2024-04-16T16:38:55.987 INFO:root:teuthology version: 0.0.1.dev267+g6c63784 2024-04-16T16:38:55.987 DEBUG:teuthology.run:Teuthology command: teuthology --verbose --description fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-workunit/suites/dbench}} --owner scheduled_mchangir@teuthology --name mchangir-2024-04-16_16:32:10-fs-wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-distro-default-smithi --archive /home/teuthworker/archive/mchangir-2024-04-16_16:32:10-fs-wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-distro-default-smithi/7658439 -- /home/teuthworker/archive/mchangir-2024-04-16_16:32:10-fs-wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-distro-default-smithi/7658439/orig.config.yaml 2024-04-16T16:38:56.043 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-16T16:38:56.136 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/mchangir-2024-04-16_16:32:10-fs-wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-distro-default-smithi/7658439 branch: wip-mchangir-qa-debug-resource-temporarily-unavailable-issue description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-workunit/suites/dbench}} email: mchangir@redhat.com first_in_suite: false job_id: '7658439' kernel: client: branch: testing kdb: 1 sha1: distro ktype: upstream last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules - exec: mon.a: - ceph mgr module enable snap_schedule - ceph config set mgr mgr/snap_schedule/allow_m_granularity true - ceph config set mgr mgr/snap_schedule/dump_on_update true name: mchangir-2024-04-16_16:32:10-fs-wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-mchangir-qa-debug-resource-temporarily-unavailable-issue ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd max_mds: 3 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug cephsqlite: 20 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10000 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - slow metadata IO - SLOW_OPS - slow request sha1: 9b42bfbeda298874861ece590aa565cf64ac8c45 subvols: create: 2 subvol_options: --namespace-isolated --size 25000000000 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - mds.root_rsnaps - mds_server.req_mksnap_latency.avgcount - mds_server.req_rmsnap_latency.avgcount install: ceph: flavor: default sha1: 9b42bfbeda298874861ece590aa565cf64ac8c45 kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=crc - wsync syntax: v2 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: qa-debug-resource-temporarily-unavailable-issue sha1: b137e4a675398e295567f9652210865a4d573fac owner: scheduled_mchangir@teuthology priority: 25 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 2381 sha1: 9b42bfbeda298874861ece590aa565cf64ac8c45 sleep_before_teardown: 0 suite: fs suite_branch: qa-debug-resource-temporarily-unavailable-issue suite_path: /home/teuthworker/src/github.com_mchangir_ceph_b137e4a675398e295567f9652210865a4d573fac/qa suite_relpath: qa suite_repo: https://github.com/mchangir/ceph.git suite_sha1: b137e4a675398e295567f9652210865a4d573fac syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi045.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNRd4Z5lIkG+aHVcNKB5bIR33xLSkjJ2+IrwRHGZWgiBdmi5xc7WlihH4ZBjgTq0kQvwjbwStiHVrSWiMstz70I= smithi104.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBE2MaJ9xDICTcctgG59YkLRA+XHL7Tqw5uUYQV9k69LVhiELc9OQJywfIZiI9Tko916Qu0n1c8yH0ufeobgw8NQ= smithi117.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGRRWVkC0WMOkt1Y4/S+07VTH5BhW2jpEz0phigKIpIr5kqVDK2y1ROw0hAw3CSDTz1vShxtrByguWwQ+BTKSQ0= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - sequential: - mgrmodules - kclient: null - exec: mon.a: - ceph fs set cephfs balance_automate true - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - exec: mon.a: - ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m - ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - date +%s > START_TIME - full_sequential_finally: - exec: mon.a: - date +%s > END_TIME - START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - background_exec: mon.a: - while sleep 13; do ceph tell mds.cephfs:0 flush journal; done - workunit: clients: all: - suites/dbench.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 6c637841c215537a4502385240412f1966e0faab timestamp: 2024-04-16_16:32:10 tube: smithi user: mchangir verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-04-16T16:38:56.137 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_mchangir_ceph_b137e4a675398e295567f9652210865a4d573fac/qa; will attempt to use it 2024-04-16T16:38:56.137 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_mchangir_ceph_b137e4a675398e295567f9652210865a4d573fac/qa/tasks 2024-04-16T16:38:56.137 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-04-16T16:38:56.139 INFO:teuthology.task.internal:Checking packages... 2024-04-16T16:38:56.162 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '9b42bfbeda298874861ece590aa565cf64ac8c45' 2024-04-16T16:38:56.162 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-16T16:38:56.162 INFO:teuthology.packaging:ref: None 2024-04-16T16:38:56.162 INFO:teuthology.packaging:tag: None 2024-04-16T16:38:56.162 INFO:teuthology.packaging:branch: wip-mchangir-qa-debug-resource-temporarily-unavailable-issue 2024-04-16T16:38:56.162 INFO:teuthology.packaging:sha1: 9b42bfbeda298874861ece590aa565cf64ac8c45 2024-04-16T16:38:56.162 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue 2024-04-16T16:38:56.379 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-3101.g9b42bfbe 2024-04-16T16:38:56.380 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-04-16T16:38:56.387 INFO:teuthology.task.internal:no buildpackages task found 2024-04-16T16:38:56.387 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-04-16T16:38:56.394 INFO:teuthology.task.internal:Saving configuration 2024-04-16T16:38:56.413 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-04-16T16:38:56.419 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-16T16:38:56.443 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi045.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/mchangir-2024-04-16_16:32:10-fs-wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-distro-default-smithi/7658439', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-16 16:34:01.459561', 'locked_by': 'scheduled_mchangir@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNRd4Z5lIkG+aHVcNKB5bIR33xLSkjJ2+IrwRHGZWgiBdmi5xc7WlihH4ZBjgTq0kQvwjbwStiHVrSWiMstz70I='} 2024-04-16T16:38:56.462 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi104.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/mchangir-2024-04-16_16:32:10-fs-wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-distro-default-smithi/7658439', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-16 16:34:01.457884', 'locked_by': 'scheduled_mchangir@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBE2MaJ9xDICTcctgG59YkLRA+XHL7Tqw5uUYQV9k69LVhiELc9OQJywfIZiI9Tko916Qu0n1c8yH0ufeobgw8NQ='} 2024-04-16T16:38:56.479 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi117.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/mchangir-2024-04-16_16:32:10-fs-wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-distro-default-smithi/7658439', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-16 16:34:01.455850', 'locked_by': 'scheduled_mchangir@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGRRWVkC0WMOkt1Y4/S+07VTH5BhW2jpEz0phigKIpIr5kqVDK2y1ROw0hAw3CSDTz1vShxtrByguWwQ+BTKSQ0='} 2024-04-16T16:38:56.480 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-04-16T16:38:56.486 INFO:teuthology.task.internal:roles: ubuntu@smithi045.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-04-16T16:38:56.486 INFO:teuthology.task.internal:roles: ubuntu@smithi104.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-04-16T16:38:56.487 INFO:teuthology.task.internal:roles: ubuntu@smithi117.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-04-16T16:38:56.487 INFO:teuthology.run_tasks:Running task console_log... 2024-04-16T16:38:56.568 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fe82dac90d0>, signals=[15]) 2024-04-16T16:38:56.568 INFO:teuthology.run_tasks:Running task internal.connect... 2024-04-16T16:38:56.575 INFO:teuthology.task.internal:Opening connections... 2024-04-16T16:38:56.575 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi045.front.sepia.ceph.com 2024-04-16T16:38:56.577 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi045.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-16T16:38:56.680 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi104.front.sepia.ceph.com 2024-04-16T16:38:56.681 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi104.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-16T16:38:56.765 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi117.front.sepia.ceph.com 2024-04-16T16:38:56.766 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi117.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-16T16:38:56.853 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-04-16T16:38:56.860 DEBUG:teuthology.orchestra.run.smithi045:> uname -m 2024-04-16T16:38:56.878 INFO:teuthology.orchestra.run.smithi045.stdout:x86_64 2024-04-16T16:38:56.879 DEBUG:teuthology.orchestra.run.smithi045:> cat /etc/os-release 2024-04-16T16:38:56.936 INFO:teuthology.orchestra.run.smithi045.stdout:NAME="CentOS Stream" 2024-04-16T16:38:56.936 INFO:teuthology.orchestra.run.smithi045.stdout:VERSION="9" 2024-04-16T16:38:56.936 INFO:teuthology.orchestra.run.smithi045.stdout:ID="centos" 2024-04-16T16:38:56.936 INFO:teuthology.orchestra.run.smithi045.stdout:ID_LIKE="rhel fedora" 2024-04-16T16:38:56.936 INFO:teuthology.orchestra.run.smithi045.stdout:VERSION_ID="9" 2024-04-16T16:38:56.936 INFO:teuthology.orchestra.run.smithi045.stdout:PLATFORM_ID="platform:el9" 2024-04-16T16:38:56.936 INFO:teuthology.orchestra.run.smithi045.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-16T16:38:56.936 INFO:teuthology.orchestra.run.smithi045.stdout:ANSI_COLOR="0;31" 2024-04-16T16:38:56.936 INFO:teuthology.orchestra.run.smithi045.stdout:LOGO="fedora-logo-icon" 2024-04-16T16:38:56.936 INFO:teuthology.orchestra.run.smithi045.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-16T16:38:56.937 INFO:teuthology.orchestra.run.smithi045.stdout:HOME_URL="https://centos.org/" 2024-04-16T16:38:56.937 INFO:teuthology.orchestra.run.smithi045.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-16T16:38:56.937 INFO:teuthology.orchestra.run.smithi045.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-16T16:38:56.937 INFO:teuthology.orchestra.run.smithi045.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-16T16:38:56.937 INFO:teuthology.lock.ops:Updating smithi045.front.sepia.ceph.com on lock server 2024-04-16T16:38:56.954 DEBUG:teuthology.orchestra.run.smithi104:> uname -m 2024-04-16T16:38:56.975 INFO:teuthology.orchestra.run.smithi104.stdout:x86_64 2024-04-16T16:38:56.975 DEBUG:teuthology.orchestra.run.smithi104:> cat /etc/os-release 2024-04-16T16:38:57.031 INFO:teuthology.orchestra.run.smithi104.stdout:NAME="CentOS Stream" 2024-04-16T16:38:57.031 INFO:teuthology.orchestra.run.smithi104.stdout:VERSION="9" 2024-04-16T16:38:57.031 INFO:teuthology.orchestra.run.smithi104.stdout:ID="centos" 2024-04-16T16:38:57.031 INFO:teuthology.orchestra.run.smithi104.stdout:ID_LIKE="rhel fedora" 2024-04-16T16:38:57.031 INFO:teuthology.orchestra.run.smithi104.stdout:VERSION_ID="9" 2024-04-16T16:38:57.031 INFO:teuthology.orchestra.run.smithi104.stdout:PLATFORM_ID="platform:el9" 2024-04-16T16:38:57.031 INFO:teuthology.orchestra.run.smithi104.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-16T16:38:57.031 INFO:teuthology.orchestra.run.smithi104.stdout:ANSI_COLOR="0;31" 2024-04-16T16:38:57.032 INFO:teuthology.orchestra.run.smithi104.stdout:LOGO="fedora-logo-icon" 2024-04-16T16:38:57.032 INFO:teuthology.orchestra.run.smithi104.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-16T16:38:57.032 INFO:teuthology.orchestra.run.smithi104.stdout:HOME_URL="https://centos.org/" 2024-04-16T16:38:57.032 INFO:teuthology.orchestra.run.smithi104.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-16T16:38:57.032 INFO:teuthology.orchestra.run.smithi104.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-16T16:38:57.032 INFO:teuthology.orchestra.run.smithi104.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-16T16:38:57.032 INFO:teuthology.lock.ops:Updating smithi104.front.sepia.ceph.com on lock server 2024-04-16T16:38:57.053 DEBUG:teuthology.orchestra.run.smithi117:> uname -m 2024-04-16T16:38:57.071 INFO:teuthology.orchestra.run.smithi117.stdout:x86_64 2024-04-16T16:38:57.071 DEBUG:teuthology.orchestra.run.smithi117:> cat /etc/os-release 2024-04-16T16:38:57.126 INFO:teuthology.orchestra.run.smithi117.stdout:NAME="CentOS Stream" 2024-04-16T16:38:57.127 INFO:teuthology.orchestra.run.smithi117.stdout:VERSION="9" 2024-04-16T16:38:57.127 INFO:teuthology.orchestra.run.smithi117.stdout:ID="centos" 2024-04-16T16:38:57.127 INFO:teuthology.orchestra.run.smithi117.stdout:ID_LIKE="rhel fedora" 2024-04-16T16:38:57.127 INFO:teuthology.orchestra.run.smithi117.stdout:VERSION_ID="9" 2024-04-16T16:38:57.127 INFO:teuthology.orchestra.run.smithi117.stdout:PLATFORM_ID="platform:el9" 2024-04-16T16:38:57.127 INFO:teuthology.orchestra.run.smithi117.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-16T16:38:57.127 INFO:teuthology.orchestra.run.smithi117.stdout:ANSI_COLOR="0;31" 2024-04-16T16:38:57.127 INFO:teuthology.orchestra.run.smithi117.stdout:LOGO="fedora-logo-icon" 2024-04-16T16:38:57.127 INFO:teuthology.orchestra.run.smithi117.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-16T16:38:57.127 INFO:teuthology.orchestra.run.smithi117.stdout:HOME_URL="https://centos.org/" 2024-04-16T16:38:57.127 INFO:teuthology.orchestra.run.smithi117.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-16T16:38:57.127 INFO:teuthology.orchestra.run.smithi117.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-16T16:38:57.127 INFO:teuthology.orchestra.run.smithi117.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-16T16:38:57.128 INFO:teuthology.lock.ops:Updating smithi117.front.sepia.ceph.com on lock server 2024-04-16T16:38:57.145 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-04-16T16:38:57.156 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-04-16T16:38:57.163 INFO:teuthology.task.internal:Checking for old test directory... 2024-04-16T16:38:57.163 DEBUG:teuthology.orchestra.run.smithi045:> test '!' -e /home/ubuntu/cephtest 2024-04-16T16:38:57.166 DEBUG:teuthology.orchestra.run.smithi104:> test '!' -e /home/ubuntu/cephtest 2024-04-16T16:38:57.168 DEBUG:teuthology.orchestra.run.smithi117:> test '!' -e /home/ubuntu/cephtest 2024-04-16T16:38:57.184 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-04-16T16:38:57.190 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-04-16T16:38:57.190 DEBUG:teuthology.orchestra.run.smithi045:> test -z $(ls -A /var/lib/ceph) 2024-04-16T16:38:57.223 DEBUG:teuthology.orchestra.run.smithi104:> test -z $(ls -A /var/lib/ceph) 2024-04-16T16:38:57.225 DEBUG:teuthology.orchestra.run.smithi117:> test -z $(ls -A /var/lib/ceph) 2024-04-16T16:38:57.329 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-04-16T16:38:57.394 INFO:teuthology.run_tasks:Running task kernel... 2024-04-16T16:38:57.409 INFO:teuthology.task.kernel:normalize config orig: {'client': {'branch': 'testing'}, 'kdb': 1, 'sha1': 'distro'} 2024-04-16T16:38:57.409 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'branch': 'testing'}} 2024-04-16T16:38:57.409 INFO:teuthology.task.kernel:config {'client.0': {'branch': 'testing'}}, timeout 300 2024-04-16T16:38:57.409 DEBUG:teuthology.orchestra.run.smithi045:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-16T16:38:57.426 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-16T16:38:57.427 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&ref=testing 2024-04-16T16:38:57.602 DEBUG:teuthology.task.kernel:sha1 for client.0 is 3f60ed14f7c110c7ce3c8d83dd2ed0d08e80d0c8 2024-04-16T16:38:57.602 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "3f60ed14f7c110c7ce3c8d83dd2ed0d08e80d0c8"... 2024-04-16T16:38:57.603 DEBUG:teuthology.orchestra.run.smithi045:> uname -r 2024-04-16T16:38:57.620 INFO:teuthology.orchestra.run.smithi045.stdout:5.14.0-361.el9.x86_64 2024-04-16T16:38:57.620 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-361.el9.x86_64 vs 3f60ed14f7c110c7ce3c8d83dd2ed0d08e80d0c8 2024-04-16T16:38:57.621 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-04-16T16:38:57.621 DEBUG:teuthology.orchestra.run.smithi045:> sudo yum upgrade -y linux-firmware 2024-04-16T16:38:59.771 INFO:teuthology.orchestra.run.smithi045.stdout:CentOS-9 - AppStream 13 MB/s | 6.7 MB 00:00 2024-04-16T16:39:01.221 INFO:teuthology.orchestra.run.smithi045.stdout:CentOS-9 - BaseOS 6.7 MB/s | 2.0 MB 00:00 2024-04-16T16:39:02.287 INFO:teuthology.orchestra.run.smithi045.stdout:CentOS Stream 9 - BaseOS 11 MB/s | 8.0 MB 00:00 2024-04-16T16:39:05.221 INFO:teuthology.orchestra.run.smithi045.stdout:CentOS Stream 9 - AppStream 11 MB/s | 19 MB 00:01 2024-04-16T16:39:09.455 INFO:teuthology.orchestra.run.smithi045.stdout:CentOS Stream 9 - CRB 11 MB/s | 6.2 MB 00:00 2024-04-16T16:39:11.106 INFO:teuthology.orchestra.run.smithi045.stdout:CentOS Stream 9 - Extras packages 40 kB/s | 16 kB 00:00 2024-04-16T16:39:14.382 INFO:teuthology.orchestra.run.smithi045.stdout:Extra Packages for Enterprise Linux 6.7 MB/s | 21 MB 00:03 2024-04-16T16:39:19.540 INFO:teuthology.orchestra.run.smithi045.stdout:lab-extras 42 kB/s | 1.7 kB 00:00 2024-04-16T16:39:22.152 INFO:teuthology.orchestra.run.smithi045.stdout:Dependencies resolved. 2024-04-16T16:39:22.163 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-04-16T16:39:22.163 INFO:teuthology.orchestra.run.smithi045.stdout: Package Arch Version Repository Size 2024-04-16T16:39:22.163 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-04-16T16:39:22.163 INFO:teuthology.orchestra.run.smithi045.stdout:Upgrading: 2024-04-16T16:39:22.163 INFO:teuthology.orchestra.run.smithi045.stdout: linux-firmware noarch 20240219-143.el9 CentOS-BaseOS 375 M 2024-04-16T16:39:22.164 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-16T16:39:22.164 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction Summary 2024-04-16T16:39:22.164 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-04-16T16:39:22.164 INFO:teuthology.orchestra.run.smithi045.stdout:Upgrade 1 Package 2024-04-16T16:39:22.164 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-16T16:39:22.164 INFO:teuthology.orchestra.run.smithi045.stdout:Total download size: 375 M 2024-04-16T16:39:22.164 INFO:teuthology.orchestra.run.smithi045.stdout:Downloading Packages: 2024-04-16T16:39:30.940 INFO:teuthology.orchestra.run.smithi045.stdout:linux-firmware-20240219-143.el9.noarch.rpm 43 MB/s | 375 MB 00:08 2024-04-16T16:39:30.942 INFO:teuthology.orchestra.run.smithi045.stdout:-------------------------------------------------------------------------------- 2024-04-16T16:39:30.942 INFO:teuthology.orchestra.run.smithi045.stdout:Total 43 MB/s | 375 MB 00:08 2024-04-16T16:39:30.942 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction check 2024-04-16T16:39:31.036 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction check succeeded. 2024-04-16T16:39:31.036 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction test 2024-04-16T16:39:33.672 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction test succeeded. 2024-04-16T16:39:33.672 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction 2024-04-16T16:39:40.408 INFO:teuthology.orchestra.run.smithi045.stdout: Preparing : 1/1 2024-04-16T16:39:40.639 INFO:teuthology.orchestra.run.smithi045.stdout: Upgrading : linux-firmware-20240219-143.el9.noarch 1/2 2024-04-16T16:39:40.727 INFO:teuthology.orchestra.run.smithi045.stdout: Cleanup : linux-firmware-20230814-139.el9.noarch 2/2 2024-04-16T16:39:42.057 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: linux-firmware-20230814-139.el9.noarch 2/2 2024-04-16T16:39:42.057 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : linux-firmware-20240219-143.el9.noarch 1/2 2024-04-16T16:39:42.438 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : linux-firmware-20230814-139.el9.noarch 2/2 2024-04-16T16:39:42.439 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-16T16:39:42.439 INFO:teuthology.orchestra.run.smithi045.stdout:Upgraded: 2024-04-16T16:39:42.439 INFO:teuthology.orchestra.run.smithi045.stdout: linux-firmware-20240219-143.el9.noarch 2024-04-16T16:39:42.439 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-16T16:39:42.439 INFO:teuthology.orchestra.run.smithi045.stdout:Complete! 2024-04-16T16:39:42.716 INFO:teuthology.task.kernel:Downloading kernel 3f60ed14f7c110c7ce3c8d83dd2ed0d08e80d0c8 on client.0... 2024-04-16T16:39:42.717 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3f60ed14f7c110c7ce3c8d83dd2ed0d08e80d0c8 2024-04-16T16:39:42.949 INFO:teuthology.task.kernel:fetching, builder baseurl is https://3.chacra.ceph.com/r/kernel/testing/3f60ed14f7c110c7ce3c8d83dd2ed0d08e80d0c8/centos/9/flavors/default/x86_64/ 2024-04-16T16:39:42.950 DEBUG:teuthology.orchestra.run.smithi045:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.9.0_rc3_g3f60ed14f7c1-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://3.chacra.ceph.com/r/kernel/testing/3f60ed14f7c110c7ce3c8d83dd2ed0d08e80d0c8/centos/9/flavors/default/x86_64/ --input-file=- 2024-04-16T16:39:42.952 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi045.front.sepia.ceph.com to complete... 2024-04-16T16:40:25.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16 16:40:25 URL:https://3.chacra.ceph.com/r/kernel/testing/3f60ed14f7c110c7ce3c8d83dd2ed0d08e80d0c8/centos/9/flavors/default/x86_64/kernel-6.9.0_rc3_g3f60ed14f7c1-1.x86_64.rpm [357344123/357344123] -> "/tmp/kernel.x86_64.rpm" [1] 2024-04-16T16:40:25.837 INFO:teuthology.orchestra.run.smithi045.stderr:FINISHED --2024-04-16 16:40:25-- 2024-04-16T16:40:25.837 INFO:teuthology.orchestra.run.smithi045.stderr:Total wall clock time: 43s 2024-04-16T16:40:25.837 INFO:teuthology.orchestra.run.smithi045.stderr:Downloaded: 1 files, 341M in 43s (8.00 MB/s) 2024-04-16T16:40:25.840 INFO:teuthology.task.kernel:Installing kernel 3f60ed14f7c110c7ce3c8d83dd2ed0d08e80d0c8 on client.0... 2024-04-16T16:40:25.840 DEBUG:teuthology.orchestra.run.smithi045:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-04-16T16:40:27.996 INFO:teuthology.orchestra.run.smithi045.stdout:Verifying... ######################################## 2024-04-16T16:40:28.050 INFO:teuthology.orchestra.run.smithi045.stdout:Preparing... ######################################## 2024-04-16T16:40:28.134 INFO:teuthology.orchestra.run.smithi045.stdout:Updating / installing... 2024-04-16T16:40:36.426 INFO:teuthology.orchestra.run.smithi045.stdout:kernel-6.9.0_rc3_g3f60ed14f7c1-1 ######################################## 2024-04-16T16:40:51.211 INFO:teuthology.orchestra.run.smithi045.stderr:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-04-16T16:41:14.952 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi045.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-04-16T16:41:14.953 DEBUG:teuthology.orchestra.run.smithi045:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-04-16T16:41:15.015 INFO:teuthology.orchestra.run.smithi045.stdout:/boot/System.map-6.9.0-rc3-g3f60ed14f7c1 2024-04-16T16:41:15.015 INFO:teuthology.orchestra.run.smithi045.stdout:/boot/config-6.9.0-rc3-g3f60ed14f7c1 2024-04-16T16:41:15.015 INFO:teuthology.orchestra.run.smithi045.stdout:/boot/vmlinuz-6.9.0-rc3-g3f60ed14f7c1 2024-04-16T16:41:15.015 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/System.map 2024-04-16T16:41:15.015 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/config 2024-04-16T16:41:15.015 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel 2024-04-16T16:41:15.015 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch 2024-04-16T16:41:15.015 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86 2024-04-16T16:41:15.015 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto 2024-04-16T16:41:15.015 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-04-16T16:41:15.016 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-04-16T16:41:15.016 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-04-16T16:41:15.016 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-04-16T16:41:15.016 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-04-16T16:41:15.016 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-04-16T16:41:15.016 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-04-16T16:41:15.016 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/crc32c-intel.ko 2024-04-16T16:41:15.016 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-04-16T16:41:15.016 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-04-16T16:41:15.016 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-04-16T16:41:15.016 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-04-16T16:41:15.016 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/serpent-avx2.ko 2024-04-16T16:41:15.016 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-04-16T16:41:15.016 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-04-16T16:41:15.016 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-04-16T16:41:15.017 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-04-16T16:41:15.017 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-04-16T16:41:15.017 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-04-16T16:41:15.017 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-04-16T16:41:15.017 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/kernel 2024-04-16T16:41:15.017 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/kernel/cpu 2024-04-16T16:41:15.017 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/kernel/cpu/mce 2024-04-16T16:41:15.017 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-04-16T16:41:15.017 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/kvm 2024-04-16T16:41:15.017 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/kvm/kvm-amd.ko 2024-04-16T16:41:15.017 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/kvm/kvm-intel.ko 2024-04-16T16:41:15.017 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/arch/x86/kvm/kvm.ko 2024-04-16T16:41:15.017 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto 2024-04-16T16:41:15.017 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/ansi_cprng.ko 2024-04-16T16:41:15.018 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/anubis.ko 2024-04-16T16:41:15.018 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/arc4.ko 2024-04-16T16:41:15.018 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/async_tx 2024-04-16T16:41:15.018 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/async_tx/async_memcpy.ko 2024-04-16T16:41:15.018 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/async_tx/async_pq.ko 2024-04-16T16:41:15.018 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/async_tx/async_raid6_recov.ko 2024-04-16T16:41:15.018 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/async_tx/async_tx.ko 2024-04-16T16:41:15.018 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/async_tx/async_xor.ko 2024-04-16T16:41:15.018 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/async_tx/raid6test.ko 2024-04-16T16:41:15.018 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/authenc.ko 2024-04-16T16:41:15.018 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/authencesn.ko 2024-04-16T16:41:15.018 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/blake2b_generic.ko 2024-04-16T16:41:15.018 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/blowfish_common.ko 2024-04-16T16:41:15.018 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/blowfish_generic.ko 2024-04-16T16:41:15.019 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/camellia_generic.ko 2024-04-16T16:41:15.019 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/cast5_generic.ko 2024-04-16T16:41:15.019 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/cast6_generic.ko 2024-04-16T16:41:15.019 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/cast_common.ko 2024-04-16T16:41:15.019 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/ccm.ko 2024-04-16T16:41:15.019 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/cmac.ko 2024-04-16T16:41:15.019 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/crc32_generic.ko 2024-04-16T16:41:15.019 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/crypto_engine.ko 2024-04-16T16:41:15.019 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/crypto_user.ko 2024-04-16T16:41:15.019 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/cts.ko 2024-04-16T16:41:15.019 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/des_generic.ko 2024-04-16T16:41:15.019 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/dh_generic.ko 2024-04-16T16:41:15.019 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/echainiv.ko 2024-04-16T16:41:15.020 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/essiv.ko 2024-04-16T16:41:15.020 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/fcrypt.ko 2024-04-16T16:41:15.020 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/khazad.ko 2024-04-16T16:41:15.020 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/lz4.ko 2024-04-16T16:41:15.020 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/lz4hc.ko 2024-04-16T16:41:15.020 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/md4.ko 2024-04-16T16:41:15.020 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/michael_mic.ko 2024-04-16T16:41:15.020 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/pcbc.ko 2024-04-16T16:41:15.020 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/pcrypt.ko 2024-04-16T16:41:15.020 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/rmd160.ko 2024-04-16T16:41:15.020 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/seed.ko 2024-04-16T16:41:15.020 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/serpent_generic.ko 2024-04-16T16:41:15.020 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/tcrypt.ko 2024-04-16T16:41:15.021 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/tea.ko 2024-04-16T16:41:15.021 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/twofish_common.ko 2024-04-16T16:41:15.021 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/twofish_generic.ko 2024-04-16T16:41:15.021 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/vmac.ko 2024-04-16T16:41:15.021 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/wp512.ko 2024-04-16T16:41:15.021 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/xcbc.ko 2024-04-16T16:41:15.021 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/xor.ko 2024-04-16T16:41:15.021 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/crypto/xxhash_generic.ko 2024-04-16T16:41:15.021 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers 2024-04-16T16:41:15.021 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/acpi 2024-04-16T16:41:15.021 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/acpi/acpi_ipmi.ko 2024-04-16T16:41:15.021 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/acpi/acpi_pad.ko 2024-04-16T16:41:15.021 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/acpi/ec_sys.ko 2024-04-16T16:41:15.022 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/acpi/platform_profile.ko 2024-04-16T16:41:15.022 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/acpi/sbs.ko 2024-04-16T16:41:15.022 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/acpi/sbshc.ko 2024-04-16T16:41:15.022 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/acpi/video.ko 2024-04-16T16:41:15.022 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata 2024-04-16T16:41:15.022 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/acard-ahci.ko 2024-04-16T16:41:15.022 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/ahci_platform.ko 2024-04-16T16:41:15.022 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/ata_generic.ko 2024-04-16T16:41:15.022 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/libahci_platform.ko 2024-04-16T16:41:15.022 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_acpi.ko 2024-04-16T16:41:15.022 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_ali.ko 2024-04-16T16:41:15.022 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_amd.ko 2024-04-16T16:41:15.022 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_artop.ko 2024-04-16T16:41:15.022 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_atiixp.ko 2024-04-16T16:41:15.022 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_atp867x.ko 2024-04-16T16:41:15.023 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_cmd640.ko 2024-04-16T16:41:15.023 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_cmd64x.ko 2024-04-16T16:41:15.023 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_cypress.ko 2024-04-16T16:41:15.023 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_efar.ko 2024-04-16T16:41:15.023 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_hpt366.ko 2024-04-16T16:41:15.023 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_hpt37x.ko 2024-04-16T16:41:15.023 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_hpt3x2n.ko 2024-04-16T16:41:15.023 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_hpt3x3.ko 2024-04-16T16:41:15.023 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_it8213.ko 2024-04-16T16:41:15.023 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_it821x.ko 2024-04-16T16:41:15.023 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_jmicron.ko 2024-04-16T16:41:15.023 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_marvell.ko 2024-04-16T16:41:15.023 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_mpiix.ko 2024-04-16T16:41:15.023 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_netcell.ko 2024-04-16T16:41:15.023 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_ninja32.ko 2024-04-16T16:41:15.024 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_ns87410.ko 2024-04-16T16:41:15.024 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_ns87415.ko 2024-04-16T16:41:15.024 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_oldpiix.ko 2024-04-16T16:41:15.024 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_opti.ko 2024-04-16T16:41:15.024 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_optidma.ko 2024-04-16T16:41:15.024 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_pcmcia.ko 2024-04-16T16:41:15.024 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_pdc2027x.ko 2024-04-16T16:41:15.024 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-04-16T16:41:15.024 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_piccolo.ko 2024-04-16T16:41:15.024 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_rdc.ko 2024-04-16T16:41:15.024 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_sch.ko 2024-04-16T16:41:15.024 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_serverworks.ko 2024-04-16T16:41:15.024 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_sil680.ko 2024-04-16T16:41:15.024 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_sis.ko 2024-04-16T16:41:15.025 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_sl82c105.ko 2024-04-16T16:41:15.025 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_triflex.ko 2024-04-16T16:41:15.025 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pata_via.ko 2024-04-16T16:41:15.025 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/pdc_adma.ko 2024-04-16T16:41:15.025 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/sata_inic162x.ko 2024-04-16T16:41:15.025 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/sata_mv.ko 2024-04-16T16:41:15.025 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/sata_nv.ko 2024-04-16T16:41:15.025 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/sata_promise.ko 2024-04-16T16:41:15.025 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/sata_qstor.ko 2024-04-16T16:41:15.025 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/sata_sil.ko 2024-04-16T16:41:15.025 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/sata_sil24.ko 2024-04-16T16:41:15.025 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/sata_sis.ko 2024-04-16T16:41:15.025 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/sata_svw.ko 2024-04-16T16:41:15.025 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/sata_sx4.ko 2024-04-16T16:41:15.025 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/sata_uli.ko 2024-04-16T16:41:15.026 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/sata_via.ko 2024-04-16T16:41:15.026 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ata/sata_vsc.ko 2024-04-16T16:41:15.026 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/auxdisplay 2024-04-16T16:41:15.026 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/auxdisplay/ks0108.ko 2024-04-16T16:41:15.026 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/base 2024-04-16T16:41:15.026 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/base/regmap 2024-04-16T16:41:15.026 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/base/regmap/regmap-i2c.ko 2024-04-16T16:41:15.026 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/base/regmap/regmap-mmio.ko 2024-04-16T16:41:15.026 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/bcma 2024-04-16T16:41:15.026 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/bcma/bcma.ko 2024-04-16T16:41:15.026 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block 2024-04-16T16:41:15.026 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/aoe 2024-04-16T16:41:15.026 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/aoe/aoe.ko 2024-04-16T16:41:15.026 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/brd.ko 2024-04-16T16:41:15.026 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/drbd 2024-04-16T16:41:15.027 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/drbd/drbd.ko 2024-04-16T16:41:15.027 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/floppy.ko 2024-04-16T16:41:15.027 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/loop.ko 2024-04-16T16:41:15.027 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/mtip32xx 2024-04-16T16:41:15.027 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-04-16T16:41:15.027 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/nbd.ko 2024-04-16T16:41:15.027 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/null_blk 2024-04-16T16:41:15.027 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/null_blk/null_blk.ko 2024-04-16T16:41:15.027 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/pktcdvd.ko 2024-04-16T16:41:15.027 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/rbd.ko 2024-04-16T16:41:15.027 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/virtio_blk.ko 2024-04-16T16:41:15.027 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/xen-blkback 2024-04-16T16:41:15.027 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-04-16T16:41:15.027 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/xen-blkfront.ko 2024-04-16T16:41:15.027 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/zram 2024-04-16T16:41:15.028 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/block/zram/zram.ko 2024-04-16T16:41:15.028 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char 2024-04-16T16:41:15.028 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/hangcheck-timer.ko 2024-04-16T16:41:15.028 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/hw_random 2024-04-16T16:41:15.028 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/hw_random/amd-rng.ko 2024-04-16T16:41:15.028 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/hw_random/intel-rng.ko 2024-04-16T16:41:15.028 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-04-16T16:41:15.028 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/hw_random/via-rng.ko 2024-04-16T16:41:15.028 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/hw_random/virtio-rng.ko 2024-04-16T16:41:15.028 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/ipmi 2024-04-16T16:41:15.028 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-04-16T16:41:15.028 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-04-16T16:41:15.028 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-04-16T16:41:15.028 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/ipmi/ipmi_si.ko 2024-04-16T16:41:15.028 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-04-16T16:41:15.028 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-04-16T16:41:15.028 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/lp.ko 2024-04-16T16:41:15.029 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/mwave 2024-04-16T16:41:15.029 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/mwave/mwave.ko 2024-04-16T16:41:15.029 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/ppdev.ko 2024-04-16T16:41:15.029 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/tlclk.ko 2024-04-16T16:41:15.029 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/tpm 2024-04-16T16:41:15.029 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/tpm/tpm.ko 2024-04-16T16:41:15.029 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/tpm/tpm_atmel.ko 2024-04-16T16:41:15.029 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/tpm/tpm_infineon.ko 2024-04-16T16:41:15.029 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/tpm/tpm_nsc.ko 2024-04-16T16:41:15.029 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/tpm/tpm_tis.ko 2024-04-16T16:41:15.029 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-04-16T16:41:15.029 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/uv_mmtimer.ko 2024-04-16T16:41:15.029 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/char/virtio_console.ko 2024-04-16T16:41:15.029 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/cpufreq 2024-04-16T16:41:15.029 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-04-16T16:41:15.030 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-04-16T16:41:15.030 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/cpufreq/p4-clockmod.ko 2024-04-16T16:41:15.030 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-04-16T16:41:15.030 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/cpufreq/powernow-k8.ko 2024-04-16T16:41:15.030 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/cpufreq/speedstep-lib.ko 2024-04-16T16:41:15.030 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto 2024-04-16T16:41:15.030 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto/ccp 2024-04-16T16:41:15.030 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-04-16T16:41:15.030 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto/ccp/ccp.ko 2024-04-16T16:41:15.030 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto/intel 2024-04-16T16:41:15.030 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto/intel/qat 2024-04-16T16:41:15.030 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto/intel/qat/qat_common 2024-04-16T16:41:15.030 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-04-16T16:41:15.030 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-04-16T16:41:15.030 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-04-16T16:41:15.031 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto/padlock-aes.ko 2024-04-16T16:41:15.031 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto/padlock-sha.ko 2024-04-16T16:41:15.031 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto/virtio 2024-04-16T16:41:15.031 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-04-16T16:41:15.031 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/dca 2024-04-16T16:41:15.031 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/dca/dca.ko 2024-04-16T16:41:15.031 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/devfreq 2024-04-16T16:41:15.031 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-04-16T16:41:15.031 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/dma 2024-04-16T16:41:15.031 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/dma/dw 2024-04-16T16:41:15.031 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/dma/dw/dw_dmac.ko 2024-04-16T16:41:15.031 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/dma/ioat 2024-04-16T16:41:15.031 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/dma/ioat/ioatdma.ko 2024-04-16T16:41:15.031 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac 2024-04-16T16:41:15.031 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac/amd64_edac.ko 2024-04-16T16:41:15.031 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac/e752x_edac.ko 2024-04-16T16:41:15.032 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac/edac_mce_amd.ko 2024-04-16T16:41:15.032 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac/i3000_edac.ko 2024-04-16T16:41:15.032 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac/i3200_edac.ko 2024-04-16T16:41:15.032 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac/i5100_edac.ko 2024-04-16T16:41:15.032 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac/i5400_edac.ko 2024-04-16T16:41:15.032 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac/i7300_edac.ko 2024-04-16T16:41:15.032 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac/i7core_edac.ko 2024-04-16T16:41:15.032 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac/i82975x_edac.ko 2024-04-16T16:41:15.032 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac/ie31200_edac.ko 2024-04-16T16:41:15.032 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac/sb_edac.ko 2024-04-16T16:41:15.032 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/edac/x38_edac.ko 2024-04-16T16:41:15.032 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/firmware 2024-04-16T16:41:15.032 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/firmware/edd.ko 2024-04-16T16:41:15.032 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/firmware/iscsi_ibft.ko 2024-04-16T16:41:15.032 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/gpio 2024-04-16T16:41:15.032 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/gpio/gpio-ich.ko 2024-04-16T16:41:15.033 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/gpio/gpio-viperboard.ko 2024-04-16T16:41:15.033 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid 2024-04-16T16:41:15.033 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-appleir.ko 2024-04-16T16:41:15.033 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-aureal.ko 2024-04-16T16:41:15.033 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-axff.ko 2024-04-16T16:41:15.033 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-dr.ko 2024-04-16T16:41:15.033 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-elecom.ko 2024-04-16T16:41:15.033 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-elo.ko 2024-04-16T16:41:15.033 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-emsff.ko 2024-04-16T16:41:15.033 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-gaff.ko 2024-04-16T16:41:15.033 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-gt683r.ko 2024-04-16T16:41:15.033 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-gyration.ko 2024-04-16T16:41:15.033 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-holtek-kbd.ko 2024-04-16T16:41:15.033 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-holtek-mouse.ko 2024-04-16T16:41:15.033 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-holtekff.ko 2024-04-16T16:41:15.033 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-hyperv.ko 2024-04-16T16:41:15.033 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-icade.ko 2024-04-16T16:41:15.034 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-keytouch.ko 2024-04-16T16:41:15.034 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-kye.ko 2024-04-16T16:41:15.034 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-lcpower.ko 2024-04-16T16:41:15.034 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-led.ko 2024-04-16T16:41:15.034 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-lenovo.ko 2024-04-16T16:41:15.034 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-logitech-dj.ko 2024-04-16T16:41:15.034 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-04-16T16:41:15.034 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-multitouch.ko 2024-04-16T16:41:15.034 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-ortek.ko 2024-04-16T16:41:15.034 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-penmount.ko 2024-04-16T16:41:15.034 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-petalynx.ko 2024-04-16T16:41:15.034 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-picolcd.ko 2024-04-16T16:41:15.034 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-pl.ko 2024-04-16T16:41:15.034 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-plantronics.ko 2024-04-16T16:41:15.034 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-primax.ko 2024-04-16T16:41:15.034 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-rmi.ko 2024-04-16T16:41:15.035 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-roccat-arvo.ko 2024-04-16T16:41:15.035 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-roccat-common.ko 2024-04-16T16:41:15.035 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-roccat-isku.ko 2024-04-16T16:41:15.035 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-roccat-kone.ko 2024-04-16T16:41:15.035 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-04-16T16:41:15.035 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-roccat-konepure.ko 2024-04-16T16:41:15.035 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-04-16T16:41:15.035 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-roccat-lua.ko 2024-04-16T16:41:15.035 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-roccat-pyra.ko 2024-04-16T16:41:15.035 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-roccat-ryos.ko 2024-04-16T16:41:15.035 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-roccat-savu.ko 2024-04-16T16:41:15.035 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-roccat.ko 2024-04-16T16:41:15.035 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-saitek.ko 2024-04-16T16:41:15.035 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-samsung.ko 2024-04-16T16:41:15.035 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-sensor-hub.ko 2024-04-16T16:41:15.035 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-sjoy.ko 2024-04-16T16:41:15.035 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-sony.ko 2024-04-16T16:41:15.036 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-speedlink.ko 2024-04-16T16:41:15.036 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-steelseries.ko 2024-04-16T16:41:15.036 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-sunplus.ko 2024-04-16T16:41:15.036 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-thrustmaster.ko 2024-04-16T16:41:15.036 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-tivo.ko 2024-04-16T16:41:15.036 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-tmff.ko 2024-04-16T16:41:15.036 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-topseed.ko 2024-04-16T16:41:15.036 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-twinhan.ko 2024-04-16T16:41:15.036 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-uclogic.ko 2024-04-16T16:41:15.036 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-waltop.ko 2024-04-16T16:41:15.036 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-wiimote.ko 2024-04-16T16:41:15.036 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-xinmo.ko 2024-04-16T16:41:15.036 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-zpff.ko 2024-04-16T16:41:15.036 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/hid-zydacron.ko 2024-04-16T16:41:15.036 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/uhid.ko 2024-04-16T16:41:15.036 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hid/wacom.ko 2024-04-16T16:41:15.036 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hv 2024-04-16T16:41:15.037 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hv/hv_balloon.ko 2024-04-16T16:41:15.037 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hv/hv_utils.ko 2024-04-16T16:41:15.037 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hv/hv_vmbus.ko 2024-04-16T16:41:15.037 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon 2024-04-16T16:41:15.037 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/abituguru.ko 2024-04-16T16:41:15.037 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/abituguru3.ko 2024-04-16T16:41:15.037 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/acpi_power_meter.ko 2024-04-16T16:41:15.037 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ad7414.ko 2024-04-16T16:41:15.037 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ad7418.ko 2024-04-16T16:41:15.037 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/adc128d818.ko 2024-04-16T16:41:15.037 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/adm1025.ko 2024-04-16T16:41:15.037 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/adm1026.ko 2024-04-16T16:41:15.037 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/adm1029.ko 2024-04-16T16:41:15.037 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/adm1031.ko 2024-04-16T16:41:15.037 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/adm9240.ko 2024-04-16T16:41:15.037 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ads7828.ko 2024-04-16T16:41:15.037 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/adt7410.ko 2024-04-16T16:41:15.038 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/adt7411.ko 2024-04-16T16:41:15.038 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/adt7462.ko 2024-04-16T16:41:15.038 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/adt7470.ko 2024-04-16T16:41:15.038 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/adt7475.ko 2024-04-16T16:41:15.038 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/adt7x10.ko 2024-04-16T16:41:15.038 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/amc6821.ko 2024-04-16T16:41:15.038 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/applesmc.ko 2024-04-16T16:41:15.038 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/asb100.ko 2024-04-16T16:41:15.038 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/asc7621.ko 2024-04-16T16:41:15.038 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/asus_atk0110.ko 2024-04-16T16:41:15.038 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/atxp1.ko 2024-04-16T16:41:15.038 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/coretemp.ko 2024-04-16T16:41:15.038 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-04-16T16:41:15.038 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/dme1737.ko 2024-04-16T16:41:15.039 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ds1621.ko 2024-04-16T16:41:15.039 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ds620.ko 2024-04-16T16:41:15.039 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/emc1403.ko 2024-04-16T16:41:15.039 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/emc6w201.ko 2024-04-16T16:41:15.039 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/f71805f.ko 2024-04-16T16:41:15.039 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/f71882fg.ko 2024-04-16T16:41:15.039 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/f75375s.ko 2024-04-16T16:41:15.039 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/fam15h_power.ko 2024-04-16T16:41:15.039 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/fschmd.ko 2024-04-16T16:41:15.039 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/g760a.ko 2024-04-16T16:41:15.039 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/g762.ko 2024-04-16T16:41:15.039 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/gl518sm.ko 2024-04-16T16:41:15.039 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/gl520sm.ko 2024-04-16T16:41:15.039 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/hwmon-vid.ko 2024-04-16T16:41:15.039 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/i5500_temp.ko 2024-04-16T16:41:15.039 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/i5k_amb.ko 2024-04-16T16:41:15.040 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ibmaem.ko 2024-04-16T16:41:15.040 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ibmpex.ko 2024-04-16T16:41:15.040 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ina209.ko 2024-04-16T16:41:15.040 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ina2xx.ko 2024-04-16T16:41:15.040 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/it87.ko 2024-04-16T16:41:15.040 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/k10temp.ko 2024-04-16T16:41:15.040 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/k8temp.ko 2024-04-16T16:41:15.040 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lineage-pem.ko 2024-04-16T16:41:15.040 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm63.ko 2024-04-16T16:41:15.040 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm73.ko 2024-04-16T16:41:15.040 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm75.ko 2024-04-16T16:41:15.040 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm77.ko 2024-04-16T16:41:15.040 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm78.ko 2024-04-16T16:41:15.040 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm80.ko 2024-04-16T16:41:15.040 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm83.ko 2024-04-16T16:41:15.040 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm85.ko 2024-04-16T16:41:15.041 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm87.ko 2024-04-16T16:41:15.041 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm90.ko 2024-04-16T16:41:15.041 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm92.ko 2024-04-16T16:41:15.041 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm93.ko 2024-04-16T16:41:15.041 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm95234.ko 2024-04-16T16:41:15.041 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm95241.ko 2024-04-16T16:41:15.041 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/lm95245.ko 2024-04-16T16:41:15.041 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ltc2945.ko 2024-04-16T16:41:15.041 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ltc4151.ko 2024-04-16T16:41:15.041 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ltc4215.ko 2024-04-16T16:41:15.041 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ltc4222.ko 2024-04-16T16:41:15.041 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ltc4245.ko 2024-04-16T16:41:15.041 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ltc4260.ko 2024-04-16T16:41:15.041 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/ltc4261.ko 2024-04-16T16:41:15.041 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/max16065.ko 2024-04-16T16:41:15.042 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/max1619.ko 2024-04-16T16:41:15.042 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/max1668.ko 2024-04-16T16:41:15.042 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/max197.ko 2024-04-16T16:41:15.042 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/max6639.ko 2024-04-16T16:41:15.042 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/max6650.ko 2024-04-16T16:41:15.042 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/max6697.ko 2024-04-16T16:41:15.042 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/mcp3021.ko 2024-04-16T16:41:15.042 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/nct6683.ko 2024-04-16T16:41:15.042 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/nct6775-core.ko 2024-04-16T16:41:15.042 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/nct6775.ko 2024-04-16T16:41:15.042 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/nct7802.ko 2024-04-16T16:41:15.042 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pc87360.ko 2024-04-16T16:41:15.042 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pc87427.ko 2024-04-16T16:41:15.042 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pcf8591.ko 2024-04-16T16:41:15.042 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pmbus 2024-04-16T16:41:15.042 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-04-16T16:41:15.043 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-04-16T16:41:15.043 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-04-16T16:41:15.043 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pmbus/max16064.ko 2024-04-16T16:41:15.043 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pmbus/max34440.ko 2024-04-16T16:41:15.043 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pmbus/max8688.ko 2024-04-16T16:41:15.043 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-04-16T16:41:15.043 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-04-16T16:41:15.043 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-04-16T16:41:15.043 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-04-16T16:41:15.043 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-04-16T16:41:15.043 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-04-16T16:41:15.043 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/powr1220.ko 2024-04-16T16:41:15.043 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/sch5627.ko 2024-04-16T16:41:15.043 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/sch5636.ko 2024-04-16T16:41:15.043 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/sch56xx-common.ko 2024-04-16T16:41:15.043 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/sht15.ko 2024-04-16T16:41:15.044 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/sht21.ko 2024-04-16T16:41:15.044 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/shtc1.ko 2024-04-16T16:41:15.044 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/sis5595.ko 2024-04-16T16:41:15.044 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/smsc47b397.ko 2024-04-16T16:41:15.044 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/smsc47m1.ko 2024-04-16T16:41:15.044 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/smsc47m192.ko 2024-04-16T16:41:15.044 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/thmc50.ko 2024-04-16T16:41:15.044 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/tmp102.ko 2024-04-16T16:41:15.044 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/tmp103.ko 2024-04-16T16:41:15.044 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/tmp401.ko 2024-04-16T16:41:15.044 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/tmp421.ko 2024-04-16T16:41:15.044 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/via-cputemp.ko 2024-04-16T16:41:15.044 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/via686a.ko 2024-04-16T16:41:15.044 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/vt1211.ko 2024-04-16T16:41:15.044 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/vt8231.ko 2024-04-16T16:41:15.044 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/w83627ehf.ko 2024-04-16T16:41:15.044 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/w83627hf.ko 2024-04-16T16:41:15.045 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/w83781d.ko 2024-04-16T16:41:15.045 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/w83791d.ko 2024-04-16T16:41:15.045 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/w83792d.ko 2024-04-16T16:41:15.045 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/w83793.ko 2024-04-16T16:41:15.045 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/w83795.ko 2024-04-16T16:41:15.045 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/w83l785ts.ko 2024-04-16T16:41:15.045 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/hwmon/w83l786ng.ko 2024-04-16T16:41:15.045 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c 2024-04-16T16:41:15.045 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/algos 2024-04-16T16:41:15.045 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-04-16T16:41:15.045 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-04-16T16:41:15.045 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses 2024-04-16T16:41:15.045 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-04-16T16:41:15.045 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-04-16T16:41:15.045 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-04-16T16:41:15.045 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-04-16T16:41:15.045 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-04-16T16:41:15.046 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-04-16T16:41:15.046 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-04-16T16:41:15.046 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-04-16T16:41:15.046 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-i801.ko 2024-04-16T16:41:15.046 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-isch.ko 2024-04-16T16:41:15.046 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-04-16T16:41:15.046 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-04-16T16:41:15.046 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-04-16T16:41:15.046 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-parport.ko 2024-04-16T16:41:15.046 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-04-16T16:41:15.046 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-04-16T16:41:15.046 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-04-16T16:41:15.046 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-04-16T16:41:15.046 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-04-16T16:41:15.046 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-04-16T16:41:15.046 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-via.ko 2024-04-16T16:41:15.047 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-04-16T16:41:15.047 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-04-16T16:41:15.047 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/i2c-dev.ko 2024-04-16T16:41:15.047 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/i2c-mux.ko 2024-04-16T16:41:15.047 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-04-16T16:41:15.047 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/i2c-smbus.ko 2024-04-16T16:41:15.047 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/i2c/i2c-stub.ko 2024-04-16T16:41:15.047 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband 2024-04-16T16:41:15.047 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/core 2024-04-16T16:41:15.047 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/core/ib_cm.ko 2024-04-16T16:41:15.047 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/core/ib_core.ko 2024-04-16T16:41:15.047 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/core/ib_umad.ko 2024-04-16T16:41:15.047 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-04-16T16:41:15.047 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/core/iw_cm.ko 2024-04-16T16:41:15.047 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/core/rdma_cm.ko 2024-04-16T16:41:15.047 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-04-16T16:41:15.047 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/hw 2024-04-16T16:41:15.048 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/hw/cxgb4 2024-04-16T16:41:15.048 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-04-16T16:41:15.048 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/hw/mlx4 2024-04-16T16:41:15.048 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-04-16T16:41:15.048 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/hw/mlx5 2024-04-16T16:41:15.048 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-04-16T16:41:15.048 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/hw/mthca 2024-04-16T16:41:15.048 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-04-16T16:41:15.048 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/ulp 2024-04-16T16:41:15.048 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/ulp/ipoib 2024-04-16T16:41:15.048 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-04-16T16:41:15.048 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/ulp/iser 2024-04-16T16:41:15.048 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-04-16T16:41:15.048 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/ulp/isert 2024-04-16T16:41:15.048 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-04-16T16:41:15.048 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/ulp/srp 2024-04-16T16:41:15.049 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-04-16T16:41:15.049 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/ulp/srpt 2024-04-16T16:41:15.049 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-04-16T16:41:15.049 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input 2024-04-16T16:41:15.049 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/gameport 2024-04-16T16:41:15.049 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-04-16T16:41:15.049 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/gameport/fm801-gp.ko 2024-04-16T16:41:15.049 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/gameport/gameport.ko 2024-04-16T16:41:15.049 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/joydev.ko 2024-04-16T16:41:15.049 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/keyboard 2024-04-16T16:41:15.049 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/keyboard/gpio_keys.ko 2024-04-16T16:41:15.049 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/mouse 2024-04-16T16:41:15.049 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/mouse/appletouch.ko 2024-04-16T16:41:15.049 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/mouse/bcm5974.ko 2024-04-16T16:41:15.049 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/mouse/cyapatp.ko 2024-04-16T16:41:15.049 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/mouse/elan_i2c.ko 2024-04-16T16:41:15.050 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/mouse/sermouse.ko 2024-04-16T16:41:15.050 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-04-16T16:41:15.050 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/mouse/synaptics_usb.ko 2024-04-16T16:41:15.050 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/mouse/vsxxxaa.ko 2024-04-16T16:41:15.050 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/rmi4 2024-04-16T16:41:15.050 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/rmi4/rmi_core.ko 2024-04-16T16:41:15.050 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/serio 2024-04-16T16:41:15.050 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/serio/altera_ps2.ko 2024-04-16T16:41:15.050 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/serio/arc_ps2.ko 2024-04-16T16:41:15.050 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-04-16T16:41:15.050 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/serio/serio_raw.ko 2024-04-16T16:41:15.050 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/input/sparse-keymap.ko 2024-04-16T16:41:15.050 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds 2024-04-16T16:41:15.050 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/leds-blinkm.ko 2024-04-16T16:41:15.050 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/leds-lm3530.ko 2024-04-16T16:41:15.051 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/leds-lp3944.ko 2024-04-16T16:41:15.051 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/leds-ss4200.ko 2024-04-16T16:41:15.051 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/trigger 2024-04-16T16:41:15.051 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-04-16T16:41:15.051 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-04-16T16:41:15.051 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-04-16T16:41:15.051 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-04-16T16:41:15.051 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-04-16T16:41:15.051 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-04-16T16:41:15.051 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-04-16T16:41:15.051 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-04-16T16:41:15.051 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-04-16T16:41:15.051 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md 2024-04-16T16:41:15.051 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/bcache 2024-04-16T16:41:15.051 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/bcache/bcache.ko 2024-04-16T16:41:15.051 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-bio-prison.ko 2024-04-16T16:41:15.051 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-cache-smq.ko 2024-04-16T16:41:15.052 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-cache.ko 2024-04-16T16:41:15.052 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-crypt.ko 2024-04-16T16:41:15.052 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-delay.ko 2024-04-16T16:41:15.052 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-flakey.ko 2024-04-16T16:41:15.052 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-log-userspace.ko 2024-04-16T16:41:15.052 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-multipath.ko 2024-04-16T16:41:15.052 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-queue-length.ko 2024-04-16T16:41:15.052 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-raid.ko 2024-04-16T16:41:15.052 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-round-robin.ko 2024-04-16T16:41:15.052 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-service-time.ko 2024-04-16T16:41:15.052 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-switch.ko 2024-04-16T16:41:15.052 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-thin-pool.ko 2024-04-16T16:41:15.052 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/dm-verity.ko 2024-04-16T16:41:15.052 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/persistent-data 2024-04-16T16:41:15.052 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-04-16T16:41:15.052 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/raid0.ko 2024-04-16T16:41:15.053 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/raid1.ko 2024-04-16T16:41:15.053 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/raid10.ko 2024-04-16T16:41:15.053 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/md/raid456.ko 2024-04-16T16:41:15.053 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/message 2024-04-16T16:41:15.053 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/message/fusion 2024-04-16T16:41:15.053 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/message/fusion/mptbase.ko 2024-04-16T16:41:15.053 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/message/fusion/mptctl.ko 2024-04-16T16:41:15.053 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/message/fusion/mptfc.ko 2024-04-16T16:41:15.053 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/message/fusion/mptlan.ko 2024-04-16T16:41:15.053 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/message/fusion/mptsas.ko 2024-04-16T16:41:15.053 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/message/fusion/mptscsih.ko 2024-04-16T16:41:15.053 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/message/fusion/mptspi.ko 2024-04-16T16:41:15.053 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/mfd 2024-04-16T16:41:15.053 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/mfd/lpc_ich.ko 2024-04-16T16:41:15.053 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/mfd/lpc_sch.ko 2024-04-16T16:41:15.053 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/mfd/mfd-core.ko 2024-04-16T16:41:15.054 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/mfd/sm501.ko 2024-04-16T16:41:15.054 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/mfd/viperboard.ko 2024-04-16T16:41:15.054 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/mfd/vx855.ko 2024-04-16T16:41:15.054 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/mfd/wl1273-core.ko 2024-04-16T16:41:15.054 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc 2024-04-16T16:41:15.054 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/altera-stapl 2024-04-16T16:41:15.054 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-04-16T16:41:15.054 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/apds9802als.ko 2024-04-16T16:41:15.054 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/apds990x.ko 2024-04-16T16:41:15.054 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/bh1770glc.ko 2024-04-16T16:41:15.054 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/cb710 2024-04-16T16:41:15.054 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/cb710/cb710.ko 2024-04-16T16:41:15.054 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/eeprom 2024-04-16T16:41:15.054 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/eeprom/at24.ko 2024-04-16T16:41:15.054 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-04-16T16:41:15.054 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/eeprom/max6875.ko 2024-04-16T16:41:15.055 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/enclosure.ko 2024-04-16T16:41:15.055 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/hpilo.ko 2024-04-16T16:41:15.055 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/isl29003.ko 2024-04-16T16:41:15.055 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/isl29020.ko 2024-04-16T16:41:15.055 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/lis3lv02d 2024-04-16T16:41:15.055 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-04-16T16:41:15.055 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-04-16T16:41:15.055 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/mei 2024-04-16T16:41:15.055 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/mei/mei-me.ko 2024-04-16T16:41:15.055 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/mei/mei-txe.ko 2024-04-16T16:41:15.055 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/mei/mei.ko 2024-04-16T16:41:15.055 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/sgi-gru 2024-04-16T16:41:15.055 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/sgi-gru/gru.ko 2024-04-16T16:41:15.055 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/sgi-xp 2024-04-16T16:41:15.055 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/sgi-xp/xp.ko 2024-04-16T16:41:15.055 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/sgi-xp/xpc.ko 2024-04-16T16:41:15.055 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-04-16T16:41:15.056 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/tifm_7xx1.ko 2024-04-16T16:41:15.056 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/tifm_core.ko 2024-04-16T16:41:15.056 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/tsl2550.ko 2024-04-16T16:41:15.056 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/vmw_balloon.ko 2024-04-16T16:41:15.056 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/vmw_vmci 2024-04-16T16:41:15.056 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-04-16T16:41:15.056 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net 2024-04-16T16:41:15.056 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/bonding 2024-04-16T16:41:15.056 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/bonding/bonding.ko 2024-04-16T16:41:15.056 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/dsa 2024-04-16T16:41:15.056 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/dsa/b53 2024-04-16T16:41:15.056 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/dsa/b53/b53_common.ko 2024-04-16T16:41:15.056 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/dsa/bcm-sf2.ko 2024-04-16T16:41:15.056 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/dsa/mv88e6060.ko 2024-04-16T16:41:15.056 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/dsa/mv88e6xxx 2024-04-16T16:41:15.056 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-04-16T16:41:15.057 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/dummy.ko 2024-04-16T16:41:15.057 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/eql.ko 2024-04-16T16:41:15.057 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet 2024-04-16T16:41:15.057 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/3com 2024-04-16T16:41:15.057 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-04-16T16:41:15.057 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-04-16T16:41:15.057 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-04-16T16:41:15.057 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-04-16T16:41:15.057 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/8390 2024-04-16T16:41:15.057 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/8390/8390.ko 2024-04-16T16:41:15.057 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-04-16T16:41:15.057 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-04-16T16:41:15.057 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-04-16T16:41:15.057 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/adaptec 2024-04-16T16:41:15.057 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-04-16T16:41:15.057 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/agere 2024-04-16T16:41:15.057 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/agere/et131x.ko 2024-04-16T16:41:15.058 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/alteon 2024-04-16T16:41:15.058 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-04-16T16:41:15.058 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/altera 2024-04-16T16:41:15.058 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-04-16T16:41:15.058 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/amd 2024-04-16T16:41:15.058 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-04-16T16:41:15.058 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-04-16T16:41:15.058 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-04-16T16:41:15.058 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/atheros 2024-04-16T16:41:15.058 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/atheros/alx 2024-04-16T16:41:15.058 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-04-16T16:41:15.058 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/atheros/atl1c 2024-04-16T16:41:15.058 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-04-16T16:41:15.058 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/atheros/atl1e 2024-04-16T16:41:15.058 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-04-16T16:41:15.058 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/atheros/atlx 2024-04-16T16:41:15.059 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-04-16T16:41:15.059 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-04-16T16:41:15.059 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/broadcom 2024-04-16T16:41:15.059 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-04-16T16:41:15.059 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-04-16T16:41:15.059 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-04-16T16:41:15.059 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-04-16T16:41:15.059 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-04-16T16:41:15.059 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/broadcom/genet 2024-04-16T16:41:15.059 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-04-16T16:41:15.059 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-04-16T16:41:15.059 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/brocade 2024-04-16T16:41:15.059 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/brocade/bna 2024-04-16T16:41:15.059 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-04-16T16:41:15.059 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/chelsio 2024-04-16T16:41:15.060 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/chelsio/cxgb 2024-04-16T16:41:15.060 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-04-16T16:41:15.060 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-04-16T16:41:15.060 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-04-16T16:41:15.060 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-04-16T16:41:15.060 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-04-16T16:41:15.060 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-04-16T16:41:15.060 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-04-16T16:41:15.060 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-04-16T16:41:15.060 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-04-16T16:41:15.060 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/cisco 2024-04-16T16:41:15.060 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/cisco/enic 2024-04-16T16:41:15.060 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-04-16T16:41:15.060 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/dec 2024-04-16T16:41:15.060 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/dec/tulip 2024-04-16T16:41:15.061 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-04-16T16:41:15.061 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-04-16T16:41:15.061 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-04-16T16:41:15.061 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-04-16T16:41:15.061 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-04-16T16:41:15.061 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-04-16T16:41:15.061 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/dlink 2024-04-16T16:41:15.061 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-04-16T16:41:15.061 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-04-16T16:41:15.061 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/dnet.ko 2024-04-16T16:41:15.061 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/emulex 2024-04-16T16:41:15.061 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/emulex/benet 2024-04-16T16:41:15.061 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-04-16T16:41:15.061 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/ethoc.ko 2024-04-16T16:41:15.061 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/fealnx.ko 2024-04-16T16:41:15.061 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel 2024-04-16T16:41:15.062 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/e100.ko 2024-04-16T16:41:15.062 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/e1000 2024-04-16T16:41:15.062 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-04-16T16:41:15.062 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/e1000e 2024-04-16T16:41:15.062 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-04-16T16:41:15.062 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/fm10k 2024-04-16T16:41:15.062 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-04-16T16:41:15.062 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/i40e 2024-04-16T16:41:15.062 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-04-16T16:41:15.062 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/iavf 2024-04-16T16:41:15.062 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-04-16T16:41:15.062 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/igb 2024-04-16T16:41:15.062 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-04-16T16:41:15.062 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/igbvf 2024-04-16T16:41:15.063 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-04-16T16:41:15.063 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/ixgbe 2024-04-16T16:41:15.063 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-04-16T16:41:15.063 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/ixgbevf 2024-04-16T16:41:15.063 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-04-16T16:41:15.063 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/jme.ko 2024-04-16T16:41:15.063 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/marvell 2024-04-16T16:41:15.063 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-04-16T16:41:15.063 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/marvell/skge.ko 2024-04-16T16:41:15.063 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-04-16T16:41:15.063 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/mellanox 2024-04-16T16:41:15.063 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/mellanox/mlx4 2024-04-16T16:41:15.063 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-04-16T16:41:15.063 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-04-16T16:41:15.063 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/mellanox/mlx5 2024-04-16T16:41:15.063 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-04-16T16:41:15.063 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-04-16T16:41:15.064 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/micrel 2024-04-16T16:41:15.064 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-04-16T16:41:15.064 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/myricom 2024-04-16T16:41:15.064 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/myricom/myri10ge 2024-04-16T16:41:15.064 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-04-16T16:41:15.064 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/natsemi 2024-04-16T16:41:15.064 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-04-16T16:41:15.064 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-04-16T16:41:15.064 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/neterion 2024-04-16T16:41:15.064 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-04-16T16:41:15.064 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/nvidia 2024-04-16T16:41:15.064 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-04-16T16:41:15.064 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/packetengines 2024-04-16T16:41:15.064 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-04-16T16:41:15.064 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-04-16T16:41:15.064 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/qlogic 2024-04-16T16:41:15.065 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/qlogic/netxen 2024-04-16T16:41:15.065 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-04-16T16:41:15.065 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-04-16T16:41:15.065 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-04-16T16:41:15.065 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-04-16T16:41:15.065 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/rdc 2024-04-16T16:41:15.065 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-04-16T16:41:15.065 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/realtek 2024-04-16T16:41:15.065 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-04-16T16:41:15.065 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-04-16T16:41:15.065 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/realtek/atp.ko 2024-04-16T16:41:15.065 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-04-16T16:41:15.065 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/rocker 2024-04-16T16:41:15.065 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-04-16T16:41:15.065 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/sfc 2024-04-16T16:41:15.066 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-04-16T16:41:15.066 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/silan 2024-04-16T16:41:15.066 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-04-16T16:41:15.066 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/sis 2024-04-16T16:41:15.066 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/sis/sis190.ko 2024-04-16T16:41:15.066 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/sis/sis900.ko 2024-04-16T16:41:15.066 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/smsc 2024-04-16T16:41:15.066 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-04-16T16:41:15.066 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-04-16T16:41:15.066 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-04-16T16:41:15.066 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-04-16T16:41:15.066 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/stmicro 2024-04-16T16:41:15.066 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/stmicro/stmmac 2024-04-16T16:41:15.066 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-04-16T16:41:15.066 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-04-16T16:41:15.066 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/sun 2024-04-16T16:41:15.067 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/sun/cassini.ko 2024-04-16T16:41:15.067 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/sun/niu.ko 2024-04-16T16:41:15.067 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/sun/sungem.ko 2024-04-16T16:41:15.067 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-04-16T16:41:15.067 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/tehuti 2024-04-16T16:41:15.067 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-04-16T16:41:15.067 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/ti 2024-04-16T16:41:15.067 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/ti/tlan.ko 2024-04-16T16:41:15.067 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/via 2024-04-16T16:41:15.067 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-04-16T16:41:15.067 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-04-16T16:41:15.067 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/wiznet 2024-04-16T16:41:15.067 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-04-16T16:41:15.067 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-04-16T16:41:15.067 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/xircom 2024-04-16T16:41:15.067 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-04-16T16:41:15.067 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/geneve.ko 2024-04-16T16:41:15.068 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/hyperv 2024-04-16T16:41:15.068 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-04-16T16:41:15.068 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ieee802154 2024-04-16T16:41:15.068 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ieee802154/fakelb.ko 2024-04-16T16:41:15.068 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ifb.ko 2024-04-16T16:41:15.068 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ipvlan 2024-04-16T16:41:15.068 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ipvlan/ipvlan.ko 2024-04-16T16:41:15.068 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/macvlan.ko 2024-04-16T16:41:15.068 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/macvtap.ko 2024-04-16T16:41:15.068 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/mdio 2024-04-16T16:41:15.068 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/mdio.ko 2024-04-16T16:41:15.068 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-04-16T16:41:15.068 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-04-16T16:41:15.068 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/mdio/mdio-regmap.ko 2024-04-16T16:41:15.068 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/mii.ko 2024-04-16T16:41:15.068 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/net_failover.ko 2024-04-16T16:41:15.069 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/netconsole.ko 2024-04-16T16:41:15.069 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/nlmon.ko 2024-04-16T16:41:15.069 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/pcs 2024-04-16T16:41:15.069 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/pcs/pcs-lynx.ko 2024-04-16T16:41:15.069 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-04-16T16:41:15.069 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy 2024-04-16T16:41:15.069 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/amd.ko 2024-04-16T16:41:15.069 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-04-16T16:41:15.069 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-04-16T16:41:15.069 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/bcm7xxx.ko 2024-04-16T16:41:15.069 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/bcm87xx.ko 2024-04-16T16:41:15.069 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/broadcom.ko 2024-04-16T16:41:15.069 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/cicada.ko 2024-04-16T16:41:15.069 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/davicom.ko 2024-04-16T16:41:15.069 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/dp83640.ko 2024-04-16T16:41:15.069 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/et1011c.ko 2024-04-16T16:41:15.070 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/icplus.ko 2024-04-16T16:41:15.070 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/lxt.ko 2024-04-16T16:41:15.070 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/marvell.ko 2024-04-16T16:41:15.070 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/micrel.ko 2024-04-16T16:41:15.070 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/national.ko 2024-04-16T16:41:15.070 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/phylink.ko 2024-04-16T16:41:15.070 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/qsemi.ko 2024-04-16T16:41:15.070 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/realtek.ko 2024-04-16T16:41:15.070 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/smsc.ko 2024-04-16T16:41:15.070 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/ste10Xp.ko 2024-04-16T16:41:15.070 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/phy/vitesse.ko 2024-04-16T16:41:15.070 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ppp 2024-04-16T16:41:15.070 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ppp/bsd_comp.ko 2024-04-16T16:41:15.070 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ppp/ppp_async.ko 2024-04-16T16:41:15.070 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ppp/ppp_deflate.ko 2024-04-16T16:41:15.070 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ppp/ppp_generic.ko 2024-04-16T16:41:15.071 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ppp/ppp_mppe.ko 2024-04-16T16:41:15.071 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ppp/ppp_synctty.ko 2024-04-16T16:41:15.071 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ppp/pppoe.ko 2024-04-16T16:41:15.071 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ppp/pppox.ko 2024-04-16T16:41:15.071 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/ppp/pptp.ko 2024-04-16T16:41:15.071 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/slip 2024-04-16T16:41:15.071 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/slip/slhc.ko 2024-04-16T16:41:15.071 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/slip/slip.ko 2024-04-16T16:41:15.071 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/sungem_phy.ko 2024-04-16T16:41:15.071 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/tap.ko 2024-04-16T16:41:15.071 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/team 2024-04-16T16:41:15.071 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/team/team.ko 2024-04-16T16:41:15.071 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/team/team_mode_activebackup.ko 2024-04-16T16:41:15.071 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/team/team_mode_broadcast.ko 2024-04-16T16:41:15.071 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-04-16T16:41:15.071 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/team/team_mode_random.ko 2024-04-16T16:41:15.071 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-04-16T16:41:15.072 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/tun.ko 2024-04-16T16:41:15.072 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/veth.ko 2024-04-16T16:41:15.072 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/virtio_net.ko 2024-04-16T16:41:15.072 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/vmxnet3 2024-04-16T16:41:15.072 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-04-16T16:41:15.072 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/vxlan 2024-04-16T16:41:15.072 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/vxlan/vxlan.ko 2024-04-16T16:41:15.072 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/xen-netback 2024-04-16T16:41:15.072 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/xen-netback/xen-netback.ko 2024-04-16T16:41:15.072 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/net/xen-netfront.ko 2024-04-16T16:41:15.072 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ntb 2024-04-16T16:41:15.072 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ntb/ntb.ko 2024-04-16T16:41:15.072 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/nvme 2024-04-16T16:41:15.072 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/nvme/host 2024-04-16T16:41:15.072 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/nvme/host/nvme-core.ko 2024-04-16T16:41:15.073 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/nvme/host/nvme.ko 2024-04-16T16:41:15.073 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/parport 2024-04-16T16:41:15.073 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/parport/parport.ko 2024-04-16T16:41:15.073 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/parport/parport_cs.ko 2024-04-16T16:41:15.073 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/parport/parport_pc.ko 2024-04-16T16:41:15.073 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/parport/parport_serial.ko 2024-04-16T16:41:15.073 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pci 2024-04-16T16:41:15.073 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pci/hotplug 2024-04-16T16:41:15.073 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-04-16T16:41:15.073 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pci/pcie 2024-04-16T16:41:15.073 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pci/pcie/aer_inject.ko 2024-04-16T16:41:15.073 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pci/xen-pcifront.ko 2024-04-16T16:41:15.073 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pcmcia 2024-04-16T16:41:15.073 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pcmcia/i82092.ko 2024-04-16T16:41:15.073 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pcmcia/pd6729.ko 2024-04-16T16:41:15.073 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pcmcia/yenta_socket.ko 2024-04-16T16:41:15.074 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pinctrl 2024-04-16T16:41:15.074 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pinctrl/intel 2024-04-16T16:41:15.074 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-04-16T16:41:15.074 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform 2024-04-16T16:41:15.074 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86 2024-04-16T16:41:15.074 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/acer-wmi.ko 2024-04-16T16:41:15.074 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/acerhdf.ko 2024-04-16T16:41:15.074 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/apple-gmux.ko 2024-04-16T16:41:15.074 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/asus-laptop.ko 2024-04-16T16:41:15.074 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-04-16T16:41:15.074 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/asus-wmi.ko 2024-04-16T16:41:15.074 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/classmate-laptop.ko 2024-04-16T16:41:15.074 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-04-16T16:41:15.074 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-04-16T16:41:15.074 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-04-16T16:41:15.074 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-04-16T16:41:15.075 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/hdaps.ko 2024-04-16T16:41:15.075 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/intel 2024-04-16T16:41:15.075 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-04-16T16:41:15.075 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/intel_ips.ko 2024-04-16T16:41:15.075 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/msi-wmi.ko 2024-04-16T16:41:15.075 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/mxm-wmi.ko 2024-04-16T16:41:15.075 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-04-16T16:41:15.075 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/samsung-laptop.ko 2024-04-16T16:41:15.075 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/samsung-q10.ko 2024-04-16T16:41:15.075 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/topstar-laptop.ko 2024-04-16T16:41:15.075 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-04-16T16:41:15.075 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/toshiba_haps.ko 2024-04-16T16:41:15.075 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/wmi-bmof.ko 2024-04-16T16:41:15.075 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/platform/x86/wmi.ko 2024-04-16T16:41:15.075 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/powercap 2024-04-16T16:41:15.075 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/powercap/intel_rapl_common.ko 2024-04-16T16:41:15.075 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/powercap/intel_rapl_msr.ko 2024-04-16T16:41:15.076 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pps 2024-04-16T16:41:15.076 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pps/clients 2024-04-16T16:41:15.076 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pps/clients/pps-gpio.ko 2024-04-16T16:41:15.076 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pps/clients/pps-ldisc.ko 2024-04-16T16:41:15.076 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pps/clients/pps_parport.ko 2024-04-16T16:41:15.076 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/pps/pps_core.ko 2024-04-16T16:41:15.076 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ptp 2024-04-16T16:41:15.076 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ptp/ptp.ko 2024-04-16T16:41:15.076 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ptp/ptp_kvm.ko 2024-04-16T16:41:15.076 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ras 2024-04-16T16:41:15.076 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ras/amd 2024-04-16T16:41:15.076 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ras/amd/atl 2024-04-16T16:41:15.076 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-04-16T16:41:15.076 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ras/amd/fmpm.ko 2024-04-16T16:41:15.076 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc 2024-04-16T16:41:15.076 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-bq32k.ko 2024-04-16T16:41:15.076 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-ds1286.ko 2024-04-16T16:41:15.077 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-ds1307.ko 2024-04-16T16:41:15.077 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-ds1374.ko 2024-04-16T16:41:15.077 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-ds1511.ko 2024-04-16T16:41:15.077 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-ds1553.ko 2024-04-16T16:41:15.077 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-ds1672.ko 2024-04-16T16:41:15.077 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-ds1742.ko 2024-04-16T16:41:15.077 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-ds2404.ko 2024-04-16T16:41:15.077 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-ds3232.ko 2024-04-16T16:41:15.077 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-em3027.ko 2024-04-16T16:41:15.077 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-fm3130.ko 2024-04-16T16:41:15.077 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-isl12022.ko 2024-04-16T16:41:15.077 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-isl1208.ko 2024-04-16T16:41:15.077 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-m41t80.ko 2024-04-16T16:41:15.077 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-m48t35.ko 2024-04-16T16:41:15.077 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-m48t59.ko 2024-04-16T16:41:15.077 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-max6900.ko 2024-04-16T16:41:15.078 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-msm6242.ko 2024-04-16T16:41:15.078 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-pcf2127.ko 2024-04-16T16:41:15.078 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-pcf85063.ko 2024-04-16T16:41:15.078 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-pcf8523.ko 2024-04-16T16:41:15.078 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-pcf8563.ko 2024-04-16T16:41:15.078 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-pcf8583.ko 2024-04-16T16:41:15.078 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-rp5c01.ko 2024-04-16T16:41:15.078 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-rs5c372.ko 2024-04-16T16:41:15.078 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-04-16T16:41:15.078 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-rx8025.ko 2024-04-16T16:41:15.078 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-rx8581.ko 2024-04-16T16:41:15.078 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-04-16T16:41:15.078 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/rtc/rtc-x1205.ko 2024-04-16T16:41:15.078 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi 2024-04-16T16:41:15.078 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/3w-9xxx.ko 2024-04-16T16:41:15.078 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/3w-sas.ko 2024-04-16T16:41:15.078 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/3w-xxxx.ko 2024-04-16T16:41:15.079 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/BusLogic.ko 2024-04-16T16:41:15.079 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/a100u2w.ko 2024-04-16T16:41:15.079 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/aacraid 2024-04-16T16:41:15.079 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/aacraid/aacraid.ko 2024-04-16T16:41:15.079 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/advansys.ko 2024-04-16T16:41:15.079 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/aic7xxx 2024-04-16T16:41:15.079 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-04-16T16:41:15.079 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-04-16T16:41:15.079 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/am53c974.ko 2024-04-16T16:41:15.079 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/arcmsr 2024-04-16T16:41:15.079 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-04-16T16:41:15.079 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/atp870u.ko 2024-04-16T16:41:15.079 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/be2iscsi 2024-04-16T16:41:15.079 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-04-16T16:41:15.079 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/bfa 2024-04-16T16:41:15.079 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/bfa/bfa.ko 2024-04-16T16:41:15.080 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/bnx2fc 2024-04-16T16:41:15.080 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-04-16T16:41:15.080 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/bnx2i 2024-04-16T16:41:15.080 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-04-16T16:41:15.080 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/ch.ko 2024-04-16T16:41:15.080 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/csiostor 2024-04-16T16:41:15.080 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/csiostor/csiostor.ko 2024-04-16T16:41:15.080 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/cxgbi 2024-04-16T16:41:15.080 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/cxgbi/cxgb3i 2024-04-16T16:41:15.080 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-04-16T16:41:15.080 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/cxgbi/cxgb4i 2024-04-16T16:41:15.080 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-04-16T16:41:15.080 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-04-16T16:41:15.080 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/dc395x.ko 2024-04-16T16:41:15.080 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/device_handler 2024-04-16T16:41:15.080 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-04-16T16:41:15.081 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-04-16T16:41:15.081 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-04-16T16:41:15.081 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-04-16T16:41:15.081 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/dmx3191d.ko 2024-04-16T16:41:15.081 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/esas2r 2024-04-16T16:41:15.081 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/esas2r/esas2r.ko 2024-04-16T16:41:15.081 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/esp_scsi.ko 2024-04-16T16:41:15.081 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/fcoe 2024-04-16T16:41:15.081 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/fcoe/fcoe.ko 2024-04-16T16:41:15.081 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-04-16T16:41:15.081 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/fnic 2024-04-16T16:41:15.081 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/fnic/fnic.ko 2024-04-16T16:41:15.081 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/hpsa.ko 2024-04-16T16:41:15.081 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/hptiop.ko 2024-04-16T16:41:15.081 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/hv_storvsc.ko 2024-04-16T16:41:15.081 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/initio.ko 2024-04-16T16:41:15.081 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/ipr.ko 2024-04-16T16:41:15.082 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/ips.ko 2024-04-16T16:41:15.082 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/isci 2024-04-16T16:41:15.082 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/isci/isci.ko 2024-04-16T16:41:15.082 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-04-16T16:41:15.082 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/iscsi_tcp.ko 2024-04-16T16:41:15.082 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/libfc 2024-04-16T16:41:15.082 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/libfc/libfc.ko 2024-04-16T16:41:15.082 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/libiscsi.ko 2024-04-16T16:41:15.082 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/libiscsi_tcp.ko 2024-04-16T16:41:15.082 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/libsas 2024-04-16T16:41:15.082 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/libsas/libsas.ko 2024-04-16T16:41:15.082 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/lpfc 2024-04-16T16:41:15.082 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/lpfc/lpfc.ko 2024-04-16T16:41:15.082 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/megaraid 2024-04-16T16:41:15.082 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/megaraid.ko 2024-04-16T16:41:15.082 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-04-16T16:41:15.083 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-04-16T16:41:15.083 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-04-16T16:41:15.083 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/mpt3sas 2024-04-16T16:41:15.083 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-04-16T16:41:15.083 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/mvsas 2024-04-16T16:41:15.083 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/mvsas/mvsas.ko 2024-04-16T16:41:15.083 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/mvumi.ko 2024-04-16T16:41:15.083 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/pm8001 2024-04-16T16:41:15.083 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-04-16T16:41:15.083 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/pmcraid.ko 2024-04-16T16:41:15.083 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/qla1280.ko 2024-04-16T16:41:15.083 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/qla2xxx 2024-04-16T16:41:15.083 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-04-16T16:41:15.083 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-04-16T16:41:15.083 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/qla4xxx 2024-04-16T16:41:15.083 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-04-16T16:41:15.083 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/raid_class.ko 2024-04-16T16:41:15.084 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/scsi_debug.ko 2024-04-16T16:41:15.084 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/scsi_transport_fc.ko 2024-04-16T16:41:15.084 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-04-16T16:41:15.084 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/scsi_transport_sas.ko 2024-04-16T16:41:15.084 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/scsi_transport_spi.ko 2024-04-16T16:41:15.084 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/scsi_transport_srp.ko 2024-04-16T16:41:15.084 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/ses.ko 2024-04-16T16:41:15.084 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/st.ko 2024-04-16T16:41:15.084 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/stex.ko 2024-04-16T16:41:15.084 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/sym53c8xx_2 2024-04-16T16:41:15.084 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-04-16T16:41:15.084 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/virtio_scsi.ko 2024-04-16T16:41:15.084 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/vmw_pvscsi.ko 2024-04-16T16:41:15.084 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/scsi/wd719x.ko 2024-04-16T16:41:15.084 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ssb 2024-04-16T16:41:15.085 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ssb/ssb.ko 2024-04-16T16:41:15.085 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/target 2024-04-16T16:41:15.085 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/target/iscsi 2024-04-16T16:41:15.085 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-04-16T16:41:15.085 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/target/loopback 2024-04-16T16:41:15.085 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/target/loopback/tcm_loop.ko 2024-04-16T16:41:15.085 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/target/target_core_file.ko 2024-04-16T16:41:15.085 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/target/target_core_iblock.ko 2024-04-16T16:41:15.085 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/target/target_core_mod.ko 2024-04-16T16:41:15.085 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/target/target_core_pscsi.ko 2024-04-16T16:41:15.085 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/target/target_core_user.ko 2024-04-16T16:41:15.085 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/target/tcm_fc 2024-04-16T16:41:15.085 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-04-16T16:41:15.085 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal 2024-04-16T16:41:15.085 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel 2024-04-16T16:41:15.085 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal 2024-04-16T16:41:15.085 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-04-16T16:41:15.086 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-04-16T16:41:15.086 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-04-16T16:41:15.086 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-04-16T16:41:15.086 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-04-16T16:41:15.086 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-04-16T16:41:15.086 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-04-16T16:41:15.086 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-04-16T16:41:15.086 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-04-16T16:41:15.086 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-04-16T16:41:15.086 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-04-16T16:41:15.086 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-04-16T16:41:15.086 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-04-16T16:41:15.086 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-04-16T16:41:15.086 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-04-16T16:41:15.086 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-04-16T16:41:15.087 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-04-16T16:41:15.087 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-04-16T16:41:15.087 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/tty 2024-04-16T16:41:15.087 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/tty/ipwireless 2024-04-16T16:41:15.087 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-04-16T16:41:15.087 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/tty/n_gsm.ko 2024-04-16T16:41:15.087 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/tty/n_hdlc.ko 2024-04-16T16:41:15.087 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/tty/nozomi.ko 2024-04-16T16:41:15.087 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/tty/serial 2024-04-16T16:41:15.087 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/tty/serial/8250 2024-04-16T16:41:15.087 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-04-16T16:41:15.087 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/tty/serial/arc_uart.ko 2024-04-16T16:41:15.087 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/tty/serial/jsm 2024-04-16T16:41:15.087 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/tty/serial/jsm/jsm.ko 2024-04-16T16:41:15.087 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ufs 2024-04-16T16:41:15.087 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ufs/core 2024-04-16T16:41:15.087 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ufs/core/ufshcd-core.ko 2024-04-16T16:41:15.088 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ufs/host 2024-04-16T16:41:15.088 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-04-16T16:41:15.088 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/uio 2024-04-16T16:41:15.088 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/uio/uio.ko 2024-04-16T16:41:15.088 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/uio/uio_aec.ko 2024-04-16T16:41:15.088 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/uio/uio_cif.ko 2024-04-16T16:41:15.088 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/uio/uio_pci_generic.ko 2024-04-16T16:41:15.088 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/uio/uio_sercos3.ko 2024-04-16T16:41:15.088 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb 2024-04-16T16:41:15.088 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/class 2024-04-16T16:41:15.088 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/class/cdc-acm.ko 2024-04-16T16:41:15.088 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/class/cdc-wdm.ko 2024-04-16T16:41:15.088 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/class/usblp.ko 2024-04-16T16:41:15.088 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/class/usbtmc.ko 2024-04-16T16:41:15.088 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/host 2024-04-16T16:41:15.088 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/host/sl811-hcd.ko 2024-04-16T16:41:15.088 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/image 2024-04-16T16:41:15.089 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/image/mdc800.ko 2024-04-16T16:41:15.089 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/image/microtek.ko 2024-04-16T16:41:15.089 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc 2024-04-16T16:41:15.089 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/adutux.ko 2024-04-16T16:41:15.089 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/appledisplay.ko 2024-04-16T16:41:15.089 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/emi26.ko 2024-04-16T16:41:15.089 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/emi62.ko 2024-04-16T16:41:15.089 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/ezusb.ko 2024-04-16T16:41:15.089 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/idmouse.ko 2024-04-16T16:41:15.089 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/iowarrior.ko 2024-04-16T16:41:15.089 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/isight_firmware.ko 2024-04-16T16:41:15.089 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/ldusb.ko 2024-04-16T16:41:15.089 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/legousbtower.ko 2024-04-16T16:41:15.089 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/sisusbvga 2024-04-16T16:41:15.089 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-04-16T16:41:15.089 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/trancevibrator.ko 2024-04-16T16:41:15.090 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/usb3503.ko 2024-04-16T16:41:15.090 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/usblcd.ko 2024-04-16T16:41:15.090 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/usbsevseg.ko 2024-04-16T16:41:15.090 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/uss720.ko 2024-04-16T16:41:15.090 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/misc/yurex.ko 2024-04-16T16:41:15.090 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/phy 2024-04-16T16:41:15.090 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/phy/phy-generic.ko 2024-04-16T16:41:15.090 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial 2024-04-16T16:41:15.090 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/aircable.ko 2024-04-16T16:41:15.090 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/ark3116.ko 2024-04-16T16:41:15.090 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/belkin_sa.ko 2024-04-16T16:41:15.090 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/ch341.ko 2024-04-16T16:41:15.090 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/cp210x.ko 2024-04-16T16:41:15.090 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/cyberjack.ko 2024-04-16T16:41:15.091 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/cypress_m8.ko 2024-04-16T16:41:15.091 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/digi_acceleport.ko 2024-04-16T16:41:15.091 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/empeg.ko 2024-04-16T16:41:15.091 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/ftdi_sio.ko 2024-04-16T16:41:15.091 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/garmin_gps.ko 2024-04-16T16:41:15.091 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/io_edgeport.ko 2024-04-16T16:41:15.091 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/io_ti.ko 2024-04-16T16:41:15.091 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/ipaq.ko 2024-04-16T16:41:15.091 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/ipw.ko 2024-04-16T16:41:15.091 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/ir-usb.ko 2024-04-16T16:41:15.091 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-04-16T16:41:15.091 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/keyspan.ko 2024-04-16T16:41:15.091 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/keyspan_pda.ko 2024-04-16T16:41:15.091 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/kl5kusb105.ko 2024-04-16T16:41:15.092 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/kobil_sct.ko 2024-04-16T16:41:15.092 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/mct_u232.ko 2024-04-16T16:41:15.092 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/mos7720.ko 2024-04-16T16:41:15.092 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/mos7840.ko 2024-04-16T16:41:15.092 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/navman.ko 2024-04-16T16:41:15.092 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/omninet.ko 2024-04-16T16:41:15.092 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/opticon.ko 2024-04-16T16:41:15.092 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/option.ko 2024-04-16T16:41:15.092 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/oti6858.ko 2024-04-16T16:41:15.092 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/pl2303.ko 2024-04-16T16:41:15.092 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/qcaux.ko 2024-04-16T16:41:15.092 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/qcserial.ko 2024-04-16T16:41:15.092 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/quatech2.ko 2024-04-16T16:41:15.092 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/safe_serial.ko 2024-04-16T16:41:15.092 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/sierra.ko 2024-04-16T16:41:15.092 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/spcp8x5.ko 2024-04-16T16:41:15.092 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/ssu100.ko 2024-04-16T16:41:15.093 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/symbolserial.ko 2024-04-16T16:41:15.093 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-04-16T16:41:15.093 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-04-16T16:41:15.093 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/usb_debug.ko 2024-04-16T16:41:15.093 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/usb_wwan.ko 2024-04-16T16:41:15.093 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/visor.ko 2024-04-16T16:41:15.093 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/whiteheat.ko 2024-04-16T16:41:15.093 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/serial/xsens_mt.ko 2024-04-16T16:41:15.093 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage 2024-04-16T16:41:15.093 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/uas.ko 2024-04-16T16:41:15.093 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/ums-alauda.ko 2024-04-16T16:41:15.093 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/ums-cypress.ko 2024-04-16T16:41:15.093 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/ums-datafab.ko 2024-04-16T16:41:15.093 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-04-16T16:41:15.093 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/ums-freecom.ko 2024-04-16T16:41:15.093 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/ums-isd200.ko 2024-04-16T16:41:15.094 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-04-16T16:41:15.094 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/ums-karma.ko 2024-04-16T16:41:15.094 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/ums-onetouch.ko 2024-04-16T16:41:15.094 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/ums-realtek.ko 2024-04-16T16:41:15.094 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/ums-sddr09.ko 2024-04-16T16:41:15.094 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/ums-sddr55.ko 2024-04-16T16:41:15.094 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/ums-usbat.ko 2024-04-16T16:41:15.094 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/storage/usb-storage.ko 2024-04-16T16:41:15.094 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/usbip 2024-04-16T16:41:15.094 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/usbip/usbip-core.ko 2024-04-16T16:41:15.094 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/usbip/usbip-host.ko 2024-04-16T16:41:15.094 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-04-16T16:41:15.094 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/vfio 2024-04-16T16:41:15.094 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/vfio/pci 2024-04-16T16:41:15.094 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-04-16T16:41:15.095 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/vfio/pci/vfio-pci.ko 2024-04-16T16:41:15.095 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/vfio/vfio.ko 2024-04-16T16:41:15.095 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-04-16T16:41:15.095 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/vhost 2024-04-16T16:41:15.095 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/vhost/vhost.ko 2024-04-16T16:41:15.095 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/vhost/vhost_iotlb.ko 2024-04-16T16:41:15.095 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/vhost/vhost_net.ko 2024-04-16T16:41:15.095 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/vhost/vhost_scsi.ko 2024-04-16T16:41:15.095 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/video 2024-04-16T16:41:15.095 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/video/backlight 2024-04-16T16:41:15.095 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/video/backlight/apple_bl.ko 2024-04-16T16:41:15.095 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/video/backlight/lcd.ko 2024-04-16T16:41:15.095 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/video/backlight/platform_lcd.ko 2024-04-16T16:41:15.095 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/virtio 2024-04-16T16:41:15.095 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/virtio/virtio.ko 2024-04-16T16:41:15.095 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/virtio/virtio_balloon.ko 2024-04-16T16:41:15.095 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/virtio/virtio_mmio.ko 2024-04-16T16:41:15.096 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/virtio/virtio_pci.ko 2024-04-16T16:41:15.096 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-04-16T16:41:15.096 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-04-16T16:41:15.096 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/virtio/virtio_ring.ko 2024-04-16T16:41:15.096 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1 2024-04-16T16:41:15.096 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/masters 2024-04-16T16:41:15.096 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/masters/ds2482.ko 2024-04-16T16:41:15.096 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/masters/ds2490.ko 2024-04-16T16:41:15.096 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/slaves 2024-04-16T16:41:15.096 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-04-16T16:41:15.096 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-04-16T16:41:15.096 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-04-16T16:41:15.096 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-04-16T16:41:15.096 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-04-16T16:41:15.096 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-04-16T16:41:15.097 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-04-16T16:41:15.097 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-04-16T16:41:15.097 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-04-16T16:41:15.097 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/slaves/w1_smem.ko 2024-04-16T16:41:15.097 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/slaves/w1_therm.ko 2024-04-16T16:41:15.097 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/w1/wire.ko 2024-04-16T16:41:15.097 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog 2024-04-16T16:41:15.097 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/alim1535_wdt.ko 2024-04-16T16:41:15.097 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/alim7101_wdt.ko 2024-04-16T16:41:15.097 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/f71808e_wdt.ko 2024-04-16T16:41:15.097 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/hpwdt.ko 2024-04-16T16:41:15.097 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/i6300esb.ko 2024-04-16T16:41:15.097 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-04-16T16:41:15.097 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/iTCO_wdt.ko 2024-04-16T16:41:15.097 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/ib700wdt.ko 2024-04-16T16:41:15.097 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/ibmasr.ko 2024-04-16T16:41:15.098 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-04-16T16:41:15.098 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/it8712f_wdt.ko 2024-04-16T16:41:15.098 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/it87_wdt.ko 2024-04-16T16:41:15.098 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/machzwd.ko 2024-04-16T16:41:15.098 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/nv_tco.ko 2024-04-16T16:41:15.098 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/pcwd_pci.ko 2024-04-16T16:41:15.098 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/pcwd_usb.ko 2024-04-16T16:41:15.098 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-04-16T16:41:15.098 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/sch311x_wdt.ko 2024-04-16T16:41:15.098 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/softdog.ko 2024-04-16T16:41:15.098 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/sp5100_tco.ko 2024-04-16T16:41:15.098 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/via_wdt.ko 2024-04-16T16:41:15.098 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-04-16T16:41:15.098 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/w83877f_wdt.ko 2024-04-16T16:41:15.098 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/w83977f_wdt.ko 2024-04-16T16:41:15.098 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/wdt_pci.ko 2024-04-16T16:41:15.099 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/watchdog/xen_wdt.ko 2024-04-16T16:41:15.099 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/xen 2024-04-16T16:41:15.099 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/xen/xen-acpi-processor.ko 2024-04-16T16:41:15.099 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/xen/xen-evtchn.ko 2024-04-16T16:41:15.099 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/xen/xen-gntalloc.ko 2024-04-16T16:41:15.099 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/xen/xen-gntdev.ko 2024-04-16T16:41:15.099 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/xen/xen-pciback 2024-04-16T16:41:15.099 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-04-16T16:41:15.099 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/xen/xen-privcmd.ko 2024-04-16T16:41:15.099 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/xen/xenfs 2024-04-16T16:41:15.099 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/drivers/xen/xenfs/xenfs.ko 2024-04-16T16:41:15.099 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs 2024-04-16T16:41:15.099 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/9p 2024-04-16T16:41:15.099 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/9p/9p.ko 2024-04-16T16:41:15.099 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/affs 2024-04-16T16:41:15.099 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/affs/affs.ko 2024-04-16T16:41:15.099 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/befs 2024-04-16T16:41:15.100 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/befs/befs.ko 2024-04-16T16:41:15.100 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/binfmt_misc.ko 2024-04-16T16:41:15.100 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/btrfs 2024-04-16T16:41:15.100 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/btrfs/btrfs.ko 2024-04-16T16:41:15.100 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/cachefiles 2024-04-16T16:41:15.100 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/cachefiles/cachefiles.ko 2024-04-16T16:41:15.100 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ceph 2024-04-16T16:41:15.100 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ceph/ceph.ko 2024-04-16T16:41:15.100 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/coda 2024-04-16T16:41:15.100 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/coda/coda.ko 2024-04-16T16:41:15.100 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/cramfs 2024-04-16T16:41:15.100 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/cramfs/cramfs.ko 2024-04-16T16:41:15.100 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/dlm 2024-04-16T16:41:15.100 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/dlm/dlm.ko 2024-04-16T16:41:15.100 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ecryptfs 2024-04-16T16:41:15.101 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ecryptfs/ecryptfs.ko 2024-04-16T16:41:15.101 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/f2fs 2024-04-16T16:41:15.101 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/f2fs/f2fs.ko 2024-04-16T16:41:15.101 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/fat 2024-04-16T16:41:15.101 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/fat/fat.ko 2024-04-16T16:41:15.101 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/fat/msdos.ko 2024-04-16T16:41:15.101 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/fat/vfat.ko 2024-04-16T16:41:15.101 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/fuse 2024-04-16T16:41:15.101 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/fuse/cuse.ko 2024-04-16T16:41:15.101 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/fuse/fuse.ko 2024-04-16T16:41:15.101 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/gfs2 2024-04-16T16:41:15.101 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/gfs2/gfs2.ko 2024-04-16T16:41:15.101 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/hfs 2024-04-16T16:41:15.101 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/hfs/hfs.ko 2024-04-16T16:41:15.102 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/hfsplus 2024-04-16T16:41:15.102 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/hfsplus/hfsplus.ko 2024-04-16T16:41:15.102 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/isofs 2024-04-16T16:41:15.102 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/isofs/isofs.ko 2024-04-16T16:41:15.102 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/jfs 2024-04-16T16:41:15.102 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/jfs/jfs.ko 2024-04-16T16:41:15.102 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/lockd 2024-04-16T16:41:15.102 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/lockd/lockd.ko 2024-04-16T16:41:15.102 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/minix 2024-04-16T16:41:15.102 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/minix/minix.ko 2024-04-16T16:41:15.102 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/netfs 2024-04-16T16:41:15.102 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/netfs/netfs.ko 2024-04-16T16:41:15.102 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfs 2024-04-16T16:41:15.102 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfs/blocklayout 2024-04-16T16:41:15.103 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-04-16T16:41:15.103 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfs/filelayout 2024-04-16T16:41:15.103 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-04-16T16:41:15.103 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfs/flexfilelayout 2024-04-16T16:41:15.103 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-04-16T16:41:15.103 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfs/nfs.ko 2024-04-16T16:41:15.103 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfs/nfsv3.ko 2024-04-16T16:41:15.103 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfs/nfsv4.ko 2024-04-16T16:41:15.103 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfs_common 2024-04-16T16:41:15.103 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfs_common/grace.ko 2024-04-16T16:41:15.103 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfs_common/nfs_acl.ko 2024-04-16T16:41:15.103 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfsd 2024-04-16T16:41:15.103 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nfsd/nfsd.ko 2024-04-16T16:41:15.103 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nilfs2 2024-04-16T16:41:15.103 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nilfs2/nilfs2.ko 2024-04-16T16:41:15.104 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls 2024-04-16T16:41:15.104 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/mac-celtic.ko 2024-04-16T16:41:15.104 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/mac-centeuro.ko 2024-04-16T16:41:15.104 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/mac-croatian.ko 2024-04-16T16:41:15.104 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/mac-cyrillic.ko 2024-04-16T16:41:15.104 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/mac-gaelic.ko 2024-04-16T16:41:15.104 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/mac-greek.ko 2024-04-16T16:41:15.104 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/mac-iceland.ko 2024-04-16T16:41:15.104 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/mac-inuit.ko 2024-04-16T16:41:15.104 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/mac-roman.ko 2024-04-16T16:41:15.104 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/mac-romanian.ko 2024-04-16T16:41:15.104 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/mac-turkish.ko 2024-04-16T16:41:15.104 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp1250.ko 2024-04-16T16:41:15.104 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp1251.ko 2024-04-16T16:41:15.104 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp1255.ko 2024-04-16T16:41:15.104 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp737.ko 2024-04-16T16:41:15.105 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp775.ko 2024-04-16T16:41:15.105 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp850.ko 2024-04-16T16:41:15.105 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp852.ko 2024-04-16T16:41:15.105 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp855.ko 2024-04-16T16:41:15.105 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp857.ko 2024-04-16T16:41:15.105 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp860.ko 2024-04-16T16:41:15.105 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp861.ko 2024-04-16T16:41:15.105 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp862.ko 2024-04-16T16:41:15.105 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp863.ko 2024-04-16T16:41:15.105 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp864.ko 2024-04-16T16:41:15.105 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp865.ko 2024-04-16T16:41:15.105 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp866.ko 2024-04-16T16:41:15.105 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp869.ko 2024-04-16T16:41:15.105 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp874.ko 2024-04-16T16:41:15.105 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp932.ko 2024-04-16T16:41:15.105 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp936.ko 2024-04-16T16:41:15.105 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp949.ko 2024-04-16T16:41:15.106 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_cp950.ko 2024-04-16T16:41:15.106 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_euc-jp.ko 2024-04-16T16:41:15.106 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_iso8859-1.ko 2024-04-16T16:41:15.106 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_iso8859-13.ko 2024-04-16T16:41:15.106 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_iso8859-14.ko 2024-04-16T16:41:15.106 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_iso8859-15.ko 2024-04-16T16:41:15.106 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_iso8859-2.ko 2024-04-16T16:41:15.106 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_iso8859-3.ko 2024-04-16T16:41:15.106 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_iso8859-4.ko 2024-04-16T16:41:15.106 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_iso8859-5.ko 2024-04-16T16:41:15.106 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_iso8859-6.ko 2024-04-16T16:41:15.106 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_iso8859-7.ko 2024-04-16T16:41:15.106 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_iso8859-9.ko 2024-04-16T16:41:15.106 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_koi8-r.ko 2024-04-16T16:41:15.106 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_koi8-ru.ko 2024-04-16T16:41:15.106 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_koi8-u.ko 2024-04-16T16:41:15.107 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_ucs2_utils.ko 2024-04-16T16:41:15.107 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/nls/nls_utf8.ko 2024-04-16T16:41:15.107 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ocfs2 2024-04-16T16:41:15.107 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ocfs2/cluster 2024-04-16T16:41:15.107 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-04-16T16:41:15.107 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ocfs2/dlm 2024-04-16T16:41:15.107 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-04-16T16:41:15.107 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ocfs2/dlmfs 2024-04-16T16:41:15.107 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-04-16T16:41:15.107 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ocfs2/ocfs2.ko 2024-04-16T16:41:15.107 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-04-16T16:41:15.107 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-04-16T16:41:15.107 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-04-16T16:41:15.107 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/overlayfs 2024-04-16T16:41:15.107 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/overlayfs/overlay.ko 2024-04-16T16:41:15.107 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/pstore 2024-04-16T16:41:15.108 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/pstore/ramoops.ko 2024-04-16T16:41:15.108 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/reiserfs 2024-04-16T16:41:15.108 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/reiserfs/reiserfs.ko 2024-04-16T16:41:15.108 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/romfs 2024-04-16T16:41:15.108 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/romfs/romfs.ko 2024-04-16T16:41:15.108 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/smb 2024-04-16T16:41:15.108 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/smb/client 2024-04-16T16:41:15.108 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/smb/client/cifs.ko 2024-04-16T16:41:15.108 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/smb/common 2024-04-16T16:41:15.108 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/smb/common/cifs_arc4.ko 2024-04-16T16:41:15.108 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/smb/common/cifs_md4.ko 2024-04-16T16:41:15.108 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/squashfs 2024-04-16T16:41:15.108 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/squashfs/squashfs.ko 2024-04-16T16:41:15.108 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/sysv 2024-04-16T16:41:15.108 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/sysv/sysv.ko 2024-04-16T16:41:15.108 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/udf 2024-04-16T16:41:15.108 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/udf/udf.ko 2024-04-16T16:41:15.109 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ufs 2024-04-16T16:41:15.109 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/ufs/ufs.ko 2024-04-16T16:41:15.109 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/xfs 2024-04-16T16:41:15.109 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/fs/xfs/xfs.ko 2024-04-16T16:41:15.109 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/kernel 2024-04-16T16:41:15.109 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/kernel/rcu 2024-04-16T16:41:15.109 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/kernel/rcu/rcutorture.ko 2024-04-16T16:41:15.109 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/kernel/torture.ko 2024-04-16T16:41:15.109 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/kernel/trace 2024-04-16T16:41:15.109 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-04-16T16:41:15.109 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib 2024-04-16T16:41:15.109 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/asn1_encoder.ko 2024-04-16T16:41:15.109 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/crc-itu-t.ko 2024-04-16T16:41:15.109 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/crc8.ko 2024-04-16T16:41:15.109 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/crypto 2024-04-16T16:41:15.110 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/crypto/libarc4.ko 2024-04-16T16:41:15.110 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/crypto/libdes.ko 2024-04-16T16:41:15.110 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/libcrc32c.ko 2024-04-16T16:41:15.110 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/lru_cache.ko 2024-04-16T16:41:15.110 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/lz4 2024-04-16T16:41:15.110 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/lz4/lz4_compress.ko 2024-04-16T16:41:15.110 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/lz4/lz4hc_compress.ko 2024-04-16T16:41:15.110 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/math 2024-04-16T16:41:15.110 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/math/cordic.ko 2024-04-16T16:41:15.110 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/raid6 2024-04-16T16:41:15.110 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/raid6/raid6_pq.ko 2024-04-16T16:41:15.110 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/reed_solomon 2024-04-16T16:41:15.110 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/reed_solomon/reed_solomon.ko 2024-04-16T16:41:15.110 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/ts_bm.ko 2024-04-16T16:41:15.110 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/ts_fsm.ko 2024-04-16T16:41:15.110 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/ts_kmp.ko 2024-04-16T16:41:15.111 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/zstd 2024-04-16T16:41:15.111 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/lib/zstd/zstd_compress.ko 2024-04-16T16:41:15.111 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/mm 2024-04-16T16:41:15.111 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/mm/hwpoison-inject.ko 2024-04-16T16:41:15.111 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net 2024-04-16T16:41:15.111 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/6lowpan 2024-04-16T16:41:15.111 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/6lowpan/6lowpan.ko 2024-04-16T16:41:15.111 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/6lowpan/nhc_dest.ko 2024-04-16T16:41:15.111 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/6lowpan/nhc_fragment.ko 2024-04-16T16:41:15.111 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/6lowpan/nhc_hop.ko 2024-04-16T16:41:15.111 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/6lowpan/nhc_ipv6.ko 2024-04-16T16:41:15.111 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/6lowpan/nhc_mobility.ko 2024-04-16T16:41:15.111 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/6lowpan/nhc_routing.ko 2024-04-16T16:41:15.111 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/6lowpan/nhc_udp.ko 2024-04-16T16:41:15.111 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/802 2024-04-16T16:41:15.111 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/802/garp.ko 2024-04-16T16:41:15.112 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/802/mrp.ko 2024-04-16T16:41:15.112 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/802/p8022.ko 2024-04-16T16:41:15.112 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/802/psnap.ko 2024-04-16T16:41:15.112 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/802/stp.ko 2024-04-16T16:41:15.112 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/8021q 2024-04-16T16:41:15.112 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/8021q/8021q.ko 2024-04-16T16:41:15.112 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/9p 2024-04-16T16:41:15.112 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/9p/9pnet.ko 2024-04-16T16:41:15.112 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/9p/9pnet_fd.ko 2024-04-16T16:41:15.112 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/9p/9pnet_rdma.ko 2024-04-16T16:41:15.112 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/9p/9pnet_virtio.ko 2024-04-16T16:41:15.112 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/batman-adv 2024-04-16T16:41:15.112 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/batman-adv/batman-adv.ko 2024-04-16T16:41:15.112 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge 2024-04-16T16:41:15.112 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/br_netfilter.ko 2024-04-16T16:41:15.112 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/bridge.ko 2024-04-16T16:41:15.112 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter 2024-04-16T16:41:15.113 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-04-16T16:41:15.113 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_among.ko 2024-04-16T16:41:15.113 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_arp.ko 2024-04-16T16:41:15.113 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-04-16T16:41:15.113 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-04-16T16:41:15.113 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_ip.ko 2024-04-16T16:41:15.113 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-04-16T16:41:15.113 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_limit.ko 2024-04-16T16:41:15.113 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_log.ko 2024-04-16T16:41:15.113 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_mark.ko 2024-04-16T16:41:15.113 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-04-16T16:41:15.113 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-04-16T16:41:15.113 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-04-16T16:41:15.113 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-04-16T16:41:15.113 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_snat.ko 2024-04-16T16:41:15.113 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_stp.ko 2024-04-16T16:41:15.114 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-04-16T16:41:15.114 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-04-16T16:41:15.114 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-04-16T16:41:15.114 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-04-16T16:41:15.114 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/ebtables.ko 2024-04-16T16:41:15.114 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-04-16T16:41:15.114 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-04-16T16:41:15.114 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-04-16T16:41:15.114 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ceph 2024-04-16T16:41:15.114 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ceph/libceph.ko 2024-04-16T16:41:15.114 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/core 2024-04-16T16:41:15.114 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/core/failover.ko 2024-04-16T16:41:15.114 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/core/pktgen.ko 2024-04-16T16:41:15.114 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/dccp 2024-04-16T16:41:15.115 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/dccp/dccp.ko 2024-04-16T16:41:15.115 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/dccp/dccp_diag.ko 2024-04-16T16:41:15.115 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/dccp/dccp_ipv4.ko 2024-04-16T16:41:15.115 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/dccp/dccp_ipv6.ko 2024-04-16T16:41:15.115 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/dns_resolver 2024-04-16T16:41:15.115 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/dns_resolver/dns_resolver.ko 2024-04-16T16:41:15.115 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/dsa 2024-04-16T16:41:15.115 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/dsa/dsa_core.ko 2024-04-16T16:41:15.115 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/dsa/tag_brcm.ko 2024-04-16T16:41:15.115 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/dsa/tag_dsa.ko 2024-04-16T16:41:15.115 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/dsa/tag_none.ko 2024-04-16T16:41:15.115 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/dsa/tag_trailer.ko 2024-04-16T16:41:15.115 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ieee802154 2024-04-16T16:41:15.115 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ieee802154/6lowpan 2024-04-16T16:41:15.115 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-04-16T16:41:15.115 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ieee802154/ieee802154.ko 2024-04-16T16:41:15.115 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ieee802154/ieee802154_socket.ko 2024-04-16T16:41:15.116 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4 2024-04-16T16:41:15.116 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/ah4.ko 2024-04-16T16:41:15.116 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/esp4.ko 2024-04-16T16:41:15.116 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/fou.ko 2024-04-16T16:41:15.116 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/gre.ko 2024-04-16T16:41:15.116 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/inet_diag.ko 2024-04-16T16:41:15.116 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/ip_gre.ko 2024-04-16T16:41:15.116 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/ip_tunnel.ko 2024-04-16T16:41:15.116 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/ip_vti.ko 2024-04-16T16:41:15.116 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/ipcomp.ko 2024-04-16T16:41:15.116 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/ipip.ko 2024-04-16T16:41:15.116 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter 2024-04-16T16:41:15.116 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/arp_tables.ko 2024-04-16T16:41:15.116 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-04-16T16:41:15.116 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-04-16T16:41:15.116 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-04-16T16:41:15.116 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-04-16T16:41:15.117 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-04-16T16:41:15.117 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-04-16T16:41:15.117 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-04-16T16:41:15.117 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-04-16T16:41:15.117 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-04-16T16:41:15.117 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/iptable_security.ko 2024-04-16T16:41:15.117 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-04-16T16:41:15.117 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-04-16T16:41:15.117 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-04-16T16:41:15.117 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-04-16T16:41:15.117 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-04-16T16:41:15.117 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-04-16T16:41:15.117 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-04-16T16:41:15.117 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-04-16T16:41:15.117 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-04-16T16:41:15.117 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-04-16T16:41:15.117 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tcp_bic.ko 2024-04-16T16:41:15.118 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tcp_dctcp.ko 2024-04-16T16:41:15.118 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tcp_diag.ko 2024-04-16T16:41:15.118 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tcp_highspeed.ko 2024-04-16T16:41:15.118 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tcp_htcp.ko 2024-04-16T16:41:15.118 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tcp_hybla.ko 2024-04-16T16:41:15.118 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tcp_illinois.ko 2024-04-16T16:41:15.118 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tcp_lp.ko 2024-04-16T16:41:15.118 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tcp_scalable.ko 2024-04-16T16:41:15.118 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tcp_vegas.ko 2024-04-16T16:41:15.118 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tcp_veno.ko 2024-04-16T16:41:15.118 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tcp_westwood.ko 2024-04-16T16:41:15.118 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tcp_yeah.ko 2024-04-16T16:41:15.118 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/tunnel4.ko 2024-04-16T16:41:15.118 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/udp_diag.ko 2024-04-16T16:41:15.118 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/udp_tunnel.ko 2024-04-16T16:41:15.118 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv4/xfrm4_tunnel.ko 2024-04-16T16:41:15.119 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6 2024-04-16T16:41:15.119 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/ah6.ko 2024-04-16T16:41:15.119 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/esp6.ko 2024-04-16T16:41:15.119 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/fou6.ko 2024-04-16T16:41:15.119 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/ip6_tunnel.ko 2024-04-16T16:41:15.119 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-04-16T16:41:15.119 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/ip6_vti.ko 2024-04-16T16:41:15.119 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/ipcomp6.ko 2024-04-16T16:41:15.119 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter 2024-04-16T16:41:15.119 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-04-16T16:41:15.119 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-04-16T16:41:15.119 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-04-16T16:41:15.119 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-04-16T16:41:15.119 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-04-16T16:41:15.119 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-04-16T16:41:15.119 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-04-16T16:41:15.119 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-04-16T16:41:15.120 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-04-16T16:41:15.120 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-04-16T16:41:15.120 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-04-16T16:41:15.120 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-04-16T16:41:15.120 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-04-16T16:41:15.120 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-04-16T16:41:15.120 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-04-16T16:41:15.120 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-04-16T16:41:15.120 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-04-16T16:41:15.120 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-04-16T16:41:15.120 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-04-16T16:41:15.120 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-04-16T16:41:15.120 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-04-16T16:41:15.120 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-04-16T16:41:15.120 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-04-16T16:41:15.121 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-04-16T16:41:15.121 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/sit.ko 2024-04-16T16:41:15.121 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/tunnel6.ko 2024-04-16T16:41:15.121 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/ipv6/xfrm6_tunnel.ko 2024-04-16T16:41:15.121 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/key 2024-04-16T16:41:15.121 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/key/af_key.ko 2024-04-16T16:41:15.121 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/l2tp 2024-04-16T16:41:15.121 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/l2tp/l2tp_core.ko 2024-04-16T16:41:15.121 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/l2tp/l2tp_debugfs.ko 2024-04-16T16:41:15.121 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/l2tp/l2tp_eth.ko 2024-04-16T16:41:15.121 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/l2tp/l2tp_ip.ko 2024-04-16T16:41:15.121 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/l2tp/l2tp_ip6.ko 2024-04-16T16:41:15.121 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/l2tp/l2tp_netlink.ko 2024-04-16T16:41:15.121 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/l2tp/l2tp_ppp.ko 2024-04-16T16:41:15.121 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/llc 2024-04-16T16:41:15.121 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/llc/llc.ko 2024-04-16T16:41:15.122 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/mac802154 2024-04-16T16:41:15.122 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/mac802154/mac802154.ko 2024-04-16T16:41:15.122 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/mpls 2024-04-16T16:41:15.122 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/mpls/mpls_gso.ko 2024-04-16T16:41:15.122 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter 2024-04-16T16:41:15.122 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset 2024-04-16T16:41:15.122 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set.ko 2024-04-16T16:41:15.122 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-04-16T16:41:15.122 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-04-16T16:41:15.122 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-04-16T16:41:15.122 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-04-16T16:41:15.122 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-04-16T16:41:15.122 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-04-16T16:41:15.122 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-04-16T16:41:15.122 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-04-16T16:41:15.122 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-04-16T16:41:15.123 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-04-16T16:41:15.123 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-04-16T16:41:15.123 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-04-16T16:41:15.123 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-04-16T16:41:15.123 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-04-16T16:41:15.123 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-04-16T16:41:15.123 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs 2024-04-16T16:41:15.123 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs.ko 2024-04-16T16:41:15.123 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-04-16T16:41:15.123 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-04-16T16:41:15.123 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-04-16T16:41:15.123 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-04-16T16:41:15.123 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-04-16T16:41:15.123 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-04-16T16:41:15.123 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-04-16T16:41:15.123 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-04-16T16:41:15.123 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-04-16T16:41:15.124 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-04-16T16:41:15.124 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-04-16T16:41:15.124 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-04-16T16:41:15.124 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-04-16T16:41:15.124 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conncount.ko 2024-04-16T16:41:15.124 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conntrack.ko 2024-04-16T16:41:15.124 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-04-16T16:41:15.124 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-04-16T16:41:15.124 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-04-16T16:41:15.124 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conntrack_h323.ko 2024-04-16T16:41:15.124 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conntrack_irc.ko 2024-04-16T16:41:15.124 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-04-16T16:41:15.124 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-04-16T16:41:15.124 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-04-16T16:41:15.124 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conntrack_sane.ko 2024-04-16T16:41:15.124 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conntrack_sip.ko 2024-04-16T16:41:15.125 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-04-16T16:41:15.125 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-04-16T16:41:15.125 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_dup_netdev.ko 2024-04-16T16:41:15.125 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_flow_table.ko 2024-04-16T16:41:15.125 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_flow_table_inet.ko 2024-04-16T16:41:15.125 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_log_syslog.ko 2024-04-16T16:41:15.125 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_nat.ko 2024-04-16T16:41:15.125 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_nat_amanda.ko 2024-04-16T16:41:15.125 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_nat_ftp.ko 2024-04-16T16:41:15.125 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_nat_irc.ko 2024-04-16T16:41:15.125 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_nat_sip.ko 2024-04-16T16:41:15.125 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_nat_tftp.ko 2024-04-16T16:41:15.125 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_synproxy_core.ko 2024-04-16T16:41:15.125 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nf_tables.ko 2024-04-16T16:41:15.125 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nfnetlink.ko 2024-04-16T16:41:15.125 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nfnetlink_acct.ko 2024-04-16T16:41:15.126 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nfnetlink_log.ko 2024-04-16T16:41:15.126 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nfnetlink_osf.ko 2024-04-16T16:41:15.126 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nfnetlink_queue.ko 2024-04-16T16:41:15.126 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_chain_nat.ko 2024-04-16T16:41:15.126 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_compat.ko 2024-04-16T16:41:15.126 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_ct.ko 2024-04-16T16:41:15.126 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_dup_netdev.ko 2024-04-16T16:41:15.126 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_fib.ko 2024-04-16T16:41:15.126 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_fwd_netdev.ko 2024-04-16T16:41:15.126 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_hash.ko 2024-04-16T16:41:15.126 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_limit.ko 2024-04-16T16:41:15.126 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_log.ko 2024-04-16T16:41:15.126 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_masq.ko 2024-04-16T16:41:15.126 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_nat.ko 2024-04-16T16:41:15.127 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_numgen.ko 2024-04-16T16:41:15.127 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_queue.ko 2024-04-16T16:41:15.127 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_quota.ko 2024-04-16T16:41:15.127 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_redir.ko 2024-04-16T16:41:15.127 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_reject.ko 2024-04-16T16:41:15.127 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_reject_inet.ko 2024-04-16T16:41:15.127 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_socket.ko 2024-04-16T16:41:15.127 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/nft_tunnel.ko 2024-04-16T16:41:15.127 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_AUDIT.ko 2024-04-16T16:41:15.127 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_CHECKSUM.ko 2024-04-16T16:41:15.127 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_CLASSIFY.ko 2024-04-16T16:41:15.127 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-04-16T16:41:15.127 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_CT.ko 2024-04-16T16:41:15.127 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_DSCP.ko 2024-04-16T16:41:15.127 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_HL.ko 2024-04-16T16:41:15.127 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_HMARK.ko 2024-04-16T16:41:15.127 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_IDLETIMER.ko 2024-04-16T16:41:15.128 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_LED.ko 2024-04-16T16:41:15.128 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_LOG.ko 2024-04-16T16:41:15.128 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_MASQUERADE.ko 2024-04-16T16:41:15.128 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_NETMAP.ko 2024-04-16T16:41:15.128 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_NFLOG.ko 2024-04-16T16:41:15.128 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_NFQUEUE.ko 2024-04-16T16:41:15.128 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_RATEEST.ko 2024-04-16T16:41:15.128 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_REDIRECT.ko 2024-04-16T16:41:15.128 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_SECMARK.ko 2024-04-16T16:41:15.128 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_TCPMSS.ko 2024-04-16T16:41:15.128 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-04-16T16:41:15.128 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_TEE.ko 2024-04-16T16:41:15.128 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_TPROXY.ko 2024-04-16T16:41:15.128 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_TRACE.ko 2024-04-16T16:41:15.128 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_addrtype.ko 2024-04-16T16:41:15.128 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_bpf.ko 2024-04-16T16:41:15.128 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_cgroup.ko 2024-04-16T16:41:15.129 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_cluster.ko 2024-04-16T16:41:15.129 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_comment.ko 2024-04-16T16:41:15.129 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_connbytes.ko 2024-04-16T16:41:15.129 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_connlabel.ko 2024-04-16T16:41:15.129 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_connlimit.ko 2024-04-16T16:41:15.129 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_connmark.ko 2024-04-16T16:41:15.129 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_conntrack.ko 2024-04-16T16:41:15.129 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_cpu.ko 2024-04-16T16:41:15.129 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_dccp.ko 2024-04-16T16:41:15.129 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_devgroup.ko 2024-04-16T16:41:15.129 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_dscp.ko 2024-04-16T16:41:15.129 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_ecn.ko 2024-04-16T16:41:15.129 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_esp.ko 2024-04-16T16:41:15.129 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_hashlimit.ko 2024-04-16T16:41:15.129 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_helper.ko 2024-04-16T16:41:15.129 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_hl.ko 2024-04-16T16:41:15.130 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_ipcomp.ko 2024-04-16T16:41:15.130 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_iprange.ko 2024-04-16T16:41:15.130 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_ipvs.ko 2024-04-16T16:41:15.130 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_l2tp.ko 2024-04-16T16:41:15.130 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_length.ko 2024-04-16T16:41:15.130 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_limit.ko 2024-04-16T16:41:15.130 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_mac.ko 2024-04-16T16:41:15.130 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_mark.ko 2024-04-16T16:41:15.130 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_multiport.ko 2024-04-16T16:41:15.130 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_nat.ko 2024-04-16T16:41:15.130 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_nfacct.ko 2024-04-16T16:41:15.130 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_osf.ko 2024-04-16T16:41:15.130 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_owner.ko 2024-04-16T16:41:15.130 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_physdev.ko 2024-04-16T16:41:15.130 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_pkttype.ko 2024-04-16T16:41:15.130 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_policy.ko 2024-04-16T16:41:15.131 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_quota.ko 2024-04-16T16:41:15.131 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_rateest.ko 2024-04-16T16:41:15.131 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_realm.ko 2024-04-16T16:41:15.131 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_recent.ko 2024-04-16T16:41:15.131 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_sctp.ko 2024-04-16T16:41:15.131 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_set.ko 2024-04-16T16:41:15.131 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_socket.ko 2024-04-16T16:41:15.131 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_state.ko 2024-04-16T16:41:15.131 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_statistic.ko 2024-04-16T16:41:15.131 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_string.ko 2024-04-16T16:41:15.131 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_tcpmss.ko 2024-04-16T16:41:15.131 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_time.ko 2024-04-16T16:41:15.131 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netfilter/xt_u32.ko 2024-04-16T16:41:15.131 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netlink 2024-04-16T16:41:15.131 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/netlink/netlink_diag.ko 2024-04-16T16:41:15.131 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/nsh 2024-04-16T16:41:15.132 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/nsh/nsh.ko 2024-04-16T16:41:15.132 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/openvswitch 2024-04-16T16:41:15.132 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/openvswitch/openvswitch.ko 2024-04-16T16:41:15.132 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/openvswitch/vport-geneve.ko 2024-04-16T16:41:15.132 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/openvswitch/vport-gre.ko 2024-04-16T16:41:15.132 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/openvswitch/vport-vxlan.ko 2024-04-16T16:41:15.132 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/packet 2024-04-16T16:41:15.132 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/packet/af_packet_diag.ko 2024-04-16T16:41:15.132 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/rds 2024-04-16T16:41:15.132 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/rds/rds.ko 2024-04-16T16:41:15.132 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/rds/rds_rdma.ko 2024-04-16T16:41:15.132 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/rds/rds_tcp.ko 2024-04-16T16:41:15.132 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched 2024-04-16T16:41:15.132 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/act_csum.ko 2024-04-16T16:41:15.132 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/act_gact.ko 2024-04-16T16:41:15.132 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/act_mirred.ko 2024-04-16T16:41:15.133 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/act_nat.ko 2024-04-16T16:41:15.133 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/act_pedit.ko 2024-04-16T16:41:15.133 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/act_police.ko 2024-04-16T16:41:15.133 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/act_simple.ko 2024-04-16T16:41:15.133 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/act_skbedit.ko 2024-04-16T16:41:15.133 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/act_vlan.ko 2024-04-16T16:41:15.133 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/cls_basic.ko 2024-04-16T16:41:15.133 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/cls_bpf.ko 2024-04-16T16:41:15.133 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/cls_flow.ko 2024-04-16T16:41:15.133 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/cls_fw.ko 2024-04-16T16:41:15.133 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/cls_route.ko 2024-04-16T16:41:15.133 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/cls_u32.ko 2024-04-16T16:41:15.133 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/em_cmp.ko 2024-04-16T16:41:15.133 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/em_ipset.ko 2024-04-16T16:41:15.133 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/em_meta.ko 2024-04-16T16:41:15.134 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/em_nbyte.ko 2024-04-16T16:41:15.134 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/em_text.ko 2024-04-16T16:41:15.134 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/em_u32.ko 2024-04-16T16:41:15.134 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_choke.ko 2024-04-16T16:41:15.134 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_codel.ko 2024-04-16T16:41:15.134 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_drr.ko 2024-04-16T16:41:15.134 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_fq.ko 2024-04-16T16:41:15.134 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_gred.ko 2024-04-16T16:41:15.134 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_hfsc.ko 2024-04-16T16:41:15.134 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_hhf.ko 2024-04-16T16:41:15.134 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_htb.ko 2024-04-16T16:41:15.134 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_ingress.ko 2024-04-16T16:41:15.134 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_mqprio.ko 2024-04-16T16:41:15.134 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_mqprio_lib.ko 2024-04-16T16:41:15.134 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_multiq.ko 2024-04-16T16:41:15.134 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_netem.ko 2024-04-16T16:41:15.135 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_pie.ko 2024-04-16T16:41:15.135 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_plug.ko 2024-04-16T16:41:15.135 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_prio.ko 2024-04-16T16:41:15.135 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_qfq.ko 2024-04-16T16:41:15.135 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_red.ko 2024-04-16T16:41:15.135 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_sfb.ko 2024-04-16T16:41:15.135 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_sfq.ko 2024-04-16T16:41:15.135 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_tbf.ko 2024-04-16T16:41:15.135 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sched/sch_teql.ko 2024-04-16T16:41:15.135 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sctp 2024-04-16T16:41:15.135 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sctp/sctp.ko 2024-04-16T16:41:15.135 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sctp/sctp_diag.ko 2024-04-16T16:41:15.135 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sunrpc 2024-04-16T16:41:15.135 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sunrpc/auth_gss 2024-04-16T16:41:15.135 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-04-16T16:41:15.135 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-04-16T16:41:15.135 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sunrpc/sunrpc.ko 2024-04-16T16:41:15.136 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sunrpc/xprtrdma 2024-04-16T16:41:15.136 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-04-16T16:41:15.136 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/tipc 2024-04-16T16:41:15.136 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/tipc/tipc.ko 2024-04-16T16:41:15.136 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/tipc/tipc_diag.ko 2024-04-16T16:41:15.136 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/unix 2024-04-16T16:41:15.136 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/unix/unix_diag.ko 2024-04-16T16:41:15.136 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/vmw_vsock 2024-04-16T16:41:15.136 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-04-16T16:41:15.136 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-04-16T16:41:15.136 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/vmw_vsock/vsock.ko 2024-04-16T16:41:15.136 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/vmw_vsock/vsock_diag.ko 2024-04-16T16:41:15.136 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/vmw_vsock/vsock_loopback.ko 2024-04-16T16:41:15.136 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/xfrm 2024-04-16T16:41:15.136 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/net/xfrm/xfrm_ipcomp.ko 2024-04-16T16:41:15.136 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/security 2024-04-16T16:41:15.136 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/security/keys 2024-04-16T16:41:15.137 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/security/keys/encrypted-keys 2024-04-16T16:41:15.137 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-04-16T16:41:15.137 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/security/keys/trusted-keys 2024-04-16T16:41:15.137 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/kernel/security/keys/trusted-keys/trusted.ko 2024-04-16T16:41:15.137 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.alias 2024-04-16T16:41:15.137 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.alias.bin 2024-04-16T16:41:15.137 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.builtin 2024-04-16T16:41:15.137 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.builtin.alias.bin 2024-04-16T16:41:15.137 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.builtin.bin 2024-04-16T16:41:15.137 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.builtin.modinfo 2024-04-16T16:41:15.137 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.dep 2024-04-16T16:41:15.137 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.dep.bin 2024-04-16T16:41:15.137 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.devname 2024-04-16T16:41:15.137 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.order 2024-04-16T16:41:15.137 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.softdep 2024-04-16T16:41:15.137 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.symbols 2024-04-16T16:41:15.137 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.symbols.bin 2024-04-16T16:41:15.137 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g3f60ed14f7c1/vmlinuz 2024-04-16T16:41:15.141 DEBUG:teuthology.task.kernel:get_image_version: 6.9.0-rc3-g3f60ed14f7c1 2024-04-16T16:41:15.141 DEBUG:teuthology.orchestra.run.smithi045:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-04-16T16:41:15.167 INFO:teuthology.orchestra.run.smithi045.stdout:postinstall scriptlet (using /bin/sh): 2024-04-16T16:41:15.167 INFO:teuthology.orchestra.run.smithi045.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-04-16T16:41:15.167 INFO:teuthology.orchestra.run.smithi045.stdout: /usr/bin/kernel-install add 6.9.0-rc3-g3f60ed14f7c1 /lib/modules/6.9.0-rc3-g3f60ed14f7c1/vmlinuz 2024-04-16T16:41:15.167 INFO:teuthology.orchestra.run.smithi045.stdout:fi 2024-04-16T16:41:15.167 INFO:teuthology.orchestra.run.smithi045.stdout:for file in vmlinuz System.map config; do 2024-04-16T16:41:15.167 INFO:teuthology.orchestra.run.smithi045.stdout: if ! cmp --silent "/lib/modules/6.9.0-rc3-g3f60ed14f7c1/${file}" "/boot/${file}-6.9.0-rc3-g3f60ed14f7c1"; then 2024-04-16T16:41:15.167 INFO:teuthology.orchestra.run.smithi045.stdout: cp "/lib/modules/6.9.0-rc3-g3f60ed14f7c1/${file}" "/boot/${file}-6.9.0-rc3-g3f60ed14f7c1" 2024-04-16T16:41:15.167 INFO:teuthology.orchestra.run.smithi045.stdout: fi 2024-04-16T16:41:15.167 INFO:teuthology.orchestra.run.smithi045.stdout:done 2024-04-16T16:41:15.168 INFO:teuthology.orchestra.run.smithi045.stdout:if [ -d "/lib/modules/6.9.0-rc3-g3f60ed14f7c1/dtb" ] && \ 2024-04-16T16:41:15.168 INFO:teuthology.orchestra.run.smithi045.stdout: ! diff -rq "/lib/modules/6.9.0-rc3-g3f60ed14f7c1/dtb" "/boot/dtb-6.9.0-rc3-g3f60ed14f7c1" >/dev/null 2>&1; then 2024-04-16T16:41:15.168 INFO:teuthology.orchestra.run.smithi045.stdout: rm -rf "/boot/dtb-6.9.0-rc3-g3f60ed14f7c1" 2024-04-16T16:41:15.168 INFO:teuthology.orchestra.run.smithi045.stdout: cp -r "/lib/modules/6.9.0-rc3-g3f60ed14f7c1/dtb" "/boot/dtb-6.9.0-rc3-g3f60ed14f7c1" 2024-04-16T16:41:15.168 INFO:teuthology.orchestra.run.smithi045.stdout:fi 2024-04-16T16:41:15.168 INFO:teuthology.orchestra.run.smithi045.stdout:if [ ! -e "/lib/modules/6.9.0-rc3-g3f60ed14f7c1/modules.dep" ]; then 2024-04-16T16:41:15.168 INFO:teuthology.orchestra.run.smithi045.stdout: /usr/sbin/depmod 6.9.0-rc3-g3f60ed14f7c1 2024-04-16T16:41:15.168 INFO:teuthology.orchestra.run.smithi045.stdout:fi 2024-04-16T16:41:15.168 INFO:teuthology.orchestra.run.smithi045.stdout:preuninstall scriptlet (using /bin/sh): 2024-04-16T16:41:15.168 INFO:teuthology.orchestra.run.smithi045.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-04-16T16:41:15.168 INFO:teuthology.orchestra.run.smithi045.stdout:kernel-install remove 6.9.0-rc3-g3f60ed14f7c1 2024-04-16T16:41:15.168 INFO:teuthology.orchestra.run.smithi045.stdout:fi 2024-04-16T16:41:15.169 DEBUG:teuthology.orchestra.run.smithi045:> sudo rpm -qi grub2-tools 2024-04-16T16:41:15.261 INFO:teuthology.orchestra.run.smithi045.stdout:Name : grub2-tools 2024-04-16T16:41:15.261 INFO:teuthology.orchestra.run.smithi045.stdout:Epoch : 1 2024-04-16T16:41:15.261 INFO:teuthology.orchestra.run.smithi045.stdout:Version : 2.06 2024-04-16T16:41:15.261 INFO:teuthology.orchestra.run.smithi045.stdout:Release : 68.el9 2024-04-16T16:41:15.261 INFO:teuthology.orchestra.run.smithi045.stdout:Architecture: x86_64 2024-04-16T16:41:15.261 INFO:teuthology.orchestra.run.smithi045.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-04-16T16:41:15.261 INFO:teuthology.orchestra.run.smithi045.stdout:Group : Unspecified 2024-04-16T16:41:15.261 INFO:teuthology.orchestra.run.smithi045.stdout:Size : 8264254 2024-04-16T16:41:15.261 INFO:teuthology.orchestra.run.smithi045.stdout:License : GPLv3+ 2024-04-16T16:41:15.261 INFO:teuthology.orchestra.run.smithi045.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-04-16T16:41:15.261 INFO:teuthology.orchestra.run.smithi045.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-04-16T16:41:15.262 INFO:teuthology.orchestra.run.smithi045.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-04-16T16:41:15.262 INFO:teuthology.orchestra.run.smithi045.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-04-16T16:41:15.262 INFO:teuthology.orchestra.run.smithi045.stdout:Packager : builder@centos.org 2024-04-16T16:41:15.262 INFO:teuthology.orchestra.run.smithi045.stdout:Vendor : CentOS 2024-04-16T16:41:15.262 INFO:teuthology.orchestra.run.smithi045.stdout:URL : http://www.gnu.org/software/grub/ 2024-04-16T16:41:15.262 INFO:teuthology.orchestra.run.smithi045.stdout:Summary : Support tools for GRUB. 2024-04-16T16:41:15.262 INFO:teuthology.orchestra.run.smithi045.stdout:Description : 2024-04-16T16:41:15.262 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-16T16:41:15.262 INFO:teuthology.orchestra.run.smithi045.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-04-16T16:41:15.262 INFO:teuthology.orchestra.run.smithi045.stdout:customizable bootloader with modular architecture. It supports a rich 2024-04-16T16:41:15.262 INFO:teuthology.orchestra.run.smithi045.stdout:variety of kernel formats, file systems, computer architectures and 2024-04-16T16:41:15.262 INFO:teuthology.orchestra.run.smithi045.stdout:hardware devices. 2024-04-16T16:41:15.262 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-16T16:41:15.262 INFO:teuthology.orchestra.run.smithi045.stdout:This subpackage provides tools for support of all platforms. 2024-04-16T16:41:15.263 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-04-16T16:41:15.263 INFO:teuthology.task.kernel:Updating grub on smithi045 to boot 6.9.0-rc3-g3f60ed14f7c1 2024-04-16T16:41:15.263 DEBUG:teuthology.orchestra.run.smithi045:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-04-16T16:41:15.904 INFO:teuthology.orchestra.run.smithi045.stderr:Generating grub configuration file ... 2024-04-16T16:41:17.490 INFO:teuthology.orchestra.run.smithi045.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-04-16T16:41:17.529 INFO:teuthology.orchestra.run.smithi045.stderr:done 2024-04-16T16:41:17.531 DEBUG:teuthology.orchestra.run.smithi045:> mktemp 2024-04-16T16:41:17.546 INFO:teuthology.orchestra.run.smithi045.stdout:/tmp/tmp.kTLa6GzMxI 2024-04-16T16:41:17.547 DEBUG:teuthology.orchestra.run.smithi045:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.kTLa6GzMxI 2024-04-16T16:41:17.631 DEBUG:teuthology.orchestra.run.smithi045:> sudo chmod 0666 /tmp/tmp.kTLa6GzMxI 2024-04-16T16:41:17.735 DEBUG:teuthology.orchestra.remote:smithi045:/tmp/tmp.kTLa6GzMxI is 6KB 2024-04-16T16:41:17.785 DEBUG:teuthology.orchestra.run.smithi045:> rm -fr /tmp/tmp.kTLa6GzMxI 2024-04-16T16:41:17.800 DEBUG:teuthology.orchestra.run.smithi045:> sudo /bin/ls /boot/loader/entries || true 2024-04-16T16:41:17.875 INFO:teuthology.orchestra.run.smithi045.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-04-16T16:41:17.875 INFO:teuthology.orchestra.run.smithi045.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-04-16T16:41:17.875 INFO:teuthology.orchestra.run.smithi045.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-04-16T16:41:17.875 INFO:teuthology.orchestra.run.smithi045.stdout:15949e3c88704300882482eddfd7baae-6.9.0-rc3-g3f60ed14f7c1.conf 2024-04-16T16:41:17.876 DEBUG:teuthology.orchestra.run.smithi045:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-6.9.0-rc3-g3f60ed14f7c1 2024-04-16T16:41:18.024 DEBUG:teuthology.orchestra.run.smithi045:> sudo shutdown -r now 2024-04-16T16:41:48.028 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-04-16T16:41:48.028 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi045.front.sepia.ceph.com' 2024-04-16T16:41:48.029 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi045.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-16T16:42:06.494 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.45 2024-04-16T16:42:15.495 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi045.front.sepia.ceph.com' 2024-04-16T16:42:15.496 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi045.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-16T16:43:15.498 DEBUG:teuthology.orchestra.remote:timed out 2024-04-16T16:43:27.499 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi045.front.sepia.ceph.com' 2024-04-16T16:43:27.502 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi045.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-16T16:43:27.896 DEBUG:teuthology.orchestra.run.smithi045:> true 2024-04-16T16:43:28.553 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi045.front.sepia.ceph.com' 2024-04-16T16:43:28.554 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "6.9.0-rc3-g3f60ed14f7c1"... 2024-04-16T16:43:28.554 DEBUG:teuthology.orchestra.run.smithi045:> uname -r 2024-04-16T16:43:28.610 INFO:teuthology.orchestra.run.smithi045.stdout:6.9.0-rc3-g3f60ed14f7c1 2024-04-16T16:43:28.610 DEBUG:teuthology.task.kernel:current kernel version is 6.9.0-rc3-g3f60ed14f7c1 vs 6.9.0-rc3-g3f60ed14f7c1 2024-04-16T16:43:28.610 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-04-16T16:43:28.610 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-04-16T16:43:29.611 DEBUG:teuthology.parallel:result is None 2024-04-16T16:43:29.611 INFO:teuthology.run_tasks:Running task internal.base... 2024-04-16T16:43:29.619 INFO:teuthology.task.internal:Creating test directory... 2024-04-16T16:43:29.619 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-16T16:43:29.627 DEBUG:teuthology.orchestra.run.smithi104:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-16T16:43:29.629 DEBUG:teuthology.orchestra.run.smithi117:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-16T16:43:29.680 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-04-16T16:43:29.688 INFO:teuthology.run_tasks:Running task internal.archive... 2024-04-16T16:43:29.695 INFO:teuthology.task.internal:Creating archive directory... 2024-04-16T16:43:29.695 DEBUG:teuthology.orchestra.run.smithi045:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-16T16:43:29.700 DEBUG:teuthology.orchestra.run.smithi104:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-16T16:43:29.702 DEBUG:teuthology.orchestra.run.smithi117:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-16T16:43:29.778 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-04-16T16:43:29.785 INFO:teuthology.task.internal:Enabling coredump saving... 2024-04-16T16:43:29.785 DEBUG:teuthology.orchestra.run.smithi104:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-16T16:43:29.800 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-16T16:43:29.801 DEBUG:teuthology.orchestra.run.smithi117:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-16T16:43:29.816 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-16T16:43:29.817 DEBUG:teuthology.orchestra.run.smithi045:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-16T16:43:29.824 DEBUG:teuthology.orchestra.run.smithi104:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-16T16:43:29.844 DEBUG:teuthology.orchestra.run.smithi117:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-16T16:43:29.900 INFO:teuthology.orchestra.run.smithi104.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-16T16:43:29.914 INFO:teuthology.orchestra.run.smithi117.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-16T16:43:29.922 INFO:teuthology.orchestra.run.smithi104.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-16T16:43:29.935 INFO:teuthology.orchestra.run.smithi117.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-16T16:43:29.980 INFO:teuthology.orchestra.run.smithi045.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-16T16:43:30.063 INFO:teuthology.orchestra.run.smithi045.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-16T16:43:30.068 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-04-16T16:43:30.075 INFO:teuthology.task.internal:Configuring sudo... 2024-04-16T16:43:30.075 DEBUG:teuthology.orchestra.run.smithi045:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-16T16:43:30.082 DEBUG:teuthology.orchestra.run.smithi104:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-16T16:43:30.084 DEBUG:teuthology.orchestra.run.smithi117:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-16T16:43:30.174 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-04-16T16:43:30.184 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-04-16T16:43:30.184 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-16T16:43:30.190 DEBUG:teuthology.orchestra.run.smithi104:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-16T16:43:30.192 DEBUG:teuthology.orchestra.run.smithi117:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-16T16:43:30.241 DEBUG:teuthology.orchestra.run.smithi045:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-16T16:43:30.327 DEBUG:teuthology.orchestra.run.smithi045:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-16T16:43:30.447 DEBUG:teuthology.orchestra.run.smithi045:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-16T16:43:30.536 DEBUG:teuthology.orchestra.run.smithi045:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-16T16:43:30.634 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-04-16T16:43:30.634 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-16T16:43:30.728 DEBUG:teuthology.orchestra.run.smithi104:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-16T16:43:30.770 DEBUG:teuthology.orchestra.run.smithi104:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-16T16:43:30.836 DEBUG:teuthology.orchestra.run.smithi104:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-16T16:43:30.880 DEBUG:teuthology.orchestra.run.smithi104:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-16T16:43:30.926 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2024-04-16T16:43:30.926 DEBUG:teuthology.orchestra.run.smithi104:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-16T16:43:30.996 DEBUG:teuthology.orchestra.run.smithi117:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-16T16:43:31.039 DEBUG:teuthology.orchestra.run.smithi117:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-16T16:43:31.106 DEBUG:teuthology.orchestra.run.smithi117:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-16T16:43:31.147 DEBUG:teuthology.orchestra.run.smithi117:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-16T16:43:31.196 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-04-16T16:43:31.196 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-16T16:43:31.266 DEBUG:teuthology.orchestra.run.smithi045:> sudo service rsyslog restart 2024-04-16T16:43:31.275 DEBUG:teuthology.orchestra.run.smithi104:> sudo service rsyslog restart 2024-04-16T16:43:31.277 DEBUG:teuthology.orchestra.run.smithi117:> sudo service rsyslog restart 2024-04-16T16:43:31.317 INFO:teuthology.orchestra.run.smithi104.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-16T16:43:31.354 INFO:teuthology.orchestra.run.smithi117.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-16T16:43:31.421 INFO:teuthology.orchestra.run.smithi045.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-16T16:43:31.802 INFO:teuthology.run_tasks:Running task internal.timer... 2024-04-16T16:43:31.810 INFO:teuthology.task.internal:Starting timer... 2024-04-16T16:43:31.810 INFO:teuthology.run_tasks:Running task pcp... 2024-04-16T16:43:31.820 INFO:teuthology.run_tasks:Running task selinux... 2024-04-16T16:43:31.829 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-04-16T16:43:31.830 DEBUG:teuthology.orchestra.run.smithi045:> sudo service auditd rotate 2024-04-16T16:43:31.947 INFO:teuthology.orchestra.run.smithi045.stdout:Rotating logs: 2024-04-16T16:43:31.951 DEBUG:teuthology.orchestra.run.smithi104:> sudo service auditd rotate 2024-04-16T16:43:32.018 INFO:teuthology.orchestra.run.smithi104.stdout:Rotating logs: 2024-04-16T16:43:32.020 DEBUG:teuthology.orchestra.run.smithi117:> sudo service auditd rotate 2024-04-16T16:43:32.074 INFO:teuthology.orchestra.run.smithi117.stdout:Rotating logs: 2024-04-16T16:43:32.076 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-04-16T16:43:32.076 DEBUG:teuthology.orchestra.run.smithi045:> /usr/sbin/getenforce 2024-04-16T16:43:32.140 INFO:teuthology.orchestra.run.smithi045.stdout:Permissive 2024-04-16T16:43:32.141 DEBUG:teuthology.orchestra.run.smithi104:> /usr/sbin/getenforce 2024-04-16T16:43:32.172 INFO:teuthology.orchestra.run.smithi104.stdout:Permissive 2024-04-16T16:43:32.172 DEBUG:teuthology.orchestra.run.smithi117:> /usr/sbin/getenforce 2024-04-16T16:43:32.202 INFO:teuthology.orchestra.run.smithi117.stdout:Permissive 2024-04-16T16:43:32.202 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi045.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi104.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi117.front.sepia.ceph.com': 'permissive'} 2024-04-16T16:43:32.202 DEBUG:teuthology.orchestra.run.smithi045:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-16T16:43:32.295 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-16T16:43:32.296 DEBUG:teuthology.orchestra.run.smithi104:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-16T16:43:32.323 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-16T16:43:32.323 DEBUG:teuthology.orchestra.run.smithi117:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-16T16:43:32.351 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-16T16:43:32.351 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-04-16T16:43:32.352 DEBUG:teuthology.orchestra.run.smithi045:> sudo /usr/sbin/setenforce permissive 2024-04-16T16:43:32.452 DEBUG:teuthology.orchestra.run.smithi104:> sudo /usr/sbin/setenforce permissive 2024-04-16T16:43:32.490 DEBUG:teuthology.orchestra.run.smithi117:> sudo /usr/sbin/setenforce permissive 2024-04-16T16:43:32.520 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-04-16T16:43:32.530 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-04-16T16:43:32.546 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-04-16T16:43:32.548 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi045.front.sepia.ceph.com,smithi104.front.sepia.ceph.com,smithi117.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-04-16T16:49:43.299 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi045.front.sepia.ceph.com'), Remote(name='ubuntu@smithi104.front.sepia.ceph.com'), Remote(name='ubuntu@smithi117.front.sepia.ceph.com')] 2024-04-16T16:49:43.301 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi045.front.sepia.ceph.com' 2024-04-16T16:49:43.302 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi045.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-16T16:49:43.462 DEBUG:teuthology.orchestra.run.smithi045:> true 2024-04-16T16:49:43.629 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi045.front.sepia.ceph.com' 2024-04-16T16:49:43.630 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi104.front.sepia.ceph.com' 2024-04-16T16:49:43.630 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi104.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-16T16:49:43.717 DEBUG:teuthology.orchestra.run.smithi104:> true 2024-04-16T16:49:43.786 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi104.front.sepia.ceph.com' 2024-04-16T16:49:43.787 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi117.front.sepia.ceph.com' 2024-04-16T16:49:43.787 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi117.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-16T16:49:43.877 DEBUG:teuthology.orchestra.run.smithi117:> true 2024-04-16T16:49:43.950 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi117.front.sepia.ceph.com' 2024-04-16T16:49:43.951 INFO:teuthology.run_tasks:Running task clock... 2024-04-16T16:49:43.964 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-04-16T16:49:43.965 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-16T16:49:43.965 DEBUG:teuthology.orchestra.run.smithi045:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-16T16:49:43.973 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-16T16:49:43.973 DEBUG:teuthology.orchestra.run.smithi104:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-16T16:49:43.976 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-16T16:49:43.976 DEBUG:teuthology.orchestra.run.smithi117:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-16T16:49:44.009 INFO:teuthology.orchestra.run.smithi104.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-16T16:49:44.023 INFO:teuthology.orchestra.run.smithi117.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-16T16:49:44.026 INFO:teuthology.orchestra.run.smithi104.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-16T16:49:44.041 INFO:teuthology.orchestra.run.smithi117.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-16T16:49:44.058 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: ntpd: command not found 2024-04-16T16:49:44.071 INFO:teuthology.orchestra.run.smithi117.stderr:sudo: ntpd: command not found 2024-04-16T16:49:44.072 INFO:teuthology.orchestra.run.smithi104.stdout:506 Cannot talk to daemon 2024-04-16T16:49:44.076 INFO:teuthology.orchestra.run.smithi045.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-16T16:49:44.085 INFO:teuthology.orchestra.run.smithi117.stdout:506 Cannot talk to daemon 2024-04-16T16:49:44.088 INFO:teuthology.orchestra.run.smithi104.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-16T16:49:44.101 INFO:teuthology.orchestra.run.smithi117.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-16T16:49:44.105 INFO:teuthology.orchestra.run.smithi104.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-16T16:49:44.118 INFO:teuthology.orchestra.run.smithi117.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-16T16:49:44.132 INFO:teuthology.orchestra.run.smithi045.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-16T16:49:44.149 INFO:teuthology.orchestra.run.smithi104.stderr:bash: line 1: ntpq: command not found 2024-04-16T16:49:44.152 INFO:teuthology.orchestra.run.smithi104.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-16T16:49:44.152 INFO:teuthology.orchestra.run.smithi104.stdout:=============================================================================== 2024-04-16T16:49:44.152 INFO:teuthology.orchestra.run.smithi104.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-16T16:49:44.152 INFO:teuthology.orchestra.run.smithi104.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-16T16:49:44.152 INFO:teuthology.orchestra.run.smithi104.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-16T16:49:44.152 INFO:teuthology.orchestra.run.smithi104.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-16T16:49:44.162 INFO:teuthology.orchestra.run.smithi117.stderr:bash: line 1: ntpq: command not found 2024-04-16T16:49:44.165 INFO:teuthology.orchestra.run.smithi117.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-16T16:49:44.165 INFO:teuthology.orchestra.run.smithi117.stdout:=============================================================================== 2024-04-16T16:49:44.165 INFO:teuthology.orchestra.run.smithi117.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-16T16:49:44.165 INFO:teuthology.orchestra.run.smithi117.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-16T16:49:44.165 INFO:teuthology.orchestra.run.smithi117.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-16T16:49:44.165 INFO:teuthology.orchestra.run.smithi117.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-16T16:49:44.237 INFO:teuthology.orchestra.run.smithi045.stderr:sudo: ntpd: command not found 2024-04-16T16:49:44.283 INFO:teuthology.orchestra.run.smithi045.stdout:506 Cannot talk to daemon 2024-04-16T16:49:44.339 INFO:teuthology.orchestra.run.smithi045.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-16T16:49:44.396 INFO:teuthology.orchestra.run.smithi045.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-16T16:49:44.550 INFO:teuthology.orchestra.run.smithi045.stderr:bash: line 1: ntpq: command not found 2024-04-16T16:49:44.559 INFO:teuthology.orchestra.run.smithi045.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-16T16:49:44.560 INFO:teuthology.orchestra.run.smithi045.stdout:=============================================================================== 2024-04-16T16:49:44.560 INFO:teuthology.orchestra.run.smithi045.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-16T16:49:44.560 INFO:teuthology.orchestra.run.smithi045.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-16T16:49:44.560 INFO:teuthology.orchestra.run.smithi045.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-16T16:49:44.560 INFO:teuthology.orchestra.run.smithi045.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-16T16:49:44.562 INFO:teuthology.run_tasks:Running task install... 2024-04-16T16:49:44.570 DEBUG:teuthology.task.install:project ceph 2024-04-16T16:49:44.571 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '9b42bfbeda298874861ece590aa565cf64ac8c45'}} 2024-04-16T16:49:44.571 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '9b42bfbeda298874861ece590aa565cf64ac8c45'} 2024-04-16T16:49:44.571 INFO:teuthology.task.install:Using flavor: default 2024-04-16T16:49:44.576 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-04-16T16:49:44.577 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-04-16T16:49:44.577 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '9b42bfbeda298874861ece590aa565cf64ac8c45', 'tag': None, 'wait_for_package': False} 2024-04-16T16:49:44.578 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=9b42bfbeda298874861ece590aa565cf64ac8c45 2024-04-16T16:49:44.580 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '9b42bfbeda298874861ece590aa565cf64ac8c45', 'tag': None, 'wait_for_package': False} 2024-04-16T16:49:44.580 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=9b42bfbeda298874861ece590aa565cf64ac8c45 2024-04-16T16:49:44.581 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '9b42bfbeda298874861ece590aa565cf64ac8c45', 'tag': None, 'wait_for_package': False} 2024-04-16T16:49:44.582 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=9b42bfbeda298874861ece590aa565cf64ac8c45 2024-04-16T16:49:44.717 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-mchangir-qa-debug-resource-temporarily-unavailable-issue/9b42bfbeda298874861ece590aa565cf64ac8c45/centos/9/flavors/default/ 2024-04-16T16:49:44.718 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3101.g9b42bfbe 2024-04-16T16:49:44.737 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-mchangir-qa-debug-resource-temporarily-unavailable-issue/9b42bfbeda298874861ece590aa565cf64ac8c45/centos/9/flavors/default/ 2024-04-16T16:49:44.737 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3101.g9b42bfbe 2024-04-16T16:49:44.829 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-mchangir-qa-debug-resource-temporarily-unavailable-issue/9b42bfbeda298874861ece590aa565cf64ac8c45/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-mchangir-qa-debug-resource-temporarily-unavailable-issue/9b42bfbeda298874861ece590aa565cf64ac8c45/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-mchangir-qa-debug-resource-temporarily-unavailable-issue/9b42bfbeda298874861ece590aa565cf64ac8c45/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-16T16:49:44.829 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-04-16T16:49:44.829 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-16T16:49:44.912 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-mchangir-qa-debug-resource-temporarily-unavailable-issue/9b42bfbeda298874861ece590aa565cf64ac8c45/centos/9/flavors/default/ 2024-04-16T16:49:44.912 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3101.g9b42bfbe 2024-04-16T16:49:44.937 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-16T16:49:44.938 DEBUG:teuthology.orchestra.run.smithi045:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/9b42bfbeda298874861ece590aa565cf64ac8c45/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-16T16:49:44.948 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-mchangir-qa-debug-resource-temporarily-unavailable-issue/9b42bfbeda298874861ece590aa565cf64ac8c45/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-mchangir-qa-debug-resource-temporarily-unavailable-issue/9b42bfbeda298874861ece590aa565cf64ac8c45/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-mchangir-qa-debug-resource-temporarily-unavailable-issue/9b42bfbeda298874861ece590aa565cf64ac8c45/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-16T16:49:44.948 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2024-04-16T16:49:44.948 DEBUG:teuthology.orchestra.run.smithi104:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-16T16:49:44.980 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-16T16:49:44.980 DEBUG:teuthology.orchestra.run.smithi104:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/9b42bfbeda298874861ece590aa565cf64ac8c45/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-16T16:49:45.050 DEBUG:teuthology.orchestra.run.smithi104:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-16T16:49:45.084 DEBUG:teuthology.orchestra.run.smithi045:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-16T16:49:45.091 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-mchangir-qa-debug-resource-temporarily-unavailable-issue/9b42bfbeda298874861ece590aa565cf64ac8c45/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-mchangir-qa-debug-resource-temporarily-unavailable-issue/9b42bfbeda298874861ece590aa565cf64ac8c45/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-mchangir-qa-debug-resource-temporarily-unavailable-issue/9b42bfbeda298874861ece590aa565cf64ac8c45/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-16T16:49:45.091 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-04-16T16:49:45.091 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-16T16:49:45.121 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-16T16:49:45.122 DEBUG:teuthology.orchestra.run.smithi117:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/9b42bfbeda298874861ece590aa565cf64ac8c45/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-16T16:49:45.134 DEBUG:teuthology.orchestra.run.smithi104:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-16T16:49:45.190 DEBUG:teuthology.orchestra.run.smithi117:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-16T16:49:45.201 INFO:teuthology.orchestra.run.smithi104.stdout:check_obsoletes = 1 2024-04-16T16:49:45.203 DEBUG:teuthology.orchestra.run.smithi104:> sudo yum clean all 2024-04-16T16:49:45.230 DEBUG:teuthology.orchestra.run.smithi045:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-16T16:49:45.271 DEBUG:teuthology.orchestra.run.smithi117:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-16T16:49:45.324 INFO:teuthology.orchestra.run.smithi045.stdout:check_obsoletes = 1 2024-04-16T16:49:45.329 DEBUG:teuthology.orchestra.run.smithi045:> sudo yum clean all 2024-04-16T16:49:45.337 INFO:teuthology.orchestra.run.smithi117.stdout:check_obsoletes = 1 2024-04-16T16:49:45.339 DEBUG:teuthology.orchestra.run.smithi117:> sudo yum clean all 2024-04-16T16:49:45.436 INFO:teuthology.orchestra.run.smithi104.stdout:74 files removed 2024-04-16T16:49:45.463 DEBUG:teuthology.orchestra.run.smithi104:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-16T16:49:45.572 INFO:teuthology.orchestra.run.smithi117.stdout:74 files removed 2024-04-16T16:49:45.599 DEBUG:teuthology.orchestra.run.smithi117:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-16T16:49:45.717 INFO:teuthology.orchestra.run.smithi045.stdout:74 files removed 2024-04-16T16:49:45.758 DEBUG:teuthology.orchestra.run.smithi045:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-16T16:49:46.225 INFO:teuthology.orchestra.run.smithi104.stdout:ceph packages for x86_64 177 kB/s | 83 kB 00:00 2024-04-16T16:49:46.366 INFO:teuthology.orchestra.run.smithi117.stdout:ceph packages for x86_64 175 kB/s | 83 kB 00:00 2024-04-16T16:49:46.567 INFO:teuthology.orchestra.run.smithi104.stdout:ceph noarch packages 39 kB/s | 12 kB 00:00 2024-04-16T16:49:46.774 INFO:teuthology.orchestra.run.smithi117.stdout:ceph noarch packages 32 kB/s | 12 kB 00:00 2024-04-16T16:49:46.851 INFO:teuthology.orchestra.run.smithi045.stdout:ceph packages for x86_64 147 kB/s | 83 kB 00:00 2024-04-16T16:49:47.018 INFO:teuthology.orchestra.run.smithi104.stdout:ceph source packages 4.5 kB/s | 1.9 kB 00:00 2024-04-16T16:49:47.100 INFO:teuthology.orchestra.run.smithi117.stdout:ceph source packages 6.4 kB/s | 1.9 kB 00:00 2024-04-16T16:49:47.377 INFO:teuthology.orchestra.run.smithi045.stdout:ceph noarch packages 27 kB/s | 12 kB 00:00 2024-04-16T16:49:47.478 INFO:teuthology.orchestra.run.smithi104.stdout:CentOS-9 - AppStream 15 MB/s | 6.7 MB 00:00 2024-04-16T16:49:47.636 INFO:teuthology.orchestra.run.smithi117.stdout:CentOS-9 - AppStream 13 MB/s | 6.7 MB 00:00 2024-04-16T16:49:48.102 INFO:teuthology.orchestra.run.smithi045.stdout:ceph source packages 2.9 kB/s | 1.9 kB 00:00 2024-04-16T16:49:48.822 INFO:teuthology.orchestra.run.smithi045.stdout:CentOS-9 - AppStream 10 MB/s | 6.7 MB 00:00 2024-04-16T16:49:48.937 INFO:teuthology.orchestra.run.smithi104.stdout:CentOS-9 - BaseOS 5.8 MB/s | 2.0 MB 00:00 2024-04-16T16:49:49.061 INFO:teuthology.orchestra.run.smithi117.stdout:CentOS-9 - BaseOS 7.0 MB/s | 2.0 MB 00:00 2024-04-16T16:49:49.694 INFO:teuthology.orchestra.run.smithi104.stdout:CentOS Stream 9 - BaseOS 19 MB/s | 8.0 MB 00:00 2024-04-16T16:49:50.523 INFO:teuthology.orchestra.run.smithi045.stdout:CentOS-9 - BaseOS 5.9 MB/s | 2.0 MB 00:00 2024-04-16T16:49:50.893 INFO:teuthology.orchestra.run.smithi117.stdout:CentOS Stream 9 - BaseOS 5.3 MB/s | 8.0 MB 00:01 2024-04-16T16:49:51.646 INFO:teuthology.orchestra.run.smithi104.stdout:CentOS Stream 9 - AppStream 24 MB/s | 19 MB 00:00 2024-04-16T16:49:51.711 INFO:teuthology.orchestra.run.smithi045.stdout:CentOS Stream 9 - BaseOS 10 MB/s | 8.0 MB 00:00 2024-04-16T16:49:53.748 INFO:teuthology.orchestra.run.smithi045.stdout:CentOS Stream 9 - AppStream 25 MB/s | 19 MB 00:00 2024-04-16T16:49:54.552 INFO:teuthology.orchestra.run.smithi117.stdout:CentOS Stream 9 - AppStream 7.5 MB/s | 19 MB 00:02 2024-04-16T16:49:56.638 INFO:teuthology.orchestra.run.smithi104.stdout:CentOS Stream 9 - CRB 4.3 MB/s | 6.2 MB 00:01 2024-04-16T16:49:58.288 INFO:teuthology.orchestra.run.smithi045.stdout:CentOS Stream 9 - CRB 10 MB/s | 6.2 MB 00:00 2024-04-16T16:49:58.346 INFO:teuthology.orchestra.run.smithi104.stdout:CentOS Stream 9 - Extras packages 31 kB/s | 16 kB 00:00 2024-04-16T16:49:59.669 INFO:teuthology.orchestra.run.smithi117.stdout:CentOS Stream 9 - CRB 4.0 MB/s | 6.2 MB 00:01 2024-04-16T16:49:59.930 INFO:teuthology.orchestra.run.smithi045.stdout:CentOS Stream 9 - Extras packages 58 kB/s | 16 kB 00:00 2024-04-16T16:50:01.237 INFO:teuthology.orchestra.run.smithi117.stdout:CentOS Stream 9 - Extras packages 40 kB/s | 16 kB 00:00 2024-04-16T16:50:01.924 INFO:teuthology.orchestra.run.smithi045.stdout:Extra Packages for Enterprise Linux 12 MB/s | 21 MB 00:01 2024-04-16T16:50:03.229 INFO:teuthology.orchestra.run.smithi117.stdout:Extra Packages for Enterprise Linux 11 MB/s | 21 MB 00:01 2024-04-16T16:50:08.257 INFO:teuthology.orchestra.run.smithi045.stdout:lab-extras 28 kB/s | 1.7 kB 00:00 2024-04-16T16:50:08.570 INFO:teuthology.orchestra.run.smithi117.stdout:lab-extras 4.7 kB/s | 1.7 kB 00:00 2024-04-16T16:50:08.613 INFO:teuthology.orchestra.run.smithi104.stdout:Extra Packages for Enterprise Linux 2.1 MB/s | 21 MB 00:10 2024-04-16T16:50:11.243 INFO:teuthology.orchestra.run.smithi117.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-16T16:50:11.243 INFO:teuthology.orchestra.run.smithi117.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-16T16:50:11.257 INFO:teuthology.orchestra.run.smithi117.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-04-16T16:50:11.258 INFO:teuthology.orchestra.run.smithi117.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-16T16:50:11.259 INFO:teuthology.orchestra.run.smithi117.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-04-16T16:50:11.260 INFO:teuthology.orchestra.run.smithi117.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-04-16T16:50:11.262 INFO:teuthology.orchestra.run.smithi117.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-16T16:50:11.265 INFO:teuthology.orchestra.run.smithi117.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-04-16T16:50:11.266 INFO:teuthology.orchestra.run.smithi117.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-04-16T16:50:11.347 INFO:teuthology.orchestra.run.smithi117.stdout:Dependencies resolved. 2024-04-16T16:50:11.366 INFO:teuthology.orchestra.run.smithi117.stdout:============================================================================================ 2024-04-16T16:50:11.366 INFO:teuthology.orchestra.run.smithi117.stdout: Package Arch Version Repository Size 2024-04-16T16:50:11.366 INFO:teuthology.orchestra.run.smithi117.stdout:============================================================================================ 2024-04-16T16:50:11.366 INFO:teuthology.orchestra.run.smithi117.stdout:Installing: 2024-04-16T16:50:11.366 INFO:teuthology.orchestra.run.smithi117.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2024-04-16T16:50:11.366 INFO:teuthology.orchestra.run.smithi117.stdout: ceph x86_64 2:19.0.0-3101.g9b42bfbe.el9 ceph 6.6 k 2024-04-16T16:50:11.366 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-base x86_64 2:19.0.0-3101.g9b42bfbe.el9 ceph 5.4 M 2024-04-16T16:50:11.367 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-fuse x86_64 2:19.0.0-3101.g9b42bfbe.el9 ceph 890 k 2024-04-16T16:50:11.367 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3101.g9b42bfbe.el9 ceph 145 k 2024-04-16T16:50:11.367 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mgr x86_64 2:19.0.0-3101.g9b42bfbe.el9 ceph 1.5 M 2024-04-16T16:50:11.367 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3101.g9b42bfbe.el9 ceph-noarch 142 k 2024-04-16T16:50:11.367 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3101.g9b42bfbe.el9 ceph-noarch 3.5 M 2024-04-16T16:50:11.367 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3101.g9b42bfbe.el9 ceph-noarch 7.4 M 2024-04-16T16:50:11.367 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mgr-rook noarch 2:19.0.0-3101.g9b42bfbe.el9 ceph-noarch 49 k 2024-04-16T16:50:11.367 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-radosgw x86_64 2:19.0.0-3101.g9b42bfbe.el9 ceph 11 M 2024-04-16T16:50:11.367 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-test x86_64 2:19.0.0-3101.g9b42bfbe.el9 ceph 49 M 2024-04-16T16:50:11.367 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-volume noarch 2:19.0.0-3101.g9b42bfbe.el9 ceph-noarch 273 k 2024-04-16T16:50:11.367 INFO:teuthology.orchestra.run.smithi117.stdout: cephadm noarch 2:19.0.0-3101.g9b42bfbe.el9 ceph-noarch 762 k 2024-04-16T16:50:11.367 INFO:teuthology.orchestra.run.smithi117.stdout: cephfs-mirror x86_64 2:19.0.0-3101.g9b42bfbe.el9 ceph 225 k 2024-04-16T16:50:11.367 INFO:teuthology.orchestra.run.smithi117.stdout: cephfs-shell x86_64 2:19.0.0-3101.g9b42bfbe.el9 ceph 27 k 2024-04-16T16:50:11.368 INFO:teuthology.orchestra.run.smithi117.stdout: cephfs-top noarch 2:19.0.0-3101.g9b42bfbe.el9 ceph-noarch 18 k 2024-04-16T16:50:11.368 INFO:teuthology.orchestra.run.smithi117.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-16T16:50:11.368 INFO:teuthology.orchestra.run.smithi117.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2024-04-16T16:50:11.368 INFO:teuthology.orchestra.run.smithi117.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2024-04-16T16:50:11.368 INFO:teuthology.orchestra.run.smithi117.stdout: libcephfs-devel x86_64 2:19.0.0-3101.g9b42bfbe.el9 ceph 32 k 2024-04-16T16:50:11.368 INFO:teuthology.orchestra.run.smithi117.stdout: libcephfs2 x86_64 2:19.0.0-3101.g9b42bfbe.el9 ceph 747 k 2024-04-16T16:50:11.368 INFO:teuthology.orchestra.run.smithi117.stdout: librados-devel x86_64 2:19.0.0-3101.g9b42bfbe.el9 ceph 127 k 2024-04-16T16:50:11.368 INFO:teuthology.orchestra.run.smithi117.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2024-04-16T16:50:11.368 INFO:teuthology.orchestra.run.smithi117.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-16T16:50:11.368 INFO:teuthology.orchestra.run.smithi117.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-04-16T16:50:11.368 INFO:teuthology.orchestra.run.smithi117.stdout: postgresql x86_64 13.14-1.el9 CentOS-AppStream 1.5 M 2024-04-16T16:50:11.368 INFO:teuthology.orchestra.run.smithi117.stdout: postgresql-contrib x86_64 13.14-1.el9 CentOS-AppStream 817 k 2024-04-16T16:50:11.368 INFO:teuthology.orchestra.run.smithi117.stdout: postgresql-server x86_64 13.14-1.el9 CentOS-AppStream 5.7 M 2024-04-16T16:50:11.368 INFO:teuthology.orchestra.run.smithi117.stdout: python3-cephfs x86_64 2:19.0.0-3101.g9b42bfbe.el9 ceph 162 k 2024-04-16T16:50:11.369 INFO:teuthology.orchestra.run.smithi117.stdout: python3-rados x86_64 2:19.0.0-3101.g9b42bfbe.el9 ceph 320 k 2024-04-16T16:50:11.369 INFO:teuthology.orchestra.run.smithi117.stdout: python3-rbd x86_64 2:19.0.0-3101.g9b42bfbe.el9 ceph 298 k 2024-04-16T16:50:11.369 INFO:teuthology.orchestra.run.smithi117.stdout: python3-rgw x86_64 2:19.0.0-3101.g9b42bfbe.el9 ceph 99 k 2024-04-16T16:50:11.369 INFO:teuthology.orchestra.run.smithi117.stdout: rbd-fuse x86_64 2:19.0.0-3101.g9b42bfbe.el9 ceph 85 k 2024-04-16T16:50:11.369 INFO:teuthology.orchestra.run.smithi117.stdout: rbd-mirror x86_64 2:19.0.0-3101.g9b42bfbe.el9 ceph 3.1 M 2024-04-16T16:50:11.369 INFO:teuthology.orchestra.run.smithi117.stdout: rbd-nbd x86_64 2:19.0.0-3101.g9b42bfbe.el9 ceph 170 k 2024-04-16T16:50:11.369 INFO:teuthology.orchestra.run.smithi117.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-16T16:50:11.369 INFO:teuthology.orchestra.run.smithi117.stdout:Upgrading: 2024-04-16T16:50:11.369 INFO:teuthology.orchestra.run.smithi117.stdout: NetworkManager x86_64 1:1.47.4-1.el9 CentOS-BaseOS 2.3 M 2024-04-16T16:50:11.369 INFO:teuthology.orchestra.run.smithi117.stdout: NetworkManager-libnm x86_64 1:1.47.4-1.el9 CentOS-BaseOS 1.8 M 2024-04-16T16:50:11.369 INFO:teuthology.orchestra.run.smithi117.stdout: NetworkManager-team x86_64 1:1.47.4-1.el9 CentOS-BaseOS 41 k 2024-04-16T16:50:11.369 INFO:teuthology.orchestra.run.smithi117.stdout: NetworkManager-tui x86_64 1:1.47.4-1.el9 CentOS-BaseOS 248 k 2024-04-16T16:50:11.369 INFO:teuthology.orchestra.run.smithi117.stdout: acl x86_64 2.3.1-4.el9 baseos 71 k 2024-04-16T16:50:11.370 INFO:teuthology.orchestra.run.smithi117.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 baseos 36 k 2024-04-16T16:50:11.370 INFO:teuthology.orchestra.run.smithi117.stdout: elfutils-libelf x86_64 0.190-2.el9 baseos 192 k 2024-04-16T16:50:11.370 INFO:teuthology.orchestra.run.smithi117.stdout: elfutils-libs x86_64 0.190-2.el9 baseos 256 k 2024-04-16T16:50:11.370 INFO:teuthology.orchestra.run.smithi117.stdout: libacl x86_64 2.3.1-4.el9 baseos 23 k 2024-04-16T16:50:11.370 INFO:teuthology.orchestra.run.smithi117.stdout: libacl-devel x86_64 2.3.1-4.el9 appstream 74 k 2024-04-16T16:50:11.370 INFO:teuthology.orchestra.run.smithi117.stdout: libblkid x86_64 2.37.4-18.el9 baseos 108 k 2024-04-16T16:50:11.370 INFO:teuthology.orchestra.run.smithi117.stdout: libfdisk x86_64 2.37.4-18.el9 baseos 155 k 2024-04-16T16:50:11.370 INFO:teuthology.orchestra.run.smithi117.stdout: libmount x86_64 2.37.4-18.el9 baseos 136 k 2024-04-16T16:50:11.370 INFO:teuthology.orchestra.run.smithi117.stdout: librados2 x86_64 2:19.0.0-3101.g9b42bfbe.el9 ceph 3.4 M 2024-04-16T16:50:11.370 INFO:teuthology.orchestra.run.smithi117.stdout: librbd1 x86_64 2:19.0.0-3101.g9b42bfbe.el9 ceph 3.1 M 2024-04-16T16:50:11.370 INFO:teuthology.orchestra.run.smithi117.stdout: libsmartcols x86_64 2.37.4-18.el9 baseos 64 k 2024-04-16T16:50:11.370 INFO:teuthology.orchestra.run.smithi117.stdout: libuuid x86_64 2.37.4-18.el9 baseos 29 k 2024-04-16T16:50:11.370 INFO:teuthology.orchestra.run.smithi117.stdout: openssl x86_64 1:3.0.7-27.el9 baseos 1.2 M 2024-04-16T16:50:11.371 INFO:teuthology.orchestra.run.smithi117.stdout: openssl-libs x86_64 1:3.0.7-27.el9 baseos 2.2 M 2024-04-16T16:50:11.371 INFO:teuthology.orchestra.run.smithi117.stdout: util-linux x86_64 2.37.4-18.el9 baseos 2.3 M 2024-04-16T16:50:11.371 INFO:teuthology.orchestra.run.smithi117.stdout: util-linux-core x86_64 2.37.4-18.el9 baseos 465 k 2024-04-16T16:50:11.371 INFO:teuthology.orchestra.run.smithi117.stdout: util-linux-user x86_64 2.37.4-18.el9 baseos 31 k 2024-04-16T16:50:11.371 INFO:teuthology.orchestra.run.smithi117.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2024-04-16T16:50:11.371 INFO:teuthology.orchestra.run.smithi117.stdout: xfsprogs x86_64 6.3.0-1.el9 baseos 1.1 M 2024-04-16T16:50:11.371 INFO:teuthology.orchestra.run.smithi117.stdout:Installing dependencies: 2024-04-16T16:50:11.371 INFO:teuthology.orchestra.run.smithi117.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-16T16:50:11.371 INFO:teuthology.orchestra.run.smithi117.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2024-04-16T16:50:11.371 INFO:teuthology.orchestra.run.smithi117.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-04-16T16:50:11.371 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-common x86_64 2:19.0.0-3101.g9b42bfbe.el9 ceph 21 M 2024-04-16T16:50:11.371 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3101.g9b42bfbe.el9 ceph-noarch 34 k 2024-04-16T16:50:11.372 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mds x86_64 2:19.0.0-3101.g9b42bfbe.el9 ceph 2.4 M 2024-04-16T16:50:11.372 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3101.g9b42bfbe.el9 ceph-noarch 249 k 2024-04-16T16:50:11.372 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mon x86_64 2:19.0.0-3101.g9b42bfbe.el9 ceph 4.7 M 2024-04-16T16:50:11.372 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-osd x86_64 2:19.0.0-3101.g9b42bfbe.el9 ceph 17 M 2024-04-16T16:50:11.372 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3101.g9b42bfbe.el9 ceph-noarch 16 k 2024-04-16T16:50:11.372 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-selinux x86_64 2:19.0.0-3101.g9b42bfbe.el9 ceph 25 k 2024-04-16T16:50:11.372 INFO:teuthology.orchestra.run.smithi117.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-04-16T16:50:11.372 INFO:teuthology.orchestra.run.smithi117.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-04-16T16:50:11.372 INFO:teuthology.orchestra.run.smithi117.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-04-16T16:50:11.372 INFO:teuthology.orchestra.run.smithi117.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-04-16T16:50:11.372 INFO:teuthology.orchestra.run.smithi117.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-16T16:50:11.372 INFO:teuthology.orchestra.run.smithi117.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-16T16:50:11.372 INFO:teuthology.orchestra.run.smithi117.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-04-16T16:50:11.373 INFO:teuthology.orchestra.run.smithi117.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-16T16:50:11.373 INFO:teuthology.orchestra.run.smithi117.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-16T16:50:11.373 INFO:teuthology.orchestra.run.smithi117.stdout: libcephsqlite x86_64 2:19.0.0-3101.g9b42bfbe.el9 ceph 163 k 2024-04-16T16:50:11.373 INFO:teuthology.orchestra.run.smithi117.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-16T16:50:11.373 INFO:teuthology.orchestra.run.smithi117.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-16T16:50:11.373 INFO:teuthology.orchestra.run.smithi117.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-04-16T16:50:11.373 INFO:teuthology.orchestra.run.smithi117.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-16T16:50:11.373 INFO:teuthology.orchestra.run.smithi117.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-04-16T16:50:11.373 INFO:teuthology.orchestra.run.smithi117.stdout: libradosstriper1 x86_64 2:19.0.0-3101.g9b42bfbe.el9 ceph 501 k 2024-04-16T16:50:11.373 INFO:teuthology.orchestra.run.smithi117.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-04-16T16:50:11.373 INFO:teuthology.orchestra.run.smithi117.stdout: librgw2 x86_64 2:19.0.0-3101.g9b42bfbe.el9 ceph 5.4 M 2024-04-16T16:50:11.373 INFO:teuthology.orchestra.run.smithi117.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-16T16:50:11.373 INFO:teuthology.orchestra.run.smithi117.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2024-04-16T16:50:11.373 INFO:teuthology.orchestra.run.smithi117.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-04-16T16:50:11.374 INFO:teuthology.orchestra.run.smithi117.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-04-16T16:50:11.374 INFO:teuthology.orchestra.run.smithi117.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-16T16:50:11.374 INFO:teuthology.orchestra.run.smithi117.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-16T16:50:11.374 INFO:teuthology.orchestra.run.smithi117.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2024-04-16T16:50:11.374 INFO:teuthology.orchestra.run.smithi117.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-04-16T16:50:11.374 INFO:teuthology.orchestra.run.smithi117.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-04-16T16:50:11.374 INFO:teuthology.orchestra.run.smithi117.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-16T16:50:11.374 INFO:teuthology.orchestra.run.smithi117.stdout: postgresql-private-libs x86_64 13.14-1.el9 CentOS-AppStream 132 k 2024-04-16T16:50:11.374 INFO:teuthology.orchestra.run.smithi117.stdout: protobuf x86_64 3.14.0-13.el9 CentOS-AppStream 1.0 M 2024-04-16T16:50:11.374 INFO:teuthology.orchestra.run.smithi117.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-16T16:50:11.374 INFO:teuthology.orchestra.run.smithi117.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-16T16:50:11.374 INFO:teuthology.orchestra.run.smithi117.stdout: python3-attrs noarch 20.3.0-7.el9 CentOS-AppStream 78 k 2024-04-16T16:50:11.374 INFO:teuthology.orchestra.run.smithi117.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-04-16T16:50:11.375 INFO:teuthology.orchestra.run.smithi117.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-16T16:50:11.375 INFO:teuthology.orchestra.run.smithi117.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-16T16:50:11.375 INFO:teuthology.orchestra.run.smithi117.stdout: python3-ceph-argparse x86_64 2:19.0.0-3101.g9b42bfbe.el9 ceph 45 k 2024-04-16T16:50:11.375 INFO:teuthology.orchestra.run.smithi117.stdout: python3-ceph-common x86_64 2:19.0.0-3101.g9b42bfbe.el9 ceph 135 k 2024-04-16T16:50:11.375 INFO:teuthology.orchestra.run.smithi117.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-16T16:50:11.375 INFO:teuthology.orchestra.run.smithi117.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-04-16T16:50:11.375 INFO:teuthology.orchestra.run.smithi117.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-16T16:50:11.375 INFO:teuthology.orchestra.run.smithi117.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-16T16:50:11.375 INFO:teuthology.orchestra.run.smithi117.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-16T16:50:11.375 INFO:teuthology.orchestra.run.smithi117.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-16T16:50:11.375 INFO:teuthology.orchestra.run.smithi117.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-04-16T16:50:11.375 INFO:teuthology.orchestra.run.smithi117.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-04-16T16:50:11.375 INFO:teuthology.orchestra.run.smithi117.stdout: python3-google-auth noarch 1:2.28.2-1.el9 epel 203 k 2024-04-16T16:50:11.375 INFO:teuthology.orchestra.run.smithi117.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-16T16:50:11.376 INFO:teuthology.orchestra.run.smithi117.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-16T16:50:11.376 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-16T16:50:11.376 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-16T16:50:11.376 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-16T16:50:11.376 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-16T16:50:11.376 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-16T16:50:11.376 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-16T16:50:11.376 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-04-16T16:50:11.376 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-04-16T16:50:11.376 INFO:teuthology.orchestra.run.smithi117.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-16T16:50:11.376 INFO:teuthology.orchestra.run.smithi117.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-16T16:50:11.376 INFO:teuthology.orchestra.run.smithi117.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-04-16T16:50:11.376 INFO:teuthology.orchestra.run.smithi117.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-04-16T16:50:11.376 INFO:teuthology.orchestra.run.smithi117.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-16T16:50:11.377 INFO:teuthology.orchestra.run.smithi117.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-16T16:50:11.377 INFO:teuthology.orchestra.run.smithi117.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-04-16T16:50:11.377 INFO:teuthology.orchestra.run.smithi117.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-04-16T16:50:11.377 INFO:teuthology.orchestra.run.smithi117.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-04-16T16:50:11.377 INFO:teuthology.orchestra.run.smithi117.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-04-16T16:50:11.377 INFO:teuthology.orchestra.run.smithi117.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-16T16:50:11.377 INFO:teuthology.orchestra.run.smithi117.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-04-16T16:50:11.377 INFO:teuthology.orchestra.run.smithi117.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-16T16:50:11.377 INFO:teuthology.orchestra.run.smithi117.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-04-16T16:50:11.377 INFO:teuthology.orchestra.run.smithi117.stdout: python3-protobuf noarch 3.14.0-13.el9 CentOS-AppStream 238 k 2024-04-16T16:50:11.377 INFO:teuthology.orchestra.run.smithi117.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-16T16:50:11.377 INFO:teuthology.orchestra.run.smithi117.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-04-16T16:50:11.377 INFO:teuthology.orchestra.run.smithi117.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-04-16T16:50:11.377 INFO:teuthology.orchestra.run.smithi117.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-04-16T16:50:11.377 INFO:teuthology.orchestra.run.smithi117.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-16T16:50:11.378 INFO:teuthology.orchestra.run.smithi117.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-04-16T16:50:11.378 INFO:teuthology.orchestra.run.smithi117.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-16T16:50:11.378 INFO:teuthology.orchestra.run.smithi117.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-16T16:50:11.378 INFO:teuthology.orchestra.run.smithi117.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-04-16T16:50:11.378 INFO:teuthology.orchestra.run.smithi117.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-16T16:50:11.378 INFO:teuthology.orchestra.run.smithi117.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-16T16:50:11.378 INFO:teuthology.orchestra.run.smithi117.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-04-16T16:50:11.378 INFO:teuthology.orchestra.run.smithi117.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-16T16:50:11.378 INFO:teuthology.orchestra.run.smithi117.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-04-16T16:50:11.378 INFO:teuthology.orchestra.run.smithi117.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-16T16:50:11.378 INFO:teuthology.orchestra.run.smithi117.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-04-16T16:50:11.378 INFO:teuthology.orchestra.run.smithi117.stdout: python3-wcwidth noarch 0.2.5-8.el9 CentOS-AppStream 41 k 2024-04-16T16:50:11.378 INFO:teuthology.orchestra.run.smithi117.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-16T16:50:11.378 INFO:teuthology.orchestra.run.smithi117.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-16T16:50:11.379 INFO:teuthology.orchestra.run.smithi117.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-16T16:50:11.379 INFO:teuthology.orchestra.run.smithi117.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-16T16:50:11.379 INFO:teuthology.orchestra.run.smithi117.stdout: qatlib x86_64 24.02.0-1.el9 CentOS-AppStream 220 k 2024-04-16T16:50:11.379 INFO:teuthology.orchestra.run.smithi117.stdout: qatzip-libs x86_64 1.2.0-1.el9 CentOS-AppStream 46 k 2024-04-16T16:50:11.379 INFO:teuthology.orchestra.run.smithi117.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-16T16:50:11.379 INFO:teuthology.orchestra.run.smithi117.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-04-16T16:50:11.379 INFO:teuthology.orchestra.run.smithi117.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-16T16:50:11.379 INFO:teuthology.orchestra.run.smithi117.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2024-04-16T16:50:11.379 INFO:teuthology.orchestra.run.smithi117.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-04-16T16:50:11.379 INFO:teuthology.orchestra.run.smithi117.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2024-04-16T16:50:11.379 INFO:teuthology.orchestra.run.smithi117.stdout:Installing weak dependencies: 2024-04-16T16:50:11.379 INFO:teuthology.orchestra.run.smithi117.stdout: qatlib-service x86_64 24.02.0-1.el9 CentOS-AppStream 35 k 2024-04-16T16:50:11.379 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-16T16:50:11.379 INFO:teuthology.orchestra.run.smithi117.stdout:Transaction Summary 2024-04-16T16:50:11.380 INFO:teuthology.orchestra.run.smithi117.stdout:============================================================================================ 2024-04-16T16:50:11.380 INFO:teuthology.orchestra.run.smithi117.stdout:Install 150 Packages 2024-04-16T16:50:11.380 INFO:teuthology.orchestra.run.smithi117.stdout:Upgrade 24 Packages 2024-04-16T16:50:11.380 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-16T16:50:11.380 INFO:teuthology.orchestra.run.smithi117.stdout:Total download size: 229 M 2024-04-16T16:50:11.380 INFO:teuthology.orchestra.run.smithi117.stdout:Downloading Packages: 2024-04-16T16:50:11.392 INFO:teuthology.orchestra.run.smithi045.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-16T16:50:11.394 INFO:teuthology.orchestra.run.smithi045.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-16T16:50:11.418 INFO:teuthology.orchestra.run.smithi045.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-04-16T16:50:11.419 INFO:teuthology.orchestra.run.smithi045.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-16T16:50:11.421 INFO:teuthology.orchestra.run.smithi045.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-04-16T16:50:11.422 INFO:teuthology.orchestra.run.smithi045.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-04-16T16:50:11.425 INFO:teuthology.orchestra.run.smithi045.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-16T16:50:11.429 INFO:teuthology.orchestra.run.smithi045.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-04-16T16:50:11.431 INFO:teuthology.orchestra.run.smithi045.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-04-16T16:50:11.524 INFO:teuthology.orchestra.run.smithi045.stdout:Dependencies resolved. 2024-04-16T16:50:11.554 INFO:teuthology.orchestra.run.smithi045.stdout:============================================================================================ 2024-04-16T16:50:11.554 INFO:teuthology.orchestra.run.smithi045.stdout: Package Arch Version Repository Size 2024-04-16T16:50:11.554 INFO:teuthology.orchestra.run.smithi045.stdout:============================================================================================ 2024-04-16T16:50:11.554 INFO:teuthology.orchestra.run.smithi045.stdout:Installing: 2024-04-16T16:50:11.554 INFO:teuthology.orchestra.run.smithi045.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2024-04-16T16:50:11.555 INFO:teuthology.orchestra.run.smithi045.stdout: ceph x86_64 2:19.0.0-3101.g9b42bfbe.el9 ceph 6.6 k 2024-04-16T16:50:11.555 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-base x86_64 2:19.0.0-3101.g9b42bfbe.el9 ceph 5.4 M 2024-04-16T16:50:11.555 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-fuse x86_64 2:19.0.0-3101.g9b42bfbe.el9 ceph 890 k 2024-04-16T16:50:11.555 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3101.g9b42bfbe.el9 ceph 145 k 2024-04-16T16:50:11.555 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr x86_64 2:19.0.0-3101.g9b42bfbe.el9 ceph 1.5 M 2024-04-16T16:50:11.555 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3101.g9b42bfbe.el9 ceph-noarch 142 k 2024-04-16T16:50:11.555 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3101.g9b42bfbe.el9 ceph-noarch 3.5 M 2024-04-16T16:50:11.555 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3101.g9b42bfbe.el9 ceph-noarch 7.4 M 2024-04-16T16:50:11.555 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr-rook noarch 2:19.0.0-3101.g9b42bfbe.el9 ceph-noarch 49 k 2024-04-16T16:50:11.555 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-radosgw x86_64 2:19.0.0-3101.g9b42bfbe.el9 ceph 11 M 2024-04-16T16:50:11.555 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-test x86_64 2:19.0.0-3101.g9b42bfbe.el9 ceph 49 M 2024-04-16T16:50:11.555 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-volume noarch 2:19.0.0-3101.g9b42bfbe.el9 ceph-noarch 273 k 2024-04-16T16:50:11.555 INFO:teuthology.orchestra.run.smithi045.stdout: cephadm noarch 2:19.0.0-3101.g9b42bfbe.el9 ceph-noarch 762 k 2024-04-16T16:50:11.555 INFO:teuthology.orchestra.run.smithi045.stdout: cephfs-mirror x86_64 2:19.0.0-3101.g9b42bfbe.el9 ceph 225 k 2024-04-16T16:50:11.556 INFO:teuthology.orchestra.run.smithi045.stdout: cephfs-shell x86_64 2:19.0.0-3101.g9b42bfbe.el9 ceph 27 k 2024-04-16T16:50:11.556 INFO:teuthology.orchestra.run.smithi045.stdout: cephfs-top noarch 2:19.0.0-3101.g9b42bfbe.el9 ceph-noarch 18 k 2024-04-16T16:50:11.556 INFO:teuthology.orchestra.run.smithi045.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-16T16:50:11.556 INFO:teuthology.orchestra.run.smithi045.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2024-04-16T16:50:11.556 INFO:teuthology.orchestra.run.smithi045.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2024-04-16T16:50:11.556 INFO:teuthology.orchestra.run.smithi045.stdout: libcephfs-devel x86_64 2:19.0.0-3101.g9b42bfbe.el9 ceph 32 k 2024-04-16T16:50:11.556 INFO:teuthology.orchestra.run.smithi045.stdout: libcephfs2 x86_64 2:19.0.0-3101.g9b42bfbe.el9 ceph 747 k 2024-04-16T16:50:11.556 INFO:teuthology.orchestra.run.smithi045.stdout: librados-devel x86_64 2:19.0.0-3101.g9b42bfbe.el9 ceph 127 k 2024-04-16T16:50:11.556 INFO:teuthology.orchestra.run.smithi045.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2024-04-16T16:50:11.556 INFO:teuthology.orchestra.run.smithi045.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-16T16:50:11.556 INFO:teuthology.orchestra.run.smithi045.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-04-16T16:50:11.556 INFO:teuthology.orchestra.run.smithi045.stdout: postgresql x86_64 13.14-1.el9 CentOS-AppStream 1.5 M 2024-04-16T16:50:11.556 INFO:teuthology.orchestra.run.smithi045.stdout: postgresql-contrib x86_64 13.14-1.el9 CentOS-AppStream 817 k 2024-04-16T16:50:11.556 INFO:teuthology.orchestra.run.smithi045.stdout: postgresql-server x86_64 13.14-1.el9 CentOS-AppStream 5.7 M 2024-04-16T16:50:11.557 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cephfs x86_64 2:19.0.0-3101.g9b42bfbe.el9 ceph 162 k 2024-04-16T16:50:11.557 INFO:teuthology.orchestra.run.smithi045.stdout: python3-rados x86_64 2:19.0.0-3101.g9b42bfbe.el9 ceph 320 k 2024-04-16T16:50:11.557 INFO:teuthology.orchestra.run.smithi045.stdout: python3-rbd x86_64 2:19.0.0-3101.g9b42bfbe.el9 ceph 298 k 2024-04-16T16:50:11.557 INFO:teuthology.orchestra.run.smithi045.stdout: python3-rgw x86_64 2:19.0.0-3101.g9b42bfbe.el9 ceph 99 k 2024-04-16T16:50:11.557 INFO:teuthology.orchestra.run.smithi045.stdout: rbd-fuse x86_64 2:19.0.0-3101.g9b42bfbe.el9 ceph 85 k 2024-04-16T16:50:11.557 INFO:teuthology.orchestra.run.smithi045.stdout: rbd-mirror x86_64 2:19.0.0-3101.g9b42bfbe.el9 ceph 3.1 M 2024-04-16T16:50:11.557 INFO:teuthology.orchestra.run.smithi045.stdout: rbd-nbd x86_64 2:19.0.0-3101.g9b42bfbe.el9 ceph 170 k 2024-04-16T16:50:11.557 INFO:teuthology.orchestra.run.smithi045.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-16T16:50:11.557 INFO:teuthology.orchestra.run.smithi045.stdout:Upgrading: 2024-04-16T16:50:11.557 INFO:teuthology.orchestra.run.smithi045.stdout: NetworkManager x86_64 1:1.47.4-1.el9 CentOS-BaseOS 2.3 M 2024-04-16T16:50:11.557 INFO:teuthology.orchestra.run.smithi045.stdout: NetworkManager-libnm x86_64 1:1.47.4-1.el9 CentOS-BaseOS 1.8 M 2024-04-16T16:50:11.557 INFO:teuthology.orchestra.run.smithi045.stdout: NetworkManager-team x86_64 1:1.47.4-1.el9 CentOS-BaseOS 41 k 2024-04-16T16:50:11.557 INFO:teuthology.orchestra.run.smithi045.stdout: NetworkManager-tui x86_64 1:1.47.4-1.el9 CentOS-BaseOS 248 k 2024-04-16T16:50:11.557 INFO:teuthology.orchestra.run.smithi045.stdout: acl x86_64 2.3.1-4.el9 baseos 71 k 2024-04-16T16:50:11.557 INFO:teuthology.orchestra.run.smithi045.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 baseos 36 k 2024-04-16T16:50:11.558 INFO:teuthology.orchestra.run.smithi045.stdout: elfutils-libelf x86_64 0.190-2.el9 baseos 192 k 2024-04-16T16:50:11.558 INFO:teuthology.orchestra.run.smithi045.stdout: elfutils-libs x86_64 0.190-2.el9 baseos 256 k 2024-04-16T16:50:11.558 INFO:teuthology.orchestra.run.smithi045.stdout: libacl x86_64 2.3.1-4.el9 baseos 23 k 2024-04-16T16:50:11.558 INFO:teuthology.orchestra.run.smithi045.stdout: libacl-devel x86_64 2.3.1-4.el9 appstream 74 k 2024-04-16T16:50:11.558 INFO:teuthology.orchestra.run.smithi045.stdout: libblkid x86_64 2.37.4-18.el9 baseos 108 k 2024-04-16T16:50:11.558 INFO:teuthology.orchestra.run.smithi045.stdout: libfdisk x86_64 2.37.4-18.el9 baseos 155 k 2024-04-16T16:50:11.558 INFO:teuthology.orchestra.run.smithi045.stdout: libmount x86_64 2.37.4-18.el9 baseos 136 k 2024-04-16T16:50:11.558 INFO:teuthology.orchestra.run.smithi045.stdout: librados2 x86_64 2:19.0.0-3101.g9b42bfbe.el9 ceph 3.4 M 2024-04-16T16:50:11.558 INFO:teuthology.orchestra.run.smithi045.stdout: librbd1 x86_64 2:19.0.0-3101.g9b42bfbe.el9 ceph 3.1 M 2024-04-16T16:50:11.558 INFO:teuthology.orchestra.run.smithi045.stdout: libsmartcols x86_64 2.37.4-18.el9 baseos 64 k 2024-04-16T16:50:11.558 INFO:teuthology.orchestra.run.smithi045.stdout: libuuid x86_64 2.37.4-18.el9 baseos 29 k 2024-04-16T16:50:11.558 INFO:teuthology.orchestra.run.smithi045.stdout: openssl x86_64 1:3.0.7-27.el9 baseos 1.2 M 2024-04-16T16:50:11.558 INFO:teuthology.orchestra.run.smithi045.stdout: openssl-libs x86_64 1:3.0.7-27.el9 baseos 2.2 M 2024-04-16T16:50:11.559 INFO:teuthology.orchestra.run.smithi045.stdout: util-linux x86_64 2.37.4-18.el9 baseos 2.3 M 2024-04-16T16:50:11.559 INFO:teuthology.orchestra.run.smithi045.stdout: util-linux-core x86_64 2.37.4-18.el9 baseos 465 k 2024-04-16T16:50:11.559 INFO:teuthology.orchestra.run.smithi045.stdout: util-linux-user x86_64 2.37.4-18.el9 baseos 31 k 2024-04-16T16:50:11.559 INFO:teuthology.orchestra.run.smithi045.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2024-04-16T16:50:11.559 INFO:teuthology.orchestra.run.smithi045.stdout: xfsprogs x86_64 6.3.0-1.el9 baseos 1.1 M 2024-04-16T16:50:11.559 INFO:teuthology.orchestra.run.smithi045.stdout:Installing dependencies: 2024-04-16T16:50:11.559 INFO:teuthology.orchestra.run.smithi045.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-16T16:50:11.559 INFO:teuthology.orchestra.run.smithi045.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2024-04-16T16:50:11.559 INFO:teuthology.orchestra.run.smithi045.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-04-16T16:50:11.559 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-common x86_64 2:19.0.0-3101.g9b42bfbe.el9 ceph 21 M 2024-04-16T16:50:11.559 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3101.g9b42bfbe.el9 ceph-noarch 34 k 2024-04-16T16:50:11.559 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mds x86_64 2:19.0.0-3101.g9b42bfbe.el9 ceph 2.4 M 2024-04-16T16:50:11.559 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3101.g9b42bfbe.el9 ceph-noarch 249 k 2024-04-16T16:50:11.559 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mon x86_64 2:19.0.0-3101.g9b42bfbe.el9 ceph 4.7 M 2024-04-16T16:50:11.560 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-osd x86_64 2:19.0.0-3101.g9b42bfbe.el9 ceph 17 M 2024-04-16T16:50:11.560 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3101.g9b42bfbe.el9 ceph-noarch 16 k 2024-04-16T16:50:11.560 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-selinux x86_64 2:19.0.0-3101.g9b42bfbe.el9 ceph 25 k 2024-04-16T16:50:11.560 INFO:teuthology.orchestra.run.smithi045.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-04-16T16:50:11.560 INFO:teuthology.orchestra.run.smithi045.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-04-16T16:50:11.560 INFO:teuthology.orchestra.run.smithi045.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-04-16T16:50:11.560 INFO:teuthology.orchestra.run.smithi045.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-04-16T16:50:11.560 INFO:teuthology.orchestra.run.smithi045.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-16T16:50:11.560 INFO:teuthology.orchestra.run.smithi045.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-16T16:50:11.560 INFO:teuthology.orchestra.run.smithi045.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-04-16T16:50:11.560 INFO:teuthology.orchestra.run.smithi045.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-16T16:50:11.560 INFO:teuthology.orchestra.run.smithi045.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-16T16:50:11.560 INFO:teuthology.orchestra.run.smithi045.stdout: libcephsqlite x86_64 2:19.0.0-3101.g9b42bfbe.el9 ceph 163 k 2024-04-16T16:50:11.561 INFO:teuthology.orchestra.run.smithi045.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-16T16:50:11.561 INFO:teuthology.orchestra.run.smithi045.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-16T16:50:11.561 INFO:teuthology.orchestra.run.smithi045.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-04-16T16:50:11.561 INFO:teuthology.orchestra.run.smithi045.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-16T16:50:11.561 INFO:teuthology.orchestra.run.smithi045.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-04-16T16:50:11.561 INFO:teuthology.orchestra.run.smithi045.stdout: libradosstriper1 x86_64 2:19.0.0-3101.g9b42bfbe.el9 ceph 501 k 2024-04-16T16:50:11.561 INFO:teuthology.orchestra.run.smithi045.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-04-16T16:50:11.561 INFO:teuthology.orchestra.run.smithi045.stdout: librgw2 x86_64 2:19.0.0-3101.g9b42bfbe.el9 ceph 5.4 M 2024-04-16T16:50:11.561 INFO:teuthology.orchestra.run.smithi045.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-16T16:50:11.561 INFO:teuthology.orchestra.run.smithi045.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2024-04-16T16:50:11.561 INFO:teuthology.orchestra.run.smithi045.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-04-16T16:50:11.561 INFO:teuthology.orchestra.run.smithi045.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-04-16T16:50:11.561 INFO:teuthology.orchestra.run.smithi045.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-16T16:50:11.561 INFO:teuthology.orchestra.run.smithi045.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-16T16:50:11.562 INFO:teuthology.orchestra.run.smithi045.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2024-04-16T16:50:11.562 INFO:teuthology.orchestra.run.smithi045.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-04-16T16:50:11.562 INFO:teuthology.orchestra.run.smithi045.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-04-16T16:50:11.562 INFO:teuthology.orchestra.run.smithi045.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-16T16:50:11.562 INFO:teuthology.orchestra.run.smithi045.stdout: postgresql-private-libs x86_64 13.14-1.el9 CentOS-AppStream 132 k 2024-04-16T16:50:11.562 INFO:teuthology.orchestra.run.smithi045.stdout: protobuf x86_64 3.14.0-13.el9 CentOS-AppStream 1.0 M 2024-04-16T16:50:11.562 INFO:teuthology.orchestra.run.smithi045.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-16T16:50:11.562 INFO:teuthology.orchestra.run.smithi045.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-16T16:50:11.562 INFO:teuthology.orchestra.run.smithi045.stdout: python3-attrs noarch 20.3.0-7.el9 CentOS-AppStream 78 k 2024-04-16T16:50:11.562 INFO:teuthology.orchestra.run.smithi045.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-04-16T16:50:11.562 INFO:teuthology.orchestra.run.smithi045.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-16T16:50:11.562 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-16T16:50:11.562 INFO:teuthology.orchestra.run.smithi045.stdout: python3-ceph-argparse x86_64 2:19.0.0-3101.g9b42bfbe.el9 ceph 45 k 2024-04-16T16:50:11.563 INFO:teuthology.orchestra.run.smithi045.stdout: python3-ceph-common x86_64 2:19.0.0-3101.g9b42bfbe.el9 ceph 135 k 2024-04-16T16:50:11.563 INFO:teuthology.orchestra.run.smithi045.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-16T16:50:11.563 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-04-16T16:50:11.563 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-16T16:50:11.563 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-16T16:50:11.563 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-16T16:50:11.563 INFO:teuthology.orchestra.run.smithi045.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-16T16:50:11.563 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-04-16T16:50:11.563 INFO:teuthology.orchestra.run.smithi045.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-04-16T16:50:11.563 INFO:teuthology.orchestra.run.smithi045.stdout: python3-google-auth noarch 1:2.28.2-1.el9 epel 203 k 2024-04-16T16:50:11.563 INFO:teuthology.orchestra.run.smithi045.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-16T16:50:11.563 INFO:teuthology.orchestra.run.smithi045.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-16T16:50:11.563 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-16T16:50:11.563 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-16T16:50:11.564 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-16T16:50:11.564 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-16T16:50:11.564 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-16T16:50:11.564 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-16T16:50:11.564 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-04-16T16:50:11.564 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-04-16T16:50:11.564 INFO:teuthology.orchestra.run.smithi045.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-16T16:50:11.564 INFO:teuthology.orchestra.run.smithi045.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-16T16:50:11.564 INFO:teuthology.orchestra.run.smithi045.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-04-16T16:50:11.564 INFO:teuthology.orchestra.run.smithi045.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-04-16T16:50:11.564 INFO:teuthology.orchestra.run.smithi045.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-16T16:50:11.564 INFO:teuthology.orchestra.run.smithi045.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-16T16:50:11.564 INFO:teuthology.orchestra.run.smithi045.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-04-16T16:50:11.564 INFO:teuthology.orchestra.run.smithi045.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-04-16T16:50:11.565 INFO:teuthology.orchestra.run.smithi045.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-04-16T16:50:11.565 INFO:teuthology.orchestra.run.smithi045.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-04-16T16:50:11.565 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-16T16:50:11.565 INFO:teuthology.orchestra.run.smithi045.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-04-16T16:50:11.565 INFO:teuthology.orchestra.run.smithi045.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-16T16:50:11.565 INFO:teuthology.orchestra.run.smithi045.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-04-16T16:50:11.565 INFO:teuthology.orchestra.run.smithi045.stdout: python3-protobuf noarch 3.14.0-13.el9 CentOS-AppStream 238 k 2024-04-16T16:50:11.565 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-16T16:50:11.565 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-04-16T16:50:11.565 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-04-16T16:50:11.565 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-04-16T16:50:11.565 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-16T16:50:11.565 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-04-16T16:50:11.565 INFO:teuthology.orchestra.run.smithi045.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-16T16:50:11.565 INFO:teuthology.orchestra.run.smithi045.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-16T16:50:11.566 INFO:teuthology.orchestra.run.smithi045.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-04-16T16:50:11.566 INFO:teuthology.orchestra.run.smithi045.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-16T16:50:11.566 INFO:teuthology.orchestra.run.smithi045.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-16T16:50:11.566 INFO:teuthology.orchestra.run.smithi045.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-04-16T16:50:11.566 INFO:teuthology.orchestra.run.smithi045.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-16T16:50:11.566 INFO:teuthology.orchestra.run.smithi045.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-04-16T16:50:11.566 INFO:teuthology.orchestra.run.smithi045.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-16T16:50:11.566 INFO:teuthology.orchestra.run.smithi045.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-04-16T16:50:11.566 INFO:teuthology.orchestra.run.smithi045.stdout: python3-wcwidth noarch 0.2.5-8.el9 CentOS-AppStream 41 k 2024-04-16T16:50:11.566 INFO:teuthology.orchestra.run.smithi045.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-16T16:50:11.566 INFO:teuthology.orchestra.run.smithi045.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-16T16:50:11.566 INFO:teuthology.orchestra.run.smithi045.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-16T16:50:11.566 INFO:teuthology.orchestra.run.smithi045.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-16T16:50:11.566 INFO:teuthology.orchestra.run.smithi045.stdout: qatlib x86_64 24.02.0-1.el9 CentOS-AppStream 220 k 2024-04-16T16:50:11.567 INFO:teuthology.orchestra.run.smithi045.stdout: qatzip-libs x86_64 1.2.0-1.el9 CentOS-AppStream 46 k 2024-04-16T16:50:11.567 INFO:teuthology.orchestra.run.smithi045.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-16T16:50:11.567 INFO:teuthology.orchestra.run.smithi045.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-04-16T16:50:11.567 INFO:teuthology.orchestra.run.smithi045.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-16T16:50:11.567 INFO:teuthology.orchestra.run.smithi045.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2024-04-16T16:50:11.567 INFO:teuthology.orchestra.run.smithi045.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-04-16T16:50:11.567 INFO:teuthology.orchestra.run.smithi045.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2024-04-16T16:50:11.567 INFO:teuthology.orchestra.run.smithi045.stdout:Installing weak dependencies: 2024-04-16T16:50:11.567 INFO:teuthology.orchestra.run.smithi045.stdout: qatlib-service x86_64 24.02.0-1.el9 CentOS-AppStream 35 k 2024-04-16T16:50:11.567 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-16T16:50:11.567 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction Summary 2024-04-16T16:50:11.567 INFO:teuthology.orchestra.run.smithi045.stdout:============================================================================================ 2024-04-16T16:50:11.567 INFO:teuthology.orchestra.run.smithi045.stdout:Install 150 Packages 2024-04-16T16:50:11.567 INFO:teuthology.orchestra.run.smithi045.stdout:Upgrade 24 Packages 2024-04-16T16:50:11.568 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-16T16:50:11.568 INFO:teuthology.orchestra.run.smithi045.stdout:Total download size: 229 M 2024-04-16T16:50:11.568 INFO:teuthology.orchestra.run.smithi045.stdout:Downloading Packages: 2024-04-16T16:50:12.263 INFO:teuthology.orchestra.run.smithi117.stdout:(1/174): ceph-19.0.0-3101.g9b42bfbe.el9.x86_64. 25 kB/s | 6.6 kB 00:00 2024-04-16T16:50:12.796 INFO:teuthology.orchestra.run.smithi117.stdout:(2/174): ceph-fuse-19.0.0-3101.g9b42bfbe.el9.x8 1.6 MB/s | 890 kB 00:00 2024-04-16T16:50:13.042 INFO:teuthology.orchestra.run.smithi045.stdout:(1/174): ceph-19.0.0-3101.g9b42bfbe.el9.x86_64. 16 kB/s | 6.6 kB 00:00 2024-04-16T16:50:13.596 INFO:teuthology.orchestra.run.smithi117.stdout:(3/174): ceph-immutable-object-cache-19.0.0-310 181 kB/s | 145 kB 00:00 2024-04-16T16:50:13.737 INFO:teuthology.orchestra.run.smithi104.stdout:lab-extras 12 kB/s | 1.7 kB 00:00 2024-04-16T16:50:13.975 INFO:teuthology.orchestra.run.smithi045.stdout:(2/174): ceph-fuse-19.0.0-3101.g9b42bfbe.el9.x8 954 kB/s | 890 kB 00:00 2024-04-16T16:50:14.126 INFO:teuthology.orchestra.run.smithi045.stdout:(3/174): ceph-immutable-object-cache-19.0.0-310 972 kB/s | 145 kB 00:00 2024-04-16T16:50:14.947 INFO:teuthology.orchestra.run.smithi117.stdout:(4/174): ceph-base-19.0.0-3101.g9b42bfbe.el9.x8 1.8 MB/s | 5.4 MB 00:02 2024-04-16T16:50:15.747 INFO:teuthology.orchestra.run.smithi117.stdout:(5/174): ceph-mds-19.0.0-3101.g9b42bfbe.el9.x86 1.1 MB/s | 2.4 MB 00:02 2024-04-16T16:50:15.830 INFO:teuthology.orchestra.run.smithi117.stdout:(6/174): ceph-mgr-19.0.0-3101.g9b42bfbe.el9.x86 1.7 MB/s | 1.5 MB 00:00 2024-04-16T16:50:15.976 INFO:teuthology.orchestra.run.smithi045.stdout:(4/174): ceph-mds-19.0.0-3101.g9b42bfbe.el9.x86 1.3 MB/s | 2.4 MB 00:01 2024-04-16T16:50:16.474 INFO:teuthology.orchestra.run.smithi104.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-16T16:50:16.475 INFO:teuthology.orchestra.run.smithi104.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-16T16:50:16.487 INFO:teuthology.orchestra.run.smithi104.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-04-16T16:50:16.488 INFO:teuthology.orchestra.run.smithi104.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-16T16:50:16.489 INFO:teuthology.orchestra.run.smithi104.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-04-16T16:50:16.490 INFO:teuthology.orchestra.run.smithi104.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-04-16T16:50:16.492 INFO:teuthology.orchestra.run.smithi104.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-16T16:50:16.494 INFO:teuthology.orchestra.run.smithi104.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-04-16T16:50:16.495 INFO:teuthology.orchestra.run.smithi104.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-04-16T16:50:16.576 INFO:teuthology.orchestra.run.smithi104.stdout:Dependencies resolved. 2024-04-16T16:50:16.606 INFO:teuthology.orchestra.run.smithi104.stdout:============================================================================================ 2024-04-16T16:50:16.606 INFO:teuthology.orchestra.run.smithi104.stdout: Package Arch Version Repository Size 2024-04-16T16:50:16.606 INFO:teuthology.orchestra.run.smithi104.stdout:============================================================================================ 2024-04-16T16:50:16.606 INFO:teuthology.orchestra.run.smithi104.stdout:Installing: 2024-04-16T16:50:16.606 INFO:teuthology.orchestra.run.smithi104.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2024-04-16T16:50:16.606 INFO:teuthology.orchestra.run.smithi104.stdout: ceph x86_64 2:19.0.0-3101.g9b42bfbe.el9 ceph 6.6 k 2024-04-16T16:50:16.606 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-base x86_64 2:19.0.0-3101.g9b42bfbe.el9 ceph 5.4 M 2024-04-16T16:50:16.607 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-fuse x86_64 2:19.0.0-3101.g9b42bfbe.el9 ceph 890 k 2024-04-16T16:50:16.607 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3101.g9b42bfbe.el9 ceph 145 k 2024-04-16T16:50:16.607 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-mgr x86_64 2:19.0.0-3101.g9b42bfbe.el9 ceph 1.5 M 2024-04-16T16:50:16.607 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3101.g9b42bfbe.el9 ceph-noarch 142 k 2024-04-16T16:50:16.607 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3101.g9b42bfbe.el9 ceph-noarch 3.5 M 2024-04-16T16:50:16.607 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3101.g9b42bfbe.el9 ceph-noarch 7.4 M 2024-04-16T16:50:16.607 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-mgr-rook noarch 2:19.0.0-3101.g9b42bfbe.el9 ceph-noarch 49 k 2024-04-16T16:50:16.607 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-radosgw x86_64 2:19.0.0-3101.g9b42bfbe.el9 ceph 11 M 2024-04-16T16:50:16.607 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-test x86_64 2:19.0.0-3101.g9b42bfbe.el9 ceph 49 M 2024-04-16T16:50:16.607 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-volume noarch 2:19.0.0-3101.g9b42bfbe.el9 ceph-noarch 273 k 2024-04-16T16:50:16.607 INFO:teuthology.orchestra.run.smithi104.stdout: cephadm noarch 2:19.0.0-3101.g9b42bfbe.el9 ceph-noarch 762 k 2024-04-16T16:50:16.607 INFO:teuthology.orchestra.run.smithi104.stdout: cephfs-mirror x86_64 2:19.0.0-3101.g9b42bfbe.el9 ceph 225 k 2024-04-16T16:50:16.607 INFO:teuthology.orchestra.run.smithi104.stdout: cephfs-shell x86_64 2:19.0.0-3101.g9b42bfbe.el9 ceph 27 k 2024-04-16T16:50:16.608 INFO:teuthology.orchestra.run.smithi104.stdout: cephfs-top noarch 2:19.0.0-3101.g9b42bfbe.el9 ceph-noarch 18 k 2024-04-16T16:50:16.608 INFO:teuthology.orchestra.run.smithi104.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-16T16:50:16.608 INFO:teuthology.orchestra.run.smithi104.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2024-04-16T16:50:16.608 INFO:teuthology.orchestra.run.smithi104.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2024-04-16T16:50:16.608 INFO:teuthology.orchestra.run.smithi104.stdout: libcephfs-devel x86_64 2:19.0.0-3101.g9b42bfbe.el9 ceph 32 k 2024-04-16T16:50:16.608 INFO:teuthology.orchestra.run.smithi104.stdout: libcephfs2 x86_64 2:19.0.0-3101.g9b42bfbe.el9 ceph 747 k 2024-04-16T16:50:16.608 INFO:teuthology.orchestra.run.smithi104.stdout: librados-devel x86_64 2:19.0.0-3101.g9b42bfbe.el9 ceph 127 k 2024-04-16T16:50:16.608 INFO:teuthology.orchestra.run.smithi104.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2024-04-16T16:50:16.608 INFO:teuthology.orchestra.run.smithi104.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-16T16:50:16.608 INFO:teuthology.orchestra.run.smithi104.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-04-16T16:50:16.608 INFO:teuthology.orchestra.run.smithi104.stdout: postgresql x86_64 13.14-1.el9 CentOS-AppStream 1.5 M 2024-04-16T16:50:16.608 INFO:teuthology.orchestra.run.smithi104.stdout: postgresql-contrib x86_64 13.14-1.el9 CentOS-AppStream 817 k 2024-04-16T16:50:16.608 INFO:teuthology.orchestra.run.smithi104.stdout: postgresql-server x86_64 13.14-1.el9 CentOS-AppStream 5.7 M 2024-04-16T16:50:16.608 INFO:teuthology.orchestra.run.smithi104.stdout: python3-cephfs x86_64 2:19.0.0-3101.g9b42bfbe.el9 ceph 162 k 2024-04-16T16:50:16.609 INFO:teuthology.orchestra.run.smithi104.stdout: python3-rados x86_64 2:19.0.0-3101.g9b42bfbe.el9 ceph 320 k 2024-04-16T16:50:16.609 INFO:teuthology.orchestra.run.smithi104.stdout: python3-rbd x86_64 2:19.0.0-3101.g9b42bfbe.el9 ceph 298 k 2024-04-16T16:50:16.609 INFO:teuthology.orchestra.run.smithi104.stdout: python3-rgw x86_64 2:19.0.0-3101.g9b42bfbe.el9 ceph 99 k 2024-04-16T16:50:16.609 INFO:teuthology.orchestra.run.smithi104.stdout: rbd-fuse x86_64 2:19.0.0-3101.g9b42bfbe.el9 ceph 85 k 2024-04-16T16:50:16.609 INFO:teuthology.orchestra.run.smithi104.stdout: rbd-mirror x86_64 2:19.0.0-3101.g9b42bfbe.el9 ceph 3.1 M 2024-04-16T16:50:16.609 INFO:teuthology.orchestra.run.smithi104.stdout: rbd-nbd x86_64 2:19.0.0-3101.g9b42bfbe.el9 ceph 170 k 2024-04-16T16:50:16.609 INFO:teuthology.orchestra.run.smithi104.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-16T16:50:16.609 INFO:teuthology.orchestra.run.smithi104.stdout:Upgrading: 2024-04-16T16:50:16.609 INFO:teuthology.orchestra.run.smithi104.stdout: NetworkManager x86_64 1:1.47.4-1.el9 CentOS-BaseOS 2.3 M 2024-04-16T16:50:16.609 INFO:teuthology.orchestra.run.smithi104.stdout: NetworkManager-libnm x86_64 1:1.47.4-1.el9 CentOS-BaseOS 1.8 M 2024-04-16T16:50:16.609 INFO:teuthology.orchestra.run.smithi104.stdout: NetworkManager-team x86_64 1:1.47.4-1.el9 CentOS-BaseOS 41 k 2024-04-16T16:50:16.609 INFO:teuthology.orchestra.run.smithi104.stdout: NetworkManager-tui x86_64 1:1.47.4-1.el9 CentOS-BaseOS 248 k 2024-04-16T16:50:16.609 INFO:teuthology.orchestra.run.smithi104.stdout: acl x86_64 2.3.1-4.el9 baseos 71 k 2024-04-16T16:50:16.609 INFO:teuthology.orchestra.run.smithi104.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 baseos 36 k 2024-04-16T16:50:16.609 INFO:teuthology.orchestra.run.smithi104.stdout: elfutils-libelf x86_64 0.190-2.el9 baseos 192 k 2024-04-16T16:50:16.610 INFO:teuthology.orchestra.run.smithi104.stdout: elfutils-libs x86_64 0.190-2.el9 baseos 256 k 2024-04-16T16:50:16.610 INFO:teuthology.orchestra.run.smithi104.stdout: libacl x86_64 2.3.1-4.el9 baseos 23 k 2024-04-16T16:50:16.610 INFO:teuthology.orchestra.run.smithi104.stdout: libacl-devel x86_64 2.3.1-4.el9 appstream 74 k 2024-04-16T16:50:16.610 INFO:teuthology.orchestra.run.smithi104.stdout: libblkid x86_64 2.37.4-18.el9 baseos 108 k 2024-04-16T16:50:16.610 INFO:teuthology.orchestra.run.smithi104.stdout: libfdisk x86_64 2.37.4-18.el9 baseos 155 k 2024-04-16T16:50:16.610 INFO:teuthology.orchestra.run.smithi104.stdout: libmount x86_64 2.37.4-18.el9 baseos 136 k 2024-04-16T16:50:16.610 INFO:teuthology.orchestra.run.smithi104.stdout: librados2 x86_64 2:19.0.0-3101.g9b42bfbe.el9 ceph 3.4 M 2024-04-16T16:50:16.610 INFO:teuthology.orchestra.run.smithi104.stdout: librbd1 x86_64 2:19.0.0-3101.g9b42bfbe.el9 ceph 3.1 M 2024-04-16T16:50:16.610 INFO:teuthology.orchestra.run.smithi104.stdout: libsmartcols x86_64 2.37.4-18.el9 baseos 64 k 2024-04-16T16:50:16.610 INFO:teuthology.orchestra.run.smithi104.stdout: libuuid x86_64 2.37.4-18.el9 baseos 29 k 2024-04-16T16:50:16.610 INFO:teuthology.orchestra.run.smithi104.stdout: openssl x86_64 1:3.0.7-27.el9 baseos 1.2 M 2024-04-16T16:50:16.610 INFO:teuthology.orchestra.run.smithi104.stdout: openssl-libs x86_64 1:3.0.7-27.el9 baseos 2.2 M 2024-04-16T16:50:16.611 INFO:teuthology.orchestra.run.smithi104.stdout: util-linux x86_64 2.37.4-18.el9 baseos 2.3 M 2024-04-16T16:50:16.611 INFO:teuthology.orchestra.run.smithi104.stdout: util-linux-core x86_64 2.37.4-18.el9 baseos 465 k 2024-04-16T16:50:16.611 INFO:teuthology.orchestra.run.smithi104.stdout: util-linux-user x86_64 2.37.4-18.el9 baseos 31 k 2024-04-16T16:50:16.611 INFO:teuthology.orchestra.run.smithi104.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2024-04-16T16:50:16.611 INFO:teuthology.orchestra.run.smithi104.stdout: xfsprogs x86_64 6.3.0-1.el9 baseos 1.1 M 2024-04-16T16:50:16.611 INFO:teuthology.orchestra.run.smithi104.stdout:Installing dependencies: 2024-04-16T16:50:16.611 INFO:teuthology.orchestra.run.smithi104.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-16T16:50:16.611 INFO:teuthology.orchestra.run.smithi104.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2024-04-16T16:50:16.611 INFO:teuthology.orchestra.run.smithi104.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-04-16T16:50:16.611 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-common x86_64 2:19.0.0-3101.g9b42bfbe.el9 ceph 21 M 2024-04-16T16:50:16.611 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3101.g9b42bfbe.el9 ceph-noarch 34 k 2024-04-16T16:50:16.611 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-mds x86_64 2:19.0.0-3101.g9b42bfbe.el9 ceph 2.4 M 2024-04-16T16:50:16.611 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3101.g9b42bfbe.el9 ceph-noarch 249 k 2024-04-16T16:50:16.611 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-mon x86_64 2:19.0.0-3101.g9b42bfbe.el9 ceph 4.7 M 2024-04-16T16:50:16.612 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-osd x86_64 2:19.0.0-3101.g9b42bfbe.el9 ceph 17 M 2024-04-16T16:50:16.612 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3101.g9b42bfbe.el9 ceph-noarch 16 k 2024-04-16T16:50:16.612 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-selinux x86_64 2:19.0.0-3101.g9b42bfbe.el9 ceph 25 k 2024-04-16T16:50:16.612 INFO:teuthology.orchestra.run.smithi104.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-04-16T16:50:16.612 INFO:teuthology.orchestra.run.smithi104.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-04-16T16:50:16.612 INFO:teuthology.orchestra.run.smithi104.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-04-16T16:50:16.612 INFO:teuthology.orchestra.run.smithi104.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-04-16T16:50:16.612 INFO:teuthology.orchestra.run.smithi104.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-16T16:50:16.612 INFO:teuthology.orchestra.run.smithi104.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-16T16:50:16.612 INFO:teuthology.orchestra.run.smithi104.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-04-16T16:50:16.612 INFO:teuthology.orchestra.run.smithi104.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-16T16:50:16.612 INFO:teuthology.orchestra.run.smithi104.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-16T16:50:16.612 INFO:teuthology.orchestra.run.smithi104.stdout: libcephsqlite x86_64 2:19.0.0-3101.g9b42bfbe.el9 ceph 163 k 2024-04-16T16:50:16.612 INFO:teuthology.orchestra.run.smithi104.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-16T16:50:16.612 INFO:teuthology.orchestra.run.smithi104.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-16T16:50:16.613 INFO:teuthology.orchestra.run.smithi104.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-04-16T16:50:16.613 INFO:teuthology.orchestra.run.smithi104.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-16T16:50:16.613 INFO:teuthology.orchestra.run.smithi104.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-04-16T16:50:16.613 INFO:teuthology.orchestra.run.smithi104.stdout: libradosstriper1 x86_64 2:19.0.0-3101.g9b42bfbe.el9 ceph 501 k 2024-04-16T16:50:16.613 INFO:teuthology.orchestra.run.smithi104.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-04-16T16:50:16.613 INFO:teuthology.orchestra.run.smithi104.stdout: librgw2 x86_64 2:19.0.0-3101.g9b42bfbe.el9 ceph 5.4 M 2024-04-16T16:50:16.613 INFO:teuthology.orchestra.run.smithi104.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-16T16:50:16.613 INFO:teuthology.orchestra.run.smithi104.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2024-04-16T16:50:16.613 INFO:teuthology.orchestra.run.smithi104.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-04-16T16:50:16.613 INFO:teuthology.orchestra.run.smithi104.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-04-16T16:50:16.613 INFO:teuthology.orchestra.run.smithi104.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-16T16:50:16.613 INFO:teuthology.orchestra.run.smithi104.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-16T16:50:16.613 INFO:teuthology.orchestra.run.smithi104.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2024-04-16T16:50:16.613 INFO:teuthology.orchestra.run.smithi104.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-04-16T16:50:16.613 INFO:teuthology.orchestra.run.smithi104.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-04-16T16:50:16.614 INFO:teuthology.orchestra.run.smithi104.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-16T16:50:16.614 INFO:teuthology.orchestra.run.smithi104.stdout: postgresql-private-libs x86_64 13.14-1.el9 CentOS-AppStream 132 k 2024-04-16T16:50:16.614 INFO:teuthology.orchestra.run.smithi104.stdout: protobuf x86_64 3.14.0-13.el9 CentOS-AppStream 1.0 M 2024-04-16T16:50:16.614 INFO:teuthology.orchestra.run.smithi104.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-16T16:50:16.614 INFO:teuthology.orchestra.run.smithi104.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-16T16:50:16.614 INFO:teuthology.orchestra.run.smithi104.stdout: python3-attrs noarch 20.3.0-7.el9 CentOS-AppStream 78 k 2024-04-16T16:50:16.614 INFO:teuthology.orchestra.run.smithi104.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-04-16T16:50:16.614 INFO:teuthology.orchestra.run.smithi104.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-16T16:50:16.614 INFO:teuthology.orchestra.run.smithi104.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-16T16:50:16.614 INFO:teuthology.orchestra.run.smithi104.stdout: python3-ceph-argparse x86_64 2:19.0.0-3101.g9b42bfbe.el9 ceph 45 k 2024-04-16T16:50:16.614 INFO:teuthology.orchestra.run.smithi104.stdout: python3-ceph-common x86_64 2:19.0.0-3101.g9b42bfbe.el9 ceph 135 k 2024-04-16T16:50:16.614 INFO:teuthology.orchestra.run.smithi104.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-16T16:50:16.614 INFO:teuthology.orchestra.run.smithi104.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-04-16T16:50:16.614 INFO:teuthology.orchestra.run.smithi104.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-16T16:50:16.615 INFO:teuthology.orchestra.run.smithi104.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-16T16:50:16.615 INFO:teuthology.orchestra.run.smithi104.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-16T16:50:16.615 INFO:teuthology.orchestra.run.smithi104.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-16T16:50:16.615 INFO:teuthology.orchestra.run.smithi104.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-04-16T16:50:16.615 INFO:teuthology.orchestra.run.smithi104.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-04-16T16:50:16.615 INFO:teuthology.orchestra.run.smithi104.stdout: python3-google-auth noarch 1:2.28.2-1.el9 epel 203 k 2024-04-16T16:50:16.615 INFO:teuthology.orchestra.run.smithi104.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-16T16:50:16.615 INFO:teuthology.orchestra.run.smithi104.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-16T16:50:16.615 INFO:teuthology.orchestra.run.smithi104.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-16T16:50:16.615 INFO:teuthology.orchestra.run.smithi104.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-16T16:50:16.615 INFO:teuthology.orchestra.run.smithi104.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-16T16:50:16.615 INFO:teuthology.orchestra.run.smithi104.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-16T16:50:16.615 INFO:teuthology.orchestra.run.smithi104.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-16T16:50:16.616 INFO:teuthology.orchestra.run.smithi104.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-16T16:50:16.616 INFO:teuthology.orchestra.run.smithi104.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-04-16T16:50:16.616 INFO:teuthology.orchestra.run.smithi104.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-04-16T16:50:16.616 INFO:teuthology.orchestra.run.smithi104.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-16T16:50:16.616 INFO:teuthology.orchestra.run.smithi104.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-16T16:50:16.616 INFO:teuthology.orchestra.run.smithi104.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-04-16T16:50:16.616 INFO:teuthology.orchestra.run.smithi104.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-04-16T16:50:16.616 INFO:teuthology.orchestra.run.smithi104.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-16T16:50:16.616 INFO:teuthology.orchestra.run.smithi104.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-16T16:50:16.616 INFO:teuthology.orchestra.run.smithi104.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-04-16T16:50:16.616 INFO:teuthology.orchestra.run.smithi104.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-04-16T16:50:16.616 INFO:teuthology.orchestra.run.smithi104.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-04-16T16:50:16.616 INFO:teuthology.orchestra.run.smithi104.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-04-16T16:50:16.616 INFO:teuthology.orchestra.run.smithi104.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-16T16:50:16.617 INFO:teuthology.orchestra.run.smithi104.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-04-16T16:50:16.617 INFO:teuthology.orchestra.run.smithi104.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-16T16:50:16.617 INFO:teuthology.orchestra.run.smithi104.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-04-16T16:50:16.617 INFO:teuthology.orchestra.run.smithi104.stdout: python3-protobuf noarch 3.14.0-13.el9 CentOS-AppStream 238 k 2024-04-16T16:50:16.617 INFO:teuthology.orchestra.run.smithi104.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-16T16:50:16.617 INFO:teuthology.orchestra.run.smithi104.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-04-16T16:50:16.617 INFO:teuthology.orchestra.run.smithi104.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-04-16T16:50:16.617 INFO:teuthology.orchestra.run.smithi104.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-04-16T16:50:16.617 INFO:teuthology.orchestra.run.smithi104.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-16T16:50:16.617 INFO:teuthology.orchestra.run.smithi104.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-04-16T16:50:16.617 INFO:teuthology.orchestra.run.smithi104.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-16T16:50:16.617 INFO:teuthology.orchestra.run.smithi104.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-16T16:50:16.617 INFO:teuthology.orchestra.run.smithi104.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-04-16T16:50:16.617 INFO:teuthology.orchestra.run.smithi104.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-16T16:50:16.618 INFO:teuthology.orchestra.run.smithi104.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-16T16:50:16.618 INFO:teuthology.orchestra.run.smithi104.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-04-16T16:50:16.618 INFO:teuthology.orchestra.run.smithi104.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-16T16:50:16.618 INFO:teuthology.orchestra.run.smithi104.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-04-16T16:50:16.618 INFO:teuthology.orchestra.run.smithi104.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-16T16:50:16.618 INFO:teuthology.orchestra.run.smithi104.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-04-16T16:50:16.618 INFO:teuthology.orchestra.run.smithi104.stdout: python3-wcwidth noarch 0.2.5-8.el9 CentOS-AppStream 41 k 2024-04-16T16:50:16.618 INFO:teuthology.orchestra.run.smithi104.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-16T16:50:16.618 INFO:teuthology.orchestra.run.smithi104.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-16T16:50:16.618 INFO:teuthology.orchestra.run.smithi104.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-16T16:50:16.618 INFO:teuthology.orchestra.run.smithi104.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-16T16:50:16.618 INFO:teuthology.orchestra.run.smithi104.stdout: qatlib x86_64 24.02.0-1.el9 CentOS-AppStream 220 k 2024-04-16T16:50:16.618 INFO:teuthology.orchestra.run.smithi104.stdout: qatzip-libs x86_64 1.2.0-1.el9 CentOS-AppStream 46 k 2024-04-16T16:50:16.618 INFO:teuthology.orchestra.run.smithi104.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-16T16:50:16.619 INFO:teuthology.orchestra.run.smithi104.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-04-16T16:50:16.619 INFO:teuthology.orchestra.run.smithi104.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-16T16:50:16.619 INFO:teuthology.orchestra.run.smithi104.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2024-04-16T16:50:16.619 INFO:teuthology.orchestra.run.smithi104.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-04-16T16:50:16.619 INFO:teuthology.orchestra.run.smithi104.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2024-04-16T16:50:16.619 INFO:teuthology.orchestra.run.smithi104.stdout:Installing weak dependencies: 2024-04-16T16:50:16.619 INFO:teuthology.orchestra.run.smithi104.stdout: qatlib-service x86_64 24.02.0-1.el9 CentOS-AppStream 35 k 2024-04-16T16:50:16.619 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-04-16T16:50:16.619 INFO:teuthology.orchestra.run.smithi104.stdout:Transaction Summary 2024-04-16T16:50:16.619 INFO:teuthology.orchestra.run.smithi104.stdout:============================================================================================ 2024-04-16T16:50:16.619 INFO:teuthology.orchestra.run.smithi104.stdout:Install 150 Packages 2024-04-16T16:50:16.619 INFO:teuthology.orchestra.run.smithi104.stdout:Upgrade 24 Packages 2024-04-16T16:50:16.619 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-04-16T16:50:16.620 INFO:teuthology.orchestra.run.smithi104.stdout:Total download size: 229 M 2024-04-16T16:50:16.620 INFO:teuthology.orchestra.run.smithi104.stdout:Downloading Packages: 2024-04-16T16:50:17.555 INFO:teuthology.orchestra.run.smithi104.stdout:(1/174): ceph-19.0.0-3101.g9b42bfbe.el9.x86_64. 18 kB/s | 6.6 kB 00:00 2024-04-16T16:50:17.726 INFO:teuthology.orchestra.run.smithi045.stdout:(5/174): ceph-mgr-19.0.0-3101.g9b42bfbe.el9.x86 875 kB/s | 1.5 MB 00:01 2024-04-16T16:50:18.238 INFO:teuthology.orchestra.run.smithi104.stdout:(2/174): ceph-fuse-19.0.0-3101.g9b42bfbe.el9.x8 1.3 MB/s | 890 kB 00:00 2024-04-16T16:50:18.405 INFO:teuthology.orchestra.run.smithi104.stdout:(3/174): ceph-immutable-object-cache-19.0.0-310 869 kB/s | 145 kB 00:00 2024-04-16T16:50:18.476 INFO:teuthology.orchestra.run.smithi045.stdout:(6/174): ceph-base-19.0.0-3101.g9b42bfbe.el9.x8 941 kB/s | 5.4 MB 00:05 2024-04-16T16:50:19.805 INFO:teuthology.orchestra.run.smithi104.stdout:(4/174): ceph-mds-19.0.0-3101.g9b42bfbe.el9.x86 1.7 MB/s | 2.4 MB 00:01 2024-04-16T16:50:20.230 INFO:teuthology.orchestra.run.smithi117.stdout:(7/174): ceph-mon-19.0.0-3101.g9b42bfbe.el9.x86 1.0 MB/s | 4.7 MB 00:04 2024-04-16T16:50:20.797 INFO:teuthology.orchestra.run.smithi104.stdout:(5/174): ceph-mgr-19.0.0-3101.g9b42bfbe.el9.x86 1.5 MB/s | 1.5 MB 00:00 2024-04-16T16:50:21.473 INFO:teuthology.orchestra.run.smithi104.stdout:(6/174): ceph-base-19.0.0-3101.g9b42bfbe.el9.x8 1.3 MB/s | 5.4 MB 00:04 2024-04-16T16:50:22.701 INFO:teuthology.orchestra.run.smithi045.stdout:(7/174): ceph-common-19.0.0-3101.g9b42bfbe.el9. 2.1 MB/s | 21 MB 00:10 2024-04-16T16:50:23.251 INFO:teuthology.orchestra.run.smithi045.stdout:(8/174): ceph-mon-19.0.0-3101.g9b42bfbe.el9.x86 871 kB/s | 4.7 MB 00:05 2024-04-16T16:50:23.351 INFO:teuthology.orchestra.run.smithi045.stdout:(9/174): ceph-selinux-19.0.0-3101.g9b42bfbe.el9 254 kB/s | 25 kB 00:00 2024-04-16T16:50:23.481 INFO:teuthology.orchestra.run.smithi104.stdout:(7/174): ceph-mon-19.0.0-3101.g9b42bfbe.el9.x86 1.8 MB/s | 4.7 MB 00:02 2024-04-16T16:50:28.129 INFO:teuthology.orchestra.run.smithi117.stdout:(8/174): ceph-common-19.0.0-3101.g9b42bfbe.el9. 1.3 MB/s | 21 MB 00:16 2024-04-16T16:50:28.237 INFO:teuthology.orchestra.run.smithi117.stdout:(9/174): ceph-selinux-19.0.0-3101.g9b42bfbe.el9 233 kB/s | 25 kB 00:00 2024-04-16T16:50:31.213 INFO:teuthology.orchestra.run.smithi104.stdout:(8/174): ceph-radosgw-19.0.0-3101.g9b42bfbe.el9 1.4 MB/s | 11 MB 00:07 2024-04-16T16:50:31.338 INFO:teuthology.orchestra.run.smithi104.stdout:(9/174): ceph-selinux-19.0.0-3101.g9b42bfbe.el9 202 kB/s | 25 kB 00:00 2024-04-16T16:50:31.453 INFO:teuthology.orchestra.run.smithi117.stdout:(10/174): ceph-radosgw-19.0.0-3101.g9b42bfbe.el 967 kB/s | 11 MB 00:11 2024-04-16T16:50:31.929 INFO:teuthology.orchestra.run.smithi117.stdout:(11/174): cephfs-mirror-19.0.0-3101.g9b42bfbe.e 472 kB/s | 225 kB 00:00 2024-04-16T16:50:32.021 INFO:teuthology.orchestra.run.smithi117.stdout:(12/174): cephfs-shell-19.0.0-3101.g9b42bfbe.el 295 kB/s | 27 kB 00:00 2024-04-16T16:50:32.113 INFO:teuthology.orchestra.run.smithi117.stdout:(13/174): libcephfs-devel-19.0.0-3101.g9b42bfbe 346 kB/s | 32 kB 00:00 2024-04-16T16:50:33.392 INFO:teuthology.orchestra.run.smithi045.stdout:(10/174): ceph-radosgw-19.0.0-3101.g9b42bfbe.el 1.0 MB/s | 11 MB 00:10 2024-04-16T16:50:33.496 INFO:teuthology.orchestra.run.smithi117.stdout:(14/174): libcephfs2-19.0.0-3101.g9b42bfbe.el9. 540 kB/s | 747 kB 00:01 2024-04-16T16:50:33.709 INFO:teuthology.orchestra.run.smithi045.stdout:(11/174): cephfs-mirror-19.0.0-3101.g9b42bfbe.e 712 kB/s | 225 kB 00:00 2024-04-16T16:50:33.843 INFO:teuthology.orchestra.run.smithi045.stdout:(12/174): cephfs-shell-19.0.0-3101.g9b42bfbe.el 204 kB/s | 27 kB 00:00 2024-04-16T16:50:33.880 INFO:teuthology.orchestra.run.smithi117.stdout:(15/174): libcephsqlite-19.0.0-3101.g9b42bfbe.e 427 kB/s | 163 kB 00:00 2024-04-16T16:50:34.105 INFO:teuthology.orchestra.run.smithi117.stdout:(16/174): librados-devel-19.0.0-3101.g9b42bfbe. 564 kB/s | 127 kB 00:00 2024-04-16T16:50:34.118 INFO:teuthology.orchestra.run.smithi045.stdout:(13/174): ceph-osd-19.0.0-3101.g9b42bfbe.el9.x8 1.1 MB/s | 17 MB 00:15 2024-04-16T16:50:34.143 INFO:teuthology.orchestra.run.smithi045.stdout:(14/174): libcephfs-devel-19.0.0-3101.g9b42bfbe 106 kB/s | 32 kB 00:00 2024-04-16T16:50:34.377 INFO:teuthology.orchestra.run.smithi045.stdout:(15/174): libcephsqlite-19.0.0-3101.g9b42bfbe.e 704 kB/s | 163 kB 00:00 2024-04-16T16:50:34.594 INFO:teuthology.orchestra.run.smithi045.stdout:(16/174): librados-devel-19.0.0-3101.g9b42bfbe. 587 kB/s | 127 kB 00:00 2024-04-16T16:50:34.802 INFO:teuthology.orchestra.run.smithi045.stdout:(17/174): libcephfs2-19.0.0-3101.g9b42bfbe.el9. 1.1 MB/s | 747 kB 00:00 2024-04-16T16:50:34.922 INFO:teuthology.orchestra.run.smithi117.stdout:(17/174): libradosstriper1-19.0.0-3101.g9b42bfb 614 kB/s | 501 kB 00:00 2024-04-16T16:50:35.211 INFO:teuthology.orchestra.run.smithi045.stdout:(18/174): libradosstriper1-19.0.0-3101.g9b42bfb 814 kB/s | 501 kB 00:00 2024-04-16T16:50:35.336 INFO:teuthology.orchestra.run.smithi045.stdout:(19/174): python3-ceph-argparse-19.0.0-3101.g9b 363 kB/s | 45 kB 00:00 2024-04-16T16:50:35.553 INFO:teuthology.orchestra.run.smithi045.stdout:(20/174): python3-ceph-common-19.0.0-3101.g9b42 625 kB/s | 135 kB 00:00 2024-04-16T16:50:36.545 INFO:teuthology.orchestra.run.smithi045.stdout:(21/174): python3-cephfs-19.0.0-3101.g9b42bfbe. 163 kB/s | 162 kB 00:00 2024-04-16T16:50:37.262 INFO:teuthology.orchestra.run.smithi045.stdout:(22/174): python3-rados-19.0.0-3101.g9b42bfbe.e 446 kB/s | 320 kB 00:00 2024-04-16T16:50:37.363 INFO:teuthology.orchestra.run.smithi104.stdout:(10/174): ceph-common-19.0.0-3101.g9b42bfbe.el9 1.0 MB/s | 21 MB 00:20 2024-04-16T16:50:37.796 INFO:teuthology.orchestra.run.smithi045.stdout:(23/174): python3-rbd-19.0.0-3101.g9b42bfbe.el9 559 kB/s | 298 kB 00:00 2024-04-16T16:50:37.804 INFO:teuthology.orchestra.run.smithi104.stdout:(11/174): cephfs-mirror-19.0.0-3101.g9b42bfbe.e 509 kB/s | 225 kB 00:00 2024-04-16T16:50:37.896 INFO:teuthology.orchestra.run.smithi104.stdout:(12/174): cephfs-shell-19.0.0-3101.g9b42bfbe.el 295 kB/s | 27 kB 00:00 2024-04-16T16:50:37.980 INFO:teuthology.orchestra.run.smithi104.stdout:(13/174): libcephfs-devel-19.0.0-3101.g9b42bfbe 381 kB/s | 32 kB 00:00 2024-04-16T16:50:37.988 INFO:teuthology.orchestra.run.smithi045.stdout:(24/174): librgw2-19.0.0-3101.g9b42bfbe.el9.x86 1.7 MB/s | 5.4 MB 00:03 2024-04-16T16:50:38.055 INFO:teuthology.orchestra.run.smithi045.stdout:(25/174): python3-rgw-19.0.0-3101.g9b42bfbe.el9 385 kB/s | 99 kB 00:00 2024-04-16T16:50:38.080 INFO:teuthology.orchestra.run.smithi045.stdout:(26/174): rbd-fuse-19.0.0-3101.g9b42bfbe.el9.x8 941 kB/s | 85 kB 00:00 2024-04-16T16:50:38.189 INFO:teuthology.orchestra.run.smithi045.stdout:(27/174): rbd-nbd-19.0.0-3101.g9b42bfbe.el9.x86 1.5 MB/s | 170 kB 00:00 2024-04-16T16:50:38.281 INFO:teuthology.orchestra.run.smithi045.stdout:(28/174): ceph-grafana-dashboards-19.0.0-3101.g 377 kB/s | 34 kB 00:00 2024-04-16T16:50:38.389 INFO:teuthology.orchestra.run.smithi045.stdout:(29/174): ceph-mgr-cephadm-19.0.0-3101.g9b42bfb 1.3 MB/s | 142 kB 00:00 2024-04-16T16:50:38.930 INFO:teuthology.orchestra.run.smithi104.stdout:(14/174): libcephfs2-19.0.0-3101.g9b42bfbe.el9. 786 kB/s | 747 kB 00:00 2024-04-16T16:50:39.188 INFO:teuthology.orchestra.run.smithi104.stdout:(15/174): libcephsqlite-19.0.0-3101.g9b42bfbe.e 633 kB/s | 163 kB 00:00 2024-04-16T16:50:39.364 INFO:teuthology.orchestra.run.smithi104.stdout:(16/174): librados-devel-19.0.0-3101.g9b42bfbe. 725 kB/s | 127 kB 00:00 2024-04-16T16:50:39.656 INFO:teuthology.orchestra.run.smithi045.stdout:(30/174): ceph-mgr-dashboard-19.0.0-3101.g9b42b 2.8 MB/s | 3.5 MB 00:01 2024-04-16T16:50:40.001 INFO:teuthology.orchestra.run.smithi104.stdout:(17/174): libradosstriper1-19.0.0-3101.g9b42bfb 788 kB/s | 501 kB 00:00 2024-04-16T16:50:42.548 INFO:teuthology.orchestra.run.smithi045.stdout:(31/174): ceph-mgr-diskprediction-local-19.0.0- 2.6 MB/s | 7.4 MB 00:02 2024-04-16T16:50:42.748 INFO:teuthology.orchestra.run.smithi045.stdout:(32/174): ceph-mgr-modules-core-19.0.0-3101.g9b 1.2 MB/s | 249 kB 00:00 2024-04-16T16:50:42.848 INFO:teuthology.orchestra.run.smithi045.stdout:(33/174): ceph-mgr-rook-19.0.0-3101.g9b42bfbe.e 498 kB/s | 49 kB 00:00 2024-04-16T16:50:42.865 INFO:teuthology.orchestra.run.smithi117.stdout:(18/174): librgw2-19.0.0-3101.g9b42bfbe.el9.x86 691 kB/s | 5.4 MB 00:07 2024-04-16T16:50:42.940 INFO:teuthology.orchestra.run.smithi045.stdout:(34/174): ceph-prometheus-alerts-19.0.0-3101.g9 181 kB/s | 16 kB 00:00 2024-04-16T16:50:42.956 INFO:teuthology.orchestra.run.smithi117.stdout:(19/174): python3-ceph-argparse-19.0.0-3101.g9b 494 kB/s | 45 kB 00:00 2024-04-16T16:50:43.049 INFO:teuthology.orchestra.run.smithi045.stdout:(35/174): rbd-mirror-19.0.0-3101.g9b42bfbe.el9. 634 kB/s | 3.1 MB 00:04 2024-04-16T16:50:43.166 INFO:teuthology.orchestra.run.smithi045.stdout:(36/174): ceph-volume-19.0.0-3101.g9b42bfbe.el9 1.2 MB/s | 273 kB 00:00 2024-04-16T16:50:43.266 INFO:teuthology.orchestra.run.smithi045.stdout:(37/174): cephfs-top-19.0.0-3101.g9b42bfbe.el9. 184 kB/s | 18 kB 00:00 2024-04-16T16:50:43.349 INFO:teuthology.orchestra.run.smithi117.stdout:(20/174): python3-ceph-common-19.0.0-3101.g9b42 344 kB/s | 135 kB 00:00 2024-04-16T16:50:43.455 INFO:teuthology.orchestra.run.smithi104.stdout:(18/174): ceph-osd-19.0.0-3101.g9b42bfbe.el9.x8 784 kB/s | 17 MB 00:21 2024-04-16T16:50:43.467 INFO:teuthology.orchestra.run.smithi045.stdout:(38/174): automake-1.16.2-8.el9.noarch.rpm 3.2 MB/s | 662 kB 00:00 2024-04-16T16:50:43.551 INFO:teuthology.orchestra.run.smithi045.stdout:(39/174): bison-3.7.4-5.el9.x86_64.rpm 11 MB/s | 920 kB 00:00 2024-04-16T16:50:43.592 INFO:teuthology.orchestra.run.smithi045.stdout:(40/174): boost-program-options-1.75.0-8.el9.x8 2.6 MB/s | 106 kB 00:00 2024-04-16T16:50:43.630 INFO:teuthology.orchestra.run.smithi104.stdout:(19/174): python3-ceph-argparse-19.0.0-3101.g9b 258 kB/s | 45 kB 00:00 2024-04-16T16:50:43.634 INFO:teuthology.orchestra.run.smithi045.stdout:(41/174): flex-2.6.4-9.el9.x86_64.rpm 7.3 MB/s | 307 kB 00:00 2024-04-16T16:50:43.641 INFO:teuthology.orchestra.run.smithi117.stdout:(21/174): python3-cephfs-19.0.0-3101.g9b42bfbe. 553 kB/s | 162 kB 00:00 2024-04-16T16:50:43.668 INFO:teuthology.orchestra.run.smithi045.stdout:(42/174): flexiblas-3.0.4-8.el9.x86_64.rpm 974 kB/s | 32 kB 00:00 2024-04-16T16:50:43.777 INFO:teuthology.orchestra.run.smithi045.stdout:(43/174): flexiblas-netlib-3.0.4-8.el9.x86_64.r 28 MB/s | 3.0 MB 00:00 2024-04-16T16:50:43.810 INFO:teuthology.orchestra.run.smithi045.stdout:(44/174): flexiblas-openblas-openmp-3.0.4-8.el9 532 kB/s | 17 kB 00:00 2024-04-16T16:50:43.822 INFO:teuthology.orchestra.run.smithi104.stdout:(20/174): python3-ceph-common-19.0.0-3101.g9b42 703 kB/s | 135 kB 00:00 2024-04-16T16:50:43.844 INFO:teuthology.orchestra.run.smithi045.stdout:(45/174): libaio-devel-0.3.111-13.el9.x86_64.rp 308 kB/s | 10 kB 00:00 2024-04-16T16:50:43.916 INFO:teuthology.orchestra.run.smithi045.stdout:(46/174): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.2 MB/s | 159 kB 00:00 2024-04-16T16:50:43.961 INFO:teuthology.orchestra.run.smithi045.stdout:(47/174): librabbitmq-0.11.0-7.el9.x86_64.rpm 1.0 MB/s | 44 kB 00:00 2024-04-16T16:50:44.012 INFO:teuthology.orchestra.run.smithi045.stdout:(48/174): librdkafka-1.6.1-102.el9.x86_64.rpm 13 MB/s | 661 kB 00:00 2024-04-16T16:50:44.062 INFO:teuthology.orchestra.run.smithi045.stdout:(49/174): libtool-2.4.6-45.el9.x86_64.rpm 12 MB/s | 578 kB 00:00 2024-04-16T16:50:44.087 INFO:teuthology.orchestra.run.smithi045.stdout:(50/174): libzstd-devel-1.5.1-2.el9.x86_64.rpm 1.9 MB/s | 46 kB 00:00 2024-04-16T16:50:44.126 INFO:teuthology.orchestra.run.smithi104.stdout:(21/174): python3-cephfs-19.0.0-3101.g9b42bfbe. 533 kB/s | 162 kB 00:00 2024-04-16T16:50:44.129 INFO:teuthology.orchestra.run.smithi045.stdout:(51/174): lttng-ust-2.12.0-6.el9.x86_64.rpm 6.8 MB/s | 282 kB 00:00 2024-04-16T16:50:44.171 INFO:teuthology.orchestra.run.smithi045.stdout:(52/174): lua-5.4.4-4.el9.x86_64.rpm 4.5 MB/s | 187 kB 00:00 2024-04-16T16:50:44.213 INFO:teuthology.orchestra.run.smithi045.stdout:(53/174): oniguruma-6.9.6-1.el9.5.x86_64.rpm 5.2 MB/s | 217 kB 00:00 2024-04-16T16:50:44.233 INFO:teuthology.orchestra.run.smithi117.stdout:(22/174): python3-rados-19.0.0-3101.g9b42bfbe.e 540 kB/s | 320 kB 00:00 2024-04-16T16:50:44.246 INFO:teuthology.orchestra.run.smithi045.stdout:(54/174): openblas-0.3.21-2.el9.x86_64.rpm 1.0 MB/s | 34 kB 00:00 2024-04-16T16:50:44.397 INFO:teuthology.orchestra.run.smithi045.stdout:(55/174): openblas-openmp-0.3.21-2.el9.x86_64.r 32 MB/s | 4.7 MB 00:00 2024-04-16T16:50:44.481 INFO:teuthology.orchestra.run.smithi045.stdout:(56/174): postgresql-13.14-1.el9.x86_64.rpm 19 MB/s | 1.5 MB 00:00 2024-04-16T16:50:44.522 INFO:teuthology.orchestra.run.smithi045.stdout:(57/174): cephadm-19.0.0-3101.g9b42bfbe.el9.noa 518 kB/s | 762 kB 00:01 2024-04-16T16:50:44.573 INFO:teuthology.orchestra.run.smithi045.stdout:(58/174): postgresql-contrib-13.14-1.el9.x86_64 8.8 MB/s | 817 kB 00:00 2024-04-16T16:50:44.598 INFO:teuthology.orchestra.run.smithi045.stdout:(59/174): postgresql-private-libs-13.14-1.el9.x 1.9 MB/s | 132 kB 00:00 2024-04-16T16:50:44.601 INFO:teuthology.orchestra.run.smithi104.stdout:(22/174): python3-rados-19.0.0-3101.g9b42bfbe.e 673 kB/s | 320 kB 00:00 2024-04-16T16:50:44.691 INFO:teuthology.orchestra.run.smithi117.stdout:(23/174): python3-rbd-19.0.0-3101.g9b42bfbe.el9 650 kB/s | 298 kB 00:00 2024-04-16T16:50:44.765 INFO:teuthology.orchestra.run.smithi045.stdout:(60/174): postgresql-server-13.14-1.el9.x86_64. 30 MB/s | 5.7 MB 00:00 2024-04-16T16:50:44.806 INFO:teuthology.orchestra.run.smithi045.stdout:(61/174): protobuf-3.14.0-13.el9.x86_64.rpm 5.3 MB/s | 1.0 MB 00:00 2024-04-16T16:50:44.865 INFO:teuthology.orchestra.run.smithi045.stdout:(62/174): python3-attrs-20.3.0-7.el9.noarch.rpm 1.3 MB/s | 78 kB 00:00 2024-04-16T16:50:44.866 INFO:teuthology.orchestra.run.smithi117.stdout:(24/174): python3-rgw-19.0.0-3101.g9b42bfbe.el9 568 kB/s | 99 kB 00:00 2024-04-16T16:50:44.974 INFO:teuthology.orchestra.run.smithi045.stdout:(63/174): python3-babel-2.9.1-2.el9.noarch.rpm 35 MB/s | 5.8 MB 00:00 2024-04-16T16:50:44.999 INFO:teuthology.orchestra.run.smithi045.stdout:(64/174): python3-jsonpatch-1.21-16.el9.noarch. 206 kB/s | 24 kB 00:00 2024-04-16T16:50:45.001 INFO:teuthology.orchestra.run.smithi104.stdout:(23/174): python3-rbd-19.0.0-3101.g9b42bfbe.el9 744 kB/s | 298 kB 00:00 2024-04-16T16:50:45.041 INFO:teuthology.orchestra.run.smithi045.stdout:(65/174): python3-jsonpointer-2.0-4.el9.noarch. 450 kB/s | 18 kB 00:00 2024-04-16T16:50:45.042 INFO:teuthology.orchestra.run.smithi117.stdout:(25/174): rbd-fuse-19.0.0-3101.g9b42bfbe.el9.x8 489 kB/s | 85 kB 00:00 2024-04-16T16:50:45.091 INFO:teuthology.orchestra.run.smithi045.stdout:(66/174): python3-mako-1.1.4-6.el9.noarch.rpm 1.6 MB/s | 144 kB 00:00 2024-04-16T16:50:45.142 INFO:teuthology.orchestra.run.smithi045.stdout:(67/174): python3-markupsafe-1.1.1-12.el9.x86_6 647 kB/s | 32 kB 00:00 2024-04-16T16:50:45.210 INFO:teuthology.orchestra.run.smithi104.stdout:(24/174): python3-rgw-19.0.0-3101.g9b42bfbe.el9 477 kB/s | 99 kB 00:00 2024-04-16T16:50:45.275 INFO:teuthology.orchestra.run.smithi045.stdout:(68/174): python3-numpy-1.20.1-5.el9.x86_64.rpm 26 MB/s | 4.8 MB 00:00 2024-04-16T16:50:45.300 INFO:teuthology.orchestra.run.smithi045.stdout:(69/174): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.5 MB/s | 217 kB 00:00 2024-04-16T16:50:45.376 INFO:teuthology.orchestra.run.smithi045.stdout:(70/174): python3-oauthlib-3.1.1-5.el9.noarch.r 2.3 MB/s | 174 kB 00:00 2024-04-16T16:50:45.401 INFO:teuthology.orchestra.run.smithi045.stdout:(71/174): python3-packaging-20.9-5.el9.noarch.r 696 kB/s | 69 kB 00:00 2024-04-16T16:50:45.427 INFO:teuthology.orchestra.run.smithi045.stdout:(72/174): python3-prettytable-0.7.2-27.el9.noar 827 kB/s | 41 kB 00:00 2024-04-16T16:50:45.427 INFO:teuthology.orchestra.run.smithi104.stdout:(25/174): rbd-fuse-19.0.0-3101.g9b42bfbe.el9.x8 394 kB/s | 85 kB 00:00 2024-04-16T16:50:45.485 INFO:teuthology.orchestra.run.smithi045.stdout:(73/174): python3-protobuf-3.14.0-13.el9.noarch 2.8 MB/s | 238 kB 00:00 2024-04-16T16:50:45.510 INFO:teuthology.orchestra.run.smithi045.stdout:(74/174): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.6 MB/s | 132 kB 00:00 2024-04-16T16:50:45.544 INFO:teuthology.orchestra.run.smithi045.stdout:(75/174): python3-pyasn1-modules-0.4.8-6.el9.no 3.6 MB/s | 211 kB 00:00 2024-04-16T16:50:45.569 INFO:teuthology.orchestra.run.smithi045.stdout:(76/174): python3-pytz-2021.1-5.el9.noarch.rpm 850 kB/s | 47 kB 00:00 2024-04-16T16:50:45.594 INFO:teuthology.orchestra.run.smithi045.stdout:(77/174): python3-requests-oauthlib-1.3.0-12.el 873 kB/s | 43 kB 00:00 2024-04-16T16:50:45.978 INFO:teuthology.orchestra.run.smithi045.stdout:(78/174): python3-scipy-1.6.2-8.el9.x86_64.rpm 38 MB/s | 16 MB 00:00 2024-04-16T16:50:46.012 INFO:teuthology.orchestra.run.smithi045.stdout:(79/174): python3-toml-0.10.2-6.el9.noarch.rpm 99 kB/s | 39 kB 00:00 2024-04-16T16:50:46.053 INFO:teuthology.orchestra.run.smithi045.stdout:(80/174): python3-wcwidth-0.2.5-8.el9.noarch.rp 1.0 MB/s | 41 kB 00:00 2024-04-16T16:50:46.093 INFO:teuthology.orchestra.run.smithi045.stdout:(81/174): qatlib-24.02.0-1.el9.x86_64.rpm 2.7 MB/s | 220 kB 00:00 2024-04-16T16:50:46.118 INFO:teuthology.orchestra.run.smithi045.stdout:(82/174): qatlib-service-24.02.0-1.el9.x86_64.r 570 kB/s | 35 kB 00:00 2024-04-16T16:50:46.143 INFO:teuthology.orchestra.run.smithi045.stdout:(83/174): qatzip-libs-1.2.0-1.el9.x86_64.rpm 946 kB/s | 46 kB 00:00 2024-04-16T16:50:46.177 INFO:teuthology.orchestra.run.smithi045.stdout:(84/174): socat-1.7.4.1-5.el9.x86_64.rpm 5.1 MB/s | 300 kB 00:00 2024-04-16T16:50:46.202 INFO:teuthology.orchestra.run.smithi045.stdout:(85/174): uuid-1.6.2-55.el9.x86_64.rpm 1.0 MB/s | 56 kB 00:00 2024-04-16T16:50:46.228 INFO:teuthology.orchestra.run.smithi045.stdout:(86/174): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 63 kB 00:00 2024-04-16T16:50:46.253 INFO:teuthology.orchestra.run.smithi045.stdout:(87/174): zlib-devel-1.2.11-40.el9.x86_64.rpm 905 kB/s | 44 kB 00:00 2024-04-16T16:50:46.278 INFO:teuthology.orchestra.run.smithi045.stdout:(88/174): fuse-2.9.9-15.el9.x86_64.rpm 1.6 MB/s | 79 kB 00:00 2024-04-16T16:50:46.334 INFO:teuthology.orchestra.run.smithi045.stdout:(89/174): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.0 MB/s | 241 kB 00:00 2024-04-16T16:50:46.387 INFO:teuthology.orchestra.run.smithi045.stdout:(90/174): python3-cryptography-36.0.1-4.el9.x86 11 MB/s | 1.1 MB 00:00 2024-04-16T16:50:46.402 INFO:teuthology.orchestra.run.smithi104.stdout:(26/174): librgw2-19.0.0-3101.g9b42bfbe.el9.x86 858 kB/s | 5.4 MB 00:06 2024-04-16T16:50:46.412 INFO:teuthology.orchestra.run.smithi045.stdout:(91/174): python3-ply-3.11-14.el9.noarch.rpm 1.5 MB/s | 103 kB 00:00 2024-04-16T16:50:46.438 INFO:teuthology.orchestra.run.smithi045.stdout:(92/174): python3-pycparser-2.20-6.el9.noarch.r 2.5 MB/s | 124 kB 00:00 2024-04-16T16:50:46.471 INFO:teuthology.orchestra.run.smithi045.stdout:(93/174): python3-urllib3-1.26.5-5.el9.noarch.r 3.2 MB/s | 187 kB 00:00 2024-04-16T16:50:46.613 INFO:teuthology.orchestra.run.smithi045.stdout:(94/174): jq-1.6-16.el9.x86_64.rpm 1.1 MB/s | 187 kB 00:00 2024-04-16T16:50:46.664 INFO:teuthology.orchestra.run.smithi045.stdout:(95/174): libquadmath-11.4.1-3.el9.x86_64.rpm 3.9 MB/s | 195 kB 00:00 2024-04-16T16:50:46.694 INFO:teuthology.orchestra.run.smithi104.stdout:(27/174): rbd-nbd-19.0.0-3101.g9b42bfbe.el9.x86 585 kB/s | 170 kB 00:00 2024-04-16T16:50:46.737 INFO:teuthology.orchestra.run.smithi045.stdout:(96/174): libgfortran-11.4.1-3.el9.x86_64.rpm 3.0 MB/s | 803 kB 00:00 2024-04-16T16:50:46.773 INFO:teuthology.orchestra.run.smithi045.stdout:(97/174): python3-requests-2.25.1-8.el9.noarch. 1.1 MB/s | 125 kB 00:00 2024-04-16T16:50:46.786 INFO:teuthology.orchestra.run.smithi104.stdout:(28/174): ceph-grafana-dashboards-19.0.0-3101.g 373 kB/s | 34 kB 00:00 2024-04-16T16:50:46.831 INFO:teuthology.orchestra.run.smithi045.stdout:(98/174): elfutils-libelf-devel-0.190-2.el9.x86 231 kB/s | 21 kB 00:00 2024-04-16T16:50:46.865 INFO:teuthology.orchestra.run.smithi045.stdout:(99/174): libuuid-devel-2.37.4-18.el9.x86_64.rp 278 kB/s | 25 kB 00:00 2024-04-16T16:50:46.957 INFO:teuthology.orchestra.run.smithi045.stdout:(100/174): python3-devel-3.9.17-2.el9.x86_64.rp 2.6 MB/s | 245 kB 00:00 2024-04-16T16:50:47.007 INFO:teuthology.orchestra.run.smithi045.stdout:(101/174): python3-jinja2-2.11.3-5.el9.noarch.r 4.9 MB/s | 248 kB 00:00 2024-04-16T16:50:47.020 INFO:teuthology.orchestra.run.smithi104.stdout:(29/174): ceph-mgr-cephadm-19.0.0-3101.g9b42bfb 609 kB/s | 142 kB 00:00 2024-04-16T16:50:47.058 INFO:teuthology.orchestra.run.smithi045.stdout:(102/174): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 1.9 MB/s | 94 kB 00:00 2024-04-16T16:50:47.191 INFO:teuthology.orchestra.run.smithi045.stdout:(103/174): openssl-devel-3.0.7-27.el9.x86_64.rp 12 MB/s | 4.1 MB 00:00 2024-04-16T16:50:47.208 INFO:teuthology.orchestra.run.smithi045.stdout:(104/174): lua-devel-5.4.4-4.el9.x86_64.rpm 149 kB/s | 22 kB 00:00 2024-04-16T16:50:47.242 INFO:teuthology.orchestra.run.smithi117.stdout:(26/174): ceph-osd-19.0.0-3101.g9b42bfbe.el9.x8 548 kB/s | 17 MB 00:31 2024-04-16T16:50:47.292 INFO:teuthology.orchestra.run.smithi045.stdout:(105/174): protobuf-compiler-3.14.0-13.el9.x86_ 8.5 MB/s | 863 kB 00:00 2024-04-16T16:50:47.525 INFO:teuthology.orchestra.run.smithi117.stdout:(27/174): rbd-nbd-19.0.0-3101.g9b42bfbe.el9.x86 603 kB/s | 170 kB 00:00 2024-04-16T16:50:47.650 INFO:teuthology.orchestra.run.smithi117.stdout:(28/174): ceph-grafana-dashboards-19.0.0-3101.g 273 kB/s | 34 kB 00:00 2024-04-16T16:50:47.850 INFO:teuthology.orchestra.run.smithi117.stdout:(29/174): ceph-mgr-cephadm-19.0.0-3101.g9b42bfb 711 kB/s | 142 kB 00:00 2024-04-16T16:50:48.384 INFO:teuthology.orchestra.run.smithi045.stdout:(106/174): gperftools-libs-2.9.1-2.el9.x86_64.r 302 kB/s | 309 kB 00:01 2024-04-16T16:50:48.492 INFO:teuthology.orchestra.run.smithi045.stdout:(107/174): grpc-data-1.46.7-10.el9.noarch.rpm 181 kB/s | 19 kB 00:00 2024-04-16T16:50:48.584 INFO:teuthology.orchestra.run.smithi045.stdout:(108/174): abseil-cpp-20211102.0-3.el9.x86_64.r 399 kB/s | 548 kB 00:01 2024-04-16T16:50:48.661 INFO:teuthology.orchestra.run.smithi104.stdout:(30/174): rbd-mirror-19.0.0-3101.g9b42bfbe.el9. 978 kB/s | 3.1 MB 00:03 2024-04-16T16:50:48.710 INFO:teuthology.orchestra.run.smithi045.stdout:(109/174): libarrow-doc-9.0.0-10.el9.noarch.rpm 209 kB/s | 26 kB 00:00 2024-04-16T16:50:48.818 INFO:teuthology.orchestra.run.smithi045.stdout:(110/174): liboath-2.6.7-2.el9.x86_64.rpm 452 kB/s | 49 kB 00:00 2024-04-16T16:50:49.010 INFO:teuthology.orchestra.run.smithi045.stdout:(111/174): libunwind-1.6.2-1.el9.x86_64.rpm 353 kB/s | 67 kB 00:00 2024-04-16T16:50:49.319 INFO:teuthology.orchestra.run.smithi045.stdout:(112/174): luarocks-3.9.2-1.el9.noarch.rpm 491 kB/s | 151 kB 00:00 2024-04-16T16:50:49.626 INFO:teuthology.orchestra.run.smithi117.stdout:(30/174): rbd-mirror-19.0.0-3101.g9b42bfbe.el9. 690 kB/s | 3.1 MB 00:04 2024-04-16T16:50:50.177 INFO:teuthology.orchestra.run.smithi045.stdout:(113/174): parquet-libs-9.0.0-10.el9.x86_64.rpm 977 kB/s | 838 kB 00:00 2024-04-16T16:50:50.328 INFO:teuthology.orchestra.run.smithi045.stdout:(114/174): python3-asyncssh-2.13.2-5.el9.noarch 3.6 MB/s | 548 kB 00:00 2024-04-16T16:50:50.418 INFO:teuthology.orchestra.run.smithi117.stdout:(31/174): ceph-mgr-dashboard-19.0.0-3101.g9b42b 1.4 MB/s | 3.5 MB 00:02 2024-04-16T16:50:50.436 INFO:teuthology.orchestra.run.smithi045.stdout:(115/174): python3-bcrypt-3.2.2-1.el9.x86_64.rp 405 kB/s | 43 kB 00:00 2024-04-16T16:50:50.545 INFO:teuthology.orchestra.run.smithi045.stdout:(116/174): python3-cachetools-4.2.4-1.el9.noarc 299 kB/s | 32 kB 00:00 2024-04-16T16:50:50.670 INFO:teuthology.orchestra.run.smithi045.stdout:(117/174): python3-certifi-2021.10.8-2.el9.noar 120 kB/s | 15 kB 00:00 2024-04-16T16:50:50.787 INFO:teuthology.orchestra.run.smithi045.stdout:(118/174): python3-cheroot-8.6.0-4.el9.noarch.r 1.4 MB/s | 172 kB 00:00 2024-04-16T16:50:50.801 INFO:teuthology.orchestra.run.smithi117.stdout:(32/174): ceph-mgr-modules-core-19.0.0-3101.g9b 650 kB/s | 249 kB 00:00 2024-04-16T16:50:50.918 INFO:teuthology.orchestra.run.smithi117.stdout:(33/174): ceph-mgr-rook-19.0.0-3101.g9b42bfbe.e 423 kB/s | 49 kB 00:00 2024-04-16T16:50:50.920 INFO:teuthology.orchestra.run.smithi045.stdout:(119/174): python3-cherrypy-18.6.1-2.el9.noarch 2.6 MB/s | 358 kB 00:00 2024-04-16T16:50:51.018 INFO:teuthology.orchestra.run.smithi117.stdout:(34/174): ceph-prometheus-alerts-19.0.0-3101.g9 165 kB/s | 16 kB 00:00 2024-04-16T16:50:51.063 INFO:teuthology.orchestra.run.smithi045.stdout:(120/174): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.1 MB/s | 303 kB 00:00 2024-04-16T16:50:51.103 INFO:teuthology.orchestra.run.smithi104.stdout:(31/174): ceph-mgr-dashboard-19.0.0-3101.g9b42b 888 kB/s | 3.5 MB 00:04 2024-04-16T16:50:51.180 INFO:teuthology.orchestra.run.smithi045.stdout:(121/174): python3-colorama-0.4.4-12.el9.noarch 305 kB/s | 35 kB 00:00 2024-04-16T16:50:51.227 INFO:teuthology.orchestra.run.smithi117.stdout:(35/174): ceph-volume-19.0.0-3101.g9b42bfbe.el9 1.3 MB/s | 273 kB 00:00 2024-04-16T16:50:51.297 INFO:teuthology.orchestra.run.smithi045.stdout:(122/174): python3-google-auth-2.28.2-1.el9.noa 1.7 MB/s | 203 kB 00:00 2024-04-16T16:50:51.436 INFO:teuthology.orchestra.run.smithi104.stdout:(32/174): ceph-mgr-modules-core-19.0.0-3101.g9b 747 kB/s | 249 kB 00:00 2024-04-16T16:50:51.595 INFO:teuthology.orchestra.run.smithi104.stdout:(33/174): ceph-mgr-rook-19.0.0-3101.g9b42bfbe.e 312 kB/s | 49 kB 00:00 2024-04-16T16:50:51.695 INFO:teuthology.orchestra.run.smithi104.stdout:(34/174): ceph-prometheus-alerts-19.0.0-3101.g9 165 kB/s | 16 kB 00:00 2024-04-16T16:50:51.697 INFO:teuthology.orchestra.run.smithi045.stdout:(123/174): python3-grpcio-1.46.7-10.el9.x86_64. 5.1 MB/s | 2.0 MB 00:00 2024-04-16T16:50:51.702 INFO:teuthology.orchestra.run.smithi117.stdout:(36/174): cephadm-19.0.0-3101.g9b42bfbe.el9.noa 1.6 MB/s | 762 kB 00:00 2024-04-16T16:50:51.803 INFO:teuthology.orchestra.run.smithi117.stdout:(37/174): cephfs-top-19.0.0-3101.g9b42bfbe.el9. 182 kB/s | 18 kB 00:00 2024-04-16T16:50:51.839 INFO:teuthology.orchestra.run.smithi045.stdout:(124/174): python3-grpcio-tools-1.46.7-10.el9.x 1.0 MB/s | 144 kB 00:00 2024-04-16T16:50:51.953 INFO:teuthology.orchestra.run.smithi117.stdout:(38/174): automake-1.16.2-8.el9.noarch.rpm 4.3 MB/s | 662 kB 00:00 2024-04-16T16:50:51.964 INFO:teuthology.orchestra.run.smithi045.stdout:(125/174): python3-jaraco-8.2.1-3.el9.noarch.rp 86 kB/s | 11 kB 00:00 2024-04-16T16:50:52.012 INFO:teuthology.orchestra.run.smithi117.stdout:(39/174): bison-3.7.4-5.el9.x86_64.rpm 15 MB/s | 920 kB 00:00 2024-04-16T16:50:52.045 INFO:teuthology.orchestra.run.smithi117.stdout:(40/174): boost-program-options-1.75.0-8.el9.x8 3.1 MB/s | 106 kB 00:00 2024-04-16T16:50:52.073 INFO:teuthology.orchestra.run.smithi045.stdout:(126/174): python3-jaraco-classes-3.2.1-5.el9.n 164 kB/s | 18 kB 00:00 2024-04-16T16:50:52.087 INFO:teuthology.orchestra.run.smithi117.stdout:(41/174): flex-2.6.4-9.el9.x86_64.rpm 7.2 MB/s | 307 kB 00:00 2024-04-16T16:50:52.112 INFO:teuthology.orchestra.run.smithi104.stdout:(35/174): ceph-volume-19.0.0-3101.g9b42bfbe.el9 655 kB/s | 273 kB 00:00 2024-04-16T16:50:52.122 INFO:teuthology.orchestra.run.smithi117.stdout:(42/174): flexiblas-3.0.4-8.el9.x86_64.rpm 955 kB/s | 32 kB 00:00 2024-04-16T16:50:52.224 INFO:teuthology.orchestra.run.smithi045.stdout:(127/174): python3-jaraco-collections-3.0.0-8.e 155 kB/s | 23 kB 00:00 2024-04-16T16:50:52.246 INFO:teuthology.orchestra.run.smithi117.stdout:(43/174): flexiblas-netlib-3.0.4-8.el9.x86_64.r 24 MB/s | 3.0 MB 00:00 2024-04-16T16:50:52.279 INFO:teuthology.orchestra.run.smithi117.stdout:(44/174): flexiblas-openblas-openmp-3.0.4-8.el9 517 kB/s | 17 kB 00:00 2024-04-16T16:50:52.313 INFO:teuthology.orchestra.run.smithi117.stdout:(45/174): libaio-devel-0.3.111-13.el9.x86_64.rp 302 kB/s | 10 kB 00:00 2024-04-16T16:50:52.332 INFO:teuthology.orchestra.run.smithi045.stdout:(128/174): python3-jaraco-functools-3.5.0-2.el9 181 kB/s | 19 kB 00:00 2024-04-16T16:50:52.346 INFO:teuthology.orchestra.run.smithi117.stdout:(46/174): libpmemobj-1.12.1-1.el9.x86_64.rpm 4.7 MB/s | 159 kB 00:00 2024-04-16T16:50:52.380 INFO:teuthology.orchestra.run.smithi117.stdout:(47/174): librabbitmq-0.11.0-7.el9.x86_64.rpm 1.3 MB/s | 44 kB 00:00 2024-04-16T16:50:52.439 INFO:teuthology.orchestra.run.smithi117.stdout:(48/174): librdkafka-1.6.1-102.el9.x86_64.rpm 11 MB/s | 661 kB 00:00 2024-04-16T16:50:52.440 INFO:teuthology.orchestra.run.smithi045.stdout:(129/174): python3-jaraco-text-3.2.0-6.el9.noar 182 kB/s | 20 kB 00:00 2024-04-16T16:50:52.489 INFO:teuthology.orchestra.run.smithi117.stdout:(49/174): libtool-2.4.6-45.el9.x86_64.rpm 11 MB/s | 578 kB 00:00 2024-04-16T16:50:52.531 INFO:teuthology.orchestra.run.smithi117.stdout:(50/174): libzstd-devel-1.5.1-2.el9.x86_64.rpm 1.1 MB/s | 46 kB 00:00 2024-04-16T16:50:52.581 INFO:teuthology.orchestra.run.smithi117.stdout:(51/174): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.5 MB/s | 282 kB 00:00 2024-04-16T16:50:52.623 INFO:teuthology.orchestra.run.smithi117.stdout:(52/174): lua-5.4.4-4.el9.x86_64.rpm 4.4 MB/s | 187 kB 00:00 2024-04-16T16:50:52.665 INFO:teuthology.orchestra.run.smithi117.stdout:(53/174): oniguruma-6.9.6-1.el9.5.x86_64.rpm 5.1 MB/s | 217 kB 00:00 2024-04-16T16:50:52.670 INFO:teuthology.orchestra.run.smithi045.stdout:(130/174): python3-kubernetes-26.1.0-2.el9.noar 4.5 MB/s | 1.0 MB 00:00 2024-04-16T16:50:52.707 INFO:teuthology.orchestra.run.smithi117.stdout:(54/174): openblas-0.3.21-2.el9.x86_64.rpm 821 kB/s | 34 kB 00:00 2024-04-16T16:50:52.812 INFO:teuthology.orchestra.run.smithi045.stdout:(131/174): python3-logutils-0.3.5-21.el9.noarch 329 kB/s | 46 kB 00:00 2024-04-16T16:50:52.866 INFO:teuthology.orchestra.run.smithi117.stdout:(55/174): openblas-openmp-0.3.21-2.el9.x86_64.r 30 MB/s | 4.7 MB 00:00 2024-04-16T16:50:52.933 INFO:teuthology.orchestra.run.smithi117.stdout:(56/174): postgresql-13.14-1.el9.x86_64.rpm 23 MB/s | 1.5 MB 00:00 2024-04-16T16:50:52.938 INFO:teuthology.orchestra.run.smithi045.stdout:(132/174): python3-more-itertools-8.12.0-2.el9. 634 kB/s | 79 kB 00:00 2024-04-16T16:50:52.983 INFO:teuthology.orchestra.run.smithi117.stdout:(57/174): postgresql-contrib-13.14-1.el9.x86_64 16 MB/s | 817 kB 00:00 2024-04-16T16:50:53.025 INFO:teuthology.orchestra.run.smithi117.stdout:(58/174): postgresql-private-libs-13.14-1.el9.x 3.1 MB/s | 132 kB 00:00 2024-04-16T16:50:53.054 INFO:teuthology.orchestra.run.smithi104.stdout:(36/174): cephadm-19.0.0-3101.g9b42bfbe.el9.noa 810 kB/s | 762 kB 00:00 2024-04-16T16:50:53.054 INFO:teuthology.orchestra.run.smithi045.stdout:(133/174): python3-natsort-7.1.1-5.el9.noarch.r 498 kB/s | 58 kB 00:00 2024-04-16T16:50:53.138 INFO:teuthology.orchestra.run.smithi104.stdout:(37/174): cephfs-top-19.0.0-3101.g9b42bfbe.el9. 219 kB/s | 18 kB 00:00 2024-04-16T16:50:53.188 INFO:teuthology.orchestra.run.smithi045.stdout:(134/174): python3-pecan-1.4.2-3.el9.noarch.rpm 2.0 MB/s | 272 kB 00:00 2024-04-16T16:50:53.217 INFO:teuthology.orchestra.run.smithi117.stdout:(59/174): postgresql-server-13.14-1.el9.x86_64. 30 MB/s | 5.7 MB 00:00 2024-04-16T16:50:53.275 INFO:teuthology.orchestra.run.smithi117.stdout:(60/174): protobuf-3.14.0-13.el9.x86_64.rpm 17 MB/s | 1.0 MB 00:00 2024-04-16T16:50:53.305 INFO:teuthology.orchestra.run.smithi045.stdout:(135/174): python3-portend-3.1.0-2.el9.noarch.r 142 kB/s | 16 kB 00:00 2024-04-16T16:50:53.317 INFO:teuthology.orchestra.run.smithi117.stdout:(61/174): python3-attrs-20.3.0-7.el9.noarch.rpm 1.8 MB/s | 78 kB 00:00 2024-04-16T16:50:53.338 INFO:teuthology.orchestra.run.smithi104.stdout:(38/174): automake-1.16.2-8.el9.noarch.rpm 3.2 MB/s | 662 kB 00:00 2024-04-16T16:50:53.530 INFO:teuthology.orchestra.run.smithi104.stdout:(39/174): bison-3.7.4-5.el9.x86_64.rpm 4.7 MB/s | 920 kB 00:00 2024-04-16T16:50:53.559 INFO:teuthology.orchestra.run.smithi117.stdout:(62/174): python3-babel-2.9.1-2.el9.noarch.rpm 24 MB/s | 5.8 MB 00:00 2024-04-16T16:50:53.563 INFO:teuthology.orchestra.run.smithi104.stdout:(40/174): boost-program-options-1.75.0-8.el9.x8 3.1 MB/s | 106 kB 00:00 2024-04-16T16:50:53.593 INFO:teuthology.orchestra.run.smithi117.stdout:(63/174): python3-jsonpatch-1.21-16.el9.noarch. 732 kB/s | 24 kB 00:00 2024-04-16T16:50:53.622 INFO:teuthology.orchestra.run.smithi104.stdout:(41/174): flex-2.6.4-9.el9.x86_64.rpm 5.2 MB/s | 307 kB 00:00 2024-04-16T16:50:53.635 INFO:teuthology.orchestra.run.smithi117.stdout:(64/174): python3-jsonpointer-2.0-4.el9.noarch. 431 kB/s | 18 kB 00:00 2024-04-16T16:50:53.664 INFO:teuthology.orchestra.run.smithi104.stdout:(42/174): flexiblas-3.0.4-8.el9.x86_64.rpm 759 kB/s | 32 kB 00:00 2024-04-16T16:50:53.685 INFO:teuthology.orchestra.run.smithi117.stdout:(65/174): python3-mako-1.1.4-6.el9.noarch.rpm 2.8 MB/s | 144 kB 00:00 2024-04-16T16:50:53.711 INFO:teuthology.orchestra.run.smithi117.stdout:(66/174): python3-markupsafe-1.1.1-12.el9.x86_6 1.2 MB/s | 32 kB 00:00 2024-04-16T16:50:53.789 INFO:teuthology.orchestra.run.smithi104.stdout:(43/174): flexiblas-netlib-3.0.4-8.el9.x86_64.r 24 MB/s | 3.0 MB 00:00 2024-04-16T16:50:53.831 INFO:teuthology.orchestra.run.smithi104.stdout:(44/174): flexiblas-openblas-openmp-3.0.4-8.el9 412 kB/s | 17 kB 00:00 2024-04-16T16:50:53.856 INFO:teuthology.orchestra.run.smithi104.stdout:(45/174): libaio-devel-0.3.111-13.el9.x86_64.rp 406 kB/s | 10 kB 00:00 2024-04-16T16:50:53.880 INFO:teuthology.orchestra.run.smithi117.stdout:(67/174): python3-numpy-1.20.1-5.el9.x86_64.rpm 28 MB/s | 4.8 MB 00:00 2024-04-16T16:50:53.898 INFO:teuthology.orchestra.run.smithi104.stdout:(46/174): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.7 MB/s | 159 kB 00:00 2024-04-16T16:50:53.922 INFO:teuthology.orchestra.run.smithi117.stdout:(68/174): python3-numpy-f2py-1.20.1-5.el9.x86_6 5.1 MB/s | 217 kB 00:00 2024-04-16T16:50:53.923 INFO:teuthology.orchestra.run.smithi104.stdout:(47/174): librabbitmq-0.11.0-7.el9.x86_64.rpm 1.7 MB/s | 44 kB 00:00 2024-04-16T16:50:53.964 INFO:teuthology.orchestra.run.smithi117.stdout:(69/174): python3-oauthlib-3.1.1-5.el9.noarch.r 4.1 MB/s | 174 kB 00:00 2024-04-16T16:50:53.998 INFO:teuthology.orchestra.run.smithi117.stdout:(70/174): python3-packaging-20.9-5.el9.noarch.r 2.0 MB/s | 69 kB 00:00 2024-04-16T16:50:53.999 INFO:teuthology.orchestra.run.smithi104.stdout:(48/174): librdkafka-1.6.1-102.el9.x86_64.rpm 8.6 MB/s | 661 kB 00:00 2024-04-16T16:50:54.032 INFO:teuthology.orchestra.run.smithi117.stdout:(71/174): python3-prettytable-0.7.2-27.el9.noar 1.2 MB/s | 41 kB 00:00 2024-04-16T16:50:54.049 INFO:teuthology.orchestra.run.smithi104.stdout:(49/174): libtool-2.4.6-45.el9.x86_64.rpm 11 MB/s | 578 kB 00:00 2024-04-16T16:50:54.073 INFO:teuthology.orchestra.run.smithi117.stdout:(72/174): python3-protobuf-3.14.0-13.el9.noarch 5.6 MB/s | 238 kB 00:00 2024-04-16T16:50:54.074 INFO:teuthology.orchestra.run.smithi104.stdout:(50/174): libzstd-devel-1.5.1-2.el9.x86_64.rpm 1.8 MB/s | 46 kB 00:00 2024-04-16T16:50:54.114 INFO:teuthology.orchestra.run.smithi045.stdout:(136/174): python3-pyOpenSSL-21.0.0-1.el9.noarc 111 kB/s | 90 kB 00:00 2024-04-16T16:50:54.115 INFO:teuthology.orchestra.run.smithi117.stdout:(73/174): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.1 MB/s | 132 kB 00:00 2024-04-16T16:50:54.116 INFO:teuthology.orchestra.run.smithi104.stdout:(51/174): lttng-ust-2.12.0-6.el9.x86_64.rpm 6.6 MB/s | 282 kB 00:00 2024-04-16T16:50:54.157 INFO:teuthology.orchestra.run.smithi117.stdout:(74/174): python3-pyasn1-modules-0.4.8-6.el9.no 4.9 MB/s | 211 kB 00:00 2024-04-16T16:50:54.158 INFO:teuthology.orchestra.run.smithi104.stdout:(52/174): lua-5.4.4-4.el9.x86_64.rpm 4.4 MB/s | 187 kB 00:00 2024-04-16T16:50:54.190 INFO:teuthology.orchestra.run.smithi117.stdout:(75/174): python3-pytz-2021.1-5.el9.noarch.rpm 1.4 MB/s | 47 kB 00:00 2024-04-16T16:50:54.200 INFO:teuthology.orchestra.run.smithi104.stdout:(53/174): oniguruma-6.9.6-1.el9.5.x86_64.rpm 5.1 MB/s | 217 kB 00:00 2024-04-16T16:50:54.224 INFO:teuthology.orchestra.run.smithi117.stdout:(76/174): python3-requests-oauthlib-1.3.0-12.el 1.3 MB/s | 43 kB 00:00 2024-04-16T16:50:54.225 INFO:teuthology.orchestra.run.smithi104.stdout:(54/174): openblas-0.3.21-2.el9.x86_64.rpm 1.3 MB/s | 34 kB 00:00 2024-04-16T16:50:54.231 INFO:teuthology.orchestra.run.smithi045.stdout:(137/174): python3-pyperclip-1.8.2-2.el9.noarch 245 kB/s | 28 kB 00:00 2024-04-16T16:50:54.373 INFO:teuthology.orchestra.run.smithi045.stdout:(138/174): python3-repoze-lru-0.7-16.el9.noarch 218 kB/s | 31 kB 00:00 2024-04-16T16:50:54.401 INFO:teuthology.orchestra.run.smithi104.stdout:(55/174): openblas-openmp-0.3.21-2.el9.x86_64.r 27 MB/s | 4.7 MB 00:00 2024-04-16T16:50:54.476 INFO:teuthology.orchestra.run.smithi104.stdout:(56/174): postgresql-13.14-1.el9.x86_64.rpm 20 MB/s | 1.5 MB 00:00 2024-04-16T16:50:54.498 INFO:teuthology.orchestra.run.smithi045.stdout:(139/174): python3-routes-2.5.1-5.el9.noarch.rp 1.5 MB/s | 188 kB 00:00 2024-04-16T16:50:54.543 INFO:teuthology.orchestra.run.smithi104.stdout:(57/174): postgresql-contrib-13.14-1.el9.x86_64 12 MB/s | 817 kB 00:00 2024-04-16T16:50:54.586 INFO:teuthology.orchestra.run.smithi104.stdout:(58/174): postgresql-private-libs-13.14-1.el9.x 3.1 MB/s | 132 kB 00:00 2024-04-16T16:50:54.690 INFO:teuthology.orchestra.run.smithi045.stdout:(140/174): libarrow-9.0.0-10.el9.x86_64.rpm 730 kB/s | 4.4 MB 00:06 2024-04-16T16:50:54.715 INFO:teuthology.orchestra.run.smithi045.stdout:(141/174): python3-rsa-4.9-2.el9.noarch.rpm 273 kB/s | 59 kB 00:00 2024-04-16T16:50:54.799 INFO:teuthology.orchestra.run.smithi045.stdout:(142/174): python3-tempora-5.0.0-2.el9.noarch.r 332 kB/s | 36 kB 00:00 2024-04-16T16:50:54.802 INFO:teuthology.orchestra.run.smithi104.stdout:(59/174): postgresql-server-13.14-1.el9.x86_64. 26 MB/s | 5.7 MB 00:00 2024-04-16T16:50:54.832 INFO:teuthology.orchestra.run.smithi045.stdout:(143/174): python3-typing-extensions-4.4.0-2.el 438 kB/s | 51 kB 00:00 2024-04-16T16:50:54.878 INFO:teuthology.orchestra.run.smithi104.stdout:(60/174): protobuf-3.14.0-13.el9.x86_64.rpm 13 MB/s | 1.0 MB 00:00 2024-04-16T16:50:54.883 INFO:teuthology.orchestra.run.smithi117.stdout:(77/174): python3-scipy-1.6.2-8.el9.x86_64.rpm 24 MB/s | 16 MB 00:00 2024-04-16T16:50:54.911 INFO:teuthology.orchestra.run.smithi104.stdout:(61/174): python3-attrs-20.3.0-7.el9.noarch.rpm 2.3 MB/s | 78 kB 00:00 2024-04-16T16:50:54.916 INFO:teuthology.orchestra.run.smithi117.stdout:(78/174): python3-toml-0.10.2-6.el9.noarch.rpm 1.1 MB/s | 39 kB 00:00 2024-04-16T16:50:54.950 INFO:teuthology.orchestra.run.smithi117.stdout:(79/174): python3-wcwidth-0.2.5-8.el9.noarch.rp 1.2 MB/s | 41 kB 00:00 2024-04-16T16:50:54.973 INFO:teuthology.orchestra.run.smithi045.stdout:(144/174): python3-websocket-client-1.2.3-2.el9 643 kB/s | 90 kB 00:00 2024-04-16T16:50:55.000 INFO:teuthology.orchestra.run.smithi117.stdout:(80/174): qatlib-24.02.0-1.el9.x86_64.rpm 4.3 MB/s | 220 kB 00:00 2024-04-16T16:50:55.050 INFO:teuthology.orchestra.run.smithi117.stdout:(81/174): qatlib-service-24.02.0-1.el9.x86_64.r 696 kB/s | 35 kB 00:00 2024-04-16T16:50:55.093 INFO:teuthology.orchestra.run.smithi117.stdout:(82/174): qatzip-libs-1.2.0-1.el9.x86_64.rpm 1.1 MB/s | 46 kB 00:00 2024-04-16T16:50:55.108 INFO:teuthology.orchestra.run.smithi045.stdout:(145/174): python3-werkzeug-2.0.3-3.el9.1.noarc 3.1 MB/s | 427 kB 00:00 2024-04-16T16:50:55.120 INFO:teuthology.orchestra.run.smithi104.stdout:(62/174): python3-babel-2.9.1-2.el9.noarch.rpm 28 MB/s | 5.8 MB 00:00 2024-04-16T16:50:55.143 INFO:teuthology.orchestra.run.smithi117.stdout:(83/174): socat-1.7.4.1-5.el9.x86_64.rpm 5.9 MB/s | 300 kB 00:00 2024-04-16T16:50:55.167 INFO:teuthology.orchestra.run.smithi045.stdout:(146/174): python3-webob-1.8.7-6.el9.noarch.rpm 626 kB/s | 230 kB 00:00 2024-04-16T16:50:55.176 INFO:teuthology.orchestra.run.smithi117.stdout:(84/174): uuid-1.6.2-55.el9.x86_64.rpm 1.7 MB/s | 56 kB 00:00 2024-04-16T16:50:55.203 INFO:teuthology.orchestra.run.smithi104.stdout:(63/174): ceph-mgr-diskprediction-local-19.0.0- 1.1 MB/s | 7.4 MB 00:06 2024-04-16T16:50:55.210 INFO:teuthology.orchestra.run.smithi117.stdout:(85/174): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.8 MB/s | 63 kB 00:00 2024-04-16T16:50:55.217 INFO:teuthology.orchestra.run.smithi045.stdout:(147/174): python3-zc-lockfile-2.0-10.el9.noarc 184 kB/s | 20 kB 00:00 2024-04-16T16:50:55.229 INFO:teuthology.orchestra.run.smithi104.stdout:(64/174): python3-jsonpatch-1.21-16.el9.noarch. 225 kB/s | 24 kB 00:00 2024-04-16T16:50:55.254 INFO:teuthology.orchestra.run.smithi104.stdout:(65/174): python3-jsonpointer-2.0-4.el9.noarch. 362 kB/s | 18 kB 00:00 2024-04-16T16:50:55.260 INFO:teuthology.orchestra.run.smithi117.stdout:(86/174): zlib-devel-1.2.11-40.el9.x86_64.rpm 883 kB/s | 44 kB 00:00 2024-04-16T16:50:55.279 INFO:teuthology.orchestra.run.smithi104.stdout:(66/174): python3-mako-1.1.4-6.el9.noarch.rpm 2.8 MB/s | 144 kB 00:00 2024-04-16T16:50:55.294 INFO:teuthology.orchestra.run.smithi117.stdout:(87/174): fuse-2.9.9-15.el9.x86_64.rpm 2.3 MB/s | 79 kB 00:00 2024-04-16T16:50:55.304 INFO:teuthology.orchestra.run.smithi104.stdout:(67/174): python3-markupsafe-1.1.1-12.el9.x86_6 629 kB/s | 32 kB 00:00 2024-04-16T16:50:55.353 INFO:teuthology.orchestra.run.smithi117.stdout:(88/174): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.0 MB/s | 241 kB 00:00 2024-04-16T16:50:55.453 INFO:teuthology.orchestra.run.smithi117.stdout:(89/174): python3-cryptography-36.0.1-4.el9.x86 11 MB/s | 1.1 MB 00:00 2024-04-16T16:50:55.471 INFO:teuthology.orchestra.run.smithi104.stdout:(68/174): python3-numpy-1.20.1-5.el9.x86_64.rpm 25 MB/s | 4.8 MB 00:00 2024-04-16T16:50:55.495 INFO:teuthology.orchestra.run.smithi117.stdout:(90/174): python3-ply-3.11-14.el9.noarch.rpm 2.4 MB/s | 103 kB 00:00 2024-04-16T16:50:55.496 INFO:teuthology.orchestra.run.smithi104.stdout:(69/174): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.1 MB/s | 217 kB 00:00 2024-04-16T16:50:55.531 INFO:teuthology.orchestra.run.smithi104.stdout:(70/174): python3-oauthlib-3.1.1-5.el9.noarch.r 2.9 MB/s | 174 kB 00:00 2024-04-16T16:50:55.536 INFO:teuthology.orchestra.run.smithi117.stdout:(91/174): python3-pycparser-2.20-6.el9.noarch.r 2.9 MB/s | 124 kB 00:00 2024-04-16T16:50:55.548 INFO:teuthology.orchestra.run.smithi104.stdout:(71/174): python3-packaging-20.9-5.el9.noarch.r 1.3 MB/s | 69 kB 00:00 2024-04-16T16:50:55.559 INFO:teuthology.orchestra.run.smithi045.stdout:(148/174): thrift-0.15.0-2.el9.x86_64.rpm 4.6 MB/s | 1.6 MB 00:00 2024-04-16T16:50:55.565 INFO:teuthology.orchestra.run.smithi104.stdout:(72/174): python3-prettytable-0.7.2-27.el9.noar 1.2 MB/s | 41 kB 00:00 2024-04-16T16:50:55.579 INFO:teuthology.orchestra.run.smithi117.stdout:(92/174): python3-urllib3-1.26.5-5.el9.noarch.r 4.4 MB/s | 187 kB 00:00 2024-04-16T16:50:55.585 INFO:teuthology.orchestra.run.smithi045.stdout:(149/174): re2-20211101-3.el9.x86_64.rpm 459 kB/s | 192 kB 00:00 2024-04-16T16:50:55.590 INFO:teuthology.orchestra.run.smithi104.stdout:(73/174): python3-protobuf-3.14.0-13.el9.noarch 5.6 MB/s | 238 kB 00:00 2024-04-16T16:50:55.615 INFO:teuthology.orchestra.run.smithi104.stdout:(74/174): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.6 MB/s | 132 kB 00:00 2024-04-16T16:50:55.641 INFO:teuthology.orchestra.run.smithi104.stdout:(75/174): python3-pyasn1-modules-0.4.8-6.el9.no 4.1 MB/s | 211 kB 00:00 2024-04-16T16:50:55.658 INFO:teuthology.orchestra.run.smithi104.stdout:(76/174): python3-pytz-2021.1-5.el9.noarch.rpm 1.1 MB/s | 47 kB 00:00 2024-04-16T16:50:55.674 INFO:teuthology.orchestra.run.smithi104.stdout:(77/174): python3-requests-oauthlib-1.3.0-12.el 1.3 MB/s | 43 kB 00:00 2024-04-16T16:50:55.704 INFO:teuthology.orchestra.run.smithi117.stdout:(93/174): jq-1.6-16.el9.x86_64.rpm 1.5 MB/s | 187 kB 00:00 2024-04-16T16:50:55.788 INFO:teuthology.orchestra.run.smithi117.stdout:(94/174): libgfortran-11.4.1-3.el9.x86_64.rpm 9.4 MB/s | 803 kB 00:00 2024-04-16T16:50:55.830 INFO:teuthology.orchestra.run.smithi117.stdout:(95/174): libquadmath-11.4.1-3.el9.x86_64.rpm 4.6 MB/s | 195 kB 00:00 2024-04-16T16:50:55.841 INFO:teuthology.orchestra.run.smithi104.stdout:(78/174): python3-toml-0.10.2-6.el9.noarch.rpm 232 kB/s | 39 kB 00:00 2024-04-16T16:50:55.872 INFO:teuthology.orchestra.run.smithi117.stdout:(96/174): python3-requests-2.25.1-8.el9.noarch. 2.9 MB/s | 125 kB 00:00 2024-04-16T16:50:55.914 INFO:teuthology.orchestra.run.smithi117.stdout:(97/174): elfutils-libelf-devel-0.190-2.el9.x86 515 kB/s | 21 kB 00:00 2024-04-16T16:50:55.956 INFO:teuthology.orchestra.run.smithi117.stdout:(98/174): libuuid-devel-2.37.4-18.el9.x86_64.rp 613 kB/s | 25 kB 00:00 2024-04-16T16:50:56.025 INFO:teuthology.orchestra.run.smithi104.stdout:(79/174): python3-wcwidth-0.2.5-8.el9.noarch.rp 225 kB/s | 41 kB 00:00 2024-04-16T16:50:56.064 INFO:teuthology.orchestra.run.smithi117.stdout:(99/174): openssl-devel-3.0.7-27.el9.x86_64.rpm 38 MB/s | 4.1 MB 00:00 2024-04-16T16:50:56.106 INFO:teuthology.orchestra.run.smithi117.stdout:(100/174): python3-devel-3.9.17-2.el9.x86_64.rp 5.8 MB/s | 245 kB 00:00 2024-04-16T16:50:56.148 INFO:teuthology.orchestra.run.smithi117.stdout:(101/174): python3-jinja2-2.11.3-5.el9.noarch.r 5.8 MB/s | 248 kB 00:00 2024-04-16T16:50:56.190 INFO:teuthology.orchestra.run.smithi117.stdout:(102/174): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 2.2 MB/s | 94 kB 00:00 2024-04-16T16:50:56.200 INFO:teuthology.orchestra.run.smithi104.stdout:(80/174): python3-scipy-1.6.2-8.el9.x86_64.rpm 29 MB/s | 16 MB 00:00 2024-04-16T16:50:56.226 INFO:teuthology.orchestra.run.smithi104.stdout:(81/174): qatlib-24.02.0-1.el9.x86_64.rpm 1.1 MB/s | 220 kB 00:00 2024-04-16T16:50:56.232 INFO:teuthology.orchestra.run.smithi117.stdout:(103/174): lua-devel-5.4.4-4.el9.x86_64.rpm 535 kB/s | 22 kB 00:00 2024-04-16T16:50:56.242 INFO:teuthology.orchestra.run.smithi104.stdout:(82/174): qatlib-service-24.02.0-1.el9.x86_64.r 841 kB/s | 35 kB 00:00 2024-04-16T16:50:56.259 INFO:teuthology.orchestra.run.smithi104.stdout:(83/174): qatzip-libs-1.2.0-1.el9.x86_64.rpm 1.4 MB/s | 46 kB 00:00 2024-04-16T16:50:56.285 INFO:teuthology.orchestra.run.smithi104.stdout:(84/174): socat-1.7.4.1-5.el9.x86_64.rpm 7.0 MB/s | 300 kB 00:00 2024-04-16T16:50:56.291 INFO:teuthology.orchestra.run.smithi117.stdout:(104/174): protobuf-compiler-3.14.0-13.el9.x86_ 14 MB/s | 863 kB 00:00 2024-04-16T16:50:56.302 INFO:teuthology.orchestra.run.smithi104.stdout:(85/174): uuid-1.6.2-55.el9.x86_64.rpm 1.3 MB/s | 56 kB 00:00 2024-04-16T16:50:56.319 INFO:teuthology.orchestra.run.smithi104.stdout:(86/174): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.8 MB/s | 63 kB 00:00 2024-04-16T16:50:56.336 INFO:teuthology.orchestra.run.smithi104.stdout:(87/174): zlib-devel-1.2.11-40.el9.x86_64.rpm 1.3 MB/s | 44 kB 00:00 2024-04-16T16:50:56.361 INFO:teuthology.orchestra.run.smithi104.stdout:(88/174): fuse-2.9.9-15.el9.x86_64.rpm 1.8 MB/s | 79 kB 00:00 2024-04-16T16:50:56.386 INFO:teuthology.orchestra.run.smithi104.stdout:(89/174): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.7 MB/s | 241 kB 00:00 2024-04-16T16:50:56.445 INFO:teuthology.orchestra.run.smithi104.stdout:(90/174): python3-cryptography-36.0.1-4.el9.x86 14 MB/s | 1.1 MB 00:00 2024-04-16T16:50:56.462 INFO:teuthology.orchestra.run.smithi104.stdout:(91/174): python3-ply-3.11-14.el9.noarch.rpm 1.3 MB/s | 103 kB 00:00 2024-04-16T16:50:56.487 INFO:teuthology.orchestra.run.smithi104.stdout:(92/174): python3-pycparser-2.20-6.el9.noarch.r 2.9 MB/s | 124 kB 00:00 2024-04-16T16:50:56.512 INFO:teuthology.orchestra.run.smithi104.stdout:(93/174): python3-urllib3-1.26.5-5.el9.noarch.r 3.7 MB/s | 187 kB 00:00 2024-04-16T16:50:56.638 INFO:teuthology.orchestra.run.smithi104.stdout:(94/174): jq-1.6-16.el9.x86_64.rpm 1.2 MB/s | 187 kB 00:00 2024-04-16T16:50:56.680 INFO:teuthology.orchestra.run.smithi104.stdout:(95/174): libquadmath-11.4.1-3.el9.x86_64.rpm 4.6 MB/s | 195 kB 00:00 2024-04-16T16:50:56.722 INFO:teuthology.orchestra.run.smithi104.stdout:(96/174): libgfortran-11.4.1-3.el9.x86_64.rpm 3.8 MB/s | 803 kB 00:00 2024-04-16T16:50:56.755 INFO:teuthology.orchestra.run.smithi104.stdout:(97/174): python3-requests-2.25.1-8.el9.noarch. 1.6 MB/s | 125 kB 00:00 2024-04-16T16:50:56.808 INFO:teuthology.orchestra.run.smithi117.stdout:(105/174): abseil-cpp-20211102.0-3.el9.x86_64.r 1.0 MB/s | 548 kB 00:00 2024-04-16T16:50:56.872 INFO:teuthology.orchestra.run.smithi104.stdout:(98/174): elfutils-libelf-devel-0.190-2.el9.x86 143 kB/s | 21 kB 00:00 2024-04-16T16:50:56.897 INFO:teuthology.orchestra.run.smithi104.stdout:(99/174): libuuid-devel-2.37.4-18.el9.x86_64.rp 180 kB/s | 25 kB 00:00 2024-04-16T16:50:57.000 INFO:teuthology.orchestra.run.smithi117.stdout:(106/174): gperftools-libs-2.9.1-2.el9.x86_64.r 1.6 MB/s | 309 kB 00:00 2024-04-16T16:50:57.039 INFO:teuthology.orchestra.run.smithi104.stdout:(100/174): python3-devel-3.9.17-2.el9.x86_64.rp 1.7 MB/s | 245 kB 00:00 2024-04-16T16:50:57.108 INFO:teuthology.orchestra.run.smithi117.stdout:(107/174): grpc-data-1.46.7-10.el9.noarch.rpm 179 kB/s | 19 kB 00:00 2024-04-16T16:50:57.165 INFO:teuthology.orchestra.run.smithi104.stdout:(101/174): python3-jinja2-2.11.3-5.el9.noarch.r 1.9 MB/s | 248 kB 00:00 2024-04-16T16:50:57.231 INFO:teuthology.orchestra.run.smithi104.stdout:(102/174): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 1.4 MB/s | 94 kB 00:00 2024-04-16T16:50:57.315 INFO:teuthology.orchestra.run.smithi104.stdout:(103/174): lua-devel-5.4.4-4.el9.x86_64.rpm 268 kB/s | 22 kB 00:00 2024-04-16T16:50:57.440 INFO:teuthology.orchestra.run.smithi104.stdout:(104/174): protobuf-compiler-3.14.0-13.el9.x86_ 6.8 MB/s | 863 kB 00:00 2024-04-16T16:50:57.625 INFO:teuthology.orchestra.run.smithi117.stdout:(108/174): ceph-mgr-diskprediction-local-19.0.0 949 kB/s | 7.4 MB 00:07 2024-04-16T16:50:57.735 INFO:teuthology.orchestra.run.smithi045.stdout:(150/174): librados2-19.0.0-3101.g9b42bfbe.el9. 1.6 MB/s | 3.4 MB 00:02 2024-04-16T16:50:57.896 INFO:teuthology.orchestra.run.smithi045.stdout:(151/174): NetworkManager-1.47.4-1.el9.x86_64.r 14 MB/s | 2.3 MB 00:00 2024-04-16T16:50:57.992 INFO:teuthology.orchestra.run.smithi117.stdout:(109/174): libarrow-doc-9.0.0-10.el9.noarch.rpm 71 kB/s | 26 kB 00:00 2024-04-16T16:50:58.011 INFO:teuthology.orchestra.run.smithi045.stdout:(152/174): NetworkManager-libnm-1.47.4-1.el9.x8 16 MB/s | 1.8 MB 00:00 2024-04-16T16:50:58.044 INFO:teuthology.orchestra.run.smithi045.stdout:(153/174): NetworkManager-team-1.47.4-1.el9.x86 1.2 MB/s | 41 kB 00:00 2024-04-16T16:50:58.086 INFO:teuthology.orchestra.run.smithi045.stdout:(154/174): NetworkManager-tui-1.47.4-1.el9.x86_ 5.9 MB/s | 248 kB 00:00 2024-04-16T16:50:58.100 INFO:teuthology.orchestra.run.smithi117.stdout:(110/174): liboath-2.6.7-2.el9.x86_64.rpm 450 kB/s | 49 kB 00:00 2024-04-16T16:50:58.137 INFO:teuthology.orchestra.run.smithi045.stdout:(155/174): xfsdump-3.1.12-4.el9.x86_64.rpm 6.4 MB/s | 321 kB 00:00 2024-04-16T16:50:58.140 INFO:teuthology.orchestra.run.smithi104.stdout:(105/174): abseil-cpp-20211102.0-3.el9.x86_64.r 783 kB/s | 548 kB 00:00 2024-04-16T16:50:58.209 INFO:teuthology.orchestra.run.smithi117.stdout:(111/174): libunwind-1.6.2-1.el9.x86_64.rpm 621 kB/s | 67 kB 00:00 2024-04-16T16:50:58.245 INFO:teuthology.orchestra.run.smithi045.stdout:(156/174): acl-2.3.1-4.el9.x86_64.rpm 663 kB/s | 71 kB 00:00 2024-04-16T16:50:58.304 INFO:teuthology.orchestra.run.smithi045.stdout:(157/174): elfutils-debuginfod-client-0.190-2.e 630 kB/s | 36 kB 00:00 2024-04-16T16:50:58.318 INFO:teuthology.orchestra.run.smithi117.stdout:(112/174): luarocks-3.9.2-1.el9.noarch.rpm 1.4 MB/s | 151 kB 00:00 2024-04-16T16:50:58.332 INFO:teuthology.orchestra.run.smithi104.stdout:(106/174): gperftools-libs-2.9.1-2.el9.x86_64.r 1.6 MB/s | 309 kB 00:00 2024-04-16T16:50:58.379 INFO:teuthology.orchestra.run.smithi045.stdout:(158/174): elfutils-libelf-0.190-2.el9.x86_64.r 2.5 MB/s | 192 kB 00:00 2024-04-16T16:50:58.391 INFO:teuthology.orchestra.run.smithi104.stdout:(107/174): openssl-devel-3.0.7-27.el9.x86_64.rp 2.8 MB/s | 4.1 MB 00:01 2024-04-16T16:50:58.414 INFO:teuthology.orchestra.run.smithi117.stdout:(113/174): libarrow-9.0.0-10.el9.x86_64.rpm 3.4 MB/s | 4.4 MB 00:01 2024-04-16T16:50:58.416 INFO:teuthology.orchestra.run.smithi104.stdout:(108/174): grpc-data-1.46.7-10.el9.noarch.rpm 233 kB/s | 19 kB 00:00 2024-04-16T16:50:58.438 INFO:teuthology.orchestra.run.smithi045.stdout:(159/174): elfutils-libs-0.190-2.el9.x86_64.rpm 4.3 MB/s | 256 kB 00:00 2024-04-16T16:50:58.483 INFO:teuthology.orchestra.run.smithi104.stdout:(109/174): libarrow-doc-9.0.0-10.el9.noarch.rpm 390 kB/s | 26 kB 00:00 2024-04-16T16:50:58.488 INFO:teuthology.orchestra.run.smithi045.stdout:(160/174): libacl-2.3.1-4.el9.x86_64.rpm 469 kB/s | 23 kB 00:00 2024-04-16T16:50:58.518 INFO:teuthology.orchestra.run.smithi117.stdout:(114/174): python3-asyncssh-2.13.2-5.el9.noarch 5.2 MB/s | 548 kB 00:00 2024-04-16T16:50:58.539 INFO:teuthology.orchestra.run.smithi045.stdout:(161/174): libblkid-2.37.4-18.el9.x86_64.rpm 2.2 MB/s | 108 kB 00:00 2024-04-16T16:50:58.567 INFO:teuthology.orchestra.run.smithi104.stdout:(110/174): liboath-2.6.7-2.el9.x86_64.rpm 582 kB/s | 49 kB 00:00 2024-04-16T16:50:58.597 INFO:teuthology.orchestra.run.smithi045.stdout:(162/174): libfdisk-2.37.4-18.el9.x86_64.rpm 2.6 MB/s | 155 kB 00:00 2024-04-16T16:50:58.618 INFO:teuthology.orchestra.run.smithi117.stdout:(115/174): python3-bcrypt-3.2.2-1.el9.x86_64.rp 434 kB/s | 43 kB 00:00 2024-04-16T16:50:58.648 INFO:teuthology.orchestra.run.smithi045.stdout:(163/174): libmount-2.37.4-18.el9.x86_64.rpm 2.7 MB/s | 136 kB 00:00 2024-04-16T16:50:58.658 INFO:teuthology.orchestra.run.smithi104.stdout:(111/174): libunwind-1.6.2-1.el9.x86_64.rpm 736 kB/s | 67 kB 00:00 2024-04-16T16:50:58.694 INFO:teuthology.orchestra.run.smithi117.stdout:(116/174): python3-cachetools-4.2.4-1.el9.noarc 429 kB/s | 32 kB 00:00 2024-04-16T16:50:58.706 INFO:teuthology.orchestra.run.smithi045.stdout:(164/174): libsmartcols-2.37.4-18.el9.x86_64.rp 1.1 MB/s | 64 kB 00:00 2024-04-16T16:50:58.757 INFO:teuthology.orchestra.run.smithi045.stdout:(165/174): libuuid-2.37.4-18.el9.x86_64.rpm 583 kB/s | 29 kB 00:00 2024-04-16T16:50:58.769 INFO:teuthology.orchestra.run.smithi117.stdout:(117/174): python3-certifi-2021.10.8-2.el9.noar 200 kB/s | 15 kB 00:00 2024-04-16T16:50:58.775 INFO:teuthology.orchestra.run.smithi104.stdout:(112/174): luarocks-3.9.2-1.el9.noarch.rpm 1.3 MB/s | 151 kB 00:00 2024-04-16T16:50:58.802 INFO:teuthology.orchestra.run.smithi117.stdout:(118/174): parquet-libs-9.0.0-10.el9.x86_64.rpm 1.7 MB/s | 838 kB 00:00 2024-04-16T16:50:58.844 INFO:teuthology.orchestra.run.smithi117.stdout:(119/174): python3-cheroot-8.6.0-4.el9.noarch.r 2.2 MB/s | 172 kB 00:00 2024-04-16T16:50:58.857 INFO:teuthology.orchestra.run.smithi045.stdout:(166/174): openssl-3.0.7-27.el9.x86_64.rpm 12 MB/s | 1.2 MB 00:00 2024-04-16T16:50:58.920 INFO:teuthology.orchestra.run.smithi117.stdout:(120/174): python3-cmd2-2.4.2-2.el9.noarch.rpm 4.0 MB/s | 303 kB 00:00 2024-04-16T16:50:58.974 INFO:teuthology.orchestra.run.smithi045.stdout:(167/174): openssl-libs-3.0.7-27.el9.x86_64.rpm 19 MB/s | 2.2 MB 00:00 2024-04-16T16:50:58.987 INFO:teuthology.orchestra.run.smithi117.stdout:(121/174): python3-colorama-0.4.4-12.el9.noarch 529 kB/s | 35 kB 00:00 2024-04-16T16:50:59.012 INFO:teuthology.orchestra.run.smithi117.stdout:(122/174): python3-cherrypy-18.6.1-2.el9.noarch 1.7 MB/s | 358 kB 00:00 2024-04-16T16:50:59.063 INFO:teuthology.orchestra.run.smithi117.stdout:(123/174): python3-google-auth-2.28.2-1.el9.noa 2.6 MB/s | 203 kB 00:00 2024-04-16T16:50:59.084 INFO:teuthology.orchestra.run.smithi104.stdout:(113/174): parquet-libs-9.0.0-10.el9.x86_64.rpm 2.7 MB/s | 838 kB 00:00 2024-04-16T16:50:59.116 INFO:teuthology.orchestra.run.smithi045.stdout:(168/174): util-linux-2.37.4-18.el9.x86_64.rpm 16 MB/s | 2.3 MB 00:00 2024-04-16T16:50:59.146 INFO:teuthology.orchestra.run.smithi117.stdout:(124/174): python3-grpcio-tools-1.46.7-10.el9.x 1.7 MB/s | 144 kB 00:00 2024-04-16T16:50:59.175 INFO:teuthology.orchestra.run.smithi045.stdout:(169/174): util-linux-core-2.37.4-18.el9.x86_64 7.9 MB/s | 465 kB 00:00 2024-04-16T16:50:59.213 INFO:teuthology.orchestra.run.smithi117.stdout:(125/174): python3-jaraco-8.2.1-3.el9.noarch.rp 161 kB/s | 11 kB 00:00 2024-04-16T16:50:59.225 INFO:teuthology.orchestra.run.smithi045.stdout:(170/174): util-linux-user-2.37.4-18.el9.x86_64 631 kB/s | 31 kB 00:00 2024-04-16T16:50:59.280 INFO:teuthology.orchestra.run.smithi117.stdout:(126/174): python3-jaraco-classes-3.2.1-5.el9.n 265 kB/s | 18 kB 00:00 2024-04-16T16:50:59.301 INFO:teuthology.orchestra.run.smithi045.stdout:(171/174): xfsprogs-6.3.0-1.el9.x86_64.rpm 14 MB/s | 1.1 MB 00:00 2024-04-16T16:50:59.309 INFO:teuthology.orchestra.run.smithi104.stdout:(114/174): python3-asyncssh-2.13.2-5.el9.noarch 2.4 MB/s | 548 kB 00:00 2024-04-16T16:50:59.338 INFO:teuthology.orchestra.run.smithi117.stdout:(127/174): python3-jaraco-collections-3.0.0-8.e 397 kB/s | 23 kB 00:00 2024-04-16T16:50:59.376 INFO:teuthology.orchestra.run.smithi104.stdout:(115/174): python3-bcrypt-3.2.2-1.el9.x86_64.rp 652 kB/s | 43 kB 00:00 2024-04-16T16:50:59.384 INFO:teuthology.orchestra.run.smithi045.stdout:(172/174): libacl-devel-2.3.1-4.el9.x86_64.rpm 904 kB/s | 74 kB 00:00 2024-04-16T16:50:59.443 INFO:teuthology.orchestra.run.smithi104.stdout:(116/174): python3-cachetools-4.2.4-1.el9.noarc 485 kB/s | 32 kB 00:00 2024-04-16T16:50:59.464 INFO:teuthology.orchestra.run.smithi117.stdout:(128/174): python3-jaraco-functools-3.5.0-2.el9 155 kB/s | 19 kB 00:00 2024-04-16T16:50:59.510 INFO:teuthology.orchestra.run.smithi104.stdout:(117/174): python3-certifi-2021.10.8-2.el9.noar 225 kB/s | 15 kB 00:00 2024-04-16T16:50:59.585 INFO:teuthology.orchestra.run.smithi104.stdout:(118/174): python3-cheroot-8.6.0-4.el9.noarch.r 2.2 MB/s | 172 kB 00:00 2024-04-16T16:50:59.631 INFO:teuthology.orchestra.run.smithi117.stdout:(129/174): python3-jaraco-text-3.2.0-6.el9.noar 118 kB/s | 20 kB 00:00 2024-04-16T16:50:59.673 INFO:teuthology.orchestra.run.smithi117.stdout:(130/174): python3-grpcio-1.46.7-10.el9.x86_64. 3.1 MB/s | 2.0 MB 00:00 2024-04-16T16:50:59.711 INFO:teuthology.orchestra.run.smithi104.stdout:(119/174): python3-cherrypy-18.6.1-2.el9.noarch 2.8 MB/s | 358 kB 00:00 2024-04-16T16:50:59.748 INFO:teuthology.orchestra.run.smithi117.stdout:(131/174): python3-logutils-0.3.5-21.el9.noarch 618 kB/s | 46 kB 00:00 2024-04-16T16:50:59.815 INFO:teuthology.orchestra.run.smithi117.stdout:(132/174): python3-more-itertools-8.12.0-2.el9. 1.2 MB/s | 79 kB 00:00 2024-04-16T16:50:59.836 INFO:teuthology.orchestra.run.smithi104.stdout:(120/174): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.4 MB/s | 303 kB 00:00 2024-04-16T16:50:59.840 INFO:teuthology.orchestra.run.smithi117.stdout:(133/174): python3-kubernetes-26.1.0-2.el9.noar 4.9 MB/s | 1.0 MB 00:00 2024-04-16T16:50:59.891 INFO:teuthology.orchestra.run.smithi117.stdout:(134/174): python3-natsort-7.1.1-5.el9.noarch.r 767 kB/s | 58 kB 00:00 2024-04-16T16:50:59.911 INFO:teuthology.orchestra.run.smithi104.stdout:(121/174): python3-colorama-0.4.4-12.el9.noarch 472 kB/s | 35 kB 00:00 2024-04-16T16:50:59.916 INFO:teuthology.orchestra.run.smithi117.stdout:(135/174): python3-pecan-1.4.2-3.el9.noarch.rpm 3.5 MB/s | 272 kB 00:00 2024-04-16T16:50:59.975 INFO:teuthology.orchestra.run.smithi117.stdout:(136/174): python3-portend-3.1.0-2.el9.noarch.r 197 kB/s | 16 kB 00:00 2024-04-16T16:51:00.008 INFO:teuthology.orchestra.run.smithi117.stdout:(137/174): python3-pyOpenSSL-21.0.0-1.el9.noarc 977 kB/s | 90 kB 00:00 2024-04-16T16:51:00.012 INFO:teuthology.orchestra.run.smithi104.stdout:(122/174): python3-google-auth-2.28.2-1.el9.noa 2.0 MB/s | 203 kB 00:00 2024-04-16T16:51:00.075 INFO:teuthology.orchestra.run.smithi117.stdout:(138/174): python3-pyperclip-1.8.2-2.el9.noarch 282 kB/s | 28 kB 00:00 2024-04-16T16:51:00.109 INFO:teuthology.orchestra.run.smithi117.stdout:(139/174): python3-repoze-lru-0.7-16.el9.noarch 307 kB/s | 31 kB 00:00 2024-04-16T16:51:00.167 INFO:teuthology.orchestra.run.smithi117.stdout:(140/174): python3-routes-2.5.1-5.el9.noarch.rp 2.0 MB/s | 188 kB 00:00 2024-04-16T16:51:00.192 INFO:teuthology.orchestra.run.smithi117.stdout:(141/174): python3-rsa-4.9-2.el9.noarch.rpm 707 kB/s | 59 kB 00:00 2024-04-16T16:51:00.251 INFO:teuthology.orchestra.run.smithi117.stdout:(142/174): python3-tempora-5.0.0-2.el9.noarch.r 429 kB/s | 36 kB 00:00 2024-04-16T16:51:00.276 INFO:teuthology.orchestra.run.smithi117.stdout:(143/174): python3-typing-extensions-4.4.0-2.el 610 kB/s | 51 kB 00:00 2024-04-16T16:51:00.352 INFO:teuthology.orchestra.run.smithi117.stdout:(144/174): python3-webob-1.8.7-6.el9.noarch.rpm 2.2 MB/s | 230 kB 00:00 2024-04-16T16:51:00.369 INFO:teuthology.orchestra.run.smithi117.stdout:(145/174): python3-websocket-client-1.2.3-2.el9 975 kB/s | 90 kB 00:00 2024-04-16T16:51:00.469 INFO:teuthology.orchestra.run.smithi117.stdout:(146/174): python3-zc-lockfile-2.0-10.el9.noarc 200 kB/s | 20 kB 00:00 2024-04-16T16:51:00.502 INFO:teuthology.orchestra.run.smithi117.stdout:(147/174): python3-werkzeug-2.0.3-3.el9.1.noarc 2.8 MB/s | 427 kB 00:00 2024-04-16T16:51:00.504 INFO:teuthology.orchestra.run.smithi104.stdout:(123/174): libarrow-9.0.0-10.el9.x86_64.rpm 2.1 MB/s | 4.4 MB 00:02 2024-04-16T16:51:00.536 INFO:teuthology.orchestra.run.smithi117.stdout:(148/174): re2-20211101-3.el9.x86_64.rpm 2.8 MB/s | 192 kB 00:00 2024-04-16T16:51:00.579 INFO:teuthology.orchestra.run.smithi104.stdout:(124/174): python3-grpcio-1.46.7-10.el9.x86_64. 3.6 MB/s | 2.0 MB 00:00 2024-04-16T16:51:00.604 INFO:teuthology.orchestra.run.smithi104.stdout:(125/174): python3-grpcio-tools-1.46.7-10.el9.x 1.4 MB/s | 144 kB 00:00 2024-04-16T16:51:00.663 INFO:teuthology.orchestra.run.smithi104.stdout:(126/174): python3-jaraco-8.2.1-3.el9.noarch.rp 128 kB/s | 11 kB 00:00 2024-04-16T16:51:00.688 INFO:teuthology.orchestra.run.smithi104.stdout:(127/174): python3-jaraco-classes-3.2.1-5.el9.n 212 kB/s | 18 kB 00:00 2024-04-16T16:51:00.739 INFO:teuthology.orchestra.run.smithi104.stdout:(128/174): python3-jaraco-collections-3.0.0-8.e 308 kB/s | 23 kB 00:00 2024-04-16T16:51:00.772 INFO:teuthology.orchestra.run.smithi104.stdout:(129/174): python3-jaraco-functools-3.5.0-2.el9 233 kB/s | 19 kB 00:00 2024-04-16T16:51:00.822 INFO:teuthology.orchestra.run.smithi104.stdout:(130/174): python3-jaraco-text-3.2.0-6.el9.noar 235 kB/s | 20 kB 00:00 2024-04-16T16:51:00.895 INFO:teuthology.orchestra.run.smithi117.stdout:(149/174): thrift-0.15.0-2.el9.x86_64.rpm 4.0 MB/s | 1.6 MB 00:00 2024-04-16T16:51:00.922 INFO:teuthology.orchestra.run.smithi104.stdout:(131/174): python3-logutils-0.3.5-21.el9.noarch 463 kB/s | 46 kB 00:00 2024-04-16T16:51:00.998 INFO:teuthology.orchestra.run.smithi104.stdout:(132/174): python3-more-itertools-8.12.0-2.el9. 1.0 MB/s | 79 kB 00:00 2024-04-16T16:51:01.040 INFO:teuthology.orchestra.run.smithi104.stdout:(133/174): python3-kubernetes-26.1.0-2.el9.noar 3.8 MB/s | 1.0 MB 00:00 2024-04-16T16:51:01.076 INFO:teuthology.orchestra.run.smithi045.stdout:(173/174): librbd1-19.0.0-3101.g9b42bfbe.el9.x8 586 kB/s | 3.1 MB 00:05 2024-04-16T16:51:01.082 INFO:teuthology.orchestra.run.smithi104.stdout:(134/174): python3-natsort-7.1.1-5.el9.noarch.r 689 kB/s | 58 kB 00:00 2024-04-16T16:51:01.134 INFO:teuthology.orchestra.run.smithi104.stdout:(135/174): python3-pecan-1.4.2-3.el9.noarch.rpm 2.9 MB/s | 272 kB 00:00 2024-04-16T16:51:01.157 INFO:teuthology.orchestra.run.smithi104.stdout:(136/174): python3-portend-3.1.0-2.el9.noarch.r 218 kB/s | 16 kB 00:00 2024-04-16T16:51:01.199 INFO:teuthology.orchestra.run.smithi104.stdout:(137/174): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.3 MB/s | 90 kB 00:00 2024-04-16T16:51:01.225 INFO:teuthology.orchestra.run.smithi104.stdout:(138/174): python3-pyperclip-1.8.2-2.el9.noarch 422 kB/s | 28 kB 00:00 2024-04-16T16:51:01.275 INFO:teuthology.orchestra.run.smithi104.stdout:(139/174): python3-repoze-lru-0.7-16.el9.noarch 409 kB/s | 31 kB 00:00 2024-04-16T16:51:01.333 INFO:teuthology.orchestra.run.smithi104.stdout:(140/174): python3-routes-2.5.1-5.el9.noarch.rp 1.7 MB/s | 188 kB 00:00 2024-04-16T16:51:01.375 INFO:teuthology.orchestra.run.smithi104.stdout:(141/174): python3-rsa-4.9-2.el9.noarch.rpm 590 kB/s | 59 kB 00:00 2024-04-16T16:51:01.400 INFO:teuthology.orchestra.run.smithi104.stdout:(142/174): python3-tempora-5.0.0-2.el9.noarch.r 536 kB/s | 36 kB 00:00 2024-04-16T16:51:01.459 INFO:teuthology.orchestra.run.smithi104.stdout:(143/174): python3-typing-extensions-4.4.0-2.el 611 kB/s | 51 kB 00:00 2024-04-16T16:51:01.484 INFO:teuthology.orchestra.run.smithi104.stdout:(144/174): python3-webob-1.8.7-6.el9.noarch.rpm 2.7 MB/s | 230 kB 00:00 2024-04-16T16:51:01.535 INFO:teuthology.orchestra.run.smithi104.stdout:(145/174): python3-websocket-client-1.2.3-2.el9 1.2 MB/s | 90 kB 00:00 2024-04-16T16:51:01.635 INFO:teuthology.orchestra.run.smithi104.stdout:(146/174): python3-zc-lockfile-2.0-10.el9.noarc 200 kB/s | 20 kB 00:00 2024-04-16T16:51:01.660 INFO:teuthology.orchestra.run.smithi104.stdout:(147/174): python3-werkzeug-2.0.3-3.el9.1.noarc 2.4 MB/s | 427 kB 00:00 2024-04-16T16:51:01.719 INFO:teuthology.orchestra.run.smithi104.stdout:(148/174): re2-20211101-3.el9.x86_64.rpm 2.2 MB/s | 192 kB 00:00 2024-04-16T16:51:01.893 INFO:teuthology.orchestra.run.smithi045.stdout:(174/174): ceph-test-19.0.0-3101.g9b42bfbe.el9. 1.3 MB/s | 49 MB 00:38 2024-04-16T16:51:01.906 INFO:teuthology.orchestra.run.smithi045.stdout:-------------------------------------------------------------------------------- 2024-04-16T16:51:01.907 INFO:teuthology.orchestra.run.smithi045.stdout:Total 4.6 MB/s | 229 MB 00:50 2024-04-16T16:51:02.270 INFO:teuthology.orchestra.run.smithi104.stdout:(149/174): ceph-test-19.0.0-3101.g9b42bfbe.el9. 1.6 MB/s | 49 MB 00:30 2024-04-16T16:51:02.311 INFO:teuthology.orchestra.run.smithi104.stdout:(150/174): thrift-0.15.0-2.el9.x86_64.rpm 2.4 MB/s | 1.6 MB 00:00 2024-04-16T16:51:02.420 INFO:teuthology.orchestra.run.smithi117.stdout:(150/174): librados2-19.0.0-3101.g9b42bfbe.el9. 1.8 MB/s | 3.4 MB 00:01 2024-04-16T16:51:02.437 INFO:teuthology.orchestra.run.smithi104.stdout:(151/174): NetworkManager-1.47.4-1.el9.x86_64.r 18 MB/s | 2.3 MB 00:00 2024-04-16T16:51:02.528 INFO:teuthology.orchestra.run.smithi117.stdout:(151/174): librbd1-19.0.0-3101.g9b42bfbe.el9.x8 1.9 MB/s | 3.1 MB 00:01 2024-04-16T16:51:02.603 INFO:teuthology.orchestra.run.smithi104.stdout:(152/174): NetworkManager-libnm-1.47.4-1.el9.x8 11 MB/s | 1.8 MB 00:00 2024-04-16T16:51:02.629 INFO:teuthology.orchestra.run.smithi117.stdout:(152/174): NetworkManager-1.47.4-1.el9.x86_64.r 11 MB/s | 2.3 MB 00:00 2024-04-16T16:51:02.645 INFO:teuthology.orchestra.run.smithi104.stdout:(153/174): NetworkManager-team-1.47.4-1.el9.x86 984 kB/s | 41 kB 00:00 2024-04-16T16:51:02.688 INFO:teuthology.orchestra.run.smithi117.stdout:(153/174): NetworkManager-libnm-1.47.4-1.el9.x8 12 MB/s | 1.8 MB 00:00 2024-04-16T16:51:02.696 INFO:teuthology.orchestra.run.smithi104.stdout:(154/174): NetworkManager-tui-1.47.4-1.el9.x86_ 4.8 MB/s | 248 kB 00:00 2024-04-16T16:51:02.712 INFO:teuthology.orchestra.run.smithi117.stdout:(154/174): NetworkManager-team-1.47.4-1.el9.x86 506 kB/s | 41 kB 00:00 2024-04-16T16:51:02.738 INFO:teuthology.orchestra.run.smithi117.stdout:(155/174): NetworkManager-tui-1.47.4-1.el9.x86_ 4.9 MB/s | 248 kB 00:00 2024-04-16T16:51:02.746 INFO:teuthology.orchestra.run.smithi104.stdout:(155/174): xfsdump-3.1.12-4.el9.x86_64.rpm 6.3 MB/s | 321 kB 00:00 2024-04-16T16:51:02.771 INFO:teuthology.orchestra.run.smithi117.stdout:(156/174): xfsdump-3.1.12-4.el9.x86_64.rpm 5.4 MB/s | 321 kB 00:00 2024-04-16T16:51:02.830 INFO:teuthology.orchestra.run.smithi104.stdout:(156/174): librbd1-19.0.0-3101.g9b42bfbe.el9.x8 5.6 MB/s | 3.1 MB 00:00 2024-04-16T16:51:02.838 INFO:teuthology.orchestra.run.smithi117.stdout:(157/174): acl-2.3.1-4.el9.x86_64.rpm 714 kB/s | 71 kB 00:00 2024-04-16T16:51:02.855 INFO:teuthology.orchestra.run.smithi104.stdout:(157/174): acl-2.3.1-4.el9.x86_64.rpm 658 kB/s | 71 kB 00:00 2024-04-16T16:51:02.864 INFO:teuthology.orchestra.run.smithi117.stdout:(158/174): elfutils-debuginfod-client-0.190-2.e 397 kB/s | 36 kB 00:00 2024-04-16T16:51:02.897 INFO:teuthology.orchestra.run.smithi104.stdout:(158/174): elfutils-debuginfod-client-0.190-2.e 546 kB/s | 36 kB 00:00 2024-04-16T16:51:02.914 INFO:teuthology.orchestra.run.smithi117.stdout:(159/174): elfutils-libelf-0.190-2.el9.x86_64.r 2.5 MB/s | 192 kB 00:00 2024-04-16T16:51:02.922 INFO:teuthology.orchestra.run.smithi104.stdout:(159/174): elfutils-libelf-0.190-2.el9.x86_64.r 2.8 MB/s | 192 kB 00:00 2024-04-16T16:51:02.956 INFO:teuthology.orchestra.run.smithi117.stdout:(160/174): libacl-2.3.1-4.el9.x86_64.rpm 554 kB/s | 23 kB 00:00 2024-04-16T16:51:02.964 INFO:teuthology.orchestra.run.smithi104.stdout:(160/174): libacl-2.3.1-4.el9.x86_64.rpm 555 kB/s | 23 kB 00:00 2024-04-16T16:51:02.981 INFO:teuthology.orchestra.run.smithi117.stdout:(161/174): elfutils-libs-0.190-2.el9.x86_64.rpm 2.1 MB/s | 256 kB 00:00 2024-04-16T16:51:02.989 INFO:teuthology.orchestra.run.smithi104.stdout:(161/174): elfutils-libs-0.190-2.el9.x86_64.rpm 2.7 MB/s | 256 kB 00:00 2024-04-16T16:51:03.006 INFO:teuthology.orchestra.run.smithi117.stdout:(162/174): libblkid-2.37.4-18.el9.x86_64.rpm 2.1 MB/s | 108 kB 00:00 2024-04-16T16:51:03.014 INFO:teuthology.orchestra.run.smithi104.stdout:(162/174): libblkid-2.37.4-18.el9.x86_64.rpm 2.1 MB/s | 108 kB 00:00 2024-04-16T16:51:03.032 INFO:teuthology.orchestra.run.smithi117.stdout:(163/174): libfdisk-2.37.4-18.el9.x86_64.rpm 3.0 MB/s | 155 kB 00:00 2024-04-16T16:51:03.039 INFO:teuthology.orchestra.run.smithi104.stdout:(163/174): libfdisk-2.37.4-18.el9.x86_64.rpm 3.0 MB/s | 155 kB 00:00 2024-04-16T16:51:03.057 INFO:teuthology.orchestra.run.smithi117.stdout:(164/174): libmount-2.37.4-18.el9.x86_64.rpm 2.6 MB/s | 136 kB 00:00 2024-04-16T16:51:03.065 INFO:teuthology.orchestra.run.smithi104.stdout:(164/174): libmount-2.37.4-18.el9.x86_64.rpm 2.6 MB/s | 136 kB 00:00 2024-04-16T16:51:03.073 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction check 2024-04-16T16:51:03.082 INFO:teuthology.orchestra.run.smithi117.stdout:(165/174): libsmartcols-2.37.4-18.el9.x86_64.rp 1.2 MB/s | 64 kB 00:00 2024-04-16T16:51:03.099 INFO:teuthology.orchestra.run.smithi117.stdout:(166/174): libuuid-2.37.4-18.el9.x86_64.rpm 686 kB/s | 29 kB 00:00 2024-04-16T16:51:03.107 INFO:teuthology.orchestra.run.smithi104.stdout:(165/174): libuuid-2.37.4-18.el9.x86_64.rpm 690 kB/s | 29 kB 00:00 2024-04-16T16:51:03.132 INFO:teuthology.orchestra.run.smithi104.stdout:(166/174): libsmartcols-2.37.4-18.el9.x86_64.rp 695 kB/s | 64 kB 00:00 2024-04-16T16:51:03.166 INFO:teuthology.orchestra.run.smithi117.stdout:(167/174): openssl-3.0.7-27.el9.x86_64.rpm 15 MB/s | 1.2 MB 00:00 2024-04-16T16:51:03.199 INFO:teuthology.orchestra.run.smithi104.stdout:(167/174): openssl-3.0.7-27.el9.x86_64.rpm 13 MB/s | 1.2 MB 00:00 2024-04-16T16:51:03.225 INFO:teuthology.orchestra.run.smithi117.stdout:(168/174): openssl-libs-3.0.7-27.el9.x86_64.rpm 17 MB/s | 2.2 MB 00:00 2024-04-16T16:51:03.266 INFO:teuthology.orchestra.run.smithi104.stdout:(168/174): openssl-libs-3.0.7-27.el9.x86_64.rpm 16 MB/s | 2.2 MB 00:00 2024-04-16T16:51:03.275 INFO:teuthology.orchestra.run.smithi117.stdout:(169/174): util-linux-2.37.4-18.el9.x86_64.rpm 21 MB/s | 2.3 MB 00:00 2024-04-16T16:51:03.300 INFO:teuthology.orchestra.run.smithi117.stdout:(170/174): util-linux-core-2.37.4-18.el9.x86_64 6.0 MB/s | 465 kB 00:00 2024-04-16T16:51:03.324 INFO:teuthology.orchestra.run.smithi104.stdout:(169/174): util-linux-2.37.4-18.el9.x86_64.rpm 18 MB/s | 2.3 MB 00:00 2024-04-16T16:51:03.325 INFO:teuthology.orchestra.run.smithi117.stdout:(171/174): util-linux-user-2.37.4-18.el9.x86_64 621 kB/s | 31 kB 00:00 2024-04-16T16:51:03.350 INFO:teuthology.orchestra.run.smithi104.stdout:(170/174): util-linux-core-2.37.4-18.el9.x86_64 5.5 MB/s | 465 kB 00:00 2024-04-16T16:51:03.359 INFO:teuthology.orchestra.run.smithi117.stdout:(172/174): xfsprogs-6.3.0-1.el9.x86_64.rpm 18 MB/s | 1.1 MB 00:00 2024-04-16T16:51:03.375 INFO:teuthology.orchestra.run.smithi104.stdout:(171/174): util-linux-user-2.37.4-18.el9.x86_64 621 kB/s | 31 kB 00:00 2024-04-16T16:51:03.384 INFO:teuthology.orchestra.run.smithi117.stdout:(173/174): libacl-devel-2.3.1-4.el9.x86_64.rpm 1.2 MB/s | 74 kB 00:00 2024-04-16T16:51:03.409 INFO:teuthology.orchestra.run.smithi104.stdout:(172/174): xfsprogs-6.3.0-1.el9.x86_64.rpm 18 MB/s | 1.1 MB 00:00 2024-04-16T16:51:03.467 INFO:teuthology.orchestra.run.smithi104.stdout:(173/174): libacl-devel-2.3.1-4.el9.x86_64.rpm 810 kB/s | 74 kB 00:00 2024-04-16T16:51:03.481 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction check succeeded. 2024-04-16T16:51:03.481 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction test 2024-04-16T16:51:04.500 INFO:teuthology.orchestra.run.smithi104.stdout:(174/174): librados2-19.0.0-3101.g9b42bfbe.el9. 1.2 MB/s | 3.4 MB 00:02 2024-04-16T16:51:04.506 INFO:teuthology.orchestra.run.smithi104.stdout:-------------------------------------------------------------------------------- 2024-04-16T16:51:04.506 INFO:teuthology.orchestra.run.smithi104.stdout:Total 4.8 MB/s | 229 MB 00:47 2024-04-16T16:51:04.866 INFO:teuthology.orchestra.run.smithi104.stdout:Running transaction check 2024-04-16T16:51:05.135 INFO:teuthology.orchestra.run.smithi104.stdout:Transaction check succeeded. 2024-04-16T16:51:05.135 INFO:teuthology.orchestra.run.smithi104.stdout:Running transaction test 2024-04-16T16:51:05.257 INFO:teuthology.orchestra.run.smithi117.stdout:(174/174): ceph-test-19.0.0-3101.g9b42bfbe.el9. 1.3 MB/s | 49 MB 00:37 2024-04-16T16:51:05.263 INFO:teuthology.orchestra.run.smithi117.stdout:-------------------------------------------------------------------------------- 2024-04-16T16:51:05.263 INFO:teuthology.orchestra.run.smithi117.stdout:Total 4.3 MB/s | 229 MB 00:53 2024-04-16T16:51:05.641 INFO:teuthology.orchestra.run.smithi117.stdout:Running transaction check 2024-04-16T16:51:05.913 INFO:teuthology.orchestra.run.smithi117.stdout:Transaction check succeeded. 2024-04-16T16:51:05.913 INFO:teuthology.orchestra.run.smithi117.stdout:Running transaction test 2024-04-16T16:51:05.968 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction test succeeded. 2024-04-16T16:51:05.969 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction 2024-04-16T16:51:06.865 INFO:teuthology.orchestra.run.smithi104.stdout:Transaction test succeeded. 2024-04-16T16:51:06.865 INFO:teuthology.orchestra.run.smithi104.stdout:Running transaction 2024-04-16T16:51:07.720 INFO:teuthology.orchestra.run.smithi117.stdout:Transaction test succeeded. 2024-04-16T16:51:07.721 INFO:teuthology.orchestra.run.smithi117.stdout:Running transaction 2024-04-16T16:51:13.627 INFO:teuthology.orchestra.run.smithi045.stdout: Preparing : 1/1 2024-04-16T16:51:13.682 INFO:teuthology.orchestra.run.smithi104.stdout: Preparing : 1/1 2024-04-16T16:51:13.716 INFO:teuthology.orchestra.run.smithi045.stdout: Upgrading : openssl-libs-1:3.0.7-27.el9.x86_64 1/198 2024-04-16T16:51:13.763 INFO:teuthology.orchestra.run.smithi045.stdout: Upgrading : libuuid-2.37.4-18.el9.x86_64 2/198 2024-04-16T16:51:13.795 INFO:teuthology.orchestra.run.smithi104.stdout: Upgrading : openssl-libs-1:3.0.7-27.el9.x86_64 1/198 2024-04-16T16:51:13.803 INFO:teuthology.orchestra.run.smithi045.stdout: Upgrading : libblkid-2.37.4-18.el9.x86_64 3/198 2024-04-16T16:51:13.830 INFO:teuthology.orchestra.run.smithi104.stdout: Upgrading : libuuid-2.37.4-18.el9.x86_64 2/198 2024-04-16T16:51:13.847 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: libblkid-2.37.4-18.el9.x86_64 3/198 2024-04-16T16:51:13.860 INFO:teuthology.orchestra.run.smithi104.stdout: Upgrading : libblkid-2.37.4-18.el9.x86_64 3/198 2024-04-16T16:51:13.869 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: libblkid-2.37.4-18.el9.x86_64 3/198 2024-04-16T16:51:13.918 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 4/198 2024-04-16T16:51:13.946 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 4/198 2024-04-16T16:51:13.972 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 5/198 2024-04-16T16:51:14.005 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/198 2024-04-16T16:51:14.031 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 7/198 2024-04-16T16:51:14.067 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 5/198 2024-04-16T16:51:14.073 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 8/198 2024-04-16T16:51:14.133 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/198 2024-04-16T16:51:14.170 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 7/198 2024-04-16T16:51:14.269 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 8/198 2024-04-16T16:51:14.392 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 9/198 2024-04-16T16:51:14.449 INFO:teuthology.orchestra.run.smithi104.stdout: Upgrading : librados2-2:19.0.0-3101.g9b42bfbe.el9.x86_64 10/198 2024-04-16T16:51:14.569 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: librados2-2:19.0.0-3101.g9b42bfbe.el9.x86_64 10/198 2024-04-16T16:51:14.591 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : libcephfs2-2:19.0.0-3101.g9b42bfbe.el9.x86_64 11/198 2024-04-16T16:51:14.682 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: libcephfs2-2:19.0.0-3101.g9b42bfbe.el9.x86_64 11/198 2024-04-16T16:51:14.683 INFO:teuthology.orchestra.run.smithi117.stdout: Preparing : 1/1 2024-04-16T16:51:14.714 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-rados-2:19.0.0-3101.g9b42bfbe.el9.x86_64 12/198 2024-04-16T16:51:14.725 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 9/198 2024-04-16T16:51:14.752 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 13/198 2024-04-16T16:51:14.788 INFO:teuthology.orchestra.run.smithi117.stdout: Upgrading : openssl-libs-1:3.0.7-27.el9.x86_64 1/198 2024-04-16T16:51:14.801 INFO:teuthology.orchestra.run.smithi045.stdout: Upgrading : librados2-2:19.0.0-3101.g9b42bfbe.el9.x86_64 10/198 2024-04-16T16:51:14.821 INFO:teuthology.orchestra.run.smithi117.stdout: Upgrading : libuuid-2.37.4-18.el9.x86_64 2/198 2024-04-16T16:51:14.861 INFO:teuthology.orchestra.run.smithi117.stdout: Upgrading : libblkid-2.37.4-18.el9.x86_64 3/198 2024-04-16T16:51:14.871 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: libblkid-2.37.4-18.el9.x86_64 3/198 2024-04-16T16:51:14.910 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 4/198 2024-04-16T16:51:14.913 INFO:teuthology.orchestra.run.smithi104.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 14/198 2024-04-16T16:51:14.963 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 5/198 2024-04-16T16:51:14.970 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : postgresql-13.14-1.el9.x86_64 15/198 2024-04-16T16:51:14.973 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: librados2-2:19.0.0-3101.g9b42bfbe.el9.x86_64 10/198 2024-04-16T16:51:14.997 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/198 2024-04-16T16:51:14.998 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : libcephsqlite-2:19.0.0-3101.g9b42bfbe.el9.x86_64 16/198 2024-04-16T16:51:15.010 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libcephfs2-2:19.0.0-3101.g9b42bfbe.el9.x86_64 11/198 2024-04-16T16:51:15.023 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 7/198 2024-04-16T16:51:15.065 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 8/198 2024-04-16T16:51:15.101 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3101.g9b42bfbe.el9.x86_64 16/198 2024-04-16T16:51:15.124 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : libradosstriper1-2:19.0.0-3101.g9b42bfbe.el9.x86 17/198 2024-04-16T16:51:15.138 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: libcephfs2-2:19.0.0-3101.g9b42bfbe.el9.x86_64 11/198 2024-04-16T16:51:15.176 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-rados-2:19.0.0-3101.g9b42bfbe.el9.x86_64 12/198 2024-04-16T16:51:15.188 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3101.g9b42bfbe.el9.x86 17/198 2024-04-16T16:51:15.227 INFO:teuthology.orchestra.run.smithi104.stdout: Upgrading : libmount-2.37.4-18.el9.x86_64 18/198 2024-04-16T16:51:15.250 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 13/198 2024-04-16T16:51:15.254 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 19/198 2024-04-16T16:51:15.265 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 19/198 2024-04-16T16:51:15.385 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 9/198 2024-04-16T16:51:15.441 INFO:teuthology.orchestra.run.smithi117.stdout: Upgrading : librados2-2:19.0.0-3101.g9b42bfbe.el9.x86_64 10/198 2024-04-16T16:51:15.450 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 20/198 2024-04-16T16:51:15.481 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 20/198 2024-04-16T16:51:15.522 INFO:teuthology.orchestra.run.smithi104.stdout: Upgrading : libsmartcols-2.37.4-18.el9.x86_64 21/198 2024-04-16T16:51:15.555 INFO:teuthology.orchestra.run.smithi045.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 14/198 2024-04-16T16:51:15.561 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: librados2-2:19.0.0-3101.g9b42bfbe.el9.x86_64 10/198 2024-04-16T16:51:15.592 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : libcephfs2-2:19.0.0-3101.g9b42bfbe.el9.x86_64 11/198 2024-04-16T16:51:15.616 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : postgresql-13.14-1.el9.x86_64 15/198 2024-04-16T16:51:15.627 INFO:teuthology.orchestra.run.smithi104.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 22/198 2024-04-16T16:51:15.641 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libcephsqlite-2:19.0.0-3101.g9b42bfbe.el9.x86_64 16/198 2024-04-16T16:51:15.682 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: libcephfs2-2:19.0.0-3101.g9b42bfbe.el9.x86_64 11/198 2024-04-16T16:51:15.713 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-rados-2:19.0.0-3101.g9b42bfbe.el9.x86_64 12/198 2024-04-16T16:51:15.728 INFO:teuthology.orchestra.run.smithi104.stdout: Upgrading : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 23/198 2024-04-16T16:51:15.752 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 13/198 2024-04-16T16:51:15.776 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 23/198 2024-04-16T16:51:15.808 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3101.g9b42bfbe.el9.x86_64 16/198 2024-04-16T16:51:15.834 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libradosstriper1-2:19.0.0-3101.g9b42bfbe.el9.x86 17/198 2024-04-16T16:51:15.837 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 24/198 2024-04-16T16:51:15.893 INFO:teuthology.orchestra.run.smithi104.stdout: Upgrading : NetworkManager-1:1.47.4-1.el9.x86_64 24/198 2024-04-16T16:51:15.929 INFO:teuthology.orchestra.run.smithi117.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 14/198 2024-04-16T16:51:15.967 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3101.g9b42bfbe.el9.x86 17/198 2024-04-16T16:51:15.977 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : postgresql-13.14-1.el9.x86_64 15/198 2024-04-16T16:51:16.005 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : libcephsqlite-2:19.0.0-3101.g9b42bfbe.el9.x86_64 16/198 2024-04-16T16:51:16.029 INFO:teuthology.orchestra.run.smithi045.stdout: Upgrading : libmount-2.37.4-18.el9.x86_64 18/198 2024-04-16T16:51:16.061 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 19/198 2024-04-16T16:51:16.100 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 24/198 2024-04-16T16:51:16.100 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3101.g9b42bfbe.el9.x86_64 16/198 2024-04-16T16:51:16.123 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : libradosstriper1-2:19.0.0-3101.g9b42bfbe.el9.x86 17/198 2024-04-16T16:51:16.135 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 19/198 2024-04-16T16:51:16.170 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3101.g9b42bfbe.el9.x86 17/198 2024-04-16T16:51:16.201 INFO:teuthology.orchestra.run.smithi117.stdout: Upgrading : libmount-2.37.4-18.el9.x86_64 18/198 2024-04-16T16:51:16.210 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : re2-1:20211101-3.el9.x86_64 25/198 2024-04-16T16:51:16.224 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 19/198 2024-04-16T16:51:16.240 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 19/198 2024-04-16T16:51:16.303 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 26/198 2024-04-16T16:51:16.356 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 27/198 2024-04-16T16:51:16.383 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : liboath-2.6.7-2.el9.x86_64 28/198 2024-04-16T16:51:16.407 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 20/198 2024-04-16T16:51:16.418 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/198 2024-04-16T16:51:16.438 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 20/198 2024-04-16T16:51:16.463 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 20/198 2024-04-16T16:51:16.470 INFO:teuthology.orchestra.run.smithi117.stdout: Upgrading : libsmartcols-2.37.4-18.el9.x86_64 21/198 2024-04-16T16:51:16.493 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/198 2024-04-16T16:51:16.519 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 20/198 2024-04-16T16:51:16.547 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 31/198 2024-04-16T16:51:16.562 INFO:teuthology.orchestra.run.smithi045.stdout: Upgrading : libsmartcols-2.37.4-18.el9.x86_64 21/198 2024-04-16T16:51:16.566 INFO:teuthology.orchestra.run.smithi117.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 22/198 2024-04-16T16:51:16.585 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 32/198 2024-04-16T16:51:16.649 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 33/198 2024-04-16T16:51:16.672 INFO:teuthology.orchestra.run.smithi117.stdout: Upgrading : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 23/198 2024-04-16T16:51:16.683 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 34/198 2024-04-16T16:51:16.716 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 23/198 2024-04-16T16:51:16.727 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 35/198 2024-04-16T16:51:16.760 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 36/198 2024-04-16T16:51:16.778 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 24/198 2024-04-16T16:51:16.788 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : lua-5.4.4-4.el9.x86_64 37/198 2024-04-16T16:51:16.842 INFO:teuthology.orchestra.run.smithi117.stdout: Upgrading : NetworkManager-1:1.47.4-1.el9.x86_64 24/198 2024-04-16T16:51:16.857 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 38/198 2024-04-16T16:51:16.892 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-ceph-common-2:19.0.0-3101.g9b42bfbe.el9. 39/198 2024-04-16T16:51:16.899 INFO:teuthology.orchestra.run.smithi045.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 22/198 2024-04-16T16:51:16.936 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-ceph-argparse-2:19.0.0-3101.g9b42bfbe.el 40/198 2024-04-16T16:51:16.965 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-cephfs-2:19.0.0-3101.g9b42bfbe.el9.x86_6 41/198 2024-04-16T16:51:17.035 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 42/198 2024-04-16T16:51:17.062 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 24/198 2024-04-16T16:51:17.104 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : luarocks-3.9.2-1.el9.noarch 43/198 2024-04-16T16:51:17.119 INFO:teuthology.orchestra.run.smithi045.stdout: Upgrading : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 23/198 2024-04-16T16:51:17.175 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : re2-1:20211101-3.el9.x86_64 25/198 2024-04-16T16:51:17.178 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 44/198 2024-04-16T16:51:17.208 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 23/198 2024-04-16T16:51:17.270 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 26/198 2024-04-16T16:51:17.315 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 45/198 2024-04-16T16:51:17.315 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 27/198 2024-04-16T16:51:17.365 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 46/198 2024-04-16T16:51:17.403 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 24/198 2024-04-16T16:51:17.422 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : liboath-2.6.7-2.el9.x86_64 28/198 2024-04-16T16:51:17.464 INFO:teuthology.orchestra.run.smithi045.stdout: Upgrading : NetworkManager-1:1.47.4-1.el9.x86_64 24/198 2024-04-16T16:51:17.568 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/198 2024-04-16T16:51:17.718 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/198 2024-04-16T16:51:17.772 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 31/198 2024-04-16T16:51:17.782 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 24/198 2024-04-16T16:51:17.810 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 32/198 2024-04-16T16:51:17.881 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 33/198 2024-04-16T16:51:17.894 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-rsa-4.9-2.el9.noarch 47/198 2024-04-16T16:51:17.916 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 34/198 2024-04-16T16:51:17.952 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : re2-1:20211101-3.el9.x86_64 25/198 2024-04-16T16:51:17.960 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 35/198 2024-04-16T16:51:17.968 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 48/198 2024-04-16T16:51:17.992 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 36/198 2024-04-16T16:51:18.020 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : lua-5.4.4-4.el9.x86_64 37/198 2024-04-16T16:51:18.043 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 49/198 2024-04-16T16:51:18.079 INFO:teuthology.orchestra.run.smithi104.stdout: Upgrading : util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-04-16T16:51:18.080 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 26/198 2024-04-16T16:51:18.089 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-04-16T16:51:18.090 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 38/198 2024-04-16T16:51:18.120 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 51/198 2024-04-16T16:51:18.120 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 27/198 2024-04-16T16:51:18.125 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-ceph-common-2:19.0.0-3101.g9b42bfbe.el9. 39/198 2024-04-16T16:51:18.156 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : liboath-2.6.7-2.el9.x86_64 28/198 2024-04-16T16:51:18.160 INFO:teuthology.orchestra.run.smithi104.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 52/198 2024-04-16T16:51:18.169 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-ceph-argparse-2:19.0.0-3101.g9b42bfbe.el 40/198 2024-04-16T16:51:18.189 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-cephfs-2:19.0.0-3101.g9b42bfbe.el9.x86_6 41/198 2024-04-16T16:51:18.198 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : librados-devel-2:19.0.0-3101.g9b42bfbe.el9.x86_6 53/198 2024-04-16T16:51:18.203 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/198 2024-04-16T16:51:18.228 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 54/198 2024-04-16T16:51:18.258 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 42/198 2024-04-16T16:51:18.304 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : luarocks-3.9.2-1.el9.noarch 43/198 2024-04-16T16:51:18.319 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/198 2024-04-16T16:51:18.385 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 44/198 2024-04-16T16:51:18.387 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 31/198 2024-04-16T16:51:18.438 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 32/198 2024-04-16T16:51:18.460 INFO:teuthology.orchestra.run.smithi104.stdout: Upgrading : libfdisk-2.37.4-18.el9.x86_64 55/198 2024-04-16T16:51:18.522 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 45/198 2024-04-16T16:51:18.525 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 33/198 2024-04-16T16:51:18.577 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 34/198 2024-04-16T16:51:18.606 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 46/198 2024-04-16T16:51:18.628 INFO:teuthology.orchestra.run.smithi104.stdout: Upgrading : util-linux-2.37.4-18.el9.x86_64 56/198 2024-04-16T16:51:18.638 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 35/198 2024-04-16T16:51:18.694 INFO:teuthology.orchestra.run.smithi104.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 57/198 2024-04-16T16:51:18.715 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 36/198 2024-04-16T16:51:18.730 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 58/198 2024-04-16T16:51:18.748 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : lua-5.4.4-4.el9.x86_64 37/198 2024-04-16T16:51:18.868 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 59/198 2024-04-16T16:51:18.874 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 38/198 2024-04-16T16:51:18.916 INFO:teuthology.orchestra.run.smithi104.stdout: Upgrading : openssl-1:3.0.7-27.el9.x86_64 60/198 2024-04-16T16:51:18.919 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-ceph-common-2:19.0.0-3101.g9b42bfbe.el9. 39/198 2024-04-16T16:51:18.971 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 61/198 2024-04-16T16:51:18.978 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-ceph-argparse-2:19.0.0-3101.g9b42bfbe.el 40/198 2024-04-16T16:51:19.009 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-cephfs-2:19.0.0-3101.g9b42bfbe.el9.x86_6 41/198 2024-04-16T16:51:19.043 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 62/198 2024-04-16T16:51:19.075 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 63/198 2024-04-16T16:51:19.111 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 64/198 2024-04-16T16:51:19.129 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-rsa-4.9-2.el9.noarch 47/198 2024-04-16T16:51:19.157 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 65/198 2024-04-16T16:51:19.171 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 42/198 2024-04-16T16:51:19.194 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 66/198 2024-04-16T16:51:19.210 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 48/198 2024-04-16T16:51:19.237 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 67/198 2024-04-16T16:51:19.241 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : luarocks-3.9.2-1.el9.noarch 43/198 2024-04-16T16:51:19.280 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 68/198 2024-04-16T16:51:19.286 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 49/198 2024-04-16T16:51:19.330 INFO:teuthology.orchestra.run.smithi117.stdout: Upgrading : util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-04-16T16:51:19.339 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-04-16T16:51:19.367 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 44/198 2024-04-16T16:51:19.370 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 51/198 2024-04-16T16:51:19.414 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 69/198 2024-04-16T16:51:19.418 INFO:teuthology.orchestra.run.smithi117.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 52/198 2024-04-16T16:51:19.457 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : librados-devel-2:19.0.0-3101.g9b42bfbe.el9.x86_6 53/198 2024-04-16T16:51:19.460 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 70/198 2024-04-16T16:51:19.486 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 54/198 2024-04-16T16:51:19.490 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 71/198 2024-04-16T16:51:19.531 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 72/198 2024-04-16T16:51:19.558 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 45/198 2024-04-16T16:51:19.626 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 73/198 2024-04-16T16:51:19.632 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 46/198 2024-04-16T16:51:19.673 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-google-auth-1:2.28.2-1.el9.noarch 74/198 2024-04-16T16:51:19.700 INFO:teuthology.orchestra.run.smithi117.stdout: Upgrading : libfdisk-2.37.4-18.el9.x86_64 55/198 2024-04-16T16:51:19.713 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 75/198 2024-04-16T16:51:19.739 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 76/198 2024-04-16T16:51:19.763 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 77/198 2024-04-16T16:51:19.837 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 78/198 2024-04-16T16:51:19.859 INFO:teuthology.orchestra.run.smithi117.stdout: Upgrading : util-linux-2.37.4-18.el9.x86_64 56/198 2024-04-16T16:51:19.923 INFO:teuthology.orchestra.run.smithi117.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 57/198 2024-04-16T16:51:19.963 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 79/198 2024-04-16T16:51:19.967 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 58/198 2024-04-16T16:51:20.079 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 80/198 2024-04-16T16:51:20.092 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 59/198 2024-04-16T16:51:20.121 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 81/198 2024-04-16T16:51:20.162 INFO:teuthology.orchestra.run.smithi117.stdout: Upgrading : openssl-1:3.0.7-27.el9.x86_64 60/198 2024-04-16T16:51:20.170 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-ply-3.11-14.el9.noarch 82/198 2024-04-16T16:51:20.217 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 61/198 2024-04-16T16:51:20.226 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 83/198 2024-04-16T16:51:20.289 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 62/198 2024-04-16T16:51:20.320 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 63/198 2024-04-16T16:51:20.357 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 64/198 2024-04-16T16:51:20.366 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-rsa-4.9-2.el9.noarch 47/198 2024-04-16T16:51:20.404 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 65/198 2024-04-16T16:51:20.411 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 84/198 2024-04-16T16:51:20.440 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 66/198 2024-04-16T16:51:20.458 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 85/198 2024-04-16T16:51:20.483 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 67/198 2024-04-16T16:51:20.483 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 48/198 2024-04-16T16:51:20.525 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 68/198 2024-04-16T16:51:20.590 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 49/198 2024-04-16T16:51:20.598 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 86/198 2024-04-16T16:51:20.648 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 87/198 2024-04-16T16:51:20.660 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 69/198 2024-04-16T16:51:20.680 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 88/198 2024-04-16T16:51:20.711 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : fuse-2.9.9-15.el9.x86_64 89/198 2024-04-16T16:51:20.714 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 70/198 2024-04-16T16:51:20.739 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 90/198 2024-04-16T16:51:20.744 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 71/198 2024-04-16T16:51:20.773 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 91/198 2024-04-16T16:51:20.787 INFO:teuthology.orchestra.run.smithi045.stdout: Upgrading : util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-04-16T16:51:20.794 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 72/198 2024-04-16T16:51:20.812 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : uuid-1.6.2-55.el9.x86_64 92/198 2024-04-16T16:51:20.818 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-04-16T16:51:20.844 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 93/198 2024-04-16T16:51:20.872 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 51/198 2024-04-16T16:51:20.874 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 94/198 2024-04-16T16:51:20.888 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 73/198 2024-04-16T16:51:20.927 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-google-auth-1:2.28.2-1.el9.noarch 74/198 2024-04-16T16:51:20.950 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 95/198 2024-04-16T16:51:20.954 INFO:teuthology.orchestra.run.smithi045.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 52/198 2024-04-16T16:51:20.967 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 75/198 2024-04-16T16:51:20.992 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 96/198 2024-04-16T16:51:21.001 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 76/198 2024-04-16T16:51:21.009 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : librados-devel-2:19.0.0-3101.g9b42bfbe.el9.x86_6 53/198 2024-04-16T16:51:21.025 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 97/198 2024-04-16T16:51:21.026 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 77/198 2024-04-16T16:51:21.064 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 54/198 2024-04-16T16:51:21.067 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 98/198 2024-04-16T16:51:21.097 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 78/198 2024-04-16T16:51:21.102 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 99/198 2024-04-16T16:51:21.210 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 100/198 2024-04-16T16:51:21.226 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 79/198 2024-04-16T16:51:21.298 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 101/198 2024-04-16T16:51:21.329 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 80/198 2024-04-16T16:51:21.340 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 102/198 2024-04-16T16:51:21.367 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 81/198 2024-04-16T16:51:21.381 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 103/198 2024-04-16T16:51:21.416 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-ply-3.11-14.el9.noarch 82/198 2024-04-16T16:51:21.427 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 104/198 2024-04-16T16:51:21.471 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 83/198 2024-04-16T16:51:21.481 INFO:teuthology.orchestra.run.smithi045.stdout: Upgrading : libfdisk-2.37.4-18.el9.x86_64 55/198 2024-04-16T16:51:21.524 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-packaging-20.9-5.el9.noarch 105/198 2024-04-16T16:51:21.574 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 106/198 2024-04-16T16:51:21.664 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 84/198 2024-04-16T16:51:21.705 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 85/198 2024-04-16T16:51:21.737 INFO:teuthology.orchestra.run.smithi045.stdout: Upgrading : util-linux-2.37.4-18.el9.x86_64 56/198 2024-04-16T16:51:21.834 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 86/198 2024-04-16T16:51:21.842 INFO:teuthology.orchestra.run.smithi045.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 57/198 2024-04-16T16:51:21.898 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 87/198 2024-04-16T16:51:21.914 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 58/198 2024-04-16T16:51:21.952 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 88/198 2024-04-16T16:51:21.982 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : fuse-2.9.9-15.el9.x86_64 89/198 2024-04-16T16:51:22.011 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 90/198 2024-04-16T16:51:22.045 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 91/198 2024-04-16T16:51:22.084 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : uuid-1.6.2-55.el9.x86_64 92/198 2024-04-16T16:51:22.115 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 93/198 2024-04-16T16:51:22.125 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 59/198 2024-04-16T16:51:22.145 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 94/198 2024-04-16T16:51:22.190 INFO:teuthology.orchestra.run.smithi045.stdout: Upgrading : openssl-1:3.0.7-27.el9.x86_64 60/198 2024-04-16T16:51:22.223 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 95/198 2024-04-16T16:51:22.256 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 96/198 2024-04-16T16:51:22.289 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 97/198 2024-04-16T16:51:22.291 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 107/198 2024-04-16T16:51:22.295 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 61/198 2024-04-16T16:51:22.322 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 98/198 2024-04-16T16:51:22.337 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 108/198 2024-04-16T16:51:22.357 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 99/198 2024-04-16T16:51:22.371 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 109/198 2024-04-16T16:51:22.394 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 62/198 2024-04-16T16:51:22.420 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 110/198 2024-04-16T16:51:22.436 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 63/198 2024-04-16T16:51:22.446 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 100/198 2024-04-16T16:51:22.481 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 64/198 2024-04-16T16:51:22.507 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 111/198 2024-04-16T16:51:22.536 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 101/198 2024-04-16T16:51:22.538 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 112/198 2024-04-16T16:51:22.552 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 65/198 2024-04-16T16:51:22.580 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 102/198 2024-04-16T16:51:22.611 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 103/198 2024-04-16T16:51:22.617 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 66/198 2024-04-16T16:51:22.657 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 104/198 2024-04-16T16:51:22.688 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 67/198 2024-04-16T16:51:22.715 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : openblas-0.3.21-2.el9.x86_64 113/198 2024-04-16T16:51:22.753 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 114/198 2024-04-16T16:51:22.755 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 68/198 2024-04-16T16:51:22.764 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-packaging-20.9-5.el9.noarch 105/198 2024-04-16T16:51:22.820 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 106/198 2024-04-16T16:51:22.830 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 115/198 2024-04-16T16:51:22.968 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 69/198 2024-04-16T16:51:23.052 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 70/198 2024-04-16T16:51:23.089 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 71/198 2024-04-16T16:51:23.148 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 72/198 2024-04-16T16:51:23.293 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 73/198 2024-04-16T16:51:23.338 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-google-auth-1:2.28.2-1.el9.noarch 74/198 2024-04-16T16:51:23.418 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 75/198 2024-04-16T16:51:23.455 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 76/198 2024-04-16T16:51:23.466 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 116/198 2024-04-16T16:51:23.487 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 77/198 2024-04-16T16:51:23.548 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 107/198 2024-04-16T16:51:23.584 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 117/198 2024-04-16T16:51:23.617 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 108/198 2024-04-16T16:51:23.651 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 109/198 2024-04-16T16:51:23.664 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 78/198 2024-04-16T16:51:23.700 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 110/198 2024-04-16T16:51:23.787 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 111/198 2024-04-16T16:51:23.828 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 112/198 2024-04-16T16:51:23.858 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 79/198 2024-04-16T16:51:24.000 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 80/198 2024-04-16T16:51:24.003 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : openblas-0.3.21-2.el9.x86_64 113/198 2024-04-16T16:51:24.033 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 114/198 2024-04-16T16:51:24.057 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 81/198 2024-04-16T16:51:24.110 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 115/198 2024-04-16T16:51:24.122 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-ply-3.11-14.el9.noarch 82/198 2024-04-16T16:51:24.342 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 83/198 2024-04-16T16:51:24.609 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 84/198 2024-04-16T16:51:24.679 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 85/198 2024-04-16T16:51:24.742 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 116/198 2024-04-16T16:51:24.792 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 118/198 2024-04-16T16:51:24.846 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 117/198 2024-04-16T16:51:24.856 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 86/198 2024-04-16T16:51:24.877 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 119/198 2024-04-16T16:51:24.908 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 120/198 2024-04-16T16:51:24.909 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 87/198 2024-04-16T16:51:24.939 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : jq-1.6-16.el9.x86_64 121/198 2024-04-16T16:51:24.967 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 122/198 2024-04-16T16:51:24.975 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 88/198 2024-04-16T16:51:25.014 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : fuse-2.9.9-15.el9.x86_64 89/198 2024-04-16T16:51:25.057 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 90/198 2024-04-16T16:51:25.106 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 91/198 2024-04-16T16:51:25.181 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : uuid-1.6.2-55.el9.x86_64 92/198 2024-04-16T16:51:25.246 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 93/198 2024-04-16T16:51:25.277 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 123/198 2024-04-16T16:51:25.293 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 94/198 2024-04-16T16:51:25.305 INFO:teuthology.orchestra.run.smithi104.stdout: Upgrading : librbd1-2:19.0.0-3101.g9b42bfbe.el9.x86_64 124/198 2024-04-16T16:51:25.379 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: librbd1-2:19.0.0-3101.g9b42bfbe.el9.x86_64 124/198 2024-04-16T16:51:25.409 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-rbd-2:19.0.0-3101.g9b42bfbe.el9.x86_64 125/198 2024-04-16T16:51:25.428 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 95/198 2024-04-16T16:51:25.445 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 126/198 2024-04-16T16:51:25.471 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 96/198 2024-04-16T16:51:25.551 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 97/198 2024-04-16T16:51:25.923 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 127/198 2024-04-16T16:51:25.966 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : librgw2-2:19.0.0-3101.g9b42bfbe.el9.x86_64 128/198 2024-04-16T16:51:25.969 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 98/198 2024-04-16T16:51:26.023 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 99/198 2024-04-16T16:51:26.028 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: librgw2-2:19.0.0-3101.g9b42bfbe.el9.x86_64 128/198 2024-04-16T16:51:26.051 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-rgw-2:19.0.0-3101.g9b42bfbe.el9.x86_64 129/198 2024-04-16T16:51:26.070 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 118/198 2024-04-16T16:51:26.150 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 119/198 2024-04-16T16:51:26.181 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 120/198 2024-04-16T16:51:26.190 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 100/198 2024-04-16T16:51:26.212 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : jq-1.6-16.el9.x86_64 121/198 2024-04-16T16:51:26.240 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 122/198 2024-04-16T16:51:26.309 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 101/198 2024-04-16T16:51:26.373 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 102/198 2024-04-16T16:51:26.409 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 103/198 2024-04-16T16:51:26.475 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 104/198 2024-04-16T16:51:26.552 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 123/198 2024-04-16T16:51:26.578 INFO:teuthology.orchestra.run.smithi117.stdout: Upgrading : librbd1-2:19.0.0-3101.g9b42bfbe.el9.x86_64 124/198 2024-04-16T16:51:26.610 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-packaging-20.9-5.el9.noarch 105/198 2024-04-16T16:51:26.669 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: librbd1-2:19.0.0-3101.g9b42bfbe.el9.x86_64 124/198 2024-04-16T16:51:26.684 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 106/198 2024-04-16T16:51:26.699 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-rbd-2:19.0.0-3101.g9b42bfbe.el9.x86_64 125/198 2024-04-16T16:51:26.736 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 126/198 2024-04-16T16:51:27.215 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 127/198 2024-04-16T16:51:27.248 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : librgw2-2:19.0.0-3101.g9b42bfbe.el9.x86_64 128/198 2024-04-16T16:51:27.321 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: librgw2-2:19.0.0-3101.g9b42bfbe.el9.x86_64 128/198 2024-04-16T16:51:27.350 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-rgw-2:19.0.0-3101.g9b42bfbe.el9.x86_64 129/198 2024-04-16T16:51:27.704 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 107/198 2024-04-16T16:51:27.766 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 108/198 2024-04-16T16:51:27.799 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 109/198 2024-04-16T16:51:27.862 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 110/198 2024-04-16T16:51:27.996 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 111/198 2024-04-16T16:51:28.034 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 112/198 2024-04-16T16:51:28.287 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : openblas-0.3.21-2.el9.x86_64 113/198 2024-04-16T16:51:28.322 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 114/198 2024-04-16T16:51:28.356 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: ceph-common-2:19.0.0-3101.g9b42bfbe.el9.x86_64 130/198 2024-04-16T16:51:28.389 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : ceph-common-2:19.0.0-3101.g9b42bfbe.el9.x86_64 130/198 2024-04-16T16:51:28.429 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 115/198 2024-04-16T16:51:28.926 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: ceph-common-2:19.0.0-3101.g9b42bfbe.el9.x86_64 130/198 2024-04-16T16:51:28.975 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : ceph-base-2:19.0.0-3101.g9b42bfbe.el9.x86_64 131/198 2024-04-16T16:51:29.056 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: ceph-base-2:19.0.0-3101.g9b42bfbe.el9.x86_64 131/198 2024-04-16T16:51:29.057 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-16T16:51:29.057 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-16T16:51:29.057 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-04-16T16:51:29.085 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : ceph-selinux-2:19.0.0-3101.g9b42bfbe.el9.x86_64 132/198 2024-04-16T16:51:29.303 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 116/198 2024-04-16T16:51:29.449 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 117/198 2024-04-16T16:51:29.637 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-common-2:19.0.0-3101.g9b42bfbe.el9.x86_64 130/198 2024-04-16T16:51:29.671 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-common-2:19.0.0-3101.g9b42bfbe.el9.x86_64 130/198 2024-04-16T16:51:30.206 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-common-2:19.0.0-3101.g9b42bfbe.el9.x86_64 130/198 2024-04-16T16:51:30.241 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-base-2:19.0.0-3101.g9b42bfbe.el9.x86_64 131/198 2024-04-16T16:51:30.322 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-base-2:19.0.0-3101.g9b42bfbe.el9.x86_64 131/198 2024-04-16T16:51:30.322 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-16T16:51:30.322 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-16T16:51:30.322 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-16T16:51:30.350 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-selinux-2:19.0.0-3101.g9b42bfbe.el9.x86_64 132/198 2024-04-16T16:51:31.104 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 118/198 2024-04-16T16:51:31.246 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 119/198 2024-04-16T16:51:31.285 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 120/198 2024-04-16T16:51:31.323 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : jq-1.6-16.el9.x86_64 121/198 2024-04-16T16:51:31.382 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 122/198 2024-04-16T16:51:31.784 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 123/198 2024-04-16T16:51:31.819 INFO:teuthology.orchestra.run.smithi045.stdout: Upgrading : librbd1-2:19.0.0-3101.g9b42bfbe.el9.x86_64 124/198 2024-04-16T16:51:31.941 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: librbd1-2:19.0.0-3101.g9b42bfbe.el9.x86_64 124/198 2024-04-16T16:51:31.976 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-rbd-2:19.0.0-3101.g9b42bfbe.el9.x86_64 125/198 2024-04-16T16:51:32.033 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 126/198 2024-04-16T16:51:32.607 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 127/198 2024-04-16T16:51:32.639 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : librgw2-2:19.0.0-3101.g9b42bfbe.el9.x86_64 128/198 2024-04-16T16:51:32.722 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: librgw2-2:19.0.0-3101.g9b42bfbe.el9.x86_64 128/198 2024-04-16T16:51:32.749 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-rgw-2:19.0.0-3101.g9b42bfbe.el9.x86_64 129/198 2024-04-16T16:51:35.606 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-common-2:19.0.0-3101.g9b42bfbe.el9.x86_64 130/198 2024-04-16T16:51:35.663 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-common-2:19.0.0-3101.g9b42bfbe.el9.x86_64 130/198 2024-04-16T16:51:36.457 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-common-2:19.0.0-3101.g9b42bfbe.el9.x86_64 130/198 2024-04-16T16:51:36.550 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-base-2:19.0.0-3101.g9b42bfbe.el9.x86_64 131/198 2024-04-16T16:51:36.747 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-base-2:19.0.0-3101.g9b42bfbe.el9.x86_64 131/198 2024-04-16T16:51:36.747 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-16T16:51:36.748 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-16T16:51:36.748 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-16T16:51:36.818 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-selinux-2:19.0.0-3101.g9b42bfbe.el9.x86_64 132/198 2024-04-16T16:51:40.867 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3101.g9b42bfbe.el9.x86_64 132/198 2024-04-16T16:51:40.867 INFO:teuthology.orchestra.run.smithi104.stdout:skipping the directory /sys 2024-04-16T16:51:40.867 INFO:teuthology.orchestra.run.smithi104.stdout:skipping the directory /proc 2024-04-16T16:51:40.867 INFO:teuthology.orchestra.run.smithi104.stdout:skipping the directory /mnt 2024-04-16T16:51:40.867 INFO:teuthology.orchestra.run.smithi104.stdout:skipping the directory /var/tmp 2024-04-16T16:51:40.868 INFO:teuthology.orchestra.run.smithi104.stdout:skipping the directory /home 2024-04-16T16:51:40.868 INFO:teuthology.orchestra.run.smithi104.stdout:skipping the directory /root 2024-04-16T16:51:40.868 INFO:teuthology.orchestra.run.smithi104.stdout:skipping the directory /tmp 2024-04-16T16:51:40.868 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-04-16T16:51:41.036 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : ceph-volume-2:19.0.0-3101.g9b42bfbe.el9.noarch 133/198 2024-04-16T16:51:41.051 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: ceph-volume-2:19.0.0-3101.g9b42bfbe.el9.noarch 133/198 2024-04-16T16:51:41.051 INFO:teuthology.orchestra.run.smithi104.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-16T16:51:41.051 INFO:teuthology.orchestra.run.smithi104.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-16T16:51:41.051 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-04-16T16:51:41.869 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3101.g9b42bfbe.el9.x86_64 132/198 2024-04-16T16:51:41.869 INFO:teuthology.orchestra.run.smithi117.stdout:skipping the directory /sys 2024-04-16T16:51:41.869 INFO:teuthology.orchestra.run.smithi117.stdout:skipping the directory /proc 2024-04-16T16:51:41.869 INFO:teuthology.orchestra.run.smithi117.stdout:skipping the directory /mnt 2024-04-16T16:51:41.870 INFO:teuthology.orchestra.run.smithi117.stdout:skipping the directory /var/tmp 2024-04-16T16:51:41.870 INFO:teuthology.orchestra.run.smithi117.stdout:skipping the directory /home 2024-04-16T16:51:41.870 INFO:teuthology.orchestra.run.smithi117.stdout:skipping the directory /root 2024-04-16T16:51:41.870 INFO:teuthology.orchestra.run.smithi117.stdout:skipping the directory /tmp 2024-04-16T16:51:41.870 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-16T16:51:42.040 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-volume-2:19.0.0-3101.g9b42bfbe.el9.noarch 133/198 2024-04-16T16:51:42.055 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-volume-2:19.0.0-3101.g9b42bfbe.el9.noarch 133/198 2024-04-16T16:51:42.055 INFO:teuthology.orchestra.run.smithi117.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-16T16:51:42.055 INFO:teuthology.orchestra.run.smithi117.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-16T16:51:42.055 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-16T16:51:42.549 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : ceph-osd-2:19.0.0-3101.g9b42bfbe.el9.x86_64 134/198 2024-04-16T16:51:42.574 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: ceph-osd-2:19.0.0-3101.g9b42bfbe.el9.x86_64 134/198 2024-04-16T16:51:42.574 INFO:teuthology.orchestra.run.smithi104.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-16T16:51:42.574 INFO:teuthology.orchestra.run.smithi104.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-16T16:51:42.574 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-16T16:51:42.574 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-16T16:51:42.575 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-04-16T16:51:42.791 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : ceph-mds-2:19.0.0-3101.g9b42bfbe.el9.x86_64 135/198 2024-04-16T16:51:42.813 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: ceph-mds-2:19.0.0-3101.g9b42bfbe.el9.x86_64 135/198 2024-04-16T16:51:42.813 INFO:teuthology.orchestra.run.smithi104.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-16T16:51:42.813 INFO:teuthology.orchestra.run.smithi104.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-16T16:51:42.813 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-16T16:51:42.813 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-16T16:51:42.813 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-04-16T16:51:43.250 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : ceph-mon-2:19.0.0-3101.g9b42bfbe.el9.x86_64 136/198 2024-04-16T16:51:43.272 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: ceph-mon-2:19.0.0-3101.g9b42bfbe.el9.x86_64 136/198 2024-04-16T16:51:43.272 INFO:teuthology.orchestra.run.smithi104.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-16T16:51:43.272 INFO:teuthology.orchestra.run.smithi104.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-16T16:51:43.272 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-16T16:51:43.272 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-16T16:51:43.272 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-04-16T16:51:43.360 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : automake-1.16.2-8.el9.noarch 137/198 2024-04-16T16:51:43.503 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-osd-2:19.0.0-3101.g9b42bfbe.el9.x86_64 134/198 2024-04-16T16:51:43.527 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-osd-2:19.0.0-3101.g9b42bfbe.el9.x86_64 134/198 2024-04-16T16:51:43.527 INFO:teuthology.orchestra.run.smithi117.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-16T16:51:43.527 INFO:teuthology.orchestra.run.smithi117.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-16T16:51:43.527 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-16T16:51:43.527 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-16T16:51:43.527 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-16T16:51:43.709 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: cephadm-2:19.0.0-3101.g9b42bfbe.el9.noarch 138/198 2024-04-16T16:51:43.742 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : cephadm-2:19.0.0-3101.g9b42bfbe.el9.noarch 138/198 2024-04-16T16:51:43.752 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-mds-2:19.0.0-3101.g9b42bfbe.el9.x86_64 135/198 2024-04-16T16:51:43.774 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3101.g9b42bfbe.e 139/198 2024-04-16T16:51:43.774 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-mds-2:19.0.0-3101.g9b42bfbe.el9.x86_64 135/198 2024-04-16T16:51:43.774 INFO:teuthology.orchestra.run.smithi117.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-16T16:51:43.775 INFO:teuthology.orchestra.run.smithi117.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-16T16:51:43.775 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-16T16:51:43.775 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-16T16:51:43.775 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-16T16:51:43.836 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3101.g9b42bfbe. 140/198 2024-04-16T16:51:43.865 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3101.g9b42bfbe.el9.noa 141/198 2024-04-16T16:51:44.211 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-mon-2:19.0.0-3101.g9b42bfbe.el9.x86_64 136/198 2024-04-16T16:51:44.233 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-mon-2:19.0.0-3101.g9b42bfbe.el9.x86_64 136/198 2024-04-16T16:51:44.233 INFO:teuthology.orchestra.run.smithi117.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-16T16:51:44.233 INFO:teuthology.orchestra.run.smithi117.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-16T16:51:44.233 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-16T16:51:44.234 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-16T16:51:44.234 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-16T16:51:44.329 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : automake-1.16.2-8.el9.noarch 137/198 2024-04-16T16:51:44.771 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: cephadm-2:19.0.0-3101.g9b42bfbe.el9.noarch 138/198 2024-04-16T16:51:44.803 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : cephadm-2:19.0.0-3101.g9b42bfbe.el9.noarch 138/198 2024-04-16T16:51:44.835 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3101.g9b42bfbe.e 139/198 2024-04-16T16:51:44.896 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3101.g9b42bfbe. 140/198 2024-04-16T16:51:44.896 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3101.g9b42bfbe.el9.noa 141/198 2024-04-16T16:51:44.925 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3101.g9b42bfbe.el9.noa 141/198 2024-04-16T16:51:44.933 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3101.g9b42bfbe.el9.n 142/198 2024-04-16T16:51:45.894 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3101.g9b42bfbe.el9.n 142/198 2024-04-16T16:51:45.917 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3101.g9b4 143/198 2024-04-16T16:51:45.961 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3101.g9b42bfbe.el9.noa 141/198 2024-04-16T16:51:45.994 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3101.g9b42bfbe.el9.n 142/198 2024-04-16T16:51:46.015 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3101.g9b4 143/198 2024-04-16T16:51:46.170 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3101.g9b42bfbe.el 144/198 2024-04-16T16:51:46.193 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : ceph-mgr-2:19.0.0-3101.g9b42bfbe.el9.x86_64 145/198 2024-04-16T16:51:46.216 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3101.g9b42bfbe.el9.x86_64 145/198 2024-04-16T16:51:46.217 INFO:teuthology.orchestra.run.smithi104.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-16T16:51:46.217 INFO:teuthology.orchestra.run.smithi104.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-16T16:51:46.217 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-16T16:51:46.217 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-16T16:51:46.217 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-04-16T16:51:46.269 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : ceph-mgr-rook-2:19.0.0-3101.g9b42bfbe.el9.noarch 146/198 2024-04-16T16:51:46.279 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3101.g9b42bfbe.el9.noarch 146/198 2024-04-16T16:51:46.325 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : ceph-2:19.0.0-3101.g9b42bfbe.el9.x86_64 147/198 2024-04-16T16:51:46.954 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3101.g9b42bfbe.el9.n 142/198 2024-04-16T16:51:46.986 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3101.g9b4 143/198 2024-04-16T16:51:47.085 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3101.g9b4 143/198 2024-04-16T16:51:47.248 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : libtool-2.4.6-45.el9.x86_64 148/198 2024-04-16T16:51:47.249 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3101.g9b42bfbe.el 144/198 2024-04-16T16:51:47.280 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-mgr-2:19.0.0-3101.g9b42bfbe.el9.x86_64 145/198 2024-04-16T16:51:47.288 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : ceph-radosgw-2:19.0.0-3101.g9b42bfbe.el9.x86_64 149/198 2024-04-16T16:51:47.303 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3101.g9b42bfbe.el9.x86_64 145/198 2024-04-16T16:51:47.303 INFO:teuthology.orchestra.run.smithi117.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-16T16:51:47.303 INFO:teuthology.orchestra.run.smithi117.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-16T16:51:47.304 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-16T16:51:47.304 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-16T16:51:47.304 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-16T16:51:47.310 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3101.g9b42bfbe.el9.x86_64 149/198 2024-04-16T16:51:47.310 INFO:teuthology.orchestra.run.smithi104.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-16T16:51:47.310 INFO:teuthology.orchestra.run.smithi104.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-16T16:51:47.310 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-16T16:51:47.310 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-16T16:51:47.311 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-04-16T16:51:47.347 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-mgr-rook-2:19.0.0-3101.g9b42bfbe.el9.noarch 146/198 2024-04-16T16:51:47.355 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3101.g9b42b 150/198 2024-04-16T16:51:47.358 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3101.g9b42bfbe.el9.noarch 146/198 2024-04-16T16:51:47.377 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3101.g9b42b 150/198 2024-04-16T16:51:47.377 INFO:teuthology.orchestra.run.smithi104.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-16T16:51:47.377 INFO:teuthology.orchestra.run.smithi104.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-16T16:51:47.377 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-04-16T16:51:47.403 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-2:19.0.0-3101.g9b42bfbe.el9.x86_64 147/198 2024-04-16T16:51:47.422 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : cephfs-mirror-2:19.0.0-3101.g9b42bfbe.el9.x86_64 151/198 2024-04-16T16:51:47.444 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3101.g9b42bfbe.el9.x86_64 151/198 2024-04-16T16:51:47.444 INFO:teuthology.orchestra.run.smithi104.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-16T16:51:47.444 INFO:teuthology.orchestra.run.smithi104.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-16T16:51:47.444 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-16T16:51:47.444 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-16T16:51:47.444 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-04-16T16:51:47.756 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : rbd-mirror-2:19.0.0-3101.g9b42bfbe.el9.x86_64 152/198 2024-04-16T16:51:47.778 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3101.g9b42bfbe.el9.x86_64 152/198 2024-04-16T16:51:47.779 INFO:teuthology.orchestra.run.smithi104.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-16T16:51:47.779 INFO:teuthology.orchestra.run.smithi104.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-16T16:51:47.779 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-16T16:51:47.779 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-16T16:51:47.779 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-04-16T16:51:48.326 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : libtool-2.4.6-45.el9.x86_64 148/198 2024-04-16T16:51:48.357 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-radosgw-2:19.0.0-3101.g9b42bfbe.el9.x86_64 149/198 2024-04-16T16:51:48.380 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3101.g9b42bfbe.el9.x86_64 149/198 2024-04-16T16:51:48.380 INFO:teuthology.orchestra.run.smithi117.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-16T16:51:48.380 INFO:teuthology.orchestra.run.smithi117.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-16T16:51:48.380 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-16T16:51:48.380 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-16T16:51:48.380 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-16T16:51:48.424 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3101.g9b42b 150/198 2024-04-16T16:51:48.446 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3101.g9b42b 150/198 2024-04-16T16:51:48.446 INFO:teuthology.orchestra.run.smithi117.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-16T16:51:48.446 INFO:teuthology.orchestra.run.smithi117.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-16T16:51:48.446 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-16T16:51:48.492 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : cephfs-mirror-2:19.0.0-3101.g9b42bfbe.el9.x86_64 151/198 2024-04-16T16:51:48.514 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3101.g9b42bfbe.el9.x86_64 151/198 2024-04-16T16:51:48.514 INFO:teuthology.orchestra.run.smithi117.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-16T16:51:48.514 INFO:teuthology.orchestra.run.smithi117.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-16T16:51:48.514 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-16T16:51:48.515 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-16T16:51:48.515 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-16T16:51:48.826 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : rbd-mirror-2:19.0.0-3101.g9b42bfbe.el9.x86_64 152/198 2024-04-16T16:51:48.848 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3101.g9b42bfbe.el9.x86_64 152/198 2024-04-16T16:51:48.848 INFO:teuthology.orchestra.run.smithi117.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-16T16:51:48.849 INFO:teuthology.orchestra.run.smithi117.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-16T16:51:48.849 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-16T16:51:48.849 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-16T16:51:48.849 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-16T16:51:50.461 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3101.g9b42bfbe.el9.x86_64 132/198 2024-04-16T16:51:50.461 INFO:teuthology.orchestra.run.smithi045.stdout:skipping the directory /sys 2024-04-16T16:51:50.461 INFO:teuthology.orchestra.run.smithi045.stdout:skipping the directory /proc 2024-04-16T16:51:50.461 INFO:teuthology.orchestra.run.smithi045.stdout:skipping the directory /mnt 2024-04-16T16:51:50.461 INFO:teuthology.orchestra.run.smithi045.stdout:skipping the directory /var/tmp 2024-04-16T16:51:50.461 INFO:teuthology.orchestra.run.smithi045.stdout:skipping the directory /home 2024-04-16T16:51:50.461 INFO:teuthology.orchestra.run.smithi045.stdout:skipping the directory /root 2024-04-16T16:51:50.462 INFO:teuthology.orchestra.run.smithi045.stdout:skipping the directory /tmp 2024-04-16T16:51:50.462 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-16T16:51:50.739 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-volume-2:19.0.0-3101.g9b42bfbe.el9.noarch 133/198 2024-04-16T16:51:50.789 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-volume-2:19.0.0-3101.g9b42bfbe.el9.noarch 133/198 2024-04-16T16:51:50.789 INFO:teuthology.orchestra.run.smithi045.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-16T16:51:50.789 INFO:teuthology.orchestra.run.smithi045.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-16T16:51:50.789 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-16T16:51:52.361 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : ceph-test-2:19.0.0-3101.g9b42bfbe.el9.x86_64 153/198 2024-04-16T16:51:52.402 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : rbd-fuse-2:19.0.0-3101.g9b42bfbe.el9.x86_64 154/198 2024-04-16T16:51:52.432 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : rbd-nbd-2:19.0.0-3101.g9b42bfbe.el9.x86_64 155/198 2024-04-16T16:51:52.459 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 156/198 2024-04-16T16:51:52.474 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-osd-2:19.0.0-3101.g9b42bfbe.el9.x86_64 134/198 2024-04-16T16:51:52.582 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-osd-2:19.0.0-3101.g9b42bfbe.el9.x86_64 134/198 2024-04-16T16:51:52.583 INFO:teuthology.orchestra.run.smithi045.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-16T16:51:52.583 INFO:teuthology.orchestra.run.smithi045.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-16T16:51:52.583 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-16T16:51:52.583 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-16T16:51:52.583 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-16T16:51:52.638 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : cephfs-shell-2:19.0.0-3101.g9b42bfbe.el9.x86_64 157/198 2024-04-16T16:51:52.750 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 158/198 2024-04-16T16:51:52.799 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : ceph-fuse-2:19.0.0-3101.g9b42bfbe.el9.x86_64 159/198 2024-04-16T16:51:52.837 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 160/198 2024-04-16T16:51:52.849 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-mds-2:19.0.0-3101.g9b42bfbe.el9.x86_64 135/198 2024-04-16T16:51:52.892 INFO:teuthology.orchestra.run.smithi104.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 161/198 2024-04-16T16:51:52.927 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-mds-2:19.0.0-3101.g9b42bfbe.el9.x86_64 135/198 2024-04-16T16:51:52.927 INFO:teuthology.orchestra.run.smithi045.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-16T16:51:52.927 INFO:teuthology.orchestra.run.smithi045.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-16T16:51:52.927 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-16T16:51:52.927 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-16T16:51:52.927 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-16T16:51:52.935 INFO:teuthology.orchestra.run.smithi104.stdout: Upgrading : util-linux-user-2.37.4-18.el9.x86_64 162/198 2024-04-16T16:51:52.968 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : libcephfs-devel-2:19.0.0-3101.g9b42bfbe.el9.x86_ 163/198 2024-04-16T16:51:53.000 INFO:teuthology.orchestra.run.smithi104.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 164/198 2024-04-16T16:51:53.029 INFO:teuthology.orchestra.run.smithi104.stdout: Upgrading : NetworkManager-team-1:1.47.4-1.el9.x86_64 165/198 2024-04-16T16:51:53.083 INFO:teuthology.orchestra.run.smithi104.stdout: Upgrading : NetworkManager-tui-1:1.47.4-1.el9.x86_64 166/198 2024-04-16T16:51:53.144 INFO:teuthology.orchestra.run.smithi104.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 167/198 2024-04-16T16:51:53.178 INFO:teuthology.orchestra.run.smithi104.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 168/198 2024-04-16T16:51:53.433 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-mon-2:19.0.0-3101.g9b42bfbe.el9.x86_64 136/198 2024-04-16T16:51:53.438 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-test-2:19.0.0-3101.g9b42bfbe.el9.x86_64 153/198 2024-04-16T16:51:53.480 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : rbd-fuse-2:19.0.0-3101.g9b42bfbe.el9.x86_64 154/198 2024-04-16T16:51:53.510 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : rbd-nbd-2:19.0.0-3101.g9b42bfbe.el9.x86_64 155/198 2024-04-16T16:51:53.525 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-mon-2:19.0.0-3101.g9b42bfbe.el9.x86_64 136/198 2024-04-16T16:51:53.525 INFO:teuthology.orchestra.run.smithi045.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-16T16:51:53.525 INFO:teuthology.orchestra.run.smithi045.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-16T16:51:53.525 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-16T16:51:53.525 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-16T16:51:53.525 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-16T16:51:53.537 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 156/198 2024-04-16T16:51:53.673 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : automake-1.16.2-8.el9.noarch 137/198 2024-04-16T16:51:53.717 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : cephfs-shell-2:19.0.0-3101.g9b42bfbe.el9.x86_64 157/198 2024-04-16T16:51:53.721 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 169/198 2024-04-16T16:51:53.758 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 169/198 2024-04-16T16:51:53.778 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 169/198 2024-04-16T16:51:53.836 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 158/198 2024-04-16T16:51:53.877 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-fuse-2:19.0.0-3101.g9b42bfbe.el9.x86_64 159/198 2024-04-16T16:51:53.916 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 160/198 2024-04-16T16:51:53.977 INFO:teuthology.orchestra.run.smithi117.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 161/198 2024-04-16T16:51:54.020 INFO:teuthology.orchestra.run.smithi117.stdout: Upgrading : util-linux-user-2.37.4-18.el9.x86_64 162/198 2024-04-16T16:51:54.053 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : libcephfs-devel-2:19.0.0-3101.g9b42bfbe.el9.x86_ 163/198 2024-04-16T16:51:54.085 INFO:teuthology.orchestra.run.smithi117.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 164/198 2024-04-16T16:51:54.114 INFO:teuthology.orchestra.run.smithi117.stdout: Upgrading : NetworkManager-team-1:1.47.4-1.el9.x86_64 165/198 2024-04-16T16:51:54.169 INFO:teuthology.orchestra.run.smithi117.stdout: Upgrading : NetworkManager-tui-1:1.47.4-1.el9.x86_64 166/198 2024-04-16T16:51:54.238 INFO:teuthology.orchestra.run.smithi117.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 167/198 2024-04-16T16:51:54.272 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: cephadm-2:19.0.0-3101.g9b42bfbe.el9.noarch 138/198 2024-04-16T16:51:54.280 INFO:teuthology.orchestra.run.smithi117.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 168/198 2024-04-16T16:51:54.305 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : cephadm-2:19.0.0-3101.g9b42bfbe.el9.noarch 138/198 2024-04-16T16:51:54.361 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3101.g9b42bfbe.e 139/198 2024-04-16T16:51:54.472 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3101.g9b42bfbe. 140/198 2024-04-16T16:51:54.508 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3101.g9b42bfbe.el9.noa 141/198 2024-04-16T16:51:54.598 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : cephfs-top-2:19.0.0-3101.g9b42bfbe.el9.noarch 170/198 2024-04-16T16:51:54.686 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 171/198 2024-04-16T16:51:54.726 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 172/198 2024-04-16T16:51:54.808 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : flex-2.6.4-9.el9.x86_64 173/198 2024-04-16T16:51:54.823 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 169/198 2024-04-16T16:51:54.857 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : bison-3.7.4-5.el9.x86_64 174/198 2024-04-16T16:51:54.861 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 169/198 2024-04-16T16:51:54.881 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 169/198 2024-04-16T16:51:54.881 INFO:teuthology.orchestra.run.smithi104.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 175/198 2024-04-16T16:51:54.907 INFO:teuthology.orchestra.run.smithi104.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 176/198 2024-04-16T16:51:54.933 INFO:teuthology.orchestra.run.smithi104.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 177/198 2024-04-16T16:51:54.933 INFO:teuthology.orchestra.run.smithi104.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-04-16T16:51:55.004 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-04-16T16:51:55.042 INFO:teuthology.orchestra.run.smithi104.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 179/198 2024-04-16T16:51:55.042 INFO:teuthology.orchestra.run.smithi104.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 180/198 2024-04-16T16:51:55.110 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 180/198 2024-04-16T16:51:55.137 INFO:teuthology.orchestra.run.smithi104.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 181/198 2024-04-16T16:51:55.169 INFO:teuthology.orchestra.run.smithi104.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 182/198 2024-04-16T16:51:55.195 INFO:teuthology.orchestra.run.smithi104.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 183/198 2024-04-16T16:51:55.220 INFO:teuthology.orchestra.run.smithi104.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 184/198 2024-04-16T16:51:55.252 INFO:teuthology.orchestra.run.smithi104.stdout: Cleanup : util-linux-user-2.37.4-15.el9.x86_64 185/198 2024-04-16T16:51:55.292 INFO:teuthology.orchestra.run.smithi104.stdout: Cleanup : util-linux-2.37.4-15.el9.x86_64 186/198 2024-04-16T16:51:55.324 INFO:teuthology.orchestra.run.smithi104.stdout: Cleanup : util-linux-core-2.37.4-15.el9.x86_64 187/198 2024-04-16T16:51:55.350 INFO:teuthology.orchestra.run.smithi104.stdout: Cleanup : libfdisk-2.37.4-15.el9.x86_64 188/198 2024-04-16T16:51:55.375 INFO:teuthology.orchestra.run.smithi104.stdout: Cleanup : libmount-2.37.4-15.el9.x86_64 189/198 2024-04-16T16:51:55.400 INFO:teuthology.orchestra.run.smithi104.stdout: Cleanup : libblkid-2.37.4-15.el9.x86_64 190/198 2024-04-16T16:51:55.434 INFO:teuthology.orchestra.run.smithi104.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 191/198 2024-04-16T16:51:55.440 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-04-16T16:51:55.440 INFO:teuthology.orchestra.run.smithi104.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-04-16T16:51:55.620 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-04-16T16:51:55.620 INFO:teuthology.orchestra.run.smithi104.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-04-16T16:51:55.690 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : cephfs-top-2:19.0.0-3101.g9b42bfbe.el9.noarch 170/198 2024-04-16T16:51:55.694 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-04-16T16:51:55.726 INFO:teuthology.orchestra.run.smithi104.stdout: Cleanup : libuuid-2.37.4-15.el9.x86_64 194/198 2024-04-16T16:51:55.753 INFO:teuthology.orchestra.run.smithi104.stdout: Cleanup : libsmartcols-2.37.4-15.el9.x86_64 195/198 2024-04-16T16:51:55.779 INFO:teuthology.orchestra.run.smithi104.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 196/198 2024-04-16T16:51:55.781 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 171/198 2024-04-16T16:51:55.803 INFO:teuthology.orchestra.run.smithi104.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 197/198 2024-04-16T16:51:55.821 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 172/198 2024-04-16T16:51:55.867 INFO:teuthology.orchestra.run.smithi104.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-04-16T16:51:55.872 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3101.g9b42bfbe.el9.noa 141/198 2024-04-16T16:51:55.902 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : flex-2.6.4-9.el9.x86_64 173/198 2024-04-16T16:51:55.918 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3101.g9b42bfbe.el9.n 142/198 2024-04-16T16:51:55.943 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : bison-3.7.4-5.el9.x86_64 174/198 2024-04-16T16:51:55.968 INFO:teuthology.orchestra.run.smithi117.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 175/198 2024-04-16T16:51:55.993 INFO:teuthology.orchestra.run.smithi117.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 176/198 2024-04-16T16:51:56.020 INFO:teuthology.orchestra.run.smithi117.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 177/198 2024-04-16T16:51:56.020 INFO:teuthology.orchestra.run.smithi117.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-04-16T16:51:56.116 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-04-16T16:51:56.146 INFO:teuthology.orchestra.run.smithi117.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 179/198 2024-04-16T16:51:56.146 INFO:teuthology.orchestra.run.smithi117.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 180/198 2024-04-16T16:51:56.222 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 180/198 2024-04-16T16:51:56.249 INFO:teuthology.orchestra.run.smithi117.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 181/198 2024-04-16T16:51:56.281 INFO:teuthology.orchestra.run.smithi117.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 182/198 2024-04-16T16:51:56.307 INFO:teuthology.orchestra.run.smithi117.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 183/198 2024-04-16T16:51:56.332 INFO:teuthology.orchestra.run.smithi117.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 184/198 2024-04-16T16:51:56.365 INFO:teuthology.orchestra.run.smithi117.stdout: Cleanup : util-linux-user-2.37.4-15.el9.x86_64 185/198 2024-04-16T16:51:56.404 INFO:teuthology.orchestra.run.smithi117.stdout: Cleanup : util-linux-2.37.4-15.el9.x86_64 186/198 2024-04-16T16:51:56.436 INFO:teuthology.orchestra.run.smithi117.stdout: Cleanup : util-linux-core-2.37.4-15.el9.x86_64 187/198 2024-04-16T16:51:56.462 INFO:teuthology.orchestra.run.smithi117.stdout: Cleanup : libfdisk-2.37.4-15.el9.x86_64 188/198 2024-04-16T16:51:56.504 INFO:teuthology.orchestra.run.smithi117.stdout: Cleanup : libmount-2.37.4-15.el9.x86_64 189/198 2024-04-16T16:51:56.529 INFO:teuthology.orchestra.run.smithi117.stdout: Cleanup : libblkid-2.37.4-15.el9.x86_64 190/198 2024-04-16T16:51:56.554 INFO:teuthology.orchestra.run.smithi117.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 191/198 2024-04-16T16:51:56.561 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-04-16T16:51:56.561 INFO:teuthology.orchestra.run.smithi117.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-04-16T16:51:56.740 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-04-16T16:51:56.741 INFO:teuthology.orchestra.run.smithi117.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-04-16T16:51:56.807 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-04-16T16:51:56.838 INFO:teuthology.orchestra.run.smithi117.stdout: Cleanup : libuuid-2.37.4-15.el9.x86_64 194/198 2024-04-16T16:51:56.865 INFO:teuthology.orchestra.run.smithi117.stdout: Cleanup : libsmartcols-2.37.4-15.el9.x86_64 195/198 2024-04-16T16:51:56.891 INFO:teuthology.orchestra.run.smithi117.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 196/198 2024-04-16T16:51:56.916 INFO:teuthology.orchestra.run.smithi117.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 197/198 2024-04-16T16:51:56.979 INFO:teuthology.orchestra.run.smithi117.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-04-16T16:51:57.082 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3101.g9b42bfbe.el9.n 142/198 2024-04-16T16:51:57.118 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3101.g9b4 143/198 2024-04-16T16:51:57.269 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3101.g9b4 143/198 2024-04-16T16:51:57.458 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3101.g9b42bfbe.el 144/198 2024-04-16T16:51:57.495 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-mgr-2:19.0.0-3101.g9b42bfbe.el9.x86_64 145/198 2024-04-16T16:51:57.596 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3101.g9b42bfbe.el9.x86_64 145/198 2024-04-16T16:51:57.596 INFO:teuthology.orchestra.run.smithi045.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-16T16:51:57.596 INFO:teuthology.orchestra.run.smithi045.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-16T16:51:57.596 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-16T16:51:57.596 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-16T16:51:57.596 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-16T16:51:57.663 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-mgr-rook-2:19.0.0-3101.g9b42bfbe.el9.noarch 146/198 2024-04-16T16:51:57.704 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3101.g9b42bfbe.el9.noarch 146/198 2024-04-16T16:51:57.784 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-2:19.0.0-3101.g9b42bfbe.el9.x86_64 147/198 2024-04-16T16:51:58.885 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libtool-2.4.6-45.el9.x86_64 148/198 2024-04-16T16:51:58.915 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-radosgw-2:19.0.0-3101.g9b42bfbe.el9.x86_64 149/198 2024-04-16T16:51:59.012 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3101.g9b42bfbe.el9.x86_64 149/198 2024-04-16T16:51:59.012 INFO:teuthology.orchestra.run.smithi045.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-16T16:51:59.012 INFO:teuthology.orchestra.run.smithi045.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-16T16:51:59.013 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-16T16:51:59.013 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-16T16:51:59.013 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-16T16:51:59.066 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3101.g9b42b 150/198 2024-04-16T16:51:59.168 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3101.g9b42b 150/198 2024-04-16T16:51:59.168 INFO:teuthology.orchestra.run.smithi045.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-16T16:51:59.168 INFO:teuthology.orchestra.run.smithi045.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-16T16:51:59.168 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-16T16:51:59.250 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : cephfs-mirror-2:19.0.0-3101.g9b42bfbe.el9.x86_64 151/198 2024-04-16T16:51:59.360 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3101.g9b42bfbe.el9.x86_64 151/198 2024-04-16T16:51:59.360 INFO:teuthology.orchestra.run.smithi045.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-16T16:51:59.360 INFO:teuthology.orchestra.run.smithi045.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-16T16:51:59.360 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-16T16:51:59.361 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-16T16:51:59.361 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-16T16:51:59.776 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : rbd-mirror-2:19.0.0-3101.g9b42bfbe.el9.x86_64 152/198 2024-04-16T16:51:59.858 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3101.g9b42bfbe.el9.x86_64 152/198 2024-04-16T16:51:59.858 INFO:teuthology.orchestra.run.smithi045.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-16T16:51:59.858 INFO:teuthology.orchestra.run.smithi045.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-16T16:51:59.858 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-16T16:51:59.858 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-16T16:51:59.858 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-16T16:52:01.341 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-04-16T16:52:01.341 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : ceph-2:19.0.0-3101.g9b42bfbe.el9.x86_64 1/198 2024-04-16T16:52:01.341 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : ceph-base-2:19.0.0-3101.g9b42bfbe.el9.x86_64 2/198 2024-04-16T16:52:01.341 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : ceph-common-2:19.0.0-3101.g9b42bfbe.el9.x86_64 3/198 2024-04-16T16:52:01.341 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : ceph-fuse-2:19.0.0-3101.g9b42bfbe.el9.x86_64 4/198 2024-04-16T16:52:01.341 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3101.g9b42b 5/198 2024-04-16T16:52:01.341 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : ceph-mds-2:19.0.0-3101.g9b42bfbe.el9.x86_64 6/198 2024-04-16T16:52:01.341 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : ceph-mgr-2:19.0.0-3101.g9b42bfbe.el9.x86_64 7/198 2024-04-16T16:52:01.341 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : ceph-mon-2:19.0.0-3101.g9b42bfbe.el9.x86_64 8/198 2024-04-16T16:52:01.342 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : ceph-osd-2:19.0.0-3101.g9b42bfbe.el9.x86_64 9/198 2024-04-16T16:52:01.342 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : ceph-radosgw-2:19.0.0-3101.g9b42bfbe.el9.x86_64 10/198 2024-04-16T16:52:01.342 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : ceph-selinux-2:19.0.0-3101.g9b42bfbe.el9.x86_64 11/198 2024-04-16T16:52:01.342 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : ceph-test-2:19.0.0-3101.g9b42bfbe.el9.x86_64 12/198 2024-04-16T16:52:01.342 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : cephfs-mirror-2:19.0.0-3101.g9b42bfbe.el9.x86_64 13/198 2024-04-16T16:52:01.342 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : cephfs-shell-2:19.0.0-3101.g9b42bfbe.el9.x86_64 14/198 2024-04-16T16:52:01.342 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : libcephfs-devel-2:19.0.0-3101.g9b42bfbe.el9.x86_ 15/198 2024-04-16T16:52:01.342 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : libcephfs2-2:19.0.0-3101.g9b42bfbe.el9.x86_64 16/198 2024-04-16T16:52:01.342 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : libcephsqlite-2:19.0.0-3101.g9b42bfbe.el9.x86_64 17/198 2024-04-16T16:52:01.342 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : librados-devel-2:19.0.0-3101.g9b42bfbe.el9.x86_6 18/198 2024-04-16T16:52:01.342 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : libradosstriper1-2:19.0.0-3101.g9b42bfbe.el9.x86 19/198 2024-04-16T16:52:01.342 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : librgw2-2:19.0.0-3101.g9b42bfbe.el9.x86_64 20/198 2024-04-16T16:52:01.342 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3101.g9b42bfbe.el 21/198 2024-04-16T16:52:01.342 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-ceph-common-2:19.0.0-3101.g9b42bfbe.el9. 22/198 2024-04-16T16:52:01.343 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-cephfs-2:19.0.0-3101.g9b42bfbe.el9.x86_6 23/198 2024-04-16T16:52:01.343 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-rados-2:19.0.0-3101.g9b42bfbe.el9.x86_64 24/198 2024-04-16T16:52:01.343 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-rbd-2:19.0.0-3101.g9b42bfbe.el9.x86_64 25/198 2024-04-16T16:52:01.343 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-rgw-2:19.0.0-3101.g9b42bfbe.el9.x86_64 26/198 2024-04-16T16:52:01.343 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : rbd-fuse-2:19.0.0-3101.g9b42bfbe.el9.x86_64 27/198 2024-04-16T16:52:01.343 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : rbd-mirror-2:19.0.0-3101.g9b42bfbe.el9.x86_64 28/198 2024-04-16T16:52:01.343 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : rbd-nbd-2:19.0.0-3101.g9b42bfbe.el9.x86_64 29/198 2024-04-16T16:52:01.343 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3101.g9b42bfbe. 30/198 2024-04-16T16:52:01.343 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3101.g9b42bfbe.el9.noa 31/198 2024-04-16T16:52:01.343 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3101.g9b42bfbe.el9.n 32/198 2024-04-16T16:52:01.343 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3101.g9b4 33/198 2024-04-16T16:52:01.343 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3101.g9b42bfbe.el 34/198 2024-04-16T16:52:01.343 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3101.g9b42bfbe.el9.noarch 35/198 2024-04-16T16:52:01.343 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3101.g9b42bfbe.e 36/198 2024-04-16T16:52:01.344 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : ceph-volume-2:19.0.0-3101.g9b42bfbe.el9.noarch 37/198 2024-04-16T16:52:01.344 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : cephadm-2:19.0.0-3101.g9b42bfbe.el9.noarch 38/198 2024-04-16T16:52:01.344 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : cephfs-top-2:19.0.0-3101.g9b42bfbe.el9.noarch 39/198 2024-04-16T16:52:01.344 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : automake-1.16.2-8.el9.noarch 40/198 2024-04-16T16:52:01.344 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : bison-3.7.4-5.el9.x86_64 41/198 2024-04-16T16:52:01.344 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 42/198 2024-04-16T16:52:01.344 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : flex-2.6.4-9.el9.x86_64 43/198 2024-04-16T16:52:01.344 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 44/198 2024-04-16T16:52:01.344 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 45/198 2024-04-16T16:52:01.344 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 46/198 2024-04-16T16:52:01.344 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 47/198 2024-04-16T16:52:01.344 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 48/198 2024-04-16T16:52:01.344 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 49/198 2024-04-16T16:52:01.344 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 50/198 2024-04-16T16:52:01.344 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 51/198 2024-04-16T16:52:01.344 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 52/198 2024-04-16T16:52:01.345 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 53/198 2024-04-16T16:52:01.345 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : lua-5.4.4-4.el9.x86_64 54/198 2024-04-16T16:52:01.345 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 55/198 2024-04-16T16:52:01.345 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 56/198 2024-04-16T16:52:01.345 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 57/198 2024-04-16T16:52:01.345 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : postgresql-13.14-1.el9.x86_64 58/198 2024-04-16T16:52:01.345 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 59/198 2024-04-16T16:52:01.345 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 60/198 2024-04-16T16:52:01.345 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 61/198 2024-04-16T16:52:01.345 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 62/198 2024-04-16T16:52:01.345 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 63/198 2024-04-16T16:52:01.345 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 64/198 2024-04-16T16:52:01.345 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 65/198 2024-04-16T16:52:01.345 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 66/198 2024-04-16T16:52:01.346 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 67/198 2024-04-16T16:52:01.346 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 68/198 2024-04-16T16:52:01.346 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 69/198 2024-04-16T16:52:01.346 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 70/198 2024-04-16T16:52:01.346 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 71/198 2024-04-16T16:52:01.346 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 72/198 2024-04-16T16:52:01.346 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 73/198 2024-04-16T16:52:01.346 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 74/198 2024-04-16T16:52:01.346 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 75/198 2024-04-16T16:52:01.346 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 76/198 2024-04-16T16:52:01.346 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 77/198 2024-04-16T16:52:01.346 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 78/198 2024-04-16T16:52:01.346 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 79/198 2024-04-16T16:52:01.346 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 80/198 2024-04-16T16:52:01.347 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 81/198 2024-04-16T16:52:01.347 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 82/198 2024-04-16T16:52:01.347 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 83/198 2024-04-16T16:52:01.347 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 84/198 2024-04-16T16:52:01.347 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 85/198 2024-04-16T16:52:01.347 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 86/198 2024-04-16T16:52:01.347 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 87/198 2024-04-16T16:52:01.347 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 88/198 2024-04-16T16:52:01.347 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 89/198 2024-04-16T16:52:01.347 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 90/198 2024-04-16T16:52:01.347 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 91/198 2024-04-16T16:52:01.347 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-ply-3.11-14.el9.noarch 92/198 2024-04-16T16:52:01.347 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 93/198 2024-04-16T16:52:01.348 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 94/198 2024-04-16T16:52:01.348 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : jq-1.6-16.el9.x86_64 95/198 2024-04-16T16:52:01.348 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 96/198 2024-04-16T16:52:01.348 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 97/198 2024-04-16T16:52:01.348 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 98/198 2024-04-16T16:52:01.348 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 99/198 2024-04-16T16:52:01.348 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 100/198 2024-04-16T16:52:01.348 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 101/198 2024-04-16T16:52:01.348 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 102/198 2024-04-16T16:52:01.348 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 103/198 2024-04-16T16:52:01.348 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 104/198 2024-04-16T16:52:01.348 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/198 2024-04-16T16:52:01.348 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/198 2024-04-16T16:52:01.348 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/198 2024-04-16T16:52:01.349 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/198 2024-04-16T16:52:01.349 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/198 2024-04-16T16:52:01.349 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 110/198 2024-04-16T16:52:01.349 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 111/198 2024-04-16T16:52:01.349 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/198 2024-04-16T16:52:01.349 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/198 2024-04-16T16:52:01.349 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 114/198 2024-04-16T16:52:01.349 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 115/198 2024-04-16T16:52:01.349 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 116/198 2024-04-16T16:52:01.349 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/198 2024-04-16T16:52:01.349 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/198 2024-04-16T16:52:01.349 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 119/198 2024-04-16T16:52:01.349 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/198 2024-04-16T16:52:01.349 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/198 2024-04-16T16:52:01.349 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/198 2024-04-16T16:52:01.350 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 123/198 2024-04-16T16:52:01.350 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-google-auth-1:2.28.2-1.el9.noarch 124/198 2024-04-16T16:52:01.350 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 125/198 2024-04-16T16:52:01.350 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 126/198 2024-04-16T16:52:01.350 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 127/198 2024-04-16T16:52:01.350 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 128/198 2024-04-16T16:52:01.350 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 129/198 2024-04-16T16:52:01.350 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 130/198 2024-04-16T16:52:01.350 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 131/198 2024-04-16T16:52:01.350 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 132/198 2024-04-16T16:52:01.350 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 133/198 2024-04-16T16:52:01.350 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 134/198 2024-04-16T16:52:01.350 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 135/198 2024-04-16T16:52:01.351 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 136/198 2024-04-16T16:52:01.351 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 137/198 2024-04-16T16:52:01.351 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 138/198 2024-04-16T16:52:01.351 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 139/198 2024-04-16T16:52:01.351 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 140/198 2024-04-16T16:52:01.351 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 141/198 2024-04-16T16:52:01.351 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 142/198 2024-04-16T16:52:01.351 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 143/198 2024-04-16T16:52:01.351 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 144/198 2024-04-16T16:52:01.351 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 145/198 2024-04-16T16:52:01.351 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 146/198 2024-04-16T16:52:01.351 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 147/198 2024-04-16T16:52:01.351 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 148/198 2024-04-16T16:52:01.351 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : re2-1:20211101-3.el9.x86_64 149/198 2024-04-16T16:52:01.352 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 150/198 2024-04-16T16:52:01.352 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : librados2-2:19.0.0-3101.g9b42bfbe.el9.x86_64 151/198 2024-04-16T16:52:01.352 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 152/198 2024-04-16T16:52:01.352 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : librbd1-2:19.0.0-3101.g9b42bfbe.el9.x86_64 153/198 2024-04-16T16:52:01.352 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 154/198 2024-04-16T16:52:01.352 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 155/198 2024-04-16T16:52:01.352 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 156/198 2024-04-16T16:52:01.352 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 157/198 2024-04-16T16:52:01.352 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 158/198 2024-04-16T16:52:01.352 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 159/198 2024-04-16T16:52:01.352 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 160/198 2024-04-16T16:52:01.352 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 161/198 2024-04-16T16:52:01.352 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 162/198 2024-04-16T16:52:01.352 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 163/198 2024-04-16T16:52:01.352 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 164/198 2024-04-16T16:52:01.353 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : acl-2.3.1-4.el9.x86_64 165/198 2024-04-16T16:52:01.353 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : acl-2.3.1-3.el9.x86_64 166/198 2024-04-16T16:52:01.353 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 167/198 2024-04-16T16:52:01.353 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 168/198 2024-04-16T16:52:01.353 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 169/198 2024-04-16T16:52:01.353 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 170/198 2024-04-16T16:52:01.353 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 171/198 2024-04-16T16:52:01.353 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 172/198 2024-04-16T16:52:01.353 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 173/198 2024-04-16T16:52:01.353 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 174/198 2024-04-16T16:52:01.353 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : libblkid-2.37.4-18.el9.x86_64 175/198 2024-04-16T16:52:01.353 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : libblkid-2.37.4-15.el9.x86_64 176/198 2024-04-16T16:52:01.353 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : libfdisk-2.37.4-18.el9.x86_64 177/198 2024-04-16T16:52:01.353 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : libfdisk-2.37.4-15.el9.x86_64 178/198 2024-04-16T16:52:01.354 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : libmount-2.37.4-18.el9.x86_64 179/198 2024-04-16T16:52:01.354 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : libmount-2.37.4-15.el9.x86_64 180/198 2024-04-16T16:52:01.354 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : libsmartcols-2.37.4-18.el9.x86_64 181/198 2024-04-16T16:52:01.354 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : libsmartcols-2.37.4-15.el9.x86_64 182/198 2024-04-16T16:52:01.354 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : libuuid-2.37.4-18.el9.x86_64 183/198 2024-04-16T16:52:01.354 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : libuuid-2.37.4-15.el9.x86_64 184/198 2024-04-16T16:52:01.354 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 185/198 2024-04-16T16:52:01.354 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 186/198 2024-04-16T16:52:01.354 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 187/198 2024-04-16T16:52:01.354 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 188/198 2024-04-16T16:52:01.354 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : util-linux-2.37.4-18.el9.x86_64 189/198 2024-04-16T16:52:01.354 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : util-linux-2.37.4-15.el9.x86_64 190/198 2024-04-16T16:52:01.354 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : util-linux-core-2.37.4-18.el9.x86_64 191/198 2024-04-16T16:52:01.355 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : util-linux-core-2.37.4-15.el9.x86_64 192/198 2024-04-16T16:52:01.355 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : util-linux-user-2.37.4-18.el9.x86_64 193/198 2024-04-16T16:52:01.355 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : util-linux-user-2.37.4-15.el9.x86_64 194/198 2024-04-16T16:52:01.355 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 195/198 2024-04-16T16:52:01.355 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 196/198 2024-04-16T16:52:01.355 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 197/198 2024-04-16T16:52:02.289 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-04-16T16:52:02.290 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-2:19.0.0-3101.g9b42bfbe.el9.x86_64 1/198 2024-04-16T16:52:02.290 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-base-2:19.0.0-3101.g9b42bfbe.el9.x86_64 2/198 2024-04-16T16:52:02.290 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-common-2:19.0.0-3101.g9b42bfbe.el9.x86_64 3/198 2024-04-16T16:52:02.290 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-fuse-2:19.0.0-3101.g9b42bfbe.el9.x86_64 4/198 2024-04-16T16:52:02.290 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3101.g9b42b 5/198 2024-04-16T16:52:02.290 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-mds-2:19.0.0-3101.g9b42bfbe.el9.x86_64 6/198 2024-04-16T16:52:02.290 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-mgr-2:19.0.0-3101.g9b42bfbe.el9.x86_64 7/198 2024-04-16T16:52:02.290 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-mon-2:19.0.0-3101.g9b42bfbe.el9.x86_64 8/198 2024-04-16T16:52:02.290 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-osd-2:19.0.0-3101.g9b42bfbe.el9.x86_64 9/198 2024-04-16T16:52:02.290 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-radosgw-2:19.0.0-3101.g9b42bfbe.el9.x86_64 10/198 2024-04-16T16:52:02.290 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-selinux-2:19.0.0-3101.g9b42bfbe.el9.x86_64 11/198 2024-04-16T16:52:02.290 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-test-2:19.0.0-3101.g9b42bfbe.el9.x86_64 12/198 2024-04-16T16:52:02.291 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : cephfs-mirror-2:19.0.0-3101.g9b42bfbe.el9.x86_64 13/198 2024-04-16T16:52:02.291 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : cephfs-shell-2:19.0.0-3101.g9b42bfbe.el9.x86_64 14/198 2024-04-16T16:52:02.291 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libcephfs-devel-2:19.0.0-3101.g9b42bfbe.el9.x86_ 15/198 2024-04-16T16:52:02.291 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libcephfs2-2:19.0.0-3101.g9b42bfbe.el9.x86_64 16/198 2024-04-16T16:52:02.291 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libcephsqlite-2:19.0.0-3101.g9b42bfbe.el9.x86_64 17/198 2024-04-16T16:52:02.291 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : librados-devel-2:19.0.0-3101.g9b42bfbe.el9.x86_6 18/198 2024-04-16T16:52:02.291 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libradosstriper1-2:19.0.0-3101.g9b42bfbe.el9.x86 19/198 2024-04-16T16:52:02.291 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : librgw2-2:19.0.0-3101.g9b42bfbe.el9.x86_64 20/198 2024-04-16T16:52:02.291 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3101.g9b42bfbe.el 21/198 2024-04-16T16:52:02.291 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-ceph-common-2:19.0.0-3101.g9b42bfbe.el9. 22/198 2024-04-16T16:52:02.291 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-cephfs-2:19.0.0-3101.g9b42bfbe.el9.x86_6 23/198 2024-04-16T16:52:02.291 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-rados-2:19.0.0-3101.g9b42bfbe.el9.x86_64 24/198 2024-04-16T16:52:02.291 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-rbd-2:19.0.0-3101.g9b42bfbe.el9.x86_64 25/198 2024-04-16T16:52:02.292 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-rgw-2:19.0.0-3101.g9b42bfbe.el9.x86_64 26/198 2024-04-16T16:52:02.292 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : rbd-fuse-2:19.0.0-3101.g9b42bfbe.el9.x86_64 27/198 2024-04-16T16:52:02.292 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : rbd-mirror-2:19.0.0-3101.g9b42bfbe.el9.x86_64 28/198 2024-04-16T16:52:02.292 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : rbd-nbd-2:19.0.0-3101.g9b42bfbe.el9.x86_64 29/198 2024-04-16T16:52:02.292 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3101.g9b42bfbe. 30/198 2024-04-16T16:52:02.292 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3101.g9b42bfbe.el9.noa 31/198 2024-04-16T16:52:02.292 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3101.g9b42bfbe.el9.n 32/198 2024-04-16T16:52:02.292 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3101.g9b4 33/198 2024-04-16T16:52:02.292 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3101.g9b42bfbe.el 34/198 2024-04-16T16:52:02.292 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3101.g9b42bfbe.el9.noarch 35/198 2024-04-16T16:52:02.292 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3101.g9b42bfbe.e 36/198 2024-04-16T16:52:02.292 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-volume-2:19.0.0-3101.g9b42bfbe.el9.noarch 37/198 2024-04-16T16:52:02.292 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : cephadm-2:19.0.0-3101.g9b42bfbe.el9.noarch 38/198 2024-04-16T16:52:02.293 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : cephfs-top-2:19.0.0-3101.g9b42bfbe.el9.noarch 39/198 2024-04-16T16:52:02.293 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : automake-1.16.2-8.el9.noarch 40/198 2024-04-16T16:52:02.293 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : bison-3.7.4-5.el9.x86_64 41/198 2024-04-16T16:52:02.293 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 42/198 2024-04-16T16:52:02.293 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : flex-2.6.4-9.el9.x86_64 43/198 2024-04-16T16:52:02.293 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 44/198 2024-04-16T16:52:02.293 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 45/198 2024-04-16T16:52:02.293 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 46/198 2024-04-16T16:52:02.293 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 47/198 2024-04-16T16:52:02.293 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 48/198 2024-04-16T16:52:02.293 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 49/198 2024-04-16T16:52:02.293 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 50/198 2024-04-16T16:52:02.293 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 51/198 2024-04-16T16:52:02.294 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 52/198 2024-04-16T16:52:02.294 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 53/198 2024-04-16T16:52:02.294 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : lua-5.4.4-4.el9.x86_64 54/198 2024-04-16T16:52:02.294 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 55/198 2024-04-16T16:52:02.294 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 56/198 2024-04-16T16:52:02.294 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 57/198 2024-04-16T16:52:02.294 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : postgresql-13.14-1.el9.x86_64 58/198 2024-04-16T16:52:02.294 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 59/198 2024-04-16T16:52:02.294 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 60/198 2024-04-16T16:52:02.294 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 61/198 2024-04-16T16:52:02.294 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 62/198 2024-04-16T16:52:02.294 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 63/198 2024-04-16T16:52:02.295 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 64/198 2024-04-16T16:52:02.295 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 65/198 2024-04-16T16:52:02.295 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 66/198 2024-04-16T16:52:02.295 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 67/198 2024-04-16T16:52:02.295 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 68/198 2024-04-16T16:52:02.295 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 69/198 2024-04-16T16:52:02.295 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 70/198 2024-04-16T16:52:02.295 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 71/198 2024-04-16T16:52:02.295 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 72/198 2024-04-16T16:52:02.295 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 73/198 2024-04-16T16:52:02.295 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 74/198 2024-04-16T16:52:02.295 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 75/198 2024-04-16T16:52:02.295 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 76/198 2024-04-16T16:52:02.295 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 77/198 2024-04-16T16:52:02.296 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 78/198 2024-04-16T16:52:02.296 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 79/198 2024-04-16T16:52:02.296 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 80/198 2024-04-16T16:52:02.296 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 81/198 2024-04-16T16:52:02.296 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 82/198 2024-04-16T16:52:02.296 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 83/198 2024-04-16T16:52:02.296 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 84/198 2024-04-16T16:52:02.296 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 85/198 2024-04-16T16:52:02.296 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 86/198 2024-04-16T16:52:02.296 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 87/198 2024-04-16T16:52:02.296 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 88/198 2024-04-16T16:52:02.296 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 89/198 2024-04-16T16:52:02.296 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 90/198 2024-04-16T16:52:02.296 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 91/198 2024-04-16T16:52:02.296 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-ply-3.11-14.el9.noarch 92/198 2024-04-16T16:52:02.297 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 93/198 2024-04-16T16:52:02.297 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 94/198 2024-04-16T16:52:02.297 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : jq-1.6-16.el9.x86_64 95/198 2024-04-16T16:52:02.297 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 96/198 2024-04-16T16:52:02.297 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 97/198 2024-04-16T16:52:02.297 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 98/198 2024-04-16T16:52:02.297 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 99/198 2024-04-16T16:52:02.297 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 100/198 2024-04-16T16:52:02.297 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 101/198 2024-04-16T16:52:02.297 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 102/198 2024-04-16T16:52:02.297 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 103/198 2024-04-16T16:52:02.297 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 104/198 2024-04-16T16:52:02.297 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/198 2024-04-16T16:52:02.297 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/198 2024-04-16T16:52:02.297 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/198 2024-04-16T16:52:02.298 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/198 2024-04-16T16:52:02.298 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/198 2024-04-16T16:52:02.298 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 110/198 2024-04-16T16:52:02.298 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 111/198 2024-04-16T16:52:02.298 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/198 2024-04-16T16:52:02.298 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/198 2024-04-16T16:52:02.298 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 114/198 2024-04-16T16:52:02.298 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 115/198 2024-04-16T16:52:02.298 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 116/198 2024-04-16T16:52:02.298 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/198 2024-04-16T16:52:02.298 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/198 2024-04-16T16:52:02.298 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 119/198 2024-04-16T16:52:02.298 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/198 2024-04-16T16:52:02.298 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/198 2024-04-16T16:52:02.299 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/198 2024-04-16T16:52:02.299 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 123/198 2024-04-16T16:52:02.299 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-google-auth-1:2.28.2-1.el9.noarch 124/198 2024-04-16T16:52:02.299 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 125/198 2024-04-16T16:52:02.299 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 126/198 2024-04-16T16:52:02.299 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 127/198 2024-04-16T16:52:02.299 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 128/198 2024-04-16T16:52:02.299 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 129/198 2024-04-16T16:52:02.299 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 130/198 2024-04-16T16:52:02.299 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 131/198 2024-04-16T16:52:02.299 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 132/198 2024-04-16T16:52:02.299 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 133/198 2024-04-16T16:52:02.299 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 134/198 2024-04-16T16:52:02.299 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 135/198 2024-04-16T16:52:02.300 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 136/198 2024-04-16T16:52:02.300 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 137/198 2024-04-16T16:52:02.300 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 138/198 2024-04-16T16:52:02.300 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 139/198 2024-04-16T16:52:02.300 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 140/198 2024-04-16T16:52:02.300 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 141/198 2024-04-16T16:52:02.300 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 142/198 2024-04-16T16:52:02.300 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 143/198 2024-04-16T16:52:02.300 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 144/198 2024-04-16T16:52:02.300 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 145/198 2024-04-16T16:52:02.300 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 146/198 2024-04-16T16:52:02.300 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 147/198 2024-04-16T16:52:02.300 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 148/198 2024-04-16T16:52:02.300 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : re2-1:20211101-3.el9.x86_64 149/198 2024-04-16T16:52:02.300 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 150/198 2024-04-16T16:52:02.301 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : librados2-2:19.0.0-3101.g9b42bfbe.el9.x86_64 151/198 2024-04-16T16:52:02.301 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 152/198 2024-04-16T16:52:02.301 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : librbd1-2:19.0.0-3101.g9b42bfbe.el9.x86_64 153/198 2024-04-16T16:52:02.301 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 154/198 2024-04-16T16:52:02.301 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 155/198 2024-04-16T16:52:02.301 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 156/198 2024-04-16T16:52:02.301 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 157/198 2024-04-16T16:52:02.301 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 158/198 2024-04-16T16:52:02.301 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 159/198 2024-04-16T16:52:02.301 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 160/198 2024-04-16T16:52:02.301 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 161/198 2024-04-16T16:52:02.301 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 162/198 2024-04-16T16:52:02.301 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 163/198 2024-04-16T16:52:02.301 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 164/198 2024-04-16T16:52:02.302 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : acl-2.3.1-4.el9.x86_64 165/198 2024-04-16T16:52:02.302 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : acl-2.3.1-3.el9.x86_64 166/198 2024-04-16T16:52:02.302 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 167/198 2024-04-16T16:52:02.302 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 168/198 2024-04-16T16:52:02.302 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 169/198 2024-04-16T16:52:02.302 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 170/198 2024-04-16T16:52:02.302 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 171/198 2024-04-16T16:52:02.302 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 172/198 2024-04-16T16:52:02.302 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 173/198 2024-04-16T16:52:02.302 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 174/198 2024-04-16T16:52:02.302 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libblkid-2.37.4-18.el9.x86_64 175/198 2024-04-16T16:52:02.302 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libblkid-2.37.4-15.el9.x86_64 176/198 2024-04-16T16:52:02.302 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libfdisk-2.37.4-18.el9.x86_64 177/198 2024-04-16T16:52:02.302 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libfdisk-2.37.4-15.el9.x86_64 178/198 2024-04-16T16:52:02.303 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libmount-2.37.4-18.el9.x86_64 179/198 2024-04-16T16:52:02.303 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libmount-2.37.4-15.el9.x86_64 180/198 2024-04-16T16:52:02.303 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libsmartcols-2.37.4-18.el9.x86_64 181/198 2024-04-16T16:52:02.303 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libsmartcols-2.37.4-15.el9.x86_64 182/198 2024-04-16T16:52:02.303 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libuuid-2.37.4-18.el9.x86_64 183/198 2024-04-16T16:52:02.303 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libuuid-2.37.4-15.el9.x86_64 184/198 2024-04-16T16:52:02.303 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 185/198 2024-04-16T16:52:02.303 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 186/198 2024-04-16T16:52:02.303 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 187/198 2024-04-16T16:52:02.303 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 188/198 2024-04-16T16:52:02.303 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : util-linux-2.37.4-18.el9.x86_64 189/198 2024-04-16T16:52:02.303 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : util-linux-2.37.4-15.el9.x86_64 190/198 2024-04-16T16:52:02.303 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : util-linux-core-2.37.4-18.el9.x86_64 191/198 2024-04-16T16:52:02.303 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : util-linux-core-2.37.4-15.el9.x86_64 192/198 2024-04-16T16:52:02.304 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : util-linux-user-2.37.4-18.el9.x86_64 193/198 2024-04-16T16:52:02.304 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : util-linux-user-2.37.4-15.el9.x86_64 194/198 2024-04-16T16:52:02.304 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 195/198 2024-04-16T16:52:02.304 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 196/198 2024-04-16T16:52:02.304 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 197/198 2024-04-16T16:52:04.055 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 198/198 2024-04-16T16:52:04.055 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-04-16T16:52:04.055 INFO:teuthology.orchestra.run.smithi104.stdout:Upgraded: 2024-04-16T16:52:04.055 INFO:teuthology.orchestra.run.smithi104.stdout: NetworkManager-1:1.47.4-1.el9.x86_64 2024-04-16T16:52:04.055 INFO:teuthology.orchestra.run.smithi104.stdout: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 2024-04-16T16:52:04.055 INFO:teuthology.orchestra.run.smithi104.stdout: NetworkManager-team-1:1.47.4-1.el9.x86_64 2024-04-16T16:52:04.055 INFO:teuthology.orchestra.run.smithi104.stdout: NetworkManager-tui-1:1.47.4-1.el9.x86_64 2024-04-16T16:52:04.055 INFO:teuthology.orchestra.run.smithi104.stdout: acl-2.3.1-4.el9.x86_64 2024-04-16T16:52:04.055 INFO:teuthology.orchestra.run.smithi104.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-04-16T16:52:04.055 INFO:teuthology.orchestra.run.smithi104.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-04-16T16:52:04.056 INFO:teuthology.orchestra.run.smithi104.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-04-16T16:52:04.056 INFO:teuthology.orchestra.run.smithi104.stdout: libacl-2.3.1-4.el9.x86_64 2024-04-16T16:52:04.056 INFO:teuthology.orchestra.run.smithi104.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-04-16T16:52:04.056 INFO:teuthology.orchestra.run.smithi104.stdout: libblkid-2.37.4-18.el9.x86_64 2024-04-16T16:52:04.056 INFO:teuthology.orchestra.run.smithi104.stdout: libfdisk-2.37.4-18.el9.x86_64 2024-04-16T16:52:04.056 INFO:teuthology.orchestra.run.smithi104.stdout: libmount-2.37.4-18.el9.x86_64 2024-04-16T16:52:04.056 INFO:teuthology.orchestra.run.smithi104.stdout: librados2-2:19.0.0-3101.g9b42bfbe.el9.x86_64 2024-04-16T16:52:04.056 INFO:teuthology.orchestra.run.smithi104.stdout: librbd1-2:19.0.0-3101.g9b42bfbe.el9.x86_64 2024-04-16T16:52:04.056 INFO:teuthology.orchestra.run.smithi104.stdout: libsmartcols-2.37.4-18.el9.x86_64 2024-04-16T16:52:04.056 INFO:teuthology.orchestra.run.smithi104.stdout: libuuid-2.37.4-18.el9.x86_64 2024-04-16T16:52:04.056 INFO:teuthology.orchestra.run.smithi104.stdout: openssl-1:3.0.7-27.el9.x86_64 2024-04-16T16:52:04.056 INFO:teuthology.orchestra.run.smithi104.stdout: openssl-libs-1:3.0.7-27.el9.x86_64 2024-04-16T16:52:04.056 INFO:teuthology.orchestra.run.smithi104.stdout: util-linux-2.37.4-18.el9.x86_64 2024-04-16T16:52:04.056 INFO:teuthology.orchestra.run.smithi104.stdout: util-linux-core-2.37.4-18.el9.x86_64 2024-04-16T16:52:04.056 INFO:teuthology.orchestra.run.smithi104.stdout: util-linux-user-2.37.4-18.el9.x86_64 2024-04-16T16:52:04.057 INFO:teuthology.orchestra.run.smithi104.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-04-16T16:52:04.057 INFO:teuthology.orchestra.run.smithi104.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-04-16T16:52:04.057 INFO:teuthology.orchestra.run.smithi104.stdout:Installed: 2024-04-16T16:52:04.057 INFO:teuthology.orchestra.run.smithi104.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-16T16:52:04.057 INFO:teuthology.orchestra.run.smithi104.stdout: automake-1.16.2-8.el9.noarch 2024-04-16T16:52:04.057 INFO:teuthology.orchestra.run.smithi104.stdout: bison-3.7.4-5.el9.x86_64 2024-04-16T16:52:04.057 INFO:teuthology.orchestra.run.smithi104.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-16T16:52:04.057 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-2:19.0.0-3101.g9b42bfbe.el9.x86_64 2024-04-16T16:52:04.057 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-base-2:19.0.0-3101.g9b42bfbe.el9.x86_64 2024-04-16T16:52:04.057 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-common-2:19.0.0-3101.g9b42bfbe.el9.x86_64 2024-04-16T16:52:04.057 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-fuse-2:19.0.0-3101.g9b42bfbe.el9.x86_64 2024-04-16T16:52:04.057 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-grafana-dashboards-2:19.0.0-3101.g9b42bfbe.el9.noarch 2024-04-16T16:52:04.057 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-immutable-object-cache-2:19.0.0-3101.g9b42bfbe.el9.x86_64 2024-04-16T16:52:04.057 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-mds-2:19.0.0-3101.g9b42bfbe.el9.x86_64 2024-04-16T16:52:04.057 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-mgr-2:19.0.0-3101.g9b42bfbe.el9.x86_64 2024-04-16T16:52:04.058 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-mgr-cephadm-2:19.0.0-3101.g9b42bfbe.el9.noarch 2024-04-16T16:52:04.058 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-mgr-dashboard-2:19.0.0-3101.g9b42bfbe.el9.noarch 2024-04-16T16:52:04.058 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3101.g9b42bfbe.el9.noarch 2024-04-16T16:52:04.058 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-mgr-modules-core-2:19.0.0-3101.g9b42bfbe.el9.noarch 2024-04-16T16:52:04.058 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-mgr-rook-2:19.0.0-3101.g9b42bfbe.el9.noarch 2024-04-16T16:52:04.058 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-mon-2:19.0.0-3101.g9b42bfbe.el9.x86_64 2024-04-16T16:52:04.058 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-osd-2:19.0.0-3101.g9b42bfbe.el9.x86_64 2024-04-16T16:52:04.058 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-prometheus-alerts-2:19.0.0-3101.g9b42bfbe.el9.noarch 2024-04-16T16:52:04.058 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-radosgw-2:19.0.0-3101.g9b42bfbe.el9.x86_64 2024-04-16T16:52:04.058 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-selinux-2:19.0.0-3101.g9b42bfbe.el9.x86_64 2024-04-16T16:52:04.058 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-test-2:19.0.0-3101.g9b42bfbe.el9.x86_64 2024-04-16T16:52:04.058 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-volume-2:19.0.0-3101.g9b42bfbe.el9.noarch 2024-04-16T16:52:04.058 INFO:teuthology.orchestra.run.smithi104.stdout: cephadm-2:19.0.0-3101.g9b42bfbe.el9.noarch 2024-04-16T16:52:04.059 INFO:teuthology.orchestra.run.smithi104.stdout: cephfs-mirror-2:19.0.0-3101.g9b42bfbe.el9.x86_64 2024-04-16T16:52:04.059 INFO:teuthology.orchestra.run.smithi104.stdout: cephfs-shell-2:19.0.0-3101.g9b42bfbe.el9.x86_64 2024-04-16T16:52:04.059 INFO:teuthology.orchestra.run.smithi104.stdout: cephfs-top-2:19.0.0-3101.g9b42bfbe.el9.noarch 2024-04-16T16:52:04.059 INFO:teuthology.orchestra.run.smithi104.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-16T16:52:04.059 INFO:teuthology.orchestra.run.smithi104.stdout: flex-2.6.4-9.el9.x86_64 2024-04-16T16:52:04.059 INFO:teuthology.orchestra.run.smithi104.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-16T16:52:04.059 INFO:teuthology.orchestra.run.smithi104.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-16T16:52:04.059 INFO:teuthology.orchestra.run.smithi104.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-16T16:52:04.059 INFO:teuthology.orchestra.run.smithi104.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-16T16:52:04.059 INFO:teuthology.orchestra.run.smithi104.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-16T16:52:04.059 INFO:teuthology.orchestra.run.smithi104.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-16T16:52:04.059 INFO:teuthology.orchestra.run.smithi104.stdout: jq-1.6-16.el9.x86_64 2024-04-16T16:52:04.059 INFO:teuthology.orchestra.run.smithi104.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-16T16:52:04.060 INFO:teuthology.orchestra.run.smithi104.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-16T16:52:04.060 INFO:teuthology.orchestra.run.smithi104.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-16T16:52:04.060 INFO:teuthology.orchestra.run.smithi104.stdout: libcephfs-devel-2:19.0.0-3101.g9b42bfbe.el9.x86_64 2024-04-16T16:52:04.060 INFO:teuthology.orchestra.run.smithi104.stdout: libcephfs2-2:19.0.0-3101.g9b42bfbe.el9.x86_64 2024-04-16T16:52:04.060 INFO:teuthology.orchestra.run.smithi104.stdout: libcephsqlite-2:19.0.0-3101.g9b42bfbe.el9.x86_64 2024-04-16T16:52:04.060 INFO:teuthology.orchestra.run.smithi104.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-16T16:52:04.060 INFO:teuthology.orchestra.run.smithi104.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-16T16:52:04.060 INFO:teuthology.orchestra.run.smithi104.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-16T16:52:04.060 INFO:teuthology.orchestra.run.smithi104.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-16T16:52:04.060 INFO:teuthology.orchestra.run.smithi104.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-16T16:52:04.060 INFO:teuthology.orchestra.run.smithi104.stdout: librados-devel-2:19.0.0-3101.g9b42bfbe.el9.x86_64 2024-04-16T16:52:04.060 INFO:teuthology.orchestra.run.smithi104.stdout: libradosstriper1-2:19.0.0-3101.g9b42bfbe.el9.x86_64 2024-04-16T16:52:04.060 INFO:teuthology.orchestra.run.smithi104.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-16T16:52:04.061 INFO:teuthology.orchestra.run.smithi104.stdout: librgw2-2:19.0.0-3101.g9b42bfbe.el9.x86_64 2024-04-16T16:52:04.061 INFO:teuthology.orchestra.run.smithi104.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-16T16:52:04.061 INFO:teuthology.orchestra.run.smithi104.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-16T16:52:04.061 INFO:teuthology.orchestra.run.smithi104.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-16T16:52:04.061 INFO:teuthology.orchestra.run.smithi104.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-16T16:52:04.061 INFO:teuthology.orchestra.run.smithi104.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-16T16:52:04.061 INFO:teuthology.orchestra.run.smithi104.stdout: lua-5.4.4-4.el9.x86_64 2024-04-16T16:52:04.061 INFO:teuthology.orchestra.run.smithi104.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-16T16:52:04.061 INFO:teuthology.orchestra.run.smithi104.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-16T16:52:04.061 INFO:teuthology.orchestra.run.smithi104.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-04-16T16:52:04.061 INFO:teuthology.orchestra.run.smithi104.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-16T16:52:04.061 INFO:teuthology.orchestra.run.smithi104.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-16T16:52:04.061 INFO:teuthology.orchestra.run.smithi104.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-04-16T16:52:04.061 INFO:teuthology.orchestra.run.smithi104.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-16T16:52:04.061 INFO:teuthology.orchestra.run.smithi104.stdout: postgresql-13.14-1.el9.x86_64 2024-04-16T16:52:04.062 INFO:teuthology.orchestra.run.smithi104.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-16T16:52:04.062 INFO:teuthology.orchestra.run.smithi104.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-16T16:52:04.062 INFO:teuthology.orchestra.run.smithi104.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-16T16:52:04.062 INFO:teuthology.orchestra.run.smithi104.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-16T16:52:04.062 INFO:teuthology.orchestra.run.smithi104.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-16T16:52:04.062 INFO:teuthology.orchestra.run.smithi104.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-16T16:52:04.062 INFO:teuthology.orchestra.run.smithi104.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-16T16:52:04.062 INFO:teuthology.orchestra.run.smithi104.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-16T16:52:04.062 INFO:teuthology.orchestra.run.smithi104.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-16T16:52:04.062 INFO:teuthology.orchestra.run.smithi104.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-16T16:52:04.062 INFO:teuthology.orchestra.run.smithi104.stdout: python3-ceph-argparse-2:19.0.0-3101.g9b42bfbe.el9.x86_64 2024-04-16T16:52:04.062 INFO:teuthology.orchestra.run.smithi104.stdout: python3-ceph-common-2:19.0.0-3101.g9b42bfbe.el9.x86_64 2024-04-16T16:52:04.063 INFO:teuthology.orchestra.run.smithi104.stdout: python3-cephfs-2:19.0.0-3101.g9b42bfbe.el9.x86_64 2024-04-16T16:52:04.063 INFO:teuthology.orchestra.run.smithi104.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-16T16:52:04.063 INFO:teuthology.orchestra.run.smithi104.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-16T16:52:04.063 INFO:teuthology.orchestra.run.smithi104.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-16T16:52:04.063 INFO:teuthology.orchestra.run.smithi104.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-16T16:52:04.063 INFO:teuthology.orchestra.run.smithi104.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-16T16:52:04.063 INFO:teuthology.orchestra.run.smithi104.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-16T16:52:04.063 INFO:teuthology.orchestra.run.smithi104.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-16T16:52:04.063 INFO:teuthology.orchestra.run.smithi104.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-04-16T16:52:04.063 INFO:teuthology.orchestra.run.smithi104.stdout: python3-google-auth-1:2.28.2-1.el9.noarch 2024-04-16T16:52:04.063 INFO:teuthology.orchestra.run.smithi104.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-16T16:52:04.063 INFO:teuthology.orchestra.run.smithi104.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-16T16:52:04.063 INFO:teuthology.orchestra.run.smithi104.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-16T16:52:04.063 INFO:teuthology.orchestra.run.smithi104.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-16T16:52:04.064 INFO:teuthology.orchestra.run.smithi104.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-16T16:52:04.064 INFO:teuthology.orchestra.run.smithi104.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-16T16:52:04.064 INFO:teuthology.orchestra.run.smithi104.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-16T16:52:04.064 INFO:teuthology.orchestra.run.smithi104.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-16T16:52:04.064 INFO:teuthology.orchestra.run.smithi104.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-16T16:52:04.064 INFO:teuthology.orchestra.run.smithi104.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-16T16:52:04.064 INFO:teuthology.orchestra.run.smithi104.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-16T16:52:04.064 INFO:teuthology.orchestra.run.smithi104.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-16T16:52:04.064 INFO:teuthology.orchestra.run.smithi104.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-16T16:52:04.064 INFO:teuthology.orchestra.run.smithi104.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-16T16:52:04.064 INFO:teuthology.orchestra.run.smithi104.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-16T16:52:04.064 INFO:teuthology.orchestra.run.smithi104.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-16T16:52:04.064 INFO:teuthology.orchestra.run.smithi104.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-16T16:52:04.064 INFO:teuthology.orchestra.run.smithi104.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-16T16:52:04.065 INFO:teuthology.orchestra.run.smithi104.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-16T16:52:04.065 INFO:teuthology.orchestra.run.smithi104.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-16T16:52:04.065 INFO:teuthology.orchestra.run.smithi104.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-16T16:52:04.065 INFO:teuthology.orchestra.run.smithi104.stdout: python3-ply-3.11-14.el9.noarch 2024-04-16T16:52:04.065 INFO:teuthology.orchestra.run.smithi104.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-16T16:52:04.065 INFO:teuthology.orchestra.run.smithi104.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-16T16:52:04.065 INFO:teuthology.orchestra.run.smithi104.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-16T16:52:04.065 INFO:teuthology.orchestra.run.smithi104.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-16T16:52:04.065 INFO:teuthology.orchestra.run.smithi104.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-16T16:52:04.065 INFO:teuthology.orchestra.run.smithi104.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-16T16:52:04.065 INFO:teuthology.orchestra.run.smithi104.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-16T16:52:04.065 INFO:teuthology.orchestra.run.smithi104.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-16T16:52:04.065 INFO:teuthology.orchestra.run.smithi104.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-16T16:52:04.065 INFO:teuthology.orchestra.run.smithi104.stdout: python3-rados-2:19.0.0-3101.g9b42bfbe.el9.x86_64 2024-04-16T16:52:04.065 INFO:teuthology.orchestra.run.smithi104.stdout: python3-rbd-2:19.0.0-3101.g9b42bfbe.el9.x86_64 2024-04-16T16:52:04.066 INFO:teuthology.orchestra.run.smithi104.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-16T16:52:04.066 INFO:teuthology.orchestra.run.smithi104.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-16T16:52:04.066 INFO:teuthology.orchestra.run.smithi104.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-16T16:52:04.066 INFO:teuthology.orchestra.run.smithi104.stdout: python3-rgw-2:19.0.0-3101.g9b42bfbe.el9.x86_64 2024-04-16T16:52:04.066 INFO:teuthology.orchestra.run.smithi104.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-16T16:52:04.066 INFO:teuthology.orchestra.run.smithi104.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-16T16:52:04.066 INFO:teuthology.orchestra.run.smithi104.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-16T16:52:04.066 INFO:teuthology.orchestra.run.smithi104.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-16T16:52:04.066 INFO:teuthology.orchestra.run.smithi104.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-16T16:52:04.066 INFO:teuthology.orchestra.run.smithi104.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-16T16:52:04.066 INFO:teuthology.orchestra.run.smithi104.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-16T16:52:04.066 INFO:teuthology.orchestra.run.smithi104.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-16T16:52:04.066 INFO:teuthology.orchestra.run.smithi104.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-16T16:52:04.066 INFO:teuthology.orchestra.run.smithi104.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-16T16:52:04.067 INFO:teuthology.orchestra.run.smithi104.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-16T16:52:04.067 INFO:teuthology.orchestra.run.smithi104.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-16T16:52:04.067 INFO:teuthology.orchestra.run.smithi104.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-16T16:52:04.067 INFO:teuthology.orchestra.run.smithi104.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-16T16:52:04.067 INFO:teuthology.orchestra.run.smithi104.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-16T16:52:04.067 INFO:teuthology.orchestra.run.smithi104.stdout: rbd-fuse-2:19.0.0-3101.g9b42bfbe.el9.x86_64 2024-04-16T16:52:04.067 INFO:teuthology.orchestra.run.smithi104.stdout: rbd-mirror-2:19.0.0-3101.g9b42bfbe.el9.x86_64 2024-04-16T16:52:04.067 INFO:teuthology.orchestra.run.smithi104.stdout: rbd-nbd-2:19.0.0-3101.g9b42bfbe.el9.x86_64 2024-04-16T16:52:04.067 INFO:teuthology.orchestra.run.smithi104.stdout: re2-1:20211101-3.el9.x86_64 2024-04-16T16:52:04.067 INFO:teuthology.orchestra.run.smithi104.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-16T16:52:04.067 INFO:teuthology.orchestra.run.smithi104.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-16T16:52:04.067 INFO:teuthology.orchestra.run.smithi104.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-16T16:52:04.067 INFO:teuthology.orchestra.run.smithi104.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-16T16:52:04.067 INFO:teuthology.orchestra.run.smithi104.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-16T16:52:04.068 INFO:teuthology.orchestra.run.smithi104.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-04-16T16:52:04.068 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-04-16T16:52:04.068 INFO:teuthology.orchestra.run.smithi104.stdout:Complete! 2024-04-16T16:52:04.572 DEBUG:teuthology.parallel:result is None 2024-04-16T16:52:05.135 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 198/198 2024-04-16T16:52:05.135 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-16T16:52:05.135 INFO:teuthology.orchestra.run.smithi117.stdout:Upgraded: 2024-04-16T16:52:05.135 INFO:teuthology.orchestra.run.smithi117.stdout: NetworkManager-1:1.47.4-1.el9.x86_64 2024-04-16T16:52:05.136 INFO:teuthology.orchestra.run.smithi117.stdout: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 2024-04-16T16:52:05.136 INFO:teuthology.orchestra.run.smithi117.stdout: NetworkManager-team-1:1.47.4-1.el9.x86_64 2024-04-16T16:52:05.136 INFO:teuthology.orchestra.run.smithi117.stdout: NetworkManager-tui-1:1.47.4-1.el9.x86_64 2024-04-16T16:52:05.136 INFO:teuthology.orchestra.run.smithi117.stdout: acl-2.3.1-4.el9.x86_64 2024-04-16T16:52:05.136 INFO:teuthology.orchestra.run.smithi117.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-04-16T16:52:05.136 INFO:teuthology.orchestra.run.smithi117.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-04-16T16:52:05.136 INFO:teuthology.orchestra.run.smithi117.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-04-16T16:52:05.136 INFO:teuthology.orchestra.run.smithi117.stdout: libacl-2.3.1-4.el9.x86_64 2024-04-16T16:52:05.136 INFO:teuthology.orchestra.run.smithi117.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-04-16T16:52:05.136 INFO:teuthology.orchestra.run.smithi117.stdout: libblkid-2.37.4-18.el9.x86_64 2024-04-16T16:52:05.136 INFO:teuthology.orchestra.run.smithi117.stdout: libfdisk-2.37.4-18.el9.x86_64 2024-04-16T16:52:05.136 INFO:teuthology.orchestra.run.smithi117.stdout: libmount-2.37.4-18.el9.x86_64 2024-04-16T16:52:05.136 INFO:teuthology.orchestra.run.smithi117.stdout: librados2-2:19.0.0-3101.g9b42bfbe.el9.x86_64 2024-04-16T16:52:05.136 INFO:teuthology.orchestra.run.smithi117.stdout: librbd1-2:19.0.0-3101.g9b42bfbe.el9.x86_64 2024-04-16T16:52:05.137 INFO:teuthology.orchestra.run.smithi117.stdout: libsmartcols-2.37.4-18.el9.x86_64 2024-04-16T16:52:05.137 INFO:teuthology.orchestra.run.smithi117.stdout: libuuid-2.37.4-18.el9.x86_64 2024-04-16T16:52:05.137 INFO:teuthology.orchestra.run.smithi117.stdout: openssl-1:3.0.7-27.el9.x86_64 2024-04-16T16:52:05.137 INFO:teuthology.orchestra.run.smithi117.stdout: openssl-libs-1:3.0.7-27.el9.x86_64 2024-04-16T16:52:05.137 INFO:teuthology.orchestra.run.smithi117.stdout: util-linux-2.37.4-18.el9.x86_64 2024-04-16T16:52:05.137 INFO:teuthology.orchestra.run.smithi117.stdout: util-linux-core-2.37.4-18.el9.x86_64 2024-04-16T16:52:05.137 INFO:teuthology.orchestra.run.smithi117.stdout: util-linux-user-2.37.4-18.el9.x86_64 2024-04-16T16:52:05.137 INFO:teuthology.orchestra.run.smithi117.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-04-16T16:52:05.137 INFO:teuthology.orchestra.run.smithi117.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-04-16T16:52:05.137 INFO:teuthology.orchestra.run.smithi117.stdout:Installed: 2024-04-16T16:52:05.137 INFO:teuthology.orchestra.run.smithi117.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-16T16:52:05.137 INFO:teuthology.orchestra.run.smithi117.stdout: automake-1.16.2-8.el9.noarch 2024-04-16T16:52:05.137 INFO:teuthology.orchestra.run.smithi117.stdout: bison-3.7.4-5.el9.x86_64 2024-04-16T16:52:05.137 INFO:teuthology.orchestra.run.smithi117.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-16T16:52:05.138 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-2:19.0.0-3101.g9b42bfbe.el9.x86_64 2024-04-16T16:52:05.138 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-base-2:19.0.0-3101.g9b42bfbe.el9.x86_64 2024-04-16T16:52:05.138 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-common-2:19.0.0-3101.g9b42bfbe.el9.x86_64 2024-04-16T16:52:05.138 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-fuse-2:19.0.0-3101.g9b42bfbe.el9.x86_64 2024-04-16T16:52:05.138 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-grafana-dashboards-2:19.0.0-3101.g9b42bfbe.el9.noarch 2024-04-16T16:52:05.138 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-immutable-object-cache-2:19.0.0-3101.g9b42bfbe.el9.x86_64 2024-04-16T16:52:05.138 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mds-2:19.0.0-3101.g9b42bfbe.el9.x86_64 2024-04-16T16:52:05.138 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mgr-2:19.0.0-3101.g9b42bfbe.el9.x86_64 2024-04-16T16:52:05.138 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mgr-cephadm-2:19.0.0-3101.g9b42bfbe.el9.noarch 2024-04-16T16:52:05.138 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mgr-dashboard-2:19.0.0-3101.g9b42bfbe.el9.noarch 2024-04-16T16:52:05.138 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3101.g9b42bfbe.el9.noarch 2024-04-16T16:52:05.138 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mgr-modules-core-2:19.0.0-3101.g9b42bfbe.el9.noarch 2024-04-16T16:52:05.138 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mgr-rook-2:19.0.0-3101.g9b42bfbe.el9.noarch 2024-04-16T16:52:05.138 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mon-2:19.0.0-3101.g9b42bfbe.el9.x86_64 2024-04-16T16:52:05.139 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-osd-2:19.0.0-3101.g9b42bfbe.el9.x86_64 2024-04-16T16:52:05.139 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-prometheus-alerts-2:19.0.0-3101.g9b42bfbe.el9.noarch 2024-04-16T16:52:05.139 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-radosgw-2:19.0.0-3101.g9b42bfbe.el9.x86_64 2024-04-16T16:52:05.139 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-selinux-2:19.0.0-3101.g9b42bfbe.el9.x86_64 2024-04-16T16:52:05.139 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-test-2:19.0.0-3101.g9b42bfbe.el9.x86_64 2024-04-16T16:52:05.139 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-volume-2:19.0.0-3101.g9b42bfbe.el9.noarch 2024-04-16T16:52:05.139 INFO:teuthology.orchestra.run.smithi117.stdout: cephadm-2:19.0.0-3101.g9b42bfbe.el9.noarch 2024-04-16T16:52:05.139 INFO:teuthology.orchestra.run.smithi117.stdout: cephfs-mirror-2:19.0.0-3101.g9b42bfbe.el9.x86_64 2024-04-16T16:52:05.139 INFO:teuthology.orchestra.run.smithi117.stdout: cephfs-shell-2:19.0.0-3101.g9b42bfbe.el9.x86_64 2024-04-16T16:52:05.139 INFO:teuthology.orchestra.run.smithi117.stdout: cephfs-top-2:19.0.0-3101.g9b42bfbe.el9.noarch 2024-04-16T16:52:05.139 INFO:teuthology.orchestra.run.smithi117.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-16T16:52:05.139 INFO:teuthology.orchestra.run.smithi117.stdout: flex-2.6.4-9.el9.x86_64 2024-04-16T16:52:05.139 INFO:teuthology.orchestra.run.smithi117.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-16T16:52:05.139 INFO:teuthology.orchestra.run.smithi117.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-16T16:52:05.139 INFO:teuthology.orchestra.run.smithi117.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-16T16:52:05.140 INFO:teuthology.orchestra.run.smithi117.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-16T16:52:05.140 INFO:teuthology.orchestra.run.smithi117.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-16T16:52:05.140 INFO:teuthology.orchestra.run.smithi117.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-16T16:52:05.140 INFO:teuthology.orchestra.run.smithi117.stdout: jq-1.6-16.el9.x86_64 2024-04-16T16:52:05.140 INFO:teuthology.orchestra.run.smithi117.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-16T16:52:05.140 INFO:teuthology.orchestra.run.smithi117.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-16T16:52:05.140 INFO:teuthology.orchestra.run.smithi117.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-16T16:52:05.140 INFO:teuthology.orchestra.run.smithi117.stdout: libcephfs-devel-2:19.0.0-3101.g9b42bfbe.el9.x86_64 2024-04-16T16:52:05.140 INFO:teuthology.orchestra.run.smithi117.stdout: libcephfs2-2:19.0.0-3101.g9b42bfbe.el9.x86_64 2024-04-16T16:52:05.140 INFO:teuthology.orchestra.run.smithi117.stdout: libcephsqlite-2:19.0.0-3101.g9b42bfbe.el9.x86_64 2024-04-16T16:52:05.140 INFO:teuthology.orchestra.run.smithi117.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-16T16:52:05.140 INFO:teuthology.orchestra.run.smithi117.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-16T16:52:05.140 INFO:teuthology.orchestra.run.smithi117.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-16T16:52:05.140 INFO:teuthology.orchestra.run.smithi117.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-16T16:52:05.140 INFO:teuthology.orchestra.run.smithi117.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-16T16:52:05.141 INFO:teuthology.orchestra.run.smithi117.stdout: librados-devel-2:19.0.0-3101.g9b42bfbe.el9.x86_64 2024-04-16T16:52:05.141 INFO:teuthology.orchestra.run.smithi117.stdout: libradosstriper1-2:19.0.0-3101.g9b42bfbe.el9.x86_64 2024-04-16T16:52:05.141 INFO:teuthology.orchestra.run.smithi117.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-16T16:52:05.141 INFO:teuthology.orchestra.run.smithi117.stdout: librgw2-2:19.0.0-3101.g9b42bfbe.el9.x86_64 2024-04-16T16:52:05.141 INFO:teuthology.orchestra.run.smithi117.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-16T16:52:05.141 INFO:teuthology.orchestra.run.smithi117.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-16T16:52:05.141 INFO:teuthology.orchestra.run.smithi117.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-16T16:52:05.141 INFO:teuthology.orchestra.run.smithi117.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-16T16:52:05.141 INFO:teuthology.orchestra.run.smithi117.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-16T16:52:05.141 INFO:teuthology.orchestra.run.smithi117.stdout: lua-5.4.4-4.el9.x86_64 2024-04-16T16:52:05.141 INFO:teuthology.orchestra.run.smithi117.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-16T16:52:05.141 INFO:teuthology.orchestra.run.smithi117.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-16T16:52:05.141 INFO:teuthology.orchestra.run.smithi117.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-04-16T16:52:05.141 INFO:teuthology.orchestra.run.smithi117.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-16T16:52:05.142 INFO:teuthology.orchestra.run.smithi117.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-16T16:52:05.142 INFO:teuthology.orchestra.run.smithi117.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-04-16T16:52:05.142 INFO:teuthology.orchestra.run.smithi117.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-16T16:52:05.142 INFO:teuthology.orchestra.run.smithi117.stdout: postgresql-13.14-1.el9.x86_64 2024-04-16T16:52:05.142 INFO:teuthology.orchestra.run.smithi117.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-16T16:52:05.142 INFO:teuthology.orchestra.run.smithi117.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-16T16:52:05.142 INFO:teuthology.orchestra.run.smithi117.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-16T16:52:05.142 INFO:teuthology.orchestra.run.smithi117.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-16T16:52:05.142 INFO:teuthology.orchestra.run.smithi117.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-16T16:52:05.142 INFO:teuthology.orchestra.run.smithi117.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-16T16:52:05.142 INFO:teuthology.orchestra.run.smithi117.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-16T16:52:05.142 INFO:teuthology.orchestra.run.smithi117.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-16T16:52:05.142 INFO:teuthology.orchestra.run.smithi117.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-16T16:52:05.142 INFO:teuthology.orchestra.run.smithi117.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-16T16:52:05.143 INFO:teuthology.orchestra.run.smithi117.stdout: python3-ceph-argparse-2:19.0.0-3101.g9b42bfbe.el9.x86_64 2024-04-16T16:52:05.143 INFO:teuthology.orchestra.run.smithi117.stdout: python3-ceph-common-2:19.0.0-3101.g9b42bfbe.el9.x86_64 2024-04-16T16:52:05.143 INFO:teuthology.orchestra.run.smithi117.stdout: python3-cephfs-2:19.0.0-3101.g9b42bfbe.el9.x86_64 2024-04-16T16:52:05.143 INFO:teuthology.orchestra.run.smithi117.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-16T16:52:05.143 INFO:teuthology.orchestra.run.smithi117.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-16T16:52:05.143 INFO:teuthology.orchestra.run.smithi117.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-16T16:52:05.143 INFO:teuthology.orchestra.run.smithi117.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-16T16:52:05.143 INFO:teuthology.orchestra.run.smithi117.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-16T16:52:05.143 INFO:teuthology.orchestra.run.smithi117.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-16T16:52:05.143 INFO:teuthology.orchestra.run.smithi117.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-16T16:52:05.143 INFO:teuthology.orchestra.run.smithi117.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-04-16T16:52:05.143 INFO:teuthology.orchestra.run.smithi117.stdout: python3-google-auth-1:2.28.2-1.el9.noarch 2024-04-16T16:52:05.143 INFO:teuthology.orchestra.run.smithi117.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-16T16:52:05.143 INFO:teuthology.orchestra.run.smithi117.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-16T16:52:05.144 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-16T16:52:05.144 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-16T16:52:05.144 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-16T16:52:05.144 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-16T16:52:05.144 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-16T16:52:05.144 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-16T16:52:05.144 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-16T16:52:05.144 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-16T16:52:05.144 INFO:teuthology.orchestra.run.smithi117.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-16T16:52:05.144 INFO:teuthology.orchestra.run.smithi117.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-16T16:52:05.144 INFO:teuthology.orchestra.run.smithi117.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-16T16:52:05.144 INFO:teuthology.orchestra.run.smithi117.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-16T16:52:05.144 INFO:teuthology.orchestra.run.smithi117.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-16T16:52:05.144 INFO:teuthology.orchestra.run.smithi117.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-16T16:52:05.144 INFO:teuthology.orchestra.run.smithi117.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-16T16:52:05.145 INFO:teuthology.orchestra.run.smithi117.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-16T16:52:05.145 INFO:teuthology.orchestra.run.smithi117.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-16T16:52:05.145 INFO:teuthology.orchestra.run.smithi117.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-16T16:52:05.145 INFO:teuthology.orchestra.run.smithi117.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-16T16:52:05.145 INFO:teuthology.orchestra.run.smithi117.stdout: python3-ply-3.11-14.el9.noarch 2024-04-16T16:52:05.145 INFO:teuthology.orchestra.run.smithi117.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-16T16:52:05.145 INFO:teuthology.orchestra.run.smithi117.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-16T16:52:05.145 INFO:teuthology.orchestra.run.smithi117.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-16T16:52:05.145 INFO:teuthology.orchestra.run.smithi117.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-16T16:52:05.145 INFO:teuthology.orchestra.run.smithi117.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-16T16:52:05.145 INFO:teuthology.orchestra.run.smithi117.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-16T16:52:05.145 INFO:teuthology.orchestra.run.smithi117.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-16T16:52:05.145 INFO:teuthology.orchestra.run.smithi117.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-16T16:52:05.145 INFO:teuthology.orchestra.run.smithi117.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-16T16:52:05.145 INFO:teuthology.orchestra.run.smithi117.stdout: python3-rados-2:19.0.0-3101.g9b42bfbe.el9.x86_64 2024-04-16T16:52:05.145 INFO:teuthology.orchestra.run.smithi117.stdout: python3-rbd-2:19.0.0-3101.g9b42bfbe.el9.x86_64 2024-04-16T16:52:05.146 INFO:teuthology.orchestra.run.smithi117.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-16T16:52:05.146 INFO:teuthology.orchestra.run.smithi117.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-16T16:52:05.146 INFO:teuthology.orchestra.run.smithi117.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-16T16:52:05.146 INFO:teuthology.orchestra.run.smithi117.stdout: python3-rgw-2:19.0.0-3101.g9b42bfbe.el9.x86_64 2024-04-16T16:52:05.146 INFO:teuthology.orchestra.run.smithi117.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-16T16:52:05.146 INFO:teuthology.orchestra.run.smithi117.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-16T16:52:05.146 INFO:teuthology.orchestra.run.smithi117.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-16T16:52:05.146 INFO:teuthology.orchestra.run.smithi117.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-16T16:52:05.146 INFO:teuthology.orchestra.run.smithi117.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-16T16:52:05.146 INFO:teuthology.orchestra.run.smithi117.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-16T16:52:05.146 INFO:teuthology.orchestra.run.smithi117.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-16T16:52:05.146 INFO:teuthology.orchestra.run.smithi117.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-16T16:52:05.146 INFO:teuthology.orchestra.run.smithi117.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-16T16:52:05.146 INFO:teuthology.orchestra.run.smithi117.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-16T16:52:05.147 INFO:teuthology.orchestra.run.smithi117.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-16T16:52:05.147 INFO:teuthology.orchestra.run.smithi117.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-16T16:52:05.147 INFO:teuthology.orchestra.run.smithi117.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-16T16:52:05.147 INFO:teuthology.orchestra.run.smithi117.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-16T16:52:05.147 INFO:teuthology.orchestra.run.smithi117.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-16T16:52:05.147 INFO:teuthology.orchestra.run.smithi117.stdout: rbd-fuse-2:19.0.0-3101.g9b42bfbe.el9.x86_64 2024-04-16T16:52:05.147 INFO:teuthology.orchestra.run.smithi117.stdout: rbd-mirror-2:19.0.0-3101.g9b42bfbe.el9.x86_64 2024-04-16T16:52:05.147 INFO:teuthology.orchestra.run.smithi117.stdout: rbd-nbd-2:19.0.0-3101.g9b42bfbe.el9.x86_64 2024-04-16T16:52:05.147 INFO:teuthology.orchestra.run.smithi117.stdout: re2-1:20211101-3.el9.x86_64 2024-04-16T16:52:05.147 INFO:teuthology.orchestra.run.smithi117.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-16T16:52:05.147 INFO:teuthology.orchestra.run.smithi117.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-16T16:52:05.147 INFO:teuthology.orchestra.run.smithi117.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-16T16:52:05.147 INFO:teuthology.orchestra.run.smithi117.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-16T16:52:05.147 INFO:teuthology.orchestra.run.smithi117.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-16T16:52:05.148 INFO:teuthology.orchestra.run.smithi117.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-04-16T16:52:05.148 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-16T16:52:05.148 INFO:teuthology.orchestra.run.smithi117.stdout:Complete! 2024-04-16T16:52:05.150 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-test-2:19.0.0-3101.g9b42bfbe.el9.x86_64 153/198 2024-04-16T16:52:05.195 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : rbd-fuse-2:19.0.0-3101.g9b42bfbe.el9.x86_64 154/198 2024-04-16T16:52:05.231 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : rbd-nbd-2:19.0.0-3101.g9b42bfbe.el9.x86_64 155/198 2024-04-16T16:52:05.269 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 156/198 2024-04-16T16:52:05.555 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : cephfs-shell-2:19.0.0-3101.g9b42bfbe.el9.x86_64 157/198 2024-04-16T16:52:05.623 DEBUG:teuthology.parallel:result is None 2024-04-16T16:52:05.691 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 158/198 2024-04-16T16:52:05.743 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-fuse-2:19.0.0-3101.g9b42bfbe.el9.x86_64 159/198 2024-04-16T16:52:05.804 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 160/198 2024-04-16T16:52:05.883 INFO:teuthology.orchestra.run.smithi045.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 161/198 2024-04-16T16:52:05.939 INFO:teuthology.orchestra.run.smithi045.stdout: Upgrading : util-linux-user-2.37.4-18.el9.x86_64 162/198 2024-04-16T16:52:05.992 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libcephfs-devel-2:19.0.0-3101.g9b42bfbe.el9.x86_ 163/198 2024-04-16T16:52:06.034 INFO:teuthology.orchestra.run.smithi045.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 164/198 2024-04-16T16:52:06.086 INFO:teuthology.orchestra.run.smithi045.stdout: Upgrading : NetworkManager-team-1:1.47.4-1.el9.x86_64 165/198 2024-04-16T16:52:06.195 INFO:teuthology.orchestra.run.smithi045.stdout: Upgrading : NetworkManager-tui-1:1.47.4-1.el9.x86_64 166/198 2024-04-16T16:52:06.308 INFO:teuthology.orchestra.run.smithi045.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 167/198 2024-04-16T16:52:06.351 INFO:teuthology.orchestra.run.smithi045.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 168/198 2024-04-16T16:52:07.337 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 169/198 2024-04-16T16:52:07.383 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 169/198 2024-04-16T16:52:07.452 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 169/198 2024-04-16T16:52:08.885 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : cephfs-top-2:19.0.0-3101.g9b42bfbe.el9.noarch 170/198 2024-04-16T16:52:08.980 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 171/198 2024-04-16T16:52:09.034 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 172/198 2024-04-16T16:52:09.180 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : flex-2.6.4-9.el9.x86_64 173/198 2024-04-16T16:52:09.225 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : bison-3.7.4-5.el9.x86_64 174/198 2024-04-16T16:52:09.290 INFO:teuthology.orchestra.run.smithi045.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 175/198 2024-04-16T16:52:09.326 INFO:teuthology.orchestra.run.smithi045.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 176/198 2024-04-16T16:52:09.370 INFO:teuthology.orchestra.run.smithi045.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 177/198 2024-04-16T16:52:09.370 INFO:teuthology.orchestra.run.smithi045.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-04-16T16:52:09.506 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-04-16T16:52:09.556 INFO:teuthology.orchestra.run.smithi045.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 179/198 2024-04-16T16:52:09.556 INFO:teuthology.orchestra.run.smithi045.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 180/198 2024-04-16T16:52:09.710 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 180/198 2024-04-16T16:52:09.756 INFO:teuthology.orchestra.run.smithi045.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 181/198 2024-04-16T16:52:09.801 INFO:teuthology.orchestra.run.smithi045.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 182/198 2024-04-16T16:52:09.843 INFO:teuthology.orchestra.run.smithi045.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 183/198 2024-04-16T16:52:09.883 INFO:teuthology.orchestra.run.smithi045.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 184/198 2024-04-16T16:52:09.972 INFO:teuthology.orchestra.run.smithi045.stdout: Cleanup : util-linux-user-2.37.4-15.el9.x86_64 185/198 2024-04-16T16:52:10.087 INFO:teuthology.orchestra.run.smithi045.stdout: Cleanup : util-linux-2.37.4-15.el9.x86_64 186/198 2024-04-16T16:52:10.139 INFO:teuthology.orchestra.run.smithi045.stdout: Cleanup : util-linux-core-2.37.4-15.el9.x86_64 187/198 2024-04-16T16:52:10.172 INFO:teuthology.orchestra.run.smithi045.stdout: Cleanup : libfdisk-2.37.4-15.el9.x86_64 188/198 2024-04-16T16:52:10.205 INFO:teuthology.orchestra.run.smithi045.stdout: Cleanup : libmount-2.37.4-15.el9.x86_64 189/198 2024-04-16T16:52:10.239 INFO:teuthology.orchestra.run.smithi045.stdout: Cleanup : libblkid-2.37.4-15.el9.x86_64 190/198 2024-04-16T16:52:10.281 INFO:teuthology.orchestra.run.smithi045.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 191/198 2024-04-16T16:52:10.319 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-04-16T16:52:10.320 INFO:teuthology.orchestra.run.smithi045.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-04-16T16:52:10.640 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-04-16T16:52:10.640 INFO:teuthology.orchestra.run.smithi045.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-04-16T16:52:10.779 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-04-16T16:52:10.831 INFO:teuthology.orchestra.run.smithi045.stdout: Cleanup : libuuid-2.37.4-15.el9.x86_64 194/198 2024-04-16T16:52:10.876 INFO:teuthology.orchestra.run.smithi045.stdout: Cleanup : libsmartcols-2.37.4-15.el9.x86_64 195/198 2024-04-16T16:52:10.912 INFO:teuthology.orchestra.run.smithi045.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 196/198 2024-04-16T16:52:10.952 INFO:teuthology.orchestra.run.smithi045.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 197/198 2024-04-16T16:52:11.077 INFO:teuthology.orchestra.run.smithi045.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-04-16T16:52:11.349 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-04-16T16:52:11.349 INFO:teuthology.orchestra.run.smithi045.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-04-16T16:52:11.349 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-16T16:52:16.737 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-2:19.0.0-3101.g9b42bfbe.el9.x86_64 1/198 2024-04-16T16:52:16.737 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-base-2:19.0.0-3101.g9b42bfbe.el9.x86_64 2/198 2024-04-16T16:52:16.737 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-common-2:19.0.0-3101.g9b42bfbe.el9.x86_64 3/198 2024-04-16T16:52:16.737 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-fuse-2:19.0.0-3101.g9b42bfbe.el9.x86_64 4/198 2024-04-16T16:52:16.738 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3101.g9b42b 5/198 2024-04-16T16:52:16.738 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-mds-2:19.0.0-3101.g9b42bfbe.el9.x86_64 6/198 2024-04-16T16:52:16.738 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-mgr-2:19.0.0-3101.g9b42bfbe.el9.x86_64 7/198 2024-04-16T16:52:16.738 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-mon-2:19.0.0-3101.g9b42bfbe.el9.x86_64 8/198 2024-04-16T16:52:16.738 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-osd-2:19.0.0-3101.g9b42bfbe.el9.x86_64 9/198 2024-04-16T16:52:16.738 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-radosgw-2:19.0.0-3101.g9b42bfbe.el9.x86_64 10/198 2024-04-16T16:52:16.738 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-selinux-2:19.0.0-3101.g9b42bfbe.el9.x86_64 11/198 2024-04-16T16:52:16.738 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-test-2:19.0.0-3101.g9b42bfbe.el9.x86_64 12/198 2024-04-16T16:52:16.738 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : cephfs-mirror-2:19.0.0-3101.g9b42bfbe.el9.x86_64 13/198 2024-04-16T16:52:16.738 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : cephfs-shell-2:19.0.0-3101.g9b42bfbe.el9.x86_64 14/198 2024-04-16T16:52:16.738 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libcephfs-devel-2:19.0.0-3101.g9b42bfbe.el9.x86_ 15/198 2024-04-16T16:52:16.738 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libcephfs2-2:19.0.0-3101.g9b42bfbe.el9.x86_64 16/198 2024-04-16T16:52:16.738 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libcephsqlite-2:19.0.0-3101.g9b42bfbe.el9.x86_64 17/198 2024-04-16T16:52:16.738 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : librados-devel-2:19.0.0-3101.g9b42bfbe.el9.x86_6 18/198 2024-04-16T16:52:16.739 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libradosstriper1-2:19.0.0-3101.g9b42bfbe.el9.x86 19/198 2024-04-16T16:52:16.739 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : librgw2-2:19.0.0-3101.g9b42bfbe.el9.x86_64 20/198 2024-04-16T16:52:16.739 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3101.g9b42bfbe.el 21/198 2024-04-16T16:52:16.739 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-ceph-common-2:19.0.0-3101.g9b42bfbe.el9. 22/198 2024-04-16T16:52:16.739 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-cephfs-2:19.0.0-3101.g9b42bfbe.el9.x86_6 23/198 2024-04-16T16:52:16.739 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-rados-2:19.0.0-3101.g9b42bfbe.el9.x86_64 24/198 2024-04-16T16:52:16.739 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-rbd-2:19.0.0-3101.g9b42bfbe.el9.x86_64 25/198 2024-04-16T16:52:16.739 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-rgw-2:19.0.0-3101.g9b42bfbe.el9.x86_64 26/198 2024-04-16T16:52:16.739 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : rbd-fuse-2:19.0.0-3101.g9b42bfbe.el9.x86_64 27/198 2024-04-16T16:52:16.754 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : rbd-mirror-2:19.0.0-3101.g9b42bfbe.el9.x86_64 28/198 2024-04-16T16:52:16.755 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : rbd-nbd-2:19.0.0-3101.g9b42bfbe.el9.x86_64 29/198 2024-04-16T16:52:16.755 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3101.g9b42bfbe. 30/198 2024-04-16T16:52:16.755 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3101.g9b42bfbe.el9.noa 31/198 2024-04-16T16:52:16.755 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3101.g9b42bfbe.el9.n 32/198 2024-04-16T16:52:16.755 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3101.g9b4 33/198 2024-04-16T16:52:16.755 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3101.g9b42bfbe.el 34/198 2024-04-16T16:52:16.755 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3101.g9b42bfbe.el9.noarch 35/198 2024-04-16T16:52:16.755 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3101.g9b42bfbe.e 36/198 2024-04-16T16:52:16.755 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-volume-2:19.0.0-3101.g9b42bfbe.el9.noarch 37/198 2024-04-16T16:52:16.755 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : cephadm-2:19.0.0-3101.g9b42bfbe.el9.noarch 38/198 2024-04-16T16:52:16.755 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : cephfs-top-2:19.0.0-3101.g9b42bfbe.el9.noarch 39/198 2024-04-16T16:52:16.755 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : automake-1.16.2-8.el9.noarch 40/198 2024-04-16T16:52:16.755 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : bison-3.7.4-5.el9.x86_64 41/198 2024-04-16T16:52:16.756 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 42/198 2024-04-16T16:52:16.756 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : flex-2.6.4-9.el9.x86_64 43/198 2024-04-16T16:52:16.756 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 44/198 2024-04-16T16:52:16.756 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 45/198 2024-04-16T16:52:16.756 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 46/198 2024-04-16T16:52:16.756 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 47/198 2024-04-16T16:52:16.756 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 48/198 2024-04-16T16:52:16.756 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 49/198 2024-04-16T16:52:16.756 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 50/198 2024-04-16T16:52:16.756 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 51/198 2024-04-16T16:52:16.756 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 52/198 2024-04-16T16:52:16.756 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 53/198 2024-04-16T16:52:16.756 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : lua-5.4.4-4.el9.x86_64 54/198 2024-04-16T16:52:16.756 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 55/198 2024-04-16T16:52:16.756 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 56/198 2024-04-16T16:52:16.757 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 57/198 2024-04-16T16:52:16.757 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : postgresql-13.14-1.el9.x86_64 58/198 2024-04-16T16:52:16.757 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 59/198 2024-04-16T16:52:16.757 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 60/198 2024-04-16T16:52:16.757 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 61/198 2024-04-16T16:52:16.757 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 62/198 2024-04-16T16:52:16.757 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 63/198 2024-04-16T16:52:16.757 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 64/198 2024-04-16T16:52:16.757 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 65/198 2024-04-16T16:52:16.757 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 66/198 2024-04-16T16:52:16.757 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 67/198 2024-04-16T16:52:16.757 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 68/198 2024-04-16T16:52:16.757 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 69/198 2024-04-16T16:52:16.757 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 70/198 2024-04-16T16:52:16.758 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 71/198 2024-04-16T16:52:16.758 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 72/198 2024-04-16T16:52:16.758 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 73/198 2024-04-16T16:52:16.758 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 74/198 2024-04-16T16:52:16.758 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 75/198 2024-04-16T16:52:16.758 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 76/198 2024-04-16T16:52:16.758 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 77/198 2024-04-16T16:52:16.758 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 78/198 2024-04-16T16:52:16.758 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 79/198 2024-04-16T16:52:16.758 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 80/198 2024-04-16T16:52:16.758 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 81/198 2024-04-16T16:52:16.758 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 82/198 2024-04-16T16:52:16.758 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 83/198 2024-04-16T16:52:16.759 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 84/198 2024-04-16T16:52:16.759 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 85/198 2024-04-16T16:52:16.759 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 86/198 2024-04-16T16:52:16.759 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 87/198 2024-04-16T16:52:16.759 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 88/198 2024-04-16T16:52:16.759 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 89/198 2024-04-16T16:52:16.759 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 90/198 2024-04-16T16:52:16.759 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 91/198 2024-04-16T16:52:16.759 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-ply-3.11-14.el9.noarch 92/198 2024-04-16T16:52:16.759 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 93/198 2024-04-16T16:52:16.759 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 94/198 2024-04-16T16:52:16.759 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : jq-1.6-16.el9.x86_64 95/198 2024-04-16T16:52:16.759 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 96/198 2024-04-16T16:52:16.759 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 97/198 2024-04-16T16:52:16.759 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 98/198 2024-04-16T16:52:16.759 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 99/198 2024-04-16T16:52:16.760 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 100/198 2024-04-16T16:52:16.760 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 101/198 2024-04-16T16:52:16.760 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 102/198 2024-04-16T16:52:16.760 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 103/198 2024-04-16T16:52:16.760 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 104/198 2024-04-16T16:52:16.760 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/198 2024-04-16T16:52:16.760 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/198 2024-04-16T16:52:16.760 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/198 2024-04-16T16:52:16.760 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/198 2024-04-16T16:52:16.760 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/198 2024-04-16T16:52:16.760 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 110/198 2024-04-16T16:52:16.760 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 111/198 2024-04-16T16:52:16.760 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/198 2024-04-16T16:52:16.760 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/198 2024-04-16T16:52:16.760 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 114/198 2024-04-16T16:52:16.761 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 115/198 2024-04-16T16:52:16.761 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 116/198 2024-04-16T16:52:16.761 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/198 2024-04-16T16:52:16.761 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/198 2024-04-16T16:52:16.761 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 119/198 2024-04-16T16:52:16.761 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/198 2024-04-16T16:52:16.761 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/198 2024-04-16T16:52:16.761 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/198 2024-04-16T16:52:16.761 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 123/198 2024-04-16T16:52:16.761 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-google-auth-1:2.28.2-1.el9.noarch 124/198 2024-04-16T16:52:16.761 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 125/198 2024-04-16T16:52:16.761 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 126/198 2024-04-16T16:52:16.761 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 127/198 2024-04-16T16:52:16.761 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 128/198 2024-04-16T16:52:16.761 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 129/198 2024-04-16T16:52:16.762 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 130/198 2024-04-16T16:52:16.762 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 131/198 2024-04-16T16:52:16.762 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 132/198 2024-04-16T16:52:16.762 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 133/198 2024-04-16T16:52:16.762 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 134/198 2024-04-16T16:52:16.762 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 135/198 2024-04-16T16:52:16.762 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 136/198 2024-04-16T16:52:16.762 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 137/198 2024-04-16T16:52:16.762 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 138/198 2024-04-16T16:52:16.762 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 139/198 2024-04-16T16:52:16.762 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 140/198 2024-04-16T16:52:16.763 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 141/198 2024-04-16T16:52:16.763 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 142/198 2024-04-16T16:52:16.763 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 143/198 2024-04-16T16:52:16.763 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 144/198 2024-04-16T16:52:16.763 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 145/198 2024-04-16T16:52:16.763 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 146/198 2024-04-16T16:52:16.763 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 147/198 2024-04-16T16:52:16.763 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 148/198 2024-04-16T16:52:16.763 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : re2-1:20211101-3.el9.x86_64 149/198 2024-04-16T16:52:16.763 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 150/198 2024-04-16T16:52:16.763 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : librados2-2:19.0.0-3101.g9b42bfbe.el9.x86_64 151/198 2024-04-16T16:52:16.763 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 152/198 2024-04-16T16:52:16.763 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : librbd1-2:19.0.0-3101.g9b42bfbe.el9.x86_64 153/198 2024-04-16T16:52:16.763 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 154/198 2024-04-16T16:52:16.763 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 155/198 2024-04-16T16:52:16.763 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 156/198 2024-04-16T16:52:16.764 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 157/198 2024-04-16T16:52:16.764 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 158/198 2024-04-16T16:52:16.764 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 159/198 2024-04-16T16:52:16.764 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 160/198 2024-04-16T16:52:16.764 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 161/198 2024-04-16T16:52:16.764 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 162/198 2024-04-16T16:52:16.764 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 163/198 2024-04-16T16:52:16.764 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 164/198 2024-04-16T16:52:16.764 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : acl-2.3.1-4.el9.x86_64 165/198 2024-04-16T16:52:16.764 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : acl-2.3.1-3.el9.x86_64 166/198 2024-04-16T16:52:16.764 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 167/198 2024-04-16T16:52:16.764 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 168/198 2024-04-16T16:52:16.764 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 169/198 2024-04-16T16:52:16.764 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 170/198 2024-04-16T16:52:16.764 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 171/198 2024-04-16T16:52:16.765 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 172/198 2024-04-16T16:52:16.765 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 173/198 2024-04-16T16:52:16.765 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 174/198 2024-04-16T16:52:16.765 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libblkid-2.37.4-18.el9.x86_64 175/198 2024-04-16T16:52:16.765 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libblkid-2.37.4-15.el9.x86_64 176/198 2024-04-16T16:52:16.765 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libfdisk-2.37.4-18.el9.x86_64 177/198 2024-04-16T16:52:16.765 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libfdisk-2.37.4-15.el9.x86_64 178/198 2024-04-16T16:52:16.765 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libmount-2.37.4-18.el9.x86_64 179/198 2024-04-16T16:52:16.765 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libmount-2.37.4-15.el9.x86_64 180/198 2024-04-16T16:52:16.765 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libsmartcols-2.37.4-18.el9.x86_64 181/198 2024-04-16T16:52:16.765 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libsmartcols-2.37.4-15.el9.x86_64 182/198 2024-04-16T16:52:16.765 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libuuid-2.37.4-18.el9.x86_64 183/198 2024-04-16T16:52:16.765 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libuuid-2.37.4-15.el9.x86_64 184/198 2024-04-16T16:52:16.765 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 185/198 2024-04-16T16:52:16.766 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 186/198 2024-04-16T16:52:16.766 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 187/198 2024-04-16T16:52:16.766 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 188/198 2024-04-16T16:52:16.766 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : util-linux-2.37.4-18.el9.x86_64 189/198 2024-04-16T16:52:16.766 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : util-linux-2.37.4-15.el9.x86_64 190/198 2024-04-16T16:52:16.766 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : util-linux-core-2.37.4-18.el9.x86_64 191/198 2024-04-16T16:52:16.766 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : util-linux-core-2.37.4-15.el9.x86_64 192/198 2024-04-16T16:52:16.766 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : util-linux-user-2.37.4-18.el9.x86_64 193/198 2024-04-16T16:52:16.766 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : util-linux-user-2.37.4-15.el9.x86_64 194/198 2024-04-16T16:52:16.766 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 195/198 2024-04-16T16:52:16.766 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 196/198 2024-04-16T16:52:16.766 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 197/198 2024-04-16T16:52:19.631 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 198/198 2024-04-16T16:52:19.631 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-16T16:52:19.631 INFO:teuthology.orchestra.run.smithi045.stdout:Upgraded: 2024-04-16T16:52:19.632 INFO:teuthology.orchestra.run.smithi045.stdout: NetworkManager-1:1.47.4-1.el9.x86_64 2024-04-16T16:52:19.632 INFO:teuthology.orchestra.run.smithi045.stdout: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 2024-04-16T16:52:19.632 INFO:teuthology.orchestra.run.smithi045.stdout: NetworkManager-team-1:1.47.4-1.el9.x86_64 2024-04-16T16:52:19.632 INFO:teuthology.orchestra.run.smithi045.stdout: NetworkManager-tui-1:1.47.4-1.el9.x86_64 2024-04-16T16:52:19.632 INFO:teuthology.orchestra.run.smithi045.stdout: acl-2.3.1-4.el9.x86_64 2024-04-16T16:52:19.632 INFO:teuthology.orchestra.run.smithi045.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-04-16T16:52:19.632 INFO:teuthology.orchestra.run.smithi045.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-04-16T16:52:19.632 INFO:teuthology.orchestra.run.smithi045.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-04-16T16:52:19.632 INFO:teuthology.orchestra.run.smithi045.stdout: libacl-2.3.1-4.el9.x86_64 2024-04-16T16:52:19.632 INFO:teuthology.orchestra.run.smithi045.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-04-16T16:52:19.632 INFO:teuthology.orchestra.run.smithi045.stdout: libblkid-2.37.4-18.el9.x86_64 2024-04-16T16:52:19.632 INFO:teuthology.orchestra.run.smithi045.stdout: libfdisk-2.37.4-18.el9.x86_64 2024-04-16T16:52:19.632 INFO:teuthology.orchestra.run.smithi045.stdout: libmount-2.37.4-18.el9.x86_64 2024-04-16T16:52:19.632 INFO:teuthology.orchestra.run.smithi045.stdout: librados2-2:19.0.0-3101.g9b42bfbe.el9.x86_64 2024-04-16T16:52:19.633 INFO:teuthology.orchestra.run.smithi045.stdout: librbd1-2:19.0.0-3101.g9b42bfbe.el9.x86_64 2024-04-16T16:52:19.633 INFO:teuthology.orchestra.run.smithi045.stdout: libsmartcols-2.37.4-18.el9.x86_64 2024-04-16T16:52:19.633 INFO:teuthology.orchestra.run.smithi045.stdout: libuuid-2.37.4-18.el9.x86_64 2024-04-16T16:52:19.633 INFO:teuthology.orchestra.run.smithi045.stdout: openssl-1:3.0.7-27.el9.x86_64 2024-04-16T16:52:19.633 INFO:teuthology.orchestra.run.smithi045.stdout: openssl-libs-1:3.0.7-27.el9.x86_64 2024-04-16T16:52:19.633 INFO:teuthology.orchestra.run.smithi045.stdout: util-linux-2.37.4-18.el9.x86_64 2024-04-16T16:52:19.633 INFO:teuthology.orchestra.run.smithi045.stdout: util-linux-core-2.37.4-18.el9.x86_64 2024-04-16T16:52:19.633 INFO:teuthology.orchestra.run.smithi045.stdout: util-linux-user-2.37.4-18.el9.x86_64 2024-04-16T16:52:19.633 INFO:teuthology.orchestra.run.smithi045.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-04-16T16:52:19.633 INFO:teuthology.orchestra.run.smithi045.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-04-16T16:52:19.633 INFO:teuthology.orchestra.run.smithi045.stdout:Installed: 2024-04-16T16:52:19.633 INFO:teuthology.orchestra.run.smithi045.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-16T16:52:19.633 INFO:teuthology.orchestra.run.smithi045.stdout: automake-1.16.2-8.el9.noarch 2024-04-16T16:52:19.633 INFO:teuthology.orchestra.run.smithi045.stdout: bison-3.7.4-5.el9.x86_64 2024-04-16T16:52:19.633 INFO:teuthology.orchestra.run.smithi045.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-16T16:52:19.634 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-2:19.0.0-3101.g9b42bfbe.el9.x86_64 2024-04-16T16:52:19.634 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-base-2:19.0.0-3101.g9b42bfbe.el9.x86_64 2024-04-16T16:52:19.634 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-common-2:19.0.0-3101.g9b42bfbe.el9.x86_64 2024-04-16T16:52:19.634 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-fuse-2:19.0.0-3101.g9b42bfbe.el9.x86_64 2024-04-16T16:52:19.634 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-grafana-dashboards-2:19.0.0-3101.g9b42bfbe.el9.noarch 2024-04-16T16:52:19.634 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-immutable-object-cache-2:19.0.0-3101.g9b42bfbe.el9.x86_64 2024-04-16T16:52:19.634 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mds-2:19.0.0-3101.g9b42bfbe.el9.x86_64 2024-04-16T16:52:19.634 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr-2:19.0.0-3101.g9b42bfbe.el9.x86_64 2024-04-16T16:52:19.634 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr-cephadm-2:19.0.0-3101.g9b42bfbe.el9.noarch 2024-04-16T16:52:19.634 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr-dashboard-2:19.0.0-3101.g9b42bfbe.el9.noarch 2024-04-16T16:52:19.634 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3101.g9b42bfbe.el9.noarch 2024-04-16T16:52:19.634 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr-modules-core-2:19.0.0-3101.g9b42bfbe.el9.noarch 2024-04-16T16:52:19.634 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr-rook-2:19.0.0-3101.g9b42bfbe.el9.noarch 2024-04-16T16:52:19.634 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mon-2:19.0.0-3101.g9b42bfbe.el9.x86_64 2024-04-16T16:52:19.634 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-osd-2:19.0.0-3101.g9b42bfbe.el9.x86_64 2024-04-16T16:52:19.635 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-prometheus-alerts-2:19.0.0-3101.g9b42bfbe.el9.noarch 2024-04-16T16:52:19.635 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-radosgw-2:19.0.0-3101.g9b42bfbe.el9.x86_64 2024-04-16T16:52:19.635 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-selinux-2:19.0.0-3101.g9b42bfbe.el9.x86_64 2024-04-16T16:52:19.635 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-test-2:19.0.0-3101.g9b42bfbe.el9.x86_64 2024-04-16T16:52:19.635 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-volume-2:19.0.0-3101.g9b42bfbe.el9.noarch 2024-04-16T16:52:19.635 INFO:teuthology.orchestra.run.smithi045.stdout: cephadm-2:19.0.0-3101.g9b42bfbe.el9.noarch 2024-04-16T16:52:19.635 INFO:teuthology.orchestra.run.smithi045.stdout: cephfs-mirror-2:19.0.0-3101.g9b42bfbe.el9.x86_64 2024-04-16T16:52:19.635 INFO:teuthology.orchestra.run.smithi045.stdout: cephfs-shell-2:19.0.0-3101.g9b42bfbe.el9.x86_64 2024-04-16T16:52:19.635 INFO:teuthology.orchestra.run.smithi045.stdout: cephfs-top-2:19.0.0-3101.g9b42bfbe.el9.noarch 2024-04-16T16:52:19.635 INFO:teuthology.orchestra.run.smithi045.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-16T16:52:19.635 INFO:teuthology.orchestra.run.smithi045.stdout: flex-2.6.4-9.el9.x86_64 2024-04-16T16:52:19.635 INFO:teuthology.orchestra.run.smithi045.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-16T16:52:19.635 INFO:teuthology.orchestra.run.smithi045.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-16T16:52:19.635 INFO:teuthology.orchestra.run.smithi045.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-16T16:52:19.636 INFO:teuthology.orchestra.run.smithi045.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-16T16:52:19.636 INFO:teuthology.orchestra.run.smithi045.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-16T16:52:19.636 INFO:teuthology.orchestra.run.smithi045.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-16T16:52:19.636 INFO:teuthology.orchestra.run.smithi045.stdout: jq-1.6-16.el9.x86_64 2024-04-16T16:52:19.636 INFO:teuthology.orchestra.run.smithi045.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-16T16:52:19.636 INFO:teuthology.orchestra.run.smithi045.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-16T16:52:19.636 INFO:teuthology.orchestra.run.smithi045.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-16T16:52:19.636 INFO:teuthology.orchestra.run.smithi045.stdout: libcephfs-devel-2:19.0.0-3101.g9b42bfbe.el9.x86_64 2024-04-16T16:52:19.636 INFO:teuthology.orchestra.run.smithi045.stdout: libcephfs2-2:19.0.0-3101.g9b42bfbe.el9.x86_64 2024-04-16T16:52:19.636 INFO:teuthology.orchestra.run.smithi045.stdout: libcephsqlite-2:19.0.0-3101.g9b42bfbe.el9.x86_64 2024-04-16T16:52:19.636 INFO:teuthology.orchestra.run.smithi045.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-16T16:52:19.636 INFO:teuthology.orchestra.run.smithi045.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-16T16:52:19.636 INFO:teuthology.orchestra.run.smithi045.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-16T16:52:19.636 INFO:teuthology.orchestra.run.smithi045.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-16T16:52:19.636 INFO:teuthology.orchestra.run.smithi045.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-16T16:52:19.637 INFO:teuthology.orchestra.run.smithi045.stdout: librados-devel-2:19.0.0-3101.g9b42bfbe.el9.x86_64 2024-04-16T16:52:19.637 INFO:teuthology.orchestra.run.smithi045.stdout: libradosstriper1-2:19.0.0-3101.g9b42bfbe.el9.x86_64 2024-04-16T16:52:19.637 INFO:teuthology.orchestra.run.smithi045.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-16T16:52:19.637 INFO:teuthology.orchestra.run.smithi045.stdout: librgw2-2:19.0.0-3101.g9b42bfbe.el9.x86_64 2024-04-16T16:52:19.637 INFO:teuthology.orchestra.run.smithi045.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-16T16:52:19.637 INFO:teuthology.orchestra.run.smithi045.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-16T16:52:19.637 INFO:teuthology.orchestra.run.smithi045.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-16T16:52:19.637 INFO:teuthology.orchestra.run.smithi045.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-16T16:52:19.637 INFO:teuthology.orchestra.run.smithi045.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-16T16:52:19.637 INFO:teuthology.orchestra.run.smithi045.stdout: lua-5.4.4-4.el9.x86_64 2024-04-16T16:52:19.637 INFO:teuthology.orchestra.run.smithi045.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-16T16:52:19.637 INFO:teuthology.orchestra.run.smithi045.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-16T16:52:19.637 INFO:teuthology.orchestra.run.smithi045.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-04-16T16:52:19.637 INFO:teuthology.orchestra.run.smithi045.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-16T16:52:19.638 INFO:teuthology.orchestra.run.smithi045.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-16T16:52:19.638 INFO:teuthology.orchestra.run.smithi045.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-04-16T16:52:19.638 INFO:teuthology.orchestra.run.smithi045.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-16T16:52:19.638 INFO:teuthology.orchestra.run.smithi045.stdout: postgresql-13.14-1.el9.x86_64 2024-04-16T16:52:19.638 INFO:teuthology.orchestra.run.smithi045.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-16T16:52:19.638 INFO:teuthology.orchestra.run.smithi045.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-16T16:52:19.638 INFO:teuthology.orchestra.run.smithi045.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-16T16:52:19.638 INFO:teuthology.orchestra.run.smithi045.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-16T16:52:19.638 INFO:teuthology.orchestra.run.smithi045.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-16T16:52:19.638 INFO:teuthology.orchestra.run.smithi045.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-16T16:52:19.638 INFO:teuthology.orchestra.run.smithi045.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-16T16:52:19.638 INFO:teuthology.orchestra.run.smithi045.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-16T16:52:19.638 INFO:teuthology.orchestra.run.smithi045.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-16T16:52:19.638 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-16T16:52:19.638 INFO:teuthology.orchestra.run.smithi045.stdout: python3-ceph-argparse-2:19.0.0-3101.g9b42bfbe.el9.x86_64 2024-04-16T16:52:19.639 INFO:teuthology.orchestra.run.smithi045.stdout: python3-ceph-common-2:19.0.0-3101.g9b42bfbe.el9.x86_64 2024-04-16T16:52:19.639 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cephfs-2:19.0.0-3101.g9b42bfbe.el9.x86_64 2024-04-16T16:52:19.639 INFO:teuthology.orchestra.run.smithi045.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-16T16:52:19.639 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-16T16:52:19.639 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-16T16:52:19.639 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-16T16:52:19.639 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-16T16:52:19.639 INFO:teuthology.orchestra.run.smithi045.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-16T16:52:19.639 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-16T16:52:19.639 INFO:teuthology.orchestra.run.smithi045.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-04-16T16:52:19.639 INFO:teuthology.orchestra.run.smithi045.stdout: python3-google-auth-1:2.28.2-1.el9.noarch 2024-04-16T16:52:19.639 INFO:teuthology.orchestra.run.smithi045.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-16T16:52:19.639 INFO:teuthology.orchestra.run.smithi045.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-16T16:52:19.639 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-16T16:52:19.639 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-16T16:52:19.640 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-16T16:52:19.640 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-16T16:52:19.640 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-16T16:52:19.640 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-16T16:52:19.640 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-16T16:52:19.640 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-16T16:52:19.640 INFO:teuthology.orchestra.run.smithi045.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-16T16:52:19.640 INFO:teuthology.orchestra.run.smithi045.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-16T16:52:19.640 INFO:teuthology.orchestra.run.smithi045.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-16T16:52:19.640 INFO:teuthology.orchestra.run.smithi045.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-16T16:52:19.640 INFO:teuthology.orchestra.run.smithi045.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-16T16:52:19.640 INFO:teuthology.orchestra.run.smithi045.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-16T16:52:19.640 INFO:teuthology.orchestra.run.smithi045.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-16T16:52:19.640 INFO:teuthology.orchestra.run.smithi045.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-16T16:52:19.641 INFO:teuthology.orchestra.run.smithi045.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-16T16:52:19.641 INFO:teuthology.orchestra.run.smithi045.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-16T16:52:19.641 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-16T16:52:19.641 INFO:teuthology.orchestra.run.smithi045.stdout: python3-ply-3.11-14.el9.noarch 2024-04-16T16:52:19.641 INFO:teuthology.orchestra.run.smithi045.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-16T16:52:19.641 INFO:teuthology.orchestra.run.smithi045.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-16T16:52:19.641 INFO:teuthology.orchestra.run.smithi045.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-16T16:52:19.641 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-16T16:52:19.641 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-16T16:52:19.641 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-16T16:52:19.641 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-16T16:52:19.641 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-16T16:52:19.641 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-16T16:52:19.641 INFO:teuthology.orchestra.run.smithi045.stdout: python3-rados-2:19.0.0-3101.g9b42bfbe.el9.x86_64 2024-04-16T16:52:19.641 INFO:teuthology.orchestra.run.smithi045.stdout: python3-rbd-2:19.0.0-3101.g9b42bfbe.el9.x86_64 2024-04-16T16:52:19.642 INFO:teuthology.orchestra.run.smithi045.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-16T16:52:19.642 INFO:teuthology.orchestra.run.smithi045.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-16T16:52:19.642 INFO:teuthology.orchestra.run.smithi045.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-16T16:52:19.642 INFO:teuthology.orchestra.run.smithi045.stdout: python3-rgw-2:19.0.0-3101.g9b42bfbe.el9.x86_64 2024-04-16T16:52:19.642 INFO:teuthology.orchestra.run.smithi045.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-16T16:52:19.642 INFO:teuthology.orchestra.run.smithi045.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-16T16:52:19.642 INFO:teuthology.orchestra.run.smithi045.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-16T16:52:19.642 INFO:teuthology.orchestra.run.smithi045.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-16T16:52:19.642 INFO:teuthology.orchestra.run.smithi045.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-16T16:52:19.642 INFO:teuthology.orchestra.run.smithi045.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-16T16:52:19.642 INFO:teuthology.orchestra.run.smithi045.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-16T16:52:19.642 INFO:teuthology.orchestra.run.smithi045.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-16T16:52:19.642 INFO:teuthology.orchestra.run.smithi045.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-16T16:52:19.643 INFO:teuthology.orchestra.run.smithi045.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-16T16:52:19.643 INFO:teuthology.orchestra.run.smithi045.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-16T16:52:19.643 INFO:teuthology.orchestra.run.smithi045.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-16T16:52:19.643 INFO:teuthology.orchestra.run.smithi045.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-16T16:52:19.643 INFO:teuthology.orchestra.run.smithi045.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-16T16:52:19.643 INFO:teuthology.orchestra.run.smithi045.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-16T16:52:19.643 INFO:teuthology.orchestra.run.smithi045.stdout: rbd-fuse-2:19.0.0-3101.g9b42bfbe.el9.x86_64 2024-04-16T16:52:19.643 INFO:teuthology.orchestra.run.smithi045.stdout: rbd-mirror-2:19.0.0-3101.g9b42bfbe.el9.x86_64 2024-04-16T16:52:19.643 INFO:teuthology.orchestra.run.smithi045.stdout: rbd-nbd-2:19.0.0-3101.g9b42bfbe.el9.x86_64 2024-04-16T16:52:19.643 INFO:teuthology.orchestra.run.smithi045.stdout: re2-1:20211101-3.el9.x86_64 2024-04-16T16:52:19.643 INFO:teuthology.orchestra.run.smithi045.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-16T16:52:19.643 INFO:teuthology.orchestra.run.smithi045.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-16T16:52:19.643 INFO:teuthology.orchestra.run.smithi045.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-16T16:52:19.643 INFO:teuthology.orchestra.run.smithi045.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-16T16:52:19.643 INFO:teuthology.orchestra.run.smithi045.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-16T16:52:19.644 INFO:teuthology.orchestra.run.smithi045.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-04-16T16:52:19.644 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-16T16:52:19.644 INFO:teuthology.orchestra.run.smithi045.stdout:Complete! 2024-04-16T16:52:20.311 DEBUG:teuthology.parallel:result is None 2024-04-16T16:52:20.312 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=9b42bfbeda298874861ece590aa565cf64ac8c45 2024-04-16T16:52:20.455 DEBUG:teuthology.orchestra.run.smithi045:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-16T16:52:20.509 INFO:teuthology.orchestra.run.smithi045.stdout:19.0.0-3101.g9b42bfbe.el9 2024-04-16T16:52:20.510 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3101.g9b42bfbe.el9 2024-04-16T16:52:20.510 INFO:teuthology.task.install:The correct ceph version 19.0.0-3101.g9b42bfbe is installed. 2024-04-16T16:52:20.512 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=9b42bfbeda298874861ece590aa565cf64ac8c45 2024-04-16T16:52:20.876 DEBUG:teuthology.orchestra.run.smithi104:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-16T16:52:20.902 INFO:teuthology.orchestra.run.smithi104.stdout:19.0.0-3101.g9b42bfbe.el9 2024-04-16T16:52:20.902 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3101.g9b42bfbe.el9 2024-04-16T16:52:20.902 INFO:teuthology.task.install:The correct ceph version 19.0.0-3101.g9b42bfbe is installed. 2024-04-16T16:52:20.904 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=9b42bfbeda298874861ece590aa565cf64ac8c45 2024-04-16T16:52:21.192 DEBUG:teuthology.orchestra.run.smithi117:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-16T16:52:21.217 INFO:teuthology.orchestra.run.smithi117.stdout:19.0.0-3101.g9b42bfbe.el9 2024-04-16T16:52:21.217 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3101.g9b42bfbe.el9 2024-04-16T16:52:21.217 INFO:teuthology.task.install:The correct ceph version 19.0.0-3101.g9b42bfbe is installed. 2024-04-16T16:52:21.219 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-04-16T16:52:21.219 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-04-16T16:52:21.219 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-16T16:52:21.295 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2024-04-16T16:52:21.295 DEBUG:teuthology.orchestra.run.smithi104:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-16T16:52:21.330 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-04-16T16:52:21.330 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-16T16:52:21.364 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-04-16T16:52:21.364 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-04-16T16:52:21.364 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd of=/usr/bin/daemon-helper 2024-04-16T16:52:21.437 DEBUG:teuthology.orchestra.run.smithi045:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-16T16:52:21.548 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2024-04-16T16:52:21.548 DEBUG:teuthology.orchestra.run.smithi104:> sudo dd of=/usr/bin/daemon-helper 2024-04-16T16:52:21.581 DEBUG:teuthology.orchestra.run.smithi104:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-16T16:52:21.650 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-04-16T16:52:21.650 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd of=/usr/bin/daemon-helper 2024-04-16T16:52:21.679 DEBUG:teuthology.orchestra.run.smithi117:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-16T16:52:21.746 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-04-16T16:52:21.746 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-04-16T16:52:21.746 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-16T16:52:21.818 DEBUG:teuthology.orchestra.run.smithi045:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-16T16:52:21.931 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2024-04-16T16:52:21.931 DEBUG:teuthology.orchestra.run.smithi104:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-16T16:52:21.958 DEBUG:teuthology.orchestra.run.smithi104:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-16T16:52:22.025 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-04-16T16:52:22.025 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-16T16:52:22.052 DEBUG:teuthology.orchestra.run.smithi117:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-16T16:52:22.120 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-04-16T16:52:22.120 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-04-16T16:52:22.120 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd of=/usr/bin/stdin-killer 2024-04-16T16:52:22.193 DEBUG:teuthology.orchestra.run.smithi045:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-16T16:52:22.310 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2024-04-16T16:52:22.310 DEBUG:teuthology.orchestra.run.smithi104:> sudo dd of=/usr/bin/stdin-killer 2024-04-16T16:52:22.337 DEBUG:teuthology.orchestra.run.smithi104:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-16T16:52:22.404 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-04-16T16:52:22.405 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd of=/usr/bin/stdin-killer 2024-04-16T16:52:22.432 DEBUG:teuthology.orchestra.run.smithi117:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-16T16:52:22.499 INFO:teuthology.run_tasks:Running task cephadm... 2024-04-16T16:52:22.596 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 3, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug cephsqlite': 20, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10000}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'slow metadata IO', 'SLOW_OPS', 'slow request'], 'sha1': '9b42bfbeda298874861ece590aa565cf64ac8c45', 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated --size 25000000000'}} 2024-04-16T16:52:22.597 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45 2024-04-16T16:52:22.597 INFO:tasks.cephadm:Cluster fsid is b1fecd38-fc11-11ee-bc90-c7b262605968 2024-04-16T16:52:22.597 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-04-16T16:52:22.597 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.45', 'mon.b': '172.21.15.104', 'mon.c': '172.21.15.117'} 2024-04-16T16:52:22.597 INFO:tasks.cephadm:First mon is mon.a on smithi045 2024-04-16T16:52:22.597 INFO:tasks.cephadm:First mgr is x 2024-04-16T16:52:22.598 INFO:tasks.cephadm:Normalizing hostnames... 2024-04-16T16:52:22.598 DEBUG:teuthology.orchestra.run.smithi045:> sudo hostname $(hostname -s) 2024-04-16T16:52:22.670 DEBUG:teuthology.orchestra.run.smithi104:> sudo hostname $(hostname -s) 2024-04-16T16:52:22.696 DEBUG:teuthology.orchestra.run.smithi117:> sudo hostname $(hostname -s) 2024-04-16T16:52:22.723 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-04-16T16:52:22.723 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=9b42bfbeda298874861ece590aa565cf64ac8c45 2024-04-16T16:52:22.963 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '9b42bfbeda298874861ece590aa565cf64ac8c45', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/78612/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-3101-g9b42bfbe', 'node_name': '172.21.5.36+adami06', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-3101.g9b42bfbe'}, 'url': 'https://3.chacra.ceph.com/r/ceph/wip-mchangir-qa-debug-resource-temporarily-unavailable-issue/9b42bfbeda298874861ece590aa565cf64ac8c45/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-04-16 07:47:33.284700', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-mchangir-qa-debug-resource-temporarily-unavailable-issue', 'chacra_url': 'https://3.chacra.ceph.com/repos/ceph/wip-mchangir-qa-debug-resource-temporarily-unavailable-issue/9b42bfbeda298874861ece590aa565cf64ac8c45/centos/9/flavors/default/', 'archs': ['x86_64', 'source'], 'distro': 'centos'}] 2024-04-16T16:52:23.099 INFO:tasks.util.chacra:got chacra host 3.chacra.ceph.com, ref wip-mchangir-qa-debug-resource-temporarily-unavailable-issue, sha1 9b42bfbeda298874861ece590aa565cf64ac8c45 from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=9b42bfbeda298874861ece590aa565cf64ac8c45 2024-04-16T16:52:23.101 INFO:tasks.cephadm:Discovered cachra url: https://3.chacra.ceph.com/binaries/ceph/wip-mchangir-qa-debug-resource-temporarily-unavailable-issue/9b42bfbeda298874861ece590aa565cf64ac8c45/centos/9/x86_64/flavors/default/cephadm 2024-04-16T16:52:23.101 DEBUG:teuthology.orchestra.run.smithi045:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/wip-mchangir-qa-debug-resource-temporarily-unavailable-issue/9b42bfbeda298874861ece590aa565cf64ac8c45/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-16T16:52:23.987 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r--. 1 ubuntu ubuntu 781394 Apr 16 16:52 /home/ubuntu/cephtest/cephadm 2024-04-16T16:52:23.987 DEBUG:teuthology.orchestra.run.smithi104:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/wip-mchangir-qa-debug-resource-temporarily-unavailable-issue/9b42bfbeda298874861ece590aa565cf64ac8c45/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-16T16:52:24.516 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r--. 1 ubuntu ubuntu 781394 Apr 16 16:52 /home/ubuntu/cephtest/cephadm 2024-04-16T16:52:24.517 DEBUG:teuthology.orchestra.run.smithi117:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/wip-mchangir-qa-debug-resource-temporarily-unavailable-issue/9b42bfbeda298874861ece590aa565cf64ac8c45/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-16T16:52:25.444 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r--. 1 ubuntu ubuntu 781394 Apr 16 16:52 /home/ubuntu/cephtest/cephadm 2024-04-16T16:52:25.445 DEBUG:teuthology.orchestra.run.smithi045:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-16T16:52:25.508 DEBUG:teuthology.orchestra.run.smithi104:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-16T16:52:25.531 DEBUG:teuthology.orchestra.run.smithi117:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-16T16:52:25.565 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45 on all hosts... 2024-04-16T16:52:25.565 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45 pull 2024-04-16T16:52:25.572 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45 pull 2024-04-16T16:52:25.574 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45 pull 2024-04-16T16:52:26.380 INFO:teuthology.orchestra.run.smithi117.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45... 2024-04-16T16:52:26.482 INFO:teuthology.orchestra.run.smithi104.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45... 2024-04-16T16:52:26.859 INFO:teuthology.orchestra.run.smithi045.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45... 2024-04-16T16:53:34.522 INFO:teuthology.orchestra.run.smithi104.stdout:{ 2024-04-16T16:53:34.523 INFO:teuthology.orchestra.run.smithi104.stdout: "ceph_version": "ceph version 19.0.0-3101-g9b42bfbe (9b42bfbeda298874861ece590aa565cf64ac8c45) squid (dev)", 2024-04-16T16:53:34.523 INFO:teuthology.orchestra.run.smithi104.stdout: "image_id": "e5ce10bdffbeeffffd1125f67b369e2151f0f17cdec854e93b6adf9321f65907", 2024-04-16T16:53:34.523 INFO:teuthology.orchestra.run.smithi104.stdout: "repo_digests": [ 2024-04-16T16:53:34.523 INFO:teuthology.orchestra.run.smithi104.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2" 2024-04-16T16:53:34.523 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-04-16T16:53:34.523 INFO:teuthology.orchestra.run.smithi104.stdout:} 2024-04-16T16:53:36.996 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-04-16T16:53:36.996 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph_version": "ceph version 19.0.0-3101-g9b42bfbe (9b42bfbeda298874861ece590aa565cf64ac8c45) squid (dev)", 2024-04-16T16:53:36.996 INFO:teuthology.orchestra.run.smithi117.stdout: "image_id": "e5ce10bdffbeeffffd1125f67b369e2151f0f17cdec854e93b6adf9321f65907", 2024-04-16T16:53:36.996 INFO:teuthology.orchestra.run.smithi117.stdout: "repo_digests": [ 2024-04-16T16:53:36.996 INFO:teuthology.orchestra.run.smithi117.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2" 2024-04-16T16:53:36.996 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-04-16T16:53:36.996 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-04-16T16:53:49.738 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-16T16:53:49.739 INFO:teuthology.orchestra.run.smithi045.stdout: "ceph_version": "ceph version 19.0.0-3101-g9b42bfbe (9b42bfbeda298874861ece590aa565cf64ac8c45) squid (dev)", 2024-04-16T16:53:49.739 INFO:teuthology.orchestra.run.smithi045.stdout: "image_id": "e5ce10bdffbeeffffd1125f67b369e2151f0f17cdec854e93b6adf9321f65907", 2024-04-16T16:53:49.739 INFO:teuthology.orchestra.run.smithi045.stdout: "repo_digests": [ 2024-04-16T16:53:49.739 INFO:teuthology.orchestra.run.smithi045.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2" 2024-04-16T16:53:49.739 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-04-16T16:53:49.739 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-16T16:53:49.790 DEBUG:teuthology.orchestra.run.smithi045:> sudo mkdir -p /etc/ceph 2024-04-16T16:53:49.885 DEBUG:teuthology.orchestra.run.smithi104:> sudo mkdir -p /etc/ceph 2024-04-16T16:53:49.920 DEBUG:teuthology.orchestra.run.smithi117:> sudo mkdir -p /etc/ceph 2024-04-16T16:53:49.956 DEBUG:teuthology.orchestra.run.smithi045:> sudo chmod 777 /etc/ceph 2024-04-16T16:53:50.065 DEBUG:teuthology.orchestra.run.smithi104:> sudo chmod 777 /etc/ceph 2024-04-16T16:53:50.096 DEBUG:teuthology.orchestra.run.smithi117:> sudo chmod 777 /etc/ceph 2024-04-16T16:53:50.126 INFO:tasks.cephadm:Writing seed config... 2024-04-16T16:53:50.127 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-04-16T16:53:50.127 INFO:tasks.cephadm: override: [client] debug client = 20 2024-04-16T16:53:50.127 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-04-16T16:53:50.127 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-04-16T16:53:50.127 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-04-16T16:53:50.128 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-04-16T16:53:50.128 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-04-16T16:53:50.128 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-04-16T16:53:50.128 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-04-16T16:53:50.128 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-04-16T16:53:50.128 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-04-16T16:53:50.128 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-04-16T16:53:50.128 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-04-16T16:53:50.128 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-04-16T16:53:50.128 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-04-16T16:53:50.128 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-04-16T16:53:50.128 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-04-16T16:53:50.128 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-04-16T16:53:50.128 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-04-16T16:53:50.128 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-04-16T16:53:50.128 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-04-16T16:53:50.129 INFO:tasks.cephadm: override: [mgr] debug cephsqlite = 20 2024-04-16T16:53:50.129 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-04-16T16:53:50.129 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-04-16T16:53:50.129 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-04-16T16:53:50.129 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-04-16T16:53:50.129 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-04-16T16:53:50.129 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-04-16T16:53:50.129 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-04-16T16:53:50.129 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-04-16T16:53:50.129 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-04-16T16:53:50.129 INFO:tasks.cephadm: override: [osd] bluestore compression mode = aggressive 2024-04-16T16:53:50.129 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-04-16T16:53:50.129 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2024-04-16T16:53:50.129 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2024-04-16T16:53:50.129 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-04-16T16:53:50.129 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-04-16T16:53:50.130 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2024-04-16T16:53:50.130 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-04-16T16:53:50.130 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-04-16T16:53:50.130 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-04-16T16:53:50.130 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-04-16T16:53:50.130 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-04-16T16:53:50.130 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-04-16T16:53:50.130 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-04-16T16:53:50.130 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10000 2024-04-16T16:53:50.131 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-04-16T16:53:50.131 DEBUG:teuthology.orchestra.run.smithi045:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-04-16T16:53:50.192 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = b1fecd38-fc11-11ee-bc90-c7b262605968 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore compression mode = aggressive bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10000 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug cephsqlite = 20 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-04-16T16:53:50.192 DEBUG:teuthology.orchestra.run.smithi045:mon.a> sudo journalctl -f -n 0 -u ceph-b1fecd38-fc11-11ee-bc90-c7b262605968@mon.a.service 2024-04-16T16:53:50.239 DEBUG:teuthology.orchestra.run.smithi045:mgr.x> sudo journalctl -f -n 0 -u ceph-b1fecd38-fc11-11ee-bc90-c7b262605968@mgr.x.service 2024-04-16T16:53:50.285 INFO:tasks.cephadm:Bootstrapping... 2024-04-16T16:53:50.285 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45 -v bootstrap --fsid b1fecd38-fc11-11ee-bc90-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.45 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-04-16T16:53:50.569 INFO:teuthology.orchestra.run.smithi045.stdout:-------------------------------------------------------------------------------- 2024-04-16T16:53:50.569 INFO:teuthology.orchestra.run.smithi045.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45', '-v', 'bootstrap', '--fsid', 'b1fecd38-fc11-11ee-bc90-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.45', '--skip-admin-label'] 2024-04-16T16:53:50.569 INFO:teuthology.orchestra.run.smithi045.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-04-16T16:53:50.570 INFO:teuthology.orchestra.run.smithi045.stdout:Verifying podman|docker is present... 2024-04-16T16:53:50.631 INFO:teuthology.orchestra.run.smithi045.stdout:/bin/podman: stdout 4.9.4-dev 2024-04-16T16:53:50.631 INFO:teuthology.orchestra.run.smithi045.stdout:Verifying lvm2 is present... 2024-04-16T16:53:50.632 INFO:teuthology.orchestra.run.smithi045.stdout:Verifying time synchronization is in place... 2024-04-16T16:53:50.650 INFO:teuthology.orchestra.run.smithi045.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-04-16T16:53:50.650 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-04-16T16:53:50.668 INFO:teuthology.orchestra.run.smithi045.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-04-16T16:53:50.668 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stdout inactive 2024-04-16T16:53:50.691 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stdout enabled 2024-04-16T16:53:50.711 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stdout active 2024-04-16T16:53:50.712 INFO:teuthology.orchestra.run.smithi045.stdout:Unit chronyd.service is enabled and running 2024-04-16T16:53:50.712 INFO:teuthology.orchestra.run.smithi045.stdout:Repeating the final host check... 2024-04-16T16:53:50.775 INFO:teuthology.orchestra.run.smithi045.stdout:/bin/podman: stdout 4.9.4-dev 2024-04-16T16:53:50.775 INFO:teuthology.orchestra.run.smithi045.stdout:podman (/bin/podman) version 4.9.4 is present 2024-04-16T16:53:50.775 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl is present 2024-04-16T16:53:50.775 INFO:teuthology.orchestra.run.smithi045.stdout:lvcreate is present 2024-04-16T16:53:50.790 INFO:teuthology.orchestra.run.smithi045.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-04-16T16:53:50.790 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-04-16T16:53:50.807 INFO:teuthology.orchestra.run.smithi045.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-04-16T16:53:50.807 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stdout inactive 2024-04-16T16:53:50.830 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stdout enabled 2024-04-16T16:53:50.850 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stdout active 2024-04-16T16:53:50.850 INFO:teuthology.orchestra.run.smithi045.stdout:Unit chronyd.service is enabled and running 2024-04-16T16:53:50.850 INFO:teuthology.orchestra.run.smithi045.stdout:Host looks OK 2024-04-16T16:53:50.850 INFO:teuthology.orchestra.run.smithi045.stdout:Cluster fsid: b1fecd38-fc11-11ee-bc90-c7b262605968 2024-04-16T16:53:50.851 INFO:teuthology.orchestra.run.smithi045.stdout:Acquiring lock 139983481344928 on /run/cephadm/b1fecd38-fc11-11ee-bc90-c7b262605968.lock 2024-04-16T16:53:50.851 INFO:teuthology.orchestra.run.smithi045.stdout:Lock 139983481344928 acquired on /run/cephadm/b1fecd38-fc11-11ee-bc90-c7b262605968.lock 2024-04-16T16:53:50.851 INFO:teuthology.orchestra.run.smithi045.stdout:Verifying IP 172.21.15.45 port 3300 ... 2024-04-16T16:53:50.851 INFO:teuthology.orchestra.run.smithi045.stdout:Verifying IP 172.21.15.45 port 6789 ... 2024-04-16T16:53:50.852 INFO:teuthology.orchestra.run.smithi045.stdout:Base mon IP(s) is [172.21.15.45:3300, 172.21.15.45:6789], mon addrv is [v2:172.21.15.45:3300,v1:172.21.15.45:6789] 2024-04-16T16:53:50.862 INFO:teuthology.orchestra.run.smithi045.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp src 172.21.15.45 metric 100 2024-04-16T16:53:50.862 INFO:teuthology.orchestra.run.smithi045.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.45 metric 100 2024-04-16T16:53:50.870 INFO:teuthology.orchestra.run.smithi045.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-04-16T16:53:50.870 INFO:teuthology.orchestra.run.smithi045.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 1024 pref medium 2024-04-16T16:53:50.870 INFO:teuthology.orchestra.run.smithi045.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 100 pref medium 2024-04-16T16:53:50.879 INFO:teuthology.orchestra.run.smithi045.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-04-16T16:53:50.879 INFO:teuthology.orchestra.run.smithi045.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-04-16T16:53:50.879 INFO:teuthology.orchestra.run.smithi045.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-04-16T16:53:50.879 INFO:teuthology.orchestra.run.smithi045.stdout:/sbin/ip: stdout 4: ens1f0: mtu 1500 state UP qlen 1000 2024-04-16T16:53:50.879 INFO:teuthology.orchestra.run.smithi045.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:febd:12be/64 scope link noprefixroute 2024-04-16T16:53:50.879 INFO:teuthology.orchestra.run.smithi045.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-04-16T16:53:50.880 INFO:teuthology.orchestra.run.smithi045.stdout:Mon IP `172.21.15.45` is in CIDR network `172.21.0.0/20` 2024-04-16T16:53:50.880 INFO:teuthology.orchestra.run.smithi045.stdout:Mon IP `172.21.15.45` is in CIDR network `172.21.0.0/20` 2024-04-16T16:53:50.880 INFO:teuthology.orchestra.run.smithi045.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-04-16T16:53:50.882 INFO:teuthology.orchestra.run.smithi045.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-04-16T16:53:50.883 INFO:teuthology.orchestra.run.smithi045.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45... 2024-04-16T16:53:51.540 INFO:teuthology.orchestra.run.smithi045.stdout:/bin/podman: stdout e5ce10bdffbeeffffd1125f67b369e2151f0f17cdec854e93b6adf9321f65907 2024-04-16T16:53:51.540 INFO:teuthology.orchestra.run.smithi045.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45... 2024-04-16T16:53:51.540 INFO:teuthology.orchestra.run.smithi045.stdout:/bin/podman: stderr Getting image source signatures 2024-04-16T16:53:51.540 INFO:teuthology.orchestra.run.smithi045.stdout:/bin/podman: stderr Copying blob sha256:1b020bf69d7c5884a90622708ad98de4ba7dfcb01b3fd49d68b46566b30aa33a 2024-04-16T16:53:51.540 INFO:teuthology.orchestra.run.smithi045.stdout:/bin/podman: stderr Copying blob sha256:9793b5b5582798bde394f4383d142e9f248a25e4ebfd72ee05ea9ec6251a877d 2024-04-16T16:53:51.540 INFO:teuthology.orchestra.run.smithi045.stdout:/bin/podman: stderr Copying config sha256:e5ce10bdffbeeffffd1125f67b369e2151f0f17cdec854e93b6adf9321f65907 2024-04-16T16:53:51.540 INFO:teuthology.orchestra.run.smithi045.stdout:/bin/podman: stderr Writing manifest to image destination 2024-04-16T16:53:52.709 INFO:teuthology.orchestra.run.smithi045.stdout:ceph: stdout ceph version 19.0.0-3101-g9b42bfbe (9b42bfbeda298874861ece590aa565cf64ac8c45) squid (dev) 2024-04-16T16:53:52.709 INFO:teuthology.orchestra.run.smithi045.stdout:Ceph version: ceph version 19.0.0-3101-g9b42bfbe (9b42bfbeda298874861ece590aa565cf64ac8c45) squid (dev) 2024-04-16T16:53:52.710 INFO:teuthology.orchestra.run.smithi045.stdout:Extracting ceph user uid/gid from container image... 2024-04-16T16:53:53.998 INFO:teuthology.orchestra.run.smithi045.stdout:stat: stdout 167 167 2024-04-16T16:53:53.998 INFO:teuthology.orchestra.run.smithi045.stdout:Creating initial keys... 2024-04-16T16:53:55.187 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph-authtool: stdout AQAirR5mJvSbHRAAdIvZQApTCmzDZPK5hQ5oQw== 2024-04-16T16:53:56.384 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph-authtool: stdout AQAjrR5moqmhJhAAWgWEODyfAEtAn/lC8fs+uw== 2024-04-16T16:53:57.689 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph-authtool: stdout AQAkrR5mrlXYNBAAvV2QY87JJuAEzhdO4LxTVg== 2024-04-16T16:53:57.690 INFO:teuthology.orchestra.run.smithi045.stdout:Creating initial monmap... 2024-04-16T16:53:58.851 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-04-16T16:53:58.851 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-04-16T16:53:58.851 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to b1fecd38-fc11-11ee-bc90-c7b262605968 2024-04-16T16:53:58.851 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-04-16T16:53:58.851 INFO:teuthology.orchestra.run.smithi045.stdout:monmaptool for a [v2:172.21.15.45:3300,v1:172.21.15.45:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-04-16T16:53:58.851 INFO:teuthology.orchestra.run.smithi045.stdout:setting min_mon_release = quincy 2024-04-16T16:53:58.851 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/monmaptool: set fsid to b1fecd38-fc11-11ee-bc90-c7b262605968 2024-04-16T16:53:58.851 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-04-16T16:53:58.851 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-16T16:53:58.852 INFO:teuthology.orchestra.run.smithi045.stdout:Creating mon... 2024-04-16T16:54:00.003 INFO:teuthology.orchestra.run.smithi045.stdout:create mon.a on 2024-04-16T16:54:00.378 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-04-16T16:54:00.628 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-04-16T16:54:00.943 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-b1fecd38-fc11-11ee-bc90-c7b262605968.target → /etc/systemd/system/ceph-b1fecd38-fc11-11ee-bc90-c7b262605968.target. 2024-04-16T16:54:00.943 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-b1fecd38-fc11-11ee-bc90-c7b262605968.target → /etc/systemd/system/ceph-b1fecd38-fc11-11ee-bc90-c7b262605968.target. 2024-04-16T16:54:01.327 INFO:teuthology.orchestra.run.smithi045.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-b1fecd38-fc11-11ee-bc90-c7b262605968@mon.a 2024-04-16T16:54:01.327 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stderr Failed to reset failed state of unit ceph-b1fecd38-fc11-11ee-bc90-c7b262605968@mon.a.service: Unit ceph-b1fecd38-fc11-11ee-bc90-c7b262605968@mon.a.service not loaded. 2024-04-16T16:54:01.609 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-b1fecd38-fc11-11ee-bc90-c7b262605968.target.wants/ceph-b1fecd38-fc11-11ee-bc90-c7b262605968@mon.a.service → /etc/systemd/system/ceph-b1fecd38-fc11-11ee-bc90-c7b262605968@.service. 2024-04-16T16:54:02.297 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:02 smithi045 podman[31288]: 2024-04-16 16:54:02.052649781 +0000 UTC m=+0.178189757 container create 0f00ccaf459552a49b8d3056c43c0bf4586ffeee0c3036368a8a852f2dbb80a3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45, name=ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mon-a, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240409, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-16T16:54:02.297 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:02 smithi045 podman[31288]: 2024-04-16 16:54:02.294928722 +0000 UTC m=+0.420468705 container init 0f00ccaf459552a49b8d3056c43c0bf4586ffeee0c3036368a8a852f2dbb80a3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45, name=ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mon-a, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240409, GIT_BRANCH=HEAD) 2024-04-16T16:54:02.391 INFO:teuthology.orchestra.run.smithi045.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-16T16:54:02.392 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stdout disabled 2024-04-16T16:54:02.407 INFO:teuthology.orchestra.run.smithi045.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-16T16:54:02.407 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stdout inactive 2024-04-16T16:54:02.407 INFO:teuthology.orchestra.run.smithi045.stdout:firewalld.service is not enabled 2024-04-16T16:54:02.408 INFO:teuthology.orchestra.run.smithi045.stdout:Not possible to enable service . firewalld.service is not available 2024-04-16T16:54:02.408 INFO:teuthology.orchestra.run.smithi045.stdout:Waiting for mon to start... 2024-04-16T16:54:02.408 INFO:teuthology.orchestra.run.smithi045.stdout:Waiting for mon... 2024-04-16T16:54:02.546 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:02 smithi045 podman[31288]: 2024-04-16 16:54:02.304554058 +0000 UTC m=+0.430094000 container start 0f00ccaf459552a49b8d3056c43c0bf4586ffeee0c3036368a8a852f2dbb80a3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45, name=ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mon-a, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240409, GIT_BRANCH=HEAD, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-16T16:54:02.547 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:02 smithi045 bash[31288]: 0f00ccaf459552a49b8d3056c43c0bf4586ffeee0c3036368a8a852f2dbb80a3 2024-04-16T16:54:02.547 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:02 smithi045 systemd[1]: Started Ceph mon.a for b1fecd38-fc11-11ee-bc90-c7b262605968. 2024-04-16T16:54:02.547 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:02 smithi045 ceph-mon[31307]: mkfs b1fecd38-fc11-11ee-bc90-c7b262605968 2024-04-16T16:54:02.547 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:02 smithi045 ceph-mon[31307]: mon.a is new leader, mons a in quorum (ranks 0) 2024-04-16T16:54:03.729 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:03 smithi045 ceph-mon[31307]: mon.a is new leader, mons a in quorum (ranks 0) 2024-04-16T16:54:03.729 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:03 smithi045 ceph-mon[31307]: monmap epoch 1 2024-04-16T16:54:03.729 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:03 smithi045 ceph-mon[31307]: fsid b1fecd38-fc11-11ee-bc90-c7b262605968 2024-04-16T16:54:03.729 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:03 smithi045 ceph-mon[31307]: last_changed 2024-04-16T16:53:58.140859+0000 2024-04-16T16:54:03.729 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:03 smithi045 ceph-mon[31307]: created 2024-04-16T16:53:58.140859+0000 2024-04-16T16:54:03.729 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:03 smithi045 ceph-mon[31307]: min_mon_release 19 (squid) 2024-04-16T16:54:03.729 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:03 smithi045 ceph-mon[31307]: election_strategy: 1 2024-04-16T16:54:03.729 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:03 smithi045 ceph-mon[31307]: 0: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.a 2024-04-16T16:54:03.729 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:03 smithi045 ceph-mon[31307]: fsmap 2024-04-16T16:54:03.729 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:03 smithi045 ceph-mon[31307]: osdmap e1: 0 total, 0 up, 0 in 2024-04-16T16:54:03.729 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:03 smithi045 ceph-mon[31307]: mgrmap e1: no daemons active 2024-04-16T16:54:03.729 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:03 smithi045 ceph-mon[31307]: from='client.? 172.21.15.45:0/2808880097' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-04-16T16:54:03.786 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout cluster: 2024-04-16T16:54:03.794 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout id: b1fecd38-fc11-11ee-bc90-c7b262605968 2024-04-16T16:54:03.794 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-04-16T16:54:03.794 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout 2024-04-16T16:54:03.794 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout services: 2024-04-16T16:54:03.794 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.701473s) 2024-04-16T16:54:03.794 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-04-16T16:54:03.794 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-04-16T16:54:03.794 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout 2024-04-16T16:54:03.794 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout data: 2024-04-16T16:54:03.794 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-04-16T16:54:03.794 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-04-16T16:54:03.794 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-04-16T16:54:03.795 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout pgs: 2024-04-16T16:54:03.795 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout 2024-04-16T16:54:03.795 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:02.992+0000 7faacda00700 1 Processor -- start 2024-04-16T16:54:03.795 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:02.993+0000 7faacda00700 1 -- start start 2024-04-16T16:54:03.795 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:02.993+0000 7faacda00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faac807dde0 0x7faac807c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:54:03.795 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:02.993+0000 7faacda00700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7faac807c820 con 0x7faac807dde0 2024-04-16T16:54:03.795 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:02.994+0000 7faac7e00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faac807dde0 0x7faac807c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:54:03.795 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:02.994+0000 7faac7e00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faac807dde0 0x7faac807c2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38348/0 (socket says 172.21.15.45:38348) 2024-04-16T16:54:03.795 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:02.994+0000 7faac7e00700 1 -- 172.21.15.45:0/3814937710 learned_addr learned my addr 172.21.15.45:0/3814937710 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T16:54:03.795 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:02.995+0000 7faac7e00700 1 -- 172.21.15.45:0/3814937710 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faac807c9a0 con 0x7faac807dde0 2024-04-16T16:54:03.795 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:02.995+0000 7faac7e00700 1 --2- 172.21.15.45:0/3814937710 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faac807dde0 0x7faac807c2e0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7faab80098a0 tx=0x7faab8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1379323d25be1242 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T16:54:03.795 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:02.995+0000 7faac7400700 1 -- 172.21.15.45:0/3814937710 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faab8004160 con 0x7faac807dde0 2024-04-16T16:54:03.795 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:02.995+0000 7faac7400700 1 -- 172.21.15.45:0/3814937710 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faab8004300 con 0x7faac807dde0 2024-04-16T16:54:03.796 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:02.995+0000 7faac7400700 1 -- 172.21.15.45:0/3814937710 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faab80036a0 con 0x7faac807dde0 2024-04-16T16:54:03.796 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:02.996+0000 7faacda00700 1 -- 172.21.15.45:0/3814937710 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faac807dde0 msgr2=0x7faac807c2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:54:03.796 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:02.996+0000 7faacda00700 1 --2- 172.21.15.45:0/3814937710 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faac807dde0 0x7faac807c2e0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7faab80098a0 tx=0x7faab8009bb0 comp rx=0 tx=0).stop 2024-04-16T16:54:03.796 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:02.996+0000 7faacda00700 1 -- 172.21.15.45:0/3814937710 shutdown_connections 2024-04-16T16:54:03.796 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:02.996+0000 7faacda00700 1 --2- 172.21.15.45:0/3814937710 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faac807dde0 0x7faac807c2e0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:54:03.796 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:02.996+0000 7faacda00700 1 -- 172.21.15.45:0/3814937710 >> 172.21.15.45:0/3814937710 conn(0x7faac81039d0 msgr2=0x7faac8105e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T16:54:03.796 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:02.996+0000 7faacda00700 1 -- 172.21.15.45:0/3814937710 shutdown_connections 2024-04-16T16:54:03.796 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:02.997+0000 7faacda00700 1 -- 172.21.15.45:0/3814937710 wait complete. 2024-04-16T16:54:03.796 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:02.998+0000 7faacda00700 1 Processor -- start 2024-04-16T16:54:03.796 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:02.998+0000 7faacda00700 1 -- start start 2024-04-16T16:54:03.796 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:02.998+0000 7faacda00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faac807dde0 0x7faac819dd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:54:03.796 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:02.998+0000 7faacda00700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7faac807ce70 con 0x7faac807dde0 2024-04-16T16:54:03.796 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:02.999+0000 7faac7e00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faac807dde0 0x7faac819dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:54:03.797 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:02.999+0000 7faac7e00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faac807dde0 0x7faac819dd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38356/0 (socket says 172.21.15.45:38356) 2024-04-16T16:54:03.797 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:02.999+0000 7faac7e00700 1 -- 172.21.15.45:0/2808880097 learned_addr learned my addr 172.21.15.45:0/2808880097 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T16:54:03.797 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:02.999+0000 7faac7e00700 1 -- 172.21.15.45:0/2808880097 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faac819e260 con 0x7faac807dde0 2024-04-16T16:54:03.797 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:02.999+0000 7faac7e00700 1 --2- 172.21.15.45:0/2808880097 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faac807dde0 0x7faac819dd20 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7faab8000c00 tx=0x7faab8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T16:54:03.797 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:03.000+0000 7faac5600700 1 -- 172.21.15.45:0/2808880097 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faab8003db0 con 0x7faac807dde0 2024-04-16T16:54:03.797 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:03.000+0000 7faac5600700 1 -- 172.21.15.45:0/2808880097 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faab8011630 con 0x7faac807dde0 2024-04-16T16:54:03.797 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:03.000+0000 7faac5600700 1 -- 172.21.15.45:0/2808880097 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faab8011950 con 0x7faac807dde0 2024-04-16T16:54:03.797 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:03.000+0000 7faacda00700 1 -- 172.21.15.45:0/2808880097 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faac819e4f0 con 0x7faac807dde0 2024-04-16T16:54:03.797 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:03.000+0000 7faacda00700 1 -- 172.21.15.45:0/2808880097 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faac81a11d0 con 0x7faac807dde0 2024-04-16T16:54:03.797 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:03.000+0000 7faac5600700 1 -- 172.21.15.45:0/2808880097 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7faab801a460 con 0x7faac807dde0 2024-04-16T16:54:03.797 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:03.001+0000 7faac5600700 1 -- 172.21.15.45:0/2808880097 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7faab801bb40 con 0x7faac807dde0 2024-04-16T16:54:03.797 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:03.001+0000 7faacda00700 1 -- 172.21.15.45:0/2808880097 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faac8064150 con 0x7faac807dde0 2024-04-16T16:54:03.797 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:03.005+0000 7faac5600700 1 -- 172.21.15.45:0/2808880097 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7faab8004260 con 0x7faac807dde0 2024-04-16T16:54:03.797 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:03.085+0000 7faacda00700 1 -- 172.21.15.45:0/2808880097 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7faac802e1f0 con 0x7faac807dde0 2024-04-16T16:54:03.798 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:03.085+0000 7faac5600700 1 -- 172.21.15.45:0/2808880097 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+317 (secure 0 0 0) 0x7faab801bd40 con 0x7faac807dde0 2024-04-16T16:54:03.798 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:03.087+0000 7faacda00700 1 -- 172.21.15.45:0/2808880097 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faac807dde0 msgr2=0x7faac819dd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:54:03.798 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:03.087+0000 7faacda00700 1 --2- 172.21.15.45:0/2808880097 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faac807dde0 0x7faac819dd20 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7faab8000c00 tx=0x7faab8004750 comp rx=0 tx=0).stop 2024-04-16T16:54:03.798 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:03.087+0000 7faacda00700 1 -- 172.21.15.45:0/2808880097 shutdown_connections 2024-04-16T16:54:03.798 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:03.087+0000 7faacda00700 1 --2- 172.21.15.45:0/2808880097 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faac807dde0 0x7faac819dd20 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:54:03.798 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:03.087+0000 7faacda00700 1 -- 172.21.15.45:0/2808880097 >> 172.21.15.45:0/2808880097 conn(0x7faac81039d0 msgr2=0x7faac8105cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T16:54:03.798 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:03.087+0000 7faacda00700 1 -- 172.21.15.45:0/2808880097 shutdown_connections 2024-04-16T16:54:03.798 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:03.087+0000 7faacda00700 1 -- 172.21.15.45:0/2808880097 wait complete. 2024-04-16T16:54:03.798 INFO:teuthology.orchestra.run.smithi045.stdout:mon is available 2024-04-16T16:54:03.798 INFO:teuthology.orchestra.run.smithi045.stdout:Assimilating anything we can from ceph.conf... 2024-04-16T16:54:04.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:04 smithi045 ceph-mon[31307]: from='client.? 172.21.15.45:0/3518384599' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-04-16T16:54:04.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:04 smithi045 ceph-mon[31307]: from='client.? 172.21.15.45:0/3518384599' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-04-16T16:54:05.164 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout 2024-04-16T16:54:05.164 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout [global] 2024-04-16T16:54:05.165 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout fsid = b1fecd38-fc11-11ee-bc90-c7b262605968 2024-04-16T16:54:05.165 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.45:3300,v1:172.21.15.45:6789] 2024-04-16T16:54:05.165 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-04-16T16:54:05.165 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-04-16T16:54:05.165 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-04-16T16:54:05.165 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-04-16T16:54:05.165 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout 2024-04-16T16:54:05.165 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout [mgr] 2024-04-16T16:54:05.165 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-04-16T16:54:05.165 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout 2024-04-16T16:54:05.165 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout [osd] 2024-04-16T16:54:05.165 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-04-16T16:54:05.165 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-04-16T16:54:05.165 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-04-16T16:54:05.166 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-04-16T16:54:05.166 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-04-16T16:54:05.166 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:04.405+0000 7f0e5e000700 1 Processor -- start 2024-04-16T16:54:05.166 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:04.406+0000 7f0e5e000700 1 -- start start 2024-04-16T16:54:05.166 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:04.406+0000 7f0e5e000700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0e5807de90 0x7f0e5807c340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:54:05.166 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:04.406+0000 7f0e5e000700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f0e5807c910 con 0x7f0e5807de90 2024-04-16T16:54:05.166 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:04.407+0000 7f0e57400700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0e5807de90 0x7f0e5807c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:54:05.166 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:04.407+0000 7f0e57400700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0e5807de90 0x7f0e5807c340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38370/0 (socket says 172.21.15.45:38370) 2024-04-16T16:54:05.166 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:04.407+0000 7f0e57400700 1 -- 172.21.15.45:0/3965903378 learned_addr learned my addr 172.21.15.45:0/3965903378 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T16:54:05.166 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:04.407+0000 7f0e57400700 1 -- 172.21.15.45:0/3965903378 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e5807ca90 con 0x7f0e5807de90 2024-04-16T16:54:05.166 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:04.408+0000 7f0e57400700 1 --2- 172.21.15.45:0/3965903378 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0e5807de90 0x7f0e5807c340 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f0e480098a0 tx=0x7f0e48009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6829b3ed73223e3e server_cookie=0 in_seq=0 out_seq=0 2024-04-16T16:54:05.166 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:04.408+0000 7f0e56000700 1 -- 172.21.15.45:0/3965903378 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0e48004160 con 0x7f0e5807de90 2024-04-16T16:54:05.166 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:04.408+0000 7f0e56000700 1 -- 172.21.15.45:0/3965903378 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e48004300 con 0x7f0e5807de90 2024-04-16T16:54:05.167 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:04.408+0000 7f0e56000700 1 -- 172.21.15.45:0/3965903378 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0e480036a0 con 0x7f0e5807de90 2024-04-16T16:54:05.170 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:04.409+0000 7f0e5e000700 1 -- 172.21.15.45:0/3965903378 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0e5807de90 msgr2=0x7f0e5807c340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:54:05.170 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:04.409+0000 7f0e5e000700 1 --2- 172.21.15.45:0/3965903378 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0e5807de90 0x7f0e5807c340 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f0e480098a0 tx=0x7f0e48009bb0 comp rx=0 tx=0).stop 2024-04-16T16:54:05.170 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:04.409+0000 7f0e5e000700 1 -- 172.21.15.45:0/3965903378 shutdown_connections 2024-04-16T16:54:05.170 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:04.409+0000 7f0e5e000700 1 --2- 172.21.15.45:0/3965903378 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0e5807de90 0x7f0e5807c340 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:54:05.171 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:04.409+0000 7f0e5e000700 1 -- 172.21.15.45:0/3965903378 >> 172.21.15.45:0/3965903378 conn(0x7f0e58103bc0 msgr2=0x7f0e58105fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T16:54:05.171 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:04.409+0000 7f0e5e000700 1 -- 172.21.15.45:0/3965903378 shutdown_connections 2024-04-16T16:54:05.171 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:04.409+0000 7f0e5e000700 1 -- 172.21.15.45:0/3965903378 wait complete. 2024-04-16T16:54:05.171 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:04.410+0000 7f0e5e000700 1 Processor -- start 2024-04-16T16:54:05.171 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:04.410+0000 7f0e5e000700 1 -- start start 2024-04-16T16:54:05.171 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:04.410+0000 7f0e5e000700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0e5807de90 0x7f0e581a2280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:54:05.171 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:04.410+0000 7f0e5e000700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f0e5807cfc0 con 0x7f0e5807de90 2024-04-16T16:54:05.171 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:04.411+0000 7f0e57400700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0e5807de90 0x7f0e581a2280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:54:05.171 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:04.411+0000 7f0e57400700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0e5807de90 0x7f0e581a2280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38384/0 (socket says 172.21.15.45:38384) 2024-04-16T16:54:05.171 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:04.411+0000 7f0e57400700 1 -- 172.21.15.45:0/3518384599 learned_addr learned my addr 172.21.15.45:0/3518384599 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T16:54:05.171 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:04.411+0000 7f0e57400700 1 -- 172.21.15.45:0/3518384599 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e581a27c0 con 0x7f0e5807de90 2024-04-16T16:54:05.172 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:04.411+0000 7f0e57400700 1 --2- 172.21.15.45:0/3518384599 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0e5807de90 0x7f0e581a2280 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f0e48000c00 tx=0x7f0e48004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T16:54:05.172 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:04.412+0000 7f0e4fe00700 1 -- 172.21.15.45:0/3518384599 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0e48003db0 con 0x7f0e5807de90 2024-04-16T16:54:05.172 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:04.412+0000 7f0e4fe00700 1 -- 172.21.15.45:0/3518384599 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e48011630 con 0x7f0e5807de90 2024-04-16T16:54:05.172 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:04.412+0000 7f0e5e000700 1 -- 172.21.15.45:0/3518384599 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e581a2a50 con 0x7f0e5807de90 2024-04-16T16:54:05.172 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:04.412+0000 7f0e4fe00700 1 -- 172.21.15.45:0/3518384599 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0e48011950 con 0x7f0e5807de90 2024-04-16T16:54:05.172 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:04.412+0000 7f0e5e000700 1 -- 172.21.15.45:0/3518384599 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e581a5740 con 0x7f0e5807de90 2024-04-16T16:54:05.172 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:04.412+0000 7f0e4fe00700 1 -- 172.21.15.45:0/3518384599 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f0e48011af0 con 0x7f0e5807de90 2024-04-16T16:54:05.172 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:04.412+0000 7f0e4fe00700 1 -- 172.21.15.45:0/3518384599 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f0e4801c790 con 0x7f0e5807de90 2024-04-16T16:54:05.172 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:04.413+0000 7f0e5e000700 1 -- 172.21.15.45:0/3518384599 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e58064150 con 0x7f0e5807de90 2024-04-16T16:54:05.172 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:04.417+0000 7f0e4fe00700 1 -- 172.21.15.45:0/3518384599 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f0e48032020 con 0x7f0e5807de90 2024-04-16T16:54:05.172 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:04.496+0000 7f0e5e000700 1 -- 172.21.15.45:0/3518384599 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f0e5802e120 con 0x7f0e5807de90 2024-04-16T16:54:05.172 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:04.503+0000 7f0e4fe00700 1 -- 172.21.15.45:0/3518384599 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) ==== 70+0+429 (secure 0 0 0) 0x7f0e48004260 con 0x7f0e5807de90 2024-04-16T16:54:05.172 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:04.503+0000 7f0e4fe00700 1 -- 172.21.15.45:0/3518384599 <== mon.0 v2:172.21.15.45:3300/0 8 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f0e4801ce50 con 0x7f0e5807de90 2024-04-16T16:54:05.173 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:04.505+0000 7f0e5e000700 1 -- 172.21.15.45:0/3518384599 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0e5807de90 msgr2=0x7f0e581a2280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:54:05.173 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:04.505+0000 7f0e5e000700 1 --2- 172.21.15.45:0/3518384599 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0e5807de90 0x7f0e581a2280 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f0e48000c00 tx=0x7f0e48004750 comp rx=0 tx=0).stop 2024-04-16T16:54:05.173 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:04.505+0000 7f0e5e000700 1 -- 172.21.15.45:0/3518384599 shutdown_connections 2024-04-16T16:54:05.173 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:04.505+0000 7f0e5e000700 1 --2- 172.21.15.45:0/3518384599 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0e5807de90 0x7f0e581a2280 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:54:05.173 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:04.505+0000 7f0e5e000700 1 -- 172.21.15.45:0/3518384599 >> 172.21.15.45:0/3518384599 conn(0x7f0e58103bc0 msgr2=0x7f0e58105fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T16:54:05.173 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:04.505+0000 7f0e5e000700 1 -- 172.21.15.45:0/3518384599 shutdown_connections 2024-04-16T16:54:05.173 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:04.505+0000 7f0e5e000700 1 -- 172.21.15.45:0/3518384599 wait complete. 2024-04-16T16:54:05.173 INFO:teuthology.orchestra.run.smithi045.stdout:Generating new minimal ceph.conf... 2024-04-16T16:54:05.978 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:05 smithi045 ceph-mon[31307]: from='client.? 172.21.15.45:0/1160873998' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:54:06.557 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:05.813+0000 7ff971e00700 1 Processor -- start 2024-04-16T16:54:06.558 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:05.813+0000 7ff971e00700 1 -- start start 2024-04-16T16:54:06.558 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:05.814+0000 7ff971e00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff96c106e80 0x7ff96c1072a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:54:06.558 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:05.814+0000 7ff971e00700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7ff96c07be50 con 0x7ff96c106e80 2024-04-16T16:54:06.559 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:05.814+0000 7ff96aa00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff96c106e80 0x7ff96c1072a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:54:06.559 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:05.814+0000 7ff96aa00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff96c106e80 0x7ff96c1072a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38386/0 (socket says 172.21.15.45:38386) 2024-04-16T16:54:06.559 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:05.814+0000 7ff96aa00700 1 -- 172.21.15.45:0/682555555 learned_addr learned my addr 172.21.15.45:0/682555555 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T16:54:06.559 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:05.815+0000 7ff96aa00700 1 -- 172.21.15.45:0/682555555 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff96c1077e0 con 0x7ff96c106e80 2024-04-16T16:54:06.559 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:05.815+0000 7ff96aa00700 1 --2- 172.21.15.45:0/682555555 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff96c106e80 0x7ff96c1072a0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7ff95c0098a0 tx=0x7ff95c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8895a6540f6167ed server_cookie=0 in_seq=0 out_seq=0 2024-04-16T16:54:06.559 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:05.815+0000 7ff969600700 1 -- 172.21.15.45:0/682555555 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff95c004160 con 0x7ff96c106e80 2024-04-16T16:54:06.559 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:05.815+0000 7ff969600700 1 -- 172.21.15.45:0/682555555 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7ff95c004300 con 0x7ff96c106e80 2024-04-16T16:54:06.560 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:05.815+0000 7ff969600700 1 -- 172.21.15.45:0/682555555 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff95c011b50 con 0x7ff96c106e80 2024-04-16T16:54:06.560 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:05.816+0000 7ff971e00700 1 -- 172.21.15.45:0/682555555 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff96c106e80 msgr2=0x7ff96c1072a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:54:06.561 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:05.816+0000 7ff971e00700 1 --2- 172.21.15.45:0/682555555 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff96c106e80 0x7ff96c1072a0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7ff95c0098a0 tx=0x7ff95c009bb0 comp rx=0 tx=0).stop 2024-04-16T16:54:06.561 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:05.816+0000 7ff971e00700 1 -- 172.21.15.45:0/682555555 shutdown_connections 2024-04-16T16:54:06.562 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:05.816+0000 7ff971e00700 1 --2- 172.21.15.45:0/682555555 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff96c106e80 0x7ff96c1072a0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:54:06.562 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:05.816+0000 7ff971e00700 1 -- 172.21.15.45:0/682555555 >> 172.21.15.45:0/682555555 conn(0x7ff96c102a60 msgr2=0x7ff96c104ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T16:54:06.562 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:05.816+0000 7ff971e00700 1 -- 172.21.15.45:0/682555555 shutdown_connections 2024-04-16T16:54:06.562 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:05.816+0000 7ff971e00700 1 -- 172.21.15.45:0/682555555 wait complete. 2024-04-16T16:54:06.562 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:05.817+0000 7ff971e00700 1 Processor -- start 2024-04-16T16:54:06.562 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:05.818+0000 7ff971e00700 1 -- start start 2024-04-16T16:54:06.562 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:05.818+0000 7ff971e00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff96c106e80 0x7ff96c19e750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:54:06.562 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:05.818+0000 7ff971e00700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7ff96c108870 con 0x7ff96c106e80 2024-04-16T16:54:06.562 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:05.818+0000 7ff96aa00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff96c106e80 0x7ff96c19e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:54:06.562 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:05.818+0000 7ff96aa00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff96c106e80 0x7ff96c19e750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38402/0 (socket says 172.21.15.45:38402) 2024-04-16T16:54:06.562 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:05.818+0000 7ff96aa00700 1 -- 172.21.15.45:0/1160873998 learned_addr learned my addr 172.21.15.45:0/1160873998 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T16:54:06.562 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:05.819+0000 7ff96aa00700 1 -- 172.21.15.45:0/1160873998 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff96c19ec90 con 0x7ff96c106e80 2024-04-16T16:54:06.564 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:05.819+0000 7ff96aa00700 1 --2- 172.21.15.45:0/1160873998 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff96c106e80 0x7ff96c19e750 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7ff95c011b60 tx=0x7ff95c011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T16:54:06.564 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:05.819+0000 7ff970a00700 1 -- 172.21.15.45:0/1160873998 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff95c003b20 con 0x7ff96c106e80 2024-04-16T16:54:06.564 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:05.819+0000 7ff970a00700 1 -- 172.21.15.45:0/1160873998 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7ff95c003cc0 con 0x7ff96c106e80 2024-04-16T16:54:06.564 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:05.819+0000 7ff970a00700 1 -- 172.21.15.45:0/1160873998 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff95c024840 con 0x7ff96c106e80 2024-04-16T16:54:06.565 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:05.819+0000 7ff971e00700 1 -- 172.21.15.45:0/1160873998 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff96c19ef20 con 0x7ff96c106e80 2024-04-16T16:54:06.565 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:05.819+0000 7ff971e00700 1 -- 172.21.15.45:0/1160873998 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff96c1a1c10 con 0x7ff96c106e80 2024-04-16T16:54:06.565 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:05.820+0000 7ff970a00700 1 -- 172.21.15.45:0/1160873998 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7ff95c022020 con 0x7ff96c106e80 2024-04-16T16:54:06.565 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:05.820+0000 7ff970a00700 1 -- 172.21.15.45:0/1160873998 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7ff95c011e00 con 0x7ff96c106e80 2024-04-16T16:54:06.565 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:05.821+0000 7ff971e00700 1 -- 172.21.15.45:0/1160873998 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff96c064150 con 0x7ff96c106e80 2024-04-16T16:54:06.565 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:05.824+0000 7ff970a00700 1 -- 172.21.15.45:0/1160873998 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7ff95c043e50 con 0x7ff96c106e80 2024-04-16T16:54:06.565 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:05.903+0000 7ff971e00700 1 -- 172.21.15.45:0/1160873998 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7ff96c02e280 con 0x7ff96c106e80 2024-04-16T16:54:06.565 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:05.903+0000 7ff970a00700 1 -- 172.21.15.45:0/1160873998 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) ==== 76+0+175 (secure 0 0 0) 0x7ff95c032020 con 0x7ff96c106e80 2024-04-16T16:54:06.565 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:05.904+0000 7ff971e00700 1 -- 172.21.15.45:0/1160873998 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff96c106e80 msgr2=0x7ff96c19e750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:54:06.565 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:05.904+0000 7ff971e00700 1 --2- 172.21.15.45:0/1160873998 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff96c106e80 0x7ff96c19e750 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7ff95c011b60 tx=0x7ff95c011c40 comp rx=0 tx=0).stop 2024-04-16T16:54:06.565 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:05.905+0000 7ff971e00700 1 -- 172.21.15.45:0/1160873998 shutdown_connections 2024-04-16T16:54:06.565 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:05.905+0000 7ff971e00700 1 --2- 172.21.15.45:0/1160873998 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff96c106e80 0x7ff96c19e750 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:54:06.566 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:05.905+0000 7ff971e00700 1 -- 172.21.15.45:0/1160873998 >> 172.21.15.45:0/1160873998 conn(0x7ff96c102a60 msgr2=0x7ff96c195430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T16:54:06.566 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:05.905+0000 7ff971e00700 1 -- 172.21.15.45:0/1160873998 shutdown_connections 2024-04-16T16:54:06.566 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:05.905+0000 7ff971e00700 1 -- 172.21.15.45:0/1160873998 wait complete. 2024-04-16T16:54:06.566 INFO:teuthology.orchestra.run.smithi045.stdout:Restarting the monitor... 2024-04-16T16:54:06.978 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:06 smithi045 systemd[1]: Stopping Ceph mon.a for b1fecd38-fc11-11ee-bc90-c7b262605968... 2024-04-16T16:54:07.439 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:07 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mon-a[31303]: 2024-04-16T16:54:07.064+0000 7f7644e00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-16T16:54:07.439 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:07 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mon-a[31303]: 2024-04-16T16:54:07.064+0000 7f7644e00700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-04-16T16:54:07.439 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:07 smithi045 podman[31548]: 2024-04-16 16:54:07.116753274 +0000 UTC m=+0.190619270 container died 0f00ccaf459552a49b8d3056c43c0bf4586ffeee0c3036368a8a852f2dbb80a3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45, name=ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mon-a, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240409, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-16T16:54:07.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:07 smithi045 podman[31548]: 2024-04-16 16:54:07.549206921 +0000 UTC m=+0.623072911 container cleanup 0f00ccaf459552a49b8d3056c43c0bf4586ffeee0c3036368a8a852f2dbb80a3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45, name=ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mon-a, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240409, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-16T16:54:07.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:07 smithi045 bash[31548]: ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mon-a 2024-04-16T16:54:08.228 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:07 smithi045 podman[31561]: 2024-04-16 16:54:07.79187709 +0000 UTC m=+0.659327826 container remove 0f00ccaf459552a49b8d3056c43c0bf4586ffeee0c3036368a8a852f2dbb80a3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45, name=ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mon-a, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240409, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-04-16T16:54:08.684 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:08 smithi045 systemd[1]: ceph-b1fecd38-fc11-11ee-bc90-c7b262605968@mon.a.service: Deactivated successfully. 2024-04-16T16:54:08.684 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:08 smithi045 systemd[1]: Stopped Ceph mon.a for b1fecd38-fc11-11ee-bc90-c7b262605968. 2024-04-16T16:54:08.684 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:08 smithi045 systemd[1]: ceph-b1fecd38-fc11-11ee-bc90-c7b262605968@mon.a.service: Consumed 1.931s CPU time. 2024-04-16T16:54:08.684 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:08 smithi045 systemd[1]: Starting Ceph mon.a for b1fecd38-fc11-11ee-bc90-c7b262605968... 2024-04-16T16:54:08.978 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:08 smithi045 podman[31658]: 2024-04-16 16:54:08.589644324 +0000 UTC m=+0.039717597 image pull e5ce10bdffbeeffffd1125f67b369e2151f0f17cdec854e93b6adf9321f65907 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45 2024-04-16T16:54:08.978 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:08 smithi045 podman[31658]: 2024-04-16 16:54:08.684245586 +0000 UTC m=+0.134318905 container create 5e03aa10df484d017679f33bd15f25d228119ead2a392d47d4e2705506ca956f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45, name=ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mon-a, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240409, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-04-16T16:54:14.229 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:13 smithi045 podman[31658]: 2024-04-16 16:54:13.846952454 +0000 UTC m=+5.297025799 container init 5e03aa10df484d017679f33bd15f25d228119ead2a392d47d4e2705506ca956f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45, name=ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mon-a, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240409, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-04-16T16:54:14.229 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:13 smithi045 podman[31658]: 2024-04-16 16:54:13.858235361 +0000 UTC m=+5.308308627 container start 5e03aa10df484d017679f33bd15f25d228119ead2a392d47d4e2705506ca956f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45, name=ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mon-a, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240409, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-16T16:54:14.229 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:13 smithi045 ceph-mon[31676]: set uid:gid to 167:167 (ceph:ceph) 2024-04-16T16:54:14.230 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:13 smithi045 ceph-mon[31676]: ceph version 19.0.0-3101-g9b42bfbe (9b42bfbeda298874861ece590aa565cf64ac8c45) squid (dev), process ceph-mon, pid 2 2024-04-16T16:54:14.230 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:13 smithi045 ceph-mon[31676]: pidfile_write: ignore empty --pid-file 2024-04-16T16:54:14.230 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:13 smithi045 ceph-mon[31676]: load: jerasure load: lrc 2024-04-16T16:54:14.230 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:13 smithi045 ceph-mon[31676]: rocksdb: RocksDB version: 7.9.2 2024-04-16T16:54:14.230 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:13 smithi045 ceph-mon[31676]: rocksdb: Git sha 0 2024-04-16T16:54:14.230 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:13 smithi045 ceph-mon[31676]: rocksdb: Compile date 2024-04-16 06:46:19 2024-04-16T16:54:14.230 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:13 smithi045 ceph-mon[31676]: rocksdb: DB SUMMARY 2024-04-16T16:54:14.230 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:13 smithi045 ceph-mon[31676]: rocksdb: DB Session ID: KZ4FSGY6F4B3SSUNGX87 2024-04-16T16:54:14.230 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:13 smithi045 ceph-mon[31676]: rocksdb: CURRENT file: CURRENT 2024-04-16T16:54:14.230 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:13 smithi045 ceph-mon[31676]: rocksdb: IDENTITY file: IDENTITY 2024-04-16T16:54:14.230 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:13 smithi045 ceph-mon[31676]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-04-16T16:54:14.230 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:13 smithi045 ceph-mon[31676]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-04-16T16:54:14.230 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:13 smithi045 ceph-mon[31676]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 105090 ; 2024-04-16T16:54:14.231 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:13 smithi045 ceph-mon[31676]: rocksdb: Options.error_if_exists: 0 2024-04-16T16:54:14.231 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:13 smithi045 ceph-mon[31676]: rocksdb: Options.create_if_missing: 0 2024-04-16T16:54:14.231 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:13 smithi045 ceph-mon[31676]: rocksdb: Options.paranoid_checks: 1 2024-04-16T16:54:14.231 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:13 smithi045 ceph-mon[31676]: rocksdb: Options.flush_verify_memtable_count: 1 2024-04-16T16:54:14.231 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:13 smithi045 ceph-mon[31676]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-04-16T16:54:14.231 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:13 smithi045 ceph-mon[31676]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-04-16T16:54:14.231 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:13 smithi045 ceph-mon[31676]: rocksdb: Options.env: 0x55727755bcc0 2024-04-16T16:54:14.231 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:13 smithi045 ceph-mon[31676]: rocksdb: Options.fs: PosixFileSystem 2024-04-16T16:54:14.231 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:13 smithi045 ceph-mon[31676]: rocksdb: Options.info_log: 0x5572a9cd7c60 2024-04-16T16:54:14.231 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:13 smithi045 ceph-mon[31676]: rocksdb: Options.max_file_opening_threads: 16 2024-04-16T16:54:14.232 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:13 smithi045 ceph-mon[31676]: rocksdb: Options.statistics: (nil) 2024-04-16T16:54:14.232 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:13 smithi045 ceph-mon[31676]: rocksdb: Options.use_fsync: 0 2024-04-16T16:54:14.232 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:13 smithi045 ceph-mon[31676]: rocksdb: Options.max_log_file_size: 0 2024-04-16T16:54:14.232 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:13 smithi045 ceph-mon[31676]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-04-16T16:54:14.232 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:13 smithi045 ceph-mon[31676]: rocksdb: Options.log_file_time_to_roll: 0 2024-04-16T16:54:14.232 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:13 smithi045 ceph-mon[31676]: rocksdb: Options.keep_log_file_num: 1000 2024-04-16T16:54:14.232 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:13 smithi045 ceph-mon[31676]: rocksdb: Options.recycle_log_file_num: 0 2024-04-16T16:54:14.232 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:13 smithi045 ceph-mon[31676]: rocksdb: Options.allow_fallocate: 1 2024-04-16T16:54:14.232 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:13 smithi045 ceph-mon[31676]: rocksdb: Options.allow_mmap_reads: 0 2024-04-16T16:54:14.232 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:13 smithi045 ceph-mon[31676]: rocksdb: Options.allow_mmap_writes: 0 2024-04-16T16:54:14.232 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:13 smithi045 ceph-mon[31676]: rocksdb: Options.use_direct_reads: 0 2024-04-16T16:54:14.232 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:13 smithi045 ceph-mon[31676]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-04-16T16:54:14.233 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:13 smithi045 ceph-mon[31676]: rocksdb: Options.create_missing_column_families: 0 2024-04-16T16:54:14.233 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:13 smithi045 ceph-mon[31676]: rocksdb: Options.db_log_dir: 2024-04-16T16:54:14.233 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:13 smithi045 ceph-mon[31676]: rocksdb: Options.wal_dir: 2024-04-16T16:54:14.233 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:13 smithi045 ceph-mon[31676]: rocksdb: Options.table_cache_numshardbits: 6 2024-04-16T16:54:14.233 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:13 smithi045 ceph-mon[31676]: rocksdb: Options.WAL_ttl_seconds: 0 2024-04-16T16:54:14.233 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:13 smithi045 ceph-mon[31676]: rocksdb: Options.WAL_size_limit_MB: 0 2024-04-16T16:54:14.233 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:13 smithi045 ceph-mon[31676]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-04-16T16:54:14.233 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:13 smithi045 ceph-mon[31676]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-04-16T16:54:14.233 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:13 smithi045 ceph-mon[31676]: rocksdb: Options.is_fd_close_on_exec: 1 2024-04-16T16:54:14.233 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:13 smithi045 ceph-mon[31676]: rocksdb: Options.advise_random_on_open: 1 2024-04-16T16:54:14.233 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:13 smithi045 ceph-mon[31676]: rocksdb: Options.db_write_buffer_size: 0 2024-04-16T16:54:14.233 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:13 smithi045 ceph-mon[31676]: rocksdb: Options.write_buffer_manager: 0x5572a9ca61e0 2024-04-16T16:54:14.233 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:13 smithi045 ceph-mon[31676]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-04-16T16:54:14.234 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:13 smithi045 ceph-mon[31676]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-04-16T16:54:14.234 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:13 smithi045 ceph-mon[31676]: rocksdb: Options.use_adaptive_mutex: 0 2024-04-16T16:54:14.234 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:13 smithi045 ceph-mon[31676]: rocksdb: Options.rate_limiter: (nil) 2024-04-16T16:54:14.234 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:13 smithi045 ceph-mon[31676]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-04-16T16:54:14.234 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:13 smithi045 ceph-mon[31676]: rocksdb: Options.wal_recovery_mode: 2 2024-04-16T16:54:14.234 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:13 smithi045 ceph-mon[31676]: rocksdb: Options.enable_thread_tracking: 0 2024-04-16T16:54:14.234 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:13 smithi045 ceph-mon[31676]: rocksdb: Options.enable_pipelined_write: 0 2024-04-16T16:54:14.234 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:13 smithi045 ceph-mon[31676]: rocksdb: Options.unordered_write: 0 2024-04-16T16:54:14.234 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:13 smithi045 ceph-mon[31676]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-04-16T16:54:14.234 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:13 smithi045 ceph-mon[31676]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-04-16T16:54:14.234 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:13 smithi045 ceph-mon[31676]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-04-16T16:54:14.234 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:13 smithi045 ceph-mon[31676]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-04-16T16:54:14.234 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:13 smithi045 ceph-mon[31676]: rocksdb: Options.row_cache: None 2024-04-16T16:54:14.234 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:13 smithi045 ceph-mon[31676]: rocksdb: Options.wal_filter: None 2024-04-16T16:54:14.235 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:13 smithi045 ceph-mon[31676]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-04-16T16:54:14.235 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:13 smithi045 ceph-mon[31676]: rocksdb: Options.allow_ingest_behind: 0 2024-04-16T16:54:14.235 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:13 smithi045 ceph-mon[31676]: rocksdb: Options.two_write_queues: 0 2024-04-16T16:54:14.235 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:13 smithi045 ceph-mon[31676]: rocksdb: Options.manual_wal_flush: 0 2024-04-16T16:54:14.235 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:13 smithi045 ceph-mon[31676]: rocksdb: Options.wal_compression: 0 2024-04-16T16:54:14.235 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:13 smithi045 ceph-mon[31676]: rocksdb: Options.atomic_flush: 0 2024-04-16T16:54:14.235 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:13 smithi045 ceph-mon[31676]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-04-16T16:54:14.235 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:13 smithi045 ceph-mon[31676]: rocksdb: Options.persist_stats_to_disk: 0 2024-04-16T16:54:14.235 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:13 smithi045 ceph-mon[31676]: rocksdb: Options.write_dbid_to_manifest: 0 2024-04-16T16:54:14.235 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:13 smithi045 ceph-mon[31676]: rocksdb: Options.log_readahead_size: 0 2024-04-16T16:54:14.235 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:13 smithi045 ceph-mon[31676]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-04-16T16:54:14.235 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:13 smithi045 ceph-mon[31676]: rocksdb: Options.best_efforts_recovery: 0 2024-04-16T16:54:14.235 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:13 smithi045 ceph-mon[31676]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-04-16T16:54:14.236 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:13 smithi045 ceph-mon[31676]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-04-16T16:54:14.236 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:13 smithi045 ceph-mon[31676]: rocksdb: Options.allow_data_in_errors: 0 2024-04-16T16:54:14.236 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:13 smithi045 ceph-mon[31676]: rocksdb: Options.db_host_id: __hostname__ 2024-04-16T16:54:14.236 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:13 smithi045 ceph-mon[31676]: rocksdb: Options.enforce_single_del_contracts: true 2024-04-16T16:54:14.236 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:13 smithi045 ceph-mon[31676]: rocksdb: Options.max_background_jobs: 2 2024-04-16T16:54:14.236 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:13 smithi045 ceph-mon[31676]: rocksdb: Options.max_background_compactions: -1 2024-04-16T16:54:14.236 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:13 smithi045 ceph-mon[31676]: rocksdb: Options.max_subcompactions: 1 2024-04-16T16:54:14.236 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:13 smithi045 ceph-mon[31676]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-04-16T16:54:14.236 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:13 smithi045 ceph-mon[31676]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-04-16T16:54:14.236 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:13 smithi045 ceph-mon[31676]: rocksdb: Options.delayed_write_rate : 16777216 2024-04-16T16:54:14.236 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:13 smithi045 ceph-mon[31676]: rocksdb: Options.max_total_wal_size: 0 2024-04-16T16:54:14.236 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:13 smithi045 ceph-mon[31676]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-04-16T16:54:14.236 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:13 smithi045 ceph-mon[31676]: rocksdb: Options.stats_dump_period_sec: 600 2024-04-16T16:54:14.236 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:13 smithi045 ceph-mon[31676]: rocksdb: Options.stats_persist_period_sec: 600 2024-04-16T16:54:14.236 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:13 smithi045 ceph-mon[31676]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-04-16T16:54:14.237 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:13 smithi045 ceph-mon[31676]: rocksdb: Options.max_open_files: -1 2024-04-16T16:54:14.237 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:13 smithi045 ceph-mon[31676]: rocksdb: Options.bytes_per_sync: 0 2024-04-16T16:54:14.237 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:13 smithi045 ceph-mon[31676]: rocksdb: Options.wal_bytes_per_sync: 0 2024-04-16T16:54:14.237 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:13 smithi045 ceph-mon[31676]: rocksdb: Options.strict_bytes_per_sync: 0 2024-04-16T16:54:14.237 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:13 smithi045 ceph-mon[31676]: rocksdb: Options.compaction_readahead_size: 0 2024-04-16T16:54:14.237 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:13 smithi045 ceph-mon[31676]: rocksdb: Options.max_background_flushes: -1 2024-04-16T16:54:14.237 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:13 smithi045 ceph-mon[31676]: rocksdb: Compression algorithms supported: 2024-04-16T16:54:14.237 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:13 smithi045 ceph-mon[31676]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-04-16T16:54:14.237 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:13 smithi045 ceph-mon[31676]: rocksdb: kZSTD supported: 0 2024-04-16T16:54:14.237 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:13 smithi045 ceph-mon[31676]: rocksdb: kXpressCompression supported: 0 2024-04-16T16:54:14.237 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:13 smithi045 ceph-mon[31676]: rocksdb: kLZ4HCCompression supported: 1 2024-04-16T16:54:14.237 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:13 smithi045 ceph-mon[31676]: rocksdb: kZlibCompression supported: 1 2024-04-16T16:54:14.237 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:13 smithi045 ceph-mon[31676]: rocksdb: kSnappyCompression supported: 1 2024-04-16T16:54:14.238 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:13 smithi045 ceph-mon[31676]: rocksdb: kLZ4Compression supported: 1 2024-04-16T16:54:14.239 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:13 smithi045 ceph-mon[31676]: rocksdb: kBZip2Compression supported: 0 2024-04-16T16:54:14.239 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:13 smithi045 ceph-mon[31676]: rocksdb: Fast CRC32 supported: Supported on x86 2024-04-16T16:54:14.239 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:13 smithi045 ceph-mon[31676]: rocksdb: DMutex implementation: pthread_mutex_t 2024-04-16T16:54:14.239 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:13 smithi045 ceph-mon[31676]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-04-16T16:54:14.239 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:13 smithi045 ceph-mon[31676]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-04-16T16:54:14.239 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:13 smithi045 ceph-mon[31676]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-04-16T16:54:14.239 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:13 smithi045 ceph-mon[31676]: rocksdb: Options.merge_operator: 2024-04-16T16:54:14.239 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:13 smithi045 ceph-mon[31676]: rocksdb: Options.compaction_filter: None 2024-04-16T16:54:14.239 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:13 smithi045 ceph-mon[31676]: rocksdb: Options.compaction_filter_factory: None 2024-04-16T16:54:14.239 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:13 smithi045 ceph-mon[31676]: rocksdb: Options.sst_partitioner_factory: None 2024-04-16T16:54:14.239 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:13 smithi045 ceph-mon[31676]: rocksdb: Options.memtable_factory: SkipListFactory 2024-04-16T16:54:14.239 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:13 smithi045 ceph-mon[31676]: rocksdb: Options.table_factory: BlockBasedTable 2024-04-16T16:54:14.239 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:13 smithi045 ceph-mon[31676]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x5572a9cd7e20) 2024-04-16T16:54:14.240 INFO:journalctl@ceph.mon.a.smithi045.stdout: cache_index_and_filter_blocks: 1 2024-04-16T16:54:14.240 INFO:journalctl@ceph.mon.a.smithi045.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-04-16T16:54:14.240 INFO:journalctl@ceph.mon.a.smithi045.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-04-16T16:54:14.240 INFO:journalctl@ceph.mon.a.smithi045.stdout: pin_top_level_index_and_filter: 1 2024-04-16T16:54:14.240 INFO:journalctl@ceph.mon.a.smithi045.stdout: index_type: 0 2024-04-16T16:54:14.240 INFO:journalctl@ceph.mon.a.smithi045.stdout: data_block_index_type: 0 2024-04-16T16:54:14.240 INFO:journalctl@ceph.mon.a.smithi045.stdout: index_shortening: 1 2024-04-16T16:54:14.240 INFO:journalctl@ceph.mon.a.smithi045.stdout: data_block_hash_table_util_ratio: 0.750000 2024-04-16T16:54:14.240 INFO:journalctl@ceph.mon.a.smithi045.stdout: checksum: 4 2024-04-16T16:54:14.240 INFO:journalctl@ceph.mon.a.smithi045.stdout: no_block_cache: 0 2024-04-16T16:54:14.240 INFO:journalctl@ceph.mon.a.smithi045.stdout: block_cache: 0x5572a9cd9090 2024-04-16T16:54:14.240 INFO:journalctl@ceph.mon.a.smithi045.stdout: block_cache_name: BinnedLRUCache 2024-04-16T16:54:14.240 INFO:journalctl@ceph.mon.a.smithi045.stdout: block_cache_options: 2024-04-16T16:54:14.240 INFO:journalctl@ceph.mon.a.smithi045.stdout: capacity : 536870912 2024-04-16T16:54:14.240 INFO:journalctl@ceph.mon.a.smithi045.stdout: num_shard_bits : 4 2024-04-16T16:54:14.240 INFO:journalctl@ceph.mon.a.smithi045.stdout: strict_capacity_limit : 0 2024-04-16T16:54:14.241 INFO:journalctl@ceph.mon.a.smithi045.stdout: high_pri_pool_ratio: 0.000 2024-04-16T16:54:14.241 INFO:journalctl@ceph.mon.a.smithi045.stdout: block_cache_compressed: (nil) 2024-04-16T16:54:14.241 INFO:journalctl@ceph.mon.a.smithi045.stdout: persistent_cache: (nil) 2024-04-16T16:54:14.241 INFO:journalctl@ceph.mon.a.smithi045.stdout: block_size: 4096 2024-04-16T16:54:14.241 INFO:journalctl@ceph.mon.a.smithi045.stdout: block_size_deviation: 10 2024-04-16T16:54:14.241 INFO:journalctl@ceph.mon.a.smithi045.stdout: block_restart_interval: 16 2024-04-16T16:54:14.241 INFO:journalctl@ceph.mon.a.smithi045.stdout: index_block_restart_interval: 1 2024-04-16T16:54:14.241 INFO:journalctl@ceph.mon.a.smithi045.stdout: metadata_block_size: 4096 2024-04-16T16:54:14.241 INFO:journalctl@ceph.mon.a.smithi045.stdout: partition_filters: 0 2024-04-16T16:54:14.241 INFO:journalctl@ceph.mon.a.smithi045.stdout: use_delta_encoding: 1 2024-04-16T16:54:14.241 INFO:journalctl@ceph.mon.a.smithi045.stdout: filter_policy: bloomfilter 2024-04-16T16:54:14.241 INFO:journalctl@ceph.mon.a.smithi045.stdout: whole_key_filtering: 1 2024-04-16T16:54:14.241 INFO:journalctl@ceph.mon.a.smithi045.stdout: verify_compression: 0 2024-04-16T16:54:14.241 INFO:journalctl@ceph.mon.a.smithi045.stdout: read_amp_bytes_per_bit: 0 2024-04-16T16:54:14.241 INFO:journalctl@ceph.mon.a.smithi045.stdout: format_version: 5 2024-04-16T16:54:14.242 INFO:journalctl@ceph.mon.a.smithi045.stdout: enable_index_compression: 1 2024-04-16T16:54:14.242 INFO:journalctl@ceph.mon.a.smithi045.stdout: block_align: 0 2024-04-16T16:54:14.242 INFO:journalctl@ceph.mon.a.smithi045.stdout: max_auto_readahead_size: 262144 2024-04-16T16:54:14.242 INFO:journalctl@ceph.mon.a.smithi045.stdout: prepopulate_block_cache: 0 2024-04-16T16:54:14.242 INFO:journalctl@ceph.mon.a.smithi045.stdout: initial_auto_readahead_size: 8192 2024-04-16T16:54:14.242 INFO:journalctl@ceph.mon.a.smithi045.stdout: num_file_reads_for_auto_readahead: 2 2024-04-16T16:54:14.242 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:13 smithi045 ceph-mon[31676]: rocksdb: Options.write_buffer_size: 33554432 2024-04-16T16:54:14.242 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:13 smithi045 ceph-mon[31676]: rocksdb: Options.max_write_buffer_number: 2 2024-04-16T16:54:14.242 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:13 smithi045 ceph-mon[31676]: rocksdb: Options.compression: NoCompression 2024-04-16T16:54:14.242 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:13 smithi045 ceph-mon[31676]: rocksdb: Options.bottommost_compression: Disabled 2024-04-16T16:54:14.242 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:13 smithi045 ceph-mon[31676]: rocksdb: Options.prefix_extractor: nullptr 2024-04-16T16:54:14.242 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:13 smithi045 ceph-mon[31676]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-04-16T16:54:14.242 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:13 smithi045 ceph-mon[31676]: rocksdb: Options.num_levels: 7 2024-04-16T16:54:14.242 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:13 smithi045 ceph-mon[31676]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-04-16T16:54:14.242 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:13 smithi045 ceph-mon[31676]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-04-16T16:54:14.243 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:13 smithi045 ceph-mon[31676]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-04-16T16:54:14.243 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:13 smithi045 ceph-mon[31676]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-04-16T16:54:14.243 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:13 smithi045 ceph-mon[31676]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-04-16T16:54:14.243 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:13 smithi045 ceph-mon[31676]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-04-16T16:54:14.243 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:13 smithi045 ceph-mon[31676]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-04-16T16:54:14.243 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:13 smithi045 ceph-mon[31676]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-04-16T16:54:14.243 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:13 smithi045 ceph-mon[31676]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-04-16T16:54:14.243 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:13 smithi045 ceph-mon[31676]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-04-16T16:54:14.243 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:13 smithi045 ceph-mon[31676]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-04-16T16:54:14.243 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:13 smithi045 ceph-mon[31676]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-04-16T16:54:14.243 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:13 smithi045 ceph-mon[31676]: rocksdb: Options.compression_opts.window_bits: -14 2024-04-16T16:54:14.243 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:13 smithi045 ceph-mon[31676]: rocksdb: Options.compression_opts.level: 32767 2024-04-16T16:54:14.243 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:13 smithi045 ceph-mon[31676]: rocksdb: Options.compression_opts.strategy: 0 2024-04-16T16:54:14.243 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:13 smithi045 ceph-mon[31676]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-04-16T16:54:14.243 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:13 smithi045 ceph-mon[31676]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-04-16T16:54:14.244 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:13 smithi045 ceph-mon[31676]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-04-16T16:54:14.244 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:13 smithi045 ceph-mon[31676]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-04-16T16:54:14.244 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:13 smithi045 ceph-mon[31676]: rocksdb: Options.compression_opts.enabled: false 2024-04-16T16:54:14.244 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:13 smithi045 ceph-mon[31676]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-04-16T16:54:14.244 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:13 smithi045 ceph-mon[31676]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-04-16T16:54:14.244 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:13 smithi045 ceph-mon[31676]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-04-16T16:54:14.244 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:13 smithi045 ceph-mon[31676]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-04-16T16:54:14.244 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:13 smithi045 ceph-mon[31676]: rocksdb: Options.target_file_size_base: 67108864 2024-04-16T16:54:14.244 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:13 smithi045 ceph-mon[31676]: rocksdb: Options.target_file_size_multiplier: 1 2024-04-16T16:54:14.244 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:13 smithi045 ceph-mon[31676]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-04-16T16:54:14.244 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:13 smithi045 ceph-mon[31676]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-04-16T16:54:14.244 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:13 smithi045 ceph-mon[31676]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-04-16T16:54:14.244 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:13 smithi045 ceph-mon[31676]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-04-16T16:54:14.244 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:13 smithi045 ceph-mon[31676]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-04-16T16:54:14.244 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:13 smithi045 ceph-mon[31676]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-04-16T16:54:14.245 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:13 smithi045 ceph-mon[31676]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-04-16T16:54:14.245 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:13 smithi045 ceph-mon[31676]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-04-16T16:54:14.245 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:13 smithi045 ceph-mon[31676]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-04-16T16:54:14.245 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:13 smithi045 ceph-mon[31676]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-04-16T16:54:14.245 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:13 smithi045 ceph-mon[31676]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-04-16T16:54:14.245 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:13 smithi045 ceph-mon[31676]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-04-16T16:54:14.245 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:13 smithi045 ceph-mon[31676]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-04-16T16:54:14.245 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:13 smithi045 ceph-mon[31676]: rocksdb: Options.arena_block_size: 1048576 2024-04-16T16:54:14.245 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:13 smithi045 ceph-mon[31676]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-04-16T16:54:14.245 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:13 smithi045 ceph-mon[31676]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-04-16T16:54:14.245 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:13 smithi045 ceph-mon[31676]: rocksdb: Options.disable_auto_compactions: 0 2024-04-16T16:54:14.245 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:13 smithi045 ceph-mon[31676]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-04-16T16:54:14.245 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:13 smithi045 ceph-mon[31676]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-04-16T16:54:14.245 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:13 smithi045 ceph-mon[31676]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-04-16T16:54:14.245 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:13 smithi045 ceph-mon[31676]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-04-16T16:54:14.246 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:13 smithi045 ceph-mon[31676]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-04-16T16:54:14.246 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:13 smithi045 ceph-mon[31676]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-04-16T16:54:14.246 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:13 smithi045 ceph-mon[31676]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-04-16T16:54:14.246 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:13 smithi045 ceph-mon[31676]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-04-16T16:54:14.246 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:13 smithi045 ceph-mon[31676]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-04-16T16:54:14.246 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:13 smithi045 ceph-mon[31676]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-04-16T16:54:14.246 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:13 smithi045 ceph-mon[31676]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-04-16T16:54:14.246 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:13 smithi045 ceph-mon[31676]: rocksdb: Options.inplace_update_support: 0 2024-04-16T16:54:14.246 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:13 smithi045 ceph-mon[31676]: rocksdb: Options.inplace_update_num_locks: 10000 2024-04-16T16:54:14.246 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:13 smithi045 ceph-mon[31676]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-04-16T16:54:14.246 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:13 smithi045 ceph-mon[31676]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-04-16T16:54:14.246 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:13 smithi045 ceph-mon[31676]: rocksdb: Options.memtable_huge_page_size: 0 2024-04-16T16:54:14.246 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:13 smithi045 ceph-mon[31676]: rocksdb: Options.bloom_locality: 0 2024-04-16T16:54:14.246 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:13 smithi045 ceph-mon[31676]: rocksdb: Options.max_successive_merges: 0 2024-04-16T16:54:14.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:13 smithi045 ceph-mon[31676]: rocksdb: Options.optimize_filters_for_hits: 0 2024-04-16T16:54:14.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:13 smithi045 ceph-mon[31676]: rocksdb: Options.paranoid_file_checks: 0 2024-04-16T16:54:14.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:13 smithi045 ceph-mon[31676]: rocksdb: Options.force_consistency_checks: 1 2024-04-16T16:54:14.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:13 smithi045 ceph-mon[31676]: rocksdb: Options.report_bg_io_stats: 0 2024-04-16T16:54:14.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:13 smithi045 ceph-mon[31676]: rocksdb: Options.ttl: 2592000 2024-04-16T16:54:14.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:13 smithi045 ceph-mon[31676]: rocksdb: Options.periodic_compaction_seconds: 0 2024-04-16T16:54:14.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:13 smithi045 ceph-mon[31676]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-04-16T16:54:14.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:13 smithi045 ceph-mon[31676]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-04-16T16:54:14.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:13 smithi045 ceph-mon[31676]: rocksdb: Options.enable_blob_files: false 2024-04-16T16:54:14.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:13 smithi045 ceph-mon[31676]: rocksdb: Options.min_blob_size: 0 2024-04-16T16:54:14.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:13 smithi045 ceph-mon[31676]: rocksdb: Options.blob_file_size: 268435456 2024-04-16T16:54:14.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:13 smithi045 ceph-mon[31676]: rocksdb: Options.blob_compression_type: NoCompression 2024-04-16T16:54:14.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:13 smithi045 ceph-mon[31676]: rocksdb: Options.enable_blob_garbage_collection: false 2024-04-16T16:54:14.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:13 smithi045 ceph-mon[31676]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-04-16T16:54:14.247 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:13 smithi045 ceph-mon[31676]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-04-16T16:54:14.248 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:13 smithi045 ceph-mon[31676]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-04-16T16:54:14.248 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:13 smithi045 ceph-mon[31676]: rocksdb: Options.blob_file_starting_level: 0 2024-04-16T16:54:14.248 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:13 smithi045 ceph-mon[31676]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-04-16T16:54:14.248 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:13 smithi045 ceph-mon[31676]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-04-16T16:54:14.248 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:13 smithi045 ceph-mon[31676]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-04-16T16:54:14.248 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:13 smithi045 ceph-mon[31676]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 5c5ca1b1-fb39-4bf5-afd0-deb5528cef1e 2024-04-16T16:54:14.248 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:13 smithi045 ceph-mon[31676]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713286453917027, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-04-16T16:54:14.248 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:13 smithi045 ceph-mon[31676]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-04-16T16:54:14.248 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:13 smithi045 ceph-mon[31676]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713286453919905, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 100208, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 326, "table_properties": {"data_size": 98096, "index_size": 304, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 773, "raw_key_size": 14165, "raw_average_key_size": 49, "raw_value_size": 90430, "raw_average_value_size": 317, "num_data_blocks": 12, "num_entries": 285, "num_filter_entries": 285, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1713286453, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "5c5ca1b1-fb39-4bf5-afd0-deb5528cef1e", "db_session_id": "KZ4FSGY6F4B3SSUNGX87", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-04-16T16:54:14.248 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:13 smithi045 ceph-mon[31676]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713286453920125, "job": 1, "event": "recovery_finished"} 2024-04-16T16:54:14.248 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:13 smithi045 ceph-mon[31676]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-04-16T16:54:14.248 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:13 smithi045 ceph-mon[31676]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-04-16T16:54:14.248 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:13 smithi045 ceph-mon[31676]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x5572a9dc6000 2024-04-16T16:54:14.248 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:13 smithi045 ceph-mon[31676]: rocksdb: DB pointer 0x5572a9db0000 2024-04-16T16:54:14.249 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:13 smithi045 ceph-mon[31676]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-04-16T16:54:14.249 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:13 smithi045 ceph-mon[31676]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-04-16T16:54:14.249 INFO:journalctl@ceph.mon.a.smithi045.stdout: ** DB Stats ** 2024-04-16T16:54:14.249 INFO:journalctl@ceph.mon.a.smithi045.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-16T16:54:14.249 INFO:journalctl@ceph.mon.a.smithi045.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-04-16T16:54:14.249 INFO:journalctl@ceph.mon.a.smithi045.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-16T16:54:14.249 INFO:journalctl@ceph.mon.a.smithi045.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-16T16:54:14.249 INFO:journalctl@ceph.mon.a.smithi045.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-04-16T16:54:14.249 INFO:journalctl@ceph.mon.a.smithi045.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-16T16:54:14.249 INFO:journalctl@ceph.mon.a.smithi045.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-16T16:54:14.249 INFO:journalctl@ceph.mon.a.smithi045.stdout: 2024-04-16T16:54:14.249 INFO:journalctl@ceph.mon.a.smithi045.stdout: ** Compaction Stats [default] ** 2024-04-16T16:54:14.249 INFO:journalctl@ceph.mon.a.smithi045.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-16T16:54:14.249 INFO:journalctl@ceph.mon.a.smithi045.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-04-16T16:54:14.249 INFO:journalctl@ceph.mon.a.smithi045.stdout: L0 2/0 99.72 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 52.4 0.00 0.00 1 0.002 0 0 0.0 0.0 2024-04-16T16:54:14.250 INFO:journalctl@ceph.mon.a.smithi045.stdout: Sum 2/0 99.72 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 52.4 0.00 0.00 1 0.002 0 0 0.0 0.0 2024-04-16T16:54:14.250 INFO:journalctl@ceph.mon.a.smithi045.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 52.4 0.00 0.00 1 0.002 0 0 0.0 0.0 2024-04-16T16:54:14.250 INFO:journalctl@ceph.mon.a.smithi045.stdout: 2024-04-16T16:54:14.250 INFO:journalctl@ceph.mon.a.smithi045.stdout: ** Compaction Stats [default] ** 2024-04-16T16:54:14.250 INFO:journalctl@ceph.mon.a.smithi045.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-16T16:54:14.250 INFO:journalctl@ceph.mon.a.smithi045.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-16T16:54:14.250 INFO:journalctl@ceph.mon.a.smithi045.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 52.4 0.00 0.00 1 0.002 0 0 0.0 0.0 2024-04-16T16:54:14.250 INFO:journalctl@ceph.mon.a.smithi045.stdout: 2024-04-16T16:54:14.250 INFO:journalctl@ceph.mon.a.smithi045.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-04-16T16:54:14.250 INFO:journalctl@ceph.mon.a.smithi045.stdout: 2024-04-16T16:54:14.250 INFO:journalctl@ceph.mon.a.smithi045.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-16T16:54:14.250 INFO:journalctl@ceph.mon.a.smithi045.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-04-16T16:54:14.250 INFO:journalctl@ceph.mon.a.smithi045.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-04-16T16:54:14.250 INFO:journalctl@ceph.mon.a.smithi045.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-04-16T16:54:14.251 INFO:journalctl@ceph.mon.a.smithi045.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-04-16T16:54:14.251 INFO:journalctl@ceph.mon.a.smithi045.stdout: AddFile(Keys): cumulative 0, interval 0 2024-04-16T16:54:14.251 INFO:journalctl@ceph.mon.a.smithi045.stdout: Cumulative compaction: 0.00 GB write, 10.88 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-16T16:54:14.251 INFO:journalctl@ceph.mon.a.smithi045.stdout: Interval compaction: 0.00 GB write, 10.88 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-16T16:54:14.251 INFO:journalctl@ceph.mon.a.smithi045.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-04-16T16:54:14.251 INFO:journalctl@ceph.mon.a.smithi045.stdout: Block cache BinnedLRUCache@0x5572a9cd9090#2 capacity: 512.00 MB usage: 1.52 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 2.2e-05 secs_since: 0 2024-04-16T16:54:14.251 INFO:journalctl@ceph.mon.a.smithi045.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,1.02 KB,0.000193715%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-04-16T16:54:14.251 INFO:journalctl@ceph.mon.a.smithi045.stdout: 2024-04-16T16:54:14.251 INFO:journalctl@ceph.mon.a.smithi045.stdout: ** File Read Latency Histogram By Level [default] ** 2024-04-16T16:54:14.251 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:13 smithi045 ceph-mon[31676]: starting mon.a rank 0 at public addrs [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] at bind addrs [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid b1fecd38-fc11-11ee-bc90-c7b262605968 2024-04-16T16:54:14.251 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:13 smithi045 ceph-mon[31676]: mon.a@-1(???) e1 preinit fsid b1fecd38-fc11-11ee-bc90-c7b262605968 2024-04-16T16:54:14.251 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:13 smithi045 ceph-mon[31676]: mon.a@-1(???).mds e1 new map 2024-04-16T16:54:14.251 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:13 smithi045 ceph-mon[31676]: mon.a@-1(???).mds e1 print_map 2024-04-16T16:54:14.251 INFO:journalctl@ceph.mon.a.smithi045.stdout: e1 2024-04-16T16:54:14.251 INFO:journalctl@ceph.mon.a.smithi045.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-04-16T16:54:14.251 INFO:journalctl@ceph.mon.a.smithi045.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-16T16:54:14.252 INFO:journalctl@ceph.mon.a.smithi045.stdout: legacy client fscid: -1 2024-04-16T16:54:14.252 INFO:journalctl@ceph.mon.a.smithi045.stdout: 2024-04-16T16:54:14.252 INFO:journalctl@ceph.mon.a.smithi045.stdout: No filesystems configured 2024-04-16T16:54:14.252 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:13 smithi045 ceph-mon[31676]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-04-16T16:54:14.252 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:13 smithi045 ceph-mon[31676]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-16T16:54:14.252 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:13 smithi045 ceph-mon[31676]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-16T16:54:14.252 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:13 smithi045 ceph-mon[31676]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-16T16:54:14.252 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:13 smithi045 ceph-mon[31676]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-04-16T16:54:14.252 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:13 smithi045 ceph-mon[31676]: mon.a is new leader, mons a in quorum (ranks 0) 2024-04-16T16:54:14.252 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:13 smithi045 ceph-mon[31676]: monmap epoch 1 2024-04-16T16:54:14.252 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:13 smithi045 ceph-mon[31676]: fsid b1fecd38-fc11-11ee-bc90-c7b262605968 2024-04-16T16:54:14.252 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:13 smithi045 ceph-mon[31676]: last_changed 2024-04-16T16:53:58.140859+0000 2024-04-16T16:54:14.252 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:13 smithi045 ceph-mon[31676]: created 2024-04-16T16:53:58.140859+0000 2024-04-16T16:54:14.252 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:13 smithi045 ceph-mon[31676]: min_mon_release 19 (squid) 2024-04-16T16:54:14.252 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:13 smithi045 ceph-mon[31676]: election_strategy: 1 2024-04-16T16:54:14.252 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:13 smithi045 ceph-mon[31676]: 0: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.a 2024-04-16T16:54:14.253 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:13 smithi045 ceph-mon[31676]: fsmap 2024-04-16T16:54:14.253 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:13 smithi045 ceph-mon[31676]: osdmap e1: 0 total, 0 up, 0 in 2024-04-16T16:54:14.253 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:13 smithi045 ceph-mon[31676]: mgrmap e1: no daemons active 2024-04-16T16:54:16.848 INFO:teuthology.orchestra.run.smithi045.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-04-16T16:54:17.147 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:16 smithi045 bash[31658]: 5e03aa10df484d017679f33bd15f25d228119ead2a392d47d4e2705506ca956f 2024-04-16T16:54:17.148 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:16 smithi045 systemd[1]: Started Ceph mon.a for b1fecd38-fc11-11ee-bc90-c7b262605968. 2024-04-16T16:54:18.355 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:17.491+0000 7fe4ad400700 1 Processor -- start 2024-04-16T16:54:18.355 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:17.492+0000 7fe4ad400700 1 -- start start 2024-04-16T16:54:18.355 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:17.492+0000 7fe4ad400700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe4a810f060 0x7fe4a810f480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:54:18.355 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:17.492+0000 7fe4ad400700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fe4a810fa50 con 0x7fe4a810f060 2024-04-16T16:54:18.355 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:17.493+0000 7fe4a6800700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe4a810f060 0x7fe4a810f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:54:18.356 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:17.493+0000 7fe4a6800700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe4a810f060 0x7fe4a810f480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:48410/0 (socket says 172.21.15.45:48410) 2024-04-16T16:54:18.356 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:17.493+0000 7fe4a6800700 1 -- 172.21.15.45:0/4034633926 learned_addr learned my addr 172.21.15.45:0/4034633926 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T16:54:18.356 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:17.494+0000 7fe4a6800700 1 -- 172.21.15.45:0/4034633926 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4a810fde0 con 0x7fe4a810f060 2024-04-16T16:54:18.356 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:17.494+0000 7fe4a6800700 1 --2- 172.21.15.45:0/4034633926 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe4a810f060 0x7fe4a810f480 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fe4980098a0 tx=0x7fe498009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ceafd21b3759f14d server_cookie=0 in_seq=0 out_seq=0 2024-04-16T16:54:18.356 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:17.494+0000 7fe4a5400700 1 -- 172.21.15.45:0/4034633926 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe498004160 con 0x7fe4a810f060 2024-04-16T16:54:18.356 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:17.494+0000 7fe4a5400700 1 -- 172.21.15.45:0/4034633926 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fe498004300 con 0x7fe4a810f060 2024-04-16T16:54:18.356 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:17.494+0000 7fe4a5400700 1 -- 172.21.15.45:0/4034633926 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe498011b50 con 0x7fe4a810f060 2024-04-16T16:54:18.356 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:17.495+0000 7fe4ad400700 1 -- 172.21.15.45:0/4034633926 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe4a810f060 msgr2=0x7fe4a810f480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:54:18.356 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:17.495+0000 7fe4ad400700 1 --2- 172.21.15.45:0/4034633926 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe4a810f060 0x7fe4a810f480 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fe4980098a0 tx=0x7fe498009bb0 comp rx=0 tx=0).stop 2024-04-16T16:54:18.356 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:17.495+0000 7fe4ad400700 1 -- 172.21.15.45:0/4034633926 shutdown_connections 2024-04-16T16:54:18.356 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:17.495+0000 7fe4ad400700 1 --2- 172.21.15.45:0/4034633926 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe4a810f060 0x7fe4a810f480 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:54:18.356 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:17.495+0000 7fe4ad400700 1 -- 172.21.15.45:0/4034633926 >> 172.21.15.45:0/4034633926 conn(0x7fe4a807de10 msgr2=0x7fe4a8080270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T16:54:18.357 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:17.495+0000 7fe4ad400700 1 -- 172.21.15.45:0/4034633926 shutdown_connections 2024-04-16T16:54:18.357 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:17.495+0000 7fe4ad400700 1 -- 172.21.15.45:0/4034633926 wait complete. 2024-04-16T16:54:18.357 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:17.496+0000 7fe4ad400700 1 Processor -- start 2024-04-16T16:54:18.357 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:17.497+0000 7fe4ad400700 1 -- start start 2024-04-16T16:54:18.357 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:17.497+0000 7fe4ad400700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe4a810f060 0x7fe4a81a2b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:54:18.357 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:17.497+0000 7fe4ad400700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fe4a8113190 con 0x7fe4a810f060 2024-04-16T16:54:18.357 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:17.497+0000 7fe4a6800700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe4a810f060 0x7fe4a81a2b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:54:18.357 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:17.497+0000 7fe4a6800700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe4a810f060 0x7fe4a81a2b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:48426/0 (socket says 172.21.15.45:48426) 2024-04-16T16:54:18.357 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:17.497+0000 7fe4a6800700 1 -- 172.21.15.45:0/3375458960 learned_addr learned my addr 172.21.15.45:0/3375458960 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T16:54:18.357 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:17.498+0000 7fe4a6800700 1 -- 172.21.15.45:0/3375458960 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4a81a30b0 con 0x7fe4a810f060 2024-04-16T16:54:18.357 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:17.498+0000 7fe4a6800700 1 --2- 172.21.15.45:0/3375458960 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe4a810f060 0x7fe4a81a2b70 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fe498011850 tx=0x7fe498011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T16:54:18.357 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:17.498+0000 7fe49f400700 1 -- 172.21.15.45:0/3375458960 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe498003af0 con 0x7fe4a810f060 2024-04-16T16:54:18.358 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:17.499+0000 7fe49f400700 1 -- 172.21.15.45:0/3375458960 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fe498020070 con 0x7fe4a810f060 2024-04-16T16:54:18.358 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:17.499+0000 7fe49f400700 1 -- 172.21.15.45:0/3375458960 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe4980195a0 con 0x7fe4a810f060 2024-04-16T16:54:18.358 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:17.499+0000 7fe4ad400700 1 -- 172.21.15.45:0/3375458960 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4a81a3340 con 0x7fe4a810f060 2024-04-16T16:54:18.358 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:17.499+0000 7fe4ad400700 1 -- 172.21.15.45:0/3375458960 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4a81a6030 con 0x7fe4a810f060 2024-04-16T16:54:18.358 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:17.499+0000 7fe49f400700 1 -- 172.21.15.45:0/3375458960 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fe498019be0 con 0x7fe4a810f060 2024-04-16T16:54:18.358 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:17.499+0000 7fe49f400700 1 -- 172.21.15.45:0/3375458960 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fe498011e00 con 0x7fe4a810f060 2024-04-16T16:54:18.358 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:17.500+0000 7fe4ad400700 1 -- 172.21.15.45:0/3375458960 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4a8064150 con 0x7fe4a810f060 2024-04-16T16:54:18.358 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:17.503+0000 7fe49f400700 1 -- 172.21.15.45:0/3375458960 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7fe49801a770 con 0x7fe4a810f060 2024-04-16T16:54:18.358 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:17.582+0000 7fe4ad400700 1 -- 172.21.15.45:0/3375458960 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) -- 0x7fe4a81a6200 con 0x7fe4a810f060 2024-04-16T16:54:18.358 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:17.585+0000 7fe49f400700 1 -- 172.21.15.45:0/3375458960 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) ==== 124+0+0 (secure 0 0 0) 0x7fe49801a950 con 0x7fe4a810f060 2024-04-16T16:54:18.358 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:17.586+0000 7fe4ad400700 1 -- 172.21.15.45:0/3375458960 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe4a810f060 msgr2=0x7fe4a81a2b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:54:18.359 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:17.586+0000 7fe4ad400700 1 --2- 172.21.15.45:0/3375458960 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe4a810f060 0x7fe4a81a2b70 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fe498011850 tx=0x7fe498011c10 comp rx=0 tx=0).stop 2024-04-16T16:54:18.359 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:17.586+0000 7fe4ad400700 1 -- 172.21.15.45:0/3375458960 shutdown_connections 2024-04-16T16:54:18.359 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:17.586+0000 7fe4ad400700 1 --2- 172.21.15.45:0/3375458960 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe4a810f060 0x7fe4a81a2b70 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:54:18.359 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:17.586+0000 7fe4ad400700 1 -- 172.21.15.45:0/3375458960 >> 172.21.15.45:0/3375458960 conn(0x7fe4a807de10 msgr2=0x7fe4a807fa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T16:54:18.359 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:17.586+0000 7fe4ad400700 1 -- 172.21.15.45:0/3375458960 shutdown_connections 2024-04-16T16:54:18.359 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:17.586+0000 7fe4ad400700 1 -- 172.21.15.45:0/3375458960 wait complete. 2024-04-16T16:54:18.359 INFO:teuthology.orchestra.run.smithi045.stdout:Wrote config to /etc/ceph/ceph.conf 2024-04-16T16:54:18.382 INFO:teuthology.orchestra.run.smithi045.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-04-16T16:54:18.382 INFO:teuthology.orchestra.run.smithi045.stdout:Creating mgr... 2024-04-16T16:54:18.383 INFO:teuthology.orchestra.run.smithi045.stdout:Verifying port 0.0.0.0:9283 ... 2024-04-16T16:54:18.384 INFO:teuthology.orchestra.run.smithi045.stdout:Verifying port 0.0.0.0:8765 ... 2024-04-16T16:54:18.852 INFO:teuthology.orchestra.run.smithi045.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-b1fecd38-fc11-11ee-bc90-c7b262605968@mgr.x 2024-04-16T16:54:18.852 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stderr Failed to reset failed state of unit ceph-b1fecd38-fc11-11ee-bc90-c7b262605968@mgr.x.service: Unit ceph-b1fecd38-fc11-11ee-bc90-c7b262605968@mgr.x.service not loaded. 2024-04-16T16:54:18.872 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:18 smithi045 ceph-mon[31676]: from='client.? 172.21.15.45:0/3375458960' entity='client.admin' 2024-04-16T16:54:19.123 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-b1fecd38-fc11-11ee-bc90-c7b262605968.target.wants/ceph-b1fecd38-fc11-11ee-bc90-c7b262605968@mgr.x.service → /etc/systemd/system/ceph-b1fecd38-fc11-11ee-bc90-c7b262605968@.service. 2024-04-16T16:54:19.478 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 16 16:54:19 smithi045 systemd[1]: Starting Ceph mgr.x for b1fecd38-fc11-11ee-bc90-c7b262605968... 2024-04-16T16:54:19.804 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 16 16:54:19 smithi045 podman[31888]: 2024-04-16 16:54:19.415089062 +0000 UTC m=+0.035454918 image pull e5ce10bdffbeeffffd1125f67b369e2151f0f17cdec854e93b6adf9321f65907 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45 2024-04-16T16:54:19.804 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 16 16:54:19 smithi045 podman[31888]: 2024-04-16 16:54:19.517946217 +0000 UTC m=+0.138312112 container create 9b59d149dd293fab06b5d2104c31d27b9ae666e74f598d5ac2c35f8f89b6fdfe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45, name=ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-x, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240409, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-04-16T16:54:19.804 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 16 16:54:19 smithi045 podman[31888]: 2024-04-16 16:54:19.738097659 +0000 UTC m=+0.358463580 container init 9b59d149dd293fab06b5d2104c31d27b9ae666e74f598d5ac2c35f8f89b6fdfe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45, name=ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-x, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240409, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb) 2024-04-16T16:54:19.804 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 16 16:54:19 smithi045 podman[31888]: 2024-04-16 16:54:19.748952409 +0000 UTC m=+0.369318256 container start 9b59d149dd293fab06b5d2104c31d27b9ae666e74f598d5ac2c35f8f89b6fdfe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45, name=ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-x, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.build-date=20240409, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD) 2024-04-16T16:54:19.804 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 16 16:54:19 smithi045 bash[31888]: 9b59d149dd293fab06b5d2104c31d27b9ae666e74f598d5ac2c35f8f89b6fdfe 2024-04-16T16:54:19.836 INFO:teuthology.orchestra.run.smithi045.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-16T16:54:19.836 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stdout disabled 2024-04-16T16:54:19.854 INFO:teuthology.orchestra.run.smithi045.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-16T16:54:19.855 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stdout inactive 2024-04-16T16:54:19.855 INFO:teuthology.orchestra.run.smithi045.stdout:firewalld.service is not enabled 2024-04-16T16:54:19.855 INFO:teuthology.orchestra.run.smithi045.stdout:Not possible to enable service . firewalld.service is not available 2024-04-16T16:54:19.877 INFO:teuthology.orchestra.run.smithi045.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-16T16:54:19.878 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stdout disabled 2024-04-16T16:54:19.895 INFO:teuthology.orchestra.run.smithi045.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-16T16:54:19.896 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stdout inactive 2024-04-16T16:54:19.896 INFO:teuthology.orchestra.run.smithi045.stdout:firewalld.service is not enabled 2024-04-16T16:54:19.896 INFO:teuthology.orchestra.run.smithi045.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-04-16T16:54:19.896 INFO:teuthology.orchestra.run.smithi045.stdout:Waiting for mgr to start... 2024-04-16T16:54:19.896 INFO:teuthology.orchestra.run.smithi045.stdout:Waiting for mgr... 2024-04-16T16:54:20.062 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 16 16:54:19 smithi045 systemd[1]: Started Ceph mgr.x for b1fecd38-fc11-11ee-bc90-c7b262605968. 2024-04-16T16:54:20.616 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 16 16:54:20 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-x[31905]: 2024-04-16T16:54:20.530+0000 7fd7c2a0d200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-16T16:54:20.973 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:20 smithi045 ceph-mon[31676]: from='client.? 172.21.15.45:0/3573024193' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-16T16:54:20.973 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 16 16:54:20 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-x[31905]: 2024-04-16T16:54:20.663+0000 7fd7c2a0d200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-16T16:54:21.314 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout 2024-04-16T16:54:21.314 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout { 2024-04-16T16:54:21.314 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "fsid": "b1fecd38-fc11-11ee-bc90-c7b262605968", 2024-04-16T16:54:21.315 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "health": { 2024-04-16T16:54:21.315 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-16T16:54:21.315 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-16T16:54:21.315 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-16T16:54:21.315 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2024-04-16T16:54:21.315 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-16T16:54:21.315 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-16T16:54:21.315 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout 0 2024-04-16T16:54:21.315 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout ], 2024-04-16T16:54:21.315 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-16T16:54:21.315 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "a" 2024-04-16T16:54:21.315 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout ], 2024-04-16T16:54:21.315 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "quorum_age": 6, 2024-04-16T16:54:21.315 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-16T16:54:21.316 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-16T16:54:21.316 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-16T16:54:21.316 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-16T16:54:21.316 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2024-04-16T16:54:21.316 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-16T16:54:21.316 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-16T16:54:21.316 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-16T16:54:21.316 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-16T16:54:21.316 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-16T16:54:21.316 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-16T16:54:21.316 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-16T16:54:21.316 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-16T16:54:21.316 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2024-04-16T16:54:21.316 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-16T16:54:21.317 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-16T16:54:21.317 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-16T16:54:21.317 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-16T16:54:21.317 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-16T16:54:21.317 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-16T16:54:21.317 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-16T16:54:21.317 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-16T16:54:21.317 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-16T16:54:21.317 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2024-04-16T16:54:21.317 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-16T16:54:21.317 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-16T16:54:21.317 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-16T16:54:21.317 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-16T16:54:21.317 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2024-04-16T16:54:21.317 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-16T16:54:21.318 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-16T16:54:21.318 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-16T16:54:21.318 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-16T16:54:21.318 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-16T16:54:21.318 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-16T16:54:21.318 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "restful" 2024-04-16T16:54:21.318 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout ], 2024-04-16T16:54:21.318 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-16T16:54:21.318 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2024-04-16T16:54:21.318 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-16T16:54:21.318 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-16T16:54:21.318 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "modified": "2024-04-16T16:54:02.386471+0000", 2024-04-16T16:54:21.318 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-16T16:54:21.318 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2024-04-16T16:54:21.319 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-16T16:54:21.319 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout } 2024-04-16T16:54:21.319 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:20.516+0000 7f0557800700 1 Processor -- start 2024-04-16T16:54:21.319 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:20.517+0000 7f0557800700 1 -- start start 2024-04-16T16:54:21.319 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:20.517+0000 7f0557800700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f05501074c0 0x7f05501078e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:54:21.319 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:20.517+0000 7f0557800700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f0550107eb0 con 0x7f05501074c0 2024-04-16T16:54:21.319 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:20.518+0000 7f0554e00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f05501074c0 0x7f05501078e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:54:21.319 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:20.518+0000 7f0554e00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f05501074c0 0x7f05501078e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35194/0 (socket says 172.21.15.45:35194) 2024-04-16T16:54:21.319 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:20.518+0000 7f0554e00700 1 -- 172.21.15.45:0/3676243530 learned_addr learned my addr 172.21.15.45:0/3676243530 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T16:54:21.319 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:20.518+0000 7f0554e00700 1 -- 172.21.15.45:0/3676243530 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05501086c0 con 0x7f05501074c0 2024-04-16T16:54:21.319 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:20.518+0000 7f0554e00700 1 --2- 172.21.15.45:0/3676243530 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f05501074c0 0x7f05501078e0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f054400bb40 tx=0x7f054400be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea552f3a53ce1744 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T16:54:21.319 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:20.519+0000 7f054f400700 1 -- 172.21.15.45:0/3676243530 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f054400dbe0 con 0x7f05501074c0 2024-04-16T16:54:21.319 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:20.519+0000 7f054f400700 1 -- 172.21.15.45:0/3676243530 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f05440044f0 con 0x7f05501074c0 2024-04-16T16:54:21.319 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:20.519+0000 7f054f400700 1 -- 172.21.15.45:0/3676243530 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0544010470 con 0x7f05501074c0 2024-04-16T16:54:21.320 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:20.519+0000 7f0557800700 1 -- 172.21.15.45:0/3676243530 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f05501074c0 msgr2=0x7f05501078e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:54:21.320 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:20.519+0000 7f0557800700 1 --2- 172.21.15.45:0/3676243530 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f05501074c0 0x7f05501078e0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f054400bb40 tx=0x7f054400be50 comp rx=0 tx=0).stop 2024-04-16T16:54:21.320 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:20.520+0000 7f0557800700 1 -- 172.21.15.45:0/3676243530 shutdown_connections 2024-04-16T16:54:21.320 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:20.520+0000 7f0557800700 1 --2- 172.21.15.45:0/3676243530 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f05501074c0 0x7f05501078e0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:54:21.320 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:20.520+0000 7f0557800700 1 -- 172.21.15.45:0/3676243530 >> 172.21.15.45:0/3676243530 conn(0x7f0550102a60 msgr2=0x7f0550104ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T16:54:21.320 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:20.520+0000 7f0557800700 1 -- 172.21.15.45:0/3676243530 shutdown_connections 2024-04-16T16:54:21.320 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:20.520+0000 7f0557800700 1 -- 172.21.15.45:0/3676243530 wait complete. 2024-04-16T16:54:21.320 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:20.521+0000 7f0557800700 1 Processor -- start 2024-04-16T16:54:21.320 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:20.522+0000 7f0557800700 1 -- start start 2024-04-16T16:54:21.320 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:20.522+0000 7f0557800700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f05501074c0 0x7f055019e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:54:21.320 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:20.522+0000 7f0557800700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f055007afd0 con 0x7f05501074c0 2024-04-16T16:54:21.320 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:20.522+0000 7f0554e00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f05501074c0 0x7f055019e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:54:21.320 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:20.522+0000 7f0554e00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f05501074c0 0x7f055019e7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35200/0 (socket says 172.21.15.45:35200) 2024-04-16T16:54:21.320 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:20.522+0000 7f0554e00700 1 -- 172.21.15.45:0/3573024193 learned_addr learned my addr 172.21.15.45:0/3573024193 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T16:54:21.321 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:20.523+0000 7f0554e00700 1 -- 172.21.15.45:0/3573024193 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f055019ed10 con 0x7f05501074c0 2024-04-16T16:54:21.321 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:20.523+0000 7f0554e00700 1 --2- 172.21.15.45:0/3573024193 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f05501074c0 0x7f055019e7d0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f054400b340 tx=0x7f054400dd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T16:54:21.321 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:20.523+0000 7f054d600700 1 -- 172.21.15.45:0/3573024193 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0544010680 con 0x7f05501074c0 2024-04-16T16:54:21.321 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:20.523+0000 7f054d600700 1 -- 172.21.15.45:0/3573024193 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f054401d070 con 0x7f05501074c0 2024-04-16T16:54:21.321 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:20.523+0000 7f054d600700 1 -- 172.21.15.45:0/3573024193 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0544021b10 con 0x7f05501074c0 2024-04-16T16:54:21.321 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:20.524+0000 7f0557800700 1 -- 172.21.15.45:0/3573024193 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f055019efa0 con 0x7f05501074c0 2024-04-16T16:54:21.321 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:20.525+0000 7f0557800700 1 -- 172.21.15.45:0/3573024193 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05501a1c40 con 0x7f05501074c0 2024-04-16T16:54:21.321 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:20.525+0000 7f054d600700 1 -- 172.21.15.45:0/3573024193 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f0544028020 con 0x7f05501074c0 2024-04-16T16:54:21.321 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:20.525+0000 7f054d600700 1 -- 172.21.15.45:0/3573024193 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f054402a3f0 con 0x7f05501074c0 2024-04-16T16:54:21.321 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:20.526+0000 7f0542a00700 1 -- 172.21.15.45:0/3573024193 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f052c005330 con 0x7f05501074c0 2024-04-16T16:54:21.321 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:20.531+0000 7f054d600700 1 -- 172.21.15.45:0/3573024193 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f054402f030 con 0x7f05501074c0 2024-04-16T16:54:21.321 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:20.611+0000 7f0542a00700 1 -- 172.21.15.45:0/3573024193 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f052c0058f0 con 0x7f05501074c0 2024-04-16T16:54:21.321 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:20.612+0000 7f054d600700 1 -- 172.21.15.45:0/3573024193 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f0544015070 con 0x7f05501074c0 2024-04-16T16:54:21.321 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:20.613+0000 7f0542a00700 1 -- 172.21.15.45:0/3573024193 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f05501074c0 msgr2=0x7f055019e7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:54:21.322 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:20.613+0000 7f0542a00700 1 --2- 172.21.15.45:0/3573024193 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f05501074c0 0x7f055019e7d0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f054400b340 tx=0x7f054400dd90 comp rx=0 tx=0).stop 2024-04-16T16:54:21.322 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:20.613+0000 7f0542a00700 1 -- 172.21.15.45:0/3573024193 shutdown_connections 2024-04-16T16:54:21.322 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:20.613+0000 7f0542a00700 1 --2- 172.21.15.45:0/3573024193 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f05501074c0 0x7f055019e7d0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:54:21.322 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:20.613+0000 7f0542a00700 1 -- 172.21.15.45:0/3573024193 >> 172.21.15.45:0/3573024193 conn(0x7f0550102a60 msgr2=0x7f0550194df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T16:54:21.322 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:20.613+0000 7f0542a00700 1 -- 172.21.15.45:0/3573024193 shutdown_connections 2024-04-16T16:54:21.322 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:20.613+0000 7f0542a00700 1 -- 172.21.15.45:0/3573024193 wait complete. 2024-04-16T16:54:21.322 INFO:teuthology.orchestra.run.smithi045.stdout:mgr not available, waiting (1/15)... 2024-04-16T16:54:21.656 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 16 16:54:21 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-x[31905]: 2024-04-16T16:54:21.519+0000 7fd7c2a0d200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-16T16:54:21.977 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 16 16:54:21 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-x[31905]: 2024-04-16T16:54:21.653+0000 7fd7c2a0d200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-16T16:54:21.977 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 16 16:54:21 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-x[31905]: 2024-04-16T16:54:21.843+0000 7fd7c2a0d200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-16T16:54:21.977 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 16 16:54:21 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-x[31905]: 2024-04-16T16:54:21.974+0000 7fd7c2a0d200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-16T16:54:22.228 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 16 16:54:22 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-x[31905]: 2024-04-16T16:54:22.088+0000 7fd7c2a0d200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-16T16:54:22.705 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 16 16:54:22 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-x[31905]: 2024-04-16T16:54:22.440+0000 7fd7c2a0d200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-16T16:54:22.705 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 16 16:54:22 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-x[31905]: 2024-04-16T16:54:22.586+0000 7fd7c2a0d200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-16T16:54:22.978 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 16 16:54:22 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-x[31905]: 2024-04-16T16:54:22.702+0000 7fd7c2a0d200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-16T16:54:22.978 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 16 16:54:22 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-x[31905]: 2024-04-16T16:54:22.830+0000 7fd7c2a0d200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-16T16:54:23.478 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 16 16:54:23 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-x[31905]: 2024-04-16T16:54:23.053+0000 7fd7c2a0d200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-16T16:54:24.406 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:24 smithi045 ceph-mon[31676]: from='client.? 172.21.15.45:0/1566998965' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-16T16:54:24.728 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 16 16:54:24 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-x[31905]: 2024-04-16T16:54:24.452+0000 7fd7c2a0d200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-16T16:54:24.728 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 16 16:54:24 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-x[31905]: 2024-04-16T16:54:24.632+0000 7fd7c2a0d200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-16T16:54:24.762 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout 2024-04-16T16:54:24.766 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout { 2024-04-16T16:54:24.766 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "fsid": "b1fecd38-fc11-11ee-bc90-c7b262605968", 2024-04-16T16:54:24.766 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "health": { 2024-04-16T16:54:24.766 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-16T16:54:24.766 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-16T16:54:24.766 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-16T16:54:24.766 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2024-04-16T16:54:24.767 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-16T16:54:24.767 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-16T16:54:24.767 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout 0 2024-04-16T16:54:24.767 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout ], 2024-04-16T16:54:24.767 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-16T16:54:24.767 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "a" 2024-04-16T16:54:24.767 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout ], 2024-04-16T16:54:24.767 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "quorum_age": 10, 2024-04-16T16:54:24.767 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-16T16:54:24.767 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-16T16:54:24.767 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-16T16:54:24.768 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-16T16:54:24.768 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2024-04-16T16:54:24.768 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-16T16:54:24.768 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-16T16:54:24.768 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-16T16:54:24.768 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-16T16:54:24.768 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-16T16:54:24.768 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-16T16:54:24.768 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-16T16:54:24.768 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-16T16:54:24.768 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2024-04-16T16:54:24.769 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-16T16:54:24.769 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-16T16:54:24.769 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-16T16:54:24.769 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-16T16:54:24.777 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-16T16:54:24.777 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-16T16:54:24.777 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-16T16:54:24.777 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-16T16:54:24.777 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-16T16:54:24.778 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2024-04-16T16:54:24.778 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-16T16:54:24.778 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-16T16:54:24.778 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-16T16:54:24.778 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-16T16:54:24.778 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2024-04-16T16:54:24.778 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-16T16:54:24.778 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-16T16:54:24.778 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-16T16:54:24.778 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-16T16:54:24.778 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-16T16:54:24.778 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-16T16:54:24.778 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "restful" 2024-04-16T16:54:24.779 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout ], 2024-04-16T16:54:24.779 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-16T16:54:24.779 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2024-04-16T16:54:24.779 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-16T16:54:24.779 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-16T16:54:24.779 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "modified": "2024-04-16T16:54:02.386471+0000", 2024-04-16T16:54:24.779 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-16T16:54:24.779 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2024-04-16T16:54:24.779 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-16T16:54:24.779 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout } 2024-04-16T16:54:24.779 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:23.922+0000 7f6084200700 1 Processor -- start 2024-04-16T16:54:24.779 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:23.922+0000 7f6084200700 1 -- start start 2024-04-16T16:54:24.780 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:23.923+0000 7f6084200700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f607c07de30 0x7f607c07c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:54:24.780 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:23.923+0000 7f6084200700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f607c07c820 con 0x7f607c07de30 2024-04-16T16:54:24.780 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:23.923+0000 7f6081800700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f607c07de30 0x7f607c07c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:54:24.780 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:23.923+0000 7f6081800700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f607c07de30 0x7f607c07c2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35208/0 (socket says 172.21.15.45:35208) 2024-04-16T16:54:24.780 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:23.923+0000 7f6081800700 1 -- 172.21.15.45:0/2318029906 learned_addr learned my addr 172.21.15.45:0/2318029906 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T16:54:24.780 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:23.924+0000 7f6081800700 1 -- 172.21.15.45:0/2318029906 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f607c07c9a0 con 0x7f607c07de30 2024-04-16T16:54:24.780 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:23.924+0000 7f6081800700 1 --2- 172.21.15.45:0/2318029906 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f607c07de30 0x7f607c07c2e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f6070009e80 tx=0x7f607000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23076f576e205b60 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T16:54:24.780 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:23.924+0000 7f607be00700 1 -- 172.21.15.45:0/2318029906 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6070017070 con 0x7f607c07de30 2024-04-16T16:54:24.780 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:23.925+0000 7f607be00700 1 -- 172.21.15.45:0/2318029906 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f607000b8c0 con 0x7f607c07de30 2024-04-16T16:54:24.780 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:23.925+0000 7f607be00700 1 -- 172.21.15.45:0/2318029906 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f60700045c0 con 0x7f607c07de30 2024-04-16T16:54:24.780 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:23.925+0000 7f6084200700 1 -- 172.21.15.45:0/2318029906 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f607c07de30 msgr2=0x7f607c07c2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:54:24.780 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:23.925+0000 7f6084200700 1 --2- 172.21.15.45:0/2318029906 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f607c07de30 0x7f607c07c2e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f6070009e80 tx=0x7f607000b330 comp rx=0 tx=0).stop 2024-04-16T16:54:24.780 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:23.925+0000 7f6084200700 1 -- 172.21.15.45:0/2318029906 shutdown_connections 2024-04-16T16:54:24.781 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:23.925+0000 7f6084200700 1 --2- 172.21.15.45:0/2318029906 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f607c07de30 0x7f607c07c2e0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:54:24.781 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:23.925+0000 7f6084200700 1 -- 172.21.15.45:0/2318029906 >> 172.21.15.45:0/2318029906 conn(0x7f607c103b90 msgr2=0x7f607c105fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T16:54:24.781 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:23.925+0000 7f6084200700 1 -- 172.21.15.45:0/2318029906 shutdown_connections 2024-04-16T16:54:24.781 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:23.926+0000 7f6084200700 1 -- 172.21.15.45:0/2318029906 wait complete. 2024-04-16T16:54:24.781 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:23.927+0000 7f6084200700 1 Processor -- start 2024-04-16T16:54:24.781 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:23.927+0000 7f6084200700 1 -- start start 2024-04-16T16:54:24.781 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:23.928+0000 7f6084200700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f607c07de30 0x7f607c19e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:54:24.781 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:23.928+0000 7f6084200700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f607c10eba0 con 0x7f607c07de30 2024-04-16T16:54:24.781 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:23.928+0000 7f6081800700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f607c07de30 0x7f607c19e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:54:24.781 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:23.928+0000 7f6081800700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f607c07de30 0x7f607c19e6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35216/0 (socket says 172.21.15.45:35216) 2024-04-16T16:54:24.782 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:23.928+0000 7f6081800700 1 -- 172.21.15.45:0/1566998965 learned_addr learned my addr 172.21.15.45:0/1566998965 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T16:54:24.782 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:23.928+0000 7f6081800700 1 -- 172.21.15.45:0/1566998965 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f607c19ec10 con 0x7f607c07de30 2024-04-16T16:54:24.782 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:23.929+0000 7f6081800700 1 --2- 172.21.15.45:0/1566998965 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f607c07de30 0x7f607c19e6d0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f6070004350 tx=0x7f607000bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T16:54:24.782 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:23.929+0000 7f607a000700 1 -- 172.21.15.45:0/1566998965 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6070020020 con 0x7f607c07de30 2024-04-16T16:54:24.782 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:23.929+0000 7f607a000700 1 -- 172.21.15.45:0/1566998965 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f6070012430 con 0x7f607c07de30 2024-04-16T16:54:24.782 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:23.929+0000 7f607a000700 1 -- 172.21.15.45:0/1566998965 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6070017040 con 0x7f607c07de30 2024-04-16T16:54:24.782 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:23.929+0000 7f6084200700 1 -- 172.21.15.45:0/1566998965 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f607c19eea0 con 0x7f607c07de30 2024-04-16T16:54:24.782 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:23.929+0000 7f6084200700 1 -- 172.21.15.45:0/1566998965 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f607c1a1b90 con 0x7f607c07de30 2024-04-16T16:54:24.782 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:23.929+0000 7f607a000700 1 -- 172.21.15.45:0/1566998965 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f6070023020 con 0x7f607c07de30 2024-04-16T16:54:24.783 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:23.930+0000 7f607a000700 1 -- 172.21.15.45:0/1566998965 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f607000f050 con 0x7f607c07de30 2024-04-16T16:54:24.783 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:23.930+0000 7f6084200700 1 -- 172.21.15.45:0/1566998965 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f607c064150 con 0x7f607c07de30 2024-04-16T16:54:24.783 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:23.935+0000 7f607a000700 1 -- 172.21.15.45:0/1566998965 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f607002a020 con 0x7f607c07de30 2024-04-16T16:54:24.783 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:24.015+0000 7f6084200700 1 -- 172.21.15.45:0/1566998965 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f607c1a2050 con 0x7f607c07de30 2024-04-16T16:54:24.783 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:24.016+0000 7f607a000700 1 -- 172.21.15.45:0/1566998965 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1240 (secure 0 0 0) 0x7f607000b860 con 0x7f607c07de30 2024-04-16T16:54:24.783 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:24.018+0000 7f6084200700 1 -- 172.21.15.45:0/1566998965 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f607c07de30 msgr2=0x7f607c19e6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:54:24.783 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:24.018+0000 7f6084200700 1 --2- 172.21.15.45:0/1566998965 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f607c07de30 0x7f607c19e6d0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f6070004350 tx=0x7f607000bf80 comp rx=0 tx=0).stop 2024-04-16T16:54:24.783 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:24.018+0000 7f6084200700 1 -- 172.21.15.45:0/1566998965 shutdown_connections 2024-04-16T16:54:24.783 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:24.018+0000 7f6084200700 1 --2- 172.21.15.45:0/1566998965 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f607c07de30 0x7f607c19e6d0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:54:24.783 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:24.018+0000 7f6084200700 1 -- 172.21.15.45:0/1566998965 >> 172.21.15.45:0/1566998965 conn(0x7f607c103b90 msgr2=0x7f607c1046b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T16:54:24.784 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:24.018+0000 7f6084200700 1 -- 172.21.15.45:0/1566998965 shutdown_connections 2024-04-16T16:54:24.784 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:24.018+0000 7f6084200700 1 -- 172.21.15.45:0/1566998965 wait complete. 2024-04-16T16:54:24.784 INFO:teuthology.orchestra.run.smithi045.stdout:mgr not available, waiting (2/15)... 2024-04-16T16:54:25.453 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 16 16:54:25 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-x[31905]: 2024-04-16T16:54:25.064+0000 7fd7c2a0d200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-16T16:54:25.453 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 16 16:54:25 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-x[31905]: 2024-04-16T16:54:25.187+0000 7fd7c2a0d200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-16T16:54:25.453 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 16 16:54:25 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-x[31905]: 2024-04-16T16:54:25.451+0000 7fd7c2a0d200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-16T16:54:25.978 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 16 16:54:25 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-x[31905]: 2024-04-16T16:54:25.704+0000 7fd7c2a0d200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-16T16:54:26.426 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 16 16:54:26 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-x[31905]: 2024-04-16T16:54:26.095+0000 7fd7c2a0d200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-16T16:54:26.728 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 16 16:54:26 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-x[31905]: 2024-04-16T16:54:26.424+0000 7fd7c2a0d200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-16T16:54:27.087 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 16 16:54:26 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-x[31905]: 2024-04-16T16:54:26.741+0000 7fd7c2a0d200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-16T16:54:27.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:27 smithi045 ceph-mon[31676]: from='client.? 172.21.15.45:0/2646982671' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-16T16:54:28.148 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 16 16:54:27 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-x[31905]: 2024-04-16T16:54:27.851+0000 7fd7c2a0d200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-16T16:54:28.166 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout 2024-04-16T16:54:28.166 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout { 2024-04-16T16:54:28.166 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "fsid": "b1fecd38-fc11-11ee-bc90-c7b262605968", 2024-04-16T16:54:28.166 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "health": { 2024-04-16T16:54:28.166 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-16T16:54:28.166 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-16T16:54:28.166 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-16T16:54:28.167 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2024-04-16T16:54:28.167 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-16T16:54:28.167 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-16T16:54:28.167 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout 0 2024-04-16T16:54:28.167 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout ], 2024-04-16T16:54:28.167 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-16T16:54:28.167 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "a" 2024-04-16T16:54:28.167 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout ], 2024-04-16T16:54:28.167 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "quorum_age": 13, 2024-04-16T16:54:28.167 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-16T16:54:28.167 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-16T16:54:28.167 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-16T16:54:28.167 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-16T16:54:28.168 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2024-04-16T16:54:28.168 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-16T16:54:28.168 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-16T16:54:28.168 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-16T16:54:28.168 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-16T16:54:28.176 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-16T16:54:28.176 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-16T16:54:28.176 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-16T16:54:28.176 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-16T16:54:28.176 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2024-04-16T16:54:28.177 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-16T16:54:28.177 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-16T16:54:28.177 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-16T16:54:28.177 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-16T16:54:28.177 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-16T16:54:28.177 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-16T16:54:28.177 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-16T16:54:28.177 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-16T16:54:28.177 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-16T16:54:28.177 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2024-04-16T16:54:28.177 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-16T16:54:28.177 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-16T16:54:28.177 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-16T16:54:28.177 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-16T16:54:28.178 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2024-04-16T16:54:28.178 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-16T16:54:28.178 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-16T16:54:28.178 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-16T16:54:28.178 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-16T16:54:28.178 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-16T16:54:28.178 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-16T16:54:28.178 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "restful" 2024-04-16T16:54:28.178 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout ], 2024-04-16T16:54:28.178 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-16T16:54:28.178 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2024-04-16T16:54:28.178 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-16T16:54:28.178 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-16T16:54:28.178 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "modified": "2024-04-16T16:54:02.386471+0000", 2024-04-16T16:54:28.179 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-16T16:54:28.179 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2024-04-16T16:54:28.179 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-16T16:54:28.179 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout } 2024-04-16T16:54:28.179 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:27.357+0000 7f603e000700 1 Processor -- start 2024-04-16T16:54:28.179 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:27.357+0000 7f603e000700 1 -- start start 2024-04-16T16:54:28.179 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:27.357+0000 7f603e000700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f60381085d0 0x7f60381089f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:54:28.179 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:27.357+0000 7f603e000700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f6038108fc0 con 0x7f60381085d0 2024-04-16T16:54:28.179 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:27.358+0000 7f6037400700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f60381085d0 0x7f60381089f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:54:28.179 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:27.358+0000 7f6037400700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f60381085d0 0x7f60381089f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35228/0 (socket says 172.21.15.45:35228) 2024-04-16T16:54:28.179 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:27.358+0000 7f6037400700 1 -- 172.21.15.45:0/748688277 learned_addr learned my addr 172.21.15.45:0/748688277 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T16:54:28.179 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:27.358+0000 7f6037400700 1 -- 172.21.15.45:0/748688277 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6038109780 con 0x7f60381085d0 2024-04-16T16:54:28.179 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:27.359+0000 7f6037400700 1 --2- 172.21.15.45:0/748688277 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f60381085d0 0x7f60381089f0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f60280098a0 tx=0x7f6028009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4db6483d3da96ef1 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T16:54:28.179 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:27.359+0000 7f6036000700 1 -- 172.21.15.45:0/748688277 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6028004160 con 0x7f60381085d0 2024-04-16T16:54:28.180 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:27.359+0000 7f6036000700 1 -- 172.21.15.45:0/748688277 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f6028004300 con 0x7f60381085d0 2024-04-16T16:54:28.180 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:27.359+0000 7f6036000700 1 -- 172.21.15.45:0/748688277 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6028011b50 con 0x7f60381085d0 2024-04-16T16:54:28.180 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:27.360+0000 7f603e000700 1 -- 172.21.15.45:0/748688277 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f60381085d0 msgr2=0x7f60381089f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:54:28.180 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:27.360+0000 7f603e000700 1 --2- 172.21.15.45:0/748688277 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f60381085d0 0x7f60381089f0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f60280098a0 tx=0x7f6028009bb0 comp rx=0 tx=0).stop 2024-04-16T16:54:28.180 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:27.360+0000 7f603e000700 1 -- 172.21.15.45:0/748688277 shutdown_connections 2024-04-16T16:54:28.180 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:27.360+0000 7f603e000700 1 --2- 172.21.15.45:0/748688277 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f60381085d0 0x7f60381089f0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:54:28.180 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:27.360+0000 7f603e000700 1 -- 172.21.15.45:0/748688277 >> 172.21.15.45:0/748688277 conn(0x7f6038103b90 msgr2=0x7f6038105fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T16:54:28.180 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:27.360+0000 7f603e000700 1 -- 172.21.15.45:0/748688277 shutdown_connections 2024-04-16T16:54:28.180 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:27.360+0000 7f603e000700 1 -- 172.21.15.45:0/748688277 wait complete. 2024-04-16T16:54:28.180 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:27.361+0000 7f603e000700 1 Processor -- start 2024-04-16T16:54:28.180 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:27.361+0000 7f603e000700 1 -- start start 2024-04-16T16:54:28.180 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:27.361+0000 7f603e000700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f60381085d0 0x7f603819c550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:54:28.180 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:27.361+0000 7f603e000700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f603810cb30 con 0x7f60381085d0 2024-04-16T16:54:28.180 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:27.362+0000 7f6037400700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f60381085d0 0x7f603819c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:54:28.181 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:27.362+0000 7f6037400700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f60381085d0 0x7f603819c550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35240/0 (socket says 172.21.15.45:35240) 2024-04-16T16:54:28.181 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:27.362+0000 7f6037400700 1 -- 172.21.15.45:0/2646982671 learned_addr learned my addr 172.21.15.45:0/2646982671 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T16:54:28.181 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:27.362+0000 7f6037400700 1 -- 172.21.15.45:0/2646982671 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f603819ca90 con 0x7f60381085d0 2024-04-16T16:54:28.181 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:27.362+0000 7f6037400700 1 --2- 172.21.15.45:0/2646982671 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f60381085d0 0x7f603819c550 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f6028000c00 tx=0x7f6028011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T16:54:28.181 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:27.363+0000 7f602fe00700 1 -- 172.21.15.45:0/2646982671 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6028003b20 con 0x7f60381085d0 2024-04-16T16:54:28.181 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:27.363+0000 7f602fe00700 1 -- 172.21.15.45:0/2646982671 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f6028003cc0 con 0x7f60381085d0 2024-04-16T16:54:28.181 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:27.363+0000 7f602fe00700 1 -- 172.21.15.45:0/2646982671 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6028024910 con 0x7f60381085d0 2024-04-16T16:54:28.181 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:27.363+0000 7f603e000700 1 -- 172.21.15.45:0/2646982671 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f603819cd20 con 0x7f60381085d0 2024-04-16T16:54:28.181 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:27.363+0000 7f603e000700 1 -- 172.21.15.45:0/2646982671 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f603819fa10 con 0x7f60381085d0 2024-04-16T16:54:28.181 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:27.363+0000 7f602fe00700 1 -- 172.21.15.45:0/2646982671 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f6028022020 con 0x7f60381085d0 2024-04-16T16:54:28.181 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:27.363+0000 7f602fe00700 1 -- 172.21.15.45:0/2646982671 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f6028011e00 con 0x7f60381085d0 2024-04-16T16:54:28.182 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:27.364+0000 7f603e000700 1 -- 172.21.15.45:0/2646982671 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6038064150 con 0x7f60381085d0 2024-04-16T16:54:28.182 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:27.367+0000 7f602fe00700 1 -- 172.21.15.45:0/2646982671 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f602801acd0 con 0x7f60381085d0 2024-04-16T16:54:28.182 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:27.448+0000 7f603e000700 1 -- 172.21.15.45:0/2646982671 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f603819fbe0 con 0x7f60381085d0 2024-04-16T16:54:28.182 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:27.449+0000 7f602fe00700 1 -- 172.21.15.45:0/2646982671 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1240 (secure 0 0 0) 0x7f6028032030 con 0x7f60381085d0 2024-04-16T16:54:28.182 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:27.450+0000 7f603e000700 1 -- 172.21.15.45:0/2646982671 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f60381085d0 msgr2=0x7f603819c550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:54:28.182 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:27.450+0000 7f603e000700 1 --2- 172.21.15.45:0/2646982671 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f60381085d0 0x7f603819c550 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f6028000c00 tx=0x7f6028011c10 comp rx=0 tx=0).stop 2024-04-16T16:54:28.182 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:27.450+0000 7f603e000700 1 -- 172.21.15.45:0/2646982671 shutdown_connections 2024-04-16T16:54:28.182 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:27.450+0000 7f603e000700 1 --2- 172.21.15.45:0/2646982671 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f60381085d0 0x7f603819c550 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:54:28.182 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:27.450+0000 7f603e000700 1 -- 172.21.15.45:0/2646982671 >> 172.21.15.45:0/2646982671 conn(0x7f6038103b90 msgr2=0x7f6038104720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T16:54:28.182 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:27.450+0000 7f603e000700 1 -- 172.21.15.45:0/2646982671 shutdown_connections 2024-04-16T16:54:28.182 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:27.450+0000 7f603e000700 1 -- 172.21.15.45:0/2646982671 wait complete. 2024-04-16T16:54:28.182 INFO:teuthology.orchestra.run.smithi045.stdout:mgr not available, waiting (3/15)... 2024-04-16T16:54:28.978 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 16 16:54:28 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-x[31905]: 2024-04-16T16:54:28.579+0000 7fd7c2a0d200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-16T16:54:28.978 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 16 16:54:28 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-x[31905]: 2024-04-16T16:54:28.713+0000 7fd7c2a0d200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-16T16:54:29.478 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 16 16:54:29 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-x[31905]: 2024-04-16T16:54:29.007+0000 7fd7c2a0d200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-16T16:54:29.479 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:29 smithi045 ceph-mon[31676]: Activating manager daemon x 2024-04-16T16:54:29.479 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:29 smithi045 ceph-mon[31676]: mgrmap e2: x(active, starting, since 0.00993408s) 2024-04-16T16:54:29.479 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:29 smithi045 ceph-mon[31676]: from='mgr.14100 172.21.15.45:0/1813451034' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-16T16:54:29.479 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:29 smithi045 ceph-mon[31676]: from='mgr.14100 172.21.15.45:0/1813451034' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-16T16:54:29.479 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:29 smithi045 ceph-mon[31676]: from='mgr.14100 172.21.15.45:0/1813451034' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-16T16:54:29.479 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:29 smithi045 ceph-mon[31676]: from='mgr.14100 172.21.15.45:0/1813451034' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-16T16:54:29.479 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:29 smithi045 ceph-mon[31676]: from='mgr.14100 172.21.15.45:0/1813451034' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-16T16:54:29.479 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:29 smithi045 ceph-mon[31676]: from='mgr.14100 172.21.15.45:0/1813451034' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-16T16:54:29.480 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:29 smithi045 ceph-mon[31676]: from='mgr.14100 172.21.15.45:0/1813451034' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-16T16:54:29.480 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:29 smithi045 ceph-mon[31676]: from='mgr.14100 172.21.15.45:0/1813451034' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-16T16:54:29.480 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:29 smithi045 ceph-mon[31676]: Manager daemon x is now available 2024-04-16T16:54:29.480 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:29 smithi045 ceph-mon[31676]: from='mgr.14100 172.21.15.45:0/1813451034' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-16T16:54:29.480 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:29 smithi045 ceph-mon[31676]: from='mgr.14100 172.21.15.45:0/1813451034' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-16T16:54:30.333 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:30 smithi045 ceph-mon[31676]: from='mgr.14100 172.21.15.45:0/1813451034' entity='mgr.x' 2024-04-16T16:54:30.334 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:30 smithi045 ceph-mon[31676]: from='mgr.14100 172.21.15.45:0/1813451034' entity='mgr.x' 2024-04-16T16:54:30.334 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:30 smithi045 ceph-mon[31676]: from='mgr.14100 172.21.15.45:0/1813451034' entity='mgr.x' 2024-04-16T16:54:30.334 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:30 smithi045 ceph-mon[31676]: mgrmap e3: x(active, since 1.01894s) 2024-04-16T16:54:31.771 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout 2024-04-16T16:54:31.771 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout { 2024-04-16T16:54:31.771 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "fsid": "b1fecd38-fc11-11ee-bc90-c7b262605968", 2024-04-16T16:54:31.771 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "health": { 2024-04-16T16:54:31.773 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-16T16:54:31.773 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-16T16:54:31.773 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-16T16:54:31.773 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2024-04-16T16:54:31.773 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-16T16:54:31.773 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-16T16:54:31.773 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout 0 2024-04-16T16:54:31.773 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout ], 2024-04-16T16:54:31.773 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-16T16:54:31.773 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "a" 2024-04-16T16:54:31.774 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout ], 2024-04-16T16:54:31.774 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "quorum_age": 17, 2024-04-16T16:54:31.774 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-16T16:54:31.775 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-16T16:54:31.775 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-16T16:54:31.775 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-16T16:54:31.775 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2024-04-16T16:54:31.775 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-16T16:54:31.775 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-16T16:54:31.776 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-16T16:54:31.776 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-16T16:54:31.776 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-16T16:54:31.776 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-16T16:54:31.780 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-16T16:54:31.780 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-16T16:54:31.780 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2024-04-16T16:54:31.780 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-16T16:54:31.780 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-16T16:54:31.780 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-16T16:54:31.781 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-16T16:54:31.781 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-16T16:54:31.781 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-16T16:54:31.781 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-16T16:54:31.781 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-16T16:54:31.781 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-16T16:54:31.781 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2024-04-16T16:54:31.781 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-16T16:54:31.781 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-16T16:54:31.781 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-16T16:54:31.781 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-16T16:54:31.781 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2024-04-16T16:54:31.782 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-16T16:54:31.782 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-16T16:54:31.782 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-16T16:54:31.782 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-16T16:54:31.782 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-16T16:54:31.782 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-16T16:54:31.782 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "restful" 2024-04-16T16:54:31.782 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout ], 2024-04-16T16:54:31.782 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-16T16:54:31.782 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2024-04-16T16:54:31.782 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-16T16:54:31.783 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-16T16:54:31.783 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "modified": "2024-04-16T16:54:02.386471+0000", 2024-04-16T16:54:31.783 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-16T16:54:31.783 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2024-04-16T16:54:31.783 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-16T16:54:31.783 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout } 2024-04-16T16:54:31.783 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:30.766+0000 7f217c200700 1 Processor -- start 2024-04-16T16:54:31.783 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:30.767+0000 7f217c200700 1 -- start start 2024-04-16T16:54:31.783 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:30.767+0000 7f217c200700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f217407de30 0x7f217407c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:54:31.783 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:30.767+0000 7f217c200700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f217407c820 con 0x7f217407de30 2024-04-16T16:54:31.783 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:30.768+0000 7f2179800700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f217407de30 0x7f217407c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:54:31.783 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:30.768+0000 7f2179800700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f217407de30 0x7f217407c2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49264/0 (socket says 172.21.15.45:49264) 2024-04-16T16:54:31.788 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:30.768+0000 7f2179800700 1 -- 172.21.15.45:0/1686984932 learned_addr learned my addr 172.21.15.45:0/1686984932 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T16:54:31.788 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:30.768+0000 7f2179800700 1 -- 172.21.15.45:0/1686984932 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f217407c9a0 con 0x7f217407de30 2024-04-16T16:54:31.788 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:30.769+0000 7f2179800700 1 --2- 172.21.15.45:0/1686984932 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f217407de30 0x7f217407c2e0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f21680098a0 tx=0x7f2168009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3206f5a385b66260 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T16:54:31.789 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:30.769+0000 7f2173e00700 1 -- 172.21.15.45:0/1686984932 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2168004160 con 0x7f217407de30 2024-04-16T16:54:31.789 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:30.769+0000 7f2173e00700 1 -- 172.21.15.45:0/1686984932 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f2168004300 con 0x7f217407de30 2024-04-16T16:54:31.789 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:30.769+0000 7f2173e00700 1 -- 172.21.15.45:0/1686984932 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2168011b50 con 0x7f217407de30 2024-04-16T16:54:31.789 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:30.770+0000 7f217c200700 1 -- 172.21.15.45:0/1686984932 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f217407de30 msgr2=0x7f217407c2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:54:31.789 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:30.770+0000 7f217c200700 1 --2- 172.21.15.45:0/1686984932 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f217407de30 0x7f217407c2e0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f21680098a0 tx=0x7f2168009bb0 comp rx=0 tx=0).stop 2024-04-16T16:54:31.789 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:30.770+0000 7f217c200700 1 -- 172.21.15.45:0/1686984932 shutdown_connections 2024-04-16T16:54:31.789 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:30.770+0000 7f217c200700 1 --2- 172.21.15.45:0/1686984932 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f217407de30 0x7f217407c2e0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:54:31.789 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:30.770+0000 7f217c200700 1 -- 172.21.15.45:0/1686984932 >> 172.21.15.45:0/1686984932 conn(0x7f2174103b90 msgr2=0x7f2174105fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T16:54:31.789 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:30.770+0000 7f217c200700 1 -- 172.21.15.45:0/1686984932 shutdown_connections 2024-04-16T16:54:31.789 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:30.770+0000 7f217c200700 1 -- 172.21.15.45:0/1686984932 wait complete. 2024-04-16T16:54:31.789 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:30.771+0000 7f217c200700 1 Processor -- start 2024-04-16T16:54:31.789 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:30.771+0000 7f217c200700 1 -- start start 2024-04-16T16:54:31.790 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:30.771+0000 7f217c200700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f217407de30 0x7f217419e730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:54:31.790 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:30.771+0000 7f217c200700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f217410eba0 con 0x7f217407de30 2024-04-16T16:54:31.790 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:30.772+0000 7f2179800700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f217407de30 0x7f217419e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:54:31.790 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:30.772+0000 7f2179800700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f217407de30 0x7f217419e730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49280/0 (socket says 172.21.15.45:49280) 2024-04-16T16:54:31.790 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:30.772+0000 7f2179800700 1 -- 172.21.15.45:0/3373300771 learned_addr learned my addr 172.21.15.45:0/3373300771 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T16:54:31.790 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:30.772+0000 7f2179800700 1 -- 172.21.15.45:0/3373300771 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f217419ec70 con 0x7f217407de30 2024-04-16T16:54:31.790 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:30.772+0000 7f2179800700 1 --2- 172.21.15.45:0/3373300771 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f217407de30 0x7f217419e730 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f2168011670 tx=0x7f2168011b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T16:54:31.790 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:30.773+0000 7f2172000700 1 -- 172.21.15.45:0/3373300771 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f21680039c0 con 0x7f217407de30 2024-04-16T16:54:31.790 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:30.773+0000 7f2172000700 1 -- 172.21.15.45:0/3373300771 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f2168011e50 con 0x7f217407de30 2024-04-16T16:54:31.790 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:30.773+0000 7f217c200700 1 -- 172.21.15.45:0/3373300771 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f217419ef00 con 0x7f217407de30 2024-04-16T16:54:31.790 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:30.773+0000 7f2172000700 1 -- 172.21.15.45:0/3373300771 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f216801a5c0 con 0x7f217407de30 2024-04-16T16:54:31.790 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:30.773+0000 7f217c200700 1 -- 172.21.15.45:0/3373300771 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21741a1bf0 con 0x7f217407de30 2024-04-16T16:54:31.790 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:30.774+0000 7f2172000700 1 -- 172.21.15.45:0/3373300771 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 3) ==== 46441+0+0 (secure 0 0 0) 0x7f2168024430 con 0x7f217407de30 2024-04-16T16:54:31.790 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:30.774+0000 7f217c200700 1 -- 172.21.15.45:0/3373300771 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2174064150 con 0x7f217407de30 2024-04-16T16:54:31.791 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:30.775+0000 7f2172000700 1 --2- 172.21.15.45:0/3373300771 >> [v2:172.21.15.45:6800/2553659622,v1:172.21.15.45:6801/2553659622] conn(0x7f215c039450 0x7f215c03b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:54:31.791 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:30.775+0000 7f2172000700 1 -- 172.21.15.45:0/3373300771 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f216804d780 con 0x7f217407de30 2024-04-16T16:54:31.791 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:30.775+0000 7f2178e00700 1 --2- 172.21.15.45:0/3373300771 >> [v2:172.21.15.45:6800/2553659622,v1:172.21.15.45:6801/2553659622] conn(0x7f215c039450 0x7f215c03b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:54:31.791 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:30.776+0000 7f2178e00700 1 --2- 172.21.15.45:0/3373300771 >> [v2:172.21.15.45:6800/2553659622,v1:172.21.15.45:6801/2553659622] conn(0x7f215c039450 0x7f215c03b910 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f2160006fd0 tx=0x7f2160006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T16:54:31.791 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:30.782+0000 7f2172000700 1 -- 172.21.15.45:0/3373300771 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f216802d3f0 con 0x7f217407de30 2024-04-16T16:54:31.791 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:31.070+0000 7f2172000700 1 -- 172.21.15.45:0/3373300771 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mgrmap(e 4) ==== 46547+0+0 (secure 0 0 0) 0x7f2168024720 con 0x7f217407de30 2024-04-16T16:54:31.791 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:31.102+0000 7f217c200700 1 -- 172.21.15.45:0/3373300771 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f21741a1dc0 con 0x7f217407de30 2024-04-16T16:54:31.791 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:31.103+0000 7f2172000700 1 -- 172.21.15.45:0/3373300771 <== mon.0 v2:172.21.15.45:3300/0 8 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f2168020e70 con 0x7f217407de30 2024-04-16T16:54:31.791 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:31.106+0000 7f217c200700 1 -- 172.21.15.45:0/3373300771 >> [v2:172.21.15.45:6800/2553659622,v1:172.21.15.45:6801/2553659622] conn(0x7f215c039450 msgr2=0x7f215c03b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:54:31.791 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:31.106+0000 7f217c200700 1 --2- 172.21.15.45:0/3373300771 >> [v2:172.21.15.45:6800/2553659622,v1:172.21.15.45:6801/2553659622] conn(0x7f215c039450 0x7f215c03b910 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f2160006fd0 tx=0x7f2160006e40 comp rx=0 tx=0).stop 2024-04-16T16:54:31.791 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:31.106+0000 7f217c200700 1 -- 172.21.15.45:0/3373300771 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f217407de30 msgr2=0x7f217419e730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:54:31.791 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:31.106+0000 7f217c200700 1 --2- 172.21.15.45:0/3373300771 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f217407de30 0x7f217419e730 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f2168011670 tx=0x7f2168011b60 comp rx=0 tx=0).stop 2024-04-16T16:54:31.791 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:31.106+0000 7f217c200700 1 -- 172.21.15.45:0/3373300771 shutdown_connections 2024-04-16T16:54:31.791 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:31.106+0000 7f217c200700 1 --2- 172.21.15.45:0/3373300771 >> [v2:172.21.15.45:6800/2553659622,v1:172.21.15.45:6801/2553659622] conn(0x7f215c039450 0x7f215c03b910 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:54:31.791 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:31.106+0000 7f217c200700 1 --2- 172.21.15.45:0/3373300771 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f217407de30 0x7f217419e730 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:54:31.792 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:31.106+0000 7f217c200700 1 -- 172.21.15.45:0/3373300771 >> 172.21.15.45:0/3373300771 conn(0x7f2174103b90 msgr2=0x7f2174104720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T16:54:31.792 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:31.106+0000 7f217c200700 1 -- 172.21.15.45:0/3373300771 shutdown_connections 2024-04-16T16:54:31.792 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:31.106+0000 7f217c200700 1 -- 172.21.15.45:0/3373300771 wait complete. 2024-04-16T16:54:31.792 INFO:teuthology.orchestra.run.smithi045.stdout:mgr is available 2024-04-16T16:54:32.341 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:32 smithi045 ceph-mon[31676]: mgrmap e4: x(active, since 2s) 2024-04-16T16:54:32.341 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:32 smithi045 ceph-mon[31676]: from='client.? 172.21.15.45:0/3373300771' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-16T16:54:33.427 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:33 smithi045 ceph-mon[31676]: from='client.? 172.21.15.45:0/1115639751' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-04-16T16:54:33.446 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout 2024-04-16T16:54:33.446 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout [global] 2024-04-16T16:54:33.446 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout fsid = b1fecd38-fc11-11ee-bc90-c7b262605968 2024-04-16T16:54:33.446 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-04-16T16:54:33.446 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-04-16T16:54:33.447 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-04-16T16:54:33.447 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-04-16T16:54:33.447 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout 2024-04-16T16:54:33.447 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout [mgr] 2024-04-16T16:54:33.447 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-04-16T16:54:33.447 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout 2024-04-16T16:54:33.447 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout [osd] 2024-04-16T16:54:33.447 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-04-16T16:54:33.447 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-04-16T16:54:33.447 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-04-16T16:54:33.447 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-04-16T16:54:33.447 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-04-16T16:54:33.447 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:32.426+0000 7fbc93200700 1 Processor -- start 2024-04-16T16:54:33.447 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:32.426+0000 7fbc93200700 1 -- start start 2024-04-16T16:54:33.447 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:32.426+0000 7fbc93200700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbc8c10f060 0x7fbc8c10f480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:54:33.448 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:32.426+0000 7fbc93200700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fbc8c10fa50 con 0x7fbc8c10f060 2024-04-16T16:54:33.448 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:32.427+0000 7fbc8be00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbc8c10f060 0x7fbc8c10f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:54:33.448 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:32.427+0000 7fbc8be00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbc8c10f060 0x7fbc8c10f480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49282/0 (socket says 172.21.15.45:49282) 2024-04-16T16:54:33.448 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:32.427+0000 7fbc8be00700 1 -- 172.21.15.45:0/1346461276 learned_addr learned my addr 172.21.15.45:0/1346461276 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T16:54:33.448 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:32.427+0000 7fbc8be00700 1 -- 172.21.15.45:0/1346461276 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc8c10fde0 con 0x7fbc8c10f060 2024-04-16T16:54:33.448 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:32.428+0000 7fbc8be00700 1 --2- 172.21.15.45:0/1346461276 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbc8c10f060 0x7fbc8c10f480 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fbc7c0098a0 tx=0x7fbc7c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf11cbeb7681d977 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T16:54:33.448 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:32.428+0000 7fbc8aa00700 1 -- 172.21.15.45:0/1346461276 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbc7c004160 con 0x7fbc8c10f060 2024-04-16T16:54:33.448 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:32.428+0000 7fbc8aa00700 1 -- 172.21.15.45:0/1346461276 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fbc7c004300 con 0x7fbc8c10f060 2024-04-16T16:54:33.448 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:32.428+0000 7fbc8aa00700 1 -- 172.21.15.45:0/1346461276 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbc7c011b50 con 0x7fbc8c10f060 2024-04-16T16:54:33.448 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:32.429+0000 7fbc93200700 1 -- 172.21.15.45:0/1346461276 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbc8c10f060 msgr2=0x7fbc8c10f480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:54:33.448 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:32.429+0000 7fbc93200700 1 --2- 172.21.15.45:0/1346461276 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbc8c10f060 0x7fbc8c10f480 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fbc7c0098a0 tx=0x7fbc7c009bb0 comp rx=0 tx=0).stop 2024-04-16T16:54:33.448 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:32.429+0000 7fbc93200700 1 -- 172.21.15.45:0/1346461276 shutdown_connections 2024-04-16T16:54:33.448 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:32.429+0000 7fbc93200700 1 --2- 172.21.15.45:0/1346461276 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbc8c10f060 0x7fbc8c10f480 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:54:33.448 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:32.429+0000 7fbc93200700 1 -- 172.21.15.45:0/1346461276 >> 172.21.15.45:0/1346461276 conn(0x7fbc8c07de10 msgr2=0x7fbc8c080270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T16:54:33.448 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:32.429+0000 7fbc93200700 1 -- 172.21.15.45:0/1346461276 shutdown_connections 2024-04-16T16:54:33.449 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:32.429+0000 7fbc93200700 1 -- 172.21.15.45:0/1346461276 wait complete. 2024-04-16T16:54:33.449 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:32.430+0000 7fbc93200700 1 Processor -- start 2024-04-16T16:54:33.449 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:32.430+0000 7fbc93200700 1 -- start start 2024-04-16T16:54:33.449 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:32.431+0000 7fbc93200700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbc8c10f060 0x7fbc8c1a2b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:54:33.449 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:32.431+0000 7fbc93200700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fbc8c113190 con 0x7fbc8c10f060 2024-04-16T16:54:33.449 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:32.431+0000 7fbc8be00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbc8c10f060 0x7fbc8c1a2b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:54:33.449 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:32.431+0000 7fbc8be00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbc8c10f060 0x7fbc8c1a2b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49290/0 (socket says 172.21.15.45:49290) 2024-04-16T16:54:33.449 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:32.431+0000 7fbc8be00700 1 -- 172.21.15.45:0/1115639751 learned_addr learned my addr 172.21.15.45:0/1115639751 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T16:54:33.449 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:32.432+0000 7fbc8be00700 1 -- 172.21.15.45:0/1115639751 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc8c1a30b0 con 0x7fbc8c10f060 2024-04-16T16:54:33.449 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:32.432+0000 7fbc8be00700 1 --2- 172.21.15.45:0/1115639751 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbc8c10f060 0x7fbc8c1a2b70 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fbc7c011850 tx=0x7fbc7c011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T16:54:33.449 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:32.432+0000 7fbc88c00700 1 -- 172.21.15.45:0/1115639751 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbc7c003af0 con 0x7fbc8c10f060 2024-04-16T16:54:33.449 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:32.432+0000 7fbc88c00700 1 -- 172.21.15.45:0/1115639751 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fbc7c020070 con 0x7fbc8c10f060 2024-04-16T16:54:33.449 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:32.432+0000 7fbc93200700 1 -- 172.21.15.45:0/1115639751 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc8c1a3340 con 0x7fbc8c10f060 2024-04-16T16:54:33.449 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:32.432+0000 7fbc88c00700 1 -- 172.21.15.45:0/1115639751 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbc7c0195a0 con 0x7fbc8c10f060 2024-04-16T16:54:33.449 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:32.432+0000 7fbc93200700 1 -- 172.21.15.45:0/1115639751 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc8c1a6030 con 0x7fbc8c10f060 2024-04-16T16:54:33.449 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:32.433+0000 7fbc88c00700 1 -- 172.21.15.45:0/1115639751 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 4) ==== 46547+0+0 (secure 0 0 0) 0x7fbc7c019850 con 0x7fbc8c10f060 2024-04-16T16:54:33.450 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:32.434+0000 7fbc93200700 1 -- 172.21.15.45:0/1115639751 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc8c064150 con 0x7fbc8c10f060 2024-04-16T16:54:33.450 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:32.434+0000 7fbc88c00700 1 --2- 172.21.15.45:0/1115639751 >> [v2:172.21.15.45:6800/2553659622,v1:172.21.15.45:6801/2553659622] conn(0x7fbc700395b0 0x7fbc7003ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:54:33.450 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:32.434+0000 7fbc88c00700 1 -- 172.21.15.45:0/1115639751 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fbc7c011e00 con 0x7fbc8c10f060 2024-04-16T16:54:33.450 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:32.434+0000 7fbc8b400700 1 --2- 172.21.15.45:0/1115639751 >> [v2:172.21.15.45:6800/2553659622,v1:172.21.15.45:6801/2553659622] conn(0x7fbc700395b0 0x7fbc7003ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:54:33.450 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:32.435+0000 7fbc8b400700 1 --2- 172.21.15.45:0/1115639751 >> [v2:172.21.15.45:6800/2553659622,v1:172.21.15.45:6801/2553659622] conn(0x7fbc700395b0 0x7fbc7003ba70 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fbc78006fd0 tx=0x7fbc78006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T16:54:33.450 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:32.440+0000 7fbc88c00700 1 -- 172.21.15.45:0/1115639751 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fbc7c00e260 con 0x7fbc8c10f060 2024-04-16T16:54:33.450 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:32.667+0000 7fbc93200700 1 -- 172.21.15.45:0/1115639751 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7fbc8c1a6200 con 0x7fbc8c10f060 2024-04-16T16:54:33.450 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:32.668+0000 7fbc88c00700 1 -- 172.21.15.45:0/1115639751 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) ==== 70+0+373 (secure 0 0 0) 0x7fbc7c003c90 con 0x7fbc8c10f060 2024-04-16T16:54:33.450 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:32.672+0000 7fbc93200700 1 -- 172.21.15.45:0/1115639751 >> [v2:172.21.15.45:6800/2553659622,v1:172.21.15.45:6801/2553659622] conn(0x7fbc700395b0 msgr2=0x7fbc7003ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:54:33.450 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:32.672+0000 7fbc93200700 1 --2- 172.21.15.45:0/1115639751 >> [v2:172.21.15.45:6800/2553659622,v1:172.21.15.45:6801/2553659622] conn(0x7fbc700395b0 0x7fbc7003ba70 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fbc78006fd0 tx=0x7fbc78006e40 comp rx=0 tx=0).stop 2024-04-16T16:54:33.450 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:32.672+0000 7fbc93200700 1 -- 172.21.15.45:0/1115639751 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbc8c10f060 msgr2=0x7fbc8c1a2b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:54:33.450 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:32.672+0000 7fbc93200700 1 --2- 172.21.15.45:0/1115639751 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbc8c10f060 0x7fbc8c1a2b70 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fbc7c011850 tx=0x7fbc7c011c10 comp rx=0 tx=0).stop 2024-04-16T16:54:33.450 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:32.672+0000 7fbc93200700 1 -- 172.21.15.45:0/1115639751 shutdown_connections 2024-04-16T16:54:33.451 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:32.672+0000 7fbc93200700 1 --2- 172.21.15.45:0/1115639751 >> [v2:172.21.15.45:6800/2553659622,v1:172.21.15.45:6801/2553659622] conn(0x7fbc700395b0 0x7fbc7003ba70 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:54:33.451 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:32.672+0000 7fbc93200700 1 --2- 172.21.15.45:0/1115639751 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbc8c10f060 0x7fbc8c1a2b70 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:54:33.451 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:32.672+0000 7fbc93200700 1 -- 172.21.15.45:0/1115639751 >> 172.21.15.45:0/1115639751 conn(0x7fbc8c07de10 msgr2=0x7fbc8c07fa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T16:54:33.451 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:32.673+0000 7fbc93200700 1 -- 172.21.15.45:0/1115639751 shutdown_connections 2024-04-16T16:54:33.451 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:32.673+0000 7fbc93200700 1 -- 172.21.15.45:0/1115639751 wait complete. 2024-04-16T16:54:33.451 INFO:teuthology.orchestra.run.smithi045.stdout:Enabling cephadm module... 2024-04-16T16:54:34.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:34 smithi045 ceph-mon[31676]: from='client.? 172.21.15.45:0/2071149514' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-04-16T16:54:34.728 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 16 16:54:34 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-x[31905]: ignoring --setuser ceph since I am not root 2024-04-16T16:54:34.728 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 16 16:54:34 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-x[31905]: ignoring --setgroup ceph since I am not root 2024-04-16T16:54:35.167 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:34.078+0000 7fafd3400700 1 Processor -- start 2024-04-16T16:54:35.167 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:34.078+0000 7fafd3400700 1 -- start start 2024-04-16T16:54:35.167 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:34.079+0000 7fafd3400700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fafcc106ed0 0x7fafcc1072f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:54:35.167 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:34.079+0000 7fafd3400700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fafcc07be10 con 0x7fafcc106ed0 2024-04-16T16:54:35.167 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:34.079+0000 7fafd0a00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fafcc106ed0 0x7fafcc1072f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:54:35.168 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:34.079+0000 7fafd0a00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fafcc106ed0 0x7fafcc1072f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49294/0 (socket says 172.21.15.45:49294) 2024-04-16T16:54:35.168 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:34.079+0000 7fafd0a00700 1 -- 172.21.15.45:0/725725253 learned_addr learned my addr 172.21.15.45:0/725725253 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T16:54:35.168 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:34.080+0000 7fafd0a00700 1 -- 172.21.15.45:0/725725253 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafcc107830 con 0x7fafcc106ed0 2024-04-16T16:54:35.168 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:34.080+0000 7fafd0a00700 1 --2- 172.21.15.45:0/725725253 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fafcc106ed0 0x7fafcc1072f0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fafc00098a0 tx=0x7fafc0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2569b0bd65fe774e server_cookie=0 in_seq=0 out_seq=0 2024-04-16T16:54:35.168 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:34.080+0000 7fafcb400700 1 -- 172.21.15.45:0/725725253 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fafc0004160 con 0x7fafcc106ed0 2024-04-16T16:54:35.168 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:34.080+0000 7fafcb400700 1 -- 172.21.15.45:0/725725253 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fafc0004300 con 0x7fafcc106ed0 2024-04-16T16:54:35.168 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:34.080+0000 7fafcb400700 1 -- 172.21.15.45:0/725725253 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fafc0011b50 con 0x7fafcc106ed0 2024-04-16T16:54:35.168 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:34.081+0000 7fafd3400700 1 -- 172.21.15.45:0/725725253 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fafcc106ed0 msgr2=0x7fafcc1072f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:54:35.168 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:34.081+0000 7fafd3400700 1 --2- 172.21.15.45:0/725725253 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fafcc106ed0 0x7fafcc1072f0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fafc00098a0 tx=0x7fafc0009bb0 comp rx=0 tx=0).stop 2024-04-16T16:54:35.168 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:34.081+0000 7fafd3400700 1 -- 172.21.15.45:0/725725253 shutdown_connections 2024-04-16T16:54:35.168 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:34.081+0000 7fafd3400700 1 --2- 172.21.15.45:0/725725253 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fafcc106ed0 0x7fafcc1072f0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:54:35.168 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:34.081+0000 7fafd3400700 1 -- 172.21.15.45:0/725725253 >> 172.21.15.45:0/725725253 conn(0x7fafcc102a70 msgr2=0x7fafcc104eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T16:54:35.168 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:34.081+0000 7fafd3400700 1 -- 172.21.15.45:0/725725253 shutdown_connections 2024-04-16T16:54:35.168 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:34.081+0000 7fafd3400700 1 -- 172.21.15.45:0/725725253 wait complete. 2024-04-16T16:54:35.169 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:34.082+0000 7fafd3400700 1 Processor -- start 2024-04-16T16:54:35.169 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:34.083+0000 7fafd3400700 1 -- start start 2024-04-16T16:54:35.169 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:34.083+0000 7fafd3400700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fafcc106ed0 0x7fafcc19e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:54:35.169 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:34.083+0000 7fafd3400700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fafcc1088c0 con 0x7fafcc106ed0 2024-04-16T16:54:35.169 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:34.083+0000 7fafd0a00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fafcc106ed0 0x7fafcc19e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:54:35.169 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:34.083+0000 7fafd0a00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fafcc106ed0 0x7fafcc19e760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49310/0 (socket says 172.21.15.45:49310) 2024-04-16T16:54:35.169 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:34.083+0000 7fafd0a00700 1 -- 172.21.15.45:0/2071149514 learned_addr learned my addr 172.21.15.45:0/2071149514 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T16:54:35.169 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:34.084+0000 7fafd0a00700 1 -- 172.21.15.45:0/2071149514 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafcc19eca0 con 0x7fafcc106ed0 2024-04-16T16:54:35.169 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:34.084+0000 7fafd0a00700 1 --2- 172.21.15.45:0/2071149514 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fafcc106ed0 0x7fafcc19e760 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fafc0011b60 tx=0x7fafc0011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T16:54:35.169 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:34.084+0000 7fafc9600700 1 -- 172.21.15.45:0/2071149514 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fafc0003970 con 0x7fafcc106ed0 2024-04-16T16:54:35.169 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:34.084+0000 7fafc9600700 1 -- 172.21.15.45:0/2071149514 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fafc0003b10 con 0x7fafcc106ed0 2024-04-16T16:54:35.169 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:34.084+0000 7fafc9600700 1 -- 172.21.15.45:0/2071149514 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fafc0024610 con 0x7fafcc106ed0 2024-04-16T16:54:35.169 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:34.084+0000 7fafd3400700 1 -- 172.21.15.45:0/2071149514 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fafcc19ef30 con 0x7fafcc106ed0 2024-04-16T16:54:35.169 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:34.084+0000 7fafd3400700 1 -- 172.21.15.45:0/2071149514 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fafcc1a1c20 con 0x7fafcc106ed0 2024-04-16T16:54:35.170 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:34.086+0000 7fafd3400700 1 -- 172.21.15.45:0/2071149514 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fafcc064150 con 0x7fafcc106ed0 2024-04-16T16:54:35.170 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:34.086+0000 7fafc9600700 1 -- 172.21.15.45:0/2071149514 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 4) ==== 46547+0+0 (secure 0 0 0) 0x7fafc001a9f0 con 0x7fafcc106ed0 2024-04-16T16:54:35.170 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:34.086+0000 7fafc9600700 1 --2- 172.21.15.45:0/2071149514 >> [v2:172.21.15.45:6800/2553659622,v1:172.21.15.45:6801/2553659622] conn(0x7fafb40395b0 0x7fafb403ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:54:35.170 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:34.086+0000 7fafc9600700 1 -- 172.21.15.45:0/2071149514 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fafc0011e00 con 0x7fafcc106ed0 2024-04-16T16:54:35.170 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:34.087+0000 7fafcbe00700 1 --2- 172.21.15.45:0/2071149514 >> [v2:172.21.15.45:6800/2553659622,v1:172.21.15.45:6801/2553659622] conn(0x7fafb40395b0 0x7fafb403ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:54:35.170 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:34.087+0000 7fafcbe00700 1 --2- 172.21.15.45:0/2071149514 >> [v2:172.21.15.45:6800/2553659622,v1:172.21.15.45:6801/2553659622] conn(0x7fafb40395b0 0x7fafb403ba70 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fafb8006fd0 tx=0x7fafb8006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T16:54:35.170 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:34.092+0000 7fafc9600700 1 -- 172.21.15.45:0/2071149514 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fafc0016260 con 0x7fafcc106ed0 2024-04-16T16:54:35.170 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:34.388+0000 7fafd3400700 1 -- 172.21.15.45:0/2071149514 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) -- 0x7fafcc02e250 con 0x7fafcc106ed0 2024-04-16T16:54:35.170 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:34.441+0000 7fafc9600700 1 -- 172.21.15.45:0/2071149514 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) ==== 86+0+0 (secure 0 0 0) 0x7fafc0019e10 con 0x7fafcc106ed0 2024-04-16T16:54:35.170 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:34.445+0000 7fafc9600700 1 -- 172.21.15.45:0/2071149514 <== mon.0 v2:172.21.15.45:3300/0 8 ==== mgrmap(e 5) ==== 46558+0+0 (secure 0 0 0) 0x7fafc0024c40 con 0x7fafcc106ed0 2024-04-16T16:54:35.170 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:34.448+0000 7fafd3400700 1 -- 172.21.15.45:0/2071149514 >> [v2:172.21.15.45:6800/2553659622,v1:172.21.15.45:6801/2553659622] conn(0x7fafb40395b0 msgr2=0x7fafb403ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:54:35.170 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:34.448+0000 7fafd3400700 1 --2- 172.21.15.45:0/2071149514 >> [v2:172.21.15.45:6800/2553659622,v1:172.21.15.45:6801/2553659622] conn(0x7fafb40395b0 0x7fafb403ba70 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fafb8006fd0 tx=0x7fafb8006e40 comp rx=0 tx=0).stop 2024-04-16T16:54:35.170 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:34.448+0000 7fafd3400700 1 -- 172.21.15.45:0/2071149514 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fafcc106ed0 msgr2=0x7fafcc19e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:54:35.170 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:34.448+0000 7fafd3400700 1 --2- 172.21.15.45:0/2071149514 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fafcc106ed0 0x7fafcc19e760 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fafc0011b60 tx=0x7fafc0011c40 comp rx=0 tx=0).stop 2024-04-16T16:54:35.171 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:34.448+0000 7fafd3400700 1 -- 172.21.15.45:0/2071149514 shutdown_connections 2024-04-16T16:54:35.171 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:34.448+0000 7fafd3400700 1 --2- 172.21.15.45:0/2071149514 >> [v2:172.21.15.45:6800/2553659622,v1:172.21.15.45:6801/2553659622] conn(0x7fafb40395b0 0x7fafb403ba70 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:54:35.171 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:34.448+0000 7fafd3400700 1 --2- 172.21.15.45:0/2071149514 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fafcc106ed0 0x7fafcc19e760 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:54:35.171 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:34.448+0000 7fafd3400700 1 -- 172.21.15.45:0/2071149514 >> 172.21.15.45:0/2071149514 conn(0x7fafcc102a70 msgr2=0x7fafcc1953d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T16:54:35.171 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:34.448+0000 7fafd3400700 1 -- 172.21.15.45:0/2071149514 shutdown_connections 2024-04-16T16:54:35.171 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:34.448+0000 7fafd3400700 1 -- 172.21.15.45:0/2071149514 wait complete. 2024-04-16T16:54:35.452 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 16 16:54:35 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-x[31905]: 2024-04-16T16:54:35.258+0000 7f79939a4200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-16T16:54:35.452 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 16 16:54:35 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-x[31905]: 2024-04-16T16:54:35.384+0000 7f79939a4200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-16T16:54:35.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:35 smithi045 ceph-mon[31676]: from='client.? 172.21.15.45:0/2071149514' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-04-16T16:54:35.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:35 smithi045 ceph-mon[31676]: mgrmap e5: x(active, since 5s) 2024-04-16T16:54:36.356 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 16 16:54:36 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-x[31905]: 2024-04-16T16:54:36.225+0000 7f79939a4200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-16T16:54:36.356 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 16 16:54:36 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-x[31905]: 2024-04-16T16:54:36.353+0000 7f79939a4200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-16T16:54:36.675 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:36 smithi045 ceph-mon[31676]: from='client.? 172.21.15.45:0/3160996587' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-04-16T16:54:36.676 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 16 16:54:36 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-x[31905]: 2024-04-16T16:54:36.541+0000 7f79939a4200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-16T16:54:36.676 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 16 16:54:36 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-x[31905]: 2024-04-16T16:54:36.673+0000 7f79939a4200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-16T16:54:36.794 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout { 2024-04-16T16:54:36.794 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-04-16T16:54:36.794 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-16T16:54:36.794 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-04-16T16:54:36.794 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-04-16T16:54:36.794 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout } 2024-04-16T16:54:36.794 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:35.802+0000 7feef0a00700 1 Processor -- start 2024-04-16T16:54:36.794 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:35.803+0000 7feef0a00700 1 -- start start 2024-04-16T16:54:36.794 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:35.803+0000 7feef0a00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feeec108450 0x7feeec108870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:54:36.799 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:35.803+0000 7feef0a00700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7feeec108e40 con 0x7feeec108450 2024-04-16T16:54:36.799 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:35.803+0000 7feeeb400700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feeec108450 0x7feeec108870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:54:36.800 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:35.803+0000 7feeeb400700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feeec108450 0x7feeec108870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49336/0 (socket says 172.21.15.45:49336) 2024-04-16T16:54:36.800 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:35.803+0000 7feeeb400700 1 -- 172.21.15.45:0/224155184 learned_addr learned my addr 172.21.15.45:0/224155184 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T16:54:36.800 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:35.804+0000 7feeeb400700 1 -- 172.21.15.45:0/224155184 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feeec1095b0 con 0x7feeec108450 2024-04-16T16:54:36.800 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:35.804+0000 7feeeb400700 1 --2- 172.21.15.45:0/224155184 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feeec108450 0x7feeec108870 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7feedc0098a0 tx=0x7feedc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3bcd0e5737d8e94 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T16:54:36.800 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:35.804+0000 7feeea000700 1 -- 172.21.15.45:0/224155184 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7feedc004160 con 0x7feeec108450 2024-04-16T16:54:36.800 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:35.804+0000 7feeea000700 1 -- 172.21.15.45:0/224155184 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7feedc004300 con 0x7feeec108450 2024-04-16T16:54:36.800 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:35.804+0000 7feeea000700 1 -- 172.21.15.45:0/224155184 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7feedc011b50 con 0x7feeec108450 2024-04-16T16:54:36.800 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:35.805+0000 7feef0a00700 1 -- 172.21.15.45:0/224155184 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feeec108450 msgr2=0x7feeec108870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:54:36.800 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:35.805+0000 7feef0a00700 1 --2- 172.21.15.45:0/224155184 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feeec108450 0x7feeec108870 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7feedc0098a0 tx=0x7feedc009bb0 comp rx=0 tx=0).stop 2024-04-16T16:54:36.800 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:35.805+0000 7feef0a00700 1 -- 172.21.15.45:0/224155184 shutdown_connections 2024-04-16T16:54:36.800 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:35.805+0000 7feef0a00700 1 --2- 172.21.15.45:0/224155184 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feeec108450 0x7feeec108870 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:54:36.800 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:35.805+0000 7feef0a00700 1 -- 172.21.15.45:0/224155184 >> 172.21.15.45:0/224155184 conn(0x7feeec1039d0 msgr2=0x7feeec105e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T16:54:36.800 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:35.805+0000 7feef0a00700 1 -- 172.21.15.45:0/224155184 shutdown_connections 2024-04-16T16:54:36.800 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:35.806+0000 7feef0a00700 1 -- 172.21.15.45:0/224155184 wait complete. 2024-04-16T16:54:36.801 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:35.807+0000 7feef0a00700 1 Processor -- start 2024-04-16T16:54:36.801 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:35.807+0000 7feef0a00700 1 -- start start 2024-04-16T16:54:36.801 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:35.807+0000 7feef0a00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feeec108450 0x7feeec07db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:54:36.801 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:35.807+0000 7feef0a00700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7feeec10c960 con 0x7feeec108450 2024-04-16T16:54:36.801 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:35.807+0000 7feeeb400700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feeec108450 0x7feeec07db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:54:36.801 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:35.807+0000 7feeeb400700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feeec108450 0x7feeec07db70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49340/0 (socket says 172.21.15.45:49340) 2024-04-16T16:54:36.801 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:35.807+0000 7feeeb400700 1 -- 172.21.15.45:0/3160996587 learned_addr learned my addr 172.21.15.45:0/3160996587 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T16:54:36.801 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:35.808+0000 7feeeb400700 1 -- 172.21.15.45:0/3160996587 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feeec07e0b0 con 0x7feeec108450 2024-04-16T16:54:36.801 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:35.808+0000 7feeeb400700 1 --2- 172.21.15.45:0/3160996587 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feeec108450 0x7feeec07db70 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7feedc011850 tx=0x7feedc011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T16:54:36.801 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:35.808+0000 7feee3e00700 1 -- 172.21.15.45:0/3160996587 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7feedc003af0 con 0x7feeec108450 2024-04-16T16:54:36.801 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:35.808+0000 7feee3e00700 1 -- 172.21.15.45:0/3160996587 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7feedc020070 con 0x7feeec108450 2024-04-16T16:54:36.801 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:35.808+0000 7feee3e00700 1 -- 172.21.15.45:0/3160996587 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7feedc0195a0 con 0x7feeec108450 2024-04-16T16:54:36.801 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:35.808+0000 7feef0a00700 1 -- 172.21.15.45:0/3160996587 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feeec07c2a0 con 0x7feeec108450 2024-04-16T16:54:36.801 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:35.809+0000 7feef0a00700 1 -- 172.21.15.45:0/3160996587 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feeec07c7e0 con 0x7feeec108450 2024-04-16T16:54:36.801 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:35.810+0000 7feef0a00700 1 -- 172.21.15.45:0/3160996587 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feeec064150 con 0x7feeec108450 2024-04-16T16:54:36.802 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:35.810+0000 7feee3e00700 1 -- 172.21.15.45:0/3160996587 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 46558+0+0 (secure 0 0 0) 0x7feedc019850 con 0x7feeec108450 2024-04-16T16:54:36.802 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:35.810+0000 7feee3e00700 1 --2- 172.21.15.45:0/3160996587 >> [v2:172.21.15.45:6800/2553659622,v1:172.21.15.45:6801/2553659622] conn(0x7feed0039600 0x7feed003bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:54:36.802 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:35.810+0000 7feee3e00700 1 -- 172.21.15.45:0/3160996587 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7feedc011e00 con 0x7feeec108450 2024-04-16T16:54:36.802 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:35.810+0000 7feeeaa00700 1 -- 172.21.15.45:0/3160996587 >> [v2:172.21.15.45:6800/2553659622,v1:172.21.15.45:6801/2553659622] conn(0x7feed0039600 msgr2=0x7feed003bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.45:6800/2553659622 2024-04-16T16:54:36.802 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:35.811+0000 7feeeaa00700 1 --2- 172.21.15.45:0/3160996587 >> [v2:172.21.15.45:6800/2553659622,v1:172.21.15.45:6801/2553659622] conn(0x7feed0039600 0x7feed003bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-16T16:54:36.802 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:35.816+0000 7feee3e00700 1 -- 172.21.15.45:0/3160996587 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7feedc003c90 con 0x7feeec108450 2024-04-16T16:54:36.802 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:36.012+0000 7feeeaa00700 1 -- 172.21.15.45:0/3160996587 >> [v2:172.21.15.45:6800/2553659622,v1:172.21.15.45:6801/2553659622] conn(0x7feed0039600 msgr2=0x7feed003bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.45:6800/2553659622 2024-04-16T16:54:36.802 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:36.012+0000 7feeeaa00700 1 --2- 172.21.15.45:0/3160996587 >> [v2:172.21.15.45:6800/2553659622,v1:172.21.15.45:6801/2553659622] conn(0x7feed0039600 0x7feed003bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-16T16:54:36.802 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:36.107+0000 7feef0a00700 1 -- 172.21.15.45:0/3160996587 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7feeec07cf80 con 0x7feeec108450 2024-04-16T16:54:36.802 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:36.107+0000 7feee3e00700 1 -- 172.21.15.45:0/3160996587 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) ==== 56+0+88 (secure 0 0 0) 0x7feedc023530 con 0x7feeec108450 2024-04-16T16:54:36.802 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:36.110+0000 7feef0a00700 1 -- 172.21.15.45:0/3160996587 >> [v2:172.21.15.45:6800/2553659622,v1:172.21.15.45:6801/2553659622] conn(0x7feed0039600 msgr2=0x7feed003bac0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-16T16:54:36.802 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:36.110+0000 7feef0a00700 1 --2- 172.21.15.45:0/3160996587 >> [v2:172.21.15.45:6800/2553659622,v1:172.21.15.45:6801/2553659622] conn(0x7feed0039600 0x7feed003bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:54:36.802 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:36.110+0000 7feef0a00700 1 -- 172.21.15.45:0/3160996587 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feeec108450 msgr2=0x7feeec07db70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:54:36.803 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:36.110+0000 7feef0a00700 1 --2- 172.21.15.45:0/3160996587 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feeec108450 0x7feeec07db70 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7feedc011850 tx=0x7feedc011c10 comp rx=0 tx=0).stop 2024-04-16T16:54:36.803 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:36.110+0000 7feef0a00700 1 -- 172.21.15.45:0/3160996587 shutdown_connections 2024-04-16T16:54:36.803 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:36.110+0000 7feef0a00700 1 --2- 172.21.15.45:0/3160996587 >> [v2:172.21.15.45:6800/2553659622,v1:172.21.15.45:6801/2553659622] conn(0x7feed0039600 0x7feed003bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:54:36.803 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:36.110+0000 7feef0a00700 1 --2- 172.21.15.45:0/3160996587 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feeec108450 0x7feeec07db70 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:54:36.803 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:36.110+0000 7feef0a00700 1 -- 172.21.15.45:0/3160996587 >> 172.21.15.45:0/3160996587 conn(0x7feeec1039d0 msgr2=0x7feeec1045f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T16:54:36.803 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:36.111+0000 7feef0a00700 1 -- 172.21.15.45:0/3160996587 shutdown_connections 2024-04-16T16:54:36.803 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:36.111+0000 7feef0a00700 1 -- 172.21.15.45:0/3160996587 wait complete. 2024-04-16T16:54:36.803 INFO:teuthology.orchestra.run.smithi045.stdout:Waiting for the mgr to restart... 2024-04-16T16:54:36.803 INFO:teuthology.orchestra.run.smithi045.stdout:Waiting for mgr epoch 5... 2024-04-16T16:54:36.947 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 16 16:54:36 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-x[31905]: 2024-04-16T16:54:36.791+0000 7f79939a4200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-16T16:54:37.228 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 16 16:54:37 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-x[31905]: 2024-04-16T16:54:37.146+0000 7f79939a4200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-16T16:54:37.551 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 16 16:54:37 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-x[31905]: 2024-04-16T16:54:37.295+0000 7f79939a4200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-16T16:54:37.551 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 16 16:54:37 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-x[31905]: 2024-04-16T16:54:37.417+0000 7f79939a4200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-16T16:54:37.551 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 16 16:54:37 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-x[31905]: 2024-04-16T16:54:37.548+0000 7f79939a4200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-16T16:54:37.978 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 16 16:54:37 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-x[31905]: 2024-04-16T16:54:37.776+0000 7f79939a4200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-16T16:54:39.478 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 16 16:54:39 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-x[31905]: 2024-04-16T16:54:39.117+0000 7f79939a4200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-16T16:54:39.479 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 16 16:54:39 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-x[31905]: 2024-04-16T16:54:39.299+0000 7f79939a4200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-16T16:54:40.131 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 16 16:54:39 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-x[31905]: 2024-04-16T16:54:39.732+0000 7f79939a4200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-16T16:54:40.132 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 16 16:54:39 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-x[31905]: 2024-04-16T16:54:39.857+0000 7f79939a4200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-16T16:54:40.388 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 16 16:54:40 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-x[31905]: 2024-04-16T16:54:40.129+0000 7f79939a4200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-16T16:54:40.389 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 16 16:54:40 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-x[31905]: 2024-04-16T16:54:40.386+0000 7f79939a4200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-16T16:54:41.113 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 16 16:54:40 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-x[31905]: 2024-04-16T16:54:40.784+0000 7f79939a4200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-16T16:54:41.113 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 16 16:54:41 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-x[31905]: 2024-04-16T16:54:41.111+0000 7f79939a4200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-16T16:54:41.728 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 16 16:54:41 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-x[31905]: 2024-04-16T16:54:41.435+0000 7f79939a4200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-16T16:54:42.978 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 16 16:54:42 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-x[31905]: 2024-04-16T16:54:42.523+0000 7f79939a4200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-16T16:54:43.659 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 16 16:54:43 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-x[31905]: 2024-04-16T16:54:43.252+0000 7f79939a4200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-16T16:54:43.659 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 16 16:54:43 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-x[31905]: 2024-04-16T16:54:43.389+0000 7f79939a4200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-16T16:54:43.978 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 16 16:54:43 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-x[31905]: 2024-04-16T16:54:43.688+0000 7f79939a4200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-16T16:54:43.978 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:43 smithi045 ceph-mon[31676]: Active manager daemon x restarted 2024-04-16T16:54:43.979 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:43 smithi045 ceph-mon[31676]: Activating manager daemon x 2024-04-16T16:54:43.979 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:43 smithi045 ceph-mon[31676]: osdmap e2: 0 total, 0 up, 0 in 2024-04-16T16:54:43.979 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:43 smithi045 ceph-mon[31676]: mgrmap e6: x(active, starting, since 0.00450372s) 2024-04-16T16:54:43.979 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:43 smithi045 ceph-mon[31676]: from='mgr.14120 172.21.15.45:0/1544209564' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-16T16:54:43.979 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:43 smithi045 ceph-mon[31676]: from='mgr.14120 172.21.15.45:0/1544209564' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-16T16:54:43.979 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:43 smithi045 ceph-mon[31676]: from='mgr.14120 172.21.15.45:0/1544209564' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-16T16:54:43.979 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:43 smithi045 ceph-mon[31676]: from='mgr.14120 172.21.15.45:0/1544209564' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-16T16:54:43.979 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:43 smithi045 ceph-mon[31676]: from='mgr.14120 172.21.15.45:0/1544209564' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-16T16:54:43.979 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:43 smithi045 ceph-mon[31676]: Manager daemon x is now available 2024-04-16T16:54:43.979 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:43 smithi045 ceph-mon[31676]: from='mgr.14120 172.21.15.45:0/1544209564' entity='mgr.x' 2024-04-16T16:54:43.979 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:43 smithi045 ceph-mon[31676]: from='mgr.14120 172.21.15.45:0/1544209564' entity='mgr.x' 2024-04-16T16:54:44.978 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:44 smithi045 ceph-mon[31676]: Found migration_current of "None". Setting to last migration. 2024-04-16T16:54:44.979 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:44 smithi045 ceph-mon[31676]: from='mgr.14120 172.21.15.45:0/1544209564' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T16:54:44.979 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:44 smithi045 ceph-mon[31676]: from='mgr.14120 172.21.15.45:0/1544209564' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T16:54:44.979 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:44 smithi045 ceph-mon[31676]: from='mgr.14120 172.21.15.45:0/1544209564' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-16T16:54:44.979 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:44 smithi045 ceph-mon[31676]: from='mgr.14120 172.21.15.45:0/1544209564' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-16T16:54:44.979 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:44 smithi045 ceph-mon[31676]: from='mgr.14120 172.21.15.45:0/1544209564' entity='mgr.x' 2024-04-16T16:54:44.979 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:44 smithi045 ceph-mon[31676]: from='mgr.14120 172.21.15.45:0/1544209564' entity='mgr.x' 2024-04-16T16:54:44.979 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:44 smithi045 ceph-mon[31676]: [16/Apr/2024:16:54:44] ENGINE Bus STARTING 2024-04-16T16:54:44.979 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:44 smithi045 ceph-mon[31676]: mgrmap e7: x(active, since 1.01554s) 2024-04-16T16:54:45.392 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout { 2024-04-16T16:54:45.392 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-04-16T16:54:45.392 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "initialized": true 2024-04-16T16:54:45.393 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout } 2024-04-16T16:54:45.393 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:37.440+0000 7fdc78a00700 1 Processor -- start 2024-04-16T16:54:45.393 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:37.440+0000 7fdc78a00700 1 -- start start 2024-04-16T16:54:45.393 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:37.441+0000 7fdc78a00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdc74108450 0x7fdc74108870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:54:45.393 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:37.441+0000 7fdc78a00700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fdc74108e40 con 0x7fdc74108450 2024-04-16T16:54:45.393 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:37.441+0000 7fdc73400700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdc74108450 0x7fdc74108870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:54:45.393 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:37.441+0000 7fdc73400700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdc74108450 0x7fdc74108870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49356/0 (socket says 172.21.15.45:49356) 2024-04-16T16:54:45.393 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:37.441+0000 7fdc73400700 1 -- 172.21.15.45:0/3691689513 learned_addr learned my addr 172.21.15.45:0/3691689513 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T16:54:45.393 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:37.442+0000 7fdc73400700 1 -- 172.21.15.45:0/3691689513 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc741095b0 con 0x7fdc74108450 2024-04-16T16:54:45.393 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:37.442+0000 7fdc73400700 1 --2- 172.21.15.45:0/3691689513 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdc74108450 0x7fdc74108870 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fdc640098a0 tx=0x7fdc64009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72bc7355997fd526 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T16:54:45.393 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:37.442+0000 7fdc72000700 1 -- 172.21.15.45:0/3691689513 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdc64004160 con 0x7fdc74108450 2024-04-16T16:54:45.393 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:37.442+0000 7fdc72000700 1 -- 172.21.15.45:0/3691689513 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fdc64004300 con 0x7fdc74108450 2024-04-16T16:54:45.394 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:37.442+0000 7fdc72000700 1 -- 172.21.15.45:0/3691689513 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdc64011b50 con 0x7fdc74108450 2024-04-16T16:54:45.394 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:37.443+0000 7fdc78a00700 1 -- 172.21.15.45:0/3691689513 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdc74108450 msgr2=0x7fdc74108870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:54:45.394 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:37.443+0000 7fdc78a00700 1 --2- 172.21.15.45:0/3691689513 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdc74108450 0x7fdc74108870 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fdc640098a0 tx=0x7fdc64009bb0 comp rx=0 tx=0).stop 2024-04-16T16:54:45.394 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:37.443+0000 7fdc78a00700 1 -- 172.21.15.45:0/3691689513 shutdown_connections 2024-04-16T16:54:45.394 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:37.443+0000 7fdc78a00700 1 --2- 172.21.15.45:0/3691689513 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdc74108450 0x7fdc74108870 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:54:45.394 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:37.443+0000 7fdc78a00700 1 -- 172.21.15.45:0/3691689513 >> 172.21.15.45:0/3691689513 conn(0x7fdc741039d0 msgr2=0x7fdc74105e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T16:54:45.394 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:37.443+0000 7fdc78a00700 1 -- 172.21.15.45:0/3691689513 shutdown_connections 2024-04-16T16:54:45.394 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:37.443+0000 7fdc78a00700 1 -- 172.21.15.45:0/3691689513 wait complete. 2024-04-16T16:54:45.394 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:37.444+0000 7fdc78a00700 1 Processor -- start 2024-04-16T16:54:45.394 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:37.444+0000 7fdc78a00700 1 -- start start 2024-04-16T16:54:45.394 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:37.445+0000 7fdc78a00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdc74108450 0x7fdc7407db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:54:45.394 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:37.445+0000 7fdc78a00700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fdc7410c960 con 0x7fdc74108450 2024-04-16T16:54:45.394 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:37.445+0000 7fdc73400700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdc74108450 0x7fdc7407db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:54:45.395 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:37.445+0000 7fdc73400700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdc74108450 0x7fdc7407db70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49372/0 (socket says 172.21.15.45:49372) 2024-04-16T16:54:45.395 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:37.445+0000 7fdc73400700 1 -- 172.21.15.45:0/1246207264 learned_addr learned my addr 172.21.15.45:0/1246207264 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T16:54:45.395 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:37.445+0000 7fdc73400700 1 -- 172.21.15.45:0/1246207264 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc7407e0b0 con 0x7fdc74108450 2024-04-16T16:54:45.395 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:37.446+0000 7fdc73400700 1 --2- 172.21.15.45:0/1246207264 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdc74108450 0x7fdc7407db70 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fdc64011850 tx=0x7fdc64011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T16:54:45.395 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:37.446+0000 7fdc6be00700 1 -- 172.21.15.45:0/1246207264 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdc64003af0 con 0x7fdc74108450 2024-04-16T16:54:45.395 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:37.446+0000 7fdc6be00700 1 -- 172.21.15.45:0/1246207264 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fdc64020070 con 0x7fdc74108450 2024-04-16T16:54:45.395 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:37.446+0000 7fdc6be00700 1 -- 172.21.15.45:0/1246207264 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdc640195a0 con 0x7fdc74108450 2024-04-16T16:54:45.395 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:37.446+0000 7fdc78a00700 1 -- 172.21.15.45:0/1246207264 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc7407c2a0 con 0x7fdc74108450 2024-04-16T16:54:45.395 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:37.446+0000 7fdc78a00700 1 -- 172.21.15.45:0/1246207264 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc7407c7e0 con 0x7fdc74108450 2024-04-16T16:54:45.395 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:37.448+0000 7fdc6be00700 1 -- 172.21.15.45:0/1246207264 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 46558+0+0 (secure 0 0 0) 0x7fdc6401a430 con 0x7fdc74108450 2024-04-16T16:54:45.395 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:37.448+0000 7fdc6be00700 1 --2- 172.21.15.45:0/1246207264 >> [v2:172.21.15.45:6800/2553659622,v1:172.21.15.45:6801/2553659622] conn(0x7fdc58039600 0x7fdc5803bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:54:45.395 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:37.448+0000 7fdc6be00700 1 -- 172.21.15.45:0/1246207264 --> [v2:172.21.15.45:6800/2553659622,v1:172.21.15.45:6801/2553659622] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fdc5803c1d0 con 0x7fdc58039600 2024-04-16T16:54:45.395 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:37.448+0000 7fdc6be00700 1 -- 172.21.15.45:0/1246207264 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fdc64011e00 con 0x7fdc74108450 2024-04-16T16:54:45.396 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:37.448+0000 7fdc72a00700 1 -- 172.21.15.45:0/1246207264 >> [v2:172.21.15.45:6800/2553659622,v1:172.21.15.45:6801/2553659622] conn(0x7fdc58039600 msgr2=0x7fdc5803bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.45:6800/2553659622 2024-04-16T16:54:45.396 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:37.448+0000 7fdc72a00700 1 --2- 172.21.15.45:0/1246207264 >> [v2:172.21.15.45:6800/2553659622,v1:172.21.15.45:6801/2553659622] conn(0x7fdc58039600 0x7fdc5803bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-16T16:54:45.396 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:37.649+0000 7fdc72a00700 1 -- 172.21.15.45:0/1246207264 >> [v2:172.21.15.45:6800/2553659622,v1:172.21.15.45:6801/2553659622] conn(0x7fdc58039600 msgr2=0x7fdc5803bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.45:6800/2553659622 2024-04-16T16:54:45.396 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:37.649+0000 7fdc72a00700 1 --2- 172.21.15.45:0/1246207264 >> [v2:172.21.15.45:6800/2553659622,v1:172.21.15.45:6801/2553659622] conn(0x7fdc58039600 0x7fdc5803bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-16T16:54:45.396 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:38.050+0000 7fdc72a00700 1 -- 172.21.15.45:0/1246207264 >> [v2:172.21.15.45:6800/2553659622,v1:172.21.15.45:6801/2553659622] conn(0x7fdc58039600 msgr2=0x7fdc5803bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.45:6800/2553659622 2024-04-16T16:54:45.396 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:38.050+0000 7fdc72a00700 1 --2- 172.21.15.45:0/1246207264 >> [v2:172.21.15.45:6800/2553659622,v1:172.21.15.45:6801/2553659622] conn(0x7fdc58039600 0x7fdc5803bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-16T16:54:45.396 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:38.851+0000 7fdc72a00700 1 -- 172.21.15.45:0/1246207264 >> [v2:172.21.15.45:6800/2553659622,v1:172.21.15.45:6801/2553659622] conn(0x7fdc58039600 msgr2=0x7fdc5803bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.45:6800/2553659622 2024-04-16T16:54:45.396 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:38.851+0000 7fdc72a00700 1 --2- 172.21.15.45:0/1246207264 >> [v2:172.21.15.45:6800/2553659622,v1:172.21.15.45:6801/2553659622] conn(0x7fdc58039600 0x7fdc5803bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-16T16:54:45.396 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:40.453+0000 7fdc72a00700 1 -- 172.21.15.45:0/1246207264 >> [v2:172.21.15.45:6800/2553659622,v1:172.21.15.45:6801/2553659622] conn(0x7fdc58039600 msgr2=0x7fdc5803bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.45:6800/2553659622 2024-04-16T16:54:45.396 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:40.453+0000 7fdc72a00700 1 --2- 172.21.15.45:0/1246207264 >> [v2:172.21.15.45:6800/2553659622,v1:172.21.15.45:6801/2553659622] conn(0x7fdc58039600 0x7fdc5803bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-04-16T16:54:45.396 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:43.657+0000 7fdc72a00700 1 -- 172.21.15.45:0/1246207264 >> [v2:172.21.15.45:6800/2553659622,v1:172.21.15.45:6801/2553659622] conn(0x7fdc58039600 msgr2=0x7fdc5803bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.45:6800/2553659622 2024-04-16T16:54:45.396 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:43.658+0000 7fdc72a00700 1 --2- 172.21.15.45:0/1246207264 >> [v2:172.21.15.45:6800/2553659622,v1:172.21.15.45:6801/2553659622] conn(0x7fdc58039600 0x7fdc5803bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 6.400000 2024-04-16T16:54:45.396 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:43.696+0000 7fdc6be00700 1 -- 172.21.15.45:0/1246207264 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mgrmap(e 6) ==== 46325+0+0 (secure 0 0 0) 0x7fdc64024ca0 con 0x7fdc74108450 2024-04-16T16:54:45.396 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:43.697+0000 7fdc6be00700 1 -- 172.21.15.45:0/1246207264 >> [v2:172.21.15.45:6800/2553659622,v1:172.21.15.45:6801/2553659622] conn(0x7fdc58039600 msgr2=0x7fdc5803bac0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-16T16:54:45.397 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:43.697+0000 7fdc6be00700 1 --2- 172.21.15.45:0/1246207264 >> [v2:172.21.15.45:6800/2553659622,v1:172.21.15.45:6801/2553659622] conn(0x7fdc58039600 0x7fdc5803bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:54:45.397 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:44.701+0000 7fdc6be00700 1 -- 172.21.15.45:0/1246207264 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mgrmap(e 7) ==== 46452+0+0 (secure 0 0 0) 0x7fdc6404f570 con 0x7fdc74108450 2024-04-16T16:54:45.397 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:44.701+0000 7fdc6be00700 1 --2- 172.21.15.45:0/1246207264 >> [v2:172.21.15.45:6800/1983630136,v1:172.21.15.45:6801/1983630136] conn(0x7fdc5803cef0 0x7fdc5803f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:54:45.397 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:44.701+0000 7fdc6be00700 1 -- 172.21.15.45:0/1246207264 --> [v2:172.21.15.45:6800/1983630136,v1:172.21.15.45:6801/1983630136] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fdc64046c50 con 0x7fdc5803cef0 2024-04-16T16:54:45.397 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:44.702+0000 7fdc72a00700 1 --2- 172.21.15.45:0/1246207264 >> [v2:172.21.15.45:6800/1983630136,v1:172.21.15.45:6801/1983630136] conn(0x7fdc5803cef0 0x7fdc5803f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:54:45.397 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:44.705+0000 7fdc72a00700 1 --2- 172.21.15.45:0/1246207264 >> [v2:172.21.15.45:6800/1983630136,v1:172.21.15.45:6801/1983630136] conn(0x7fdc5803cef0 0x7fdc5803f2e0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fdc60003a10 tx=0x7fdc600092b0 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T16:54:45.397 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:44.706+0000 7fdc6be00700 1 -- 172.21.15.45:0/1246207264 <== mgr.14120 v2:172.21.15.45:6800/1983630136 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7774 (secure 0 0 0) 0x7fdc64046c50 con 0x7fdc5803cef0 2024-04-16T16:54:45.397 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:44.715+0000 7fdc78a00700 1 -- 172.21.15.45:0/1246207264 --> [v2:172.21.15.45:6800/1983630136,v1:172.21.15.45:6801/1983630136] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7fdc74064150 con 0x7fdc5803cef0 2024-04-16T16:54:45.397 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:44.716+0000 7fdc6be00700 1 -- 172.21.15.45:0/1246207264 <== mgr.14120 v2:172.21.15.45:6800/1983630136 2 ==== command_reply(tid 1: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7fdc74064150 con 0x7fdc5803cef0 2024-04-16T16:54:45.397 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:44.716+0000 7fdc78a00700 1 -- 172.21.15.45:0/1246207264 >> [v2:172.21.15.45:6800/1983630136,v1:172.21.15.45:6801/1983630136] conn(0x7fdc5803cef0 msgr2=0x7fdc5803f2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:54:45.397 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:44.716+0000 7fdc78a00700 1 --2- 172.21.15.45:0/1246207264 >> [v2:172.21.15.45:6800/1983630136,v1:172.21.15.45:6801/1983630136] conn(0x7fdc5803cef0 0x7fdc5803f2e0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fdc60003a10 tx=0x7fdc600092b0 comp rx=0 tx=0).stop 2024-04-16T16:54:45.397 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:44.716+0000 7fdc78a00700 1 -- 172.21.15.45:0/1246207264 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdc74108450 msgr2=0x7fdc7407db70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:54:45.397 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:44.716+0000 7fdc78a00700 1 --2- 172.21.15.45:0/1246207264 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdc74108450 0x7fdc7407db70 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fdc64011850 tx=0x7fdc64011c10 comp rx=0 tx=0).stop 2024-04-16T16:54:45.397 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:44.716+0000 7fdc78a00700 1 -- 172.21.15.45:0/1246207264 shutdown_connections 2024-04-16T16:54:45.398 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:44.716+0000 7fdc78a00700 1 --2- 172.21.15.45:0/1246207264 >> [v2:172.21.15.45:6800/1983630136,v1:172.21.15.45:6801/1983630136] conn(0x7fdc5803cef0 0x7fdc5803f2e0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:54:45.398 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:44.716+0000 7fdc78a00700 1 --2- 172.21.15.45:0/1246207264 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdc74108450 0x7fdc7407db70 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:54:45.398 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:44.716+0000 7fdc78a00700 1 --2- 172.21.15.45:0/1246207264 >> [v2:172.21.15.45:6800/2553659622,v1:172.21.15.45:6801/2553659622] conn(0x7fdc58039600 0x7fdc5803bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:54:45.398 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:44.716+0000 7fdc78a00700 1 -- 172.21.15.45:0/1246207264 >> 172.21.15.45:0/1246207264 conn(0x7fdc741039d0 msgr2=0x7fdc74104640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T16:54:45.398 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:44.717+0000 7fdc78a00700 1 -- 172.21.15.45:0/1246207264 shutdown_connections 2024-04-16T16:54:45.398 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:44.717+0000 7fdc78a00700 1 -- 172.21.15.45:0/1246207264 wait complete. 2024-04-16T16:54:45.398 INFO:teuthology.orchestra.run.smithi045.stdout:mgr epoch 5 is available 2024-04-16T16:54:45.398 INFO:teuthology.orchestra.run.smithi045.stdout:Setting orchestrator backend to cephadm... 2024-04-16T16:54:45.967 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:45 smithi045 ceph-mon[31676]: [16/Apr/2024:16:54:44] ENGINE Serving on http://172.21.15.45:8765 2024-04-16T16:54:45.967 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:45 smithi045 ceph-mon[31676]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-04-16T16:54:45.967 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:45 smithi045 ceph-mon[31676]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-04-16T16:54:45.967 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:45 smithi045 ceph-mon[31676]: [16/Apr/2024:16:54:44] ENGINE Serving on https://172.21.15.45:7150 2024-04-16T16:54:45.967 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:45 smithi045 ceph-mon[31676]: [16/Apr/2024:16:54:44] ENGINE Bus STARTED 2024-04-16T16:54:45.967 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:45 smithi045 ceph-mon[31676]: from='mgr.14120 172.21.15.45:0/1544209564' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T16:54:47.002 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:46 smithi045 ceph-mon[31676]: mgrmap e8: x(active, since 2s) 2024-04-16T16:54:47.002 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:46 smithi045 ceph-mon[31676]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T16:54:47.002 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:46 smithi045 ceph-mon[31676]: from='mgr.14120 172.21.15.45:0/1544209564' entity='mgr.x' 2024-04-16T16:54:47.002 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:46 smithi045 ceph-mon[31676]: from='mgr.14120 172.21.15.45:0/1544209564' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T16:54:47.016 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:46.136+0000 7fd069200700 1 Processor -- start 2024-04-16T16:54:47.016 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:46.136+0000 7fd069200700 1 -- start start 2024-04-16T16:54:47.016 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:46.136+0000 7fd069200700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd06410a5d0 0x7fd06410a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:54:47.016 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:46.136+0000 7fd069200700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fd06410afc0 con 0x7fd06410a5d0 2024-04-16T16:54:47.016 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:46.137+0000 7fd063400700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd06410a5d0 0x7fd06410a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:54:47.016 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:46.137+0000 7fd063400700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd06410a5d0 0x7fd06410a9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49704/0 (socket says 172.21.15.45:49704) 2024-04-16T16:54:47.016 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:46.137+0000 7fd063400700 1 -- 172.21.15.45:0/487767092 learned_addr learned my addr 172.21.15.45:0/487767092 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T16:54:47.021 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:46.137+0000 7fd063400700 1 -- 172.21.15.45:0/487767092 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd06410b7d0 con 0x7fd06410a5d0 2024-04-16T16:54:47.021 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:46.138+0000 7fd063400700 1 --2- 172.21.15.45:0/487767092 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd06410a5d0 0x7fd06410a9f0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fd054009e80 tx=0x7fd05400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b8d1aa0ab6a7374 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T16:54:47.021 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:46.138+0000 7fd062a00700 1 -- 172.21.15.45:0/487767092 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd054017070 con 0x7fd06410a5d0 2024-04-16T16:54:47.021 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:46.138+0000 7fd062a00700 1 -- 172.21.15.45:0/487767092 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fd05400b8c0 con 0x7fd06410a5d0 2024-04-16T16:54:47.021 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:46.138+0000 7fd062a00700 1 -- 172.21.15.45:0/487767092 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd0540045c0 con 0x7fd06410a5d0 2024-04-16T16:54:47.021 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:46.139+0000 7fd069200700 1 -- 172.21.15.45:0/487767092 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd06410a5d0 msgr2=0x7fd06410a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:54:47.021 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:46.139+0000 7fd069200700 1 --2- 172.21.15.45:0/487767092 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd06410a5d0 0x7fd06410a9f0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fd054009e80 tx=0x7fd05400b330 comp rx=0 tx=0).stop 2024-04-16T16:54:47.021 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:46.139+0000 7fd069200700 1 -- 172.21.15.45:0/487767092 shutdown_connections 2024-04-16T16:54:47.021 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:46.139+0000 7fd069200700 1 --2- 172.21.15.45:0/487767092 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd06410a5d0 0x7fd06410a9f0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:54:47.021 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:46.139+0000 7fd069200700 1 -- 172.21.15.45:0/487767092 >> 172.21.15.45:0/487767092 conn(0x7fd06407d9b0 msgr2=0x7fd0641084d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T16:54:47.021 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:46.139+0000 7fd069200700 1 -- 172.21.15.45:0/487767092 shutdown_connections 2024-04-16T16:54:47.021 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:46.139+0000 7fd069200700 1 -- 172.21.15.45:0/487767092 wait complete. 2024-04-16T16:54:47.022 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:46.140+0000 7fd069200700 1 Processor -- start 2024-04-16T16:54:47.022 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:46.141+0000 7fd069200700 1 -- start start 2024-04-16T16:54:47.022 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:46.141+0000 7fd069200700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd06410a5d0 0x7fd06419e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:54:47.022 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:46.141+0000 7fd069200700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fd06410eb80 con 0x7fd06410a5d0 2024-04-16T16:54:47.022 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:46.141+0000 7fd063400700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd06410a5d0 0x7fd06419e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:54:47.022 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:46.141+0000 7fd063400700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd06410a5d0 0x7fd06419e580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49716/0 (socket says 172.21.15.45:49716) 2024-04-16T16:54:47.022 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:46.141+0000 7fd063400700 1 -- 172.21.15.45:0/323714246 learned_addr learned my addr 172.21.15.45:0/323714246 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T16:54:47.022 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:46.142+0000 7fd063400700 1 -- 172.21.15.45:0/323714246 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd06419eac0 con 0x7fd06410a5d0 2024-04-16T16:54:47.022 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:46.142+0000 7fd063400700 1 --2- 172.21.15.45:0/323714246 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd06410a5d0 0x7fd06419e580 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fd054009e50 tx=0x7fd05400bd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T16:54:47.022 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:46.142+0000 7fd060c00700 1 -- 172.21.15.45:0/323714246 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd054020020 con 0x7fd06410a5d0 2024-04-16T16:54:47.022 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:46.143+0000 7fd060c00700 1 -- 172.21.15.45:0/323714246 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fd054003b50 con 0x7fd06410a5d0 2024-04-16T16:54:47.022 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:46.143+0000 7fd060c00700 1 -- 172.21.15.45:0/323714246 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd054017040 con 0x7fd06410a5d0 2024-04-16T16:54:47.022 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:46.143+0000 7fd069200700 1 -- 172.21.15.45:0/323714246 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd06419ed50 con 0x7fd06410a5d0 2024-04-16T16:54:47.022 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:46.143+0000 7fd069200700 1 -- 172.21.15.45:0/323714246 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0641a1a40 con 0x7fd06410a5d0 2024-04-16T16:54:47.023 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:46.144+0000 7fd060c00700 1 -- 172.21.15.45:0/323714246 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7fd054023020 con 0x7fd06410a5d0 2024-04-16T16:54:47.023 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:46.144+0000 7fd069200700 1 -- 172.21.15.45:0/323714246 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd064064150 con 0x7fd06410a5d0 2024-04-16T16:54:47.023 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:46.144+0000 7fd060c00700 1 --2- 172.21.15.45:0/323714246 >> [v2:172.21.15.45:6800/1983630136,v1:172.21.15.45:6801/1983630136] conn(0x7fd0480395b0 0x7fd04803ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:54:47.023 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:46.144+0000 7fd060c00700 1 -- 172.21.15.45:0/323714246 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fd05400f050 con 0x7fd06410a5d0 2024-04-16T16:54:47.023 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:46.145+0000 7fd05aa00700 1 --2- 172.21.15.45:0/323714246 >> [v2:172.21.15.45:6800/1983630136,v1:172.21.15.45:6801/1983630136] conn(0x7fd0480395b0 0x7fd04803ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:54:47.023 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:46.145+0000 7fd05aa00700 1 --2- 172.21.15.45:0/323714246 >> [v2:172.21.15.45:6800/1983630136,v1:172.21.15.45:6801/1983630136] conn(0x7fd0480395b0 0x7fd04803ba70 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fd04c006fd0 tx=0x7fd04c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T16:54:47.023 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:46.151+0000 7fd060c00700 1 -- 172.21.15.45:0/323714246 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd05402a020 con 0x7fd06410a5d0 2024-04-16T16:54:47.023 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:46.400+0000 7fd069200700 1 -- 172.21.15.45:0/323714246 --> [v2:172.21.15.45:6800/1983630136,v1:172.21.15.45:6801/1983630136] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) -- 0x7fd064108310 con 0x7fd0480395b0 2024-04-16T16:54:47.023 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:46.412+0000 7fd060c00700 1 -- 172.21.15.45:0/323714246 <== mgr.14120 v2:172.21.15.45:6800/1983630136 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fd064108310 con 0x7fd0480395b0 2024-04-16T16:54:47.023 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:46.416+0000 7fd069200700 1 -- 172.21.15.45:0/323714246 >> [v2:172.21.15.45:6800/1983630136,v1:172.21.15.45:6801/1983630136] conn(0x7fd0480395b0 msgr2=0x7fd04803ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:54:47.023 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:46.416+0000 7fd069200700 1 --2- 172.21.15.45:0/323714246 >> [v2:172.21.15.45:6800/1983630136,v1:172.21.15.45:6801/1983630136] conn(0x7fd0480395b0 0x7fd04803ba70 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fd04c006fd0 tx=0x7fd04c006e40 comp rx=0 tx=0).stop 2024-04-16T16:54:47.023 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:46.416+0000 7fd069200700 1 -- 172.21.15.45:0/323714246 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd06410a5d0 msgr2=0x7fd06419e580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:54:47.023 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:46.416+0000 7fd069200700 1 --2- 172.21.15.45:0/323714246 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd06410a5d0 0x7fd06419e580 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fd054009e50 tx=0x7fd05400bd30 comp rx=0 tx=0).stop 2024-04-16T16:54:47.023 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:46.416+0000 7fd069200700 1 -- 172.21.15.45:0/323714246 shutdown_connections 2024-04-16T16:54:47.024 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:46.416+0000 7fd069200700 1 --2- 172.21.15.45:0/323714246 >> [v2:172.21.15.45:6800/1983630136,v1:172.21.15.45:6801/1983630136] conn(0x7fd0480395b0 0x7fd04803ba70 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:54:47.024 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:46.416+0000 7fd069200700 1 --2- 172.21.15.45:0/323714246 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd06410a5d0 0x7fd06419e580 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:54:47.024 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:46.416+0000 7fd069200700 1 -- 172.21.15.45:0/323714246 >> 172.21.15.45:0/323714246 conn(0x7fd06407d9b0 msgr2=0x7fd064107c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T16:54:47.024 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:46.416+0000 7fd069200700 1 -- 172.21.15.45:0/323714246 shutdown_connections 2024-04-16T16:54:47.024 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:46.416+0000 7fd069200700 1 -- 172.21.15.45:0/323714246 wait complete. 2024-04-16T16:54:48.542 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout value unchanged 2024-04-16T16:54:48.543 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:47.644+0000 7f02ca600700 1 Processor -- start 2024-04-16T16:54:48.543 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:47.644+0000 7f02ca600700 1 -- start start 2024-04-16T16:54:48.543 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:47.645+0000 7f02ca600700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f02c410a7c0 0x7f02c410abe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:54:48.543 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:47.645+0000 7f02ca600700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f02c410b1b0 con 0x7f02c410a7c0 2024-04-16T16:54:48.543 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:47.645+0000 7f02c3400700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f02c410a7c0 0x7f02c410abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:54:48.543 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:47.646+0000 7f02c3400700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f02c410a7c0 0x7f02c410abe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49728/0 (socket says 172.21.15.45:49728) 2024-04-16T16:54:48.543 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:47.646+0000 7f02c3400700 1 -- 172.21.15.45:0/2121317255 learned_addr learned my addr 172.21.15.45:0/2121317255 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T16:54:48.543 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:47.646+0000 7f02c3400700 1 -- 172.21.15.45:0/2121317255 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02c410b9c0 con 0x7f02c410a7c0 2024-04-16T16:54:48.543 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:47.647+0000 7f02c3400700 1 --2- 172.21.15.45:0/2121317255 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f02c410a7c0 0x7f02c410abe0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f02b40098a0 tx=0x7f02b4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8884319a9b9abdbc server_cookie=0 in_seq=0 out_seq=0 2024-04-16T16:54:48.543 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:47.647+0000 7f02c2000700 1 -- 172.21.15.45:0/2121317255 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f02b4004160 con 0x7f02c410a7c0 2024-04-16T16:54:48.543 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:47.647+0000 7f02c2000700 1 -- 172.21.15.45:0/2121317255 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f02b4004300 con 0x7f02c410a7c0 2024-04-16T16:54:48.544 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:47.647+0000 7f02c2000700 1 -- 172.21.15.45:0/2121317255 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f02b4011b50 con 0x7f02c410a7c0 2024-04-16T16:54:48.544 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:47.648+0000 7f02ca600700 1 -- 172.21.15.45:0/2121317255 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f02c410a7c0 msgr2=0x7f02c410abe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:54:48.544 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:47.648+0000 7f02ca600700 1 --2- 172.21.15.45:0/2121317255 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f02c410a7c0 0x7f02c410abe0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f02b40098a0 tx=0x7f02b4009bb0 comp rx=0 tx=0).stop 2024-04-16T16:54:48.544 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:47.648+0000 7f02ca600700 1 -- 172.21.15.45:0/2121317255 shutdown_connections 2024-04-16T16:54:48.544 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:47.648+0000 7f02ca600700 1 --2- 172.21.15.45:0/2121317255 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f02c410a7c0 0x7f02c410abe0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:54:48.544 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:47.648+0000 7f02ca600700 1 -- 172.21.15.45:0/2121317255 >> 172.21.15.45:0/2121317255 conn(0x7f02c407de10 msgr2=0x7f02c41084d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T16:54:48.544 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:47.648+0000 7f02ca600700 1 -- 172.21.15.45:0/2121317255 shutdown_connections 2024-04-16T16:54:48.544 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:47.648+0000 7f02ca600700 1 -- 172.21.15.45:0/2121317255 wait complete. 2024-04-16T16:54:48.544 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:47.649+0000 7f02ca600700 1 Processor -- start 2024-04-16T16:54:48.544 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:47.649+0000 7f02ca600700 1 -- start start 2024-04-16T16:54:48.544 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:47.650+0000 7f02ca600700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f02c410a7c0 0x7f02c419e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:54:48.544 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:47.650+0000 7f02ca600700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f02c410ed70 con 0x7f02c410a7c0 2024-04-16T16:54:48.544 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:47.650+0000 7f02c3400700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f02c410a7c0 0x7f02c419e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:54:48.544 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:47.650+0000 7f02c3400700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f02c410a7c0 0x7f02c419e790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49732/0 (socket says 172.21.15.45:49732) 2024-04-16T16:54:48.545 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:47.650+0000 7f02c3400700 1 -- 172.21.15.45:0/781005339 learned_addr learned my addr 172.21.15.45:0/781005339 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T16:54:48.545 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:47.650+0000 7f02c3400700 1 -- 172.21.15.45:0/781005339 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02c419ecd0 con 0x7f02c410a7c0 2024-04-16T16:54:48.545 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:47.651+0000 7f02c3400700 1 --2- 172.21.15.45:0/781005339 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f02c410a7c0 0x7f02c419e790 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f02b4009fb0 tx=0x7f02b4011bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T16:54:48.545 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:47.651+0000 7f02bbe00700 1 -- 172.21.15.45:0/781005339 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f02b40038b0 con 0x7f02c410a7c0 2024-04-16T16:54:48.545 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:47.651+0000 7f02bbe00700 1 -- 172.21.15.45:0/781005339 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f02b4011e50 con 0x7f02c410a7c0 2024-04-16T16:54:48.545 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:47.651+0000 7f02ca600700 1 -- 172.21.15.45:0/781005339 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02c419ef60 con 0x7f02c410a7c0 2024-04-16T16:54:48.545 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:47.651+0000 7f02bbe00700 1 -- 172.21.15.45:0/781005339 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f02b4019550 con 0x7f02c410a7c0 2024-04-16T16:54:48.545 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:47.651+0000 7f02ca600700 1 -- 172.21.15.45:0/781005339 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02c41a1c50 con 0x7f02c410a7c0 2024-04-16T16:54:48.545 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:47.652+0000 7f02bbe00700 1 -- 172.21.15.45:0/781005339 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f02b4003a50 con 0x7f02c410a7c0 2024-04-16T16:54:48.545 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:47.652+0000 7f02ca600700 1 -- 172.21.15.45:0/781005339 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f02c4064150 con 0x7f02c410a7c0 2024-04-16T16:54:48.545 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:47.653+0000 7f02bbe00700 1 --2- 172.21.15.45:0/781005339 >> [v2:172.21.15.45:6800/1983630136,v1:172.21.15.45:6801/1983630136] conn(0x7f02a80395b0 0x7f02a803ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:54:48.545 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:47.653+0000 7f02bbe00700 1 -- 172.21.15.45:0/781005339 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f02b404d9b0 con 0x7f02c410a7c0 2024-04-16T16:54:48.545 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:47.653+0000 7f02c2a00700 1 --2- 172.21.15.45:0/781005339 >> [v2:172.21.15.45:6800/1983630136,v1:172.21.15.45:6801/1983630136] conn(0x7f02a80395b0 0x7f02a803ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:54:48.545 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:47.654+0000 7f02c2a00700 1 --2- 172.21.15.45:0/781005339 >> [v2:172.21.15.45:6800/1983630136,v1:172.21.15.45:6801/1983630136] conn(0x7f02a80395b0 0x7f02a803ba70 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f02b0006fd0 tx=0x7f02b0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T16:54:48.545 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:47.658+0000 7f02bbe00700 1 -- 172.21.15.45:0/781005339 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f02b4020240 con 0x7f02c410a7c0 2024-04-16T16:54:48.546 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:47.890+0000 7f02ca600700 1 -- 172.21.15.45:0/781005339 --> [v2:172.21.15.45:6800/1983630136,v1:172.21.15.45:6801/1983630136] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) -- 0x7f02c4108330 con 0x7f02a80395b0 2024-04-16T16:54:48.546 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:47.892+0000 7f02bbe00700 1 -- 172.21.15.45:0/781005339 <== mgr.14120 v2:172.21.15.45:6800/1983630136 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+16 (secure 0 0 0) 0x7f02c4108330 con 0x7f02a80395b0 2024-04-16T16:54:48.546 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:47.895+0000 7f02ca600700 1 -- 172.21.15.45:0/781005339 >> [v2:172.21.15.45:6800/1983630136,v1:172.21.15.45:6801/1983630136] conn(0x7f02a80395b0 msgr2=0x7f02a803ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:54:48.546 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:47.895+0000 7f02ca600700 1 --2- 172.21.15.45:0/781005339 >> [v2:172.21.15.45:6800/1983630136,v1:172.21.15.45:6801/1983630136] conn(0x7f02a80395b0 0x7f02a803ba70 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f02b0006fd0 tx=0x7f02b0006e40 comp rx=0 tx=0).stop 2024-04-16T16:54:48.546 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:47.895+0000 7f02ca600700 1 -- 172.21.15.45:0/781005339 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f02c410a7c0 msgr2=0x7f02c419e790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:54:48.546 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:47.895+0000 7f02ca600700 1 --2- 172.21.15.45:0/781005339 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f02c410a7c0 0x7f02c419e790 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f02b4009fb0 tx=0x7f02b4011bf0 comp rx=0 tx=0).stop 2024-04-16T16:54:48.546 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:47.895+0000 7f02ca600700 1 -- 172.21.15.45:0/781005339 shutdown_connections 2024-04-16T16:54:48.546 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:47.895+0000 7f02ca600700 1 --2- 172.21.15.45:0/781005339 >> [v2:172.21.15.45:6800/1983630136,v1:172.21.15.45:6801/1983630136] conn(0x7f02a80395b0 0x7f02a803ba70 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:54:48.546 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:47.895+0000 7f02ca600700 1 --2- 172.21.15.45:0/781005339 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f02c410a7c0 0x7f02c419e790 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:54:48.546 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:47.895+0000 7f02ca600700 1 -- 172.21.15.45:0/781005339 >> 172.21.15.45:0/781005339 conn(0x7f02c407de10 msgr2=0x7f02c4107c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T16:54:48.546 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:47.895+0000 7f02ca600700 1 -- 172.21.15.45:0/781005339 shutdown_connections 2024-04-16T16:54:48.546 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:47.895+0000 7f02ca600700 1 -- 172.21.15.45:0/781005339 wait complete. 2024-04-16T16:54:48.546 INFO:teuthology.orchestra.run.smithi045.stdout:Generating ssh key... 2024-04-16T16:54:48.827 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:48 smithi045 ceph-mon[31676]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T16:54:49.971 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:49 smithi045 ceph-mon[31676]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T16:54:49.971 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:49 smithi045 ceph-mon[31676]: Generating ssh key... 2024-04-16T16:54:49.971 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:49 smithi045 ceph-mon[31676]: from='mgr.14120 172.21.15.45:0/1544209564' entity='mgr.x' 2024-04-16T16:54:49.972 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:49 smithi045 ceph-mon[31676]: from='mgr.14120 172.21.15.45:0/1544209564' entity='mgr.x' 2024-04-16T16:54:49.972 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 16 16:54:49 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-x[31905]: Generating public/private rsa key pair. 2024-04-16T16:54:49.972 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 16 16:54:49 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-x[31905]: Your identification has been saved in /tmp/tmpw21fu2o3/key. 2024-04-16T16:54:49.972 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 16 16:54:49 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-x[31905]: Your public key has been saved in /tmp/tmpw21fu2o3/key.pub. 2024-04-16T16:54:49.972 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 16 16:54:49 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-x[31905]: The key fingerprint is: 2024-04-16T16:54:49.972 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 16 16:54:49 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-x[31905]: SHA256:hLW1HNk/UOlnYg/ma8umGDe93YMKMxpn5ghPd9dxefk ceph-b1fecd38-fc11-11ee-bc90-c7b262605968 2024-04-16T16:54:49.973 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 16 16:54:49 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-x[31905]: The key's randomart image is: 2024-04-16T16:54:49.973 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 16 16:54:49 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-x[31905]: +---[RSA 3072]----+ 2024-04-16T16:54:49.973 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 16 16:54:49 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-x[31905]: | . oo ... | 2024-04-16T16:54:49.973 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 16 16:54:49 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-x[31905]: | o +.oo . | 2024-04-16T16:54:49.973 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 16 16:54:49 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-x[31905]: | . o o + | 2024-04-16T16:54:49.973 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 16 16:54:49 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-x[31905]: | . O =| 2024-04-16T16:54:49.973 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 16 16:54:49 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-x[31905]: | S + @o| 2024-04-16T16:54:49.973 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 16 16:54:49 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-x[31905]: | .o *| 2024-04-16T16:54:49.973 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 16 16:54:49 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-x[31905]: | . o O..o.ooE| 2024-04-16T16:54:49.973 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 16 16:54:49 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-x[31905]: | + X =+.+++.| 2024-04-16T16:54:49.973 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 16 16:54:49 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-x[31905]: | + ...o++.+| 2024-04-16T16:54:49.973 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 16 16:54:49 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-x[31905]: +----[SHA256]-----+ 2024-04-16T16:54:50.348 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:49.111+0000 7f0e06800700 1 Processor -- start 2024-04-16T16:54:50.348 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:49.112+0000 7f0e06800700 1 -- start start 2024-04-16T16:54:50.348 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:49.112+0000 7f0e06800700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0e0010a7c0 0x7f0e0010abe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:54:50.348 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:49.112+0000 7f0e06800700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f0e0010b1b0 con 0x7f0e0010a7c0 2024-04-16T16:54:50.348 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:49.113+0000 7f0dff400700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0e0010a7c0 0x7f0e0010abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:54:50.348 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:49.113+0000 7f0dff400700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0e0010a7c0 0x7f0e0010abe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49738/0 (socket says 172.21.15.45:49738) 2024-04-16T16:54:50.348 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:49.113+0000 7f0dff400700 1 -- 172.21.15.45:0/1409933482 learned_addr learned my addr 172.21.15.45:0/1409933482 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T16:54:50.348 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:49.113+0000 7f0dff400700 1 -- 172.21.15.45:0/1409933482 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e0010b9c0 con 0x7f0e0010a7c0 2024-04-16T16:54:50.348 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:49.114+0000 7f0dff400700 1 --2- 172.21.15.45:0/1409933482 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0e0010a7c0 0x7f0e0010abe0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f0df00098a0 tx=0x7f0df0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74de0c3ee8758755 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T16:54:50.348 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:49.114+0000 7f0dfe000700 1 -- 172.21.15.45:0/1409933482 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0df0004160 con 0x7f0e0010a7c0 2024-04-16T16:54:50.348 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:49.114+0000 7f0dfe000700 1 -- 172.21.15.45:0/1409933482 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f0df0004300 con 0x7f0e0010a7c0 2024-04-16T16:54:50.348 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:49.114+0000 7f0dfe000700 1 -- 172.21.15.45:0/1409933482 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0df0011b50 con 0x7f0e0010a7c0 2024-04-16T16:54:50.349 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:49.115+0000 7f0e06800700 1 -- 172.21.15.45:0/1409933482 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0e0010a7c0 msgr2=0x7f0e0010abe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:54:50.349 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:49.115+0000 7f0e06800700 1 --2- 172.21.15.45:0/1409933482 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0e0010a7c0 0x7f0e0010abe0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f0df00098a0 tx=0x7f0df0009bb0 comp rx=0 tx=0).stop 2024-04-16T16:54:50.349 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:49.115+0000 7f0e06800700 1 -- 172.21.15.45:0/1409933482 shutdown_connections 2024-04-16T16:54:50.349 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:49.115+0000 7f0e06800700 1 --2- 172.21.15.45:0/1409933482 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0e0010a7c0 0x7f0e0010abe0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:54:50.349 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:49.115+0000 7f0e06800700 1 -- 172.21.15.45:0/1409933482 >> 172.21.15.45:0/1409933482 conn(0x7f0e0007de10 msgr2=0x7f0e001084d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T16:54:50.349 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:49.115+0000 7f0e06800700 1 -- 172.21.15.45:0/1409933482 shutdown_connections 2024-04-16T16:54:50.349 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:49.115+0000 7f0e06800700 1 -- 172.21.15.45:0/1409933482 wait complete. 2024-04-16T16:54:50.349 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:49.117+0000 7f0e06800700 1 Processor -- start 2024-04-16T16:54:50.349 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:49.117+0000 7f0e06800700 1 -- start start 2024-04-16T16:54:50.349 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:49.117+0000 7f0e06800700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0e0010a7c0 0x7f0e0019e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:54:50.349 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:49.117+0000 7f0e06800700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f0e0010ed70 con 0x7f0e0010a7c0 2024-04-16T16:54:50.349 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:49.118+0000 7f0dff400700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0e0010a7c0 0x7f0e0019e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:54:50.349 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:49.118+0000 7f0dff400700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0e0010a7c0 0x7f0e0019e790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49746/0 (socket says 172.21.15.45:49746) 2024-04-16T16:54:50.349 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:49.118+0000 7f0dff400700 1 -- 172.21.15.45:0/776746063 learned_addr learned my addr 172.21.15.45:0/776746063 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T16:54:50.350 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:49.118+0000 7f0dff400700 1 -- 172.21.15.45:0/776746063 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e0019ecd0 con 0x7f0e0010a7c0 2024-04-16T16:54:50.350 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:49.118+0000 7f0dff400700 1 --2- 172.21.15.45:0/776746063 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0e0010a7c0 0x7f0e0019e790 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f0df0011850 tx=0x7f0df0011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T16:54:50.350 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:49.119+0000 7f0df7e00700 1 -- 172.21.15.45:0/776746063 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0df0003af0 con 0x7f0e0010a7c0 2024-04-16T16:54:50.350 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:49.119+0000 7f0df7e00700 1 -- 172.21.15.45:0/776746063 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f0df0020070 con 0x7f0e0010a7c0 2024-04-16T16:54:50.350 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:49.119+0000 7f0df7e00700 1 -- 172.21.15.45:0/776746063 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0df00195a0 con 0x7f0e0010a7c0 2024-04-16T16:54:50.350 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:49.119+0000 7f0e06800700 1 -- 172.21.15.45:0/776746063 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e0019ef60 con 0x7f0e0010a7c0 2024-04-16T16:54:50.350 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:49.119+0000 7f0e06800700 1 -- 172.21.15.45:0/776746063 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e001a1c50 con 0x7f0e0010a7c0 2024-04-16T16:54:50.350 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:49.120+0000 7f0df7e00700 1 -- 172.21.15.45:0/776746063 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f0df0019850 con 0x7f0e0010a7c0 2024-04-16T16:54:50.350 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:49.120+0000 7f0e06800700 1 -- 172.21.15.45:0/776746063 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e00064150 con 0x7f0e0010a7c0 2024-04-16T16:54:50.350 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:49.121+0000 7f0df7e00700 1 --2- 172.21.15.45:0/776746063 >> [v2:172.21.15.45:6800/1983630136,v1:172.21.15.45:6801/1983630136] conn(0x7f0de4039600 0x7f0de403bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:54:50.350 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:49.121+0000 7f0df7e00700 1 -- 172.21.15.45:0/776746063 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f0df0011e00 con 0x7f0e0010a7c0 2024-04-16T16:54:50.350 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:49.121+0000 7f0dfea00700 1 --2- 172.21.15.45:0/776746063 >> [v2:172.21.15.45:6800/1983630136,v1:172.21.15.45:6801/1983630136] conn(0x7f0de4039600 0x7f0de403bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:54:50.351 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:49.122+0000 7f0dfea00700 1 --2- 172.21.15.45:0/776746063 >> [v2:172.21.15.45:6800/1983630136,v1:172.21.15.45:6801/1983630136] conn(0x7f0de4039600 0x7f0de403bac0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f0dec006fd0 tx=0x7f0dec006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T16:54:50.351 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:49.128+0000 7f0df7e00700 1 -- 172.21.15.45:0/776746063 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0df000e260 con 0x7f0e0010a7c0 2024-04-16T16:54:50.351 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:49.360+0000 7f0e06800700 1 -- 172.21.15.45:0/776746063 --> [v2:172.21.15.45:6800/1983630136,v1:172.21.15.45:6801/1983630136] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) -- 0x7f0e00108320 con 0x7f0de4039600 2024-04-16T16:54:50.351 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:49.606+0000 7f0df7e00700 1 -- 172.21.15.45:0/776746063 <== mgr.14120 v2:172.21.15.45:6800/1983630136 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f0e00108320 con 0x7f0de4039600 2024-04-16T16:54:50.351 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:49.609+0000 7f0e06800700 1 -- 172.21.15.45:0/776746063 >> [v2:172.21.15.45:6800/1983630136,v1:172.21.15.45:6801/1983630136] conn(0x7f0de4039600 msgr2=0x7f0de403bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:54:50.351 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:49.609+0000 7f0e06800700 1 --2- 172.21.15.45:0/776746063 >> [v2:172.21.15.45:6800/1983630136,v1:172.21.15.45:6801/1983630136] conn(0x7f0de4039600 0x7f0de403bac0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f0dec006fd0 tx=0x7f0dec006e40 comp rx=0 tx=0).stop 2024-04-16T16:54:50.351 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:49.609+0000 7f0e06800700 1 -- 172.21.15.45:0/776746063 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0e0010a7c0 msgr2=0x7f0e0019e790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:54:50.351 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:49.609+0000 7f0e06800700 1 --2- 172.21.15.45:0/776746063 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0e0010a7c0 0x7f0e0019e790 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f0df0011850 tx=0x7f0df0011c10 comp rx=0 tx=0).stop 2024-04-16T16:54:50.351 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:49.610+0000 7f0e06800700 1 -- 172.21.15.45:0/776746063 shutdown_connections 2024-04-16T16:54:50.351 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:49.610+0000 7f0e06800700 1 --2- 172.21.15.45:0/776746063 >> [v2:172.21.15.45:6800/1983630136,v1:172.21.15.45:6801/1983630136] conn(0x7f0de4039600 0x7f0de403bac0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:54:50.351 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:49.610+0000 7f0e06800700 1 --2- 172.21.15.45:0/776746063 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0e0010a7c0 0x7f0e0019e790 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:54:50.351 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:49.610+0000 7f0e06800700 1 -- 172.21.15.45:0/776746063 >> 172.21.15.45:0/776746063 conn(0x7f0e0007de10 msgr2=0x7f0e00107c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T16:54:50.351 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:49.610+0000 7f0e06800700 1 -- 172.21.15.45:0/776746063 shutdown_connections 2024-04-16T16:54:50.351 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:49.610+0000 7f0e06800700 1 -- 172.21.15.45:0/776746063 wait complete. 2024-04-16T16:54:51.836 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:51 smithi045 ceph-mon[31676]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T16:54:51.845 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC0HRkdd/nb7cJP9WLx41ZcucBZg8S6GffuqGVcdJyHYPttolr/DWL/oBs2x7IOjrQPQqKf7RfTWyErPsM/AVI8zSZ50wIhZ68382uHkFZ+h/rrraimekgWFstBalbFzu8D5rhwb44Zm6/2Yz836n3ujVG3MmvZjmDzfYURKFaVHMnndHW705uMZwigCkYSEfBxSZaMzdkoLHAOUBy79OzHGTpXKIHm63RhQi2CC+GaSMte3tYDI7ZifWzr8GP4Xt+XYC1gtp4mYXwsu9hALSreJeO+p/XTidVbapL8pyaGfJ/ZjAPTgtemAc7WzI2+0S0BYinFyzdbsv7XJ4Ico2MWCBNBy2brAYTdQX1DEGzSjlIMZLv+hMpnY+4Hh/PebySReztCYlm41lYeNTc/rEYu0f1vUwHQC3wCuyQbHDLmR8HxWV/5h3Ubq4W7faPxsaPXk7k2qbTPBgjKreITsGXcjbxraR7yXaIO+cNFf+ps6kEVUCeZXdV7DtmDetRHh7c= ceph-b1fecd38-fc11-11ee-bc90-c7b262605968 2024-04-16T16:54:51.851 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:50.921+0000 7f90be400700 1 Processor -- start 2024-04-16T16:54:51.851 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:50.922+0000 7f90be400700 1 -- start start 2024-04-16T16:54:51.851 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:50.922+0000 7f90be400700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f90b8106cc0 0x7f90b81070e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:54:51.851 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:50.922+0000 7f90be400700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f90b807be30 con 0x7f90b8106cc0 2024-04-16T16:54:51.851 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:50.923+0000 7f90bd000700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f90b8106cc0 0x7f90b81070e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:54:51.851 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:50.923+0000 7f90bd000700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f90b8106cc0 0x7f90b81070e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35062/0 (socket says 172.21.15.45:35062) 2024-04-16T16:54:51.851 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:50.923+0000 7f90bd000700 1 -- 172.21.15.45:0/2720176330 learned_addr learned my addr 172.21.15.45:0/2720176330 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T16:54:51.851 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:50.923+0000 7f90bd000700 1 -- 172.21.15.45:0/2720176330 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90b8107620 con 0x7f90b8106cc0 2024-04-16T16:54:51.851 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:50.923+0000 7f90bd000700 1 --2- 172.21.15.45:0/2720176330 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f90b8106cc0 0x7f90b81070e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f90ac0098a0 tx=0x7f90ac009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4cc2a0abc8b956b server_cookie=0 in_seq=0 out_seq=0 2024-04-16T16:54:51.852 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:50.924+0000 7f90b7400700 1 -- 172.21.15.45:0/2720176330 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f90ac004160 con 0x7f90b8106cc0 2024-04-16T16:54:51.852 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:50.924+0000 7f90b7400700 1 -- 172.21.15.45:0/2720176330 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f90ac004300 con 0x7f90b8106cc0 2024-04-16T16:54:51.852 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:50.924+0000 7f90b7400700 1 -- 172.21.15.45:0/2720176330 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f90ac011b50 con 0x7f90b8106cc0 2024-04-16T16:54:51.852 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:50.924+0000 7f90be400700 1 -- 172.21.15.45:0/2720176330 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f90b8106cc0 msgr2=0x7f90b81070e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:54:51.852 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:50.924+0000 7f90be400700 1 --2- 172.21.15.45:0/2720176330 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f90b8106cc0 0x7f90b81070e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f90ac0098a0 tx=0x7f90ac009bb0 comp rx=0 tx=0).stop 2024-04-16T16:54:51.852 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:50.925+0000 7f90be400700 1 -- 172.21.15.45:0/2720176330 shutdown_connections 2024-04-16T16:54:51.852 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:50.925+0000 7f90be400700 1 --2- 172.21.15.45:0/2720176330 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f90b8106cc0 0x7f90b81070e0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:54:51.852 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:50.925+0000 7f90be400700 1 -- 172.21.15.45:0/2720176330 >> 172.21.15.45:0/2720176330 conn(0x7f90b81028c0 msgr2=0x7f90b8104ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T16:54:51.852 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:50.925+0000 7f90be400700 1 -- 172.21.15.45:0/2720176330 shutdown_connections 2024-04-16T16:54:51.852 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:50.925+0000 7f90be400700 1 -- 172.21.15.45:0/2720176330 wait complete. 2024-04-16T16:54:51.852 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:50.926+0000 7f90be400700 1 Processor -- start 2024-04-16T16:54:51.852 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:50.926+0000 7f90be400700 1 -- start start 2024-04-16T16:54:51.852 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:50.927+0000 7f90be400700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f90b8106cc0 0x7f90b819e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:54:51.852 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:50.927+0000 7f90be400700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f90b81086b0 con 0x7f90b8106cc0 2024-04-16T16:54:51.853 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:50.927+0000 7f90bd000700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f90b8106cc0 0x7f90b819e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:54:51.853 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:50.927+0000 7f90bd000700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f90b8106cc0 0x7f90b819e5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35066/0 (socket says 172.21.15.45:35066) 2024-04-16T16:54:51.853 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:50.927+0000 7f90bd000700 1 -- 172.21.15.45:0/4273786799 learned_addr learned my addr 172.21.15.45:0/4273786799 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T16:54:51.853 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:50.927+0000 7f90bd000700 1 -- 172.21.15.45:0/4273786799 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90b819eb20 con 0x7f90b8106cc0 2024-04-16T16:54:51.853 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:50.928+0000 7f90bd000700 1 --2- 172.21.15.45:0/4273786799 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f90b8106cc0 0x7f90b819e5e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f90ac011b60 tx=0x7f90ac011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T16:54:51.853 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:50.928+0000 7f90b5600700 1 -- 172.21.15.45:0/4273786799 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f90ac003970 con 0x7f90b8106cc0 2024-04-16T16:54:51.853 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:50.928+0000 7f90b5600700 1 -- 172.21.15.45:0/4273786799 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f90ac003b10 con 0x7f90b8106cc0 2024-04-16T16:54:51.853 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:50.928+0000 7f90b5600700 1 -- 172.21.15.45:0/4273786799 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f90ac024610 con 0x7f90b8106cc0 2024-04-16T16:54:51.853 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:50.928+0000 7f90be400700 1 -- 172.21.15.45:0/4273786799 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90b819edb0 con 0x7f90b8106cc0 2024-04-16T16:54:51.853 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:50.928+0000 7f90be400700 1 -- 172.21.15.45:0/4273786799 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90b81a1aa0 con 0x7f90b8106cc0 2024-04-16T16:54:51.853 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:50.929+0000 7f90b5600700 1 -- 172.21.15.45:0/4273786799 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f90ac01a9f0 con 0x7f90b8106cc0 2024-04-16T16:54:51.853 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:50.929+0000 7f90be400700 1 -- 172.21.15.45:0/4273786799 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f90b8064150 con 0x7f90b8106cc0 2024-04-16T16:54:51.853 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:50.930+0000 7f90b5600700 1 --2- 172.21.15.45:0/4273786799 >> [v2:172.21.15.45:6800/1983630136,v1:172.21.15.45:6801/1983630136] conn(0x7f90a0039600 0x7f90a003bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:54:51.853 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:50.930+0000 7f90b5600700 1 -- 172.21.15.45:0/4273786799 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f90ac011e00 con 0x7f90b8106cc0 2024-04-16T16:54:51.854 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:50.930+0000 7f90b7e00700 1 --2- 172.21.15.45:0/4273786799 >> [v2:172.21.15.45:6800/1983630136,v1:172.21.15.45:6801/1983630136] conn(0x7f90a0039600 0x7f90a003bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:54:51.854 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:50.931+0000 7f90b7e00700 1 --2- 172.21.15.45:0/4273786799 >> [v2:172.21.15.45:6800/1983630136,v1:172.21.15.45:6801/1983630136] conn(0x7f90a0039600 0x7f90a003bac0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f90a4006fd0 tx=0x7f90a4006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T16:54:51.854 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:50.936+0000 7f90b5600700 1 -- 172.21.15.45:0/4273786799 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f90ac016260 con 0x7f90b8106cc0 2024-04-16T16:54:51.854 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:51.161+0000 7f90be400700 1 -- 172.21.15.45:0/4273786799 --> [v2:172.21.15.45:6800/1983630136,v1:172.21.15.45:6801/1983630136] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) -- 0x7f90b802e2c0 con 0x7f90a0039600 2024-04-16T16:54:51.854 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:51.162+0000 7f90b5600700 1 -- 172.21.15.45:0/4273786799 <== mgr.14120 v2:172.21.15.45:6800/1983630136 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+595 (secure 0 0 0) 0x7f90b802e2c0 con 0x7f90a0039600 2024-04-16T16:54:51.854 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:51.165+0000 7f90be400700 1 -- 172.21.15.45:0/4273786799 >> [v2:172.21.15.45:6800/1983630136,v1:172.21.15.45:6801/1983630136] conn(0x7f90a0039600 msgr2=0x7f90a003bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:54:51.854 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:51.165+0000 7f90be400700 1 --2- 172.21.15.45:0/4273786799 >> [v2:172.21.15.45:6800/1983630136,v1:172.21.15.45:6801/1983630136] conn(0x7f90a0039600 0x7f90a003bac0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f90a4006fd0 tx=0x7f90a4006e40 comp rx=0 tx=0).stop 2024-04-16T16:54:51.854 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:51.165+0000 7f90be400700 1 -- 172.21.15.45:0/4273786799 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f90b8106cc0 msgr2=0x7f90b819e5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:54:51.854 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:51.165+0000 7f90be400700 1 --2- 172.21.15.45:0/4273786799 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f90b8106cc0 0x7f90b819e5e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f90ac011b60 tx=0x7f90ac011c40 comp rx=0 tx=0).stop 2024-04-16T16:54:51.854 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:51.165+0000 7f90be400700 1 -- 172.21.15.45:0/4273786799 shutdown_connections 2024-04-16T16:54:51.854 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:51.165+0000 7f90be400700 1 --2- 172.21.15.45:0/4273786799 >> [v2:172.21.15.45:6800/1983630136,v1:172.21.15.45:6801/1983630136] conn(0x7f90a0039600 0x7f90a003bac0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:54:51.854 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:51.165+0000 7f90be400700 1 --2- 172.21.15.45:0/4273786799 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f90b8106cc0 0x7f90b819e5e0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:54:51.854 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:51.165+0000 7f90be400700 1 -- 172.21.15.45:0/4273786799 >> 172.21.15.45:0/4273786799 conn(0x7f90b81028c0 msgr2=0x7f90b81951c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T16:54:51.854 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:51.165+0000 7f90be400700 1 -- 172.21.15.45:0/4273786799 shutdown_connections 2024-04-16T16:54:51.855 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:51.166+0000 7f90be400700 1 -- 172.21.15.45:0/4273786799 wait complete. 2024-04-16T16:54:51.855 INFO:teuthology.orchestra.run.smithi045.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-04-16T16:54:51.855 INFO:teuthology.orchestra.run.smithi045.stdout:Adding key to root@localhost authorized_keys... 2024-04-16T16:54:51.855 INFO:teuthology.orchestra.run.smithi045.stdout:Adding host smithi045... 2024-04-16T16:54:53.978 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:53 smithi045 ceph-mon[31676]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi045", "addr": "172.21.15.45", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T16:54:54.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:54:54 smithi045 ceph-mon[31676]: Deploying cephadm binary to smithi045 2024-04-16T16:55:00.691 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout Added host 'smithi045' with addr '172.21.15.45' 2024-04-16T16:55:00.696 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:52.581+0000 7f4264e00700 1 Processor -- start 2024-04-16T16:55:00.696 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:52.582+0000 7f4264e00700 1 -- start start 2024-04-16T16:55:00.697 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:52.582+0000 7f4264e00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f426007de00 0x7f426007c300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:55:00.697 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:52.582+0000 7f4264e00700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f426007c840 con 0x7f426007de00 2024-04-16T16:55:00.697 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:52.583+0000 7f425f400700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f426007de00 0x7f426007c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:55:00.697 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:52.583+0000 7f425f400700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f426007de00 0x7f426007c300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35076/0 (socket says 172.21.15.45:35076) 2024-04-16T16:55:00.697 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:52.583+0000 7f425f400700 1 -- 172.21.15.45:0/2201329918 learned_addr learned my addr 172.21.15.45:0/2201329918 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T16:55:00.697 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:52.583+0000 7f425f400700 1 -- 172.21.15.45:0/2201329918 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f426007c9c0 con 0x7f426007de00 2024-04-16T16:55:00.697 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:52.584+0000 7f425f400700 1 --2- 172.21.15.45:0/2201329918 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f426007de00 0x7f426007c300 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f42500098a0 tx=0x7f4250009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=262c6095be41162b server_cookie=0 in_seq=0 out_seq=0 2024-04-16T16:55:00.697 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:52.584+0000 7f425ea00700 1 -- 172.21.15.45:0/2201329918 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4250004160 con 0x7f426007de00 2024-04-16T16:55:00.697 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:52.584+0000 7f425ea00700 1 -- 172.21.15.45:0/2201329918 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f4250004300 con 0x7f426007de00 2024-04-16T16:55:00.697 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:52.584+0000 7f425ea00700 1 -- 172.21.15.45:0/2201329918 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4250011b50 con 0x7f426007de00 2024-04-16T16:55:00.697 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:52.585+0000 7f4264e00700 1 -- 172.21.15.45:0/2201329918 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f426007de00 msgr2=0x7f426007c300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:55:00.697 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:52.585+0000 7f4264e00700 1 --2- 172.21.15.45:0/2201329918 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f426007de00 0x7f426007c300 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f42500098a0 tx=0x7f4250009bb0 comp rx=0 tx=0).stop 2024-04-16T16:55:00.697 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:52.585+0000 7f4264e00700 1 -- 172.21.15.45:0/2201329918 shutdown_connections 2024-04-16T16:55:00.698 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:52.585+0000 7f4264e00700 1 --2- 172.21.15.45:0/2201329918 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f426007de00 0x7f426007c300 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:55:00.698 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:52.585+0000 7f4264e00700 1 -- 172.21.15.45:0/2201329918 >> 172.21.15.45:0/2201329918 conn(0x7f42601039d0 msgr2=0x7f4260105e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T16:55:00.698 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:52.585+0000 7f4264e00700 1 -- 172.21.15.45:0/2201329918 shutdown_connections 2024-04-16T16:55:00.698 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:52.585+0000 7f4264e00700 1 -- 172.21.15.45:0/2201329918 wait complete. 2024-04-16T16:55:00.698 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:52.586+0000 7f4264e00700 1 Processor -- start 2024-04-16T16:55:00.698 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:52.586+0000 7f4264e00700 1 -- start start 2024-04-16T16:55:00.698 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:52.587+0000 7f4264e00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f426007de00 0x7f426019e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:55:00.698 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:52.587+0000 7f4264e00700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f426010ea00 con 0x7f426007de00 2024-04-16T16:55:00.698 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:52.587+0000 7f425f400700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f426007de00 0x7f426019e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:55:00.698 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:52.587+0000 7f425f400700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f426007de00 0x7f426019e5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35088/0 (socket says 172.21.15.45:35088) 2024-04-16T16:55:00.698 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:52.587+0000 7f425f400700 1 -- 172.21.15.45:0/3562071552 learned_addr learned my addr 172.21.15.45:0/3562071552 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T16:55:00.698 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:52.587+0000 7f425f400700 1 -- 172.21.15.45:0/3562071552 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f426019eb00 con 0x7f426007de00 2024-04-16T16:55:00.698 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:52.588+0000 7f425f400700 1 --2- 172.21.15.45:0/3562071552 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f426007de00 0x7f426019e5c0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f4250011b60 tx=0x7f4250011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T16:55:00.698 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:52.588+0000 7f425cc00700 1 -- 172.21.15.45:0/3562071552 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4250003970 con 0x7f426007de00 2024-04-16T16:55:00.699 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:52.588+0000 7f425cc00700 1 -- 172.21.15.45:0/3562071552 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f4250003b10 con 0x7f426007de00 2024-04-16T16:55:00.699 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:52.588+0000 7f425cc00700 1 -- 172.21.15.45:0/3562071552 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4250024610 con 0x7f426007de00 2024-04-16T16:55:00.699 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:52.588+0000 7f4264e00700 1 -- 172.21.15.45:0/3562071552 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f426019ed90 con 0x7f426007de00 2024-04-16T16:55:00.699 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:52.588+0000 7f4264e00700 1 -- 172.21.15.45:0/3562071552 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42601a1a80 con 0x7f426007de00 2024-04-16T16:55:00.699 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:52.589+0000 7f425cc00700 1 -- 172.21.15.45:0/3562071552 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f425001a9f0 con 0x7f426007de00 2024-04-16T16:55:00.699 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:52.589+0000 7f4264e00700 1 -- 172.21.15.45:0/3562071552 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4260064150 con 0x7f426007de00 2024-04-16T16:55:00.700 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:52.590+0000 7f425cc00700 1 --2- 172.21.15.45:0/3562071552 >> [v2:172.21.15.45:6800/1983630136,v1:172.21.15.45:6801/1983630136] conn(0x7f4244039600 0x7f424403bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:55:00.700 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:52.590+0000 7f425cc00700 1 -- 172.21.15.45:0/3562071552 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f4250011e00 con 0x7f426007de00 2024-04-16T16:55:00.700 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:52.591+0000 7f4256a00700 1 --2- 172.21.15.45:0/3562071552 >> [v2:172.21.15.45:6800/1983630136,v1:172.21.15.45:6801/1983630136] conn(0x7f4244039600 0x7f424403bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:55:00.700 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:52.591+0000 7f4256a00700 1 --2- 172.21.15.45:0/3562071552 >> [v2:172.21.15.45:6800/1983630136,v1:172.21.15.45:6801/1983630136] conn(0x7f4244039600 0x7f424403bac0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f4248006fd0 tx=0x7f4248006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T16:55:00.705 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:52.596+0000 7f425cc00700 1 -- 172.21.15.45:0/3562071552 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f425000b3e0 con 0x7f426007de00 2024-04-16T16:55:00.705 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:52.833+0000 7f4264e00700 1 -- 172.21.15.45:0/3562071552 --> [v2:172.21.15.45:6800/1983630136,v1:172.21.15.45:6801/1983630136] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi045", "addr": "172.21.15.45", "target": ["mon-mgr", ""]}) -- 0x7f4260104d10 con 0x7f4244039600 2024-04-16T16:55:00.705 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:59.805+0000 7f425cc00700 1 -- 172.21.15.45:0/3562071552 <== mgr.14120 v2:172.21.15.45:6800/1983630136 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7f4260104d10 con 0x7f4244039600 2024-04-16T16:55:00.705 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:59.809+0000 7f4264e00700 1 -- 172.21.15.45:0/3562071552 >> [v2:172.21.15.45:6800/1983630136,v1:172.21.15.45:6801/1983630136] conn(0x7f4244039600 msgr2=0x7f424403bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:55:00.705 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:59.810+0000 7f4264e00700 1 --2- 172.21.15.45:0/3562071552 >> [v2:172.21.15.45:6800/1983630136,v1:172.21.15.45:6801/1983630136] conn(0x7f4244039600 0x7f424403bac0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f4248006fd0 tx=0x7f4248006e40 comp rx=0 tx=0).stop 2024-04-16T16:55:00.706 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:59.810+0000 7f4264e00700 1 -- 172.21.15.45:0/3562071552 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f426007de00 msgr2=0x7f426019e5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:55:00.706 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:59.810+0000 7f4264e00700 1 --2- 172.21.15.45:0/3562071552 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f426007de00 0x7f426019e5c0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f4250011b60 tx=0x7f4250011c40 comp rx=0 tx=0).stop 2024-04-16T16:55:00.706 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:59.810+0000 7f4264e00700 1 -- 172.21.15.45:0/3562071552 shutdown_connections 2024-04-16T16:55:00.710 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:59.810+0000 7f4264e00700 1 --2- 172.21.15.45:0/3562071552 >> [v2:172.21.15.45:6800/1983630136,v1:172.21.15.45:6801/1983630136] conn(0x7f4244039600 0x7f424403bac0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:55:00.710 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:59.810+0000 7f4264e00700 1 --2- 172.21.15.45:0/3562071552 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f426007de00 0x7f426019e5c0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:55:00.710 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:59.810+0000 7f4264e00700 1 -- 172.21.15.45:0/3562071552 >> 172.21.15.45:0/3562071552 conn(0x7f42601039d0 msgr2=0x7f4260104600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T16:55:00.710 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:59.810+0000 7f4264e00700 1 -- 172.21.15.45:0/3562071552 shutdown_connections 2024-04-16T16:55:00.710 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:54:59.810+0000 7f4264e00700 1 -- 172.21.15.45:0/3562071552 wait complete. 2024-04-16T16:55:00.710 INFO:teuthology.orchestra.run.smithi045.stdout:Deploying unmanaged mon service... 2024-04-16T16:55:00.978 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:55:00 smithi045 ceph-mon[31676]: from='mgr.14120 172.21.15.45:0/1544209564' entity='mgr.x' 2024-04-16T16:55:00.978 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:55:00 smithi045 ceph-mon[31676]: Added host smithi045 2024-04-16T16:55:00.978 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:55:00 smithi045 ceph-mon[31676]: from='mgr.14120 172.21.15.45:0/1544209564' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T16:55:02.765 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-04-16T16:55:02.765 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:01.396+0000 7f21eb800700 1 Processor -- start 2024-04-16T16:55:02.766 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:01.397+0000 7f21eb800700 1 -- start start 2024-04-16T16:55:02.771 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:01.397+0000 7f21eb800700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f21e407de70 0x7f21e407c320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:55:02.771 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:01.397+0000 7f21eb800700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f21e407c8f0 con 0x7f21e407de70 2024-04-16T16:55:02.771 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:01.398+0000 7f21e8e00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f21e407de70 0x7f21e407c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:55:02.771 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:01.398+0000 7f21e8e00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f21e407de70 0x7f21e407c320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56624/0 (socket says 172.21.15.45:56624) 2024-04-16T16:55:02.771 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:01.398+0000 7f21e8e00700 1 -- 172.21.15.45:0/1055209388 learned_addr learned my addr 172.21.15.45:0/1055209388 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T16:55:02.771 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:01.398+0000 7f21e8e00700 1 -- 172.21.15.45:0/1055209388 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21e407ca70 con 0x7f21e407de70 2024-04-16T16:55:02.771 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:01.398+0000 7f21e8e00700 1 --2- 172.21.15.45:0/1055209388 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f21e407de70 0x7f21e407c320 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f21d80098a0 tx=0x7f21d8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd0fa224c680eee1 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T16:55:02.771 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:01.399+0000 7f21e3400700 1 -- 172.21.15.45:0/1055209388 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f21d8004160 con 0x7f21e407de70 2024-04-16T16:55:02.771 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:01.399+0000 7f21e3400700 1 -- 172.21.15.45:0/1055209388 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f21d8004300 con 0x7f21e407de70 2024-04-16T16:55:02.771 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:01.399+0000 7f21e3400700 1 -- 172.21.15.45:0/1055209388 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f21d8011b50 con 0x7f21e407de70 2024-04-16T16:55:02.771 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:01.399+0000 7f21eb800700 1 -- 172.21.15.45:0/1055209388 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f21e407de70 msgr2=0x7f21e407c320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:55:02.771 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:01.399+0000 7f21eb800700 1 --2- 172.21.15.45:0/1055209388 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f21e407de70 0x7f21e407c320 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f21d80098a0 tx=0x7f21d8009bb0 comp rx=0 tx=0).stop 2024-04-16T16:55:02.771 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:01.400+0000 7f21eb800700 1 -- 172.21.15.45:0/1055209388 shutdown_connections 2024-04-16T16:55:02.772 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:01.400+0000 7f21eb800700 1 --2- 172.21.15.45:0/1055209388 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f21e407de70 0x7f21e407c320 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:55:02.772 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:01.400+0000 7f21eb800700 1 -- 172.21.15.45:0/1055209388 >> 172.21.15.45:0/1055209388 conn(0x7f21e4103b90 msgr2=0x7f21e4105fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T16:55:02.772 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:01.400+0000 7f21eb800700 1 -- 172.21.15.45:0/1055209388 shutdown_connections 2024-04-16T16:55:02.772 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:01.400+0000 7f21eb800700 1 -- 172.21.15.45:0/1055209388 wait complete. 2024-04-16T16:55:02.772 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:01.401+0000 7f21eb800700 1 Processor -- start 2024-04-16T16:55:02.772 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:01.401+0000 7f21eb800700 1 -- start start 2024-04-16T16:55:02.772 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:01.402+0000 7f21eb800700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f21e407de70 0x7f21e419e700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:55:02.772 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:01.402+0000 7f21eb800700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f21e410eb60 con 0x7f21e407de70 2024-04-16T16:55:02.772 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:01.402+0000 7f21e8e00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f21e407de70 0x7f21e419e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:55:02.772 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:01.402+0000 7f21e8e00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f21e407de70 0x7f21e419e700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56628/0 (socket says 172.21.15.45:56628) 2024-04-16T16:55:02.772 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:01.402+0000 7f21e8e00700 1 -- 172.21.15.45:0/100739770 learned_addr learned my addr 172.21.15.45:0/100739770 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T16:55:02.772 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:01.403+0000 7f21e8e00700 1 -- 172.21.15.45:0/100739770 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21e419ec40 con 0x7f21e407de70 2024-04-16T16:55:02.772 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:01.403+0000 7f21e8e00700 1 --2- 172.21.15.45:0/100739770 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f21e407de70 0x7f21e419e700 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f21d8011b60 tx=0x7f21d8011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T16:55:02.772 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:01.403+0000 7f21e1600700 1 -- 172.21.15.45:0/100739770 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f21d8003b20 con 0x7f21e407de70 2024-04-16T16:55:02.773 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:01.403+0000 7f21e1600700 1 -- 172.21.15.45:0/100739770 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f21d8003cc0 con 0x7f21e407de70 2024-04-16T16:55:02.773 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:01.403+0000 7f21eb800700 1 -- 172.21.15.45:0/100739770 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21e419eed0 con 0x7f21e407de70 2024-04-16T16:55:02.773 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:01.403+0000 7f21eb800700 1 -- 172.21.15.45:0/100739770 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21e41a1bc0 con 0x7f21e407de70 2024-04-16T16:55:02.773 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:01.404+0000 7f21e1600700 1 -- 172.21.15.45:0/100739770 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f21d8024840 con 0x7f21e407de70 2024-04-16T16:55:02.773 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:01.405+0000 7f21eb800700 1 -- 172.21.15.45:0/100739770 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f21c4005360 con 0x7f21e407de70 2024-04-16T16:55:02.773 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:01.405+0000 7f21e1600700 1 -- 172.21.15.45:0/100739770 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f21d8022020 con 0x7f21e407de70 2024-04-16T16:55:02.773 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:01.405+0000 7f21e1600700 1 --2- 172.21.15.45:0/100739770 >> [v2:172.21.15.45:6800/1983630136,v1:172.21.15.45:6801/1983630136] conn(0x7f21cc0395b0 0x7f21cc03ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:55:02.773 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:01.405+0000 7f21e1600700 1 -- 172.21.15.45:0/100739770 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f21d8011e00 con 0x7f21e407de70 2024-04-16T16:55:02.773 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:01.406+0000 7f21e3e00700 1 --2- 172.21.15.45:0/100739770 >> [v2:172.21.15.45:6800/1983630136,v1:172.21.15.45:6801/1983630136] conn(0x7f21cc0395b0 0x7f21cc03ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:55:02.773 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:01.406+0000 7f21e3e00700 1 --2- 172.21.15.45:0/100739770 >> [v2:172.21.15.45:6800/1983630136,v1:172.21.15.45:6801/1983630136] conn(0x7f21cc0395b0 0x7f21cc03ba70 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f21d0006fd0 tx=0x7f21d0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T16:55:02.773 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:01.411+0000 7f21e1600700 1 -- 172.21.15.45:0/100739770 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f21d804d1d0 con 0x7f21e407de70 2024-04-16T16:55:02.773 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:01.668+0000 7f21eb800700 1 -- 172.21.15.45:0/100739770 --> [v2:172.21.15.45:6800/1983630136,v1:172.21.15.45:6801/1983630136] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f21c4000bf0 con 0x7f21cc0395b0 2024-04-16T16:55:02.773 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:01.678+0000 7f21e1600700 1 -- 172.21.15.45:0/100739770 <== mgr.14120 v2:172.21.15.45:6800/1983630136 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f21c4000bf0 con 0x7f21cc0395b0 2024-04-16T16:55:02.773 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:01.681+0000 7f21eb800700 1 -- 172.21.15.45:0/100739770 >> [v2:172.21.15.45:6800/1983630136,v1:172.21.15.45:6801/1983630136] conn(0x7f21cc0395b0 msgr2=0x7f21cc03ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:55:02.773 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:01.681+0000 7f21eb800700 1 --2- 172.21.15.45:0/100739770 >> [v2:172.21.15.45:6800/1983630136,v1:172.21.15.45:6801/1983630136] conn(0x7f21cc0395b0 0x7f21cc03ba70 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f21d0006fd0 tx=0x7f21d0006e40 comp rx=0 tx=0).stop 2024-04-16T16:55:02.774 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:01.681+0000 7f21eb800700 1 -- 172.21.15.45:0/100739770 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f21e407de70 msgr2=0x7f21e419e700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:55:02.774 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:01.681+0000 7f21eb800700 1 --2- 172.21.15.45:0/100739770 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f21e407de70 0x7f21e419e700 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f21d8011b60 tx=0x7f21d8011c40 comp rx=0 tx=0).stop 2024-04-16T16:55:02.774 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:01.681+0000 7f21eb800700 1 -- 172.21.15.45:0/100739770 shutdown_connections 2024-04-16T16:55:02.774 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:01.681+0000 7f21eb800700 1 --2- 172.21.15.45:0/100739770 >> [v2:172.21.15.45:6800/1983630136,v1:172.21.15.45:6801/1983630136] conn(0x7f21cc0395b0 0x7f21cc03ba70 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:55:02.774 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:01.681+0000 7f21eb800700 1 --2- 172.21.15.45:0/100739770 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f21e407de70 0x7f21e419e700 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:55:02.774 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:01.681+0000 7f21eb800700 1 -- 172.21.15.45:0/100739770 >> 172.21.15.45:0/100739770 conn(0x7f21e4103b90 msgr2=0x7f21e4104710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T16:55:02.774 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:01.681+0000 7f21eb800700 1 -- 172.21.15.45:0/100739770 shutdown_connections 2024-04-16T16:55:02.774 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:01.681+0000 7f21eb800700 1 -- 172.21.15.45:0/100739770 wait complete. 2024-04-16T16:55:02.774 INFO:teuthology.orchestra.run.smithi045.stdout:Deploying unmanaged mgr service... 2024-04-16T16:55:02.978 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:55:02 smithi045 ceph-mon[31676]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T16:55:02.978 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:55:02 smithi045 ceph-mon[31676]: Saving service mon spec with placement count:5 2024-04-16T16:55:02.978 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:55:02 smithi045 ceph-mon[31676]: from='mgr.14120 172.21.15.45:0/1544209564' entity='mgr.x' 2024-04-16T16:55:04.685 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:55:04 smithi045 ceph-mon[31676]: from='mgr.14120 172.21.15.45:0/1544209564' entity='mgr.x' 2024-04-16T16:55:04.685 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:55:04 smithi045 ceph-mon[31676]: from='mgr.14120 172.21.15.45:0/1544209564' entity='mgr.x' 2024-04-16T16:55:04.686 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:55:04 smithi045 ceph-mon[31676]: from='mgr.14120 172.21.15.45:0/1544209564' entity='mgr.x' 2024-04-16T16:55:04.986 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-04-16T16:55:04.986 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:04.048+0000 7f1153000700 1 Processor -- start 2024-04-16T16:55:04.986 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:04.049+0000 7f1153000700 1 -- start start 2024-04-16T16:55:04.986 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:04.049+0000 7f1153000700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f114c106ea0 0x7f114c1072c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:55:04.986 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:04.049+0000 7f1153000700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f114c07be30 con 0x7f114c106ea0 2024-04-16T16:55:04.986 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:04.049+0000 7f114be00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f114c106ea0 0x7f114c1072c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:55:04.986 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:04.049+0000 7f114be00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f114c106ea0 0x7f114c1072c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56634/0 (socket says 172.21.15.45:56634) 2024-04-16T16:55:04.986 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:04.049+0000 7f114be00700 1 -- 172.21.15.45:0/2392288465 learned_addr learned my addr 172.21.15.45:0/2392288465 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T16:55:04.987 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:04.050+0000 7f114be00700 1 -- 172.21.15.45:0/2392288465 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f114c107800 con 0x7f114c106ea0 2024-04-16T16:55:04.987 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:04.050+0000 7f114be00700 1 --2- 172.21.15.45:0/2392288465 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f114c106ea0 0x7f114c1072c0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f113c009e80 tx=0x7f113c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8485bab5bd352c71 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T16:55:04.987 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:04.050+0000 7f114aa00700 1 -- 172.21.15.45:0/2392288465 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f113c017070 con 0x7f114c106ea0 2024-04-16T16:55:04.987 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:04.051+0000 7f114aa00700 1 -- 172.21.15.45:0/2392288465 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f113c00b8c0 con 0x7f114c106ea0 2024-04-16T16:55:04.987 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:04.051+0000 7f114aa00700 1 -- 172.21.15.45:0/2392288465 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f113c0045c0 con 0x7f114c106ea0 2024-04-16T16:55:04.987 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:04.051+0000 7f1153000700 1 -- 172.21.15.45:0/2392288465 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f114c106ea0 msgr2=0x7f114c1072c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:55:04.987 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:04.051+0000 7f1153000700 1 --2- 172.21.15.45:0/2392288465 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f114c106ea0 0x7f114c1072c0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f113c009e80 tx=0x7f113c00b330 comp rx=0 tx=0).stop 2024-04-16T16:55:04.987 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:04.051+0000 7f1153000700 1 -- 172.21.15.45:0/2392288465 shutdown_connections 2024-04-16T16:55:04.987 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:04.051+0000 7f1153000700 1 --2- 172.21.15.45:0/2392288465 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f114c106ea0 0x7f114c1072c0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:55:04.987 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:04.051+0000 7f1153000700 1 -- 172.21.15.45:0/2392288465 >> 172.21.15.45:0/2392288465 conn(0x7f114c102a60 msgr2=0x7f114c104ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T16:55:04.987 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:04.052+0000 7f1153000700 1 -- 172.21.15.45:0/2392288465 shutdown_connections 2024-04-16T16:55:04.987 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:04.052+0000 7f1153000700 1 -- 172.21.15.45:0/2392288465 wait complete. 2024-04-16T16:55:04.987 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:04.053+0000 7f1153000700 1 Processor -- start 2024-04-16T16:55:04.987 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:04.053+0000 7f1153000700 1 -- start start 2024-04-16T16:55:04.988 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:04.054+0000 7f1153000700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f114c106ea0 0x7f114c19e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:55:04.988 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:04.054+0000 7f1153000700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f114c1088d0 con 0x7f114c106ea0 2024-04-16T16:55:04.988 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:04.054+0000 7f114be00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f114c106ea0 0x7f114c19e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:55:04.988 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:04.054+0000 7f114be00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f114c106ea0 0x7f114c19e6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56642/0 (socket says 172.21.15.45:56642) 2024-04-16T16:55:04.988 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:04.054+0000 7f114be00700 1 -- 172.21.15.45:0/1850439890 learned_addr learned my addr 172.21.15.45:0/1850439890 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T16:55:04.988 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:04.055+0000 7f114be00700 1 -- 172.21.15.45:0/1850439890 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f114c19ec30 con 0x7f114c106ea0 2024-04-16T16:55:04.988 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:04.055+0000 7f114be00700 1 --2- 172.21.15.45:0/1850439890 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f114c106ea0 0x7f114c19e6f0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f113c009fd0 tx=0x7f113c003d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T16:55:04.988 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:04.055+0000 7f1148c00700 1 -- 172.21.15.45:0/1850439890 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f113c020020 con 0x7f114c106ea0 2024-04-16T16:55:04.988 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:04.055+0000 7f1148c00700 1 -- 172.21.15.45:0/1850439890 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f113c00bba0 con 0x7f114c106ea0 2024-04-16T16:55:04.988 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:04.055+0000 7f1148c00700 1 -- 172.21.15.45:0/1850439890 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f113c017040 con 0x7f114c106ea0 2024-04-16T16:55:04.988 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:04.055+0000 7f1153000700 1 -- 172.21.15.45:0/1850439890 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f114c19eec0 con 0x7f114c106ea0 2024-04-16T16:55:04.988 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:04.055+0000 7f1153000700 1 -- 172.21.15.45:0/1850439890 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f114c1a1bb0 con 0x7f114c106ea0 2024-04-16T16:55:04.988 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:04.057+0000 7f1153000700 1 -- 172.21.15.45:0/1850439890 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f114c064150 con 0x7f114c106ea0 2024-04-16T16:55:04.988 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:04.057+0000 7f1148c00700 1 -- 172.21.15.45:0/1850439890 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f113c023020 con 0x7f114c106ea0 2024-04-16T16:55:04.989 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:04.058+0000 7f1148c00700 1 --2- 172.21.15.45:0/1850439890 >> [v2:172.21.15.45:6800/1983630136,v1:172.21.15.45:6801/1983630136] conn(0x7f1130039600 0x7f113003bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:55:04.989 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:04.058+0000 7f1148c00700 1 -- 172.21.15.45:0/1850439890 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f113c00f050 con 0x7f114c106ea0 2024-04-16T16:55:04.989 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:04.058+0000 7f114b400700 1 --2- 172.21.15.45:0/1850439890 >> [v2:172.21.15.45:6800/1983630136,v1:172.21.15.45:6801/1983630136] conn(0x7f1130039600 0x7f113003bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:55:04.989 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:04.059+0000 7f114b400700 1 --2- 172.21.15.45:0/1850439890 >> [v2:172.21.15.45:6800/1983630136,v1:172.21.15.45:6801/1983630136] conn(0x7f1130039600 0x7f113003bac0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f1138006fd0 tx=0x7f1138006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T16:55:04.989 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:04.065+0000 7f1148c00700 1 -- 172.21.15.45:0/1850439890 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f113c02a020 con 0x7f114c106ea0 2024-04-16T16:55:04.989 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:04.316+0000 7f1153000700 1 -- 172.21.15.45:0/1850439890 --> [v2:172.21.15.45:6800/1983630136,v1:172.21.15.45:6801/1983630136] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f114c02e290 con 0x7f1130039600 2024-04-16T16:55:04.989 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:04.323+0000 7f1148c00700 1 -- 172.21.15.45:0/1850439890 <== mgr.14120 v2:172.21.15.45:6800/1983630136 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f114c02e290 con 0x7f1130039600 2024-04-16T16:55:04.989 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:04.326+0000 7f1153000700 1 -- 172.21.15.45:0/1850439890 >> [v2:172.21.15.45:6800/1983630136,v1:172.21.15.45:6801/1983630136] conn(0x7f1130039600 msgr2=0x7f113003bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:55:04.989 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:04.326+0000 7f1153000700 1 --2- 172.21.15.45:0/1850439890 >> [v2:172.21.15.45:6800/1983630136,v1:172.21.15.45:6801/1983630136] conn(0x7f1130039600 0x7f113003bac0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f1138006fd0 tx=0x7f1138006e40 comp rx=0 tx=0).stop 2024-04-16T16:55:04.989 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:04.326+0000 7f1153000700 1 -- 172.21.15.45:0/1850439890 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f114c106ea0 msgr2=0x7f114c19e6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:55:04.989 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:04.326+0000 7f1153000700 1 --2- 172.21.15.45:0/1850439890 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f114c106ea0 0x7f114c19e6f0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f113c009fd0 tx=0x7f113c003d00 comp rx=0 tx=0).stop 2024-04-16T16:55:04.989 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:04.327+0000 7f1153000700 1 -- 172.21.15.45:0/1850439890 shutdown_connections 2024-04-16T16:55:04.989 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:04.327+0000 7f1153000700 1 --2- 172.21.15.45:0/1850439890 >> [v2:172.21.15.45:6800/1983630136,v1:172.21.15.45:6801/1983630136] conn(0x7f1130039600 0x7f113003bac0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:55:04.990 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:04.327+0000 7f1153000700 1 --2- 172.21.15.45:0/1850439890 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f114c106ea0 0x7f114c19e6f0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:55:04.990 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:04.327+0000 7f1153000700 1 -- 172.21.15.45:0/1850439890 >> 172.21.15.45:0/1850439890 conn(0x7f114c102a60 msgr2=0x7f114c195360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T16:55:04.990 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:04.327+0000 7f1153000700 1 -- 172.21.15.45:0/1850439890 shutdown_connections 2024-04-16T16:55:04.990 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:04.327+0000 7f1153000700 1 -- 172.21.15.45:0/1850439890 wait complete. 2024-04-16T16:55:05.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:55:05 smithi045 ceph-mon[31676]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:55:05.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:55:05 smithi045 ceph-mon[31676]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T16:55:05.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:55:05 smithi045 ceph-mon[31676]: Saving service mgr spec with placement count:2 2024-04-16T16:55:06.591 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:05.545+0000 7fdf1ec00700 1 Processor -- start 2024-04-16T16:55:06.592 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:05.545+0000 7fdf1ec00700 1 -- start start 2024-04-16T16:55:06.592 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:05.545+0000 7fdf1ec00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdf1807de30 0x7fdf1807c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:55:06.592 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:05.545+0000 7fdf1ec00700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fdf1807c820 con 0x7fdf1807de30 2024-04-16T16:55:06.593 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:05.546+0000 7fdf17e00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdf1807de30 0x7fdf1807c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:55:06.593 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:05.546+0000 7fdf17e00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdf1807de30 0x7fdf1807c2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56650/0 (socket says 172.21.15.45:56650) 2024-04-16T16:55:06.594 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:05.546+0000 7fdf17e00700 1 -- 172.21.15.45:0/2815503218 learned_addr learned my addr 172.21.15.45:0/2815503218 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T16:55:06.594 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:05.547+0000 7fdf17e00700 1 -- 172.21.15.45:0/2815503218 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf1807c9a0 con 0x7fdf1807de30 2024-04-16T16:55:06.594 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:05.547+0000 7fdf17e00700 1 --2- 172.21.15.45:0/2815503218 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdf1807de30 0x7fdf1807c2e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fdf080098a0 tx=0x7fdf08009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fe0c039d955f32d server_cookie=0 in_seq=0 out_seq=0 2024-04-16T16:55:06.594 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:05.548+0000 7fdf16a00700 1 -- 172.21.15.45:0/2815503218 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdf08004160 con 0x7fdf1807de30 2024-04-16T16:55:06.594 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:05.548+0000 7fdf16a00700 1 -- 172.21.15.45:0/2815503218 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdf08004300 con 0x7fdf1807de30 2024-04-16T16:55:06.594 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:05.548+0000 7fdf16a00700 1 -- 172.21.15.45:0/2815503218 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdf08011b20 con 0x7fdf1807de30 2024-04-16T16:55:06.594 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:05.548+0000 7fdf1ec00700 1 -- 172.21.15.45:0/2815503218 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdf1807de30 msgr2=0x7fdf1807c2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:55:06.595 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:05.548+0000 7fdf1ec00700 1 --2- 172.21.15.45:0/2815503218 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdf1807de30 0x7fdf1807c2e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fdf080098a0 tx=0x7fdf08009bb0 comp rx=0 tx=0).stop 2024-04-16T16:55:06.595 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:05.549+0000 7fdf1ec00700 1 -- 172.21.15.45:0/2815503218 shutdown_connections 2024-04-16T16:55:06.595 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:05.549+0000 7fdf1ec00700 1 --2- 172.21.15.45:0/2815503218 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdf1807de30 0x7fdf1807c2e0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:55:06.595 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:05.549+0000 7fdf1ec00700 1 -- 172.21.15.45:0/2815503218 >> 172.21.15.45:0/2815503218 conn(0x7fdf18103b90 msgr2=0x7fdf18105fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T16:55:06.595 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:05.549+0000 7fdf1ec00700 1 -- 172.21.15.45:0/2815503218 shutdown_connections 2024-04-16T16:55:06.597 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:05.549+0000 7fdf1ec00700 1 -- 172.21.15.45:0/2815503218 wait complete. 2024-04-16T16:55:06.597 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:05.550+0000 7fdf1ec00700 1 Processor -- start 2024-04-16T16:55:06.597 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:05.551+0000 7fdf1ec00700 1 -- start start 2024-04-16T16:55:06.597 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:05.551+0000 7fdf1ec00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdf1807de30 0x7fdf1819e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:55:06.597 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:05.551+0000 7fdf1ec00700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fdf1810eb90 con 0x7fdf1807de30 2024-04-16T16:55:06.597 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:05.551+0000 7fdf17e00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdf1807de30 0x7fdf1819e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:55:06.597 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:05.551+0000 7fdf17e00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdf1807de30 0x7fdf1819e6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56660/0 (socket says 172.21.15.45:56660) 2024-04-16T16:55:06.597 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:05.551+0000 7fdf17e00700 1 -- 172.21.15.45:0/1812396842 learned_addr learned my addr 172.21.15.45:0/1812396842 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T16:55:06.597 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:05.552+0000 7fdf17e00700 1 -- 172.21.15.45:0/1812396842 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf1819ebf0 con 0x7fdf1807de30 2024-04-16T16:55:06.597 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:05.552+0000 7fdf17e00700 1 --2- 172.21.15.45:0/1812396842 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdf1807de30 0x7fdf1819e6b0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fdf080117e0 tx=0x7fdf08011ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T16:55:06.597 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:05.552+0000 7fdf14c00700 1 -- 172.21.15.45:0/1812396842 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdf0801a430 con 0x7fdf1807de30 2024-04-16T16:55:06.597 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:05.552+0000 7fdf14c00700 1 -- 172.21.15.45:0/1812396842 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdf0801aab0 con 0x7fdf1807de30 2024-04-16T16:55:06.597 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:05.553+0000 7fdf1ec00700 1 -- 172.21.15.45:0/1812396842 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf1819ee80 con 0x7fdf1807de30 2024-04-16T16:55:06.600 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:05.553+0000 7fdf14c00700 1 -- 172.21.15.45:0/1812396842 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdf08024a60 con 0x7fdf1807de30 2024-04-16T16:55:06.600 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:05.553+0000 7fdf1ec00700 1 -- 172.21.15.45:0/1812396842 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf1819f2e0 con 0x7fdf1807de30 2024-04-16T16:55:06.600 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:05.554+0000 7fdf14c00700 1 -- 172.21.15.45:0/1812396842 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7fdf08024d10 con 0x7fdf1807de30 2024-04-16T16:55:06.600 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:05.554+0000 7fdf1ec00700 1 -- 172.21.15.45:0/1812396842 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf18064150 con 0x7fdf1807de30 2024-04-16T16:55:06.600 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:05.555+0000 7fdf14c00700 1 --2- 172.21.15.45:0/1812396842 >> [v2:172.21.15.45:6800/1983630136,v1:172.21.15.45:6801/1983630136] conn(0x7fdefc0395b0 0x7fdefc03ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:55:06.600 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:05.555+0000 7fdf14c00700 1 -- 172.21.15.45:0/1812396842 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fdf0804df00 con 0x7fdf1807de30 2024-04-16T16:55:06.600 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:05.555+0000 7fdf17400700 1 --2- 172.21.15.45:0/1812396842 >> [v2:172.21.15.45:6800/1983630136,v1:172.21.15.45:6801/1983630136] conn(0x7fdefc0395b0 0x7fdefc03ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:55:06.600 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:05.556+0000 7fdf17400700 1 --2- 172.21.15.45:0/1812396842 >> [v2:172.21.15.45:6800/1983630136,v1:172.21.15.45:6801/1983630136] conn(0x7fdefc0395b0 0x7fdefc03ba70 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fdf04006fd0 tx=0x7fdf04006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T16:55:06.600 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:05.561+0000 7fdf14c00700 1 -- 172.21.15.45:0/1812396842 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fdf0802cdf0 con 0x7fdf1807de30 2024-04-16T16:55:06.600 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:05.800+0000 7fdf1ec00700 1 -- 172.21.15.45:0/1812396842 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) -- 0x7fdf181a1d70 con 0x7fdf1807de30 2024-04-16T16:55:06.600 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:05.805+0000 7fdf14c00700 1 -- 172.21.15.45:0/1812396842 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) ==== 142+0+0 (secure 0 0 0) 0x7fdf08020c10 con 0x7fdf1807de30 2024-04-16T16:55:06.600 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:05.809+0000 7fdf1ec00700 1 -- 172.21.15.45:0/1812396842 >> [v2:172.21.15.45:6800/1983630136,v1:172.21.15.45:6801/1983630136] conn(0x7fdefc0395b0 msgr2=0x7fdefc03ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:55:06.601 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:05.809+0000 7fdf1ec00700 1 --2- 172.21.15.45:0/1812396842 >> [v2:172.21.15.45:6800/1983630136,v1:172.21.15.45:6801/1983630136] conn(0x7fdefc0395b0 0x7fdefc03ba70 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fdf04006fd0 tx=0x7fdf04006e40 comp rx=0 tx=0).stop 2024-04-16T16:55:06.601 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:05.809+0000 7fdf1ec00700 1 -- 172.21.15.45:0/1812396842 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdf1807de30 msgr2=0x7fdf1819e6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:55:06.601 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:05.809+0000 7fdf1ec00700 1 --2- 172.21.15.45:0/1812396842 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdf1807de30 0x7fdf1819e6b0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fdf080117e0 tx=0x7fdf08011ce0 comp rx=0 tx=0).stop 2024-04-16T16:55:06.601 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:05.809+0000 7fdf1ec00700 1 -- 172.21.15.45:0/1812396842 shutdown_connections 2024-04-16T16:55:06.601 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:05.809+0000 7fdf1ec00700 1 --2- 172.21.15.45:0/1812396842 >> [v2:172.21.15.45:6800/1983630136,v1:172.21.15.45:6801/1983630136] conn(0x7fdefc0395b0 0x7fdefc03ba70 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:55:06.601 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:05.809+0000 7fdf1ec00700 1 --2- 172.21.15.45:0/1812396842 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdf1807de30 0x7fdf1819e6b0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:55:06.601 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:05.809+0000 7fdf1ec00700 1 -- 172.21.15.45:0/1812396842 >> 172.21.15.45:0/1812396842 conn(0x7fdf18103b90 msgr2=0x7fdf18104680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T16:55:06.601 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:05.809+0000 7fdf1ec00700 1 -- 172.21.15.45:0/1812396842 shutdown_connections 2024-04-16T16:55:06.601 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:05.809+0000 7fdf1ec00700 1 -- 172.21.15.45:0/1812396842 wait complete. 2024-04-16T16:55:06.978 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:55:06 smithi045 ceph-mon[31676]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:55:06.978 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:55:06 smithi045 ceph-mon[31676]: from='client.? 172.21.15.45:0/1812396842' entity='client.admin' 2024-04-16T16:55:08.636 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:07.615+0000 7f06dbc00700 1 Processor -- start 2024-04-16T16:55:08.637 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:07.615+0000 7f06dbc00700 1 -- start start 2024-04-16T16:55:08.637 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:07.615+0000 7f06dbc00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f06d410a7e0 0x7f06d410ac00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:55:08.637 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:07.615+0000 7f06dbc00700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f06d410b1d0 con 0x7f06d410a7e0 2024-04-16T16:55:08.637 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:07.616+0000 7f06d9200700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f06d410a7e0 0x7f06d410ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:55:08.637 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:07.616+0000 7f06d9200700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f06d410a7e0 0x7f06d410ac00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56666/0 (socket says 172.21.15.45:56666) 2024-04-16T16:55:08.637 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:07.616+0000 7f06d9200700 1 -- 172.21.15.45:0/901657737 learned_addr learned my addr 172.21.15.45:0/901657737 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T16:55:08.637 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:07.616+0000 7f06d9200700 1 -- 172.21.15.45:0/901657737 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06d410b990 con 0x7f06d410a7e0 2024-04-16T16:55:08.637 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:07.617+0000 7f06d9200700 1 --2- 172.21.15.45:0/901657737 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f06d410a7e0 0x7f06d410ac00 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f06c80098a0 tx=0x7f06c8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7c6fb45f21d4487 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T16:55:08.637 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:07.617+0000 7f06d3400700 1 -- 172.21.15.45:0/901657737 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f06c8004160 con 0x7f06d410a7e0 2024-04-16T16:55:08.637 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:07.617+0000 7f06d3400700 1 -- 172.21.15.45:0/901657737 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f06c8004300 con 0x7f06d410a7e0 2024-04-16T16:55:08.637 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:07.617+0000 7f06d3400700 1 -- 172.21.15.45:0/901657737 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f06c8011b20 con 0x7f06d410a7e0 2024-04-16T16:55:08.637 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:07.618+0000 7f06dbc00700 1 -- 172.21.15.45:0/901657737 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f06d410a7e0 msgr2=0x7f06d410ac00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:55:08.637 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:07.618+0000 7f06dbc00700 1 --2- 172.21.15.45:0/901657737 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f06d410a7e0 0x7f06d410ac00 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f06c80098a0 tx=0x7f06c8009bb0 comp rx=0 tx=0).stop 2024-04-16T16:55:08.638 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:07.618+0000 7f06dbc00700 1 -- 172.21.15.45:0/901657737 shutdown_connections 2024-04-16T16:55:08.638 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:07.618+0000 7f06dbc00700 1 --2- 172.21.15.45:0/901657737 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f06d410a7e0 0x7f06d410ac00 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:55:08.638 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:07.618+0000 7f06dbc00700 1 -- 172.21.15.45:0/901657737 >> 172.21.15.45:0/901657737 conn(0x7f06d407dd80 msgr2=0x7f06d41084d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T16:55:08.638 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:07.618+0000 7f06dbc00700 1 -- 172.21.15.45:0/901657737 shutdown_connections 2024-04-16T16:55:08.638 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:07.618+0000 7f06dbc00700 1 -- 172.21.15.45:0/901657737 wait complete. 2024-04-16T16:55:08.638 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:07.619+0000 7f06dbc00700 1 Processor -- start 2024-04-16T16:55:08.638 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:07.620+0000 7f06dbc00700 1 -- start start 2024-04-16T16:55:08.638 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:07.620+0000 7f06dbc00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f06d410a7e0 0x7f06d4081b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:55:08.638 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:07.620+0000 7f06dbc00700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f06d410ed40 con 0x7f06d410a7e0 2024-04-16T16:55:08.638 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:07.620+0000 7f06d9200700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f06d410a7e0 0x7f06d4081b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:55:08.638 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:07.620+0000 7f06d9200700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f06d410a7e0 0x7f06d4081b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56668/0 (socket says 172.21.15.45:56668) 2024-04-16T16:55:08.638 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:07.620+0000 7f06d9200700 1 -- 172.21.15.45:0/257535282 learned_addr learned my addr 172.21.15.45:0/257535282 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T16:55:08.638 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:07.621+0000 7f06d9200700 1 -- 172.21.15.45:0/257535282 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06d40820c0 con 0x7f06d410a7e0 2024-04-16T16:55:08.638 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:07.621+0000 7f06d9200700 1 --2- 172.21.15.45:0/257535282 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f06d410a7e0 0x7f06d4081b80 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f06c80117e0 tx=0x7f06c8011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T16:55:08.638 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:07.621+0000 7f06d1600700 1 -- 172.21.15.45:0/257535282 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f06c8003bf0 con 0x7f06d410a7e0 2024-04-16T16:55:08.639 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:07.621+0000 7f06d1600700 1 -- 172.21.15.45:0/257535282 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f06c801a910 con 0x7f06d410a7e0 2024-04-16T16:55:08.639 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:07.621+0000 7f06d1600700 1 -- 172.21.15.45:0/257535282 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f06c80249a0 con 0x7f06d410a7e0 2024-04-16T16:55:08.639 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:07.621+0000 7f06dbc00700 1 -- 172.21.15.45:0/257535282 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06d4082350 con 0x7f06d410a7e0 2024-04-16T16:55:08.639 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:07.622+0000 7f06dbc00700 1 -- 172.21.15.45:0/257535282 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06d407e890 con 0x7f06d410a7e0 2024-04-16T16:55:08.639 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:07.623+0000 7f06d1600700 1 -- 172.21.15.45:0/257535282 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f06c8022020 con 0x7f06d410a7e0 2024-04-16T16:55:08.639 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:07.623+0000 7f06dbc00700 1 -- 172.21.15.45:0/257535282 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06d4082520 con 0x7f06d410a7e0 2024-04-16T16:55:08.639 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:07.624+0000 7f06d1600700 1 --2- 172.21.15.45:0/257535282 >> [v2:172.21.15.45:6800/1983630136,v1:172.21.15.45:6801/1983630136] conn(0x7f06bc0395b0 0x7f06bc03ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:55:08.639 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:07.624+0000 7f06d1600700 1 -- 172.21.15.45:0/257535282 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f06c804d9a0 con 0x7f06d410a7e0 2024-04-16T16:55:08.639 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:07.624+0000 7f06d3e00700 1 --2- 172.21.15.45:0/257535282 >> [v2:172.21.15.45:6800/1983630136,v1:172.21.15.45:6801/1983630136] conn(0x7f06bc0395b0 0x7f06bc03ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:55:08.639 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:07.625+0000 7f06d3e00700 1 --2- 172.21.15.45:0/257535282 >> [v2:172.21.15.45:6800/1983630136,v1:172.21.15.45:6801/1983630136] conn(0x7f06bc0395b0 0x7f06bc03ba70 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f06c0006fd0 tx=0x7f06c0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T16:55:08.639 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:07.629+0000 7f06d1600700 1 -- 172.21.15.45:0/257535282 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f06d4082520 con 0x7f06d410a7e0 2024-04-16T16:55:08.639 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:07.863+0000 7f06dbc00700 1 -- 172.21.15.45:0/257535282 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) -- 0x7f06d407eea0 con 0x7f06d410a7e0 2024-04-16T16:55:08.639 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:07.869+0000 7f06d1600700 1 -- 172.21.15.45:0/257535282 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) ==== 130+0+0 (secure 0 0 0) 0x7f06c8011e20 con 0x7f06d410a7e0 2024-04-16T16:55:08.639 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:07.873+0000 7f06dbc00700 1 -- 172.21.15.45:0/257535282 >> [v2:172.21.15.45:6800/1983630136,v1:172.21.15.45:6801/1983630136] conn(0x7f06bc0395b0 msgr2=0x7f06bc03ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:55:08.640 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:07.873+0000 7f06dbc00700 1 --2- 172.21.15.45:0/257535282 >> [v2:172.21.15.45:6800/1983630136,v1:172.21.15.45:6801/1983630136] conn(0x7f06bc0395b0 0x7f06bc03ba70 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f06c0006fd0 tx=0x7f06c0006e40 comp rx=0 tx=0).stop 2024-04-16T16:55:08.640 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:07.873+0000 7f06dbc00700 1 -- 172.21.15.45:0/257535282 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f06d410a7e0 msgr2=0x7f06d4081b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:55:08.640 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:07.873+0000 7f06dbc00700 1 --2- 172.21.15.45:0/257535282 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f06d410a7e0 0x7f06d4081b80 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f06c80117e0 tx=0x7f06c8011c10 comp rx=0 tx=0).stop 2024-04-16T16:55:08.640 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:07.873+0000 7f06dbc00700 1 -- 172.21.15.45:0/257535282 shutdown_connections 2024-04-16T16:55:08.640 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:07.873+0000 7f06dbc00700 1 --2- 172.21.15.45:0/257535282 >> [v2:172.21.15.45:6800/1983630136,v1:172.21.15.45:6801/1983630136] conn(0x7f06bc0395b0 0x7f06bc03ba70 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:55:08.640 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:07.873+0000 7f06dbc00700 1 --2- 172.21.15.45:0/257535282 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f06d410a7e0 0x7f06d4081b80 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:55:08.640 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:07.873+0000 7f06dbc00700 1 -- 172.21.15.45:0/257535282 >> 172.21.15.45:0/257535282 conn(0x7f06d407dd80 msgr2=0x7f06d4107ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T16:55:08.640 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:07.873+0000 7f06dbc00700 1 -- 172.21.15.45:0/257535282 shutdown_connections 2024-04-16T16:55:08.640 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:07.873+0000 7f06dbc00700 1 -- 172.21.15.45:0/257535282 wait complete. 2024-04-16T16:55:08.640 INFO:teuthology.orchestra.run.smithi045.stdout:Enabling the dashboard module... 2024-04-16T16:55:08.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:55:08 smithi045 ceph-mon[31676]: from='mgr.14120 172.21.15.45:0/1544209564' entity='mgr.x' 2024-04-16T16:55:08.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:55:08 smithi045 ceph-mon[31676]: from='client.? 172.21.15.45:0/257535282' entity='client.admin' 2024-04-16T16:55:09.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:55:09 smithi045 ceph-mon[31676]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:55:10.978 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:55:10 smithi045 ceph-mon[31676]: from='mgr.14120 172.21.15.45:0/1544209564' entity='mgr.x' 2024-04-16T16:55:10.978 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:55:10 smithi045 ceph-mon[31676]: from='client.? 172.21.15.45:0/1730277990' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-04-16T16:55:10.978 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:55:10 smithi045 ceph-mon[31676]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:55:10.979 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 16 16:55:10 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-x[31905]: ignoring --setuser ceph since I am not root 2024-04-16T16:55:10.979 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 16 16:55:10 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-x[31905]: ignoring --setgroup ceph since I am not root 2024-04-16T16:55:11.429 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:09.322+0000 7fbae3200700 1 Processor -- start 2024-04-16T16:55:11.429 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:09.323+0000 7fbae3200700 1 -- start start 2024-04-16T16:55:11.429 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:09.323+0000 7fbae3200700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbae407ddc0 0x7fbae407c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:55:11.429 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:09.323+0000 7fbae3200700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fbae407c800 con 0x7fbae407ddc0 2024-04-16T16:55:11.429 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:09.324+0000 7fbae1e00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbae407ddc0 0x7fbae407c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:55:11.429 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:09.324+0000 7fbae1e00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbae407ddc0 0x7fbae407c2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56682/0 (socket says 172.21.15.45:56682) 2024-04-16T16:55:11.430 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:09.324+0000 7fbae1e00700 1 -- 172.21.15.45:0/1382556809 learned_addr learned my addr 172.21.15.45:0/1382556809 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T16:55:11.430 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:09.324+0000 7fbae1e00700 1 -- 172.21.15.45:0/1382556809 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbae407c980 con 0x7fbae407ddc0 2024-04-16T16:55:11.430 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:09.324+0000 7fbae1e00700 1 --2- 172.21.15.45:0/1382556809 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbae407ddc0 0x7fbae407c2c0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fbad40098a0 tx=0x7fbad4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c73aadce9e0e6e8 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T16:55:11.430 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:09.325+0000 7fbae0a00700 1 -- 172.21.15.45:0/1382556809 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbad4004160 con 0x7fbae407ddc0 2024-04-16T16:55:11.430 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:09.325+0000 7fbae0a00700 1 -- 172.21.15.45:0/1382556809 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbad4004300 con 0x7fbae407ddc0 2024-04-16T16:55:11.430 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:09.325+0000 7fbae0a00700 1 -- 172.21.15.45:0/1382556809 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbad4011b20 con 0x7fbae407ddc0 2024-04-16T16:55:11.430 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:09.325+0000 7fbae3200700 1 -- 172.21.15.45:0/1382556809 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbae407ddc0 msgr2=0x7fbae407c2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:55:11.430 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:09.325+0000 7fbae3200700 1 --2- 172.21.15.45:0/1382556809 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbae407ddc0 0x7fbae407c2c0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fbad40098a0 tx=0x7fbad4009bb0 comp rx=0 tx=0).stop 2024-04-16T16:55:11.430 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:09.326+0000 7fbae3200700 1 -- 172.21.15.45:0/1382556809 shutdown_connections 2024-04-16T16:55:11.430 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:09.326+0000 7fbae3200700 1 --2- 172.21.15.45:0/1382556809 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbae407ddc0 0x7fbae407c2c0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:55:11.430 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:09.326+0000 7fbae3200700 1 -- 172.21.15.45:0/1382556809 >> 172.21.15.45:0/1382556809 conn(0x7fbae41039d0 msgr2=0x7fbae4105e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T16:55:11.430 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:09.326+0000 7fbae3200700 1 -- 172.21.15.45:0/1382556809 shutdown_connections 2024-04-16T16:55:11.430 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:09.326+0000 7fbae3200700 1 -- 172.21.15.45:0/1382556809 wait complete. 2024-04-16T16:55:11.430 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:09.327+0000 7fbae3200700 1 Processor -- start 2024-04-16T16:55:11.431 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:09.327+0000 7fbae3200700 1 -- start start 2024-04-16T16:55:11.431 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:09.327+0000 7fbae3200700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbae407ddc0 0x7fbae419e570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:55:11.431 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:09.327+0000 7fbae3200700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fbae410e9d0 con 0x7fbae407ddc0 2024-04-16T16:55:11.431 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:09.328+0000 7fbae1e00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbae407ddc0 0x7fbae419e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:55:11.431 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:09.328+0000 7fbae1e00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbae407ddc0 0x7fbae419e570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56684/0 (socket says 172.21.15.45:56684) 2024-04-16T16:55:11.431 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:09.328+0000 7fbae1e00700 1 -- 172.21.15.45:0/1730277990 learned_addr learned my addr 172.21.15.45:0/1730277990 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T16:55:11.431 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:09.328+0000 7fbae1e00700 1 -- 172.21.15.45:0/1730277990 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbae419eab0 con 0x7fbae407ddc0 2024-04-16T16:55:11.431 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:09.329+0000 7fbae1e00700 1 --2- 172.21.15.45:0/1730277990 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbae407ddc0 0x7fbae419e570 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fbad4011b60 tx=0x7fbad4011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T16:55:11.431 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:09.329+0000 7fbadaa00700 1 -- 172.21.15.45:0/1730277990 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbad4003b60 con 0x7fbae407ddc0 2024-04-16T16:55:11.431 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:09.329+0000 7fbadaa00700 1 -- 172.21.15.45:0/1730277990 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbad401a910 con 0x7fbae407ddc0 2024-04-16T16:55:11.431 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:09.329+0000 7fbadaa00700 1 -- 172.21.15.45:0/1730277990 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbad4024930 con 0x7fbae407ddc0 2024-04-16T16:55:11.431 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:09.329+0000 7fbae3200700 1 -- 172.21.15.45:0/1730277990 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbae419ed40 con 0x7fbae407ddc0 2024-04-16T16:55:11.431 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:09.329+0000 7fbae3200700 1 -- 172.21.15.45:0/1730277990 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbae419f1a0 con 0x7fbae407ddc0 2024-04-16T16:55:11.431 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:09.330+0000 7fbae3200700 1 -- 172.21.15.45:0/1730277990 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbae4064150 con 0x7fbae407ddc0 2024-04-16T16:55:11.432 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:09.331+0000 7fbadaa00700 1 -- 172.21.15.45:0/1730277990 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7fbad4022020 con 0x7fbae407ddc0 2024-04-16T16:55:11.432 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:09.331+0000 7fbadaa00700 1 --2- 172.21.15.45:0/1730277990 >> [v2:172.21.15.45:6800/1983630136,v1:172.21.15.45:6801/1983630136] conn(0x7fbac8039600 0x7fbac803bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:55:11.432 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:09.331+0000 7fbadaa00700 1 -- 172.21.15.45:0/1730277990 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fbad404d960 con 0x7fbae407ddc0 2024-04-16T16:55:11.432 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:09.331+0000 7fbae1400700 1 --2- 172.21.15.45:0/1730277990 >> [v2:172.21.15.45:6800/1983630136,v1:172.21.15.45:6801/1983630136] conn(0x7fbac8039600 0x7fbac803bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:55:11.432 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:09.332+0000 7fbae1400700 1 --2- 172.21.15.45:0/1730277990 >> [v2:172.21.15.45:6800/1983630136,v1:172.21.15.45:6801/1983630136] conn(0x7fbac8039600 0x7fbac803bac0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fbad0006fd0 tx=0x7fbad0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T16:55:11.432 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:09.337+0000 7fbadaa00700 1 -- 172.21.15.45:0/1730277990 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fbad400b3e0 con 0x7fbae407ddc0 2024-04-16T16:55:11.432 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:09.654+0000 7fbae3200700 1 -- 172.21.15.45:0/1730277990 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) -- 0x7fbae41a1ed0 con 0x7fbae407ddc0 2024-04-16T16:55:11.432 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:10.609+0000 7fbadaa00700 1 -- 172.21.15.45:0/1730277990 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) ==== 88+0+0 (secure 0 0 0) 0x7fbad404d2b0 con 0x7fbae407ddc0 2024-04-16T16:55:11.432 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:10.611+0000 7fbadaa00700 1 -- 172.21.15.45:0/1730277990 <== mon.0 v2:172.21.15.45:3300/0 8 ==== mgrmap(e 9) ==== 46571+0+0 (secure 0 0 0) 0x7fbad4024ad0 con 0x7fbae407ddc0 2024-04-16T16:55:11.432 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:10.614+0000 7fbae3200700 1 -- 172.21.15.45:0/1730277990 >> [v2:172.21.15.45:6800/1983630136,v1:172.21.15.45:6801/1983630136] conn(0x7fbac8039600 msgr2=0x7fbac803bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:55:11.432 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:10.614+0000 7fbae3200700 1 --2- 172.21.15.45:0/1730277990 >> [v2:172.21.15.45:6800/1983630136,v1:172.21.15.45:6801/1983630136] conn(0x7fbac8039600 0x7fbac803bac0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fbad0006fd0 tx=0x7fbad0006e40 comp rx=0 tx=0).stop 2024-04-16T16:55:11.432 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:10.614+0000 7fbae3200700 1 -- 172.21.15.45:0/1730277990 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbae407ddc0 msgr2=0x7fbae419e570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:55:11.432 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:10.614+0000 7fbae3200700 1 --2- 172.21.15.45:0/1730277990 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbae407ddc0 0x7fbae419e570 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fbad4011b60 tx=0x7fbad4011c40 comp rx=0 tx=0).stop 2024-04-16T16:55:11.432 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:10.614+0000 7fbae3200700 1 -- 172.21.15.45:0/1730277990 shutdown_connections 2024-04-16T16:55:11.433 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:10.614+0000 7fbae3200700 1 --2- 172.21.15.45:0/1730277990 >> [v2:172.21.15.45:6800/1983630136,v1:172.21.15.45:6801/1983630136] conn(0x7fbac8039600 0x7fbac803bac0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:55:11.433 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:10.614+0000 7fbae3200700 1 --2- 172.21.15.45:0/1730277990 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbae407ddc0 0x7fbae419e570 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:55:11.433 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:10.614+0000 7fbae3200700 1 -- 172.21.15.45:0/1730277990 >> 172.21.15.45:0/1730277990 conn(0x7fbae41039d0 msgr2=0x7fbae4104550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T16:55:11.433 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:10.614+0000 7fbae3200700 1 -- 172.21.15.45:0/1730277990 shutdown_connections 2024-04-16T16:55:11.433 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:10.614+0000 7fbae3200700 1 -- 172.21.15.45:0/1730277990 wait complete. 2024-04-16T16:55:11.729 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:55:11 smithi045 ceph-mon[31676]: from='client.? 172.21.15.45:0/1730277990' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-04-16T16:55:11.729 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:55:11 smithi045 ceph-mon[31676]: mgrmap e9: x(active, since 26s) 2024-04-16T16:55:11.729 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 16 16:55:11 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-x[31905]: 2024-04-16T16:55:11.463+0000 7f1ad8270200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-16T16:55:11.729 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 16 16:55:11 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-x[31905]: 2024-04-16T16:55:11.590+0000 7f1ad8270200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-16T16:55:12.602 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 16 16:55:12 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-x[31905]: 2024-04-16T16:55:12.438+0000 7f1ad8270200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-16T16:55:12.602 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 16 16:55:12 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-x[31905]: 2024-04-16T16:55:12.566+0000 7f1ad8270200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-16T16:55:12.851 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 16 16:55:12 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-x[31905]: 2024-04-16T16:55:12.758+0000 7f1ad8270200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-16T16:55:13.146 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 16 16:55:12 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-x[31905]: 2024-04-16T16:55:12.893+0000 7f1ad8270200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-16T16:55:13.146 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 16 16:55:13 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-x[31905]: 2024-04-16T16:55:13.009+0000 7f1ad8270200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-16T16:55:13.478 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:55:13 smithi045 ceph-mon[31676]: from='client.? 172.21.15.45:0/937699134' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-04-16T16:55:13.478 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 16 16:55:13 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-x[31905]: 2024-04-16T16:55:13.358+0000 7f1ad8270200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-16T16:55:13.760 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 16 16:55:13 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-x[31905]: 2024-04-16T16:55:13.507+0000 7f1ad8270200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-16T16:55:13.760 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 16 16:55:13 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-x[31905]: 2024-04-16T16:55:13.628+0000 7f1ad8270200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-16T16:55:13.877 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout { 2024-04-16T16:55:13.877 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-04-16T16:55:13.877 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-16T16:55:13.877 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-04-16T16:55:13.877 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-04-16T16:55:13.877 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout } 2024-04-16T16:55:13.877 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:12.845+0000 7f7e4e000700 1 Processor -- start 2024-04-16T16:55:13.877 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:12.845+0000 7f7e4e000700 1 -- start start 2024-04-16T16:55:13.877 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:12.846+0000 7f7e4e000700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7e4807de90 0x7f7e4807c340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:55:13.878 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:12.846+0000 7f7e4e000700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f7e4807c910 con 0x7f7e4807de90 2024-04-16T16:55:13.878 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:12.846+0000 7f7e47400700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7e4807de90 0x7f7e4807c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:55:13.878 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:12.846+0000 7f7e47400700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7e4807de90 0x7f7e4807c340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36440/0 (socket says 172.21.15.45:36440) 2024-04-16T16:55:13.878 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:12.846+0000 7f7e47400700 1 -- 172.21.15.45:0/2629033661 learned_addr learned my addr 172.21.15.45:0/2629033661 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T16:55:13.878 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:12.847+0000 7f7e47400700 1 -- 172.21.15.45:0/2629033661 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e4807ca90 con 0x7f7e4807de90 2024-04-16T16:55:13.878 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:12.847+0000 7f7e47400700 1 --2- 172.21.15.45:0/2629033661 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7e4807de90 0x7f7e4807c340 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f7e380098a0 tx=0x7f7e38009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9c6c919cc61efcd server_cookie=0 in_seq=0 out_seq=0 2024-04-16T16:55:13.878 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:12.847+0000 7f7e46000700 1 -- 172.21.15.45:0/2629033661 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7e38004160 con 0x7f7e4807de90 2024-04-16T16:55:13.878 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:12.847+0000 7f7e46000700 1 -- 172.21.15.45:0/2629033661 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7e38004300 con 0x7f7e4807de90 2024-04-16T16:55:13.878 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:12.847+0000 7f7e46000700 1 -- 172.21.15.45:0/2629033661 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7e38011b20 con 0x7f7e4807de90 2024-04-16T16:55:13.878 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:12.848+0000 7f7e4e000700 1 -- 172.21.15.45:0/2629033661 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7e4807de90 msgr2=0x7f7e4807c340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:55:13.878 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:12.848+0000 7f7e4e000700 1 --2- 172.21.15.45:0/2629033661 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7e4807de90 0x7f7e4807c340 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f7e380098a0 tx=0x7f7e38009bb0 comp rx=0 tx=0).stop 2024-04-16T16:55:13.878 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:12.848+0000 7f7e4e000700 1 -- 172.21.15.45:0/2629033661 shutdown_connections 2024-04-16T16:55:13.878 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:12.848+0000 7f7e4e000700 1 --2- 172.21.15.45:0/2629033661 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7e4807de90 0x7f7e4807c340 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:55:13.878 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:12.848+0000 7f7e4e000700 1 -- 172.21.15.45:0/2629033661 >> 172.21.15.45:0/2629033661 conn(0x7f7e48103b80 msgr2=0x7f7e48105fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T16:55:13.878 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:12.848+0000 7f7e4e000700 1 -- 172.21.15.45:0/2629033661 shutdown_connections 2024-04-16T16:55:13.879 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:12.848+0000 7f7e4e000700 1 -- 172.21.15.45:0/2629033661 wait complete. 2024-04-16T16:55:13.879 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:12.849+0000 7f7e4e000700 1 Processor -- start 2024-04-16T16:55:13.879 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:12.849+0000 7f7e4e000700 1 -- start start 2024-04-16T16:55:13.879 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:12.850+0000 7f7e4e000700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7e4807de90 0x7f7e481a2b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:55:13.879 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:12.850+0000 7f7e4e000700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f7e4810a7f0 con 0x7f7e4807de90 2024-04-16T16:55:13.879 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:12.850+0000 7f7e47400700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7e4807de90 0x7f7e481a2b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:55:13.879 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:12.850+0000 7f7e47400700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7e4807de90 0x7f7e481a2b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36448/0 (socket says 172.21.15.45:36448) 2024-04-16T16:55:13.879 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:12.850+0000 7f7e47400700 1 -- 172.21.15.45:0/937699134 learned_addr learned my addr 172.21.15.45:0/937699134 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T16:55:13.879 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:12.850+0000 7f7e47400700 1 -- 172.21.15.45:0/937699134 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e481a3060 con 0x7f7e4807de90 2024-04-16T16:55:13.879 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:12.851+0000 7f7e47400700 1 --2- 172.21.15.45:0/937699134 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7e4807de90 0x7f7e481a2b20 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f7e38011b60 tx=0x7f7e38011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T16:55:13.879 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:12.851+0000 7f7e3fe00700 1 -- 172.21.15.45:0/937699134 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7e38003b60 con 0x7f7e4807de90 2024-04-16T16:55:13.879 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:12.851+0000 7f7e3fe00700 1 -- 172.21.15.45:0/937699134 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7e3801a910 con 0x7f7e4807de90 2024-04-16T16:55:13.879 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:12.851+0000 7f7e4e000700 1 -- 172.21.15.45:0/937699134 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e481a32f0 con 0x7f7e4807de90 2024-04-16T16:55:13.879 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:12.851+0000 7f7e3fe00700 1 -- 172.21.15.45:0/937699134 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7e38024930 con 0x7f7e4807de90 2024-04-16T16:55:13.879 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:12.851+0000 7f7e4e000700 1 -- 172.21.15.45:0/937699134 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e481a3750 con 0x7f7e4807de90 2024-04-16T16:55:13.879 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:12.852+0000 7f7e4e000700 1 -- 172.21.15.45:0/937699134 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e48064150 con 0x7f7e4807de90 2024-04-16T16:55:13.880 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:12.853+0000 7f7e3fe00700 1 -- 172.21.15.45:0/937699134 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 9) ==== 46571+0+0 (secure 0 0 0) 0x7f7e38022020 con 0x7f7e4807de90 2024-04-16T16:55:13.880 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:12.853+0000 7f7e3fe00700 1 --2- 172.21.15.45:0/937699134 >> [v2:172.21.15.45:6800/1983630136,v1:172.21.15.45:6801/1983630136] conn(0x7f7e2c039660 0x7f7e2c03bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:55:13.880 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:12.854+0000 7f7e3fe00700 1 -- 172.21.15.45:0/937699134 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f7e3804d9e0 con 0x7f7e4807de90 2024-04-16T16:55:13.880 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:12.854+0000 7f7e46a00700 1 -- 172.21.15.45:0/937699134 >> [v2:172.21.15.45:6800/1983630136,v1:172.21.15.45:6801/1983630136] conn(0x7f7e2c039660 msgr2=0x7f7e2c03bb20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.45:6800/1983630136 2024-04-16T16:55:13.880 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:12.854+0000 7f7e46a00700 1 --2- 172.21.15.45:0/937699134 >> [v2:172.21.15.45:6800/1983630136,v1:172.21.15.45:6801/1983630136] conn(0x7f7e2c039660 0x7f7e2c03bb20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-16T16:55:13.880 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:12.859+0000 7f7e3fe00700 1 -- 172.21.15.45:0/937699134 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7e38016260 con 0x7f7e4807de90 2024-04-16T16:55:13.880 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:13.055+0000 7f7e46a00700 1 -- 172.21.15.45:0/937699134 >> [v2:172.21.15.45:6800/1983630136,v1:172.21.15.45:6801/1983630136] conn(0x7f7e2c039660 msgr2=0x7f7e2c03bb20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.45:6800/1983630136 2024-04-16T16:55:13.880 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:13.055+0000 7f7e46a00700 1 --2- 172.21.15.45:0/937699134 >> [v2:172.21.15.45:6800/1983630136,v1:172.21.15.45:6801/1983630136] conn(0x7f7e2c039660 0x7f7e2c03bb20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-16T16:55:13.880 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:13.145+0000 7f7e4e000700 1 -- 172.21.15.45:0/937699134 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f7e4819c080 con 0x7f7e4807de90 2024-04-16T16:55:13.880 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:13.145+0000 7f7e3fe00700 1 -- 172.21.15.45:0/937699134 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) ==== 56+0+88 (secure 0 0 0) 0x7f7e4819c080 con 0x7f7e4807de90 2024-04-16T16:55:13.880 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:13.148+0000 7f7e4e000700 1 -- 172.21.15.45:0/937699134 >> [v2:172.21.15.45:6800/1983630136,v1:172.21.15.45:6801/1983630136] conn(0x7f7e2c039660 msgr2=0x7f7e2c03bb20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-16T16:55:13.880 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:13.148+0000 7f7e4e000700 1 --2- 172.21.15.45:0/937699134 >> [v2:172.21.15.45:6800/1983630136,v1:172.21.15.45:6801/1983630136] conn(0x7f7e2c039660 0x7f7e2c03bb20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:55:13.880 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:13.148+0000 7f7e4e000700 1 -- 172.21.15.45:0/937699134 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7e4807de90 msgr2=0x7f7e481a2b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:55:13.880 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:13.148+0000 7f7e4e000700 1 --2- 172.21.15.45:0/937699134 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7e4807de90 0x7f7e481a2b20 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f7e38011b60 tx=0x7f7e38011c40 comp rx=0 tx=0).stop 2024-04-16T16:55:13.880 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:13.148+0000 7f7e4e000700 1 -- 172.21.15.45:0/937699134 shutdown_connections 2024-04-16T16:55:13.881 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:13.148+0000 7f7e4e000700 1 --2- 172.21.15.45:0/937699134 >> [v2:172.21.15.45:6800/1983630136,v1:172.21.15.45:6801/1983630136] conn(0x7f7e2c039660 0x7f7e2c03bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:55:13.881 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:13.148+0000 7f7e4e000700 1 --2- 172.21.15.45:0/937699134 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7e4807de90 0x7f7e481a2b20 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:55:13.881 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:13.148+0000 7f7e4e000700 1 -- 172.21.15.45:0/937699134 >> 172.21.15.45:0/937699134 conn(0x7f7e48103b80 msgr2=0x7f7e48105fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T16:55:13.881 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:13.149+0000 7f7e4e000700 1 -- 172.21.15.45:0/937699134 shutdown_connections 2024-04-16T16:55:13.881 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:13.149+0000 7f7e4e000700 1 -- 172.21.15.45:0/937699134 wait complete. 2024-04-16T16:55:13.881 INFO:teuthology.orchestra.run.smithi045.stdout:Waiting for the mgr to restart... 2024-04-16T16:55:13.881 INFO:teuthology.orchestra.run.smithi045.stdout:Waiting for mgr epoch 9... 2024-04-16T16:55:14.028 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 16 16:55:13 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-x[31905]: 2024-04-16T16:55:13.759+0000 7f1ad8270200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-16T16:55:14.028 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 16 16:55:14 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-x[31905]: 2024-04-16T16:55:14.000+0000 7f1ad8270200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-16T16:55:15.478 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 16 16:55:15 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-x[31905]: 2024-04-16T16:55:15.362+0000 7f1ad8270200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-16T16:55:15.921 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 16 16:55:15 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-x[31905]: 2024-04-16T16:55:15.543+0000 7f1ad8270200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-16T16:55:16.228 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 16 16:55:15 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-x[31905]: 2024-04-16T16:55:15.979+0000 7f1ad8270200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-16T16:55:16.228 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 16 16:55:16 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-x[31905]: 2024-04-16T16:55:16.104+0000 7f1ad8270200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-16T16:55:16.633 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 16 16:55:16 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-x[31905]: 2024-04-16T16:55:16.373+0000 7f1ad8270200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-16T16:55:16.633 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 16 16:55:16 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-x[31905]: 2024-04-16T16:55:16.630+0000 7f1ad8270200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-16T16:55:17.359 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 16 16:55:17 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-x[31905]: 2024-04-16T16:55:17.029+0000 7f1ad8270200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-16T16:55:17.359 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 16 16:55:17 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-x[31905]: 2024-04-16T16:55:17.357+0000 7f1ad8270200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-16T16:55:17.978 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 16 16:55:17 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-x[31905]: 2024-04-16T16:55:17.679+0000 7f1ad8270200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-16T16:55:19.229 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 16 16:55:18 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-x[31905]: 2024-04-16T16:55:18.763+0000 7f1ad8270200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-16T16:55:19.946 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 16 16:55:19 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-x[31905]: 2024-04-16T16:55:19.505+0000 7f1ad8270200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-16T16:55:19.946 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 16 16:55:19 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-x[31905]: 2024-04-16T16:55:19.646+0000 7f1ad8270200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-16T16:55:20.203 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 16 16:55:19 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-x[31905]: 2024-04-16T16:55:19.943+0000 7f1ad8270200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-16T16:55:20.204 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:55:20 smithi045 ceph-mon[31676]: Active manager daemon x restarted 2024-04-16T16:55:20.204 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:55:20 smithi045 ceph-mon[31676]: Activating manager daemon x 2024-04-16T16:55:20.204 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:55:20 smithi045 ceph-mon[31676]: osdmap e3: 0 total, 0 up, 0 in 2024-04-16T16:55:20.204 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:55:20 smithi045 ceph-mon[31676]: mgrmap e10: x(active, starting, since 0.00997766s) 2024-04-16T16:55:20.204 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:55:20 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-16T16:55:20.204 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:55:20 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-16T16:55:20.204 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:55:20 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-16T16:55:20.204 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:55:20 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-16T16:55:20.205 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:55:20 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-16T16:55:20.205 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:55:20 smithi045 ceph-mon[31676]: Manager daemon x is now available 2024-04-16T16:55:21.228 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:55:21 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T16:55:21.228 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:55:21 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-16T16:55:21.228 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:55:21 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-16T16:55:21.228 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:55:21 smithi045 ceph-mon[31676]: [16/Apr/2024:16:55:20] ENGINE Bus STARTING 2024-04-16T16:55:21.228 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:55:21 smithi045 ceph-mon[31676]: mgrmap e11: x(active, since 1.01133s) 2024-04-16T16:55:21.726 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout { 2024-04-16T16:55:21.726 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-04-16T16:55:21.727 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "initialized": true 2024-04-16T16:55:21.727 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout } 2024-04-16T16:55:21.727 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:14.498+0000 7f9aa9200700 1 Processor -- start 2024-04-16T16:55:21.727 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:14.499+0000 7f9aa9200700 1 -- start start 2024-04-16T16:55:21.727 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:14.499+0000 7f9aa9200700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9aa4073d70 0x7f9aa4074190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:55:21.727 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:14.500+0000 7f9aa3400700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9aa4073d70 0x7f9aa4074190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:55:21.727 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:14.500+0000 7f9aa3400700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9aa4073d70 0x7f9aa4074190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36462/0 (socket says 172.21.15.45:36462) 2024-04-16T16:55:21.727 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:14.500+0000 7f9aa3400700 1 -- 172.21.15.45:0/3648303666 learned_addr learned my addr 172.21.15.45:0/3648303666 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T16:55:21.727 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:14.501+0000 7f9aa9200700 1 -- 172.21.15.45:0/3648303666 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f9aa4074760 con 0x7f9aa4073d70 2024-04-16T16:55:21.727 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:14.502+0000 7f9aa3400700 1 -- 172.21.15.45:0/3648303666 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9aa410ffa0 con 0x7f9aa4073d70 2024-04-16T16:55:21.727 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:14.502+0000 7f9aa3400700 1 --2- 172.21.15.45:0/3648303666 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9aa4073d70 0x7f9aa4074190 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f9a90009b00 tx=0x7f9a90009e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=984bb4c8c36784dd server_cookie=0 in_seq=0 out_seq=0 2024-04-16T16:55:21.727 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:14.503+0000 7f9aa2000700 1 -- 172.21.15.45:0/3648303666 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9a90004160 con 0x7f9aa4073d70 2024-04-16T16:55:21.727 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:14.503+0000 7f9aa2000700 1 -- 172.21.15.45:0/3648303666 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9a90004300 con 0x7f9aa4073d70 2024-04-16T16:55:21.727 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:14.503+0000 7f9aa2000700 1 -- 172.21.15.45:0/3648303666 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9a9000bac0 con 0x7f9aa4073d70 2024-04-16T16:55:21.728 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:14.504+0000 7f9aa9200700 1 -- 172.21.15.45:0/3648303666 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9aa4073d70 msgr2=0x7f9aa4074190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:55:21.728 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:14.504+0000 7f9aa9200700 1 --2- 172.21.15.45:0/3648303666 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9aa4073d70 0x7f9aa4074190 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f9a90009b00 tx=0x7f9a90009e10 comp rx=0 tx=0).stop 2024-04-16T16:55:21.728 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:14.505+0000 7f9aa9200700 1 -- 172.21.15.45:0/3648303666 shutdown_connections 2024-04-16T16:55:21.728 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:14.505+0000 7f9aa9200700 1 --2- 172.21.15.45:0/3648303666 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9aa4073d70 0x7f9aa4074190 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:55:21.728 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:14.506+0000 7f9aa9200700 1 -- 172.21.15.45:0/3648303666 >> 172.21.15.45:0/3648303666 conn(0x7f9aa406eef0 msgr2=0x7f9aa4071350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T16:55:21.728 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:14.506+0000 7f9aa9200700 1 -- 172.21.15.45:0/3648303666 shutdown_connections 2024-04-16T16:55:21.728 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:14.506+0000 7f9aa9200700 1 -- 172.21.15.45:0/3648303666 wait complete. 2024-04-16T16:55:21.728 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:14.509+0000 7f9aa9200700 1 Processor -- start 2024-04-16T16:55:21.728 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:14.509+0000 7f9aa9200700 1 -- start start 2024-04-16T16:55:21.728 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:14.510+0000 7f9aa9200700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9aa4073d70 0x7f9aa41a2b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:55:21.728 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:14.510+0000 7f9aa9200700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f9aa41133e0 con 0x7f9aa4073d70 2024-04-16T16:55:21.728 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:14.511+0000 7f9aa3400700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9aa4073d70 0x7f9aa41a2b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:55:21.728 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:14.511+0000 7f9aa3400700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9aa4073d70 0x7f9aa41a2b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36468/0 (socket says 172.21.15.45:36468) 2024-04-16T16:55:21.728 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:14.511+0000 7f9aa3400700 1 -- 172.21.15.45:0/1506658091 learned_addr learned my addr 172.21.15.45:0/1506658091 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T16:55:21.728 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:14.511+0000 7f9aa3400700 1 -- 172.21.15.45:0/1506658091 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9aa4072fe0 con 0x7f9aa4073d70 2024-04-16T16:55:21.729 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:14.512+0000 7f9aa3400700 1 --2- 172.21.15.45:0/1506658091 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9aa4073d70 0x7f9aa41a2b30 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f9a9000b6d0 tx=0x7f9a90004610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T16:55:21.729 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:14.513+0000 7f9a9be00700 1 -- 172.21.15.45:0/1506658091 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9a900038b0 con 0x7f9aa4073d70 2024-04-16T16:55:21.729 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:14.513+0000 7f9a9be00700 1 -- 172.21.15.45:0/1506658091 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9a90003a50 con 0x7f9aa4073d70 2024-04-16T16:55:21.729 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:14.513+0000 7f9a9be00700 1 -- 172.21.15.45:0/1506658091 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9a900115b0 con 0x7f9aa4073d70 2024-04-16T16:55:21.729 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:14.514+0000 7f9aa9200700 1 -- 172.21.15.45:0/1506658091 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9aa41a3070 con 0x7f9aa4073d70 2024-04-16T16:55:21.729 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:14.514+0000 7f9aa9200700 1 -- 172.21.15.45:0/1506658091 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9aa41a34f0 con 0x7f9aa4073d70 2024-04-16T16:55:21.729 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:14.516+0000 7f9a9be00700 1 -- 172.21.15.45:0/1506658091 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 9) ==== 46571+0+0 (secure 0 0 0) 0x7f9a9001a430 con 0x7f9aa4073d70 2024-04-16T16:55:21.729 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:14.517+0000 7f9a9be00700 1 --2- 172.21.15.45:0/1506658091 >> [v2:172.21.15.45:6800/1983630136,v1:172.21.15.45:6801/1983630136] conn(0x7f9a88039640 0x7f9a8803bb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:55:21.729 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:14.517+0000 7f9a9be00700 1 -- 172.21.15.45:0/1506658091 --> [v2:172.21.15.45:6800/1983630136,v1:172.21.15.45:6801/1983630136] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f9a8803c210 con 0x7f9a88039640 2024-04-16T16:55:21.729 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:14.517+0000 7f9aa2a00700 1 -- 172.21.15.45:0/1506658091 >> [v2:172.21.15.45:6800/1983630136,v1:172.21.15.45:6801/1983630136] conn(0x7f9a88039640 msgr2=0x7f9a8803bb00 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.45:6800/1983630136 2024-04-16T16:55:21.729 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:14.517+0000 7f9aa2a00700 1 --2- 172.21.15.45:0/1506658091 >> [v2:172.21.15.45:6800/1983630136,v1:172.21.15.45:6801/1983630136] conn(0x7f9a88039640 0x7f9a8803bb00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-16T16:55:21.729 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:14.519+0000 7f9a9be00700 1 -- 172.21.15.45:0/1506658091 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f9a9001a720 con 0x7f9aa4073d70 2024-04-16T16:55:21.729 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:14.717+0000 7f9aa2a00700 1 -- 172.21.15.45:0/1506658091 >> [v2:172.21.15.45:6800/1983630136,v1:172.21.15.45:6801/1983630136] conn(0x7f9a88039640 msgr2=0x7f9a8803bb00 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.45:6800/1983630136 2024-04-16T16:55:21.729 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:14.717+0000 7f9aa2a00700 1 --2- 172.21.15.45:0/1506658091 >> [v2:172.21.15.45:6800/1983630136,v1:172.21.15.45:6801/1983630136] conn(0x7f9a88039640 0x7f9a8803bb00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-16T16:55:21.730 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:15.118+0000 7f9aa2a00700 1 -- 172.21.15.45:0/1506658091 >> [v2:172.21.15.45:6800/1983630136,v1:172.21.15.45:6801/1983630136] conn(0x7f9a88039640 msgr2=0x7f9a8803bb00 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.45:6800/1983630136 2024-04-16T16:55:21.730 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:15.118+0000 7f9aa2a00700 1 --2- 172.21.15.45:0/1506658091 >> [v2:172.21.15.45:6800/1983630136,v1:172.21.15.45:6801/1983630136] conn(0x7f9a88039640 0x7f9a8803bb00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-16T16:55:21.730 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:15.919+0000 7f9aa2a00700 1 -- 172.21.15.45:0/1506658091 >> [v2:172.21.15.45:6800/1983630136,v1:172.21.15.45:6801/1983630136] conn(0x7f9a88039640 msgr2=0x7f9a8803bb00 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.45:6800/1983630136 2024-04-16T16:55:21.730 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:15.919+0000 7f9aa2a00700 1 --2- 172.21.15.45:0/1506658091 >> [v2:172.21.15.45:6800/1983630136,v1:172.21.15.45:6801/1983630136] conn(0x7f9a88039640 0x7f9a8803bb00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-16T16:55:21.730 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:17.521+0000 7f9aa2a00700 1 -- 172.21.15.45:0/1506658091 >> [v2:172.21.15.45:6800/1983630136,v1:172.21.15.45:6801/1983630136] conn(0x7f9a88039640 msgr2=0x7f9a8803bb00 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.45:6800/1983630136 2024-04-16T16:55:21.730 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:17.521+0000 7f9aa2a00700 1 --2- 172.21.15.45:0/1506658091 >> [v2:172.21.15.45:6800/1983630136,v1:172.21.15.45:6801/1983630136] conn(0x7f9a88039640 0x7f9a8803bb00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-04-16T16:55:21.730 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:19.953+0000 7f9a9be00700 1 -- 172.21.15.45:0/1506658091 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mgrmap(e 10) ==== 46338+0+0 (secure 0 0 0) 0x7f9a90022cd0 con 0x7f9aa4073d70 2024-04-16T16:55:21.730 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:19.953+0000 7f9a9be00700 1 -- 172.21.15.45:0/1506658091 >> [v2:172.21.15.45:6800/1983630136,v1:172.21.15.45:6801/1983630136] conn(0x7f9a88039640 msgr2=0x7f9a8803bb00 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-16T16:55:21.730 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:19.953+0000 7f9a9be00700 1 --2- 172.21.15.45:0/1506658091 >> [v2:172.21.15.45:6800/1983630136,v1:172.21.15.45:6801/1983630136] conn(0x7f9a88039640 0x7f9a8803bb00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:55:21.730 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:20.954+0000 7f9a9be00700 1 -- 172.21.15.45:0/1506658091 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mgrmap(e 11) ==== 46465+0+0 (secure 0 0 0) 0x7f9a9004ef30 con 0x7f9aa4073d70 2024-04-16T16:55:21.730 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:20.954+0000 7f9a9be00700 1 --2- 172.21.15.45:0/1506658091 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f9a8803cfd0 0x7f9a8803f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:55:21.730 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:20.954+0000 7f9a9be00700 1 -- 172.21.15.45:0/1506658091 --> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f9a9001bca0 con 0x7f9a8803cfd0 2024-04-16T16:55:21.731 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:20.955+0000 7f9aa2a00700 1 --2- 172.21.15.45:0/1506658091 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f9a8803cfd0 0x7f9a8803f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:55:21.731 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:20.957+0000 7f9aa2a00700 1 --2- 172.21.15.45:0/1506658091 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f9a8803cfd0 0x7f9a8803f3c0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f9a9c003d90 tx=0x7f9a9c007220 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T16:55:21.731 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:20.957+0000 7f9a9be00700 1 -- 172.21.15.45:0/1506658091 <== mgr.14152 v2:172.21.15.45:6800/268030699 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7774 (secure 0 0 0) 0x7f9a9001bca0 con 0x7f9a8803cfd0 2024-04-16T16:55:21.731 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:20.969+0000 7f9aa9200700 1 -- 172.21.15.45:0/1506658091 --> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f9aa4064150 con 0x7f9a8803cfd0 2024-04-16T16:55:21.731 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:20.970+0000 7f9a9be00700 1 -- 172.21.15.45:0/1506658091 <== mgr.14152 v2:172.21.15.45:6800/268030699 2 ==== command_reply(tid 1: 0 ) ==== 8+0+52 (secure 0 0 0) 0x7f9aa4064150 con 0x7f9a8803cfd0 2024-04-16T16:55:21.731 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:20.970+0000 7f9aa9200700 1 -- 172.21.15.45:0/1506658091 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f9a8803cfd0 msgr2=0x7f9a8803f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:55:21.731 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:20.970+0000 7f9aa9200700 1 --2- 172.21.15.45:0/1506658091 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f9a8803cfd0 0x7f9a8803f3c0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f9a9c003d90 tx=0x7f9a9c007220 comp rx=0 tx=0).stop 2024-04-16T16:55:21.731 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:20.970+0000 7f9aa9200700 1 -- 172.21.15.45:0/1506658091 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9aa4073d70 msgr2=0x7f9aa41a2b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:55:21.731 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:20.970+0000 7f9aa9200700 1 --2- 172.21.15.45:0/1506658091 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9aa4073d70 0x7f9aa41a2b30 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f9a9000b6d0 tx=0x7f9a90004610 comp rx=0 tx=0).stop 2024-04-16T16:55:21.731 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:20.970+0000 7f9aa9200700 1 -- 172.21.15.45:0/1506658091 shutdown_connections 2024-04-16T16:55:21.731 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:20.970+0000 7f9aa9200700 1 --2- 172.21.15.45:0/1506658091 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f9a8803cfd0 0x7f9a8803f3c0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:55:21.731 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:20.970+0000 7f9aa9200700 1 --2- 172.21.15.45:0/1506658091 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9aa4073d70 0x7f9aa41a2b30 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:55:21.731 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:20.970+0000 7f9aa9200700 1 --2- 172.21.15.45:0/1506658091 >> [v2:172.21.15.45:6800/1983630136,v1:172.21.15.45:6801/1983630136] conn(0x7f9a88039640 0x7f9a8803bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:55:21.732 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:20.970+0000 7f9aa9200700 1 -- 172.21.15.45:0/1506658091 >> 172.21.15.45:0/1506658091 conn(0x7f9aa406eef0 msgr2=0x7f9aa406f930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T16:55:21.732 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:20.970+0000 7f9aa9200700 1 -- 172.21.15.45:0/1506658091 shutdown_connections 2024-04-16T16:55:21.732 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:20.971+0000 7f9aa9200700 1 -- 172.21.15.45:0/1506658091 wait complete. 2024-04-16T16:55:21.732 INFO:teuthology.orchestra.run.smithi045.stdout:mgr epoch 9 is available 2024-04-16T16:55:21.732 INFO:teuthology.orchestra.run.smithi045.stdout:Generating a dashboard self-signed certificate... 2024-04-16T16:55:22.228 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:55:22 smithi045 ceph-mon[31676]: [16/Apr/2024:16:55:20] ENGINE Serving on https://172.21.15.45:7150 2024-04-16T16:55:22.228 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:55:22 smithi045 ceph-mon[31676]: [16/Apr/2024:16:55:20] ENGINE Serving on http://172.21.15.45:8765 2024-04-16T16:55:22.228 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:55:22 smithi045 ceph-mon[31676]: [16/Apr/2024:16:55:20] ENGINE Bus STARTED 2024-04-16T16:55:22.228 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:55:22 smithi045 ceph-mon[31676]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-04-16T16:55:22.228 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:55:22 smithi045 ceph-mon[31676]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-04-16T16:55:23.362 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:55:23 smithi045 ceph-mon[31676]: mgrmap e12: x(active, since 2s) 2024-04-16T16:55:23.362 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:55:23 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:55:24.193 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-04-16T16:55:24.193 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:23.109+0000 7fd322800700 1 Processor -- start 2024-04-16T16:55:24.193 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:23.109+0000 7fd322800700 1 -- start start 2024-04-16T16:55:24.194 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:23.109+0000 7fd322800700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd31c1085e0 0x7fd31c108a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:55:24.194 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:23.109+0000 7fd322800700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fd31c108fd0 con 0x7fd31c1085e0 2024-04-16T16:55:24.194 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:23.110+0000 7fd31b400700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd31c1085e0 0x7fd31c108a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:55:24.194 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:23.110+0000 7fd31b400700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd31c1085e0 0x7fd31c108a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51858/0 (socket says 172.21.15.45:51858) 2024-04-16T16:55:24.194 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:23.110+0000 7fd31b400700 1 -- 172.21.15.45:0/535696698 learned_addr learned my addr 172.21.15.45:0/535696698 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T16:55:24.194 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:23.110+0000 7fd31b400700 1 -- 172.21.15.45:0/535696698 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd31c109790 con 0x7fd31c1085e0 2024-04-16T16:55:24.194 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:23.111+0000 7fd31b400700 1 --2- 172.21.15.45:0/535696698 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd31c1085e0 0x7fd31c108a00 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fd30c0098a0 tx=0x7fd30c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31da04ebf98ae6a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T16:55:24.194 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:23.111+0000 7fd31aa00700 1 -- 172.21.15.45:0/535696698 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd30c004160 con 0x7fd31c1085e0 2024-04-16T16:55:24.194 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:23.111+0000 7fd31aa00700 1 -- 172.21.15.45:0/535696698 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd30c004300 con 0x7fd31c1085e0 2024-04-16T16:55:24.194 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:23.111+0000 7fd31aa00700 1 -- 172.21.15.45:0/535696698 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd30c011b20 con 0x7fd31c1085e0 2024-04-16T16:55:24.194 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:23.112+0000 7fd322800700 1 -- 172.21.15.45:0/535696698 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd31c1085e0 msgr2=0x7fd31c108a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:55:24.194 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:23.112+0000 7fd322800700 1 --2- 172.21.15.45:0/535696698 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd31c1085e0 0x7fd31c108a00 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fd30c0098a0 tx=0x7fd30c009bb0 comp rx=0 tx=0).stop 2024-04-16T16:55:24.194 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:23.112+0000 7fd322800700 1 -- 172.21.15.45:0/535696698 shutdown_connections 2024-04-16T16:55:24.194 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:23.112+0000 7fd322800700 1 --2- 172.21.15.45:0/535696698 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd31c1085e0 0x7fd31c108a00 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:55:24.195 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:23.112+0000 7fd322800700 1 -- 172.21.15.45:0/535696698 >> 172.21.15.45:0/535696698 conn(0x7fd31c103bc0 msgr2=0x7fd31c105fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T16:55:24.195 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:23.112+0000 7fd322800700 1 -- 172.21.15.45:0/535696698 shutdown_connections 2024-04-16T16:55:24.195 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:23.112+0000 7fd322800700 1 -- 172.21.15.45:0/535696698 wait complete. 2024-04-16T16:55:24.195 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:23.113+0000 7fd322800700 1 Processor -- start 2024-04-16T16:55:24.195 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:23.113+0000 7fd322800700 1 -- start start 2024-04-16T16:55:24.195 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:23.114+0000 7fd322800700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd31c1085e0 0x7fd31c19c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:55:24.195 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:23.114+0000 7fd322800700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fd31c10cb40 con 0x7fd31c1085e0 2024-04-16T16:55:24.195 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:23.114+0000 7fd31b400700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd31c1085e0 0x7fd31c19c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:55:24.195 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:23.114+0000 7fd31b400700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd31c1085e0 0x7fd31c19c560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51864/0 (socket says 172.21.15.45:51864) 2024-04-16T16:55:24.195 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:23.114+0000 7fd31b400700 1 -- 172.21.15.45:0/1721998767 learned_addr learned my addr 172.21.15.45:0/1721998767 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T16:55:24.195 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:23.114+0000 7fd31b400700 1 -- 172.21.15.45:0/1721998767 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd31c19caa0 con 0x7fd31c1085e0 2024-04-16T16:55:24.195 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:23.115+0000 7fd31b400700 1 --2- 172.21.15.45:0/1721998767 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd31c1085e0 0x7fd31c19c560 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fd30c0116a0 tx=0x7fd30c011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T16:55:24.195 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:23.115+0000 7fd318c00700 1 -- 172.21.15.45:0/1721998767 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd30c003a40 con 0x7fd31c1085e0 2024-04-16T16:55:24.195 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:23.115+0000 7fd318c00700 1 -- 172.21.15.45:0/1721998767 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd30c003be0 con 0x7fd31c1085e0 2024-04-16T16:55:24.195 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:23.115+0000 7fd318c00700 1 -- 172.21.15.45:0/1721998767 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd30c01a720 con 0x7fd31c1085e0 2024-04-16T16:55:24.196 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:23.115+0000 7fd322800700 1 -- 172.21.15.45:0/1721998767 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd31c19cd30 con 0x7fd31c1085e0 2024-04-16T16:55:24.196 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:23.115+0000 7fd322800700 1 -- 172.21.15.45:0/1721998767 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd31c19d190 con 0x7fd31c1085e0 2024-04-16T16:55:24.196 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:23.117+0000 7fd322800700 1 -- 172.21.15.45:0/1721998767 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd31c064150 con 0x7fd31c1085e0 2024-04-16T16:55:24.196 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:23.117+0000 7fd318c00700 1 -- 172.21.15.45:0/1721998767 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 12) ==== 46571+0+0 (secure 0 0 0) 0x7fd30c024aa0 con 0x7fd31c1085e0 2024-04-16T16:55:24.196 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:23.117+0000 7fd318c00700 1 --2- 172.21.15.45:0/1721998767 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fd300039610 0x7fd30003bad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:55:24.196 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:23.117+0000 7fd318c00700 1 -- 172.21.15.45:0/1721998767 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fd30c04dd60 con 0x7fd31c1085e0 2024-04-16T16:55:24.196 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:23.118+0000 7fd312a00700 1 --2- 172.21.15.45:0/1721998767 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fd300039610 0x7fd30003bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:55:24.196 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:23.119+0000 7fd312a00700 1 --2- 172.21.15.45:0/1721998767 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fd300039610 0x7fd30003bad0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fd304006fd0 tx=0x7fd304006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T16:55:24.196 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:23.124+0000 7fd318c00700 1 -- 172.21.15.45:0/1721998767 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd30c011e20 con 0x7fd31c1085e0 2024-04-16T16:55:24.196 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:23.359+0000 7fd322800700 1 -- 172.21.15.45:0/1721998767 --> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) -- 0x7fd31c104e10 con 0x7fd300039610 2024-04-16T16:55:24.196 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:23.518+0000 7fd318c00700 1 -- 172.21.15.45:0/1721998767 <== mgr.14152 v2:172.21.15.45:6800/268030699 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+32 (secure 0 0 0) 0x7fd31c104e10 con 0x7fd300039610 2024-04-16T16:55:24.196 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:23.521+0000 7fd322800700 1 -- 172.21.15.45:0/1721998767 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fd300039610 msgr2=0x7fd30003bad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:55:24.196 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:23.521+0000 7fd322800700 1 --2- 172.21.15.45:0/1721998767 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fd300039610 0x7fd30003bad0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fd304006fd0 tx=0x7fd304006e40 comp rx=0 tx=0).stop 2024-04-16T16:55:24.196 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:23.521+0000 7fd322800700 1 -- 172.21.15.45:0/1721998767 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd31c1085e0 msgr2=0x7fd31c19c560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:55:24.196 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:23.521+0000 7fd322800700 1 --2- 172.21.15.45:0/1721998767 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd31c1085e0 0x7fd31c19c560 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fd30c0116a0 tx=0x7fd30c011ba0 comp rx=0 tx=0).stop 2024-04-16T16:55:24.196 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:23.521+0000 7fd322800700 1 -- 172.21.15.45:0/1721998767 shutdown_connections 2024-04-16T16:55:24.197 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:23.521+0000 7fd322800700 1 --2- 172.21.15.45:0/1721998767 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fd300039610 0x7fd30003bad0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:55:24.197 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:23.521+0000 7fd322800700 1 --2- 172.21.15.45:0/1721998767 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd31c1085e0 0x7fd31c19c560 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:55:24.197 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:23.521+0000 7fd322800700 1 -- 172.21.15.45:0/1721998767 >> 172.21.15.45:0/1721998767 conn(0x7fd31c103bc0 msgr2=0x7fd31c104700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T16:55:24.197 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:23.521+0000 7fd322800700 1 -- 172.21.15.45:0/1721998767 shutdown_connections 2024-04-16T16:55:24.197 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:23.521+0000 7fd322800700 1 -- 172.21.15.45:0/1721998767 wait complete. 2024-04-16T16:55:24.197 INFO:teuthology.orchestra.run.smithi045.stdout:Creating initial admin user... 2024-04-16T16:55:24.797 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:55:24 smithi045 ceph-mon[31676]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T16:55:24.797 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:55:24 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:55:24.797 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:55:24 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:55:25.978 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:55:25 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:55:25.978 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:55:25 smithi045 ceph-mon[31676]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T16:55:25.978 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:55:25 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:55:26.064 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$Zz6OgBofiV7Uk46Z9wRrE.nFQ/vt9qe.VZikIyf/M04smEzKErLhK", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1713286525, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-04-16T16:55:26.065 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:24.794+0000 7fe122e00700 1 Processor -- start 2024-04-16T16:55:26.065 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:24.795+0000 7fe122e00700 1 -- start start 2024-04-16T16:55:26.065 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:24.795+0000 7fe122e00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe11c108600 0x7fe11c108a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:55:26.065 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:24.795+0000 7fe122e00700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fe11c108ff0 con 0x7fe11c108600 2024-04-16T16:55:26.067 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:24.795+0000 7fe11be00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe11c108600 0x7fe11c108a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:55:26.067 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:24.796+0000 7fe11be00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe11c108600 0x7fe11c108a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51876/0 (socket says 172.21.15.45:51876) 2024-04-16T16:55:26.067 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:24.796+0000 7fe11be00700 1 -- 172.21.15.45:0/2002687730 learned_addr learned my addr 172.21.15.45:0/2002687730 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T16:55:26.067 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:24.796+0000 7fe11be00700 1 -- 172.21.15.45:0/2002687730 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe11c1097b0 con 0x7fe11c108600 2024-04-16T16:55:26.067 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:24.797+0000 7fe11be00700 1 --2- 172.21.15.45:0/2002687730 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe11c108600 0x7fe11c108a20 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fe10c009e80 tx=0x7fe10c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5ccf406ee23b8e server_cookie=0 in_seq=0 out_seq=0 2024-04-16T16:55:26.067 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:24.797+0000 7fe11b400700 1 -- 172.21.15.45:0/2002687730 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe10c017070 con 0x7fe11c108600 2024-04-16T16:55:26.067 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:24.797+0000 7fe11b400700 1 -- 172.21.15.45:0/2002687730 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe10c00b8c0 con 0x7fe11c108600 2024-04-16T16:55:26.067 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:24.797+0000 7fe11b400700 1 -- 172.21.15.45:0/2002687730 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe10c0045c0 con 0x7fe11c108600 2024-04-16T16:55:26.069 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:24.798+0000 7fe122e00700 1 -- 172.21.15.45:0/2002687730 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe11c108600 msgr2=0x7fe11c108a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:55:26.069 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:24.798+0000 7fe122e00700 1 --2- 172.21.15.45:0/2002687730 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe11c108600 0x7fe11c108a20 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fe10c009e80 tx=0x7fe10c00b330 comp rx=0 tx=0).stop 2024-04-16T16:55:26.070 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:24.798+0000 7fe122e00700 1 -- 172.21.15.45:0/2002687730 shutdown_connections 2024-04-16T16:55:26.070 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:24.798+0000 7fe122e00700 1 --2- 172.21.15.45:0/2002687730 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe11c108600 0x7fe11c108a20 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:55:26.070 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:24.798+0000 7fe122e00700 1 -- 172.21.15.45:0/2002687730 >> 172.21.15.45:0/2002687730 conn(0x7fe11c103bc0 msgr2=0x7fe11c105fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T16:55:26.070 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:24.798+0000 7fe122e00700 1 -- 172.21.15.45:0/2002687730 shutdown_connections 2024-04-16T16:55:26.070 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:24.798+0000 7fe122e00700 1 -- 172.21.15.45:0/2002687730 wait complete. 2024-04-16T16:55:26.070 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:24.800+0000 7fe122e00700 1 Processor -- start 2024-04-16T16:55:26.070 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:24.800+0000 7fe122e00700 1 -- start start 2024-04-16T16:55:26.070 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:24.800+0000 7fe122e00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe11c108600 0x7fe11c07db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:55:26.070 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:24.800+0000 7fe122e00700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fe11c10cb60 con 0x7fe11c108600 2024-04-16T16:55:26.070 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:24.801+0000 7fe11be00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe11c108600 0x7fe11c07db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:55:26.070 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:24.801+0000 7fe11be00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe11c108600 0x7fe11c07db70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51890/0 (socket says 172.21.15.45:51890) 2024-04-16T16:55:26.070 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:24.801+0000 7fe11be00700 1 -- 172.21.15.45:0/2611068994 learned_addr learned my addr 172.21.15.45:0/2611068994 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T16:55:26.070 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:24.801+0000 7fe11be00700 1 -- 172.21.15.45:0/2611068994 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe11c07e0b0 con 0x7fe11c108600 2024-04-16T16:55:26.070 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:24.801+0000 7fe11be00700 1 --2- 172.21.15.45:0/2611068994 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe11c108600 0x7fe11c07db70 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fe10c009fd0 tx=0x7fe10c003d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T16:55:26.073 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:24.802+0000 7fe119600700 1 -- 172.21.15.45:0/2611068994 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe10c020020 con 0x7fe11c108600 2024-04-16T16:55:26.074 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:24.802+0000 7fe119600700 1 -- 172.21.15.45:0/2611068994 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe10c00bba0 con 0x7fe11c108600 2024-04-16T16:55:26.074 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:24.802+0000 7fe119600700 1 -- 172.21.15.45:0/2611068994 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe10c017040 con 0x7fe11c108600 2024-04-16T16:55:26.074 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:24.802+0000 7fe122e00700 1 -- 172.21.15.45:0/2611068994 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe11c07c2a0 con 0x7fe11c108600 2024-04-16T16:55:26.074 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:24.802+0000 7fe122e00700 1 -- 172.21.15.45:0/2611068994 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe11c07c7e0 con 0x7fe11c108600 2024-04-16T16:55:26.074 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:24.803+0000 7fe119600700 1 -- 172.21.15.45:0/2611068994 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 12) ==== 46571+0+0 (secure 0 0 0) 0x7fe10c023020 con 0x7fe11c108600 2024-04-16T16:55:26.074 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:24.803+0000 7fe122e00700 1 -- 172.21.15.45:0/2611068994 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe11c064150 con 0x7fe11c108600 2024-04-16T16:55:26.074 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:24.804+0000 7fe119600700 1 --2- 172.21.15.45:0/2611068994 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fe100039660 0x7fe10003bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:55:26.074 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:24.804+0000 7fe119600700 1 -- 172.21.15.45:0/2611068994 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fe10c00f050 con 0x7fe11c108600 2024-04-16T16:55:26.074 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:24.804+0000 7fe113400700 1 --2- 172.21.15.45:0/2611068994 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fe100039660 0x7fe10003bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:55:26.074 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:24.805+0000 7fe113400700 1 --2- 172.21.15.45:0/2611068994 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fe100039660 0x7fe10003bb20 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fe104006fd0 tx=0x7fe104006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T16:55:26.074 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:24.810+0000 7fe119600700 1 -- 172.21.15.45:0/2611068994 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe10c02a020 con 0x7fe11c108600 2024-04-16T16:55:26.074 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:25.078+0000 7fe122e00700 1 -- 172.21.15.45:0/2611068994 --> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) -- 0x7fe11c07c9b0 con 0x7fe100039660 2024-04-16T16:55:26.074 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:25.358+0000 7fe119600700 1 -- 172.21.15.45:0/2611068994 <== mgr.14152 v2:172.21.15.45:6800/268030699 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+252 (secure 0 0 0) 0x7fe11c07c9b0 con 0x7fe100039660 2024-04-16T16:55:26.075 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:25.363+0000 7fe122e00700 1 -- 172.21.15.45:0/2611068994 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fe100039660 msgr2=0x7fe10003bb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:55:26.075 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:25.363+0000 7fe122e00700 1 --2- 172.21.15.45:0/2611068994 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fe100039660 0x7fe10003bb20 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fe104006fd0 tx=0x7fe104006e40 comp rx=0 tx=0).stop 2024-04-16T16:55:26.075 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:25.363+0000 7fe122e00700 1 -- 172.21.15.45:0/2611068994 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe11c108600 msgr2=0x7fe11c07db70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:55:26.075 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:25.363+0000 7fe122e00700 1 --2- 172.21.15.45:0/2611068994 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe11c108600 0x7fe11c07db70 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fe10c009fd0 tx=0x7fe10c003d00 comp rx=0 tx=0).stop 2024-04-16T16:55:26.075 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:25.363+0000 7fe122e00700 1 -- 172.21.15.45:0/2611068994 shutdown_connections 2024-04-16T16:55:26.075 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:25.363+0000 7fe122e00700 1 --2- 172.21.15.45:0/2611068994 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fe100039660 0x7fe10003bb20 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:55:26.075 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:25.363+0000 7fe122e00700 1 --2- 172.21.15.45:0/2611068994 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe11c108600 0x7fe11c07db70 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:55:26.075 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:25.363+0000 7fe122e00700 1 -- 172.21.15.45:0/2611068994 >> 172.21.15.45:0/2611068994 conn(0x7fe11c103bc0 msgr2=0x7fe11c104710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T16:55:26.075 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:25.363+0000 7fe122e00700 1 -- 172.21.15.45:0/2611068994 shutdown_connections 2024-04-16T16:55:26.075 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:25.364+0000 7fe122e00700 1 -- 172.21.15.45:0/2611068994 wait complete. 2024-04-16T16:55:26.076 INFO:teuthology.orchestra.run.smithi045.stdout:Fetching dashboard port number... 2024-04-16T16:55:27.978 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:55:27 smithi045 ceph-mon[31676]: mgrmap e13: x(active, since 6s) 2024-04-16T16:55:27.978 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:55:27 smithi045 ceph-mon[31676]: from='client.? 172.21.15.45:0/1927084740' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-04-16T16:55:28.403 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout 8443 2024-04-16T16:55:28.403 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:26.963+0000 7fdf9d800700 1 Processor -- start 2024-04-16T16:55:28.403 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:26.963+0000 7fdf9d800700 1 -- start start 2024-04-16T16:55:28.403 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:26.963+0000 7fdf9d800700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdf9810a7e0 0x7fdf9810ac00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:55:28.403 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:26.963+0000 7fdf9d800700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fdf9810b1d0 con 0x7fdf9810a7e0 2024-04-16T16:55:28.403 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:26.964+0000 7fdf96a00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdf9810a7e0 0x7fdf9810ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:55:28.407 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:26.964+0000 7fdf96a00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdf9810a7e0 0x7fdf9810ac00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51892/0 (socket says 172.21.15.45:51892) 2024-04-16T16:55:28.407 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:26.964+0000 7fdf96a00700 1 -- 172.21.15.45:0/879653267 learned_addr learned my addr 172.21.15.45:0/879653267 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T16:55:28.407 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:26.964+0000 7fdf96a00700 1 -- 172.21.15.45:0/879653267 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf9810b990 con 0x7fdf9810a7e0 2024-04-16T16:55:28.407 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:26.965+0000 7fdf96a00700 1 --2- 172.21.15.45:0/879653267 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdf9810a7e0 0x7fdf9810ac00 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fdf880098a0 tx=0x7fdf88009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64b29b0700f7148f server_cookie=0 in_seq=0 out_seq=0 2024-04-16T16:55:28.407 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:26.965+0000 7fdf96000700 1 -- 172.21.15.45:0/879653267 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdf88004160 con 0x7fdf9810a7e0 2024-04-16T16:55:28.407 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:26.965+0000 7fdf96000700 1 -- 172.21.15.45:0/879653267 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdf88004300 con 0x7fdf9810a7e0 2024-04-16T16:55:28.407 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:26.965+0000 7fdf96000700 1 -- 172.21.15.45:0/879653267 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdf88011b20 con 0x7fdf9810a7e0 2024-04-16T16:55:28.407 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:26.965+0000 7fdf9d800700 1 -- 172.21.15.45:0/879653267 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdf9810a7e0 msgr2=0x7fdf9810ac00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:55:28.407 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:26.965+0000 7fdf9d800700 1 --2- 172.21.15.45:0/879653267 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdf9810a7e0 0x7fdf9810ac00 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fdf880098a0 tx=0x7fdf88009bb0 comp rx=0 tx=0).stop 2024-04-16T16:55:28.407 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:26.966+0000 7fdf9d800700 1 -- 172.21.15.45:0/879653267 shutdown_connections 2024-04-16T16:55:28.407 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:26.966+0000 7fdf9d800700 1 --2- 172.21.15.45:0/879653267 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdf9810a7e0 0x7fdf9810ac00 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:55:28.408 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:26.966+0000 7fdf9d800700 1 -- 172.21.15.45:0/879653267 >> 172.21.15.45:0/879653267 conn(0x7fdf9807dd80 msgr2=0x7fdf981084d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T16:55:28.408 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:26.966+0000 7fdf9d800700 1 -- 172.21.15.45:0/879653267 shutdown_connections 2024-04-16T16:55:28.408 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:26.966+0000 7fdf9d800700 1 -- 172.21.15.45:0/879653267 wait complete. 2024-04-16T16:55:28.408 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:26.967+0000 7fdf9d800700 1 Processor -- start 2024-04-16T16:55:28.408 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:26.967+0000 7fdf9d800700 1 -- start start 2024-04-16T16:55:28.408 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:26.967+0000 7fdf9d800700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdf9810a7e0 0x7fdf9819e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:55:28.408 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:26.967+0000 7fdf9d800700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fdf9810ed40 con 0x7fdf9810a7e0 2024-04-16T16:55:28.408 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:26.968+0000 7fdf96a00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdf9810a7e0 0x7fdf9819e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:55:28.408 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:26.968+0000 7fdf96a00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdf9810a7e0 0x7fdf9819e6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51908/0 (socket says 172.21.15.45:51908) 2024-04-16T16:55:28.408 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:26.968+0000 7fdf96a00700 1 -- 172.21.15.45:0/1927084740 learned_addr learned my addr 172.21.15.45:0/1927084740 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T16:55:28.408 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:26.968+0000 7fdf96a00700 1 -- 172.21.15.45:0/1927084740 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf9819ebe0 con 0x7fdf9810a7e0 2024-04-16T16:55:28.408 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:26.968+0000 7fdf96a00700 1 --2- 172.21.15.45:0/1927084740 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdf9810a7e0 0x7fdf9819e6a0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fdf880117e0 tx=0x7fdf88011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T16:55:28.408 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:26.968+0000 7fdf8f400700 1 -- 172.21.15.45:0/1927084740 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdf88003bf0 con 0x7fdf9810a7e0 2024-04-16T16:55:28.408 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:26.968+0000 7fdf8f400700 1 -- 172.21.15.45:0/1927084740 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdf8801a910 con 0x7fdf9810a7e0 2024-04-16T16:55:28.408 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:26.968+0000 7fdf8f400700 1 -- 172.21.15.45:0/1927084740 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdf88024970 con 0x7fdf9810a7e0 2024-04-16T16:55:28.409 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:26.969+0000 7fdf9d800700 1 -- 172.21.15.45:0/1927084740 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf9819ee70 con 0x7fdf9810a7e0 2024-04-16T16:55:28.409 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:26.969+0000 7fdf9d800700 1 -- 172.21.15.45:0/1927084740 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf9819f2d0 con 0x7fdf9810a7e0 2024-04-16T16:55:28.411 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:26.970+0000 7fdf8f400700 1 -- 172.21.15.45:0/1927084740 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7fdf88022020 con 0x7fdf9810a7e0 2024-04-16T16:55:28.411 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:26.970+0000 7fdf9d800700 1 -- 172.21.15.45:0/1927084740 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf98064150 con 0x7fdf9810a7e0 2024-04-16T16:55:28.411 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:26.970+0000 7fdf8f400700 1 --2- 172.21.15.45:0/1927084740 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fdf7c039700 0x7fdf7c03bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:55:28.411 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:26.971+0000 7fdf8f400700 1 -- 172.21.15.45:0/1927084740 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fdf8804dd10 con 0x7fdf9810a7e0 2024-04-16T16:55:28.411 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:26.971+0000 7fdf8fe00700 1 --2- 172.21.15.45:0/1927084740 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fdf7c039700 0x7fdf7c03bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:55:28.411 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:26.971+0000 7fdf8fe00700 1 --2- 172.21.15.45:0/1927084740 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fdf7c039700 0x7fdf7c03bbc0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fdf80006fd0 tx=0x7fdf80006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T16:55:28.411 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:26.976+0000 7fdf8f400700 1 -- 172.21.15.45:0/1927084740 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fdf8802d910 con 0x7fdf9810a7e0 2024-04-16T16:55:28.411 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:27.211+0000 7fdf9d800700 1 -- 172.21.15.45:0/1927084740 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) -- 0x7fdf981a1ca0 con 0x7fdf9810a7e0 2024-04-16T16:55:28.411 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:27.212+0000 7fdf8f400700 1 -- 172.21.15.45:0/1927084740 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) ==== 112+0+5 (secure 0 0 0) 0x7fdf8802d3f0 con 0x7fdf9810a7e0 2024-04-16T16:55:28.411 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:27.215+0000 7fdf9d800700 1 -- 172.21.15.45:0/1927084740 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fdf7c039700 msgr2=0x7fdf7c03bbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:55:28.411 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:27.215+0000 7fdf9d800700 1 --2- 172.21.15.45:0/1927084740 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fdf7c039700 0x7fdf7c03bbc0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fdf80006fd0 tx=0x7fdf80006e40 comp rx=0 tx=0).stop 2024-04-16T16:55:28.412 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:27.215+0000 7fdf9d800700 1 -- 172.21.15.45:0/1927084740 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdf9810a7e0 msgr2=0x7fdf9819e6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:55:28.412 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:27.215+0000 7fdf9d800700 1 --2- 172.21.15.45:0/1927084740 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdf9810a7e0 0x7fdf9819e6a0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fdf880117e0 tx=0x7fdf88011ba0 comp rx=0 tx=0).stop 2024-04-16T16:55:28.412 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:27.216+0000 7fdf9d800700 1 -- 172.21.15.45:0/1927084740 shutdown_connections 2024-04-16T16:55:28.412 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:27.216+0000 7fdf9d800700 1 --2- 172.21.15.45:0/1927084740 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fdf7c039700 0x7fdf7c03bbc0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:55:28.412 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:27.216+0000 7fdf9d800700 1 --2- 172.21.15.45:0/1927084740 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdf9810a7e0 0x7fdf9819e6a0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:55:28.412 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:27.216+0000 7fdf9d800700 1 -- 172.21.15.45:0/1927084740 >> 172.21.15.45:0/1927084740 conn(0x7fdf9807dd80 msgr2=0x7fdf98107ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T16:55:28.412 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:27.216+0000 7fdf9d800700 1 -- 172.21.15.45:0/1927084740 shutdown_connections 2024-04-16T16:55:28.412 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:27.216+0000 7fdf9d800700 1 -- 172.21.15.45:0/1927084740 wait complete. 2024-04-16T16:55:28.442 INFO:teuthology.orchestra.run.smithi045.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-16T16:55:28.442 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stdout disabled 2024-04-16T16:55:28.466 INFO:teuthology.orchestra.run.smithi045.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-16T16:55:28.466 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stdout inactive 2024-04-16T16:55:28.466 INFO:teuthology.orchestra.run.smithi045.stdout:firewalld.service is not enabled 2024-04-16T16:55:28.467 INFO:teuthology.orchestra.run.smithi045.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-04-16T16:55:28.473 INFO:teuthology.orchestra.run.smithi045.stdout:Ceph Dashboard is now available at: 2024-04-16T16:55:28.473 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-16T16:55:28.474 INFO:teuthology.orchestra.run.smithi045.stdout: URL: https://smithi045.front.sepia.ceph.com:8443/ 2024-04-16T16:55:28.474 INFO:teuthology.orchestra.run.smithi045.stdout: User: admin 2024-04-16T16:55:28.474 INFO:teuthology.orchestra.run.smithi045.stdout: Password: eqmm1d86l9 2024-04-16T16:55:28.474 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-16T16:55:28.475 INFO:teuthology.orchestra.run.smithi045.stdout:Saving cluster configuration to /var/lib/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/config directory 2024-04-16T16:55:28.477 INFO:teuthology.orchestra.run.smithi045.stdout:Enabling autotune for osd_memory_target 2024-04-16T16:55:30.080 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:29.132+0000 7f4db7200700 1 Processor -- start 2024-04-16T16:55:30.080 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:29.132+0000 7f4db7200700 1 -- start start 2024-04-16T16:55:30.080 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:29.133+0000 7f4db7200700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4db807de00 0x7f4db807c300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:55:30.080 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:29.133+0000 7f4db7200700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f4db807c840 con 0x7f4db807de00 2024-04-16T16:55:30.080 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:29.133+0000 7f4db5e00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4db807de00 0x7f4db807c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:55:30.080 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:29.133+0000 7f4db5e00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4db807de00 0x7f4db807c300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51924/0 (socket says 172.21.15.45:51924) 2024-04-16T16:55:30.081 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:29.133+0000 7f4db5e00700 1 -- 172.21.15.45:0/3566488697 learned_addr learned my addr 172.21.15.45:0/3566488697 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T16:55:30.081 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:29.134+0000 7f4db5e00700 1 -- 172.21.15.45:0/3566488697 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4db807c9c0 con 0x7f4db807de00 2024-04-16T16:55:30.081 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:29.134+0000 7f4db5e00700 1 --2- 172.21.15.45:0/3566488697 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4db807de00 0x7f4db807c300 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f4da80098a0 tx=0x7f4da8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db2ef16e6120c6c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T16:55:30.081 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:29.134+0000 7f4db4a00700 1 -- 172.21.15.45:0/3566488697 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4da8004160 con 0x7f4db807de00 2024-04-16T16:55:30.081 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:29.134+0000 7f4db4a00700 1 -- 172.21.15.45:0/3566488697 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4da8004300 con 0x7f4db807de00 2024-04-16T16:55:30.081 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:29.134+0000 7f4db4a00700 1 -- 172.21.15.45:0/3566488697 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4da8011b20 con 0x7f4db807de00 2024-04-16T16:55:30.081 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:29.135+0000 7f4db7200700 1 -- 172.21.15.45:0/3566488697 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4db807de00 msgr2=0x7f4db807c300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:55:30.081 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:29.135+0000 7f4db7200700 1 --2- 172.21.15.45:0/3566488697 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4db807de00 0x7f4db807c300 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f4da80098a0 tx=0x7f4da8009bb0 comp rx=0 tx=0).stop 2024-04-16T16:55:30.081 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:29.135+0000 7f4db7200700 1 -- 172.21.15.45:0/3566488697 shutdown_connections 2024-04-16T16:55:30.081 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:29.135+0000 7f4db7200700 1 --2- 172.21.15.45:0/3566488697 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4db807de00 0x7f4db807c300 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:55:30.081 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:29.135+0000 7f4db7200700 1 -- 172.21.15.45:0/3566488697 >> 172.21.15.45:0/3566488697 conn(0x7f4db81039d0 msgr2=0x7f4db8105e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T16:55:30.081 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:29.135+0000 7f4db7200700 1 -- 172.21.15.45:0/3566488697 shutdown_connections 2024-04-16T16:55:30.081 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:29.135+0000 7f4db7200700 1 -- 172.21.15.45:0/3566488697 wait complete. 2024-04-16T16:55:30.081 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:29.136+0000 7f4db7200700 1 Processor -- start 2024-04-16T16:55:30.082 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:29.137+0000 7f4db7200700 1 -- start start 2024-04-16T16:55:30.082 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:29.137+0000 7f4db7200700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4db807de00 0x7f4db819e530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:55:30.082 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:29.137+0000 7f4db7200700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f4db810e9f0 con 0x7f4db807de00 2024-04-16T16:55:30.082 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:29.137+0000 7f4db5e00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4db807de00 0x7f4db819e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:55:30.082 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:29.137+0000 7f4db5e00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4db807de00 0x7f4db819e530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51938/0 (socket says 172.21.15.45:51938) 2024-04-16T16:55:30.082 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:29.137+0000 7f4db5e00700 1 -- 172.21.15.45:0/1640925206 learned_addr learned my addr 172.21.15.45:0/1640925206 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T16:55:30.085 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:29.138+0000 7f4db5e00700 1 -- 172.21.15.45:0/1640925206 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4db819ea70 con 0x7f4db807de00 2024-04-16T16:55:30.085 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:29.138+0000 7f4db5e00700 1 --2- 172.21.15.45:0/1640925206 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4db807de00 0x7f4db819e530 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f4da8011b60 tx=0x7f4da8011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T16:55:30.085 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:29.138+0000 7f4daea00700 1 -- 172.21.15.45:0/1640925206 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4da8003b60 con 0x7f4db807de00 2024-04-16T16:55:30.085 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:29.138+0000 7f4daea00700 1 -- 172.21.15.45:0/1640925206 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4da801a910 con 0x7f4db807de00 2024-04-16T16:55:30.085 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:29.138+0000 7f4daea00700 1 -- 172.21.15.45:0/1640925206 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4da8024930 con 0x7f4db807de00 2024-04-16T16:55:30.085 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:29.138+0000 7f4db7200700 1 -- 172.21.15.45:0/1640925206 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4db819ed00 con 0x7f4db807de00 2024-04-16T16:55:30.085 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:29.139+0000 7f4db7200700 1 -- 172.21.15.45:0/1640925206 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4db819f160 con 0x7f4db807de00 2024-04-16T16:55:30.085 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:29.140+0000 7f4db7200700 1 -- 172.21.15.45:0/1640925206 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4db8064150 con 0x7f4db807de00 2024-04-16T16:55:30.086 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:29.140+0000 7f4daea00700 1 -- 172.21.15.45:0/1640925206 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f4da8022020 con 0x7f4db807de00 2024-04-16T16:55:30.086 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:29.141+0000 7f4daea00700 1 --2- 172.21.15.45:0/1640925206 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f4d9c039700 0x7f4d9c03bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:55:30.086 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:29.141+0000 7f4daea00700 1 -- 172.21.15.45:0/1640925206 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f4da804db80 con 0x7f4db807de00 2024-04-16T16:55:30.086 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:29.141+0000 7f4db5400700 1 --2- 172.21.15.45:0/1640925206 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f4d9c039700 0x7f4d9c03bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:55:30.086 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:29.142+0000 7f4db5400700 1 --2- 172.21.15.45:0/1640925206 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f4d9c039700 0x7f4d9c03bbc0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f4da4006fd0 tx=0x7f4da4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T16:55:30.086 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:29.147+0000 7f4daea00700 1 -- 172.21.15.45:0/1640925206 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4da800b3e0 con 0x7f4db807de00 2024-04-16T16:55:30.086 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:29.378+0000 7f4db7200700 1 -- 172.21.15.45:0/1640925206 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) -- 0x7f4db81a1b30 con 0x7f4db807de00 2024-04-16T16:55:30.086 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:29.379+0000 7f4daea00700 1 -- 172.21.15.45:0/1640925206 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) ==== 127+0+0 (secure 0 0 0) 0x7f4da804de20 con 0x7f4db807de00 2024-04-16T16:55:30.086 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:29.382+0000 7f4db7200700 1 -- 172.21.15.45:0/1640925206 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f4d9c039700 msgr2=0x7f4d9c03bbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:55:30.086 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:29.382+0000 7f4db7200700 1 --2- 172.21.15.45:0/1640925206 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f4d9c039700 0x7f4d9c03bbc0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f4da4006fd0 tx=0x7f4da4006e40 comp rx=0 tx=0).stop 2024-04-16T16:55:30.086 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:29.382+0000 7f4db7200700 1 -- 172.21.15.45:0/1640925206 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4db807de00 msgr2=0x7f4db819e530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:55:30.086 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:29.382+0000 7f4db7200700 1 --2- 172.21.15.45:0/1640925206 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4db807de00 0x7f4db819e530 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f4da8011b60 tx=0x7f4da8011c40 comp rx=0 tx=0).stop 2024-04-16T16:55:30.086 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:29.382+0000 7f4db7200700 1 -- 172.21.15.45:0/1640925206 shutdown_connections 2024-04-16T16:55:30.086 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:29.382+0000 7f4db7200700 1 --2- 172.21.15.45:0/1640925206 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f4d9c039700 0x7f4d9c03bbc0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:55:30.087 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:29.382+0000 7f4db7200700 1 --2- 172.21.15.45:0/1640925206 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4db807de00 0x7f4db819e530 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:55:30.087 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:29.382+0000 7f4db7200700 1 -- 172.21.15.45:0/1640925206 >> 172.21.15.45:0/1640925206 conn(0x7f4db81039d0 msgr2=0x7f4db81045b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T16:55:30.087 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:29.382+0000 7f4db7200700 1 -- 172.21.15.45:0/1640925206 shutdown_connections 2024-04-16T16:55:30.087 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:29.382+0000 7f4db7200700 1 -- 172.21.15.45:0/1640925206 wait complete. 2024-04-16T16:55:31.791 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:30.695+0000 7f5209600700 1 Processor -- start 2024-04-16T16:55:31.791 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:30.695+0000 7f5209600700 1 -- start start 2024-04-16T16:55:31.791 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:30.695+0000 7f5209600700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5204109c80 0x7f520410a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:55:31.791 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:30.695+0000 7f5209600700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f52041081c0 con 0x7f5204109c80 2024-04-16T16:55:31.791 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:30.696+0000 7f5202a00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5204109c80 0x7f520410a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:55:31.792 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:30.696+0000 7f5202a00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5204109c80 0x7f520410a0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41976/0 (socket says 172.21.15.45:41976) 2024-04-16T16:55:31.792 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:30.696+0000 7f5202a00700 1 -- 172.21.15.45:0/1197221053 learned_addr learned my addr 172.21.15.45:0/1197221053 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T16:55:31.792 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:30.696+0000 7f5202a00700 1 -- 172.21.15.45:0/1197221053 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5204108340 con 0x7f5204109c80 2024-04-16T16:55:31.792 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:30.697+0000 7f5202a00700 1 --2- 172.21.15.45:0/1197221053 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5204109c80 0x7f520410a0a0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f51f40098a0 tx=0x7f51f4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d907b5afae3703fc server_cookie=0 in_seq=0 out_seq=0 2024-04-16T16:55:31.792 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:30.697+0000 7f5201600700 1 -- 172.21.15.45:0/1197221053 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f51f4004160 con 0x7f5204109c80 2024-04-16T16:55:31.792 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:30.697+0000 7f5201600700 1 -- 172.21.15.45:0/1197221053 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f51f4004300 con 0x7f5204109c80 2024-04-16T16:55:31.792 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:30.697+0000 7f5201600700 1 -- 172.21.15.45:0/1197221053 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f51f4011b20 con 0x7f5204109c80 2024-04-16T16:55:31.792 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:30.698+0000 7f5209600700 1 -- 172.21.15.45:0/1197221053 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5204109c80 msgr2=0x7f520410a0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:55:31.792 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:30.698+0000 7f5209600700 1 --2- 172.21.15.45:0/1197221053 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5204109c80 0x7f520410a0a0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f51f40098a0 tx=0x7f51f4009bb0 comp rx=0 tx=0).stop 2024-04-16T16:55:31.792 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:30.698+0000 7f5209600700 1 -- 172.21.15.45:0/1197221053 shutdown_connections 2024-04-16T16:55:31.792 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:30.698+0000 7f5209600700 1 --2- 172.21.15.45:0/1197221053 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5204109c80 0x7f520410a0a0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:55:31.793 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:30.698+0000 7f5209600700 1 -- 172.21.15.45:0/1197221053 >> 172.21.15.45:0/1197221053 conn(0x7f5204103b20 msgr2=0x7f5204105f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T16:55:31.793 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:30.698+0000 7f5209600700 1 -- 172.21.15.45:0/1197221053 shutdown_connections 2024-04-16T16:55:31.793 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:30.698+0000 7f5209600700 1 -- 172.21.15.45:0/1197221053 wait complete. 2024-04-16T16:55:31.793 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:30.700+0000 7f5209600700 1 Processor -- start 2024-04-16T16:55:31.793 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:30.700+0000 7f5209600700 1 -- start start 2024-04-16T16:55:31.793 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:30.701+0000 7f5209600700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f520407db30 0x7f520407c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:55:31.793 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:30.701+0000 7f5209600700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f520410cb50 con 0x7f520407db30 2024-04-16T16:55:31.793 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:30.701+0000 7f5202a00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f520407db30 0x7f520407c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:55:31.793 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:30.701+0000 7f5202a00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f520407db30 0x7f520407c2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41990/0 (socket says 172.21.15.45:41990) 2024-04-16T16:55:31.793 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:30.701+0000 7f5202a00700 1 -- 172.21.15.45:0/1960574747 learned_addr learned my addr 172.21.15.45:0/1960574747 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T16:55:31.793 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:30.701+0000 7f5202a00700 1 -- 172.21.15.45:0/1960574747 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f520407df50 con 0x7f520407db30 2024-04-16T16:55:31.793 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:30.702+0000 7f5202a00700 1 --2- 172.21.15.45:0/1960574747 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f520407db30 0x7f520407c2a0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f5204109600 tx=0x7f51f4011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T16:55:31.794 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:30.702+0000 7f51fb400700 1 -- 172.21.15.45:0/1960574747 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f51f40036a0 con 0x7f520407db30 2024-04-16T16:55:31.794 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:30.702+0000 7f51fb400700 1 -- 172.21.15.45:0/1960574747 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f51f401a460 con 0x7f520407db30 2024-04-16T16:55:31.794 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:30.702+0000 7f5209600700 1 -- 172.21.15.45:0/1960574747 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f520407c7e0 con 0x7f520407db30 2024-04-16T16:55:31.794 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:30.702+0000 7f51fb400700 1 -- 172.21.15.45:0/1960574747 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f51f401b440 con 0x7f520407db30 2024-04-16T16:55:31.794 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:30.702+0000 7f5209600700 1 -- 172.21.15.45:0/1960574747 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f520407cc40 con 0x7f520407db30 2024-04-16T16:55:31.794 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:30.703+0000 7f5209600700 1 -- 172.21.15.45:0/1960574747 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f520407e120 con 0x7f520407db30 2024-04-16T16:55:31.794 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:30.704+0000 7f51fb400700 1 -- 172.21.15.45:0/1960574747 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f51f401a610 con 0x7f520407db30 2024-04-16T16:55:31.794 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:30.704+0000 7f51fb400700 1 --2- 172.21.15.45:0/1960574747 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f51e80396a0 0x7f51e803bb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:55:31.794 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:30.704+0000 7f51fb400700 1 -- 172.21.15.45:0/1960574747 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f51f404df70 con 0x7f520407db30 2024-04-16T16:55:31.794 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:30.705+0000 7f5202000700 1 --2- 172.21.15.45:0/1960574747 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f51e80396a0 0x7f51e803bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:55:31.794 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:30.706+0000 7f5202000700 1 --2- 172.21.15.45:0/1960574747 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f51e80396a0 0x7f51e803bb60 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f51f0006fd0 tx=0x7f51f0006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T16:55:31.794 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:30.710+0000 7f51fb400700 1 -- 172.21.15.45:0/1960574747 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f51f402a2b0 con 0x7f520407db30 2024-04-16T16:55:31.795 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:31.062+0000 7f5209600700 1 -- 172.21.15.45:0/1960574747 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) -- 0x7f520407d0d0 con 0x7f520407db30 2024-04-16T16:55:31.795 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:31.068+0000 7f51fb400700 1 -- 172.21.15.45:0/1960574747 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v28)=0 set mgr/dashboard/cluster/status v28) ==== 153+0+0 (secure 0 0 0) 0x7f51f4003840 con 0x7f520407db30 2024-04-16T16:55:31.795 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:31.071+0000 7f5209600700 1 -- 172.21.15.45:0/1960574747 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f51e80396a0 msgr2=0x7f51e803bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:55:31.795 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:31.071+0000 7f5209600700 1 --2- 172.21.15.45:0/1960574747 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f51e80396a0 0x7f51e803bb60 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f51f0006fd0 tx=0x7f51f0006e40 comp rx=0 tx=0).stop 2024-04-16T16:55:31.795 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:31.071+0000 7f5209600700 1 -- 172.21.15.45:0/1960574747 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f520407db30 msgr2=0x7f520407c2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:55:31.795 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:31.071+0000 7f5209600700 1 --2- 172.21.15.45:0/1960574747 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f520407db30 0x7f520407c2a0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f5204109600 tx=0x7f51f4011ba0 comp rx=0 tx=0).stop 2024-04-16T16:55:31.795 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:31.071+0000 7f5209600700 1 -- 172.21.15.45:0/1960574747 shutdown_connections 2024-04-16T16:55:31.795 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:31.071+0000 7f5209600700 1 --2- 172.21.15.45:0/1960574747 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f51e80396a0 0x7f51e803bb60 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:55:31.795 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:31.071+0000 7f5209600700 1 --2- 172.21.15.45:0/1960574747 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f520407db30 0x7f520407c2a0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:55:31.795 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:31.071+0000 7f5209600700 1 -- 172.21.15.45:0/1960574747 >> 172.21.15.45:0/1960574747 conn(0x7f5204103b20 msgr2=0x7f5204104720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T16:55:31.795 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:31.072+0000 7f5209600700 1 -- 172.21.15.45:0/1960574747 shutdown_connections 2024-04-16T16:55:31.795 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-16T16:55:31.072+0000 7f5209600700 1 -- 172.21.15.45:0/1960574747 wait complete. 2024-04-16T16:55:31.796 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-04-16T16:55:31.796 INFO:teuthology.orchestra.run.smithi045.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-04-16T16:55:31.796 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-16T16:55:31.796 INFO:teuthology.orchestra.run.smithi045.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid b1fecd38-fc11-11ee-bc90-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-04-16T16:55:31.796 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-16T16:55:31.796 INFO:teuthology.orchestra.run.smithi045.stdout:Or, if you are only running a single cluster on this host: 2024-04-16T16:55:31.796 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-16T16:55:31.796 INFO:teuthology.orchestra.run.smithi045.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-04-16T16:55:31.796 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-16T16:55:31.796 INFO:teuthology.orchestra.run.smithi045.stdout:Please consider enabling telemetry to help improve Ceph: 2024-04-16T16:55:31.796 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-16T16:55:31.796 INFO:teuthology.orchestra.run.smithi045.stdout: ceph telemetry on 2024-04-16T16:55:31.796 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-16T16:55:31.797 INFO:teuthology.orchestra.run.smithi045.stdout:For more information see: 2024-04-16T16:55:31.797 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-16T16:55:31.797 INFO:teuthology.orchestra.run.smithi045.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-04-16T16:55:31.797 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-16T16:55:31.797 INFO:teuthology.orchestra.run.smithi045.stdout:Bootstrap complete. 2024-04-16T16:55:31.918 INFO:tasks.cephadm:Fetching config... 2024-04-16T16:55:31.918 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-04-16T16:55:31.919 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-04-16T16:55:32.006 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-04-16T16:55:32.006 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-04-16T16:55:32.006 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-04-16T16:55:32.097 INFO:tasks.cephadm:Fetching mon keyring... 2024-04-16T16:55:32.097 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-04-16T16:55:32.097 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd if=/var/lib/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/mon.a/keyring of=/dev/stdout 2024-04-16T16:55:32.232 INFO:tasks.cephadm:Fetching pub ssh key... 2024-04-16T16:55:32.233 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-04-16T16:55:32.233 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-04-16T16:55:32.281 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-04-16T16:55:32.281 DEBUG:teuthology.orchestra.run.smithi045:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC0HRkdd/nb7cJP9WLx41ZcucBZg8S6GffuqGVcdJyHYPttolr/DWL/oBs2x7IOjrQPQqKf7RfTWyErPsM/AVI8zSZ50wIhZ68382uHkFZ+h/rrraimekgWFstBalbFzu8D5rhwb44Zm6/2Yz836n3ujVG3MmvZjmDzfYURKFaVHMnndHW705uMZwigCkYSEfBxSZaMzdkoLHAOUBy79OzHGTpXKIHm63RhQi2CC+GaSMte3tYDI7ZifWzr8GP4Xt+XYC1gtp4mYXwsu9hALSreJeO+p/XTidVbapL8pyaGfJ/ZjAPTgtemAc7WzI2+0S0BYinFyzdbsv7XJ4Ico2MWCBNBy2brAYTdQX1DEGzSjlIMZLv+hMpnY+4Hh/PebySReztCYlm41lYeNTc/rEYu0f1vUwHQC3wCuyQbHDLmR8HxWV/5h3Ubq4W7faPxsaPXk7k2qbTPBgjKreITsGXcjbxraR7yXaIO+cNFf+ps6kEVUCeZXdV7DtmDetRHh7c= ceph-b1fecd38-fc11-11ee-bc90-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-16T16:55:32.405 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:55:32 smithi045 ceph-mon[31676]: from='client.? 172.21.15.45:0/1960574747' entity='client.admin' 2024-04-16T16:55:32.472 INFO:teuthology.orchestra.run.smithi045.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC0HRkdd/nb7cJP9WLx41ZcucBZg8S6GffuqGVcdJyHYPttolr/DWL/oBs2x7IOjrQPQqKf7RfTWyErPsM/AVI8zSZ50wIhZ68382uHkFZ+h/rrraimekgWFstBalbFzu8D5rhwb44Zm6/2Yz836n3ujVG3MmvZjmDzfYURKFaVHMnndHW705uMZwigCkYSEfBxSZaMzdkoLHAOUBy79OzHGTpXKIHm63RhQi2CC+GaSMte3tYDI7ZifWzr8GP4Xt+XYC1gtp4mYXwsu9hALSreJeO+p/XTidVbapL8pyaGfJ/ZjAPTgtemAc7WzI2+0S0BYinFyzdbsv7XJ4Ico2MWCBNBy2brAYTdQX1DEGzSjlIMZLv+hMpnY+4Hh/PebySReztCYlm41lYeNTc/rEYu0f1vUwHQC3wCuyQbHDLmR8HxWV/5h3Ubq4W7faPxsaPXk7k2qbTPBgjKreITsGXcjbxraR7yXaIO+cNFf+ps6kEVUCeZXdV7DtmDetRHh7c= ceph-b1fecd38-fc11-11ee-bc90-c7b262605968 2024-04-16T16:55:32.532 DEBUG:teuthology.orchestra.run.smithi104:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC0HRkdd/nb7cJP9WLx41ZcucBZg8S6GffuqGVcdJyHYPttolr/DWL/oBs2x7IOjrQPQqKf7RfTWyErPsM/AVI8zSZ50wIhZ68382uHkFZ+h/rrraimekgWFstBalbFzu8D5rhwb44Zm6/2Yz836n3ujVG3MmvZjmDzfYURKFaVHMnndHW705uMZwigCkYSEfBxSZaMzdkoLHAOUBy79OzHGTpXKIHm63RhQi2CC+GaSMte3tYDI7ZifWzr8GP4Xt+XYC1gtp4mYXwsu9hALSreJeO+p/XTidVbapL8pyaGfJ/ZjAPTgtemAc7WzI2+0S0BYinFyzdbsv7XJ4Ico2MWCBNBy2brAYTdQX1DEGzSjlIMZLv+hMpnY+4Hh/PebySReztCYlm41lYeNTc/rEYu0f1vUwHQC3wCuyQbHDLmR8HxWV/5h3Ubq4W7faPxsaPXk7k2qbTPBgjKreITsGXcjbxraR7yXaIO+cNFf+ps6kEVUCeZXdV7DtmDetRHh7c= ceph-b1fecd38-fc11-11ee-bc90-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-16T16:55:32.578 INFO:teuthology.orchestra.run.smithi104.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC0HRkdd/nb7cJP9WLx41ZcucBZg8S6GffuqGVcdJyHYPttolr/DWL/oBs2x7IOjrQPQqKf7RfTWyErPsM/AVI8zSZ50wIhZ68382uHkFZ+h/rrraimekgWFstBalbFzu8D5rhwb44Zm6/2Yz836n3ujVG3MmvZjmDzfYURKFaVHMnndHW705uMZwigCkYSEfBxSZaMzdkoLHAOUBy79OzHGTpXKIHm63RhQi2CC+GaSMte3tYDI7ZifWzr8GP4Xt+XYC1gtp4mYXwsu9hALSreJeO+p/XTidVbapL8pyaGfJ/ZjAPTgtemAc7WzI2+0S0BYinFyzdbsv7XJ4Ico2MWCBNBy2brAYTdQX1DEGzSjlIMZLv+hMpnY+4Hh/PebySReztCYlm41lYeNTc/rEYu0f1vUwHQC3wCuyQbHDLmR8HxWV/5h3Ubq4W7faPxsaPXk7k2qbTPBgjKreITsGXcjbxraR7yXaIO+cNFf+ps6kEVUCeZXdV7DtmDetRHh7c= ceph-b1fecd38-fc11-11ee-bc90-c7b262605968 2024-04-16T16:55:32.593 DEBUG:teuthology.orchestra.run.smithi117:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC0HRkdd/nb7cJP9WLx41ZcucBZg8S6GffuqGVcdJyHYPttolr/DWL/oBs2x7IOjrQPQqKf7RfTWyErPsM/AVI8zSZ50wIhZ68382uHkFZ+h/rrraimekgWFstBalbFzu8D5rhwb44Zm6/2Yz836n3ujVG3MmvZjmDzfYURKFaVHMnndHW705uMZwigCkYSEfBxSZaMzdkoLHAOUBy79OzHGTpXKIHm63RhQi2CC+GaSMte3tYDI7ZifWzr8GP4Xt+XYC1gtp4mYXwsu9hALSreJeO+p/XTidVbapL8pyaGfJ/ZjAPTgtemAc7WzI2+0S0BYinFyzdbsv7XJ4Ico2MWCBNBy2brAYTdQX1DEGzSjlIMZLv+hMpnY+4Hh/PebySReztCYlm41lYeNTc/rEYu0f1vUwHQC3wCuyQbHDLmR8HxWV/5h3Ubq4W7faPxsaPXk7k2qbTPBgjKreITsGXcjbxraR7yXaIO+cNFf+ps6kEVUCeZXdV7DtmDetRHh7c= ceph-b1fecd38-fc11-11ee-bc90-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-16T16:55:32.638 INFO:teuthology.orchestra.run.smithi117.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC0HRkdd/nb7cJP9WLx41ZcucBZg8S6GffuqGVcdJyHYPttolr/DWL/oBs2x7IOjrQPQqKf7RfTWyErPsM/AVI8zSZ50wIhZ68382uHkFZ+h/rrraimekgWFstBalbFzu8D5rhwb44Zm6/2Yz836n3ujVG3MmvZjmDzfYURKFaVHMnndHW705uMZwigCkYSEfBxSZaMzdkoLHAOUBy79OzHGTpXKIHm63RhQi2CC+GaSMte3tYDI7ZifWzr8GP4Xt+XYC1gtp4mYXwsu9hALSreJeO+p/XTidVbapL8pyaGfJ/ZjAPTgtemAc7WzI2+0S0BYinFyzdbsv7XJ4Ico2MWCBNBy2brAYTdQX1DEGzSjlIMZLv+hMpnY+4Hh/PebySReztCYlm41lYeNTc/rEYu0f1vUwHQC3wCuyQbHDLmR8HxWV/5h3Ubq4W7faPxsaPXk7k2qbTPBgjKreITsGXcjbxraR7yXaIO+cNFf+ps6kEVUCeZXdV7DtmDetRHh7c= ceph-b1fecd38-fc11-11ee-bc90-c7b262605968 2024-04-16T16:55:32.653 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b1fecd38-fc11-11ee-bc90-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-04-16T16:55:33.121 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/mon.a/config 2024-04-16T16:55:36.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:55:35.962+0000 7fc7d5e00700 1 -- 172.21.15.45:0/1199805530 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc7d0074f80 msgr2=0x7fc7d00753a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:55:36.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:55:35.962+0000 7fc7d5e00700 1 --2- 172.21.15.45:0/1199805530 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc7d0074f80 0x7fc7d00753a0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fc7c0009910 tx=0x7fc7c0009c20 comp rx=0 tx=0).stop 2024-04-16T16:55:36.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:55:35.962+0000 7fc7d5e00700 1 -- 172.21.15.45:0/1199805530 shutdown_connections 2024-04-16T16:55:36.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:55:35.962+0000 7fc7d5e00700 1 --2- 172.21.15.45:0/1199805530 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc7d0074f80 0x7fc7d00753a0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:55:36.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:55:35.962+0000 7fc7d5e00700 1 -- 172.21.15.45:0/1199805530 >> 172.21.15.45:0/1199805530 conn(0x7fc7d00fdb40 msgr2=0x7fc7d00fff60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T16:55:36.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:55:35.962+0000 7fc7d5e00700 1 -- 172.21.15.45:0/1199805530 shutdown_connections 2024-04-16T16:55:36.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:55:35.963+0000 7fc7d5e00700 1 -- 172.21.15.45:0/1199805530 wait complete. 2024-04-16T16:55:36.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:55:35.964+0000 7fc7d5e00700 1 Processor -- start 2024-04-16T16:55:36.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:55:35.964+0000 7fc7d5e00700 1 -- start start 2024-04-16T16:55:36.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:55:35.964+0000 7fc7d5e00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc7d0074f80 0x7fc7d0199f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:55:36.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:55:35.964+0000 7fc7d5e00700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fc7d010a910 con 0x7fc7d0074f80 2024-04-16T16:55:36.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:55:35.965+0000 7fc7cea00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc7d0074f80 0x7fc7d0199f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:55:36.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:55:35.965+0000 7fc7cea00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc7d0074f80 0x7fc7d0199f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42004/0 (socket says 172.21.15.45:42004) 2024-04-16T16:55:36.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:55:35.965+0000 7fc7cea00700 1 -- 172.21.15.45:0/620635658 learned_addr learned my addr 172.21.15.45:0/620635658 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T16:55:36.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:55:35.965+0000 7fc7cea00700 1 -- 172.21.15.45:0/620635658 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7d019a490 con 0x7fc7d0074f80 2024-04-16T16:55:36.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:55:35.965+0000 7fc7cea00700 1 --2- 172.21.15.45:0/620635658 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc7d0074f80 0x7fc7d0199f50 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fc7c0005950 tx=0x7fc7c0005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T16:55:36.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:55:35.966+0000 7fc7d4a00700 1 -- 172.21.15.45:0/620635658 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc7c0026070 con 0x7fc7d0074f80 2024-04-16T16:55:36.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:55:35.966+0000 7fc7d4a00700 1 -- 172.21.15.45:0/620635658 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc7c000bc30 con 0x7fc7d0074f80 2024-04-16T16:55:36.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:55:35.966+0000 7fc7d4a00700 1 -- 172.21.15.45:0/620635658 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc7c0003b30 con 0x7fc7d0074f80 2024-04-16T16:55:36.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:55:35.966+0000 7fc7d5e00700 1 -- 172.21.15.45:0/620635658 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7d019a720 con 0x7fc7d0074f80 2024-04-16T16:55:36.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:55:35.966+0000 7fc7d5e00700 1 -- 172.21.15.45:0/620635658 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7d019d410 con 0x7fc7d0074f80 2024-04-16T16:55:36.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:55:35.967+0000 7fc7d5e00700 1 -- 172.21.15.45:0/620635658 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7d010b090 con 0x7fc7d0074f80 2024-04-16T16:55:36.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:55:35.967+0000 7fc7d4a00700 1 -- 172.21.15.45:0/620635658 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7fc7c0029070 con 0x7fc7d0074f80 2024-04-16T16:55:36.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:55:35.968+0000 7fc7d4a00700 1 --2- 172.21.15.45:0/620635658 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fc7b40396b0 0x7fc7b403bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:55:36.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:55:35.968+0000 7fc7d4a00700 1 -- 172.21.15.45:0/620635658 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fc7c001e070 con 0x7fc7d0074f80 2024-04-16T16:55:36.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:55:35.968+0000 7fc7ce000700 1 --2- 172.21.15.45:0/620635658 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fc7b40396b0 0x7fc7b403bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:55:36.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:55:35.969+0000 7fc7ce000700 1 --2- 172.21.15.45:0/620635658 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fc7b40396b0 0x7fc7b403bb70 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fc7bc006fd0 tx=0x7fc7bc006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T16:55:36.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:55:35.975+0000 7fc7d4a00700 1 -- 172.21.15.45:0/620635658 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc7c0013070 con 0x7fc7d0074f80 2024-04-16T16:55:36.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:55:36.209+0000 7fc7d5e00700 1 -- 172.21.15.45:0/620635658 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) -- 0x7fc7d002dfe0 con 0x7fc7d0074f80 2024-04-16T16:55:36.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:55:36.215+0000 7fc7d4a00700 1 -- 172.21.15.45:0/620635658 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) ==== 125+0+0 (secure 0 0 0) 0x7fc7c00300c0 con 0x7fc7d0074f80 2024-04-16T16:55:36.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:55:36.219+0000 7fc7d5e00700 1 -- 172.21.15.45:0/620635658 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fc7b40396b0 msgr2=0x7fc7b403bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:55:36.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:55:36.219+0000 7fc7d5e00700 1 --2- 172.21.15.45:0/620635658 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fc7b40396b0 0x7fc7b403bb70 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fc7bc006fd0 tx=0x7fc7bc006e40 comp rx=0 tx=0).stop 2024-04-16T16:55:36.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:55:36.219+0000 7fc7d5e00700 1 -- 172.21.15.45:0/620635658 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc7d0074f80 msgr2=0x7fc7d0199f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:55:36.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:55:36.219+0000 7fc7d5e00700 1 --2- 172.21.15.45:0/620635658 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc7d0074f80 0x7fc7d0199f50 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fc7c0005950 tx=0x7fc7c0005030 comp rx=0 tx=0).stop 2024-04-16T16:55:36.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:55:36.220+0000 7fc7d5e00700 1 -- 172.21.15.45:0/620635658 shutdown_connections 2024-04-16T16:55:36.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:55:36.220+0000 7fc7d5e00700 1 --2- 172.21.15.45:0/620635658 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fc7b40396b0 0x7fc7b403bb70 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:55:36.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:55:36.220+0000 7fc7d5e00700 1 --2- 172.21.15.45:0/620635658 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc7d0074f80 0x7fc7d0199f50 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:55:36.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:55:36.220+0000 7fc7d5e00700 1 -- 172.21.15.45:0/620635658 >> 172.21.15.45:0/620635658 conn(0x7fc7d00fdb40 msgr2=0x7fc7d00fe520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T16:55:36.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:55:36.220+0000 7fc7d5e00700 1 -- 172.21.15.45:0/620635658 shutdown_connections 2024-04-16T16:55:36.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:55:36.220+0000 7fc7d5e00700 1 -- 172.21.15.45:0/620635658 wait complete. 2024-04-16T16:55:36.478 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:55:36 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:55:36.478 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:55:36 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:55:36.478 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:55:36 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi045", "name": "osd_memory_target"}]: dispatch 2024-04-16T16:55:36.478 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:55:36 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:55:36.478 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:55:36 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T16:55:36.478 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:55:36 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:55:36.478 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:55:36 smithi045 ceph-mon[31676]: from='client.? 172.21.15.45:0/620635658' entity='client.admin' 2024-04-16T16:55:36.478 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:55:36 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T16:55:36.478 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:55:36 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:55:36.984 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-04-16T16:55:36.984 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b1fecd38-fc11-11ee-bc90-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-04-16T16:55:37.433 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/mon.a/config 2024-04-16T16:55:40.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:55:40.127+0000 7f0f31e00700 1 -- 172.21.15.45:0/1238726737 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0f2c104790 msgr2=0x7f0f2c104bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:55:40.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:55:40.127+0000 7f0f31e00700 1 --2- 172.21.15.45:0/1238726737 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0f2c104790 0x7f0f2c104bb0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f0f1c009910 tx=0x7f0f1c009c20 comp rx=0 tx=0).stop 2024-04-16T16:55:40.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:55:40.127+0000 7f0f31e00700 1 -- 172.21.15.45:0/1238726737 shutdown_connections 2024-04-16T16:55:40.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:55:40.127+0000 7f0f31e00700 1 --2- 172.21.15.45:0/1238726737 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0f2c104790 0x7f0f2c104bb0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:55:40.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:55:40.127+0000 7f0f31e00700 1 -- 172.21.15.45:0/1238726737 >> 172.21.15.45:0/1238726737 conn(0x7f0f2c0ffd10 msgr2=0x7f0f2c102170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T16:55:40.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:55:40.127+0000 7f0f31e00700 1 -- 172.21.15.45:0/1238726737 shutdown_connections 2024-04-16T16:55:40.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:55:40.127+0000 7f0f31e00700 1 -- 172.21.15.45:0/1238726737 wait complete. 2024-04-16T16:55:40.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:55:40.128+0000 7f0f31e00700 1 Processor -- start 2024-04-16T16:55:40.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:55:40.129+0000 7f0f31e00700 1 -- start start 2024-04-16T16:55:40.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:55:40.129+0000 7f0f31e00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0f2c104790 0x7f0f2c199e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:55:40.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:55:40.129+0000 7f0f31e00700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f0f2c10a820 con 0x7f0f2c104790 2024-04-16T16:55:40.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:55:40.129+0000 7f0f2aa00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0f2c104790 0x7f0f2c199e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:55:40.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:55:40.129+0000 7f0f2aa00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0f2c104790 0x7f0f2c199e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:48842/0 (socket says 172.21.15.45:48842) 2024-04-16T16:55:40.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:55:40.129+0000 7f0f2aa00700 1 -- 172.21.15.45:0/3674938406 learned_addr learned my addr 172.21.15.45:0/3674938406 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T16:55:40.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:55:40.130+0000 7f0f2aa00700 1 -- 172.21.15.45:0/3674938406 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f2c19a3a0 con 0x7f0f2c104790 2024-04-16T16:55:40.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:55:40.130+0000 7f0f2aa00700 1 --2- 172.21.15.45:0/3674938406 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0f2c104790 0x7f0f2c199e60 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f0f1c006010 tx=0x7f0f1c005050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T16:55:40.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:55:40.130+0000 7f0f30a00700 1 -- 172.21.15.45:0/3674938406 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0f1c027070 con 0x7f0f2c104790 2024-04-16T16:55:40.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:55:40.130+0000 7f0f30a00700 1 -- 172.21.15.45:0/3674938406 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0f1c00be40 con 0x7f0f2c104790 2024-04-16T16:55:40.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:55:40.130+0000 7f0f30a00700 1 -- 172.21.15.45:0/3674938406 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0f1c00f460 con 0x7f0f2c104790 2024-04-16T16:55:40.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:55:40.130+0000 7f0f31e00700 1 -- 172.21.15.45:0/3674938406 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f2c19a630 con 0x7f0f2c104790 2024-04-16T16:55:40.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:55:40.131+0000 7f0f31e00700 1 -- 172.21.15.45:0/3674938406 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f2c19d320 con 0x7f0f2c104790 2024-04-16T16:55:40.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:55:40.132+0000 7f0f30a00700 1 -- 172.21.15.45:0/3674938406 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f0f1c02a070 con 0x7f0f2c104790 2024-04-16T16:55:40.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:55:40.132+0000 7f0f31e00700 1 -- 172.21.15.45:0/3674938406 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f2c10afa0 con 0x7f0f2c104790 2024-04-16T16:55:40.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:55:40.132+0000 7f0f30a00700 1 --2- 172.21.15.45:0/3674938406 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f0f100396b0 0x7f0f1003bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:55:40.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:55:40.132+0000 7f0f30a00700 1 -- 172.21.15.45:0/3674938406 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f0f1c01e030 con 0x7f0f2c104790 2024-04-16T16:55:40.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:55:40.133+0000 7f0f23e00700 1 --2- 172.21.15.45:0/3674938406 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f0f100396b0 0x7f0f1003bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:55:40.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:55:40.133+0000 7f0f23e00700 1 --2- 172.21.15.45:0/3674938406 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f0f100396b0 0x7f0f1003bb70 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f0f14006fd0 tx=0x7f0f14006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T16:55:40.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:55:40.138+0000 7f0f30a00700 1 -- 172.21.15.45:0/3674938406 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0f1c013070 con 0x7f0f2c104790 2024-04-16T16:55:40.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:55:40.380+0000 7f0f31e00700 1 -- 172.21.15.45:0/3674938406 --> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) -- 0x7f0f2c02e0f0 con 0x7f0f100396b0 2024-04-16T16:55:40.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:55:40.388+0000 7f0f30a00700 1 -- 172.21.15.45:0/3674938406 <== mgr.14152 v2:172.21.15.45:6800/268030699 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f0f2c02e0f0 con 0x7f0f100396b0 2024-04-16T16:55:40.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:55:40.391+0000 7f0f31e00700 1 -- 172.21.15.45:0/3674938406 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f0f100396b0 msgr2=0x7f0f1003bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:55:40.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:55:40.391+0000 7f0f31e00700 1 --2- 172.21.15.45:0/3674938406 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f0f100396b0 0x7f0f1003bb70 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f0f14006fd0 tx=0x7f0f14006e40 comp rx=0 tx=0).stop 2024-04-16T16:55:40.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:55:40.391+0000 7f0f31e00700 1 -- 172.21.15.45:0/3674938406 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0f2c104790 msgr2=0x7f0f2c199e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:55:40.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:55:40.391+0000 7f0f31e00700 1 --2- 172.21.15.45:0/3674938406 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0f2c104790 0x7f0f2c199e60 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f0f1c006010 tx=0x7f0f1c005050 comp rx=0 tx=0).stop 2024-04-16T16:55:40.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:55:40.392+0000 7f0f31e00700 1 -- 172.21.15.45:0/3674938406 shutdown_connections 2024-04-16T16:55:40.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:55:40.392+0000 7f0f31e00700 1 --2- 172.21.15.45:0/3674938406 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f0f100396b0 0x7f0f1003bb70 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:55:40.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:55:40.392+0000 7f0f31e00700 1 --2- 172.21.15.45:0/3674938406 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0f2c104790 0x7f0f2c199e60 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:55:40.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:55:40.392+0000 7f0f31e00700 1 -- 172.21.15.45:0/3674938406 >> 172.21.15.45:0/3674938406 conn(0x7f0f2c0ffd10 msgr2=0x7f0f2c100740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T16:55:40.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:55:40.392+0000 7f0f31e00700 1 -- 172.21.15.45:0/3674938406 shutdown_connections 2024-04-16T16:55:40.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:55:40.392+0000 7f0f31e00700 1 -- 172.21.15.45:0/3674938406 wait complete. 2024-04-16T16:55:41.236 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi104 2024-04-16T16:55:41.236 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2024-04-16T16:55:41.236 DEBUG:teuthology.orchestra.run.smithi104:> dd of=/etc/ceph/ceph.conf 2024-04-16T16:55:41.254 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2024-04-16T16:55:41.255 DEBUG:teuthology.orchestra.run.smithi104:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-16T16:55:41.312 INFO:tasks.cephadm:Adding host smithi104 to orchestrator... 2024-04-16T16:55:41.312 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b1fecd38-fc11-11ee-bc90-c7b262605968 -- ceph orch host add smithi104 2024-04-16T16:55:41.478 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:55:41 smithi045 ceph-mon[31676]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:55:41.478 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:55:41 smithi045 ceph-mon[31676]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T16:55:41.478 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:55:41 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:55:41.478 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:55:41 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T16:55:41.478 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:55:41 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:55:41.478 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:55:41 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T16:55:41.478 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:55:41 smithi045 ceph-mon[31676]: Updating smithi045:/etc/ceph/ceph.conf 2024-04-16T16:55:41.738 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/mon.a/config 2024-04-16T16:55:42.478 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:55:42 smithi045 ceph-mon[31676]: Updating smithi045:/var/lib/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/config/ceph.conf 2024-04-16T16:55:42.478 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:55:42 smithi045 ceph-mon[31676]: Updating smithi045:/etc/ceph/ceph.client.admin.keyring 2024-04-16T16:55:43.617 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:55:43 smithi045 ceph-mon[31676]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:55:43.617 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:55:43 smithi045 ceph-mon[31676]: Updating smithi045:/var/lib/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/config/ceph.client.admin.keyring 2024-04-16T16:55:43.617 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:55:43 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:55:43.617 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:55:43 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:55:43.617 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:55:43 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:55:43.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:55:43.884+0000 7fa0b8600700 1 -- 172.21.15.45:0/178785495 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa0b010daa0 msgr2=0x7fa0b010de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:55:43.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:55:43.884+0000 7fa0b8600700 1 --2- 172.21.15.45:0/178785495 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa0b010daa0 0x7fa0b010de80 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fa0a4009910 tx=0x7fa0a4009c20 comp rx=0 tx=0).stop 2024-04-16T16:55:43.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:55:43.884+0000 7fa0b8600700 1 -- 172.21.15.45:0/178785495 shutdown_connections 2024-04-16T16:55:43.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:55:43.884+0000 7fa0b8600700 1 --2- 172.21.15.45:0/178785495 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa0b010daa0 0x7fa0b010de80 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:55:43.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:55:43.884+0000 7fa0b8600700 1 -- 172.21.15.45:0/178785495 >> 172.21.15.45:0/178785495 conn(0x7fa0b0078290 msgr2=0x7fa0b007a6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T16:55:43.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:55:43.884+0000 7fa0b8600700 1 -- 172.21.15.45:0/178785495 shutdown_connections 2024-04-16T16:55:43.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:55:43.884+0000 7fa0b8600700 1 -- 172.21.15.45:0/178785495 wait complete. 2024-04-16T16:55:43.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:55:43.886+0000 7fa0b8600700 1 Processor -- start 2024-04-16T16:55:43.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:55:43.886+0000 7fa0b8600700 1 -- start start 2024-04-16T16:55:43.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:55:43.886+0000 7fa0b8600700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa0b010daa0 0x7fa0b019e600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:55:43.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:55:43.886+0000 7fa0b8600700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fa0b0113160 con 0x7fa0b010daa0 2024-04-16T16:55:43.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:55:43.887+0000 7fa0b5c00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa0b010daa0 0x7fa0b019e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:55:43.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:55:43.887+0000 7fa0b5c00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa0b010daa0 0x7fa0b019e600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:48854/0 (socket says 172.21.15.45:48854) 2024-04-16T16:55:43.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:55:43.887+0000 7fa0b5c00700 1 -- 172.21.15.45:0/2141514259 learned_addr learned my addr 172.21.15.45:0/2141514259 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T16:55:43.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:55:43.887+0000 7fa0b5c00700 1 -- 172.21.15.45:0/2141514259 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0b019eb40 con 0x7fa0b010daa0 2024-04-16T16:55:43.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:55:43.888+0000 7fa0b5c00700 1 --2- 172.21.15.45:0/2141514259 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa0b010daa0 0x7fa0b019e600 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fa0a4006010 tx=0x7fa0a4005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T16:55:43.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:55:43.888+0000 7fa0ae000700 1 -- 172.21.15.45:0/2141514259 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa0a4026070 con 0x7fa0b010daa0 2024-04-16T16:55:43.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:55:43.888+0000 7fa0ae000700 1 -- 172.21.15.45:0/2141514259 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa0a400bbd0 con 0x7fa0b010daa0 2024-04-16T16:55:43.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:55:43.888+0000 7fa0ae000700 1 -- 172.21.15.45:0/2141514259 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa0a40039d0 con 0x7fa0b010daa0 2024-04-16T16:55:43.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:55:43.888+0000 7fa0b8600700 1 -- 172.21.15.45:0/2141514259 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0b019ee30 con 0x7fa0b010daa0 2024-04-16T16:55:43.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:55:43.888+0000 7fa0b8600700 1 -- 172.21.15.45:0/2141514259 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0b01a2ed0 con 0x7fa0b010daa0 2024-04-16T16:55:43.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:55:43.889+0000 7fa0ae000700 1 -- 172.21.15.45:0/2141514259 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7fa0a4029070 con 0x7fa0b010daa0 2024-04-16T16:55:43.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:55:43.889+0000 7fa0b8600700 1 -- 172.21.15.45:0/2141514259 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0b010a1f0 con 0x7fa0b010daa0 2024-04-16T16:55:43.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:55:43.890+0000 7fa0ae000700 1 --2- 172.21.15.45:0/2141514259 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fa0980396b0 0x7fa09803bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:55:43.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:55:43.890+0000 7fa0ae000700 1 -- 172.21.15.45:0/2141514259 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fa0a401e070 con 0x7fa0b010daa0 2024-04-16T16:55:43.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:55:43.890+0000 7fa0b5200700 1 --2- 172.21.15.45:0/2141514259 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fa0980396b0 0x7fa09803bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:55:43.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:55:43.891+0000 7fa0b5200700 1 --2- 172.21.15.45:0/2141514259 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fa0980396b0 0x7fa09803bb70 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fa09c006fd0 tx=0x7fa09c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T16:55:43.897 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:55:43.896+0000 7fa0ae000700 1 -- 172.21.15.45:0/2141514259 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa0a4013070 con 0x7fa0b010daa0 2024-04-16T16:55:44.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:55:44.131+0000 7fa0b8600700 1 -- 172.21.15.45:0/2141514259 --> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi104", "target": ["mon-mgr", ""]}) -- 0x7fa0b019f960 con 0x7fa0980396b0 2024-04-16T16:55:45.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:55:45 smithi045 ceph-mon[31676]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:55:45.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:55:45 smithi045 ceph-mon[31676]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi104", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T16:55:46.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:55:46 smithi045 ceph-mon[31676]: Deploying cephadm binary to smithi104 2024-04-16T16:55:47.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:55:47 smithi045 ceph-mon[31676]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:55:49.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:55:49.036+0000 7fa0ae000700 1 -- 172.21.15.45:0/2141514259 <== mgr.14152 v2:172.21.15.45:6800/268030699 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7fa0b019f960 con 0x7fa0980396b0 2024-04-16T16:55:49.038 INFO:teuthology.orchestra.run.smithi045.stdout:Added host 'smithi104' with addr '172.21.15.104' 2024-04-16T16:55:49.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:55:49.039+0000 7fa0b8600700 1 -- 172.21.15.45:0/2141514259 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fa0980396b0 msgr2=0x7fa09803bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:55:49.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:55:49.039+0000 7fa0b8600700 1 --2- 172.21.15.45:0/2141514259 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fa0980396b0 0x7fa09803bb70 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fa09c006fd0 tx=0x7fa09c006e40 comp rx=0 tx=0).stop 2024-04-16T16:55:49.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:55:49.040+0000 7fa0b8600700 1 -- 172.21.15.45:0/2141514259 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa0b010daa0 msgr2=0x7fa0b019e600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:55:49.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:55:49.040+0000 7fa0b8600700 1 --2- 172.21.15.45:0/2141514259 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa0b010daa0 0x7fa0b019e600 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fa0a4006010 tx=0x7fa0a4005030 comp rx=0 tx=0).stop 2024-04-16T16:55:49.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:55:49.040+0000 7fa0b8600700 1 -- 172.21.15.45:0/2141514259 shutdown_connections 2024-04-16T16:55:49.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:55:49.040+0000 7fa0b8600700 1 --2- 172.21.15.45:0/2141514259 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fa0980396b0 0x7fa09803bb70 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:55:49.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:55:49.040+0000 7fa0b8600700 1 --2- 172.21.15.45:0/2141514259 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa0b010daa0 0x7fa0b019e600 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:55:49.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:55:49.040+0000 7fa0b8600700 1 -- 172.21.15.45:0/2141514259 >> 172.21.15.45:0/2141514259 conn(0x7fa0b0078290 msgr2=0x7fa0b01104c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T16:55:49.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:55:49.040+0000 7fa0b8600700 1 -- 172.21.15.45:0/2141514259 shutdown_connections 2024-04-16T16:55:49.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:55:49.040+0000 7fa0b8600700 1 -- 172.21.15.45:0/2141514259 wait complete. 2024-04-16T16:55:49.653 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:55:49 smithi045 ceph-mon[31676]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:55:49.653 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:55:49 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:55:49.653 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:55:49 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T16:55:49.702 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b1fecd38-fc11-11ee-bc90-c7b262605968 -- ceph orch host ls --format=json 2024-04-16T16:55:50.168 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/mon.a/config 2024-04-16T16:55:50.978 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:55:50 smithi045 ceph-mon[31676]: Added host smithi104 2024-04-16T16:55:50.978 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:55:50 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:55:50.978 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:55:50 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:55:51.862 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:55:51 smithi045 ceph-mon[31676]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:55:52.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:55:52.125+0000 7f3ec3e00700 1 -- 172.21.15.45:0/3451702760 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3ec4074e90 msgr2=0x7f3ec4075270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:55:52.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:55:52.125+0000 7f3ec3e00700 1 --2- 172.21.15.45:0/3451702760 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3ec4074e90 0x7f3ec4075270 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f3eb4009910 tx=0x7f3eb4009c20 comp rx=0 tx=0).stop 2024-04-16T16:55:52.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:55:52.125+0000 7f3ec3e00700 1 -- 172.21.15.45:0/3451702760 shutdown_connections 2024-04-16T16:55:52.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:55:52.125+0000 7f3ec3e00700 1 --2- 172.21.15.45:0/3451702760 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3ec4074e90 0x7f3ec4075270 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:55:52.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:55:52.125+0000 7f3ec3e00700 1 -- 172.21.15.45:0/3451702760 >> 172.21.15.45:0/3451702760 conn(0x7f3ec40fde50 msgr2=0x7f3ec4100270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T16:55:52.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:55:52.125+0000 7f3ec3e00700 1 -- 172.21.15.45:0/3451702760 shutdown_connections 2024-04-16T16:55:52.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:55:52.125+0000 7f3ec3e00700 1 -- 172.21.15.45:0/3451702760 wait complete. 2024-04-16T16:55:52.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:55:52.126+0000 7f3ec3e00700 1 Processor -- start 2024-04-16T16:55:52.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:55:52.127+0000 7f3ec3e00700 1 -- start start 2024-04-16T16:55:52.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:55:52.127+0000 7f3ec3e00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3ec4074e90 0x7f3ec419e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:55:52.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:55:52.127+0000 7f3ec3e00700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f3ec4112fe0 con 0x7f3ec4074e90 2024-04-16T16:55:52.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:55:52.128+0000 7f3ec2a00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3ec4074e90 0x7f3ec419e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:55:52.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:55:52.128+0000 7f3ec2a00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3ec4074e90 0x7f3ec419e4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54038/0 (socket says 172.21.15.45:54038) 2024-04-16T16:55:52.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:55:52.128+0000 7f3ec2a00700 1 -- 172.21.15.45:0/1770534740 learned_addr learned my addr 172.21.15.45:0/1770534740 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T16:55:52.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:55:52.128+0000 7f3ec2a00700 1 -- 172.21.15.45:0/1770534740 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ec419e9e0 con 0x7f3ec4074e90 2024-04-16T16:55:52.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:55:52.128+0000 7f3ec2a00700 1 --2- 172.21.15.45:0/1770534740 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3ec4074e90 0x7f3ec419e4a0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f3eb4006010 tx=0x7f3eb4005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T16:55:52.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:55:52.129+0000 7f3ebb400700 1 -- 172.21.15.45:0/1770534740 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3eb4026070 con 0x7f3ec4074e90 2024-04-16T16:55:52.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:55:52.129+0000 7f3ebb400700 1 -- 172.21.15.45:0/1770534740 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3eb400bc30 con 0x7f3ec4074e90 2024-04-16T16:55:52.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:55:52.129+0000 7f3ec3e00700 1 -- 172.21.15.45:0/1770534740 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ec419ec70 con 0x7f3ec4074e90 2024-04-16T16:55:52.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:55:52.129+0000 7f3ec3e00700 1 -- 172.21.15.45:0/1770534740 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ec419f0d0 con 0x7f3ec4074e90 2024-04-16T16:55:52.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:55:52.129+0000 7f3ebb400700 1 -- 172.21.15.45:0/1770534740 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3eb40039a0 con 0x7f3ec4074e90 2024-04-16T16:55:52.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:55:52.130+0000 7f3ec3e00700 1 -- 172.21.15.45:0/1770534740 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ea4005330 con 0x7f3ec4074e90 2024-04-16T16:55:52.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:55:52.130+0000 7f3ebb400700 1 -- 172.21.15.45:0/1770534740 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f3eb4029070 con 0x7f3ec4074e90 2024-04-16T16:55:52.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:55:52.131+0000 7f3ebb400700 1 --2- 172.21.15.45:0/1770534740 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f3ea8039700 0x7f3ea803bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:55:52.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:55:52.131+0000 7f3ebb400700 1 -- 172.21.15.45:0/1770534740 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f3eb401e070 con 0x7f3ec4074e90 2024-04-16T16:55:52.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:55:52.131+0000 7f3ec2000700 1 --2- 172.21.15.45:0/1770534740 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f3ea8039700 0x7f3ea803bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:55:52.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:55:52.132+0000 7f3ec2000700 1 --2- 172.21.15.45:0/1770534740 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f3ea8039700 0x7f3ea803bbc0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f3eb0006fd0 tx=0x7f3eb0006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T16:55:52.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:55:52.138+0000 7f3ebb400700 1 -- 172.21.15.45:0/1770534740 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3eb4013070 con 0x7f3ec4074e90 2024-04-16T16:55:52.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:55:52.375+0000 7f3ec3e00700 1 -- 172.21.15.45:0/1770534740 --> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f3ea4000bc0 con 0x7f3ea8039700 2024-04-16T16:55:52.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:55:52.377+0000 7f3ebb400700 1 -- 172.21.15.45:0/1770534740 <== mgr.14152 v2:172.21.15.45:6800/268030699 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+160 (secure 0 0 0) 0x7f3ea4000bc0 con 0x7f3ea8039700 2024-04-16T16:55:52.378 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-16T16:55:52.378 INFO:teuthology.orchestra.run.smithi045.stdout:[{"addr": "172.21.15.45", "hostname": "smithi045", "labels": [], "status": ""}, {"addr": "172.21.15.104", "hostname": "smithi104", "labels": [], "status": ""}] 2024-04-16T16:55:52.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:55:52.380+0000 7f3ec3e00700 1 -- 172.21.15.45:0/1770534740 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f3ea8039700 msgr2=0x7f3ea803bbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:55:52.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:55:52.380+0000 7f3ec3e00700 1 --2- 172.21.15.45:0/1770534740 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f3ea8039700 0x7f3ea803bbc0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f3eb0006fd0 tx=0x7f3eb0006e40 comp rx=0 tx=0).stop 2024-04-16T16:55:52.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:55:52.380+0000 7f3ec3e00700 1 -- 172.21.15.45:0/1770534740 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3ec4074e90 msgr2=0x7f3ec419e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:55:52.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:55:52.380+0000 7f3ec3e00700 1 --2- 172.21.15.45:0/1770534740 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3ec4074e90 0x7f3ec419e4a0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f3eb4006010 tx=0x7f3eb4005030 comp rx=0 tx=0).stop 2024-04-16T16:55:52.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:55:52.380+0000 7f3ec3e00700 1 -- 172.21.15.45:0/1770534740 shutdown_connections 2024-04-16T16:55:52.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:55:52.380+0000 7f3ec3e00700 1 --2- 172.21.15.45:0/1770534740 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f3ea8039700 0x7f3ea803bbc0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:55:52.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:55:52.380+0000 7f3ec3e00700 1 --2- 172.21.15.45:0/1770534740 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3ec4074e90 0x7f3ec419e4a0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:55:52.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:55:52.380+0000 7f3ec3e00700 1 -- 172.21.15.45:0/1770534740 >> 172.21.15.45:0/1770534740 conn(0x7f3ec40fde50 msgr2=0x7f3ec4104090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T16:55:52.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:55:52.380+0000 7f3ec3e00700 1 -- 172.21.15.45:0/1770534740 shutdown_connections 2024-04-16T16:55:52.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:55:52.380+0000 7f3ec3e00700 1 -- 172.21.15.45:0/1770534740 wait complete. 2024-04-16T16:55:53.064 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:55:52 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:55:53.064 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:55:52 smithi045 ceph-mon[31676]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:55:53.065 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:55:52 smithi045 ceph-mon[31676]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-16T16:55:53.130 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi117 2024-04-16T16:55:53.130 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-04-16T16:55:53.130 DEBUG:teuthology.orchestra.run.smithi117:> dd of=/etc/ceph/ceph.conf 2024-04-16T16:55:53.149 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-04-16T16:55:53.149 DEBUG:teuthology.orchestra.run.smithi117:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-16T16:55:53.206 INFO:tasks.cephadm:Adding host smithi117 to orchestrator... 2024-04-16T16:55:53.206 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b1fecd38-fc11-11ee-bc90-c7b262605968 -- ceph orch host add smithi117 2024-04-16T16:55:53.636 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/mon.a/config 2024-04-16T16:55:55.228 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:55:55 smithi045 ceph-mon[31676]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:55:55.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:55:55.755+0000 7f6c76600700 1 -- 172.21.15.45:0/3815068283 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6c70074e90 msgr2=0x7f6c70075270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:55:55.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:55:55.755+0000 7f6c76600700 1 --2- 172.21.15.45:0/3815068283 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6c70074e90 0x7f6c70075270 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f6c6000b0a0 tx=0x7f6c6000b3b0 comp rx=0 tx=0).stop 2024-04-16T16:55:55.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:55:55.755+0000 7f6c76600700 1 -- 172.21.15.45:0/3815068283 shutdown_connections 2024-04-16T16:55:55.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:55:55.756+0000 7f6c76600700 1 --2- 172.21.15.45:0/3815068283 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6c70074e90 0x7f6c70075270 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:55:55.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:55:55.756+0000 7f6c76600700 1 -- 172.21.15.45:0/3815068283 >> 172.21.15.45:0/3815068283 conn(0x7f6c700fe030 msgr2=0x7f6c70100450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T16:55:55.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:55:55.756+0000 7f6c76600700 1 -- 172.21.15.45:0/3815068283 shutdown_connections 2024-04-16T16:55:55.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:55:55.756+0000 7f6c76600700 1 -- 172.21.15.45:0/3815068283 wait complete. 2024-04-16T16:55:55.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:55:55.757+0000 7f6c76600700 1 Processor -- start 2024-04-16T16:55:55.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:55:55.757+0000 7f6c76600700 1 -- start start 2024-04-16T16:55:55.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:55:55.757+0000 7f6c76600700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6c70074e90 0x7f6c7019e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:55:55.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:55:55.757+0000 7f6c76600700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f6c701131a0 con 0x7f6c70074e90 2024-04-16T16:55:55.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:55:55.758+0000 7f6c6f400700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6c70074e90 0x7f6c7019e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:55:55.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:55:55.758+0000 7f6c6f400700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6c70074e90 0x7f6c7019e5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54066/0 (socket says 172.21.15.45:54066) 2024-04-16T16:55:55.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:55:55.758+0000 7f6c6f400700 1 -- 172.21.15.45:0/1848143028 learned_addr learned my addr 172.21.15.45:0/1848143028 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T16:55:55.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:55:55.758+0000 7f6c6f400700 1 -- 172.21.15.45:0/1848143028 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c7019eb10 con 0x7f6c70074e90 2024-04-16T16:55:55.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:55:55.759+0000 7f6c6f400700 1 --2- 172.21.15.45:0/1848143028 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6c70074e90 0x7f6c7019e5d0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f6c60000f80 tx=0x7f6c6000bd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T16:55:55.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:55:55.759+0000 7f6c6cc00700 1 -- 172.21.15.45:0/1848143028 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6c60020020 con 0x7f6c70074e90 2024-04-16T16:55:55.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:55:55.759+0000 7f6c6cc00700 1 -- 172.21.15.45:0/1848143028 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c60018040 con 0x7f6c70074e90 2024-04-16T16:55:55.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:55:55.759+0000 7f6c76600700 1 -- 172.21.15.45:0/1848143028 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c7019eda0 con 0x7f6c70074e90 2024-04-16T16:55:55.761 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:55:55.759+0000 7f6c6cc00700 1 -- 172.21.15.45:0/1848143028 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6c60017070 con 0x7f6c70074e90 2024-04-16T16:55:55.761 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:55:55.759+0000 7f6c76600700 1 -- 172.21.15.45:0/1848143028 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c701a2ea0 con 0x7f6c70074e90 2024-04-16T16:55:55.761 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:55:55.760+0000 7f6c6cc00700 1 -- 172.21.15.45:0/1848143028 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f6c60023020 con 0x7f6c70074e90 2024-04-16T16:55:55.761 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:55:55.760+0000 7f6c76600700 1 -- 172.21.15.45:0/1848143028 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6c700765d0 con 0x7f6c70074e90 2024-04-16T16:55:55.761 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:55:55.761+0000 7f6c6cc00700 1 --2- 172.21.15.45:0/1848143028 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f6c540396f0 0x7f6c5403bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:55:55.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:55:55.761+0000 7f6c6cc00700 1 -- 172.21.15.45:0/1848143028 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f6c6000f050 con 0x7f6c70074e90 2024-04-16T16:55:55.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:55:55.761+0000 7f6c66a00700 1 --2- 172.21.15.45:0/1848143028 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f6c540396f0 0x7f6c5403bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:55:55.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:55:55.762+0000 7f6c66a00700 1 --2- 172.21.15.45:0/1848143028 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f6c540396f0 0x7f6c5403bbb0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f6c58006fd0 tx=0x7f6c58006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T16:55:55.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:55:55.767+0000 7f6c6cc00700 1 -- 172.21.15.45:0/1848143028 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6c6002a020 con 0x7f6c70074e90 2024-04-16T16:55:56.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:55:56.006+0000 7f6c76600700 1 -- 172.21.15.45:0/1848143028 --> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi117", "target": ["mon-mgr", ""]}) -- 0x7f6c7019f930 con 0x7f6c540396f0 2024-04-16T16:55:56.478 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:55:56 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:55:56.479 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:55:56 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:55:56.479 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:55:56 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:55:56.479 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:55:56 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:55:56.479 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:55:56 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi104", "name": "osd_memory_target"}]: dispatch 2024-04-16T16:55:56.479 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:55:56 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:55:56.479 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:55:56 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T16:55:56.479 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:55:56 smithi045 ceph-mon[31676]: Updating smithi104:/etc/ceph/ceph.conf 2024-04-16T16:55:56.479 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:55:56 smithi045 ceph-mon[31676]: Updating smithi104:/var/lib/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/config/ceph.conf 2024-04-16T16:55:56.479 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:55:56 smithi045 ceph-mon[31676]: Updating smithi104:/etc/ceph/ceph.client.admin.keyring 2024-04-16T16:55:57.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:55:57 smithi045 ceph-mon[31676]: Updating smithi104:/var/lib/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/config/ceph.client.admin.keyring 2024-04-16T16:55:57.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:55:57 smithi045 ceph-mon[31676]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:55:57.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:55:57 smithi045 ceph-mon[31676]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi117", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T16:55:57.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:55:57 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:55:57.729 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:55:57 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:55:57.729 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:55:57 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:55:57.729 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:55:57 smithi045 ceph-mon[31676]: Deploying cephadm binary to smithi117 2024-04-16T16:55:59.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:55:59 smithi045 ceph-mon[31676]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:56:00.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:56:00.947+0000 7f6c6cc00700 1 -- 172.21.15.45:0/1848143028 <== mgr.14152 v2:172.21.15.45:6800/268030699 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f6c7019f930 con 0x7f6c540396f0 2024-04-16T16:56:00.948 INFO:teuthology.orchestra.run.smithi045.stdout:Added host 'smithi117' with addr '172.21.15.117' 2024-04-16T16:56:00.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:56:00.950+0000 7f6c76600700 1 -- 172.21.15.45:0/1848143028 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f6c540396f0 msgr2=0x7f6c5403bbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:56:00.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:56:00.950+0000 7f6c76600700 1 --2- 172.21.15.45:0/1848143028 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f6c540396f0 0x7f6c5403bbb0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f6c58006fd0 tx=0x7f6c58006e40 comp rx=0 tx=0).stop 2024-04-16T16:56:00.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:56:00.950+0000 7f6c76600700 1 -- 172.21.15.45:0/1848143028 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6c70074e90 msgr2=0x7f6c7019e5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:56:00.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:56:00.950+0000 7f6c76600700 1 --2- 172.21.15.45:0/1848143028 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6c70074e90 0x7f6c7019e5d0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f6c60000f80 tx=0x7f6c6000bd20 comp rx=0 tx=0).stop 2024-04-16T16:56:00.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:56:00.951+0000 7f6c76600700 1 -- 172.21.15.45:0/1848143028 shutdown_connections 2024-04-16T16:56:00.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:56:00.951+0000 7f6c76600700 1 --2- 172.21.15.45:0/1848143028 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f6c540396f0 0x7f6c5403bbb0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:56:00.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:56:00.951+0000 7f6c76600700 1 --2- 172.21.15.45:0/1848143028 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6c70074e90 0x7f6c7019e5d0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:56:00.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:56:00.951+0000 7f6c76600700 1 -- 172.21.15.45:0/1848143028 >> 172.21.15.45:0/1848143028 conn(0x7f6c700fe030 msgr2=0x7f6c700ff9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T16:56:00.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:56:00.951+0000 7f6c76600700 1 -- 172.21.15.45:0/1848143028 shutdown_connections 2024-04-16T16:56:00.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:56:00.951+0000 7f6c76600700 1 -- 172.21.15.45:0/1848143028 wait complete. 2024-04-16T16:56:01.617 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:56:01 smithi045 ceph-mon[31676]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:56:01.617 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:56:01 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:56:01.618 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:56:01 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T16:56:01.697 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b1fecd38-fc11-11ee-bc90-c7b262605968 -- ceph orch host ls --format=json 2024-04-16T16:56:02.164 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/mon.a/config 2024-04-16T16:56:02.629 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:56:02 smithi045 ceph-mon[31676]: Added host smithi117 2024-04-16T16:56:02.629 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:56:02 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:56:02.629 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:56:02 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:56:03.806 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:56:03 smithi045 ceph-mon[31676]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:56:04.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:56:04.310+0000 7f5582400700 1 -- 172.21.15.45:0/690756609 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f557c106aa0 msgr2=0x7f557c106e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:56:04.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:56:04.310+0000 7f5582400700 1 --2- 172.21.15.45:0/690756609 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f557c106aa0 0x7f557c106e80 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f557000aa20 tx=0x7f557000ad30 comp rx=0 tx=0).stop 2024-04-16T16:56:04.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:56:04.311+0000 7f5582400700 1 -- 172.21.15.45:0/690756609 shutdown_connections 2024-04-16T16:56:04.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:56:04.311+0000 7f5582400700 1 --2- 172.21.15.45:0/690756609 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f557c106aa0 0x7f557c106e80 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:56:04.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:56:04.311+0000 7f5582400700 1 -- 172.21.15.45:0/690756609 >> 172.21.15.45:0/690756609 conn(0x7f557c0fc0a0 msgr2=0x7f557c0fe4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T16:56:04.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:56:04.311+0000 7f5582400700 1 -- 172.21.15.45:0/690756609 shutdown_connections 2024-04-16T16:56:04.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:56:04.311+0000 7f5582400700 1 -- 172.21.15.45:0/690756609 wait complete. 2024-04-16T16:56:04.315 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:56:04.312+0000 7f5582400700 1 Processor -- start 2024-04-16T16:56:04.315 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:56:04.312+0000 7f5582400700 1 -- start start 2024-04-16T16:56:04.315 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:56:04.312+0000 7f5582400700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f557c106aa0 0x7f557c102570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:56:04.315 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:56:04.312+0000 7f5582400700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f557c10a7d0 con 0x7f557c106aa0 2024-04-16T16:56:04.315 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:56:04.313+0000 7f5581000700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f557c106aa0 0x7f557c102570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:56:04.315 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:56:04.313+0000 7f5581000700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f557c106aa0 0x7f557c102570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:44450/0 (socket says 172.21.15.45:44450) 2024-04-16T16:56:04.315 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:56:04.313+0000 7f5581000700 1 -- 172.21.15.45:0/3940584444 learned_addr learned my addr 172.21.15.45:0/3940584444 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T16:56:04.315 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:56:04.313+0000 7f5581000700 1 -- 172.21.15.45:0/3940584444 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f557c104460 con 0x7f557c106aa0 2024-04-16T16:56:04.315 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:56:04.313+0000 7f5581000700 1 --2- 172.21.15.45:0/3940584444 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f557c106aa0 0x7f557c102570 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f5570004cd0 tx=0x7f5570004db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T16:56:04.315 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:56:04.314+0000 7f5579600700 1 -- 172.21.15.45:0/3940584444 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5570005310 con 0x7f557c106aa0 2024-04-16T16:56:04.315 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:56:04.314+0000 7f5579600700 1 -- 172.21.15.45:0/3940584444 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f55700054b0 con 0x7f557c106aa0 2024-04-16T16:56:04.315 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:56:04.314+0000 7f5579600700 1 -- 172.21.15.45:0/3940584444 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f557000e7f0 con 0x7f557c106aa0 2024-04-16T16:56:04.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:56:04.314+0000 7f5582400700 1 -- 172.21.15.45:0/3940584444 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f557c102ab0 con 0x7f557c106aa0 2024-04-16T16:56:04.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:56:04.314+0000 7f5582400700 1 -- 172.21.15.45:0/3940584444 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f557c102f70 con 0x7f557c106aa0 2024-04-16T16:56:04.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:56:04.315+0000 7f5579600700 1 -- 172.21.15.45:0/3940584444 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f5570016ac0 con 0x7f557c106aa0 2024-04-16T16:56:04.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:56:04.315+0000 7f5582400700 1 -- 172.21.15.45:0/3940584444 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f557c1020e0 con 0x7f557c106aa0 2024-04-16T16:56:04.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:56:04.316+0000 7f5579600700 1 --2- 172.21.15.45:0/3940584444 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f55640396b0 0x7f556403bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:56:04.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:56:04.316+0000 7f5579600700 1 -- 172.21.15.45:0/3940584444 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f557001e070 con 0x7f557c106aa0 2024-04-16T16:56:04.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:56:04.316+0000 7f557be00700 1 --2- 172.21.15.45:0/3940584444 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f55640396b0 0x7f556403bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:56:04.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:56:04.317+0000 7f557be00700 1 --2- 172.21.15.45:0/3940584444 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f55640396b0 0x7f556403bb70 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f5568006fd0 tx=0x7f5568006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T16:56:04.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:56:04.321+0000 7f5579600700 1 -- 172.21.15.45:0/3940584444 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5570005660 con 0x7f557c106aa0 2024-04-16T16:56:04.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:56:04.564+0000 7f5582400700 1 -- 172.21.15.45:0/3940584444 --> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f557c1031a0 con 0x7f55640396b0 2024-04-16T16:56:04.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:56:04.566+0000 7f5579600700 1 -- 172.21.15.45:0/3940584444 <== mgr.14152 v2:172.21.15.45:6800/268030699 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+240 (secure 0 0 0) 0x7f557c1031a0 con 0x7f55640396b0 2024-04-16T16:56:04.567 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-16T16:56:04.567 INFO:teuthology.orchestra.run.smithi045.stdout:[{"addr": "172.21.15.45", "hostname": "smithi045", "labels": [], "status": ""}, {"addr": "172.21.15.104", "hostname": "smithi104", "labels": [], "status": ""}, {"addr": "172.21.15.117", "hostname": "smithi117", "labels": [], "status": ""}] 2024-04-16T16:56:04.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:56:04.568+0000 7f5582400700 1 -- 172.21.15.45:0/3940584444 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f55640396b0 msgr2=0x7f556403bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:56:04.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:56:04.568+0000 7f5582400700 1 --2- 172.21.15.45:0/3940584444 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f55640396b0 0x7f556403bb70 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f5568006fd0 tx=0x7f5568006e40 comp rx=0 tx=0).stop 2024-04-16T16:56:04.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:56:04.568+0000 7f5582400700 1 -- 172.21.15.45:0/3940584444 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f557c106aa0 msgr2=0x7f557c102570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:56:04.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:56:04.568+0000 7f5582400700 1 --2- 172.21.15.45:0/3940584444 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f557c106aa0 0x7f557c102570 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f5570004cd0 tx=0x7f5570004db0 comp rx=0 tx=0).stop 2024-04-16T16:56:04.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:56:04.569+0000 7f5582400700 1 -- 172.21.15.45:0/3940584444 shutdown_connections 2024-04-16T16:56:04.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:56:04.569+0000 7f5582400700 1 --2- 172.21.15.45:0/3940584444 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f55640396b0 0x7f556403bb70 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:56:04.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:56:04.569+0000 7f5582400700 1 --2- 172.21.15.45:0/3940584444 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f557c106aa0 0x7f557c102570 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:56:04.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:56:04.569+0000 7f5582400700 1 -- 172.21.15.45:0/3940584444 >> 172.21.15.45:0/3940584444 conn(0x7f557c0fc0a0 msgr2=0x7f557c108490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T16:56:04.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:56:04.569+0000 7f5582400700 1 -- 172.21.15.45:0/3940584444 shutdown_connections 2024-04-16T16:56:04.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:56:04.570+0000 7f5582400700 1 -- 172.21.15.45:0/3940584444 wait complete. 2024-04-16T16:56:04.931 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:56:04 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:56:05.361 INFO:tasks.cephadm:Setting crush tunables to default 2024-04-16T16:56:05.361 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b1fecd38-fc11-11ee-bc90-c7b262605968 -- ceph osd crush tunables default 2024-04-16T16:56:05.821 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/mon.a/config 2024-04-16T16:56:05.966 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:56:05 smithi045 ceph-mon[31676]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:56:05.966 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:56:05 smithi045 ceph-mon[31676]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-16T16:56:07.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:56:07.901+0000 7fc247200700 1 -- 172.21.15.45:0/698373734 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc24810a630 msgr2=0x7fc24810aa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:56:07.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:56:07.901+0000 7fc247200700 1 --2- 172.21.15.45:0/698373734 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc24810a630 0x7fc24810aa10 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fc238009910 tx=0x7fc238009c20 comp rx=0 tx=0).stop 2024-04-16T16:56:07.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:56:07.901+0000 7fc247200700 1 -- 172.21.15.45:0/698373734 shutdown_connections 2024-04-16T16:56:07.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:56:07.901+0000 7fc247200700 1 --2- 172.21.15.45:0/698373734 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc24810a630 0x7fc24810aa10 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:56:07.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:56:07.901+0000 7fc247200700 1 -- 172.21.15.45:0/698373734 >> 172.21.15.45:0/698373734 conn(0x7fc248100070 msgr2=0x7fc248102490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T16:56:07.903 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:56:07 smithi045 ceph-mon[31676]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:56:07.903 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:56:07 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:56:07.903 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:56:07 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:56:07.903 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:56:07 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:56:07.904 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:56:07 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:56:07.904 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:56:07 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi117", "name": "osd_memory_target"}]: dispatch 2024-04-16T16:56:07.904 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:56:07 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:56:07.904 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:56:07 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T16:56:07.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:56:07.901+0000 7fc247200700 1 -- 172.21.15.45:0/698373734 shutdown_connections 2024-04-16T16:56:07.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:56:07.902+0000 7fc247200700 1 -- 172.21.15.45:0/698373734 wait complete. 2024-04-16T16:56:07.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:56:07.903+0000 7fc247200700 1 Processor -- start 2024-04-16T16:56:07.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:56:07.903+0000 7fc247200700 1 -- start start 2024-04-16T16:56:07.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:56:07.904+0000 7fc247200700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc24810a630 0x7fc24819a030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:56:07.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:56:07.904+0000 7fc247200700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fc24810ebf0 con 0x7fc24810a630 2024-04-16T16:56:07.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:56:07.904+0000 7fc245e00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc24810a630 0x7fc24819a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:56:07.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:56:07.904+0000 7fc245e00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc24810a630 0x7fc24819a030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:44476/0 (socket says 172.21.15.45:44476) 2024-04-16T16:56:07.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:56:07.904+0000 7fc245e00700 1 -- 172.21.15.45:0/1006631336 learned_addr learned my addr 172.21.15.45:0/1006631336 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T16:56:07.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:56:07.905+0000 7fc245e00700 1 -- 172.21.15.45:0/1006631336 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc24819a570 con 0x7fc24810a630 2024-04-16T16:56:07.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:56:07.905+0000 7fc245e00700 1 --2- 172.21.15.45:0/1006631336 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc24810a630 0x7fc24819a030 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fc238006010 tx=0x7fc238005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T16:56:07.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:56:07.905+0000 7fc23ea00700 1 -- 172.21.15.45:0/1006631336 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc238026070 con 0x7fc24810a630 2024-04-16T16:56:07.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:56:07.905+0000 7fc23ea00700 1 -- 172.21.15.45:0/1006631336 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc23800bc30 con 0x7fc24810a630 2024-04-16T16:56:07.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:56:07.905+0000 7fc23ea00700 1 -- 172.21.15.45:0/1006631336 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc2380039d0 con 0x7fc24810a630 2024-04-16T16:56:07.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:56:07.905+0000 7fc247200700 1 -- 172.21.15.45:0/1006631336 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc24819a800 con 0x7fc24810a630 2024-04-16T16:56:07.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:56:07.905+0000 7fc247200700 1 -- 172.21.15.45:0/1006631336 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc24819e900 con 0x7fc24810a630 2024-04-16T16:56:07.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:56:07.907+0000 7fc23ea00700 1 -- 172.21.15.45:0/1006631336 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7fc238029070 con 0x7fc24810a630 2024-04-16T16:56:07.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:56:07.907+0000 7fc247200700 1 -- 172.21.15.45:0/1006631336 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc248105ca0 con 0x7fc24810a630 2024-04-16T16:56:07.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:56:07.907+0000 7fc23ea00700 1 --2- 172.21.15.45:0/1006631336 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fc22c039650 0x7fc22c03bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:56:07.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:56:07.907+0000 7fc23ea00700 1 -- 172.21.15.45:0/1006631336 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fc23801e070 con 0x7fc24810a630 2024-04-16T16:56:07.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:56:07.907+0000 7fc245400700 1 --2- 172.21.15.45:0/1006631336 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fc22c039650 0x7fc22c03bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:56:07.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:56:07.908+0000 7fc245400700 1 --2- 172.21.15.45:0/1006631336 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fc22c039650 0x7fc22c03bb10 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fc234006fd0 tx=0x7fc234006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T16:56:07.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:56:07.913+0000 7fc23ea00700 1 -- 172.21.15.45:0/1006631336 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc238013070 con 0x7fc24810a630 2024-04-16T16:56:08.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:56:08.154+0000 7fc247200700 1 -- 172.21.15.45:0/1006631336 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7fc24819b390 con 0x7fc24810a630 2024-04-16T16:56:08.978 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:56:08 smithi045 ceph-mon[31676]: Updating smithi117:/etc/ceph/ceph.conf 2024-04-16T16:56:08.978 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:56:08 smithi045 ceph-mon[31676]: Updating smithi117:/var/lib/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/config/ceph.conf 2024-04-16T16:56:08.979 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:56:08 smithi045 ceph-mon[31676]: Updating smithi117:/etc/ceph/ceph.client.admin.keyring 2024-04-16T16:56:08.979 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:56:08 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:56:08.979 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:56:08 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:56:08.979 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:56:08 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:56:08.979 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:56:08 smithi045 ceph-mon[31676]: from='client.? 172.21.15.45:0/1006631336' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-04-16T16:56:09.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:56:09.086+0000 7fc23ea00700 1 -- 172.21.15.45:0/1006631336 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) ==== 124+0+0 (secure 0 0 0) 0x7fc23800de00 con 0x7fc24810a630 2024-04-16T16:56:09.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:56:09.089+0000 7fc247200700 1 -- 172.21.15.45:0/1006631336 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fc22c039650 msgr2=0x7fc22c03bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:56:09.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:56:09.089+0000 7fc247200700 1 --2- 172.21.15.45:0/1006631336 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fc22c039650 0x7fc22c03bb10 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fc234006fd0 tx=0x7fc234006e40 comp rx=0 tx=0).stop 2024-04-16T16:56:09.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:56:09.090+0000 7fc247200700 1 -- 172.21.15.45:0/1006631336 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc24810a630 msgr2=0x7fc24819a030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:56:09.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:56:09.090+0000 7fc247200700 1 --2- 172.21.15.45:0/1006631336 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc24810a630 0x7fc24819a030 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fc238006010 tx=0x7fc238005030 comp rx=0 tx=0).stop 2024-04-16T16:56:09.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:56:09.090+0000 7fc247200700 1 -- 172.21.15.45:0/1006631336 shutdown_connections 2024-04-16T16:56:09.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:56:09.090+0000 7fc247200700 1 --2- 172.21.15.45:0/1006631336 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fc22c039650 0x7fc22c03bb10 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:56:09.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:56:09.090+0000 7fc247200700 1 --2- 172.21.15.45:0/1006631336 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc24810a630 0x7fc24819a030 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:56:09.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:56:09.090+0000 7fc247200700 1 -- 172.21.15.45:0/1006631336 >> 172.21.15.45:0/1006631336 conn(0x7fc248100070 msgr2=0x7fc2481018c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T16:56:09.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:56:09.090+0000 7fc247200700 1 -- 172.21.15.45:0/1006631336 shutdown_connections 2024-04-16T16:56:09.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:56:09.090+0000 7fc247200700 1 -- 172.21.15.45:0/1006631336 wait complete. 2024-04-16T16:56:09.093 INFO:teuthology.orchestra.run.smithi045.stderr:adjusted tunables profile to default 2024-04-16T16:56:09.690 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:56:09 smithi045 ceph-mon[31676]: Updating smithi117:/var/lib/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/config/ceph.client.admin.keyring 2024-04-16T16:56:09.690 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:56:09 smithi045 ceph-mon[31676]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:56:09.690 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:56:09 smithi045 ceph-mon[31676]: from='client.? 172.21.15.45:0/1006631336' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-04-16T16:56:09.690 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:56:09 smithi045 ceph-mon[31676]: osdmap e4: 0 total, 0 up, 0 in 2024-04-16T16:56:09.769 INFO:tasks.cephadm:Adding mon.a on smithi045 2024-04-16T16:56:09.769 INFO:tasks.cephadm:Adding mon.b on smithi104 2024-04-16T16:56:09.769 INFO:tasks.cephadm:Adding mon.c on smithi117 2024-04-16T16:56:09.769 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b1fecd38-fc11-11ee-bc90-c7b262605968 -- ceph orch apply mon '3;smithi045:172.21.15.45=a;smithi104:172.21.15.104=b;smithi117:172.21.15.117=c' 2024-04-16T16:56:10.008 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/config/ceph.conf 2024-04-16T16:56:11.523 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:56:11.521+0000 7f24accbc700 1 -- 172.21.15.117:0/2638021746 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f24a81074c0 msgr2=0x7f24a81078a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:56:11.523 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:56:11.521+0000 7f24accbc700 1 --2- 172.21.15.117:0/2638021746 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f24a81074c0 0x7f24a81078a0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f2498009910 tx=0x7f2498009c20 comp rx=0 tx=0).stop 2024-04-16T16:56:11.523 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:56:11.521+0000 7f24accbc700 1 -- 172.21.15.117:0/2638021746 shutdown_connections 2024-04-16T16:56:11.523 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:56:11.521+0000 7f24accbc700 1 --2- 172.21.15.117:0/2638021746 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f24a81074c0 0x7f24a81078a0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:56:11.523 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:56:11.521+0000 7f24accbc700 1 -- 172.21.15.117:0/2638021746 >> 172.21.15.117:0/2638021746 conn(0x7f24a8078a20 msgr2=0x7f24a8078e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T16:56:11.523 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:56:11.521+0000 7f24accbc700 1 -- 172.21.15.117:0/2638021746 shutdown_connections 2024-04-16T16:56:11.524 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:56:11.521+0000 7f24accbc700 1 -- 172.21.15.117:0/2638021746 wait complete. 2024-04-16T16:56:11.524 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:56:11.522+0000 7f24accbc700 1 Processor -- start 2024-04-16T16:56:11.524 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:56:11.522+0000 7f24accbc700 1 -- start start 2024-04-16T16:56:11.524 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:56:11.522+0000 7f24accbc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f24a81074c0 0x7f24a8197f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:56:11.524 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:56:11.522+0000 7f24accbc700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f24a810ba00 con 0x7f24a81074c0 2024-04-16T16:56:11.525 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:56:11.522+0000 7f24a659c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f24a81074c0 0x7f24a8197f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:56:11.525 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:56:11.522+0000 7f24a659c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f24a81074c0 0x7f24a8197f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.117:55826/0 (socket says 172.21.15.117:55826) 2024-04-16T16:56:11.525 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:56:11.522+0000 7f24a659c700 1 -- 172.21.15.117:0/2264729498 learned_addr learned my addr 172.21.15.117:0/2264729498 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-16T16:56:11.525 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:56:11.523+0000 7f24a659c700 1 -- 172.21.15.117:0/2264729498 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24a81984c0 con 0x7f24a81074c0 2024-04-16T16:56:11.525 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:56:11.523+0000 7f24a659c700 1 --2- 172.21.15.117:0/2264729498 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f24a81074c0 0x7f24a8197f80 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f24980092e0 tx=0x7f2498005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T16:56:11.525 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:56:11.524+0000 7f24977fe700 1 -- 172.21.15.117:0/2264729498 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2498026070 con 0x7f24a81074c0 2024-04-16T16:56:11.525 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:56:11.524+0000 7f24accbc700 1 -- 172.21.15.117:0/2264729498 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24a81987b0 con 0x7f24a81074c0 2024-04-16T16:56:11.525 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:56:11.524+0000 7f24accbc700 1 -- 172.21.15.117:0/2264729498 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24a819c800 con 0x7f24a81074c0 2024-04-16T16:56:11.526 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:56:11.524+0000 7f24977fe700 1 -- 172.21.15.117:0/2264729498 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2498004740 con 0x7f24a81074c0 2024-04-16T16:56:11.526 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:56:11.524+0000 7f24977fe700 1 -- 172.21.15.117:0/2264729498 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2498030060 con 0x7f24a81074c0 2024-04-16T16:56:11.526 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:56:11.524+0000 7f24accbc700 1 -- 172.21.15.117:0/2264729498 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2488005330 con 0x7f24a81074c0 2024-04-16T16:56:11.526 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:56:11.525+0000 7f24977fe700 1 -- 172.21.15.117:0/2264729498 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f2498029070 con 0x7f24a81074c0 2024-04-16T16:56:11.527 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:56:11.525+0000 7f24977fe700 1 --2- 172.21.15.117:0/2264729498 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f2490039650 0x7f249003bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:56:11.527 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:56:11.525+0000 7f24977fe700 1 -- 172.21.15.117:0/2264729498 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f249801e070 con 0x7f24a81074c0 2024-04-16T16:56:11.527 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:56:11.526+0000 7f24a5d9b700 1 --2- 172.21.15.117:0/2264729498 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f2490039650 0x7f249003bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:56:11.528 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:56:11.527+0000 7f24a5d9b700 1 --2- 172.21.15.117:0/2264729498 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f2490039650 0x7f249003bb10 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f249c006fd0 tx=0x7f249c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T16:56:11.542 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:56:11.541+0000 7f24977fe700 1 -- 172.21.15.117:0/2264729498 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2498013070 con 0x7f24a81074c0 2024-04-16T16:56:11.750 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:56:11.749+0000 7f24accbc700 1 -- 172.21.15.117:0/2264729498 --> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi045:172.21.15.45=a;smithi104:172.21.15.104=b;smithi117:172.21.15.117=c", "target": ["mon-mgr", ""]}) -- 0x7f2488000c80 con 0x7f2490039650 2024-04-16T16:56:11.765 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:56:11.764+0000 7f24977fe700 1 -- 172.21.15.117:0/2264729498 <== mgr.14152 v2:172.21.15.45:6800/268030699 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f2488000c80 con 0x7f2490039650 2024-04-16T16:56:11.765 INFO:teuthology.orchestra.run.smithi117.stdout:Scheduled mon update... 2024-04-16T16:56:11.767 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:56:11.766+0000 7f24accbc700 1 -- 172.21.15.117:0/2264729498 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f2490039650 msgr2=0x7f249003bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:56:11.768 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:56:11.766+0000 7f24accbc700 1 --2- 172.21.15.117:0/2264729498 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f2490039650 0x7f249003bb10 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f249c006fd0 tx=0x7f249c006e40 comp rx=0 tx=0).stop 2024-04-16T16:56:11.768 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:56:11.766+0000 7f24accbc700 1 -- 172.21.15.117:0/2264729498 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f24a81074c0 msgr2=0x7f24a8197f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:56:11.768 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:56:11.766+0000 7f24accbc700 1 --2- 172.21.15.117:0/2264729498 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f24a81074c0 0x7f24a8197f80 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f24980092e0 tx=0x7f2498005030 comp rx=0 tx=0).stop 2024-04-16T16:56:11.768 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:56:11.766+0000 7f24accbc700 1 -- 172.21.15.117:0/2264729498 shutdown_connections 2024-04-16T16:56:11.768 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:56:11.766+0000 7f24accbc700 1 --2- 172.21.15.117:0/2264729498 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f2490039650 0x7f249003bb10 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:56:11.768 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:56:11.766+0000 7f24accbc700 1 --2- 172.21.15.117:0/2264729498 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f24a81074c0 0x7f24a8197f80 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:56:11.768 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:56:11.766+0000 7f24accbc700 1 -- 172.21.15.117:0/2264729498 >> 172.21.15.117:0/2264729498 conn(0x7f24a8078a20 msgr2=0x7f24a8100190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T16:56:11.768 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:56:11.766+0000 7f24accbc700 1 -- 172.21.15.117:0/2264729498 shutdown_connections 2024-04-16T16:56:11.768 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:56:11.766+0000 7f24accbc700 1 -- 172.21.15.117:0/2264729498 wait complete. 2024-04-16T16:56:11.978 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:56:11 smithi045 ceph-mon[31676]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:56:12.401 DEBUG:teuthology.orchestra.run.smithi104:mon.b> sudo journalctl -f -n 0 -u ceph-b1fecd38-fc11-11ee-bc90-c7b262605968@mon.b.service 2024-04-16T16:56:12.404 DEBUG:teuthology.orchestra.run.smithi117:mon.c> sudo journalctl -f -n 0 -u ceph-b1fecd38-fc11-11ee-bc90-c7b262605968@mon.c.service 2024-04-16T16:56:12.406 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-16T16:56:12.406 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b1fecd38-fc11-11ee-bc90-c7b262605968 -- ceph mon dump -f json 2024-04-16T16:56:13.228 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:56:12 smithi045 ceph-mon[31676]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi045:172.21.15.45=a;smithi104:172.21.15.104=b;smithi117:172.21.15.117=c", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T16:56:13.228 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:56:12 smithi045 ceph-mon[31676]: Saving service mon spec with placement smithi045:172.21.15.45=a;smithi104:172.21.15.104=b;smithi117:172.21.15.117=c;count:3 2024-04-16T16:56:13.229 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:56:12 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:56:13.229 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:56:12 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T16:56:13.229 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:56:12 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:56:13.229 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:56:12 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T16:56:13.229 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:56:12 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:56:13.229 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:56:12 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-16T16:56:13.229 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:56:12 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:56:13.290 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/config/ceph.conf 2024-04-16T16:56:14.228 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:56:13 smithi045 ceph-mon[31676]: Deploying daemon mon.c on smithi117 2024-04-16T16:56:14.228 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:56:13 smithi045 ceph-mon[31676]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:56:15.743 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:56:15.740+0000 7fd570967700 1 -- 172.21.15.117:0/605910940 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd56c108580 msgr2=0x7fd56c108960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:56:15.743 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:56:15.740+0000 7fd570967700 1 --2- 172.21.15.117:0/605910940 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd56c108580 0x7fd56c108960 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fd55c009910 tx=0x7fd55c009c20 comp rx=0 tx=0).stop 2024-04-16T16:56:15.744 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:56:15.741+0000 7fd570967700 1 -- 172.21.15.117:0/605910940 shutdown_connections 2024-04-16T16:56:15.744 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:56:15.741+0000 7fd570967700 1 --2- 172.21.15.117:0/605910940 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd56c108580 0x7fd56c108960 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:56:15.744 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:56:15.741+0000 7fd570967700 1 -- 172.21.15.117:0/605910940 >> 172.21.15.117:0/605910940 conn(0x7fd56c076f50 msgr2=0x7fd56c077360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T16:56:15.744 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:56:15.741+0000 7fd570967700 1 -- 172.21.15.117:0/605910940 shutdown_connections 2024-04-16T16:56:15.744 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:56:15.741+0000 7fd570967700 1 -- 172.21.15.117:0/605910940 wait complete. 2024-04-16T16:56:15.744 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:56:15.741+0000 7fd570967700 1 Processor -- start 2024-04-16T16:56:15.744 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:56:15.741+0000 7fd570967700 1 -- start start 2024-04-16T16:56:15.744 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:56:15.741+0000 7fd570967700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd56c108580 0x7fd56c101510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:56:15.744 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:56:15.741+0000 7fd570967700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fd56c10cac0 con 0x7fd56c108580 2024-04-16T16:56:15.744 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:56:15.742+0000 7fd56a59c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd56c108580 0x7fd56c101510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:56:15.744 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:56:15.742+0000 7fd56a59c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd56c108580 0x7fd56c101510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.117:55834/0 (socket says 172.21.15.117:55834) 2024-04-16T16:56:15.744 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:56:15.742+0000 7fd56a59c700 1 -- 172.21.15.117:0/1087836869 learned_addr learned my addr 172.21.15.117:0/1087836869 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-16T16:56:15.745 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:56:15.743+0000 7fd56a59c700 1 -- 172.21.15.117:0/1087836869 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd56c103400 con 0x7fd56c108580 2024-04-16T16:56:15.746 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:56:15.743+0000 7fd56a59c700 1 --2- 172.21.15.117:0/1087836869 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd56c108580 0x7fd56c101510 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fd55c005850 tx=0x7fd55c005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T16:56:15.746 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:56:15.743+0000 7fd55b7fe700 1 -- 172.21.15.117:0/1087836869 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd55c026070 con 0x7fd56c108580 2024-04-16T16:56:15.746 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:56:15.743+0000 7fd570967700 1 -- 172.21.15.117:0/1087836869 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd56c101ab0 con 0x7fd56c108580 2024-04-16T16:56:15.746 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:56:15.743+0000 7fd570967700 1 -- 172.21.15.117:0/1087836869 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd56c101f70 con 0x7fd56c108580 2024-04-16T16:56:15.746 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:56:15.743+0000 7fd55b7fe700 1 -- 172.21.15.117:0/1087836869 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd55c00b750 con 0x7fd56c108580 2024-04-16T16:56:15.746 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:56:15.743+0000 7fd55b7fe700 1 -- 172.21.15.117:0/1087836869 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd55c030060 con 0x7fd56c108580 2024-04-16T16:56:15.746 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:56:15.744+0000 7fd570967700 1 -- 172.21.15.117:0/1087836869 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd54c005330 con 0x7fd56c108580 2024-04-16T16:56:15.747 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:56:15.745+0000 7fd55b7fe700 1 -- 172.21.15.117:0/1087836869 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7fd55c029070 con 0x7fd56c108580 2024-04-16T16:56:15.747 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:56:15.745+0000 7fd55b7fe700 1 --2- 172.21.15.117:0/1087836869 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fd554039600 0x7fd55403bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:56:15.747 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:56:15.745+0000 7fd55b7fe700 1 -- 172.21.15.117:0/1087836869 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fd55c01e070 con 0x7fd56c108580 2024-04-16T16:56:15.747 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:56:15.746+0000 7fd569d9b700 1 --2- 172.21.15.117:0/1087836869 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fd554039600 0x7fd55403bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:56:15.748 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:56:15.747+0000 7fd569d9b700 1 --2- 172.21.15.117:0/1087836869 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fd554039600 0x7fd55403bac0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fd560006fd0 tx=0x7fd560006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T16:56:15.762 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:56:15.761+0000 7fd55b7fe700 1 -- 172.21.15.117:0/1087836869 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd55c013070 con 0x7fd56c108580 2024-04-16T16:56:16.067 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:56:16.066+0000 7fd570967700 1 -- 172.21.15.117:0/1087836869 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fd54c0058f0 con 0x7fd56c108580 2024-04-16T16:56:16.068 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:56:16.067+0000 7fd55b7fe700 1 -- 172.21.15.117:0/1087836869 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+745 (secure 0 0 0) 0x7fd55c00e840 con 0x7fd56c108580 2024-04-16T16:56:16.068 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-16T16:56:16.068 INFO:teuthology.orchestra.run.smithi117.stdout:{"epoch":1,"fsid":"b1fecd38-fc11-11ee-bc90-c7b262605968","modified":"2024-04-16T16:53:58.140859Z","created":"2024-04-16T16:53:58.140859Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:3300","nonce":0},{"type":"v1","addr":"172.21.15.45:6789","nonce":0}]},"addr":"172.21.15.45:6789/0","public_addr":"172.21.15.45:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-16T16:56:16.070 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:56:16.069+0000 7fd570967700 1 -- 172.21.15.117:0/1087836869 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fd554039600 msgr2=0x7fd55403bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:56:16.070 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:56:16.069+0000 7fd570967700 1 --2- 172.21.15.117:0/1087836869 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fd554039600 0x7fd55403bac0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fd560006fd0 tx=0x7fd560006e40 comp rx=0 tx=0).stop 2024-04-16T16:56:16.070 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:56:16.069+0000 7fd570967700 1 -- 172.21.15.117:0/1087836869 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd56c108580 msgr2=0x7fd56c101510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:56:16.070 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:56:16.069+0000 7fd570967700 1 --2- 172.21.15.117:0/1087836869 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd56c108580 0x7fd56c101510 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fd55c005850 tx=0x7fd55c005030 comp rx=0 tx=0).stop 2024-04-16T16:56:16.071 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:56:16.069+0000 7fd570967700 1 -- 172.21.15.117:0/1087836869 shutdown_connections 2024-04-16T16:56:16.071 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:56:16.069+0000 7fd570967700 1 --2- 172.21.15.117:0/1087836869 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fd554039600 0x7fd55403bac0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:56:16.071 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:56:16.069+0000 7fd570967700 1 --2- 172.21.15.117:0/1087836869 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd56c108580 0x7fd56c101510 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:56:16.071 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:56:16.069+0000 7fd570967700 1 -- 172.21.15.117:0/1087836869 >> 172.21.15.117:0/1087836869 conn(0x7fd56c076f50 msgr2=0x7fd56c0fe980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T16:56:16.071 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:56:16.069+0000 7fd570967700 1 -- 172.21.15.117:0/1087836869 shutdown_connections 2024-04-16T16:56:16.071 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:56:16.069+0000 7fd570967700 1 -- 172.21.15.117:0/1087836869 wait complete. 2024-04-16T16:56:16.071 INFO:teuthology.orchestra.run.smithi117.stderr:dumped monmap epoch 1 2024-04-16T16:56:16.228 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:56:15 smithi045 ceph-mon[31676]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:56:16.306 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:16 smithi117 systemd[1]: Starting Ceph mon.c for b1fecd38-fc11-11ee-bc90-c7b262605968... 2024-04-16T16:56:17.102 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:16 smithi117 podman[36711]: 2024-04-16 16:56:16.833200204 +0000 UTC m=+0.106236306 container create a26181e132ad7b008b206cb47b80c6b217ca6d390204bc4a88c6fd7eb7a28317 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mon-c, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240409, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-16T16:56:17.102 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:16 smithi117 podman[36711]: 2024-04-16 16:56:16.74468112 +0000 UTC m=+0.017717221 image pull e5ce10bdffbeeffffd1125f67b369e2151f0f17cdec854e93b6adf9321f65907 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2 2024-04-16T16:56:17.228 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:56:16 smithi045 ceph-mon[31676]: from='client.? 172.21.15.117:0/1087836869' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-16T16:56:17.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 podman[36711]: 2024-04-16 16:56:17.193708704 +0000 UTC m=+0.466744798 container init a26181e132ad7b008b206cb47b80c6b217ca6d390204bc4a88c6fd7eb7a28317 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mon-c, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, org.label-schema.build-date=20240409, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS) 2024-04-16T16:56:17.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 podman[36711]: 2024-04-16 16:56:17.196651662 +0000 UTC m=+0.469687742 container start a26181e132ad7b008b206cb47b80c6b217ca6d390204bc4a88c6fd7eb7a28317 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mon-c, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240409) 2024-04-16T16:56:17.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: set uid:gid to 167:167 (ceph:ceph) 2024-04-16T16:56:17.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: ceph version 19.0.0-3101-g9b42bfbe (9b42bfbeda298874861ece590aa565cf64ac8c45) squid (dev), process ceph-mon, pid 2 2024-04-16T16:56:17.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: pidfile_write: ignore empty --pid-file 2024-04-16T16:56:17.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: load: jerasure load: lrc 2024-04-16T16:56:17.557 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: rocksdb: RocksDB version: 7.9.2 2024-04-16T16:56:17.557 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: rocksdb: Git sha 0 2024-04-16T16:56:17.557 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: rocksdb: Compile date 2024-04-16 06:46:19 2024-04-16T16:56:17.557 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: rocksdb: DB SUMMARY 2024-04-16T16:56:17.557 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: rocksdb: DB Session ID: UEXL7SUV6JAM0GRUNF3U 2024-04-16T16:56:17.557 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: rocksdb: CURRENT file: CURRENT 2024-04-16T16:56:17.557 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: rocksdb: IDENTITY file: IDENTITY 2024-04-16T16:56:17.557 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-04-16T16:56:17.557 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2024-04-16T16:56:17.557 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 511 ; 2024-04-16T16:56:17.557 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: rocksdb: Options.error_if_exists: 0 2024-04-16T16:56:17.557 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: rocksdb: Options.create_if_missing: 0 2024-04-16T16:56:17.557 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: rocksdb: Options.paranoid_checks: 1 2024-04-16T16:56:17.557 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: rocksdb: Options.flush_verify_memtable_count: 1 2024-04-16T16:56:17.557 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-04-16T16:56:17.558 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-04-16T16:56:17.558 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: rocksdb: Options.env: 0x563dac98acc0 2024-04-16T16:56:17.558 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: rocksdb: Options.fs: PosixFileSystem 2024-04-16T16:56:17.558 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: rocksdb: Options.info_log: 0x563dae087c20 2024-04-16T16:56:17.559 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: rocksdb: Options.max_file_opening_threads: 16 2024-04-16T16:56:17.560 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: rocksdb: Options.statistics: (nil) 2024-04-16T16:56:17.560 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: rocksdb: Options.use_fsync: 0 2024-04-16T16:56:17.560 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: rocksdb: Options.max_log_file_size: 0 2024-04-16T16:56:17.560 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-04-16T16:56:17.560 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: rocksdb: Options.log_file_time_to_roll: 0 2024-04-16T16:56:17.560 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: rocksdb: Options.keep_log_file_num: 1000 2024-04-16T16:56:17.560 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: rocksdb: Options.recycle_log_file_num: 0 2024-04-16T16:56:17.560 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: rocksdb: Options.allow_fallocate: 1 2024-04-16T16:56:17.560 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: rocksdb: Options.allow_mmap_reads: 0 2024-04-16T16:56:17.560 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: rocksdb: Options.allow_mmap_writes: 0 2024-04-16T16:56:17.560 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: rocksdb: Options.use_direct_reads: 0 2024-04-16T16:56:17.560 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-04-16T16:56:17.561 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: rocksdb: Options.create_missing_column_families: 0 2024-04-16T16:56:17.561 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: rocksdb: Options.db_log_dir: 2024-04-16T16:56:17.561 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: rocksdb: Options.wal_dir: 2024-04-16T16:56:17.561 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: rocksdb: Options.table_cache_numshardbits: 6 2024-04-16T16:56:17.561 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: rocksdb: Options.WAL_ttl_seconds: 0 2024-04-16T16:56:17.561 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: rocksdb: Options.WAL_size_limit_MB: 0 2024-04-16T16:56:17.561 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-04-16T16:56:17.561 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-04-16T16:56:17.561 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: rocksdb: Options.is_fd_close_on_exec: 1 2024-04-16T16:56:17.561 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: rocksdb: Options.advise_random_on_open: 1 2024-04-16T16:56:17.561 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: rocksdb: Options.db_write_buffer_size: 0 2024-04-16T16:56:17.561 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: rocksdb: Options.write_buffer_manager: 0x563dae0561e0 2024-04-16T16:56:17.561 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-04-16T16:56:17.561 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-04-16T16:56:17.561 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: rocksdb: Options.use_adaptive_mutex: 0 2024-04-16T16:56:17.562 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: rocksdb: Options.rate_limiter: (nil) 2024-04-16T16:56:17.562 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-04-16T16:56:17.562 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: rocksdb: Options.wal_recovery_mode: 2 2024-04-16T16:56:17.562 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: rocksdb: Options.enable_thread_tracking: 0 2024-04-16T16:56:17.562 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: rocksdb: Options.enable_pipelined_write: 0 2024-04-16T16:56:17.562 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: rocksdb: Options.unordered_write: 0 2024-04-16T16:56:17.562 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-04-16T16:56:17.562 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-04-16T16:56:17.562 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-04-16T16:56:17.562 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-04-16T16:56:17.562 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: rocksdb: Options.row_cache: None 2024-04-16T16:56:17.562 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: rocksdb: Options.wal_filter: None 2024-04-16T16:56:17.562 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-04-16T16:56:17.563 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: rocksdb: Options.allow_ingest_behind: 0 2024-04-16T16:56:17.563 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: rocksdb: Options.two_write_queues: 0 2024-04-16T16:56:17.563 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: rocksdb: Options.manual_wal_flush: 0 2024-04-16T16:56:17.563 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: rocksdb: Options.wal_compression: 0 2024-04-16T16:56:17.563 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: rocksdb: Options.atomic_flush: 0 2024-04-16T16:56:17.563 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-04-16T16:56:17.563 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: rocksdb: Options.persist_stats_to_disk: 0 2024-04-16T16:56:17.563 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: rocksdb: Options.write_dbid_to_manifest: 0 2024-04-16T16:56:17.563 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: rocksdb: Options.log_readahead_size: 0 2024-04-16T16:56:17.563 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-04-16T16:56:17.563 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: rocksdb: Options.best_efforts_recovery: 0 2024-04-16T16:56:17.563 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-04-16T16:56:17.563 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-04-16T16:56:17.563 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: rocksdb: Options.allow_data_in_errors: 0 2024-04-16T16:56:17.563 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: rocksdb: Options.db_host_id: __hostname__ 2024-04-16T16:56:17.564 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: rocksdb: Options.enforce_single_del_contracts: true 2024-04-16T16:56:17.564 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: rocksdb: Options.max_background_jobs: 2 2024-04-16T16:56:17.564 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: rocksdb: Options.max_background_compactions: -1 2024-04-16T16:56:17.564 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: rocksdb: Options.max_subcompactions: 1 2024-04-16T16:56:17.564 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-04-16T16:56:17.564 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-04-16T16:56:17.564 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: rocksdb: Options.delayed_write_rate : 16777216 2024-04-16T16:56:17.564 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: rocksdb: Options.max_total_wal_size: 0 2024-04-16T16:56:17.564 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-04-16T16:56:17.564 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: rocksdb: Options.stats_dump_period_sec: 600 2024-04-16T16:56:17.564 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: rocksdb: Options.stats_persist_period_sec: 600 2024-04-16T16:56:17.564 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-04-16T16:56:17.564 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: rocksdb: Options.max_open_files: -1 2024-04-16T16:56:17.564 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: rocksdb: Options.bytes_per_sync: 0 2024-04-16T16:56:17.564 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: rocksdb: Options.wal_bytes_per_sync: 0 2024-04-16T16:56:17.565 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: rocksdb: Options.strict_bytes_per_sync: 0 2024-04-16T16:56:17.565 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: rocksdb: Options.compaction_readahead_size: 0 2024-04-16T16:56:17.565 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: rocksdb: Options.max_background_flushes: -1 2024-04-16T16:56:17.565 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: rocksdb: Compression algorithms supported: 2024-04-16T16:56:17.565 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-04-16T16:56:17.565 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: rocksdb: kZSTD supported: 0 2024-04-16T16:56:17.565 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: rocksdb: kXpressCompression supported: 0 2024-04-16T16:56:17.565 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: rocksdb: kLZ4HCCompression supported: 1 2024-04-16T16:56:17.565 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: rocksdb: kZlibCompression supported: 1 2024-04-16T16:56:17.565 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: rocksdb: kSnappyCompression supported: 1 2024-04-16T16:56:17.565 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: rocksdb: kLZ4Compression supported: 1 2024-04-16T16:56:17.565 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: rocksdb: kBZip2Compression supported: 0 2024-04-16T16:56:17.565 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: rocksdb: Fast CRC32 supported: Supported on x86 2024-04-16T16:56:17.565 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: rocksdb: DMutex implementation: pthread_mutex_t 2024-04-16T16:56:17.565 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 2024-04-16T16:56:17.566 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-04-16T16:56:17.566 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-04-16T16:56:17.566 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: rocksdb: Options.merge_operator: 2024-04-16T16:56:17.566 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: rocksdb: Options.compaction_filter: None 2024-04-16T16:56:17.566 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: rocksdb: Options.compaction_filter_factory: None 2024-04-16T16:56:17.566 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: rocksdb: Options.sst_partitioner_factory: None 2024-04-16T16:56:17.566 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: rocksdb: Options.memtable_factory: SkipListFactory 2024-04-16T16:56:17.566 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: rocksdb: Options.table_factory: BlockBasedTable 2024-04-16T16:56:17.566 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x563dae087de0) 2024-04-16T16:56:17.566 INFO:journalctl@ceph.mon.c.smithi117.stdout: cache_index_and_filter_blocks: 1 2024-04-16T16:56:17.566 INFO:journalctl@ceph.mon.c.smithi117.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-04-16T16:56:17.566 INFO:journalctl@ceph.mon.c.smithi117.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-04-16T16:56:17.566 INFO:journalctl@ceph.mon.c.smithi117.stdout: pin_top_level_index_and_filter: 1 2024-04-16T16:56:17.566 INFO:journalctl@ceph.mon.c.smithi117.stdout: index_type: 0 2024-04-16T16:56:17.566 INFO:journalctl@ceph.mon.c.smithi117.stdout: data_block_index_type: 0 2024-04-16T16:56:17.567 INFO:journalctl@ceph.mon.c.smithi117.stdout: index_shortening: 1 2024-04-16T16:56:17.567 INFO:journalctl@ceph.mon.c.smithi117.stdout: data_block_hash_table_util_ratio: 0.750000 2024-04-16T16:56:17.567 INFO:journalctl@ceph.mon.c.smithi117.stdout: checksum: 4 2024-04-16T16:56:17.567 INFO:journalctl@ceph.mon.c.smithi117.stdout: no_block_cache: 0 2024-04-16T16:56:17.567 INFO:journalctl@ceph.mon.c.smithi117.stdout: block_cache: 0x563dae089090 2024-04-16T16:56:17.567 INFO:journalctl@ceph.mon.c.smithi117.stdout: block_cache_name: BinnedLRUCache 2024-04-16T16:56:17.567 INFO:journalctl@ceph.mon.c.smithi117.stdout: block_cache_options: 2024-04-16T16:56:17.567 INFO:journalctl@ceph.mon.c.smithi117.stdout: capacity : 536870912 2024-04-16T16:56:17.567 INFO:journalctl@ceph.mon.c.smithi117.stdout: num_shard_bits : 4 2024-04-16T16:56:17.567 INFO:journalctl@ceph.mon.c.smithi117.stdout: strict_capacity_limit : 0 2024-04-16T16:56:17.567 INFO:journalctl@ceph.mon.c.smithi117.stdout: high_pri_pool_ratio: 0.000 2024-04-16T16:56:17.567 INFO:journalctl@ceph.mon.c.smithi117.stdout: block_cache_compressed: (nil) 2024-04-16T16:56:17.567 INFO:journalctl@ceph.mon.c.smithi117.stdout: persistent_cache: (nil) 2024-04-16T16:56:17.567 INFO:journalctl@ceph.mon.c.smithi117.stdout: block_size: 4096 2024-04-16T16:56:17.567 INFO:journalctl@ceph.mon.c.smithi117.stdout: block_size_deviation: 10 2024-04-16T16:56:17.568 INFO:journalctl@ceph.mon.c.smithi117.stdout: block_restart_interval: 16 2024-04-16T16:56:17.568 INFO:journalctl@ceph.mon.c.smithi117.stdout: index_block_restart_interval: 1 2024-04-16T16:56:17.568 INFO:journalctl@ceph.mon.c.smithi117.stdout: metadata_block_size: 4096 2024-04-16T16:56:17.568 INFO:journalctl@ceph.mon.c.smithi117.stdout: partition_filters: 0 2024-04-16T16:56:17.568 INFO:journalctl@ceph.mon.c.smithi117.stdout: use_delta_encoding: 1 2024-04-16T16:56:17.568 INFO:journalctl@ceph.mon.c.smithi117.stdout: filter_policy: bloomfilter 2024-04-16T16:56:17.568 INFO:journalctl@ceph.mon.c.smithi117.stdout: whole_key_filtering: 1 2024-04-16T16:56:17.568 INFO:journalctl@ceph.mon.c.smithi117.stdout: verify_compression: 0 2024-04-16T16:56:17.568 INFO:journalctl@ceph.mon.c.smithi117.stdout: read_amp_bytes_per_bit: 0 2024-04-16T16:56:17.568 INFO:journalctl@ceph.mon.c.smithi117.stdout: format_version: 5 2024-04-16T16:56:17.568 INFO:journalctl@ceph.mon.c.smithi117.stdout: enable_index_compression: 1 2024-04-16T16:56:17.568 INFO:journalctl@ceph.mon.c.smithi117.stdout: block_align: 0 2024-04-16T16:56:17.568 INFO:journalctl@ceph.mon.c.smithi117.stdout: max_auto_readahead_size: 262144 2024-04-16T16:56:17.568 INFO:journalctl@ceph.mon.c.smithi117.stdout: prepopulate_block_cache: 0 2024-04-16T16:56:17.568 INFO:journalctl@ceph.mon.c.smithi117.stdout: initial_auto_readahead_size: 8192 2024-04-16T16:56:17.568 INFO:journalctl@ceph.mon.c.smithi117.stdout: num_file_reads_for_auto_readahead: 2 2024-04-16T16:56:17.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: rocksdb: Options.write_buffer_size: 33554432 2024-04-16T16:56:17.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: rocksdb: Options.max_write_buffer_number: 2 2024-04-16T16:56:17.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: rocksdb: Options.compression: NoCompression 2024-04-16T16:56:17.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: rocksdb: Options.bottommost_compression: Disabled 2024-04-16T16:56:17.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: rocksdb: Options.prefix_extractor: nullptr 2024-04-16T16:56:17.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-04-16T16:56:17.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: rocksdb: Options.num_levels: 7 2024-04-16T16:56:17.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-04-16T16:56:17.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-04-16T16:56:17.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-04-16T16:56:17.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-04-16T16:56:17.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-04-16T16:56:17.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-04-16T16:56:17.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-04-16T16:56:17.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-04-16T16:56:17.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-04-16T16:56:17.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-04-16T16:56:17.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-04-16T16:56:17.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-04-16T16:56:17.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: rocksdb: Options.compression_opts.window_bits: -14 2024-04-16T16:56:17.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: rocksdb: Options.compression_opts.level: 32767 2024-04-16T16:56:17.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: rocksdb: Options.compression_opts.strategy: 0 2024-04-16T16:56:17.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-04-16T16:56:17.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-04-16T16:56:17.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-04-16T16:56:17.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-04-16T16:56:17.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: rocksdb: Options.compression_opts.enabled: false 2024-04-16T16:56:17.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-04-16T16:56:17.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-04-16T16:56:17.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-04-16T16:56:17.571 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-04-16T16:56:17.571 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: rocksdb: Options.target_file_size_base: 67108864 2024-04-16T16:56:17.571 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: rocksdb: Options.target_file_size_multiplier: 1 2024-04-16T16:56:17.571 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-04-16T16:56:17.571 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-04-16T16:56:17.571 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-04-16T16:56:17.571 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-04-16T16:56:17.571 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-04-16T16:56:17.571 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-04-16T16:56:17.571 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-04-16T16:56:17.571 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-04-16T16:56:17.571 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-04-16T16:56:17.571 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-04-16T16:56:17.571 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-04-16T16:56:17.571 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-04-16T16:56:17.572 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-04-16T16:56:17.572 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: rocksdb: Options.arena_block_size: 1048576 2024-04-16T16:56:17.572 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-04-16T16:56:17.572 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-04-16T16:56:17.572 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: rocksdb: Options.disable_auto_compactions: 0 2024-04-16T16:56:17.572 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-04-16T16:56:17.572 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-04-16T16:56:17.572 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-04-16T16:56:17.572 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-04-16T16:56:17.572 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-04-16T16:56:17.572 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-04-16T16:56:17.572 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-04-16T16:56:17.572 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-04-16T16:56:17.572 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-04-16T16:56:17.573 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-04-16T16:56:17.573 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-04-16T16:56:17.573 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: rocksdb: Options.inplace_update_support: 0 2024-04-16T16:56:17.573 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: rocksdb: Options.inplace_update_num_locks: 10000 2024-04-16T16:56:17.573 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-04-16T16:56:17.573 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-04-16T16:56:17.573 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: rocksdb: Options.memtable_huge_page_size: 0 2024-04-16T16:56:17.573 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: rocksdb: Options.bloom_locality: 0 2024-04-16T16:56:17.573 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: rocksdb: Options.max_successive_merges: 0 2024-04-16T16:56:17.573 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: rocksdb: Options.optimize_filters_for_hits: 0 2024-04-16T16:56:17.573 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: rocksdb: Options.paranoid_file_checks: 0 2024-04-16T16:56:17.573 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: rocksdb: Options.force_consistency_checks: 1 2024-04-16T16:56:17.573 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: rocksdb: Options.report_bg_io_stats: 0 2024-04-16T16:56:17.573 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: rocksdb: Options.ttl: 2592000 2024-04-16T16:56:17.573 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: rocksdb: Options.periodic_compaction_seconds: 0 2024-04-16T16:56:17.573 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-04-16T16:56:17.574 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-04-16T16:56:17.574 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: rocksdb: Options.enable_blob_files: false 2024-04-16T16:56:17.574 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: rocksdb: Options.min_blob_size: 0 2024-04-16T16:56:17.574 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: rocksdb: Options.blob_file_size: 268435456 2024-04-16T16:56:17.574 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: rocksdb: Options.blob_compression_type: NoCompression 2024-04-16T16:56:17.574 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: rocksdb: Options.enable_blob_garbage_collection: false 2024-04-16T16:56:17.574 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-04-16T16:56:17.574 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-04-16T16:56:17.574 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-04-16T16:56:17.574 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: rocksdb: Options.blob_file_starting_level: 0 2024-04-16T16:56:17.574 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-04-16T16:56:17.574 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-04-16T16:56:17.574 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-04-16T16:56:17.575 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: ff2a3ca1-e3ae-40e9-a8ff-091439340eef 2024-04-16T16:56:17.575 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713286577226082, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-04-16T16:56:17.575 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-04-16T16:56:17.575 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713286577226491, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1648, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1713286577, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "ff2a3ca1-e3ae-40e9-a8ff-091439340eef", "db_session_id": "UEXL7SUV6JAM0GRUNF3U", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-04-16T16:56:17.575 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713286577226568, "job": 1, "event": "recovery_finished"} 2024-04-16T16:56:17.575 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-04-16T16:56:17.575 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-04-16T16:56:17.575 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x563dae176000 2024-04-16T16:56:17.575 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: rocksdb: DB pointer 0x563dae160000 2024-04-16T16:56:17.575 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-04-16T16:56:17.575 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-04-16T16:56:17.575 INFO:journalctl@ceph.mon.c.smithi117.stdout: ** DB Stats ** 2024-04-16T16:56:17.575 INFO:journalctl@ceph.mon.c.smithi117.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-16T16:56:17.575 INFO:journalctl@ceph.mon.c.smithi117.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-04-16T16:56:17.575 INFO:journalctl@ceph.mon.c.smithi117.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-16T16:56:17.576 INFO:journalctl@ceph.mon.c.smithi117.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-16T16:56:17.576 INFO:journalctl@ceph.mon.c.smithi117.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-04-16T16:56:17.576 INFO:journalctl@ceph.mon.c.smithi117.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-16T16:56:17.576 INFO:journalctl@ceph.mon.c.smithi117.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-16T16:56:17.576 INFO:journalctl@ceph.mon.c.smithi117.stdout: 2024-04-16T16:56:17.576 INFO:journalctl@ceph.mon.c.smithi117.stdout: ** Compaction Stats [default] ** 2024-04-16T16:56:17.576 INFO:journalctl@ceph.mon.c.smithi117.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-16T16:56:17.576 INFO:journalctl@ceph.mon.c.smithi117.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-04-16T16:56:17.576 INFO:journalctl@ceph.mon.c.smithi117.stdout: L0 1/0 1.61 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.0 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-04-16T16:56:17.576 INFO:journalctl@ceph.mon.c.smithi117.stdout: Sum 1/0 1.61 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.0 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-04-16T16:56:17.576 INFO:journalctl@ceph.mon.c.smithi117.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.0 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-04-16T16:56:17.576 INFO:journalctl@ceph.mon.c.smithi117.stdout: 2024-04-16T16:56:17.576 INFO:journalctl@ceph.mon.c.smithi117.stdout: ** Compaction Stats [default] ** 2024-04-16T16:56:17.576 INFO:journalctl@ceph.mon.c.smithi117.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-16T16:56:17.576 INFO:journalctl@ceph.mon.c.smithi117.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-16T16:56:17.577 INFO:journalctl@ceph.mon.c.smithi117.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 4.0 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-04-16T16:56:17.577 INFO:journalctl@ceph.mon.c.smithi117.stdout: 2024-04-16T16:56:17.577 INFO:journalctl@ceph.mon.c.smithi117.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-04-16T16:56:17.577 INFO:journalctl@ceph.mon.c.smithi117.stdout: 2024-04-16T16:56:17.577 INFO:journalctl@ceph.mon.c.smithi117.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-16T16:56:17.577 INFO:journalctl@ceph.mon.c.smithi117.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-04-16T16:56:17.577 INFO:journalctl@ceph.mon.c.smithi117.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-04-16T16:56:17.577 INFO:journalctl@ceph.mon.c.smithi117.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-04-16T16:56:17.577 INFO:journalctl@ceph.mon.c.smithi117.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-04-16T16:56:17.577 INFO:journalctl@ceph.mon.c.smithi117.stdout: AddFile(Keys): cumulative 0, interval 0 2024-04-16T16:56:17.577 INFO:journalctl@ceph.mon.c.smithi117.stdout: Cumulative compaction: 0.00 GB write, 0.67 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-16T16:56:17.577 INFO:journalctl@ceph.mon.c.smithi117.stdout: Interval compaction: 0.00 GB write, 0.67 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-16T16:56:17.577 INFO:journalctl@ceph.mon.c.smithi117.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-04-16T16:56:17.577 INFO:journalctl@ceph.mon.c.smithi117.stdout: Block cache BinnedLRUCache@0x563dae089090#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 8e-06 secs_since: 0 2024-04-16T16:56:17.578 INFO:journalctl@ceph.mon.c.smithi117.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-04-16T16:56:17.578 INFO:journalctl@ceph.mon.c.smithi117.stdout: 2024-04-16T16:56:17.578 INFO:journalctl@ceph.mon.c.smithi117.stdout: ** File Read Latency Histogram By Level [default] ** 2024-04-16T16:56:17.578 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: mon.c does not exist in monmap, will attempt to join an existing cluster 2024-04-16T16:56:17.578 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: using public_addr v2:172.21.15.117:0/0 -> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] 2024-04-16T16:56:17.578 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: starting mon.c rank -1 at public addrs [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] at bind addrs [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] mon_data /var/lib/ceph/mon/ceph-c fsid b1fecd38-fc11-11ee-bc90-c7b262605968 2024-04-16T16:56:17.578 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: mon.c@-1(???) e0 preinit fsid b1fecd38-fc11-11ee-bc90-c7b262605968 2024-04-16T16:56:17.578 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: mon.c@-1(synchronizing).mds e1 new map 2024-04-16T16:56:17.578 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: mon.c@-1(synchronizing).mds e1 print_map 2024-04-16T16:56:17.578 INFO:journalctl@ceph.mon.c.smithi117.stdout: e1 2024-04-16T16:56:17.578 INFO:journalctl@ceph.mon.c.smithi117.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-04-16T16:56:17.578 INFO:journalctl@ceph.mon.c.smithi117.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-16T16:56:17.578 INFO:journalctl@ceph.mon.c.smithi117.stdout: legacy client fscid: -1 2024-04-16T16:56:17.578 INFO:journalctl@ceph.mon.c.smithi117.stdout: 2024-04-16T16:56:17.579 INFO:journalctl@ceph.mon.c.smithi117.stdout: No filesystems configured 2024-04-16T16:56:17.579 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-04-16T16:56:17.579 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-04-16T16:56:17.579 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-04-16T16:56:17.579 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-04-16T16:56:17.579 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-04-16T16:56:17.579 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-04-16T16:56:17.579 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-04-16T16:56:17.579 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-16T16:56:17.579 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-16T16:56:17.579 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-16T16:56:17.579 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:56:17.579 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:56:17.579 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T16:56:17.579 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: Added host smithi104 2024-04-16T16:56:17.580 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:56:17.580 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:56:17.580 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:56:17.580 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:56:17.580 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:56:17.580 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-16T16:56:17.580 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:56:17.580 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:56:17.580 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:56:17.580 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:56:17.580 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:56:17.580 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi104", "name": "osd_memory_target"}]: dispatch 2024-04-16T16:56:17.580 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:56:17.580 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T16:56:17.580 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: Updating smithi104:/etc/ceph/ceph.conf 2024-04-16T16:56:17.581 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: Updating smithi104:/var/lib/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/config/ceph.conf 2024-04-16T16:56:17.581 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: Updating smithi104:/etc/ceph/ceph.client.admin.keyring 2024-04-16T16:56:17.581 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: Updating smithi104:/var/lib/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/config/ceph.client.admin.keyring 2024-04-16T16:56:17.581 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:56:17.581 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi117", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T16:56:17.581 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:56:17.581 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:56:17.581 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:56:17.581 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: Deploying cephadm binary to smithi117 2024-04-16T16:56:17.581 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:56:17.581 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:56:17.581 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:56:17.581 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T16:56:17.581 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: Added host smithi117 2024-04-16T16:56:17.581 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:56:17.582 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:56:17.582 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:56:17.582 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:56:17.582 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:56:17.582 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-16T16:56:17.582 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:56:17.582 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:56:17.582 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:56:17.582 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:56:17.582 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:56:17.582 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi117", "name": "osd_memory_target"}]: dispatch 2024-04-16T16:56:17.582 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:56:17.582 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T16:56:17.582 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: Updating smithi117:/etc/ceph/ceph.conf 2024-04-16T16:56:17.583 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: Updating smithi117:/var/lib/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/config/ceph.conf 2024-04-16T16:56:17.583 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: Updating smithi117:/etc/ceph/ceph.client.admin.keyring 2024-04-16T16:56:17.583 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:56:17.583 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:56:17.583 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:56:17.583 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: from='client.? 172.21.15.45:0/1006631336' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-04-16T16:56:17.583 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: Updating smithi117:/var/lib/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/config/ceph.client.admin.keyring 2024-04-16T16:56:17.583 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:56:17.583 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: from='client.? 172.21.15.45:0/1006631336' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-04-16T16:56:17.583 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: osdmap e4: 0 total, 0 up, 0 in 2024-04-16T16:56:17.583 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:56:17.583 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi045:172.21.15.45=a;smithi104:172.21.15.104=b;smithi117:172.21.15.117=c", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T16:56:17.583 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: Saving service mon spec with placement smithi045:172.21.15.45=a;smithi104:172.21.15.104=b;smithi117:172.21.15.117=c;count:3 2024-04-16T16:56:17.583 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:56:17.583 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T16:56:17.584 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:56:17.584 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T16:56:17.584 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:56:17.584 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-16T16:56:17.584 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:56:17.584 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: Deploying daemon mon.c on smithi117 2024-04-16T16:56:17.584 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:56:17.584 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:56:17.584 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: from='client.? 172.21.15.117:0/1087836869' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-16T16:56:17.584 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 ceph-mon[36730]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-04-16T16:56:17.584 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 bash[36711]: a26181e132ad7b008b206cb47b80c6b217ca6d390204bc4a88c6fd7eb7a28317 2024-04-16T16:56:17.584 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:17 smithi117 systemd[1]: Started Ceph mon.c for b1fecd38-fc11-11ee-bc90-c7b262605968. 2024-04-16T16:56:17.644 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-16T16:56:17.644 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b1fecd38-fc11-11ee-bc90-c7b262605968 -- ceph mon dump -f json 2024-04-16T16:56:17.877 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/mon.c/config 2024-04-16T16:56:22.363 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:56:22.361+0000 7fa0e8e45700 1 -- 172.21.15.117:0/3780625527 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7fa0e0017670 con 0x7fa0e41037a0 2024-04-16T16:56:22.364 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:56:22.362+0000 7fa0ec0ab700 1 -- 172.21.15.117:0/3780625527 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa0e41037a0 msgr2=0x7fa0d0005610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:56:22.364 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:56:22.362+0000 7fa0ec0ab700 1 --2- 172.21.15.117:0/3780625527 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa0e41037a0 0x7fa0d0005610 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fa0e0009bf0 tx=0x7fa0e0009f00 comp rx=0 tx=0).stop 2024-04-16T16:56:22.364 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:56:22.362+0000 7fa0ec0ab700 1 -- 172.21.15.117:0/3780625527 shutdown_connections 2024-04-16T16:56:22.364 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:56:22.362+0000 7fa0ec0ab700 1 --2- 172.21.15.117:0/3780625527 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa0e41037a0 0x7fa0d0005610 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:56:22.364 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:56:22.362+0000 7fa0ec0ab700 1 -- 172.21.15.117:0/3780625527 >> 172.21.15.117:0/3780625527 conn(0x7fa0e40fcfd0 msgr2=0x7fa0e40ff3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T16:56:22.364 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:56:22.362+0000 7fa0ec0ab700 1 -- 172.21.15.117:0/3780625527 shutdown_connections 2024-04-16T16:56:22.364 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:56:22.362+0000 7fa0ec0ab700 1 -- 172.21.15.117:0/3780625527 wait complete. 2024-04-16T16:56:22.365 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:56:22.363+0000 7fa0ec0ab700 1 Processor -- start 2024-04-16T16:56:22.365 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:56:22.363+0000 7fa0ec0ab700 1 -- start start 2024-04-16T16:56:22.366 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:56:22.363+0000 7fa0ec0ab700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa0e41037a0 0x7fa0e41980d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:56:22.366 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:56:22.363+0000 7fa0ec0ab700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa0e4198610 0x7fa0e419c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:56:22.366 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:56:22.363+0000 7fa0ec0ab700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fa0e410a680 con 0x7fa0e41037a0 2024-04-16T16:56:22.366 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:56:22.363+0000 7fa0ec0ab700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa0e410a500 con 0x7fa0e4198610 2024-04-16T16:56:22.366 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:56:22.364+0000 7fa0e9646700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa0e4198610 0x7fa0e419c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:56:22.366 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:56:22.364+0000 7fa0e9646700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa0e4198610 0x7fa0e419c9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:51628/0 (socket says 172.21.15.117:51628) 2024-04-16T16:56:22.366 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:56:22.364+0000 7fa0e9646700 1 -- 172.21.15.117:0/773820889 learned_addr learned my addr 172.21.15.117:0/773820889 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-16T16:56:22.366 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:56:22.364+0000 7fa0e9646700 1 -- 172.21.15.117:0/773820889 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa0e4198610 msgr2=0x7fa0e419c9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-04-16T16:56:22.366 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:56:22.364+0000 7fa0e9646700 1 -- 172.21.15.117:0/773820889 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa0e4198610 msgr2=0x7fa0e419c9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-16T16:56:22.366 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:56:22.364+0000 7fa0e9646700 1 --2- 172.21.15.117:0/773820889 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa0e4198610 0x7fa0e419c9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-16T16:56:22.366 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:56:22.364+0000 7fa0e9646700 1 --2- 172.21.15.117:0/773820889 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa0e4198610 0x7fa0e419c9a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-16T16:56:22.366 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:56:22.364+0000 7fa0e9e47700 1 --2- 172.21.15.117:0/773820889 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa0e41037a0 0x7fa0e41980d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:56:22.367 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:56:22.365+0000 7fa0e9e47700 1 -- 172.21.15.117:0/773820889 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa0e4198610 msgr2=0x7fa0e419c9a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-16T16:56:22.367 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:56:22.365+0000 7fa0e9e47700 1 --2- 172.21.15.117:0/773820889 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa0e4198610 0x7fa0e419c9a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:56:22.367 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:56:22.365+0000 7fa0e9e47700 1 -- 172.21.15.117:0/773820889 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0e4198c50 con 0x7fa0e41037a0 2024-04-16T16:56:22.367 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:56:22.365+0000 7fa0e9e47700 1 --2- 172.21.15.117:0/773820889 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa0e41037a0 0x7fa0e41980d0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fa0e0007950 tx=0x7fa0e0003e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T16:56:22.368 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:56:22.367+0000 7fa0daffd700 1 -- 172.21.15.117:0/773820889 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7fa0e0004c40 con 0x7fa0e41037a0 2024-04-16T16:56:22.369 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:56:22.367+0000 7fa0ec0ab700 1 -- 172.21.15.117:0/773820889 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0e419cee0 con 0x7fa0e41037a0 2024-04-16T16:56:22.369 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:56:22.367+0000 7fa0daffd700 1 -- 172.21.15.117:0/773820889 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa0e0004de0 con 0x7fa0e41037a0 2024-04-16T16:56:22.369 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:56:22.367+0000 7fa0daffd700 1 -- 172.21.15.117:0/773820889 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7fa0e0005210 con 0x7fa0e41037a0 2024-04-16T16:56:22.369 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:56:22.367+0000 7fa0ec0ab700 1 -- 172.21.15.117:0/773820889 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0e419d440 con 0x7fa0e41037a0 2024-04-16T16:56:22.370 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:56:22.368+0000 7fa0ec0ab700 1 -- 172.21.15.117:0/773820889 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0e406ad50 con 0x7fa0e41037a0 2024-04-16T16:56:22.370 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:56:22.369+0000 7fa0daffd700 1 -- 172.21.15.117:0/773820889 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7fa0e001e070 con 0x7fa0e41037a0 2024-04-16T16:56:22.370 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:56:22.369+0000 7fa0daffd700 1 --2- 172.21.15.117:0/773820889 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fa0c80393a0 0x7fa0c803b860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:56:22.371 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:56:22.369+0000 7fa0daffd700 1 -- 172.21.15.117:0/773820889 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fa0e001c070 con 0x7fa0e41037a0 2024-04-16T16:56:22.371 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:56:22.370+0000 7fa0e9646700 1 --2- 172.21.15.117:0/773820889 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fa0c80393a0 0x7fa0c803b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:56:22.372 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:56:22.371+0000 7fa0e9646700 1 --2- 172.21.15.117:0/773820889 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fa0c80393a0 0x7fa0c803b860 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fa0d4009180 tx=0x7fa0d4008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T16:56:22.386 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:56:22.385+0000 7fa0daffd700 1 -- 172.21.15.117:0/773820889 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa0e001edc0 con 0x7fa0e41037a0 2024-04-16T16:56:22.636 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:56:22 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-16T16:56:22.636 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:56:22 smithi045 ceph-mon[31676]: mon.a calling monitor election 2024-04-16T16:56:22.636 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:56:22 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-16T16:56:22.636 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:56:22 smithi045 ceph-mon[31676]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:56:22.637 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:56:22 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-16T16:56:22.637 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:56:22 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-16T16:56:22.637 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:56:22 smithi045 ceph-mon[31676]: mon.c calling monitor election 2024-04-16T16:56:22.637 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:56:22 smithi045 ceph-mon[31676]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:56:22.637 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:56:22 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-16T16:56:22.637 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:56:22 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-16T16:56:22.637 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:56:22 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-16T16:56:22.637 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:56:22 smithi045 ceph-mon[31676]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-16T16:56:22.637 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:56:22 smithi045 ceph-mon[31676]: monmap epoch 2 2024-04-16T16:56:22.637 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:56:22 smithi045 ceph-mon[31676]: fsid b1fecd38-fc11-11ee-bc90-c7b262605968 2024-04-16T16:56:22.637 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:56:22 smithi045 ceph-mon[31676]: last_changed 2024-04-16T16:56:17.259442+0000 2024-04-16T16:56:22.637 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:56:22 smithi045 ceph-mon[31676]: created 2024-04-16T16:53:58.140859+0000 2024-04-16T16:56:22.637 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:56:22 smithi045 ceph-mon[31676]: min_mon_release 19 (squid) 2024-04-16T16:56:22.637 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:56:22 smithi045 ceph-mon[31676]: election_strategy: 1 2024-04-16T16:56:22.637 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:56:22 smithi045 ceph-mon[31676]: 0: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.a 2024-04-16T16:56:22.637 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:56:22 smithi045 ceph-mon[31676]: 1: [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] mon.c 2024-04-16T16:56:22.638 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:56:22 smithi045 ceph-mon[31676]: fsmap 2024-04-16T16:56:22.638 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:56:22 smithi045 ceph-mon[31676]: osdmap e4: 0 total, 0 up, 0 in 2024-04-16T16:56:22.638 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:56:22 smithi045 ceph-mon[31676]: mgrmap e13: x(active, since 62s) 2024-04-16T16:56:22.638 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:56:22 smithi045 ceph-mon[31676]: overall HEALTH_OK 2024-04-16T16:56:22.638 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:56:22 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:56:22.638 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:56:22 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:56:22.638 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:56:22 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:56:22.723 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:56:22.722+0000 7fa0ec0ab700 1 -- 172.21.15.117:0/773820889 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fa0e41994c0 con 0x7fa0e41037a0 2024-04-16T16:56:22.724 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:22 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-16T16:56:22.724 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:22 smithi117 ceph-mon[36730]: mon.a calling monitor election 2024-04-16T16:56:22.724 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:22 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-16T16:56:22.724 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:22 smithi117 ceph-mon[36730]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:56:22.724 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:22 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-16T16:56:22.724 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:22 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-16T16:56:22.724 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:22 smithi117 ceph-mon[36730]: mon.c calling monitor election 2024-04-16T16:56:22.724 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:22 smithi117 ceph-mon[36730]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:56:22.724 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:22 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-16T16:56:22.724 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:22 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-16T16:56:22.724 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:22 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-16T16:56:22.724 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:22 smithi117 ceph-mon[36730]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-16T16:56:22.724 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:22 smithi117 ceph-mon[36730]: monmap epoch 2 2024-04-16T16:56:22.724 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:22 smithi117 ceph-mon[36730]: fsid b1fecd38-fc11-11ee-bc90-c7b262605968 2024-04-16T16:56:22.725 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:22 smithi117 ceph-mon[36730]: last_changed 2024-04-16T16:56:17.259442+0000 2024-04-16T16:56:22.725 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:22 smithi117 ceph-mon[36730]: created 2024-04-16T16:53:58.140859+0000 2024-04-16T16:56:22.725 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:22 smithi117 ceph-mon[36730]: min_mon_release 19 (squid) 2024-04-16T16:56:22.725 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:22 smithi117 ceph-mon[36730]: election_strategy: 1 2024-04-16T16:56:22.725 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:22 smithi117 ceph-mon[36730]: 0: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.a 2024-04-16T16:56:22.725 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:22 smithi117 ceph-mon[36730]: 1: [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] mon.c 2024-04-16T16:56:22.725 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:22 smithi117 ceph-mon[36730]: fsmap 2024-04-16T16:56:22.725 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:22 smithi117 ceph-mon[36730]: osdmap e4: 0 total, 0 up, 0 in 2024-04-16T16:56:22.725 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:22 smithi117 ceph-mon[36730]: mgrmap e13: x(active, since 62s) 2024-04-16T16:56:22.725 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:22 smithi117 ceph-mon[36730]: overall HEALTH_OK 2024-04-16T16:56:22.725 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:22 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:56:22.725 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:22 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:56:22.725 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:22 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:56:22.726 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:56:22.723+0000 7fa0daffd700 1 -- 172.21.15.117:0/773820889 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1015 (secure 0 0 0) 0x7fa0e41994c0 con 0x7fa0e41037a0 2024-04-16T16:56:22.726 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-16T16:56:22.726 INFO:teuthology.orchestra.run.smithi117.stdout:{"epoch":2,"fsid":"b1fecd38-fc11-11ee-bc90-c7b262605968","modified":"2024-04-16T16:56:17.259442Z","created":"2024-04-16T16:53:58.140859Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:3300","nonce":0},{"type":"v1","addr":"172.21.15.45:6789","nonce":0}]},"addr":"172.21.15.45:6789/0","public_addr":"172.21.15.45:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:3300","nonce":0},{"type":"v1","addr":"172.21.15.117:6789","nonce":0}]},"addr":"172.21.15.117:6789/0","public_addr":"172.21.15.117:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-04-16T16:56:22.727 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:56:22.726+0000 7fa0ec0ab700 1 -- 172.21.15.117:0/773820889 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fa0c80393a0 msgr2=0x7fa0c803b860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:56:22.727 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:56:22.726+0000 7fa0ec0ab700 1 --2- 172.21.15.117:0/773820889 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fa0c80393a0 0x7fa0c803b860 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fa0d4009180 tx=0x7fa0d4008040 comp rx=0 tx=0).stop 2024-04-16T16:56:22.727 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:56:22.726+0000 7fa0ec0ab700 1 -- 172.21.15.117:0/773820889 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa0e41037a0 msgr2=0x7fa0e41980d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:56:22.727 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:56:22.726+0000 7fa0ec0ab700 1 --2- 172.21.15.117:0/773820889 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa0e41037a0 0x7fa0e41980d0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fa0e0007950 tx=0x7fa0e0003e00 comp rx=0 tx=0).stop 2024-04-16T16:56:22.727 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:56:22.726+0000 7fa0ec0ab700 1 -- 172.21.15.117:0/773820889 shutdown_connections 2024-04-16T16:56:22.727 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:56:22.726+0000 7fa0ec0ab700 1 --2- 172.21.15.117:0/773820889 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa0e41037a0 0x7fa0e41980d0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:56:22.727 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:56:22.726+0000 7fa0ec0ab700 1 --2- 172.21.15.117:0/773820889 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fa0c80393a0 0x7fa0c803b860 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:56:22.727 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:56:22.726+0000 7fa0ec0ab700 1 --2- 172.21.15.117:0/773820889 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa0e4198610 0x7fa0e419c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:56:22.728 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:56:22.726+0000 7fa0ec0ab700 1 -- 172.21.15.117:0/773820889 >> 172.21.15.117:0/773820889 conn(0x7fa0e40fcfd0 msgr2=0x7fa0e40fed10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T16:56:22.728 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:56:22.726+0000 7fa0ec0ab700 1 -- 172.21.15.117:0/773820889 shutdown_connections 2024-04-16T16:56:22.728 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:56:22.726+0000 7fa0ec0ab700 1 -- 172.21.15.117:0/773820889 wait complete. 2024-04-16T16:56:22.728 INFO:teuthology.orchestra.run.smithi117.stderr:dumped monmap epoch 2 2024-04-16T16:56:23.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:23 smithi117 ceph-mon[36730]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:56:23.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:23 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-16T16:56:23.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:23 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:56:23.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:23 smithi117 ceph-mon[36730]: Deploying daemon mon.b on smithi104 2024-04-16T16:56:23.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:23 smithi117 ceph-mon[36730]: from='client.? 172.21.15.117:0/773820889' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-16T16:56:23.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:23 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-16T16:56:23.661 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:56:23 smithi045 ceph-mon[31676]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:56:23.661 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:56:23 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-16T16:56:23.661 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:56:23 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:56:23.661 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:56:23 smithi045 ceph-mon[31676]: Deploying daemon mon.b on smithi104 2024-04-16T16:56:23.661 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:56:23 smithi045 ceph-mon[31676]: from='client.? 172.21.15.117:0/773820889' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-16T16:56:23.661 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:56:23 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-16T16:56:24.324 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-16T16:56:24.325 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b1fecd38-fc11-11ee-bc90-c7b262605968 -- ceph mon dump -f json 2024-04-16T16:56:24.365 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 16 16:56:24 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-x[31905]: 2024-04-16T16:56:24.259+0000 7f1a8d400700 -1 mgr.server handle_report got status from non-daemon mon.c 2024-04-16T16:56:24.567 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/mon.c/config 2024-04-16T16:56:25.577 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:25 smithi117 ceph-mon[36730]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:56:25.729 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:56:25 smithi045 ceph-mon[31676]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:56:26.057 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:56:26.056+0000 7f3c1d630700 1 -- 172.21.15.117:0/160368941 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3c181086a0 msgr2=0x7f3c18108a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:56:26.058 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:56:26.056+0000 7f3c1d630700 1 --2- 172.21.15.117:0/160368941 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3c181086a0 0x7f3c18108a80 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f3c0c009910 tx=0x7f3c0c009c20 comp rx=0 tx=0).stop 2024-04-16T16:56:26.058 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:56:26.056+0000 7f3c1d630700 1 -- 172.21.15.117:0/160368941 shutdown_connections 2024-04-16T16:56:26.058 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:56:26.056+0000 7f3c1d630700 1 --2- 172.21.15.117:0/160368941 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3c181086a0 0x7f3c18108a80 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:56:26.058 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:56:26.056+0000 7f3c1d630700 1 -- 172.21.15.117:0/160368941 >> 172.21.15.117:0/160368941 conn(0x7f3c180fe100 msgr2=0x7f3c18100520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T16:56:26.058 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:56:26.056+0000 7f3c1d630700 1 -- 172.21.15.117:0/160368941 shutdown_connections 2024-04-16T16:56:26.058 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:56:26.056+0000 7f3c1d630700 1 -- 172.21.15.117:0/160368941 wait complete. 2024-04-16T16:56:26.058 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:56:26.056+0000 7f3c1d630700 1 Processor -- start 2024-04-16T16:56:26.058 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:56:26.057+0000 7f3c1d630700 1 -- start start 2024-04-16T16:56:26.058 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:56:26.057+0000 7f3c1d630700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3c181086a0 0x7f3c18198250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:56:26.058 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:56:26.057+0000 7f3c1d630700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3c18198790 0x7f3c1819cb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:56:26.058 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:56:26.057+0000 7f3c1d630700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f3c1810cc50 con 0x7f3c181086a0 2024-04-16T16:56:26.058 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:56:26.057+0000 7f3c1d630700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f3c1810cad0 con 0x7f3c18198790 2024-04-16T16:56:26.059 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:56:26.057+0000 7f3c167fc700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3c18198790 0x7f3c1819cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:56:26.059 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:56:26.057+0000 7f3c167fc700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3c18198790 0x7f3c1819cb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:51632/0 (socket says 172.21.15.117:51632) 2024-04-16T16:56:26.059 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:56:26.057+0000 7f3c167fc700 1 -- 172.21.15.117:0/1671446774 learned_addr learned my addr 172.21.15.117:0/1671446774 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-16T16:56:26.059 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:56:26.057+0000 7f3c167fc700 1 -- 172.21.15.117:0/1671446774 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3c18198790 msgr2=0x7f3c1819cb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-04-16T16:56:26.059 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:56:26.057+0000 7f3c167fc700 1 -- 172.21.15.117:0/1671446774 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3c18198790 msgr2=0x7f3c1819cb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-16T16:56:26.059 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:56:26.057+0000 7f3c167fc700 1 --2- 172.21.15.117:0/1671446774 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3c18198790 0x7f3c1819cb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-16T16:56:26.059 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:56:26.057+0000 7f3c167fc700 1 --2- 172.21.15.117:0/1671446774 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3c18198790 0x7f3c1819cb20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-16T16:56:26.059 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:56:26.057+0000 7f3c16ffd700 1 --2- 172.21.15.117:0/1671446774 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3c181086a0 0x7f3c18198250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:56:26.060 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:56:26.058+0000 7f3c16ffd700 1 -- 172.21.15.117:0/1671446774 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3c18198790 msgr2=0x7f3c1819cb20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-16T16:56:26.061 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:56:26.058+0000 7f3c16ffd700 1 --2- 172.21.15.117:0/1671446774 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3c18198790 0x7f3c1819cb20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:56:26.061 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:56:26.058+0000 7f3c16ffd700 1 -- 172.21.15.117:0/1671446774 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c18198dd0 con 0x7f3c181086a0 2024-04-16T16:56:26.061 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:56:26.058+0000 7f3c16ffd700 1 --2- 172.21.15.117:0/1671446774 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3c181086a0 0x7f3c18198250 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f3c18103750 tx=0x7f3c0c005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T16:56:26.061 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:56:26.059+0000 7f3bfffff700 1 -- 172.21.15.117:0/1671446774 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f3c0c005dc0 con 0x7f3c181086a0 2024-04-16T16:56:26.061 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:56:26.059+0000 7f3c1d630700 1 -- 172.21.15.117:0/1671446774 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c1819d060 con 0x7f3c181086a0 2024-04-16T16:56:26.061 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:56:26.059+0000 7f3c1d630700 1 -- 172.21.15.117:0/1671446774 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c1819d640 con 0x7f3c181086a0 2024-04-16T16:56:26.061 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:56:26.059+0000 7f3bfffff700 1 -- 172.21.15.117:0/1671446774 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3c0c00b6f0 con 0x7f3c181086a0 2024-04-16T16:56:26.061 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:56:26.059+0000 7f3bfffff700 1 -- 172.21.15.117:0/1671446774 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f3c0c003840 con 0x7f3c181086a0 2024-04-16T16:56:26.061 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:56:26.059+0000 7f3c1d630700 1 -- 172.21.15.117:0/1671446774 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c04005330 con 0x7f3c181086a0 2024-04-16T16:56:26.061 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:56:26.060+0000 7f3bfffff700 1 -- 172.21.15.117:0/1671446774 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f3c0c029070 con 0x7f3c181086a0 2024-04-16T16:56:26.061 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:56:26.060+0000 7f3bfffff700 1 --2- 172.21.15.117:0/1671446774 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f3c000393f0 0x7f3c0003b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:56:26.061 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:56:26.060+0000 7f3bfffff700 1 -- 172.21.15.117:0/1671446774 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f3c0c01e070 con 0x7f3c181086a0 2024-04-16T16:56:26.062 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:56:26.061+0000 7f3c167fc700 1 --2- 172.21.15.117:0/1671446774 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f3c000393f0 0x7f3c0003b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:56:26.062 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:56:26.062+0000 7f3c167fc700 1 --2- 172.21.15.117:0/1671446774 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f3c000393f0 0x7f3c0003b8b0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f3c08009180 tx=0x7f3c08008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T16:56:26.076 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:56:26.076+0000 7f3bfffff700 1 -- 172.21.15.117:0/1671446774 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3c0c013070 con 0x7f3c181086a0 2024-04-16T16:56:26.155 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 podman[36437]: 2024-04-16 16:56:26.131153909 +0000 UTC m=+0.344823445 container start 68a342e84dea98df618d136c2aca253c177fb630c8b2f18d159f2106c3f1c4f7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mon-b, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240409, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-16T16:56:26.190 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:56:26.190+0000 7f3bfffff700 1 -- 172.21.15.117:0/1671446774 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c0c00e3e0 con 0x7f3c181086a0 2024-04-16T16:56:26.356 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:56:26.356+0000 7f3c1d630700 1 -- 172.21.15.117:0/1671446774 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f3c040058f0 con 0x7f3c181086a0 2024-04-16T16:56:26.429 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: set uid:gid to 167:167 (ceph:ceph) 2024-04-16T16:56:26.429 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: ceph version 19.0.0-3101-g9b42bfbe (9b42bfbeda298874861ece590aa565cf64ac8c45) squid (dev), process ceph-mon, pid 2 2024-04-16T16:56:26.429 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: pidfile_write: ignore empty --pid-file 2024-04-16T16:56:26.429 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: load: jerasure load: lrc 2024-04-16T16:56:26.429 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: rocksdb: RocksDB version: 7.9.2 2024-04-16T16:56:26.429 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: rocksdb: Git sha 0 2024-04-16T16:56:26.429 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: rocksdb: Compile date 2024-04-16 06:46:19 2024-04-16T16:56:26.429 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: rocksdb: DB SUMMARY 2024-04-16T16:56:26.430 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: rocksdb: DB Session ID: 7ZZC0KSLS1MQVRCUWQ2I 2024-04-16T16:56:26.430 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: rocksdb: CURRENT file: CURRENT 2024-04-16T16:56:26.430 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: rocksdb: IDENTITY file: IDENTITY 2024-04-16T16:56:26.430 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-04-16T16:56:26.430 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: rocksdb: SST files in /var/lib/ceph/mon/ceph-b/store.db dir, Total Num: 0, files: 2024-04-16T16:56:26.430 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-b/store.db: 000004.log size: 636 ; 2024-04-16T16:56:26.430 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: rocksdb: Options.error_if_exists: 0 2024-04-16T16:56:26.430 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: rocksdb: Options.create_if_missing: 0 2024-04-16T16:56:26.430 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: rocksdb: Options.paranoid_checks: 1 2024-04-16T16:56:26.430 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: rocksdb: Options.flush_verify_memtable_count: 1 2024-04-16T16:56:26.430 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-04-16T16:56:26.431 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-04-16T16:56:26.431 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: rocksdb: Options.env: 0x5559b2b6ecc0 2024-04-16T16:56:26.431 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: rocksdb: Options.fs: PosixFileSystem 2024-04-16T16:56:26.431 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: rocksdb: Options.info_log: 0x5559b3453c20 2024-04-16T16:56:26.431 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: rocksdb: Options.max_file_opening_threads: 16 2024-04-16T16:56:26.431 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: rocksdb: Options.statistics: (nil) 2024-04-16T16:56:26.431 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: rocksdb: Options.use_fsync: 0 2024-04-16T16:56:26.431 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: rocksdb: Options.max_log_file_size: 0 2024-04-16T16:56:26.431 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-04-16T16:56:26.431 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: rocksdb: Options.log_file_time_to_roll: 0 2024-04-16T16:56:26.431 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: rocksdb: Options.keep_log_file_num: 1000 2024-04-16T16:56:26.431 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: rocksdb: Options.recycle_log_file_num: 0 2024-04-16T16:56:26.432 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: rocksdb: Options.allow_fallocate: 1 2024-04-16T16:56:26.432 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: rocksdb: Options.allow_mmap_reads: 0 2024-04-16T16:56:26.432 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: rocksdb: Options.allow_mmap_writes: 0 2024-04-16T16:56:26.432 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: rocksdb: Options.use_direct_reads: 0 2024-04-16T16:56:26.432 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-04-16T16:56:26.432 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: rocksdb: Options.create_missing_column_families: 0 2024-04-16T16:56:26.432 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: rocksdb: Options.db_log_dir: 2024-04-16T16:56:26.432 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: rocksdb: Options.wal_dir: 2024-04-16T16:56:26.432 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: rocksdb: Options.table_cache_numshardbits: 6 2024-04-16T16:56:26.432 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: rocksdb: Options.WAL_ttl_seconds: 0 2024-04-16T16:56:26.432 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: rocksdb: Options.WAL_size_limit_MB: 0 2024-04-16T16:56:26.432 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-04-16T16:56:26.433 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-04-16T16:56:26.433 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: rocksdb: Options.is_fd_close_on_exec: 1 2024-04-16T16:56:26.433 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: rocksdb: Options.advise_random_on_open: 1 2024-04-16T16:56:26.433 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: rocksdb: Options.db_write_buffer_size: 0 2024-04-16T16:56:26.433 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: rocksdb: Options.write_buffer_manager: 0x5559b34221e0 2024-04-16T16:56:26.433 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-04-16T16:56:26.433 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-04-16T16:56:26.433 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: rocksdb: Options.use_adaptive_mutex: 0 2024-04-16T16:56:26.433 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: rocksdb: Options.rate_limiter: (nil) 2024-04-16T16:56:26.433 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-04-16T16:56:26.433 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: rocksdb: Options.wal_recovery_mode: 2 2024-04-16T16:56:26.433 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: rocksdb: Options.enable_thread_tracking: 0 2024-04-16T16:56:26.433 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: rocksdb: Options.enable_pipelined_write: 0 2024-04-16T16:56:26.433 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: rocksdb: Options.unordered_write: 0 2024-04-16T16:56:26.434 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-04-16T16:56:26.434 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-04-16T16:56:26.434 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-04-16T16:56:26.434 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-04-16T16:56:26.434 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: rocksdb: Options.row_cache: None 2024-04-16T16:56:26.434 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: rocksdb: Options.wal_filter: None 2024-04-16T16:56:26.434 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-04-16T16:56:26.434 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: rocksdb: Options.allow_ingest_behind: 0 2024-04-16T16:56:26.434 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: rocksdb: Options.two_write_queues: 0 2024-04-16T16:56:26.434 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: rocksdb: Options.manual_wal_flush: 0 2024-04-16T16:56:26.434 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: rocksdb: Options.wal_compression: 0 2024-04-16T16:56:26.434 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: rocksdb: Options.atomic_flush: 0 2024-04-16T16:56:26.434 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-04-16T16:56:26.434 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: rocksdb: Options.persist_stats_to_disk: 0 2024-04-16T16:56:26.435 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: rocksdb: Options.write_dbid_to_manifest: 0 2024-04-16T16:56:26.435 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: rocksdb: Options.log_readahead_size: 0 2024-04-16T16:56:26.435 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-04-16T16:56:26.435 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: rocksdb: Options.best_efforts_recovery: 0 2024-04-16T16:56:26.435 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-04-16T16:56:26.435 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-04-16T16:56:26.435 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: rocksdb: Options.allow_data_in_errors: 0 2024-04-16T16:56:26.435 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: rocksdb: Options.db_host_id: __hostname__ 2024-04-16T16:56:26.435 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: rocksdb: Options.enforce_single_del_contracts: true 2024-04-16T16:56:26.437 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: rocksdb: Options.max_background_jobs: 2 2024-04-16T16:56:26.437 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: rocksdb: Options.max_background_compactions: -1 2024-04-16T16:56:26.437 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: rocksdb: Options.max_subcompactions: 1 2024-04-16T16:56:26.437 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-04-16T16:56:26.437 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-04-16T16:56:26.437 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: rocksdb: Options.delayed_write_rate : 16777216 2024-04-16T16:56:26.437 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: rocksdb: Options.max_total_wal_size: 0 2024-04-16T16:56:26.437 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-04-16T16:56:26.437 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: rocksdb: Options.stats_dump_period_sec: 600 2024-04-16T16:56:26.437 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: rocksdb: Options.stats_persist_period_sec: 600 2024-04-16T16:56:26.437 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-04-16T16:56:26.437 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: rocksdb: Options.max_open_files: -1 2024-04-16T16:56:26.437 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: rocksdb: Options.bytes_per_sync: 0 2024-04-16T16:56:26.438 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: rocksdb: Options.wal_bytes_per_sync: 0 2024-04-16T16:56:26.438 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: rocksdb: Options.strict_bytes_per_sync: 0 2024-04-16T16:56:26.438 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: rocksdb: Options.compaction_readahead_size: 0 2024-04-16T16:56:26.438 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: rocksdb: Options.max_background_flushes: -1 2024-04-16T16:56:26.438 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: rocksdb: Compression algorithms supported: 2024-04-16T16:56:26.438 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-04-16T16:56:26.438 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: rocksdb: kZSTD supported: 0 2024-04-16T16:56:26.438 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: rocksdb: kXpressCompression supported: 0 2024-04-16T16:56:26.438 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: rocksdb: kLZ4HCCompression supported: 1 2024-04-16T16:56:26.438 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: rocksdb: kZlibCompression supported: 1 2024-04-16T16:56:26.438 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: rocksdb: kSnappyCompression supported: 1 2024-04-16T16:56:26.438 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: rocksdb: kLZ4Compression supported: 1 2024-04-16T16:56:26.438 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: rocksdb: kBZip2Compression supported: 0 2024-04-16T16:56:26.438 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: rocksdb: Fast CRC32 supported: Supported on x86 2024-04-16T16:56:26.438 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: rocksdb: DMutex implementation: pthread_mutex_t 2024-04-16T16:56:26.439 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 2024-04-16T16:56:26.439 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-04-16T16:56:26.439 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-04-16T16:56:26.439 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: rocksdb: Options.merge_operator: 2024-04-16T16:56:26.439 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: rocksdb: Options.compaction_filter: None 2024-04-16T16:56:26.439 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: rocksdb: Options.compaction_filter_factory: None 2024-04-16T16:56:26.439 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: rocksdb: Options.sst_partitioner_factory: None 2024-04-16T16:56:26.439 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: rocksdb: Options.memtable_factory: SkipListFactory 2024-04-16T16:56:26.439 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: rocksdb: Options.table_factory: BlockBasedTable 2024-04-16T16:56:26.439 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x5559b3453de0) 2024-04-16T16:56:26.439 INFO:journalctl@ceph.mon.b.smithi104.stdout: cache_index_and_filter_blocks: 1 2024-04-16T16:56:26.439 INFO:journalctl@ceph.mon.b.smithi104.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-04-16T16:56:26.439 INFO:journalctl@ceph.mon.b.smithi104.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-04-16T16:56:26.439 INFO:journalctl@ceph.mon.b.smithi104.stdout: pin_top_level_index_and_filter: 1 2024-04-16T16:56:26.439 INFO:journalctl@ceph.mon.b.smithi104.stdout: index_type: 0 2024-04-16T16:56:26.439 INFO:journalctl@ceph.mon.b.smithi104.stdout: data_block_index_type: 0 2024-04-16T16:56:26.440 INFO:journalctl@ceph.mon.b.smithi104.stdout: index_shortening: 1 2024-04-16T16:56:26.440 INFO:journalctl@ceph.mon.b.smithi104.stdout: data_block_hash_table_util_ratio: 0.750000 2024-04-16T16:56:26.440 INFO:journalctl@ceph.mon.b.smithi104.stdout: checksum: 4 2024-04-16T16:56:26.440 INFO:journalctl@ceph.mon.b.smithi104.stdout: no_block_cache: 0 2024-04-16T16:56:26.440 INFO:journalctl@ceph.mon.b.smithi104.stdout: block_cache: 0x5559b3455090 2024-04-16T16:56:26.440 INFO:journalctl@ceph.mon.b.smithi104.stdout: block_cache_name: BinnedLRUCache 2024-04-16T16:56:26.440 INFO:journalctl@ceph.mon.b.smithi104.stdout: block_cache_options: 2024-04-16T16:56:26.440 INFO:journalctl@ceph.mon.b.smithi104.stdout: capacity : 536870912 2024-04-16T16:56:26.440 INFO:journalctl@ceph.mon.b.smithi104.stdout: num_shard_bits : 4 2024-04-16T16:56:26.440 INFO:journalctl@ceph.mon.b.smithi104.stdout: strict_capacity_limit : 0 2024-04-16T16:56:26.440 INFO:journalctl@ceph.mon.b.smithi104.stdout: high_pri_pool_ratio: 0.000 2024-04-16T16:56:26.440 INFO:journalctl@ceph.mon.b.smithi104.stdout: block_cache_compressed: (nil) 2024-04-16T16:56:26.440 INFO:journalctl@ceph.mon.b.smithi104.stdout: persistent_cache: (nil) 2024-04-16T16:56:26.440 INFO:journalctl@ceph.mon.b.smithi104.stdout: block_size: 4096 2024-04-16T16:56:26.440 INFO:journalctl@ceph.mon.b.smithi104.stdout: block_size_deviation: 10 2024-04-16T16:56:26.440 INFO:journalctl@ceph.mon.b.smithi104.stdout: block_restart_interval: 16 2024-04-16T16:56:26.441 INFO:journalctl@ceph.mon.b.smithi104.stdout: index_block_restart_interval: 1 2024-04-16T16:56:26.441 INFO:journalctl@ceph.mon.b.smithi104.stdout: metadata_block_size: 4096 2024-04-16T16:56:26.441 INFO:journalctl@ceph.mon.b.smithi104.stdout: partition_filters: 0 2024-04-16T16:56:26.441 INFO:journalctl@ceph.mon.b.smithi104.stdout: use_delta_encoding: 1 2024-04-16T16:56:26.441 INFO:journalctl@ceph.mon.b.smithi104.stdout: filter_policy: bloomfilter 2024-04-16T16:56:26.441 INFO:journalctl@ceph.mon.b.smithi104.stdout: whole_key_filtering: 1 2024-04-16T16:56:26.441 INFO:journalctl@ceph.mon.b.smithi104.stdout: verify_compression: 0 2024-04-16T16:56:26.441 INFO:journalctl@ceph.mon.b.smithi104.stdout: read_amp_bytes_per_bit: 0 2024-04-16T16:56:26.441 INFO:journalctl@ceph.mon.b.smithi104.stdout: format_version: 5 2024-04-16T16:56:26.441 INFO:journalctl@ceph.mon.b.smithi104.stdout: enable_index_compression: 1 2024-04-16T16:56:26.441 INFO:journalctl@ceph.mon.b.smithi104.stdout: block_align: 0 2024-04-16T16:56:26.441 INFO:journalctl@ceph.mon.b.smithi104.stdout: max_auto_readahead_size: 262144 2024-04-16T16:56:26.441 INFO:journalctl@ceph.mon.b.smithi104.stdout: prepopulate_block_cache: 0 2024-04-16T16:56:26.441 INFO:journalctl@ceph.mon.b.smithi104.stdout: initial_auto_readahead_size: 8192 2024-04-16T16:56:26.441 INFO:journalctl@ceph.mon.b.smithi104.stdout: num_file_reads_for_auto_readahead: 2 2024-04-16T16:56:26.442 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: rocksdb: Options.write_buffer_size: 33554432 2024-04-16T16:56:26.442 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: rocksdb: Options.max_write_buffer_number: 2 2024-04-16T16:56:26.442 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: rocksdb: Options.compression: NoCompression 2024-04-16T16:56:26.442 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: rocksdb: Options.bottommost_compression: Disabled 2024-04-16T16:56:26.442 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: rocksdb: Options.prefix_extractor: nullptr 2024-04-16T16:56:26.442 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-04-16T16:56:26.442 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: rocksdb: Options.num_levels: 7 2024-04-16T16:56:26.442 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-04-16T16:56:26.442 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-04-16T16:56:26.442 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-04-16T16:56:26.442 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-04-16T16:56:26.442 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-04-16T16:56:26.442 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-04-16T16:56:26.442 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-04-16T16:56:26.442 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-04-16T16:56:26.443 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-04-16T16:56:26.443 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-04-16T16:56:26.443 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-04-16T16:56:26.443 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-04-16T16:56:26.443 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: rocksdb: Options.compression_opts.window_bits: -14 2024-04-16T16:56:26.443 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: rocksdb: Options.compression_opts.level: 32767 2024-04-16T16:56:26.443 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: rocksdb: Options.compression_opts.strategy: 0 2024-04-16T16:56:26.443 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-04-16T16:56:26.443 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-04-16T16:56:26.443 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-04-16T16:56:26.443 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-04-16T16:56:26.443 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: rocksdb: Options.compression_opts.enabled: false 2024-04-16T16:56:26.443 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-04-16T16:56:26.443 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-04-16T16:56:26.443 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-04-16T16:56:26.443 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-04-16T16:56:26.444 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: rocksdb: Options.target_file_size_base: 67108864 2024-04-16T16:56:26.444 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: rocksdb: Options.target_file_size_multiplier: 1 2024-04-16T16:56:26.444 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-04-16T16:56:26.444 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-04-16T16:56:26.444 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-04-16T16:56:26.444 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-04-16T16:56:26.444 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-04-16T16:56:26.444 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-04-16T16:56:26.444 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-04-16T16:56:26.444 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-04-16T16:56:26.444 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-04-16T16:56:26.444 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-04-16T16:56:26.444 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-04-16T16:56:26.444 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-04-16T16:56:26.444 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-04-16T16:56:26.444 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: rocksdb: Options.arena_block_size: 1048576 2024-04-16T16:56:26.444 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-04-16T16:56:26.445 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-04-16T16:56:26.445 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: rocksdb: Options.disable_auto_compactions: 0 2024-04-16T16:56:26.445 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-04-16T16:56:26.445 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-04-16T16:56:26.445 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-04-16T16:56:26.445 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-04-16T16:56:26.445 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-04-16T16:56:26.445 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-04-16T16:56:26.445 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-04-16T16:56:26.445 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-04-16T16:56:26.445 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-04-16T16:56:26.445 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-04-16T16:56:26.445 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-04-16T16:56:26.445 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: rocksdb: Options.inplace_update_support: 0 2024-04-16T16:56:26.445 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: rocksdb: Options.inplace_update_num_locks: 10000 2024-04-16T16:56:26.446 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-04-16T16:56:26.446 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-04-16T16:56:26.446 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: rocksdb: Options.memtable_huge_page_size: 0 2024-04-16T16:56:26.446 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: rocksdb: Options.bloom_locality: 0 2024-04-16T16:56:26.446 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: rocksdb: Options.max_successive_merges: 0 2024-04-16T16:56:26.446 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: rocksdb: Options.optimize_filters_for_hits: 0 2024-04-16T16:56:26.446 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: rocksdb: Options.paranoid_file_checks: 0 2024-04-16T16:56:26.446 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: rocksdb: Options.force_consistency_checks: 1 2024-04-16T16:56:26.446 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: rocksdb: Options.report_bg_io_stats: 0 2024-04-16T16:56:26.446 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: rocksdb: Options.ttl: 2592000 2024-04-16T16:56:26.446 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: rocksdb: Options.periodic_compaction_seconds: 0 2024-04-16T16:56:26.446 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-04-16T16:56:26.446 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-04-16T16:56:26.446 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: rocksdb: Options.enable_blob_files: false 2024-04-16T16:56:26.446 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: rocksdb: Options.min_blob_size: 0 2024-04-16T16:56:26.446 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: rocksdb: Options.blob_file_size: 268435456 2024-04-16T16:56:26.446 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: rocksdb: Options.blob_compression_type: NoCompression 2024-04-16T16:56:26.447 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: rocksdb: Options.enable_blob_garbage_collection: false 2024-04-16T16:56:26.447 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-04-16T16:56:26.447 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-04-16T16:56:26.447 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-04-16T16:56:26.447 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: rocksdb: Options.blob_file_starting_level: 0 2024-04-16T16:56:26.447 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-04-16T16:56:26.447 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-04-16T16:56:26.447 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-04-16T16:56:26.447 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: f031e7bd-2490-4878-bc37-bf5164054182 2024-04-16T16:56:26.447 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713286586161228, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-04-16T16:56:26.447 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-04-16T16:56:26.447 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713286586161801, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1773, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 648, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 526, "raw_average_value_size": 105, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1713286586, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "f031e7bd-2490-4878-bc37-bf5164054182", "db_session_id": "7ZZC0KSLS1MQVRCUWQ2I", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-04-16T16:56:26.447 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713286586161905, "job": 1, "event": "recovery_finished"} 2024-04-16T16:56:26.447 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-04-16T16:56:26.447 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-b/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-04-16T16:56:26.447 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x5559b3542000 2024-04-16T16:56:26.448 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: rocksdb: DB pointer 0x5559b352c000 2024-04-16T16:56:26.448 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-04-16T16:56:26.448 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-04-16T16:56:26.448 INFO:journalctl@ceph.mon.b.smithi104.stdout: ** DB Stats ** 2024-04-16T16:56:26.448 INFO:journalctl@ceph.mon.b.smithi104.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-16T16:56:26.448 INFO:journalctl@ceph.mon.b.smithi104.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-04-16T16:56:26.448 INFO:journalctl@ceph.mon.b.smithi104.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-16T16:56:26.448 INFO:journalctl@ceph.mon.b.smithi104.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-16T16:56:26.448 INFO:journalctl@ceph.mon.b.smithi104.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-04-16T16:56:26.448 INFO:journalctl@ceph.mon.b.smithi104.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-16T16:56:26.448 INFO:journalctl@ceph.mon.b.smithi104.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-16T16:56:26.448 INFO:journalctl@ceph.mon.b.smithi104.stdout: 2024-04-16T16:56:26.448 INFO:journalctl@ceph.mon.b.smithi104.stdout: ** Compaction Stats [default] ** 2024-04-16T16:56:26.448 INFO:journalctl@ceph.mon.b.smithi104.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-16T16:56:26.448 INFO:journalctl@ceph.mon.b.smithi104.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-04-16T16:56:26.448 INFO:journalctl@ceph.mon.b.smithi104.stdout: L0 1/0 1.73 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-16T16:56:26.449 INFO:journalctl@ceph.mon.b.smithi104.stdout: Sum 1/0 1.73 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-16T16:56:26.449 INFO:journalctl@ceph.mon.b.smithi104.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-16T16:56:26.449 INFO:journalctl@ceph.mon.b.smithi104.stdout: 2024-04-16T16:56:26.449 INFO:journalctl@ceph.mon.b.smithi104.stdout: ** Compaction Stats [default] ** 2024-04-16T16:56:26.449 INFO:journalctl@ceph.mon.b.smithi104.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-16T16:56:26.449 INFO:journalctl@ceph.mon.b.smithi104.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-16T16:56:26.449 INFO:journalctl@ceph.mon.b.smithi104.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 3.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-16T16:56:26.449 INFO:journalctl@ceph.mon.b.smithi104.stdout: 2024-04-16T16:56:26.449 INFO:journalctl@ceph.mon.b.smithi104.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-04-16T16:56:26.449 INFO:journalctl@ceph.mon.b.smithi104.stdout: 2024-04-16T16:56:26.449 INFO:journalctl@ceph.mon.b.smithi104.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-16T16:56:26.449 INFO:journalctl@ceph.mon.b.smithi104.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-04-16T16:56:26.449 INFO:journalctl@ceph.mon.b.smithi104.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-04-16T16:56:26.449 INFO:journalctl@ceph.mon.b.smithi104.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-04-16T16:56:26.449 INFO:journalctl@ceph.mon.b.smithi104.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-04-16T16:56:26.449 INFO:journalctl@ceph.mon.b.smithi104.stdout: AddFile(Keys): cumulative 0, interval 0 2024-04-16T16:56:26.450 INFO:journalctl@ceph.mon.b.smithi104.stdout: Cumulative compaction: 0.00 GB write, 0.49 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-16T16:56:26.450 INFO:journalctl@ceph.mon.b.smithi104.stdout: Interval compaction: 0.00 GB write, 0.49 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-16T16:56:26.450 INFO:journalctl@ceph.mon.b.smithi104.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-04-16T16:56:26.450 INFO:journalctl@ceph.mon.b.smithi104.stdout: Block cache BinnedLRUCache@0x5559b3455090#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-04-16T16:56:26.450 INFO:journalctl@ceph.mon.b.smithi104.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-04-16T16:56:26.450 INFO:journalctl@ceph.mon.b.smithi104.stdout: 2024-04-16T16:56:26.450 INFO:journalctl@ceph.mon.b.smithi104.stdout: ** File Read Latency Histogram By Level [default] ** 2024-04-16T16:56:26.450 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: mon.b does not exist in monmap, will attempt to join an existing cluster 2024-04-16T16:56:26.450 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: using public_addr v2:172.21.15.104:0/0 -> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] 2024-04-16T16:56:26.450 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: starting mon.b rank -1 at public addrs [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] at bind addrs [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] mon_data /var/lib/ceph/mon/ceph-b fsid b1fecd38-fc11-11ee-bc90-c7b262605968 2024-04-16T16:56:26.450 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: mon.b@-1(???) e0 preinit fsid b1fecd38-fc11-11ee-bc90-c7b262605968 2024-04-16T16:56:26.450 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: mon.b@-1(synchronizing).mds e1 new map 2024-04-16T16:56:26.450 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: mon.b@-1(synchronizing).mds e1 print_map 2024-04-16T16:56:26.450 INFO:journalctl@ceph.mon.b.smithi104.stdout: e1 2024-04-16T16:56:26.450 INFO:journalctl@ceph.mon.b.smithi104.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-04-16T16:56:26.451 INFO:journalctl@ceph.mon.b.smithi104.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-16T16:56:26.451 INFO:journalctl@ceph.mon.b.smithi104.stdout: legacy client fscid: -1 2024-04-16T16:56:26.451 INFO:journalctl@ceph.mon.b.smithi104.stdout: 2024-04-16T16:56:26.451 INFO:journalctl@ceph.mon.b.smithi104.stdout: No filesystems configured 2024-04-16T16:56:26.451 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-04-16T16:56:26.451 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-04-16T16:56:26.451 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-04-16T16:56:26.451 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-04-16T16:56:26.451 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-04-16T16:56:26.451 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-04-16T16:56:26.451 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-04-16T16:56:26.451 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-16T16:56:26.451 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-16T16:56:26.451 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-16T16:56:26.451 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:56:26.451 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:56:26.452 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T16:56:26.452 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: Added host smithi104 2024-04-16T16:56:26.452 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:56:26.452 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:56:26.452 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:56:26.452 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:56:26.452 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:56:26.452 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-16T16:56:26.452 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:56:26.452 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:56:26.452 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:56:26.452 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:56:26.452 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:56:26.452 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi104", "name": "osd_memory_target"}]: dispatch 2024-04-16T16:56:26.452 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:56:26.452 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T16:56:26.453 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: Updating smithi104:/etc/ceph/ceph.conf 2024-04-16T16:56:26.453 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: Updating smithi104:/var/lib/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/config/ceph.conf 2024-04-16T16:56:26.453 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: Updating smithi104:/etc/ceph/ceph.client.admin.keyring 2024-04-16T16:56:26.453 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: Updating smithi104:/var/lib/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/config/ceph.client.admin.keyring 2024-04-16T16:56:26.453 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:56:26.453 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi117", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T16:56:26.453 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:56:26.453 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:56:26.453 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:56:26.453 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: Deploying cephadm binary to smithi117 2024-04-16T16:56:26.453 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:56:26.453 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:56:26.453 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:56:26.453 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T16:56:26.453 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: Added host smithi117 2024-04-16T16:56:26.453 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:56:26.454 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:56:26.454 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:56:26.454 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:56:26.454 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:56:26.454 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-16T16:56:26.454 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:56:26.454 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:56:26.454 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:56:26.454 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:56:26.454 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:56:26.454 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi117", "name": "osd_memory_target"}]: dispatch 2024-04-16T16:56:26.454 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:56:26.454 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T16:56:26.454 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: Updating smithi117:/etc/ceph/ceph.conf 2024-04-16T16:56:26.454 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: Updating smithi117:/var/lib/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/config/ceph.conf 2024-04-16T16:56:26.455 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: Updating smithi117:/etc/ceph/ceph.client.admin.keyring 2024-04-16T16:56:26.455 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:56:26.455 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:56:26.455 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:56:26.455 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: from='client.? 172.21.15.45:0/1006631336' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-04-16T16:56:26.455 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: Updating smithi117:/var/lib/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/config/ceph.client.admin.keyring 2024-04-16T16:56:26.455 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:56:26.455 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: from='client.? 172.21.15.45:0/1006631336' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-04-16T16:56:26.455 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: osdmap e4: 0 total, 0 up, 0 in 2024-04-16T16:56:26.455 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:56:26.455 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi045:172.21.15.45=a;smithi104:172.21.15.104=b;smithi117:172.21.15.117=c", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T16:56:26.455 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: Saving service mon spec with placement smithi045:172.21.15.45=a;smithi104:172.21.15.104=b;smithi117:172.21.15.117=c;count:3 2024-04-16T16:56:26.455 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:56:26.455 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T16:56:26.455 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:56:26.455 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T16:56:26.456 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:56:26.456 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-16T16:56:26.456 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:56:26.456 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: Deploying daemon mon.c on smithi117 2024-04-16T16:56:26.456 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:56:26.456 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:56:26.456 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: from='client.? 172.21.15.117:0/1087836869' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-16T16:56:26.456 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-16T16:56:26.456 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: mon.a calling monitor election 2024-04-16T16:56:26.456 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-16T16:56:26.456 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:56:26.456 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-16T16:56:26.456 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-16T16:56:26.456 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: mon.c calling monitor election 2024-04-16T16:56:26.456 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:56:26.456 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-16T16:56:26.457 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-16T16:56:26.457 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-16T16:56:26.457 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-16T16:56:26.457 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: monmap epoch 2 2024-04-16T16:56:26.457 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: fsid b1fecd38-fc11-11ee-bc90-c7b262605968 2024-04-16T16:56:26.457 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: last_changed 2024-04-16T16:56:17.259442+0000 2024-04-16T16:56:26.457 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: created 2024-04-16T16:53:58.140859+0000 2024-04-16T16:56:26.457 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: min_mon_release 19 (squid) 2024-04-16T16:56:26.457 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: election_strategy: 1 2024-04-16T16:56:26.457 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: 0: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.a 2024-04-16T16:56:26.457 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: 1: [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] mon.c 2024-04-16T16:56:26.457 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: fsmap 2024-04-16T16:56:26.457 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: osdmap e4: 0 total, 0 up, 0 in 2024-04-16T16:56:26.457 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: mgrmap e13: x(active, since 62s) 2024-04-16T16:56:26.457 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: overall HEALTH_OK 2024-04-16T16:56:26.457 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:56:26.457 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:56:26.458 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:56:26.458 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:56:26.458 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-16T16:56:26.458 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:56:26.458 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: Deploying daemon mon.b on smithi104 2024-04-16T16:56:26.458 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: from='client.? 172.21.15.117:0/773820889' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-16T16:56:26.458 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-16T16:56:26.458 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:56:26.458 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 ceph-mon[36457]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-04-16T16:56:26.458 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 bash[36437]: 68a342e84dea98df618d136c2aca253c177fb630c8b2f18d159f2106c3f1c4f7 2024-04-16T16:56:26.458 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:26 smithi104 systemd[1]: Started Ceph mon.b for b1fecd38-fc11-11ee-bc90-c7b262605968. 2024-04-16T16:56:31.210 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:56:31.209+0000 7f3bfffff700 1 -- 172.21.15.117:0/1671446774 <== mon.0 v2:172.21.15.45:3300/0 8 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) ==== 95+0+1283 (secure 0 0 0) 0x7f3c0c038520 con 0x7f3c181086a0 2024-04-16T16:56:31.211 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-04-16T16:56:31.211 INFO:teuthology.orchestra.run.smithi117.stdout:{"epoch":3,"fsid":"b1fecd38-fc11-11ee-bc90-c7b262605968","modified":"2024-04-16T16:56:26.187742Z","created":"2024-04-16T16:53:58.140859Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:3300","nonce":0},{"type":"v1","addr":"172.21.15.45:6789","nonce":0}]},"addr":"172.21.15.45:6789/0","public_addr":"172.21.15.45:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:3300","nonce":0},{"type":"v1","addr":"172.21.15.117:6789","nonce":0}]},"addr":"172.21.15.117:6789/0","public_addr":"172.21.15.117:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:3300","nonce":0},{"type":"v1","addr":"172.21.15.104:6789","nonce":0}]},"addr":"172.21.15.104:6789/0","public_addr":"172.21.15.104:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-04-16T16:56:31.213 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:56:31.212+0000 7f3c1d630700 1 -- 172.21.15.117:0/1671446774 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f3c000393f0 msgr2=0x7f3c0003b8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:56:31.213 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:56:31.212+0000 7f3c1d630700 1 --2- 172.21.15.117:0/1671446774 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f3c000393f0 0x7f3c0003b8b0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f3c08009180 tx=0x7f3c08008040 comp rx=0 tx=0).stop 2024-04-16T16:56:31.213 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:56:31.212+0000 7f3c1d630700 1 -- 172.21.15.117:0/1671446774 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3c181086a0 msgr2=0x7f3c18198250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:56:31.213 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:56:31.212+0000 7f3c1d630700 1 --2- 172.21.15.117:0/1671446774 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3c181086a0 0x7f3c18198250 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f3c18103750 tx=0x7f3c0c005030 comp rx=0 tx=0).stop 2024-04-16T16:56:31.213 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:56:31.212+0000 7f3c1d630700 1 -- 172.21.15.117:0/1671446774 shutdown_connections 2024-04-16T16:56:31.213 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:56:31.212+0000 7f3c1d630700 1 --2- 172.21.15.117:0/1671446774 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3c181086a0 0x7f3c18198250 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:56:31.214 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:56:31.212+0000 7f3c1d630700 1 --2- 172.21.15.117:0/1671446774 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f3c000393f0 0x7f3c0003b8b0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:56:31.214 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:56:31.212+0000 7f3c1d630700 1 --2- 172.21.15.117:0/1671446774 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3c18198790 0x7f3c1819cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:56:31.214 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:56:31.212+0000 7f3c1d630700 1 -- 172.21.15.117:0/1671446774 >> 172.21.15.117:0/1671446774 conn(0x7f3c180fe100 msgr2=0x7f3c180fe4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T16:56:31.214 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:56:31.212+0000 7f3c1d630700 1 -- 172.21.15.117:0/1671446774 shutdown_connections 2024-04-16T16:56:31.214 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:56:31.212+0000 7f3c1d630700 1 -- 172.21.15.117:0/1671446774 wait complete. 2024-04-16T16:56:31.214 INFO:teuthology.orchestra.run.smithi117.stderr:dumped monmap epoch 3 2024-04-16T16:56:31.466 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:31 smithi117 ceph-mon[36730]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:56:31.466 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:31 smithi117 ceph-mon[36730]: mon.c calling monitor election 2024-04-16T16:56:31.466 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:31 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-16T16:56:31.467 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:31 smithi117 ceph-mon[36730]: mon.a calling monitor election 2024-04-16T16:56:31.467 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:31 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-16T16:56:31.467 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:31 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-16T16:56:31.467 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:31 smithi117 ceph-mon[36730]: from='client.? 172.21.15.117:0/1671446774' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-16T16:56:31.467 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:31 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-16T16:56:31.467 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:31 smithi117 ceph-mon[36730]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:56:31.467 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:31 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-16T16:56:31.467 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:31 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-16T16:56:31.467 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:31 smithi117 ceph-mon[36730]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:56:31.467 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:31 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-16T16:56:31.467 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:31 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-16T16:56:31.467 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:31 smithi117 ceph-mon[36730]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-16T16:56:31.467 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:31 smithi117 ceph-mon[36730]: monmap epoch 3 2024-04-16T16:56:31.467 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:31 smithi117 ceph-mon[36730]: fsid b1fecd38-fc11-11ee-bc90-c7b262605968 2024-04-16T16:56:31.468 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:31 smithi117 ceph-mon[36730]: last_changed 2024-04-16T16:56:26.187742+0000 2024-04-16T16:56:31.468 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:31 smithi117 ceph-mon[36730]: created 2024-04-16T16:53:58.140859+0000 2024-04-16T16:56:31.468 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:31 smithi117 ceph-mon[36730]: min_mon_release 19 (squid) 2024-04-16T16:56:31.468 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:31 smithi117 ceph-mon[36730]: election_strategy: 1 2024-04-16T16:56:31.468 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:31 smithi117 ceph-mon[36730]: 0: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.a 2024-04-16T16:56:31.468 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:31 smithi117 ceph-mon[36730]: 1: [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] mon.c 2024-04-16T16:56:31.468 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:31 smithi117 ceph-mon[36730]: 2: [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] mon.b 2024-04-16T16:56:31.468 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:31 smithi117 ceph-mon[36730]: fsmap 2024-04-16T16:56:31.468 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:31 smithi117 ceph-mon[36730]: osdmap e4: 0 total, 0 up, 0 in 2024-04-16T16:56:31.468 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:31 smithi117 ceph-mon[36730]: mgrmap e13: x(active, since 71s) 2024-04-16T16:56:31.468 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:31 smithi117 ceph-mon[36730]: overall HEALTH_OK 2024-04-16T16:56:31.468 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:31 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:56:31.468 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:31 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:56:31.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:56:31 smithi045 ceph-mon[31676]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:56:31.729 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:56:31 smithi045 ceph-mon[31676]: mon.c calling monitor election 2024-04-16T16:56:31.729 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:56:31 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-16T16:56:31.729 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:56:31 smithi045 ceph-mon[31676]: mon.a calling monitor election 2024-04-16T16:56:31.729 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:56:31 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-16T16:56:31.729 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:56:31 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-16T16:56:31.729 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:56:31 smithi045 ceph-mon[31676]: from='client.? 172.21.15.117:0/1671446774' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-16T16:56:31.729 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:56:31 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-16T16:56:31.729 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:56:31 smithi045 ceph-mon[31676]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:56:31.729 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:56:31 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-16T16:56:31.729 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:56:31 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-16T16:56:31.729 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:56:31 smithi045 ceph-mon[31676]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:56:31.729 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:56:31 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-16T16:56:31.729 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:56:31 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-16T16:56:31.730 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:56:31 smithi045 ceph-mon[31676]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-16T16:56:31.730 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:56:31 smithi045 ceph-mon[31676]: monmap epoch 3 2024-04-16T16:56:31.730 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:56:31 smithi045 ceph-mon[31676]: fsid b1fecd38-fc11-11ee-bc90-c7b262605968 2024-04-16T16:56:31.730 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:56:31 smithi045 ceph-mon[31676]: last_changed 2024-04-16T16:56:26.187742+0000 2024-04-16T16:56:31.730 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:56:31 smithi045 ceph-mon[31676]: created 2024-04-16T16:53:58.140859+0000 2024-04-16T16:56:31.730 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:56:31 smithi045 ceph-mon[31676]: min_mon_release 19 (squid) 2024-04-16T16:56:31.730 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:56:31 smithi045 ceph-mon[31676]: election_strategy: 1 2024-04-16T16:56:31.730 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:56:31 smithi045 ceph-mon[31676]: 0: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.a 2024-04-16T16:56:31.730 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:56:31 smithi045 ceph-mon[31676]: 1: [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] mon.c 2024-04-16T16:56:31.730 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:56:31 smithi045 ceph-mon[31676]: 2: [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] mon.b 2024-04-16T16:56:31.730 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:56:31 smithi045 ceph-mon[31676]: fsmap 2024-04-16T16:56:31.730 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:56:31 smithi045 ceph-mon[31676]: osdmap e4: 0 total, 0 up, 0 in 2024-04-16T16:56:31.730 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:56:31 smithi045 ceph-mon[31676]: mgrmap e13: x(active, since 71s) 2024-04-16T16:56:31.730 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:56:31 smithi045 ceph-mon[31676]: overall HEALTH_OK 2024-04-16T16:56:31.731 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:56:31 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:56:31.731 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:56:31 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:56:31.786 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-04-16T16:56:31.787 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b1fecd38-fc11-11ee-bc90-c7b262605968 -- ceph config generate-minimal-conf 2024-04-16T16:56:32.253 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/mon.a/config 2024-04-16T16:56:32.524 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:32 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:56:32.525 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:32 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:56:32.525 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:32 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T16:56:32.525 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:32 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-16T16:56:32.692 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:56:32 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:56:32.692 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:56:32 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:56:32.692 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:56:32 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T16:56:32.692 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:56:32 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-16T16:56:33.679 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:33 smithi104 ceph-mon[36457]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:56:33.679 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:33 smithi104 ceph-mon[36457]: mon.c calling monitor election 2024-04-16T16:56:33.679 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:33 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-16T16:56:33.679 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:33 smithi104 ceph-mon[36457]: mon.a calling monitor election 2024-04-16T16:56:33.679 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:33 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-16T16:56:33.679 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:33 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-16T16:56:33.679 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:33 smithi104 ceph-mon[36457]: from='client.? 172.21.15.117:0/1671446774' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-16T16:56:33.679 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:33 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-16T16:56:33.679 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:33 smithi104 ceph-mon[36457]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:56:33.679 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:33 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-16T16:56:33.679 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:33 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-16T16:56:33.679 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:33 smithi104 ceph-mon[36457]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:56:33.679 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:33 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-16T16:56:33.680 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:33 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-16T16:56:33.680 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:33 smithi104 ceph-mon[36457]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-16T16:56:33.680 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:33 smithi104 ceph-mon[36457]: monmap epoch 3 2024-04-16T16:56:33.680 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:33 smithi104 ceph-mon[36457]: fsid b1fecd38-fc11-11ee-bc90-c7b262605968 2024-04-16T16:56:33.680 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:33 smithi104 ceph-mon[36457]: last_changed 2024-04-16T16:56:26.187742+0000 2024-04-16T16:56:33.680 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:33 smithi104 ceph-mon[36457]: created 2024-04-16T16:53:58.140859+0000 2024-04-16T16:56:33.680 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:33 smithi104 ceph-mon[36457]: min_mon_release 19 (squid) 2024-04-16T16:56:33.680 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:33 smithi104 ceph-mon[36457]: election_strategy: 1 2024-04-16T16:56:33.680 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:33 smithi104 ceph-mon[36457]: 0: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.a 2024-04-16T16:56:33.680 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:33 smithi104 ceph-mon[36457]: 1: [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] mon.c 2024-04-16T16:56:33.680 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:33 smithi104 ceph-mon[36457]: 2: [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] mon.b 2024-04-16T16:56:33.680 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:33 smithi104 ceph-mon[36457]: fsmap 2024-04-16T16:56:33.680 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:33 smithi104 ceph-mon[36457]: osdmap e4: 0 total, 0 up, 0 in 2024-04-16T16:56:33.680 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:33 smithi104 ceph-mon[36457]: mgrmap e13: x(active, since 71s) 2024-04-16T16:56:33.681 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:33 smithi104 ceph-mon[36457]: overall HEALTH_OK 2024-04-16T16:56:33.681 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:33 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:56:33.681 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:33 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:56:33.681 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:33 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:56:33.681 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:33 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:56:33.681 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:33 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T16:56:33.681 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:33 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-16T16:56:34.229 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:56:33 smithi045 ceph-mon[31676]: mon.b calling monitor election 2024-04-16T16:56:34.229 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:56:33 smithi045 ceph-mon[31676]: Updating smithi045:/etc/ceph/ceph.conf 2024-04-16T16:56:34.229 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:56:33 smithi045 ceph-mon[31676]: Updating smithi104:/etc/ceph/ceph.conf 2024-04-16T16:56:34.229 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:56:33 smithi045 ceph-mon[31676]: Updating smithi117:/etc/ceph/ceph.conf 2024-04-16T16:56:34.229 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:56:33 smithi045 ceph-mon[31676]: mon.b calling monitor election 2024-04-16T16:56:34.229 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:56:33 smithi045 ceph-mon[31676]: mon.c calling monitor election 2024-04-16T16:56:34.229 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:56:33 smithi045 ceph-mon[31676]: mon.a calling monitor election 2024-04-16T16:56:34.229 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:56:33 smithi045 ceph-mon[31676]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-16T16:56:34.229 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:56:33 smithi045 ceph-mon[31676]: monmap epoch 3 2024-04-16T16:56:34.229 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:56:33 smithi045 ceph-mon[31676]: fsid b1fecd38-fc11-11ee-bc90-c7b262605968 2024-04-16T16:56:34.229 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:56:33 smithi045 ceph-mon[31676]: last_changed 2024-04-16T16:56:26.187742+0000 2024-04-16T16:56:34.229 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:56:33 smithi045 ceph-mon[31676]: created 2024-04-16T16:53:58.140859+0000 2024-04-16T16:56:34.230 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:56:33 smithi045 ceph-mon[31676]: min_mon_release 19 (squid) 2024-04-16T16:56:34.230 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:56:33 smithi045 ceph-mon[31676]: election_strategy: 1 2024-04-16T16:56:34.230 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:56:33 smithi045 ceph-mon[31676]: 0: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.a 2024-04-16T16:56:34.230 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:56:33 smithi045 ceph-mon[31676]: 1: [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] mon.c 2024-04-16T16:56:34.230 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:56:33 smithi045 ceph-mon[31676]: 2: [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] mon.b 2024-04-16T16:56:34.230 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:56:33 smithi045 ceph-mon[31676]: fsmap 2024-04-16T16:56:34.230 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:56:33 smithi045 ceph-mon[31676]: osdmap e4: 0 total, 0 up, 0 in 2024-04-16T16:56:34.230 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:56:33 smithi045 ceph-mon[31676]: mgrmap e13: x(active, since 73s) 2024-04-16T16:56:34.230 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:56:33 smithi045 ceph-mon[31676]: overall HEALTH_OK 2024-04-16T16:56:34.230 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:56:33 smithi045 ceph-mon[31676]: Updating smithi104:/var/lib/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/config/ceph.conf 2024-04-16T16:56:34.230 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:56:33 smithi045 ceph-mon[31676]: Updating smithi117:/var/lib/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/config/ceph.conf 2024-04-16T16:56:34.230 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:56:33 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:56:34.230 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:56:33 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:56:34.230 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:56:33 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:56:34.230 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:56:33 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:56:34.231 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:56:33 smithi045 ceph-mon[31676]: Updating smithi045:/var/lib/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/config/ceph.conf 2024-04-16T16:56:34.306 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:33 smithi117 ceph-mon[36730]: mon.b calling monitor election 2024-04-16T16:56:34.306 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:33 smithi117 ceph-mon[36730]: Updating smithi045:/etc/ceph/ceph.conf 2024-04-16T16:56:34.306 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:33 smithi117 ceph-mon[36730]: Updating smithi104:/etc/ceph/ceph.conf 2024-04-16T16:56:34.306 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:33 smithi117 ceph-mon[36730]: Updating smithi117:/etc/ceph/ceph.conf 2024-04-16T16:56:34.306 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:33 smithi117 ceph-mon[36730]: mon.b calling monitor election 2024-04-16T16:56:34.306 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:33 smithi117 ceph-mon[36730]: mon.c calling monitor election 2024-04-16T16:56:34.306 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:33 smithi117 ceph-mon[36730]: mon.a calling monitor election 2024-04-16T16:56:34.306 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:33 smithi117 ceph-mon[36730]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-16T16:56:34.307 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:33 smithi117 ceph-mon[36730]: monmap epoch 3 2024-04-16T16:56:34.307 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:33 smithi117 ceph-mon[36730]: fsid b1fecd38-fc11-11ee-bc90-c7b262605968 2024-04-16T16:56:34.307 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:33 smithi117 ceph-mon[36730]: last_changed 2024-04-16T16:56:26.187742+0000 2024-04-16T16:56:34.307 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:33 smithi117 ceph-mon[36730]: created 2024-04-16T16:53:58.140859+0000 2024-04-16T16:56:34.307 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:33 smithi117 ceph-mon[36730]: min_mon_release 19 (squid) 2024-04-16T16:56:34.307 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:33 smithi117 ceph-mon[36730]: election_strategy: 1 2024-04-16T16:56:34.307 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:33 smithi117 ceph-mon[36730]: 0: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.a 2024-04-16T16:56:34.307 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:33 smithi117 ceph-mon[36730]: 1: [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] mon.c 2024-04-16T16:56:34.307 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:33 smithi117 ceph-mon[36730]: 2: [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] mon.b 2024-04-16T16:56:34.307 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:33 smithi117 ceph-mon[36730]: fsmap 2024-04-16T16:56:34.307 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:33 smithi117 ceph-mon[36730]: osdmap e4: 0 total, 0 up, 0 in 2024-04-16T16:56:34.307 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:33 smithi117 ceph-mon[36730]: mgrmap e13: x(active, since 73s) 2024-04-16T16:56:34.307 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:33 smithi117 ceph-mon[36730]: overall HEALTH_OK 2024-04-16T16:56:34.307 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:33 smithi117 ceph-mon[36730]: Updating smithi104:/var/lib/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/config/ceph.conf 2024-04-16T16:56:34.308 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:33 smithi117 ceph-mon[36730]: Updating smithi117:/var/lib/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/config/ceph.conf 2024-04-16T16:56:34.308 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:33 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:56:34.308 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:33 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:56:34.308 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:33 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:56:34.308 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:33 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:56:34.308 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:33 smithi117 ceph-mon[36730]: Updating smithi045:/var/lib/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/config/ceph.conf 2024-04-16T16:56:34.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:56:34.394+0000 7f1b53400700 1 -- 172.21.15.45:0/3657169476 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1b4c074ec0 msgr2=0x7f1b4c0752a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:56:34.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:56:34.394+0000 7f1b53400700 1 --2- 172.21.15.45:0/3657169476 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1b4c074ec0 0x7f1b4c0752a0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f1b40009910 tx=0x7f1b40009c20 comp rx=0 tx=0).stop 2024-04-16T16:56:34.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:56:34.394+0000 7f1b53400700 1 -- 172.21.15.45:0/3657169476 shutdown_connections 2024-04-16T16:56:34.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:56:34.394+0000 7f1b53400700 1 --2- 172.21.15.45:0/3657169476 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1b4c074ec0 0x7f1b4c0752a0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:56:34.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:56:34.394+0000 7f1b53400700 1 -- 172.21.15.45:0/3657169476 >> 172.21.15.45:0/3657169476 conn(0x7f1b4c0fe030 msgr2=0x7f1b4c100450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T16:56:34.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:56:34.394+0000 7f1b53400700 1 -- 172.21.15.45:0/3657169476 shutdown_connections 2024-04-16T16:56:34.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:56:34.394+0000 7f1b53400700 1 -- 172.21.15.45:0/3657169476 wait complete. 2024-04-16T16:56:34.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:56:34.396+0000 7f1b53400700 1 Processor -- start 2024-04-16T16:56:34.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:56:34.396+0000 7f1b53400700 1 -- start start 2024-04-16T16:56:34.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:56:34.396+0000 7f1b53400700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1b4c074ec0 0x7f1b4c19a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:56:34.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:56:34.396+0000 7f1b53400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1b4c19aa50 0x7f1b4c19ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:56:34.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:56:34.396+0000 7f1b53400700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1b4c19f320 0x7f1b4c19f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:56:34.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:56:34.396+0000 7f1b53400700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f1b4c10ed50 con 0x7f1b4c074ec0 2024-04-16T16:56:34.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:56:34.396+0000 7f1b53400700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f1b4c10ebd0 con 0x7f1b4c19f320 2024-04-16T16:56:34.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:56:34.396+0000 7f1b53400700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f1b4c10eed0 con 0x7f1b4c19aa50 2024-04-16T16:56:34.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:56:34.397+0000 7f1b50a00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1b4c074ec0 0x7f1b4c19a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:56:34.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:56:34.397+0000 7f1b50a00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1b4c074ec0 0x7f1b4c19a510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:44384/0 (socket says 172.21.15.45:44384) 2024-04-16T16:56:34.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:56:34.397+0000 7f1b50a00700 1 -- 172.21.15.45:0/359825857 learned_addr learned my addr 172.21.15.45:0/359825857 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T16:56:34.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:56:34.397+0000 7f1b51400700 1 --2- 172.21.15.45:0/359825857 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1b4c19f320 0x7f1b4c19f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:56:34.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:56:34.397+0000 7f1b4be00700 1 --2- 172.21.15.45:0/359825857 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1b4c19aa50 0x7f1b4c19ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:56:34.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:56:34.397+0000 7f1b50a00700 1 -- 172.21.15.45:0/359825857 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1b4c19aa50 msgr2=0x7f1b4c19ede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:56:34.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:56:34.397+0000 7f1b50a00700 1 --2- 172.21.15.45:0/359825857 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1b4c19aa50 0x7f1b4c19ede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:56:34.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:56:34.397+0000 7f1b50a00700 1 -- 172.21.15.45:0/359825857 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1b4c19f320 msgr2=0x7f1b4c19f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:56:34.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:56:34.397+0000 7f1b50a00700 1 --2- 172.21.15.45:0/359825857 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1b4c19f320 0x7f1b4c19f7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:56:34.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:56:34.397+0000 7f1b50a00700 1 -- 172.21.15.45:0/359825857 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b4c1a7090 con 0x7f1b4c074ec0 2024-04-16T16:56:34.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:56:34.398+0000 7f1b50a00700 1 --2- 172.21.15.45:0/359825857 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1b4c074ec0 0x7f1b4c19a510 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f1b40009fd0 tx=0x7f1b4000bde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T16:56:34.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:56:34.398+0000 7f1b49600700 1 -- 172.21.15.45:0/359825857 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b40026070 con 0x7f1b4c074ec0 2024-04-16T16:56:34.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:56:34.398+0000 7f1b49600700 1 -- 172.21.15.45:0/359825857 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1b40005da0 con 0x7f1b4c074ec0 2024-04-16T16:56:34.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:56:34.398+0000 7f1b49600700 1 -- 172.21.15.45:0/359825857 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b400038e0 con 0x7f1b4c074ec0 2024-04-16T16:56:34.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:56:34.398+0000 7f1b53400700 1 -- 172.21.15.45:0/359825857 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b4c1a7260 con 0x7f1b4c074ec0 2024-04-16T16:56:34.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:56:34.398+0000 7f1b53400700 1 -- 172.21.15.45:0/359825857 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b4c1a7740 con 0x7f1b4c074ec0 2024-04-16T16:56:34.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:56:34.399+0000 7f1b49600700 1 -- 172.21.15.45:0/359825857 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f1b40029070 con 0x7f1b4c074ec0 2024-04-16T16:56:34.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:56:34.399+0000 7f1b53400700 1 -- 172.21.15.45:0/359825857 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b4c064150 con 0x7f1b4c074ec0 2024-04-16T16:56:34.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:56:34.400+0000 7f1b49600700 1 --2- 172.21.15.45:0/359825857 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f1b34039ae0 0x7f1b3403bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:56:34.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:56:34.400+0000 7f1b49600700 1 -- 172.21.15.45:0/359825857 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f1b4004d4e0 con 0x7f1b4c074ec0 2024-04-16T16:56:34.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:56:34.400+0000 7f1b4be00700 1 --2- 172.21.15.45:0/359825857 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f1b34039ae0 0x7f1b3403bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:56:34.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:56:34.401+0000 7f1b4be00700 1 --2- 172.21.15.45:0/359825857 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f1b34039ae0 0x7f1b3403bfa0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f1b38009ad0 tx=0x7f1b38009210 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T16:56:34.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:56:34.406+0000 7f1b49600700 1 -- 172.21.15.45:0/359825857 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1b40013070 con 0x7f1b4c074ec0 2024-04-16T16:56:34.428 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:33 smithi104 ceph-mon[36457]: mon.b calling monitor election 2024-04-16T16:56:34.428 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:33 smithi104 ceph-mon[36457]: Updating smithi045:/etc/ceph/ceph.conf 2024-04-16T16:56:34.428 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:33 smithi104 ceph-mon[36457]: Updating smithi104:/etc/ceph/ceph.conf 2024-04-16T16:56:34.428 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:33 smithi104 ceph-mon[36457]: Updating smithi117:/etc/ceph/ceph.conf 2024-04-16T16:56:34.428 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:33 smithi104 ceph-mon[36457]: mon.b calling monitor election 2024-04-16T16:56:34.429 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:33 smithi104 ceph-mon[36457]: mon.c calling monitor election 2024-04-16T16:56:34.429 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:33 smithi104 ceph-mon[36457]: mon.a calling monitor election 2024-04-16T16:56:34.429 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:33 smithi104 ceph-mon[36457]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-16T16:56:34.429 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:33 smithi104 ceph-mon[36457]: monmap epoch 3 2024-04-16T16:56:34.429 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:33 smithi104 ceph-mon[36457]: fsid b1fecd38-fc11-11ee-bc90-c7b262605968 2024-04-16T16:56:34.429 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:33 smithi104 ceph-mon[36457]: last_changed 2024-04-16T16:56:26.187742+0000 2024-04-16T16:56:34.429 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:33 smithi104 ceph-mon[36457]: created 2024-04-16T16:53:58.140859+0000 2024-04-16T16:56:34.429 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:33 smithi104 ceph-mon[36457]: min_mon_release 19 (squid) 2024-04-16T16:56:34.429 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:33 smithi104 ceph-mon[36457]: election_strategy: 1 2024-04-16T16:56:34.429 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:33 smithi104 ceph-mon[36457]: 0: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.a 2024-04-16T16:56:34.429 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:33 smithi104 ceph-mon[36457]: 1: [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] mon.c 2024-04-16T16:56:34.429 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:33 smithi104 ceph-mon[36457]: 2: [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] mon.b 2024-04-16T16:56:34.429 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:33 smithi104 ceph-mon[36457]: fsmap 2024-04-16T16:56:34.429 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:33 smithi104 ceph-mon[36457]: osdmap e4: 0 total, 0 up, 0 in 2024-04-16T16:56:34.429 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:33 smithi104 ceph-mon[36457]: mgrmap e13: x(active, since 73s) 2024-04-16T16:56:34.430 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:33 smithi104 ceph-mon[36457]: overall HEALTH_OK 2024-04-16T16:56:34.430 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:33 smithi104 ceph-mon[36457]: Updating smithi104:/var/lib/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/config/ceph.conf 2024-04-16T16:56:34.430 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:33 smithi104 ceph-mon[36457]: Updating smithi117:/var/lib/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/config/ceph.conf 2024-04-16T16:56:34.430 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:33 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:56:34.430 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:33 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:56:34.430 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:33 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:56:34.430 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:33 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:56:34.430 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:33 smithi104 ceph-mon[36457]: Updating smithi045:/var/lib/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/config/ceph.conf 2024-04-16T16:56:34.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:56:34.646+0000 7f1b53400700 1 -- 172.21.15.45:0/359825857 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f1b4c1a79a0 con 0x7f1b4c074ec0 2024-04-16T16:56:34.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:56:34.648+0000 7f1b49600700 1 -- 172.21.15.45:0/359825857 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) ==== 76+0+275 (secure 0 0 0) 0x7f1b40030090 con 0x7f1b4c074ec0 2024-04-16T16:56:34.650 INFO:teuthology.orchestra.run.smithi045.stdout:# minimal ceph.conf for b1fecd38-fc11-11ee-bc90-c7b262605968 2024-04-16T16:56:34.650 INFO:teuthology.orchestra.run.smithi045.stdout:[global] 2024-04-16T16:56:34.650 INFO:teuthology.orchestra.run.smithi045.stdout: fsid = b1fecd38-fc11-11ee-bc90-c7b262605968 2024-04-16T16:56:34.650 INFO:teuthology.orchestra.run.smithi045.stdout: mon_host = [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] 2024-04-16T16:56:34.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:56:34.650+0000 7f1b53400700 1 -- 172.21.15.45:0/359825857 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f1b34039ae0 msgr2=0x7f1b3403bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:56:34.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:56:34.650+0000 7f1b53400700 1 --2- 172.21.15.45:0/359825857 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f1b34039ae0 0x7f1b3403bfa0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f1b38009ad0 tx=0x7f1b38009210 comp rx=0 tx=0).stop 2024-04-16T16:56:34.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:56:34.650+0000 7f1b53400700 1 -- 172.21.15.45:0/359825857 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1b4c074ec0 msgr2=0x7f1b4c19a510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:56:34.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:56:34.650+0000 7f1b53400700 1 --2- 172.21.15.45:0/359825857 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1b4c074ec0 0x7f1b4c19a510 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f1b40009fd0 tx=0x7f1b4000bde0 comp rx=0 tx=0).stop 2024-04-16T16:56:34.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:56:34.651+0000 7f1b53400700 1 -- 172.21.15.45:0/359825857 shutdown_connections 2024-04-16T16:56:34.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:56:34.651+0000 7f1b53400700 1 --2- 172.21.15.45:0/359825857 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1b4c19f320 0x7f1b4c19f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:56:34.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:56:34.651+0000 7f1b53400700 1 --2- 172.21.15.45:0/359825857 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1b4c074ec0 0x7f1b4c19a510 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:56:34.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:56:34.651+0000 7f1b53400700 1 --2- 172.21.15.45:0/359825857 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f1b34039ae0 0x7f1b3403bfa0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:56:34.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:56:34.651+0000 7f1b53400700 1 --2- 172.21.15.45:0/359825857 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1b4c19aa50 0x7f1b4c19ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:56:34.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:56:34.651+0000 7f1b53400700 1 -- 172.21.15.45:0/359825857 >> 172.21.15.45:0/359825857 conn(0x7f1b4c0fe030 msgr2=0x7f1b4c106940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T16:56:34.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:56:34.651+0000 7f1b53400700 1 -- 172.21.15.45:0/359825857 shutdown_connections 2024-04-16T16:56:34.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:56:34.651+0000 7f1b53400700 1 -- 172.21.15.45:0/359825857 wait complete. 2024-04-16T16:56:35.306 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:34 smithi117 ceph-mon[36730]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:56:35.306 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:34 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-16T16:56:35.306 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:34 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:56:35.306 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:34 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:56:35.306 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:34 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:56:35.307 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:34 smithi117 ceph-mon[36730]: from='client.? 172.21.15.45:0/359825857' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:56:35.307 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:34 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:56:35.307 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:34 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:56:35.307 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:34 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:56:35.307 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:34 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:56:35.307 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:34 smithi117 ceph-mon[36730]: Reconfiguring mon.a (unknown last config time)... 2024-04-16T16:56:35.307 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:34 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-16T16:56:35.307 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:34 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-16T16:56:35.307 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:34 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:56:35.307 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:34 smithi117 ceph-mon[36730]: Reconfiguring daemon mon.a on smithi045 2024-04-16T16:56:35.428 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:34 smithi104 ceph-mon[36457]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:56:35.429 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:34 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-16T16:56:35.429 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:34 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:56:35.429 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:34 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:56:35.429 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:34 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:56:35.429 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:34 smithi104 ceph-mon[36457]: from='client.? 172.21.15.45:0/359825857' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:56:35.429 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:34 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:56:35.429 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:34 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:56:35.429 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:34 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:56:35.429 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:34 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:56:35.429 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:34 smithi104 ceph-mon[36457]: Reconfiguring mon.a (unknown last config time)... 2024-04-16T16:56:35.429 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:34 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-16T16:56:35.429 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:34 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-16T16:56:35.429 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:34 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:56:35.429 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:34 smithi104 ceph-mon[36457]: Reconfiguring daemon mon.a on smithi045 2024-04-16T16:56:35.445 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:56:34 smithi045 ceph-mon[31676]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:56:35.445 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:56:34 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-16T16:56:35.445 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:56:34 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:56:35.445 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:56:34 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:56:35.445 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:56:34 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:56:35.445 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:56:34 smithi045 ceph-mon[31676]: from='client.? 172.21.15.45:0/359825857' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:56:35.445 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:56:34 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:56:35.445 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:56:34 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:56:35.445 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:56:34 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:56:35.445 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:56:34 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:56:35.445 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:56:34 smithi045 ceph-mon[31676]: Reconfiguring mon.a (unknown last config time)... 2024-04-16T16:56:35.445 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:56:34 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-16T16:56:35.445 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:56:34 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-16T16:56:35.446 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:56:34 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:56:35.446 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:56:34 smithi045 ceph-mon[31676]: Reconfiguring daemon mon.a on smithi045 2024-04-16T16:56:35.499 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-04-16T16:56:35.499 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-04-16T16:56:35.499 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd of=/etc/ceph/ceph.conf 2024-04-16T16:56:35.580 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-04-16T16:56:35.580 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-16T16:56:35.715 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2024-04-16T16:56:35.716 DEBUG:teuthology.orchestra.run.smithi104:> sudo dd of=/etc/ceph/ceph.conf 2024-04-16T16:56:35.750 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2024-04-16T16:56:35.750 DEBUG:teuthology.orchestra.run.smithi104:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-16T16:56:35.820 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-04-16T16:56:35.820 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd of=/etc/ceph/ceph.conf 2024-04-16T16:56:35.850 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-04-16T16:56:35.850 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-16T16:56:35.916 INFO:tasks.cephadm:Adding mgr.x on smithi045 2024-04-16T16:56:35.916 INFO:tasks.cephadm:Adding mgr.y on smithi104 2024-04-16T16:56:35.916 INFO:tasks.cephadm:Adding mgr.z on smithi117 2024-04-16T16:56:35.917 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b1fecd38-fc11-11ee-bc90-c7b262605968 -- ceph orch apply mgr '3;smithi045=x;smithi104=y;smithi117=z' 2024-04-16T16:56:36.186 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/mon.c/config 2024-04-16T16:56:36.279 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:36 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:56:36.428 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:36 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:56:36.438 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:56:36 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:56:37.369 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:37 smithi117 ceph-mon[36730]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:56:37.369 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:37 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:56:37.369 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:37 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:56:37.370 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:37 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-16T16:56:37.370 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:37 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-16T16:56:37.370 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:37 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:56:37.428 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:37 smithi104 ceph-mon[36457]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:56:37.428 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:37 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:56:37.428 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:37 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:56:37.428 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:37 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-16T16:56:37.429 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:37 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-16T16:56:37.429 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:37 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:56:37.478 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:56:37 smithi045 ceph-mon[31676]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:56:37.478 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:56:37 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:56:37.478 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:56:37 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:56:37.478 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:56:37 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-16T16:56:37.478 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:56:37 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-16T16:56:37.479 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:56:37 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:56:37.575 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:56:37.571+0000 7f8ebe70a700 1 -- 172.21.15.117:0/3244141558 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8eb8102680 msgr2=0x7f8eb8102aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:56:37.575 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:56:37.571+0000 7f8ebe70a700 1 --2- 172.21.15.117:0/3244141558 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8eb8102680 0x7f8eb8102aa0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f8ea8009910 tx=0x7f8ea8009c20 comp rx=0 tx=0).stop 2024-04-16T16:56:37.575 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:56:37.572+0000 7f8ebe70a700 1 -- 172.21.15.117:0/3244141558 shutdown_connections 2024-04-16T16:56:37.576 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:56:37.572+0000 7f8ebe70a700 1 --2- 172.21.15.117:0/3244141558 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8eb8102680 0x7f8eb8102aa0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:56:37.576 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:56:37.572+0000 7f8ebe70a700 1 -- 172.21.15.117:0/3244141558 >> 172.21.15.117:0/3244141558 conn(0x7f8eb80fdc20 msgr2=0x7f8eb8100060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T16:56:37.576 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:56:37.572+0000 7f8ebe70a700 1 -- 172.21.15.117:0/3244141558 shutdown_connections 2024-04-16T16:56:37.576 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:56:37.572+0000 7f8ebe70a700 1 -- 172.21.15.117:0/3244141558 wait complete. 2024-04-16T16:56:37.576 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:56:37.572+0000 7f8ebe70a700 1 Processor -- start 2024-04-16T16:56:37.576 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:56:37.572+0000 7f8ebe70a700 1 -- start start 2024-04-16T16:56:37.576 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:56:37.572+0000 7f8ebe70a700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8eb8102680 0x7f8eb8076890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:56:37.576 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:56:37.572+0000 7f8ebe70a700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8eb8074ee0 0x7f8eb8075360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:56:37.576 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:56:37.572+0000 7f8ebe70a700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8eb80758a0 0x7f8eb81a1f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:56:37.576 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:56:37.572+0000 7f8ebe70a700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f8eb81086c0 con 0x7f8eb8102680 2024-04-16T16:56:37.576 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:56:37.572+0000 7f8ebe70a700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f8eb8108540 con 0x7f8eb80758a0 2024-04-16T16:56:37.576 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:56:37.572+0000 7f8ebe70a700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f8eb8108840 con 0x7f8eb8074ee0 2024-04-16T16:56:37.576 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:56:37.573+0000 7f8eb77fe700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8eb8074ee0 0x7f8eb8075360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:56:37.577 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:56:37.573+0000 7f8eb77fe700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8eb8074ee0 0x7f8eb8075360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:38918/0 (socket says 172.21.15.117:38918) 2024-04-16T16:56:37.577 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:56:37.573+0000 7f8eb77fe700 1 -- 172.21.15.117:0/1958657797 learned_addr learned my addr 172.21.15.117:0/1958657797 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-16T16:56:37.577 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:56:37.573+0000 7f8ebcca7700 1 --2- 172.21.15.117:0/1958657797 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8eb80758a0 0x7f8eb81a1f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:56:37.577 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:56:37.573+0000 7f8eb7fff700 1 --2- 172.21.15.117:0/1958657797 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8eb8102680 0x7f8eb8076890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:56:37.577 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:56:37.573+0000 7f8eb77fe700 1 -- 172.21.15.117:0/1958657797 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8eb80758a0 msgr2=0x7f8eb81a1f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:56:37.577 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:56:37.573+0000 7f8eb77fe700 1 --2- 172.21.15.117:0/1958657797 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8eb80758a0 0x7f8eb81a1f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:56:37.577 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:56:37.573+0000 7f8eb77fe700 1 -- 172.21.15.117:0/1958657797 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8eb8102680 msgr2=0x7f8eb8076890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:56:37.577 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:56:37.573+0000 7f8eb77fe700 1 --2- 172.21.15.117:0/1958657797 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8eb8102680 0x7f8eb8076890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:56:37.577 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:56:37.573+0000 7f8eb77fe700 1 -- 172.21.15.117:0/1958657797 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8eb81a24d0 con 0x7f8eb8074ee0 2024-04-16T16:56:37.577 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:56:37.573+0000 7f8ebcca7700 1 --2- 172.21.15.117:0/1958657797 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8eb80758a0 0x7f8eb81a1f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T16:56:37.577 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:56:37.573+0000 7f8eb7fff700 1 --2- 172.21.15.117:0/1958657797 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8eb8102680 0x7f8eb8076890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T16:56:37.577 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:56:37.573+0000 7f8eb77fe700 1 --2- 172.21.15.117:0/1958657797 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8eb8074ee0 0x7f8eb8075360 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f8eac00b780 tx=0x7f8eac00bb40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T16:56:37.577 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:56:37.573+0000 7f8eb57fa700 1 -- 172.21.15.117:0/1958657797 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8eac004230 con 0x7f8eb8074ee0 2024-04-16T16:56:37.577 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:56:37.573+0000 7f8eb57fa700 1 -- 172.21.15.117:0/1958657797 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8eac0043d0 con 0x7f8eb8074ee0 2024-04-16T16:56:37.578 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:56:37.573+0000 7f8ebe70a700 1 -- 172.21.15.117:0/1958657797 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8eb81a27c0 con 0x7f8eb8074ee0 2024-04-16T16:56:37.578 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:56:37.573+0000 7f8eb57fa700 1 -- 172.21.15.117:0/1958657797 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8eac011c60 con 0x7f8eb8074ee0 2024-04-16T16:56:37.578 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:56:37.574+0000 7f8ebe70a700 1 -- 172.21.15.117:0/1958657797 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8eb81a2d80 con 0x7f8eb8074ee0 2024-04-16T16:56:37.578 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:56:37.574+0000 7f8eb57fa700 1 -- 172.21.15.117:0/1958657797 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f8eac01a400 con 0x7f8eb8074ee0 2024-04-16T16:56:37.579 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:56:37.574+0000 7f8ebe70a700 1 -- 172.21.15.117:0/1958657797 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8eb8064150 con 0x7f8eb8074ee0 2024-04-16T16:56:37.579 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:56:37.575+0000 7f8eb57fa700 1 --2- 172.21.15.117:0/1958657797 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f8ea0039ae0 0x7f8ea003bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:56:37.579 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:56:37.575+0000 7f8eb57fa700 1 -- 172.21.15.117:0/1958657797 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f8eac04c6a0 con 0x7f8eb8074ee0 2024-04-16T16:56:37.579 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:56:37.575+0000 7f8eb7fff700 1 --2- 172.21.15.117:0/1958657797 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f8ea0039ae0 0x7f8ea003bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:56:37.579 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:56:37.576+0000 7f8eb7fff700 1 --2- 172.21.15.117:0/1958657797 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f8ea0039ae0 0x7f8ea003bfa0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f8ea8005330 tx=0x7f8ea8005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T16:56:37.581 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:56:37.580+0000 7f8eb57fa700 1 -- 172.21.15.117:0/1958657797 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8eac018350 con 0x7f8eb8074ee0 2024-04-16T16:56:37.789 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:56:37.788+0000 7f8ebe70a700 1 -- 172.21.15.117:0/1958657797 --> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi045=x;smithi104=y;smithi117=z", "target": ["mon-mgr", ""]}) -- 0x7f8eb8062fc0 con 0x7f8ea0039ae0 2024-04-16T16:56:37.803 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:56:37.802+0000 7f8eb57fa700 1 -- 172.21.15.117:0/1958657797 <== mgr.14152 v2:172.21.15.45:6800/268030699 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f8eb8062fc0 con 0x7f8ea0039ae0 2024-04-16T16:56:37.803 INFO:teuthology.orchestra.run.smithi117.stdout:Scheduled mgr update... 2024-04-16T16:56:37.805 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:56:37.804+0000 7f8ebe70a700 1 -- 172.21.15.117:0/1958657797 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f8ea0039ae0 msgr2=0x7f8ea003bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:56:37.805 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:56:37.804+0000 7f8ebe70a700 1 --2- 172.21.15.117:0/1958657797 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f8ea0039ae0 0x7f8ea003bfa0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f8ea8005330 tx=0x7f8ea8005fb0 comp rx=0 tx=0).stop 2024-04-16T16:56:37.805 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:56:37.804+0000 7f8ebe70a700 1 -- 172.21.15.117:0/1958657797 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8eb8074ee0 msgr2=0x7f8eb8075360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:56:37.806 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:56:37.804+0000 7f8ebe70a700 1 --2- 172.21.15.117:0/1958657797 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8eb8074ee0 0x7f8eb8075360 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f8eac00b780 tx=0x7f8eac00bb40 comp rx=0 tx=0).stop 2024-04-16T16:56:37.806 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:56:37.804+0000 7f8ebe70a700 1 -- 172.21.15.117:0/1958657797 shutdown_connections 2024-04-16T16:56:37.806 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:56:37.804+0000 7f8ebe70a700 1 --2- 172.21.15.117:0/1958657797 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8eb80758a0 0x7f8eb81a1f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:56:37.806 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:56:37.804+0000 7f8ebe70a700 1 --2- 172.21.15.117:0/1958657797 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8eb8102680 0x7f8eb8076890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:56:37.806 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:56:37.804+0000 7f8ebe70a700 1 --2- 172.21.15.117:0/1958657797 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f8ea0039ae0 0x7f8ea003bfa0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:56:37.806 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:56:37.804+0000 7f8ebe70a700 1 --2- 172.21.15.117:0/1958657797 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8eb8074ee0 0x7f8eb8075360 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:56:37.806 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:56:37.804+0000 7f8ebe70a700 1 -- 172.21.15.117:0/1958657797 >> 172.21.15.117:0/1958657797 conn(0x7f8eb80fdc20 msgr2=0x7f8eb80fe900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T16:56:37.806 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:56:37.804+0000 7f8ebe70a700 1 -- 172.21.15.117:0/1958657797 shutdown_connections 2024-04-16T16:56:37.806 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:56:37.804+0000 7f8ebe70a700 1 -- 172.21.15.117:0/1958657797 wait complete. 2024-04-16T16:56:38.347 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:38 smithi117 ceph-mon[36730]: Reconfiguring mon.b (monmap changed)... 2024-04-16T16:56:38.347 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:38 smithi117 ceph-mon[36730]: Reconfiguring daemon mon.b on smithi104 2024-04-16T16:56:38.348 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:38 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:56:38.348 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:38 smithi104 ceph-mon[36457]: Reconfiguring mon.b (monmap changed)... 2024-04-16T16:56:38.348 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:38 smithi104 ceph-mon[36457]: Reconfiguring daemon mon.b on smithi104 2024-04-16T16:56:38.348 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:38 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:56:38.349 DEBUG:teuthology.orchestra.run.smithi104:mgr.y> sudo journalctl -f -n 0 -u ceph-b1fecd38-fc11-11ee-bc90-c7b262605968@mgr.y.service 2024-04-16T16:56:38.351 DEBUG:teuthology.orchestra.run.smithi117:mgr.z> sudo journalctl -f -n 0 -u ceph-b1fecd38-fc11-11ee-bc90-c7b262605968@mgr.z.service 2024-04-16T16:56:38.353 INFO:tasks.cephadm:Deploying OSDs... 2024-04-16T16:56:38.353 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-04-16T16:56:38.353 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/scratch_devs of=/dev/stdout 2024-04-16T16:56:38.415 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-16T16:56:38.415 DEBUG:teuthology.orchestra.run.smithi045:> stat /dev/vg_nvme/lv_1 2024-04-16T16:56:38.470 INFO:teuthology.orchestra.run.smithi045.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-16T16:56:38.470 INFO:teuthology.orchestra.run.smithi045.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-16T16:56:38.470 INFO:teuthology.orchestra.run.smithi045.stdout:Device: 6h/6d Inode: 632 Links: 1 2024-04-16T16:56:38.470 INFO:teuthology.orchestra.run.smithi045.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-16T16:56:38.470 INFO:teuthology.orchestra.run.smithi045.stdout:Context: system_u:object_r:device_t:s0 2024-04-16T16:56:38.470 INFO:teuthology.orchestra.run.smithi045.stdout:Access: 2024-04-16 16:55:37.213653479 +0000 2024-04-16T16:56:38.470 INFO:teuthology.orchestra.run.smithi045.stdout:Modify: 2024-04-16 16:49:11.019439727 +0000 2024-04-16T16:56:38.471 INFO:teuthology.orchestra.run.smithi045.stdout:Change: 2024-04-16 16:49:11.019439727 +0000 2024-04-16T16:56:38.471 INFO:teuthology.orchestra.run.smithi045.stdout: Birth: 2024-04-16 16:49:11.019439727 +0000 2024-04-16T16:56:38.471 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-16T16:56:38.477 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:56:38 smithi045 ceph-mon[31676]: Reconfiguring mon.b (monmap changed)... 2024-04-16T16:56:38.478 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:56:38 smithi045 ceph-mon[31676]: Reconfiguring daemon mon.b on smithi104 2024-04-16T16:56:38.478 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:56:38 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:56:38.552 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-16T16:56:38.553 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-16T16:56:38.553 INFO:teuthology.orchestra.run.smithi045.stderr:512 bytes copied, 0.000288789 s, 1.8 MB/s 2024-04-16T16:56:38.557 DEBUG:teuthology.orchestra.run.smithi045:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-16T16:56:38.659 DEBUG:teuthology.orchestra.run.smithi045:> stat /dev/vg_nvme/lv_2 2024-04-16T16:56:38.715 INFO:teuthology.orchestra.run.smithi045.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-16T16:56:38.715 INFO:teuthology.orchestra.run.smithi045.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-16T16:56:38.715 INFO:teuthology.orchestra.run.smithi045.stdout:Device: 6h/6d Inode: 650 Links: 1 2024-04-16T16:56:38.716 INFO:teuthology.orchestra.run.smithi045.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-16T16:56:38.716 INFO:teuthology.orchestra.run.smithi045.stdout:Context: system_u:object_r:device_t:s0 2024-04-16T16:56:38.716 INFO:teuthology.orchestra.run.smithi045.stdout:Access: 2024-04-16 16:55:37.271652588 +0000 2024-04-16T16:56:38.716 INFO:teuthology.orchestra.run.smithi045.stdout:Modify: 2024-04-16 16:49:11.641430180 +0000 2024-04-16T16:56:38.716 INFO:teuthology.orchestra.run.smithi045.stdout:Change: 2024-04-16 16:49:11.641430180 +0000 2024-04-16T16:56:38.716 INFO:teuthology.orchestra.run.smithi045.stdout: Birth: 2024-04-16 16:49:11.641430180 +0000 2024-04-16T16:56:38.716 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-16T16:56:38.810 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-16T16:56:38.810 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-16T16:56:38.810 INFO:teuthology.orchestra.run.smithi045.stderr:512 bytes copied, 0.000278636 s, 1.8 MB/s 2024-04-16T16:56:38.814 DEBUG:teuthology.orchestra.run.smithi045:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-16T16:56:38.871 DEBUG:teuthology.orchestra.run.smithi045:> stat /dev/vg_nvme/lv_3 2024-04-16T16:56:38.923 INFO:teuthology.orchestra.run.smithi045.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-16T16:56:38.923 INFO:teuthology.orchestra.run.smithi045.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-16T16:56:38.923 INFO:teuthology.orchestra.run.smithi045.stdout:Device: 6h/6d Inode: 670 Links: 1 2024-04-16T16:56:38.923 INFO:teuthology.orchestra.run.smithi045.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-16T16:56:38.923 INFO:teuthology.orchestra.run.smithi045.stdout:Context: system_u:object_r:device_t:s0 2024-04-16T16:56:38.923 INFO:teuthology.orchestra.run.smithi045.stdout:Access: 2024-04-16 16:55:37.323651789 +0000 2024-04-16T16:56:38.923 INFO:teuthology.orchestra.run.smithi045.stdout:Modify: 2024-04-16 16:49:12.267420571 +0000 2024-04-16T16:56:38.924 INFO:teuthology.orchestra.run.smithi045.stdout:Change: 2024-04-16 16:49:12.267420571 +0000 2024-04-16T16:56:38.924 INFO:teuthology.orchestra.run.smithi045.stdout: Birth: 2024-04-16 16:49:12.267420571 +0000 2024-04-16T16:56:38.924 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-16T16:56:39.016 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-16T16:56:39.016 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-16T16:56:39.016 INFO:teuthology.orchestra.run.smithi045.stderr:512 bytes copied, 0.000275415 s, 1.9 MB/s 2024-04-16T16:56:39.019 DEBUG:teuthology.orchestra.run.smithi045:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-16T16:56:39.079 DEBUG:teuthology.orchestra.run.smithi045:> stat /dev/vg_nvme/lv_4 2024-04-16T16:56:39.135 INFO:teuthology.orchestra.run.smithi045.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-16T16:56:39.135 INFO:teuthology.orchestra.run.smithi045.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-16T16:56:39.135 INFO:teuthology.orchestra.run.smithi045.stdout:Device: 6h/6d Inode: 689 Links: 1 2024-04-16T16:56:39.135 INFO:teuthology.orchestra.run.smithi045.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-16T16:56:39.135 INFO:teuthology.orchestra.run.smithi045.stdout:Context: system_u:object_r:device_t:s0 2024-04-16T16:56:39.135 INFO:teuthology.orchestra.run.smithi045.stdout:Access: 2024-04-16 16:55:37.378650944 +0000 2024-04-16T16:56:39.135 INFO:teuthology.orchestra.run.smithi045.stdout:Modify: 2024-04-16 16:49:12.824412022 +0000 2024-04-16T16:56:39.135 INFO:teuthology.orchestra.run.smithi045.stdout:Change: 2024-04-16 16:49:12.824412022 +0000 2024-04-16T16:56:39.136 INFO:teuthology.orchestra.run.smithi045.stdout: Birth: 2024-04-16 16:49:12.824412022 +0000 2024-04-16T16:56:39.136 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-16T16:56:39.224 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-16T16:56:39.225 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-16T16:56:39.225 INFO:teuthology.orchestra.run.smithi045.stderr:512 bytes copied, 0.000272726 s, 1.9 MB/s 2024-04-16T16:56:39.229 DEBUG:teuthology.orchestra.run.smithi045:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-16T16:56:39.289 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2024-04-16T16:56:39.289 DEBUG:teuthology.orchestra.run.smithi104:> dd if=/scratch_devs of=/dev/stdout 2024-04-16T16:56:39.308 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-16T16:56:39.308 DEBUG:teuthology.orchestra.run.smithi104:> stat /dev/vg_nvme/lv_1 2024-04-16T16:56:39.365 INFO:teuthology.orchestra.run.smithi104.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-16T16:56:39.365 INFO:teuthology.orchestra.run.smithi104.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-16T16:56:39.365 INFO:teuthology.orchestra.run.smithi104.stdout:Device: 5h/5d Inode: 663 Links: 1 2024-04-16T16:56:39.365 INFO:teuthology.orchestra.run.smithi104.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-16T16:56:39.365 INFO:teuthology.orchestra.run.smithi104.stdout:Context: system_u:object_r:device_t:s0 2024-04-16T16:56:39.365 INFO:teuthology.orchestra.run.smithi104.stdout:Access: 2024-04-16 16:56:25.620700424 +0000 2024-04-16T16:56:39.365 INFO:teuthology.orchestra.run.smithi104.stdout:Modify: 2024-04-16 16:47:40.833598148 +0000 2024-04-16T16:56:39.365 INFO:teuthology.orchestra.run.smithi104.stdout:Change: 2024-04-16 16:47:40.833598148 +0000 2024-04-16T16:56:39.365 INFO:teuthology.orchestra.run.smithi104.stdout: Birth: 2024-04-16 16:47:40.833598148 +0000 2024-04-16T16:56:39.365 DEBUG:teuthology.orchestra.run.smithi104:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-16T16:56:39.433 INFO:teuthology.orchestra.run.smithi104.stderr:1+0 records in 2024-04-16T16:56:39.433 INFO:teuthology.orchestra.run.smithi104.stderr:1+0 records out 2024-04-16T16:56:39.433 INFO:teuthology.orchestra.run.smithi104.stderr:512 bytes copied, 0.000283517 s, 1.8 MB/s 2024-04-16T16:56:39.435 DEBUG:teuthology.orchestra.run.smithi104:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-16T16:56:39.478 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:56:39 smithi045 ceph-mon[31676]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi045=x;smithi104=y;smithi117=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T16:56:39.478 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:56:39 smithi045 ceph-mon[31676]: Saving service mgr spec with placement smithi045=x;smithi104=y;smithi117=z;count:3 2024-04-16T16:56:39.478 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:56:39 smithi045 ceph-mon[31676]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:56:39.478 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:56:39 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:56:39.479 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:56:39 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:56:39.479 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:56:39 smithi045 ceph-mon[31676]: Reconfiguring mon.c (monmap changed)... 2024-04-16T16:56:39.479 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:56:39 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-16T16:56:39.479 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:56:39 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-16T16:56:39.479 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:56:39 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:56:39.479 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:56:39 smithi045 ceph-mon[31676]: Reconfiguring daemon mon.c on smithi117 2024-04-16T16:56:39.492 DEBUG:teuthology.orchestra.run.smithi104:> stat /dev/vg_nvme/lv_2 2024-04-16T16:56:39.547 INFO:teuthology.orchestra.run.smithi104.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-16T16:56:39.548 INFO:teuthology.orchestra.run.smithi104.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-16T16:56:39.548 INFO:teuthology.orchestra.run.smithi104.stdout:Device: 5h/5d Inode: 683 Links: 1 2024-04-16T16:56:39.548 INFO:teuthology.orchestra.run.smithi104.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-16T16:56:39.548 INFO:teuthology.orchestra.run.smithi104.stdout:Context: system_u:object_r:device_t:s0 2024-04-16T16:56:39.548 INFO:teuthology.orchestra.run.smithi104.stdout:Access: 2024-04-16 16:56:25.620700424 +0000 2024-04-16T16:56:39.548 INFO:teuthology.orchestra.run.smithi104.stdout:Modify: 2024-04-16 16:47:41.179593637 +0000 2024-04-16T16:56:39.548 INFO:teuthology.orchestra.run.smithi104.stdout:Change: 2024-04-16 16:47:41.179593637 +0000 2024-04-16T16:56:39.548 INFO:teuthology.orchestra.run.smithi104.stdout: Birth: 2024-04-16 16:47:41.179593637 +0000 2024-04-16T16:56:39.549 DEBUG:teuthology.orchestra.run.smithi104:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-16T16:56:39.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:39 smithi117 ceph-mon[36730]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi045=x;smithi104=y;smithi117=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T16:56:39.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:39 smithi117 ceph-mon[36730]: Saving service mgr spec with placement smithi045=x;smithi104=y;smithi117=z;count:3 2024-04-16T16:56:39.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:39 smithi117 ceph-mon[36730]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:56:39.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:39 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:56:39.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:39 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:56:39.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:39 smithi117 ceph-mon[36730]: Reconfiguring mon.c (monmap changed)... 2024-04-16T16:56:39.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:39 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-16T16:56:39.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:39 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-16T16:56:39.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:39 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:56:39.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:39 smithi117 ceph-mon[36730]: Reconfiguring daemon mon.c on smithi117 2024-04-16T16:56:39.614 INFO:teuthology.orchestra.run.smithi104.stderr:1+0 records in 2024-04-16T16:56:39.615 INFO:teuthology.orchestra.run.smithi104.stderr:1+0 records out 2024-04-16T16:56:39.615 INFO:teuthology.orchestra.run.smithi104.stderr:512 bytes copied, 0.000205966 s, 2.5 MB/s 2024-04-16T16:56:39.616 DEBUG:teuthology.orchestra.run.smithi104:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-16T16:56:39.673 DEBUG:teuthology.orchestra.run.smithi104:> stat /dev/vg_nvme/lv_3 2024-04-16T16:56:39.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:39 smithi104 ceph-mon[36457]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi045=x;smithi104=y;smithi117=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T16:56:39.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:39 smithi104 ceph-mon[36457]: Saving service mgr spec with placement smithi045=x;smithi104=y;smithi117=z;count:3 2024-04-16T16:56:39.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:39 smithi104 ceph-mon[36457]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:56:39.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:39 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:56:39.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:39 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:56:39.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:39 smithi104 ceph-mon[36457]: Reconfiguring mon.c (monmap changed)... 2024-04-16T16:56:39.679 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:39 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-16T16:56:39.679 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:39 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-16T16:56:39.679 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:39 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:56:39.679 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:39 smithi104 ceph-mon[36457]: Reconfiguring daemon mon.c on smithi117 2024-04-16T16:56:39.693 INFO:teuthology.orchestra.run.smithi104.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-16T16:56:39.693 INFO:teuthology.orchestra.run.smithi104.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-16T16:56:39.693 INFO:teuthology.orchestra.run.smithi104.stdout:Device: 5h/5d Inode: 699 Links: 1 2024-04-16T16:56:39.693 INFO:teuthology.orchestra.run.smithi104.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-16T16:56:39.693 INFO:teuthology.orchestra.run.smithi104.stdout:Context: system_u:object_r:device_t:s0 2024-04-16T16:56:39.693 INFO:teuthology.orchestra.run.smithi104.stdout:Access: 2024-04-16 16:56:25.620700424 +0000 2024-04-16T16:56:39.694 INFO:teuthology.orchestra.run.smithi104.stdout:Modify: 2024-04-16 16:47:41.527589099 +0000 2024-04-16T16:56:39.694 INFO:teuthology.orchestra.run.smithi104.stdout:Change: 2024-04-16 16:47:41.527589099 +0000 2024-04-16T16:56:39.694 INFO:teuthology.orchestra.run.smithi104.stdout: Birth: 2024-04-16 16:47:41.527589099 +0000 2024-04-16T16:56:39.694 DEBUG:teuthology.orchestra.run.smithi104:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-16T16:56:39.759 INFO:teuthology.orchestra.run.smithi104.stderr:1+0 records in 2024-04-16T16:56:39.759 INFO:teuthology.orchestra.run.smithi104.stderr:1+0 records out 2024-04-16T16:56:39.759 INFO:teuthology.orchestra.run.smithi104.stderr:512 bytes copied, 0.000182122 s, 2.8 MB/s 2024-04-16T16:56:39.760 DEBUG:teuthology.orchestra.run.smithi104:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-16T16:56:39.816 DEBUG:teuthology.orchestra.run.smithi104:> stat /dev/vg_nvme/lv_4 2024-04-16T16:56:39.872 INFO:teuthology.orchestra.run.smithi104.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-16T16:56:39.872 INFO:teuthology.orchestra.run.smithi104.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-16T16:56:39.872 INFO:teuthology.orchestra.run.smithi104.stdout:Device: 5h/5d Inode: 719 Links: 1 2024-04-16T16:56:39.872 INFO:teuthology.orchestra.run.smithi104.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-16T16:56:39.873 INFO:teuthology.orchestra.run.smithi104.stdout:Context: system_u:object_r:device_t:s0 2024-04-16T16:56:39.873 INFO:teuthology.orchestra.run.smithi104.stdout:Access: 2024-04-16 16:56:25.621700411 +0000 2024-04-16T16:56:39.873 INFO:teuthology.orchestra.run.smithi104.stdout:Modify: 2024-04-16 16:47:41.883584457 +0000 2024-04-16T16:56:39.873 INFO:teuthology.orchestra.run.smithi104.stdout:Change: 2024-04-16 16:47:41.883584457 +0000 2024-04-16T16:56:39.873 INFO:teuthology.orchestra.run.smithi104.stdout: Birth: 2024-04-16 16:47:41.883584457 +0000 2024-04-16T16:56:39.873 DEBUG:teuthology.orchestra.run.smithi104:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-16T16:56:39.937 INFO:teuthology.orchestra.run.smithi104.stderr:1+0 records in 2024-04-16T16:56:39.937 INFO:teuthology.orchestra.run.smithi104.stderr:1+0 records out 2024-04-16T16:56:39.937 INFO:teuthology.orchestra.run.smithi104.stderr:512 bytes copied, 0.000187265 s, 2.7 MB/s 2024-04-16T16:56:39.939 DEBUG:teuthology.orchestra.run.smithi104:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-16T16:56:39.994 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-04-16T16:56:39.995 DEBUG:teuthology.orchestra.run.smithi117:> dd if=/scratch_devs of=/dev/stdout 2024-04-16T16:56:40.010 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-16T16:56:40.010 DEBUG:teuthology.orchestra.run.smithi117:> stat /dev/vg_nvme/lv_1 2024-04-16T16:56:40.065 INFO:teuthology.orchestra.run.smithi117.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-16T16:56:40.065 INFO:teuthology.orchestra.run.smithi117.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-16T16:56:40.065 INFO:teuthology.orchestra.run.smithi117.stdout:Device: 5h/5d Inode: 663 Links: 1 2024-04-16T16:56:40.065 INFO:teuthology.orchestra.run.smithi117.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-16T16:56:40.065 INFO:teuthology.orchestra.run.smithi117.stdout:Context: system_u:object_r:device_t:s0 2024-04-16T16:56:40.065 INFO:teuthology.orchestra.run.smithi117.stdout:Access: 2024-04-16 16:56:16.125225236 +0000 2024-04-16T16:56:40.066 INFO:teuthology.orchestra.run.smithi117.stdout:Modify: 2024-04-16 16:47:40.838821408 +0000 2024-04-16T16:56:40.066 INFO:teuthology.orchestra.run.smithi117.stdout:Change: 2024-04-16 16:47:40.838821408 +0000 2024-04-16T16:56:40.066 INFO:teuthology.orchestra.run.smithi117.stdout: Birth: 2024-04-16 16:47:40.838821408 +0000 2024-04-16T16:56:40.066 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-16T16:56:40.132 INFO:teuthology.orchestra.run.smithi117.stderr:1+0 records in 2024-04-16T16:56:40.132 INFO:teuthology.orchestra.run.smithi117.stderr:1+0 records out 2024-04-16T16:56:40.132 INFO:teuthology.orchestra.run.smithi117.stderr:512 bytes copied, 0.000257484 s, 2.0 MB/s 2024-04-16T16:56:40.133 DEBUG:teuthology.orchestra.run.smithi117:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-16T16:56:40.189 DEBUG:teuthology.orchestra.run.smithi117:> stat /dev/vg_nvme/lv_2 2024-04-16T16:56:40.244 INFO:teuthology.orchestra.run.smithi117.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-16T16:56:40.244 INFO:teuthology.orchestra.run.smithi117.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-16T16:56:40.244 INFO:teuthology.orchestra.run.smithi117.stdout:Device: 5h/5d Inode: 680 Links: 1 2024-04-16T16:56:40.244 INFO:teuthology.orchestra.run.smithi117.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-16T16:56:40.244 INFO:teuthology.orchestra.run.smithi117.stdout:Context: system_u:object_r:device_t:s0 2024-04-16T16:56:40.244 INFO:teuthology.orchestra.run.smithi117.stdout:Access: 2024-04-16 16:56:16.125225236 +0000 2024-04-16T16:56:40.244 INFO:teuthology.orchestra.run.smithi117.stdout:Modify: 2024-04-16 16:47:41.178815033 +0000 2024-04-16T16:56:40.244 INFO:teuthology.orchestra.run.smithi117.stdout:Change: 2024-04-16 16:47:41.178815033 +0000 2024-04-16T16:56:40.244 INFO:teuthology.orchestra.run.smithi117.stdout: Birth: 2024-04-16 16:47:41.178815033 +0000 2024-04-16T16:56:40.244 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-16T16:56:40.310 INFO:teuthology.orchestra.run.smithi117.stderr:1+0 records in 2024-04-16T16:56:40.310 INFO:teuthology.orchestra.run.smithi117.stderr:1+0 records out 2024-04-16T16:56:40.310 INFO:teuthology.orchestra.run.smithi117.stderr:512 bytes copied, 0.000200938 s, 2.5 MB/s 2024-04-16T16:56:40.311 DEBUG:teuthology.orchestra.run.smithi117:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-16T16:56:40.367 DEBUG:teuthology.orchestra.run.smithi117:> stat /dev/vg_nvme/lv_3 2024-04-16T16:56:40.422 INFO:teuthology.orchestra.run.smithi117.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-16T16:56:40.422 INFO:teuthology.orchestra.run.smithi117.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-16T16:56:40.422 INFO:teuthology.orchestra.run.smithi117.stdout:Device: 5h/5d Inode: 699 Links: 1 2024-04-16T16:56:40.423 INFO:teuthology.orchestra.run.smithi117.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-16T16:56:40.423 INFO:teuthology.orchestra.run.smithi117.stdout:Context: system_u:object_r:device_t:s0 2024-04-16T16:56:40.423 INFO:teuthology.orchestra.run.smithi117.stdout:Access: 2024-04-16 16:56:16.125225236 +0000 2024-04-16T16:56:40.423 INFO:teuthology.orchestra.run.smithi117.stdout:Modify: 2024-04-16 16:47:41.543808195 +0000 2024-04-16T16:56:40.423 INFO:teuthology.orchestra.run.smithi117.stdout:Change: 2024-04-16 16:47:41.543808195 +0000 2024-04-16T16:56:40.423 INFO:teuthology.orchestra.run.smithi117.stdout: Birth: 2024-04-16 16:47:41.543808195 +0000 2024-04-16T16:56:40.424 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-16T16:56:40.489 INFO:teuthology.orchestra.run.smithi117.stderr:1+0 records in 2024-04-16T16:56:40.489 INFO:teuthology.orchestra.run.smithi117.stderr:1+0 records out 2024-04-16T16:56:40.489 INFO:teuthology.orchestra.run.smithi117.stderr:512 bytes copied, 0.000192787 s, 2.7 MB/s 2024-04-16T16:56:40.491 DEBUG:teuthology.orchestra.run.smithi117:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-16T16:56:40.546 DEBUG:teuthology.orchestra.run.smithi117:> stat /dev/vg_nvme/lv_4 2024-04-16T16:56:40.600 INFO:teuthology.orchestra.run.smithi117.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-16T16:56:40.601 INFO:teuthology.orchestra.run.smithi117.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-16T16:56:40.601 INFO:teuthology.orchestra.run.smithi117.stdout:Device: 5h/5d Inode: 719 Links: 1 2024-04-16T16:56:40.601 INFO:teuthology.orchestra.run.smithi117.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-16T16:56:40.601 INFO:teuthology.orchestra.run.smithi117.stdout:Context: system_u:object_r:device_t:s0 2024-04-16T16:56:40.601 INFO:teuthology.orchestra.run.smithi117.stdout:Access: 2024-04-16 16:56:16.126225218 +0000 2024-04-16T16:56:40.601 INFO:teuthology.orchestra.run.smithi117.stdout:Modify: 2024-04-16 16:47:41.884801806 +0000 2024-04-16T16:56:40.601 INFO:teuthology.orchestra.run.smithi117.stdout:Change: 2024-04-16 16:47:41.884801806 +0000 2024-04-16T16:56:40.601 INFO:teuthology.orchestra.run.smithi117.stdout: Birth: 2024-04-16 16:47:41.884801806 +0000 2024-04-16T16:56:40.601 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-16T16:56:40.666 INFO:teuthology.orchestra.run.smithi117.stderr:1+0 records in 2024-04-16T16:56:40.666 INFO:teuthology.orchestra.run.smithi117.stderr:1+0 records out 2024-04-16T16:56:40.666 INFO:teuthology.orchestra.run.smithi117.stderr:512 bytes copied, 0.000215645 s, 2.4 MB/s 2024-04-16T16:56:40.668 DEBUG:teuthology.orchestra.run.smithi117:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-16T16:56:40.723 INFO:tasks.cephadm:Deploying osd.0 on smithi045 with /dev/vg_nvme/lv_4... 2024-04-16T16:56:40.723 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b1fecd38-fc11-11ee-bc90-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-16T16:56:41.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:40 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:56:41.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:40 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:56:41.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:40 smithi117 ceph-mon[36730]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:56:41.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:40 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T16:56:41.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:40 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:56:41.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:40 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T16:56:41.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:40 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:56:41.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:40 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-16T16:56:41.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:40 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-16T16:56:41.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:40 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-16T16:56:41.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:40 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:56:41.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:40 smithi117 ceph-mon[36730]: Deploying daemon mgr.z on smithi117 2024-04-16T16:56:41.174 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/mon.a/config 2024-04-16T16:56:41.228 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:56:40 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:56:41.228 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:56:40 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:56:41.228 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:56:40 smithi045 ceph-mon[31676]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:56:41.228 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:56:40 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T16:56:41.229 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:56:40 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:56:41.229 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:56:40 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T16:56:41.229 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:56:40 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:56:41.229 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:56:40 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-16T16:56:41.229 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:56:40 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-16T16:56:41.229 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:56:40 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-16T16:56:41.229 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:56:40 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:56:41.229 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:56:40 smithi045 ceph-mon[31676]: Deploying daemon mgr.z on smithi117 2024-04-16T16:56:41.428 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:40 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:56:41.428 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:40 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:56:41.429 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:40 smithi104 ceph-mon[36457]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:56:41.429 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:40 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T16:56:41.429 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:40 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:56:41.429 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:40 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T16:56:41.429 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:40 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:56:41.429 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:40 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-16T16:56:41.429 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:40 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-16T16:56:41.429 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:40 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-16T16:56:41.429 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:40 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:56:41.429 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:40 smithi104 ceph-mon[36457]: Deploying daemon mgr.z on smithi117 2024-04-16T16:56:42.215 INFO:journalctl@ceph.mgr.z.smithi117.stdout:Apr 16 16:56:42 smithi117 systemd[1]: Starting Ceph mgr.z for b1fecd38-fc11-11ee-bc90-c7b262605968... 2024-04-16T16:56:42.468 INFO:journalctl@ceph.mgr.z.smithi117.stdout:Apr 16 16:56:42 smithi117 podman[38577]: 2024-04-16 16:56:42.117445912 +0000 UTC m=+0.017039225 image pull e5ce10bdffbeeffffd1125f67b369e2151f0f17cdec854e93b6adf9321f65907 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2 2024-04-16T16:56:42.468 INFO:journalctl@ceph.mgr.z.smithi117.stdout:Apr 16 16:56:42 smithi117 podman[38577]: 2024-04-16 16:56:42.221978557 +0000 UTC m=+0.121571848 container create 08195d3a5a957145dbd9722387eac83b6d2e2e5aa30ef32e995d2df61803426a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-z, org.label-schema.schema-version=1.0, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240409, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True) 2024-04-16T16:56:42.468 INFO:journalctl@ceph.mgr.z.smithi117.stdout:Apr 16 16:56:42 smithi117 podman[38577]: 2024-04-16 16:56:42.467370119 +0000 UTC m=+0.366963416 container init 08195d3a5a957145dbd9722387eac83b6d2e2e5aa30ef32e995d2df61803426a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-z, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240409) 2024-04-16T16:56:42.806 INFO:journalctl@ceph.mgr.z.smithi117.stdout:Apr 16 16:56:42 smithi117 podman[38577]: 2024-04-16 16:56:42.470283976 +0000 UTC m=+0.369877273 container start 08195d3a5a957145dbd9722387eac83b6d2e2e5aa30ef32e995d2df61803426a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-z, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, org.label-schema.build-date=20240409, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, ceph=True) 2024-04-16T16:56:42.806 INFO:journalctl@ceph.mgr.z.smithi117.stdout:Apr 16 16:56:42 smithi117 ceph-mgr[38596]: -- 172.21.15.117:0/906118463 <== mon.1 v2:172.21.15.117:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55e2ded765a0 con 0x55e2ded4b000 2024-04-16T16:56:42.806 INFO:journalctl@ceph.mgr.z.smithi117.stdout:Apr 16 16:56:42 smithi117 bash[38577]: 08195d3a5a957145dbd9722387eac83b6d2e2e5aa30ef32e995d2df61803426a 2024-04-16T16:56:42.806 INFO:journalctl@ceph.mgr.z.smithi117.stdout:Apr 16 16:56:42 smithi117 systemd[1]: Started Ceph mgr.z for b1fecd38-fc11-11ee-bc90-c7b262605968. 2024-04-16T16:56:43.306 INFO:journalctl@ceph.mgr.z.smithi117.stdout:Apr 16 16:56:43 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-z[38592]: 2024-04-16T16:56:43.013+0000 7f9591153200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-16T16:56:43.306 INFO:journalctl@ceph.mgr.z.smithi117.stdout:Apr 16 16:56:43 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-z[38592]: 2024-04-16T16:56:43.104+0000 7f9591153200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-16T16:56:43.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:43 smithi104 ceph-mon[36457]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:56:43.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:43 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:56:43.929 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:43 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:56:43.929 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:43 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:56:43.929 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:43 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-16T16:56:43.929 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:43 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-16T16:56:43.929 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:43 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-16T16:56:43.929 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:43 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:56:43.929 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:43 smithi104 ceph-mon[36457]: Deploying daemon mgr.y on smithi104 2024-04-16T16:56:43.968 INFO:journalctl@ceph.mgr.z.smithi117.stdout:Apr 16 16:56:43 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-z[38592]: 2024-04-16T16:56:43.738+0000 7f9591153200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-16T16:56:43.968 INFO:journalctl@ceph.mgr.z.smithi117.stdout:Apr 16 16:56:43 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-z[38592]: 2024-04-16T16:56:43.828+0000 7f9591153200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-16T16:56:43.968 INFO:journalctl@ceph.mgr.z.smithi117.stdout:Apr 16 16:56:43 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-z[38592]: 2024-04-16T16:56:43.966+0000 7f9591153200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-16T16:56:43.968 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:43 smithi117 ceph-mon[36730]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:56:43.968 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:43 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:56:43.968 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:43 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:56:43.968 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:43 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:56:43.968 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:43 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-16T16:56:43.968 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:43 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-16T16:56:43.969 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:43 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-16T16:56:43.969 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:43 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:56:43.969 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:43 smithi117 ceph-mon[36730]: Deploying daemon mgr.y on smithi104 2024-04-16T16:56:43.979 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:56:43 smithi045 ceph-mon[31676]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:56:43.980 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:56:43 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:56:43.980 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:56:43 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:56:43.980 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:56:43 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:56:43.980 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:56:43 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-16T16:56:43.980 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:56:43 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-16T16:56:43.980 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:56:43 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-16T16:56:43.980 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:56:43 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:56:43.980 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:56:43 smithi045 ceph-mon[31676]: Deploying daemon mgr.y on smithi104 2024-04-16T16:56:44.026 INFO:teuthology.orchestra.run.smithi045.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-16T16:56:44.026 INFO:teuthology.orchestra.run.smithi045.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-16T16:56:44.027 INFO:teuthology.orchestra.run.smithi045.stderr: stderr: 10+0 records in 2024-04-16T16:56:44.027 INFO:teuthology.orchestra.run.smithi045.stderr:10+0 records out 2024-04-16T16:56:44.027 INFO:teuthology.orchestra.run.smithi045.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0388984 s, 270 MB/s 2024-04-16T16:56:44.027 INFO:teuthology.orchestra.run.smithi045.stderr:--> Zapping successful for: 2024-04-16T16:56:44.306 INFO:journalctl@ceph.mgr.z.smithi117.stdout:Apr 16 16:56:44 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-z[38592]: 2024-04-16T16:56:44.060+0000 7f9591153200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-16T16:56:44.306 INFO:journalctl@ceph.mgr.z.smithi117.stdout:Apr 16 16:56:44 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-z[38592]: 2024-04-16T16:56:44.142+0000 7f9591153200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-16T16:56:44.678 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Apr 16 16:56:44 smithi104 systemd[1]: Starting Ceph mgr.y for b1fecd38-fc11-11ee-bc90-c7b262605968... 2024-04-16T16:56:44.687 INFO:journalctl@ceph.mgr.z.smithi117.stdout:Apr 16 16:56:44 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-z[38592]: 2024-04-16T16:56:44.403+0000 7f9591153200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-16T16:56:44.687 INFO:journalctl@ceph.mgr.z.smithi117.stdout:Apr 16 16:56:44 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-z[38592]: 2024-04-16T16:56:44.509+0000 7f9591153200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-16T16:56:44.687 INFO:journalctl@ceph.mgr.z.smithi117.stdout:Apr 16 16:56:44 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-z[38592]: 2024-04-16T16:56:44.593+0000 7f9591153200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-16T16:56:44.687 INFO:journalctl@ceph.mgr.z.smithi117.stdout:Apr 16 16:56:44 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-z[38592]: 2024-04-16T16:56:44.686+0000 7f9591153200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-16T16:56:44.776 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b1fecd38-fc11-11ee-bc90-c7b262605968 -- ceph orch daemon add osd smithi045:vg_nvme/lv_4 2024-04-16T16:56:45.056 INFO:journalctl@ceph.mgr.z.smithi117.stdout:Apr 16 16:56:44 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-z[38592]: 2024-04-16T16:56:44.858+0000 7f9591153200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-16T16:56:45.072 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Apr 16 16:56:44 smithi104 podman[37826]: 2024-04-16 16:56:44.786050808 +0000 UTC m=+0.098282129 container create ae9050b2d6c5b6535e7df159b86e5de8d321a91a254ac5f4b6821fd18700a66b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-y, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.build-date=20240409, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD) 2024-04-16T16:56:45.072 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Apr 16 16:56:44 smithi104 podman[37826]: 2024-04-16 16:56:44.705361883 +0000 UTC m=+0.017593218 image pull e5ce10bdffbeeffffd1125f67b369e2151f0f17cdec854e93b6adf9321f65907 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2 2024-04-16T16:56:45.072 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Apr 16 16:56:44 smithi104 podman[37826]: 2024-04-16 16:56:44.988353116 +0000 UTC m=+0.300584445 container init ae9050b2d6c5b6535e7df159b86e5de8d321a91a254ac5f4b6821fd18700a66b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-y, io.buildah.version=1.33.5, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240409, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-04-16T16:56:45.072 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Apr 16 16:56:44 smithi104 podman[37826]: 2024-04-16 16:56:44.990834441 +0000 UTC m=+0.303065765 container start ae9050b2d6c5b6535e7df159b86e5de8d321a91a254ac5f4b6821fd18700a66b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-y, org.label-schema.build-date=20240409, org.label-schema.vendor=CentOS, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-04-16T16:56:45.072 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Apr 16 16:56:45 smithi104 ceph-mgr[37845]: -- 172.21.15.104:0/2482493363 <== mon.2 v2:172.21.15.104:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x558b342265a0 con 0x558b341fac00 2024-04-16T16:56:45.072 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Apr 16 16:56:45 smithi104 bash[37826]: ae9050b2d6c5b6535e7df159b86e5de8d321a91a254ac5f4b6821fd18700a66b 2024-04-16T16:56:45.234 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/mon.a/config 2024-04-16T16:56:45.348 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Apr 16 16:56:45 smithi104 systemd[1]: Started Ceph mgr.y for b1fecd38-fc11-11ee-bc90-c7b262605968. 2024-04-16T16:56:45.623 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Apr 16 16:56:45 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-y[37841]: 2024-04-16T16:56:45.562+0000 7f92d3525200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-16T16:56:45.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:56:45 smithi045 ceph-mon[31676]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:56:45.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:56:45 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:56:45.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:56:45 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:56:45.729 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:56:45 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:56:45.729 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:56:45 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:56:45.729 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:56:45 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T16:56:45.928 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Apr 16 16:56:45 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-y[37841]: 2024-04-16T16:56:45.662+0000 7f92d3525200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-16T16:56:45.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:45 smithi104 ceph-mon[36457]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:56:45.929 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:45 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:56:45.929 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:45 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:56:45.929 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:45 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:56:45.929 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:45 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:56:45.929 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:45 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T16:56:45.973 INFO:journalctl@ceph.mgr.z.smithi117.stdout:Apr 16 16:56:45 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-z[38592]: 2024-04-16T16:56:45.831+0000 7f9591153200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-16T16:56:45.973 INFO:journalctl@ceph.mgr.z.smithi117.stdout:Apr 16 16:56:45 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-z[38592]: 2024-04-16T16:56:45.971+0000 7f9591153200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-16T16:56:45.973 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:45 smithi117 ceph-mon[36730]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:56:45.973 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:45 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:56:45.973 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:45 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:56:45.973 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:45 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:56:45.973 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:45 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:56:45.973 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:45 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T16:56:46.527 INFO:journalctl@ceph.mgr.z.smithi117.stdout:Apr 16 16:56:46 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-z[38592]: 2024-04-16T16:56:46.321+0000 7f9591153200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-16T16:56:46.527 INFO:journalctl@ceph.mgr.z.smithi117.stdout:Apr 16 16:56:46 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-z[38592]: 2024-04-16T16:56:46.414+0000 7f9591153200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-16T16:56:46.617 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Apr 16 16:56:46 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-y[37841]: 2024-04-16T16:56:46.384+0000 7f92d3525200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-16T16:56:46.617 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Apr 16 16:56:46 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-y[37841]: 2024-04-16T16:56:46.476+0000 7f92d3525200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-16T16:56:46.802 INFO:journalctl@ceph.mgr.z.smithi117.stdout:Apr 16 16:56:46 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-z[38592]: 2024-04-16T16:56:46.611+0000 7f9591153200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-16T16:56:46.802 INFO:journalctl@ceph.mgr.z.smithi117.stdout:Apr 16 16:56:46 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-z[38592]: 2024-04-16T16:56:46.800+0000 7f9591153200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-16T16:56:46.928 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Apr 16 16:56:46 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-y[37841]: 2024-04-16T16:56:46.616+0000 7f92d3525200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-16T16:56:46.928 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Apr 16 16:56:46 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-y[37841]: 2024-04-16T16:56:46.726+0000 7f92d3525200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-16T16:56:46.928 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Apr 16 16:56:46 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-y[37841]: 2024-04-16T16:56:46.810+0000 7f92d3525200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-16T16:56:47.345 INFO:journalctl@ceph.mgr.z.smithi117.stdout:Apr 16 16:56:47 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-z[38592]: 2024-04-16T16:56:47.084+0000 7f9591153200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-16T16:56:47.345 INFO:journalctl@ceph.mgr.z.smithi117.stdout:Apr 16 16:56:47 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-z[38592]: 2024-04-16T16:56:47.343+0000 7f9591153200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-16T16:56:47.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:56:47.343+0000 7f0602000700 1 -- 172.21.15.45:0/2016223216 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f05fc076aa0 msgr2=0x7f05fc074f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:56:47.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:56:47.343+0000 7f0602000700 1 --2- 172.21.15.45:0/2016223216 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f05fc076aa0 0x7f05fc074f00 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f05ec009990 tx=0x7f05ec009ca0 comp rx=0 tx=0).stop 2024-04-16T16:56:47.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:56:47.343+0000 7f0602000700 1 -- 172.21.15.45:0/2016223216 shutdown_connections 2024-04-16T16:56:47.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:56:47.343+0000 7f0602000700 1 --2- 172.21.15.45:0/2016223216 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f05fc10aaa0 0x7f05fc10cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:56:47.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:56:47.343+0000 7f0602000700 1 --2- 172.21.15.45:0/2016223216 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f05fc076aa0 0x7f05fc074f00 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:56:47.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:56:47.343+0000 7f0602000700 1 --2- 172.21.15.45:0/2016223216 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f05fc0754d0 0x7f05fc075950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:56:47.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:56:47.343+0000 7f0602000700 1 -- 172.21.15.45:0/2016223216 >> 172.21.15.45:0/2016223216 conn(0x7f05fc0fe140 msgr2=0x7f05fc100580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T16:56:47.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:56:47.343+0000 7f0602000700 1 -- 172.21.15.45:0/2016223216 shutdown_connections 2024-04-16T16:56:47.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:56:47.343+0000 7f0602000700 1 -- 172.21.15.45:0/2016223216 wait complete. 2024-04-16T16:56:47.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:56:47.344+0000 7f0602000700 1 Processor -- start 2024-04-16T16:56:47.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:56:47.344+0000 7f0602000700 1 -- start start 2024-04-16T16:56:47.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:56:47.345+0000 7f0602000700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f05fc0754d0 0x7f05fc19af80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:56:47.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:56:47.345+0000 7f0602000700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f05fc076aa0 0x7f05fc19b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:56:47.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:56:47.345+0000 7f0602000700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f05fc10aaa0 0x7f05fc1a24f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:56:47.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:56:47.345+0000 7f0602000700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f05fc110990 con 0x7f05fc0754d0 2024-04-16T16:56:47.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:56:47.345+0000 7f0602000700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f05fc110810 con 0x7f05fc076aa0 2024-04-16T16:56:47.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:56:47.345+0000 7f0602000700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f05fc110b10 con 0x7f05fc10aaa0 2024-04-16T16:56:47.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:56:47.345+0000 7f05faa00700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f05fc076aa0 0x7f05fc19b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:56:47.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:56:47.345+0000 7f05faa00700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f05fc076aa0 0x7f05fc19b4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:60210/0 (socket says 172.21.15.45:60210) 2024-04-16T16:56:47.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:56:47.345+0000 7f05fb400700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f05fc0754d0 0x7f05fc19af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:56:47.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:56:47.345+0000 7f05faa00700 1 -- 172.21.15.45:0/401005744 learned_addr learned my addr 172.21.15.45:0/401005744 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T16:56:47.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:56:47.346+0000 7f05faa00700 1 -- 172.21.15.45:0/401005744 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f05fc10aaa0 msgr2=0x7f05fc1a24f0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-16T16:56:47.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:56:47.346+0000 7f05faa00700 1 --2- 172.21.15.45:0/401005744 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f05fc10aaa0 0x7f05fc1a24f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:56:47.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:56:47.346+0000 7f05faa00700 1 -- 172.21.15.45:0/401005744 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f05fc0754d0 msgr2=0x7f05fc19af80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:56:47.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:56:47.346+0000 7f05faa00700 1 --2- 172.21.15.45:0/401005744 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f05fc0754d0 0x7f05fc19af80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:56:47.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:56:47.346+0000 7f05faa00700 1 -- 172.21.15.45:0/401005744 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05fc1a2b90 con 0x7f05fc076aa0 2024-04-16T16:56:47.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:56:47.346+0000 7f05faa00700 1 --2- 172.21.15.45:0/401005744 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f05fc076aa0 0x7f05fc19b4c0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f05e800e9a0 tx=0x7f05e800ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T16:56:47.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:56:47.346+0000 7f05f3e00700 1 -- 172.21.15.45:0/401005744 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05e800cbd0 con 0x7f05fc076aa0 2024-04-16T16:56:47.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:56:47.346+0000 7f05f3e00700 1 -- 172.21.15.45:0/401005744 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f05e800cd70 con 0x7f05fc076aa0 2024-04-16T16:56:47.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:56:47.346+0000 7f0602000700 1 -- 172.21.15.45:0/401005744 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05fc1a2e80 con 0x7f05fc076aa0 2024-04-16T16:56:47.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:56:47.347+0000 7f05f3e00700 1 -- 172.21.15.45:0/401005744 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05e80187a0 con 0x7f05fc076aa0 2024-04-16T16:56:47.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:56:47.347+0000 7f0602000700 1 -- 172.21.15.45:0/401005744 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05fc1a33e0 con 0x7f05fc076aa0 2024-04-16T16:56:47.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:56:47.348+0000 7f05f3e00700 1 -- 172.21.15.45:0/401005744 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f05e8018940 con 0x7f05fc076aa0 2024-04-16T16:56:47.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:56:47.348+0000 7f05f3e00700 1 --2- 172.21.15.45:0/401005744 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f05e0039890 0x7f05e003bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:56:47.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:56:47.348+0000 7f05f3e00700 1 -- 172.21.15.45:0/401005744 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f05e804cb30 con 0x7f05fc076aa0 2024-04-16T16:56:47.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:56:47.348+0000 7f0602000700 1 -- 172.21.15.45:0/401005744 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05fc0697e0 con 0x7f05fc076aa0 2024-04-16T16:56:47.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:56:47.349+0000 7f05fb400700 1 --2- 172.21.15.45:0/401005744 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f05e0039890 0x7f05e003bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:56:47.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:56:47.349+0000 7f05fb400700 1 --2- 172.21.15.45:0/401005744 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f05e0039890 0x7f05e003bd50 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f05ec00b2e0 tx=0x7f05ec005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T16:56:47.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:56:47.354+0000 7f05f3e00700 1 -- 172.21.15.45:0/401005744 <== mon.2 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f05e80071c0 con 0x7f05fc076aa0 2024-04-16T16:56:47.375 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Apr 16 16:56:47 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-y[37841]: 2024-04-16T16:56:47.084+0000 7f92d3525200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-16T16:56:47.375 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Apr 16 16:56:47 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-y[37841]: 2024-04-16T16:56:47.194+0000 7f92d3525200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-16T16:56:47.375 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Apr 16 16:56:47 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-y[37841]: 2024-04-16T16:56:47.278+0000 7f92d3525200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-16T16:56:47.375 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Apr 16 16:56:47 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-y[37841]: 2024-04-16T16:56:47.373+0000 7f92d3525200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-16T16:56:47.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:56:47.596+0000 7f0602000700 1 -- 172.21.15.45:0/401005744 --> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi045:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f05fc062f30 con 0x7f05e0039890 2024-04-16T16:56:47.678 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Apr 16 16:56:47 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-y[37841]: 2024-04-16T16:56:47.547+0000 7f92d3525200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-16T16:56:47.748 INFO:journalctl@ceph.mgr.z.smithi117.stdout:Apr 16 16:56:47 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-z[38592]: 2024-04-16T16:56:47.578+0000 7f9591153200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-16T16:56:47.978 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:56:47 smithi045 ceph-mon[31676]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:56:47.978 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:56:47 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:56:47.978 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:56:47 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:56:47.978 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:56:47 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:56:47.978 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:56:47 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:56:47.978 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:56:47 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:56:47.979 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:56:47 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T16:56:47.979 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:56:47 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:56:47.979 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:56:47 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-16T16:56:47.979 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:56:47 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-16T16:56:47.979 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:56:47 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:56:47.979 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:56:47 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-16T16:56:47.979 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:56:47 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-16T16:56:47.979 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:56:47 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:56:48.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:47 smithi117 ceph-mon[36730]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:56:48.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:47 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:56:48.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:47 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:56:48.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:47 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:56:48.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:47 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:56:48.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:47 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:56:48.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:47 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T16:56:48.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:47 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:56:48.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:47 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-16T16:56:48.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:47 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-16T16:56:48.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:47 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:56:48.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:47 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-16T16:56:48.057 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:47 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-16T16:56:48.057 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:47 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:56:48.178 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:47 smithi104 ceph-mon[36457]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:56:48.200 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:47 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:56:48.200 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:47 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:56:48.200 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:47 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:56:48.201 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:47 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:56:48.201 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:47 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:56:48.201 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:47 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T16:56:48.201 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:47 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:56:48.201 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:47 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-16T16:56:48.201 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:47 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-16T16:56:48.201 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:47 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:56:48.201 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:47 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-16T16:56:48.201 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:47 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-16T16:56:48.201 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:47 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:56:48.756 INFO:journalctl@ceph.mgr.z.smithi117.stdout:Apr 16 16:56:48 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-z[38592]: 2024-04-16T16:56:48.371+0000 7f9591153200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-16T16:56:48.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:48 smithi104 ceph-mon[36457]: Reconfiguring mgr.x (unknown last config time)... 2024-04-16T16:56:48.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:48 smithi104 ceph-mon[36457]: Reconfiguring daemon mgr.x on smithi045 2024-04-16T16:56:48.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:48 smithi104 ceph-mon[36457]: from='client.24119 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi045:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T16:56:48.929 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Apr 16 16:56:48 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-y[37841]: 2024-04-16T16:56:48.514+0000 7f92d3525200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-16T16:56:48.929 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Apr 16 16:56:48 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-y[37841]: 2024-04-16T16:56:48.644+0000 7f92d3525200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-16T16:56:49.030 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:48 smithi117 ceph-mon[36730]: Reconfiguring mgr.x (unknown last config time)... 2024-04-16T16:56:49.030 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:48 smithi117 ceph-mon[36730]: Reconfiguring daemon mgr.x on smithi045 2024-04-16T16:56:49.031 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:48 smithi117 ceph-mon[36730]: from='client.24119 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi045:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T16:56:49.031 INFO:journalctl@ceph.mgr.z.smithi117.stdout:Apr 16 16:56:48 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-z[38592]: 2024-04-16T16:56:48.934+0000 7f9591153200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-16T16:56:49.031 INFO:journalctl@ceph.mgr.z.smithi117.stdout:Apr 16 16:56:49 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-z[38592]: 2024-04-16T16:56:49.029+0000 7f9591153200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-16T16:56:49.146 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:56:48 smithi045 ceph-mon[31676]: Reconfiguring mgr.x (unknown last config time)... 2024-04-16T16:56:49.146 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:56:48 smithi045 ceph-mon[31676]: Reconfiguring daemon mgr.x on smithi045 2024-04-16T16:56:49.146 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:56:48 smithi045 ceph-mon[31676]: from='client.24119 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi045:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T16:56:49.257 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Apr 16 16:56:48 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-y[37841]: 2024-04-16T16:56:48.974+0000 7f92d3525200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-16T16:56:49.257 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Apr 16 16:56:49 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-y[37841]: 2024-04-16T16:56:49.062+0000 7f92d3525200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-16T16:56:49.257 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Apr 16 16:56:49 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-y[37841]: 2024-04-16T16:56:49.255+0000 7f92d3525200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-16T16:56:49.306 INFO:journalctl@ceph.mgr.z.smithi117.stdout:Apr 16 16:56:49 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-z[38592]: 2024-04-16T16:56:49.240+0000 7f9591153200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-16T16:56:49.678 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Apr 16 16:56:49 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-y[37841]: 2024-04-16T16:56:49.439+0000 7f92d3525200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-16T16:56:49.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:56:49.764+0000 7f05f3e00700 1 -- 172.21.15.45:0/401005744 <== mon.2 v2:172.21.15.104:3300/0 7 ==== mgrmap(e 14) ==== 92602+0+0 (secure 0 0 0) 0x7f05e80109a0 con 0x7f05fc076aa0 2024-04-16T16:56:49.984 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Apr 16 16:56:49 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-y[37841]: 2024-04-16T16:56:49.725+0000 7f92d3525200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-16T16:56:49.984 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Apr 16 16:56:49 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-y[37841]: 2024-04-16T16:56:49.982+0000 7f92d3525200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-16T16:56:49.984 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:49 smithi104 ceph-mon[36457]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:56:49.984 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:49 smithi104 ceph-mon[36457]: Standby manager daemon z started 2024-04-16T16:56:49.985 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:49 smithi104 ceph-mon[36457]: from='mgr.? 172.21.15.117:0/3968342475' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-16T16:56:49.985 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:49 smithi104 ceph-mon[36457]: from='mgr.? 172.21.15.117:0/3968342475' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-16T16:56:49.985 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:49 smithi104 ceph-mon[36457]: from='mgr.? 172.21.15.117:0/3968342475' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-16T16:56:49.985 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:49 smithi104 ceph-mon[36457]: from='mgr.? 172.21.15.117:0/3968342475' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-16T16:56:49.985 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:49 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:56:49.985 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:49 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:56:49.985 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:49 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T16:56:49.985 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:49 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:56:49.985 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:49 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T16:56:49.985 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:49 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:56:50.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:49 smithi117 ceph-mon[36730]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:56:50.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:49 smithi117 ceph-mon[36730]: Standby manager daemon z started 2024-04-16T16:56:50.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:49 smithi117 ceph-mon[36730]: from='mgr.? 172.21.15.117:0/3968342475' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-16T16:56:50.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:49 smithi117 ceph-mon[36730]: from='mgr.? 172.21.15.117:0/3968342475' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-16T16:56:50.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:49 smithi117 ceph-mon[36730]: from='mgr.? 172.21.15.117:0/3968342475' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-16T16:56:50.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:49 smithi117 ceph-mon[36730]: from='mgr.? 172.21.15.117:0/3968342475' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-16T16:56:50.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:49 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:56:50.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:49 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:56:50.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:49 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T16:56:50.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:49 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:56:50.057 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:49 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T16:56:50.057 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:49 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:56:50.228 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:56:49 smithi045 ceph-mon[31676]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:56:50.229 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:56:49 smithi045 ceph-mon[31676]: Standby manager daemon z started 2024-04-16T16:56:50.229 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:56:49 smithi045 ceph-mon[31676]: from='mgr.? 172.21.15.117:0/3968342475' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-16T16:56:50.229 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:56:49 smithi045 ceph-mon[31676]: from='mgr.? 172.21.15.117:0/3968342475' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-16T16:56:50.229 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:56:49 smithi045 ceph-mon[31676]: from='mgr.? 172.21.15.117:0/3968342475' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-16T16:56:50.229 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:56:49 smithi045 ceph-mon[31676]: from='mgr.? 172.21.15.117:0/3968342475' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-16T16:56:50.229 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:56:49 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:56:50.229 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:56:49 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:56:50.229 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:56:49 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T16:56:50.229 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:56:49 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:56:50.229 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:56:49 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T16:56:50.229 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:56:49 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:56:50.428 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Apr 16 16:56:50 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-y[37841]: 2024-04-16T16:56:50.219+0000 7f92d3525200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-16T16:56:50.950 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:56:50 smithi045 ceph-mon[31676]: mgrmap e14: x(active, since 89s), standbys: z 2024-04-16T16:56:50.950 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:56:50 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-16T16:56:50.950 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:56:50 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:56:51.055 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:50 smithi117 ceph-mon[36730]: mgrmap e14: x(active, since 89s), standbys: z 2024-04-16T16:56:51.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:50 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-16T16:56:51.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:50 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:56:51.178 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:50 smithi104 ceph-mon[36457]: mgrmap e14: x(active, since 89s), standbys: z 2024-04-16T16:56:51.178 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:50 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-16T16:56:51.178 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:50 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:56:51.179 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Apr 16 16:56:51 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-y[37841]: 2024-04-16T16:56:51.021+0000 7f92d3525200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-16T16:56:51.901 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:51 smithi104 ceph-mon[36457]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:56:51.901 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Apr 16 16:56:51 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-y[37841]: 2024-04-16T16:56:51.587+0000 7f92d3525200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-16T16:56:51.901 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Apr 16 16:56:51 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-y[37841]: 2024-04-16T16:56:51.684+0000 7f92d3525200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-16T16:56:51.901 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Apr 16 16:56:51 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-y[37841]: 2024-04-16T16:56:51.899+0000 7f92d3525200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-16T16:56:52.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:51 smithi117 ceph-mon[36730]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:56:52.228 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:56:51 smithi045 ceph-mon[31676]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:56:52.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:56:52.806+0000 7f05f3e00700 1 -- 172.21.15.45:0/401005744 <== mon.2 v2:172.21.15.104:3300/0 8 ==== mgrmap(e 15) ==== 138590+0+0 (secure 0 0 0) 0x7f05e80173d0 con 0x7f05fc076aa0 2024-04-16T16:56:53.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:52 smithi117 ceph-mon[36730]: Standby manager daemon y started 2024-04-16T16:56:53.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:52 smithi117 ceph-mon[36730]: from='mgr.? 172.21.15.104:0/3952644145' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-16T16:56:53.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:52 smithi117 ceph-mon[36730]: from='mgr.? 172.21.15.104:0/3952644145' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-16T16:56:53.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:52 smithi117 ceph-mon[36730]: from='mgr.? 172.21.15.104:0/3952644145' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-16T16:56:53.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:52 smithi117 ceph-mon[36730]: from='mgr.? 172.21.15.104:0/3952644145' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-16T16:56:53.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:52 smithi117 ceph-mon[36730]: from='client.? 172.21.15.45:0/1556792849' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b655de2c-59f8-4fd4-ac98-0cc222229f56"}]: dispatch 2024-04-16T16:56:53.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:52 smithi117 ceph-mon[36730]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b655de2c-59f8-4fd4-ac98-0cc222229f56"}]: dispatch 2024-04-16T16:56:53.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:52 smithi117 ceph-mon[36730]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b655de2c-59f8-4fd4-ac98-0cc222229f56"}]': finished 2024-04-16T16:56:53.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:52 smithi117 ceph-mon[36730]: osdmap e5: 1 total, 0 up, 1 in 2024-04-16T16:56:53.057 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:52 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-16T16:56:53.178 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:52 smithi104 ceph-mon[36457]: Standby manager daemon y started 2024-04-16T16:56:53.178 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:52 smithi104 ceph-mon[36457]: from='mgr.? 172.21.15.104:0/3952644145' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-16T16:56:53.178 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:52 smithi104 ceph-mon[36457]: from='mgr.? 172.21.15.104:0/3952644145' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-16T16:56:53.178 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:52 smithi104 ceph-mon[36457]: from='mgr.? 172.21.15.104:0/3952644145' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-16T16:56:53.178 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:52 smithi104 ceph-mon[36457]: from='mgr.? 172.21.15.104:0/3952644145' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-16T16:56:53.178 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:52 smithi104 ceph-mon[36457]: from='client.? 172.21.15.45:0/1556792849' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b655de2c-59f8-4fd4-ac98-0cc222229f56"}]: dispatch 2024-04-16T16:56:53.179 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:52 smithi104 ceph-mon[36457]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b655de2c-59f8-4fd4-ac98-0cc222229f56"}]: dispatch 2024-04-16T16:56:53.179 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:52 smithi104 ceph-mon[36457]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b655de2c-59f8-4fd4-ac98-0cc222229f56"}]': finished 2024-04-16T16:56:53.179 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:52 smithi104 ceph-mon[36457]: osdmap e5: 1 total, 0 up, 1 in 2024-04-16T16:56:53.179 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:52 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-16T16:56:53.184 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:56:52 smithi045 ceph-mon[31676]: Standby manager daemon y started 2024-04-16T16:56:53.184 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:56:52 smithi045 ceph-mon[31676]: from='mgr.? 172.21.15.104:0/3952644145' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-16T16:56:53.184 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:56:52 smithi045 ceph-mon[31676]: from='mgr.? 172.21.15.104:0/3952644145' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-16T16:56:53.184 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:56:52 smithi045 ceph-mon[31676]: from='mgr.? 172.21.15.104:0/3952644145' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-16T16:56:53.184 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:56:52 smithi045 ceph-mon[31676]: from='mgr.? 172.21.15.104:0/3952644145' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-16T16:56:53.184 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:56:52 smithi045 ceph-mon[31676]: from='client.? 172.21.15.45:0/1556792849' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b655de2c-59f8-4fd4-ac98-0cc222229f56"}]: dispatch 2024-04-16T16:56:53.184 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:56:52 smithi045 ceph-mon[31676]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b655de2c-59f8-4fd4-ac98-0cc222229f56"}]: dispatch 2024-04-16T16:56:53.184 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:56:52 smithi045 ceph-mon[31676]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b655de2c-59f8-4fd4-ac98-0cc222229f56"}]': finished 2024-04-16T16:56:53.184 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:56:52 smithi045 ceph-mon[31676]: osdmap e5: 1 total, 0 up, 1 in 2024-04-16T16:56:53.184 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:56:52 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-16T16:56:53.899 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:56:53 smithi045 ceph-mon[31676]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:56:53.899 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:56:53 smithi045 ceph-mon[31676]: mgrmap e15: x(active, since 92s), standbys: z, y 2024-04-16T16:56:53.899 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:56:53 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-16T16:56:53.900 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:56:53 smithi045 ceph-mon[31676]: from='client.? 172.21.15.45:0/42996788' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-16T16:56:54.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:53 smithi117 ceph-mon[36730]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:56:54.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:53 smithi117 ceph-mon[36730]: mgrmap e15: x(active, since 92s), standbys: z, y 2024-04-16T16:56:54.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:53 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-16T16:56:54.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:53 smithi117 ceph-mon[36730]: from='client.? 172.21.15.45:0/42996788' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-16T16:56:54.178 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:53 smithi104 ceph-mon[36457]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:56:54.179 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:53 smithi104 ceph-mon[36457]: mgrmap e15: x(active, since 92s), standbys: z, y 2024-04-16T16:56:54.179 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:53 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-16T16:56:54.179 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:53 smithi104 ceph-mon[36457]: from='client.? 172.21.15.45:0/42996788' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-16T16:56:55.178 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:54 smithi104 ceph-mon[36457]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:56:55.229 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:56:54 smithi045 ceph-mon[31676]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:56:55.306 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:54 smithi117 ceph-mon[36730]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:56:57.194 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:56:57 smithi045 ceph-mon[31676]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:56:57.306 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:57 smithi117 ceph-mon[36730]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:56:57.428 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:57 smithi104 ceph-mon[36457]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:56:59.228 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:56:59 smithi045 ceph-mon[31676]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:56:59.305 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:56:59 smithi117 ceph-mon[36730]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:56:59.428 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:56:59 smithi104 ceph-mon[36457]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:57:01.306 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:57:01 smithi117 ceph-mon[36730]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:57:01.428 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:57:01 smithi104 ceph-mon[36457]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:57:01.459 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:57:01 smithi045 ceph-mon[31676]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:57:02.306 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:57:02 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-16T16:57:02.306 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:57:02 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:57:02.306 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:57:02 smithi117 ceph-mon[36730]: Deploying daemon osd.0 on smithi045 2024-04-16T16:57:02.428 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:57:02 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-16T16:57:02.429 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:57:02 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:57:02.429 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:57:02 smithi104 ceph-mon[36457]: Deploying daemon osd.0 on smithi045 2024-04-16T16:57:02.478 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:57:02 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-16T16:57:02.478 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:57:02 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:57:02.478 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:57:02 smithi045 ceph-mon[31676]: Deploying daemon osd.0 on smithi045 2024-04-16T16:57:03.228 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:57:03 smithi045 ceph-mon[31676]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:57:03.428 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:57:03 smithi104 ceph-mon[36457]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:57:03.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:57:03 smithi117 ceph-mon[36730]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:57:05.358 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:57:05 smithi117 ceph-mon[36730]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:57:05.428 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:57:05 smithi104 ceph-mon[36457]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:57:05.430 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:57:05 smithi045 ceph-mon[31676]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:57:07.205 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:57:07 smithi045 ceph-mon[31676]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:57:07.428 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:57:07 smithi104 ceph-mon[36457]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:57:07.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:57:07 smithi117 ceph-mon[36730]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:57:09.228 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:57:09 smithi045 ceph-mon[31676]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:57:09.428 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:57:09 smithi104 ceph-mon[36457]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:57:09.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:57:09 smithi117 ceph-mon[36730]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:57:11.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:57:10 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:57:11.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:57:10 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:57:11.051 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:57:10 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T16:57:11.055 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:57:10 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:57:11.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:57:10 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:57:11.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:57:10 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T16:57:11.178 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:57:10 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:57:11.178 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:57:10 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:57:11.178 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:57:10 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T16:57:12.055 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:57:11 smithi117 ceph-mon[36730]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:57:12.178 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:57:11 smithi104 ceph-mon[36457]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:57:12.181 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:57:11 smithi045 ceph-mon[31676]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:57:14.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:57:13 smithi117 ceph-mon[36730]: pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:57:14.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:57:13 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:57:14.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:57:13 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:57:14.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:57:13 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:57:14.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:57:13 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T16:57:14.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:57:13 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:57:14.057 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:57:13 smithi117 ceph-mon[36730]: from='osd.0 [v2:172.21.15.45:6802/2290170881,v1:172.21.15.45:6803/2290170881]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-16T16:57:14.057 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:57:13 smithi117 ceph-mon[36730]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-16T16:57:14.117 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:57:13 smithi045 ceph-mon[31676]: pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:57:14.117 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:57:13 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:57:14.118 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:57:13 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:57:14.118 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:57:13 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:57:14.118 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:57:13 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T16:57:14.118 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:57:13 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:57:14.118 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:57:13 smithi045 ceph-mon[31676]: from='osd.0 [v2:172.21.15.45:6802/2290170881,v1:172.21.15.45:6803/2290170881]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-16T16:57:14.118 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:57:13 smithi045 ceph-mon[31676]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-16T16:57:14.178 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:57:13 smithi104 ceph-mon[36457]: pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:57:14.178 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:57:13 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:57:14.178 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:57:13 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:57:14.178 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:57:13 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:57:14.178 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:57:13 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T16:57:14.179 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:57:13 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:57:14.179 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:57:13 smithi104 ceph-mon[36457]: from='osd.0 [v2:172.21.15.45:6802/2290170881,v1:172.21.15.45:6803/2290170881]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-16T16:57:14.179 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:57:13 smithi104 ceph-mon[36457]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-16T16:57:14.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:57:14.606+0000 7f05f3e00700 1 -- 172.21.15.45:0/401005744 <== mgr.14152 v2:172.21.15.45:6800/268030699 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f05fc062f30 con 0x7f05e0039890 2024-04-16T16:57:14.607 INFO:teuthology.orchestra.run.smithi045.stdout:Created osd(s) 0 on host 'smithi045' 2024-04-16T16:57:14.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:57:14.609+0000 7f0602000700 1 -- 172.21.15.45:0/401005744 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f05e0039890 msgr2=0x7f05e003bd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:57:14.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:57:14.609+0000 7f0602000700 1 --2- 172.21.15.45:0/401005744 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f05e0039890 0x7f05e003bd50 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f05ec00b2e0 tx=0x7f05ec005fd0 comp rx=0 tx=0).stop 2024-04-16T16:57:14.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:57:14.609+0000 7f0602000700 1 -- 172.21.15.45:0/401005744 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f05fc076aa0 msgr2=0x7f05fc19b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:57:14.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:57:14.609+0000 7f0602000700 1 --2- 172.21.15.45:0/401005744 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f05fc076aa0 0x7f05fc19b4c0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f05e800e9a0 tx=0x7f05e800ecb0 comp rx=0 tx=0).stop 2024-04-16T16:57:14.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:57:14.610+0000 7f0602000700 1 -- 172.21.15.45:0/401005744 shutdown_connections 2024-04-16T16:57:14.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:57:14.610+0000 7f0602000700 1 --2- 172.21.15.45:0/401005744 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f05e0039890 0x7f05e003bd50 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:57:14.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:57:14.610+0000 7f0602000700 1 --2- 172.21.15.45:0/401005744 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f05fc10aaa0 0x7f05fc1a24f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:57:14.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:57:14.610+0000 7f0602000700 1 --2- 172.21.15.45:0/401005744 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f05fc0754d0 0x7f05fc19af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:57:14.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:57:14.610+0000 7f0602000700 1 --2- 172.21.15.45:0/401005744 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f05fc076aa0 0x7f05fc19b4c0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:57:14.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:57:14.610+0000 7f0602000700 1 -- 172.21.15.45:0/401005744 >> 172.21.15.45:0/401005744 conn(0x7f05fc0fe140 msgr2=0x7f05fc1069f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T16:57:14.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:57:14.610+0000 7f0602000700 1 -- 172.21.15.45:0/401005744 shutdown_connections 2024-04-16T16:57:14.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:57:14.610+0000 7f0602000700 1 -- 172.21.15.45:0/401005744 wait complete. 2024-04-16T16:57:15.404 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:57:15 smithi045 ceph-mon[31676]: pgmap v52: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:57:15.404 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:57:15 smithi045 ceph-mon[31676]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-16T16:57:15.404 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:57:15 smithi045 ceph-mon[31676]: from='osd.0 [v2:172.21.15.45:6802/2290170881,v1:172.21.15.45:6803/2290170881]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]: dispatch 2024-04-16T16:57:15.404 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:57:15 smithi045 ceph-mon[31676]: osdmap e6: 1 total, 0 up, 1 in 2024-04-16T16:57:15.404 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:57:15 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-16T16:57:15.404 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:57:15 smithi045 ceph-mon[31676]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]: dispatch 2024-04-16T16:57:15.404 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:57:15 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:57:15.404 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:57:15 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T16:57:15.404 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:57:15 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:57:15.516 DEBUG:teuthology.orchestra.run.smithi045:osd.0> sudo journalctl -f -n 0 -u ceph-b1fecd38-fc11-11ee-bc90-c7b262605968@osd.0.service 2024-04-16T16:57:15.521 INFO:tasks.cephadm:Deploying osd.1 on smithi104 with /dev/vg_nvme/lv_4... 2024-04-16T16:57:15.521 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b1fecd38-fc11-11ee-bc90-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-16T16:57:15.547 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:57:15 smithi104 ceph-mon[36457]: pgmap v52: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:57:15.547 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:57:15 smithi104 ceph-mon[36457]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-16T16:57:15.547 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:57:15 smithi104 ceph-mon[36457]: from='osd.0 [v2:172.21.15.45:6802/2290170881,v1:172.21.15.45:6803/2290170881]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]: dispatch 2024-04-16T16:57:15.547 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:57:15 smithi104 ceph-mon[36457]: osdmap e6: 1 total, 0 up, 1 in 2024-04-16T16:57:15.547 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:57:15 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-16T16:57:15.547 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:57:15 smithi104 ceph-mon[36457]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]: dispatch 2024-04-16T16:57:15.547 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:57:15 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:57:15.547 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:57:15 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T16:57:15.547 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:57:15 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:57:15.558 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:57:15 smithi117 ceph-mon[36730]: pgmap v52: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:57:15.558 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:57:15 smithi117 ceph-mon[36730]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-16T16:57:15.558 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:57:15 smithi117 ceph-mon[36730]: from='osd.0 [v2:172.21.15.45:6802/2290170881,v1:172.21.15.45:6803/2290170881]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]: dispatch 2024-04-16T16:57:15.559 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:57:15 smithi117 ceph-mon[36730]: osdmap e6: 1 total, 0 up, 1 in 2024-04-16T16:57:15.559 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:57:15 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-16T16:57:15.559 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:57:15 smithi117 ceph-mon[36730]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]: dispatch 2024-04-16T16:57:15.559 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:57:15 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:57:15.559 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:57:15 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T16:57:15.559 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:57:15 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:57:15.752 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/mon.b/config 2024-04-16T16:57:16.116 INFO:journalctl@ceph.osd.0.smithi045.stdout:Apr 16 16:57:15 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-0[46414]: 2024-04-16T16:57:15.893+0000 7fad22a00700 -1 osd.0 0 waiting for initial osdmap 2024-04-16T16:57:16.116 INFO:journalctl@ceph.osd.0.smithi045.stdout:Apr 16 16:57:15 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-0[46414]: 2024-04-16T16:57:15.908+0000 7fad17e00700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-16T16:57:16.478 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:57:16 smithi045 ceph-mon[31676]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]': finished 2024-04-16T16:57:16.478 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:57:16 smithi045 ceph-mon[31676]: osdmap e7: 1 total, 0 up, 1 in 2024-04-16T16:57:16.479 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:57:16 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-16T16:57:16.479 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:57:16 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-16T16:57:16.479 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:57:16 smithi045 ceph-mon[31676]: from='osd.0 ' entity='osd.0' 2024-04-16T16:57:16.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:57:16 smithi117 ceph-mon[36730]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]': finished 2024-04-16T16:57:16.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:57:16 smithi117 ceph-mon[36730]: osdmap e7: 1 total, 0 up, 1 in 2024-04-16T16:57:16.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:57:16 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-16T16:57:16.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:57:16 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-16T16:57:16.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:57:16 smithi117 ceph-mon[36730]: from='osd.0 ' entity='osd.0' 2024-04-16T16:57:16.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:57:16 smithi104 ceph-mon[36457]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]': finished 2024-04-16T16:57:16.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:57:16 smithi104 ceph-mon[36457]: osdmap e7: 1 total, 0 up, 1 in 2024-04-16T16:57:16.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:57:16 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-16T16:57:16.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:57:16 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-16T16:57:16.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:57:16 smithi104 ceph-mon[36457]: from='osd.0 ' entity='osd.0' 2024-04-16T16:57:17.428 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:57:17 smithi104 ceph-mon[36457]: purged_snaps scrub starts 2024-04-16T16:57:17.428 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:57:17 smithi104 ceph-mon[36457]: purged_snaps scrub ok 2024-04-16T16:57:17.429 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:57:17 smithi104 ceph-mon[36457]: pgmap v55: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:57:17.429 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:57:17 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-16T16:57:17.429 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:57:17 smithi104 ceph-mon[36457]: osd.0 [v2:172.21.15.45:6802/2290170881,v1:172.21.15.45:6803/2290170881] boot 2024-04-16T16:57:17.429 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:57:17 smithi104 ceph-mon[36457]: osdmap e8: 1 total, 1 up, 1 in 2024-04-16T16:57:17.429 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:57:17 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-16T16:57:17.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:57:17 smithi117 ceph-mon[36730]: purged_snaps scrub starts 2024-04-16T16:57:17.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:57:17 smithi117 ceph-mon[36730]: purged_snaps scrub ok 2024-04-16T16:57:17.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:57:17 smithi117 ceph-mon[36730]: pgmap v55: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:57:17.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:57:17 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-16T16:57:17.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:57:17 smithi117 ceph-mon[36730]: osd.0 [v2:172.21.15.45:6802/2290170881,v1:172.21.15.45:6803/2290170881] boot 2024-04-16T16:57:17.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:57:17 smithi117 ceph-mon[36730]: osdmap e8: 1 total, 1 up, 1 in 2024-04-16T16:57:17.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:57:17 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-16T16:57:17.623 INFO:teuthology.orchestra.run.smithi104.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-16T16:57:17.624 INFO:teuthology.orchestra.run.smithi104.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-16T16:57:17.624 INFO:teuthology.orchestra.run.smithi104.stderr: stderr: 10+0 records in 2024-04-16T16:57:17.624 INFO:teuthology.orchestra.run.smithi104.stderr:10+0 records out 2024-04-16T16:57:17.624 INFO:teuthology.orchestra.run.smithi104.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.011374 s, 922 MB/s 2024-04-16T16:57:17.624 INFO:teuthology.orchestra.run.smithi104.stderr:--> Zapping successful for: 2024-04-16T16:57:17.673 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:57:17 smithi045 ceph-mon[31676]: purged_snaps scrub starts 2024-04-16T16:57:17.673 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:57:17 smithi045 ceph-mon[31676]: purged_snaps scrub ok 2024-04-16T16:57:17.674 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:57:17 smithi045 ceph-mon[31676]: pgmap v55: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-16T16:57:17.674 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:57:17 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-16T16:57:17.674 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:57:17 smithi045 ceph-mon[31676]: osd.0 [v2:172.21.15.45:6802/2290170881,v1:172.21.15.45:6803/2290170881] boot 2024-04-16T16:57:17.674 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:57:17 smithi045 ceph-mon[31676]: osdmap e8: 1 total, 1 up, 1 in 2024-04-16T16:57:17.674 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:57:17 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-16T16:57:18.194 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b1fecd38-fc11-11ee-bc90-c7b262605968 -- ceph orch daemon add osd smithi104:vg_nvme/lv_4 2024-04-16T16:57:18.469 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/mon.b/config 2024-04-16T16:57:19.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:57:19 smithi117 ceph-mon[36730]: pgmap v57: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-16T16:57:19.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:57:19 smithi117 ceph-mon[36730]: osdmap e9: 1 total, 1 up, 1 in 2024-04-16T16:57:19.665 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:57:19 smithi104 ceph-mon[36457]: pgmap v57: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-16T16:57:19.665 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:57:19 smithi104 ceph-mon[36457]: osdmap e9: 1 total, 1 up, 1 in 2024-04-16T16:57:19.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:57:19 smithi045 ceph-mon[31676]: pgmap v57: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-16T16:57:19.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:57:19 smithi045 ceph-mon[31676]: osdmap e9: 1 total, 1 up, 1 in 2024-04-16T16:57:19.939 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-16T16:57:19.938+0000 7f0dda3d6700 1 -- 172.21.15.104:0/3886124611 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0dd4103660 msgr2=0x7f0dd4109ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:57:19.939 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-16T16:57:19.938+0000 7f0dda3d6700 1 --2- 172.21.15.104:0/3886124611 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0dd4103660 0x7f0dd4109ef0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f0dcc00b0a0 tx=0x7f0dcc00b3b0 comp rx=0 tx=0).stop 2024-04-16T16:57:19.939 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-16T16:57:19.938+0000 7f0dda3d6700 1 -- 172.21.15.104:0/3886124611 shutdown_connections 2024-04-16T16:57:19.939 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-16T16:57:19.938+0000 7f0dda3d6700 1 --2- 172.21.15.104:0/3886124611 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0dd4101f80 0x7f0dd41023a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:57:19.939 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-16T16:57:19.938+0000 7f0dda3d6700 1 --2- 172.21.15.104:0/3886124611 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0dd4103660 0x7f0dd4109ef0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:57:19.939 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-16T16:57:19.938+0000 7f0dda3d6700 1 --2- 172.21.15.104:0/3886124611 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0dd4102ca0 0x7f0dd4103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:57:19.939 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-16T16:57:19.938+0000 7f0dda3d6700 1 -- 172.21.15.104:0/3886124611 >> 172.21.15.104:0/3886124611 conn(0x7f0dd40fc760 msgr2=0x7f0dd40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T16:57:19.939 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-16T16:57:19.938+0000 7f0dda3d6700 1 -- 172.21.15.104:0/3886124611 shutdown_connections 2024-04-16T16:57:19.939 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-16T16:57:19.938+0000 7f0dda3d6700 1 -- 172.21.15.104:0/3886124611 wait complete. 2024-04-16T16:57:19.939 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-16T16:57:19.938+0000 7f0dda3d6700 1 Processor -- start 2024-04-16T16:57:19.939 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-16T16:57:19.938+0000 7f0dda3d6700 1 -- start start 2024-04-16T16:57:19.941 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-16T16:57:19.939+0000 7f0dda3d6700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0dd4101f80 0x7f0dd4197d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:57:19.941 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-16T16:57:19.939+0000 7f0dda3d6700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0dd4102ca0 0x7f0dd4198270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:57:19.941 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-16T16:57:19.939+0000 7f0dda3d6700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0dd4103660 0x7f0dd419f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:57:19.941 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-16T16:57:19.939+0000 7f0dda3d6700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f0dd410d780 con 0x7f0dd4102ca0 2024-04-16T16:57:19.941 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-16T16:57:19.939+0000 7f0dda3d6700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f0dd410d600 con 0x7f0dd4103660 2024-04-16T16:57:19.941 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-16T16:57:19.939+0000 7f0dda3d6700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f0dd410d900 con 0x7f0dd4101f80 2024-04-16T16:57:19.942 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-16T16:57:19.939+0000 7f0dd8973700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0dd4103660 0x7f0dd419f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:57:19.942 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-16T16:57:19.939+0000 7f0dd8973700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0dd4103660 0x7f0dd419f2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:53478/0 (socket says 172.21.15.104:53478) 2024-04-16T16:57:19.942 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-16T16:57:19.939+0000 7f0dd8973700 1 -- 172.21.15.104:0/2638505666 learned_addr learned my addr 172.21.15.104:0/2638505666 (peer_addr_for_me v2:172.21.15.104:0/0) 2024-04-16T16:57:19.942 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-16T16:57:19.939+0000 7f0dd3fff700 1 --2- 172.21.15.104:0/2638505666 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0dd4101f80 0x7f0dd4197d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:57:19.942 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-16T16:57:19.939+0000 7f0dd8973700 1 -- 172.21.15.104:0/2638505666 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0dd4101f80 msgr2=0x7f0dd4197d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:57:19.942 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-16T16:57:19.939+0000 7f0dd8973700 1 --2- 172.21.15.104:0/2638505666 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0dd4101f80 0x7f0dd4197d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:57:19.942 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-16T16:57:19.939+0000 7f0dd8973700 1 -- 172.21.15.104:0/2638505666 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0dd4102ca0 msgr2=0x7f0dd4198270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:57:19.942 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-16T16:57:19.939+0000 7f0dd8973700 1 --2- 172.21.15.104:0/2638505666 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0dd4102ca0 0x7f0dd4198270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:57:19.942 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-16T16:57:19.939+0000 7f0dd8973700 1 -- 172.21.15.104:0/2638505666 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0dd4100be0 con 0x7f0dd4103660 2024-04-16T16:57:19.942 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-16T16:57:19.939+0000 7f0dd8973700 1 --2- 172.21.15.104:0/2638505666 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0dd4103660 0x7f0dd419f2f0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f0dcc00b830 tx=0x7f0dcc007d90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T16:57:19.942 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-16T16:57:19.939+0000 7f0dd17fa700 1 -- 172.21.15.104:0/2638505666 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dcc01f070 con 0x7f0dd4103660 2024-04-16T16:57:19.943 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-16T16:57:19.939+0000 7f0dda3d6700 1 -- 172.21.15.104:0/2638505666 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0dd4100e70 con 0x7f0dd4103660 2024-04-16T16:57:19.943 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-16T16:57:19.939+0000 7f0dd17fa700 1 -- 172.21.15.104:0/2638505666 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0dcc003e10 con 0x7f0dd4103660 2024-04-16T16:57:19.943 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-16T16:57:19.940+0000 7f0dd17fa700 1 -- 172.21.15.104:0/2638505666 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dcc01baf0 con 0x7f0dd4103660 2024-04-16T16:57:19.943 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-16T16:57:19.940+0000 7f0dda3d6700 1 -- 172.21.15.104:0/2638505666 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0dd41013d0 con 0x7f0dd4103660 2024-04-16T16:57:19.943 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-16T16:57:19.940+0000 7f0dda3d6700 1 -- 172.21.15.104:0/2638505666 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0dd40697e0 con 0x7f0dd4103660 2024-04-16T16:57:19.943 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-16T16:57:19.942+0000 7f0dd17fa700 1 -- 172.21.15.104:0/2638505666 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 15) ==== 138590+0+0 (secure 0 0 0) 0x7f0dcc022070 con 0x7f0dd4103660 2024-04-16T16:57:19.944 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-16T16:57:19.942+0000 7f0dd17fa700 1 --2- 172.21.15.104:0/2638505666 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f0dbc0a3bb0 0x7f0dbc0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:57:19.944 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-16T16:57:19.943+0000 7f0dd17fa700 1 -- 172.21.15.104:0/2638505666 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1767+0+0 (secure 0 0 0) 0x7f0dcc00f020 con 0x7f0dd4103660 2024-04-16T16:57:19.944 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-16T16:57:19.943+0000 7f0dd3fff700 1 --2- 172.21.15.104:0/2638505666 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f0dbc0a3bb0 0x7f0dbc0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:57:19.944 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-16T16:57:19.944+0000 7f0dd3fff700 1 --2- 172.21.15.104:0/2638505666 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f0dbc0a3bb0 0x7f0dbc0a6070 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f0dd406ae10 tx=0x7f0dc4009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T16:57:19.947 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-16T16:57:19.947+0000 7f0dd17fa700 1 -- 172.21.15.104:0/2638505666 <== mon.2 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0dcc029020 con 0x7f0dd4103660 2024-04-16T16:57:20.152 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-16T16:57:20.152+0000 7f0dda3d6700 1 -- 172.21.15.104:0/2638505666 --> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi104:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f0dd4062f30 con 0x7f0dbc0a3bb0 2024-04-16T16:57:20.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:57:20 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-16T16:57:20.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:57:20 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-16T16:57:20.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:57:20 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:57:20.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:57:20 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-16T16:57:20.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:57:20 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-16T16:57:20.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:57:20 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:57:20.705 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:57:20 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-16T16:57:20.705 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:57:20 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-16T16:57:20.705 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:57:20 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:57:21.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:57:21 smithi117 ceph-mon[36730]: pgmap v59: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-16T16:57:21.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:57:21 smithi117 ceph-mon[36730]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi104:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T16:57:21.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:57:21 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:57:21.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:57:21 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:57:21.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:57:21 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-16T16:57:21.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:57:21 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:57:21.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:57:21 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:57:21.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:57:21 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T16:57:21.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:57:21 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:57:21.624 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:57:21 smithi104 ceph-mon[36457]: pgmap v59: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-16T16:57:21.624 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:57:21 smithi104 ceph-mon[36457]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi104:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T16:57:21.624 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:57:21 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:57:21.624 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:57:21 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:57:21.624 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:57:21 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-16T16:57:21.624 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:57:21 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:57:21.624 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:57:21 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:57:21.624 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:57:21 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T16:57:21.624 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:57:21 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:57:21.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:57:21 smithi045 ceph-mon[31676]: pgmap v59: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-16T16:57:21.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:57:21 smithi045 ceph-mon[31676]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi104:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T16:57:21.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:57:21 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:57:21.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:57:21 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:57:21.729 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:57:21 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-16T16:57:21.729 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:57:21 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:57:21.729 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:57:21 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:57:21.729 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:57:21 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T16:57:21.729 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:57:21 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:57:22.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:57:22 smithi117 ceph-mon[36730]: Detected new or changed devices on smithi045 2024-04-16T16:57:22.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:57:22 smithi117 ceph-mon[36730]: Adjusting osd_memory_target on smithi045 to 17094M 2024-04-16T16:57:22.632 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:57:22 smithi045 ceph-mon[31676]: Detected new or changed devices on smithi045 2024-04-16T16:57:22.632 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:57:22 smithi045 ceph-mon[31676]: Adjusting osd_memory_target on smithi045 to 17094M 2024-04-16T16:57:22.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:57:22 smithi104 ceph-mon[36457]: Detected new or changed devices on smithi045 2024-04-16T16:57:22.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:57:22 smithi104 ceph-mon[36457]: Adjusting osd_memory_target on smithi045 to 17094M 2024-04-16T16:57:23.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:57:23 smithi104 ceph-mon[36457]: pgmap v60: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-16T16:57:23.679 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:57:23 smithi104 ceph-mon[36457]: from='client.? 172.21.15.104:0/2234159986' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ebd450ec-08fd-4c3d-8b24-4e10634ddb7e"}]: dispatch 2024-04-16T16:57:23.679 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:57:23 smithi104 ceph-mon[36457]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ebd450ec-08fd-4c3d-8b24-4e10634ddb7e"}]: dispatch 2024-04-16T16:57:23.679 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:57:23 smithi104 ceph-mon[36457]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ebd450ec-08fd-4c3d-8b24-4e10634ddb7e"}]': finished 2024-04-16T16:57:23.679 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:57:23 smithi104 ceph-mon[36457]: osdmap e10: 2 total, 1 up, 2 in 2024-04-16T16:57:23.679 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:57:23 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-16T16:57:23.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:57:23 smithi045 ceph-mon[31676]: pgmap v60: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-16T16:57:23.729 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:57:23 smithi045 ceph-mon[31676]: from='client.? 172.21.15.104:0/2234159986' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ebd450ec-08fd-4c3d-8b24-4e10634ddb7e"}]: dispatch 2024-04-16T16:57:23.729 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:57:23 smithi045 ceph-mon[31676]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ebd450ec-08fd-4c3d-8b24-4e10634ddb7e"}]: dispatch 2024-04-16T16:57:23.729 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:57:23 smithi045 ceph-mon[31676]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ebd450ec-08fd-4c3d-8b24-4e10634ddb7e"}]': finished 2024-04-16T16:57:23.729 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:57:23 smithi045 ceph-mon[31676]: osdmap e10: 2 total, 1 up, 2 in 2024-04-16T16:57:23.729 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:57:23 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-16T16:57:23.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:57:23 smithi117 ceph-mon[36730]: pgmap v60: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-16T16:57:23.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:57:23 smithi117 ceph-mon[36730]: from='client.? 172.21.15.104:0/2234159986' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ebd450ec-08fd-4c3d-8b24-4e10634ddb7e"}]: dispatch 2024-04-16T16:57:23.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:57:23 smithi117 ceph-mon[36730]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ebd450ec-08fd-4c3d-8b24-4e10634ddb7e"}]: dispatch 2024-04-16T16:57:23.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:57:23 smithi117 ceph-mon[36730]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ebd450ec-08fd-4c3d-8b24-4e10634ddb7e"}]': finished 2024-04-16T16:57:23.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:57:23 smithi117 ceph-mon[36730]: osdmap e10: 2 total, 1 up, 2 in 2024-04-16T16:57:23.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:57:23 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-16T16:57:24.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:57:24 smithi104 ceph-mon[36457]: from='client.? 172.21.15.104:0/836237568' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-16T16:57:24.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:57:24 smithi045 ceph-mon[31676]: from='client.? 172.21.15.104:0/836237568' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-16T16:57:24.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:57:24 smithi117 ceph-mon[36730]: from='client.? 172.21.15.104:0/836237568' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-16T16:57:25.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:57:25 smithi104 ceph-mon[36457]: pgmap v62: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-16T16:57:25.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:57:25 smithi045 ceph-mon[31676]: pgmap v62: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-16T16:57:25.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:57:25 smithi117 ceph-mon[36730]: pgmap v62: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-16T16:57:27.582 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:57:27 smithi104 ceph-mon[36457]: pgmap v63: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-16T16:57:27.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:57:27 smithi045 ceph-mon[31676]: pgmap v63: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-16T16:57:27.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:57:27 smithi117 ceph-mon[36730]: pgmap v63: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-16T16:57:29.428 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:57:29 smithi104 ceph-mon[36457]: pgmap v64: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-16T16:57:29.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:57:29 smithi045 ceph-mon[31676]: pgmap v64: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-16T16:57:29.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:57:29 smithi117 ceph-mon[36730]: pgmap v64: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-16T16:57:30.597 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:57:30 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-16T16:57:30.597 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:57:30 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:57:30.598 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:57:30 smithi104 ceph-mon[36457]: Deploying daemon osd.1 on smithi104 2024-04-16T16:57:30.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:57:30 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-16T16:57:30.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:57:30 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:57:30.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:57:30 smithi045 ceph-mon[31676]: Deploying daemon osd.1 on smithi104 2024-04-16T16:57:30.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:57:30 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-16T16:57:30.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:57:30 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:57:30.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:57:30 smithi117 ceph-mon[36730]: Deploying daemon osd.1 on smithi104 2024-04-16T16:57:31.598 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:57:31 smithi104 ceph-mon[36457]: pgmap v65: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-16T16:57:31.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:57:31 smithi045 ceph-mon[31676]: pgmap v65: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-16T16:57:31.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:57:31 smithi117 ceph-mon[36730]: pgmap v65: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-16T16:57:33.428 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:57:33 smithi104 ceph-mon[36457]: pgmap v66: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-16T16:57:33.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:57:33 smithi045 ceph-mon[31676]: pgmap v66: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-16T16:57:33.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:57:33 smithi117 ceph-mon[36730]: pgmap v66: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-16T16:57:35.608 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:57:35 smithi117 ceph-mon[36730]: pgmap v67: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-16T16:57:35.608 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:57:35 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:57:35.608 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:57:35 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:57:35.608 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:57:35 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T16:57:35.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:57:35 smithi104 ceph-mon[36457]: pgmap v67: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-16T16:57:35.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:57:35 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:57:35.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:57:35 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:57:35.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:57:35 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T16:57:35.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:57:35 smithi045 ceph-mon[31676]: pgmap v67: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-16T16:57:35.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:57:35 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:57:35.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:57:35 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:57:35.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:57:35 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T16:57:37.642 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:57:37 smithi104 ceph-mon[36457]: pgmap v68: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-16T16:57:37.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:57:37 smithi045 ceph-mon[31676]: pgmap v68: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-16T16:57:37.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:57:37 smithi117 ceph-mon[36730]: pgmap v68: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-16T16:57:38.284 INFO:teuthology.orchestra.run.smithi104.stdout:Created osd(s) 1 on host 'smithi104' 2024-04-16T16:57:38.284 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-16T16:57:38.282+0000 7f0dd17fa700 1 -- 172.21.15.104:0/2638505666 <== mgr.14152 v2:172.21.15.45:6800/268030699 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f0dd4062f30 con 0x7f0dbc0a3bb0 2024-04-16T16:57:38.286 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-16T16:57:38.284+0000 7f0dda3d6700 1 -- 172.21.15.104:0/2638505666 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f0dbc0a3bb0 msgr2=0x7f0dbc0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:57:38.286 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-16T16:57:38.284+0000 7f0dda3d6700 1 --2- 172.21.15.104:0/2638505666 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f0dbc0a3bb0 0x7f0dbc0a6070 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f0dd406ae10 tx=0x7f0dc4009190 comp rx=0 tx=0).stop 2024-04-16T16:57:38.286 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-16T16:57:38.284+0000 7f0dda3d6700 1 -- 172.21.15.104:0/2638505666 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0dd4103660 msgr2=0x7f0dd419f2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:57:38.286 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-16T16:57:38.284+0000 7f0dda3d6700 1 --2- 172.21.15.104:0/2638505666 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0dd4103660 0x7f0dd419f2f0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f0dcc00b830 tx=0x7f0dcc007d90 comp rx=0 tx=0).stop 2024-04-16T16:57:38.286 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-16T16:57:38.284+0000 7f0dda3d6700 1 -- 172.21.15.104:0/2638505666 shutdown_connections 2024-04-16T16:57:38.286 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-16T16:57:38.284+0000 7f0dda3d6700 1 --2- 172.21.15.104:0/2638505666 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f0dbc0a3bb0 0x7f0dbc0a6070 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:57:38.286 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-16T16:57:38.284+0000 7f0dda3d6700 1 --2- 172.21.15.104:0/2638505666 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0dd4101f80 0x7f0dd4197d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:57:38.286 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-16T16:57:38.284+0000 7f0dda3d6700 1 --2- 172.21.15.104:0/2638505666 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0dd4103660 0x7f0dd419f2f0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:57:38.286 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-16T16:57:38.284+0000 7f0dda3d6700 1 --2- 172.21.15.104:0/2638505666 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0dd4102ca0 0x7f0dd4198270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:57:38.286 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-16T16:57:38.284+0000 7f0dda3d6700 1 -- 172.21.15.104:0/2638505666 >> 172.21.15.104:0/2638505666 conn(0x7f0dd40fc760 msgr2=0x7f0dd4105fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T16:57:38.286 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-16T16:57:38.284+0000 7f0dda3d6700 1 -- 172.21.15.104:0/2638505666 shutdown_connections 2024-04-16T16:57:38.286 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-16T16:57:38.284+0000 7f0dda3d6700 1 -- 172.21.15.104:0/2638505666 wait complete. 2024-04-16T16:57:38.900 DEBUG:teuthology.orchestra.run.smithi104:osd.1> sudo journalctl -f -n 0 -u ceph-b1fecd38-fc11-11ee-bc90-c7b262605968@osd.1.service 2024-04-16T16:57:38.902 INFO:tasks.cephadm:Deploying osd.2 on smithi117 with /dev/vg_nvme/lv_4... 2024-04-16T16:57:38.902 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b1fecd38-fc11-11ee-bc90-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-16T16:57:39.144 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/mon.c/config 2024-04-16T16:57:39.497 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:57:39 smithi117 ceph-mon[36730]: pgmap v69: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-16T16:57:39.497 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:57:39 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:57:39.497 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:57:39 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:57:39.497 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:57:39 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:57:39.497 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:57:39 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:57:39.497 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:57:39 smithi117 ceph-mon[36730]: from='osd.1 [v2:172.21.15.104:6800/1462942067,v1:172.21.15.104:6801/1462942067]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-16T16:57:39.497 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:57:39 smithi117 ceph-mon[36730]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-16T16:57:39.506 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:57:39 smithi104 ceph-mon[36457]: pgmap v69: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-16T16:57:39.506 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:57:39 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:57:39.506 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:57:39 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:57:39.506 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:57:39 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:57:39.506 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:57:39 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:57:39.506 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:57:39 smithi104 ceph-mon[36457]: from='osd.1 [v2:172.21.15.104:6800/1462942067,v1:172.21.15.104:6801/1462942067]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-16T16:57:39.507 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:57:39 smithi104 ceph-mon[36457]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-16T16:57:39.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:57:39 smithi045 ceph-mon[31676]: pgmap v69: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-16T16:57:39.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:57:39 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:57:39.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:57:39 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:57:39.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:57:39 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:57:39.729 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:57:39 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:57:39.729 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:57:39 smithi045 ceph-mon[31676]: from='osd.1 [v2:172.21.15.104:6800/1462942067,v1:172.21.15.104:6801/1462942067]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-16T16:57:39.729 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:57:39 smithi045 ceph-mon[31676]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-16T16:57:40.679 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:57:40 smithi104 ceph-mon[36457]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-16T16:57:40.679 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:57:40 smithi104 ceph-mon[36457]: from='osd.1 [v2:172.21.15.104:6800/1462942067,v1:172.21.15.104:6801/1462942067]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2024-04-16T16:57:40.679 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:57:40 smithi104 ceph-mon[36457]: osdmap e11: 2 total, 1 up, 2 in 2024-04-16T16:57:40.679 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:57:40 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-16T16:57:40.679 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:57:40 smithi104 ceph-mon[36457]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2024-04-16T16:57:40.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:57:40 smithi117 ceph-mon[36730]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-16T16:57:40.807 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:57:40 smithi117 ceph-mon[36730]: from='osd.1 [v2:172.21.15.104:6800/1462942067,v1:172.21.15.104:6801/1462942067]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2024-04-16T16:57:40.807 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:57:40 smithi117 ceph-mon[36730]: osdmap e11: 2 total, 1 up, 2 in 2024-04-16T16:57:40.807 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:57:40 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-16T16:57:40.807 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:57:40 smithi117 ceph-mon[36730]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2024-04-16T16:57:40.860 INFO:teuthology.orchestra.run.smithi117.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-16T16:57:40.860 INFO:teuthology.orchestra.run.smithi117.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-16T16:57:40.860 INFO:teuthology.orchestra.run.smithi117.stderr: stderr: 10+0 records in 2024-04-16T16:57:40.860 INFO:teuthology.orchestra.run.smithi117.stderr:10+0 records out 2024-04-16T16:57:40.860 INFO:teuthology.orchestra.run.smithi117.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0114035 s, 920 MB/s 2024-04-16T16:57:40.860 INFO:teuthology.orchestra.run.smithi117.stderr:--> Zapping successful for: 2024-04-16T16:57:40.978 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:57:40 smithi045 ceph-mon[31676]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-16T16:57:40.978 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:57:40 smithi045 ceph-mon[31676]: from='osd.1 [v2:172.21.15.104:6800/1462942067,v1:172.21.15.104:6801/1462942067]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2024-04-16T16:57:40.978 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:57:40 smithi045 ceph-mon[31676]: osdmap e11: 2 total, 1 up, 2 in 2024-04-16T16:57:40.978 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:57:40 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-16T16:57:40.978 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:57:40 smithi045 ceph-mon[31676]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2024-04-16T16:57:41.428 INFO:journalctl@ceph.osd.1.smithi104.stdout:Apr 16 16:57:41 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-1[41050]: 2024-04-16T16:57:41.038+0000 7f42cae13700 -1 osd.1 0 waiting for initial osdmap 2024-04-16T16:57:41.429 INFO:journalctl@ceph.osd.1.smithi104.stdout:Apr 16 16:57:41 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-1[41050]: 2024-04-16T16:57:41.042+0000 7f42c2bfe700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-16T16:57:41.500 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b1fecd38-fc11-11ee-bc90-c7b262605968 -- ceph orch daemon add osd smithi117:vg_nvme/lv_4 2024-04-16T16:57:41.743 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/mon.c/config 2024-04-16T16:57:41.774 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:57:41 smithi117 ceph-mon[36730]: pgmap v71: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-16T16:57:41.774 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:57:41 smithi117 ceph-mon[36730]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]': finished 2024-04-16T16:57:41.774 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:57:41 smithi117 ceph-mon[36730]: osdmap e12: 2 total, 1 up, 2 in 2024-04-16T16:57:41.774 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:57:41 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-16T16:57:41.774 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:57:41 smithi117 ceph-mon[36730]: from='osd.1 ' entity='osd.1' 2024-04-16T16:57:41.899 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:57:41 smithi104 ceph-mon[36457]: pgmap v71: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-16T16:57:41.900 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:57:41 smithi104 ceph-mon[36457]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]': finished 2024-04-16T16:57:41.900 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:57:41 smithi104 ceph-mon[36457]: osdmap e12: 2 total, 1 up, 2 in 2024-04-16T16:57:41.900 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:57:41 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-16T16:57:41.900 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:57:41 smithi104 ceph-mon[36457]: from='osd.1 ' entity='osd.1' 2024-04-16T16:57:41.978 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:57:41 smithi045 ceph-mon[31676]: pgmap v71: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-16T16:57:41.978 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:57:41 smithi045 ceph-mon[31676]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]': finished 2024-04-16T16:57:41.978 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:57:41 smithi045 ceph-mon[31676]: osdmap e12: 2 total, 1 up, 2 in 2024-04-16T16:57:41.978 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:57:41 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-16T16:57:41.978 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:57:41 smithi045 ceph-mon[31676]: from='osd.1 ' entity='osd.1' 2024-04-16T16:57:42.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:57:42 smithi117 ceph-mon[36730]: purged_snaps scrub starts 2024-04-16T16:57:42.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:57:42 smithi117 ceph-mon[36730]: purged_snaps scrub ok 2024-04-16T16:57:42.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:57:42 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-16T16:57:42.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:57:42 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:57:42.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:57:42 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:57:42.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:57:42 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-16T16:57:42.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:57:42 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:57:42.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:57:42 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:57:42.807 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:57:42 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T16:57:42.807 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:57:42 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:57:42.807 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:57:42 smithi117 ceph-mon[36730]: osd.1 [v2:172.21.15.104:6800/1462942067,v1:172.21.15.104:6801/1462942067] boot 2024-04-16T16:57:42.807 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:57:42 smithi117 ceph-mon[36730]: osdmap e13: 2 total, 2 up, 2 in 2024-04-16T16:57:42.807 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:57:42 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-16T16:57:42.807 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:57:42 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T16:57:42.807 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:57:42 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:57:42.807 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:57:42 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T16:57:42.807 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:57:42 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:57:42.929 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:57:42 smithi104 ceph-mon[36457]: purged_snaps scrub starts 2024-04-16T16:57:42.929 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:57:42 smithi104 ceph-mon[36457]: purged_snaps scrub ok 2024-04-16T16:57:42.929 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:57:42 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-16T16:57:42.929 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:57:42 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:57:42.929 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:57:42 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:57:42.929 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:57:42 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-16T16:57:42.929 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:57:42 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:57:42.929 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:57:42 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:57:42.929 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:57:42 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T16:57:42.929 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:57:42 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:57:42.929 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:57:42 smithi104 ceph-mon[36457]: osd.1 [v2:172.21.15.104:6800/1462942067,v1:172.21.15.104:6801/1462942067] boot 2024-04-16T16:57:42.929 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:57:42 smithi104 ceph-mon[36457]: osdmap e13: 2 total, 2 up, 2 in 2024-04-16T16:57:42.929 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:57:42 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-16T16:57:42.929 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:57:42 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T16:57:42.930 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:57:42 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:57:42.930 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:57:42 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T16:57:42.930 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:57:42 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:57:42.978 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:57:42 smithi045 ceph-mon[31676]: purged_snaps scrub starts 2024-04-16T16:57:42.979 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:57:42 smithi045 ceph-mon[31676]: purged_snaps scrub ok 2024-04-16T16:57:42.979 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:57:42 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-16T16:57:42.979 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:57:42 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:57:42.979 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:57:42 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:57:42.979 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:57:42 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-16T16:57:42.979 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:57:42 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:57:42.979 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:57:42 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:57:42.979 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:57:42 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T16:57:42.979 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:57:42 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:57:42.979 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:57:42 smithi045 ceph-mon[31676]: osd.1 [v2:172.21.15.104:6800/1462942067,v1:172.21.15.104:6801/1462942067] boot 2024-04-16T16:57:42.979 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:57:42 smithi045 ceph-mon[31676]: osdmap e13: 2 total, 2 up, 2 in 2024-04-16T16:57:42.979 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:57:42 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-16T16:57:42.979 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:57:42 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T16:57:42.979 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:57:42 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:57:42.979 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:57:42 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T16:57:42.980 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:57:42 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:57:43.320 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:57:43.316+0000 7fc0af0a9700 1 -- 172.21.15.117:0/4112671226 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc0a8101a70 msgr2=0x7fc0a8101e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:57:43.320 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:57:43.316+0000 7fc0af0a9700 1 --2- 172.21.15.117:0/4112671226 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc0a8101a70 0x7fc0a8101e90 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fc09c009990 tx=0x7fc09c009ca0 comp rx=0 tx=0).stop 2024-04-16T16:57:43.320 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:57:43.316+0000 7fc0af0a9700 1 -- 172.21.15.117:0/4112671226 shutdown_connections 2024-04-16T16:57:43.320 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:57:43.316+0000 7fc0af0a9700 1 --2- 172.21.15.117:0/4112671226 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc0a8101a70 0x7fc0a8101e90 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:57:43.320 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:57:43.316+0000 7fc0af0a9700 1 --2- 172.21.15.117:0/4112671226 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc0a8103610 0x7fc0a8109ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:57:43.320 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:57:43.316+0000 7fc0af0a9700 1 --2- 172.21.15.117:0/4112671226 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc0a8102c70 0x7fc0a81030d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:57:43.320 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:57:43.316+0000 7fc0af0a9700 1 -- 172.21.15.117:0/4112671226 >> 172.21.15.117:0/4112671226 conn(0x7fc0a80fd010 msgr2=0x7fc0a80ff450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T16:57:43.320 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:57:43.316+0000 7fc0af0a9700 1 -- 172.21.15.117:0/4112671226 shutdown_connections 2024-04-16T16:57:43.320 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:57:43.316+0000 7fc0af0a9700 1 -- 172.21.15.117:0/4112671226 wait complete. 2024-04-16T16:57:43.321 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:57:43.317+0000 7fc0af0a9700 1 Processor -- start 2024-04-16T16:57:43.321 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:57:43.317+0000 7fc0af0a9700 1 -- start start 2024-04-16T16:57:43.321 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:57:43.317+0000 7fc0af0a9700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc0a8101a70 0x7fc0a8197da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:57:43.321 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:57:43.317+0000 7fc0af0a9700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc0a8102c70 0x7fc0a81982e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:57:43.321 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:57:43.317+0000 7fc0af0a9700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc0a8103610 0x7fc0a819f360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:57:43.321 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:57:43.317+0000 7fc0af0a9700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fc0a810d730 con 0x7fc0a8102c70 2024-04-16T16:57:43.321 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:57:43.317+0000 7fc0af0a9700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fc0a810d5b0 con 0x7fc0a8103610 2024-04-16T16:57:43.321 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:57:43.317+0000 7fc0af0a9700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fc0a810d8b0 con 0x7fc0a8101a70 2024-04-16T16:57:43.321 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:57:43.317+0000 7fc0ace45700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc0a8101a70 0x7fc0a8197da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:57:43.321 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:57:43.317+0000 7fc0ace45700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc0a8101a70 0x7fc0a8197da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:41672/0 (socket says 172.21.15.117:41672) 2024-04-16T16:57:43.321 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:57:43.317+0000 7fc0ace45700 1 -- 172.21.15.117:0/3301018125 learned_addr learned my addr 172.21.15.117:0/3301018125 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-16T16:57:43.322 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:57:43.318+0000 7fc0ace45700 1 -- 172.21.15.117:0/3301018125 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc0a8103610 msgr2=0x7fc0a819f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:57:43.322 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:57:43.318+0000 7fc0ace45700 1 --2- 172.21.15.117:0/3301018125 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc0a8103610 0x7fc0a819f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:57:43.322 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:57:43.318+0000 7fc0ace45700 1 -- 172.21.15.117:0/3301018125 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc0a8102c70 msgr2=0x7fc0a81982e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:57:43.322 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:57:43.318+0000 7fc0ace45700 1 --2- 172.21.15.117:0/3301018125 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc0a8102c70 0x7fc0a81982e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:57:43.322 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:57:43.318+0000 7fc0ace45700 1 -- 172.21.15.117:0/3301018125 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0a819fa00 con 0x7fc0a8101a70 2024-04-16T16:57:43.322 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:57:43.318+0000 7fc0ace45700 1 --2- 172.21.15.117:0/3301018125 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc0a8101a70 0x7fc0a8197da0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fc09c0047b0 tx=0x7fc09c009f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T16:57:43.322 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:57:43.318+0000 7fc0a5ffb700 1 -- 172.21.15.117:0/3301018125 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc09c027070 con 0x7fc0a8101a70 2024-04-16T16:57:43.322 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:57:43.318+0000 7fc0a5ffb700 1 -- 172.21.15.117:0/3301018125 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc09c022470 con 0x7fc0a8101a70 2024-04-16T16:57:43.322 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:57:43.318+0000 7fc0a5ffb700 1 -- 172.21.15.117:0/3301018125 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc09c00f6c0 con 0x7fc0a8101a70 2024-04-16T16:57:43.322 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:57:43.318+0000 7fc0af0a9700 1 -- 172.21.15.117:0/3301018125 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0a819fc90 con 0x7fc0a8101a70 2024-04-16T16:57:43.322 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:57:43.318+0000 7fc0af0a9700 1 -- 172.21.15.117:0/3301018125 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0a81a0160 con 0x7fc0a8101a70 2024-04-16T16:57:43.323 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:57:43.319+0000 7fc0af0a9700 1 -- 172.21.15.117:0/3301018125 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0a80697e0 con 0x7fc0a8101a70 2024-04-16T16:57:43.323 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:57:43.320+0000 7fc0a5ffb700 1 -- 172.21.15.117:0/3301018125 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 15) ==== 138590+0+0 (secure 0 0 0) 0x7fc09c02a070 con 0x7fc0a8101a70 2024-04-16T16:57:43.324 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:57:43.321+0000 7fc0a5ffb700 1 --2- 172.21.15.117:0/3301018125 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fc0980a3bb0 0x7fc0980a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:57:43.324 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:57:43.321+0000 7fc0a5ffb700 1 -- 172.21.15.117:0/3301018125 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(13..13 src has 1..13) ==== 2309+0+0 (secure 0 0 0) 0x7fc09c0cfc70 con 0x7fc0a8101a70 2024-04-16T16:57:43.324 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:57:43.321+0000 7fc0a7fff700 1 --2- 172.21.15.117:0/3301018125 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fc0980a3bb0 0x7fc0980a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:57:43.324 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:57:43.322+0000 7fc0a7fff700 1 --2- 172.21.15.117:0/3301018125 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fc0980a3bb0 0x7fc0980a6070 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fc0940097b0 tx=0x7fc094006d20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T16:57:43.326 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:57:43.325+0000 7fc0a5ffb700 1 -- 172.21.15.117:0/3301018125 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc09c05eff0 con 0x7fc0a8101a70 2024-04-16T16:57:43.530 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:57:43.528+0000 7fc0af0a9700 1 -- 172.21.15.117:0/3301018125 --> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi117:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7fc0a8062f30 con 0x7fc0980a3bb0 2024-04-16T16:57:43.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:57:43 smithi117 ceph-mon[36730]: Detected new or changed devices on smithi104 2024-04-16T16:57:43.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:57:43 smithi117 ceph-mon[36730]: Adjusting osd_memory_target on smithi104 to 17080M 2024-04-16T16:57:43.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:57:43 smithi117 ceph-mon[36730]: pgmap v73: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-16T16:57:43.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:57:43 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-16T16:57:43.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:57:43 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-16T16:57:43.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:57:43 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:57:43.846 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:57:43 smithi104 ceph-mon[36457]: Detected new or changed devices on smithi104 2024-04-16T16:57:43.846 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:57:43 smithi104 ceph-mon[36457]: Adjusting osd_memory_target on smithi104 to 17080M 2024-04-16T16:57:43.846 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:57:43 smithi104 ceph-mon[36457]: pgmap v73: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-16T16:57:43.846 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:57:43 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-16T16:57:43.846 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:57:43 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-16T16:57:43.846 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:57:43 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:57:43.978 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:57:43 smithi045 ceph-mon[31676]: Detected new or changed devices on smithi104 2024-04-16T16:57:43.978 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:57:43 smithi045 ceph-mon[31676]: Adjusting osd_memory_target on smithi104 to 17080M 2024-04-16T16:57:43.979 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:57:43 smithi045 ceph-mon[31676]: pgmap v73: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-16T16:57:43.979 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:57:43 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-16T16:57:43.979 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:57:43 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-16T16:57:43.979 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:57:43 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:57:44.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:57:44 smithi117 ceph-mon[36730]: from='client.24154 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi117:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T16:57:44.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:57:44 smithi117 ceph-mon[36730]: osdmap e14: 2 total, 2 up, 2 in 2024-04-16T16:57:44.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:57:44 smithi104 ceph-mon[36457]: from='client.24154 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi117:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T16:57:44.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:57:44 smithi104 ceph-mon[36457]: osdmap e14: 2 total, 2 up, 2 in 2024-04-16T16:57:44.978 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:57:44 smithi045 ceph-mon[31676]: from='client.24154 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi117:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T16:57:44.978 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:57:44 smithi045 ceph-mon[31676]: osdmap e14: 2 total, 2 up, 2 in 2024-04-16T16:57:45.929 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:57:45 smithi104 ceph-mon[36457]: pgmap v75: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-16T16:57:45.978 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:57:45 smithi045 ceph-mon[31676]: pgmap v75: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-16T16:57:46.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:57:45 smithi117 ceph-mon[36730]: pgmap v75: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-16T16:57:46.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:57:46 smithi117 ceph-mon[36730]: from='client.? 172.21.15.117:0/1535690333' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b318a512-4190-42a2-8104-dcc4026ad8ad"}]: dispatch 2024-04-16T16:57:46.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:57:46 smithi117 ceph-mon[36730]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b318a512-4190-42a2-8104-dcc4026ad8ad"}]: dispatch 2024-04-16T16:57:46.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:57:46 smithi117 ceph-mon[36730]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b318a512-4190-42a2-8104-dcc4026ad8ad"}]': finished 2024-04-16T16:57:46.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:57:46 smithi117 ceph-mon[36730]: osdmap e15: 3 total, 2 up, 3 in 2024-04-16T16:57:46.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:57:46 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-16T16:57:46.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:57:46 smithi104 ceph-mon[36457]: from='client.? 172.21.15.117:0/1535690333' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b318a512-4190-42a2-8104-dcc4026ad8ad"}]: dispatch 2024-04-16T16:57:46.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:57:46 smithi104 ceph-mon[36457]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b318a512-4190-42a2-8104-dcc4026ad8ad"}]: dispatch 2024-04-16T16:57:46.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:57:46 smithi104 ceph-mon[36457]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b318a512-4190-42a2-8104-dcc4026ad8ad"}]': finished 2024-04-16T16:57:46.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:57:46 smithi104 ceph-mon[36457]: osdmap e15: 3 total, 2 up, 3 in 2024-04-16T16:57:46.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:57:46 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-16T16:57:46.978 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:57:46 smithi045 ceph-mon[31676]: from='client.? 172.21.15.117:0/1535690333' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b318a512-4190-42a2-8104-dcc4026ad8ad"}]: dispatch 2024-04-16T16:57:46.978 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:57:46 smithi045 ceph-mon[31676]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b318a512-4190-42a2-8104-dcc4026ad8ad"}]: dispatch 2024-04-16T16:57:46.978 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:57:46 smithi045 ceph-mon[31676]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b318a512-4190-42a2-8104-dcc4026ad8ad"}]': finished 2024-04-16T16:57:46.978 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:57:46 smithi045 ceph-mon[31676]: osdmap e15: 3 total, 2 up, 3 in 2024-04-16T16:57:46.978 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:57:46 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-16T16:57:47.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:57:47 smithi104 ceph-mon[36457]: pgmap v77: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-16T16:57:47.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:57:47 smithi104 ceph-mon[36457]: from='client.? 172.21.15.117:0/3534230260' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-16T16:57:47.978 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:57:47 smithi045 ceph-mon[31676]: pgmap v77: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-16T16:57:47.978 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:57:47 smithi045 ceph-mon[31676]: from='client.? 172.21.15.117:0/3534230260' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-16T16:57:48.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:57:47 smithi117 ceph-mon[36730]: pgmap v77: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-16T16:57:48.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:57:47 smithi117 ceph-mon[36730]: from='client.? 172.21.15.117:0/3534230260' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-16T16:57:49.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:57:49 smithi104 ceph-mon[36457]: pgmap v79: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-16T16:57:49.978 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:57:49 smithi045 ceph-mon[31676]: pgmap v79: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-16T16:57:50.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:57:49 smithi117 ceph-mon[36730]: pgmap v79: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-16T16:57:51.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:57:51 smithi104 ceph-mon[36457]: pgmap v80: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-16T16:57:51.975 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:57:51 smithi117 ceph-mon[36730]: pgmap v80: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-16T16:57:51.978 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:57:51 smithi045 ceph-mon[31676]: pgmap v80: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-16T16:57:53.919 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:57:53 smithi117 ceph-mon[36730]: pgmap v81: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-16T16:57:53.919 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:57:53 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-16T16:57:53.920 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:57:53 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:57:53.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:57:53 smithi104 ceph-mon[36457]: pgmap v81: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-16T16:57:53.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:57:53 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-16T16:57:53.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:57:53 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:57:53.978 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:57:53 smithi045 ceph-mon[31676]: pgmap v81: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-16T16:57:53.978 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:57:53 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-16T16:57:53.978 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:57:53 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:57:54.912 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:57:54 smithi117 ceph-mon[36730]: Deploying daemon osd.2 on smithi117 2024-04-16T16:57:54.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:57:54 smithi104 ceph-mon[36457]: Deploying daemon osd.2 on smithi117 2024-04-16T16:57:54.978 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:57:54 smithi045 ceph-mon[31676]: Deploying daemon osd.2 on smithi117 2024-04-16T16:57:55.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:57:55 smithi104 ceph-mon[36457]: pgmap v82: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-16T16:57:55.953 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:57:55 smithi117 ceph-mon[36730]: pgmap v82: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-16T16:57:55.978 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:57:55 smithi045 ceph-mon[31676]: pgmap v82: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-16T16:57:57.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:57:57 smithi117 ceph-mon[36730]: pgmap v83: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-16T16:57:57.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:57:57 smithi104 ceph-mon[36457]: pgmap v83: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-16T16:57:57.978 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:57:57 smithi045 ceph-mon[31676]: pgmap v83: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-16T16:57:59.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:57:59 smithi045 ceph-mon[31676]: pgmap v84: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-16T16:57:59.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:57:59 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:57:59.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:57:59 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:57:59.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:57:59 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T16:57:59.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:57:59 smithi117 ceph-mon[36730]: pgmap v84: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-16T16:57:59.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:57:59 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:57:59.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:57:59 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:57:59.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:57:59 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T16:57:59.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:57:59 smithi104 ceph-mon[36457]: pgmap v84: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-16T16:57:59.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:57:59 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:57:59.929 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:57:59 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:57:59.929 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:57:59 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T16:58:01.666 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:58:01 smithi117 ceph-mon[36730]: pgmap v85: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-16T16:58:01.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:58:01 smithi045 ceph-mon[31676]: pgmap v85: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-16T16:58:01.761 INFO:teuthology.orchestra.run.smithi117.stdout:Created osd(s) 2 on host 'smithi117' 2024-04-16T16:58:01.761 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:58:01.759+0000 7fc0a5ffb700 1 -- 172.21.15.117:0/3301018125 <== mgr.14152 v2:172.21.15.45:6800/268030699 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fc0a8062f30 con 0x7fc0980a3bb0 2024-04-16T16:58:01.763 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:58:01.761+0000 7fc0af0a9700 1 -- 172.21.15.117:0/3301018125 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fc0980a3bb0 msgr2=0x7fc0980a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:58:01.763 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:58:01.761+0000 7fc0af0a9700 1 --2- 172.21.15.117:0/3301018125 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fc0980a3bb0 0x7fc0980a6070 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fc0940097b0 tx=0x7fc094006d20 comp rx=0 tx=0).stop 2024-04-16T16:58:01.763 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:58:01.761+0000 7fc0af0a9700 1 -- 172.21.15.117:0/3301018125 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc0a8101a70 msgr2=0x7fc0a8197da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:58:01.763 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:58:01.761+0000 7fc0af0a9700 1 --2- 172.21.15.117:0/3301018125 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc0a8101a70 0x7fc0a8197da0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fc09c0047b0 tx=0x7fc09c009f90 comp rx=0 tx=0).stop 2024-04-16T16:58:01.763 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:58:01.761+0000 7fc0af0a9700 1 -- 172.21.15.117:0/3301018125 shutdown_connections 2024-04-16T16:58:01.763 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:58:01.762+0000 7fc0af0a9700 1 --2- 172.21.15.117:0/3301018125 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fc0980a3bb0 0x7fc0980a6070 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:58:01.763 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:58:01.762+0000 7fc0af0a9700 1 --2- 172.21.15.117:0/3301018125 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc0a8101a70 0x7fc0a8197da0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:58:01.763 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:58:01.762+0000 7fc0af0a9700 1 --2- 172.21.15.117:0/3301018125 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc0a8103610 0x7fc0a819f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:58:01.763 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:58:01.762+0000 7fc0af0a9700 1 --2- 172.21.15.117:0/3301018125 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc0a8102c70 0x7fc0a81982e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:58:01.763 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:58:01.762+0000 7fc0af0a9700 1 -- 172.21.15.117:0/3301018125 >> 172.21.15.117:0/3301018125 conn(0x7fc0a80fd010 msgr2=0x7fc0a8105f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T16:58:01.764 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:58:01.762+0000 7fc0af0a9700 1 -- 172.21.15.117:0/3301018125 shutdown_connections 2024-04-16T16:58:01.764 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:58:01.762+0000 7fc0af0a9700 1 -- 172.21.15.117:0/3301018125 wait complete. 2024-04-16T16:58:01.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:58:01 smithi104 ceph-mon[36457]: pgmap v85: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-16T16:58:02.358 DEBUG:teuthology.orchestra.run.smithi117:osd.2> sudo journalctl -f -n 0 -u ceph-b1fecd38-fc11-11ee-bc90-c7b262605968@osd.2.service 2024-04-16T16:58:02.360 INFO:tasks.cephadm:Deploying osd.3 on smithi045 with /dev/vg_nvme/lv_3... 2024-04-16T16:58:02.360 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b1fecd38-fc11-11ee-bc90-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-16T16:58:02.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:58:02 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:58:02.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:58:02 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:58:02.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:58:02 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:58:02.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:58:02 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:58:02.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:58:02 smithi045 ceph-mon[31676]: from='osd.2 [v2:172.21.15.117:6800/2829260735,v1:172.21.15.117:6801/2829260735]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-16T16:58:02.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:58:02 smithi045 ceph-mon[31676]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-16T16:58:02.818 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/mon.a/config 2024-04-16T16:58:03.014 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:58:02 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:58:03.015 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:58:02 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:58:03.015 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:58:02 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:58:03.015 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:58:02 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:58:03.015 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:58:02 smithi117 ceph-mon[36730]: from='osd.2 [v2:172.21.15.117:6800/2829260735,v1:172.21.15.117:6801/2829260735]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-16T16:58:03.015 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:58:02 smithi117 ceph-mon[36730]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-16T16:58:03.178 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:58:02 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:58:03.179 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:58:02 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:58:03.179 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:58:02 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:58:03.179 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:58:02 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:58:03.179 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:58:02 smithi104 ceph-mon[36457]: from='osd.2 [v2:172.21.15.117:6800/2829260735,v1:172.21.15.117:6801/2829260735]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-16T16:58:03.179 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:58:02 smithi104 ceph-mon[36457]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-16T16:58:03.948 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:58:03 smithi045 ceph-mon[31676]: pgmap v86: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-16T16:58:03.948 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:58:03 smithi045 ceph-mon[31676]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-16T16:58:03.948 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:58:03 smithi045 ceph-mon[31676]: osdmap e16: 3 total, 2 up, 3 in 2024-04-16T16:58:03.948 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:58:03 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-16T16:58:03.948 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:58:03 smithi045 ceph-mon[31676]: from='osd.2 [v2:172.21.15.117:6800/2829260735,v1:172.21.15.117:6801/2829260735]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]: dispatch 2024-04-16T16:58:03.948 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:58:03 smithi045 ceph-mon[31676]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]: dispatch 2024-04-16T16:58:03.994 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:58:03 smithi117 ceph-mon[36730]: pgmap v86: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-16T16:58:03.994 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:58:03 smithi117 ceph-mon[36730]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-16T16:58:03.994 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:58:03 smithi117 ceph-mon[36730]: osdmap e16: 3 total, 2 up, 3 in 2024-04-16T16:58:03.994 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:58:03 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-16T16:58:03.994 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:58:03 smithi117 ceph-mon[36730]: from='osd.2 [v2:172.21.15.117:6800/2829260735,v1:172.21.15.117:6801/2829260735]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]: dispatch 2024-04-16T16:58:03.994 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:58:03 smithi117 ceph-mon[36730]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]: dispatch 2024-04-16T16:58:04.178 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:58:03 smithi104 ceph-mon[36457]: pgmap v86: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-16T16:58:04.178 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:58:03 smithi104 ceph-mon[36457]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-16T16:58:04.179 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:58:03 smithi104 ceph-mon[36457]: osdmap e16: 3 total, 2 up, 3 in 2024-04-16T16:58:04.179 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:58:03 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-16T16:58:04.179 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:58:03 smithi104 ceph-mon[36457]: from='osd.2 [v2:172.21.15.117:6800/2829260735,v1:172.21.15.117:6801/2829260735]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]: dispatch 2024-04-16T16:58:04.179 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:58:03 smithi104 ceph-mon[36457]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]: dispatch 2024-04-16T16:58:04.306 INFO:journalctl@ceph.osd.2.smithi117.stdout:Apr 16 16:58:04 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-2[41881]: 2024-04-16T16:58:04.059+0000 7fb2a5b1e700 -1 osd.2 0 waiting for initial osdmap 2024-04-16T16:58:04.306 INFO:journalctl@ceph.osd.2.smithi117.stdout:Apr 16 16:58:04 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-2[41881]: 2024-04-16T16:58:04.063+0000 7fb2a1911700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-16T16:58:05.170 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:58:05 smithi045 ceph-mon[31676]: pgmap v88: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-16T16:58:05.170 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:58:05 smithi045 ceph-mon[31676]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]': finished 2024-04-16T16:58:05.170 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:58:05 smithi045 ceph-mon[31676]: osdmap e17: 3 total, 2 up, 3 in 2024-04-16T16:58:05.170 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:58:05 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-16T16:58:05.170 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:58:05 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-16T16:58:05.428 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:58:05 smithi104 ceph-mon[36457]: pgmap v88: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-16T16:58:05.428 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:58:05 smithi104 ceph-mon[36457]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]': finished 2024-04-16T16:58:05.428 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:58:05 smithi104 ceph-mon[36457]: osdmap e17: 3 total, 2 up, 3 in 2024-04-16T16:58:05.428 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:58:05 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-16T16:58:05.428 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:58:05 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-16T16:58:05.467 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:58:05 smithi117 ceph-mon[36730]: pgmap v88: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-16T16:58:05.467 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:58:05 smithi117 ceph-mon[36730]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]': finished 2024-04-16T16:58:05.467 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:58:05 smithi117 ceph-mon[36730]: osdmap e17: 3 total, 2 up, 3 in 2024-04-16T16:58:05.467 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:58:05 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-16T16:58:05.467 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:58:05 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-16T16:58:05.909 INFO:teuthology.orchestra.run.smithi045.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-16T16:58:05.909 INFO:teuthology.orchestra.run.smithi045.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-16T16:58:05.909 INFO:teuthology.orchestra.run.smithi045.stderr: stderr: 10+0 records in 2024-04-16T16:58:05.909 INFO:teuthology.orchestra.run.smithi045.stderr:10+0 records out 2024-04-16T16:58:05.909 INFO:teuthology.orchestra.run.smithi045.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0397478 s, 264 MB/s 2024-04-16T16:58:05.909 INFO:teuthology.orchestra.run.smithi045.stderr:--> Zapping successful for: 2024-04-16T16:58:06.229 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:58:06 smithi045 ceph-mon[31676]: purged_snaps scrub starts 2024-04-16T16:58:06.229 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:58:06 smithi045 ceph-mon[31676]: purged_snaps scrub ok 2024-04-16T16:58:06.229 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:58:06 smithi045 ceph-mon[31676]: osd.2 [v2:172.21.15.117:6800/2829260735,v1:172.21.15.117:6801/2829260735] boot 2024-04-16T16:58:06.229 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:58:06 smithi045 ceph-mon[31676]: osdmap e18: 3 total, 3 up, 3 in 2024-04-16T16:58:06.229 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:58:06 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-16T16:58:06.229 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:58:06 smithi045 ceph-mon[31676]: Detected new or changed devices on smithi117 2024-04-16T16:58:06.229 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:58:06 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:58:06.229 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:58:06 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:58:06.229 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:58:06 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-16T16:58:06.229 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:58:06 smithi045 ceph-mon[31676]: Adjusting osd_memory_target on smithi117 to 17080M 2024-04-16T16:58:06.229 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:58:06 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:58:06.229 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:58:06 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:58:06.229 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:58:06 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T16:58:06.230 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:58:06 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:58:06.230 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:58:06 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T16:58:06.230 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:58:06 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:58:06.230 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:58:06 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T16:58:06.230 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:58:06 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:58:06.306 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:58:06 smithi117 ceph-mon[36730]: purged_snaps scrub starts 2024-04-16T16:58:06.306 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:58:06 smithi117 ceph-mon[36730]: purged_snaps scrub ok 2024-04-16T16:58:06.306 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:58:06 smithi117 ceph-mon[36730]: osd.2 [v2:172.21.15.117:6800/2829260735,v1:172.21.15.117:6801/2829260735] boot 2024-04-16T16:58:06.307 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:58:06 smithi117 ceph-mon[36730]: osdmap e18: 3 total, 3 up, 3 in 2024-04-16T16:58:06.307 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:58:06 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-16T16:58:06.307 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:58:06 smithi117 ceph-mon[36730]: Detected new or changed devices on smithi117 2024-04-16T16:58:06.307 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:58:06 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:58:06.307 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:58:06 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:58:06.307 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:58:06 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-16T16:58:06.307 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:58:06 smithi117 ceph-mon[36730]: Adjusting osd_memory_target on smithi117 to 17080M 2024-04-16T16:58:06.307 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:58:06 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:58:06.307 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:58:06 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:58:06.307 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:58:06 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T16:58:06.307 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:58:06 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:58:06.307 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:58:06 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T16:58:06.307 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:58:06 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:58:06.307 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:58:06 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T16:58:06.307 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:58:06 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:58:06.428 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:58:06 smithi104 ceph-mon[36457]: purged_snaps scrub starts 2024-04-16T16:58:06.428 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:58:06 smithi104 ceph-mon[36457]: purged_snaps scrub ok 2024-04-16T16:58:06.428 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:58:06 smithi104 ceph-mon[36457]: osd.2 [v2:172.21.15.117:6800/2829260735,v1:172.21.15.117:6801/2829260735] boot 2024-04-16T16:58:06.429 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:58:06 smithi104 ceph-mon[36457]: osdmap e18: 3 total, 3 up, 3 in 2024-04-16T16:58:06.429 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:58:06 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-16T16:58:06.429 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:58:06 smithi104 ceph-mon[36457]: Detected new or changed devices on smithi117 2024-04-16T16:58:06.429 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:58:06 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:58:06.429 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:58:06 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:58:06.429 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:58:06 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-16T16:58:06.429 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:58:06 smithi104 ceph-mon[36457]: Adjusting osd_memory_target on smithi117 to 17080M 2024-04-16T16:58:06.429 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:58:06 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:58:06.429 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:58:06 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:58:06.429 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:58:06 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T16:58:06.429 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:58:06 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:58:06.429 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:58:06 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T16:58:06.429 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:58:06 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:58:06.429 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:58:06 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T16:58:06.429 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:58:06 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:58:06.841 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b1fecd38-fc11-11ee-bc90-c7b262605968 -- ceph orch daemon add osd smithi045:vg_nvme/lv_3 2024-04-16T16:58:07.315 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/mon.a/config 2024-04-16T16:58:07.428 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:58:07 smithi104 ceph-mon[36457]: pgmap v91: 0 pgs: ; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-16T16:58:07.428 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:58:07 smithi104 ceph-mon[36457]: osdmap e19: 3 total, 3 up, 3 in 2024-04-16T16:58:07.428 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:58:07 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-16T16:58:07.470 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:58:07 smithi045 ceph-mon[31676]: pgmap v91: 0 pgs: ; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-16T16:58:07.470 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:58:07 smithi045 ceph-mon[31676]: osdmap e19: 3 total, 3 up, 3 in 2024-04-16T16:58:07.470 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:58:07 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-16T16:58:07.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:58:07 smithi117 ceph-mon[36730]: pgmap v91: 0 pgs: ; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-16T16:58:07.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:58:07 smithi117 ceph-mon[36730]: osdmap e19: 3 total, 3 up, 3 in 2024-04-16T16:58:07.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:58:07 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-16T16:58:08.428 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:58:08 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-16T16:58:08.428 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:58:08 smithi104 ceph-mon[36457]: osdmap e20: 3 total, 3 up, 3 in 2024-04-16T16:58:08.429 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:58:08 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-16T16:58:08.475 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:58:08 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-16T16:58:08.475 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:58:08 smithi045 ceph-mon[31676]: osdmap e20: 3 total, 3 up, 3 in 2024-04-16T16:58:08.475 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:58:08 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-16T16:58:08.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:58:08 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-16T16:58:08.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:58:08 smithi117 ceph-mon[36730]: osdmap e20: 3 total, 3 up, 3 in 2024-04-16T16:58:08.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:58:08 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-16T16:58:08.729 INFO:journalctl@ceph.osd.0.smithi045.stdout:Apr 16 16:58:08 smithi045 sudo[51037]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-16T16:58:08.729 INFO:journalctl@ceph.osd.0.smithi045.stdout:Apr 16 16:58:08 smithi045 sudo[51037]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-16T16:58:08.729 INFO:journalctl@ceph.osd.0.smithi045.stdout:Apr 16 16:58:08 smithi045 sudo[51037]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-16T16:58:08.729 INFO:journalctl@ceph.osd.0.smithi045.stdout:Apr 16 16:58:08 smithi045 sudo[51037]: pam_unix(sudo:session): session closed for user root 2024-04-16T16:58:08.729 INFO:journalctl@ceph.osd.0.smithi045.stdout:Apr 16 16:58:08 smithi045 sudo[51040]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-16T16:58:08.729 INFO:journalctl@ceph.osd.0.smithi045.stdout:Apr 16 16:58:08 smithi045 sudo[51040]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-16T16:58:08.729 INFO:journalctl@ceph.osd.0.smithi045.stdout:Apr 16 16:58:08 smithi045 sudo[51040]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-16T16:58:08.729 INFO:journalctl@ceph.osd.0.smithi045.stdout:Apr 16 16:58:08 smithi045 sudo[51040]: pam_unix(sudo:session): session closed for user root 2024-04-16T16:58:08.917 INFO:journalctl@ceph.osd.1.smithi104.stdout:Apr 16 16:58:08 smithi104 sudo[45216]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-16T16:58:08.917 INFO:journalctl@ceph.osd.1.smithi104.stdout:Apr 16 16:58:08 smithi104 sudo[45216]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-16T16:58:08.917 INFO:journalctl@ceph.osd.1.smithi104.stdout:Apr 16 16:58:08 smithi104 sudo[45216]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-16T16:58:08.917 INFO:journalctl@ceph.osd.1.smithi104.stdout:Apr 16 16:58:08 smithi104 sudo[45216]: pam_unix(sudo:session): session closed for user root 2024-04-16T16:58:08.917 INFO:journalctl@ceph.osd.1.smithi104.stdout:Apr 16 16:58:08 smithi104 sudo[45219]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-16T16:58:08.918 INFO:journalctl@ceph.osd.1.smithi104.stdout:Apr 16 16:58:08 smithi104 sudo[45219]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-16T16:58:08.918 INFO:journalctl@ceph.osd.1.smithi104.stdout:Apr 16 16:58:08 smithi104 sudo[45219]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-16T16:58:08.918 INFO:journalctl@ceph.osd.1.smithi104.stdout:Apr 16 16:58:08 smithi104 sudo[45219]: pam_unix(sudo:session): session closed for user root 2024-04-16T16:58:09.056 INFO:journalctl@ceph.osd.2.smithi117.stdout:Apr 16 16:58:08 smithi117 sudo[45943]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-16T16:58:09.056 INFO:journalctl@ceph.osd.2.smithi117.stdout:Apr 16 16:58:08 smithi117 sudo[45943]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-16T16:58:09.056 INFO:journalctl@ceph.osd.2.smithi117.stdout:Apr 16 16:58:08 smithi117 sudo[45943]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-16T16:58:09.056 INFO:journalctl@ceph.osd.2.smithi117.stdout:Apr 16 16:58:08 smithi117 sudo[45943]: pam_unix(sudo:session): session closed for user root 2024-04-16T16:58:09.056 INFO:journalctl@ceph.osd.2.smithi117.stdout:Apr 16 16:58:08 smithi117 sudo[45946]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-16T16:58:09.056 INFO:journalctl@ceph.osd.2.smithi117.stdout:Apr 16 16:58:08 smithi117 sudo[45946]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-16T16:58:09.057 INFO:journalctl@ceph.osd.2.smithi117.stdout:Apr 16 16:58:08 smithi117 sudo[45946]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-16T16:58:09.057 INFO:journalctl@ceph.osd.2.smithi117.stdout:Apr 16 16:58:08 smithi117 sudo[45946]: pam_unix(sudo:session): session closed for user root 2024-04-16T16:58:09.057 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:58:08 smithi117 sudo[45949]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-16T16:58:09.057 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:58:08 smithi117 sudo[45949]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-16T16:58:09.057 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:58:08 smithi117 sudo[45949]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-16T16:58:09.057 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:58:08 smithi117 sudo[45949]: pam_unix(sudo:session): session closed for user root 2024-04-16T16:58:09.057 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:58:08 smithi117 sudo[45952]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-16T16:58:09.057 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:58:08 smithi117 sudo[45952]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-16T16:58:09.057 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:58:08 smithi117 sudo[45952]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-16T16:58:09.057 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:58:08 smithi117 sudo[45952]: pam_unix(sudo:session): session closed for user root 2024-04-16T16:58:09.084 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:58:08 smithi045 sudo[51043]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-16T16:58:09.084 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:58:08 smithi045 sudo[51043]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-16T16:58:09.084 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:58:08 smithi045 sudo[51043]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-16T16:58:09.084 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:58:08 smithi045 sudo[51043]: pam_unix(sudo:session): session closed for user root 2024-04-16T16:58:09.084 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:58:08 smithi045 sudo[51048]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-16T16:58:09.084 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:58:08 smithi045 sudo[51048]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-16T16:58:09.084 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:58:08 smithi045 sudo[51048]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-16T16:58:09.084 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:58:08 smithi045 sudo[51048]: pam_unix(sudo:session): session closed for user root 2024-04-16T16:58:09.179 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:58:08 smithi104 sudo[45222]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-16T16:58:09.179 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:58:08 smithi104 sudo[45222]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-16T16:58:09.179 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:58:08 smithi104 sudo[45222]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-16T16:58:09.179 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:58:08 smithi104 sudo[45222]: pam_unix(sudo:session): session closed for user root 2024-04-16T16:58:09.179 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:58:08 smithi104 sudo[45225]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-16T16:58:09.179 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:58:08 smithi104 sudo[45225]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-16T16:58:09.179 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:58:08 smithi104 sudo[45225]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-16T16:58:09.179 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:58:08 smithi104 sudo[45225]: pam_unix(sudo:session): session closed for user root 2024-04-16T16:58:09.179 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:58:09 smithi104 ceph-mon[36457]: pgmap v94: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-16T16:58:09.179 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:58:09 smithi104 ceph-mon[36457]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-16T16:58:09.180 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:58:09 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-16T16:58:09.180 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:58:09 smithi104 ceph-mon[36457]: osdmap e21: 3 total, 3 up, 3 in 2024-04-16T16:58:09.180 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:58:09 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:58:09.180 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:58:09 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:58:09.180 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:58:09 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:58:09.180 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:58:09 smithi104 ceph-mon[36457]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-16T16:58:09.180 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:58:09 smithi104 ceph-mon[36457]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-16T16:58:09.180 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:58:09 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-16T16:58:09.180 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:58:09 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-16T16:58:09.180 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:58:09 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-16T16:58:09.180 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:58:09 smithi104 ceph-mon[36457]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-16T16:58:09.180 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:58:09 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-16T16:58:09.181 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:58:09 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-16T16:58:09.181 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:58:09 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-16T16:58:09.181 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:58:09 smithi104 ceph-mon[36457]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-16T16:58:09.181 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:58:09 smithi104 ceph-mon[36457]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-16T16:58:09.181 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:58:09 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-16T16:58:09.181 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:58:09 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-16T16:58:09.181 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:58:09 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-16T16:58:09.181 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:58:09 smithi104 ceph-mon[36457]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-16T16:58:09.352 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:58:09 smithi045 ceph-mon[31676]: pgmap v94: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-16T16:58:09.352 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:58:09 smithi045 ceph-mon[31676]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-16T16:58:09.352 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:58:09 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-16T16:58:09.353 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:58:09 smithi045 ceph-mon[31676]: osdmap e21: 3 total, 3 up, 3 in 2024-04-16T16:58:09.353 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:58:09 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:58:09.353 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:58:09 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:58:09.353 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:58:09 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:58:09.353 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:58:09 smithi045 ceph-mon[31676]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-16T16:58:09.353 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:58:09 smithi045 ceph-mon[31676]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-16T16:58:09.353 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:58:09 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-16T16:58:09.353 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:58:09 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-16T16:58:09.353 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:58:09 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-16T16:58:09.353 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:58:09 smithi045 ceph-mon[31676]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-16T16:58:09.353 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:58:09 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-16T16:58:09.353 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:58:09 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-16T16:58:09.353 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:58:09 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-16T16:58:09.353 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:58:09 smithi045 ceph-mon[31676]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-16T16:58:09.353 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:58:09 smithi045 ceph-mon[31676]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-16T16:58:09.354 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:58:09 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-16T16:58:09.354 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:58:09 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-16T16:58:09.354 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:58:09 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-16T16:58:09.354 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:58:09 smithi045 ceph-mon[31676]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-16T16:58:09.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:58:09 smithi117 ceph-mon[36730]: pgmap v94: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-16T16:58:09.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:58:09 smithi117 ceph-mon[36730]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-16T16:58:09.557 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:58:09 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-16T16:58:09.557 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:58:09 smithi117 ceph-mon[36730]: osdmap e21: 3 total, 3 up, 3 in 2024-04-16T16:58:09.557 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:58:09 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:58:09.557 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:58:09 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:58:09.557 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:58:09 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:58:09.557 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:58:09 smithi117 ceph-mon[36730]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-16T16:58:09.557 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:58:09 smithi117 ceph-mon[36730]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-16T16:58:09.557 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:58:09 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-16T16:58:09.557 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:58:09 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-16T16:58:09.557 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:58:09 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-16T16:58:09.557 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:58:09 smithi117 ceph-mon[36730]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-16T16:58:09.557 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:58:09 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-16T16:58:09.557 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:58:09 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-16T16:58:09.557 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:58:09 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-16T16:58:09.558 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:58:09 smithi117 ceph-mon[36730]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-16T16:58:09.558 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:58:09 smithi117 ceph-mon[36730]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-16T16:58:09.558 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:58:09 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-16T16:58:09.558 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:58:09 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-16T16:58:09.558 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:58:09 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-16T16:58:09.558 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:58:09 smithi117 ceph-mon[36730]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-16T16:58:09.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:58:09.648+0000 7f4828c00700 1 -- 172.21.15.45:0/3970811821 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f482410a970 msgr2=0x7f482410ce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:58:09.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:58:09.648+0000 7f4828c00700 1 --2- 172.21.15.45:0/3970811821 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f482410a970 0x7f482410ce00 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f481c00b0a0 tx=0x7f481c00b3b0 comp rx=0 tx=0).stop 2024-04-16T16:58:09.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:58:09.648+0000 7f4828c00700 1 -- 172.21.15.45:0/3970811821 shutdown_connections 2024-04-16T16:58:09.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:58:09.648+0000 7f4828c00700 1 --2- 172.21.15.45:0/3970811821 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f482410a970 0x7f482410ce00 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:58:09.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:58:09.648+0000 7f4828c00700 1 --2- 172.21.15.45:0/3970811821 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4824076ac0 0x7f4824074f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:58:09.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:58:09.648+0000 7f4828c00700 1 --2- 172.21.15.45:0/3970811821 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f48240754f0 0x7f4824075970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:58:09.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:58:09.648+0000 7f4828c00700 1 -- 172.21.15.45:0/3970811821 >> 172.21.15.45:0/3970811821 conn(0x7f48240fe030 msgr2=0x7f4824100470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T16:58:09.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:58:09.648+0000 7f4828c00700 1 -- 172.21.15.45:0/3970811821 shutdown_connections 2024-04-16T16:58:09.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:58:09.648+0000 7f4828c00700 1 -- 172.21.15.45:0/3970811821 wait complete. 2024-04-16T16:58:09.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:58:09.649+0000 7f4828c00700 1 Processor -- start 2024-04-16T16:58:09.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:58:09.650+0000 7f4828c00700 1 -- start start 2024-04-16T16:58:09.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:58:09.650+0000 7f4828c00700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f48240754f0 0x7f482419ae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:58:09.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:58:09.650+0000 7f4828c00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4824076ac0 0x7f482419b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:58:09.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:58:09.650+0000 7f4828c00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f482410a970 0x7f48241a23d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:58:09.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:58:09.650+0000 7f4828c00700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f48241108b0 con 0x7f482410a970 2024-04-16T16:58:09.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:58:09.650+0000 7f4828c00700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f4824110730 con 0x7f48240754f0 2024-04-16T16:58:09.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:58:09.650+0000 7f4828c00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f4824110a30 con 0x7f4824076ac0 2024-04-16T16:58:09.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:58:09.650+0000 7f4823400700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f48240754f0 0x7f482419ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:58:09.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:58:09.650+0000 7f481be00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4824076ac0 0x7f482419b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:58:09.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:58:09.650+0000 7f4823400700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f48240754f0 0x7f482419ae60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:39584/0 (socket says 172.21.15.45:39584) 2024-04-16T16:58:09.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:58:09.650+0000 7f4823400700 1 -- 172.21.15.45:0/2824558918 learned_addr learned my addr 172.21.15.45:0/2824558918 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T16:58:09.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:58:09.651+0000 7f4823e00700 1 --2- 172.21.15.45:0/2824558918 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f482410a970 0x7f48241a23d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:58:09.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:58:09.651+0000 7f4823400700 1 -- 172.21.15.45:0/2824558918 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4824076ac0 msgr2=0x7f482419b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:58:09.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:58:09.651+0000 7f4823400700 1 --2- 172.21.15.45:0/2824558918 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4824076ac0 0x7f482419b3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:58:09.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:58:09.651+0000 7f4823400700 1 -- 172.21.15.45:0/2824558918 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f482410a970 msgr2=0x7f48241a23d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:58:09.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:58:09.651+0000 7f4823400700 1 --2- 172.21.15.45:0/2824558918 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f482410a970 0x7f48241a23d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:58:09.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:58:09.651+0000 7f4823400700 1 -- 172.21.15.45:0/2824558918 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48241a2a70 con 0x7f48240754f0 2024-04-16T16:58:09.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:58:09.651+0000 7f4823400700 1 --2- 172.21.15.45:0/2824558918 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f48240754f0 0x7f482419ae60 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f481400e9a0 tx=0x7f481400ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T16:58:09.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:58:09.651+0000 7f4820c00700 1 -- 172.21.15.45:0/2824558918 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f481400cbd0 con 0x7f48240754f0 2024-04-16T16:58:09.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:58:09.652+0000 7f4820c00700 1 -- 172.21.15.45:0/2824558918 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f481400cd70 con 0x7f48240754f0 2024-04-16T16:58:09.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:58:09.652+0000 7f4820c00700 1 -- 172.21.15.45:0/2824558918 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48140187a0 con 0x7f48240754f0 2024-04-16T16:58:09.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:58:09.652+0000 7f4828c00700 1 -- 172.21.15.45:0/2824558918 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48241a2d60 con 0x7f48240754f0 2024-04-16T16:58:09.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:58:09.652+0000 7f4828c00700 1 -- 172.21.15.45:0/2824558918 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48241a3200 con 0x7f48240754f0 2024-04-16T16:58:09.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:58:09.653+0000 7f4828c00700 1 -- 172.21.15.45:0/2824558918 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48240697e0 con 0x7f48240754f0 2024-04-16T16:58:09.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:58:09.655+0000 7f4820c00700 1 -- 172.21.15.45:0/2824558918 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 15) ==== 138590+0+0 (secure 0 0 0) 0x7f48140109a0 con 0x7f48240754f0 2024-04-16T16:58:09.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:58:09.656+0000 7f4820c00700 1 --2- 172.21.15.45:0/2824558918 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f48080a3c30 0x7f48080a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:58:09.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:58:09.656+0000 7f4820c00700 1 -- 172.21.15.45:0/2824558918 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 3237+0+0 (secure 0 0 0) 0x7f48140cf100 con 0x7f48240754f0 2024-04-16T16:58:09.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:58:09.657+0000 7f481be00700 1 --2- 172.21.15.45:0/2824558918 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f48080a3c30 0x7f48080a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:58:09.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:58:09.657+0000 7f481be00700 1 --2- 172.21.15.45:0/2824558918 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f48080a3c30 0x7f48080a60f0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f480c009c60 tx=0x7f480c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T16:58:09.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:58:09.659+0000 7f4820c00700 1 -- 172.21.15.45:0/2824558918 <== mon.2 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f481409cf80 con 0x7f48240754f0 2024-04-16T16:58:09.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:58:09.904+0000 7f4828c00700 1 -- 172.21.15.45:0/2824558918 --> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi045:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f4824062f30 con 0x7f48080a3c30 2024-04-16T16:58:10.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:58:10.141+0000 7f4820c00700 1 -- 172.21.15.45:0/2824558918 <== mon.2 v2:172.21.15.104:3300/0 7 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f4814098ef0 con 0x7f48240754f0 2024-04-16T16:58:10.228 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:58:10 smithi045 ceph-mon[31676]: osdmap e22: 3 total, 3 up, 3 in 2024-04-16T16:58:10.228 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:58:10 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-16T16:58:10.228 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:58:10 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-16T16:58:10.228 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:58:10 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:58:10.428 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:58:10 smithi104 ceph-mon[36457]: osdmap e22: 3 total, 3 up, 3 in 2024-04-16T16:58:10.429 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:58:10 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-16T16:58:10.429 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:58:10 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-16T16:58:10.429 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:58:10 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:58:10.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:58:10 smithi117 ceph-mon[36730]: osdmap e22: 3 total, 3 up, 3 in 2024-04-16T16:58:10.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:58:10 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-16T16:58:10.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:58:10 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-16T16:58:10.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:58:10 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:58:11.228 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:58:11 smithi045 ceph-mon[31676]: from='client.24194 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi045:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T16:58:11.228 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:58:11 smithi045 ceph-mon[31676]: pgmap v97: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-16T16:58:11.229 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:58:11 smithi045 ceph-mon[31676]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-16T16:58:11.229 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:58:11 smithi045 ceph-mon[31676]: Cluster is now healthy 2024-04-16T16:58:11.229 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:58:11 smithi045 ceph-mon[31676]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-04-16T16:58:11.428 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:58:11 smithi104 ceph-mon[36457]: from='client.24194 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi045:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T16:58:11.428 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:58:11 smithi104 ceph-mon[36457]: pgmap v97: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-16T16:58:11.428 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:58:11 smithi104 ceph-mon[36457]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-16T16:58:11.428 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:58:11 smithi104 ceph-mon[36457]: Cluster is now healthy 2024-04-16T16:58:11.428 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:58:11 smithi104 ceph-mon[36457]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-04-16T16:58:11.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:58:11 smithi117 ceph-mon[36730]: from='client.24194 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi045:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T16:58:11.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:58:11 smithi117 ceph-mon[36730]: pgmap v97: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-16T16:58:11.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:58:11 smithi117 ceph-mon[36730]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-16T16:58:11.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:58:11 smithi117 ceph-mon[36730]: Cluster is now healthy 2024-04-16T16:58:11.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:58:11 smithi117 ceph-mon[36730]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-04-16T16:58:13.428 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:58:13 smithi104 ceph-mon[36457]: pgmap v98: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-16T16:58:13.478 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:58:13 smithi045 ceph-mon[31676]: pgmap v98: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-16T16:58:13.555 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:58:13 smithi117 ceph-mon[36730]: pgmap v98: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-16T16:58:15.428 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:58:15 smithi104 ceph-mon[36457]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-16T16:58:15.429 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:58:15 smithi104 ceph-mon[36457]: from='client.? 172.21.15.45:0/1645973553' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ec6490cb-a715-4c12-a54a-225aae26fcb9"}]: dispatch 2024-04-16T16:58:15.429 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:58:15 smithi104 ceph-mon[36457]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ec6490cb-a715-4c12-a54a-225aae26fcb9"}]: dispatch 2024-04-16T16:58:15.429 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:58:15 smithi104 ceph-mon[36457]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ec6490cb-a715-4c12-a54a-225aae26fcb9"}]': finished 2024-04-16T16:58:15.429 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:58:15 smithi104 ceph-mon[36457]: osdmap e23: 4 total, 3 up, 4 in 2024-04-16T16:58:15.429 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:58:15 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-16T16:58:15.429 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:58:15 smithi104 ceph-mon[36457]: from='client.? 172.21.15.45:0/3649512467' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-16T16:58:15.478 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:58:15 smithi045 ceph-mon[31676]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-16T16:58:15.479 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:58:15 smithi045 ceph-mon[31676]: from='client.? 172.21.15.45:0/1645973553' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ec6490cb-a715-4c12-a54a-225aae26fcb9"}]: dispatch 2024-04-16T16:58:15.479 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:58:15 smithi045 ceph-mon[31676]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ec6490cb-a715-4c12-a54a-225aae26fcb9"}]: dispatch 2024-04-16T16:58:15.479 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:58:15 smithi045 ceph-mon[31676]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ec6490cb-a715-4c12-a54a-225aae26fcb9"}]': finished 2024-04-16T16:58:15.479 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:58:15 smithi045 ceph-mon[31676]: osdmap e23: 4 total, 3 up, 4 in 2024-04-16T16:58:15.479 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:58:15 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-16T16:58:15.479 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:58:15 smithi045 ceph-mon[31676]: from='client.? 172.21.15.45:0/3649512467' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-16T16:58:15.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:58:15 smithi117 ceph-mon[36730]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-16T16:58:15.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:58:15 smithi117 ceph-mon[36730]: from='client.? 172.21.15.45:0/1645973553' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ec6490cb-a715-4c12-a54a-225aae26fcb9"}]: dispatch 2024-04-16T16:58:15.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:58:15 smithi117 ceph-mon[36730]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ec6490cb-a715-4c12-a54a-225aae26fcb9"}]: dispatch 2024-04-16T16:58:15.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:58:15 smithi117 ceph-mon[36730]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ec6490cb-a715-4c12-a54a-225aae26fcb9"}]': finished 2024-04-16T16:58:15.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:58:15 smithi117 ceph-mon[36730]: osdmap e23: 4 total, 3 up, 4 in 2024-04-16T16:58:15.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:58:15 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-16T16:58:15.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:58:15 smithi117 ceph-mon[36730]: from='client.? 172.21.15.45:0/3649512467' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-16T16:58:17.428 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:58:17 smithi104 ceph-mon[36457]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-16T16:58:17.478 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:58:17 smithi045 ceph-mon[31676]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-16T16:58:17.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:58:17 smithi117 ceph-mon[36730]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-16T16:58:19.428 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:58:19 smithi104 ceph-mon[36457]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-16T16:58:19.478 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:58:19 smithi045 ceph-mon[31676]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-16T16:58:19.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:58:19 smithi117 ceph-mon[36730]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-16T16:58:21.428 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:58:21 smithi104 ceph-mon[36457]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-16T16:58:21.478 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:58:21 smithi045 ceph-mon[31676]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-16T16:58:21.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:58:21 smithi117 ceph-mon[36730]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-16T16:58:23.428 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:58:23 smithi104 ceph-mon[36457]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-16T16:58:23.478 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:58:23 smithi045 ceph-mon[31676]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-16T16:58:23.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:58:23 smithi117 ceph-mon[36730]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-16T16:58:24.428 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:58:24 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-16T16:58:24.428 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:58:24 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:58:24.428 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:58:24 smithi104 ceph-mon[36457]: Deploying daemon osd.3 on smithi045 2024-04-16T16:58:24.478 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:58:24 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-16T16:58:24.478 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:58:24 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:58:24.478 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:58:24 smithi045 ceph-mon[31676]: Deploying daemon osd.3 on smithi045 2024-04-16T16:58:24.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:58:24 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-16T16:58:24.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:58:24 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:58:24.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:58:24 smithi117 ceph-mon[36730]: Deploying daemon osd.3 on smithi045 2024-04-16T16:58:25.478 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:58:25 smithi045 ceph-mon[31676]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-16T16:58:25.555 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:58:25 smithi117 ceph-mon[36730]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-16T16:58:25.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:58:25 smithi104 ceph-mon[36457]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-16T16:58:27.478 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:58:27 smithi045 ceph-mon[31676]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-16T16:58:27.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:58:27 smithi117 ceph-mon[36730]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-16T16:58:27.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:58:27 smithi104 ceph-mon[36457]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-16T16:58:29.429 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:58:29 smithi045 ceph-mon[31676]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-16T16:58:29.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:58:29 smithi117 ceph-mon[36730]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-16T16:58:29.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:58:29 smithi104 ceph-mon[36457]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-16T16:58:31.479 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:58:31 smithi045 ceph-mon[31676]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-16T16:58:31.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:58:31 smithi117 ceph-mon[36730]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-16T16:58:31.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:58:31 smithi104 ceph-mon[36457]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-16T16:58:33.603 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:58:33 smithi045 ceph-mon[31676]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-16T16:58:33.603 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:58:33 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:58:33.603 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:58:33 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:58:33.603 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:58:33 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T16:58:33.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:58:33 smithi104 ceph-mon[36457]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-16T16:58:33.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:58:33 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:58:33.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:58:33 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:58:33.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:58:33 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T16:58:33.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:58:33 smithi117 ceph-mon[36730]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-16T16:58:33.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:58:33 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:58:33.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:58:33 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:58:33.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:58:33 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T16:58:35.574 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:58:35 smithi045 ceph-mon[31676]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-16T16:58:35.608 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:58:35 smithi117 ceph-mon[36730]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-16T16:58:35.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:58:35 smithi104 ceph-mon[36457]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-16T16:58:37.200 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:58:36 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:58:37.200 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:58:36 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:58:37.200 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:58:36 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:58:37.200 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:58:36 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T16:58:37.201 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:58:36 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:58:37.201 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:58:36 smithi045 ceph-mon[31676]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-16T16:58:37.201 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:58:36 smithi045 ceph-mon[31676]: from='osd.3 [v2:172.21.15.45:6810/2823712137,v1:172.21.15.45:6811/2823712137]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-16T16:58:37.305 INFO:teuthology.orchestra.run.smithi045.stdout:Created osd(s) 3 on host 'smithi045' 2024-04-16T16:58:37.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:58:37.303+0000 7f4820c00700 1 -- 172.21.15.45:0/2824558918 <== mgr.14152 v2:172.21.15.45:6800/268030699 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f4824062f30 con 0x7f48080a3c30 2024-04-16T16:58:37.306 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:58:36 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:58:37.306 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:58:36 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:58:37.306 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:58:36 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:58:37.306 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:58:36 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T16:58:37.306 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:58:36 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:58:37.307 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:58:36 smithi117 ceph-mon[36730]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-16T16:58:37.307 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:58:36 smithi117 ceph-mon[36730]: from='osd.3 [v2:172.21.15.45:6810/2823712137,v1:172.21.15.45:6811/2823712137]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-16T16:58:37.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:58:37.307+0000 7f4828c00700 1 -- 172.21.15.45:0/2824558918 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f48080a3c30 msgr2=0x7f48080a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:58:37.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:58:37.307+0000 7f4828c00700 1 --2- 172.21.15.45:0/2824558918 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f48080a3c30 0x7f48080a60f0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f480c009c60 tx=0x7f480c009260 comp rx=0 tx=0).stop 2024-04-16T16:58:37.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:58:37.307+0000 7f4828c00700 1 -- 172.21.15.45:0/2824558918 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f48240754f0 msgr2=0x7f482419ae60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:58:37.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:58:37.307+0000 7f4828c00700 1 --2- 172.21.15.45:0/2824558918 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f48240754f0 0x7f482419ae60 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f481400e9a0 tx=0x7f481400ecb0 comp rx=0 tx=0).stop 2024-04-16T16:58:37.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:58:37.307+0000 7f4828c00700 1 -- 172.21.15.45:0/2824558918 shutdown_connections 2024-04-16T16:58:37.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:58:37.307+0000 7f4828c00700 1 --2- 172.21.15.45:0/2824558918 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f482410a970 0x7f48241a23d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:58:37.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:58:37.307+0000 7f4828c00700 1 --2- 172.21.15.45:0/2824558918 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f48240754f0 0x7f482419ae60 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:58:37.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:58:37.307+0000 7f4828c00700 1 --2- 172.21.15.45:0/2824558918 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f48080a3c30 0x7f48080a60f0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:58:37.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:58:37.307+0000 7f4828c00700 1 --2- 172.21.15.45:0/2824558918 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4824076ac0 0x7f482419b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:58:37.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:58:37.307+0000 7f4828c00700 1 -- 172.21.15.45:0/2824558918 >> 172.21.15.45:0/2824558918 conn(0x7f48240fe030 msgr2=0x7f48241068c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T16:58:37.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:58:37.307+0000 7f4828c00700 1 -- 172.21.15.45:0/2824558918 shutdown_connections 2024-04-16T16:58:37.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:58:37.307+0000 7f4828c00700 1 -- 172.21.15.45:0/2824558918 wait complete. 2024-04-16T16:58:37.428 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:58:36 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:58:37.428 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:58:36 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:58:37.428 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:58:36 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:58:37.428 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:58:36 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T16:58:37.429 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:58:36 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:58:37.429 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:58:36 smithi104 ceph-mon[36457]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-16T16:58:37.429 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:58:36 smithi104 ceph-mon[36457]: from='osd.3 [v2:172.21.15.45:6810/2823712137,v1:172.21.15.45:6811/2823712137]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-16T16:58:38.108 DEBUG:teuthology.orchestra.run.smithi045:osd.3> sudo journalctl -f -n 0 -u ceph-b1fecd38-fc11-11ee-bc90-c7b262605968@osd.3.service 2024-04-16T16:58:38.114 INFO:tasks.cephadm:Deploying osd.4 on smithi104 with /dev/vg_nvme/lv_3... 2024-04-16T16:58:38.114 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b1fecd38-fc11-11ee-bc90-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-16T16:58:38.228 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:58:37 smithi045 ceph-mon[31676]: from='osd.3 [v2:172.21.15.45:6810/2823712137,v1:172.21.15.45:6811/2823712137]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-16T16:58:38.228 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:58:37 smithi045 ceph-mon[31676]: osdmap e24: 4 total, 3 up, 4 in 2024-04-16T16:58:38.228 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:58:37 smithi045 ceph-mon[31676]: from='osd.3 [v2:172.21.15.45:6810/2823712137,v1:172.21.15.45:6811/2823712137]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]: dispatch 2024-04-16T16:58:38.228 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:58:37 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-16T16:58:38.228 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:58:37 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:58:38.228 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:58:37 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T16:58:38.228 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:58:37 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:58:38.306 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:58:37 smithi117 ceph-mon[36730]: from='osd.3 [v2:172.21.15.45:6810/2823712137,v1:172.21.15.45:6811/2823712137]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-16T16:58:38.306 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:58:37 smithi117 ceph-mon[36730]: osdmap e24: 4 total, 3 up, 4 in 2024-04-16T16:58:38.306 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:58:37 smithi117 ceph-mon[36730]: from='osd.3 [v2:172.21.15.45:6810/2823712137,v1:172.21.15.45:6811/2823712137]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]: dispatch 2024-04-16T16:58:38.306 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:58:37 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-16T16:58:38.306 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:58:37 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:58:38.306 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:58:37 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T16:58:38.306 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:58:37 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:58:38.358 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/mon.b/config 2024-04-16T16:58:38.428 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:58:37 smithi104 ceph-mon[36457]: from='osd.3 [v2:172.21.15.45:6810/2823712137,v1:172.21.15.45:6811/2823712137]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-16T16:58:38.428 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:58:37 smithi104 ceph-mon[36457]: osdmap e24: 4 total, 3 up, 4 in 2024-04-16T16:58:38.428 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:58:37 smithi104 ceph-mon[36457]: from='osd.3 [v2:172.21.15.45:6810/2823712137,v1:172.21.15.45:6811/2823712137]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]: dispatch 2024-04-16T16:58:38.428 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:58:37 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-16T16:58:38.428 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:58:37 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:58:38.428 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:58:37 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T16:58:38.428 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:58:37 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:58:38.976 INFO:journalctl@ceph.osd.3.smithi045.stdout:Apr 16 16:58:38 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-3[53367]: 2024-04-16T16:58:38.578+0000 7fc3da800700 -1 osd.3 0 waiting for initial osdmap 2024-04-16T16:58:38.976 INFO:journalctl@ceph.osd.3.smithi045.stdout:Apr 16 16:58:38 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-3[53367]: 2024-04-16T16:58:38.606+0000 7fc3d0600700 -1 osd.3 25 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-16T16:58:39.178 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:58:38 smithi104 ceph-mon[36457]: from='osd.3 [v2:172.21.15.45:6810/2823712137,v1:172.21.15.45:6811/2823712137]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]': finished 2024-04-16T16:58:39.178 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:58:38 smithi104 ceph-mon[36457]: osdmap e25: 4 total, 3 up, 4 in 2024-04-16T16:58:39.178 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:58:38 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-16T16:58:39.178 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:58:38 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-16T16:58:39.178 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:58:38 smithi104 ceph-mon[36457]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-16T16:58:39.179 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:58:38 smithi104 ceph-mon[36457]: from='osd.3 [v2:172.21.15.45:6810/2823712137,v1:172.21.15.45:6811/2823712137]' entity='osd.3' 2024-04-16T16:58:39.228 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:58:38 smithi045 ceph-mon[31676]: from='osd.3 [v2:172.21.15.45:6810/2823712137,v1:172.21.15.45:6811/2823712137]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]': finished 2024-04-16T16:58:39.228 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:58:38 smithi045 ceph-mon[31676]: osdmap e25: 4 total, 3 up, 4 in 2024-04-16T16:58:39.228 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:58:38 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-16T16:58:39.228 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:58:38 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-16T16:58:39.228 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:58:38 smithi045 ceph-mon[31676]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-16T16:58:39.228 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:58:38 smithi045 ceph-mon[31676]: from='osd.3 [v2:172.21.15.45:6810/2823712137,v1:172.21.15.45:6811/2823712137]' entity='osd.3' 2024-04-16T16:58:39.306 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:58:38 smithi117 ceph-mon[36730]: from='osd.3 [v2:172.21.15.45:6810/2823712137,v1:172.21.15.45:6811/2823712137]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]': finished 2024-04-16T16:58:39.306 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:58:38 smithi117 ceph-mon[36730]: osdmap e25: 4 total, 3 up, 4 in 2024-04-16T16:58:39.306 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:58:38 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-16T16:58:39.306 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:58:38 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-16T16:58:39.306 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:58:38 smithi117 ceph-mon[36730]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-16T16:58:39.306 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:58:38 smithi117 ceph-mon[36730]: from='osd.3 [v2:172.21.15.45:6810/2823712137,v1:172.21.15.45:6811/2823712137]' entity='osd.3' 2024-04-16T16:58:40.178 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:58:39 smithi104 ceph-mon[36457]: purged_snaps scrub starts 2024-04-16T16:58:40.178 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:58:39 smithi104 ceph-mon[36457]: purged_snaps scrub ok 2024-04-16T16:58:40.178 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:58:39 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-16T16:58:40.178 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:58:39 smithi104 ceph-mon[36457]: osd.3 [v2:172.21.15.45:6810/2823712137,v1:172.21.15.45:6811/2823712137] boot 2024-04-16T16:58:40.179 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:58:39 smithi104 ceph-mon[36457]: osdmap e26: 4 total, 4 up, 4 in 2024-04-16T16:58:40.179 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:58:39 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-16T16:58:40.179 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:58:39 smithi104 ceph-mon[36457]: osdmap e27: 4 total, 4 up, 4 in 2024-04-16T16:58:40.228 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:58:39 smithi045 ceph-mon[31676]: purged_snaps scrub starts 2024-04-16T16:58:40.229 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:58:39 smithi045 ceph-mon[31676]: purged_snaps scrub ok 2024-04-16T16:58:40.229 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:58:39 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-16T16:58:40.229 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:58:39 smithi045 ceph-mon[31676]: osd.3 [v2:172.21.15.45:6810/2823712137,v1:172.21.15.45:6811/2823712137] boot 2024-04-16T16:58:40.229 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:58:39 smithi045 ceph-mon[31676]: osdmap e26: 4 total, 4 up, 4 in 2024-04-16T16:58:40.229 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:58:39 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-16T16:58:40.229 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:58:39 smithi045 ceph-mon[31676]: osdmap e27: 4 total, 4 up, 4 in 2024-04-16T16:58:40.306 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:58:39 smithi117 ceph-mon[36730]: purged_snaps scrub starts 2024-04-16T16:58:40.306 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:58:39 smithi117 ceph-mon[36730]: purged_snaps scrub ok 2024-04-16T16:58:40.306 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:58:39 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-16T16:58:40.306 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:58:39 smithi117 ceph-mon[36730]: osd.3 [v2:172.21.15.45:6810/2823712137,v1:172.21.15.45:6811/2823712137] boot 2024-04-16T16:58:40.306 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:58:39 smithi117 ceph-mon[36730]: osdmap e26: 4 total, 4 up, 4 in 2024-04-16T16:58:40.306 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:58:39 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-16T16:58:40.306 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:58:39 smithi117 ceph-mon[36730]: osdmap e27: 4 total, 4 up, 4 in 2024-04-16T16:58:40.453 INFO:teuthology.orchestra.run.smithi104.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-16T16:58:40.453 INFO:teuthology.orchestra.run.smithi104.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-16T16:58:40.453 INFO:teuthology.orchestra.run.smithi104.stderr: stderr: 10+0 records in 2024-04-16T16:58:40.453 INFO:teuthology.orchestra.run.smithi104.stderr:10+0 records out 2024-04-16T16:58:40.453 INFO:teuthology.orchestra.run.smithi104.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0114315 s, 917 MB/s 2024-04-16T16:58:40.453 INFO:teuthology.orchestra.run.smithi104.stderr:--> Zapping successful for: 2024-04-16T16:58:41.114 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:58:41 smithi104 ceph-mon[36457]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-04-16T16:58:41.114 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:58:41 smithi104 ceph-mon[36457]: osdmap e28: 4 total, 4 up, 4 in 2024-04-16T16:58:41.140 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b1fecd38-fc11-11ee-bc90-c7b262605968 -- ceph orch daemon add osd smithi104:vg_nvme/lv_3 2024-04-16T16:58:41.306 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:58:41 smithi117 ceph-mon[36730]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-04-16T16:58:41.306 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:58:41 smithi117 ceph-mon[36730]: osdmap e28: 4 total, 4 up, 4 in 2024-04-16T16:58:41.362 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:58:41 smithi045 ceph-mon[31676]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-04-16T16:58:41.363 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:58:41 smithi045 ceph-mon[31676]: osdmap e28: 4 total, 4 up, 4 in 2024-04-16T16:58:41.382 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/mon.b/config 2024-04-16T16:58:42.306 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:58:42 smithi117 ceph-mon[36730]: osdmap e29: 4 total, 4 up, 4 in 2024-04-16T16:58:42.306 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:58:42 smithi117 ceph-mon[36730]: osdmap e30: 4 total, 4 up, 4 in 2024-04-16T16:58:42.428 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:58:42 smithi104 ceph-mon[36457]: osdmap e29: 4 total, 4 up, 4 in 2024-04-16T16:58:42.429 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:58:42 smithi104 ceph-mon[36457]: osdmap e30: 4 total, 4 up, 4 in 2024-04-16T16:58:42.478 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:58:42 smithi045 ceph-mon[31676]: osdmap e29: 4 total, 4 up, 4 in 2024-04-16T16:58:42.478 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:58:42 smithi045 ceph-mon[31676]: osdmap e30: 4 total, 4 up, 4 in 2024-04-16T16:58:43.066 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-16T16:58:43.061+0000 7f312c729700 1 -- 172.21.15.104:0/115528041 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f31241035f0 msgr2=0x7f3124109ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:58:43.067 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-16T16:58:43.061+0000 7f312c729700 1 --2- 172.21.15.104:0/115528041 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f31241035f0 0x7f3124109ea0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f311c00b0d0 tx=0x7f311c00b3e0 comp rx=0 tx=0).stop 2024-04-16T16:58:43.067 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-16T16:58:43.061+0000 7f312c729700 1 -- 172.21.15.104:0/115528041 shutdown_connections 2024-04-16T16:58:43.067 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-16T16:58:43.061+0000 7f312c729700 1 --2- 172.21.15.104:0/115528041 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f31241035f0 0x7f3124109ea0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:58:43.067 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-16T16:58:43.061+0000 7f312c729700 1 --2- 172.21.15.104:0/115528041 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3124101a50 0x7f3124101e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:58:43.067 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-16T16:58:43.061+0000 7f312c729700 1 --2- 172.21.15.104:0/115528041 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3124102c50 0x7f31241030b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:58:43.067 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-16T16:58:43.061+0000 7f312c729700 1 -- 172.21.15.104:0/115528041 >> 172.21.15.104:0/115528041 conn(0x7f31240fcff0 msgr2=0x7f31240ff430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T16:58:43.067 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-16T16:58:43.061+0000 7f312c729700 1 -- 172.21.15.104:0/115528041 shutdown_connections 2024-04-16T16:58:43.067 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-16T16:58:43.062+0000 7f312c729700 1 -- 172.21.15.104:0/115528041 wait complete. 2024-04-16T16:58:43.067 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-16T16:58:43.062+0000 7f312c729700 1 Processor -- start 2024-04-16T16:58:43.067 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-16T16:58:43.062+0000 7f312c729700 1 -- start start 2024-04-16T16:58:43.067 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-16T16:58:43.062+0000 7f312c729700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3124101a50 0x7f312406b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:58:43.067 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-16T16:58:43.062+0000 7f312c729700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3124102c50 0x7f31241a0010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:58:43.068 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-16T16:58:43.062+0000 7f312c729700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f31241035f0 0x7f31241a2560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:58:43.068 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-16T16:58:43.062+0000 7f312c729700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f312410d730 con 0x7f31241035f0 2024-04-16T16:58:43.068 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-16T16:58:43.062+0000 7f312c729700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f312410d5b0 con 0x7f3124101a50 2024-04-16T16:58:43.068 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-16T16:58:43.062+0000 7f312c729700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f312410d8b0 con 0x7f3124102c50 2024-04-16T16:58:43.068 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-16T16:58:43.062+0000 7f312a4c5700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3124101a50 0x7f312406b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:58:43.068 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-16T16:58:43.062+0000 7f312a4c5700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3124101a50 0x7f312406b330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:55622/0 (socket says 172.21.15.104:55622) 2024-04-16T16:58:43.068 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-16T16:58:43.062+0000 7f312a4c5700 1 -- 172.21.15.104:0/1904304084 learned_addr learned my addr 172.21.15.104:0/1904304084 (peer_addr_for_me v2:172.21.15.104:0/0) 2024-04-16T16:58:43.068 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-16T16:58:43.063+0000 7f312a4c5700 1 -- 172.21.15.104:0/1904304084 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3124102c50 msgr2=0x7f31241a0010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:58:43.068 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-16T16:58:43.063+0000 7f312a4c5700 1 --2- 172.21.15.104:0/1904304084 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3124102c50 0x7f31241a0010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:58:43.068 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-16T16:58:43.063+0000 7f312a4c5700 1 -- 172.21.15.104:0/1904304084 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f31241035f0 msgr2=0x7f31241a2560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:58:43.068 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-16T16:58:43.063+0000 7f312a4c5700 1 --2- 172.21.15.104:0/1904304084 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f31241035f0 0x7f31241a2560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:58:43.068 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-16T16:58:43.063+0000 7f312a4c5700 1 -- 172.21.15.104:0/1904304084 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31241a2d40 con 0x7f3124101a50 2024-04-16T16:58:43.068 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-16T16:58:43.063+0000 7f312a4c5700 1 --2- 172.21.15.104:0/1904304084 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3124101a50 0x7f312406b330 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f312000b500 tx=0x7f312000b8c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T16:58:43.068 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-16T16:58:43.063+0000 7f311b7fe700 1 -- 172.21.15.104:0/1904304084 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3120015020 con 0x7f3124101a50 2024-04-16T16:58:43.069 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-16T16:58:43.063+0000 7f311b7fe700 1 -- 172.21.15.104:0/1904304084 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3120011d10 con 0x7f3124101a50 2024-04-16T16:58:43.069 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-16T16:58:43.063+0000 7f312c729700 1 -- 172.21.15.104:0/1904304084 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31241a2fd0 con 0x7f3124101a50 2024-04-16T16:58:43.069 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-16T16:58:43.063+0000 7f311b7fe700 1 -- 172.21.15.104:0/1904304084 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f312000f2e0 con 0x7f3124101a50 2024-04-16T16:58:43.069 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-16T16:58:43.063+0000 7f312c729700 1 -- 172.21.15.104:0/1904304084 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31241a3560 con 0x7f3124101a50 2024-04-16T16:58:43.069 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-16T16:58:43.064+0000 7f312c729700 1 -- 172.21.15.104:0/1904304084 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f31240697e0 con 0x7f3124101a50 2024-04-16T16:58:43.069 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-16T16:58:43.065+0000 7f311b7fe700 1 -- 172.21.15.104:0/1904304084 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f3120018020 con 0x7f3124101a50 2024-04-16T16:58:43.069 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-16T16:58:43.065+0000 7f311b7fe700 1 --2- 172.21.15.104:0/1904304084 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f31100a3c40 0x7f31100a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:58:43.069 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-16T16:58:43.066+0000 7f311b7fe700 1 -- 172.21.15.104:0/1904304084 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 3669+0+0 (secure 0 0 0) 0x7f31200cede0 con 0x7f3124101a50 2024-04-16T16:58:43.069 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-16T16:58:43.066+0000 7f3129cc4700 1 --2- 172.21.15.104:0/1904304084 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f31100a3c40 0x7f31100a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:58:43.070 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-16T16:58:43.067+0000 7f3129cc4700 1 --2- 172.21.15.104:0/1904304084 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f31100a3c40 0x7f31100a6100 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f312406b8b0 tx=0x7f3114006d20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T16:58:43.070 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-16T16:58:43.069+0000 7f311b7fe700 1 -- 172.21.15.104:0/1904304084 <== mon.2 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f312001f020 con 0x7f3124101a50 2024-04-16T16:58:43.178 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:58:43 smithi104 ceph-mon[36457]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-04-16T16:58:43.277 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-16T16:58:43.276+0000 7f312c729700 1 -- 172.21.15.104:0/1904304084 --> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi104:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f3124062f30 con 0x7f31100a3c40 2024-04-16T16:58:43.306 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:58:43 smithi117 ceph-mon[36730]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-04-16T16:58:43.385 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:58:43 smithi045 ceph-mon[31676]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-04-16T16:58:44.169 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:58:44 smithi104 ceph-mon[36457]: from='client.24221 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi104:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T16:58:44.169 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:58:44 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-16T16:58:44.169 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:58:44 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-16T16:58:44.169 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:58:44 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:58:44.228 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:58:44 smithi045 ceph-mon[31676]: from='client.24221 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi104:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T16:58:44.228 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:58:44 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-16T16:58:44.228 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:58:44 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-16T16:58:44.228 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:58:44 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:58:44.306 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:58:44 smithi117 ceph-mon[36730]: from='client.24221 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi104:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T16:58:44.306 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:58:44 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-16T16:58:44.306 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:58:44 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-16T16:58:44.306 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:58:44 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:58:45.642 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:58:45 smithi045 ceph-mon[31676]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 115 KiB/s, 0 objects/s recovering 2024-04-16T16:58:45.642 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:58:45 smithi045 ceph-mon[31676]: Detected new or changed devices on smithi045 2024-04-16T16:58:45.642 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:58:45 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:58:45.642 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:58:45 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:58:45.643 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:58:45 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-16T16:58:45.643 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:58:45 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-16T16:58:45.643 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:58:45 smithi045 ceph-mon[31676]: Adjusting osd_memory_target on smithi045 to 8547M 2024-04-16T16:58:45.643 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:58:45 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:58:45.643 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:58:45 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:58:45.643 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:58:45 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T16:58:45.643 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:58:45 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:58:45.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:58:45 smithi104 ceph-mon[36457]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 115 KiB/s, 0 objects/s recovering 2024-04-16T16:58:45.679 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:58:45 smithi104 ceph-mon[36457]: Detected new or changed devices on smithi045 2024-04-16T16:58:45.679 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:58:45 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:58:45.679 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:58:45 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:58:45.679 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:58:45 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-16T16:58:45.679 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:58:45 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-16T16:58:45.679 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:58:45 smithi104 ceph-mon[36457]: Adjusting osd_memory_target on smithi045 to 8547M 2024-04-16T16:58:45.679 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:58:45 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:58:45.679 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:58:45 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:58:45.679 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:58:45 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T16:58:45.679 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:58:45 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:58:45.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:58:45 smithi117 ceph-mon[36730]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 115 KiB/s, 0 objects/s recovering 2024-04-16T16:58:45.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:58:45 smithi117 ceph-mon[36730]: Detected new or changed devices on smithi045 2024-04-16T16:58:45.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:58:45 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:58:45.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:58:45 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:58:45.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:58:45 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-16T16:58:45.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:58:45 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-16T16:58:45.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:58:45 smithi117 ceph-mon[36730]: Adjusting osd_memory_target on smithi045 to 8547M 2024-04-16T16:58:45.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:58:45 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:58:45.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:58:45 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:58:45.807 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:58:45 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T16:58:45.807 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:58:45 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:58:46.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:58:46 smithi104 ceph-mon[36457]: from='client.? 172.21.15.104:0/227279641' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5d91d568-0250-4aae-8dc2-a0e079f92062"}]: dispatch 2024-04-16T16:58:46.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:58:46 smithi104 ceph-mon[36457]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5d91d568-0250-4aae-8dc2-a0e079f92062"}]: dispatch 2024-04-16T16:58:46.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:58:46 smithi104 ceph-mon[36457]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5d91d568-0250-4aae-8dc2-a0e079f92062"}]': finished 2024-04-16T16:58:46.679 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:58:46 smithi104 ceph-mon[36457]: osdmap e31: 5 total, 4 up, 5 in 2024-04-16T16:58:46.679 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:58:46 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-16T16:58:46.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:58:46 smithi045 ceph-mon[31676]: from='client.? 172.21.15.104:0/227279641' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5d91d568-0250-4aae-8dc2-a0e079f92062"}]: dispatch 2024-04-16T16:58:46.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:58:46 smithi045 ceph-mon[31676]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5d91d568-0250-4aae-8dc2-a0e079f92062"}]: dispatch 2024-04-16T16:58:46.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:58:46 smithi045 ceph-mon[31676]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5d91d568-0250-4aae-8dc2-a0e079f92062"}]': finished 2024-04-16T16:58:46.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:58:46 smithi045 ceph-mon[31676]: osdmap e31: 5 total, 4 up, 5 in 2024-04-16T16:58:46.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:58:46 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-16T16:58:46.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:58:46 smithi117 ceph-mon[36730]: from='client.? 172.21.15.104:0/227279641' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5d91d568-0250-4aae-8dc2-a0e079f92062"}]: dispatch 2024-04-16T16:58:46.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:58:46 smithi117 ceph-mon[36730]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5d91d568-0250-4aae-8dc2-a0e079f92062"}]: dispatch 2024-04-16T16:58:46.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:58:46 smithi117 ceph-mon[36730]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5d91d568-0250-4aae-8dc2-a0e079f92062"}]': finished 2024-04-16T16:58:46.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:58:46 smithi117 ceph-mon[36730]: osdmap e31: 5 total, 4 up, 5 in 2024-04-16T16:58:46.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:58:46 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-16T16:58:47.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:58:47 smithi104 ceph-mon[36457]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-16T16:58:47.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:58:47 smithi104 ceph-mon[36457]: from='client.? 172.21.15.104:0/1531125566' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-16T16:58:47.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:58:47 smithi045 ceph-mon[31676]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-16T16:58:47.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:58:47 smithi045 ceph-mon[31676]: from='client.? 172.21.15.104:0/1531125566' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-16T16:58:47.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:58:47 smithi117 ceph-mon[36730]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-16T16:58:47.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:58:47 smithi117 ceph-mon[36730]: from='client.? 172.21.15.104:0/1531125566' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-16T16:58:49.666 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:58:49 smithi104 ceph-mon[36457]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 82 KiB/s, 0 objects/s recovering 2024-04-16T16:58:49.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:58:49 smithi045 ceph-mon[31676]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 82 KiB/s, 0 objects/s recovering 2024-04-16T16:58:49.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:58:49 smithi117 ceph-mon[36730]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 82 KiB/s, 0 objects/s recovering 2024-04-16T16:58:51.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:58:51 smithi104 ceph-mon[36457]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-16T16:58:51.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:58:51 smithi045 ceph-mon[31676]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-16T16:58:51.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:58:51 smithi117 ceph-mon[36730]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-16T16:58:53.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:58:53 smithi104 ceph-mon[36457]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-16T16:58:53.679 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:58:53 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-16T16:58:53.679 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:58:53 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:58:53.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:58:53 smithi045 ceph-mon[31676]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-16T16:58:53.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:58:53 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-16T16:58:53.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:58:53 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:58:53.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:58:53 smithi117 ceph-mon[36730]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-16T16:58:53.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:58:53 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-16T16:58:53.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:58:53 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:58:54.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:58:54 smithi104 ceph-mon[36457]: Deploying daemon osd.4 on smithi104 2024-04-16T16:58:54.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:58:54 smithi045 ceph-mon[31676]: Deploying daemon osd.4 on smithi104 2024-04-16T16:58:54.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:58:54 smithi117 ceph-mon[36730]: Deploying daemon osd.4 on smithi104 2024-04-16T16:58:55.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:58:55 smithi104 ceph-mon[36457]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-04-16T16:58:55.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:58:55 smithi045 ceph-mon[31676]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-04-16T16:58:55.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:58:55 smithi117 ceph-mon[36730]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-04-16T16:58:57.679 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:58:57 smithi104 ceph-mon[36457]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-04-16T16:58:57.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:58:57 smithi045 ceph-mon[31676]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-04-16T16:58:57.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:58:57 smithi117 ceph-mon[36730]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-04-16T16:59:00.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:58:59 smithi117 ceph-mon[36730]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-04-16T16:59:00.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:58:59 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:59:00.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:58:59 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:59:00.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:58:59 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T16:59:00.101 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:58:59 smithi104 ceph-mon[36457]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-04-16T16:59:00.102 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:58:59 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:59:00.102 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:58:59 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:59:00.102 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:58:59 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T16:59:00.228 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:58:59 smithi045 ceph-mon[31676]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-04-16T16:59:00.228 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:58:59 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:59:00.228 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:58:59 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:59:00.228 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:58:59 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T16:59:02.045 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:59:01 smithi104 ceph-mon[36457]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-04-16T16:59:02.045 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:59:01 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:59:02.045 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:59:01 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:59:02.045 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:59:01 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:59:02.045 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:59:01 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T16:59:02.045 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:59:01 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:59:02.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:59:01 smithi117 ceph-mon[36730]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-04-16T16:59:02.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:59:01 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:59:02.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:59:01 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:59:02.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:59:01 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:59:02.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:59:01 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T16:59:02.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:59:01 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:59:02.228 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:59:01 smithi045 ceph-mon[31676]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-04-16T16:59:02.228 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:59:01 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:59:02.228 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:59:01 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:59:02.228 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:59:01 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:59:02.229 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:59:01 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T16:59:02.229 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:59:01 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:59:02.361 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-16T16:59:02.360+0000 7f311b7fe700 1 -- 172.21.15.104:0/1904304084 <== mgr.14152 v2:172.21.15.45:6800/268030699 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f3124062f30 con 0x7f31100a3c40 2024-04-16T16:59:02.361 INFO:teuthology.orchestra.run.smithi104.stdout:Created osd(s) 4 on host 'smithi104' 2024-04-16T16:59:02.364 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-16T16:59:02.362+0000 7f312c729700 1 -- 172.21.15.104:0/1904304084 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f31100a3c40 msgr2=0x7f31100a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:59:02.364 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-16T16:59:02.362+0000 7f312c729700 1 --2- 172.21.15.104:0/1904304084 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f31100a3c40 0x7f31100a6100 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f312406b8b0 tx=0x7f3114006d20 comp rx=0 tx=0).stop 2024-04-16T16:59:02.364 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-16T16:59:02.362+0000 7f312c729700 1 -- 172.21.15.104:0/1904304084 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3124101a50 msgr2=0x7f312406b330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:59:02.364 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-16T16:59:02.362+0000 7f312c729700 1 --2- 172.21.15.104:0/1904304084 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3124101a50 0x7f312406b330 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f312000b500 tx=0x7f312000b8c0 comp rx=0 tx=0).stop 2024-04-16T16:59:02.364 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-16T16:59:02.362+0000 7f312c729700 1 -- 172.21.15.104:0/1904304084 shutdown_connections 2024-04-16T16:59:02.364 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-16T16:59:02.362+0000 7f312c729700 1 --2- 172.21.15.104:0/1904304084 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f31241035f0 0x7f31241a2560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:59:02.364 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-16T16:59:02.362+0000 7f312c729700 1 --2- 172.21.15.104:0/1904304084 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3124101a50 0x7f312406b330 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:59:02.364 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-16T16:59:02.362+0000 7f312c729700 1 --2- 172.21.15.104:0/1904304084 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f31100a3c40 0x7f31100a6100 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:59:02.364 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-16T16:59:02.362+0000 7f312c729700 1 --2- 172.21.15.104:0/1904304084 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3124102c50 0x7f31241a0010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:59:02.364 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-16T16:59:02.362+0000 7f312c729700 1 -- 172.21.15.104:0/1904304084 >> 172.21.15.104:0/1904304084 conn(0x7f31240fcff0 msgr2=0x7f3124106010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T16:59:02.365 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-16T16:59:02.362+0000 7f312c729700 1 -- 172.21.15.104:0/1904304084 shutdown_connections 2024-04-16T16:59:02.365 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-16T16:59:02.363+0000 7f312c729700 1 -- 172.21.15.104:0/1904304084 wait complete. 2024-04-16T16:59:03.034 DEBUG:teuthology.orchestra.run.smithi104:osd.4> sudo journalctl -f -n 0 -u ceph-b1fecd38-fc11-11ee-bc90-c7b262605968@osd.4.service 2024-04-16T16:59:03.036 INFO:tasks.cephadm:Deploying osd.5 on smithi117 with /dev/vg_nvme/lv_3... 2024-04-16T16:59:03.036 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b1fecd38-fc11-11ee-bc90-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-16T16:59:03.281 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/mon.c/config 2024-04-16T16:59:03.578 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:59:03 smithi104 ceph-mon[36457]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-04-16T16:59:03.578 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:59:03 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:59:03.578 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:59:03 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T16:59:03.578 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:59:03 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:59:03.578 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:59:03 smithi104 ceph-mon[36457]: from='osd.4 [v2:172.21.15.104:6808/1396278127,v1:172.21.15.104:6809/1396278127]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-16T16:59:03.578 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:59:03 smithi104 ceph-mon[36457]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-16T16:59:03.597 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:59:03 smithi117 ceph-mon[36730]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-04-16T16:59:03.597 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:59:03 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:59:03.597 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:59:03 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T16:59:03.597 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:59:03 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:59:03.597 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:59:03 smithi117 ceph-mon[36730]: from='osd.4 [v2:172.21.15.104:6808/1396278127,v1:172.21.15.104:6809/1396278127]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-16T16:59:03.597 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:59:03 smithi117 ceph-mon[36730]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-16T16:59:03.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:59:03 smithi045 ceph-mon[31676]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-04-16T16:59:03.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:59:03 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:59:03.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:59:03 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T16:59:03.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:59:03 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:59:03.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:59:03 smithi045 ceph-mon[31676]: from='osd.4 [v2:172.21.15.104:6808/1396278127,v1:172.21.15.104:6809/1396278127]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-16T16:59:03.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:59:03 smithi045 ceph-mon[31676]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-16T16:59:04.428 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:59:04 smithi104 ceph-mon[36457]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-16T16:59:04.429 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:59:04 smithi104 ceph-mon[36457]: from='osd.4 [v2:172.21.15.104:6808/1396278127,v1:172.21.15.104:6809/1396278127]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2024-04-16T16:59:04.429 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:59:04 smithi104 ceph-mon[36457]: osdmap e32: 5 total, 4 up, 5 in 2024-04-16T16:59:04.429 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:59:04 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-16T16:59:04.429 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:59:04 smithi104 ceph-mon[36457]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2024-04-16T16:59:04.429 INFO:journalctl@ceph.osd.4.smithi104.stdout:Apr 16 16:59:04 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-4[48045]: 2024-04-16T16:59:04.347+0000 7fa47c9ea700 -1 osd.4 0 waiting for initial osdmap 2024-04-16T16:59:04.429 INFO:journalctl@ceph.osd.4.smithi104.stdout:Apr 16 16:59:04 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-4[48045]: 2024-04-16T16:59:04.354+0000 7fa4787dd700 -1 osd.4 33 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-16T16:59:04.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:59:04 smithi045 ceph-mon[31676]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-16T16:59:04.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:59:04 smithi045 ceph-mon[31676]: from='osd.4 [v2:172.21.15.104:6808/1396278127,v1:172.21.15.104:6809/1396278127]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2024-04-16T16:59:04.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:59:04 smithi045 ceph-mon[31676]: osdmap e32: 5 total, 4 up, 5 in 2024-04-16T16:59:04.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:59:04 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-16T16:59:04.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:59:04 smithi045 ceph-mon[31676]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2024-04-16T16:59:04.764 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:59:04 smithi117 ceph-mon[36730]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-16T16:59:04.765 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:59:04 smithi117 ceph-mon[36730]: from='osd.4 [v2:172.21.15.104:6808/1396278127,v1:172.21.15.104:6809/1396278127]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2024-04-16T16:59:04.765 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:59:04 smithi117 ceph-mon[36730]: osdmap e32: 5 total, 4 up, 5 in 2024-04-16T16:59:04.765 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:59:04 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-16T16:59:04.765 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:59:04 smithi117 ceph-mon[36730]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2024-04-16T16:59:05.440 INFO:teuthology.orchestra.run.smithi117.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-16T16:59:05.440 INFO:teuthology.orchestra.run.smithi117.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-16T16:59:05.440 INFO:teuthology.orchestra.run.smithi117.stderr: stderr: 10+0 records in 2024-04-16T16:59:05.440 INFO:teuthology.orchestra.run.smithi117.stderr:10+0 records out 2024-04-16T16:59:05.440 INFO:teuthology.orchestra.run.smithi117.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0111767 s, 938 MB/s 2024-04-16T16:59:05.440 INFO:teuthology.orchestra.run.smithi117.stderr:--> Zapping successful for: 2024-04-16T16:59:05.644 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:59:05 smithi045 ceph-mon[31676]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-04-16T16:59:05.644 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:59:05 smithi045 ceph-mon[31676]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]': finished 2024-04-16T16:59:05.644 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:59:05 smithi045 ceph-mon[31676]: osdmap e33: 5 total, 4 up, 5 in 2024-04-16T16:59:05.644 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:59:05 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-16T16:59:05.645 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:59:05 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-16T16:59:05.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:59:05 smithi104 ceph-mon[36457]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-04-16T16:59:05.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:59:05 smithi104 ceph-mon[36457]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]': finished 2024-04-16T16:59:05.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:59:05 smithi104 ceph-mon[36457]: osdmap e33: 5 total, 4 up, 5 in 2024-04-16T16:59:05.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:59:05 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-16T16:59:05.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:59:05 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-16T16:59:05.737 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:59:05 smithi117 ceph-mon[36730]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-04-16T16:59:05.738 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:59:05 smithi117 ceph-mon[36730]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]': finished 2024-04-16T16:59:05.738 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:59:05 smithi117 ceph-mon[36730]: osdmap e33: 5 total, 4 up, 5 in 2024-04-16T16:59:05.738 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:59:05 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-16T16:59:05.738 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:59:05 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-16T16:59:06.084 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b1fecd38-fc11-11ee-bc90-c7b262605968 -- ceph orch daemon add osd smithi117:vg_nvme/lv_3 2024-04-16T16:59:06.328 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/mon.c/config 2024-04-16T16:59:06.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:59:06 smithi104 ceph-mon[36457]: purged_snaps scrub starts 2024-04-16T16:59:06.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:59:06 smithi104 ceph-mon[36457]: purged_snaps scrub ok 2024-04-16T16:59:06.679 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:59:06 smithi104 ceph-mon[36457]: osd.4 [v2:172.21.15.104:6808/1396278127,v1:172.21.15.104:6809/1396278127] boot 2024-04-16T16:59:06.679 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:59:06 smithi104 ceph-mon[36457]: osdmap e34: 5 total, 5 up, 5 in 2024-04-16T16:59:06.679 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:59:06 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-16T16:59:06.679 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:59:06 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:59:06.679 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:59:06 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:59:06.679 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:59:06 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-16T16:59:06.679 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:59:06 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-16T16:59:06.679 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:59:06 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:59:06.679 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:59:06 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:59:06.679 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:59:06 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T16:59:06.679 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:59:06 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:59:06.684 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:59:06 smithi117 ceph-mon[36730]: purged_snaps scrub starts 2024-04-16T16:59:06.684 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:59:06 smithi117 ceph-mon[36730]: purged_snaps scrub ok 2024-04-16T16:59:06.684 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:59:06 smithi117 ceph-mon[36730]: osd.4 [v2:172.21.15.104:6808/1396278127,v1:172.21.15.104:6809/1396278127] boot 2024-04-16T16:59:06.684 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:59:06 smithi117 ceph-mon[36730]: osdmap e34: 5 total, 5 up, 5 in 2024-04-16T16:59:06.684 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:59:06 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-16T16:59:06.684 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:59:06 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:59:06.684 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:59:06 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:59:06.684 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:59:06 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-16T16:59:06.684 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:59:06 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-16T16:59:06.685 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:59:06 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:59:06.685 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:59:06 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:59:06.685 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:59:06 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T16:59:06.685 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:59:06 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:59:06.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:59:06 smithi045 ceph-mon[31676]: purged_snaps scrub starts 2024-04-16T16:59:06.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:59:06 smithi045 ceph-mon[31676]: purged_snaps scrub ok 2024-04-16T16:59:06.729 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:59:06 smithi045 ceph-mon[31676]: osd.4 [v2:172.21.15.104:6808/1396278127,v1:172.21.15.104:6809/1396278127] boot 2024-04-16T16:59:06.729 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:59:06 smithi045 ceph-mon[31676]: osdmap e34: 5 total, 5 up, 5 in 2024-04-16T16:59:06.729 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:59:06 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-16T16:59:06.729 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:59:06 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:59:06.729 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:59:06 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:59:06.729 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:59:06 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-16T16:59:06.729 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:59:06 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-16T16:59:06.729 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:59:06 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:59:06.729 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:59:06 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:59:06.729 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:59:06 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T16:59:06.729 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:59:06 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:59:07.463 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:59:07 smithi117 ceph-mon[36730]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-04-16T16:59:07.463 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:59:07 smithi117 ceph-mon[36730]: Detected new or changed devices on smithi104 2024-04-16T16:59:07.463 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:59:07 smithi117 ceph-mon[36730]: Adjusting osd_memory_target on smithi104 to 8540M 2024-04-16T16:59:07.463 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:59:07 smithi117 ceph-mon[36730]: osdmap e35: 5 total, 5 up, 5 in 2024-04-16T16:59:07.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:59:07 smithi104 ceph-mon[36457]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-04-16T16:59:07.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:59:07 smithi104 ceph-mon[36457]: Detected new or changed devices on smithi104 2024-04-16T16:59:07.679 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:59:07 smithi104 ceph-mon[36457]: Adjusting osd_memory_target on smithi104 to 8540M 2024-04-16T16:59:07.679 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:59:07 smithi104 ceph-mon[36457]: osdmap e35: 5 total, 5 up, 5 in 2024-04-16T16:59:07.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:59:07 smithi045 ceph-mon[31676]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-04-16T16:59:07.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:59:07 smithi045 ceph-mon[31676]: Detected new or changed devices on smithi104 2024-04-16T16:59:07.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:59:07 smithi045 ceph-mon[31676]: Adjusting osd_memory_target on smithi104 to 8540M 2024-04-16T16:59:07.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:59:07 smithi045 ceph-mon[31676]: osdmap e35: 5 total, 5 up, 5 in 2024-04-16T16:59:07.986 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:59:07.982+0000 7fe9476f5700 1 -- 172.21.15.117:0/1370279903 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe9401047e0 msgr2=0x7fe94010b090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:59:07.986 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:59:07.982+0000 7fe9476f5700 1 --2- 172.21.15.117:0/1370279903 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe9401047e0 0x7fe94010b090 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fe93800b0a0 tx=0x7fe93800b3b0 comp rx=0 tx=0).stop 2024-04-16T16:59:07.986 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:59:07.982+0000 7fe9476f5700 1 -- 172.21.15.117:0/1370279903 shutdown_connections 2024-04-16T16:59:07.986 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:59:07.982+0000 7fe9476f5700 1 --2- 172.21.15.117:0/1370279903 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe9401047e0 0x7fe94010b090 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:59:07.986 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:59:07.982+0000 7fe9476f5700 1 --2- 172.21.15.117:0/1370279903 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe940100e60 0x7fe940101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:59:07.986 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:59:07.982+0000 7fe9476f5700 1 --2- 172.21.15.117:0/1370279903 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe940103e40 0x7fe9401042a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:59:07.986 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:59:07.982+0000 7fe9476f5700 1 -- 172.21.15.117:0/1370279903 >> 172.21.15.117:0/1370279903 conn(0x7fe9400fc7c0 msgr2=0x7fe9400fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T16:59:07.987 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:59:07.982+0000 7fe9476f5700 1 -- 172.21.15.117:0/1370279903 shutdown_connections 2024-04-16T16:59:07.987 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:59:07.982+0000 7fe9476f5700 1 -- 172.21.15.117:0/1370279903 wait complete. 2024-04-16T16:59:07.987 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:59:07.983+0000 7fe9476f5700 1 Processor -- start 2024-04-16T16:59:07.987 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:59:07.983+0000 7fe9476f5700 1 -- start start 2024-04-16T16:59:07.987 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:59:07.983+0000 7fe9476f5700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe940100e60 0x7fe940198dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:59:07.987 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:59:07.983+0000 7fe9476f5700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe940103e40 0x7fe940199310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:59:07.987 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:59:07.983+0000 7fe9476f5700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe9401047e0 0x7fe9401a0390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:59:07.987 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:59:07.983+0000 7fe9476f5700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fe94010e870 con 0x7fe940100e60 2024-04-16T16:59:07.987 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:59:07.983+0000 7fe9476f5700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fe94010e6f0 con 0x7fe9401047e0 2024-04-16T16:59:07.987 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:59:07.983+0000 7fe9476f5700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fe94010e9f0 con 0x7fe940103e40 2024-04-16T16:59:07.987 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:59:07.983+0000 7fe944c90700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe940103e40 0x7fe940199310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:59:07.987 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:59:07.983+0000 7fe944c90700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe940103e40 0x7fe940199310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:53014/0 (socket says 172.21.15.117:53014) 2024-04-16T16:59:07.987 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:59:07.983+0000 7fe944c90700 1 -- 172.21.15.117:0/102307543 learned_addr learned my addr 172.21.15.117:0/102307543 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-16T16:59:07.987 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:59:07.983+0000 7fe945c92700 1 --2- 172.21.15.117:0/102307543 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe9401047e0 0x7fe9401a0390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:59:07.988 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:59:07.984+0000 7fe944c90700 1 -- 172.21.15.117:0/102307543 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe9401047e0 msgr2=0x7fe9401a0390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:59:07.988 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:59:07.984+0000 7fe944c90700 1 --2- 172.21.15.117:0/102307543 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe9401047e0 0x7fe9401a0390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:59:07.988 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:59:07.984+0000 7fe944c90700 1 -- 172.21.15.117:0/102307543 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe940100e60 msgr2=0x7fe940198dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:59:07.988 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:59:07.984+0000 7fe944c90700 1 --2- 172.21.15.117:0/102307543 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe940100e60 0x7fe940198dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:59:07.988 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:59:07.984+0000 7fe944c90700 1 -- 172.21.15.117:0/102307543 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9401a0a30 con 0x7fe940103e40 2024-04-16T16:59:07.988 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:59:07.984+0000 7fe944c90700 1 --2- 172.21.15.117:0/102307543 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe940103e40 0x7fe940199310 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fe93c00b560 tx=0x7fe93c00b920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T16:59:07.988 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:59:07.984+0000 7fe9367fc700 1 -- 172.21.15.117:0/102307543 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe93c015020 con 0x7fe940103e40 2024-04-16T16:59:07.988 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:59:07.984+0000 7fe9476f5700 1 -- 172.21.15.117:0/102307543 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9401a0d20 con 0x7fe940103e40 2024-04-16T16:59:07.988 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:59:07.984+0000 7fe9367fc700 1 -- 172.21.15.117:0/102307543 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe93c011d10 con 0x7fe940103e40 2024-04-16T16:59:07.988 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:59:07.984+0000 7fe9367fc700 1 -- 172.21.15.117:0/102307543 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe93c00f2e0 con 0x7fe940103e40 2024-04-16T16:59:07.988 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:59:07.984+0000 7fe9476f5700 1 -- 172.21.15.117:0/102307543 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9401a12d0 con 0x7fe940103e40 2024-04-16T16:59:07.988 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:59:07.985+0000 7fe9476f5700 1 -- 172.21.15.117:0/102307543 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9400697e0 con 0x7fe940103e40 2024-04-16T16:59:07.989 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:59:07.986+0000 7fe9367fc700 1 -- 172.21.15.117:0/102307543 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fe93c018020 con 0x7fe940103e40 2024-04-16T16:59:07.989 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:59:07.987+0000 7fe9367fc700 1 --2- 172.21.15.117:0/102307543 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fe92c0a3c40 0x7fe92c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:59:07.989 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:59:07.987+0000 7fe9367fc700 1 -- 172.21.15.117:0/102307543 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 4101+0+0 (secure 0 0 0) 0x7fe93c0cf0d0 con 0x7fe940103e40 2024-04-16T16:59:07.989 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:59:07.987+0000 7fe945491700 1 --2- 172.21.15.117:0/102307543 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fe92c0a3c40 0x7fe92c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:59:07.989 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:59:07.988+0000 7fe945491700 1 --2- 172.21.15.117:0/102307543 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fe92c0a3c40 0x7fe92c0a6100 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fe94006b040 tx=0x7fe930006d20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T16:59:07.992 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:59:07.991+0000 7fe9367fc700 1 -- 172.21.15.117:0/102307543 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe93c01f020 con 0x7fe940103e40 2024-04-16T16:59:08.200 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:59:08.199+0000 7fe9476f5700 1 -- 172.21.15.117:0/102307543 --> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi117:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fe940062f30 con 0x7fe92c0a3c40 2024-04-16T16:59:08.555 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:59:08 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-16T16:59:08.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:59:08 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-16T16:59:08.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:59:08 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:59:08.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:59:08 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-16T16:59:08.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:59:08 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-16T16:59:08.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:59:08 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:59:08.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:59:08 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-16T16:59:08.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:59:08 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-16T16:59:08.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:59:08 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:59:09.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:59:09 smithi104 ceph-mon[36457]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-16T16:59:09.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:59:09 smithi104 ceph-mon[36457]: from='client.24223 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi117:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T16:59:09.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:59:09 smithi045 ceph-mon[31676]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-16T16:59:09.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:59:09 smithi045 ceph-mon[31676]: from='client.24223 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi117:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T16:59:09.747 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:59:09 smithi117 ceph-mon[36730]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-16T16:59:09.747 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:59:09 smithi117 ceph-mon[36730]: from='client.24223 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi117:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T16:59:11.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:59:11 smithi104 ceph-mon[36457]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-16T16:59:11.679 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:59:11 smithi104 ceph-mon[36457]: from='client.? 172.21.15.117:0/1126682552' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ac510b43-6fb2-4d0f-bc5f-16ac080d9e9d"}]: dispatch 2024-04-16T16:59:11.679 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:59:11 smithi104 ceph-mon[36457]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ac510b43-6fb2-4d0f-bc5f-16ac080d9e9d"}]: dispatch 2024-04-16T16:59:11.679 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:59:11 smithi104 ceph-mon[36457]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ac510b43-6fb2-4d0f-bc5f-16ac080d9e9d"}]': finished 2024-04-16T16:59:11.679 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:59:11 smithi104 ceph-mon[36457]: osdmap e36: 6 total, 5 up, 6 in 2024-04-16T16:59:11.679 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:59:11 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-16T16:59:11.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:59:11 smithi045 ceph-mon[31676]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-16T16:59:11.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:59:11 smithi045 ceph-mon[31676]: from='client.? 172.21.15.117:0/1126682552' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ac510b43-6fb2-4d0f-bc5f-16ac080d9e9d"}]: dispatch 2024-04-16T16:59:11.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:59:11 smithi045 ceph-mon[31676]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ac510b43-6fb2-4d0f-bc5f-16ac080d9e9d"}]: dispatch 2024-04-16T16:59:11.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:59:11 smithi045 ceph-mon[31676]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ac510b43-6fb2-4d0f-bc5f-16ac080d9e9d"}]': finished 2024-04-16T16:59:11.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:59:11 smithi045 ceph-mon[31676]: osdmap e36: 6 total, 5 up, 6 in 2024-04-16T16:59:11.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:59:11 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-16T16:59:11.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:59:11 smithi117 ceph-mon[36730]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-16T16:59:11.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:59:11 smithi117 ceph-mon[36730]: from='client.? 172.21.15.117:0/1126682552' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ac510b43-6fb2-4d0f-bc5f-16ac080d9e9d"}]: dispatch 2024-04-16T16:59:11.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:59:11 smithi117 ceph-mon[36730]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ac510b43-6fb2-4d0f-bc5f-16ac080d9e9d"}]: dispatch 2024-04-16T16:59:11.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:59:11 smithi117 ceph-mon[36730]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ac510b43-6fb2-4d0f-bc5f-16ac080d9e9d"}]': finished 2024-04-16T16:59:11.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:59:11 smithi117 ceph-mon[36730]: osdmap e36: 6 total, 5 up, 6 in 2024-04-16T16:59:11.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:59:11 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-16T16:59:12.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:59:12 smithi104 ceph-mon[36457]: from='client.? 172.21.15.117:0/3466247823' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-16T16:59:12.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:59:12 smithi045 ceph-mon[31676]: from='client.? 172.21.15.117:0/3466247823' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-16T16:59:12.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:59:12 smithi117 ceph-mon[36730]: from='client.? 172.21.15.117:0/3466247823' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-16T16:59:13.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:59:13 smithi104 ceph-mon[36457]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-16T16:59:13.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:59:13 smithi045 ceph-mon[31676]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-16T16:59:13.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:59:13 smithi117 ceph-mon[36730]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-16T16:59:15.679 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:59:15 smithi104 ceph-mon[36457]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-16T16:59:15.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:59:15 smithi045 ceph-mon[31676]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-16T16:59:15.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:59:15 smithi117 ceph-mon[36730]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-16T16:59:17.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:59:17 smithi104 ceph-mon[36457]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-16T16:59:17.711 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:59:17 smithi117 ceph-mon[36730]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-16T16:59:17.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:59:17 smithi045 ceph-mon[31676]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-16T16:59:18.696 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:59:18 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-16T16:59:18.696 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:59:18 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:59:18.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:59:18 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-16T16:59:18.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:59:18 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:59:18.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:59:18 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-16T16:59:18.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:59:18 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:59:19.706 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:59:19 smithi117 ceph-mon[36730]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-16T16:59:19.706 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:59:19 smithi117 ceph-mon[36730]: Deploying daemon osd.5 on smithi117 2024-04-16T16:59:19.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:59:19 smithi045 ceph-mon[31676]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-16T16:59:19.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:59:19 smithi045 ceph-mon[31676]: Deploying daemon osd.5 on smithi117 2024-04-16T16:59:19.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:59:19 smithi104 ceph-mon[36457]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-16T16:59:19.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:59:19 smithi104 ceph-mon[36457]: Deploying daemon osd.5 on smithi117 2024-04-16T16:59:21.719 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:59:21 smithi117 ceph-mon[36730]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-16T16:59:21.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:59:21 smithi045 ceph-mon[31676]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-16T16:59:21.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:59:21 smithi104 ceph-mon[36457]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-16T16:59:23.500 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:59:23 smithi117 ceph-mon[36730]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-16T16:59:23.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:59:23 smithi045 ceph-mon[31676]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-16T16:59:23.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:59:23 smithi104 ceph-mon[36457]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-16T16:59:24.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:59:24 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:59:24.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:59:24 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:59:24.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:59:24 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T16:59:24.978 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:59:24 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:59:24.978 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:59:24 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:59:24.978 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:59:24 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T16:59:25.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:59:24 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:59:25.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:59:24 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:59:25.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:59:24 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T16:59:25.919 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:59:25 smithi117 ceph-mon[36730]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-16T16:59:25.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:59:25 smithi104 ceph-mon[36457]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-16T16:59:25.978 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:59:25 smithi045 ceph-mon[31676]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-16T16:59:27.284 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:59:27.282+0000 7fe9367fc700 1 -- 172.21.15.117:0/102307543 <== mgr.14152 v2:172.21.15.45:6800/268030699 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fe940062f30 con 0x7fe92c0a3c40 2024-04-16T16:59:27.284 INFO:teuthology.orchestra.run.smithi117.stdout:Created osd(s) 5 on host 'smithi117' 2024-04-16T16:59:27.286 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:59:27.284+0000 7fe9476f5700 1 -- 172.21.15.117:0/102307543 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fe92c0a3c40 msgr2=0x7fe92c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:59:27.286 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:59:27.284+0000 7fe9476f5700 1 --2- 172.21.15.117:0/102307543 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fe92c0a3c40 0x7fe92c0a6100 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fe94006b040 tx=0x7fe930006d20 comp rx=0 tx=0).stop 2024-04-16T16:59:27.286 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:59:27.284+0000 7fe9476f5700 1 -- 172.21.15.117:0/102307543 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe940103e40 msgr2=0x7fe940199310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:59:27.286 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:59:27.284+0000 7fe9476f5700 1 --2- 172.21.15.117:0/102307543 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe940103e40 0x7fe940199310 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fe93c00b560 tx=0x7fe93c00b920 comp rx=0 tx=0).stop 2024-04-16T16:59:27.286 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:59:27.284+0000 7fe9476f5700 1 -- 172.21.15.117:0/102307543 shutdown_connections 2024-04-16T16:59:27.286 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:59:27.284+0000 7fe9476f5700 1 --2- 172.21.15.117:0/102307543 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe9401047e0 0x7fe9401a0390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:59:27.286 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:59:27.284+0000 7fe9476f5700 1 --2- 172.21.15.117:0/102307543 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe940100e60 0x7fe940198dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:59:27.286 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:59:27.284+0000 7fe9476f5700 1 --2- 172.21.15.117:0/102307543 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fe92c0a3c40 0x7fe92c0a6100 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:59:27.286 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:59:27.284+0000 7fe9476f5700 1 --2- 172.21.15.117:0/102307543 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe940103e40 0x7fe940199310 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:59:27.286 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:59:27.284+0000 7fe9476f5700 1 -- 172.21.15.117:0/102307543 >> 172.21.15.117:0/102307543 conn(0x7fe9400fc7c0 msgr2=0x7fe9401070e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T16:59:27.286 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:59:27.284+0000 7fe9476f5700 1 -- 172.21.15.117:0/102307543 shutdown_connections 2024-04-16T16:59:27.286 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T16:59:27.284+0000 7fe9476f5700 1 -- 172.21.15.117:0/102307543 wait complete. 2024-04-16T16:59:27.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:59:27 smithi117 ceph-mon[36730]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-16T16:59:27.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:59:27 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:59:27.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:59:27 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:59:27.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:59:27 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:59:27.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:59:27 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:59:27.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:59:27 smithi117 ceph-mon[36730]: from='osd.5 [v2:172.21.15.117:6808/4145720047,v1:172.21.15.117:6809/4145720047]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-16T16:59:27.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:59:27 smithi117 ceph-mon[36730]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-16T16:59:27.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:59:27 smithi104 ceph-mon[36457]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-16T16:59:27.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:59:27 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:59:27.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:59:27 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:59:27.929 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:59:27 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:59:27.929 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:59:27 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:59:27.929 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:59:27 smithi104 ceph-mon[36457]: from='osd.5 [v2:172.21.15.117:6808/4145720047,v1:172.21.15.117:6809/4145720047]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-16T16:59:27.929 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:59:27 smithi104 ceph-mon[36457]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-16T16:59:27.935 DEBUG:teuthology.orchestra.run.smithi117:osd.5> sudo journalctl -f -n 0 -u ceph-b1fecd38-fc11-11ee-bc90-c7b262605968@osd.5.service 2024-04-16T16:59:27.937 INFO:tasks.cephadm:Deploying osd.6 on smithi045 with /dev/vg_nvme/lv_2... 2024-04-16T16:59:27.937 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b1fecd38-fc11-11ee-bc90-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-16T16:59:27.978 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:59:27 smithi045 ceph-mon[31676]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-16T16:59:27.978 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:59:27 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:59:27.978 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:59:27 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:59:27.978 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:59:27 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:59:27.978 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:59:27 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:59:27.978 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:59:27 smithi045 ceph-mon[31676]: from='osd.5 [v2:172.21.15.117:6808/4145720047,v1:172.21.15.117:6809/4145720047]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-16T16:59:27.978 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:59:27 smithi045 ceph-mon[31676]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-16T16:59:28.427 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/mon.a/config 2024-04-16T16:59:29.679 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:59:29 smithi104 ceph-mon[36457]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-16T16:59:29.679 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:59:29 smithi104 ceph-mon[36457]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-16T16:59:29.679 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:59:29 smithi104 ceph-mon[36457]: from='osd.5 [v2:172.21.15.117:6808/4145720047,v1:172.21.15.117:6809/4145720047]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]: dispatch 2024-04-16T16:59:29.679 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:59:29 smithi104 ceph-mon[36457]: osdmap e37: 6 total, 5 up, 6 in 2024-04-16T16:59:29.679 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:59:29 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-16T16:59:29.679 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:59:29 smithi104 ceph-mon[36457]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]: dispatch 2024-04-16T16:59:29.722 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:59:29 smithi045 ceph-mon[31676]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-16T16:59:29.722 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:59:29 smithi045 ceph-mon[31676]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-16T16:59:29.722 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:59:29 smithi045 ceph-mon[31676]: from='osd.5 [v2:172.21.15.117:6808/4145720047,v1:172.21.15.117:6809/4145720047]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]: dispatch 2024-04-16T16:59:29.722 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:59:29 smithi045 ceph-mon[31676]: osdmap e37: 6 total, 5 up, 6 in 2024-04-16T16:59:29.722 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:59:29 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-16T16:59:29.722 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:59:29 smithi045 ceph-mon[31676]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]: dispatch 2024-04-16T16:59:29.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:59:29 smithi117 ceph-mon[36730]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-16T16:59:29.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:59:29 smithi117 ceph-mon[36730]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-16T16:59:29.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:59:29 smithi117 ceph-mon[36730]: from='osd.5 [v2:172.21.15.117:6808/4145720047,v1:172.21.15.117:6809/4145720047]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]: dispatch 2024-04-16T16:59:29.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:59:29 smithi117 ceph-mon[36730]: osdmap e37: 6 total, 5 up, 6 in 2024-04-16T16:59:29.807 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:59:29 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-16T16:59:29.807 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:59:29 smithi117 ceph-mon[36730]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]: dispatch 2024-04-16T16:59:29.807 INFO:journalctl@ceph.osd.5.smithi117.stdout:Apr 16 16:59:29 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-5[48830]: 2024-04-16T16:59:29.399+0000 7fe9df616700 -1 osd.5 0 waiting for initial osdmap 2024-04-16T16:59:29.807 INFO:journalctl@ceph.osd.5.smithi117.stdout:Apr 16 16:59:29 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-5[48830]: 2024-04-16T16:59:29.408+0000 7fe9db409700 -1 osd.5 38 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-16T16:59:30.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:59:30 smithi117 ceph-mon[36730]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]': finished 2024-04-16T16:59:30.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:59:30 smithi117 ceph-mon[36730]: osdmap e38: 6 total, 5 up, 6 in 2024-04-16T16:59:30.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:59:30 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-16T16:59:30.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:59:30 smithi117 ceph-mon[36730]: osd.5 [v2:172.21.15.117:6808/4145720047,v1:172.21.15.117:6809/4145720047] boot 2024-04-16T16:59:30.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:59:30 smithi117 ceph-mon[36730]: osdmap e39: 6 total, 6 up, 6 in 2024-04-16T16:59:30.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:59:30 smithi104 ceph-mon[36457]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]': finished 2024-04-16T16:59:30.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:59:30 smithi104 ceph-mon[36457]: osdmap e38: 6 total, 5 up, 6 in 2024-04-16T16:59:30.679 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:59:30 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-16T16:59:30.679 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:59:30 smithi104 ceph-mon[36457]: osd.5 [v2:172.21.15.117:6808/4145720047,v1:172.21.15.117:6809/4145720047] boot 2024-04-16T16:59:30.679 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:59:30 smithi104 ceph-mon[36457]: osdmap e39: 6 total, 6 up, 6 in 2024-04-16T16:59:30.714 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:59:30 smithi045 ceph-mon[31676]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]': finished 2024-04-16T16:59:30.714 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:59:30 smithi045 ceph-mon[31676]: osdmap e38: 6 total, 5 up, 6 in 2024-04-16T16:59:30.715 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:59:30 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-16T16:59:30.715 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:59:30 smithi045 ceph-mon[31676]: osd.5 [v2:172.21.15.117:6808/4145720047,v1:172.21.15.117:6809/4145720047] boot 2024-04-16T16:59:30.715 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:59:30 smithi045 ceph-mon[31676]: osdmap e39: 6 total, 6 up, 6 in 2024-04-16T16:59:31.470 INFO:teuthology.orchestra.run.smithi045.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-16T16:59:31.470 INFO:teuthology.orchestra.run.smithi045.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-16T16:59:31.470 INFO:teuthology.orchestra.run.smithi045.stderr: stderr: 10+0 records in 2024-04-16T16:59:31.471 INFO:teuthology.orchestra.run.smithi045.stderr:10+0 records out 2024-04-16T16:59:31.471 INFO:teuthology.orchestra.run.smithi045.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0584932 s, 179 MB/s 2024-04-16T16:59:31.471 INFO:teuthology.orchestra.run.smithi045.stderr:--> Zapping successful for: 2024-04-16T16:59:31.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:59:31 smithi104 ceph-mon[36457]: purged_snaps scrub starts 2024-04-16T16:59:31.679 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:59:31 smithi104 ceph-mon[36457]: purged_snaps scrub ok 2024-04-16T16:59:31.679 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:59:31 smithi104 ceph-mon[36457]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-16T16:59:31.679 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:59:31 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-16T16:59:31.679 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:59:31 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:59:31.679 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:59:31 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:59:31.679 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:59:31 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-16T16:59:31.679 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:59:31 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-16T16:59:31.679 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:59:31 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:59:31.679 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:59:31 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:59:31.679 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:59:31 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T16:59:31.679 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:59:31 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:59:31.679 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:59:31 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T16:59:31.679 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:59:31 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:59:31.679 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:59:31 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T16:59:31.679 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:59:31 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:59:31.680 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:59:31 smithi104 ceph-mon[36457]: osdmap e40: 6 total, 6 up, 6 in 2024-04-16T16:59:31.729 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:59:31 smithi045 ceph-mon[31676]: purged_snaps scrub starts 2024-04-16T16:59:31.729 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:59:31 smithi045 ceph-mon[31676]: purged_snaps scrub ok 2024-04-16T16:59:31.729 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:59:31 smithi045 ceph-mon[31676]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-16T16:59:31.729 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:59:31 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-16T16:59:31.729 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:59:31 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:59:31.729 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:59:31 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:59:31.729 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:59:31 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-16T16:59:31.729 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:59:31 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-16T16:59:31.729 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:59:31 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:59:31.729 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:59:31 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:59:31.729 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:59:31 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T16:59:31.729 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:59:31 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:59:31.729 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:59:31 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T16:59:31.729 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:59:31 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:59:31.730 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:59:31 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T16:59:31.730 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:59:31 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:59:31.730 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:59:31 smithi045 ceph-mon[31676]: osdmap e40: 6 total, 6 up, 6 in 2024-04-16T16:59:31.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:59:31 smithi117 ceph-mon[36730]: purged_snaps scrub starts 2024-04-16T16:59:31.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:59:31 smithi117 ceph-mon[36730]: purged_snaps scrub ok 2024-04-16T16:59:31.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:59:31 smithi117 ceph-mon[36730]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-16T16:59:31.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:59:31 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-16T16:59:31.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:59:31 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:59:31.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:59:31 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:59:31.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:59:31 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-16T16:59:31.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:59:31 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-16T16:59:31.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:59:31 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:59:31.807 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:59:31 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:59:31.807 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:59:31 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T16:59:31.807 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:59:31 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:59:31.807 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:59:31 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T16:59:31.807 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:59:31 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:59:31.807 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:59:31 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T16:59:31.807 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:59:31 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:59:31.807 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:59:31 smithi117 ceph-mon[36730]: osdmap e40: 6 total, 6 up, 6 in 2024-04-16T16:59:32.603 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b1fecd38-fc11-11ee-bc90-c7b262605968 -- ceph orch daemon add osd smithi045:vg_nvme/lv_2 2024-04-16T16:59:32.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:59:32 smithi045 ceph-mon[31676]: Detected new or changed devices on smithi117 2024-04-16T16:59:32.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:59:32 smithi045 ceph-mon[31676]: Adjusting osd_memory_target on smithi117 to 8540M 2024-04-16T16:59:32.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:59:32 smithi045 ceph-mon[31676]: osdmap e41: 6 total, 6 up, 6 in 2024-04-16T16:59:32.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:59:32 smithi117 ceph-mon[36730]: Detected new or changed devices on smithi117 2024-04-16T16:59:32.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:59:32 smithi117 ceph-mon[36730]: Adjusting osd_memory_target on smithi117 to 8540M 2024-04-16T16:59:32.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:59:32 smithi117 ceph-mon[36730]: osdmap e41: 6 total, 6 up, 6 in 2024-04-16T16:59:32.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:59:32 smithi104 ceph-mon[36457]: Detected new or changed devices on smithi117 2024-04-16T16:59:32.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:59:32 smithi104 ceph-mon[36457]: Adjusting osd_memory_target on smithi117 to 8540M 2024-04-16T16:59:32.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:59:32 smithi104 ceph-mon[36457]: osdmap e41: 6 total, 6 up, 6 in 2024-04-16T16:59:33.109 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/mon.a/config 2024-04-16T16:59:33.518 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:59:33 smithi045 ceph-mon[31676]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-16T16:59:33.518 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:59:33 smithi045 ceph-mon[31676]: osdmap e42: 6 total, 6 up, 6 in 2024-04-16T16:59:33.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:59:33 smithi117 ceph-mon[36730]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-16T16:59:33.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:59:33 smithi117 ceph-mon[36730]: osdmap e42: 6 total, 6 up, 6 in 2024-04-16T16:59:33.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:59:33 smithi104 ceph-mon[36457]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-16T16:59:33.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:59:33 smithi104 ceph-mon[36457]: osdmap e42: 6 total, 6 up, 6 in 2024-04-16T16:59:35.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:59:35.530+0000 7fd9eb800700 1 -- 172.21.15.45:0/4253616742 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd9e410f140 msgr2=0x7fd9e4111530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:59:35.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:59:35.530+0000 7fd9eb800700 1 --2- 172.21.15.45:0/4253616742 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd9e410f140 0x7fd9e4111530 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fd9dc00b0a0 tx=0x7fd9dc00b3b0 comp rx=0 tx=0).stop 2024-04-16T16:59:35.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:59:35.530+0000 7fd9eb800700 1 -- 172.21.15.45:0/4253616742 shutdown_connections 2024-04-16T16:59:35.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:59:35.530+0000 7fd9eb800700 1 --2- 172.21.15.45:0/4253616742 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd9e410f140 0x7fd9e4111530 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:59:35.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:59:35.530+0000 7fd9eb800700 1 --2- 172.21.15.45:0/4253616742 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd9e4076b20 0x7fd9e4074f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:59:35.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:59:35.530+0000 7fd9eb800700 1 --2- 172.21.15.45:0/4253616742 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd9e4075550 0x7fd9e40759d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:59:35.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:59:35.530+0000 7fd9eb800700 1 -- 172.21.15.45:0/4253616742 >> 172.21.15.45:0/4253616742 conn(0x7fd9e40fe0c0 msgr2=0x7fd9e4100540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T16:59:35.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:59:35.530+0000 7fd9eb800700 1 -- 172.21.15.45:0/4253616742 shutdown_connections 2024-04-16T16:59:35.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:59:35.531+0000 7fd9eb800700 1 -- 172.21.15.45:0/4253616742 wait complete. 2024-04-16T16:59:35.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:59:35.531+0000 7fd9eb800700 1 Processor -- start 2024-04-16T16:59:35.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:59:35.532+0000 7fd9eb800700 1 -- start start 2024-04-16T16:59:35.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:59:35.532+0000 7fd9eb800700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd9e4075550 0x7fd9e419f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:59:35.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:59:35.532+0000 7fd9eb800700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd9e4076b20 0x7fd9e419f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:59:35.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:59:35.532+0000 7fd9eb800700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd9e410f140 0x7fd9e41a68b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:59:35.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:59:35.532+0000 7fd9eb800700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fd9e4114d40 con 0x7fd9e4076b20 2024-04-16T16:59:35.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:59:35.532+0000 7fd9eb800700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fd9e4114bc0 con 0x7fd9e410f140 2024-04-16T16:59:35.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:59:35.532+0000 7fd9eb800700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fd9e4114ec0 con 0x7fd9e4075550 2024-04-16T16:59:35.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:59:35.532+0000 7fd9e8e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd9e4075550 0x7fd9e419f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:59:35.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:59:35.532+0000 7fd9dbe00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd9e4076b20 0x7fd9e419f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:59:35.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:59:35.532+0000 7fd9e8e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd9e4075550 0x7fd9e419f2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:58684/0 (socket says 172.21.15.45:58684) 2024-04-16T16:59:35.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:59:35.532+0000 7fd9e8e00700 1 -- 172.21.15.45:0/1167931047 learned_addr learned my addr 172.21.15.45:0/1167931047 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T16:59:35.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:59:35.533+0000 7fd9e9800700 1 --2- 172.21.15.45:0/1167931047 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd9e410f140 0x7fd9e41a68b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:59:35.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:59:35.533+0000 7fd9e8e00700 1 -- 172.21.15.45:0/1167931047 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd9e410f140 msgr2=0x7fd9e41a68b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:59:35.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:59:35.533+0000 7fd9e8e00700 1 --2- 172.21.15.45:0/1167931047 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd9e410f140 0x7fd9e41a68b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:59:35.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:59:35.533+0000 7fd9e8e00700 1 -- 172.21.15.45:0/1167931047 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd9e4076b20 msgr2=0x7fd9e419f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T16:59:35.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:59:35.533+0000 7fd9e8e00700 1 --2- 172.21.15.45:0/1167931047 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd9e4076b20 0x7fd9e419f830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T16:59:35.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:59:35.533+0000 7fd9e8e00700 1 -- 172.21.15.45:0/1167931047 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9e41a6f50 con 0x7fd9e4075550 2024-04-16T16:59:35.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:59:35.533+0000 7fd9dbe00700 1 --2- 172.21.15.45:0/1167931047 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd9e4076b20 0x7fd9e419f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T16:59:35.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:59:35.533+0000 7fd9e8e00700 1 --2- 172.21.15.45:0/1167931047 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd9e4075550 0x7fd9e419f2f0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fd9d400e9a0 tx=0x7fd9d400ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T16:59:35.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:59:35.533+0000 7fd9e2000700 1 -- 172.21.15.45:0/1167931047 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9d4004a30 con 0x7fd9e4075550 2024-04-16T16:59:35.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:59:35.534+0000 7fd9e2000700 1 -- 172.21.15.45:0/1167931047 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd9d400ccf0 con 0x7fd9e4075550 2024-04-16T16:59:35.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:59:35.534+0000 7fd9eb800700 1 -- 172.21.15.45:0/1167931047 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9e41a7240 con 0x7fd9e4075550 2024-04-16T16:59:35.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:59:35.534+0000 7fd9e2000700 1 -- 172.21.15.45:0/1167931047 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9d4005680 con 0x7fd9e4075550 2024-04-16T16:59:35.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:59:35.534+0000 7fd9eb800700 1 -- 172.21.15.45:0/1167931047 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9e41a77d0 con 0x7fd9e4075550 2024-04-16T16:59:35.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:59:35.535+0000 7fd9eb800700 1 -- 172.21.15.45:0/1167931047 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd9e40697e0 con 0x7fd9e4075550 2024-04-16T16:59:35.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:59:35.537+0000 7fd9e2000700 1 -- 172.21.15.45:0/1167931047 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fd9d400c810 con 0x7fd9e4075550 2024-04-16T16:59:35.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:59:35.538+0000 7fd9e2000700 1 --2- 172.21.15.45:0/1167931047 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fd9c80a3cc0 0x7fd9c80a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T16:59:35.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:59:35.538+0000 7fd9e2000700 1 -- 172.21.15.45:0/1167931047 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 4533+0+0 (secure 0 0 0) 0x7fd9d40cf6d0 con 0x7fd9e4075550 2024-04-16T16:59:35.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:59:35.538+0000 7fd9dbe00700 1 --2- 172.21.15.45:0/1167931047 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fd9c80a3cc0 0x7fd9c80a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T16:59:35.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:59:35.539+0000 7fd9dbe00700 1 --2- 172.21.15.45:0/1167931047 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fd9c80a3cc0 0x7fd9c80a6180 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fd9e4076850 tx=0x7fd9cc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T16:59:35.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:59:35.543+0000 7fd9e2000700 1 -- 172.21.15.45:0/1167931047 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd9d409ced0 con 0x7fd9e4075550 2024-04-16T16:59:35.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:59:35 smithi104 ceph-mon[36457]: pgmap v159: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-16T16:59:35.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:59:35 smithi104 ceph-mon[36457]: osdmap e43: 6 total, 6 up, 6 in 2024-04-16T16:59:35.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:59:35 smithi045 ceph-mon[31676]: pgmap v159: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-16T16:59:35.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:59:35 smithi045 ceph-mon[31676]: osdmap e43: 6 total, 6 up, 6 in 2024-04-16T16:59:35.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T16:59:35.786+0000 7fd9eb800700 1 -- 172.21.15.45:0/1167931047 --> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi045:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7fd9e4062f30 con 0x7fd9c80a3cc0 2024-04-16T16:59:35.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:59:35 smithi117 ceph-mon[36730]: pgmap v159: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-16T16:59:35.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:59:35 smithi117 ceph-mon[36730]: osdmap e43: 6 total, 6 up, 6 in 2024-04-16T16:59:36.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:59:36 smithi104 ceph-mon[36457]: from='client.24250 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi045:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T16:59:36.679 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:59:36 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-16T16:59:36.679 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:59:36 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-16T16:59:36.679 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:59:36 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:59:36.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:59:36 smithi045 ceph-mon[31676]: from='client.24250 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi045:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T16:59:36.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:59:36 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-16T16:59:36.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:59:36 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-16T16:59:36.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:59:36 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:59:36.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:59:36 smithi117 ceph-mon[36730]: from='client.24250 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi045:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T16:59:36.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:59:36 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-16T16:59:36.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:59:36 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-16T16:59:36.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:59:36 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:59:37.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:59:37 smithi104 ceph-mon[36457]: pgmap v161: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 125 KiB/s, 0 objects/s recovering 2024-04-16T16:59:37.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:59:37 smithi045 ceph-mon[31676]: pgmap v161: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 125 KiB/s, 0 objects/s recovering 2024-04-16T16:59:37.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:59:37 smithi117 ceph-mon[36730]: pgmap v161: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 125 KiB/s, 0 objects/s recovering 2024-04-16T16:59:39.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:59:39 smithi104 ceph-mon[36457]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-16T16:59:39.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:59:39 smithi045 ceph-mon[31676]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-16T16:59:39.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:59:39 smithi117 ceph-mon[36730]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-16T16:59:40.679 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:59:40 smithi104 ceph-mon[36457]: from='client.? 172.21.15.45:0/4049581477' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a41adc18-d88a-419e-b2c2-b089c1f32472"}]: dispatch 2024-04-16T16:59:40.679 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:59:40 smithi104 ceph-mon[36457]: from='client.? 172.21.15.45:0/4049581477' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a41adc18-d88a-419e-b2c2-b089c1f32472"}]': finished 2024-04-16T16:59:40.679 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:59:40 smithi104 ceph-mon[36457]: osdmap e44: 7 total, 6 up, 7 in 2024-04-16T16:59:40.679 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:59:40 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-16T16:59:40.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:59:40 smithi045 ceph-mon[31676]: from='client.? 172.21.15.45:0/4049581477' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a41adc18-d88a-419e-b2c2-b089c1f32472"}]: dispatch 2024-04-16T16:59:40.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:59:40 smithi045 ceph-mon[31676]: from='client.? 172.21.15.45:0/4049581477' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a41adc18-d88a-419e-b2c2-b089c1f32472"}]': finished 2024-04-16T16:59:40.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:59:40 smithi045 ceph-mon[31676]: osdmap e44: 7 total, 6 up, 7 in 2024-04-16T16:59:40.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:59:40 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-16T16:59:40.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:59:40 smithi117 ceph-mon[36730]: from='client.? 172.21.15.45:0/4049581477' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a41adc18-d88a-419e-b2c2-b089c1f32472"}]: dispatch 2024-04-16T16:59:40.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:59:40 smithi117 ceph-mon[36730]: from='client.? 172.21.15.45:0/4049581477' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a41adc18-d88a-419e-b2c2-b089c1f32472"}]': finished 2024-04-16T16:59:40.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:59:40 smithi117 ceph-mon[36730]: osdmap e44: 7 total, 6 up, 7 in 2024-04-16T16:59:40.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:59:40 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-16T16:59:41.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:59:41 smithi045 ceph-mon[31676]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-16T16:59:41.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:59:41 smithi045 ceph-mon[31676]: from='client.? 172.21.15.45:0/3712966753' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-16T16:59:41.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:59:41 smithi117 ceph-mon[36730]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-16T16:59:41.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:59:41 smithi117 ceph-mon[36730]: from='client.? 172.21.15.45:0/3712966753' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-16T16:59:41.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:59:41 smithi104 ceph-mon[36457]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-16T16:59:41.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:59:41 smithi104 ceph-mon[36457]: from='client.? 172.21.15.45:0/3712966753' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-16T16:59:43.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:59:43 smithi045 ceph-mon[31676]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-16T16:59:43.730 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:59:43 smithi104 ceph-mon[36457]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-16T16:59:43.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:59:43 smithi117 ceph-mon[36730]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-16T16:59:45.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:59:45 smithi045 ceph-mon[31676]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-16T16:59:45.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:59:45 smithi117 ceph-mon[36730]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-16T16:59:45.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:59:45 smithi104 ceph-mon[36457]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-16T16:59:47.729 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:59:47 smithi045 ceph-mon[31676]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-16T16:59:47.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:59:47 smithi117 ceph-mon[36730]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-16T16:59:47.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:59:47 smithi104 ceph-mon[36457]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-16T16:59:49.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:59:49 smithi045 ceph-mon[31676]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-16T16:59:49.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:59:49 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-16T16:59:49.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:59:49 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:59:49.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:59:49 smithi117 ceph-mon[36730]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-16T16:59:49.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:59:49 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-16T16:59:49.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:59:49 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:59:49.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:59:49 smithi104 ceph-mon[36457]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-16T16:59:49.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:59:49 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-16T16:59:49.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:59:49 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T16:59:50.639 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:59:50 smithi045 ceph-mon[31676]: Deploying daemon osd.6 on smithi045 2024-04-16T16:59:50.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:59:50 smithi117 ceph-mon[36730]: Deploying daemon osd.6 on smithi045 2024-04-16T16:59:50.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:59:50 smithi104 ceph-mon[36457]: Deploying daemon osd.6 on smithi045 2024-04-16T16:59:51.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:59:51 smithi045 ceph-mon[31676]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-16T16:59:51.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:59:51 smithi117 ceph-mon[36730]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-16T16:59:51.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:59:51 smithi104 ceph-mon[36457]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-16T16:59:53.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:59:53 smithi045 ceph-mon[31676]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-16T16:59:53.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:59:53 smithi117 ceph-mon[36730]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-16T16:59:53.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:59:53 smithi104 ceph-mon[36457]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-16T16:59:55.729 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:59:55 smithi045 ceph-mon[31676]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-16T16:59:55.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:59:55 smithi117 ceph-mon[36730]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-16T16:59:55.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:59:55 smithi104 ceph-mon[36457]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-16T16:59:57.729 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:59:57 smithi045 ceph-mon[31676]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-16T16:59:57.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:59:57 smithi117 ceph-mon[36730]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-16T16:59:57.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:59:57 smithi104 ceph-mon[36457]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-16T16:59:59.204 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:59:58 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:59:59.204 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:59:58 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:59:59.204 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:59:58 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T16:59:59.204 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 16:59:58 smithi045 ceph-mon[31676]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-16T16:59:59.306 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:59:58 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:59:59.306 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:59:58 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:59:59.306 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:59:58 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T16:59:59.306 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 16:59:58 smithi117 ceph-mon[36730]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-16T16:59:59.428 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:59:58 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:59:59.428 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:59:58 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T16:59:59.428 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:59:58 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T16:59:59.428 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 16:59:58 smithi104 ceph-mon[36457]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-16T17:00:00.232 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:00:00 smithi045 ceph-mon[31676]: overall HEALTH_OK 2024-04-16T17:00:00.306 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:00:00 smithi117 ceph-mon[36730]: overall HEALTH_OK 2024-04-16T17:00:00.428 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:00:00 smithi104 ceph-mon[36457]: overall HEALTH_OK 2024-04-16T17:00:01.229 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:00:01 smithi045 ceph-mon[31676]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-16T17:00:01.428 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:00:01 smithi104 ceph-mon[36457]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-16T17:00:01.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:00:01 smithi117 ceph-mon[36730]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-16T17:00:02.978 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:00:02 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:00:02.978 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:00:02 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:00:02.978 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:00:02 smithi045 ceph-mon[31676]: from='osd.6 [v2:172.21.15.45:6818/3254229176,v1:172.21.15.45:6819/3254229176]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-16T17:00:02.979 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:00:02 smithi045 ceph-mon[31676]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-16T17:00:02.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:00:02.993+0000 7fd9e2000700 1 -- 172.21.15.45:0/1167931047 <== mgr.14152 v2:172.21.15.45:6800/268030699 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fd9e4062f30 con 0x7fd9c80a3cc0 2024-04-16T17:00:02.996 INFO:teuthology.orchestra.run.smithi045.stdout:Created osd(s) 6 on host 'smithi045' 2024-04-16T17:00:02.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:00:02.996+0000 7fd9eb800700 1 -- 172.21.15.45:0/1167931047 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fd9c80a3cc0 msgr2=0x7fd9c80a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:00:02.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:00:02.996+0000 7fd9eb800700 1 --2- 172.21.15.45:0/1167931047 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fd9c80a3cc0 0x7fd9c80a6180 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fd9e4076850 tx=0x7fd9cc009260 comp rx=0 tx=0).stop 2024-04-16T17:00:02.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:00:02.996+0000 7fd9eb800700 1 -- 172.21.15.45:0/1167931047 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd9e4075550 msgr2=0x7fd9e419f2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:00:02.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:00:02.996+0000 7fd9eb800700 1 --2- 172.21.15.45:0/1167931047 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd9e4075550 0x7fd9e419f2f0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fd9d400e9a0 tx=0x7fd9d400ecb0 comp rx=0 tx=0).stop 2024-04-16T17:00:02.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:00:02.996+0000 7fd9eb800700 1 -- 172.21.15.45:0/1167931047 shutdown_connections 2024-04-16T17:00:02.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:00:02.996+0000 7fd9eb800700 1 --2- 172.21.15.45:0/1167931047 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fd9c80a3cc0 0x7fd9c80a6180 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:00:02.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:00:02.996+0000 7fd9eb800700 1 --2- 172.21.15.45:0/1167931047 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd9e4075550 0x7fd9e419f2f0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:00:02.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:00:02.996+0000 7fd9eb800700 1 --2- 172.21.15.45:0/1167931047 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd9e410f140 0x7fd9e41a68b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:00:02.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:00:02.996+0000 7fd9eb800700 1 --2- 172.21.15.45:0/1167931047 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd9e4076b20 0x7fd9e419f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:00:02.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:00:02.996+0000 7fd9eb800700 1 -- 172.21.15.45:0/1167931047 >> 172.21.15.45:0/1167931047 conn(0x7fd9e40fe0c0 msgr2=0x7fd9e41024e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:00:02.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:00:02.997+0000 7fd9eb800700 1 -- 172.21.15.45:0/1167931047 shutdown_connections 2024-04-16T17:00:02.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:00:02.997+0000 7fd9eb800700 1 -- 172.21.15.45:0/1167931047 wait complete. 2024-04-16T17:00:03.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:00:02 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:00:03.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:00:02 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:00:03.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:00:02 smithi117 ceph-mon[36730]: from='osd.6 [v2:172.21.15.45:6818/3254229176,v1:172.21.15.45:6819/3254229176]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-16T17:00:03.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:00:02 smithi117 ceph-mon[36730]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-16T17:00:03.178 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:00:02 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:00:03.178 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:00:02 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:00:03.178 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:00:02 smithi104 ceph-mon[36457]: from='osd.6 [v2:172.21.15.45:6818/3254229176,v1:172.21.15.45:6819/3254229176]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-16T17:00:03.178 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:00:02 smithi104 ceph-mon[36457]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-16T17:00:03.831 DEBUG:teuthology.orchestra.run.smithi045:osd.6> sudo journalctl -f -n 0 -u ceph-b1fecd38-fc11-11ee-bc90-c7b262605968@osd.6.service 2024-04-16T17:00:03.836 INFO:tasks.cephadm:Deploying osd.7 on smithi104 with /dev/vg_nvme/lv_2... 2024-04-16T17:00:03.836 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b1fecd38-fc11-11ee-bc90-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-16T17:00:03.978 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:00:03 smithi045 ceph-mon[31676]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-16T17:00:03.978 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:00:03 smithi045 ceph-mon[31676]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-16T17:00:03.978 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:00:03 smithi045 ceph-mon[31676]: from='osd.6 [v2:172.21.15.45:6818/3254229176,v1:172.21.15.45:6819/3254229176]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]: dispatch 2024-04-16T17:00:03.978 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:00:03 smithi045 ceph-mon[31676]: osdmap e45: 7 total, 6 up, 7 in 2024-04-16T17:00:03.978 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:00:03 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-16T17:00:03.978 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:00:03 smithi045 ceph-mon[31676]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]: dispatch 2024-04-16T17:00:03.979 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:00:03 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:00:03.979 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:00:03 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:00:04.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:00:03 smithi117 ceph-mon[36730]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-16T17:00:04.057 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:00:03 smithi117 ceph-mon[36730]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-16T17:00:04.057 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:00:03 smithi117 ceph-mon[36730]: from='osd.6 [v2:172.21.15.45:6818/3254229176,v1:172.21.15.45:6819/3254229176]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]: dispatch 2024-04-16T17:00:04.057 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:00:03 smithi117 ceph-mon[36730]: osdmap e45: 7 total, 6 up, 7 in 2024-04-16T17:00:04.057 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:00:03 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-16T17:00:04.057 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:00:03 smithi117 ceph-mon[36730]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]: dispatch 2024-04-16T17:00:04.057 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:00:03 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:00:04.057 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:00:03 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:00:04.095 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/mon.b/config 2024-04-16T17:00:04.178 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:00:03 smithi104 ceph-mon[36457]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-16T17:00:04.178 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:00:03 smithi104 ceph-mon[36457]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-16T17:00:04.178 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:00:03 smithi104 ceph-mon[36457]: from='osd.6 [v2:172.21.15.45:6818/3254229176,v1:172.21.15.45:6819/3254229176]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]: dispatch 2024-04-16T17:00:04.178 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:00:03 smithi104 ceph-mon[36457]: osdmap e45: 7 total, 6 up, 7 in 2024-04-16T17:00:04.178 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:00:03 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-16T17:00:04.179 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:00:03 smithi104 ceph-mon[36457]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]: dispatch 2024-04-16T17:00:04.179 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:00:03 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:00:04.179 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:00:03 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:00:04.729 INFO:journalctl@ceph.osd.6.smithi045.stdout:Apr 16 17:00:04 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-6[59926]: 2024-04-16T17:00:04.424+0000 7fae91600700 -1 osd.6 0 waiting for initial osdmap 2024-04-16T17:00:04.729 INFO:journalctl@ceph.osd.6.smithi045.stdout:Apr 16 17:00:04 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-6[59926]: 2024-04-16T17:00:04.456+0000 7fae86a00700 -1 osd.6 46 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-16T17:00:05.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:00:04 smithi117 ceph-mon[36730]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]': finished 2024-04-16T17:00:05.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:00:04 smithi117 ceph-mon[36730]: osdmap e46: 7 total, 6 up, 7 in 2024-04-16T17:00:05.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:00:04 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-16T17:00:05.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:00:04 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-16T17:00:05.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:00:04 smithi117 ceph-mon[36730]: from='osd.6 ' entity='osd.6' 2024-04-16T17:00:05.178 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:00:04 smithi104 ceph-mon[36457]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]': finished 2024-04-16T17:00:05.178 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:00:04 smithi104 ceph-mon[36457]: osdmap e46: 7 total, 6 up, 7 in 2024-04-16T17:00:05.179 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:00:04 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-16T17:00:05.179 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:00:04 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-16T17:00:05.179 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:00:04 smithi104 ceph-mon[36457]: from='osd.6 ' entity='osd.6' 2024-04-16T17:00:05.228 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:00:04 smithi045 ceph-mon[31676]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]': finished 2024-04-16T17:00:05.228 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:00:04 smithi045 ceph-mon[31676]: osdmap e46: 7 total, 6 up, 7 in 2024-04-16T17:00:05.228 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:00:04 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-16T17:00:05.228 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:00:04 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-16T17:00:05.228 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:00:04 smithi045 ceph-mon[31676]: from='osd.6 ' entity='osd.6' 2024-04-16T17:00:05.929 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:00:05 smithi104 ceph-mon[36457]: purged_snaps scrub starts 2024-04-16T17:00:05.929 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:00:05 smithi104 ceph-mon[36457]: purged_snaps scrub ok 2024-04-16T17:00:05.929 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:00:05 smithi104 ceph-mon[36457]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-16T17:00:05.929 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:00:05 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-16T17:00:05.929 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:00:05 smithi104 ceph-mon[36457]: osd.6 [v2:172.21.15.45:6818/3254229176,v1:172.21.15.45:6819/3254229176] boot 2024-04-16T17:00:05.929 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:00:05 smithi104 ceph-mon[36457]: osdmap e47: 7 total, 7 up, 7 in 2024-04-16T17:00:05.929 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:00:05 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-16T17:00:06.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:00:05 smithi117 ceph-mon[36730]: purged_snaps scrub starts 2024-04-16T17:00:06.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:00:05 smithi117 ceph-mon[36730]: purged_snaps scrub ok 2024-04-16T17:00:06.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:00:05 smithi117 ceph-mon[36730]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-16T17:00:06.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:00:05 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-16T17:00:06.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:00:05 smithi117 ceph-mon[36730]: osd.6 [v2:172.21.15.45:6818/3254229176,v1:172.21.15.45:6819/3254229176] boot 2024-04-16T17:00:06.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:00:05 smithi117 ceph-mon[36730]: osdmap e47: 7 total, 7 up, 7 in 2024-04-16T17:00:06.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:00:05 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-16T17:00:06.091 INFO:teuthology.orchestra.run.smithi104.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-16T17:00:06.091 INFO:teuthology.orchestra.run.smithi104.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-16T17:00:06.091 INFO:teuthology.orchestra.run.smithi104.stderr: stderr: 10+0 records in 2024-04-16T17:00:06.091 INFO:teuthology.orchestra.run.smithi104.stderr:10+0 records out 2024-04-16T17:00:06.091 INFO:teuthology.orchestra.run.smithi104.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.011517 s, 910 MB/s 2024-04-16T17:00:06.092 INFO:teuthology.orchestra.run.smithi104.stderr:--> Zapping successful for: 2024-04-16T17:00:06.167 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:00:05 smithi045 ceph-mon[31676]: purged_snaps scrub starts 2024-04-16T17:00:06.168 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:00:05 smithi045 ceph-mon[31676]: purged_snaps scrub ok 2024-04-16T17:00:06.168 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:00:05 smithi045 ceph-mon[31676]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-16T17:00:06.168 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:00:05 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-16T17:00:06.168 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:00:05 smithi045 ceph-mon[31676]: osd.6 [v2:172.21.15.45:6818/3254229176,v1:172.21.15.45:6819/3254229176] boot 2024-04-16T17:00:06.168 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:00:05 smithi045 ceph-mon[31676]: osdmap e47: 7 total, 7 up, 7 in 2024-04-16T17:00:06.168 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:00:05 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-16T17:00:06.749 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b1fecd38-fc11-11ee-bc90-c7b262605968 -- ceph orch daemon add osd smithi104:vg_nvme/lv_2 2024-04-16T17:00:06.987 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/mon.b/config 2024-04-16T17:00:08.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:00:07 smithi117 ceph-mon[36730]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-04-16T17:00:08.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:00:07 smithi117 ceph-mon[36730]: osdmap e48: 7 total, 7 up, 7 in 2024-04-16T17:00:08.094 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:00:07 smithi104 ceph-mon[36457]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-04-16T17:00:08.094 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:00:07 smithi104 ceph-mon[36457]: osdmap e48: 7 total, 7 up, 7 in 2024-04-16T17:00:08.228 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:00:07 smithi045 ceph-mon[31676]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-04-16T17:00:08.228 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:00:07 smithi045 ceph-mon[31676]: osdmap e48: 7 total, 7 up, 7 in 2024-04-16T17:00:08.624 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-16T17:00:08.622+0000 7f8011b7b700 1 -- 172.21.15.104:0/4045247560 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f800c108880 msgr2=0x7f800c10acf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:00:08.624 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-16T17:00:08.622+0000 7f8011b7b700 1 --2- 172.21.15.104:0/4045247560 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f800c108880 0x7f800c10acf0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f800400b0a0 tx=0x7f800400b3b0 comp rx=0 tx=0).stop 2024-04-16T17:00:08.624 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-16T17:00:08.622+0000 7f8011b7b700 1 -- 172.21.15.104:0/4045247560 shutdown_connections 2024-04-16T17:00:08.624 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-16T17:00:08.622+0000 7f8011b7b700 1 --2- 172.21.15.104:0/4045247560 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f800c108880 0x7f800c10acf0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:00:08.624 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-16T17:00:08.622+0000 7f8011b7b700 1 --2- 172.21.15.104:0/4045247560 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f800c103620 0x7f800c105a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:00:08.624 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-16T17:00:08.622+0000 7f8011b7b700 1 --2- 172.21.15.104:0/4045247560 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f800c105f50 0x7f800c108340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:00:08.624 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-16T17:00:08.622+0000 7f8011b7b700 1 -- 172.21.15.104:0/4045247560 >> 172.21.15.104:0/4045247560 conn(0x7f800c0fd050 msgr2=0x7f800c0ff470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:00:08.624 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-16T17:00:08.622+0000 7f8011b7b700 1 -- 172.21.15.104:0/4045247560 shutdown_connections 2024-04-16T17:00:08.624 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-16T17:00:08.622+0000 7f8011b7b700 1 -- 172.21.15.104:0/4045247560 wait complete. 2024-04-16T17:00:08.625 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-16T17:00:08.622+0000 7f8011b7b700 1 Processor -- start 2024-04-16T17:00:08.625 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-16T17:00:08.623+0000 7f8011b7b700 1 -- start start 2024-04-16T17:00:08.626 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-16T17:00:08.623+0000 7f8011b7b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f800c103620 0x7f800c198d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:00:08.626 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-16T17:00:08.623+0000 7f8011b7b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f800c105f50 0x7f800c199250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:00:08.626 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-16T17:00:08.623+0000 7f8011b7b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f800c108880 0x7f800c1a0290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:00:08.626 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-16T17:00:08.623+0000 7f8011b7b700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f800c10e7c0 con 0x7f800c103620 2024-04-16T17:00:08.626 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-16T17:00:08.623+0000 7f8011b7b700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f800c10e640 con 0x7f800c108880 2024-04-16T17:00:08.626 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-16T17:00:08.623+0000 7f8011b7b700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f800c10e940 con 0x7f800c105f50 2024-04-16T17:00:08.627 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-16T17:00:08.623+0000 7f800bfff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f800c108880 0x7f800c1a0290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:00:08.627 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-16T17:00:08.623+0000 7f800bfff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f800c108880 0x7f800c1a0290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:42306/0 (socket says 172.21.15.104:42306) 2024-04-16T17:00:08.627 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-16T17:00:08.623+0000 7f800bfff700 1 -- 172.21.15.104:0/4005503982 learned_addr learned my addr 172.21.15.104:0/4005503982 (peer_addr_for_me v2:172.21.15.104:0/0) 2024-04-16T17:00:08.627 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-16T17:00:08.623+0000 7f800affd700 1 --2- 172.21.15.104:0/4005503982 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f800c105f50 0x7f800c199250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:00:08.627 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-16T17:00:08.623+0000 7f800bfff700 1 -- 172.21.15.104:0/4005503982 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f800c105f50 msgr2=0x7f800c199250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:00:08.627 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-16T17:00:08.623+0000 7f800bfff700 1 --2- 172.21.15.104:0/4005503982 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f800c105f50 0x7f800c199250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:00:08.627 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-16T17:00:08.623+0000 7f800bfff700 1 -- 172.21.15.104:0/4005503982 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f800c103620 msgr2=0x7f800c198d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:00:08.627 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-16T17:00:08.623+0000 7f800bfff700 1 --2- 172.21.15.104:0/4005503982 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f800c103620 0x7f800c198d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:00:08.627 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-16T17:00:08.623+0000 7f800bfff700 1 -- 172.21.15.104:0/4005503982 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f800c1a0960 con 0x7f800c108880 2024-04-16T17:00:08.627 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-16T17:00:08.623+0000 7f800affd700 1 --2- 172.21.15.104:0/4005503982 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f800c105f50 0x7f800c199250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:00:08.627 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-16T17:00:08.623+0000 7f800bfff700 1 --2- 172.21.15.104:0/4005503982 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f800c108880 0x7f800c1a0290 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f8004007f00 tx=0x7f8004004ef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:00:08.627 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-16T17:00:08.624+0000 7f8008ff9700 1 -- 172.21.15.104:0/4005503982 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8004021070 con 0x7f800c108880 2024-04-16T17:00:08.627 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-16T17:00:08.624+0000 7f8011b7b700 1 -- 172.21.15.104:0/4005503982 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f800c1a0bf0 con 0x7f800c108880 2024-04-16T17:00:08.627 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-16T17:00:08.624+0000 7f8008ff9700 1 -- 172.21.15.104:0/4005503982 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f800401d930 con 0x7f800c108880 2024-04-16T17:00:08.627 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-16T17:00:08.624+0000 7f8008ff9700 1 -- 172.21.15.104:0/4005503982 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8004005640 con 0x7f800c108880 2024-04-16T17:00:08.628 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-16T17:00:08.624+0000 7f8011b7b700 1 -- 172.21.15.104:0/4005503982 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f800c1a1050 con 0x7f800c108880 2024-04-16T17:00:08.628 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-16T17:00:08.625+0000 7f8011b7b700 1 -- 172.21.15.104:0/4005503982 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f800c0697e0 con 0x7f800c108880 2024-04-16T17:00:08.628 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-16T17:00:08.626+0000 7f8008ff9700 1 -- 172.21.15.104:0/4005503982 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f8004024070 con 0x7f800c108880 2024-04-16T17:00:08.628 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-16T17:00:08.626+0000 7f8008ff9700 1 --2- 172.21.15.104:0/4005503982 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f7ff40a3c40 0x7f7ff40a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:00:08.628 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-16T17:00:08.626+0000 7f8008ff9700 1 -- 172.21.15.104:0/4005503982 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 4965+0+0 (secure 0 0 0) 0x7f80040d0b20 con 0x7f800c108880 2024-04-16T17:00:08.628 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-16T17:00:08.627+0000 7f800b7fe700 1 --2- 172.21.15.104:0/4005503982 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f7ff40a3c40 0x7f7ff40a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:00:08.629 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-16T17:00:08.628+0000 7f800b7fe700 1 --2- 172.21.15.104:0/4005503982 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f7ff40a3c40 0x7f7ff40a6100 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f800c19a1f0 tx=0x7f7ffc006d20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:00:08.632 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-16T17:00:08.631+0000 7f8008ff9700 1 -- 172.21.15.104:0/4005503982 <== mon.2 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f800409e2a0 con 0x7f800c108880 2024-04-16T17:00:08.842 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-16T17:00:08.840+0000 7f8011b7b700 1 -- 172.21.15.104:0/4005503982 --> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi104:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f800c062f30 con 0x7f7ff40a3c40 2024-04-16T17:00:10.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:00:09 smithi117 ceph-mon[36730]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-04-16T17:00:10.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:00:09 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-16T17:00:10.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:00:09 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-16T17:00:10.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:00:09 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:00:10.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:00:09 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:00:10.057 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:00:09 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:00:10.057 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:00:09 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:00:10.057 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:00:09 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:00:10.057 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:00:09 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:00:10.057 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:00:09 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:00:10.057 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:00:09 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:00:10.057 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:00:09 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T17:00:10.057 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:00:09 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:00:10.057 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:00:09 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T17:00:10.057 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:00:09 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:00:10.057 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:00:09 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T17:00:10.057 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:00:09 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:00:10.178 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:00:09 smithi104 ceph-mon[36457]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-04-16T17:00:10.178 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:00:09 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-16T17:00:10.178 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:00:09 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-16T17:00:10.178 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:00:09 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:00:10.178 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:00:09 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:00:10.178 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:00:09 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:00:10.179 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:00:09 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:00:10.179 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:00:09 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:00:10.179 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:00:09 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:00:10.179 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:00:09 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:00:10.179 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:00:09 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:00:10.179 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:00:09 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T17:00:10.179 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:00:09 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:00:10.179 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:00:09 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T17:00:10.179 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:00:09 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:00:10.179 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:00:09 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T17:00:10.179 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:00:09 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:00:10.229 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:00:09 smithi045 ceph-mon[31676]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-04-16T17:00:10.229 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:00:09 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-16T17:00:10.229 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:00:09 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-16T17:00:10.229 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:00:09 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:00:10.229 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:00:09 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:00:10.229 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:00:09 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:00:10.229 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:00:09 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:00:10.229 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:00:09 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:00:10.229 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:00:09 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:00:10.229 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:00:09 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:00:10.229 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:00:09 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:00:10.229 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:00:09 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T17:00:10.229 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:00:09 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:00:10.230 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:00:09 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T17:00:10.230 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:00:09 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:00:10.230 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:00:09 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T17:00:10.230 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:00:09 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:00:11.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:00:10 smithi117 ceph-mon[36730]: from='client.24284 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi104:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T17:00:11.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:00:10 smithi117 ceph-mon[36730]: Detected new or changed devices on smithi045 2024-04-16T17:00:11.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:00:10 smithi117 ceph-mon[36730]: Adjusting osd_memory_target on smithi045 to 5698M 2024-04-16T17:00:11.157 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:00:10 smithi045 ceph-mon[31676]: from='client.24284 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi104:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T17:00:11.158 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:00:10 smithi045 ceph-mon[31676]: Detected new or changed devices on smithi045 2024-04-16T17:00:11.158 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:00:10 smithi045 ceph-mon[31676]: Adjusting osd_memory_target on smithi045 to 5698M 2024-04-16T17:00:11.178 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:00:10 smithi104 ceph-mon[36457]: from='client.24284 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi104:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T17:00:11.178 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:00:10 smithi104 ceph-mon[36457]: Detected new or changed devices on smithi045 2024-04-16T17:00:11.178 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:00:10 smithi104 ceph-mon[36457]: Adjusting osd_memory_target on smithi045 to 5698M 2024-04-16T17:00:12.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:00:11 smithi117 ceph-mon[36730]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-04-16T17:00:12.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:00:11 smithi117 ceph-mon[36730]: from='client.? 172.21.15.104:0/3852151008' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b71712a4-7cf6-454d-9fd1-565e3df21c1b"}]: dispatch 2024-04-16T17:00:12.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:00:11 smithi117 ceph-mon[36730]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b71712a4-7cf6-454d-9fd1-565e3df21c1b"}]: dispatch 2024-04-16T17:00:12.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:00:11 smithi117 ceph-mon[36730]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b71712a4-7cf6-454d-9fd1-565e3df21c1b"}]': finished 2024-04-16T17:00:12.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:00:11 smithi117 ceph-mon[36730]: osdmap e49: 8 total, 7 up, 8 in 2024-04-16T17:00:12.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:00:11 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-16T17:00:12.178 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:00:11 smithi104 ceph-mon[36457]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-04-16T17:00:12.178 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:00:11 smithi104 ceph-mon[36457]: from='client.? 172.21.15.104:0/3852151008' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b71712a4-7cf6-454d-9fd1-565e3df21c1b"}]: dispatch 2024-04-16T17:00:12.178 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:00:11 smithi104 ceph-mon[36457]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b71712a4-7cf6-454d-9fd1-565e3df21c1b"}]: dispatch 2024-04-16T17:00:12.178 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:00:11 smithi104 ceph-mon[36457]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b71712a4-7cf6-454d-9fd1-565e3df21c1b"}]': finished 2024-04-16T17:00:12.178 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:00:11 smithi104 ceph-mon[36457]: osdmap e49: 8 total, 7 up, 8 in 2024-04-16T17:00:12.178 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:00:11 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-16T17:00:12.228 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:00:11 smithi045 ceph-mon[31676]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-04-16T17:00:12.228 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:00:11 smithi045 ceph-mon[31676]: from='client.? 172.21.15.104:0/3852151008' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b71712a4-7cf6-454d-9fd1-565e3df21c1b"}]: dispatch 2024-04-16T17:00:12.228 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:00:11 smithi045 ceph-mon[31676]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b71712a4-7cf6-454d-9fd1-565e3df21c1b"}]: dispatch 2024-04-16T17:00:12.229 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:00:11 smithi045 ceph-mon[31676]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b71712a4-7cf6-454d-9fd1-565e3df21c1b"}]': finished 2024-04-16T17:00:12.229 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:00:11 smithi045 ceph-mon[31676]: osdmap e49: 8 total, 7 up, 8 in 2024-04-16T17:00:12.229 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:00:11 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-16T17:00:13.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:00:12 smithi117 ceph-mon[36730]: from='client.? 172.21.15.104:0/4233349585' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-16T17:00:13.079 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:00:12 smithi104 ceph-mon[36457]: from='client.? 172.21.15.104:0/4233349585' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-16T17:00:13.228 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:00:12 smithi045 ceph-mon[31676]: from='client.? 172.21.15.104:0/4233349585' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-16T17:00:14.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:00:13 smithi117 ceph-mon[36730]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-04-16T17:00:14.178 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:00:13 smithi104 ceph-mon[36457]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-04-16T17:00:14.228 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:00:13 smithi045 ceph-mon[31676]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-04-16T17:00:15.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:00:15 smithi104 ceph-mon[36457]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-04-16T17:00:16.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:00:15 smithi117 ceph-mon[36730]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-04-16T17:00:16.228 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:00:15 smithi045 ceph-mon[31676]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-04-16T17:00:18.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:00:17 smithi117 ceph-mon[36730]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-04-16T17:00:18.064 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:00:17 smithi104 ceph-mon[36457]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-04-16T17:00:18.228 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:00:17 smithi045 ceph-mon[31676]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-04-16T17:00:19.855 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:00:19 smithi104 ceph-mon[36457]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-04-16T17:00:19.856 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:00:19 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-16T17:00:19.856 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:00:19 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:00:20.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:00:19 smithi117 ceph-mon[36730]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-04-16T17:00:20.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:00:19 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-16T17:00:20.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:00:19 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:00:20.228 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:00:19 smithi045 ceph-mon[31676]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-04-16T17:00:20.228 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:00:19 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-16T17:00:20.228 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:00:19 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:00:20.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:00:20 smithi104 ceph-mon[36457]: Deploying daemon osd.7 on smithi104 2024-04-16T17:00:21.228 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:00:20 smithi045 ceph-mon[31676]: Deploying daemon osd.7 on smithi104 2024-04-16T17:00:21.306 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:00:20 smithi117 ceph-mon[36730]: Deploying daemon osd.7 on smithi104 2024-04-16T17:00:21.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:00:21 smithi104 ceph-mon[36457]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-04-16T17:00:22.228 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:00:21 smithi045 ceph-mon[31676]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-04-16T17:00:22.306 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:00:21 smithi117 ceph-mon[36730]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-04-16T17:00:24.082 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:00:23 smithi104 ceph-mon[36457]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-04-16T17:00:24.228 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:00:23 smithi045 ceph-mon[31676]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-04-16T17:00:24.305 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:00:23 smithi117 ceph-mon[36730]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-04-16T17:00:25.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:00:25 smithi104 ceph-mon[36457]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-04-16T17:00:25.929 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:00:25 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:00:25.929 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:00:25 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:00:25.929 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:00:25 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T17:00:25.978 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:00:25 smithi045 ceph-mon[31676]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-04-16T17:00:25.978 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:00:25 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:00:25.978 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:00:25 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:00:25.978 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:00:25 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T17:00:26.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:00:25 smithi117 ceph-mon[36730]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-04-16T17:00:26.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:00:25 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:00:26.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:00:25 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:00:26.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:00:25 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T17:00:27.898 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:00:27 smithi104 ceph-mon[36457]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-04-16T17:00:27.978 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:00:27 smithi045 ceph-mon[31676]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-04-16T17:00:28.058 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:00:27 smithi117 ceph-mon[36730]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-04-16T17:00:28.584 INFO:teuthology.orchestra.run.smithi104.stdout:Created osd(s) 7 on host 'smithi104' 2024-04-16T17:00:28.584 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-16T17:00:28.581+0000 7f8008ff9700 1 -- 172.21.15.104:0/4005503982 <== mgr.14152 v2:172.21.15.45:6800/268030699 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f800c062f30 con 0x7f7ff40a3c40 2024-04-16T17:00:28.585 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-16T17:00:28.584+0000 7f8011b7b700 1 -- 172.21.15.104:0/4005503982 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f7ff40a3c40 msgr2=0x7f7ff40a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:00:28.585 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-16T17:00:28.584+0000 7f8011b7b700 1 --2- 172.21.15.104:0/4005503982 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f7ff40a3c40 0x7f7ff40a6100 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f800c19a1f0 tx=0x7f7ffc006d20 comp rx=0 tx=0).stop 2024-04-16T17:00:28.585 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-16T17:00:28.584+0000 7f8011b7b700 1 -- 172.21.15.104:0/4005503982 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f800c108880 msgr2=0x7f800c1a0290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:00:28.586 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-16T17:00:28.584+0000 7f8011b7b700 1 --2- 172.21.15.104:0/4005503982 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f800c108880 0x7f800c1a0290 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f8004007f00 tx=0x7f8004004ef0 comp rx=0 tx=0).stop 2024-04-16T17:00:28.586 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-16T17:00:28.584+0000 7f8011b7b700 1 -- 172.21.15.104:0/4005503982 shutdown_connections 2024-04-16T17:00:28.586 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-16T17:00:28.584+0000 7f8011b7b700 1 --2- 172.21.15.104:0/4005503982 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f800c108880 0x7f800c1a0290 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:00:28.586 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-16T17:00:28.584+0000 7f8011b7b700 1 --2- 172.21.15.104:0/4005503982 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f800c103620 0x7f800c198d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:00:28.586 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-16T17:00:28.584+0000 7f8011b7b700 1 --2- 172.21.15.104:0/4005503982 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f7ff40a3c40 0x7f7ff40a6100 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:00:28.586 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-16T17:00:28.584+0000 7f8011b7b700 1 --2- 172.21.15.104:0/4005503982 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f800c105f50 0x7f800c199250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:00:28.586 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-16T17:00:28.584+0000 7f8011b7b700 1 -- 172.21.15.104:0/4005503982 >> 172.21.15.104:0/4005503982 conn(0x7f800c0fd050 msgr2=0x7f800c103f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:00:28.586 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-16T17:00:28.584+0000 7f8011b7b700 1 -- 172.21.15.104:0/4005503982 shutdown_connections 2024-04-16T17:00:28.586 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-16T17:00:28.584+0000 7f8011b7b700 1 -- 172.21.15.104:0/4005503982 wait complete. 2024-04-16T17:00:29.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:00:28 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:00:29.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:00:28 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:00:29.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:00:28 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:00:29.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:00:28 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T17:00:29.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:00:28 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:00:29.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:00:28 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:00:29.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:00:28 smithi117 ceph-mon[36730]: from='osd.7 [v2:172.21.15.104:6816/4151077186,v1:172.21.15.104:6817/4151077186]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-16T17:00:29.057 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:00:28 smithi117 ceph-mon[36730]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-16T17:00:29.057 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:00:28 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:00:29.057 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:00:28 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T17:00:29.178 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:00:28 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:00:29.178 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:00:28 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:00:29.178 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:00:28 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:00:29.178 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:00:28 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T17:00:29.178 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:00:28 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:00:29.179 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:00:28 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:00:29.179 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:00:28 smithi104 ceph-mon[36457]: from='osd.7 [v2:172.21.15.104:6816/4151077186,v1:172.21.15.104:6817/4151077186]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-16T17:00:29.179 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:00:28 smithi104 ceph-mon[36457]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-16T17:00:29.179 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:00:28 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:00:29.179 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:00:28 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T17:00:29.228 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:00:28 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:00:29.228 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:00:28 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:00:29.228 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:00:28 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:00:29.228 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:00:28 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T17:00:29.229 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:00:28 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:00:29.229 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:00:28 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:00:29.229 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:00:28 smithi045 ceph-mon[31676]: from='osd.7 [v2:172.21.15.104:6816/4151077186,v1:172.21.15.104:6817/4151077186]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-16T17:00:29.229 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:00:28 smithi045 ceph-mon[31676]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-16T17:00:29.229 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:00:28 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:00:29.229 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:00:28 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T17:00:29.244 DEBUG:teuthology.orchestra.run.smithi104:osd.7> sudo journalctl -f -n 0 -u ceph-b1fecd38-fc11-11ee-bc90-c7b262605968@osd.7.service 2024-04-16T17:00:29.246 INFO:tasks.cephadm:Deploying osd.8 on smithi117 with /dev/vg_nvme/lv_2... 2024-04-16T17:00:29.247 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b1fecd38-fc11-11ee-bc90-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-16T17:00:29.491 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/mon.c/config 2024-04-16T17:00:29.918 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:00:29 smithi117 ceph-mon[36730]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-04-16T17:00:29.918 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:00:29 smithi117 ceph-mon[36730]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-16T17:00:29.918 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:00:29 smithi117 ceph-mon[36730]: from='osd.7 [v2:172.21.15.104:6816/4151077186,v1:172.21.15.104:6817/4151077186]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2024-04-16T17:00:29.919 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:00:29 smithi117 ceph-mon[36730]: osdmap e50: 8 total, 7 up, 8 in 2024-04-16T17:00:29.919 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:00:29 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-16T17:00:29.919 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:00:29 smithi117 ceph-mon[36730]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2024-04-16T17:00:29.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:00:29 smithi104 ceph-mon[36457]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-04-16T17:00:29.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:00:29 smithi104 ceph-mon[36457]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-16T17:00:29.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:00:29 smithi104 ceph-mon[36457]: from='osd.7 [v2:172.21.15.104:6816/4151077186,v1:172.21.15.104:6817/4151077186]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2024-04-16T17:00:29.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:00:29 smithi104 ceph-mon[36457]: osdmap e50: 8 total, 7 up, 8 in 2024-04-16T17:00:29.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:00:29 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-16T17:00:29.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:00:29 smithi104 ceph-mon[36457]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2024-04-16T17:00:30.228 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:00:29 smithi045 ceph-mon[31676]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-04-16T17:00:30.229 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:00:29 smithi045 ceph-mon[31676]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-16T17:00:30.229 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:00:29 smithi045 ceph-mon[31676]: from='osd.7 [v2:172.21.15.104:6816/4151077186,v1:172.21.15.104:6817/4151077186]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2024-04-16T17:00:30.229 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:00:29 smithi045 ceph-mon[31676]: osdmap e50: 8 total, 7 up, 8 in 2024-04-16T17:00:30.229 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:00:29 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-16T17:00:30.229 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:00:29 smithi045 ceph-mon[31676]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2024-04-16T17:00:31.429 INFO:journalctl@ceph.osd.7.smithi104.stdout:Apr 16 17:00:31 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-7[55033]: 2024-04-16T17:00:31.038+0000 7f08b9ee7700 -1 osd.7 0 waiting for initial osdmap 2024-04-16T17:00:31.429 INFO:journalctl@ceph.osd.7.smithi104.stdout:Apr 16 17:00:31 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-7[55033]: 2024-04-16T17:00:31.050+0000 7f08b1cd2700 -1 osd.7 51 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-16T17:00:31.536 INFO:teuthology.orchestra.run.smithi117.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-16T17:00:31.536 INFO:teuthology.orchestra.run.smithi117.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-16T17:00:31.536 INFO:teuthology.orchestra.run.smithi117.stderr: stderr: 10+0 records in 2024-04-16T17:00:31.536 INFO:teuthology.orchestra.run.smithi117.stderr:10+0 records out 2024-04-16T17:00:31.536 INFO:teuthology.orchestra.run.smithi117.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0114666 s, 914 MB/s 2024-04-16T17:00:31.536 INFO:teuthology.orchestra.run.smithi117.stderr:--> Zapping successful for: 2024-04-16T17:00:31.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:00:31 smithi104 ceph-mon[36457]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-04-16T17:00:31.929 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:00:31 smithi104 ceph-mon[36457]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]': finished 2024-04-16T17:00:31.929 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:00:31 smithi104 ceph-mon[36457]: osdmap e51: 8 total, 7 up, 8 in 2024-04-16T17:00:31.929 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:00:31 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-16T17:00:31.929 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:00:31 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-16T17:00:31.929 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:00:31 smithi104 ceph-mon[36457]: from='osd.7 ' entity='osd.7' 2024-04-16T17:00:31.929 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:00:31 smithi117 ceph-mon[36730]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-04-16T17:00:31.929 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:00:31 smithi117 ceph-mon[36730]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]': finished 2024-04-16T17:00:31.929 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:00:31 smithi117 ceph-mon[36730]: osdmap e51: 8 total, 7 up, 8 in 2024-04-16T17:00:31.929 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:00:31 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-16T17:00:31.929 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:00:31 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-16T17:00:31.930 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:00:31 smithi117 ceph-mon[36730]: from='osd.7 ' entity='osd.7' 2024-04-16T17:00:31.978 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:00:31 smithi045 ceph-mon[31676]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-04-16T17:00:31.978 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:00:31 smithi045 ceph-mon[31676]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]': finished 2024-04-16T17:00:31.978 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:00:31 smithi045 ceph-mon[31676]: osdmap e51: 8 total, 7 up, 8 in 2024-04-16T17:00:31.978 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:00:31 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-16T17:00:31.978 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:00:31 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-16T17:00:31.978 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:00:31 smithi045 ceph-mon[31676]: from='osd.7 ' entity='osd.7' 2024-04-16T17:00:32.256 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b1fecd38-fc11-11ee-bc90-c7b262605968 -- ceph orch daemon add osd smithi117:vg_nvme/lv_2 2024-04-16T17:00:32.502 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/mon.c/config 2024-04-16T17:00:32.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:00:32 smithi104 ceph-mon[36457]: purged_snaps scrub starts 2024-04-16T17:00:32.679 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:00:32 smithi104 ceph-mon[36457]: purged_snaps scrub ok 2024-04-16T17:00:32.679 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:00:32 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-16T17:00:32.679 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:00:32 smithi104 ceph-mon[36457]: osd.7 [v2:172.21.15.104:6816/4151077186,v1:172.21.15.104:6817/4151077186] boot 2024-04-16T17:00:32.679 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:00:32 smithi104 ceph-mon[36457]: osdmap e52: 8 total, 8 up, 8 in 2024-04-16T17:00:32.679 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:00:32 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-16T17:00:32.679 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:00:32 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:00:32.679 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:00:32 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:00:32.679 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:00:32 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:00:32.679 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:00:32 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:00:32.679 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:00:32 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:00:32.679 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:00:32 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:00:32.679 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:00:32 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:00:32.679 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:00:32 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T17:00:32.680 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:00:32 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:00:32.797 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:00:32 smithi117 ceph-mon[36730]: purged_snaps scrub starts 2024-04-16T17:00:32.797 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:00:32 smithi117 ceph-mon[36730]: purged_snaps scrub ok 2024-04-16T17:00:32.797 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:00:32 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-16T17:00:32.798 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:00:32 smithi117 ceph-mon[36730]: osd.7 [v2:172.21.15.104:6816/4151077186,v1:172.21.15.104:6817/4151077186] boot 2024-04-16T17:00:32.798 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:00:32 smithi117 ceph-mon[36730]: osdmap e52: 8 total, 8 up, 8 in 2024-04-16T17:00:32.798 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:00:32 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-16T17:00:32.798 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:00:32 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:00:32.798 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:00:32 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:00:32.798 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:00:32 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:00:32.798 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:00:32 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:00:32.798 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:00:32 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:00:32.798 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:00:32 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:00:32.798 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:00:32 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:00:32.798 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:00:32 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T17:00:32.798 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:00:32 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:00:32.978 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:00:32 smithi045 ceph-mon[31676]: purged_snaps scrub starts 2024-04-16T17:00:32.979 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:00:32 smithi045 ceph-mon[31676]: purged_snaps scrub ok 2024-04-16T17:00:32.979 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:00:32 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-16T17:00:32.979 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:00:32 smithi045 ceph-mon[31676]: osd.7 [v2:172.21.15.104:6816/4151077186,v1:172.21.15.104:6817/4151077186] boot 2024-04-16T17:00:32.979 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:00:32 smithi045 ceph-mon[31676]: osdmap e52: 8 total, 8 up, 8 in 2024-04-16T17:00:32.979 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:00:32 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-16T17:00:32.979 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:00:32 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:00:32.979 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:00:32 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:00:32.979 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:00:32 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:00:32.979 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:00:32 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:00:32.979 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:00:32 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:00:32.979 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:00:32 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:00:32.979 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:00:32 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:00:32.979 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:00:32 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T17:00:32.979 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:00:32 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:00:33.678 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:00:33 smithi117 ceph-mon[36730]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-04-16T17:00:33.678 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:00:33 smithi117 ceph-mon[36730]: Detected new or changed devices on smithi104 2024-04-16T17:00:33.678 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:00:33 smithi117 ceph-mon[36730]: Adjusting osd_memory_target on smithi104 to 5693M 2024-04-16T17:00:33.678 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:00:33 smithi117 ceph-mon[36730]: osdmap e53: 8 total, 8 up, 8 in 2024-04-16T17:00:33.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:00:33 smithi104 ceph-mon[36457]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-04-16T17:00:33.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:00:33 smithi104 ceph-mon[36457]: Detected new or changed devices on smithi104 2024-04-16T17:00:33.929 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:00:33 smithi104 ceph-mon[36457]: Adjusting osd_memory_target on smithi104 to 5693M 2024-04-16T17:00:33.929 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:00:33 smithi104 ceph-mon[36457]: osdmap e53: 8 total, 8 up, 8 in 2024-04-16T17:00:33.978 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:00:33 smithi045 ceph-mon[31676]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-04-16T17:00:33.978 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:00:33 smithi045 ceph-mon[31676]: Detected new or changed devices on smithi104 2024-04-16T17:00:33.978 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:00:33 smithi045 ceph-mon[31676]: Adjusting osd_memory_target on smithi104 to 5693M 2024-04-16T17:00:33.978 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:00:33 smithi045 ceph-mon[31676]: osdmap e53: 8 total, 8 up, 8 in 2024-04-16T17:00:34.228 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T17:00:34.224+0000 7f02dfe1b700 1 -- 172.21.15.117:0/2078402465 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f02d8103d50 msgr2=0x7f02d81041b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:00:34.228 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T17:00:34.224+0000 7f02dfe1b700 1 --2- 172.21.15.117:0/2078402465 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f02d8103d50 0x7f02d81041b0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f02d4009990 tx=0x7f02d4009ca0 comp rx=0 tx=0).stop 2024-04-16T17:00:34.228 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T17:00:34.224+0000 7f02dfe1b700 1 -- 172.21.15.117:0/2078402465 shutdown_connections 2024-04-16T17:00:34.228 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T17:00:34.224+0000 7f02dfe1b700 1 --2- 172.21.15.117:0/2078402465 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f02d81046f0 0x7f02d810afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:00:34.228 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T17:00:34.224+0000 7f02dfe1b700 1 --2- 172.21.15.117:0/2078402465 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f02d8102b50 0x7f02d8102f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:00:34.228 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T17:00:34.224+0000 7f02dfe1b700 1 --2- 172.21.15.117:0/2078402465 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f02d8103d50 0x7f02d81041b0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:00:34.228 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T17:00:34.224+0000 7f02dfe1b700 1 -- 172.21.15.117:0/2078402465 >> 172.21.15.117:0/2078402465 conn(0x7f02d80fe0f0 msgr2=0x7f02d8100530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:00:34.228 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T17:00:34.224+0000 7f02dfe1b700 1 -- 172.21.15.117:0/2078402465 shutdown_connections 2024-04-16T17:00:34.228 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T17:00:34.224+0000 7f02dfe1b700 1 -- 172.21.15.117:0/2078402465 wait complete. 2024-04-16T17:00:34.228 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T17:00:34.225+0000 7f02dfe1b700 1 Processor -- start 2024-04-16T17:00:34.228 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T17:00:34.225+0000 7f02dfe1b700 1 -- start start 2024-04-16T17:00:34.228 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T17:00:34.225+0000 7f02dfe1b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f02d8102b50 0x7f02d8198de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:00:34.229 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T17:00:34.225+0000 7f02dfe1b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f02d8103d50 0x7f02d8199320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:00:34.229 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T17:00:34.225+0000 7f02dfe1b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f02d81046f0 0x7f02d81a03a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:00:34.229 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T17:00:34.225+0000 7f02dfe1b700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f02d810e7b0 con 0x7f02d81046f0 2024-04-16T17:00:34.229 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T17:00:34.225+0000 7f02dfe1b700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f02d810e630 con 0x7f02d8103d50 2024-04-16T17:00:34.229 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T17:00:34.225+0000 7f02dfe1b700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f02d810e930 con 0x7f02d8102b50 2024-04-16T17:00:34.229 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T17:00:34.225+0000 7f02ddbb7700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f02d8102b50 0x7f02d8198de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:00:34.229 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T17:00:34.225+0000 7f02ddbb7700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f02d8102b50 0x7f02d8198de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:51878/0 (socket says 172.21.15.117:51878) 2024-04-16T17:00:34.229 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T17:00:34.225+0000 7f02ddbb7700 1 -- 172.21.15.117:0/1419965694 learned_addr learned my addr 172.21.15.117:0/1419965694 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-16T17:00:34.229 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T17:00:34.226+0000 7f02ddbb7700 1 -- 172.21.15.117:0/1419965694 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f02d8103d50 msgr2=0x7f02d8199320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:00:34.229 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T17:00:34.226+0000 7f02ddbb7700 1 --2- 172.21.15.117:0/1419965694 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f02d8103d50 0x7f02d8199320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:00:34.229 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T17:00:34.226+0000 7f02ddbb7700 1 -- 172.21.15.117:0/1419965694 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f02d81046f0 msgr2=0x7f02d81a03a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:00:34.229 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T17:00:34.226+0000 7f02ddbb7700 1 --2- 172.21.15.117:0/1419965694 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f02d81046f0 0x7f02d81a03a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:00:34.229 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T17:00:34.226+0000 7f02ddbb7700 1 -- 172.21.15.117:0/1419965694 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02d81a0a40 con 0x7f02d8102b50 2024-04-16T17:00:34.229 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T17:00:34.226+0000 7f02ddbb7700 1 --2- 172.21.15.117:0/1419965694 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f02d8102b50 0x7f02d8198de0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f02cc00b530 tx=0x7f02cc00b8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:00:34.230 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T17:00:34.226+0000 7f02caffd700 1 -- 172.21.15.117:0/1419965694 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02cc015020 con 0x7f02d8102b50 2024-04-16T17:00:34.230 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T17:00:34.226+0000 7f02dfe1b700 1 -- 172.21.15.117:0/1419965694 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02d81a0d30 con 0x7f02d8102b50 2024-04-16T17:00:34.230 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T17:00:34.226+0000 7f02caffd700 1 -- 172.21.15.117:0/1419965694 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f02cc011d10 con 0x7f02d8102b50 2024-04-16T17:00:34.230 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T17:00:34.226+0000 7f02caffd700 1 -- 172.21.15.117:0/1419965694 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02cc00f2e0 con 0x7f02d8102b50 2024-04-16T17:00:34.230 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T17:00:34.226+0000 7f02dfe1b700 1 -- 172.21.15.117:0/1419965694 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02d81a1290 con 0x7f02d8102b50 2024-04-16T17:00:34.231 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T17:00:34.227+0000 7f02dfe1b700 1 -- 172.21.15.117:0/1419965694 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f02d802dfe0 con 0x7f02d8102b50 2024-04-16T17:00:34.231 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T17:00:34.228+0000 7f02caffd700 1 -- 172.21.15.117:0/1419965694 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f02cc018020 con 0x7f02d8102b50 2024-04-16T17:00:34.231 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T17:00:34.229+0000 7f02caffd700 1 --2- 172.21.15.117:0/1419965694 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f02c40a3c40 0x7f02c40a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:00:34.231 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T17:00:34.229+0000 7f02caffd700 1 -- 172.21.15.117:0/1419965694 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 5397+0+0 (secure 0 0 0) 0x7f02cc0cf5d0 con 0x7f02d8102b50 2024-04-16T17:00:34.231 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T17:00:34.229+0000 7f02dd3b6700 1 --2- 172.21.15.117:0/1419965694 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f02c40a3c40 0x7f02c40a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:00:34.231 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T17:00:34.230+0000 7f02dd3b6700 1 --2- 172.21.15.117:0/1419965694 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f02c40a3c40 0x7f02c40a6100 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f02d40039c0 tx=0x7f02d4005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:00:34.234 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T17:00:34.233+0000 7f02caffd700 1 -- 172.21.15.117:0/1419965694 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f02cc01f020 con 0x7f02d8102b50 2024-04-16T17:00:34.443 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T17:00:34.441+0000 7f02dfe1b700 1 -- 172.21.15.117:0/1419965694 --> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi117:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f02d8062f30 con 0x7f02c40a3c40 2024-04-16T17:00:34.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:00:34 smithi117 ceph-mon[36730]: osdmap e54: 8 total, 8 up, 8 in 2024-04-16T17:00:34.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:00:34 smithi117 ceph-mon[36730]: osdmap e55: 8 total, 8 up, 8 in 2024-04-16T17:00:34.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:00:34 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-16T17:00:34.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:00:34 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-16T17:00:34.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:00:34 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:00:34.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:00:34 smithi104 ceph-mon[36457]: osdmap e54: 8 total, 8 up, 8 in 2024-04-16T17:00:34.929 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:00:34 smithi104 ceph-mon[36457]: osdmap e55: 8 total, 8 up, 8 in 2024-04-16T17:00:34.929 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:00:34 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-16T17:00:34.929 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:00:34 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-16T17:00:34.929 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:00:34 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:00:34.978 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:00:34 smithi045 ceph-mon[31676]: osdmap e54: 8 total, 8 up, 8 in 2024-04-16T17:00:34.978 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:00:34 smithi045 ceph-mon[31676]: osdmap e55: 8 total, 8 up, 8 in 2024-04-16T17:00:34.978 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:00:34 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-16T17:00:34.979 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:00:34 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-16T17:00:34.979 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:00:34 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:00:35.755 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:00:35 smithi117 ceph-mon[36730]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-04-16T17:00:35.755 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:00:35 smithi117 ceph-mon[36730]: from='client.24292 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi117:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T17:00:35.755 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:00:35 smithi117 ceph-mon[36730]: osdmap e56: 8 total, 8 up, 8 in 2024-04-16T17:00:35.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:00:35 smithi104 ceph-mon[36457]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-04-16T17:00:35.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:00:35 smithi104 ceph-mon[36457]: from='client.24292 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi117:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T17:00:35.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:00:35 smithi104 ceph-mon[36457]: osdmap e56: 8 total, 8 up, 8 in 2024-04-16T17:00:35.978 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:00:35 smithi045 ceph-mon[31676]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-04-16T17:00:35.978 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:00:35 smithi045 ceph-mon[31676]: from='client.24292 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi117:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T17:00:35.978 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:00:35 smithi045 ceph-mon[31676]: osdmap e56: 8 total, 8 up, 8 in 2024-04-16T17:00:37.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:00:37 smithi104 ceph-mon[36457]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-04-16T17:00:37.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:00:37 smithi104 ceph-mon[36457]: from='client.? 172.21.15.117:0/2607329089' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9e572aef-59a8-450e-a92e-df9dff4d57a4"}]: dispatch 2024-04-16T17:00:37.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:00:37 smithi104 ceph-mon[36457]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9e572aef-59a8-450e-a92e-df9dff4d57a4"}]: dispatch 2024-04-16T17:00:37.929 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:00:37 smithi104 ceph-mon[36457]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9e572aef-59a8-450e-a92e-df9dff4d57a4"}]': finished 2024-04-16T17:00:37.929 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:00:37 smithi104 ceph-mon[36457]: osdmap e57: 9 total, 8 up, 9 in 2024-04-16T17:00:37.929 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:00:37 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-16T17:00:37.978 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:00:37 smithi045 ceph-mon[31676]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-04-16T17:00:37.978 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:00:37 smithi045 ceph-mon[31676]: from='client.? 172.21.15.117:0/2607329089' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9e572aef-59a8-450e-a92e-df9dff4d57a4"}]: dispatch 2024-04-16T17:00:37.978 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:00:37 smithi045 ceph-mon[31676]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9e572aef-59a8-450e-a92e-df9dff4d57a4"}]: dispatch 2024-04-16T17:00:37.978 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:00:37 smithi045 ceph-mon[31676]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9e572aef-59a8-450e-a92e-df9dff4d57a4"}]': finished 2024-04-16T17:00:37.978 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:00:37 smithi045 ceph-mon[31676]: osdmap e57: 9 total, 8 up, 9 in 2024-04-16T17:00:37.978 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:00:37 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-16T17:00:38.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:00:37 smithi117 ceph-mon[36730]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-04-16T17:00:38.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:00:37 smithi117 ceph-mon[36730]: from='client.? 172.21.15.117:0/2607329089' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9e572aef-59a8-450e-a92e-df9dff4d57a4"}]: dispatch 2024-04-16T17:00:38.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:00:37 smithi117 ceph-mon[36730]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9e572aef-59a8-450e-a92e-df9dff4d57a4"}]: dispatch 2024-04-16T17:00:38.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:00:37 smithi117 ceph-mon[36730]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9e572aef-59a8-450e-a92e-df9dff4d57a4"}]': finished 2024-04-16T17:00:38.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:00:37 smithi117 ceph-mon[36730]: osdmap e57: 9 total, 8 up, 9 in 2024-04-16T17:00:38.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:00:37 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-16T17:00:38.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:00:38 smithi104 ceph-mon[36457]: from='client.? 172.21.15.117:0/3425521962' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-16T17:00:38.978 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:00:38 smithi045 ceph-mon[31676]: from='client.? 172.21.15.117:0/3425521962' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-16T17:00:39.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:00:38 smithi117 ceph-mon[36730]: from='client.? 172.21.15.117:0/3425521962' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-16T17:00:39.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:00:39 smithi104 ceph-mon[36457]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-04-16T17:00:39.978 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:00:39 smithi045 ceph-mon[31676]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-04-16T17:00:40.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:00:39 smithi117 ceph-mon[36730]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-04-16T17:00:41.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:00:41 smithi117 ceph-mon[36730]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-04-16T17:00:41.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:00:41 smithi104 ceph-mon[36457]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-04-16T17:00:41.978 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:00:41 smithi045 ceph-mon[31676]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-04-16T17:00:43.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:00:43 smithi117 ceph-mon[36730]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-04-16T17:00:43.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:00:43 smithi104 ceph-mon[36457]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-04-16T17:00:43.978 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:00:43 smithi045 ceph-mon[31676]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-04-16T17:00:44.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:00:44 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-16T17:00:44.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:00:44 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:00:44.978 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:00:44 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-16T17:00:44.978 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:00:44 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:00:45.055 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:00:44 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-16T17:00:45.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:00:44 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:00:45.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:00:45 smithi117 ceph-mon[36730]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-04-16T17:00:45.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:00:45 smithi117 ceph-mon[36730]: Deploying daemon osd.8 on smithi117 2024-04-16T17:00:45.921 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:00:45 smithi104 ceph-mon[36457]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-04-16T17:00:45.921 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:00:45 smithi104 ceph-mon[36457]: Deploying daemon osd.8 on smithi117 2024-04-16T17:00:45.978 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:00:45 smithi045 ceph-mon[31676]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-04-16T17:00:45.978 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:00:45 smithi045 ceph-mon[31676]: Deploying daemon osd.8 on smithi117 2024-04-16T17:00:47.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:00:47 smithi104 ceph-mon[36457]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-04-16T17:00:47.978 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:00:47 smithi045 ceph-mon[31676]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-04-16T17:00:47.983 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:00:47 smithi117 ceph-mon[36730]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-04-16T17:00:49.778 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:00:49 smithi117 ceph-mon[36730]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-04-16T17:00:49.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:00:49 smithi104 ceph-mon[36457]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-04-16T17:00:49.978 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:00:49 smithi045 ceph-mon[31676]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-04-16T17:00:51.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:00:51 smithi104 ceph-mon[36457]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-04-16T17:00:51.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:00:51 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:00:51.679 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:00:51 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T17:00:51.679 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:00:51 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:00:51.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:00:51 smithi045 ceph-mon[31676]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-04-16T17:00:51.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:00:51 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:00:51.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:00:51 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T17:00:51.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:00:51 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:00:51.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:00:51 smithi117 ceph-mon[36730]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-04-16T17:00:51.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:00:51 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:00:51.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:00:51 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T17:00:51.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:00:51 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:00:53.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:00:53 smithi104 ceph-mon[36457]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-04-16T17:00:53.701 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:00:53 smithi117 ceph-mon[36730]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-04-16T17:00:53.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:00:53 smithi045 ceph-mon[31676]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-04-16T17:00:54.087 INFO:teuthology.orchestra.run.smithi117.stdout:Created osd(s) 8 on host 'smithi117' 2024-04-16T17:00:54.087 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T17:00:54.085+0000 7f02caffd700 1 -- 172.21.15.117:0/1419965694 <== mgr.14152 v2:172.21.15.45:6800/268030699 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f02d8062f30 con 0x7f02c40a3c40 2024-04-16T17:00:54.090 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T17:00:54.088+0000 7f02dfe1b700 1 -- 172.21.15.117:0/1419965694 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f02c40a3c40 msgr2=0x7f02c40a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:00:54.090 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T17:00:54.088+0000 7f02dfe1b700 1 --2- 172.21.15.117:0/1419965694 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f02c40a3c40 0x7f02c40a6100 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f02d40039c0 tx=0x7f02d4005fb0 comp rx=0 tx=0).stop 2024-04-16T17:00:54.090 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T17:00:54.088+0000 7f02dfe1b700 1 -- 172.21.15.117:0/1419965694 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f02d8102b50 msgr2=0x7f02d8198de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:00:54.090 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T17:00:54.088+0000 7f02dfe1b700 1 --2- 172.21.15.117:0/1419965694 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f02d8102b50 0x7f02d8198de0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f02cc00b530 tx=0x7f02cc00b8f0 comp rx=0 tx=0).stop 2024-04-16T17:00:54.090 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T17:00:54.088+0000 7f02dfe1b700 1 -- 172.21.15.117:0/1419965694 shutdown_connections 2024-04-16T17:00:54.090 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T17:00:54.088+0000 7f02dfe1b700 1 --2- 172.21.15.117:0/1419965694 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f02c40a3c40 0x7f02c40a6100 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:00:54.090 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T17:00:54.088+0000 7f02dfe1b700 1 --2- 172.21.15.117:0/1419965694 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f02d8102b50 0x7f02d8198de0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:00:54.090 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T17:00:54.088+0000 7f02dfe1b700 1 --2- 172.21.15.117:0/1419965694 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f02d81046f0 0x7f02d81a03a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:00:54.090 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T17:00:54.088+0000 7f02dfe1b700 1 --2- 172.21.15.117:0/1419965694 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f02d8103d50 0x7f02d8199320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:00:54.090 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T17:00:54.088+0000 7f02dfe1b700 1 -- 172.21.15.117:0/1419965694 >> 172.21.15.117:0/1419965694 conn(0x7f02d80fe0f0 msgr2=0x7f02d8106ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:00:54.090 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T17:00:54.088+0000 7f02dfe1b700 1 -- 172.21.15.117:0/1419965694 shutdown_connections 2024-04-16T17:00:54.090 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T17:00:54.088+0000 7f02dfe1b700 1 -- 172.21.15.117:0/1419965694 wait complete. 2024-04-16T17:00:54.793 DEBUG:teuthology.orchestra.run.smithi117:osd.8> sudo journalctl -f -n 0 -u ceph-b1fecd38-fc11-11ee-bc90-c7b262605968@osd.8.service 2024-04-16T17:00:54.795 INFO:tasks.cephadm:Deploying osd.9 on smithi045 with /dev/vg_nvme/lv_1... 2024-04-16T17:00:54.795 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b1fecd38-fc11-11ee-bc90-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-16T17:00:55.228 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:00:55 smithi045 ceph-mon[31676]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-04-16T17:00:55.228 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:00:55 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:00:55.228 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:00:55 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:00:55.228 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:00:55 smithi045 ceph-mon[31676]: from='osd.8 [v2:172.21.15.117:6816/1500286830,v1:172.21.15.117:6817/1500286830]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-16T17:00:55.229 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:00:55 smithi045 ceph-mon[31676]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-16T17:00:55.229 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:00:55 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:00:55.229 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:00:55 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:00:55.274 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/mon.a/config 2024-04-16T17:00:55.306 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:00:55 smithi117 ceph-mon[36730]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-04-16T17:00:55.306 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:00:55 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:00:55.306 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:00:55 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:00:55.306 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:00:55 smithi117 ceph-mon[36730]: from='osd.8 [v2:172.21.15.117:6816/1500286830,v1:172.21.15.117:6817/1500286830]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-16T17:00:55.306 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:00:55 smithi117 ceph-mon[36730]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-16T17:00:55.306 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:00:55 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:00:55.306 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:00:55 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:00:55.422 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:00:55 smithi104 ceph-mon[36457]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-04-16T17:00:55.423 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:00:55 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:00:55.423 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:00:55 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:00:55.423 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:00:55 smithi104 ceph-mon[36457]: from='osd.8 [v2:172.21.15.117:6816/1500286830,v1:172.21.15.117:6817/1500286830]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-16T17:00:55.423 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:00:55 smithi104 ceph-mon[36457]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-16T17:00:55.423 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:00:55 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:00:55.423 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:00:55 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:00:56.229 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:00:56 smithi045 ceph-mon[31676]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-16T17:00:56.229 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:00:56 smithi045 ceph-mon[31676]: from='osd.8 [v2:172.21.15.117:6816/1500286830,v1:172.21.15.117:6817/1500286830]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]: dispatch 2024-04-16T17:00:56.229 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:00:56 smithi045 ceph-mon[31676]: osdmap e58: 9 total, 8 up, 9 in 2024-04-16T17:00:56.229 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:00:56 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-16T17:00:56.229 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:00:56 smithi045 ceph-mon[31676]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]: dispatch 2024-04-16T17:00:56.306 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:00:56 smithi117 ceph-mon[36730]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-16T17:00:56.306 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:00:56 smithi117 ceph-mon[36730]: from='osd.8 [v2:172.21.15.117:6816/1500286830,v1:172.21.15.117:6817/1500286830]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]: dispatch 2024-04-16T17:00:56.306 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:00:56 smithi117 ceph-mon[36730]: osdmap e58: 9 total, 8 up, 9 in 2024-04-16T17:00:56.306 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:00:56 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-16T17:00:56.306 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:00:56 smithi117 ceph-mon[36730]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]: dispatch 2024-04-16T17:00:56.428 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:00:56 smithi104 ceph-mon[36457]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-16T17:00:56.428 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:00:56 smithi104 ceph-mon[36457]: from='osd.8 [v2:172.21.15.117:6816/1500286830,v1:172.21.15.117:6817/1500286830]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]: dispatch 2024-04-16T17:00:56.428 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:00:56 smithi104 ceph-mon[36457]: osdmap e58: 9 total, 8 up, 9 in 2024-04-16T17:00:56.428 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:00:56 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-16T17:00:56.429 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:00:56 smithi104 ceph-mon[36457]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]: dispatch 2024-04-16T17:00:56.806 INFO:journalctl@ceph.osd.8.smithi117.stdout:Apr 16 17:00:56 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-8[55817]: 2024-04-16T17:00:56.520+0000 7f65b1daa700 -1 osd.8 0 waiting for initial osdmap 2024-04-16T17:00:56.806 INFO:journalctl@ceph.osd.8.smithi117.stdout:Apr 16 17:00:56 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-8[55817]: 2024-04-16T17:00:56.532+0000 7f65a9b95700 -1 osd.8 59 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-16T17:00:57.352 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:00:57 smithi045 ceph-mon[31676]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-04-16T17:00:57.353 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:00:57 smithi045 ceph-mon[31676]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]': finished 2024-04-16T17:00:57.353 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:00:57 smithi045 ceph-mon[31676]: osdmap e59: 9 total, 8 up, 9 in 2024-04-16T17:00:57.353 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:00:57 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-16T17:00:57.353 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:00:57 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-16T17:00:57.353 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:00:57 smithi045 ceph-mon[31676]: from='osd.8 ' entity='osd.8' 2024-04-16T17:00:57.428 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:00:57 smithi104 ceph-mon[36457]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-04-16T17:00:57.428 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:00:57 smithi104 ceph-mon[36457]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]': finished 2024-04-16T17:00:57.428 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:00:57 smithi104 ceph-mon[36457]: osdmap e59: 9 total, 8 up, 9 in 2024-04-16T17:00:57.428 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:00:57 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-16T17:00:57.428 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:00:57 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-16T17:00:57.429 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:00:57 smithi104 ceph-mon[36457]: from='osd.8 ' entity='osd.8' 2024-04-16T17:00:57.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:00:57 smithi117 ceph-mon[36730]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-04-16T17:00:57.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:00:57 smithi117 ceph-mon[36730]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]': finished 2024-04-16T17:00:57.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:00:57 smithi117 ceph-mon[36730]: osdmap e59: 9 total, 8 up, 9 in 2024-04-16T17:00:57.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:00:57 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-16T17:00:57.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:00:57 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-16T17:00:57.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:00:57 smithi117 ceph-mon[36730]: from='osd.8 ' entity='osd.8' 2024-04-16T17:00:58.306 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:00:58 smithi117 ceph-mon[36730]: purged_snaps scrub starts 2024-04-16T17:00:58.306 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:00:58 smithi117 ceph-mon[36730]: purged_snaps scrub ok 2024-04-16T17:00:58.306 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:00:58 smithi117 ceph-mon[36730]: osd.8 [v2:172.21.15.117:6816/1500286830,v1:172.21.15.117:6817/1500286830] boot 2024-04-16T17:00:58.306 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:00:58 smithi117 ceph-mon[36730]: osdmap e60: 9 total, 9 up, 9 in 2024-04-16T17:00:58.306 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:00:58 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-16T17:00:58.307 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:00:58 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:00:58.307 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:00:58 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:00:58.307 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:00:58 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:00:58.307 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:00:58 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:00:58.307 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:00:58 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:00:58.307 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:00:58 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:00:58.307 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:00:58 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:00:58.307 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:00:58 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T17:00:58.307 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:00:58 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:00:58.428 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:00:58 smithi104 ceph-mon[36457]: purged_snaps scrub starts 2024-04-16T17:00:58.428 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:00:58 smithi104 ceph-mon[36457]: purged_snaps scrub ok 2024-04-16T17:00:58.429 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:00:58 smithi104 ceph-mon[36457]: osd.8 [v2:172.21.15.117:6816/1500286830,v1:172.21.15.117:6817/1500286830] boot 2024-04-16T17:00:58.429 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:00:58 smithi104 ceph-mon[36457]: osdmap e60: 9 total, 9 up, 9 in 2024-04-16T17:00:58.429 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:00:58 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-16T17:00:58.429 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:00:58 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:00:58.429 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:00:58 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:00:58.429 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:00:58 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:00:58.429 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:00:58 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:00:58.429 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:00:58 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:00:58.429 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:00:58 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:00:58.429 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:00:58 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:00:58.429 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:00:58 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T17:00:58.429 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:00:58 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:00:58.471 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:00:58 smithi045 ceph-mon[31676]: purged_snaps scrub starts 2024-04-16T17:00:58.471 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:00:58 smithi045 ceph-mon[31676]: purged_snaps scrub ok 2024-04-16T17:00:58.471 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:00:58 smithi045 ceph-mon[31676]: osd.8 [v2:172.21.15.117:6816/1500286830,v1:172.21.15.117:6817/1500286830] boot 2024-04-16T17:00:58.471 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:00:58 smithi045 ceph-mon[31676]: osdmap e60: 9 total, 9 up, 9 in 2024-04-16T17:00:58.472 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:00:58 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-16T17:00:58.472 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:00:58 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:00:58.472 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:00:58 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:00:58.472 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:00:58 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:00:58.472 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:00:58 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:00:58.472 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:00:58 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:00:58.472 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:00:58 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:00:58.472 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:00:58 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:00:58.472 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:00:58 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T17:00:58.472 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:00:58 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:00:58.472 INFO:teuthology.orchestra.run.smithi045.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-16T17:00:58.473 INFO:teuthology.orchestra.run.smithi045.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-16T17:00:58.473 INFO:teuthology.orchestra.run.smithi045.stderr: stderr: 10+0 records in 2024-04-16T17:00:58.473 INFO:teuthology.orchestra.run.smithi045.stderr:10+0 records out 2024-04-16T17:00:58.473 INFO:teuthology.orchestra.run.smithi045.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0350845 s, 299 MB/s 2024-04-16T17:00:58.473 INFO:teuthology.orchestra.run.smithi045.stderr:--> Zapping successful for: 2024-04-16T17:00:59.118 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:00:59 smithi045 ceph-mon[31676]: Detected new or changed devices on smithi117 2024-04-16T17:00:59.118 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:00:59 smithi045 ceph-mon[31676]: Adjusting osd_memory_target on smithi117 to 5693M 2024-04-16T17:00:59.321 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b1fecd38-fc11-11ee-bc90-c7b262605968 -- ceph orch daemon add osd smithi045:vg_nvme/lv_1 2024-04-16T17:00:59.413 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:00:59 smithi045 ceph-mon[31676]: pgmap v219: 1 pgs: 1 remapped+peering; 577 KiB data, 288 MiB used, 804 GiB / 805 GiB avail 2024-04-16T17:00:59.413 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:00:59 smithi045 ceph-mon[31676]: osdmap e61: 9 total, 9 up, 9 in 2024-04-16T17:00:59.413 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:00:59 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T17:00:59.414 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:00:59 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:00:59.414 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:00:59 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T17:00:59.414 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:00:59 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:00:59.428 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:00:59 smithi104 ceph-mon[36457]: Detected new or changed devices on smithi117 2024-04-16T17:00:59.428 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:00:59 smithi104 ceph-mon[36457]: Adjusting osd_memory_target on smithi117 to 5693M 2024-04-16T17:00:59.428 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:00:59 smithi104 ceph-mon[36457]: pgmap v219: 1 pgs: 1 remapped+peering; 577 KiB data, 288 MiB used, 804 GiB / 805 GiB avail 2024-04-16T17:00:59.429 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:00:59 smithi104 ceph-mon[36457]: osdmap e61: 9 total, 9 up, 9 in 2024-04-16T17:00:59.429 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:00:59 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T17:00:59.429 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:00:59 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:00:59.429 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:00:59 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T17:00:59.429 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:00:59 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:00:59.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:00:59 smithi117 ceph-mon[36730]: Detected new or changed devices on smithi117 2024-04-16T17:00:59.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:00:59 smithi117 ceph-mon[36730]: Adjusting osd_memory_target on smithi117 to 5693M 2024-04-16T17:00:59.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:00:59 smithi117 ceph-mon[36730]: pgmap v219: 1 pgs: 1 remapped+peering; 577 KiB data, 288 MiB used, 804 GiB / 805 GiB avail 2024-04-16T17:00:59.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:00:59 smithi117 ceph-mon[36730]: osdmap e61: 9 total, 9 up, 9 in 2024-04-16T17:00:59.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:00:59 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T17:00:59.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:00:59 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:00:59.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:00:59 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T17:00:59.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:00:59 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:00:59.780 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/mon.a/config 2024-04-16T17:01:00.228 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:01:00 smithi045 ceph-mon[31676]: osdmap e62: 9 total, 9 up, 9 in 2024-04-16T17:01:00.428 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:01:00 smithi104 ceph-mon[36457]: osdmap e62: 9 total, 9 up, 9 in 2024-04-16T17:01:00.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:01:00 smithi117 ceph-mon[36730]: osdmap e62: 9 total, 9 up, 9 in 2024-04-16T17:01:01.228 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:01:01 smithi045 ceph-mon[31676]: pgmap v222: 1 pgs: 1 remapped+peering; 577 KiB data, 288 MiB used, 804 GiB / 805 GiB avail 2024-04-16T17:01:01.228 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:01:01 smithi045 ceph-mon[31676]: osdmap e63: 9 total, 9 up, 9 in 2024-04-16T17:01:01.228 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:01:01 smithi045 ceph-mon[31676]: osdmap e64: 9 total, 9 up, 9 in 2024-04-16T17:01:01.388 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:01:01 smithi104 ceph-mon[36457]: pgmap v222: 1 pgs: 1 remapped+peering; 577 KiB data, 288 MiB used, 804 GiB / 805 GiB avail 2024-04-16T17:01:01.388 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:01:01 smithi104 ceph-mon[36457]: osdmap e63: 9 total, 9 up, 9 in 2024-04-16T17:01:01.388 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:01:01 smithi104 ceph-mon[36457]: osdmap e64: 9 total, 9 up, 9 in 2024-04-16T17:01:01.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:01:01 smithi117 ceph-mon[36730]: pgmap v222: 1 pgs: 1 remapped+peering; 577 KiB data, 288 MiB used, 804 GiB / 805 GiB avail 2024-04-16T17:01:01.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:01:01 smithi117 ceph-mon[36730]: osdmap e63: 9 total, 9 up, 9 in 2024-04-16T17:01:01.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:01:01 smithi117 ceph-mon[36730]: osdmap e64: 9 total, 9 up, 9 in 2024-04-16T17:01:01.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:01:01.974+0000 7f19cd800700 1 -- 172.21.15.45:0/664916203 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f19c8105e30 msgr2=0x7f19c8108220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:01:01.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:01:01.974+0000 7f19cd800700 1 --2- 172.21.15.45:0/664916203 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f19c8105e30 0x7f19c8108220 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f19b4009990 tx=0x7f19b4009ca0 comp rx=0 tx=0).stop 2024-04-16T17:01:01.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:01:01.974+0000 7f19cd800700 1 -- 172.21.15.45:0/664916203 shutdown_connections 2024-04-16T17:01:01.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:01:01.974+0000 7f19cd800700 1 --2- 172.21.15.45:0/664916203 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f19c8103500 0x7f19c81058f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:01.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:01:01.974+0000 7f19cd800700 1 --2- 172.21.15.45:0/664916203 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f19c8108760 0x7f19c810abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:01.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:01:01.974+0000 7f19cd800700 1 --2- 172.21.15.45:0/664916203 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f19c8105e30 0x7f19c8108220 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:01.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:01:01.974+0000 7f19cd800700 1 -- 172.21.15.45:0/664916203 >> 172.21.15.45:0/664916203 conn(0x7f19c80fcf10 msgr2=0x7f19c80ff350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:01:01.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:01:01.974+0000 7f19cd800700 1 -- 172.21.15.45:0/664916203 shutdown_connections 2024-04-16T17:01:01.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:01:01.974+0000 7f19cd800700 1 -- 172.21.15.45:0/664916203 wait complete. 2024-04-16T17:01:01.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:01:01.976+0000 7f19cd800700 1 Processor -- start 2024-04-16T17:01:01.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:01:01.976+0000 7f19cd800700 1 -- start start 2024-04-16T17:01:01.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:01:01.976+0000 7f19cd800700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f19c8103500 0x7f19c819aea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:01:01.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:01:01.976+0000 7f19cd800700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f19c8105e30 0x7f19c819b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:01:01.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:01:01.976+0000 7f19cd800700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f19c8108760 0x7f19c81a2460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:01:01.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:01:01.976+0000 7f19cd800700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f19c810e6a0 con 0x7f19c8108760 2024-04-16T17:01:01.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:01:01.976+0000 7f19cd800700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f19c810e520 con 0x7f19c8103500 2024-04-16T17:01:01.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:01:01.976+0000 7f19cd800700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f19c810e820 con 0x7f19c8105e30 2024-04-16T17:01:01.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:01:01.977+0000 7f19c7e00700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f19c8103500 0x7f19c819aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:01:01.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:01:01.977+0000 7f19cce00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f19c8108760 0x7f19c81a2460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:01:01.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:01:01.977+0000 7f19c7e00700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f19c8103500 0x7f19c819aea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:43100/0 (socket says 172.21.15.45:43100) 2024-04-16T17:01:01.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:01:01.977+0000 7f19c7e00700 1 -- 172.21.15.45:0/140071 learned_addr learned my addr 172.21.15.45:0/140071 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:01:01.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:01:01.977+0000 7f19c7400700 1 --2- 172.21.15.45:0/140071 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f19c8105e30 0x7f19c819b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:01:01.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:01:01.977+0000 7f19c7e00700 1 -- 172.21.15.45:0/140071 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f19c8105e30 msgr2=0x7f19c819b3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:01:01.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:01:01.977+0000 7f19c7e00700 1 --2- 172.21.15.45:0/140071 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f19c8105e30 0x7f19c819b3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:01.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:01:01.977+0000 7f19c7e00700 1 -- 172.21.15.45:0/140071 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f19c8108760 msgr2=0x7f19c81a2460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:01:01.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:01:01.977+0000 7f19c7e00700 1 --2- 172.21.15.45:0/140071 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f19c8108760 0x7f19c81a2460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:01.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:01:01.977+0000 7f19c7e00700 1 -- 172.21.15.45:0/140071 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19c81a2b00 con 0x7f19c8103500 2024-04-16T17:01:01.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:01:01.977+0000 7f19c7400700 1 --2- 172.21.15.45:0/140071 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f19c8105e30 0x7f19c819b3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:01:01.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:01:01.978+0000 7f19c7e00700 1 --2- 172.21.15.45:0/140071 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f19c8103500 0x7f19c819aea0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f19b800e9a0 tx=0x7f19b800ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:01:01.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:01:01.978+0000 7f19c4c00700 1 -- 172.21.15.45:0/140071 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19b800cbd0 con 0x7f19c8103500 2024-04-16T17:01:01.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:01:01.978+0000 7f19c4c00700 1 -- 172.21.15.45:0/140071 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f19b800cd70 con 0x7f19c8103500 2024-04-16T17:01:01.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:01:01.978+0000 7f19cd800700 1 -- 172.21.15.45:0/140071 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19c81a2df0 con 0x7f19c8103500 2024-04-16T17:01:01.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:01:01.978+0000 7f19c4c00700 1 -- 172.21.15.45:0/140071 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19b80187a0 con 0x7f19c8103500 2024-04-16T17:01:01.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:01:01.978+0000 7f19cd800700 1 -- 172.21.15.45:0/140071 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19c81a3250 con 0x7f19c8103500 2024-04-16T17:01:01.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:01:01.979+0000 7f19cd800700 1 -- 172.21.15.45:0/140071 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19c80697e0 con 0x7f19c8103500 2024-04-16T17:01:01.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:01:01.981+0000 7f19c4c00700 1 -- 172.21.15.45:0/140071 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f19b80109a0 con 0x7f19c8103500 2024-04-16T17:01:01.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:01:01.983+0000 7f19c4c00700 1 --2- 172.21.15.45:0/140071 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f19ac0a3cc0 0x7f19ac0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:01:01.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:01:01.983+0000 7f19c4c00700 1 -- 172.21.15.45:0/140071 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 5829+0+0 (secure 0 0 0) 0x7f19b80cfc20 con 0x7f19c8103500 2024-04-16T17:01:01.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:01:01.983+0000 7f19c7400700 1 --2- 172.21.15.45:0/140071 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f19ac0a3cc0 0x7f19ac0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:01:01.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:01:01.984+0000 7f19c7400700 1 --2- 172.21.15.45:0/140071 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f19ac0a3cc0 0x7f19ac0a6180 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f19b400b2e0 tx=0x7f19b401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:01:01.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:01:01.986+0000 7f19c4c00700 1 -- 172.21.15.45:0/140071 <== mon.2 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f19b809cf10 con 0x7f19c8103500 2024-04-16T17:01:02.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:01:02.234+0000 7f19cd800700 1 -- 172.21.15.45:0/140071 --> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi045:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f19c8062f30 con 0x7f19ac0a3cc0 2024-04-16T17:01:03.424 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:01:03 smithi045 ceph-mon[31676]: pgmap v225: 1 pgs: 1 remapped+peering; 577 KiB data, 288 MiB used, 804 GiB / 805 GiB avail 2024-04-16T17:01:03.424 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:01:03 smithi045 ceph-mon[31676]: from='client.24320 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi045:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T17:01:03.424 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:01:03 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-16T17:01:03.424 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:01:03 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-16T17:01:03.424 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:01:03 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:01:03.428 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:01:03 smithi104 ceph-mon[36457]: pgmap v225: 1 pgs: 1 remapped+peering; 577 KiB data, 288 MiB used, 804 GiB / 805 GiB avail 2024-04-16T17:01:03.428 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:01:03 smithi104 ceph-mon[36457]: from='client.24320 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi045:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T17:01:03.428 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:01:03 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-16T17:01:03.429 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:01:03 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-16T17:01:03.429 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:01:03 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:01:03.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:01:03 smithi117 ceph-mon[36730]: pgmap v225: 1 pgs: 1 remapped+peering; 577 KiB data, 288 MiB used, 804 GiB / 805 GiB avail 2024-04-16T17:01:03.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:01:03 smithi117 ceph-mon[36730]: from='client.24320 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi045:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T17:01:03.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:01:03 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-16T17:01:03.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:01:03 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-16T17:01:03.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:01:03 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:01:05.428 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:01:05 smithi104 ceph-mon[36457]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 97 KiB/s, 0 objects/s recovering 2024-04-16T17:01:05.478 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:01:05 smithi045 ceph-mon[31676]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 97 KiB/s, 0 objects/s recovering 2024-04-16T17:01:05.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:01:05 smithi117 ceph-mon[36730]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 97 KiB/s, 0 objects/s recovering 2024-04-16T17:01:07.428 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:01:07 smithi104 ceph-mon[36457]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 83 KiB/s, 0 objects/s recovering 2024-04-16T17:01:07.429 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:01:07 smithi104 ceph-mon[36457]: from='client.? 172.21.15.45:0/2724412549' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "014e9598-c413-4cc4-9bb8-61c6e8f59d30"}]: dispatch 2024-04-16T17:01:07.429 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:01:07 smithi104 ceph-mon[36457]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "014e9598-c413-4cc4-9bb8-61c6e8f59d30"}]: dispatch 2024-04-16T17:01:07.429 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:01:07 smithi104 ceph-mon[36457]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "014e9598-c413-4cc4-9bb8-61c6e8f59d30"}]': finished 2024-04-16T17:01:07.429 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:01:07 smithi104 ceph-mon[36457]: osdmap e65: 10 total, 9 up, 10 in 2024-04-16T17:01:07.429 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:01:07 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-16T17:01:07.460 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:01:07 smithi045 ceph-mon[31676]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 83 KiB/s, 0 objects/s recovering 2024-04-16T17:01:07.460 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:01:07 smithi045 ceph-mon[31676]: from='client.? 172.21.15.45:0/2724412549' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "014e9598-c413-4cc4-9bb8-61c6e8f59d30"}]: dispatch 2024-04-16T17:01:07.460 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:01:07 smithi045 ceph-mon[31676]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "014e9598-c413-4cc4-9bb8-61c6e8f59d30"}]: dispatch 2024-04-16T17:01:07.461 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:01:07 smithi045 ceph-mon[31676]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "014e9598-c413-4cc4-9bb8-61c6e8f59d30"}]': finished 2024-04-16T17:01:07.461 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:01:07 smithi045 ceph-mon[31676]: osdmap e65: 10 total, 9 up, 10 in 2024-04-16T17:01:07.461 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:01:07 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-16T17:01:07.553 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:01:07 smithi117 ceph-mon[36730]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 83 KiB/s, 0 objects/s recovering 2024-04-16T17:01:07.553 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:01:07 smithi117 ceph-mon[36730]: from='client.? 172.21.15.45:0/2724412549' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "014e9598-c413-4cc4-9bb8-61c6e8f59d30"}]: dispatch 2024-04-16T17:01:07.553 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:01:07 smithi117 ceph-mon[36730]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "014e9598-c413-4cc4-9bb8-61c6e8f59d30"}]: dispatch 2024-04-16T17:01:07.553 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:01:07 smithi117 ceph-mon[36730]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "014e9598-c413-4cc4-9bb8-61c6e8f59d30"}]': finished 2024-04-16T17:01:07.553 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:01:07 smithi117 ceph-mon[36730]: osdmap e65: 10 total, 9 up, 10 in 2024-04-16T17:01:07.553 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:01:07 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-16T17:01:08.428 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:01:08 smithi104 ceph-mon[36457]: from='client.? 172.21.15.45:0/3129858325' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-16T17:01:08.478 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:01:08 smithi045 ceph-mon[31676]: from='client.? 172.21.15.45:0/3129858325' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-16T17:01:08.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:01:08 smithi117 ceph-mon[36730]: from='client.? 172.21.15.45:0/3129858325' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-16T17:01:09.428 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:01:09 smithi104 ceph-mon[36457]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-04-16T17:01:09.478 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:01:09 smithi045 ceph-mon[31676]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-04-16T17:01:09.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:01:09 smithi117 ceph-mon[36730]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-04-16T17:01:11.428 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:01:11 smithi104 ceph-mon[36457]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-04-16T17:01:11.478 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:01:11 smithi045 ceph-mon[31676]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-04-16T17:01:11.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:01:11 smithi117 ceph-mon[36730]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-04-16T17:01:13.428 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:01:13 smithi104 ceph-mon[36457]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-16T17:01:13.478 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:01:13 smithi045 ceph-mon[31676]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-16T17:01:13.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:01:13 smithi117 ceph-mon[36730]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-16T17:01:15.428 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:01:15 smithi104 ceph-mon[36457]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-04-16T17:01:15.478 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:01:15 smithi045 ceph-mon[31676]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-04-16T17:01:15.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:01:15 smithi117 ceph-mon[36730]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-04-16T17:01:17.306 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:01:17 smithi117 ceph-mon[36730]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-04-16T17:01:17.306 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:01:17 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-16T17:01:17.307 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:01:17 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:01:17.307 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:01:17 smithi117 ceph-mon[36730]: Deploying daemon osd.9 on smithi045 2024-04-16T17:01:17.428 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:01:17 smithi104 ceph-mon[36457]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-04-16T17:01:17.428 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:01:17 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-16T17:01:17.428 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:01:17 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:01:17.429 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:01:17 smithi104 ceph-mon[36457]: Deploying daemon osd.9 on smithi045 2024-04-16T17:01:17.431 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:01:17 smithi045 ceph-mon[31676]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-04-16T17:01:17.431 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:01:17 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-16T17:01:17.431 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:01:17 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:01:17.431 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:01:17 smithi045 ceph-mon[31676]: Deploying daemon osd.9 on smithi045 2024-04-16T17:01:19.478 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:01:19 smithi045 ceph-mon[31676]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-04-16T17:01:19.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:01:19 smithi117 ceph-mon[36730]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-04-16T17:01:19.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:01:19 smithi104 ceph-mon[36457]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-04-16T17:01:21.416 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:01:21 smithi045 ceph-mon[31676]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-04-16T17:01:21.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:01:21 smithi117 ceph-mon[36730]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-04-16T17:01:21.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:01:21 smithi104 ceph-mon[36457]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-04-16T17:01:23.479 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:01:23 smithi045 ceph-mon[31676]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-04-16T17:01:23.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:01:23 smithi117 ceph-mon[36730]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-04-16T17:01:23.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:01:23 smithi104 ceph-mon[36457]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-04-16T17:01:25.478 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:01:25 smithi045 ceph-mon[31676]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-04-16T17:01:25.478 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:01:25 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:01:25.478 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:01:25 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:01:25.478 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:01:25 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T17:01:25.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:01:25 smithi117 ceph-mon[36730]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-04-16T17:01:25.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:01:25 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:01:25.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:01:25 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:01:25.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:01:25 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T17:01:25.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:01:25 smithi104 ceph-mon[36457]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-04-16T17:01:25.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:01:25 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:01:25.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:01:25 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:01:25.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:01:25 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T17:01:27.478 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:01:27 smithi045 ceph-mon[31676]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-04-16T17:01:27.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:01:27 smithi117 ceph-mon[36730]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-04-16T17:01:27.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:01:27 smithi104 ceph-mon[36457]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-04-16T17:01:29.907 INFO:teuthology.orchestra.run.smithi045.stdout:Created osd(s) 9 on host 'smithi045' 2024-04-16T17:01:29.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:01:29.901+0000 7f19c4c00700 1 -- 172.21.15.45:0/140071 <== mgr.14152 v2:172.21.15.45:6800/268030699 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f19c8062f30 con 0x7f19ac0a3cc0 2024-04-16T17:01:29.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:01:29.905+0000 7f19cd800700 1 -- 172.21.15.45:0/140071 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f19ac0a3cc0 msgr2=0x7f19ac0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:01:29.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:01:29.905+0000 7f19cd800700 1 --2- 172.21.15.45:0/140071 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f19ac0a3cc0 0x7f19ac0a6180 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f19b400b2e0 tx=0x7f19b401a040 comp rx=0 tx=0).stop 2024-04-16T17:01:29.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:01:29.905+0000 7f19cd800700 1 -- 172.21.15.45:0/140071 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f19c8103500 msgr2=0x7f19c819aea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:01:29.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:01:29.905+0000 7f19cd800700 1 --2- 172.21.15.45:0/140071 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f19c8103500 0x7f19c819aea0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f19b800e9a0 tx=0x7f19b800ecb0 comp rx=0 tx=0).stop 2024-04-16T17:01:29.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:01:29.905+0000 7f19cd800700 1 -- 172.21.15.45:0/140071 shutdown_connections 2024-04-16T17:01:29.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:01:29.905+0000 7f19cd800700 1 --2- 172.21.15.45:0/140071 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f19c8108760 0x7f19c81a2460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:29.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:01:29.905+0000 7f19cd800700 1 --2- 172.21.15.45:0/140071 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f19c8103500 0x7f19c819aea0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:29.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:01:29.905+0000 7f19cd800700 1 --2- 172.21.15.45:0/140071 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f19ac0a3cc0 0x7f19ac0a6180 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:29.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:01:29.905+0000 7f19cd800700 1 --2- 172.21.15.45:0/140071 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f19c8105e30 0x7f19c819b3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:29.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:01:29.905+0000 7f19cd800700 1 -- 172.21.15.45:0/140071 >> 172.21.15.45:0/140071 conn(0x7f19c80fcf10 msgr2=0x7f19c8102c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:01:29.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:01:29.905+0000 7f19cd800700 1 -- 172.21.15.45:0/140071 shutdown_connections 2024-04-16T17:01:29.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:01:29.906+0000 7f19cd800700 1 -- 172.21.15.45:0/140071 wait complete. 2024-04-16T17:01:29.978 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:01:29 smithi045 ceph-mon[31676]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-04-16T17:01:29.978 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:01:29 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:01:29.978 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:01:29 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:01:29.978 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:01:29 smithi045 ceph-mon[31676]: from='osd.9 [v2:172.21.15.45:6826/3106734074,v1:172.21.15.45:6827/3106734074]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-16T17:01:30.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:01:29 smithi117 ceph-mon[36730]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-04-16T17:01:30.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:01:29 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:01:30.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:01:29 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:01:30.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:01:29 smithi117 ceph-mon[36730]: from='osd.9 [v2:172.21.15.45:6826/3106734074,v1:172.21.15.45:6827/3106734074]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-16T17:01:30.178 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:01:29 smithi104 ceph-mon[36457]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-04-16T17:01:30.178 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:01:29 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:01:30.178 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:01:29 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:01:30.179 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:01:29 smithi104 ceph-mon[36457]: from='osd.9 [v2:172.21.15.45:6826/3106734074,v1:172.21.15.45:6827/3106734074]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-16T17:01:30.659 DEBUG:teuthology.orchestra.run.smithi045:osd.9> sudo journalctl -f -n 0 -u ceph-b1fecd38-fc11-11ee-bc90-c7b262605968@osd.9.service 2024-04-16T17:01:30.664 INFO:tasks.cephadm:Deploying osd.10 on smithi104 with /dev/vg_nvme/lv_1... 2024-04-16T17:01:30.664 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b1fecd38-fc11-11ee-bc90-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-16T17:01:30.910 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/mon.b/config 2024-04-16T17:01:30.923 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:01:30 smithi045 ceph-mon[31676]: from='osd.9 [v2:172.21.15.45:6826/3106734074,v1:172.21.15.45:6827/3106734074]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-16T17:01:30.923 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:01:30 smithi045 ceph-mon[31676]: osdmap e66: 10 total, 9 up, 10 in 2024-04-16T17:01:30.923 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:01:30 smithi045 ceph-mon[31676]: from='osd.9 [v2:172.21.15.45:6826/3106734074,v1:172.21.15.45:6827/3106734074]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]: dispatch 2024-04-16T17:01:30.923 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:01:30 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-16T17:01:30.923 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:01:30 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:01:30.923 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:01:30 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:01:31.032 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:01:30 smithi104 ceph-mon[36457]: from='osd.9 [v2:172.21.15.45:6826/3106734074,v1:172.21.15.45:6827/3106734074]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-16T17:01:31.032 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:01:30 smithi104 ceph-mon[36457]: osdmap e66: 10 total, 9 up, 10 in 2024-04-16T17:01:31.032 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:01:30 smithi104 ceph-mon[36457]: from='osd.9 [v2:172.21.15.45:6826/3106734074,v1:172.21.15.45:6827/3106734074]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]: dispatch 2024-04-16T17:01:31.032 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:01:30 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-16T17:01:31.032 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:01:30 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:01:31.032 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:01:30 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:01:31.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:01:30 smithi117 ceph-mon[36730]: from='osd.9 [v2:172.21.15.45:6826/3106734074,v1:172.21.15.45:6827/3106734074]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-16T17:01:31.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:01:30 smithi117 ceph-mon[36730]: osdmap e66: 10 total, 9 up, 10 in 2024-04-16T17:01:31.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:01:30 smithi117 ceph-mon[36730]: from='osd.9 [v2:172.21.15.45:6826/3106734074,v1:172.21.15.45:6827/3106734074]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]: dispatch 2024-04-16T17:01:31.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:01:30 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-16T17:01:31.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:01:30 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:01:31.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:01:30 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:01:31.663 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 16 17:01:31 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-9[66545]: 2024-04-16T17:01:31.334+0000 7f5940400700 -1 osd.9 0 waiting for initial osdmap 2024-04-16T17:01:31.663 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 16 17:01:31 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-9[66545]: 2024-04-16T17:01:31.360+0000 7f5935800700 -1 osd.9 67 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-16T17:01:31.978 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:01:31 smithi045 ceph-mon[31676]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-04-16T17:01:31.978 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:01:31 smithi045 ceph-mon[31676]: from='osd.9 [v2:172.21.15.45:6826/3106734074,v1:172.21.15.45:6827/3106734074]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]': finished 2024-04-16T17:01:31.978 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:01:31 smithi045 ceph-mon[31676]: osdmap e67: 10 total, 9 up, 10 in 2024-04-16T17:01:31.978 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:01:31 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-16T17:01:31.978 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:01:31 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-16T17:01:31.979 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:01:31 smithi045 ceph-mon[31676]: from='osd.9 [v2:172.21.15.45:6826/3106734074,v1:172.21.15.45:6827/3106734074]' entity='osd.9' 2024-04-16T17:01:32.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:01:31 smithi117 ceph-mon[36730]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-04-16T17:01:32.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:01:31 smithi117 ceph-mon[36730]: from='osd.9 [v2:172.21.15.45:6826/3106734074,v1:172.21.15.45:6827/3106734074]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]': finished 2024-04-16T17:01:32.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:01:31 smithi117 ceph-mon[36730]: osdmap e67: 10 total, 9 up, 10 in 2024-04-16T17:01:32.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:01:31 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-16T17:01:32.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:01:31 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-16T17:01:32.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:01:31 smithi117 ceph-mon[36730]: from='osd.9 [v2:172.21.15.45:6826/3106734074,v1:172.21.15.45:6827/3106734074]' entity='osd.9' 2024-04-16T17:01:32.070 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:01:31 smithi104 ceph-mon[36457]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-04-16T17:01:32.070 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:01:31 smithi104 ceph-mon[36457]: from='osd.9 [v2:172.21.15.45:6826/3106734074,v1:172.21.15.45:6827/3106734074]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]': finished 2024-04-16T17:01:32.070 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:01:31 smithi104 ceph-mon[36457]: osdmap e67: 10 total, 9 up, 10 in 2024-04-16T17:01:32.070 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:01:31 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-16T17:01:32.071 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:01:31 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-16T17:01:32.071 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:01:31 smithi104 ceph-mon[36457]: from='osd.9 [v2:172.21.15.45:6826/3106734074,v1:172.21.15.45:6827/3106734074]' entity='osd.9' 2024-04-16T17:01:32.854 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:01:32 smithi045 ceph-mon[31676]: purged_snaps scrub starts 2024-04-16T17:01:32.855 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:01:32 smithi045 ceph-mon[31676]: purged_snaps scrub ok 2024-04-16T17:01:32.855 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:01:32 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-16T17:01:32.855 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:01:32 smithi045 ceph-mon[31676]: osd.9 [v2:172.21.15.45:6826/3106734074,v1:172.21.15.45:6827/3106734074] boot 2024-04-16T17:01:32.855 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:01:32 smithi045 ceph-mon[31676]: osdmap e68: 10 total, 10 up, 10 in 2024-04-16T17:01:32.855 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:01:32 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-16T17:01:32.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:01:32 smithi104 ceph-mon[36457]: purged_snaps scrub starts 2024-04-16T17:01:32.929 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:01:32 smithi104 ceph-mon[36457]: purged_snaps scrub ok 2024-04-16T17:01:32.929 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:01:32 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-16T17:01:32.929 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:01:32 smithi104 ceph-mon[36457]: osd.9 [v2:172.21.15.45:6826/3106734074,v1:172.21.15.45:6827/3106734074] boot 2024-04-16T17:01:32.929 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:01:32 smithi104 ceph-mon[36457]: osdmap e68: 10 total, 10 up, 10 in 2024-04-16T17:01:32.929 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:01:32 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-16T17:01:33.050 INFO:teuthology.orchestra.run.smithi104.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-16T17:01:33.050 INFO:teuthology.orchestra.run.smithi104.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-16T17:01:33.050 INFO:teuthology.orchestra.run.smithi104.stderr: stderr: 10+0 records in 2024-04-16T17:01:33.050 INFO:teuthology.orchestra.run.smithi104.stderr:10+0 records out 2024-04-16T17:01:33.050 INFO:teuthology.orchestra.run.smithi104.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0115303 s, 909 MB/s 2024-04-16T17:01:33.050 INFO:teuthology.orchestra.run.smithi104.stderr:--> Zapping successful for: 2024-04-16T17:01:33.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:01:32 smithi117 ceph-mon[36730]: purged_snaps scrub starts 2024-04-16T17:01:33.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:01:32 smithi117 ceph-mon[36730]: purged_snaps scrub ok 2024-04-16T17:01:33.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:01:32 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-16T17:01:33.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:01:32 smithi117 ceph-mon[36730]: osd.9 [v2:172.21.15.45:6826/3106734074,v1:172.21.15.45:6827/3106734074] boot 2024-04-16T17:01:33.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:01:32 smithi117 ceph-mon[36730]: osdmap e68: 10 total, 10 up, 10 in 2024-04-16T17:01:33.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:01:32 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-16T17:01:33.703 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b1fecd38-fc11-11ee-bc90-c7b262605968 -- ceph orch daemon add osd smithi104:vg_nvme/lv_1 2024-04-16T17:01:33.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:01:33 smithi104 ceph-mon[36457]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-04-16T17:01:33.944 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/mon.b/config 2024-04-16T17:01:33.978 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:01:33 smithi045 ceph-mon[31676]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-04-16T17:01:34.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:01:33 smithi117 ceph-mon[36730]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-04-16T17:01:35.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:01:34 smithi117 ceph-mon[36730]: osdmap e69: 10 total, 10 up, 10 in 2024-04-16T17:01:35.124 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:01:34 smithi045 ceph-mon[31676]: osdmap e69: 10 total, 10 up, 10 in 2024-04-16T17:01:35.163 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:01:34 smithi104 ceph-mon[36457]: osdmap e69: 10 total, 10 up, 10 in 2024-04-16T17:01:35.714 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-16T17:01:35.711+0000 7fef78b67700 1 -- 172.21.15.104:0/230881946 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fef74102c40 msgr2=0x7fef741030a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:01:35.715 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-16T17:01:35.711+0000 7fef78b67700 1 --2- 172.21.15.104:0/230881946 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fef74102c40 0x7fef741030a0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fef680099c0 tx=0x7fef68009cd0 comp rx=0 tx=0).stop 2024-04-16T17:01:35.715 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-16T17:01:35.711+0000 7fef78b67700 1 -- 172.21.15.104:0/230881946 shutdown_connections 2024-04-16T17:01:35.715 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-16T17:01:35.711+0000 7fef78b67700 1 --2- 172.21.15.104:0/230881946 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fef74101a40 0x7fef74101e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:35.715 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-16T17:01:35.711+0000 7fef78b67700 1 --2- 172.21.15.104:0/230881946 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef741035e0 0x7fef74109e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:35.715 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-16T17:01:35.711+0000 7fef78b67700 1 --2- 172.21.15.104:0/230881946 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fef74102c40 0x7fef741030a0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:35.715 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-16T17:01:35.711+0000 7fef78b67700 1 -- 172.21.15.104:0/230881946 >> 172.21.15.104:0/230881946 conn(0x7fef740fcfe0 msgr2=0x7fef740ff420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:01:35.715 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-16T17:01:35.711+0000 7fef78b67700 1 -- 172.21.15.104:0/230881946 shutdown_connections 2024-04-16T17:01:35.715 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-16T17:01:35.711+0000 7fef78b67700 1 -- 172.21.15.104:0/230881946 wait complete. 2024-04-16T17:01:35.715 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-16T17:01:35.712+0000 7fef78b67700 1 Processor -- start 2024-04-16T17:01:35.715 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-16T17:01:35.712+0000 7fef78b67700 1 -- start start 2024-04-16T17:01:35.715 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-16T17:01:35.712+0000 7fef78b67700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fef74101a40 0x7fef74198d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:01:35.715 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-16T17:01:35.712+0000 7fef78b67700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fef74102c40 0x7fef74199270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:01:35.715 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-16T17:01:35.712+0000 7fef78b67700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef741035e0 0x7fef741a02f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:01:35.716 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-16T17:01:35.712+0000 7fef78b67700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fef7410d680 con 0x7fef741035e0 2024-04-16T17:01:35.716 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-16T17:01:35.712+0000 7fef78b67700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fef7410d500 con 0x7fef74101a40 2024-04-16T17:01:35.716 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-16T17:01:35.712+0000 7fef78b67700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fef7410d800 con 0x7fef74102c40 2024-04-16T17:01:35.716 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-16T17:01:35.712+0000 7fef7259c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fef74101a40 0x7fef74198d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:01:35.716 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-16T17:01:35.712+0000 7fef7259c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fef74101a40 0x7fef74198d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:57846/0 (socket says 172.21.15.104:57846) 2024-04-16T17:01:35.716 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-16T17:01:35.712+0000 7fef7259c700 1 -- 172.21.15.104:0/1471317546 learned_addr learned my addr 172.21.15.104:0/1471317546 (peer_addr_for_me v2:172.21.15.104:0/0) 2024-04-16T17:01:35.716 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-16T17:01:35.712+0000 7fef71d9b700 1 --2- 172.21.15.104:0/1471317546 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fef74102c40 0x7fef74199270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:01:35.716 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-16T17:01:35.712+0000 7fef7259c700 1 -- 172.21.15.104:0/1471317546 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fef74102c40 msgr2=0x7fef74199270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:01:35.716 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-16T17:01:35.712+0000 7fef7259c700 1 --2- 172.21.15.104:0/1471317546 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fef74102c40 0x7fef74199270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:35.716 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-16T17:01:35.712+0000 7fef72d9d700 1 --2- 172.21.15.104:0/1471317546 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef741035e0 0x7fef741a02f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:01:35.716 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-16T17:01:35.712+0000 7fef7259c700 1 -- 172.21.15.104:0/1471317546 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef741035e0 msgr2=0x7fef741a02f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:01:35.716 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-16T17:01:35.712+0000 7fef7259c700 1 --2- 172.21.15.104:0/1471317546 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef741035e0 0x7fef741a02f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:35.716 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-16T17:01:35.712+0000 7fef7259c700 1 -- 172.21.15.104:0/1471317546 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef741a0990 con 0x7fef74101a40 2024-04-16T17:01:35.717 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-16T17:01:35.713+0000 7fef7259c700 1 --2- 172.21.15.104:0/1471317546 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fef74101a40 0x7fef74198d30 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fef6400c990 tx=0x7fef6400cd50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:01:35.717 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-16T17:01:35.713+0000 7fef637fe700 1 -- 172.21.15.104:0/1471317546 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef64007b00 con 0x7fef74101a40 2024-04-16T17:01:35.717 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-16T17:01:35.713+0000 7fef637fe700 1 -- 172.21.15.104:0/1471317546 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fef64007ca0 con 0x7fef74101a40 2024-04-16T17:01:35.717 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-16T17:01:35.713+0000 7fef78b67700 1 -- 172.21.15.104:0/1471317546 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef741a0c80 con 0x7fef74101a40 2024-04-16T17:01:35.717 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-16T17:01:35.713+0000 7fef637fe700 1 -- 172.21.15.104:0/1471317546 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef64018780 con 0x7fef74101a40 2024-04-16T17:01:35.718 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-16T17:01:35.713+0000 7fef78b67700 1 -- 172.21.15.104:0/1471317546 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef741a11c0 con 0x7fef74101a40 2024-04-16T17:01:35.718 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-16T17:01:35.714+0000 7fef78b67700 1 -- 172.21.15.104:0/1471317546 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef74069810 con 0x7fef74101a40 2024-04-16T17:01:35.718 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-16T17:01:35.715+0000 7fef637fe700 1 -- 172.21.15.104:0/1471317546 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fef6400f9c0 con 0x7fef74101a40 2024-04-16T17:01:35.718 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-16T17:01:35.715+0000 7fef637fe700 1 --2- 172.21.15.104:0/1471317546 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fef5c0a3c40 0x7fef5c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:01:35.718 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-16T17:01:35.716+0000 7fef637fe700 1 -- 172.21.15.104:0/1471317546 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 6261+0+0 (secure 0 0 0) 0x7fef640cfcf0 con 0x7fef74101a40 2024-04-16T17:01:35.718 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-16T17:01:35.716+0000 7fef71d9b700 1 --2- 172.21.15.104:0/1471317546 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fef5c0a3c40 0x7fef5c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:01:35.718 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-16T17:01:35.717+0000 7fef71d9b700 1 --2- 172.21.15.104:0/1471317546 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fef5c0a3c40 0x7fef5c0a6100 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fef68000c00 tx=0x7fef68005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:01:35.720 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-16T17:01:35.719+0000 7fef637fe700 1 -- 172.21.15.104:0/1471317546 <== mon.2 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fef74069810 con 0x7fef74101a40 2024-04-16T17:01:35.926 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-16T17:01:35.924+0000 7fef78b67700 1 -- 172.21.15.104:0/1471317546 --> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi104:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fef740ff120 con 0x7fef5c0a3c40 2024-04-16T17:01:35.978 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:01:35 smithi045 ceph-mon[31676]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-04-16T17:01:36.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:01:35 smithi117 ceph-mon[36730]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-04-16T17:01:36.178 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:01:35 smithi104 ceph-mon[36457]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-04-16T17:01:36.845 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:01:36 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-16T17:01:36.845 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:01:36 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-16T17:01:36.845 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:01:36 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:01:36.845 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:01:36 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:01:36.846 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:01:36 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:01:36.846 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:01:36 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:01:36.846 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:01:36 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:01:36.846 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:01:36 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:01:36.846 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:01:36 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:01:36.846 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:01:36 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:01:36.846 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:01:36 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:01:36.846 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:01:36 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T17:01:36.846 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:01:36 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:01:36.846 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:01:36 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T17:01:36.846 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:01:36 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:01:36.846 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:01:36 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T17:01:36.847 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:01:36 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:01:37.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:01:36 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-16T17:01:37.057 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:01:36 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-16T17:01:37.057 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:01:36 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:01:37.057 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:01:36 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:01:37.057 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:01:36 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:01:37.057 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:01:36 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:01:37.057 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:01:36 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:01:37.057 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:01:36 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:01:37.057 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:01:36 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:01:37.057 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:01:36 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:01:37.057 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:01:36 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:01:37.057 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:01:36 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T17:01:37.057 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:01:36 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:01:37.057 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:01:36 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T17:01:37.057 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:01:36 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:01:37.058 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:01:36 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T17:01:37.058 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:01:36 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:01:37.229 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:01:36 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-16T17:01:37.229 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:01:36 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-16T17:01:37.229 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:01:36 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:01:37.229 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:01:36 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:01:37.229 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:01:36 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:01:37.229 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:01:36 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:01:37.229 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:01:36 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:01:37.229 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:01:36 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:01:37.229 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:01:36 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:01:37.229 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:01:36 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:01:37.229 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:01:36 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:01:37.229 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:01:36 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T17:01:37.230 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:01:36 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:01:37.230 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:01:36 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T17:01:37.230 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:01:36 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:01:37.230 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:01:36 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T17:01:37.230 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:01:36 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:01:37.889 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:01:37 smithi104 ceph-mon[36457]: from='client.24347 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi104:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T17:01:37.889 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:01:37 smithi104 ceph-mon[36457]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-04-16T17:01:37.890 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:01:37 smithi104 ceph-mon[36457]: Detected new or changed devices on smithi045 2024-04-16T17:01:37.890 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:01:37 smithi104 ceph-mon[36457]: Adjusting osd_memory_target on smithi045 to 4273M 2024-04-16T17:01:38.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:01:37 smithi117 ceph-mon[36730]: from='client.24347 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi104:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T17:01:38.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:01:37 smithi117 ceph-mon[36730]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-04-16T17:01:38.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:01:37 smithi117 ceph-mon[36730]: Detected new or changed devices on smithi045 2024-04-16T17:01:38.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:01:37 smithi117 ceph-mon[36730]: Adjusting osd_memory_target on smithi045 to 4273M 2024-04-16T17:01:38.134 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:01:37 smithi045 ceph-mon[31676]: from='client.24347 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi104:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T17:01:38.134 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:01:37 smithi045 ceph-mon[31676]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-04-16T17:01:38.134 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:01:37 smithi045 ceph-mon[31676]: Detected new or changed devices on smithi045 2024-04-16T17:01:38.134 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:01:37 smithi045 ceph-mon[31676]: Adjusting osd_memory_target on smithi045 to 4273M 2024-04-16T17:01:40.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:01:39 smithi117 ceph-mon[36730]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-04-16T17:01:40.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:01:39 smithi117 ceph-mon[36730]: from='client.? 172.21.15.104:0/2318820414' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2deba364-3e39-47be-8a1f-8256df5b82d6"}]: dispatch 2024-04-16T17:01:40.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:01:39 smithi117 ceph-mon[36730]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2deba364-3e39-47be-8a1f-8256df5b82d6"}]: dispatch 2024-04-16T17:01:40.057 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:01:39 smithi117 ceph-mon[36730]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2deba364-3e39-47be-8a1f-8256df5b82d6"}]': finished 2024-04-16T17:01:40.057 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:01:39 smithi117 ceph-mon[36730]: osdmap e70: 11 total, 10 up, 11 in 2024-04-16T17:01:40.057 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:01:39 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-16T17:01:40.057 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:01:39 smithi117 ceph-mon[36730]: from='client.? 172.21.15.104:0/3779584893' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-16T17:01:40.179 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:01:39 smithi104 ceph-mon[36457]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-04-16T17:01:40.179 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:01:39 smithi104 ceph-mon[36457]: from='client.? 172.21.15.104:0/2318820414' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2deba364-3e39-47be-8a1f-8256df5b82d6"}]: dispatch 2024-04-16T17:01:40.179 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:01:39 smithi104 ceph-mon[36457]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2deba364-3e39-47be-8a1f-8256df5b82d6"}]: dispatch 2024-04-16T17:01:40.179 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:01:39 smithi104 ceph-mon[36457]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2deba364-3e39-47be-8a1f-8256df5b82d6"}]': finished 2024-04-16T17:01:40.179 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:01:39 smithi104 ceph-mon[36457]: osdmap e70: 11 total, 10 up, 11 in 2024-04-16T17:01:40.179 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:01:39 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-16T17:01:40.179 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:01:39 smithi104 ceph-mon[36457]: from='client.? 172.21.15.104:0/3779584893' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-16T17:01:40.228 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:01:39 smithi045 ceph-mon[31676]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-04-16T17:01:40.228 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:01:39 smithi045 ceph-mon[31676]: from='client.? 172.21.15.104:0/2318820414' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2deba364-3e39-47be-8a1f-8256df5b82d6"}]: dispatch 2024-04-16T17:01:40.229 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:01:39 smithi045 ceph-mon[31676]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2deba364-3e39-47be-8a1f-8256df5b82d6"}]: dispatch 2024-04-16T17:01:40.229 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:01:39 smithi045 ceph-mon[31676]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2deba364-3e39-47be-8a1f-8256df5b82d6"}]': finished 2024-04-16T17:01:40.229 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:01:39 smithi045 ceph-mon[31676]: osdmap e70: 11 total, 10 up, 11 in 2024-04-16T17:01:40.229 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:01:39 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-16T17:01:40.229 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:01:39 smithi045 ceph-mon[31676]: from='client.? 172.21.15.104:0/3779584893' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-16T17:01:42.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:01:41 smithi117 ceph-mon[36730]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-04-16T17:01:42.178 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:01:41 smithi104 ceph-mon[36457]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-04-16T17:01:42.228 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:01:41 smithi045 ceph-mon[31676]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-04-16T17:01:44.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:01:43 smithi117 ceph-mon[36730]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-04-16T17:01:44.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:01:43 smithi104 ceph-mon[36457]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-04-16T17:01:44.228 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:01:43 smithi045 ceph-mon[31676]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-04-16T17:01:45.883 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:01:45 smithi104 ceph-mon[36457]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-04-16T17:01:46.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:01:45 smithi117 ceph-mon[36730]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-04-16T17:01:46.228 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:01:45 smithi045 ceph-mon[31676]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-04-16T17:01:47.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:01:46 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-16T17:01:47.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:01:46 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:01:47.070 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:01:46 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-16T17:01:47.070 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:01:46 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:01:47.228 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:01:46 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-16T17:01:47.228 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:01:46 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:01:48.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:01:47 smithi117 ceph-mon[36730]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-04-16T17:01:48.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:01:47 smithi117 ceph-mon[36730]: Deploying daemon osd.10 on smithi104 2024-04-16T17:01:48.178 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:01:47 smithi104 ceph-mon[36457]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-04-16T17:01:48.178 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:01:47 smithi104 ceph-mon[36457]: Deploying daemon osd.10 on smithi104 2024-04-16T17:01:48.228 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:01:47 smithi045 ceph-mon[31676]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-04-16T17:01:48.228 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:01:47 smithi045 ceph-mon[31676]: Deploying daemon osd.10 on smithi104 2024-04-16T17:01:50.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:01:49 smithi117 ceph-mon[36730]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-04-16T17:01:50.123 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:01:49 smithi104 ceph-mon[36457]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-04-16T17:01:50.228 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:01:49 smithi045 ceph-mon[31676]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-04-16T17:01:52.046 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:01:51 smithi104 ceph-mon[36457]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-04-16T17:01:52.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:01:51 smithi117 ceph-mon[36730]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-04-16T17:01:52.228 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:01:51 smithi045 ceph-mon[31676]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-04-16T17:01:53.429 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:01:53 smithi104 ceph-mon[36457]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-04-16T17:01:53.429 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:01:53 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:01:53.429 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:01:53 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T17:01:53.429 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:01:53 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:01:53.478 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:01:53 smithi045 ceph-mon[31676]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-04-16T17:01:53.478 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:01:53 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:01:53.478 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:01:53 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T17:01:53.478 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:01:53 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:01:53.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:01:53 smithi117 ceph-mon[36730]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-04-16T17:01:53.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:01:53 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:01:53.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:01:53 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T17:01:53.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:01:53 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:01:55.398 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:01:55 smithi104 ceph-mon[36457]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-04-16T17:01:55.478 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:01:55 smithi045 ceph-mon[31676]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-04-16T17:01:55.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:01:55 smithi117 ceph-mon[36730]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-04-16T17:01:56.171 INFO:teuthology.orchestra.run.smithi104.stdout:Created osd(s) 10 on host 'smithi104' 2024-04-16T17:01:56.171 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-16T17:01:56.168+0000 7fef637fe700 1 -- 172.21.15.104:0/1471317546 <== mgr.14152 v2:172.21.15.45:6800/268030699 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7fef740ff120 con 0x7fef5c0a3c40 2024-04-16T17:01:56.172 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-16T17:01:56.171+0000 7fef78b67700 1 -- 172.21.15.104:0/1471317546 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fef5c0a3c40 msgr2=0x7fef5c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:01:56.173 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-16T17:01:56.171+0000 7fef78b67700 1 --2- 172.21.15.104:0/1471317546 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fef5c0a3c40 0x7fef5c0a6100 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fef68000c00 tx=0x7fef68005fd0 comp rx=0 tx=0).stop 2024-04-16T17:01:56.173 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-16T17:01:56.171+0000 7fef78b67700 1 -- 172.21.15.104:0/1471317546 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fef74101a40 msgr2=0x7fef74198d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:01:56.173 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-16T17:01:56.171+0000 7fef78b67700 1 --2- 172.21.15.104:0/1471317546 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fef74101a40 0x7fef74198d30 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fef6400c990 tx=0x7fef6400cd50 comp rx=0 tx=0).stop 2024-04-16T17:01:56.173 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-16T17:01:56.171+0000 7fef78b67700 1 -- 172.21.15.104:0/1471317546 shutdown_connections 2024-04-16T17:01:56.173 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-16T17:01:56.171+0000 7fef78b67700 1 --2- 172.21.15.104:0/1471317546 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef741035e0 0x7fef741a02f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:56.173 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-16T17:01:56.171+0000 7fef78b67700 1 --2- 172.21.15.104:0/1471317546 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fef74101a40 0x7fef74198d30 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:56.173 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-16T17:01:56.171+0000 7fef78b67700 1 --2- 172.21.15.104:0/1471317546 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fef5c0a3c40 0x7fef5c0a6100 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:56.173 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-16T17:01:56.171+0000 7fef78b67700 1 --2- 172.21.15.104:0/1471317546 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fef74102c40 0x7fef74199270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:01:56.173 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-16T17:01:56.171+0000 7fef78b67700 1 -- 172.21.15.104:0/1471317546 >> 172.21.15.104:0/1471317546 conn(0x7fef740fcfe0 msgr2=0x7fef740fea10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:01:56.174 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-16T17:01:56.171+0000 7fef78b67700 1 -- 172.21.15.104:0/1471317546 shutdown_connections 2024-04-16T17:01:56.174 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-16T17:01:56.171+0000 7fef78b67700 1 -- 172.21.15.104:0/1471317546 wait complete. 2024-04-16T17:01:56.429 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:01:56 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:01:56.429 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:01:56 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:01:56.429 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:01:56 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:01:56.429 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:01:56 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T17:01:56.429 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:01:56 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:01:56.429 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:01:56 smithi104 ceph-mon[36457]: from='osd.10 [v2:172.21.15.104:6824/1404794652,v1:172.21.15.104:6825/1404794652]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-16T17:01:56.429 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:01:56 smithi104 ceph-mon[36457]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-16T17:01:56.429 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:01:56 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:01:56.429 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:01:56 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:01:56.429 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:01:56 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T17:01:56.479 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:01:56 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:01:56.479 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:01:56 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:01:56.479 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:01:56 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:01:56.479 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:01:56 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T17:01:56.479 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:01:56 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:01:56.479 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:01:56 smithi045 ceph-mon[31676]: from='osd.10 [v2:172.21.15.104:6824/1404794652,v1:172.21.15.104:6825/1404794652]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-16T17:01:56.479 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:01:56 smithi045 ceph-mon[31676]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-16T17:01:56.479 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:01:56 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:01:56.479 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:01:56 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:01:56.480 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:01:56 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T17:01:56.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:01:56 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:01:56.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:01:56 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:01:56.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:01:56 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:01:56.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:01:56 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T17:01:56.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:01:56 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:01:56.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:01:56 smithi117 ceph-mon[36730]: from='osd.10 [v2:172.21.15.104:6824/1404794652,v1:172.21.15.104:6825/1404794652]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-16T17:01:56.557 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:01:56 smithi117 ceph-mon[36730]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-16T17:01:56.557 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:01:56 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:01:56.557 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:01:56 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:01:56.557 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:01:56 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T17:01:56.819 DEBUG:teuthology.orchestra.run.smithi104:osd.10> sudo journalctl -f -n 0 -u ceph-b1fecd38-fc11-11ee-bc90-c7b262605968@osd.10.service 2024-04-16T17:01:56.822 INFO:tasks.cephadm:Deploying osd.11 on smithi117 with /dev/vg_nvme/lv_1... 2024-04-16T17:01:56.823 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b1fecd38-fc11-11ee-bc90-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-16T17:01:57.066 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/mon.c/config 2024-04-16T17:01:57.428 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:01:57 smithi104 ceph-mon[36457]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-04-16T17:01:57.428 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:01:57 smithi104 ceph-mon[36457]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-16T17:01:57.429 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:01:57 smithi104 ceph-mon[36457]: from='osd.10 [v2:172.21.15.104:6824/1404794652,v1:172.21.15.104:6825/1404794652]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2024-04-16T17:01:57.429 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:01:57 smithi104 ceph-mon[36457]: osdmap e71: 11 total, 10 up, 11 in 2024-04-16T17:01:57.429 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:01:57 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-16T17:01:57.429 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:01:57 smithi104 ceph-mon[36457]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2024-04-16T17:01:57.535 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:01:57 smithi117 ceph-mon[36730]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-04-16T17:01:57.535 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:01:57 smithi117 ceph-mon[36730]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-16T17:01:57.535 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:01:57 smithi117 ceph-mon[36730]: from='osd.10 [v2:172.21.15.104:6824/1404794652,v1:172.21.15.104:6825/1404794652]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2024-04-16T17:01:57.535 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:01:57 smithi117 ceph-mon[36730]: osdmap e71: 11 total, 10 up, 11 in 2024-04-16T17:01:57.535 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:01:57 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-16T17:01:57.535 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:01:57 smithi117 ceph-mon[36730]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2024-04-16T17:01:57.729 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:01:57 smithi045 ceph-mon[31676]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-04-16T17:01:57.729 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:01:57 smithi045 ceph-mon[31676]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-16T17:01:57.729 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:01:57 smithi045 ceph-mon[31676]: from='osd.10 [v2:172.21.15.104:6824/1404794652,v1:172.21.15.104:6825/1404794652]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2024-04-16T17:01:57.729 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:01:57 smithi045 ceph-mon[31676]: osdmap e71: 11 total, 10 up, 11 in 2024-04-16T17:01:57.729 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:01:57 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-16T17:01:57.729 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:01:57 smithi045 ceph-mon[31676]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2024-04-16T17:01:58.003 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:01:57 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:01:57.694+0000 7fc224f0f700 -1 osd.10 0 waiting for initial osdmap 2024-04-16T17:01:58.003 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:01:57 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:01:57.706+0000 7fc21c4f9700 -1 osd.10 72 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-16T17:01:58.479 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:01:58 smithi117 ceph-mon[36730]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]': finished 2024-04-16T17:01:58.479 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:01:58 smithi117 ceph-mon[36730]: osdmap e72: 11 total, 10 up, 11 in 2024-04-16T17:01:58.479 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:01:58 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-16T17:01:58.479 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:01:58 smithi117 ceph-mon[36730]: from='osd.10 ' entity='osd.10' 2024-04-16T17:01:58.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:01:58 smithi104 ceph-mon[36457]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]': finished 2024-04-16T17:01:58.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:01:58 smithi104 ceph-mon[36457]: osdmap e72: 11 total, 10 up, 11 in 2024-04-16T17:01:58.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:01:58 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-16T17:01:58.679 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:01:58 smithi104 ceph-mon[36457]: from='osd.10 ' entity='osd.10' 2024-04-16T17:01:58.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:01:58 smithi045 ceph-mon[31676]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]': finished 2024-04-16T17:01:58.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:01:58 smithi045 ceph-mon[31676]: osdmap e72: 11 total, 10 up, 11 in 2024-04-16T17:01:58.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:01:58 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-16T17:01:58.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:01:58 smithi045 ceph-mon[31676]: from='osd.10 ' entity='osd.10' 2024-04-16T17:01:59.198 INFO:teuthology.orchestra.run.smithi117.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-16T17:01:59.198 INFO:teuthology.orchestra.run.smithi117.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-16T17:01:59.198 INFO:teuthology.orchestra.run.smithi117.stderr: stderr: 10+0 records in 2024-04-16T17:01:59.198 INFO:teuthology.orchestra.run.smithi117.stderr:10+0 records out 2024-04-16T17:01:59.198 INFO:teuthology.orchestra.run.smithi117.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0176263 s, 595 MB/s 2024-04-16T17:01:59.198 INFO:teuthology.orchestra.run.smithi117.stderr:--> Zapping successful for: 2024-04-16T17:01:59.428 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:01:59 smithi104 ceph-mon[36457]: purged_snaps scrub starts 2024-04-16T17:01:59.428 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:01:59 smithi104 ceph-mon[36457]: purged_snaps scrub ok 2024-04-16T17:01:59.428 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:01:59 smithi104 ceph-mon[36457]: pgmap v261: 1 pgs: 1 peering; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-04-16T17:01:59.428 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:01:59 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-16T17:01:59.429 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:01:59 smithi104 ceph-mon[36457]: osd.10 [v2:172.21.15.104:6824/1404794652,v1:172.21.15.104:6825/1404794652] boot 2024-04-16T17:01:59.429 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:01:59 smithi104 ceph-mon[36457]: osdmap e73: 11 total, 11 up, 11 in 2024-04-16T17:01:59.429 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:01:59 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-16T17:01:59.429 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:01:59 smithi104 ceph-mon[36457]: osdmap e74: 11 total, 11 up, 11 in 2024-04-16T17:01:59.505 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:01:59 smithi117 ceph-mon[36730]: purged_snaps scrub starts 2024-04-16T17:01:59.505 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:01:59 smithi117 ceph-mon[36730]: purged_snaps scrub ok 2024-04-16T17:01:59.505 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:01:59 smithi117 ceph-mon[36730]: pgmap v261: 1 pgs: 1 peering; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-04-16T17:01:59.505 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:01:59 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-16T17:01:59.505 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:01:59 smithi117 ceph-mon[36730]: osd.10 [v2:172.21.15.104:6824/1404794652,v1:172.21.15.104:6825/1404794652] boot 2024-04-16T17:01:59.505 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:01:59 smithi117 ceph-mon[36730]: osdmap e73: 11 total, 11 up, 11 in 2024-04-16T17:01:59.505 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:01:59 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-16T17:01:59.505 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:01:59 smithi117 ceph-mon[36730]: osdmap e74: 11 total, 11 up, 11 in 2024-04-16T17:01:59.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:01:59 smithi045 ceph-mon[31676]: purged_snaps scrub starts 2024-04-16T17:01:59.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:01:59 smithi045 ceph-mon[31676]: purged_snaps scrub ok 2024-04-16T17:01:59.729 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:01:59 smithi045 ceph-mon[31676]: pgmap v261: 1 pgs: 1 peering; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-04-16T17:01:59.729 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:01:59 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-16T17:01:59.729 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:01:59 smithi045 ceph-mon[31676]: osd.10 [v2:172.21.15.104:6824/1404794652,v1:172.21.15.104:6825/1404794652] boot 2024-04-16T17:01:59.729 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:01:59 smithi045 ceph-mon[31676]: osdmap e73: 11 total, 11 up, 11 in 2024-04-16T17:01:59.729 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:01:59 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-16T17:01:59.729 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:01:59 smithi045 ceph-mon[31676]: osdmap e74: 11 total, 11 up, 11 in 2024-04-16T17:01:59.883 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b1fecd38-fc11-11ee-bc90-c7b262605968 -- ceph orch daemon add osd smithi117:vg_nvme/lv_1 2024-04-16T17:02:00.130 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/mon.c/config 2024-04-16T17:02:01.298 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:02:01 smithi117 ceph-mon[36730]: pgmap v264: 1 pgs: 1 peering; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-04-16T17:02:01.298 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:02:01 smithi117 ceph-mon[36730]: osdmap e75: 11 total, 11 up, 11 in 2024-04-16T17:02:01.298 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:02:01 smithi117 ceph-mon[36730]: Detected new or changed devices on smithi104 2024-04-16T17:02:01.298 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:02:01 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:02:01.298 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:02:01 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:02:01.298 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:02:01 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:02:01.298 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:02:01 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:02:01.298 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:02:01 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:02:01.298 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:02:01 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:02:01.299 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:02:01 smithi117 ceph-mon[36730]: Adjusting osd_memory_target on smithi104 to 4270M 2024-04-16T17:02:01.299 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:02:01 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:02:01.299 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:02:01 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:02:01.299 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:02:01 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T17:02:01.299 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:02:01 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:02:01.428 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:02:01 smithi104 ceph-mon[36457]: pgmap v264: 1 pgs: 1 peering; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-04-16T17:02:01.428 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:02:01 smithi104 ceph-mon[36457]: osdmap e75: 11 total, 11 up, 11 in 2024-04-16T17:02:01.428 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:02:01 smithi104 ceph-mon[36457]: Detected new or changed devices on smithi104 2024-04-16T17:02:01.428 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:02:01 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:02:01.428 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:02:01 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:02:01.428 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:02:01 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:02:01.429 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:02:01 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:02:01.429 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:02:01 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:02:01.429 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:02:01 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:02:01.429 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:02:01 smithi104 ceph-mon[36457]: Adjusting osd_memory_target on smithi104 to 4270M 2024-04-16T17:02:01.429 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:02:01 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:02:01.429 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:02:01 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:02:01.429 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:02:01 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T17:02:01.429 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:02:01 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:02:01.479 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:02:01 smithi045 ceph-mon[31676]: pgmap v264: 1 pgs: 1 peering; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-04-16T17:02:01.479 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:02:01 smithi045 ceph-mon[31676]: osdmap e75: 11 total, 11 up, 11 in 2024-04-16T17:02:01.479 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:02:01 smithi045 ceph-mon[31676]: Detected new or changed devices on smithi104 2024-04-16T17:02:01.479 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:02:01 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:02:01.479 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:02:01 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:02:01.479 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:02:01 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:02:01.479 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:02:01 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:02:01.479 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:02:01 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:02:01.479 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:02:01 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:02:01.480 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:02:01 smithi045 ceph-mon[31676]: Adjusting osd_memory_target on smithi104 to 4270M 2024-04-16T17:02:01.480 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:02:01 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:02:01.480 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:02:01 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:02:01.480 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:02:01 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T17:02:01.480 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:02:01 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:02:01.901 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T17:02:01.893+0000 7f1e33f16700 1 -- 172.21.15.117:0/1010304791 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1e2c104780 msgr2=0x7f1e2c10b030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:02:01.901 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T17:02:01.893+0000 7f1e33f16700 1 --2- 172.21.15.117:0/1010304791 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1e2c104780 0x7f1e2c10b030 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f1e2400b0a0 tx=0x7f1e2400b3b0 comp rx=0 tx=0).stop 2024-04-16T17:02:01.901 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T17:02:01.894+0000 7f1e33f16700 1 -- 172.21.15.117:0/1010304791 shutdown_connections 2024-04-16T17:02:01.901 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T17:02:01.894+0000 7f1e33f16700 1 --2- 172.21.15.117:0/1010304791 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1e2c104780 0x7f1e2c10b030 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:01.901 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T17:02:01.894+0000 7f1e33f16700 1 --2- 172.21.15.117:0/1010304791 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1e2c100e00 0x7f1e2c101220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:01.902 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T17:02:01.894+0000 7f1e33f16700 1 --2- 172.21.15.117:0/1010304791 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1e2c103de0 0x7f1e2c104240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:01.902 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T17:02:01.894+0000 7f1e33f16700 1 -- 172.21.15.117:0/1010304791 >> 172.21.15.117:0/1010304791 conn(0x7f1e2c0fc780 msgr2=0x7f1e2c0feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:02:01.902 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T17:02:01.894+0000 7f1e33f16700 1 -- 172.21.15.117:0/1010304791 shutdown_connections 2024-04-16T17:02:01.902 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T17:02:01.894+0000 7f1e33f16700 1 -- 172.21.15.117:0/1010304791 wait complete. 2024-04-16T17:02:01.902 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T17:02:01.894+0000 7f1e33f16700 1 Processor -- start 2024-04-16T17:02:01.902 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T17:02:01.894+0000 7f1e33f16700 1 -- start start 2024-04-16T17:02:01.902 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T17:02:01.895+0000 7f1e33f16700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1e2c100e00 0x7f1e2c198d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:02:01.902 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T17:02:01.895+0000 7f1e33f16700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1e2c103de0 0x7f1e2c199290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:02:01.902 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T17:02:01.895+0000 7f1e33f16700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1e2c104780 0x7f1e2c1a0310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:02:01.902 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T17:02:01.895+0000 7f1e33f16700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f1e2c10e770 con 0x7f1e2c104780 2024-04-16T17:02:01.902 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T17:02:01.895+0000 7f1e33f16700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f1e2c10e5f0 con 0x7f1e2c103de0 2024-04-16T17:02:01.902 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T17:02:01.895+0000 7f1e33f16700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f1e2c10e8f0 con 0x7f1e2c100e00 2024-04-16T17:02:01.902 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T17:02:01.895+0000 7f1e31cb2700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1e2c100e00 0x7f1e2c198d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:02:01.902 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T17:02:01.895+0000 7f1e31cb2700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1e2c100e00 0x7f1e2c198d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:39666/0 (socket says 172.21.15.117:39666) 2024-04-16T17:02:01.903 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T17:02:01.895+0000 7f1e31cb2700 1 -- 172.21.15.117:0/1829494456 learned_addr learned my addr 172.21.15.117:0/1829494456 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-16T17:02:01.903 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T17:02:01.895+0000 7f1e314b1700 1 --2- 172.21.15.117:0/1829494456 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1e2c103de0 0x7f1e2c199290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:02:01.903 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T17:02:01.895+0000 7f1e31cb2700 1 -- 172.21.15.117:0/1829494456 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1e2c103de0 msgr2=0x7f1e2c199290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:02:01.903 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T17:02:01.895+0000 7f1e31cb2700 1 --2- 172.21.15.117:0/1829494456 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1e2c103de0 0x7f1e2c199290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:01.903 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T17:02:01.895+0000 7f1e31cb2700 1 -- 172.21.15.117:0/1829494456 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1e2c104780 msgr2=0x7f1e2c1a0310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:02:01.903 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T17:02:01.895+0000 7f1e31cb2700 1 --2- 172.21.15.117:0/1829494456 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1e2c104780 0x7f1e2c1a0310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:01.903 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T17:02:01.895+0000 7f1e31cb2700 1 -- 172.21.15.117:0/1829494456 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e2c1a09b0 con 0x7f1e2c100e00 2024-04-16T17:02:01.903 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T17:02:01.895+0000 7f1e31cb2700 1 --2- 172.21.15.117:0/1829494456 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1e2c100e00 0x7f1e2c198d50 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f1e2800d9b0 tx=0x7f1e2800dcc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:02:01.903 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T17:02:01.895+0000 7f1e22ffd700 1 -- 172.21.15.117:0/1829494456 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e28017030 con 0x7f1e2c100e00 2024-04-16T17:02:01.903 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T17:02:01.895+0000 7f1e33f16700 1 -- 172.21.15.117:0/1829494456 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e2c1a0ca0 con 0x7f1e2c100e00 2024-04-16T17:02:01.903 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T17:02:01.895+0000 7f1e22ffd700 1 -- 172.21.15.117:0/1829494456 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1e28004500 con 0x7f1e2c100e00 2024-04-16T17:02:01.903 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T17:02:01.895+0000 7f1e22ffd700 1 -- 172.21.15.117:0/1829494456 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e28003aa0 con 0x7f1e2c100e00 2024-04-16T17:02:01.903 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T17:02:01.895+0000 7f1e33f16700 1 -- 172.21.15.117:0/1829494456 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e2c1a1180 con 0x7f1e2c100e00 2024-04-16T17:02:01.903 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T17:02:01.896+0000 7f1e33f16700 1 -- 172.21.15.117:0/1829494456 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e2c02dfe0 con 0x7f1e2c100e00 2024-04-16T17:02:01.903 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T17:02:01.897+0000 7f1e22ffd700 1 -- 172.21.15.117:0/1829494456 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f1e2801a070 con 0x7f1e2c100e00 2024-04-16T17:02:01.904 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T17:02:01.898+0000 7f1e22ffd700 1 --2- 172.21.15.117:0/1829494456 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f1e180a3c40 0x7f1e180a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:02:01.904 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T17:02:01.898+0000 7f1e22ffd700 1 -- 172.21.15.117:0/1829494456 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 6694+0+0 (secure 0 0 0) 0x7f1e280cf730 con 0x7f1e2c100e00 2024-04-16T17:02:01.904 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T17:02:01.898+0000 7f1e314b1700 1 --2- 172.21.15.117:0/1829494456 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f1e180a3c40 0x7f1e180a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:02:01.904 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T17:02:01.899+0000 7f1e314b1700 1 --2- 172.21.15.117:0/1829494456 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f1e180a3c40 0x7f1e180a6100 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f1e1c009a10 tx=0x7f1e1c009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:02:01.904 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T17:02:01.903+0000 7f1e22ffd700 1 -- 172.21.15.117:0/1829494456 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1e280232f0 con 0x7f1e2c100e00 2024-04-16T17:02:02.110 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T17:02:02.109+0000 7f1e33f16700 1 -- 172.21.15.117:0/1829494456 --> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi117:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f1e2c062f30 con 0x7f1e180a3c40 2024-04-16T17:02:02.306 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:02:02 smithi117 ceph-mon[36730]: osdmap e76: 11 total, 11 up, 11 in 2024-04-16T17:02:02.428 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:02:02 smithi104 ceph-mon[36457]: osdmap e76: 11 total, 11 up, 11 in 2024-04-16T17:02:02.478 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:02:02 smithi045 ceph-mon[31676]: osdmap e76: 11 total, 11 up, 11 in 2024-04-16T17:02:03.306 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:02:03 smithi117 ceph-mon[36730]: pgmap v267: 1 pgs: 1 peering; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-04-16T17:02:03.306 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:02:03 smithi117 ceph-mon[36730]: osdmap e77: 11 total, 11 up, 11 in 2024-04-16T17:02:03.306 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:02:03 smithi117 ceph-mon[36730]: from='client.24361 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi117:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T17:02:03.306 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:02:03 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-16T17:02:03.306 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:02:03 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-16T17:02:03.306 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:02:03 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:02:03.428 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:02:03 smithi104 ceph-mon[36457]: pgmap v267: 1 pgs: 1 peering; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-04-16T17:02:03.428 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:02:03 smithi104 ceph-mon[36457]: osdmap e77: 11 total, 11 up, 11 in 2024-04-16T17:02:03.429 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:02:03 smithi104 ceph-mon[36457]: from='client.24361 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi117:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T17:02:03.429 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:02:03 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-16T17:02:03.429 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:02:03 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-16T17:02:03.429 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:02:03 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:02:03.478 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:02:03 smithi045 ceph-mon[31676]: pgmap v267: 1 pgs: 1 peering; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-04-16T17:02:03.478 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:02:03 smithi045 ceph-mon[31676]: osdmap e77: 11 total, 11 up, 11 in 2024-04-16T17:02:03.478 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:02:03 smithi045 ceph-mon[31676]: from='client.24361 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi117:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T17:02:03.478 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:02:03 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-16T17:02:03.478 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:02:03 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-16T17:02:03.479 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:02:03 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:02:05.428 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:02:05 smithi104 ceph-mon[36457]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail; 115 KiB/s, 0 objects/s recovering 2024-04-16T17:02:05.429 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:02:05 smithi104 ceph-mon[36457]: from='client.? 172.21.15.117:0/969485587' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3fda4fea-59ad-4d1a-9fe6-fcb30e31fd7d"}]: dispatch 2024-04-16T17:02:05.429 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:02:05 smithi104 ceph-mon[36457]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3fda4fea-59ad-4d1a-9fe6-fcb30e31fd7d"}]: dispatch 2024-04-16T17:02:05.429 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:02:05 smithi104 ceph-mon[36457]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3fda4fea-59ad-4d1a-9fe6-fcb30e31fd7d"}]': finished 2024-04-16T17:02:05.429 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:02:05 smithi104 ceph-mon[36457]: osdmap e78: 12 total, 11 up, 12 in 2024-04-16T17:02:05.429 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:02:05 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-16T17:02:05.478 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:02:05 smithi045 ceph-mon[31676]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail; 115 KiB/s, 0 objects/s recovering 2024-04-16T17:02:05.478 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:02:05 smithi045 ceph-mon[31676]: from='client.? 172.21.15.117:0/969485587' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3fda4fea-59ad-4d1a-9fe6-fcb30e31fd7d"}]: dispatch 2024-04-16T17:02:05.479 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:02:05 smithi045 ceph-mon[31676]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3fda4fea-59ad-4d1a-9fe6-fcb30e31fd7d"}]: dispatch 2024-04-16T17:02:05.479 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:02:05 smithi045 ceph-mon[31676]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3fda4fea-59ad-4d1a-9fe6-fcb30e31fd7d"}]': finished 2024-04-16T17:02:05.479 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:02:05 smithi045 ceph-mon[31676]: osdmap e78: 12 total, 11 up, 12 in 2024-04-16T17:02:05.479 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:02:05 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-16T17:02:05.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:02:05 smithi117 ceph-mon[36730]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail; 115 KiB/s, 0 objects/s recovering 2024-04-16T17:02:05.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:02:05 smithi117 ceph-mon[36730]: from='client.? 172.21.15.117:0/969485587' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3fda4fea-59ad-4d1a-9fe6-fcb30e31fd7d"}]: dispatch 2024-04-16T17:02:05.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:02:05 smithi117 ceph-mon[36730]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3fda4fea-59ad-4d1a-9fe6-fcb30e31fd7d"}]: dispatch 2024-04-16T17:02:05.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:02:05 smithi117 ceph-mon[36730]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3fda4fea-59ad-4d1a-9fe6-fcb30e31fd7d"}]': finished 2024-04-16T17:02:05.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:02:05 smithi117 ceph-mon[36730]: osdmap e78: 12 total, 11 up, 12 in 2024-04-16T17:02:05.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:02:05 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-16T17:02:06.390 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:02:06 smithi045 ceph-mon[31676]: from='client.? 172.21.15.117:0/2556043370' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-16T17:02:06.428 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:02:06 smithi104 ceph-mon[36457]: from='client.? 172.21.15.117:0/2556043370' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-16T17:02:06.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:02:06 smithi117 ceph-mon[36730]: from='client.? 172.21.15.117:0/2556043370' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-16T17:02:07.428 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:02:07 smithi104 ceph-mon[36457]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-16T17:02:07.478 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:02:07 smithi045 ceph-mon[31676]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-16T17:02:07.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:02:07 smithi117 ceph-mon[36730]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-16T17:02:09.227 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:02:09 smithi117 ceph-mon[36730]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail; 82 KiB/s, 0 objects/s recovering 2024-04-16T17:02:09.429 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:02:09 smithi104 ceph-mon[36457]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail; 82 KiB/s, 0 objects/s recovering 2024-04-16T17:02:09.478 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:02:09 smithi045 ceph-mon[31676]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail; 82 KiB/s, 0 objects/s recovering 2024-04-16T17:02:11.254 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:02:11 smithi117 ceph-mon[36730]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-16T17:02:11.428 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:02:11 smithi104 ceph-mon[36457]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-16T17:02:11.478 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:02:11 smithi045 ceph-mon[31676]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-16T17:02:13.396 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:02:13 smithi117 ceph-mon[36730]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-16T17:02:13.396 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:02:13 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-16T17:02:13.396 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:02:13 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:02:13.396 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:02:13 smithi117 ceph-mon[36730]: Deploying daemon osd.11 on smithi117 2024-04-16T17:02:13.428 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:02:13 smithi104 ceph-mon[36457]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-16T17:02:13.428 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:02:13 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-16T17:02:13.429 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:02:13 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:02:13.429 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:02:13 smithi104 ceph-mon[36457]: Deploying daemon osd.11 on smithi117 2024-04-16T17:02:13.478 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:02:13 smithi045 ceph-mon[31676]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-16T17:02:13.478 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:02:13 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-16T17:02:13.479 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:02:13 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:02:13.479 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:02:13 smithi045 ceph-mon[31676]: Deploying daemon osd.11 on smithi117 2024-04-16T17:02:15.429 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:02:15 smithi104 ceph-mon[36457]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-04-16T17:02:15.474 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:02:15 smithi117 ceph-mon[36730]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-04-16T17:02:15.478 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:02:15 smithi045 ceph-mon[31676]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-04-16T17:02:17.306 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:02:17 smithi117 ceph-mon[36730]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-04-16T17:02:17.428 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:02:17 smithi104 ceph-mon[36457]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-04-16T17:02:17.478 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:02:17 smithi045 ceph-mon[31676]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-04-16T17:02:19.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:02:19 smithi117 ceph-mon[36730]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-04-16T17:02:19.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:02:19 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:02:19.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:02:19 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T17:02:19.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:02:19 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:02:19.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:02:19 smithi104 ceph-mon[36457]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-04-16T17:02:19.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:02:19 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:02:19.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:02:19 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T17:02:19.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:02:19 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:02:19.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:02:19 smithi045 ceph-mon[31676]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-04-16T17:02:19.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:02:19 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:02:19.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:02:19 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T17:02:19.729 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:02:19 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:02:21.501 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:02:21 smithi117 ceph-mon[36730]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-04-16T17:02:21.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:02:21 smithi104 ceph-mon[36457]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-04-16T17:02:21.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:02:21 smithi045 ceph-mon[31676]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-04-16T17:02:22.201 INFO:teuthology.orchestra.run.smithi117.stdout:Created osd(s) 11 on host 'smithi117' 2024-04-16T17:02:22.201 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T17:02:22.199+0000 7f1e22ffd700 1 -- 172.21.15.117:0/1829494456 <== mgr.14152 v2:172.21.15.45:6800/268030699 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f1e2c062f30 con 0x7f1e180a3c40 2024-04-16T17:02:22.203 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T17:02:22.201+0000 7f1e33f16700 1 -- 172.21.15.117:0/1829494456 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f1e180a3c40 msgr2=0x7f1e180a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:02:22.203 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T17:02:22.201+0000 7f1e33f16700 1 --2- 172.21.15.117:0/1829494456 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f1e180a3c40 0x7f1e180a6100 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f1e1c009a10 tx=0x7f1e1c009190 comp rx=0 tx=0).stop 2024-04-16T17:02:22.203 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T17:02:22.201+0000 7f1e33f16700 1 -- 172.21.15.117:0/1829494456 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1e2c100e00 msgr2=0x7f1e2c198d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:02:22.203 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T17:02:22.201+0000 7f1e33f16700 1 --2- 172.21.15.117:0/1829494456 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1e2c100e00 0x7f1e2c198d50 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f1e2800d9b0 tx=0x7f1e2800dcc0 comp rx=0 tx=0).stop 2024-04-16T17:02:22.203 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T17:02:22.201+0000 7f1e33f16700 1 -- 172.21.15.117:0/1829494456 shutdown_connections 2024-04-16T17:02:22.203 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T17:02:22.201+0000 7f1e33f16700 1 --2- 172.21.15.117:0/1829494456 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f1e180a3c40 0x7f1e180a6100 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:22.203 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T17:02:22.201+0000 7f1e33f16700 1 --2- 172.21.15.117:0/1829494456 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1e2c100e00 0x7f1e2c198d50 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:22.203 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T17:02:22.201+0000 7f1e33f16700 1 --2- 172.21.15.117:0/1829494456 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1e2c104780 0x7f1e2c1a0310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:22.203 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T17:02:22.201+0000 7f1e33f16700 1 --2- 172.21.15.117:0/1829494456 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1e2c103de0 0x7f1e2c199290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:22.203 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T17:02:22.201+0000 7f1e33f16700 1 -- 172.21.15.117:0/1829494456 >> 172.21.15.117:0/1829494456 conn(0x7f1e2c0fc780 msgr2=0x7f1e2c1070e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:02:22.204 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T17:02:22.201+0000 7f1e33f16700 1 -- 172.21.15.117:0/1829494456 shutdown_connections 2024-04-16T17:02:22.204 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T17:02:22.201+0000 7f1e33f16700 1 -- 172.21.15.117:0/1829494456 wait complete. 2024-04-16T17:02:22.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:02:22 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:02:22.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:02:22 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:02:22.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:02:22 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:02:22.679 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:02:22 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T17:02:22.679 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:02:22 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:02:22.679 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:02:22 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:02:22.679 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:02:22 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T17:02:22.679 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:02:22 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:02:22.679 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:02:22 smithi104 ceph-mon[36457]: from='osd.11 [v2:172.21.15.117:6824/3384003243,v1:172.21.15.117:6825/3384003243]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-16T17:02:22.679 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:02:22 smithi104 ceph-mon[36457]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-16T17:02:22.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:02:22 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:02:22.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:02:22 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:02:22.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:02:22 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:02:22.729 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:02:22 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T17:02:22.729 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:02:22 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:02:22.729 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:02:22 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:02:22.729 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:02:22 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T17:02:22.729 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:02:22 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:02:22.729 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:02:22 smithi045 ceph-mon[31676]: from='osd.11 [v2:172.21.15.117:6824/3384003243,v1:172.21.15.117:6825/3384003243]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-16T17:02:22.729 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:02:22 smithi045 ceph-mon[31676]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-16T17:02:22.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:02:22 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:02:22.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:02:22 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:02:22.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:02:22 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:02:22.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:02:22 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T17:02:22.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:02:22 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:02:22.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:02:22 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:02:22.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:02:22 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T17:02:22.807 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:02:22 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:02:22.807 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:02:22 smithi117 ceph-mon[36730]: from='osd.11 [v2:172.21.15.117:6824/3384003243,v1:172.21.15.117:6825/3384003243]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-16T17:02:22.807 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:02:22 smithi117 ceph-mon[36730]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-16T17:02:22.914 DEBUG:teuthology.orchestra.run.smithi117:osd.11> sudo journalctl -f -n 0 -u ceph-b1fecd38-fc11-11ee-bc90-c7b262605968@osd.11.service 2024-04-16T17:02:22.916 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-04-16T17:02:22.916 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b1fecd38-fc11-11ee-bc90-c7b262605968 -- ceph osd stat -f json 2024-04-16T17:02:23.392 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/mon.a/config 2024-04-16T17:02:23.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:02:23 smithi117 ceph-mon[36730]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-04-16T17:02:23.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:02:23 smithi117 ceph-mon[36730]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-16T17:02:23.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:02:23 smithi117 ceph-mon[36730]: from='osd.11 [v2:172.21.15.117:6824/3384003243,v1:172.21.15.117:6825/3384003243]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]: dispatch 2024-04-16T17:02:23.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:02:23 smithi117 ceph-mon[36730]: osdmap e79: 12 total, 11 up, 12 in 2024-04-16T17:02:23.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:02:23 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-16T17:02:23.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:02:23 smithi117 ceph-mon[36730]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]: dispatch 2024-04-16T17:02:23.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:02:23 smithi104 ceph-mon[36457]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-04-16T17:02:23.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:02:23 smithi104 ceph-mon[36457]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-16T17:02:23.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:02:23 smithi104 ceph-mon[36457]: from='osd.11 [v2:172.21.15.117:6824/3384003243,v1:172.21.15.117:6825/3384003243]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]: dispatch 2024-04-16T17:02:23.679 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:02:23 smithi104 ceph-mon[36457]: osdmap e79: 12 total, 11 up, 12 in 2024-04-16T17:02:23.679 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:02:23 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-16T17:02:23.679 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:02:23 smithi104 ceph-mon[36457]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]: dispatch 2024-04-16T17:02:23.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:02:23 smithi045 ceph-mon[31676]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-04-16T17:02:23.729 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:02:23 smithi045 ceph-mon[31676]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-16T17:02:23.729 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:02:23 smithi045 ceph-mon[31676]: from='osd.11 [v2:172.21.15.117:6824/3384003243,v1:172.21.15.117:6825/3384003243]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]: dispatch 2024-04-16T17:02:23.729 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:02:23 smithi045 ceph-mon[31676]: osdmap e79: 12 total, 11 up, 12 in 2024-04-16T17:02:23.729 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:02:23 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-16T17:02:23.729 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:02:23 smithi045 ceph-mon[31676]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]: dispatch 2024-04-16T17:02:24.427 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:02:24 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:02:24.169+0000 7f713c50b700 -1 osd.11 0 waiting for initial osdmap 2024-04-16T17:02:24.427 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:02:24 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:02:24.183+0000 7f71382fe700 -1 osd.11 80 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-16T17:02:25.428 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:02:25 smithi104 ceph-mon[36457]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-04-16T17:02:25.429 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:02:25 smithi104 ceph-mon[36457]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]': finished 2024-04-16T17:02:25.429 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:02:25 smithi104 ceph-mon[36457]: osdmap e80: 12 total, 11 up, 12 in 2024-04-16T17:02:25.429 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:02:25 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-16T17:02:25.429 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:02:25 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-16T17:02:25.462 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:02:25 smithi045 ceph-mon[31676]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-04-16T17:02:25.462 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:02:25 smithi045 ceph-mon[31676]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]': finished 2024-04-16T17:02:25.462 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:02:25 smithi045 ceph-mon[31676]: osdmap e80: 12 total, 11 up, 12 in 2024-04-16T17:02:25.462 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:02:25 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-16T17:02:25.462 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:02:25 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-16T17:02:25.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:02:25 smithi117 ceph-mon[36730]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-04-16T17:02:25.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:02:25 smithi117 ceph-mon[36730]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]': finished 2024-04-16T17:02:25.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:02:25 smithi117 ceph-mon[36730]: osdmap e80: 12 total, 11 up, 12 in 2024-04-16T17:02:25.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:02:25 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-16T17:02:25.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:02:25 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-16T17:02:25.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:25.915+0000 7f6e26e00700 1 -- 172.21.15.45:0/3646690365 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6e20103c70 msgr2=0x7f6e201040d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:02:25.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:25.915+0000 7f6e26e00700 1 --2- 172.21.15.45:0/3646690365 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6e20103c70 0x7f6e201040d0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f6e0c0098f0 tx=0x7f6e0c009c00 comp rx=0 tx=0).stop 2024-04-16T17:02:25.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:25.915+0000 7f6e26e00700 1 -- 172.21.15.45:0/3646690365 shutdown_connections 2024-04-16T17:02:25.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:25.915+0000 7f6e26e00700 1 --2- 172.21.15.45:0/3646690365 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6e20104610 0x7f6e2010aec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:25.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:25.915+0000 7f6e26e00700 1 --2- 172.21.15.45:0/3646690365 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6e20102a70 0x7f6e20102e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:25.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:25.915+0000 7f6e26e00700 1 --2- 172.21.15.45:0/3646690365 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6e20103c70 0x7f6e201040d0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:25.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:25.915+0000 7f6e26e00700 1 -- 172.21.15.45:0/3646690365 >> 172.21.15.45:0/3646690365 conn(0x7f6e200fe030 msgr2=0x7f6e20100450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:02:25.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:25.916+0000 7f6e26e00700 1 -- 172.21.15.45:0/3646690365 shutdown_connections 2024-04-16T17:02:25.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:25.916+0000 7f6e26e00700 1 -- 172.21.15.45:0/3646690365 wait complete. 2024-04-16T17:02:25.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:25.917+0000 7f6e26e00700 1 Processor -- start 2024-04-16T17:02:25.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:25.917+0000 7f6e26e00700 1 -- start start 2024-04-16T17:02:25.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:25.917+0000 7f6e26e00700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6e20102a70 0x7f6e20198d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:02:25.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:25.917+0000 7f6e26e00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6e20103c70 0x7f6e20199240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:02:25.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:25.918+0000 7f6e26e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6e20104610 0x7f6e201a0280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:02:25.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:25.918+0000 7f6e26e00700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f6e2010e6a0 con 0x7f6e20103c70 2024-04-16T17:02:25.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:25.918+0000 7f6e26e00700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f6e2010e520 con 0x7f6e20102a70 2024-04-16T17:02:25.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:25.918+0000 7f6e26e00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f6e2010e820 con 0x7f6e20104610 2024-04-16T17:02:25.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:25.918+0000 7f6e25000700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6e20103c70 0x7f6e20199240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:02:25.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:25.918+0000 7f6e25a00700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6e20102a70 0x7f6e20198d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:02:25.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:25.918+0000 7f6e25a00700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6e20102a70 0x7f6e20198d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:34778/0 (socket says 172.21.15.45:34778) 2024-04-16T17:02:25.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:25.918+0000 7f6e25a00700 1 -- 172.21.15.45:0/1937268766 learned_addr learned my addr 172.21.15.45:0/1937268766 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:02:25.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:25.918+0000 7f6e26400700 1 --2- 172.21.15.45:0/1937268766 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6e20104610 0x7f6e201a0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:02:25.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:25.918+0000 7f6e25000700 1 -- 172.21.15.45:0/1937268766 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6e20104610 msgr2=0x7f6e201a0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:02:25.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:25.918+0000 7f6e25000700 1 --2- 172.21.15.45:0/1937268766 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6e20104610 0x7f6e201a0280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:25.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:25.918+0000 7f6e25000700 1 -- 172.21.15.45:0/1937268766 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6e20102a70 msgr2=0x7f6e20198d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:02:25.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:25.918+0000 7f6e25000700 1 --2- 172.21.15.45:0/1937268766 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6e20102a70 0x7f6e20198d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:25.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:25.919+0000 7f6e25000700 1 -- 172.21.15.45:0/1937268766 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e201a0950 con 0x7f6e20103c70 2024-04-16T17:02:25.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:25.919+0000 7f6e25a00700 1 --2- 172.21.15.45:0/1937268766 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6e20102a70 0x7f6e20198d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:02:25.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:25.919+0000 7f6e25000700 1 --2- 172.21.15.45:0/1937268766 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6e20103c70 0x7f6e20199240 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f6e0c009230 tx=0x7f6e0c0052f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:02:25.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:25.919+0000 7f6e1e000700 1 -- 172.21.15.45:0/1937268766 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e0c00b560 con 0x7f6e20103c70 2024-04-16T17:02:25.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:25.919+0000 7f6e1e000700 1 -- 172.21.15.45:0/1937268766 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6e0c00bbe0 con 0x7f6e20103c70 2024-04-16T17:02:25.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:25.919+0000 7f6e1e000700 1 -- 172.21.15.45:0/1937268766 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e0c020410 con 0x7f6e20103c70 2024-04-16T17:02:25.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:25.919+0000 7f6e26e00700 1 -- 172.21.15.45:0/1937268766 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e201a0be0 con 0x7f6e20103c70 2024-04-16T17:02:25.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:25.919+0000 7f6e26e00700 1 -- 172.21.15.45:0/1937268766 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e201a0fe0 con 0x7f6e20103c70 2024-04-16T17:02:25.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:25.921+0000 7f6e26e00700 1 -- 172.21.15.45:0/1937268766 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e200697e0 con 0x7f6e20103c70 2024-04-16T17:02:25.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:25.925+0000 7f6e1e000700 1 -- 172.21.15.45:0/1937268766 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f6e0c003680 con 0x7f6e20103c70 2024-04-16T17:02:25.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:25.926+0000 7f6e1e000700 1 --2- 172.21.15.45:0/1937268766 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f6e080a3cc0 0x7f6e080a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:02:25.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:25.926+0000 7f6e1e000700 1 -- 172.21.15.45:0/1937268766 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f6e0c0d0a60 con 0x7f6e20103c70 2024-04-16T17:02:25.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:25.927+0000 7f6e25a00700 1 --2- 172.21.15.45:0/1937268766 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f6e080a3cc0 0x7f6e080a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:02:25.929 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:25.928+0000 7f6e25a00700 1 --2- 172.21.15.45:0/1937268766 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f6e080a3cc0 0x7f6e080a6180 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f6e2019a1e0 tx=0x7f6e14009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:02:25.931 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:25.930+0000 7f6e1e000700 1 -- 172.21.15.45:0/1937268766 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6e0c013030 con 0x7f6e20103c70 2024-04-16T17:02:26.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:26.175+0000 7f6e26e00700 1 -- 172.21.15.45:0/1937268766 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f6e20063700 con 0x7f6e20103c70 2024-04-16T17:02:26.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:26.177+0000 7f6e1e000700 1 -- 172.21.15.45:0/1937268766 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v81) ==== 74+0+133 (secure 0 0 0) 0x7f6e0c0094d0 con 0x7f6e20103c70 2024-04-16T17:02:26.178 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-16T17:02:26.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:26.179+0000 7f6e26e00700 1 -- 172.21.15.45:0/1937268766 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f6e080a3cc0 msgr2=0x7f6e080a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:02:26.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:26.179+0000 7f6e26e00700 1 --2- 172.21.15.45:0/1937268766 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f6e080a3cc0 0x7f6e080a6180 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f6e2019a1e0 tx=0x7f6e14009260 comp rx=0 tx=0).stop 2024-04-16T17:02:26.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:26.179+0000 7f6e26e00700 1 -- 172.21.15.45:0/1937268766 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6e20103c70 msgr2=0x7f6e20199240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:02:26.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:26.179+0000 7f6e26e00700 1 --2- 172.21.15.45:0/1937268766 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6e20103c70 0x7f6e20199240 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f6e0c009230 tx=0x7f6e0c0052f0 comp rx=0 tx=0).stop 2024-04-16T17:02:26.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:26.180+0000 7f6e26e00700 1 -- 172.21.15.45:0/1937268766 shutdown_connections 2024-04-16T17:02:26.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:26.180+0000 7f6e26e00700 1 --2- 172.21.15.45:0/1937268766 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f6e080a3cc0 0x7f6e080a6180 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:26.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:26.180+0000 7f6e26e00700 1 --2- 172.21.15.45:0/1937268766 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6e20104610 0x7f6e201a0280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:26.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:26.180+0000 7f6e26e00700 1 --2- 172.21.15.45:0/1937268766 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6e20102a70 0x7f6e20198d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:26.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:26.180+0000 7f6e26e00700 1 --2- 172.21.15.45:0/1937268766 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6e20103c70 0x7f6e20199240 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:26.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:26.180+0000 7f6e26e00700 1 -- 172.21.15.45:0/1937268766 >> 172.21.15.45:0/1937268766 conn(0x7f6e200fe030 msgr2=0x7f6e20106f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:02:26.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:26.180+0000 7f6e26e00700 1 -- 172.21.15.45:0/1937268766 shutdown_connections 2024-04-16T17:02:26.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:26.180+0000 7f6e26e00700 1 -- 172.21.15.45:0/1937268766 wait complete. 2024-04-16T17:02:26.306 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:02:26 smithi117 ceph-mon[36730]: purged_snaps scrub starts 2024-04-16T17:02:26.306 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:02:26 smithi117 ceph-mon[36730]: purged_snaps scrub ok 2024-04-16T17:02:26.306 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:02:26 smithi117 ceph-mon[36730]: osd.11 [v2:172.21.15.117:6824/3384003243,v1:172.21.15.117:6825/3384003243] boot 2024-04-16T17:02:26.306 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:02:26 smithi117 ceph-mon[36730]: osdmap e81: 12 total, 12 up, 12 in 2024-04-16T17:02:26.306 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:02:26 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-16T17:02:26.306 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:02:26 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:02:26.306 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:02:26 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:02:26.306 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:02:26 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:02:26.307 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:02:26 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:02:26.307 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:02:26 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:02:26.307 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:02:26 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:02:26.307 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:02:26 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:02:26.307 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:02:26 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:02:26.307 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:02:26 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T17:02:26.307 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:02:26 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:02:26.428 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:02:26 smithi104 ceph-mon[36457]: purged_snaps scrub starts 2024-04-16T17:02:26.429 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:02:26 smithi104 ceph-mon[36457]: purged_snaps scrub ok 2024-04-16T17:02:26.429 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:02:26 smithi104 ceph-mon[36457]: osd.11 [v2:172.21.15.117:6824/3384003243,v1:172.21.15.117:6825/3384003243] boot 2024-04-16T17:02:26.429 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:02:26 smithi104 ceph-mon[36457]: osdmap e81: 12 total, 12 up, 12 in 2024-04-16T17:02:26.429 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:02:26 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-16T17:02:26.429 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:02:26 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:02:26.429 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:02:26 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:02:26.429 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:02:26 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:02:26.429 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:02:26 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:02:26.429 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:02:26 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:02:26.429 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:02:26 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:02:26.429 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:02:26 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:02:26.429 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:02:26 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:02:26.429 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:02:26 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T17:02:26.429 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:02:26 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:02:26.479 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:02:26 smithi045 ceph-mon[31676]: purged_snaps scrub starts 2024-04-16T17:02:26.479 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:02:26 smithi045 ceph-mon[31676]: purged_snaps scrub ok 2024-04-16T17:02:26.479 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:02:26 smithi045 ceph-mon[31676]: osd.11 [v2:172.21.15.117:6824/3384003243,v1:172.21.15.117:6825/3384003243] boot 2024-04-16T17:02:26.479 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:02:26 smithi045 ceph-mon[31676]: osdmap e81: 12 total, 12 up, 12 in 2024-04-16T17:02:26.479 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:02:26 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-16T17:02:26.479 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:02:26 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:02:26.479 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:02:26 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:02:26.479 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:02:26 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:02:26.479 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:02:26 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:02:26.479 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:02:26 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:02:26.479 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:02:26 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:02:26.479 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:02:26 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:02:26.479 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:02:26 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:02:26.480 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:02:26 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T17:02:26.480 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:02:26 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:02:27.005 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":81,"num_osds":12,"num_up_osds":12,"osd_up_since":1713286945,"num_in_osds":12,"osd_in_since":1713286924,"num_remapped_pgs":0} 2024-04-16T17:02:27.006 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45 shell --fsid b1fecd38-fc11-11ee-bc90-c7b262605968 -- ceph osd dump --format=json 2024-04-16T17:02:27.228 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:02:27 smithi045 ceph-mon[31676]: Detected new or changed devices on smithi117 2024-04-16T17:02:27.228 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:02:27 smithi045 ceph-mon[31676]: Adjusting osd_memory_target on smithi117 to 4270M 2024-04-16T17:02:27.229 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:02:27 smithi045 ceph-mon[31676]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T17:02:27.229 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:02:27 smithi045 ceph-mon[31676]: from='client.? 172.21.15.45:0/1937268766' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-16T17:02:27.428 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:02:27 smithi104 ceph-mon[36457]: Detected new or changed devices on smithi117 2024-04-16T17:02:27.428 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:02:27 smithi104 ceph-mon[36457]: Adjusting osd_memory_target on smithi117 to 4270M 2024-04-16T17:02:27.429 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:02:27 smithi104 ceph-mon[36457]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T17:02:27.429 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:02:27 smithi104 ceph-mon[36457]: from='client.? 172.21.15.45:0/1937268766' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-16T17:02:27.485 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/mon.a/config 2024-04-16T17:02:27.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:02:27 smithi117 ceph-mon[36730]: Detected new or changed devices on smithi117 2024-04-16T17:02:27.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:02:27 smithi117 ceph-mon[36730]: Adjusting osd_memory_target on smithi117 to 4270M 2024-04-16T17:02:27.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:02:27 smithi117 ceph-mon[36730]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T17:02:27.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:02:27 smithi117 ceph-mon[36730]: from='client.? 172.21.15.45:0/1937268766' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-16T17:02:28.228 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:02:28 smithi045 ceph-mon[31676]: osdmap e82: 12 total, 12 up, 12 in 2024-04-16T17:02:28.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:02:28 smithi117 ceph-mon[36730]: osdmap e82: 12 total, 12 up, 12 in 2024-04-16T17:02:28.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:02:28 smithi104 ceph-mon[36457]: osdmap e82: 12 total, 12 up, 12 in 2024-04-16T17:02:29.358 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:02:29 smithi045 ceph-mon[31676]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T17:02:29.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:02:29 smithi117 ceph-mon[36730]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T17:02:29.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:02:29 smithi104 ceph-mon[36457]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T17:02:30.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:30.125+0000 7f2a4a800700 1 -- 172.21.15.45:0/3790825714 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2a4406b5a0 msgr2=0x7f2a4406ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:02:30.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:30.125+0000 7f2a4a800700 1 --2- 172.21.15.45:0/3790825714 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2a4406b5a0 0x7f2a4406ba20 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f2a30009990 tx=0x7f2a30009ca0 comp rx=0 tx=0).stop 2024-04-16T17:02:30.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:30.126+0000 7f2a4a800700 1 -- 172.21.15.45:0/3790825714 shutdown_connections 2024-04-16T17:02:30.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:30.126+0000 7f2a4a800700 1 --2- 172.21.15.45:0/3790825714 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2a44105470 0x7f2a441058f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:30.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:30.126+0000 7f2a4a800700 1 --2- 172.21.15.45:0/3790825714 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2a441079d0 0x7f2a4406b060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:30.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:30.126+0000 7f2a4a800700 1 --2- 172.21.15.45:0/3790825714 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2a4406b5a0 0x7f2a4406ba20 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:30.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:30.126+0000 7f2a4a800700 1 -- 172.21.15.45:0/3790825714 >> 172.21.15.45:0/3790825714 conn(0x7f2a44078830 msgr2=0x7f2a44078c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:02:30.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:30.126+0000 7f2a4a800700 1 -- 172.21.15.45:0/3790825714 shutdown_connections 2024-04-16T17:02:30.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:30.126+0000 7f2a4a800700 1 -- 172.21.15.45:0/3790825714 wait complete. 2024-04-16T17:02:30.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:30.127+0000 7f2a4a800700 1 Processor -- start 2024-04-16T17:02:30.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:30.127+0000 7f2a4a800700 1 -- start start 2024-04-16T17:02:30.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:30.127+0000 7f2a4a800700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2a4406b5a0 0x7f2a4419f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:02:30.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:30.127+0000 7f2a4a800700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2a44105470 0x7f2a4419fb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:02:30.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:30.127+0000 7f2a4a800700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2a441079d0 0x7f2a441a3ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:02:30.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:30.127+0000 7f2a4a800700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f2a44119260 con 0x7f2a441079d0 2024-04-16T17:02:30.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:30.127+0000 7f2a4a800700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f2a441190e0 con 0x7f2a4406b5a0 2024-04-16T17:02:30.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:30.127+0000 7f2a4a800700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f2a441193e0 con 0x7f2a44105470 2024-04-16T17:02:30.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:30.128+0000 7f2a49400700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2a4406b5a0 0x7f2a4419f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:02:30.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:30.128+0000 7f2a49400700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2a4406b5a0 0x7f2a4419f5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:55852/0 (socket says 172.21.15.45:55852) 2024-04-16T17:02:30.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:30.128+0000 7f2a49400700 1 -- 172.21.15.45:0/1504373157 learned_addr learned my addr 172.21.15.45:0/1504373157 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:02:30.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:30.128+0000 7f2a49e00700 1 --2- 172.21.15.45:0/1504373157 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2a441079d0 0x7f2a441a3ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:02:30.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:30.128+0000 7f2a48a00700 1 --2- 172.21.15.45:0/1504373157 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2a44105470 0x7f2a4419fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:02:30.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:30.128+0000 7f2a48a00700 1 -- 172.21.15.45:0/1504373157 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2a4406b5a0 msgr2=0x7f2a4419f5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:02:30.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:30.128+0000 7f2a48a00700 1 --2- 172.21.15.45:0/1504373157 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2a4406b5a0 0x7f2a4419f5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:30.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:30.128+0000 7f2a48a00700 1 -- 172.21.15.45:0/1504373157 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2a441079d0 msgr2=0x7f2a441a3ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:02:30.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:30.128+0000 7f2a48a00700 1 --2- 172.21.15.45:0/1504373157 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2a441079d0 0x7f2a441a3ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:30.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:30.128+0000 7f2a48a00700 1 -- 172.21.15.45:0/1504373157 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a441a4600 con 0x7f2a44105470 2024-04-16T17:02:30.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:30.128+0000 7f2a49400700 1 --2- 172.21.15.45:0/1504373157 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2a4406b5a0 0x7f2a4419f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:02:30.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:30.129+0000 7f2a48a00700 1 --2- 172.21.15.45:0/1504373157 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2a44105470 0x7f2a4419fb30 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f2a30006010 tx=0x7f2a3000bab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:02:30.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:30.129+0000 7f2a42000700 1 -- 172.21.15.45:0/1504373157 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a30027070 con 0x7f2a44105470 2024-04-16T17:02:30.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:30.129+0000 7f2a4a800700 1 -- 172.21.15.45:0/1504373157 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a441a4890 con 0x7f2a44105470 2024-04-16T17:02:30.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:30.129+0000 7f2a42000700 1 -- 172.21.15.45:0/1504373157 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2a300056d0 con 0x7f2a44105470 2024-04-16T17:02:30.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:30.129+0000 7f2a42000700 1 -- 172.21.15.45:0/1504373157 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a30022d90 con 0x7f2a44105470 2024-04-16T17:02:30.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:30.129+0000 7f2a4a800700 1 -- 172.21.15.45:0/1504373157 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a441ac170 con 0x7f2a44105470 2024-04-16T17:02:30.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:30.130+0000 7f2a4a800700 1 -- 172.21.15.45:0/1504373157 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a440697e0 con 0x7f2a44105470 2024-04-16T17:02:30.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:30.132+0000 7f2a42000700 1 -- 172.21.15.45:0/1504373157 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f2a3002a070 con 0x7f2a44105470 2024-04-16T17:02:30.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:30.134+0000 7f2a42000700 1 --2- 172.21.15.45:0/1504373157 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f2a2c0a3cc0 0x7f2a2c0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:02:30.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:30.134+0000 7f2a42000700 1 -- 172.21.15.45:0/1504373157 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f2a300d10e0 con 0x7f2a44105470 2024-04-16T17:02:30.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:30.134+0000 7f2a49400700 1 --2- 172.21.15.45:0/1504373157 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f2a2c0a3cc0 0x7f2a2c0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:02:30.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:30.135+0000 7f2a49400700 1 --2- 172.21.15.45:0/1504373157 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f2a2c0a3cc0 0x7f2a2c0a6180 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f2a38011550 tx=0x7f2a38009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:02:30.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:30.138+0000 7f2a42000700 1 -- 172.21.15.45:0/1504373157 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2a30013070 con 0x7f2a44105470 2024-04-16T17:02:30.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:30.372+0000 7f2a4a800700 1 -- 172.21.15.45:0/1504373157 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2a441ac400 con 0x7f2a44105470 2024-04-16T17:02:30.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:30.374+0000 7f2a42000700 1 -- 172.21.15.45:0/1504373157 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+18492 (secure 0 0 0) 0x7f2a3009df50 con 0x7f2a44105470 2024-04-16T17:02:30.376 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-16T17:02:30.376 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":82,"fsid":"b1fecd38-fc11-11ee-bc90-c7b262605968","created":"2024-04-16T16:54:02.385906+0000","modified":"2024-04-16T17:02:27.173577+0000","last_up_change":"2024-04-16T17:02:25.162624+0000","last_in_change":"2024-04-16T17:02:04.903274+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-16T16:58:06.166659+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b655de2c-59f8-4fd4-ac98-0cc222229f56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":2290170881},{"type":"v1","addr":"172.21.15.45:6803","nonce":2290170881}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":2290170881},{"type":"v1","addr":"172.21.15.45:6805","nonce":2290170881}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":2290170881},{"type":"v1","addr":"172.21.15.45:6809","nonce":2290170881}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":2290170881},{"type":"v1","addr":"172.21.15.45:6807","nonce":2290170881}]},"public_addr":"172.21.15.45:6803/2290170881","cluster_addr":"172.21.15.45:6805/2290170881","heartbeat_back_addr":"172.21.15.45:6809/2290170881","heartbeat_front_addr":"172.21.15.45:6807/2290170881","state":["exists","up"]},{"osd":1,"uuid":"ebd450ec-08fd-4c3d-8b24-4e10634ddb7e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6800","nonce":1462942067},{"type":"v1","addr":"172.21.15.104:6801","nonce":1462942067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":1462942067},{"type":"v1","addr":"172.21.15.104:6803","nonce":1462942067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":1462942067},{"type":"v1","addr":"172.21.15.104:6807","nonce":1462942067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":1462942067},{"type":"v1","addr":"172.21.15.104:6805","nonce":1462942067}]},"public_addr":"172.21.15.104:6801/1462942067","cluster_addr":"172.21.15.104:6803/1462942067","heartbeat_back_addr":"172.21.15.104:6807/1462942067","heartbeat_front_addr":"172.21.15.104:6805/1462942067","state":["exists","up"]},{"osd":2,"uuid":"b318a512-4190-42a2-8104-dcc4026ad8ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":2829260735},{"type":"v1","addr":"172.21.15.117:6801","nonce":2829260735}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":2829260735},{"type":"v1","addr":"172.21.15.117:6803","nonce":2829260735}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":2829260735},{"type":"v1","addr":"172.21.15.117:6807","nonce":2829260735}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":2829260735},{"type":"v1","addr":"172.21.15.117:6805","nonce":2829260735}]},"public_addr":"172.21.15.117:6801/2829260735","cluster_addr":"172.21.15.117:6803/2829260735","heartbeat_back_addr":"172.21.15.117:6807/2829260735","heartbeat_front_addr":"172.21.15.117:6805/2829260735","state":["exists","up"]},{"osd":3,"uuid":"ec6490cb-a715-4c12-a54a-225aae26fcb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":2823712137},{"type":"v1","addr":"172.21.15.45:6811","nonce":2823712137}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":2823712137},{"type":"v1","addr":"172.21.15.45:6813","nonce":2823712137}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":2823712137},{"type":"v1","addr":"172.21.15.45:6817","nonce":2823712137}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":2823712137},{"type":"v1","addr":"172.21.15.45:6815","nonce":2823712137}]},"public_addr":"172.21.15.45:6811/2823712137","cluster_addr":"172.21.15.45:6813/2823712137","heartbeat_back_addr":"172.21.15.45:6817/2823712137","heartbeat_front_addr":"172.21.15.45:6815/2823712137","state":["exists","up"]},{"osd":4,"uuid":"5d91d568-0250-4aae-8dc2-a0e079f92062","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":1396278127},{"type":"v1","addr":"172.21.15.104:6809","nonce":1396278127}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":1396278127},{"type":"v1","addr":"172.21.15.104:6811","nonce":1396278127}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":1396278127},{"type":"v1","addr":"172.21.15.104:6815","nonce":1396278127}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":1396278127},{"type":"v1","addr":"172.21.15.104:6813","nonce":1396278127}]},"public_addr":"172.21.15.104:6809/1396278127","cluster_addr":"172.21.15.104:6811/1396278127","heartbeat_back_addr":"172.21.15.104:6815/1396278127","heartbeat_front_addr":"172.21.15.104:6813/1396278127","state":["exists","up"]},{"osd":5,"uuid":"ac510b43-6fb2-4d0f-bc5f-16ac080d9e9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":4145720047},{"type":"v1","addr":"172.21.15.117:6809","nonce":4145720047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":4145720047},{"type":"v1","addr":"172.21.15.117:6811","nonce":4145720047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":4145720047},{"type":"v1","addr":"172.21.15.117:6815","nonce":4145720047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":4145720047},{"type":"v1","addr":"172.21.15.117:6813","nonce":4145720047}]},"public_addr":"172.21.15.117:6809/4145720047","cluster_addr":"172.21.15.117:6811/4145720047","heartbeat_back_addr":"172.21.15.117:6815/4145720047","heartbeat_front_addr":"172.21.15.117:6813/4145720047","state":["exists","up"]},{"osd":6,"uuid":"a41adc18-d88a-419e-b2c2-b089c1f32472","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":3254229176},{"type":"v1","addr":"172.21.15.45:6819","nonce":3254229176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":3254229176},{"type":"v1","addr":"172.21.15.45:6821","nonce":3254229176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":3254229176},{"type":"v1","addr":"172.21.15.45:6825","nonce":3254229176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":3254229176},{"type":"v1","addr":"172.21.15.45:6823","nonce":3254229176}]},"public_addr":"172.21.15.45:6819/3254229176","cluster_addr":"172.21.15.45:6821/3254229176","heartbeat_back_addr":"172.21.15.45:6825/3254229176","heartbeat_front_addr":"172.21.15.45:6823/3254229176","state":["exists","up"]},{"osd":7,"uuid":"b71712a4-7cf6-454d-9fd1-565e3df21c1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":4151077186},{"type":"v1","addr":"172.21.15.104:6817","nonce":4151077186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":4151077186},{"type":"v1","addr":"172.21.15.104:6819","nonce":4151077186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":4151077186},{"type":"v1","addr":"172.21.15.104:6823","nonce":4151077186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":4151077186},{"type":"v1","addr":"172.21.15.104:6821","nonce":4151077186}]},"public_addr":"172.21.15.104:6817/4151077186","cluster_addr":"172.21.15.104:6819/4151077186","heartbeat_back_addr":"172.21.15.104:6823/4151077186","heartbeat_front_addr":"172.21.15.104:6821/4151077186","state":["exists","up"]},{"osd":8,"uuid":"9e572aef-59a8-450e-a92e-df9dff4d57a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":1500286830},{"type":"v1","addr":"172.21.15.117:6817","nonce":1500286830}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":1500286830},{"type":"v1","addr":"172.21.15.117:6819","nonce":1500286830}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":1500286830},{"type":"v1","addr":"172.21.15.117:6823","nonce":1500286830}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":1500286830},{"type":"v1","addr":"172.21.15.117:6821","nonce":1500286830}]},"public_addr":"172.21.15.117:6817/1500286830","cluster_addr":"172.21.15.117:6819/1500286830","heartbeat_back_addr":"172.21.15.117:6823/1500286830","heartbeat_front_addr":"172.21.15.117:6821/1500286830","state":["exists","up"]},{"osd":9,"uuid":"014e9598-c413-4cc4-9bb8-61c6e8f59d30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":3106734074},{"type":"v1","addr":"172.21.15.45:6827","nonce":3106734074}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":3106734074},{"type":"v1","addr":"172.21.15.45:6829","nonce":3106734074}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":3106734074},{"type":"v1","addr":"172.21.15.45:6833","nonce":3106734074}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":3106734074},{"type":"v1","addr":"172.21.15.45:6831","nonce":3106734074}]},"public_addr":"172.21.15.45:6827/3106734074","cluster_addr":"172.21.15.45:6829/3106734074","heartbeat_back_addr":"172.21.15.45:6833/3106734074","heartbeat_front_addr":"172.21.15.45:6831/3106734074","state":["exists","up"]},{"osd":10,"uuid":"2deba364-3e39-47be-8a1f-8256df5b82d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":1404794652},{"type":"v1","addr":"172.21.15.104:6825","nonce":1404794652}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":1404794652},{"type":"v1","addr":"172.21.15.104:6827","nonce":1404794652}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":1404794652},{"type":"v1","addr":"172.21.15.104:6831","nonce":1404794652}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":1404794652},{"type":"v1","addr":"172.21.15.104:6829","nonce":1404794652}]},"public_addr":"172.21.15.104:6825/1404794652","cluster_addr":"172.21.15.104:6827/1404794652","heartbeat_back_addr":"172.21.15.104:6831/1404794652","heartbeat_front_addr":"172.21.15.104:6829/1404794652","state":["exists","up"]},{"osd":11,"uuid":"3fda4fea-59ad-4d1a-9fe6-fcb30e31fd7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":3384003243},{"type":"v1","addr":"172.21.15.117:6825","nonce":3384003243}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":3384003243},{"type":"v1","addr":"172.21.15.117:6827","nonce":3384003243}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":3384003243},{"type":"v1","addr":"172.21.15.117:6831","nonce":3384003243}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":3384003243},{"type":"v1","addr":"172.21.15.117:6829","nonce":3384003243}]},"public_addr":"172.21.15.117:6825/3384003243","cluster_addr":"172.21.15.117:6827/3384003243","heartbeat_back_addr":"172.21.15.117:6831/3384003243","heartbeat_front_addr":"172.21.15.117:6829/3384003243","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:57:14.799608+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:57:39.884668+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:58:03.327057+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:58:37.314531+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:59:03.667650+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:59:28.462823+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T17:00:02.968574+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T17:00:29.559190+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T17:00:55.291007+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T17:01:29.815977+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T17:01:57.015580+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T17:02:23.233151+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.45:0/4074982598":"2024-04-17T16:55:19.948001+0000","172.21.15.45:0/2443572576":"2024-04-17T16:55:19.948001+0000","172.21.15.45:6801/1983630136":"2024-04-17T16:55:19.948001+0000","172.21.15.45:6800/1983630136":"2024-04-17T16:55:19.948001+0000","172.21.15.45:0/3150465248":"2024-04-17T16:54:43.692466+0000","172.21.15.45:0/556738623":"2024-04-17T16:54:43.692466+0000","172.21.15.45:0/445676297":"2024-04-17T16:55:19.948001+0000","172.21.15.45:6800/2553659622":"2024-04-17T16:54:43.692466+0000","172.21.15.45:6801/2553659622":"2024-04-17T16:54:43.692466+0000","172.21.15.45:0/1184615408":"2024-04-17T16:54:43.692466+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-16T17:02:30.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:30.377+0000 7f2a4a800700 1 -- 172.21.15.45:0/1504373157 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f2a2c0a3cc0 msgr2=0x7f2a2c0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:02:30.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:30.377+0000 7f2a4a800700 1 --2- 172.21.15.45:0/1504373157 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f2a2c0a3cc0 0x7f2a2c0a6180 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f2a38011550 tx=0x7f2a38009260 comp rx=0 tx=0).stop 2024-04-16T17:02:30.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:30.377+0000 7f2a4a800700 1 -- 172.21.15.45:0/1504373157 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2a44105470 msgr2=0x7f2a4419fb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:02:30.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:30.377+0000 7f2a4a800700 1 --2- 172.21.15.45:0/1504373157 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2a44105470 0x7f2a4419fb30 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f2a30006010 tx=0x7f2a3000bab0 comp rx=0 tx=0).stop 2024-04-16T17:02:30.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:30.377+0000 7f2a4a800700 1 -- 172.21.15.45:0/1504373157 shutdown_connections 2024-04-16T17:02:30.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:30.377+0000 7f2a4a800700 1 --2- 172.21.15.45:0/1504373157 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2a441079d0 0x7f2a441a3ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:30.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:30.377+0000 7f2a4a800700 1 --2- 172.21.15.45:0/1504373157 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2a4406b5a0 0x7f2a4419f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:30.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:30.377+0000 7f2a4a800700 1 --2- 172.21.15.45:0/1504373157 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f2a2c0a3cc0 0x7f2a2c0a6180 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:30.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:30.377+0000 7f2a4a800700 1 --2- 172.21.15.45:0/1504373157 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2a44105470 0x7f2a4419fb30 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:30.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:30.377+0000 7f2a4a800700 1 -- 172.21.15.45:0/1504373157 >> 172.21.15.45:0/1504373157 conn(0x7f2a44078830 msgr2=0x7f2a44100ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:02:30.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:30.377+0000 7f2a4a800700 1 -- 172.21.15.45:0/1504373157 shutdown_connections 2024-04-16T17:02:30.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:30.377+0000 7f2a4a800700 1 -- 172.21.15.45:0/1504373157 wait complete. 2024-04-16T17:02:31.161 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-16T16:58:06.166659+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-16T17:02:31.162 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45 shell --fsid b1fecd38-fc11-11ee-bc90-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-04-16T17:02:31.478 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:02:31 smithi045 ceph-mon[31676]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T17:02:31.478 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:02:31 smithi045 ceph-mon[31676]: from='client.? 172.21.15.45:0/1504373157' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T17:02:31.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:02:31 smithi117 ceph-mon[36730]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T17:02:31.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:02:31 smithi117 ceph-mon[36730]: from='client.? 172.21.15.45:0/1504373157' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T17:02:31.652 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/mon.a/config 2024-04-16T17:02:31.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:02:31 smithi104 ceph-mon[36457]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T17:02:31.679 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:02:31 smithi104 ceph-mon[36457]: from='client.? 172.21.15.45:0/1504373157' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T17:02:33.478 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:02:33 smithi045 ceph-mon[31676]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T17:02:33.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:02:33 smithi117 ceph-mon[36730]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T17:02:33.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:02:33 smithi104 ceph-mon[36457]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T17:02:34.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:34.167+0000 7f0dd8600700 1 -- 172.21.15.45:0/1015513878 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0dd010f4f0 msgr2=0x7f0dd01118e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:02:34.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:34.167+0000 7f0dd8600700 1 --2- 172.21.15.45:0/1015513878 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0dd010f4f0 0x7f0dd01118e0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f0dc800b0a0 tx=0x7f0dc800b3b0 comp rx=0 tx=0).stop 2024-04-16T17:02:34.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:34.167+0000 7f0dd8600700 1 -- 172.21.15.45:0/1015513878 shutdown_connections 2024-04-16T17:02:34.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:34.167+0000 7f0dd8600700 1 --2- 172.21.15.45:0/1015513878 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0dd010f4f0 0x7f0dd01118e0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:34.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:34.167+0000 7f0dd8600700 1 --2- 172.21.15.45:0/1015513878 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0dd0074d20 0x7f0dd0075100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:34.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:34.167+0000 7f0dd8600700 1 --2- 172.21.15.45:0/1015513878 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0dd0075640 0x7f0dd010efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:34.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:34.167+0000 7f0dd8600700 1 -- 172.21.15.45:0/1015513878 >> 172.21.15.45:0/1015513878 conn(0x7f0dd0079c50 msgr2=0x7f0dd007a060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:02:34.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:34.167+0000 7f0dd8600700 1 -- 172.21.15.45:0/1015513878 shutdown_connections 2024-04-16T17:02:34.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:34.168+0000 7f0dd8600700 1 -- 172.21.15.45:0/1015513878 wait complete. 2024-04-16T17:02:34.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:34.169+0000 7f0dd8600700 1 Processor -- start 2024-04-16T17:02:34.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:34.169+0000 7f0dd8600700 1 -- start start 2024-04-16T17:02:34.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:34.170+0000 7f0dd8600700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0dd0074d20 0x7f0dd019b470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:02:34.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:34.170+0000 7f0dd8600700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0dd0075640 0x7f0dd019b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:02:34.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:34.170+0000 7f0dd8600700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0dd010f4f0 0x7f0dd019fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:02:34.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:34.170+0000 7f0dd8600700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f0dd0115020 con 0x7f0dd0075640 2024-04-16T17:02:34.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:34.170+0000 7f0dd8600700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f0dd0114ea0 con 0x7f0dd0074d20 2024-04-16T17:02:34.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:34.170+0000 7f0dd8600700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f0dd01151a0 con 0x7f0dd010f4f0 2024-04-16T17:02:34.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:34.170+0000 7f0dd5c00700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0dd0074d20 0x7f0dd019b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:02:34.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:34.170+0000 7f0dd5200700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0dd0075640 0x7f0dd019b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:02:34.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:34.170+0000 7f0dd5c00700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0dd0074d20 0x7f0dd019b470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:55858/0 (socket says 172.21.15.45:55858) 2024-04-16T17:02:34.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:34.170+0000 7f0dd5c00700 1 -- 172.21.15.45:0/1006111179 learned_addr learned my addr 172.21.15.45:0/1006111179 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:02:34.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:34.170+0000 7f0dd6600700 1 --2- 172.21.15.45:0/1006111179 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0dd010f4f0 0x7f0dd019fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:02:34.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:34.171+0000 7f0dd5c00700 1 -- 172.21.15.45:0/1006111179 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0dd010f4f0 msgr2=0x7f0dd019fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:02:34.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:34.171+0000 7f0dd5c00700 1 --2- 172.21.15.45:0/1006111179 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0dd010f4f0 0x7f0dd019fcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:34.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:34.171+0000 7f0dd5c00700 1 -- 172.21.15.45:0/1006111179 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0dd0075640 msgr2=0x7f0dd019b9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:02:34.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:34.171+0000 7f0dd5c00700 1 --2- 172.21.15.45:0/1006111179 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0dd0075640 0x7f0dd019b9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:34.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:34.171+0000 7f0dd5c00700 1 -- 172.21.15.45:0/1006111179 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0dd01a0430 con 0x7f0dd0074d20 2024-04-16T17:02:34.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:34.171+0000 7f0dd5c00700 1 --2- 172.21.15.45:0/1006111179 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0dd0074d20 0x7f0dd019b470 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f0dc400e9a0 tx=0x7f0dc400ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:02:34.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:34.171+0000 7f0dce000700 1 -- 172.21.15.45:0/1006111179 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dc400ca10 con 0x7f0dd0074d20 2024-04-16T17:02:34.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:34.171+0000 7f0dd8600700 1 -- 172.21.15.45:0/1006111179 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0dd01a0720 con 0x7f0dd0074d20 2024-04-16T17:02:34.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:34.171+0000 7f0dce000700 1 -- 172.21.15.45:0/1006111179 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0dc400cbb0 con 0x7f0dd0074d20 2024-04-16T17:02:34.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:34.171+0000 7f0dce000700 1 -- 172.21.15.45:0/1006111179 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dc4018680 con 0x7f0dd0074d20 2024-04-16T17:02:34.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:34.172+0000 7f0dd8600700 1 -- 172.21.15.45:0/1006111179 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0dd01a7fa0 con 0x7f0dd0074d20 2024-04-16T17:02:34.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:34.173+0000 7f0dd8600700 1 -- 172.21.15.45:0/1006111179 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0dd00697e0 con 0x7f0dd0074d20 2024-04-16T17:02:34.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:34.175+0000 7f0dce000700 1 -- 172.21.15.45:0/1006111179 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f0dc4010430 con 0x7f0dd0074d20 2024-04-16T17:02:34.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:34.176+0000 7f0dce000700 1 --2- 172.21.15.45:0/1006111179 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f0db80a3c40 0x7f0db80a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:02:34.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:34.176+0000 7f0dce000700 1 -- 172.21.15.45:0/1006111179 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f0dc40d01f0 con 0x7f0dd0074d20 2024-04-16T17:02:34.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:34.176+0000 7f0dd5200700 1 --2- 172.21.15.45:0/1006111179 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f0db80a3c40 0x7f0db80a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:02:34.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:34.177+0000 7f0dd5200700 1 --2- 172.21.15.45:0/1006111179 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f0db80a3c40 0x7f0db80a6100 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f0dbc009680 tx=0x7f0dbc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:02:34.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:34.180+0000 7f0dce000700 1 -- 172.21.15.45:0/1006111179 <== mon.2 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0dc409cfd0 con 0x7f0dd0074d20 2024-04-16T17:02:34.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:34.411+0000 7f0dd8600700 1 -- 172.21.15.45:0/1006111179 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f0dd01a8200 con 0x7f0dd0074d20 2024-04-16T17:02:34.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:34.412+0000 7f0dce000700 1 -- 172.21.15.45:0/1006111179 <== mon.2 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) ==== 93+0+10 (secure 0 0 0) 0x7f0dc409c8d0 con 0x7f0dd0074d20 2024-04-16T17:02:34.414 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 1 2024-04-16T17:02:34.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:34.415+0000 7f0dd8600700 1 -- 172.21.15.45:0/1006111179 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f0db80a3c40 msgr2=0x7f0db80a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:02:34.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:34.415+0000 7f0dd8600700 1 --2- 172.21.15.45:0/1006111179 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f0db80a3c40 0x7f0db80a6100 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f0dbc009680 tx=0x7f0dbc009260 comp rx=0 tx=0).stop 2024-04-16T17:02:34.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:34.415+0000 7f0dd8600700 1 -- 172.21.15.45:0/1006111179 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0dd0074d20 msgr2=0x7f0dd019b470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:02:34.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:34.415+0000 7f0dd8600700 1 --2- 172.21.15.45:0/1006111179 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0dd0074d20 0x7f0dd019b470 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f0dc400e9a0 tx=0x7f0dc400ecb0 comp rx=0 tx=0).stop 2024-04-16T17:02:34.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:34.415+0000 7f0dd8600700 1 -- 172.21.15.45:0/1006111179 shutdown_connections 2024-04-16T17:02:34.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:34.415+0000 7f0dd8600700 1 --2- 172.21.15.45:0/1006111179 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f0db80a3c40 0x7f0db80a6100 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:34.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:34.415+0000 7f0dd8600700 1 --2- 172.21.15.45:0/1006111179 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0dd010f4f0 0x7f0dd019fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:34.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:34.415+0000 7f0dd8600700 1 --2- 172.21.15.45:0/1006111179 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0dd0074d20 0x7f0dd019b470 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:34.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:34.415+0000 7f0dd8600700 1 --2- 172.21.15.45:0/1006111179 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0dd0075640 0x7f0dd019b9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:34.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:34.415+0000 7f0dd8600700 1 -- 172.21.15.45:0/1006111179 >> 172.21.15.45:0/1006111179 conn(0x7f0dd0079c50 msgr2=0x7f0dd00ffd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:02:34.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:34.416+0000 7f0dd8600700 1 -- 172.21.15.45:0/1006111179 shutdown_connections 2024-04-16T17:02:34.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:34.416+0000 7f0dd8600700 1 -- 172.21.15.45:0/1006111179 wait complete. 2024-04-16T17:02:35.253 INFO:tasks.cephadm:Adding mds.a on smithi045 2024-04-16T17:02:35.253 INFO:tasks.cephadm:Adding mds.d on smithi045 2024-04-16T17:02:35.253 INFO:tasks.cephadm:Adding mds.g on smithi045 2024-04-16T17:02:35.253 INFO:tasks.cephadm:Adding mds.j on smithi045 2024-04-16T17:02:35.253 INFO:tasks.cephadm:Adding mds.b on smithi104 2024-04-16T17:02:35.253 INFO:tasks.cephadm:Adding mds.e on smithi104 2024-04-16T17:02:35.253 INFO:tasks.cephadm:Adding mds.h on smithi104 2024-04-16T17:02:35.253 INFO:tasks.cephadm:Adding mds.k on smithi104 2024-04-16T17:02:35.254 INFO:tasks.cephadm:Adding mds.c on smithi117 2024-04-16T17:02:35.254 INFO:tasks.cephadm:Adding mds.f on smithi117 2024-04-16T17:02:35.254 INFO:tasks.cephadm:Adding mds.i on smithi117 2024-04-16T17:02:35.254 INFO:tasks.cephadm:Adding mds.l on smithi117 2024-04-16T17:02:35.254 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b1fecd38-fc11-11ee-bc90-c7b262605968 -- ceph orch apply mds all '12;smithi045=a;smithi045=d;smithi045=g;smithi045=j;smithi104=b;smithi104=e;smithi104=h;smithi104=k;smithi117=c;smithi117=f;smithi117=i;smithi117=l' 2024-04-16T17:02:35.478 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:02:35 smithi045 ceph-mon[31676]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T17:02:35.478 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:02:35 smithi045 ceph-mon[31676]: from='client.? 172.21.15.45:0/1006111179' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-16T17:02:35.517 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/mon.c/config 2024-04-16T17:02:35.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:02:35 smithi117 ceph-mon[36730]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T17:02:35.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:02:35 smithi117 ceph-mon[36730]: from='client.? 172.21.15.45:0/1006111179' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-16T17:02:35.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:02:35 smithi104 ceph-mon[36457]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T17:02:35.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:02:35 smithi104 ceph-mon[36457]: from='client.? 172.21.15.45:0/1006111179' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-16T17:02:37.253 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T17:02:37.250+0000 7f5d438eb700 1 -- 172.21.15.117:0/2641408562 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5d3c105f10 msgr2=0x7f5d3c108300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:02:37.254 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T17:02:37.250+0000 7f5d438eb700 1 --2- 172.21.15.117:0/2641408562 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5d3c105f10 0x7f5d3c108300 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f5d2c009990 tx=0x7f5d2c009ca0 comp rx=0 tx=0).stop 2024-04-16T17:02:37.254 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T17:02:37.250+0000 7f5d438eb700 1 -- 172.21.15.117:0/2641408562 shutdown_connections 2024-04-16T17:02:37.254 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T17:02:37.250+0000 7f5d438eb700 1 --2- 172.21.15.117:0/2641408562 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5d3c108840 0x7f5d3c10acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:37.254 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T17:02:37.250+0000 7f5d438eb700 1 --2- 172.21.15.117:0/2641408562 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5d3c1035e0 0x7f5d3c1059d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:37.254 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T17:02:37.250+0000 7f5d438eb700 1 --2- 172.21.15.117:0/2641408562 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5d3c105f10 0x7f5d3c108300 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:37.254 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T17:02:37.250+0000 7f5d438eb700 1 -- 172.21.15.117:0/2641408562 >> 172.21.15.117:0/2641408562 conn(0x7f5d3c0fcff0 msgr2=0x7f5d3c0ff430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:02:37.254 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T17:02:37.250+0000 7f5d438eb700 1 -- 172.21.15.117:0/2641408562 shutdown_connections 2024-04-16T17:02:37.254 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T17:02:37.250+0000 7f5d438eb700 1 -- 172.21.15.117:0/2641408562 wait complete. 2024-04-16T17:02:37.254 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T17:02:37.251+0000 7f5d438eb700 1 Processor -- start 2024-04-16T17:02:37.254 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T17:02:37.251+0000 7f5d438eb700 1 -- start start 2024-04-16T17:02:37.254 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T17:02:37.251+0000 7f5d438eb700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5d3c1035e0 0x7f5d3c198f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:02:37.254 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T17:02:37.251+0000 7f5d438eb700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5d3c105f10 0x7f5d3c199450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:02:37.255 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T17:02:37.251+0000 7f5d438eb700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5d3c108840 0x7f5d3c1a04d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:02:37.255 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T17:02:37.251+0000 7f5d438eb700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f5d3c10e820 con 0x7f5d3c105f10 2024-04-16T17:02:37.255 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T17:02:37.251+0000 7f5d438eb700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f5d3c10e6a0 con 0x7f5d3c108840 2024-04-16T17:02:37.255 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T17:02:37.251+0000 7f5d438eb700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f5d3c10e9a0 con 0x7f5d3c1035e0 2024-04-16T17:02:37.255 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T17:02:37.251+0000 7f5d41687700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5d3c1035e0 0x7f5d3c198f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:02:37.255 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T17:02:37.251+0000 7f5d41687700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5d3c1035e0 0x7f5d3c198f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:57910/0 (socket says 172.21.15.117:57910) 2024-04-16T17:02:37.255 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T17:02:37.251+0000 7f5d41687700 1 -- 172.21.15.117:0/411141344 learned_addr learned my addr 172.21.15.117:0/411141344 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-04-16T17:02:37.255 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T17:02:37.251+0000 7f5d41e88700 1 --2- 172.21.15.117:0/411141344 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5d3c108840 0x7f5d3c1a04d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:02:37.255 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T17:02:37.251+0000 7f5d41687700 1 -- 172.21.15.117:0/411141344 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5d3c108840 msgr2=0x7f5d3c1a04d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:02:37.255 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T17:02:37.251+0000 7f5d41687700 1 --2- 172.21.15.117:0/411141344 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5d3c108840 0x7f5d3c1a04d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:37.255 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T17:02:37.251+0000 7f5d41687700 1 -- 172.21.15.117:0/411141344 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5d3c105f10 msgr2=0x7f5d3c199450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:02:37.255 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T17:02:37.251+0000 7f5d41687700 1 --2- 172.21.15.117:0/411141344 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5d3c105f10 0x7f5d3c199450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:37.255 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T17:02:37.251+0000 7f5d41687700 1 -- 172.21.15.117:0/411141344 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d3c1a0b70 con 0x7f5d3c1035e0 2024-04-16T17:02:37.255 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T17:02:37.251+0000 7f5d41e88700 1 --2- 172.21.15.117:0/411141344 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5d3c108840 0x7f5d3c1a04d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:02:37.255 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T17:02:37.251+0000 7f5d41687700 1 --2- 172.21.15.117:0/411141344 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5d3c1035e0 0x7f5d3c198f10 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f5d3800b560 tx=0x7f5d3800b920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:02:37.256 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T17:02:37.252+0000 7f5d327fc700 1 -- 172.21.15.117:0/411141344 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d38015020 con 0x7f5d3c1035e0 2024-04-16T17:02:37.256 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T17:02:37.252+0000 7f5d327fc700 1 -- 172.21.15.117:0/411141344 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5d38011d10 con 0x7f5d3c1035e0 2024-04-16T17:02:37.256 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T17:02:37.252+0000 7f5d327fc700 1 -- 172.21.15.117:0/411141344 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d3800f2e0 con 0x7f5d3c1035e0 2024-04-16T17:02:37.256 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T17:02:37.252+0000 7f5d438eb700 1 -- 172.21.15.117:0/411141344 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d3c1a0e60 con 0x7f5d3c1035e0 2024-04-16T17:02:37.256 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T17:02:37.252+0000 7f5d438eb700 1 -- 172.21.15.117:0/411141344 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d3c1a12d0 con 0x7f5d3c1035e0 2024-04-16T17:02:37.257 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T17:02:37.253+0000 7f5d438eb700 1 -- 172.21.15.117:0/411141344 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d3c0697e0 con 0x7f5d3c1035e0 2024-04-16T17:02:37.257 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T17:02:37.254+0000 7f5d327fc700 1 -- 172.21.15.117:0/411141344 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f5d38018020 con 0x7f5d3c1035e0 2024-04-16T17:02:37.257 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T17:02:37.255+0000 7f5d327fc700 1 --2- 172.21.15.117:0/411141344 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f5d280a3c40 0x7f5d280a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:02:37.257 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T17:02:37.255+0000 7f5d327fc700 1 -- 172.21.15.117:0/411141344 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f5d380cf7c0 con 0x7f5d3c1035e0 2024-04-16T17:02:37.257 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T17:02:37.255+0000 7f5d40e86700 1 --2- 172.21.15.117:0/411141344 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f5d280a3c40 0x7f5d280a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:02:37.258 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T17:02:37.256+0000 7f5d40e86700 1 --2- 172.21.15.117:0/411141344 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f5d280a3c40 0x7f5d280a6100 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f5d2c0098e0 tx=0x7f5d2c005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:02:37.261 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T17:02:37.259+0000 7f5d327fc700 1 -- 172.21.15.117:0/411141344 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5d3801f020 con 0x7f5d3c1035e0 2024-04-16T17:02:37.306 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:02:37 smithi117 ceph-mon[36730]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T17:02:37.470 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T17:02:37.468+0000 7f5d438eb700 1 -- 172.21.15.117:0/411141344 --> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi045=a;smithi045=d;smithi045=g;smithi045=j;smithi104=b;smithi104=e;smithi104=h;smithi104=k;smithi117=c;smithi117=f;smithi117=i;smithi117=l", "target": ["mon-mgr", ""]}) -- 0x7f5d3c063130 con 0x7f5d280a3c40 2024-04-16T17:02:37.478 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:02:37 smithi045 ceph-mon[31676]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T17:02:37.486 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T17:02:37.485+0000 7f5d327fc700 1 -- 172.21.15.117:0/411141344 <== mgr.14152 v2:172.21.15.45:6800/268030699 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7f5d3c063130 con 0x7f5d280a3c40 2024-04-16T17:02:37.487 INFO:teuthology.orchestra.run.smithi117.stdout:Scheduled mds.all update... 2024-04-16T17:02:37.489 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T17:02:37.487+0000 7f5d438eb700 1 -- 172.21.15.117:0/411141344 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f5d280a3c40 msgr2=0x7f5d280a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:02:37.489 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T17:02:37.487+0000 7f5d438eb700 1 --2- 172.21.15.117:0/411141344 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f5d280a3c40 0x7f5d280a6100 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f5d2c0098e0 tx=0x7f5d2c005fb0 comp rx=0 tx=0).stop 2024-04-16T17:02:37.489 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T17:02:37.487+0000 7f5d438eb700 1 -- 172.21.15.117:0/411141344 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5d3c1035e0 msgr2=0x7f5d3c198f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:02:37.489 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T17:02:37.487+0000 7f5d438eb700 1 --2- 172.21.15.117:0/411141344 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5d3c1035e0 0x7f5d3c198f10 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f5d3800b560 tx=0x7f5d3800b920 comp rx=0 tx=0).stop 2024-04-16T17:02:37.489 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T17:02:37.487+0000 7f5d438eb700 1 -- 172.21.15.117:0/411141344 shutdown_connections 2024-04-16T17:02:37.489 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T17:02:37.487+0000 7f5d438eb700 1 --2- 172.21.15.117:0/411141344 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f5d280a3c40 0x7f5d280a6100 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:37.489 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T17:02:37.487+0000 7f5d438eb700 1 --2- 172.21.15.117:0/411141344 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5d3c1035e0 0x7f5d3c198f10 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:37.489 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T17:02:37.487+0000 7f5d438eb700 1 --2- 172.21.15.117:0/411141344 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5d3c108840 0x7f5d3c1a04d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:37.489 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T17:02:37.487+0000 7f5d438eb700 1 --2- 172.21.15.117:0/411141344 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5d3c105f10 0x7f5d3c199450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:37.489 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T17:02:37.487+0000 7f5d438eb700 1 -- 172.21.15.117:0/411141344 >> 172.21.15.117:0/411141344 conn(0x7f5d3c0fcff0 msgr2=0x7f5d3c104040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:02:37.489 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T17:02:37.487+0000 7f5d438eb700 1 -- 172.21.15.117:0/411141344 shutdown_connections 2024-04-16T17:02:37.489 INFO:teuthology.orchestra.run.smithi117.stderr:2024-04-16T17:02:37.487+0000 7f5d438eb700 1 -- 172.21.15.117:0/411141344 wait complete. 2024-04-16T17:02:37.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:02:37 smithi104 ceph-mon[36457]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T17:02:38.153 DEBUG:teuthology.orchestra.run.smithi045:mds.a> sudo journalctl -f -n 0 -u ceph-b1fecd38-fc11-11ee-bc90-c7b262605968@mds.a.service 2024-04-16T17:02:38.161 DEBUG:teuthology.orchestra.run.smithi045:mds.d> sudo journalctl -f -n 0 -u ceph-b1fecd38-fc11-11ee-bc90-c7b262605968@mds.d.service 2024-04-16T17:02:38.207 DEBUG:teuthology.orchestra.run.smithi045:mds.g> sudo journalctl -f -n 0 -u ceph-b1fecd38-fc11-11ee-bc90-c7b262605968@mds.g.service 2024-04-16T17:02:38.255 DEBUG:teuthology.orchestra.run.smithi045:mds.j> sudo journalctl -f -n 0 -u ceph-b1fecd38-fc11-11ee-bc90-c7b262605968@mds.j.service 2024-04-16T17:02:38.300 DEBUG:teuthology.orchestra.run.smithi104:mds.b> sudo journalctl -f -n 0 -u ceph-b1fecd38-fc11-11ee-bc90-c7b262605968@mds.b.service 2024-04-16T17:02:38.302 DEBUG:teuthology.orchestra.run.smithi104:mds.e> sudo journalctl -f -n 0 -u ceph-b1fecd38-fc11-11ee-bc90-c7b262605968@mds.e.service 2024-04-16T17:02:38.345 DEBUG:teuthology.orchestra.run.smithi104:mds.h> sudo journalctl -f -n 0 -u ceph-b1fecd38-fc11-11ee-bc90-c7b262605968@mds.h.service 2024-04-16T17:02:38.388 DEBUG:teuthology.orchestra.run.smithi104:mds.k> sudo journalctl -f -n 0 -u ceph-b1fecd38-fc11-11ee-bc90-c7b262605968@mds.k.service 2024-04-16T17:02:38.430 DEBUG:teuthology.orchestra.run.smithi117:mds.c> sudo journalctl -f -n 0 -u ceph-b1fecd38-fc11-11ee-bc90-c7b262605968@mds.c.service 2024-04-16T17:02:38.433 DEBUG:teuthology.orchestra.run.smithi117:mds.f> sudo journalctl -f -n 0 -u ceph-b1fecd38-fc11-11ee-bc90-c7b262605968@mds.f.service 2024-04-16T17:02:38.475 DEBUG:teuthology.orchestra.run.smithi117:mds.i> sudo journalctl -f -n 0 -u ceph-b1fecd38-fc11-11ee-bc90-c7b262605968@mds.i.service 2024-04-16T17:02:38.518 DEBUG:teuthology.orchestra.run.smithi117:mds.l> sudo journalctl -f -n 0 -u ceph-b1fecd38-fc11-11ee-bc90-c7b262605968@mds.l.service 2024-04-16T17:02:38.560 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-04-16T17:02:38.560 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-16T17:02:38.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:02:38 smithi104 ceph-mon[36457]: from='client.24406 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi045=a;smithi045=d;smithi045=g;smithi045=j;smithi104=b;smithi104=e;smithi104=h;smithi104=k;smithi117=c;smithi117=f;smithi117=i;smithi117=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T17:02:38.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:02:38 smithi104 ceph-mon[36457]: Saving service mds.all spec with placement smithi045=a;smithi045=d;smithi045=g;smithi045=j;smithi104=b;smithi104=e;smithi104=h;smithi104=k;smithi117=c;smithi117=f;smithi117=i;smithi117=l;count:12 2024-04-16T17:02:38.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:02:38 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:02:38.679 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:02:38 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T17:02:38.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:02:38 smithi117 ceph-mon[36730]: from='client.24406 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi045=a;smithi045=d;smithi045=g;smithi045=j;smithi104=b;smithi104=e;smithi104=h;smithi104=k;smithi117=c;smithi117=f;smithi117=i;smithi117=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T17:02:38.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:02:38 smithi117 ceph-mon[36730]: Saving service mds.all spec with placement smithi045=a;smithi045=d;smithi045=g;smithi045=j;smithi104=b;smithi104=e;smithi104=h;smithi104=k;smithi117=c;smithi117=f;smithi117=i;smithi117=l;count:12 2024-04-16T17:02:38.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:02:38 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:02:38.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:02:38 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T17:02:38.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:38.816+0000 7f913b800640 1 -- 172.21.15.45:0/3385182809 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9134109850 msgr2=0x7f9134109cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:02:38.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:38.816+0000 7f913b800640 1 --2- 172.21.15.45:0/3385182809 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9134109850 0x7f9134109cd0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f9120009860 tx=0x7f912002bb70 comp rx=0 tx=0).stop 2024-04-16T17:02:38.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:38.817+0000 7f913b800640 1 -- 172.21.15.45:0/3385182809 shutdown_connections 2024-04-16T17:02:38.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:38.817+0000 7f913b800640 1 --2- 172.21.15.45:0/3385182809 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9134109850 0x7f9134109cd0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:38.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:38.817+0000 7f913b800640 1 --2- 172.21.15.45:0/3385182809 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f913410a210 0x7f9134110aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:38.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:38.817+0000 7f913b800640 1 --2- 172.21.15.45:0/3385182809 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9134108660 0x7f9134108a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:38.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:38.817+0000 7f913b800640 1 -- 172.21.15.45:0/3385182809 >> 172.21.15.45:0/3385182809 conn(0x7f9134077620 msgr2=0x7f9134079a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:02:38.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:38.817+0000 7f913b800640 1 -- 172.21.15.45:0/3385182809 shutdown_connections 2024-04-16T17:02:38.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:38.817+0000 7f913b800640 1 -- 172.21.15.45:0/3385182809 wait complete. 2024-04-16T17:02:38.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:38.818+0000 7f913b800640 1 Processor -- start 2024-04-16T17:02:38.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:38.818+0000 7f913b800640 1 -- start start 2024-04-16T17:02:38.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:38.818+0000 7f913b800640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9134108660 0x7f913419e990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:02:38.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:38.818+0000 7f913b800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9134109850 0x7f913419eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:02:38.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:38.818+0000 7f913b800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f913410a210 0x7f91341a5f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:02:38.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:38.818+0000 7f913b800640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f91341142d0 con 0x7f9134109850 2024-04-16T17:02:38.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:38.818+0000 7f913b800640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f9134114150 con 0x7f9134108660 2024-04-16T17:02:38.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:38.818+0000 7f913b800640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f9134114450 con 0x7f913410a210 2024-04-16T17:02:38.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:38.818+0000 7f9139800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f913410a210 0x7f91341a5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:02:38.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:38.818+0000 7f9133e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9134109850 0x7f913419eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:02:38.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:38.818+0000 7f9139800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f913410a210 0x7f91341a5f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:43798/0 (socket says 172.21.15.45:43798) 2024-04-16T17:02:38.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:38.818+0000 7f9139800640 1 -- 172.21.15.45:0/2198935209 learned_addr learned my addr 172.21.15.45:0/2198935209 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:02:38.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:38.819+0000 7f9138e00640 1 --2- 172.21.15.45:0/2198935209 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9134108660 0x7f913419e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:02:38.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:38.819+0000 7f9139800640 1 -- 172.21.15.45:0/2198935209 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9134108660 msgr2=0x7f913419e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:02:38.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:38.819+0000 7f9139800640 1 --2- 172.21.15.45:0/2198935209 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9134108660 0x7f913419e990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:38.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:38.819+0000 7f9139800640 1 -- 172.21.15.45:0/2198935209 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9134109850 msgr2=0x7f913419eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:02:38.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:38.819+0000 7f9139800640 1 --2- 172.21.15.45:0/2198935209 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9134109850 0x7f913419eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:38.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:38.819+0000 7f9139800640 1 -- 172.21.15.45:0/2198935209 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91341a65b0 con 0x7f913410a210 2024-04-16T17:02:38.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:38.819+0000 7f9133e00640 1 --2- 172.21.15.45:0/2198935209 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9134109850 0x7f913419eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:02:38.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:38.819+0000 7f9139800640 1 --2- 172.21.15.45:0/2198935209 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f913410a210 0x7f91341a5f40 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f912c00e940 tx=0x7f912c00ee00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:02:38.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:38.819+0000 7f9131600640 1 -- 172.21.15.45:0/2198935209 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f912c00cd80 con 0x7f913410a210 2024-04-16T17:02:38.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:38.819+0000 7f9131600640 1 -- 172.21.15.45:0/2198935209 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f912c0035f0 con 0x7f913410a210 2024-04-16T17:02:38.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:38.819+0000 7f9131600640 1 -- 172.21.15.45:0/2198935209 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f912c010690 con 0x7f913410a210 2024-04-16T17:02:38.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:38.819+0000 7f913b800640 1 -- 172.21.15.45:0/2198935209 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91341a68a0 con 0x7f913410a210 2024-04-16T17:02:38.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:38.819+0000 7f913b800640 1 -- 172.21.15.45:0/2198935209 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91341a6dc0 con 0x7f913410a210 2024-04-16T17:02:38.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:38.821+0000 7f913b800640 1 -- 172.21.15.45:0/2198935209 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f913410f980 con 0x7f913410a210 2024-04-16T17:02:38.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:38.822+0000 7f9131600640 1 -- 172.21.15.45:0/2198935209 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f912c0037a0 con 0x7f913410a210 2024-04-16T17:02:38.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:38.824+0000 7f9131600640 1 --2- 172.21.15.45:0/2198935209 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f91180a3c30 0x7f91180a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:02:38.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:38.824+0000 7f9131600640 1 -- 172.21.15.45:0/2198935209 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f912c0d02b0 con 0x7f913410a210 2024-04-16T17:02:38.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:38.824+0000 7f9138e00640 1 --2- 172.21.15.45:0/2198935209 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f91180a3c30 0x7f91180a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:02:38.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:38.825+0000 7f9138e00640 1 --2- 172.21.15.45:0/2198935209 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f91180a3c30 0x7f91180a60f0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f91240099a0 tx=0x7f9124009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:02:38.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:38.828+0000 7f9131600640 1 -- 172.21.15.45:0/2198935209 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f912c09d090 con 0x7f913410a210 2024-04-16T17:02:38.978 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:02:38 smithi045 ceph-mon[31676]: from='client.24406 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi045=a;smithi045=d;smithi045=g;smithi045=j;smithi104=b;smithi104=e;smithi104=h;smithi104=k;smithi117=c;smithi117=f;smithi117=i;smithi117=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T17:02:38.978 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:02:38 smithi045 ceph-mon[31676]: Saving service mds.all spec with placement smithi045=a;smithi045=d;smithi045=g;smithi045=j;smithi104=b;smithi104=e;smithi104=h;smithi104=k;smithi117=c;smithi117=f;smithi117=i;smithi117=l;count:12 2024-04-16T17:02:38.979 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:02:38 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:02:38.979 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:02:38 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T17:02:39.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:39.036+0000 7f913b800640 1 -- 172.21.15.45:0/2198935209 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f913402d620 con 0x7f913410a210 2024-04-16T17:02:39.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:39.037+0000 7f9131600640 1 -- 172.21.15.45:0/2198935209 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+18492 (secure 0 0 0) 0x7f912c09c990 con 0x7f913410a210 2024-04-16T17:02:39.039 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-16T17:02:39.039 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":82,"fsid":"b1fecd38-fc11-11ee-bc90-c7b262605968","created":"2024-04-16T16:54:02.385906+0000","modified":"2024-04-16T17:02:27.173577+0000","last_up_change":"2024-04-16T17:02:25.162624+0000","last_in_change":"2024-04-16T17:02:04.903274+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-16T16:58:06.166659+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b655de2c-59f8-4fd4-ac98-0cc222229f56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":2290170881},{"type":"v1","addr":"172.21.15.45:6803","nonce":2290170881}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":2290170881},{"type":"v1","addr":"172.21.15.45:6805","nonce":2290170881}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":2290170881},{"type":"v1","addr":"172.21.15.45:6809","nonce":2290170881}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":2290170881},{"type":"v1","addr":"172.21.15.45:6807","nonce":2290170881}]},"public_addr":"172.21.15.45:6803/2290170881","cluster_addr":"172.21.15.45:6805/2290170881","heartbeat_back_addr":"172.21.15.45:6809/2290170881","heartbeat_front_addr":"172.21.15.45:6807/2290170881","state":["exists","up"]},{"osd":1,"uuid":"ebd450ec-08fd-4c3d-8b24-4e10634ddb7e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6800","nonce":1462942067},{"type":"v1","addr":"172.21.15.104:6801","nonce":1462942067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":1462942067},{"type":"v1","addr":"172.21.15.104:6803","nonce":1462942067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":1462942067},{"type":"v1","addr":"172.21.15.104:6807","nonce":1462942067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":1462942067},{"type":"v1","addr":"172.21.15.104:6805","nonce":1462942067}]},"public_addr":"172.21.15.104:6801/1462942067","cluster_addr":"172.21.15.104:6803/1462942067","heartbeat_back_addr":"172.21.15.104:6807/1462942067","heartbeat_front_addr":"172.21.15.104:6805/1462942067","state":["exists","up"]},{"osd":2,"uuid":"b318a512-4190-42a2-8104-dcc4026ad8ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":2829260735},{"type":"v1","addr":"172.21.15.117:6801","nonce":2829260735}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":2829260735},{"type":"v1","addr":"172.21.15.117:6803","nonce":2829260735}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":2829260735},{"type":"v1","addr":"172.21.15.117:6807","nonce":2829260735}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":2829260735},{"type":"v1","addr":"172.21.15.117:6805","nonce":2829260735}]},"public_addr":"172.21.15.117:6801/2829260735","cluster_addr":"172.21.15.117:6803/2829260735","heartbeat_back_addr":"172.21.15.117:6807/2829260735","heartbeat_front_addr":"172.21.15.117:6805/2829260735","state":["exists","up"]},{"osd":3,"uuid":"ec6490cb-a715-4c12-a54a-225aae26fcb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":2823712137},{"type":"v1","addr":"172.21.15.45:6811","nonce":2823712137}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":2823712137},{"type":"v1","addr":"172.21.15.45:6813","nonce":2823712137}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":2823712137},{"type":"v1","addr":"172.21.15.45:6817","nonce":2823712137}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":2823712137},{"type":"v1","addr":"172.21.15.45:6815","nonce":2823712137}]},"public_addr":"172.21.15.45:6811/2823712137","cluster_addr":"172.21.15.45:6813/2823712137","heartbeat_back_addr":"172.21.15.45:6817/2823712137","heartbeat_front_addr":"172.21.15.45:6815/2823712137","state":["exists","up"]},{"osd":4,"uuid":"5d91d568-0250-4aae-8dc2-a0e079f92062","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":1396278127},{"type":"v1","addr":"172.21.15.104:6809","nonce":1396278127}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":1396278127},{"type":"v1","addr":"172.21.15.104:6811","nonce":1396278127}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":1396278127},{"type":"v1","addr":"172.21.15.104:6815","nonce":1396278127}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":1396278127},{"type":"v1","addr":"172.21.15.104:6813","nonce":1396278127}]},"public_addr":"172.21.15.104:6809/1396278127","cluster_addr":"172.21.15.104:6811/1396278127","heartbeat_back_addr":"172.21.15.104:6815/1396278127","heartbeat_front_addr":"172.21.15.104:6813/1396278127","state":["exists","up"]},{"osd":5,"uuid":"ac510b43-6fb2-4d0f-bc5f-16ac080d9e9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":4145720047},{"type":"v1","addr":"172.21.15.117:6809","nonce":4145720047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":4145720047},{"type":"v1","addr":"172.21.15.117:6811","nonce":4145720047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":4145720047},{"type":"v1","addr":"172.21.15.117:6815","nonce":4145720047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":4145720047},{"type":"v1","addr":"172.21.15.117:6813","nonce":4145720047}]},"public_addr":"172.21.15.117:6809/4145720047","cluster_addr":"172.21.15.117:6811/4145720047","heartbeat_back_addr":"172.21.15.117:6815/4145720047","heartbeat_front_addr":"172.21.15.117:6813/4145720047","state":["exists","up"]},{"osd":6,"uuid":"a41adc18-d88a-419e-b2c2-b089c1f32472","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":3254229176},{"type":"v1","addr":"172.21.15.45:6819","nonce":3254229176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":3254229176},{"type":"v1","addr":"172.21.15.45:6821","nonce":3254229176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":3254229176},{"type":"v1","addr":"172.21.15.45:6825","nonce":3254229176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":3254229176},{"type":"v1","addr":"172.21.15.45:6823","nonce":3254229176}]},"public_addr":"172.21.15.45:6819/3254229176","cluster_addr":"172.21.15.45:6821/3254229176","heartbeat_back_addr":"172.21.15.45:6825/3254229176","heartbeat_front_addr":"172.21.15.45:6823/3254229176","state":["exists","up"]},{"osd":7,"uuid":"b71712a4-7cf6-454d-9fd1-565e3df21c1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":4151077186},{"type":"v1","addr":"172.21.15.104:6817","nonce":4151077186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":4151077186},{"type":"v1","addr":"172.21.15.104:6819","nonce":4151077186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":4151077186},{"type":"v1","addr":"172.21.15.104:6823","nonce":4151077186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":4151077186},{"type":"v1","addr":"172.21.15.104:6821","nonce":4151077186}]},"public_addr":"172.21.15.104:6817/4151077186","cluster_addr":"172.21.15.104:6819/4151077186","heartbeat_back_addr":"172.21.15.104:6823/4151077186","heartbeat_front_addr":"172.21.15.104:6821/4151077186","state":["exists","up"]},{"osd":8,"uuid":"9e572aef-59a8-450e-a92e-df9dff4d57a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":1500286830},{"type":"v1","addr":"172.21.15.117:6817","nonce":1500286830}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":1500286830},{"type":"v1","addr":"172.21.15.117:6819","nonce":1500286830}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":1500286830},{"type":"v1","addr":"172.21.15.117:6823","nonce":1500286830}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":1500286830},{"type":"v1","addr":"172.21.15.117:6821","nonce":1500286830}]},"public_addr":"172.21.15.117:6817/1500286830","cluster_addr":"172.21.15.117:6819/1500286830","heartbeat_back_addr":"172.21.15.117:6823/1500286830","heartbeat_front_addr":"172.21.15.117:6821/1500286830","state":["exists","up"]},{"osd":9,"uuid":"014e9598-c413-4cc4-9bb8-61c6e8f59d30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":3106734074},{"type":"v1","addr":"172.21.15.45:6827","nonce":3106734074}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":3106734074},{"type":"v1","addr":"172.21.15.45:6829","nonce":3106734074}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":3106734074},{"type":"v1","addr":"172.21.15.45:6833","nonce":3106734074}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":3106734074},{"type":"v1","addr":"172.21.15.45:6831","nonce":3106734074}]},"public_addr":"172.21.15.45:6827/3106734074","cluster_addr":"172.21.15.45:6829/3106734074","heartbeat_back_addr":"172.21.15.45:6833/3106734074","heartbeat_front_addr":"172.21.15.45:6831/3106734074","state":["exists","up"]},{"osd":10,"uuid":"2deba364-3e39-47be-8a1f-8256df5b82d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":1404794652},{"type":"v1","addr":"172.21.15.104:6825","nonce":1404794652}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":1404794652},{"type":"v1","addr":"172.21.15.104:6827","nonce":1404794652}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":1404794652},{"type":"v1","addr":"172.21.15.104:6831","nonce":1404794652}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":1404794652},{"type":"v1","addr":"172.21.15.104:6829","nonce":1404794652}]},"public_addr":"172.21.15.104:6825/1404794652","cluster_addr":"172.21.15.104:6827/1404794652","heartbeat_back_addr":"172.21.15.104:6831/1404794652","heartbeat_front_addr":"172.21.15.104:6829/1404794652","state":["exists","up"]},{"osd":11,"uuid":"3fda4fea-59ad-4d1a-9fe6-fcb30e31fd7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":3384003243},{"type":"v1","addr":"172.21.15.117:6825","nonce":3384003243}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":3384003243},{"type":"v1","addr":"172.21.15.117:6827","nonce":3384003243}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":3384003243},{"type":"v1","addr":"172.21.15.117:6831","nonce":3384003243}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":3384003243},{"type":"v1","addr":"172.21.15.117:6829","nonce":3384003243}]},"public_addr":"172.21.15.117:6825/3384003243","cluster_addr":"172.21.15.117:6827/3384003243","heartbeat_back_addr":"172.21.15.117:6831/3384003243","heartbeat_front_addr":"172.21.15.117:6829/3384003243","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:57:14.799608+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:57:39.884668+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:58:03.327057+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:58:37.314531+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:59:03.667650+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:59:28.462823+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T17:00:02.968574+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T17:00:29.559190+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T17:00:55.291007+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T17:01:29.815977+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T17:01:57.015580+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T17:02:23.233151+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.45:0/4074982598":"2024-04-17T16:55:19.948001+0000","172.21.15.45:0/2443572576":"2024-04-17T16:55:19.948001+0000","172.21.15.45:6801/1983630136":"2024-04-17T16:55:19.948001+0000","172.21.15.45:6800/1983630136":"2024-04-17T16:55:19.948001+0000","172.21.15.45:0/3150465248":"2024-04-17T16:54:43.692466+0000","172.21.15.45:0/556738623":"2024-04-17T16:54:43.692466+0000","172.21.15.45:0/445676297":"2024-04-17T16:55:19.948001+0000","172.21.15.45:6800/2553659622":"2024-04-17T16:54:43.692466+0000","172.21.15.45:6801/2553659622":"2024-04-17T16:54:43.692466+0000","172.21.15.45:0/1184615408":"2024-04-17T16:54:43.692466+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-16T17:02:39.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:39.041+0000 7f913b800640 1 -- 172.21.15.45:0/2198935209 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f91180a3c30 msgr2=0x7f91180a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:02:39.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:39.041+0000 7f913b800640 1 --2- 172.21.15.45:0/2198935209 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f91180a3c30 0x7f91180a60f0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f91240099a0 tx=0x7f9124009500 comp rx=0 tx=0).stop 2024-04-16T17:02:39.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:39.041+0000 7f913b800640 1 -- 172.21.15.45:0/2198935209 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f913410a210 msgr2=0x7f91341a5f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:02:39.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:39.041+0000 7f913b800640 1 --2- 172.21.15.45:0/2198935209 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f913410a210 0x7f91341a5f40 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f912c00e940 tx=0x7f912c00ee00 comp rx=0 tx=0).stop 2024-04-16T17:02:39.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:39.041+0000 7f913b800640 1 -- 172.21.15.45:0/2198935209 shutdown_connections 2024-04-16T17:02:39.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:39.041+0000 7f913b800640 1 --2- 172.21.15.45:0/2198935209 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f91180a3c30 0x7f91180a60f0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:39.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:39.041+0000 7f913b800640 1 --2- 172.21.15.45:0/2198935209 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f913410a210 0x7f91341a5f40 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:39.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:39.041+0000 7f913b800640 1 --2- 172.21.15.45:0/2198935209 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9134109850 0x7f913419eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:39.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:39.041+0000 7f913b800640 1 --2- 172.21.15.45:0/2198935209 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9134108660 0x7f913419e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:39.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:39.041+0000 7f913b800640 1 -- 172.21.15.45:0/2198935209 >> 172.21.15.45:0/2198935209 conn(0x7f9134077620 msgr2=0x7f913410cb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:02:39.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:39.042+0000 7f913b800640 1 -- 172.21.15.45:0/2198935209 shutdown_connections 2024-04-16T17:02:39.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:39.042+0000 7f913b800640 1 -- 172.21.15.45:0/2198935209 wait complete. 2024-04-16T17:02:39.063 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-16T16:58:06.166659+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-16T17:02:39.063 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-16T17:02:39.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:39.294+0000 7f9dc0c00640 1 -- 172.21.15.45:0/1226627009 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9dbc108660 msgr2=0x7f9dbc108a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:02:39.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:39.294+0000 7f9dc0c00640 1 --2- 172.21.15.45:0/1226627009 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9dbc108660 0x7f9dbc108a80 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f9dac009900 tx=0x7f9dac02bc30 comp rx=0 tx=0).stop 2024-04-16T17:02:39.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:39.294+0000 7f9dc0c00640 1 -- 172.21.15.45:0/1226627009 shutdown_connections 2024-04-16T17:02:39.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:39.294+0000 7f9dc0c00640 1 --2- 172.21.15.45:0/1226627009 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9dbc10a210 0x7f9dbc110aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:39.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:39.294+0000 7f9dc0c00640 1 --2- 172.21.15.45:0/1226627009 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9dbc109850 0x7f9dbc109cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:39.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:39.294+0000 7f9dc0c00640 1 --2- 172.21.15.45:0/1226627009 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9dbc108660 0x7f9dbc108a80 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:39.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:39.294+0000 7f9dc0c00640 1 -- 172.21.15.45:0/1226627009 >> 172.21.15.45:0/1226627009 conn(0x7f9dbc077600 msgr2=0x7f9dbc079a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:02:39.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:39.294+0000 7f9dc0c00640 1 -- 172.21.15.45:0/1226627009 shutdown_connections 2024-04-16T17:02:39.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:39.294+0000 7f9dc0c00640 1 -- 172.21.15.45:0/1226627009 wait complete. 2024-04-16T17:02:39.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:39.295+0000 7f9dc0c00640 1 Processor -- start 2024-04-16T17:02:39.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:39.296+0000 7f9dc0c00640 1 -- start start 2024-04-16T17:02:39.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:39.296+0000 7f9dc0c00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9dbc108660 0x7f9dbc19e950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:02:39.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:39.296+0000 7f9dc0c00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9dbc109850 0x7f9dbc19ee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:02:39.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:39.296+0000 7f9dc0c00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9dbc10a210 0x7f9dbc1a5f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:02:39.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:39.296+0000 7f9dc0c00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f9dbc114330 con 0x7f9dbc10a210 2024-04-16T17:02:39.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:39.296+0000 7f9dc0c00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f9dbc1141b0 con 0x7f9dbc108660 2024-04-16T17:02:39.298 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:39.296+0000 7f9dc0c00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f9dbc1144b0 con 0x7f9dbc109850 2024-04-16T17:02:39.298 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:39.296+0000 7f9db9e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9dbc108660 0x7f9dbc19e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:02:39.298 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:39.296+0000 7f9db1400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9dbc109850 0x7f9dbc19ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:02:39.298 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:39.296+0000 7f9db9e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9dbc108660 0x7f9dbc19e950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:55894/0 (socket says 172.21.15.45:55894) 2024-04-16T17:02:39.298 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:39.296+0000 7f9db1400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9dbc109850 0x7f9dbc19ee90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:43806/0 (socket says 172.21.15.45:43806) 2024-04-16T17:02:39.298 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:39.296+0000 7f9db9e00640 1 -- 172.21.15.45:0/1225243169 learned_addr learned my addr 172.21.15.45:0/1225243169 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:02:39.298 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:39.296+0000 7f9dba800640 1 --2- 172.21.15.45:0/1225243169 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9dbc10a210 0x7f9dbc1a5f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:02:39.299 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:39.297+0000 7f9db9e00640 1 -- 172.21.15.45:0/1225243169 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9dbc109850 msgr2=0x7f9dbc19ee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:02:39.299 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:39.297+0000 7f9db9e00640 1 --2- 172.21.15.45:0/1225243169 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9dbc109850 0x7f9dbc19ee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:39.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:39.297+0000 7f9db9e00640 1 -- 172.21.15.45:0/1225243169 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9dbc10a210 msgr2=0x7f9dbc1a5f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:02:39.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:39.297+0000 7f9db9e00640 1 --2- 172.21.15.45:0/1225243169 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9dbc10a210 0x7f9dbc1a5f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:39.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:39.297+0000 7f9db9e00640 1 -- 172.21.15.45:0/1225243169 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9dbc1a6570 con 0x7f9dbc108660 2024-04-16T17:02:39.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:39.297+0000 7f9db1400640 1 --2- 172.21.15.45:0/1225243169 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9dbc109850 0x7f9dbc19ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:02:39.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:39.297+0000 7f9dba800640 1 --2- 172.21.15.45:0/1225243169 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9dbc10a210 0x7f9dbc1a5f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:02:39.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:39.297+0000 7f9db9e00640 1 --2- 172.21.15.45:0/1225243169 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9dbc108660 0x7f9dbc19e950 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f9dac000c00 tx=0x7f9dac02c650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:02:39.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:39.297+0000 7f9db3400640 1 -- 172.21.15.45:0/1225243169 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dac043020 con 0x7f9dbc108660 2024-04-16T17:02:39.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:39.297+0000 7f9db3400640 1 -- 172.21.15.45:0/1225243169 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9dac0054e0 con 0x7f9dbc108660 2024-04-16T17:02:39.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:39.297+0000 7f9dc0c00640 1 -- 172.21.15.45:0/1225243169 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9dbc1a6800 con 0x7f9dbc108660 2024-04-16T17:02:39.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:39.297+0000 7f9db3400640 1 -- 172.21.15.45:0/1225243169 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dac003c40 con 0x7f9dbc108660 2024-04-16T17:02:39.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:39.298+0000 7f9dc0c00640 1 -- 172.21.15.45:0/1225243169 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9dbc1a6da0 con 0x7f9dbc108660 2024-04-16T17:02:39.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:39.299+0000 7f9dc0c00640 1 -- 172.21.15.45:0/1225243169 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9dbc10f980 con 0x7f9dbc108660 2024-04-16T17:02:39.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:39.300+0000 7f9db3400640 1 -- 172.21.15.45:0/1225243169 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f9dac046020 con 0x7f9dbc108660 2024-04-16T17:02:39.303 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:39.302+0000 7f9db3400640 1 --2- 172.21.15.45:0/1225243169 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f9da00a3c30 0x7f9da00a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:02:39.303 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:39.302+0000 7f9db3400640 1 -- 172.21.15.45:0/1225243169 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f9dac0f1f20 con 0x7f9dbc108660 2024-04-16T17:02:39.303 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:39.302+0000 7f9db1400640 1 --2- 172.21.15.45:0/1225243169 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f9da00a3c30 0x7f9da00a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:02:39.303 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:39.303+0000 7f9db1400640 1 --2- 172.21.15.45:0/1225243169 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f9da00a3c30 0x7f9da00a60f0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f9dbc19fe60 tx=0x7f9da4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:02:39.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:39.306+0000 7f9db3400640 1 -- 172.21.15.45:0/1225243169 <== mon.2 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9dac04d020 con 0x7f9dbc108660 2024-04-16T17:02:39.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:39.515+0000 7f9dc0c00640 1 -- 172.21.15.45:0/1225243169 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f9dbc02d7b0 con 0x7f9dbc108660 2024-04-16T17:02:39.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:39.516+0000 7f9db3400640 1 -- 172.21.15.45:0/1225243169 <== mon.2 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) ==== 93+0+10 (secure 0 0 0) 0x7f9dac0bed00 con 0x7f9dbc108660 2024-04-16T17:02:39.517 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 1 2024-04-16T17:02:39.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:39.519+0000 7f9dc0c00640 1 -- 172.21.15.45:0/1225243169 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f9da00a3c30 msgr2=0x7f9da00a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:02:39.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:39.519+0000 7f9dc0c00640 1 --2- 172.21.15.45:0/1225243169 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f9da00a3c30 0x7f9da00a60f0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f9dbc19fe60 tx=0x7f9da4005f90 comp rx=0 tx=0).stop 2024-04-16T17:02:39.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:39.519+0000 7f9dc0c00640 1 -- 172.21.15.45:0/1225243169 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9dbc108660 msgr2=0x7f9dbc19e950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:02:39.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:39.519+0000 7f9dc0c00640 1 --2- 172.21.15.45:0/1225243169 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9dbc108660 0x7f9dbc19e950 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f9dac000c00 tx=0x7f9dac02c650 comp rx=0 tx=0).stop 2024-04-16T17:02:39.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:39.519+0000 7f9dc0c00640 1 -- 172.21.15.45:0/1225243169 shutdown_connections 2024-04-16T17:02:39.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:39.519+0000 7f9dc0c00640 1 --2- 172.21.15.45:0/1225243169 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f9da00a3c30 0x7f9da00a60f0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:39.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:39.519+0000 7f9dc0c00640 1 --2- 172.21.15.45:0/1225243169 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9dbc109850 0x7f9dbc19ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:39.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:39.519+0000 7f9dc0c00640 1 --2- 172.21.15.45:0/1225243169 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9dbc10a210 0x7f9dbc1a5f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:39.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:39.519+0000 7f9dc0c00640 1 --2- 172.21.15.45:0/1225243169 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9dbc108660 0x7f9dbc19e950 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:39.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:39.519+0000 7f9dc0c00640 1 -- 172.21.15.45:0/1225243169 >> 172.21.15.45:0/1225243169 conn(0x7f9dbc077600 msgr2=0x7f9dbc10cb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:02:39.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:39.519+0000 7f9dc0c00640 1 -- 172.21.15.45:0/1225243169 shutdown_connections 2024-04-16T17:02:39.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:39.519+0000 7f9dc0c00640 1 -- 172.21.15.45:0/1225243169 wait complete. 2024-04-16T17:02:39.542 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-16T17:02:39.707 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:02:39 smithi117 ceph-mon[36730]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T17:02:39.707 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:02:39 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:02:39.707 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:02:39 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T17:02:39.707 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:02:39 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:02:39.707 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:02:39 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-16T17:02:39.707 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:02:39 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-16T17:02:39.707 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:02:39 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:02:39.707 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:02:39 smithi117 ceph-mon[36730]: Deploying daemon mds.l on smithi117 2024-04-16T17:02:39.708 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:02:39 smithi117 ceph-mon[36730]: from='client.? 172.21.15.45:0/2198935209' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T17:02:39.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:39.765+0000 7f71abe00640 1 -- 172.21.15.45:0/792841739 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f71ac1052b0 msgr2=0x7f71ac1076a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:02:39.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:39.765+0000 7f71abe00640 1 --2- 172.21.15.45:0/792841739 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f71ac1052b0 0x7f71ac1076a0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f7198009900 tx=0x7f719802bc50 comp rx=0 tx=0).stop 2024-04-16T17:02:39.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:39.766+0000 7f71abe00640 1 -- 172.21.15.45:0/792841739 shutdown_connections 2024-04-16T17:02:39.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:39.766+0000 7f71abe00640 1 --2- 172.21.15.45:0/792841739 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f71ac107be0 0x7f71ac109fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:39.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:39.766+0000 7f71abe00640 1 --2- 172.21.15.45:0/792841739 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f71ac1052b0 0x7f71ac1076a0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:39.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:39.766+0000 7f71abe00640 1 --2- 172.21.15.45:0/792841739 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f71ac102980 0x7f71ac104d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:39.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:39.766+0000 7f71abe00640 1 -- 172.21.15.45:0/792841739 >> 172.21.15.45:0/792841739 conn(0x7f71ac0fc2d0 msgr2=0x7f71ac0fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:02:39.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:39.766+0000 7f71abe00640 1 -- 172.21.15.45:0/792841739 shutdown_connections 2024-04-16T17:02:39.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:39.766+0000 7f71abe00640 1 -- 172.21.15.45:0/792841739 wait complete. 2024-04-16T17:02:39.769 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:39.767+0000 7f71abe00640 1 Processor -- start 2024-04-16T17:02:39.769 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:39.767+0000 7f71abe00640 1 -- start start 2024-04-16T17:02:39.769 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:39.767+0000 7f71abe00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f71ac102980 0x7f71ac19a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:02:39.769 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:39.767+0000 7f71abe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f71ac1052b0 0x7f71ac19a950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:02:39.769 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:39.767+0000 7f71abe00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f71ac107be0 0x7f71ac1a19c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:02:39.769 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:39.767+0000 7f71abe00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f71ac10db40 con 0x7f71ac1052b0 2024-04-16T17:02:39.769 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:39.767+0000 7f71abe00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f71ac10d9c0 con 0x7f71ac107be0 2024-04-16T17:02:39.769 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:39.767+0000 7f71abe00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f71ac10dcc0 con 0x7f71ac102980 2024-04-16T17:02:39.769 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:39.767+0000 7f71aaa00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f71ac102980 0x7f71ac19a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:02:39.769 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:39.767+0000 7f71aaa00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f71ac102980 0x7f71ac19a410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:43810/0 (socket says 172.21.15.45:43810) 2024-04-16T17:02:39.769 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:39.767+0000 7f71ab400640 1 --2- 172.21.15.45:0/2818970888 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f71ac107be0 0x7f71ac1a19c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:02:39.769 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:39.767+0000 7f71aa000640 1 --2- 172.21.15.45:0/2818970888 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f71ac1052b0 0x7f71ac19a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:02:39.769 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:39.767+0000 7f71aaa00640 1 -- 172.21.15.45:0/2818970888 learned_addr learned my addr 172.21.15.45:0/2818970888 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:02:39.770 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:39.768+0000 7f71aa000640 1 -- 172.21.15.45:0/2818970888 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f71ac102980 msgr2=0x7f71ac19a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:02:39.770 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:39.768+0000 7f71aa000640 1 --2- 172.21.15.45:0/2818970888 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f71ac102980 0x7f71ac19a410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:39.770 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:39.768+0000 7f71aa000640 1 -- 172.21.15.45:0/2818970888 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f71ac107be0 msgr2=0x7f71ac1a19c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:02:39.770 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:39.768+0000 7f71aa000640 1 --2- 172.21.15.45:0/2818970888 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f71ac107be0 0x7f71ac1a19c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:39.770 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:39.768+0000 7f71aa000640 1 -- 172.21.15.45:0/2818970888 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71ac1a2030 con 0x7f71ac1052b0 2024-04-16T17:02:39.770 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:39.768+0000 7f71ab400640 1 --2- 172.21.15.45:0/2818970888 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f71ac107be0 0x7f71ac1a19c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:02:39.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:39.768+0000 7f71aa000640 1 --2- 172.21.15.45:0/2818970888 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f71ac1052b0 0x7f71ac19a950 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f7198009230 tx=0x7f7198005dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:02:39.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:39.769+0000 7f71b0a00640 1 -- 172.21.15.45:0/2818970888 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7198041070 con 0x7f71ac1052b0 2024-04-16T17:02:39.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:39.769+0000 7f71abe00640 1 -- 172.21.15.45:0/2818970888 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71ac1a22c0 con 0x7f71ac1052b0 2024-04-16T17:02:39.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:39.769+0000 7f71b0a00640 1 -- 172.21.15.45:0/2818970888 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f71980054e0 con 0x7f71ac1052b0 2024-04-16T17:02:39.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:39.769+0000 7f71b0a00640 1 -- 172.21.15.45:0/2818970888 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7198034440 con 0x7f71ac1052b0 2024-04-16T17:02:39.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:39.769+0000 7f71abe00640 1 -- 172.21.15.45:0/2818970888 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71ac1a2830 con 0x7f71ac1052b0 2024-04-16T17:02:39.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:39.770+0000 7f71abe00640 1 -- 172.21.15.45:0/2818970888 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71ac1096d0 con 0x7f71ac1052b0 2024-04-16T17:02:39.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:39.772+0000 7f71b0a00640 1 -- 172.21.15.45:0/2818970888 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f7198044070 con 0x7f71ac1052b0 2024-04-16T17:02:39.774 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:39.773+0000 7f71b0a00640 1 --2- 172.21.15.45:0/2818970888 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f71900a3c30 0x7f71900a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:02:39.774 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:39.773+0000 7f71b0a00640 1 -- 172.21.15.45:0/2818970888 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f71980f1e90 con 0x7f71ac1052b0 2024-04-16T17:02:39.774 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:39.774+0000 7f71aaa00640 1 --2- 172.21.15.45:0/2818970888 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f71900a3c30 0x7f71900a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:02:39.775 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:39.774+0000 7f71aaa00640 1 --2- 172.21.15.45:0/2818970888 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f71900a3c30 0x7f71900a60f0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f719c009270 tx=0x7f719c009700 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:02:39.777 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:39.776+0000 7f71b0a00640 1 -- 172.21.15.45:0/2818970888 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f71980be620 con 0x7f71ac1052b0 2024-04-16T17:02:39.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:02:39 smithi104 ceph-mon[36457]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T17:02:39.929 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:02:39 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:02:39.929 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:02:39 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T17:02:39.929 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:02:39 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:02:39.929 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:02:39 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-16T17:02:39.929 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:02:39 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-16T17:02:39.929 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:02:39 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:02:39.929 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:02:39 smithi104 ceph-mon[36457]: Deploying daemon mds.l on smithi117 2024-04-16T17:02:39.929 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:02:39 smithi104 ceph-mon[36457]: from='client.? 172.21.15.45:0/2198935209' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T17:02:39.978 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:02:39 smithi045 ceph-mon[31676]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T17:02:39.978 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:02:39 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:02:39.979 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:02:39 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T17:02:39.979 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:02:39 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:02:39.979 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:02:39 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-16T17:02:39.979 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:02:39 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-16T17:02:39.979 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:02:39 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:02:39.979 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:02:39 smithi045 ceph-mon[31676]: Deploying daemon mds.l on smithi117 2024-04-16T17:02:39.979 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:02:39 smithi045 ceph-mon[31676]: from='client.? 172.21.15.45:0/2198935209' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T17:02:40.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:40.028+0000 7f71abe00640 1 -- 172.21.15.45:0/2818970888 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f71ac02d620 con 0x7f71ac1052b0 2024-04-16T17:02:40.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:40.028+0000 7f71b0a00640 1 -- 172.21.15.45:0/2818970888 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) ==== 93+0+569 (secure 0 0 0) 0x7f71980f1020 con 0x7f71ac1052b0 2024-04-16T17:02:40.030 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-16T17:02:40.030 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":1,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-04-16T17:02:40.030 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 1 2024-04-16T17:02:40.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:40.031+0000 7f71abe00640 1 -- 172.21.15.45:0/2818970888 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f71900a3c30 msgr2=0x7f71900a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:02:40.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:40.031+0000 7f71abe00640 1 --2- 172.21.15.45:0/2818970888 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f71900a3c30 0x7f71900a60f0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f719c009270 tx=0x7f719c009700 comp rx=0 tx=0).stop 2024-04-16T17:02:40.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:40.031+0000 7f71abe00640 1 -- 172.21.15.45:0/2818970888 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f71ac1052b0 msgr2=0x7f71ac19a950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:02:40.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:40.031+0000 7f71abe00640 1 --2- 172.21.15.45:0/2818970888 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f71ac1052b0 0x7f71ac19a950 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f7198009230 tx=0x7f7198005dd0 comp rx=0 tx=0).stop 2024-04-16T17:02:40.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:40.031+0000 7f71abe00640 1 -- 172.21.15.45:0/2818970888 shutdown_connections 2024-04-16T17:02:40.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:40.031+0000 7f71abe00640 1 --2- 172.21.15.45:0/2818970888 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f71900a3c30 0x7f71900a60f0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:40.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:40.031+0000 7f71abe00640 1 --2- 172.21.15.45:0/2818970888 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f71ac107be0 0x7f71ac1a19c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:40.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:40.031+0000 7f71abe00640 1 --2- 172.21.15.45:0/2818970888 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f71ac1052b0 0x7f71ac19a950 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:40.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:40.031+0000 7f71abe00640 1 --2- 172.21.15.45:0/2818970888 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f71ac102980 0x7f71ac19a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:40.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:40.031+0000 7f71abe00640 1 -- 172.21.15.45:0/2818970888 >> 172.21.15.45:0/2818970888 conn(0x7f71ac0fc2d0 msgr2=0x7f71ac102190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:02:40.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:40.032+0000 7f71abe00640 1 -- 172.21.15.45:0/2818970888 shutdown_connections 2024-04-16T17:02:40.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:40.032+0000 7f71abe00640 1 -- 172.21.15.45:0/2818970888 wait complete. 2024-04-16T17:02:40.788 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:02:40 smithi117 ceph-mon[36730]: from='client.? 172.21.15.45:0/1225243169' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-16T17:02:40.788 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:02:40 smithi117 ceph-mon[36730]: from='client.? 172.21.15.45:0/2818970888' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:02:40.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:02:40 smithi104 ceph-mon[36457]: from='client.? 172.21.15.45:0/1225243169' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-16T17:02:40.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:02:40 smithi104 ceph-mon[36457]: from='client.? 172.21.15.45:0/2818970888' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:02:40.978 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:02:40 smithi045 ceph-mon[31676]: from='client.? 172.21.15.45:0/1225243169' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-16T17:02:40.978 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:02:40 smithi045 ceph-mon[31676]: from='client.? 172.21.15.45:0/2818970888' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:02:41.056 INFO:journalctl@ceph.mds.l.smithi117.stdout:Apr 16 17:02:40 smithi117 systemd[1]: Starting Ceph mds.l for b1fecd38-fc11-11ee-bc90-c7b262605968... 2024-04-16T17:02:41.505 INFO:journalctl@ceph.mds.l.smithi117.stdout:Apr 16 17:02:41 smithi117 podman[67037]: 2024-04-16 17:02:41.092920614 +0000 UTC m=+0.017662446 image pull e5ce10bdffbeeffffd1125f67b369e2151f0f17cdec854e93b6adf9321f65907 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2 2024-04-16T17:02:41.505 INFO:journalctl@ceph.mds.l.smithi117.stdout:Apr 16 17:02:41 smithi117 podman[67037]: 2024-04-16 17:02:41.213751603 +0000 UTC m=+0.138493422 container create 888d9071f1761f69b31f6e2d66fdaa707f15a1c9447b8bcfd2af62059fa3c184 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mds-l, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240409, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-04-16T17:02:41.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:02:41 smithi117 ceph-mon[36730]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T17:02:41.806 INFO:journalctl@ceph.mds.l.smithi117.stdout:Apr 16 17:02:41 smithi117 podman[67037]: 2024-04-16 17:02:41.549990082 +0000 UTC m=+0.474731914 container init 888d9071f1761f69b31f6e2d66fdaa707f15a1c9447b8bcfd2af62059fa3c184 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mds-l, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240409) 2024-04-16T17:02:41.806 INFO:journalctl@ceph.mds.l.smithi117.stdout:Apr 16 17:02:41 smithi117 podman[67037]: 2024-04-16 17:02:41.552749079 +0000 UTC m=+0.477490897 container start 888d9071f1761f69b31f6e2d66fdaa707f15a1c9447b8bcfd2af62059fa3c184 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mds-l, org.label-schema.build-date=20240409, ceph=True, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-04-16T17:02:41.806 INFO:journalctl@ceph.mds.l.smithi117.stdout:Apr 16 17:02:41 smithi117 ceph-mds[67056]: -- 172.21.15.117:0/3412977117 <== mon.1 v2:172.21.15.117:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55a11c6c1a40 con 0x55a11c695000 2024-04-16T17:02:41.807 INFO:journalctl@ceph.mds.l.smithi117.stdout:Apr 16 17:02:41 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mds-l[67052]: starting mds.l at 2024-04-16T17:02:41.807 INFO:journalctl@ceph.mds.l.smithi117.stdout:Apr 16 17:02:41 smithi117 bash[67037]: 888d9071f1761f69b31f6e2d66fdaa707f15a1c9447b8bcfd2af62059fa3c184 2024-04-16T17:02:41.807 INFO:journalctl@ceph.mds.l.smithi117.stdout:Apr 16 17:02:41 smithi117 systemd[1]: Started Ceph mds.l for b1fecd38-fc11-11ee-bc90-c7b262605968. 2024-04-16T17:02:41.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:02:41 smithi104 ceph-mon[36457]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T17:02:41.978 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:02:41 smithi045 ceph-mon[31676]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T17:02:42.054 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-16T17:02:42.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:42.280+0000 7f3b91400640 1 -- 172.21.15.45:0/1482682614 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3b8c105e30 msgr2=0x7f3b8c10c6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:02:42.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:42.280+0000 7f3b91400640 1 --2- 172.21.15.45:0/1482682614 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3b8c105e30 0x7f3b8c10c6c0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f3b84009be0 tx=0x7f3b8402bef0 comp rx=0 tx=0).stop 2024-04-16T17:02:42.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:42.280+0000 7f3b91400640 1 -- 172.21.15.45:0/1482682614 shutdown_connections 2024-04-16T17:02:42.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:42.280+0000 7f3b91400640 1 --2- 172.21.15.45:0/1482682614 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3b8c105470 0x7f3b8c1058f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:42.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:42.280+0000 7f3b91400640 1 --2- 172.21.15.45:0/1482682614 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3b8c105e30 0x7f3b8c10c6c0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:42.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:42.280+0000 7f3b91400640 1 --2- 172.21.15.45:0/1482682614 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3b8c104280 0x7f3b8c1046a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:42.282 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:42.280+0000 7f3b91400640 1 -- 172.21.15.45:0/1482682614 >> 172.21.15.45:0/1482682614 conn(0x7f3b8c0ff7f0 msgr2=0x7f3b8c101c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:02:42.282 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:42.280+0000 7f3b91400640 1 -- 172.21.15.45:0/1482682614 shutdown_connections 2024-04-16T17:02:42.282 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:42.280+0000 7f3b91400640 1 -- 172.21.15.45:0/1482682614 wait complete. 2024-04-16T17:02:42.284 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:42.281+0000 7f3b91400640 1 Processor -- start 2024-04-16T17:02:42.284 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:42.281+0000 7f3b91400640 1 -- start start 2024-04-16T17:02:42.284 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:42.281+0000 7f3b91400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3b8c104280 0x7f3b8c19a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:02:42.284 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:42.281+0000 7f3b91400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3b8c105470 0x7f3b8c19aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:02:42.284 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:42.282+0000 7f3b91400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3b8c105e30 0x7f3b8c1a1a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:02:42.284 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:42.282+0000 7f3b91400640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f3b8c10ff30 con 0x7f3b8c105e30 2024-04-16T17:02:42.284 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:42.282+0000 7f3b91400640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f3b8c10fdb0 con 0x7f3b8c105470 2024-04-16T17:02:42.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:42.282+0000 7f3b91400640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f3b8c1100b0 con 0x7f3b8c104280 2024-04-16T17:02:42.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:42.282+0000 7f3b81e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3b8c105470 0x7f3b8c19aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:02:42.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:42.282+0000 7f3b8b200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3b8c105e30 0x7f3b8c1a1a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:02:42.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:42.282+0000 7f3b81e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3b8c105470 0x7f3b8c19aa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:36362/0 (socket says 172.21.15.45:36362) 2024-04-16T17:02:42.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:42.282+0000 7f3b81e00640 1 -- 172.21.15.45:0/1473108449 learned_addr learned my addr 172.21.15.45:0/1473108449 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:02:42.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:42.282+0000 7f3b8a800640 1 --2- 172.21.15.45:0/1473108449 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3b8c104280 0x7f3b8c19a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:02:42.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:42.282+0000 7f3b81e00640 1 -- 172.21.15.45:0/1473108449 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3b8c104280 msgr2=0x7f3b8c19a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:02:42.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:42.282+0000 7f3b81e00640 1 --2- 172.21.15.45:0/1473108449 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3b8c104280 0x7f3b8c19a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:42.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:42.282+0000 7f3b81e00640 1 -- 172.21.15.45:0/1473108449 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3b8c105e30 msgr2=0x7f3b8c1a1a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:02:42.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:42.282+0000 7f3b81e00640 1 --2- 172.21.15.45:0/1473108449 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3b8c105e30 0x7f3b8c1a1a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:42.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:42.282+0000 7f3b81e00640 1 -- 172.21.15.45:0/1473108449 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b8c1a2100 con 0x7f3b8c105470 2024-04-16T17:02:42.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:42.283+0000 7f3b81e00640 1 --2- 172.21.15.45:0/1473108449 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3b8c105470 0x7f3b8c19aa20 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f3b7400e9f0 tx=0x7f3b7400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:02:42.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:42.283+0000 7f3b83e00640 1 -- 172.21.15.45:0/1473108449 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b7400cd80 con 0x7f3b8c105470 2024-04-16T17:02:42.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:42.283+0000 7f3b83e00640 1 -- 172.21.15.45:0/1473108449 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3b74010910 con 0x7f3b8c105470 2024-04-16T17:02:42.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:42.283+0000 7f3b91400640 1 -- 172.21.15.45:0/1473108449 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b8c1a23f0 con 0x7f3b8c105470 2024-04-16T17:02:42.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:42.283+0000 7f3b83e00640 1 -- 172.21.15.45:0/1473108449 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b740188f0 con 0x7f3b8c105470 2024-04-16T17:02:42.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:42.283+0000 7f3b91400640 1 -- 172.21.15.45:0/1473108449 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b8c1a2940 con 0x7f3b8c105470 2024-04-16T17:02:42.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:42.284+0000 7f3b91400640 1 -- 172.21.15.45:0/1473108449 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b8c10b540 con 0x7f3b8c105470 2024-04-16T17:02:42.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:42.286+0000 7f3b83e00640 1 -- 172.21.15.45:0/1473108449 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f3b74010ac0 con 0x7f3b8c105470 2024-04-16T17:02:42.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:42.288+0000 7f3b83e00640 1 --2- 172.21.15.45:0/1473108449 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f3b700a3bb0 0x7f3b700a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:02:42.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:42.288+0000 7f3b83e00640 1 -- 172.21.15.45:0/1473108449 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f3b740d0390 con 0x7f3b8c105470 2024-04-16T17:02:42.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:42.288+0000 7f3b8a800640 1 --2- 172.21.15.45:0/1473108449 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f3b700a3bb0 0x7f3b700a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:02:42.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:42.289+0000 7f3b8a800640 1 --2- 172.21.15.45:0/1473108449 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f3b700a3bb0 0x7f3b700a6070 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f3b7c009530 tx=0x7f3b7c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:02:42.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:42.292+0000 7f3b83e00640 1 -- 172.21.15.45:0/1473108449 <== mon.2 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3b7409cb20 con 0x7f3b8c105470 2024-04-16T17:02:42.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:42.544+0000 7f3b91400640 1 -- 172.21.15.45:0/1473108449 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3b8c02d620 con 0x7f3b8c105470 2024-04-16T17:02:42.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:42.545+0000 7f3b83e00640 1 -- 172.21.15.45:0/1473108449 <== mon.2 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1396 (secure 0 0 0) 0x7f3b740a0e20 con 0x7f3b8c105470 2024-04-16T17:02:42.546 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-16T17:02:42.546 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/1768242489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1768242489},{"type":"v1","addr":"172.21.15.117:6833","nonce":1768242489}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-16T17:02:42.546 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 2 2024-04-16T17:02:42.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:42.547+0000 7f3b91400640 1 -- 172.21.15.45:0/1473108449 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f3b700a3bb0 msgr2=0x7f3b700a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:02:42.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:42.547+0000 7f3b91400640 1 --2- 172.21.15.45:0/1473108449 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f3b700a3bb0 0x7f3b700a6070 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f3b7c009530 tx=0x7f3b7c009260 comp rx=0 tx=0).stop 2024-04-16T17:02:42.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:42.547+0000 7f3b91400640 1 -- 172.21.15.45:0/1473108449 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3b8c105470 msgr2=0x7f3b8c19aa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:02:42.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:42.547+0000 7f3b91400640 1 --2- 172.21.15.45:0/1473108449 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3b8c105470 0x7f3b8c19aa20 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f3b7400e9f0 tx=0x7f3b7400eeb0 comp rx=0 tx=0).stop 2024-04-16T17:02:42.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:42.548+0000 7f3b91400640 1 -- 172.21.15.45:0/1473108449 shutdown_connections 2024-04-16T17:02:42.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:42.548+0000 7f3b91400640 1 --2- 172.21.15.45:0/1473108449 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f3b700a3bb0 0x7f3b700a6070 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:42.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:42.548+0000 7f3b91400640 1 --2- 172.21.15.45:0/1473108449 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3b8c105e30 0x7f3b8c1a1a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:42.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:42.548+0000 7f3b91400640 1 --2- 172.21.15.45:0/1473108449 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3b8c105470 0x7f3b8c19aa20 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:42.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:42.548+0000 7f3b91400640 1 --2- 172.21.15.45:0/1473108449 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3b8c104280 0x7f3b8c19a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:42.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:42.548+0000 7f3b91400640 1 -- 172.21.15.45:0/1473108449 >> 172.21.15.45:0/1473108449 conn(0x7f3b8c0ff7f0 msgr2=0x7f3b8c108710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:02:42.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:42.548+0000 7f3b91400640 1 -- 172.21.15.45:0/1473108449 shutdown_connections 2024-04-16T17:02:42.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:42.548+0000 7f3b91400640 1 -- 172.21.15.45:0/1473108449 wait complete. 2024-04-16T17:02:42.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:02:42 smithi104 ceph-mon[36457]: mds.? [v2:172.21.15.117:6832/1768242489,v1:172.21.15.117:6833/1768242489] up:boot 2024-04-16T17:02:42.679 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:02:42 smithi104 ceph-mon[36457]: fsmap 1 up:standby 2024-04-16T17:02:42.679 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:02:42 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-16T17:02:42.679 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:02:42 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:02:42.679 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:02:42 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:02:42.679 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:02:42 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:02:42.679 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:02:42 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-16T17:02:42.679 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:02:42 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-16T17:02:42.679 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:02:42 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:02:42.679 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:02:42 smithi104 ceph-mon[36457]: Deploying daemon mds.k on smithi104 2024-04-16T17:02:42.679 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:02:42 smithi104 ceph-mon[36457]: from='client.? 172.21.15.45:0/1473108449' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:02:42.979 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:02:42 smithi045 ceph-mon[31676]: mds.? [v2:172.21.15.117:6832/1768242489,v1:172.21.15.117:6833/1768242489] up:boot 2024-04-16T17:02:42.979 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:02:42 smithi045 ceph-mon[31676]: fsmap 1 up:standby 2024-04-16T17:02:42.979 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:02:42 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-16T17:02:42.979 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:02:42 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:02:42.979 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:02:42 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:02:42.980 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:02:42 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:02:42.980 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:02:42 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-16T17:02:42.980 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:02:42 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-16T17:02:42.980 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:02:42 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:02:42.980 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:02:42 smithi045 ceph-mon[31676]: Deploying daemon mds.k on smithi104 2024-04-16T17:02:42.980 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:02:42 smithi045 ceph-mon[31676]: from='client.? 172.21.15.45:0/1473108449' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:02:43.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:02:42 smithi117 ceph-mon[36730]: mds.? [v2:172.21.15.117:6832/1768242489,v1:172.21.15.117:6833/1768242489] up:boot 2024-04-16T17:02:43.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:02:42 smithi117 ceph-mon[36730]: fsmap 1 up:standby 2024-04-16T17:02:43.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:02:42 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-16T17:02:43.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:02:42 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:02:43.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:02:42 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:02:43.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:02:42 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:02:43.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:02:42 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-16T17:02:43.057 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:02:42 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-16T17:02:43.057 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:02:42 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:02:43.057 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:02:42 smithi117 ceph-mon[36730]: Deploying daemon mds.k on smithi104 2024-04-16T17:02:43.057 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:02:42 smithi117 ceph-mon[36730]: from='client.? 172.21.15.45:0/1473108449' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:02:43.860 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:02:43 smithi104 ceph-mon[36457]: pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T17:02:44.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:02:43 smithi117 ceph-mon[36730]: pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T17:02:44.178 INFO:journalctl@ceph.mds.k.smithi104.stdout:Apr 16 17:02:44 smithi104 systemd[1]: Starting Ceph mds.k for b1fecd38-fc11-11ee-bc90-c7b262605968... 2024-04-16T17:02:44.228 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:02:43 smithi045 ceph-mon[31676]: pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T17:02:44.511 INFO:journalctl@ceph.mds.k.smithi104.stdout:Apr 16 17:02:44 smithi104 podman[66830]: 2024-04-16 17:02:44.16128604 +0000 UTC m=+0.017645737 image pull e5ce10bdffbeeffffd1125f67b369e2151f0f17cdec854e93b6adf9321f65907 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2 2024-04-16T17:02:44.511 INFO:journalctl@ceph.mds.k.smithi104.stdout:Apr 16 17:02:44 smithi104 podman[66830]: 2024-04-16 17:02:44.274311753 +0000 UTC m=+0.130671465 container create c06d93d410ffc0ac248cc71d43f501f82405fade8d5b13d9ff1b8bbcf7678129 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mds-k, GIT_CLEAN=True, ceph=True, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240409, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-04-16T17:02:44.569 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-16T17:02:44.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:44.804+0000 7f760e200640 1 -- 172.21.15.45:0/2262730272 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7608103c00 msgr2=0x7f760810a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:02:44.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:44.804+0000 7f760e200640 1 --2- 172.21.15.45:0/2262730272 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7608103c00 0x7f760810a490 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f7600009be0 tx=0x7f760002bef0 comp rx=0 tx=0).stop 2024-04-16T17:02:44.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:44.804+0000 7f760e200640 1 -- 172.21.15.45:0/2262730272 shutdown_connections 2024-04-16T17:02:44.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:44.804+0000 7f760e200640 1 --2- 172.21.15.45:0/2262730272 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7608103c00 0x7f760810a490 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:44.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:44.804+0000 7f760e200640 1 --2- 172.21.15.45:0/2262730272 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7608103240 0x7f76081036c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:44.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:44.804+0000 7f760e200640 1 --2- 172.21.15.45:0/2262730272 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7608102050 0x7f7608102470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:44.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:44.804+0000 7f760e200640 1 -- 172.21.15.45:0/2262730272 >> 172.21.15.45:0/2262730272 conn(0x7f76080fd5e0 msgr2=0x7f76080ffa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:02:44.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:44.804+0000 7f760e200640 1 -- 172.21.15.45:0/2262730272 shutdown_connections 2024-04-16T17:02:44.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:44.804+0000 7f760e200640 1 -- 172.21.15.45:0/2262730272 wait complete. 2024-04-16T17:02:44.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:44.805+0000 7f760e200640 1 Processor -- start 2024-04-16T17:02:44.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:44.805+0000 7f760e200640 1 -- start start 2024-04-16T17:02:44.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:44.806+0000 7f760e200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7608102050 0x7f7608075db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:02:44.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:44.806+0000 7f760e200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7608103240 0x7f76080743f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:02:44.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:44.806+0000 7f760e200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7608103c00 0x7f7608074930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:02:44.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:44.806+0000 7f760e200640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f760810dce0 con 0x7f7608102050 2024-04-16T17:02:44.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:44.806+0000 7f760e200640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f760810db60 con 0x7f7608103c00 2024-04-16T17:02:44.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:44.806+0000 7f760e200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f760810de60 con 0x7f7608103240 2024-04-16T17:02:44.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:44.806+0000 7f7607400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7608102050 0x7f7608075db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:02:44.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:44.806+0000 7f75ffe00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7608103240 0x7f76080743f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:02:44.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:44.806+0000 7f7607400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7608102050 0x7f7608075db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39514/0 (socket says 172.21.15.45:39514) 2024-04-16T17:02:44.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:44.806+0000 7f7607400640 1 -- 172.21.15.45:0/4183725948 learned_addr learned my addr 172.21.15.45:0/4183725948 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:02:44.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:44.806+0000 7f7607e00640 1 --2- 172.21.15.45:0/4183725948 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7608103c00 0x7f7608074930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:02:44.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:44.807+0000 7f7607e00640 1 -- 172.21.15.45:0/4183725948 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7608103240 msgr2=0x7f76080743f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:02:44.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:44.807+0000 7f7607e00640 1 --2- 172.21.15.45:0/4183725948 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7608103240 0x7f76080743f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:44.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:44.807+0000 7f7607e00640 1 -- 172.21.15.45:0/4183725948 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7608102050 msgr2=0x7f7608075db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:02:44.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:44.807+0000 7f7607e00640 1 --2- 172.21.15.45:0/4183725948 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7608102050 0x7f7608075db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:44.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:44.807+0000 7f7607e00640 1 -- 172.21.15.45:0/4183725948 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76080751c0 con 0x7f7608103c00 2024-04-16T17:02:44.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:44.807+0000 7f7607400640 1 --2- 172.21.15.45:0/4183725948 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7608102050 0x7f7608075db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:02:44.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:44.807+0000 7f7607e00640 1 --2- 172.21.15.45:0/4183725948 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7608103c00 0x7f7608074930 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f760002c430 tx=0x7f760002cb90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:02:44.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:44.807+0000 7f7604c00640 1 -- 172.21.15.45:0/4183725948 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7600043020 con 0x7f7608103c00 2024-04-16T17:02:44.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:44.807+0000 7f7604c00640 1 -- 172.21.15.45:0/4183725948 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7600003e50 con 0x7f7608103c00 2024-04-16T17:02:44.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:44.807+0000 7f7604c00640 1 -- 172.21.15.45:0/4183725948 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7600034430 con 0x7f7608103c00 2024-04-16T17:02:44.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:44.808+0000 7f760e200640 1 -- 172.21.15.45:0/4183725948 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76081a23f0 con 0x7f7608103c00 2024-04-16T17:02:44.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:44.808+0000 7f760e200640 1 -- 172.21.15.45:0/4183725948 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76081a28f0 con 0x7f7608103c00 2024-04-16T17:02:44.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:44.809+0000 7f760e200640 1 -- 172.21.15.45:0/4183725948 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7608109370 con 0x7f7608103c00 2024-04-16T17:02:44.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:44.811+0000 7f7604c00640 1 -- 172.21.15.45:0/4183725948 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f7600046020 con 0x7f7608103c00 2024-04-16T17:02:44.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:44.812+0000 7f7604c00640 1 --2- 172.21.15.45:0/4183725948 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f75ec0a3c30 0x7f75ec0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:02:44.814 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:44.813+0000 7f7604c00640 1 -- 172.21.15.45:0/4183725948 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f76000f24c0 con 0x7f7608103c00 2024-04-16T17:02:44.814 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:44.813+0000 7f7607400640 1 --2- 172.21.15.45:0/4183725948 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f75ec0a3c30 0x7f75ec0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:02:44.814 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:44.813+0000 7f7607400640 1 --2- 172.21.15.45:0/4183725948 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f75ec0a3c30 0x7f75ec0a60f0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f75f80058c0 tx=0x7f75f8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:02:44.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:44.816+0000 7f7604c00640 1 -- 172.21.15.45:0/4183725948 <== mon.2 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f760004d020 con 0x7f7608103c00 2024-04-16T17:02:44.928 INFO:journalctl@ceph.mds.k.smithi104.stdout:Apr 16 17:02:44 smithi104 podman[66830]: 2024-04-16 17:02:44.510193985 +0000 UTC m=+0.366553678 container init c06d93d410ffc0ac248cc71d43f501f82405fade8d5b13d9ff1b8bbcf7678129 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mds-k, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240409, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-04-16T17:02:44.929 INFO:journalctl@ceph.mds.k.smithi104.stdout:Apr 16 17:02:44 smithi104 podman[66830]: 2024-04-16 17:02:44.512834486 +0000 UTC m=+0.369194179 container start c06d93d410ffc0ac248cc71d43f501f82405fade8d5b13d9ff1b8bbcf7678129 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mds-k, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240409, org.label-schema.license=GPLv2) 2024-04-16T17:02:44.929 INFO:journalctl@ceph.mds.k.smithi104.stdout:Apr 16 17:02:44 smithi104 ceph-mds[66849]: -- 172.21.15.104:0/2216092063 <== mon.2 v2:172.21.15.104:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56133c03ba40 con 0x56133c00e800 2024-04-16T17:02:44.929 INFO:journalctl@ceph.mds.k.smithi104.stdout:Apr 16 17:02:44 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mds-k[66845]: starting mds.k at 2024-04-16T17:02:44.929 INFO:journalctl@ceph.mds.k.smithi104.stdout:Apr 16 17:02:44 smithi104 bash[66830]: c06d93d410ffc0ac248cc71d43f501f82405fade8d5b13d9ff1b8bbcf7678129 2024-04-16T17:02:44.929 INFO:journalctl@ceph.mds.k.smithi104.stdout:Apr 16 17:02:44 smithi104 systemd[1]: Started Ceph mds.k for b1fecd38-fc11-11ee-bc90-c7b262605968. 2024-04-16T17:02:45.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:45.072+0000 7f760e200640 1 -- 172.21.15.45:0/4183725948 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7608062c00 con 0x7f7608103c00 2024-04-16T17:02:45.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:45.073+0000 7f7604c00640 1 -- 172.21.15.45:0/4183725948 <== mon.2 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2224 (secure 0 0 0) 0x7f76000bf2a0 con 0x7f7608103c00 2024-04-16T17:02:45.075 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-16T17:02:45.075 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/1768242489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1768242489},{"type":"v1","addr":"172.21.15.117:6833","nonce":1768242489}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6833/3558902585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":3558902585},{"type":"v1","addr":"172.21.15.104:6833","nonce":3558902585}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-04-16T17:02:45.075 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 3 2024-04-16T17:02:45.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:45.077+0000 7f760e200640 1 -- 172.21.15.45:0/4183725948 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f75ec0a3c30 msgr2=0x7f75ec0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:02:45.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:45.077+0000 7f760e200640 1 --2- 172.21.15.45:0/4183725948 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f75ec0a3c30 0x7f75ec0a60f0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f75f80058c0 tx=0x7f75f8005f90 comp rx=0 tx=0).stop 2024-04-16T17:02:45.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:45.077+0000 7f760e200640 1 -- 172.21.15.45:0/4183725948 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7608103c00 msgr2=0x7f7608074930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:02:45.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:45.077+0000 7f760e200640 1 --2- 172.21.15.45:0/4183725948 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7608103c00 0x7f7608074930 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f760002c430 tx=0x7f760002cb90 comp rx=0 tx=0).stop 2024-04-16T17:02:45.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:45.077+0000 7f760e200640 1 -- 172.21.15.45:0/4183725948 shutdown_connections 2024-04-16T17:02:45.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:45.077+0000 7f760e200640 1 --2- 172.21.15.45:0/4183725948 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f75ec0a3c30 0x7f75ec0a60f0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:45.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:45.077+0000 7f760e200640 1 --2- 172.21.15.45:0/4183725948 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7608103240 0x7f76080743f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:45.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:45.077+0000 7f760e200640 1 --2- 172.21.15.45:0/4183725948 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7608103c00 0x7f7608074930 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:45.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:45.077+0000 7f760e200640 1 --2- 172.21.15.45:0/4183725948 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7608102050 0x7f7608075db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:45.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:45.077+0000 7f760e200640 1 -- 172.21.15.45:0/4183725948 >> 172.21.15.45:0/4183725948 conn(0x7f76080fd5e0 msgr2=0x7f7608106540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:02:45.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:45.078+0000 7f760e200640 1 -- 172.21.15.45:0/4183725948 shutdown_connections 2024-04-16T17:02:45.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:45.078+0000 7f760e200640 1 -- 172.21.15.45:0/4183725948 wait complete. 2024-04-16T17:02:45.939 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:02:45 smithi045 ceph-mon[31676]: pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T17:02:45.940 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:02:45 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:02:45.940 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:02:45 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:02:45.940 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:02:45 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:02:45.940 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:02:45 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-16T17:02:45.940 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:02:45 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-16T17:02:45.940 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:02:45 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:02:45.940 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:02:45 smithi045 ceph-mon[31676]: Deploying daemon mds.j on smithi045 2024-04-16T17:02:45.940 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:02:45 smithi045 ceph-mon[31676]: mds.? [v2:172.21.15.104:6832/3558902585,v1:172.21.15.104:6833/3558902585] up:boot 2024-04-16T17:02:45.940 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:02:45 smithi045 ceph-mon[31676]: fsmap 2 up:standby 2024-04-16T17:02:45.940 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:02:45 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-16T17:02:45.940 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:02:45 smithi045 ceph-mon[31676]: from='client.? 172.21.15.45:0/4183725948' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:02:46.058 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:02:45 smithi117 ceph-mon[36730]: pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T17:02:46.058 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:02:45 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:02:46.058 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:02:45 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:02:46.058 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:02:45 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:02:46.058 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:02:45 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-16T17:02:46.058 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:02:45 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-16T17:02:46.058 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:02:45 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:02:46.058 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:02:45 smithi117 ceph-mon[36730]: Deploying daemon mds.j on smithi045 2024-04-16T17:02:46.059 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:02:45 smithi117 ceph-mon[36730]: mds.? [v2:172.21.15.104:6832/3558902585,v1:172.21.15.104:6833/3558902585] up:boot 2024-04-16T17:02:46.059 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:02:45 smithi117 ceph-mon[36730]: fsmap 2 up:standby 2024-04-16T17:02:46.059 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:02:45 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-16T17:02:46.059 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:02:45 smithi117 ceph-mon[36730]: from='client.? 172.21.15.45:0/4183725948' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:02:46.178 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:02:45 smithi104 ceph-mon[36457]: pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T17:02:46.179 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:02:45 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:02:46.179 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:02:45 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:02:46.179 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:02:45 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:02:46.179 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:02:45 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-16T17:02:46.179 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:02:45 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-16T17:02:46.179 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:02:45 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:02:46.179 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:02:45 smithi104 ceph-mon[36457]: Deploying daemon mds.j on smithi045 2024-04-16T17:02:46.179 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:02:45 smithi104 ceph-mon[36457]: mds.? [v2:172.21.15.104:6832/3558902585,v1:172.21.15.104:6833/3558902585] up:boot 2024-04-16T17:02:46.179 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:02:45 smithi104 ceph-mon[36457]: fsmap 2 up:standby 2024-04-16T17:02:46.179 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:02:45 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-16T17:02:46.179 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:02:45 smithi104 ceph-mon[36457]: from='client.? 172.21.15.45:0/4183725948' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:02:47.098 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-16T17:02:47.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:47.352+0000 7f2a2e200640 1 -- 172.21.15.45:0/1204495499 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2a28102900 msgr2=0x7f2a28104cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:02:47.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:47.352+0000 7f2a2e200640 1 --2- 172.21.15.45:0/1204495499 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2a28102900 0x7f2a28104cf0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f2a1c009900 tx=0x7f2a1c02bc50 comp rx=0 tx=0).stop 2024-04-16T17:02:47.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:47.352+0000 7f2a2e200640 1 -- 172.21.15.45:0/1204495499 shutdown_connections 2024-04-16T17:02:47.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:47.352+0000 7f2a2e200640 1 --2- 172.21.15.45:0/1204495499 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2a28107b60 0x7f2a28109fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:47.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:47.352+0000 7f2a2e200640 1 --2- 172.21.15.45:0/1204495499 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2a28105230 0x7f2a28107620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:47.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:47.352+0000 7f2a2e200640 1 --2- 172.21.15.45:0/1204495499 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2a28102900 0x7f2a28104cf0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:47.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:47.352+0000 7f2a2e200640 1 -- 172.21.15.45:0/1204495499 >> 172.21.15.45:0/1204495499 conn(0x7f2a280fc2d0 msgr2=0x7f2a280fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:02:47.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:47.352+0000 7f2a2e200640 1 -- 172.21.15.45:0/1204495499 shutdown_connections 2024-04-16T17:02:47.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:47.352+0000 7f2a2e200640 1 -- 172.21.15.45:0/1204495499 wait complete. 2024-04-16T17:02:47.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:47.354+0000 7f2a2e200640 1 Processor -- start 2024-04-16T17:02:47.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:47.354+0000 7f2a2e200640 1 -- start start 2024-04-16T17:02:47.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:47.354+0000 7f2a2e200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2a28102900 0x7f2a281981b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:02:47.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:47.354+0000 7f2a2e200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2a28105230 0x7f2a281986f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:02:47.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:47.354+0000 7f2a2e200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2a28107b60 0x7f2a2819f760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:02:47.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:47.354+0000 7f2a2e200640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f2a2810db70 con 0x7f2a28105230 2024-04-16T17:02:47.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:47.354+0000 7f2a2e200640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f2a2810d9f0 con 0x7f2a28102900 2024-04-16T17:02:47.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:47.354+0000 7f2a2e200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f2a2810dcf0 con 0x7f2a28107b60 2024-04-16T17:02:47.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:47.355+0000 7f2a27e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2a28105230 0x7f2a281986f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:02:47.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:47.355+0000 7f2a2ce00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2a28102900 0x7f2a281981b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:02:47.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:47.355+0000 7f2a27e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2a28105230 0x7f2a281986f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39534/0 (socket says 172.21.15.45:39534) 2024-04-16T17:02:47.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:47.355+0000 7f2a27e00640 1 -- 172.21.15.45:0/2299376003 learned_addr learned my addr 172.21.15.45:0/2299376003 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:02:47.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:47.355+0000 7f2a2d800640 1 --2- 172.21.15.45:0/2299376003 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2a28107b60 0x7f2a2819f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:02:47.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:47.355+0000 7f2a27e00640 1 -- 172.21.15.45:0/2299376003 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2a28107b60 msgr2=0x7f2a2819f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:02:47.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:47.355+0000 7f2a27e00640 1 --2- 172.21.15.45:0/2299376003 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2a28107b60 0x7f2a2819f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:47.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:47.355+0000 7f2a27e00640 1 -- 172.21.15.45:0/2299376003 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2a28102900 msgr2=0x7f2a281981b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:02:47.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:47.355+0000 7f2a27e00640 1 --2- 172.21.15.45:0/2299376003 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2a28102900 0x7f2a281981b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:47.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:47.355+0000 7f2a27e00640 1 -- 172.21.15.45:0/2299376003 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a2819fdd0 con 0x7f2a28105230 2024-04-16T17:02:47.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:47.355+0000 7f2a2ce00640 1 --2- 172.21.15.45:0/2299376003 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2a28102900 0x7f2a281981b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:02:47.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:47.355+0000 7f2a27e00640 1 --2- 172.21.15.45:0/2299376003 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2a28105230 0x7f2a281986f0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f2a1400e9c0 tx=0x7f2a1400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:02:47.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:47.356+0000 7f2a25600640 1 -- 172.21.15.45:0/2299376003 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a1400cd80 con 0x7f2a28105230 2024-04-16T17:02:47.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:47.356+0000 7f2a2e200640 1 -- 172.21.15.45:0/2299376003 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a281a00c0 con 0x7f2a28105230 2024-04-16T17:02:47.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:47.356+0000 7f2a25600640 1 -- 172.21.15.45:0/2299376003 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2a14005500 con 0x7f2a28105230 2024-04-16T17:02:47.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:47.356+0000 7f2a25600640 1 -- 172.21.15.45:0/2299376003 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a14010690 con 0x7f2a28105230 2024-04-16T17:02:47.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:47.356+0000 7f2a2e200640 1 -- 172.21.15.45:0/2299376003 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a281a0610 con 0x7f2a28105230 2024-04-16T17:02:47.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:47.357+0000 7f2a2e200640 1 -- 172.21.15.45:0/2299376003 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a280fddc0 con 0x7f2a28105230 2024-04-16T17:02:47.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:47.360+0000 7f2a25600640 1 -- 172.21.15.45:0/2299376003 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f2a140046c0 con 0x7f2a28105230 2024-04-16T17:02:47.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:47.362+0000 7f2a25600640 1 --2- 172.21.15.45:0/2299376003 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f2a100a3bb0 0x7f2a100a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:02:47.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:47.363+0000 7f2a25600640 1 -- 172.21.15.45:0/2299376003 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f2a140d0360 con 0x7f2a28105230 2024-04-16T17:02:47.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:47.363+0000 7f2a2ce00640 1 --2- 172.21.15.45:0/2299376003 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f2a100a3bb0 0x7f2a100a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:02:47.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:47.364+0000 7f2a2ce00640 1 --2- 172.21.15.45:0/2299376003 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f2a100a3bb0 0x7f2a100a6070 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f2a1c02c260 tx=0x7f2a1c000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:02:47.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:47.367+0000 7f2a25600640 1 -- 172.21.15.45:0/2299376003 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2a1409d140 con 0x7f2a28105230 2024-04-16T17:02:47.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:47.626+0000 7f2a2e200640 1 -- 172.21.15.45:0/2299376003 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2a2802d620 con 0x7f2a28105230 2024-04-16T17:02:47.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:47.628+0000 7f2a25600640 1 -- 172.21.15.45:0/2299376003 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2224 (secure 0 0 0) 0x7f2a1409ca40 con 0x7f2a28105230 2024-04-16T17:02:47.633 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-16T17:02:47.633 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/1768242489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1768242489},{"type":"v1","addr":"172.21.15.117:6833","nonce":1768242489}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6833/3558902585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":3558902585},{"type":"v1","addr":"172.21.15.104:6833","nonce":3558902585}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-04-16T17:02:47.633 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 3 2024-04-16T17:02:47.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:47.632+0000 7f2a2e200640 1 -- 172.21.15.45:0/2299376003 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f2a100a3bb0 msgr2=0x7f2a100a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:02:47.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:47.632+0000 7f2a2e200640 1 --2- 172.21.15.45:0/2299376003 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f2a100a3bb0 0x7f2a100a6070 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f2a1c02c260 tx=0x7f2a1c000bc0 comp rx=0 tx=0).stop 2024-04-16T17:02:47.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:47.632+0000 7f2a2e200640 1 -- 172.21.15.45:0/2299376003 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2a28105230 msgr2=0x7f2a281986f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:02:47.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:47.632+0000 7f2a2e200640 1 --2- 172.21.15.45:0/2299376003 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2a28105230 0x7f2a281986f0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f2a1400e9c0 tx=0x7f2a1400ee80 comp rx=0 tx=0).stop 2024-04-16T17:02:47.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:47.632+0000 7f2a2e200640 1 -- 172.21.15.45:0/2299376003 shutdown_connections 2024-04-16T17:02:47.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:47.633+0000 7f2a2e200640 1 --2- 172.21.15.45:0/2299376003 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f2a100a3bb0 0x7f2a100a6070 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:47.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:47.633+0000 7f2a2e200640 1 --2- 172.21.15.45:0/2299376003 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2a28107b60 0x7f2a2819f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:47.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:47.633+0000 7f2a2e200640 1 --2- 172.21.15.45:0/2299376003 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2a28105230 0x7f2a281986f0 secure :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f2a1400e9c0 tx=0x7f2a1400ee80 comp rx=0 tx=0).stop 2024-04-16T17:02:47.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:47.633+0000 7f2a2e200640 1 --2- 172.21.15.45:0/2299376003 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2a28102900 0x7f2a281981b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:47.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:47.633+0000 7f2a2e200640 1 -- 172.21.15.45:0/2299376003 >> 172.21.15.45:0/2299376003 conn(0x7f2a280fc2d0 msgr2=0x7f2a28103320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:02:47.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:47.636+0000 7f2a2e200640 1 -- 172.21.15.45:0/2299376003 shutdown_connections 2024-04-16T17:02:47.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:47.636+0000 7f2a2e200640 1 -- 172.21.15.45:0/2299376003 wait complete. 2024-04-16T17:02:47.866 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:02:47 smithi045 ceph-mon[31676]: pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T17:02:47.866 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:02:47 smithi045 ceph-mon[31676]: from='client.? 172.21.15.45:0/2299376003' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:02:48.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:02:47 smithi117 ceph-mon[36730]: pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T17:02:48.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:02:47 smithi117 ceph-mon[36730]: from='client.? 172.21.15.45:0/2299376003' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:02:48.178 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:02:47 smithi104 ceph-mon[36457]: pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T17:02:48.179 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:02:47 smithi104 ceph-mon[36457]: from='client.? 172.21.15.45:0/2299376003' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:02:48.229 INFO:journalctl@ceph.mds.j.smithi045.stdout:Apr 16 17:02:47 smithi045 systemd[1]: Starting Ceph mds.j for b1fecd38-fc11-11ee-bc90-c7b262605968... 2024-04-16T17:02:48.564 INFO:journalctl@ceph.mds.j.smithi045.stdout:Apr 16 17:02:48 smithi045 podman[72000]: 2024-04-16 17:02:48.131866547 +0000 UTC m=+0.032524382 image pull e5ce10bdffbeeffffd1125f67b369e2151f0f17cdec854e93b6adf9321f65907 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2 2024-04-16T17:02:48.564 INFO:journalctl@ceph.mds.j.smithi045.stdout:Apr 16 17:02:48 smithi045 podman[72000]: 2024-04-16 17:02:48.378750981 +0000 UTC m=+0.279408841 container create f9a5b8751be267ac3b34bf014287140eda4b857862f23800aff7207d6d31d163 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mds-j, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.build-date=20240409, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb) 2024-04-16T17:02:48.979 INFO:journalctl@ceph.mds.j.smithi045.stdout:Apr 16 17:02:48 smithi045 podman[72000]: 2024-04-16 17:02:48.643912166 +0000 UTC m=+0.544570033 container init f9a5b8751be267ac3b34bf014287140eda4b857862f23800aff7207d6d31d163 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mds-j, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, org.label-schema.build-date=20240409, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-04-16T17:02:48.979 INFO:journalctl@ceph.mds.j.smithi045.stdout:Apr 16 17:02:48 smithi045 podman[72000]: 2024-04-16 17:02:48.654635705 +0000 UTC m=+0.555293558 container start f9a5b8751be267ac3b34bf014287140eda4b857862f23800aff7207d6d31d163 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mds-j, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240409, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, io.buildah.version=1.33.5) 2024-04-16T17:02:48.979 INFO:journalctl@ceph.mds.j.smithi045.stdout:Apr 16 17:02:48 smithi045 ceph-mds[72020]: -- 172.21.15.45:0/55807433 <== mon.0 v2:172.21.15.45:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x559e07193a40 con 0x559e07166800 2024-04-16T17:02:48.979 INFO:journalctl@ceph.mds.j.smithi045.stdout:Apr 16 17:02:48 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mds-j[72016]: starting mds.j at 2024-04-16T17:02:48.979 INFO:journalctl@ceph.mds.j.smithi045.stdout:Apr 16 17:02:48 smithi045 bash[72000]: f9a5b8751be267ac3b34bf014287140eda4b857862f23800aff7207d6d31d163 2024-04-16T17:02:48.979 INFO:journalctl@ceph.mds.j.smithi045.stdout:Apr 16 17:02:48 smithi045 systemd[1]: Started Ceph mds.j for b1fecd38-fc11-11ee-bc90-c7b262605968. 2024-04-16T17:02:49.666 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-16T17:02:49.869 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:49.866+0000 7f87cd200640 1 -- 172.21.15.45:0/1095186061 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f87c81019f0 msgr2=0x7f87c8101e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:02:49.869 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:49.866+0000 7f87cd200640 1 --2- 172.21.15.45:0/1095186061 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f87c81019f0 0x7f87c8101e70 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f87c0009ca0 tx=0x7f87c002c0f0 comp rx=0 tx=0).stop 2024-04-16T17:02:49.869 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:49.866+0000 7f87cd200640 1 -- 172.21.15.45:0/1095186061 shutdown_connections 2024-04-16T17:02:49.869 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:49.866+0000 7f87cd200640 1 --2- 172.21.15.45:0/1095186061 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f87c81019f0 0x7f87c8101e70 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:49.869 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:49.866+0000 7f87cd200640 1 --2- 172.21.15.45:0/1095186061 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f87c8101050 0x7f87c81014b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:49.869 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:49.866+0000 7f87cd200640 1 --2- 172.21.15.45:0/1095186061 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f87c81006f0 0x7f87c8100b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:49.869 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:49.866+0000 7f87cd200640 1 -- 172.21.15.45:0/1095186061 >> 172.21.15.45:0/1095186061 conn(0x7f87c80fc2d0 msgr2=0x7f87c80fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:02:49.869 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:49.866+0000 7f87cd200640 1 -- 172.21.15.45:0/1095186061 shutdown_connections 2024-04-16T17:02:49.869 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:49.867+0000 7f87cd200640 1 -- 172.21.15.45:0/1095186061 wait complete. 2024-04-16T17:02:49.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:49.867+0000 7f87cd200640 1 Processor -- start 2024-04-16T17:02:49.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:49.868+0000 7f87cd200640 1 -- start start 2024-04-16T17:02:49.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:49.868+0000 7f87cd200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f87c81006f0 0x7f87c819a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:02:49.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:49.868+0000 7f87cd200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f87c8101050 0x7f87c819a860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:02:49.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:49.868+0000 7f87cd200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f87c81019f0 0x7f87c81a18d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:02:49.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:49.868+0000 7f87cd200640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f87c810fd20 con 0x7f87c81019f0 2024-04-16T17:02:49.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:49.868+0000 7f87cd200640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f87c810fba0 con 0x7f87c81006f0 2024-04-16T17:02:49.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:49.868+0000 7f87cd200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f87c810fea0 con 0x7f87c8101050 2024-04-16T17:02:49.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:49.868+0000 7f87c7e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f87c81019f0 0x7f87c81a18d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:02:49.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:49.868+0000 7f87c7400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f87c81006f0 0x7f87c819a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:02:49.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:49.868+0000 7f87c6a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f87c8101050 0x7f87c819a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:02:49.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:49.868+0000 7f87c7e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f87c81019f0 0x7f87c81a18d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50132/0 (socket says 172.21.15.45:50132) 2024-04-16T17:02:49.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:49.868+0000 7f87c7e00640 1 -- 172.21.15.45:0/4143271409 learned_addr learned my addr 172.21.15.45:0/4143271409 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:02:49.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:49.869+0000 7f87c7e00640 1 -- 172.21.15.45:0/4143271409 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f87c8101050 msgr2=0x7f87c819a860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:02:49.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:49.869+0000 7f87c7e00640 1 --2- 172.21.15.45:0/4143271409 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f87c8101050 0x7f87c819a860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:49.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:49.869+0000 7f87c7e00640 1 -- 172.21.15.45:0/4143271409 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f87c81006f0 msgr2=0x7f87c819a320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:02:49.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:49.869+0000 7f87c7e00640 1 --2- 172.21.15.45:0/4143271409 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f87c81006f0 0x7f87c819a320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:49.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:49.869+0000 7f87c7e00640 1 -- 172.21.15.45:0/4143271409 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87c81a1f40 con 0x7f87c81019f0 2024-04-16T17:02:49.873 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:49.869+0000 7f87c7400640 1 --2- 172.21.15.45:0/4143271409 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f87c81006f0 0x7f87c819a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:02:49.873 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:49.869+0000 7f87c7e00640 1 --2- 172.21.15.45:0/4143271409 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f87c81019f0 0x7f87c81a18d0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f87c002c630 tx=0x7f87c0032ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:02:49.873 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:49.869+0000 7f87bfe00640 1 -- 172.21.15.45:0/4143271409 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87c0041020 con 0x7f87c81019f0 2024-04-16T17:02:49.873 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:49.869+0000 7f87bfe00640 1 -- 172.21.15.45:0/4143271409 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f87c0034cd0 con 0x7f87c81019f0 2024-04-16T17:02:49.873 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:49.869+0000 7f87bfe00640 1 -- 172.21.15.45:0/4143271409 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87c0039040 con 0x7f87c81019f0 2024-04-16T17:02:49.873 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:49.869+0000 7f87cd200640 1 -- 172.21.15.45:0/4143271409 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87c81a21d0 con 0x7f87c81019f0 2024-04-16T17:02:49.873 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:49.869+0000 7f87cd200640 1 -- 172.21.15.45:0/4143271409 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87c81a2670 con 0x7f87c81019f0 2024-04-16T17:02:49.873 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:49.870+0000 7f87cd200640 1 -- 172.21.15.45:0/4143271409 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f87c810b440 con 0x7f87c81019f0 2024-04-16T17:02:49.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:49.872+0000 7f87bfe00640 1 -- 172.21.15.45:0/4143271409 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f87c0044020 con 0x7f87c81019f0 2024-04-16T17:02:49.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:49.874+0000 7f87bfe00640 1 --2- 172.21.15.45:0/4143271409 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f87ac0a3c30 0x7f87ac0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:02:49.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:49.874+0000 7f87bfe00640 1 -- 172.21.15.45:0/4143271409 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f87c00f2160 con 0x7f87c81019f0 2024-04-16T17:02:49.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:49.874+0000 7f87c7400640 1 --2- 172.21.15.45:0/4143271409 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f87ac0a3c30 0x7f87ac0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:02:49.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:49.875+0000 7f87c7400640 1 --2- 172.21.15.45:0/4143271409 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f87ac0a3c30 0x7f87ac0a60f0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f87b8009fc0 tx=0x7f87b8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:02:49.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:49.877+0000 7f87bfe00640 1 -- 172.21.15.45:0/4143271409 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f87c004b020 con 0x7f87c81019f0 2024-04-16T17:02:50.097 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:02:49 smithi045 ceph-mon[31676]: pgmap v296: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T17:02:50.097 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:02:49 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:02:50.097 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:02:49 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:02:50.098 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:02:49 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:02:50.098 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:02:49 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-16T17:02:50.098 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:02:49 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-16T17:02:50.098 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:02:49 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:02:50.098 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:02:49 smithi045 ceph-mon[31676]: mds.? [v2:172.21.15.45:6834/1748842443,v1:172.21.15.45:6835/1748842443] up:boot 2024-04-16T17:02:50.098 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:02:49 smithi045 ceph-mon[31676]: fsmap 3 up:standby 2024-04-16T17:02:50.098 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:02:49 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-16T17:02:50.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:50.129+0000 7f87cd200640 1 -- 172.21.15.45:0/4143271409 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f87c802d620 con 0x7f87c81019f0 2024-04-16T17:02:50.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:50.130+0000 7f87bfe00640 1 -- 172.21.15.45:0/4143271409 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3049 (secure 0 0 0) 0x7f87c00befd0 con 0x7f87c81019f0 2024-04-16T17:02:50.133 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-16T17:02:50.133 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/1748842443","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":1748842443},{"type":"v1","addr":"172.21.15.45:6835","nonce":1748842443}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/1768242489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1768242489},{"type":"v1","addr":"172.21.15.117:6833","nonce":1768242489}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6833/3558902585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":3558902585},{"type":"v1","addr":"172.21.15.104:6833","nonce":3558902585}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-04-16T17:02:50.133 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 4 2024-04-16T17:02:50.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:50.133+0000 7f87cd200640 1 -- 172.21.15.45:0/4143271409 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f87ac0a3c30 msgr2=0x7f87ac0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:02:50.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:50.133+0000 7f87cd200640 1 --2- 172.21.15.45:0/4143271409 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f87ac0a3c30 0x7f87ac0a60f0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f87b8009fc0 tx=0x7f87b8005f90 comp rx=0 tx=0).stop 2024-04-16T17:02:50.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:50.133+0000 7f87cd200640 1 -- 172.21.15.45:0/4143271409 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f87c81019f0 msgr2=0x7f87c81a18d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:02:50.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:50.133+0000 7f87cd200640 1 --2- 172.21.15.45:0/4143271409 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f87c81019f0 0x7f87c81a18d0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f87c002c630 tx=0x7f87c0032ea0 comp rx=0 tx=0).stop 2024-04-16T17:02:50.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:50.133+0000 7f87cd200640 1 -- 172.21.15.45:0/4143271409 shutdown_connections 2024-04-16T17:02:50.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:50.133+0000 7f87cd200640 1 --2- 172.21.15.45:0/4143271409 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f87ac0a3c30 0x7f87ac0a60f0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:50.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:50.133+0000 7f87cd200640 1 --2- 172.21.15.45:0/4143271409 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f87c8101050 0x7f87c819a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:50.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:50.133+0000 7f87cd200640 1 --2- 172.21.15.45:0/4143271409 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f87c81019f0 0x7f87c81a18d0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:50.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:50.133+0000 7f87cd200640 1 --2- 172.21.15.45:0/4143271409 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f87c81006f0 0x7f87c819a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:50.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:50.133+0000 7f87cd200640 1 -- 172.21.15.45:0/4143271409 >> 172.21.15.45:0/4143271409 conn(0x7f87c80fc2d0 msgr2=0x7f87c80fddd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:02:50.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:50.133+0000 7f87cd200640 1 -- 172.21.15.45:0/4143271409 shutdown_connections 2024-04-16T17:02:50.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:50.133+0000 7f87cd200640 1 -- 172.21.15.45:0/4143271409 wait complete. 2024-04-16T17:02:50.178 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:02:49 smithi104 ceph-mon[36457]: pgmap v296: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T17:02:50.178 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:02:49 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:02:50.179 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:02:49 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:02:50.179 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:02:49 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:02:50.179 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:02:49 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-16T17:02:50.179 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:02:49 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-16T17:02:50.179 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:02:49 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:02:50.179 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:02:49 smithi104 ceph-mon[36457]: mds.? [v2:172.21.15.45:6834/1748842443,v1:172.21.15.45:6835/1748842443] up:boot 2024-04-16T17:02:50.179 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:02:49 smithi104 ceph-mon[36457]: fsmap 3 up:standby 2024-04-16T17:02:50.179 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:02:49 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-16T17:02:50.306 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:02:49 smithi117 ceph-mon[36730]: pgmap v296: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T17:02:50.306 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:02:49 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:02:50.306 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:02:49 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:02:50.306 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:02:49 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:02:50.306 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:02:49 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-16T17:02:50.306 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:02:49 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-16T17:02:50.306 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:02:49 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:02:50.306 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:02:49 smithi117 ceph-mon[36730]: mds.? [v2:172.21.15.45:6834/1748842443,v1:172.21.15.45:6835/1748842443] up:boot 2024-04-16T17:02:50.306 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:02:49 smithi117 ceph-mon[36730]: fsmap 3 up:standby 2024-04-16T17:02:50.306 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:02:49 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-16T17:02:51.178 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:02:50 smithi104 ceph-mon[36457]: Deploying daemon mds.d on smithi045 2024-04-16T17:02:51.179 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:02:50 smithi104 ceph-mon[36457]: from='client.? 172.21.15.45:0/4143271409' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:02:51.228 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:02:50 smithi045 ceph-mon[31676]: Deploying daemon mds.d on smithi045 2024-04-16T17:02:51.228 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:02:50 smithi045 ceph-mon[31676]: from='client.? 172.21.15.45:0/4143271409' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:02:51.306 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:02:50 smithi117 ceph-mon[36730]: Deploying daemon mds.d on smithi045 2024-04-16T17:02:51.306 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:02:50 smithi117 ceph-mon[36730]: from='client.? 172.21.15.45:0/4143271409' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:02:52.085 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:02:51 smithi045 ceph-mon[31676]: pgmap v297: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T17:02:52.154 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-16T17:02:52.178 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:02:51 smithi104 ceph-mon[36457]: pgmap v297: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T17:02:52.306 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:02:51 smithi117 ceph-mon[36730]: pgmap v297: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T17:02:52.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:52.360+0000 7ff976000640 1 -- 172.21.15.45:0/3970000642 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff9700749e0 msgr2=0x7ff970074e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:02:52.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:52.360+0000 7ff976000640 1 --2- 172.21.15.45:0/3970000642 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff9700749e0 0x7ff970074e40 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7ff95c009900 tx=0x7ff95c02bcd0 comp rx=0 tx=0).stop 2024-04-16T17:02:52.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:52.360+0000 7ff976000640 1 -- 172.21.15.45:0/3970000642 shutdown_connections 2024-04-16T17:02:52.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:52.360+0000 7ff976000640 1 --2- 172.21.15.45:0/3970000642 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff970109f00 0x7ff97010c390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:52.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:52.360+0000 7ff976000640 1 --2- 172.21.15.45:0/3970000642 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff9700749e0 0x7ff970074e40 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:52.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:52.360+0000 7ff976000640 1 --2- 172.21.15.45:0/3970000642 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff970075fc0 0x7ff970074410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:52.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:52.360+0000 7ff976000640 1 -- 172.21.15.45:0/3970000642 >> 172.21.15.45:0/3970000642 conn(0x7ff9700fd5e0 msgr2=0x7ff9700ffa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:02:52.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:52.361+0000 7ff976000640 1 -- 172.21.15.45:0/3970000642 shutdown_connections 2024-04-16T17:02:52.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:52.361+0000 7ff976000640 1 -- 172.21.15.45:0/3970000642 wait complete. 2024-04-16T17:02:52.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:52.362+0000 7ff976000640 1 Processor -- start 2024-04-16T17:02:52.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:52.362+0000 7ff976000640 1 -- start start 2024-04-16T17:02:52.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:52.362+0000 7ff976000640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff9700749e0 0x7ff9701053f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:02:52.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:52.362+0000 7ff976000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff970075fc0 0x7ff970105930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:02:52.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:52.362+0000 7ff976000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff970109f00 0x7ff970101f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:02:52.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:52.362+0000 7ff976000640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7ff97010ff40 con 0x7ff970109f00 2024-04-16T17:02:52.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:52.362+0000 7ff976000640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7ff97010fdc0 con 0x7ff9700749e0 2024-04-16T17:02:52.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:52.362+0000 7ff976000640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7ff9701100c0 con 0x7ff970075fc0 2024-04-16T17:02:52.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:52.363+0000 7ff96f400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff9700749e0 0x7ff9701053f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:02:52.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:52.363+0000 7ff96ea00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff970075fc0 0x7ff970105930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:02:52.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:52.363+0000 7ff96f400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff9700749e0 0x7ff9701053f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:36396/0 (socket says 172.21.15.45:36396) 2024-04-16T17:02:52.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:52.363+0000 7ff96f400640 1 -- 172.21.15.45:0/1458714416 learned_addr learned my addr 172.21.15.45:0/1458714416 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:02:52.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:52.363+0000 7ff96ea00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff970075fc0 0x7ff970105930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:55634/0 (socket says 172.21.15.45:55634) 2024-04-16T17:02:52.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:52.363+0000 7ff96fe00640 1 --2- 172.21.15.45:0/1458714416 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff970109f00 0x7ff970101f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:02:52.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:52.363+0000 7ff96f400640 1 -- 172.21.15.45:0/1458714416 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff970075fc0 msgr2=0x7ff970105930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:02:52.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:52.363+0000 7ff96f400640 1 --2- 172.21.15.45:0/1458714416 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff970075fc0 0x7ff970105930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:52.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:52.363+0000 7ff96f400640 1 -- 172.21.15.45:0/1458714416 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff970109f00 msgr2=0x7ff970101f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:02:52.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:52.363+0000 7ff96f400640 1 --2- 172.21.15.45:0/1458714416 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff970109f00 0x7ff970101f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:52.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:52.363+0000 7ff96f400640 1 -- 172.21.15.45:0/1458714416 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff970102820 con 0x7ff9700749e0 2024-04-16T17:02:52.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:52.363+0000 7ff96f400640 1 --2- 172.21.15.45:0/1458714416 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff9700749e0 0x7ff9701053f0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7ff96000e9f0 tx=0x7ff96000edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:02:52.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:52.364+0000 7ff967e00640 1 -- 172.21.15.45:0/1458714416 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff96000ccc0 con 0x7ff9700749e0 2024-04-16T17:02:52.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:52.364+0000 7ff967e00640 1 -- 172.21.15.45:0/1458714416 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff96000ce60 con 0x7ff9700749e0 2024-04-16T17:02:52.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:52.364+0000 7ff967e00640 1 -- 172.21.15.45:0/1458714416 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff960010690 con 0x7ff9700749e0 2024-04-16T17:02:52.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:52.364+0000 7ff976000640 1 -- 172.21.15.45:0/1458714416 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9701a69b0 con 0x7ff9700749e0 2024-04-16T17:02:52.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:52.364+0000 7ff976000640 1 -- 172.21.15.45:0/1458714416 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9701a6d60 con 0x7ff9700749e0 2024-04-16T17:02:52.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:52.365+0000 7ff976000640 1 -- 172.21.15.45:0/1458714416 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff970069270 con 0x7ff9700749e0 2024-04-16T17:02:52.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:52.367+0000 7ff967e00640 1 -- 172.21.15.45:0/1458714416 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7ff9600046c0 con 0x7ff9700749e0 2024-04-16T17:02:52.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:52.369+0000 7ff967e00640 1 --2- 172.21.15.45:0/1458714416 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7ff9540a3c30 0x7ff9540a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:02:52.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:52.369+0000 7ff967e00640 1 -- 172.21.15.45:0/1458714416 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7ff9600d0340 con 0x7ff9700749e0 2024-04-16T17:02:52.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:52.369+0000 7ff96ea00640 1 --2- 172.21.15.45:0/1458714416 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7ff9540a3c30 0x7ff9540a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:02:52.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:52.370+0000 7ff96ea00640 1 --2- 172.21.15.45:0/1458714416 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7ff9540a3c30 0x7ff9540a60f0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7ff95c02c2e0 tx=0x7ff95c02c920 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:02:52.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:52.371+0000 7ff967e00640 1 -- 172.21.15.45:0/1458714416 <== mon.2 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff96009d120 con 0x7ff9700749e0 2024-04-16T17:02:52.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:52.623+0000 7ff976000640 1 -- 172.21.15.45:0/1458714416 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff970108cc0 con 0x7ff9700749e0 2024-04-16T17:02:52.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:52.624+0000 7ff967e00640 1 -- 172.21.15.45:0/1458714416 <== mon.2 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3049 (secure 0 0 0) 0x7ff970108cc0 con 0x7ff9700749e0 2024-04-16T17:02:52.627 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-16T17:02:52.627 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/1748842443","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":1748842443},{"type":"v1","addr":"172.21.15.45:6835","nonce":1748842443}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/1768242489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1768242489},{"type":"v1","addr":"172.21.15.117:6833","nonce":1768242489}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6833/3558902585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":3558902585},{"type":"v1","addr":"172.21.15.104:6833","nonce":3558902585}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-04-16T17:02:52.627 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 4 2024-04-16T17:02:52.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:52.627+0000 7ff976000640 1 -- 172.21.15.45:0/1458714416 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7ff9540a3c30 msgr2=0x7ff9540a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:02:52.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:52.627+0000 7ff976000640 1 --2- 172.21.15.45:0/1458714416 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7ff9540a3c30 0x7ff9540a60f0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7ff95c02c2e0 tx=0x7ff95c02c920 comp rx=0 tx=0).stop 2024-04-16T17:02:52.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:52.627+0000 7ff976000640 1 -- 172.21.15.45:0/1458714416 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff9700749e0 msgr2=0x7ff9701053f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:02:52.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:52.627+0000 7ff976000640 1 --2- 172.21.15.45:0/1458714416 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff9700749e0 0x7ff9701053f0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7ff96000e9f0 tx=0x7ff96000edf0 comp rx=0 tx=0).stop 2024-04-16T17:02:52.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:52.627+0000 7ff976000640 1 -- 172.21.15.45:0/1458714416 shutdown_connections 2024-04-16T17:02:52.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:52.628+0000 7ff976000640 1 --2- 172.21.15.45:0/1458714416 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7ff9540a3c30 0x7ff9540a60f0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:52.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:52.628+0000 7ff976000640 1 --2- 172.21.15.45:0/1458714416 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff970075fc0 0x7ff970105930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:52.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:52.628+0000 7ff976000640 1 --2- 172.21.15.45:0/1458714416 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff970109f00 0x7ff970101f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:52.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:52.628+0000 7ff976000640 1 --2- 172.21.15.45:0/1458714416 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff9700749e0 0x7ff9701053f0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:52.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:52.628+0000 7ff976000640 1 -- 172.21.15.45:0/1458714416 >> 172.21.15.45:0/1458714416 conn(0x7ff9700fd5e0 msgr2=0x7ff970105e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:02:52.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:52.628+0000 7ff976000640 1 -- 172.21.15.45:0/1458714416 shutdown_connections 2024-04-16T17:02:52.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:52.628+0000 7ff976000640 1 -- 172.21.15.45:0/1458714416 wait complete. 2024-04-16T17:02:53.178 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:02:52 smithi104 ceph-mon[36457]: pgmap v298: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T17:02:53.178 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:02:52 smithi104 ceph-mon[36457]: from='client.? 172.21.15.45:0/1458714416' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:02:53.228 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:02:52 smithi045 ceph-mon[31676]: pgmap v298: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T17:02:53.228 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:02:52 smithi045 ceph-mon[31676]: from='client.? 172.21.15.45:0/1458714416' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:02:53.229 INFO:journalctl@ceph.mds.d.smithi045.stdout:Apr 16 17:02:52 smithi045 systemd[1]: Started Ceph mds.d for b1fecd38-fc11-11ee-bc90-c7b262605968. 2024-04-16T17:02:53.306 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:02:52 smithi117 ceph-mon[36730]: pgmap v298: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T17:02:53.306 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:02:52 smithi117 ceph-mon[36730]: from='client.? 172.21.15.45:0/1458714416' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:02:54.429 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:02:54 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:02:54.429 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:02:54 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:02:54.429 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:02:54 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:02:54.429 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:02:54 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-16T17:02:54.429 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:02:54 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-16T17:02:54.429 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:02:54 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:02:54.429 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:02:54 smithi104 ceph-mon[36457]: Deploying daemon mds.e on smithi104 2024-04-16T17:02:54.429 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:02:54 smithi104 ceph-mon[36457]: mds.? [v2:172.21.15.45:6836/1308142903,v1:172.21.15.45:6837/1308142903] up:boot 2024-04-16T17:02:54.429 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:02:54 smithi104 ceph-mon[36457]: fsmap 4 up:standby 2024-04-16T17:02:54.429 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:02:54 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-16T17:02:54.479 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:02:54 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:02:54.479 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:02:54 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:02:54.479 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:02:54 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:02:54.479 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:02:54 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-16T17:02:54.479 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:02:54 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-16T17:02:54.479 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:02:54 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:02:54.479 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:02:54 smithi045 ceph-mon[31676]: Deploying daemon mds.e on smithi104 2024-04-16T17:02:54.479 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:02:54 smithi045 ceph-mon[31676]: mds.? [v2:172.21.15.45:6836/1308142903,v1:172.21.15.45:6837/1308142903] up:boot 2024-04-16T17:02:54.479 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:02:54 smithi045 ceph-mon[31676]: fsmap 4 up:standby 2024-04-16T17:02:54.479 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:02:54 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-16T17:02:54.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:02:54 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:02:54.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:02:54 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:02:54.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:02:54 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:02:54.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:02:54 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-16T17:02:54.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:02:54 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-16T17:02:54.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:02:54 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:02:54.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:02:54 smithi117 ceph-mon[36730]: Deploying daemon mds.e on smithi104 2024-04-16T17:02:54.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:02:54 smithi117 ceph-mon[36730]: mds.? [v2:172.21.15.45:6836/1308142903,v1:172.21.15.45:6837/1308142903] up:boot 2024-04-16T17:02:54.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:02:54 smithi117 ceph-mon[36730]: fsmap 4 up:standby 2024-04-16T17:02:54.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:02:54 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-16T17:02:54.650 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-16T17:02:54.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:54.885+0000 7f077d200640 1 -- 172.21.15.45:0/2200256958 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f077810a030 msgr2=0x7f07781108c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:02:54.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:54.885+0000 7f077d200640 1 --2- 172.21.15.45:0/2200256958 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f077810a030 0x7f07781108c0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f0770009ca0 tx=0x7f077002c0f0 comp rx=0 tx=0).stop 2024-04-16T17:02:54.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:54.885+0000 7f077d200640 1 -- 172.21.15.45:0/2200256958 shutdown_connections 2024-04-16T17:02:54.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:54.885+0000 7f077d200640 1 --2- 172.21.15.45:0/2200256958 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0778109670 0x7f0778109af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:54.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:54.885+0000 7f077d200640 1 --2- 172.21.15.45:0/2200256958 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f077810a030 0x7f07781108c0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:54.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:54.885+0000 7f077d200640 1 --2- 172.21.15.45:0/2200256958 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0778108480 0x7f07781088a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:54.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:54.885+0000 7f077d200640 1 -- 172.21.15.45:0/2200256958 >> 172.21.15.45:0/2200256958 conn(0x7f0778077440 msgr2=0x7f0778079860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:02:54.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:54.885+0000 7f077d200640 1 -- 172.21.15.45:0/2200256958 shutdown_connections 2024-04-16T17:02:54.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:54.885+0000 7f077d200640 1 -- 172.21.15.45:0/2200256958 wait complete. 2024-04-16T17:02:54.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:54.886+0000 7f077d200640 1 Processor -- start 2024-04-16T17:02:54.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:54.886+0000 7f077d200640 1 -- start start 2024-04-16T17:02:54.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:54.887+0000 7f077d200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0778108480 0x7f077819e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:02:54.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:54.887+0000 7f077d200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0778109670 0x7f077819ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:02:54.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:54.887+0000 7f077d200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f077810a030 0x7f07781a5cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:02:54.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:54.887+0000 7f077d200640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f0778114110 con 0x7f0778108480 2024-04-16T17:02:54.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:54.887+0000 7f077d200640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f0778113f90 con 0x7f077810a030 2024-04-16T17:02:54.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:54.887+0000 7f077d200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f0778114290 con 0x7f0778109670 2024-04-16T17:02:54.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:54.887+0000 7f0777400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0778108480 0x7f077819e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:02:54.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:54.887+0000 7f0777400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0778108480 0x7f077819e790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50186/0 (socket says 172.21.15.45:50186) 2024-04-16T17:02:54.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:54.887+0000 7f0777400640 1 -- 172.21.15.45:0/2946812372 learned_addr learned my addr 172.21.15.45:0/2946812372 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:02:54.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:54.887+0000 7f0777e00640 1 --2- 172.21.15.45:0/2946812372 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f077810a030 0x7f07781a5cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:02:54.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:54.887+0000 7f076ea00640 1 --2- 172.21.15.45:0/2946812372 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0778109670 0x7f077819ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:02:54.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:54.887+0000 7f0777400640 1 -- 172.21.15.45:0/2946812372 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0778109670 msgr2=0x7f077819ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:02:54.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:54.887+0000 7f0777400640 1 --2- 172.21.15.45:0/2946812372 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0778109670 0x7f077819ecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:54.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:54.887+0000 7f0777400640 1 -- 172.21.15.45:0/2946812372 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f077810a030 msgr2=0x7f07781a5cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:02:54.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:54.887+0000 7f0777400640 1 --2- 172.21.15.45:0/2946812372 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f077810a030 0x7f07781a5cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:54.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:54.887+0000 7f0777400640 1 -- 172.21.15.45:0/2946812372 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07781a6360 con 0x7f0778108480 2024-04-16T17:02:54.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:54.887+0000 7f0777e00640 1 --2- 172.21.15.45:0/2946812372 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f077810a030 0x7f07781a5cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:02:54.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:54.888+0000 7f0777400640 1 --2- 172.21.15.45:0/2946812372 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0778108480 0x7f077819e790 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f076800e9c0 tx=0x7f076800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:02:54.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:54.888+0000 7f0774c00640 1 -- 172.21.15.45:0/2946812372 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f076800cde0 con 0x7f0778108480 2024-04-16T17:02:54.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:54.888+0000 7f0774c00640 1 -- 172.21.15.45:0/2946812372 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0768010910 con 0x7f0778108480 2024-04-16T17:02:54.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:54.888+0000 7f0774c00640 1 -- 172.21.15.45:0/2946812372 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0768018a00 con 0x7f0778108480 2024-04-16T17:02:54.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:54.888+0000 7f077d200640 1 -- 172.21.15.45:0/2946812372 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07781a6650 con 0x7f0778108480 2024-04-16T17:02:54.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:54.888+0000 7f077d200640 1 -- 172.21.15.45:0/2946812372 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07781a6bf0 con 0x7f0778108480 2024-04-16T17:02:54.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:54.889+0000 7f077d200640 1 -- 172.21.15.45:0/2946812372 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f077810f7a0 con 0x7f0778108480 2024-04-16T17:02:54.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:54.891+0000 7f0774c00640 1 -- 172.21.15.45:0/2946812372 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f076801f020 con 0x7f0778108480 2024-04-16T17:02:54.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:54.893+0000 7f0774c00640 1 --2- 172.21.15.45:0/2946812372 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f075c0a3c30 0x7f075c0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:02:54.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:54.893+0000 7f0774c00640 1 -- 172.21.15.45:0/2946812372 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f07680d02a0 con 0x7f0778108480 2024-04-16T17:02:54.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:54.893+0000 7f076ea00640 1 --2- 172.21.15.45:0/2946812372 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f075c0a3c30 0x7f075c0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:02:54.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:54.894+0000 7f076ea00640 1 --2- 172.21.15.45:0/2946812372 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f075c0a3c30 0x7f075c0a60f0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f077819fc50 tx=0x7f0760005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:02:54.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:54.898+0000 7f0774c00640 1 -- 172.21.15.45:0/2946812372 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f076801e2c0 con 0x7f0778108480 2024-04-16T17:02:55.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:55.148+0000 7f077d200640 1 -- 172.21.15.45:0/2946812372 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f077802d620 con 0x7f0778108480 2024-04-16T17:02:55.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:55.149+0000 7f0774c00640 1 -- 172.21.15.45:0/2946812372 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) ==== 93+0+3874 (secure 0 0 0) 0x7f076809d080 con 0x7f0778108480 2024-04-16T17:02:55.152 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-16T17:02:55.152 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/1748842443","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":1748842443},{"type":"v1","addr":"172.21.15.45:6835","nonce":1748842443}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/1768242489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1768242489},{"type":"v1","addr":"172.21.15.117:6833","nonce":1768242489}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6833/3558902585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":3558902585},{"type":"v1","addr":"172.21.15.104:6833","nonce":3558902585}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24478,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/1308142903","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":1308142903},{"type":"v1","addr":"172.21.15.45:6837","nonce":1308142903}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[]} 2024-04-16T17:02:55.152 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 5 2024-04-16T17:02:55.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:55.152+0000 7f077d200640 1 -- 172.21.15.45:0/2946812372 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f075c0a3c30 msgr2=0x7f075c0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:02:55.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:55.152+0000 7f077d200640 1 --2- 172.21.15.45:0/2946812372 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f075c0a3c30 0x7f075c0a60f0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f077819fc50 tx=0x7f0760005f90 comp rx=0 tx=0).stop 2024-04-16T17:02:55.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:55.152+0000 7f077d200640 1 -- 172.21.15.45:0/2946812372 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0778108480 msgr2=0x7f077819e790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:02:55.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:55.152+0000 7f077d200640 1 --2- 172.21.15.45:0/2946812372 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0778108480 0x7f077819e790 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f076800e9c0 tx=0x7f076800ee80 comp rx=0 tx=0).stop 2024-04-16T17:02:55.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:55.152+0000 7f077d200640 1 -- 172.21.15.45:0/2946812372 shutdown_connections 2024-04-16T17:02:55.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:55.152+0000 7f077d200640 1 --2- 172.21.15.45:0/2946812372 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f075c0a3c30 0x7f075c0a60f0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:55.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:55.152+0000 7f077d200640 1 --2- 172.21.15.45:0/2946812372 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0778109670 0x7f077819ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:55.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:55.152+0000 7f077d200640 1 --2- 172.21.15.45:0/2946812372 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f077810a030 0x7f07781a5cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:55.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:55.152+0000 7f077d200640 1 --2- 172.21.15.45:0/2946812372 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0778108480 0x7f077819e790 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:55.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:55.152+0000 7f077d200640 1 -- 172.21.15.45:0/2946812372 >> 172.21.15.45:0/2946812372 conn(0x7f0778077440 msgr2=0x7f077810c970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:02:55.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:55.153+0000 7f077d200640 1 -- 172.21.15.45:0/2946812372 shutdown_connections 2024-04-16T17:02:55.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:55.153+0000 7f077d200640 1 -- 172.21.15.45:0/2946812372 wait complete. 2024-04-16T17:02:55.428 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:02:55 smithi104 ceph-mon[36457]: pgmap v299: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T17:02:55.429 INFO:journalctl@ceph.mds.e.smithi104.stdout:Apr 16 17:02:55 smithi104 systemd[1]: Starting Ceph mds.e for b1fecd38-fc11-11ee-bc90-c7b262605968... 2024-04-16T17:02:55.473 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:02:55 smithi045 ceph-mon[31676]: pgmap v299: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T17:02:55.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:02:55 smithi117 ceph-mon[36730]: pgmap v299: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T17:02:55.831 INFO:journalctl@ceph.mds.e.smithi104.stdout:Apr 16 17:02:55 smithi104 podman[67151]: 2024-04-16 17:02:55.448003591 +0000 UTC m=+0.017577505 image pull e5ce10bdffbeeffffd1125f67b369e2151f0f17cdec854e93b6adf9321f65907 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2 2024-04-16T17:02:55.831 INFO:journalctl@ceph.mds.e.smithi104.stdout:Apr 16 17:02:55 smithi104 podman[67151]: 2024-04-16 17:02:55.569756332 +0000 UTC m=+0.139330237 container create 27e929370e371825dd4c5123c564baa959d6ca7f6cde48e83b6d5f6e03e2eb1a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mds-e, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240409, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb) 2024-04-16T17:02:55.831 INFO:journalctl@ceph.mds.e.smithi104.stdout:Apr 16 17:02:55 smithi104 podman[67151]: 2024-04-16 17:02:55.830523407 +0000 UTC m=+0.400097326 container init 27e929370e371825dd4c5123c564baa959d6ca7f6cde48e83b6d5f6e03e2eb1a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mds-e, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240409, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS) 2024-04-16T17:02:56.093 INFO:journalctl@ceph.mds.e.smithi104.stdout:Apr 16 17:02:55 smithi104 podman[67151]: 2024-04-16 17:02:55.833286616 +0000 UTC m=+0.402860530 container start 27e929370e371825dd4c5123c564baa959d6ca7f6cde48e83b6d5f6e03e2eb1a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mds-e, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, org.label-schema.build-date=20240409, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-16T17:02:56.093 INFO:journalctl@ceph.mds.e.smithi104.stdout:Apr 16 17:02:55 smithi104 ceph-mds[67171]: -- 172.21.15.104:0/861525379 <== mon.2 v2:172.21.15.104:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x555c484ffa40 con 0x555c484d2800 2024-04-16T17:02:56.093 INFO:journalctl@ceph.mds.e.smithi104.stdout:Apr 16 17:02:55 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mds-e[67167]: starting mds.e at 2024-04-16T17:02:56.094 INFO:journalctl@ceph.mds.e.smithi104.stdout:Apr 16 17:02:55 smithi104 bash[67151]: 27e929370e371825dd4c5123c564baa959d6ca7f6cde48e83b6d5f6e03e2eb1a 2024-04-16T17:02:56.094 INFO:journalctl@ceph.mds.e.smithi104.stdout:Apr 16 17:02:55 smithi104 systemd[1]: Started Ceph mds.e for b1fecd38-fc11-11ee-bc90-c7b262605968. 2024-04-16T17:02:56.428 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:02:56 smithi104 ceph-mon[36457]: from='client.? 172.21.15.45:0/2946812372' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:02:56.429 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:02:56 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:02:56.429 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:02:56 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:02:56.429 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:02:56 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:02:56.429 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:02:56 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-16T17:02:56.429 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:02:56 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-16T17:02:56.429 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:02:56 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:02:56.478 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:02:56 smithi045 ceph-mon[31676]: from='client.? 172.21.15.45:0/2946812372' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:02:56.478 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:02:56 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:02:56.478 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:02:56 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:02:56.478 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:02:56 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:02:56.479 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:02:56 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-16T17:02:56.479 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:02:56 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-16T17:02:56.479 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:02:56 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:02:56.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:02:56 smithi117 ceph-mon[36730]: from='client.? 172.21.15.45:0/2946812372' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:02:56.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:02:56 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:02:56.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:02:56 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:02:56.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:02:56 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:02:56.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:02:56 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-16T17:02:56.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:02:56 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-16T17:02:56.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:02:56 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:02:57.175 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-16T17:02:57.306 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:02:57 smithi117 ceph-mon[36730]: Deploying daemon mds.f on smithi117 2024-04-16T17:02:57.306 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:02:57 smithi117 ceph-mon[36730]: pgmap v300: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T17:02:57.306 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:02:57 smithi117 ceph-mon[36730]: mds.? [v2:172.21.15.104:6834/3430679937,v1:172.21.15.104:6835/3430679937] up:boot 2024-04-16T17:02:57.306 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:02:57 smithi117 ceph-mon[36730]: fsmap 5 up:standby 2024-04-16T17:02:57.306 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:02:57 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-16T17:02:57.375 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:02:57 smithi104 ceph-mon[36457]: Deploying daemon mds.f on smithi117 2024-04-16T17:02:57.375 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:02:57 smithi104 ceph-mon[36457]: pgmap v300: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T17:02:57.375 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:02:57 smithi104 ceph-mon[36457]: mds.? [v2:172.21.15.104:6834/3430679937,v1:172.21.15.104:6835/3430679937] up:boot 2024-04-16T17:02:57.375 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:02:57 smithi104 ceph-mon[36457]: fsmap 5 up:standby 2024-04-16T17:02:57.375 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:02:57 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-16T17:02:57.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:57.408+0000 7fa237200640 1 -- 172.21.15.45:0/789352576 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa2381020b0 msgr2=0x7fa238108940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:02:57.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:57.408+0000 7fa237200640 1 --2- 172.21.15.45:0/789352576 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa2381020b0 0x7fa238108940 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fa23000aa10 tx=0x7fa23002cf60 comp rx=0 tx=0).stop 2024-04-16T17:02:57.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:57.408+0000 7fa237200640 1 -- 172.21.15.45:0/789352576 shutdown_connections 2024-04-16T17:02:57.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:57.408+0000 7fa237200640 1 --2- 172.21.15.45:0/789352576 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa2381020b0 0x7fa238108940 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:57.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:57.408+0000 7fa237200640 1 --2- 172.21.15.45:0/789352576 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa2381016f0 0x7fa238101b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:57.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:57.408+0000 7fa237200640 1 --2- 172.21.15.45:0/789352576 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa238100500 0x7fa238100920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:57.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:57.408+0000 7fa237200640 1 -- 172.21.15.45:0/789352576 >> 172.21.15.45:0/789352576 conn(0x7fa2380fba90 msgr2=0x7fa2380fdeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:02:57.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:57.408+0000 7fa237200640 1 -- 172.21.15.45:0/789352576 shutdown_connections 2024-04-16T17:02:57.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:57.408+0000 7fa237200640 1 -- 172.21.15.45:0/789352576 wait complete. 2024-04-16T17:02:57.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:57.409+0000 7fa237200640 1 Processor -- start 2024-04-16T17:02:57.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:57.409+0000 7fa237200640 1 -- start start 2024-04-16T17:02:57.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:57.410+0000 7fa237200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa238100500 0x7fa23819a340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:02:57.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:57.410+0000 7fa237200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa2381016f0 0x7fa23819a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:02:57.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:57.410+0000 7fa237200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa2381020b0 0x7fa2381a18f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:02:57.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:57.410+0000 7fa237200640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fa23810b910 con 0x7fa2381016f0 2024-04-16T17:02:57.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:57.410+0000 7fa237200640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fa23810b790 con 0x7fa238100500 2024-04-16T17:02:57.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:57.410+0000 7fa237200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa23810ba90 con 0x7fa2381020b0 2024-04-16T17:02:57.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:57.410+0000 7fa22d400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa2381016f0 0x7fa23819a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:02:57.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:57.410+0000 7fa235e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa238100500 0x7fa23819a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:02:57.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:57.410+0000 7fa235e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa238100500 0x7fa23819a340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:36442/0 (socket says 172.21.15.45:36442) 2024-04-16T17:02:57.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:57.410+0000 7fa235e00640 1 -- 172.21.15.45:0/2153386670 learned_addr learned my addr 172.21.15.45:0/2153386670 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:02:57.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:57.410+0000 7fa236800640 1 --2- 172.21.15.45:0/2153386670 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa2381020b0 0x7fa2381a18f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:02:57.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:57.410+0000 7fa235e00640 1 -- 172.21.15.45:0/2153386670 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa2381020b0 msgr2=0x7fa2381a18f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:02:57.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:57.410+0000 7fa235e00640 1 --2- 172.21.15.45:0/2153386670 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa2381020b0 0x7fa2381a18f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:57.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:57.410+0000 7fa235e00640 1 -- 172.21.15.45:0/2153386670 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa2381016f0 msgr2=0x7fa23819a880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:02:57.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:57.410+0000 7fa235e00640 1 --2- 172.21.15.45:0/2153386670 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa2381016f0 0x7fa23819a880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:57.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:57.410+0000 7fa235e00640 1 -- 172.21.15.45:0/2153386670 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2381a1f60 con 0x7fa238100500 2024-04-16T17:02:57.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:57.411+0000 7fa235e00640 1 --2- 172.21.15.45:0/2153386670 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa238100500 0x7fa23819a340 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fa22800ebc0 tx=0x7fa22800c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:02:57.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:57.411+0000 7fa22f400640 1 -- 172.21.15.45:0/2153386670 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa228003b70 con 0x7fa238100500 2024-04-16T17:02:57.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:57.411+0000 7fa237200640 1 -- 172.21.15.45:0/2153386670 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2381a2250 con 0x7fa238100500 2024-04-16T17:02:57.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:57.411+0000 7fa22f400640 1 -- 172.21.15.45:0/2153386670 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa228003d10 con 0x7fa238100500 2024-04-16T17:02:57.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:57.411+0000 7fa22f400640 1 -- 172.21.15.45:0/2153386670 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa228019980 con 0x7fa238100500 2024-04-16T17:02:57.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:57.411+0000 7fa237200640 1 -- 172.21.15.45:0/2153386670 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2381a27a0 con 0x7fa238100500 2024-04-16T17:02:57.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:57.412+0000 7fa237200640 1 -- 172.21.15.45:0/2153386670 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2381077c0 con 0x7fa238100500 2024-04-16T17:02:57.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:57.414+0000 7fa22f400640 1 -- 172.21.15.45:0/2153386670 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fa228019b20 con 0x7fa238100500 2024-04-16T17:02:57.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:57.415+0000 7fa22f400640 1 --2- 172.21.15.45:0/2153386670 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fa21c0a3bb0 0x7fa21c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:02:57.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:57.416+0000 7fa22f400640 1 -- 172.21.15.45:0/2153386670 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fa2280d0800 con 0x7fa238100500 2024-04-16T17:02:57.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:57.416+0000 7fa22d400640 1 --2- 172.21.15.45:0/2153386670 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fa21c0a3bb0 0x7fa21c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:02:57.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:57.416+0000 7fa22d400640 1 --2- 172.21.15.45:0/2153386670 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fa21c0a3bb0 0x7fa21c0a6070 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fa2200095f0 tx=0x7fa220005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:02:57.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:57.420+0000 7fa22f400640 1 -- 172.21.15.45:0/2153386670 <== mon.2 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa22809d5e0 con 0x7fa238100500 2024-04-16T17:02:57.478 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:02:57 smithi045 ceph-mon[31676]: Deploying daemon mds.f on smithi117 2024-04-16T17:02:57.478 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:02:57 smithi045 ceph-mon[31676]: pgmap v300: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T17:02:57.478 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:02:57 smithi045 ceph-mon[31676]: mds.? [v2:172.21.15.104:6834/3430679937,v1:172.21.15.104:6835/3430679937] up:boot 2024-04-16T17:02:57.478 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:02:57 smithi045 ceph-mon[31676]: fsmap 5 up:standby 2024-04-16T17:02:57.478 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:02:57 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-16T17:02:57.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:57.671+0000 7fa237200640 1 -- 172.21.15.45:0/2153386670 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa23802d620 con 0x7fa238100500 2024-04-16T17:02:57.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:57.673+0000 7fa22f400640 1 -- 172.21.15.45:0/2153386670 <== mon.2 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+4702 (secure 0 0 0) 0x7fa22809cee0 con 0x7fa238100500 2024-04-16T17:02:57.674 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-16T17:02:57.674 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/1748842443","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":1748842443},{"type":"v1","addr":"172.21.15.45:6835","nonce":1748842443}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/1768242489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1768242489},{"type":"v1","addr":"172.21.15.117:6833","nonce":1768242489}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6833/3558902585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":3558902585},{"type":"v1","addr":"172.21.15.104:6833","nonce":3558902585}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24478,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/1308142903","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":1308142903},{"type":"v1","addr":"172.21.15.45:6837","nonce":1308142903}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24491,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6835/3430679937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":3430679937},{"type":"v1","addr":"172.21.15.104:6835","nonce":3430679937}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-04-16T17:02:57.675 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 6 2024-04-16T17:02:57.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:57.675+0000 7fa237200640 1 -- 172.21.15.45:0/2153386670 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fa21c0a3bb0 msgr2=0x7fa21c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:02:57.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:57.675+0000 7fa237200640 1 --2- 172.21.15.45:0/2153386670 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fa21c0a3bb0 0x7fa21c0a6070 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fa2200095f0 tx=0x7fa220005f90 comp rx=0 tx=0).stop 2024-04-16T17:02:57.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:57.675+0000 7fa237200640 1 -- 172.21.15.45:0/2153386670 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa238100500 msgr2=0x7fa23819a340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:02:57.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:57.675+0000 7fa237200640 1 --2- 172.21.15.45:0/2153386670 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa238100500 0x7fa23819a340 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fa22800ebc0 tx=0x7fa22800c6a0 comp rx=0 tx=0).stop 2024-04-16T17:02:57.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:57.675+0000 7fa237200640 1 -- 172.21.15.45:0/2153386670 shutdown_connections 2024-04-16T17:02:57.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:57.675+0000 7fa237200640 1 --2- 172.21.15.45:0/2153386670 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fa21c0a3bb0 0x7fa21c0a6070 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:57.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:57.675+0000 7fa237200640 1 --2- 172.21.15.45:0/2153386670 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa2381020b0 0x7fa2381a18f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:57.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:57.675+0000 7fa237200640 1 --2- 172.21.15.45:0/2153386670 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa2381016f0 0x7fa23819a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:57.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:57.675+0000 7fa237200640 1 --2- 172.21.15.45:0/2153386670 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa238100500 0x7fa23819a340 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:57.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:57.675+0000 7fa237200640 1 -- 172.21.15.45:0/2153386670 >> 172.21.15.45:0/2153386670 conn(0x7fa2380fba90 msgr2=0x7fa238104990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:02:57.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:57.675+0000 7fa237200640 1 -- 172.21.15.45:0/2153386670 shutdown_connections 2024-04-16T17:02:57.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:57.676+0000 7fa237200640 1 -- 172.21.15.45:0/2153386670 wait complete. 2024-04-16T17:02:58.428 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:02:58 smithi104 ceph-mon[36457]: from='client.? 172.21.15.45:0/2153386670' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:02:58.478 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:02:58 smithi045 ceph-mon[31676]: from='client.? 172.21.15.45:0/2153386670' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:02:58.488 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:02:58 smithi117 ceph-mon[36730]: from='client.? 172.21.15.45:0/2153386670' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:02:58.488 INFO:journalctl@ceph.mds.f.smithi117.stdout:Apr 16 17:02:58 smithi117 systemd[1]: Starting Ceph mds.f for b1fecd38-fc11-11ee-bc90-c7b262605968... 2024-04-16T17:02:58.753 INFO:journalctl@ceph.mds.f.smithi117.stdout:Apr 16 17:02:58 smithi117 podman[67348]: 2024-04-16 17:02:58.389764776 +0000 UTC m=+0.017745656 image pull e5ce10bdffbeeffffd1125f67b369e2151f0f17cdec854e93b6adf9321f65907 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2 2024-04-16T17:02:58.753 INFO:journalctl@ceph.mds.f.smithi117.stdout:Apr 16 17:02:58 smithi117 podman[67348]: 2024-04-16 17:02:58.51685874 +0000 UTC m=+0.144839597 container create a9eb34cce0d4060ed494edc24b958a7702fc5d52a29ed990a4a05231c7677f93 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mds-f, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240409, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-16T17:02:58.753 INFO:journalctl@ceph.mds.f.smithi117.stdout:Apr 16 17:02:58 smithi117 podman[67348]: 2024-04-16 17:02:58.752645907 +0000 UTC m=+0.380626773 container init a9eb34cce0d4060ed494edc24b958a7702fc5d52a29ed990a4a05231c7677f93 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mds-f, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240409, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-16T17:02:59.056 INFO:journalctl@ceph.mds.f.smithi117.stdout:Apr 16 17:02:58 smithi117 podman[67348]: 2024-04-16 17:02:58.755359952 +0000 UTC m=+0.383340827 container start a9eb34cce0d4060ed494edc24b958a7702fc5d52a29ed990a4a05231c7677f93 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mds-f, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240409, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-04-16T17:02:59.056 INFO:journalctl@ceph.mds.f.smithi117.stdout:Apr 16 17:02:58 smithi117 ceph-mds[67367]: -- 172.21.15.117:0/2645262279 <== mon.1 v2:172.21.15.117:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55b6b93d5a40 con 0x55b6b93a9000 2024-04-16T17:02:59.056 INFO:journalctl@ceph.mds.f.smithi117.stdout:Apr 16 17:02:58 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mds-f[67363]: starting mds.f at 2024-04-16T17:02:59.056 INFO:journalctl@ceph.mds.f.smithi117.stdout:Apr 16 17:02:58 smithi117 bash[67348]: a9eb34cce0d4060ed494edc24b958a7702fc5d52a29ed990a4a05231c7677f93 2024-04-16T17:02:59.056 INFO:journalctl@ceph.mds.f.smithi117.stdout:Apr 16 17:02:58 smithi117 systemd[1]: Started Ceph mds.f for b1fecd38-fc11-11ee-bc90-c7b262605968. 2024-04-16T17:02:59.428 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:02:59 smithi104 ceph-mon[36457]: pgmap v301: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T17:02:59.429 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:02:59 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:02:59.429 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:02:59 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:02:59.429 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:02:59 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:02:59.429 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:02:59 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-16T17:02:59.429 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:02:59 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-16T17:02:59.429 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:02:59 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:02:59.478 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:02:59 smithi045 ceph-mon[31676]: pgmap v301: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T17:02:59.478 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:02:59 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:02:59.478 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:02:59 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:02:59.478 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:02:59 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:02:59.479 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:02:59 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-16T17:02:59.479 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:02:59 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-16T17:02:59.479 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:02:59 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:02:59.555 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:02:59 smithi117 ceph-mon[36730]: pgmap v301: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T17:02:59.555 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:02:59 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:02:59.555 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:02:59 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:02:59.555 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:02:59 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:02:59.555 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:02:59 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-16T17:02:59.555 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:02:59 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-16T17:02:59.555 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:02:59 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:02:59.698 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-16T17:02:59.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:59.931+0000 7fc30e600640 1 -- 172.21.15.45:0/239279701 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc3080684b0 msgr2=0x7fc308101290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:02:59.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:59.931+0000 7fc30e600640 1 --2- 172.21.15.45:0/239279701 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc3080684b0 0x7fc308101290 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fc2f800aa40 tx=0x7fc2f802cf50 comp rx=0 tx=0).stop 2024-04-16T17:02:59.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:59.932+0000 7fc30e600640 1 -- 172.21.15.45:0/239279701 shutdown_connections 2024-04-16T17:02:59.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:59.932+0000 7fc30e600640 1 --2- 172.21.15.45:0/239279701 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc3081064d0 0x7fc3081088c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:59.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:59.932+0000 7fc30e600640 1 --2- 172.21.15.45:0/239279701 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc3081017d0 0x7fc308101c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:59.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:59.932+0000 7fc30e600640 1 --2- 172.21.15.45:0/239279701 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc3080684b0 0x7fc308101290 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:59.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:59.932+0000 7fc30e600640 1 -- 172.21.15.45:0/239279701 >> 172.21.15.45:0/239279701 conn(0x7fc3080fb630 msgr2=0x7fc3080fda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:02:59.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:59.932+0000 7fc30e600640 1 -- 172.21.15.45:0/239279701 shutdown_connections 2024-04-16T17:02:59.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:59.932+0000 7fc30e600640 1 -- 172.21.15.45:0/239279701 wait complete. 2024-04-16T17:02:59.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:59.933+0000 7fc30e600640 1 Processor -- start 2024-04-16T17:02:59.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:59.933+0000 7fc30e600640 1 -- start start 2024-04-16T17:02:59.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:59.933+0000 7fc30e600640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc3080684b0 0x7fc30819a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:02:59.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:59.933+0000 7fc30e600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc3081017d0 0x7fc30819a960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:02:59.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:59.933+0000 7fc30e600640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc3081064d0 0x7fc3081a19d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:02:59.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:59.933+0000 7fc30e600640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fc30810dac0 con 0x7fc3081017d0 2024-04-16T17:02:59.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:59.933+0000 7fc30e600640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fc30810d940 con 0x7fc3081064d0 2024-04-16T17:02:59.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:59.933+0000 7fc30e600640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fc30810dc40 con 0x7fc3080684b0 2024-04-16T17:02:59.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:59.933+0000 7fc307e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc3081017d0 0x7fc30819a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:02:59.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:59.933+0000 7fc307e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc3081017d0 0x7fc30819a960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39144/0 (socket says 172.21.15.45:39144) 2024-04-16T17:02:59.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:59.933+0000 7fc307e00640 1 -- 172.21.15.45:0/123749970 learned_addr learned my addr 172.21.15.45:0/123749970 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:02:59.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:59.933+0000 7fc30d200640 1 --2- 172.21.15.45:0/123749970 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc3080684b0 0x7fc30819a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:02:59.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:59.933+0000 7fc30dc00640 1 --2- 172.21.15.45:0/123749970 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc3081064d0 0x7fc3081a19d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:02:59.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:59.934+0000 7fc30d200640 1 -- 172.21.15.45:0/123749970 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc3081064d0 msgr2=0x7fc3081a19d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:02:59.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:59.934+0000 7fc30d200640 1 --2- 172.21.15.45:0/123749970 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc3081064d0 0x7fc3081a19d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:59.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:59.934+0000 7fc30d200640 1 -- 172.21.15.45:0/123749970 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc3081017d0 msgr2=0x7fc30819a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:02:59.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:59.934+0000 7fc30d200640 1 --2- 172.21.15.45:0/123749970 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc3081017d0 0x7fc30819a960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:02:59.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:59.934+0000 7fc30d200640 1 -- 172.21.15.45:0/123749970 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3081a2040 con 0x7fc3080684b0 2024-04-16T17:02:59.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:59.934+0000 7fc307e00640 1 --2- 172.21.15.45:0/123749970 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc3081017d0 0x7fc30819a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:02:59.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:59.934+0000 7fc30d200640 1 --2- 172.21.15.45:0/123749970 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc3080684b0 0x7fc30819a420 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fc2f8037040 tx=0x7fc2f8005200 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:02:59.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:59.934+0000 7fc305600640 1 -- 172.21.15.45:0/123749970 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2f8035c00 con 0x7fc3080684b0 2024-04-16T17:02:59.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:59.935+0000 7fc30e600640 1 -- 172.21.15.45:0/123749970 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3081a22d0 con 0x7fc3080684b0 2024-04-16T17:02:59.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:59.935+0000 7fc305600640 1 -- 172.21.15.45:0/123749970 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc2f8035da0 con 0x7fc3080684b0 2024-04-16T17:02:59.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:59.935+0000 7fc305600640 1 -- 172.21.15.45:0/123749970 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2f8003850 con 0x7fc3080684b0 2024-04-16T17:02:59.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:59.935+0000 7fc30e600640 1 -- 172.21.15.45:0/123749970 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3081a2790 con 0x7fc3080684b0 2024-04-16T17:02:59.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:59.936+0000 7fc30e600640 1 -- 172.21.15.45:0/123749970 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3081a2ab0 con 0x7fc3080684b0 2024-04-16T17:02:59.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:59.938+0000 7fc305600640 1 -- 172.21.15.45:0/123749970 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fc2f8045020 con 0x7fc3080684b0 2024-04-16T17:02:59.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:59.939+0000 7fc305600640 1 --2- 172.21.15.45:0/123749970 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fc2ec0a3bb0 0x7fc2ec0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:02:59.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:59.939+0000 7fc305600640 1 -- 172.21.15.45:0/123749970 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fc2f80f1950 con 0x7fc3080684b0 2024-04-16T17:02:59.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:59.939+0000 7fc307e00640 1 --2- 172.21.15.45:0/123749970 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fc2ec0a3bb0 0x7fc2ec0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:02:59.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:59.940+0000 7fc307e00640 1 --2- 172.21.15.45:0/123749970 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fc2ec0a3bb0 0x7fc2ec0a6070 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fc30819b930 tx=0x7fc2f4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:02:59.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:02:59.943+0000 7fc305600640 1 -- 172.21.15.45:0/123749970 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc2f804c020 con 0x7fc3080684b0 2024-04-16T17:03:00.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:00.196+0000 7fc30e600640 1 -- 172.21.15.45:0/123749970 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc3081a2c80 con 0x7fc3080684b0 2024-04-16T17:03:00.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:00.197+0000 7fc305600640 1 -- 172.21.15.45:0/123749970 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5530 (secure 0 0 0) 0x7fc2f80be730 con 0x7fc3080684b0 2024-04-16T17:03:00.199 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-16T17:03:00.199 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/1748842443","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":1748842443},{"type":"v1","addr":"172.21.15.45:6835","nonce":1748842443}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/1768242489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1768242489},{"type":"v1","addr":"172.21.15.117:6833","nonce":1768242489}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6833/3558902585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":3558902585},{"type":"v1","addr":"172.21.15.104:6833","nonce":3558902585}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24478,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/1308142903","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":1308142903},{"type":"v1","addr":"172.21.15.45:6837","nonce":1308142903}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24491,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6835/3430679937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":3430679937},{"type":"v1","addr":"172.21.15.104:6835","nonce":3430679937}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24499,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/2224123841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2224123841},{"type":"v1","addr":"172.21.15.117:6835","nonce":2224123841}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[]} 2024-04-16T17:03:00.200 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 7 2024-04-16T17:03:00.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:00.200+0000 7fc30e600640 1 -- 172.21.15.45:0/123749970 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fc2ec0a3bb0 msgr2=0x7fc2ec0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:00.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:00.200+0000 7fc30e600640 1 --2- 172.21.15.45:0/123749970 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fc2ec0a3bb0 0x7fc2ec0a6070 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fc30819b930 tx=0x7fc2f4005f90 comp rx=0 tx=0).stop 2024-04-16T17:03:00.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:00.200+0000 7fc30e600640 1 -- 172.21.15.45:0/123749970 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc3080684b0 msgr2=0x7fc30819a420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:00.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:00.200+0000 7fc30e600640 1 --2- 172.21.15.45:0/123749970 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc3080684b0 0x7fc30819a420 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fc2f8037040 tx=0x7fc2f8005200 comp rx=0 tx=0).stop 2024-04-16T17:03:00.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:00.200+0000 7fc30e600640 1 -- 172.21.15.45:0/123749970 shutdown_connections 2024-04-16T17:03:00.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:00.200+0000 7fc30e600640 1 --2- 172.21.15.45:0/123749970 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fc2ec0a3bb0 0x7fc2ec0a6070 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:00.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:00.200+0000 7fc30e600640 1 --2- 172.21.15.45:0/123749970 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc3081064d0 0x7fc3081a19d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:00.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:00.200+0000 7fc30e600640 1 --2- 172.21.15.45:0/123749970 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc3081017d0 0x7fc30819a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:00.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:00.200+0000 7fc30e600640 1 --2- 172.21.15.45:0/123749970 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc3080684b0 0x7fc30819a420 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:00.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:00.200+0000 7fc30e600640 1 -- 172.21.15.45:0/123749970 >> 172.21.15.45:0/123749970 conn(0x7fc3080fb630 msgr2=0x7fc3080fbd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:03:00.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:00.200+0000 7fc30e600640 1 -- 172.21.15.45:0/123749970 shutdown_connections 2024-04-16T17:03:00.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:00.200+0000 7fc30e600640 1 -- 172.21.15.45:0/123749970 wait complete. 2024-04-16T17:03:00.428 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:03:00 smithi104 ceph-mon[36457]: Deploying daemon mds.c on smithi117 2024-04-16T17:03:00.428 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:03:00 smithi104 ceph-mon[36457]: mds.? [v2:172.21.15.117:6834/2224123841,v1:172.21.15.117:6835/2224123841] up:boot 2024-04-16T17:03:00.428 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:03:00 smithi104 ceph-mon[36457]: fsmap 6 up:standby 2024-04-16T17:03:00.429 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:03:00 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-16T17:03:00.478 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:03:00 smithi045 ceph-mon[31676]: Deploying daemon mds.c on smithi117 2024-04-16T17:03:00.478 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:03:00 smithi045 ceph-mon[31676]: mds.? [v2:172.21.15.117:6834/2224123841,v1:172.21.15.117:6835/2224123841] up:boot 2024-04-16T17:03:00.478 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:03:00 smithi045 ceph-mon[31676]: fsmap 6 up:standby 2024-04-16T17:03:00.478 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:03:00 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-16T17:03:00.479 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:03:00 smithi117 ceph-mon[36730]: Deploying daemon mds.c on smithi117 2024-04-16T17:03:00.479 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:03:00 smithi117 ceph-mon[36730]: mds.? [v2:172.21.15.117:6834/2224123841,v1:172.21.15.117:6835/2224123841] up:boot 2024-04-16T17:03:00.479 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:03:00 smithi117 ceph-mon[36730]: fsmap 6 up:standby 2024-04-16T17:03:00.479 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:03:00 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-16T17:03:01.056 INFO:journalctl@ceph.mds.c.smithi117.stdout:Apr 16 17:03:00 smithi117 systemd[1]: Starting Ceph mds.c for b1fecd38-fc11-11ee-bc90-c7b262605968... 2024-04-16T17:03:01.429 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:03:01 smithi104 ceph-mon[36457]: pgmap v302: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T17:03:01.429 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:03:01 smithi104 ceph-mon[36457]: from='client.? 172.21.15.45:0/123749970' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:03:01.478 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:03:01 smithi045 ceph-mon[31676]: pgmap v302: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T17:03:01.478 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:03:01 smithi045 ceph-mon[31676]: from='client.? 172.21.15.45:0/123749970' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:03:01.534 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:03:01 smithi117 ceph-mon[36730]: pgmap v302: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T17:03:01.534 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:03:01 smithi117 ceph-mon[36730]: from='client.? 172.21.15.45:0/123749970' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:03:01.534 INFO:journalctl@ceph.mds.c.smithi117.stdout:Apr 16 17:03:01 smithi117 podman[67654]: 2024-04-16 17:03:01.0847659 +0000 UTC m=+0.017804687 image pull e5ce10bdffbeeffffd1125f67b369e2151f0f17cdec854e93b6adf9321f65907 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2 2024-04-16T17:03:01.535 INFO:journalctl@ceph.mds.c.smithi117.stdout:Apr 16 17:03:01 smithi117 podman[67654]: 2024-04-16 17:03:01.206088151 +0000 UTC m=+0.139126938 container create 13fd9e10d5813e0ed2550e2603de9ddae30bddb663b77b3fee60d61ba0b890cb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mds-c, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240409) 2024-04-16T17:03:01.535 INFO:journalctl@ceph.mds.c.smithi117.stdout:Apr 16 17:03:01 smithi117 podman[67654]: 2024-04-16 17:03:01.533898618 +0000 UTC m=+0.466937405 container init 13fd9e10d5813e0ed2550e2603de9ddae30bddb663b77b3fee60d61ba0b890cb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mds-c, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240409, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-04-16T17:03:01.806 INFO:journalctl@ceph.mds.c.smithi117.stdout:Apr 16 17:03:01 smithi117 podman[67654]: 2024-04-16 17:03:01.536703085 +0000 UTC m=+0.469741880 container start 13fd9e10d5813e0ed2550e2603de9ddae30bddb663b77b3fee60d61ba0b890cb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mds-c, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240409, org.label-schema.schema-version=1.0, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5) 2024-04-16T17:03:01.806 INFO:journalctl@ceph.mds.c.smithi117.stdout:Apr 16 17:03:01 smithi117 ceph-mds[67672]: -- 172.21.15.117:0/1635326107 <== mon.1 v2:172.21.15.117:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x556c9a163a40 con 0x556c9a136c00 2024-04-16T17:03:01.806 INFO:journalctl@ceph.mds.c.smithi117.stdout:Apr 16 17:03:01 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mds-c[67668]: starting mds.c at 2024-04-16T17:03:01.806 INFO:journalctl@ceph.mds.c.smithi117.stdout:Apr 16 17:03:01 smithi117 bash[67654]: 13fd9e10d5813e0ed2550e2603de9ddae30bddb663b77b3fee60d61ba0b890cb 2024-04-16T17:03:01.807 INFO:journalctl@ceph.mds.c.smithi117.stdout:Apr 16 17:03:01 smithi117 systemd[1]: Started Ceph mds.c for b1fecd38-fc11-11ee-bc90-c7b262605968. 2024-04-16T17:03:02.221 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-16T17:03:02.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:02.457+0000 7f50d0a00640 1 -- 172.21.15.45:0/2399032409 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f50cc074a60 msgr2=0x7f50cc074ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:02.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:02.457+0000 7f50d0a00640 1 --2- 172.21.15.45:0/2399032409 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f50cc074a60 0x7f50cc074ee0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f50b4009900 tx=0x7f50b402bc30 comp rx=0 tx=0).stop 2024-04-16T17:03:02.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:02.457+0000 7f50d0a00640 1 -- 172.21.15.45:0/2399032409 shutdown_connections 2024-04-16T17:03:02.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:02.457+0000 7f50d0a00640 1 --2- 172.21.15.45:0/2399032409 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f50cc074a60 0x7f50cc074ee0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:02.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:02.457+0000 7f50d0a00640 1 --2- 172.21.15.45:0/2399032409 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f50cc10e440 0x7f50cc110830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:02.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:02.457+0000 7f50d0a00640 1 --2- 172.21.15.45:0/2399032409 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f50cc076040 0x7f50cc074490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:02.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:02.457+0000 7f50d0a00640 1 -- 172.21.15.45:0/2399032409 >> 172.21.15.45:0/2399032409 conn(0x7f50cc0fd3e0 msgr2=0x7f50cc0ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:03:02.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:02.457+0000 7f50d0a00640 1 -- 172.21.15.45:0/2399032409 shutdown_connections 2024-04-16T17:03:02.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:02.457+0000 7f50d0a00640 1 -- 172.21.15.45:0/2399032409 wait complete. 2024-04-16T17:03:02.462 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:02.458+0000 7f50d0a00640 1 Processor -- start 2024-04-16T17:03:02.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:02.458+0000 7f50d0a00640 1 -- start start 2024-04-16T17:03:02.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:02.459+0000 7f50d0a00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f50cc074a60 0x7f50cc19e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:02.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:02.459+0000 7f50d0a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f50cc076040 0x7f50cc19ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:02.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:02.459+0000 7f50d0a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f50cc10e440 0x7f50cc1a5d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:02.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:02.459+0000 7f50d0a00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f50cc114150 con 0x7f50cc10e440 2024-04-16T17:03:02.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:02.459+0000 7f50d0a00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f50cc113fd0 con 0x7f50cc074a60 2024-04-16T17:03:02.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:02.459+0000 7f50d0a00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f50cc1142d0 con 0x7f50cc076040 2024-04-16T17:03:02.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:02.459+0000 7f50cb400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f50cc074a60 0x7f50cc19e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:02.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:02.459+0000 7f50caa00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f50cc076040 0x7f50cc19ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:02.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:02.459+0000 7f50caa00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f50cc076040 0x7f50cc19ece0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:37748/0 (socket says 172.21.15.45:37748) 2024-04-16T17:03:02.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:02.459+0000 7f50caa00640 1 -- 172.21.15.45:0/1820123576 learned_addr learned my addr 172.21.15.45:0/1820123576 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:03:02.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:02.459+0000 7f50cb400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f50cc074a60 0x7f50cc19e7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:60632/0 (socket says 172.21.15.45:60632) 2024-04-16T17:03:02.464 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:02.459+0000 7f50cbe00640 1 --2- 172.21.15.45:0/1820123576 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f50cc10e440 0x7f50cc1a5d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:02.464 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:02.460+0000 7f50cb400640 1 -- 172.21.15.45:0/1820123576 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f50cc076040 msgr2=0x7f50cc19ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:02.464 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:02.460+0000 7f50cb400640 1 --2- 172.21.15.45:0/1820123576 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f50cc076040 0x7f50cc19ece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:02.464 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:02.460+0000 7f50cb400640 1 -- 172.21.15.45:0/1820123576 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f50cc10e440 msgr2=0x7f50cc1a5d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:02.464 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:02.460+0000 7f50cb400640 1 --2- 172.21.15.45:0/1820123576 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f50cc10e440 0x7f50cc1a5d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:02.464 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:02.460+0000 7f50cb400640 1 -- 172.21.15.45:0/1820123576 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50cc1a63c0 con 0x7f50cc074a60 2024-04-16T17:03:02.464 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:02.460+0000 7f50cb400640 1 --2- 172.21.15.45:0/1820123576 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f50cc074a60 0x7f50cc19e7a0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f50bc00eb00 tx=0x7f50bc00c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:03:02.464 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:02.460+0000 7f50c3e00640 1 -- 172.21.15.45:0/1820123576 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50bc010430 con 0x7f50cc074a60 2024-04-16T17:03:02.464 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:02.460+0000 7f50c3e00640 1 -- 172.21.15.45:0/1820123576 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f50bc010ab0 con 0x7f50cc074a60 2024-04-16T17:03:02.464 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:02.460+0000 7f50c3e00640 1 -- 172.21.15.45:0/1820123576 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50bc017bb0 con 0x7f50cc074a60 2024-04-16T17:03:02.464 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:02.460+0000 7f50d0a00640 1 -- 172.21.15.45:0/1820123576 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50cc1a66b0 con 0x7f50cc074a60 2024-04-16T17:03:02.464 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:02.461+0000 7f50d0a00640 1 -- 172.21.15.45:0/1820123576 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50cc1a6b80 con 0x7f50cc074a60 2024-04-16T17:03:02.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:02.462+0000 7f50d0a00640 1 -- 172.21.15.45:0/1820123576 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f50cc104610 con 0x7f50cc074a60 2024-04-16T17:03:02.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:02.464+0000 7f50c3e00640 1 -- 172.21.15.45:0/1820123576 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f50bc010c60 con 0x7f50cc074a60 2024-04-16T17:03:02.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:02.465+0000 7f50c3e00640 1 --2- 172.21.15.45:0/1820123576 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f50b00a3c30 0x7f50b00a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:02.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:02.465+0000 7f50c3e00640 1 -- 172.21.15.45:0/1820123576 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f50bc0d08d0 con 0x7f50cc074a60 2024-04-16T17:03:02.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:02.465+0000 7f50caa00640 1 --2- 172.21.15.45:0/1820123576 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f50b00a3c30 0x7f50b00a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:02.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:02.466+0000 7f50caa00640 1 --2- 172.21.15.45:0/1820123576 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f50b00a3c30 0x7f50b00a60f0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f50b4006010 tx=0x7f50b4000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:03:02.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:02.469+0000 7f50c3e00640 1 -- 172.21.15.45:0/1820123576 <== mon.2 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f50bc09d6b0 con 0x7f50cc074a60 2024-04-16T17:03:02.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:02.721+0000 7f50d0a00640 1 -- 172.21.15.45:0/1820123576 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f50cc02d620 con 0x7f50cc074a60 2024-04-16T17:03:02.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:02.722+0000 7f50c3e00640 1 -- 172.21.15.45:0/1820123576 <== mon.2 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6358 (secure 0 0 0) 0x7f50bc09cfb0 con 0x7f50cc074a60 2024-04-16T17:03:02.724 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-16T17:03:02.724 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/1748842443","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":1748842443},{"type":"v1","addr":"172.21.15.45:6835","nonce":1748842443}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/1768242489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1768242489},{"type":"v1","addr":"172.21.15.117:6833","nonce":1768242489}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6833/3558902585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":3558902585},{"type":"v1","addr":"172.21.15.104:6833","nonce":3558902585}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24478,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/1308142903","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":1308142903},{"type":"v1","addr":"172.21.15.45:6837","nonce":1308142903}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24491,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6835/3430679937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":3430679937},{"type":"v1","addr":"172.21.15.104:6835","nonce":3430679937}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24499,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/2224123841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2224123841},{"type":"v1","addr":"172.21.15.117:6835","nonce":2224123841}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24511,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/2471356815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":2471356815},{"type":"v1","addr":"172.21.15.117:6837","nonce":2471356815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-04-16T17:03:02.724 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 8 2024-04-16T17:03:02.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:02.725+0000 7f50d0a00640 1 -- 172.21.15.45:0/1820123576 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f50b00a3c30 msgr2=0x7f50b00a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:02.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:02.725+0000 7f50d0a00640 1 --2- 172.21.15.45:0/1820123576 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f50b00a3c30 0x7f50b00a60f0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f50b4006010 tx=0x7f50b4000bc0 comp rx=0 tx=0).stop 2024-04-16T17:03:02.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:02.725+0000 7f50d0a00640 1 -- 172.21.15.45:0/1820123576 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f50cc074a60 msgr2=0x7f50cc19e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:02.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:02.725+0000 7f50d0a00640 1 --2- 172.21.15.45:0/1820123576 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f50cc074a60 0x7f50cc19e7a0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f50bc00eb00 tx=0x7f50bc00c2d0 comp rx=0 tx=0).stop 2024-04-16T17:03:02.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:02.725+0000 7f50d0a00640 1 -- 172.21.15.45:0/1820123576 shutdown_connections 2024-04-16T17:03:02.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:02.725+0000 7f50d0a00640 1 --2- 172.21.15.45:0/1820123576 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f50b00a3c30 0x7f50b00a60f0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:02.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:02.725+0000 7f50d0a00640 1 --2- 172.21.15.45:0/1820123576 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f50cc076040 0x7f50cc19ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:02.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:02.725+0000 7f50d0a00640 1 --2- 172.21.15.45:0/1820123576 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f50cc10e440 0x7f50cc1a5d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:02.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:02.725+0000 7f50d0a00640 1 --2- 172.21.15.45:0/1820123576 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f50cc074a60 0x7f50cc19e7a0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:02.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:02.725+0000 7f50d0a00640 1 -- 172.21.15.45:0/1820123576 >> 172.21.15.45:0/1820123576 conn(0x7f50cc0fd3e0 msgr2=0x7f50cc1017e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:03:02.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:02.725+0000 7f50d0a00640 1 -- 172.21.15.45:0/1820123576 shutdown_connections 2024-04-16T17:03:02.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:02.725+0000 7f50d0a00640 1 -- 172.21.15.45:0/1820123576 wait complete. 2024-04-16T17:03:02.979 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:03:02 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:03:02.979 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:03:02 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:03:02.979 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:03:02 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:03:02.979 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:03:02 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-16T17:03:02.979 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:03:02 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-16T17:03:02.979 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:03:02 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:03:02.979 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:03:02 smithi045 ceph-mon[31676]: Deploying daemon mds.h on smithi104 2024-04-16T17:03:02.979 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:03:02 smithi045 ceph-mon[31676]: mds.? [v2:172.21.15.117:6836/2471356815,v1:172.21.15.117:6837/2471356815] up:boot 2024-04-16T17:03:02.979 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:03:02 smithi045 ceph-mon[31676]: fsmap 7 up:standby 2024-04-16T17:03:02.979 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:03:02 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-16T17:03:03.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:03:02 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:03:03.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:03:02 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:03:03.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:03:02 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:03:03.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:03:02 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-16T17:03:03.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:03:02 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-16T17:03:03.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:03:02 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:03:03.057 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:03:02 smithi117 ceph-mon[36730]: Deploying daemon mds.h on smithi104 2024-04-16T17:03:03.057 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:03:02 smithi117 ceph-mon[36730]: mds.? [v2:172.21.15.117:6836/2471356815,v1:172.21.15.117:6837/2471356815] up:boot 2024-04-16T17:03:03.057 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:03:02 smithi117 ceph-mon[36730]: fsmap 7 up:standby 2024-04-16T17:03:03.057 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:03:02 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-16T17:03:03.088 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:03:02 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:03:03.088 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:03:02 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:03:03.088 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:03:02 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:03:03.088 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:03:02 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-16T17:03:03.088 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:03:02 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-16T17:03:03.088 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:03:02 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:03:03.088 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:03:02 smithi104 ceph-mon[36457]: Deploying daemon mds.h on smithi104 2024-04-16T17:03:03.088 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:03:02 smithi104 ceph-mon[36457]: mds.? [v2:172.21.15.117:6836/2471356815,v1:172.21.15.117:6837/2471356815] up:boot 2024-04-16T17:03:03.089 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:03:02 smithi104 ceph-mon[36457]: fsmap 7 up:standby 2024-04-16T17:03:03.089 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:03:02 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-16T17:03:03.978 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:03:03 smithi045 ceph-mon[31676]: pgmap v303: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T17:03:03.978 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:03:03 smithi045 ceph-mon[31676]: from='client.? 172.21.15.45:0/1820123576' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:03:04.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:03:03 smithi117 ceph-mon[36730]: pgmap v303: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T17:03:04.057 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:03:03 smithi117 ceph-mon[36730]: from='client.? 172.21.15.45:0/1820123576' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:03:04.099 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:03:03 smithi104 ceph-mon[36457]: pgmap v303: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T17:03:04.099 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:03:03 smithi104 ceph-mon[36457]: from='client.? 172.21.15.45:0/1820123576' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:03:04.099 INFO:journalctl@ceph.mds.h.smithi104.stdout:Apr 16 17:03:03 smithi104 systemd[1]: Starting Ceph mds.h for b1fecd38-fc11-11ee-bc90-c7b262605968... 2024-04-16T17:03:04.369 INFO:journalctl@ceph.mds.h.smithi104.stdout:Apr 16 17:03:04 smithi104 podman[67461]: 2024-04-16 17:03:04.001189235 +0000 UTC m=+0.017961875 image pull e5ce10bdffbeeffffd1125f67b369e2151f0f17cdec854e93b6adf9321f65907 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2 2024-04-16T17:03:04.369 INFO:journalctl@ceph.mds.h.smithi104.stdout:Apr 16 17:03:04 smithi104 podman[67461]: 2024-04-16 17:03:04.10737438 +0000 UTC m=+0.124147004 container create 664cad82bac4b0446c60c0ad8d361a02cf8d3a5f87e8c95af01034eaaf37cd2f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mds-h, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240409, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, ceph=True) 2024-04-16T17:03:04.369 INFO:journalctl@ceph.mds.h.smithi104.stdout:Apr 16 17:03:04 smithi104 podman[67461]: 2024-04-16 17:03:04.368663051 +0000 UTC m=+0.385435681 container init 664cad82bac4b0446c60c0ad8d361a02cf8d3a5f87e8c95af01034eaaf37cd2f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mds-h, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240409) 2024-04-16T17:03:04.678 INFO:journalctl@ceph.mds.h.smithi104.stdout:Apr 16 17:03:04 smithi104 podman[67461]: 2024-04-16 17:03:04.371382517 +0000 UTC m=+0.388155142 container start 664cad82bac4b0446c60c0ad8d361a02cf8d3a5f87e8c95af01034eaaf37cd2f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mds-h, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, org.label-schema.build-date=20240409, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2) 2024-04-16T17:03:04.679 INFO:journalctl@ceph.mds.h.smithi104.stdout:Apr 16 17:03:04 smithi104 ceph-mds[67481]: -- 172.21.15.104:0/3858904322 <== mon.2 v2:172.21.15.104:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56333bce3a40 con 0x56333bcb6800 2024-04-16T17:03:04.679 INFO:journalctl@ceph.mds.h.smithi104.stdout:Apr 16 17:03:04 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mds-h[67477]: starting mds.h at 2024-04-16T17:03:04.679 INFO:journalctl@ceph.mds.h.smithi104.stdout:Apr 16 17:03:04 smithi104 bash[67461]: 664cad82bac4b0446c60c0ad8d361a02cf8d3a5f87e8c95af01034eaaf37cd2f 2024-04-16T17:03:04.679 INFO:journalctl@ceph.mds.h.smithi104.stdout:Apr 16 17:03:04 smithi104 systemd[1]: Started Ceph mds.h for b1fecd38-fc11-11ee-bc90-c7b262605968. 2024-04-16T17:03:04.751 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-16T17:03:04.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:04.983+0000 7ff8afe00640 1 -- 172.21.15.45:0/3894597494 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff8b010a030 msgr2=0x7ff8b01108c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:04.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:04.983+0000 7ff8afe00640 1 --2- 172.21.15.45:0/3894597494 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff8b010a030 0x7ff8b01108c0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7ff8a8009ca0 tx=0x7ff8a802c100 comp rx=0 tx=0).stop 2024-04-16T17:03:04.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:04.983+0000 7ff8afe00640 1 -- 172.21.15.45:0/3894597494 shutdown_connections 2024-04-16T17:03:04.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:04.983+0000 7ff8afe00640 1 --2- 172.21.15.45:0/3894597494 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff8b010a030 0x7ff8b01108c0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:04.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:04.983+0000 7ff8afe00640 1 --2- 172.21.15.45:0/3894597494 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff8b0109670 0x7ff8b0109af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:04.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:04.983+0000 7ff8afe00640 1 --2- 172.21.15.45:0/3894597494 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff8b0108480 0x7ff8b01088a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:04.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:04.983+0000 7ff8afe00640 1 -- 172.21.15.45:0/3894597494 >> 172.21.15.45:0/3894597494 conn(0x7ff8b0077440 msgr2=0x7ff8b0079860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:03:04.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:04.984+0000 7ff8afe00640 1 -- 172.21.15.45:0/3894597494 shutdown_connections 2024-04-16T17:03:04.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:04.984+0000 7ff8afe00640 1 -- 172.21.15.45:0/3894597494 wait complete. 2024-04-16T17:03:04.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:04.985+0000 7ff8afe00640 1 Processor -- start 2024-04-16T17:03:04.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:04.985+0000 7ff8afe00640 1 -- start start 2024-04-16T17:03:04.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:04.985+0000 7ff8afe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff8b0108480 0x7ff8b019e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:04.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:04.985+0000 7ff8afe00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff8b0109670 0x7ff8b019ec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:04.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:04.986+0000 7ff8afe00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff8b010a030 0x7ff8b01a5c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:04.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:04.986+0000 7ff8afe00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7ff8b0114110 con 0x7ff8b0108480 2024-04-16T17:03:04.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:04.986+0000 7ff8afe00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7ff8b0113f90 con 0x7ff8b0109670 2024-04-16T17:03:04.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:04.986+0000 7ff8afe00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7ff8b0114290 con 0x7ff8b010a030 2024-04-16T17:03:04.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:04.986+0000 7ff8af400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff8b010a030 0x7ff8b01a5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:04.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:04.986+0000 7ff8aea00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff8b0108480 0x7ff8b019e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:04.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:04.986+0000 7ff8af400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff8b010a030 0x7ff8b01a5c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:37772/0 (socket says 172.21.15.45:37772) 2024-04-16T17:03:04.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:04.986+0000 7ff8af400640 1 -- 172.21.15.45:0/2018117423 learned_addr learned my addr 172.21.15.45:0/2018117423 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:03:04.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:04.986+0000 7ff8a7e00640 1 --2- 172.21.15.45:0/2018117423 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff8b0109670 0x7ff8b019ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:04.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:04.986+0000 7ff8af400640 1 -- 172.21.15.45:0/2018117423 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff8b0109670 msgr2=0x7ff8b019ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:04.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:04.986+0000 7ff8af400640 1 --2- 172.21.15.45:0/2018117423 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff8b0109670 0x7ff8b019ec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:04.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:04.986+0000 7ff8af400640 1 -- 172.21.15.45:0/2018117423 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff8b0108480 msgr2=0x7ff8b019e6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:04.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:04.986+0000 7ff8af400640 1 --2- 172.21.15.45:0/2018117423 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff8b0108480 0x7ff8b019e6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:04.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:04.986+0000 7ff8af400640 1 -- 172.21.15.45:0/2018117423 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8b01a6300 con 0x7ff8b010a030 2024-04-16T17:03:04.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:04.987+0000 7ff8af400640 1 --2- 172.21.15.45:0/2018117423 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff8b010a030 0x7ff8b01a5c90 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7ff8a802c640 tx=0x7ff8a8035d10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:03:04.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:04.987+0000 7ff8a7400640 1 -- 172.21.15.45:0/2018117423 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8a8039040 con 0x7ff8b010a030 2024-04-16T17:03:04.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:04.987+0000 7ff8a7400640 1 -- 172.21.15.45:0/2018117423 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff8a8044070 con 0x7ff8b010a030 2024-04-16T17:03:04.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:04.987+0000 7ff8afe00640 1 -- 172.21.15.45:0/2018117423 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8b01a6590 con 0x7ff8b010a030 2024-04-16T17:03:04.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:04.987+0000 7ff8a7400640 1 -- 172.21.15.45:0/2018117423 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8a80043c0 con 0x7ff8b010a030 2024-04-16T17:03:04.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:04.987+0000 7ff8afe00640 1 -- 172.21.15.45:0/2018117423 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8b01a6a80 con 0x7ff8b010a030 2024-04-16T17:03:04.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:04.988+0000 7ff8afe00640 1 -- 172.21.15.45:0/2018117423 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8b010f740 con 0x7ff8b010a030 2024-04-16T17:03:04.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:04.990+0000 7ff8a7400640 1 -- 172.21.15.45:0/2018117423 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7ff8a8004560 con 0x7ff8b010a030 2024-04-16T17:03:04.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:04.992+0000 7ff8a7400640 1 --2- 172.21.15.45:0/2018117423 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7ff8940a3bb0 0x7ff8940a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:04.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:04.992+0000 7ff8a7400640 1 -- 172.21.15.45:0/2018117423 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7ff8a80f2680 con 0x7ff8b010a030 2024-04-16T17:03:04.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:04.992+0000 7ff8aea00640 1 --2- 172.21.15.45:0/2018117423 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7ff8940a3bb0 0x7ff8940a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:04.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:04.993+0000 7ff8aea00640 1 --2- 172.21.15.45:0/2018117423 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7ff8940a3bb0 0x7ff8940a6070 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7ff8b01094d0 tx=0x7ff8a0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:03:04.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:04.994+0000 7ff8a7400640 1 -- 172.21.15.45:0/2018117423 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff8a80bf460 con 0x7ff8b010a030 2024-04-16T17:03:05.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:05.244+0000 7ff8afe00640 1 -- 172.21.15.45:0/2018117423 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff8b002d620 con 0x7ff8b010a030 2024-04-16T17:03:05.247 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-16T17:03:05.248 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/1748842443","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":1748842443},{"type":"v1","addr":"172.21.15.45:6835","nonce":1748842443}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/1768242489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1768242489},{"type":"v1","addr":"172.21.15.117:6833","nonce":1768242489}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6833/3558902585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":3558902585},{"type":"v1","addr":"172.21.15.104:6833","nonce":3558902585}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24478,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/1308142903","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":1308142903},{"type":"v1","addr":"172.21.15.45:6837","nonce":1308142903}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24491,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6835/3430679937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":3430679937},{"type":"v1","addr":"172.21.15.104:6835","nonce":3430679937}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24499,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/2224123841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2224123841},{"type":"v1","addr":"172.21.15.117:6835","nonce":2224123841}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24511,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/2471356815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":2471356815},{"type":"v1","addr":"172.21.15.117:6837","nonce":2471356815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24524,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6837/2647756509","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":2647756509},{"type":"v1","addr":"172.21.15.104:6837","nonce":2647756509}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-04-16T17:03:05.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:05.245+0000 7ff8a7400640 1 -- 172.21.15.45:0/2018117423 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+7186 (secure 0 0 0) 0x7ff8a80098b0 con 0x7ff8b010a030 2024-04-16T17:03:05.248 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 9 2024-04-16T17:03:05.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:05.248+0000 7ff8afe00640 1 -- 172.21.15.45:0/2018117423 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7ff8940a3bb0 msgr2=0x7ff8940a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:05.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:05.248+0000 7ff8afe00640 1 --2- 172.21.15.45:0/2018117423 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7ff8940a3bb0 0x7ff8940a6070 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7ff8b01094d0 tx=0x7ff8a0005f90 comp rx=0 tx=0).stop 2024-04-16T17:03:05.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:05.248+0000 7ff8afe00640 1 -- 172.21.15.45:0/2018117423 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff8b010a030 msgr2=0x7ff8b01a5c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:05.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:05.248+0000 7ff8afe00640 1 --2- 172.21.15.45:0/2018117423 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff8b010a030 0x7ff8b01a5c90 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7ff8a802c640 tx=0x7ff8a8035d10 comp rx=0 tx=0).stop 2024-04-16T17:03:05.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:05.248+0000 7ff8afe00640 1 -- 172.21.15.45:0/2018117423 shutdown_connections 2024-04-16T17:03:05.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:05.248+0000 7ff8afe00640 1 --2- 172.21.15.45:0/2018117423 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7ff8940a3bb0 0x7ff8940a6070 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:05.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:05.248+0000 7ff8afe00640 1 --2- 172.21.15.45:0/2018117423 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff8b010a030 0x7ff8b01a5c90 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:05.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:05.248+0000 7ff8afe00640 1 --2- 172.21.15.45:0/2018117423 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff8b0109670 0x7ff8b019ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:05.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:05.248+0000 7ff8afe00640 1 --2- 172.21.15.45:0/2018117423 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff8b0108480 0x7ff8b019e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:05.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:05.248+0000 7ff8afe00640 1 -- 172.21.15.45:0/2018117423 >> 172.21.15.45:0/2018117423 conn(0x7ff8b0077440 msgr2=0x7ff8b010c910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:03:05.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:05.248+0000 7ff8afe00640 1 -- 172.21.15.45:0/2018117423 shutdown_connections 2024-04-16T17:03:05.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:05.248+0000 7ff8afe00640 1 -- 172.21.15.45:0/2018117423 wait complete. 2024-04-16T17:03:05.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:03:05 smithi117 ceph-mon[36730]: pgmap v304: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T17:03:05.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:03:05 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:03:05.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:03:05 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:03:05.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:03:05 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:03:05.807 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:03:05 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-16T17:03:05.807 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:03:05 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-16T17:03:05.807 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:03:05 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:03:05.807 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:03:05 smithi117 ceph-mon[36730]: Deploying daemon mds.i on smithi117 2024-04-16T17:03:05.807 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:03:05 smithi117 ceph-mon[36730]: mds.? [v2:172.21.15.104:6836/2647756509,v1:172.21.15.104:6837/2647756509] up:boot 2024-04-16T17:03:05.807 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:03:05 smithi117 ceph-mon[36730]: fsmap 8 up:standby 2024-04-16T17:03:05.807 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:03:05 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-16T17:03:05.807 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:03:05 smithi117 ceph-mon[36730]: from='client.? 172.21.15.45:0/2018117423' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:03:05.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:03:05 smithi104 ceph-mon[36457]: pgmap v304: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T17:03:05.929 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:03:05 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:03:05.929 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:03:05 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:03:05.929 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:03:05 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:03:05.929 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:03:05 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-16T17:03:05.929 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:03:05 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-16T17:03:05.929 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:03:05 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:03:05.929 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:03:05 smithi104 ceph-mon[36457]: Deploying daemon mds.i on smithi117 2024-04-16T17:03:05.929 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:03:05 smithi104 ceph-mon[36457]: mds.? [v2:172.21.15.104:6836/2647756509,v1:172.21.15.104:6837/2647756509] up:boot 2024-04-16T17:03:05.929 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:03:05 smithi104 ceph-mon[36457]: fsmap 8 up:standby 2024-04-16T17:03:05.929 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:03:05 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-16T17:03:05.929 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:03:05 smithi104 ceph-mon[36457]: from='client.? 172.21.15.45:0/2018117423' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:03:05.978 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:03:05 smithi045 ceph-mon[31676]: pgmap v304: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T17:03:05.979 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:03:05 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:03:05.979 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:03:05 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:03:05.979 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:03:05 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:03:05.979 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:03:05 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-16T17:03:05.979 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:03:05 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-16T17:03:05.979 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:03:05 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:03:05.979 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:03:05 smithi045 ceph-mon[31676]: Deploying daemon mds.i on smithi117 2024-04-16T17:03:05.979 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:03:05 smithi045 ceph-mon[31676]: mds.? [v2:172.21.15.104:6836/2647756509,v1:172.21.15.104:6837/2647756509] up:boot 2024-04-16T17:03:05.979 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:03:05 smithi045 ceph-mon[31676]: fsmap 8 up:standby 2024-04-16T17:03:05.979 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:03:05 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-16T17:03:05.979 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:03:05 smithi045 ceph-mon[31676]: from='client.? 172.21.15.45:0/2018117423' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:03:06.976 INFO:journalctl@ceph.mds.i.smithi117.stdout:Apr 16 17:03:06 smithi117 systemd[1]: Starting Ceph mds.i for b1fecd38-fc11-11ee-bc90-c7b262605968... 2024-04-16T17:03:07.243 INFO:journalctl@ceph.mds.i.smithi117.stdout:Apr 16 17:03:06 smithi117 podman[67959]: 2024-04-16 17:03:06.877253075 +0000 UTC m=+0.017342779 image pull e5ce10bdffbeeffffd1125f67b369e2151f0f17cdec854e93b6adf9321f65907 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2 2024-04-16T17:03:07.244 INFO:journalctl@ceph.mds.i.smithi117.stdout:Apr 16 17:03:06 smithi117 podman[67959]: 2024-04-16 17:03:06.988798867 +0000 UTC m=+0.128888574 container create 02de05e9e9c27b0cde2597955d7d8807e919a9894e529a77f59804366d42bca2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mds-i, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240409) 2024-04-16T17:03:07.244 INFO:journalctl@ceph.mds.i.smithi117.stdout:Apr 16 17:03:07 smithi117 podman[67959]: 2024-04-16 17:03:07.2165472 +0000 UTC m=+0.356636911 container init 02de05e9e9c27b0cde2597955d7d8807e919a9894e529a77f59804366d42bca2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mds-i, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240409, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-04-16T17:03:07.244 INFO:journalctl@ceph.mds.i.smithi117.stdout:Apr 16 17:03:07 smithi117 podman[67959]: 2024-04-16 17:03:07.219222087 +0000 UTC m=+0.359311801 container start 02de05e9e9c27b0cde2597955d7d8807e919a9894e529a77f59804366d42bca2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mds-i, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, io.buildah.version=1.33.5, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240409, ceph=True) 2024-04-16T17:03:07.272 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-16T17:03:07.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:07.504+0000 7fcad0000640 1 -- 172.21.15.45:0/338590314 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcac8105470 msgr2=0x7fcac81058f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:07.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:07.504+0000 7fcad0000640 1 --2- 172.21.15.45:0/338590314 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcac8105470 0x7fcac81058f0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fcab4009ca0 tx=0x7fcab402c100 comp rx=0 tx=0).stop 2024-04-16T17:03:07.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:07.504+0000 7fcad0000640 1 -- 172.21.15.45:0/338590314 shutdown_connections 2024-04-16T17:03:07.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:07.504+0000 7fcad0000640 1 --2- 172.21.15.45:0/338590314 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcac8105e30 0x7fcac810c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:07.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:07.504+0000 7fcad0000640 1 --2- 172.21.15.45:0/338590314 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcac8105470 0x7fcac81058f0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:07.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:07.504+0000 7fcad0000640 1 --2- 172.21.15.45:0/338590314 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcac8104280 0x7fcac81046a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:07.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:07.504+0000 7fcad0000640 1 -- 172.21.15.45:0/338590314 >> 172.21.15.45:0/338590314 conn(0x7fcac80ff7f0 msgr2=0x7fcac8101c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:03:07.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:07.504+0000 7fcad0000640 1 -- 172.21.15.45:0/338590314 shutdown_connections 2024-04-16T17:03:07.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:07.504+0000 7fcad0000640 1 -- 172.21.15.45:0/338590314 wait complete. 2024-04-16T17:03:07.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:07.505+0000 7fcad0000640 1 Processor -- start 2024-04-16T17:03:07.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:07.505+0000 7fcad0000640 1 -- start start 2024-04-16T17:03:07.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:07.506+0000 7fcad0000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcac8104280 0x7fcac819a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:07.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:07.506+0000 7fcad0000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcac8105470 0x7fcac819aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:07.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:07.506+0000 7fcad0000640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcac8105e30 0x7fcac81a1af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:07.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:07.506+0000 7fcad0000640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fcac810ff10 con 0x7fcac8104280 2024-04-16T17:03:07.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:07.506+0000 7fcad0000640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fcac810fd90 con 0x7fcac8105e30 2024-04-16T17:03:07.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:07.506+0000 7fcad0000640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fcac8110090 con 0x7fcac8105470 2024-04-16T17:03:07.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:07.506+0000 7fcacd600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcac8104280 0x7fcac819a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:07.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:07.506+0000 7fcaccc00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcac8105470 0x7fcac819aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:07.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:07.506+0000 7fcacd600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcac8104280 0x7fcac819a540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39182/0 (socket says 172.21.15.45:39182) 2024-04-16T17:03:07.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:07.506+0000 7fcacd600640 1 -- 172.21.15.45:0/1601072760 learned_addr learned my addr 172.21.15.45:0/1601072760 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:03:07.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:07.506+0000 7fcace000640 1 --2- 172.21.15.45:0/1601072760 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcac8105e30 0x7fcac81a1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:07.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:07.507+0000 7fcacd600640 1 -- 172.21.15.45:0/1601072760 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcac8105470 msgr2=0x7fcac819aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:07.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:07.507+0000 7fcacd600640 1 --2- 172.21.15.45:0/1601072760 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcac8105470 0x7fcac819aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:07.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:07.507+0000 7fcacd600640 1 -- 172.21.15.45:0/1601072760 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcac8105e30 msgr2=0x7fcac81a1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:07.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:07.507+0000 7fcacd600640 1 --2- 172.21.15.45:0/1601072760 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcac8105e30 0x7fcac81a1af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:07.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:07.507+0000 7fcacd600640 1 -- 172.21.15.45:0/1601072760 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcac81a2160 con 0x7fcac8104280 2024-04-16T17:03:07.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:07.507+0000 7fcace000640 1 --2- 172.21.15.45:0/1601072760 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcac8105e30 0x7fcac81a1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:03:07.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:07.507+0000 7fcaccc00640 1 --2- 172.21.15.45:0/1601072760 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcac8105470 0x7fcac819aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:03:07.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:07.507+0000 7fcacd600640 1 --2- 172.21.15.45:0/1601072760 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcac8104280 0x7fcac819a540 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fcac000e9c0 tx=0x7fcac000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:03:07.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:07.507+0000 7fcac6000640 1 -- 172.21.15.45:0/1601072760 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcac000cd80 con 0x7fcac8104280 2024-04-16T17:03:07.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:07.507+0000 7fcac6000640 1 -- 172.21.15.45:0/1601072760 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcac00035f0 con 0x7fcac8104280 2024-04-16T17:03:07.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:07.507+0000 7fcad0000640 1 -- 172.21.15.45:0/1601072760 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcac81a2450 con 0x7fcac8104280 2024-04-16T17:03:07.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:07.507+0000 7fcac6000640 1 -- 172.21.15.45:0/1601072760 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcac0010690 con 0x7fcac8104280 2024-04-16T17:03:07.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:07.507+0000 7fcad0000640 1 -- 172.21.15.45:0/1601072760 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcac81a29c0 con 0x7fcac8104280 2024-04-16T17:03:07.512 INFO:journalctl@ceph.mds.i.smithi117.stdout:Apr 16 17:03:07 smithi117 ceph-mds[67979]: -- 172.21.15.117:0/1296907540 <== mon.1 v2:172.21.15.117:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55965e63fa40 con 0x55965e612800 2024-04-16T17:03:07.512 INFO:journalctl@ceph.mds.i.smithi117.stdout:Apr 16 17:03:07 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mds-i[67974]: starting mds.i at 2024-04-16T17:03:07.512 INFO:journalctl@ceph.mds.i.smithi117.stdout:Apr 16 17:03:07 smithi117 bash[67959]: 02de05e9e9c27b0cde2597955d7d8807e919a9894e529a77f59804366d42bca2 2024-04-16T17:03:07.512 INFO:journalctl@ceph.mds.i.smithi117.stdout:Apr 16 17:03:07 smithi117 systemd[1]: Started Ceph mds.i for b1fecd38-fc11-11ee-bc90-c7b262605968. 2024-04-16T17:03:07.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:07.509+0000 7fcad0000640 1 -- 172.21.15.45:0/1601072760 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcac810b5a0 con 0x7fcac8104280 2024-04-16T17:03:07.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:07.512+0000 7fcac6000640 1 -- 172.21.15.45:0/1601072760 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fcac00037a0 con 0x7fcac8104280 2024-04-16T17:03:07.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:07.514+0000 7fcac6000640 1 --2- 172.21.15.45:0/1601072760 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fcaac0a3c30 0x7fcaac0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:07.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:07.514+0000 7fcac6000640 1 -- 172.21.15.45:0/1601072760 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fcac00d04b0 con 0x7fcac8104280 2024-04-16T17:03:07.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:07.514+0000 7fcaccc00640 1 --2- 172.21.15.45:0/1601072760 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fcaac0a3c30 0x7fcaac0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:07.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:07.515+0000 7fcaccc00640 1 --2- 172.21.15.45:0/1601072760 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fcaac0a3c30 0x7fcaac0a60f0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fcab402c710 tx=0x7fcab4005ef0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:03:07.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:07.517+0000 7fcac6000640 1 -- 172.21.15.45:0/1601072760 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fcac009d320 con 0x7fcac8104280 2024-04-16T17:03:07.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:03:07 smithi045 ceph-mon[31676]: pgmap v305: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T17:03:07.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:03:07 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:03:07.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:03:07 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:03:07.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:03:07 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:03:07.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:03:07 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-16T17:03:07.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:03:07 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-16T17:03:07.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:03:07 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:03:07.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:07.771+0000 7fcad0000640 1 -- 172.21.15.45:0/1601072760 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcac802d620 con 0x7fcac8104280 2024-04-16T17:03:07.774 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:07.772+0000 7fcac6000640 1 -- 172.21.15.45:0/1601072760 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+8016 (secure 0 0 0) 0x7fcac009cc20 con 0x7fcac8104280 2024-04-16T17:03:07.774 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-16T17:03:07.775 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/1748842443","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":1748842443},{"type":"v1","addr":"172.21.15.45:6835","nonce":1748842443}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/1768242489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1768242489},{"type":"v1","addr":"172.21.15.117:6833","nonce":1768242489}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6833/3558902585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":3558902585},{"type":"v1","addr":"172.21.15.104:6833","nonce":3558902585}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24478,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/1308142903","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":1308142903},{"type":"v1","addr":"172.21.15.45:6837","nonce":1308142903}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24491,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6835/3430679937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":3430679937},{"type":"v1","addr":"172.21.15.104:6835","nonce":3430679937}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24499,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/2224123841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2224123841},{"type":"v1","addr":"172.21.15.117:6835","nonce":2224123841}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24511,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/2471356815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":2471356815},{"type":"v1","addr":"172.21.15.117:6837","nonce":2471356815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24524,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6837/2647756509","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":2647756509},{"type":"v1","addr":"172.21.15.104:6837","nonce":2647756509}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24532,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6839/2378675949","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":2378675949},{"type":"v1","addr":"172.21.15.117:6839","nonce":2378675949}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-04-16T17:03:07.775 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 10 2024-04-16T17:03:07.777 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:07.774+0000 7fcad0000640 1 -- 172.21.15.45:0/1601072760 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fcaac0a3c30 msgr2=0x7fcaac0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:07.777 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:07.774+0000 7fcad0000640 1 --2- 172.21.15.45:0/1601072760 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fcaac0a3c30 0x7fcaac0a60f0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fcab402c710 tx=0x7fcab4005ef0 comp rx=0 tx=0).stop 2024-04-16T17:03:07.777 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:07.775+0000 7fcad0000640 1 -- 172.21.15.45:0/1601072760 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcac8104280 msgr2=0x7fcac819a540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:07.777 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:07.775+0000 7fcad0000640 1 --2- 172.21.15.45:0/1601072760 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcac8104280 0x7fcac819a540 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fcac000e9c0 tx=0x7fcac000ee80 comp rx=0 tx=0).stop 2024-04-16T17:03:07.777 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:07.775+0000 7fcad0000640 1 -- 172.21.15.45:0/1601072760 shutdown_connections 2024-04-16T17:03:07.777 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:07.775+0000 7fcad0000640 1 --2- 172.21.15.45:0/1601072760 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fcaac0a3c30 0x7fcaac0a60f0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:07.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:07.775+0000 7fcad0000640 1 --2- 172.21.15.45:0/1601072760 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcac8105470 0x7fcac819aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:07.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:07.775+0000 7fcad0000640 1 --2- 172.21.15.45:0/1601072760 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcac8105e30 0x7fcac81a1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:07.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:07.775+0000 7fcad0000640 1 --2- 172.21.15.45:0/1601072760 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcac8104280 0x7fcac819a540 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:07.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:07.775+0000 7fcad0000640 1 -- 172.21.15.45:0/1601072760 >> 172.21.15.45:0/1601072760 conn(0x7fcac80ff7f0 msgr2=0x7fcac8108770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:03:07.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:07.775+0000 7fcad0000640 1 -- 172.21.15.45:0/1601072760 shutdown_connections 2024-04-16T17:03:07.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:07.775+0000 7fcad0000640 1 -- 172.21.15.45:0/1601072760 wait complete. 2024-04-16T17:03:07.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:03:07 smithi117 ceph-mon[36730]: pgmap v305: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T17:03:07.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:03:07 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:03:07.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:03:07 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:03:07.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:03:07 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:03:07.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:03:07 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-16T17:03:07.807 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:03:07 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-16T17:03:07.807 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:03:07 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:03:07.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:03:07 smithi104 ceph-mon[36457]: pgmap v305: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T17:03:07.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:03:07 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:03:07.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:03:07 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:03:07.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:03:07 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:03:07.929 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:03:07 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-16T17:03:07.929 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:03:07 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-16T17:03:07.929 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:03:07 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:03:08.685 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:03:08 smithi045 ceph-mon[31676]: Deploying daemon mds.g on smithi045 2024-04-16T17:03:08.685 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:03:08 smithi045 ceph-mon[31676]: mds.? [v2:172.21.15.117:6838/2378675949,v1:172.21.15.117:6839/2378675949] up:boot 2024-04-16T17:03:08.685 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:03:08 smithi045 ceph-mon[31676]: fsmap 9 up:standby 2024-04-16T17:03:08.685 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:03:08 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-16T17:03:08.685 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:03:08 smithi045 ceph-mon[31676]: from='client.? 172.21.15.45:0/1601072760' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:03:08.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:03:08 smithi117 ceph-mon[36730]: Deploying daemon mds.g on smithi045 2024-04-16T17:03:08.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:03:08 smithi117 ceph-mon[36730]: mds.? [v2:172.21.15.117:6838/2378675949,v1:172.21.15.117:6839/2378675949] up:boot 2024-04-16T17:03:08.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:03:08 smithi117 ceph-mon[36730]: fsmap 9 up:standby 2024-04-16T17:03:08.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:03:08 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-16T17:03:08.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:03:08 smithi117 ceph-mon[36730]: from='client.? 172.21.15.45:0/1601072760' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:03:08.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:03:08 smithi104 ceph-mon[36457]: Deploying daemon mds.g on smithi045 2024-04-16T17:03:08.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:03:08 smithi104 ceph-mon[36457]: mds.? [v2:172.21.15.117:6838/2378675949,v1:172.21.15.117:6839/2378675949] up:boot 2024-04-16T17:03:08.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:03:08 smithi104 ceph-mon[36457]: fsmap 9 up:standby 2024-04-16T17:03:08.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:03:08 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-16T17:03:08.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:03:08 smithi104 ceph-mon[36457]: from='client.? 172.21.15.45:0/1601072760' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:03:09.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:03:09 smithi045 ceph-mon[31676]: pgmap v306: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T17:03:09.807 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-16T17:03:09.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:03:09 smithi104 ceph-mon[36457]: pgmap v306: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T17:03:10.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:10.040+0000 7f2d33400640 1 -- 172.21.15.45:0/3341906546 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2d2c0769c0 msgr2=0x7f2d2c076de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:10.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:10.040+0000 7f2d33400640 1 --2- 172.21.15.45:0/3341906546 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2d2c0769c0 0x7f2d2c076de0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f2d20009900 tx=0x7f2d2002bc40 comp rx=0 tx=0).stop 2024-04-16T17:03:10.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:10.040+0000 7f2d33400640 1 -- 172.21.15.45:0/3341906546 shutdown_connections 2024-04-16T17:03:10.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:10.040+0000 7f2d33400640 1 --2- 172.21.15.45:0/3341906546 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2d2c0785a0 0x7f2d2c078e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:10.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:10.040+0000 7f2d33400640 1 --2- 172.21.15.45:0/3341906546 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2d2c077c00 0x7f2d2c078060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:10.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:10.040+0000 7f2d33400640 1 --2- 172.21.15.45:0/3341906546 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2d2c0769c0 0x7f2d2c076de0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:10.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:10.040+0000 7f2d33400640 1 -- 172.21.15.45:0/3341906546 >> 172.21.15.45:0/3341906546 conn(0x7f2d2c0ff7f0 msgr2=0x7f2d2c101c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:03:10.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:10.040+0000 7f2d33400640 1 -- 172.21.15.45:0/3341906546 shutdown_connections 2024-04-16T17:03:10.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:10.040+0000 7f2d33400640 1 -- 172.21.15.45:0/3341906546 wait complete. 2024-04-16T17:03:10.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:10.041+0000 7f2d33400640 1 Processor -- start 2024-04-16T17:03:10.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:10.041+0000 7f2d33400640 1 -- start start 2024-04-16T17:03:10.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:10.042+0000 7f2d33400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2d2c0769c0 0x7f2d2c19e980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:10.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:10.042+0000 7f2d33400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2d2c077c00 0x7f2d2c19eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:10.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:10.042+0000 7f2d33400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2d2c0785a0 0x7f2d2c1a5f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:10.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:10.042+0000 7f2d33400640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f2d2c114350 con 0x7f2d2c077c00 2024-04-16T17:03:10.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:10.042+0000 7f2d33400640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f2d2c1141d0 con 0x7f2d2c0785a0 2024-04-16T17:03:10.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:10.042+0000 7f2d33400640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f2d2c1144d0 con 0x7f2d2c0769c0 2024-04-16T17:03:10.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:10.042+0000 7f2d2be00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2d2c077c00 0x7f2d2c19eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:10.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:10.042+0000 7f2d31400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2d2c0785a0 0x7f2d2c1a5f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:10.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:10.042+0000 7f2d2be00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2d2c077c00 0x7f2d2c19eec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39804/0 (socket says 172.21.15.45:39804) 2024-04-16T17:03:10.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:10.042+0000 7f2d2be00640 1 -- 172.21.15.45:0/267065548 learned_addr learned my addr 172.21.15.45:0/267065548 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:03:10.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:10.042+0000 7f2d30a00640 1 --2- 172.21.15.45:0/267065548 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2d2c0769c0 0x7f2d2c19e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:10.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:10.042+0000 7f2d31400640 1 -- 172.21.15.45:0/267065548 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2d2c0769c0 msgr2=0x7f2d2c19e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:10.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:10.042+0000 7f2d31400640 1 --2- 172.21.15.45:0/267065548 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2d2c0769c0 0x7f2d2c19e980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:10.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:10.042+0000 7f2d31400640 1 -- 172.21.15.45:0/267065548 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2d2c077c00 msgr2=0x7f2d2c19eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:10.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:10.042+0000 7f2d31400640 1 --2- 172.21.15.45:0/267065548 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2d2c077c00 0x7f2d2c19eec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:10.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:10.042+0000 7f2d31400640 1 -- 172.21.15.45:0/267065548 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d2c1a65a0 con 0x7f2d2c0785a0 2024-04-16T17:03:10.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:10.042+0000 7f2d2be00640 1 --2- 172.21.15.45:0/267065548 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2d2c077c00 0x7f2d2c19eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:03:10.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:10.043+0000 7f2d31400640 1 --2- 172.21.15.45:0/267065548 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2d2c0785a0 0x7f2d2c1a5f30 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f2d2400e940 tx=0x7f2d2400ee00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:03:10.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:10.043+0000 7f2d29600640 1 -- 172.21.15.45:0/267065548 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d24004d20 con 0x7f2d2c0785a0 2024-04-16T17:03:10.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:10.043+0000 7f2d29600640 1 -- 172.21.15.45:0/267065548 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2d24005020 con 0x7f2d2c0785a0 2024-04-16T17:03:10.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:10.043+0000 7f2d33400640 1 -- 172.21.15.45:0/267065548 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d2c1a6890 con 0x7f2d2c0785a0 2024-04-16T17:03:10.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:10.043+0000 7f2d29600640 1 -- 172.21.15.45:0/267065548 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d240116b0 con 0x7f2d2c0785a0 2024-04-16T17:03:10.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:10.043+0000 7f2d33400640 1 -- 172.21.15.45:0/267065548 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d2c1a6d90 con 0x7f2d2c0785a0 2024-04-16T17:03:10.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:10.045+0000 7f2d33400640 1 -- 172.21.15.45:0/267065548 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d2c110d80 con 0x7f2d2c0785a0 2024-04-16T17:03:10.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:10.047+0000 7f2d29600640 1 -- 172.21.15.45:0/267065548 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f2d240054d0 con 0x7f2d2c0785a0 2024-04-16T17:03:10.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:10.048+0000 7f2d29600640 1 --2- 172.21.15.45:0/267065548 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f2d140a3bb0 0x7f2d140a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:10.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:10.048+0000 7f2d29600640 1 -- 172.21.15.45:0/267065548 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f2d240d0060 con 0x7f2d2c0785a0 2024-04-16T17:03:10.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:10.048+0000 7f2d30a00640 1 --2- 172.21.15.45:0/267065548 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f2d140a3bb0 0x7f2d140a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:10.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:10.049+0000 7f2d30a00640 1 --2- 172.21.15.45:0/267065548 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f2d140a3bb0 0x7f2d140a6070 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f2d2002c250 tx=0x7f2d2002cf90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:03:10.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:10.053+0000 7f2d29600640 1 -- 172.21.15.45:0/267065548 <== mon.2 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2d24016030 con 0x7f2d2c0785a0 2024-04-16T17:03:10.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:03:09 smithi117 ceph-mon[36730]: pgmap v306: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T17:03:10.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:10.322+0000 7f2d33400640 1 -- 172.21.15.45:0/267065548 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2d2c062c00 con 0x7f2d2c0785a0 2024-04-16T17:03:10.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:10.323+0000 7f2d29600640 1 -- 172.21.15.45:0/267065548 <== mon.2 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+8016 (secure 0 0 0) 0x7f2d2409c7f0 con 0x7f2d2c0785a0 2024-04-16T17:03:10.325 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-16T17:03:10.325 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/1748842443","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":1748842443},{"type":"v1","addr":"172.21.15.45:6835","nonce":1748842443}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/1768242489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1768242489},{"type":"v1","addr":"172.21.15.117:6833","nonce":1768242489}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6833/3558902585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":3558902585},{"type":"v1","addr":"172.21.15.104:6833","nonce":3558902585}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24478,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/1308142903","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":1308142903},{"type":"v1","addr":"172.21.15.45:6837","nonce":1308142903}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24491,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6835/3430679937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":3430679937},{"type":"v1","addr":"172.21.15.104:6835","nonce":3430679937}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24499,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/2224123841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2224123841},{"type":"v1","addr":"172.21.15.117:6835","nonce":2224123841}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24511,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/2471356815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":2471356815},{"type":"v1","addr":"172.21.15.117:6837","nonce":2471356815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24524,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6837/2647756509","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":2647756509},{"type":"v1","addr":"172.21.15.104:6837","nonce":2647756509}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24532,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6839/2378675949","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":2378675949},{"type":"v1","addr":"172.21.15.117:6839","nonce":2378675949}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-04-16T17:03:10.325 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 10 2024-04-16T17:03:10.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:10.326+0000 7f2d33400640 1 -- 172.21.15.45:0/267065548 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f2d140a3bb0 msgr2=0x7f2d140a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:10.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:10.326+0000 7f2d33400640 1 --2- 172.21.15.45:0/267065548 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f2d140a3bb0 0x7f2d140a6070 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f2d2002c250 tx=0x7f2d2002cf90 comp rx=0 tx=0).stop 2024-04-16T17:03:10.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:10.326+0000 7f2d33400640 1 -- 172.21.15.45:0/267065548 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2d2c0785a0 msgr2=0x7f2d2c1a5f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:10.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:10.326+0000 7f2d33400640 1 --2- 172.21.15.45:0/267065548 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2d2c0785a0 0x7f2d2c1a5f30 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f2d2400e940 tx=0x7f2d2400ee00 comp rx=0 tx=0).stop 2024-04-16T17:03:10.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:10.326+0000 7f2d33400640 1 -- 172.21.15.45:0/267065548 shutdown_connections 2024-04-16T17:03:10.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:10.326+0000 7f2d33400640 1 --2- 172.21.15.45:0/267065548 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f2d140a3bb0 0x7f2d140a6070 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:10.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:10.326+0000 7f2d33400640 1 --2- 172.21.15.45:0/267065548 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2d2c0785a0 0x7f2d2c1a5f30 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:10.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:10.326+0000 7f2d33400640 1 --2- 172.21.15.45:0/267065548 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2d2c077c00 0x7f2d2c19eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:10.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:10.326+0000 7f2d33400640 1 --2- 172.21.15.45:0/267065548 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2d2c0769c0 0x7f2d2c19e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:10.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:10.326+0000 7f2d33400640 1 -- 172.21.15.45:0/267065548 >> 172.21.15.45:0/267065548 conn(0x7f2d2c0ff7f0 msgr2=0x7f2d2c100f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:03:10.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:10.326+0000 7f2d33400640 1 -- 172.21.15.45:0/267065548 shutdown_connections 2024-04-16T17:03:10.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:10.326+0000 7f2d33400640 1 -- 172.21.15.45:0/267065548 wait complete. 2024-04-16T17:03:10.722 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:03:10 smithi045 ceph-mon[31676]: from='client.? 172.21.15.45:0/267065548' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:03:10.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:03:10 smithi104 ceph-mon[36457]: from='client.? 172.21.15.45:0/267065548' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:03:11.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:03:10 smithi117 ceph-mon[36730]: from='client.? 172.21.15.45:0/267065548' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:03:11.345 INFO:journalctl@ceph.mds.g.smithi045.stdout:Apr 16 17:03:11 smithi045 podman[72984]: 2024-04-16 17:03:11.131124139 +0000 UTC m=+0.157656970 container create 0cf613ae6b13474562a8d1843026f3ca9a669b3b6a1ea1583c2c761e145a7b80 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mds-g, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.build-date=20240409, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-04-16T17:03:11.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:03:11 smithi045 ceph-mon[31676]: pgmap v307: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T17:03:11.729 INFO:journalctl@ceph.mds.g.smithi045.stdout:Apr 16 17:03:11 smithi045 podman[72984]: 2024-04-16 17:03:11.444879178 +0000 UTC m=+0.471412016 container init 0cf613ae6b13474562a8d1843026f3ca9a669b3b6a1ea1583c2c761e145a7b80 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mds-g, GIT_BRANCH=HEAD, org.label-schema.build-date=20240409, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-16T17:03:11.729 INFO:journalctl@ceph.mds.g.smithi045.stdout:Apr 16 17:03:11 smithi045 podman[72984]: 2024-04-16 17:03:11.456618629 +0000 UTC m=+0.483151493 container start 0cf613ae6b13474562a8d1843026f3ca9a669b3b6a1ea1583c2c761e145a7b80 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mds-g, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240409, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-04-16T17:03:11.729 INFO:journalctl@ceph.mds.g.smithi045.stdout:Apr 16 17:03:11 smithi045 ceph-mds[73009]: -- 172.21.15.45:0/2111675834 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5609303824e0 con 0x560930359000 2024-04-16T17:03:11.729 INFO:journalctl@ceph.mds.g.smithi045.stdout:Apr 16 17:03:11 smithi045 ceph-mds[73009]: -- 172.21.15.45:0/2111675834 <== mon.0 v2:172.21.15.45:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x560930385a40 con 0x560930359000 2024-04-16T17:03:11.729 INFO:journalctl@ceph.mds.g.smithi045.stdout:Apr 16 17:03:11 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mds-g[73003]: starting mds.g at 2024-04-16T17:03:11.729 INFO:journalctl@ceph.mds.g.smithi045.stdout:Apr 16 17:03:11 smithi045 bash[72984]: 0cf613ae6b13474562a8d1843026f3ca9a669b3b6a1ea1583c2c761e145a7b80 2024-04-16T17:03:11.730 INFO:journalctl@ceph.mds.g.smithi045.stdout:Apr 16 17:03:11 smithi045 systemd[1]: Started Ceph mds.g for b1fecd38-fc11-11ee-bc90-c7b262605968. 2024-04-16T17:03:11.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:03:11 smithi104 ceph-mon[36457]: pgmap v307: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T17:03:12.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:03:11 smithi117 ceph-mon[36730]: pgmap v307: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T17:03:12.349 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-16T17:03:12.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:12.566+0000 7f2a6ea00640 1 -- 172.21.15.45:0/2429772115 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2a68105e10 msgr2=0x7f2a6810c6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:12.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:12.566+0000 7f2a6ea00640 1 --2- 172.21.15.45:0/2429772115 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2a68105e10 0x7f2a6810c6a0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f2a60009ca0 tx=0x7f2a6002c0f0 comp rx=0 tx=0).stop 2024-04-16T17:03:12.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:12.566+0000 7f2a6ea00640 1 -- 172.21.15.45:0/2429772115 shutdown_connections 2024-04-16T17:03:12.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:12.566+0000 7f2a6ea00640 1 --2- 172.21.15.45:0/2429772115 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2a68105450 0x7f2a681058d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:12.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:12.566+0000 7f2a6ea00640 1 --2- 172.21.15.45:0/2429772115 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2a68105e10 0x7f2a6810c6a0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:12.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:12.566+0000 7f2a6ea00640 1 --2- 172.21.15.45:0/2429772115 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2a68104260 0x7f2a68104680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:12.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:12.566+0000 7f2a6ea00640 1 -- 172.21.15.45:0/2429772115 >> 172.21.15.45:0/2429772115 conn(0x7f2a680ff7f0 msgr2=0x7f2a68101c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:03:12.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:12.566+0000 7f2a6ea00640 1 -- 172.21.15.45:0/2429772115 shutdown_connections 2024-04-16T17:03:12.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:12.566+0000 7f2a6ea00640 1 -- 172.21.15.45:0/2429772115 wait complete. 2024-04-16T17:03:12.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:12.568+0000 7f2a6ea00640 1 Processor -- start 2024-04-16T17:03:12.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:12.568+0000 7f2a6ea00640 1 -- start start 2024-04-16T17:03:12.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:12.568+0000 7f2a6ea00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2a68104260 0x7f2a6819a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:12.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:12.568+0000 7f2a6ea00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2a68105450 0x7f2a6819aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:12.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:12.568+0000 7f2a6ea00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2a68105e10 0x7f2a681a1af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:12.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:12.568+0000 7f2a6ea00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f2a6810fef0 con 0x7f2a68104260 2024-04-16T17:03:12.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:12.568+0000 7f2a6ea00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f2a6810fd70 con 0x7f2a68105450 2024-04-16T17:03:12.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:12.568+0000 7f2a6ea00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f2a68110070 con 0x7f2a68105e10 2024-04-16T17:03:12.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:12.569+0000 7f2a67e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2a68104260 0x7f2a6819a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:12.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:12.569+0000 7f2a67e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2a68104260 0x7f2a6819a540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39844/0 (socket says 172.21.15.45:39844) 2024-04-16T17:03:12.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:12.569+0000 7f2a67e00640 1 -- 172.21.15.45:0/2703744424 learned_addr learned my addr 172.21.15.45:0/2703744424 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:03:12.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:12.569+0000 7f2a6ca00640 1 --2- 172.21.15.45:0/2703744424 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2a68105e10 0x7f2a681a1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:12.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:12.569+0000 7f2a67400640 1 --2- 172.21.15.45:0/2703744424 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2a68105450 0x7f2a6819aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:12.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:12.569+0000 7f2a67e00640 1 -- 172.21.15.45:0/2703744424 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2a68105e10 msgr2=0x7f2a681a1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:12.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:12.569+0000 7f2a67e00640 1 --2- 172.21.15.45:0/2703744424 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2a68105e10 0x7f2a681a1af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:12.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:12.569+0000 7f2a67e00640 1 -- 172.21.15.45:0/2703744424 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2a68105450 msgr2=0x7f2a6819aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:12.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:12.569+0000 7f2a67e00640 1 --2- 172.21.15.45:0/2703744424 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2a68105450 0x7f2a6819aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:12.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:12.569+0000 7f2a67e00640 1 -- 172.21.15.45:0/2703744424 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a681a2160 con 0x7f2a68104260 2024-04-16T17:03:12.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:12.569+0000 7f2a67e00640 1 --2- 172.21.15.45:0/2703744424 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2a68104260 0x7f2a6819a540 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f2a5800e9c0 tx=0x7f2a5800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:03:12.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:12.570+0000 7f2a64c00640 1 -- 172.21.15.45:0/2703744424 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a5800cde0 con 0x7f2a68104260 2024-04-16T17:03:12.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:12.570+0000 7f2a6ea00640 1 -- 172.21.15.45:0/2703744424 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a681a2450 con 0x7f2a68104260 2024-04-16T17:03:12.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:12.570+0000 7f2a64c00640 1 -- 172.21.15.45:0/2703744424 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2a58010910 con 0x7f2a68104260 2024-04-16T17:03:12.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:12.570+0000 7f2a64c00640 1 -- 172.21.15.45:0/2703744424 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a58018a10 con 0x7f2a68104260 2024-04-16T17:03:12.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:12.570+0000 7f2a6ea00640 1 -- 172.21.15.45:0/2703744424 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a681a29f0 con 0x7f2a68104260 2024-04-16T17:03:12.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:12.571+0000 7f2a6ea00640 1 -- 172.21.15.45:0/2703744424 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a6810b580 con 0x7f2a68104260 2024-04-16T17:03:12.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:12.573+0000 7f2a64c00640 1 -- 172.21.15.45:0/2703744424 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f2a5801f020 con 0x7f2a68104260 2024-04-16T17:03:12.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:12.575+0000 7f2a64c00640 1 --2- 172.21.15.45:0/2703744424 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f2a4c0a3c30 0x7f2a4c0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:12.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:12.575+0000 7f2a64c00640 1 -- 172.21.15.45:0/2703744424 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f2a580d0120 con 0x7f2a68104260 2024-04-16T17:03:12.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:12.575+0000 7f2a67400640 1 --2- 172.21.15.45:0/2703744424 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f2a4c0a3c30 0x7f2a4c0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:12.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:12.576+0000 7f2a67400640 1 --2- 172.21.15.45:0/2703744424 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f2a4c0a3c30 0x7f2a4c0a60f0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f2a6819ba50 tx=0x7f2a54005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:03:12.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:12.583+0000 7f2a64c00640 1 -- 172.21.15.45:0/2703744424 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2a5809cf00 con 0x7f2a68104260 2024-04-16T17:03:12.711 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:03:12 smithi045 ceph-mon[31676]: mds.? [v2:172.21.15.45:6838/1308627051,v1:172.21.15.45:6839/1308627051] up:boot 2024-04-16T17:03:12.711 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:03:12 smithi045 ceph-mon[31676]: fsmap 10 up:standby 2024-04-16T17:03:12.711 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:03:12 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-16T17:03:12.711 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:03:12 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:03:12.711 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:03:12 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:03:12.711 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:03:12 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:03:12.711 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:03:12 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-16T17:03:12.711 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:03:12 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-16T17:03:12.711 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:03:12 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:03:12.711 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:03:12 smithi045 ceph-mon[31676]: Deploying daemon mds.a on smithi045 2024-04-16T17:03:12.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:12.859+0000 7f2a6ea00640 1 -- 172.21.15.45:0/2703744424 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2a6802d620 con 0x7f2a68104260 2024-04-16T17:03:12.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:12.860+0000 7f2a64c00640 1 -- 172.21.15.45:0/2703744424 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+8842 (secure 0 0 0) 0x7f2a5809c800 con 0x7f2a68104260 2024-04-16T17:03:12.862 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-16T17:03:12.863 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/1748842443","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":1748842443},{"type":"v1","addr":"172.21.15.45:6835","nonce":1748842443}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6839/1308627051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":1308627051},{"type":"v1","addr":"172.21.15.45:6839","nonce":1308627051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/1768242489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1768242489},{"type":"v1","addr":"172.21.15.117:6833","nonce":1768242489}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6833/3558902585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":3558902585},{"type":"v1","addr":"172.21.15.104:6833","nonce":3558902585}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24478,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/1308142903","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":1308142903},{"type":"v1","addr":"172.21.15.45:6837","nonce":1308142903}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24491,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6835/3430679937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":3430679937},{"type":"v1","addr":"172.21.15.104:6835","nonce":3430679937}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24499,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/2224123841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2224123841},{"type":"v1","addr":"172.21.15.117:6835","nonce":2224123841}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24511,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/2471356815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":2471356815},{"type":"v1","addr":"172.21.15.117:6837","nonce":2471356815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24524,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6837/2647756509","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":2647756509},{"type":"v1","addr":"172.21.15.104:6837","nonce":2647756509}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24532,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6839/2378675949","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":2378675949},{"type":"v1","addr":"172.21.15.117:6839","nonce":2378675949}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-04-16T17:03:12.863 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 11 2024-04-16T17:03:12.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:12.863+0000 7f2a6ea00640 1 -- 172.21.15.45:0/2703744424 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f2a4c0a3c30 msgr2=0x7f2a4c0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:12.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:12.863+0000 7f2a6ea00640 1 --2- 172.21.15.45:0/2703744424 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f2a4c0a3c30 0x7f2a4c0a60f0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f2a6819ba50 tx=0x7f2a54005f90 comp rx=0 tx=0).stop 2024-04-16T17:03:12.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:12.863+0000 7f2a6ea00640 1 -- 172.21.15.45:0/2703744424 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2a68104260 msgr2=0x7f2a6819a540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:12.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:12.863+0000 7f2a6ea00640 1 --2- 172.21.15.45:0/2703744424 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2a68104260 0x7f2a6819a540 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f2a5800e9c0 tx=0x7f2a5800ee80 comp rx=0 tx=0).stop 2024-04-16T17:03:12.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:12.863+0000 7f2a6ea00640 1 -- 172.21.15.45:0/2703744424 shutdown_connections 2024-04-16T17:03:12.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:12.863+0000 7f2a6ea00640 1 --2- 172.21.15.45:0/2703744424 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f2a4c0a3c30 0x7f2a4c0a60f0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:12.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:12.863+0000 7f2a6ea00640 1 --2- 172.21.15.45:0/2703744424 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2a68105e10 0x7f2a681a1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:12.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:12.863+0000 7f2a6ea00640 1 --2- 172.21.15.45:0/2703744424 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2a68105450 0x7f2a6819aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:12.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:12.863+0000 7f2a6ea00640 1 --2- 172.21.15.45:0/2703744424 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2a68104260 0x7f2a6819a540 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:12.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:12.863+0000 7f2a6ea00640 1 -- 172.21.15.45:0/2703744424 >> 172.21.15.45:0/2703744424 conn(0x7f2a680ff7f0 msgr2=0x7f2a68108750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:03:12.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:12.863+0000 7f2a6ea00640 1 -- 172.21.15.45:0/2703744424 shutdown_connections 2024-04-16T17:03:12.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:12.863+0000 7f2a6ea00640 1 -- 172.21.15.45:0/2703744424 wait complete. 2024-04-16T17:03:12.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:03:12 smithi104 ceph-mon[36457]: mds.? [v2:172.21.15.45:6838/1308627051,v1:172.21.15.45:6839/1308627051] up:boot 2024-04-16T17:03:12.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:03:12 smithi104 ceph-mon[36457]: fsmap 10 up:standby 2024-04-16T17:03:12.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:03:12 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-16T17:03:12.929 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:03:12 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:03:12.929 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:03:12 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:03:12.929 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:03:12 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:03:12.929 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:03:12 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-16T17:03:12.929 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:03:12 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-16T17:03:12.929 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:03:12 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:03:12.929 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:03:12 smithi104 ceph-mon[36457]: Deploying daemon mds.a on smithi045 2024-04-16T17:03:13.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:03:12 smithi117 ceph-mon[36730]: mds.? [v2:172.21.15.45:6838/1308627051,v1:172.21.15.45:6839/1308627051] up:boot 2024-04-16T17:03:13.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:03:12 smithi117 ceph-mon[36730]: fsmap 10 up:standby 2024-04-16T17:03:13.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:03:12 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-16T17:03:13.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:03:12 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:03:13.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:03:12 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:03:13.057 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:03:12 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:03:13.057 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:03:12 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-16T17:03:13.057 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:03:12 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-16T17:03:13.057 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:03:12 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:03:13.057 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:03:12 smithi117 ceph-mon[36730]: Deploying daemon mds.a on smithi045 2024-04-16T17:03:13.978 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:03:13 smithi045 ceph-mon[31676]: pgmap v308: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T17:03:13.979 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:03:13 smithi045 ceph-mon[31676]: from='client.? 172.21.15.45:0/2703744424' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:03:14.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:03:13 smithi117 ceph-mon[36730]: pgmap v308: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T17:03:14.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:03:13 smithi117 ceph-mon[36730]: from='client.? 172.21.15.45:0/2703744424' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:03:14.098 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:03:13 smithi104 ceph-mon[36457]: pgmap v308: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T17:03:14.098 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:03:13 smithi104 ceph-mon[36457]: from='client.? 172.21.15.45:0/2703744424' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:03:14.885 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-16T17:03:15.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:15.119+0000 7fc612800640 1 -- 172.21.15.45:0/3766619302 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc60c101210 msgr2=0x7fc60c101670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:15.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:15.119+0000 7fc612800640 1 --2- 172.21.15.45:0/3766619302 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc60c101210 0x7fc60c101670 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fc5f8009860 tx=0x7fc5f802bb70 comp rx=0 tx=0).stop 2024-04-16T17:03:15.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:15.119+0000 7fc612800640 1 -- 172.21.15.45:0/3766619302 shutdown_connections 2024-04-16T17:03:15.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:15.119+0000 7fc612800640 1 --2- 172.21.15.45:0/3766619302 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc60c101210 0x7fc60c101670 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:15.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:15.119+0000 7fc612800640 1 --2- 172.21.15.45:0/3766619302 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc60c101bb0 0x7fc60c102030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:15.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:15.119+0000 7fc612800640 1 --2- 172.21.15.45:0/3766619302 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc60c1008b0 0x7fc60c100cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:15.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:15.119+0000 7fc612800640 1 -- 172.21.15.45:0/3766619302 >> 172.21.15.45:0/3766619302 conn(0x7fc60c0fc4b0 msgr2=0x7fc60c0fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:03:15.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:15.119+0000 7fc612800640 1 -- 172.21.15.45:0/3766619302 shutdown_connections 2024-04-16T17:03:15.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:15.119+0000 7fc612800640 1 -- 172.21.15.45:0/3766619302 wait complete. 2024-04-16T17:03:15.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:15.120+0000 7fc612800640 1 Processor -- start 2024-04-16T17:03:15.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:15.120+0000 7fc612800640 1 -- start start 2024-04-16T17:03:15.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:15.121+0000 7fc612800640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc60c1008b0 0x7fc60c19a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:15.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:15.121+0000 7fc612800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc60c101210 0x7fc60c19aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:15.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:15.121+0000 7fc612800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc60c101bb0 0x7fc60c1a1ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:15.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:15.121+0000 7fc612800640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fc60c10fee0 con 0x7fc60c101bb0 2024-04-16T17:03:15.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:15.121+0000 7fc612800640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fc60c10fd60 con 0x7fc60c1008b0 2024-04-16T17:03:15.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:15.121+0000 7fc612800640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fc60c110060 con 0x7fc60c101210 2024-04-16T17:03:15.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:15.121+0000 7fc60b400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc60c1008b0 0x7fc60c19a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:15.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:15.121+0000 7fc60be00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc60c101bb0 0x7fc60c1a1ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:15.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:15.121+0000 7fc60b400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc60c1008b0 0x7fc60c19a500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:52878/0 (socket says 172.21.15.45:52878) 2024-04-16T17:03:15.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:15.121+0000 7fc60b400640 1 -- 172.21.15.45:0/1541223130 learned_addr learned my addr 172.21.15.45:0/1541223130 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:03:15.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:15.121+0000 7fc60aa00640 1 --2- 172.21.15.45:0/1541223130 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc60c101210 0x7fc60c19aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:15.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:15.122+0000 7fc60b400640 1 -- 172.21.15.45:0/1541223130 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc60c101210 msgr2=0x7fc60c19aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:15.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:15.122+0000 7fc60b400640 1 --2- 172.21.15.45:0/1541223130 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc60c101210 0x7fc60c19aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:15.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:15.122+0000 7fc60b400640 1 -- 172.21.15.45:0/1541223130 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc60c101bb0 msgr2=0x7fc60c1a1ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:15.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:15.122+0000 7fc60b400640 1 --2- 172.21.15.45:0/1541223130 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc60c101bb0 0x7fc60c1a1ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:15.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:15.122+0000 7fc60b400640 1 -- 172.21.15.45:0/1541223130 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc60c1a2120 con 0x7fc60c1008b0 2024-04-16T17:03:15.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:15.122+0000 7fc60be00640 1 --2- 172.21.15.45:0/1541223130 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc60c101bb0 0x7fc60c1a1ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:03:15.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:15.122+0000 7fc60b400640 1 --2- 172.21.15.45:0/1541223130 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc60c1008b0 0x7fc60c19a500 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fc5fc00e9c0 tx=0x7fc5fc00ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:03:15.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:15.122+0000 7fc603e00640 1 -- 172.21.15.45:0/1541223130 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5fc00cde0 con 0x7fc60c1008b0 2024-04-16T17:03:15.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:15.122+0000 7fc603e00640 1 -- 172.21.15.45:0/1541223130 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc5fc0035f0 con 0x7fc60c1008b0 2024-04-16T17:03:15.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:15.122+0000 7fc612800640 1 -- 172.21.15.45:0/1541223130 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc60c1a2410 con 0x7fc60c1008b0 2024-04-16T17:03:15.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:15.122+0000 7fc603e00640 1 -- 172.21.15.45:0/1541223130 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5fc010690 con 0x7fc60c1008b0 2024-04-16T17:03:15.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:15.122+0000 7fc612800640 1 -- 172.21.15.45:0/1541223130 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc60c1a2930 con 0x7fc60c1008b0 2024-04-16T17:03:15.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:15.124+0000 7fc612800640 1 -- 172.21.15.45:0/1541223130 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc60c10b7b0 con 0x7fc60c1008b0 2024-04-16T17:03:15.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:15.126+0000 7fc603e00640 1 -- 172.21.15.45:0/1541223130 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fc5fc0037a0 con 0x7fc60c1008b0 2024-04-16T17:03:15.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:15.127+0000 7fc603e00640 1 --2- 172.21.15.45:0/1541223130 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fc5f00a3bb0 0x7fc5f00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:15.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:15.127+0000 7fc603e00640 1 -- 172.21.15.45:0/1541223130 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fc5fc0d0320 con 0x7fc60c1008b0 2024-04-16T17:03:15.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:15.127+0000 7fc60aa00640 1 --2- 172.21.15.45:0/1541223130 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fc5f00a3bb0 0x7fc5f00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:15.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:15.128+0000 7fc60aa00640 1 --2- 172.21.15.45:0/1541223130 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fc5f00a3bb0 0x7fc5f00a6070 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fc5f802c180 tx=0x7fc5f8034ad0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:03:15.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:15.131+0000 7fc603e00640 1 -- 172.21.15.45:0/1541223130 <== mon.2 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc5fc09d100 con 0x7fc60c1008b0 2024-04-16T17:03:15.228 INFO:journalctl@ceph.mds.a.smithi045.stdout:Apr 16 17:03:15 smithi045 systemd[1]: Starting Ceph mds.a for b1fecd38-fc11-11ee-bc90-c7b262605968... 2024-04-16T17:03:15.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:15.390+0000 7fc612800640 1 -- 172.21.15.45:0/1541223130 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc60c02d620 con 0x7fc60c1008b0 2024-04-16T17:03:15.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:15.391+0000 7fc603e00640 1 -- 172.21.15.45:0/1541223130 <== mon.2 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+8842 (secure 0 0 0) 0x7fc5fc09ca00 con 0x7fc60c1008b0 2024-04-16T17:03:15.393 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-16T17:03:15.393 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/1748842443","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":1748842443},{"type":"v1","addr":"172.21.15.45:6835","nonce":1748842443}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6839/1308627051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":1308627051},{"type":"v1","addr":"172.21.15.45:6839","nonce":1308627051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/1768242489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1768242489},{"type":"v1","addr":"172.21.15.117:6833","nonce":1768242489}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6833/3558902585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":3558902585},{"type":"v1","addr":"172.21.15.104:6833","nonce":3558902585}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24478,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/1308142903","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":1308142903},{"type":"v1","addr":"172.21.15.45:6837","nonce":1308142903}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24491,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6835/3430679937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":3430679937},{"type":"v1","addr":"172.21.15.104:6835","nonce":3430679937}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24499,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/2224123841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2224123841},{"type":"v1","addr":"172.21.15.117:6835","nonce":2224123841}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24511,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/2471356815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":2471356815},{"type":"v1","addr":"172.21.15.117:6837","nonce":2471356815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24524,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6837/2647756509","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":2647756509},{"type":"v1","addr":"172.21.15.104:6837","nonce":2647756509}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24532,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6839/2378675949","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":2378675949},{"type":"v1","addr":"172.21.15.117:6839","nonce":2378675949}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-04-16T17:03:15.393 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 11 2024-04-16T17:03:15.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:15.393+0000 7fc612800640 1 -- 172.21.15.45:0/1541223130 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fc5f00a3bb0 msgr2=0x7fc5f00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:15.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:15.393+0000 7fc612800640 1 --2- 172.21.15.45:0/1541223130 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fc5f00a3bb0 0x7fc5f00a6070 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fc5f802c180 tx=0x7fc5f8034ad0 comp rx=0 tx=0).stop 2024-04-16T17:03:15.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:15.393+0000 7fc612800640 1 -- 172.21.15.45:0/1541223130 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc60c1008b0 msgr2=0x7fc60c19a500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:15.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:15.393+0000 7fc612800640 1 --2- 172.21.15.45:0/1541223130 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc60c1008b0 0x7fc60c19a500 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fc5fc00e9c0 tx=0x7fc5fc00ee80 comp rx=0 tx=0).stop 2024-04-16T17:03:15.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:15.394+0000 7fc612800640 1 -- 172.21.15.45:0/1541223130 shutdown_connections 2024-04-16T17:03:15.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:15.394+0000 7fc612800640 1 --2- 172.21.15.45:0/1541223130 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fc5f00a3bb0 0x7fc5f00a6070 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:15.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:15.394+0000 7fc612800640 1 --2- 172.21.15.45:0/1541223130 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc60c101210 0x7fc60c19aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:15.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:15.394+0000 7fc612800640 1 --2- 172.21.15.45:0/1541223130 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc60c101bb0 0x7fc60c1a1ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:15.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:15.394+0000 7fc612800640 1 --2- 172.21.15.45:0/1541223130 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc60c1008b0 0x7fc60c19a500 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:15.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:15.394+0000 7fc612800640 1 -- 172.21.15.45:0/1541223130 >> 172.21.15.45:0/1541223130 conn(0x7fc60c0fc4b0 msgr2=0x7fc60c0fdbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:03:15.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:15.394+0000 7fc612800640 1 -- 172.21.15.45:0/1541223130 shutdown_connections 2024-04-16T17:03:15.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:15.394+0000 7fc612800640 1 -- 172.21.15.45:0/1541223130 wait complete. 2024-04-16T17:03:15.634 INFO:journalctl@ceph.mds.a.smithi045.stdout:Apr 16 17:03:15 smithi045 podman[73368]: 2024-04-16 17:03:15.377916824 +0000 UTC m=+0.129482853 container create 070e1db7eaf3b2fb2d8a18804a330f93c3a73c41dc144ec08371030cf1c4d759 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mds-a, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240409, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True) 2024-04-16T17:03:15.634 INFO:journalctl@ceph.mds.a.smithi045.stdout:Apr 16 17:03:15 smithi045 podman[73368]: 2024-04-16 17:03:15.286400254 +0000 UTC m=+0.037966259 image pull e5ce10bdffbeeffffd1125f67b369e2151f0f17cdec854e93b6adf9321f65907 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2 2024-04-16T17:03:15.634 INFO:journalctl@ceph.mds.a.smithi045.stdout:Apr 16 17:03:15 smithi045 podman[73368]: 2024-04-16 17:03:15.631898247 +0000 UTC m=+0.383464283 container init 070e1db7eaf3b2fb2d8a18804a330f93c3a73c41dc144ec08371030cf1c4d759 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mds-a, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240409, org.label-schema.schema-version=1.0) 2024-04-16T17:03:15.978 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:03:15 smithi045 ceph-mon[31676]: pgmap v309: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T17:03:15.978 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:03:15 smithi045 ceph-mon[31676]: from='client.? 172.21.15.45:0/1541223130' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:03:15.979 INFO:journalctl@ceph.mds.a.smithi045.stdout:Apr 16 17:03:15 smithi045 podman[73368]: 2024-04-16 17:03:15.643115505 +0000 UTC m=+0.394681510 container start 070e1db7eaf3b2fb2d8a18804a330f93c3a73c41dc144ec08371030cf1c4d759 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mds-a, io.buildah.version=1.33.5, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240409, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-04-16T17:03:15.979 INFO:journalctl@ceph.mds.a.smithi045.stdout:Apr 16 17:03:15 smithi045 ceph-mds[73388]: -- 172.21.15.45:0/443712108 <== mon.2 v2:172.21.15.104:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5651ada13a40 con 0x5651ad9e7000 2024-04-16T17:03:15.979 INFO:journalctl@ceph.mds.a.smithi045.stdout:Apr 16 17:03:15 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mds-a[73384]: starting mds.a at 2024-04-16T17:03:15.979 INFO:journalctl@ceph.mds.a.smithi045.stdout:Apr 16 17:03:15 smithi045 bash[73368]: 070e1db7eaf3b2fb2d8a18804a330f93c3a73c41dc144ec08371030cf1c4d759 2024-04-16T17:03:15.979 INFO:journalctl@ceph.mds.a.smithi045.stdout:Apr 16 17:03:15 smithi045 systemd[1]: Started Ceph mds.a for b1fecd38-fc11-11ee-bc90-c7b262605968. 2024-04-16T17:03:16.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:03:15 smithi117 ceph-mon[36730]: pgmap v309: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T17:03:16.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:03:15 smithi117 ceph-mon[36730]: from='client.? 172.21.15.45:0/1541223130' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:03:16.178 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:03:15 smithi104 ceph-mon[36457]: pgmap v309: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T17:03:16.178 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:03:15 smithi104 ceph-mon[36457]: from='client.? 172.21.15.45:0/1541223130' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:03:16.751 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:03:16 smithi104 ceph-mon[36457]: mds.? [v2:172.21.15.45:6840/3755310822,v1:172.21.15.45:6841/3755310822] up:boot 2024-04-16T17:03:16.751 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:03:16 smithi104 ceph-mon[36457]: fsmap 11 up:standby 2024-04-16T17:03:16.751 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:03:16 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-16T17:03:16.751 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:03:16 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:03:16.751 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:03:16 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:03:16.751 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:03:16 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:03:16.751 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:03:16 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-16T17:03:16.751 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:03:16 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-16T17:03:16.751 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:03:16 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:03:17.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:03:16 smithi117 ceph-mon[36730]: mds.? [v2:172.21.15.45:6840/3755310822,v1:172.21.15.45:6841/3755310822] up:boot 2024-04-16T17:03:17.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:03:16 smithi117 ceph-mon[36730]: fsmap 11 up:standby 2024-04-16T17:03:17.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:03:16 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-16T17:03:17.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:03:16 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:03:17.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:03:16 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:03:17.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:03:16 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:03:17.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:03:16 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-16T17:03:17.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:03:16 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-16T17:03:17.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:03:16 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:03:17.228 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:03:16 smithi045 ceph-mon[31676]: mds.? [v2:172.21.15.45:6840/3755310822,v1:172.21.15.45:6841/3755310822] up:boot 2024-04-16T17:03:17.228 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:03:16 smithi045 ceph-mon[31676]: fsmap 11 up:standby 2024-04-16T17:03:17.228 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:03:16 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-16T17:03:17.228 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:03:16 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:03:17.229 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:03:16 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:03:17.229 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:03:16 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:03:17.229 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:03:16 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-16T17:03:17.229 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:03:16 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-16T17:03:17.229 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:03:16 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:03:17.414 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-16T17:03:17.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:17.653+0000 7f31e1000640 1 -- 172.21.15.45:0/3525141998 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f31dc0785a0 msgr2=0x7f31dc078e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:17.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:17.653+0000 7f31e1000640 1 --2- 172.21.15.45:0/3525141998 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f31dc0785a0 0x7f31dc078e40 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f31d4009be0 tx=0x7f31d402bef0 comp rx=0 tx=0).stop 2024-04-16T17:03:17.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:17.653+0000 7f31e1000640 1 -- 172.21.15.45:0/3525141998 shutdown_connections 2024-04-16T17:03:17.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:17.653+0000 7f31e1000640 1 --2- 172.21.15.45:0/3525141998 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f31dc0785a0 0x7f31dc078e40 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:17.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:17.653+0000 7f31e1000640 1 --2- 172.21.15.45:0/3525141998 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f31dc077c00 0x7f31dc078060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:17.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:17.653+0000 7f31e1000640 1 --2- 172.21.15.45:0/3525141998 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f31dc0769c0 0x7f31dc076de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:17.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:17.653+0000 7f31e1000640 1 -- 172.21.15.45:0/3525141998 >> 172.21.15.45:0/3525141998 conn(0x7f31dc0ff7f0 msgr2=0x7f31dc101c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:03:17.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:17.653+0000 7f31e1000640 1 -- 172.21.15.45:0/3525141998 shutdown_connections 2024-04-16T17:03:17.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:17.653+0000 7f31e1000640 1 -- 172.21.15.45:0/3525141998 wait complete. 2024-04-16T17:03:17.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:17.654+0000 7f31e1000640 1 Processor -- start 2024-04-16T17:03:17.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:17.654+0000 7f31e1000640 1 -- start start 2024-04-16T17:03:17.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:17.654+0000 7f31e1000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f31dc0769c0 0x7f31dc19e900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:17.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:17.654+0000 7f31e1000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f31dc077c00 0x7f31dc19ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:17.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:17.654+0000 7f31e1000640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f31dc0785a0 0x7f31dc1a5eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:17.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:17.654+0000 7f31e1000640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f31dc114330 con 0x7f31dc0769c0 2024-04-16T17:03:17.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:17.654+0000 7f31e1000640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f31dc1141b0 con 0x7f31dc0785a0 2024-04-16T17:03:17.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:17.654+0000 7f31e1000640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f31dc1144b0 con 0x7f31dc077c00 2024-04-16T17:03:17.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:17.655+0000 7f31da800640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f31dc0785a0 0x7f31dc1a5eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:17.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:17.655+0000 7f31d9e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f31dc0769c0 0x7f31dc19e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:17.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:17.655+0000 7f31da800640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f31dc0785a0 0x7f31dc1a5eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:52922/0 (socket says 172.21.15.45:52922) 2024-04-16T17:03:17.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:17.655+0000 7f31da800640 1 -- 172.21.15.45:0/1931951337 learned_addr learned my addr 172.21.15.45:0/1931951337 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:03:17.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:17.655+0000 7f31d9400640 1 --2- 172.21.15.45:0/1931951337 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f31dc077c00 0x7f31dc19ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:17.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:17.655+0000 7f31da800640 1 -- 172.21.15.45:0/1931951337 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f31dc077c00 msgr2=0x7f31dc19ee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:17.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:17.655+0000 7f31da800640 1 --2- 172.21.15.45:0/1931951337 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f31dc077c00 0x7f31dc19ee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:17.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:17.655+0000 7f31da800640 1 -- 172.21.15.45:0/1931951337 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f31dc0769c0 msgr2=0x7f31dc19e900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:17.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:17.655+0000 7f31da800640 1 --2- 172.21.15.45:0/1931951337 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f31dc0769c0 0x7f31dc19e900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:17.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:17.655+0000 7f31da800640 1 -- 172.21.15.45:0/1931951337 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31dc1a6520 con 0x7f31dc0785a0 2024-04-16T17:03:17.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:17.655+0000 7f31d9e00640 1 --2- 172.21.15.45:0/1931951337 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f31dc0769c0 0x7f31dc19e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:03:17.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:17.655+0000 7f31da800640 1 --2- 172.21.15.45:0/1931951337 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f31dc0785a0 0x7f31dc1a5eb0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f31d4036040 tx=0x7f31d40337b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:03:17.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:17.656+0000 7f31d2a00640 1 -- 172.21.15.45:0/1931951337 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31d4043020 con 0x7f31dc0785a0 2024-04-16T17:03:17.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:17.656+0000 7f31d2a00640 1 -- 172.21.15.45:0/1931951337 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f31d402ca60 con 0x7f31dc0785a0 2024-04-16T17:03:17.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:17.656+0000 7f31e1000640 1 -- 172.21.15.45:0/1931951337 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31dc1a67b0 con 0x7f31dc0785a0 2024-04-16T17:03:17.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:17.656+0000 7f31d2a00640 1 -- 172.21.15.45:0/1931951337 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31d4034690 con 0x7f31dc0785a0 2024-04-16T17:03:17.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:17.656+0000 7f31e1000640 1 -- 172.21.15.45:0/1931951337 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31dc1a6d50 con 0x7f31dc0785a0 2024-04-16T17:03:17.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:17.657+0000 7f31e1000640 1 -- 172.21.15.45:0/1931951337 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f31dc110d10 con 0x7f31dc0785a0 2024-04-16T17:03:17.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:17.660+0000 7f31d2a00640 1 -- 172.21.15.45:0/1931951337 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f31d4046020 con 0x7f31dc0785a0 2024-04-16T17:03:17.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:17.660+0000 7f31d2a00640 1 --2- 172.21.15.45:0/1931951337 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f31c00a3c30 0x7f31c00a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:17.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:17.660+0000 7f31d2a00640 1 -- 172.21.15.45:0/1931951337 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f31d40f2580 con 0x7f31dc0785a0 2024-04-16T17:03:17.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:17.660+0000 7f31d9e00640 1 --2- 172.21.15.45:0/1931951337 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f31c00a3c30 0x7f31c00a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:17.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:17.661+0000 7f31d9e00640 1 --2- 172.21.15.45:0/1931951337 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f31c00a3c30 0x7f31c00a60f0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f31c4005fd0 tx=0x7f31c4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:03:17.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:17.665+0000 7f31d2a00640 1 -- 172.21.15.45:0/1931951337 <== mon.2 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f31d404d020 con 0x7f31dc0785a0 2024-04-16T17:03:17.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:17.918+0000 7f31e1000640 1 -- 172.21.15.45:0/1931951337 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f31dc062c00 con 0x7f31dc0785a0 2024-04-16T17:03:17.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:17.919+0000 7f31d2a00640 1 -- 172.21.15.45:0/1931951337 <== mon.2 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9668 (secure 0 0 0) 0x7f31d40dde20 con 0x7f31dc0785a0 2024-04-16T17:03:17.921 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-16T17:03:17.921 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/1748842443","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":1748842443},{"type":"v1","addr":"172.21.15.45:6835","nonce":1748842443}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6839/1308627051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":1308627051},{"type":"v1","addr":"172.21.15.45:6839","nonce":1308627051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/1768242489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1768242489},{"type":"v1","addr":"172.21.15.117:6833","nonce":1768242489}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6833/3558902585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":3558902585},{"type":"v1","addr":"172.21.15.104:6833","nonce":3558902585}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24478,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/1308142903","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":1308142903},{"type":"v1","addr":"172.21.15.45:6837","nonce":1308142903}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24491,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6835/3430679937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":3430679937},{"type":"v1","addr":"172.21.15.104:6835","nonce":3430679937}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24499,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/2224123841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2224123841},{"type":"v1","addr":"172.21.15.117:6835","nonce":2224123841}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24511,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/2471356815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":2471356815},{"type":"v1","addr":"172.21.15.117:6837","nonce":2471356815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24524,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6837/2647756509","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":2647756509},{"type":"v1","addr":"172.21.15.104:6837","nonce":2647756509}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24532,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6839/2378675949","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":2378675949},{"type":"v1","addr":"172.21.15.117:6839","nonce":2378675949}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24569,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6841/3755310822","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6840","nonce":3755310822},{"type":"v1","addr":"172.21.15.45:6841","nonce":3755310822}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[]} 2024-04-16T17:03:17.922 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 12 2024-04-16T17:03:17.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:17.922+0000 7f31e1000640 1 -- 172.21.15.45:0/1931951337 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f31c00a3c30 msgr2=0x7f31c00a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:17.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:17.922+0000 7f31e1000640 1 --2- 172.21.15.45:0/1931951337 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f31c00a3c30 0x7f31c00a60f0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f31c4005fd0 tx=0x7f31c4009260 comp rx=0 tx=0).stop 2024-04-16T17:03:17.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:17.922+0000 7f31e1000640 1 -- 172.21.15.45:0/1931951337 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f31dc0785a0 msgr2=0x7f31dc1a5eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:17.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:17.922+0000 7f31e1000640 1 --2- 172.21.15.45:0/1931951337 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f31dc0785a0 0x7f31dc1a5eb0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f31d4036040 tx=0x7f31d40337b0 comp rx=0 tx=0).stop 2024-04-16T17:03:17.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:17.922+0000 7f31e1000640 1 -- 172.21.15.45:0/1931951337 shutdown_connections 2024-04-16T17:03:17.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:17.922+0000 7f31e1000640 1 --2- 172.21.15.45:0/1931951337 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f31c00a3c30 0x7f31c00a60f0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:17.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:17.922+0000 7f31e1000640 1 --2- 172.21.15.45:0/1931951337 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f31dc077c00 0x7f31dc19ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:17.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:17.922+0000 7f31e1000640 1 --2- 172.21.15.45:0/1931951337 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f31dc0785a0 0x7f31dc1a5eb0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:17.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:17.922+0000 7f31e1000640 1 --2- 172.21.15.45:0/1931951337 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f31dc0769c0 0x7f31dc19e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:17.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:17.922+0000 7f31e1000640 1 -- 172.21.15.45:0/1931951337 >> 172.21.15.45:0/1931951337 conn(0x7f31dc0ff7f0 msgr2=0x7f31dc100e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:03:17.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:17.922+0000 7f31e1000640 1 -- 172.21.15.45:0/1931951337 shutdown_connections 2024-04-16T17:03:17.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:17.922+0000 7f31e1000640 1 -- 172.21.15.45:0/1931951337 wait complete. 2024-04-16T17:03:18.178 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:03:17 smithi104 ceph-mon[36457]: Deploying daemon mds.b on smithi104 2024-04-16T17:03:18.178 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:03:17 smithi104 ceph-mon[36457]: pgmap v310: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T17:03:18.179 INFO:journalctl@ceph.mds.b.smithi104.stdout:Apr 16 17:03:17 smithi104 systemd[1]: Starting Ceph mds.b for b1fecd38-fc11-11ee-bc90-c7b262605968... 2024-04-16T17:03:18.228 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:03:17 smithi045 ceph-mon[31676]: Deploying daemon mds.b on smithi104 2024-04-16T17:03:18.228 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:03:17 smithi045 ceph-mon[31676]: pgmap v310: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T17:03:18.306 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:03:17 smithi117 ceph-mon[36730]: Deploying daemon mds.b on smithi104 2024-04-16T17:03:18.306 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:03:17 smithi117 ceph-mon[36730]: pgmap v310: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T17:03:18.473 INFO:journalctl@ceph.mds.b.smithi104.stdout:Apr 16 17:03:18 smithi104 podman[67783]: 2024-04-16 17:03:18.093570731 +0000 UTC m=+0.017810444 image pull e5ce10bdffbeeffffd1125f67b369e2151f0f17cdec854e93b6adf9321f65907 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2 2024-04-16T17:03:18.473 INFO:journalctl@ceph.mds.b.smithi104.stdout:Apr 16 17:03:18 smithi104 podman[67783]: 2024-04-16 17:03:18.213604114 +0000 UTC m=+0.137843806 container create 7f17776ddeedcf6bda64cedb064a9ae5afbea853a9470eff8ce2335d835a2b7a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mds-b, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, GIT_BRANCH=HEAD, org.label-schema.build-date=20240409, org.label-schema.vendor=CentOS) 2024-04-16T17:03:18.473 INFO:journalctl@ceph.mds.b.smithi104.stdout:Apr 16 17:03:18 smithi104 podman[67783]: 2024-04-16 17:03:18.472271698 +0000 UTC m=+0.396511397 container init 7f17776ddeedcf6bda64cedb064a9ae5afbea853a9470eff8ce2335d835a2b7a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mds-b, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240409, org.label-schema.schema-version=1.0) 2024-04-16T17:03:18.864 INFO:journalctl@ceph.mds.b.smithi104.stdout:Apr 16 17:03:18 smithi104 podman[67783]: 2024-04-16 17:03:18.475058328 +0000 UTC m=+0.399298037 container start 7f17776ddeedcf6bda64cedb064a9ae5afbea853a9470eff8ce2335d835a2b7a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mds-b, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240409, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-16T17:03:18.864 INFO:journalctl@ceph.mds.b.smithi104.stdout:Apr 16 17:03:18 smithi104 ceph-mds[67802]: -- 172.21.15.104:0/231240451 <== mon.2 v2:172.21.15.104:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55c731d7ba40 con 0x55c731d4ec00 2024-04-16T17:03:18.864 INFO:journalctl@ceph.mds.b.smithi104.stdout:Apr 16 17:03:18 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mds-b[67798]: starting mds.b at 2024-04-16T17:03:18.864 INFO:journalctl@ceph.mds.b.smithi104.stdout:Apr 16 17:03:18 smithi104 bash[67783]: 7f17776ddeedcf6bda64cedb064a9ae5afbea853a9470eff8ce2335d835a2b7a 2024-04-16T17:03:18.865 INFO:journalctl@ceph.mds.b.smithi104.stdout:Apr 16 17:03:18 smithi104 systemd[1]: Started Ceph mds.b for b1fecd38-fc11-11ee-bc90-c7b262605968. 2024-04-16T17:03:19.178 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:03:18 smithi104 ceph-mon[36457]: from='client.? 172.21.15.45:0/1931951337' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:03:19.179 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:03:18 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:03:19.179 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:03:18 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:03:19.179 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:03:18 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:03:19.179 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:03:18 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:03:19.179 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:03:18 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:03:19.228 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:03:18 smithi045 ceph-mon[31676]: from='client.? 172.21.15.45:0/1931951337' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:03:19.228 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:03:18 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:03:19.228 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:03:18 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:03:19.228 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:03:18 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:03:19.228 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:03:18 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:03:19.228 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:03:18 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:03:19.305 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:03:18 smithi117 ceph-mon[36730]: from='client.? 172.21.15.45:0/1931951337' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:03:19.306 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:03:18 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:03:19.306 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:03:18 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:03:19.306 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:03:18 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:03:19.306 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:03:18 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:03:19.306 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:03:18 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:03:19.945 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-16T17:03:20.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:03:19 smithi117 ceph-mon[36730]: pgmap v311: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T17:03:20.057 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:03:19 smithi117 ceph-mon[36730]: mds.? [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] up:boot 2024-04-16T17:03:20.057 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:03:19 smithi117 ceph-mon[36730]: fsmap 12 up:standby 2024-04-16T17:03:20.057 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:03:19 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-16T17:03:20.057 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:03:19 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T17:03:20.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:20.159+0000 7f3fba000640 1 -- 172.21.15.45:0/4042395249 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3fb41040a0 msgr2=0x7f3fb41044c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:20.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:20.159+0000 7f3fba000640 1 --2- 172.21.15.45:0/4042395249 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3fb41040a0 0x7f3fb41044c0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f3fa8009900 tx=0x7f3fa802bc40 comp rx=0 tx=0).stop 2024-04-16T17:03:20.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:20.160+0000 7f3fba000640 1 -- 172.21.15.45:0/4042395249 shutdown_connections 2024-04-16T17:03:20.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:20.160+0000 7f3fba000640 1 --2- 172.21.15.45:0/4042395249 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3fb4105c50 0x7f3fb410c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:20.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:20.160+0000 7f3fba000640 1 --2- 172.21.15.45:0/4042395249 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3fb4105290 0x7f3fb4105710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:20.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:20.160+0000 7f3fba000640 1 --2- 172.21.15.45:0/4042395249 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3fb41040a0 0x7f3fb41044c0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:20.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:20.160+0000 7f3fba000640 1 -- 172.21.15.45:0/4042395249 >> 172.21.15.45:0/4042395249 conn(0x7f3fb40ff610 msgr2=0x7f3fb4101a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:03:20.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:20.160+0000 7f3fba000640 1 -- 172.21.15.45:0/4042395249 shutdown_connections 2024-04-16T17:03:20.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:20.160+0000 7f3fba000640 1 -- 172.21.15.45:0/4042395249 wait complete. 2024-04-16T17:03:20.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:20.161+0000 7f3fba000640 1 Processor -- start 2024-04-16T17:03:20.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:20.161+0000 7f3fba000640 1 -- start start 2024-04-16T17:03:20.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:20.162+0000 7f3fba000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3fb41040a0 0x7f3fb419a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:20.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:20.162+0000 7f3fba000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3fb4105290 0x7f3fb419a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:20.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:20.162+0000 7f3fba000640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3fb4105c50 0x7f3fb41a1970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:20.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:20.162+0000 7f3fba000640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f3fb410fd80 con 0x7f3fb4105290 2024-04-16T17:03:20.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:20.162+0000 7f3fba000640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f3fb410fc00 con 0x7f3fb4105c50 2024-04-16T17:03:20.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:20.162+0000 7f3fba000640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f3fb410ff00 con 0x7f3fb41040a0 2024-04-16T17:03:20.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:20.162+0000 7f3fb8c00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3fb41040a0 0x7f3fb419a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:20.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:20.162+0000 7f3fb8c00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3fb41040a0 0x7f3fb419a3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:40100/0 (socket says 172.21.15.45:40100) 2024-04-16T17:03:20.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:20.162+0000 7f3fb8c00640 1 -- 172.21.15.45:0/796282306 learned_addr learned my addr 172.21.15.45:0/796282306 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:03:20.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:20.162+0000 7f3fb3e00640 1 --2- 172.21.15.45:0/796282306 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3fb4105290 0x7f3fb419a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:20.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:20.162+0000 7f3fb9600640 1 --2- 172.21.15.45:0/796282306 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3fb4105c50 0x7f3fb41a1970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:20.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:20.162+0000 7f3fb8c00640 1 -- 172.21.15.45:0/796282306 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3fb4105c50 msgr2=0x7f3fb41a1970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:20.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:20.162+0000 7f3fb8c00640 1 --2- 172.21.15.45:0/796282306 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3fb4105c50 0x7f3fb41a1970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:20.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:20.162+0000 7f3fb8c00640 1 -- 172.21.15.45:0/796282306 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3fb4105290 msgr2=0x7f3fb419a900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:20.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:20.162+0000 7f3fb8c00640 1 --2- 172.21.15.45:0/796282306 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3fb4105290 0x7f3fb419a900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:20.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:20.162+0000 7f3fb8c00640 1 -- 172.21.15.45:0/796282306 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3fb41a1fe0 con 0x7f3fb41040a0 2024-04-16T17:03:20.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:20.163+0000 7f3fb8c00640 1 --2- 172.21.15.45:0/796282306 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3fb41040a0 0x7f3fb419a3c0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f3fa802efd0 tx=0x7f3fa802c660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:03:20.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:20.163+0000 7f3fb1600640 1 -- 172.21.15.45:0/796282306 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fa8041070 con 0x7f3fb41040a0 2024-04-16T17:03:20.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:20.163+0000 7f3fba000640 1 -- 172.21.15.45:0/796282306 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3fb41a2270 con 0x7f3fb41040a0 2024-04-16T17:03:20.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:20.163+0000 7f3fba000640 1 -- 172.21.15.45:0/796282306 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3fb41a2780 con 0x7f3fb41040a0 2024-04-16T17:03:20.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:20.163+0000 7f3fb1600640 1 -- 172.21.15.45:0/796282306 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3fa8004450 con 0x7f3fb41040a0 2024-04-16T17:03:20.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:20.163+0000 7f3fb1600640 1 -- 172.21.15.45:0/796282306 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fa804b060 con 0x7f3fb41040a0 2024-04-16T17:03:20.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:20.164+0000 7f3fba000640 1 -- 172.21.15.45:0/796282306 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3fb410b3e0 con 0x7f3fb41040a0 2024-04-16T17:03:20.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:20.166+0000 7f3fb1600640 1 -- 172.21.15.45:0/796282306 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f3fa8044070 con 0x7f3fb41040a0 2024-04-16T17:03:20.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:20.167+0000 7f3fb1600640 1 --2- 172.21.15.45:0/796282306 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f3f9c0a3c30 0x7f3f9c0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:20.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:20.168+0000 7f3fb1600640 1 -- 172.21.15.45:0/796282306 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f3fa80f1630 con 0x7f3fb41040a0 2024-04-16T17:03:20.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:20.168+0000 7f3fb3e00640 1 --2- 172.21.15.45:0/796282306 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f3f9c0a3c30 0x7f3f9c0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:20.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:20.168+0000 7f3fb3e00640 1 --2- 172.21.15.45:0/796282306 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f3f9c0a3c30 0x7f3f9c0a60f0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f3fb419b8d0 tx=0x7f3fa0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:03:20.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:20.170+0000 7f3fb1600640 1 -- 172.21.15.45:0/796282306 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3fa80bddc0 con 0x7f3fb41040a0 2024-04-16T17:03:20.179 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:03:19 smithi104 ceph-mon[36457]: pgmap v311: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T17:03:20.179 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:03:19 smithi104 ceph-mon[36457]: mds.? [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] up:boot 2024-04-16T17:03:20.179 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:03:19 smithi104 ceph-mon[36457]: fsmap 12 up:standby 2024-04-16T17:03:20.179 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:03:19 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-16T17:03:20.179 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:03:19 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T17:03:20.228 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:03:19 smithi045 ceph-mon[31676]: pgmap v311: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T17:03:20.228 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:03:19 smithi045 ceph-mon[31676]: mds.? [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] up:boot 2024-04-16T17:03:20.228 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:03:19 smithi045 ceph-mon[31676]: fsmap 12 up:standby 2024-04-16T17:03:20.229 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:03:19 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-16T17:03:20.229 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:03:19 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T17:03:20.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:20.429+0000 7f3fba000640 1 -- 172.21.15.45:0/796282306 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3fb402d620 con 0x7f3fb41040a0 2024-04-16T17:03:20.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:20.430+0000 7f3fb1600640 1 -- 172.21.15.45:0/796282306 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10497 (secure 0 0 0) 0x7f3fa80c20c0 con 0x7f3fb41040a0 2024-04-16T17:03:20.432 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-16T17:03:20.432 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/1748842443","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":1748842443},{"type":"v1","addr":"172.21.15.45:6835","nonce":1748842443}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6839/1308627051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":1308627051},{"type":"v1","addr":"172.21.15.45:6839","nonce":1308627051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/1768242489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1768242489},{"type":"v1","addr":"172.21.15.117:6833","nonce":1768242489}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6833/3558902585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":3558902585},{"type":"v1","addr":"172.21.15.104:6833","nonce":3558902585}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24478,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/1308142903","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":1308142903},{"type":"v1","addr":"172.21.15.45:6837","nonce":1308142903}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24491,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6835/3430679937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":3430679937},{"type":"v1","addr":"172.21.15.104:6835","nonce":3430679937}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24499,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/2224123841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2224123841},{"type":"v1","addr":"172.21.15.117:6835","nonce":2224123841}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24511,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/2471356815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":2471356815},{"type":"v1","addr":"172.21.15.117:6837","nonce":2471356815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24524,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6837/2647756509","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":2647756509},{"type":"v1","addr":"172.21.15.104:6837","nonce":2647756509}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24532,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6839/2378675949","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":2378675949},{"type":"v1","addr":"172.21.15.117:6839","nonce":2378675949}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24569,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6841/3755310822","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6840","nonce":3755310822},{"type":"v1","addr":"172.21.15.45:6841","nonce":3755310822}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24581,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6839/3660070592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6838","nonce":3660070592},{"type":"v1","addr":"172.21.15.104:6839","nonce":3660070592}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[]} 2024-04-16T17:03:20.432 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 13 2024-04-16T17:03:20.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:20.433+0000 7f3fba000640 1 -- 172.21.15.45:0/796282306 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f3f9c0a3c30 msgr2=0x7f3f9c0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:20.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:20.433+0000 7f3fba000640 1 --2- 172.21.15.45:0/796282306 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f3f9c0a3c30 0x7f3f9c0a60f0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f3fb419b8d0 tx=0x7f3fa0005f90 comp rx=0 tx=0).stop 2024-04-16T17:03:20.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:20.433+0000 7f3fba000640 1 -- 172.21.15.45:0/796282306 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3fb41040a0 msgr2=0x7f3fb419a3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:20.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:20.433+0000 7f3fba000640 1 --2- 172.21.15.45:0/796282306 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3fb41040a0 0x7f3fb419a3c0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f3fa802efd0 tx=0x7f3fa802c660 comp rx=0 tx=0).stop 2024-04-16T17:03:20.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:20.433+0000 7f3fba000640 1 -- 172.21.15.45:0/796282306 shutdown_connections 2024-04-16T17:03:20.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:20.433+0000 7f3fba000640 1 --2- 172.21.15.45:0/796282306 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f3f9c0a3c30 0x7f3f9c0a60f0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:20.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:20.433+0000 7f3fba000640 1 --2- 172.21.15.45:0/796282306 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3fb4105c50 0x7f3fb41a1970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:20.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:20.433+0000 7f3fba000640 1 --2- 172.21.15.45:0/796282306 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3fb4105290 0x7f3fb419a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:20.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:20.433+0000 7f3fba000640 1 --2- 172.21.15.45:0/796282306 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3fb41040a0 0x7f3fb419a3c0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:20.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:20.433+0000 7f3fba000640 1 -- 172.21.15.45:0/796282306 >> 172.21.15.45:0/796282306 conn(0x7f3fb40ff610 msgr2=0x7f3fb41085b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:03:20.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:20.433+0000 7f3fba000640 1 -- 172.21.15.45:0/796282306 shutdown_connections 2024-04-16T17:03:20.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:20.433+0000 7f3fba000640 1 -- 172.21.15.45:0/796282306 wait complete. 2024-04-16T17:03:20.454 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-16T17:03:20.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:20.657+0000 7f6567200640 1 -- 172.21.15.45:0/268248244 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f65681039b0 msgr2=0x7f656810a2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:20.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:20.657+0000 7f6567200640 1 --2- 172.21.15.45:0/268248244 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f65681039b0 0x7f656810a2d0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f6560009ca0 tx=0x7f656002c100 comp rx=0 tx=0).stop 2024-04-16T17:03:20.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:20.657+0000 7f6567200640 1 -- 172.21.15.45:0/268248244 shutdown_connections 2024-04-16T17:03:20.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:20.657+0000 7f6567200640 1 --2- 172.21.15.45:0/268248244 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6568102ff0 0x7f6568103470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:20.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:20.657+0000 7f6567200640 1 --2- 172.21.15.45:0/268248244 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f65681039b0 0x7f656810a2d0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:20.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:20.657+0000 7f6567200640 1 --2- 172.21.15.45:0/268248244 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6568101ea0 0x7f65681022c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:20.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:20.657+0000 7f6567200640 1 -- 172.21.15.45:0/268248244 >> 172.21.15.45:0/268248244 conn(0x7f65680fd410 msgr2=0x7f65680ff850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:03:20.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:20.657+0000 7f6567200640 1 -- 172.21.15.45:0/268248244 shutdown_connections 2024-04-16T17:03:20.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:20.657+0000 7f6567200640 1 -- 172.21.15.45:0/268248244 wait complete. 2024-04-16T17:03:20.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:20.658+0000 7f6567200640 1 Processor -- start 2024-04-16T17:03:20.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:20.659+0000 7f6567200640 1 -- start start 2024-04-16T17:03:20.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:20.659+0000 7f6567200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6568101ea0 0x7f656819a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:20.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:20.659+0000 7f6567200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6568102ff0 0x7f656819a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:20.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:20.659+0000 7f6567200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f65681039b0 0x7f65681a18a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:20.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:20.659+0000 7f6567200640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f656810db40 con 0x7f65681039b0 2024-04-16T17:03:20.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:20.659+0000 7f6567200640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f656810d9c0 con 0x7f6568101ea0 2024-04-16T17:03:20.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:20.659+0000 7f6567200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f656810dcc0 con 0x7f6568102ff0 2024-04-16T17:03:20.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:20.659+0000 7f6566800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f65681039b0 0x7f65681a18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:20.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:20.659+0000 7f6565400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6568102ff0 0x7f656819a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:20.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:20.659+0000 7f6566800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f65681039b0 0x7f65681a18a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42812/0 (socket says 172.21.15.45:42812) 2024-04-16T17:03:20.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:20.659+0000 7f6565e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6568101ea0 0x7f656819a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:20.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:20.659+0000 7f6566800640 1 -- 172.21.15.45:0/1378996348 learned_addr learned my addr 172.21.15.45:0/1378996348 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:03:20.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:20.660+0000 7f6565400640 1 -- 172.21.15.45:0/1378996348 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6568101ea0 msgr2=0x7f656819a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:20.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:20.660+0000 7f6565400640 1 --2- 172.21.15.45:0/1378996348 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6568101ea0 0x7f656819a2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:20.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:20.660+0000 7f6565400640 1 -- 172.21.15.45:0/1378996348 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f65681039b0 msgr2=0x7f65681a18a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:20.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:20.660+0000 7f6565400640 1 --2- 172.21.15.45:0/1378996348 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f65681039b0 0x7f65681a18a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:20.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:20.660+0000 7f6565400640 1 -- 172.21.15.45:0/1378996348 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65681a1f10 con 0x7f6568102ff0 2024-04-16T17:03:20.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:20.660+0000 7f6566800640 1 --2- 172.21.15.45:0/1378996348 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f65681039b0 0x7f65681a18a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:03:20.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:20.660+0000 7f6565400640 1 --2- 172.21.15.45:0/1378996348 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6568102ff0 0x7f656819a830 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f655400e9c0 tx=0x7f655400ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:03:20.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:20.660+0000 7f655ea00640 1 -- 172.21.15.45:0/1378996348 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f655400ccc0 con 0x7f6568102ff0 2024-04-16T17:03:20.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:20.660+0000 7f655ea00640 1 -- 172.21.15.45:0/1378996348 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f65540035f0 con 0x7f6568102ff0 2024-04-16T17:03:20.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:20.660+0000 7f655ea00640 1 -- 172.21.15.45:0/1378996348 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f655400ce60 con 0x7f6568102ff0 2024-04-16T17:03:20.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:20.660+0000 7f6567200640 1 -- 172.21.15.45:0/1378996348 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65681a2200 con 0x7f6568102ff0 2024-04-16T17:03:20.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:20.660+0000 7f6567200640 1 -- 172.21.15.45:0/1378996348 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65681a2750 con 0x7f6568102ff0 2024-04-16T17:03:20.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:20.662+0000 7f6567200640 1 -- 172.21.15.45:0/1378996348 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65681a2a10 con 0x7f6568102ff0 2024-04-16T17:03:20.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:20.663+0000 7f655ea00640 1 -- 172.21.15.45:0/1378996348 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f65540037a0 con 0x7f6568102ff0 2024-04-16T17:03:20.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:20.664+0000 7f655ea00640 1 --2- 172.21.15.45:0/1378996348 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f654c0a3c30 0x7f654c0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:20.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:20.665+0000 7f655ea00640 1 -- 172.21.15.45:0/1378996348 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f65540d0460 con 0x7f6568102ff0 2024-04-16T17:03:20.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:20.665+0000 7f6565e00640 1 --2- 172.21.15.45:0/1378996348 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f654c0a3c30 0x7f654c0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:20.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:20.665+0000 7f6565e00640 1 --2- 172.21.15.45:0/1378996348 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f654c0a3c30 0x7f654c0a60f0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f6558009bd0 tx=0x7f6558005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:03:20.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:20.669+0000 7f655ea00640 1 -- 172.21.15.45:0/1378996348 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f655409d240 con 0x7f6568102ff0 2024-04-16T17:03:20.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:20.891+0000 7f6567200640 1 -- 172.21.15.45:0/1378996348 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f65681a2be0 con 0x7f6568102ff0 2024-04-16T17:03:20.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:20.892+0000 7f655ea00640 1 -- 172.21.15.45:0/1378996348 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+18492 (secure 0 0 0) 0x7f655409cb40 con 0x7f6568102ff0 2024-04-16T17:03:20.894 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-16T17:03:20.894 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":82,"fsid":"b1fecd38-fc11-11ee-bc90-c7b262605968","created":"2024-04-16T16:54:02.385906+0000","modified":"2024-04-16T17:02:27.173577+0000","last_up_change":"2024-04-16T17:02:25.162624+0000","last_in_change":"2024-04-16T17:02:04.903274+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-16T16:58:06.166659+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b655de2c-59f8-4fd4-ac98-0cc222229f56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":2290170881},{"type":"v1","addr":"172.21.15.45:6803","nonce":2290170881}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":2290170881},{"type":"v1","addr":"172.21.15.45:6805","nonce":2290170881}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":2290170881},{"type":"v1","addr":"172.21.15.45:6809","nonce":2290170881}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":2290170881},{"type":"v1","addr":"172.21.15.45:6807","nonce":2290170881}]},"public_addr":"172.21.15.45:6803/2290170881","cluster_addr":"172.21.15.45:6805/2290170881","heartbeat_back_addr":"172.21.15.45:6809/2290170881","heartbeat_front_addr":"172.21.15.45:6807/2290170881","state":["exists","up"]},{"osd":1,"uuid":"ebd450ec-08fd-4c3d-8b24-4e10634ddb7e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6800","nonce":1462942067},{"type":"v1","addr":"172.21.15.104:6801","nonce":1462942067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":1462942067},{"type":"v1","addr":"172.21.15.104:6803","nonce":1462942067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":1462942067},{"type":"v1","addr":"172.21.15.104:6807","nonce":1462942067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":1462942067},{"type":"v1","addr":"172.21.15.104:6805","nonce":1462942067}]},"public_addr":"172.21.15.104:6801/1462942067","cluster_addr":"172.21.15.104:6803/1462942067","heartbeat_back_addr":"172.21.15.104:6807/1462942067","heartbeat_front_addr":"172.21.15.104:6805/1462942067","state":["exists","up"]},{"osd":2,"uuid":"b318a512-4190-42a2-8104-dcc4026ad8ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":2829260735},{"type":"v1","addr":"172.21.15.117:6801","nonce":2829260735}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":2829260735},{"type":"v1","addr":"172.21.15.117:6803","nonce":2829260735}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":2829260735},{"type":"v1","addr":"172.21.15.117:6807","nonce":2829260735}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":2829260735},{"type":"v1","addr":"172.21.15.117:6805","nonce":2829260735}]},"public_addr":"172.21.15.117:6801/2829260735","cluster_addr":"172.21.15.117:6803/2829260735","heartbeat_back_addr":"172.21.15.117:6807/2829260735","heartbeat_front_addr":"172.21.15.117:6805/2829260735","state":["exists","up"]},{"osd":3,"uuid":"ec6490cb-a715-4c12-a54a-225aae26fcb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":2823712137},{"type":"v1","addr":"172.21.15.45:6811","nonce":2823712137}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":2823712137},{"type":"v1","addr":"172.21.15.45:6813","nonce":2823712137}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":2823712137},{"type":"v1","addr":"172.21.15.45:6817","nonce":2823712137}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":2823712137},{"type":"v1","addr":"172.21.15.45:6815","nonce":2823712137}]},"public_addr":"172.21.15.45:6811/2823712137","cluster_addr":"172.21.15.45:6813/2823712137","heartbeat_back_addr":"172.21.15.45:6817/2823712137","heartbeat_front_addr":"172.21.15.45:6815/2823712137","state":["exists","up"]},{"osd":4,"uuid":"5d91d568-0250-4aae-8dc2-a0e079f92062","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":1396278127},{"type":"v1","addr":"172.21.15.104:6809","nonce":1396278127}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":1396278127},{"type":"v1","addr":"172.21.15.104:6811","nonce":1396278127}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":1396278127},{"type":"v1","addr":"172.21.15.104:6815","nonce":1396278127}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":1396278127},{"type":"v1","addr":"172.21.15.104:6813","nonce":1396278127}]},"public_addr":"172.21.15.104:6809/1396278127","cluster_addr":"172.21.15.104:6811/1396278127","heartbeat_back_addr":"172.21.15.104:6815/1396278127","heartbeat_front_addr":"172.21.15.104:6813/1396278127","state":["exists","up"]},{"osd":5,"uuid":"ac510b43-6fb2-4d0f-bc5f-16ac080d9e9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":4145720047},{"type":"v1","addr":"172.21.15.117:6809","nonce":4145720047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":4145720047},{"type":"v1","addr":"172.21.15.117:6811","nonce":4145720047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":4145720047},{"type":"v1","addr":"172.21.15.117:6815","nonce":4145720047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":4145720047},{"type":"v1","addr":"172.21.15.117:6813","nonce":4145720047}]},"public_addr":"172.21.15.117:6809/4145720047","cluster_addr":"172.21.15.117:6811/4145720047","heartbeat_back_addr":"172.21.15.117:6815/4145720047","heartbeat_front_addr":"172.21.15.117:6813/4145720047","state":["exists","up"]},{"osd":6,"uuid":"a41adc18-d88a-419e-b2c2-b089c1f32472","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":3254229176},{"type":"v1","addr":"172.21.15.45:6819","nonce":3254229176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":3254229176},{"type":"v1","addr":"172.21.15.45:6821","nonce":3254229176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":3254229176},{"type":"v1","addr":"172.21.15.45:6825","nonce":3254229176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":3254229176},{"type":"v1","addr":"172.21.15.45:6823","nonce":3254229176}]},"public_addr":"172.21.15.45:6819/3254229176","cluster_addr":"172.21.15.45:6821/3254229176","heartbeat_back_addr":"172.21.15.45:6825/3254229176","heartbeat_front_addr":"172.21.15.45:6823/3254229176","state":["exists","up"]},{"osd":7,"uuid":"b71712a4-7cf6-454d-9fd1-565e3df21c1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":4151077186},{"type":"v1","addr":"172.21.15.104:6817","nonce":4151077186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":4151077186},{"type":"v1","addr":"172.21.15.104:6819","nonce":4151077186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":4151077186},{"type":"v1","addr":"172.21.15.104:6823","nonce":4151077186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":4151077186},{"type":"v1","addr":"172.21.15.104:6821","nonce":4151077186}]},"public_addr":"172.21.15.104:6817/4151077186","cluster_addr":"172.21.15.104:6819/4151077186","heartbeat_back_addr":"172.21.15.104:6823/4151077186","heartbeat_front_addr":"172.21.15.104:6821/4151077186","state":["exists","up"]},{"osd":8,"uuid":"9e572aef-59a8-450e-a92e-df9dff4d57a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":1500286830},{"type":"v1","addr":"172.21.15.117:6817","nonce":1500286830}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":1500286830},{"type":"v1","addr":"172.21.15.117:6819","nonce":1500286830}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":1500286830},{"type":"v1","addr":"172.21.15.117:6823","nonce":1500286830}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":1500286830},{"type":"v1","addr":"172.21.15.117:6821","nonce":1500286830}]},"public_addr":"172.21.15.117:6817/1500286830","cluster_addr":"172.21.15.117:6819/1500286830","heartbeat_back_addr":"172.21.15.117:6823/1500286830","heartbeat_front_addr":"172.21.15.117:6821/1500286830","state":["exists","up"]},{"osd":9,"uuid":"014e9598-c413-4cc4-9bb8-61c6e8f59d30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":3106734074},{"type":"v1","addr":"172.21.15.45:6827","nonce":3106734074}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":3106734074},{"type":"v1","addr":"172.21.15.45:6829","nonce":3106734074}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":3106734074},{"type":"v1","addr":"172.21.15.45:6833","nonce":3106734074}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":3106734074},{"type":"v1","addr":"172.21.15.45:6831","nonce":3106734074}]},"public_addr":"172.21.15.45:6827/3106734074","cluster_addr":"172.21.15.45:6829/3106734074","heartbeat_back_addr":"172.21.15.45:6833/3106734074","heartbeat_front_addr":"172.21.15.45:6831/3106734074","state":["exists","up"]},{"osd":10,"uuid":"2deba364-3e39-47be-8a1f-8256df5b82d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":1404794652},{"type":"v1","addr":"172.21.15.104:6825","nonce":1404794652}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":1404794652},{"type":"v1","addr":"172.21.15.104:6827","nonce":1404794652}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":1404794652},{"type":"v1","addr":"172.21.15.104:6831","nonce":1404794652}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":1404794652},{"type":"v1","addr":"172.21.15.104:6829","nonce":1404794652}]},"public_addr":"172.21.15.104:6825/1404794652","cluster_addr":"172.21.15.104:6827/1404794652","heartbeat_back_addr":"172.21.15.104:6831/1404794652","heartbeat_front_addr":"172.21.15.104:6829/1404794652","state":["exists","up"]},{"osd":11,"uuid":"3fda4fea-59ad-4d1a-9fe6-fcb30e31fd7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":3384003243},{"type":"v1","addr":"172.21.15.117:6825","nonce":3384003243}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":3384003243},{"type":"v1","addr":"172.21.15.117:6827","nonce":3384003243}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":3384003243},{"type":"v1","addr":"172.21.15.117:6831","nonce":3384003243}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":3384003243},{"type":"v1","addr":"172.21.15.117:6829","nonce":3384003243}]},"public_addr":"172.21.15.117:6825/3384003243","cluster_addr":"172.21.15.117:6827/3384003243","heartbeat_back_addr":"172.21.15.117:6831/3384003243","heartbeat_front_addr":"172.21.15.117:6829/3384003243","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:57:14.799608+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:57:39.884668+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:58:03.327057+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:58:37.314531+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:59:03.667650+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:59:28.462823+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T17:00:02.968574+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T17:00:29.559190+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T17:00:55.291007+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T17:01:29.815977+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T17:01:57.015580+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T17:02:23.233151+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.45:0/4074982598":"2024-04-17T16:55:19.948001+0000","172.21.15.45:0/2443572576":"2024-04-17T16:55:19.948001+0000","172.21.15.45:6801/1983630136":"2024-04-17T16:55:19.948001+0000","172.21.15.45:6800/1983630136":"2024-04-17T16:55:19.948001+0000","172.21.15.45:0/3150465248":"2024-04-17T16:54:43.692466+0000","172.21.15.45:0/556738623":"2024-04-17T16:54:43.692466+0000","172.21.15.45:0/445676297":"2024-04-17T16:55:19.948001+0000","172.21.15.45:6800/2553659622":"2024-04-17T16:54:43.692466+0000","172.21.15.45:6801/2553659622":"2024-04-17T16:54:43.692466+0000","172.21.15.45:0/1184615408":"2024-04-17T16:54:43.692466+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-16T17:03:20.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:20.895+0000 7f6567200640 1 -- 172.21.15.45:0/1378996348 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f654c0a3c30 msgr2=0x7f654c0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:20.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:20.895+0000 7f6567200640 1 --2- 172.21.15.45:0/1378996348 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f654c0a3c30 0x7f654c0a60f0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f6558009bd0 tx=0x7f6558005f90 comp rx=0 tx=0).stop 2024-04-16T17:03:20.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:20.895+0000 7f6567200640 1 -- 172.21.15.45:0/1378996348 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6568102ff0 msgr2=0x7f656819a830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:20.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:20.895+0000 7f6567200640 1 --2- 172.21.15.45:0/1378996348 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6568102ff0 0x7f656819a830 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f655400e9c0 tx=0x7f655400ee80 comp rx=0 tx=0).stop 2024-04-16T17:03:20.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:20.895+0000 7f6567200640 1 -- 172.21.15.45:0/1378996348 shutdown_connections 2024-04-16T17:03:20.897 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:20.895+0000 7f6567200640 1 --2- 172.21.15.45:0/1378996348 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f654c0a3c30 0x7f654c0a60f0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:20.897 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:20.895+0000 7f6567200640 1 --2- 172.21.15.45:0/1378996348 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6568102ff0 0x7f656819a830 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:20.897 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:20.895+0000 7f6567200640 1 --2- 172.21.15.45:0/1378996348 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f65681039b0 0x7f65681a18a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:20.897 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:20.895+0000 7f6567200640 1 --2- 172.21.15.45:0/1378996348 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6568101ea0 0x7f656819a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:20.897 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:20.895+0000 7f6567200640 1 -- 172.21.15.45:0/1378996348 >> 172.21.15.45:0/1378996348 conn(0x7f65680fd410 msgr2=0x7f6568074230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:03:20.897 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:20.895+0000 7f6567200640 1 -- 172.21.15.45:0/1378996348 shutdown_connections 2024-04-16T17:03:20.897 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:20.895+0000 7f6567200640 1 -- 172.21.15.45:0/1378996348 wait complete. 2024-04-16T17:03:20.915 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-16T16:58:06.166659+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-16T17:03:20.915 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-16T17:03:21.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:21.111+0000 7fbfdee00640 1 -- 172.21.15.45:0/1012264091 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbfd8076040 msgr2=0x7fbfd8074490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:21.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:21.111+0000 7fbfdee00640 1 --2- 172.21.15.45:0/1012264091 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbfd8076040 0x7fbfd8074490 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fbfcc009900 tx=0x7fbfcc02bc40 comp rx=0 tx=0).stop 2024-04-16T17:03:21.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:21.112+0000 7fbfdee00640 1 -- 172.21.15.45:0/1012264091 shutdown_connections 2024-04-16T17:03:21.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:21.112+0000 7fbfdee00640 1 --2- 172.21.15.45:0/1012264091 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbfd8074a60 0x7fbfd8074ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:21.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:21.112+0000 7fbfdee00640 1 --2- 172.21.15.45:0/1012264091 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbfd8109d00 0x7fbfd810c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:21.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:21.112+0000 7fbfdee00640 1 --2- 172.21.15.45:0/1012264091 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbfd8076040 0x7fbfd8074490 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:21.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:21.112+0000 7fbfdee00640 1 -- 172.21.15.45:0/1012264091 >> 172.21.15.45:0/1012264091 conn(0x7fbfd80fd3b0 msgr2=0x7fbfd80ff7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:03:21.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:21.112+0000 7fbfdee00640 1 -- 172.21.15.45:0/1012264091 shutdown_connections 2024-04-16T17:03:21.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:21.112+0000 7fbfdee00640 1 -- 172.21.15.45:0/1012264091 wait complete. 2024-04-16T17:03:21.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:21.113+0000 7fbfdee00640 1 Processor -- start 2024-04-16T17:03:21.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:21.113+0000 7fbfdee00640 1 -- start start 2024-04-16T17:03:21.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:21.114+0000 7fbfdee00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbfd8074a60 0x7fbfd819a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:21.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:21.114+0000 7fbfdee00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbfd8076040 0x7fbfd819a860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:21.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:21.114+0000 7fbfdee00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbfd8109d00 0x7fbfd81a18d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:21.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:21.114+0000 7fbfdee00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fbfd810fcd0 con 0x7fbfd8076040 2024-04-16T17:03:21.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:21.114+0000 7fbfdee00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fbfd810fb50 con 0x7fbfd8109d00 2024-04-16T17:03:21.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:21.114+0000 7fbfdee00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fbfd810fe50 con 0x7fbfd8074a60 2024-04-16T17:03:21.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:21.114+0000 7fbfdd000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbfd8076040 0x7fbfd819a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:21.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:21.114+0000 7fbfdda00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbfd8074a60 0x7fbfd819a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:21.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:21.114+0000 7fbfdd000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbfd8076040 0x7fbfd819a860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42840/0 (socket says 172.21.15.45:42840) 2024-04-16T17:03:21.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:21.114+0000 7fbfdd000640 1 -- 172.21.15.45:0/3873305125 learned_addr learned my addr 172.21.15.45:0/3873305125 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:03:21.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:21.114+0000 7fbfde400640 1 --2- 172.21.15.45:0/3873305125 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbfd8109d00 0x7fbfd81a18d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:21.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:21.114+0000 7fbfdda00640 1 -- 172.21.15.45:0/3873305125 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbfd8109d00 msgr2=0x7fbfd81a18d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:21.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:21.114+0000 7fbfdda00640 1 --2- 172.21.15.45:0/3873305125 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbfd8109d00 0x7fbfd81a18d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:21.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:21.114+0000 7fbfdda00640 1 -- 172.21.15.45:0/3873305125 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbfd8076040 msgr2=0x7fbfd819a860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:21.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:21.114+0000 7fbfdda00640 1 --2- 172.21.15.45:0/3873305125 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbfd8076040 0x7fbfd819a860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:21.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:21.114+0000 7fbfdda00640 1 -- 172.21.15.45:0/3873305125 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfd81a1f40 con 0x7fbfd8074a60 2024-04-16T17:03:21.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:21.114+0000 7fbfdd000640 1 --2- 172.21.15.45:0/3873305125 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbfd8076040 0x7fbfd819a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:03:21.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:21.114+0000 7fbfde400640 1 --2- 172.21.15.45:0/3873305125 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbfd8109d00 0x7fbfd81a18d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:03:21.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:21.115+0000 7fbfdda00640 1 --2- 172.21.15.45:0/3873305125 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbfd8074a60 0x7fbfd819a320 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fbfcc036040 tx=0x7fbfcc005d50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:03:21.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:21.115+0000 7fbfd6000640 1 -- 172.21.15.45:0/3873305125 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfcc041070 con 0x7fbfd8074a60 2024-04-16T17:03:21.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:21.115+0000 7fbfdee00640 1 -- 172.21.15.45:0/3873305125 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbfd81a21d0 con 0x7fbfd8074a60 2024-04-16T17:03:21.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:21.115+0000 7fbfd6000640 1 -- 172.21.15.45:0/3873305125 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbfcc0044d0 con 0x7fbfd8074a60 2024-04-16T17:03:21.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:21.115+0000 7fbfd6000640 1 -- 172.21.15.45:0/3873305125 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfcc034440 con 0x7fbfd8074a60 2024-04-16T17:03:21.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:21.115+0000 7fbfdee00640 1 -- 172.21.15.45:0/3873305125 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbfd81a26e0 con 0x7fbfd8074a60 2024-04-16T17:03:21.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:21.117+0000 7fbfdee00640 1 -- 172.21.15.45:0/3873305125 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbfd8108a30 con 0x7fbfd8074a60 2024-04-16T17:03:21.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:21.118+0000 7fbfd6000640 1 -- 172.21.15.45:0/3873305125 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fbfcc044070 con 0x7fbfd8074a60 2024-04-16T17:03:21.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:21.120+0000 7fbfd6000640 1 --2- 172.21.15.45:0/3873305125 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fbfc00a3bb0 0x7fbfc00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:21.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:21.120+0000 7fbfd6000640 1 -- 172.21.15.45:0/3873305125 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fbfcc0f1a80 con 0x7fbfd8074a60 2024-04-16T17:03:21.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:21.120+0000 7fbfdd000640 1 --2- 172.21.15.45:0/3873305125 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fbfc00a3bb0 0x7fbfc00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:21.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:21.121+0000 7fbfdd000640 1 --2- 172.21.15.45:0/3873305125 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fbfc00a3bb0 0x7fbfc00a6070 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fbfd819b830 tx=0x7fbfc4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:03:21.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:21.123+0000 7fbfd6000640 1 -- 172.21.15.45:0/3873305125 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fbfcc0f3e20 con 0x7fbfd8074a60 2024-04-16T17:03:21.228 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:03:21 smithi045 ceph-mon[31676]: pgmap v312: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T17:03:21.228 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:03:21 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:03:21.228 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:03:21 smithi045 ceph-mon[31676]: from='client.? 172.21.15.45:0/796282306' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:03:21.229 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:03:21 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:03:21.229 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:03:21 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:03:21.229 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:03:21 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:03:21.229 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:03:21 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:03:21.229 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:03:21 smithi045 ceph-mon[31676]: from='client.? 172.21.15.45:0/1378996348' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T17:03:21.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:21.344+0000 7fbfdee00640 1 -- 172.21.15.45:0/3873305125 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fbfd8062c00 con 0x7fbfd8074a60 2024-04-16T17:03:21.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:21.345+0000 7fbfd6000640 1 -- 172.21.15.45:0/3873305125 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) ==== 93+0+10 (secure 0 0 0) 0x7fbfcc0be210 con 0x7fbfd8074a60 2024-04-16T17:03:21.346 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 1 2024-04-16T17:03:21.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:21.347+0000 7fbfdee00640 1 -- 172.21.15.45:0/3873305125 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fbfc00a3bb0 msgr2=0x7fbfc00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:21.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:21.347+0000 7fbfdee00640 1 --2- 172.21.15.45:0/3873305125 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fbfc00a3bb0 0x7fbfc00a6070 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fbfd819b830 tx=0x7fbfc4005f90 comp rx=0 tx=0).stop 2024-04-16T17:03:21.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:21.347+0000 7fbfdee00640 1 -- 172.21.15.45:0/3873305125 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbfd8074a60 msgr2=0x7fbfd819a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:21.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:21.347+0000 7fbfdee00640 1 --2- 172.21.15.45:0/3873305125 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbfd8074a60 0x7fbfd819a320 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fbfcc036040 tx=0x7fbfcc005d50 comp rx=0 tx=0).stop 2024-04-16T17:03:21.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:21.347+0000 7fbfdee00640 1 -- 172.21.15.45:0/3873305125 shutdown_connections 2024-04-16T17:03:21.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:21.347+0000 7fbfdee00640 1 --2- 172.21.15.45:0/3873305125 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fbfc00a3bb0 0x7fbfc00a6070 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:21.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:21.347+0000 7fbfdee00640 1 --2- 172.21.15.45:0/3873305125 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbfd8109d00 0x7fbfd81a18d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:21.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:21.347+0000 7fbfdee00640 1 --2- 172.21.15.45:0/3873305125 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbfd8076040 0x7fbfd819a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:21.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:21.347+0000 7fbfdee00640 1 --2- 172.21.15.45:0/3873305125 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbfd8074a60 0x7fbfd819a320 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:21.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:21.347+0000 7fbfdee00640 1 -- 172.21.15.45:0/3873305125 >> 172.21.15.45:0/3873305125 conn(0x7fbfd80fd3b0 msgr2=0x7fbfd8105c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:03:21.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:21.348+0000 7fbfdee00640 1 -- 172.21.15.45:0/3873305125 shutdown_connections 2024-04-16T17:03:21.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:21.348+0000 7fbfdee00640 1 -- 172.21.15.45:0/3873305125 wait complete. 2024-04-16T17:03:21.370 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-16T17:03:21.428 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:03:21 smithi104 ceph-mon[36457]: pgmap v312: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T17:03:21.429 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:03:21 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:03:21.429 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:03:21 smithi104 ceph-mon[36457]: from='client.? 172.21.15.45:0/796282306' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:03:21.429 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:03:21 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:03:21.429 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:03:21 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:03:21.429 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:03:21 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:03:21.429 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:03:21 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:03:21.429 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:03:21 smithi104 ceph-mon[36457]: from='client.? 172.21.15.45:0/1378996348' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T17:03:21.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:03:21 smithi117 ceph-mon[36730]: pgmap v312: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T17:03:21.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:03:21 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:03:21.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:03:21 smithi117 ceph-mon[36730]: from='client.? 172.21.15.45:0/796282306' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:03:21.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:03:21 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:03:21.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:03:21 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:03:21.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:03:21 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:03:21.557 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:03:21 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:03:21.557 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:03:21 smithi117 ceph-mon[36730]: from='client.? 172.21.15.45:0/1378996348' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T17:03:21.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:21.571+0000 7fb3d1a00640 1 -- 172.21.15.45:0/3771155468 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb3cc105470 msgr2=0x7fb3cc1058f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:21.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:21.571+0000 7fb3d1a00640 1 --2- 172.21.15.45:0/3771155468 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb3cc105470 0x7fb3cc1058f0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fb3b4009900 tx=0x7fb3b402bc30 comp rx=0 tx=0).stop 2024-04-16T17:03:21.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:21.571+0000 7fb3d1a00640 1 -- 172.21.15.45:0/3771155468 shutdown_connections 2024-04-16T17:03:21.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:21.571+0000 7fb3d1a00640 1 --2- 172.21.15.45:0/3771155468 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb3cc105e30 0x7fb3cc10c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:21.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:21.571+0000 7fb3d1a00640 1 --2- 172.21.15.45:0/3771155468 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb3cc105470 0x7fb3cc1058f0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:21.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:21.571+0000 7fb3d1a00640 1 --2- 172.21.15.45:0/3771155468 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb3cc104280 0x7fb3cc1046a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:21.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:21.571+0000 7fb3d1a00640 1 -- 172.21.15.45:0/3771155468 >> 172.21.15.45:0/3771155468 conn(0x7fb3cc0ff7f0 msgr2=0x7fb3cc101c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:03:21.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:21.571+0000 7fb3d1a00640 1 -- 172.21.15.45:0/3771155468 shutdown_connections 2024-04-16T17:03:21.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:21.571+0000 7fb3d1a00640 1 -- 172.21.15.45:0/3771155468 wait complete. 2024-04-16T17:03:21.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:21.572+0000 7fb3d1a00640 1 Processor -- start 2024-04-16T17:03:21.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:21.572+0000 7fb3d1a00640 1 -- start start 2024-04-16T17:03:21.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:21.572+0000 7fb3d1a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb3cc104280 0x7fb3cc19a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:21.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:21.572+0000 7fb3d1a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb3cc105470 0x7fb3cc19aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:21.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:21.573+0000 7fb3d1a00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb3cc105e30 0x7fb3cc1a1b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:21.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:21.573+0000 7fb3d1a00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fb3cc10ff60 con 0x7fb3cc104280 2024-04-16T17:03:21.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:21.573+0000 7fb3d1a00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fb3cc10fde0 con 0x7fb3cc105e30 2024-04-16T17:03:21.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:21.573+0000 7fb3d1a00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb3cc1100e0 con 0x7fb3cc105470 2024-04-16T17:03:21.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:21.573+0000 7fb3caa00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb3cc104280 0x7fb3cc19a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:21.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:21.573+0000 7fb3caa00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb3cc104280 0x7fb3cc19a560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42858/0 (socket says 172.21.15.45:42858) 2024-04-16T17:03:21.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:21.573+0000 7fb3caa00640 1 -- 172.21.15.45:0/287854665 learned_addr learned my addr 172.21.15.45:0/287854665 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:03:21.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:21.573+0000 7fb3cb400640 1 --2- 172.21.15.45:0/287854665 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb3cc105e30 0x7fb3cc1a1b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:21.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:21.573+0000 7fb3c2000640 1 --2- 172.21.15.45:0/287854665 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb3cc105470 0x7fb3cc19aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:21.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:21.573+0000 7fb3caa00640 1 -- 172.21.15.45:0/287854665 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb3cc105470 msgr2=0x7fb3cc19aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:21.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:21.573+0000 7fb3caa00640 1 --2- 172.21.15.45:0/287854665 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb3cc105470 0x7fb3cc19aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:21.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:21.573+0000 7fb3caa00640 1 -- 172.21.15.45:0/287854665 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb3cc105e30 msgr2=0x7fb3cc1a1b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:21.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:21.573+0000 7fb3caa00640 1 --2- 172.21.15.45:0/287854665 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb3cc105e30 0x7fb3cc1a1b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:21.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:21.573+0000 7fb3caa00640 1 -- 172.21.15.45:0/287854665 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3cc1a2180 con 0x7fb3cc104280 2024-04-16T17:03:21.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:21.574+0000 7fb3caa00640 1 --2- 172.21.15.45:0/287854665 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb3cc104280 0x7fb3cc19a560 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fb3bc00ebc0 tx=0x7fb3bc00c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:03:21.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:21.574+0000 7fb3c3e00640 1 -- 172.21.15.45:0/287854665 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3bc003b70 con 0x7fb3cc104280 2024-04-16T17:03:21.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:21.574+0000 7fb3c3e00640 1 -- 172.21.15.45:0/287854665 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb3bc003d10 con 0x7fb3cc104280 2024-04-16T17:03:21.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:21.574+0000 7fb3d1a00640 1 -- 172.21.15.45:0/287854665 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3cc1a2470 con 0x7fb3cc104280 2024-04-16T17:03:21.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:21.574+0000 7fb3c3e00640 1 -- 172.21.15.45:0/287854665 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3bc019980 con 0x7fb3cc104280 2024-04-16T17:03:21.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:21.574+0000 7fb3d1a00640 1 -- 172.21.15.45:0/287854665 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3cc1a2990 con 0x7fb3cc104280 2024-04-16T17:03:21.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:21.575+0000 7fb3d1a00640 1 -- 172.21.15.45:0/287854665 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb3cc10b540 con 0x7fb3cc104280 2024-04-16T17:03:21.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:21.578+0000 7fb3c3e00640 1 -- 172.21.15.45:0/287854665 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fb3bc019b20 con 0x7fb3cc104280 2024-04-16T17:03:21.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:21.579+0000 7fb3c3e00640 1 --2- 172.21.15.45:0/287854665 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fb3b00a3bb0 0x7fb3b00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:21.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:21.579+0000 7fb3c3e00640 1 -- 172.21.15.45:0/287854665 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fb3bc0d0900 con 0x7fb3cc104280 2024-04-16T17:03:21.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:21.580+0000 7fb3c2000640 1 --2- 172.21.15.45:0/287854665 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fb3b00a3bb0 0x7fb3b00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:21.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:21.580+0000 7fb3c2000640 1 --2- 172.21.15.45:0/287854665 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fb3b00a3bb0 0x7fb3b00a6070 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fb3b402c240 tx=0x7fb3b402df90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:03:21.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:21.582+0000 7fb3c3e00640 1 -- 172.21.15.45:0/287854665 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb3bc09d770 con 0x7fb3cc104280 2024-04-16T17:03:21.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:21.811+0000 7fb3d1a00640 1 -- 172.21.15.45:0/287854665 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fb3cc062c00 con 0x7fb3cc104280 2024-04-16T17:03:21.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:21.811+0000 7fb3c3e00640 1 -- 172.21.15.45:0/287854665 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v82) ==== 84+0+65 (secure 0 0 0) 0x7fb3bc09d070 con 0x7fb3cc104280 2024-04-16T17:03:21.812 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-16T17:03:21.812 INFO:teuthology.orchestra.run.smithi045.stdout:[ 2024-04-16T17:03:21.813 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-16T17:03:21.813 INFO:teuthology.orchestra.run.smithi045.stdout: "poolnum": 1, 2024-04-16T17:03:21.813 INFO:teuthology.orchestra.run.smithi045.stdout: "poolname": ".mgr" 2024-04-16T17:03:21.813 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:03:21.813 INFO:teuthology.orchestra.run.smithi045.stdout:] 2024-04-16T17:03:21.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:21.814+0000 7fb3d1a00640 1 -- 172.21.15.45:0/287854665 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fb3b00a3bb0 msgr2=0x7fb3b00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:21.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:21.814+0000 7fb3d1a00640 1 --2- 172.21.15.45:0/287854665 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fb3b00a3bb0 0x7fb3b00a6070 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fb3b402c240 tx=0x7fb3b402df90 comp rx=0 tx=0).stop 2024-04-16T17:03:21.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:21.814+0000 7fb3d1a00640 1 -- 172.21.15.45:0/287854665 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb3cc104280 msgr2=0x7fb3cc19a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:21.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:21.814+0000 7fb3d1a00640 1 --2- 172.21.15.45:0/287854665 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb3cc104280 0x7fb3cc19a560 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fb3bc00ebc0 tx=0x7fb3bc00c6a0 comp rx=0 tx=0).stop 2024-04-16T17:03:21.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:21.814+0000 7fb3d1a00640 1 -- 172.21.15.45:0/287854665 shutdown_connections 2024-04-16T17:03:21.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:21.814+0000 7fb3d1a00640 1 --2- 172.21.15.45:0/287854665 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fb3b00a3bb0 0x7fb3b00a6070 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:21.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:21.814+0000 7fb3d1a00640 1 --2- 172.21.15.45:0/287854665 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb3cc105470 0x7fb3cc19aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:21.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:21.814+0000 7fb3d1a00640 1 --2- 172.21.15.45:0/287854665 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb3cc105e30 0x7fb3cc1a1b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:21.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:21.814+0000 7fb3d1a00640 1 --2- 172.21.15.45:0/287854665 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb3cc104280 0x7fb3cc19a560 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:21.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:21.814+0000 7fb3d1a00640 1 -- 172.21.15.45:0/287854665 >> 172.21.15.45:0/287854665 conn(0x7fb3cc0ff7f0 msgr2=0x7fb3cc108710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:03:21.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:21.814+0000 7fb3d1a00640 1 -- 172.21.15.45:0/287854665 shutdown_connections 2024-04-16T17:03:21.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:21.814+0000 7fb3d1a00640 1 -- 172.21.15.45:0/287854665 wait complete. 2024-04-16T17:03:21.836 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-16T17:03:21.836 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-16T17:03:22.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:22.062+0000 7f1872e00640 1 -- 172.21.15.45:0/2895209567 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f186c102ae0 msgr2=0x7f186c104ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:22.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:22.062+0000 7f1872e00640 1 --2- 172.21.15.45:0/2895209567 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f186c102ae0 0x7f186c104ed0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f185c009900 tx=0x7f185c02bc50 comp rx=0 tx=0).stop 2024-04-16T17:03:22.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:22.063+0000 7f1872e00640 1 -- 172.21.15.45:0/2895209567 shutdown_connections 2024-04-16T17:03:22.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:22.063+0000 7f1872e00640 1 --2- 172.21.15.45:0/2895209567 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f186c107d40 0x7f186c10a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:22.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:22.063+0000 7f1872e00640 1 --2- 172.21.15.45:0/2895209567 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f186c105410 0x7f186c107800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:22.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:22.063+0000 7f1872e00640 1 --2- 172.21.15.45:0/2895209567 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f186c102ae0 0x7f186c104ed0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:22.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:22.063+0000 7f1872e00640 1 -- 172.21.15.45:0/2895209567 >> 172.21.15.45:0/2895209567 conn(0x7f186c0fc4d0 msgr2=0x7f186c0fe910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:03:22.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:22.063+0000 7f1872e00640 1 -- 172.21.15.45:0/2895209567 shutdown_connections 2024-04-16T17:03:22.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:22.063+0000 7f1872e00640 1 -- 172.21.15.45:0/2895209567 wait complete. 2024-04-16T17:03:22.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:22.064+0000 7f1872e00640 1 Processor -- start 2024-04-16T17:03:22.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:22.064+0000 7f1872e00640 1 -- start start 2024-04-16T17:03:22.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:22.064+0000 7f1872e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f186c102ae0 0x7f186c19a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:22.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:22.064+0000 7f1872e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f186c105410 0x7f186c19ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:22.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:22.064+0000 7f1872e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f186c107d40 0x7f186c1a1ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:22.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:22.064+0000 7f1872e00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f186c10dd40 con 0x7f186c107d40 2024-04-16T17:03:22.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:22.064+0000 7f1872e00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f186c10dbc0 con 0x7f186c102ae0 2024-04-16T17:03:22.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:22.064+0000 7f1872e00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f186c10dec0 con 0x7f186c105410 2024-04-16T17:03:22.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:22.065+0000 7f186be00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f186c102ae0 0x7f186c19a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:22.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:22.065+0000 7f186be00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f186c102ae0 0x7f186c19a5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:52010/0 (socket says 172.21.15.45:52010) 2024-04-16T17:03:22.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:22.065+0000 7f186be00640 1 -- 172.21.15.45:0/250618073 learned_addr learned my addr 172.21.15.45:0/250618073 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:03:22.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:22.065+0000 7f186b400640 1 --2- 172.21.15.45:0/250618073 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f186c105410 0x7f186c19ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:22.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:22.065+0000 7f1870e00640 1 --2- 172.21.15.45:0/250618073 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f186c107d40 0x7f186c1a1ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:22.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:22.065+0000 7f186be00640 1 -- 172.21.15.45:0/250618073 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f186c105410 msgr2=0x7f186c19ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:22.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:22.065+0000 7f186be00640 1 --2- 172.21.15.45:0/250618073 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f186c105410 0x7f186c19ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:22.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:22.065+0000 7f186be00640 1 -- 172.21.15.45:0/250618073 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f186c107d40 msgr2=0x7f186c1a1ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:22.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:22.065+0000 7f186be00640 1 --2- 172.21.15.45:0/250618073 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f186c107d40 0x7f186c1a1ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:22.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:22.065+0000 7f186be00640 1 -- 172.21.15.45:0/250618073 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f186c1a2210 con 0x7f186c102ae0 2024-04-16T17:03:22.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:22.065+0000 7f1870e00640 1 --2- 172.21.15.45:0/250618073 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f186c107d40 0x7f186c1a1ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:03:22.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:22.066+0000 7f186be00640 1 --2- 172.21.15.45:0/250618073 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f186c102ae0 0x7f186c19a5f0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f185c02c260 tx=0x7f185c02c670 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:03:22.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:22.066+0000 7f1868c00640 1 -- 172.21.15.45:0/250618073 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f185c041070 con 0x7f186c102ae0 2024-04-16T17:03:22.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:22.066+0000 7f1868c00640 1 -- 172.21.15.45:0/250618073 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f185c004450 con 0x7f186c102ae0 2024-04-16T17:03:22.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:22.066+0000 7f1872e00640 1 -- 172.21.15.45:0/250618073 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f186c1a24a0 con 0x7f186c102ae0 2024-04-16T17:03:22.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:22.066+0000 7f1868c00640 1 -- 172.21.15.45:0/250618073 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f185c034440 con 0x7f186c102ae0 2024-04-16T17:03:22.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:22.066+0000 7f1872e00640 1 -- 172.21.15.45:0/250618073 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f186c1a29f0 con 0x7f186c102ae0 2024-04-16T17:03:22.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:22.068+0000 7f1872e00640 1 -- 172.21.15.45:0/250618073 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f186c109890 con 0x7f186c102ae0 2024-04-16T17:03:22.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:22.070+0000 7f1868c00640 1 -- 172.21.15.45:0/250618073 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f185c044070 con 0x7f186c102ae0 2024-04-16T17:03:22.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:22.071+0000 7f1868c00640 1 --2- 172.21.15.45:0/250618073 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f18500a3bb0 0x7f18500a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:22.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:22.071+0000 7f1868c00640 1 -- 172.21.15.45:0/250618073 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f185c0f1b40 con 0x7f186c102ae0 2024-04-16T17:03:22.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:22.071+0000 7f186b400640 1 --2- 172.21.15.45:0/250618073 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f18500a3bb0 0x7f18500a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:22.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:22.072+0000 7f186b400640 1 --2- 172.21.15.45:0/250618073 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f18500a3bb0 0x7f18500a6070 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f186c19bb00 tx=0x7f1858005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:03:22.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:22.076+0000 7f1868c00640 1 -- 172.21.15.45:0/250618073 <== mon.2 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f185c0be920 con 0x7f186c102ae0 2024-04-16T17:03:22.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:22.290+0000 7f1872e00640 1 -- 172.21.15.45:0/250618073 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f186c02d620 con 0x7f186c102ae0 2024-04-16T17:03:22.428 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:03:22 smithi104 ceph-mon[36457]: from='client.? 172.21.15.45:0/3873305125' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-16T17:03:22.428 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:03:22 smithi104 ceph-mon[36457]: from='client.? 172.21.15.45:0/287854665' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-16T17:03:22.428 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:03:22 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:03:22.428 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:03:22 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:03:22.428 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:03:22 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:03:22.428 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:03:22 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T17:03:22.429 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:03:22 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:03:22.478 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:03:22 smithi045 ceph-mon[31676]: from='client.? 172.21.15.45:0/3873305125' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-16T17:03:22.478 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:03:22 smithi045 ceph-mon[31676]: from='client.? 172.21.15.45:0/287854665' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-16T17:03:22.479 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:03:22 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:03:22.479 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:03:22 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:03:22.479 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:03:22 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:03:22.479 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:03:22 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T17:03:22.479 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:03:22 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:03:22.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:03:22 smithi117 ceph-mon[36730]: from='client.? 172.21.15.45:0/3873305125' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-16T17:03:22.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:03:22 smithi117 ceph-mon[36730]: from='client.? 172.21.15.45:0/287854665' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-16T17:03:22.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:03:22 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:03:22.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:03:22 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:03:22.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:03:22 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:03:22.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:03:22 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T17:03:22.557 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:03:22 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:03:23.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:23.138+0000 7f1868c00640 1 -- 172.21.15.45:0/250618073 <== mon.2 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v83) ==== 138+0+0 (secure 0 0 0) 0x7f185c0be220 con 0x7f186c102ae0 2024-04-16T17:03:23.140 INFO:teuthology.orchestra.run.smithi045.stderr:pool 'cephfs_metadata' created 2024-04-16T17:03:23.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:23.145+0000 7f1872e00640 1 -- 172.21.15.45:0/250618073 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f18500a3bb0 msgr2=0x7f18500a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:23.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:23.146+0000 7f1872e00640 1 --2- 172.21.15.45:0/250618073 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f18500a3bb0 0x7f18500a6070 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f186c19bb00 tx=0x7f1858005f90 comp rx=0 tx=0).stop 2024-04-16T17:03:23.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:23.146+0000 7f1872e00640 1 -- 172.21.15.45:0/250618073 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f186c102ae0 msgr2=0x7f186c19a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:23.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:23.146+0000 7f1872e00640 1 --2- 172.21.15.45:0/250618073 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f186c102ae0 0x7f186c19a5f0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f185c02c260 tx=0x7f185c02c670 comp rx=0 tx=0).stop 2024-04-16T17:03:23.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:23.146+0000 7f1872e00640 1 -- 172.21.15.45:0/250618073 shutdown_connections 2024-04-16T17:03:23.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:23.146+0000 7f1872e00640 1 --2- 172.21.15.45:0/250618073 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f18500a3bb0 0x7f18500a6070 secure :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f186c19bb00 tx=0x7f1858005f90 comp rx=0 tx=0).stop 2024-04-16T17:03:23.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:23.146+0000 7f1872e00640 1 --2- 172.21.15.45:0/250618073 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f186c105410 0x7f186c19ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:23.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:23.146+0000 7f1872e00640 1 --2- 172.21.15.45:0/250618073 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f186c107d40 0x7f186c1a1ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:23.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:23.146+0000 7f1872e00640 1 --2- 172.21.15.45:0/250618073 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f186c102ae0 0x7f186c19a5f0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:23.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:23.146+0000 7f1872e00640 1 -- 172.21.15.45:0/250618073 >> 172.21.15.45:0/250618073 conn(0x7f186c0fc4d0 msgr2=0x7f186c1022d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:03:23.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:23.146+0000 7f1872e00640 1 -- 172.21.15.45:0/250618073 shutdown_connections 2024-04-16T17:03:23.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:23.146+0000 7f1872e00640 1 -- 172.21.15.45:0/250618073 wait complete. 2024-04-16T17:03:23.165 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2024-04-16T17:03:23.306 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:03:23 smithi117 ceph-mon[36730]: pgmap v313: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T17:03:23.306 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:03:23 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T17:03:23.306 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:03:23 smithi117 ceph-mon[36730]: from='client.? 172.21.15.45:0/250618073' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-16T17:03:23.306 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:03:23 smithi117 ceph-mon[36730]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-16T17:03:23.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:23.377+0000 7f69efe00640 1 -- 172.21.15.45:0/1268078728 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f69f0100510 msgr2=0x7f69f0107710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:23.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:23.377+0000 7f69efe00640 1 --2- 172.21.15.45:0/1268078728 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f69f0100510 0x7f69f0107710 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f69d8009900 tx=0x7f69d802bc30 comp rx=0 tx=0).stop 2024-04-16T17:03:23.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:23.377+0000 7f69efe00640 1 -- 172.21.15.45:0/1268078728 shutdown_connections 2024-04-16T17:03:23.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:23.377+0000 7f69efe00640 1 --2- 172.21.15.45:0/1268078728 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f69f0100510 0x7f69f0107710 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:23.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:23.377+0000 7f69efe00640 1 --2- 172.21.15.45:0/1268078728 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f69f0107c50 0x7f69f010a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:23.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:23.377+0000 7f69efe00640 1 --2- 172.21.15.45:0/1268078728 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f69f00ffb40 0x7f69f00fff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:23.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:23.377+0000 7f69efe00640 1 -- 172.21.15.45:0/1268078728 >> 172.21.15.45:0/1268078728 conn(0x7f69f00fb6c0 msgr2=0x7f69f00fdb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:03:23.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:23.377+0000 7f69efe00640 1 -- 172.21.15.45:0/1268078728 shutdown_connections 2024-04-16T17:03:23.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:23.377+0000 7f69efe00640 1 -- 172.21.15.45:0/1268078728 wait complete. 2024-04-16T17:03:23.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:23.378+0000 7f69efe00640 1 Processor -- start 2024-04-16T17:03:23.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:23.379+0000 7f69efe00640 1 -- start start 2024-04-16T17:03:23.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:23.379+0000 7f69efe00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f69f00ffb40 0x7f69f0198260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:23.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:23.379+0000 7f69efe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f69f0100510 0x7f69f01987a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:23.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:23.379+0000 7f69efe00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f69f0107c50 0x7f69f019f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:23.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:23.379+0000 7f69efe00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f69f010dbf0 con 0x7f69f0100510 2024-04-16T17:03:23.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:23.379+0000 7f69efe00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f69f010da70 con 0x7f69f00ffb40 2024-04-16T17:03:23.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:23.379+0000 7f69efe00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f69f010dd70 con 0x7f69f0107c50 2024-04-16T17:03:23.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:23.379+0000 7f69ef400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f69f0107c50 0x7f69f019f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:23.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:23.379+0000 7f69eea00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f69f00ffb40 0x7f69f0198260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:23.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:23.379+0000 7f69e7e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f69f0100510 0x7f69f01987a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:23.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:23.379+0000 7f69ef400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f69f0107c50 0x7f69f019f810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:40194/0 (socket says 172.21.15.45:40194) 2024-04-16T17:03:23.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:23.379+0000 7f69ef400640 1 -- 172.21.15.45:0/656004658 learned_addr learned my addr 172.21.15.45:0/656004658 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:03:23.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:23.380+0000 7f69ef400640 1 -- 172.21.15.45:0/656004658 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f69f00ffb40 msgr2=0x7f69f0198260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:23.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:23.380+0000 7f69ef400640 1 --2- 172.21.15.45:0/656004658 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f69f00ffb40 0x7f69f0198260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:23.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:23.380+0000 7f69ef400640 1 -- 172.21.15.45:0/656004658 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f69f0100510 msgr2=0x7f69f01987a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:23.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:23.380+0000 7f69ef400640 1 --2- 172.21.15.45:0/656004658 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f69f0100510 0x7f69f01987a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:23.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:23.380+0000 7f69ef400640 1 -- 172.21.15.45:0/656004658 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69f019fe80 con 0x7f69f0107c50 2024-04-16T17:03:23.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:23.380+0000 7f69eea00640 1 --2- 172.21.15.45:0/656004658 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f69f00ffb40 0x7f69f0198260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:03:23.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:23.380+0000 7f69ef400640 1 --2- 172.21.15.45:0/656004658 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f69f0107c50 0x7f69f019f810 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f69e800e9d0 tx=0x7f69e800ee90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:03:23.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:23.380+0000 7f69f4a00640 1 -- 172.21.15.45:0/656004658 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69e800cd80 con 0x7f69f0107c50 2024-04-16T17:03:23.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:23.380+0000 7f69f4a00640 1 -- 172.21.15.45:0/656004658 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f69e8005020 con 0x7f69f0107c50 2024-04-16T17:03:23.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:23.380+0000 7f69efe00640 1 -- 172.21.15.45:0/656004658 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69f01a0170 con 0x7f69f0107c50 2024-04-16T17:03:23.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:23.381+0000 7f69f4a00640 1 -- 172.21.15.45:0/656004658 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69e80116d0 con 0x7f69f0107c50 2024-04-16T17:03:23.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:23.381+0000 7f69efe00640 1 -- 172.21.15.45:0/656004658 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69f01a06e0 con 0x7f69f0107c50 2024-04-16T17:03:23.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:23.382+0000 7f69efe00640 1 -- 172.21.15.45:0/656004658 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69f0106520 con 0x7f69f0107c50 2024-04-16T17:03:23.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:23.384+0000 7f69f4a00640 1 -- 172.21.15.45:0/656004658 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f69e80051d0 con 0x7f69f0107c50 2024-04-16T17:03:23.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:23.385+0000 7f69f4a00640 1 --2- 172.21.15.45:0/656004658 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f69d40a3c30 0x7f69d40a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:23.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:23.385+0000 7f69f4a00640 1 -- 172.21.15.45:0/656004658 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7497+0+0 (secure 0 0 0) 0x7f69e80d09a0 con 0x7f69f0107c50 2024-04-16T17:03:23.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:23.385+0000 7f69eea00640 1 --2- 172.21.15.45:0/656004658 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f69d40a3c30 0x7f69d40a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:23.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:23.386+0000 7f69eea00640 1 --2- 172.21.15.45:0/656004658 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f69d40a3c30 0x7f69d40a60f0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f69e0005d00 tx=0x7f69e000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:03:23.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:23.388+0000 7f69f4a00640 1 -- 172.21.15.45:0/656004658 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f69e8016030 con 0x7f69f0107c50 2024-04-16T17:03:23.428 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:03:23 smithi104 ceph-mon[36457]: pgmap v313: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T17:03:23.428 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:03:23 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T17:03:23.428 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:03:23 smithi104 ceph-mon[36457]: from='client.? 172.21.15.45:0/250618073' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-16T17:03:23.428 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:03:23 smithi104 ceph-mon[36457]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-16T17:03:23.478 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:03:23 smithi045 ceph-mon[31676]: pgmap v313: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T17:03:23.478 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:03:23 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T17:03:23.479 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:03:23 smithi045 ceph-mon[31676]: from='client.? 172.21.15.45:0/250618073' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-16T17:03:23.479 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:03:23 smithi045 ceph-mon[31676]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-16T17:03:23.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:23.602+0000 7f69efe00640 1 -- 172.21.15.45:0/656004658 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) -- 0x7f69f002d620 con 0x7f69f0107c50 2024-04-16T17:03:24.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:24.143+0000 7f69f4a00640 1 -- 172.21.15.45:0/656004658 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v84) ==== 171+0+0 (secure 0 0 0) 0x7f69e809d040 con 0x7f69f0107c50 2024-04-16T17:03:24.144 INFO:teuthology.orchestra.run.smithi045.stderr:pool 'cephfs_data' created 2024-04-16T17:03:24.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:24.152+0000 7f69efe00640 1 -- 172.21.15.45:0/656004658 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f69d40a3c30 msgr2=0x7f69d40a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:24.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:24.152+0000 7f69efe00640 1 --2- 172.21.15.45:0/656004658 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f69d40a3c30 0x7f69d40a60f0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f69e0005d00 tx=0x7f69e000b410 comp rx=0 tx=0).stop 2024-04-16T17:03:24.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:24.152+0000 7f69efe00640 1 -- 172.21.15.45:0/656004658 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f69f0107c50 msgr2=0x7f69f019f810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:24.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:24.152+0000 7f69efe00640 1 --2- 172.21.15.45:0/656004658 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f69f0107c50 0x7f69f019f810 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f69e800e9d0 tx=0x7f69e800ee90 comp rx=0 tx=0).stop 2024-04-16T17:03:24.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:24.154+0000 7f69efe00640 1 -- 172.21.15.45:0/656004658 shutdown_connections 2024-04-16T17:03:24.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:24.154+0000 7f69efe00640 1 --2- 172.21.15.45:0/656004658 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f69d40a3c30 0x7f69d40a60f0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:24.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:24.154+0000 7f69efe00640 1 --2- 172.21.15.45:0/656004658 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f69f0107c50 0x7f69f019f810 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:24.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:24.154+0000 7f69efe00640 1 --2- 172.21.15.45:0/656004658 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f69f0100510 0x7f69f01987a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:24.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:24.154+0000 7f69efe00640 1 --2- 172.21.15.45:0/656004658 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f69f00ffb40 0x7f69f0198260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:24.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:24.154+0000 7f69efe00640 1 -- 172.21.15.45:0/656004658 >> 172.21.15.45:0/656004658 conn(0x7f69f00fb6c0 msgr2=0x7f69f01036f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:03:24.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:24.155+0000 7f69efe00640 1 -- 172.21.15.45:0/656004658 shutdown_connections 2024-04-16T17:03:24.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:24.158+0000 7f69efe00640 1 -- 172.21.15.45:0/656004658 wait complete. 2024-04-16T17:03:24.190 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-16T17:03:24.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:24.391+0000 7fbfe1800640 1 -- 172.21.15.45:0/4177246793 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbfdc103240 msgr2=0x7fbfdc1036c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:24.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:24.391+0000 7fbfe1800640 1 --2- 172.21.15.45:0/4177246793 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbfdc103240 0x7fbfdc1036c0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fbfc4009900 tx=0x7fbfc402bc30 comp rx=0 tx=0).stop 2024-04-16T17:03:24.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:24.391+0000 7fbfe1800640 1 -- 172.21.15.45:0/4177246793 shutdown_connections 2024-04-16T17:03:24.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:24.391+0000 7fbfe1800640 1 --2- 172.21.15.45:0/4177246793 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbfdc103240 0x7fbfdc1036c0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:24.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:24.391+0000 7fbfe1800640 1 --2- 172.21.15.45:0/4177246793 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbfdc103c00 0x7fbfdc10a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:24.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:24.391+0000 7fbfe1800640 1 --2- 172.21.15.45:0/4177246793 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbfdc102050 0x7fbfdc102470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:24.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:24.391+0000 7fbfe1800640 1 -- 172.21.15.45:0/4177246793 >> 172.21.15.45:0/4177246793 conn(0x7fbfdc0fd5e0 msgr2=0x7fbfdc0ffa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:03:24.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:24.391+0000 7fbfe1800640 1 -- 172.21.15.45:0/4177246793 shutdown_connections 2024-04-16T17:03:24.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:24.391+0000 7fbfe1800640 1 -- 172.21.15.45:0/4177246793 wait complete. 2024-04-16T17:03:24.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:24.392+0000 7fbfe1800640 1 Processor -- start 2024-04-16T17:03:24.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:24.392+0000 7fbfe1800640 1 -- start start 2024-04-16T17:03:24.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:24.392+0000 7fbfe1800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbfdc102050 0x7fbfdc075db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:24.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:24.392+0000 7fbfe1800640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbfdc103240 0x7fbfdc0743f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:24.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:24.392+0000 7fbfe1800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbfdc103c00 0x7fbfdc074930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:24.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:24.392+0000 7fbfe1800640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fbfdc10dce0 con 0x7fbfdc102050 2024-04-16T17:03:24.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:24.392+0000 7fbfe1800640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fbfdc10db60 con 0x7fbfdc103240 2024-04-16T17:03:24.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:24.392+0000 7fbfe1800640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fbfdc10de60 con 0x7fbfdc103c00 2024-04-16T17:03:24.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:24.393+0000 7fbfdaa00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbfdc102050 0x7fbfdc075db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:24.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:24.393+0000 7fbfdaa00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbfdc102050 0x7fbfdc075db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42910/0 (socket says 172.21.15.45:42910) 2024-04-16T17:03:24.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:24.393+0000 7fbfdaa00640 1 -- 172.21.15.45:0/147626882 learned_addr learned my addr 172.21.15.45:0/147626882 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:03:24.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:24.393+0000 7fbfdb400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbfdc103c00 0x7fbfdc074930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:24.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:24.393+0000 7fbfd2000640 1 --2- 172.21.15.45:0/147626882 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbfdc103240 0x7fbfdc0743f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:24.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:24.393+0000 7fbfd2000640 1 -- 172.21.15.45:0/147626882 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbfdc103c00 msgr2=0x7fbfdc074930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:24.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:24.393+0000 7fbfd2000640 1 --2- 172.21.15.45:0/147626882 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbfdc103c00 0x7fbfdc074930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:24.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:24.393+0000 7fbfd2000640 1 -- 172.21.15.45:0/147626882 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbfdc102050 msgr2=0x7fbfdc075db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:24.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:24.393+0000 7fbfd2000640 1 --2- 172.21.15.45:0/147626882 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbfdc102050 0x7fbfdc075db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:24.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:24.393+0000 7fbfd2000640 1 -- 172.21.15.45:0/147626882 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfdc0751c0 con 0x7fbfdc103240 2024-04-16T17:03:24.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:24.393+0000 7fbfd2000640 1 --2- 172.21.15.45:0/147626882 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbfdc103240 0x7fbfdc0743f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fbfc4009230 tx=0x7fbfc402c6c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:03:24.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:24.394+0000 7fbfd3e00640 1 -- 172.21.15.45:0/147626882 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfc4041070 con 0x7fbfdc103240 2024-04-16T17:03:24.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:24.394+0000 7fbfd3e00640 1 -- 172.21.15.45:0/147626882 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbfc4004450 con 0x7fbfdc103240 2024-04-16T17:03:24.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:24.394+0000 7fbfd3e00640 1 -- 172.21.15.45:0/147626882 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfc4034440 con 0x7fbfdc103240 2024-04-16T17:03:24.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:24.394+0000 7fbfe1800640 1 -- 172.21.15.45:0/147626882 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbfdc1a2400 con 0x7fbfdc103240 2024-04-16T17:03:24.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:24.394+0000 7fbfe1800640 1 -- 172.21.15.45:0/147626882 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbfdc1a2900 con 0x7fbfdc103240 2024-04-16T17:03:24.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:24.395+0000 7fbfe1800640 1 -- 172.21.15.45:0/147626882 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbfdc109440 con 0x7fbfdc103240 2024-04-16T17:03:24.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:24.397+0000 7fbfd3e00640 1 -- 172.21.15.45:0/147626882 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fbfc4044070 con 0x7fbfdc103240 2024-04-16T17:03:24.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:24.398+0000 7fbfd3e00640 1 --2- 172.21.15.45:0/147626882 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fbfc00a3bb0 0x7fbfc00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:24.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:24.398+0000 7fbfd3e00640 1 -- 172.21.15.45:0/147626882 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7879+0+0 (secure 0 0 0) 0x7fbfc40f1e10 con 0x7fbfdc103240 2024-04-16T17:03:24.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:24.398+0000 7fbfdaa00640 1 --2- 172.21.15.45:0/147626882 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fbfc00a3bb0 0x7fbfc00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:24.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:24.399+0000 7fbfdaa00640 1 --2- 172.21.15.45:0/147626882 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fbfc00a3bb0 0x7fbfc00a6070 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fbfcc005f40 tx=0x7fbfcc009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:03:24.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:24.402+0000 7fbfd3e00640 1 -- 172.21.15.45:0/147626882 <== mon.2 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fbfc40f5030 con 0x7fbfdc103240 2024-04-16T17:03:24.429 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:03:24 smithi104 ceph-mon[36457]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-16T17:03:24.430 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:03:24 smithi104 ceph-mon[36457]: osdmap e83: 12 total, 12 up, 12 in 2024-04-16T17:03:24.430 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:03:24 smithi104 ceph-mon[36457]: from='client.? 172.21.15.45:0/656004658' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-04-16T17:03:24.430 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:03:24 smithi104 ceph-mon[36457]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-04-16T17:03:24.430 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:03:24 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:03:24.430 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:03:24 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:03:24.430 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:03:24 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:03:24.430 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:03:24 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:03:24.478 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:03:24 smithi045 ceph-mon[31676]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-16T17:03:24.478 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:03:24 smithi045 ceph-mon[31676]: osdmap e83: 12 total, 12 up, 12 in 2024-04-16T17:03:24.478 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:03:24 smithi045 ceph-mon[31676]: from='client.? 172.21.15.45:0/656004658' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-04-16T17:03:24.478 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:03:24 smithi045 ceph-mon[31676]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-04-16T17:03:24.478 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:03:24 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:03:24.478 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:03:24 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:03:24.478 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:03:24 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:03:24.478 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:03:24 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:03:24.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:03:24 smithi117 ceph-mon[36730]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-16T17:03:24.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:03:24 smithi117 ceph-mon[36730]: osdmap e83: 12 total, 12 up, 12 in 2024-04-16T17:03:24.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:03:24 smithi117 ceph-mon[36730]: from='client.? 172.21.15.45:0/656004658' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-04-16T17:03:24.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:03:24 smithi117 ceph-mon[36730]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-04-16T17:03:24.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:03:24 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:03:24.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:03:24 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:03:24.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:03:24 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:03:24.557 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:03:24 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:03:24.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:24.662+0000 7fbfe1800640 1 -- 172.21.15.45:0/147626882 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fbfdc062c00 con 0x7fbfdc103240 2024-04-16T17:03:25.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:25.166+0000 7fbfd3e00640 1 -- 172.21.15.45:0/147626882 <== mon.2 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7fbfc40b8e20 con 0x7fbfdc103240 2024-04-16T17:03:25.167 INFO:teuthology.orchestra.run.smithi045.stderr:new fs with metadata pool 2 and data pool 3 2024-04-16T17:03:25.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:25.172+0000 7fbfe1800640 1 -- 172.21.15.45:0/147626882 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fbfc00a3bb0 msgr2=0x7fbfc00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:25.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:25.172+0000 7fbfe1800640 1 --2- 172.21.15.45:0/147626882 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fbfc00a3bb0 0x7fbfc00a6070 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fbfcc005f40 tx=0x7fbfcc009500 comp rx=0 tx=0).stop 2024-04-16T17:03:25.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:25.172+0000 7fbfe1800640 1 -- 172.21.15.45:0/147626882 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbfdc103240 msgr2=0x7fbfdc0743f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:25.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:25.172+0000 7fbfe1800640 1 --2- 172.21.15.45:0/147626882 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbfdc103240 0x7fbfdc0743f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fbfc4009230 tx=0x7fbfc402c6c0 comp rx=0 tx=0).stop 2024-04-16T17:03:25.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:25.173+0000 7fbfe1800640 1 -- 172.21.15.45:0/147626882 shutdown_connections 2024-04-16T17:03:25.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:25.173+0000 7fbfe1800640 1 --2- 172.21.15.45:0/147626882 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fbfc00a3bb0 0x7fbfc00a6070 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:25.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:25.173+0000 7fbfe1800640 1 --2- 172.21.15.45:0/147626882 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbfdc103c00 0x7fbfdc074930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:25.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:25.173+0000 7fbfe1800640 1 --2- 172.21.15.45:0/147626882 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbfdc103240 0x7fbfdc0743f0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:25.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:25.173+0000 7fbfe1800640 1 --2- 172.21.15.45:0/147626882 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbfdc102050 0x7fbfdc075db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:25.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:25.173+0000 7fbfe1800640 1 -- 172.21.15.45:0/147626882 >> 172.21.15.45:0/147626882 conn(0x7fbfdc0fd5e0 msgr2=0x7fbfdc106610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:03:25.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:25.175+0000 7fbfe1800640 1 -- 172.21.15.45:0/147626882 shutdown_connections 2024-04-16T17:03:25.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:25.175+0000 7fbfe1800640 1 -- 172.21.15.45:0/147626882 wait complete. 2024-04-16T17:03:25.202 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2024-04-16T17:03:25.202 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2024-04-16T17:03:25.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:25.409+0000 7f9767000640 1 -- 172.21.15.45:0/3580092070 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9760105cf0 msgr2=0x7f976010c580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:25.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:25.409+0000 7f9767000640 1 --2- 172.21.15.45:0/3580092070 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9760105cf0 0x7f976010c580 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f9758009ca0 tx=0x7f975802c0f0 comp rx=0 tx=0).stop 2024-04-16T17:03:25.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:25.409+0000 7f9767000640 1 -- 172.21.15.45:0/3580092070 shutdown_connections 2024-04-16T17:03:25.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:25.409+0000 7f9767000640 1 --2- 172.21.15.45:0/3580092070 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9760105cf0 0x7f976010c580 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:25.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:25.409+0000 7f9767000640 1 --2- 172.21.15.45:0/3580092070 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9760105330 0x7f97601057b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:25.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:25.409+0000 7f9767000640 1 --2- 172.21.15.45:0/3580092070 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9760104140 0x7f9760104560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:25.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:25.409+0000 7f9767000640 1 -- 172.21.15.45:0/3580092070 >> 172.21.15.45:0/3580092070 conn(0x7f97600ff6b0 msgr2=0x7f9760101af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:03:25.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:25.409+0000 7f9767000640 1 -- 172.21.15.45:0/3580092070 shutdown_connections 2024-04-16T17:03:25.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:25.409+0000 7f9767000640 1 -- 172.21.15.45:0/3580092070 wait complete. 2024-04-16T17:03:25.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:25.410+0000 7f9767000640 1 Processor -- start 2024-04-16T17:03:25.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:25.410+0000 7f9767000640 1 -- start start 2024-04-16T17:03:25.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:25.411+0000 7f9767000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9760104140 0x7f976019a380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:25.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:25.411+0000 7f9767000640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9760105330 0x7f976019a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:25.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:25.411+0000 7f9767000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9760105cf0 0x7f97601a1930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:25.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:25.411+0000 7f9767000640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f976010fda0 con 0x7f9760104140 2024-04-16T17:03:25.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:25.411+0000 7f9767000640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f976010fc20 con 0x7f9760105330 2024-04-16T17:03:25.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:25.411+0000 7f9767000640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f976010ff20 con 0x7f9760105cf0 2024-04-16T17:03:25.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:25.411+0000 7f9765c00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9760104140 0x7f976019a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:25.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:25.411+0000 7f9766600640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9760105cf0 0x7f97601a1930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:25.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:25.411+0000 7f9765c00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9760104140 0x7f976019a380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42922/0 (socket says 172.21.15.45:42922) 2024-04-16T17:03:25.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:25.411+0000 7f9765c00640 1 -- 172.21.15.45:0/1516453355 learned_addr learned my addr 172.21.15.45:0/1516453355 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:03:25.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:25.411+0000 7f9765200640 1 --2- 172.21.15.45:0/1516453355 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9760105330 0x7f976019a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:25.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:25.411+0000 7f9766600640 1 -- 172.21.15.45:0/1516453355 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9760105330 msgr2=0x7f976019a8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:25.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:25.411+0000 7f9766600640 1 --2- 172.21.15.45:0/1516453355 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9760105330 0x7f976019a8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:25.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:25.411+0000 7f9766600640 1 -- 172.21.15.45:0/1516453355 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9760104140 msgr2=0x7f976019a380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:25.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:25.411+0000 7f9766600640 1 --2- 172.21.15.45:0/1516453355 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9760104140 0x7f976019a380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:25.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:25.411+0000 7f9766600640 1 -- 172.21.15.45:0/1516453355 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97601a1fa0 con 0x7f9760105cf0 2024-04-16T17:03:25.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:25.412+0000 7f9766600640 1 --2- 172.21.15.45:0/1516453355 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9760105cf0 0x7f97601a1930 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f9758037040 tx=0x7f9758032ea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:03:25.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:25.412+0000 7f975e000640 1 -- 172.21.15.45:0/1516453355 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9758041020 con 0x7f9760105cf0 2024-04-16T17:03:25.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:25.412+0000 7f975e000640 1 -- 172.21.15.45:0/1516453355 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9758034cd0 con 0x7f9760105cf0 2024-04-16T17:03:25.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:25.412+0000 7f975e000640 1 -- 172.21.15.45:0/1516453355 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9758039040 con 0x7f9760105cf0 2024-04-16T17:03:25.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:25.412+0000 7f9767000640 1 -- 172.21.15.45:0/1516453355 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97601a2230 con 0x7f9760105cf0 2024-04-16T17:03:25.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:25.412+0000 7f9767000640 1 -- 172.21.15.45:0/1516453355 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97601a26a0 con 0x7f9760105cf0 2024-04-16T17:03:25.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:25.414+0000 7f9767000640 1 -- 172.21.15.45:0/1516453355 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f976010b460 con 0x7f9760105cf0 2024-04-16T17:03:25.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:25.417+0000 7f975e000640 1 -- 172.21.15.45:0/1516453355 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f9758044020 con 0x7f9760105cf0 2024-04-16T17:03:25.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:25.418+0000 7f975e000640 1 --2- 172.21.15.45:0/1516453355 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f97480a3bb0 0x7f97480a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:25.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:25.418+0000 7f975e000640 1 -- 172.21.15.45:0/1516453355 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f97580f20e0 con 0x7f9760105cf0 2024-04-16T17:03:25.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:25.419+0000 7f9765c00640 1 --2- 172.21.15.45:0/1516453355 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f97480a3bb0 0x7f97480a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:25.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:25.419+0000 7f9765c00640 1 --2- 172.21.15.45:0/1516453355 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f97480a3bb0 0x7f97480a6070 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f97540058c0 tx=0x7f9754005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:03:25.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:25.423+0000 7f975e000640 1 -- 172.21.15.45:0/1516453355 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f975804b020 con 0x7f9760105cf0 2024-04-16T17:03:25.478 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:03:25 smithi045 ceph-mon[31676]: pgmap v315: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T17:03:25.478 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:03:25 smithi045 ceph-mon[31676]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-04-16T17:03:25.478 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:03:25 smithi045 ceph-mon[31676]: osdmap e84: 12 total, 12 up, 12 in 2024-04-16T17:03:25.478 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:03:25 smithi045 ceph-mon[31676]: from='client.? 172.21.15.45:0/147626882' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-16T17:03:25.479 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:03:25 smithi045 ceph-mon[31676]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-16T17:03:25.479 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:03:25 smithi045 ceph-mon[31676]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-04-16T17:03:25.479 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:03:25 smithi045 ceph-mon[31676]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-16T17:03:25.479 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:03:25 smithi045 ceph-mon[31676]: osdmap e85: 12 total, 12 up, 12 in 2024-04-16T17:03:25.479 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:03:25 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:03:25.479 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:03:25 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:03:25.479 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:03:25 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:03:25.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:03:25 smithi117 ceph-mon[36730]: pgmap v315: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T17:03:25.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:03:25 smithi117 ceph-mon[36730]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-04-16T17:03:25.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:03:25 smithi117 ceph-mon[36730]: osdmap e84: 12 total, 12 up, 12 in 2024-04-16T17:03:25.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:03:25 smithi117 ceph-mon[36730]: from='client.? 172.21.15.45:0/147626882' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-16T17:03:25.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:03:25 smithi117 ceph-mon[36730]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-16T17:03:25.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:03:25 smithi117 ceph-mon[36730]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-04-16T17:03:25.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:03:25 smithi117 ceph-mon[36730]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-16T17:03:25.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:03:25 smithi117 ceph-mon[36730]: osdmap e85: 12 total, 12 up, 12 in 2024-04-16T17:03:25.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:03:25 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:03:25.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:03:25 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:03:25.557 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:03:25 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:03:25.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:25.645+0000 7f9767000640 1 -- 172.21.15.45:0/1516453355 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) -- 0x7f976002d620 con 0x7f9760105cf0 2024-04-16T17:03:25.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:03:25 smithi104 ceph-mon[36457]: pgmap v315: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T17:03:25.679 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:03:25 smithi104 ceph-mon[36457]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-04-16T17:03:25.679 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:03:25 smithi104 ceph-mon[36457]: osdmap e84: 12 total, 12 up, 12 in 2024-04-16T17:03:25.679 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:03:25 smithi104 ceph-mon[36457]: from='client.? 172.21.15.45:0/147626882' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-16T17:03:25.679 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:03:25 smithi104 ceph-mon[36457]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-16T17:03:25.679 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:03:25 smithi104 ceph-mon[36457]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-04-16T17:03:25.679 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:03:25 smithi104 ceph-mon[36457]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-16T17:03:25.679 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:03:25 smithi104 ceph-mon[36457]: osdmap e85: 12 total, 12 up, 12 in 2024-04-16T17:03:25.679 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:03:25 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:03:25.679 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:03:25 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:03:25.679 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:03:25 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:03:25.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:25.954+0000 7f975e000640 1 -- 172.21.15.45:0/1516453355 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v86) ==== 157+0+0 (secure 0 0 0) 0x7f97580bec00 con 0x7f9760105cf0 2024-04-16T17:03:25.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:25.958+0000 7f9767000640 1 -- 172.21.15.45:0/1516453355 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f97480a3bb0 msgr2=0x7f97480a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:25.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:25.958+0000 7f9767000640 1 --2- 172.21.15.45:0/1516453355 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f97480a3bb0 0x7f97480a6070 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f97540058c0 tx=0x7f9754005f90 comp rx=0 tx=0).stop 2024-04-16T17:03:25.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:25.958+0000 7f9767000640 1 -- 172.21.15.45:0/1516453355 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9760105cf0 msgr2=0x7f97601a1930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:25.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:25.958+0000 7f9767000640 1 --2- 172.21.15.45:0/1516453355 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9760105cf0 0x7f97601a1930 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f9758037040 tx=0x7f9758032ea0 comp rx=0 tx=0).stop 2024-04-16T17:03:25.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:25.958+0000 7f9767000640 1 -- 172.21.15.45:0/1516453355 shutdown_connections 2024-04-16T17:03:25.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:25.958+0000 7f9767000640 1 --2- 172.21.15.45:0/1516453355 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f97480a3bb0 0x7f97480a6070 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:25.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:25.958+0000 7f9767000640 1 --2- 172.21.15.45:0/1516453355 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9760105cf0 0x7f97601a1930 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:25.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:25.958+0000 7f9767000640 1 --2- 172.21.15.45:0/1516453355 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9760105330 0x7f976019a8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:25.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:25.958+0000 7f9767000640 1 --2- 172.21.15.45:0/1516453355 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9760104140 0x7f976019a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:25.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:25.958+0000 7f9767000640 1 -- 172.21.15.45:0/1516453355 >> 172.21.15.45:0/1516453355 conn(0x7f97600ff6b0 msgr2=0x7f9760108630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:03:25.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:25.958+0000 7f9767000640 1 -- 172.21.15.45:0/1516453355 shutdown_connections 2024-04-16T17:03:25.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:25.959+0000 7f9767000640 1 -- 172.21.15.45:0/1516453355 wait complete. 2024-04-16T17:03:25.986 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2024-04-16T17:03:26.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:26.205+0000 7f2bc4c00640 1 -- 172.21.15.45:0/2209703832 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2bc0105450 msgr2=0x7f2bc01058d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:26.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:26.205+0000 7f2bc4c00640 1 --2- 172.21.15.45:0/2209703832 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2bc0105450 0x7f2bc01058d0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f2ba8009860 tx=0x7f2ba802bb70 comp rx=0 tx=0).stop 2024-04-16T17:03:26.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:26.205+0000 7f2bc4c00640 1 -- 172.21.15.45:0/2209703832 shutdown_connections 2024-04-16T17:03:26.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:26.205+0000 7f2bc4c00640 1 --2- 172.21.15.45:0/2209703832 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2bc0105e10 0x7f2bc010c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:26.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:26.205+0000 7f2bc4c00640 1 --2- 172.21.15.45:0/2209703832 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2bc0105450 0x7f2bc01058d0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:26.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:26.205+0000 7f2bc4c00640 1 --2- 172.21.15.45:0/2209703832 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2bc0104260 0x7f2bc0104680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:26.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:26.205+0000 7f2bc4c00640 1 -- 172.21.15.45:0/2209703832 >> 172.21.15.45:0/2209703832 conn(0x7f2bc00ff7f0 msgr2=0x7f2bc0101c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:03:26.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:26.206+0000 7f2bc4c00640 1 -- 172.21.15.45:0/2209703832 shutdown_connections 2024-04-16T17:03:26.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:26.206+0000 7f2bc4c00640 1 -- 172.21.15.45:0/2209703832 wait complete. 2024-04-16T17:03:26.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:26.207+0000 7f2bc4c00640 1 Processor -- start 2024-04-16T17:03:26.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:26.207+0000 7f2bc4c00640 1 -- start start 2024-04-16T17:03:26.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:26.207+0000 7f2bc4c00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2bc0104260 0x7f2bc019a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:26.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:26.207+0000 7f2bc4c00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2bc0105450 0x7f2bc019aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:26.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:26.207+0000 7f2bc4c00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2bc0105e10 0x7f2bc01a1af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:26.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:26.207+0000 7f2bc4c00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f2bc010fef0 con 0x7f2bc0105e10 2024-04-16T17:03:26.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:26.207+0000 7f2bc4c00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f2bc010fd70 con 0x7f2bc0105450 2024-04-16T17:03:26.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:26.207+0000 7f2bc4c00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f2bc0110070 con 0x7f2bc0104260 2024-04-16T17:03:26.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:26.208+0000 7f2bbde00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2bc0104260 0x7f2bc019a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:26.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:26.208+0000 7f2bbe800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2bc0105e10 0x7f2bc01a1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:26.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:26.208+0000 7f2bbd400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2bc0105450 0x7f2bc019aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:26.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:26.208+0000 7f2bbde00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2bc0104260 0x7f2bc019a540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:40224/0 (socket says 172.21.15.45:40224) 2024-04-16T17:03:26.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:26.208+0000 7f2bbde00640 1 -- 172.21.15.45:0/1336381293 learned_addr learned my addr 172.21.15.45:0/1336381293 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:03:26.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:26.208+0000 7f2bbde00640 1 -- 172.21.15.45:0/1336381293 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2bc0105450 msgr2=0x7f2bc019aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:26.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:26.208+0000 7f2bbde00640 1 --2- 172.21.15.45:0/1336381293 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2bc0105450 0x7f2bc019aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:26.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:26.208+0000 7f2bbde00640 1 -- 172.21.15.45:0/1336381293 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2bc0105e10 msgr2=0x7f2bc01a1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:26.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:26.208+0000 7f2bbde00640 1 --2- 172.21.15.45:0/1336381293 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2bc0105e10 0x7f2bc01a1af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:26.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:26.208+0000 7f2bbde00640 1 -- 172.21.15.45:0/1336381293 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2bc01a2160 con 0x7f2bc0104260 2024-04-16T17:03:26.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:26.208+0000 7f2bbd400640 1 --2- 172.21.15.45:0/1336381293 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2bc0105450 0x7f2bc019aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:03:26.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:26.208+0000 7f2bbe800640 1 --2- 172.21.15.45:0/1336381293 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2bc0105e10 0x7f2bc01a1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:03:26.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:26.208+0000 7f2bbde00640 1 --2- 172.21.15.45:0/1336381293 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2bc0104260 0x7f2bc019a540 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f2bb000e9a0 tx=0x7f2bb000ee60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:03:26.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:26.209+0000 7f2bb6a00640 1 -- 172.21.15.45:0/1336381293 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bb000cd80 con 0x7f2bc0104260 2024-04-16T17:03:26.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:26.209+0000 7f2bc4c00640 1 -- 172.21.15.45:0/1336381293 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2bc01a2450 con 0x7f2bc0104260 2024-04-16T17:03:26.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:26.209+0000 7f2bb6a00640 1 -- 172.21.15.45:0/1336381293 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2bb0010910 con 0x7f2bc0104260 2024-04-16T17:03:26.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:26.209+0000 7f2bb6a00640 1 -- 172.21.15.45:0/1336381293 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bb0018a30 con 0x7f2bc0104260 2024-04-16T17:03:26.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:26.209+0000 7f2bc4c00640 1 -- 172.21.15.45:0/1336381293 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2bc01a29c0 con 0x7f2bc0104260 2024-04-16T17:03:26.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:26.210+0000 7f2bc4c00640 1 -- 172.21.15.45:0/1336381293 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2bc010b580 con 0x7f2bc0104260 2024-04-16T17:03:26.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:26.212+0000 7f2bb6a00640 1 -- 172.21.15.45:0/1336381293 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f2bb0010ac0 con 0x7f2bc0104260 2024-04-16T17:03:26.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:26.213+0000 7f2bb6a00640 1 --2- 172.21.15.45:0/1336381293 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f2ba40a3c30 0x7f2ba40a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:26.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:26.214+0000 7f2bb6a00640 1 -- 172.21.15.45:0/1336381293 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8277+0+0 (secure 0 0 0) 0x7f2bb00d07a0 con 0x7f2bc0104260 2024-04-16T17:03:26.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:26.214+0000 7f2bbd400640 1 --2- 172.21.15.45:0/1336381293 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f2ba40a3c30 0x7f2ba40a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:26.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:26.215+0000 7f2bbd400640 1 --2- 172.21.15.45:0/1336381293 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f2ba40a3c30 0x7f2ba40a60f0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f2ba8009830 tx=0x7f2ba802e040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:03:26.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:26.218+0000 7f2bb6a00640 1 -- 172.21.15.45:0/1336381293 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2bb0053ff0 con 0x7f2bc0104260 2024-04-16T17:03:26.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:26.416+0000 7f2bc4c00640 1 -- 172.21.15.45:0/1336381293 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f2bc002d620 con 0x7f2bc0104260 2024-04-16T17:03:26.478 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:03:26 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T17:03:26.478 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:03:26 smithi045 ceph-mon[31676]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-16T17:03:26.478 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:03:26 smithi045 ceph-mon[31676]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-04-16T17:03:26.479 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:03:26 smithi045 ceph-mon[31676]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-04-16T17:03:26.479 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:03:26 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:03:26.479 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:03:26 smithi045 ceph-mon[31676]: from='client.? 172.21.15.45:0/1516453355' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-04-16T17:03:26.479 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:03:26 smithi045 ceph-mon[31676]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-04-16T17:03:26.479 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:03:26 smithi045 ceph-mon[31676]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-04-16T17:03:26.479 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:03:26 smithi045 ceph-mon[31676]: osdmap e86: 12 total, 12 up, 12 in 2024-04-16T17:03:26.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:03:26 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T17:03:26.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:03:26 smithi117 ceph-mon[36730]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-16T17:03:26.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:03:26 smithi117 ceph-mon[36730]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-04-16T17:03:26.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:03:26 smithi117 ceph-mon[36730]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-04-16T17:03:26.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:03:26 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:03:26.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:03:26 smithi117 ceph-mon[36730]: from='client.? 172.21.15.45:0/1516453355' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-04-16T17:03:26.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:03:26 smithi117 ceph-mon[36730]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-04-16T17:03:26.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:03:26 smithi117 ceph-mon[36730]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-04-16T17:03:26.557 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:03:26 smithi117 ceph-mon[36730]: osdmap e86: 12 total, 12 up, 12 in 2024-04-16T17:03:26.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:03:26 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T17:03:26.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:03:26 smithi104 ceph-mon[36457]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-16T17:03:26.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:03:26 smithi104 ceph-mon[36457]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-04-16T17:03:26.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:03:26 smithi104 ceph-mon[36457]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-04-16T17:03:26.679 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:03:26 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:03:26.679 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:03:26 smithi104 ceph-mon[36457]: from='client.? 172.21.15.45:0/1516453355' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-04-16T17:03:26.679 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:03:26 smithi104 ceph-mon[36457]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-04-16T17:03:26.679 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:03:26 smithi104 ceph-mon[36457]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-04-16T17:03:26.679 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:03:26 smithi104 ceph-mon[36457]: osdmap e86: 12 total, 12 up, 12 in 2024-04-16T17:03:27.479 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:03:27 smithi045 ceph-mon[31676]: pgmap v319: 97 pgs: 4 creating+peering, 47 active+clean, 46 unknown; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 1 op/s 2024-04-16T17:03:27.479 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:03:27 smithi045 ceph-mon[31676]: mds.? [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] up:active 2024-04-16T17:03:27.479 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:03:27 smithi045 ceph-mon[31676]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-16T17:03:27.479 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:03:27 smithi045 ceph-mon[31676]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-16T17:03:27.479 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:03:27 smithi045 ceph-mon[31676]: from='client.? 172.21.15.45:0/1336381293' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-16T17:03:27.479 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:03:27 smithi045 ceph-mon[31676]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-16T17:03:27.479 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:03:27 smithi045 ceph-mon[31676]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-16T17:03:27.479 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:03:27 smithi045 ceph-mon[31676]: Cluster is now healthy 2024-04-16T17:03:27.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:03:27 smithi117 ceph-mon[36730]: pgmap v319: 97 pgs: 4 creating+peering, 47 active+clean, 46 unknown; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 1 op/s 2024-04-16T17:03:27.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:03:27 smithi117 ceph-mon[36730]: mds.? [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] up:active 2024-04-16T17:03:27.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:03:27 smithi117 ceph-mon[36730]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-16T17:03:27.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:03:27 smithi117 ceph-mon[36730]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-16T17:03:27.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:03:27 smithi117 ceph-mon[36730]: from='client.? 172.21.15.45:0/1336381293' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-16T17:03:27.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:03:27 smithi117 ceph-mon[36730]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-16T17:03:27.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:03:27 smithi117 ceph-mon[36730]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-16T17:03:27.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:03:27 smithi117 ceph-mon[36730]: Cluster is now healthy 2024-04-16T17:03:27.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:03:27 smithi104 ceph-mon[36457]: pgmap v319: 97 pgs: 4 creating+peering, 47 active+clean, 46 unknown; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 1 op/s 2024-04-16T17:03:27.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:03:27 smithi104 ceph-mon[36457]: mds.? [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] up:active 2024-04-16T17:03:27.679 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:03:27 smithi104 ceph-mon[36457]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-16T17:03:27.679 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:03:27 smithi104 ceph-mon[36457]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-16T17:03:27.679 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:03:27 smithi104 ceph-mon[36457]: from='client.? 172.21.15.45:0/1336381293' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-16T17:03:27.679 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:03:27 smithi104 ceph-mon[36457]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-16T17:03:27.679 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:03:27 smithi104 ceph-mon[36457]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-16T17:03:27.679 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:03:27 smithi104 ceph-mon[36457]: Cluster is now healthy 2024-04-16T17:03:28.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:28.221+0000 7f2bb6a00640 1 -- 172.21.15.45:0/1336381293 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v88) ==== 228+0+0 (secure 0 0 0) 0x7f2bb009d0f0 con 0x7f2bc0104260 2024-04-16T17:03:28.223 INFO:teuthology.orchestra.run.smithi045.stderr:pool 'cephfs_data_ec' created 2024-04-16T17:03:28.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:28.228+0000 7f2bc4c00640 1 -- 172.21.15.45:0/1336381293 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f2ba40a3c30 msgr2=0x7f2ba40a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:28.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:28.228+0000 7f2bc4c00640 1 --2- 172.21.15.45:0/1336381293 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f2ba40a3c30 0x7f2ba40a60f0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f2ba8009830 tx=0x7f2ba802e040 comp rx=0 tx=0).stop 2024-04-16T17:03:28.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:28.228+0000 7f2bc4c00640 1 -- 172.21.15.45:0/1336381293 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2bc0104260 msgr2=0x7f2bc019a540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:28.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:28.228+0000 7f2bc4c00640 1 --2- 172.21.15.45:0/1336381293 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2bc0104260 0x7f2bc019a540 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f2bb000e9a0 tx=0x7f2bb000ee60 comp rx=0 tx=0).stop 2024-04-16T17:03:28.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:28.229+0000 7f2bc4c00640 1 -- 172.21.15.45:0/1336381293 shutdown_connections 2024-04-16T17:03:28.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:28.229+0000 7f2bc4c00640 1 --2- 172.21.15.45:0/1336381293 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f2ba40a3c30 0x7f2ba40a60f0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:28.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:28.229+0000 7f2bc4c00640 1 --2- 172.21.15.45:0/1336381293 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2bc0105e10 0x7f2bc01a1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:28.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:28.229+0000 7f2bc4c00640 1 --2- 172.21.15.45:0/1336381293 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2bc0105450 0x7f2bc019aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:28.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:28.229+0000 7f2bc4c00640 1 --2- 172.21.15.45:0/1336381293 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2bc0104260 0x7f2bc019a540 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:28.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:28.229+0000 7f2bc4c00640 1 -- 172.21.15.45:0/1336381293 >> 172.21.15.45:0/1336381293 conn(0x7f2bc00ff7f0 msgr2=0x7f2bc0108750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:03:28.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:28.230+0000 7f2bc4c00640 1 -- 172.21.15.45:0/1336381293 shutdown_connections 2024-04-16T17:03:28.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:28.231+0000 7f2bc4c00640 1 -- 172.21.15.45:0/1336381293 wait complete. 2024-04-16T17:03:28.257 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2024-04-16T17:03:28.464 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:28.463+0000 7fe571e00640 1 -- 172.21.15.45:0/2359962180 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe56c105cf0 msgr2=0x7fe56c10c580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:28.464 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:28.463+0000 7fe571e00640 1 --2- 172.21.15.45:0/2359962180 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe56c105cf0 0x7fe56c10c580 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fe564009ca0 tx=0x7fe56402c100 comp rx=0 tx=0).stop 2024-04-16T17:03:28.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:28.463+0000 7fe571e00640 1 -- 172.21.15.45:0/2359962180 shutdown_connections 2024-04-16T17:03:28.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:28.463+0000 7fe571e00640 1 --2- 172.21.15.45:0/2359962180 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe56c105330 0x7fe56c1057b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:28.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:28.463+0000 7fe571e00640 1 --2- 172.21.15.45:0/2359962180 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe56c105cf0 0x7fe56c10c580 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:28.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:28.463+0000 7fe571e00640 1 --2- 172.21.15.45:0/2359962180 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe56c104140 0x7fe56c104560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:28.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:28.463+0000 7fe571e00640 1 -- 172.21.15.45:0/2359962180 >> 172.21.15.45:0/2359962180 conn(0x7fe56c0ff6b0 msgr2=0x7fe56c101af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:03:28.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:28.463+0000 7fe571e00640 1 -- 172.21.15.45:0/2359962180 shutdown_connections 2024-04-16T17:03:28.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:28.463+0000 7fe571e00640 1 -- 172.21.15.45:0/2359962180 wait complete. 2024-04-16T17:03:28.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:28.465+0000 7fe571e00640 1 Processor -- start 2024-04-16T17:03:28.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:28.465+0000 7fe571e00640 1 -- start start 2024-04-16T17:03:28.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:28.465+0000 7fe571e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe56c104140 0x7fe56c19a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:28.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:28.465+0000 7fe571e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe56c105330 0x7fe56c19a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:28.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:28.465+0000 7fe571e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe56c105cf0 0x7fe56c1a19b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:28.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:28.465+0000 7fe571e00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fe56c10fdd0 con 0x7fe56c105330 2024-04-16T17:03:28.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:28.465+0000 7fe571e00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fe56c10fc50 con 0x7fe56c104140 2024-04-16T17:03:28.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:28.465+0000 7fe571e00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fe56c10ff50 con 0x7fe56c105cf0 2024-04-16T17:03:28.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:28.466+0000 7fe570a00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe56c104140 0x7fe56c19a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:28.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:28.466+0000 7fe570a00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe56c104140 0x7fe56c19a400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:52090/0 (socket says 172.21.15.45:52090) 2024-04-16T17:03:28.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:28.466+0000 7fe570a00640 1 -- 172.21.15.45:0/209928873 learned_addr learned my addr 172.21.15.45:0/209928873 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:03:28.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:28.466+0000 7fe56be00640 1 --2- 172.21.15.45:0/209928873 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe56c105330 0x7fe56c19a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:28.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:28.466+0000 7fe571400640 1 --2- 172.21.15.45:0/209928873 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe56c105cf0 0x7fe56c1a19b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:28.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:28.466+0000 7fe570a00640 1 -- 172.21.15.45:0/209928873 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe56c105cf0 msgr2=0x7fe56c1a19b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:28.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:28.466+0000 7fe570a00640 1 --2- 172.21.15.45:0/209928873 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe56c105cf0 0x7fe56c1a19b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:28.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:28.466+0000 7fe570a00640 1 -- 172.21.15.45:0/209928873 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe56c105330 msgr2=0x7fe56c19a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:28.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:28.466+0000 7fe570a00640 1 --2- 172.21.15.45:0/209928873 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe56c105330 0x7fe56c19a940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:28.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:28.466+0000 7fe570a00640 1 -- 172.21.15.45:0/209928873 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe56c1a2020 con 0x7fe56c104140 2024-04-16T17:03:28.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:28.466+0000 7fe56be00640 1 --2- 172.21.15.45:0/209928873 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe56c105330 0x7fe56c19a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:03:28.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:28.466+0000 7fe571400640 1 --2- 172.21.15.45:0/209928873 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe56c105cf0 0x7fe56c1a19b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:03:28.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:28.467+0000 7fe570a00640 1 --2- 172.21.15.45:0/209928873 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe56c104140 0x7fe56c19a400 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fe56000e9c0 tx=0x7fe56000ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:03:28.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:28.467+0000 7fe569600640 1 -- 172.21.15.45:0/209928873 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe56000cd80 con 0x7fe56c104140 2024-04-16T17:03:28.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:28.467+0000 7fe571e00640 1 -- 172.21.15.45:0/209928873 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe56c1a2310 con 0x7fe56c104140 2024-04-16T17:03:28.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:28.467+0000 7fe569600640 1 -- 172.21.15.45:0/209928873 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe5600035f0 con 0x7fe56c104140 2024-04-16T17:03:28.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:28.467+0000 7fe569600640 1 -- 172.21.15.45:0/209928873 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe560010690 con 0x7fe56c104140 2024-04-16T17:03:28.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:28.467+0000 7fe571e00640 1 -- 172.21.15.45:0/209928873 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe56c1a2860 con 0x7fe56c104140 2024-04-16T17:03:28.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:28.468+0000 7fe571e00640 1 -- 172.21.15.45:0/209928873 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe56c10b460 con 0x7fe56c104140 2024-04-16T17:03:28.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:28.471+0000 7fe569600640 1 -- 172.21.15.45:0/209928873 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fe5600037a0 con 0x7fe56c104140 2024-04-16T17:03:28.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:28.472+0000 7fe569600640 1 --2- 172.21.15.45:0/209928873 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fe5540a3bb0 0x7fe5540a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:28.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:28.473+0000 7fe569600640 1 -- 172.21.15.45:0/209928873 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8770+0+0 (secure 0 0 0) 0x7fe5600d0af0 con 0x7fe56c104140 2024-04-16T17:03:28.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:28.473+0000 7fe56be00640 1 --2- 172.21.15.45:0/209928873 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fe5540a3bb0 0x7fe5540a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:28.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:28.473+0000 7fe56be00640 1 --2- 172.21.15.45:0/209928873 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fe5540a3bb0 0x7fe5540a6070 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fe56c19b910 tx=0x7fe558005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:03:28.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:28.474+0000 7fe569600640 1 -- 172.21.15.45:0/209928873 <== mon.2 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe56009d2e0 con 0x7fe56c104140 2024-04-16T17:03:28.477 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:03:28 smithi045 ceph-mon[31676]: osdmap e87: 12 total, 12 up, 12 in 2024-04-16T17:03:28.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:03:28 smithi117 ceph-mon[36730]: osdmap e87: 12 total, 12 up, 12 in 2024-04-16T17:03:28.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:03:28 smithi104 ceph-mon[36457]: osdmap e87: 12 total, 12 up, 12 in 2024-04-16T17:03:28.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:28.683+0000 7fe571e00640 1 -- 172.21.15.45:0/209928873 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) -- 0x7fe56c02d620 con 0x7fe56c104140 2024-04-16T17:03:29.178 INFO:journalctl@ceph.mds.b.smithi104.stdout:Apr 16 17:03:28 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mds-b[67798]: 2024-04-16T17:03:28.882+0000 7fde231b9700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-16T17:03:29.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:29.226+0000 7fe569600640 1 -- 172.21.15.45:0/209928873 <== mon.2 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v89) ==== 169+0+0 (secure 0 0 0) 0x7fe56009cbe0 con 0x7fe56c104140 2024-04-16T17:03:29.230 INFO:teuthology.orchestra.run.smithi045.stderr:set pool 4 allow_ec_overwrites to true 2024-04-16T17:03:29.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:29.233+0000 7fe571e00640 1 -- 172.21.15.45:0/209928873 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fe5540a3bb0 msgr2=0x7fe5540a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:29.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:29.233+0000 7fe571e00640 1 --2- 172.21.15.45:0/209928873 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fe5540a3bb0 0x7fe5540a6070 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fe56c19b910 tx=0x7fe558005f90 comp rx=0 tx=0).stop 2024-04-16T17:03:29.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:29.233+0000 7fe571e00640 1 -- 172.21.15.45:0/209928873 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe56c104140 msgr2=0x7fe56c19a400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:29.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:29.233+0000 7fe571e00640 1 --2- 172.21.15.45:0/209928873 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe56c104140 0x7fe56c19a400 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fe56000e9c0 tx=0x7fe56000ee80 comp rx=0 tx=0).stop 2024-04-16T17:03:29.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:29.234+0000 7fe571e00640 1 -- 172.21.15.45:0/209928873 shutdown_connections 2024-04-16T17:03:29.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:29.234+0000 7fe571e00640 1 --2- 172.21.15.45:0/209928873 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fe5540a3bb0 0x7fe5540a6070 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:29.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:29.234+0000 7fe571e00640 1 --2- 172.21.15.45:0/209928873 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe56c105cf0 0x7fe56c1a19b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:29.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:29.234+0000 7fe571e00640 1 --2- 172.21.15.45:0/209928873 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe56c105330 0x7fe56c19a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:29.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:29.234+0000 7fe571e00640 1 --2- 172.21.15.45:0/209928873 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe56c104140 0x7fe56c19a400 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:29.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:29.235+0000 7fe571e00640 1 -- 172.21.15.45:0/209928873 >> 172.21.15.45:0/209928873 conn(0x7fe56c0ff6b0 msgr2=0x7fe56c108630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:03:29.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:29.235+0000 7fe571e00640 1 -- 172.21.15.45:0/209928873 shutdown_connections 2024-04-16T17:03:29.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:29.235+0000 7fe571e00640 1 -- 172.21.15.45:0/209928873 wait complete. 2024-04-16T17:03:29.270 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs cephfs_data_ec 2024-04-16T17:03:29.462 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:29.461+0000 7fd9e2e00640 1 -- 172.21.15.45:0/2129759122 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd9dc102050 msgr2=0x7fd9dc102470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:29.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:29.461+0000 7fd9e2e00640 1 --2- 172.21.15.45:0/2129759122 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd9dc102050 0x7fd9dc102470 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fd9cc009900 tx=0x7fd9cc02bc40 comp rx=0 tx=0).stop 2024-04-16T17:03:29.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:29.461+0000 7fd9e2e00640 1 -- 172.21.15.45:0/2129759122 shutdown_connections 2024-04-16T17:03:29.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:29.461+0000 7fd9e2e00640 1 --2- 172.21.15.45:0/2129759122 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd9dc103c00 0x7fd9dc10a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:29.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:29.461+0000 7fd9e2e00640 1 --2- 172.21.15.45:0/2129759122 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd9dc103240 0x7fd9dc1036c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:29.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:29.461+0000 7fd9e2e00640 1 --2- 172.21.15.45:0/2129759122 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd9dc102050 0x7fd9dc102470 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:29.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:29.461+0000 7fd9e2e00640 1 -- 172.21.15.45:0/2129759122 >> 172.21.15.45:0/2129759122 conn(0x7fd9dc0fd5e0 msgr2=0x7fd9dc0ffa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:03:29.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:29.462+0000 7fd9e2e00640 1 -- 172.21.15.45:0/2129759122 shutdown_connections 2024-04-16T17:03:29.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:29.462+0000 7fd9e2e00640 1 -- 172.21.15.45:0/2129759122 wait complete. 2024-04-16T17:03:29.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:29.463+0000 7fd9e2e00640 1 Processor -- start 2024-04-16T17:03:29.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:29.463+0000 7fd9e2e00640 1 -- start start 2024-04-16T17:03:29.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:29.463+0000 7fd9e2e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd9dc102050 0x7fd9dc19a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:29.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:29.463+0000 7fd9e2e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd9dc103240 0x7fd9dc19aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:29.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:29.463+0000 7fd9e2e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd9dc103c00 0x7fd9dc1a1b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:29.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:29.463+0000 7fd9e2e00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fd9dc10dd30 con 0x7fd9dc103240 2024-04-16T17:03:29.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:29.463+0000 7fd9e2e00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fd9dc10dbb0 con 0x7fd9dc102050 2024-04-16T17:03:29.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:29.463+0000 7fd9e2e00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fd9dc10deb0 con 0x7fd9dc103c00 2024-04-16T17:03:29.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:29.464+0000 7fd9dbe00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd9dc102050 0x7fd9dc19a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:29.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:29.464+0000 7fd9dbe00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd9dc102050 0x7fd9dc19a5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:52110/0 (socket says 172.21.15.45:52110) 2024-04-16T17:03:29.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:29.464+0000 7fd9e0e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd9dc103c00 0x7fd9dc1a1b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:29.466 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:29.464+0000 7fd9dbe00640 1 -- 172.21.15.45:0/2259016244 learned_addr learned my addr 172.21.15.45:0/2259016244 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:03:29.466 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:29.464+0000 7fd9d3400640 1 --2- 172.21.15.45:0/2259016244 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd9dc103240 0x7fd9dc19aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:29.466 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:29.464+0000 7fd9dbe00640 1 -- 172.21.15.45:0/2259016244 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd9dc103c00 msgr2=0x7fd9dc1a1b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:29.466 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:29.464+0000 7fd9dbe00640 1 --2- 172.21.15.45:0/2259016244 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd9dc103c00 0x7fd9dc1a1b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:29.466 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:29.464+0000 7fd9dbe00640 1 -- 172.21.15.45:0/2259016244 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd9dc103240 msgr2=0x7fd9dc19aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:29.466 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:29.464+0000 7fd9dbe00640 1 --2- 172.21.15.45:0/2259016244 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd9dc103240 0x7fd9dc19aae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:29.466 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:29.464+0000 7fd9dbe00640 1 -- 172.21.15.45:0/2259016244 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9dc1a21c0 con 0x7fd9dc102050 2024-04-16T17:03:29.466 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:29.464+0000 7fd9d3400640 1 --2- 172.21.15.45:0/2259016244 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd9dc103240 0x7fd9dc19aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:03:29.466 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:29.464+0000 7fd9dbe00640 1 --2- 172.21.15.45:0/2259016244 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd9dc102050 0x7fd9dc19a5a0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fd9cc02efd0 tx=0x7fd9cc02c660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:03:29.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:29.465+0000 7fd9d9600640 1 -- 172.21.15.45:0/2259016244 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9cc041070 con 0x7fd9dc102050 2024-04-16T17:03:29.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:29.465+0000 7fd9d9600640 1 -- 172.21.15.45:0/2259016244 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd9cc004450 con 0x7fd9dc102050 2024-04-16T17:03:29.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:29.465+0000 7fd9e2e00640 1 -- 172.21.15.45:0/2259016244 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9dc1a2450 con 0x7fd9dc102050 2024-04-16T17:03:29.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:29.465+0000 7fd9d9600640 1 -- 172.21.15.45:0/2259016244 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9cc003e50 con 0x7fd9dc102050 2024-04-16T17:03:29.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:29.465+0000 7fd9e2e00640 1 -- 172.21.15.45:0/2259016244 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9dc1a2910 con 0x7fd9dc102050 2024-04-16T17:03:29.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:29.466+0000 7fd9e2e00640 1 -- 172.21.15.45:0/2259016244 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd9dc1a2b70 con 0x7fd9dc102050 2024-04-16T17:03:29.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:29.468+0000 7fd9d9600640 1 -- 172.21.15.45:0/2259016244 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fd9cc044070 con 0x7fd9dc102050 2024-04-16T17:03:29.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:29.469+0000 7fd9d9600640 1 --2- 172.21.15.45:0/2259016244 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fd9c00a3d50 0x7fd9c00a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:29.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:29.469+0000 7fd9d9600640 1 -- 172.21.15.45:0/2259016244 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8770+0+0 (secure 0 0 0) 0x7fd9cc0f2250 con 0x7fd9dc102050 2024-04-16T17:03:29.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:29.470+0000 7fd9d3400640 1 --2- 172.21.15.45:0/2259016244 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fd9c00a3d50 0x7fd9c00a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:29.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:29.470+0000 7fd9d3400640 1 --2- 172.21.15.45:0/2259016244 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fd9c00a3d50 0x7fd9c00a6210 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fd9dc19bab0 tx=0x7fd9c4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:03:29.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:29.473+0000 7fd9d9600640 1 -- 172.21.15.45:0/2259016244 <== mon.2 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd9cc0be9b0 con 0x7fd9dc102050 2024-04-16T17:03:29.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:03:29 smithi117 ceph-mon[36730]: pgmap v321: 97 pgs: 4 creating+peering, 93 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s wr, 8 op/s 2024-04-16T17:03:29.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:03:29 smithi117 ceph-mon[36730]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-16T17:03:29.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:03:29 smithi117 ceph-mon[36730]: osdmap e88: 12 total, 12 up, 12 in 2024-04-16T17:03:29.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:03:29 smithi117 ceph-mon[36730]: from='client.? 172.21.15.45:0/209928873' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-04-16T17:03:29.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:03:29 smithi117 ceph-mon[36730]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-04-16T17:03:29.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:03:29 smithi104 ceph-mon[36457]: pgmap v321: 97 pgs: 4 creating+peering, 93 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s wr, 8 op/s 2024-04-16T17:03:29.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:03:29 smithi104 ceph-mon[36457]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-16T17:03:29.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:03:29 smithi104 ceph-mon[36457]: osdmap e88: 12 total, 12 up, 12 in 2024-04-16T17:03:29.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:03:29 smithi104 ceph-mon[36457]: from='client.? 172.21.15.45:0/209928873' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-04-16T17:03:29.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:03:29 smithi104 ceph-mon[36457]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-04-16T17:03:29.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:29.718+0000 7fd9e2e00640 1 -- 172.21.15.45:0/2259016244 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) -- 0x7fd9dc1a2e40 con 0x7fd9dc102050 2024-04-16T17:03:29.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:03:29 smithi045 ceph-mon[31676]: pgmap v321: 97 pgs: 4 creating+peering, 93 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s wr, 8 op/s 2024-04-16T17:03:29.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:03:29 smithi045 ceph-mon[31676]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-16T17:03:29.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:03:29 smithi045 ceph-mon[31676]: osdmap e88: 12 total, 12 up, 12 in 2024-04-16T17:03:29.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:03:29 smithi045 ceph-mon[31676]: from='client.? 172.21.15.45:0/209928873' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-04-16T17:03:29.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:03:29 smithi045 ceph-mon[31676]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-04-16T17:03:30.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:30.263+0000 7fd9d9600640 1 -- 172.21.15.45:0/2259016244 <== mon.2 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v17) ==== 137+0+0 (secure 0 0 0) 0x7fd9cc0b8e20 con 0x7fd9dc102050 2024-04-16T17:03:30.265 INFO:teuthology.orchestra.run.smithi045.stderr:added data pool 4 to fsmap 2024-04-16T17:03:30.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:30.266+0000 7fd9e2e00640 1 -- 172.21.15.45:0/2259016244 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fd9c00a3d50 msgr2=0x7fd9c00a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:30.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:30.267+0000 7fd9e2e00640 1 --2- 172.21.15.45:0/2259016244 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fd9c00a3d50 0x7fd9c00a6210 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fd9dc19bab0 tx=0x7fd9c4005f90 comp rx=0 tx=0).stop 2024-04-16T17:03:30.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:30.267+0000 7fd9e2e00640 1 -- 172.21.15.45:0/2259016244 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd9dc102050 msgr2=0x7fd9dc19a5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:30.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:30.267+0000 7fd9e2e00640 1 --2- 172.21.15.45:0/2259016244 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd9dc102050 0x7fd9dc19a5a0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fd9cc02efd0 tx=0x7fd9cc02c660 comp rx=0 tx=0).stop 2024-04-16T17:03:30.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:30.267+0000 7fd9e2e00640 1 -- 172.21.15.45:0/2259016244 shutdown_connections 2024-04-16T17:03:30.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:30.267+0000 7fd9e2e00640 1 --2- 172.21.15.45:0/2259016244 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fd9c00a3d50 0x7fd9c00a6210 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:30.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:30.267+0000 7fd9e2e00640 1 --2- 172.21.15.45:0/2259016244 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd9dc103c00 0x7fd9dc1a1b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:30.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:30.267+0000 7fd9e2e00640 1 --2- 172.21.15.45:0/2259016244 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd9dc103240 0x7fd9dc19aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:30.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:30.267+0000 7fd9e2e00640 1 --2- 172.21.15.45:0/2259016244 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd9dc102050 0x7fd9dc19a5a0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:30.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:30.267+0000 7fd9e2e00640 1 -- 172.21.15.45:0/2259016244 >> 172.21.15.45:0/2259016244 conn(0x7fd9dc0fd5e0 msgr2=0x7fd9dc074230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:03:30.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:30.267+0000 7fd9e2e00640 1 -- 172.21.15.45:0/2259016244 shutdown_connections 2024-04-16T17:03:30.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:30.267+0000 7fd9e2e00640 1 -- 172.21.15.45:0/2259016244 wait complete. 2024-04-16T17:03:30.290 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-16T17:03:30.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:30.512+0000 7fe6eb200640 1 -- 172.21.15.45:0/3268943656 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe6ec1039b0 msgr2=0x7fe6ec10a2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:30.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:30.512+0000 7fe6eb200640 1 --2- 172.21.15.45:0/3268943656 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe6ec1039b0 0x7fe6ec10a2d0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fe6e4009860 tx=0x7fe6e402bb70 comp rx=0 tx=0).stop 2024-04-16T17:03:30.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:30.512+0000 7fe6eb200640 1 -- 172.21.15.45:0/3268943656 shutdown_connections 2024-04-16T17:03:30.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:30.512+0000 7fe6eb200640 1 --2- 172.21.15.45:0/3268943656 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe6ec1039b0 0x7fe6ec10a2d0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:30.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:30.512+0000 7fe6eb200640 1 --2- 172.21.15.45:0/3268943656 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe6ec102ff0 0x7fe6ec103470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:30.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:30.512+0000 7fe6eb200640 1 --2- 172.21.15.45:0/3268943656 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe6ec101ea0 0x7fe6ec1022c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:30.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:30.512+0000 7fe6eb200640 1 -- 172.21.15.45:0/3268943656 >> 172.21.15.45:0/3268943656 conn(0x7fe6ec0fd410 msgr2=0x7fe6ec0ff850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:03:30.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:30.513+0000 7fe6eb200640 1 -- 172.21.15.45:0/3268943656 shutdown_connections 2024-04-16T17:03:30.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:30.513+0000 7fe6eb200640 1 -- 172.21.15.45:0/3268943656 wait complete. 2024-04-16T17:03:30.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:30.513+0000 7fe6eb200640 1 Processor -- start 2024-04-16T17:03:30.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:30.514+0000 7fe6eb200640 1 -- start start 2024-04-16T17:03:30.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:30.514+0000 7fe6eb200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe6ec101ea0 0x7fe6ec071d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:30.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:30.514+0000 7fe6eb200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe6ec102ff0 0x7fe6ec072290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:30.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:30.514+0000 7fe6eb200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe6ec1039b0 0x7fe6ec0752e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:30.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:30.514+0000 7fe6eb200640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fe6ec06c470 con 0x7fe6ec101ea0 2024-04-16T17:03:30.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:30.514+0000 7fe6eb200640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fe6ec06c2f0 con 0x7fe6ec102ff0 2024-04-16T17:03:30.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:30.514+0000 7fe6eb200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fe6ec06c5f0 con 0x7fe6ec1039b0 2024-04-16T17:03:30.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:30.514+0000 7fe6e9e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe6ec101ea0 0x7fe6ec071d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:30.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:30.514+0000 7fe6e9400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe6ec102ff0 0x7fe6ec072290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:30.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:30.514+0000 7fe6ea800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe6ec1039b0 0x7fe6ec0752e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:30.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:30.514+0000 7fe6e9e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe6ec101ea0 0x7fe6ec071d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43752/0 (socket says 172.21.15.45:43752) 2024-04-16T17:03:30.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:30.514+0000 7fe6e9e00640 1 -- 172.21.15.45:0/1971560110 learned_addr learned my addr 172.21.15.45:0/1971560110 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:03:30.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:30.515+0000 7fe6e9400640 1 -- 172.21.15.45:0/1971560110 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe6ec1039b0 msgr2=0x7fe6ec0752e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:30.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:30.515+0000 7fe6e9400640 1 --2- 172.21.15.45:0/1971560110 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe6ec1039b0 0x7fe6ec0752e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:30.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:30.515+0000 7fe6e9400640 1 -- 172.21.15.45:0/1971560110 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe6ec101ea0 msgr2=0x7fe6ec071d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:30.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:30.515+0000 7fe6e9400640 1 --2- 172.21.15.45:0/1971560110 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe6ec101ea0 0x7fe6ec071d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:30.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:30.515+0000 7fe6e9400640 1 -- 172.21.15.45:0/1971560110 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6ec075950 con 0x7fe6ec102ff0 2024-04-16T17:03:30.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:30.515+0000 7fe6ea800640 1 --2- 172.21.15.45:0/1971560110 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe6ec1039b0 0x7fe6ec0752e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:03:30.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:30.515+0000 7fe6e9e00640 1 --2- 172.21.15.45:0/1971560110 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe6ec101ea0 0x7fe6ec071d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:03:30.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:30.515+0000 7fe6e9400640 1 --2- 172.21.15.45:0/1971560110 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe6ec102ff0 0x7fe6ec072290 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fe6d4005fd0 tx=0x7fe6d400ec20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:03:30.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:30.515+0000 7fe6e2a00640 1 -- 172.21.15.45:0/1971560110 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6d400ccc0 con 0x7fe6ec102ff0 2024-04-16T17:03:30.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:30.515+0000 7fe6e2a00640 1 -- 172.21.15.45:0/1971560110 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe6d400ee70 con 0x7fe6ec102ff0 2024-04-16T17:03:30.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:30.515+0000 7fe6eb200640 1 -- 172.21.15.45:0/1971560110 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6ec075c40 con 0x7fe6ec102ff0 2024-04-16T17:03:30.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:30.515+0000 7fe6e2a00640 1 -- 172.21.15.45:0/1971560110 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6d4010430 con 0x7fe6ec102ff0 2024-04-16T17:03:30.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:30.515+0000 7fe6eb200640 1 -- 172.21.15.45:0/1971560110 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6ec076190 con 0x7fe6ec102ff0 2024-04-16T17:03:30.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:30.516+0000 7fe6eb200640 1 -- 172.21.15.45:0/1971560110 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6ec1091b0 con 0x7fe6ec102ff0 2024-04-16T17:03:30.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:30.519+0000 7fe6e2a00640 1 -- 172.21.15.45:0/1971560110 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fe6d4003ed0 con 0x7fe6ec102ff0 2024-04-16T17:03:30.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:30.520+0000 7fe6e2a00640 1 --2- 172.21.15.45:0/1971560110 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fe6d00a3bb0 0x7fe6d00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:30.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:30.520+0000 7fe6e2a00640 1 -- 172.21.15.45:0/1971560110 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7fe6d40d09f0 con 0x7fe6ec102ff0 2024-04-16T17:03:30.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:30.520+0000 7fe6e9e00640 1 --2- 172.21.15.45:0/1971560110 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fe6d00a3bb0 0x7fe6d00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:30.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:30.521+0000 7fe6e9e00640 1 --2- 172.21.15.45:0/1971560110 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fe6d00a3bb0 0x7fe6d00a6070 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fe6dc009fd0 tx=0x7fe6dc009720 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:03:30.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:30.522+0000 7fe6e2a00640 1 -- 172.21.15.45:0/1971560110 <== mon.2 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe6d409d1c0 con 0x7fe6ec102ff0 2024-04-16T17:03:30.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:03:30 smithi117 ceph-mon[36730]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-04-16T17:03:30.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:03:30 smithi117 ceph-mon[36730]: osdmap e89: 12 total, 12 up, 12 in 2024-04-16T17:03:30.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:03:30 smithi117 ceph-mon[36730]: from='client.? 172.21.15.45:0/2259016244' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-04-16T17:03:30.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:03:30 smithi117 ceph-mon[36730]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-04-16T17:03:30.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:03:30 smithi104 ceph-mon[36457]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-04-16T17:03:30.679 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:03:30 smithi104 ceph-mon[36457]: osdmap e89: 12 total, 12 up, 12 in 2024-04-16T17:03:30.679 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:03:30 smithi104 ceph-mon[36457]: from='client.? 172.21.15.45:0/2259016244' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-04-16T17:03:30.679 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:03:30 smithi104 ceph-mon[36457]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-04-16T17:03:30.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:03:30 smithi045 ceph-mon[31676]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-04-16T17:03:30.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:03:30 smithi045 ceph-mon[31676]: osdmap e89: 12 total, 12 up, 12 in 2024-04-16T17:03:30.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:03:30 smithi045 ceph-mon[31676]: from='client.? 172.21.15.45:0/2259016244' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-04-16T17:03:30.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:03:30 smithi045 ceph-mon[31676]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-04-16T17:03:30.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:30.771+0000 7fe6eb200640 1 -- 172.21.15.45:0/1971560110 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe6ec02d620 con 0x7fe6ec102ff0 2024-04-16T17:03:30.774 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:30.773+0000 7fe6e2a00640 1 -- 172.21.15.45:0/1971560110 <== mon.2 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+11815 (secure 0 0 0) 0x7fe6d409cac0 con 0x7fe6ec102ff0 2024-04-16T17:03:30.774 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-16T17:03:30.774 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/1748842443","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":1748842443},{"type":"v1","addr":"172.21.15.45:6835","nonce":1748842443}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6839/1308627051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":1308627051},{"type":"v1","addr":"172.21.15.45:6839","nonce":1308627051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/1768242489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1768242489},{"type":"v1","addr":"172.21.15.117:6833","nonce":1768242489}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6833/3558902585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":3558902585},{"type":"v1","addr":"172.21.15.104:6833","nonce":3558902585}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24478,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/1308142903","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":1308142903},{"type":"v1","addr":"172.21.15.45:6837","nonce":1308142903}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24491,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6835/3430679937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":3430679937},{"type":"v1","addr":"172.21.15.104:6835","nonce":3430679937}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24499,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/2224123841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2224123841},{"type":"v1","addr":"172.21.15.117:6835","nonce":2224123841}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24511,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/2471356815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":2471356815},{"type":"v1","addr":"172.21.15.117:6837","nonce":2471356815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24524,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6837/2647756509","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":2647756509},{"type":"v1","addr":"172.21.15.104:6837","nonce":2647756509}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24532,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6839/2378675949","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":2378675949},{"type":"v1","addr":"172.21.15.117:6839","nonce":2378675949}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24569,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6841/3755310822","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6840","nonce":3755310822},{"type":"v1","addr":"172.21.15.45:6841","nonce":3755310822}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-16T17:03:24.663611+0000","modified":"2024-04-16T17:03:29.720950+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24581},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24581":{"gid":24581,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.104:6839/3660070592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6838","nonce":3660070592},{"type":"v1","addr":"172.21.15.104:6839","nonce":3660070592}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24581,"qdb_cluster":[24581]},"id":1}]} 2024-04-16T17:03:30.775 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 17 2024-04-16T17:03:30.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:30.777+0000 7fe6eb200640 1 -- 172.21.15.45:0/1971560110 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fe6d00a3bb0 msgr2=0x7fe6d00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:30.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:30.777+0000 7fe6eb200640 1 --2- 172.21.15.45:0/1971560110 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fe6d00a3bb0 0x7fe6d00a6070 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fe6dc009fd0 tx=0x7fe6dc009720 comp rx=0 tx=0).stop 2024-04-16T17:03:30.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:30.777+0000 7fe6eb200640 1 -- 172.21.15.45:0/1971560110 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe6ec102ff0 msgr2=0x7fe6ec072290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:30.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:30.777+0000 7fe6eb200640 1 --2- 172.21.15.45:0/1971560110 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe6ec102ff0 0x7fe6ec072290 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fe6d4005fd0 tx=0x7fe6d400ec20 comp rx=0 tx=0).stop 2024-04-16T17:03:30.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:30.777+0000 7fe6eb200640 1 -- 172.21.15.45:0/1971560110 shutdown_connections 2024-04-16T17:03:30.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:30.777+0000 7fe6eb200640 1 --2- 172.21.15.45:0/1971560110 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fe6d00a3bb0 0x7fe6d00a6070 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:30.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:30.777+0000 7fe6eb200640 1 --2- 172.21.15.45:0/1971560110 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe6ec1039b0 0x7fe6ec0752e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:30.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:30.777+0000 7fe6eb200640 1 --2- 172.21.15.45:0/1971560110 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe6ec102ff0 0x7fe6ec072290 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:30.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:30.777+0000 7fe6eb200640 1 --2- 172.21.15.45:0/1971560110 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe6ec101ea0 0x7fe6ec071d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:30.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:30.777+0000 7fe6eb200640 1 -- 172.21.15.45:0/1971560110 >> 172.21.15.45:0/1971560110 conn(0x7fe6ec0fd410 msgr2=0x7fe6ec106380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:03:30.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:30.777+0000 7fe6eb200640 1 -- 172.21.15.45:0/1971560110 shutdown_connections 2024-04-16T17:03:30.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:30.777+0000 7fe6eb200640 1 -- 172.21.15.45:0/1971560110 wait complete. 2024-04-16T17:03:30.797 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-16T17:03:31.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:31.025+0000 7f46ef000640 1 -- 172.21.15.45:0/12709052 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f46e8109670 msgr2=0x7f46e8109af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:31.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:31.025+0000 7f46ef000640 1 --2- 172.21.15.45:0/12709052 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f46e8109670 0x7f46e8109af0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f46d4009900 tx=0x7f46d402bc50 comp rx=0 tx=0).stop 2024-04-16T17:03:31.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:31.025+0000 7f46ef000640 1 -- 172.21.15.45:0/12709052 shutdown_connections 2024-04-16T17:03:31.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:31.025+0000 7f46ef000640 1 --2- 172.21.15.45:0/12709052 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f46e8109670 0x7f46e8109af0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:31.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:31.025+0000 7f46ef000640 1 --2- 172.21.15.45:0/12709052 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f46e810a030 0x7f46e81108c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:31.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:31.025+0000 7f46ef000640 1 --2- 172.21.15.45:0/12709052 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f46e8108480 0x7f46e81088a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:31.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:31.025+0000 7f46ef000640 1 -- 172.21.15.45:0/12709052 >> 172.21.15.45:0/12709052 conn(0x7f46e8077440 msgr2=0x7f46e8079860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:03:31.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:31.025+0000 7f46ef000640 1 -- 172.21.15.45:0/12709052 shutdown_connections 2024-04-16T17:03:31.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:31.025+0000 7f46ef000640 1 -- 172.21.15.45:0/12709052 wait complete. 2024-04-16T17:03:31.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:31.027+0000 7f46ef000640 1 Processor -- start 2024-04-16T17:03:31.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:31.027+0000 7f46ef000640 1 -- start start 2024-04-16T17:03:31.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:31.027+0000 7f46ef000640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f46e8108480 0x7f46e819e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:31.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:31.027+0000 7f46ef000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f46e8109670 0x7f46e819eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:31.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:31.027+0000 7f46ef000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f46e810a030 0x7f46e81a5dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:31.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:31.027+0000 7f46ef000640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f46e8114110 con 0x7f46e810a030 2024-04-16T17:03:31.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:31.027+0000 7f46ef000640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f46e8113f90 con 0x7f46e8108480 2024-04-16T17:03:31.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:31.027+0000 7f46ef000640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f46e8114290 con 0x7f46e8109670 2024-04-16T17:03:31.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:31.028+0000 7f46edc00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f46e8108480 0x7f46e819e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:31.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:31.028+0000 7f46edc00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f46e8108480 0x7f46e819e860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:49118/0 (socket says 172.21.15.45:49118) 2024-04-16T17:03:31.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:31.028+0000 7f46edc00640 1 -- 172.21.15.45:0/3432094455 learned_addr learned my addr 172.21.15.45:0/3432094455 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:03:31.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:31.028+0000 7f46ee600640 1 --2- 172.21.15.45:0/3432094455 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f46e810a030 0x7f46e81a5dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:31.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:31.028+0000 7f46ed200640 1 --2- 172.21.15.45:0/3432094455 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f46e8109670 0x7f46e819eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:31.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:31.028+0000 7f46edc00640 1 -- 172.21.15.45:0/3432094455 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f46e8109670 msgr2=0x7f46e819eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:31.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:31.028+0000 7f46edc00640 1 --2- 172.21.15.45:0/3432094455 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f46e8109670 0x7f46e819eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:31.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:31.028+0000 7f46edc00640 1 -- 172.21.15.45:0/3432094455 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f46e810a030 msgr2=0x7f46e81a5dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:31.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:31.028+0000 7f46edc00640 1 --2- 172.21.15.45:0/3432094455 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f46e810a030 0x7f46e81a5dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:31.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:31.028+0000 7f46edc00640 1 -- 172.21.15.45:0/3432094455 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46e81a6430 con 0x7f46e8108480 2024-04-16T17:03:31.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:31.029+0000 7f46edc00640 1 --2- 172.21.15.45:0/3432094455 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f46e8108480 0x7f46e819e860 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f46dc00e9c0 tx=0x7f46dc00ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:03:31.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:31.029+0000 7f46e6000640 1 -- 172.21.15.45:0/3432094455 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46dc00cd80 con 0x7f46e8108480 2024-04-16T17:03:31.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:31.029+0000 7f46e6000640 1 -- 172.21.15.45:0/3432094455 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f46dc0035f0 con 0x7f46e8108480 2024-04-16T17:03:31.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:31.029+0000 7f46e6000640 1 -- 172.21.15.45:0/3432094455 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46dc010690 con 0x7f46e8108480 2024-04-16T17:03:31.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:31.029+0000 7f46ef000640 1 -- 172.21.15.45:0/3432094455 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46e81a6720 con 0x7f46e8108480 2024-04-16T17:03:31.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:31.029+0000 7f46ef000640 1 -- 172.21.15.45:0/3432094455 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46e81a6bf0 con 0x7f46e8108480 2024-04-16T17:03:31.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:31.030+0000 7f46ef000640 1 -- 172.21.15.45:0/3432094455 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46e810f7a0 con 0x7f46e8108480 2024-04-16T17:03:31.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:31.032+0000 7f46e6000640 1 -- 172.21.15.45:0/3432094455 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f46dc0037a0 con 0x7f46e8108480 2024-04-16T17:03:31.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:31.034+0000 7f46e6000640 1 --2- 172.21.15.45:0/3432094455 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f46d00a3c30 0x7f46d00a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:31.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:31.034+0000 7f46e6000640 1 -- 172.21.15.45:0/3432094455 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f46dc0d0aa0 con 0x7f46e8108480 2024-04-16T17:03:31.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:31.034+0000 7f46ed200640 1 --2- 172.21.15.45:0/3432094455 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f46d00a3c30 0x7f46d00a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:31.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:31.035+0000 7f46ed200640 1 --2- 172.21.15.45:0/3432094455 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f46d00a3c30 0x7f46d00a60f0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f46d402c7d0 tx=0x7f46d4005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:03:31.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:31.036+0000 7f46e6000640 1 -- 172.21.15.45:0/3432094455 <== mon.2 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f46dc09d270 con 0x7f46e8108480 2024-04-16T17:03:31.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:31.246+0000 7f46ef000640 1 -- 172.21.15.45:0/3432094455 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f46e802d8d0 con 0x7f46e8108480 2024-04-16T17:03:31.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:31.248+0000 7f46e6000640 1 -- 172.21.15.45:0/3432094455 <== mon.2 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+24164 (secure 0 0 0) 0x7f46dc09cb70 con 0x7f46e8108480 2024-04-16T17:03:31.250 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-16T17:03:31.250 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":90,"fsid":"b1fecd38-fc11-11ee-bc90-c7b262605968","created":"2024-04-16T16:54:02.385906+0000","modified":"2024-04-16T17:03:29.720565+0000","last_up_change":"2024-04-16T17:02:25.162624+0000","last_in_change":"2024-04-16T17:02:04.903274+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-16T16:58:06.166659+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-16T17:03:22.304198+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-16T17:03:23.616450+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-16T17:03:27.226289+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b655de2c-59f8-4fd4-ac98-0cc222229f56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":2290170881},{"type":"v1","addr":"172.21.15.45:6803","nonce":2290170881}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":2290170881},{"type":"v1","addr":"172.21.15.45:6805","nonce":2290170881}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":2290170881},{"type":"v1","addr":"172.21.15.45:6809","nonce":2290170881}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":2290170881},{"type":"v1","addr":"172.21.15.45:6807","nonce":2290170881}]},"public_addr":"172.21.15.45:6803/2290170881","cluster_addr":"172.21.15.45:6805/2290170881","heartbeat_back_addr":"172.21.15.45:6809/2290170881","heartbeat_front_addr":"172.21.15.45:6807/2290170881","state":["exists","up"]},{"osd":1,"uuid":"ebd450ec-08fd-4c3d-8b24-4e10634ddb7e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6800","nonce":1462942067},{"type":"v1","addr":"172.21.15.104:6801","nonce":1462942067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":1462942067},{"type":"v1","addr":"172.21.15.104:6803","nonce":1462942067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":1462942067},{"type":"v1","addr":"172.21.15.104:6807","nonce":1462942067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":1462942067},{"type":"v1","addr":"172.21.15.104:6805","nonce":1462942067}]},"public_addr":"172.21.15.104:6801/1462942067","cluster_addr":"172.21.15.104:6803/1462942067","heartbeat_back_addr":"172.21.15.104:6807/1462942067","heartbeat_front_addr":"172.21.15.104:6805/1462942067","state":["exists","up"]},{"osd":2,"uuid":"b318a512-4190-42a2-8104-dcc4026ad8ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":2829260735},{"type":"v1","addr":"172.21.15.117:6801","nonce":2829260735}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":2829260735},{"type":"v1","addr":"172.21.15.117:6803","nonce":2829260735}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":2829260735},{"type":"v1","addr":"172.21.15.117:6807","nonce":2829260735}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":2829260735},{"type":"v1","addr":"172.21.15.117:6805","nonce":2829260735}]},"public_addr":"172.21.15.117:6801/2829260735","cluster_addr":"172.21.15.117:6803/2829260735","heartbeat_back_addr":"172.21.15.117:6807/2829260735","heartbeat_front_addr":"172.21.15.117:6805/2829260735","state":["exists","up"]},{"osd":3,"uuid":"ec6490cb-a715-4c12-a54a-225aae26fcb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":2823712137},{"type":"v1","addr":"172.21.15.45:6811","nonce":2823712137}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":2823712137},{"type":"v1","addr":"172.21.15.45:6813","nonce":2823712137}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":2823712137},{"type":"v1","addr":"172.21.15.45:6817","nonce":2823712137}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":2823712137},{"type":"v1","addr":"172.21.15.45:6815","nonce":2823712137}]},"public_addr":"172.21.15.45:6811/2823712137","cluster_addr":"172.21.15.45:6813/2823712137","heartbeat_back_addr":"172.21.15.45:6817/2823712137","heartbeat_front_addr":"172.21.15.45:6815/2823712137","state":["exists","up"]},{"osd":4,"uuid":"5d91d568-0250-4aae-8dc2-a0e079f92062","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":1396278127},{"type":"v1","addr":"172.21.15.104:6809","nonce":1396278127}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":1396278127},{"type":"v1","addr":"172.21.15.104:6811","nonce":1396278127}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":1396278127},{"type":"v1","addr":"172.21.15.104:6815","nonce":1396278127}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":1396278127},{"type":"v1","addr":"172.21.15.104:6813","nonce":1396278127}]},"public_addr":"172.21.15.104:6809/1396278127","cluster_addr":"172.21.15.104:6811/1396278127","heartbeat_back_addr":"172.21.15.104:6815/1396278127","heartbeat_front_addr":"172.21.15.104:6813/1396278127","state":["exists","up"]},{"osd":5,"uuid":"ac510b43-6fb2-4d0f-bc5f-16ac080d9e9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":4145720047},{"type":"v1","addr":"172.21.15.117:6809","nonce":4145720047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":4145720047},{"type":"v1","addr":"172.21.15.117:6811","nonce":4145720047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":4145720047},{"type":"v1","addr":"172.21.15.117:6815","nonce":4145720047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":4145720047},{"type":"v1","addr":"172.21.15.117:6813","nonce":4145720047}]},"public_addr":"172.21.15.117:6809/4145720047","cluster_addr":"172.21.15.117:6811/4145720047","heartbeat_back_addr":"172.21.15.117:6815/4145720047","heartbeat_front_addr":"172.21.15.117:6813/4145720047","state":["exists","up"]},{"osd":6,"uuid":"a41adc18-d88a-419e-b2c2-b089c1f32472","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":3254229176},{"type":"v1","addr":"172.21.15.45:6819","nonce":3254229176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":3254229176},{"type":"v1","addr":"172.21.15.45:6821","nonce":3254229176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":3254229176},{"type":"v1","addr":"172.21.15.45:6825","nonce":3254229176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":3254229176},{"type":"v1","addr":"172.21.15.45:6823","nonce":3254229176}]},"public_addr":"172.21.15.45:6819/3254229176","cluster_addr":"172.21.15.45:6821/3254229176","heartbeat_back_addr":"172.21.15.45:6825/3254229176","heartbeat_front_addr":"172.21.15.45:6823/3254229176","state":["exists","up"]},{"osd":7,"uuid":"b71712a4-7cf6-454d-9fd1-565e3df21c1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":4151077186},{"type":"v1","addr":"172.21.15.104:6817","nonce":4151077186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":4151077186},{"type":"v1","addr":"172.21.15.104:6819","nonce":4151077186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":4151077186},{"type":"v1","addr":"172.21.15.104:6823","nonce":4151077186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":4151077186},{"type":"v1","addr":"172.21.15.104:6821","nonce":4151077186}]},"public_addr":"172.21.15.104:6817/4151077186","cluster_addr":"172.21.15.104:6819/4151077186","heartbeat_back_addr":"172.21.15.104:6823/4151077186","heartbeat_front_addr":"172.21.15.104:6821/4151077186","state":["exists","up"]},{"osd":8,"uuid":"9e572aef-59a8-450e-a92e-df9dff4d57a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":1500286830},{"type":"v1","addr":"172.21.15.117:6817","nonce":1500286830}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":1500286830},{"type":"v1","addr":"172.21.15.117:6819","nonce":1500286830}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":1500286830},{"type":"v1","addr":"172.21.15.117:6823","nonce":1500286830}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":1500286830},{"type":"v1","addr":"172.21.15.117:6821","nonce":1500286830}]},"public_addr":"172.21.15.117:6817/1500286830","cluster_addr":"172.21.15.117:6819/1500286830","heartbeat_back_addr":"172.21.15.117:6823/1500286830","heartbeat_front_addr":"172.21.15.117:6821/1500286830","state":["exists","up"]},{"osd":9,"uuid":"014e9598-c413-4cc4-9bb8-61c6e8f59d30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":3106734074},{"type":"v1","addr":"172.21.15.45:6827","nonce":3106734074}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":3106734074},{"type":"v1","addr":"172.21.15.45:6829","nonce":3106734074}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":3106734074},{"type":"v1","addr":"172.21.15.45:6833","nonce":3106734074}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":3106734074},{"type":"v1","addr":"172.21.15.45:6831","nonce":3106734074}]},"public_addr":"172.21.15.45:6827/3106734074","cluster_addr":"172.21.15.45:6829/3106734074","heartbeat_back_addr":"172.21.15.45:6833/3106734074","heartbeat_front_addr":"172.21.15.45:6831/3106734074","state":["exists","up"]},{"osd":10,"uuid":"2deba364-3e39-47be-8a1f-8256df5b82d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":1404794652},{"type":"v1","addr":"172.21.15.104:6825","nonce":1404794652}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":1404794652},{"type":"v1","addr":"172.21.15.104:6827","nonce":1404794652}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":1404794652},{"type":"v1","addr":"172.21.15.104:6831","nonce":1404794652}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":1404794652},{"type":"v1","addr":"172.21.15.104:6829","nonce":1404794652}]},"public_addr":"172.21.15.104:6825/1404794652","cluster_addr":"172.21.15.104:6827/1404794652","heartbeat_back_addr":"172.21.15.104:6831/1404794652","heartbeat_front_addr":"172.21.15.104:6829/1404794652","state":["exists","up"]},{"osd":11,"uuid":"3fda4fea-59ad-4d1a-9fe6-fcb30e31fd7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":3384003243},{"type":"v1","addr":"172.21.15.117:6825","nonce":3384003243}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":3384003243},{"type":"v1","addr":"172.21.15.117:6827","nonce":3384003243}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":3384003243},{"type":"v1","addr":"172.21.15.117:6831","nonce":3384003243}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":3384003243},{"type":"v1","addr":"172.21.15.117:6829","nonce":3384003243}]},"public_addr":"172.21.15.117:6825/3384003243","cluster_addr":"172.21.15.117:6827/3384003243","heartbeat_back_addr":"172.21.15.117:6831/3384003243","heartbeat_front_addr":"172.21.15.117:6829/3384003243","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:57:14.799608+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:57:39.884668+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:58:03.327057+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:58:37.314531+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:59:03.667650+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:59:28.462823+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T17:00:02.968574+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T17:00:29.559190+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T17:00:55.291007+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T17:01:29.815977+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T17:01:57.015580+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T17:02:23.233151+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.45:0/4074982598":"2024-04-17T16:55:19.948001+0000","172.21.15.45:0/2443572576":"2024-04-17T16:55:19.948001+0000","172.21.15.45:6801/1983630136":"2024-04-17T16:55:19.948001+0000","172.21.15.45:6800/1983630136":"2024-04-17T16:55:19.948001+0000","172.21.15.45:0/3150465248":"2024-04-17T16:54:43.692466+0000","172.21.15.45:0/556738623":"2024-04-17T16:54:43.692466+0000","172.21.15.45:0/445676297":"2024-04-17T16:55:19.948001+0000","172.21.15.45:6800/2553659622":"2024-04-17T16:54:43.692466+0000","172.21.15.45:6801/2553659622":"2024-04-17T16:54:43.692466+0000","172.21.15.45:0/1184615408":"2024-04-17T16:54:43.692466+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-16T17:03:31.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:31.252+0000 7f46ef000640 1 -- 172.21.15.45:0/3432094455 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f46d00a3c30 msgr2=0x7f46d00a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:31.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:31.252+0000 7f46ef000640 1 --2- 172.21.15.45:0/3432094455 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f46d00a3c30 0x7f46d00a60f0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f46d402c7d0 tx=0x7f46d4005fd0 comp rx=0 tx=0).stop 2024-04-16T17:03:31.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:31.252+0000 7f46ef000640 1 -- 172.21.15.45:0/3432094455 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f46e8108480 msgr2=0x7f46e819e860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:31.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:31.252+0000 7f46ef000640 1 --2- 172.21.15.45:0/3432094455 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f46e8108480 0x7f46e819e860 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f46dc00e9c0 tx=0x7f46dc00ee80 comp rx=0 tx=0).stop 2024-04-16T17:03:31.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:31.252+0000 7f46ef000640 1 -- 172.21.15.45:0/3432094455 shutdown_connections 2024-04-16T17:03:31.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:31.252+0000 7f46ef000640 1 --2- 172.21.15.45:0/3432094455 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f46d00a3c30 0x7f46d00a60f0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:31.254 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:31.252+0000 7f46ef000640 1 --2- 172.21.15.45:0/3432094455 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f46e8109670 0x7f46e819eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:31.254 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:31.252+0000 7f46ef000640 1 --2- 172.21.15.45:0/3432094455 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f46e810a030 0x7f46e81a5dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:31.254 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:31.252+0000 7f46ef000640 1 --2- 172.21.15.45:0/3432094455 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f46e8108480 0x7f46e819e860 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:31.254 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:31.252+0000 7f46ef000640 1 -- 172.21.15.45:0/3432094455 >> 172.21.15.45:0/3432094455 conn(0x7f46e8077440 msgr2=0x7f46e810c970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:03:31.254 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:31.252+0000 7f46ef000640 1 -- 172.21.15.45:0/3432094455 shutdown_connections 2024-04-16T17:03:31.254 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:31.253+0000 7f46ef000640 1 -- 172.21.15.45:0/3432094455 wait complete. 2024-04-16T17:03:31.275 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-16T17:03:31.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:31.491+0000 7fb18f200640 1 -- 172.21.15.45:0/3899676454 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb1880785a0 msgr2=0x7fb188078e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:31.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:31.491+0000 7fb18f200640 1 --2- 172.21.15.45:0/3899676454 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb1880785a0 0x7fb188078e40 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fb178009900 tx=0x7fb17802bc30 comp rx=0 tx=0).stop 2024-04-16T17:03:31.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:31.491+0000 7fb18f200640 1 -- 172.21.15.45:0/3899676454 shutdown_connections 2024-04-16T17:03:31.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:31.491+0000 7fb18f200640 1 --2- 172.21.15.45:0/3899676454 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb188077c00 0x7fb188078060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:31.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:31.491+0000 7fb18f200640 1 --2- 172.21.15.45:0/3899676454 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb1880785a0 0x7fb188078e40 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:31.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:31.491+0000 7fb18f200640 1 --2- 172.21.15.45:0/3899676454 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb1880769c0 0x7fb188076de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:31.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:31.491+0000 7fb18f200640 1 -- 172.21.15.45:0/3899676454 >> 172.21.15.45:0/3899676454 conn(0x7fb1880ff7f0 msgr2=0x7fb188101c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:03:31.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:31.491+0000 7fb18f200640 1 -- 172.21.15.45:0/3899676454 shutdown_connections 2024-04-16T17:03:31.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:31.491+0000 7fb18f200640 1 -- 172.21.15.45:0/3899676454 wait complete. 2024-04-16T17:03:31.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:31.492+0000 7fb18f200640 1 Processor -- start 2024-04-16T17:03:31.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:31.493+0000 7fb18f200640 1 -- start start 2024-04-16T17:03:31.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:31.493+0000 7fb18f200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb1880769c0 0x7fb18819e960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:31.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:31.493+0000 7fb18f200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb188077c00 0x7fb18819eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:31.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:31.493+0000 7fb18f200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb1880785a0 0x7fb1881a5f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:31.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:31.493+0000 7fb18f200640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fb188114310 con 0x7fb188077c00 2024-04-16T17:03:31.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:31.493+0000 7fb18f200640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fb188114190 con 0x7fb1880769c0 2024-04-16T17:03:31.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:31.493+0000 7fb18f200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb188114490 con 0x7fb1880785a0 2024-04-16T17:03:31.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:31.493+0000 7fb187400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb188077c00 0x7fb18819eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:31.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:31.493+0000 7fb187400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb188077c00 0x7fb18819eea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43796/0 (socket says 172.21.15.45:43796) 2024-04-16T17:03:31.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:31.493+0000 7fb187400640 1 -- 172.21.15.45:0/4244088699 learned_addr learned my addr 172.21.15.45:0/4244088699 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:03:31.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:31.493+0000 7fb187e00640 1 --2- 172.21.15.45:0/4244088699 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb1880769c0 0x7fb18819e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:31.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:31.493+0000 7fb18d200640 1 --2- 172.21.15.45:0/4244088699 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb1880785a0 0x7fb1881a5f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:31.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:31.494+0000 7fb187400640 1 -- 172.21.15.45:0/4244088699 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb1880785a0 msgr2=0x7fb1881a5f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:31.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:31.494+0000 7fb187400640 1 --2- 172.21.15.45:0/4244088699 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb1880785a0 0x7fb1881a5f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:31.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:31.494+0000 7fb187400640 1 -- 172.21.15.45:0/4244088699 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb1880769c0 msgr2=0x7fb18819e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:31.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:31.494+0000 7fb187400640 1 --2- 172.21.15.45:0/4244088699 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb1880769c0 0x7fb18819e960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:31.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:31.494+0000 7fb187400640 1 -- 172.21.15.45:0/4244088699 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1881a6580 con 0x7fb188077c00 2024-04-16T17:03:31.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:31.494+0000 7fb187e00640 1 --2- 172.21.15.45:0/4244088699 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb1880769c0 0x7fb18819e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:03:31.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:31.494+0000 7fb187400640 1 --2- 172.21.15.45:0/4244088699 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb188077c00 0x7fb18819eea0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fb17400e9f0 tx=0x7fb17400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:03:31.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:31.494+0000 7fb184c00640 1 -- 172.21.15.45:0/4244088699 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb17400cd80 con 0x7fb188077c00 2024-04-16T17:03:31.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:31.494+0000 7fb184c00640 1 -- 172.21.15.45:0/4244088699 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb174005500 con 0x7fb188077c00 2024-04-16T17:03:31.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:31.494+0000 7fb18f200640 1 -- 172.21.15.45:0/4244088699 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1881a6870 con 0x7fb188077c00 2024-04-16T17:03:31.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:31.494+0000 7fb184c00640 1 -- 172.21.15.45:0/4244088699 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb174010690 con 0x7fb188077c00 2024-04-16T17:03:31.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:31.495+0000 7fb18f200640 1 -- 172.21.15.45:0/4244088699 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1881a6d70 con 0x7fb188077c00 2024-04-16T17:03:31.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:31.496+0000 7fb18f200640 1 -- 172.21.15.45:0/4244088699 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb188110bf0 con 0x7fb188077c00 2024-04-16T17:03:31.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:31.499+0000 7fb184c00640 1 -- 172.21.15.45:0/4244088699 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fb1740046c0 con 0x7fb188077c00 2024-04-16T17:03:31.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:31.501+0000 7fb184c00640 1 --2- 172.21.15.45:0/4244088699 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fb16c0a3bb0 0x7fb16c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:31.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:31.501+0000 7fb184c00640 1 -- 172.21.15.45:0/4244088699 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7fb1740d0ca0 con 0x7fb188077c00 2024-04-16T17:03:31.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:31.502+0000 7fb187e00640 1 --2- 172.21.15.45:0/4244088699 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fb16c0a3bb0 0x7fb16c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:31.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:31.502+0000 7fb187e00640 1 --2- 172.21.15.45:0/4244088699 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fb16c0a3bb0 0x7fb16c0a6070 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fb1800099a0 tx=0x7fb180009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:03:31.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:31.504+0000 7fb184c00640 1 -- 172.21.15.45:0/4244088699 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb174053ff0 con 0x7fb188077c00 2024-04-16T17:03:31.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:03:31 smithi117 ceph-mon[36730]: pgmap v324: 129 pgs: 30 unknown, 99 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 8 op/s 2024-04-16T17:03:31.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:03:31 smithi117 ceph-mon[36730]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-16T17:03:31.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:03:31 smithi117 ceph-mon[36730]: osdmap e90: 12 total, 12 up, 12 in 2024-04-16T17:03:31.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:03:31 smithi117 ceph-mon[36730]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-04-16T17:03:31.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:03:31 smithi117 ceph-mon[36730]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-16T17:03:31.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:03:31 smithi117 ceph-mon[36730]: from='client.? 172.21.15.45:0/1971560110' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:03:31.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:03:31 smithi104 ceph-mon[36457]: pgmap v324: 129 pgs: 30 unknown, 99 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 8 op/s 2024-04-16T17:03:31.679 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:03:31 smithi104 ceph-mon[36457]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-16T17:03:31.679 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:03:31 smithi104 ceph-mon[36457]: osdmap e90: 12 total, 12 up, 12 in 2024-04-16T17:03:31.679 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:03:31 smithi104 ceph-mon[36457]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-04-16T17:03:31.679 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:03:31 smithi104 ceph-mon[36457]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-16T17:03:31.679 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:03:31 smithi104 ceph-mon[36457]: from='client.? 172.21.15.45:0/1971560110' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:03:31.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:31.713+0000 7fb18f200640 1 -- 172.21.15.45:0/4244088699 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb18802d870 con 0x7fb188077c00 2024-04-16T17:03:31.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:31.715+0000 7fb184c00640 1 -- 172.21.15.45:0/4244088699 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+24164 (secure 0 0 0) 0x7fb17409d3e0 con 0x7fb188077c00 2024-04-16T17:03:31.716 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-16T17:03:31.716 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":90,"fsid":"b1fecd38-fc11-11ee-bc90-c7b262605968","created":"2024-04-16T16:54:02.385906+0000","modified":"2024-04-16T17:03:29.720565+0000","last_up_change":"2024-04-16T17:02:25.162624+0000","last_in_change":"2024-04-16T17:02:04.903274+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-16T16:58:06.166659+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-16T17:03:22.304198+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-16T17:03:23.616450+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-16T17:03:27.226289+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b655de2c-59f8-4fd4-ac98-0cc222229f56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":2290170881},{"type":"v1","addr":"172.21.15.45:6803","nonce":2290170881}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":2290170881},{"type":"v1","addr":"172.21.15.45:6805","nonce":2290170881}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":2290170881},{"type":"v1","addr":"172.21.15.45:6809","nonce":2290170881}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":2290170881},{"type":"v1","addr":"172.21.15.45:6807","nonce":2290170881}]},"public_addr":"172.21.15.45:6803/2290170881","cluster_addr":"172.21.15.45:6805/2290170881","heartbeat_back_addr":"172.21.15.45:6809/2290170881","heartbeat_front_addr":"172.21.15.45:6807/2290170881","state":["exists","up"]},{"osd":1,"uuid":"ebd450ec-08fd-4c3d-8b24-4e10634ddb7e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6800","nonce":1462942067},{"type":"v1","addr":"172.21.15.104:6801","nonce":1462942067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":1462942067},{"type":"v1","addr":"172.21.15.104:6803","nonce":1462942067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":1462942067},{"type":"v1","addr":"172.21.15.104:6807","nonce":1462942067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":1462942067},{"type":"v1","addr":"172.21.15.104:6805","nonce":1462942067}]},"public_addr":"172.21.15.104:6801/1462942067","cluster_addr":"172.21.15.104:6803/1462942067","heartbeat_back_addr":"172.21.15.104:6807/1462942067","heartbeat_front_addr":"172.21.15.104:6805/1462942067","state":["exists","up"]},{"osd":2,"uuid":"b318a512-4190-42a2-8104-dcc4026ad8ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":2829260735},{"type":"v1","addr":"172.21.15.117:6801","nonce":2829260735}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":2829260735},{"type":"v1","addr":"172.21.15.117:6803","nonce":2829260735}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":2829260735},{"type":"v1","addr":"172.21.15.117:6807","nonce":2829260735}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":2829260735},{"type":"v1","addr":"172.21.15.117:6805","nonce":2829260735}]},"public_addr":"172.21.15.117:6801/2829260735","cluster_addr":"172.21.15.117:6803/2829260735","heartbeat_back_addr":"172.21.15.117:6807/2829260735","heartbeat_front_addr":"172.21.15.117:6805/2829260735","state":["exists","up"]},{"osd":3,"uuid":"ec6490cb-a715-4c12-a54a-225aae26fcb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":2823712137},{"type":"v1","addr":"172.21.15.45:6811","nonce":2823712137}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":2823712137},{"type":"v1","addr":"172.21.15.45:6813","nonce":2823712137}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":2823712137},{"type":"v1","addr":"172.21.15.45:6817","nonce":2823712137}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":2823712137},{"type":"v1","addr":"172.21.15.45:6815","nonce":2823712137}]},"public_addr":"172.21.15.45:6811/2823712137","cluster_addr":"172.21.15.45:6813/2823712137","heartbeat_back_addr":"172.21.15.45:6817/2823712137","heartbeat_front_addr":"172.21.15.45:6815/2823712137","state":["exists","up"]},{"osd":4,"uuid":"5d91d568-0250-4aae-8dc2-a0e079f92062","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":1396278127},{"type":"v1","addr":"172.21.15.104:6809","nonce":1396278127}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":1396278127},{"type":"v1","addr":"172.21.15.104:6811","nonce":1396278127}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":1396278127},{"type":"v1","addr":"172.21.15.104:6815","nonce":1396278127}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":1396278127},{"type":"v1","addr":"172.21.15.104:6813","nonce":1396278127}]},"public_addr":"172.21.15.104:6809/1396278127","cluster_addr":"172.21.15.104:6811/1396278127","heartbeat_back_addr":"172.21.15.104:6815/1396278127","heartbeat_front_addr":"172.21.15.104:6813/1396278127","state":["exists","up"]},{"osd":5,"uuid":"ac510b43-6fb2-4d0f-bc5f-16ac080d9e9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":4145720047},{"type":"v1","addr":"172.21.15.117:6809","nonce":4145720047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":4145720047},{"type":"v1","addr":"172.21.15.117:6811","nonce":4145720047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":4145720047},{"type":"v1","addr":"172.21.15.117:6815","nonce":4145720047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":4145720047},{"type":"v1","addr":"172.21.15.117:6813","nonce":4145720047}]},"public_addr":"172.21.15.117:6809/4145720047","cluster_addr":"172.21.15.117:6811/4145720047","heartbeat_back_addr":"172.21.15.117:6815/4145720047","heartbeat_front_addr":"172.21.15.117:6813/4145720047","state":["exists","up"]},{"osd":6,"uuid":"a41adc18-d88a-419e-b2c2-b089c1f32472","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":3254229176},{"type":"v1","addr":"172.21.15.45:6819","nonce":3254229176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":3254229176},{"type":"v1","addr":"172.21.15.45:6821","nonce":3254229176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":3254229176},{"type":"v1","addr":"172.21.15.45:6825","nonce":3254229176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":3254229176},{"type":"v1","addr":"172.21.15.45:6823","nonce":3254229176}]},"public_addr":"172.21.15.45:6819/3254229176","cluster_addr":"172.21.15.45:6821/3254229176","heartbeat_back_addr":"172.21.15.45:6825/3254229176","heartbeat_front_addr":"172.21.15.45:6823/3254229176","state":["exists","up"]},{"osd":7,"uuid":"b71712a4-7cf6-454d-9fd1-565e3df21c1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":4151077186},{"type":"v1","addr":"172.21.15.104:6817","nonce":4151077186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":4151077186},{"type":"v1","addr":"172.21.15.104:6819","nonce":4151077186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":4151077186},{"type":"v1","addr":"172.21.15.104:6823","nonce":4151077186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":4151077186},{"type":"v1","addr":"172.21.15.104:6821","nonce":4151077186}]},"public_addr":"172.21.15.104:6817/4151077186","cluster_addr":"172.21.15.104:6819/4151077186","heartbeat_back_addr":"172.21.15.104:6823/4151077186","heartbeat_front_addr":"172.21.15.104:6821/4151077186","state":["exists","up"]},{"osd":8,"uuid":"9e572aef-59a8-450e-a92e-df9dff4d57a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":1500286830},{"type":"v1","addr":"172.21.15.117:6817","nonce":1500286830}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":1500286830},{"type":"v1","addr":"172.21.15.117:6819","nonce":1500286830}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":1500286830},{"type":"v1","addr":"172.21.15.117:6823","nonce":1500286830}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":1500286830},{"type":"v1","addr":"172.21.15.117:6821","nonce":1500286830}]},"public_addr":"172.21.15.117:6817/1500286830","cluster_addr":"172.21.15.117:6819/1500286830","heartbeat_back_addr":"172.21.15.117:6823/1500286830","heartbeat_front_addr":"172.21.15.117:6821/1500286830","state":["exists","up"]},{"osd":9,"uuid":"014e9598-c413-4cc4-9bb8-61c6e8f59d30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":3106734074},{"type":"v1","addr":"172.21.15.45:6827","nonce":3106734074}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":3106734074},{"type":"v1","addr":"172.21.15.45:6829","nonce":3106734074}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":3106734074},{"type":"v1","addr":"172.21.15.45:6833","nonce":3106734074}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":3106734074},{"type":"v1","addr":"172.21.15.45:6831","nonce":3106734074}]},"public_addr":"172.21.15.45:6827/3106734074","cluster_addr":"172.21.15.45:6829/3106734074","heartbeat_back_addr":"172.21.15.45:6833/3106734074","heartbeat_front_addr":"172.21.15.45:6831/3106734074","state":["exists","up"]},{"osd":10,"uuid":"2deba364-3e39-47be-8a1f-8256df5b82d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":1404794652},{"type":"v1","addr":"172.21.15.104:6825","nonce":1404794652}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":1404794652},{"type":"v1","addr":"172.21.15.104:6827","nonce":1404794652}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":1404794652},{"type":"v1","addr":"172.21.15.104:6831","nonce":1404794652}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":1404794652},{"type":"v1","addr":"172.21.15.104:6829","nonce":1404794652}]},"public_addr":"172.21.15.104:6825/1404794652","cluster_addr":"172.21.15.104:6827/1404794652","heartbeat_back_addr":"172.21.15.104:6831/1404794652","heartbeat_front_addr":"172.21.15.104:6829/1404794652","state":["exists","up"]},{"osd":11,"uuid":"3fda4fea-59ad-4d1a-9fe6-fcb30e31fd7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":3384003243},{"type":"v1","addr":"172.21.15.117:6825","nonce":3384003243}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":3384003243},{"type":"v1","addr":"172.21.15.117:6827","nonce":3384003243}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":3384003243},{"type":"v1","addr":"172.21.15.117:6831","nonce":3384003243}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":3384003243},{"type":"v1","addr":"172.21.15.117:6829","nonce":3384003243}]},"public_addr":"172.21.15.117:6825/3384003243","cluster_addr":"172.21.15.117:6827/3384003243","heartbeat_back_addr":"172.21.15.117:6831/3384003243","heartbeat_front_addr":"172.21.15.117:6829/3384003243","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:57:14.799608+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:57:39.884668+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:58:03.327057+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:58:37.314531+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:59:03.667650+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:59:28.462823+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T17:00:02.968574+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T17:00:29.559190+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T17:00:55.291007+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T17:01:29.815977+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T17:01:57.015580+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T17:02:23.233151+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.45:0/4074982598":"2024-04-17T16:55:19.948001+0000","172.21.15.45:0/2443572576":"2024-04-17T16:55:19.948001+0000","172.21.15.45:6801/1983630136":"2024-04-17T16:55:19.948001+0000","172.21.15.45:6800/1983630136":"2024-04-17T16:55:19.948001+0000","172.21.15.45:0/3150465248":"2024-04-17T16:54:43.692466+0000","172.21.15.45:0/556738623":"2024-04-17T16:54:43.692466+0000","172.21.15.45:0/445676297":"2024-04-17T16:55:19.948001+0000","172.21.15.45:6800/2553659622":"2024-04-17T16:54:43.692466+0000","172.21.15.45:6801/2553659622":"2024-04-17T16:54:43.692466+0000","172.21.15.45:0/1184615408":"2024-04-17T16:54:43.692466+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-16T17:03:31.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:31.718+0000 7fb18f200640 1 -- 172.21.15.45:0/4244088699 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fb16c0a3bb0 msgr2=0x7fb16c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:31.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:31.718+0000 7fb18f200640 1 --2- 172.21.15.45:0/4244088699 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fb16c0a3bb0 0x7fb16c0a6070 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fb1800099a0 tx=0x7fb180009500 comp rx=0 tx=0).stop 2024-04-16T17:03:31.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:31.718+0000 7fb18f200640 1 -- 172.21.15.45:0/4244088699 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb188077c00 msgr2=0x7fb18819eea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:31.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:31.718+0000 7fb18f200640 1 --2- 172.21.15.45:0/4244088699 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb188077c00 0x7fb18819eea0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fb17400e9f0 tx=0x7fb17400edf0 comp rx=0 tx=0).stop 2024-04-16T17:03:31.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:31.718+0000 7fb18f200640 1 -- 172.21.15.45:0/4244088699 shutdown_connections 2024-04-16T17:03:31.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:31.718+0000 7fb18f200640 1 --2- 172.21.15.45:0/4244088699 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fb16c0a3bb0 0x7fb16c0a6070 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:31.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:31.718+0000 7fb18f200640 1 --2- 172.21.15.45:0/4244088699 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb1880785a0 0x7fb1881a5f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:31.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:31.718+0000 7fb18f200640 1 --2- 172.21.15.45:0/4244088699 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb188077c00 0x7fb18819eea0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:31.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:31.718+0000 7fb18f200640 1 --2- 172.21.15.45:0/4244088699 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb1880769c0 0x7fb18819e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:31.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:31.718+0000 7fb18f200640 1 -- 172.21.15.45:0/4244088699 >> 172.21.15.45:0/4244088699 conn(0x7fb1880ff7f0 msgr2=0x7fb1881012f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:03:31.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:31.718+0000 7fb18f200640 1 -- 172.21.15.45:0/4244088699 shutdown_connections 2024-04-16T17:03:31.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:31.719+0000 7fb18f200640 1 -- 172.21.15.45:0/4244088699 wait complete. 2024-04-16T17:03:31.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:03:31 smithi045 ceph-mon[31676]: pgmap v324: 129 pgs: 30 unknown, 99 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 8 op/s 2024-04-16T17:03:31.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:03:31 smithi045 ceph-mon[31676]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-16T17:03:31.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:03:31 smithi045 ceph-mon[31676]: osdmap e90: 12 total, 12 up, 12 in 2024-04-16T17:03:31.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:03:31 smithi045 ceph-mon[31676]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-04-16T17:03:31.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:03:31 smithi045 ceph-mon[31676]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-16T17:03:31.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:03:31 smithi045 ceph-mon[31676]: from='client.? 172.21.15.45:0/1971560110' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:03:31.753 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-16T17:03:31.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:31.969+0000 7f7ce7e00640 1 -- 172.21.15.45:0/3454764051 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7ce8074a00 msgr2=0x7f7ce8074e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:31.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:31.969+0000 7f7ce7e00640 1 --2- 172.21.15.45:0/3454764051 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7ce8074a00 0x7f7ce8074e60 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f7cd8009ca0 tx=0x7f7cd802c180 comp rx=0 tx=0).stop 2024-04-16T17:03:31.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:31.969+0000 7f7ce7e00640 1 -- 172.21.15.45:0/3454764051 shutdown_connections 2024-04-16T17:03:31.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:31.969+0000 7f7ce7e00640 1 --2- 172.21.15.45:0/3454764051 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7ce8109d30 0x7f7ce810c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:31.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:31.969+0000 7f7ce7e00640 1 --2- 172.21.15.45:0/3454764051 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7ce8074a00 0x7f7ce8074e60 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:31.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:31.969+0000 7f7ce7e00640 1 --2- 172.21.15.45:0/3454764051 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7ce8075fe0 0x7f7ce8074430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:31.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:31.969+0000 7f7ce7e00640 1 -- 172.21.15.45:0/3454764051 >> 172.21.15.45:0/3454764051 conn(0x7f7ce80fd410 msgr2=0x7f7ce80ff850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:03:31.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:31.969+0000 7f7ce7e00640 1 -- 172.21.15.45:0/3454764051 shutdown_connections 2024-04-16T17:03:31.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:31.970+0000 7f7ce7e00640 1 -- 172.21.15.45:0/3454764051 wait complete. 2024-04-16T17:03:31.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:31.971+0000 7f7ce7e00640 1 Processor -- start 2024-04-16T17:03:31.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:31.971+0000 7f7ce7e00640 1 -- start start 2024-04-16T17:03:31.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:31.971+0000 7f7ce7e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7ce8074a00 0x7f7ce819a340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:31.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:31.971+0000 7f7ce7e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7ce8075fe0 0x7f7ce819a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:31.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:31.971+0000 7f7ce7e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7ce8109d30 0x7f7ce81a18f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:31.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:31.971+0000 7f7ce7e00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f7ce810fd30 con 0x7f7ce8075fe0 2024-04-16T17:03:31.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:31.971+0000 7f7ce7e00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f7ce810fbb0 con 0x7f7ce8074a00 2024-04-16T17:03:31.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:31.971+0000 7f7ce7e00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f7ce810feb0 con 0x7f7ce8109d30 2024-04-16T17:03:31.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:31.972+0000 7f7ce6a00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7ce8074a00 0x7f7ce819a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:31.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:31.972+0000 7f7ce6a00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7ce8074a00 0x7f7ce819a340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:49164/0 (socket says 172.21.15.45:49164) 2024-04-16T17:03:31.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:31.972+0000 7f7ce6a00640 1 -- 172.21.15.45:0/3954058389 learned_addr learned my addr 172.21.15.45:0/3954058389 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:03:31.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:31.972+0000 7f7ce6000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7ce8075fe0 0x7f7ce819a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:31.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:31.972+0000 7f7ce7400640 1 --2- 172.21.15.45:0/3954058389 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7ce8109d30 0x7f7ce81a18f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:31.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:31.972+0000 7f7ce6a00640 1 -- 172.21.15.45:0/3954058389 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7ce8109d30 msgr2=0x7f7ce81a18f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:31.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:31.972+0000 7f7ce6a00640 1 --2- 172.21.15.45:0/3954058389 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7ce8109d30 0x7f7ce81a18f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:31.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:31.972+0000 7f7ce6a00640 1 -- 172.21.15.45:0/3954058389 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7ce8075fe0 msgr2=0x7f7ce819a880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:31.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:31.972+0000 7f7ce6a00640 1 --2- 172.21.15.45:0/3954058389 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7ce8075fe0 0x7f7ce819a880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:31.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:31.972+0000 7f7ce6a00640 1 -- 172.21.15.45:0/3954058389 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ce81a1f60 con 0x7f7ce8074a00 2024-04-16T17:03:31.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:31.972+0000 7f7ce6a00640 1 --2- 172.21.15.45:0/3954058389 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7ce8074a00 0x7f7ce819a340 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f7cd000e9f0 tx=0x7f7cd000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:03:31.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:31.972+0000 7f7ceca00640 1 -- 172.21.15.45:0/3954058389 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cd000cd80 con 0x7f7ce8074a00 2024-04-16T17:03:31.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:31.973+0000 7f7ceca00640 1 -- 172.21.15.45:0/3954058389 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7cd0010910 con 0x7f7ce8074a00 2024-04-16T17:03:31.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:31.973+0000 7f7ceca00640 1 -- 172.21.15.45:0/3954058389 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cd0018990 con 0x7f7ce8074a00 2024-04-16T17:03:31.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:31.973+0000 7f7ce7e00640 1 -- 172.21.15.45:0/3954058389 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ce81a2250 con 0x7f7ce8074a00 2024-04-16T17:03:31.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:31.973+0000 7f7ce7e00640 1 -- 172.21.15.45:0/3954058389 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ce81a2720 con 0x7f7ce8074a00 2024-04-16T17:03:31.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:31.974+0000 7f7ce7e00640 1 -- 172.21.15.45:0/3954058389 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ce8108af0 con 0x7f7ce8074a00 2024-04-16T17:03:31.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:31.976+0000 7f7ceca00640 1 -- 172.21.15.45:0/3954058389 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f7cd0010ac0 con 0x7f7ce8074a00 2024-04-16T17:03:31.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:31.977+0000 7f7ceca00640 1 --2- 172.21.15.45:0/3954058389 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f7ccc0a3c30 0x7f7ccc0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:31.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:31.977+0000 7f7ceca00640 1 -- 172.21.15.45:0/3954058389 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f7cd00d09c0 con 0x7f7ce8074a00 2024-04-16T17:03:31.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:31.978+0000 7f7ce6000640 1 --2- 172.21.15.45:0/3954058389 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f7ccc0a3c30 0x7f7ccc0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:31.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:31.978+0000 7f7ce6000640 1 --2- 172.21.15.45:0/3954058389 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f7ccc0a3c30 0x7f7ccc0a60f0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f7cd802c790 tx=0x7f7cd8005f70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:03:31.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:31.982+0000 7f7ceca00640 1 -- 172.21.15.45:0/3954058389 <== mon.2 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7cd009d100 con 0x7f7ce8074a00 2024-04-16T17:03:32.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.235+0000 7f7ce7e00640 1 -- 172.21.15.45:0/3954058389 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7ce8062c30 con 0x7f7ce8074a00 2024-04-16T17:03:32.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.237+0000 7f7ceca00640 1 -- 172.21.15.45:0/3954058389 <== mon.2 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+11815 (secure 0 0 0) 0x7f7cd009ca00 con 0x7f7ce8074a00 2024-04-16T17:03:32.238 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-16T17:03:32.238 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/1748842443","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":1748842443},{"type":"v1","addr":"172.21.15.45:6835","nonce":1748842443}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6839/1308627051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":1308627051},{"type":"v1","addr":"172.21.15.45:6839","nonce":1308627051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/1768242489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1768242489},{"type":"v1","addr":"172.21.15.117:6833","nonce":1768242489}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6833/3558902585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":3558902585},{"type":"v1","addr":"172.21.15.104:6833","nonce":3558902585}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24478,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/1308142903","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":1308142903},{"type":"v1","addr":"172.21.15.45:6837","nonce":1308142903}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24491,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6835/3430679937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":3430679937},{"type":"v1","addr":"172.21.15.104:6835","nonce":3430679937}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24499,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/2224123841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2224123841},{"type":"v1","addr":"172.21.15.117:6835","nonce":2224123841}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24511,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/2471356815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":2471356815},{"type":"v1","addr":"172.21.15.117:6837","nonce":2471356815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24524,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6837/2647756509","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":2647756509},{"type":"v1","addr":"172.21.15.104:6837","nonce":2647756509}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24532,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6839/2378675949","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":2378675949},{"type":"v1","addr":"172.21.15.117:6839","nonce":2378675949}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24569,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6841/3755310822","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6840","nonce":3755310822},{"type":"v1","addr":"172.21.15.45:6841","nonce":3755310822}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-16T17:03:24.663611+0000","modified":"2024-04-16T17:03:29.720950+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24581},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24581":{"gid":24581,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.104:6839/3660070592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6838","nonce":3660070592},{"type":"v1","addr":"172.21.15.104:6839","nonce":3660070592}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24581,"qdb_cluster":[24581]},"id":1}]} 2024-04-16T17:03:32.238 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 17 2024-04-16T17:03:32.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.240+0000 7f7ce7e00640 1 -- 172.21.15.45:0/3954058389 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f7ccc0a3c30 msgr2=0x7f7ccc0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:32.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.240+0000 7f7ce7e00640 1 --2- 172.21.15.45:0/3954058389 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f7ccc0a3c30 0x7f7ccc0a60f0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f7cd802c790 tx=0x7f7cd8005f70 comp rx=0 tx=0).stop 2024-04-16T17:03:32.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.240+0000 7f7ce7e00640 1 -- 172.21.15.45:0/3954058389 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7ce8074a00 msgr2=0x7f7ce819a340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:32.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.240+0000 7f7ce7e00640 1 --2- 172.21.15.45:0/3954058389 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7ce8074a00 0x7f7ce819a340 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f7cd000e9f0 tx=0x7f7cd000eeb0 comp rx=0 tx=0).stop 2024-04-16T17:03:32.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.240+0000 7f7ce7e00640 1 -- 172.21.15.45:0/3954058389 shutdown_connections 2024-04-16T17:03:32.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.240+0000 7f7ce7e00640 1 --2- 172.21.15.45:0/3954058389 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f7ccc0a3c30 0x7f7ccc0a60f0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:32.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.240+0000 7f7ce7e00640 1 --2- 172.21.15.45:0/3954058389 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7ce8109d30 0x7f7ce81a18f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:32.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.240+0000 7f7ce7e00640 1 --2- 172.21.15.45:0/3954058389 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7ce8075fe0 0x7f7ce819a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:32.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.240+0000 7f7ce7e00640 1 --2- 172.21.15.45:0/3954058389 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7ce8074a00 0x7f7ce819a340 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:32.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.240+0000 7f7ce7e00640 1 -- 172.21.15.45:0/3954058389 >> 172.21.15.45:0/3954058389 conn(0x7f7ce80fd410 msgr2=0x7f7ce8105cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:03:32.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.240+0000 7f7ce7e00640 1 -- 172.21.15.45:0/3954058389 shutdown_connections 2024-04-16T17:03:32.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.240+0000 7f7ce7e00640 1 -- 172.21.15.45:0/3954058389 wait complete. 2024-04-16T17:03:32.263 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-16T17:03:32.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.480+0000 7fefcf000640 1 -- 172.21.15.45:0/3917169597 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fefc8101f70 msgr2=0x7fefc81023f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:32.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.480+0000 7fefcf000640 1 --2- 172.21.15.45:0/3917169597 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fefc8101f70 0x7fefc81023f0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fefb4009860 tx=0x7fefb402bb70 comp rx=0 tx=0).stop 2024-04-16T17:03:32.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.481+0000 7fefcf000640 1 -- 172.21.15.45:0/3917169597 shutdown_connections 2024-04-16T17:03:32.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.481+0000 7fefcf000640 1 --2- 172.21.15.45:0/3917169597 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fefc8102930 0x7fefc81091c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:32.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.481+0000 7fefcf000640 1 --2- 172.21.15.45:0/3917169597 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fefc8101f70 0x7fefc81023f0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:32.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.481+0000 7fefcf000640 1 --2- 172.21.15.45:0/3917169597 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fefc8100d80 0x7fefc81011a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:32.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.481+0000 7fefcf000640 1 -- 172.21.15.45:0/3917169597 >> 172.21.15.45:0/3917169597 conn(0x7fefc80fc310 msgr2=0x7fefc80fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:03:32.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.481+0000 7fefcf000640 1 -- 172.21.15.45:0/3917169597 shutdown_connections 2024-04-16T17:03:32.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.481+0000 7fefcf000640 1 -- 172.21.15.45:0/3917169597 wait complete. 2024-04-16T17:03:32.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.482+0000 7fefcf000640 1 Processor -- start 2024-04-16T17:03:32.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.482+0000 7fefcf000640 1 -- start start 2024-04-16T17:03:32.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.482+0000 7fefcf000640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fefc8100d80 0x7fefc819a370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:32.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.482+0000 7fefcf000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fefc8101f70 0x7fefc819a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:32.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.482+0000 7fefcf000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fefc8102930 0x7fefc81a1920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:32.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.482+0000 7fefcf000640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fefc810db20 con 0x7fefc8102930 2024-04-16T17:03:32.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.482+0000 7fefcf000640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fefc810d9a0 con 0x7fefc8100d80 2024-04-16T17:03:32.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.482+0000 7fefcf000640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fefc810dca0 con 0x7fefc8101f70 2024-04-16T17:03:32.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.482+0000 7fefcdc00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fefc8100d80 0x7fefc819a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:32.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.482+0000 7fefce600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fefc8102930 0x7fefc81a1920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:32.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.482+0000 7fefcdc00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fefc8100d80 0x7fefc819a370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:49182/0 (socket says 172.21.15.45:49182) 2024-04-16T17:03:32.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.482+0000 7fefcdc00640 1 -- 172.21.15.45:0/1295138279 learned_addr learned my addr 172.21.15.45:0/1295138279 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:03:32.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.483+0000 7fefcd200640 1 --2- 172.21.15.45:0/1295138279 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fefc8101f70 0x7fefc819a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:32.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.483+0000 7fefcdc00640 1 -- 172.21.15.45:0/1295138279 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fefc8101f70 msgr2=0x7fefc819a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:32.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.483+0000 7fefcdc00640 1 --2- 172.21.15.45:0/1295138279 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fefc8101f70 0x7fefc819a8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:32.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.483+0000 7fefcdc00640 1 -- 172.21.15.45:0/1295138279 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fefc8102930 msgr2=0x7fefc81a1920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:32.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.483+0000 7fefcdc00640 1 --2- 172.21.15.45:0/1295138279 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fefc8102930 0x7fefc81a1920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:32.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.483+0000 7fefcdc00640 1 -- 172.21.15.45:0/1295138279 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefc81a1f90 con 0x7fefc8100d80 2024-04-16T17:03:32.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.483+0000 7fefcdc00640 1 --2- 172.21.15.45:0/1295138279 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fefc8100d80 0x7fefc819a370 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fefbc00e9f0 tx=0x7fefbc00edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:03:32.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.483+0000 7fefc6000640 1 -- 172.21.15.45:0/1295138279 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefbc00ccc0 con 0x7fefc8100d80 2024-04-16T17:03:32.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.483+0000 7fefc6000640 1 -- 172.21.15.45:0/1295138279 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fefbc0035f0 con 0x7fefc8100d80 2024-04-16T17:03:32.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.483+0000 7fefc6000640 1 -- 172.21.15.45:0/1295138279 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefbc00ce60 con 0x7fefc8100d80 2024-04-16T17:03:32.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.483+0000 7fefcf000640 1 -- 172.21.15.45:0/1295138279 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fefc81a2280 con 0x7fefc8100d80 2024-04-16T17:03:32.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.484+0000 7fefcf000640 1 -- 172.21.15.45:0/1295138279 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fefc81a27f0 con 0x7fefc8100d80 2024-04-16T17:03:32.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.485+0000 7fefcf000640 1 -- 172.21.15.45:0/1295138279 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fefc81a2ab0 con 0x7fefc8100d80 2024-04-16T17:03:32.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.487+0000 7fefc6000640 1 -- 172.21.15.45:0/1295138279 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fefbc0037a0 con 0x7fefc8100d80 2024-04-16T17:03:32.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.488+0000 7fefc6000640 1 --2- 172.21.15.45:0/1295138279 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fefb00a3c30 0x7fefb00a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:32.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.488+0000 7fefc6000640 1 -- 172.21.15.45:0/1295138279 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7fefbc0d0850 con 0x7fefc8100d80 2024-04-16T17:03:32.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.488+0000 7fefcd200640 1 --2- 172.21.15.45:0/1295138279 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fefb00a3c30 0x7fefb00a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:32.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.489+0000 7fefcd200640 1 --2- 172.21.15.45:0/1295138279 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fefb00a3c30 0x7fefb00a60f0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fefb402c180 tx=0x7fefb4034ad0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:03:32.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.492+0000 7fefc6000640 1 -- 172.21.15.45:0/1295138279 <== mon.2 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fefbc09cf90 con 0x7fefc8100d80 2024-04-16T17:03:32.555 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:03:32 smithi117 ceph-mon[36730]: from='client.? 172.21.15.45:0/3432094455' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T17:03:32.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:03:32 smithi117 ceph-mon[36730]: from='client.? 172.21.15.45:0/4244088699' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T17:03:32.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:03:32 smithi117 ceph-mon[36730]: from='client.? 172.21.15.45:0/3954058389' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:03:32.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:03:32 smithi104 ceph-mon[36457]: from='client.? 172.21.15.45:0/3432094455' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T17:03:32.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:03:32 smithi104 ceph-mon[36457]: from='client.? 172.21.15.45:0/4244088699' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T17:03:32.679 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:03:32 smithi104 ceph-mon[36457]: from='client.? 172.21.15.45:0/3954058389' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:03:32.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.704+0000 7fefcf000640 1 -- 172.21.15.45:0/1295138279 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fefc81a2c80 con 0x7fefc8100d80 2024-04-16T17:03:32.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.706+0000 7fefc6000640 1 -- 172.21.15.45:0/1295138279 <== mon.2 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+24164 (secure 0 0 0) 0x7fefbc085e20 con 0x7fefc8100d80 2024-04-16T17:03:32.707 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-16T17:03:32.708 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":90,"fsid":"b1fecd38-fc11-11ee-bc90-c7b262605968","created":"2024-04-16T16:54:02.385906+0000","modified":"2024-04-16T17:03:29.720565+0000","last_up_change":"2024-04-16T17:02:25.162624+0000","last_in_change":"2024-04-16T17:02:04.903274+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-16T16:58:06.166659+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-16T17:03:22.304198+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-16T17:03:23.616450+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-16T17:03:27.226289+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b655de2c-59f8-4fd4-ac98-0cc222229f56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":2290170881},{"type":"v1","addr":"172.21.15.45:6803","nonce":2290170881}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":2290170881},{"type":"v1","addr":"172.21.15.45:6805","nonce":2290170881}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":2290170881},{"type":"v1","addr":"172.21.15.45:6809","nonce":2290170881}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":2290170881},{"type":"v1","addr":"172.21.15.45:6807","nonce":2290170881}]},"public_addr":"172.21.15.45:6803/2290170881","cluster_addr":"172.21.15.45:6805/2290170881","heartbeat_back_addr":"172.21.15.45:6809/2290170881","heartbeat_front_addr":"172.21.15.45:6807/2290170881","state":["exists","up"]},{"osd":1,"uuid":"ebd450ec-08fd-4c3d-8b24-4e10634ddb7e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6800","nonce":1462942067},{"type":"v1","addr":"172.21.15.104:6801","nonce":1462942067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":1462942067},{"type":"v1","addr":"172.21.15.104:6803","nonce":1462942067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":1462942067},{"type":"v1","addr":"172.21.15.104:6807","nonce":1462942067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":1462942067},{"type":"v1","addr":"172.21.15.104:6805","nonce":1462942067}]},"public_addr":"172.21.15.104:6801/1462942067","cluster_addr":"172.21.15.104:6803/1462942067","heartbeat_back_addr":"172.21.15.104:6807/1462942067","heartbeat_front_addr":"172.21.15.104:6805/1462942067","state":["exists","up"]},{"osd":2,"uuid":"b318a512-4190-42a2-8104-dcc4026ad8ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":2829260735},{"type":"v1","addr":"172.21.15.117:6801","nonce":2829260735}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":2829260735},{"type":"v1","addr":"172.21.15.117:6803","nonce":2829260735}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":2829260735},{"type":"v1","addr":"172.21.15.117:6807","nonce":2829260735}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":2829260735},{"type":"v1","addr":"172.21.15.117:6805","nonce":2829260735}]},"public_addr":"172.21.15.117:6801/2829260735","cluster_addr":"172.21.15.117:6803/2829260735","heartbeat_back_addr":"172.21.15.117:6807/2829260735","heartbeat_front_addr":"172.21.15.117:6805/2829260735","state":["exists","up"]},{"osd":3,"uuid":"ec6490cb-a715-4c12-a54a-225aae26fcb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":2823712137},{"type":"v1","addr":"172.21.15.45:6811","nonce":2823712137}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":2823712137},{"type":"v1","addr":"172.21.15.45:6813","nonce":2823712137}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":2823712137},{"type":"v1","addr":"172.21.15.45:6817","nonce":2823712137}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":2823712137},{"type":"v1","addr":"172.21.15.45:6815","nonce":2823712137}]},"public_addr":"172.21.15.45:6811/2823712137","cluster_addr":"172.21.15.45:6813/2823712137","heartbeat_back_addr":"172.21.15.45:6817/2823712137","heartbeat_front_addr":"172.21.15.45:6815/2823712137","state":["exists","up"]},{"osd":4,"uuid":"5d91d568-0250-4aae-8dc2-a0e079f92062","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":1396278127},{"type":"v1","addr":"172.21.15.104:6809","nonce":1396278127}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":1396278127},{"type":"v1","addr":"172.21.15.104:6811","nonce":1396278127}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":1396278127},{"type":"v1","addr":"172.21.15.104:6815","nonce":1396278127}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":1396278127},{"type":"v1","addr":"172.21.15.104:6813","nonce":1396278127}]},"public_addr":"172.21.15.104:6809/1396278127","cluster_addr":"172.21.15.104:6811/1396278127","heartbeat_back_addr":"172.21.15.104:6815/1396278127","heartbeat_front_addr":"172.21.15.104:6813/1396278127","state":["exists","up"]},{"osd":5,"uuid":"ac510b43-6fb2-4d0f-bc5f-16ac080d9e9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":4145720047},{"type":"v1","addr":"172.21.15.117:6809","nonce":4145720047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":4145720047},{"type":"v1","addr":"172.21.15.117:6811","nonce":4145720047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":4145720047},{"type":"v1","addr":"172.21.15.117:6815","nonce":4145720047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":4145720047},{"type":"v1","addr":"172.21.15.117:6813","nonce":4145720047}]},"public_addr":"172.21.15.117:6809/4145720047","cluster_addr":"172.21.15.117:6811/4145720047","heartbeat_back_addr":"172.21.15.117:6815/4145720047","heartbeat_front_addr":"172.21.15.117:6813/4145720047","state":["exists","up"]},{"osd":6,"uuid":"a41adc18-d88a-419e-b2c2-b089c1f32472","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":3254229176},{"type":"v1","addr":"172.21.15.45:6819","nonce":3254229176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":3254229176},{"type":"v1","addr":"172.21.15.45:6821","nonce":3254229176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":3254229176},{"type":"v1","addr":"172.21.15.45:6825","nonce":3254229176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":3254229176},{"type":"v1","addr":"172.21.15.45:6823","nonce":3254229176}]},"public_addr":"172.21.15.45:6819/3254229176","cluster_addr":"172.21.15.45:6821/3254229176","heartbeat_back_addr":"172.21.15.45:6825/3254229176","heartbeat_front_addr":"172.21.15.45:6823/3254229176","state":["exists","up"]},{"osd":7,"uuid":"b71712a4-7cf6-454d-9fd1-565e3df21c1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":4151077186},{"type":"v1","addr":"172.21.15.104:6817","nonce":4151077186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":4151077186},{"type":"v1","addr":"172.21.15.104:6819","nonce":4151077186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":4151077186},{"type":"v1","addr":"172.21.15.104:6823","nonce":4151077186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":4151077186},{"type":"v1","addr":"172.21.15.104:6821","nonce":4151077186}]},"public_addr":"172.21.15.104:6817/4151077186","cluster_addr":"172.21.15.104:6819/4151077186","heartbeat_back_addr":"172.21.15.104:6823/4151077186","heartbeat_front_addr":"172.21.15.104:6821/4151077186","state":["exists","up"]},{"osd":8,"uuid":"9e572aef-59a8-450e-a92e-df9dff4d57a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":1500286830},{"type":"v1","addr":"172.21.15.117:6817","nonce":1500286830}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":1500286830},{"type":"v1","addr":"172.21.15.117:6819","nonce":1500286830}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":1500286830},{"type":"v1","addr":"172.21.15.117:6823","nonce":1500286830}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":1500286830},{"type":"v1","addr":"172.21.15.117:6821","nonce":1500286830}]},"public_addr":"172.21.15.117:6817/1500286830","cluster_addr":"172.21.15.117:6819/1500286830","heartbeat_back_addr":"172.21.15.117:6823/1500286830","heartbeat_front_addr":"172.21.15.117:6821/1500286830","state":["exists","up"]},{"osd":9,"uuid":"014e9598-c413-4cc4-9bb8-61c6e8f59d30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":3106734074},{"type":"v1","addr":"172.21.15.45:6827","nonce":3106734074}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":3106734074},{"type":"v1","addr":"172.21.15.45:6829","nonce":3106734074}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":3106734074},{"type":"v1","addr":"172.21.15.45:6833","nonce":3106734074}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":3106734074},{"type":"v1","addr":"172.21.15.45:6831","nonce":3106734074}]},"public_addr":"172.21.15.45:6827/3106734074","cluster_addr":"172.21.15.45:6829/3106734074","heartbeat_back_addr":"172.21.15.45:6833/3106734074","heartbeat_front_addr":"172.21.15.45:6831/3106734074","state":["exists","up"]},{"osd":10,"uuid":"2deba364-3e39-47be-8a1f-8256df5b82d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":1404794652},{"type":"v1","addr":"172.21.15.104:6825","nonce":1404794652}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":1404794652},{"type":"v1","addr":"172.21.15.104:6827","nonce":1404794652}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":1404794652},{"type":"v1","addr":"172.21.15.104:6831","nonce":1404794652}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":1404794652},{"type":"v1","addr":"172.21.15.104:6829","nonce":1404794652}]},"public_addr":"172.21.15.104:6825/1404794652","cluster_addr":"172.21.15.104:6827/1404794652","heartbeat_back_addr":"172.21.15.104:6831/1404794652","heartbeat_front_addr":"172.21.15.104:6829/1404794652","state":["exists","up"]},{"osd":11,"uuid":"3fda4fea-59ad-4d1a-9fe6-fcb30e31fd7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":3384003243},{"type":"v1","addr":"172.21.15.117:6825","nonce":3384003243}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":3384003243},{"type":"v1","addr":"172.21.15.117:6827","nonce":3384003243}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":3384003243},{"type":"v1","addr":"172.21.15.117:6831","nonce":3384003243}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":3384003243},{"type":"v1","addr":"172.21.15.117:6829","nonce":3384003243}]},"public_addr":"172.21.15.117:6825/3384003243","cluster_addr":"172.21.15.117:6827/3384003243","heartbeat_back_addr":"172.21.15.117:6831/3384003243","heartbeat_front_addr":"172.21.15.117:6829/3384003243","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:57:14.799608+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:57:39.884668+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:58:03.327057+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:58:37.314531+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:59:03.667650+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:59:28.462823+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T17:00:02.968574+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T17:00:29.559190+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T17:00:55.291007+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T17:01:29.815977+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T17:01:57.015580+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T17:02:23.233151+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.45:0/4074982598":"2024-04-17T16:55:19.948001+0000","172.21.15.45:0/2443572576":"2024-04-17T16:55:19.948001+0000","172.21.15.45:6801/1983630136":"2024-04-17T16:55:19.948001+0000","172.21.15.45:6800/1983630136":"2024-04-17T16:55:19.948001+0000","172.21.15.45:0/3150465248":"2024-04-17T16:54:43.692466+0000","172.21.15.45:0/556738623":"2024-04-17T16:54:43.692466+0000","172.21.15.45:0/445676297":"2024-04-17T16:55:19.948001+0000","172.21.15.45:6800/2553659622":"2024-04-17T16:54:43.692466+0000","172.21.15.45:6801/2553659622":"2024-04-17T16:54:43.692466+0000","172.21.15.45:0/1184615408":"2024-04-17T16:54:43.692466+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-16T17:03:32.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.709+0000 7fefcf000640 1 -- 172.21.15.45:0/1295138279 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fefb00a3c30 msgr2=0x7fefb00a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:32.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.709+0000 7fefcf000640 1 --2- 172.21.15.45:0/1295138279 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fefb00a3c30 0x7fefb00a60f0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fefb402c180 tx=0x7fefb4034ad0 comp rx=0 tx=0).stop 2024-04-16T17:03:32.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.709+0000 7fefcf000640 1 -- 172.21.15.45:0/1295138279 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fefc8100d80 msgr2=0x7fefc819a370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:32.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.709+0000 7fefcf000640 1 --2- 172.21.15.45:0/1295138279 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fefc8100d80 0x7fefc819a370 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fefbc00e9f0 tx=0x7fefbc00edf0 comp rx=0 tx=0).stop 2024-04-16T17:03:32.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.709+0000 7fefcf000640 1 -- 172.21.15.45:0/1295138279 shutdown_connections 2024-04-16T17:03:32.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.709+0000 7fefcf000640 1 --2- 172.21.15.45:0/1295138279 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fefb00a3c30 0x7fefb00a60f0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:32.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.709+0000 7fefcf000640 1 --2- 172.21.15.45:0/1295138279 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fefc8101f70 0x7fefc819a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:32.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.709+0000 7fefcf000640 1 --2- 172.21.15.45:0/1295138279 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fefc8102930 0x7fefc81a1920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:32.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.709+0000 7fefcf000640 1 --2- 172.21.15.45:0/1295138279 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fefc8100d80 0x7fefc819a370 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:32.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.709+0000 7fefcf000640 1 -- 172.21.15.45:0/1295138279 >> 172.21.15.45:0/1295138279 conn(0x7fefc80fc310 msgr2=0x7fefc8109b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:03:32.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.709+0000 7fefcf000640 1 -- 172.21.15.45:0/1295138279 shutdown_connections 2024-04-16T17:03:32.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.709+0000 7fefcf000640 1 -- 172.21.15.45:0/1295138279 wait complete. 2024-04-16T17:03:32.726 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:03:32 smithi045 ceph-mon[31676]: from='client.? 172.21.15.45:0/3432094455' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T17:03:32.726 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:03:32 smithi045 ceph-mon[31676]: from='client.? 172.21.15.45:0/4244088699' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T17:03:32.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:03:32 smithi045 ceph-mon[31676]: from='client.? 172.21.15.45:0/3954058389' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:03:32.730 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-16T17:03:24.663611+0000', 'modified': '2024-04-16T17:03:29.720950+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 24581}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24581': {'gid': 24581, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.104:6839/3660070592', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.104:6838', 'nonce': 3660070592}, {'type': 'v1', 'addr': '172.21.15.104:6839', 'nonce': 3660070592}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1, 'qdb_leader': 24581, 'qdb_cluster': [24581]} 2024-04-16T17:03:32.730 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-16T17:03:32.730 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-16T17:03:32.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.950+0000 7f4f91a00640 1 -- 172.21.15.45:0/602658111 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4f8c104260 msgr2=0x7f4f8c104680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:32.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.950+0000 7f4f91a00640 1 --2- 172.21.15.45:0/602658111 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4f8c104260 0x7f4f8c104680 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f4f84009ca0 tx=0x7f4f8402c0f0 comp rx=0 tx=0).stop 2024-04-16T17:03:32.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.950+0000 7f4f91a00640 1 -- 172.21.15.45:0/602658111 shutdown_connections 2024-04-16T17:03:32.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.950+0000 7f4f91a00640 1 --2- 172.21.15.45:0/602658111 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4f8c105e10 0x7f4f8c10c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:32.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.950+0000 7f4f91a00640 1 --2- 172.21.15.45:0/602658111 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4f8c105450 0x7f4f8c1058d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:32.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.950+0000 7f4f91a00640 1 --2- 172.21.15.45:0/602658111 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4f8c104260 0x7f4f8c104680 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:32.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.950+0000 7f4f91a00640 1 -- 172.21.15.45:0/602658111 >> 172.21.15.45:0/602658111 conn(0x7f4f8c0ff7b0 msgr2=0x7f4f8c101c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:03:32.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.950+0000 7f4f91a00640 1 -- 172.21.15.45:0/602658111 shutdown_connections 2024-04-16T17:03:32.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.950+0000 7f4f91a00640 1 -- 172.21.15.45:0/602658111 wait complete. 2024-04-16T17:03:32.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.951+0000 7f4f91a00640 1 Processor -- start 2024-04-16T17:03:32.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.952+0000 7f4f91a00640 1 -- start start 2024-04-16T17:03:32.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.952+0000 7f4f91a00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4f8c104260 0x7f4f8c19a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:32.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.952+0000 7f4f91a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4f8c105450 0x7f4f8c19ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:32.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.952+0000 7f4f91a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4f8c105e10 0x7f4f8c1a1ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:32.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.952+0000 7f4f91a00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f4f8c10ff40 con 0x7f4f8c105e10 2024-04-16T17:03:32.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.952+0000 7f4f91a00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f4f8c10fdc0 con 0x7f4f8c104260 2024-04-16T17:03:32.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.952+0000 7f4f91a00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f4f8c1100c0 con 0x7f4f8c105450 2024-04-16T17:03:32.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.952+0000 7f4f8b400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4f8c105e10 0x7f4f8c1a1ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:32.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.952+0000 7f4f8a000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4f8c105450 0x7f4f8c19ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:32.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.952+0000 7f4f8b400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4f8c105e10 0x7f4f8c1a1ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43854/0 (socket says 172.21.15.45:43854) 2024-04-16T17:03:32.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.952+0000 7f4f8b400640 1 -- 172.21.15.45:0/2517142406 learned_addr learned my addr 172.21.15.45:0/2517142406 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:03:32.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.952+0000 7f4f8aa00640 1 --2- 172.21.15.45:0/2517142406 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4f8c104260 0x7f4f8c19a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:32.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.953+0000 7f4f8a000640 1 -- 172.21.15.45:0/2517142406 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4f8c104260 msgr2=0x7f4f8c19a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:32.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.953+0000 7f4f8a000640 1 --2- 172.21.15.45:0/2517142406 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4f8c104260 0x7f4f8c19a5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:32.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.953+0000 7f4f8a000640 1 -- 172.21.15.45:0/2517142406 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4f8c105e10 msgr2=0x7f4f8c1a1ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:32.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.953+0000 7f4f8a000640 1 --2- 172.21.15.45:0/2517142406 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4f8c105e10 0x7f4f8c1a1ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:32.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.953+0000 7f4f8a000640 1 -- 172.21.15.45:0/2517142406 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f8c1a2210 con 0x7f4f8c105450 2024-04-16T17:03:32.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.953+0000 7f4f8b400640 1 --2- 172.21.15.45:0/2517142406 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4f8c105e10 0x7f4f8c1a1ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:03:32.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.953+0000 7f4f8aa00640 1 --2- 172.21.15.45:0/2517142406 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4f8c104260 0x7f4f8c19a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:03:32.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.953+0000 7f4f8a000640 1 --2- 172.21.15.45:0/2517142406 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4f8c105450 0x7f4f8c19ab30 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f4f7400e9f0 tx=0x7f4f7400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:03:32.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.953+0000 7f4f83400640 1 -- 172.21.15.45:0/2517142406 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f7400cd80 con 0x7f4f8c105450 2024-04-16T17:03:32.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.953+0000 7f4f91a00640 1 -- 172.21.15.45:0/2517142406 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f8c1a2500 con 0x7f4f8c105450 2024-04-16T17:03:32.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.953+0000 7f4f83400640 1 -- 172.21.15.45:0/2517142406 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4f740035f0 con 0x7f4f8c105450 2024-04-16T17:03:32.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.953+0000 7f4f83400640 1 -- 172.21.15.45:0/2517142406 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f74010670 con 0x7f4f8c105450 2024-04-16T17:03:32.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.953+0000 7f4f91a00640 1 -- 172.21.15.45:0/2517142406 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f8c1a2aa0 con 0x7f4f8c105450 2024-04-16T17:03:32.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.957+0000 7f4f9463c740 1 Processor -- start 2024-04-16T17:03:32.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.957+0000 7f4f83400640 1 -- 172.21.15.45:0/2517142406 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f4f740037a0 con 0x7f4f8c105450 2024-04-16T17:03:32.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.957+0000 7f4f9463c740 1 -- start start 2024-04-16T17:03:32.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.957+0000 7f4f9463c740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5561b417f2d0 0x5561b417f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:32.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.957+0000 7f4f9463c740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5561b417fcc0 0x5561b41847c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:32.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.957+0000 7f4f9463c740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5561b4184e30 0x5561b41872c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:32.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.957+0000 7f4f9463c740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5561b40e5980 con 0x5561b4184e30 2024-04-16T17:03:32.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.957+0000 7f4f9463c740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5561b408c220 con 0x5561b417f2d0 2024-04-16T17:03:32.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.957+0000 7f4f9463c740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5561b40e1940 con 0x5561b417fcc0 2024-04-16T17:03:32.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.957+0000 7f4f8a000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5561b4184e30 0x5561b41872c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:32.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.957+0000 7f4f8aa00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5561b417fcc0 0x5561b41847c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:32.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.957+0000 7f4f8b400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5561b417f2d0 0x5561b417f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:32.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.957+0000 7f4f8b400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5561b417f2d0 0x5561b417f6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:49194/0 (socket says 172.21.15.45:49194) 2024-04-16T17:03:32.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.957+0000 7f4f8b400640 1 -- 172.21.15.45:0/1484439036 learned_addr learned my addr 172.21.15.45:0/1484439036 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:03:32.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.958+0000 7f4f8aa00640 1 -- 172.21.15.45:0/1484439036 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5561b417f2d0 msgr2=0x5561b417f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:03:32.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.958+0000 7f4f8aa00640 1 --2- 172.21.15.45:0/1484439036 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5561b417f2d0 0x5561b417f6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:32.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.958+0000 7f4f8aa00640 1 -- 172.21.15.45:0/1484439036 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5561b4184e30 msgr2=0x5561b41872c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:03:32.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.958+0000 7f4f8aa00640 1 --2- 172.21.15.45:0/1484439036 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5561b4184e30 0x5561b41872c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:32.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.958+0000 7f4f8aa00640 1 -- 172.21.15.45:0/1484439036 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5561b3df82d0 con 0x5561b417fcc0 2024-04-16T17:03:32.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.958+0000 7f4f8b400640 1 --2- 172.21.15.45:0/1484439036 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5561b417f2d0 0x5561b417f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:03:32.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.958+0000 7f4f8aa00640 1 --2- 172.21.15.45:0/1484439036 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5561b417fcc0 0x5561b41847c0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f4f84037040 tx=0x7f4f840323d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8f28dc9ecab7815a server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:03:32.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.959+0000 7f4f83400640 1 --2- 172.21.15.45:0/2517142406 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f4f700a3c00 0x7f4f700a60c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:32.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.959+0000 7f4f7be00640 1 -- 172.21.15.45:0/1484439036 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f84043070 con 0x5561b417fcc0 2024-04-16T17:03:32.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.959+0000 7f4f7be00640 1 -- 172.21.15.45:0/1484439036 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4f8402cca0 con 0x5561b417fcc0 2024-04-16T17:03:32.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.959+0000 7f4f7be00640 1 -- 172.21.15.45:0/1484439036 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f84039040 con 0x5561b417fcc0 2024-04-16T17:03:32.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.959+0000 7f4f83400640 1 -- 172.21.15.45:0/2517142406 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f4f740d0970 con 0x7f4f8c105450 2024-04-16T17:03:32.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.959+0000 7f4f8b400640 1 --2- 172.21.15.45:0/2517142406 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f4f700a3c00 0x7f4f700a60c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:32.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.959+0000 7f4f9463c740 1 -- 172.21.15.45:0/1484439036 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5561b417fcc0 msgr2=0x5561b41847c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:32.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.959+0000 7f4f9463c740 1 --2- 172.21.15.45:0/1484439036 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5561b417fcc0 0x5561b41847c0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f4f84037040 tx=0x7f4f840323d0 comp rx=0 tx=0).stop 2024-04-16T17:03:32.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.960+0000 7f4f9463c740 1 -- 172.21.15.45:0/1484439036 shutdown_connections 2024-04-16T17:03:32.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.960+0000 7f4f9463c740 1 --2- 172.21.15.45:0/1484439036 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5561b417fcc0 0x5561b41847c0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:32.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.960+0000 7f4f9463c740 1 --2- 172.21.15.45:0/1484439036 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5561b4184e30 0x5561b41872c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:32.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.960+0000 7f4f9463c740 1 --2- 172.21.15.45:0/1484439036 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5561b417f2d0 0x5561b417f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:32.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.960+0000 7f4f9463c740 1 -- 172.21.15.45:0/1484439036 >> 172.21.15.45:0/1484439036 conn(0x5561b417a840 msgr2=0x5561b417ac40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:03:32.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.960+0000 7f4f8b400640 1 --2- 172.21.15.45:0/2517142406 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f4f700a3c00 0x7f4f700a60c0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f4f7c009fd0 tx=0x7f4f7c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:03:32.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.960+0000 7f4f9463c740 1 -- 172.21.15.45:0/1484439036 shutdown_connections 2024-04-16T17:03:32.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.960+0000 7f4f9463c740 1 -- 172.21.15.45:0/1484439036 wait complete. 2024-04-16T17:03:32.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.960+0000 7f4f9463c740 1 -- start start 2024-04-16T17:03:32.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.960+0000 7f4f9463c740 1 Processor -- start 2024-04-16T17:03:32.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.961+0000 7f4f9463c740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5561b417f2d0 0x5561b4189850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:32.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.961+0000 7f4f9463c740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5561b417fcc0 0x5561b4189d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:32.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.961+0000 7f4f9463c740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5561b4184e30 0x5561b4190d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:32.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.961+0000 7f4f9463c740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5561b4187d80 con 0x5561b417f2d0 2024-04-16T17:03:32.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.961+0000 7f4f9463c740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5561b409f590 con 0x5561b4184e30 2024-04-16T17:03:32.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.961+0000 7f4f9463c740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5561b4187f00 con 0x5561b417fcc0 2024-04-16T17:03:32.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.961+0000 7f4f9463c740 10 client.0 _ll_register_callbacks cb 0x5561b40a7050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:03:32.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.961+0000 7f4f8b400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5561b4184e30 0x5561b4190d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:32.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.961+0000 7f4f8a000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5561b417fcc0 0x5561b4189d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:32.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.961+0000 7f4f8aa00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5561b417f2d0 0x5561b4189850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:32.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.961+0000 7f4f8a000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5561b417fcc0 0x5561b4189d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:46590/0 (socket says 172.21.15.45:46590) 2024-04-16T17:03:32.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.961+0000 7f4f8a000640 1 -- 172.21.15.45:0/3408342068 learned_addr learned my addr 172.21.15.45:0/3408342068 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:03:32.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.962+0000 7f4f8a000640 1 -- 172.21.15.45:0/3408342068 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5561b4184e30 msgr2=0x5561b4190d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:03:32.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.962+0000 7f4f8a000640 1 --2- 172.21.15.45:0/3408342068 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5561b4184e30 0x5561b4190d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:32.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.962+0000 7f4f8a000640 1 -- 172.21.15.45:0/3408342068 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5561b417f2d0 msgr2=0x5561b4189850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:03:32.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.962+0000 7f4f8a000640 1 --2- 172.21.15.45:0/3408342068 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5561b417f2d0 0x5561b4189850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:32.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.962+0000 7f4f8a000640 1 -- 172.21.15.45:0/3408342068 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5561b3ec26b0 con 0x5561b417fcc0 2024-04-16T17:03:32.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.962+0000 7f4f8a000640 1 --2- 172.21.15.45:0/3408342068 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5561b417fcc0 0x5561b4189d90 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f4f740a50f0 tx=0x7f4f74098620 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cb322eb0ea697315 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:03:32.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.962+0000 7f4f78c00640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-04-16T17:03:32.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.962+0000 7f4f78c00640 1 -- 172.21.15.45:0/3408342068 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f7409c310 con 0x5561b417fcc0 2024-04-16T17:03:32.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.962+0000 7f4f78c00640 1 -- 172.21.15.45:0/3408342068 <== mon.1 v2:172.21.15.117:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f4f74099020 con 0x5561b417fcc0 2024-04-16T17:03:32.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.962+0000 7f4f9463c740 1 -- 172.21.15.45:0/3408342068 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4f8402cca0 con 0x5561b417fcc0 2024-04-16T17:03:32.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.963+0000 7f4f78c00640 1 -- 172.21.15.45:0/3408342068 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7f4f7409d0b0 con 0x5561b417fcc0 2024-04-16T17:03:32.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.963+0000 7f4f9463c740 10 client.24685 fetch_fsmap learned FSMap version 17 2024-04-16T17:03:32.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.963+0000 7f4f9463c740 1 -- 172.21.15.45:0/3408342068 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) -- 0x5561b4194190 con 0x5561b417fcc0 2024-04-16T17:03:32.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.963+0000 7f4f78c00640 1 -- 172.21.15.45:0/3408342068 <== mon.1 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4f8402cca0 con 0x5561b417fcc0 2024-04-16T17:03:32.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.963+0000 7f4f78c00640 1 -- 172.21.15.45:0/3408342068 <== mon.1 v2:172.21.15.117:3300/0 5 ==== fsmap(e 17) ==== 8342+0+0 (secure 0 0 0) 0x7f4f740ba020 con 0x5561b417fcc0 2024-04-16T17:03:32.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.963+0000 7f4f9463c740 10 client.24685 fetch_fsmap finished waiting for FSMap version 17 2024-04-16T17:03:32.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.963+0000 7f4f9463c740 10 client.24685 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-16T17:03:32.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.963+0000 7f4f9463c740 20 client.24685 populate_metadata read hostname 'smithi045' 2024-04-16T17:03:32.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.963+0000 7f4f9463c740 1 --2- 172.21.15.45:0/3408342068 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x5561b41967f0 0x5561b4197c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:32.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.963+0000 7f4f9463c740 4 client.24685 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:03:32.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.963+0000 7f4f9463c740 1 -- 172.21.15.45:0/3408342068 --> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5561b40f33a0 con 0x5561b41967f0 2024-04-16T17:03:32.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.964+0000 7f4f8aa00640 1 --2- 172.21.15.45:0/3408342068 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x5561b41967f0 0x5561b4197c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:32.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.964+0000 7f4f78c00640 10 client.24685 ms_handle_connect on v2:172.21.15.104:6838/3660070592 2024-04-16T17:03:32.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.964+0000 7f4f8aa00640 1 --2- 172.21.15.45:0/3408342068 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x5561b41967f0 0x5561b4197c60 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b4f65acddfb0705c server_cookie=1c504dc34701e3b5 in_seq=0 out_seq=0 2024-04-16T17:03:32.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.965+0000 7f4f78c00640 1 -- 172.21.15.45:0/3408342068 <== mds.0 v2:172.21.15.104:6838/3660070592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5561b40f33a0 con 0x5561b41967f0 2024-04-16T17:03:32.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.965+0000 7f4f78c00640 10 client.24685 handle_command_reply: tid=0 2024-04-16T17:03:32.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.965+0000 7f4f9463c740 1 client.24685 shutdown 2024-04-16T17:03:32.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.965+0000 7f4f9463c740 1 -- 172.21.15.45:0/3408342068 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5561b417fcc0 msgr2=0x5561b4189d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:32.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.965+0000 7f4f9463c740 1 --2- 172.21.15.45:0/3408342068 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5561b417fcc0 0x5561b4189d90 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f4f740a50f0 tx=0x7f4f74098620 comp rx=0 tx=0).stop 2024-04-16T17:03:32.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.966+0000 7f4f9463c740 1 -- 172.21.15.45:0/3408342068 shutdown_connections 2024-04-16T17:03:32.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.966+0000 7f4f9463c740 1 --2- 172.21.15.45:0/3408342068 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x5561b41967f0 0x5561b4197c60 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:32.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.966+0000 7f4f9463c740 1 --2- 172.21.15.45:0/3408342068 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5561b417fcc0 0x5561b4189d90 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:32.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.966+0000 7f4f9463c740 1 --2- 172.21.15.45:0/3408342068 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5561b4184e30 0x5561b4190d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:32.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.966+0000 7f4f9463c740 1 --2- 172.21.15.45:0/3408342068 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5561b417f2d0 0x5561b4189850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:32.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.966+0000 7f4f78c00640 0 client.24685 ms_handle_reset on v2:172.21.15.104:6838/3660070592 2024-04-16T17:03:32.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.966+0000 7f4f9463c740 1 -- 172.21.15.45:0/3408342068 >> 172.21.15.45:0/3408342068 conn(0x5561b417a840 msgr2=0x5561b4179a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:03:32.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.966+0000 7f4f9463c740 1 -- 172.21.15.45:0/3408342068 shutdown_connections 2024-04-16T17:03:32.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.966+0000 7f4f9463c740 1 -- 172.21.15.45:0/3408342068 wait complete. 2024-04-16T17:03:32.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.966+0000 7f4f9463c740 20 client.24685 trim_cache size 0 max 16384 2024-04-16T17:03:32.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.990+0000 7f4f9463c740 1 Processor -- start 2024-04-16T17:03:32.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.990+0000 7f4f9463c740 1 -- start start 2024-04-16T17:03:32.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.990+0000 7f4f9463c740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5561b417f2d0 0x5561b417ac20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:32.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.990+0000 7f4f9463c740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5561b417fcc0 0x5561b4190d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:32.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.990+0000 7f4f9463c740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5561b4184e30 0x5561b41791f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:32.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.990+0000 7f4f9463c740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5561b4195fa0 con 0x5561b417fcc0 2024-04-16T17:03:32.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.990+0000 7f4f9463c740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5561b4196670 con 0x5561b4184e30 2024-04-16T17:03:32.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.990+0000 7f4f9463c740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5561b4196d40 con 0x5561b417f2d0 2024-04-16T17:03:32.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.991+0000 7f4f8aa00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5561b417f2d0 0x5561b417ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:32.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.991+0000 7f4f8aa00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5561b417f2d0 0x5561b417ac20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:46604/0 (socket says 172.21.15.45:46604) 2024-04-16T17:03:32.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.991+0000 7f4f8aa00640 1 -- 172.21.15.45:0/2427445313 learned_addr learned my addr 172.21.15.45:0/2427445313 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:03:32.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.991+0000 7f4f8b400640 1 --2- 172.21.15.45:0/2427445313 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5561b4184e30 0x5561b41791f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:32.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.991+0000 7f4f8a000640 1 --2- 172.21.15.45:0/2427445313 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5561b417fcc0 0x5561b4190d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:32.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.991+0000 7f4f8aa00640 1 -- 172.21.15.45:0/2427445313 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5561b4184e30 msgr2=0x5561b41791f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:03:32.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.991+0000 7f4f8aa00640 1 --2- 172.21.15.45:0/2427445313 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5561b4184e30 0x5561b41791f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:32.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.991+0000 7f4f8aa00640 1 -- 172.21.15.45:0/2427445313 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5561b417fcc0 msgr2=0x5561b4190d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:03:32.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.991+0000 7f4f8aa00640 1 --2- 172.21.15.45:0/2427445313 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5561b417fcc0 0x5561b4190d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:32.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.991+0000 7f4f8aa00640 1 -- 172.21.15.45:0/2427445313 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5561b417b5b0 con 0x5561b417f2d0 2024-04-16T17:03:32.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.992+0000 7f4f8aa00640 1 --2- 172.21.15.45:0/2427445313 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5561b417f2d0 0x5561b417ac20 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x5561b4180730 tx=0x7f4f84009800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2a595cc7784f96e8 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:03:32.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.992+0000 7f4f7b400640 1 -- 172.21.15.45:0/2427445313 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f8402de60 con 0x5561b417f2d0 2024-04-16T17:03:32.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.992+0000 7f4f7b400640 1 -- 172.21.15.45:0/2427445313 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4f84052bd0 con 0x5561b417f2d0 2024-04-16T17:03:32.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.992+0000 7f4f7b400640 1 -- 172.21.15.45:0/2427445313 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f84039040 con 0x5561b417f2d0 2024-04-16T17:03:32.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.993+0000 7f4f9463c740 1 -- 172.21.15.45:0/2427445313 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5561b417f2d0 msgr2=0x5561b417ac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:32.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.993+0000 7f4f9463c740 1 --2- 172.21.15.45:0/2427445313 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5561b417f2d0 0x5561b417ac20 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x5561b4180730 tx=0x7f4f84009800 comp rx=0 tx=0).stop 2024-04-16T17:03:32.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.993+0000 7f4f9463c740 1 -- 172.21.15.45:0/2427445313 shutdown_connections 2024-04-16T17:03:32.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.993+0000 7f4f9463c740 1 --2- 172.21.15.45:0/2427445313 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5561b4184e30 0x5561b41791f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:32.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.993+0000 7f4f9463c740 1 --2- 172.21.15.45:0/2427445313 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5561b417fcc0 0x5561b4190d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:32.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.993+0000 7f4f9463c740 1 --2- 172.21.15.45:0/2427445313 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5561b417f2d0 0x5561b417ac20 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:32.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.993+0000 7f4f9463c740 1 -- 172.21.15.45:0/2427445313 >> 172.21.15.45:0/2427445313 conn(0x5561b417a840 msgr2=0x5561b41882d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:03:32.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.993+0000 7f4f9463c740 1 -- 172.21.15.45:0/2427445313 shutdown_connections 2024-04-16T17:03:32.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.993+0000 7f4f9463c740 1 -- 172.21.15.45:0/2427445313 wait complete. 2024-04-16T17:03:32.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.994+0000 7f4f9463c740 1 -- start start 2024-04-16T17:03:32.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.994+0000 7f4f9463c740 1 Processor -- start 2024-04-16T17:03:32.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.995+0000 7f4f9463c740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5561b417f2d0 0x5561b418c370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:32.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.995+0000 7f4f9463c740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5561b417fcc0 0x5561b418c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:32.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.995+0000 7f4f9463c740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5561b4184e30 0x5561b418f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:32.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.995+0000 7f4f9463c740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5561b4197ae0 con 0x5561b4184e30 2024-04-16T17:03:32.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.995+0000 7f4f9463c740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5561b4197410 con 0x5561b417f2d0 2024-04-16T17:03:32.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.995+0000 7f4f9463c740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5561b41981b0 con 0x5561b417fcc0 2024-04-16T17:03:32.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.995+0000 7f4f9463c740 10 client.0 _ll_register_callbacks cb 0x7f4f740d6220 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:03:32.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.995+0000 7f4f8aa00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5561b417f2d0 0x5561b418c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:33.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.995+0000 7f4f8aa00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5561b417f2d0 0x5561b418c370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:49224/0 (socket says 172.21.15.45:49224) 2024-04-16T17:03:33.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.995+0000 7f4f8aa00640 1 -- 172.21.15.45:0/1135008969 learned_addr learned my addr 172.21.15.45:0/1135008969 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:03:33.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.995+0000 7f4f8b400640 1 --2- 172.21.15.45:0/1135008969 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5561b4184e30 0x5561b418f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:33.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.995+0000 7f4f8a000640 1 --2- 172.21.15.45:0/1135008969 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5561b417fcc0 0x5561b418c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:33.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.995+0000 7f4f8aa00640 1 -- 172.21.15.45:0/1135008969 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5561b417fcc0 msgr2=0x5561b418c8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:03:33.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.995+0000 7f4f8aa00640 1 --2- 172.21.15.45:0/1135008969 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5561b417fcc0 0x5561b418c8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:33.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.995+0000 7f4f8aa00640 1 -- 172.21.15.45:0/1135008969 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5561b4184e30 msgr2=0x5561b418f8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:03:33.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.995+0000 7f4f8aa00640 1 --2- 172.21.15.45:0/1135008969 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5561b4184e30 0x5561b418f8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:33.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.995+0000 7f4f8aa00640 1 -- 172.21.15.45:0/1135008969 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5561b418fec0 con 0x5561b417f2d0 2024-04-16T17:03:33.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.996+0000 7f4f8aa00640 1 --2- 172.21.15.45:0/1135008969 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5561b417f2d0 0x5561b418c370 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f4f84009ca0 tx=0x7f4f84009540 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2ddb627b04d01ae4 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:03:33.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.996+0000 7f4f78c00640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-04-16T17:03:33.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.996+0000 7f4f78c00640 1 -- 172.21.15.45:0/1135008969 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f84039040 con 0x5561b417f2d0 2024-04-16T17:03:33.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.996+0000 7f4f78c00640 1 -- 172.21.15.45:0/1135008969 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f4f84032c90 con 0x5561b417f2d0 2024-04-16T17:03:33.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.996+0000 7f4f9463c740 1 -- 172.21.15.45:0/1135008969 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4f84052bd0 con 0x5561b417f2d0 2024-04-16T17:03:33.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.996+0000 7f4f78c00640 1 -- 172.21.15.45:0/1135008969 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7f4f8405e050 con 0x5561b417f2d0 2024-04-16T17:03:33.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.996+0000 7f4f9463c740 10 client.24695 fetch_fsmap learned FSMap version 17 2024-04-16T17:03:33.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.996+0000 7f4f9463c740 1 -- 172.21.15.45:0/1135008969 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) -- 0x5561b41fc7a0 con 0x5561b417f2d0 2024-04-16T17:03:33.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.996+0000 7f4f78c00640 1 -- 172.21.15.45:0/1135008969 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4f84052bd0 con 0x5561b417f2d0 2024-04-16T17:03:33.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.997+0000 7f4f78c00640 1 -- 172.21.15.45:0/1135008969 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 17) ==== 8342+0+0 (secure 0 0 0) 0x7f4f84004040 con 0x5561b417f2d0 2024-04-16T17:03:33.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.997+0000 7f4f9463c740 10 client.24695 fetch_fsmap finished waiting for FSMap version 17 2024-04-16T17:03:33.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.997+0000 7f4f9463c740 10 client.24695 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-16T17:03:33.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.997+0000 7f4f9463c740 20 client.24695 populate_metadata read hostname 'smithi045' 2024-04-16T17:03:33.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.997+0000 7f4f9463c740 1 --2- 172.21.15.45:0/1135008969 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x5561b4200c30 0x5561b41ff940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:33.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.997+0000 7f4f9463c740 4 client.24695 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-16T17:03:33.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.997+0000 7f4f9463c740 1 -- 172.21.15.45:0/1135008969 --> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] -- command(tid 0: {"prefix": "status"}) -- 0x5561b40ed210 con 0x5561b4200c30 2024-04-16T17:03:33.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.997+0000 7f4f8a000640 1 --2- 172.21.15.45:0/1135008969 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x5561b4200c30 0x5561b41ff940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:33.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.998+0000 7f4f8a000640 1 --2- 172.21.15.45:0/1135008969 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x5561b4200c30 0x5561b41ff940 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=697f674db0b2395c server_cookie=a0d2767d2f7ae949 in_seq=0 out_seq=0 2024-04-16T17:03:33.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.998+0000 7f4f78c00640 10 client.24695 ms_handle_connect on v2:172.21.15.104:6838/3660070592 2024-04-16T17:03:33.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.998+0000 7f4f78c00640 1 -- 172.21.15.45:0/1135008969 <== mds.0 v2:172.21.15.104:6838/3660070592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x5561b40ed210 con 0x5561b4200c30 2024-04-16T17:03:33.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.998+0000 7f4f78c00640 10 client.24695 handle_command_reply: tid=0 2024-04-16T17:03:33.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.998+0000 7f4f9463c740 1 client.24695 shutdown 2024-04-16T17:03:33.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.998+0000 7f4f9463c740 1 -- 172.21.15.45:0/1135008969 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5561b417f2d0 msgr2=0x5561b418c370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:33.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.998+0000 7f4f9463c740 1 --2- 172.21.15.45:0/1135008969 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5561b417f2d0 0x5561b418c370 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f4f84009ca0 tx=0x7f4f84009540 comp rx=0 tx=0).stop 2024-04-16T17:03:33.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.998+0000 7f4f9463c740 1 -- 172.21.15.45:0/1135008969 shutdown_connections 2024-04-16T17:03:33.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.998+0000 7f4f9463c740 1 --2- 172.21.15.45:0/1135008969 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x5561b4200c30 0x5561b41ff940 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:33.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.998+0000 7f4f9463c740 1 --2- 172.21.15.45:0/1135008969 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5561b417fcc0 0x5561b418c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:33.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.998+0000 7f4f78c00640 0 client.24695 ms_handle_reset on v2:172.21.15.104:6838/3660070592 2024-04-16T17:03:33.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.998+0000 7f4f9463c740 1 --2- 172.21.15.45:0/1135008969 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5561b4184e30 0x5561b418f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:33.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.998+0000 7f4f9463c740 1 --2- 172.21.15.45:0/1135008969 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5561b417f2d0 0x5561b418c370 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:33.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.998+0000 7f4f9463c740 1 -- 172.21.15.45:0/1135008969 >> 172.21.15.45:0/1135008969 conn(0x5561b417a840 msgr2=0x5561b417ac20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:03:33.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.998+0000 7f4f9463c740 1 -- 172.21.15.45:0/1135008969 shutdown_connections 2024-04-16T17:03:33.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.998+0000 7f4f9463c740 1 -- 172.21.15.45:0/1135008969 wait complete. 2024-04-16T17:03:33.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.999+0000 7f4f9463c740 20 client.24695 trim_cache size 0 max 16384 2024-04-16T17:03:33.007 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-16T17:03:33.007 INFO:teuthology.orchestra.run.smithi045.stdout: "cluster_fsid": "b1fecd38-fc11-11ee-bc90-c7b262605968", 2024-04-16T17:03:33.007 INFO:teuthology.orchestra.run.smithi045.stdout: "whoami": 0, 2024-04-16T17:03:33.007 INFO:teuthology.orchestra.run.smithi045.stdout: "id": 24581, 2024-04-16T17:03:33.007 INFO:teuthology.orchestra.run.smithi045.stdout: "want_state": "up:active", 2024-04-16T17:03:33.007 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "up:active", 2024-04-16T17:03:33.007 INFO:teuthology.orchestra.run.smithi045.stdout: "fs_name": "cephfs", 2024-04-16T17:03:33.007 INFO:teuthology.orchestra.run.smithi045.stdout: "rank_uptime": 7.8339624829999996, 2024-04-16T17:03:33.007 INFO:teuthology.orchestra.run.smithi045.stdout: "mdsmap_epoch": 17, 2024-04-16T17:03:33.007 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_epoch": 90, 2024-04-16T17:03:33.007 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_epoch_barrier": 90, 2024-04-16T17:03:33.007 INFO:teuthology.orchestra.run.smithi045.stdout: "uptime": 14.499058376000001 2024-04-16T17:03:33.007 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-16T17:03:33.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.999+0000 7f4f91a00640 1 -- 172.21.15.45:0/2517142406 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f4f700a3c00 msgr2=0x7f4f700a60c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:33.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.999+0000 7f4f91a00640 1 --2- 172.21.15.45:0/2517142406 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f4f700a3c00 0x7f4f700a60c0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f4f7c009fd0 tx=0x7f4f7c005f90 comp rx=0 tx=0).stop 2024-04-16T17:03:33.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.999+0000 7f4f91a00640 1 -- 172.21.15.45:0/2517142406 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4f8c105450 msgr2=0x7f4f8c19ab30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:33.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:32.999+0000 7f4f91a00640 1 --2- 172.21.15.45:0/2517142406 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4f8c105450 0x7f4f8c19ab30 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f4f7400e9f0 tx=0x7f4f7400eeb0 comp rx=0 tx=0).stop 2024-04-16T17:03:33.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:33.000+0000 7f4f91a00640 1 -- 172.21.15.45:0/2517142406 shutdown_connections 2024-04-16T17:03:33.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:33.000+0000 7f4f91a00640 1 --2- 172.21.15.45:0/2517142406 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f4f700a3c00 0x7f4f700a60c0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:33.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:33.000+0000 7f4f91a00640 1 --2- 172.21.15.45:0/2517142406 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4f8c105450 0x7f4f8c19ab30 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:33.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:33.000+0000 7f4f91a00640 1 --2- 172.21.15.45:0/2517142406 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4f8c105e10 0x7f4f8c1a1ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:33.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:33.000+0000 7f4f91a00640 1 --2- 172.21.15.45:0/2517142406 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4f8c104260 0x7f4f8c19a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:33.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:33.000+0000 7f4f91a00640 1 -- 172.21.15.45:0/2517142406 >> 172.21.15.45:0/2517142406 conn(0x7f4f8c0ff7b0 msgr2=0x7f4f8c1087a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:03:33.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:33.000+0000 7f4f91a00640 1 -- 172.21.15.45:0/2517142406 shutdown_connections 2024-04-16T17:03:33.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:33.000+0000 7f4f91a00640 1 -- 172.21.15.45:0/2517142406 wait complete. 2024-04-16T17:03:33.017 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-16T17:03:33.017 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-04-16T17:03:33.018 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45 shell --fsid b1fecd38-fc11-11ee-bc90-c7b262605968 -- ceph fs ls 2024-04-16T17:03:33.454 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/mon.a/config 2024-04-16T17:03:33.478 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:03:33 smithi045 ceph-mon[31676]: pgmap v326: 129 pgs: 9 unknown, 120 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s wr, 2 op/s 2024-04-16T17:03:33.478 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:03:33 smithi045 ceph-mon[31676]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-16T17:03:33.478 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:03:33 smithi045 ceph-mon[31676]: Cluster is now healthy 2024-04-16T17:03:33.478 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:03:33 smithi045 ceph-mon[31676]: from='client.? 172.21.15.45:0/1295138279' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T17:03:33.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:03:33 smithi117 ceph-mon[36730]: pgmap v326: 129 pgs: 9 unknown, 120 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s wr, 2 op/s 2024-04-16T17:03:33.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:03:33 smithi117 ceph-mon[36730]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-16T17:03:33.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:03:33 smithi117 ceph-mon[36730]: Cluster is now healthy 2024-04-16T17:03:33.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:03:33 smithi117 ceph-mon[36730]: from='client.? 172.21.15.45:0/1295138279' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T17:03:33.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:03:33 smithi104 ceph-mon[36457]: pgmap v326: 129 pgs: 9 unknown, 120 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s wr, 2 op/s 2024-04-16T17:03:33.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:03:33 smithi104 ceph-mon[36457]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-16T17:03:33.679 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:03:33 smithi104 ceph-mon[36457]: Cluster is now healthy 2024-04-16T17:03:33.679 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:03:33 smithi104 ceph-mon[36457]: from='client.? 172.21.15.45:0/1295138279' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T17:03:35.535 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:03:35 smithi045 ceph-mon[31676]: pgmap v327: 129 pgs: 129 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 1 op/s 2024-04-16T17:03:35.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:03:35 smithi117 ceph-mon[36730]: pgmap v327: 129 pgs: 129 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 1 op/s 2024-04-16T17:03:35.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:03:35 smithi104 ceph-mon[36457]: pgmap v327: 129 pgs: 129 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 1 op/s 2024-04-16T17:03:35.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:35.837+0000 7f3cb6000700 1 -- 172.21.15.45:0/3322909394 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3cb006ba30 msgr2=0x7f3cb010f9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:35.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:35.837+0000 7f3cb6000700 1 --2- 172.21.15.45:0/3322909394 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3cb006ba30 0x7f3cb010f9a0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f3ca8009f90 tx=0x7f3ca800b330 comp rx=0 tx=0).stop 2024-04-16T17:03:35.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:35.837+0000 7f3cb6000700 1 -- 172.21.15.45:0/3322909394 shutdown_connections 2024-04-16T17:03:35.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:35.837+0000 7f3cb6000700 1 --2- 172.21.15.45:0/3322909394 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3cb0108b60 0x7f3cb0108f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:35.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:35.837+0000 7f3cb6000700 1 --2- 172.21.15.45:0/3322909394 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3cb006ba30 0x7f3cb010f9a0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:35.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:35.837+0000 7f3cb6000700 1 --2- 172.21.15.45:0/3322909394 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3cb006b090 0x7f3cb006b4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:35.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:35.837+0000 7f3cb6000700 1 -- 172.21.15.45:0/3322909394 >> 172.21.15.45:0/3322909394 conn(0x7f3cb0078960 msgr2=0x7f3cb0078d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:03:35.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:35.838+0000 7f3cb6000700 1 -- 172.21.15.45:0/3322909394 shutdown_connections 2024-04-16T17:03:35.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:35.838+0000 7f3cb6000700 1 -- 172.21.15.45:0/3322909394 wait complete. 2024-04-16T17:03:35.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:35.839+0000 7f3cb6000700 1 Processor -- start 2024-04-16T17:03:35.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:35.839+0000 7f3cb6000700 1 -- start start 2024-04-16T17:03:35.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:35.840+0000 7f3cb6000700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3cb006b090 0x7f3cb0101ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:35.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:35.840+0000 7f3cb6000700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3cb006ba30 0x7f3cb0101ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:35.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:35.840+0000 7f3cb6000700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3cb0108b60 0x7f3cb0102530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:35.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:35.840+0000 7f3cb6000700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f3cb0112db0 con 0x7f3cb006ba30 2024-04-16T17:03:35.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:35.840+0000 7f3cb6000700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f3cb0112c30 con 0x7f3cb0108b60 2024-04-16T17:03:35.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:35.840+0000 7f3cb6000700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f3cb0112f30 con 0x7f3cb006b090 2024-04-16T17:03:35.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:35.840+0000 7f3caea00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3cb006ba30 0x7f3cb0101ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:35.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:35.840+0000 7f3caea00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3cb006ba30 0x7f3cb0101ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43902/0 (socket says 172.21.15.45:43902) 2024-04-16T17:03:35.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:35.840+0000 7f3caea00700 1 -- 172.21.15.45:0/1647253035 learned_addr learned my addr 172.21.15.45:0/1647253035 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:03:35.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:35.840+0000 7f3cafe00700 1 --2- 172.21.15.45:0/1647253035 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3cb0108b60 0x7f3cb0102530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:35.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:35.840+0000 7f3caf400700 1 --2- 172.21.15.45:0/1647253035 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3cb006b090 0x7f3cb0101ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:35.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:35.840+0000 7f3cafe00700 1 -- 172.21.15.45:0/1647253035 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3cb006b090 msgr2=0x7f3cb0101ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:35.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:35.840+0000 7f3cafe00700 1 --2- 172.21.15.45:0/1647253035 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3cb006b090 0x7f3cb0101ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:35.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:35.840+0000 7f3cafe00700 1 -- 172.21.15.45:0/1647253035 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3cb006ba30 msgr2=0x7f3cb0101ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:35.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:35.840+0000 7f3cafe00700 1 --2- 172.21.15.45:0/1647253035 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3cb006ba30 0x7f3cb0101ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:35.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:35.841+0000 7f3cafe00700 1 -- 172.21.15.45:0/1647253035 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3cb01a7770 con 0x7f3cb0108b60 2024-04-16T17:03:35.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:35.841+0000 7f3caea00700 1 --2- 172.21.15.45:0/1647253035 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3cb006ba30 0x7f3cb0101ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:03:35.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:35.841+0000 7f3cafe00700 1 --2- 172.21.15.45:0/1647253035 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3cb0108b60 0x7f3cb0102530 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f3ca8015000 tx=0x7f3ca8004f00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:03:35.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:35.841+0000 7f3ca7e00700 1 -- 172.21.15.45:0/1647253035 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ca8021070 con 0x7f3cb0108b60 2024-04-16T17:03:35.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:35.841+0000 7f3ca7e00700 1 -- 172.21.15.45:0/1647253035 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ca8007ac0 con 0x7f3cb0108b60 2024-04-16T17:03:35.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:35.841+0000 7f3ca7e00700 1 -- 172.21.15.45:0/1647253035 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ca80052a0 con 0x7f3cb0108b60 2024-04-16T17:03:35.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:35.841+0000 7f3cb6000700 1 -- 172.21.15.45:0/1647253035 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3cb01a7a00 con 0x7f3cb0108b60 2024-04-16T17:03:35.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:35.841+0000 7f3cb6000700 1 -- 172.21.15.45:0/1647253035 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3cb01a7e60 con 0x7f3cb0108b60 2024-04-16T17:03:35.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:35.842+0000 7f3cb6000700 1 -- 172.21.15.45:0/1647253035 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3cb00697e0 con 0x7f3cb0108b60 2024-04-16T17:03:35.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:35.844+0000 7f3ca7e00700 1 -- 172.21.15.45:0/1647253035 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f3ca8024070 con 0x7f3cb0108b60 2024-04-16T17:03:35.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:35.846+0000 7f3ca7e00700 1 --2- 172.21.15.45:0/1647253035 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f3c940a3c40 0x7f3c940a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:35.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:35.846+0000 7f3ca7e00700 1 -- 172.21.15.45:0/1647253035 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f3ca800f020 con 0x7f3cb0108b60 2024-04-16T17:03:35.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:35.846+0000 7f3caf400700 1 --2- 172.21.15.45:0/1647253035 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f3c940a3c40 0x7f3c940a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:35.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:35.847+0000 7f3caf400700 1 --2- 172.21.15.45:0/1647253035 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f3c940a3c40 0x7f3c940a6100 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f3ca0009be0 tx=0x7f3ca0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:03:35.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:35.848+0000 7f3ca7e00700 1 -- 172.21.15.45:0/1647253035 <== mon.2 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3ca802b020 con 0x7f3cb0108b60 2024-04-16T17:03:36.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:36.132+0000 7f3cb6000700 1 -- 172.21.15.45:0/1647253035 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f3cb002dfe0 con 0x7f3cb0108b60 2024-04-16T17:03:36.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:36.133+0000 7f3ca7e00700 1 -- 172.21.15.45:0/1647253035 <== mon.2 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) ==== 53+0+88 (secure 0 0 0) 0x7f3ca809e6e0 con 0x7f3cb0108b60 2024-04-16T17:03:36.134 INFO:teuthology.orchestra.run.smithi045.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-04-16T17:03:36.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:36.136+0000 7f3cb6000700 1 -- 172.21.15.45:0/1647253035 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f3c940a3c40 msgr2=0x7f3c940a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:36.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:36.136+0000 7f3cb6000700 1 --2- 172.21.15.45:0/1647253035 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f3c940a3c40 0x7f3c940a6100 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f3ca0009be0 tx=0x7f3ca0009260 comp rx=0 tx=0).stop 2024-04-16T17:03:36.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:36.136+0000 7f3cb6000700 1 -- 172.21.15.45:0/1647253035 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3cb0108b60 msgr2=0x7f3cb0102530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:36.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:36.136+0000 7f3cb6000700 1 --2- 172.21.15.45:0/1647253035 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3cb0108b60 0x7f3cb0102530 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f3ca8015000 tx=0x7f3ca8004f00 comp rx=0 tx=0).stop 2024-04-16T17:03:36.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:36.136+0000 7f3cb6000700 1 -- 172.21.15.45:0/1647253035 shutdown_connections 2024-04-16T17:03:36.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:36.136+0000 7f3cb6000700 1 --2- 172.21.15.45:0/1647253035 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f3c940a3c40 0x7f3c940a6100 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:36.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:36.136+0000 7f3cb6000700 1 --2- 172.21.15.45:0/1647253035 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3cb006b090 0x7f3cb0101ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:36.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:36.136+0000 7f3cb6000700 1 --2- 172.21.15.45:0/1647253035 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3cb0108b60 0x7f3cb0102530 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:36.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:36.136+0000 7f3cb6000700 1 --2- 172.21.15.45:0/1647253035 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3cb006ba30 0x7f3cb0101ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:36.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:36.136+0000 7f3cb6000700 1 -- 172.21.15.45:0/1647253035 >> 172.21.15.45:0/1647253035 conn(0x7f3cb0078960 msgr2=0x7f3cb00ff990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:03:36.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:36.136+0000 7f3cb6000700 1 -- 172.21.15.45:0/1647253035 shutdown_connections 2024-04-16T17:03:36.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:36.136+0000 7f3cb6000700 1 -- 172.21.15.45:0/1647253035 wait complete. 2024-04-16T17:03:36.478 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:03:36 smithi045 ceph-mon[31676]: from='client.? 172.21.15.45:0/1647253035' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-16T17:03:36.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:03:36 smithi117 ceph-mon[36730]: from='client.? 172.21.15.45:0/1647253035' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-16T17:03:36.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:03:36 smithi104 ceph-mon[36457]: from='client.? 172.21.15.45:0/1647253035' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-16T17:03:37.050 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-16T17:03:37.050 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-16T17:03:37.051 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi045.front.sepia.ceph.com 2024-04-16T17:03:37.051 INFO:tasks.cephfs.mount:self.client.name = client.admin 2024-04-16T17:03:37.051 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-04-16T17:03:37.051 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-16T17:03:37.051 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-16T17:03:37.051 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-16T17:03:37.051 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2024-04-16T17:03:37.051 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T17:03:37.052 DEBUG:teuthology.orchestra.run.smithi045:> ip addr 2024-04-16T17:03:37.112 INFO:teuthology.orchestra.run.smithi045.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-16T17:03:37.113 INFO:teuthology.orchestra.run.smithi045.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-16T17:03:37.113 INFO:teuthology.orchestra.run.smithi045.stdout: inet 127.0.0.1/8 scope host lo 2024-04-16T17:03:37.113 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-04-16T17:03:37.113 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 ::1/128 scope host 2024-04-16T17:03:37.113 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-04-16T17:03:37.113 INFO:teuthology.orchestra.run.smithi045.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-16T17:03:37.113 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:6b:fd:a6 brd ff:ff:ff:ff:ff:ff 2024-04-16T17:03:37.113 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp7s0f0 2024-04-16T17:03:37.113 INFO:teuthology.orchestra.run.smithi045.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-16T17:03:37.113 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:6b:fd:a7 brd ff:ff:ff:ff:ff:ff 2024-04-16T17:03:37.114 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp7s0f1 2024-04-16T17:03:37.114 INFO:teuthology.orchestra.run.smithi045.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-16T17:03:37.114 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:bd:12:be brd ff:ff:ff:ff:ff:ff 2024-04-16T17:03:37.114 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp1s0f0 2024-04-16T17:03:37.114 INFO:teuthology.orchestra.run.smithi045.stdout: inet 172.21.15.45/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-16T17:03:37.114 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft 41958sec preferred_lft 41958sec 2024-04-16T17:03:37.114 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::ec4:7aff:febd:12be/64 scope link noprefixroute 2024-04-16T17:03:37.114 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-04-16T17:03:37.114 INFO:teuthology.orchestra.run.smithi045.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-16T17:03:37.114 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:bd:12:bf brd ff:ff:ff:ff:ff:ff 2024-04-16T17:03:37.115 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp1s0f1 2024-04-16T17:03:37.116 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-16T17:03:37.116 DEBUG:teuthology.orchestra.run.smithi045:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-16T17:03:37.116 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-04-16T17:03:37.117 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip link add name ceph-brx type bridge 2024-04-16T17:03:37.117 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip addr flush dev ceph-brx 2024-04-16T17:03:37.117 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip link set ceph-brx up 2024-04-16T17:03:37.117 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-16T17:03:37.117 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-04-16T17:03:37.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:37 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-16T17:03:37.518 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:03:37 smithi045 ceph-mon[31676]: pgmap v328: 129 pgs: 129 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s wr, 1 op/s 2024-04-16T17:03:37.550 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:03:37 smithi117 ceph-mon[36730]: pgmap v328: 129 pgs: 129 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s wr, 1 op/s 2024-04-16T17:03:37.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-16T17:03:37.673 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T17:03:37.673 DEBUG:teuthology.orchestra.run.smithi045:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-16T17:03:37.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:03:37 smithi104 ceph-mon[36457]: pgmap v328: 129 pgs: 129 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s wr, 1 op/s 2024-04-16T17:03:37.753 INFO:teuthology.orchestra.run.smithi045.stdout:1 2024-04-16T17:03:37.757 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T17:03:37.757 DEBUG:teuthology.orchestra.run.smithi045:> route 2024-04-16T17:03:37.815 INFO:teuthology.orchestra.run.smithi045.stdout:Kernel IP routing table 2024-04-16T17:03:37.815 INFO:teuthology.orchestra.run.smithi045.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-16T17:03:37.815 INFO:teuthology.orchestra.run.smithi045.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-16T17:03:37.816 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-16T17:03:37.816 INFO:teuthology.orchestra.run.smithi045.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-16T17:03:37.817 DEBUG:teuthology.orchestra.run.smithi045:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-16T17:03:37.817 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-04-16T17:03:37.817 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-16T17:03:37.817 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-16T17:03:37.817 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-16T17:03:37.817 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-04-16T17:03:37.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:37 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-16T17:03:38.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-16T17:03:38.639 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T17:03:38.640 DEBUG:teuthology.orchestra.run.smithi045:> ip netns list 2024-04-16T17:03:38.696 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T17:03:38.696 DEBUG:teuthology.orchestra.run.smithi045:> ip netns list-id 2024-04-16T17:03:38.750 DEBUG:teuthology.orchestra.run.smithi045:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-16T17:03:38.751 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-04-16T17:03:38.751 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-16T17:03:38.751 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2024-04-16T17:03:38.751 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-04-16T17:03:38.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:38 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-16T17:03:38.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-16T17:03:38.962 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2024-04-16T17:03:38.962 DEBUG:teuthology.orchestra.run.smithi045:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-16T17:03:38.962 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-04-16T17:03:38.962 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2024-04-16T17:03:38.962 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-16T17:03:38.963 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2024-04-16T17:03:38.963 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2024-04-16T17:03:38.963 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2024-04-16T17:03:38.963 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-04-16T17:03:39.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:39 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-16T17:03:39.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:39 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-16T17:03:39.330 DEBUG:teuthology.orchestra.run.smithi045:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-16T17:03:39.330 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-04-16T17:03:39.330 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip link set brx.0 up 2024-04-16T17:03:39.330 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip link set dev brx.0 master ceph-brx 2024-04-16T17:03:39.330 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-04-16T17:03:39.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:39 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-16T17:03:39.478 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:03:39 smithi045 ceph-mon[31676]: pgmap v329: 129 pgs: 129 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s wr, 1 op/s 2024-04-16T17:03:39.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:03:39 smithi117 ceph-mon[36730]: pgmap v329: 129 pgs: 129 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s wr, 1 op/s 2024-04-16T17:03:39.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:39 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-16T17:03:39.580 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2024-04-16T17:03:39.580 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-16T17:03:39.580 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2024-04-16T17:03:39.628 INFO:teuthology.orchestra.run.smithi045.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2024-04-16T17:03:39.628 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-16T17:03:39.628 DEBUG:teuthology.orchestra.run.smithi045:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2024-04-16T17:03:39.677 DEBUG:teuthology.orchestra.run.smithi045:> sudo modprobe fuse 2024-04-16T17:03:39.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:03:39 smithi104 ceph-mon[36457]: pgmap v329: 129 pgs: 129 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s wr, 1 op/s 2024-04-16T17:03:39.766 DEBUG:teuthology.orchestra.run.smithi045:> cat /proc/self/mounts | awk '{print $2}' 2024-04-16T17:03:39.820 INFO:teuthology.orchestra.run.smithi045.stdout:/proc 2024-04-16T17:03:39.820 INFO:teuthology.orchestra.run.smithi045.stdout:/sys 2024-04-16T17:03:39.820 INFO:teuthology.orchestra.run.smithi045.stdout:/dev 2024-04-16T17:03:39.820 INFO:teuthology.orchestra.run.smithi045.stdout:/sys/kernel/security 2024-04-16T17:03:39.820 INFO:teuthology.orchestra.run.smithi045.stdout:/dev/shm 2024-04-16T17:03:39.820 INFO:teuthology.orchestra.run.smithi045.stdout:/dev/pts 2024-04-16T17:03:39.820 INFO:teuthology.orchestra.run.smithi045.stdout:/run 2024-04-16T17:03:39.820 INFO:teuthology.orchestra.run.smithi045.stdout:/sys/fs/cgroup 2024-04-16T17:03:39.821 INFO:teuthology.orchestra.run.smithi045.stdout:/sys/fs/pstore 2024-04-16T17:03:39.821 INFO:teuthology.orchestra.run.smithi045.stdout:/sys/fs/bpf 2024-04-16T17:03:39.821 INFO:teuthology.orchestra.run.smithi045.stdout:/ 2024-04-16T17:03:39.821 INFO:teuthology.orchestra.run.smithi045.stdout:/sys/fs/selinux 2024-04-16T17:03:39.821 INFO:teuthology.orchestra.run.smithi045.stdout:/proc/sys/fs/binfmt_misc 2024-04-16T17:03:39.821 INFO:teuthology.orchestra.run.smithi045.stdout:/dev/hugepages 2024-04-16T17:03:39.821 INFO:teuthology.orchestra.run.smithi045.stdout:/dev/mqueue 2024-04-16T17:03:39.821 INFO:teuthology.orchestra.run.smithi045.stdout:/sys/kernel/debug 2024-04-16T17:03:39.821 INFO:teuthology.orchestra.run.smithi045.stdout:/sys/kernel/tracing 2024-04-16T17:03:39.821 INFO:teuthology.orchestra.run.smithi045.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-16T17:03:39.821 INFO:teuthology.orchestra.run.smithi045.stdout:/sys/fs/fuse/connections 2024-04-16T17:03:39.821 INFO:teuthology.orchestra.run.smithi045.stdout:/sys/kernel/config 2024-04-16T17:03:39.822 INFO:teuthology.orchestra.run.smithi045.stdout:/run/credentials/systemd-sysctl.service 2024-04-16T17:03:39.822 INFO:teuthology.orchestra.run.smithi045.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-16T17:03:39.822 INFO:teuthology.orchestra.run.smithi045.stdout:/var/lib/nfs/rpc_pipefs 2024-04-16T17:03:39.822 INFO:teuthology.orchestra.run.smithi045.stdout:/run/user/1000 2024-04-16T17:03:39.822 INFO:teuthology.orchestra.run.smithi045.stdout:/var/lib/ceph 2024-04-16T17:03:39.822 INFO:teuthology.orchestra.run.smithi045.stdout:/var/lib/containers/storage/overlay 2024-04-16T17:03:39.822 INFO:teuthology.orchestra.run.smithi045.stdout:/var/lib/containers/storage/overlay/3f91d22174ed37c5feb685df89197b179afed416bc0da2e26485b5107b96d657/merged 2024-04-16T17:03:39.822 INFO:teuthology.orchestra.run.smithi045.stdout:/var/lib/containers/storage/overlay/09149817d89b6908cbe2a45e6aebbfda1f5e7becfec4cba92be3eba0bb729a15/merged 2024-04-16T17:03:39.822 INFO:teuthology.orchestra.run.smithi045.stdout:/run/user/0 2024-04-16T17:03:39.822 INFO:teuthology.orchestra.run.smithi045.stdout:/proc/sys/fs/binfmt_misc 2024-04-16T17:03:39.822 INFO:teuthology.orchestra.run.smithi045.stdout:/var/lib/containers/storage/overlay/c48eee27b040c08172c1b1362f5c61e9985204605bb27540039fb8fb7227ce42/merged 2024-04-16T17:03:39.823 INFO:teuthology.orchestra.run.smithi045.stdout:/var/lib/containers/storage/overlay/792925c1eb4c2623f3271c7f92c37b50c1e555849c3e192ed941d438aef5aa0a/merged 2024-04-16T17:03:39.823 INFO:teuthology.orchestra.run.smithi045.stdout:/var/lib/containers/storage/overlay/17b1e9320387ec8e36ef717afb4ee617e6b9fe0bb52458747997b7425a5c994e/merged 2024-04-16T17:03:39.823 INFO:teuthology.orchestra.run.smithi045.stdout:/var/lib/containers/storage/overlay/51a2adb03cd0c1671f3a954c6c0d1c05840e22f9dd5db65f99190810d29ec977/merged 2024-04-16T17:03:39.823 INFO:teuthology.orchestra.run.smithi045.stdout:/var/lib/containers/storage/overlay/80547563580a1e3552ef649c209c0d4bd8fc006a206b3e6f3cb78d4a04ba3292/merged 2024-04-16T17:03:39.823 INFO:teuthology.orchestra.run.smithi045.stdout:/var/lib/containers/storage/overlay/85ea1f8c8a32a776a49cb9cfafb5cec3642d4edc135a3b18a9eeb1e5c887b2ec/merged 2024-04-16T17:03:39.823 INFO:teuthology.orchestra.run.smithi045.stdout:/var/lib/containers/storage/overlay/d51bec82bdae12d6640ecc21033a21dcae2c1fb85e78c2cec712182a60a83ddd/merged 2024-04-16T17:03:39.823 INFO:teuthology.orchestra.run.smithi045.stdout:/var/lib/containers/storage/overlay/1767b54943cb11cc485544a9b7a28be53801244bddf366fc0088d215902506da/merged 2024-04-16T17:03:39.823 INFO:teuthology.orchestra.run.smithi045.stdout:/run/netns 2024-04-16T17:03:39.823 INFO:teuthology.orchestra.run.smithi045.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-16T17:03:39.823 INFO:teuthology.orchestra.run.smithi045.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-16T17:03:39.824 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T17:03:39.824 DEBUG:teuthology.orchestra.run.smithi045:> ls /sys/fs/fuse/connections 2024-04-16T17:03:39.874 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-16T17:03:39.874 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2024-04-16T17:03:39.879 DEBUG:teuthology.orchestra.run.smithi045:> sudo modprobe fuse 2024-04-16T17:03:39.956 DEBUG:teuthology.orchestra.run.smithi045:> cat /proc/self/mounts | awk '{print $2}' 2024-04-16T17:03:40.000 INFO:teuthology.orchestra.run.smithi045.stdout:/proc 2024-04-16T17:03:40.001 INFO:teuthology.orchestra.run.smithi045.stdout:/sys 2024-04-16T17:03:40.001 INFO:teuthology.orchestra.run.smithi045.stdout:/dev 2024-04-16T17:03:40.001 INFO:teuthology.orchestra.run.smithi045.stdout:/sys/kernel/security 2024-04-16T17:03:40.001 INFO:teuthology.orchestra.run.smithi045.stdout:/dev/shm 2024-04-16T17:03:40.001 INFO:teuthology.orchestra.run.smithi045.stdout:/dev/pts 2024-04-16T17:03:40.001 INFO:teuthology.orchestra.run.smithi045.stdout:/run 2024-04-16T17:03:40.001 INFO:teuthology.orchestra.run.smithi045.stdout:/sys/fs/cgroup 2024-04-16T17:03:40.001 INFO:teuthology.orchestra.run.smithi045.stdout:/sys/fs/pstore 2024-04-16T17:03:40.001 INFO:teuthology.orchestra.run.smithi045.stdout:/sys/fs/bpf 2024-04-16T17:03:40.001 INFO:teuthology.orchestra.run.smithi045.stdout:/ 2024-04-16T17:03:40.002 INFO:teuthology.orchestra.run.smithi045.stdout:/sys/fs/selinux 2024-04-16T17:03:40.002 INFO:teuthology.orchestra.run.smithi045.stdout:/proc/sys/fs/binfmt_misc 2024-04-16T17:03:40.002 INFO:teuthology.orchestra.run.smithi045.stdout:/dev/hugepages 2024-04-16T17:03:40.002 INFO:teuthology.orchestra.run.smithi045.stdout:/dev/mqueue 2024-04-16T17:03:40.002 INFO:teuthology.orchestra.run.smithi045.stdout:/sys/kernel/debug 2024-04-16T17:03:40.002 INFO:teuthology.orchestra.run.smithi045.stdout:/sys/kernel/tracing 2024-04-16T17:03:40.002 INFO:teuthology.orchestra.run.smithi045.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-16T17:03:40.002 INFO:teuthology.orchestra.run.smithi045.stdout:/sys/fs/fuse/connections 2024-04-16T17:03:40.002 INFO:teuthology.orchestra.run.smithi045.stdout:/sys/kernel/config 2024-04-16T17:03:40.002 INFO:teuthology.orchestra.run.smithi045.stdout:/run/credentials/systemd-sysctl.service 2024-04-16T17:03:40.002 INFO:teuthology.orchestra.run.smithi045.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-16T17:03:40.002 INFO:teuthology.orchestra.run.smithi045.stdout:/var/lib/nfs/rpc_pipefs 2024-04-16T17:03:40.003 INFO:teuthology.orchestra.run.smithi045.stdout:/run/user/1000 2024-04-16T17:03:40.003 INFO:teuthology.orchestra.run.smithi045.stdout:/var/lib/ceph 2024-04-16T17:03:40.003 INFO:teuthology.orchestra.run.smithi045.stdout:/var/lib/containers/storage/overlay 2024-04-16T17:03:40.003 INFO:teuthology.orchestra.run.smithi045.stdout:/var/lib/containers/storage/overlay/3f91d22174ed37c5feb685df89197b179afed416bc0da2e26485b5107b96d657/merged 2024-04-16T17:03:40.003 INFO:teuthology.orchestra.run.smithi045.stdout:/var/lib/containers/storage/overlay/09149817d89b6908cbe2a45e6aebbfda1f5e7becfec4cba92be3eba0bb729a15/merged 2024-04-16T17:03:40.003 INFO:teuthology.orchestra.run.smithi045.stdout:/run/user/0 2024-04-16T17:03:40.003 INFO:teuthology.orchestra.run.smithi045.stdout:/proc/sys/fs/binfmt_misc 2024-04-16T17:03:40.003 INFO:teuthology.orchestra.run.smithi045.stdout:/var/lib/containers/storage/overlay/c48eee27b040c08172c1b1362f5c61e9985204605bb27540039fb8fb7227ce42/merged 2024-04-16T17:03:40.003 INFO:teuthology.orchestra.run.smithi045.stdout:/var/lib/containers/storage/overlay/792925c1eb4c2623f3271c7f92c37b50c1e555849c3e192ed941d438aef5aa0a/merged 2024-04-16T17:03:40.003 INFO:teuthology.orchestra.run.smithi045.stdout:/var/lib/containers/storage/overlay/17b1e9320387ec8e36ef717afb4ee617e6b9fe0bb52458747997b7425a5c994e/merged 2024-04-16T17:03:40.003 INFO:teuthology.orchestra.run.smithi045.stdout:/var/lib/containers/storage/overlay/51a2adb03cd0c1671f3a954c6c0d1c05840e22f9dd5db65f99190810d29ec977/merged 2024-04-16T17:03:40.003 INFO:teuthology.orchestra.run.smithi045.stdout:/var/lib/containers/storage/overlay/80547563580a1e3552ef649c209c0d4bd8fc006a206b3e6f3cb78d4a04ba3292/merged 2024-04-16T17:03:40.004 INFO:teuthology.orchestra.run.smithi045.stdout:/var/lib/containers/storage/overlay/85ea1f8c8a32a776a49cb9cfafb5cec3642d4edc135a3b18a9eeb1e5c887b2ec/merged 2024-04-16T17:03:40.004 INFO:teuthology.orchestra.run.smithi045.stdout:/var/lib/containers/storage/overlay/d51bec82bdae12d6640ecc21033a21dcae2c1fb85e78c2cec712182a60a83ddd/merged 2024-04-16T17:03:40.004 INFO:teuthology.orchestra.run.smithi045.stdout:/var/lib/containers/storage/overlay/1767b54943cb11cc485544a9b7a28be53801244bddf366fc0088d215902506da/merged 2024-04-16T17:03:40.004 INFO:teuthology.orchestra.run.smithi045.stdout:/run/netns 2024-04-16T17:03:40.004 INFO:teuthology.orchestra.run.smithi045.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-16T17:03:40.004 INFO:teuthology.orchestra.run.smithi045.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-16T17:03:40.005 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T17:03:40.005 DEBUG:teuthology.orchestra.run.smithi045:> ls /sys/fs/fuse/connections 2024-04-16T17:03:40.086 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi045.stderr:2024-04-16T17:03:40.085+0000 7fad90038480 -1 init, newargv = 0x562278640f60 newargc=15 2024-04-16T17:03:40.086 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi045.stderr:ceph-fuse[75514]: starting ceph client 2024-04-16T17:03:40.097 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi045.stderr:ceph-fuse[75514]: starting fuse 2024-04-16T17:03:41.190 DEBUG:teuthology.orchestra.run.smithi045:> sudo modprobe fuse 2024-04-16T17:03:41.284 DEBUG:teuthology.orchestra.run.smithi045:> cat /proc/self/mounts | awk '{print $2}' 2024-04-16T17:03:41.336 INFO:teuthology.orchestra.run.smithi045.stdout:/proc 2024-04-16T17:03:41.336 INFO:teuthology.orchestra.run.smithi045.stdout:/sys 2024-04-16T17:03:41.336 INFO:teuthology.orchestra.run.smithi045.stdout:/dev 2024-04-16T17:03:41.337 INFO:teuthology.orchestra.run.smithi045.stdout:/sys/kernel/security 2024-04-16T17:03:41.337 INFO:teuthology.orchestra.run.smithi045.stdout:/dev/shm 2024-04-16T17:03:41.337 INFO:teuthology.orchestra.run.smithi045.stdout:/dev/pts 2024-04-16T17:03:41.337 INFO:teuthology.orchestra.run.smithi045.stdout:/run 2024-04-16T17:03:41.337 INFO:teuthology.orchestra.run.smithi045.stdout:/sys/fs/cgroup 2024-04-16T17:03:41.337 INFO:teuthology.orchestra.run.smithi045.stdout:/sys/fs/pstore 2024-04-16T17:03:41.337 INFO:teuthology.orchestra.run.smithi045.stdout:/sys/fs/bpf 2024-04-16T17:03:41.337 INFO:teuthology.orchestra.run.smithi045.stdout:/ 2024-04-16T17:03:41.337 INFO:teuthology.orchestra.run.smithi045.stdout:/sys/fs/selinux 2024-04-16T17:03:41.337 INFO:teuthology.orchestra.run.smithi045.stdout:/proc/sys/fs/binfmt_misc 2024-04-16T17:03:41.337 INFO:teuthology.orchestra.run.smithi045.stdout:/dev/hugepages 2024-04-16T17:03:41.337 INFO:teuthology.orchestra.run.smithi045.stdout:/dev/mqueue 2024-04-16T17:03:41.337 INFO:teuthology.orchestra.run.smithi045.stdout:/sys/kernel/debug 2024-04-16T17:03:41.337 INFO:teuthology.orchestra.run.smithi045.stdout:/sys/kernel/tracing 2024-04-16T17:03:41.337 INFO:teuthology.orchestra.run.smithi045.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-16T17:03:41.338 INFO:teuthology.orchestra.run.smithi045.stdout:/sys/fs/fuse/connections 2024-04-16T17:03:41.338 INFO:teuthology.orchestra.run.smithi045.stdout:/sys/kernel/config 2024-04-16T17:03:41.338 INFO:teuthology.orchestra.run.smithi045.stdout:/run/credentials/systemd-sysctl.service 2024-04-16T17:03:41.338 INFO:teuthology.orchestra.run.smithi045.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-16T17:03:41.338 INFO:teuthology.orchestra.run.smithi045.stdout:/var/lib/nfs/rpc_pipefs 2024-04-16T17:03:41.338 INFO:teuthology.orchestra.run.smithi045.stdout:/run/user/1000 2024-04-16T17:03:41.338 INFO:teuthology.orchestra.run.smithi045.stdout:/var/lib/ceph 2024-04-16T17:03:41.338 INFO:teuthology.orchestra.run.smithi045.stdout:/var/lib/containers/storage/overlay 2024-04-16T17:03:41.338 INFO:teuthology.orchestra.run.smithi045.stdout:/var/lib/containers/storage/overlay/3f91d22174ed37c5feb685df89197b179afed416bc0da2e26485b5107b96d657/merged 2024-04-16T17:03:41.338 INFO:teuthology.orchestra.run.smithi045.stdout:/var/lib/containers/storage/overlay/09149817d89b6908cbe2a45e6aebbfda1f5e7becfec4cba92be3eba0bb729a15/merged 2024-04-16T17:03:41.338 INFO:teuthology.orchestra.run.smithi045.stdout:/run/user/0 2024-04-16T17:03:41.338 INFO:teuthology.orchestra.run.smithi045.stdout:/proc/sys/fs/binfmt_misc 2024-04-16T17:03:41.338 INFO:teuthology.orchestra.run.smithi045.stdout:/var/lib/containers/storage/overlay/c48eee27b040c08172c1b1362f5c61e9985204605bb27540039fb8fb7227ce42/merged 2024-04-16T17:03:41.338 INFO:teuthology.orchestra.run.smithi045.stdout:/var/lib/containers/storage/overlay/792925c1eb4c2623f3271c7f92c37b50c1e555849c3e192ed941d438aef5aa0a/merged 2024-04-16T17:03:41.338 INFO:teuthology.orchestra.run.smithi045.stdout:/var/lib/containers/storage/overlay/17b1e9320387ec8e36ef717afb4ee617e6b9fe0bb52458747997b7425a5c994e/merged 2024-04-16T17:03:41.339 INFO:teuthology.orchestra.run.smithi045.stdout:/var/lib/containers/storage/overlay/51a2adb03cd0c1671f3a954c6c0d1c05840e22f9dd5db65f99190810d29ec977/merged 2024-04-16T17:03:41.339 INFO:teuthology.orchestra.run.smithi045.stdout:/var/lib/containers/storage/overlay/80547563580a1e3552ef649c209c0d4bd8fc006a206b3e6f3cb78d4a04ba3292/merged 2024-04-16T17:03:41.339 INFO:teuthology.orchestra.run.smithi045.stdout:/var/lib/containers/storage/overlay/85ea1f8c8a32a776a49cb9cfafb5cec3642d4edc135a3b18a9eeb1e5c887b2ec/merged 2024-04-16T17:03:41.339 INFO:teuthology.orchestra.run.smithi045.stdout:/var/lib/containers/storage/overlay/d51bec82bdae12d6640ecc21033a21dcae2c1fb85e78c2cec712182a60a83ddd/merged 2024-04-16T17:03:41.339 INFO:teuthology.orchestra.run.smithi045.stdout:/var/lib/containers/storage/overlay/1767b54943cb11cc485544a9b7a28be53801244bddf366fc0088d215902506da/merged 2024-04-16T17:03:41.339 INFO:teuthology.orchestra.run.smithi045.stdout:/run/netns 2024-04-16T17:03:41.339 INFO:teuthology.orchestra.run.smithi045.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-16T17:03:41.339 INFO:teuthology.orchestra.run.smithi045.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-16T17:03:41.339 INFO:teuthology.orchestra.run.smithi045.stdout:/home/ubuntu/cephtest/mnt.admin 2024-04-16T17:03:41.340 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T17:03:41.340 DEBUG:teuthology.orchestra.run.smithi045:> ls /sys/fs/fuse/connections 2024-04-16T17:03:41.389 INFO:teuthology.orchestra.run.smithi045.stdout:119 2024-04-16T17:03:41.389 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [119] 2024-04-16T17:03:41.389 DEBUG:teuthology.orchestra.run.smithi045:> sudo stdin-killer -- python3 -c ' 2024-04-16T17:03:41.389 DEBUG:teuthology.orchestra.run.smithi045:> import glob 2024-04-16T17:03:41.389 DEBUG:teuthology.orchestra.run.smithi045:> import re 2024-04-16T17:03:41.390 DEBUG:teuthology.orchestra.run.smithi045:> import os 2024-04-16T17:03:41.390 DEBUG:teuthology.orchestra.run.smithi045:> import subprocess 2024-04-16T17:03:41.390 DEBUG:teuthology.orchestra.run.smithi045:> 2024-04-16T17:03:41.390 DEBUG:teuthology.orchestra.run.smithi045:> def _find_admin_socket(client_name): 2024-04-16T17:03:41.390 DEBUG:teuthology.orchestra.run.smithi045:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2024-04-16T17:03:41.390 DEBUG:teuthology.orchestra.run.smithi045:> files = glob.glob(asok_path) 2024-04-16T17:03:41.390 DEBUG:teuthology.orchestra.run.smithi045:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2024-04-16T17:03:41.390 DEBUG:teuthology.orchestra.run.smithi045:> 2024-04-16T17:03:41.390 DEBUG:teuthology.orchestra.run.smithi045:> # Given a non-glob path, it better be there 2024-04-16T17:03:41.390 DEBUG:teuthology.orchestra.run.smithi045:> if "*" not in asok_path: 2024-04-16T17:03:41.390 DEBUG:teuthology.orchestra.run.smithi045:> assert(len(files) == 1) 2024-04-16T17:03:41.390 DEBUG:teuthology.orchestra.run.smithi045:> return files[0] 2024-04-16T17:03:41.390 DEBUG:teuthology.orchestra.run.smithi045:> 2024-04-16T17:03:41.390 DEBUG:teuthology.orchestra.run.smithi045:> for f in files: 2024-04-16T17:03:41.390 DEBUG:teuthology.orchestra.run.smithi045:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-16T17:03:41.391 DEBUG:teuthology.orchestra.run.smithi045:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-16T17:03:41.391 DEBUG:teuthology.orchestra.run.smithi045:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-16T17:03:41.391 DEBUG:teuthology.orchestra.run.smithi045:> contents = proc_f.read() 2024-04-16T17:03:41.391 DEBUG:teuthology.orchestra.run.smithi045:> if mountpoint in contents: 2024-04-16T17:03:41.391 DEBUG:teuthology.orchestra.run.smithi045:> return f 2024-04-16T17:03:41.391 DEBUG:teuthology.orchestra.run.smithi045:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-16T17:03:41.391 DEBUG:teuthology.orchestra.run.smithi045:> 2024-04-16T17:03:41.391 DEBUG:teuthology.orchestra.run.smithi045:> print(_find_admin_socket("client.admin")) 2024-04-16T17:03:41.391 DEBUG:teuthology.orchestra.run.smithi045:> ' 2024-04-16T17:03:41.550 INFO:teuthology.orchestra.run.smithi045.stdout:/var/run/ceph/ceph-client.admin.75514.asok 2024-04-16T17:03:41.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:41 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-16T17:03:41.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:03:41 smithi117 ceph-mon[36730]: pgmap v330: 129 pgs: 129 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 716 B/s wr, 0 op/s 2024-04-16T17:03:41.563 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:03:41 smithi045 ceph-mon[31676]: pgmap v330: 129 pgs: 129 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 716 B/s wr, 0 op/s 2024-04-16T17:03:41.566 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.75514.asok 2024-04-16T17:03:41.566 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T17:03:41.567 DEBUG:teuthology.orchestra.run.smithi045:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.75514.asok status 2024-04-16T17:03:41.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:03:41 smithi104 ceph-mon[36457]: pgmap v330: 129 pgs: 129 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 716 B/s wr, 0 op/s 2024-04-16T17:03:41.812 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-16T17:03:41.812 INFO:teuthology.orchestra.run.smithi045.stdout: "metadata": { 2024-04-16T17:03:41.813 INFO:teuthology.orchestra.run.smithi045.stdout: "ceph_sha1": "9b42bfbeda298874861ece590aa565cf64ac8c45", 2024-04-16T17:03:41.813 INFO:teuthology.orchestra.run.smithi045.stdout: "ceph_version": "ceph version 19.0.0-3101-g9b42bfbe (9b42bfbeda298874861ece590aa565cf64ac8c45) squid (dev)", 2024-04-16T17:03:41.813 INFO:teuthology.orchestra.run.smithi045.stdout: "entity_id": "admin", 2024-04-16T17:03:41.813 INFO:teuthology.orchestra.run.smithi045.stdout: "hostname": "smithi045", 2024-04-16T17:03:41.813 INFO:teuthology.orchestra.run.smithi045.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2024-04-16T17:03:41.813 INFO:teuthology.orchestra.run.smithi045.stdout: "pid": "75514", 2024-04-16T17:03:41.813 INFO:teuthology.orchestra.run.smithi045.stdout: "root": "/" 2024-04-16T17:03:41.813 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:03:41.813 INFO:teuthology.orchestra.run.smithi045.stdout: "dentry_count": 0, 2024-04-16T17:03:41.813 INFO:teuthology.orchestra.run.smithi045.stdout: "dentry_pinned_count": 0, 2024-04-16T17:03:41.813 INFO:teuthology.orchestra.run.smithi045.stdout: "id": 24703, 2024-04-16T17:03:41.813 INFO:teuthology.orchestra.run.smithi045.stdout: "inst": { 2024-04-16T17:03:41.813 INFO:teuthology.orchestra.run.smithi045.stdout: "name": { 2024-04-16T17:03:41.813 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "client", 2024-04-16T17:03:41.813 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 24703 2024-04-16T17:03:41.814 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:03:41.814 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": { 2024-04-16T17:03:41.814 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-04-16T17:03:41.814 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "192.168.0.1:0", 2024-04-16T17:03:41.814 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 2182114945 2024-04-16T17:03:41.814 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:03:41.814 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:03:41.814 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": { 2024-04-16T17:03:41.814 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-04-16T17:03:41.814 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "192.168.0.1:0", 2024-04-16T17:03:41.814 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 2182114945 2024-04-16T17:03:41.814 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:03:41.814 INFO:teuthology.orchestra.run.smithi045.stdout: "inst_str": "client.24703 192.168.0.1:0/2182114945", 2024-04-16T17:03:41.814 INFO:teuthology.orchestra.run.smithi045.stdout: "addr_str": "192.168.0.1:0/2182114945", 2024-04-16T17:03:41.815 INFO:teuthology.orchestra.run.smithi045.stdout: "inode_count": 1, 2024-04-16T17:03:41.815 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_epoch": 17, 2024-04-16T17:03:41.815 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_epoch": 90, 2024-04-16T17:03:41.815 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_epoch_barrier": 0, 2024-04-16T17:03:41.815 INFO:teuthology.orchestra.run.smithi045.stdout: "blocklisted": false, 2024-04-16T17:03:41.815 INFO:teuthology.orchestra.run.smithi045.stdout: "fs_name": "cephfs" 2024-04-16T17:03:41.815 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-16T17:03:41.828 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T17:03:41.828 DEBUG:teuthology.orchestra.run.smithi045:> stat --file-system '--printf=%T 2024-04-16T17:03:41.828 DEBUG:teuthology.orchestra.run.smithi045:> ' -- /home/ubuntu/cephtest/mnt.admin 2024-04-16T17:03:41.879 INFO:teuthology.orchestra.run.smithi045.stdout:fuseblk 2024-04-16T17:03:41.879 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2024-04-16T17:03:41.879 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T17:03:41.879 DEBUG:teuthology.orchestra.run.smithi045:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2024-04-16T17:03:41.968 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.admin && exec stdin-killer --timeout=900 -- bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2024-04-16T17:03:42.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:42 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-16T17:03:42.121 INFO:teuthology.orchestra.run.smithi045.stdout:# file: . 2024-04-16T17:03:42.121 INFO:teuthology.orchestra.run.smithi045.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2024-04-16T17:03:42.121 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-16T17:03:42.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-16T17:03:42.131 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-04-16T17:03:42.131 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-16T17:03:42.184 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-16T17:03:42.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:42.408+0000 7fb707200640 1 -- 172.21.15.45:0/1917336864 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb708105290 msgr2=0x7fb708105710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:42.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:42.408+0000 7fb707200640 1 --2- 172.21.15.45:0/1917336864 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb708105290 0x7fb708105710 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fb6f0009ca0 tx=0x7fb6f002c100 comp rx=0 tx=0).stop 2024-04-16T17:03:42.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:42.409+0000 7fb707200640 1 -- 172.21.15.45:0/1917336864 shutdown_connections 2024-04-16T17:03:42.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:42.409+0000 7fb707200640 1 --2- 172.21.15.45:0/1917336864 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb708105c50 0x7fb70810c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:42.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:42.409+0000 7fb707200640 1 --2- 172.21.15.45:0/1917336864 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb708105290 0x7fb708105710 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:42.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:42.409+0000 7fb707200640 1 --2- 172.21.15.45:0/1917336864 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb7081040a0 0x7fb7081044c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:42.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:42.409+0000 7fb707200640 1 -- 172.21.15.45:0/1917336864 >> 172.21.15.45:0/1917336864 conn(0x7fb7080ff610 msgr2=0x7fb708101a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:03:42.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:42.409+0000 7fb707200640 1 -- 172.21.15.45:0/1917336864 shutdown_connections 2024-04-16T17:03:42.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:42.409+0000 7fb707200640 1 -- 172.21.15.45:0/1917336864 wait complete. 2024-04-16T17:03:42.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:42.410+0000 7fb707200640 1 Processor -- start 2024-04-16T17:03:42.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:42.410+0000 7fb707200640 1 -- start start 2024-04-16T17:03:42.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:42.411+0000 7fb707200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb7081040a0 0x7fb70819a380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:42.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:42.411+0000 7fb707200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb708105290 0x7fb70819a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:42.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:42.411+0000 7fb707200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb708105c50 0x7fb7081a1930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:42.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:42.411+0000 7fb707200640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fb70810fd30 con 0x7fb708105290 2024-04-16T17:03:42.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:42.411+0000 7fb707200640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fb70810fbb0 con 0x7fb708105c50 2024-04-16T17:03:42.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:42.411+0000 7fb707200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb70810feb0 con 0x7fb7081040a0 2024-04-16T17:03:42.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:42.411+0000 7fb705400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb708105290 0x7fb70819a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:42.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:42.411+0000 7fb705400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb708105290 0x7fb70819a8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:44298/0 (socket says 172.21.15.45:44298) 2024-04-16T17:03:42.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:42.411+0000 7fb705400640 1 -- 172.21.15.45:0/120829070 learned_addr learned my addr 172.21.15.45:0/120829070 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:03:42.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:42.411+0000 7fb705e00640 1 --2- 172.21.15.45:0/120829070 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb7081040a0 0x7fb70819a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:42.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:42.411+0000 7fb706800640 1 --2- 172.21.15.45:0/120829070 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb708105c50 0x7fb7081a1930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:42.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:42.411+0000 7fb705400640 1 -- 172.21.15.45:0/120829070 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb7081040a0 msgr2=0x7fb70819a380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:42.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:42.411+0000 7fb705400640 1 --2- 172.21.15.45:0/120829070 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb7081040a0 0x7fb70819a380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:42.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:42.411+0000 7fb705400640 1 -- 172.21.15.45:0/120829070 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb708105c50 msgr2=0x7fb7081a1930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:42.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:42.411+0000 7fb705400640 1 --2- 172.21.15.45:0/120829070 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb708105c50 0x7fb7081a1930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:42.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:42.411+0000 7fb705400640 1 -- 172.21.15.45:0/120829070 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7081a1fa0 con 0x7fb708105290 2024-04-16T17:03:42.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:42.412+0000 7fb705400640 1 --2- 172.21.15.45:0/120829070 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb708105290 0x7fb70819a8c0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fb6f00325f0 tx=0x7fb6f0035d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:03:42.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:42.412+0000 7fb6fea00640 1 -- 172.21.15.45:0/120829070 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6f0039040 con 0x7fb708105290 2024-04-16T17:03:42.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:42.412+0000 7fb6fea00640 1 -- 172.21.15.45:0/120829070 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb6f0044070 con 0x7fb708105290 2024-04-16T17:03:42.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:42.412+0000 7fb707200640 1 -- 172.21.15.45:0/120829070 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7081a2230 con 0x7fb708105290 2024-04-16T17:03:42.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:42.412+0000 7fb6fea00640 1 -- 172.21.15.45:0/120829070 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6f0004540 con 0x7fb708105290 2024-04-16T17:03:42.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:42.412+0000 7fb707200640 1 -- 172.21.15.45:0/120829070 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7081a26f0 con 0x7fb708105290 2024-04-16T17:03:42.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:42.413+0000 7fb707200640 1 -- 172.21.15.45:0/120829070 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb70810b3c0 con 0x7fb708105290 2024-04-16T17:03:42.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:42.415+0000 7fb6fea00640 1 -- 172.21.15.45:0/120829070 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fb6f0032890 con 0x7fb708105290 2024-04-16T17:03:42.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:42.417+0000 7fb6fea00640 1 --2- 172.21.15.45:0/120829070 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fb6e80a3bb0 0x7fb6e80a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:42.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:42.417+0000 7fb6fea00640 1 -- 172.21.15.45:0/120829070 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7fb6f00f2c90 con 0x7fb708105290 2024-04-16T17:03:42.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:42.417+0000 7fb705e00640 1 --2- 172.21.15.45:0/120829070 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fb6e80a3bb0 0x7fb6e80a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:42.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:42.418+0000 7fb705e00640 1 --2- 172.21.15.45:0/120829070 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fb6e80a3bb0 0x7fb6e80a6070 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fb7081050f0 tx=0x7fb6f8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:03:42.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:42.422+0000 7fb6fea00640 1 -- 172.21.15.45:0/120829070 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb6f00bf3d0 con 0x7fb708105290 2024-04-16T17:03:42.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:42.629+0000 7fb707200640 1 -- 172.21.15.45:0/120829070 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb708062c00 con 0x7fb708105290 2024-04-16T17:03:42.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:42.634+0000 7fb6fea00640 1 -- 172.21.15.45:0/120829070 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+24164 (secure 0 0 0) 0x7fb6f00becd0 con 0x7fb708105290 2024-04-16T17:03:42.635 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-16T17:03:42.636 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":90,"fsid":"b1fecd38-fc11-11ee-bc90-c7b262605968","created":"2024-04-16T16:54:02.385906+0000","modified":"2024-04-16T17:03:29.720565+0000","last_up_change":"2024-04-16T17:02:25.162624+0000","last_in_change":"2024-04-16T17:02:04.903274+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-16T16:58:06.166659+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-16T17:03:22.304198+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-16T17:03:23.616450+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-16T17:03:27.226289+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b655de2c-59f8-4fd4-ac98-0cc222229f56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":2290170881},{"type":"v1","addr":"172.21.15.45:6803","nonce":2290170881}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":2290170881},{"type":"v1","addr":"172.21.15.45:6805","nonce":2290170881}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":2290170881},{"type":"v1","addr":"172.21.15.45:6809","nonce":2290170881}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":2290170881},{"type":"v1","addr":"172.21.15.45:6807","nonce":2290170881}]},"public_addr":"172.21.15.45:6803/2290170881","cluster_addr":"172.21.15.45:6805/2290170881","heartbeat_back_addr":"172.21.15.45:6809/2290170881","heartbeat_front_addr":"172.21.15.45:6807/2290170881","state":["exists","up"]},{"osd":1,"uuid":"ebd450ec-08fd-4c3d-8b24-4e10634ddb7e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6800","nonce":1462942067},{"type":"v1","addr":"172.21.15.104:6801","nonce":1462942067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":1462942067},{"type":"v1","addr":"172.21.15.104:6803","nonce":1462942067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":1462942067},{"type":"v1","addr":"172.21.15.104:6807","nonce":1462942067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":1462942067},{"type":"v1","addr":"172.21.15.104:6805","nonce":1462942067}]},"public_addr":"172.21.15.104:6801/1462942067","cluster_addr":"172.21.15.104:6803/1462942067","heartbeat_back_addr":"172.21.15.104:6807/1462942067","heartbeat_front_addr":"172.21.15.104:6805/1462942067","state":["exists","up"]},{"osd":2,"uuid":"b318a512-4190-42a2-8104-dcc4026ad8ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":2829260735},{"type":"v1","addr":"172.21.15.117:6801","nonce":2829260735}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":2829260735},{"type":"v1","addr":"172.21.15.117:6803","nonce":2829260735}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":2829260735},{"type":"v1","addr":"172.21.15.117:6807","nonce":2829260735}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":2829260735},{"type":"v1","addr":"172.21.15.117:6805","nonce":2829260735}]},"public_addr":"172.21.15.117:6801/2829260735","cluster_addr":"172.21.15.117:6803/2829260735","heartbeat_back_addr":"172.21.15.117:6807/2829260735","heartbeat_front_addr":"172.21.15.117:6805/2829260735","state":["exists","up"]},{"osd":3,"uuid":"ec6490cb-a715-4c12-a54a-225aae26fcb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":2823712137},{"type":"v1","addr":"172.21.15.45:6811","nonce":2823712137}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":2823712137},{"type":"v1","addr":"172.21.15.45:6813","nonce":2823712137}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":2823712137},{"type":"v1","addr":"172.21.15.45:6817","nonce":2823712137}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":2823712137},{"type":"v1","addr":"172.21.15.45:6815","nonce":2823712137}]},"public_addr":"172.21.15.45:6811/2823712137","cluster_addr":"172.21.15.45:6813/2823712137","heartbeat_back_addr":"172.21.15.45:6817/2823712137","heartbeat_front_addr":"172.21.15.45:6815/2823712137","state":["exists","up"]},{"osd":4,"uuid":"5d91d568-0250-4aae-8dc2-a0e079f92062","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":1396278127},{"type":"v1","addr":"172.21.15.104:6809","nonce":1396278127}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":1396278127},{"type":"v1","addr":"172.21.15.104:6811","nonce":1396278127}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":1396278127},{"type":"v1","addr":"172.21.15.104:6815","nonce":1396278127}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":1396278127},{"type":"v1","addr":"172.21.15.104:6813","nonce":1396278127}]},"public_addr":"172.21.15.104:6809/1396278127","cluster_addr":"172.21.15.104:6811/1396278127","heartbeat_back_addr":"172.21.15.104:6815/1396278127","heartbeat_front_addr":"172.21.15.104:6813/1396278127","state":["exists","up"]},{"osd":5,"uuid":"ac510b43-6fb2-4d0f-bc5f-16ac080d9e9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":4145720047},{"type":"v1","addr":"172.21.15.117:6809","nonce":4145720047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":4145720047},{"type":"v1","addr":"172.21.15.117:6811","nonce":4145720047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":4145720047},{"type":"v1","addr":"172.21.15.117:6815","nonce":4145720047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":4145720047},{"type":"v1","addr":"172.21.15.117:6813","nonce":4145720047}]},"public_addr":"172.21.15.117:6809/4145720047","cluster_addr":"172.21.15.117:6811/4145720047","heartbeat_back_addr":"172.21.15.117:6815/4145720047","heartbeat_front_addr":"172.21.15.117:6813/4145720047","state":["exists","up"]},{"osd":6,"uuid":"a41adc18-d88a-419e-b2c2-b089c1f32472","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":3254229176},{"type":"v1","addr":"172.21.15.45:6819","nonce":3254229176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":3254229176},{"type":"v1","addr":"172.21.15.45:6821","nonce":3254229176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":3254229176},{"type":"v1","addr":"172.21.15.45:6825","nonce":3254229176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":3254229176},{"type":"v1","addr":"172.21.15.45:6823","nonce":3254229176}]},"public_addr":"172.21.15.45:6819/3254229176","cluster_addr":"172.21.15.45:6821/3254229176","heartbeat_back_addr":"172.21.15.45:6825/3254229176","heartbeat_front_addr":"172.21.15.45:6823/3254229176","state":["exists","up"]},{"osd":7,"uuid":"b71712a4-7cf6-454d-9fd1-565e3df21c1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":4151077186},{"type":"v1","addr":"172.21.15.104:6817","nonce":4151077186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":4151077186},{"type":"v1","addr":"172.21.15.104:6819","nonce":4151077186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":4151077186},{"type":"v1","addr":"172.21.15.104:6823","nonce":4151077186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":4151077186},{"type":"v1","addr":"172.21.15.104:6821","nonce":4151077186}]},"public_addr":"172.21.15.104:6817/4151077186","cluster_addr":"172.21.15.104:6819/4151077186","heartbeat_back_addr":"172.21.15.104:6823/4151077186","heartbeat_front_addr":"172.21.15.104:6821/4151077186","state":["exists","up"]},{"osd":8,"uuid":"9e572aef-59a8-450e-a92e-df9dff4d57a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":1500286830},{"type":"v1","addr":"172.21.15.117:6817","nonce":1500286830}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":1500286830},{"type":"v1","addr":"172.21.15.117:6819","nonce":1500286830}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":1500286830},{"type":"v1","addr":"172.21.15.117:6823","nonce":1500286830}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":1500286830},{"type":"v1","addr":"172.21.15.117:6821","nonce":1500286830}]},"public_addr":"172.21.15.117:6817/1500286830","cluster_addr":"172.21.15.117:6819/1500286830","heartbeat_back_addr":"172.21.15.117:6823/1500286830","heartbeat_front_addr":"172.21.15.117:6821/1500286830","state":["exists","up"]},{"osd":9,"uuid":"014e9598-c413-4cc4-9bb8-61c6e8f59d30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":3106734074},{"type":"v1","addr":"172.21.15.45:6827","nonce":3106734074}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":3106734074},{"type":"v1","addr":"172.21.15.45:6829","nonce":3106734074}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":3106734074},{"type":"v1","addr":"172.21.15.45:6833","nonce":3106734074}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":3106734074},{"type":"v1","addr":"172.21.15.45:6831","nonce":3106734074}]},"public_addr":"172.21.15.45:6827/3106734074","cluster_addr":"172.21.15.45:6829/3106734074","heartbeat_back_addr":"172.21.15.45:6833/3106734074","heartbeat_front_addr":"172.21.15.45:6831/3106734074","state":["exists","up"]},{"osd":10,"uuid":"2deba364-3e39-47be-8a1f-8256df5b82d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":1404794652},{"type":"v1","addr":"172.21.15.104:6825","nonce":1404794652}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":1404794652},{"type":"v1","addr":"172.21.15.104:6827","nonce":1404794652}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":1404794652},{"type":"v1","addr":"172.21.15.104:6831","nonce":1404794652}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":1404794652},{"type":"v1","addr":"172.21.15.104:6829","nonce":1404794652}]},"public_addr":"172.21.15.104:6825/1404794652","cluster_addr":"172.21.15.104:6827/1404794652","heartbeat_back_addr":"172.21.15.104:6831/1404794652","heartbeat_front_addr":"172.21.15.104:6829/1404794652","state":["exists","up"]},{"osd":11,"uuid":"3fda4fea-59ad-4d1a-9fe6-fcb30e31fd7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":3384003243},{"type":"v1","addr":"172.21.15.117:6825","nonce":3384003243}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":3384003243},{"type":"v1","addr":"172.21.15.117:6827","nonce":3384003243}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":3384003243},{"type":"v1","addr":"172.21.15.117:6831","nonce":3384003243}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":3384003243},{"type":"v1","addr":"172.21.15.117:6829","nonce":3384003243}]},"public_addr":"172.21.15.117:6825/3384003243","cluster_addr":"172.21.15.117:6827/3384003243","heartbeat_back_addr":"172.21.15.117:6831/3384003243","heartbeat_front_addr":"172.21.15.117:6829/3384003243","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:57:14.799608+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:57:39.884668+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:58:03.327057+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:58:37.314531+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:59:03.667650+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:59:28.462823+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T17:00:02.968574+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T17:00:29.559190+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T17:00:55.291007+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T17:01:29.815977+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T17:01:57.015580+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T17:02:23.233151+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.45:0/4074982598":"2024-04-17T16:55:19.948001+0000","172.21.15.45:0/2443572576":"2024-04-17T16:55:19.948001+0000","172.21.15.45:6801/1983630136":"2024-04-17T16:55:19.948001+0000","172.21.15.45:6800/1983630136":"2024-04-17T16:55:19.948001+0000","172.21.15.45:0/3150465248":"2024-04-17T16:54:43.692466+0000","172.21.15.45:0/556738623":"2024-04-17T16:54:43.692466+0000","172.21.15.45:0/445676297":"2024-04-17T16:55:19.948001+0000","172.21.15.45:6800/2553659622":"2024-04-17T16:54:43.692466+0000","172.21.15.45:6801/2553659622":"2024-04-17T16:54:43.692466+0000","172.21.15.45:0/1184615408":"2024-04-17T16:54:43.692466+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-16T17:03:42.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:42.636+0000 7fb707200640 1 -- 172.21.15.45:0/120829070 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fb6e80a3bb0 msgr2=0x7fb6e80a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:42.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:42.636+0000 7fb707200640 1 --2- 172.21.15.45:0/120829070 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fb6e80a3bb0 0x7fb6e80a6070 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fb7081050f0 tx=0x7fb6f8005f90 comp rx=0 tx=0).stop 2024-04-16T17:03:42.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:42.636+0000 7fb707200640 1 -- 172.21.15.45:0/120829070 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb708105290 msgr2=0x7fb70819a8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:42.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:42.636+0000 7fb707200640 1 --2- 172.21.15.45:0/120829070 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb708105290 0x7fb70819a8c0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fb6f00325f0 tx=0x7fb6f0035d10 comp rx=0 tx=0).stop 2024-04-16T17:03:42.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:42.637+0000 7fb707200640 1 -- 172.21.15.45:0/120829070 shutdown_connections 2024-04-16T17:03:42.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:42.637+0000 7fb707200640 1 --2- 172.21.15.45:0/120829070 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fb6e80a3bb0 0x7fb6e80a6070 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:42.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:42.637+0000 7fb707200640 1 --2- 172.21.15.45:0/120829070 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb708105c50 0x7fb7081a1930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:42.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:42.637+0000 7fb707200640 1 --2- 172.21.15.45:0/120829070 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb708105290 0x7fb70819a8c0 secure :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fb6f00325f0 tx=0x7fb6f0035d10 comp rx=0 tx=0).stop 2024-04-16T17:03:42.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:42.637+0000 7fb707200640 1 --2- 172.21.15.45:0/120829070 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb7081040a0 0x7fb70819a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:42.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:42.637+0000 7fb707200640 1 -- 172.21.15.45:0/120829070 >> 172.21.15.45:0/120829070 conn(0x7fb7080ff610 msgr2=0x7fb708108590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:03:42.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:42.637+0000 7fb707200640 1 -- 172.21.15.45:0/120829070 shutdown_connections 2024-04-16T17:03:42.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:42.637+0000 7fb707200640 1 -- 172.21.15.45:0/120829070 wait complete. 2024-04-16T17:03:42.659 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-16T16:58:06.166659+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-16T17:03:22.304198+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-16T17:03:23.616450+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-16T17:03:27.226289+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-16T17:03:42.659 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-16T17:03:42.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:42.888+0000 7f93ab200640 1 -- 172.21.15.45:0/3250736334 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f93a4105250 msgr2=0x7f93a41056d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:42.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:42.888+0000 7f93ab200640 1 --2- 172.21.15.45:0/3250736334 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f93a4105250 0x7f93a41056d0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f9394009900 tx=0x7f939402bc50 comp rx=0 tx=0).stop 2024-04-16T17:03:42.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:42.888+0000 7f93ab200640 1 -- 172.21.15.45:0/3250736334 shutdown_connections 2024-04-16T17:03:42.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:42.888+0000 7f93ab200640 1 --2- 172.21.15.45:0/3250736334 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f93a4105c10 0x7f93a410c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:42.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:42.888+0000 7f93ab200640 1 --2- 172.21.15.45:0/3250736334 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f93a4105250 0x7f93a41056d0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:42.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:42.888+0000 7f93ab200640 1 --2- 172.21.15.45:0/3250736334 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f93a4104060 0x7f93a4104480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:42.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:42.888+0000 7f93ab200640 1 -- 172.21.15.45:0/3250736334 >> 172.21.15.45:0/3250736334 conn(0x7f93a40ff5d0 msgr2=0x7f93a4101a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:03:42.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:42.888+0000 7f93ab200640 1 -- 172.21.15.45:0/3250736334 shutdown_connections 2024-04-16T17:03:42.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:42.888+0000 7f93ab200640 1 -- 172.21.15.45:0/3250736334 wait complete. 2024-04-16T17:03:42.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:42.889+0000 7f93ab200640 1 Processor -- start 2024-04-16T17:03:42.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:42.890+0000 7f93ab200640 1 -- start start 2024-04-16T17:03:42.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:42.890+0000 7f93ab200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f93a4104060 0x7f93a419a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:42.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:42.890+0000 7f93ab200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f93a4105250 0x7f93a419a980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:42.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:42.890+0000 7f93ab200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f93a4105c10 0x7f93a41a19a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:42.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:42.890+0000 7f93ab200640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f93a410fcf0 con 0x7f93a4105c10 2024-04-16T17:03:42.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:42.890+0000 7f93ab200640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f93a410fb70 con 0x7f93a4104060 2024-04-16T17:03:42.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:42.890+0000 7f93ab200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f93a410fe70 con 0x7f93a4105250 2024-04-16T17:03:42.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:42.890+0000 7f93a9400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f93a4105250 0x7f93a419a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:42.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:42.890+0000 7f93aa800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f93a4105c10 0x7f93a41a19a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:42.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:42.890+0000 7f93a9400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f93a4105250 0x7f93a419a980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:51524/0 (socket says 172.21.15.45:51524) 2024-04-16T17:03:42.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:42.890+0000 7f93a9e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f93a4104060 0x7f93a419a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:42.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:42.890+0000 7f93a9400640 1 -- 172.21.15.45:0/2149093964 learned_addr learned my addr 172.21.15.45:0/2149093964 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:03:42.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:42.891+0000 7f93a9400640 1 -- 172.21.15.45:0/2149093964 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f93a4104060 msgr2=0x7f93a419a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:42.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:42.891+0000 7f93a9400640 1 --2- 172.21.15.45:0/2149093964 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f93a4104060 0x7f93a419a440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:42.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:42.891+0000 7f93a9400640 1 -- 172.21.15.45:0/2149093964 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f93a4105c10 msgr2=0x7f93a41a19a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:42.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:42.891+0000 7f93a9400640 1 --2- 172.21.15.45:0/2149093964 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f93a4105c10 0x7f93a41a19a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:42.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:42.891+0000 7f93a9400640 1 -- 172.21.15.45:0/2149093964 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93a41a2010 con 0x7f93a4105250 2024-04-16T17:03:42.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:42.891+0000 7f93a9e00640 1 --2- 172.21.15.45:0/2149093964 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f93a4104060 0x7f93a419a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:03:42.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:42.891+0000 7f93a9400640 1 --2- 172.21.15.45:0/2149093964 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f93a4105250 0x7f93a419a980 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f9394009230 tx=0x7f9394005dd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:03:42.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:42.891+0000 7f939ea00640 1 -- 172.21.15.45:0/2149093964 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9394042020 con 0x7f93a4105250 2024-04-16T17:03:42.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:42.891+0000 7f939ea00640 1 -- 172.21.15.45:0/2149093964 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f93940054e0 con 0x7f93a4105250 2024-04-16T17:03:42.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:42.891+0000 7f93ab200640 1 -- 172.21.15.45:0/2149093964 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93a41a22a0 con 0x7f93a4105250 2024-04-16T17:03:42.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:42.892+0000 7f939ea00640 1 -- 172.21.15.45:0/2149093964 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f939403e460 con 0x7f93a4105250 2024-04-16T17:03:42.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:42.892+0000 7f93ab200640 1 -- 172.21.15.45:0/2149093964 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93a41a27f0 con 0x7f93a4105250 2024-04-16T17:03:42.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:42.893+0000 7f93ab200640 1 -- 172.21.15.45:0/2149093964 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f93a410b380 con 0x7f93a4105250 2024-04-16T17:03:42.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:42.895+0000 7f939ea00640 1 -- 172.21.15.45:0/2149093964 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f9394045020 con 0x7f93a4105250 2024-04-16T17:03:42.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:42.896+0000 7f939ea00640 1 --2- 172.21.15.45:0/2149093964 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f938c0a3bb0 0x7f938c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:42.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:42.896+0000 7f939ea00640 1 -- 172.21.15.45:0/2149093964 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f93940f2770 con 0x7f93a4105250 2024-04-16T17:03:42.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:42.896+0000 7f93a9e00640 1 --2- 172.21.15.45:0/2149093964 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f938c0a3bb0 0x7f938c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:42.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:42.897+0000 7f93a9e00640 1 --2- 172.21.15.45:0/2149093964 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f938c0a3bb0 0x7f938c0a6070 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f93a41050b0 tx=0x7f939800b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:03:42.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:42.901+0000 7f939ea00640 1 -- 172.21.15.45:0/2149093964 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f939404c020 con 0x7f93a4105250 2024-04-16T17:03:43.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:43.109+0000 7f93ab200640 1 -- 172.21.15.45:0/2149093964 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f93a4062c00 con 0x7f93a4105250 2024-04-16T17:03:43.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:43.110+0000 7f939ea00640 1 -- 172.21.15.45:0/2149093964 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v90) ==== 93+0+10 (secure 0 0 0) 0x7f93940beeb0 con 0x7f93a4105250 2024-04-16T17:03:43.110 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 1 2024-04-16T17:03:43.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:43.112+0000 7f93ab200640 1 -- 172.21.15.45:0/2149093964 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f938c0a3bb0 msgr2=0x7f938c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:43.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:43.112+0000 7f93ab200640 1 --2- 172.21.15.45:0/2149093964 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f938c0a3bb0 0x7f938c0a6070 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f93a41050b0 tx=0x7f939800b390 comp rx=0 tx=0).stop 2024-04-16T17:03:43.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:43.112+0000 7f93ab200640 1 -- 172.21.15.45:0/2149093964 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f93a4105250 msgr2=0x7f93a419a980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:43.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:43.112+0000 7f93ab200640 1 --2- 172.21.15.45:0/2149093964 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f93a4105250 0x7f93a419a980 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f9394009230 tx=0x7f9394005dd0 comp rx=0 tx=0).stop 2024-04-16T17:03:43.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:43.112+0000 7f93ab200640 1 -- 172.21.15.45:0/2149093964 shutdown_connections 2024-04-16T17:03:43.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:43.112+0000 7f93ab200640 1 --2- 172.21.15.45:0/2149093964 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f938c0a3bb0 0x7f938c0a6070 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:43.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:43.112+0000 7f93ab200640 1 --2- 172.21.15.45:0/2149093964 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f93a4105250 0x7f93a419a980 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:43.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:43.112+0000 7f93ab200640 1 --2- 172.21.15.45:0/2149093964 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f93a4105c10 0x7f93a41a19a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:43.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:43.112+0000 7f93ab200640 1 --2- 172.21.15.45:0/2149093964 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f93a4104060 0x7f93a419a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:43.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:43.112+0000 7f93ab200640 1 -- 172.21.15.45:0/2149093964 >> 172.21.15.45:0/2149093964 conn(0x7f93a40ff5d0 msgr2=0x7f93a4108550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:03:43.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:43.112+0000 7f93ab200640 1 -- 172.21.15.45:0/2149093964 shutdown_connections 2024-04-16T17:03:43.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:43.112+0000 7f93ab200640 1 -- 172.21.15.45:0/2149093964 wait complete. 2024-04-16T17:03:43.135 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-16T17:03:43.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:43.357+0000 7f8af1400640 1 -- 172.21.15.45:0/2171461812 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8aec105330 msgr2=0x7f8aec1057b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:43.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:43.357+0000 7f8af1400640 1 --2- 172.21.15.45:0/2171461812 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8aec105330 0x7f8aec1057b0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f8ad4009900 tx=0x7f8ad402bc30 comp rx=0 tx=0).stop 2024-04-16T17:03:43.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:43.357+0000 7f8af1400640 1 -- 172.21.15.45:0/2171461812 shutdown_connections 2024-04-16T17:03:43.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:43.357+0000 7f8af1400640 1 --2- 172.21.15.45:0/2171461812 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8aec105cf0 0x7f8aec10c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:43.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:43.357+0000 7f8af1400640 1 --2- 172.21.15.45:0/2171461812 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8aec105330 0x7f8aec1057b0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:43.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:43.357+0000 7f8af1400640 1 --2- 172.21.15.45:0/2171461812 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8aec104140 0x7f8aec104560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:43.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:43.357+0000 7f8af1400640 1 -- 172.21.15.45:0/2171461812 >> 172.21.15.45:0/2171461812 conn(0x7f8aec0ff6b0 msgr2=0x7f8aec101af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:03:43.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:43.357+0000 7f8af1400640 1 -- 172.21.15.45:0/2171461812 shutdown_connections 2024-04-16T17:03:43.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:43.358+0000 7f8af1400640 1 -- 172.21.15.45:0/2171461812 wait complete. 2024-04-16T17:03:43.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:43.358+0000 7f8af1400640 1 Processor -- start 2024-04-16T17:03:43.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:43.359+0000 7f8af1400640 1 -- start start 2024-04-16T17:03:43.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:43.359+0000 7f8af1400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8aec104140 0x7f8aec19a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:43.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:43.359+0000 7f8af1400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8aec105330 0x7f8aec19a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:43.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:43.359+0000 7f8af1400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8aec105cf0 0x7f8aec1a1980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:43.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:43.359+0000 7f8af1400640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f8aec10fdd0 con 0x7f8aec104140 2024-04-16T17:03:43.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:43.359+0000 7f8af1400640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f8aec10fc50 con 0x7f8aec105330 2024-04-16T17:03:43.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:43.359+0000 7f8af1400640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f8aec10ff50 con 0x7f8aec105cf0 2024-04-16T17:03:43.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:43.359+0000 7f8aebe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8aec104140 0x7f8aec19a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:43.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:43.359+0000 7f8aebe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8aec104140 0x7f8aec19a3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:44336/0 (socket says 172.21.15.45:44336) 2024-04-16T17:03:43.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:43.359+0000 7f8aebe00640 1 -- 172.21.15.45:0/3916268659 learned_addr learned my addr 172.21.15.45:0/3916268659 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:03:43.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:43.359+0000 7f8ae3e00640 1 --2- 172.21.15.45:0/3916268659 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8aec105330 0x7f8aec19a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:43.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:43.359+0000 7f8af0a00640 1 --2- 172.21.15.45:0/3916268659 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8aec105cf0 0x7f8aec1a1980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:43.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:43.360+0000 7f8aebe00640 1 -- 172.21.15.45:0/3916268659 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8aec105cf0 msgr2=0x7f8aec1a1980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:43.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:43.360+0000 7f8aebe00640 1 --2- 172.21.15.45:0/3916268659 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8aec105cf0 0x7f8aec1a1980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:43.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:43.360+0000 7f8aebe00640 1 -- 172.21.15.45:0/3916268659 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8aec105330 msgr2=0x7f8aec19a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:43.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:43.360+0000 7f8aebe00640 1 --2- 172.21.15.45:0/3916268659 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8aec105330 0x7f8aec19a910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:43.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:43.360+0000 7f8aebe00640 1 -- 172.21.15.45:0/3916268659 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8aec1a1ff0 con 0x7f8aec104140 2024-04-16T17:03:43.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:43.360+0000 7f8aebe00640 1 --2- 172.21.15.45:0/3916268659 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8aec104140 0x7f8aec19a3d0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f8adc00e9c0 tx=0x7f8adc00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:03:43.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:43.360+0000 7f8ae9600640 1 -- 172.21.15.45:0/3916268659 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8adc00cd80 con 0x7f8aec104140 2024-04-16T17:03:43.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:43.360+0000 7f8ae9600640 1 -- 172.21.15.45:0/3916268659 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8adc0035f0 con 0x7f8aec104140 2024-04-16T17:03:43.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:43.361+0000 7f8af1400640 1 -- 172.21.15.45:0/3916268659 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8aec1a22e0 con 0x7f8aec104140 2024-04-16T17:03:43.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:43.361+0000 7f8ae9600640 1 -- 172.21.15.45:0/3916268659 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8adc010690 con 0x7f8aec104140 2024-04-16T17:03:43.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:43.361+0000 7f8af1400640 1 -- 172.21.15.45:0/3916268659 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8aec1a2850 con 0x7f8aec104140 2024-04-16T17:03:43.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:43.362+0000 7f8af1400640 1 -- 172.21.15.45:0/3916268659 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8aec10b4e0 con 0x7f8aec104140 2024-04-16T17:03:43.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:43.364+0000 7f8ae9600640 1 -- 172.21.15.45:0/3916268659 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f8adc0037a0 con 0x7f8aec104140 2024-04-16T17:03:43.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:43.365+0000 7f8ae9600640 1 --2- 172.21.15.45:0/3916268659 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f8ad00a3c30 0x7f8ad00a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:43.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:43.365+0000 7f8ae9600640 1 -- 172.21.15.45:0/3916268659 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f8adc0d0a00 con 0x7f8aec104140 2024-04-16T17:03:43.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:43.366+0000 7f8ae3e00640 1 --2- 172.21.15.45:0/3916268659 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f8ad00a3c30 0x7f8ad00a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:43.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:43.366+0000 7f8ae3e00640 1 --2- 172.21.15.45:0/3916268659 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f8ad00a3c30 0x7f8ad00a60f0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f8ad402c7b0 tx=0x7f8ad4005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:03:43.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:43.369+0000 7f8ae9600640 1 -- 172.21.15.45:0/3916268659 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8adc09d140 con 0x7f8aec104140 2024-04-16T17:03:43.478 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:03:43 smithi045 ceph-mon[31676]: pgmap v331: 129 pgs: 129 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T17:03:43.478 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:03:43 smithi045 ceph-mon[31676]: from='client.? 172.21.15.45:0/120829070' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T17:03:43.478 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:03:43 smithi045 ceph-mon[31676]: from='client.? 172.21.15.45:0/2149093964' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-16T17:03:43.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:03:43 smithi117 ceph-mon[36730]: pgmap v331: 129 pgs: 129 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T17:03:43.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:03:43 smithi117 ceph-mon[36730]: from='client.? 172.21.15.45:0/120829070' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T17:03:43.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:03:43 smithi117 ceph-mon[36730]: from='client.? 172.21.15.45:0/2149093964' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-16T17:03:43.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:43.575+0000 7f8af1400640 1 -- 172.21.15.45:0/3916268659 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f8aec062c00 con 0x7f8aec104140 2024-04-16T17:03:43.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:43.576+0000 7f8ae9600640 1 -- 172.21.15.45:0/3916268659 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v90) ==== 104+0+11 (secure 0 0 0) 0x7f8adc09ca40 con 0x7f8aec104140 2024-04-16T17:03:43.577 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 32 2024-04-16T17:03:43.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:43.579+0000 7f8af1400640 1 -- 172.21.15.45:0/3916268659 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f8ad00a3c30 msgr2=0x7f8ad00a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:43.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:43.579+0000 7f8af1400640 1 --2- 172.21.15.45:0/3916268659 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f8ad00a3c30 0x7f8ad00a60f0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f8ad402c7b0 tx=0x7f8ad4005fd0 comp rx=0 tx=0).stop 2024-04-16T17:03:43.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:43.579+0000 7f8af1400640 1 -- 172.21.15.45:0/3916268659 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8aec104140 msgr2=0x7f8aec19a3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:43.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:43.579+0000 7f8af1400640 1 --2- 172.21.15.45:0/3916268659 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8aec104140 0x7f8aec19a3d0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f8adc00e9c0 tx=0x7f8adc00ee80 comp rx=0 tx=0).stop 2024-04-16T17:03:43.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:43.579+0000 7f8af1400640 1 -- 172.21.15.45:0/3916268659 shutdown_connections 2024-04-16T17:03:43.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:43.579+0000 7f8af1400640 1 --2- 172.21.15.45:0/3916268659 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f8ad00a3c30 0x7f8ad00a60f0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:43.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:43.579+0000 7f8af1400640 1 --2- 172.21.15.45:0/3916268659 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8aec105cf0 0x7f8aec1a1980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:43.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:43.579+0000 7f8af1400640 1 --2- 172.21.15.45:0/3916268659 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8aec105330 0x7f8aec19a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:43.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:43.579+0000 7f8af1400640 1 --2- 172.21.15.45:0/3916268659 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8aec104140 0x7f8aec19a3d0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:43.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:43.579+0000 7f8af1400640 1 -- 172.21.15.45:0/3916268659 >> 172.21.15.45:0/3916268659 conn(0x7f8aec0ff6b0 msgr2=0x7f8aec1086b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:03:43.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:43.580+0000 7f8af1400640 1 -- 172.21.15.45:0/3916268659 shutdown_connections 2024-04-16T17:03:43.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:43.580+0000 7f8af1400640 1 -- 172.21.15.45:0/3916268659 wait complete. 2024-04-16T17:03:43.598 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-16T17:03:43.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:03:43 smithi104 ceph-mon[36457]: pgmap v331: 129 pgs: 129 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T17:03:43.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:03:43 smithi104 ceph-mon[36457]: from='client.? 172.21.15.45:0/120829070' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T17:03:43.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:03:43 smithi104 ceph-mon[36457]: from='client.? 172.21.15.45:0/2149093964' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-16T17:03:43.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:43.830+0000 7f4e22400640 1 -- 172.21.15.45:0/957921929 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4e1c108540 msgr2=0x7f4e1c108960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:43.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:43.830+0000 7f4e22400640 1 --2- 172.21.15.45:0/957921929 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4e1c108540 0x7f4e1c108960 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f4e10009900 tx=0x7f4e1002bc40 comp rx=0 tx=0).stop 2024-04-16T17:03:43.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:43.830+0000 7f4e22400640 1 -- 172.21.15.45:0/957921929 shutdown_connections 2024-04-16T17:03:43.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:43.830+0000 7f4e22400640 1 --2- 172.21.15.45:0/957921929 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4e1c109730 0x7f4e1c109bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:43.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:43.830+0000 7f4e22400640 1 --2- 172.21.15.45:0/957921929 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4e1c10a0f0 0x7f4e1c110980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:43.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:43.830+0000 7f4e22400640 1 --2- 172.21.15.45:0/957921929 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4e1c108540 0x7f4e1c108960 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:43.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:43.830+0000 7f4e22400640 1 -- 172.21.15.45:0/957921929 >> 172.21.15.45:0/957921929 conn(0x7f4e1c0774d0 msgr2=0x7f4e1c079910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:03:43.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:43.830+0000 7f4e22400640 1 -- 172.21.15.45:0/957921929 shutdown_connections 2024-04-16T17:03:43.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:43.830+0000 7f4e22400640 1 -- 172.21.15.45:0/957921929 wait complete. 2024-04-16T17:03:43.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:43.831+0000 7f4e22400640 1 Processor -- start 2024-04-16T17:03:43.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:43.831+0000 7f4e22400640 1 -- start start 2024-04-16T17:03:43.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:43.832+0000 7f4e22400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4e1c108540 0x7f4e1c19e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:43.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:43.832+0000 7f4e22400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4e1c109730 0x7f4e1c19edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:43.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:43.832+0000 7f4e22400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4e1c10a0f0 0x7f4e1c1a5e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:43.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:43.832+0000 7f4e22400640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f4e1c1141d0 con 0x7f4e1c108540 2024-04-16T17:03:43.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:43.832+0000 7f4e22400640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f4e1c114050 con 0x7f4e1c10a0f0 2024-04-16T17:03:43.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:43.832+0000 7f4e22400640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f4e1c114350 con 0x7f4e1c109730 2024-04-16T17:03:43.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:43.832+0000 7f4e21000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4e1c108540 0x7f4e1c19e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:43.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:43.832+0000 7f4e21000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4e1c108540 0x7f4e1c19e870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:44348/0 (socket says 172.21.15.45:44348) 2024-04-16T17:03:43.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:43.832+0000 7f4e21000640 1 -- 172.21.15.45:0/3660024322 learned_addr learned my addr 172.21.15.45:0/3660024322 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:03:43.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:43.832+0000 7f4e1be00640 1 --2- 172.21.15.45:0/3660024322 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4e1c109730 0x7f4e1c19edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:43.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:43.832+0000 7f4e21a00640 1 --2- 172.21.15.45:0/3660024322 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4e1c10a0f0 0x7f4e1c1a5e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:43.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:43.832+0000 7f4e21000640 1 -- 172.21.15.45:0/3660024322 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4e1c109730 msgr2=0x7f4e1c19edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:43.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:43.832+0000 7f4e21000640 1 --2- 172.21.15.45:0/3660024322 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4e1c109730 0x7f4e1c19edb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:43.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:43.832+0000 7f4e21000640 1 -- 172.21.15.45:0/3660024322 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4e1c10a0f0 msgr2=0x7f4e1c1a5e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:43.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:43.832+0000 7f4e21000640 1 --2- 172.21.15.45:0/3660024322 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4e1c10a0f0 0x7f4e1c1a5e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:43.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:43.832+0000 7f4e21000640 1 -- 172.21.15.45:0/3660024322 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e1c1a6490 con 0x7f4e1c108540 2024-04-16T17:03:43.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:43.833+0000 7f4e21000640 1 --2- 172.21.15.45:0/3660024322 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4e1c108540 0x7f4e1c19e870 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f4e10000c00 tx=0x7f4e1002c650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:03:43.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:43.833+0000 7f4e19600640 1 -- 172.21.15.45:0/3660024322 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e10041070 con 0x7f4e1c108540 2024-04-16T17:03:43.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:43.833+0000 7f4e19600640 1 -- 172.21.15.45:0/3660024322 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4e10004450 con 0x7f4e1c108540 2024-04-16T17:03:43.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:43.833+0000 7f4e22400640 1 -- 172.21.15.45:0/3660024322 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e1c1a6720 con 0x7f4e1c108540 2024-04-16T17:03:43.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:43.833+0000 7f4e19600640 1 -- 172.21.15.45:0/3660024322 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e10034440 con 0x7f4e1c108540 2024-04-16T17:03:43.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:43.833+0000 7f4e22400640 1 -- 172.21.15.45:0/3660024322 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e1c1a6c10 con 0x7f4e1c108540 2024-04-16T17:03:43.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:43.835+0000 7f4e22400640 1 -- 172.21.15.45:0/3660024322 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e1c10f860 con 0x7f4e1c108540 2024-04-16T17:03:43.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:43.837+0000 7f4e19600640 1 -- 172.21.15.45:0/3660024322 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f4e10044070 con 0x7f4e1c108540 2024-04-16T17:03:43.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:43.838+0000 7f4e19600640 1 --2- 172.21.15.45:0/3660024322 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f4e040a3bb0 0x7f4e040a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:43.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:43.838+0000 7f4e19600640 1 -- 172.21.15.45:0/3660024322 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f4e100f2180 con 0x7f4e1c108540 2024-04-16T17:03:43.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:43.838+0000 7f4e1be00640 1 --2- 172.21.15.45:0/3660024322 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f4e040a3bb0 0x7f4e040a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:43.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:43.839+0000 7f4e1be00640 1 --2- 172.21.15.45:0/3660024322 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f4e040a3bb0 0x7f4e040a6070 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f4e1c19fd80 tx=0x7f4e08005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:03:43.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:43.842+0000 7f4e19600640 1 -- 172.21.15.45:0/3660024322 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4e100be300 con 0x7f4e1c108540 2024-04-16T17:03:44.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:44.047+0000 7f4e22400640 1 -- 172.21.15.45:0/3660024322 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f4e1c062c00 con 0x7f4e1c108540 2024-04-16T17:03:44.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:44.048+0000 7f4e19600640 1 -- 172.21.15.45:0/3660024322 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v90) ==== 100+0+11 (secure 0 0 0) 0x7f4e100c2600 con 0x7f4e1c108540 2024-04-16T17:03:44.049 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 64 2024-04-16T17:03:44.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:44.050+0000 7f4e22400640 1 -- 172.21.15.45:0/3660024322 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f4e040a3bb0 msgr2=0x7f4e040a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:44.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:44.050+0000 7f4e22400640 1 --2- 172.21.15.45:0/3660024322 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f4e040a3bb0 0x7f4e040a6070 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f4e1c19fd80 tx=0x7f4e08005f90 comp rx=0 tx=0).stop 2024-04-16T17:03:44.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:44.050+0000 7f4e22400640 1 -- 172.21.15.45:0/3660024322 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4e1c108540 msgr2=0x7f4e1c19e870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:44.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:44.050+0000 7f4e22400640 1 --2- 172.21.15.45:0/3660024322 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4e1c108540 0x7f4e1c19e870 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f4e10000c00 tx=0x7f4e1002c650 comp rx=0 tx=0).stop 2024-04-16T17:03:44.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:44.051+0000 7f4e22400640 1 -- 172.21.15.45:0/3660024322 shutdown_connections 2024-04-16T17:03:44.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:44.051+0000 7f4e22400640 1 --2- 172.21.15.45:0/3660024322 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f4e040a3bb0 0x7f4e040a6070 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:44.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:44.051+0000 7f4e22400640 1 --2- 172.21.15.45:0/3660024322 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4e1c109730 0x7f4e1c19edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:44.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:44.051+0000 7f4e22400640 1 --2- 172.21.15.45:0/3660024322 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4e1c10a0f0 0x7f4e1c1a5e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:44.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:44.051+0000 7f4e22400640 1 --2- 172.21.15.45:0/3660024322 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4e1c108540 0x7f4e1c19e870 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:44.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:44.051+0000 7f4e22400640 1 -- 172.21.15.45:0/3660024322 >> 172.21.15.45:0/3660024322 conn(0x7f4e1c0774d0 msgr2=0x7f4e1c10ca30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:03:44.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:44.051+0000 7f4e22400640 1 -- 172.21.15.45:0/3660024322 shutdown_connections 2024-04-16T17:03:44.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:44.051+0000 7f4e22400640 1 -- 172.21.15.45:0/3660024322 wait complete. 2024-04-16T17:03:44.073 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-16T17:03:44.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:44.311+0000 7f7a6f800640 1 -- 172.21.15.45:0/762145359 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7a68104260 msgr2=0x7f7a68104680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:44.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:44.311+0000 7f7a6f800640 1 --2- 172.21.15.45:0/762145359 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7a68104260 0x7f7a68104680 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f7a5c009860 tx=0x7f7a5c02bb70 comp rx=0 tx=0).stop 2024-04-16T17:03:44.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:44.311+0000 7f7a6f800640 1 -- 172.21.15.45:0/762145359 shutdown_connections 2024-04-16T17:03:44.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:44.312+0000 7f7a6f800640 1 --2- 172.21.15.45:0/762145359 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7a68105e10 0x7f7a6810c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:44.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:44.312+0000 7f7a6f800640 1 --2- 172.21.15.45:0/762145359 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7a68105450 0x7f7a681058d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:44.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:44.312+0000 7f7a6f800640 1 --2- 172.21.15.45:0/762145359 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7a68104260 0x7f7a68104680 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:44.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:44.312+0000 7f7a6f800640 1 -- 172.21.15.45:0/762145359 >> 172.21.15.45:0/762145359 conn(0x7f7a680ff7b0 msgr2=0x7f7a68101c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:03:44.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:44.312+0000 7f7a6f800640 1 -- 172.21.15.45:0/762145359 shutdown_connections 2024-04-16T17:03:44.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:44.312+0000 7f7a6f800640 1 -- 172.21.15.45:0/762145359 wait complete. 2024-04-16T17:03:44.315 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:44.313+0000 7f7a6f800640 1 Processor -- start 2024-04-16T17:03:44.315 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:44.313+0000 7f7a6f800640 1 -- start start 2024-04-16T17:03:44.315 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:44.314+0000 7f7a6f800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7a68104260 0x7f7a6819a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:44.315 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:44.314+0000 7f7a6f800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7a68105450 0x7f7a6819aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:44.315 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:44.314+0000 7f7a6f800640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7a68105e10 0x7f7a681a1b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:44.315 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:44.314+0000 7f7a6f800640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f7a6810fef0 con 0x7f7a68105450 2024-04-16T17:03:44.315 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:44.314+0000 7f7a6f800640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f7a6810fd70 con 0x7f7a68105e10 2024-04-16T17:03:44.315 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:44.314+0000 7f7a6f800640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f7a68110070 con 0x7f7a68104260 2024-04-16T17:03:44.315 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:44.314+0000 7f7a67e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7a68105450 0x7f7a6819aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:44.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:44.314+0000 7f7a6d800640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7a68105e10 0x7f7a681a1b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:44.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:44.314+0000 7f7a67e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7a68105450 0x7f7a6819aab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:44368/0 (socket says 172.21.15.45:44368) 2024-04-16T17:03:44.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:44.314+0000 7f7a67e00640 1 -- 172.21.15.45:0/2079128721 learned_addr learned my addr 172.21.15.45:0/2079128721 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:03:44.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:44.314+0000 7f7a6ce00640 1 --2- 172.21.15.45:0/2079128721 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7a68104260 0x7f7a6819a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:44.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:44.314+0000 7f7a6d800640 1 -- 172.21.15.45:0/2079128721 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7a68104260 msgr2=0x7f7a6819a570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:44.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:44.314+0000 7f7a6d800640 1 --2- 172.21.15.45:0/2079128721 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7a68104260 0x7f7a6819a570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:44.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:44.314+0000 7f7a6d800640 1 -- 172.21.15.45:0/2079128721 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7a68105450 msgr2=0x7f7a6819aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:44.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:44.314+0000 7f7a6d800640 1 --2- 172.21.15.45:0/2079128721 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7a68105450 0x7f7a6819aab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:44.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:44.314+0000 7f7a6d800640 1 -- 172.21.15.45:0/2079128721 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a681a2190 con 0x7f7a68105e10 2024-04-16T17:03:44.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:44.315+0000 7f7a6d800640 1 --2- 172.21.15.45:0/2079128721 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7a68105e10 0x7f7a681a1b20 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f7a60009fd0 tx=0x7f7a6000ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:03:44.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:44.315+0000 7f7a65600640 1 -- 172.21.15.45:0/2079128721 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a6000ce20 con 0x7f7a68105e10 2024-04-16T17:03:44.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:44.315+0000 7f7a65600640 1 -- 172.21.15.45:0/2079128721 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7a60005020 con 0x7f7a68105e10 2024-04-16T17:03:44.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:44.315+0000 7f7a65600640 1 -- 172.21.15.45:0/2079128721 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a60010720 con 0x7f7a68105e10 2024-04-16T17:03:44.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:44.315+0000 7f7a6f800640 1 -- 172.21.15.45:0/2079128721 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a681a2480 con 0x7f7a68105e10 2024-04-16T17:03:44.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:44.315+0000 7f7a6f800640 1 -- 172.21.15.45:0/2079128721 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a681a29d0 con 0x7f7a68105e10 2024-04-16T17:03:44.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:44.317+0000 7f7a6f800640 1 -- 172.21.15.45:0/2079128721 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a6810b580 con 0x7f7a68105e10 2024-04-16T17:03:44.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:44.318+0000 7f7a65600640 1 -- 172.21.15.45:0/2079128721 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f7a60005540 con 0x7f7a68105e10 2024-04-16T17:03:44.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:44.319+0000 7f7a65600640 1 --2- 172.21.15.45:0/2079128721 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f7a500a3bb0 0x7f7a500a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:44.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:44.320+0000 7f7a65600640 1 -- 172.21.15.45:0/2079128721 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f7a600d0c70 con 0x7f7a68105e10 2024-04-16T17:03:44.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:44.320+0000 7f7a6ce00640 1 --2- 172.21.15.45:0/2079128721 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f7a500a3bb0 0x7f7a500a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:44.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:44.321+0000 7f7a6ce00640 1 --2- 172.21.15.45:0/2079128721 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f7a500a3bb0 0x7f7a500a6070 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f7a5c0349e0 tx=0x7f7a5c034ab0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:03:44.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:44.324+0000 7f7a65600640 1 -- 172.21.15.45:0/2079128721 <== mon.2 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7a6009d3b0 con 0x7f7a68105e10 2024-04-16T17:03:44.478 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:03:44 smithi045 ceph-mon[31676]: from='client.? 172.21.15.45:0/3916268659' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-16T17:03:44.478 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:03:44 smithi045 ceph-mon[31676]: from='client.? 172.21.15.45:0/3660024322' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-16T17:03:44.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:44.530+0000 7f7a6f800640 1 -- 172.21.15.45:0/2079128721 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f7a68062c00 con 0x7f7a68105e10 2024-04-16T17:03:44.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:44.531+0000 7f7a65600640 1 -- 172.21.15.45:0/2079128721 <== mon.2 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v90) ==== 103+0+11 (secure 0 0 0) 0x7f7a6009ccb0 con 0x7f7a68105e10 2024-04-16T17:03:44.532 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 32 2024-04-16T17:03:44.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:44.534+0000 7f7a6f800640 1 -- 172.21.15.45:0/2079128721 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f7a500a3bb0 msgr2=0x7f7a500a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:44.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:44.534+0000 7f7a6f800640 1 --2- 172.21.15.45:0/2079128721 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f7a500a3bb0 0x7f7a500a6070 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f7a5c0349e0 tx=0x7f7a5c034ab0 comp rx=0 tx=0).stop 2024-04-16T17:03:44.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:44.534+0000 7f7a6f800640 1 -- 172.21.15.45:0/2079128721 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7a68105e10 msgr2=0x7f7a681a1b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:44.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:44.534+0000 7f7a6f800640 1 --2- 172.21.15.45:0/2079128721 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7a68105e10 0x7f7a681a1b20 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f7a60009fd0 tx=0x7f7a6000ef10 comp rx=0 tx=0).stop 2024-04-16T17:03:44.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:44.534+0000 7f7a6f800640 1 -- 172.21.15.45:0/2079128721 shutdown_connections 2024-04-16T17:03:44.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:44.534+0000 7f7a6f800640 1 --2- 172.21.15.45:0/2079128721 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f7a500a3bb0 0x7f7a500a6070 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:44.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:44.534+0000 7f7a6f800640 1 --2- 172.21.15.45:0/2079128721 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7a68105e10 0x7f7a681a1b20 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:44.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:44.534+0000 7f7a6f800640 1 --2- 172.21.15.45:0/2079128721 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7a68105450 0x7f7a6819aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:44.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:44.534+0000 7f7a6f800640 1 --2- 172.21.15.45:0/2079128721 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7a68104260 0x7f7a6819a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:44.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:44.534+0000 7f7a6f800640 1 -- 172.21.15.45:0/2079128721 >> 172.21.15.45:0/2079128721 conn(0x7f7a680ff7b0 msgr2=0x7f7a68108750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:03:44.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:44.534+0000 7f7a6f800640 1 -- 172.21.15.45:0/2079128721 shutdown_connections 2024-04-16T17:03:44.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:44.534+0000 7f7a6f800640 1 -- 172.21.15.45:0/2079128721 wait complete. 2024-04-16T17:03:44.554 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-16T17:03:44.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:03:44 smithi117 ceph-mon[36730]: from='client.? 172.21.15.45:0/3916268659' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-16T17:03:44.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:03:44 smithi117 ceph-mon[36730]: from='client.? 172.21.15.45:0/3660024322' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-16T17:03:44.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:03:44 smithi104 ceph-mon[36457]: from='client.? 172.21.15.45:0/3916268659' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-16T17:03:44.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:03:44 smithi104 ceph-mon[36457]: from='client.? 172.21.15.45:0/3660024322' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-16T17:03:44.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:44.778+0000 7fda9b200640 1 -- 172.21.15.45:0/1423665610 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fda9c10a0f0 msgr2=0x7fda9c110980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:44.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:44.778+0000 7fda9b200640 1 --2- 172.21.15.45:0/1423665610 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fda9c10a0f0 0x7fda9c110980 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fda94009ca0 tx=0x7fda9402c100 comp rx=0 tx=0).stop 2024-04-16T17:03:44.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:44.778+0000 7fda9b200640 1 -- 172.21.15.45:0/1423665610 shutdown_connections 2024-04-16T17:03:44.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:44.778+0000 7fda9b200640 1 --2- 172.21.15.45:0/1423665610 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fda9c109730 0x7fda9c109bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:44.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:44.778+0000 7fda9b200640 1 --2- 172.21.15.45:0/1423665610 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fda9c10a0f0 0x7fda9c110980 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:44.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:44.778+0000 7fda9b200640 1 --2- 172.21.15.45:0/1423665610 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fda9c108540 0x7fda9c108960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:44.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:44.778+0000 7fda9b200640 1 -- 172.21.15.45:0/1423665610 >> 172.21.15.45:0/1423665610 conn(0x7fda9c0774d0 msgr2=0x7fda9c079910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:03:44.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:44.778+0000 7fda9b200640 1 -- 172.21.15.45:0/1423665610 shutdown_connections 2024-04-16T17:03:44.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:44.778+0000 7fda9b200640 1 -- 172.21.15.45:0/1423665610 wait complete. 2024-04-16T17:03:44.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:44.779+0000 7fda9b200640 1 Processor -- start 2024-04-16T17:03:44.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:44.779+0000 7fda9b200640 1 -- start start 2024-04-16T17:03:44.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:44.779+0000 7fda9b200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fda9c108540 0x7fda9c19e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:44.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:44.779+0000 7fda9b200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fda9c109730 0x7fda9c19edf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:44.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:44.780+0000 7fda9b200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fda9c10a0f0 0x7fda9c1a5d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:44.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:44.780+0000 7fda9b200640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fda9c1141a0 con 0x7fda9c108540 2024-04-16T17:03:44.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:44.780+0000 7fda9b200640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fda9c114020 con 0x7fda9c109730 2024-04-16T17:03:44.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:44.780+0000 7fda9b200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fda9c114320 con 0x7fda9c10a0f0 2024-04-16T17:03:44.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:44.780+0000 7fda99e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fda9c108540 0x7fda9c19e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:44.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:44.780+0000 7fda99e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fda9c108540 0x7fda9c19e8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:44384/0 (socket says 172.21.15.45:44384) 2024-04-16T17:03:44.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:44.780+0000 7fda99e00640 1 -- 172.21.15.45:0/3072679464 learned_addr learned my addr 172.21.15.45:0/3072679464 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:03:44.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:44.780+0000 7fda99400640 1 --2- 172.21.15.45:0/3072679464 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fda9c109730 0x7fda9c19edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:44.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:44.780+0000 7fda9a800640 1 --2- 172.21.15.45:0/3072679464 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fda9c10a0f0 0x7fda9c1a5d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:44.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:44.780+0000 7fda99e00640 1 -- 172.21.15.45:0/3072679464 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fda9c10a0f0 msgr2=0x7fda9c1a5d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:44.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:44.780+0000 7fda99e00640 1 --2- 172.21.15.45:0/3072679464 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fda9c10a0f0 0x7fda9c1a5d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:44.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:44.780+0000 7fda99e00640 1 -- 172.21.15.45:0/3072679464 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fda9c109730 msgr2=0x7fda9c19edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:44.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:44.780+0000 7fda99e00640 1 --2- 172.21.15.45:0/3072679464 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fda9c109730 0x7fda9c19edf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:44.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:44.780+0000 7fda99e00640 1 -- 172.21.15.45:0/3072679464 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda9c1a6390 con 0x7fda9c108540 2024-04-16T17:03:44.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:44.780+0000 7fda99400640 1 --2- 172.21.15.45:0/3072679464 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fda9c109730 0x7fda9c19edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:03:44.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:44.781+0000 7fda99e00640 1 --2- 172.21.15.45:0/3072679464 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fda9c108540 0x7fda9c19e8b0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fda8c00e9c0 tx=0x7fda8c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:03:44.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:44.781+0000 7fda92a00640 1 -- 172.21.15.45:0/3072679464 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda8c00cd80 con 0x7fda9c108540 2024-04-16T17:03:44.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:44.781+0000 7fda92a00640 1 -- 172.21.15.45:0/3072679464 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fda8c0035f0 con 0x7fda9c108540 2024-04-16T17:03:44.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:44.781+0000 7fda9b200640 1 -- 172.21.15.45:0/3072679464 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda9c1a6680 con 0x7fda9c108540 2024-04-16T17:03:44.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:44.781+0000 7fda92a00640 1 -- 172.21.15.45:0/3072679464 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda8c010690 con 0x7fda9c108540 2024-04-16T17:03:44.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:44.781+0000 7fda9b200640 1 -- 172.21.15.45:0/3072679464 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda9c1a6bd0 con 0x7fda9c108540 2024-04-16T17:03:44.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:44.782+0000 7fda9b200640 1 -- 172.21.15.45:0/3072679464 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda9c10f800 con 0x7fda9c108540 2024-04-16T17:03:44.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:44.784+0000 7fda92a00640 1 -- 172.21.15.45:0/3072679464 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fda8c0037a0 con 0x7fda9c108540 2024-04-16T17:03:44.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:44.786+0000 7fda92a00640 1 --2- 172.21.15.45:0/3072679464 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fda800a3bb0 0x7fda800a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:44.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:44.786+0000 7fda92a00640 1 -- 172.21.15.45:0/3072679464 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7fda8c0d0aa0 con 0x7fda9c108540 2024-04-16T17:03:44.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:44.786+0000 7fda99400640 1 --2- 172.21.15.45:0/3072679464 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fda800a3bb0 0x7fda800a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:44.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:44.787+0000 7fda99400640 1 --2- 172.21.15.45:0/3072679464 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fda800a3bb0 0x7fda800a6070 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fda880095f0 tx=0x7fda88005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:03:44.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:44.790+0000 7fda92a00640 1 -- 172.21.15.45:0/3072679464 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fda8c09d1e0 con 0x7fda9c108540 2024-04-16T17:03:45.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:45.010+0000 7fda9b200640 1 -- 172.21.15.45:0/3072679464 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fda9c02d620 con 0x7fda9c108540 2024-04-16T17:03:45.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:45.011+0000 7fda92a00640 1 -- 172.21.15.45:0/3072679464 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v90) ==== 81+0+590 (secure 0 0 0) 0x7fda8c09cae0 con 0x7fda9c108540 2024-04-16T17:03:45.012 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.45:0/4074982598 2024-04-17T16:55:19.948001+0000 2024-04-16T17:03:45.012 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.45:0/2443572576 2024-04-17T16:55:19.948001+0000 2024-04-16T17:03:45.012 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.45:6801/1983630136 2024-04-17T16:55:19.948001+0000 2024-04-16T17:03:45.012 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.45:6800/1983630136 2024-04-17T16:55:19.948001+0000 2024-04-16T17:03:45.012 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.45:0/3150465248 2024-04-17T16:54:43.692466+0000 2024-04-16T17:03:45.012 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.45:0/556738623 2024-04-17T16:54:43.692466+0000 2024-04-16T17:03:45.013 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.45:0/445676297 2024-04-17T16:55:19.948001+0000 2024-04-16T17:03:45.013 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.45:6800/2553659622 2024-04-17T16:54:43.692466+0000 2024-04-16T17:03:45.013 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.45:6801/2553659622 2024-04-17T16:54:43.692466+0000 2024-04-16T17:03:45.013 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.45:0/1184615408 2024-04-17T16:54:43.692466+0000 2024-04-16T17:03:45.013 INFO:teuthology.orchestra.run.smithi045.stderr:listed 10 entries 2024-04-16T17:03:45.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:45.014+0000 7fda9b200640 1 -- 172.21.15.45:0/3072679464 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fda800a3bb0 msgr2=0x7fda800a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:45.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:45.014+0000 7fda9b200640 1 --2- 172.21.15.45:0/3072679464 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fda800a3bb0 0x7fda800a6070 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fda880095f0 tx=0x7fda88005f90 comp rx=0 tx=0).stop 2024-04-16T17:03:45.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:45.014+0000 7fda9b200640 1 -- 172.21.15.45:0/3072679464 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fda9c108540 msgr2=0x7fda9c19e8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:45.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:45.014+0000 7fda9b200640 1 --2- 172.21.15.45:0/3072679464 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fda9c108540 0x7fda9c19e8b0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fda8c00e9c0 tx=0x7fda8c00ee80 comp rx=0 tx=0).stop 2024-04-16T17:03:45.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:45.014+0000 7fda9b200640 1 -- 172.21.15.45:0/3072679464 shutdown_connections 2024-04-16T17:03:45.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:45.014+0000 7fda9b200640 1 --2- 172.21.15.45:0/3072679464 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fda800a3bb0 0x7fda800a6070 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:45.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:45.014+0000 7fda9b200640 1 --2- 172.21.15.45:0/3072679464 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fda9c10a0f0 0x7fda9c1a5d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:45.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:45.014+0000 7fda9b200640 1 --2- 172.21.15.45:0/3072679464 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fda9c109730 0x7fda9c19edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:45.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:45.014+0000 7fda9b200640 1 --2- 172.21.15.45:0/3072679464 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fda9c108540 0x7fda9c19e8b0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:45.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:45.014+0000 7fda9b200640 1 -- 172.21.15.45:0/3072679464 >> 172.21.15.45:0/3072679464 conn(0x7fda9c0774d0 msgr2=0x7fda9c10c9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:03:45.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:45.014+0000 7fda9b200640 1 -- 172.21.15.45:0/3072679464 shutdown_connections 2024-04-16T17:03:45.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:45.015+0000 7fda9b200640 1 -- 172.21.15.45:0/3072679464 wait complete. 2024-04-16T17:03:45.035 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-04-16T17:03:45.035 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-16T17:03:45.085 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-16T17:03:45.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:45.313+0000 7f142b200640 1 -- 172.21.15.45:0/1381909081 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f142c109670 msgr2=0x7f142c109af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:45.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:45.313+0000 7f142b200640 1 --2- 172.21.15.45:0/1381909081 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f142c109670 0x7f142c109af0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f1414009900 tx=0x7f141402bc50 comp rx=0 tx=0).stop 2024-04-16T17:03:45.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:45.313+0000 7f142b200640 1 -- 172.21.15.45:0/1381909081 shutdown_connections 2024-04-16T17:03:45.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:45.313+0000 7f142b200640 1 --2- 172.21.15.45:0/1381909081 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f142c10a030 0x7f142c1108c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:45.315 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:45.313+0000 7f142b200640 1 --2- 172.21.15.45:0/1381909081 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f142c109670 0x7f142c109af0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:45.315 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:45.313+0000 7f142b200640 1 --2- 172.21.15.45:0/1381909081 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f142c108480 0x7f142c1088a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:45.315 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:45.313+0000 7f142b200640 1 -- 172.21.15.45:0/1381909081 >> 172.21.15.45:0/1381909081 conn(0x7f142c077440 msgr2=0x7f142c079860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:03:45.315 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:45.313+0000 7f142b200640 1 -- 172.21.15.45:0/1381909081 shutdown_connections 2024-04-16T17:03:45.315 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:45.313+0000 7f142b200640 1 -- 172.21.15.45:0/1381909081 wait complete. 2024-04-16T17:03:45.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:45.314+0000 7f142b200640 1 Processor -- start 2024-04-16T17:03:45.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:45.314+0000 7f142b200640 1 -- start start 2024-04-16T17:03:45.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:45.315+0000 7f142b200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f142c108480 0x7f142c19e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:45.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:45.315+0000 7f142b200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f142c109670 0x7f142c19eca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:45.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:45.315+0000 7f142b200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f142c10a030 0x7f142c1a5d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:45.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:45.315+0000 7f142b200640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f142c114160 con 0x7f142c108480 2024-04-16T17:03:45.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:45.315+0000 7f142b200640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f142c113fe0 con 0x7f142c109670 2024-04-16T17:03:45.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:45.315+0000 7f142b200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f142c1142e0 con 0x7f142c10a030 2024-04-16T17:03:45.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:45.315+0000 7f1429e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f142c108480 0x7f142c19e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:45.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:45.315+0000 7f1429e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f142c108480 0x7f142c19e760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:44408/0 (socket says 172.21.15.45:44408) 2024-04-16T17:03:45.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:45.315+0000 7f1429e00640 1 -- 172.21.15.45:0/3897194833 learned_addr learned my addr 172.21.15.45:0/3897194833 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:03:45.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:45.315+0000 7f142a800640 1 --2- 172.21.15.45:0/3897194833 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f142c10a030 0x7f142c1a5d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:45.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:45.315+0000 7f1423e00640 1 --2- 172.21.15.45:0/3897194833 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f142c109670 0x7f142c19eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:45.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:45.315+0000 7f1429e00640 1 -- 172.21.15.45:0/3897194833 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f142c10a030 msgr2=0x7f142c1a5d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:45.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:45.315+0000 7f1429e00640 1 --2- 172.21.15.45:0/3897194833 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f142c10a030 0x7f142c1a5d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:45.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:45.315+0000 7f1429e00640 1 -- 172.21.15.45:0/3897194833 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f142c109670 msgr2=0x7f142c19eca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:45.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:45.315+0000 7f1429e00640 1 --2- 172.21.15.45:0/3897194833 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f142c109670 0x7f142c19eca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:45.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:45.315+0000 7f1429e00640 1 -- 172.21.15.45:0/3897194833 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f142c1a6380 con 0x7f142c108480 2024-04-16T17:03:45.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:45.315+0000 7f1423e00640 1 --2- 172.21.15.45:0/3897194833 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f142c109670 0x7f142c19eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:03:45.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:45.316+0000 7f1429e00640 1 --2- 172.21.15.45:0/3897194833 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f142c108480 0x7f142c19e760 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f141c00e9c0 tx=0x7f141c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:03:45.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:45.316+0000 7f1422a00640 1 -- 172.21.15.45:0/3897194833 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f141c00cd80 con 0x7f142c108480 2024-04-16T17:03:45.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:45.316+0000 7f1422a00640 1 -- 172.21.15.45:0/3897194833 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f141c0035f0 con 0x7f142c108480 2024-04-16T17:03:45.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:45.316+0000 7f142b200640 1 -- 172.21.15.45:0/3897194833 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f142c1a6670 con 0x7f142c108480 2024-04-16T17:03:45.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:45.316+0000 7f1422a00640 1 -- 172.21.15.45:0/3897194833 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f141c010690 con 0x7f142c108480 2024-04-16T17:03:45.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:45.316+0000 7f142b200640 1 -- 172.21.15.45:0/3897194833 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f142c1a6b90 con 0x7f142c108480 2024-04-16T17:03:45.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:45.317+0000 7f142b200640 1 -- 172.21.15.45:0/3897194833 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f142c10f7a0 con 0x7f142c108480 2024-04-16T17:03:45.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:45.319+0000 7f1422a00640 1 -- 172.21.15.45:0/3897194833 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f141c0037a0 con 0x7f142c108480 2024-04-16T17:03:45.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:45.321+0000 7f1422a00640 1 --2- 172.21.15.45:0/3897194833 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f14100a3c30 0x7f14100a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:45.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:45.321+0000 7f1422a00640 1 -- 172.21.15.45:0/3897194833 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f141c0d0a30 con 0x7f142c108480 2024-04-16T17:03:45.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:45.321+0000 7f1423e00640 1 --2- 172.21.15.45:0/3897194833 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f14100a3c30 0x7f14100a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:45.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:45.322+0000 7f1423e00640 1 --2- 172.21.15.45:0/3897194833 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f14100a3c30 0x7f14100a60f0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f141402c260 tx=0x7f1414005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:03:45.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:45.325+0000 7f1422a00640 1 -- 172.21.15.45:0/3897194833 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f141c09d170 con 0x7f142c108480 2024-04-16T17:03:45.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:45.532+0000 7f142b200640 1 -- 172.21.15.45:0/3897194833 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f142c062c00 con 0x7f142c108480 2024-04-16T17:03:45.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:45.534+0000 7f1422a00640 1 -- 172.21.15.45:0/3897194833 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+24164 (secure 0 0 0) 0x7f141c09ca70 con 0x7f142c108480 2024-04-16T17:03:45.536 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-16T17:03:45.536 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":90,"fsid":"b1fecd38-fc11-11ee-bc90-c7b262605968","created":"2024-04-16T16:54:02.385906+0000","modified":"2024-04-16T17:03:29.720565+0000","last_up_change":"2024-04-16T17:02:25.162624+0000","last_in_change":"2024-04-16T17:02:04.903274+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-16T16:58:06.166659+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-16T17:03:22.304198+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-16T17:03:23.616450+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-16T17:03:27.226289+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b655de2c-59f8-4fd4-ac98-0cc222229f56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":2290170881},{"type":"v1","addr":"172.21.15.45:6803","nonce":2290170881}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":2290170881},{"type":"v1","addr":"172.21.15.45:6805","nonce":2290170881}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":2290170881},{"type":"v1","addr":"172.21.15.45:6809","nonce":2290170881}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":2290170881},{"type":"v1","addr":"172.21.15.45:6807","nonce":2290170881}]},"public_addr":"172.21.15.45:6803/2290170881","cluster_addr":"172.21.15.45:6805/2290170881","heartbeat_back_addr":"172.21.15.45:6809/2290170881","heartbeat_front_addr":"172.21.15.45:6807/2290170881","state":["exists","up"]},{"osd":1,"uuid":"ebd450ec-08fd-4c3d-8b24-4e10634ddb7e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6800","nonce":1462942067},{"type":"v1","addr":"172.21.15.104:6801","nonce":1462942067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":1462942067},{"type":"v1","addr":"172.21.15.104:6803","nonce":1462942067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":1462942067},{"type":"v1","addr":"172.21.15.104:6807","nonce":1462942067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":1462942067},{"type":"v1","addr":"172.21.15.104:6805","nonce":1462942067}]},"public_addr":"172.21.15.104:6801/1462942067","cluster_addr":"172.21.15.104:6803/1462942067","heartbeat_back_addr":"172.21.15.104:6807/1462942067","heartbeat_front_addr":"172.21.15.104:6805/1462942067","state":["exists","up"]},{"osd":2,"uuid":"b318a512-4190-42a2-8104-dcc4026ad8ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":2829260735},{"type":"v1","addr":"172.21.15.117:6801","nonce":2829260735}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":2829260735},{"type":"v1","addr":"172.21.15.117:6803","nonce":2829260735}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":2829260735},{"type":"v1","addr":"172.21.15.117:6807","nonce":2829260735}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":2829260735},{"type":"v1","addr":"172.21.15.117:6805","nonce":2829260735}]},"public_addr":"172.21.15.117:6801/2829260735","cluster_addr":"172.21.15.117:6803/2829260735","heartbeat_back_addr":"172.21.15.117:6807/2829260735","heartbeat_front_addr":"172.21.15.117:6805/2829260735","state":["exists","up"]},{"osd":3,"uuid":"ec6490cb-a715-4c12-a54a-225aae26fcb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":2823712137},{"type":"v1","addr":"172.21.15.45:6811","nonce":2823712137}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":2823712137},{"type":"v1","addr":"172.21.15.45:6813","nonce":2823712137}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":2823712137},{"type":"v1","addr":"172.21.15.45:6817","nonce":2823712137}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":2823712137},{"type":"v1","addr":"172.21.15.45:6815","nonce":2823712137}]},"public_addr":"172.21.15.45:6811/2823712137","cluster_addr":"172.21.15.45:6813/2823712137","heartbeat_back_addr":"172.21.15.45:6817/2823712137","heartbeat_front_addr":"172.21.15.45:6815/2823712137","state":["exists","up"]},{"osd":4,"uuid":"5d91d568-0250-4aae-8dc2-a0e079f92062","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":1396278127},{"type":"v1","addr":"172.21.15.104:6809","nonce":1396278127}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":1396278127},{"type":"v1","addr":"172.21.15.104:6811","nonce":1396278127}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":1396278127},{"type":"v1","addr":"172.21.15.104:6815","nonce":1396278127}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":1396278127},{"type":"v1","addr":"172.21.15.104:6813","nonce":1396278127}]},"public_addr":"172.21.15.104:6809/1396278127","cluster_addr":"172.21.15.104:6811/1396278127","heartbeat_back_addr":"172.21.15.104:6815/1396278127","heartbeat_front_addr":"172.21.15.104:6813/1396278127","state":["exists","up"]},{"osd":5,"uuid":"ac510b43-6fb2-4d0f-bc5f-16ac080d9e9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":4145720047},{"type":"v1","addr":"172.21.15.117:6809","nonce":4145720047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":4145720047},{"type":"v1","addr":"172.21.15.117:6811","nonce":4145720047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":4145720047},{"type":"v1","addr":"172.21.15.117:6815","nonce":4145720047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":4145720047},{"type":"v1","addr":"172.21.15.117:6813","nonce":4145720047}]},"public_addr":"172.21.15.117:6809/4145720047","cluster_addr":"172.21.15.117:6811/4145720047","heartbeat_back_addr":"172.21.15.117:6815/4145720047","heartbeat_front_addr":"172.21.15.117:6813/4145720047","state":["exists","up"]},{"osd":6,"uuid":"a41adc18-d88a-419e-b2c2-b089c1f32472","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":3254229176},{"type":"v1","addr":"172.21.15.45:6819","nonce":3254229176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":3254229176},{"type":"v1","addr":"172.21.15.45:6821","nonce":3254229176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":3254229176},{"type":"v1","addr":"172.21.15.45:6825","nonce":3254229176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":3254229176},{"type":"v1","addr":"172.21.15.45:6823","nonce":3254229176}]},"public_addr":"172.21.15.45:6819/3254229176","cluster_addr":"172.21.15.45:6821/3254229176","heartbeat_back_addr":"172.21.15.45:6825/3254229176","heartbeat_front_addr":"172.21.15.45:6823/3254229176","state":["exists","up"]},{"osd":7,"uuid":"b71712a4-7cf6-454d-9fd1-565e3df21c1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":4151077186},{"type":"v1","addr":"172.21.15.104:6817","nonce":4151077186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":4151077186},{"type":"v1","addr":"172.21.15.104:6819","nonce":4151077186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":4151077186},{"type":"v1","addr":"172.21.15.104:6823","nonce":4151077186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":4151077186},{"type":"v1","addr":"172.21.15.104:6821","nonce":4151077186}]},"public_addr":"172.21.15.104:6817/4151077186","cluster_addr":"172.21.15.104:6819/4151077186","heartbeat_back_addr":"172.21.15.104:6823/4151077186","heartbeat_front_addr":"172.21.15.104:6821/4151077186","state":["exists","up"]},{"osd":8,"uuid":"9e572aef-59a8-450e-a92e-df9dff4d57a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":1500286830},{"type":"v1","addr":"172.21.15.117:6817","nonce":1500286830}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":1500286830},{"type":"v1","addr":"172.21.15.117:6819","nonce":1500286830}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":1500286830},{"type":"v1","addr":"172.21.15.117:6823","nonce":1500286830}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":1500286830},{"type":"v1","addr":"172.21.15.117:6821","nonce":1500286830}]},"public_addr":"172.21.15.117:6817/1500286830","cluster_addr":"172.21.15.117:6819/1500286830","heartbeat_back_addr":"172.21.15.117:6823/1500286830","heartbeat_front_addr":"172.21.15.117:6821/1500286830","state":["exists","up"]},{"osd":9,"uuid":"014e9598-c413-4cc4-9bb8-61c6e8f59d30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":3106734074},{"type":"v1","addr":"172.21.15.45:6827","nonce":3106734074}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":3106734074},{"type":"v1","addr":"172.21.15.45:6829","nonce":3106734074}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":3106734074},{"type":"v1","addr":"172.21.15.45:6833","nonce":3106734074}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":3106734074},{"type":"v1","addr":"172.21.15.45:6831","nonce":3106734074}]},"public_addr":"172.21.15.45:6827/3106734074","cluster_addr":"172.21.15.45:6829/3106734074","heartbeat_back_addr":"172.21.15.45:6833/3106734074","heartbeat_front_addr":"172.21.15.45:6831/3106734074","state":["exists","up"]},{"osd":10,"uuid":"2deba364-3e39-47be-8a1f-8256df5b82d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":1404794652},{"type":"v1","addr":"172.21.15.104:6825","nonce":1404794652}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":1404794652},{"type":"v1","addr":"172.21.15.104:6827","nonce":1404794652}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":1404794652},{"type":"v1","addr":"172.21.15.104:6831","nonce":1404794652}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":1404794652},{"type":"v1","addr":"172.21.15.104:6829","nonce":1404794652}]},"public_addr":"172.21.15.104:6825/1404794652","cluster_addr":"172.21.15.104:6827/1404794652","heartbeat_back_addr":"172.21.15.104:6831/1404794652","heartbeat_front_addr":"172.21.15.104:6829/1404794652","state":["exists","up"]},{"osd":11,"uuid":"3fda4fea-59ad-4d1a-9fe6-fcb30e31fd7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":3384003243},{"type":"v1","addr":"172.21.15.117:6825","nonce":3384003243}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":3384003243},{"type":"v1","addr":"172.21.15.117:6827","nonce":3384003243}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":3384003243},{"type":"v1","addr":"172.21.15.117:6831","nonce":3384003243}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":3384003243},{"type":"v1","addr":"172.21.15.117:6829","nonce":3384003243}]},"public_addr":"172.21.15.117:6825/3384003243","cluster_addr":"172.21.15.117:6827/3384003243","heartbeat_back_addr":"172.21.15.117:6831/3384003243","heartbeat_front_addr":"172.21.15.117:6829/3384003243","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:57:14.799608+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:57:39.884668+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:58:03.327057+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:58:37.314531+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:59:03.667650+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:59:28.462823+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T17:00:02.968574+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T17:00:29.559190+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T17:00:55.291007+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T17:01:29.815977+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T17:01:57.015580+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T17:02:23.233151+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.45:0/4074982598":"2024-04-17T16:55:19.948001+0000","172.21.15.45:0/2443572576":"2024-04-17T16:55:19.948001+0000","172.21.15.45:6801/1983630136":"2024-04-17T16:55:19.948001+0000","172.21.15.45:6800/1983630136":"2024-04-17T16:55:19.948001+0000","172.21.15.45:0/3150465248":"2024-04-17T16:54:43.692466+0000","172.21.15.45:0/556738623":"2024-04-17T16:54:43.692466+0000","172.21.15.45:0/445676297":"2024-04-17T16:55:19.948001+0000","172.21.15.45:6800/2553659622":"2024-04-17T16:54:43.692466+0000","172.21.15.45:6801/2553659622":"2024-04-17T16:54:43.692466+0000","172.21.15.45:0/1184615408":"2024-04-17T16:54:43.692466+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-16T17:03:45.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:45.537+0000 7f142b200640 1 -- 172.21.15.45:0/3897194833 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f14100a3c30 msgr2=0x7f14100a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:45.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:45.537+0000 7f142b200640 1 --2- 172.21.15.45:0/3897194833 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f14100a3c30 0x7f14100a60f0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f141402c260 tx=0x7f1414005fd0 comp rx=0 tx=0).stop 2024-04-16T17:03:45.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:45.537+0000 7f142b200640 1 -- 172.21.15.45:0/3897194833 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f142c108480 msgr2=0x7f142c19e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:45.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:45.537+0000 7f142b200640 1 --2- 172.21.15.45:0/3897194833 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f142c108480 0x7f142c19e760 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f141c00e9c0 tx=0x7f141c00ee80 comp rx=0 tx=0).stop 2024-04-16T17:03:45.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:45.537+0000 7f142b200640 1 -- 172.21.15.45:0/3897194833 shutdown_connections 2024-04-16T17:03:45.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:45.537+0000 7f142b200640 1 --2- 172.21.15.45:0/3897194833 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f14100a3c30 0x7f14100a60f0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:45.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:45.537+0000 7f142b200640 1 --2- 172.21.15.45:0/3897194833 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f142c10a030 0x7f142c1a5d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:45.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:45.537+0000 7f142b200640 1 --2- 172.21.15.45:0/3897194833 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f142c109670 0x7f142c19eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:45.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:45.537+0000 7f142b200640 1 --2- 172.21.15.45:0/3897194833 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f142c108480 0x7f142c19e760 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:45.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:45.537+0000 7f142b200640 1 -- 172.21.15.45:0/3897194833 >> 172.21.15.45:0/3897194833 conn(0x7f142c077440 msgr2=0x7f142c10c970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:03:45.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:45.537+0000 7f142b200640 1 -- 172.21.15.45:0/3897194833 shutdown_connections 2024-04-16T17:03:45.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:45.537+0000 7f142b200640 1 -- 172.21.15.45:0/3897194833 wait complete. 2024-04-16T17:03:45.554 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:03:45 smithi045 ceph-mon[31676]: pgmap v332: 129 pgs: 129 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T17:03:45.554 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:03:45 smithi045 ceph-mon[31676]: from='client.? 172.21.15.45:0/2079128721' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-16T17:03:45.554 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:03:45 smithi045 ceph-mon[31676]: from='client.? 172.21.15.45:0/3072679464' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-16T17:03:45.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:03:45 smithi117 ceph-mon[36730]: pgmap v332: 129 pgs: 129 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T17:03:45.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:03:45 smithi117 ceph-mon[36730]: from='client.? 172.21.15.45:0/2079128721' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-16T17:03:45.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:03:45 smithi117 ceph-mon[36730]: from='client.? 172.21.15.45:0/3072679464' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-16T17:03:45.558 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-16T16:58:06.166659+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-16T17:03:22.304198+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-16T17:03:23.616450+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-16T17:03:27.226289+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-16T17:03:45.558 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-16T17:03:45.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:03:45 smithi104 ceph-mon[36457]: pgmap v332: 129 pgs: 129 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T17:03:45.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:03:45 smithi104 ceph-mon[36457]: from='client.? 172.21.15.45:0/2079128721' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-16T17:03:45.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:03:45 smithi104 ceph-mon[36457]: from='client.? 172.21.15.45:0/3072679464' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-16T17:03:45.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:45.785+0000 7eff09000640 1 -- 172.21.15.45:0/2902117276 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7eff04108660 msgr2=0x7eff04108a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:45.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:45.785+0000 7eff09000640 1 --2- 172.21.15.45:0/2902117276 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7eff04108660 0x7eff04108a80 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7efef4009900 tx=0x7efef402bc40 comp rx=0 tx=0).stop 2024-04-16T17:03:45.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:45.785+0000 7eff09000640 1 -- 172.21.15.45:0/2902117276 shutdown_connections 2024-04-16T17:03:45.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:45.785+0000 7eff09000640 1 --2- 172.21.15.45:0/2902117276 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7eff0410a210 0x7eff04110aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:45.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:45.785+0000 7eff09000640 1 --2- 172.21.15.45:0/2902117276 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7eff04109850 0x7eff04109cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:45.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:45.785+0000 7eff09000640 1 --2- 172.21.15.45:0/2902117276 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7eff04108660 0x7eff04108a80 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:45.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:45.785+0000 7eff09000640 1 -- 172.21.15.45:0/2902117276 >> 172.21.15.45:0/2902117276 conn(0x7eff04077600 msgr2=0x7eff04079a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:03:45.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:45.785+0000 7eff09000640 1 -- 172.21.15.45:0/2902117276 shutdown_connections 2024-04-16T17:03:45.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:45.785+0000 7eff09000640 1 -- 172.21.15.45:0/2902117276 wait complete. 2024-04-16T17:03:45.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:45.786+0000 7eff09000640 1 Processor -- start 2024-04-16T17:03:45.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:45.786+0000 7eff09000640 1 -- start start 2024-04-16T17:03:45.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:45.786+0000 7eff09000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7eff04108660 0x7eff0419e940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:45.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:45.786+0000 7eff09000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7eff04109850 0x7eff0419ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:45.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:45.786+0000 7eff09000640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7eff0410a210 0x7eff041a5ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:45.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:45.786+0000 7eff09000640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7eff04114300 con 0x7eff04108660 2024-04-16T17:03:45.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:45.786+0000 7eff09000640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7eff04114180 con 0x7eff0410a210 2024-04-16T17:03:45.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:45.786+0000 7eff09000640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7eff04114480 con 0x7eff04109850 2024-04-16T17:03:45.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:45.787+0000 7eff01e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7eff04108660 0x7eff0419e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:45.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:45.787+0000 7eff01e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7eff04108660 0x7eff0419e940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:44426/0 (socket says 172.21.15.45:44426) 2024-04-16T17:03:45.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:45.787+0000 7eff01e00640 1 -- 172.21.15.45:0/843939065 learned_addr learned my addr 172.21.15.45:0/843939065 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:03:45.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:45.787+0000 7eff02800640 1 --2- 172.21.15.45:0/843939065 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7eff0410a210 0x7eff041a5ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:45.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:45.787+0000 7efef9400640 1 --2- 172.21.15.45:0/843939065 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7eff04109850 0x7eff0419ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:45.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:45.787+0000 7eff01e00640 1 -- 172.21.15.45:0/843939065 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7eff04109850 msgr2=0x7eff0419ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:45.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:45.787+0000 7eff01e00640 1 --2- 172.21.15.45:0/843939065 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7eff04109850 0x7eff0419ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:45.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:45.787+0000 7eff01e00640 1 -- 172.21.15.45:0/843939065 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7eff0410a210 msgr2=0x7eff041a5ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:45.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:45.787+0000 7eff01e00640 1 --2- 172.21.15.45:0/843939065 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7eff0410a210 0x7eff041a5ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:45.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:45.787+0000 7eff01e00640 1 -- 172.21.15.45:0/843939065 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff041a6560 con 0x7eff04108660 2024-04-16T17:03:45.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:45.787+0000 7eff01e00640 1 --2- 172.21.15.45:0/843939065 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7eff04108660 0x7eff0419e940 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7efef402efd0 tx=0x7efef402c660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:03:45.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:45.788+0000 7efefb400640 1 -- 172.21.15.45:0/843939065 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efef4041070 con 0x7eff04108660 2024-04-16T17:03:45.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:45.788+0000 7eff09000640 1 -- 172.21.15.45:0/843939065 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff041a67f0 con 0x7eff04108660 2024-04-16T17:03:45.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:45.788+0000 7efefb400640 1 -- 172.21.15.45:0/843939065 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efef4004450 con 0x7eff04108660 2024-04-16T17:03:45.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:45.788+0000 7eff09000640 1 -- 172.21.15.45:0/843939065 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff041a6d00 con 0x7eff04108660 2024-04-16T17:03:45.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:45.788+0000 7efefb400640 1 -- 172.21.15.45:0/843939065 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efef404b060 con 0x7eff04108660 2024-04-16T17:03:45.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:45.789+0000 7eff09000640 1 -- 172.21.15.45:0/843939065 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff0410f980 con 0x7eff04108660 2024-04-16T17:03:45.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:45.791+0000 7efefb400640 1 -- 172.21.15.45:0/843939065 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7efef4044070 con 0x7eff04108660 2024-04-16T17:03:45.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:45.792+0000 7efefb400640 1 --2- 172.21.15.45:0/843939065 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7efee80a3bb0 0x7efee80a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:45.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:45.792+0000 7efefb400640 1 -- 172.21.15.45:0/843939065 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7efef40f2550 con 0x7eff04108660 2024-04-16T17:03:45.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:45.793+0000 7efef9400640 1 --2- 172.21.15.45:0/843939065 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7efee80a3bb0 0x7efee80a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:45.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:45.793+0000 7efef9400640 1 --2- 172.21.15.45:0/843939065 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7efee80a3bb0 0x7efee80a6070 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7eff0419fe50 tx=0x7efeec005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:03:45.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:45.795+0000 7efefb400640 1 -- 172.21.15.45:0/843939065 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7efef40be640 con 0x7eff04108660 2024-04-16T17:03:46.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:46.000+0000 7eff09000640 1 -- 172.21.15.45:0/843939065 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7eff0402d870 con 0x7eff04108660 2024-04-16T17:03:46.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:46.001+0000 7efefb400640 1 -- 172.21.15.45:0/843939065 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v90) ==== 93+0+10 (secure 0 0 0) 0x7efef40ba020 con 0x7eff04108660 2024-04-16T17:03:46.002 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 1 2024-04-16T17:03:46.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:46.003+0000 7eff09000640 1 -- 172.21.15.45:0/843939065 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7efee80a3bb0 msgr2=0x7efee80a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:46.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:46.003+0000 7eff09000640 1 --2- 172.21.15.45:0/843939065 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7efee80a3bb0 0x7efee80a6070 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7eff0419fe50 tx=0x7efeec005f90 comp rx=0 tx=0).stop 2024-04-16T17:03:46.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:46.003+0000 7eff09000640 1 -- 172.21.15.45:0/843939065 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7eff04108660 msgr2=0x7eff0419e940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:46.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:46.003+0000 7eff09000640 1 --2- 172.21.15.45:0/843939065 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7eff04108660 0x7eff0419e940 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7efef402efd0 tx=0x7efef402c660 comp rx=0 tx=0).stop 2024-04-16T17:03:46.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:46.003+0000 7eff09000640 1 -- 172.21.15.45:0/843939065 shutdown_connections 2024-04-16T17:03:46.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:46.003+0000 7eff09000640 1 --2- 172.21.15.45:0/843939065 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7efee80a3bb0 0x7efee80a6070 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:46.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:46.003+0000 7eff09000640 1 --2- 172.21.15.45:0/843939065 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7eff04109850 0x7eff0419ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:46.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:46.003+0000 7eff09000640 1 --2- 172.21.15.45:0/843939065 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7eff0410a210 0x7eff041a5ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:46.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:46.003+0000 7eff09000640 1 --2- 172.21.15.45:0/843939065 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7eff04108660 0x7eff0419e940 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:46.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:46.003+0000 7eff09000640 1 -- 172.21.15.45:0/843939065 >> 172.21.15.45:0/843939065 conn(0x7eff04077600 msgr2=0x7eff0410cb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:03:46.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:46.004+0000 7eff09000640 1 -- 172.21.15.45:0/843939065 shutdown_connections 2024-04-16T17:03:46.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:46.004+0000 7eff09000640 1 -- 172.21.15.45:0/843939065 wait complete. 2024-04-16T17:03:46.024 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-16T17:03:46.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:46.258+0000 7fd057e00640 1 -- 172.21.15.45:0/477629479 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd0580769c0 msgr2=0x7fd058076de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:46.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:46.258+0000 7fd057e00640 1 --2- 172.21.15.45:0/477629479 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd0580769c0 0x7fd058076de0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fd050009ca0 tx=0x7fd05002c0f0 comp rx=0 tx=0).stop 2024-04-16T17:03:46.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:46.259+0000 7fd057e00640 1 -- 172.21.15.45:0/477629479 shutdown_connections 2024-04-16T17:03:46.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:46.259+0000 7fd057e00640 1 --2- 172.21.15.45:0/477629479 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd0580785a0 0x7fd058078e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:46.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:46.259+0000 7fd057e00640 1 --2- 172.21.15.45:0/477629479 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd058077c00 0x7fd058078060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:46.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:46.259+0000 7fd057e00640 1 --2- 172.21.15.45:0/477629479 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd0580769c0 0x7fd058076de0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:46.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:46.259+0000 7fd057e00640 1 -- 172.21.15.45:0/477629479 >> 172.21.15.45:0/477629479 conn(0x7fd0580ff6b0 msgr2=0x7fd058101af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:03:46.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:46.259+0000 7fd057e00640 1 -- 172.21.15.45:0/477629479 shutdown_connections 2024-04-16T17:03:46.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:46.259+0000 7fd057e00640 1 -- 172.21.15.45:0/477629479 wait complete. 2024-04-16T17:03:46.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:46.260+0000 7fd057e00640 1 Processor -- start 2024-04-16T17:03:46.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:46.260+0000 7fd057e00640 1 -- start start 2024-04-16T17:03:46.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:46.260+0000 7fd057e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd0580769c0 0x7fd05819e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:46.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:46.260+0000 7fd057e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd058077c00 0x7fd05819ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:46.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:46.260+0000 7fd057e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd0580785a0 0x7fd0581a5df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:46.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:46.260+0000 7fd057e00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fd058114190 con 0x7fd058077c00 2024-04-16T17:03:46.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:46.260+0000 7fd057e00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fd058114010 con 0x7fd0580785a0 2024-04-16T17:03:46.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:46.260+0000 7fd057e00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fd058114310 con 0x7fd0580769c0 2024-04-16T17:03:46.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:46.261+0000 7fd056000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd058077c00 0x7fd05819ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:46.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:46.261+0000 7fd056000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd058077c00 0x7fd05819ed80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:44452/0 (socket says 172.21.15.45:44452) 2024-04-16T17:03:46.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:46.261+0000 7fd056000640 1 -- 172.21.15.45:0/1747166142 learned_addr learned my addr 172.21.15.45:0/1747166142 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:03:46.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:46.261+0000 7fd056a00640 1 --2- 172.21.15.45:0/1747166142 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd0580769c0 0x7fd05819e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:46.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:46.261+0000 7fd057400640 1 --2- 172.21.15.45:0/1747166142 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd0580785a0 0x7fd0581a5df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:46.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:46.261+0000 7fd056000640 1 -- 172.21.15.45:0/1747166142 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd0580769c0 msgr2=0x7fd05819e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:46.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:46.261+0000 7fd056000640 1 --2- 172.21.15.45:0/1747166142 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd0580769c0 0x7fd05819e840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:46.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:46.261+0000 7fd056000640 1 -- 172.21.15.45:0/1747166142 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd0580785a0 msgr2=0x7fd0581a5df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:46.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:46.261+0000 7fd056000640 1 --2- 172.21.15.45:0/1747166142 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd0580785a0 0x7fd0581a5df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:46.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:46.261+0000 7fd056000640 1 -- 172.21.15.45:0/1747166142 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0581a6460 con 0x7fd058077c00 2024-04-16T17:03:46.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:46.261+0000 7fd056a00640 1 --2- 172.21.15.45:0/1747166142 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd0580769c0 0x7fd05819e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:03:46.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:46.261+0000 7fd056000640 1 --2- 172.21.15.45:0/1747166142 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd058077c00 0x7fd05819ed80 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fd04400e9f0 tx=0x7fd04400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:03:46.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:46.262+0000 7fd04f400640 1 -- 172.21.15.45:0/1747166142 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd04400cd80 con 0x7fd058077c00 2024-04-16T17:03:46.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:46.262+0000 7fd057e00640 1 -- 172.21.15.45:0/1747166142 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0581a6750 con 0x7fd058077c00 2024-04-16T17:03:46.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:46.262+0000 7fd04f400640 1 -- 172.21.15.45:0/1747166142 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd044010910 con 0x7fd058077c00 2024-04-16T17:03:46.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:46.262+0000 7fd04f400640 1 -- 172.21.15.45:0/1747166142 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd044018990 con 0x7fd058077c00 2024-04-16T17:03:46.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:46.262+0000 7fd057e00640 1 -- 172.21.15.45:0/1747166142 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0581a6ca0 con 0x7fd058077c00 2024-04-16T17:03:46.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:46.263+0000 7fd057e00640 1 -- 172.21.15.45:0/1747166142 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd058110c90 con 0x7fd058077c00 2024-04-16T17:03:46.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:46.266+0000 7fd04f400640 1 -- 172.21.15.45:0/1747166142 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fd044010ac0 con 0x7fd058077c00 2024-04-16T17:03:46.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:46.268+0000 7fd04f400640 1 --2- 172.21.15.45:0/1747166142 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fd03c0a3bb0 0x7fd03c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:46.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:46.268+0000 7fd04f400640 1 -- 172.21.15.45:0/1747166142 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7fd0440d09c0 con 0x7fd058077c00 2024-04-16T17:03:46.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:46.268+0000 7fd056a00640 1 --2- 172.21.15.45:0/1747166142 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fd03c0a3bb0 0x7fd03c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:46.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:46.269+0000 7fd056a00640 1 --2- 172.21.15.45:0/1747166142 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fd03c0a3bb0 0x7fd03c0a6070 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fd05002c700 tx=0x7fd050005ef0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:03:46.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:46.272+0000 7fd04f400640 1 -- 172.21.15.45:0/1747166142 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd04409cab0 con 0x7fd058077c00 2024-04-16T17:03:46.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:46.485+0000 7fd057e00640 1 -- 172.21.15.45:0/1747166142 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fd058062c00 con 0x7fd058077c00 2024-04-16T17:03:46.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:46.486+0000 7fd04f400640 1 -- 172.21.15.45:0/1747166142 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v90) ==== 104+0+11 (secure 0 0 0) 0x7fd0440a0db0 con 0x7fd058077c00 2024-04-16T17:03:46.486 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 32 2024-04-16T17:03:46.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:46.488+0000 7fd057e00640 1 -- 172.21.15.45:0/1747166142 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fd03c0a3bb0 msgr2=0x7fd03c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:46.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:46.488+0000 7fd057e00640 1 --2- 172.21.15.45:0/1747166142 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fd03c0a3bb0 0x7fd03c0a6070 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fd05002c700 tx=0x7fd050005ef0 comp rx=0 tx=0).stop 2024-04-16T17:03:46.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:46.488+0000 7fd057e00640 1 -- 172.21.15.45:0/1747166142 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd058077c00 msgr2=0x7fd05819ed80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:46.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:46.488+0000 7fd057e00640 1 --2- 172.21.15.45:0/1747166142 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd058077c00 0x7fd05819ed80 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fd04400e9f0 tx=0x7fd04400eeb0 comp rx=0 tx=0).stop 2024-04-16T17:03:46.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:46.488+0000 7fd057e00640 1 -- 172.21.15.45:0/1747166142 shutdown_connections 2024-04-16T17:03:46.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:46.488+0000 7fd057e00640 1 --2- 172.21.15.45:0/1747166142 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fd03c0a3bb0 0x7fd03c0a6070 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:46.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:46.488+0000 7fd057e00640 1 --2- 172.21.15.45:0/1747166142 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd0580785a0 0x7fd0581a5df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:46.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:46.488+0000 7fd057e00640 1 --2- 172.21.15.45:0/1747166142 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd058077c00 0x7fd05819ed80 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:46.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:46.488+0000 7fd057e00640 1 --2- 172.21.15.45:0/1747166142 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd0580769c0 0x7fd05819e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:46.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:46.488+0000 7fd057e00640 1 -- 172.21.15.45:0/1747166142 >> 172.21.15.45:0/1747166142 conn(0x7fd0580ff6b0 msgr2=0x7fd058100e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:03:46.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:46.489+0000 7fd057e00640 1 -- 172.21.15.45:0/1747166142 shutdown_connections 2024-04-16T17:03:46.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:46.489+0000 7fd057e00640 1 -- 172.21.15.45:0/1747166142 wait complete. 2024-04-16T17:03:46.507 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-16T17:03:46.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:03:46 smithi117 ceph-mon[36730]: from='client.? 172.21.15.45:0/3897194833' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T17:03:46.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:03:46 smithi117 ceph-mon[36730]: from='client.? 172.21.15.45:0/843939065' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-16T17:03:46.585 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:03:46 smithi045 ceph-mon[31676]: from='client.? 172.21.15.45:0/3897194833' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T17:03:46.585 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:03:46 smithi045 ceph-mon[31676]: from='client.? 172.21.15.45:0/843939065' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-16T17:03:46.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:03:46 smithi104 ceph-mon[36457]: from='client.? 172.21.15.45:0/3897194833' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T17:03:46.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:03:46 smithi104 ceph-mon[36457]: from='client.? 172.21.15.45:0/843939065' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-16T17:03:46.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:46.731+0000 7ffabb200640 1 -- 172.21.15.45:0/2884254432 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffabc108540 msgr2=0x7ffabc108960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:46.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:46.731+0000 7ffabb200640 1 --2- 172.21.15.45:0/2884254432 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffabc108540 0x7ffabc108960 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7ffaac009900 tx=0x7ffaac02bc40 comp rx=0 tx=0).stop 2024-04-16T17:03:46.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:46.731+0000 7ffabb200640 1 -- 172.21.15.45:0/2884254432 shutdown_connections 2024-04-16T17:03:46.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:46.731+0000 7ffabb200640 1 --2- 172.21.15.45:0/2884254432 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffabc10a0f0 0x7ffabc110980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:46.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:46.731+0000 7ffabb200640 1 --2- 172.21.15.45:0/2884254432 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ffabc109730 0x7ffabc109bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:46.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:46.731+0000 7ffabb200640 1 --2- 172.21.15.45:0/2884254432 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffabc108540 0x7ffabc108960 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:46.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:46.731+0000 7ffabb200640 1 -- 172.21.15.45:0/2884254432 >> 172.21.15.45:0/2884254432 conn(0x7ffabc0774d0 msgr2=0x7ffabc079910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:03:46.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:46.731+0000 7ffabb200640 1 -- 172.21.15.45:0/2884254432 shutdown_connections 2024-04-16T17:03:46.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:46.731+0000 7ffabb200640 1 -- 172.21.15.45:0/2884254432 wait complete. 2024-04-16T17:03:46.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:46.732+0000 7ffabb200640 1 Processor -- start 2024-04-16T17:03:46.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:46.732+0000 7ffabb200640 1 -- start start 2024-04-16T17:03:46.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:46.732+0000 7ffabb200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ffabc108540 0x7ffabc19e910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:46.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:46.732+0000 7ffabb200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffabc109730 0x7ffabc19ee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:46.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:46.733+0000 7ffabb200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffabc10a0f0 0x7ffabc1a5dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:46.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:46.733+0000 7ffabb200640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7ffabc114220 con 0x7ffabc10a0f0 2024-04-16T17:03:46.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:46.733+0000 7ffabb200640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7ffabc1140a0 con 0x7ffabc108540 2024-04-16T17:03:46.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:46.733+0000 7ffabb200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7ffabc1143a0 con 0x7ffabc109730 2024-04-16T17:03:46.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:46.733+0000 7ffaba800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffabc10a0f0 0x7ffabc1a5dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:46.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:46.733+0000 7ffab9e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ffabc108540 0x7ffabc19e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:46.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:46.733+0000 7ffab9e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ffabc108540 0x7ffabc19e910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:43050/0 (socket says 172.21.15.45:43050) 2024-04-16T17:03:46.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:46.733+0000 7ffab9e00640 1 -- 172.21.15.45:0/154949562 learned_addr learned my addr 172.21.15.45:0/154949562 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:03:46.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:46.733+0000 7ffab1400640 1 --2- 172.21.15.45:0/154949562 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffabc109730 0x7ffabc19ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:46.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:46.733+0000 7ffab9e00640 1 -- 172.21.15.45:0/154949562 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffabc109730 msgr2=0x7ffabc19ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:46.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:46.733+0000 7ffab9e00640 1 --2- 172.21.15.45:0/154949562 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffabc109730 0x7ffabc19ee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:46.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:46.733+0000 7ffab9e00640 1 -- 172.21.15.45:0/154949562 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffabc10a0f0 msgr2=0x7ffabc1a5dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:46.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:46.733+0000 7ffab9e00640 1 --2- 172.21.15.45:0/154949562 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffabc10a0f0 0x7ffabc1a5dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:46.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:46.734+0000 7ffab9e00640 1 -- 172.21.15.45:0/154949562 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffabc1a6440 con 0x7ffabc108540 2024-04-16T17:03:46.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:46.734+0000 7ffab1400640 1 --2- 172.21.15.45:0/154949562 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffabc109730 0x7ffabc19ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:03:46.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:46.734+0000 7ffaba800640 1 --2- 172.21.15.45:0/154949562 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffabc10a0f0 0x7ffabc1a5dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:03:46.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:46.734+0000 7ffab9e00640 1 --2- 172.21.15.45:0/154949562 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ffabc108540 0x7ffabc19e910 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7ffaac02efd0 tx=0x7ffaac02c660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:03:46.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:46.734+0000 7ffab3400640 1 -- 172.21.15.45:0/154949562 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaac041070 con 0x7ffabc108540 2024-04-16T17:03:46.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:46.734+0000 7ffabb200640 1 -- 172.21.15.45:0/154949562 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffabc1a66d0 con 0x7ffabc108540 2024-04-16T17:03:46.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:46.734+0000 7ffab3400640 1 -- 172.21.15.45:0/154949562 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffaac004450 con 0x7ffabc108540 2024-04-16T17:03:46.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:46.734+0000 7ffab3400640 1 -- 172.21.15.45:0/154949562 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaac003e50 con 0x7ffabc108540 2024-04-16T17:03:46.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:46.734+0000 7ffabb200640 1 -- 172.21.15.45:0/154949562 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffabc1a6b90 con 0x7ffabc108540 2024-04-16T17:03:46.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:46.736+0000 7ffabb200640 1 -- 172.21.15.45:0/154949562 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffabc10f800 con 0x7ffabc108540 2024-04-16T17:03:46.738 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:46.737+0000 7ffab3400640 1 -- 172.21.15.45:0/154949562 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7ffaac044070 con 0x7ffabc108540 2024-04-16T17:03:46.739 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:46.739+0000 7ffab3400640 1 --2- 172.21.15.45:0/154949562 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7ffaa00a3d50 0x7ffaa00a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:46.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:46.739+0000 7ffab3400640 1 -- 172.21.15.45:0/154949562 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7ffaac0f21c0 con 0x7ffabc108540 2024-04-16T17:03:46.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:46.739+0000 7ffab1400640 1 --2- 172.21.15.45:0/154949562 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7ffaa00a3d50 0x7ffaa00a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:46.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:46.740+0000 7ffab1400640 1 --2- 172.21.15.45:0/154949562 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7ffaa00a3d50 0x7ffaa00a6210 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7ffabc19fd30 tx=0x7ffaa4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:03:46.744 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:46.743+0000 7ffab3400640 1 -- 172.21.15.45:0/154949562 <== mon.2 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ffaac0be2b0 con 0x7ffabc108540 2024-04-16T17:03:46.947 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:46.947+0000 7ffabb200640 1 -- 172.21.15.45:0/154949562 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7ffabc062c00 con 0x7ffabc108540 2024-04-16T17:03:46.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:46.948+0000 7ffab3400640 1 -- 172.21.15.45:0/154949562 <== mon.2 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v90) ==== 100+0+11 (secure 0 0 0) 0x7ffaac0b8e20 con 0x7ffabc108540 2024-04-16T17:03:46.949 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 64 2024-04-16T17:03:46.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:46.950+0000 7ffabb200640 1 -- 172.21.15.45:0/154949562 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7ffaa00a3d50 msgr2=0x7ffaa00a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:46.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:46.950+0000 7ffabb200640 1 --2- 172.21.15.45:0/154949562 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7ffaa00a3d50 0x7ffaa00a6210 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7ffabc19fd30 tx=0x7ffaa4005f90 comp rx=0 tx=0).stop 2024-04-16T17:03:46.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:46.950+0000 7ffabb200640 1 -- 172.21.15.45:0/154949562 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ffabc108540 msgr2=0x7ffabc19e910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:46.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:46.950+0000 7ffabb200640 1 --2- 172.21.15.45:0/154949562 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ffabc108540 0x7ffabc19e910 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7ffaac02efd0 tx=0x7ffaac02c660 comp rx=0 tx=0).stop 2024-04-16T17:03:46.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:46.950+0000 7ffabb200640 1 -- 172.21.15.45:0/154949562 shutdown_connections 2024-04-16T17:03:46.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:46.950+0000 7ffabb200640 1 --2- 172.21.15.45:0/154949562 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7ffaa00a3d50 0x7ffaa00a6210 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:46.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:46.950+0000 7ffabb200640 1 --2- 172.21.15.45:0/154949562 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffabc109730 0x7ffabc19ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:46.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:46.950+0000 7ffabb200640 1 --2- 172.21.15.45:0/154949562 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffabc10a0f0 0x7ffabc1a5dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:46.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:46.950+0000 7ffabb200640 1 --2- 172.21.15.45:0/154949562 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ffabc108540 0x7ffabc19e910 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:46.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:46.950+0000 7ffabb200640 1 -- 172.21.15.45:0/154949562 >> 172.21.15.45:0/154949562 conn(0x7ffabc0774d0 msgr2=0x7ffabc10c9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:03:46.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:46.951+0000 7ffabb200640 1 -- 172.21.15.45:0/154949562 shutdown_connections 2024-04-16T17:03:46.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:46.951+0000 7ffabb200640 1 -- 172.21.15.45:0/154949562 wait complete. 2024-04-16T17:03:46.972 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-16T17:03:47.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:47.206+0000 7f5137200640 1 -- 172.21.15.45:0/558586920 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5138108540 msgr2=0x7f5138108960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:47.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:47.206+0000 7f5137200640 1 --2- 172.21.15.45:0/558586920 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5138108540 0x7f5138108960 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f5128009be0 tx=0x7f512802bef0 comp rx=0 tx=0).stop 2024-04-16T17:03:47.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:47.206+0000 7f5137200640 1 -- 172.21.15.45:0/558586920 shutdown_connections 2024-04-16T17:03:47.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:47.206+0000 7f5137200640 1 --2- 172.21.15.45:0/558586920 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f513810a0f0 0x7f5138110980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:47.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:47.206+0000 7f5137200640 1 --2- 172.21.15.45:0/558586920 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5138109730 0x7f5138109bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:47.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:47.206+0000 7f5137200640 1 --2- 172.21.15.45:0/558586920 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5138108540 0x7f5138108960 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:47.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:47.206+0000 7f5137200640 1 -- 172.21.15.45:0/558586920 >> 172.21.15.45:0/558586920 conn(0x7f51380774d0 msgr2=0x7f5138079910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:03:47.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:47.206+0000 7f5137200640 1 -- 172.21.15.45:0/558586920 shutdown_connections 2024-04-16T17:03:47.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:47.207+0000 7f5137200640 1 -- 172.21.15.45:0/558586920 wait complete. 2024-04-16T17:03:47.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:47.208+0000 7f5137200640 1 Processor -- start 2024-04-16T17:03:47.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:47.208+0000 7f5137200640 1 -- start start 2024-04-16T17:03:47.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:47.208+0000 7f5137200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5138108540 0x7f513819e830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:47.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:47.208+0000 7f5137200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5138109730 0x7f513819ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:47.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:47.208+0000 7f5137200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f513810a0f0 0x7f51381a5de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:47.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:47.208+0000 7f5137200640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f51381141d0 con 0x7f5138108540 2024-04-16T17:03:47.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:47.208+0000 7f5137200640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f5138114050 con 0x7f513810a0f0 2024-04-16T17:03:47.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:47.208+0000 7f5137200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f5138114350 con 0x7f5138109730 2024-04-16T17:03:47.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:47.208+0000 7f5135e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5138108540 0x7f513819e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:47.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:47.208+0000 7f5135e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5138108540 0x7f513819e830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:44484/0 (socket says 172.21.15.45:44484) 2024-04-16T17:03:47.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:47.208+0000 7f5135e00640 1 -- 172.21.15.45:0/1847014935 learned_addr learned my addr 172.21.15.45:0/1847014935 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:03:47.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:47.208+0000 7f5135400640 1 --2- 172.21.15.45:0/1847014935 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5138109730 0x7f513819ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:47.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:47.209+0000 7f5136800640 1 --2- 172.21.15.45:0/1847014935 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f513810a0f0 0x7f51381a5de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:47.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:47.209+0000 7f5135e00640 1 -- 172.21.15.45:0/1847014935 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5138109730 msgr2=0x7f513819ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:47.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:47.209+0000 7f5135e00640 1 --2- 172.21.15.45:0/1847014935 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5138109730 0x7f513819ed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:47.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:47.209+0000 7f5135e00640 1 -- 172.21.15.45:0/1847014935 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f513810a0f0 msgr2=0x7f51381a5de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:47.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:47.209+0000 7f5135e00640 1 --2- 172.21.15.45:0/1847014935 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f513810a0f0 0x7f51381a5de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:47.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:47.209+0000 7f5135e00640 1 -- 172.21.15.45:0/1847014935 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51381a6450 con 0x7f5138108540 2024-04-16T17:03:47.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:47.209+0000 7f5135e00640 1 --2- 172.21.15.45:0/1847014935 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5138108540 0x7f513819e830 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f5128036040 tx=0x7f5128033580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:03:47.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:47.209+0000 7f512ea00640 1 -- 172.21.15.45:0/1847014935 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5128043020 con 0x7f5138108540 2024-04-16T17:03:47.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:47.210+0000 7f512ea00640 1 -- 172.21.15.45:0/1847014935 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5128003e50 con 0x7f5138108540 2024-04-16T17:03:47.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:47.210+0000 7f5137200640 1 -- 172.21.15.45:0/1847014935 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51381a66e0 con 0x7f5138108540 2024-04-16T17:03:47.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:47.210+0000 7f512ea00640 1 -- 172.21.15.45:0/1847014935 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5128034630 con 0x7f5138108540 2024-04-16T17:03:47.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:47.210+0000 7f5137200640 1 -- 172.21.15.45:0/1847014935 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51381a6c30 con 0x7f5138108540 2024-04-16T17:03:47.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:47.211+0000 7f5137200640 1 -- 172.21.15.45:0/1847014935 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f513810f860 con 0x7f5138108540 2024-04-16T17:03:47.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:47.213+0000 7f512ea00640 1 -- 172.21.15.45:0/1847014935 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f5128046020 con 0x7f5138108540 2024-04-16T17:03:47.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:47.214+0000 7f512ea00640 1 --2- 172.21.15.45:0/1847014935 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f511c0a3bb0 0x7f511c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:47.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:47.215+0000 7f512ea00640 1 -- 172.21.15.45:0/1847014935 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f51280f2c20 con 0x7f5138108540 2024-04-16T17:03:47.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:47.215+0000 7f5135400640 1 --2- 172.21.15.45:0/1847014935 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f511c0a3bb0 0x7f511c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:47.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:47.216+0000 7f5135400640 1 --2- 172.21.15.45:0/1847014935 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f511c0a3bb0 0x7f511c0a6070 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f513819fd40 tx=0x7f5120009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:03:47.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:47.219+0000 7f512ea00640 1 -- 172.21.15.45:0/1847014935 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f512804d020 con 0x7f5138108540 2024-04-16T17:03:47.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:47.426+0000 7f5137200640 1 -- 172.21.15.45:0/1847014935 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f5138062c00 con 0x7f5138108540 2024-04-16T17:03:47.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:47.427+0000 7f512ea00640 1 -- 172.21.15.45:0/1847014935 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v90) ==== 103+0+11 (secure 0 0 0) 0x7f51280dde20 con 0x7f5138108540 2024-04-16T17:03:47.428 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 32 2024-04-16T17:03:47.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:47.430+0000 7f5137200640 1 -- 172.21.15.45:0/1847014935 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f511c0a3bb0 msgr2=0x7f511c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:47.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:47.430+0000 7f5137200640 1 --2- 172.21.15.45:0/1847014935 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f511c0a3bb0 0x7f511c0a6070 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f513819fd40 tx=0x7f5120009260 comp rx=0 tx=0).stop 2024-04-16T17:03:47.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:47.430+0000 7f5137200640 1 -- 172.21.15.45:0/1847014935 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5138108540 msgr2=0x7f513819e830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:47.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:47.430+0000 7f5137200640 1 --2- 172.21.15.45:0/1847014935 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5138108540 0x7f513819e830 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f5128036040 tx=0x7f5128033580 comp rx=0 tx=0).stop 2024-04-16T17:03:47.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:47.430+0000 7f5137200640 1 -- 172.21.15.45:0/1847014935 shutdown_connections 2024-04-16T17:03:47.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:47.430+0000 7f5137200640 1 --2- 172.21.15.45:0/1847014935 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f511c0a3bb0 0x7f511c0a6070 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:47.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:47.430+0000 7f5137200640 1 --2- 172.21.15.45:0/1847014935 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5138109730 0x7f513819ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:47.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:47.430+0000 7f5137200640 1 --2- 172.21.15.45:0/1847014935 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f513810a0f0 0x7f51381a5de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:47.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:47.430+0000 7f5137200640 1 --2- 172.21.15.45:0/1847014935 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5138108540 0x7f513819e830 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:47.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:47.430+0000 7f5137200640 1 -- 172.21.15.45:0/1847014935 >> 172.21.15.45:0/1847014935 conn(0x7f51380774d0 msgr2=0x7f513810ca30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:03:47.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:47.430+0000 7f5137200640 1 -- 172.21.15.45:0/1847014935 shutdown_connections 2024-04-16T17:03:47.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:47.430+0000 7f5137200640 1 -- 172.21.15.45:0/1847014935 wait complete. 2024-04-16T17:03:47.452 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-16T17:03:47.617 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:03:47 smithi117 ceph-mon[36730]: pgmap v333: 129 pgs: 129 active+clean; 581 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-04-16T17:03:47.618 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:03:47 smithi117 ceph-mon[36730]: from='client.? 172.21.15.45:0/1747166142' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-16T17:03:47.618 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:03:47 smithi117 ceph-mon[36730]: from='client.? 172.21.15.45:0/154949562' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-16T17:03:47.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:47.676+0000 7fd06c800640 1 -- 172.21.15.45:0/1962777530 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd064105470 msgr2=0x7fd0641058f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:47.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:47.676+0000 7fd06c800640 1 --2- 172.21.15.45:0/1962777530 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd064105470 0x7fd0641058f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fd054009900 tx=0x7fd05402bc30 comp rx=0 tx=0).stop 2024-04-16T17:03:47.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:47.676+0000 7fd06c800640 1 -- 172.21.15.45:0/1962777530 shutdown_connections 2024-04-16T17:03:47.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:47.676+0000 7fd06c800640 1 --2- 172.21.15.45:0/1962777530 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd064105e30 0x7fd06410c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:47.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:47.676+0000 7fd06c800640 1 --2- 172.21.15.45:0/1962777530 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd064105470 0x7fd0641058f0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:47.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:47.676+0000 7fd06c800640 1 --2- 172.21.15.45:0/1962777530 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd064104280 0x7fd0641046a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:47.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:47.676+0000 7fd06c800640 1 -- 172.21.15.45:0/1962777530 >> 172.21.15.45:0/1962777530 conn(0x7fd0640ff7f0 msgr2=0x7fd064101c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:03:47.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:47.676+0000 7fd06c800640 1 -- 172.21.15.45:0/1962777530 shutdown_connections 2024-04-16T17:03:47.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:47.676+0000 7fd06c800640 1 -- 172.21.15.45:0/1962777530 wait complete. 2024-04-16T17:03:47.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:03:47 smithi104 ceph-mon[36457]: pgmap v333: 129 pgs: 129 active+clean; 581 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-04-16T17:03:47.679 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:03:47 smithi104 ceph-mon[36457]: from='client.? 172.21.15.45:0/1747166142' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-16T17:03:47.679 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:03:47 smithi104 ceph-mon[36457]: from='client.? 172.21.15.45:0/154949562' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-16T17:03:47.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:47.677+0000 7fd06c800640 1 Processor -- start 2024-04-16T17:03:47.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:47.678+0000 7fd06c800640 1 -- start start 2024-04-16T17:03:47.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:47.678+0000 7fd06c800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd064104280 0x7fd06419a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:47.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:47.678+0000 7fd06c800640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd064105470 0x7fd06419aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:47.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:47.678+0000 7fd06c800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd064105e30 0x7fd0641a1b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:47.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:47.678+0000 7fd06c800640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fd06410ff60 con 0x7fd064105e30 2024-04-16T17:03:47.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:47.678+0000 7fd06c800640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fd06410fde0 con 0x7fd064105470 2024-04-16T17:03:47.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:47.678+0000 7fd06c800640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fd0641100e0 con 0x7fd064104280 2024-04-16T17:03:47.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:47.678+0000 7fd069400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd064105470 0x7fd06419aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:47.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:47.678+0000 7fd069400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd064105470 0x7fd06419aac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:43076/0 (socket says 172.21.15.45:43076) 2024-04-16T17:03:47.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:47.678+0000 7fd06a800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd064105e30 0x7fd0641a1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:47.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:47.678+0000 7fd069400640 1 -- 172.21.15.45:0/1328737616 learned_addr learned my addr 172.21.15.45:0/1328737616 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:03:47.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:47.678+0000 7fd069e00640 1 --2- 172.21.15.45:0/1328737616 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd064104280 0x7fd06419a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:47.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:47.679+0000 7fd069400640 1 -- 172.21.15.45:0/1328737616 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd064104280 msgr2=0x7fd06419a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:47.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:47.679+0000 7fd069400640 1 --2- 172.21.15.45:0/1328737616 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd064104280 0x7fd06419a580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:47.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:47.679+0000 7fd069400640 1 -- 172.21.15.45:0/1328737616 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd064105e30 msgr2=0x7fd0641a1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:47.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:47.679+0000 7fd069400640 1 --2- 172.21.15.45:0/1328737616 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd064105e30 0x7fd0641a1b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:47.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:47.679+0000 7fd069400640 1 -- 172.21.15.45:0/1328737616 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0641a21a0 con 0x7fd064105470 2024-04-16T17:03:47.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:47.679+0000 7fd069400640 1 --2- 172.21.15.45:0/1328737616 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd064105470 0x7fd06419aac0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fd054000c00 tx=0x7fd05402c650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:03:47.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:47.679+0000 7fd05ea00640 1 -- 172.21.15.45:0/1328737616 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd054043020 con 0x7fd064105470 2024-04-16T17:03:47.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:47.679+0000 7fd06c800640 1 -- 172.21.15.45:0/1328737616 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0641a2430 con 0x7fd064105470 2024-04-16T17:03:47.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:47.679+0000 7fd05ea00640 1 -- 172.21.15.45:0/1328737616 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd0540054e0 con 0x7fd064105470 2024-04-16T17:03:47.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:47.679+0000 7fd05ea00640 1 -- 172.21.15.45:0/1328737616 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd054003c40 con 0x7fd064105470 2024-04-16T17:03:47.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:47.680+0000 7fd06c800640 1 -- 172.21.15.45:0/1328737616 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0641a2980 con 0x7fd064105470 2024-04-16T17:03:47.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:47.681+0000 7fd06c800640 1 -- 172.21.15.45:0/1328737616 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd06410b620 con 0x7fd064105470 2024-04-16T17:03:47.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:47.683+0000 7fd05ea00640 1 -- 172.21.15.45:0/1328737616 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fd054046020 con 0x7fd064105470 2024-04-16T17:03:47.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:47.684+0000 7fd05ea00640 1 --2- 172.21.15.45:0/1328737616 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fd04c0a3bb0 0x7fd04c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:47.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:47.684+0000 7fd05ea00640 1 -- 172.21.15.45:0/1328737616 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7fd0540f2670 con 0x7fd064105470 2024-04-16T17:03:47.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:47.684+0000 7fd069e00640 1 --2- 172.21.15.45:0/1328737616 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fd04c0a3bb0 0x7fd04c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:47.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:47.685+0000 7fd069e00640 1 --2- 172.21.15.45:0/1328737616 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fd04c0a3bb0 0x7fd04c0a6070 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fd0641052d0 tx=0x7fd058005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:03:47.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:47.689+0000 7fd05ea00640 1 -- 172.21.15.45:0/1328737616 <== mon.2 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd05404d020 con 0x7fd064105470 2024-04-16T17:03:47.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:03:47 smithi045 ceph-mon[31676]: pgmap v333: 129 pgs: 129 active+clean; 581 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-04-16T17:03:47.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:03:47 smithi045 ceph-mon[31676]: from='client.? 172.21.15.45:0/1747166142' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-16T17:03:47.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:03:47 smithi045 ceph-mon[31676]: from='client.? 172.21.15.45:0/154949562' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-16T17:03:47.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:47.913+0000 7fd06c800640 1 -- 172.21.15.45:0/1328737616 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fd06402d620 con 0x7fd064105470 2024-04-16T17:03:47.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:47.914+0000 7fd05ea00640 1 -- 172.21.15.45:0/1328737616 <== mon.2 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v90) ==== 81+0+590 (secure 0 0 0) 0x7fd0540bedb0 con 0x7fd064105470 2024-04-16T17:03:47.915 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.45:0/4074982598 2024-04-17T16:55:19.948001+0000 2024-04-16T17:03:47.915 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.45:0/2443572576 2024-04-17T16:55:19.948001+0000 2024-04-16T17:03:47.915 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.45:6801/1983630136 2024-04-17T16:55:19.948001+0000 2024-04-16T17:03:47.915 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.45:6800/1983630136 2024-04-17T16:55:19.948001+0000 2024-04-16T17:03:47.915 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.45:0/3150465248 2024-04-17T16:54:43.692466+0000 2024-04-16T17:03:47.915 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.45:0/556738623 2024-04-17T16:54:43.692466+0000 2024-04-16T17:03:47.915 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.45:0/445676297 2024-04-17T16:55:19.948001+0000 2024-04-16T17:03:47.915 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.45:6800/2553659622 2024-04-17T16:54:43.692466+0000 2024-04-16T17:03:47.915 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.45:6801/2553659622 2024-04-17T16:54:43.692466+0000 2024-04-16T17:03:47.916 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.45:0/1184615408 2024-04-17T16:54:43.692466+0000 2024-04-16T17:03:47.916 INFO:teuthology.orchestra.run.smithi045.stderr:listed 10 entries 2024-04-16T17:03:47.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:47.917+0000 7fd06c800640 1 -- 172.21.15.45:0/1328737616 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fd04c0a3bb0 msgr2=0x7fd04c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:47.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:47.917+0000 7fd06c800640 1 --2- 172.21.15.45:0/1328737616 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fd04c0a3bb0 0x7fd04c0a6070 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fd0641052d0 tx=0x7fd058005f90 comp rx=0 tx=0).stop 2024-04-16T17:03:47.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:47.917+0000 7fd06c800640 1 -- 172.21.15.45:0/1328737616 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd064105470 msgr2=0x7fd06419aac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:47.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:47.917+0000 7fd06c800640 1 --2- 172.21.15.45:0/1328737616 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd064105470 0x7fd06419aac0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fd054000c00 tx=0x7fd05402c650 comp rx=0 tx=0).stop 2024-04-16T17:03:47.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:47.917+0000 7fd06c800640 1 -- 172.21.15.45:0/1328737616 shutdown_connections 2024-04-16T17:03:47.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:47.917+0000 7fd06c800640 1 --2- 172.21.15.45:0/1328737616 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fd04c0a3bb0 0x7fd04c0a6070 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:47.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:47.917+0000 7fd06c800640 1 --2- 172.21.15.45:0/1328737616 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd064105e30 0x7fd0641a1b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:47.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:47.917+0000 7fd06c800640 1 --2- 172.21.15.45:0/1328737616 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd064105470 0x7fd06419aac0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:47.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:47.917+0000 7fd06c800640 1 --2- 172.21.15.45:0/1328737616 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd064104280 0x7fd06419a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:47.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:47.917+0000 7fd06c800640 1 -- 172.21.15.45:0/1328737616 >> 172.21.15.45:0/1328737616 conn(0x7fd0640ff7f0 msgr2=0x7fd0641087f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:03:47.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:47.917+0000 7fd06c800640 1 -- 172.21.15.45:0/1328737616 shutdown_connections 2024-04-16T17:03:47.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:47.917+0000 7fd06c800640 1 -- 172.21.15.45:0/1328737616 wait complete. 2024-04-16T17:03:47.937 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi045.front.sepia.ceph.com... 2024-04-16T17:03:47.937 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T17:03:47.937 DEBUG:teuthology.orchestra.run.smithi045:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2024-04-16T17:03:48.026 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi045.stderr:ceph-fuse[75514]: fuse finished with error 0 and tester_r 0 2024-04-16T17:03:48.031 INFO:teuthology.orchestra.run:waiting for 300 2024-04-16T17:03:48.064 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi045.front.sepia.ceph.com 2024-04-16T17:03:48.064 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T17:03:48.064 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2024-04-16T17:03:48.178 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-16T17:03:48.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:48.405+0000 7ff235800640 1 -- 172.21.15.45:0/705146849 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff23010a030 msgr2=0x7ff2301108c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:48.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:48.405+0000 7ff235800640 1 --2- 172.21.15.45:0/705146849 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff23010a030 0x7ff2301108c0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7ff228009900 tx=0x7ff22802bc30 comp rx=0 tx=0).stop 2024-04-16T17:03:48.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:48.405+0000 7ff235800640 1 -- 172.21.15.45:0/705146849 shutdown_connections 2024-04-16T17:03:48.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:48.405+0000 7ff235800640 1 --2- 172.21.15.45:0/705146849 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff23010a030 0x7ff2301108c0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:48.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:48.405+0000 7ff235800640 1 --2- 172.21.15.45:0/705146849 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff230109670 0x7ff230109af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:48.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:48.405+0000 7ff235800640 1 --2- 172.21.15.45:0/705146849 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff230108480 0x7ff2301088a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:48.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:48.405+0000 7ff235800640 1 -- 172.21.15.45:0/705146849 >> 172.21.15.45:0/705146849 conn(0x7ff230077440 msgr2=0x7ff230079860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:03:48.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:48.405+0000 7ff235800640 1 -- 172.21.15.45:0/705146849 shutdown_connections 2024-04-16T17:03:48.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:48.405+0000 7ff235800640 1 -- 172.21.15.45:0/705146849 wait complete. 2024-04-16T17:03:48.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:48.406+0000 7ff235800640 1 Processor -- start 2024-04-16T17:03:48.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:48.406+0000 7ff235800640 1 -- start start 2024-04-16T17:03:48.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:48.406+0000 7ff235800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff230108480 0x7ff23019e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:48.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:48.406+0000 7ff235800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff230109670 0x7ff23019ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:48.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:48.406+0000 7ff235800640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff23010a030 0x7ff2301a5cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:48.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:48.407+0000 7ff235800640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7ff230114130 con 0x7ff230108480 2024-04-16T17:03:48.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:48.407+0000 7ff235800640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7ff230113fb0 con 0x7ff23010a030 2024-04-16T17:03:48.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:48.407+0000 7ff235800640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7ff2301142b0 con 0x7ff230109670 2024-04-16T17:03:48.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:48.407+0000 7ff22f400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff230109670 0x7ff23019ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:48.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:48.407+0000 7ff22fe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff230108480 0x7ff23019e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:48.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:48.407+0000 7ff22f400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff230109670 0x7ff23019ecb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:51712/0 (socket says 172.21.15.45:51712) 2024-04-16T17:03:48.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:48.407+0000 7ff22f400640 1 -- 172.21.15.45:0/3697534986 learned_addr learned my addr 172.21.15.45:0/3697534986 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:03:48.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:48.407+0000 7ff234e00640 1 --2- 172.21.15.45:0/3697534986 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff23010a030 0x7ff2301a5cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:48.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:48.407+0000 7ff22f400640 1 -- 172.21.15.45:0/3697534986 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff23010a030 msgr2=0x7ff2301a5cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:48.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:48.407+0000 7ff22f400640 1 --2- 172.21.15.45:0/3697534986 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff23010a030 0x7ff2301a5cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:48.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:48.407+0000 7ff22f400640 1 -- 172.21.15.45:0/3697534986 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff230108480 msgr2=0x7ff23019e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:48.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:48.407+0000 7ff22f400640 1 --2- 172.21.15.45:0/3697534986 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff230108480 0x7ff23019e770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:48.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:48.407+0000 7ff22f400640 1 -- 172.21.15.45:0/3697534986 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2301a6340 con 0x7ff230109670 2024-04-16T17:03:48.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:48.407+0000 7ff22fe00640 1 --2- 172.21.15.45:0/3697534986 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff230108480 0x7ff23019e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:03:48.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:48.408+0000 7ff22f400640 1 --2- 172.21.15.45:0/3697534986 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff230109670 0x7ff23019ecb0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7ff22000ebc0 tx=0x7ff22000c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:03:48.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:48.408+0000 7ff22cc00640 1 -- 172.21.15.45:0/3697534986 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff220010630 con 0x7ff230109670 2024-04-16T17:03:48.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:48.408+0000 7ff22cc00640 1 -- 172.21.15.45:0/3697534986 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff220010cb0 con 0x7ff230109670 2024-04-16T17:03:48.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:48.408+0000 7ff22cc00640 1 -- 172.21.15.45:0/3697534986 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff220018bd0 con 0x7ff230109670 2024-04-16T17:03:48.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:48.408+0000 7ff235800640 1 -- 172.21.15.45:0/3697534986 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2301a6630 con 0x7ff230109670 2024-04-16T17:03:48.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:48.408+0000 7ff235800640 1 -- 172.21.15.45:0/3697534986 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2301a6b30 con 0x7ff230109670 2024-04-16T17:03:48.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:48.410+0000 7ff235800640 1 -- 172.21.15.45:0/3697534986 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff23010f740 con 0x7ff230109670 2024-04-16T17:03:48.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:48.411+0000 7ff22cc00640 1 -- 172.21.15.45:0/3697534986 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7ff2200107d0 con 0x7ff230109670 2024-04-16T17:03:48.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:48.412+0000 7ff22cc00640 1 --2- 172.21.15.45:0/3697534986 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7ff2140a3bb0 0x7ff2140a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:48.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:48.413+0000 7ff22cc00640 1 -- 172.21.15.45:0/3697534986 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7ff2200d0c80 con 0x7ff230109670 2024-04-16T17:03:48.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:48.413+0000 7ff22fe00640 1 --2- 172.21.15.45:0/3697534986 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7ff2140a3bb0 0x7ff2140a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:48.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:48.414+0000 7ff22fe00640 1 --2- 172.21.15.45:0/3697534986 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7ff2140a3bb0 0x7ff2140a6070 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7ff2301094d0 tx=0x7ff218005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:03:48.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:48.417+0000 7ff22cc00640 1 -- 172.21.15.45:0/3697534986 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff22009cd70 con 0x7ff230109670 2024-04-16T17:03:48.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:48.623+0000 7ff235800640 1 -- 172.21.15.45:0/3697534986 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff230062c00 con 0x7ff230109670 2024-04-16T17:03:48.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:48.625+0000 7ff22cc00640 1 -- 172.21.15.45:0/3697534986 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+24164 (secure 0 0 0) 0x7ff2200a1070 con 0x7ff230109670 2024-04-16T17:03:48.627 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-16T17:03:48.627 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":90,"fsid":"b1fecd38-fc11-11ee-bc90-c7b262605968","created":"2024-04-16T16:54:02.385906+0000","modified":"2024-04-16T17:03:29.720565+0000","last_up_change":"2024-04-16T17:02:25.162624+0000","last_in_change":"2024-04-16T17:02:04.903274+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-16T16:58:06.166659+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-16T17:03:22.304198+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-16T17:03:23.616450+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-16T17:03:27.226289+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b655de2c-59f8-4fd4-ac98-0cc222229f56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":2290170881},{"type":"v1","addr":"172.21.15.45:6803","nonce":2290170881}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":2290170881},{"type":"v1","addr":"172.21.15.45:6805","nonce":2290170881}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":2290170881},{"type":"v1","addr":"172.21.15.45:6809","nonce":2290170881}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":2290170881},{"type":"v1","addr":"172.21.15.45:6807","nonce":2290170881}]},"public_addr":"172.21.15.45:6803/2290170881","cluster_addr":"172.21.15.45:6805/2290170881","heartbeat_back_addr":"172.21.15.45:6809/2290170881","heartbeat_front_addr":"172.21.15.45:6807/2290170881","state":["exists","up"]},{"osd":1,"uuid":"ebd450ec-08fd-4c3d-8b24-4e10634ddb7e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6800","nonce":1462942067},{"type":"v1","addr":"172.21.15.104:6801","nonce":1462942067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":1462942067},{"type":"v1","addr":"172.21.15.104:6803","nonce":1462942067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":1462942067},{"type":"v1","addr":"172.21.15.104:6807","nonce":1462942067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":1462942067},{"type":"v1","addr":"172.21.15.104:6805","nonce":1462942067}]},"public_addr":"172.21.15.104:6801/1462942067","cluster_addr":"172.21.15.104:6803/1462942067","heartbeat_back_addr":"172.21.15.104:6807/1462942067","heartbeat_front_addr":"172.21.15.104:6805/1462942067","state":["exists","up"]},{"osd":2,"uuid":"b318a512-4190-42a2-8104-dcc4026ad8ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":2829260735},{"type":"v1","addr":"172.21.15.117:6801","nonce":2829260735}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":2829260735},{"type":"v1","addr":"172.21.15.117:6803","nonce":2829260735}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":2829260735},{"type":"v1","addr":"172.21.15.117:6807","nonce":2829260735}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":2829260735},{"type":"v1","addr":"172.21.15.117:6805","nonce":2829260735}]},"public_addr":"172.21.15.117:6801/2829260735","cluster_addr":"172.21.15.117:6803/2829260735","heartbeat_back_addr":"172.21.15.117:6807/2829260735","heartbeat_front_addr":"172.21.15.117:6805/2829260735","state":["exists","up"]},{"osd":3,"uuid":"ec6490cb-a715-4c12-a54a-225aae26fcb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":2823712137},{"type":"v1","addr":"172.21.15.45:6811","nonce":2823712137}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":2823712137},{"type":"v1","addr":"172.21.15.45:6813","nonce":2823712137}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":2823712137},{"type":"v1","addr":"172.21.15.45:6817","nonce":2823712137}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":2823712137},{"type":"v1","addr":"172.21.15.45:6815","nonce":2823712137}]},"public_addr":"172.21.15.45:6811/2823712137","cluster_addr":"172.21.15.45:6813/2823712137","heartbeat_back_addr":"172.21.15.45:6817/2823712137","heartbeat_front_addr":"172.21.15.45:6815/2823712137","state":["exists","up"]},{"osd":4,"uuid":"5d91d568-0250-4aae-8dc2-a0e079f92062","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":1396278127},{"type":"v1","addr":"172.21.15.104:6809","nonce":1396278127}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":1396278127},{"type":"v1","addr":"172.21.15.104:6811","nonce":1396278127}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":1396278127},{"type":"v1","addr":"172.21.15.104:6815","nonce":1396278127}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":1396278127},{"type":"v1","addr":"172.21.15.104:6813","nonce":1396278127}]},"public_addr":"172.21.15.104:6809/1396278127","cluster_addr":"172.21.15.104:6811/1396278127","heartbeat_back_addr":"172.21.15.104:6815/1396278127","heartbeat_front_addr":"172.21.15.104:6813/1396278127","state":["exists","up"]},{"osd":5,"uuid":"ac510b43-6fb2-4d0f-bc5f-16ac080d9e9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":4145720047},{"type":"v1","addr":"172.21.15.117:6809","nonce":4145720047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":4145720047},{"type":"v1","addr":"172.21.15.117:6811","nonce":4145720047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":4145720047},{"type":"v1","addr":"172.21.15.117:6815","nonce":4145720047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":4145720047},{"type":"v1","addr":"172.21.15.117:6813","nonce":4145720047}]},"public_addr":"172.21.15.117:6809/4145720047","cluster_addr":"172.21.15.117:6811/4145720047","heartbeat_back_addr":"172.21.15.117:6815/4145720047","heartbeat_front_addr":"172.21.15.117:6813/4145720047","state":["exists","up"]},{"osd":6,"uuid":"a41adc18-d88a-419e-b2c2-b089c1f32472","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":3254229176},{"type":"v1","addr":"172.21.15.45:6819","nonce":3254229176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":3254229176},{"type":"v1","addr":"172.21.15.45:6821","nonce":3254229176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":3254229176},{"type":"v1","addr":"172.21.15.45:6825","nonce":3254229176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":3254229176},{"type":"v1","addr":"172.21.15.45:6823","nonce":3254229176}]},"public_addr":"172.21.15.45:6819/3254229176","cluster_addr":"172.21.15.45:6821/3254229176","heartbeat_back_addr":"172.21.15.45:6825/3254229176","heartbeat_front_addr":"172.21.15.45:6823/3254229176","state":["exists","up"]},{"osd":7,"uuid":"b71712a4-7cf6-454d-9fd1-565e3df21c1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":4151077186},{"type":"v1","addr":"172.21.15.104:6817","nonce":4151077186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":4151077186},{"type":"v1","addr":"172.21.15.104:6819","nonce":4151077186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":4151077186},{"type":"v1","addr":"172.21.15.104:6823","nonce":4151077186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":4151077186},{"type":"v1","addr":"172.21.15.104:6821","nonce":4151077186}]},"public_addr":"172.21.15.104:6817/4151077186","cluster_addr":"172.21.15.104:6819/4151077186","heartbeat_back_addr":"172.21.15.104:6823/4151077186","heartbeat_front_addr":"172.21.15.104:6821/4151077186","state":["exists","up"]},{"osd":8,"uuid":"9e572aef-59a8-450e-a92e-df9dff4d57a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":1500286830},{"type":"v1","addr":"172.21.15.117:6817","nonce":1500286830}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":1500286830},{"type":"v1","addr":"172.21.15.117:6819","nonce":1500286830}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":1500286830},{"type":"v1","addr":"172.21.15.117:6823","nonce":1500286830}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":1500286830},{"type":"v1","addr":"172.21.15.117:6821","nonce":1500286830}]},"public_addr":"172.21.15.117:6817/1500286830","cluster_addr":"172.21.15.117:6819/1500286830","heartbeat_back_addr":"172.21.15.117:6823/1500286830","heartbeat_front_addr":"172.21.15.117:6821/1500286830","state":["exists","up"]},{"osd":9,"uuid":"014e9598-c413-4cc4-9bb8-61c6e8f59d30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":3106734074},{"type":"v1","addr":"172.21.15.45:6827","nonce":3106734074}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":3106734074},{"type":"v1","addr":"172.21.15.45:6829","nonce":3106734074}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":3106734074},{"type":"v1","addr":"172.21.15.45:6833","nonce":3106734074}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":3106734074},{"type":"v1","addr":"172.21.15.45:6831","nonce":3106734074}]},"public_addr":"172.21.15.45:6827/3106734074","cluster_addr":"172.21.15.45:6829/3106734074","heartbeat_back_addr":"172.21.15.45:6833/3106734074","heartbeat_front_addr":"172.21.15.45:6831/3106734074","state":["exists","up"]},{"osd":10,"uuid":"2deba364-3e39-47be-8a1f-8256df5b82d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":1404794652},{"type":"v1","addr":"172.21.15.104:6825","nonce":1404794652}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":1404794652},{"type":"v1","addr":"172.21.15.104:6827","nonce":1404794652}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":1404794652},{"type":"v1","addr":"172.21.15.104:6831","nonce":1404794652}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":1404794652},{"type":"v1","addr":"172.21.15.104:6829","nonce":1404794652}]},"public_addr":"172.21.15.104:6825/1404794652","cluster_addr":"172.21.15.104:6827/1404794652","heartbeat_back_addr":"172.21.15.104:6831/1404794652","heartbeat_front_addr":"172.21.15.104:6829/1404794652","state":["exists","up"]},{"osd":11,"uuid":"3fda4fea-59ad-4d1a-9fe6-fcb30e31fd7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":3384003243},{"type":"v1","addr":"172.21.15.117:6825","nonce":3384003243}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":3384003243},{"type":"v1","addr":"172.21.15.117:6827","nonce":3384003243}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":3384003243},{"type":"v1","addr":"172.21.15.117:6831","nonce":3384003243}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":3384003243},{"type":"v1","addr":"172.21.15.117:6829","nonce":3384003243}]},"public_addr":"172.21.15.117:6825/3384003243","cluster_addr":"172.21.15.117:6827/3384003243","heartbeat_back_addr":"172.21.15.117:6831/3384003243","heartbeat_front_addr":"172.21.15.117:6829/3384003243","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:57:14.799608+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:57:39.884668+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:58:03.327057+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:58:37.314531+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:59:03.667650+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:59:28.462823+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T17:00:02.968574+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T17:00:29.559190+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T17:00:55.291007+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T17:01:29.815977+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T17:01:57.015580+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T17:02:23.233151+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.45:0/4074982598":"2024-04-17T16:55:19.948001+0000","172.21.15.45:0/2443572576":"2024-04-17T16:55:19.948001+0000","172.21.15.45:6801/1983630136":"2024-04-17T16:55:19.948001+0000","172.21.15.45:6800/1983630136":"2024-04-17T16:55:19.948001+0000","172.21.15.45:0/3150465248":"2024-04-17T16:54:43.692466+0000","172.21.15.45:0/556738623":"2024-04-17T16:54:43.692466+0000","172.21.15.45:0/445676297":"2024-04-17T16:55:19.948001+0000","172.21.15.45:6800/2553659622":"2024-04-17T16:54:43.692466+0000","172.21.15.45:6801/2553659622":"2024-04-17T16:54:43.692466+0000","172.21.15.45:0/1184615408":"2024-04-17T16:54:43.692466+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-16T17:03:48.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:48.628+0000 7ff235800640 1 -- 172.21.15.45:0/3697534986 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7ff2140a3bb0 msgr2=0x7ff2140a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:48.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:48.628+0000 7ff235800640 1 --2- 172.21.15.45:0/3697534986 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7ff2140a3bb0 0x7ff2140a6070 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7ff2301094d0 tx=0x7ff218005f90 comp rx=0 tx=0).stop 2024-04-16T17:03:48.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:48.628+0000 7ff235800640 1 -- 172.21.15.45:0/3697534986 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff230109670 msgr2=0x7ff23019ecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:48.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:48.628+0000 7ff235800640 1 --2- 172.21.15.45:0/3697534986 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff230109670 0x7ff23019ecb0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7ff22000ebc0 tx=0x7ff22000c6a0 comp rx=0 tx=0).stop 2024-04-16T17:03:48.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:48.628+0000 7ff235800640 1 -- 172.21.15.45:0/3697534986 shutdown_connections 2024-04-16T17:03:48.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:48.628+0000 7ff235800640 1 --2- 172.21.15.45:0/3697534986 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7ff2140a3bb0 0x7ff2140a6070 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:48.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:48.628+0000 7ff235800640 1 --2- 172.21.15.45:0/3697534986 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff230109670 0x7ff23019ecb0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:48.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:48.628+0000 7ff235800640 1 --2- 172.21.15.45:0/3697534986 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff23010a030 0x7ff2301a5cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:48.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:48.628+0000 7ff235800640 1 --2- 172.21.15.45:0/3697534986 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff230108480 0x7ff23019e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:48.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:48.628+0000 7ff235800640 1 -- 172.21.15.45:0/3697534986 >> 172.21.15.45:0/3697534986 conn(0x7ff230077440 msgr2=0x7ff23010c910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:03:48.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:48.628+0000 7ff235800640 1 -- 172.21.15.45:0/3697534986 shutdown_connections 2024-04-16T17:03:48.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:48.628+0000 7ff235800640 1 -- 172.21.15.45:0/3697534986 wait complete. 2024-04-16T17:03:48.646 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:03:48 smithi045 ceph-mon[31676]: from='client.? 172.21.15.45:0/1847014935' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-16T17:03:48.646 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:03:48 smithi045 ceph-mon[31676]: from='client.? 172.21.15.45:0/1328737616' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-16T17:03:48.649 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-16T17:03:48.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:03:48 smithi104 ceph-mon[36457]: from='client.? 172.21.15.45:0/1847014935' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-16T17:03:48.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:03:48 smithi104 ceph-mon[36457]: from='client.? 172.21.15.45:0/1328737616' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-16T17:03:48.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:03:48 smithi117 ceph-mon[36730]: from='client.? 172.21.15.45:0/1847014935' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-16T17:03:48.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:03:48 smithi117 ceph-mon[36730]: from='client.? 172.21.15.45:0/1328737616' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-16T17:03:48.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:48.870+0000 7fb027200640 1 -- 172.21.15.45:0/2395971140 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb02810e440 msgr2=0x7fb028110830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:48.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:48.870+0000 7fb027200640 1 --2- 172.21.15.45:0/2395971140 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb02810e440 0x7fb028110830 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fb020009ca0 tx=0x7fb02002c100 comp rx=0 tx=0).stop 2024-04-16T17:03:48.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:48.870+0000 7fb027200640 1 -- 172.21.15.45:0/2395971140 shutdown_connections 2024-04-16T17:03:48.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:48.870+0000 7fb027200640 1 --2- 172.21.15.45:0/2395971140 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb02810e440 0x7fb028110830 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:48.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:48.870+0000 7fb027200640 1 --2- 172.21.15.45:0/2395971140 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb028074a60 0x7fb028074ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:48.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:48.870+0000 7fb027200640 1 --2- 172.21.15.45:0/2395971140 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb028076040 0x7fb028074490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:48.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:48.870+0000 7fb027200640 1 -- 172.21.15.45:0/2395971140 >> 172.21.15.45:0/2395971140 conn(0x7fb0280fd3e0 msgr2=0x7fb0280ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:03:48.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:48.870+0000 7fb027200640 1 -- 172.21.15.45:0/2395971140 shutdown_connections 2024-04-16T17:03:48.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:48.870+0000 7fb027200640 1 -- 172.21.15.45:0/2395971140 wait complete. 2024-04-16T17:03:48.873 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:48.871+0000 7fb027200640 1 Processor -- start 2024-04-16T17:03:48.873 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:48.871+0000 7fb027200640 1 -- start start 2024-04-16T17:03:48.873 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:48.872+0000 7fb027200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb028074a60 0x7fb02819e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:48.873 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:48.872+0000 7fb027200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb028076040 0x7fb02819ec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:48.873 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:48.872+0000 7fb027200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb02810e440 0x7fb0281a5cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:48.873 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:48.872+0000 7fb027200640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fb028114100 con 0x7fb02810e440 2024-04-16T17:03:48.873 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:48.872+0000 7fb027200640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fb028113f80 con 0x7fb028076040 2024-04-16T17:03:48.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:48.872+0000 7fb027200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb028114280 con 0x7fb028074a60 2024-04-16T17:03:48.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:48.872+0000 7fb026800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb02810e440 0x7fb0281a5cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:48.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:48.872+0000 7fb026800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb02810e440 0x7fb0281a5cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:44548/0 (socket says 172.21.15.45:44548) 2024-04-16T17:03:48.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:48.872+0000 7fb026800640 1 -- 172.21.15.45:0/1403698062 learned_addr learned my addr 172.21.15.45:0/1403698062 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:03:48.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:48.872+0000 7fb025e00640 1 --2- 172.21.15.45:0/1403698062 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb028074a60 0x7fb02819e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:48.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:48.872+0000 7fb01fe00640 1 --2- 172.21.15.45:0/1403698062 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb028076040 0x7fb02819ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:48.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:48.873+0000 7fb026800640 1 -- 172.21.15.45:0/1403698062 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb028074a60 msgr2=0x7fb02819e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:48.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:48.873+0000 7fb026800640 1 --2- 172.21.15.45:0/1403698062 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb028074a60 0x7fb02819e710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:48.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:48.873+0000 7fb026800640 1 -- 172.21.15.45:0/1403698062 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb028076040 msgr2=0x7fb02819ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:48.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:48.873+0000 7fb026800640 1 --2- 172.21.15.45:0/1403698062 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb028076040 0x7fb02819ec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:48.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:48.873+0000 7fb026800640 1 -- 172.21.15.45:0/1403698062 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0281a6330 con 0x7fb02810e440 2024-04-16T17:03:48.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:48.873+0000 7fb026800640 1 --2- 172.21.15.45:0/1403698062 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb02810e440 0x7fb0281a5cc0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fb02002c640 tx=0x7fb020034670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:03:48.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:48.873+0000 7fb01ea00640 1 -- 172.21.15.45:0/1403698062 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb020039050 con 0x7fb02810e440 2024-04-16T17:03:48.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:48.873+0000 7fb027200640 1 -- 172.21.15.45:0/1403698062 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0281a65c0 con 0x7fb02810e440 2024-04-16T17:03:48.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:48.873+0000 7fb01ea00640 1 -- 172.21.15.45:0/1403698062 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb020045070 con 0x7fb02810e440 2024-04-16T17:03:48.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:48.873+0000 7fb01ea00640 1 -- 172.21.15.45:0/1403698062 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0200046d0 con 0x7fb02810e440 2024-04-16T17:03:48.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:48.873+0000 7fb027200640 1 -- 172.21.15.45:0/1403698062 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0281a6ab0 con 0x7fb02810e440 2024-04-16T17:03:48.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:48.875+0000 7fb027200640 1 -- 172.21.15.45:0/1403698062 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb028104610 con 0x7fb02810e440 2024-04-16T17:03:48.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:48.877+0000 7fb01ea00640 1 -- 172.21.15.45:0/1403698062 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fb020003970 con 0x7fb02810e440 2024-04-16T17:03:48.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:48.878+0000 7fb01ea00640 1 --2- 172.21.15.45:0/1403698062 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fb00c0a3bb0 0x7fb00c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:48.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:48.878+0000 7fb01ea00640 1 -- 172.21.15.45:0/1403698062 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7fb0200f2e70 con 0x7fb02810e440 2024-04-16T17:03:48.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:48.878+0000 7fb025e00640 1 --2- 172.21.15.45:0/1403698062 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fb00c0a3bb0 0x7fb00c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:48.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:48.879+0000 7fb025e00640 1 --2- 172.21.15.45:0/1403698062 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fb00c0a3bb0 0x7fb00c0a6070 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fb01800bbe0 tx=0x7fb01800b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:03:48.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:48.883+0000 7fb01ea00640 1 -- 172.21.15.45:0/1403698062 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb0200bf5b0 con 0x7fb02810e440 2024-04-16T17:03:49.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:49.100+0000 7fb027200640 1 -- 172.21.15.45:0/1403698062 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb028062c00 con 0x7fb02810e440 2024-04-16T17:03:49.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:49.104+0000 7fb01ea00640 1 -- 172.21.15.45:0/1403698062 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+24164 (secure 0 0 0) 0x7fb0200098b0 con 0x7fb02810e440 2024-04-16T17:03:49.106 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-16T17:03:49.106 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":90,"fsid":"b1fecd38-fc11-11ee-bc90-c7b262605968","created":"2024-04-16T16:54:02.385906+0000","modified":"2024-04-16T17:03:29.720565+0000","last_up_change":"2024-04-16T17:02:25.162624+0000","last_in_change":"2024-04-16T17:02:04.903274+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-16T16:58:06.166659+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-16T17:03:22.304198+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-16T17:03:23.616450+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-16T17:03:27.226289+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b655de2c-59f8-4fd4-ac98-0cc222229f56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":2290170881},{"type":"v1","addr":"172.21.15.45:6803","nonce":2290170881}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":2290170881},{"type":"v1","addr":"172.21.15.45:6805","nonce":2290170881}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":2290170881},{"type":"v1","addr":"172.21.15.45:6809","nonce":2290170881}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":2290170881},{"type":"v1","addr":"172.21.15.45:6807","nonce":2290170881}]},"public_addr":"172.21.15.45:6803/2290170881","cluster_addr":"172.21.15.45:6805/2290170881","heartbeat_back_addr":"172.21.15.45:6809/2290170881","heartbeat_front_addr":"172.21.15.45:6807/2290170881","state":["exists","up"]},{"osd":1,"uuid":"ebd450ec-08fd-4c3d-8b24-4e10634ddb7e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6800","nonce":1462942067},{"type":"v1","addr":"172.21.15.104:6801","nonce":1462942067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":1462942067},{"type":"v1","addr":"172.21.15.104:6803","nonce":1462942067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":1462942067},{"type":"v1","addr":"172.21.15.104:6807","nonce":1462942067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":1462942067},{"type":"v1","addr":"172.21.15.104:6805","nonce":1462942067}]},"public_addr":"172.21.15.104:6801/1462942067","cluster_addr":"172.21.15.104:6803/1462942067","heartbeat_back_addr":"172.21.15.104:6807/1462942067","heartbeat_front_addr":"172.21.15.104:6805/1462942067","state":["exists","up"]},{"osd":2,"uuid":"b318a512-4190-42a2-8104-dcc4026ad8ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":2829260735},{"type":"v1","addr":"172.21.15.117:6801","nonce":2829260735}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":2829260735},{"type":"v1","addr":"172.21.15.117:6803","nonce":2829260735}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":2829260735},{"type":"v1","addr":"172.21.15.117:6807","nonce":2829260735}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":2829260735},{"type":"v1","addr":"172.21.15.117:6805","nonce":2829260735}]},"public_addr":"172.21.15.117:6801/2829260735","cluster_addr":"172.21.15.117:6803/2829260735","heartbeat_back_addr":"172.21.15.117:6807/2829260735","heartbeat_front_addr":"172.21.15.117:6805/2829260735","state":["exists","up"]},{"osd":3,"uuid":"ec6490cb-a715-4c12-a54a-225aae26fcb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":2823712137},{"type":"v1","addr":"172.21.15.45:6811","nonce":2823712137}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":2823712137},{"type":"v1","addr":"172.21.15.45:6813","nonce":2823712137}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":2823712137},{"type":"v1","addr":"172.21.15.45:6817","nonce":2823712137}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":2823712137},{"type":"v1","addr":"172.21.15.45:6815","nonce":2823712137}]},"public_addr":"172.21.15.45:6811/2823712137","cluster_addr":"172.21.15.45:6813/2823712137","heartbeat_back_addr":"172.21.15.45:6817/2823712137","heartbeat_front_addr":"172.21.15.45:6815/2823712137","state":["exists","up"]},{"osd":4,"uuid":"5d91d568-0250-4aae-8dc2-a0e079f92062","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":1396278127},{"type":"v1","addr":"172.21.15.104:6809","nonce":1396278127}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":1396278127},{"type":"v1","addr":"172.21.15.104:6811","nonce":1396278127}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":1396278127},{"type":"v1","addr":"172.21.15.104:6815","nonce":1396278127}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":1396278127},{"type":"v1","addr":"172.21.15.104:6813","nonce":1396278127}]},"public_addr":"172.21.15.104:6809/1396278127","cluster_addr":"172.21.15.104:6811/1396278127","heartbeat_back_addr":"172.21.15.104:6815/1396278127","heartbeat_front_addr":"172.21.15.104:6813/1396278127","state":["exists","up"]},{"osd":5,"uuid":"ac510b43-6fb2-4d0f-bc5f-16ac080d9e9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":4145720047},{"type":"v1","addr":"172.21.15.117:6809","nonce":4145720047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":4145720047},{"type":"v1","addr":"172.21.15.117:6811","nonce":4145720047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":4145720047},{"type":"v1","addr":"172.21.15.117:6815","nonce":4145720047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":4145720047},{"type":"v1","addr":"172.21.15.117:6813","nonce":4145720047}]},"public_addr":"172.21.15.117:6809/4145720047","cluster_addr":"172.21.15.117:6811/4145720047","heartbeat_back_addr":"172.21.15.117:6815/4145720047","heartbeat_front_addr":"172.21.15.117:6813/4145720047","state":["exists","up"]},{"osd":6,"uuid":"a41adc18-d88a-419e-b2c2-b089c1f32472","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":3254229176},{"type":"v1","addr":"172.21.15.45:6819","nonce":3254229176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":3254229176},{"type":"v1","addr":"172.21.15.45:6821","nonce":3254229176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":3254229176},{"type":"v1","addr":"172.21.15.45:6825","nonce":3254229176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":3254229176},{"type":"v1","addr":"172.21.15.45:6823","nonce":3254229176}]},"public_addr":"172.21.15.45:6819/3254229176","cluster_addr":"172.21.15.45:6821/3254229176","heartbeat_back_addr":"172.21.15.45:6825/3254229176","heartbeat_front_addr":"172.21.15.45:6823/3254229176","state":["exists","up"]},{"osd":7,"uuid":"b71712a4-7cf6-454d-9fd1-565e3df21c1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":4151077186},{"type":"v1","addr":"172.21.15.104:6817","nonce":4151077186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":4151077186},{"type":"v1","addr":"172.21.15.104:6819","nonce":4151077186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":4151077186},{"type":"v1","addr":"172.21.15.104:6823","nonce":4151077186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":4151077186},{"type":"v1","addr":"172.21.15.104:6821","nonce":4151077186}]},"public_addr":"172.21.15.104:6817/4151077186","cluster_addr":"172.21.15.104:6819/4151077186","heartbeat_back_addr":"172.21.15.104:6823/4151077186","heartbeat_front_addr":"172.21.15.104:6821/4151077186","state":["exists","up"]},{"osd":8,"uuid":"9e572aef-59a8-450e-a92e-df9dff4d57a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":1500286830},{"type":"v1","addr":"172.21.15.117:6817","nonce":1500286830}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":1500286830},{"type":"v1","addr":"172.21.15.117:6819","nonce":1500286830}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":1500286830},{"type":"v1","addr":"172.21.15.117:6823","nonce":1500286830}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":1500286830},{"type":"v1","addr":"172.21.15.117:6821","nonce":1500286830}]},"public_addr":"172.21.15.117:6817/1500286830","cluster_addr":"172.21.15.117:6819/1500286830","heartbeat_back_addr":"172.21.15.117:6823/1500286830","heartbeat_front_addr":"172.21.15.117:6821/1500286830","state":["exists","up"]},{"osd":9,"uuid":"014e9598-c413-4cc4-9bb8-61c6e8f59d30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":3106734074},{"type":"v1","addr":"172.21.15.45:6827","nonce":3106734074}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":3106734074},{"type":"v1","addr":"172.21.15.45:6829","nonce":3106734074}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":3106734074},{"type":"v1","addr":"172.21.15.45:6833","nonce":3106734074}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":3106734074},{"type":"v1","addr":"172.21.15.45:6831","nonce":3106734074}]},"public_addr":"172.21.15.45:6827/3106734074","cluster_addr":"172.21.15.45:6829/3106734074","heartbeat_back_addr":"172.21.15.45:6833/3106734074","heartbeat_front_addr":"172.21.15.45:6831/3106734074","state":["exists","up"]},{"osd":10,"uuid":"2deba364-3e39-47be-8a1f-8256df5b82d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":1404794652},{"type":"v1","addr":"172.21.15.104:6825","nonce":1404794652}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":1404794652},{"type":"v1","addr":"172.21.15.104:6827","nonce":1404794652}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":1404794652},{"type":"v1","addr":"172.21.15.104:6831","nonce":1404794652}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":1404794652},{"type":"v1","addr":"172.21.15.104:6829","nonce":1404794652}]},"public_addr":"172.21.15.104:6825/1404794652","cluster_addr":"172.21.15.104:6827/1404794652","heartbeat_back_addr":"172.21.15.104:6831/1404794652","heartbeat_front_addr":"172.21.15.104:6829/1404794652","state":["exists","up"]},{"osd":11,"uuid":"3fda4fea-59ad-4d1a-9fe6-fcb30e31fd7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":3384003243},{"type":"v1","addr":"172.21.15.117:6825","nonce":3384003243}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":3384003243},{"type":"v1","addr":"172.21.15.117:6827","nonce":3384003243}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":3384003243},{"type":"v1","addr":"172.21.15.117:6831","nonce":3384003243}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":3384003243},{"type":"v1","addr":"172.21.15.117:6829","nonce":3384003243}]},"public_addr":"172.21.15.117:6825/3384003243","cluster_addr":"172.21.15.117:6827/3384003243","heartbeat_back_addr":"172.21.15.117:6831/3384003243","heartbeat_front_addr":"172.21.15.117:6829/3384003243","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:57:14.799608+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:57:39.884668+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:58:03.327057+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:58:37.314531+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:59:03.667650+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:59:28.462823+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T17:00:02.968574+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T17:00:29.559190+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T17:00:55.291007+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T17:01:29.815977+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T17:01:57.015580+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T17:02:23.233151+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.45:0/4074982598":"2024-04-17T16:55:19.948001+0000","172.21.15.45:0/2443572576":"2024-04-17T16:55:19.948001+0000","172.21.15.45:6801/1983630136":"2024-04-17T16:55:19.948001+0000","172.21.15.45:6800/1983630136":"2024-04-17T16:55:19.948001+0000","172.21.15.45:0/3150465248":"2024-04-17T16:54:43.692466+0000","172.21.15.45:0/556738623":"2024-04-17T16:54:43.692466+0000","172.21.15.45:0/445676297":"2024-04-17T16:55:19.948001+0000","172.21.15.45:6800/2553659622":"2024-04-17T16:54:43.692466+0000","172.21.15.45:6801/2553659622":"2024-04-17T16:54:43.692466+0000","172.21.15.45:0/1184615408":"2024-04-17T16:54:43.692466+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-16T17:03:49.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:49.107+0000 7fb027200640 1 -- 172.21.15.45:0/1403698062 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fb00c0a3bb0 msgr2=0x7fb00c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:49.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:49.107+0000 7fb027200640 1 --2- 172.21.15.45:0/1403698062 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fb00c0a3bb0 0x7fb00c0a6070 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fb01800bbe0 tx=0x7fb01800b390 comp rx=0 tx=0).stop 2024-04-16T17:03:49.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:49.107+0000 7fb027200640 1 -- 172.21.15.45:0/1403698062 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb02810e440 msgr2=0x7fb0281a5cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:49.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:49.107+0000 7fb027200640 1 --2- 172.21.15.45:0/1403698062 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb02810e440 0x7fb0281a5cc0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fb02002c640 tx=0x7fb020034670 comp rx=0 tx=0).stop 2024-04-16T17:03:49.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:49.107+0000 7fb027200640 1 -- 172.21.15.45:0/1403698062 shutdown_connections 2024-04-16T17:03:49.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:49.107+0000 7fb027200640 1 --2- 172.21.15.45:0/1403698062 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fb00c0a3bb0 0x7fb00c0a6070 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:49.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:49.107+0000 7fb027200640 1 --2- 172.21.15.45:0/1403698062 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb02810e440 0x7fb0281a5cc0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:49.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:49.107+0000 7fb027200640 1 --2- 172.21.15.45:0/1403698062 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb028076040 0x7fb02819ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:49.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:49.107+0000 7fb027200640 1 --2- 172.21.15.45:0/1403698062 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb028074a60 0x7fb02819e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:49.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:49.107+0000 7fb027200640 1 -- 172.21.15.45:0/1403698062 >> 172.21.15.45:0/1403698062 conn(0x7fb0280fd3e0 msgr2=0x7fb0281017e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:03:49.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:49.107+0000 7fb027200640 1 -- 172.21.15.45:0/1403698062 shutdown_connections 2024-04-16T17:03:49.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:49.107+0000 7fb027200640 1 -- 172.21.15.45:0/1403698062 wait complete. 2024-04-16T17:03:49.128 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-16T17:03:49.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:49.357+0000 7f1d00600640 1 -- 172.21.15.45:0/50402486 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1cf8104260 msgr2=0x7f1cf8104680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:49.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:49.358+0000 7f1d00600640 1 --2- 172.21.15.45:0/50402486 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1cf8104260 0x7f1cf8104680 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f1cec009900 tx=0x7f1cec02bc30 comp rx=0 tx=0).stop 2024-04-16T17:03:49.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:49.358+0000 7f1d00600640 1 -- 172.21.15.45:0/50402486 shutdown_connections 2024-04-16T17:03:49.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:49.358+0000 7f1d00600640 1 --2- 172.21.15.45:0/50402486 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1cf8105e10 0x7f1cf810c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:49.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:49.358+0000 7f1d00600640 1 --2- 172.21.15.45:0/50402486 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1cf8105450 0x7f1cf81058d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:49.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:49.358+0000 7f1d00600640 1 --2- 172.21.15.45:0/50402486 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1cf8104260 0x7f1cf8104680 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:49.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:49.358+0000 7f1d00600640 1 -- 172.21.15.45:0/50402486 >> 172.21.15.45:0/50402486 conn(0x7f1cf80ff7f0 msgr2=0x7f1cf8101c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:03:49.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:49.358+0000 7f1d00600640 1 -- 172.21.15.45:0/50402486 shutdown_connections 2024-04-16T17:03:49.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:49.358+0000 7f1d00600640 1 -- 172.21.15.45:0/50402486 wait complete. 2024-04-16T17:03:49.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:49.359+0000 7f1d00600640 1 Processor -- start 2024-04-16T17:03:49.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:49.359+0000 7f1d00600640 1 -- start start 2024-04-16T17:03:49.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:49.359+0000 7f1d00600640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1cf8104260 0x7f1cf819a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:49.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:49.359+0000 7f1d00600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1cf8105450 0x7f1cf819aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:49.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:49.360+0000 7f1d00600640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1cf8105e10 0x7f1cf81a1af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:49.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:49.360+0000 7f1d00600640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f1cf810fef0 con 0x7f1cf8105450 2024-04-16T17:03:49.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:49.360+0000 7f1d00600640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f1cf810fd70 con 0x7f1cf8105e10 2024-04-16T17:03:49.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:49.360+0000 7f1d00600640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f1cf8110070 con 0x7f1cf8104260 2024-04-16T17:03:49.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:49.360+0000 7f1cfd200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1cf8105450 0x7f1cf819aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:49.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:49.360+0000 7f1cfdc00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1cf8104260 0x7f1cf819a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:49.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:49.360+0000 7f1cfd200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1cf8105450 0x7f1cf819aa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:44566/0 (socket says 172.21.15.45:44566) 2024-04-16T17:03:49.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:49.360+0000 7f1cfd200640 1 -- 172.21.15.45:0/2495912894 learned_addr learned my addr 172.21.15.45:0/2495912894 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:03:49.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:49.360+0000 7f1cfe600640 1 --2- 172.21.15.45:0/2495912894 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1cf8105e10 0x7f1cf81a1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:49.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:49.360+0000 7f1cfd200640 1 -- 172.21.15.45:0/2495912894 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1cf8104260 msgr2=0x7f1cf819a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:49.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:49.360+0000 7f1cfd200640 1 --2- 172.21.15.45:0/2495912894 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1cf8104260 0x7f1cf819a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:49.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:49.360+0000 7f1cfd200640 1 -- 172.21.15.45:0/2495912894 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1cf8105e10 msgr2=0x7f1cf81a1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:49.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:49.360+0000 7f1cfd200640 1 --2- 172.21.15.45:0/2495912894 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1cf8105e10 0x7f1cf81a1af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:49.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:49.360+0000 7f1cfd200640 1 -- 172.21.15.45:0/2495912894 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cf81a2160 con 0x7f1cf8105450 2024-04-16T17:03:49.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:49.360+0000 7f1cfe600640 1 --2- 172.21.15.45:0/2495912894 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1cf8105e10 0x7f1cf81a1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:03:49.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:49.361+0000 7f1cfd200640 1 --2- 172.21.15.45:0/2495912894 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1cf8105450 0x7f1cf819aa80 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f1ce400e9f0 tx=0x7f1ce400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:03:49.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:49.361+0000 7f1cf6000640 1 -- 172.21.15.45:0/2495912894 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ce400cd80 con 0x7f1cf8105450 2024-04-16T17:03:49.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:49.361+0000 7f1cf6000640 1 -- 172.21.15.45:0/2495912894 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ce4010910 con 0x7f1cf8105450 2024-04-16T17:03:49.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:49.361+0000 7f1cf6000640 1 -- 172.21.15.45:0/2495912894 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ce4018990 con 0x7f1cf8105450 2024-04-16T17:03:49.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:49.361+0000 7f1d00600640 1 -- 172.21.15.45:0/2495912894 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1cf81a2450 con 0x7f1cf8105450 2024-04-16T17:03:49.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:49.361+0000 7f1d00600640 1 -- 172.21.15.45:0/2495912894 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1cf81a29a0 con 0x7f1cf8105450 2024-04-16T17:03:49.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:49.362+0000 7f1d00600640 1 -- 172.21.15.45:0/2495912894 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1cf810b650 con 0x7f1cf8105450 2024-04-16T17:03:49.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:49.365+0000 7f1cf6000640 1 -- 172.21.15.45:0/2495912894 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f1ce4010ac0 con 0x7f1cf8105450 2024-04-16T17:03:49.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:49.366+0000 7f1cf6000640 1 --2- 172.21.15.45:0/2495912894 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f1ce00a3bb0 0x7f1ce00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:49.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:49.367+0000 7f1cf6000640 1 -- 172.21.15.45:0/2495912894 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f1ce40d0a70 con 0x7f1cf8105450 2024-04-16T17:03:49.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:49.367+0000 7f1cfdc00640 1 --2- 172.21.15.45:0/2495912894 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f1ce00a3bb0 0x7f1ce00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:49.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:49.368+0000 7f1cfdc00640 1 --2- 172.21.15.45:0/2495912894 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f1ce00a3bb0 0x7f1ce00a6070 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f1cec000c00 tx=0x7f1cec005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:03:49.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:49.372+0000 7f1cf6000640 1 -- 172.21.15.45:0/2495912894 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1ce409d1b0 con 0x7f1cf8105450 2024-04-16T17:03:49.478 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:03:49 smithi045 ceph-mon[31676]: pgmap v334: 129 pgs: 129 active+clean; 581 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-04-16T17:03:49.478 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:03:49 smithi045 ceph-mon[31676]: from='client.? 172.21.15.45:0/3697534986' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T17:03:49.478 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:03:49 smithi045 ceph-mon[31676]: from='client.? 172.21.15.45:0/1403698062' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T17:03:49.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:49.621+0000 7f1d00600640 1 -- 172.21.15.45:0/2495912894 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f1cf8062c00 con 0x7f1cf8105450 2024-04-16T17:03:49.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:03:49 smithi104 ceph-mon[36457]: pgmap v334: 129 pgs: 129 active+clean; 581 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-04-16T17:03:49.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:03:49 smithi104 ceph-mon[36457]: from='client.? 172.21.15.45:0/3697534986' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T17:03:49.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:03:49 smithi104 ceph-mon[36457]: from='client.? 172.21.15.45:0/1403698062' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T17:03:49.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:03:49 smithi117 ceph-mon[36730]: pgmap v334: 129 pgs: 129 active+clean; 581 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-04-16T17:03:49.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:03:49 smithi117 ceph-mon[36730]: from='client.? 172.21.15.45:0/3697534986' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T17:03:49.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:03:49 smithi117 ceph-mon[36730]: from='client.? 172.21.15.45:0/1403698062' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T17:03:50.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:50.335+0000 7f1cf6000640 1 -- 172.21.15.45:0/2495912894 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v18) ==== 118+0+0 (secure 0 0 0) 0x7f1ce409cab0 con 0x7f1cf8105450 2024-04-16T17:03:50.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:50.339+0000 7f1d00600640 1 -- 172.21.15.45:0/2495912894 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f1ce00a3bb0 msgr2=0x7f1ce00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:50.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:50.339+0000 7f1d00600640 1 --2- 172.21.15.45:0/2495912894 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f1ce00a3bb0 0x7f1ce00a6070 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f1cec000c00 tx=0x7f1cec005fd0 comp rx=0 tx=0).stop 2024-04-16T17:03:50.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:50.339+0000 7f1d00600640 1 -- 172.21.15.45:0/2495912894 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1cf8105450 msgr2=0x7f1cf819aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:50.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:50.339+0000 7f1d00600640 1 --2- 172.21.15.45:0/2495912894 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1cf8105450 0x7f1cf819aa80 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f1ce400e9f0 tx=0x7f1ce400eeb0 comp rx=0 tx=0).stop 2024-04-16T17:03:50.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:50.340+0000 7f1d00600640 1 -- 172.21.15.45:0/2495912894 shutdown_connections 2024-04-16T17:03:50.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:50.340+0000 7f1d00600640 1 --2- 172.21.15.45:0/2495912894 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f1ce00a3bb0 0x7f1ce00a6070 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:50.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:50.340+0000 7f1d00600640 1 --2- 172.21.15.45:0/2495912894 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1cf8105e10 0x7f1cf81a1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:50.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:50.340+0000 7f1d00600640 1 --2- 172.21.15.45:0/2495912894 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1cf8105450 0x7f1cf819aa80 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:50.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:50.340+0000 7f1d00600640 1 --2- 172.21.15.45:0/2495912894 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1cf8104260 0x7f1cf819a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:50.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:50.340+0000 7f1d00600640 1 -- 172.21.15.45:0/2495912894 >> 172.21.15.45:0/2495912894 conn(0x7f1cf80ff7f0 msgr2=0x7f1cf8108820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:03:50.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:50.340+0000 7f1d00600640 1 -- 172.21.15.45:0/2495912894 shutdown_connections 2024-04-16T17:03:50.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:50.340+0000 7f1d00600640 1 -- 172.21.15.45:0/2495912894 wait complete. 2024-04-16T17:03:50.366 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 3, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated --size 25000000000'}} 2024-04-16T17:03:50.367 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 3 2024-04-16T17:03:50.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:50.600+0000 7f924ce00640 1 -- 172.21.15.45:0/2067975568 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f924810a010 msgr2=0x7f92481108a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:50.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:50.600+0000 7f924ce00640 1 --2- 172.21.15.45:0/2067975568 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f924810a010 0x7f92481108a0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f9240009900 tx=0x7f924002bc40 comp rx=0 tx=0).stop 2024-04-16T17:03:50.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:50.600+0000 7f924ce00640 1 -- 172.21.15.45:0/2067975568 shutdown_connections 2024-04-16T17:03:50.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:50.600+0000 7f924ce00640 1 --2- 172.21.15.45:0/2067975568 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f924810a010 0x7f92481108a0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:50.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:50.600+0000 7f924ce00640 1 --2- 172.21.15.45:0/2067975568 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9248109650 0x7f9248109ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:50.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:50.600+0000 7f924ce00640 1 --2- 172.21.15.45:0/2067975568 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9248108460 0x7f9248108880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:50.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:50.600+0000 7f924ce00640 1 -- 172.21.15.45:0/2067975568 >> 172.21.15.45:0/2067975568 conn(0x7f9248077400 msgr2=0x7f9248079840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:03:50.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:50.600+0000 7f924ce00640 1 -- 172.21.15.45:0/2067975568 shutdown_connections 2024-04-16T17:03:50.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:50.600+0000 7f924ce00640 1 -- 172.21.15.45:0/2067975568 wait complete. 2024-04-16T17:03:50.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:50.602+0000 7f924ce00640 1 Processor -- start 2024-04-16T17:03:50.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:50.602+0000 7f924ce00640 1 -- start start 2024-04-16T17:03:50.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:50.602+0000 7f924ce00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9248108460 0x7f924819e690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:50.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:50.602+0000 7f924ce00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9248109650 0x7f924819ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:50.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:50.602+0000 7f924ce00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f924810a010 0x7f92481a5c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:50.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:50.602+0000 7f924ce00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f92481140f0 con 0x7f9248108460 2024-04-16T17:03:50.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:50.602+0000 7f924ce00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f9248113f70 con 0x7f924810a010 2024-04-16T17:03:50.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:50.602+0000 7f924ce00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f9248114270 con 0x7f9248109650 2024-04-16T17:03:50.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:50.602+0000 7f923ea00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9248109650 0x7f924819ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:50.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:50.602+0000 7f9247400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9248108460 0x7f924819e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:50.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:50.603+0000 7f923ea00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9248109650 0x7f924819ebd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:55660/0 (socket says 172.21.15.45:55660) 2024-04-16T17:03:50.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:50.603+0000 7f923ea00640 1 -- 172.21.15.45:0/1705195706 learned_addr learned my addr 172.21.15.45:0/1705195706 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:03:50.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:50.603+0000 7f9247e00640 1 --2- 172.21.15.45:0/1705195706 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f924810a010 0x7f92481a5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:50.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:50.603+0000 7f923ea00640 1 -- 172.21.15.45:0/1705195706 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f924810a010 msgr2=0x7f92481a5c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:50.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:50.603+0000 7f923ea00640 1 --2- 172.21.15.45:0/1705195706 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f924810a010 0x7f92481a5c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:50.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:50.603+0000 7f923ea00640 1 -- 172.21.15.45:0/1705195706 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9248108460 msgr2=0x7f924819e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:50.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:50.603+0000 7f923ea00640 1 --2- 172.21.15.45:0/1705195706 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9248108460 0x7f924819e690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:50.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:50.603+0000 7f923ea00640 1 -- 172.21.15.45:0/1705195706 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92481a62b0 con 0x7f9248109650 2024-04-16T17:03:50.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:50.603+0000 7f923ea00640 1 --2- 172.21.15.45:0/1705195706 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9248109650 0x7f924819ebd0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f923000e9f0 tx=0x7f923000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:03:50.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:50.604+0000 7f9244c00640 1 -- 172.21.15.45:0/1705195706 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f923000cd80 con 0x7f9248109650 2024-04-16T17:03:50.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:50.604+0000 7f924ce00640 1 -- 172.21.15.45:0/1705195706 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92481a65a0 con 0x7f9248109650 2024-04-16T17:03:50.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:50.604+0000 7f9244c00640 1 -- 172.21.15.45:0/1705195706 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9230010910 con 0x7f9248109650 2024-04-16T17:03:50.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:50.604+0000 7f924ce00640 1 -- 172.21.15.45:0/1705195706 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92481a6b40 con 0x7f9248109650 2024-04-16T17:03:50.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:50.604+0000 7f9244c00640 1 -- 172.21.15.45:0/1705195706 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9230018980 con 0x7f9248109650 2024-04-16T17:03:50.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:50.605+0000 7f924ce00640 1 -- 172.21.15.45:0/1705195706 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f92240052c0 con 0x7f9248109650 2024-04-16T17:03:50.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:50.607+0000 7f9244c00640 1 -- 172.21.15.45:0/1705195706 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f9230010ac0 con 0x7f9248109650 2024-04-16T17:03:50.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:50.608+0000 7f9244c00640 1 --2- 172.21.15.45:0/1705195706 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f922c0a3bb0 0x7f922c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:50.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:50.608+0000 7f9244c00640 1 -- 172.21.15.45:0/1705195706 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f92300d0950 con 0x7f9248109650 2024-04-16T17:03:50.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:50.609+0000 7f9247400640 1 --2- 172.21.15.45:0/1705195706 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f922c0a3bb0 0x7f922c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:50.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:50.609+0000 7f9247400640 1 --2- 172.21.15.45:0/1705195706 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f922c0a3bb0 0x7f922c0a6070 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f92380099a0 tx=0x7f9238009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:03:50.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:50.611+0000 7f9244c00640 1 -- 172.21.15.45:0/1705195706 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f923009ca40 con 0x7f9248109650 2024-04-16T17:03:50.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:03:50 smithi104 ceph-mon[36457]: from='client.? 172.21.15.45:0/2495912894' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-16T17:03:50.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:03:50 smithi045 ceph-mon[31676]: from='client.? 172.21.15.45:0/2495912894' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-16T17:03:50.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:03:50 smithi117 ceph-mon[36730]: from='client.? 172.21.15.45:0/2495912894' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-16T17:03:50.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:50.862+0000 7f924ce00640 1 -- 172.21.15.45:0/1705195706 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) -- 0x7f9224005880 con 0x7f9248109650 2024-04-16T17:03:51.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:51.350+0000 7f9244c00640 1 -- 172.21.15.45:0/1705195706 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v19) ==== 105+0+0 (secure 0 0 0) 0x7f92300a0d40 con 0x7f9248109650 2024-04-16T17:03:51.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:51.354+0000 7f924ce00640 1 -- 172.21.15.45:0/1705195706 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f922c0a3bb0 msgr2=0x7f922c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:51.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:51.354+0000 7f924ce00640 1 --2- 172.21.15.45:0/1705195706 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f922c0a3bb0 0x7f922c0a6070 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f92380099a0 tx=0x7f9238009500 comp rx=0 tx=0).stop 2024-04-16T17:03:51.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:51.354+0000 7f924ce00640 1 -- 172.21.15.45:0/1705195706 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9248109650 msgr2=0x7f924819ebd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:51.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:51.354+0000 7f924ce00640 1 --2- 172.21.15.45:0/1705195706 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9248109650 0x7f924819ebd0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f923000e9f0 tx=0x7f923000eeb0 comp rx=0 tx=0).stop 2024-04-16T17:03:51.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:51.355+0000 7f924ce00640 1 -- 172.21.15.45:0/1705195706 shutdown_connections 2024-04-16T17:03:51.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:51.355+0000 7f924ce00640 1 --2- 172.21.15.45:0/1705195706 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f922c0a3bb0 0x7f922c0a6070 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:51.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:51.355+0000 7f924ce00640 1 --2- 172.21.15.45:0/1705195706 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9248109650 0x7f924819ebd0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:51.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:51.355+0000 7f924ce00640 1 --2- 172.21.15.45:0/1705195706 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f924810a010 0x7f92481a5c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:51.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:51.355+0000 7f924ce00640 1 --2- 172.21.15.45:0/1705195706 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9248108460 0x7f924819e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:51.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:51.355+0000 7f924ce00640 1 -- 172.21.15.45:0/1705195706 >> 172.21.15.45:0/1705195706 conn(0x7f9248077400 msgr2=0x7f924810c950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:03:51.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:51.355+0000 7f924ce00640 1 -- 172.21.15.45:0/1705195706 shutdown_connections 2024-04-16T17:03:51.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:51.355+0000 7f924ce00640 1 -- 172.21.15.45:0/1705195706 wait complete. 2024-04-16T17:03:51.386 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay true 2024-04-16T17:03:51.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:51.598+0000 7f09b3c00640 1 -- 172.21.15.45:0/2355728943 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f09ac108660 msgr2=0x7f09ac108a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:51.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:51.598+0000 7f09b3c00640 1 --2- 172.21.15.45:0/2355728943 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f09ac108660 0x7f09ac108a80 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f09a0009900 tx=0x7f09a002bc50 comp rx=0 tx=0).stop 2024-04-16T17:03:51.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:51.598+0000 7f09b3c00640 1 -- 172.21.15.45:0/2355728943 shutdown_connections 2024-04-16T17:03:51.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:51.598+0000 7f09b3c00640 1 --2- 172.21.15.45:0/2355728943 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f09ac109850 0x7f09ac109cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:51.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:51.598+0000 7f09b3c00640 1 --2- 172.21.15.45:0/2355728943 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f09ac10a210 0x7f09ac110aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:51.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:51.598+0000 7f09b3c00640 1 --2- 172.21.15.45:0/2355728943 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f09ac108660 0x7f09ac108a80 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:51.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:51.598+0000 7f09b3c00640 1 -- 172.21.15.45:0/2355728943 >> 172.21.15.45:0/2355728943 conn(0x7f09ac077600 msgr2=0x7f09ac079a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:03:51.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:51.598+0000 7f09b3c00640 1 -- 172.21.15.45:0/2355728943 shutdown_connections 2024-04-16T17:03:51.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:51.598+0000 7f09b3c00640 1 -- 172.21.15.45:0/2355728943 wait complete. 2024-04-16T17:03:51.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:51.599+0000 7f09b3c00640 1 Processor -- start 2024-04-16T17:03:51.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:51.599+0000 7f09b3c00640 1 -- start start 2024-04-16T17:03:51.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:51.599+0000 7f09b3c00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f09ac108660 0x7f09ac19e970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:51.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:51.599+0000 7f09b3c00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f09ac109850 0x7f09ac19eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:51.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:51.600+0000 7f09b3c00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f09ac10a210 0x7f09ac1a5f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:51.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:51.600+0000 7f09b3c00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f09ac114300 con 0x7f09ac108660 2024-04-16T17:03:51.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:51.600+0000 7f09b3c00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f09ac114180 con 0x7f09ac10a210 2024-04-16T17:03:51.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:51.600+0000 7f09b3c00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f09ac114480 con 0x7f09ac109850 2024-04-16T17:03:51.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:51.600+0000 7f09b1200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f09ac108660 0x7f09ac19e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:51.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:51.600+0000 7f09b1200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f09ac108660 0x7f09ac19e970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:44870/0 (socket says 172.21.15.45:44870) 2024-04-16T17:03:51.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:51.600+0000 7f09b1200640 1 -- 172.21.15.45:0/909731725 learned_addr learned my addr 172.21.15.45:0/909731725 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:03:51.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:51.600+0000 7f09abe00640 1 --2- 172.21.15.45:0/909731725 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f09ac109850 0x7f09ac19eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:51.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:51.600+0000 7f09b1c00640 1 --2- 172.21.15.45:0/909731725 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f09ac10a210 0x7f09ac1a5f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:51.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:51.600+0000 7f09b1200640 1 -- 172.21.15.45:0/909731725 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f09ac109850 msgr2=0x7f09ac19eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:51.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:51.600+0000 7f09b1200640 1 --2- 172.21.15.45:0/909731725 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f09ac109850 0x7f09ac19eeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:51.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:51.600+0000 7f09b1200640 1 -- 172.21.15.45:0/909731725 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f09ac10a210 msgr2=0x7f09ac1a5f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:51.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:51.600+0000 7f09b1200640 1 --2- 172.21.15.45:0/909731725 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f09ac10a210 0x7f09ac1a5f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:51.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:51.600+0000 7f09b1200640 1 -- 172.21.15.45:0/909731725 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09ac1a6590 con 0x7f09ac108660 2024-04-16T17:03:51.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:51.600+0000 7f09abe00640 1 --2- 172.21.15.45:0/909731725 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f09ac109850 0x7f09ac19eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:03:51.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:51.601+0000 7f09b1200640 1 --2- 172.21.15.45:0/909731725 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f09ac108660 0x7f09ac19e970 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f09a002c260 tx=0x7f09a002c670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:03:51.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:51.601+0000 7f09a9600640 1 -- 172.21.15.45:0/909731725 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09a0041070 con 0x7f09ac108660 2024-04-16T17:03:51.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:51.601+0000 7f09a9600640 1 -- 172.21.15.45:0/909731725 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f09a0004450 con 0x7f09ac108660 2024-04-16T17:03:51.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:51.601+0000 7f09b3c00640 1 -- 172.21.15.45:0/909731725 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09ac1a6820 con 0x7f09ac108660 2024-04-16T17:03:51.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:51.601+0000 7f09a9600640 1 -- 172.21.15.45:0/909731725 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09a0003e20 con 0x7f09ac108660 2024-04-16T17:03:51.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:51.601+0000 7f09b3c00640 1 -- 172.21.15.45:0/909731725 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09ac1a6d70 con 0x7f09ac108660 2024-04-16T17:03:51.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:51.603+0000 7f09b3c00640 1 -- 172.21.15.45:0/909731725 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09ac10f980 con 0x7f09ac108660 2024-04-16T17:03:51.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:51.604+0000 7f09a9600640 1 -- 172.21.15.45:0/909731725 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f09a0044070 con 0x7f09ac108660 2024-04-16T17:03:51.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:51.606+0000 7f09a9600640 1 --2- 172.21.15.45:0/909731725 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f09940a3bb0 0x7f09940a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:51.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:51.606+0000 7f09a9600640 1 -- 172.21.15.45:0/909731725 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f09a00f2100 con 0x7f09ac108660 2024-04-16T17:03:51.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:51.606+0000 7f09abe00640 1 --2- 172.21.15.45:0/909731725 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f09940a3bb0 0x7f09940a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:51.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:51.607+0000 7f09abe00640 1 --2- 172.21.15.45:0/909731725 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f09940a3bb0 0x7f09940a6070 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f099c0095f0 tx=0x7f099c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:03:51.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:51.610+0000 7f09a9600640 1 -- 172.21.15.45:0/909731725 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f09a00be840 con 0x7f09ac108660 2024-04-16T17:03:51.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:03:51 smithi104 ceph-mon[36457]: pgmap v335: 129 pgs: 129 active+clean; 581 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-04-16T17:03:51.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:03:51 smithi104 ceph-mon[36457]: from='client.? 172.21.15.45:0/2495912894' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-16T17:03:51.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:03:51 smithi104 ceph-mon[36457]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-16T17:03:51.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:03:51 smithi104 ceph-mon[36457]: from='client.? 172.21.15.45:0/1705195706' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-04-16T17:03:51.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:03:51 smithi104 ceph-mon[36457]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-04-16T17:03:51.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:03:51 smithi045 ceph-mon[31676]: pgmap v335: 129 pgs: 129 active+clean; 581 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-04-16T17:03:51.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:03:51 smithi045 ceph-mon[31676]: from='client.? 172.21.15.45:0/2495912894' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-16T17:03:51.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:03:51 smithi045 ceph-mon[31676]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-16T17:03:51.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:03:51 smithi045 ceph-mon[31676]: from='client.? 172.21.15.45:0/1705195706' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-04-16T17:03:51.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:03:51 smithi045 ceph-mon[31676]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-04-16T17:03:51.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:03:51 smithi117 ceph-mon[36730]: pgmap v335: 129 pgs: 129 active+clean; 581 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-04-16T17:03:51.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:03:51 smithi117 ceph-mon[36730]: from='client.? 172.21.15.45:0/2495912894' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-16T17:03:51.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:03:51 smithi117 ceph-mon[36730]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-16T17:03:51.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:03:51 smithi117 ceph-mon[36730]: from='client.? 172.21.15.45:0/1705195706' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-04-16T17:03:51.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:03:51 smithi117 ceph-mon[36730]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-04-16T17:03:51.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:51.864+0000 7f09b3c00640 1 -- 172.21.15.45:0/909731725 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7f09ac02d620 con 0x7f09ac108660 2024-04-16T17:03:52.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:52.359+0000 7f09a9600640 1 -- 172.21.15.45:0/909731725 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v21) ==== 121+0+0 (secure 0 0 0) 0x7f09a00be140 con 0x7f09ac108660 2024-04-16T17:03:52.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:52.364+0000 7f09b3c00640 1 -- 172.21.15.45:0/909731725 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f09940a3bb0 msgr2=0x7f09940a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:52.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:52.364+0000 7f09b3c00640 1 --2- 172.21.15.45:0/909731725 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f09940a3bb0 0x7f09940a6070 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f099c0095f0 tx=0x7f099c005f90 comp rx=0 tx=0).stop 2024-04-16T17:03:52.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:52.364+0000 7f09b3c00640 1 -- 172.21.15.45:0/909731725 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f09ac108660 msgr2=0x7f09ac19e970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:52.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:52.364+0000 7f09b3c00640 1 --2- 172.21.15.45:0/909731725 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f09ac108660 0x7f09ac19e970 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f09a002c260 tx=0x7f09a002c670 comp rx=0 tx=0).stop 2024-04-16T17:03:52.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:52.364+0000 7f09b3c00640 1 -- 172.21.15.45:0/909731725 shutdown_connections 2024-04-16T17:03:52.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:52.364+0000 7f09b3c00640 1 --2- 172.21.15.45:0/909731725 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f09940a3bb0 0x7f09940a6070 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:52.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:52.364+0000 7f09b3c00640 1 --2- 172.21.15.45:0/909731725 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f09ac109850 0x7f09ac19eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:52.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:52.364+0000 7f09b3c00640 1 --2- 172.21.15.45:0/909731725 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f09ac10a210 0x7f09ac1a5f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:52.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:52.364+0000 7f09b3c00640 1 --2- 172.21.15.45:0/909731725 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f09ac108660 0x7f09ac19e970 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:52.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:52.364+0000 7f09b3c00640 1 -- 172.21.15.45:0/909731725 >> 172.21.15.45:0/909731725 conn(0x7f09ac077600 msgr2=0x7f09ac10cb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:03:52.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:52.364+0000 7f09b3c00640 1 -- 172.21.15.45:0/909731725 shutdown_connections 2024-04-16T17:03:52.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:52.364+0000 7f09b3c00640 1 -- 172.21.15.45:0/909731725 wait complete. 2024-04-16T17:03:52.393 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs session_timeout 300 2024-04-16T17:03:52.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:52.593+0000 7f8d40a00640 1 -- 172.21.15.45:0/2014782549 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8d3c102050 msgr2=0x7f8d3c102470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:52.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:52.593+0000 7f8d40a00640 1 --2- 172.21.15.45:0/2014782549 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8d3c102050 0x7f8d3c102470 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f8d2c009860 tx=0x7f8d2c02bb70 comp rx=0 tx=0).stop 2024-04-16T17:03:52.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:52.594+0000 7f8d40a00640 1 -- 172.21.15.45:0/2014782549 shutdown_connections 2024-04-16T17:03:52.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:52.594+0000 7f8d40a00640 1 --2- 172.21.15.45:0/2014782549 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8d3c103240 0x7f8d3c1036c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:52.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:52.594+0000 7f8d40a00640 1 --2- 172.21.15.45:0/2014782549 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8d3c103c00 0x7f8d3c10a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:52.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:52.594+0000 7f8d40a00640 1 --2- 172.21.15.45:0/2014782549 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8d3c102050 0x7f8d3c102470 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:52.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:52.594+0000 7f8d40a00640 1 -- 172.21.15.45:0/2014782549 >> 172.21.15.45:0/2014782549 conn(0x7f8d3c0fd5e0 msgr2=0x7f8d3c0ffa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:03:52.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:52.594+0000 7f8d40a00640 1 -- 172.21.15.45:0/2014782549 shutdown_connections 2024-04-16T17:03:52.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:52.594+0000 7f8d40a00640 1 -- 172.21.15.45:0/2014782549 wait complete. 2024-04-16T17:03:52.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:52.595+0000 7f8d40a00640 1 Processor -- start 2024-04-16T17:03:52.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:52.595+0000 7f8d40a00640 1 -- start start 2024-04-16T17:03:52.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:52.596+0000 7f8d40a00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8d3c102050 0x7f8d3c19a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:52.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:52.596+0000 7f8d40a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8d3c103240 0x7f8d3c19aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:52.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:52.596+0000 7f8d40a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8d3c103c00 0x7f8d3c1a1b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:52.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:52.596+0000 7f8d40a00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f8d3c10dce0 con 0x7f8d3c103240 2024-04-16T17:03:52.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:52.596+0000 7f8d40a00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f8d3c10db60 con 0x7f8d3c102050 2024-04-16T17:03:52.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:52.596+0000 7f8d40a00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f8d3c10de60 con 0x7f8d3c103c00 2024-04-16T17:03:52.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:52.596+0000 7f8d39e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8d3c102050 0x7f8d3c19a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:52.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:52.596+0000 7f8d39400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8d3c103240 0x7f8d3c19aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:52.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:52.596+0000 7f8d39e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8d3c102050 0x7f8d3c19a5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:34266/0 (socket says 172.21.15.45:34266) 2024-04-16T17:03:52.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:52.596+0000 7f8d39e00640 1 -- 172.21.15.45:0/1243102437 learned_addr learned my addr 172.21.15.45:0/1243102437 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:03:52.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:52.596+0000 7f8d3a800640 1 --2- 172.21.15.45:0/1243102437 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8d3c103c00 0x7f8d3c1a1b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:52.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:52.596+0000 7f8d39e00640 1 -- 172.21.15.45:0/1243102437 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8d3c103c00 msgr2=0x7f8d3c1a1b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:52.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:52.596+0000 7f8d39e00640 1 --2- 172.21.15.45:0/1243102437 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8d3c103c00 0x7f8d3c1a1b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:52.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:52.596+0000 7f8d39e00640 1 -- 172.21.15.45:0/1243102437 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8d3c103240 msgr2=0x7f8d3c19aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:52.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:52.596+0000 7f8d39e00640 1 --2- 172.21.15.45:0/1243102437 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8d3c103240 0x7f8d3c19aaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:52.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:52.596+0000 7f8d39e00640 1 -- 172.21.15.45:0/1243102437 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d3c1a21d0 con 0x7f8d3c102050 2024-04-16T17:03:52.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:52.597+0000 7f8d39e00640 1 --2- 172.21.15.45:0/1243102437 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8d3c102050 0x7f8d3c19a5b0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f8d2c02c0b0 tx=0x7f8d2c032430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:03:52.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:52.597+0000 7f8d32a00640 1 -- 172.21.15.45:0/1243102437 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d2c041020 con 0x7f8d3c102050 2024-04-16T17:03:52.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:52.597+0000 7f8d32a00640 1 -- 172.21.15.45:0/1243102437 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8d2c02c9f0 con 0x7f8d3c102050 2024-04-16T17:03:52.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:52.597+0000 7f8d40a00640 1 -- 172.21.15.45:0/1243102437 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d3c1a2460 con 0x7f8d3c102050 2024-04-16T17:03:52.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:52.597+0000 7f8d32a00640 1 -- 172.21.15.45:0/1243102437 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d2c039040 con 0x7f8d3c102050 2024-04-16T17:03:52.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:52.597+0000 7f8d40a00640 1 -- 172.21.15.45:0/1243102437 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d3c1a29a0 con 0x7f8d3c102050 2024-04-16T17:03:52.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:52.598+0000 7f8d40a00640 1 -- 172.21.15.45:0/1243102437 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8d3c1a2c00 con 0x7f8d3c102050 2024-04-16T17:03:52.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:52.600+0000 7f8d32a00640 1 -- 172.21.15.45:0/1243102437 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f8d2c044020 con 0x7f8d3c102050 2024-04-16T17:03:52.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:52.601+0000 7f8d32a00640 1 --2- 172.21.15.45:0/1243102437 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f8d200a3c30 0x7f8d200a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:52.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:52.602+0000 7f8d32a00640 1 -- 172.21.15.45:0/1243102437 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f8d2c0f1ef0 con 0x7f8d3c102050 2024-04-16T17:03:52.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:52.602+0000 7f8d39400640 1 --2- 172.21.15.45:0/1243102437 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f8d200a3c30 0x7f8d200a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:52.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:52.603+0000 7f8d39400640 1 --2- 172.21.15.45:0/1243102437 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f8d200a3c30 0x7f8d200a60f0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f8d3c19bac0 tx=0x7f8d28005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:03:52.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:52.606+0000 7f8d32a00640 1 -- 172.21.15.45:0/1243102437 <== mon.2 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8d2c04b020 con 0x7f8d3c102050 2024-04-16T17:03:52.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:03:52 smithi104 ceph-mon[36457]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-16T17:03:52.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:03:52 smithi104 ceph-mon[36457]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-04-16T17:03:52.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:03:52 smithi104 ceph-mon[36457]: daemon mds.a assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-04-16T17:03:52.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:03:52 smithi104 ceph-mon[36457]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-16T17:03:52.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:03:52 smithi104 ceph-mon[36457]: fsmap cephfs:2 {0=b=up:active,1=a=up:creating} 10 up:standby 2024-04-16T17:03:52.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:03:52 smithi104 ceph-mon[36457]: daemon mds.a is now active in filesystem cephfs as rank 1 2024-04-16T17:03:52.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:03:52 smithi104 ceph-mon[36457]: from='client.? 172.21.15.45:0/909731725' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-16T17:03:52.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:03:52 smithi045 ceph-mon[31676]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-16T17:03:52.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:03:52 smithi045 ceph-mon[31676]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-04-16T17:03:52.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:03:52 smithi045 ceph-mon[31676]: daemon mds.a assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-04-16T17:03:52.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:03:52 smithi045 ceph-mon[31676]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-16T17:03:52.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:03:52 smithi045 ceph-mon[31676]: fsmap cephfs:2 {0=b=up:active,1=a=up:creating} 10 up:standby 2024-04-16T17:03:52.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:03:52 smithi045 ceph-mon[31676]: daemon mds.a is now active in filesystem cephfs as rank 1 2024-04-16T17:03:52.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:03:52 smithi045 ceph-mon[31676]: from='client.? 172.21.15.45:0/909731725' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-16T17:03:52.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:03:52 smithi117 ceph-mon[36730]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-16T17:03:52.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:03:52 smithi117 ceph-mon[36730]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-04-16T17:03:52.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:03:52 smithi117 ceph-mon[36730]: daemon mds.a assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-04-16T17:03:52.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:03:52 smithi117 ceph-mon[36730]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-16T17:03:52.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:03:52 smithi117 ceph-mon[36730]: fsmap cephfs:2 {0=b=up:active,1=a=up:creating} 10 up:standby 2024-04-16T17:03:52.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:03:52 smithi117 ceph-mon[36730]: daemon mds.a is now active in filesystem cephfs as rank 1 2024-04-16T17:03:52.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:03:52 smithi117 ceph-mon[36730]: from='client.? 172.21.15.45:0/909731725' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-16T17:03:52.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:52.854+0000 7f8d40a00640 1 -- 172.21.15.45:0/1243102437 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7f8d3c1a2dd0 con 0x7f8d3c102050 2024-04-16T17:03:53.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:53.376+0000 7f8d32a00640 1 -- 172.21.15.45:0/1243102437 <== mon.2 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v23) ==== 115+0+0 (secure 0 0 0) 0x7f8d2c0be630 con 0x7f8d3c102050 2024-04-16T17:03:53.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:53.380+0000 7f8d40a00640 1 -- 172.21.15.45:0/1243102437 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f8d200a3c30 msgr2=0x7f8d200a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:53.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:53.380+0000 7f8d40a00640 1 --2- 172.21.15.45:0/1243102437 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f8d200a3c30 0x7f8d200a60f0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f8d3c19bac0 tx=0x7f8d28005f90 comp rx=0 tx=0).stop 2024-04-16T17:03:53.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:53.380+0000 7f8d40a00640 1 -- 172.21.15.45:0/1243102437 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8d3c102050 msgr2=0x7f8d3c19a5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:53.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:53.380+0000 7f8d40a00640 1 --2- 172.21.15.45:0/1243102437 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8d3c102050 0x7f8d3c19a5b0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f8d2c02c0b0 tx=0x7f8d2c032430 comp rx=0 tx=0).stop 2024-04-16T17:03:53.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:53.380+0000 7f8d40a00640 1 -- 172.21.15.45:0/1243102437 shutdown_connections 2024-04-16T17:03:53.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:53.380+0000 7f8d40a00640 1 --2- 172.21.15.45:0/1243102437 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f8d200a3c30 0x7f8d200a60f0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:53.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:53.380+0000 7f8d40a00640 1 --2- 172.21.15.45:0/1243102437 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8d3c103c00 0x7f8d3c1a1b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:53.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:53.380+0000 7f8d40a00640 1 --2- 172.21.15.45:0/1243102437 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8d3c103240 0x7f8d3c19aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:53.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:53.380+0000 7f8d40a00640 1 --2- 172.21.15.45:0/1243102437 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8d3c102050 0x7f8d3c19a5b0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:53.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:53.380+0000 7f8d40a00640 1 -- 172.21.15.45:0/1243102437 >> 172.21.15.45:0/1243102437 conn(0x7f8d3c0fd5e0 msgr2=0x7f8d3c074230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:03:53.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:53.381+0000 7f8d40a00640 1 -- 172.21.15.45:0/1243102437 shutdown_connections 2024-04-16T17:03:53.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:53.381+0000 7f8d40a00640 1 -- 172.21.15.45:0/1243102437 wait complete. 2024-04-16T17:03:53.405 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--namespace-isolated --size 25000000000'} subvols for filesystem 'cephfs' 2024-04-16T17:03:53.405 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup create cephfs qa 2024-04-16T17:03:53.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:53.625+0000 7fdfbd000640 1 -- 172.21.15.45:0/1787170994 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdfb8105cf0 msgr2=0x7fdfb810c580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:53.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:53.625+0000 7fdfbd000640 1 --2- 172.21.15.45:0/1787170994 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdfb8105cf0 0x7fdfb810c580 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fdfb0009900 tx=0x7fdfb002bc30 comp rx=0 tx=0).stop 2024-04-16T17:03:53.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:53.626+0000 7fdfbd000640 1 -- 172.21.15.45:0/1787170994 shutdown_connections 2024-04-16T17:03:53.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:53.626+0000 7fdfbd000640 1 --2- 172.21.15.45:0/1787170994 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdfb8105cf0 0x7fdfb810c580 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:53.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:53.626+0000 7fdfbd000640 1 --2- 172.21.15.45:0/1787170994 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdfb8105330 0x7fdfb81057b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:53.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:53.626+0000 7fdfbd000640 1 --2- 172.21.15.45:0/1787170994 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdfb8104140 0x7fdfb8104560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:53.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:53.626+0000 7fdfbd000640 1 -- 172.21.15.45:0/1787170994 >> 172.21.15.45:0/1787170994 conn(0x7fdfb80ff6b0 msgr2=0x7fdfb8101af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:03:53.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:53.626+0000 7fdfbd000640 1 -- 172.21.15.45:0/1787170994 shutdown_connections 2024-04-16T17:03:53.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:53.626+0000 7fdfbd000640 1 -- 172.21.15.45:0/1787170994 wait complete. 2024-04-16T17:03:53.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:53.627+0000 7fdfbd000640 1 Processor -- start 2024-04-16T17:03:53.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:53.627+0000 7fdfbd000640 1 -- start start 2024-04-16T17:03:53.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:53.627+0000 7fdfbd000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdfb8104140 0x7fdfb819a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:53.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:53.627+0000 7fdfbd000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdfb8105330 0x7fdfb819a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:53.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:53.627+0000 7fdfbd000640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdfb8105cf0 0x7fdfb81a1990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:53.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:53.627+0000 7fdfbd000640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fdfb810fe20 con 0x7fdfb8105330 2024-04-16T17:03:53.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:53.627+0000 7fdfbd000640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fdfb810fca0 con 0x7fdfb8105cf0 2024-04-16T17:03:53.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:53.627+0000 7fdfbd000640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fdfb810ffa0 con 0x7fdfb8104140 2024-04-16T17:03:53.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:53.628+0000 7fdfb7400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdfb8104140 0x7fdfb819a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:53.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:53.628+0000 7fdfb6a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdfb8105330 0x7fdfb819a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:53.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:53.628+0000 7fdfb7400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdfb8104140 0x7fdfb819a3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:55716/0 (socket says 172.21.15.45:55716) 2024-04-16T17:03:53.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:53.628+0000 7fdfb7400640 1 -- 172.21.15.45:0/3161464934 learned_addr learned my addr 172.21.15.45:0/3161464934 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:03:53.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:53.628+0000 7fdfb7e00640 1 --2- 172.21.15.45:0/3161464934 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdfb8105cf0 0x7fdfb81a1990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:53.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:53.628+0000 7fdfb7400640 1 -- 172.21.15.45:0/3161464934 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdfb8105cf0 msgr2=0x7fdfb81a1990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:53.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:53.628+0000 7fdfb7400640 1 --2- 172.21.15.45:0/3161464934 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdfb8105cf0 0x7fdfb81a1990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:53.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:53.628+0000 7fdfb7400640 1 -- 172.21.15.45:0/3161464934 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdfb8105330 msgr2=0x7fdfb819a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:53.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:53.628+0000 7fdfb7400640 1 --2- 172.21.15.45:0/3161464934 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdfb8105330 0x7fdfb819a920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:53.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:53.628+0000 7fdfb7400640 1 -- 172.21.15.45:0/3161464934 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfb81a2000 con 0x7fdfb8104140 2024-04-16T17:03:53.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:53.628+0000 7fdfb7e00640 1 --2- 172.21.15.45:0/3161464934 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdfb8105cf0 0x7fdfb81a1990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:03:53.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:53.628+0000 7fdfb7400640 1 --2- 172.21.15.45:0/3161464934 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdfb8104140 0x7fdfb819a3e0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fdfa800eb70 tx=0x7fdfa800c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:03:53.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:53.629+0000 7fdfafe00640 1 -- 172.21.15.45:0/3161464934 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfa8003b70 con 0x7fdfb8104140 2024-04-16T17:03:53.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:53.629+0000 7fdfafe00640 1 -- 172.21.15.45:0/3161464934 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdfa800ee70 con 0x7fdfb8104140 2024-04-16T17:03:53.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:53.629+0000 7fdfafe00640 1 -- 172.21.15.45:0/3161464934 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfa8010830 con 0x7fdfb8104140 2024-04-16T17:03:53.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:53.629+0000 7fdfbd000640 1 -- 172.21.15.45:0/3161464934 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdfb81a22f0 con 0x7fdfb8104140 2024-04-16T17:03:53.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:53.629+0000 7fdfbd000640 1 -- 172.21.15.45:0/3161464934 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdfb81a27f0 con 0x7fdfb8104140 2024-04-16T17:03:53.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:53.630+0000 7fdfbd000640 1 -- 172.21.15.45:0/3161464934 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdfb810b390 con 0x7fdfb8104140 2024-04-16T17:03:53.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:53.632+0000 7fdfafe00640 1 -- 172.21.15.45:0/3161464934 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fdfa8004020 con 0x7fdfb8104140 2024-04-16T17:03:53.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:53.634+0000 7fdfafe00640 1 --2- 172.21.15.45:0/3161464934 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fdf9c0a3c30 0x7fdf9c0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:53.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:53.634+0000 7fdfafe00640 1 -- 172.21.15.45:0/3161464934 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7fdfa80d0d50 con 0x7fdfb8104140 2024-04-16T17:03:53.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:53.634+0000 7fdfb6a00640 1 --2- 172.21.15.45:0/3161464934 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fdf9c0a3c30 0x7fdf9c0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:53.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:53.635+0000 7fdfb6a00640 1 --2- 172.21.15.45:0/3161464934 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fdf9c0a3c30 0x7fdf9c0a60f0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fdfb819b8f0 tx=0x7fdfa0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:03:53.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:53.638+0000 7fdfafe00640 1 -- 172.21.15.45:0/3161464934 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fdfa809d490 con 0x7fdfb8104140 2024-04-16T17:03:53.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:03:53 smithi104 ceph-mon[36457]: pgmap v336: 129 pgs: 129 active+clean; 581 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s wr, 0 op/s 2024-04-16T17:03:53.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:03:53 smithi104 ceph-mon[36457]: from='client.? 172.21.15.45:0/909731725' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-16T17:03:53.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:03:53 smithi104 ceph-mon[36457]: mds.? [v2:172.21.15.45:6840/3755310822,v1:172.21.15.45:6841/3755310822] up:active 2024-04-16T17:03:53.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:03:53 smithi104 ceph-mon[36457]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-04-16T17:03:53.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:03:53 smithi104 ceph-mon[36457]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-16T17:03:53.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:03:53 smithi104 ceph-mon[36457]: Cluster is now healthy 2024-04-16T17:03:53.679 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:03:53 smithi104 ceph-mon[36457]: fsmap cephfs:2 {0=b=up:active,1=a=up:active} 10 up:standby 2024-04-16T17:03:53.679 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:03:53 smithi104 ceph-mon[36457]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:creating} 9 up:standby 2024-04-16T17:03:53.679 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:03:53 smithi104 ceph-mon[36457]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-04-16T17:03:53.679 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:03:53 smithi104 ceph-mon[36457]: from='client.? 172.21.15.45:0/1243102437' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-16T17:03:53.679 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:03:53 smithi104 ceph-mon[36457]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-16T17:03:53.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:03:53 smithi045 ceph-mon[31676]: pgmap v336: 129 pgs: 129 active+clean; 581 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s wr, 0 op/s 2024-04-16T17:03:53.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:03:53 smithi045 ceph-mon[31676]: from='client.? 172.21.15.45:0/909731725' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-16T17:03:53.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:03:53 smithi045 ceph-mon[31676]: mds.? [v2:172.21.15.45:6840/3755310822,v1:172.21.15.45:6841/3755310822] up:active 2024-04-16T17:03:53.729 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:03:53 smithi045 ceph-mon[31676]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-04-16T17:03:53.729 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:03:53 smithi045 ceph-mon[31676]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-16T17:03:53.729 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:03:53 smithi045 ceph-mon[31676]: Cluster is now healthy 2024-04-16T17:03:53.729 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:03:53 smithi045 ceph-mon[31676]: fsmap cephfs:2 {0=b=up:active,1=a=up:active} 10 up:standby 2024-04-16T17:03:53.729 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:03:53 smithi045 ceph-mon[31676]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:creating} 9 up:standby 2024-04-16T17:03:53.729 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:03:53 smithi045 ceph-mon[31676]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-04-16T17:03:53.729 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:03:53 smithi045 ceph-mon[31676]: from='client.? 172.21.15.45:0/1243102437' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-16T17:03:53.729 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:03:53 smithi045 ceph-mon[31676]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-16T17:03:53.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:03:53 smithi117 ceph-mon[36730]: pgmap v336: 129 pgs: 129 active+clean; 581 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s wr, 0 op/s 2024-04-16T17:03:53.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:03:53 smithi117 ceph-mon[36730]: from='client.? 172.21.15.45:0/909731725' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-16T17:03:53.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:03:53 smithi117 ceph-mon[36730]: mds.? [v2:172.21.15.45:6840/3755310822,v1:172.21.15.45:6841/3755310822] up:active 2024-04-16T17:03:53.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:03:53 smithi117 ceph-mon[36730]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-04-16T17:03:53.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:03:53 smithi117 ceph-mon[36730]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-16T17:03:53.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:03:53 smithi117 ceph-mon[36730]: Cluster is now healthy 2024-04-16T17:03:53.807 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:03:53 smithi117 ceph-mon[36730]: fsmap cephfs:2 {0=b=up:active,1=a=up:active} 10 up:standby 2024-04-16T17:03:53.807 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:03:53 smithi117 ceph-mon[36730]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:creating} 9 up:standby 2024-04-16T17:03:53.807 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:03:53 smithi117 ceph-mon[36730]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-04-16T17:03:53.807 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:03:53 smithi117 ceph-mon[36730]: from='client.? 172.21.15.45:0/1243102437' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-16T17:03:53.807 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:03:53 smithi117 ceph-mon[36730]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-16T17:03:53.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:53.872+0000 7fdfbd000640 1 -- 172.21.15.45:0/3161464934 --> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7fdfb8062440 con 0x7fdf9c0a3c30 2024-04-16T17:03:53.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:53.909+0000 7fdfafe00640 1 -- 172.21.15.45:0/3161464934 <== mgr.14152 v2:172.21.15.45:6800/268030699 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fdfb8062440 con 0x7fdf9c0a3c30 2024-04-16T17:03:53.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:53.911+0000 7fdfbd000640 1 -- 172.21.15.45:0/3161464934 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fdf9c0a3c30 msgr2=0x7fdf9c0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:53.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:53.911+0000 7fdfbd000640 1 --2- 172.21.15.45:0/3161464934 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fdf9c0a3c30 0x7fdf9c0a60f0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fdfb819b8f0 tx=0x7fdfa0005f90 comp rx=0 tx=0).stop 2024-04-16T17:03:53.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:53.911+0000 7fdfbd000640 1 -- 172.21.15.45:0/3161464934 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdfb8104140 msgr2=0x7fdfb819a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:53.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:53.911+0000 7fdfbd000640 1 --2- 172.21.15.45:0/3161464934 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdfb8104140 0x7fdfb819a3e0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fdfa800eb70 tx=0x7fdfa800c6a0 comp rx=0 tx=0).stop 2024-04-16T17:03:53.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:53.911+0000 7fdfbd000640 1 -- 172.21.15.45:0/3161464934 shutdown_connections 2024-04-16T17:03:53.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:53.911+0000 7fdfbd000640 1 --2- 172.21.15.45:0/3161464934 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fdf9c0a3c30 0x7fdf9c0a60f0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:53.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:53.911+0000 7fdfbd000640 1 --2- 172.21.15.45:0/3161464934 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdfb8105cf0 0x7fdfb81a1990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:53.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:53.911+0000 7fdfbd000640 1 --2- 172.21.15.45:0/3161464934 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdfb8105330 0x7fdfb819a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:53.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:53.911+0000 7fdfbd000640 1 --2- 172.21.15.45:0/3161464934 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdfb8104140 0x7fdfb819a3e0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:53.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:53.911+0000 7fdfbd000640 1 -- 172.21.15.45:0/3161464934 >> 172.21.15.45:0/3161464934 conn(0x7fdfb80ff6b0 msgr2=0x7fdfb8108560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:03:53.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:53.911+0000 7fdfbd000640 1 -- 172.21.15.45:0/3161464934 shutdown_connections 2024-04-16T17:03:53.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:53.912+0000 7fdfbd000640 1 -- 172.21.15.45:0/3161464934 wait complete. 2024-04-16T17:03:53.934 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_0 --group_name qa 2024-04-16T17:03:54.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:54.145+0000 7f76cea00640 1 -- 172.21.15.45:0/3137227211 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f76c8104140 msgr2=0x7f76c8104560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:54.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:54.145+0000 7f76cea00640 1 --2- 172.21.15.45:0/3137227211 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f76c8104140 0x7f76c8104560 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f76bc009900 tx=0x7f76bc02bc30 comp rx=0 tx=0).stop 2024-04-16T17:03:54.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:54.145+0000 7f76cea00640 1 -- 172.21.15.45:0/3137227211 shutdown_connections 2024-04-16T17:03:54.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:54.145+0000 7f76cea00640 1 --2- 172.21.15.45:0/3137227211 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f76c8105330 0x7f76c81057b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:54.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:54.145+0000 7f76cea00640 1 --2- 172.21.15.45:0/3137227211 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f76c8105cf0 0x7f76c810c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:54.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:54.145+0000 7f76cea00640 1 --2- 172.21.15.45:0/3137227211 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f76c8104140 0x7f76c8104560 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:54.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:54.145+0000 7f76cea00640 1 -- 172.21.15.45:0/3137227211 >> 172.21.15.45:0/3137227211 conn(0x7f76c80ff6b0 msgr2=0x7f76c8101af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:03:54.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:54.145+0000 7f76cea00640 1 -- 172.21.15.45:0/3137227211 shutdown_connections 2024-04-16T17:03:54.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:54.145+0000 7f76cea00640 1 -- 172.21.15.45:0/3137227211 wait complete. 2024-04-16T17:03:54.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:54.146+0000 7f76cea00640 1 Processor -- start 2024-04-16T17:03:54.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:54.147+0000 7f76cea00640 1 -- start start 2024-04-16T17:03:54.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:54.147+0000 7f76cea00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f76c8104140 0x7f76c819a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:54.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:54.147+0000 7f76cea00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f76c8105330 0x7f76c819a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:54.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:54.147+0000 7f76cea00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f76c8105cf0 0x7f76c81a19f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:54.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:54.147+0000 7f76cea00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f76c810fe20 con 0x7f76c8105cf0 2024-04-16T17:03:54.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:54.147+0000 7f76cea00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f76c810fca0 con 0x7f76c8104140 2024-04-16T17:03:54.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:54.147+0000 7f76cea00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f76c810ffa0 con 0x7f76c8105330 2024-04-16T17:03:54.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:54.148+0000 7f76cd600640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f76c8104140 0x7f76c819a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:54.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:54.148+0000 7f76cd600640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f76c8104140 0x7f76c819a490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:34326/0 (socket says 172.21.15.45:34326) 2024-04-16T17:03:54.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:54.148+0000 7f76cd600640 1 -- 172.21.15.45:0/2743288458 learned_addr learned my addr 172.21.15.45:0/2743288458 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:03:54.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:54.148+0000 7f76ce000640 1 --2- 172.21.15.45:0/2743288458 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f76c8105cf0 0x7f76c81a19f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:54.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:54.148+0000 7f76ccc00640 1 --2- 172.21.15.45:0/2743288458 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f76c8105330 0x7f76c819a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:54.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:54.148+0000 7f76cd600640 1 -- 172.21.15.45:0/2743288458 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f76c8105330 msgr2=0x7f76c819a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:54.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:54.148+0000 7f76cd600640 1 --2- 172.21.15.45:0/2743288458 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f76c8105330 0x7f76c819a9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:54.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:54.148+0000 7f76cd600640 1 -- 172.21.15.45:0/2743288458 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f76c8105cf0 msgr2=0x7f76c81a19f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:54.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:54.148+0000 7f76cd600640 1 --2- 172.21.15.45:0/2743288458 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f76c8105cf0 0x7f76c81a19f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:54.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:54.148+0000 7f76cd600640 1 -- 172.21.15.45:0/2743288458 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76c81a2060 con 0x7f76c8104140 2024-04-16T17:03:54.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:54.148+0000 7f76cd600640 1 --2- 172.21.15.45:0/2743288458 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f76c8104140 0x7f76c819a490 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f76bc000c00 tx=0x7f76bc02c6c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:03:54.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:54.149+0000 7f76c6000640 1 -- 172.21.15.45:0/2743288458 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76bc043020 con 0x7f76c8104140 2024-04-16T17:03:54.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:54.149+0000 7f76c6000640 1 -- 172.21.15.45:0/2743288458 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f76bc0054e0 con 0x7f76c8104140 2024-04-16T17:03:54.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:54.149+0000 7f76c6000640 1 -- 172.21.15.45:0/2743288458 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76bc003ce0 con 0x7f76c8104140 2024-04-16T17:03:54.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:54.149+0000 7f76cea00640 1 -- 172.21.15.45:0/2743288458 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76c81a22f0 con 0x7f76c8104140 2024-04-16T17:03:54.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:54.149+0000 7f76cea00640 1 -- 172.21.15.45:0/2743288458 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76c81a27c0 con 0x7f76c8104140 2024-04-16T17:03:54.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:54.150+0000 7f76cea00640 1 -- 172.21.15.45:0/2743288458 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76c810b530 con 0x7f76c8104140 2024-04-16T17:03:54.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:54.152+0000 7f76c6000640 1 -- 172.21.15.45:0/2743288458 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f76bc046020 con 0x7f76c8104140 2024-04-16T17:03:54.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:54.153+0000 7f76c6000640 1 --2- 172.21.15.45:0/2743288458 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f76b00a3c30 0x7f76b00a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:54.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:54.153+0000 7f76c6000640 1 -- 172.21.15.45:0/2743288458 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f76bc0f26b0 con 0x7f76c8104140 2024-04-16T17:03:54.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:54.154+0000 7f76ccc00640 1 --2- 172.21.15.45:0/2743288458 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f76b00a3c30 0x7f76b00a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:54.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:54.154+0000 7f76ccc00640 1 --2- 172.21.15.45:0/2743288458 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f76b00a3c30 0x7f76b00a60f0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f76c819b950 tx=0x7f76b400b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:03:54.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:54.157+0000 7f76c6000640 1 -- 172.21.15.45:0/2743288458 <== mon.2 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f76bc04d020 con 0x7f76c8104140 2024-04-16T17:03:54.178 INFO:journalctl@ceph.mds.b.smithi104.stdout:Apr 16 17:03:53 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mds-b[67798]: 2024-04-16T17:03:53.883+0000 7fde231b9700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-04-16T17:03:54.178 INFO:journalctl@ceph.mds.b.smithi104.stdout:Apr 16 17:03:53 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mds-b[67798]: 2024-04-16T17:03:53.883+0000 7fde231b9700 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-04-16T17:03:54.229 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 16 17:03:53 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-x[31905]: 2024-04-16T17:03:53.884+0000 7f1a2dc00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-16T17:03:54.229 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 16 17:03:53 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-x[31905]: 2024-04-16T17:03:53.884+0000 7f1a2dc00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-16T17:03:54.229 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 16 17:03:53 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-x[31905]: 2024-04-16T17:03:53.884+0000 7f1a2dc00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-16T17:03:54.229 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 16 17:03:53 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-x[31905]: 2024-04-16T17:03:53.884+0000 7f1a2dc00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-16T17:03:54.229 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 16 17:03:53 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-x[31905]: 2024-04-16T17:03:53.884+0000 7f1a2dc00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-16T17:03:54.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:54.401+0000 7f76cea00640 1 -- 172.21.15.45:0/2743288458 --> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f76c8062440 con 0x7f76b00a3c30 2024-04-16T17:03:54.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:54.409+0000 7f76c6000640 1 -- 172.21.15.45:0/2743288458 <== mon.2 v2:172.21.15.104:3300/0 7 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f76bc0bad60 con 0x7f76c8104140 2024-04-16T17:03:54.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:54.449+0000 7f76c6000640 1 -- 172.21.15.45:0/2743288458 <== mgr.14152 v2:172.21.15.45:6800/268030699 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f76c8062440 con 0x7f76b00a3c30 2024-04-16T17:03:54.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:54.452+0000 7f76cea00640 1 -- 172.21.15.45:0/2743288458 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f76b00a3c30 msgr2=0x7f76b00a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:54.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:54.452+0000 7f76cea00640 1 --2- 172.21.15.45:0/2743288458 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f76b00a3c30 0x7f76b00a60f0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f76c819b950 tx=0x7f76b400b390 comp rx=0 tx=0).stop 2024-04-16T17:03:54.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:54.452+0000 7f76cea00640 1 -- 172.21.15.45:0/2743288458 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f76c8104140 msgr2=0x7f76c819a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:54.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:54.452+0000 7f76cea00640 1 --2- 172.21.15.45:0/2743288458 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f76c8104140 0x7f76c819a490 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f76bc000c00 tx=0x7f76bc02c6c0 comp rx=0 tx=0).stop 2024-04-16T17:03:54.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:54.452+0000 7f76cea00640 1 -- 172.21.15.45:0/2743288458 shutdown_connections 2024-04-16T17:03:54.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:54.452+0000 7f76cea00640 1 --2- 172.21.15.45:0/2743288458 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f76b00a3c30 0x7f76b00a60f0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:54.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:54.452+0000 7f76cea00640 1 --2- 172.21.15.45:0/2743288458 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f76c8105330 0x7f76c819a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:54.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:54.452+0000 7f76cea00640 1 --2- 172.21.15.45:0/2743288458 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f76c8105cf0 0x7f76c81a19f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:54.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:54.452+0000 7f76cea00640 1 --2- 172.21.15.45:0/2743288458 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f76c8104140 0x7f76c819a490 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:54.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:54.452+0000 7f76cea00640 1 -- 172.21.15.45:0/2743288458 >> 172.21.15.45:0/2743288458 conn(0x7f76c80ff6b0 msgr2=0x7f76c8108700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:03:54.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:54.452+0000 7f76cea00640 1 -- 172.21.15.45:0/2743288458 shutdown_connections 2024-04-16T17:03:54.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:54.452+0000 7f76cea00640 1 -- 172.21.15.45:0/2743288458 wait complete. 2024-04-16T17:03:54.474 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-04-16T17:03:54.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:03:54 smithi104 ceph-mon[36457]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-16T17:03:54.679 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:03:54 smithi104 ceph-mon[36457]: mds.? [v2:172.21.15.117:6838/2378675949,v1:172.21.15.117:6839/2378675949] up:active 2024-04-16T17:03:54.679 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:03:54 smithi104 ceph-mon[36457]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 9 up:standby 2024-04-16T17:03:54.679 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:03:54 smithi104 ceph-mon[36457]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-16T17:03:54.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:54.682+0000 7fededa00640 1 -- 172.21.15.45:0/2060514935 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fede8103080 msgr2=0x7fede8103500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:54.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:54.682+0000 7fededa00640 1 --2- 172.21.15.45:0/2060514935 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fede8103080 0x7fede8103500 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fedd4009900 tx=0x7fedd402bc40 comp rx=0 tx=0).stop 2024-04-16T17:03:54.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:54.682+0000 7fededa00640 1 -- 172.21.15.45:0/2060514935 shutdown_connections 2024-04-16T17:03:54.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:54.682+0000 7fededa00640 1 --2- 172.21.15.45:0/2060514935 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fede8103a40 0x7fede810a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:54.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:54.682+0000 7fededa00640 1 --2- 172.21.15.45:0/2060514935 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fede8103080 0x7fede8103500 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:54.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:54.682+0000 7fededa00640 1 --2- 172.21.15.45:0/2060514935 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fede8101f30 0x7fede8102350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:54.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:54.682+0000 7fededa00640 1 -- 172.21.15.45:0/2060514935 >> 172.21.15.45:0/2060514935 conn(0x7fede80fd4a0 msgr2=0x7fede80ff8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:03:54.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:54.683+0000 7fededa00640 1 -- 172.21.15.45:0/2060514935 shutdown_connections 2024-04-16T17:03:54.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:54.683+0000 7fededa00640 1 -- 172.21.15.45:0/2060514935 wait complete. 2024-04-16T17:03:54.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:54.683+0000 7fededa00640 1 Processor -- start 2024-04-16T17:03:54.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:54.684+0000 7fededa00640 1 -- start start 2024-04-16T17:03:54.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:54.684+0000 7fededa00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fede8101f30 0x7fede819a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:54.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:54.684+0000 7fededa00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fede8103080 0x7fede819a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:54.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:54.684+0000 7fededa00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fede8103a40 0x7fede81a1a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:54.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:54.684+0000 7fededa00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fede810dc10 con 0x7fede8103080 2024-04-16T17:03:54.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:54.684+0000 7fededa00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fede810da90 con 0x7fede8101f30 2024-04-16T17:03:54.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:54.684+0000 7fededa00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fede810dd90 con 0x7fede8103a40 2024-04-16T17:03:54.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:54.684+0000 7fede7e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fede8101f30 0x7fede819a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:54.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:54.684+0000 7fede7e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fede8101f30 0x7fede819a450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:34338/0 (socket says 172.21.15.45:34338) 2024-04-16T17:03:54.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:54.684+0000 7fede7e00640 1 -- 172.21.15.45:0/2274433644 learned_addr learned my addr 172.21.15.45:0/2274433644 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:03:54.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:54.684+0000 7feded000640 1 --2- 172.21.15.45:0/2274433644 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fede8103a40 0x7fede81a1a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:54.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:54.685+0000 7fede7400640 1 --2- 172.21.15.45:0/2274433644 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fede8103080 0x7fede819a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:54.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:54.685+0000 7fede7e00640 1 -- 172.21.15.45:0/2274433644 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fede8103a40 msgr2=0x7fede81a1a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:54.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:54.685+0000 7fede7e00640 1 --2- 172.21.15.45:0/2274433644 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fede8103a40 0x7fede81a1a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:54.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:54.685+0000 7fede7e00640 1 -- 172.21.15.45:0/2274433644 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fede8103080 msgr2=0x7fede819a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:54.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:54.685+0000 7fede7e00640 1 --2- 172.21.15.45:0/2274433644 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fede8103080 0x7fede819a990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:54.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:54.685+0000 7fede7e00640 1 -- 172.21.15.45:0/2274433644 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fede81a2070 con 0x7fede8101f30 2024-04-16T17:03:54.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:54.685+0000 7fede7e00640 1 --2- 172.21.15.45:0/2274433644 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fede8101f30 0x7fede819a450 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fedd800d950 tx=0x7fedd800de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:03:54.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:54.685+0000 7fede4c00640 1 -- 172.21.15.45:0/2274433644 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedd8017030 con 0x7fede8101f30 2024-04-16T17:03:54.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:54.686+0000 7fede4c00640 1 -- 172.21.15.45:0/2274433644 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fedd8004020 con 0x7fede8101f30 2024-04-16T17:03:54.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:54.686+0000 7fede4c00640 1 -- 172.21.15.45:0/2274433644 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedd80047a0 con 0x7fede8101f30 2024-04-16T17:03:54.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:54.686+0000 7fededa00640 1 -- 172.21.15.45:0/2274433644 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fede81a2360 con 0x7fede8101f30 2024-04-16T17:03:54.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:54.686+0000 7fededa00640 1 -- 172.21.15.45:0/2274433644 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fede81a2830 con 0x7fede8101f30 2024-04-16T17:03:54.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:54.687+0000 7fededa00640 1 -- 172.21.15.45:0/2274433644 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fede81a2af0 con 0x7fede8101f30 2024-04-16T17:03:54.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:54.689+0000 7fede4c00640 1 -- 172.21.15.45:0/2274433644 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fedd80041d0 con 0x7fede8101f30 2024-04-16T17:03:54.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:54.690+0000 7fede4c00640 1 --2- 172.21.15.45:0/2274433644 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fedcc0a3cc0 0x7fedcc0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:54.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:54.690+0000 7fede4c00640 1 -- 172.21.15.45:0/2274433644 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7fedd80d0240 con 0x7fede8101f30 2024-04-16T17:03:54.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:54.690+0000 7fede7400640 1 --2- 172.21.15.45:0/2274433644 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fedcc0a3cc0 0x7fedcc0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:54.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:54.691+0000 7fede7400640 1 --2- 172.21.15.45:0/2274433644 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fedcc0a3cc0 0x7fedcc0a6180 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fedd402c180 tx=0x7fedd4000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:03:54.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:54.695+0000 7fede4c00640 1 -- 172.21.15.45:0/2274433644 <== mon.2 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fedd809c820 con 0x7fede8101f30 2024-04-16T17:03:54.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:03:54 smithi045 ceph-mon[31676]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-16T17:03:54.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:03:54 smithi045 ceph-mon[31676]: mds.? [v2:172.21.15.117:6838/2378675949,v1:172.21.15.117:6839/2378675949] up:active 2024-04-16T17:03:54.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:03:54 smithi045 ceph-mon[31676]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 9 up:standby 2024-04-16T17:03:54.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:03:54 smithi045 ceph-mon[31676]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-16T17:03:54.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:03:54 smithi117 ceph-mon[36730]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-16T17:03:54.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:03:54 smithi117 ceph-mon[36730]: mds.? [v2:172.21.15.117:6838/2378675949,v1:172.21.15.117:6839/2378675949] up:active 2024-04-16T17:03:54.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:03:54 smithi117 ceph-mon[36730]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 9 up:standby 2024-04-16T17:03:54.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:03:54 smithi117 ceph-mon[36730]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-16T17:03:54.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:54.926+0000 7fededa00640 1 -- 172.21.15.45:0/2274433644 --> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7fede8075950 con 0x7fedcc0a3cc0 2024-04-16T17:03:54.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:54.938+0000 7fede4c00640 1 -- 172.21.15.45:0/2274433644 <== mgr.14152 v2:172.21.15.45:6800/268030699 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7fede8075950 con 0x7fedcc0a3cc0 2024-04-16T17:03:54.941 INFO:teuthology.orchestra.run.smithi045.stdout:/volumes/qa/sv_0/08dbda30-b0b4-4c24-b79e-74f280589b6c 2024-04-16T17:03:54.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:54.941+0000 7fededa00640 1 -- 172.21.15.45:0/2274433644 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fedcc0a3cc0 msgr2=0x7fedcc0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:54.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:54.941+0000 7fededa00640 1 --2- 172.21.15.45:0/2274433644 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fedcc0a3cc0 0x7fedcc0a6180 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fedd402c180 tx=0x7fedd4000bc0 comp rx=0 tx=0).stop 2024-04-16T17:03:54.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:54.941+0000 7fededa00640 1 -- 172.21.15.45:0/2274433644 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fede8101f30 msgr2=0x7fede819a450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:54.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:54.941+0000 7fededa00640 1 --2- 172.21.15.45:0/2274433644 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fede8101f30 0x7fede819a450 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fedd800d950 tx=0x7fedd800de10 comp rx=0 tx=0).stop 2024-04-16T17:03:54.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:54.941+0000 7fededa00640 1 -- 172.21.15.45:0/2274433644 shutdown_connections 2024-04-16T17:03:54.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:54.941+0000 7fededa00640 1 --2- 172.21.15.45:0/2274433644 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fedcc0a3cc0 0x7fedcc0a6180 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:54.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:54.941+0000 7fededa00640 1 --2- 172.21.15.45:0/2274433644 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fede8103a40 0x7fede81a1a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:54.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:54.941+0000 7fededa00640 1 --2- 172.21.15.45:0/2274433644 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fede8103080 0x7fede819a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:54.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:54.941+0000 7fededa00640 1 --2- 172.21.15.45:0/2274433644 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fede8101f30 0x7fede819a450 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:54.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:54.941+0000 7fededa00640 1 -- 172.21.15.45:0/2274433644 >> 172.21.15.45:0/2274433644 conn(0x7fede80fd4a0 msgr2=0x7fede8074230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:03:54.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:54.941+0000 7fededa00640 1 -- 172.21.15.45:0/2274433644 shutdown_connections 2024-04-16T17:03:54.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:54.941+0000 7fededa00640 1 -- 172.21.15.45:0/2274433644 wait complete. 2024-04-16T17:03:54.960 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_1 --group_name qa 2024-04-16T17:03:55.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:55.177+0000 7fadeae00640 1 -- 172.21.15.45:0/131400291 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fade4101f30 msgr2=0x7fade4102350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:55.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:55.177+0000 7fadeae00640 1 --2- 172.21.15.45:0/131400291 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fade4101f30 0x7fade4102350 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fadd8009900 tx=0x7fadd802bc30 comp rx=0 tx=0).stop 2024-04-16T17:03:55.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:55.177+0000 7fadeae00640 1 -- 172.21.15.45:0/131400291 shutdown_connections 2024-04-16T17:03:55.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:55.177+0000 7fadeae00640 1 --2- 172.21.15.45:0/131400291 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fade4103a40 0x7fade410a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:55.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:55.177+0000 7fadeae00640 1 --2- 172.21.15.45:0/131400291 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fade4103080 0x7fade4103500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:55.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:55.177+0000 7fadeae00640 1 --2- 172.21.15.45:0/131400291 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fade4101f30 0x7fade4102350 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:55.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:55.177+0000 7fadeae00640 1 -- 172.21.15.45:0/131400291 >> 172.21.15.45:0/131400291 conn(0x7fade40fd4a0 msgr2=0x7fade40ff8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:03:55.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:55.177+0000 7fadeae00640 1 -- 172.21.15.45:0/131400291 shutdown_connections 2024-04-16T17:03:55.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:55.177+0000 7fadeae00640 1 -- 172.21.15.45:0/131400291 wait complete. 2024-04-16T17:03:55.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:55.178+0000 7fadeae00640 1 Processor -- start 2024-04-16T17:03:55.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:55.178+0000 7fadeae00640 1 -- start start 2024-04-16T17:03:55.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:55.179+0000 7fadeae00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fade4101f30 0x7fade4198220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:55.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:55.179+0000 7fadeae00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fade4103080 0x7fade4198760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:55.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:55.179+0000 7fadeae00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fade4103a40 0x7fade419f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:55.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:55.179+0000 7fadeae00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fade410dc10 con 0x7fade4103a40 2024-04-16T17:03:55.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:55.179+0000 7fadeae00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fade410da90 con 0x7fade4103080 2024-04-16T17:03:55.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:55.179+0000 7fadeae00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fade410dd90 con 0x7fade4101f30 2024-04-16T17:03:55.184 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:55.179+0000 7fade9000640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fade4103080 0x7fade4198760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:55.184 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:55.179+0000 7fade9000640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fade4103080 0x7fade4198760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:34358/0 (socket says 172.21.15.45:34358) 2024-04-16T17:03:55.184 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:55.179+0000 7fade9000640 1 -- 172.21.15.45:0/2995110340 learned_addr learned my addr 172.21.15.45:0/2995110340 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:03:55.184 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:55.179+0000 7fadea400640 1 --2- 172.21.15.45:0/2995110340 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fade4103a40 0x7fade419f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:55.184 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:55.179+0000 7fade9a00640 1 --2- 172.21.15.45:0/2995110340 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fade4101f30 0x7fade4198220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:55.184 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:55.180+0000 7fade9000640 1 -- 172.21.15.45:0/2995110340 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fade4101f30 msgr2=0x7fade4198220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:55.184 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:55.180+0000 7fade9000640 1 --2- 172.21.15.45:0/2995110340 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fade4101f30 0x7fade4198220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:55.184 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:55.180+0000 7fade9000640 1 -- 172.21.15.45:0/2995110340 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fade4103a40 msgr2=0x7fade419f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:55.184 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:55.180+0000 7fade9000640 1 --2- 172.21.15.45:0/2995110340 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fade4103a40 0x7fade419f7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:55.184 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:55.180+0000 7fade9000640 1 -- 172.21.15.45:0/2995110340 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fade419fe40 con 0x7fade4103080 2024-04-16T17:03:55.184 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:55.180+0000 7fade9000640 1 --2- 172.21.15.45:0/2995110340 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fade4103080 0x7fade4198760 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fadd000e9c0 tx=0x7fadd000ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:03:55.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:55.180+0000 7fade2000640 1 -- 172.21.15.45:0/2995110340 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadd000cd80 con 0x7fade4103080 2024-04-16T17:03:55.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:55.180+0000 7fade2000640 1 -- 172.21.15.45:0/2995110340 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fadd00035f0 con 0x7fade4103080 2024-04-16T17:03:55.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:55.180+0000 7fadeae00640 1 -- 172.21.15.45:0/2995110340 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fade41a0130 con 0x7fade4103080 2024-04-16T17:03:55.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:55.180+0000 7fade2000640 1 -- 172.21.15.45:0/2995110340 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadd0010690 con 0x7fade4103080 2024-04-16T17:03:55.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:55.180+0000 7fadeae00640 1 -- 172.21.15.45:0/2995110340 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fade41a06d0 con 0x7fade4103080 2024-04-16T17:03:55.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:55.182+0000 7fadeae00640 1 -- 172.21.15.45:0/2995110340 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fade41092c0 con 0x7fade4103080 2024-04-16T17:03:55.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:55.185+0000 7fade2000640 1 -- 172.21.15.45:0/2995110340 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fadd00037a0 con 0x7fade4103080 2024-04-16T17:03:55.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:55.186+0000 7fade2000640 1 --2- 172.21.15.45:0/2995110340 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fadcc0a3cc0 0x7fadcc0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:55.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:55.186+0000 7fade2000640 1 -- 172.21.15.45:0/2995110340 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7fadd00d0d40 con 0x7fade4103080 2024-04-16T17:03:55.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:55.186+0000 7fade9a00640 1 --2- 172.21.15.45:0/2995110340 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fadcc0a3cc0 0x7fadcc0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:55.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:55.187+0000 7fade9a00640 1 --2- 172.21.15.45:0/2995110340 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fadcc0a3cc0 0x7fadcc0a6180 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fadd802c240 tx=0x7fadd802df90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:03:55.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:55.187+0000 7fade2000640 1 -- 172.21.15.45:0/2995110340 <== mon.2 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fadd009d320 con 0x7fade4103080 2024-04-16T17:03:55.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:55.431+0000 7fadeae00640 1 -- 172.21.15.45:0/2995110340 --> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7fade4062440 con 0x7fadcc0a3cc0 2024-04-16T17:03:55.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:55.465+0000 7fade2000640 1 -- 172.21.15.45:0/2995110340 <== mgr.14152 v2:172.21.15.45:6800/268030699 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fade4062440 con 0x7fadcc0a3cc0 2024-04-16T17:03:55.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:55.468+0000 7fadeae00640 1 -- 172.21.15.45:0/2995110340 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fadcc0a3cc0 msgr2=0x7fadcc0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:55.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:55.468+0000 7fadeae00640 1 --2- 172.21.15.45:0/2995110340 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fadcc0a3cc0 0x7fadcc0a6180 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fadd802c240 tx=0x7fadd802df90 comp rx=0 tx=0).stop 2024-04-16T17:03:55.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:55.468+0000 7fadeae00640 1 -- 172.21.15.45:0/2995110340 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fade4103080 msgr2=0x7fade4198760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:55.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:55.468+0000 7fadeae00640 1 --2- 172.21.15.45:0/2995110340 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fade4103080 0x7fade4198760 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fadd000e9c0 tx=0x7fadd000ee80 comp rx=0 tx=0).stop 2024-04-16T17:03:55.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:55.468+0000 7fadeae00640 1 -- 172.21.15.45:0/2995110340 shutdown_connections 2024-04-16T17:03:55.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:55.468+0000 7fadeae00640 1 --2- 172.21.15.45:0/2995110340 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fadcc0a3cc0 0x7fadcc0a6180 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:55.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:55.468+0000 7fadeae00640 1 --2- 172.21.15.45:0/2995110340 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fade4103a40 0x7fade419f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:55.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:55.468+0000 7fadeae00640 1 --2- 172.21.15.45:0/2995110340 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fade4103080 0x7fade4198760 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:55.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:55.468+0000 7fadeae00640 1 --2- 172.21.15.45:0/2995110340 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fade4101f30 0x7fade4198220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:55.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:55.468+0000 7fadeae00640 1 -- 172.21.15.45:0/2995110340 >> 172.21.15.45:0/2995110340 conn(0x7fade40fd4a0 msgr2=0x7fade4106490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:03:55.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:55.469+0000 7fadeae00640 1 -- 172.21.15.45:0/2995110340 shutdown_connections 2024-04-16T17:03:55.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:55.469+0000 7fadeae00640 1 -- 172.21.15.45:0/2995110340 wait complete. 2024-04-16T17:03:55.487 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:03:55 smithi045 ceph-mon[31676]: from='client.24817 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T17:03:55.487 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:03:55 smithi045 ceph-mon[31676]: pgmap v337: 129 pgs: 129 active+clean; 581 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s wr, 2 op/s 2024-04-16T17:03:55.487 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:03:55 smithi045 ceph-mon[31676]: from='client.24833 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T17:03:55.488 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:03:55 smithi045 ceph-mon[31676]: mgrmap e17: x(active, since 8m), standbys: z, y 2024-04-16T17:03:55.491 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-04-16T17:03:55.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:03:55 smithi104 ceph-mon[36457]: from='client.24817 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T17:03:55.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:03:55 smithi104 ceph-mon[36457]: pgmap v337: 129 pgs: 129 active+clean; 581 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s wr, 2 op/s 2024-04-16T17:03:55.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:03:55 smithi104 ceph-mon[36457]: from='client.24833 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T17:03:55.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:03:55 smithi104 ceph-mon[36457]: mgrmap e17: x(active, since 8m), standbys: z, y 2024-04-16T17:03:55.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:55.709+0000 7fe905600640 1 -- 172.21.15.45:0/4109729979 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe900103c00 msgr2=0x7fe90010a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:55.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:55.709+0000 7fe905600640 1 --2- 172.21.15.45:0/4109729979 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe900103c00 0x7fe90010a490 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fe8f8009900 tx=0x7fe8f802bc30 comp rx=0 tx=0).stop 2024-04-16T17:03:55.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:55.709+0000 7fe905600640 1 -- 172.21.15.45:0/4109729979 shutdown_connections 2024-04-16T17:03:55.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:55.709+0000 7fe905600640 1 --2- 172.21.15.45:0/4109729979 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe900103c00 0x7fe90010a490 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:55.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:55.709+0000 7fe905600640 1 --2- 172.21.15.45:0/4109729979 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe900103240 0x7fe9001036c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:55.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:55.709+0000 7fe905600640 1 --2- 172.21.15.45:0/4109729979 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe900102050 0x7fe900102470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:55.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:55.709+0000 7fe905600640 1 -- 172.21.15.45:0/4109729979 >> 172.21.15.45:0/4109729979 conn(0x7fe9000fd5e0 msgr2=0x7fe9000ffa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:03:55.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:55.709+0000 7fe905600640 1 -- 172.21.15.45:0/4109729979 shutdown_connections 2024-04-16T17:03:55.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:55.709+0000 7fe905600640 1 -- 172.21.15.45:0/4109729979 wait complete. 2024-04-16T17:03:55.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:55.710+0000 7fe905600640 1 Processor -- start 2024-04-16T17:03:55.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:55.710+0000 7fe905600640 1 -- start start 2024-04-16T17:03:55.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:55.711+0000 7fe905600640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe900102050 0x7fe90019a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:55.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:55.711+0000 7fe905600640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe900103240 0x7fe90019aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:55.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:55.711+0000 7fe905600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe900103c00 0x7fe9001a1aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:55.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:55.711+0000 7fe905600640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fe90010dce0 con 0x7fe900103c00 2024-04-16T17:03:55.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:55.711+0000 7fe905600640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fe90010db60 con 0x7fe900103240 2024-04-16T17:03:55.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:55.711+0000 7fe905600640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fe90010de60 con 0x7fe900102050 2024-04-16T17:03:55.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:55.711+0000 7fe8ff400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe900103c00 0x7fe9001a1aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:55.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:55.711+0000 7fe8ff400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe900103c00 0x7fe9001a1aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45006/0 (socket says 172.21.15.45:45006) 2024-04-16T17:03:55.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:55.711+0000 7fe8f6000640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe900103240 0x7fe90019aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:55.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:55.711+0000 7fe8ff400640 1 -- 172.21.15.45:0/1920840054 learned_addr learned my addr 172.21.15.45:0/1920840054 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:03:55.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:55.711+0000 7fe8fea00640 1 --2- 172.21.15.45:0/1920840054 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe900102050 0x7fe90019a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:55.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:55.712+0000 7fe8f6000640 1 -- 172.21.15.45:0/1920840054 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe900102050 msgr2=0x7fe90019a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:55.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:55.712+0000 7fe8f6000640 1 --2- 172.21.15.45:0/1920840054 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe900102050 0x7fe90019a4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:55.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:55.712+0000 7fe8f6000640 1 -- 172.21.15.45:0/1920840054 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe900103c00 msgr2=0x7fe9001a1aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:55.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:55.712+0000 7fe8f6000640 1 --2- 172.21.15.45:0/1920840054 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe900103c00 0x7fe9001a1aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:55.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:55.712+0000 7fe8f6000640 1 -- 172.21.15.45:0/1920840054 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9001a2110 con 0x7fe900103240 2024-04-16T17:03:55.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:55.712+0000 7fe8ff400640 1 --2- 172.21.15.45:0/1920840054 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe900103c00 0x7fe9001a1aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:03:55.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:55.712+0000 7fe8fea00640 1 --2- 172.21.15.45:0/1920840054 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe900102050 0x7fe90019a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:03:55.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:55.712+0000 7fe8f6000640 1 --2- 172.21.15.45:0/1920840054 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe900103240 0x7fe90019aa30 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fe8e800e9f0 tx=0x7fe8e800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:03:55.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:55.712+0000 7fe8f7e00640 1 -- 172.21.15.45:0/1920840054 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8e800ccc0 con 0x7fe900103240 2024-04-16T17:03:55.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:55.713+0000 7fe905600640 1 -- 172.21.15.45:0/1920840054 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9001a2400 con 0x7fe900103240 2024-04-16T17:03:55.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:55.713+0000 7fe8f7e00640 1 -- 172.21.15.45:0/1920840054 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe8e8010910 con 0x7fe900103240 2024-04-16T17:03:55.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:55.713+0000 7fe8f7e00640 1 -- 172.21.15.45:0/1920840054 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8e8018960 con 0x7fe900103240 2024-04-16T17:03:55.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:55.713+0000 7fe905600640 1 -- 172.21.15.45:0/1920840054 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9001a29a0 con 0x7fe900103240 2024-04-16T17:03:55.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:55.714+0000 7fe905600640 1 -- 172.21.15.45:0/1920840054 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9001a2cc0 con 0x7fe900103240 2024-04-16T17:03:55.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:55.718+0000 7fe8f7e00640 1 -- 172.21.15.45:0/1920840054 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fe8e8010ac0 con 0x7fe900103240 2024-04-16T17:03:55.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:55.719+0000 7fe8f7e00640 1 --2- 172.21.15.45:0/1920840054 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fe8e40a3c40 0x7fe8e40a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:55.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:55.720+0000 7fe8f7e00640 1 -- 172.21.15.45:0/1920840054 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7fe8e80d1020 con 0x7fe900103240 2024-04-16T17:03:55.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:55.720+0000 7fe8fea00640 1 --2- 172.21.15.45:0/1920840054 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fe8e40a3c40 0x7fe8e40a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:55.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:55.720+0000 7fe8f7e00640 1 -- 172.21.15.45:0/1920840054 <== mon.2 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe8e809d010 con 0x7fe900103240 2024-04-16T17:03:55.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:55.720+0000 7fe8fea00640 1 --2- 172.21.15.45:0/1920840054 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fe8e40a3c40 0x7fe8e40a6100 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fe9001030a0 tx=0x7fe8f0009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:03:55.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:03:55 smithi117 ceph-mon[36730]: from='client.24817 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T17:03:55.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:03:55 smithi117 ceph-mon[36730]: pgmap v337: 129 pgs: 129 active+clean; 581 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s wr, 2 op/s 2024-04-16T17:03:55.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:03:55 smithi117 ceph-mon[36730]: from='client.24833 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T17:03:55.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:03:55 smithi117 ceph-mon[36730]: mgrmap e17: x(active, since 8m), standbys: z, y 2024-04-16T17:03:55.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:55.961+0000 7fe905600640 1 -- 172.21.15.45:0/1920840054 --> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7fe900075950 con 0x7fe8e40a3c40 2024-04-16T17:03:55.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:55.970+0000 7fe8f7e00640 1 -- 172.21.15.45:0/1920840054 <== mgr.14152 v2:172.21.15.45:6800/268030699 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7fe900075950 con 0x7fe8e40a3c40 2024-04-16T17:03:55.971 INFO:teuthology.orchestra.run.smithi045.stdout:/volumes/qa/sv_1/584ee6b0-43a5-48e1-b843-9914b8e65254 2024-04-16T17:03:55.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:55.972+0000 7fe905600640 1 -- 172.21.15.45:0/1920840054 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fe8e40a3c40 msgr2=0x7fe8e40a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:55.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:55.972+0000 7fe905600640 1 --2- 172.21.15.45:0/1920840054 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fe8e40a3c40 0x7fe8e40a6100 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fe9001030a0 tx=0x7fe8f0009500 comp rx=0 tx=0).stop 2024-04-16T17:03:55.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:55.972+0000 7fe905600640 1 -- 172.21.15.45:0/1920840054 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe900103240 msgr2=0x7fe90019aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:55.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:55.972+0000 7fe905600640 1 --2- 172.21.15.45:0/1920840054 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe900103240 0x7fe90019aa30 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fe8e800e9f0 tx=0x7fe8e800eeb0 comp rx=0 tx=0).stop 2024-04-16T17:03:55.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:55.972+0000 7fe905600640 1 -- 172.21.15.45:0/1920840054 shutdown_connections 2024-04-16T17:03:55.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:55.972+0000 7fe905600640 1 --2- 172.21.15.45:0/1920840054 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fe8e40a3c40 0x7fe8e40a6100 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:55.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:55.972+0000 7fe905600640 1 --2- 172.21.15.45:0/1920840054 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe900103c00 0x7fe9001a1aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:55.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:55.972+0000 7fe905600640 1 --2- 172.21.15.45:0/1920840054 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe900103240 0x7fe90019aa30 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:55.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:55.972+0000 7fe905600640 1 --2- 172.21.15.45:0/1920840054 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe900102050 0x7fe90019a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:55.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:55.972+0000 7fe905600640 1 -- 172.21.15.45:0/1920840054 >> 172.21.15.45:0/1920840054 conn(0x7fe9000fd5e0 msgr2=0x7fe900074230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:03:55.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:55.973+0000 7fe905600640 1 -- 172.21.15.45:0/1920840054 shutdown_connections 2024-04-16T17:03:55.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:55.973+0000 7fe905600640 1 -- 172.21.15.45:0/1920840054 wait complete. 2024-04-16T17:03:55.994 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-16T17:03:56.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:56.190+0000 7f04fe000640 1 -- 172.21.15.45:0/3088920989 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04f002cc80 con 0x7f0500101ea0 2024-04-16T17:03:56.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:56.191+0000 7f04ffe00640 1 -- 172.21.15.45:0/3088920989 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0500101ea0 msgr2=0x7f05001022c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:56.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:56.191+0000 7f04ffe00640 1 --2- 172.21.15.45:0/3088920989 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0500101ea0 0x7f05001022c0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f04f0009900 tx=0x7f04f002bc30 comp rx=0 tx=0).stop 2024-04-16T17:03:56.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:56.191+0000 7f04ffe00640 1 -- 172.21.15.45:0/3088920989 shutdown_connections 2024-04-16T17:03:56.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:56.191+0000 7f04ffe00640 1 --2- 172.21.15.45:0/3088920989 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0500102ff0 0x7f0500103470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:56.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:56.191+0000 7f04ffe00640 1 --2- 172.21.15.45:0/3088920989 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f05001039b0 0x7f050010a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:56.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:56.191+0000 7f04ffe00640 1 --2- 172.21.15.45:0/3088920989 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0500101ea0 0x7f05001022c0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:56.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:56.191+0000 7f04ffe00640 1 -- 172.21.15.45:0/3088920989 >> 172.21.15.45:0/3088920989 conn(0x7f05000fd410 msgr2=0x7f05000ff850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:03:56.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:56.191+0000 7f04ffe00640 1 -- 172.21.15.45:0/3088920989 shutdown_connections 2024-04-16T17:03:56.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:56.191+0000 7f04ffe00640 1 -- 172.21.15.45:0/3088920989 wait complete. 2024-04-16T17:03:56.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:56.192+0000 7f04ffe00640 1 Processor -- start 2024-04-16T17:03:56.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:56.192+0000 7f04ffe00640 1 -- start start 2024-04-16T17:03:56.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:56.192+0000 7f04ffe00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0500101ea0 0x7f050019a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:56.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:56.192+0000 7f04ffe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0500102ff0 0x7f050019a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:56.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:56.192+0000 7f04ffe00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f05001039b0 0x7f05001a1990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:56.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:56.192+0000 7f04ffe00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f05000752e0 con 0x7f0500102ff0 2024-04-16T17:03:56.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:56.192+0000 7f04ffe00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f0500075160 con 0x7f05001039b0 2024-04-16T17:03:56.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:56.192+0000 7f04ffe00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f0500075460 con 0x7f0500101ea0 2024-04-16T17:03:56.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:56.193+0000 7f04f7e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0500102ff0 0x7f050019a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:56.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:56.193+0000 7f04fea00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0500101ea0 0x7f050019a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:56.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:56.193+0000 7f04fea00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0500101ea0 0x7f050019a3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:55834/0 (socket says 172.21.15.45:55834) 2024-04-16T17:03:56.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:56.193+0000 7f04fea00640 1 -- 172.21.15.45:0/3853156587 learned_addr learned my addr 172.21.15.45:0/3853156587 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:03:56.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:56.193+0000 7f04f7e00640 1 --2- 172.21.15.45:0/3853156587 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0500102ff0 0x7f050019a920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45026/0 (socket says 172.21.15.45:45026) 2024-04-16T17:03:56.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:56.193+0000 7f04ff400640 1 --2- 172.21.15.45:0/3853156587 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f05001039b0 0x7f05001a1990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:56.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:56.193+0000 7f04f7e00640 1 -- 172.21.15.45:0/3853156587 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0500101ea0 msgr2=0x7f050019a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:56.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:56.193+0000 7f04f7e00640 1 --2- 172.21.15.45:0/3853156587 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0500101ea0 0x7f050019a3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:56.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:56.193+0000 7f04f7e00640 1 -- 172.21.15.45:0/3853156587 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f05001039b0 msgr2=0x7f05001a1990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:56.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:56.193+0000 7f04f7e00640 1 --2- 172.21.15.45:0/3853156587 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f05001039b0 0x7f05001a1990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:56.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:56.193+0000 7f04f7e00640 1 -- 172.21.15.45:0/3853156587 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05001a2000 con 0x7f0500102ff0 2024-04-16T17:03:56.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:56.193+0000 7f04f7e00640 1 --2- 172.21.15.45:0/3853156587 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0500102ff0 0x7f050019a920 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f04e800e9c0 tx=0x7f04e800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:03:56.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:56.194+0000 7f0504a00640 1 -- 172.21.15.45:0/3853156587 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04e800ccc0 con 0x7f0500102ff0 2024-04-16T17:03:56.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:56.194+0000 7f0504a00640 1 -- 172.21.15.45:0/3853156587 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f04e80035f0 con 0x7f0500102ff0 2024-04-16T17:03:56.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:56.194+0000 7f0504a00640 1 -- 172.21.15.45:0/3853156587 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04e800ce60 con 0x7f0500102ff0 2024-04-16T17:03:56.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:56.194+0000 7f04ffe00640 1 -- 172.21.15.45:0/3853156587 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05001a22f0 con 0x7f0500102ff0 2024-04-16T17:03:56.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:56.194+0000 7f04ffe00640 1 -- 172.21.15.45:0/3853156587 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05001a27f0 con 0x7f0500102ff0 2024-04-16T17:03:56.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:56.195+0000 7f04ffe00640 1 -- 172.21.15.45:0/3853156587 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05001089d0 con 0x7f0500102ff0 2024-04-16T17:03:56.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:56.197+0000 7f0504a00640 1 -- 172.21.15.45:0/3853156587 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f04e80037a0 con 0x7f0500102ff0 2024-04-16T17:03:56.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:56.199+0000 7f0504a00640 1 --2- 172.21.15.45:0/3853156587 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f04e40a3cc0 0x7f04e40a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:56.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:56.199+0000 7f0504a00640 1 -- 172.21.15.45:0/3853156587 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f04e80d0d40 con 0x7f0500102ff0 2024-04-16T17:03:56.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:56.199+0000 7f04fea00640 1 --2- 172.21.15.45:0/3853156587 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f04e40a3cc0 0x7f04e40a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:56.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:56.200+0000 7f04fea00640 1 --2- 172.21.15.45:0/3853156587 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f04e40a3cc0 0x7f04e40a6180 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f04f0000c00 tx=0x7f04f002c880 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:03:56.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:56.203+0000 7f0504a00640 1 -- 172.21.15.45:0/3853156587 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f04e809d320 con 0x7f0500102ff0 2024-04-16T17:03:56.478 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:03:56 smithi045 ceph-mon[31676]: from='client.24839 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T17:03:56.478 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:03:56 smithi045 ceph-mon[31676]: from='client.24845 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T17:03:56.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:56.498+0000 7f04ffe00640 1 -- 172.21.15.45:0/3853156587 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f050002d620 con 0x7f0500102ff0 2024-04-16T17:03:56.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:56.500+0000 7f0504a00640 1 -- 172.21.15.45:0/3853156587 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11887 (secure 0 0 0) 0x7f04e8087090 con 0x7f0500102ff0 2024-04-16T17:03:56.502 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-16T17:03:56.503 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/1748842443","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":1748842443},{"type":"v1","addr":"172.21.15.45:6835","nonce":1748842443}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6839/1308627051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":1308627051},{"type":"v1","addr":"172.21.15.45:6839","nonce":1308627051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/1768242489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1768242489},{"type":"v1","addr":"172.21.15.117:6833","nonce":1768242489}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6833/3558902585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":3558902585},{"type":"v1","addr":"172.21.15.104:6833","nonce":3558902585}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24478,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/1308142903","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":1308142903},{"type":"v1","addr":"172.21.15.45:6837","nonce":1308142903}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24491,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6835/3430679937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":3430679937},{"type":"v1","addr":"172.21.15.104:6835","nonce":3430679937}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-16T17:03:24.663611+0000","modified":"2024-04-16T17:03:53.381209+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24581,"mds_1":24569,"mds_2":24532},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24499":{"gid":24499,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/2224123841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2224123841},{"type":"v1","addr":"172.21.15.117:6835","nonce":2224123841}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6837/2471356815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":2471356815},{"type":"v1","addr":"172.21.15.117:6837","nonce":2471356815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24524":{"gid":24524,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6837/2647756509","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":2647756509},{"type":"v1","addr":"172.21.15.104:6837","nonce":2647756509}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24532":{"gid":24532,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":13,"addr":"172.21.15.117:6839/2378675949","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":2378675949},{"type":"v1","addr":"172.21.15.117:6839","nonce":2378675949}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24569":{"gid":24569,"name":"a","rank":1,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.45:6841/3755310822","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6840","nonce":3755310822},{"type":"v1","addr":"172.21.15.45:6841","nonce":3755310822}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24581":{"gid":24581,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.104:6839/3660070592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6838","nonce":3660070592},{"type":"v1","addr":"172.21.15.104:6839","nonce":3660070592}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24581,"qdb_cluster":[24569,24532,24581]},"id":1}]} 2024-04-16T17:03:56.503 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 24 2024-04-16T17:03:56.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:56.503+0000 7f04ffe00640 1 -- 172.21.15.45:0/3853156587 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f04e40a3cc0 msgr2=0x7f04e40a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:56.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:56.503+0000 7f04ffe00640 1 --2- 172.21.15.45:0/3853156587 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f04e40a3cc0 0x7f04e40a6180 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f04f0000c00 tx=0x7f04f002c880 comp rx=0 tx=0).stop 2024-04-16T17:03:56.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:56.503+0000 7f04ffe00640 1 -- 172.21.15.45:0/3853156587 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0500102ff0 msgr2=0x7f050019a920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:56.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:56.503+0000 7f04ffe00640 1 --2- 172.21.15.45:0/3853156587 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0500102ff0 0x7f050019a920 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f04e800e9c0 tx=0x7f04e800ee80 comp rx=0 tx=0).stop 2024-04-16T17:03:56.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:56.503+0000 7f04ffe00640 1 -- 172.21.15.45:0/3853156587 shutdown_connections 2024-04-16T17:03:56.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:56.503+0000 7f04ffe00640 1 --2- 172.21.15.45:0/3853156587 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f04e40a3cc0 0x7f04e40a6180 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:56.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:56.503+0000 7f04ffe00640 1 --2- 172.21.15.45:0/3853156587 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f05001039b0 0x7f05001a1990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:56.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:56.503+0000 7f04ffe00640 1 --2- 172.21.15.45:0/3853156587 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0500102ff0 0x7f050019a920 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:56.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:56.503+0000 7f04ffe00640 1 --2- 172.21.15.45:0/3853156587 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0500101ea0 0x7f050019a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:56.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:56.503+0000 7f04ffe00640 1 -- 172.21.15.45:0/3853156587 >> 172.21.15.45:0/3853156587 conn(0x7f05000fd410 msgr2=0x7f05000fe8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:03:56.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:56.504+0000 7f04ffe00640 1 -- 172.21.15.45:0/3853156587 shutdown_connections 2024-04-16T17:03:56.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:56.504+0000 7f04ffe00640 1 -- 172.21.15.45:0/3853156587 wait complete. 2024-04-16T17:03:56.523 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-16T17:03:56.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:03:56 smithi104 ceph-mon[36457]: from='client.24839 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T17:03:56.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:03:56 smithi104 ceph-mon[36457]: from='client.24845 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T17:03:56.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:56.734+0000 7f3e0b400640 1 -- 172.21.15.45:0/2198851812 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3e04104280 msgr2=0x7f3e041046a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:56.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:56.734+0000 7f3e0b400640 1 --2- 172.21.15.45:0/2198851812 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3e04104280 0x7f3e041046a0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f3df8009900 tx=0x7f3df802bc30 comp rx=0 tx=0).stop 2024-04-16T17:03:56.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:56.734+0000 7f3e0b400640 1 -- 172.21.15.45:0/2198851812 shutdown_connections 2024-04-16T17:03:56.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:56.735+0000 7f3e0b400640 1 --2- 172.21.15.45:0/2198851812 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3e04105470 0x7f3e041058f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:56.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:56.735+0000 7f3e0b400640 1 --2- 172.21.15.45:0/2198851812 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3e04105e30 0x7f3e0410c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:56.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:56.735+0000 7f3e0b400640 1 --2- 172.21.15.45:0/2198851812 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3e04104280 0x7f3e041046a0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:56.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:56.735+0000 7f3e0b400640 1 -- 172.21.15.45:0/2198851812 >> 172.21.15.45:0/2198851812 conn(0x7f3e040ff7f0 msgr2=0x7f3e04101c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:03:56.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:56.735+0000 7f3e0b400640 1 -- 172.21.15.45:0/2198851812 shutdown_connections 2024-04-16T17:03:56.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:56.735+0000 7f3e0b400640 1 -- 172.21.15.45:0/2198851812 wait complete. 2024-04-16T17:03:56.739 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:56.736+0000 7f3e0b400640 1 Processor -- start 2024-04-16T17:03:56.739 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:56.736+0000 7f3e0b400640 1 -- start start 2024-04-16T17:03:56.739 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:56.736+0000 7f3e0b400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3e04104280 0x7f3e0419a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:56.739 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:56.736+0000 7f3e0b400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3e04105470 0x7f3e0419aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:56.739 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:56.736+0000 7f3e0b400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3e04105e30 0x7f3e041a1b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:56.739 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:56.736+0000 7f3e0b400640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f3e0410ff60 con 0x7f3e04104280 2024-04-16T17:03:56.739 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:56.736+0000 7f3e0b400640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f3e0410fde0 con 0x7f3e04105470 2024-04-16T17:03:56.739 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:56.736+0000 7f3e0b400640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f3e041100e0 con 0x7f3e04105e30 2024-04-16T17:03:56.739 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:56.736+0000 7f3e08a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3e04104280 0x7f3e0419a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:56.739 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:56.736+0000 7f3e03e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3e04105470 0x7f3e0419aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:56.739 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:56.737+0000 7f3e03e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3e04105470 0x7f3e0419aac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:34422/0 (socket says 172.21.15.45:34422) 2024-04-16T17:03:56.739 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:56.737+0000 7f3e08a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3e04104280 0x7f3e0419a580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45054/0 (socket says 172.21.15.45:45054) 2024-04-16T17:03:56.739 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:56.737+0000 7f3e03e00640 1 -- 172.21.15.45:0/760600030 learned_addr learned my addr 172.21.15.45:0/760600030 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:03:56.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:56.737+0000 7f3e09400640 1 --2- 172.21.15.45:0/760600030 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3e04105e30 0x7f3e041a1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:56.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:56.737+0000 7f3e03e00640 1 -- 172.21.15.45:0/760600030 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3e04105e30 msgr2=0x7f3e041a1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:56.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:56.737+0000 7f3e03e00640 1 --2- 172.21.15.45:0/760600030 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3e04105e30 0x7f3e041a1b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:56.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:56.737+0000 7f3e03e00640 1 -- 172.21.15.45:0/760600030 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3e04104280 msgr2=0x7f3e0419a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:56.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:56.737+0000 7f3e03e00640 1 --2- 172.21.15.45:0/760600030 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3e04104280 0x7f3e0419a580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:56.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:56.737+0000 7f3e03e00640 1 -- 172.21.15.45:0/760600030 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e041a21a0 con 0x7f3e04105470 2024-04-16T17:03:56.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:56.737+0000 7f3e03e00640 1 --2- 172.21.15.45:0/760600030 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3e04105470 0x7f3e0419aac0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f3df000e9f0 tx=0x7f3df000edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:03:56.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:56.738+0000 7f3e01600640 1 -- 172.21.15.45:0/760600030 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3df000cd80 con 0x7f3e04105470 2024-04-16T17:03:56.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:56.738+0000 7f3e01600640 1 -- 172.21.15.45:0/760600030 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3df0005500 con 0x7f3e04105470 2024-04-16T17:03:56.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:56.738+0000 7f3e0b400640 1 -- 172.21.15.45:0/760600030 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e041a2490 con 0x7f3e04105470 2024-04-16T17:03:56.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:56.738+0000 7f3e01600640 1 -- 172.21.15.45:0/760600030 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3df0010690 con 0x7f3e04105470 2024-04-16T17:03:56.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:56.738+0000 7f3e0b400640 1 -- 172.21.15.45:0/760600030 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e041a2990 con 0x7f3e04105470 2024-04-16T17:03:56.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:56.739+0000 7f3e0b400640 1 -- 172.21.15.45:0/760600030 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e0410b5a0 con 0x7f3e04105470 2024-04-16T17:03:56.743 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:56.742+0000 7f3e01600640 1 -- 172.21.15.45:0/760600030 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f3df00046c0 con 0x7f3e04105470 2024-04-16T17:03:56.744 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:56.743+0000 7f3e01600640 1 --2- 172.21.15.45:0/760600030 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f3dec0a3c40 0x7f3dec0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:56.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:56.743+0000 7f3e01600640 1 -- 172.21.15.45:0/760600030 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f3df00d0d50 con 0x7f3e04105470 2024-04-16T17:03:56.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:56.743+0000 7f3e08a00640 1 --2- 172.21.15.45:0/760600030 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f3dec0a3c40 0x7f3dec0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:56.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:56.744+0000 7f3e08a00640 1 --2- 172.21.15.45:0/760600030 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f3dec0a3c40 0x7f3dec0a6100 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f3df802c170 tx=0x7f3df802df90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:03:56.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:56.745+0000 7f3e01600640 1 -- 172.21.15.45:0/760600030 <== mon.2 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3df009d330 con 0x7f3e04105470 2024-04-16T17:03:56.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:03:56 smithi117 ceph-mon[36730]: from='client.24839 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T17:03:56.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:03:56 smithi117 ceph-mon[36730]: from='client.24845 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T17:03:56.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:56.948+0000 7f3e0b400640 1 -- 172.21.15.45:0/760600030 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3e0402d870 con 0x7f3e04105470 2024-04-16T17:03:56.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:56.950+0000 7f3e01600640 1 -- 172.21.15.45:0/760600030 <== mon.2 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+24164 (secure 0 0 0) 0x7f3df009cc30 con 0x7f3e04105470 2024-04-16T17:03:56.953 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-16T17:03:56.953 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":90,"fsid":"b1fecd38-fc11-11ee-bc90-c7b262605968","created":"2024-04-16T16:54:02.385906+0000","modified":"2024-04-16T17:03:29.720565+0000","last_up_change":"2024-04-16T17:02:25.162624+0000","last_in_change":"2024-04-16T17:02:04.903274+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-16T16:58:06.166659+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-16T17:03:22.304198+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-16T17:03:23.616450+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-16T17:03:27.226289+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b655de2c-59f8-4fd4-ac98-0cc222229f56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":2290170881},{"type":"v1","addr":"172.21.15.45:6803","nonce":2290170881}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":2290170881},{"type":"v1","addr":"172.21.15.45:6805","nonce":2290170881}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":2290170881},{"type":"v1","addr":"172.21.15.45:6809","nonce":2290170881}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":2290170881},{"type":"v1","addr":"172.21.15.45:6807","nonce":2290170881}]},"public_addr":"172.21.15.45:6803/2290170881","cluster_addr":"172.21.15.45:6805/2290170881","heartbeat_back_addr":"172.21.15.45:6809/2290170881","heartbeat_front_addr":"172.21.15.45:6807/2290170881","state":["exists","up"]},{"osd":1,"uuid":"ebd450ec-08fd-4c3d-8b24-4e10634ddb7e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6800","nonce":1462942067},{"type":"v1","addr":"172.21.15.104:6801","nonce":1462942067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":1462942067},{"type":"v1","addr":"172.21.15.104:6803","nonce":1462942067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":1462942067},{"type":"v1","addr":"172.21.15.104:6807","nonce":1462942067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":1462942067},{"type":"v1","addr":"172.21.15.104:6805","nonce":1462942067}]},"public_addr":"172.21.15.104:6801/1462942067","cluster_addr":"172.21.15.104:6803/1462942067","heartbeat_back_addr":"172.21.15.104:6807/1462942067","heartbeat_front_addr":"172.21.15.104:6805/1462942067","state":["exists","up"]},{"osd":2,"uuid":"b318a512-4190-42a2-8104-dcc4026ad8ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":2829260735},{"type":"v1","addr":"172.21.15.117:6801","nonce":2829260735}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":2829260735},{"type":"v1","addr":"172.21.15.117:6803","nonce":2829260735}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":2829260735},{"type":"v1","addr":"172.21.15.117:6807","nonce":2829260735}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":2829260735},{"type":"v1","addr":"172.21.15.117:6805","nonce":2829260735}]},"public_addr":"172.21.15.117:6801/2829260735","cluster_addr":"172.21.15.117:6803/2829260735","heartbeat_back_addr":"172.21.15.117:6807/2829260735","heartbeat_front_addr":"172.21.15.117:6805/2829260735","state":["exists","up"]},{"osd":3,"uuid":"ec6490cb-a715-4c12-a54a-225aae26fcb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":2823712137},{"type":"v1","addr":"172.21.15.45:6811","nonce":2823712137}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":2823712137},{"type":"v1","addr":"172.21.15.45:6813","nonce":2823712137}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":2823712137},{"type":"v1","addr":"172.21.15.45:6817","nonce":2823712137}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":2823712137},{"type":"v1","addr":"172.21.15.45:6815","nonce":2823712137}]},"public_addr":"172.21.15.45:6811/2823712137","cluster_addr":"172.21.15.45:6813/2823712137","heartbeat_back_addr":"172.21.15.45:6817/2823712137","heartbeat_front_addr":"172.21.15.45:6815/2823712137","state":["exists","up"]},{"osd":4,"uuid":"5d91d568-0250-4aae-8dc2-a0e079f92062","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":1396278127},{"type":"v1","addr":"172.21.15.104:6809","nonce":1396278127}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":1396278127},{"type":"v1","addr":"172.21.15.104:6811","nonce":1396278127}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":1396278127},{"type":"v1","addr":"172.21.15.104:6815","nonce":1396278127}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":1396278127},{"type":"v1","addr":"172.21.15.104:6813","nonce":1396278127}]},"public_addr":"172.21.15.104:6809/1396278127","cluster_addr":"172.21.15.104:6811/1396278127","heartbeat_back_addr":"172.21.15.104:6815/1396278127","heartbeat_front_addr":"172.21.15.104:6813/1396278127","state":["exists","up"]},{"osd":5,"uuid":"ac510b43-6fb2-4d0f-bc5f-16ac080d9e9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":4145720047},{"type":"v1","addr":"172.21.15.117:6809","nonce":4145720047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":4145720047},{"type":"v1","addr":"172.21.15.117:6811","nonce":4145720047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":4145720047},{"type":"v1","addr":"172.21.15.117:6815","nonce":4145720047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":4145720047},{"type":"v1","addr":"172.21.15.117:6813","nonce":4145720047}]},"public_addr":"172.21.15.117:6809/4145720047","cluster_addr":"172.21.15.117:6811/4145720047","heartbeat_back_addr":"172.21.15.117:6815/4145720047","heartbeat_front_addr":"172.21.15.117:6813/4145720047","state":["exists","up"]},{"osd":6,"uuid":"a41adc18-d88a-419e-b2c2-b089c1f32472","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":3254229176},{"type":"v1","addr":"172.21.15.45:6819","nonce":3254229176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":3254229176},{"type":"v1","addr":"172.21.15.45:6821","nonce":3254229176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":3254229176},{"type":"v1","addr":"172.21.15.45:6825","nonce":3254229176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":3254229176},{"type":"v1","addr":"172.21.15.45:6823","nonce":3254229176}]},"public_addr":"172.21.15.45:6819/3254229176","cluster_addr":"172.21.15.45:6821/3254229176","heartbeat_back_addr":"172.21.15.45:6825/3254229176","heartbeat_front_addr":"172.21.15.45:6823/3254229176","state":["exists","up"]},{"osd":7,"uuid":"b71712a4-7cf6-454d-9fd1-565e3df21c1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":4151077186},{"type":"v1","addr":"172.21.15.104:6817","nonce":4151077186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":4151077186},{"type":"v1","addr":"172.21.15.104:6819","nonce":4151077186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":4151077186},{"type":"v1","addr":"172.21.15.104:6823","nonce":4151077186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":4151077186},{"type":"v1","addr":"172.21.15.104:6821","nonce":4151077186}]},"public_addr":"172.21.15.104:6817/4151077186","cluster_addr":"172.21.15.104:6819/4151077186","heartbeat_back_addr":"172.21.15.104:6823/4151077186","heartbeat_front_addr":"172.21.15.104:6821/4151077186","state":["exists","up"]},{"osd":8,"uuid":"9e572aef-59a8-450e-a92e-df9dff4d57a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":1500286830},{"type":"v1","addr":"172.21.15.117:6817","nonce":1500286830}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":1500286830},{"type":"v1","addr":"172.21.15.117:6819","nonce":1500286830}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":1500286830},{"type":"v1","addr":"172.21.15.117:6823","nonce":1500286830}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":1500286830},{"type":"v1","addr":"172.21.15.117:6821","nonce":1500286830}]},"public_addr":"172.21.15.117:6817/1500286830","cluster_addr":"172.21.15.117:6819/1500286830","heartbeat_back_addr":"172.21.15.117:6823/1500286830","heartbeat_front_addr":"172.21.15.117:6821/1500286830","state":["exists","up"]},{"osd":9,"uuid":"014e9598-c413-4cc4-9bb8-61c6e8f59d30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":3106734074},{"type":"v1","addr":"172.21.15.45:6827","nonce":3106734074}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":3106734074},{"type":"v1","addr":"172.21.15.45:6829","nonce":3106734074}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":3106734074},{"type":"v1","addr":"172.21.15.45:6833","nonce":3106734074}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":3106734074},{"type":"v1","addr":"172.21.15.45:6831","nonce":3106734074}]},"public_addr":"172.21.15.45:6827/3106734074","cluster_addr":"172.21.15.45:6829/3106734074","heartbeat_back_addr":"172.21.15.45:6833/3106734074","heartbeat_front_addr":"172.21.15.45:6831/3106734074","state":["exists","up"]},{"osd":10,"uuid":"2deba364-3e39-47be-8a1f-8256df5b82d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":1404794652},{"type":"v1","addr":"172.21.15.104:6825","nonce":1404794652}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":1404794652},{"type":"v1","addr":"172.21.15.104:6827","nonce":1404794652}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":1404794652},{"type":"v1","addr":"172.21.15.104:6831","nonce":1404794652}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":1404794652},{"type":"v1","addr":"172.21.15.104:6829","nonce":1404794652}]},"public_addr":"172.21.15.104:6825/1404794652","cluster_addr":"172.21.15.104:6827/1404794652","heartbeat_back_addr":"172.21.15.104:6831/1404794652","heartbeat_front_addr":"172.21.15.104:6829/1404794652","state":["exists","up"]},{"osd":11,"uuid":"3fda4fea-59ad-4d1a-9fe6-fcb30e31fd7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":3384003243},{"type":"v1","addr":"172.21.15.117:6825","nonce":3384003243}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":3384003243},{"type":"v1","addr":"172.21.15.117:6827","nonce":3384003243}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":3384003243},{"type":"v1","addr":"172.21.15.117:6831","nonce":3384003243}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":3384003243},{"type":"v1","addr":"172.21.15.117:6829","nonce":3384003243}]},"public_addr":"172.21.15.117:6825/3384003243","cluster_addr":"172.21.15.117:6827/3384003243","heartbeat_back_addr":"172.21.15.117:6831/3384003243","heartbeat_front_addr":"172.21.15.117:6829/3384003243","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:57:14.799608+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:57:39.884668+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:58:03.327057+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:58:37.314531+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:59:03.667650+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:59:28.462823+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T17:00:02.968574+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T17:00:29.559190+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T17:00:55.291007+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T17:01:29.815977+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T17:01:57.015580+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T17:02:23.233151+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.45:0/4074982598":"2024-04-17T16:55:19.948001+0000","172.21.15.45:0/2443572576":"2024-04-17T16:55:19.948001+0000","172.21.15.45:6801/1983630136":"2024-04-17T16:55:19.948001+0000","172.21.15.45:6800/1983630136":"2024-04-17T16:55:19.948001+0000","172.21.15.45:0/3150465248":"2024-04-17T16:54:43.692466+0000","172.21.15.45:0/556738623":"2024-04-17T16:54:43.692466+0000","172.21.15.45:0/445676297":"2024-04-17T16:55:19.948001+0000","172.21.15.45:6800/2553659622":"2024-04-17T16:54:43.692466+0000","172.21.15.45:6801/2553659622":"2024-04-17T16:54:43.692466+0000","172.21.15.45:0/1184615408":"2024-04-17T16:54:43.692466+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-16T17:03:56.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:56.953+0000 7f3e0b400640 1 -- 172.21.15.45:0/760600030 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f3dec0a3c40 msgr2=0x7f3dec0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:56.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:56.953+0000 7f3e0b400640 1 --2- 172.21.15.45:0/760600030 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f3dec0a3c40 0x7f3dec0a6100 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f3df802c170 tx=0x7f3df802df90 comp rx=0 tx=0).stop 2024-04-16T17:03:56.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:56.953+0000 7f3e0b400640 1 -- 172.21.15.45:0/760600030 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3e04105470 msgr2=0x7f3e0419aac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:56.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:56.953+0000 7f3e0b400640 1 --2- 172.21.15.45:0/760600030 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3e04105470 0x7f3e0419aac0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f3df000e9f0 tx=0x7f3df000edf0 comp rx=0 tx=0).stop 2024-04-16T17:03:56.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:56.953+0000 7f3e0b400640 1 -- 172.21.15.45:0/760600030 shutdown_connections 2024-04-16T17:03:56.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:56.953+0000 7f3e0b400640 1 --2- 172.21.15.45:0/760600030 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f3dec0a3c40 0x7f3dec0a6100 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:56.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:56.953+0000 7f3e0b400640 1 --2- 172.21.15.45:0/760600030 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3e04105e30 0x7f3e041a1b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:56.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:56.953+0000 7f3e0b400640 1 --2- 172.21.15.45:0/760600030 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3e04105470 0x7f3e0419aac0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:56.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:56.953+0000 7f3e0b400640 1 --2- 172.21.15.45:0/760600030 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3e04104280 0x7f3e0419a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:56.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:56.953+0000 7f3e0b400640 1 -- 172.21.15.45:0/760600030 >> 172.21.15.45:0/760600030 conn(0x7f3e040ff7f0 msgr2=0x7f3e04108770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:03:56.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:56.953+0000 7f3e0b400640 1 -- 172.21.15.45:0/760600030 shutdown_connections 2024-04-16T17:03:56.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:56.953+0000 7f3e0b400640 1 -- 172.21.15.45:0/760600030 wait complete. 2024-04-16T17:03:56.975 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-16T17:03:56.975 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-16T17:03:57.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:57.187+0000 7fb827e00640 1 -- 172.21.15.45:0/1230176449 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb828074a60 msgr2=0x7fb828074ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:57.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:57.187+0000 7fb827e00640 1 --2- 172.21.15.45:0/1230176449 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb828074a60 0x7fb828074ee0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fb814009900 tx=0x7fb81402bc40 comp rx=0 tx=0).stop 2024-04-16T17:03:57.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:57.187+0000 7fb827e00640 1 -- 172.21.15.45:0/1230176449 shutdown_connections 2024-04-16T17:03:57.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:57.187+0000 7fb827e00640 1 --2- 172.21.15.45:0/1230176449 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb828109d60 0x7fb82810c1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:57.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:57.187+0000 7fb827e00640 1 --2- 172.21.15.45:0/1230176449 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb828074a60 0x7fb828074ee0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:57.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:57.187+0000 7fb827e00640 1 --2- 172.21.15.45:0/1230176449 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb828076040 0x7fb828074490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:57.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:57.187+0000 7fb827e00640 1 -- 172.21.15.45:0/1230176449 >> 172.21.15.45:0/1230176449 conn(0x7fb8280fd410 msgr2=0x7fb8280ff850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:03:57.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:57.187+0000 7fb827e00640 1 -- 172.21.15.45:0/1230176449 shutdown_connections 2024-04-16T17:03:57.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:57.187+0000 7fb827e00640 1 -- 172.21.15.45:0/1230176449 wait complete. 2024-04-16T17:03:57.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:57.188+0000 7fb827e00640 1 Processor -- start 2024-04-16T17:03:57.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:57.188+0000 7fb827e00640 1 -- start start 2024-04-16T17:03:57.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:57.189+0000 7fb827e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb828074a60 0x7fb82819a370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:57.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:57.189+0000 7fb827e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb828076040 0x7fb82819a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:57.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:57.189+0000 7fb827e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb828109d60 0x7fb8281a1920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:57.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:57.189+0000 7fb827e00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fb82810fd70 con 0x7fb828074a60 2024-04-16T17:03:57.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:57.189+0000 7fb827e00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fb82810fbf0 con 0x7fb828076040 2024-04-16T17:03:57.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:57.189+0000 7fb827e00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb82810fef0 con 0x7fb828109d60 2024-04-16T17:03:57.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:57.189+0000 7fb826a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb828074a60 0x7fb82819a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:57.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:57.189+0000 7fb826a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb828074a60 0x7fb82819a370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45082/0 (socket says 172.21.15.45:45082) 2024-04-16T17:03:57.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:57.189+0000 7fb826a00640 1 -- 172.21.15.45:0/3371023053 learned_addr learned my addr 172.21.15.45:0/3371023053 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:03:57.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:57.189+0000 7fb826000640 1 --2- 172.21.15.45:0/3371023053 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb828076040 0x7fb82819a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:57.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:57.189+0000 7fb827400640 1 --2- 172.21.15.45:0/3371023053 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb828109d60 0x7fb8281a1920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:57.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:57.190+0000 7fb826000640 1 -- 172.21.15.45:0/3371023053 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb828109d60 msgr2=0x7fb8281a1920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:57.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:57.190+0000 7fb826000640 1 --2- 172.21.15.45:0/3371023053 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb828109d60 0x7fb8281a1920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:57.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:57.190+0000 7fb826000640 1 -- 172.21.15.45:0/3371023053 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb828074a60 msgr2=0x7fb82819a370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:57.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:57.190+0000 7fb826000640 1 --2- 172.21.15.45:0/3371023053 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb828074a60 0x7fb82819a370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:57.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:57.190+0000 7fb826000640 1 -- 172.21.15.45:0/3371023053 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8281a1f90 con 0x7fb828076040 2024-04-16T17:03:57.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:57.190+0000 7fb827400640 1 --2- 172.21.15.45:0/3371023053 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb828109d60 0x7fb8281a1920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:03:57.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:57.190+0000 7fb826a00640 1 --2- 172.21.15.45:0/3371023053 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb828074a60 0x7fb82819a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:03:57.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:57.190+0000 7fb826000640 1 --2- 172.21.15.45:0/3371023053 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb828076040 0x7fb82819a8b0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fb814036040 tx=0x7fb81402c660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:03:57.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:57.190+0000 7fb81f400640 1 -- 172.21.15.45:0/3371023053 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb814041070 con 0x7fb828076040 2024-04-16T17:03:57.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:57.190+0000 7fb81f400640 1 -- 172.21.15.45:0/3371023053 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb814004450 con 0x7fb828076040 2024-04-16T17:03:57.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:57.190+0000 7fb827e00640 1 -- 172.21.15.45:0/3371023053 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8281a2220 con 0x7fb828076040 2024-04-16T17:03:57.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:57.190+0000 7fb81f400640 1 -- 172.21.15.45:0/3371023053 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb814003df0 con 0x7fb828076040 2024-04-16T17:03:57.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:57.191+0000 7fb827e00640 1 -- 172.21.15.45:0/3371023053 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8281a26e0 con 0x7fb828076040 2024-04-16T17:03:57.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:57.192+0000 7fb827e00640 1 -- 172.21.15.45:0/3371023053 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb828108a90 con 0x7fb828076040 2024-04-16T17:03:57.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:57.193+0000 7fb81f400640 1 -- 172.21.15.45:0/3371023053 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fb814044070 con 0x7fb828076040 2024-04-16T17:03:57.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:57.195+0000 7fb81f400640 1 --2- 172.21.15.45:0/3371023053 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fb80c0a3c40 0x7fb80c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:57.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:57.195+0000 7fb81f400640 1 -- 172.21.15.45:0/3371023053 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7fb8140f2180 con 0x7fb828076040 2024-04-16T17:03:57.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:57.195+0000 7fb826a00640 1 --2- 172.21.15.45:0/3371023053 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fb80c0a3c40 0x7fb80c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:57.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:57.196+0000 7fb826a00640 1 --2- 172.21.15.45:0/3371023053 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fb80c0a3c40 0x7fb80c0a6100 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fb818011500 tx=0x7fb818005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:03:57.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:57.199+0000 7fb81f400640 1 -- 172.21.15.45:0/3371023053 <== mon.2 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb8140be870 con 0x7fb828076040 2024-04-16T17:03:57.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:57.401+0000 7fb827e00640 1 -- 172.21.15.45:0/3371023053 --> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb828062440 con 0x7fb80c0a3c40 2024-04-16T17:03:57.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:57.416+0000 7fb81f400640 1 -- 172.21.15.45:0/3371023053 <== mgr.14152 v2:172.21.15.45:6800/268030699 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340627 (secure 0 0 0) 0x7fb828062440 con 0x7fb80c0a3c40 2024-04-16T17:03:57.423 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-16T17:03:57.425 INFO:teuthology.orchestra.run.smithi045.stderr:dumped all 2024-04-16T17:03:57.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:57.423+0000 7fb827e00640 1 -- 172.21.15.45:0/3371023053 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fb80c0a3c40 msgr2=0x7fb80c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:57.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:57.423+0000 7fb827e00640 1 --2- 172.21.15.45:0/3371023053 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fb80c0a3c40 0x7fb80c0a6100 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fb818011500 tx=0x7fb818005f90 comp rx=0 tx=0).stop 2024-04-16T17:03:57.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:57.423+0000 7fb827e00640 1 -- 172.21.15.45:0/3371023053 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb828076040 msgr2=0x7fb82819a8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:57.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:57.424+0000 7fb827e00640 1 --2- 172.21.15.45:0/3371023053 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb828076040 0x7fb82819a8b0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fb814036040 tx=0x7fb81402c660 comp rx=0 tx=0).stop 2024-04-16T17:03:57.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:57.424+0000 7fb827e00640 1 -- 172.21.15.45:0/3371023053 shutdown_connections 2024-04-16T17:03:57.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:57.424+0000 7fb827e00640 1 --2- 172.21.15.45:0/3371023053 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fb80c0a3c40 0x7fb80c0a6100 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:57.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:57.424+0000 7fb827e00640 1 --2- 172.21.15.45:0/3371023053 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb828109d60 0x7fb8281a1920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:57.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:57.424+0000 7fb827e00640 1 --2- 172.21.15.45:0/3371023053 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb828076040 0x7fb82819a8b0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:57.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:57.424+0000 7fb827e00640 1 --2- 172.21.15.45:0/3371023053 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb828074a60 0x7fb82819a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:57.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:57.424+0000 7fb827e00640 1 -- 172.21.15.45:0/3371023053 >> 172.21.15.45:0/3371023053 conn(0x7fb8280fd410 msgr2=0x7fb828105c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:03:57.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:57.424+0000 7fb827e00640 1 -- 172.21.15.45:0/3371023053 shutdown_connections 2024-04-16T17:03:57.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:57.424+0000 7fb827e00640 1 -- 172.21.15.45:0/3371023053 wait complete. 2024-04-16T17:03:57.445 INFO:teuthology.orchestra.run.smithi045.stdout:{"pg_ready":true,"pg_map":{"version":338,"stamp":"2024-04-16T17:03:56.109071+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":604077,"num_objects":54,"num_object_clones":0,"num_object_copies":162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":0,"num_read":154,"num_read_kb":136,"num_write":303,"num_write_kb":2980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":36,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":242,"ondisk_log_size":242,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":369224,"kb_used_data":11740,"kb_used_omap":18,"kb_used_meta":357357,"kb_avail":1124572600,"statfs":{"total":1151940427776,"available":1151562342400,"internally_reserved":0,"allocated":12021760,"data_stored":7582181,"data_compressed":284742,"data_compressed_allocated":380928,"data_compressed_original":2260992,"omap_allocated":19143,"internal_metadata":365933881},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":11299,"num_objects":30,"num_object_clones":0,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":18,"num_read_kb":17,"num_write":77,"num_write_kb":28,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":22,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006640"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:30.400293+0000","last_change":"2024-04-16T17:03:24.989508+0000","last_active":"2024-04-16T17:03:30.400293+0000","last_peered":"2024-04-16T17:03:30.400293+0000","last_clean":"2024-04-16T17:03:30.400293+0000","last_became_active":"2024-04-16T17:03:24.989178+0000","last_became_peered":"2024-04-16T17:03:24.989178+0000","last_unstale":"2024-04-16T17:03:30.400293+0000","last_undegraded":"2024-04-16T17:03:30.400293+0000","last_fullsized":"2024-04-16T17:03:30.400293+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:18:56.488358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:30.512048+0000","last_change":"2024-04-16T17:03:24.986378+0000","last_active":"2024-04-16T17:03:30.512048+0000","last_peered":"2024-04-16T17:03:30.512048+0000","last_clean":"2024-04-16T17:03:30.512048+0000","last_became_active":"2024-04-16T17:03:24.986030+0000","last_became_peered":"2024-04-16T17:03:24.986030+0000","last_unstale":"2024-04-16T17:03:30.512048+0000","last_undegraded":"2024-04-16T17:03:30.512048+0000","last_fullsized":"2024-04-16T17:03:30.512048+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:21:14.071109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:30.511599+0000","last_change":"2024-04-16T17:03:24.976232+0000","last_active":"2024-04-16T17:03:30.511599+0000","last_peered":"2024-04-16T17:03:30.511599+0000","last_clean":"2024-04-16T17:03:30.511599+0000","last_became_active":"2024-04-16T17:03:24.975873+0000","last_became_peered":"2024-04-16T17:03:24.975873+0000","last_unstale":"2024-04-16T17:03:30.511599+0000","last_undegraded":"2024-04-16T17:03:30.511599+0000","last_fullsized":"2024-04-16T17:03:30.511599+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:07:53.523101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:30.512686+0000","last_change":"2024-04-16T17:03:24.967995+0000","last_active":"2024-04-16T17:03:30.512686+0000","last_peered":"2024-04-16T17:03:30.512686+0000","last_clean":"2024-04-16T17:03:30.512686+0000","last_became_active":"2024-04-16T17:03:24.967675+0000","last_became_peered":"2024-04-16T17:03:24.967675+0000","last_unstale":"2024-04-16T17:03:30.512686+0000","last_undegraded":"2024-04-16T17:03:30.512686+0000","last_fullsized":"2024-04-16T17:03:30.512686+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:44:38.474399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:30.510925+0000","last_change":"2024-04-16T17:03:24.985609+0000","last_active":"2024-04-16T17:03:30.510925+0000","last_peered":"2024-04-16T17:03:30.510925+0000","last_clean":"2024-04-16T17:03:30.510925+0000","last_became_active":"2024-04-16T17:03:24.985239+0000","last_became_peered":"2024-04-16T17:03:24.985239+0000","last_unstale":"2024-04-16T17:03:30.510925+0000","last_undegraded":"2024-04-16T17:03:30.510925+0000","last_fullsized":"2024-04-16T17:03:30.510925+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:55:58.619210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:30.511963+0000","last_change":"2024-04-16T17:03:24.966862+0000","last_active":"2024-04-16T17:03:30.511963+0000","last_peered":"2024-04-16T17:03:30.511963+0000","last_clean":"2024-04-16T17:03:30.511963+0000","last_became_active":"2024-04-16T17:03:24.966234+0000","last_became_peered":"2024-04-16T17:03:24.966234+0000","last_unstale":"2024-04-16T17:03:30.511963+0000","last_undegraded":"2024-04-16T17:03:30.511963+0000","last_fullsized":"2024-04-16T17:03:30.511963+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:21:07.018281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:30.511080+0000","last_change":"2024-04-16T17:03:24.971219+0000","last_active":"2024-04-16T17:03:30.511080+0000","last_peered":"2024-04-16T17:03:30.511080+0000","last_clean":"2024-04-16T17:03:30.511080+0000","last_became_active":"2024-04-16T17:03:24.970876+0000","last_became_peered":"2024-04-16T17:03:24.970876+0000","last_unstale":"2024-04-16T17:03:30.511080+0000","last_undegraded":"2024-04-16T17:03:30.511080+0000","last_fullsized":"2024-04-16T17:03:30.511080+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:44:16.311167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:30.510880+0000","last_change":"2024-04-16T17:03:24.994694+0000","last_active":"2024-04-16T17:03:30.510880+0000","last_peered":"2024-04-16T17:03:30.510880+0000","last_clean":"2024-04-16T17:03:30.510880+0000","last_became_active":"2024-04-16T17:03:24.994315+0000","last_became_peered":"2024-04-16T17:03:24.994315+0000","last_unstale":"2024-04-16T17:03:30.510880+0000","last_undegraded":"2024-04-16T17:03:30.510880+0000","last_fullsized":"2024-04-16T17:03:30.510880+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:04:59.543816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:30.251023+0000","last_change":"2024-04-16T17:03:24.964654+0000","last_active":"2024-04-16T17:03:30.251023+0000","last_peered":"2024-04-16T17:03:30.251023+0000","last_clean":"2024-04-16T17:03:30.251023+0000","last_became_active":"2024-04-16T17:03:24.964349+0000","last_became_peered":"2024-04-16T17:03:24.964349+0000","last_unstale":"2024-04-16T17:03:30.251023+0000","last_undegraded":"2024-04-16T17:03:30.251023+0000","last_fullsized":"2024-04-16T17:03:30.251023+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:38:55.880948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:30.512071+0000","last_change":"2024-04-16T17:03:24.966364+0000","last_active":"2024-04-16T17:03:30.512071+0000","last_peered":"2024-04-16T17:03:30.512071+0000","last_clean":"2024-04-16T17:03:30.512071+0000","last_became_active":"2024-04-16T17:03:24.966061+0000","last_became_peered":"2024-04-16T17:03:24.966061+0000","last_unstale":"2024-04-16T17:03:30.512071+0000","last_undegraded":"2024-04-16T17:03:30.512071+0000","last_fullsized":"2024-04-16T17:03:30.512071+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:18:12.642602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:30.512774+0000","last_change":"2024-04-16T17:03:24.969874+0000","last_active":"2024-04-16T17:03:30.512774+0000","last_peered":"2024-04-16T17:03:30.512774+0000","last_clean":"2024-04-16T17:03:30.512774+0000","last_became_active":"2024-04-16T17:03:24.969523+0000","last_became_peered":"2024-04-16T17:03:24.969523+0000","last_unstale":"2024-04-16T17:03:30.512774+0000","last_undegraded":"2024-04-16T17:03:30.512774+0000","last_fullsized":"2024-04-16T17:03:30.512774+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:17:49.704855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:30.512389+0000","last_change":"2024-04-16T17:03:24.968660+0000","last_active":"2024-04-16T17:03:30.512389+0000","last_peered":"2024-04-16T17:03:30.512389+0000","last_clean":"2024-04-16T17:03:30.512389+0000","last_became_active":"2024-04-16T17:03:24.968348+0000","last_became_peered":"2024-04-16T17:03:24.968348+0000","last_unstale":"2024-04-16T17:03:30.512389+0000","last_undegraded":"2024-04-16T17:03:30.512389+0000","last_fullsized":"2024-04-16T17:03:30.512389+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:06:06.319391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:30.664287+0000","last_change":"2024-04-16T17:03:24.974741+0000","last_active":"2024-04-16T17:03:30.664287+0000","last_peered":"2024-04-16T17:03:30.664287+0000","last_clean":"2024-04-16T17:03:30.664287+0000","last_became_active":"2024-04-16T17:03:24.973568+0000","last_became_peered":"2024-04-16T17:03:24.973568+0000","last_unstale":"2024-04-16T17:03:30.664287+0000","last_undegraded":"2024-04-16T17:03:30.664287+0000","last_fullsized":"2024-04-16T17:03:30.664287+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:30:49.325469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:30.511469+0000","last_change":"2024-04-16T17:03:24.981629+0000","last_active":"2024-04-16T17:03:30.511469+0000","last_peered":"2024-04-16T17:03:30.511469+0000","last_clean":"2024-04-16T17:03:30.511469+0000","last_became_active":"2024-04-16T17:03:24.981111+0000","last_became_peered":"2024-04-16T17:03:24.981111+0000","last_unstale":"2024-04-16T17:03:30.511469+0000","last_undegraded":"2024-04-16T17:03:30.511469+0000","last_fullsized":"2024-04-16T17:03:30.511469+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:41:29.641610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:30.511218+0000","last_change":"2024-04-16T17:03:24.977890+0000","last_active":"2024-04-16T17:03:30.511218+0000","last_peered":"2024-04-16T17:03:30.511218+0000","last_clean":"2024-04-16T17:03:30.511218+0000","last_became_active":"2024-04-16T17:03:24.976500+0000","last_became_peered":"2024-04-16T17:03:24.976500+0000","last_unstale":"2024-04-16T17:03:30.511218+0000","last_undegraded":"2024-04-16T17:03:30.511218+0000","last_fullsized":"2024-04-16T17:03:30.511218+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:17:51.604701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:30.250242+0000","last_change":"2024-04-16T17:03:24.976085+0000","last_active":"2024-04-16T17:03:30.250242+0000","last_peered":"2024-04-16T17:03:30.250242+0000","last_clean":"2024-04-16T17:03:30.250242+0000","last_became_active":"2024-04-16T17:03:24.975804+0000","last_became_peered":"2024-04-16T17:03:24.975804+0000","last_unstale":"2024-04-16T17:03:30.250242+0000","last_undegraded":"2024-04-16T17:03:30.250242+0000","last_fullsized":"2024-04-16T17:03:30.250242+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:19:13.706889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:30.400057+0000","last_change":"2024-04-16T17:03:24.989456+0000","last_active":"2024-04-16T17:03:30.400057+0000","last_peered":"2024-04-16T17:03:30.400057+0000","last_clean":"2024-04-16T17:03:30.400057+0000","last_became_active":"2024-04-16T17:03:24.989161+0000","last_became_peered":"2024-04-16T17:03:24.989161+0000","last_unstale":"2024-04-16T17:03:30.400057+0000","last_undegraded":"2024-04-16T17:03:30.400057+0000","last_fullsized":"2024-04-16T17:03:30.400057+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:52:59.333625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:30.254501+0000","last_change":"2024-04-16T17:03:24.963428+0000","last_active":"2024-04-16T17:03:30.254501+0000","last_peered":"2024-04-16T17:03:30.254501+0000","last_clean":"2024-04-16T17:03:30.254501+0000","last_became_active":"2024-04-16T17:03:24.962523+0000","last_became_peered":"2024-04-16T17:03:24.962523+0000","last_unstale":"2024-04-16T17:03:30.254501+0000","last_undegraded":"2024-04-16T17:03:30.254501+0000","last_fullsized":"2024-04-16T17:03:30.254501+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:22:19.523901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:30.251382+0000","last_change":"2024-04-16T17:03:24.981122+0000","last_active":"2024-04-16T17:03:30.251382+0000","last_peered":"2024-04-16T17:03:30.251382+0000","last_clean":"2024-04-16T17:03:30.251382+0000","last_became_active":"2024-04-16T17:03:24.980533+0000","last_became_peered":"2024-04-16T17:03:24.980533+0000","last_unstale":"2024-04-16T17:03:30.251382+0000","last_undegraded":"2024-04-16T17:03:30.251382+0000","last_fullsized":"2024-04-16T17:03:30.251382+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:51:08.580965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:30.251470+0000","last_change":"2024-04-16T17:03:29.257239+0000","last_active":"2024-04-16T17:03:30.251470+0000","last_peered":"2024-04-16T17:03:30.251470+0000","last_clean":"2024-04-16T17:03:30.251470+0000","last_became_active":"2024-04-16T17:03:29.256827+0000","last_became_peered":"2024-04-16T17:03:29.256827+0000","last_unstale":"2024-04-16T17:03:30.251470+0000","last_undegraded":"2024-04-16T17:03:30.251470+0000","last_fullsized":"2024-04-16T17:03:30.251470+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:57:28.373386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"90'9","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:52.384026+0000","last_change":"2024-04-16T17:03:24.157157+0000","last_active":"2024-04-16T17:03:52.384026+0000","last_peered":"2024-04-16T17:03:52.384026+0000","last_clean":"2024-04-16T17:03:52.384026+0000","last_became_active":"2024-04-16T17:03:24.154813+0000","last_became_peered":"2024-04-16T17:03:24.154813+0000","last_unstale":"2024-04-16T17:03:52.384026+0000","last_undegraded":"2024-04-16T17:03:52.384026+0000","last_fullsized":"2024-04-16T17:03:52.384026+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:39:31.343583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:30.512100+0000","last_change":"2024-04-16T17:03:24.994921+0000","last_active":"2024-04-16T17:03:30.512100+0000","last_peered":"2024-04-16T17:03:30.512100+0000","last_clean":"2024-04-16T17:03:30.512100+0000","last_became_active":"2024-04-16T17:03:24.994412+0000","last_became_peered":"2024-04-16T17:03:24.994412+0000","last_unstale":"2024-04-16T17:03:30.512100+0000","last_undegraded":"2024-04-16T17:03:30.512100+0000","last_fullsized":"2024-04-16T17:03:30.512100+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:11:40.748198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:30.511269+0000","last_change":"2024-04-16T17:03:29.259884+0000","last_active":"2024-04-16T17:03:30.511269+0000","last_peered":"2024-04-16T17:03:30.511269+0000","last_clean":"2024-04-16T17:03:30.511269+0000","last_became_active":"2024-04-16T17:03:29.259319+0000","last_became_peered":"2024-04-16T17:03:29.259319+0000","last_unstale":"2024-04-16T17:03:30.511269+0000","last_undegraded":"2024-04-16T17:03:30.511269+0000","last_fullsized":"2024-04-16T17:03:30.511269+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:05:18.244682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"90'2","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:30.512508+0000","last_change":"2024-04-16T17:03:24.179308+0000","last_active":"2024-04-16T17:03:30.512508+0000","last_peered":"2024-04-16T17:03:30.512508+0000","last_clean":"2024-04-16T17:03:30.512508+0000","last_became_active":"2024-04-16T17:03:24.178894+0000","last_became_peered":"2024-04-16T17:03:24.178894+0000","last_unstale":"2024-04-16T17:03:30.512508+0000","last_undegraded":"2024-04-16T17:03:30.512508+0000","last_fullsized":"2024-04-16T17:03:30.512508+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:17:46.831871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:30.512411+0000","last_change":"2024-04-16T17:03:24.982391+0000","last_active":"2024-04-16T17:03:30.512411+0000","last_peered":"2024-04-16T17:03:30.512411+0000","last_clean":"2024-04-16T17:03:30.512411+0000","last_became_active":"2024-04-16T17:03:24.981761+0000","last_became_peered":"2024-04-16T17:03:24.981761+0000","last_unstale":"2024-04-16T17:03:30.512411+0000","last_undegraded":"2024-04-16T17:03:30.512411+0000","last_fullsized":"2024-04-16T17:03:30.512411+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:40:57.657170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:30.512302+0000","last_change":"2024-04-16T17:03:29.259705+0000","last_active":"2024-04-16T17:03:30.512302+0000","last_peered":"2024-04-16T17:03:30.512302+0000","last_clean":"2024-04-16T17:03:30.512302+0000","last_became_active":"2024-04-16T17:03:29.259336+0000","last_became_peered":"2024-04-16T17:03:29.259336+0000","last_unstale":"2024-04-16T17:03:30.512302+0000","last_undegraded":"2024-04-16T17:03:30.512302+0000","last_fullsized":"2024-04-16T17:03:30.512302+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:53:29.042933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"85'2","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:30.511835+0000","last_change":"2024-04-16T17:03:24.152641+0000","last_active":"2024-04-16T17:03:30.511835+0000","last_peered":"2024-04-16T17:03:30.511835+0000","last_clean":"2024-04-16T17:03:30.511835+0000","last_became_active":"2024-04-16T17:03:24.152310+0000","last_became_peered":"2024-04-16T17:03:24.152310+0000","last_unstale":"2024-04-16T17:03:30.511835+0000","last_undegraded":"2024-04-16T17:03:30.511835+0000","last_fullsized":"2024-04-16T17:03:30.511835+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:50:11.362075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:30.250552+0000","last_change":"2024-04-16T17:03:24.973205+0000","last_active":"2024-04-16T17:03:30.250552+0000","last_peered":"2024-04-16T17:03:30.250552+0000","last_clean":"2024-04-16T17:03:30.250552+0000","last_became_active":"2024-04-16T17:03:24.972863+0000","last_became_peered":"2024-04-16T17:03:24.972863+0000","last_unstale":"2024-04-16T17:03:30.250552+0000","last_undegraded":"2024-04-16T17:03:30.250552+0000","last_fullsized":"2024-04-16T17:03:30.250552+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:28:29.005783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:30.511362+0000","last_change":"2024-04-16T17:03:29.262930+0000","last_active":"2024-04-16T17:03:30.511362+0000","last_peered":"2024-04-16T17:03:30.511362+0000","last_clean":"2024-04-16T17:03:30.511362+0000","last_became_active":"2024-04-16T17:03:29.262621+0000","last_became_peered":"2024-04-16T17:03:29.262621+0000","last_unstale":"2024-04-16T17:03:30.511362+0000","last_undegraded":"2024-04-16T17:03:30.511362+0000","last_fullsized":"2024-04-16T17:03:30.511362+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:22:31.555143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:30.511245+0000","last_change":"2024-04-16T17:03:24.150503+0000","last_active":"2024-04-16T17:03:30.511245+0000","last_peered":"2024-04-16T17:03:30.511245+0000","last_clean":"2024-04-16T17:03:30.511245+0000","last_became_active":"2024-04-16T17:03:24.150021+0000","last_became_peered":"2024-04-16T17:03:24.150021+0000","last_unstale":"2024-04-16T17:03:30.511245+0000","last_undegraded":"2024-04-16T17:03:30.511245+0000","last_fullsized":"2024-04-16T17:03:30.511245+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:21:36.119497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:30.511870+0000","last_change":"2024-04-16T17:03:24.965049+0000","last_active":"2024-04-16T17:03:30.511870+0000","last_peered":"2024-04-16T17:03:30.511870+0000","last_clean":"2024-04-16T17:03:30.511870+0000","last_became_active":"2024-04-16T17:03:24.963120+0000","last_became_peered":"2024-04-16T17:03:24.963120+0000","last_unstale":"2024-04-16T17:03:30.511870+0000","last_undegraded":"2024-04-16T17:03:30.511870+0000","last_fullsized":"2024-04-16T17:03:30.511870+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:17:49.725113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:30.251263+0000","last_change":"2024-04-16T17:03:29.247013+0000","last_active":"2024-04-16T17:03:30.251263+0000","last_peered":"2024-04-16T17:03:30.251263+0000","last_clean":"2024-04-16T17:03:30.251263+0000","last_became_active":"2024-04-16T17:03:29.246477+0000","last_became_peered":"2024-04-16T17:03:29.246477+0000","last_unstale":"2024-04-16T17:03:30.251263+0000","last_undegraded":"2024-04-16T17:03:30.251263+0000","last_fullsized":"2024-04-16T17:03:30.251263+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:52:19.525076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"90'6","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:54.393619+0000","last_change":"2024-04-16T17:03:24.155333+0000","last_active":"2024-04-16T17:03:54.393619+0000","last_peered":"2024-04-16T17:03:54.393619+0000","last_clean":"2024-04-16T17:03:54.393619+0000","last_became_active":"2024-04-16T17:03:24.154848+0000","last_became_peered":"2024-04-16T17:03:24.154848+0000","last_unstale":"2024-04-16T17:03:54.393619+0000","last_undegraded":"2024-04-16T17:03:54.393619+0000","last_fullsized":"2024-04-16T17:03:54.393619+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:03:34.769573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:30.510981+0000","last_change":"2024-04-16T17:03:24.967627+0000","last_active":"2024-04-16T17:03:30.510981+0000","last_peered":"2024-04-16T17:03:30.510981+0000","last_clean":"2024-04-16T17:03:30.510981+0000","last_became_active":"2024-04-16T17:03:24.967264+0000","last_became_peered":"2024-04-16T17:03:24.967264+0000","last_unstale":"2024-04-16T17:03:30.510981+0000","last_undegraded":"2024-04-16T17:03:30.510981+0000","last_fullsized":"2024-04-16T17:03:30.510981+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:35:29.182374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:30.251205+0000","last_change":"2024-04-16T17:03:29.264625+0000","last_active":"2024-04-16T17:03:30.251205+0000","last_peered":"2024-04-16T17:03:30.251205+0000","last_clean":"2024-04-16T17:03:30.251205+0000","last_became_active":"2024-04-16T17:03:29.264114+0000","last_became_peered":"2024-04-16T17:03:29.264114+0000","last_unstale":"2024-04-16T17:03:30.251205+0000","last_undegraded":"2024-04-16T17:03:30.251205+0000","last_fullsized":"2024-04-16T17:03:30.251205+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:14:49.449980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"90'3","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:52.382001+0000","last_change":"2024-04-16T17:03:24.160457+0000","last_active":"2024-04-16T17:03:52.382001+0000","last_peered":"2024-04-16T17:03:52.382001+0000","last_clean":"2024-04-16T17:03:52.382001+0000","last_became_active":"2024-04-16T17:03:24.160195+0000","last_became_peered":"2024-04-16T17:03:24.160195+0000","last_unstale":"2024-04-16T17:03:52.382001+0000","last_undegraded":"2024-04-16T17:03:52.382001+0000","last_fullsized":"2024-04-16T17:03:52.382001+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:23:07.064539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:30.511680+0000","last_change":"2024-04-16T17:03:24.991249+0000","last_active":"2024-04-16T17:03:30.511680+0000","last_peered":"2024-04-16T17:03:30.511680+0000","last_clean":"2024-04-16T17:03:30.511680+0000","last_became_active":"2024-04-16T17:03:24.990927+0000","last_became_peered":"2024-04-16T17:03:24.990927+0000","last_unstale":"2024-04-16T17:03:30.511680+0000","last_undegraded":"2024-04-16T17:03:30.511680+0000","last_fullsized":"2024-04-16T17:03:30.511680+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:36:39.975616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:30.400144+0000","last_change":"2024-04-16T17:03:29.253704+0000","last_active":"2024-04-16T17:03:30.400144+0000","last_peered":"2024-04-16T17:03:30.400144+0000","last_clean":"2024-04-16T17:03:30.400144+0000","last_became_active":"2024-04-16T17:03:29.253334+0000","last_became_peered":"2024-04-16T17:03:29.253334+0000","last_unstale":"2024-04-16T17:03:30.400144+0000","last_undegraded":"2024-04-16T17:03:30.400144+0000","last_fullsized":"2024-04-16T17:03:30.400144+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:19:12.611731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"90'2","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:52.386383+0000","last_change":"2024-04-16T17:03:24.158875+0000","last_active":"2024-04-16T17:03:52.386383+0000","last_peered":"2024-04-16T17:03:52.386383+0000","last_clean":"2024-04-16T17:03:52.386383+0000","last_became_active":"2024-04-16T17:03:24.158465+0000","last_became_peered":"2024-04-16T17:03:24.158465+0000","last_unstale":"2024-04-16T17:03:52.386383+0000","last_undegraded":"2024-04-16T17:03:52.386383+0000","last_fullsized":"2024-04-16T17:03:52.386383+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:21:29.882251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:30.511047+0000","last_change":"2024-04-16T17:03:24.994476+0000","last_active":"2024-04-16T17:03:30.511047+0000","last_peered":"2024-04-16T17:03:30.511047+0000","last_clean":"2024-04-16T17:03:30.511047+0000","last_became_active":"2024-04-16T17:03:24.993532+0000","last_became_peered":"2024-04-16T17:03:24.993532+0000","last_unstale":"2024-04-16T17:03:30.511047+0000","last_undegraded":"2024-04-16T17:03:30.511047+0000","last_fullsized":"2024-04-16T17:03:30.511047+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:04:44.062444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:30.511983+0000","last_change":"2024-04-16T17:03:29.255446+0000","last_active":"2024-04-16T17:03:30.511983+0000","last_peered":"2024-04-16T17:03:30.511983+0000","last_clean":"2024-04-16T17:03:30.511983+0000","last_became_active":"2024-04-16T17:03:29.255051+0000","last_became_peered":"2024-04-16T17:03:29.255051+0000","last_unstale":"2024-04-16T17:03:30.511983+0000","last_undegraded":"2024-04-16T17:03:30.511983+0000","last_fullsized":"2024-04-16T17:03:30.511983+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:24:04.946342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"90'1","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:30.511394+0000","last_change":"2024-04-16T17:03:24.154904+0000","last_active":"2024-04-16T17:03:30.511394+0000","last_peered":"2024-04-16T17:03:30.511394+0000","last_clean":"2024-04-16T17:03:30.511394+0000","last_became_active":"2024-04-16T17:03:24.154617+0000","last_became_peered":"2024-04-16T17:03:24.154617+0000","last_unstale":"2024-04-16T17:03:30.511394+0000","last_undegraded":"2024-04-16T17:03:30.511394+0000","last_fullsized":"2024-04-16T17:03:30.511394+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:35:55.498855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:30.253013+0000","last_change":"2024-04-16T17:03:24.979908+0000","last_active":"2024-04-16T17:03:30.253013+0000","last_peered":"2024-04-16T17:03:30.253013+0000","last_clean":"2024-04-16T17:03:30.253013+0000","last_became_active":"2024-04-16T17:03:24.979501+0000","last_became_peered":"2024-04-16T17:03:24.979501+0000","last_unstale":"2024-04-16T17:03:30.253013+0000","last_undegraded":"2024-04-16T17:03:30.253013+0000","last_fullsized":"2024-04-16T17:03:30.253013+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:51:05.421459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:30.511352+0000","last_change":"2024-04-16T17:03:29.258809+0000","last_active":"2024-04-16T17:03:30.511352+0000","last_peered":"2024-04-16T17:03:30.511352+0000","last_clean":"2024-04-16T17:03:30.511352+0000","last_became_active":"2024-04-16T17:03:29.256629+0000","last_became_peered":"2024-04-16T17:03:29.256629+0000","last_unstale":"2024-04-16T17:03:30.511352+0000","last_undegraded":"2024-04-16T17:03:30.511352+0000","last_fullsized":"2024-04-16T17:03:30.511352+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:46:51.634944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"90'3","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:51.374270+0000","last_change":"2024-04-16T17:03:24.153774+0000","last_active":"2024-04-16T17:03:51.374270+0000","last_peered":"2024-04-16T17:03:51.374270+0000","last_clean":"2024-04-16T17:03:51.374270+0000","last_became_active":"2024-04-16T17:03:24.153418+0000","last_became_peered":"2024-04-16T17:03:24.153418+0000","last_unstale":"2024-04-16T17:03:51.374270+0000","last_undegraded":"2024-04-16T17:03:51.374270+0000","last_fullsized":"2024-04-16T17:03:51.374270+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:58:36.093789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:30.251009+0000","last_change":"2024-04-16T17:03:24.962838+0000","last_active":"2024-04-16T17:03:30.251009+0000","last_peered":"2024-04-16T17:03:30.251009+0000","last_clean":"2024-04-16T17:03:30.251009+0000","last_became_active":"2024-04-16T17:03:24.960163+0000","last_became_peered":"2024-04-16T17:03:24.960163+0000","last_unstale":"2024-04-16T17:03:30.251009+0000","last_undegraded":"2024-04-16T17:03:30.251009+0000","last_fullsized":"2024-04-16T17:03:30.251009+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:48:18.723499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:30.253053+0000","last_change":"2024-04-16T17:03:29.251177+0000","last_active":"2024-04-16T17:03:30.253053+0000","last_peered":"2024-04-16T17:03:30.253053+0000","last_clean":"2024-04-16T17:03:30.253053+0000","last_became_active":"2024-04-16T17:03:29.250657+0000","last_became_peered":"2024-04-16T17:03:29.250657+0000","last_unstale":"2024-04-16T17:03:30.253053+0000","last_undegraded":"2024-04-16T17:03:30.253053+0000","last_fullsized":"2024-04-16T17:03:30.253053+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:16:47.217193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"90'2","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:51.372780+0000","last_change":"2024-04-16T17:03:24.154245+0000","last_active":"2024-04-16T17:03:51.372780+0000","last_peered":"2024-04-16T17:03:51.372780+0000","last_clean":"2024-04-16T17:03:51.372780+0000","last_became_active":"2024-04-16T17:03:24.153297+0000","last_became_peered":"2024-04-16T17:03:24.153297+0000","last_unstale":"2024-04-16T17:03:51.372780+0000","last_undegraded":"2024-04-16T17:03:51.372780+0000","last_fullsized":"2024-04-16T17:03:51.372780+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:23:49.490238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:30.512505+0000","last_change":"2024-04-16T17:03:24.984572+0000","last_active":"2024-04-16T17:03:30.512505+0000","last_peered":"2024-04-16T17:03:30.512505+0000","last_clean":"2024-04-16T17:03:30.512505+0000","last_became_active":"2024-04-16T17:03:24.983588+0000","last_became_peered":"2024-04-16T17:03:24.983588+0000","last_unstale":"2024-04-16T17:03:30.512505+0000","last_undegraded":"2024-04-16T17:03:30.512505+0000","last_fullsized":"2024-04-16T17:03:30.512505+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:41:24.704602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:30.512717+0000","last_change":"2024-04-16T17:03:29.245129+0000","last_active":"2024-04-16T17:03:30.512717+0000","last_peered":"2024-04-16T17:03:30.512717+0000","last_clean":"2024-04-16T17:03:30.512717+0000","last_became_active":"2024-04-16T17:03:29.244720+0000","last_became_peered":"2024-04-16T17:03:29.244720+0000","last_unstale":"2024-04-16T17:03:30.512717+0000","last_undegraded":"2024-04-16T17:03:30.512717+0000","last_fullsized":"2024-04-16T17:03:30.512717+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:55:26.612176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"90'4","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:51.372651+0000","last_change":"2024-04-16T17:03:24.165248+0000","last_active":"2024-04-16T17:03:51.372651+0000","last_peered":"2024-04-16T17:03:51.372651+0000","last_clean":"2024-04-16T17:03:51.372651+0000","last_became_active":"2024-04-16T17:03:24.164915+0000","last_became_peered":"2024-04-16T17:03:24.164915+0000","last_unstale":"2024-04-16T17:03:51.372651+0000","last_undegraded":"2024-04-16T17:03:51.372651+0000","last_fullsized":"2024-04-16T17:03:51.372651+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:04:47.207554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:30.250692+0000","last_change":"2024-04-16T17:03:24.967292+0000","last_active":"2024-04-16T17:03:30.250692+0000","last_peered":"2024-04-16T17:03:30.250692+0000","last_clean":"2024-04-16T17:03:30.250692+0000","last_became_active":"2024-04-16T17:03:24.966952+0000","last_became_peered":"2024-04-16T17:03:24.966952+0000","last_unstale":"2024-04-16T17:03:30.250692+0000","last_undegraded":"2024-04-16T17:03:30.250692+0000","last_fullsized":"2024-04-16T17:03:30.250692+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:42:54.969478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:30.254085+0000","last_change":"2024-04-16T17:03:29.257836+0000","last_active":"2024-04-16T17:03:30.254085+0000","last_peered":"2024-04-16T17:03:30.254085+0000","last_clean":"2024-04-16T17:03:30.254085+0000","last_became_active":"2024-04-16T17:03:29.257409+0000","last_became_peered":"2024-04-16T17:03:29.257409+0000","last_unstale":"2024-04-16T17:03:30.254085+0000","last_undegraded":"2024-04-16T17:03:30.254085+0000","last_fullsized":"2024-04-16T17:03:30.254085+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:25:21.288266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"90'15","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:53.902286+0000","last_change":"2024-04-16T17:03:24.150943+0000","last_active":"2024-04-16T17:03:53.902286+0000","last_peered":"2024-04-16T17:03:53.902286+0000","last_clean":"2024-04-16T17:03:53.902286+0000","last_became_active":"2024-04-16T17:03:24.150563+0000","last_became_peered":"2024-04-16T17:03:24.150563+0000","last_unstale":"2024-04-16T17:03:53.902286+0000","last_undegraded":"2024-04-16T17:03:53.902286+0000","last_fullsized":"2024-04-16T17:03:53.902286+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:05:07.380440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10442,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":6,"num_write":16,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:30.511933+0000","last_change":"2024-04-16T17:03:24.966393+0000","last_active":"2024-04-16T17:03:30.511933+0000","last_peered":"2024-04-16T17:03:30.511933+0000","last_clean":"2024-04-16T17:03:30.511933+0000","last_became_active":"2024-04-16T17:03:24.966016+0000","last_became_peered":"2024-04-16T17:03:24.966016+0000","last_unstale":"2024-04-16T17:03:30.511933+0000","last_undegraded":"2024-04-16T17:03:30.511933+0000","last_fullsized":"2024-04-16T17:03:30.511933+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:30:25.203262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:30.512504+0000","last_change":"2024-04-16T17:03:29.265371+0000","last_active":"2024-04-16T17:03:30.512504+0000","last_peered":"2024-04-16T17:03:30.512504+0000","last_clean":"2024-04-16T17:03:30.512504+0000","last_became_active":"2024-04-16T17:03:29.264933+0000","last_became_peered":"2024-04-16T17:03:29.264933+0000","last_unstale":"2024-04-16T17:03:30.512504+0000","last_undegraded":"2024-04-16T17:03:30.512504+0000","last_fullsized":"2024-04-16T17:03:30.512504+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:50:18.203015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"90'6","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:52.380772+0000","last_change":"2024-04-16T17:03:24.166899+0000","last_active":"2024-04-16T17:03:52.380772+0000","last_peered":"2024-04-16T17:03:52.380772+0000","last_clean":"2024-04-16T17:03:52.380772+0000","last_became_active":"2024-04-16T17:03:24.166529+0000","last_became_peered":"2024-04-16T17:03:24.166529+0000","last_unstale":"2024-04-16T17:03:52.380772+0000","last_undegraded":"2024-04-16T17:03:52.380772+0000","last_fullsized":"2024-04-16T17:03:52.380772+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:59:07.156150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:30.663656+0000","last_change":"2024-04-16T17:03:24.986228+0000","last_active":"2024-04-16T17:03:30.663656+0000","last_peered":"2024-04-16T17:03:30.663656+0000","last_clean":"2024-04-16T17:03:30.663656+0000","last_became_active":"2024-04-16T17:03:24.985916+0000","last_became_peered":"2024-04-16T17:03:24.985916+0000","last_unstale":"2024-04-16T17:03:30.663656+0000","last_undegraded":"2024-04-16T17:03:30.663656+0000","last_fullsized":"2024-04-16T17:03:30.663656+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:23:54.309203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:30.400136+0000","last_change":"2024-04-16T17:03:29.264636+0000","last_active":"2024-04-16T17:03:30.400136+0000","last_peered":"2024-04-16T17:03:30.400136+0000","last_clean":"2024-04-16T17:03:30.400136+0000","last_became_active":"2024-04-16T17:03:29.264117+0000","last_became_peered":"2024-04-16T17:03:29.264117+0000","last_unstale":"2024-04-16T17:03:30.400136+0000","last_undegraded":"2024-04-16T17:03:30.400136+0000","last_fullsized":"2024-04-16T17:03:30.400136+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:42:36.123154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"90'3","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:51.375761+0000","last_change":"2024-04-16T17:03:24.166844+0000","last_active":"2024-04-16T17:03:51.375761+0000","last_peered":"2024-04-16T17:03:51.375761+0000","last_clean":"2024-04-16T17:03:51.375761+0000","last_became_active":"2024-04-16T17:03:24.166479+0000","last_became_peered":"2024-04-16T17:03:24.166479+0000","last_unstale":"2024-04-16T17:03:51.375761+0000","last_undegraded":"2024-04-16T17:03:51.375761+0000","last_fullsized":"2024-04-16T17:03:51.375761+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:02:20.139865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:30.512599+0000","last_change":"2024-04-16T17:03:24.973057+0000","last_active":"2024-04-16T17:03:30.512599+0000","last_peered":"2024-04-16T17:03:30.512599+0000","last_clean":"2024-04-16T17:03:30.512599+0000","last_became_active":"2024-04-16T17:03:24.972731+0000","last_became_peered":"2024-04-16T17:03:24.972731+0000","last_unstale":"2024-04-16T17:03:30.512599+0000","last_undegraded":"2024-04-16T17:03:30.512599+0000","last_fullsized":"2024-04-16T17:03:30.512599+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:37:38.702919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:30.253798+0000","last_change":"2024-04-16T17:03:29.259167+0000","last_active":"2024-04-16T17:03:30.253798+0000","last_peered":"2024-04-16T17:03:30.253798+0000","last_clean":"2024-04-16T17:03:30.253798+0000","last_became_active":"2024-04-16T17:03:29.258737+0000","last_became_peered":"2024-04-16T17:03:29.258737+0000","last_unstale":"2024-04-16T17:03:30.253798+0000","last_undegraded":"2024-04-16T17:03:30.253798+0000","last_fullsized":"2024-04-16T17:03:30.253798+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:01:13.774037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:30.511357+0000","last_change":"2024-04-16T17:03:24.151064+0000","last_active":"2024-04-16T17:03:30.511357+0000","last_peered":"2024-04-16T17:03:30.511357+0000","last_clean":"2024-04-16T17:03:30.511357+0000","last_became_active":"2024-04-16T17:03:24.150745+0000","last_became_peered":"2024-04-16T17:03:24.150745+0000","last_unstale":"2024-04-16T17:03:30.511357+0000","last_undegraded":"2024-04-16T17:03:30.511357+0000","last_fullsized":"2024-04-16T17:03:30.511357+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:32:26.271008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:30.511652+0000","last_change":"2024-04-16T17:03:24.971948+0000","last_active":"2024-04-16T17:03:30.511652+0000","last_peered":"2024-04-16T17:03:30.511652+0000","last_clean":"2024-04-16T17:03:30.511652+0000","last_became_active":"2024-04-16T17:03:24.971530+0000","last_became_peered":"2024-04-16T17:03:24.971530+0000","last_unstale":"2024-04-16T17:03:30.511652+0000","last_undegraded":"2024-04-16T17:03:30.511652+0000","last_fullsized":"2024-04-16T17:03:30.511652+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:03:29.286160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:30.665300+0000","last_change":"2024-04-16T17:03:29.264821+0000","last_active":"2024-04-16T17:03:30.665300+0000","last_peered":"2024-04-16T17:03:30.665300+0000","last_clean":"2024-04-16T17:03:30.665300+0000","last_became_active":"2024-04-16T17:03:29.264287+0000","last_became_peered":"2024-04-16T17:03:29.264287+0000","last_unstale":"2024-04-16T17:03:30.665300+0000","last_undegraded":"2024-04-16T17:03:30.665300+0000","last_fullsized":"2024-04-16T17:03:30.665300+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:33:28.400663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"90'10","reported_seq":33,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:51.372926+0000","last_change":"2024-04-16T17:03:24.158844+0000","last_active":"2024-04-16T17:03:51.372926+0000","last_peered":"2024-04-16T17:03:51.372926+0000","last_clean":"2024-04-16T17:03:51.372926+0000","last_became_active":"2024-04-16T17:03:24.158537+0000","last_became_peered":"2024-04-16T17:03:24.158537+0000","last_unstale":"2024-04-16T17:03:51.372926+0000","last_undegraded":"2024-04-16T17:03:51.372926+0000","last_fullsized":"2024-04-16T17:03:51.372926+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:42:22.772452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:30.511509+0000","last_change":"2024-04-16T17:03:24.962522+0000","last_active":"2024-04-16T17:03:30.511509+0000","last_peered":"2024-04-16T17:03:30.511509+0000","last_clean":"2024-04-16T17:03:30.511509+0000","last_became_active":"2024-04-16T17:03:24.962169+0000","last_became_peered":"2024-04-16T17:03:24.962169+0000","last_unstale":"2024-04-16T17:03:30.511509+0000","last_undegraded":"2024-04-16T17:03:30.511509+0000","last_fullsized":"2024-04-16T17:03:30.511509+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:28:17.147491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:30.511508+0000","last_change":"2024-04-16T17:03:29.259539+0000","last_active":"2024-04-16T17:03:30.511508+0000","last_peered":"2024-04-16T17:03:30.511508+0000","last_clean":"2024-04-16T17:03:30.511508+0000","last_became_active":"2024-04-16T17:03:29.259123+0000","last_became_peered":"2024-04-16T17:03:29.259123+0000","last_unstale":"2024-04-16T17:03:30.511508+0000","last_undegraded":"2024-04-16T17:03:30.511508+0000","last_fullsized":"2024-04-16T17:03:30.511508+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:15:17.142636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"85'1","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:30.511961+0000","last_change":"2024-04-16T17:03:24.147526+0000","last_active":"2024-04-16T17:03:30.511961+0000","last_peered":"2024-04-16T17:03:30.511961+0000","last_clean":"2024-04-16T17:03:30.511961+0000","last_became_active":"2024-04-16T17:03:24.147119+0000","last_became_peered":"2024-04-16T17:03:24.147119+0000","last_unstale":"2024-04-16T17:03:30.511961+0000","last_undegraded":"2024-04-16T17:03:30.511961+0000","last_fullsized":"2024-04-16T17:03:30.511961+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:30:38.742122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:30.400101+0000","last_change":"2024-04-16T17:03:24.970179+0000","last_active":"2024-04-16T17:03:30.400101+0000","last_peered":"2024-04-16T17:03:30.400101+0000","last_clean":"2024-04-16T17:03:30.400101+0000","last_became_active":"2024-04-16T17:03:24.969819+0000","last_became_peered":"2024-04-16T17:03:24.969819+0000","last_unstale":"2024-04-16T17:03:30.400101+0000","last_undegraded":"2024-04-16T17:03:30.400101+0000","last_fullsized":"2024-04-16T17:03:30.400101+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:40:19.897034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:30.511208+0000","last_change":"2024-04-16T17:03:29.301135+0000","last_active":"2024-04-16T17:03:30.511208+0000","last_peered":"2024-04-16T17:03:30.511208+0000","last_clean":"2024-04-16T17:03:30.511208+0000","last_became_active":"2024-04-16T17:03:29.300672+0000","last_became_peered":"2024-04-16T17:03:29.300672+0000","last_unstale":"2024-04-16T17:03:30.511208+0000","last_undegraded":"2024-04-16T17:03:30.511208+0000","last_fullsized":"2024-04-16T17:03:30.511208+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:55:41.599984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:30.254270+0000","last_change":"2024-04-16T17:03:24.162105+0000","last_active":"2024-04-16T17:03:30.254270+0000","last_peered":"2024-04-16T17:03:30.254270+0000","last_clean":"2024-04-16T17:03:30.254270+0000","last_became_active":"2024-04-16T17:03:24.161631+0000","last_became_peered":"2024-04-16T17:03:24.161631+0000","last_unstale":"2024-04-16T17:03:30.254270+0000","last_undegraded":"2024-04-16T17:03:30.254270+0000","last_fullsized":"2024-04-16T17:03:30.254270+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:22:33.677848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:30.511660+0000","last_change":"2024-04-16T17:03:24.978253+0000","last_active":"2024-04-16T17:03:30.511660+0000","last_peered":"2024-04-16T17:03:30.511660+0000","last_clean":"2024-04-16T17:03:30.511660+0000","last_became_active":"2024-04-16T17:03:24.977956+0000","last_became_peered":"2024-04-16T17:03:24.977956+0000","last_unstale":"2024-04-16T17:03:30.511660+0000","last_undegraded":"2024-04-16T17:03:30.511660+0000","last_fullsized":"2024-04-16T17:03:30.511660+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:59:45.102383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:30.664893+0000","last_change":"2024-04-16T17:03:29.265312+0000","last_active":"2024-04-16T17:03:30.664893+0000","last_peered":"2024-04-16T17:03:30.664893+0000","last_clean":"2024-04-16T17:03:30.664893+0000","last_became_active":"2024-04-16T17:03:29.264891+0000","last_became_peered":"2024-04-16T17:03:29.264891+0000","last_unstale":"2024-04-16T17:03:30.664893+0000","last_undegraded":"2024-04-16T17:03:30.664893+0000","last_fullsized":"2024-04-16T17:03:30.664893+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:26:40.088127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"90'1","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:30.511321+0000","last_change":"2024-04-16T17:03:24.170935+0000","last_active":"2024-04-16T17:03:30.511321+0000","last_peered":"2024-04-16T17:03:30.511321+0000","last_clean":"2024-04-16T17:03:30.511321+0000","last_became_active":"2024-04-16T17:03:24.170609+0000","last_became_peered":"2024-04-16T17:03:24.170609+0000","last_unstale":"2024-04-16T17:03:30.511321+0000","last_undegraded":"2024-04-16T17:03:30.511321+0000","last_fullsized":"2024-04-16T17:03:30.511321+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:18:33.516309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:30.511334+0000","last_change":"2024-04-16T17:03:24.985434+0000","last_active":"2024-04-16T17:03:30.511334+0000","last_peered":"2024-04-16T17:03:30.511334+0000","last_clean":"2024-04-16T17:03:30.511334+0000","last_became_active":"2024-04-16T17:03:24.984388+0000","last_became_peered":"2024-04-16T17:03:24.984388+0000","last_unstale":"2024-04-16T17:03:30.511334+0000","last_undegraded":"2024-04-16T17:03:30.511334+0000","last_fullsized":"2024-04-16T17:03:30.511334+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:43:26.355572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:30.664876+0000","last_change":"2024-04-16T17:03:29.301455+0000","last_active":"2024-04-16T17:03:30.664876+0000","last_peered":"2024-04-16T17:03:30.664876+0000","last_clean":"2024-04-16T17:03:30.664876+0000","last_became_active":"2024-04-16T17:03:29.301028+0000","last_became_peered":"2024-04-16T17:03:29.301028+0000","last_unstale":"2024-04-16T17:03:30.664876+0000","last_undegraded":"2024-04-16T17:03:30.664876+0000","last_fullsized":"2024-04-16T17:03:30.664876+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:54:49.305379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:30.511787+0000","last_change":"2024-04-16T17:03:24.962786+0000","last_active":"2024-04-16T17:03:30.511787+0000","last_peered":"2024-04-16T17:03:30.511787+0000","last_clean":"2024-04-16T17:03:30.511787+0000","last_became_active":"2024-04-16T17:03:24.961704+0000","last_became_peered":"2024-04-16T17:03:24.961704+0000","last_unstale":"2024-04-16T17:03:30.511787+0000","last_undegraded":"2024-04-16T17:03:30.511787+0000","last_fullsized":"2024-04-16T17:03:30.511787+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:22:27.957607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"90'4","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:54.394175+0000","last_change":"2024-04-16T17:03:24.175617+0000","last_active":"2024-04-16T17:03:54.394175+0000","last_peered":"2024-04-16T17:03:54.394175+0000","last_clean":"2024-04-16T17:03:54.394175+0000","last_became_active":"2024-04-16T17:03:24.175272+0000","last_became_peered":"2024-04-16T17:03:24.175272+0000","last_unstale":"2024-04-16T17:03:54.394175+0000","last_undegraded":"2024-04-16T17:03:54.394175+0000","last_fullsized":"2024-04-16T17:03:54.394175+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:27:44.738030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:30.512775+0000","last_change":"2024-04-16T17:03:29.243467+0000","last_active":"2024-04-16T17:03:30.512775+0000","last_peered":"2024-04-16T17:03:30.512775+0000","last_clean":"2024-04-16T17:03:30.512775+0000","last_became_active":"2024-04-16T17:03:29.242746+0000","last_became_peered":"2024-04-16T17:03:29.242746+0000","last_unstale":"2024-04-16T17:03:30.512775+0000","last_undegraded":"2024-04-16T17:03:30.512775+0000","last_fullsized":"2024-04-16T17:03:30.512775+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:43:23.893380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:30.511885+0000","last_change":"2024-04-16T17:03:24.983629+0000","last_active":"2024-04-16T17:03:30.511885+0000","last_peered":"2024-04-16T17:03:30.511885+0000","last_clean":"2024-04-16T17:03:30.511885+0000","last_became_active":"2024-04-16T17:03:24.979969+0000","last_became_peered":"2024-04-16T17:03:24.979969+0000","last_unstale":"2024-04-16T17:03:30.511885+0000","last_undegraded":"2024-04-16T17:03:30.511885+0000","last_fullsized":"2024-04-16T17:03:30.511885+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:01:24.391732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"90'2","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:51.374054+0000","last_change":"2024-04-16T17:03:24.152475+0000","last_active":"2024-04-16T17:03:51.374054+0000","last_peered":"2024-04-16T17:03:51.374054+0000","last_clean":"2024-04-16T17:03:51.374054+0000","last_became_active":"2024-04-16T17:03:24.152198+0000","last_became_peered":"2024-04-16T17:03:24.152198+0000","last_unstale":"2024-04-16T17:03:51.374054+0000","last_undegraded":"2024-04-16T17:03:51.374054+0000","last_fullsized":"2024-04-16T17:03:51.374054+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:49:33.914325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":355,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:30.511441+0000","last_change":"2024-04-16T17:02:02.111335+0000","last_active":"2024-04-16T17:03:30.511441+0000","last_peered":"2024-04-16T17:03:30.511441+0000","last_clean":"2024-04-16T17:03:30.511441+0000","last_became_active":"2024-04-16T17:02:02.110928+0000","last_became_peered":"2024-04-16T17:02:02.110928+0000","last_unstale":"2024-04-16T17:03:30.511441+0000","last_undegraded":"2024-04-16T17:03:30.511441+0000","last_fullsized":"2024-04-16T17:03:30.511441+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T16:58:07.081753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T16:58:07.081753+0000","last_clean_scrub_stamp":"2024-04-16T16:58:07.081753+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:27:02.865664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:30.511786+0000","last_change":"2024-04-16T17:03:29.301691+0000","last_active":"2024-04-16T17:03:30.511786+0000","last_peered":"2024-04-16T17:03:30.511786+0000","last_clean":"2024-04-16T17:03:30.511786+0000","last_became_active":"2024-04-16T17:03:29.301277+0000","last_became_peered":"2024-04-16T17:03:29.301277+0000","last_unstale":"2024-04-16T17:03:30.511786+0000","last_undegraded":"2024-04-16T17:03:30.511786+0000","last_fullsized":"2024-04-16T17:03:30.511786+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:45:16.727610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:30.663231+0000","last_change":"2024-04-16T17:03:24.970939+0000","last_active":"2024-04-16T17:03:30.663231+0000","last_peered":"2024-04-16T17:03:30.663231+0000","last_clean":"2024-04-16T17:03:30.663231+0000","last_became_active":"2024-04-16T17:03:24.970581+0000","last_became_peered":"2024-04-16T17:03:24.970581+0000","last_unstale":"2024-04-16T17:03:30.663231+0000","last_undegraded":"2024-04-16T17:03:30.663231+0000","last_fullsized":"2024-04-16T17:03:30.663231+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:33:07.448597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"90'3","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:52.378753+0000","last_change":"2024-04-16T17:03:24.155454+0000","last_active":"2024-04-16T17:03:52.378753+0000","last_peered":"2024-04-16T17:03:52.378753+0000","last_clean":"2024-04-16T17:03:52.378753+0000","last_became_active":"2024-04-16T17:03:24.155155+0000","last_became_peered":"2024-04-16T17:03:24.155155+0000","last_unstale":"2024-04-16T17:03:52.378753+0000","last_undegraded":"2024-04-16T17:03:52.378753+0000","last_fullsized":"2024-04-16T17:03:52.378753+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:34:08.618002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:30.400042+0000","last_change":"2024-04-16T17:03:29.255538+0000","last_active":"2024-04-16T17:03:30.400042+0000","last_peered":"2024-04-16T17:03:30.400042+0000","last_clean":"2024-04-16T17:03:30.400042+0000","last_became_active":"2024-04-16T17:03:29.255107+0000","last_became_peered":"2024-04-16T17:03:29.255107+0000","last_unstale":"2024-04-16T17:03:30.400042+0000","last_undegraded":"2024-04-16T17:03:30.400042+0000","last_fullsized":"2024-04-16T17:03:30.400042+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:31:58.458641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"90'1","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:30.511711+0000","last_change":"2024-04-16T17:03:24.158816+0000","last_active":"2024-04-16T17:03:30.511711+0000","last_peered":"2024-04-16T17:03:30.511711+0000","last_clean":"2024-04-16T17:03:30.511711+0000","last_became_active":"2024-04-16T17:03:24.158518+0000","last_became_peered":"2024-04-16T17:03:24.158518+0000","last_unstale":"2024-04-16T17:03:30.511711+0000","last_undegraded":"2024-04-16T17:03:30.511711+0000","last_fullsized":"2024-04-16T17:03:30.511711+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:54:55.807975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:30.664497+0000","last_change":"2024-04-16T17:03:24.977190+0000","last_active":"2024-04-16T17:03:30.664497+0000","last_peered":"2024-04-16T17:03:30.664497+0000","last_clean":"2024-04-16T17:03:30.664497+0000","last_became_active":"2024-04-16T17:03:24.976840+0000","last_became_peered":"2024-04-16T17:03:24.976840+0000","last_unstale":"2024-04-16T17:03:30.664497+0000","last_undegraded":"2024-04-16T17:03:30.664497+0000","last_fullsized":"2024-04-16T17:03:30.664497+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:21:51.294150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:30.511059+0000","last_change":"2024-04-16T17:03:29.301423+0000","last_active":"2024-04-16T17:03:30.511059+0000","last_peered":"2024-04-16T17:03:30.511059+0000","last_clean":"2024-04-16T17:03:30.511059+0000","last_became_active":"2024-04-16T17:03:29.300888+0000","last_became_peered":"2024-04-16T17:03:29.300888+0000","last_unstale":"2024-04-16T17:03:30.511059+0000","last_undegraded":"2024-04-16T17:03:30.511059+0000","last_fullsized":"2024-04-16T17:03:30.511059+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:13:54.188090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"90'3","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:52.386116+0000","last_change":"2024-04-16T17:03:24.167643+0000","last_active":"2024-04-16T17:03:52.386116+0000","last_peered":"2024-04-16T17:03:52.386116+0000","last_clean":"2024-04-16T17:03:52.386116+0000","last_became_active":"2024-04-16T17:03:24.167103+0000","last_became_peered":"2024-04-16T17:03:24.167103+0000","last_unstale":"2024-04-16T17:03:52.386116+0000","last_undegraded":"2024-04-16T17:03:52.386116+0000","last_fullsized":"2024-04-16T17:03:52.386116+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:33:16.809371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:30.512376+0000","last_change":"2024-04-16T17:03:24.976981+0000","last_active":"2024-04-16T17:03:30.512376+0000","last_peered":"2024-04-16T17:03:30.512376+0000","last_clean":"2024-04-16T17:03:30.512376+0000","last_became_active":"2024-04-16T17:03:24.975385+0000","last_became_peered":"2024-04-16T17:03:24.975385+0000","last_unstale":"2024-04-16T17:03:30.512376+0000","last_undegraded":"2024-04-16T17:03:30.512376+0000","last_fullsized":"2024-04-16T17:03:30.512376+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:27:03.104838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:30.250799+0000","last_change":"2024-04-16T17:03:29.244767+0000","last_active":"2024-04-16T17:03:30.250799+0000","last_peered":"2024-04-16T17:03:30.250799+0000","last_clean":"2024-04-16T17:03:30.250799+0000","last_became_active":"2024-04-16T17:03:29.244205+0000","last_became_peered":"2024-04-16T17:03:29.244205+0000","last_unstale":"2024-04-16T17:03:30.250799+0000","last_undegraded":"2024-04-16T17:03:30.250799+0000","last_fullsized":"2024-04-16T17:03:30.250799+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:51:46.652387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"90'4","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:51.372155+0000","last_change":"2024-04-16T17:03:24.164916+0000","last_active":"2024-04-16T17:03:51.372155+0000","last_peered":"2024-04-16T17:03:51.372155+0000","last_clean":"2024-04-16T17:03:51.372155+0000","last_became_active":"2024-04-16T17:03:24.164648+0000","last_became_peered":"2024-04-16T17:03:24.164648+0000","last_unstale":"2024-04-16T17:03:51.372155+0000","last_undegraded":"2024-04-16T17:03:51.372155+0000","last_fullsized":"2024-04-16T17:03:51.372155+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:23:00.198785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:30.512143+0000","last_change":"2024-04-16T17:03:24.994393+0000","last_active":"2024-04-16T17:03:30.512143+0000","last_peered":"2024-04-16T17:03:30.512143+0000","last_clean":"2024-04-16T17:03:30.512143+0000","last_became_active":"2024-04-16T17:03:24.994002+0000","last_became_peered":"2024-04-16T17:03:24.994002+0000","last_unstale":"2024-04-16T17:03:30.512143+0000","last_undegraded":"2024-04-16T17:03:30.512143+0000","last_fullsized":"2024-04-16T17:03:30.512143+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:23:44.596781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:30.512679+0000","last_change":"2024-04-16T17:03:29.302009+0000","last_active":"2024-04-16T17:03:30.512679+0000","last_peered":"2024-04-16T17:03:30.512679+0000","last_clean":"2024-04-16T17:03:30.512679+0000","last_became_active":"2024-04-16T17:03:29.301183+0000","last_became_peered":"2024-04-16T17:03:29.301183+0000","last_unstale":"2024-04-16T17:03:30.512679+0000","last_undegraded":"2024-04-16T17:03:30.512679+0000","last_fullsized":"2024-04-16T17:03:30.512679+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:30:19.561875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"90'2","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:51.373227+0000","last_change":"2024-04-16T17:03:24.176829+0000","last_active":"2024-04-16T17:03:51.373227+0000","last_peered":"2024-04-16T17:03:51.373227+0000","last_clean":"2024-04-16T17:03:51.373227+0000","last_became_active":"2024-04-16T17:03:24.176555+0000","last_became_peered":"2024-04-16T17:03:24.176555+0000","last_unstale":"2024-04-16T17:03:51.373227+0000","last_undegraded":"2024-04-16T17:03:51.373227+0000","last_fullsized":"2024-04-16T17:03:51.373227+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:05:40.602150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:30.510743+0000","last_change":"2024-04-16T17:03:24.965746+0000","last_active":"2024-04-16T17:03:30.510743+0000","last_peered":"2024-04-16T17:03:30.510743+0000","last_clean":"2024-04-16T17:03:30.510743+0000","last_became_active":"2024-04-16T17:03:24.965241+0000","last_became_peered":"2024-04-16T17:03:24.965241+0000","last_unstale":"2024-04-16T17:03:30.510743+0000","last_undegraded":"2024-04-16T17:03:30.510743+0000","last_fullsized":"2024-04-16T17:03:30.510743+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:51:34.407565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:30.254317+0000","last_change":"2024-04-16T17:03:29.261441+0000","last_active":"2024-04-16T17:03:30.254317+0000","last_peered":"2024-04-16T17:03:30.254317+0000","last_clean":"2024-04-16T17:03:30.254317+0000","last_became_active":"2024-04-16T17:03:29.259538+0000","last_became_peered":"2024-04-16T17:03:29.259538+0000","last_unstale":"2024-04-16T17:03:30.254317+0000","last_undegraded":"2024-04-16T17:03:30.254317+0000","last_fullsized":"2024-04-16T17:03:30.254317+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:12:27.092053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"90'3","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:51.375506+0000","last_change":"2024-04-16T17:03:24.172053+0000","last_active":"2024-04-16T17:03:51.375506+0000","last_peered":"2024-04-16T17:03:51.375506+0000","last_clean":"2024-04-16T17:03:51.375506+0000","last_became_active":"2024-04-16T17:03:24.171709+0000","last_became_peered":"2024-04-16T17:03:24.171709+0000","last_unstale":"2024-04-16T17:03:51.375506+0000","last_undegraded":"2024-04-16T17:03:51.375506+0000","last_fullsized":"2024-04-16T17:03:51.375506+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:40:12.307756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:30.399747+0000","last_change":"2024-04-16T17:03:24.984267+0000","last_active":"2024-04-16T17:03:30.399747+0000","last_peered":"2024-04-16T17:03:30.399747+0000","last_clean":"2024-04-16T17:03:30.399747+0000","last_became_active":"2024-04-16T17:03:24.983888+0000","last_became_peered":"2024-04-16T17:03:24.983888+0000","last_unstale":"2024-04-16T17:03:30.399747+0000","last_undegraded":"2024-04-16T17:03:30.399747+0000","last_fullsized":"2024-04-16T17:03:30.399747+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:16:23.814917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:30.512852+0000","last_change":"2024-04-16T17:03:29.251716+0000","last_active":"2024-04-16T17:03:30.512852+0000","last_peered":"2024-04-16T17:03:30.512852+0000","last_clean":"2024-04-16T17:03:30.512852+0000","last_became_active":"2024-04-16T17:03:29.251317+0000","last_became_peered":"2024-04-16T17:03:29.251317+0000","last_unstale":"2024-04-16T17:03:30.512852+0000","last_undegraded":"2024-04-16T17:03:30.512852+0000","last_fullsized":"2024-04-16T17:03:30.512852+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:48:54.680195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"90'1","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:30.250985+0000","last_change":"2024-04-16T17:03:24.167511+0000","last_active":"2024-04-16T17:03:30.250985+0000","last_peered":"2024-04-16T17:03:30.250985+0000","last_clean":"2024-04-16T17:03:30.250985+0000","last_became_active":"2024-04-16T17:03:24.167170+0000","last_became_peered":"2024-04-16T17:03:24.167170+0000","last_unstale":"2024-04-16T17:03:30.250985+0000","last_undegraded":"2024-04-16T17:03:30.250985+0000","last_fullsized":"2024-04-16T17:03:30.250985+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:31:49.892269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:30.511721+0000","last_change":"2024-04-16T17:03:24.995547+0000","last_active":"2024-04-16T17:03:30.511721+0000","last_peered":"2024-04-16T17:03:30.511721+0000","last_clean":"2024-04-16T17:03:30.511721+0000","last_became_active":"2024-04-16T17:03:24.995096+0000","last_became_peered":"2024-04-16T17:03:24.995096+0000","last_unstale":"2024-04-16T17:03:30.511721+0000","last_undegraded":"2024-04-16T17:03:30.511721+0000","last_fullsized":"2024-04-16T17:03:30.511721+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:54:51.193117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:30.512519+0000","last_change":"2024-04-16T17:03:29.301526+0000","last_active":"2024-04-16T17:03:30.512519+0000","last_peered":"2024-04-16T17:03:30.512519+0000","last_clean":"2024-04-16T17:03:30.512519+0000","last_became_active":"2024-04-16T17:03:29.300808+0000","last_became_peered":"2024-04-16T17:03:29.300808+0000","last_unstale":"2024-04-16T17:03:30.512519+0000","last_undegraded":"2024-04-16T17:03:30.512519+0000","last_fullsized":"2024-04-16T17:03:30.512519+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:01:14.717548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"90'4","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:51.372404+0000","last_change":"2024-04-16T17:03:24.152645+0000","last_active":"2024-04-16T17:03:51.372404+0000","last_peered":"2024-04-16T17:03:51.372404+0000","last_clean":"2024-04-16T17:03:51.372404+0000","last_became_active":"2024-04-16T17:03:24.152281+0000","last_became_peered":"2024-04-16T17:03:24.152281+0000","last_unstale":"2024-04-16T17:03:51.372404+0000","last_undegraded":"2024-04-16T17:03:51.372404+0000","last_fullsized":"2024-04-16T17:03:51.372404+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:33:51.377421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:30.511887+0000","last_change":"2024-04-16T17:03:24.979827+0000","last_active":"2024-04-16T17:03:30.511887+0000","last_peered":"2024-04-16T17:03:30.511887+0000","last_clean":"2024-04-16T17:03:30.511887+0000","last_became_active":"2024-04-16T17:03:24.979452+0000","last_became_peered":"2024-04-16T17:03:24.979452+0000","last_unstale":"2024-04-16T17:03:30.511887+0000","last_undegraded":"2024-04-16T17:03:30.511887+0000","last_fullsized":"2024-04-16T17:03:30.511887+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:07:09.259366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:30.250611+0000","last_change":"2024-04-16T17:03:29.255519+0000","last_active":"2024-04-16T17:03:30.250611+0000","last_peered":"2024-04-16T17:03:30.250611+0000","last_clean":"2024-04-16T17:03:30.250611+0000","last_became_active":"2024-04-16T17:03:29.255104+0000","last_became_peered":"2024-04-16T17:03:29.255104+0000","last_unstale":"2024-04-16T17:03:30.250611+0000","last_undegraded":"2024-04-16T17:03:30.250611+0000","last_fullsized":"2024-04-16T17:03:30.250611+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:27:34.496199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"90'3","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:52.386419+0000","last_change":"2024-04-16T17:03:24.161145+0000","last_active":"2024-04-16T17:03:52.386419+0000","last_peered":"2024-04-16T17:03:52.386419+0000","last_clean":"2024-04-16T17:03:52.386419+0000","last_became_active":"2024-04-16T17:03:24.160824+0000","last_became_peered":"2024-04-16T17:03:24.160824+0000","last_unstale":"2024-04-16T17:03:52.386419+0000","last_undegraded":"2024-04-16T17:03:52.386419+0000","last_fullsized":"2024-04-16T17:03:52.386419+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:02:48.805638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:30.513026+0000","last_change":"2024-04-16T17:03:24.966513+0000","last_active":"2024-04-16T17:03:30.513026+0000","last_peered":"2024-04-16T17:03:30.513026+0000","last_clean":"2024-04-16T17:03:30.513026+0000","last_became_active":"2024-04-16T17:03:24.966074+0000","last_became_peered":"2024-04-16T17:03:24.966074+0000","last_unstale":"2024-04-16T17:03:30.513026+0000","last_undegraded":"2024-04-16T17:03:30.513026+0000","last_fullsized":"2024-04-16T17:03:30.513026+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:50:21.579901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:30.511846+0000","last_change":"2024-04-16T17:03:29.244236+0000","last_active":"2024-04-16T17:03:30.511846+0000","last_peered":"2024-04-16T17:03:30.511846+0000","last_clean":"2024-04-16T17:03:30.511846+0000","last_became_active":"2024-04-16T17:03:29.242809+0000","last_became_peered":"2024-04-16T17:03:29.242809+0000","last_unstale":"2024-04-16T17:03:30.511846+0000","last_undegraded":"2024-04-16T17:03:30.511846+0000","last_fullsized":"2024-04-16T17:03:30.511846+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:38:01.788299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"85'1","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:30.512271+0000","last_change":"2024-04-16T17:03:24.158032+0000","last_active":"2024-04-16T17:03:30.512271+0000","last_peered":"2024-04-16T17:03:30.512271+0000","last_clean":"2024-04-16T17:03:30.512271+0000","last_became_active":"2024-04-16T17:03:24.157689+0000","last_became_peered":"2024-04-16T17:03:24.157689+0000","last_unstale":"2024-04-16T17:03:30.512271+0000","last_undegraded":"2024-04-16T17:03:30.512271+0000","last_fullsized":"2024-04-16T17:03:30.512271+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:38:18.240889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:30.512181+0000","last_change":"2024-04-16T17:03:24.977780+0000","last_active":"2024-04-16T17:03:30.512181+0000","last_peered":"2024-04-16T17:03:30.512181+0000","last_clean":"2024-04-16T17:03:30.512181+0000","last_became_active":"2024-04-16T17:03:24.977397+0000","last_became_peered":"2024-04-16T17:03:24.977397+0000","last_unstale":"2024-04-16T17:03:30.512181+0000","last_undegraded":"2024-04-16T17:03:30.512181+0000","last_fullsized":"2024-04-16T17:03:30.512181+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:37:30.701763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:30.512202+0000","last_change":"2024-04-16T17:03:29.245176+0000","last_active":"2024-04-16T17:03:30.512202+0000","last_peered":"2024-04-16T17:03:30.512202+0000","last_clean":"2024-04-16T17:03:30.512202+0000","last_became_active":"2024-04-16T17:03:29.244763+0000","last_became_peered":"2024-04-16T17:03:29.244763+0000","last_unstale":"2024-04-16T17:03:30.512202+0000","last_undegraded":"2024-04-16T17:03:30.512202+0000","last_fullsized":"2024-04-16T17:03:30.512202+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:52:30.381417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:30.664851+0000","last_change":"2024-04-16T17:03:24.168068+0000","last_active":"2024-04-16T17:03:30.664851+0000","last_peered":"2024-04-16T17:03:30.664851+0000","last_clean":"2024-04-16T17:03:30.664851+0000","last_became_active":"2024-04-16T17:03:24.167694+0000","last_became_peered":"2024-04-16T17:03:24.167694+0000","last_unstale":"2024-04-16T17:03:30.664851+0000","last_undegraded":"2024-04-16T17:03:30.664851+0000","last_fullsized":"2024-04-16T17:03:30.664851+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:13:35.692306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:30.513055+0000","last_change":"2024-04-16T17:03:24.966145+0000","last_active":"2024-04-16T17:03:30.513055+0000","last_peered":"2024-04-16T17:03:30.513055+0000","last_clean":"2024-04-16T17:03:30.513055+0000","last_became_active":"2024-04-16T17:03:24.965813+0000","last_became_peered":"2024-04-16T17:03:24.965813+0000","last_unstale":"2024-04-16T17:03:30.513055+0000","last_undegraded":"2024-04-16T17:03:30.513055+0000","last_fullsized":"2024-04-16T17:03:30.513055+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:43:43.224130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:30.399863+0000","last_change":"2024-04-16T17:03:24.970941+0000","last_active":"2024-04-16T17:03:30.399863+0000","last_peered":"2024-04-16T17:03:30.399863+0000","last_clean":"2024-04-16T17:03:30.399863+0000","last_became_active":"2024-04-16T17:03:24.970587+0000","last_became_peered":"2024-04-16T17:03:24.970587+0000","last_unstale":"2024-04-16T17:03:30.399863+0000","last_undegraded":"2024-04-16T17:03:30.399863+0000","last_fullsized":"2024-04-16T17:03:30.399863+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:14:13.930011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:30.251358+0000","last_change":"2024-04-16T17:03:24.967167+0000","last_active":"2024-04-16T17:03:30.251358+0000","last_peered":"2024-04-16T17:03:30.251358+0000","last_clean":"2024-04-16T17:03:30.251358+0000","last_became_active":"2024-04-16T17:03:24.966787+0000","last_became_peered":"2024-04-16T17:03:24.966787+0000","last_unstale":"2024-04-16T17:03:30.251358+0000","last_undegraded":"2024-04-16T17:03:30.251358+0000","last_fullsized":"2024-04-16T17:03:30.251358+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:51:39.510220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:30.250802+0000","last_change":"2024-04-16T17:03:24.981115+0000","last_active":"2024-04-16T17:03:30.250802+0000","last_peered":"2024-04-16T17:03:30.250802+0000","last_clean":"2024-04-16T17:03:30.250802+0000","last_became_active":"2024-04-16T17:03:24.980685+0000","last_became_peered":"2024-04-16T17:03:24.980685+0000","last_unstale":"2024-04-16T17:03:30.250802+0000","last_undegraded":"2024-04-16T17:03:30.250802+0000","last_fullsized":"2024-04-16T17:03:30.250802+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:20:44.066932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:30.511844+0000","last_change":"2024-04-16T17:03:24.990421+0000","last_active":"2024-04-16T17:03:30.511844+0000","last_peered":"2024-04-16T17:03:30.511844+0000","last_clean":"2024-04-16T17:03:30.511844+0000","last_became_active":"2024-04-16T17:03:24.989940+0000","last_became_peered":"2024-04-16T17:03:24.989940+0000","last_unstale":"2024-04-16T17:03:30.511844+0000","last_undegraded":"2024-04-16T17:03:30.511844+0000","last_fullsized":"2024-04-16T17:03:30.511844+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:19:57.548067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:30.664720+0000","last_change":"2024-04-16T17:03:24.991854+0000","last_active":"2024-04-16T17:03:30.664720+0000","last_peered":"2024-04-16T17:03:30.664720+0000","last_clean":"2024-04-16T17:03:30.664720+0000","last_became_active":"2024-04-16T17:03:24.990964+0000","last_became_peered":"2024-04-16T17:03:24.990964+0000","last_unstale":"2024-04-16T17:03:30.664720+0000","last_undegraded":"2024-04-16T17:03:30.664720+0000","last_fullsized":"2024-04-16T17:03:30.664720+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:10:10.618708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:30.400274+0000","last_change":"2024-04-16T17:03:24.971332+0000","last_active":"2024-04-16T17:03:30.400274+0000","last_peered":"2024-04-16T17:03:30.400274+0000","last_clean":"2024-04-16T17:03:30.400274+0000","last_became_active":"2024-04-16T17:03:24.970975+0000","last_became_peered":"2024-04-16T17:03:24.970975+0000","last_unstale":"2024-04-16T17:03:30.400274+0000","last_undegraded":"2024-04-16T17:03:30.400274+0000","last_fullsized":"2024-04-16T17:03:30.400274+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:18:01.046445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:30.251550+0000","last_change":"2024-04-16T17:03:24.970377+0000","last_active":"2024-04-16T17:03:30.251550+0000","last_peered":"2024-04-16T17:03:30.251550+0000","last_clean":"2024-04-16T17:03:30.251550+0000","last_became_active":"2024-04-16T17:03:24.968856+0000","last_became_peered":"2024-04-16T17:03:24.968856+0000","last_unstale":"2024-04-16T17:03:30.251550+0000","last_undegraded":"2024-04-16T17:03:30.251550+0000","last_fullsized":"2024-04-16T17:03:30.251550+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:47:28.543120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:30.512482+0000","last_change":"2024-04-16T17:03:24.992877+0000","last_active":"2024-04-16T17:03:30.512482+0000","last_peered":"2024-04-16T17:03:30.512482+0000","last_clean":"2024-04-16T17:03:30.512482+0000","last_became_active":"2024-04-16T17:03:24.992484+0000","last_became_peered":"2024-04-16T17:03:24.992484+0000","last_unstale":"2024-04-16T17:03:30.512482+0000","last_undegraded":"2024-04-16T17:03:30.512482+0000","last_fullsized":"2024-04-16T17:03:30.512482+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:00:45.204782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:30.511453+0000","last_change":"2024-04-16T17:03:24.977794+0000","last_active":"2024-04-16T17:03:30.511453+0000","last_peered":"2024-04-16T17:03:30.511453+0000","last_clean":"2024-04-16T17:03:30.511453+0000","last_became_active":"2024-04-16T17:03:24.977434+0000","last_became_peered":"2024-04-16T17:03:24.977434+0000","last_unstale":"2024-04-16T17:03:30.511453+0000","last_undegraded":"2024-04-16T17:03:30.511453+0000","last_fullsized":"2024-04-16T17:03:30.511453+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:46:35.670357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:30.511470+0000","last_change":"2024-04-16T17:03:24.992225+0000","last_active":"2024-04-16T17:03:30.511470+0000","last_peered":"2024-04-16T17:03:30.511470+0000","last_clean":"2024-04-16T17:03:30.511470+0000","last_became_active":"2024-04-16T17:03:24.991904+0000","last_became_peered":"2024-04-16T17:03:24.991904+0000","last_unstale":"2024-04-16T17:03:30.511470+0000","last_undegraded":"2024-04-16T17:03:30.511470+0000","last_fullsized":"2024-04-16T17:03:30.511470+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:01:09.710698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:30.250680+0000","last_change":"2024-04-16T17:03:24.981322+0000","last_active":"2024-04-16T17:03:30.250680+0000","last_peered":"2024-04-16T17:03:30.250680+0000","last_clean":"2024-04-16T17:03:30.250680+0000","last_became_active":"2024-04-16T17:03:24.980920+0000","last_became_peered":"2024-04-16T17:03:24.980920+0000","last_unstale":"2024-04-16T17:03:30.250680+0000","last_undegraded":"2024-04-16T17:03:30.250680+0000","last_fullsized":"2024-04-16T17:03:30.250680+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:07:59.378512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:30.663982+0000","last_change":"2024-04-16T17:03:24.990117+0000","last_active":"2024-04-16T17:03:30.663982+0000","last_peered":"2024-04-16T17:03:30.663982+0000","last_clean":"2024-04-16T17:03:30.663982+0000","last_became_active":"2024-04-16T17:03:24.989734+0000","last_became_peered":"2024-04-16T17:03:24.989734+0000","last_unstale":"2024-04-16T17:03:30.663982+0000","last_undegraded":"2024-04-16T17:03:30.663982+0000","last_fullsized":"2024-04-16T17:03:30.663982+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:36:55.000702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:30.400073+0000","last_change":"2024-04-16T17:03:24.970032+0000","last_active":"2024-04-16T17:03:30.400073+0000","last_peered":"2024-04-16T17:03:30.400073+0000","last_clean":"2024-04-16T17:03:30.400073+0000","last_became_active":"2024-04-16T17:03:24.968608+0000","last_became_peered":"2024-04-16T17:03:24.968608+0000","last_unstale":"2024-04-16T17:03:30.400073+0000","last_undegraded":"2024-04-16T17:03:30.400073+0000","last_fullsized":"2024-04-16T17:03:30.400073+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:24:03.918418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":283858,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":13709,"num_objects":52,"num_object_clones":0,"num_object_copies":156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":0,"num_read":18,"num_read_kb":17,"num_write":122,"num_write_kb":42,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":36,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":241664,"data_stored":47910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":105,"ondisk_log_size":105,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892350995,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":978944,"data_stored":484944,"data_compressed":14704,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612634,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32216,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712936,"statfs":{"total":95995035648,"available":95962046464,"internally_reserved":0,"allocated":1064960,"data_stored":1076054,"data_compressed":50802,"data_compressed_allocated":65536,"data_compressed_original":630784,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776158,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32140,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713012,"statfs":{"total":95995035648,"available":95962124288,"internally_reserved":0,"allocated":987136,"data_stored":481050,"data_compressed":14704,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037797,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32136,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713016,"statfs":{"total":95995035648,"available":95962128384,"internally_reserved":0,"allocated":983040,"data_stored":480892,"data_compressed":14704,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299435,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32184,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712968,"statfs":{"total":95995035648,"available":95962079232,"internally_reserved":0,"allocated":1032192,"data_stored":1071838,"data_compressed":50802,"data_compressed_allocated":65536,"data_compressed_original":630784,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462961,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32128,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713024,"statfs":{"total":95995035648,"available":95962136576,"internally_reserved":0,"allocated":974848,"data_stored":480356,"data_compressed":14704,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574925,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32136,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713016,"statfs":{"total":95995035648,"available":95962128384,"internally_reserved":0,"allocated":983040,"data_stored":484108,"data_compressed":14704,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738449,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32136,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713016,"statfs":{"total":95995035648,"available":95962128384,"internally_reserved":0,"allocated":974848,"data_stored":480356,"data_compressed":14704,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411399,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1028096,"data_stored":505148,"data_compressed":14704,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149761,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32140,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713012,"statfs":{"total":95995035648,"available":95962124288,"internally_reserved":0,"allocated":987136,"data_stored":480550,"data_compressed":14704,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888123,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":974848,"data_stored":480288,"data_compressed":14704,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724599,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1052672,"data_stored":1076597,"data_compressed":50802,"data_compressed_allocated":65536,"data_compressed_original":630784,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34456,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34456,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34456,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36098,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36098,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36098,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36098,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36098,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":24882,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":12411,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-16T17:03:57.453 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-16T17:03:57.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:03:57 smithi104 ceph-mon[36457]: from='client.24851 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T17:03:57.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:03:57 smithi104 ceph-mon[36457]: pgmap v338: 129 pgs: 129 active+clean; 590 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2.3 KiB/s wr, 7 op/s 2024-04-16T17:03:57.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:03:57 smithi104 ceph-mon[36457]: from='client.? 172.21.15.45:0/3853156587' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:03:57.679 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:03:57 smithi104 ceph-mon[36457]: from='client.? 172.21.15.45:0/760600030' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T17:03:57.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:57.714+0000 7fb152a00640 1 -- 172.21.15.45:0/1268736040 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb14c108480 msgr2=0x7fb14c1088a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:57.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:57.714+0000 7fb152a00640 1 --2- 172.21.15.45:0/1268736040 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb14c108480 0x7fb14c1088a0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fb140009900 tx=0x7fb14002bc30 comp rx=0 tx=0).stop 2024-04-16T17:03:57.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:57.714+0000 7fb152a00640 1 -- 172.21.15.45:0/1268736040 shutdown_connections 2024-04-16T17:03:57.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:57.714+0000 7fb152a00640 1 --2- 172.21.15.45:0/1268736040 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb14c109670 0x7fb14c109af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:57.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:57.714+0000 7fb152a00640 1 --2- 172.21.15.45:0/1268736040 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb14c10a030 0x7fb14c1108c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:57.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:57.714+0000 7fb152a00640 1 --2- 172.21.15.45:0/1268736040 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb14c108480 0x7fb14c1088a0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:57.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:57.714+0000 7fb152a00640 1 -- 172.21.15.45:0/1268736040 >> 172.21.15.45:0/1268736040 conn(0x7fb14c077440 msgr2=0x7fb14c079860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:03:57.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:57.714+0000 7fb152a00640 1 -- 172.21.15.45:0/1268736040 shutdown_connections 2024-04-16T17:03:57.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:57.714+0000 7fb152a00640 1 -- 172.21.15.45:0/1268736040 wait complete. 2024-04-16T17:03:57.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:57.715+0000 7fb152a00640 1 Processor -- start 2024-04-16T17:03:57.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:57.715+0000 7fb152a00640 1 -- start start 2024-04-16T17:03:57.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:57.715+0000 7fb152a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb14c108480 0x7fb14c19e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:57.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:57.715+0000 7fb152a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb14c109670 0x7fb14c19ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:57.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:57.715+0000 7fb152a00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb14c10a030 0x7fb14c1a5d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:57.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:57.715+0000 7fb152a00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fb14c114110 con 0x7fb14c108480 2024-04-16T17:03:57.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:57.715+0000 7fb152a00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fb14c113f90 con 0x7fb14c10a030 2024-04-16T17:03:57.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:57.715+0000 7fb152a00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb14c114290 con 0x7fb14c109670 2024-04-16T17:03:57.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:57.716+0000 7fb151600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb14c108480 0x7fb14c19e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:57.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:57.716+0000 7fb151600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb14c108480 0x7fb14c19e7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45104/0 (socket says 172.21.15.45:45104) 2024-04-16T17:03:57.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:57.716+0000 7fb150c00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb14c109670 0x7fb14c19ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:57.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:57.716+0000 7fb151600640 1 -- 172.21.15.45:0/3663985178 learned_addr learned my addr 172.21.15.45:0/3663985178 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:03:57.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:57.716+0000 7fb152000640 1 --2- 172.21.15.45:0/3663985178 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb14c10a030 0x7fb14c1a5d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:57.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:57.716+0000 7fb150c00640 1 -- 172.21.15.45:0/3663985178 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb14c10a030 msgr2=0x7fb14c1a5d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:57.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:57.716+0000 7fb150c00640 1 --2- 172.21.15.45:0/3663985178 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb14c10a030 0x7fb14c1a5d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:57.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:57.716+0000 7fb150c00640 1 -- 172.21.15.45:0/3663985178 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb14c108480 msgr2=0x7fb14c19e7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:57.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:57.716+0000 7fb150c00640 1 --2- 172.21.15.45:0/3663985178 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb14c108480 0x7fb14c19e7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:57.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:57.716+0000 7fb150c00640 1 -- 172.21.15.45:0/3663985178 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb14c1a63c0 con 0x7fb14c109670 2024-04-16T17:03:57.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:57.716+0000 7fb151600640 1 --2- 172.21.15.45:0/3663985178 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb14c108480 0x7fb14c19e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:03:57.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:57.716+0000 7fb150c00640 1 --2- 172.21.15.45:0/3663985178 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb14c109670 0x7fb14c19ece0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fb13800e9f0 tx=0x7fb13800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:03:57.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:57.717+0000 7fb14a000640 1 -- 172.21.15.45:0/3663985178 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb13800ccc0 con 0x7fb14c109670 2024-04-16T17:03:57.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:57.717+0000 7fb14a000640 1 -- 172.21.15.45:0/3663985178 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb138005020 con 0x7fb14c109670 2024-04-16T17:03:57.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:57.717+0000 7fb14a000640 1 -- 172.21.15.45:0/3663985178 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1380049e0 con 0x7fb14c109670 2024-04-16T17:03:57.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:57.717+0000 7fb152a00640 1 -- 172.21.15.45:0/3663985178 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb14c1a66b0 con 0x7fb14c109670 2024-04-16T17:03:57.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:57.717+0000 7fb152a00640 1 -- 172.21.15.45:0/3663985178 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb14c1a6bb0 con 0x7fb14c109670 2024-04-16T17:03:57.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:57.718+0000 7fb152a00640 1 -- 172.21.15.45:0/3663985178 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb14c10f820 con 0x7fb14c109670 2024-04-16T17:03:57.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:57.721+0000 7fb14a000640 1 -- 172.21.15.45:0/3663985178 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fb138005560 con 0x7fb14c109670 2024-04-16T17:03:57.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:57.723+0000 7fb14a000640 1 --2- 172.21.15.45:0/3663985178 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fb1340a3c40 0x7fb1340a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:03:57.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:57.723+0000 7fb14a000640 1 -- 172.21.15.45:0/3663985178 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7fb1380d0910 con 0x7fb14c109670 2024-04-16T17:03:57.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:57.723+0000 7fb151600640 1 --2- 172.21.15.45:0/3663985178 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fb1340a3c40 0x7fb1340a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:03:57.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:57.724+0000 7fb151600640 1 --2- 172.21.15.45:0/3663985178 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fb1340a3c40 0x7fb1340a6100 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fb14002c8e0 tx=0x7fb140005b50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:03:57.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:57.724+0000 7fb14a000640 1 -- 172.21.15.45:0/3663985178 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb1380051d0 con 0x7fb14c109670 2024-04-16T17:03:57.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:03:57 smithi045 ceph-mon[31676]: from='client.24851 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T17:03:57.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:03:57 smithi045 ceph-mon[31676]: pgmap v338: 129 pgs: 129 active+clean; 590 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2.3 KiB/s wr, 7 op/s 2024-04-16T17:03:57.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:03:57 smithi045 ceph-mon[31676]: from='client.? 172.21.15.45:0/3853156587' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:03:57.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:03:57 smithi045 ceph-mon[31676]: from='client.? 172.21.15.45:0/760600030' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T17:03:57.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:03:57 smithi117 ceph-mon[36730]: from='client.24851 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T17:03:57.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:03:57 smithi117 ceph-mon[36730]: pgmap v338: 129 pgs: 129 active+clean; 590 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2.3 KiB/s wr, 7 op/s 2024-04-16T17:03:57.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:03:57 smithi117 ceph-mon[36730]: from='client.? 172.21.15.45:0/3853156587' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:03:57.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:03:57 smithi117 ceph-mon[36730]: from='client.? 172.21.15.45:0/760600030' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T17:03:57.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:57.931+0000 7fb152a00640 1 -- 172.21.15.45:0/3663985178 --> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb14c062440 con 0x7fb1340a3c40 2024-04-16T17:03:57.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:57.961+0000 7fb14a000640 1 -- 172.21.15.45:0/3663985178 <== mgr.14152 v2:172.21.15.45:6800/268030699 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340627 (secure 0 0 0) 0x7fb14c062440 con 0x7fb1340a3c40 2024-04-16T17:03:57.965 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-16T17:03:57.969 INFO:teuthology.orchestra.run.smithi045.stderr:dumped all 2024-04-16T17:03:57.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:57.969+0000 7fb152a00640 1 -- 172.21.15.45:0/3663985178 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fb1340a3c40 msgr2=0x7fb1340a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:57.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:57.969+0000 7fb152a00640 1 --2- 172.21.15.45:0/3663985178 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fb1340a3c40 0x7fb1340a6100 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fb14002c8e0 tx=0x7fb140005b50 comp rx=0 tx=0).stop 2024-04-16T17:03:57.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:57.969+0000 7fb152a00640 1 -- 172.21.15.45:0/3663985178 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb14c109670 msgr2=0x7fb14c19ece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:03:57.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:57.969+0000 7fb152a00640 1 --2- 172.21.15.45:0/3663985178 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb14c109670 0x7fb14c19ece0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fb13800e9f0 tx=0x7fb13800eeb0 comp rx=0 tx=0).stop 2024-04-16T17:03:57.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:57.969+0000 7fb152a00640 1 -- 172.21.15.45:0/3663985178 shutdown_connections 2024-04-16T17:03:57.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:57.969+0000 7fb152a00640 1 --2- 172.21.15.45:0/3663985178 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fb1340a3c40 0x7fb1340a6100 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:57.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:57.969+0000 7fb152a00640 1 --2- 172.21.15.45:0/3663985178 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb14c109670 0x7fb14c19ece0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:57.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:57.969+0000 7fb152a00640 1 --2- 172.21.15.45:0/3663985178 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb14c10a030 0x7fb14c1a5d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:57.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:57.969+0000 7fb152a00640 1 --2- 172.21.15.45:0/3663985178 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb14c108480 0x7fb14c19e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:03:57.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:57.969+0000 7fb152a00640 1 -- 172.21.15.45:0/3663985178 >> 172.21.15.45:0/3663985178 conn(0x7fb14c077440 msgr2=0x7fb14c10c9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:03:57.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:57.969+0000 7fb152a00640 1 -- 172.21.15.45:0/3663985178 shutdown_connections 2024-04-16T17:03:57.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:03:57.969+0000 7fb152a00640 1 -- 172.21.15.45:0/3663985178 wait complete. 2024-04-16T17:03:57.993 INFO:teuthology.orchestra.run.smithi045.stdout:{"pg_ready":true,"pg_map":{"version":338,"stamp":"2024-04-16T17:03:56.109071+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":604077,"num_objects":54,"num_object_clones":0,"num_object_copies":162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":0,"num_read":154,"num_read_kb":136,"num_write":303,"num_write_kb":2980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":36,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":242,"ondisk_log_size":242,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":369224,"kb_used_data":11740,"kb_used_omap":18,"kb_used_meta":357357,"kb_avail":1124572600,"statfs":{"total":1151940427776,"available":1151562342400,"internally_reserved":0,"allocated":12021760,"data_stored":7582181,"data_compressed":284742,"data_compressed_allocated":380928,"data_compressed_original":2260992,"omap_allocated":19143,"internal_metadata":365933881},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":11299,"num_objects":30,"num_object_clones":0,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":18,"num_read_kb":17,"num_write":77,"num_write_kb":28,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":22,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006640"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:30.400293+0000","last_change":"2024-04-16T17:03:24.989508+0000","last_active":"2024-04-16T17:03:30.400293+0000","last_peered":"2024-04-16T17:03:30.400293+0000","last_clean":"2024-04-16T17:03:30.400293+0000","last_became_active":"2024-04-16T17:03:24.989178+0000","last_became_peered":"2024-04-16T17:03:24.989178+0000","last_unstale":"2024-04-16T17:03:30.400293+0000","last_undegraded":"2024-04-16T17:03:30.400293+0000","last_fullsized":"2024-04-16T17:03:30.400293+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:18:56.488358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:30.512048+0000","last_change":"2024-04-16T17:03:24.986378+0000","last_active":"2024-04-16T17:03:30.512048+0000","last_peered":"2024-04-16T17:03:30.512048+0000","last_clean":"2024-04-16T17:03:30.512048+0000","last_became_active":"2024-04-16T17:03:24.986030+0000","last_became_peered":"2024-04-16T17:03:24.986030+0000","last_unstale":"2024-04-16T17:03:30.512048+0000","last_undegraded":"2024-04-16T17:03:30.512048+0000","last_fullsized":"2024-04-16T17:03:30.512048+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:21:14.071109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:30.511599+0000","last_change":"2024-04-16T17:03:24.976232+0000","last_active":"2024-04-16T17:03:30.511599+0000","last_peered":"2024-04-16T17:03:30.511599+0000","last_clean":"2024-04-16T17:03:30.511599+0000","last_became_active":"2024-04-16T17:03:24.975873+0000","last_became_peered":"2024-04-16T17:03:24.975873+0000","last_unstale":"2024-04-16T17:03:30.511599+0000","last_undegraded":"2024-04-16T17:03:30.511599+0000","last_fullsized":"2024-04-16T17:03:30.511599+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:07:53.523101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:30.512686+0000","last_change":"2024-04-16T17:03:24.967995+0000","last_active":"2024-04-16T17:03:30.512686+0000","last_peered":"2024-04-16T17:03:30.512686+0000","last_clean":"2024-04-16T17:03:30.512686+0000","last_became_active":"2024-04-16T17:03:24.967675+0000","last_became_peered":"2024-04-16T17:03:24.967675+0000","last_unstale":"2024-04-16T17:03:30.512686+0000","last_undegraded":"2024-04-16T17:03:30.512686+0000","last_fullsized":"2024-04-16T17:03:30.512686+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:44:38.474399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:30.510925+0000","last_change":"2024-04-16T17:03:24.985609+0000","last_active":"2024-04-16T17:03:30.510925+0000","last_peered":"2024-04-16T17:03:30.510925+0000","last_clean":"2024-04-16T17:03:30.510925+0000","last_became_active":"2024-04-16T17:03:24.985239+0000","last_became_peered":"2024-04-16T17:03:24.985239+0000","last_unstale":"2024-04-16T17:03:30.510925+0000","last_undegraded":"2024-04-16T17:03:30.510925+0000","last_fullsized":"2024-04-16T17:03:30.510925+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:55:58.619210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:30.511963+0000","last_change":"2024-04-16T17:03:24.966862+0000","last_active":"2024-04-16T17:03:30.511963+0000","last_peered":"2024-04-16T17:03:30.511963+0000","last_clean":"2024-04-16T17:03:30.511963+0000","last_became_active":"2024-04-16T17:03:24.966234+0000","last_became_peered":"2024-04-16T17:03:24.966234+0000","last_unstale":"2024-04-16T17:03:30.511963+0000","last_undegraded":"2024-04-16T17:03:30.511963+0000","last_fullsized":"2024-04-16T17:03:30.511963+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:21:07.018281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:30.511080+0000","last_change":"2024-04-16T17:03:24.971219+0000","last_active":"2024-04-16T17:03:30.511080+0000","last_peered":"2024-04-16T17:03:30.511080+0000","last_clean":"2024-04-16T17:03:30.511080+0000","last_became_active":"2024-04-16T17:03:24.970876+0000","last_became_peered":"2024-04-16T17:03:24.970876+0000","last_unstale":"2024-04-16T17:03:30.511080+0000","last_undegraded":"2024-04-16T17:03:30.511080+0000","last_fullsized":"2024-04-16T17:03:30.511080+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:44:16.311167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:30.510880+0000","last_change":"2024-04-16T17:03:24.994694+0000","last_active":"2024-04-16T17:03:30.510880+0000","last_peered":"2024-04-16T17:03:30.510880+0000","last_clean":"2024-04-16T17:03:30.510880+0000","last_became_active":"2024-04-16T17:03:24.994315+0000","last_became_peered":"2024-04-16T17:03:24.994315+0000","last_unstale":"2024-04-16T17:03:30.510880+0000","last_undegraded":"2024-04-16T17:03:30.510880+0000","last_fullsized":"2024-04-16T17:03:30.510880+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:04:59.543816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:30.251023+0000","last_change":"2024-04-16T17:03:24.964654+0000","last_active":"2024-04-16T17:03:30.251023+0000","last_peered":"2024-04-16T17:03:30.251023+0000","last_clean":"2024-04-16T17:03:30.251023+0000","last_became_active":"2024-04-16T17:03:24.964349+0000","last_became_peered":"2024-04-16T17:03:24.964349+0000","last_unstale":"2024-04-16T17:03:30.251023+0000","last_undegraded":"2024-04-16T17:03:30.251023+0000","last_fullsized":"2024-04-16T17:03:30.251023+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:38:55.880948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:30.512071+0000","last_change":"2024-04-16T17:03:24.966364+0000","last_active":"2024-04-16T17:03:30.512071+0000","last_peered":"2024-04-16T17:03:30.512071+0000","last_clean":"2024-04-16T17:03:30.512071+0000","last_became_active":"2024-04-16T17:03:24.966061+0000","last_became_peered":"2024-04-16T17:03:24.966061+0000","last_unstale":"2024-04-16T17:03:30.512071+0000","last_undegraded":"2024-04-16T17:03:30.512071+0000","last_fullsized":"2024-04-16T17:03:30.512071+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:18:12.642602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:30.512774+0000","last_change":"2024-04-16T17:03:24.969874+0000","last_active":"2024-04-16T17:03:30.512774+0000","last_peered":"2024-04-16T17:03:30.512774+0000","last_clean":"2024-04-16T17:03:30.512774+0000","last_became_active":"2024-04-16T17:03:24.969523+0000","last_became_peered":"2024-04-16T17:03:24.969523+0000","last_unstale":"2024-04-16T17:03:30.512774+0000","last_undegraded":"2024-04-16T17:03:30.512774+0000","last_fullsized":"2024-04-16T17:03:30.512774+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:17:49.704855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:30.512389+0000","last_change":"2024-04-16T17:03:24.968660+0000","last_active":"2024-04-16T17:03:30.512389+0000","last_peered":"2024-04-16T17:03:30.512389+0000","last_clean":"2024-04-16T17:03:30.512389+0000","last_became_active":"2024-04-16T17:03:24.968348+0000","last_became_peered":"2024-04-16T17:03:24.968348+0000","last_unstale":"2024-04-16T17:03:30.512389+0000","last_undegraded":"2024-04-16T17:03:30.512389+0000","last_fullsized":"2024-04-16T17:03:30.512389+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:06:06.319391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:30.664287+0000","last_change":"2024-04-16T17:03:24.974741+0000","last_active":"2024-04-16T17:03:30.664287+0000","last_peered":"2024-04-16T17:03:30.664287+0000","last_clean":"2024-04-16T17:03:30.664287+0000","last_became_active":"2024-04-16T17:03:24.973568+0000","last_became_peered":"2024-04-16T17:03:24.973568+0000","last_unstale":"2024-04-16T17:03:30.664287+0000","last_undegraded":"2024-04-16T17:03:30.664287+0000","last_fullsized":"2024-04-16T17:03:30.664287+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:30:49.325469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:30.511469+0000","last_change":"2024-04-16T17:03:24.981629+0000","last_active":"2024-04-16T17:03:30.511469+0000","last_peered":"2024-04-16T17:03:30.511469+0000","last_clean":"2024-04-16T17:03:30.511469+0000","last_became_active":"2024-04-16T17:03:24.981111+0000","last_became_peered":"2024-04-16T17:03:24.981111+0000","last_unstale":"2024-04-16T17:03:30.511469+0000","last_undegraded":"2024-04-16T17:03:30.511469+0000","last_fullsized":"2024-04-16T17:03:30.511469+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:41:29.641610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:30.511218+0000","last_change":"2024-04-16T17:03:24.977890+0000","last_active":"2024-04-16T17:03:30.511218+0000","last_peered":"2024-04-16T17:03:30.511218+0000","last_clean":"2024-04-16T17:03:30.511218+0000","last_became_active":"2024-04-16T17:03:24.976500+0000","last_became_peered":"2024-04-16T17:03:24.976500+0000","last_unstale":"2024-04-16T17:03:30.511218+0000","last_undegraded":"2024-04-16T17:03:30.511218+0000","last_fullsized":"2024-04-16T17:03:30.511218+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:17:51.604701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:30.250242+0000","last_change":"2024-04-16T17:03:24.976085+0000","last_active":"2024-04-16T17:03:30.250242+0000","last_peered":"2024-04-16T17:03:30.250242+0000","last_clean":"2024-04-16T17:03:30.250242+0000","last_became_active":"2024-04-16T17:03:24.975804+0000","last_became_peered":"2024-04-16T17:03:24.975804+0000","last_unstale":"2024-04-16T17:03:30.250242+0000","last_undegraded":"2024-04-16T17:03:30.250242+0000","last_fullsized":"2024-04-16T17:03:30.250242+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:19:13.706889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:30.400057+0000","last_change":"2024-04-16T17:03:24.989456+0000","last_active":"2024-04-16T17:03:30.400057+0000","last_peered":"2024-04-16T17:03:30.400057+0000","last_clean":"2024-04-16T17:03:30.400057+0000","last_became_active":"2024-04-16T17:03:24.989161+0000","last_became_peered":"2024-04-16T17:03:24.989161+0000","last_unstale":"2024-04-16T17:03:30.400057+0000","last_undegraded":"2024-04-16T17:03:30.400057+0000","last_fullsized":"2024-04-16T17:03:30.400057+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:52:59.333625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:30.254501+0000","last_change":"2024-04-16T17:03:24.963428+0000","last_active":"2024-04-16T17:03:30.254501+0000","last_peered":"2024-04-16T17:03:30.254501+0000","last_clean":"2024-04-16T17:03:30.254501+0000","last_became_active":"2024-04-16T17:03:24.962523+0000","last_became_peered":"2024-04-16T17:03:24.962523+0000","last_unstale":"2024-04-16T17:03:30.254501+0000","last_undegraded":"2024-04-16T17:03:30.254501+0000","last_fullsized":"2024-04-16T17:03:30.254501+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:22:19.523901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:30.251382+0000","last_change":"2024-04-16T17:03:24.981122+0000","last_active":"2024-04-16T17:03:30.251382+0000","last_peered":"2024-04-16T17:03:30.251382+0000","last_clean":"2024-04-16T17:03:30.251382+0000","last_became_active":"2024-04-16T17:03:24.980533+0000","last_became_peered":"2024-04-16T17:03:24.980533+0000","last_unstale":"2024-04-16T17:03:30.251382+0000","last_undegraded":"2024-04-16T17:03:30.251382+0000","last_fullsized":"2024-04-16T17:03:30.251382+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:51:08.580965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:30.251470+0000","last_change":"2024-04-16T17:03:29.257239+0000","last_active":"2024-04-16T17:03:30.251470+0000","last_peered":"2024-04-16T17:03:30.251470+0000","last_clean":"2024-04-16T17:03:30.251470+0000","last_became_active":"2024-04-16T17:03:29.256827+0000","last_became_peered":"2024-04-16T17:03:29.256827+0000","last_unstale":"2024-04-16T17:03:30.251470+0000","last_undegraded":"2024-04-16T17:03:30.251470+0000","last_fullsized":"2024-04-16T17:03:30.251470+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:57:28.373386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"90'9","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:52.384026+0000","last_change":"2024-04-16T17:03:24.157157+0000","last_active":"2024-04-16T17:03:52.384026+0000","last_peered":"2024-04-16T17:03:52.384026+0000","last_clean":"2024-04-16T17:03:52.384026+0000","last_became_active":"2024-04-16T17:03:24.154813+0000","last_became_peered":"2024-04-16T17:03:24.154813+0000","last_unstale":"2024-04-16T17:03:52.384026+0000","last_undegraded":"2024-04-16T17:03:52.384026+0000","last_fullsized":"2024-04-16T17:03:52.384026+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:39:31.343583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:30.512100+0000","last_change":"2024-04-16T17:03:24.994921+0000","last_active":"2024-04-16T17:03:30.512100+0000","last_peered":"2024-04-16T17:03:30.512100+0000","last_clean":"2024-04-16T17:03:30.512100+0000","last_became_active":"2024-04-16T17:03:24.994412+0000","last_became_peered":"2024-04-16T17:03:24.994412+0000","last_unstale":"2024-04-16T17:03:30.512100+0000","last_undegraded":"2024-04-16T17:03:30.512100+0000","last_fullsized":"2024-04-16T17:03:30.512100+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:11:40.748198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:30.511269+0000","last_change":"2024-04-16T17:03:29.259884+0000","last_active":"2024-04-16T17:03:30.511269+0000","last_peered":"2024-04-16T17:03:30.511269+0000","last_clean":"2024-04-16T17:03:30.511269+0000","last_became_active":"2024-04-16T17:03:29.259319+0000","last_became_peered":"2024-04-16T17:03:29.259319+0000","last_unstale":"2024-04-16T17:03:30.511269+0000","last_undegraded":"2024-04-16T17:03:30.511269+0000","last_fullsized":"2024-04-16T17:03:30.511269+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:05:18.244682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"90'2","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:30.512508+0000","last_change":"2024-04-16T17:03:24.179308+0000","last_active":"2024-04-16T17:03:30.512508+0000","last_peered":"2024-04-16T17:03:30.512508+0000","last_clean":"2024-04-16T17:03:30.512508+0000","last_became_active":"2024-04-16T17:03:24.178894+0000","last_became_peered":"2024-04-16T17:03:24.178894+0000","last_unstale":"2024-04-16T17:03:30.512508+0000","last_undegraded":"2024-04-16T17:03:30.512508+0000","last_fullsized":"2024-04-16T17:03:30.512508+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:17:46.831871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:30.512411+0000","last_change":"2024-04-16T17:03:24.982391+0000","last_active":"2024-04-16T17:03:30.512411+0000","last_peered":"2024-04-16T17:03:30.512411+0000","last_clean":"2024-04-16T17:03:30.512411+0000","last_became_active":"2024-04-16T17:03:24.981761+0000","last_became_peered":"2024-04-16T17:03:24.981761+0000","last_unstale":"2024-04-16T17:03:30.512411+0000","last_undegraded":"2024-04-16T17:03:30.512411+0000","last_fullsized":"2024-04-16T17:03:30.512411+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:40:57.657170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:30.512302+0000","last_change":"2024-04-16T17:03:29.259705+0000","last_active":"2024-04-16T17:03:30.512302+0000","last_peered":"2024-04-16T17:03:30.512302+0000","last_clean":"2024-04-16T17:03:30.512302+0000","last_became_active":"2024-04-16T17:03:29.259336+0000","last_became_peered":"2024-04-16T17:03:29.259336+0000","last_unstale":"2024-04-16T17:03:30.512302+0000","last_undegraded":"2024-04-16T17:03:30.512302+0000","last_fullsized":"2024-04-16T17:03:30.512302+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:53:29.042933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"85'2","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:30.511835+0000","last_change":"2024-04-16T17:03:24.152641+0000","last_active":"2024-04-16T17:03:30.511835+0000","last_peered":"2024-04-16T17:03:30.511835+0000","last_clean":"2024-04-16T17:03:30.511835+0000","last_became_active":"2024-04-16T17:03:24.152310+0000","last_became_peered":"2024-04-16T17:03:24.152310+0000","last_unstale":"2024-04-16T17:03:30.511835+0000","last_undegraded":"2024-04-16T17:03:30.511835+0000","last_fullsized":"2024-04-16T17:03:30.511835+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:50:11.362075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:30.250552+0000","last_change":"2024-04-16T17:03:24.973205+0000","last_active":"2024-04-16T17:03:30.250552+0000","last_peered":"2024-04-16T17:03:30.250552+0000","last_clean":"2024-04-16T17:03:30.250552+0000","last_became_active":"2024-04-16T17:03:24.972863+0000","last_became_peered":"2024-04-16T17:03:24.972863+0000","last_unstale":"2024-04-16T17:03:30.250552+0000","last_undegraded":"2024-04-16T17:03:30.250552+0000","last_fullsized":"2024-04-16T17:03:30.250552+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:28:29.005783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:30.511362+0000","last_change":"2024-04-16T17:03:29.262930+0000","last_active":"2024-04-16T17:03:30.511362+0000","last_peered":"2024-04-16T17:03:30.511362+0000","last_clean":"2024-04-16T17:03:30.511362+0000","last_became_active":"2024-04-16T17:03:29.262621+0000","last_became_peered":"2024-04-16T17:03:29.262621+0000","last_unstale":"2024-04-16T17:03:30.511362+0000","last_undegraded":"2024-04-16T17:03:30.511362+0000","last_fullsized":"2024-04-16T17:03:30.511362+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:22:31.555143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:30.511245+0000","last_change":"2024-04-16T17:03:24.150503+0000","last_active":"2024-04-16T17:03:30.511245+0000","last_peered":"2024-04-16T17:03:30.511245+0000","last_clean":"2024-04-16T17:03:30.511245+0000","last_became_active":"2024-04-16T17:03:24.150021+0000","last_became_peered":"2024-04-16T17:03:24.150021+0000","last_unstale":"2024-04-16T17:03:30.511245+0000","last_undegraded":"2024-04-16T17:03:30.511245+0000","last_fullsized":"2024-04-16T17:03:30.511245+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:21:36.119497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:30.511870+0000","last_change":"2024-04-16T17:03:24.965049+0000","last_active":"2024-04-16T17:03:30.511870+0000","last_peered":"2024-04-16T17:03:30.511870+0000","last_clean":"2024-04-16T17:03:30.511870+0000","last_became_active":"2024-04-16T17:03:24.963120+0000","last_became_peered":"2024-04-16T17:03:24.963120+0000","last_unstale":"2024-04-16T17:03:30.511870+0000","last_undegraded":"2024-04-16T17:03:30.511870+0000","last_fullsized":"2024-04-16T17:03:30.511870+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:17:49.725113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:30.251263+0000","last_change":"2024-04-16T17:03:29.247013+0000","last_active":"2024-04-16T17:03:30.251263+0000","last_peered":"2024-04-16T17:03:30.251263+0000","last_clean":"2024-04-16T17:03:30.251263+0000","last_became_active":"2024-04-16T17:03:29.246477+0000","last_became_peered":"2024-04-16T17:03:29.246477+0000","last_unstale":"2024-04-16T17:03:30.251263+0000","last_undegraded":"2024-04-16T17:03:30.251263+0000","last_fullsized":"2024-04-16T17:03:30.251263+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:52:19.525076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"90'6","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:54.393619+0000","last_change":"2024-04-16T17:03:24.155333+0000","last_active":"2024-04-16T17:03:54.393619+0000","last_peered":"2024-04-16T17:03:54.393619+0000","last_clean":"2024-04-16T17:03:54.393619+0000","last_became_active":"2024-04-16T17:03:24.154848+0000","last_became_peered":"2024-04-16T17:03:24.154848+0000","last_unstale":"2024-04-16T17:03:54.393619+0000","last_undegraded":"2024-04-16T17:03:54.393619+0000","last_fullsized":"2024-04-16T17:03:54.393619+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:03:34.769573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:30.510981+0000","last_change":"2024-04-16T17:03:24.967627+0000","last_active":"2024-04-16T17:03:30.510981+0000","last_peered":"2024-04-16T17:03:30.510981+0000","last_clean":"2024-04-16T17:03:30.510981+0000","last_became_active":"2024-04-16T17:03:24.967264+0000","last_became_peered":"2024-04-16T17:03:24.967264+0000","last_unstale":"2024-04-16T17:03:30.510981+0000","last_undegraded":"2024-04-16T17:03:30.510981+0000","last_fullsized":"2024-04-16T17:03:30.510981+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:35:29.182374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:30.251205+0000","last_change":"2024-04-16T17:03:29.264625+0000","last_active":"2024-04-16T17:03:30.251205+0000","last_peered":"2024-04-16T17:03:30.251205+0000","last_clean":"2024-04-16T17:03:30.251205+0000","last_became_active":"2024-04-16T17:03:29.264114+0000","last_became_peered":"2024-04-16T17:03:29.264114+0000","last_unstale":"2024-04-16T17:03:30.251205+0000","last_undegraded":"2024-04-16T17:03:30.251205+0000","last_fullsized":"2024-04-16T17:03:30.251205+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:14:49.449980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"90'3","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:52.382001+0000","last_change":"2024-04-16T17:03:24.160457+0000","last_active":"2024-04-16T17:03:52.382001+0000","last_peered":"2024-04-16T17:03:52.382001+0000","last_clean":"2024-04-16T17:03:52.382001+0000","last_became_active":"2024-04-16T17:03:24.160195+0000","last_became_peered":"2024-04-16T17:03:24.160195+0000","last_unstale":"2024-04-16T17:03:52.382001+0000","last_undegraded":"2024-04-16T17:03:52.382001+0000","last_fullsized":"2024-04-16T17:03:52.382001+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:23:07.064539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:30.511680+0000","last_change":"2024-04-16T17:03:24.991249+0000","last_active":"2024-04-16T17:03:30.511680+0000","last_peered":"2024-04-16T17:03:30.511680+0000","last_clean":"2024-04-16T17:03:30.511680+0000","last_became_active":"2024-04-16T17:03:24.990927+0000","last_became_peered":"2024-04-16T17:03:24.990927+0000","last_unstale":"2024-04-16T17:03:30.511680+0000","last_undegraded":"2024-04-16T17:03:30.511680+0000","last_fullsized":"2024-04-16T17:03:30.511680+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:36:39.975616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:30.400144+0000","last_change":"2024-04-16T17:03:29.253704+0000","last_active":"2024-04-16T17:03:30.400144+0000","last_peered":"2024-04-16T17:03:30.400144+0000","last_clean":"2024-04-16T17:03:30.400144+0000","last_became_active":"2024-04-16T17:03:29.253334+0000","last_became_peered":"2024-04-16T17:03:29.253334+0000","last_unstale":"2024-04-16T17:03:30.400144+0000","last_undegraded":"2024-04-16T17:03:30.400144+0000","last_fullsized":"2024-04-16T17:03:30.400144+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:19:12.611731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"90'2","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:52.386383+0000","last_change":"2024-04-16T17:03:24.158875+0000","last_active":"2024-04-16T17:03:52.386383+0000","last_peered":"2024-04-16T17:03:52.386383+0000","last_clean":"2024-04-16T17:03:52.386383+0000","last_became_active":"2024-04-16T17:03:24.158465+0000","last_became_peered":"2024-04-16T17:03:24.158465+0000","last_unstale":"2024-04-16T17:03:52.386383+0000","last_undegraded":"2024-04-16T17:03:52.386383+0000","last_fullsized":"2024-04-16T17:03:52.386383+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:21:29.882251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:30.511047+0000","last_change":"2024-04-16T17:03:24.994476+0000","last_active":"2024-04-16T17:03:30.511047+0000","last_peered":"2024-04-16T17:03:30.511047+0000","last_clean":"2024-04-16T17:03:30.511047+0000","last_became_active":"2024-04-16T17:03:24.993532+0000","last_became_peered":"2024-04-16T17:03:24.993532+0000","last_unstale":"2024-04-16T17:03:30.511047+0000","last_undegraded":"2024-04-16T17:03:30.511047+0000","last_fullsized":"2024-04-16T17:03:30.511047+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:04:44.062444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:30.511983+0000","last_change":"2024-04-16T17:03:29.255446+0000","last_active":"2024-04-16T17:03:30.511983+0000","last_peered":"2024-04-16T17:03:30.511983+0000","last_clean":"2024-04-16T17:03:30.511983+0000","last_became_active":"2024-04-16T17:03:29.255051+0000","last_became_peered":"2024-04-16T17:03:29.255051+0000","last_unstale":"2024-04-16T17:03:30.511983+0000","last_undegraded":"2024-04-16T17:03:30.511983+0000","last_fullsized":"2024-04-16T17:03:30.511983+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:24:04.946342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"90'1","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:30.511394+0000","last_change":"2024-04-16T17:03:24.154904+0000","last_active":"2024-04-16T17:03:30.511394+0000","last_peered":"2024-04-16T17:03:30.511394+0000","last_clean":"2024-04-16T17:03:30.511394+0000","last_became_active":"2024-04-16T17:03:24.154617+0000","last_became_peered":"2024-04-16T17:03:24.154617+0000","last_unstale":"2024-04-16T17:03:30.511394+0000","last_undegraded":"2024-04-16T17:03:30.511394+0000","last_fullsized":"2024-04-16T17:03:30.511394+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:35:55.498855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:30.253013+0000","last_change":"2024-04-16T17:03:24.979908+0000","last_active":"2024-04-16T17:03:30.253013+0000","last_peered":"2024-04-16T17:03:30.253013+0000","last_clean":"2024-04-16T17:03:30.253013+0000","last_became_active":"2024-04-16T17:03:24.979501+0000","last_became_peered":"2024-04-16T17:03:24.979501+0000","last_unstale":"2024-04-16T17:03:30.253013+0000","last_undegraded":"2024-04-16T17:03:30.253013+0000","last_fullsized":"2024-04-16T17:03:30.253013+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:51:05.421459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:30.511352+0000","last_change":"2024-04-16T17:03:29.258809+0000","last_active":"2024-04-16T17:03:30.511352+0000","last_peered":"2024-04-16T17:03:30.511352+0000","last_clean":"2024-04-16T17:03:30.511352+0000","last_became_active":"2024-04-16T17:03:29.256629+0000","last_became_peered":"2024-04-16T17:03:29.256629+0000","last_unstale":"2024-04-16T17:03:30.511352+0000","last_undegraded":"2024-04-16T17:03:30.511352+0000","last_fullsized":"2024-04-16T17:03:30.511352+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:46:51.634944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"90'3","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:51.374270+0000","last_change":"2024-04-16T17:03:24.153774+0000","last_active":"2024-04-16T17:03:51.374270+0000","last_peered":"2024-04-16T17:03:51.374270+0000","last_clean":"2024-04-16T17:03:51.374270+0000","last_became_active":"2024-04-16T17:03:24.153418+0000","last_became_peered":"2024-04-16T17:03:24.153418+0000","last_unstale":"2024-04-16T17:03:51.374270+0000","last_undegraded":"2024-04-16T17:03:51.374270+0000","last_fullsized":"2024-04-16T17:03:51.374270+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:58:36.093789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:30.251009+0000","last_change":"2024-04-16T17:03:24.962838+0000","last_active":"2024-04-16T17:03:30.251009+0000","last_peered":"2024-04-16T17:03:30.251009+0000","last_clean":"2024-04-16T17:03:30.251009+0000","last_became_active":"2024-04-16T17:03:24.960163+0000","last_became_peered":"2024-04-16T17:03:24.960163+0000","last_unstale":"2024-04-16T17:03:30.251009+0000","last_undegraded":"2024-04-16T17:03:30.251009+0000","last_fullsized":"2024-04-16T17:03:30.251009+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:48:18.723499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:30.253053+0000","last_change":"2024-04-16T17:03:29.251177+0000","last_active":"2024-04-16T17:03:30.253053+0000","last_peered":"2024-04-16T17:03:30.253053+0000","last_clean":"2024-04-16T17:03:30.253053+0000","last_became_active":"2024-04-16T17:03:29.250657+0000","last_became_peered":"2024-04-16T17:03:29.250657+0000","last_unstale":"2024-04-16T17:03:30.253053+0000","last_undegraded":"2024-04-16T17:03:30.253053+0000","last_fullsized":"2024-04-16T17:03:30.253053+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:16:47.217193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"90'2","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:51.372780+0000","last_change":"2024-04-16T17:03:24.154245+0000","last_active":"2024-04-16T17:03:51.372780+0000","last_peered":"2024-04-16T17:03:51.372780+0000","last_clean":"2024-04-16T17:03:51.372780+0000","last_became_active":"2024-04-16T17:03:24.153297+0000","last_became_peered":"2024-04-16T17:03:24.153297+0000","last_unstale":"2024-04-16T17:03:51.372780+0000","last_undegraded":"2024-04-16T17:03:51.372780+0000","last_fullsized":"2024-04-16T17:03:51.372780+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:23:49.490238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:30.512505+0000","last_change":"2024-04-16T17:03:24.984572+0000","last_active":"2024-04-16T17:03:30.512505+0000","last_peered":"2024-04-16T17:03:30.512505+0000","last_clean":"2024-04-16T17:03:30.512505+0000","last_became_active":"2024-04-16T17:03:24.983588+0000","last_became_peered":"2024-04-16T17:03:24.983588+0000","last_unstale":"2024-04-16T17:03:30.512505+0000","last_undegraded":"2024-04-16T17:03:30.512505+0000","last_fullsized":"2024-04-16T17:03:30.512505+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:41:24.704602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:30.512717+0000","last_change":"2024-04-16T17:03:29.245129+0000","last_active":"2024-04-16T17:03:30.512717+0000","last_peered":"2024-04-16T17:03:30.512717+0000","last_clean":"2024-04-16T17:03:30.512717+0000","last_became_active":"2024-04-16T17:03:29.244720+0000","last_became_peered":"2024-04-16T17:03:29.244720+0000","last_unstale":"2024-04-16T17:03:30.512717+0000","last_undegraded":"2024-04-16T17:03:30.512717+0000","last_fullsized":"2024-04-16T17:03:30.512717+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:55:26.612176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"90'4","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:51.372651+0000","last_change":"2024-04-16T17:03:24.165248+0000","last_active":"2024-04-16T17:03:51.372651+0000","last_peered":"2024-04-16T17:03:51.372651+0000","last_clean":"2024-04-16T17:03:51.372651+0000","last_became_active":"2024-04-16T17:03:24.164915+0000","last_became_peered":"2024-04-16T17:03:24.164915+0000","last_unstale":"2024-04-16T17:03:51.372651+0000","last_undegraded":"2024-04-16T17:03:51.372651+0000","last_fullsized":"2024-04-16T17:03:51.372651+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:04:47.207554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:30.250692+0000","last_change":"2024-04-16T17:03:24.967292+0000","last_active":"2024-04-16T17:03:30.250692+0000","last_peered":"2024-04-16T17:03:30.250692+0000","last_clean":"2024-04-16T17:03:30.250692+0000","last_became_active":"2024-04-16T17:03:24.966952+0000","last_became_peered":"2024-04-16T17:03:24.966952+0000","last_unstale":"2024-04-16T17:03:30.250692+0000","last_undegraded":"2024-04-16T17:03:30.250692+0000","last_fullsized":"2024-04-16T17:03:30.250692+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:42:54.969478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:30.254085+0000","last_change":"2024-04-16T17:03:29.257836+0000","last_active":"2024-04-16T17:03:30.254085+0000","last_peered":"2024-04-16T17:03:30.254085+0000","last_clean":"2024-04-16T17:03:30.254085+0000","last_became_active":"2024-04-16T17:03:29.257409+0000","last_became_peered":"2024-04-16T17:03:29.257409+0000","last_unstale":"2024-04-16T17:03:30.254085+0000","last_undegraded":"2024-04-16T17:03:30.254085+0000","last_fullsized":"2024-04-16T17:03:30.254085+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:25:21.288266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"90'15","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:53.902286+0000","last_change":"2024-04-16T17:03:24.150943+0000","last_active":"2024-04-16T17:03:53.902286+0000","last_peered":"2024-04-16T17:03:53.902286+0000","last_clean":"2024-04-16T17:03:53.902286+0000","last_became_active":"2024-04-16T17:03:24.150563+0000","last_became_peered":"2024-04-16T17:03:24.150563+0000","last_unstale":"2024-04-16T17:03:53.902286+0000","last_undegraded":"2024-04-16T17:03:53.902286+0000","last_fullsized":"2024-04-16T17:03:53.902286+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:05:07.380440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10442,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":6,"num_write":16,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:30.511933+0000","last_change":"2024-04-16T17:03:24.966393+0000","last_active":"2024-04-16T17:03:30.511933+0000","last_peered":"2024-04-16T17:03:30.511933+0000","last_clean":"2024-04-16T17:03:30.511933+0000","last_became_active":"2024-04-16T17:03:24.966016+0000","last_became_peered":"2024-04-16T17:03:24.966016+0000","last_unstale":"2024-04-16T17:03:30.511933+0000","last_undegraded":"2024-04-16T17:03:30.511933+0000","last_fullsized":"2024-04-16T17:03:30.511933+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:30:25.203262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:30.512504+0000","last_change":"2024-04-16T17:03:29.265371+0000","last_active":"2024-04-16T17:03:30.512504+0000","last_peered":"2024-04-16T17:03:30.512504+0000","last_clean":"2024-04-16T17:03:30.512504+0000","last_became_active":"2024-04-16T17:03:29.264933+0000","last_became_peered":"2024-04-16T17:03:29.264933+0000","last_unstale":"2024-04-16T17:03:30.512504+0000","last_undegraded":"2024-04-16T17:03:30.512504+0000","last_fullsized":"2024-04-16T17:03:30.512504+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:50:18.203015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"90'6","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:52.380772+0000","last_change":"2024-04-16T17:03:24.166899+0000","last_active":"2024-04-16T17:03:52.380772+0000","last_peered":"2024-04-16T17:03:52.380772+0000","last_clean":"2024-04-16T17:03:52.380772+0000","last_became_active":"2024-04-16T17:03:24.166529+0000","last_became_peered":"2024-04-16T17:03:24.166529+0000","last_unstale":"2024-04-16T17:03:52.380772+0000","last_undegraded":"2024-04-16T17:03:52.380772+0000","last_fullsized":"2024-04-16T17:03:52.380772+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:59:07.156150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:30.663656+0000","last_change":"2024-04-16T17:03:24.986228+0000","last_active":"2024-04-16T17:03:30.663656+0000","last_peered":"2024-04-16T17:03:30.663656+0000","last_clean":"2024-04-16T17:03:30.663656+0000","last_became_active":"2024-04-16T17:03:24.985916+0000","last_became_peered":"2024-04-16T17:03:24.985916+0000","last_unstale":"2024-04-16T17:03:30.663656+0000","last_undegraded":"2024-04-16T17:03:30.663656+0000","last_fullsized":"2024-04-16T17:03:30.663656+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:23:54.309203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:30.400136+0000","last_change":"2024-04-16T17:03:29.264636+0000","last_active":"2024-04-16T17:03:30.400136+0000","last_peered":"2024-04-16T17:03:30.400136+0000","last_clean":"2024-04-16T17:03:30.400136+0000","last_became_active":"2024-04-16T17:03:29.264117+0000","last_became_peered":"2024-04-16T17:03:29.264117+0000","last_unstale":"2024-04-16T17:03:30.400136+0000","last_undegraded":"2024-04-16T17:03:30.400136+0000","last_fullsized":"2024-04-16T17:03:30.400136+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:42:36.123154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"90'3","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:51.375761+0000","last_change":"2024-04-16T17:03:24.166844+0000","last_active":"2024-04-16T17:03:51.375761+0000","last_peered":"2024-04-16T17:03:51.375761+0000","last_clean":"2024-04-16T17:03:51.375761+0000","last_became_active":"2024-04-16T17:03:24.166479+0000","last_became_peered":"2024-04-16T17:03:24.166479+0000","last_unstale":"2024-04-16T17:03:51.375761+0000","last_undegraded":"2024-04-16T17:03:51.375761+0000","last_fullsized":"2024-04-16T17:03:51.375761+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:02:20.139865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:30.512599+0000","last_change":"2024-04-16T17:03:24.973057+0000","last_active":"2024-04-16T17:03:30.512599+0000","last_peered":"2024-04-16T17:03:30.512599+0000","last_clean":"2024-04-16T17:03:30.512599+0000","last_became_active":"2024-04-16T17:03:24.972731+0000","last_became_peered":"2024-04-16T17:03:24.972731+0000","last_unstale":"2024-04-16T17:03:30.512599+0000","last_undegraded":"2024-04-16T17:03:30.512599+0000","last_fullsized":"2024-04-16T17:03:30.512599+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:37:38.702919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:30.253798+0000","last_change":"2024-04-16T17:03:29.259167+0000","last_active":"2024-04-16T17:03:30.253798+0000","last_peered":"2024-04-16T17:03:30.253798+0000","last_clean":"2024-04-16T17:03:30.253798+0000","last_became_active":"2024-04-16T17:03:29.258737+0000","last_became_peered":"2024-04-16T17:03:29.258737+0000","last_unstale":"2024-04-16T17:03:30.253798+0000","last_undegraded":"2024-04-16T17:03:30.253798+0000","last_fullsized":"2024-04-16T17:03:30.253798+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:01:13.774037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:30.511357+0000","last_change":"2024-04-16T17:03:24.151064+0000","last_active":"2024-04-16T17:03:30.511357+0000","last_peered":"2024-04-16T17:03:30.511357+0000","last_clean":"2024-04-16T17:03:30.511357+0000","last_became_active":"2024-04-16T17:03:24.150745+0000","last_became_peered":"2024-04-16T17:03:24.150745+0000","last_unstale":"2024-04-16T17:03:30.511357+0000","last_undegraded":"2024-04-16T17:03:30.511357+0000","last_fullsized":"2024-04-16T17:03:30.511357+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:32:26.271008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:30.511652+0000","last_change":"2024-04-16T17:03:24.971948+0000","last_active":"2024-04-16T17:03:30.511652+0000","last_peered":"2024-04-16T17:03:30.511652+0000","last_clean":"2024-04-16T17:03:30.511652+0000","last_became_active":"2024-04-16T17:03:24.971530+0000","last_became_peered":"2024-04-16T17:03:24.971530+0000","last_unstale":"2024-04-16T17:03:30.511652+0000","last_undegraded":"2024-04-16T17:03:30.511652+0000","last_fullsized":"2024-04-16T17:03:30.511652+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:03:29.286160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:30.665300+0000","last_change":"2024-04-16T17:03:29.264821+0000","last_active":"2024-04-16T17:03:30.665300+0000","last_peered":"2024-04-16T17:03:30.665300+0000","last_clean":"2024-04-16T17:03:30.665300+0000","last_became_active":"2024-04-16T17:03:29.264287+0000","last_became_peered":"2024-04-16T17:03:29.264287+0000","last_unstale":"2024-04-16T17:03:30.665300+0000","last_undegraded":"2024-04-16T17:03:30.665300+0000","last_fullsized":"2024-04-16T17:03:30.665300+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:33:28.400663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"90'10","reported_seq":33,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:51.372926+0000","last_change":"2024-04-16T17:03:24.158844+0000","last_active":"2024-04-16T17:03:51.372926+0000","last_peered":"2024-04-16T17:03:51.372926+0000","last_clean":"2024-04-16T17:03:51.372926+0000","last_became_active":"2024-04-16T17:03:24.158537+0000","last_became_peered":"2024-04-16T17:03:24.158537+0000","last_unstale":"2024-04-16T17:03:51.372926+0000","last_undegraded":"2024-04-16T17:03:51.372926+0000","last_fullsized":"2024-04-16T17:03:51.372926+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:42:22.772452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:30.511509+0000","last_change":"2024-04-16T17:03:24.962522+0000","last_active":"2024-04-16T17:03:30.511509+0000","last_peered":"2024-04-16T17:03:30.511509+0000","last_clean":"2024-04-16T17:03:30.511509+0000","last_became_active":"2024-04-16T17:03:24.962169+0000","last_became_peered":"2024-04-16T17:03:24.962169+0000","last_unstale":"2024-04-16T17:03:30.511509+0000","last_undegraded":"2024-04-16T17:03:30.511509+0000","last_fullsized":"2024-04-16T17:03:30.511509+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:28:17.147491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:30.511508+0000","last_change":"2024-04-16T17:03:29.259539+0000","last_active":"2024-04-16T17:03:30.511508+0000","last_peered":"2024-04-16T17:03:30.511508+0000","last_clean":"2024-04-16T17:03:30.511508+0000","last_became_active":"2024-04-16T17:03:29.259123+0000","last_became_peered":"2024-04-16T17:03:29.259123+0000","last_unstale":"2024-04-16T17:03:30.511508+0000","last_undegraded":"2024-04-16T17:03:30.511508+0000","last_fullsized":"2024-04-16T17:03:30.511508+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:15:17.142636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"85'1","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:30.511961+0000","last_change":"2024-04-16T17:03:24.147526+0000","last_active":"2024-04-16T17:03:30.511961+0000","last_peered":"2024-04-16T17:03:30.511961+0000","last_clean":"2024-04-16T17:03:30.511961+0000","last_became_active":"2024-04-16T17:03:24.147119+0000","last_became_peered":"2024-04-16T17:03:24.147119+0000","last_unstale":"2024-04-16T17:03:30.511961+0000","last_undegraded":"2024-04-16T17:03:30.511961+0000","last_fullsized":"2024-04-16T17:03:30.511961+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:30:38.742122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:30.400101+0000","last_change":"2024-04-16T17:03:24.970179+0000","last_active":"2024-04-16T17:03:30.400101+0000","last_peered":"2024-04-16T17:03:30.400101+0000","last_clean":"2024-04-16T17:03:30.400101+0000","last_became_active":"2024-04-16T17:03:24.969819+0000","last_became_peered":"2024-04-16T17:03:24.969819+0000","last_unstale":"2024-04-16T17:03:30.400101+0000","last_undegraded":"2024-04-16T17:03:30.400101+0000","last_fullsized":"2024-04-16T17:03:30.400101+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:40:19.897034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:30.511208+0000","last_change":"2024-04-16T17:03:29.301135+0000","last_active":"2024-04-16T17:03:30.511208+0000","last_peered":"2024-04-16T17:03:30.511208+0000","last_clean":"2024-04-16T17:03:30.511208+0000","last_became_active":"2024-04-16T17:03:29.300672+0000","last_became_peered":"2024-04-16T17:03:29.300672+0000","last_unstale":"2024-04-16T17:03:30.511208+0000","last_undegraded":"2024-04-16T17:03:30.511208+0000","last_fullsized":"2024-04-16T17:03:30.511208+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:55:41.599984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:30.254270+0000","last_change":"2024-04-16T17:03:24.162105+0000","last_active":"2024-04-16T17:03:30.254270+0000","last_peered":"2024-04-16T17:03:30.254270+0000","last_clean":"2024-04-16T17:03:30.254270+0000","last_became_active":"2024-04-16T17:03:24.161631+0000","last_became_peered":"2024-04-16T17:03:24.161631+0000","last_unstale":"2024-04-16T17:03:30.254270+0000","last_undegraded":"2024-04-16T17:03:30.254270+0000","last_fullsized":"2024-04-16T17:03:30.254270+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:22:33.677848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:30.511660+0000","last_change":"2024-04-16T17:03:24.978253+0000","last_active":"2024-04-16T17:03:30.511660+0000","last_peered":"2024-04-16T17:03:30.511660+0000","last_clean":"2024-04-16T17:03:30.511660+0000","last_became_active":"2024-04-16T17:03:24.977956+0000","last_became_peered":"2024-04-16T17:03:24.977956+0000","last_unstale":"2024-04-16T17:03:30.511660+0000","last_undegraded":"2024-04-16T17:03:30.511660+0000","last_fullsized":"2024-04-16T17:03:30.511660+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:59:45.102383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:30.664893+0000","last_change":"2024-04-16T17:03:29.265312+0000","last_active":"2024-04-16T17:03:30.664893+0000","last_peered":"2024-04-16T17:03:30.664893+0000","last_clean":"2024-04-16T17:03:30.664893+0000","last_became_active":"2024-04-16T17:03:29.264891+0000","last_became_peered":"2024-04-16T17:03:29.264891+0000","last_unstale":"2024-04-16T17:03:30.664893+0000","last_undegraded":"2024-04-16T17:03:30.664893+0000","last_fullsized":"2024-04-16T17:03:30.664893+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:26:40.088127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"90'1","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:30.511321+0000","last_change":"2024-04-16T17:03:24.170935+0000","last_active":"2024-04-16T17:03:30.511321+0000","last_peered":"2024-04-16T17:03:30.511321+0000","last_clean":"2024-04-16T17:03:30.511321+0000","last_became_active":"2024-04-16T17:03:24.170609+0000","last_became_peered":"2024-04-16T17:03:24.170609+0000","last_unstale":"2024-04-16T17:03:30.511321+0000","last_undegraded":"2024-04-16T17:03:30.511321+0000","last_fullsized":"2024-04-16T17:03:30.511321+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:18:33.516309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:30.511334+0000","last_change":"2024-04-16T17:03:24.985434+0000","last_active":"2024-04-16T17:03:30.511334+0000","last_peered":"2024-04-16T17:03:30.511334+0000","last_clean":"2024-04-16T17:03:30.511334+0000","last_became_active":"2024-04-16T17:03:24.984388+0000","last_became_peered":"2024-04-16T17:03:24.984388+0000","last_unstale":"2024-04-16T17:03:30.511334+0000","last_undegraded":"2024-04-16T17:03:30.511334+0000","last_fullsized":"2024-04-16T17:03:30.511334+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:43:26.355572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:30.664876+0000","last_change":"2024-04-16T17:03:29.301455+0000","last_active":"2024-04-16T17:03:30.664876+0000","last_peered":"2024-04-16T17:03:30.664876+0000","last_clean":"2024-04-16T17:03:30.664876+0000","last_became_active":"2024-04-16T17:03:29.301028+0000","last_became_peered":"2024-04-16T17:03:29.301028+0000","last_unstale":"2024-04-16T17:03:30.664876+0000","last_undegraded":"2024-04-16T17:03:30.664876+0000","last_fullsized":"2024-04-16T17:03:30.664876+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:54:49.305379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:30.511787+0000","last_change":"2024-04-16T17:03:24.962786+0000","last_active":"2024-04-16T17:03:30.511787+0000","last_peered":"2024-04-16T17:03:30.511787+0000","last_clean":"2024-04-16T17:03:30.511787+0000","last_became_active":"2024-04-16T17:03:24.961704+0000","last_became_peered":"2024-04-16T17:03:24.961704+0000","last_unstale":"2024-04-16T17:03:30.511787+0000","last_undegraded":"2024-04-16T17:03:30.511787+0000","last_fullsized":"2024-04-16T17:03:30.511787+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:22:27.957607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"90'4","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:54.394175+0000","last_change":"2024-04-16T17:03:24.175617+0000","last_active":"2024-04-16T17:03:54.394175+0000","last_peered":"2024-04-16T17:03:54.394175+0000","last_clean":"2024-04-16T17:03:54.394175+0000","last_became_active":"2024-04-16T17:03:24.175272+0000","last_became_peered":"2024-04-16T17:03:24.175272+0000","last_unstale":"2024-04-16T17:03:54.394175+0000","last_undegraded":"2024-04-16T17:03:54.394175+0000","last_fullsized":"2024-04-16T17:03:54.394175+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:27:44.738030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:30.512775+0000","last_change":"2024-04-16T17:03:29.243467+0000","last_active":"2024-04-16T17:03:30.512775+0000","last_peered":"2024-04-16T17:03:30.512775+0000","last_clean":"2024-04-16T17:03:30.512775+0000","last_became_active":"2024-04-16T17:03:29.242746+0000","last_became_peered":"2024-04-16T17:03:29.242746+0000","last_unstale":"2024-04-16T17:03:30.512775+0000","last_undegraded":"2024-04-16T17:03:30.512775+0000","last_fullsized":"2024-04-16T17:03:30.512775+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:43:23.893380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:30.511885+0000","last_change":"2024-04-16T17:03:24.983629+0000","last_active":"2024-04-16T17:03:30.511885+0000","last_peered":"2024-04-16T17:03:30.511885+0000","last_clean":"2024-04-16T17:03:30.511885+0000","last_became_active":"2024-04-16T17:03:24.979969+0000","last_became_peered":"2024-04-16T17:03:24.979969+0000","last_unstale":"2024-04-16T17:03:30.511885+0000","last_undegraded":"2024-04-16T17:03:30.511885+0000","last_fullsized":"2024-04-16T17:03:30.511885+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:01:24.391732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"90'2","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:51.374054+0000","last_change":"2024-04-16T17:03:24.152475+0000","last_active":"2024-04-16T17:03:51.374054+0000","last_peered":"2024-04-16T17:03:51.374054+0000","last_clean":"2024-04-16T17:03:51.374054+0000","last_became_active":"2024-04-16T17:03:24.152198+0000","last_became_peered":"2024-04-16T17:03:24.152198+0000","last_unstale":"2024-04-16T17:03:51.374054+0000","last_undegraded":"2024-04-16T17:03:51.374054+0000","last_fullsized":"2024-04-16T17:03:51.374054+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:49:33.914325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":355,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:30.511441+0000","last_change":"2024-04-16T17:02:02.111335+0000","last_active":"2024-04-16T17:03:30.511441+0000","last_peered":"2024-04-16T17:03:30.511441+0000","last_clean":"2024-04-16T17:03:30.511441+0000","last_became_active":"2024-04-16T17:02:02.110928+0000","last_became_peered":"2024-04-16T17:02:02.110928+0000","last_unstale":"2024-04-16T17:03:30.511441+0000","last_undegraded":"2024-04-16T17:03:30.511441+0000","last_fullsized":"2024-04-16T17:03:30.511441+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T16:58:07.081753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T16:58:07.081753+0000","last_clean_scrub_stamp":"2024-04-16T16:58:07.081753+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:27:02.865664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:30.511786+0000","last_change":"2024-04-16T17:03:29.301691+0000","last_active":"2024-04-16T17:03:30.511786+0000","last_peered":"2024-04-16T17:03:30.511786+0000","last_clean":"2024-04-16T17:03:30.511786+0000","last_became_active":"2024-04-16T17:03:29.301277+0000","last_became_peered":"2024-04-16T17:03:29.301277+0000","last_unstale":"2024-04-16T17:03:30.511786+0000","last_undegraded":"2024-04-16T17:03:30.511786+0000","last_fullsized":"2024-04-16T17:03:30.511786+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:45:16.727610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:30.663231+0000","last_change":"2024-04-16T17:03:24.970939+0000","last_active":"2024-04-16T17:03:30.663231+0000","last_peered":"2024-04-16T17:03:30.663231+0000","last_clean":"2024-04-16T17:03:30.663231+0000","last_became_active":"2024-04-16T17:03:24.970581+0000","last_became_peered":"2024-04-16T17:03:24.970581+0000","last_unstale":"2024-04-16T17:03:30.663231+0000","last_undegraded":"2024-04-16T17:03:30.663231+0000","last_fullsized":"2024-04-16T17:03:30.663231+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:33:07.448597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"90'3","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:52.378753+0000","last_change":"2024-04-16T17:03:24.155454+0000","last_active":"2024-04-16T17:03:52.378753+0000","last_peered":"2024-04-16T17:03:52.378753+0000","last_clean":"2024-04-16T17:03:52.378753+0000","last_became_active":"2024-04-16T17:03:24.155155+0000","last_became_peered":"2024-04-16T17:03:24.155155+0000","last_unstale":"2024-04-16T17:03:52.378753+0000","last_undegraded":"2024-04-16T17:03:52.378753+0000","last_fullsized":"2024-04-16T17:03:52.378753+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:34:08.618002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:30.400042+0000","last_change":"2024-04-16T17:03:29.255538+0000","last_active":"2024-04-16T17:03:30.400042+0000","last_peered":"2024-04-16T17:03:30.400042+0000","last_clean":"2024-04-16T17:03:30.400042+0000","last_became_active":"2024-04-16T17:03:29.255107+0000","last_became_peered":"2024-04-16T17:03:29.255107+0000","last_unstale":"2024-04-16T17:03:30.400042+0000","last_undegraded":"2024-04-16T17:03:30.400042+0000","last_fullsized":"2024-04-16T17:03:30.400042+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:31:58.458641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"90'1","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:30.511711+0000","last_change":"2024-04-16T17:03:24.158816+0000","last_active":"2024-04-16T17:03:30.511711+0000","last_peered":"2024-04-16T17:03:30.511711+0000","last_clean":"2024-04-16T17:03:30.511711+0000","last_became_active":"2024-04-16T17:03:24.158518+0000","last_became_peered":"2024-04-16T17:03:24.158518+0000","last_unstale":"2024-04-16T17:03:30.511711+0000","last_undegraded":"2024-04-16T17:03:30.511711+0000","last_fullsized":"2024-04-16T17:03:30.511711+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:54:55.807975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:30.664497+0000","last_change":"2024-04-16T17:03:24.977190+0000","last_active":"2024-04-16T17:03:30.664497+0000","last_peered":"2024-04-16T17:03:30.664497+0000","last_clean":"2024-04-16T17:03:30.664497+0000","last_became_active":"2024-04-16T17:03:24.976840+0000","last_became_peered":"2024-04-16T17:03:24.976840+0000","last_unstale":"2024-04-16T17:03:30.664497+0000","last_undegraded":"2024-04-16T17:03:30.664497+0000","last_fullsized":"2024-04-16T17:03:30.664497+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:21:51.294150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:30.511059+0000","last_change":"2024-04-16T17:03:29.301423+0000","last_active":"2024-04-16T17:03:30.511059+0000","last_peered":"2024-04-16T17:03:30.511059+0000","last_clean":"2024-04-16T17:03:30.511059+0000","last_became_active":"2024-04-16T17:03:29.300888+0000","last_became_peered":"2024-04-16T17:03:29.300888+0000","last_unstale":"2024-04-16T17:03:30.511059+0000","last_undegraded":"2024-04-16T17:03:30.511059+0000","last_fullsized":"2024-04-16T17:03:30.511059+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:13:54.188090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"90'3","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:52.386116+0000","last_change":"2024-04-16T17:03:24.167643+0000","last_active":"2024-04-16T17:03:52.386116+0000","last_peered":"2024-04-16T17:03:52.386116+0000","last_clean":"2024-04-16T17:03:52.386116+0000","last_became_active":"2024-04-16T17:03:24.167103+0000","last_became_peered":"2024-04-16T17:03:24.167103+0000","last_unstale":"2024-04-16T17:03:52.386116+0000","last_undegraded":"2024-04-16T17:03:52.386116+0000","last_fullsized":"2024-04-16T17:03:52.386116+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:33:16.809371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:30.512376+0000","last_change":"2024-04-16T17:03:24.976981+0000","last_active":"2024-04-16T17:03:30.512376+0000","last_peered":"2024-04-16T17:03:30.512376+0000","last_clean":"2024-04-16T17:03:30.512376+0000","last_became_active":"2024-04-16T17:03:24.975385+0000","last_became_peered":"2024-04-16T17:03:24.975385+0000","last_unstale":"2024-04-16T17:03:30.512376+0000","last_undegraded":"2024-04-16T17:03:30.512376+0000","last_fullsized":"2024-04-16T17:03:30.512376+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:27:03.104838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:30.250799+0000","last_change":"2024-04-16T17:03:29.244767+0000","last_active":"2024-04-16T17:03:30.250799+0000","last_peered":"2024-04-16T17:03:30.250799+0000","last_clean":"2024-04-16T17:03:30.250799+0000","last_became_active":"2024-04-16T17:03:29.244205+0000","last_became_peered":"2024-04-16T17:03:29.244205+0000","last_unstale":"2024-04-16T17:03:30.250799+0000","last_undegraded":"2024-04-16T17:03:30.250799+0000","last_fullsized":"2024-04-16T17:03:30.250799+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:51:46.652387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"90'4","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:51.372155+0000","last_change":"2024-04-16T17:03:24.164916+0000","last_active":"2024-04-16T17:03:51.372155+0000","last_peered":"2024-04-16T17:03:51.372155+0000","last_clean":"2024-04-16T17:03:51.372155+0000","last_became_active":"2024-04-16T17:03:24.164648+0000","last_became_peered":"2024-04-16T17:03:24.164648+0000","last_unstale":"2024-04-16T17:03:51.372155+0000","last_undegraded":"2024-04-16T17:03:51.372155+0000","last_fullsized":"2024-04-16T17:03:51.372155+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:23:00.198785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:30.512143+0000","last_change":"2024-04-16T17:03:24.994393+0000","last_active":"2024-04-16T17:03:30.512143+0000","last_peered":"2024-04-16T17:03:30.512143+0000","last_clean":"2024-04-16T17:03:30.512143+0000","last_became_active":"2024-04-16T17:03:24.994002+0000","last_became_peered":"2024-04-16T17:03:24.994002+0000","last_unstale":"2024-04-16T17:03:30.512143+0000","last_undegraded":"2024-04-16T17:03:30.512143+0000","last_fullsized":"2024-04-16T17:03:30.512143+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:23:44.596781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:30.512679+0000","last_change":"2024-04-16T17:03:29.302009+0000","last_active":"2024-04-16T17:03:30.512679+0000","last_peered":"2024-04-16T17:03:30.512679+0000","last_clean":"2024-04-16T17:03:30.512679+0000","last_became_active":"2024-04-16T17:03:29.301183+0000","last_became_peered":"2024-04-16T17:03:29.301183+0000","last_unstale":"2024-04-16T17:03:30.512679+0000","last_undegraded":"2024-04-16T17:03:30.512679+0000","last_fullsized":"2024-04-16T17:03:30.512679+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:30:19.561875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"90'2","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:51.373227+0000","last_change":"2024-04-16T17:03:24.176829+0000","last_active":"2024-04-16T17:03:51.373227+0000","last_peered":"2024-04-16T17:03:51.373227+0000","last_clean":"2024-04-16T17:03:51.373227+0000","last_became_active":"2024-04-16T17:03:24.176555+0000","last_became_peered":"2024-04-16T17:03:24.176555+0000","last_unstale":"2024-04-16T17:03:51.373227+0000","last_undegraded":"2024-04-16T17:03:51.373227+0000","last_fullsized":"2024-04-16T17:03:51.373227+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:05:40.602150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:30.510743+0000","last_change":"2024-04-16T17:03:24.965746+0000","last_active":"2024-04-16T17:03:30.510743+0000","last_peered":"2024-04-16T17:03:30.510743+0000","last_clean":"2024-04-16T17:03:30.510743+0000","last_became_active":"2024-04-16T17:03:24.965241+0000","last_became_peered":"2024-04-16T17:03:24.965241+0000","last_unstale":"2024-04-16T17:03:30.510743+0000","last_undegraded":"2024-04-16T17:03:30.510743+0000","last_fullsized":"2024-04-16T17:03:30.510743+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:51:34.407565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:30.254317+0000","last_change":"2024-04-16T17:03:29.261441+0000","last_active":"2024-04-16T17:03:30.254317+0000","last_peered":"2024-04-16T17:03:30.254317+0000","last_clean":"2024-04-16T17:03:30.254317+0000","last_became_active":"2024-04-16T17:03:29.259538+0000","last_became_peered":"2024-04-16T17:03:29.259538+0000","last_unstale":"2024-04-16T17:03:30.254317+0000","last_undegraded":"2024-04-16T17:03:30.254317+0000","last_fullsized":"2024-04-16T17:03:30.254317+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:12:27.092053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"90'3","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:51.375506+0000","last_change":"2024-04-16T17:03:24.172053+0000","last_active":"2024-04-16T17:03:51.375506+0000","last_peered":"2024-04-16T17:03:51.375506+0000","last_clean":"2024-04-16T17:03:51.375506+0000","last_became_active":"2024-04-16T17:03:24.171709+0000","last_became_peered":"2024-04-16T17:03:24.171709+0000","last_unstale":"2024-04-16T17:03:51.375506+0000","last_undegraded":"2024-04-16T17:03:51.375506+0000","last_fullsized":"2024-04-16T17:03:51.375506+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:40:12.307756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:30.399747+0000","last_change":"2024-04-16T17:03:24.984267+0000","last_active":"2024-04-16T17:03:30.399747+0000","last_peered":"2024-04-16T17:03:30.399747+0000","last_clean":"2024-04-16T17:03:30.399747+0000","last_became_active":"2024-04-16T17:03:24.983888+0000","last_became_peered":"2024-04-16T17:03:24.983888+0000","last_unstale":"2024-04-16T17:03:30.399747+0000","last_undegraded":"2024-04-16T17:03:30.399747+0000","last_fullsized":"2024-04-16T17:03:30.399747+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:16:23.814917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:30.512852+0000","last_change":"2024-04-16T17:03:29.251716+0000","last_active":"2024-04-16T17:03:30.512852+0000","last_peered":"2024-04-16T17:03:30.512852+0000","last_clean":"2024-04-16T17:03:30.512852+0000","last_became_active":"2024-04-16T17:03:29.251317+0000","last_became_peered":"2024-04-16T17:03:29.251317+0000","last_unstale":"2024-04-16T17:03:30.512852+0000","last_undegraded":"2024-04-16T17:03:30.512852+0000","last_fullsized":"2024-04-16T17:03:30.512852+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:48:54.680195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"90'1","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:30.250985+0000","last_change":"2024-04-16T17:03:24.167511+0000","last_active":"2024-04-16T17:03:30.250985+0000","last_peered":"2024-04-16T17:03:30.250985+0000","last_clean":"2024-04-16T17:03:30.250985+0000","last_became_active":"2024-04-16T17:03:24.167170+0000","last_became_peered":"2024-04-16T17:03:24.167170+0000","last_unstale":"2024-04-16T17:03:30.250985+0000","last_undegraded":"2024-04-16T17:03:30.250985+0000","last_fullsized":"2024-04-16T17:03:30.250985+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:31:49.892269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:30.511721+0000","last_change":"2024-04-16T17:03:24.995547+0000","last_active":"2024-04-16T17:03:30.511721+0000","last_peered":"2024-04-16T17:03:30.511721+0000","last_clean":"2024-04-16T17:03:30.511721+0000","last_became_active":"2024-04-16T17:03:24.995096+0000","last_became_peered":"2024-04-16T17:03:24.995096+0000","last_unstale":"2024-04-16T17:03:30.511721+0000","last_undegraded":"2024-04-16T17:03:30.511721+0000","last_fullsized":"2024-04-16T17:03:30.511721+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:54:51.193117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:30.512519+0000","last_change":"2024-04-16T17:03:29.301526+0000","last_active":"2024-04-16T17:03:30.512519+0000","last_peered":"2024-04-16T17:03:30.512519+0000","last_clean":"2024-04-16T17:03:30.512519+0000","last_became_active":"2024-04-16T17:03:29.300808+0000","last_became_peered":"2024-04-16T17:03:29.300808+0000","last_unstale":"2024-04-16T17:03:30.512519+0000","last_undegraded":"2024-04-16T17:03:30.512519+0000","last_fullsized":"2024-04-16T17:03:30.512519+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:01:14.717548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"90'4","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:51.372404+0000","last_change":"2024-04-16T17:03:24.152645+0000","last_active":"2024-04-16T17:03:51.372404+0000","last_peered":"2024-04-16T17:03:51.372404+0000","last_clean":"2024-04-16T17:03:51.372404+0000","last_became_active":"2024-04-16T17:03:24.152281+0000","last_became_peered":"2024-04-16T17:03:24.152281+0000","last_unstale":"2024-04-16T17:03:51.372404+0000","last_undegraded":"2024-04-16T17:03:51.372404+0000","last_fullsized":"2024-04-16T17:03:51.372404+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:33:51.377421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:30.511887+0000","last_change":"2024-04-16T17:03:24.979827+0000","last_active":"2024-04-16T17:03:30.511887+0000","last_peered":"2024-04-16T17:03:30.511887+0000","last_clean":"2024-04-16T17:03:30.511887+0000","last_became_active":"2024-04-16T17:03:24.979452+0000","last_became_peered":"2024-04-16T17:03:24.979452+0000","last_unstale":"2024-04-16T17:03:30.511887+0000","last_undegraded":"2024-04-16T17:03:30.511887+0000","last_fullsized":"2024-04-16T17:03:30.511887+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:07:09.259366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:30.250611+0000","last_change":"2024-04-16T17:03:29.255519+0000","last_active":"2024-04-16T17:03:30.250611+0000","last_peered":"2024-04-16T17:03:30.250611+0000","last_clean":"2024-04-16T17:03:30.250611+0000","last_became_active":"2024-04-16T17:03:29.255104+0000","last_became_peered":"2024-04-16T17:03:29.255104+0000","last_unstale":"2024-04-16T17:03:30.250611+0000","last_undegraded":"2024-04-16T17:03:30.250611+0000","last_fullsized":"2024-04-16T17:03:30.250611+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:27:34.496199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"90'3","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:52.386419+0000","last_change":"2024-04-16T17:03:24.161145+0000","last_active":"2024-04-16T17:03:52.386419+0000","last_peered":"2024-04-16T17:03:52.386419+0000","last_clean":"2024-04-16T17:03:52.386419+0000","last_became_active":"2024-04-16T17:03:24.160824+0000","last_became_peered":"2024-04-16T17:03:24.160824+0000","last_unstale":"2024-04-16T17:03:52.386419+0000","last_undegraded":"2024-04-16T17:03:52.386419+0000","last_fullsized":"2024-04-16T17:03:52.386419+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:02:48.805638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:30.513026+0000","last_change":"2024-04-16T17:03:24.966513+0000","last_active":"2024-04-16T17:03:30.513026+0000","last_peered":"2024-04-16T17:03:30.513026+0000","last_clean":"2024-04-16T17:03:30.513026+0000","last_became_active":"2024-04-16T17:03:24.966074+0000","last_became_peered":"2024-04-16T17:03:24.966074+0000","last_unstale":"2024-04-16T17:03:30.513026+0000","last_undegraded":"2024-04-16T17:03:30.513026+0000","last_fullsized":"2024-04-16T17:03:30.513026+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:50:21.579901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:30.511846+0000","last_change":"2024-04-16T17:03:29.244236+0000","last_active":"2024-04-16T17:03:30.511846+0000","last_peered":"2024-04-16T17:03:30.511846+0000","last_clean":"2024-04-16T17:03:30.511846+0000","last_became_active":"2024-04-16T17:03:29.242809+0000","last_became_peered":"2024-04-16T17:03:29.242809+0000","last_unstale":"2024-04-16T17:03:30.511846+0000","last_undegraded":"2024-04-16T17:03:30.511846+0000","last_fullsized":"2024-04-16T17:03:30.511846+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:38:01.788299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"85'1","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:30.512271+0000","last_change":"2024-04-16T17:03:24.158032+0000","last_active":"2024-04-16T17:03:30.512271+0000","last_peered":"2024-04-16T17:03:30.512271+0000","last_clean":"2024-04-16T17:03:30.512271+0000","last_became_active":"2024-04-16T17:03:24.157689+0000","last_became_peered":"2024-04-16T17:03:24.157689+0000","last_unstale":"2024-04-16T17:03:30.512271+0000","last_undegraded":"2024-04-16T17:03:30.512271+0000","last_fullsized":"2024-04-16T17:03:30.512271+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:38:18.240889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:30.512181+0000","last_change":"2024-04-16T17:03:24.977780+0000","last_active":"2024-04-16T17:03:30.512181+0000","last_peered":"2024-04-16T17:03:30.512181+0000","last_clean":"2024-04-16T17:03:30.512181+0000","last_became_active":"2024-04-16T17:03:24.977397+0000","last_became_peered":"2024-04-16T17:03:24.977397+0000","last_unstale":"2024-04-16T17:03:30.512181+0000","last_undegraded":"2024-04-16T17:03:30.512181+0000","last_fullsized":"2024-04-16T17:03:30.512181+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:37:30.701763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:30.512202+0000","last_change":"2024-04-16T17:03:29.245176+0000","last_active":"2024-04-16T17:03:30.512202+0000","last_peered":"2024-04-16T17:03:30.512202+0000","last_clean":"2024-04-16T17:03:30.512202+0000","last_became_active":"2024-04-16T17:03:29.244763+0000","last_became_peered":"2024-04-16T17:03:29.244763+0000","last_unstale":"2024-04-16T17:03:30.512202+0000","last_undegraded":"2024-04-16T17:03:30.512202+0000","last_fullsized":"2024-04-16T17:03:30.512202+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:52:30.381417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:30.664851+0000","last_change":"2024-04-16T17:03:24.168068+0000","last_active":"2024-04-16T17:03:30.664851+0000","last_peered":"2024-04-16T17:03:30.664851+0000","last_clean":"2024-04-16T17:03:30.664851+0000","last_became_active":"2024-04-16T17:03:24.167694+0000","last_became_peered":"2024-04-16T17:03:24.167694+0000","last_unstale":"2024-04-16T17:03:30.664851+0000","last_undegraded":"2024-04-16T17:03:30.664851+0000","last_fullsized":"2024-04-16T17:03:30.664851+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:13:35.692306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:30.513055+0000","last_change":"2024-04-16T17:03:24.966145+0000","last_active":"2024-04-16T17:03:30.513055+0000","last_peered":"2024-04-16T17:03:30.513055+0000","last_clean":"2024-04-16T17:03:30.513055+0000","last_became_active":"2024-04-16T17:03:24.965813+0000","last_became_peered":"2024-04-16T17:03:24.965813+0000","last_unstale":"2024-04-16T17:03:30.513055+0000","last_undegraded":"2024-04-16T17:03:30.513055+0000","last_fullsized":"2024-04-16T17:03:30.513055+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:43:43.224130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:30.399863+0000","last_change":"2024-04-16T17:03:24.970941+0000","last_active":"2024-04-16T17:03:30.399863+0000","last_peered":"2024-04-16T17:03:30.399863+0000","last_clean":"2024-04-16T17:03:30.399863+0000","last_became_active":"2024-04-16T17:03:24.970587+0000","last_became_peered":"2024-04-16T17:03:24.970587+0000","last_unstale":"2024-04-16T17:03:30.399863+0000","last_undegraded":"2024-04-16T17:03:30.399863+0000","last_fullsized":"2024-04-16T17:03:30.399863+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:14:13.930011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:30.251358+0000","last_change":"2024-04-16T17:03:24.967167+0000","last_active":"2024-04-16T17:03:30.251358+0000","last_peered":"2024-04-16T17:03:30.251358+0000","last_clean":"2024-04-16T17:03:30.251358+0000","last_became_active":"2024-04-16T17:03:24.966787+0000","last_became_peered":"2024-04-16T17:03:24.966787+0000","last_unstale":"2024-04-16T17:03:30.251358+0000","last_undegraded":"2024-04-16T17:03:30.251358+0000","last_fullsized":"2024-04-16T17:03:30.251358+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:51:39.510220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:30.250802+0000","last_change":"2024-04-16T17:03:24.981115+0000","last_active":"2024-04-16T17:03:30.250802+0000","last_peered":"2024-04-16T17:03:30.250802+0000","last_clean":"2024-04-16T17:03:30.250802+0000","last_became_active":"2024-04-16T17:03:24.980685+0000","last_became_peered":"2024-04-16T17:03:24.980685+0000","last_unstale":"2024-04-16T17:03:30.250802+0000","last_undegraded":"2024-04-16T17:03:30.250802+0000","last_fullsized":"2024-04-16T17:03:30.250802+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:20:44.066932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:30.511844+0000","last_change":"2024-04-16T17:03:24.990421+0000","last_active":"2024-04-16T17:03:30.511844+0000","last_peered":"2024-04-16T17:03:30.511844+0000","last_clean":"2024-04-16T17:03:30.511844+0000","last_became_active":"2024-04-16T17:03:24.989940+0000","last_became_peered":"2024-04-16T17:03:24.989940+0000","last_unstale":"2024-04-16T17:03:30.511844+0000","last_undegraded":"2024-04-16T17:03:30.511844+0000","last_fullsized":"2024-04-16T17:03:30.511844+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:19:57.548067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:30.664720+0000","last_change":"2024-04-16T17:03:24.991854+0000","last_active":"2024-04-16T17:03:30.664720+0000","last_peered":"2024-04-16T17:03:30.664720+0000","last_clean":"2024-04-16T17:03:30.664720+0000","last_became_active":"2024-04-16T17:03:24.990964+0000","last_became_peered":"2024-04-16T17:03:24.990964+0000","last_unstale":"2024-04-16T17:03:30.664720+0000","last_undegraded":"2024-04-16T17:03:30.664720+0000","last_fullsized":"2024-04-16T17:03:30.664720+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:10:10.618708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:30.400274+0000","last_change":"2024-04-16T17:03:24.971332+0000","last_active":"2024-04-16T17:03:30.400274+0000","last_peered":"2024-04-16T17:03:30.400274+0000","last_clean":"2024-04-16T17:03:30.400274+0000","last_became_active":"2024-04-16T17:03:24.970975+0000","last_became_peered":"2024-04-16T17:03:24.970975+0000","last_unstale":"2024-04-16T17:03:30.400274+0000","last_undegraded":"2024-04-16T17:03:30.400274+0000","last_fullsized":"2024-04-16T17:03:30.400274+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:18:01.046445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:30.251550+0000","last_change":"2024-04-16T17:03:24.970377+0000","last_active":"2024-04-16T17:03:30.251550+0000","last_peered":"2024-04-16T17:03:30.251550+0000","last_clean":"2024-04-16T17:03:30.251550+0000","last_became_active":"2024-04-16T17:03:24.968856+0000","last_became_peered":"2024-04-16T17:03:24.968856+0000","last_unstale":"2024-04-16T17:03:30.251550+0000","last_undegraded":"2024-04-16T17:03:30.251550+0000","last_fullsized":"2024-04-16T17:03:30.251550+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:47:28.543120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:30.512482+0000","last_change":"2024-04-16T17:03:24.992877+0000","last_active":"2024-04-16T17:03:30.512482+0000","last_peered":"2024-04-16T17:03:30.512482+0000","last_clean":"2024-04-16T17:03:30.512482+0000","last_became_active":"2024-04-16T17:03:24.992484+0000","last_became_peered":"2024-04-16T17:03:24.992484+0000","last_unstale":"2024-04-16T17:03:30.512482+0000","last_undegraded":"2024-04-16T17:03:30.512482+0000","last_fullsized":"2024-04-16T17:03:30.512482+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:00:45.204782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:30.511453+0000","last_change":"2024-04-16T17:03:24.977794+0000","last_active":"2024-04-16T17:03:30.511453+0000","last_peered":"2024-04-16T17:03:30.511453+0000","last_clean":"2024-04-16T17:03:30.511453+0000","last_became_active":"2024-04-16T17:03:24.977434+0000","last_became_peered":"2024-04-16T17:03:24.977434+0000","last_unstale":"2024-04-16T17:03:30.511453+0000","last_undegraded":"2024-04-16T17:03:30.511453+0000","last_fullsized":"2024-04-16T17:03:30.511453+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:46:35.670357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:30.511470+0000","last_change":"2024-04-16T17:03:24.992225+0000","last_active":"2024-04-16T17:03:30.511470+0000","last_peered":"2024-04-16T17:03:30.511470+0000","last_clean":"2024-04-16T17:03:30.511470+0000","last_became_active":"2024-04-16T17:03:24.991904+0000","last_became_peered":"2024-04-16T17:03:24.991904+0000","last_unstale":"2024-04-16T17:03:30.511470+0000","last_undegraded":"2024-04-16T17:03:30.511470+0000","last_fullsized":"2024-04-16T17:03:30.511470+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:01:09.710698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:30.250680+0000","last_change":"2024-04-16T17:03:24.981322+0000","last_active":"2024-04-16T17:03:30.250680+0000","last_peered":"2024-04-16T17:03:30.250680+0000","last_clean":"2024-04-16T17:03:30.250680+0000","last_became_active":"2024-04-16T17:03:24.980920+0000","last_became_peered":"2024-04-16T17:03:24.980920+0000","last_unstale":"2024-04-16T17:03:30.250680+0000","last_undegraded":"2024-04-16T17:03:30.250680+0000","last_fullsized":"2024-04-16T17:03:30.250680+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:07:59.378512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:30.663982+0000","last_change":"2024-04-16T17:03:24.990117+0000","last_active":"2024-04-16T17:03:30.663982+0000","last_peered":"2024-04-16T17:03:30.663982+0000","last_clean":"2024-04-16T17:03:30.663982+0000","last_became_active":"2024-04-16T17:03:24.989734+0000","last_became_peered":"2024-04-16T17:03:24.989734+0000","last_unstale":"2024-04-16T17:03:30.663982+0000","last_undegraded":"2024-04-16T17:03:30.663982+0000","last_fullsized":"2024-04-16T17:03:30.663982+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:36:55.000702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-16T17:03:30.400073+0000","last_change":"2024-04-16T17:03:24.970032+0000","last_active":"2024-04-16T17:03:30.400073+0000","last_peered":"2024-04-16T17:03:30.400073+0000","last_clean":"2024-04-16T17:03:30.400073+0000","last_became_active":"2024-04-16T17:03:24.968608+0000","last_became_peered":"2024-04-16T17:03:24.968608+0000","last_unstale":"2024-04-16T17:03:30.400073+0000","last_undegraded":"2024-04-16T17:03:30.400073+0000","last_fullsized":"2024-04-16T17:03:30.400073+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:24:03.918418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":283858,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":13709,"num_objects":52,"num_object_clones":0,"num_object_copies":156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":0,"num_read":18,"num_read_kb":17,"num_write":122,"num_write_kb":42,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":36,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":241664,"data_stored":47910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":105,"ondisk_log_size":105,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892350995,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":978944,"data_stored":484944,"data_compressed":14704,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612634,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32216,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712936,"statfs":{"total":95995035648,"available":95962046464,"internally_reserved":0,"allocated":1064960,"data_stored":1076054,"data_compressed":50802,"data_compressed_allocated":65536,"data_compressed_original":630784,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776158,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32140,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713012,"statfs":{"total":95995035648,"available":95962124288,"internally_reserved":0,"allocated":987136,"data_stored":481050,"data_compressed":14704,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037797,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32136,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713016,"statfs":{"total":95995035648,"available":95962128384,"internally_reserved":0,"allocated":983040,"data_stored":480892,"data_compressed":14704,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299435,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32184,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712968,"statfs":{"total":95995035648,"available":95962079232,"internally_reserved":0,"allocated":1032192,"data_stored":1071838,"data_compressed":50802,"data_compressed_allocated":65536,"data_compressed_original":630784,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462961,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32128,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713024,"statfs":{"total":95995035648,"available":95962136576,"internally_reserved":0,"allocated":974848,"data_stored":480356,"data_compressed":14704,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574925,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32136,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713016,"statfs":{"total":95995035648,"available":95962128384,"internally_reserved":0,"allocated":983040,"data_stored":484108,"data_compressed":14704,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738449,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32136,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713016,"statfs":{"total":95995035648,"available":95962128384,"internally_reserved":0,"allocated":974848,"data_stored":480356,"data_compressed":14704,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411399,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1028096,"data_stored":505148,"data_compressed":14704,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149761,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32140,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713012,"statfs":{"total":95995035648,"available":95962124288,"internally_reserved":0,"allocated":987136,"data_stored":480550,"data_compressed":14704,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888123,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":974848,"data_stored":480288,"data_compressed":14704,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724599,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1052672,"data_stored":1076597,"data_compressed":50802,"data_compressed_allocated":65536,"data_compressed_original":630784,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34456,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34456,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34456,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36098,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36098,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36098,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36098,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36098,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":24882,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":12411,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-16T17:03:57.998 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-16T17:03:57.998 INFO:tasks.cephadm:Setting up client nodes... 2024-04-16T17:03:57.999 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b1fecd38-fc11-11ee-bc90-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-04-16T17:03:58.515 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/mon.a/config 2024-04-16T17:03:58.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:03:58 smithi104 ceph-mon[36457]: from='client.24869 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-16T17:03:58.711 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:03:58 smithi045 ceph-mon[31676]: from='client.24869 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-16T17:03:58.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:03:58 smithi117 ceph-mon[36730]: from='client.24869 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-16T17:03:59.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:03:59 smithi104 ceph-mon[36457]: from='client.24871 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-16T17:03:59.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:03:59 smithi104 ceph-mon[36457]: pgmap v339: 129 pgs: 129 active+clean; 591 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 3.3 KiB/s wr, 10 op/s 2024-04-16T17:03:59.724 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:03:59 smithi045 ceph-mon[31676]: from='client.24871 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-16T17:03:59.724 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:03:59 smithi045 ceph-mon[31676]: pgmap v339: 129 pgs: 129 active+clean; 591 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 3.3 KiB/s wr, 10 op/s 2024-04-16T17:03:59.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:03:59 smithi117 ceph-mon[36730]: from='client.24871 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-16T17:03:59.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:03:59 smithi117 ceph-mon[36730]: pgmap v339: 129 pgs: 129 active+clean; 591 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 3.3 KiB/s wr, 10 op/s 2024-04-16T17:04:00.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:00.838+0000 7f260b200700 1 -- 172.21.15.45:0/4266015869 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f260410f160 msgr2=0x7f2604111550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:00.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:00.838+0000 7f260b200700 1 --2- 172.21.15.45:0/4266015869 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f260410f160 0x7f2604111550 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f25fc00b0a0 tx=0x7f25fc00b3b0 comp rx=0 tx=0).stop 2024-04-16T17:04:00.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:00.838+0000 7f260b200700 1 -- 172.21.15.45:0/4266015869 shutdown_connections 2024-04-16T17:04:00.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:00.838+0000 7f260b200700 1 --2- 172.21.15.45:0/4266015869 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f260410f160 0x7f2604111550 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:00.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:00.838+0000 7f260b200700 1 --2- 172.21.15.45:0/4266015869 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2604076b20 0x7f2604074f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:00.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:00.838+0000 7f260b200700 1 --2- 172.21.15.45:0/4266015869 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2604075550 0x7f26040759d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:00.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:00.838+0000 7f260b200700 1 -- 172.21.15.45:0/4266015869 >> 172.21.15.45:0/4266015869 conn(0x7f26040fe140 msgr2=0x7f2604100560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:04:00.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:00.838+0000 7f260b200700 1 -- 172.21.15.45:0/4266015869 shutdown_connections 2024-04-16T17:04:00.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:00.838+0000 7f260b200700 1 -- 172.21.15.45:0/4266015869 wait complete. 2024-04-16T17:04:00.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:00.840+0000 7f260b200700 1 Processor -- start 2024-04-16T17:04:00.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:00.840+0000 7f260b200700 1 -- start start 2024-04-16T17:04:00.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:00.840+0000 7f260b200700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2604075550 0x7f260419f280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:00.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:00.840+0000 7f260b200700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2604076b20 0x7f260419f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:00.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:00.840+0000 7f260b200700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f260410f160 0x7f26041a6840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:00.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:00.840+0000 7f260b200700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f2604114dc0 con 0x7f260410f160 2024-04-16T17:04:00.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:00.840+0000 7f260b200700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f2604114c40 con 0x7f2604076b20 2024-04-16T17:04:00.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:00.840+0000 7f260b200700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f2604114f40 con 0x7f2604075550 2024-04-16T17:04:00.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:00.841+0000 7f2603400700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2604076b20 0x7f260419f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:00.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:00.841+0000 7f2609200700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f260410f160 0x7f26041a6840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:00.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:00.841+0000 7f2603400700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2604076b20 0x7f260419f7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:47386/0 (socket says 172.21.15.45:47386) 2024-04-16T17:04:00.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:00.841+0000 7f2603400700 1 -- 172.21.15.45:0/327567375 learned_addr learned my addr 172.21.15.45:0/327567375 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:04:00.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:00.841+0000 7f2603e00700 1 --2- 172.21.15.45:0/327567375 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2604075550 0x7f260419f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:00.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:00.841+0000 7f2609200700 1 -- 172.21.15.45:0/327567375 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2604075550 msgr2=0x7f260419f280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:00.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:00.841+0000 7f2609200700 1 --2- 172.21.15.45:0/327567375 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2604075550 0x7f260419f280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:00.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:00.841+0000 7f2609200700 1 -- 172.21.15.45:0/327567375 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2604076b20 msgr2=0x7f260419f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:00.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:00.841+0000 7f2609200700 1 --2- 172.21.15.45:0/327567375 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2604076b20 0x7f260419f7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:00.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:00.841+0000 7f2609200700 1 -- 172.21.15.45:0/327567375 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26041a6ee0 con 0x7f260410f160 2024-04-16T17:04:00.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:00.842+0000 7f2609200700 1 --2- 172.21.15.45:0/327567375 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f260410f160 0x7f26041a6840 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f25fc007fa0 tx=0x7f25fc004c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:04:00.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:00.842+0000 7f2600c00700 1 -- 172.21.15.45:0/327567375 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25fc021070 con 0x7f260410f160 2024-04-16T17:04:00.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:00.842+0000 7f2600c00700 1 -- 172.21.15.45:0/327567375 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f25fc004e50 con 0x7f260410f160 2024-04-16T17:04:00.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:00.842+0000 7f260b200700 1 -- 172.21.15.45:0/327567375 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26041a7170 con 0x7f260410f160 2024-04-16T17:04:00.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:00.842+0000 7f2600c00700 1 -- 172.21.15.45:0/327567375 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25fc004780 con 0x7f260410f160 2024-04-16T17:04:00.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:00.842+0000 7f260b200700 1 -- 172.21.15.45:0/327567375 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26041a7670 con 0x7f260410f160 2024-04-16T17:04:00.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:00.844+0000 7f260b200700 1 -- 172.21.15.45:0/327567375 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2604051030 con 0x7f260410f160 2024-04-16T17:04:00.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:00.847+0000 7f2600c00700 1 -- 172.21.15.45:0/327567375 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f25fc003a00 con 0x7f260410f160 2024-04-16T17:04:00.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:00.849+0000 7f2600c00700 1 --2- 172.21.15.45:0/327567375 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f25e80a3e00 0x7f25e80a62c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:00.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:00.849+0000 7f2600c00700 1 -- 172.21.15.45:0/327567375 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f25fc00f020 con 0x7f260410f160 2024-04-16T17:04:00.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:00.849+0000 7f2603e00700 1 --2- 172.21.15.45:0/327567375 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f25e80a3e00 0x7f25e80a62c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:00.852 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:00.850+0000 7f2603e00700 1 --2- 172.21.15.45:0/327567375 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f25e80a3e00 0x7f25e80a62c0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f2604076f00 tx=0x7f25f4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:04:00.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:00.852+0000 7f2600c00700 1 -- 172.21.15.45:0/327567375 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f25fc02a020 con 0x7f260410f160 2024-04-16T17:04:01.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:01.171+0000 7f260b200700 1 -- 172.21.15.45:0/327567375 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7f260402dfe0 con 0x7f260410f160 2024-04-16T17:04:01.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:01.175+0000 7f2600c00700 1 -- 172.21.15.45:0/327567375 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) ==== 170+0+59 (secure 0 0 0) 0x7f25fc09e560 con 0x7f260410f160 2024-04-16T17:04:01.177 INFO:teuthology.orchestra.run.smithi045.stdout:[client.0] 2024-04-16T17:04:01.177 INFO:teuthology.orchestra.run.smithi045.stdout: key = AQCBrx5m3BxWChAAEN47i8K/EoMGSu+5IPzZSg== 2024-04-16T17:04:01.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:01.178+0000 7f260b200700 1 -- 172.21.15.45:0/327567375 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f25e80a3e00 msgr2=0x7f25e80a62c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:01.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:01.178+0000 7f260b200700 1 --2- 172.21.15.45:0/327567375 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f25e80a3e00 0x7f25e80a62c0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f2604076f00 tx=0x7f25f4009260 comp rx=0 tx=0).stop 2024-04-16T17:04:01.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:01.178+0000 7f260b200700 1 -- 172.21.15.45:0/327567375 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f260410f160 msgr2=0x7f26041a6840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:01.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:01.178+0000 7f260b200700 1 --2- 172.21.15.45:0/327567375 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f260410f160 0x7f26041a6840 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f25fc007fa0 tx=0x7f25fc004c60 comp rx=0 tx=0).stop 2024-04-16T17:04:01.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:01.178+0000 7f260b200700 1 -- 172.21.15.45:0/327567375 shutdown_connections 2024-04-16T17:04:01.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:01.178+0000 7f260b200700 1 --2- 172.21.15.45:0/327567375 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f25e80a3e00 0x7f25e80a62c0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:01.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:01.178+0000 7f260b200700 1 --2- 172.21.15.45:0/327567375 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2604075550 0x7f260419f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:01.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:01.178+0000 7f260b200700 1 --2- 172.21.15.45:0/327567375 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f260410f160 0x7f26041a6840 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:01.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:01.178+0000 7f260b200700 1 --2- 172.21.15.45:0/327567375 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2604076b20 0x7f260419f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:01.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:01.178+0000 7f260b200700 1 -- 172.21.15.45:0/327567375 >> 172.21.15.45:0/327567375 conn(0x7f26040fe140 msgr2=0x7f2604102500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:04:01.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:01.178+0000 7f260b200700 1 -- 172.21.15.45:0/327567375 shutdown_connections 2024-04-16T17:04:01.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:01.178+0000 7f260b200700 1 -- 172.21.15.45:0/327567375 wait complete. 2024-04-16T17:04:01.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:04:01 smithi104 ceph-mon[36457]: pgmap v340: 129 pgs: 129 active+clean; 626 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 6.6 KiB/s wr, 12 op/s 2024-04-16T17:04:01.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:04:01 smithi104 ceph-mon[36457]: from='client.? 172.21.15.45:0/327567375' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-16T17:04:01.679 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:04:01 smithi104 ceph-mon[36457]: from='client.? 172.21.15.45:0/327567375' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-16T17:04:01.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:04:01 smithi045 ceph-mon[31676]: pgmap v340: 129 pgs: 129 active+clean; 626 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 6.6 KiB/s wr, 12 op/s 2024-04-16T17:04:01.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:04:01 smithi045 ceph-mon[31676]: from='client.? 172.21.15.45:0/327567375' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-16T17:04:01.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:04:01 smithi045 ceph-mon[31676]: from='client.? 172.21.15.45:0/327567375' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-16T17:04:01.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:04:01 smithi117 ceph-mon[36730]: pgmap v340: 129 pgs: 129 active+clean; 626 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 6.6 KiB/s wr, 12 op/s 2024-04-16T17:04:01.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:04:01 smithi117 ceph-mon[36730]: from='client.? 172.21.15.45:0/327567375' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-16T17:04:01.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:04:01 smithi117 ceph-mon[36730]: from='client.? 172.21.15.45:0/327567375' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-16T17:04:02.030 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-04-16T17:04:02.030 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-04-16T17:04:02.031 DEBUG:teuthology.orchestra.run.smithi045:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-04-16T17:04:02.201 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-04-16T17:04:02.201 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-04-16T17:04:02.201 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45 shell --fsid b1fecd38-fc11-11ee-bc90-c7b262605968 -- ceph mgr dump --format=json 2024-04-16T17:04:02.709 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/mon.a/config 2024-04-16T17:04:03.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:04:03 smithi045 ceph-mon[31676]: pgmap v341: 129 pgs: 129 active+clean; 626 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 6.5 KiB/s wr, 14 op/s 2024-04-16T17:04:03.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:04:03 smithi117 ceph-mon[36730]: pgmap v341: 129 pgs: 129 active+clean; 626 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 6.5 KiB/s wr, 14 op/s 2024-04-16T17:04:03.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:04:03 smithi104 ceph-mon[36457]: pgmap v341: 129 pgs: 129 active+clean; 626 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 6.5 KiB/s wr, 14 op/s 2024-04-16T17:04:05.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:05.119+0000 7fcb4cc00700 1 -- 172.21.15.45:0/2937010438 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcb4810f520 msgr2=0x7fcb48111910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:05.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:05.119+0000 7fcb4cc00700 1 --2- 172.21.15.45:0/2937010438 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcb4810f520 0x7fcb48111910 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fcb4000b0a0 tx=0x7fcb4000b3b0 comp rx=0 tx=0).stop 2024-04-16T17:04:05.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:05.119+0000 7fcb4cc00700 1 -- 172.21.15.45:0/2937010438 shutdown_connections 2024-04-16T17:04:05.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:05.119+0000 7fcb4cc00700 1 --2- 172.21.15.45:0/2937010438 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcb4810f520 0x7fcb48111910 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:05.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:05.119+0000 7fcb4cc00700 1 --2- 172.21.15.45:0/2937010438 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcb48074d20 0x7fcb48075100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:05.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:05.119+0000 7fcb4cc00700 1 --2- 172.21.15.45:0/2937010438 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcb48075640 0x7fcb4810efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:05.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:05.119+0000 7fcb4cc00700 1 -- 172.21.15.45:0/2937010438 >> 172.21.15.45:0/2937010438 conn(0x7fcb48079bc0 msgr2=0x7fcb48079fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:04:05.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:05.119+0000 7fcb4cc00700 1 -- 172.21.15.45:0/2937010438 shutdown_connections 2024-04-16T17:04:05.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:05.119+0000 7fcb4cc00700 1 -- 172.21.15.45:0/2937010438 wait complete. 2024-04-16T17:04:05.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:05.120+0000 7fcb4cc00700 1 Processor -- start 2024-04-16T17:04:05.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:05.121+0000 7fcb4cc00700 1 -- start start 2024-04-16T17:04:05.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:05.121+0000 7fcb4cc00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcb48074d20 0x7fcb4819b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:05.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:05.121+0000 7fcb4cc00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcb48075640 0x7fcb4819b920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:05.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:05.121+0000 7fcb4cc00700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcb4810f520 0x7fcb4819fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:05.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:05.121+0000 7fcb4cc00700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fcb48115050 con 0x7fcb48075640 2024-04-16T17:04:05.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:05.121+0000 7fcb4cc00700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fcb48114ed0 con 0x7fcb4810f520 2024-04-16T17:04:05.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:05.121+0000 7fcb4cc00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fcb481151d0 con 0x7fcb48074d20 2024-04-16T17:04:05.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:05.121+0000 7fcb3d400700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcb48075640 0x7fcb4819b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:05.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:05.121+0000 7fcb46800700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcb4810f520 0x7fcb4819fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:05.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:05.122+0000 7fcb45e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcb48074d20 0x7fcb4819b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:05.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:05.122+0000 7fcb46800700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcb4810f520 0x7fcb4819fcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:47402/0 (socket says 172.21.15.45:47402) 2024-04-16T17:04:05.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:05.122+0000 7fcb46800700 1 -- 172.21.15.45:0/4259549106 learned_addr learned my addr 172.21.15.45:0/4259549106 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:04:05.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:05.122+0000 7fcb46800700 1 -- 172.21.15.45:0/4259549106 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcb48074d20 msgr2=0x7fcb4819b3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:05.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:05.122+0000 7fcb46800700 1 --2- 172.21.15.45:0/4259549106 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcb48074d20 0x7fcb4819b3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:05.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:05.122+0000 7fcb46800700 1 -- 172.21.15.45:0/4259549106 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcb48075640 msgr2=0x7fcb4819b920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:05.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:05.122+0000 7fcb46800700 1 --2- 172.21.15.45:0/4259549106 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcb48075640 0x7fcb4819b920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:05.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:05.122+0000 7fcb46800700 1 -- 172.21.15.45:0/4259549106 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb481a03f0 con 0x7fcb4810f520 2024-04-16T17:04:05.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:05.122+0000 7fcb46800700 1 --2- 172.21.15.45:0/4259549106 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcb4810f520 0x7fcb4819fcb0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fcb40006070 tx=0x7fcb40004c60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:04:05.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:05.123+0000 7fcb3f400700 1 -- 172.21.15.45:0/4259549106 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb40021070 con 0x7fcb4810f520 2024-04-16T17:04:05.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:05.123+0000 7fcb3f400700 1 -- 172.21.15.45:0/4259549106 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcb40003c00 con 0x7fcb4810f520 2024-04-16T17:04:05.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:05.123+0000 7fcb4cc00700 1 -- 172.21.15.45:0/4259549106 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb481a0680 con 0x7fcb4810f520 2024-04-16T17:04:05.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:05.123+0000 7fcb3f400700 1 -- 172.21.15.45:0/4259549106 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb400046c0 con 0x7fcb4810f520 2024-04-16T17:04:05.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:05.123+0000 7fcb4cc00700 1 -- 172.21.15.45:0/4259549106 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb481a7f60 con 0x7fcb4810f520 2024-04-16T17:04:05.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:05.124+0000 7fcb4cc00700 1 -- 172.21.15.45:0/4259549106 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb480697e0 con 0x7fcb4810f520 2024-04-16T17:04:05.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:05.126+0000 7fcb3f400700 1 -- 172.21.15.45:0/4259549106 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fcb40007980 con 0x7fcb4810f520 2024-04-16T17:04:05.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:05.127+0000 7fcb3f400700 1 --2- 172.21.15.45:0/4259549106 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fcb2c0a3cd0 0x7fcb2c0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:05.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:05.128+0000 7fcb3f400700 1 -- 172.21.15.45:0/4259549106 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7fcb4000f020 con 0x7fcb4810f520 2024-04-16T17:04:05.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:05.128+0000 7fcb45e00700 1 --2- 172.21.15.45:0/4259549106 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fcb2c0a3cd0 0x7fcb2c0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:05.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:05.128+0000 7fcb45e00700 1 --2- 172.21.15.45:0/4259549106 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fcb2c0a3cd0 0x7fcb2c0a6190 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fcb38009c60 tx=0x7fcb38009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:04:05.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:05.130+0000 7fcb3f400700 1 -- 172.21.15.45:0/4259549106 <== mon.2 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fcb4002a020 con 0x7fcb4810f520 2024-04-16T17:04:05.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:05.426+0000 7fcb4cc00700 1 -- 172.21.15.45:0/4259549106 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7fcb481a8130 con 0x7fcb4810f520 2024-04-16T17:04:05.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:05.432+0000 7fcb3f400700 1 -- 172.21.15.45:0/4259549106 <== mon.2 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+264991 (secure 0 0 0) 0x7fcb4009e530 con 0x7fcb4810f520 2024-04-16T17:04:05.436 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-16T17:04:05.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:05.438+0000 7fcb4cc00700 1 -- 172.21.15.45:0/4259549106 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fcb2c0a3cd0 msgr2=0x7fcb2c0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:05.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:05.438+0000 7fcb4cc00700 1 --2- 172.21.15.45:0/4259549106 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fcb2c0a3cd0 0x7fcb2c0a6190 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fcb38009c60 tx=0x7fcb38009260 comp rx=0 tx=0).stop 2024-04-16T17:04:05.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:05.438+0000 7fcb4cc00700 1 -- 172.21.15.45:0/4259549106 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcb4810f520 msgr2=0x7fcb4819fcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:05.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:05.438+0000 7fcb4cc00700 1 --2- 172.21.15.45:0/4259549106 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcb4810f520 0x7fcb4819fcb0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fcb40006070 tx=0x7fcb40004c60 comp rx=0 tx=0).stop 2024-04-16T17:04:05.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:05.439+0000 7fcb4cc00700 1 -- 172.21.15.45:0/4259549106 shutdown_connections 2024-04-16T17:04:05.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:05.439+0000 7fcb4cc00700 1 --2- 172.21.15.45:0/4259549106 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fcb2c0a3cd0 0x7fcb2c0a6190 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:05.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:05.439+0000 7fcb4cc00700 1 --2- 172.21.15.45:0/4259549106 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcb48074d20 0x7fcb4819b3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:05.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:05.439+0000 7fcb4cc00700 1 --2- 172.21.15.45:0/4259549106 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcb4810f520 0x7fcb4819fcb0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:05.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:05.439+0000 7fcb4cc00700 1 --2- 172.21.15.45:0/4259549106 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcb48075640 0x7fcb4819b920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:05.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:05.439+0000 7fcb4cc00700 1 -- 172.21.15.45:0/4259549106 >> 172.21.15.45:0/4259549106 conn(0x7fcb48079bc0 msgr2=0x7fcb480ffbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:04:05.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:05.440+0000 7fcb4cc00700 1 -- 172.21.15.45:0/4259549106 shutdown_connections 2024-04-16T17:04:05.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:05.440+0000 7fcb4cc00700 1 -- 172.21.15.45:0/4259549106 wait complete. 2024-04-16T17:04:05.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:04:05 smithi045 ceph-mon[31676]: pgmap v342: 129 pgs: 129 active+clean; 626 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 6.1 KiB/s wr, 14 op/s 2024-04-16T17:04:05.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:04:05 smithi045 ceph-mon[31676]: from='client.? 172.21.15.45:0/4259549106' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-16T17:04:05.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:04:05 smithi117 ceph-mon[36730]: pgmap v342: 129 pgs: 129 active+clean; 626 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 6.1 KiB/s wr, 14 op/s 2024-04-16T17:04:05.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:04:05 smithi117 ceph-mon[36730]: from='client.? 172.21.15.45:0/4259549106' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-16T17:04:05.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:04:05 smithi104 ceph-mon[36457]: pgmap v342: 129 pgs: 129 active+clean; 626 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 6.1 KiB/s wr, 14 op/s 2024-04-16T17:04:05.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:04:05 smithi104 ceph-mon[36457]: from='client.? 172.21.15.45:0/4259549106' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-16T17:04:06.351 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":17,"flags":0,"active_gid":14152,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":268030699},{"type":"v1","addr":"172.21.15.45:6801","nonce":268030699}]},"active_addr":"172.21.15.45:6801/268030699","active_change":"2024-04-16T16:55:19.948348+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24109,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24113,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.45:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.45:0","nonce":1915281176}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.45:0","nonce":1500147666}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.45:0","nonce":12001708}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.45:0","nonce":265066921}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.45:0","nonce":414304644}]}]} 2024-04-16T17:04:06.356 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-04-16T17:04:06.356 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-04-16T17:04:06.356 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45 shell --fsid b1fecd38-fc11-11ee-bc90-c7b262605968 -- ceph osd dump --format=json 2024-04-16T17:04:06.877 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/mon.a/config 2024-04-16T17:04:07.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:04:07 smithi045 ceph-mon[31676]: pgmap v343: 129 pgs: 129 active+clean; 638 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 7.2 KiB/s wr, 14 op/s 2024-04-16T17:04:07.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:04:07 smithi117 ceph-mon[36730]: pgmap v343: 129 pgs: 129 active+clean; 638 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 7.2 KiB/s wr, 14 op/s 2024-04-16T17:04:07.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:04:07 smithi104 ceph-mon[36457]: pgmap v343: 129 pgs: 129 active+clean; 638 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 7.2 KiB/s wr, 14 op/s 2024-04-16T17:04:09.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:09.222+0000 7fad9bc00700 1 -- 172.21.15.45:0/4002987754 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fad9406ba30 msgr2=0x7fad9410f9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:09.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:09.222+0000 7fad9bc00700 1 --2- 172.21.15.45:0/4002987754 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fad9406ba30 0x7fad9410f9a0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fad8c00b0a0 tx=0x7fad8c00b3b0 comp rx=0 tx=0).stop 2024-04-16T17:04:09.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:09.223+0000 7fad9bc00700 1 -- 172.21.15.45:0/4002987754 shutdown_connections 2024-04-16T17:04:09.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:09.223+0000 7fad9bc00700 1 --2- 172.21.15.45:0/4002987754 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fad9406ba30 0x7fad9410f9a0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:09.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:09.223+0000 7fad9bc00700 1 --2- 172.21.15.45:0/4002987754 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fad94108b60 0x7fad94108f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:09.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:09.223+0000 7fad9bc00700 1 --2- 172.21.15.45:0/4002987754 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fad9406b090 0x7fad9406b4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:09.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:09.223+0000 7fad9bc00700 1 -- 172.21.15.45:0/4002987754 >> 172.21.15.45:0/4002987754 conn(0x7fad94078960 msgr2=0x7fad94078d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:04:09.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:09.223+0000 7fad9bc00700 1 -- 172.21.15.45:0/4002987754 shutdown_connections 2024-04-16T17:04:09.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:09.223+0000 7fad9bc00700 1 -- 172.21.15.45:0/4002987754 wait complete. 2024-04-16T17:04:09.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:09.224+0000 7fad9bc00700 1 Processor -- start 2024-04-16T17:04:09.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:09.224+0000 7fad9bc00700 1 -- start start 2024-04-16T17:04:09.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:09.224+0000 7fad9bc00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fad9406b090 0x7fad9419b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:09.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:09.224+0000 7fad9bc00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fad9406ba30 0x7fad9419ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:09.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:09.224+0000 7fad9bc00700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fad94108b60 0x7fad9419fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:09.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:09.225+0000 7fad9bc00700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fad94112e10 con 0x7fad9406b090 2024-04-16T17:04:09.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:09.225+0000 7fad9bc00700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fad94112c90 con 0x7fad94108b60 2024-04-16T17:04:09.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:09.225+0000 7fad9bc00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fad94112f90 con 0x7fad9406ba30 2024-04-16T17:04:09.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:09.225+0000 7fad99200700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fad9406b090 0x7fad9419b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:09.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:09.225+0000 7fad8be00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fad9406ba30 0x7fad9419ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:09.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:09.225+0000 7fad8be00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fad9406ba30 0x7fad9419ba20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:33556/0 (socket says 172.21.15.45:33556) 2024-04-16T17:04:09.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:09.225+0000 7fad8be00700 1 -- 172.21.15.45:0/627674829 learned_addr learned my addr 172.21.15.45:0/627674829 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:04:09.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:09.225+0000 7fad99c00700 1 --2- 172.21.15.45:0/627674829 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fad94108b60 0x7fad9419fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:09.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:09.225+0000 7fad8be00700 1 -- 172.21.15.45:0/627674829 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fad94108b60 msgr2=0x7fad9419fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:09.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:09.225+0000 7fad8be00700 1 --2- 172.21.15.45:0/627674829 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fad94108b60 0x7fad9419fdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:09.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:09.225+0000 7fad8be00700 1 -- 172.21.15.45:0/627674829 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fad9406b090 msgr2=0x7fad9419b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:09.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:09.225+0000 7fad8be00700 1 --2- 172.21.15.45:0/627674829 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fad9406b090 0x7fad9419b4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:09.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:09.225+0000 7fad8be00700 1 -- 172.21.15.45:0/627674829 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad941a04f0 con 0x7fad9406ba30 2024-04-16T17:04:09.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:09.225+0000 7fad99200700 1 --2- 172.21.15.45:0/627674829 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fad9406b090 0x7fad9419b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:04:09.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:09.226+0000 7fad8be00700 1 --2- 172.21.15.45:0/627674829 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fad9406ba30 0x7fad9419ba20 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fad7c00e9a0 tx=0x7fad7c00ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:04:09.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:09.226+0000 7fad92000700 1 -- 172.21.15.45:0/627674829 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad7c00ca10 con 0x7fad9406ba30 2024-04-16T17:04:09.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:09.226+0000 7fad92000700 1 -- 172.21.15.45:0/627674829 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fad7c00cbb0 con 0x7fad9406ba30 2024-04-16T17:04:09.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:09.226+0000 7fad9bc00700 1 -- 172.21.15.45:0/627674829 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad941a07e0 con 0x7fad9406ba30 2024-04-16T17:04:09.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:09.226+0000 7fad92000700 1 -- 172.21.15.45:0/627674829 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad7c018680 con 0x7fad9406ba30 2024-04-16T17:04:09.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:09.226+0000 7fad9bc00700 1 -- 172.21.15.45:0/627674829 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad941a8060 con 0x7fad9406ba30 2024-04-16T17:04:09.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:09.227+0000 7fad9bc00700 1 -- 172.21.15.45:0/627674829 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad940697e0 con 0x7fad9406ba30 2024-04-16T17:04:09.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:09.230+0000 7fad92000700 1 -- 172.21.15.45:0/627674829 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fad7c010430 con 0x7fad9406ba30 2024-04-16T17:04:09.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:09.231+0000 7fad92000700 1 --2- 172.21.15.45:0/627674829 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fad780a3d50 0x7fad780a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:09.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:09.231+0000 7fad92000700 1 -- 172.21.15.45:0/627674829 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7fad7c0d0e40 con 0x7fad9406ba30 2024-04-16T17:04:09.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:09.231+0000 7fad99200700 1 --2- 172.21.15.45:0/627674829 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fad780a3d50 0x7fad780a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:09.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:09.232+0000 7fad99200700 1 --2- 172.21.15.45:0/627674829 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fad780a3d50 0x7fad780a6210 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fad84009fd0 tx=0x7fad84009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:04:09.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:09.233+0000 7fad92000700 1 -- 172.21.15.45:0/627674829 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fad7c09d420 con 0x7fad9406ba30 2024-04-16T17:04:09.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:09.474+0000 7fad9bc00700 1 -- 172.21.15.45:0/627674829 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fad9402dfe0 con 0x7fad9406ba30 2024-04-16T17:04:09.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:09.476+0000 7fad92000700 1 -- 172.21.15.45:0/627674829 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+24164 (secure 0 0 0) 0x7fad7c09cd20 con 0x7fad9406ba30 2024-04-16T17:04:09.479 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-16T17:04:09.479 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":90,"fsid":"b1fecd38-fc11-11ee-bc90-c7b262605968","created":"2024-04-16T16:54:02.385906+0000","modified":"2024-04-16T17:03:29.720565+0000","last_up_change":"2024-04-16T17:02:25.162624+0000","last_in_change":"2024-04-16T17:02:04.903274+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-16T16:58:06.166659+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-16T17:03:22.304198+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-16T17:03:23.616450+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-16T17:03:27.226289+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b655de2c-59f8-4fd4-ac98-0cc222229f56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":2290170881},{"type":"v1","addr":"172.21.15.45:6803","nonce":2290170881}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":2290170881},{"type":"v1","addr":"172.21.15.45:6805","nonce":2290170881}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":2290170881},{"type":"v1","addr":"172.21.15.45:6809","nonce":2290170881}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":2290170881},{"type":"v1","addr":"172.21.15.45:6807","nonce":2290170881}]},"public_addr":"172.21.15.45:6803/2290170881","cluster_addr":"172.21.15.45:6805/2290170881","heartbeat_back_addr":"172.21.15.45:6809/2290170881","heartbeat_front_addr":"172.21.15.45:6807/2290170881","state":["exists","up"]},{"osd":1,"uuid":"ebd450ec-08fd-4c3d-8b24-4e10634ddb7e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6800","nonce":1462942067},{"type":"v1","addr":"172.21.15.104:6801","nonce":1462942067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":1462942067},{"type":"v1","addr":"172.21.15.104:6803","nonce":1462942067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":1462942067},{"type":"v1","addr":"172.21.15.104:6807","nonce":1462942067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":1462942067},{"type":"v1","addr":"172.21.15.104:6805","nonce":1462942067}]},"public_addr":"172.21.15.104:6801/1462942067","cluster_addr":"172.21.15.104:6803/1462942067","heartbeat_back_addr":"172.21.15.104:6807/1462942067","heartbeat_front_addr":"172.21.15.104:6805/1462942067","state":["exists","up"]},{"osd":2,"uuid":"b318a512-4190-42a2-8104-dcc4026ad8ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":2829260735},{"type":"v1","addr":"172.21.15.117:6801","nonce":2829260735}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":2829260735},{"type":"v1","addr":"172.21.15.117:6803","nonce":2829260735}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":2829260735},{"type":"v1","addr":"172.21.15.117:6807","nonce":2829260735}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":2829260735},{"type":"v1","addr":"172.21.15.117:6805","nonce":2829260735}]},"public_addr":"172.21.15.117:6801/2829260735","cluster_addr":"172.21.15.117:6803/2829260735","heartbeat_back_addr":"172.21.15.117:6807/2829260735","heartbeat_front_addr":"172.21.15.117:6805/2829260735","state":["exists","up"]},{"osd":3,"uuid":"ec6490cb-a715-4c12-a54a-225aae26fcb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":2823712137},{"type":"v1","addr":"172.21.15.45:6811","nonce":2823712137}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":2823712137},{"type":"v1","addr":"172.21.15.45:6813","nonce":2823712137}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":2823712137},{"type":"v1","addr":"172.21.15.45:6817","nonce":2823712137}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":2823712137},{"type":"v1","addr":"172.21.15.45:6815","nonce":2823712137}]},"public_addr":"172.21.15.45:6811/2823712137","cluster_addr":"172.21.15.45:6813/2823712137","heartbeat_back_addr":"172.21.15.45:6817/2823712137","heartbeat_front_addr":"172.21.15.45:6815/2823712137","state":["exists","up"]},{"osd":4,"uuid":"5d91d568-0250-4aae-8dc2-a0e079f92062","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":1396278127},{"type":"v1","addr":"172.21.15.104:6809","nonce":1396278127}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":1396278127},{"type":"v1","addr":"172.21.15.104:6811","nonce":1396278127}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":1396278127},{"type":"v1","addr":"172.21.15.104:6815","nonce":1396278127}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":1396278127},{"type":"v1","addr":"172.21.15.104:6813","nonce":1396278127}]},"public_addr":"172.21.15.104:6809/1396278127","cluster_addr":"172.21.15.104:6811/1396278127","heartbeat_back_addr":"172.21.15.104:6815/1396278127","heartbeat_front_addr":"172.21.15.104:6813/1396278127","state":["exists","up"]},{"osd":5,"uuid":"ac510b43-6fb2-4d0f-bc5f-16ac080d9e9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":4145720047},{"type":"v1","addr":"172.21.15.117:6809","nonce":4145720047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":4145720047},{"type":"v1","addr":"172.21.15.117:6811","nonce":4145720047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":4145720047},{"type":"v1","addr":"172.21.15.117:6815","nonce":4145720047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":4145720047},{"type":"v1","addr":"172.21.15.117:6813","nonce":4145720047}]},"public_addr":"172.21.15.117:6809/4145720047","cluster_addr":"172.21.15.117:6811/4145720047","heartbeat_back_addr":"172.21.15.117:6815/4145720047","heartbeat_front_addr":"172.21.15.117:6813/4145720047","state":["exists","up"]},{"osd":6,"uuid":"a41adc18-d88a-419e-b2c2-b089c1f32472","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":3254229176},{"type":"v1","addr":"172.21.15.45:6819","nonce":3254229176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":3254229176},{"type":"v1","addr":"172.21.15.45:6821","nonce":3254229176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":3254229176},{"type":"v1","addr":"172.21.15.45:6825","nonce":3254229176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":3254229176},{"type":"v1","addr":"172.21.15.45:6823","nonce":3254229176}]},"public_addr":"172.21.15.45:6819/3254229176","cluster_addr":"172.21.15.45:6821/3254229176","heartbeat_back_addr":"172.21.15.45:6825/3254229176","heartbeat_front_addr":"172.21.15.45:6823/3254229176","state":["exists","up"]},{"osd":7,"uuid":"b71712a4-7cf6-454d-9fd1-565e3df21c1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":4151077186},{"type":"v1","addr":"172.21.15.104:6817","nonce":4151077186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":4151077186},{"type":"v1","addr":"172.21.15.104:6819","nonce":4151077186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":4151077186},{"type":"v1","addr":"172.21.15.104:6823","nonce":4151077186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":4151077186},{"type":"v1","addr":"172.21.15.104:6821","nonce":4151077186}]},"public_addr":"172.21.15.104:6817/4151077186","cluster_addr":"172.21.15.104:6819/4151077186","heartbeat_back_addr":"172.21.15.104:6823/4151077186","heartbeat_front_addr":"172.21.15.104:6821/4151077186","state":["exists","up"]},{"osd":8,"uuid":"9e572aef-59a8-450e-a92e-df9dff4d57a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":1500286830},{"type":"v1","addr":"172.21.15.117:6817","nonce":1500286830}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":1500286830},{"type":"v1","addr":"172.21.15.117:6819","nonce":1500286830}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":1500286830},{"type":"v1","addr":"172.21.15.117:6823","nonce":1500286830}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":1500286830},{"type":"v1","addr":"172.21.15.117:6821","nonce":1500286830}]},"public_addr":"172.21.15.117:6817/1500286830","cluster_addr":"172.21.15.117:6819/1500286830","heartbeat_back_addr":"172.21.15.117:6823/1500286830","heartbeat_front_addr":"172.21.15.117:6821/1500286830","state":["exists","up"]},{"osd":9,"uuid":"014e9598-c413-4cc4-9bb8-61c6e8f59d30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":3106734074},{"type":"v1","addr":"172.21.15.45:6827","nonce":3106734074}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":3106734074},{"type":"v1","addr":"172.21.15.45:6829","nonce":3106734074}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":3106734074},{"type":"v1","addr":"172.21.15.45:6833","nonce":3106734074}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":3106734074},{"type":"v1","addr":"172.21.15.45:6831","nonce":3106734074}]},"public_addr":"172.21.15.45:6827/3106734074","cluster_addr":"172.21.15.45:6829/3106734074","heartbeat_back_addr":"172.21.15.45:6833/3106734074","heartbeat_front_addr":"172.21.15.45:6831/3106734074","state":["exists","up"]},{"osd":10,"uuid":"2deba364-3e39-47be-8a1f-8256df5b82d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":1404794652},{"type":"v1","addr":"172.21.15.104:6825","nonce":1404794652}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":1404794652},{"type":"v1","addr":"172.21.15.104:6827","nonce":1404794652}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":1404794652},{"type":"v1","addr":"172.21.15.104:6831","nonce":1404794652}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":1404794652},{"type":"v1","addr":"172.21.15.104:6829","nonce":1404794652}]},"public_addr":"172.21.15.104:6825/1404794652","cluster_addr":"172.21.15.104:6827/1404794652","heartbeat_back_addr":"172.21.15.104:6831/1404794652","heartbeat_front_addr":"172.21.15.104:6829/1404794652","state":["exists","up"]},{"osd":11,"uuid":"3fda4fea-59ad-4d1a-9fe6-fcb30e31fd7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":3384003243},{"type":"v1","addr":"172.21.15.117:6825","nonce":3384003243}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":3384003243},{"type":"v1","addr":"172.21.15.117:6827","nonce":3384003243}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":3384003243},{"type":"v1","addr":"172.21.15.117:6831","nonce":3384003243}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":3384003243},{"type":"v1","addr":"172.21.15.117:6829","nonce":3384003243}]},"public_addr":"172.21.15.117:6825/3384003243","cluster_addr":"172.21.15.117:6827/3384003243","heartbeat_back_addr":"172.21.15.117:6831/3384003243","heartbeat_front_addr":"172.21.15.117:6829/3384003243","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:57:14.799608+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:57:39.884668+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:58:03.327057+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:58:37.314531+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:59:03.667650+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:59:28.462823+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T17:00:02.968574+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T17:00:29.559190+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T17:00:55.291007+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T17:01:29.815977+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T17:01:57.015580+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T17:02:23.233151+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.45:0/4074982598":"2024-04-17T16:55:19.948001+0000","172.21.15.45:0/2443572576":"2024-04-17T16:55:19.948001+0000","172.21.15.45:6801/1983630136":"2024-04-17T16:55:19.948001+0000","172.21.15.45:6800/1983630136":"2024-04-17T16:55:19.948001+0000","172.21.15.45:0/3150465248":"2024-04-17T16:54:43.692466+0000","172.21.15.45:0/556738623":"2024-04-17T16:54:43.692466+0000","172.21.15.45:0/445676297":"2024-04-17T16:55:19.948001+0000","172.21.15.45:6800/2553659622":"2024-04-17T16:54:43.692466+0000","172.21.15.45:6801/2553659622":"2024-04-17T16:54:43.692466+0000","172.21.15.45:0/1184615408":"2024-04-17T16:54:43.692466+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-16T17:04:09.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:09.479+0000 7fad9bc00700 1 -- 172.21.15.45:0/627674829 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fad780a3d50 msgr2=0x7fad780a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:09.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:09.479+0000 7fad9bc00700 1 --2- 172.21.15.45:0/627674829 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fad780a3d50 0x7fad780a6210 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fad84009fd0 tx=0x7fad84009260 comp rx=0 tx=0).stop 2024-04-16T17:04:09.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:09.480+0000 7fad9bc00700 1 -- 172.21.15.45:0/627674829 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fad9406ba30 msgr2=0x7fad9419ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:09.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:09.480+0000 7fad9bc00700 1 --2- 172.21.15.45:0/627674829 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fad9406ba30 0x7fad9419ba20 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fad7c00e9a0 tx=0x7fad7c00ecb0 comp rx=0 tx=0).stop 2024-04-16T17:04:09.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:09.480+0000 7fad9bc00700 1 -- 172.21.15.45:0/627674829 shutdown_connections 2024-04-16T17:04:09.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:09.480+0000 7fad9bc00700 1 --2- 172.21.15.45:0/627674829 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fad94108b60 0x7fad9419fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:09.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:09.480+0000 7fad9bc00700 1 --2- 172.21.15.45:0/627674829 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fad9406b090 0x7fad9419b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:09.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:09.480+0000 7fad9bc00700 1 --2- 172.21.15.45:0/627674829 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fad780a3d50 0x7fad780a6210 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:09.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:09.480+0000 7fad9bc00700 1 --2- 172.21.15.45:0/627674829 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fad9406ba30 0x7fad9419ba20 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:09.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:09.480+0000 7fad9bc00700 1 -- 172.21.15.45:0/627674829 >> 172.21.15.45:0/627674829 conn(0x7fad94078960 msgr2=0x7fad9410df40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:04:09.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:09.480+0000 7fad9bc00700 1 -- 172.21.15.45:0/627674829 shutdown_connections 2024-04-16T17:04:09.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:09.480+0000 7fad9bc00700 1 -- 172.21.15.45:0/627674829 wait complete. 2024-04-16T17:04:09.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:04:09 smithi045 ceph-mon[31676]: pgmap v344: 129 pgs: 129 active+clean; 638 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 5.9 KiB/s wr, 10 op/s 2024-04-16T17:04:09.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:04:09 smithi117 ceph-mon[36730]: pgmap v344: 129 pgs: 129 active+clean; 638 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 5.9 KiB/s wr, 10 op/s 2024-04-16T17:04:09.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:04:09 smithi104 ceph-mon[36457]: pgmap v344: 129 pgs: 129 active+clean; 638 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 5.9 KiB/s wr, 10 op/s 2024-04-16T17:04:10.226 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-04-16T17:04:10.226 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45 shell --fsid b1fecd38-fc11-11ee-bc90-c7b262605968 -- ceph osd dump --format=json 2024-04-16T17:04:10.711 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/mon.a/config 2024-04-16T17:04:10.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:04:10 smithi045 ceph-mon[31676]: from='client.? 172.21.15.45:0/627674829' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T17:04:10.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:04:10 smithi117 ceph-mon[36730]: from='client.? 172.21.15.45:0/627674829' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T17:04:10.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:04:10 smithi104 ceph-mon[36457]: from='client.? 172.21.15.45:0/627674829' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T17:04:11.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:04:11 smithi045 ceph-mon[31676]: pgmap v345: 129 pgs: 129 active+clean; 638 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 4.6 KiB/s wr, 8 op/s 2024-04-16T17:04:11.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:04:11 smithi117 ceph-mon[36730]: pgmap v345: 129 pgs: 129 active+clean; 638 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 4.6 KiB/s wr, 8 op/s 2024-04-16T17:04:11.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:04:11 smithi104 ceph-mon[36457]: pgmap v345: 129 pgs: 129 active+clean; 638 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 4.6 KiB/s wr, 8 op/s 2024-04-16T17:04:13.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:13.062+0000 7fb9bba00700 1 -- 172.21.15.45:0/2669471846 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb9b4102ba0 msgr2=0x7fb9b4103000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:13.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:13.062+0000 7fb9bba00700 1 --2- 172.21.15.45:0/2669471846 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb9b4102ba0 0x7fb9b4103000 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fb9a0009990 tx=0x7fb9a0009ca0 comp rx=0 tx=0).stop 2024-04-16T17:04:13.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:13.062+0000 7fb9bba00700 1 -- 172.21.15.45:0/2669471846 shutdown_connections 2024-04-16T17:04:13.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:13.062+0000 7fb9bba00700 1 --2- 172.21.15.45:0/2669471846 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb9b4107ab0 0x7fb9b4107e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:13.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:13.062+0000 7fb9bba00700 1 --2- 172.21.15.45:0/2669471846 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb9b4103540 0x7fb9b410fa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:13.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:13.062+0000 7fb9bba00700 1 --2- 172.21.15.45:0/2669471846 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb9b4102ba0 0x7fb9b4103000 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:13.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:13.062+0000 7fb9bba00700 1 -- 172.21.15.45:0/2669471846 >> 172.21.15.45:0/2669471846 conn(0x7fb9b4079c50 msgr2=0x7fb9b407a060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:04:13.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:13.062+0000 7fb9bba00700 1 -- 172.21.15.45:0/2669471846 shutdown_connections 2024-04-16T17:04:13.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:13.062+0000 7fb9bba00700 1 -- 172.21.15.45:0/2669471846 wait complete. 2024-04-16T17:04:13.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:13.063+0000 7fb9bba00700 1 Processor -- start 2024-04-16T17:04:13.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:13.064+0000 7fb9bba00700 1 -- start start 2024-04-16T17:04:13.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:13.064+0000 7fb9bba00700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb9b4102ba0 0x7fb9b4074df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:13.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:13.064+0000 7fb9bba00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb9b4103540 0x7fb9b4075330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:13.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:13.064+0000 7fb9bba00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb9b4107ab0 0x7fb9b4075870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:13.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:13.064+0000 7fb9bba00700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fb9b4112e30 con 0x7fb9b4107ab0 2024-04-16T17:04:13.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:13.064+0000 7fb9bba00700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fb9b4112cb0 con 0x7fb9b4102ba0 2024-04-16T17:04:13.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:13.064+0000 7fb9bba00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb9b4112fb0 con 0x7fb9b4103540 2024-04-16T17:04:13.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:13.064+0000 7fb9b9000700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb9b4102ba0 0x7fb9b4074df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:13.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:13.064+0000 7fb9b9a00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb9b4107ab0 0x7fb9b4075870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:13.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:13.065+0000 7fb9abe00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb9b4103540 0x7fb9b4075330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:13.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:13.065+0000 7fb9b9000700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb9b4102ba0 0x7fb9b4074df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:54392/0 (socket says 172.21.15.45:54392) 2024-04-16T17:04:13.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:13.065+0000 7fb9b9000700 1 -- 172.21.15.45:0/2668372015 learned_addr learned my addr 172.21.15.45:0/2668372015 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:04:13.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:13.064+0000 7fb9b9a00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb9b4107ab0 0x7fb9b4075870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:47656/0 (socket says 172.21.15.45:47656) 2024-04-16T17:04:13.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:13.065+0000 7fb9b9000700 1 -- 172.21.15.45:0/2668372015 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb9b4103540 msgr2=0x7fb9b4075330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:13.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:13.065+0000 7fb9b9000700 1 --2- 172.21.15.45:0/2668372015 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb9b4103540 0x7fb9b4075330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:13.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:13.065+0000 7fb9b9000700 1 -- 172.21.15.45:0/2668372015 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb9b4107ab0 msgr2=0x7fb9b4075870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:13.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:13.065+0000 7fb9b9000700 1 --2- 172.21.15.45:0/2668372015 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb9b4107ab0 0x7fb9b4075870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:13.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:13.065+0000 7fb9b9000700 1 -- 172.21.15.45:0/2668372015 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9b41a7840 con 0x7fb9b4102ba0 2024-04-16T17:04:13.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:13.065+0000 7fb9abe00700 1 --2- 172.21.15.45:0/2668372015 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb9b4103540 0x7fb9b4075330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:04:13.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:13.065+0000 7fb9b9000700 1 --2- 172.21.15.45:0/2668372015 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb9b4102ba0 0x7fb9b4074df0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fb9a400efc0 tx=0x7fb9a400c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:04:13.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:13.066+0000 7fb9b2000700 1 -- 172.21.15.45:0/2668372015 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9a40092e0 con 0x7fb9b4102ba0 2024-04-16T17:04:13.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:13.066+0000 7fb9b2000700 1 -- 172.21.15.45:0/2668372015 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb9a400c9c0 con 0x7fb9b4102ba0 2024-04-16T17:04:13.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:13.066+0000 7fb9bba00700 1 -- 172.21.15.45:0/2668372015 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9b41a7ad0 con 0x7fb9b4102ba0 2024-04-16T17:04:13.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:13.066+0000 7fb9b2000700 1 -- 172.21.15.45:0/2668372015 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9a4004610 con 0x7fb9b4102ba0 2024-04-16T17:04:13.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:13.066+0000 7fb9bba00700 1 -- 172.21.15.45:0/2668372015 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9b41a8030 con 0x7fb9b4102ba0 2024-04-16T17:04:13.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:13.067+0000 7fb9bba00700 1 -- 172.21.15.45:0/2668372015 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb9b40697e0 con 0x7fb9b4102ba0 2024-04-16T17:04:13.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:13.069+0000 7fb9b2000700 1 -- 172.21.15.45:0/2668372015 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fb9a401b030 con 0x7fb9b4102ba0 2024-04-16T17:04:13.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:13.070+0000 7fb9b2000700 1 --2- 172.21.15.45:0/2668372015 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fb9980a3d50 0x7fb9980a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:13.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:13.071+0000 7fb9b2000700 1 -- 172.21.15.45:0/2668372015 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7fb9a40d0600 con 0x7fb9b4102ba0 2024-04-16T17:04:13.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:13.071+0000 7fb9abe00700 1 --2- 172.21.15.45:0/2668372015 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fb9980a3d50 0x7fb9980a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:13.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:13.072+0000 7fb9abe00700 1 --2- 172.21.15.45:0/2668372015 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fb9980a3d50 0x7fb9980a6210 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fb9a00098e0 tx=0x7fb9a0005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:04:13.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:13.075+0000 7fb9b2000700 1 -- 172.21.15.45:0/2668372015 <== mon.2 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb9a4022020 con 0x7fb9b4102ba0 2024-04-16T17:04:13.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:13.308+0000 7fb9bba00700 1 -- 172.21.15.45:0/2668372015 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb9b402dfe0 con 0x7fb9b4102ba0 2024-04-16T17:04:13.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:13.310+0000 7fb9b2000700 1 -- 172.21.15.45:0/2668372015 <== mon.2 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+24164 (secure 0 0 0) 0x7fb9a4010040 con 0x7fb9b4102ba0 2024-04-16T17:04:13.313 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-16T17:04:13.313 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":90,"fsid":"b1fecd38-fc11-11ee-bc90-c7b262605968","created":"2024-04-16T16:54:02.385906+0000","modified":"2024-04-16T17:03:29.720565+0000","last_up_change":"2024-04-16T17:02:25.162624+0000","last_in_change":"2024-04-16T17:02:04.903274+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-16T16:58:06.166659+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-16T17:03:22.304198+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-16T17:03:23.616450+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-16T17:03:27.226289+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b655de2c-59f8-4fd4-ac98-0cc222229f56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":2290170881},{"type":"v1","addr":"172.21.15.45:6803","nonce":2290170881}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":2290170881},{"type":"v1","addr":"172.21.15.45:6805","nonce":2290170881}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":2290170881},{"type":"v1","addr":"172.21.15.45:6809","nonce":2290170881}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":2290170881},{"type":"v1","addr":"172.21.15.45:6807","nonce":2290170881}]},"public_addr":"172.21.15.45:6803/2290170881","cluster_addr":"172.21.15.45:6805/2290170881","heartbeat_back_addr":"172.21.15.45:6809/2290170881","heartbeat_front_addr":"172.21.15.45:6807/2290170881","state":["exists","up"]},{"osd":1,"uuid":"ebd450ec-08fd-4c3d-8b24-4e10634ddb7e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6800","nonce":1462942067},{"type":"v1","addr":"172.21.15.104:6801","nonce":1462942067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":1462942067},{"type":"v1","addr":"172.21.15.104:6803","nonce":1462942067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":1462942067},{"type":"v1","addr":"172.21.15.104:6807","nonce":1462942067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":1462942067},{"type":"v1","addr":"172.21.15.104:6805","nonce":1462942067}]},"public_addr":"172.21.15.104:6801/1462942067","cluster_addr":"172.21.15.104:6803/1462942067","heartbeat_back_addr":"172.21.15.104:6807/1462942067","heartbeat_front_addr":"172.21.15.104:6805/1462942067","state":["exists","up"]},{"osd":2,"uuid":"b318a512-4190-42a2-8104-dcc4026ad8ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":2829260735},{"type":"v1","addr":"172.21.15.117:6801","nonce":2829260735}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":2829260735},{"type":"v1","addr":"172.21.15.117:6803","nonce":2829260735}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":2829260735},{"type":"v1","addr":"172.21.15.117:6807","nonce":2829260735}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":2829260735},{"type":"v1","addr":"172.21.15.117:6805","nonce":2829260735}]},"public_addr":"172.21.15.117:6801/2829260735","cluster_addr":"172.21.15.117:6803/2829260735","heartbeat_back_addr":"172.21.15.117:6807/2829260735","heartbeat_front_addr":"172.21.15.117:6805/2829260735","state":["exists","up"]},{"osd":3,"uuid":"ec6490cb-a715-4c12-a54a-225aae26fcb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":2823712137},{"type":"v1","addr":"172.21.15.45:6811","nonce":2823712137}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":2823712137},{"type":"v1","addr":"172.21.15.45:6813","nonce":2823712137}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":2823712137},{"type":"v1","addr":"172.21.15.45:6817","nonce":2823712137}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":2823712137},{"type":"v1","addr":"172.21.15.45:6815","nonce":2823712137}]},"public_addr":"172.21.15.45:6811/2823712137","cluster_addr":"172.21.15.45:6813/2823712137","heartbeat_back_addr":"172.21.15.45:6817/2823712137","heartbeat_front_addr":"172.21.15.45:6815/2823712137","state":["exists","up"]},{"osd":4,"uuid":"5d91d568-0250-4aae-8dc2-a0e079f92062","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":1396278127},{"type":"v1","addr":"172.21.15.104:6809","nonce":1396278127}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":1396278127},{"type":"v1","addr":"172.21.15.104:6811","nonce":1396278127}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":1396278127},{"type":"v1","addr":"172.21.15.104:6815","nonce":1396278127}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":1396278127},{"type":"v1","addr":"172.21.15.104:6813","nonce":1396278127}]},"public_addr":"172.21.15.104:6809/1396278127","cluster_addr":"172.21.15.104:6811/1396278127","heartbeat_back_addr":"172.21.15.104:6815/1396278127","heartbeat_front_addr":"172.21.15.104:6813/1396278127","state":["exists","up"]},{"osd":5,"uuid":"ac510b43-6fb2-4d0f-bc5f-16ac080d9e9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":4145720047},{"type":"v1","addr":"172.21.15.117:6809","nonce":4145720047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":4145720047},{"type":"v1","addr":"172.21.15.117:6811","nonce":4145720047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":4145720047},{"type":"v1","addr":"172.21.15.117:6815","nonce":4145720047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":4145720047},{"type":"v1","addr":"172.21.15.117:6813","nonce":4145720047}]},"public_addr":"172.21.15.117:6809/4145720047","cluster_addr":"172.21.15.117:6811/4145720047","heartbeat_back_addr":"172.21.15.117:6815/4145720047","heartbeat_front_addr":"172.21.15.117:6813/4145720047","state":["exists","up"]},{"osd":6,"uuid":"a41adc18-d88a-419e-b2c2-b089c1f32472","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":3254229176},{"type":"v1","addr":"172.21.15.45:6819","nonce":3254229176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":3254229176},{"type":"v1","addr":"172.21.15.45:6821","nonce":3254229176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":3254229176},{"type":"v1","addr":"172.21.15.45:6825","nonce":3254229176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":3254229176},{"type":"v1","addr":"172.21.15.45:6823","nonce":3254229176}]},"public_addr":"172.21.15.45:6819/3254229176","cluster_addr":"172.21.15.45:6821/3254229176","heartbeat_back_addr":"172.21.15.45:6825/3254229176","heartbeat_front_addr":"172.21.15.45:6823/3254229176","state":["exists","up"]},{"osd":7,"uuid":"b71712a4-7cf6-454d-9fd1-565e3df21c1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":4151077186},{"type":"v1","addr":"172.21.15.104:6817","nonce":4151077186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":4151077186},{"type":"v1","addr":"172.21.15.104:6819","nonce":4151077186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":4151077186},{"type":"v1","addr":"172.21.15.104:6823","nonce":4151077186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":4151077186},{"type":"v1","addr":"172.21.15.104:6821","nonce":4151077186}]},"public_addr":"172.21.15.104:6817/4151077186","cluster_addr":"172.21.15.104:6819/4151077186","heartbeat_back_addr":"172.21.15.104:6823/4151077186","heartbeat_front_addr":"172.21.15.104:6821/4151077186","state":["exists","up"]},{"osd":8,"uuid":"9e572aef-59a8-450e-a92e-df9dff4d57a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":1500286830},{"type":"v1","addr":"172.21.15.117:6817","nonce":1500286830}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":1500286830},{"type":"v1","addr":"172.21.15.117:6819","nonce":1500286830}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":1500286830},{"type":"v1","addr":"172.21.15.117:6823","nonce":1500286830}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":1500286830},{"type":"v1","addr":"172.21.15.117:6821","nonce":1500286830}]},"public_addr":"172.21.15.117:6817/1500286830","cluster_addr":"172.21.15.117:6819/1500286830","heartbeat_back_addr":"172.21.15.117:6823/1500286830","heartbeat_front_addr":"172.21.15.117:6821/1500286830","state":["exists","up"]},{"osd":9,"uuid":"014e9598-c413-4cc4-9bb8-61c6e8f59d30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":3106734074},{"type":"v1","addr":"172.21.15.45:6827","nonce":3106734074}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":3106734074},{"type":"v1","addr":"172.21.15.45:6829","nonce":3106734074}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":3106734074},{"type":"v1","addr":"172.21.15.45:6833","nonce":3106734074}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":3106734074},{"type":"v1","addr":"172.21.15.45:6831","nonce":3106734074}]},"public_addr":"172.21.15.45:6827/3106734074","cluster_addr":"172.21.15.45:6829/3106734074","heartbeat_back_addr":"172.21.15.45:6833/3106734074","heartbeat_front_addr":"172.21.15.45:6831/3106734074","state":["exists","up"]},{"osd":10,"uuid":"2deba364-3e39-47be-8a1f-8256df5b82d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":1404794652},{"type":"v1","addr":"172.21.15.104:6825","nonce":1404794652}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":1404794652},{"type":"v1","addr":"172.21.15.104:6827","nonce":1404794652}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":1404794652},{"type":"v1","addr":"172.21.15.104:6831","nonce":1404794652}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":1404794652},{"type":"v1","addr":"172.21.15.104:6829","nonce":1404794652}]},"public_addr":"172.21.15.104:6825/1404794652","cluster_addr":"172.21.15.104:6827/1404794652","heartbeat_back_addr":"172.21.15.104:6831/1404794652","heartbeat_front_addr":"172.21.15.104:6829/1404794652","state":["exists","up"]},{"osd":11,"uuid":"3fda4fea-59ad-4d1a-9fe6-fcb30e31fd7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":3384003243},{"type":"v1","addr":"172.21.15.117:6825","nonce":3384003243}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":3384003243},{"type":"v1","addr":"172.21.15.117:6827","nonce":3384003243}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":3384003243},{"type":"v1","addr":"172.21.15.117:6831","nonce":3384003243}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":3384003243},{"type":"v1","addr":"172.21.15.117:6829","nonce":3384003243}]},"public_addr":"172.21.15.117:6825/3384003243","cluster_addr":"172.21.15.117:6827/3384003243","heartbeat_back_addr":"172.21.15.117:6831/3384003243","heartbeat_front_addr":"172.21.15.117:6829/3384003243","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:57:14.799608+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:57:39.884668+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:58:03.327057+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:58:37.314531+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:59:03.667650+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:59:28.462823+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T17:00:02.968574+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T17:00:29.559190+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T17:00:55.291007+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T17:01:29.815977+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T17:01:57.015580+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T17:02:23.233151+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.45:0/4074982598":"2024-04-17T16:55:19.948001+0000","172.21.15.45:0/2443572576":"2024-04-17T16:55:19.948001+0000","172.21.15.45:6801/1983630136":"2024-04-17T16:55:19.948001+0000","172.21.15.45:6800/1983630136":"2024-04-17T16:55:19.948001+0000","172.21.15.45:0/3150465248":"2024-04-17T16:54:43.692466+0000","172.21.15.45:0/556738623":"2024-04-17T16:54:43.692466+0000","172.21.15.45:0/445676297":"2024-04-17T16:55:19.948001+0000","172.21.15.45:6800/2553659622":"2024-04-17T16:54:43.692466+0000","172.21.15.45:6801/2553659622":"2024-04-17T16:54:43.692466+0000","172.21.15.45:0/1184615408":"2024-04-17T16:54:43.692466+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-16T17:04:13.315 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:13.313+0000 7fb9bba00700 1 -- 172.21.15.45:0/2668372015 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fb9980a3d50 msgr2=0x7fb9980a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:13.315 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:13.313+0000 7fb9bba00700 1 --2- 172.21.15.45:0/2668372015 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fb9980a3d50 0x7fb9980a6210 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fb9a00098e0 tx=0x7fb9a0005960 comp rx=0 tx=0).stop 2024-04-16T17:04:13.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:13.313+0000 7fb9bba00700 1 -- 172.21.15.45:0/2668372015 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb9b4102ba0 msgr2=0x7fb9b4074df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:13.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:13.313+0000 7fb9bba00700 1 --2- 172.21.15.45:0/2668372015 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb9b4102ba0 0x7fb9b4074df0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fb9a400efc0 tx=0x7fb9a400c3a0 comp rx=0 tx=0).stop 2024-04-16T17:04:13.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:13.313+0000 7fb9bba00700 1 -- 172.21.15.45:0/2668372015 shutdown_connections 2024-04-16T17:04:13.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:13.313+0000 7fb9bba00700 1 --2- 172.21.15.45:0/2668372015 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb9b4107ab0 0x7fb9b4075870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:13.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:13.313+0000 7fb9bba00700 1 --2- 172.21.15.45:0/2668372015 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb9b4102ba0 0x7fb9b4074df0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:13.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:13.313+0000 7fb9bba00700 1 --2- 172.21.15.45:0/2668372015 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fb9980a3d50 0x7fb9980a6210 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:13.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:13.313+0000 7fb9bba00700 1 --2- 172.21.15.45:0/2668372015 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb9b4103540 0x7fb9b4075330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:13.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:13.313+0000 7fb9bba00700 1 -- 172.21.15.45:0/2668372015 >> 172.21.15.45:0/2668372015 conn(0x7fb9b4079c50 msgr2=0x7fb9b4100c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:04:13.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:13.314+0000 7fb9bba00700 1 -- 172.21.15.45:0/2668372015 shutdown_connections 2024-04-16T17:04:13.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:13.314+0000 7fb9bba00700 1 -- 172.21.15.45:0/2668372015 wait complete. 2024-04-16T17:04:13.716 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:04:13 smithi045 ceph-mon[31676]: pgmap v346: 129 pgs: 129 active+clean; 638 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1.3 KiB/s wr, 7 op/s 2024-04-16T17:04:13.716 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:04:13 smithi045 ceph-mon[31676]: from='client.? 172.21.15.45:0/2668372015' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T17:04:13.729 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:04:13 smithi104 ceph-mon[36457]: pgmap v346: 129 pgs: 129 active+clean; 638 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1.3 KiB/s wr, 7 op/s 2024-04-16T17:04:13.730 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:04:13 smithi104 ceph-mon[36457]: from='client.? 172.21.15.45:0/2668372015' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T17:04:13.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:04:13 smithi117 ceph-mon[36730]: pgmap v346: 129 pgs: 129 active+clean; 638 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1.3 KiB/s wr, 7 op/s 2024-04-16T17:04:13.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:04:13 smithi117 ceph-mon[36730]: from='client.? 172.21.15.45:0/2668372015' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T17:04:14.235 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45 shell --fsid b1fecd38-fc11-11ee-bc90-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-04-16T17:04:14.235 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45 shell --fsid b1fecd38-fc11-11ee-bc90-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-04-16T17:04:14.236 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45 shell --fsid b1fecd38-fc11-11ee-bc90-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-04-16T17:04:14.236 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45 shell --fsid b1fecd38-fc11-11ee-bc90-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-04-16T17:04:14.236 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45 shell --fsid b1fecd38-fc11-11ee-bc90-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-04-16T17:04:14.237 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45 shell --fsid b1fecd38-fc11-11ee-bc90-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-04-16T17:04:14.237 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45 shell --fsid b1fecd38-fc11-11ee-bc90-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-04-16T17:04:14.237 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45 shell --fsid b1fecd38-fc11-11ee-bc90-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-04-16T17:04:14.238 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45 shell --fsid b1fecd38-fc11-11ee-bc90-c7b262605968 -- ceph tell osd.8 flush_pg_stats 2024-04-16T17:04:14.238 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45 shell --fsid b1fecd38-fc11-11ee-bc90-c7b262605968 -- ceph tell osd.9 flush_pg_stats 2024-04-16T17:04:14.239 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45 shell --fsid b1fecd38-fc11-11ee-bc90-c7b262605968 -- ceph tell osd.10 flush_pg_stats 2024-04-16T17:04:14.239 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45 shell --fsid b1fecd38-fc11-11ee-bc90-c7b262605968 -- ceph tell osd.11 flush_pg_stats 2024-04-16T17:04:15.244 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/mon.a/config 2024-04-16T17:04:15.268 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/mon.a/config 2024-04-16T17:04:15.272 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/mon.a/config 2024-04-16T17:04:15.282 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/mon.a/config 2024-04-16T17:04:15.701 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:04:15 smithi045 ceph-mon[31676]: pgmap v347: 129 pgs: 129 active+clean; 638 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 1.3 KiB/s wr, 6 op/s 2024-04-16T17:04:15.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:04:15 smithi117 ceph-mon[36730]: pgmap v347: 129 pgs: 129 active+clean; 638 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 1.3 KiB/s wr, 6 op/s 2024-04-16T17:04:15.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:04:15 smithi104 ceph-mon[36457]: pgmap v347: 129 pgs: 129 active+clean; 638 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 1.3 KiB/s wr, 6 op/s 2024-04-16T17:04:17.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:04:17 smithi045 ceph-mon[31676]: pgmap v348: 129 pgs: 129 active+clean; 641 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-04-16T17:04:17.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:04:17 smithi117 ceph-mon[36730]: pgmap v348: 129 pgs: 129 active+clean; 641 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-04-16T17:04:17.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:04:17 smithi104 ceph-mon[36457]: pgmap v348: 129 pgs: 129 active+clean; 641 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-04-16T17:04:18.220 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/mon.a/config 2024-04-16T17:04:18.224 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/mon.a/config 2024-04-16T17:04:18.224 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/mon.a/config 2024-04-16T17:04:18.228 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/mon.a/config 2024-04-16T17:04:18.253 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/mon.a/config 2024-04-16T17:04:18.801 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/mon.a/config 2024-04-16T17:04:18.808 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/mon.a/config 2024-04-16T17:04:18.809 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/mon.a/config 2024-04-16T17:04:19.591 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:04:19 smithi045 ceph-mon[31676]: pgmap v349: 129 pgs: 129 active+clean; 641 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 426 B/s wr, 5 op/s 2024-04-16T17:04:19.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:04:19 smithi117 ceph-mon[36730]: pgmap v349: 129 pgs: 129 active+clean; 641 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 426 B/s wr, 5 op/s 2024-04-16T17:04:19.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:04:19 smithi104 ceph-mon[36457]: pgmap v349: 129 pgs: 129 active+clean; 641 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 426 B/s wr, 5 op/s 2024-04-16T17:04:20.705 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:04:20 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-04-16T17:04:20.705 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:04:20 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]: dispatch 2024-04-16T17:04:20.706 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:04:20 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 4]}]: dispatch 2024-04-16T17:04:20.706 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:04:20 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [8, 1]}]: dispatch 2024-04-16T17:04:20.706 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:04:20 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]}]: dispatch 2024-04-16T17:04:20.706 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:04:20 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 8]}]: dispatch 2024-04-16T17:04:20.706 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:04:20 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]}]: dispatch 2024-04-16T17:04:20.706 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:04:20 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]: dispatch 2024-04-16T17:04:20.706 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:04:20 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.5", "id": [9, 10]}]: dispatch 2024-04-16T17:04:20.706 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:04:20 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [9, 8]}]: dispatch 2024-04-16T17:04:20.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:04:20 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-04-16T17:04:20.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:04:20 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]: dispatch 2024-04-16T17:04:20.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:04:20 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 4]}]: dispatch 2024-04-16T17:04:20.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:04:20 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [8, 1]}]: dispatch 2024-04-16T17:04:20.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:04:20 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]}]: dispatch 2024-04-16T17:04:20.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:04:20 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 8]}]: dispatch 2024-04-16T17:04:20.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:04:20 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]}]: dispatch 2024-04-16T17:04:20.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:04:20 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]: dispatch 2024-04-16T17:04:20.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:04:20 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.5", "id": [9, 10]}]: dispatch 2024-04-16T17:04:20.807 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:04:20 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [9, 8]}]: dispatch 2024-04-16T17:04:20.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:04:20 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-04-16T17:04:20.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:04:20 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]: dispatch 2024-04-16T17:04:20.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:04:20 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 4]}]: dispatch 2024-04-16T17:04:20.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:04:20 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [8, 1]}]: dispatch 2024-04-16T17:04:20.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:04:20 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]}]: dispatch 2024-04-16T17:04:20.929 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:04:20 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 8]}]: dispatch 2024-04-16T17:04:20.929 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:04:20 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]}]: dispatch 2024-04-16T17:04:20.929 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:04:20 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]: dispatch 2024-04-16T17:04:20.929 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:04:20 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.5", "id": [9, 10]}]: dispatch 2024-04-16T17:04:20.929 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:04:20 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [9, 8]}]: dispatch 2024-04-16T17:04:21.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:04:21 smithi045 ceph-mon[31676]: pgmap v350: 129 pgs: 129 active+clean; 641 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 6 op/s 2024-04-16T17:04:21.729 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:04:21 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-04-16T17:04:21.729 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:04:21 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]': finished 2024-04-16T17:04:21.729 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:04:21 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 4]}]': finished 2024-04-16T17:04:21.729 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:04:21 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [8, 1]}]': finished 2024-04-16T17:04:21.729 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:04:21 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]}]': finished 2024-04-16T17:04:21.729 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:04:21 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 8]}]': finished 2024-04-16T17:04:21.729 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:04:21 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]}]': finished 2024-04-16T17:04:21.729 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:04:21 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]': finished 2024-04-16T17:04:21.729 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:04:21 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.5", "id": [9, 10]}]': finished 2024-04-16T17:04:21.729 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:04:21 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [9, 8]}]': finished 2024-04-16T17:04:21.729 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:04:21 smithi045 ceph-mon[31676]: osdmap e91: 12 total, 12 up, 12 in 2024-04-16T17:04:21.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:04:21 smithi117 ceph-mon[36730]: pgmap v350: 129 pgs: 129 active+clean; 641 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 6 op/s 2024-04-16T17:04:21.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:04:21 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-04-16T17:04:21.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:04:21 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]': finished 2024-04-16T17:04:21.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:04:21 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 4]}]': finished 2024-04-16T17:04:21.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:04:21 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [8, 1]}]': finished 2024-04-16T17:04:21.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:04:21 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]}]': finished 2024-04-16T17:04:21.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:04:21 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 8]}]': finished 2024-04-16T17:04:21.807 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:04:21 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]}]': finished 2024-04-16T17:04:21.807 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:04:21 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]': finished 2024-04-16T17:04:21.807 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:04:21 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.5", "id": [9, 10]}]': finished 2024-04-16T17:04:21.807 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:04:21 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [9, 8]}]': finished 2024-04-16T17:04:21.807 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:04:21 smithi117 ceph-mon[36730]: osdmap e91: 12 total, 12 up, 12 in 2024-04-16T17:04:21.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:04:21 smithi104 ceph-mon[36457]: pgmap v350: 129 pgs: 129 active+clean; 641 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 6 op/s 2024-04-16T17:04:21.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:04:21 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-04-16T17:04:21.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:04:21 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]': finished 2024-04-16T17:04:21.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:04:21 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 4]}]': finished 2024-04-16T17:04:21.929 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:04:21 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [8, 1]}]': finished 2024-04-16T17:04:21.929 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:04:21 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]}]': finished 2024-04-16T17:04:21.929 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:04:21 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 8]}]': finished 2024-04-16T17:04:21.929 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:04:21 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]}]': finished 2024-04-16T17:04:21.929 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:04:21 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]': finished 2024-04-16T17:04:21.929 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:04:21 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.5", "id": [9, 10]}]': finished 2024-04-16T17:04:21.929 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:04:21 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [9, 8]}]': finished 2024-04-16T17:04:21.929 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:04:21 smithi104 ceph-mon[36457]: osdmap e91: 12 total, 12 up, 12 in 2024-04-16T17:04:22.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:19.884+0000 7f16bf200700 1 -- 172.21.15.45:0/3077652351 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f16b8104f90 msgr2=0x7f16b8105370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:22.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:19.884+0000 7f16bf200700 1 --2- 172.21.15.45:0/3077652351 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f16b8104f90 0x7f16b8105370 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f16a80099c0 tx=0x7f16a8009cd0 comp rx=0 tx=0).stop 2024-04-16T17:04:22.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:19.884+0000 7f16bf200700 1 -- 172.21.15.45:0/3077652351 shutdown_connections 2024-04-16T17:04:22.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:19.884+0000 7f16bf200700 1 --2- 172.21.15.45:0/3077652351 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f16b8104f90 0x7f16b8105370 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:22.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:19.884+0000 7f16bf200700 1 --2- 172.21.15.45:0/3077652351 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f16b810a070 0x7f16b8111c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:22.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:19.884+0000 7f16bf200700 1 --2- 172.21.15.45:0/3077652351 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f16b8105940 0x7f16b8109990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:22.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:19.884+0000 7f16bf200700 1 -- 172.21.15.45:0/3077652351 >> 172.21.15.45:0/3077652351 conn(0x7f16b8100820 msgr2=0x7f16b8102c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:04:22.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:19.885+0000 7f16bf200700 1 -- 172.21.15.45:0/3077652351 shutdown_connections 2024-04-16T17:04:22.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:19.885+0000 7f16bf200700 1 -- 172.21.15.45:0/3077652351 wait complete. 2024-04-16T17:04:22.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:19.886+0000 7f16bf200700 1 Processor -- start 2024-04-16T17:04:22.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:19.886+0000 7f16bf200700 1 -- start start 2024-04-16T17:04:22.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:19.886+0000 7f16bf200700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f16b8104f90 0x7f16b819b530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:22.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:19.886+0000 7f16bf200700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f16b8105940 0x7f16b819ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:22.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:19.887+0000 7f16bf200700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f16b810a070 0x7f16b819fe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:22.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:19.887+0000 7f16bf200700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f16b81150c0 con 0x7f16b810a070 2024-04-16T17:04:22.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:19.887+0000 7f16bf200700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f16b8114f40 con 0x7f16b8104f90 2024-04-16T17:04:22.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:19.887+0000 7f16bf200700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f16b8115240 con 0x7f16b8105940 2024-04-16T17:04:22.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:19.887+0000 7f16bd200700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f16b810a070 0x7f16b819fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:22.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:19.887+0000 7f16bd200700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f16b810a070 0x7f16b819fe00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35484/0 (socket says 172.21.15.45:35484) 2024-04-16T17:04:22.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:19.887+0000 7f16bd200700 1 -- 172.21.15.45:0/2647980399 learned_addr learned my addr 172.21.15.45:0/2647980399 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:04:22.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:19.887+0000 7f16b7e00700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f16b8104f90 0x7f16b819b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:22.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:19.887+0000 7f16b7400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f16b8105940 0x7f16b819ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:22.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:19.887+0000 7f16b7400700 1 -- 172.21.15.45:0/2647980399 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f16b8104f90 msgr2=0x7f16b819b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:22.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:19.887+0000 7f16b7400700 1 --2- 172.21.15.45:0/2647980399 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f16b8104f90 0x7f16b819b530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:22.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:19.887+0000 7f16b7400700 1 -- 172.21.15.45:0/2647980399 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f16b810a070 msgr2=0x7f16b819fe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:22.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:19.887+0000 7f16b7400700 1 --2- 172.21.15.45:0/2647980399 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f16b810a070 0x7f16b819fe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:22.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:19.887+0000 7f16b7400700 1 -- 172.21.15.45:0/2647980399 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16b81a0540 con 0x7f16b8105940 2024-04-16T17:04:22.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:19.887+0000 7f16bd200700 1 --2- 172.21.15.45:0/2647980399 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f16b810a070 0x7f16b819fe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:04:22.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:19.888+0000 7f16b7400700 1 --2- 172.21.15.45:0/2647980399 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f16b8105940 0x7f16b819ba70 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f16a400e9c0 tx=0x7f16a400ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:04:22.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:19.888+0000 7f16b4c00700 1 -- 172.21.15.45:0/2647980399 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16a4004220 con 0x7f16b8105940 2024-04-16T17:04:22.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:19.888+0000 7f16b4c00700 1 -- 172.21.15.45:0/2647980399 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f16a4004d10 con 0x7f16b8105940 2024-04-16T17:04:22.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:19.888+0000 7f16bf200700 1 -- 172.21.15.45:0/2647980399 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16b81a0830 con 0x7f16b8105940 2024-04-16T17:04:22.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:19.888+0000 7f16b4c00700 1 -- 172.21.15.45:0/2647980399 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16a4005020 con 0x7f16b8105940 2024-04-16T17:04:22.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:19.888+0000 7f16bf200700 1 -- 172.21.15.45:0/2647980399 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16b81a80b0 con 0x7f16b8105940 2024-04-16T17:04:22.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:19.889+0000 7f16bf200700 1 -- 172.21.15.45:0/2647980399 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f16b8063280 con 0x7f16b8105940 2024-04-16T17:04:22.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:19.891+0000 7f16b4c00700 1 -- 172.21.15.45:0/2647980399 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f16a4004750 con 0x7f16b8105940 2024-04-16T17:04:22.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:19.893+0000 7f16b4c00700 1 --2- 172.21.15.45:0/2647980399 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f169c0a3d50 0x7f169c0a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:22.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:19.893+0000 7f16b4c00700 1 -- 172.21.15.45:0/2647980399 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f16a40d0990 con 0x7f16b8105940 2024-04-16T17:04:22.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:19.893+0000 7f16b7e00700 1 --2- 172.21.15.45:0/2647980399 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f169c0a3d50 0x7f169c0a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:22.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:19.893+0000 7f16b4c00700 1 --2- 172.21.15.45:0/2647980399 >> [v2:172.21.15.117:6808/4145720047,v1:172.21.15.117:6809/4145720047] conn(0x7f169c0aea10 0x7f169c0b0e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:22.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:19.893+0000 7f16b4c00700 1 -- 172.21.15.45:0/2647980399 --> [v2:172.21.15.117:6808/4145720047,v1:172.21.15.117:6809/4145720047] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f16a401ee50 con 0x7f169c0aea10 2024-04-16T17:04:22.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:19.893+0000 7f16b4c00700 1 -- 172.21.15.45:0/2647980399 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7f16a40d0d40 con 0x7f16b8105940 2024-04-16T17:04:22.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:19.894+0000 7f16bd200700 1 --2- 172.21.15.45:0/2647980399 >> [v2:172.21.15.117:6808/4145720047,v1:172.21.15.117:6809/4145720047] conn(0x7f169c0aea10 0x7f169c0b0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:22.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:19.894+0000 7f16b7e00700 1 --2- 172.21.15.45:0/2647980399 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f169c0a3d50 0x7f169c0a6210 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f16a8000c00 tx=0x7f16a8005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:04:22.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:19.894+0000 7f16bd200700 1 --2- 172.21.15.45:0/2647980399 >> [v2:172.21.15.117:6808/4145720047,v1:172.21.15.117:6809/4145720047] conn(0x7f169c0aea10 0x7f169c0b0e50 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:04:22.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:19.895+0000 7f16b4c00700 1 -- 172.21.15.45:0/2647980399 <== osd.5 v2:172.21.15.117:6808/4145720047 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f16a401ee50 con 0x7f169c0aea10 2024-04-16T17:04:22.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:19.914+0000 7f16bf200700 1 -- 172.21.15.45:0/2647980399 --> [v2:172.21.15.117:6808/4145720047,v1:172.21.15.117:6809/4145720047] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f16b8063700 con 0x7f169c0aea10 2024-04-16T17:04:22.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:19.915+0000 7f16b4c00700 1 -- 172.21.15.45:0/2647980399 <== osd.5 v2:172.21.15.117:6808/4145720047 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f16b8063700 con 0x7f169c0aea10 2024-04-16T17:04:22.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:19.915+0000 7f16bf200700 1 -- 172.21.15.45:0/2647980399 >> [v2:172.21.15.117:6808/4145720047,v1:172.21.15.117:6809/4145720047] conn(0x7f169c0aea10 msgr2=0x7f169c0b0e50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:22.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:19.915+0000 7f16bf200700 1 --2- 172.21.15.45:0/2647980399 >> [v2:172.21.15.117:6808/4145720047,v1:172.21.15.117:6809/4145720047] conn(0x7f169c0aea10 0x7f169c0b0e50 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:22.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:19.915+0000 7f16bf200700 1 -- 172.21.15.45:0/2647980399 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f169c0a3d50 msgr2=0x7f169c0a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:22.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:19.915+0000 7f16bf200700 1 --2- 172.21.15.45:0/2647980399 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f169c0a3d50 0x7f169c0a6210 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f16a8000c00 tx=0x7f16a8005fb0 comp rx=0 tx=0).stop 2024-04-16T17:04:22.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:19.915+0000 7f16bf200700 1 -- 172.21.15.45:0/2647980399 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f16b8105940 msgr2=0x7f16b819ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:22.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:19.915+0000 7f16bf200700 1 --2- 172.21.15.45:0/2647980399 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f16b8105940 0x7f16b819ba70 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f16a400e9c0 tx=0x7f16a400ecd0 comp rx=0 tx=0).stop 2024-04-16T17:04:22.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:19.915+0000 7f16bd200700 1 -- 172.21.15.45:0/2647980399 reap_dead start 2024-04-16T17:04:22.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:19.916+0000 7f16bf200700 1 -- 172.21.15.45:0/2647980399 shutdown_connections 2024-04-16T17:04:22.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:19.916+0000 7f16bf200700 1 -- 172.21.15.45:0/2647980399 >> 172.21.15.45:0/2647980399 conn(0x7f16b8100820 msgr2=0x7f16b8102600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:04:22.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:19.916+0000 7f16bf200700 1 -- 172.21.15.45:0/2647980399 shutdown_connections 2024-04-16T17:04:22.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:19.916+0000 7f16bf200700 1 -- 172.21.15.45:0/2647980399 wait complete. 2024-04-16T17:04:22.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:04:22 smithi045 ceph-mon[31676]: osdmap e92: 12 total, 12 up, 12 in 2024-04-16T17:04:22.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:04:22 smithi117 ceph-mon[36730]: osdmap e92: 12 total, 12 up, 12 in 2024-04-16T17:04:22.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:04:22 smithi104 ceph-mon[36457]: osdmap e92: 12 total, 12 up, 12 in 2024-04-16T17:04:23.690 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:04:23 smithi045 ceph-mon[31676]: pgmap v353: 129 pgs: 129 active+clean; 641 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 639 B/s wr, 7 op/s 2024-04-16T17:04:23.743 INFO:teuthology.orchestra.run.smithi045.stdout:167503724604 2024-04-16T17:04:23.744 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45 shell --fsid b1fecd38-fc11-11ee-bc90-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-04-16T17:04:23.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:04:23 smithi117 ceph-mon[36730]: pgmap v353: 129 pgs: 129 active+clean; 641 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 639 B/s wr, 7 op/s 2024-04-16T17:04:23.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:04:23 smithi104 ceph-mon[36457]: pgmap v353: 129 pgs: 129 active+clean; 641 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 639 B/s wr, 7 op/s 2024-04-16T17:04:24.704 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:04:24 smithi045 ceph-mon[31676]: Health check failed: Degraded data redundancy: 2/194 objects degraded (1.031%), 1 pg degraded (PG_DEGRADED) 2024-04-16T17:04:24.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:04:24 smithi104 ceph-mon[36457]: Health check failed: Degraded data redundancy: 2/194 objects degraded (1.031%), 1 pg degraded (PG_DEGRADED) 2024-04-16T17:04:25.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:04:24 smithi117 ceph-mon[36730]: Health check failed: Degraded data redundancy: 2/194 objects degraded (1.031%), 1 pg degraded (PG_DEGRADED) 2024-04-16T17:04:25.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:04:25 smithi045 ceph-mon[31676]: pgmap v354: 129 pgs: 1 activating+degraded, 128 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s; 2/194 objects degraded (1.031%); 87 B/s, 0 objects/s recovering 2024-04-16T17:04:25.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:04:25 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T17:04:25.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:04:25 smithi104 ceph-mon[36457]: pgmap v354: 129 pgs: 1 activating+degraded, 128 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s; 2/194 objects degraded (1.031%); 87 B/s, 0 objects/s recovering 2024-04-16T17:04:25.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:04:25 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T17:04:26.055 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:04:25 smithi117 ceph-mon[36730]: pgmap v354: 129 pgs: 1 activating+degraded, 128 active+clean; 641 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s; 2/194 objects degraded (1.031%); 87 B/s, 0 objects/s recovering 2024-04-16T17:04:26.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:04:25 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T17:04:26.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:26.258+0000 7f1f4b200700 1 -- 172.21.15.45:0/2577625258 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1f44109380 msgr2=0x7f1f44109760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:26.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:26.258+0000 7f1f4b200700 1 --2- 172.21.15.45:0/2577625258 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1f44109380 0x7f1f44109760 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f1f34009990 tx=0x7f1f34009ca0 comp rx=0 tx=0).stop 2024-04-16T17:04:26.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:26.258+0000 7f1f4b200700 1 -- 172.21.15.45:0/2577625258 shutdown_connections 2024-04-16T17:04:26.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:26.258+0000 7f1f4b200700 1 --2- 172.21.15.45:0/2577625258 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1f4410e460 0x7f1f44116030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:26.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:26.258+0000 7f1f4b200700 1 --2- 172.21.15.45:0/2577625258 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1f44109380 0x7f1f44109760 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:26.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:26.258+0000 7f1f4b200700 1 --2- 172.21.15.45:0/2577625258 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1f44109d30 0x7f1f4410dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:26.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:26.258+0000 7f1f4b200700 1 -- 172.21.15.45:0/2577625258 >> 172.21.15.45:0/2577625258 conn(0x7f1f440762c0 msgr2=0x7f1f44107910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:04:26.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:26.258+0000 7f1f4b200700 1 -- 172.21.15.45:0/2577625258 shutdown_connections 2024-04-16T17:04:26.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:26.258+0000 7f1f4b200700 1 -- 172.21.15.45:0/2577625258 wait complete. 2024-04-16T17:04:26.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:26.259+0000 7f1f4b200700 1 Processor -- start 2024-04-16T17:04:26.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:26.260+0000 7f1f4b200700 1 -- start start 2024-04-16T17:04:26.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:26.260+0000 7f1f4b200700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1f44109380 0x7f1f441a5630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:26.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:26.260+0000 7f1f4b200700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1f44109d30 0x7f1f441a5b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:26.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:26.260+0000 7f1f4b200700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1f4410e460 0x7f1f4419f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:26.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:26.260+0000 7f1f4b200700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f1f44119490 con 0x7f1f44109380 2024-04-16T17:04:26.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:26.260+0000 7f1f4b200700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f1f44119310 con 0x7f1f44109d30 2024-04-16T17:04:26.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:26.260+0000 7f1f4b200700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f1f44119610 con 0x7f1f4410e460 2024-04-16T17:04:26.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:26.260+0000 7f1f43400700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1f44109d30 0x7f1f441a5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:26.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:26.260+0000 7f1f43e00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1f44109380 0x7f1f441a5630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:26.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:26.260+0000 7f1f43400700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1f44109d30 0x7f1f441a5b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:35244/0 (socket says 172.21.15.45:35244) 2024-04-16T17:04:26.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:26.260+0000 7f1f43400700 1 -- 172.21.15.45:0/647909871 learned_addr learned my addr 172.21.15.45:0/647909871 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:04:26.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:26.261+0000 7f1f49200700 1 --2- 172.21.15.45:0/647909871 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1f4410e460 0x7f1f4419f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:26.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:26.261+0000 7f1f43400700 1 -- 172.21.15.45:0/647909871 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1f4410e460 msgr2=0x7f1f4419f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:26.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:26.261+0000 7f1f43400700 1 --2- 172.21.15.45:0/647909871 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1f4410e460 0x7f1f4419f750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:26.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:26.261+0000 7f1f43400700 1 -- 172.21.15.45:0/647909871 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1f44109380 msgr2=0x7f1f441a5630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:26.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:26.261+0000 7f1f43400700 1 --2- 172.21.15.45:0/647909871 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1f44109380 0x7f1f441a5630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:26.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:26.261+0000 7f1f43400700 1 -- 172.21.15.45:0/647909871 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f441a0030 con 0x7f1f44109d30 2024-04-16T17:04:26.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:26.261+0000 7f1f43e00700 1 --2- 172.21.15.45:0/647909871 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1f44109380 0x7f1f441a5630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:04:26.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:26.261+0000 7f1f43400700 1 --2- 172.21.15.45:0/647909871 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1f44109d30 0x7f1f441a5b70 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f1f3000e970 tx=0x7f1f3000ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:04:26.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:26.262+0000 7f1f40c00700 1 -- 172.21.15.45:0/647909871 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f30004a30 con 0x7f1f44109d30 2024-04-16T17:04:26.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:26.262+0000 7f1f40c00700 1 -- 172.21.15.45:0/647909871 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1f3000ccc0 con 0x7f1f44109d30 2024-04-16T17:04:26.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:26.262+0000 7f1f4b200700 1 -- 172.21.15.45:0/647909871 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f441a0320 con 0x7f1f44109d30 2024-04-16T17:04:26.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:26.262+0000 7f1f40c00700 1 -- 172.21.15.45:0/647909871 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f30005650 con 0x7f1f44109d30 2024-04-16T17:04:26.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:26.262+0000 7f1f4b200700 1 -- 172.21.15.45:0/647909871 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f441ac5b0 con 0x7f1f44109d30 2024-04-16T17:04:26.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:26.263+0000 7f1f4b200700 1 -- 172.21.15.45:0/647909871 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f1f44063240 con 0x7f1f44109d30 2024-04-16T17:04:26.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:26.265+0000 7f1f40c00700 1 -- 172.21.15.45:0/647909871 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f1f30004bd0 con 0x7f1f44109d30 2024-04-16T17:04:26.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:26.266+0000 7f1f40c00700 1 --2- 172.21.15.45:0/647909871 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f1f280a3d50 0x7f1f280a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:26.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:26.267+0000 7f1f40c00700 1 -- 172.21.15.45:0/647909871 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f1f300d0d80 con 0x7f1f44109d30 2024-04-16T17:04:26.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:26.267+0000 7f1f43e00700 1 --2- 172.21.15.45:0/647909871 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f1f280a3d50 0x7f1f280a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:26.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:26.267+0000 7f1f40c00700 1 --2- 172.21.15.45:0/647909871 >> [v2:172.21.15.117:6800/2829260735,v1:172.21.15.117:6801/2829260735] conn(0x7f1f280aeeb0 0x7f1f280b1350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:26.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:26.267+0000 7f1f40c00700 1 -- 172.21.15.45:0/647909871 --> [v2:172.21.15.117:6800/2829260735,v1:172.21.15.117:6801/2829260735] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f1f280b1a20 con 0x7f1f280aeeb0 2024-04-16T17:04:26.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:26.267+0000 7f1f40c00700 1 -- 172.21.15.45:0/647909871 <== mon.2 v2:172.21.15.104:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7f1f300d1130 con 0x7f1f44109d30 2024-04-16T17:04:26.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:26.267+0000 7f1f49200700 1 --2- 172.21.15.45:0/647909871 >> [v2:172.21.15.117:6800/2829260735,v1:172.21.15.117:6801/2829260735] conn(0x7f1f280aeeb0 0x7f1f280b1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:26.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:26.268+0000 7f1f43e00700 1 --2- 172.21.15.45:0/647909871 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f1f280a3d50 0x7f1f280a6210 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f1f34006010 tx=0x7f1f3400b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:04:26.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:26.268+0000 7f1f49200700 1 --2- 172.21.15.45:0/647909871 >> [v2:172.21.15.117:6800/2829260735,v1:172.21.15.117:6801/2829260735] conn(0x7f1f280aeeb0 0x7f1f280b1350 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:04:26.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:26.269+0000 7f1f40c00700 1 -- 172.21.15.45:0/647909871 <== osd.2 v2:172.21.15.117:6800/2829260735 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f1f280b1a20 con 0x7f1f280aeeb0 2024-04-16T17:04:26.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:26.288+0000 7f1f4b200700 1 -- 172.21.15.45:0/647909871 --> [v2:172.21.15.117:6800/2829260735,v1:172.21.15.117:6801/2829260735] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f1f44063640 con 0x7f1f280aeeb0 2024-04-16T17:04:26.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:26.289+0000 7f1f40c00700 1 -- 172.21.15.45:0/647909871 <== osd.2 v2:172.21.15.117:6800/2829260735 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f1f44063640 con 0x7f1f280aeeb0 2024-04-16T17:04:26.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:26.290+0000 7f1f4b200700 1 -- 172.21.15.45:0/647909871 >> [v2:172.21.15.117:6800/2829260735,v1:172.21.15.117:6801/2829260735] conn(0x7f1f280aeeb0 msgr2=0x7f1f280b1350 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:26.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:26.290+0000 7f1f4b200700 1 --2- 172.21.15.45:0/647909871 >> [v2:172.21.15.117:6800/2829260735,v1:172.21.15.117:6801/2829260735] conn(0x7f1f280aeeb0 0x7f1f280b1350 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:26.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:26.290+0000 7f1f4b200700 1 -- 172.21.15.45:0/647909871 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f1f280a3d50 msgr2=0x7f1f280a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:26.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:26.290+0000 7f1f4b200700 1 --2- 172.21.15.45:0/647909871 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f1f280a3d50 0x7f1f280a6210 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f1f34006010 tx=0x7f1f3400b260 comp rx=0 tx=0).stop 2024-04-16T17:04:26.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:26.290+0000 7f1f4b200700 1 -- 172.21.15.45:0/647909871 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1f44109d30 msgr2=0x7f1f441a5b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:26.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:26.290+0000 7f1f4b200700 1 --2- 172.21.15.45:0/647909871 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1f44109d30 0x7f1f441a5b70 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f1f3000e970 tx=0x7f1f3000ec80 comp rx=0 tx=0).stop 2024-04-16T17:04:26.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:26.290+0000 7f1f49200700 1 -- 172.21.15.45:0/647909871 reap_dead start 2024-04-16T17:04:26.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:26.290+0000 7f1f4b200700 1 -- 172.21.15.45:0/647909871 shutdown_connections 2024-04-16T17:04:26.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:26.290+0000 7f1f4b200700 1 -- 172.21.15.45:0/647909871 >> 172.21.15.45:0/647909871 conn(0x7f1f440762c0 msgr2=0x7f1f44077930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:04:26.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:26.290+0000 7f1f4b200700 1 -- 172.21.15.45:0/647909871 shutdown_connections 2024-04-16T17:04:26.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:26.290+0000 7f1f4b200700 1 -- 172.21.15.45:0/647909871 wait complete. 2024-04-16T17:04:27.637 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:04:27 smithi045 ceph-mon[31676]: pgmap v355: 129 pgs: 1 activating+degraded, 128 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 6 op/s; 2/194 objects degraded (1.031%); 91 B/s, 1 keys/s, 1 objects/s recovering 2024-04-16T17:04:27.637 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:04:27 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:04:27.637 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:04:27 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:04:27.637 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:04:27 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:04:27.637 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:04:27 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:04:27.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:04:27 smithi104 ceph-mon[36457]: pgmap v355: 129 pgs: 1 activating+degraded, 128 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 6 op/s; 2/194 objects degraded (1.031%); 91 B/s, 1 keys/s, 1 objects/s recovering 2024-04-16T17:04:27.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:04:27 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:04:27.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:04:27 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:04:27.929 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:04:27 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:04:27.929 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:04:27 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:04:28.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:04:27 smithi117 ceph-mon[36730]: pgmap v355: 129 pgs: 1 activating+degraded, 128 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 6 op/s; 2/194 objects degraded (1.031%); 91 B/s, 1 keys/s, 1 objects/s recovering 2024-04-16T17:04:28.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:04:27 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:04:28.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:04:27 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:04:28.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:04:27 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:04:28.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:04:27 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:04:28.925 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:04:28 smithi045 ceph-mon[31676]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/194 objects degraded (1.031%), 1 pg degraded) 2024-04-16T17:04:28.925 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:04:28 smithi045 ceph-mon[31676]: Cluster is now healthy 2024-04-16T17:04:28.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:04:28 smithi104 ceph-mon[36457]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/194 objects degraded (1.031%), 1 pg degraded) 2024-04-16T17:04:28.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:04:28 smithi104 ceph-mon[36457]: Cluster is now healthy 2024-04-16T17:04:29.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:04:28 smithi117 ceph-mon[36730]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/194 objects degraded (1.031%), 1 pg degraded) 2024-04-16T17:04:29.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:04:28 smithi117 ceph-mon[36730]: Cluster is now healthy 2024-04-16T17:04:29.858 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:04:29 smithi045 ceph-mon[31676]: pgmap v356: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 255 B/s wr, 5 op/s; 91 B/s, 2 keys/s, 1 objects/s recovering 2024-04-16T17:04:29.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:04:29 smithi104 ceph-mon[36457]: pgmap v356: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 255 B/s wr, 5 op/s; 91 B/s, 2 keys/s, 1 objects/s recovering 2024-04-16T17:04:30.055 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:04:29 smithi117 ceph-mon[36730]: pgmap v356: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 255 B/s wr, 5 op/s; 91 B/s, 2 keys/s, 1 objects/s recovering 2024-04-16T17:04:30.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:29.779+0000 7f10d5a00700 1 -- 172.21.15.45:0/1712773714 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f10d0109380 msgr2=0x7f10d0109760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:30.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:29.779+0000 7f10d5a00700 1 --2- 172.21.15.45:0/1712773714 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f10d0109380 0x7f10d0109760 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f10c00099c0 tx=0x7f10c0009cd0 comp rx=0 tx=0).stop 2024-04-16T17:04:30.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:29.779+0000 7f10d5a00700 1 -- 172.21.15.45:0/1712773714 shutdown_connections 2024-04-16T17:04:30.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:29.779+0000 7f10d5a00700 1 --2- 172.21.15.45:0/1712773714 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f10d010e460 0x7f10d0116030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:30.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:29.779+0000 7f10d5a00700 1 --2- 172.21.15.45:0/1712773714 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f10d0109380 0x7f10d0109760 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:30.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:29.779+0000 7f10d5a00700 1 --2- 172.21.15.45:0/1712773714 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f10d0109d30 0x7f10d010dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:30.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:29.779+0000 7f10d5a00700 1 -- 172.21.15.45:0/1712773714 >> 172.21.15.45:0/1712773714 conn(0x7f10d00762c0 msgr2=0x7f10d00786e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:04:30.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:29.779+0000 7f10d5a00700 1 -- 172.21.15.45:0/1712773714 shutdown_connections 2024-04-16T17:04:30.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:29.779+0000 7f10d5a00700 1 -- 172.21.15.45:0/1712773714 wait complete. 2024-04-16T17:04:30.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:29.780+0000 7f10d5a00700 1 Processor -- start 2024-04-16T17:04:30.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:29.780+0000 7f10d5a00700 1 -- start start 2024-04-16T17:04:30.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:29.781+0000 7f10d5a00700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f10d0109380 0x7f10d01a5670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:30.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:29.781+0000 7f10d5a00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f10d0109d30 0x7f10d01a5bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:30.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:29.781+0000 7f10d5a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f10d010e460 0x7f10d019f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:30.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:29.781+0000 7f10d5a00700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f10d0119490 con 0x7f10d0109d30 2024-04-16T17:04:30.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:29.781+0000 7f10d5a00700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f10d0119310 con 0x7f10d0109380 2024-04-16T17:04:30.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:29.781+0000 7f10d5a00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f10d0119610 con 0x7f10d010e460 2024-04-16T17:04:30.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:29.781+0000 7f10cea00700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f10d0109380 0x7f10d01a5670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:30.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:29.781+0000 7f10ce000700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f10d0109d30 0x7f10d01a5bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:30.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:29.781+0000 7f10cea00700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f10d0109380 0x7f10d01a5670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:35256/0 (socket says 172.21.15.45:35256) 2024-04-16T17:04:30.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:29.781+0000 7f10cea00700 1 -- 172.21.15.45:0/1031912631 learned_addr learned my addr 172.21.15.45:0/1031912631 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:04:30.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:29.781+0000 7f10cf400700 1 --2- 172.21.15.45:0/1031912631 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f10d010e460 0x7f10d019f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:30.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:29.782+0000 7f10cea00700 1 -- 172.21.15.45:0/1031912631 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f10d010e460 msgr2=0x7f10d019f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:30.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:29.782+0000 7f10cea00700 1 --2- 172.21.15.45:0/1031912631 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f10d010e460 0x7f10d019f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:30.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:29.782+0000 7f10cea00700 1 -- 172.21.15.45:0/1031912631 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f10d0109d30 msgr2=0x7f10d01a5bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:30.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:29.782+0000 7f10cea00700 1 --2- 172.21.15.45:0/1031912631 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f10d0109d30 0x7f10d01a5bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:30.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:29.782+0000 7f10cea00700 1 -- 172.21.15.45:0/1031912631 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10d01a0070 con 0x7f10d0109380 2024-04-16T17:04:30.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:29.782+0000 7f10ce000700 1 --2- 172.21.15.45:0/1031912631 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f10d0109d30 0x7f10d01a5bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:04:30.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:29.782+0000 7f10cea00700 1 --2- 172.21.15.45:0/1031912631 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f10d0109380 0x7f10d01a5670 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f10c0009990 tx=0x7f10c0009f80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:04:30.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:29.782+0000 7f10c7400700 1 -- 172.21.15.45:0/1031912631 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10c0027070 con 0x7f10d0109380 2024-04-16T17:04:30.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:29.782+0000 7f10c7400700 1 -- 172.21.15.45:0/1031912631 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f10c0003e40 con 0x7f10d0109380 2024-04-16T17:04:30.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:29.782+0000 7f10d5a00700 1 -- 172.21.15.45:0/1031912631 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10d01a0300 con 0x7f10d0109380 2024-04-16T17:04:30.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:29.782+0000 7f10c7400700 1 -- 172.21.15.45:0/1031912631 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10c0022bf0 con 0x7f10d0109380 2024-04-16T17:04:30.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:29.783+0000 7f10d5a00700 1 -- 172.21.15.45:0/1031912631 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10d01ac420 con 0x7f10d0109380 2024-04-16T17:04:30.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:29.783+0000 7f10d5a00700 1 -- 172.21.15.45:0/1031912631 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f10d0063240 con 0x7f10d0109380 2024-04-16T17:04:30.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:29.785+0000 7f10c7400700 1 -- 172.21.15.45:0/1031912631 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f10c002a070 con 0x7f10d0109380 2024-04-16T17:04:30.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:29.787+0000 7f10c7400700 1 --2- 172.21.15.45:0/1031912631 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f10b40a3cd0 0x7f10b40a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:30.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:29.787+0000 7f10c7400700 1 -- 172.21.15.45:0/1031912631 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f10c00d1f90 con 0x7f10d0109380 2024-04-16T17:04:30.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:29.787+0000 7f10ce000700 1 --2- 172.21.15.45:0/1031912631 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f10b40a3cd0 0x7f10b40a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:30.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:29.787+0000 7f10c7400700 1 --2- 172.21.15.45:0/1031912631 >> [v2:172.21.15.117:6824/3384003243,v1:172.21.15.117:6825/3384003243] conn(0x7f10b40aee30 0x7f10b40b12d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:30.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:29.787+0000 7f10c7400700 1 -- 172.21.15.45:0/1031912631 --> [v2:172.21.15.117:6824/3384003243,v1:172.21.15.117:6825/3384003243] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f10c00043d0 con 0x7f10b40aee30 2024-04-16T17:04:30.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:29.787+0000 7f10c7400700 1 -- 172.21.15.45:0/1031912631 <== mon.2 v2:172.21.15.104:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7f10c00d2340 con 0x7f10d0109380 2024-04-16T17:04:30.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:29.788+0000 7f10cf400700 1 --2- 172.21.15.45:0/1031912631 >> [v2:172.21.15.117:6824/3384003243,v1:172.21.15.117:6825/3384003243] conn(0x7f10b40aee30 0x7f10b40b12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:30.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:29.788+0000 7f10ce000700 1 --2- 172.21.15.45:0/1031912631 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f10b40a3cd0 0x7f10b40a6190 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f10d01a0f00 tx=0x7f10bc0073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:04:30.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:29.788+0000 7f10cf400700 1 --2- 172.21.15.45:0/1031912631 >> [v2:172.21.15.117:6824/3384003243,v1:172.21.15.117:6825/3384003243] conn(0x7f10b40aee30 0x7f10b40b12d0 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:04:30.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:29.789+0000 7f10c7400700 1 -- 172.21.15.45:0/1031912631 <== osd.11 v2:172.21.15.117:6824/3384003243 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f10c00043d0 con 0x7f10b40aee30 2024-04-16T17:04:30.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:29.807+0000 7f10d5a00700 1 -- 172.21.15.45:0/1031912631 --> [v2:172.21.15.117:6824/3384003243,v1:172.21.15.117:6825/3384003243] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f10d00636c0 con 0x7f10b40aee30 2024-04-16T17:04:30.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:29.808+0000 7f10c7400700 1 -- 172.21.15.45:0/1031912631 <== osd.11 v2:172.21.15.117:6824/3384003243 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f10d00636c0 con 0x7f10b40aee30 2024-04-16T17:04:30.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:29.808+0000 7f10d5a00700 1 -- 172.21.15.45:0/1031912631 >> [v2:172.21.15.117:6824/3384003243,v1:172.21.15.117:6825/3384003243] conn(0x7f10b40aee30 msgr2=0x7f10b40b12d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:30.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:29.809+0000 7f10d5a00700 1 --2- 172.21.15.45:0/1031912631 >> [v2:172.21.15.117:6824/3384003243,v1:172.21.15.117:6825/3384003243] conn(0x7f10b40aee30 0x7f10b40b12d0 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:30.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:29.809+0000 7f10d5a00700 1 -- 172.21.15.45:0/1031912631 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f10b40a3cd0 msgr2=0x7f10b40a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:30.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:29.809+0000 7f10d5a00700 1 --2- 172.21.15.45:0/1031912631 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f10b40a3cd0 0x7f10b40a6190 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f10d01a0f00 tx=0x7f10bc0073d0 comp rx=0 tx=0).stop 2024-04-16T17:04:30.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:29.809+0000 7f10d5a00700 1 -- 172.21.15.45:0/1031912631 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f10d0109380 msgr2=0x7f10d01a5670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:30.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:29.809+0000 7f10d5a00700 1 --2- 172.21.15.45:0/1031912631 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f10d0109380 0x7f10d01a5670 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f10c0009990 tx=0x7f10c0009f80 comp rx=0 tx=0).stop 2024-04-16T17:04:30.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:29.809+0000 7f10cf400700 1 -- 172.21.15.45:0/1031912631 reap_dead start 2024-04-16T17:04:30.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:29.809+0000 7f10d5a00700 1 -- 172.21.15.45:0/1031912631 shutdown_connections 2024-04-16T17:04:30.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:29.809+0000 7f10d5a00700 1 -- 172.21.15.45:0/1031912631 >> 172.21.15.45:0/1031912631 conn(0x7f10d00762c0 msgr2=0x7f10d0078600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:04:30.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:29.809+0000 7f10d5a00700 1 -- 172.21.15.45:0/1031912631 shutdown_connections 2024-04-16T17:04:30.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:29.809+0000 7f10d5a00700 1 -- 172.21.15.45:0/1031912631 wait complete. 2024-04-16T17:04:31.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:04:31 smithi104 ceph-mon[36457]: pgmap v357: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 212 B/s wr, 5 op/s; 76 B/s, 2 keys/s, 1 objects/s recovering 2024-04-16T17:04:31.978 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:04:31 smithi045 ceph-mon[31676]: pgmap v357: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 212 B/s wr, 5 op/s; 76 B/s, 2 keys/s, 1 objects/s recovering 2024-04-16T17:04:32.055 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:04:31 smithi117 ceph-mon[36730]: pgmap v357: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 212 B/s wr, 5 op/s; 76 B/s, 2 keys/s, 1 objects/s recovering 2024-04-16T17:04:32.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:32.434+0000 7f373da00700 1 -- 172.21.15.45:0/1226291015 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3738103be0 msgr2=0x7f3738103fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:32.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:32.434+0000 7f373da00700 1 --2- 172.21.15.45:0/1226291015 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3738103be0 0x7f3738103fc0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f3728009990 tx=0x7f3728009ca0 comp rx=0 tx=0).stop 2024-04-16T17:04:32.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:32.434+0000 7f373da00700 1 -- 172.21.15.45:0/1226291015 shutdown_connections 2024-04-16T17:04:32.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:32.434+0000 7f373da00700 1 --2- 172.21.15.45:0/1226291015 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f373810d010 0x7f373810f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:32.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:32.434+0000 7f373da00700 1 --2- 172.21.15.45:0/1226291015 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3738103be0 0x7f3738103fc0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:32.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:32.434+0000 7f373da00700 1 --2- 172.21.15.45:0/1226291015 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3738104500 0x7f373810c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:32.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:32.434+0000 7f373da00700 1 -- 172.21.15.45:0/1226291015 >> 172.21.15.45:0/1226291015 conn(0x7f37380fd410 msgr2=0x7f37380ff830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:04:32.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:32.434+0000 7f373da00700 1 -- 172.21.15.45:0/1226291015 shutdown_connections 2024-04-16T17:04:32.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:32.434+0000 7f373da00700 1 -- 172.21.15.45:0/1226291015 wait complete. 2024-04-16T17:04:32.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:32.435+0000 7f373da00700 1 Processor -- start 2024-04-16T17:04:32.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:32.436+0000 7f373da00700 1 -- start start 2024-04-16T17:04:32.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:32.436+0000 7f373da00700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3738103be0 0x7f373819b420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:32.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:32.436+0000 7f373da00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3738104500 0x7f373819b960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:32.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:32.436+0000 7f373da00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f373810d010 0x7f373819fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:32.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:32.436+0000 7f373da00700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f3738112dc0 con 0x7f3738104500 2024-04-16T17:04:32.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:32.436+0000 7f373da00700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f3738112c40 con 0x7f3738103be0 2024-04-16T17:04:32.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:32.436+0000 7f373da00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f3738112f40 con 0x7f373810d010 2024-04-16T17:04:32.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:32.436+0000 7f3737e00700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3738103be0 0x7f373819b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:32.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:32.436+0000 7f3737400700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3738104500 0x7f373819b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:32.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:32.436+0000 7f3737e00700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3738103be0 0x7f373819b420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:54654/0 (socket says 172.21.15.45:54654) 2024-04-16T17:04:32.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:32.436+0000 7f3737e00700 1 -- 172.21.15.45:0/158609747 learned_addr learned my addr 172.21.15.45:0/158609747 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:04:32.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:32.436+0000 7f373d000700 1 --2- 172.21.15.45:0/158609747 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f373810d010 0x7f373819fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:32.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:32.437+0000 7f3737400700 1 -- 172.21.15.45:0/158609747 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f373810d010 msgr2=0x7f373819fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:32.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:32.437+0000 7f3737400700 1 --2- 172.21.15.45:0/158609747 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f373810d010 0x7f373819fcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:32.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:32.437+0000 7f3737400700 1 -- 172.21.15.45:0/158609747 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3738103be0 msgr2=0x7f373819b420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:32.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:32.437+0000 7f3737400700 1 --2- 172.21.15.45:0/158609747 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3738103be0 0x7f373819b420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:32.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:32.437+0000 7f3737400700 1 -- 172.21.15.45:0/158609747 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37381a0430 con 0x7f3738104500 2024-04-16T17:04:32.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:32.437+0000 7f373d000700 1 --2- 172.21.15.45:0/158609747 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f373810d010 0x7f373819fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:04:32.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:32.437+0000 7f3737400700 1 --2- 172.21.15.45:0/158609747 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3738104500 0x7f373819b960 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f372400e9c0 tx=0x7f372400ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:04:32.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:32.437+0000 7f3734c00700 1 -- 172.21.15.45:0/158609747 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f372400c9c0 con 0x7f3738104500 2024-04-16T17:04:32.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:32.438+0000 7f3734c00700 1 -- 172.21.15.45:0/158609747 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f372400cb60 con 0x7f3738104500 2024-04-16T17:04:32.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:32.438+0000 7f3734c00700 1 -- 172.21.15.45:0/158609747 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3724018680 con 0x7f3738104500 2024-04-16T17:04:32.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:32.438+0000 7f373da00700 1 -- 172.21.15.45:0/158609747 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37381a0720 con 0x7f3738104500 2024-04-16T17:04:32.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:32.438+0000 7f373da00700 1 -- 172.21.15.45:0/158609747 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37381a7fa0 con 0x7f3738104500 2024-04-16T17:04:32.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:32.439+0000 7f373da00700 1 -- 172.21.15.45:0/158609747 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f3738103340 con 0x7f3738104500 2024-04-16T17:04:32.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:32.441+0000 7f3734c00700 1 -- 172.21.15.45:0/158609747 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f3724010430 con 0x7f3738104500 2024-04-16T17:04:32.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:32.442+0000 7f3734c00700 1 --2- 172.21.15.45:0/158609747 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f371c0a3d50 0x7f371c0a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:32.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:32.442+0000 7f3734c00700 1 -- 172.21.15.45:0/158609747 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f37240d0fc0 con 0x7f3738104500 2024-04-16T17:04:32.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:32.443+0000 7f3737e00700 1 --2- 172.21.15.45:0/158609747 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f371c0a3d50 0x7f371c0a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:32.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:32.443+0000 7f3734c00700 1 --2- 172.21.15.45:0/158609747 >> [v2:172.21.15.104:6800/1462942067,v1:172.21.15.104:6801/1462942067] conn(0x7f371c0aeeb0 0x7f371c0b1350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:32.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:32.443+0000 7f3734c00700 1 -- 172.21.15.45:0/158609747 --> [v2:172.21.15.104:6800/1462942067,v1:172.21.15.104:6801/1462942067] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f3724019ec0 con 0x7f371c0aeeb0 2024-04-16T17:04:32.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:32.443+0000 7f3734c00700 1 -- 172.21.15.45:0/158609747 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7f37240d1370 con 0x7f3738104500 2024-04-16T17:04:32.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:32.443+0000 7f3737e00700 1 --2- 172.21.15.45:0/158609747 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f371c0a3d50 0x7f371c0a6210 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f3728009750 tx=0x7f372800b280 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:04:32.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:32.443+0000 7f373d000700 1 --2- 172.21.15.45:0/158609747 >> [v2:172.21.15.104:6800/1462942067,v1:172.21.15.104:6801/1462942067] conn(0x7f371c0aeeb0 0x7f371c0b1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:32.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:32.444+0000 7f373d000700 1 --2- 172.21.15.45:0/158609747 >> [v2:172.21.15.104:6800/1462942067,v1:172.21.15.104:6801/1462942067] conn(0x7f371c0aeeb0 0x7f371c0b1350 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:04:32.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:32.445+0000 7f3734c00700 1 -- 172.21.15.45:0/158609747 <== osd.1 v2:172.21.15.104:6800/1462942067 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f3724019ec0 con 0x7f371c0aeeb0 2024-04-16T17:04:32.464 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:32.464+0000 7f373da00700 1 -- 172.21.15.45:0/158609747 --> [v2:172.21.15.104:6800/1462942067,v1:172.21.15.104:6801/1462942067] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f373806b0b0 con 0x7f371c0aeeb0 2024-04-16T17:04:32.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:32.465+0000 7f3734c00700 1 -- 172.21.15.45:0/158609747 <== osd.1 v2:172.21.15.104:6800/1462942067 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f373806b0b0 con 0x7f371c0aeeb0 2024-04-16T17:04:32.466 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:32.465+0000 7f373da00700 1 -- 172.21.15.45:0/158609747 >> [v2:172.21.15.104:6800/1462942067,v1:172.21.15.104:6801/1462942067] conn(0x7f371c0aeeb0 msgr2=0x7f371c0b1350 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:32.466 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:32.465+0000 7f373da00700 1 --2- 172.21.15.45:0/158609747 >> [v2:172.21.15.104:6800/1462942067,v1:172.21.15.104:6801/1462942067] conn(0x7f371c0aeeb0 0x7f371c0b1350 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:32.466 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:32.465+0000 7f373da00700 1 -- 172.21.15.45:0/158609747 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f371c0a3d50 msgr2=0x7f371c0a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:32.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:32.465+0000 7f373da00700 1 --2- 172.21.15.45:0/158609747 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f371c0a3d50 0x7f371c0a6210 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f3728009750 tx=0x7f372800b280 comp rx=0 tx=0).stop 2024-04-16T17:04:32.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:32.466+0000 7f373da00700 1 -- 172.21.15.45:0/158609747 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3738104500 msgr2=0x7f373819b960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:32.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:32.466+0000 7f373da00700 1 --2- 172.21.15.45:0/158609747 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3738104500 0x7f373819b960 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f372400e9c0 tx=0x7f372400ecd0 comp rx=0 tx=0).stop 2024-04-16T17:04:32.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:32.466+0000 7f373d000700 1 -- 172.21.15.45:0/158609747 reap_dead start 2024-04-16T17:04:32.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:32.466+0000 7f373da00700 1 -- 172.21.15.45:0/158609747 shutdown_connections 2024-04-16T17:04:32.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:32.466+0000 7f373da00700 1 -- 172.21.15.45:0/158609747 >> 172.21.15.45:0/158609747 conn(0x7f37380fd410 msgr2=0x7f373810f2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:04:32.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:32.466+0000 7f373da00700 1 -- 172.21.15.45:0/158609747 shutdown_connections 2024-04-16T17:04:32.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:32.466+0000 7f373da00700 1 -- 172.21.15.45:0/158609747 wait complete. 2024-04-16T17:04:33.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:33.281+0000 7f689b200700 1 -- 172.21.15.45:0/2619872749 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6894102b40 msgr2=0x7f6894102fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:33.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:33.281+0000 7f689b200700 1 --2- 172.21.15.45:0/2619872749 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6894102b40 0x7f6894102fc0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f6880009990 tx=0x7f6880009ca0 comp rx=0 tx=0).stop 2024-04-16T17:04:33.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:33.281+0000 7f689b200700 1 -- 172.21.15.45:0/2619872749 shutdown_connections 2024-04-16T17:04:33.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:33.281+0000 7f689b200700 1 --2- 172.21.15.45:0/2619872749 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6894108b90 0x7f6894108f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:33.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:33.281+0000 7f689b200700 1 --2- 172.21.15.45:0/2619872749 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6894103500 0x7f689410f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:33.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:33.281+0000 7f689b200700 1 --2- 172.21.15.45:0/2619872749 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6894102b40 0x7f6894102fc0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:33.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:33.281+0000 7f689b200700 1 -- 172.21.15.45:0/2619872749 >> 172.21.15.45:0/2619872749 conn(0x7f6894079c50 msgr2=0x7f689407a060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:04:33.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:33.282+0000 7f689b200700 1 -- 172.21.15.45:0/2619872749 shutdown_connections 2024-04-16T17:04:33.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:33.282+0000 7f689b200700 1 -- 172.21.15.45:0/2619872749 wait complete. 2024-04-16T17:04:33.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:33.283+0000 7f689b200700 1 Processor -- start 2024-04-16T17:04:33.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:33.283+0000 7f689b200700 1 -- start start 2024-04-16T17:04:33.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:33.283+0000 7f689b200700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6894102b40 0x7f689419b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:33.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:33.283+0000 7f689b200700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6894103500 0x7f689419baf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:33.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:33.283+0000 7f689b200700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6894108b90 0x7f689419fde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:33.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:33.283+0000 7f689b200700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f6894112e40 con 0x7f6894102b40 2024-04-16T17:04:33.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:33.283+0000 7f689b200700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f6894112cc0 con 0x7f6894108b90 2024-04-16T17:04:33.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:33.283+0000 7f689b200700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f6894112fc0 con 0x7f6894103500 2024-04-16T17:04:33.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:33.284+0000 7f6893e00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6894102b40 0x7f689419b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:33.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:33.284+0000 7f6893e00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6894102b40 0x7f689419b5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52062/0 (socket says 172.21.15.45:52062) 2024-04-16T17:04:33.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:33.284+0000 7f6893e00700 1 -- 172.21.15.45:0/2805631283 learned_addr learned my addr 172.21.15.45:0/2805631283 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:04:33.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:33.284+0000 7f6893400700 1 --2- 172.21.15.45:0/2805631283 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6894103500 0x7f689419baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:33.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:33.284+0000 7f6899200700 1 --2- 172.21.15.45:0/2805631283 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6894108b90 0x7f689419fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:33.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:33.284+0000 7f6893e00700 1 -- 172.21.15.45:0/2805631283 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6894103500 msgr2=0x7f689419baf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:33.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:33.284+0000 7f6893e00700 1 --2- 172.21.15.45:0/2805631283 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6894103500 0x7f689419baf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:33.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:33.284+0000 7f6893e00700 1 -- 172.21.15.45:0/2805631283 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6894108b90 msgr2=0x7f689419fde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:33.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:33.284+0000 7f6893e00700 1 --2- 172.21.15.45:0/2805631283 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6894108b90 0x7f689419fde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:33.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:33.284+0000 7f6893e00700 1 -- 172.21.15.45:0/2805631283 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68941a0520 con 0x7f6894102b40 2024-04-16T17:04:33.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:33.284+0000 7f6893400700 1 --2- 172.21.15.45:0/2805631283 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6894103500 0x7f689419baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:04:33.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:33.284+0000 7f6899200700 1 --2- 172.21.15.45:0/2805631283 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6894108b90 0x7f689419fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:04:33.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:33.285+0000 7f6893e00700 1 --2- 172.21.15.45:0/2805631283 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6894102b40 0x7f689419b5b0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f688400e9a0 tx=0x7f688400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:04:33.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:33.285+0000 7f6890c00700 1 -- 172.21.15.45:0/2805631283 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f688400c9c0 con 0x7f6894102b40 2024-04-16T17:04:33.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:33.285+0000 7f6890c00700 1 -- 172.21.15.45:0/2805631283 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f688400cb60 con 0x7f6894102b40 2024-04-16T17:04:33.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:33.285+0000 7f689b200700 1 -- 172.21.15.45:0/2805631283 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68941a0810 con 0x7f6894102b40 2024-04-16T17:04:33.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:33.285+0000 7f6890c00700 1 -- 172.21.15.45:0/2805631283 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6884018660 con 0x7f6894102b40 2024-04-16T17:04:33.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:33.285+0000 7f689b200700 1 -- 172.21.15.45:0/2805631283 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68941a8090 con 0x7f6894102b40 2024-04-16T17:04:33.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:33.286+0000 7f689b200700 1 -- 172.21.15.45:0/2805631283 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f6894102050 con 0x7f6894102b40 2024-04-16T17:04:33.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:33.288+0000 7f6890c00700 1 -- 172.21.15.45:0/2805631283 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f688400cd10 con 0x7f6894102b40 2024-04-16T17:04:33.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:33.290+0000 7f6890c00700 1 --2- 172.21.15.45:0/2805631283 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f68780a3cd0 0x7f68780a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:33.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:33.290+0000 7f6890c00700 1 -- 172.21.15.45:0/2805631283 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f68840d0f20 con 0x7f6894102b40 2024-04-16T17:04:33.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:33.290+0000 7f6893400700 1 --2- 172.21.15.45:0/2805631283 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f68780a3cd0 0x7f68780a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:33.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:33.290+0000 7f6890c00700 1 --2- 172.21.15.45:0/2805631283 >> [v2:172.21.15.45:6802/2290170881,v1:172.21.15.45:6803/2290170881] conn(0x7f68780aee30 0x7f68780b12d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:33.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:33.290+0000 7f6890c00700 1 -- 172.21.15.45:0/2805631283 --> [v2:172.21.15.45:6802/2290170881,v1:172.21.15.45:6803/2290170881] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6884019ec0 con 0x7f68780aee30 2024-04-16T17:04:33.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:33.290+0000 7f6890c00700 1 -- 172.21.15.45:0/2805631283 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7f68840d12d0 con 0x7f6894102b40 2024-04-16T17:04:33.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:33.291+0000 7f6899200700 1 --2- 172.21.15.45:0/2805631283 >> [v2:172.21.15.45:6802/2290170881,v1:172.21.15.45:6803/2290170881] conn(0x7f68780aee30 0x7f68780b12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:33.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:33.291+0000 7f6893400700 1 --2- 172.21.15.45:0/2805631283 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f68780a3cd0 0x7f68780a6190 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f689410fda0 tx=0x7f688000b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:04:33.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:33.291+0000 7f6899200700 1 --2- 172.21.15.45:0/2805631283 >> [v2:172.21.15.45:6802/2290170881,v1:172.21.15.45:6803/2290170881] conn(0x7f68780aee30 0x7f68780b12d0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:04:33.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:33.292+0000 7f6890c00700 1 -- 172.21.15.45:0/2805631283 <== osd.0 v2:172.21.15.45:6802/2290170881 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f6884019ec0 con 0x7f68780aee30 2024-04-16T17:04:33.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:33.312+0000 7f689b200700 1 -- 172.21.15.45:0/2805631283 --> [v2:172.21.15.45:6802/2290170881,v1:172.21.15.45:6803/2290170881] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f68941041f0 con 0x7f68780aee30 2024-04-16T17:04:33.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:33.313+0000 7f6890c00700 1 -- 172.21.15.45:0/2805631283 <== osd.0 v2:172.21.15.45:6802/2290170881 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f68941041f0 con 0x7f68780aee30 2024-04-16T17:04:33.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:33.313+0000 7f689b200700 1 -- 172.21.15.45:0/2805631283 >> [v2:172.21.15.45:6802/2290170881,v1:172.21.15.45:6803/2290170881] conn(0x7f68780aee30 msgr2=0x7f68780b12d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:33.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:33.313+0000 7f689b200700 1 --2- 172.21.15.45:0/2805631283 >> [v2:172.21.15.45:6802/2290170881,v1:172.21.15.45:6803/2290170881] conn(0x7f68780aee30 0x7f68780b12d0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:33.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:33.313+0000 7f689b200700 1 -- 172.21.15.45:0/2805631283 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f68780a3cd0 msgr2=0x7f68780a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:33.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:33.313+0000 7f689b200700 1 --2- 172.21.15.45:0/2805631283 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f68780a3cd0 0x7f68780a6190 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f689410fda0 tx=0x7f688000b260 comp rx=0 tx=0).stop 2024-04-16T17:04:33.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:33.313+0000 7f689b200700 1 -- 172.21.15.45:0/2805631283 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6894102b40 msgr2=0x7f689419b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:33.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:33.313+0000 7f689b200700 1 --2- 172.21.15.45:0/2805631283 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6894102b40 0x7f689419b5b0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f688400e9a0 tx=0x7f688400ecb0 comp rx=0 tx=0).stop 2024-04-16T17:04:33.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:33.313+0000 7f6899200700 1 -- 172.21.15.45:0/2805631283 reap_dead start 2024-04-16T17:04:33.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:33.313+0000 7f689b200700 1 -- 172.21.15.45:0/2805631283 shutdown_connections 2024-04-16T17:04:33.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:33.313+0000 7f689b200700 1 -- 172.21.15.45:0/2805631283 >> 172.21.15.45:0/2805631283 conn(0x7f6894079c50 msgr2=0x7f68941015f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:04:33.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:33.314+0000 7f689b200700 1 -- 172.21.15.45:0/2805631283 shutdown_connections 2024-04-16T17:04:33.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:33.314+0000 7f689b200700 1 -- 172.21.15.45:0/2805631283 wait complete. 2024-04-16T17:04:33.920 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:04:33 smithi045 ceph-mon[31676]: pgmap v358: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 192 B/s wr, 5 op/s; 68 B/s, 1 keys/s, 1 objects/s recovering 2024-04-16T17:04:33.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:04:33 smithi104 ceph-mon[36457]: pgmap v358: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 192 B/s wr, 5 op/s; 68 B/s, 1 keys/s, 1 objects/s recovering 2024-04-16T17:04:33.976 INFO:teuthology.orchestra.run.smithi045.stdout:77309411406 2024-04-16T17:04:33.976 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45 shell --fsid b1fecd38-fc11-11ee-bc90-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-04-16T17:04:34.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:04:33 smithi117 ceph-mon[36730]: pgmap v358: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 192 B/s wr, 5 op/s; 68 B/s, 1 keys/s, 1 objects/s recovering 2024-04-16T17:04:35.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:35.494+0000 7f0325e00700 1 -- 172.21.15.45:0/3062914664 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0320105920 msgr2=0x7f0320109970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:35.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:35.494+0000 7f0325e00700 1 --2- 172.21.15.45:0/3062914664 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0320105920 0x7f0320109970 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f030c009990 tx=0x7f030c009ca0 comp rx=0 tx=0).stop 2024-04-16T17:04:35.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:35.494+0000 7f0325e00700 1 -- 172.21.15.45:0/3062914664 shutdown_connections 2024-04-16T17:04:35.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:35.494+0000 7f0325e00700 1 --2- 172.21.15.45:0/3062914664 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0320104f70 0x7f0320105350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:35.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:35.494+0000 7f0325e00700 1 --2- 172.21.15.45:0/3062914664 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f032010a050 0x7f0320111c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:35.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:35.494+0000 7f0325e00700 1 --2- 172.21.15.45:0/3062914664 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0320105920 0x7f0320109970 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:35.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:35.494+0000 7f0325e00700 1 -- 172.21.15.45:0/3062914664 >> 172.21.15.45:0/3062914664 conn(0x7f0320100800 msgr2=0x7f0320102c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:04:35.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:35.494+0000 7f0325e00700 1 -- 172.21.15.45:0/3062914664 shutdown_connections 2024-04-16T17:04:35.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:35.494+0000 7f0325e00700 1 -- 172.21.15.45:0/3062914664 wait complete. 2024-04-16T17:04:35.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:35.496+0000 7f0325e00700 1 Processor -- start 2024-04-16T17:04:35.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:35.496+0000 7f0325e00700 1 -- start start 2024-04-16T17:04:35.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:35.497+0000 7f0325e00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0320104f70 0x7f032019b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:35.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:35.497+0000 7f0325e00700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0320105920 0x7f032019bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:35.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:35.497+0000 7f0325e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f032010a050 0x7f032019fe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:35.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:35.497+0000 7f0325e00700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f0320115030 con 0x7f0320104f70 2024-04-16T17:04:35.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:35.497+0000 7f0325e00700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f0320114eb0 con 0x7f0320105920 2024-04-16T17:04:35.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:35.497+0000 7f0325e00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f03201151b0 con 0x7f032010a050 2024-04-16T17:04:35.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:35.497+0000 7f031ea00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0320104f70 0x7f032019b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:35.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:35.497+0000 7f031ea00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0320104f70 0x7f032019b580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52078/0 (socket says 172.21.15.45:52078) 2024-04-16T17:04:35.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:35.497+0000 7f031ea00700 1 -- 172.21.15.45:0/2081206795 learned_addr learned my addr 172.21.15.45:0/2081206795 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:04:35.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:35.497+0000 7f031e000700 1 --2- 172.21.15.45:0/2081206795 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0320105920 0x7f032019bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:35.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:35.497+0000 7f031f400700 1 --2- 172.21.15.45:0/2081206795 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f032010a050 0x7f032019fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:35.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:35.498+0000 7f031e000700 1 -- 172.21.15.45:0/2081206795 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f032010a050 msgr2=0x7f032019fe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:35.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:35.498+0000 7f031e000700 1 --2- 172.21.15.45:0/2081206795 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f032010a050 0x7f032019fe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:35.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:35.498+0000 7f031e000700 1 -- 172.21.15.45:0/2081206795 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0320104f70 msgr2=0x7f032019b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:35.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:35.498+0000 7f031e000700 1 --2- 172.21.15.45:0/2081206795 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0320104f70 0x7f032019b580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:35.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:35.498+0000 7f031e000700 1 -- 172.21.15.45:0/2081206795 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03201a0540 con 0x7f0320105920 2024-04-16T17:04:35.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:35.498+0000 7f031ea00700 1 --2- 172.21.15.45:0/2081206795 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0320104f70 0x7f032019b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:04:35.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:35.498+0000 7f031e000700 1 --2- 172.21.15.45:0/2081206795 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0320105920 0x7f032019bac0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f030c0059d0 tx=0x7f030c005590 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:04:35.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:35.498+0000 7f0324a00700 1 -- 172.21.15.45:0/2081206795 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f030c027070 con 0x7f0320105920 2024-04-16T17:04:35.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:35.498+0000 7f0324a00700 1 -- 172.21.15.45:0/2081206795 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f030c0056c0 con 0x7f0320105920 2024-04-16T17:04:35.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:35.498+0000 7f0325e00700 1 -- 172.21.15.45:0/2081206795 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03201a07d0 con 0x7f0320105920 2024-04-16T17:04:35.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:35.498+0000 7f0324a00700 1 -- 172.21.15.45:0/2081206795 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f030c0173f0 con 0x7f0320105920 2024-04-16T17:04:35.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:35.499+0000 7f0325e00700 1 -- 172.21.15.45:0/2081206795 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03201a80b0 con 0x7f0320105920 2024-04-16T17:04:35.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:35.500+0000 7f0325e00700 1 -- 172.21.15.45:0/2081206795 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f0320063200 con 0x7f0320105920 2024-04-16T17:04:35.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:35.502+0000 7f0324a00700 1 -- 172.21.15.45:0/2081206795 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f030c02a070 con 0x7f0320105920 2024-04-16T17:04:35.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:35.503+0000 7f0324a00700 1 --2- 172.21.15.45:0/2081206795 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f03040a3d50 0x7f03040a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:35.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:35.503+0000 7f0324a00700 1 -- 172.21.15.45:0/2081206795 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f030c0d1b90 con 0x7f0320105920 2024-04-16T17:04:35.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:35.503+0000 7f031ea00700 1 --2- 172.21.15.45:0/2081206795 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f03040a3d50 0x7f03040a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:35.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:35.504+0000 7f0324a00700 1 --2- 172.21.15.45:0/2081206795 >> [v2:172.21.15.104:6808/1396278127,v1:172.21.15.104:6809/1396278127] conn(0x7f03040aed80 0x7f03040b1220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:35.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:35.504+0000 7f0324a00700 1 -- 172.21.15.45:0/2081206795 --> [v2:172.21.15.104:6808/1396278127,v1:172.21.15.104:6809/1396278127] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f03040b18f0 con 0x7f03040aed80 2024-04-16T17:04:35.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:35.504+0000 7f0324a00700 1 -- 172.21.15.45:0/2081206795 <== mon.2 v2:172.21.15.104:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7f030c0d1f40 con 0x7f0320105920 2024-04-16T17:04:35.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:35.504+0000 7f031f400700 1 --2- 172.21.15.45:0/2081206795 >> [v2:172.21.15.104:6808/1396278127,v1:172.21.15.104:6809/1396278127] conn(0x7f03040aed80 0x7f03040b1220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:35.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:35.504+0000 7f031ea00700 1 --2- 172.21.15.45:0/2081206795 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f03040a3d50 0x7f03040a6210 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f0310011590 tx=0x7f0310009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:04:35.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:35.505+0000 7f031f400700 1 --2- 172.21.15.45:0/2081206795 >> [v2:172.21.15.104:6808/1396278127,v1:172.21.15.104:6809/1396278127] conn(0x7f03040aed80 0x7f03040b1220 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:04:35.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:35.506+0000 7f0324a00700 1 -- 172.21.15.45:0/2081206795 <== osd.4 v2:172.21.15.104:6808/1396278127 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f03040b18f0 con 0x7f03040aed80 2024-04-16T17:04:35.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:35.524+0000 7f0325e00700 1 -- 172.21.15.45:0/2081206795 --> [v2:172.21.15.104:6808/1396278127,v1:172.21.15.104:6809/1396278127] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f0320063600 con 0x7f03040aed80 2024-04-16T17:04:35.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:35.524+0000 7f0324a00700 1 -- 172.21.15.45:0/2081206795 <== osd.4 v2:172.21.15.104:6808/1396278127 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f0320063600 con 0x7f03040aed80 2024-04-16T17:04:35.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:35.525+0000 7f0325e00700 1 -- 172.21.15.45:0/2081206795 >> [v2:172.21.15.104:6808/1396278127,v1:172.21.15.104:6809/1396278127] conn(0x7f03040aed80 msgr2=0x7f03040b1220 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:35.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:35.525+0000 7f0325e00700 1 --2- 172.21.15.45:0/2081206795 >> [v2:172.21.15.104:6808/1396278127,v1:172.21.15.104:6809/1396278127] conn(0x7f03040aed80 0x7f03040b1220 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:35.527 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:35.525+0000 7f0325e00700 1 -- 172.21.15.45:0/2081206795 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f03040a3d50 msgr2=0x7f03040a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:35.527 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:35.525+0000 7f0325e00700 1 --2- 172.21.15.45:0/2081206795 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f03040a3d50 0x7f03040a6210 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f0310011590 tx=0x7f0310009260 comp rx=0 tx=0).stop 2024-04-16T17:04:35.527 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:35.525+0000 7f0325e00700 1 -- 172.21.15.45:0/2081206795 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0320105920 msgr2=0x7f032019bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:35.527 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:35.525+0000 7f0325e00700 1 --2- 172.21.15.45:0/2081206795 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0320105920 0x7f032019bac0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f030c0059d0 tx=0x7f030c005590 comp rx=0 tx=0).stop 2024-04-16T17:04:35.527 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:35.525+0000 7f031f400700 1 -- 172.21.15.45:0/2081206795 reap_dead start 2024-04-16T17:04:35.527 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:35.525+0000 7f0325e00700 1 -- 172.21.15.45:0/2081206795 shutdown_connections 2024-04-16T17:04:35.527 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:35.525+0000 7f0325e00700 1 -- 172.21.15.45:0/2081206795 >> 172.21.15.45:0/2081206795 conn(0x7f0320100800 msgr2=0x7f03201013e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:04:35.527 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:35.525+0000 7f0325e00700 1 -- 172.21.15.45:0/2081206795 shutdown_connections 2024-04-16T17:04:35.527 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:35.526+0000 7f0325e00700 1 -- 172.21.15.45:0/2081206795 wait complete. 2024-04-16T17:04:35.876 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:04:35 smithi045 ceph-mon[31676]: pgmap v359: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s; 61 B/s, 1 keys/s, 0 objects/s recovering 2024-04-16T17:04:35.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:04:35 smithi104 ceph-mon[36457]: pgmap v359: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s; 61 B/s, 1 keys/s, 0 objects/s recovering 2024-04-16T17:04:36.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:04:35 smithi117 ceph-mon[36730]: pgmap v359: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s; 61 B/s, 1 keys/s, 0 objects/s recovering 2024-04-16T17:04:36.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:36.354+0000 7f015f600700 1 -- 172.21.15.45:0/4153491126 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f015810e460 msgr2=0x7f0158116030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:36.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:36.354+0000 7f015f600700 1 --2- 172.21.15.45:0/4153491126 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f015810e460 0x7f0158116030 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f0150009920 tx=0x7f0150009c30 comp rx=0 tx=0).stop 2024-04-16T17:04:36.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:36.354+0000 7f015f600700 1 -- 172.21.15.45:0/4153491126 shutdown_connections 2024-04-16T17:04:36.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:36.354+0000 7f015f600700 1 --2- 172.21.15.45:0/4153491126 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f015810e460 0x7f0158116030 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:36.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:36.354+0000 7f015f600700 1 --2- 172.21.15.45:0/4153491126 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0158109380 0x7f0158109760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:36.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:36.354+0000 7f015f600700 1 --2- 172.21.15.45:0/4153491126 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0158109d30 0x7f015810dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:36.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:36.354+0000 7f015f600700 1 -- 172.21.15.45:0/4153491126 >> 172.21.15.45:0/4153491126 conn(0x7f01580762c0 msgr2=0x7f0158107910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:04:36.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:36.354+0000 7f015f600700 1 -- 172.21.15.45:0/4153491126 shutdown_connections 2024-04-16T17:04:36.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:36.354+0000 7f015f600700 1 -- 172.21.15.45:0/4153491126 wait complete. 2024-04-16T17:04:36.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:36.356+0000 7f015f600700 1 Processor -- start 2024-04-16T17:04:36.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:36.356+0000 7f015f600700 1 -- start start 2024-04-16T17:04:36.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:36.357+0000 7f015f600700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0158109380 0x7f01581a5650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:36.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:36.357+0000 7f015f600700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0158109d30 0x7f01581a5b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:36.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:36.357+0000 7f015f600700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f015810e460 0x7f015819f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:36.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:36.357+0000 7f015f600700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f01581194a0 con 0x7f015810e460 2024-04-16T17:04:36.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:36.357+0000 7f015f600700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f0158119320 con 0x7f0158109380 2024-04-16T17:04:36.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:36.357+0000 7f015f600700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f0158119620 con 0x7f0158109d30 2024-04-16T17:04:36.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:36.357+0000 7f015cc00700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0158109380 0x7f01581a5650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:36.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:36.357+0000 7f0157e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0158109d30 0x7f01581a5b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:36.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:36.357+0000 7f015cc00700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0158109380 0x7f01581a5650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:54720/0 (socket says 172.21.15.45:54720) 2024-04-16T17:04:36.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:36.357+0000 7f0157e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0158109d30 0x7f01581a5b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:53914/0 (socket says 172.21.15.45:53914) 2024-04-16T17:04:36.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:36.357+0000 7f015cc00700 1 -- 172.21.15.45:0/2590303571 learned_addr learned my addr 172.21.15.45:0/2590303571 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:04:36.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:36.357+0000 7f015d600700 1 --2- 172.21.15.45:0/2590303571 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f015810e460 0x7f015819f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:36.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:36.357+0000 7f0157e00700 1 -- 172.21.15.45:0/2590303571 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0158109380 msgr2=0x7f01581a5650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:36.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:36.357+0000 7f0157e00700 1 --2- 172.21.15.45:0/2590303571 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0158109380 0x7f01581a5650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:36.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:36.357+0000 7f0157e00700 1 -- 172.21.15.45:0/2590303571 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f015810e460 msgr2=0x7f015819f820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:36.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:36.357+0000 7f0157e00700 1 --2- 172.21.15.45:0/2590303571 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f015810e460 0x7f015819f820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:36.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:36.357+0000 7f0157e00700 1 -- 172.21.15.45:0/2590303571 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01581a0000 con 0x7f0158109d30 2024-04-16T17:04:36.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:36.358+0000 7f0157e00700 1 --2- 172.21.15.45:0/2590303571 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0158109d30 0x7f01581a5b90 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f014400e9c0 tx=0x7f014400ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:04:36.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:36.358+0000 7f0155600700 1 -- 172.21.15.45:0/2590303571 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f014400cbd0 con 0x7f0158109d30 2024-04-16T17:04:36.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:36.358+0000 7f0155600700 1 -- 172.21.15.45:0/2590303571 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f014400cd70 con 0x7f0158109d30 2024-04-16T17:04:36.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:36.358+0000 7f0155600700 1 -- 172.21.15.45:0/2590303571 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0144018680 con 0x7f0158109d30 2024-04-16T17:04:36.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:36.358+0000 7f015f600700 1 -- 172.21.15.45:0/2590303571 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01581a02f0 con 0x7f0158109d30 2024-04-16T17:04:36.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:36.358+0000 7f015f600700 1 -- 172.21.15.45:0/2590303571 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01581ac4b0 con 0x7f0158109d30 2024-04-16T17:04:36.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:36.359+0000 7f015f600700 1 -- 172.21.15.45:0/2590303571 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f0158063200 con 0x7f0158109d30 2024-04-16T17:04:36.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:36.361+0000 7f0155600700 1 -- 172.21.15.45:0/2590303571 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f0144010430 con 0x7f0158109d30 2024-04-16T17:04:36.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:36.362+0000 7f0155600700 1 --2- 172.21.15.45:0/2590303571 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f01400a3cd0 0x7f01400a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:36.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:36.363+0000 7f0155600700 1 -- 172.21.15.45:0/2590303571 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f01440d0c70 con 0x7f0158109d30 2024-04-16T17:04:36.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:36.363+0000 7f015cc00700 1 --2- 172.21.15.45:0/2590303571 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f01400a3cd0 0x7f01400a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:36.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:36.363+0000 7f0155600700 1 --2- 172.21.15.45:0/2590303571 >> [v2:172.21.15.45:6810/2823712137,v1:172.21.15.45:6811/2823712137] conn(0x7f01400aee30 0x7f01400b12d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:36.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:36.363+0000 7f0155600700 1 -- 172.21.15.45:0/2590303571 --> [v2:172.21.15.45:6810/2823712137,v1:172.21.15.45:6811/2823712137] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0144015e80 con 0x7f01400aee30 2024-04-16T17:04:36.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:36.363+0000 7f0155600700 1 -- 172.21.15.45:0/2590303571 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7f01440d1020 con 0x7f0158109d30 2024-04-16T17:04:36.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:36.363+0000 7f015d600700 1 --2- 172.21.15.45:0/2590303571 >> [v2:172.21.15.45:6810/2823712137,v1:172.21.15.45:6811/2823712137] conn(0x7f01400aee30 0x7f01400b12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:36.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:36.364+0000 7f015cc00700 1 --2- 172.21.15.45:0/2590303571 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f01400a3cd0 0x7f01400a6190 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f014c009fd0 tx=0x7f014c00b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:04:36.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:36.364+0000 7f015d600700 1 --2- 172.21.15.45:0/2590303571 >> [v2:172.21.15.45:6810/2823712137,v1:172.21.15.45:6811/2823712137] conn(0x7f01400aee30 0x7f01400b12d0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:04:36.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:36.365+0000 7f0155600700 1 -- 172.21.15.45:0/2590303571 <== osd.3 v2:172.21.15.45:6810/2823712137 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f0144015e80 con 0x7f01400aee30 2024-04-16T17:04:36.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:36.384+0000 7f015f600700 1 -- 172.21.15.45:0/2590303571 --> [v2:172.21.15.45:6810/2823712137,v1:172.21.15.45:6811/2823712137] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f0158063680 con 0x7f01400aee30 2024-04-16T17:04:36.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:36.384+0000 7f0155600700 1 -- 172.21.15.45:0/2590303571 <== osd.3 v2:172.21.15.45:6810/2823712137 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f0158063680 con 0x7f01400aee30 2024-04-16T17:04:36.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:36.385+0000 7f015f600700 1 -- 172.21.15.45:0/2590303571 >> [v2:172.21.15.45:6810/2823712137,v1:172.21.15.45:6811/2823712137] conn(0x7f01400aee30 msgr2=0x7f01400b12d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:36.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:36.385+0000 7f015f600700 1 --2- 172.21.15.45:0/2590303571 >> [v2:172.21.15.45:6810/2823712137,v1:172.21.15.45:6811/2823712137] conn(0x7f01400aee30 0x7f01400b12d0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:36.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:36.385+0000 7f015f600700 1 -- 172.21.15.45:0/2590303571 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f01400a3cd0 msgr2=0x7f01400a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:36.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:36.385+0000 7f015f600700 1 --2- 172.21.15.45:0/2590303571 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f01400a3cd0 0x7f01400a6190 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f014c009fd0 tx=0x7f014c00b040 comp rx=0 tx=0).stop 2024-04-16T17:04:36.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:36.385+0000 7f015f600700 1 -- 172.21.15.45:0/2590303571 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0158109d30 msgr2=0x7f01581a5b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:36.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:36.385+0000 7f015f600700 1 --2- 172.21.15.45:0/2590303571 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0158109d30 0x7f01581a5b90 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f014400e9c0 tx=0x7f014400ecd0 comp rx=0 tx=0).stop 2024-04-16T17:04:36.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:36.385+0000 7f015d600700 1 -- 172.21.15.45:0/2590303571 reap_dead start 2024-04-16T17:04:36.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:36.385+0000 7f015f600700 1 -- 172.21.15.45:0/2590303571 shutdown_connections 2024-04-16T17:04:36.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:36.385+0000 7f015f600700 1 -- 172.21.15.45:0/2590303571 >> 172.21.15.45:0/2590303571 conn(0x7f01580762c0 msgr2=0x7f0158076920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:04:36.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:36.385+0000 7f015f600700 1 -- 172.21.15.45:0/2590303571 shutdown_connections 2024-04-16T17:04:36.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:36.386+0000 7f015f600700 1 -- 172.21.15.45:0/2590303571 wait complete. 2024-04-16T17:04:37.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:37.112+0000 7f4823800700 1 -- 172.21.15.45:0/2386738066 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f481c10f550 msgr2=0x7f481c111940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:37.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:37.112+0000 7f4823800700 1 --2- 172.21.15.45:0/2386738066 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f481c10f550 0x7f481c111940 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f481400b0a0 tx=0x7f481400b3b0 comp rx=0 tx=0).stop 2024-04-16T17:04:37.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:37.112+0000 7f4823800700 1 -- 172.21.15.45:0/2386738066 shutdown_connections 2024-04-16T17:04:37.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:37.112+0000 7f4823800700 1 --2- 172.21.15.45:0/2386738066 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f481c074d20 0x7f481c075100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:37.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:37.112+0000 7f4823800700 1 --2- 172.21.15.45:0/2386738066 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f481c10f550 0x7f481c111940 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:37.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:37.112+0000 7f4823800700 1 --2- 172.21.15.45:0/2386738066 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f481c075640 0x7f481c10f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:37.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:37.112+0000 7f4823800700 1 -- 172.21.15.45:0/2386738066 >> 172.21.15.45:0/2386738066 conn(0x7f481c079c50 msgr2=0x7f481c07a060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:04:37.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:37.112+0000 7f4823800700 1 -- 172.21.15.45:0/2386738066 shutdown_connections 2024-04-16T17:04:37.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:37.112+0000 7f4823800700 1 -- 172.21.15.45:0/2386738066 wait complete. 2024-04-16T17:04:37.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:37.113+0000 7f4823800700 1 Processor -- start 2024-04-16T17:04:37.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:37.113+0000 7f4823800700 1 -- start start 2024-04-16T17:04:37.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:37.114+0000 7f4823800700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f481c074d20 0x7f481c19b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:37.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:37.114+0000 7f4823800700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f481c075640 0x7f481c19b930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:37.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:37.114+0000 7f4823800700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f481c10f550 0x7f481c19fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:37.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:37.114+0000 7f4823800700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f481c115080 con 0x7f481c075640 2024-04-16T17:04:37.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:37.114+0000 7f4823800700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f481c114f00 con 0x7f481c074d20 2024-04-16T17:04:37.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:37.114+0000 7f4823800700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f481c115200 con 0x7f481c10f550 2024-04-16T17:04:37.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:37.114+0000 7f4813e00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f481c075640 0x7f481c19b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:37.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:37.114+0000 7f4820e00700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f481c074d20 0x7f481c19b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:37.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:37.114+0000 7f4813e00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f481c075640 0x7f481c19b930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52108/0 (socket says 172.21.15.45:52108) 2024-04-16T17:04:37.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:37.114+0000 7f4813e00700 1 -- 172.21.15.45:0/1715897983 learned_addr learned my addr 172.21.15.45:0/1715897983 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:04:37.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:37.114+0000 7f4821800700 1 --2- 172.21.15.45:0/1715897983 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f481c10f550 0x7f481c19fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:37.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:37.115+0000 7f4820e00700 1 -- 172.21.15.45:0/1715897983 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f481c10f550 msgr2=0x7f481c19fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:37.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:37.115+0000 7f4820e00700 1 --2- 172.21.15.45:0/1715897983 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f481c10f550 0x7f481c19fcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:37.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:37.115+0000 7f4820e00700 1 -- 172.21.15.45:0/1715897983 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f481c075640 msgr2=0x7f481c19b930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:37.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:37.115+0000 7f4820e00700 1 --2- 172.21.15.45:0/1715897983 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f481c075640 0x7f481c19b930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:37.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:37.115+0000 7f4820e00700 1 -- 172.21.15.45:0/1715897983 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f481c1a0400 con 0x7f481c074d20 2024-04-16T17:04:37.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:37.115+0000 7f4813e00700 1 --2- 172.21.15.45:0/1715897983 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f481c075640 0x7f481c19b930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:04:37.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:37.115+0000 7f4820e00700 1 --2- 172.21.15.45:0/1715897983 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f481c074d20 0x7f481c19b3f0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f480c00e9f0 tx=0x7f480c00ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:04:37.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:37.115+0000 7f481a000700 1 -- 172.21.15.45:0/1715897983 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f480c00c9c0 con 0x7f481c074d20 2024-04-16T17:04:37.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:37.115+0000 7f481a000700 1 -- 172.21.15.45:0/1715897983 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f480c00cb60 con 0x7f481c074d20 2024-04-16T17:04:37.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:37.116+0000 7f481a000700 1 -- 172.21.15.45:0/1715897983 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f480c018680 con 0x7f481c074d20 2024-04-16T17:04:37.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:37.116+0000 7f4823800700 1 -- 172.21.15.45:0/1715897983 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f481c1a06f0 con 0x7f481c074d20 2024-04-16T17:04:37.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:37.116+0000 7f4823800700 1 -- 172.21.15.45:0/1715897983 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f481c1a7f70 con 0x7f481c074d20 2024-04-16T17:04:37.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:37.117+0000 7f4823800700 1 -- 172.21.15.45:0/1715897983 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f481c100660 con 0x7f481c074d20 2024-04-16T17:04:37.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:37.119+0000 7f481a000700 1 -- 172.21.15.45:0/1715897983 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f480c010430 con 0x7f481c074d20 2024-04-16T17:04:37.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:37.120+0000 7f481a000700 1 --2- 172.21.15.45:0/1715897983 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f48000a3d50 0x7f48000a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:37.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:37.120+0000 7f481a000700 1 -- 172.21.15.45:0/1715897983 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f480c0d0e30 con 0x7f481c074d20 2024-04-16T17:04:37.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:37.121+0000 7f4813e00700 1 --2- 172.21.15.45:0/1715897983 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f48000a3d50 0x7f48000a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:37.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:37.121+0000 7f481a000700 1 --2- 172.21.15.45:0/1715897983 >> [v2:172.21.15.45:6818/3254229176,v1:172.21.15.45:6819/3254229176] conn(0x7f48000aeeb0 0x7f48000b1350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:37.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:37.121+0000 7f481a000700 1 -- 172.21.15.45:0/1715897983 --> [v2:172.21.15.45:6818/3254229176,v1:172.21.15.45:6819/3254229176] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f480c016ec0 con 0x7f48000aeeb0 2024-04-16T17:04:37.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:37.121+0000 7f481a000700 1 -- 172.21.15.45:0/1715897983 <== mon.2 v2:172.21.15.104:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7f480c0d11e0 con 0x7f481c074d20 2024-04-16T17:04:37.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:37.121+0000 7f4821800700 1 --2- 172.21.15.45:0/1715897983 >> [v2:172.21.15.45:6818/3254229176,v1:172.21.15.45:6819/3254229176] conn(0x7f48000aeeb0 0x7f48000b1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:37.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:37.121+0000 7f4813e00700 1 --2- 172.21.15.45:0/1715897983 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f48000a3d50 0x7f48000a6210 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f481c1110c0 tx=0x7f4804009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:04:37.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:37.122+0000 7f4821800700 1 --2- 172.21.15.45:0/1715897983 >> [v2:172.21.15.45:6818/3254229176,v1:172.21.15.45:6819/3254229176] conn(0x7f48000aeeb0 0x7f48000b1350 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:04:37.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:37.123+0000 7f481a000700 1 -- 172.21.15.45:0/1715897983 <== osd.6 v2:172.21.15.45:6818/3254229176 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f480c016ec0 con 0x7f48000aeeb0 2024-04-16T17:04:37.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:37.142+0000 7f4823800700 1 -- 172.21.15.45:0/1715897983 --> [v2:172.21.15.45:6818/3254229176,v1:172.21.15.45:6819/3254229176] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f481c076400 con 0x7f48000aeeb0 2024-04-16T17:04:37.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:37.142+0000 7f481a000700 1 -- 172.21.15.45:0/1715897983 <== osd.6 v2:172.21.15.45:6818/3254229176 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f481c076400 con 0x7f48000aeeb0 2024-04-16T17:04:37.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:37.143+0000 7f4823800700 1 -- 172.21.15.45:0/1715897983 >> [v2:172.21.15.45:6818/3254229176,v1:172.21.15.45:6819/3254229176] conn(0x7f48000aeeb0 msgr2=0x7f48000b1350 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:37.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:37.143+0000 7f4823800700 1 --2- 172.21.15.45:0/1715897983 >> [v2:172.21.15.45:6818/3254229176,v1:172.21.15.45:6819/3254229176] conn(0x7f48000aeeb0 0x7f48000b1350 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:37.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:37.143+0000 7f4823800700 1 -- 172.21.15.45:0/1715897983 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f48000a3d50 msgr2=0x7f48000a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:37.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:37.143+0000 7f4823800700 1 --2- 172.21.15.45:0/1715897983 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f48000a3d50 0x7f48000a6210 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f481c1110c0 tx=0x7f4804009260 comp rx=0 tx=0).stop 2024-04-16T17:04:37.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:37.143+0000 7f4823800700 1 -- 172.21.15.45:0/1715897983 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f481c074d20 msgr2=0x7f481c19b3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:37.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:37.143+0000 7f4823800700 1 --2- 172.21.15.45:0/1715897983 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f481c074d20 0x7f481c19b3f0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f480c00e9f0 tx=0x7f480c00ed00 comp rx=0 tx=0).stop 2024-04-16T17:04:37.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:37.143+0000 7f4821800700 1 -- 172.21.15.45:0/1715897983 reap_dead start 2024-04-16T17:04:37.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:37.143+0000 7f4823800700 1 -- 172.21.15.45:0/1715897983 shutdown_connections 2024-04-16T17:04:37.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:37.143+0000 7f4823800700 1 -- 172.21.15.45:0/1715897983 >> 172.21.15.45:0/1715897983 conn(0x7f481c079c50 msgr2=0x7f481c0ffc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:04:37.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:37.144+0000 7f4823800700 1 -- 172.21.15.45:0/1715897983 shutdown_connections 2024-04-16T17:04:37.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:37.144+0000 7f4823800700 1 -- 172.21.15.45:0/1715897983 wait complete. 2024-04-16T17:04:37.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:04:37 smithi104 ceph-mon[36457]: pgmap v360: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 170 B/s wr, 7 op/s; 2 B/s, 1 keys/s, 0 objects/s recovering 2024-04-16T17:04:37.978 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:04:37 smithi045 ceph-mon[31676]: pgmap v360: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 170 B/s wr, 7 op/s; 2 B/s, 1 keys/s, 0 objects/s recovering 2024-04-16T17:04:38.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:04:37 smithi117 ceph-mon[36730]: pgmap v360: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 170 B/s wr, 7 op/s; 2 B/s, 1 keys/s, 0 objects/s recovering 2024-04-16T17:04:38.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:37.478+0000 7f0fe3600700 1 -- 172.21.15.45:0/181604186 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0fdc109380 msgr2=0x7f0fdc109760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:38.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:37.478+0000 7f0fe3600700 1 --2- 172.21.15.45:0/181604186 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0fdc109380 0x7f0fdc109760 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f0fcc0099c0 tx=0x7f0fcc009cd0 comp rx=0 tx=0).stop 2024-04-16T17:04:38.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:37.478+0000 7f0fe3600700 1 -- 172.21.15.45:0/181604186 shutdown_connections 2024-04-16T17:04:38.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:37.478+0000 7f0fe3600700 1 --2- 172.21.15.45:0/181604186 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0fdc10e460 0x7f0fdc116030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:38.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:37.478+0000 7f0fe3600700 1 --2- 172.21.15.45:0/181604186 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0fdc109380 0x7f0fdc109760 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:38.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:37.478+0000 7f0fe3600700 1 --2- 172.21.15.45:0/181604186 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0fdc109d30 0x7f0fdc10dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:38.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:37.478+0000 7f0fe3600700 1 -- 172.21.15.45:0/181604186 >> 172.21.15.45:0/181604186 conn(0x7f0fdc0762c0 msgr2=0x7f0fdc107910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:04:38.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:37.478+0000 7f0fe3600700 1 -- 172.21.15.45:0/181604186 shutdown_connections 2024-04-16T17:04:38.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:37.478+0000 7f0fe3600700 1 -- 172.21.15.45:0/181604186 wait complete. 2024-04-16T17:04:38.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:37.479+0000 7f0fe3600700 1 Processor -- start 2024-04-16T17:04:38.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:37.480+0000 7f0fe3600700 1 -- start start 2024-04-16T17:04:38.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:37.480+0000 7f0fe3600700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0fdc109380 0x7f0fdc1a5610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:38.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:37.480+0000 7f0fe3600700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0fdc109d30 0x7f0fdc1a5b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:38.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:37.480+0000 7f0fe3600700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0fdc10e460 0x7f0fdc19f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:38.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:37.480+0000 7f0fe3600700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f0fdc1194c0 con 0x7f0fdc10e460 2024-04-16T17:04:38.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:37.480+0000 7f0fe3600700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f0fdc119340 con 0x7f0fdc109380 2024-04-16T17:04:38.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:37.480+0000 7f0fe3600700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f0fdc119640 con 0x7f0fdc109d30 2024-04-16T17:04:38.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:37.481+0000 7f0fe0c00700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0fdc109380 0x7f0fdc1a5610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:38.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:37.480+0000 7f0fd3e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0fdc109d30 0x7f0fdc1a5b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:38.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:37.481+0000 7f0fe1600700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0fdc10e460 0x7f0fdc19f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:38.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:37.481+0000 7f0fe0c00700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0fdc109380 0x7f0fdc1a5610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:54770/0 (socket says 172.21.15.45:54770) 2024-04-16T17:04:38.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:37.481+0000 7f0fe0c00700 1 -- 172.21.15.45:0/1924193100 learned_addr learned my addr 172.21.15.45:0/1924193100 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:04:38.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:37.481+0000 7f0fd3e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0fdc109d30 0x7f0fdc1a5b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:53938/0 (socket says 172.21.15.45:53938) 2024-04-16T17:04:38.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:37.481+0000 7f0fe0c00700 1 -- 172.21.15.45:0/1924193100 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0fdc109d30 msgr2=0x7f0fdc1a5b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:38.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:37.481+0000 7f0fe0c00700 1 --2- 172.21.15.45:0/1924193100 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0fdc109d30 0x7f0fdc1a5b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:38.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:37.481+0000 7f0fe0c00700 1 -- 172.21.15.45:0/1924193100 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0fdc10e460 msgr2=0x7f0fdc19f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:38.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:37.481+0000 7f0fe0c00700 1 --2- 172.21.15.45:0/1924193100 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0fdc10e460 0x7f0fdc19f7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:38.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:37.481+0000 7f0fe0c00700 1 -- 172.21.15.45:0/1924193100 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fdc19ffc0 con 0x7f0fdc109380 2024-04-16T17:04:38.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:37.481+0000 7f0fd3e00700 1 --2- 172.21.15.45:0/1924193100 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0fdc109d30 0x7f0fdc1a5b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:04:38.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:37.481+0000 7f0fe0c00700 1 --2- 172.21.15.45:0/1924193100 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0fdc109380 0x7f0fdc1a5610 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f0fcc005910 tx=0x7f0fcc00f690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:04:38.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:37.482+0000 7f0fda000700 1 -- 172.21.15.45:0/1924193100 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fcc027070 con 0x7f0fdc109380 2024-04-16T17:04:38.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:37.482+0000 7f0fda000700 1 -- 172.21.15.45:0/1924193100 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0fcc003680 con 0x7f0fdc109380 2024-04-16T17:04:38.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:37.482+0000 7f0fda000700 1 -- 172.21.15.45:0/1924193100 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fcc0173f0 con 0x7f0fdc109380 2024-04-16T17:04:38.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:37.482+0000 7f0fe3600700 1 -- 172.21.15.45:0/1924193100 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0fdc1a0250 con 0x7f0fdc109380 2024-04-16T17:04:38.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:37.482+0000 7f0fe3600700 1 -- 172.21.15.45:0/1924193100 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0fdc1ac470 con 0x7f0fdc109380 2024-04-16T17:04:38.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:37.483+0000 7f0fe3600700 1 -- 172.21.15.45:0/1924193100 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f0fdc0631c0 con 0x7f0fdc109380 2024-04-16T17:04:38.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:37.485+0000 7f0fda000700 1 -- 172.21.15.45:0/1924193100 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f0fcc022590 con 0x7f0fdc109380 2024-04-16T17:04:38.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:37.487+0000 7f0fda000700 1 --2- 172.21.15.45:0/1924193100 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f0fc00a3d50 0x7f0fc00a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:38.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:37.487+0000 7f0fda000700 1 -- 172.21.15.45:0/1924193100 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f0fcc0d18c0 con 0x7f0fdc109380 2024-04-16T17:04:38.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:37.487+0000 7f0fd3e00700 1 --2- 172.21.15.45:0/1924193100 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f0fc00a3d50 0x7f0fc00a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:38.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:37.487+0000 7f0fda000700 1 --2- 172.21.15.45:0/1924193100 >> [v2:172.21.15.104:6816/4151077186,v1:172.21.15.104:6817/4151077186] conn(0x7f0fc00aeeb0 0x7f0fc00b1350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:38.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:37.488+0000 7f0fda000700 1 -- 172.21.15.45:0/1924193100 --> [v2:172.21.15.104:6816/4151077186,v1:172.21.15.104:6817/4151077186] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0fcc00b0b0 con 0x7f0fc00aeeb0 2024-04-16T17:04:38.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:37.488+0000 7f0fda000700 1 -- 172.21.15.45:0/1924193100 <== mon.2 v2:172.21.15.104:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7f0fcc0d1c70 con 0x7f0fdc109380 2024-04-16T17:04:38.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:37.488+0000 7f0fe1600700 1 --2- 172.21.15.45:0/1924193100 >> [v2:172.21.15.104:6816/4151077186,v1:172.21.15.104:6817/4151077186] conn(0x7f0fc00aeeb0 0x7f0fc00b1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:38.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:37.488+0000 7f0fd3e00700 1 --2- 172.21.15.45:0/1924193100 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f0fc00a3d50 0x7f0fc00a6210 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f0fdc108990 tx=0x7f0fc4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:04:38.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:37.489+0000 7f0fe1600700 1 --2- 172.21.15.45:0/1924193100 >> [v2:172.21.15.104:6816/4151077186,v1:172.21.15.104:6817/4151077186] conn(0x7f0fc00aeeb0 0x7f0fc00b1350 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:04:38.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:37.490+0000 7f0fda000700 1 -- 172.21.15.45:0/1924193100 <== osd.7 v2:172.21.15.104:6816/4151077186 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f0fcc00b0b0 con 0x7f0fc00aeeb0 2024-04-16T17:04:38.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:37.509+0000 7f0fe3600700 1 -- 172.21.15.45:0/1924193100 --> [v2:172.21.15.104:6816/4151077186,v1:172.21.15.104:6817/4151077186] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f0fdc063640 con 0x7f0fc00aeeb0 2024-04-16T17:04:38.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:37.510+0000 7f0fda000700 1 -- 172.21.15.45:0/1924193100 <== osd.7 v2:172.21.15.104:6816/4151077186 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f0fdc063640 con 0x7f0fc00aeeb0 2024-04-16T17:04:38.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:37.511+0000 7f0fe3600700 1 -- 172.21.15.45:0/1924193100 >> [v2:172.21.15.104:6816/4151077186,v1:172.21.15.104:6817/4151077186] conn(0x7f0fc00aeeb0 msgr2=0x7f0fc00b1350 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:38.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:37.511+0000 7f0fe3600700 1 --2- 172.21.15.45:0/1924193100 >> [v2:172.21.15.104:6816/4151077186,v1:172.21.15.104:6817/4151077186] conn(0x7f0fc00aeeb0 0x7f0fc00b1350 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:38.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:37.511+0000 7f0fe3600700 1 -- 172.21.15.45:0/1924193100 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f0fc00a3d50 msgr2=0x7f0fc00a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:38.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:37.511+0000 7f0fe3600700 1 --2- 172.21.15.45:0/1924193100 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f0fc00a3d50 0x7f0fc00a6210 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f0fdc108990 tx=0x7f0fc4009260 comp rx=0 tx=0).stop 2024-04-16T17:04:38.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:37.511+0000 7f0fe3600700 1 -- 172.21.15.45:0/1924193100 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0fdc109380 msgr2=0x7f0fdc1a5610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:38.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:37.511+0000 7f0fe3600700 1 --2- 172.21.15.45:0/1924193100 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0fdc109380 0x7f0fdc1a5610 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f0fcc005910 tx=0x7f0fcc00f690 comp rx=0 tx=0).stop 2024-04-16T17:04:38.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:37.511+0000 7f0fe1600700 1 -- 172.21.15.45:0/1924193100 reap_dead start 2024-04-16T17:04:38.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:37.511+0000 7f0fe3600700 1 -- 172.21.15.45:0/1924193100 shutdown_connections 2024-04-16T17:04:38.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:37.511+0000 7f0fe3600700 1 -- 172.21.15.45:0/1924193100 >> 172.21.15.45:0/1924193100 conn(0x7f0fdc0762c0 msgr2=0x7f0fdc076920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:04:38.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:37.511+0000 7f0fe3600700 1 -- 172.21.15.45:0/1924193100 shutdown_connections 2024-04-16T17:04:38.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:37.511+0000 7f0fe3600700 1 -- 172.21.15.45:0/1924193100 wait complete. 2024-04-16T17:04:38.641 INFO:teuthology.orchestra.run.smithi045.stdout:347892351002 2024-04-16T17:04:38.641 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45 shell --fsid b1fecd38-fc11-11ee-bc90-c7b262605968 -- ceph osd last-stat-seq osd.11 2024-04-16T17:04:39.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:04:39 smithi104 ceph-mon[36457]: pgmap v361: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-04-16T17:04:39.978 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:04:39 smithi045 ceph-mon[31676]: pgmap v361: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-04-16T17:04:40.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:04:39 smithi117 ceph-mon[36730]: pgmap v361: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-04-16T17:04:40.093 INFO:teuthology.orchestra.run.smithi045.stdout:34359738458 2024-04-16T17:04:40.093 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45 shell --fsid b1fecd38-fc11-11ee-bc90-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-04-16T17:04:40.194 INFO:teuthology.orchestra.run.smithi045.stdout:111669149770 2024-04-16T17:04:40.195 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45 shell --fsid b1fecd38-fc11-11ee-bc90-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-04-16T17:04:40.693 INFO:teuthology.orchestra.run.smithi045.stdout:146028888132 2024-04-16T17:04:40.694 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45 shell --fsid b1fecd38-fc11-11ee-bc90-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-04-16T17:04:41.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:41.264+0000 7ff3ede00700 1 -- 172.21.15.45:0/4137077709 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff3e8074d20 msgr2=0x7ff3e8075100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:41.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:41.264+0000 7ff3ede00700 1 --2- 172.21.15.45:0/4137077709 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff3e8074d20 0x7ff3e8075100 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7ff3d8009990 tx=0x7ff3d8009ca0 comp rx=0 tx=0).stop 2024-04-16T17:04:41.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:41.264+0000 7ff3ede00700 1 -- 172.21.15.45:0/4137077709 shutdown_connections 2024-04-16T17:04:41.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:41.264+0000 7ff3ede00700 1 --2- 172.21.15.45:0/4137077709 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff3e810f4f0 0x7ff3e81118e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:41.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:41.264+0000 7ff3ede00700 1 --2- 172.21.15.45:0/4137077709 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff3e8074d20 0x7ff3e8075100 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:41.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:41.264+0000 7ff3ede00700 1 --2- 172.21.15.45:0/4137077709 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff3e8075640 0x7ff3e810efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:41.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:41.264+0000 7ff3ede00700 1 -- 172.21.15.45:0/4137077709 >> 172.21.15.45:0/4137077709 conn(0x7ff3e8079c50 msgr2=0x7ff3e807a060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:04:41.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:41.264+0000 7ff3ede00700 1 -- 172.21.15.45:0/4137077709 shutdown_connections 2024-04-16T17:04:41.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:41.264+0000 7ff3ede00700 1 -- 172.21.15.45:0/4137077709 wait complete. 2024-04-16T17:04:41.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:41.265+0000 7ff3ede00700 1 Processor -- start 2024-04-16T17:04:41.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:41.265+0000 7ff3ede00700 1 -- start start 2024-04-16T17:04:41.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:41.266+0000 7ff3ede00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff3e8074d20 0x7ff3e819b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:41.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:41.266+0000 7ff3ede00700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff3e8075640 0x7ff3e819b980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:41.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:41.266+0000 7ff3ede00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff3e810f4f0 0x7ff3e819fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:41.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:41.266+0000 7ff3ede00700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7ff3e8114ff0 con 0x7ff3e8074d20 2024-04-16T17:04:41.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:41.266+0000 7ff3ede00700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7ff3e8114e70 con 0x7ff3e8075640 2024-04-16T17:04:41.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:41.266+0000 7ff3ede00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7ff3e8115170 con 0x7ff3e810f4f0 2024-04-16T17:04:41.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:41.266+0000 7ff3e6000700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff3e8075640 0x7ff3e819b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:41.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:41.266+0000 7ff3e6a00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff3e8074d20 0x7ff3e819b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:41.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:41.266+0000 7ff3e6000700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff3e8075640 0x7ff3e819b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:43038/0 (socket says 172.21.15.45:43038) 2024-04-16T17:04:41.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:41.266+0000 7ff3e6000700 1 -- 172.21.15.45:0/2121754545 learned_addr learned my addr 172.21.15.45:0/2121754545 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:04:41.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:41.266+0000 7ff3e7400700 1 --2- 172.21.15.45:0/2121754545 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff3e810f4f0 0x7ff3e819fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:41.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:41.267+0000 7ff3e6000700 1 -- 172.21.15.45:0/2121754545 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff3e810f4f0 msgr2=0x7ff3e819fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:41.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:41.267+0000 7ff3e6000700 1 --2- 172.21.15.45:0/2121754545 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff3e810f4f0 0x7ff3e819fcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:41.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:41.267+0000 7ff3e6000700 1 -- 172.21.15.45:0/2121754545 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff3e8074d20 msgr2=0x7ff3e819b440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:41.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:41.267+0000 7ff3e6000700 1 --2- 172.21.15.45:0/2121754545 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff3e8074d20 0x7ff3e819b440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:41.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:41.267+0000 7ff3e6000700 1 -- 172.21.15.45:0/2121754545 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3e81a0400 con 0x7ff3e8075640 2024-04-16T17:04:41.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:41.267+0000 7ff3e6000700 1 --2- 172.21.15.45:0/2121754545 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff3e8075640 0x7ff3e819b980 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7ff3d400d980 tx=0x7ff3d400dc90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:04:41.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:41.268+0000 7ff3eca00700 1 -- 172.21.15.45:0/2121754545 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3d4017030 con 0x7ff3e8075640 2024-04-16T17:04:41.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:41.268+0000 7ff3eca00700 1 -- 172.21.15.45:0/2121754545 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff3d4004500 con 0x7ff3e8075640 2024-04-16T17:04:41.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:41.268+0000 7ff3ede00700 1 -- 172.21.15.45:0/2121754545 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3e81a06f0 con 0x7ff3e8075640 2024-04-16T17:04:41.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:41.268+0000 7ff3eca00700 1 -- 172.21.15.45:0/2121754545 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3d40039c0 con 0x7ff3e8075640 2024-04-16T17:04:41.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:41.268+0000 7ff3ede00700 1 -- 172.21.15.45:0/2121754545 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3e81a7f70 con 0x7ff3e8075640 2024-04-16T17:04:41.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:41.269+0000 7ff3ede00700 1 -- 172.21.15.45:0/2121754545 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ff3e8063240 con 0x7ff3e8075640 2024-04-16T17:04:41.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:41.271+0000 7ff3eca00700 1 -- 172.21.15.45:0/2121754545 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7ff3d401a070 con 0x7ff3e8075640 2024-04-16T17:04:41.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:41.272+0000 7ff3eca00700 1 --2- 172.21.15.45:0/2121754545 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7ff3cc0a3cd0 0x7ff3cc0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:41.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:41.272+0000 7ff3eca00700 1 -- 172.21.15.45:0/2121754545 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7ff3d40d01d0 con 0x7ff3e8075640 2024-04-16T17:04:41.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:41.273+0000 7ff3e6a00700 1 --2- 172.21.15.45:0/2121754545 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7ff3cc0a3cd0 0x7ff3cc0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:41.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:41.273+0000 7ff3eca00700 1 --2- 172.21.15.45:0/2121754545 >> [v2:172.21.15.45:6826/3106734074,v1:172.21.15.45:6827/3106734074] conn(0x7ff3cc0aee30 0x7ff3cc0b12d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:41.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:41.273+0000 7ff3eca00700 1 -- 172.21.15.45:0/2121754545 --> [v2:172.21.15.45:6826/3106734074,v1:172.21.15.45:6827/3106734074] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff3d4018e50 con 0x7ff3cc0aee30 2024-04-16T17:04:41.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:41.273+0000 7ff3eca00700 1 -- 172.21.15.45:0/2121754545 <== mon.2 v2:172.21.15.104:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7ff3d40d7050 con 0x7ff3e8075640 2024-04-16T17:04:41.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:41.273+0000 7ff3e7400700 1 --2- 172.21.15.45:0/2121754545 >> [v2:172.21.15.45:6826/3106734074,v1:172.21.15.45:6827/3106734074] conn(0x7ff3cc0aee30 0x7ff3cc0b12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:41.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:41.273+0000 7ff3e6a00700 1 --2- 172.21.15.45:0/2121754545 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7ff3cc0a3cd0 0x7ff3cc0a6190 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7ff3d80098e0 tx=0x7ff3d8005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:04:41.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:41.274+0000 7ff3e7400700 1 --2- 172.21.15.45:0/2121754545 >> [v2:172.21.15.45:6826/3106734074,v1:172.21.15.45:6827/3106734074] conn(0x7ff3cc0aee30 0x7ff3cc0b12d0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:04:41.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:41.275+0000 7ff3eca00700 1 -- 172.21.15.45:0/2121754545 <== osd.9 v2:172.21.15.45:6826/3106734074 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff3d4018e50 con 0x7ff3cc0aee30 2024-04-16T17:04:41.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:41.293+0000 7ff3ede00700 1 -- 172.21.15.45:0/2121754545 --> [v2:172.21.15.45:6826/3106734074,v1:172.21.15.45:6827/3106734074] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7ff3e80636c0 con 0x7ff3cc0aee30 2024-04-16T17:04:41.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:41.294+0000 7ff3eca00700 1 -- 172.21.15.45:0/2121754545 <== osd.9 v2:172.21.15.45:6826/3106734074 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7ff3e80636c0 con 0x7ff3cc0aee30 2024-04-16T17:04:41.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:41.294+0000 7ff3ede00700 1 -- 172.21.15.45:0/2121754545 >> [v2:172.21.15.45:6826/3106734074,v1:172.21.15.45:6827/3106734074] conn(0x7ff3cc0aee30 msgr2=0x7ff3cc0b12d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:41.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:41.294+0000 7ff3ede00700 1 --2- 172.21.15.45:0/2121754545 >> [v2:172.21.15.45:6826/3106734074,v1:172.21.15.45:6827/3106734074] conn(0x7ff3cc0aee30 0x7ff3cc0b12d0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:41.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:41.295+0000 7ff3ede00700 1 -- 172.21.15.45:0/2121754545 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7ff3cc0a3cd0 msgr2=0x7ff3cc0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:41.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:41.295+0000 7ff3ede00700 1 --2- 172.21.15.45:0/2121754545 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7ff3cc0a3cd0 0x7ff3cc0a6190 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7ff3d80098e0 tx=0x7ff3d8005960 comp rx=0 tx=0).stop 2024-04-16T17:04:41.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:41.295+0000 7ff3ede00700 1 -- 172.21.15.45:0/2121754545 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff3e8075640 msgr2=0x7ff3e819b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:41.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:41.295+0000 7ff3ede00700 1 --2- 172.21.15.45:0/2121754545 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff3e8075640 0x7ff3e819b980 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7ff3d400d980 tx=0x7ff3d400dc90 comp rx=0 tx=0).stop 2024-04-16T17:04:41.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:41.295+0000 7ff3e7400700 1 -- 172.21.15.45:0/2121754545 reap_dead start 2024-04-16T17:04:41.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:41.295+0000 7ff3ede00700 1 -- 172.21.15.45:0/2121754545 shutdown_connections 2024-04-16T17:04:41.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:41.295+0000 7ff3ede00700 1 -- 172.21.15.45:0/2121754545 >> 172.21.15.45:0/2121754545 conn(0x7ff3e8079c50 msgr2=0x7ff3e80ffc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:04:41.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:41.295+0000 7ff3ede00700 1 -- 172.21.15.45:0/2121754545 shutdown_connections 2024-04-16T17:04:41.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:41.295+0000 7ff3ede00700 1 -- 172.21.15.45:0/2121754545 wait complete. 2024-04-16T17:04:41.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:04:41 smithi104 ceph-mon[36457]: pgmap v362: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 6 op/s 2024-04-16T17:04:41.933 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:04:41 smithi045 ceph-mon[31676]: pgmap v362: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 6 op/s 2024-04-16T17:04:42.055 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:04:41 smithi117 ceph-mon[36730]: pgmap v362: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 6 op/s 2024-04-16T17:04:42.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:42.434+0000 7fdf4e800700 1 -- 172.21.15.45:0/2883911159 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdf48075640 msgr2=0x7fdf4810efb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:42.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:42.434+0000 7fdf4e800700 1 --2- 172.21.15.45:0/2883911159 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdf48075640 0x7fdf4810efb0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fdf34009990 tx=0x7fdf34009ca0 comp rx=0 tx=0).stop 2024-04-16T17:04:42.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:42.435+0000 7fdf4e800700 1 -- 172.21.15.45:0/2883911159 shutdown_connections 2024-04-16T17:04:42.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:42.435+0000 7fdf4e800700 1 --2- 172.21.15.45:0/2883911159 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdf48074d20 0x7fdf48075100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:42.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:42.435+0000 7fdf4e800700 1 --2- 172.21.15.45:0/2883911159 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdf4810f4f0 0x7fdf481118e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:42.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:42.435+0000 7fdf4e800700 1 --2- 172.21.15.45:0/2883911159 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdf48075640 0x7fdf4810efb0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:42.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:42.435+0000 7fdf4e800700 1 -- 172.21.15.45:0/2883911159 >> 172.21.15.45:0/2883911159 conn(0x7fdf48079c50 msgr2=0x7fdf4807a060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:04:42.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:42.435+0000 7fdf4e800700 1 -- 172.21.15.45:0/2883911159 shutdown_connections 2024-04-16T17:04:42.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:42.435+0000 7fdf4e800700 1 -- 172.21.15.45:0/2883911159 wait complete. 2024-04-16T17:04:42.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:42.436+0000 7fdf4e800700 1 Processor -- start 2024-04-16T17:04:42.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:42.436+0000 7fdf4e800700 1 -- start start 2024-04-16T17:04:42.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:42.437+0000 7fdf4e800700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdf48074d20 0x7fdf4819b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:42.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:42.437+0000 7fdf4e800700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdf48075640 0x7fdf4819ba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:42.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:42.437+0000 7fdf4e800700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdf4810f4f0 0x7fdf4819fda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:42.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:42.437+0000 7fdf4e800700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fdf48115050 con 0x7fdf48074d20 2024-04-16T17:04:42.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:42.437+0000 7fdf4e800700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fdf48114ed0 con 0x7fdf48075640 2024-04-16T17:04:42.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:42.437+0000 7fdf4e800700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fdf481151d0 con 0x7fdf4810f4f0 2024-04-16T17:04:42.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:42.437+0000 7fdf47400700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdf48074d20 0x7fdf4819b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:42.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:42.437+0000 7fdf47400700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdf48074d20 0x7fdf4819b520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:44040/0 (socket says 172.21.15.45:44040) 2024-04-16T17:04:42.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:42.437+0000 7fdf47400700 1 -- 172.21.15.45:0/834000065 learned_addr learned my addr 172.21.15.45:0/834000065 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:04:42.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:42.437+0000 7fdf47e00700 1 --2- 172.21.15.45:0/834000065 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdf4810f4f0 0x7fdf4819fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:42.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:42.437+0000 7fdf46a00700 1 --2- 172.21.15.45:0/834000065 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdf48075640 0x7fdf4819ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:42.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:42.438+0000 7fdf47400700 1 -- 172.21.15.45:0/834000065 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdf4810f4f0 msgr2=0x7fdf4819fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:42.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:42.438+0000 7fdf47400700 1 --2- 172.21.15.45:0/834000065 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdf4810f4f0 0x7fdf4819fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:42.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:42.438+0000 7fdf47400700 1 -- 172.21.15.45:0/834000065 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdf48075640 msgr2=0x7fdf4819ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:42.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:42.438+0000 7fdf47400700 1 --2- 172.21.15.45:0/834000065 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdf48075640 0x7fdf4819ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:42.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:42.438+0000 7fdf47400700 1 -- 172.21.15.45:0/834000065 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf481a04e0 con 0x7fdf48074d20 2024-04-16T17:04:42.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:42.438+0000 7fdf47e00700 1 --2- 172.21.15.45:0/834000065 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdf4810f4f0 0x7fdf4819fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:04:42.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:42.438+0000 7fdf47400700 1 --2- 172.21.15.45:0/834000065 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdf48074d20 0x7fdf4819b520 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fdf3800e9a0 tx=0x7fdf3800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:04:42.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:42.438+0000 7fdf3fe00700 1 -- 172.21.15.45:0/834000065 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf3800c9c0 con 0x7fdf48074d20 2024-04-16T17:04:42.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:42.438+0000 7fdf3fe00700 1 -- 172.21.15.45:0/834000065 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdf3800cb60 con 0x7fdf48074d20 2024-04-16T17:04:42.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:42.438+0000 7fdf4e800700 1 -- 172.21.15.45:0/834000065 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf481a07d0 con 0x7fdf48074d20 2024-04-16T17:04:42.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:42.438+0000 7fdf3fe00700 1 -- 172.21.15.45:0/834000065 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf38018660 con 0x7fdf48074d20 2024-04-16T17:04:42.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:42.439+0000 7fdf4e800700 1 -- 172.21.15.45:0/834000065 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf481a8050 con 0x7fdf48074d20 2024-04-16T17:04:42.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:42.439+0000 7fdf4e800700 1 -- 172.21.15.45:0/834000065 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fdf48100760 con 0x7fdf48074d20 2024-04-16T17:04:42.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:42.442+0000 7fdf3fe00700 1 -- 172.21.15.45:0/834000065 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fdf3800cd10 con 0x7fdf48074d20 2024-04-16T17:04:42.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:42.443+0000 7fdf3fe00700 1 --2- 172.21.15.45:0/834000065 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fdf2c0a3d50 0x7fdf2c0a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:42.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:42.443+0000 7fdf3fe00700 1 -- 172.21.15.45:0/834000065 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7fdf380d0ef0 con 0x7fdf48074d20 2024-04-16T17:04:42.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:42.444+0000 7fdf46a00700 1 --2- 172.21.15.45:0/834000065 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fdf2c0a3d50 0x7fdf2c0a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:42.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:42.444+0000 7fdf3fe00700 1 --2- 172.21.15.45:0/834000065 >> [v2:172.21.15.104:6824/1404794652,v1:172.21.15.104:6825/1404794652] conn(0x7fdf2c0aeeb0 0x7fdf2c0b1350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:42.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:42.444+0000 7fdf3fe00700 1 -- 172.21.15.45:0/834000065 --> [v2:172.21.15.104:6824/1404794652,v1:172.21.15.104:6825/1404794652] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fdf38019ec0 con 0x7fdf2c0aeeb0 2024-04-16T17:04:42.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:42.444+0000 7fdf3fe00700 1 -- 172.21.15.45:0/834000065 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7fdf380d12a0 con 0x7fdf48074d20 2024-04-16T17:04:42.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:42.444+0000 7fdf47e00700 1 --2- 172.21.15.45:0/834000065 >> [v2:172.21.15.104:6824/1404794652,v1:172.21.15.104:6825/1404794652] conn(0x7fdf2c0aeeb0 0x7fdf2c0b1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:42.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:42.445+0000 7fdf46a00700 1 --2- 172.21.15.45:0/834000065 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fdf2c0a3d50 0x7fdf2c0a6210 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fdf48111180 tx=0x7fdf34005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:04:42.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:42.445+0000 7fdf47e00700 1 --2- 172.21.15.45:0/834000065 >> [v2:172.21.15.104:6824/1404794652,v1:172.21.15.104:6825/1404794652] conn(0x7fdf2c0aeeb0 0x7fdf2c0b1350 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:04:42.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:42.446+0000 7fdf3fe00700 1 -- 172.21.15.45:0/834000065 <== osd.10 v2:172.21.15.104:6824/1404794652 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fdf38019ec0 con 0x7fdf2c0aeeb0 2024-04-16T17:04:42.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:42.464+0000 7fdf4e800700 1 -- 172.21.15.45:0/834000065 --> [v2:172.21.15.104:6824/1404794652,v1:172.21.15.104:6825/1404794652] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fdf48076410 con 0x7fdf2c0aeeb0 2024-04-16T17:04:42.466 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:42.465+0000 7fdf3fe00700 1 -- 172.21.15.45:0/834000065 <== osd.10 v2:172.21.15.104:6824/1404794652 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fdf48076410 con 0x7fdf2c0aeeb0 2024-04-16T17:04:42.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:42.466+0000 7fdf4e800700 1 -- 172.21.15.45:0/834000065 >> [v2:172.21.15.104:6824/1404794652,v1:172.21.15.104:6825/1404794652] conn(0x7fdf2c0aeeb0 msgr2=0x7fdf2c0b1350 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:42.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:42.466+0000 7fdf4e800700 1 --2- 172.21.15.45:0/834000065 >> [v2:172.21.15.104:6824/1404794652,v1:172.21.15.104:6825/1404794652] conn(0x7fdf2c0aeeb0 0x7fdf2c0b1350 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:42.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:42.466+0000 7fdf4e800700 1 -- 172.21.15.45:0/834000065 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fdf2c0a3d50 msgr2=0x7fdf2c0a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:42.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:42.466+0000 7fdf4e800700 1 --2- 172.21.15.45:0/834000065 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fdf2c0a3d50 0x7fdf2c0a6210 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fdf48111180 tx=0x7fdf34005fd0 comp rx=0 tx=0).stop 2024-04-16T17:04:42.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:42.466+0000 7fdf4e800700 1 -- 172.21.15.45:0/834000065 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdf48074d20 msgr2=0x7fdf4819b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:42.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:42.466+0000 7fdf4e800700 1 --2- 172.21.15.45:0/834000065 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdf48074d20 0x7fdf4819b520 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fdf3800e9a0 tx=0x7fdf3800ecb0 comp rx=0 tx=0).stop 2024-04-16T17:04:42.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:42.466+0000 7fdf47e00700 1 -- 172.21.15.45:0/834000065 reap_dead start 2024-04-16T17:04:42.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:42.466+0000 7fdf4e800700 1 -- 172.21.15.45:0/834000065 shutdown_connections 2024-04-16T17:04:42.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:42.466+0000 7fdf4e800700 1 -- 172.21.15.45:0/834000065 >> 172.21.15.45:0/834000065 conn(0x7fdf48079c50 msgr2=0x7fdf480ffd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:04:42.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:42.466+0000 7fdf4e800700 1 -- 172.21.15.45:0/834000065 shutdown_connections 2024-04-16T17:04:42.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:42.466+0000 7fdf4e800700 1 -- 172.21.15.45:0/834000065 wait complete. 2024-04-16T17:04:42.815 INFO:teuthology.orchestra.run.smithi045.stdout:223338299444 2024-04-16T17:04:42.815 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45 shell --fsid b1fecd38-fc11-11ee-bc90-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-04-16T17:04:43.496 INFO:teuthology.orchestra.run.smithi045.stdout:292057776168 2024-04-16T17:04:43.496 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45 shell --fsid b1fecd38-fc11-11ee-bc90-c7b262605968 -- ceph osd last-stat-seq osd.9 2024-04-16T17:04:43.693 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:04:43 smithi045 ceph-mon[31676]: pgmap v363: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-16T17:04:43.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:04:43 smithi104 ceph-mon[36457]: pgmap v363: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-16T17:04:44.055 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:04:43 smithi117 ceph-mon[36730]: pgmap v363: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-16T17:04:44.424 INFO:teuthology.orchestra.run.smithi045.stdout:201863462970 2024-04-16T17:04:44.424 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45 shell --fsid b1fecd38-fc11-11ee-bc90-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-04-16T17:04:44.512 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/mon.a/config 2024-04-16T17:04:44.515 INFO:teuthology.orchestra.run.smithi045.stdout:55834574933 2024-04-16T17:04:44.515 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45 shell --fsid b1fecd38-fc11-11ee-bc90-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-04-16T17:04:45.466 INFO:teuthology.orchestra.run.smithi045.stdout:313532612644 2024-04-16T17:04:45.466 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45 shell --fsid b1fecd38-fc11-11ee-bc90-c7b262605968 -- ceph osd last-stat-seq osd.10 2024-04-16T17:04:45.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:44.792+0000 7f9bbd600700 1 -- 172.21.15.45:0/2615482179 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9bb8074d20 msgr2=0x7f9bb8075100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:45.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:44.792+0000 7f9bbd600700 1 --2- 172.21.15.45:0/2615482179 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9bb8074d20 0x7f9bb8075100 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f9ba8009990 tx=0x7f9ba8009ca0 comp rx=0 tx=0).stop 2024-04-16T17:04:45.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:44.792+0000 7f9bbd600700 1 -- 172.21.15.45:0/2615482179 shutdown_connections 2024-04-16T17:04:45.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:44.792+0000 7f9bbd600700 1 --2- 172.21.15.45:0/2615482179 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9bb810f550 0x7f9bb8111940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:45.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:44.792+0000 7f9bbd600700 1 --2- 172.21.15.45:0/2615482179 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9bb8074d20 0x7f9bb8075100 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:45.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:44.792+0000 7f9bbd600700 1 --2- 172.21.15.45:0/2615482179 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9bb8075640 0x7f9bb810f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:45.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:44.792+0000 7f9bbd600700 1 -- 172.21.15.45:0/2615482179 >> 172.21.15.45:0/2615482179 conn(0x7f9bb8079c50 msgr2=0x7f9bb807a060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:04:45.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:44.792+0000 7f9bbd600700 1 -- 172.21.15.45:0/2615482179 shutdown_connections 2024-04-16T17:04:45.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:44.792+0000 7f9bbd600700 1 -- 172.21.15.45:0/2615482179 wait complete. 2024-04-16T17:04:45.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:44.794+0000 7f9bbd600700 1 Processor -- start 2024-04-16T17:04:45.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:44.794+0000 7f9bbd600700 1 -- start start 2024-04-16T17:04:45.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:44.794+0000 7f9bbd600700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9bb8074d20 0x7f9bb819b410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:45.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:44.794+0000 7f9bbd600700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9bb8075640 0x7f9bb819b950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:45.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:44.794+0000 7f9bbd600700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9bb810f550 0x7f9bb819fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:45.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:44.794+0000 7f9bbd600700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f9bb8115040 con 0x7f9bb8075640 2024-04-16T17:04:45.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:44.794+0000 7f9bbd600700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f9bb8114ec0 con 0x7f9bb8074d20 2024-04-16T17:04:45.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:44.794+0000 7f9bbd600700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f9bb81151c0 con 0x7f9bb810f550 2024-04-16T17:04:45.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:44.795+0000 7f9bb6a00700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9bb8074d20 0x7f9bb819b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:45.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:44.795+0000 7f9bb6a00700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9bb8074d20 0x7f9bb819b410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:43058/0 (socket says 172.21.15.45:43058) 2024-04-16T17:04:45.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:44.795+0000 7f9bb6a00700 1 -- 172.21.15.45:0/5603537 learned_addr learned my addr 172.21.15.45:0/5603537 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:04:45.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:44.795+0000 7f9bb6000700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9bb8075640 0x7f9bb819b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:45.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:44.795+0000 7f9bb7400700 1 --2- 172.21.15.45:0/5603537 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9bb810f550 0x7f9bb819fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:45.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:44.795+0000 7f9bb6a00700 1 -- 172.21.15.45:0/5603537 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9bb810f550 msgr2=0x7f9bb819fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:45.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:44.795+0000 7f9bb6a00700 1 --2- 172.21.15.45:0/5603537 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9bb810f550 0x7f9bb819fc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:45.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:44.795+0000 7f9bb6a00700 1 -- 172.21.15.45:0/5603537 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9bb8075640 msgr2=0x7f9bb819b950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:45.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:44.795+0000 7f9bb6a00700 1 --2- 172.21.15.45:0/5603537 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9bb8075640 0x7f9bb819b950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:45.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:44.795+0000 7f9bb6a00700 1 -- 172.21.15.45:0/5603537 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9bb81a03d0 con 0x7f9bb8074d20 2024-04-16T17:04:45.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:44.796+0000 7f9bb6a00700 1 --2- 172.21.15.45:0/5603537 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9bb8074d20 0x7f9bb819b410 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f9ba800bae0 tx=0x7f9ba800bb10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:04:45.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:44.796+0000 7f9baf400700 1 -- 172.21.15.45:0/5603537 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ba8026070 con 0x7f9bb8074d20 2024-04-16T17:04:45.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:44.796+0000 7f9baf400700 1 -- 172.21.15.45:0/5603537 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9ba8003d70 con 0x7f9bb8074d20 2024-04-16T17:04:45.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:44.796+0000 7f9baf400700 1 -- 172.21.15.45:0/5603537 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ba8021410 con 0x7f9bb8074d20 2024-04-16T17:04:45.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:44.796+0000 7f9bbd600700 1 -- 172.21.15.45:0/5603537 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9bb81a0660 con 0x7f9bb8074d20 2024-04-16T17:04:45.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:44.796+0000 7f9bbd600700 1 -- 172.21.15.45:0/5603537 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9bb81a7f40 con 0x7f9bb8074d20 2024-04-16T17:04:45.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:44.797+0000 7f9bacc00700 1 -- 172.21.15.45:0/5603537 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f9b90001030 con 0x7f9bb8074d20 2024-04-16T17:04:45.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:44.801+0000 7f9baf400700 1 -- 172.21.15.45:0/5603537 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f9ba8029070 con 0x7f9bb8074d20 2024-04-16T17:04:45.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:44.802+0000 7f9baf400700 1 --2- 172.21.15.45:0/5603537 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f9b9c0a3cd0 0x7f9b9c0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:45.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:44.803+0000 7f9baf400700 1 -- 172.21.15.45:0/5603537 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f9ba80d1980 con 0x7f9bb8074d20 2024-04-16T17:04:45.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:44.803+0000 7f9baf400700 1 --2- 172.21.15.45:0/5603537 >> [v2:172.21.15.117:6816/1500286830,v1:172.21.15.117:6817/1500286830] conn(0x7f9b9c0aee30 0x7f9b9c0b12d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:45.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:44.803+0000 7f9bb6000700 1 --2- 172.21.15.45:0/5603537 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f9b9c0a3cd0 0x7f9b9c0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:45.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:44.803+0000 7f9baf400700 1 -- 172.21.15.45:0/5603537 --> [v2:172.21.15.117:6816/1500286830,v1:172.21.15.117:6817/1500286830] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9b9c0b19a0 con 0x7f9b9c0aee30 2024-04-16T17:04:45.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:44.803+0000 7f9baf400700 1 -- 172.21.15.45:0/5603537 <== mon.2 v2:172.21.15.104:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7f9ba80d1d30 con 0x7f9bb8074d20 2024-04-16T17:04:45.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:44.803+0000 7f9bb7400700 1 --2- 172.21.15.45:0/5603537 >> [v2:172.21.15.117:6816/1500286830,v1:172.21.15.117:6817/1500286830] conn(0x7f9b9c0aee30 0x7f9b9c0b12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:45.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:44.804+0000 7f9bb6000700 1 --2- 172.21.15.45:0/5603537 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f9b9c0a3cd0 0x7f9b9c0a6190 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f9bb8111030 tx=0x7f9ba4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:04:45.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:44.804+0000 7f9bb7400700 1 --2- 172.21.15.45:0/5603537 >> [v2:172.21.15.117:6816/1500286830,v1:172.21.15.117:6817/1500286830] conn(0x7f9b9c0aee30 0x7f9b9c0b12d0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:04:45.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:44.805+0000 7f9baf400700 1 -- 172.21.15.45:0/5603537 <== osd.8 v2:172.21.15.117:6816/1500286830 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f9b9c0b19a0 con 0x7f9b9c0aee30 2024-04-16T17:04:45.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:44.824+0000 7f9bacc00700 1 -- 172.21.15.45:0/5603537 --> [v2:172.21.15.117:6816/1500286830,v1:172.21.15.117:6817/1500286830] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f9b90002e60 con 0x7f9b9c0aee30 2024-04-16T17:04:45.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:44.824+0000 7f9baf400700 1 -- 172.21.15.45:0/5603537 <== osd.8 v2:172.21.15.117:6816/1500286830 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f9b90002e60 con 0x7f9b9c0aee30 2024-04-16T17:04:45.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:44.825+0000 7f9bacc00700 1 -- 172.21.15.45:0/5603537 >> [v2:172.21.15.117:6816/1500286830,v1:172.21.15.117:6817/1500286830] conn(0x7f9b9c0aee30 msgr2=0x7f9b9c0b12d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:45.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:44.825+0000 7f9bacc00700 1 --2- 172.21.15.45:0/5603537 >> [v2:172.21.15.117:6816/1500286830,v1:172.21.15.117:6817/1500286830] conn(0x7f9b9c0aee30 0x7f9b9c0b12d0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:45.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:44.825+0000 7f9bacc00700 1 -- 172.21.15.45:0/5603537 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f9b9c0a3cd0 msgr2=0x7f9b9c0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:45.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:44.825+0000 7f9bacc00700 1 --2- 172.21.15.45:0/5603537 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f9b9c0a3cd0 0x7f9b9c0a6190 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f9bb8111030 tx=0x7f9ba4009260 comp rx=0 tx=0).stop 2024-04-16T17:04:45.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:44.825+0000 7f9bacc00700 1 -- 172.21.15.45:0/5603537 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9bb8074d20 msgr2=0x7f9bb819b410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:45.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:44.825+0000 7f9bacc00700 1 --2- 172.21.15.45:0/5603537 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9bb8074d20 0x7f9bb819b410 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f9ba800bae0 tx=0x7f9ba800bb10 comp rx=0 tx=0).stop 2024-04-16T17:04:45.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:44.825+0000 7f9bb7400700 1 -- 172.21.15.45:0/5603537 reap_dead start 2024-04-16T17:04:45.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:44.825+0000 7f9bacc00700 1 -- 172.21.15.45:0/5603537 shutdown_connections 2024-04-16T17:04:45.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:44.825+0000 7f9bacc00700 1 -- 172.21.15.45:0/5603537 >> 172.21.15.45:0/5603537 conn(0x7f9bb8079c50 msgr2=0x7f9bb80ffc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:04:45.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:44.826+0000 7f9bacc00700 1 -- 172.21.15.45:0/5603537 shutdown_connections 2024-04-16T17:04:45.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:44.826+0000 7f9bacc00700 1 -- 172.21.15.45:0/5603537 wait complete. 2024-04-16T17:04:45.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:04:45 smithi045 ceph-mon[31676]: pgmap v364: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 5 op/s 2024-04-16T17:04:45.944 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:04:45 smithi104 ceph-mon[36457]: pgmap v364: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 5 op/s 2024-04-16T17:04:46.055 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:04:45 smithi117 ceph-mon[36730]: pgmap v364: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 5 op/s 2024-04-16T17:04:47.964 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:04:47 smithi045 ceph-mon[31676]: pgmap v365: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-16T17:04:48.027 INFO:teuthology.orchestra.run.smithi045.stdout:257698037808 2024-04-16T17:04:48.028 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45 shell --fsid b1fecd38-fc11-11ee-bc90-c7b262605968 -- ceph osd last-stat-seq osd.8 2024-04-16T17:04:48.055 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:04:47 smithi117 ceph-mon[36730]: pgmap v365: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-16T17:04:48.178 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:04:47 smithi104 ceph-mon[36457]: pgmap v365: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-16T17:04:49.150 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/mon.a/config 2024-04-16T17:04:49.154 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/mon.a/config 2024-04-16T17:04:49.495 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/mon.a/config 2024-04-16T17:04:49.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:49.499+0000 7f2626400700 1 -- 172.21.15.45:0/3296158869 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f262006ad40 msgr2=0x7f262006b1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:49.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:49.499+0000 7f2626400700 1 --2- 172.21.15.45:0/3296158869 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f262006ad40 0x7f262006b1a0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f260c01a9a0 tx=0x7f260c01acb0 comp rx=0 tx=0).stop 2024-04-16T17:04:49.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:49.499+0000 7f2626400700 1 -- 172.21.15.45:0/3296158869 shutdown_connections 2024-04-16T17:04:49.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:49.499+0000 7f2626400700 1 --2- 172.21.15.45:0/3296158869 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2620105530 0x7f26201059c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:49.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:49.499+0000 7f2626400700 1 --2- 172.21.15.45:0/3296158869 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2620107a20 0x7f262006a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:49.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:49.499+0000 7f2626400700 1 --2- 172.21.15.45:0/3296158869 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f262006ad40 0x7f262006b1a0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:49.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:49.499+0000 7f2626400700 1 -- 172.21.15.45:0/3296158869 >> 172.21.15.45:0/3296158869 conn(0x7f26200789a0 msgr2=0x7f2620078db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:04:49.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:49.499+0000 7f2626400700 1 -- 172.21.15.45:0/3296158869 shutdown_connections 2024-04-16T17:04:49.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:49.500+0000 7f2626400700 1 -- 172.21.15.45:0/3296158869 wait complete. 2024-04-16T17:04:49.501 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/mon.a/config 2024-04-16T17:04:49.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:49.501+0000 7f2626400700 1 Processor -- start 2024-04-16T17:04:49.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:49.502+0000 7f2626400700 1 -- start start 2024-04-16T17:04:49.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:49.502+0000 7f2626400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f262006ad40 0x7f2620115eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:49.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:49.502+0000 7f2626400700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2620105530 0x7f2620111fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:49.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:49.502+0000 7f2626400700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2620107a20 0x7f26201124e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:49.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:49.502+0000 7f2626400700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f262010a930 con 0x7f2620105530 2024-04-16T17:04:49.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:49.502+0000 7f2626400700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f262010a7b0 con 0x7f2620107a20 2024-04-16T17:04:49.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:49.502+0000 7f2626400700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f262010aab0 con 0x7f262006ad40 2024-04-16T17:04:49.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:49.503+0000 7f261fe00700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2620107a20 0x7f26201124e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:49.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:49.503+0000 7f261fe00700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2620107a20 0x7f26201124e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:43074/0 (socket says 172.21.15.45:43074) 2024-04-16T17:04:49.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:49.503+0000 7f261fe00700 1 -- 172.21.15.45:0/1370710486 learned_addr learned my addr 172.21.15.45:0/1370710486 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:04:49.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:49.503+0000 7f261fe00700 1 -- 172.21.15.45:0/1370710486 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f262006ad40 msgr2=0x7f2620115eb0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-16T17:04:49.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:49.503+0000 7f261fe00700 1 --2- 172.21.15.45:0/1370710486 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f262006ad40 0x7f2620115eb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:49.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:49.503+0000 7f261fe00700 1 -- 172.21.15.45:0/1370710486 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2620105530 msgr2=0x7f2620111fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:49.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:49.503+0000 7f261fe00700 1 --2- 172.21.15.45:0/1370710486 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2620105530 0x7f2620111fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:49.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:49.504+0000 7f261fe00700 1 -- 172.21.15.45:0/1370710486 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2620112b20 con 0x7f2620107a20 2024-04-16T17:04:49.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:49.504+0000 7f261fe00700 1 --2- 172.21.15.45:0/1370710486 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2620107a20 0x7f26201124e0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f261800ef60 tx=0x7f261800ef90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:04:49.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:49.505+0000 7f2617e00700 1 -- 172.21.15.45:0/1370710486 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26180092e0 con 0x7f2620107a20 2024-04-16T17:04:49.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:49.505+0000 7f2617e00700 1 -- 172.21.15.45:0/1370710486 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2618004bb0 con 0x7f2620107a20 2024-04-16T17:04:49.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:49.505+0000 7f2617e00700 1 -- 172.21.15.45:0/1370710486 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2618004310 con 0x7f2620107a20 2024-04-16T17:04:49.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:49.505+0000 7f2626400700 1 -- 172.21.15.45:0/1370710486 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2620112db0 con 0x7f2620107a20 2024-04-16T17:04:49.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:49.505+0000 7f2626400700 1 -- 172.21.15.45:0/1370710486 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f262006c800 con 0x7f2620107a20 2024-04-16T17:04:49.507 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/mon.a/config 2024-04-16T17:04:49.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:49.508+0000 7f2626400700 1 -- 172.21.15.45:0/1370710486 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2620102010 con 0x7f2620107a20 2024-04-16T17:04:49.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:49.510+0000 7f2617e00700 1 -- 172.21.15.45:0/1370710486 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f261801b030 con 0x7f2620107a20 2024-04-16T17:04:49.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:49.513+0000 7f2617e00700 1 --2- 172.21.15.45:0/1370710486 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f26040a3d50 0x7f26040a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:49.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:49.513+0000 7f2617e00700 1 -- 172.21.15.45:0/1370710486 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f26180d0300 con 0x7f2620107a20 2024-04-16T17:04:49.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:49.516+0000 7f261f400700 1 --2- 172.21.15.45:0/1370710486 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f26040a3d50 0x7f26040a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:49.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:49.517+0000 7f261f400700 1 --2- 172.21.15.45:0/1370710486 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f26040a3d50 0x7f26040a6210 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f2610009730 tx=0x7f2610006cb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:04:49.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:49.518+0000 7f2617e00700 1 -- 172.21.15.45:0/1370710486 <== mon.2 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2618022020 con 0x7f2620107a20 2024-04-16T17:04:49.522 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/mon.a/config 2024-04-16T17:04:49.704 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:04:49 smithi045 ceph-mon[31676]: pgmap v366: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T17:04:49.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:49.815+0000 7f2626400700 1 -- 172.21.15.45:0/1370710486 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f262006cb20 con 0x7f2620107a20 2024-04-16T17:04:49.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:49.816+0000 7f2617e00700 1 -- 172.21.15.45:0/1370710486 <== mon.2 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f2618010040 con 0x7f2620107a20 2024-04-16T17:04:49.817 INFO:teuthology.orchestra.run.smithi045.stdout:167503724610 2024-04-16T17:04:49.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:49.818+0000 7f2626400700 1 -- 172.21.15.45:0/1370710486 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f26040a3d50 msgr2=0x7f26040a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:49.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:49.818+0000 7f2626400700 1 --2- 172.21.15.45:0/1370710486 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f26040a3d50 0x7f26040a6210 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f2610009730 tx=0x7f2610006cb0 comp rx=0 tx=0).stop 2024-04-16T17:04:49.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:49.818+0000 7f2626400700 1 -- 172.21.15.45:0/1370710486 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2620107a20 msgr2=0x7f26201124e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:49.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:49.818+0000 7f2626400700 1 --2- 172.21.15.45:0/1370710486 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2620107a20 0x7f26201124e0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f261800ef60 tx=0x7f261800ef90 comp rx=0 tx=0).stop 2024-04-16T17:04:49.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:49.818+0000 7f2626400700 1 -- 172.21.15.45:0/1370710486 shutdown_connections 2024-04-16T17:04:49.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:49.818+0000 7f2626400700 1 --2- 172.21.15.45:0/1370710486 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f26040a3d50 0x7f26040a6210 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:49.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:49.818+0000 7f2626400700 1 --2- 172.21.15.45:0/1370710486 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f262006ad40 0x7f2620115eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:49.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:49.818+0000 7f2626400700 1 --2- 172.21.15.45:0/1370710486 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2620107a20 0x7f26201124e0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:49.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:49.818+0000 7f2626400700 1 --2- 172.21.15.45:0/1370710486 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2620105530 0x7f2620111fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:49.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:49.818+0000 7f2626400700 1 -- 172.21.15.45:0/1370710486 >> 172.21.15.45:0/1370710486 conn(0x7f26200789a0 msgr2=0x7f2620104510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:04:49.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:49.819+0000 7f2626400700 1 -- 172.21.15.45:0/1370710486 shutdown_connections 2024-04-16T17:04:49.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:49.819+0000 7f2626400700 1 -- 172.21.15.45:0/1370710486 wait complete. 2024-04-16T17:04:50.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:04:49 smithi117 ceph-mon[36730]: pgmap v366: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T17:04:50.178 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:04:49 smithi104 ceph-mon[36457]: pgmap v366: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T17:04:50.939 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:04:50 smithi045 ceph-mon[31676]: from='client.? 172.21.15.45:0/1370710486' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-16T17:04:51.055 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:04:50 smithi117 ceph-mon[36730]: from='client.? 172.21.15.45:0/1370710486' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-16T17:04:51.178 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:04:50 smithi104 ceph-mon[36457]: from='client.? 172.21.15.45:0/1370710486' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-16T17:04:51.789 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:04:51 smithi045 ceph-mon[31676]: pgmap v367: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-04-16T17:04:52.055 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:04:51 smithi117 ceph-mon[36730]: pgmap v367: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-04-16T17:04:52.178 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:04:51 smithi104 ceph-mon[36457]: pgmap v367: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-04-16T17:04:52.844 INFO:tasks.cephadm.ceph_manager.ceph:need seq 167503724604 got 167503724610 for osd.5 2024-04-16T17:04:52.845 DEBUG:teuthology.parallel:result is None 2024-04-16T17:04:53.978 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:04:53 smithi045 ceph-mon[31676]: pgmap v368: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 6 op/s 2024-04-16T17:04:54.055 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:04:53 smithi117 ceph-mon[36730]: pgmap v368: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 6 op/s 2024-04-16T17:04:54.178 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:04:53 smithi104 ceph-mon[36457]: pgmap v368: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 6 op/s 2024-04-16T17:04:54.738 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/mon.a/config 2024-04-16T17:04:55.739 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:04:55 smithi045 ceph-mon[31676]: pgmap v369: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T17:04:56.014 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:04:55 smithi104 ceph-mon[36457]: pgmap v369: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T17:04:56.055 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:04:55 smithi117 ceph-mon[36730]: pgmap v369: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T17:04:57.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:57.074+0000 7ff524200700 1 -- 172.21.15.45:0/1651879344 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff51c10f4f0 msgr2=0x7ff51c1118e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:57.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:57.074+0000 7ff524200700 1 --2- 172.21.15.45:0/1651879344 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff51c10f4f0 0x7ff51c1118e0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7ff51400b0a0 tx=0x7ff51400b3b0 comp rx=0 tx=0).stop 2024-04-16T17:04:57.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:57.074+0000 7ff524200700 1 -- 172.21.15.45:0/1651879344 shutdown_connections 2024-04-16T17:04:57.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:57.074+0000 7ff524200700 1 --2- 172.21.15.45:0/1651879344 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff51c10f4f0 0x7ff51c1118e0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:57.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:57.074+0000 7ff524200700 1 --2- 172.21.15.45:0/1651879344 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff51c074d20 0x7ff51c075100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:57.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:57.074+0000 7ff524200700 1 --2- 172.21.15.45:0/1651879344 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff51c075640 0x7ff51c10efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:57.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:57.074+0000 7ff524200700 1 -- 172.21.15.45:0/1651879344 >> 172.21.15.45:0/1651879344 conn(0x7ff51c079ca0 msgr2=0x7ff51c07a0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:04:57.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:57.074+0000 7ff524200700 1 -- 172.21.15.45:0/1651879344 shutdown_connections 2024-04-16T17:04:57.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:57.074+0000 7ff524200700 1 -- 172.21.15.45:0/1651879344 wait complete. 2024-04-16T17:04:57.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:57.075+0000 7ff524200700 1 Processor -- start 2024-04-16T17:04:57.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:57.076+0000 7ff524200700 1 -- start start 2024-04-16T17:04:57.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:57.076+0000 7ff524200700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff51c074d20 0x7ff51c19b500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:57.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:57.076+0000 7ff524200700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff51c075640 0x7ff51c19ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:57.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:57.076+0000 7ff524200700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff51c10f4f0 0x7ff51c19fd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:57.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:57.076+0000 7ff524200700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7ff51c115070 con 0x7ff51c074d20 2024-04-16T17:04:57.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:57.076+0000 7ff524200700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7ff51c114ef0 con 0x7ff51c075640 2024-04-16T17:04:57.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:57.076+0000 7ff524200700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7ff51c1151f0 con 0x7ff51c10f4f0 2024-04-16T17:04:57.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:57.076+0000 7ff521800700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff51c074d20 0x7ff51c19b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:57.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:57.077+0000 7ff520e00700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff51c075640 0x7ff51c19ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:57.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:57.077+0000 7ff520e00700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff51c075640 0x7ff51c19ba40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:58046/0 (socket says 172.21.15.45:58046) 2024-04-16T17:04:57.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:57.077+0000 7ff520e00700 1 -- 172.21.15.45:0/273956507 learned_addr learned my addr 172.21.15.45:0/273956507 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:04:57.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:57.077+0000 7ff522200700 1 --2- 172.21.15.45:0/273956507 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff51c10f4f0 0x7ff51c19fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:57.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:57.077+0000 7ff520e00700 1 -- 172.21.15.45:0/273956507 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff51c10f4f0 msgr2=0x7ff51c19fd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:57.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:57.077+0000 7ff520e00700 1 --2- 172.21.15.45:0/273956507 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff51c10f4f0 0x7ff51c19fd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:57.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:57.077+0000 7ff520e00700 1 -- 172.21.15.45:0/273956507 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff51c074d20 msgr2=0x7ff51c19b500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:57.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:57.077+0000 7ff520e00700 1 --2- 172.21.15.45:0/273956507 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff51c074d20 0x7ff51c19b500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:57.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:57.077+0000 7ff520e00700 1 -- 172.21.15.45:0/273956507 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff51c1a04c0 con 0x7ff51c075640 2024-04-16T17:04:57.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:57.077+0000 7ff520e00700 1 --2- 172.21.15.45:0/273956507 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff51c075640 0x7ff51c19ba40 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7ff50800e9a0 tx=0x7ff50800ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:04:57.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:57.078+0000 7ff51a000700 1 -- 172.21.15.45:0/273956507 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff50800ca10 con 0x7ff51c075640 2024-04-16T17:04:57.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:57.078+0000 7ff51a000700 1 -- 172.21.15.45:0/273956507 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff50800cbb0 con 0x7ff51c075640 2024-04-16T17:04:57.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:57.078+0000 7ff51a000700 1 -- 172.21.15.45:0/273956507 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff508018680 con 0x7ff51c075640 2024-04-16T17:04:57.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:57.078+0000 7ff524200700 1 -- 172.21.15.45:0/273956507 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff51c1a07b0 con 0x7ff51c075640 2024-04-16T17:04:57.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:57.078+0000 7ff524200700 1 -- 172.21.15.45:0/273956507 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff51c1a8030 con 0x7ff51c075640 2024-04-16T17:04:57.621 INFO:teuthology.orchestra.run.smithi045.stdout:146028888136 2024-04-16T17:04:57.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:57.079+0000 7ff524200700 1 -- 172.21.15.45:0/273956507 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff51c0697e0 con 0x7ff51c075640 2024-04-16T17:04:57.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:57.081+0000 7ff51a000700 1 -- 172.21.15.45:0/273956507 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7ff508010430 con 0x7ff51c075640 2024-04-16T17:04:57.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:57.083+0000 7ff51a000700 1 --2- 172.21.15.45:0/273956507 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7ff5040a3cd0 0x7ff5040a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:04:57.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:57.083+0000 7ff51a000700 1 -- 172.21.15.45:0/273956507 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7ff5080d0f80 con 0x7ff51c075640 2024-04-16T17:04:57.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:57.084+0000 7ff521800700 1 --2- 172.21.15.45:0/273956507 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7ff5040a3cd0 0x7ff5040a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:04:57.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:57.084+0000 7ff521800700 1 --2- 172.21.15.45:0/273956507 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7ff5040a3cd0 0x7ff5040a6190 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7ff510009c60 tx=0x7ff510009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:04:57.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:57.085+0000 7ff51a000700 1 -- 172.21.15.45:0/273956507 <== mon.2 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff50809d440 con 0x7ff51c075640 2024-04-16T17:04:57.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:57.329+0000 7ff524200700 1 -- 172.21.15.45:0/273956507 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7ff51c19c810 con 0x7ff51c075640 2024-04-16T17:04:57.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:57.330+0000 7ff51a000700 1 -- 172.21.15.45:0/273956507 <== mon.2 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7ff50809cd40 con 0x7ff51c075640 2024-04-16T17:04:57.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:57.332+0000 7ff524200700 1 -- 172.21.15.45:0/273956507 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7ff5040a3cd0 msgr2=0x7ff5040a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:57.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:57.332+0000 7ff524200700 1 --2- 172.21.15.45:0/273956507 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7ff5040a3cd0 0x7ff5040a6190 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7ff510009c60 tx=0x7ff510009260 comp rx=0 tx=0).stop 2024-04-16T17:04:57.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:57.332+0000 7ff524200700 1 -- 172.21.15.45:0/273956507 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff51c075640 msgr2=0x7ff51c19ba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:04:57.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:57.332+0000 7ff524200700 1 --2- 172.21.15.45:0/273956507 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff51c075640 0x7ff51c19ba40 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7ff50800e9a0 tx=0x7ff50800ecb0 comp rx=0 tx=0).stop 2024-04-16T17:04:57.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:57.332+0000 7ff524200700 1 -- 172.21.15.45:0/273956507 shutdown_connections 2024-04-16T17:04:57.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:57.332+0000 7ff524200700 1 --2- 172.21.15.45:0/273956507 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7ff5040a3cd0 0x7ff5040a6190 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:57.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:57.332+0000 7ff524200700 1 --2- 172.21.15.45:0/273956507 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff51c10f4f0 0x7ff51c19fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:57.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:57.332+0000 7ff524200700 1 --2- 172.21.15.45:0/273956507 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff51c074d20 0x7ff51c19b500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:57.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:57.332+0000 7ff524200700 1 --2- 172.21.15.45:0/273956507 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff51c075640 0x7ff51c19ba40 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:04:57.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:57.332+0000 7ff524200700 1 -- 172.21.15.45:0/273956507 >> 172.21.15.45:0/273956507 conn(0x7ff51c079ca0 msgr2=0x7ff51c100d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:04:57.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:57.333+0000 7ff524200700 1 -- 172.21.15.45:0/273956507 shutdown_connections 2024-04-16T17:04:57.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:04:57.333+0000 7ff524200700 1 -- 172.21.15.45:0/273956507 wait complete. 2024-04-16T17:04:57.978 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:04:57 smithi045 ceph-mon[31676]: pgmap v370: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-04-16T17:04:57.978 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:04:57 smithi045 ceph-mon[31676]: from='client.? 172.21.15.45:0/273956507' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-16T17:04:58.055 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:04:57 smithi117 ceph-mon[36730]: pgmap v370: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-04-16T17:04:58.055 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:04:57 smithi117 ceph-mon[36730]: from='client.? 172.21.15.45:0/273956507' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-16T17:04:58.178 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:04:57 smithi104 ceph-mon[36457]: pgmap v370: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-04-16T17:04:58.178 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:04:57 smithi104 ceph-mon[36457]: from='client.? 172.21.15.45:0/273956507' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-16T17:04:58.795 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/mon.a/config 2024-04-16T17:04:58.797 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/mon.a/config 2024-04-16T17:04:59.729 INFO:tasks.cephadm.ceph_manager.ceph:need seq 146028888132 got 146028888136 for osd.4 2024-04-16T17:04:59.729 DEBUG:teuthology.parallel:result is None 2024-04-16T17:04:59.906 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:04:59 smithi045 ceph-mon[31676]: pgmap v371: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-16T17:05:00.055 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:04:59 smithi117 ceph-mon[36730]: pgmap v371: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-16T17:05:00.178 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:04:59 smithi104 ceph-mon[36457]: pgmap v371: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-16T17:05:00.725 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/mon.a/config 2024-04-16T17:05:01.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:01.010+0000 7f489b200700 1 -- 172.21.15.45:0/4058104560 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f489c101f30 msgr2=0x7f489c107710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:01.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:01.010+0000 7f489b200700 1 --2- 172.21.15.45:0/4058104560 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f489c101f30 0x7f489c107710 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f4888009990 tx=0x7f4888009ca0 comp rx=0 tx=0).stop 2024-04-16T17:05:01.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:01.011+0000 7f489b200700 1 -- 172.21.15.45:0/4058104560 shutdown_connections 2024-04-16T17:05:01.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:01.011+0000 7f489b200700 1 --2- 172.21.15.45:0/4058104560 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f489c101580 0x7f489c101960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:01.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:01.011+0000 7f489b200700 1 --2- 172.21.15.45:0/4058104560 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f489c107df0 0x7f489c10f9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:01.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:01.011+0000 7f489b200700 1 --2- 172.21.15.45:0/4058104560 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f489c101f30 0x7f489c107710 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:01.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:01.011+0000 7f489b200700 1 -- 172.21.15.45:0/4058104560 >> 172.21.15.45:0/4058104560 conn(0x7f489c076ca0 msgr2=0x7f489c0770b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:05:01.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:01.011+0000 7f489b200700 1 -- 172.21.15.45:0/4058104560 shutdown_connections 2024-04-16T17:05:01.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:01.011+0000 7f489b200700 1 -- 172.21.15.45:0/4058104560 wait complete. 2024-04-16T17:05:01.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:01.012+0000 7f489b200700 1 Processor -- start 2024-04-16T17:05:01.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:01.013+0000 7f489b200700 1 -- start start 2024-04-16T17:05:01.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:01.013+0000 7f489b200700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f489c101580 0x7f489c19b420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:01.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:01.013+0000 7f489b200700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f489c101f30 0x7f489c19b960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:01.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:01.013+0000 7f489b200700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f489c107df0 0x7f489c19fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:01.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:01.013+0000 7f489b200700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f489c112e00 con 0x7f489c101580 2024-04-16T17:05:01.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:01.013+0000 7f489b200700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f489c112c80 con 0x7f489c101f30 2024-04-16T17:05:01.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:01.013+0000 7f489b200700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f489c112f80 con 0x7f489c107df0 2024-04-16T17:05:01.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:01.013+0000 7f4899e00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f489c101580 0x7f489c19b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:01.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:01.013+0000 7f4899400700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f489c101f30 0x7f489c19b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:01.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:01.013+0000 7f4899e00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f489c101580 0x7f489c19b420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43356/0 (socket says 172.21.15.45:43356) 2024-04-16T17:05:01.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:01.013+0000 7f4899400700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f489c101f30 0x7f489c19b960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:32926/0 (socket says 172.21.15.45:32926) 2024-04-16T17:05:01.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:01.013+0000 7f4899e00700 1 -- 172.21.15.45:0/1009850741 learned_addr learned my addr 172.21.15.45:0/1009850741 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:05:01.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:01.014+0000 7f489a800700 1 --2- 172.21.15.45:0/1009850741 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f489c107df0 0x7f489c19fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:01.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:01.014+0000 7f4899400700 1 -- 172.21.15.45:0/1009850741 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f489c107df0 msgr2=0x7f489c19fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:01.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:01.014+0000 7f4899400700 1 --2- 172.21.15.45:0/1009850741 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f489c107df0 0x7f489c19fcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:01.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:01.014+0000 7f4899400700 1 -- 172.21.15.45:0/1009850741 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f489c101580 msgr2=0x7f489c19b420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:01.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:01.014+0000 7f4899400700 1 --2- 172.21.15.45:0/1009850741 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f489c101580 0x7f489c19b420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:01.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:01.014+0000 7f4899400700 1 -- 172.21.15.45:0/1009850741 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f489c1a0430 con 0x7f489c101f30 2024-04-16T17:05:01.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:01.014+0000 7f4899e00700 1 --2- 172.21.15.45:0/1009850741 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f489c101580 0x7f489c19b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:05:01.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:01.014+0000 7f4899400700 1 --2- 172.21.15.45:0/1009850741 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f489c101f30 0x7f489c19b960 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f48880098e0 tx=0x7f488800b8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:05:01.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:01.015+0000 7f4892a00700 1 -- 172.21.15.45:0/1009850741 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4888027070 con 0x7f489c101f30 2024-04-16T17:05:01.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:01.015+0000 7f4892a00700 1 -- 172.21.15.45:0/1009850741 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f488800be60 con 0x7f489c101f30 2024-04-16T17:05:01.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:01.015+0000 7f4892a00700 1 -- 172.21.15.45:0/1009850741 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4888022dd0 con 0x7f489c101f30 2024-04-16T17:05:01.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:01.015+0000 7f489b200700 1 -- 172.21.15.45:0/1009850741 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f489c1a06c0 con 0x7f489c101f30 2024-04-16T17:05:01.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:01.015+0000 7f489b200700 1 -- 172.21.15.45:0/1009850741 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f489c1a7fa0 con 0x7f489c101f30 2024-04-16T17:05:01.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:01.016+0000 7f489b200700 1 -- 172.21.15.45:0/1009850741 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f489c06a360 con 0x7f489c101f30 2024-04-16T17:05:01.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:01.018+0000 7f4892a00700 1 -- 172.21.15.45:0/1009850741 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f488802a070 con 0x7f489c101f30 2024-04-16T17:05:01.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:01.019+0000 7f4892a00700 1 --2- 172.21.15.45:0/1009850741 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f48800a3cd0 0x7f48800a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:01.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:01.020+0000 7f4892a00700 1 -- 172.21.15.45:0/1009850741 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f48880d1a60 con 0x7f489c101f30 2024-04-16T17:05:01.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:01.020+0000 7f4899e00700 1 --2- 172.21.15.45:0/1009850741 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f48800a3cd0 0x7f48800a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:01.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:01.021+0000 7f4899e00700 1 --2- 172.21.15.45:0/1009850741 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f48800a3cd0 0x7f48800a6190 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f488c011590 tx=0x7f488c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:05:01.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:01.022+0000 7f4892a00700 1 -- 172.21.15.45:0/1009850741 <== mon.2 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4888013070 con 0x7f489c101f30 2024-04-16T17:05:01.100 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:05:00 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:05:01.100 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:05:00 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:05:01.178 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:05:00 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:05:01.178 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:05:00 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:05:01.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:01.266+0000 7f489b200700 1 -- 172.21.15.45:0/1009850741 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7f489c19c780 con 0x7f489c101f30 2024-04-16T17:05:01.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:01.267+0000 7f4892a00700 1 -- 172.21.15.45:0/1009850741 <== mon.2 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f488809e040 con 0x7f489c101f30 2024-04-16T17:05:01.269 INFO:teuthology.orchestra.run.smithi045.stdout:292057776172 2024-04-16T17:05:01.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:01.270+0000 7f489b200700 1 -- 172.21.15.45:0/1009850741 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f48800a3cd0 msgr2=0x7f48800a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:01.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:01.270+0000 7f489b200700 1 --2- 172.21.15.45:0/1009850741 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f48800a3cd0 0x7f48800a6190 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f488c011590 tx=0x7f488c009260 comp rx=0 tx=0).stop 2024-04-16T17:05:01.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:01.270+0000 7f489b200700 1 -- 172.21.15.45:0/1009850741 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f489c101f30 msgr2=0x7f489c19b960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:01.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:01.270+0000 7f489b200700 1 --2- 172.21.15.45:0/1009850741 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f489c101f30 0x7f489c19b960 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f48880098e0 tx=0x7f488800b8a0 comp rx=0 tx=0).stop 2024-04-16T17:05:01.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:01.270+0000 7f489b200700 1 -- 172.21.15.45:0/1009850741 shutdown_connections 2024-04-16T17:05:01.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:01.270+0000 7f489b200700 1 --2- 172.21.15.45:0/1009850741 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f48800a3cd0 0x7f48800a6190 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:01.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:01.270+0000 7f489b200700 1 --2- 172.21.15.45:0/1009850741 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f489c107df0 0x7f489c19fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:01.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:01.270+0000 7f489b200700 1 --2- 172.21.15.45:0/1009850741 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f489c101580 0x7f489c19b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:01.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:01.270+0000 7f489b200700 1 --2- 172.21.15.45:0/1009850741 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f489c101f30 0x7f489c19b960 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:01.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:01.270+0000 7f489b200700 1 -- 172.21.15.45:0/1009850741 >> 172.21.15.45:0/1009850741 conn(0x7f489c076ca0 msgr2=0x7f489c0ff6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:05:01.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:01.271+0000 7f489b200700 1 -- 172.21.15.45:0/1009850741 shutdown_connections 2024-04-16T17:05:01.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:01.271+0000 7f489b200700 1 -- 172.21.15.45:0/1009850741 wait complete. 2024-04-16T17:05:01.305 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:05:00 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:05:01.305 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:05:00 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:05:02.016 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:05:01 smithi045 ceph-mon[31676]: pgmap v372: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-16T17:05:02.016 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:05:01 smithi045 ceph-mon[31676]: from='client.? 172.21.15.45:0/1009850741' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-16T17:05:02.017 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:05:01 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:05:02.017 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:05:01 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T17:05:02.017 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:05:01 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:05:02.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:02.097+0000 7fed3ea00700 1 -- 172.21.15.45:0/166907057 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fed38102a20 msgr2=0x7fed38102ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:02.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:02.097+0000 7fed3ea00700 1 --2- 172.21.15.45:0/166907057 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fed38102a20 0x7fed38102ea0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fed24009990 tx=0x7fed24009ca0 comp rx=0 tx=0).stop 2024-04-16T17:05:02.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:02.098+0000 7fed3ea00700 1 -- 172.21.15.45:0/166907057 shutdown_connections 2024-04-16T17:05:02.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:02.098+0000 7fed3ea00700 1 --2- 172.21.15.45:0/166907057 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fed381033e0 0x7fed3810f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:02.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:02.098+0000 7fed3ea00700 1 --2- 172.21.15.45:0/166907057 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fed38108a20 0x7fed38108e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:02.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:02.098+0000 7fed3ea00700 1 --2- 172.21.15.45:0/166907057 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fed38102a20 0x7fed38102ea0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:02.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:02.098+0000 7fed3ea00700 1 -- 172.21.15.45:0/166907057 >> 172.21.15.45:0/166907057 conn(0x7fed380fe570 msgr2=0x7fed38100990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:05:02.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:02.098+0000 7fed3ea00700 1 -- 172.21.15.45:0/166907057 shutdown_connections 2024-04-16T17:05:02.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:02.098+0000 7fed3ea00700 1 -- 172.21.15.45:0/166907057 wait complete. 2024-04-16T17:05:02.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:02.099+0000 7fed3ea00700 1 Processor -- start 2024-04-16T17:05:02.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:02.099+0000 7fed3ea00700 1 -- start start 2024-04-16T17:05:02.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:02.100+0000 7fed3ea00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fed38102a20 0x7fed3819b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:02.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:02.100+0000 7fed3ea00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fed381033e0 0x7fed3819b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:02.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:02.100+0000 7fed3ea00700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fed38108a20 0x7fed3819fc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:02.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:02.100+0000 7fed3ea00700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fed38112db0 con 0x7fed381033e0 2024-04-16T17:05:02.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:02.100+0000 7fed3ea00700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fed38112c30 con 0x7fed38108a20 2024-04-16T17:05:02.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:02.100+0000 7fed3ea00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fed38112f30 con 0x7fed38102a20 2024-04-16T17:05:02.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:02.100+0000 7fed3cc00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fed381033e0 0x7fed3819b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:02.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:02.100+0000 7fed3d600700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fed38102a20 0x7fed3819b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:02.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:02.100+0000 7fed3d600700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fed38102a20 0x7fed3819b380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:58828/0 (socket says 172.21.15.45:58828) 2024-04-16T17:05:02.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:02.100+0000 7fed3d600700 1 -- 172.21.15.45:0/2201292198 learned_addr learned my addr 172.21.15.45:0/2201292198 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:05:02.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:02.100+0000 7fed3e000700 1 --2- 172.21.15.45:0/2201292198 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fed38108a20 0x7fed3819fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:02.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:02.101+0000 7fed3d600700 1 -- 172.21.15.45:0/2201292198 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fed38108a20 msgr2=0x7fed3819fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:02.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:02.101+0000 7fed3d600700 1 --2- 172.21.15.45:0/2201292198 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fed38108a20 0x7fed3819fc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:02.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:02.101+0000 7fed3d600700 1 -- 172.21.15.45:0/2201292198 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fed381033e0 msgr2=0x7fed3819b8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:02.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:02.101+0000 7fed3d600700 1 --2- 172.21.15.45:0/2201292198 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fed381033e0 0x7fed3819b8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:02.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:02.101+0000 7fed3d600700 1 -- 172.21.15.45:0/2201292198 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed381a0390 con 0x7fed38102a20 2024-04-16T17:05:02.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:02.101+0000 7fed3e000700 1 --2- 172.21.15.45:0/2201292198 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fed38108a20 0x7fed3819fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:05:02.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:02.101+0000 7fed3d600700 1 --2- 172.21.15.45:0/2201292198 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fed38102a20 0x7fed3819b380 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fed2c00ef10 tx=0x7fed2c00c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:05:02.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:02.101+0000 7fed36000700 1 -- 172.21.15.45:0/2201292198 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed2c00c9c0 con 0x7fed38102a20 2024-04-16T17:05:02.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:02.101+0000 7fed36000700 1 -- 172.21.15.45:0/2201292198 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed2c019070 con 0x7fed38102a20 2024-04-16T17:05:02.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:02.101+0000 7fed3ea00700 1 -- 172.21.15.45:0/2201292198 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed381a0620 con 0x7fed38102a20 2024-04-16T17:05:02.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:02.102+0000 7fed36000700 1 -- 172.21.15.45:0/2201292198 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed2c007e60 con 0x7fed38102a20 2024-04-16T17:05:02.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:02.102+0000 7fed3ea00700 1 -- 172.21.15.45:0/2201292198 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed381a7f00 con 0x7fed38102a20 2024-04-16T17:05:02.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:02.103+0000 7fed3ea00700 1 -- 172.21.15.45:0/2201292198 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed380697e0 con 0x7fed38102a20 2024-04-16T17:05:02.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:02.106+0000 7fed36000700 1 -- 172.21.15.45:0/2201292198 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fed2c012050 con 0x7fed38102a20 2024-04-16T17:05:02.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:02.107+0000 7fed36000700 1 --2- 172.21.15.45:0/2201292198 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fed200a3d50 0x7fed200a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:02.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:02.108+0000 7fed36000700 1 -- 172.21.15.45:0/2201292198 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7fed2c0d07f0 con 0x7fed38102a20 2024-04-16T17:05:02.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:02.108+0000 7fed3cc00700 1 --2- 172.21.15.45:0/2201292198 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fed200a3d50 0x7fed200a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:02.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:02.109+0000 7fed3cc00700 1 --2- 172.21.15.45:0/2201292198 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fed200a3d50 0x7fed200a6210 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fed2400b2e0 tx=0x7fed24005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:05:02.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:02.112+0000 7fed36000700 1 -- 172.21.15.45:0/2201292198 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fed2c010040 con 0x7fed38102a20 2024-04-16T17:05:02.137 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/mon.a/config 2024-04-16T17:05:02.178 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:05:01 smithi104 ceph-mon[36457]: pgmap v372: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-16T17:05:02.178 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:05:01 smithi104 ceph-mon[36457]: from='client.? 172.21.15.45:0/1009850741' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-16T17:05:02.178 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:05:01 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:05:02.178 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:05:01 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T17:05:02.178 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:05:01 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:05:02.305 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:05:01 smithi117 ceph-mon[36730]: pgmap v372: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-16T17:05:02.306 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:05:01 smithi117 ceph-mon[36730]: from='client.? 172.21.15.45:0/1009850741' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-16T17:05:02.306 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:05:01 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:05:02.306 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:05:01 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T17:05:02.306 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:05:01 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:05:02.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:02.362+0000 7fed3ea00700 1 -- 172.21.15.45:0/2201292198 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7fed381a80d0 con 0x7fed38102a20 2024-04-16T17:05:02.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:02.363+0000 7fed36000700 1 -- 172.21.15.45:0/2201292198 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7fed2c09ccb0 con 0x7fed38102a20 2024-04-16T17:05:02.365 INFO:teuthology.orchestra.run.smithi045.stdout:347892351008 2024-04-16T17:05:02.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:02.366+0000 7fed3ea00700 1 -- 172.21.15.45:0/2201292198 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fed200a3d50 msgr2=0x7fed200a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:02.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:02.366+0000 7fed3ea00700 1 --2- 172.21.15.45:0/2201292198 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fed200a3d50 0x7fed200a6210 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fed2400b2e0 tx=0x7fed24005960 comp rx=0 tx=0).stop 2024-04-16T17:05:02.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:02.366+0000 7fed3ea00700 1 -- 172.21.15.45:0/2201292198 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fed38102a20 msgr2=0x7fed3819b380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:02.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:02.366+0000 7fed3ea00700 1 --2- 172.21.15.45:0/2201292198 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fed38102a20 0x7fed3819b380 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fed2c00ef10 tx=0x7fed2c00c3a0 comp rx=0 tx=0).stop 2024-04-16T17:05:02.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:02.366+0000 7fed3ea00700 1 -- 172.21.15.45:0/2201292198 shutdown_connections 2024-04-16T17:05:02.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:02.366+0000 7fed3ea00700 1 --2- 172.21.15.45:0/2201292198 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fed200a3d50 0x7fed200a6210 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:02.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:02.366+0000 7fed3ea00700 1 --2- 172.21.15.45:0/2201292198 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fed38102a20 0x7fed3819b380 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:02.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:02.366+0000 7fed3ea00700 1 --2- 172.21.15.45:0/2201292198 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fed38108a20 0x7fed3819fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:02.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:02.366+0000 7fed3ea00700 1 --2- 172.21.15.45:0/2201292198 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fed381033e0 0x7fed3819b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:02.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:02.366+0000 7fed3ea00700 1 -- 172.21.15.45:0/2201292198 >> 172.21.15.45:0/2201292198 conn(0x7fed380fe570 msgr2=0x7fed380feac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:05:02.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:02.366+0000 7fed3ea00700 1 -- 172.21.15.45:0/2201292198 shutdown_connections 2024-04-16T17:05:02.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:02.366+0000 7fed3ea00700 1 -- 172.21.15.45:0/2201292198 wait complete. 2024-04-16T17:05:03.178 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:05:02 smithi104 ceph-mon[36457]: from='client.? 172.21.15.45:0/2201292198' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-16T17:05:03.228 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:05:02 smithi045 ceph-mon[31676]: from='client.? 172.21.15.45:0/2201292198' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-16T17:05:03.305 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:05:02 smithi117 ceph-mon[36730]: from='client.? 172.21.15.45:0/2201292198' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-16T17:05:03.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:03.515+0000 7f780fc00700 1 -- 172.21.15.45:0/1678309995 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7808074d20 msgr2=0x7f7808075100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:03.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:03.515+0000 7f780fc00700 1 --2- 172.21.15.45:0/1678309995 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7808074d20 0x7f7808075100 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f77fc009990 tx=0x7f77fc009ca0 comp rx=0 tx=0).stop 2024-04-16T17:05:03.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:03.515+0000 7f780fc00700 1 -- 172.21.15.45:0/1678309995 shutdown_connections 2024-04-16T17:05:03.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:03.515+0000 7f780fc00700 1 --2- 172.21.15.45:0/1678309995 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f780810f4f0 0x7f78081118e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:03.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:03.515+0000 7f780fc00700 1 --2- 172.21.15.45:0/1678309995 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7808074d20 0x7f7808075100 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:03.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:03.515+0000 7f780fc00700 1 --2- 172.21.15.45:0/1678309995 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7808075640 0x7f780810efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:03.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:03.515+0000 7f780fc00700 1 -- 172.21.15.45:0/1678309995 >> 172.21.15.45:0/1678309995 conn(0x7f7808079ca0 msgr2=0x7f780807a0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:05:03.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:03.515+0000 7f780fc00700 1 -- 172.21.15.45:0/1678309995 shutdown_connections 2024-04-16T17:05:03.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:03.515+0000 7f780fc00700 1 -- 172.21.15.45:0/1678309995 wait complete. 2024-04-16T17:05:03.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:03.517+0000 7f780fc00700 1 Processor -- start 2024-04-16T17:05:03.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:03.518+0000 7f780fc00700 1 -- start start 2024-04-16T17:05:03.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:03.518+0000 7f780fc00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7808074d20 0x7f780819b590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:03.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:03.519+0000 7f780fc00700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7808075640 0x7f780819bad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:03.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:03.519+0000 7f780fc00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f780810f4f0 0x7f780819fe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:03.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:03.519+0000 7f780fc00700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f7808115060 con 0x7f780810f4f0 2024-04-16T17:05:03.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:03.519+0000 7f780fc00700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f7808114ee0 con 0x7f7808075640 2024-04-16T17:05:03.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:03.519+0000 7f780fc00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f78081151e0 con 0x7f7808074d20 2024-04-16T17:05:03.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:03.519+0000 7f7807e00700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7808075640 0x7f780819bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:03.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:03.519+0000 7f7807e00700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7808075640 0x7f780819bad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:32976/0 (socket says 172.21.15.45:32976) 2024-04-16T17:05:03.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:03.519+0000 7f780d200700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7808074d20 0x7f780819b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:03.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:03.519+0000 7f7807e00700 1 -- 172.21.15.45:0/2011430028 learned_addr learned my addr 172.21.15.45:0/2011430028 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:05:03.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:03.519+0000 7f780dc00700 1 --2- 172.21.15.45:0/2011430028 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f780810f4f0 0x7f780819fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:03.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:03.520+0000 7f7807e00700 1 -- 172.21.15.45:0/2011430028 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7808074d20 msgr2=0x7f780819b590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:03.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:03.520+0000 7f7807e00700 1 --2- 172.21.15.45:0/2011430028 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7808074d20 0x7f780819b590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:03.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:03.520+0000 7f7807e00700 1 -- 172.21.15.45:0/2011430028 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f780810f4f0 msgr2=0x7f780819fe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:03.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:03.520+0000 7f7807e00700 1 --2- 172.21.15.45:0/2011430028 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f780810f4f0 0x7f780819fe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:03.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:03.520+0000 7f7807e00700 1 -- 172.21.15.45:0/2011430028 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78081a05a0 con 0x7f7808075640 2024-04-16T17:05:03.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:03.520+0000 7f780dc00700 1 --2- 172.21.15.45:0/2011430028 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f780810f4f0 0x7f780819fe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:05:03.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:03.520+0000 7f7807e00700 1 --2- 172.21.15.45:0/2011430028 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7808075640 0x7f780819bad0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f77f400e9a0 tx=0x7f77f400ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:05:03.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:03.520+0000 7f7805600700 1 -- 172.21.15.45:0/2011430028 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77f400cbd0 con 0x7f7808075640 2024-04-16T17:05:03.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:03.520+0000 7f7805600700 1 -- 172.21.15.45:0/2011430028 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f77f400cd70 con 0x7f7808075640 2024-04-16T17:05:03.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:03.520+0000 7f780fc00700 1 -- 172.21.15.45:0/2011430028 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78081a0890 con 0x7f7808075640 2024-04-16T17:05:03.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:03.521+0000 7f7805600700 1 -- 172.21.15.45:0/2011430028 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77f4018750 con 0x7f7808075640 2024-04-16T17:05:03.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:03.521+0000 7f780fc00700 1 -- 172.21.15.45:0/2011430028 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78081a8110 con 0x7f7808075640 2024-04-16T17:05:03.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:03.522+0000 7f780fc00700 1 -- 172.21.15.45:0/2011430028 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78080697e0 con 0x7f7808075640 2024-04-16T17:05:03.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:03.524+0000 7f7805600700 1 -- 172.21.15.45:0/2011430028 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f77f4010430 con 0x7f7808075640 2024-04-16T17:05:03.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:03.525+0000 7f7805600700 1 --2- 172.21.15.45:0/2011430028 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f77f00a3d50 0x7f77f00a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:03.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:03.526+0000 7f7805600700 1 -- 172.21.15.45:0/2011430028 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f77f40d0b90 con 0x7f7808075640 2024-04-16T17:05:03.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:03.526+0000 7f780d200700 1 --2- 172.21.15.45:0/2011430028 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f77f00a3d50 0x7f77f00a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:03.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:03.526+0000 7f780d200700 1 --2- 172.21.15.45:0/2011430028 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f77f00a3d50 0x7f77f00a6210 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f77fc0098e0 tx=0x7f77fc005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:05:03.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:03.530+0000 7f7805600700 1 -- 172.21.15.45:0/2011430028 <== mon.2 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f77f409d050 con 0x7f7808075640 2024-04-16T17:05:03.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:03.661+0000 7fcb28200700 1 -- 172.21.15.45:0/1698323158 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcb20109d30 msgr2=0x7fcb2010dd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:03.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:03.661+0000 7fcb28200700 1 --2- 172.21.15.45:0/1698323158 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcb20109d30 0x7fcb2010dd80 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fcb0c0099c0 tx=0x7fcb0c009cd0 comp rx=0 tx=0).stop 2024-04-16T17:05:03.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:03.661+0000 7fcb28200700 1 -- 172.21.15.45:0/1698323158 shutdown_connections 2024-04-16T17:05:03.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:03.661+0000 7fcb28200700 1 --2- 172.21.15.45:0/1698323158 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcb2010e460 0x7fcb20116030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:03.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:03.661+0000 7fcb28200700 1 --2- 172.21.15.45:0/1698323158 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcb20109380 0x7fcb20109760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:03.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:03.661+0000 7fcb28200700 1 --2- 172.21.15.45:0/1698323158 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcb20109d30 0x7fcb2010dd80 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:03.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:03.661+0000 7fcb28200700 1 -- 172.21.15.45:0/1698323158 >> 172.21.15.45:0/1698323158 conn(0x7fcb200762c0 msgr2=0x7fcb20107910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:05:03.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:03.661+0000 7fcb28200700 1 -- 172.21.15.45:0/1698323158 shutdown_connections 2024-04-16T17:05:03.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:03.661+0000 7fcb28200700 1 -- 172.21.15.45:0/1698323158 wait complete. 2024-04-16T17:05:03.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:03.662+0000 7fcb28200700 1 Processor -- start 2024-04-16T17:05:03.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:03.662+0000 7fcb28200700 1 -- start start 2024-04-16T17:05:03.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:03.663+0000 7fcb28200700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcb20109380 0x7fcb201a56a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:03.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:03.663+0000 7fcb28200700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcb20109d30 0x7fcb201a5be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:03.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:03.663+0000 7fcb28200700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcb2010e460 0x7fcb2019f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:03.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:03.663+0000 7fcb28200700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fcb20119490 con 0x7fcb20109380 2024-04-16T17:05:03.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:03.663+0000 7fcb28200700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fcb20119310 con 0x7fcb20109d30 2024-04-16T17:05:03.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:03.663+0000 7fcb28200700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fcb20119610 con 0x7fcb2010e460 2024-04-16T17:05:03.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:03.663+0000 7fcb25800700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcb20109380 0x7fcb201a56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:03.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:03.663+0000 7fcb25800700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcb20109380 0x7fcb201a56a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43404/0 (socket says 172.21.15.45:43404) 2024-04-16T17:05:03.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:03.663+0000 7fcb25800700 1 -- 172.21.15.45:0/935922996 learned_addr learned my addr 172.21.15.45:0/935922996 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:05:03.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:03.663+0000 7fcb24e00700 1 --2- 172.21.15.45:0/935922996 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcb20109d30 0x7fcb201a5be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:03.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:03.663+0000 7fcb26200700 1 --2- 172.21.15.45:0/935922996 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcb2010e460 0x7fcb2019f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:03.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:03.663+0000 7fcb25800700 1 -- 172.21.15.45:0/935922996 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcb2010e460 msgr2=0x7fcb2019f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:03.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:03.663+0000 7fcb25800700 1 --2- 172.21.15.45:0/935922996 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcb2010e460 0x7fcb2019f770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:03.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:03.663+0000 7fcb25800700 1 -- 172.21.15.45:0/935922996 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcb20109d30 msgr2=0x7fcb201a5be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:03.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:03.663+0000 7fcb25800700 1 --2- 172.21.15.45:0/935922996 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcb20109d30 0x7fcb201a5be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:03.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:03.664+0000 7fcb25800700 1 -- 172.21.15.45:0/935922996 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb201a0050 con 0x7fcb20109380 2024-04-16T17:05:03.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:03.664+0000 7fcb26200700 1 --2- 172.21.15.45:0/935922996 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcb2010e460 0x7fcb2019f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:05:03.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:03.664+0000 7fcb24e00700 1 --2- 172.21.15.45:0/935922996 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcb20109d30 0x7fcb201a5be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:05:03.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:03.664+0000 7fcb25800700 1 --2- 172.21.15.45:0/935922996 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcb20109380 0x7fcb201a56a0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fcb1400e9a0 tx=0x7fcb1400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:05:03.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:03.664+0000 7fcb1e000700 1 -- 172.21.15.45:0/935922996 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb1400cbd0 con 0x7fcb20109380 2024-04-16T17:05:03.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:03.664+0000 7fcb1e000700 1 -- 172.21.15.45:0/935922996 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcb1400cd70 con 0x7fcb20109380 2024-04-16T17:05:03.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:03.664+0000 7fcb28200700 1 -- 172.21.15.45:0/935922996 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb201a0340 con 0x7fcb20109380 2024-04-16T17:05:03.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:03.664+0000 7fcb1e000700 1 -- 172.21.15.45:0/935922996 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb14018750 con 0x7fcb20109380 2024-04-16T17:05:03.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:03.664+0000 7fcb28200700 1 -- 172.21.15.45:0/935922996 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb201ac620 con 0x7fcb20109380 2024-04-16T17:05:03.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:03.666+0000 7fcb28200700 1 -- 172.21.15.45:0/935922996 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb2010a860 con 0x7fcb20109380 2024-04-16T17:05:03.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:03.668+0000 7fcb1e000700 1 -- 172.21.15.45:0/935922996 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fcb14010430 con 0x7fcb20109380 2024-04-16T17:05:03.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:03.669+0000 7fcb1e000700 1 --2- 172.21.15.45:0/935922996 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fcb080a3cd0 0x7fcb080a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:03.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:03.669+0000 7fcb1e000700 1 -- 172.21.15.45:0/935922996 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7fcb140d1040 con 0x7fcb20109380 2024-04-16T17:05:03.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:03.670+0000 7fcb24e00700 1 --2- 172.21.15.45:0/935922996 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fcb080a3cd0 0x7fcb080a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:03.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:03.671+0000 7fcb24e00700 1 --2- 172.21.15.45:0/935922996 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fcb080a3cd0 0x7fcb080a6190 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fcb0c000c00 tx=0x7fcb0c005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:05:03.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:03.672+0000 7fcb1e000700 1 -- 172.21.15.45:0/935922996 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fcb1409d560 con 0x7fcb20109380 2024-04-16T17:05:03.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:03.790+0000 7f780fc00700 1 -- 172.21.15.45:0/2011430028 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f780802dfe0 con 0x7f7808075640 2024-04-16T17:05:03.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:03.791+0000 7f7805600700 1 -- 172.21.15.45:0/2011430028 <== mon.2 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f77f409c950 con 0x7f7808075640 2024-04-16T17:05:03.793 INFO:teuthology.orchestra.run.smithi045.stdout:34359738463 2024-04-16T17:05:03.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:03.794+0000 7f780fc00700 1 -- 172.21.15.45:0/2011430028 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f77f00a3d50 msgr2=0x7f77f00a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:03.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:03.794+0000 7f780fc00700 1 --2- 172.21.15.45:0/2011430028 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f77f00a3d50 0x7f77f00a6210 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f77fc0098e0 tx=0x7f77fc005960 comp rx=0 tx=0).stop 2024-04-16T17:05:03.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:03.794+0000 7f780fc00700 1 -- 172.21.15.45:0/2011430028 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7808075640 msgr2=0x7f780819bad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:03.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:03.794+0000 7f780fc00700 1 --2- 172.21.15.45:0/2011430028 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7808075640 0x7f780819bad0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f77f400e9a0 tx=0x7f77f400ecb0 comp rx=0 tx=0).stop 2024-04-16T17:05:03.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:03.794+0000 7f780fc00700 1 -- 172.21.15.45:0/2011430028 shutdown_connections 2024-04-16T17:05:03.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:03.794+0000 7f780fc00700 1 --2- 172.21.15.45:0/2011430028 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f77f00a3d50 0x7f77f00a6210 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:03.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:03.794+0000 7f780fc00700 1 --2- 172.21.15.45:0/2011430028 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7808074d20 0x7f780819b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:03.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:03.794+0000 7f780fc00700 1 --2- 172.21.15.45:0/2011430028 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f780810f4f0 0x7f780819fe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:03.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:03.794+0000 7f780fc00700 1 --2- 172.21.15.45:0/2011430028 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7808075640 0x7f780819bad0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:03.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:03.794+0000 7f780fc00700 1 -- 172.21.15.45:0/2011430028 >> 172.21.15.45:0/2011430028 conn(0x7f7808079ca0 msgr2=0x7f78080ffd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:05:03.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:03.795+0000 7f780fc00700 1 -- 172.21.15.45:0/2011430028 shutdown_connections 2024-04-16T17:05:03.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:03.795+0000 7f780fc00700 1 -- 172.21.15.45:0/2011430028 wait complete. 2024-04-16T17:05:03.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:03.922+0000 7fcb28200700 1 -- 172.21.15.45:0/935922996 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7fcb201a0ca0 con 0x7fcb20109380 2024-04-16T17:05:03.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:03.923+0000 7fcb1e000700 1 -- 172.21.15.45:0/935922996 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fcb1409ce60 con 0x7fcb20109380 2024-04-16T17:05:03.924 INFO:teuthology.orchestra.run.smithi045.stdout:111669149775 2024-04-16T17:05:03.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:03.926+0000 7fcb28200700 1 -- 172.21.15.45:0/935922996 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fcb080a3cd0 msgr2=0x7fcb080a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:03.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:03.926+0000 7fcb28200700 1 --2- 172.21.15.45:0/935922996 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fcb080a3cd0 0x7fcb080a6190 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fcb0c000c00 tx=0x7fcb0c005fb0 comp rx=0 tx=0).stop 2024-04-16T17:05:03.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:03.926+0000 7fcb28200700 1 -- 172.21.15.45:0/935922996 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcb20109380 msgr2=0x7fcb201a56a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:03.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:03.926+0000 7fcb28200700 1 --2- 172.21.15.45:0/935922996 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcb20109380 0x7fcb201a56a0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fcb1400e9a0 tx=0x7fcb1400ecb0 comp rx=0 tx=0).stop 2024-04-16T17:05:03.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:03.926+0000 7fcb28200700 1 -- 172.21.15.45:0/935922996 shutdown_connections 2024-04-16T17:05:03.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:03.926+0000 7fcb28200700 1 --2- 172.21.15.45:0/935922996 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fcb080a3cd0 0x7fcb080a6190 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:03.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:03.926+0000 7fcb28200700 1 --2- 172.21.15.45:0/935922996 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcb2010e460 0x7fcb2019f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:03.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:03.926+0000 7fcb28200700 1 --2- 172.21.15.45:0/935922996 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcb20109380 0x7fcb201a56a0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:03.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:03.926+0000 7fcb28200700 1 --2- 172.21.15.45:0/935922996 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcb20109d30 0x7fcb201a5be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:03.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:03.926+0000 7fcb28200700 1 -- 172.21.15.45:0/935922996 >> 172.21.15.45:0/935922996 conn(0x7fcb200762c0 msgr2=0x7fcb20076840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:05:03.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:03.926+0000 7fcb28200700 1 -- 172.21.15.45:0/935922996 shutdown_connections 2024-04-16T17:05:03.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:03.926+0000 7fcb28200700 1 -- 172.21.15.45:0/935922996 wait complete. 2024-04-16T17:05:04.178 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:05:03 smithi104 ceph-mon[36457]: pgmap v373: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-16T17:05:04.178 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:05:03 smithi104 ceph-mon[36457]: from='client.? 172.21.15.45:0/2011430028' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-16T17:05:04.228 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:05:03 smithi045 ceph-mon[31676]: pgmap v373: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-16T17:05:04.228 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:05:03 smithi045 ceph-mon[31676]: from='client.? 172.21.15.45:0/2011430028' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-16T17:05:04.305 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:05:03 smithi117 ceph-mon[36730]: pgmap v373: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-16T17:05:04.305 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:05:03 smithi117 ceph-mon[36730]: from='client.? 172.21.15.45:0/2011430028' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-16T17:05:05.178 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:05:04 smithi104 ceph-mon[36457]: from='client.? 172.21.15.45:0/935922996' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-16T17:05:05.178 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:05:04 smithi104 ceph-mon[36457]: pgmap v374: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T17:05:05.206 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:05:04 smithi117 ceph-mon[36730]: from='client.? 172.21.15.45:0/935922996' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-16T17:05:05.206 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:05:04 smithi117 ceph-mon[36730]: pgmap v374: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T17:05:05.228 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:05:04 smithi045 ceph-mon[31676]: from='client.? 172.21.15.45:0/935922996' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-16T17:05:05.228 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:05:04 smithi045 ceph-mon[31676]: pgmap v374: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T17:05:05.320 INFO:tasks.cephadm.ceph_manager.ceph:need seq 111669149770 got 111669149775 for osd.3 2024-04-16T17:05:05.320 DEBUG:teuthology.parallel:result is None 2024-04-16T17:05:07.410 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:05:07 smithi045 ceph-mon[31676]: pgmap v375: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-16T17:05:07.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:05:07 smithi117 ceph-mon[36730]: pgmap v375: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-16T17:05:07.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:05:07 smithi104 ceph-mon[36457]: pgmap v375: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-16T17:05:07.963 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738458 got 34359738463 for osd.0 2024-04-16T17:05:07.963 DEBUG:teuthology.parallel:result is None 2024-04-16T17:05:08.043 INFO:tasks.cephadm.ceph_manager.ceph:need seq 292057776168 got 292057776172 for osd.9 2024-04-16T17:05:08.043 DEBUG:teuthology.parallel:result is None 2024-04-16T17:05:08.937 INFO:tasks.cephadm.ceph_manager.ceph:need seq 347892351002 got 347892351008 for osd.11 2024-04-16T17:05:08.937 DEBUG:teuthology.parallel:result is None 2024-04-16T17:05:09.479 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:05:09 smithi045 ceph-mon[31676]: pgmap v376: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-16T17:05:09.555 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:05:09 smithi117 ceph-mon[36730]: pgmap v376: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-16T17:05:09.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:05:09 smithi104 ceph-mon[36457]: pgmap v376: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-16T17:05:11.478 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:05:11 smithi045 ceph-mon[31676]: pgmap v377: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-16T17:05:11.478 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:05:11 smithi045 ceph-mon[31676]: from='client.? 172.21.15.45:0/652769763' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-16T17:05:11.555 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:05:11 smithi117 ceph-mon[36730]: pgmap v377: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-16T17:05:11.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:05:11 smithi117 ceph-mon[36730]: from='client.? 172.21.15.45:0/652769763' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-16T17:05:11.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:05:11 smithi104 ceph-mon[36457]: pgmap v377: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-16T17:05:11.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:05:11 smithi104 ceph-mon[36457]: from='client.? 172.21.15.45:0/652769763' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-16T17:05:12.947 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:12.941+0000 7fcadd000700 1 -- 172.21.15.45:0/4108307631 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcad806a800 msgr2=0x7fcad806ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:12.947 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:12.941+0000 7fcadd000700 1 --2- 172.21.15.45:0/4108307631 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcad806a800 0x7fcad806ac60 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fcac40098f0 tx=0x7fcac4009c00 comp rx=0 tx=0).stop 2024-04-16T17:05:12.947 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:12.941+0000 7fcadd000700 1 -- 172.21.15.45:0/4108307631 shutdown_connections 2024-04-16T17:05:12.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:12.941+0000 7fcadd000700 1 --2- 172.21.15.45:0/4108307631 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcad806b1a0 0x7fcad810f9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:12.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:12.941+0000 7fcadd000700 1 --2- 172.21.15.45:0/4108307631 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcad8108b70 0x7fcad8108f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:12.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:12.941+0000 7fcadd000700 1 --2- 172.21.15.45:0/4108307631 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcad806a800 0x7fcad806ac60 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:12.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:12.942+0000 7fcadd000700 1 -- 172.21.15.45:0/4108307631 >> 172.21.15.45:0/4108307631 conn(0x7fcad80789a0 msgr2=0x7fcad8078db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:05:12.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:12.942+0000 7fcadd000700 1 -- 172.21.15.45:0/4108307631 shutdown_connections 2024-04-16T17:05:12.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:12.942+0000 7fcadd000700 1 -- 172.21.15.45:0/4108307631 wait complete. 2024-04-16T17:05:12.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:12.943+0000 7fcadd000700 1 Processor -- start 2024-04-16T17:05:12.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:12.943+0000 7fcadd000700 1 -- start start 2024-04-16T17:05:12.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:12.943+0000 7fcadd000700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcad806a800 0x7fcad819b570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:12.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:12.944+0000 7fcadd000700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcad806b1a0 0x7fcad819bab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:12.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:12.944+0000 7fcadd000700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcad8108b70 0x7fcad819fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:12.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:12.944+0000 7fcadd000700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fcad8112e60 con 0x7fcad806a800 2024-04-16T17:05:12.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:12.944+0000 7fcadd000700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fcad8112ce0 con 0x7fcad8108b70 2024-04-16T17:05:12.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:12.944+0000 7fcadd000700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fcad8112fe0 con 0x7fcad806b1a0 2024-04-16T17:05:12.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:12.944+0000 7fcad5e00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcad806a800 0x7fcad819b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:12.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:12.944+0000 7fcad5e00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcad806a800 0x7fcad819b570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55576/0 (socket says 172.21.15.45:55576) 2024-04-16T17:05:12.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:12.944+0000 7fcad5e00700 1 -- 172.21.15.45:0/4099184272 learned_addr learned my addr 172.21.15.45:0/4099184272 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:05:12.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:12.944+0000 7fcad5400700 1 --2- 172.21.15.45:0/4099184272 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcad806b1a0 0x7fcad819bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:12.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:12.944+0000 7fcad6800700 1 --2- 172.21.15.45:0/4099184272 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcad8108b70 0x7fcad819fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:12.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:12.944+0000 7fcad5e00700 1 -- 172.21.15.45:0/4099184272 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcad806b1a0 msgr2=0x7fcad819bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:12.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:12.944+0000 7fcad5e00700 1 --2- 172.21.15.45:0/4099184272 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcad806b1a0 0x7fcad819bab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:12.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:12.944+0000 7fcad5e00700 1 -- 172.21.15.45:0/4099184272 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcad8108b70 msgr2=0x7fcad819fdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:12.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:12.944+0000 7fcad5e00700 1 --2- 172.21.15.45:0/4099184272 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcad8108b70 0x7fcad819fdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:12.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:12.944+0000 7fcad5e00700 1 -- 172.21.15.45:0/4099184272 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcad81a0530 con 0x7fcad806a800 2024-04-16T17:05:12.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:12.945+0000 7fcad5e00700 1 --2- 172.21.15.45:0/4099184272 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcad806a800 0x7fcad819b570 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fcac800ec10 tx=0x7fcac800ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:05:12.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:12.945+0000 7fcacea00700 1 -- 172.21.15.45:0/4099184272 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcac800cc90 con 0x7fcad806a800 2024-04-16T17:05:12.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:12.945+0000 7fcacea00700 1 -- 172.21.15.45:0/4099184272 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcac800ce30 con 0x7fcad806a800 2024-04-16T17:05:12.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:12.945+0000 7fcacea00700 1 -- 172.21.15.45:0/4099184272 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcac8010690 con 0x7fcad806a800 2024-04-16T17:05:12.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:12.945+0000 7fcadd000700 1 -- 172.21.15.45:0/4099184272 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcad81a0820 con 0x7fcad806a800 2024-04-16T17:05:12.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:12.945+0000 7fcadd000700 1 -- 172.21.15.45:0/4099184272 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcad81a80a0 con 0x7fcad806a800 2024-04-16T17:05:12.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:12.947+0000 7fcadd000700 1 -- 172.21.15.45:0/4099184272 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcad80697e0 con 0x7fcad806a800 2024-04-16T17:05:12.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:12.949+0000 7fcacea00700 1 -- 172.21.15.45:0/4099184272 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fcac8003680 con 0x7fcad806a800 2024-04-16T17:05:12.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:12.951+0000 7fcacea00700 1 --2- 172.21.15.45:0/4099184272 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fcabc0a3d50 0x7fcabc0a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:12.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:12.951+0000 7fcacea00700 1 -- 172.21.15.45:0/4099184272 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7fcac80d11b0 con 0x7fcad806a800 2024-04-16T17:05:12.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:12.951+0000 7fcad5400700 1 --2- 172.21.15.45:0/4099184272 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fcabc0a3d50 0x7fcabc0a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:12.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:12.952+0000 7fcad5400700 1 --2- 172.21.15.45:0/4099184272 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fcabc0a3d50 0x7fcabc0a6210 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fcac4009810 tx=0x7fcac4019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:05:12.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:12.953+0000 7fcacea00700 1 -- 172.21.15.45:0/4099184272 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fcac801d060 con 0x7fcad806a800 2024-04-16T17:05:13.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:10.781+0000 7faf4f200700 1 -- 172.21.15.45:0/2693456059 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faf48108b70 msgr2=0x7faf48108f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:13.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:10.781+0000 7faf4f200700 1 --2- 172.21.15.45:0/2693456059 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faf48108b70 0x7faf48108f50 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7faf38009990 tx=0x7faf38009ca0 comp rx=0 tx=0).stop 2024-04-16T17:05:13.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:10.781+0000 7faf4f200700 1 -- 172.21.15.45:0/2693456059 shutdown_connections 2024-04-16T17:05:13.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:10.781+0000 7faf4f200700 1 --2- 172.21.15.45:0/2693456059 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faf4806b1a0 0x7faf4810f9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:13.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:10.781+0000 7faf4f200700 1 --2- 172.21.15.45:0/2693456059 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faf48108b70 0x7faf48108f50 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:13.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:10.781+0000 7faf4f200700 1 --2- 172.21.15.45:0/2693456059 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7faf4806a800 0x7faf4806ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:13.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:10.781+0000 7faf4f200700 1 -- 172.21.15.45:0/2693456059 >> 172.21.15.45:0/2693456059 conn(0x7faf480789a0 msgr2=0x7faf48078db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:05:13.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:10.782+0000 7faf4f200700 1 -- 172.21.15.45:0/2693456059 shutdown_connections 2024-04-16T17:05:13.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:10.782+0000 7faf4f200700 1 -- 172.21.15.45:0/2693456059 wait complete. 2024-04-16T17:05:13.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:10.784+0000 7faf4f200700 1 Processor -- start 2024-04-16T17:05:13.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:10.784+0000 7faf4f200700 1 -- start start 2024-04-16T17:05:13.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:10.784+0000 7faf4f200700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faf4806a800 0x7faf4819b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:13.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:10.785+0000 7faf4f200700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faf4806b1a0 0x7faf4819ba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:13.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:10.785+0000 7faf4f200700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7faf48108b70 0x7faf4819fde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:13.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:10.785+0000 7faf4f200700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7faf48112e00 con 0x7faf4806a800 2024-04-16T17:05:13.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:10.785+0000 7faf4f200700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7faf48112c80 con 0x7faf48108b70 2024-04-16T17:05:13.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:10.785+0000 7faf4f200700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7faf48112f80 con 0x7faf4806b1a0 2024-04-16T17:05:13.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:10.785+0000 7faf47e00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faf4806a800 0x7faf4819b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:13.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:10.785+0000 7faf47e00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faf4806a800 0x7faf4819b510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55560/0 (socket says 172.21.15.45:55560) 2024-04-16T17:05:13.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:10.785+0000 7faf47e00700 1 -- 172.21.15.45:0/652769763 learned_addr learned my addr 172.21.15.45:0/652769763 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:05:13.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:10.785+0000 7faf3f400700 1 --2- 172.21.15.45:0/652769763 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faf4806b1a0 0x7faf4819ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:13.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:10.785+0000 7faf4d200700 1 --2- 172.21.15.45:0/652769763 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7faf48108b70 0x7faf4819fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:13.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:10.785+0000 7faf47e00700 1 -- 172.21.15.45:0/652769763 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faf4806b1a0 msgr2=0x7faf4819ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:13.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:10.785+0000 7faf47e00700 1 --2- 172.21.15.45:0/652769763 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faf4806b1a0 0x7faf4819ba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:13.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:10.785+0000 7faf47e00700 1 -- 172.21.15.45:0/652769763 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7faf48108b70 msgr2=0x7faf4819fde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:13.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:10.785+0000 7faf47e00700 1 --2- 172.21.15.45:0/652769763 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7faf48108b70 0x7faf4819fde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:13.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:10.785+0000 7faf47e00700 1 -- 172.21.15.45:0/652769763 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf481a0520 con 0x7faf4806a800 2024-04-16T17:05:13.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:10.785+0000 7faf4d200700 1 --2- 172.21.15.45:0/652769763 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7faf48108b70 0x7faf4819fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:05:13.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:10.786+0000 7faf47e00700 1 --2- 172.21.15.45:0/652769763 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faf4806a800 0x7faf4819b510 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7faf38005910 tx=0x7faf3800bab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:05:13.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:10.786+0000 7faf45600700 1 -- 172.21.15.45:0/652769763 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf38027070 con 0x7faf4806a800 2024-04-16T17:05:13.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:10.786+0000 7faf45600700 1 -- 172.21.15.45:0/652769763 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faf38003d70 con 0x7faf4806a800 2024-04-16T17:05:13.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:10.786+0000 7faf45600700 1 -- 172.21.15.45:0/652769763 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf3800f460 con 0x7faf4806a800 2024-04-16T17:05:13.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:10.786+0000 7faf4f200700 1 -- 172.21.15.45:0/652769763 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf481a07b0 con 0x7faf4806a800 2024-04-16T17:05:13.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:10.786+0000 7faf4f200700 1 -- 172.21.15.45:0/652769763 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf481a8090 con 0x7faf4806a800 2024-04-16T17:05:13.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:10.788+0000 7faf4f200700 1 -- 172.21.15.45:0/652769763 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf480697e0 con 0x7faf4806a800 2024-04-16T17:05:13.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:10.789+0000 7faf45600700 1 -- 172.21.15.45:0/652769763 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7faf3802a070 con 0x7faf4806a800 2024-04-16T17:05:13.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:10.791+0000 7faf45600700 1 --2- 172.21.15.45:0/652769763 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7faf2c0a3d50 0x7faf2c0a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:13.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:10.791+0000 7faf45600700 1 -- 172.21.15.45:0/652769763 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7faf380d17b0 con 0x7faf4806a800 2024-04-16T17:05:13.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:10.791+0000 7faf3f400700 1 --2- 172.21.15.45:0/652769763 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7faf2c0a3d50 0x7faf2c0a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:13.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:10.792+0000 7faf3f400700 1 --2- 172.21.15.45:0/652769763 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7faf2c0a3d50 0x7faf2c0a6210 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7faf30009680 tx=0x7faf30009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:05:13.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:10.796+0000 7faf45600700 1 -- 172.21.15.45:0/652769763 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7faf38013070 con 0x7faf4806a800 2024-04-16T17:05:13.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:11.040+0000 7faf4f200700 1 -- 172.21.15.45:0/652769763 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7faf4802dfe0 con 0x7faf4806a800 2024-04-16T17:05:13.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:11.040+0000 7faf45600700 1 -- 172.21.15.45:0/652769763 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7faf3809de20 con 0x7faf4806a800 2024-04-16T17:05:13.076 INFO:teuthology.orchestra.run.smithi045.stdout:77309411415 2024-04-16T17:05:13.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:11.044+0000 7faf4f200700 1 -- 172.21.15.45:0/652769763 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7faf2c0a3d50 msgr2=0x7faf2c0a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:13.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:11.044+0000 7faf4f200700 1 --2- 172.21.15.45:0/652769763 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7faf2c0a3d50 0x7faf2c0a6210 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7faf30009680 tx=0x7faf30009260 comp rx=0 tx=0).stop 2024-04-16T17:05:13.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:11.044+0000 7faf4f200700 1 -- 172.21.15.45:0/652769763 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faf4806a800 msgr2=0x7faf4819b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:13.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:11.044+0000 7faf4f200700 1 --2- 172.21.15.45:0/652769763 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faf4806a800 0x7faf4819b510 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7faf38005910 tx=0x7faf3800bab0 comp rx=0 tx=0).stop 2024-04-16T17:05:13.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:11.044+0000 7faf4f200700 1 -- 172.21.15.45:0/652769763 shutdown_connections 2024-04-16T17:05:13.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:11.044+0000 7faf4f200700 1 --2- 172.21.15.45:0/652769763 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7faf48108b70 0x7faf4819fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:13.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:11.044+0000 7faf4f200700 1 --2- 172.21.15.45:0/652769763 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faf4806a800 0x7faf4819b510 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:13.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:11.044+0000 7faf4f200700 1 --2- 172.21.15.45:0/652769763 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7faf2c0a3d50 0x7faf2c0a6210 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:13.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:11.044+0000 7faf4f200700 1 --2- 172.21.15.45:0/652769763 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faf4806b1a0 0x7faf4819ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:13.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:11.044+0000 7faf4f200700 1 -- 172.21.15.45:0/652769763 >> 172.21.15.45:0/652769763 conn(0x7faf480789a0 msgr2=0x7faf4810dfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:05:13.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:11.044+0000 7faf4f200700 1 -- 172.21.15.45:0/652769763 shutdown_connections 2024-04-16T17:05:13.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:11.044+0000 7faf4f200700 1 -- 172.21.15.45:0/652769763 wait complete. 2024-04-16T17:05:13.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:13.196+0000 7fcadd000700 1 -- 172.21.15.45:0/4099184272 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7fcad819c880 con 0x7fcad806a800 2024-04-16T17:05:13.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:13.197+0000 7fcacea00700 1 -- 172.21.15.45:0/4099184272 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fcac801f2f0 con 0x7fcad806a800 2024-04-16T17:05:13.198 INFO:teuthology.orchestra.run.smithi045.stdout:201863462977 2024-04-16T17:05:13.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:13.200+0000 7fcadd000700 1 -- 172.21.15.45:0/4099184272 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fcabc0a3d50 msgr2=0x7fcabc0a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:13.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:13.200+0000 7fcadd000700 1 --2- 172.21.15.45:0/4099184272 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fcabc0a3d50 0x7fcabc0a6210 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fcac4009810 tx=0x7fcac4019040 comp rx=0 tx=0).stop 2024-04-16T17:05:13.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:13.200+0000 7fcadd000700 1 -- 172.21.15.45:0/4099184272 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcad806a800 msgr2=0x7fcad819b570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:13.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:13.200+0000 7fcadd000700 1 --2- 172.21.15.45:0/4099184272 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcad806a800 0x7fcad819b570 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fcac800ec10 tx=0x7fcac800ef20 comp rx=0 tx=0).stop 2024-04-16T17:05:13.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:13.200+0000 7fcadd000700 1 -- 172.21.15.45:0/4099184272 shutdown_connections 2024-04-16T17:05:13.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:13.200+0000 7fcadd000700 1 --2- 172.21.15.45:0/4099184272 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcad8108b70 0x7fcad819fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:13.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:13.200+0000 7fcadd000700 1 --2- 172.21.15.45:0/4099184272 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcad806a800 0x7fcad819b570 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:13.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:13.200+0000 7fcadd000700 1 --2- 172.21.15.45:0/4099184272 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fcabc0a3d50 0x7fcabc0a6210 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:13.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:13.200+0000 7fcadd000700 1 --2- 172.21.15.45:0/4099184272 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcad806b1a0 0x7fcad819bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:13.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:13.200+0000 7fcadd000700 1 -- 172.21.15.45:0/4099184272 >> 172.21.15.45:0/4099184272 conn(0x7fcad80789a0 msgr2=0x7fcad810dff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:05:13.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:13.200+0000 7fcadd000700 1 -- 172.21.15.45:0/4099184272 shutdown_connections 2024-04-16T17:05:13.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:13.200+0000 7fcadd000700 1 -- 172.21.15.45:0/4099184272 wait complete. 2024-04-16T17:05:13.468 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:05:13 smithi045 ceph-mon[31676]: pgmap v378: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-16T17:05:13.468 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:05:13 smithi045 ceph-mon[31676]: from='client.? 172.21.15.45:0/1315475673' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-16T17:05:13.468 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:05:13 smithi045 ceph-mon[31676]: from='client.? 172.21.15.45:0/4099184272' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-16T17:05:13.480 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:05:13 smithi104 ceph-mon[36457]: pgmap v378: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-16T17:05:13.480 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:05:13 smithi104 ceph-mon[36457]: from='client.? 172.21.15.45:0/1315475673' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-16T17:05:13.480 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:05:13 smithi104 ceph-mon[36457]: from='client.? 172.21.15.45:0/4099184272' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-16T17:05:13.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:05:13 smithi117 ceph-mon[36730]: pgmap v378: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-16T17:05:13.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:05:13 smithi117 ceph-mon[36730]: from='client.? 172.21.15.45:0/1315475673' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-16T17:05:13.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:05:13 smithi117 ceph-mon[36730]: from='client.? 172.21.15.45:0/4099184272' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-16T17:05:13.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:13.639+0000 7ffbb7000700 1 -- 172.21.15.45:0/777238462 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffbb006bb60 msgr2=0x7ffbb010f9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:13.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:13.639+0000 7ffbb7000700 1 --2- 172.21.15.45:0/777238462 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffbb006bb60 0x7ffbb010f9d0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7ffba800b0a0 tx=0x7ffba800b3b0 comp rx=0 tx=0).stop 2024-04-16T17:05:13.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:13.639+0000 7ffbb7000700 1 -- 172.21.15.45:0/777238462 shutdown_connections 2024-04-16T17:05:13.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:13.639+0000 7ffbb7000700 1 --2- 172.21.15.45:0/777238462 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffbb006bb60 0x7ffbb010f9d0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:13.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:13.639+0000 7ffbb7000700 1 --2- 172.21.15.45:0/777238462 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ffbb0108b90 0x7ffbb0108f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:13.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:13.639+0000 7ffbb7000700 1 --2- 172.21.15.45:0/777238462 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffbb006b1c0 0x7ffbb006b620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:13.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:13.639+0000 7ffbb7000700 1 -- 172.21.15.45:0/777238462 >> 172.21.15.45:0/777238462 conn(0x7ffbb00789a0 msgr2=0x7ffbb0078db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:05:13.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:13.640+0000 7ffbb7000700 1 -- 172.21.15.45:0/777238462 shutdown_connections 2024-04-16T17:05:13.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:13.640+0000 7ffbb7000700 1 -- 172.21.15.45:0/777238462 wait complete. 2024-04-16T17:05:13.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:13.641+0000 7ffbb7000700 1 Processor -- start 2024-04-16T17:05:13.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:13.641+0000 7ffbb7000700 1 -- start start 2024-04-16T17:05:13.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:13.641+0000 7ffbb7000700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffbb006b1c0 0x7ffbb019b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:13.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:13.641+0000 7ffbb7000700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ffbb006bb60 0x7ffbb019b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:13.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:13.641+0000 7ffbb7000700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffbb0108b90 0x7ffbb019fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:13.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:13.641+0000 7ffbb7000700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7ffbb0112df0 con 0x7ffbb006b1c0 2024-04-16T17:05:13.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:13.641+0000 7ffbb7000700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7ffbb0112c70 con 0x7ffbb006bb60 2024-04-16T17:05:13.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:13.641+0000 7ffbb7000700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7ffbb0112f70 con 0x7ffbb0108b90 2024-04-16T17:05:13.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:13.642+0000 7ffbafe00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffbb006b1c0 0x7ffbb019b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:13.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:13.642+0000 7ffbaf400700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ffbb006bb60 0x7ffbb019b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:13.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:13.642+0000 7ffbafe00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffbb006b1c0 0x7ffbb019b4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55588/0 (socket says 172.21.15.45:55588) 2024-04-16T17:05:13.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:13.642+0000 7ffbafe00700 1 -- 172.21.15.45:0/310716817 learned_addr learned my addr 172.21.15.45:0/310716817 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:05:13.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:13.642+0000 7ffbb5000700 1 --2- 172.21.15.45:0/310716817 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffbb0108b90 0x7ffbb019fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:13.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:13.642+0000 7ffbaf400700 1 -- 172.21.15.45:0/310716817 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffbb0108b90 msgr2=0x7ffbb019fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:13.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:13.642+0000 7ffbaf400700 1 --2- 172.21.15.45:0/310716817 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffbb0108b90 0x7ffbb019fd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:13.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:13.642+0000 7ffbaf400700 1 -- 172.21.15.45:0/310716817 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffbb006b1c0 msgr2=0x7ffbb019b4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:13.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:13.642+0000 7ffbaf400700 1 --2- 172.21.15.45:0/310716817 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffbb006b1c0 0x7ffbb019b4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:13.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:13.642+0000 7ffbaf400700 1 -- 172.21.15.45:0/310716817 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffbb01a0470 con 0x7ffbb006bb60 2024-04-16T17:05:13.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:13.642+0000 7ffbb5000700 1 --2- 172.21.15.45:0/310716817 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffbb0108b90 0x7ffbb019fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:05:13.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:13.643+0000 7ffbaf400700 1 --2- 172.21.15.45:0/310716817 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ffbb006bb60 0x7ffbb019b9f0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7ffb9c00d750 tx=0x7ffb9c00da60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:05:13.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:13.643+0000 7ffbacc00700 1 -- 172.21.15.45:0/310716817 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb9c014030 con 0x7ffbb006bb60 2024-04-16T17:05:13.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:13.643+0000 7ffbb7000700 1 -- 172.21.15.45:0/310716817 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffbb01a0760 con 0x7ffbb006bb60 2024-04-16T17:05:13.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:13.643+0000 7ffbacc00700 1 -- 172.21.15.45:0/310716817 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffb9c004500 con 0x7ffbb006bb60 2024-04-16T17:05:13.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:13.643+0000 7ffbacc00700 1 -- 172.21.15.45:0/310716817 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb9c0054e0 con 0x7ffbb006bb60 2024-04-16T17:05:13.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:13.643+0000 7ffbb7000700 1 -- 172.21.15.45:0/310716817 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffbb01a7fe0 con 0x7ffbb006bb60 2024-04-16T17:05:13.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:13.644+0000 7ffbb7000700 1 -- 172.21.15.45:0/310716817 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffbb0104240 con 0x7ffbb006bb60 2024-04-16T17:05:13.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:13.646+0000 7ffbacc00700 1 -- 172.21.15.45:0/310716817 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7ffb9c004750 con 0x7ffbb006bb60 2024-04-16T17:05:13.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:13.648+0000 7ffbacc00700 1 --2- 172.21.15.45:0/310716817 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7ffb940a3d50 0x7ffb940a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:13.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:13.648+0000 7ffbacc00700 1 -- 172.21.15.45:0/310716817 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7ffb9c0d0120 con 0x7ffbb006bb60 2024-04-16T17:05:13.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:13.648+0000 7ffbafe00700 1 --2- 172.21.15.45:0/310716817 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7ffb940a3d50 0x7ffb940a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:13.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:13.649+0000 7ffbafe00700 1 --2- 172.21.15.45:0/310716817 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7ffb940a3d50 0x7ffb940a6210 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7ffba0009ce0 tx=0x7ffba0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:05:13.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:13.652+0000 7ffbacc00700 1 -- 172.21.15.45:0/310716817 <== mon.2 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ffb9c09c5e0 con 0x7ffbb006bb60 2024-04-16T17:05:13.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:13.890+0000 7ffbb7000700 1 -- 172.21.15.45:0/310716817 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7ffbb01a81b0 con 0x7ffbb006bb60 2024-04-16T17:05:13.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:13.891+0000 7ffbacc00700 1 -- 172.21.15.45:0/310716817 <== mon.2 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7ffb9c09bee0 con 0x7ffbb006bb60 2024-04-16T17:05:13.893 INFO:teuthology.orchestra.run.smithi045.stdout:223338299451 2024-04-16T17:05:13.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:12.748+0000 7fa85b200700 1 -- 172.21.15.45:0/3469135886 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa854108b70 msgr2=0x7fa854108f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:13.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:12.748+0000 7fa85b200700 1 --2- 172.21.15.45:0/3469135886 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa854108b70 0x7fa854108f50 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fa84400b0a0 tx=0x7fa84400b3b0 comp rx=0 tx=0).stop 2024-04-16T17:05:13.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:12.748+0000 7fa85b200700 1 -- 172.21.15.45:0/3469135886 shutdown_connections 2024-04-16T17:05:13.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:12.748+0000 7fa85b200700 1 --2- 172.21.15.45:0/3469135886 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa85406b1a0 0x7fa85410f9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:13.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:12.748+0000 7fa85b200700 1 --2- 172.21.15.45:0/3469135886 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa854108b70 0x7fa854108f50 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:13.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:12.748+0000 7fa85b200700 1 --2- 172.21.15.45:0/3469135886 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa85406a800 0x7fa85406ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:13.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:12.748+0000 7fa85b200700 1 -- 172.21.15.45:0/3469135886 >> 172.21.15.45:0/3469135886 conn(0x7fa8540789a0 msgr2=0x7fa854078db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:05:13.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:12.748+0000 7fa85b200700 1 -- 172.21.15.45:0/3469135886 shutdown_connections 2024-04-16T17:05:13.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:12.748+0000 7fa85b200700 1 -- 172.21.15.45:0/3469135886 wait complete. 2024-04-16T17:05:13.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:12.749+0000 7fa85b200700 1 Processor -- start 2024-04-16T17:05:13.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:12.749+0000 7fa85b200700 1 -- start start 2024-04-16T17:05:13.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:12.750+0000 7fa85b200700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa85406a800 0x7fa8541992f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:13.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:12.750+0000 7fa85b200700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa85406b1a0 0x7fa854199830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:13.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:12.750+0000 7fa85b200700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa854108b70 0x7fa85419dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:13.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:12.750+0000 7fa85b200700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fa854112e00 con 0x7fa85406b1a0 2024-04-16T17:05:13.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:12.750+0000 7fa85b200700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fa854112c80 con 0x7fa85406a800 2024-04-16T17:05:13.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:12.750+0000 7fa85b200700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa854112f80 con 0x7fa854108b70 2024-04-16T17:05:13.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:12.750+0000 7fa84b400700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa85406b1a0 0x7fa854199830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:13.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:12.750+0000 7fa84b400700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa85406b1a0 0x7fa854199830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55568/0 (socket says 172.21.15.45:55568) 2024-04-16T17:05:13.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:12.750+0000 7fa84b400700 1 -- 172.21.15.45:0/1315475673 learned_addr learned my addr 172.21.15.45:0/1315475673 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:05:13.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:12.750+0000 7fa853e00700 1 --2- 172.21.15.45:0/1315475673 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa85406a800 0x7fa8541992f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:13.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:12.750+0000 7fa859200700 1 --2- 172.21.15.45:0/1315475673 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa854108b70 0x7fa85419dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:13.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:12.750+0000 7fa853e00700 1 -- 172.21.15.45:0/1315475673 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa854108b70 msgr2=0x7fa85419dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:13.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:12.750+0000 7fa853e00700 1 --2- 172.21.15.45:0/1315475673 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa854108b70 0x7fa85419dbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:13.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:12.750+0000 7fa853e00700 1 -- 172.21.15.45:0/1315475673 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa85406b1a0 msgr2=0x7fa854199830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:13.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:12.750+0000 7fa853e00700 1 --2- 172.21.15.45:0/1315475673 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa85406b1a0 0x7fa854199830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:13.897 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:12.751+0000 7fa853e00700 1 -- 172.21.15.45:0/1315475673 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa85419e300 con 0x7fa85406a800 2024-04-16T17:05:13.897 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:12.751+0000 7fa84b400700 1 --2- 172.21.15.45:0/1315475673 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa85406b1a0 0x7fa854199830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:05:13.897 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:12.751+0000 7fa859200700 1 --2- 172.21.15.45:0/1315475673 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa854108b70 0x7fa85419dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:05:13.897 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:12.751+0000 7fa853e00700 1 --2- 172.21.15.45:0/1315475673 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa85406a800 0x7fa8541992f0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fa84400b780 tx=0x7fa844004d00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:05:13.897 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:12.751+0000 7fa851600700 1 -- 172.21.15.45:0/1315475673 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa844021070 con 0x7fa85406a800 2024-04-16T17:05:13.897 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:12.751+0000 7fa85b200700 1 -- 172.21.15.45:0/1315475673 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa85419e590 con 0x7fa85406a800 2024-04-16T17:05:13.897 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:12.751+0000 7fa851600700 1 -- 172.21.15.45:0/1315475673 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa844007b80 con 0x7fa85406a800 2024-04-16T17:05:13.897 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:12.751+0000 7fa851600700 1 -- 172.21.15.45:0/1315475673 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8440052c0 con 0x7fa85406a800 2024-04-16T17:05:13.897 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:12.751+0000 7fa85b200700 1 -- 172.21.15.45:0/1315475673 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8541a5e70 con 0x7fa85406a800 2024-04-16T17:05:13.897 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:12.752+0000 7fa85b200700 1 -- 172.21.15.45:0/1315475673 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8540697e0 con 0x7fa85406a800 2024-04-16T17:05:13.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:12.755+0000 7fa851600700 1 -- 172.21.15.45:0/1315475673 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fa844024070 con 0x7fa85406a800 2024-04-16T17:05:13.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:12.756+0000 7fa851600700 1 --2- 172.21.15.45:0/1315475673 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fa8380a3cd0 0x7fa8380a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:13.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:12.756+0000 7fa851600700 1 -- 172.21.15.45:0/1315475673 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7fa84400f020 con 0x7fa85406a800 2024-04-16T17:05:13.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:12.756+0000 7fa84b400700 1 --2- 172.21.15.45:0/1315475673 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fa8380a3cd0 0x7fa8380a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:13.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:12.757+0000 7fa84b400700 1 --2- 172.21.15.45:0/1315475673 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fa8380a3cd0 0x7fa8380a6190 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fa854078fa0 tx=0x7fa83c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:05:13.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:12.760+0000 7fa851600700 1 -- 172.21.15.45:0/1315475673 <== mon.2 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa84402b020 con 0x7fa85406a800 2024-04-16T17:05:13.899 INFO:teuthology.orchestra.run.smithi045.stdout:257698037814 2024-04-16T17:05:13.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:12.998+0000 7fa85b200700 1 -- 172.21.15.45:0/1315475673 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7fa85402dfe0 con 0x7fa85406a800 2024-04-16T17:05:13.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:12.999+0000 7fa851600700 1 -- 172.21.15.45:0/1315475673 <== mon.2 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fa84409e220 con 0x7fa85406a800 2024-04-16T17:05:13.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:13.003+0000 7fa85b200700 1 -- 172.21.15.45:0/1315475673 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fa8380a3cd0 msgr2=0x7fa8380a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:13.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:13.003+0000 7fa85b200700 1 --2- 172.21.15.45:0/1315475673 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fa8380a3cd0 0x7fa8380a6190 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fa854078fa0 tx=0x7fa83c009260 comp rx=0 tx=0).stop 2024-04-16T17:05:13.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:13.003+0000 7fa85b200700 1 -- 172.21.15.45:0/1315475673 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa85406a800 msgr2=0x7fa8541992f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:13.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:13.003+0000 7fa85b200700 1 --2- 172.21.15.45:0/1315475673 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa85406a800 0x7fa8541992f0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fa84400b780 tx=0x7fa844004d00 comp rx=0 tx=0).stop 2024-04-16T17:05:13.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:13.003+0000 7fa85b200700 1 -- 172.21.15.45:0/1315475673 shutdown_connections 2024-04-16T17:05:13.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:13.003+0000 7fa85b200700 1 --2- 172.21.15.45:0/1315475673 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fa8380a3cd0 0x7fa8380a6190 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:13.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:13.003+0000 7fa85b200700 1 --2- 172.21.15.45:0/1315475673 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa854108b70 0x7fa85419dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:13.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:13.003+0000 7fa85b200700 1 --2- 172.21.15.45:0/1315475673 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa85406a800 0x7fa8541992f0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:13.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:13.003+0000 7fa85b200700 1 --2- 172.21.15.45:0/1315475673 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa85406b1a0 0x7fa854199830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:13.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:13.003+0000 7fa85b200700 1 -- 172.21.15.45:0/1315475673 >> 172.21.15.45:0/1315475673 conn(0x7fa8540789a0 msgr2=0x7fa8540ffaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:05:13.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:13.003+0000 7fa85b200700 1 -- 172.21.15.45:0/1315475673 shutdown_connections 2024-04-16T17:05:13.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:13.003+0000 7fa85b200700 1 -- 172.21.15.45:0/1315475673 wait complete. 2024-04-16T17:05:13.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:13.895+0000 7ffbb7000700 1 -- 172.21.15.45:0/310716817 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7ffb940a3d50 msgr2=0x7ffb940a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:13.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:13.895+0000 7ffbb7000700 1 --2- 172.21.15.45:0/310716817 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7ffb940a3d50 0x7ffb940a6210 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7ffba0009ce0 tx=0x7ffba0009260 comp rx=0 tx=0).stop 2024-04-16T17:05:13.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:13.895+0000 7ffbb7000700 1 -- 172.21.15.45:0/310716817 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ffbb006bb60 msgr2=0x7ffbb019b9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:13.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:13.895+0000 7ffbb7000700 1 --2- 172.21.15.45:0/310716817 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ffbb006bb60 0x7ffbb019b9f0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7ffb9c00d750 tx=0x7ffb9c00da60 comp rx=0 tx=0).stop 2024-04-16T17:05:13.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:13.895+0000 7ffbb7000700 1 -- 172.21.15.45:0/310716817 shutdown_connections 2024-04-16T17:05:13.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:13.895+0000 7ffbb7000700 1 --2- 172.21.15.45:0/310716817 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7ffb940a3d50 0x7ffb940a6210 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:13.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:13.895+0000 7ffbb7000700 1 --2- 172.21.15.45:0/310716817 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffbb0108b90 0x7ffbb019fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:13.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:13.895+0000 7ffbb7000700 1 --2- 172.21.15.45:0/310716817 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffbb006b1c0 0x7ffbb019b4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:13.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:13.895+0000 7ffbb7000700 1 --2- 172.21.15.45:0/310716817 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ffbb006bb60 0x7ffbb019b9f0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:13.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:13.895+0000 7ffbb7000700 1 -- 172.21.15.45:0/310716817 >> 172.21.15.45:0/310716817 conn(0x7ffbb00789a0 msgr2=0x7ffbb0100760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:05:13.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:13.895+0000 7ffbb7000700 1 -- 172.21.15.45:0/310716817 shutdown_connections 2024-04-16T17:05:13.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:13.895+0000 7ffbb7000700 1 -- 172.21.15.45:0/310716817 wait complete. 2024-04-16T17:05:14.478 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:05:14 smithi045 ceph-mon[31676]: from='client.? 172.21.15.45:0/310716817' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-16T17:05:14.555 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:05:14 smithi117 ceph-mon[36730]: from='client.? 172.21.15.45:0/310716817' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-16T17:05:14.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:05:14 smithi104 ceph-mon[36457]: from='client.? 172.21.15.45:0/310716817' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-16T17:05:15.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:05:15 smithi117 ceph-mon[36730]: pgmap v379: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T17:05:15.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:05:15 smithi117 ceph-mon[36730]: from='client.? 172.21.15.45:0/4155154185' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-16T17:05:15.646 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:05:15 smithi045 ceph-mon[31676]: pgmap v379: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T17:05:15.646 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:05:15 smithi045 ceph-mon[31676]: from='client.? 172.21.15.45:0/4155154185' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-16T17:05:15.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:05:15 smithi104 ceph-mon[36457]: pgmap v379: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T17:05:15.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:05:15 smithi104 ceph-mon[36457]: from='client.? 172.21.15.45:0/4155154185' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-16T17:05:15.707 INFO:tasks.cephadm.ceph_manager.ceph:need seq 201863462970 got 201863462977 for osd.6 2024-04-16T17:05:15.708 DEBUG:teuthology.parallel:result is None 2024-04-16T17:05:15.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:13.988+0000 7fdaf7e00700 1 -- 172.21.15.45:0/472568734 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdaf8104e80 msgr2=0x7fdaf8105260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:15.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:13.988+0000 7fdaf7e00700 1 --2- 172.21.15.45:0/472568734 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdaf8104e80 0x7fdaf8105260 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fdae80099c0 tx=0x7fdae8009cd0 comp rx=0 tx=0).stop 2024-04-16T17:05:15.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:13.988+0000 7fdaf7e00700 1 -- 172.21.15.45:0/472568734 shutdown_connections 2024-04-16T17:05:15.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:13.988+0000 7fdaf7e00700 1 --2- 172.21.15.45:0/472568734 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdaf8109f60 0x7fdaf8111b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:15.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:13.988+0000 7fdaf7e00700 1 --2- 172.21.15.45:0/472568734 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdaf8104e80 0x7fdaf8105260 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:15.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:13.988+0000 7fdaf7e00700 1 --2- 172.21.15.45:0/472568734 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdaf8105830 0x7fdaf8109880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:15.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:13.988+0000 7fdaf7e00700 1 -- 172.21.15.45:0/472568734 >> 172.21.15.45:0/472568734 conn(0x7fdaf8100710 msgr2=0x7fdaf8102b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:05:15.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:13.989+0000 7fdaf7e00700 1 -- 172.21.15.45:0/472568734 shutdown_connections 2024-04-16T17:05:15.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:13.989+0000 7fdaf7e00700 1 -- 172.21.15.45:0/472568734 wait complete. 2024-04-16T17:05:15.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:13.990+0000 7fdaf7e00700 1 Processor -- start 2024-04-16T17:05:15.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:13.990+0000 7fdaf7e00700 1 -- start start 2024-04-16T17:05:15.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:13.990+0000 7fdaf7e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdaf8104e80 0x7fdaf819b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:15.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:13.990+0000 7fdaf7e00700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdaf8105830 0x7fdaf819b890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:15.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:13.990+0000 7fdaf7e00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdaf8109f60 0x7fdaf819fc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:15.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:13.990+0000 7fdaf7e00700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fdaf8114f40 con 0x7fdaf8109f60 2024-04-16T17:05:15.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:13.990+0000 7fdaf7e00700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fdaf8114dc0 con 0x7fdaf8105830 2024-04-16T17:05:15.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:13.990+0000 7fdaf7e00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fdaf81150c0 con 0x7fdaf8104e80 2024-04-16T17:05:15.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:13.990+0000 7fdaf6a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdaf8104e80 0x7fdaf819b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:15.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:13.990+0000 7fdaf7400700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdaf8109f60 0x7fdaf819fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:15.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:13.991+0000 7fdaefe00700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdaf8105830 0x7fdaf819b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:15.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:13.991+0000 7fdaf6a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdaf8104e80 0x7fdaf819b350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:41870/0 (socket says 172.21.15.45:41870) 2024-04-16T17:05:15.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:13.991+0000 7fdaefe00700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdaf8105830 0x7fdaf819b890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:47806/0 (socket says 172.21.15.45:47806) 2024-04-16T17:05:15.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:13.991+0000 7fdaf6a00700 1 -- 172.21.15.45:0/4155154185 learned_addr learned my addr 172.21.15.45:0/4155154185 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:05:15.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:13.991+0000 7fdaefe00700 1 -- 172.21.15.45:0/4155154185 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdaf8104e80 msgr2=0x7fdaf819b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:15.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:13.991+0000 7fdaefe00700 1 --2- 172.21.15.45:0/4155154185 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdaf8104e80 0x7fdaf819b350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:15.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:13.991+0000 7fdaefe00700 1 -- 172.21.15.45:0/4155154185 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdaf8109f60 msgr2=0x7fdaf819fc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:15.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:13.991+0000 7fdaefe00700 1 --2- 172.21.15.45:0/4155154185 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdaf8109f60 0x7fdaf819fc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:15.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:13.991+0000 7fdaefe00700 1 -- 172.21.15.45:0/4155154185 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdaf81a0360 con 0x7fdaf8105830 2024-04-16T17:05:15.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:13.991+0000 7fdaf6a00700 1 --2- 172.21.15.45:0/4155154185 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdaf8104e80 0x7fdaf819b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:05:15.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:13.991+0000 7fdaefe00700 1 --2- 172.21.15.45:0/4155154185 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdaf8105830 0x7fdaf819b890 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fdae000e9c0 tx=0x7fdae000ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:05:15.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:13.992+0000 7fdafca00700 1 -- 172.21.15.45:0/4155154185 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdae000c9c0 con 0x7fdaf8105830 2024-04-16T17:05:15.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:13.992+0000 7fdaf7e00700 1 -- 172.21.15.45:0/4155154185 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdaf81a0650 con 0x7fdaf8105830 2024-04-16T17:05:15.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:13.992+0000 7fdafca00700 1 -- 172.21.15.45:0/4155154185 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdae000cb60 con 0x7fdaf8105830 2024-04-16T17:05:15.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:13.992+0000 7fdafca00700 1 -- 172.21.15.45:0/4155154185 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdae0018680 con 0x7fdaf8105830 2024-04-16T17:05:15.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:13.992+0000 7fdaf7e00700 1 -- 172.21.15.45:0/4155154185 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdaf81a7ed0 con 0x7fdaf8105830 2024-04-16T17:05:15.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:13.993+0000 7fdaf7e00700 1 -- 172.21.15.45:0/4155154185 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdaf80697e0 con 0x7fdaf8105830 2024-04-16T17:05:15.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:13.995+0000 7fdafca00700 1 -- 172.21.15.45:0/4155154185 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fdae0010430 con 0x7fdaf8105830 2024-04-16T17:05:15.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:13.996+0000 7fdafca00700 1 --2- 172.21.15.45:0/4155154185 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fdadc0a3cd0 0x7fdadc0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:15.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:13.997+0000 7fdafca00700 1 -- 172.21.15.45:0/4155154185 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7fdae00d0c20 con 0x7fdaf8105830 2024-04-16T17:05:15.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:13.997+0000 7fdaf6a00700 1 --2- 172.21.15.45:0/4155154185 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fdadc0a3cd0 0x7fdadc0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:15.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:13.997+0000 7fdaf6a00700 1 --2- 172.21.15.45:0/4155154185 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fdadc0a3cd0 0x7fdadc0a6190 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fdae8009990 tx=0x7fdae8005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:05:15.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:13.999+0000 7fdafca00700 1 -- 172.21.15.45:0/4155154185 <== mon.2 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fdae009d170 con 0x7fdaf8105830 2024-04-16T17:05:15.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:14.231+0000 7fdaf7e00700 1 -- 172.21.15.45:0/4155154185 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7fdaf802dfe0 con 0x7fdaf8105830 2024-04-16T17:05:15.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:14.231+0000 7fdafca00700 1 -- 172.21.15.45:0/4155154185 <== mon.2 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fdae009ca70 con 0x7fdaf8105830 2024-04-16T17:05:15.759 INFO:teuthology.orchestra.run.smithi045.stdout:55834574942 2024-04-16T17:05:15.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:14.234+0000 7fdaf7e00700 1 -- 172.21.15.45:0/4155154185 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fdadc0a3cd0 msgr2=0x7fdadc0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:15.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:14.234+0000 7fdaf7e00700 1 --2- 172.21.15.45:0/4155154185 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fdadc0a3cd0 0x7fdadc0a6190 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fdae8009990 tx=0x7fdae8005960 comp rx=0 tx=0).stop 2024-04-16T17:05:15.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:14.234+0000 7fdaf7e00700 1 -- 172.21.15.45:0/4155154185 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdaf8105830 msgr2=0x7fdaf819b890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:15.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:14.234+0000 7fdaf7e00700 1 --2- 172.21.15.45:0/4155154185 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdaf8105830 0x7fdaf819b890 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fdae000e9c0 tx=0x7fdae000ecd0 comp rx=0 tx=0).stop 2024-04-16T17:05:15.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:14.235+0000 7fdaf7e00700 1 -- 172.21.15.45:0/4155154185 shutdown_connections 2024-04-16T17:05:15.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:14.235+0000 7fdaf7e00700 1 --2- 172.21.15.45:0/4155154185 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fdadc0a3cd0 0x7fdadc0a6190 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:15.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:14.235+0000 7fdaf7e00700 1 --2- 172.21.15.45:0/4155154185 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdaf8104e80 0x7fdaf819b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:15.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:14.235+0000 7fdaf7e00700 1 --2- 172.21.15.45:0/4155154185 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdaf8109f60 0x7fdaf819fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:15.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:14.235+0000 7fdaf7e00700 1 --2- 172.21.15.45:0/4155154185 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdaf8105830 0x7fdaf819b890 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:15.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:14.235+0000 7fdaf7e00700 1 -- 172.21.15.45:0/4155154185 >> 172.21.15.45:0/4155154185 conn(0x7fdaf8100710 msgr2=0x7fdaf8101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:05:15.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:14.235+0000 7fdaf7e00700 1 -- 172.21.15.45:0/4155154185 shutdown_connections 2024-04-16T17:05:15.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:14.235+0000 7fdaf7e00700 1 -- 172.21.15.45:0/4155154185 wait complete. 2024-04-16T17:05:16.631 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574933 got 55834574942 for osd.1 2024-04-16T17:05:16.631 DEBUG:teuthology.parallel:result is None 2024-04-16T17:05:16.895 INFO:tasks.cephadm.ceph_manager.ceph:need seq 223338299444 got 223338299451 for osd.7 2024-04-16T17:05:16.895 DEBUG:teuthology.parallel:result is None 2024-04-16T17:05:17.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:17.104+0000 7f5ece000700 1 -- 172.21.15.45:0/893932757 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5ec8102a20 msgr2=0x7f5ec8102ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:17.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:17.104+0000 7f5ece000700 1 --2- 172.21.15.45:0/893932757 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5ec8102a20 0x7f5ec8102ea0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f5eb40098f0 tx=0x7f5eb4009c00 comp rx=0 tx=0).stop 2024-04-16T17:05:17.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:17.105+0000 7f5ece000700 1 -- 172.21.15.45:0/893932757 shutdown_connections 2024-04-16T17:05:17.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:17.105+0000 7f5ece000700 1 --2- 172.21.15.45:0/893932757 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5ec8108a20 0x7f5ec8108e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:17.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:17.105+0000 7f5ece000700 1 --2- 172.21.15.45:0/893932757 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5ec81033e0 0x7f5ec810f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:17.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:17.105+0000 7f5ece000700 1 --2- 172.21.15.45:0/893932757 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5ec8102a20 0x7f5ec8102ea0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:17.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:17.105+0000 7f5ece000700 1 -- 172.21.15.45:0/893932757 >> 172.21.15.45:0/893932757 conn(0x7f5ec80fe570 msgr2=0x7f5ec8100990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:05:17.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:17.105+0000 7f5ece000700 1 -- 172.21.15.45:0/893932757 shutdown_connections 2024-04-16T17:05:17.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:17.105+0000 7f5ece000700 1 -- 172.21.15.45:0/893932757 wait complete. 2024-04-16T17:05:17.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:17.106+0000 7f5ece000700 1 Processor -- start 2024-04-16T17:05:17.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:17.106+0000 7f5ece000700 1 -- start start 2024-04-16T17:05:17.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:17.106+0000 7f5ece000700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5ec8102a20 0x7f5ec819b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:17.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:17.107+0000 7f5ece000700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5ec81033e0 0x7f5ec819b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:17.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:17.107+0000 7f5ece000700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5ec8108a20 0x7f5ec819fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:17.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:17.107+0000 7f5ece000700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f5ec8112d60 con 0x7f5ec8102a20 2024-04-16T17:05:17.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:17.107+0000 7f5ece000700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f5ec8112be0 con 0x7f5ec81033e0 2024-04-16T17:05:17.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:17.107+0000 7f5ece000700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f5ec8112ee0 con 0x7f5ec8108a20 2024-04-16T17:05:17.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:17.107+0000 7f5eccc00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5ec8102a20 0x7f5ec819b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:17.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:17.107+0000 7f5ec7e00700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5ec81033e0 0x7f5ec819b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:17.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:17.107+0000 7f5ec7e00700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5ec81033e0 0x7f5ec819b910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:47818/0 (socket says 172.21.15.45:47818) 2024-04-16T17:05:17.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:17.107+0000 7f5ec7e00700 1 -- 172.21.15.45:0/1432069712 learned_addr learned my addr 172.21.15.45:0/1432069712 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:05:17.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:17.107+0000 7f5eccc00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5ec8102a20 0x7f5ec819b3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55638/0 (socket says 172.21.15.45:55638) 2024-04-16T17:05:17.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:17.107+0000 7f5ecd600700 1 --2- 172.21.15.45:0/1432069712 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5ec8108a20 0x7f5ec819fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:17.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:17.108+0000 7f5ecd600700 1 -- 172.21.15.45:0/1432069712 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5ec81033e0 msgr2=0x7f5ec819b910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:17.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:17.108+0000 7f5ecd600700 1 --2- 172.21.15.45:0/1432069712 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5ec81033e0 0x7f5ec819b910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:17.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:17.108+0000 7f5ecd600700 1 -- 172.21.15.45:0/1432069712 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5ec8102a20 msgr2=0x7f5ec819b3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:17.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:17.108+0000 7f5ecd600700 1 --2- 172.21.15.45:0/1432069712 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5ec8102a20 0x7f5ec819b3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:17.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:17.108+0000 7f5ecd600700 1 -- 172.21.15.45:0/1432069712 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ec81a03e0 con 0x7f5ec8108a20 2024-04-16T17:05:17.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:17.108+0000 7f5ec7e00700 1 --2- 172.21.15.45:0/1432069712 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5ec81033e0 0x7f5ec819b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:05:17.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:17.108+0000 7f5ecd600700 1 --2- 172.21.15.45:0/1432069712 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5ec8108a20 0x7f5ec819fca0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f5ec000efc0 tx=0x7f5ec000c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:05:17.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:17.108+0000 7f5ec5600700 1 -- 172.21.15.45:0/1432069712 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ec00092e0 con 0x7f5ec8108a20 2024-04-16T17:05:17.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:17.109+0000 7f5ec5600700 1 -- 172.21.15.45:0/1432069712 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5ec0004bb0 con 0x7f5ec8108a20 2024-04-16T17:05:17.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:17.109+0000 7f5ece000700 1 -- 172.21.15.45:0/1432069712 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ec81a0670 con 0x7f5ec8108a20 2024-04-16T17:05:17.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:17.109+0000 7f5ece000700 1 -- 172.21.15.45:0/1432069712 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ec81a7f50 con 0x7f5ec8108a20 2024-04-16T17:05:17.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:17.109+0000 7f5ec5600700 1 -- 172.21.15.45:0/1432069712 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ec0004280 con 0x7f5ec8108a20 2024-04-16T17:05:17.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:17.110+0000 7f5ece000700 1 -- 172.21.15.45:0/1432069712 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ea8005330 con 0x7f5ec8108a20 2024-04-16T17:05:17.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:17.112+0000 7f5ec5600700 1 -- 172.21.15.45:0/1432069712 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f5ec001b030 con 0x7f5ec8108a20 2024-04-16T17:05:17.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:17.113+0000 7f5ec5600700 1 --2- 172.21.15.45:0/1432069712 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f5eb00a3d50 0x7f5eb00a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:17.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:17.113+0000 7f5ec5600700 1 -- 172.21.15.45:0/1432069712 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f5ec00d0210 con 0x7f5ec8108a20 2024-04-16T17:05:17.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:17.114+0000 7f5eccc00700 1 --2- 172.21.15.45:0/1432069712 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f5eb00a3d50 0x7f5eb00a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:17.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:17.114+0000 7f5eccc00700 1 --2- 172.21.15.45:0/1432069712 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f5eb00a3d50 0x7f5eb00a6210 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f5ebc005fd0 tx=0x7f5ebc009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:05:17.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:17.118+0000 7f5ec5600700 1 -- 172.21.15.45:0/1432069712 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5ec0022020 con 0x7f5ec8108a20 2024-04-16T17:05:17.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:17.357+0000 7f5ece000700 1 -- 172.21.15.45:0/1432069712 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7f5ea80058f0 con 0x7f5ec8108a20 2024-04-16T17:05:17.359 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:05:17 smithi045 ceph-mon[31676]: pgmap v380: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-16T17:05:17.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:17.358+0000 7f5ec5600700 1 -- 172.21.15.45:0/1432069712 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f5ec0010040 con 0x7f5ec8108a20 2024-04-16T17:05:17.360 INFO:teuthology.orchestra.run.smithi045.stdout:313532612651 2024-04-16T17:05:17.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:17.360+0000 7f5ece000700 1 -- 172.21.15.45:0/1432069712 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f5eb00a3d50 msgr2=0x7f5eb00a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:17.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:17.360+0000 7f5ece000700 1 --2- 172.21.15.45:0/1432069712 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f5eb00a3d50 0x7f5eb00a6210 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f5ebc005fd0 tx=0x7f5ebc009310 comp rx=0 tx=0).stop 2024-04-16T17:05:17.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:17.360+0000 7f5ece000700 1 -- 172.21.15.45:0/1432069712 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5ec8108a20 msgr2=0x7f5ec819fca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:17.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:17.360+0000 7f5ece000700 1 --2- 172.21.15.45:0/1432069712 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5ec8108a20 0x7f5ec819fca0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f5ec000efc0 tx=0x7f5ec000c3a0 comp rx=0 tx=0).stop 2024-04-16T17:05:17.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:17.361+0000 7f5ece000700 1 -- 172.21.15.45:0/1432069712 shutdown_connections 2024-04-16T17:05:17.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:17.361+0000 7f5ece000700 1 --2- 172.21.15.45:0/1432069712 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f5eb00a3d50 0x7f5eb00a6210 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:17.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:17.361+0000 7f5ece000700 1 --2- 172.21.15.45:0/1432069712 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5ec8108a20 0x7f5ec819fca0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:17.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:17.361+0000 7f5ece000700 1 --2- 172.21.15.45:0/1432069712 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5ec8102a20 0x7f5ec819b3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:17.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:17.361+0000 7f5ece000700 1 --2- 172.21.15.45:0/1432069712 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5ec81033e0 0x7f5ec819b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:17.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:17.361+0000 7f5ece000700 1 -- 172.21.15.45:0/1432069712 >> 172.21.15.45:0/1432069712 conn(0x7f5ec80fe570 msgr2=0x7f5ec810dd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:05:17.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:17.361+0000 7f5ece000700 1 -- 172.21.15.45:0/1432069712 shutdown_connections 2024-04-16T17:05:17.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:17.361+0000 7f5ece000700 1 -- 172.21.15.45:0/1432069712 wait complete. 2024-04-16T17:05:17.555 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:05:17 smithi117 ceph-mon[36730]: pgmap v380: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-16T17:05:17.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:05:17 smithi104 ceph-mon[36457]: pgmap v380: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-16T17:05:18.172 INFO:tasks.cephadm.ceph_manager.ceph:need seq 257698037808 got 257698037814 for osd.8 2024-04-16T17:05:18.173 DEBUG:teuthology.parallel:result is None 2024-04-16T17:05:18.282 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411406 got 77309411415 for osd.2 2024-04-16T17:05:18.282 DEBUG:teuthology.parallel:result is None 2024-04-16T17:05:18.394 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:05:18 smithi045 ceph-mon[31676]: from='client.? 172.21.15.45:0/1432069712' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-16T17:05:18.555 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:05:18 smithi117 ceph-mon[36730]: from='client.? 172.21.15.45:0/1432069712' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-16T17:05:18.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:05:18 smithi104 ceph-mon[36457]: from='client.? 172.21.15.45:0/1432069712' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-16T17:05:18.909 INFO:tasks.cephadm.ceph_manager.ceph:need seq 313532612644 got 313532612651 for osd.10 2024-04-16T17:05:18.909 DEBUG:teuthology.parallel:result is None 2024-04-16T17:05:18.909 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-04-16T17:05:18.909 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45 shell --fsid b1fecd38-fc11-11ee-bc90-c7b262605968 -- ceph pg dump --format=json 2024-04-16T17:05:19.405 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/mon.a/config 2024-04-16T17:05:19.555 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:05:19 smithi117 ceph-mon[36730]: pgmap v381: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-16T17:05:19.560 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:05:19 smithi045 ceph-mon[31676]: pgmap v381: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-16T17:05:19.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:05:19 smithi104 ceph-mon[36457]: pgmap v381: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-16T17:05:21.478 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:05:21 smithi045 ceph-mon[31676]: pgmap v382: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-16T17:05:21.478 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:05:21 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [9, 1]}]: dispatch 2024-04-16T17:05:21.479 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:05:21 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.7", "id": [5, 8]}]: dispatch 2024-04-16T17:05:21.555 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:05:21 smithi117 ceph-mon[36730]: pgmap v382: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-16T17:05:21.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:05:21 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [9, 1]}]: dispatch 2024-04-16T17:05:21.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:05:21 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.7", "id": [5, 8]}]: dispatch 2024-04-16T17:05:21.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:05:21 smithi104 ceph-mon[36457]: pgmap v382: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-16T17:05:21.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:05:21 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [9, 1]}]: dispatch 2024-04-16T17:05:21.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:05:21 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.7", "id": [5, 8]}]: dispatch 2024-04-16T17:05:21.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:21.698+0000 7f9a47e00700 1 -- 172.21.15.45:0/291586973 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9a48103be0 msgr2=0x7f9a48103fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:21.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:21.698+0000 7f9a47e00700 1 --2- 172.21.15.45:0/291586973 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9a48103be0 0x7f9a48103fc0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f9a38009990 tx=0x7f9a38009ca0 comp rx=0 tx=0).stop 2024-04-16T17:05:21.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:21.698+0000 7f9a47e00700 1 -- 172.21.15.45:0/291586973 shutdown_connections 2024-04-16T17:05:21.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:21.698+0000 7f9a47e00700 1 --2- 172.21.15.45:0/291586973 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9a4810d010 0x7f9a4810f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:21.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:21.698+0000 7f9a47e00700 1 --2- 172.21.15.45:0/291586973 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9a48103be0 0x7f9a48103fc0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:21.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:21.698+0000 7f9a47e00700 1 --2- 172.21.15.45:0/291586973 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9a48104500 0x7f9a4810c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:21.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:21.698+0000 7f9a47e00700 1 -- 172.21.15.45:0/291586973 >> 172.21.15.45:0/291586973 conn(0x7f9a480fd410 msgr2=0x7f9a480ff830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:05:21.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:21.698+0000 7f9a47e00700 1 -- 172.21.15.45:0/291586973 shutdown_connections 2024-04-16T17:05:21.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:21.698+0000 7f9a47e00700 1 -- 172.21.15.45:0/291586973 wait complete. 2024-04-16T17:05:21.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:21.699+0000 7f9a47e00700 1 Processor -- start 2024-04-16T17:05:21.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:21.699+0000 7f9a47e00700 1 -- start start 2024-04-16T17:05:21.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:21.700+0000 7f9a47e00700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9a48103be0 0x7f9a48101840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:21.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:21.700+0000 7f9a47e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9a48104500 0x7f9a48101d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:21.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:21.700+0000 7f9a47e00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9a4810d010 0x7f9a481022c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:21.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:21.700+0000 7f9a47e00700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f9a48112d70 con 0x7f9a4810d010 2024-04-16T17:05:21.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:21.700+0000 7f9a47e00700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f9a48112bf0 con 0x7f9a48103be0 2024-04-16T17:05:21.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:21.700+0000 7f9a47e00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f9a48112ef0 con 0x7f9a48104500 2024-04-16T17:05:21.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:21.700+0000 7f9a47400700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9a4810d010 0x7f9a481022c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:21.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:21.700+0000 7f9a46000700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9a48104500 0x7f9a48101d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:21.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:21.700+0000 7f9a47400700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9a4810d010 0x7f9a481022c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58132/0 (socket says 172.21.15.45:58132) 2024-04-16T17:05:21.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:21.700+0000 7f9a46000700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9a48104500 0x7f9a48101d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:56808/0 (socket says 172.21.15.45:56808) 2024-04-16T17:05:21.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:21.700+0000 7f9a46000700 1 -- 172.21.15.45:0/2368813939 learned_addr learned my addr 172.21.15.45:0/2368813939 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:05:21.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:21.700+0000 7f9a46a00700 1 --2- 172.21.15.45:0/2368813939 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9a48103be0 0x7f9a48101840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:21.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:21.701+0000 7f9a46000700 1 -- 172.21.15.45:0/2368813939 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9a48103be0 msgr2=0x7f9a48101840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:21.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:21.701+0000 7f9a46000700 1 --2- 172.21.15.45:0/2368813939 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9a48103be0 0x7f9a48101840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:21.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:21.701+0000 7f9a46000700 1 -- 172.21.15.45:0/2368813939 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9a4810d010 msgr2=0x7f9a481022c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:21.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:21.701+0000 7f9a46000700 1 --2- 172.21.15.45:0/2368813939 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9a4810d010 0x7f9a481022c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:21.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:21.701+0000 7f9a46000700 1 -- 172.21.15.45:0/2368813939 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a481a7700 con 0x7f9a48104500 2024-04-16T17:05:21.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:21.701+0000 7f9a46000700 1 --2- 172.21.15.45:0/2368813939 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9a48104500 0x7f9a48101d80 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f9a3400e980 tx=0x7f9a3400ec90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:05:21.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:21.701+0000 7f9a3f400700 1 -- 172.21.15.45:0/2368813939 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a34004220 con 0x7f9a48104500 2024-04-16T17:05:21.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:21.701+0000 7f9a3f400700 1 -- 172.21.15.45:0/2368813939 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9a340043c0 con 0x7f9a48104500 2024-04-16T17:05:21.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:21.701+0000 7f9a3f400700 1 -- 172.21.15.45:0/2368813939 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a34010430 con 0x7f9a48104500 2024-04-16T17:05:21.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:21.702+0000 7f9a47e00700 1 -- 172.21.15.45:0/2368813939 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a481a79f0 con 0x7f9a48104500 2024-04-16T17:05:21.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:21.702+0000 7f9a47e00700 1 -- 172.21.15.45:0/2368813939 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a481a7ee0 con 0x7f9a48104500 2024-04-16T17:05:21.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:21.703+0000 7f9a47e00700 1 -- 172.21.15.45:0/2368813939 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a480697e0 con 0x7f9a48104500 2024-04-16T17:05:21.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:21.705+0000 7f9a3f400700 1 -- 172.21.15.45:0/2368813939 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f9a3400c810 con 0x7f9a48104500 2024-04-16T17:05:21.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:21.706+0000 7f9a3f400700 1 --2- 172.21.15.45:0/2368813939 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f9a2c0a3cc0 0x7f9a2c0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:21.708 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:21.706+0000 7f9a3f400700 1 -- 172.21.15.45:0/2368813939 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7f9a340d0840 con 0x7f9a48104500 2024-04-16T17:05:21.708 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:21.706+0000 7f9a46a00700 1 --2- 172.21.15.45:0/2368813939 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f9a2c0a3cc0 0x7f9a2c0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:21.708 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:21.707+0000 7f9a46a00700 1 --2- 172.21.15.45:0/2368813939 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f9a2c0a3cc0 0x7f9a2c0a6180 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f9a380098e0 tx=0x7f9a38005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:05:21.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:21.711+0000 7f9a3f400700 1 -- 172.21.15.45:0/2368813939 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9a3409ce30 con 0x7f9a48104500 2024-04-16T17:05:21.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:21.941+0000 7f9a47e00700 1 -- 172.21.15.45:0/2368813939 --> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f9a48062f30 con 0x7f9a2c0a3cc0 2024-04-16T17:05:21.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:21.977+0000 7f9a3f400700 1 -- 172.21.15.45:0/2368813939 <== mgr.14152 v2:172.21.15.45:6800/268030699 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340783 (secure 0 0 0) 0x7f9a48062f30 con 0x7f9a2c0a3cc0 2024-04-16T17:05:21.981 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-16T17:05:21.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:21.984+0000 7f9a47e00700 1 -- 172.21.15.45:0/2368813939 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f9a2c0a3cc0 msgr2=0x7f9a2c0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:21.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:21.984+0000 7f9a47e00700 1 --2- 172.21.15.45:0/2368813939 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f9a2c0a3cc0 0x7f9a2c0a6180 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f9a380098e0 tx=0x7f9a38005960 comp rx=0 tx=0).stop 2024-04-16T17:05:21.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:21.984+0000 7f9a47e00700 1 -- 172.21.15.45:0/2368813939 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9a48104500 msgr2=0x7f9a48101d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:21.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:21.984+0000 7f9a47e00700 1 --2- 172.21.15.45:0/2368813939 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9a48104500 0x7f9a48101d80 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f9a3400e980 tx=0x7f9a3400ec90 comp rx=0 tx=0).stop 2024-04-16T17:05:21.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:21.984+0000 7f9a47e00700 1 -- 172.21.15.45:0/2368813939 shutdown_connections 2024-04-16T17:05:21.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:21.984+0000 7f9a47e00700 1 --2- 172.21.15.45:0/2368813939 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9a4810d010 0x7f9a481022c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:21.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:21.984+0000 7f9a47e00700 1 --2- 172.21.15.45:0/2368813939 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9a48103be0 0x7f9a48101840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:21.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:21.985+0000 7f9a47e00700 1 --2- 172.21.15.45:0/2368813939 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f9a2c0a3cc0 0x7f9a2c0a6180 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:21.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:21.985+0000 7f9a47e00700 1 --2- 172.21.15.45:0/2368813939 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9a48104500 0x7f9a48101d80 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:21.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:21.985+0000 7f9a47e00700 1 -- 172.21.15.45:0/2368813939 >> 172.21.15.45:0/2368813939 conn(0x7f9a480fd410 msgr2=0x7f9a480fe120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:05:21.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:21.985+0000 7f9a47e00700 1 -- 172.21.15.45:0/2368813939 shutdown_connections 2024-04-16T17:05:21.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:21.985+0000 7f9a47e00700 1 -- 172.21.15.45:0/2368813939 wait complete. 2024-04-16T17:05:21.989 INFO:teuthology.orchestra.run.smithi045.stderr:dumped all 2024-04-16T17:05:22.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:05:22 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [9, 1]}]': finished 2024-04-16T17:05:22.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:05:22 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.7", "id": [5, 8]}]': finished 2024-04-16T17:05:22.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:05:22 smithi117 ceph-mon[36730]: osdmap e93: 12 total, 12 up, 12 in 2024-04-16T17:05:22.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:05:22 smithi117 ceph-mon[36730]: mgrmap e18: x(active, since 10m), standbys: z, y 2024-04-16T17:05:22.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:05:22 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [9, 1]}]': finished 2024-04-16T17:05:22.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:05:22 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.7", "id": [5, 8]}]': finished 2024-04-16T17:05:22.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:05:22 smithi104 ceph-mon[36457]: osdmap e93: 12 total, 12 up, 12 in 2024-04-16T17:05:22.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:05:22 smithi104 ceph-mon[36457]: mgrmap e18: x(active, since 10m), standbys: z, y 2024-04-16T17:05:22.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:05:22 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [9, 1]}]': finished 2024-04-16T17:05:22.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:05:22 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.7", "id": [5, 8]}]': finished 2024-04-16T17:05:22.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:05:22 smithi045 ceph-mon[31676]: osdmap e93: 12 total, 12 up, 12 in 2024-04-16T17:05:22.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:05:22 smithi045 ceph-mon[31676]: mgrmap e18: x(active, since 10m), standbys: z, y 2024-04-16T17:05:22.872 INFO:teuthology.orchestra.run.smithi045.stdout:{"pg_ready":true,"pg_map":{"version":383,"stamp":"2024-04-16T17:05:21.273806+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":658491,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":681,"num_read_kb":466,"num_write":351,"num_write_kb":3060,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":1181469,"num_keys_recovered":20,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":428,"ondisk_log_size":428,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":373924,"kb_used_data":12248,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124567900,"statfs":{"total":1151940427776,"available":1151557529600,"internally_reserved":0,"allocated":12541952,"data_stored":7997417,"data_compressed":361473,"data_compressed_allocated":491520,"data_compressed_original":2482176,"omap_allocated":19143,"internal_metadata":370193721},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":141,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":70,"num_read_kb":35,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"11.127988"},"pg_stats":[{"pgid":"3.3f","version":"92'8","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T17:05:20.165027+0000","last_change":"2024-04-16T17:03:24.989508+0000","last_active":"2024-04-16T17:05:20.165027+0000","last_peered":"2024-04-16T17:05:20.165027+0000","last_clean":"2024-04-16T17:05:20.165027+0000","last_became_active":"2024-04-16T17:03:24.989178+0000","last_became_peered":"2024-04-16T17:03:24.989178+0000","last_unstale":"2024-04-16T17:05:20.165027+0000","last_undegraded":"2024-04-16T17:05:20.165027+0000","last_fullsized":"2024-04-16T17:05:20.165027+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:18:56.488358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T17:04:21.505697+0000","last_change":"2024-04-16T17:03:24.986378+0000","last_active":"2024-04-16T17:04:21.505697+0000","last_peered":"2024-04-16T17:04:21.505697+0000","last_clean":"2024-04-16T17:04:21.505697+0000","last_became_active":"2024-04-16T17:03:24.986030+0000","last_became_peered":"2024-04-16T17:03:24.986030+0000","last_unstale":"2024-04-16T17:04:21.505697+0000","last_undegraded":"2024-04-16T17:04:21.505697+0000","last_fullsized":"2024-04-16T17:04:21.505697+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:21:14.071109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T17:04:21.504778+0000","last_change":"2024-04-16T17:03:24.976232+0000","last_active":"2024-04-16T17:04:21.504778+0000","last_peered":"2024-04-16T17:04:21.504778+0000","last_clean":"2024-04-16T17:04:21.504778+0000","last_became_active":"2024-04-16T17:03:24.975873+0000","last_became_peered":"2024-04-16T17:03:24.975873+0000","last_unstale":"2024-04-16T17:04:21.504778+0000","last_undegraded":"2024-04-16T17:04:21.504778+0000","last_fullsized":"2024-04-16T17:04:21.504778+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:07:53.523101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T17:04:21.520574+0000","last_change":"2024-04-16T17:04:21.517830+0000","last_active":"2024-04-16T17:04:21.520574+0000","last_peered":"2024-04-16T17:04:21.520574+0000","last_clean":"2024-04-16T17:04:21.520574+0000","last_became_active":"2024-04-16T17:04:21.516749+0000","last_became_peered":"2024-04-16T17:04:21.516749+0000","last_unstale":"2024-04-16T17:04:21.520574+0000","last_undegraded":"2024-04-16T17:04:21.520574+0000","last_fullsized":"2024-04-16T17:04:21.520574+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:02:33.344034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T17:04:21.503817+0000","last_change":"2024-04-16T17:03:24.985609+0000","last_active":"2024-04-16T17:04:21.503817+0000","last_peered":"2024-04-16T17:04:21.503817+0000","last_clean":"2024-04-16T17:04:21.503817+0000","last_became_active":"2024-04-16T17:03:24.985239+0000","last_became_peered":"2024-04-16T17:03:24.985239+0000","last_unstale":"2024-04-16T17:04:21.503817+0000","last_undegraded":"2024-04-16T17:04:21.503817+0000","last_fullsized":"2024-04-16T17:04:21.503817+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:55:58.619210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T17:04:21.497420+0000","last_change":"2024-04-16T17:03:24.966862+0000","last_active":"2024-04-16T17:04:21.497420+0000","last_peered":"2024-04-16T17:04:21.497420+0000","last_clean":"2024-04-16T17:04:21.497420+0000","last_became_active":"2024-04-16T17:03:24.966234+0000","last_became_peered":"2024-04-16T17:03:24.966234+0000","last_unstale":"2024-04-16T17:04:21.497420+0000","last_undegraded":"2024-04-16T17:04:21.497420+0000","last_fullsized":"2024-04-16T17:04:21.497420+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:21:07.018281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T17:04:21.501565+0000","last_change":"2024-04-16T17:03:24.971219+0000","last_active":"2024-04-16T17:04:21.501565+0000","last_peered":"2024-04-16T17:04:21.501565+0000","last_clean":"2024-04-16T17:04:21.501565+0000","last_became_active":"2024-04-16T17:03:24.970876+0000","last_became_peered":"2024-04-16T17:03:24.970876+0000","last_unstale":"2024-04-16T17:04:21.501565+0000","last_undegraded":"2024-04-16T17:04:21.501565+0000","last_fullsized":"2024-04-16T17:04:21.501565+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:44:16.311167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T17:04:21.500129+0000","last_change":"2024-04-16T17:03:24.994694+0000","last_active":"2024-04-16T17:04:21.500129+0000","last_peered":"2024-04-16T17:04:21.500129+0000","last_clean":"2024-04-16T17:04:21.500129+0000","last_became_active":"2024-04-16T17:03:24.994315+0000","last_became_peered":"2024-04-16T17:03:24.994315+0000","last_unstale":"2024-04-16T17:04:21.500129+0000","last_undegraded":"2024-04-16T17:04:21.500129+0000","last_fullsized":"2024-04-16T17:04:21.500129+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:04:59.543816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T17:04:21.881206+0000","last_change":"2024-04-16T17:03:24.964654+0000","last_active":"2024-04-16T17:04:21.881206+0000","last_peered":"2024-04-16T17:04:21.881206+0000","last_clean":"2024-04-16T17:04:21.881206+0000","last_became_active":"2024-04-16T17:03:24.964349+0000","last_became_peered":"2024-04-16T17:03:24.964349+0000","last_unstale":"2024-04-16T17:04:21.881206+0000","last_undegraded":"2024-04-16T17:04:21.881206+0000","last_fullsized":"2024-04-16T17:04:21.881206+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:38:55.880948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T17:04:21.497850+0000","last_change":"2024-04-16T17:03:24.966364+0000","last_active":"2024-04-16T17:04:21.497850+0000","last_peered":"2024-04-16T17:04:21.497850+0000","last_clean":"2024-04-16T17:04:21.497850+0000","last_became_active":"2024-04-16T17:03:24.966061+0000","last_became_peered":"2024-04-16T17:03:24.966061+0000","last_unstale":"2024-04-16T17:04:21.497850+0000","last_undegraded":"2024-04-16T17:04:21.497850+0000","last_fullsized":"2024-04-16T17:04:21.497850+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:18:12.642602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T17:04:21.502473+0000","last_change":"2024-04-16T17:03:24.969874+0000","last_active":"2024-04-16T17:04:21.502473+0000","last_peered":"2024-04-16T17:04:21.502473+0000","last_clean":"2024-04-16T17:04:21.502473+0000","last_became_active":"2024-04-16T17:03:24.969523+0000","last_became_peered":"2024-04-16T17:03:24.969523+0000","last_unstale":"2024-04-16T17:04:21.502473+0000","last_undegraded":"2024-04-16T17:04:21.502473+0000","last_fullsized":"2024-04-16T17:04:21.502473+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:17:49.704855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T17:04:22.147861+0000","last_change":"2024-04-16T17:03:24.968660+0000","last_active":"2024-04-16T17:04:22.147861+0000","last_peered":"2024-04-16T17:04:22.147861+0000","last_clean":"2024-04-16T17:04:22.147861+0000","last_became_active":"2024-04-16T17:03:24.968348+0000","last_became_peered":"2024-04-16T17:03:24.968348+0000","last_unstale":"2024-04-16T17:04:22.147861+0000","last_undegraded":"2024-04-16T17:04:22.147861+0000","last_fullsized":"2024-04-16T17:04:22.147861+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:06:06.319391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"92'10","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T17:05:20.156899+0000","last_change":"2024-04-16T17:03:24.974741+0000","last_active":"2024-04-16T17:05:20.156899+0000","last_peered":"2024-04-16T17:05:20.156899+0000","last_clean":"2024-04-16T17:05:20.156899+0000","last_became_active":"2024-04-16T17:03:24.973568+0000","last_became_peered":"2024-04-16T17:03:24.973568+0000","last_unstale":"2024-04-16T17:05:20.156899+0000","last_undegraded":"2024-04-16T17:05:20.156899+0000","last_fullsized":"2024-04-16T17:05:20.156899+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:30:49.325469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T17:04:21.501626+0000","last_change":"2024-04-16T17:03:24.981629+0000","last_active":"2024-04-16T17:04:21.501626+0000","last_peered":"2024-04-16T17:04:21.501626+0000","last_clean":"2024-04-16T17:04:21.501626+0000","last_became_active":"2024-04-16T17:03:24.981111+0000","last_became_peered":"2024-04-16T17:03:24.981111+0000","last_unstale":"2024-04-16T17:04:21.501626+0000","last_undegraded":"2024-04-16T17:04:21.501626+0000","last_fullsized":"2024-04-16T17:04:21.501626+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:41:29.641610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T17:04:21.501250+0000","last_change":"2024-04-16T17:03:24.977890+0000","last_active":"2024-04-16T17:04:21.501250+0000","last_peered":"2024-04-16T17:04:21.501250+0000","last_clean":"2024-04-16T17:04:21.501250+0000","last_became_active":"2024-04-16T17:03:24.976500+0000","last_became_peered":"2024-04-16T17:03:24.976500+0000","last_unstale":"2024-04-16T17:04:21.501250+0000","last_undegraded":"2024-04-16T17:04:21.501250+0000","last_fullsized":"2024-04-16T17:04:21.501250+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:17:51.604701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T17:04:21.880287+0000","last_change":"2024-04-16T17:03:24.976085+0000","last_active":"2024-04-16T17:04:21.880287+0000","last_peered":"2024-04-16T17:04:21.880287+0000","last_clean":"2024-04-16T17:04:21.880287+0000","last_became_active":"2024-04-16T17:03:24.975804+0000","last_became_peered":"2024-04-16T17:03:24.975804+0000","last_unstale":"2024-04-16T17:04:21.880287+0000","last_undegraded":"2024-04-16T17:04:21.880287+0000","last_fullsized":"2024-04-16T17:04:21.880287+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:19:13.706889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"92'8","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T17:05:20.164869+0000","last_change":"2024-04-16T17:03:24.989456+0000","last_active":"2024-04-16T17:05:20.164869+0000","last_peered":"2024-04-16T17:05:20.164869+0000","last_clean":"2024-04-16T17:05:20.164869+0000","last_became_active":"2024-04-16T17:03:24.989161+0000","last_became_peered":"2024-04-16T17:03:24.989161+0000","last_unstale":"2024-04-16T17:05:20.164869+0000","last_undegraded":"2024-04-16T17:05:20.164869+0000","last_fullsized":"2024-04-16T17:05:20.164869+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:52:59.333625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T17:04:21.519756+0000","last_change":"2024-04-16T17:03:24.963428+0000","last_active":"2024-04-16T17:04:21.519756+0000","last_peered":"2024-04-16T17:04:21.519756+0000","last_clean":"2024-04-16T17:04:21.519756+0000","last_became_active":"2024-04-16T17:03:24.962523+0000","last_became_peered":"2024-04-16T17:03:24.962523+0000","last_unstale":"2024-04-16T17:04:21.519756+0000","last_undegraded":"2024-04-16T17:04:21.519756+0000","last_fullsized":"2024-04-16T17:04:21.519756+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:22:19.523901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T17:04:21.502387+0000","last_change":"2024-04-16T17:03:24.981122+0000","last_active":"2024-04-16T17:04:21.502387+0000","last_peered":"2024-04-16T17:04:21.502387+0000","last_clean":"2024-04-16T17:04:21.502387+0000","last_became_active":"2024-04-16T17:03:24.980533+0000","last_became_peered":"2024-04-16T17:03:24.980533+0000","last_unstale":"2024-04-16T17:04:21.502387+0000","last_undegraded":"2024-04-16T17:04:21.502387+0000","last_fullsized":"2024-04-16T17:04:21.502387+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:51:08.580965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T17:04:21.502277+0000","last_change":"2024-04-16T17:03:29.257239+0000","last_active":"2024-04-16T17:04:21.502277+0000","last_peered":"2024-04-16T17:04:21.502277+0000","last_clean":"2024-04-16T17:04:21.502277+0000","last_became_active":"2024-04-16T17:03:29.256827+0000","last_became_peered":"2024-04-16T17:03:29.256827+0000","last_unstale":"2024-04-16T17:04:21.502277+0000","last_undegraded":"2024-04-16T17:04:21.502277+0000","last_fullsized":"2024-04-16T17:04:21.502277+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:57:28.373386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"90'9","reported_seq":53,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T17:04:21.896665+0000","last_change":"2024-04-16T17:04:21.896665+0000","last_active":"2024-04-16T17:04:21.896665+0000","last_peered":"2024-04-16T17:04:21.896665+0000","last_clean":"2024-04-16T17:04:21.896665+0000","last_became_active":"2024-04-16T17:04:21.882573+0000","last_became_peered":"2024-04-16T17:04:21.882573+0000","last_unstale":"2024-04-16T17:04:21.896665+0000","last_undegraded":"2024-04-16T17:04:21.896665+0000","last_fullsized":"2024-04-16T17:04:21.896665+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:37:07.210368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,1],"acting":[9,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T17:04:21.503068+0000","last_change":"2024-04-16T17:03:24.994921+0000","last_active":"2024-04-16T17:04:21.503068+0000","last_peered":"2024-04-16T17:04:21.503068+0000","last_clean":"2024-04-16T17:04:21.503068+0000","last_became_active":"2024-04-16T17:03:24.994412+0000","last_became_peered":"2024-04-16T17:03:24.994412+0000","last_unstale":"2024-04-16T17:04:21.503068+0000","last_undegraded":"2024-04-16T17:04:21.503068+0000","last_fullsized":"2024-04-16T17:04:21.503068+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:11:40.748198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T17:04:22.147204+0000","last_change":"2024-04-16T17:03:29.259884+0000","last_active":"2024-04-16T17:04:22.147204+0000","last_peered":"2024-04-16T17:04:22.147204+0000","last_clean":"2024-04-16T17:04:22.147204+0000","last_became_active":"2024-04-16T17:03:29.259319+0000","last_became_peered":"2024-04-16T17:03:29.259319+0000","last_unstale":"2024-04-16T17:04:22.147204+0000","last_undegraded":"2024-04-16T17:04:22.147204+0000","last_fullsized":"2024-04-16T17:04:22.147204+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:05:18.244682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"90'2","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T17:04:21.504276+0000","last_change":"2024-04-16T17:03:24.179308+0000","last_active":"2024-04-16T17:04:21.504276+0000","last_peered":"2024-04-16T17:04:21.504276+0000","last_clean":"2024-04-16T17:04:21.504276+0000","last_became_active":"2024-04-16T17:03:24.178894+0000","last_became_peered":"2024-04-16T17:03:24.178894+0000","last_unstale":"2024-04-16T17:04:21.504276+0000","last_undegraded":"2024-04-16T17:04:21.504276+0000","last_fullsized":"2024-04-16T17:04:21.504276+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:17:46.831871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T17:04:21.502714+0000","last_change":"2024-04-16T17:03:24.982391+0000","last_active":"2024-04-16T17:04:21.502714+0000","last_peered":"2024-04-16T17:04:21.502714+0000","last_clean":"2024-04-16T17:04:21.502714+0000","last_became_active":"2024-04-16T17:03:24.981761+0000","last_became_peered":"2024-04-16T17:03:24.981761+0000","last_unstale":"2024-04-16T17:04:21.502714+0000","last_undegraded":"2024-04-16T17:04:21.502714+0000","last_fullsized":"2024-04-16T17:04:21.502714+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:40:57.657170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T17:04:22.147440+0000","last_change":"2024-04-16T17:03:29.259705+0000","last_active":"2024-04-16T17:04:22.147440+0000","last_peered":"2024-04-16T17:04:22.147440+0000","last_clean":"2024-04-16T17:04:22.147440+0000","last_became_active":"2024-04-16T17:03:29.259336+0000","last_became_peered":"2024-04-16T17:03:29.259336+0000","last_unstale":"2024-04-16T17:04:22.147440+0000","last_undegraded":"2024-04-16T17:04:22.147440+0000","last_fullsized":"2024-04-16T17:04:22.147440+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:53:29.042933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"90'4","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T17:04:22.153413+0000","last_change":"2024-04-16T17:04:22.153413+0000","last_active":"2024-04-16T17:04:22.153413+0000","last_peered":"2024-04-16T17:04:22.153413+0000","last_clean":"2024-04-16T17:04:22.153413+0000","last_became_active":"2024-04-16T17:04:22.148972+0000","last_became_peered":"2024-04-16T17:04:22.148972+0000","last_unstale":"2024-04-16T17:04:22.153413+0000","last_undegraded":"2024-04-16T17:04:22.153413+0000","last_fullsized":"2024-04-16T17:04:22.153413+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:05:57.513052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T17:04:21.880746+0000","last_change":"2024-04-16T17:03:24.973205+0000","last_active":"2024-04-16T17:04:21.880746+0000","last_peered":"2024-04-16T17:04:21.880746+0000","last_clean":"2024-04-16T17:04:21.880746+0000","last_became_active":"2024-04-16T17:03:24.972863+0000","last_became_peered":"2024-04-16T17:03:24.972863+0000","last_unstale":"2024-04-16T17:04:21.880746+0000","last_undegraded":"2024-04-16T17:04:21.880746+0000","last_fullsized":"2024-04-16T17:04:21.880746+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:28:29.005783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T17:04:22.147250+0000","last_change":"2024-04-16T17:03:29.262930+0000","last_active":"2024-04-16T17:04:22.147250+0000","last_peered":"2024-04-16T17:04:22.147250+0000","last_clean":"2024-04-16T17:04:22.147250+0000","last_became_active":"2024-04-16T17:03:29.262621+0000","last_became_peered":"2024-04-16T17:03:29.262621+0000","last_unstale":"2024-04-16T17:04:22.147250+0000","last_undegraded":"2024-04-16T17:04:22.147250+0000","last_fullsized":"2024-04-16T17:04:22.147250+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:22:31.555143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T17:04:21.503444+0000","last_change":"2024-04-16T17:03:24.150503+0000","last_active":"2024-04-16T17:04:21.503444+0000","last_peered":"2024-04-16T17:04:21.503444+0000","last_clean":"2024-04-16T17:04:21.503444+0000","last_became_active":"2024-04-16T17:03:24.150021+0000","last_became_peered":"2024-04-16T17:03:24.150021+0000","last_unstale":"2024-04-16T17:04:21.503444+0000","last_undegraded":"2024-04-16T17:04:21.503444+0000","last_fullsized":"2024-04-16T17:04:21.503444+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:21:36.119497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"92'15","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T17:05:20.158223+0000","last_change":"2024-04-16T17:03:24.965049+0000","last_active":"2024-04-16T17:05:20.158223+0000","last_peered":"2024-04-16T17:05:20.158223+0000","last_clean":"2024-04-16T17:05:20.158223+0000","last_became_active":"2024-04-16T17:03:24.963120+0000","last_became_peered":"2024-04-16T17:03:24.963120+0000","last_unstale":"2024-04-16T17:05:20.158223+0000","last_undegraded":"2024-04-16T17:05:20.158223+0000","last_fullsized":"2024-04-16T17:05:20.158223+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:17:49.725113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T17:04:21.502466+0000","last_change":"2024-04-16T17:03:29.247013+0000","last_active":"2024-04-16T17:04:21.502466+0000","last_peered":"2024-04-16T17:04:21.502466+0000","last_clean":"2024-04-16T17:04:21.502466+0000","last_became_active":"2024-04-16T17:03:29.246477+0000","last_became_peered":"2024-04-16T17:03:29.246477+0000","last_unstale":"2024-04-16T17:04:21.502466+0000","last_undegraded":"2024-04-16T17:04:21.502466+0000","last_fullsized":"2024-04-16T17:04:21.502466+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:52:19.525076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"92'7","reported_seq":124,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T17:05:19.512951+0000","last_change":"2024-04-16T17:03:24.155333+0000","last_active":"2024-04-16T17:05:19.512951+0000","last_peered":"2024-04-16T17:05:19.512951+0000","last_clean":"2024-04-16T17:05:19.512951+0000","last_became_active":"2024-04-16T17:03:24.154848+0000","last_became_peered":"2024-04-16T17:03:24.154848+0000","last_unstale":"2024-04-16T17:05:19.512951+0000","last_undegraded":"2024-04-16T17:05:19.512951+0000","last_fullsized":"2024-04-16T17:05:19.512951+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:03:34.769573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1154,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":90,"num_read_kb":3,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T17:04:21.503489+0000","last_change":"2024-04-16T17:03:24.967627+0000","last_active":"2024-04-16T17:04:21.503489+0000","last_peered":"2024-04-16T17:04:21.503489+0000","last_clean":"2024-04-16T17:04:21.503489+0000","last_became_active":"2024-04-16T17:03:24.967264+0000","last_became_peered":"2024-04-16T17:03:24.967264+0000","last_unstale":"2024-04-16T17:04:21.503489+0000","last_undegraded":"2024-04-16T17:04:21.503489+0000","last_fullsized":"2024-04-16T17:04:21.503489+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:35:29.182374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T17:04:21.501736+0000","last_change":"2024-04-16T17:03:29.264625+0000","last_active":"2024-04-16T17:04:21.501736+0000","last_peered":"2024-04-16T17:04:21.501736+0000","last_clean":"2024-04-16T17:04:21.501736+0000","last_became_active":"2024-04-16T17:03:29.264114+0000","last_became_peered":"2024-04-16T17:03:29.264114+0000","last_unstale":"2024-04-16T17:04:21.501736+0000","last_undegraded":"2024-04-16T17:04:21.501736+0000","last_fullsized":"2024-04-16T17:04:21.501736+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:14:49.449980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"90'3","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T17:04:21.497406+0000","last_change":"2024-04-16T17:03:24.160457+0000","last_active":"2024-04-16T17:04:21.497406+0000","last_peered":"2024-04-16T17:04:21.497406+0000","last_clean":"2024-04-16T17:04:21.497406+0000","last_became_active":"2024-04-16T17:03:24.160195+0000","last_became_peered":"2024-04-16T17:03:24.160195+0000","last_unstale":"2024-04-16T17:04:21.497406+0000","last_undegraded":"2024-04-16T17:04:21.497406+0000","last_fullsized":"2024-04-16T17:04:21.497406+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:23:07.064539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"92'8","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T17:05:20.167765+0000","last_change":"2024-04-16T17:03:24.991249+0000","last_active":"2024-04-16T17:05:20.167765+0000","last_peered":"2024-04-16T17:05:20.167765+0000","last_clean":"2024-04-16T17:05:20.167765+0000","last_became_active":"2024-04-16T17:03:24.990927+0000","last_became_peered":"2024-04-16T17:03:24.990927+0000","last_unstale":"2024-04-16T17:05:20.167765+0000","last_undegraded":"2024-04-16T17:05:20.167765+0000","last_fullsized":"2024-04-16T17:05:20.167765+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:36:39.975616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T17:04:21.499715+0000","last_change":"2024-04-16T17:03:29.253704+0000","last_active":"2024-04-16T17:04:21.499715+0000","last_peered":"2024-04-16T17:04:21.499715+0000","last_clean":"2024-04-16T17:04:21.499715+0000","last_became_active":"2024-04-16T17:03:29.253334+0000","last_became_peered":"2024-04-16T17:03:29.253334+0000","last_unstale":"2024-04-16T17:04:21.499715+0000","last_undegraded":"2024-04-16T17:04:21.499715+0000","last_fullsized":"2024-04-16T17:04:21.499715+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:19:12.611731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"90'2","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T17:04:22.198461+0000","last_change":"2024-04-16T17:03:24.158875+0000","last_active":"2024-04-16T17:04:22.198461+0000","last_peered":"2024-04-16T17:04:22.198461+0000","last_clean":"2024-04-16T17:04:22.198461+0000","last_became_active":"2024-04-16T17:03:24.158465+0000","last_became_peered":"2024-04-16T17:03:24.158465+0000","last_unstale":"2024-04-16T17:04:22.198461+0000","last_undegraded":"2024-04-16T17:04:22.198461+0000","last_fullsized":"2024-04-16T17:04:22.198461+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:21:29.882251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T17:04:21.500496+0000","last_change":"2024-04-16T17:03:24.994476+0000","last_active":"2024-04-16T17:04:21.500496+0000","last_peered":"2024-04-16T17:04:21.500496+0000","last_clean":"2024-04-16T17:04:21.500496+0000","last_became_active":"2024-04-16T17:03:24.993532+0000","last_became_peered":"2024-04-16T17:03:24.993532+0000","last_unstale":"2024-04-16T17:04:21.500496+0000","last_undegraded":"2024-04-16T17:04:21.500496+0000","last_fullsized":"2024-04-16T17:04:21.500496+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:04:44.062444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T17:04:21.505370+0000","last_change":"2024-04-16T17:03:29.255446+0000","last_active":"2024-04-16T17:04:21.505370+0000","last_peered":"2024-04-16T17:04:21.505370+0000","last_clean":"2024-04-16T17:04:21.505370+0000","last_became_active":"2024-04-16T17:03:29.255051+0000","last_became_peered":"2024-04-16T17:03:29.255051+0000","last_unstale":"2024-04-16T17:04:21.505370+0000","last_undegraded":"2024-04-16T17:04:21.505370+0000","last_fullsized":"2024-04-16T17:04:21.505370+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:24:04.946342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"90'1","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T17:04:21.504332+0000","last_change":"2024-04-16T17:03:24.154904+0000","last_active":"2024-04-16T17:04:21.504332+0000","last_peered":"2024-04-16T17:04:21.504332+0000","last_clean":"2024-04-16T17:04:21.504332+0000","last_became_active":"2024-04-16T17:03:24.154617+0000","last_became_peered":"2024-04-16T17:03:24.154617+0000","last_unstale":"2024-04-16T17:04:21.504332+0000","last_undegraded":"2024-04-16T17:04:21.504332+0000","last_fullsized":"2024-04-16T17:04:21.504332+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:35:55.498855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T17:04:21.514884+0000","last_change":"2024-04-16T17:03:24.979908+0000","last_active":"2024-04-16T17:04:21.514884+0000","last_peered":"2024-04-16T17:04:21.514884+0000","last_clean":"2024-04-16T17:04:21.514884+0000","last_became_active":"2024-04-16T17:03:24.979501+0000","last_became_peered":"2024-04-16T17:03:24.979501+0000","last_unstale":"2024-04-16T17:04:21.514884+0000","last_undegraded":"2024-04-16T17:04:21.514884+0000","last_fullsized":"2024-04-16T17:04:21.514884+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:51:05.421459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T17:04:21.500874+0000","last_change":"2024-04-16T17:03:29.258809+0000","last_active":"2024-04-16T17:04:21.500874+0000","last_peered":"2024-04-16T17:04:21.500874+0000","last_clean":"2024-04-16T17:04:21.500874+0000","last_became_active":"2024-04-16T17:03:29.256629+0000","last_became_peered":"2024-04-16T17:03:29.256629+0000","last_unstale":"2024-04-16T17:04:21.500874+0000","last_undegraded":"2024-04-16T17:04:21.500874+0000","last_fullsized":"2024-04-16T17:04:21.500874+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:46:51.634944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"90'3","reported_seq":116,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T17:05:17.509576+0000","last_change":"2024-04-16T17:03:24.153774+0000","last_active":"2024-04-16T17:05:17.509576+0000","last_peered":"2024-04-16T17:05:17.509576+0000","last_clean":"2024-04-16T17:05:17.509576+0000","last_became_active":"2024-04-16T17:03:24.153418+0000","last_became_peered":"2024-04-16T17:03:24.153418+0000","last_unstale":"2024-04-16T17:05:17.509576+0000","last_undegraded":"2024-04-16T17:05:17.509576+0000","last_fullsized":"2024-04-16T17:05:17.509576+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:58:36.093789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":86,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T17:04:21.880853+0000","last_change":"2024-04-16T17:03:24.962838+0000","last_active":"2024-04-16T17:04:21.880853+0000","last_peered":"2024-04-16T17:04:21.880853+0000","last_clean":"2024-04-16T17:04:21.880853+0000","last_became_active":"2024-04-16T17:03:24.960163+0000","last_became_peered":"2024-04-16T17:03:24.960163+0000","last_unstale":"2024-04-16T17:04:21.880853+0000","last_undegraded":"2024-04-16T17:04:21.880853+0000","last_fullsized":"2024-04-16T17:04:21.880853+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:48:18.723499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T17:04:21.514860+0000","last_change":"2024-04-16T17:03:29.251177+0000","last_active":"2024-04-16T17:04:21.514860+0000","last_peered":"2024-04-16T17:04:21.514860+0000","last_clean":"2024-04-16T17:04:21.514860+0000","last_became_active":"2024-04-16T17:03:29.250657+0000","last_became_peered":"2024-04-16T17:03:29.250657+0000","last_unstale":"2024-04-16T17:04:21.514860+0000","last_undegraded":"2024-04-16T17:04:21.514860+0000","last_fullsized":"2024-04-16T17:04:21.514860+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:16:47.217193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"90'2","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T17:04:21.501989+0000","last_change":"2024-04-16T17:03:24.154245+0000","last_active":"2024-04-16T17:04:21.501989+0000","last_peered":"2024-04-16T17:04:21.501989+0000","last_clean":"2024-04-16T17:04:21.501989+0000","last_became_active":"2024-04-16T17:03:24.153297+0000","last_became_peered":"2024-04-16T17:03:24.153297+0000","last_unstale":"2024-04-16T17:04:21.501989+0000","last_undegraded":"2024-04-16T17:04:21.501989+0000","last_fullsized":"2024-04-16T17:04:21.501989+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:23:49.490238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T17:04:21.502999+0000","last_change":"2024-04-16T17:03:24.984572+0000","last_active":"2024-04-16T17:04:21.502999+0000","last_peered":"2024-04-16T17:04:21.502999+0000","last_clean":"2024-04-16T17:04:21.502999+0000","last_became_active":"2024-04-16T17:03:24.983588+0000","last_became_peered":"2024-04-16T17:03:24.983588+0000","last_unstale":"2024-04-16T17:04:21.502999+0000","last_undegraded":"2024-04-16T17:04:21.502999+0000","last_fullsized":"2024-04-16T17:04:21.502999+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:41:24.704602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T17:04:22.147593+0000","last_change":"2024-04-16T17:03:29.245129+0000","last_active":"2024-04-16T17:04:22.147593+0000","last_peered":"2024-04-16T17:04:22.147593+0000","last_clean":"2024-04-16T17:04:22.147593+0000","last_became_active":"2024-04-16T17:03:29.244720+0000","last_became_peered":"2024-04-16T17:03:29.244720+0000","last_unstale":"2024-04-16T17:04:22.147593+0000","last_undegraded":"2024-04-16T17:04:22.147593+0000","last_fullsized":"2024-04-16T17:04:22.147593+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:55:26.612176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"90'4","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T17:04:21.505070+0000","last_change":"2024-04-16T17:03:24.165248+0000","last_active":"2024-04-16T17:04:21.505070+0000","last_peered":"2024-04-16T17:04:21.505070+0000","last_clean":"2024-04-16T17:04:21.505070+0000","last_became_active":"2024-04-16T17:03:24.164915+0000","last_became_peered":"2024-04-16T17:03:24.164915+0000","last_unstale":"2024-04-16T17:04:21.505070+0000","last_undegraded":"2024-04-16T17:04:21.505070+0000","last_fullsized":"2024-04-16T17:04:21.505070+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:04:47.207554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T17:04:21.881013+0000","last_change":"2024-04-16T17:03:24.967292+0000","last_active":"2024-04-16T17:04:21.881013+0000","last_peered":"2024-04-16T17:04:21.881013+0000","last_clean":"2024-04-16T17:04:21.881013+0000","last_became_active":"2024-04-16T17:03:24.966952+0000","last_became_peered":"2024-04-16T17:03:24.966952+0000","last_unstale":"2024-04-16T17:04:21.881013+0000","last_undegraded":"2024-04-16T17:04:21.881013+0000","last_fullsized":"2024-04-16T17:04:21.881013+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:42:54.969478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T17:04:21.514953+0000","last_change":"2024-04-16T17:03:29.257836+0000","last_active":"2024-04-16T17:04:21.514953+0000","last_peered":"2024-04-16T17:04:21.514953+0000","last_clean":"2024-04-16T17:04:21.514953+0000","last_became_active":"2024-04-16T17:03:29.257409+0000","last_became_peered":"2024-04-16T17:03:29.257409+0000","last_unstale":"2024-04-16T17:04:21.514953+0000","last_undegraded":"2024-04-16T17:04:21.514953+0000","last_fullsized":"2024-04-16T17:04:21.514953+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:25:21.288266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"92'33","reported_seq":241,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T17:05:18.510684+0000","last_change":"2024-04-16T17:03:24.150943+0000","last_active":"2024-04-16T17:05:18.510684+0000","last_peered":"2024-04-16T17:05:18.510684+0000","last_clean":"2024-04-16T17:05:18.510684+0000","last_became_active":"2024-04-16T17:03:24.150563+0000","last_became_peered":"2024-04-16T17:03:24.150563+0000","last_unstale":"2024-04-16T17:05:18.510684+0000","last_undegraded":"2024-04-16T17:05:18.510684+0000","last_fullsized":"2024-04-16T17:05:18.510684+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:05:07.380440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":63356,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":181,"num_read_kb":153,"num_write":34,"num_write_kb":77,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T17:04:21.502792+0000","last_change":"2024-04-16T17:03:24.966393+0000","last_active":"2024-04-16T17:04:21.502792+0000","last_peered":"2024-04-16T17:04:21.502792+0000","last_clean":"2024-04-16T17:04:21.502792+0000","last_became_active":"2024-04-16T17:03:24.966016+0000","last_became_peered":"2024-04-16T17:03:24.966016+0000","last_unstale":"2024-04-16T17:04:21.502792+0000","last_undegraded":"2024-04-16T17:04:21.502792+0000","last_fullsized":"2024-04-16T17:04:21.502792+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:30:25.203262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T17:04:21.505554+0000","last_change":"2024-04-16T17:03:29.265371+0000","last_active":"2024-04-16T17:04:21.505554+0000","last_peered":"2024-04-16T17:04:21.505554+0000","last_clean":"2024-04-16T17:04:21.505554+0000","last_became_active":"2024-04-16T17:03:29.264933+0000","last_became_peered":"2024-04-16T17:03:29.264933+0000","last_unstale":"2024-04-16T17:04:21.505554+0000","last_undegraded":"2024-04-16T17:04:21.505554+0000","last_fullsized":"2024-04-16T17:04:21.505554+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:50:18.203015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"90'6","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T17:04:21.527209+0000","last_change":"2024-04-16T17:04:21.526796+0000","last_active":"2024-04-16T17:04:21.527209+0000","last_peered":"2024-04-16T17:04:21.527209+0000","last_clean":"2024-04-16T17:04:21.527209+0000","last_became_active":"2024-04-16T17:04:21.508364+0000","last_became_peered":"2024-04-16T17:04:21.508364+0000","last_unstale":"2024-04-16T17:04:21.527209+0000","last_undegraded":"2024-04-16T17:04:21.527209+0000","last_fullsized":"2024-04-16T17:04:21.527209+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:45:42.726648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"92'9","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T17:05:20.162304+0000","last_change":"2024-04-16T17:03:24.986228+0000","last_active":"2024-04-16T17:05:20.162304+0000","last_peered":"2024-04-16T17:05:20.162304+0000","last_clean":"2024-04-16T17:05:20.162304+0000","last_became_active":"2024-04-16T17:03:24.985916+0000","last_became_peered":"2024-04-16T17:03:24.985916+0000","last_unstale":"2024-04-16T17:05:20.162304+0000","last_undegraded":"2024-04-16T17:05:20.162304+0000","last_fullsized":"2024-04-16T17:05:20.162304+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:23:54.309203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T17:04:21.499369+0000","last_change":"2024-04-16T17:03:29.264636+0000","last_active":"2024-04-16T17:04:21.499369+0000","last_peered":"2024-04-16T17:04:21.499369+0000","last_clean":"2024-04-16T17:04:21.499369+0000","last_became_active":"2024-04-16T17:03:29.264117+0000","last_became_peered":"2024-04-16T17:03:29.264117+0000","last_unstale":"2024-04-16T17:04:21.499369+0000","last_undegraded":"2024-04-16T17:04:21.499369+0000","last_fullsized":"2024-04-16T17:04:21.499369+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:42:36.123154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"90'3","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T17:04:21.505264+0000","last_change":"2024-04-16T17:03:24.166844+0000","last_active":"2024-04-16T17:04:21.505264+0000","last_peered":"2024-04-16T17:04:21.505264+0000","last_clean":"2024-04-16T17:04:21.505264+0000","last_became_active":"2024-04-16T17:03:24.166479+0000","last_became_peered":"2024-04-16T17:03:24.166479+0000","last_unstale":"2024-04-16T17:04:21.505264+0000","last_undegraded":"2024-04-16T17:04:21.505264+0000","last_fullsized":"2024-04-16T17:04:21.505264+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:02:20.139865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T17:04:21.502098+0000","last_change":"2024-04-16T17:03:24.973057+0000","last_active":"2024-04-16T17:04:21.502098+0000","last_peered":"2024-04-16T17:04:21.502098+0000","last_clean":"2024-04-16T17:04:21.502098+0000","last_became_active":"2024-04-16T17:03:24.972731+0000","last_became_peered":"2024-04-16T17:03:24.972731+0000","last_unstale":"2024-04-16T17:04:21.502098+0000","last_undegraded":"2024-04-16T17:04:21.502098+0000","last_fullsized":"2024-04-16T17:04:21.502098+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:37:38.702919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T17:04:21.519660+0000","last_change":"2024-04-16T17:03:29.259167+0000","last_active":"2024-04-16T17:04:21.519660+0000","last_peered":"2024-04-16T17:04:21.519660+0000","last_clean":"2024-04-16T17:04:21.519660+0000","last_became_active":"2024-04-16T17:03:29.258737+0000","last_became_peered":"2024-04-16T17:03:29.258737+0000","last_unstale":"2024-04-16T17:04:21.519660+0000","last_undegraded":"2024-04-16T17:04:21.519660+0000","last_fullsized":"2024-04-16T17:04:21.519660+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:01:13.774037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"90'4","reported_seq":117,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T17:05:14.568362+0000","last_change":"2024-04-16T17:03:24.151064+0000","last_active":"2024-04-16T17:05:14.568362+0000","last_peered":"2024-04-16T17:05:14.568362+0000","last_clean":"2024-04-16T17:05:14.568362+0000","last_became_active":"2024-04-16T17:03:24.150745+0000","last_became_peered":"2024-04-16T17:03:24.150745+0000","last_unstale":"2024-04-16T17:05:14.568362+0000","last_undegraded":"2024-04-16T17:05:14.568362+0000","last_fullsized":"2024-04-16T17:05:14.568362+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:32:26.271008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":87,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T17:04:21.501506+0000","last_change":"2024-04-16T17:03:24.971948+0000","last_active":"2024-04-16T17:04:21.501506+0000","last_peered":"2024-04-16T17:04:21.501506+0000","last_clean":"2024-04-16T17:04:21.501506+0000","last_became_active":"2024-04-16T17:03:24.971530+0000","last_became_peered":"2024-04-16T17:03:24.971530+0000","last_unstale":"2024-04-16T17:04:21.501506+0000","last_undegraded":"2024-04-16T17:04:21.501506+0000","last_fullsized":"2024-04-16T17:04:21.501506+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:03:29.286160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T17:04:22.199376+0000","last_change":"2024-04-16T17:03:29.264821+0000","last_active":"2024-04-16T17:04:22.199376+0000","last_peered":"2024-04-16T17:04:22.199376+0000","last_clean":"2024-04-16T17:04:22.199376+0000","last_became_active":"2024-04-16T17:03:29.264287+0000","last_became_peered":"2024-04-16T17:03:29.264287+0000","last_unstale":"2024-04-16T17:04:22.199376+0000","last_undegraded":"2024-04-16T17:04:22.199376+0000","last_fullsized":"2024-04-16T17:04:22.199376+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:33:28.400663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"90'15","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T17:04:21.503693+0000","last_change":"2024-04-16T17:03:24.158844+0000","last_active":"2024-04-16T17:04:21.503693+0000","last_peered":"2024-04-16T17:04:21.503693+0000","last_clean":"2024-04-16T17:04:21.503693+0000","last_became_active":"2024-04-16T17:03:24.158537+0000","last_became_peered":"2024-04-16T17:03:24.158537+0000","last_unstale":"2024-04-16T17:04:21.503693+0000","last_undegraded":"2024-04-16T17:04:21.503693+0000","last_fullsized":"2024-04-16T17:04:21.503693+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:42:22.772452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"92'6","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T17:05:20.151688+0000","last_change":"2024-04-16T17:03:24.962522+0000","last_active":"2024-04-16T17:05:20.151688+0000","last_peered":"2024-04-16T17:05:20.151688+0000","last_clean":"2024-04-16T17:05:20.151688+0000","last_became_active":"2024-04-16T17:03:24.962169+0000","last_became_peered":"2024-04-16T17:03:24.962169+0000","last_unstale":"2024-04-16T17:05:20.151688+0000","last_undegraded":"2024-04-16T17:05:20.151688+0000","last_fullsized":"2024-04-16T17:05:20.151688+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:28:17.147491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T17:04:21.504843+0000","last_change":"2024-04-16T17:03:29.259539+0000","last_active":"2024-04-16T17:04:21.504843+0000","last_peered":"2024-04-16T17:04:21.504843+0000","last_clean":"2024-04-16T17:04:21.504843+0000","last_became_active":"2024-04-16T17:03:29.259123+0000","last_became_peered":"2024-04-16T17:03:29.259123+0000","last_unstale":"2024-04-16T17:04:21.504843+0000","last_undegraded":"2024-04-16T17:04:21.504843+0000","last_fullsized":"2024-04-16T17:04:21.504843+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:15:17.142636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"90'2","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T17:04:21.502952+0000","last_change":"2024-04-16T17:03:24.147526+0000","last_active":"2024-04-16T17:04:21.502952+0000","last_peered":"2024-04-16T17:04:21.502952+0000","last_clean":"2024-04-16T17:04:21.502952+0000","last_became_active":"2024-04-16T17:03:24.147119+0000","last_became_peered":"2024-04-16T17:03:24.147119+0000","last_unstale":"2024-04-16T17:04:21.502952+0000","last_undegraded":"2024-04-16T17:04:21.502952+0000","last_fullsized":"2024-04-16T17:04:21.502952+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:30:38.742122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"92'5","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T17:05:20.144526+0000","last_change":"2024-04-16T17:04:21.507113+0000","last_active":"2024-04-16T17:05:20.144526+0000","last_peered":"2024-04-16T17:05:20.144526+0000","last_clean":"2024-04-16T17:05:20.144526+0000","last_became_active":"2024-04-16T17:04:21.506680+0000","last_became_peered":"2024-04-16T17:04:21.506680+0000","last_unstale":"2024-04-16T17:05:20.144526+0000","last_undegraded":"2024-04-16T17:05:20.144526+0000","last_fullsized":"2024-04-16T17:05:20.144526+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:08:47.971317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,8],"acting":[11,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T17:04:21.502671+0000","last_change":"2024-04-16T17:03:29.301135+0000","last_active":"2024-04-16T17:04:21.502671+0000","last_peered":"2024-04-16T17:04:21.502671+0000","last_clean":"2024-04-16T17:04:21.502671+0000","last_became_active":"2024-04-16T17:03:29.300672+0000","last_became_peered":"2024-04-16T17:03:29.300672+0000","last_unstale":"2024-04-16T17:04:21.502671+0000","last_undegraded":"2024-04-16T17:04:21.502671+0000","last_fullsized":"2024-04-16T17:04:21.502671+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:55:41.599984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T17:04:21.518291+0000","last_change":"2024-04-16T17:03:24.162105+0000","last_active":"2024-04-16T17:04:21.518291+0000","last_peered":"2024-04-16T17:04:21.518291+0000","last_clean":"2024-04-16T17:04:21.518291+0000","last_became_active":"2024-04-16T17:03:24.161631+0000","last_became_peered":"2024-04-16T17:03:24.161631+0000","last_unstale":"2024-04-16T17:04:21.518291+0000","last_undegraded":"2024-04-16T17:04:21.518291+0000","last_fullsized":"2024-04-16T17:04:21.518291+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:22:33.677848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T17:04:21.497711+0000","last_change":"2024-04-16T17:03:24.978253+0000","last_active":"2024-04-16T17:04:21.497711+0000","last_peered":"2024-04-16T17:04:21.497711+0000","last_clean":"2024-04-16T17:04:21.497711+0000","last_became_active":"2024-04-16T17:03:24.977956+0000","last_became_peered":"2024-04-16T17:03:24.977956+0000","last_unstale":"2024-04-16T17:04:21.497711+0000","last_undegraded":"2024-04-16T17:04:21.497711+0000","last_fullsized":"2024-04-16T17:04:21.497711+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:59:45.102383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T17:04:22.198914+0000","last_change":"2024-04-16T17:03:29.265312+0000","last_active":"2024-04-16T17:04:22.198914+0000","last_peered":"2024-04-16T17:04:22.198914+0000","last_clean":"2024-04-16T17:04:22.198914+0000","last_became_active":"2024-04-16T17:03:29.264891+0000","last_became_peered":"2024-04-16T17:03:29.264891+0000","last_unstale":"2024-04-16T17:04:22.198914+0000","last_undegraded":"2024-04-16T17:04:22.198914+0000","last_fullsized":"2024-04-16T17:04:22.198914+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:26:40.088127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"90'1","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T17:04:21.501679+0000","last_change":"2024-04-16T17:03:24.170935+0000","last_active":"2024-04-16T17:04:21.501679+0000","last_peered":"2024-04-16T17:04:21.501679+0000","last_clean":"2024-04-16T17:04:21.501679+0000","last_became_active":"2024-04-16T17:03:24.170609+0000","last_became_peered":"2024-04-16T17:03:24.170609+0000","last_unstale":"2024-04-16T17:04:21.501679+0000","last_undegraded":"2024-04-16T17:04:21.501679+0000","last_fullsized":"2024-04-16T17:04:21.501679+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:18:33.516309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T17:04:21.497654+0000","last_change":"2024-04-16T17:03:24.985434+0000","last_active":"2024-04-16T17:04:21.497654+0000","last_peered":"2024-04-16T17:04:21.497654+0000","last_clean":"2024-04-16T17:04:21.497654+0000","last_became_active":"2024-04-16T17:03:24.984388+0000","last_became_peered":"2024-04-16T17:03:24.984388+0000","last_unstale":"2024-04-16T17:04:21.497654+0000","last_undegraded":"2024-04-16T17:04:21.497654+0000","last_fullsized":"2024-04-16T17:04:21.497654+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:43:26.355572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T17:04:22.199112+0000","last_change":"2024-04-16T17:03:29.301455+0000","last_active":"2024-04-16T17:04:22.199112+0000","last_peered":"2024-04-16T17:04:22.199112+0000","last_clean":"2024-04-16T17:04:22.199112+0000","last_became_active":"2024-04-16T17:03:29.301028+0000","last_became_peered":"2024-04-16T17:03:29.301028+0000","last_unstale":"2024-04-16T17:04:22.199112+0000","last_undegraded":"2024-04-16T17:04:22.199112+0000","last_fullsized":"2024-04-16T17:04:22.199112+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:54:49.305379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"92'8","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T17:05:20.151834+0000","last_change":"2024-04-16T17:03:24.962786+0000","last_active":"2024-04-16T17:05:20.151834+0000","last_peered":"2024-04-16T17:05:20.151834+0000","last_clean":"2024-04-16T17:05:20.151834+0000","last_became_active":"2024-04-16T17:03:24.961704+0000","last_became_peered":"2024-04-16T17:03:24.961704+0000","last_unstale":"2024-04-16T17:05:20.151834+0000","last_undegraded":"2024-04-16T17:05:20.151834+0000","last_fullsized":"2024-04-16T17:05:20.151834+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:22:27.957607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"92'5","reported_seq":120,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T17:05:20.132338+0000","last_change":"2024-04-16T17:03:24.175617+0000","last_active":"2024-04-16T17:05:20.132338+0000","last_peered":"2024-04-16T17:05:20.132338+0000","last_clean":"2024-04-16T17:05:20.132338+0000","last_became_active":"2024-04-16T17:03:24.175272+0000","last_became_peered":"2024-04-16T17:03:24.175272+0000","last_unstale":"2024-04-16T17:05:20.132338+0000","last_undegraded":"2024-04-16T17:05:20.132338+0000","last_fullsized":"2024-04-16T17:05:20.132338+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:27:44.738030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":88,"num_read_kb":88,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"90'1","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T17:04:21.512033+0000","last_change":"2024-04-16T17:04:21.512033+0000","last_active":"2024-04-16T17:04:21.512033+0000","last_peered":"2024-04-16T17:04:21.512033+0000","last_clean":"2024-04-16T17:04:21.512033+0000","last_became_active":"2024-04-16T17:04:21.506718+0000","last_became_peered":"2024-04-16T17:04:21.506718+0000","last_unstale":"2024-04-16T17:04:21.512033+0000","last_undegraded":"2024-04-16T17:04:21.512033+0000","last_fullsized":"2024-04-16T17:04:21.512033+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:53:48.953887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,10],"acting":[7,5,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T17:04:21.501025+0000","last_change":"2024-04-16T17:03:24.983629+0000","last_active":"2024-04-16T17:04:21.501025+0000","last_peered":"2024-04-16T17:04:21.501025+0000","last_clean":"2024-04-16T17:04:21.501025+0000","last_became_active":"2024-04-16T17:03:24.979969+0000","last_became_peered":"2024-04-16T17:03:24.979969+0000","last_unstale":"2024-04-16T17:04:21.501025+0000","last_undegraded":"2024-04-16T17:04:21.501025+0000","last_fullsized":"2024-04-16T17:04:21.501025+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:01:24.391732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"90'3","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T17:04:22.147596+0000","last_change":"2024-04-16T17:03:24.152475+0000","last_active":"2024-04-16T17:04:22.147596+0000","last_peered":"2024-04-16T17:04:22.147596+0000","last_clean":"2024-04-16T17:04:22.147596+0000","last_became_active":"2024-04-16T17:03:24.152198+0000","last_became_peered":"2024-04-16T17:03:24.152198+0000","last_unstale":"2024-04-16T17:04:22.147596+0000","last_undegraded":"2024-04-16T17:04:22.147596+0000","last_fullsized":"2024-04-16T17:04:22.147596+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:49:33.914325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":360,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T17:04:21.501447+0000","last_change":"2024-04-16T17:02:02.111335+0000","last_active":"2024-04-16T17:04:21.501447+0000","last_peered":"2024-04-16T17:04:21.501447+0000","last_clean":"2024-04-16T17:04:21.501447+0000","last_became_active":"2024-04-16T17:02:02.110928+0000","last_became_peered":"2024-04-16T17:02:02.110928+0000","last_unstale":"2024-04-16T17:04:21.501447+0000","last_undegraded":"2024-04-16T17:04:21.501447+0000","last_fullsized":"2024-04-16T17:04:21.501447+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T16:58:07.081753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T16:58:07.081753+0000","last_clean_scrub_stamp":"2024-04-16T16:58:07.081753+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:27:02.865664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T17:04:21.505247+0000","last_change":"2024-04-16T17:03:29.301691+0000","last_active":"2024-04-16T17:04:21.505247+0000","last_peered":"2024-04-16T17:04:21.505247+0000","last_clean":"2024-04-16T17:04:21.505247+0000","last_became_active":"2024-04-16T17:03:29.301277+0000","last_became_peered":"2024-04-16T17:03:29.301277+0000","last_unstale":"2024-04-16T17:04:21.505247+0000","last_undegraded":"2024-04-16T17:04:21.505247+0000","last_fullsized":"2024-04-16T17:04:21.505247+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:45:16.727610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"92'5","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T17:05:20.150650+0000","last_change":"2024-04-16T17:03:24.970939+0000","last_active":"2024-04-16T17:05:20.150650+0000","last_peered":"2024-04-16T17:05:20.150650+0000","last_clean":"2024-04-16T17:05:20.150650+0000","last_became_active":"2024-04-16T17:03:24.970581+0000","last_became_peered":"2024-04-16T17:03:24.970581+0000","last_unstale":"2024-04-16T17:05:20.150650+0000","last_undegraded":"2024-04-16T17:05:20.150650+0000","last_fullsized":"2024-04-16T17:05:20.150650+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:33:07.448597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"90'3","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T17:04:21.501457+0000","last_change":"2024-04-16T17:03:24.155454+0000","last_active":"2024-04-16T17:04:21.501457+0000","last_peered":"2024-04-16T17:04:21.501457+0000","last_clean":"2024-04-16T17:04:21.501457+0000","last_became_active":"2024-04-16T17:03:24.155155+0000","last_became_peered":"2024-04-16T17:03:24.155155+0000","last_unstale":"2024-04-16T17:04:21.501457+0000","last_undegraded":"2024-04-16T17:04:21.501457+0000","last_fullsized":"2024-04-16T17:04:21.501457+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:34:08.618002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T17:04:21.500178+0000","last_change":"2024-04-16T17:03:29.255538+0000","last_active":"2024-04-16T17:04:21.500178+0000","last_peered":"2024-04-16T17:04:21.500178+0000","last_clean":"2024-04-16T17:04:21.500178+0000","last_became_active":"2024-04-16T17:03:29.255107+0000","last_became_peered":"2024-04-16T17:03:29.255107+0000","last_unstale":"2024-04-16T17:04:21.500178+0000","last_undegraded":"2024-04-16T17:04:21.500178+0000","last_fullsized":"2024-04-16T17:04:21.500178+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:31:58.458641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"90'1","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T17:04:21.502646+0000","last_change":"2024-04-16T17:03:24.158816+0000","last_active":"2024-04-16T17:04:21.502646+0000","last_peered":"2024-04-16T17:04:21.502646+0000","last_clean":"2024-04-16T17:04:21.502646+0000","last_became_active":"2024-04-16T17:03:24.158518+0000","last_became_peered":"2024-04-16T17:03:24.158518+0000","last_unstale":"2024-04-16T17:04:21.502646+0000","last_undegraded":"2024-04-16T17:04:21.502646+0000","last_fullsized":"2024-04-16T17:04:21.502646+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:54:55.807975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"92'9","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T17:05:20.149239+0000","last_change":"2024-04-16T17:03:24.977190+0000","last_active":"2024-04-16T17:05:20.149239+0000","last_peered":"2024-04-16T17:05:20.149239+0000","last_clean":"2024-04-16T17:05:20.149239+0000","last_became_active":"2024-04-16T17:03:24.976840+0000","last_became_peered":"2024-04-16T17:03:24.976840+0000","last_unstale":"2024-04-16T17:05:20.149239+0000","last_undegraded":"2024-04-16T17:05:20.149239+0000","last_fullsized":"2024-04-16T17:05:20.149239+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:21:51.294150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T17:04:21.500534+0000","last_change":"2024-04-16T17:03:29.301423+0000","last_active":"2024-04-16T17:04:21.500534+0000","last_peered":"2024-04-16T17:04:21.500534+0000","last_clean":"2024-04-16T17:04:21.500534+0000","last_became_active":"2024-04-16T17:03:29.300888+0000","last_became_peered":"2024-04-16T17:03:29.300888+0000","last_unstale":"2024-04-16T17:04:21.500534+0000","last_undegraded":"2024-04-16T17:04:21.500534+0000","last_fullsized":"2024-04-16T17:04:21.500534+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:13:54.188090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"90'10","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T17:04:21.519592+0000","last_change":"2024-04-16T17:03:24.167643+0000","last_active":"2024-04-16T17:04:21.519592+0000","last_peered":"2024-04-16T17:04:21.519592+0000","last_clean":"2024-04-16T17:04:21.519592+0000","last_became_active":"2024-04-16T17:03:24.167103+0000","last_became_peered":"2024-04-16T17:03:24.167103+0000","last_unstale":"2024-04-16T17:04:21.519592+0000","last_undegraded":"2024-04-16T17:04:21.519592+0000","last_fullsized":"2024-04-16T17:04:21.519592+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:33:16.809371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T17:04:21.505980+0000","last_change":"2024-04-16T17:03:24.976981+0000","last_active":"2024-04-16T17:04:21.505980+0000","last_peered":"2024-04-16T17:04:21.505980+0000","last_clean":"2024-04-16T17:04:21.505980+0000","last_became_active":"2024-04-16T17:03:24.975385+0000","last_became_peered":"2024-04-16T17:03:24.975385+0000","last_unstale":"2024-04-16T17:04:21.505980+0000","last_undegraded":"2024-04-16T17:04:21.505980+0000","last_fullsized":"2024-04-16T17:04:21.505980+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:27:03.104838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T17:04:21.501741+0000","last_change":"2024-04-16T17:03:29.244767+0000","last_active":"2024-04-16T17:04:21.501741+0000","last_peered":"2024-04-16T17:04:21.501741+0000","last_clean":"2024-04-16T17:04:21.501741+0000","last_became_active":"2024-04-16T17:03:29.244205+0000","last_became_peered":"2024-04-16T17:03:29.244205+0000","last_unstale":"2024-04-16T17:04:21.501741+0000","last_undegraded":"2024-04-16T17:04:21.501741+0000","last_fullsized":"2024-04-16T17:04:21.501741+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:51:46.652387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"90'4","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T17:04:22.147868+0000","last_change":"2024-04-16T17:03:24.164916+0000","last_active":"2024-04-16T17:04:22.147868+0000","last_peered":"2024-04-16T17:04:22.147868+0000","last_clean":"2024-04-16T17:04:22.147868+0000","last_became_active":"2024-04-16T17:03:24.164648+0000","last_became_peered":"2024-04-16T17:03:24.164648+0000","last_unstale":"2024-04-16T17:04:22.147868+0000","last_undegraded":"2024-04-16T17:04:22.147868+0000","last_fullsized":"2024-04-16T17:04:22.147868+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:23:00.198785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T17:04:21.505965+0000","last_change":"2024-04-16T17:03:24.994393+0000","last_active":"2024-04-16T17:04:21.505965+0000","last_peered":"2024-04-16T17:04:21.505965+0000","last_clean":"2024-04-16T17:04:21.505965+0000","last_became_active":"2024-04-16T17:03:24.994002+0000","last_became_peered":"2024-04-16T17:03:24.994002+0000","last_unstale":"2024-04-16T17:04:21.505965+0000","last_undegraded":"2024-04-16T17:04:21.505965+0000","last_fullsized":"2024-04-16T17:04:21.505965+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:23:44.596781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T17:04:22.147612+0000","last_change":"2024-04-16T17:03:29.302009+0000","last_active":"2024-04-16T17:04:22.147612+0000","last_peered":"2024-04-16T17:04:22.147612+0000","last_clean":"2024-04-16T17:04:22.147612+0000","last_became_active":"2024-04-16T17:03:29.301183+0000","last_became_peered":"2024-04-16T17:03:29.301183+0000","last_unstale":"2024-04-16T17:04:22.147612+0000","last_undegraded":"2024-04-16T17:04:22.147612+0000","last_fullsized":"2024-04-16T17:04:22.147612+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:30:19.561875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"90'2","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T17:04:21.880364+0000","last_change":"2024-04-16T17:03:24.176829+0000","last_active":"2024-04-16T17:04:21.880364+0000","last_peered":"2024-04-16T17:04:21.880364+0000","last_clean":"2024-04-16T17:04:21.880364+0000","last_became_active":"2024-04-16T17:03:24.176555+0000","last_became_peered":"2024-04-16T17:03:24.176555+0000","last_unstale":"2024-04-16T17:04:21.880364+0000","last_undegraded":"2024-04-16T17:04:21.880364+0000","last_fullsized":"2024-04-16T17:04:21.880364+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:05:40.602150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T17:04:21.502854+0000","last_change":"2024-04-16T17:03:24.965746+0000","last_active":"2024-04-16T17:04:21.502854+0000","last_peered":"2024-04-16T17:04:21.502854+0000","last_clean":"2024-04-16T17:04:21.502854+0000","last_became_active":"2024-04-16T17:03:24.965241+0000","last_became_peered":"2024-04-16T17:03:24.965241+0000","last_unstale":"2024-04-16T17:04:21.502854+0000","last_undegraded":"2024-04-16T17:04:21.502854+0000","last_fullsized":"2024-04-16T17:04:21.502854+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:51:34.407565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T17:04:21.520587+0000","last_change":"2024-04-16T17:03:29.261441+0000","last_active":"2024-04-16T17:04:21.520587+0000","last_peered":"2024-04-16T17:04:21.520587+0000","last_clean":"2024-04-16T17:04:21.520587+0000","last_became_active":"2024-04-16T17:03:29.259538+0000","last_became_peered":"2024-04-16T17:03:29.259538+0000","last_unstale":"2024-04-16T17:04:21.520587+0000","last_undegraded":"2024-04-16T17:04:21.520587+0000","last_fullsized":"2024-04-16T17:04:21.520587+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:12:27.092053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"90'3","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T17:04:21.501265+0000","last_change":"2024-04-16T17:03:24.172053+0000","last_active":"2024-04-16T17:04:21.501265+0000","last_peered":"2024-04-16T17:04:21.501265+0000","last_clean":"2024-04-16T17:04:21.501265+0000","last_became_active":"2024-04-16T17:03:24.171709+0000","last_became_peered":"2024-04-16T17:03:24.171709+0000","last_unstale":"2024-04-16T17:04:21.501265+0000","last_undegraded":"2024-04-16T17:04:21.501265+0000","last_fullsized":"2024-04-16T17:04:21.501265+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:40:12.307756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"92'9","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T17:05:20.147345+0000","last_change":"2024-04-16T17:03:24.984267+0000","last_active":"2024-04-16T17:05:20.147345+0000","last_peered":"2024-04-16T17:05:20.147345+0000","last_clean":"2024-04-16T17:05:20.147345+0000","last_became_active":"2024-04-16T17:03:24.983888+0000","last_became_peered":"2024-04-16T17:03:24.983888+0000","last_unstale":"2024-04-16T17:05:20.147345+0000","last_undegraded":"2024-04-16T17:05:20.147345+0000","last_fullsized":"2024-04-16T17:05:20.147345+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:16:23.814917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"90'1","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T17:04:22.147835+0000","last_change":"2024-04-16T17:03:29.251716+0000","last_active":"2024-04-16T17:04:22.147835+0000","last_peered":"2024-04-16T17:04:22.147835+0000","last_clean":"2024-04-16T17:04:22.147835+0000","last_became_active":"2024-04-16T17:03:29.251317+0000","last_became_peered":"2024-04-16T17:03:29.251317+0000","last_unstale":"2024-04-16T17:04:22.147835+0000","last_undegraded":"2024-04-16T17:04:22.147835+0000","last_fullsized":"2024-04-16T17:04:22.147835+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:48:54.680195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"90'1","reported_seq":15,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T17:04:21.515765+0000","last_change":"2024-04-16T17:04:21.515306+0000","last_active":"2024-04-16T17:04:21.515765+0000","last_peered":"2024-04-16T17:04:21.515765+0000","last_clean":"2024-04-16T17:04:21.515765+0000","last_became_active":"2024-04-16T17:04:21.508377+0000","last_became_peered":"2024-04-16T17:04:21.508377+0000","last_unstale":"2024-04-16T17:04:21.515765+0000","last_undegraded":"2024-04-16T17:04:21.515765+0000","last_fullsized":"2024-04-16T17:04:21.515765+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:28:25.000406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T17:04:21.500332+0000","last_change":"2024-04-16T17:03:24.995547+0000","last_active":"2024-04-16T17:04:21.500332+0000","last_peered":"2024-04-16T17:04:21.500332+0000","last_clean":"2024-04-16T17:04:21.500332+0000","last_became_active":"2024-04-16T17:03:24.995096+0000","last_became_peered":"2024-04-16T17:03:24.995096+0000","last_unstale":"2024-04-16T17:04:21.500332+0000","last_undegraded":"2024-04-16T17:04:21.500332+0000","last_fullsized":"2024-04-16T17:04:21.500332+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:54:51.193117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T17:04:22.147757+0000","last_change":"2024-04-16T17:03:29.301526+0000","last_active":"2024-04-16T17:04:22.147757+0000","last_peered":"2024-04-16T17:04:22.147757+0000","last_clean":"2024-04-16T17:04:22.147757+0000","last_became_active":"2024-04-16T17:03:29.300808+0000","last_became_peered":"2024-04-16T17:03:29.300808+0000","last_unstale":"2024-04-16T17:04:22.147757+0000","last_undegraded":"2024-04-16T17:04:22.147757+0000","last_fullsized":"2024-04-16T17:04:22.147757+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:01:14.717548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"90'4","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T17:04:21.503006+0000","last_change":"2024-04-16T17:03:24.152645+0000","last_active":"2024-04-16T17:04:21.503006+0000","last_peered":"2024-04-16T17:04:21.503006+0000","last_clean":"2024-04-16T17:04:21.503006+0000","last_became_active":"2024-04-16T17:03:24.152281+0000","last_became_peered":"2024-04-16T17:03:24.152281+0000","last_unstale":"2024-04-16T17:04:21.503006+0000","last_undegraded":"2024-04-16T17:04:21.503006+0000","last_fullsized":"2024-04-16T17:04:21.503006+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:33:51.377421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T17:04:21.501980+0000","last_change":"2024-04-16T17:03:24.979827+0000","last_active":"2024-04-16T17:04:21.501980+0000","last_peered":"2024-04-16T17:04:21.501980+0000","last_clean":"2024-04-16T17:04:21.501980+0000","last_became_active":"2024-04-16T17:03:24.979452+0000","last_became_peered":"2024-04-16T17:03:24.979452+0000","last_unstale":"2024-04-16T17:04:21.501980+0000","last_undegraded":"2024-04-16T17:04:21.501980+0000","last_fullsized":"2024-04-16T17:04:21.501980+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:07:09.259366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T17:04:21.880461+0000","last_change":"2024-04-16T17:03:29.255519+0000","last_active":"2024-04-16T17:04:21.880461+0000","last_peered":"2024-04-16T17:04:21.880461+0000","last_clean":"2024-04-16T17:04:21.880461+0000","last_became_active":"2024-04-16T17:03:29.255104+0000","last_became_peered":"2024-04-16T17:03:29.255104+0000","last_unstale":"2024-04-16T17:04:21.880461+0000","last_undegraded":"2024-04-16T17:04:21.880461+0000","last_fullsized":"2024-04-16T17:04:21.880461+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:27:34.496199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"90'3","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T17:04:21.502127+0000","last_change":"2024-04-16T17:03:24.161145+0000","last_active":"2024-04-16T17:04:21.502127+0000","last_peered":"2024-04-16T17:04:21.502127+0000","last_clean":"2024-04-16T17:04:21.502127+0000","last_became_active":"2024-04-16T17:03:24.160824+0000","last_became_peered":"2024-04-16T17:03:24.160824+0000","last_unstale":"2024-04-16T17:04:21.502127+0000","last_undegraded":"2024-04-16T17:04:21.502127+0000","last_fullsized":"2024-04-16T17:04:21.502127+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:02:48.805638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T17:04:21.502579+0000","last_change":"2024-04-16T17:03:24.966513+0000","last_active":"2024-04-16T17:04:21.502579+0000","last_peered":"2024-04-16T17:04:21.502579+0000","last_clean":"2024-04-16T17:04:21.502579+0000","last_became_active":"2024-04-16T17:03:24.966074+0000","last_became_peered":"2024-04-16T17:03:24.966074+0000","last_unstale":"2024-04-16T17:04:21.502579+0000","last_undegraded":"2024-04-16T17:04:21.502579+0000","last_fullsized":"2024-04-16T17:04:21.502579+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:50:21.579901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T17:04:22.150786+0000","last_change":"2024-04-16T17:04:22.150786+0000","last_active":"2024-04-16T17:04:22.150786+0000","last_peered":"2024-04-16T17:04:22.150786+0000","last_clean":"2024-04-16T17:04:22.150786+0000","last_became_active":"2024-04-16T17:04:22.149344+0000","last_became_peered":"2024-04-16T17:04:22.149344+0000","last_unstale":"2024-04-16T17:04:22.150786+0000","last_undegraded":"2024-04-16T17:04:22.150786+0000","last_fullsized":"2024-04-16T17:04:22.150786+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:18:49.421095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,1,4],"acting":[0,8,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"85'1","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T17:04:21.505354+0000","last_change":"2024-04-16T17:03:24.158032+0000","last_active":"2024-04-16T17:04:21.505354+0000","last_peered":"2024-04-16T17:04:21.505354+0000","last_clean":"2024-04-16T17:04:21.505354+0000","last_became_active":"2024-04-16T17:03:24.157689+0000","last_became_peered":"2024-04-16T17:03:24.157689+0000","last_unstale":"2024-04-16T17:04:21.505354+0000","last_undegraded":"2024-04-16T17:04:21.505354+0000","last_fullsized":"2024-04-16T17:04:21.505354+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:38:18.240889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T17:04:21.519999+0000","last_change":"2024-04-16T17:04:21.519999+0000","last_active":"2024-04-16T17:04:21.519999+0000","last_peered":"2024-04-16T17:04:21.519999+0000","last_clean":"2024-04-16T17:04:21.519999+0000","last_became_active":"2024-04-16T17:04:21.519621+0000","last_became_peered":"2024-04-16T17:04:21.519621+0000","last_unstale":"2024-04-16T17:04:21.519999+0000","last_undegraded":"2024-04-16T17:04:21.519999+0000","last_fullsized":"2024-04-16T17:04:21.519999+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:50:13.266369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T17:04:21.504113+0000","last_change":"2024-04-16T17:03:29.245176+0000","last_active":"2024-04-16T17:04:21.504113+0000","last_peered":"2024-04-16T17:04:21.504113+0000","last_clean":"2024-04-16T17:04:21.504113+0000","last_became_active":"2024-04-16T17:03:29.244763+0000","last_became_peered":"2024-04-16T17:03:29.244763+0000","last_unstale":"2024-04-16T17:04:21.504113+0000","last_undegraded":"2024-04-16T17:04:21.504113+0000","last_fullsized":"2024-04-16T17:04:21.504113+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:52:30.381417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T17:04:22.199271+0000","last_change":"2024-04-16T17:03:24.168068+0000","last_active":"2024-04-16T17:04:22.199271+0000","last_peered":"2024-04-16T17:04:22.199271+0000","last_clean":"2024-04-16T17:04:22.199271+0000","last_became_active":"2024-04-16T17:03:24.167694+0000","last_became_peered":"2024-04-16T17:03:24.167694+0000","last_unstale":"2024-04-16T17:04:22.199271+0000","last_undegraded":"2024-04-16T17:04:22.199271+0000","last_fullsized":"2024-04-16T17:04:22.199271+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:13:35.692306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T17:04:21.502355+0000","last_change":"2024-04-16T17:03:24.966145+0000","last_active":"2024-04-16T17:04:21.502355+0000","last_peered":"2024-04-16T17:04:21.502355+0000","last_clean":"2024-04-16T17:04:21.502355+0000","last_became_active":"2024-04-16T17:03:24.965813+0000","last_became_peered":"2024-04-16T17:03:24.965813+0000","last_unstale":"2024-04-16T17:04:21.502355+0000","last_undegraded":"2024-04-16T17:04:21.502355+0000","last_fullsized":"2024-04-16T17:04:21.502355+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:43:43.224130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"92'9","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T17:05:20.145157+0000","last_change":"2024-04-16T17:03:24.970941+0000","last_active":"2024-04-16T17:05:20.145157+0000","last_peered":"2024-04-16T17:05:20.145157+0000","last_clean":"2024-04-16T17:05:20.145157+0000","last_became_active":"2024-04-16T17:03:24.970587+0000","last_became_peered":"2024-04-16T17:03:24.970587+0000","last_unstale":"2024-04-16T17:05:20.145157+0000","last_undegraded":"2024-04-16T17:05:20.145157+0000","last_fullsized":"2024-04-16T17:05:20.145157+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:14:13.930011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T17:04:21.502798+0000","last_change":"2024-04-16T17:03:24.967167+0000","last_active":"2024-04-16T17:04:21.502798+0000","last_peered":"2024-04-16T17:04:21.502798+0000","last_clean":"2024-04-16T17:04:21.502798+0000","last_became_active":"2024-04-16T17:03:24.966787+0000","last_became_peered":"2024-04-16T17:03:24.966787+0000","last_unstale":"2024-04-16T17:04:21.502798+0000","last_undegraded":"2024-04-16T17:04:21.502798+0000","last_fullsized":"2024-04-16T17:04:21.502798+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:51:39.510220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T17:04:21.502859+0000","last_change":"2024-04-16T17:03:24.981115+0000","last_active":"2024-04-16T17:04:21.502859+0000","last_peered":"2024-04-16T17:04:21.502859+0000","last_clean":"2024-04-16T17:04:21.502859+0000","last_became_active":"2024-04-16T17:03:24.980685+0000","last_became_peered":"2024-04-16T17:03:24.980685+0000","last_unstale":"2024-04-16T17:04:21.502859+0000","last_undegraded":"2024-04-16T17:04:21.502859+0000","last_fullsized":"2024-04-16T17:04:21.502859+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:20:44.066932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T17:04:21.504785+0000","last_change":"2024-04-16T17:03:24.990421+0000","last_active":"2024-04-16T17:04:21.504785+0000","last_peered":"2024-04-16T17:04:21.504785+0000","last_clean":"2024-04-16T17:04:21.504785+0000","last_became_active":"2024-04-16T17:03:24.989940+0000","last_became_peered":"2024-04-16T17:03:24.989940+0000","last_unstale":"2024-04-16T17:04:21.504785+0000","last_undegraded":"2024-04-16T17:04:21.504785+0000","last_fullsized":"2024-04-16T17:04:21.504785+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:19:57.548067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"92'4","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T17:05:20.157624+0000","last_change":"2024-04-16T17:03:24.991854+0000","last_active":"2024-04-16T17:05:20.157624+0000","last_peered":"2024-04-16T17:05:20.157624+0000","last_clean":"2024-04-16T17:05:20.157624+0000","last_became_active":"2024-04-16T17:03:24.990964+0000","last_became_peered":"2024-04-16T17:03:24.990964+0000","last_unstale":"2024-04-16T17:05:20.157624+0000","last_undegraded":"2024-04-16T17:05:20.157624+0000","last_fullsized":"2024-04-16T17:05:20.157624+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:10:10.618708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"92'6","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T17:05:20.141704+0000","last_change":"2024-04-16T17:04:21.517466+0000","last_active":"2024-04-16T17:05:20.141704+0000","last_peered":"2024-04-16T17:05:20.141704+0000","last_clean":"2024-04-16T17:05:20.141704+0000","last_became_active":"2024-04-16T17:04:21.517086+0000","last_became_peered":"2024-04-16T17:04:21.517086+0000","last_unstale":"2024-04-16T17:05:20.141704+0000","last_undegraded":"2024-04-16T17:05:20.141704+0000","last_fullsized":"2024-04-16T17:05:20.141704+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:54:33.417452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,9],"acting":[11,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T17:04:21.502644+0000","last_change":"2024-04-16T17:03:24.970377+0000","last_active":"2024-04-16T17:04:21.502644+0000","last_peered":"2024-04-16T17:04:21.502644+0000","last_clean":"2024-04-16T17:04:21.502644+0000","last_became_active":"2024-04-16T17:03:24.968856+0000","last_became_peered":"2024-04-16T17:03:24.968856+0000","last_unstale":"2024-04-16T17:04:21.502644+0000","last_undegraded":"2024-04-16T17:04:21.502644+0000","last_fullsized":"2024-04-16T17:04:21.502644+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:47:28.543120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T17:04:21.506256+0000","last_change":"2024-04-16T17:03:24.992877+0000","last_active":"2024-04-16T17:04:21.506256+0000","last_peered":"2024-04-16T17:04:21.506256+0000","last_clean":"2024-04-16T17:04:21.506256+0000","last_became_active":"2024-04-16T17:03:24.992484+0000","last_became_peered":"2024-04-16T17:03:24.992484+0000","last_unstale":"2024-04-16T17:04:21.506256+0000","last_undegraded":"2024-04-16T17:04:21.506256+0000","last_fullsized":"2024-04-16T17:04:21.506256+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:00:45.204782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"92'6","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T17:05:20.171924+0000","last_change":"2024-04-16T17:03:24.977794+0000","last_active":"2024-04-16T17:05:20.171924+0000","last_peered":"2024-04-16T17:05:20.171924+0000","last_clean":"2024-04-16T17:05:20.171924+0000","last_became_active":"2024-04-16T17:03:24.977434+0000","last_became_peered":"2024-04-16T17:03:24.977434+0000","last_unstale":"2024-04-16T17:05:20.171924+0000","last_undegraded":"2024-04-16T17:05:20.171924+0000","last_fullsized":"2024-04-16T17:05:20.171924+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:46:35.670357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T17:04:21.501616+0000","last_change":"2024-04-16T17:03:24.992225+0000","last_active":"2024-04-16T17:04:21.501616+0000","last_peered":"2024-04-16T17:04:21.501616+0000","last_clean":"2024-04-16T17:04:21.501616+0000","last_became_active":"2024-04-16T17:03:24.991904+0000","last_became_peered":"2024-04-16T17:03:24.991904+0000","last_unstale":"2024-04-16T17:04:21.501616+0000","last_undegraded":"2024-04-16T17:04:21.501616+0000","last_fullsized":"2024-04-16T17:04:21.501616+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:01:09.710698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T17:04:21.880876+0000","last_change":"2024-04-16T17:03:24.981322+0000","last_active":"2024-04-16T17:04:21.880876+0000","last_peered":"2024-04-16T17:04:21.880876+0000","last_clean":"2024-04-16T17:04:21.880876+0000","last_became_active":"2024-04-16T17:03:24.980920+0000","last_became_peered":"2024-04-16T17:03:24.980920+0000","last_unstale":"2024-04-16T17:04:21.880876+0000","last_undegraded":"2024-04-16T17:04:21.880876+0000","last_fullsized":"2024-04-16T17:04:21.880876+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:07:59.378512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"92'11","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T17:05:20.175123+0000","last_change":"2024-04-16T17:03:24.990117+0000","last_active":"2024-04-16T17:05:20.175123+0000","last_peered":"2024-04-16T17:05:20.175123+0000","last_clean":"2024-04-16T17:05:20.175123+0000","last_became_active":"2024-04-16T17:03:24.989734+0000","last_became_peered":"2024-04-16T17:03:24.989734+0000","last_unstale":"2024-04-16T17:05:20.175123+0000","last_undegraded":"2024-04-16T17:05:20.175123+0000","last_fullsized":"2024-04-16T17:05:20.175123+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:36:55.000702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"92'8","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T17:05:20.140582+0000","last_change":"2024-04-16T17:03:24.970032+0000","last_active":"2024-04-16T17:05:20.140582+0000","last_peered":"2024-04-16T17:05:20.140582+0000","last_clean":"2024-04-16T17:05:20.140582+0000","last_became_active":"2024-04-16T17:03:24.968608+0000","last_became_peered":"2024-04-16T17:03:24.968608+0000","last_unstale":"2024-04-16T17:05:20.140582+0000","last_undegraded":"2024-04-16T17:05:20.140582+0000","last_fullsized":"2024-04-16T17:05:20.140582+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:24:03.918418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":144,"ondisk_log_size":144,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":283858,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":67889,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":545,"num_read_kb":347,"num_write":168,"num_write_kb":120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":616,"num_keys_recovered":20,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":203699,"data_compressed":4887,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":145,"ondisk_log_size":145,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351013,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":1007616,"data_stored":503892,"data_compressed":20691,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612652,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32300,"kb_used_data":1124,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712852,"statfs":{"total":95995035648,"available":95961960448,"internally_reserved":0,"allocated":1150976,"data_stored":1158969,"data_compressed":58418,"data_compressed_allocated":77824,"data_compressed_original":655360,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776176,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32180,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712972,"statfs":{"total":95995035648,"available":95962083328,"internally_reserved":0,"allocated":1019904,"data_stored":504094,"data_compressed":20691,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037815,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32172,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712980,"statfs":{"total":95995035648,"available":95962091520,"internally_reserved":0,"allocated":1015808,"data_stored":503432,"data_compressed":20691,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299453,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36388,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708764,"statfs":{"total":95995035648,"available":95957774336,"internally_reserved":0,"allocated":1069056,"data_stored":1100314,"data_compressed":56789,"data_compressed_allocated":73728,"data_compressed_original":647168,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462979,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32152,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713000,"statfs":{"total":95995035648,"available":95962112000,"internally_reserved":0,"allocated":999424,"data_stored":499270,"data_compressed":20691,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574943,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32220,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712932,"statfs":{"total":95995035648,"available":95962042368,"internally_reserved":0,"allocated":1069056,"data_stored":563118,"data_compressed":22320,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738467,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32184,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712968,"statfs":{"total":95995035648,"available":95962079232,"internally_reserved":0,"allocated":1024000,"data_stored":500658,"data_compressed":20691,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411417,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28076,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717076,"statfs":{"total":95995035648,"available":95966285824,"internally_reserved":0,"allocated":1085440,"data_stored":564243,"data_compressed":22320,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149779,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32168,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712984,"statfs":{"total":95995035648,"available":95962095616,"internally_reserved":0,"allocated":1011712,"data_stored":499464,"data_compressed":20691,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888141,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":1011712,"data_stored":504311,"data_compressed":20691,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724617,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28068,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717084,"statfs":{"total":95995035648,"available":95966294016,"internally_reserved":0,"allocated":1077248,"data_stored":1095652,"data_compressed":56789,"data_compressed_allocated":73728,"data_compressed_original":647168,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34456,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34456,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34456,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36098,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36098,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36098,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36098,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36098,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":63938,"data_compressed":1629,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":65377,"data_compressed":1629,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":156,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":65325,"data_compressed":1629,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-16T17:05:22.878 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45 shell --fsid b1fecd38-fc11-11ee-bc90-c7b262605968 -- ceph pg dump --format=json 2024-04-16T17:05:23.384 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/mon.a/config 2024-04-16T17:05:23.538 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:05:23 smithi045 ceph-mon[31676]: from='client.25042 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-16T17:05:23.538 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:05:23 smithi045 ceph-mon[31676]: pgmap v384: 129 pgs: 1 peering, 128 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 307 B/s wr, 6 op/s 2024-04-16T17:05:23.538 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:05:23 smithi045 ceph-mon[31676]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-04-16T17:05:23.538 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:05:23 smithi045 ceph-mon[31676]: osdmap e94: 12 total, 12 up, 12 in 2024-04-16T17:05:23.555 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:05:23 smithi117 ceph-mon[36730]: from='client.25042 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-16T17:05:23.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:05:23 smithi117 ceph-mon[36730]: pgmap v384: 129 pgs: 1 peering, 128 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 307 B/s wr, 6 op/s 2024-04-16T17:05:23.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:05:23 smithi117 ceph-mon[36730]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-04-16T17:05:23.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:05:23 smithi117 ceph-mon[36730]: osdmap e94: 12 total, 12 up, 12 in 2024-04-16T17:05:23.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:05:23 smithi104 ceph-mon[36457]: from='client.25042 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-16T17:05:23.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:05:23 smithi104 ceph-mon[36457]: pgmap v384: 129 pgs: 1 peering, 128 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 307 B/s wr, 6 op/s 2024-04-16T17:05:23.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:05:23 smithi104 ceph-mon[36457]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-04-16T17:05:23.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:05:23 smithi104 ceph-mon[36457]: osdmap e94: 12 total, 12 up, 12 in 2024-04-16T17:05:25.483 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:05:25 smithi045 ceph-mon[31676]: pgmap v386: 129 pgs: 1 peering, 128 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 383 B/s wr, 5 op/s 2024-04-16T17:05:25.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:05:25 smithi117 ceph-mon[36730]: pgmap v386: 129 pgs: 1 peering, 128 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 383 B/s wr, 5 op/s 2024-04-16T17:05:25.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:05:25 smithi104 ceph-mon[36457]: pgmap v386: 129 pgs: 1 peering, 128 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 383 B/s wr, 5 op/s 2024-04-16T17:05:25.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:25.774+0000 7fdaee400700 1 -- 172.21.15.45:0/2202771156 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdae8105910 msgr2=0x7fdae8109960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:25.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:25.774+0000 7fdaee400700 1 --2- 172.21.15.45:0/2202771156 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdae8105910 0x7fdae8109960 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fdad4009990 tx=0x7fdad4009ca0 comp rx=0 tx=0).stop 2024-04-16T17:05:25.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:25.774+0000 7fdaee400700 1 -- 172.21.15.45:0/2202771156 shutdown_connections 2024-04-16T17:05:25.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:25.774+0000 7fdaee400700 1 --2- 172.21.15.45:0/2202771156 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdae8104f60 0x7fdae8105340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:25.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:25.774+0000 7fdaee400700 1 --2- 172.21.15.45:0/2202771156 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdae810a040 0x7fdae8111c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:25.777 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:25.774+0000 7fdaee400700 1 --2- 172.21.15.45:0/2202771156 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdae8105910 0x7fdae8109960 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:25.777 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:25.774+0000 7fdaee400700 1 -- 172.21.15.45:0/2202771156 >> 172.21.15.45:0/2202771156 conn(0x7fdae81007d0 msgr2=0x7fdae8102bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:05:25.777 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:25.774+0000 7fdaee400700 1 -- 172.21.15.45:0/2202771156 shutdown_connections 2024-04-16T17:05:25.777 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:25.775+0000 7fdaee400700 1 -- 172.21.15.45:0/2202771156 wait complete. 2024-04-16T17:05:25.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:25.776+0000 7fdaee400700 1 Processor -- start 2024-04-16T17:05:25.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:25.777+0000 7fdaee400700 1 -- start start 2024-04-16T17:05:25.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:25.777+0000 7fdaee400700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdae8104f60 0x7fdae819b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:25.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:25.777+0000 7fdaee400700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdae8105910 0x7fdae819ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:25.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:25.778+0000 7fdaee400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdae810a040 0x7fdae819fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:25.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:25.778+0000 7fdaee400700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fdae8115070 con 0x7fdae8105910 2024-04-16T17:05:25.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:25.778+0000 7fdaee400700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fdae8114ef0 con 0x7fdae8104f60 2024-04-16T17:05:25.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:25.778+0000 7fdaee400700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fdae81151f0 con 0x7fdae810a040 2024-04-16T17:05:25.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:25.778+0000 7fdae6a00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdae8105910 0x7fdae819ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:25.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:25.778+0000 7fdae6a00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdae8105910 0x7fdae819ba20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58138/0 (socket says 172.21.15.45:58138) 2024-04-16T17:05:25.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:25.778+0000 7fdae6a00700 1 -- 172.21.15.45:0/1540231049 learned_addr learned my addr 172.21.15.45:0/1540231049 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:05:25.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:25.778+0000 7fdae7400700 1 --2- 172.21.15.45:0/1540231049 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdae8104f60 0x7fdae819b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:25.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:25.778+0000 7fdae7e00700 1 --2- 172.21.15.45:0/1540231049 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdae810a040 0x7fdae819fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:25.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:25.778+0000 7fdae6a00700 1 -- 172.21.15.45:0/1540231049 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdae810a040 msgr2=0x7fdae819fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:25.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:25.778+0000 7fdae6a00700 1 --2- 172.21.15.45:0/1540231049 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdae810a040 0x7fdae819fdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:25.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:25.778+0000 7fdae6a00700 1 -- 172.21.15.45:0/1540231049 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdae8104f60 msgr2=0x7fdae819b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:25.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:25.778+0000 7fdae6a00700 1 --2- 172.21.15.45:0/1540231049 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdae8104f60 0x7fdae819b4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:25.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:25.778+0000 7fdae6a00700 1 -- 172.21.15.45:0/1540231049 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdae81a04f0 con 0x7fdae8105910 2024-04-16T17:05:25.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:25.779+0000 7fdae6a00700 1 --2- 172.21.15.45:0/1540231049 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdae8105910 0x7fdae819ba20 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fdad400b2e0 tx=0x7fdad40047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:05:25.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:25.779+0000 7fdadfe00700 1 -- 172.21.15.45:0/1540231049 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdad4026070 con 0x7fdae8105910 2024-04-16T17:05:25.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:25.779+0000 7fdaee400700 1 -- 172.21.15.45:0/1540231049 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdae81a0780 con 0x7fdae8105910 2024-04-16T17:05:25.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:25.779+0000 7fdadfe00700 1 -- 172.21.15.45:0/1540231049 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdad400fde0 con 0x7fdae8105910 2024-04-16T17:05:25.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:25.779+0000 7fdadfe00700 1 -- 172.21.15.45:0/1540231049 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdad40173f0 con 0x7fdae8105910 2024-04-16T17:05:25.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:25.779+0000 7fdaee400700 1 -- 172.21.15.45:0/1540231049 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdae81a8060 con 0x7fdae8105910 2024-04-16T17:05:25.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:25.781+0000 7fdaee400700 1 -- 172.21.15.45:0/1540231049 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdae8106440 con 0x7fdae8105910 2024-04-16T17:05:25.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:25.784+0000 7fdadfe00700 1 -- 172.21.15.45:0/1540231049 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fdad4029070 con 0x7fdae8105910 2024-04-16T17:05:25.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:25.786+0000 7fdadfe00700 1 --2- 172.21.15.45:0/1540231049 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fdacc0a3c40 0x7fdacc0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:25.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:25.786+0000 7fdadfe00700 1 -- 172.21.15.45:0/1540231049 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fdad40d1c90 con 0x7fdae8105910 2024-04-16T17:05:25.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:25.786+0000 7fdae7400700 1 --2- 172.21.15.45:0/1540231049 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fdacc0a3c40 0x7fdacc0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:25.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:25.787+0000 7fdae7400700 1 --2- 172.21.15.45:0/1540231049 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fdacc0a3c40 0x7fdacc0a6100 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fdad8009c60 tx=0x7fdad8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:05:25.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:25.789+0000 7fdadfe00700 1 -- 172.21.15.45:0/1540231049 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fdad4030070 con 0x7fdae8105910 2024-04-16T17:05:26.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:26.015+0000 7fdaee400700 1 -- 172.21.15.45:0/1540231049 --> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fdae819c840 con 0x7fdacc0a3c40 2024-04-16T17:05:26.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:26.034+0000 7fdadfe00700 1 -- 172.21.15.45:0/1540231049 <== mgr.14152 v2:172.21.15.45:6800/268030699 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340875 (secure 0 0 0) 0x7fdae819c840 con 0x7fdacc0a3c40 2024-04-16T17:05:26.037 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-16T17:05:26.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:26.043+0000 7fdaee400700 1 -- 172.21.15.45:0/1540231049 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fdacc0a3c40 msgr2=0x7fdacc0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:26.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:26.043+0000 7fdaee400700 1 --2- 172.21.15.45:0/1540231049 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fdacc0a3c40 0x7fdacc0a6100 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fdad8009c60 tx=0x7fdad8009260 comp rx=0 tx=0).stop 2024-04-16T17:05:26.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:26.043+0000 7fdaee400700 1 -- 172.21.15.45:0/1540231049 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdae8105910 msgr2=0x7fdae819ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:26.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:26.043+0000 7fdaee400700 1 --2- 172.21.15.45:0/1540231049 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdae8105910 0x7fdae819ba20 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fdad400b2e0 tx=0x7fdad40047b0 comp rx=0 tx=0).stop 2024-04-16T17:05:26.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:26.044+0000 7fdaee400700 1 -- 172.21.15.45:0/1540231049 shutdown_connections 2024-04-16T17:05:26.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:26.044+0000 7fdaee400700 1 --2- 172.21.15.45:0/1540231049 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fdacc0a3c40 0x7fdacc0a6100 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:26.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:26.044+0000 7fdaee400700 1 --2- 172.21.15.45:0/1540231049 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdae810a040 0x7fdae819fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:26.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:26.044+0000 7fdaee400700 1 --2- 172.21.15.45:0/1540231049 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdae8104f60 0x7fdae819b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:26.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:26.044+0000 7fdaee400700 1 --2- 172.21.15.45:0/1540231049 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdae8105910 0x7fdae819ba20 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:26.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:26.044+0000 7fdaee400700 1 -- 172.21.15.45:0/1540231049 >> 172.21.15.45:0/1540231049 conn(0x7fdae81007d0 msgr2=0x7fdae8101340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:05:26.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:26.044+0000 7fdaee400700 1 -- 172.21.15.45:0/1540231049 shutdown_connections 2024-04-16T17:05:26.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:26.044+0000 7fdaee400700 1 -- 172.21.15.45:0/1540231049 wait complete. 2024-04-16T17:05:26.048 INFO:teuthology.orchestra.run.smithi045.stderr:dumped all 2024-04-16T17:05:26.876 INFO:teuthology.orchestra.run.smithi045.stdout:{"pg_ready":true,"pg_map":{"version":386,"stamp":"2024-04-16T17:05:24.152982+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":658632,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":687,"num_read_kb":467,"num_write":354,"num_write_kb":3061,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":1181469,"num_keys_recovered":20,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":658,"ondisk_log_size":658,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":374004,"kb_used_data":12332,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124567820,"statfs":{"total":1151940427776,"available":1151557447680,"internally_reserved":0,"allocated":12627968,"data_stored":8063861,"data_compressed":382438,"data_compressed_allocated":520192,"data_compressed_original":2539520,"omap_allocated":19143,"internal_metadata":370193721},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":32,"apply_latency_ms":32,"commit_latency_ns":32000000,"apply_latency_ns":32000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":282,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":36,"num_read_kb":16,"num_write":5,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.004538"},"pg_stats":[{"pgid":"3.3f","version":"92'8","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T17:05:20.165027+0000","last_change":"2024-04-16T17:03:24.989508+0000","last_active":"2024-04-16T17:05:20.165027+0000","last_peered":"2024-04-16T17:05:20.165027+0000","last_clean":"2024-04-16T17:05:20.165027+0000","last_became_active":"2024-04-16T17:03:24.989178+0000","last_became_peered":"2024-04-16T17:03:24.989178+0000","last_unstale":"2024-04-16T17:05:20.165027+0000","last_undegraded":"2024-04-16T17:05:20.165027+0000","last_fullsized":"2024-04-16T17:05:20.165027+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:18:56.488358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"92'4","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:05:21.301274+0000","last_change":"2024-04-16T17:03:24.986378+0000","last_active":"2024-04-16T17:05:21.301274+0000","last_peered":"2024-04-16T17:05:21.301274+0000","last_clean":"2024-04-16T17:05:21.301274+0000","last_became_active":"2024-04-16T17:03:24.986030+0000","last_became_peered":"2024-04-16T17:03:24.986030+0000","last_unstale":"2024-04-16T17:05:21.301274+0000","last_undegraded":"2024-04-16T17:05:21.301274+0000","last_fullsized":"2024-04-16T17:05:21.301274+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:21:14.071109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"92'7","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:05:21.301167+0000","last_change":"2024-04-16T17:03:24.976232+0000","last_active":"2024-04-16T17:05:21.301167+0000","last_peered":"2024-04-16T17:05:21.301167+0000","last_clean":"2024-04-16T17:05:21.301167+0000","last_became_active":"2024-04-16T17:03:24.975873+0000","last_became_peered":"2024-04-16T17:03:24.975873+0000","last_unstale":"2024-04-16T17:05:21.301167+0000","last_undegraded":"2024-04-16T17:05:21.301167+0000","last_fullsized":"2024-04-16T17:05:21.301167+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:07:53.523101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"92'8","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.285223+0000","last_change":"2024-04-16T17:04:21.517830+0000","last_active":"2024-04-16T17:05:22.285223+0000","last_peered":"2024-04-16T17:05:22.285223+0000","last_clean":"2024-04-16T17:05:22.285223+0000","last_became_active":"2024-04-16T17:04:21.516749+0000","last_became_peered":"2024-04-16T17:04:21.516749+0000","last_unstale":"2024-04-16T17:05:22.285223+0000","last_undegraded":"2024-04-16T17:05:22.285223+0000","last_fullsized":"2024-04-16T17:05:22.285223+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:02:33.344034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"92'4","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:05:21.282122+0000","last_change":"2024-04-16T17:03:24.985609+0000","last_active":"2024-04-16T17:05:21.282122+0000","last_peered":"2024-04-16T17:05:21.282122+0000","last_clean":"2024-04-16T17:05:21.282122+0000","last_became_active":"2024-04-16T17:03:24.985239+0000","last_became_peered":"2024-04-16T17:03:24.985239+0000","last_unstale":"2024-04-16T17:05:21.282122+0000","last_undegraded":"2024-04-16T17:05:21.282122+0000","last_fullsized":"2024-04-16T17:05:21.282122+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:55:58.619210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T17:04:21.497420+0000","last_change":"2024-04-16T17:03:24.966862+0000","last_active":"2024-04-16T17:04:21.497420+0000","last_peered":"2024-04-16T17:04:21.497420+0000","last_clean":"2024-04-16T17:04:21.497420+0000","last_became_active":"2024-04-16T17:03:24.966234+0000","last_became_peered":"2024-04-16T17:03:24.966234+0000","last_unstale":"2024-04-16T17:04:21.497420+0000","last_undegraded":"2024-04-16T17:04:21.497420+0000","last_fullsized":"2024-04-16T17:04:21.497420+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:21:07.018281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"92'5","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.510598+0000","last_change":"2024-04-16T17:03:24.971219+0000","last_active":"2024-04-16T17:05:22.510598+0000","last_peered":"2024-04-16T17:05:22.510598+0000","last_clean":"2024-04-16T17:05:22.510598+0000","last_became_active":"2024-04-16T17:03:24.970876+0000","last_became_peered":"2024-04-16T17:03:24.970876+0000","last_unstale":"2024-04-16T17:05:22.510598+0000","last_undegraded":"2024-04-16T17:05:22.510598+0000","last_fullsized":"2024-04-16T17:05:22.510598+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:44:16.311167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T17:04:21.500129+0000","last_change":"2024-04-16T17:03:24.994694+0000","last_active":"2024-04-16T17:04:21.500129+0000","last_peered":"2024-04-16T17:04:21.500129+0000","last_clean":"2024-04-16T17:04:21.500129+0000","last_became_active":"2024-04-16T17:03:24.994315+0000","last_became_peered":"2024-04-16T17:03:24.994315+0000","last_unstale":"2024-04-16T17:04:21.500129+0000","last_undegraded":"2024-04-16T17:04:21.500129+0000","last_fullsized":"2024-04-16T17:04:21.500129+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:04:59.543816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"92'11","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.512218+0000","last_change":"2024-04-16T17:03:24.964654+0000","last_active":"2024-04-16T17:05:22.512218+0000","last_peered":"2024-04-16T17:05:22.512218+0000","last_clean":"2024-04-16T17:05:22.512218+0000","last_became_active":"2024-04-16T17:03:24.964349+0000","last_became_peered":"2024-04-16T17:03:24.964349+0000","last_unstale":"2024-04-16T17:05:22.512218+0000","last_undegraded":"2024-04-16T17:05:22.512218+0000","last_fullsized":"2024-04-16T17:05:22.512218+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:38:55.880948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T17:04:21.497850+0000","last_change":"2024-04-16T17:03:24.966364+0000","last_active":"2024-04-16T17:04:21.497850+0000","last_peered":"2024-04-16T17:04:21.497850+0000","last_clean":"2024-04-16T17:04:21.497850+0000","last_became_active":"2024-04-16T17:03:24.966061+0000","last_became_peered":"2024-04-16T17:03:24.966061+0000","last_unstale":"2024-04-16T17:04:21.497850+0000","last_undegraded":"2024-04-16T17:04:21.497850+0000","last_fullsized":"2024-04-16T17:04:21.497850+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:18:12.642602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"92'6","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.511278+0000","last_change":"2024-04-16T17:03:24.969874+0000","last_active":"2024-04-16T17:05:22.511278+0000","last_peered":"2024-04-16T17:05:22.511278+0000","last_clean":"2024-04-16T17:05:22.511278+0000","last_became_active":"2024-04-16T17:03:24.969523+0000","last_became_peered":"2024-04-16T17:03:24.969523+0000","last_unstale":"2024-04-16T17:05:22.511278+0000","last_undegraded":"2024-04-16T17:05:22.511278+0000","last_fullsized":"2024-04-16T17:05:22.511278+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:17:49.704855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"92'7","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:05:21.276228+0000","last_change":"2024-04-16T17:03:24.968660+0000","last_active":"2024-04-16T17:05:21.276228+0000","last_peered":"2024-04-16T17:05:21.276228+0000","last_clean":"2024-04-16T17:05:21.276228+0000","last_became_active":"2024-04-16T17:03:24.968348+0000","last_became_peered":"2024-04-16T17:03:24.968348+0000","last_unstale":"2024-04-16T17:05:21.276228+0000","last_undegraded":"2024-04-16T17:05:21.276228+0000","last_fullsized":"2024-04-16T17:05:21.276228+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:06:06.319391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"92'10","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T17:05:20.156899+0000","last_change":"2024-04-16T17:03:24.974741+0000","last_active":"2024-04-16T17:05:20.156899+0000","last_peered":"2024-04-16T17:05:20.156899+0000","last_clean":"2024-04-16T17:05:20.156899+0000","last_became_active":"2024-04-16T17:03:24.973568+0000","last_became_peered":"2024-04-16T17:03:24.973568+0000","last_unstale":"2024-04-16T17:05:20.156899+0000","last_undegraded":"2024-04-16T17:05:20.156899+0000","last_fullsized":"2024-04-16T17:05:20.156899+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:30:49.325469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"92'7","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:05:21.300862+0000","last_change":"2024-04-16T17:03:24.981629+0000","last_active":"2024-04-16T17:05:21.300862+0000","last_peered":"2024-04-16T17:05:21.300862+0000","last_clean":"2024-04-16T17:05:21.300862+0000","last_became_active":"2024-04-16T17:03:24.981111+0000","last_became_peered":"2024-04-16T17:03:24.981111+0000","last_unstale":"2024-04-16T17:05:21.300862+0000","last_undegraded":"2024-04-16T17:05:21.300862+0000","last_fullsized":"2024-04-16T17:05:21.300862+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:41:29.641610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"92'7","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:05:21.300939+0000","last_change":"2024-04-16T17:03:24.977890+0000","last_active":"2024-04-16T17:05:21.300939+0000","last_peered":"2024-04-16T17:05:21.300939+0000","last_clean":"2024-04-16T17:05:21.300939+0000","last_became_active":"2024-04-16T17:03:24.976500+0000","last_became_peered":"2024-04-16T17:03:24.976500+0000","last_unstale":"2024-04-16T17:05:21.300939+0000","last_undegraded":"2024-04-16T17:05:21.300939+0000","last_fullsized":"2024-04-16T17:05:21.300939+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:17:51.604701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"92'9","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.509773+0000","last_change":"2024-04-16T17:03:24.976085+0000","last_active":"2024-04-16T17:05:22.509773+0000","last_peered":"2024-04-16T17:05:22.509773+0000","last_clean":"2024-04-16T17:05:22.509773+0000","last_became_active":"2024-04-16T17:03:24.975804+0000","last_became_peered":"2024-04-16T17:03:24.975804+0000","last_unstale":"2024-04-16T17:05:22.509773+0000","last_undegraded":"2024-04-16T17:05:22.509773+0000","last_fullsized":"2024-04-16T17:05:22.509773+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:19:13.706889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"92'8","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T17:05:20.164869+0000","last_change":"2024-04-16T17:03:24.989456+0000","last_active":"2024-04-16T17:05:20.164869+0000","last_peered":"2024-04-16T17:05:20.164869+0000","last_clean":"2024-04-16T17:05:20.164869+0000","last_became_active":"2024-04-16T17:03:24.989161+0000","last_became_peered":"2024-04-16T17:03:24.989161+0000","last_unstale":"2024-04-16T17:05:20.164869+0000","last_undegraded":"2024-04-16T17:05:20.164869+0000","last_fullsized":"2024-04-16T17:05:20.164869+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:52:59.333625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"92'10","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.287957+0000","last_change":"2024-04-16T17:03:24.963428+0000","last_active":"2024-04-16T17:05:22.287957+0000","last_peered":"2024-04-16T17:05:22.287957+0000","last_clean":"2024-04-16T17:05:22.287957+0000","last_became_active":"2024-04-16T17:03:24.962523+0000","last_became_peered":"2024-04-16T17:03:24.962523+0000","last_unstale":"2024-04-16T17:05:22.287957+0000","last_undegraded":"2024-04-16T17:05:22.287957+0000","last_fullsized":"2024-04-16T17:05:22.287957+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:22:19.523901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T17:04:21.502387+0000","last_change":"2024-04-16T17:03:24.981122+0000","last_active":"2024-04-16T17:04:21.502387+0000","last_peered":"2024-04-16T17:04:21.502387+0000","last_clean":"2024-04-16T17:04:21.502387+0000","last_became_active":"2024-04-16T17:03:24.980533+0000","last_became_peered":"2024-04-16T17:03:24.980533+0000","last_unstale":"2024-04-16T17:04:21.502387+0000","last_undegraded":"2024-04-16T17:04:21.502387+0000","last_fullsized":"2024-04-16T17:04:21.502387+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:51:08.580965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T17:04:21.502277+0000","last_change":"2024-04-16T17:03:29.257239+0000","last_active":"2024-04-16T17:04:21.502277+0000","last_peered":"2024-04-16T17:04:21.502277+0000","last_clean":"2024-04-16T17:04:21.502277+0000","last_became_active":"2024-04-16T17:03:29.256827+0000","last_became_peered":"2024-04-16T17:03:29.256827+0000","last_unstale":"2024-04-16T17:04:21.502277+0000","last_undegraded":"2024-04-16T17:04:21.502277+0000","last_fullsized":"2024-04-16T17:04:21.502277+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:57:28.373386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"90'9","reported_seq":57,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.286649+0000","last_change":"2024-04-16T17:04:21.896665+0000","last_active":"2024-04-16T17:05:22.286649+0000","last_peered":"2024-04-16T17:05:22.286649+0000","last_clean":"2024-04-16T17:05:22.286649+0000","last_became_active":"2024-04-16T17:04:21.882573+0000","last_became_peered":"2024-04-16T17:04:21.882573+0000","last_unstale":"2024-04-16T17:05:22.286649+0000","last_undegraded":"2024-04-16T17:05:22.286649+0000","last_fullsized":"2024-04-16T17:05:22.286649+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:37:07.210368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,1],"acting":[9,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T17:04:21.503068+0000","last_change":"2024-04-16T17:03:24.994921+0000","last_active":"2024-04-16T17:04:21.503068+0000","last_peered":"2024-04-16T17:04:21.503068+0000","last_clean":"2024-04-16T17:04:21.503068+0000","last_became_active":"2024-04-16T17:03:24.994412+0000","last_became_peered":"2024-04-16T17:03:24.994412+0000","last_unstale":"2024-04-16T17:04:21.503068+0000","last_undegraded":"2024-04-16T17:04:21.503068+0000","last_fullsized":"2024-04-16T17:04:21.503068+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:11:40.748198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":20,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:05:21.275792+0000","last_change":"2024-04-16T17:03:29.259884+0000","last_active":"2024-04-16T17:05:21.275792+0000","last_peered":"2024-04-16T17:05:21.275792+0000","last_clean":"2024-04-16T17:05:21.275792+0000","last_became_active":"2024-04-16T17:03:29.259319+0000","last_became_peered":"2024-04-16T17:03:29.259319+0000","last_unstale":"2024-04-16T17:05:21.275792+0000","last_undegraded":"2024-04-16T17:05:21.275792+0000","last_fullsized":"2024-04-16T17:05:21.275792+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:05:18.244682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"90'2","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T17:04:21.504276+0000","last_change":"2024-04-16T17:03:24.179308+0000","last_active":"2024-04-16T17:04:21.504276+0000","last_peered":"2024-04-16T17:04:21.504276+0000","last_clean":"2024-04-16T17:04:21.504276+0000","last_became_active":"2024-04-16T17:03:24.178894+0000","last_became_peered":"2024-04-16T17:03:24.178894+0000","last_unstale":"2024-04-16T17:04:21.504276+0000","last_undegraded":"2024-04-16T17:04:21.504276+0000","last_fullsized":"2024-04-16T17:04:21.504276+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:17:46.831871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"92'6","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:05:21.302067+0000","last_change":"2024-04-16T17:03:24.982391+0000","last_active":"2024-04-16T17:05:21.302067+0000","last_peered":"2024-04-16T17:05:21.302067+0000","last_clean":"2024-04-16T17:05:21.302067+0000","last_became_active":"2024-04-16T17:03:24.981761+0000","last_became_peered":"2024-04-16T17:03:24.981761+0000","last_unstale":"2024-04-16T17:05:21.302067+0000","last_undegraded":"2024-04-16T17:05:21.302067+0000","last_fullsized":"2024-04-16T17:05:21.302067+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:40:57.657170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":20,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:05:21.275973+0000","last_change":"2024-04-16T17:03:29.259705+0000","last_active":"2024-04-16T17:05:21.275973+0000","last_peered":"2024-04-16T17:05:21.275973+0000","last_clean":"2024-04-16T17:05:21.275973+0000","last_became_active":"2024-04-16T17:03:29.259336+0000","last_became_peered":"2024-04-16T17:03:29.259336+0000","last_unstale":"2024-04-16T17:05:21.275973+0000","last_undegraded":"2024-04-16T17:05:21.275973+0000","last_fullsized":"2024-04-16T17:05:21.275973+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:53:29.042933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"90'4","reported_seq":45,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:05:21.282233+0000","last_change":"2024-04-16T17:04:22.153413+0000","last_active":"2024-04-16T17:05:21.282233+0000","last_peered":"2024-04-16T17:05:21.282233+0000","last_clean":"2024-04-16T17:05:21.282233+0000","last_became_active":"2024-04-16T17:04:22.148972+0000","last_became_peered":"2024-04-16T17:04:22.148972+0000","last_unstale":"2024-04-16T17:05:21.282233+0000","last_undegraded":"2024-04-16T17:05:21.282233+0000","last_fullsized":"2024-04-16T17:05:21.282233+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:05:57.513052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"92'9","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.510097+0000","last_change":"2024-04-16T17:03:24.973205+0000","last_active":"2024-04-16T17:05:22.510097+0000","last_peered":"2024-04-16T17:05:22.510097+0000","last_clean":"2024-04-16T17:05:22.510097+0000","last_became_active":"2024-04-16T17:03:24.972863+0000","last_became_peered":"2024-04-16T17:03:24.972863+0000","last_unstale":"2024-04-16T17:05:22.510097+0000","last_undegraded":"2024-04-16T17:05:22.510097+0000","last_fullsized":"2024-04-16T17:05:22.510097+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:28:29.005783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":20,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:05:21.275945+0000","last_change":"2024-04-16T17:03:29.262930+0000","last_active":"2024-04-16T17:05:21.275945+0000","last_peered":"2024-04-16T17:05:21.275945+0000","last_clean":"2024-04-16T17:05:21.275945+0000","last_became_active":"2024-04-16T17:03:29.262621+0000","last_became_peered":"2024-04-16T17:03:29.262621+0000","last_unstale":"2024-04-16T17:05:21.275945+0000","last_undegraded":"2024-04-16T17:05:21.275945+0000","last_fullsized":"2024-04-16T17:05:21.275945+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:22:31.555143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:05:21.282525+0000","last_change":"2024-04-16T17:03:24.150503+0000","last_active":"2024-04-16T17:05:21.282525+0000","last_peered":"2024-04-16T17:05:21.282525+0000","last_clean":"2024-04-16T17:05:21.282525+0000","last_became_active":"2024-04-16T17:03:24.150021+0000","last_became_peered":"2024-04-16T17:03:24.150021+0000","last_unstale":"2024-04-16T17:05:21.282525+0000","last_undegraded":"2024-04-16T17:05:21.282525+0000","last_fullsized":"2024-04-16T17:05:21.282525+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:21:36.119497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"92'15","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T17:05:20.158223+0000","last_change":"2024-04-16T17:03:24.965049+0000","last_active":"2024-04-16T17:05:20.158223+0000","last_peered":"2024-04-16T17:05:20.158223+0000","last_clean":"2024-04-16T17:05:20.158223+0000","last_became_active":"2024-04-16T17:03:24.963120+0000","last_became_peered":"2024-04-16T17:03:24.963120+0000","last_unstale":"2024-04-16T17:05:20.158223+0000","last_undegraded":"2024-04-16T17:05:20.158223+0000","last_fullsized":"2024-04-16T17:05:20.158223+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:17:49.725113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T17:04:21.502466+0000","last_change":"2024-04-16T17:03:29.247013+0000","last_active":"2024-04-16T17:04:21.502466+0000","last_peered":"2024-04-16T17:04:21.502466+0000","last_clean":"2024-04-16T17:04:21.502466+0000","last_became_active":"2024-04-16T17:03:29.246477+0000","last_became_peered":"2024-04-16T17:03:29.246477+0000","last_unstale":"2024-04-16T17:04:21.502466+0000","last_undegraded":"2024-04-16T17:04:21.502466+0000","last_fullsized":"2024-04-16T17:04:21.502466+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:52:19.525076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"92'7","reported_seq":124,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T17:05:19.512951+0000","last_change":"2024-04-16T17:03:24.155333+0000","last_active":"2024-04-16T17:05:19.512951+0000","last_peered":"2024-04-16T17:05:19.512951+0000","last_clean":"2024-04-16T17:05:19.512951+0000","last_became_active":"2024-04-16T17:03:24.154848+0000","last_became_peered":"2024-04-16T17:03:24.154848+0000","last_unstale":"2024-04-16T17:05:19.512951+0000","last_undegraded":"2024-04-16T17:05:19.512951+0000","last_fullsized":"2024-04-16T17:05:19.512951+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:03:34.769573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1154,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":90,"num_read_kb":3,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"92'7","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:05:21.283418+0000","last_change":"2024-04-16T17:03:24.967627+0000","last_active":"2024-04-16T17:05:21.283418+0000","last_peered":"2024-04-16T17:05:21.283418+0000","last_clean":"2024-04-16T17:05:21.283418+0000","last_became_active":"2024-04-16T17:03:24.967264+0000","last_became_peered":"2024-04-16T17:03:24.967264+0000","last_unstale":"2024-04-16T17:05:21.283418+0000","last_undegraded":"2024-04-16T17:05:21.283418+0000","last_fullsized":"2024-04-16T17:05:21.283418+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:35:29.182374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T17:04:21.501736+0000","last_change":"2024-04-16T17:03:29.264625+0000","last_active":"2024-04-16T17:04:21.501736+0000","last_peered":"2024-04-16T17:04:21.501736+0000","last_clean":"2024-04-16T17:04:21.501736+0000","last_became_active":"2024-04-16T17:03:29.264114+0000","last_became_peered":"2024-04-16T17:03:29.264114+0000","last_unstale":"2024-04-16T17:04:21.501736+0000","last_undegraded":"2024-04-16T17:04:21.501736+0000","last_fullsized":"2024-04-16T17:04:21.501736+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:14:49.449980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"90'3","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T17:04:21.497406+0000","last_change":"2024-04-16T17:03:24.160457+0000","last_active":"2024-04-16T17:04:21.497406+0000","last_peered":"2024-04-16T17:04:21.497406+0000","last_clean":"2024-04-16T17:04:21.497406+0000","last_became_active":"2024-04-16T17:03:24.160195+0000","last_became_peered":"2024-04-16T17:03:24.160195+0000","last_unstale":"2024-04-16T17:04:21.497406+0000","last_undegraded":"2024-04-16T17:04:21.497406+0000","last_fullsized":"2024-04-16T17:04:21.497406+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:23:07.064539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"92'8","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T17:05:20.167765+0000","last_change":"2024-04-16T17:03:24.991249+0000","last_active":"2024-04-16T17:05:20.167765+0000","last_peered":"2024-04-16T17:05:20.167765+0000","last_clean":"2024-04-16T17:05:20.167765+0000","last_became_active":"2024-04-16T17:03:24.990927+0000","last_became_peered":"2024-04-16T17:03:24.990927+0000","last_unstale":"2024-04-16T17:05:20.167765+0000","last_undegraded":"2024-04-16T17:05:20.167765+0000","last_fullsized":"2024-04-16T17:05:20.167765+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:36:39.975616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T17:04:21.499715+0000","last_change":"2024-04-16T17:03:29.253704+0000","last_active":"2024-04-16T17:04:21.499715+0000","last_peered":"2024-04-16T17:04:21.499715+0000","last_clean":"2024-04-16T17:04:21.499715+0000","last_became_active":"2024-04-16T17:03:29.253334+0000","last_became_peered":"2024-04-16T17:03:29.253334+0000","last_unstale":"2024-04-16T17:04:21.499715+0000","last_undegraded":"2024-04-16T17:04:21.499715+0000","last_fullsized":"2024-04-16T17:04:21.499715+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:19:12.611731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"90'2","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T17:04:22.198461+0000","last_change":"2024-04-16T17:03:24.158875+0000","last_active":"2024-04-16T17:04:22.198461+0000","last_peered":"2024-04-16T17:04:22.198461+0000","last_clean":"2024-04-16T17:04:22.198461+0000","last_became_active":"2024-04-16T17:03:24.158465+0000","last_became_peered":"2024-04-16T17:03:24.158465+0000","last_unstale":"2024-04-16T17:04:22.198461+0000","last_undegraded":"2024-04-16T17:04:22.198461+0000","last_fullsized":"2024-04-16T17:04:22.198461+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:21:29.882251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T17:04:21.500496+0000","last_change":"2024-04-16T17:03:24.994476+0000","last_active":"2024-04-16T17:04:21.500496+0000","last_peered":"2024-04-16T17:04:21.500496+0000","last_clean":"2024-04-16T17:04:21.500496+0000","last_became_active":"2024-04-16T17:03:24.993532+0000","last_became_peered":"2024-04-16T17:03:24.993532+0000","last_unstale":"2024-04-16T17:04:21.500496+0000","last_undegraded":"2024-04-16T17:04:21.500496+0000","last_fullsized":"2024-04-16T17:04:21.500496+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:04:44.062444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":20,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:05:21.301242+0000","last_change":"2024-04-16T17:03:29.255446+0000","last_active":"2024-04-16T17:05:21.301242+0000","last_peered":"2024-04-16T17:05:21.301242+0000","last_clean":"2024-04-16T17:05:21.301242+0000","last_became_active":"2024-04-16T17:03:29.255051+0000","last_became_peered":"2024-04-16T17:03:29.255051+0000","last_unstale":"2024-04-16T17:05:21.301242+0000","last_undegraded":"2024-04-16T17:05:21.301242+0000","last_fullsized":"2024-04-16T17:05:21.301242+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:24:04.946342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"90'1","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T17:04:21.504332+0000","last_change":"2024-04-16T17:03:24.154904+0000","last_active":"2024-04-16T17:04:21.504332+0000","last_peered":"2024-04-16T17:04:21.504332+0000","last_clean":"2024-04-16T17:04:21.504332+0000","last_became_active":"2024-04-16T17:03:24.154617+0000","last_became_peered":"2024-04-16T17:03:24.154617+0000","last_unstale":"2024-04-16T17:04:21.504332+0000","last_undegraded":"2024-04-16T17:04:21.504332+0000","last_fullsized":"2024-04-16T17:04:21.504332+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:35:55.498855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"92'11","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.284812+0000","last_change":"2024-04-16T17:03:24.979908+0000","last_active":"2024-04-16T17:05:22.284812+0000","last_peered":"2024-04-16T17:05:22.284812+0000","last_clean":"2024-04-16T17:05:22.284812+0000","last_became_active":"2024-04-16T17:03:24.979501+0000","last_became_peered":"2024-04-16T17:03:24.979501+0000","last_unstale":"2024-04-16T17:05:22.284812+0000","last_undegraded":"2024-04-16T17:05:22.284812+0000","last_fullsized":"2024-04-16T17:05:22.284812+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:51:05.421459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T17:04:21.500874+0000","last_change":"2024-04-16T17:03:29.258809+0000","last_active":"2024-04-16T17:04:21.500874+0000","last_peered":"2024-04-16T17:04:21.500874+0000","last_clean":"2024-04-16T17:04:21.500874+0000","last_became_active":"2024-04-16T17:03:29.256629+0000","last_became_peered":"2024-04-16T17:03:29.256629+0000","last_unstale":"2024-04-16T17:04:21.500874+0000","last_undegraded":"2024-04-16T17:04:21.500874+0000","last_fullsized":"2024-04-16T17:04:21.500874+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:46:51.634944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"90'3","reported_seq":116,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T17:05:17.509576+0000","last_change":"2024-04-16T17:03:24.153774+0000","last_active":"2024-04-16T17:05:17.509576+0000","last_peered":"2024-04-16T17:05:17.509576+0000","last_clean":"2024-04-16T17:05:17.509576+0000","last_became_active":"2024-04-16T17:03:24.153418+0000","last_became_peered":"2024-04-16T17:03:24.153418+0000","last_unstale":"2024-04-16T17:05:17.509576+0000","last_undegraded":"2024-04-16T17:05:17.509576+0000","last_fullsized":"2024-04-16T17:05:17.509576+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:58:36.093789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":86,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"92'10","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.511791+0000","last_change":"2024-04-16T17:03:24.962838+0000","last_active":"2024-04-16T17:05:22.511791+0000","last_peered":"2024-04-16T17:05:22.511791+0000","last_clean":"2024-04-16T17:05:22.511791+0000","last_became_active":"2024-04-16T17:03:24.960163+0000","last_became_peered":"2024-04-16T17:03:24.960163+0000","last_unstale":"2024-04-16T17:05:22.511791+0000","last_undegraded":"2024-04-16T17:05:22.511791+0000","last_fullsized":"2024-04-16T17:05:22.511791+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:48:18.723499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.284550+0000","last_change":"2024-04-16T17:03:29.251177+0000","last_active":"2024-04-16T17:05:22.284550+0000","last_peered":"2024-04-16T17:05:22.284550+0000","last_clean":"2024-04-16T17:05:22.284550+0000","last_became_active":"2024-04-16T17:03:29.250657+0000","last_became_peered":"2024-04-16T17:03:29.250657+0000","last_unstale":"2024-04-16T17:05:22.284550+0000","last_undegraded":"2024-04-16T17:05:22.284550+0000","last_fullsized":"2024-04-16T17:05:22.284550+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:16:47.217193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"90'2","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.510573+0000","last_change":"2024-04-16T17:03:24.154245+0000","last_active":"2024-04-16T17:05:22.510573+0000","last_peered":"2024-04-16T17:05:22.510573+0000","last_clean":"2024-04-16T17:05:22.510573+0000","last_became_active":"2024-04-16T17:03:24.153297+0000","last_became_peered":"2024-04-16T17:03:24.153297+0000","last_unstale":"2024-04-16T17:05:22.510573+0000","last_undegraded":"2024-04-16T17:05:22.510573+0000","last_fullsized":"2024-04-16T17:05:22.510573+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:23:49.490238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"92'9","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:05:21.302114+0000","last_change":"2024-04-16T17:03:24.984572+0000","last_active":"2024-04-16T17:05:21.302114+0000","last_peered":"2024-04-16T17:05:21.302114+0000","last_clean":"2024-04-16T17:05:21.302114+0000","last_became_active":"2024-04-16T17:03:24.983588+0000","last_became_peered":"2024-04-16T17:03:24.983588+0000","last_unstale":"2024-04-16T17:05:21.302114+0000","last_undegraded":"2024-04-16T17:05:21.302114+0000","last_fullsized":"2024-04-16T17:05:21.302114+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:41:24.704602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":20,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:05:21.276496+0000","last_change":"2024-04-16T17:03:29.245129+0000","last_active":"2024-04-16T17:05:21.276496+0000","last_peered":"2024-04-16T17:05:21.276496+0000","last_clean":"2024-04-16T17:05:21.276496+0000","last_became_active":"2024-04-16T17:03:29.244720+0000","last_became_peered":"2024-04-16T17:03:29.244720+0000","last_unstale":"2024-04-16T17:05:21.276496+0000","last_undegraded":"2024-04-16T17:05:21.276496+0000","last_fullsized":"2024-04-16T17:05:21.276496+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:55:26.612176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"90'4","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T17:04:21.505070+0000","last_change":"2024-04-16T17:03:24.165248+0000","last_active":"2024-04-16T17:04:21.505070+0000","last_peered":"2024-04-16T17:04:21.505070+0000","last_clean":"2024-04-16T17:04:21.505070+0000","last_became_active":"2024-04-16T17:03:24.164915+0000","last_became_peered":"2024-04-16T17:03:24.164915+0000","last_unstale":"2024-04-16T17:04:21.505070+0000","last_undegraded":"2024-04-16T17:04:21.505070+0000","last_fullsized":"2024-04-16T17:04:21.505070+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:04:47.207554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"92'11","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.510492+0000","last_change":"2024-04-16T17:03:24.967292+0000","last_active":"2024-04-16T17:05:22.510492+0000","last_peered":"2024-04-16T17:05:22.510492+0000","last_clean":"2024-04-16T17:05:22.510492+0000","last_became_active":"2024-04-16T17:03:24.966952+0000","last_became_peered":"2024-04-16T17:03:24.966952+0000","last_unstale":"2024-04-16T17:05:22.510492+0000","last_undegraded":"2024-04-16T17:05:22.510492+0000","last_fullsized":"2024-04-16T17:05:22.510492+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:42:54.969478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.285084+0000","last_change":"2024-04-16T17:03:29.257836+0000","last_active":"2024-04-16T17:05:22.285084+0000","last_peered":"2024-04-16T17:05:22.285084+0000","last_clean":"2024-04-16T17:05:22.285084+0000","last_became_active":"2024-04-16T17:03:29.257409+0000","last_became_peered":"2024-04-16T17:03:29.257409+0000","last_unstale":"2024-04-16T17:05:22.285084+0000","last_undegraded":"2024-04-16T17:05:22.285084+0000","last_fullsized":"2024-04-16T17:05:22.285084+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:25:21.288266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"92'33","reported_seq":241,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T17:05:18.510684+0000","last_change":"2024-04-16T17:03:24.150943+0000","last_active":"2024-04-16T17:05:18.510684+0000","last_peered":"2024-04-16T17:05:18.510684+0000","last_clean":"2024-04-16T17:05:18.510684+0000","last_became_active":"2024-04-16T17:03:24.150563+0000","last_became_peered":"2024-04-16T17:03:24.150563+0000","last_unstale":"2024-04-16T17:05:18.510684+0000","last_undegraded":"2024-04-16T17:05:18.510684+0000","last_fullsized":"2024-04-16T17:05:18.510684+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:05:07.380440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":63356,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":181,"num_read_kb":153,"num_write":34,"num_write_kb":77,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"92'6","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:05:21.281933+0000","last_change":"2024-04-16T17:03:24.966393+0000","last_active":"2024-04-16T17:05:21.281933+0000","last_peered":"2024-04-16T17:05:21.281933+0000","last_clean":"2024-04-16T17:05:21.281933+0000","last_became_active":"2024-04-16T17:03:24.966016+0000","last_became_peered":"2024-04-16T17:03:24.966016+0000","last_unstale":"2024-04-16T17:05:21.281933+0000","last_undegraded":"2024-04-16T17:05:21.281933+0000","last_fullsized":"2024-04-16T17:05:21.281933+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:30:25.203262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T17:04:21.505554+0000","last_change":"2024-04-16T17:03:29.265371+0000","last_active":"2024-04-16T17:04:21.505554+0000","last_peered":"2024-04-16T17:04:21.505554+0000","last_clean":"2024-04-16T17:04:21.505554+0000","last_became_active":"2024-04-16T17:03:29.264933+0000","last_became_peered":"2024-04-16T17:03:29.264933+0000","last_unstale":"2024-04-16T17:04:21.505554+0000","last_undegraded":"2024-04-16T17:04:21.505554+0000","last_fullsized":"2024-04-16T17:04:21.505554+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:50:18.203015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"90'6","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T17:04:21.527209+0000","last_change":"2024-04-16T17:04:21.526796+0000","last_active":"2024-04-16T17:04:21.527209+0000","last_peered":"2024-04-16T17:04:21.527209+0000","last_clean":"2024-04-16T17:04:21.527209+0000","last_became_active":"2024-04-16T17:04:21.508364+0000","last_became_peered":"2024-04-16T17:04:21.508364+0000","last_unstale":"2024-04-16T17:04:21.527209+0000","last_undegraded":"2024-04-16T17:04:21.527209+0000","last_fullsized":"2024-04-16T17:04:21.527209+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:45:42.726648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"92'9","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T17:05:20.162304+0000","last_change":"2024-04-16T17:03:24.986228+0000","last_active":"2024-04-16T17:05:20.162304+0000","last_peered":"2024-04-16T17:05:20.162304+0000","last_clean":"2024-04-16T17:05:20.162304+0000","last_became_active":"2024-04-16T17:03:24.985916+0000","last_became_peered":"2024-04-16T17:03:24.985916+0000","last_unstale":"2024-04-16T17:05:20.162304+0000","last_undegraded":"2024-04-16T17:05:20.162304+0000","last_fullsized":"2024-04-16T17:05:20.162304+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:23:54.309203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T17:04:21.499369+0000","last_change":"2024-04-16T17:03:29.264636+0000","last_active":"2024-04-16T17:04:21.499369+0000","last_peered":"2024-04-16T17:04:21.499369+0000","last_clean":"2024-04-16T17:04:21.499369+0000","last_became_active":"2024-04-16T17:03:29.264117+0000","last_became_peered":"2024-04-16T17:03:29.264117+0000","last_unstale":"2024-04-16T17:04:21.499369+0000","last_undegraded":"2024-04-16T17:04:21.499369+0000","last_fullsized":"2024-04-16T17:04:21.499369+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:42:36.123154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"90'3","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T17:04:21.505264+0000","last_change":"2024-04-16T17:03:24.166844+0000","last_active":"2024-04-16T17:04:21.505264+0000","last_peered":"2024-04-16T17:04:21.505264+0000","last_clean":"2024-04-16T17:04:21.505264+0000","last_became_active":"2024-04-16T17:03:24.166479+0000","last_became_peered":"2024-04-16T17:03:24.166479+0000","last_unstale":"2024-04-16T17:04:21.505264+0000","last_undegraded":"2024-04-16T17:04:21.505264+0000","last_fullsized":"2024-04-16T17:04:21.505264+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:02:20.139865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"92'15","reported_seq":44,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.510958+0000","last_change":"2024-04-16T17:03:24.973057+0000","last_active":"2024-04-16T17:05:22.510958+0000","last_peered":"2024-04-16T17:05:22.510958+0000","last_clean":"2024-04-16T17:05:22.510958+0000","last_became_active":"2024-04-16T17:03:24.972731+0000","last_became_peered":"2024-04-16T17:03:24.972731+0000","last_unstale":"2024-04-16T17:05:22.510958+0000","last_undegraded":"2024-04-16T17:05:22.510958+0000","last_fullsized":"2024-04-16T17:05:22.510958+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:37:38.702919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.286263+0000","last_change":"2024-04-16T17:03:29.259167+0000","last_active":"2024-04-16T17:05:22.286263+0000","last_peered":"2024-04-16T17:05:22.286263+0000","last_clean":"2024-04-16T17:05:22.286263+0000","last_became_active":"2024-04-16T17:03:29.258737+0000","last_became_peered":"2024-04-16T17:03:29.258737+0000","last_unstale":"2024-04-16T17:05:22.286263+0000","last_undegraded":"2024-04-16T17:05:22.286263+0000","last_fullsized":"2024-04-16T17:05:22.286263+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:01:13.774037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"92'5","reported_seq":126,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:05:21.275997+0000","last_change":"2024-04-16T17:03:24.151064+0000","last_active":"2024-04-16T17:05:21.275997+0000","last_peered":"2024-04-16T17:05:21.275997+0000","last_clean":"2024-04-16T17:05:21.275997+0000","last_became_active":"2024-04-16T17:03:24.150745+0000","last_became_peered":"2024-04-16T17:03:24.150745+0000","last_unstale":"2024-04-16T17:05:21.275997+0000","last_undegraded":"2024-04-16T17:05:21.275997+0000","last_fullsized":"2024-04-16T17:05:21.275997+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:32:26.271008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":93,"num_read_kb":5,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"92'6","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.510687+0000","last_change":"2024-04-16T17:03:24.971948+0000","last_active":"2024-04-16T17:05:22.510687+0000","last_peered":"2024-04-16T17:05:22.510687+0000","last_clean":"2024-04-16T17:05:22.510687+0000","last_became_active":"2024-04-16T17:03:24.971530+0000","last_became_peered":"2024-04-16T17:03:24.971530+0000","last_unstale":"2024-04-16T17:05:22.510687+0000","last_undegraded":"2024-04-16T17:05:22.510687+0000","last_fullsized":"2024-04-16T17:05:22.510687+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:03:29.286160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T17:04:22.199376+0000","last_change":"2024-04-16T17:03:29.264821+0000","last_active":"2024-04-16T17:04:22.199376+0000","last_peered":"2024-04-16T17:04:22.199376+0000","last_clean":"2024-04-16T17:04:22.199376+0000","last_became_active":"2024-04-16T17:03:29.264287+0000","last_became_peered":"2024-04-16T17:03:29.264287+0000","last_unstale":"2024-04-16T17:04:22.199376+0000","last_undegraded":"2024-04-16T17:04:22.199376+0000","last_fullsized":"2024-04-16T17:04:22.199376+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:33:28.400663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"90'15","reported_seq":45,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:05:21.281634+0000","last_change":"2024-04-16T17:03:24.158844+0000","last_active":"2024-04-16T17:05:21.281634+0000","last_peered":"2024-04-16T17:05:21.281634+0000","last_clean":"2024-04-16T17:05:21.281634+0000","last_became_active":"2024-04-16T17:03:24.158537+0000","last_became_peered":"2024-04-16T17:03:24.158537+0000","last_unstale":"2024-04-16T17:05:21.281634+0000","last_undegraded":"2024-04-16T17:05:21.281634+0000","last_fullsized":"2024-04-16T17:05:21.281634+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:42:22.772452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"92'6","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T17:05:20.151688+0000","last_change":"2024-04-16T17:03:24.962522+0000","last_active":"2024-04-16T17:05:20.151688+0000","last_peered":"2024-04-16T17:05:20.151688+0000","last_clean":"2024-04-16T17:05:20.151688+0000","last_became_active":"2024-04-16T17:03:24.962169+0000","last_became_peered":"2024-04-16T17:03:24.962169+0000","last_unstale":"2024-04-16T17:05:20.151688+0000","last_undegraded":"2024-04-16T17:05:20.151688+0000","last_fullsized":"2024-04-16T17:05:20.151688+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:28:17.147491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T17:04:21.504843+0000","last_change":"2024-04-16T17:03:29.259539+0000","last_active":"2024-04-16T17:04:21.504843+0000","last_peered":"2024-04-16T17:04:21.504843+0000","last_clean":"2024-04-16T17:04:21.504843+0000","last_became_active":"2024-04-16T17:03:29.259123+0000","last_became_peered":"2024-04-16T17:03:29.259123+0000","last_unstale":"2024-04-16T17:04:21.504843+0000","last_undegraded":"2024-04-16T17:04:21.504843+0000","last_fullsized":"2024-04-16T17:04:21.504843+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:15:17.142636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"90'2","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:05:21.282283+0000","last_change":"2024-04-16T17:03:24.147526+0000","last_active":"2024-04-16T17:05:21.282283+0000","last_peered":"2024-04-16T17:05:21.282283+0000","last_clean":"2024-04-16T17:05:21.282283+0000","last_became_active":"2024-04-16T17:03:24.147119+0000","last_became_peered":"2024-04-16T17:03:24.147119+0000","last_unstale":"2024-04-16T17:05:21.282283+0000","last_undegraded":"2024-04-16T17:05:21.282283+0000","last_fullsized":"2024-04-16T17:05:21.282283+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:30:38.742122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"92'5","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T17:05:20.144526+0000","last_change":"2024-04-16T17:04:21.507113+0000","last_active":"2024-04-16T17:05:20.144526+0000","last_peered":"2024-04-16T17:05:20.144526+0000","last_clean":"2024-04-16T17:05:20.144526+0000","last_became_active":"2024-04-16T17:04:21.506680+0000","last_became_peered":"2024-04-16T17:04:21.506680+0000","last_unstale":"2024-04-16T17:05:20.144526+0000","last_undegraded":"2024-04-16T17:05:20.144526+0000","last_fullsized":"2024-04-16T17:05:20.144526+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:08:47.971317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,8],"acting":[11,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":19,"reported_epoch":92,"state":"peering","last_fresh":"2024-04-16T17:04:21.503198+0000","last_change":"2024-04-16T17:05:21.282684+0000","last_active":"2024-04-16T17:05:21.282684+0000","last_peered":"2024-04-16T17:04:21.503198+0000","last_clean":"2024-04-16T17:04:21.503198+0000","last_became_active":"2024-04-16T17:03:29.300672+0000","last_became_peered":"2024-04-16T17:03:29.300672+0000","last_unstale":"2024-04-16T17:04:21.503198+0000","last_undegraded":"2024-04-16T17:04:21.503198+0000","last_fullsized":"2024-04-16T17:04:21.503198+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,1],"acting":[8,10,11,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,9,10,11],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.285619+0000","last_change":"2024-04-16T17:03:24.162105+0000","last_active":"2024-04-16T17:05:22.285619+0000","last_peered":"2024-04-16T17:05:22.285619+0000","last_clean":"2024-04-16T17:05:22.285619+0000","last_became_active":"2024-04-16T17:03:24.161631+0000","last_became_peered":"2024-04-16T17:03:24.161631+0000","last_unstale":"2024-04-16T17:05:22.285619+0000","last_undegraded":"2024-04-16T17:05:22.285619+0000","last_fullsized":"2024-04-16T17:05:22.285619+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:22:33.677848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T17:04:21.497711+0000","last_change":"2024-04-16T17:03:24.978253+0000","last_active":"2024-04-16T17:04:21.497711+0000","last_peered":"2024-04-16T17:04:21.497711+0000","last_clean":"2024-04-16T17:04:21.497711+0000","last_became_active":"2024-04-16T17:03:24.977956+0000","last_became_peered":"2024-04-16T17:03:24.977956+0000","last_unstale":"2024-04-16T17:04:21.497711+0000","last_undegraded":"2024-04-16T17:04:21.497711+0000","last_fullsized":"2024-04-16T17:04:21.497711+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:59:45.102383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T17:04:22.198914+0000","last_change":"2024-04-16T17:03:29.265312+0000","last_active":"2024-04-16T17:04:22.198914+0000","last_peered":"2024-04-16T17:04:22.198914+0000","last_clean":"2024-04-16T17:04:22.198914+0000","last_became_active":"2024-04-16T17:03:29.264891+0000","last_became_peered":"2024-04-16T17:03:29.264891+0000","last_unstale":"2024-04-16T17:04:22.198914+0000","last_undegraded":"2024-04-16T17:04:22.198914+0000","last_fullsized":"2024-04-16T17:04:22.198914+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:26:40.088127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"90'1","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.510918+0000","last_change":"2024-04-16T17:03:24.170935+0000","last_active":"2024-04-16T17:05:22.510918+0000","last_peered":"2024-04-16T17:05:22.510918+0000","last_clean":"2024-04-16T17:05:22.510918+0000","last_became_active":"2024-04-16T17:03:24.170609+0000","last_became_peered":"2024-04-16T17:03:24.170609+0000","last_unstale":"2024-04-16T17:05:22.510918+0000","last_undegraded":"2024-04-16T17:05:22.510918+0000","last_fullsized":"2024-04-16T17:05:22.510918+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:18:33.516309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T17:04:21.497654+0000","last_change":"2024-04-16T17:03:24.985434+0000","last_active":"2024-04-16T17:04:21.497654+0000","last_peered":"2024-04-16T17:04:21.497654+0000","last_clean":"2024-04-16T17:04:21.497654+0000","last_became_active":"2024-04-16T17:03:24.984388+0000","last_became_peered":"2024-04-16T17:03:24.984388+0000","last_unstale":"2024-04-16T17:04:21.497654+0000","last_undegraded":"2024-04-16T17:04:21.497654+0000","last_fullsized":"2024-04-16T17:04:21.497654+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:43:26.355572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T17:04:22.199112+0000","last_change":"2024-04-16T17:03:29.301455+0000","last_active":"2024-04-16T17:04:22.199112+0000","last_peered":"2024-04-16T17:04:22.199112+0000","last_clean":"2024-04-16T17:04:22.199112+0000","last_became_active":"2024-04-16T17:03:29.301028+0000","last_became_peered":"2024-04-16T17:03:29.301028+0000","last_unstale":"2024-04-16T17:04:22.199112+0000","last_undegraded":"2024-04-16T17:04:22.199112+0000","last_fullsized":"2024-04-16T17:04:22.199112+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:54:49.305379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"92'8","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T17:05:20.151834+0000","last_change":"2024-04-16T17:03:24.962786+0000","last_active":"2024-04-16T17:05:20.151834+0000","last_peered":"2024-04-16T17:05:20.151834+0000","last_clean":"2024-04-16T17:05:20.151834+0000","last_became_active":"2024-04-16T17:03:24.961704+0000","last_became_peered":"2024-04-16T17:03:24.961704+0000","last_unstale":"2024-04-16T17:05:20.151834+0000","last_undegraded":"2024-04-16T17:05:20.151834+0000","last_fullsized":"2024-04-16T17:05:20.151834+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:22:27.957607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"92'5","reported_seq":120,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T17:05:20.132338+0000","last_change":"2024-04-16T17:03:24.175617+0000","last_active":"2024-04-16T17:05:20.132338+0000","last_peered":"2024-04-16T17:05:20.132338+0000","last_clean":"2024-04-16T17:05:20.132338+0000","last_became_active":"2024-04-16T17:03:24.175272+0000","last_became_peered":"2024-04-16T17:03:24.175272+0000","last_unstale":"2024-04-16T17:05:20.132338+0000","last_undegraded":"2024-04-16T17:05:20.132338+0000","last_fullsized":"2024-04-16T17:05:20.132338+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:27:44.738030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":88,"num_read_kb":88,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"90'1","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.511030+0000","last_change":"2024-04-16T17:04:21.512033+0000","last_active":"2024-04-16T17:05:22.511030+0000","last_peered":"2024-04-16T17:05:22.511030+0000","last_clean":"2024-04-16T17:05:22.511030+0000","last_became_active":"2024-04-16T17:04:21.506718+0000","last_became_peered":"2024-04-16T17:04:21.506718+0000","last_unstale":"2024-04-16T17:05:22.511030+0000","last_undegraded":"2024-04-16T17:05:22.511030+0000","last_fullsized":"2024-04-16T17:05:22.511030+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:53:48.953887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,10],"acting":[7,5,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T17:04:21.501025+0000","last_change":"2024-04-16T17:03:24.983629+0000","last_active":"2024-04-16T17:04:21.501025+0000","last_peered":"2024-04-16T17:04:21.501025+0000","last_clean":"2024-04-16T17:04:21.501025+0000","last_became_active":"2024-04-16T17:03:24.979969+0000","last_became_peered":"2024-04-16T17:03:24.979969+0000","last_unstale":"2024-04-16T17:04:21.501025+0000","last_undegraded":"2024-04-16T17:04:21.501025+0000","last_fullsized":"2024-04-16T17:04:21.501025+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:01:24.391732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"90'3","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:05:21.276492+0000","last_change":"2024-04-16T17:03:24.152475+0000","last_active":"2024-04-16T17:05:21.276492+0000","last_peered":"2024-04-16T17:05:21.276492+0000","last_clean":"2024-04-16T17:05:21.276492+0000","last_became_active":"2024-04-16T17:03:24.152198+0000","last_became_peered":"2024-04-16T17:03:24.152198+0000","last_unstale":"2024-04-16T17:05:21.276492+0000","last_undegraded":"2024-04-16T17:05:21.276492+0000","last_fullsized":"2024-04-16T17:05:21.276492+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:49:33.914325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":364,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.510449+0000","last_change":"2024-04-16T17:02:02.111335+0000","last_active":"2024-04-16T17:05:22.510449+0000","last_peered":"2024-04-16T17:05:22.510449+0000","last_clean":"2024-04-16T17:05:22.510449+0000","last_became_active":"2024-04-16T17:02:02.110928+0000","last_became_peered":"2024-04-16T17:02:02.110928+0000","last_unstale":"2024-04-16T17:05:22.510449+0000","last_undegraded":"2024-04-16T17:05:22.510449+0000","last_fullsized":"2024-04-16T17:05:22.510449+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T16:58:07.081753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T16:58:07.081753+0000","last_clean_scrub_stamp":"2024-04-16T16:58:07.081753+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:27:02.865664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T17:04:21.505247+0000","last_change":"2024-04-16T17:03:29.301691+0000","last_active":"2024-04-16T17:04:21.505247+0000","last_peered":"2024-04-16T17:04:21.505247+0000","last_clean":"2024-04-16T17:04:21.505247+0000","last_became_active":"2024-04-16T17:03:29.301277+0000","last_became_peered":"2024-04-16T17:03:29.301277+0000","last_unstale":"2024-04-16T17:04:21.505247+0000","last_undegraded":"2024-04-16T17:04:21.505247+0000","last_fullsized":"2024-04-16T17:04:21.505247+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:45:16.727610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"92'5","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T17:05:20.150650+0000","last_change":"2024-04-16T17:03:24.970939+0000","last_active":"2024-04-16T17:05:20.150650+0000","last_peered":"2024-04-16T17:05:20.150650+0000","last_clean":"2024-04-16T17:05:20.150650+0000","last_became_active":"2024-04-16T17:03:24.970581+0000","last_became_peered":"2024-04-16T17:03:24.970581+0000","last_unstale":"2024-04-16T17:05:20.150650+0000","last_undegraded":"2024-04-16T17:05:20.150650+0000","last_fullsized":"2024-04-16T17:05:20.150650+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:33:07.448597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"90'3","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.510566+0000","last_change":"2024-04-16T17:03:24.155454+0000","last_active":"2024-04-16T17:05:22.510566+0000","last_peered":"2024-04-16T17:05:22.510566+0000","last_clean":"2024-04-16T17:05:22.510566+0000","last_became_active":"2024-04-16T17:03:24.155155+0000","last_became_peered":"2024-04-16T17:03:24.155155+0000","last_unstale":"2024-04-16T17:05:22.510566+0000","last_undegraded":"2024-04-16T17:05:22.510566+0000","last_fullsized":"2024-04-16T17:05:22.510566+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:34:08.618002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T17:04:21.500178+0000","last_change":"2024-04-16T17:03:29.255538+0000","last_active":"2024-04-16T17:04:21.500178+0000","last_peered":"2024-04-16T17:04:21.500178+0000","last_clean":"2024-04-16T17:04:21.500178+0000","last_became_active":"2024-04-16T17:03:29.255107+0000","last_became_peered":"2024-04-16T17:03:29.255107+0000","last_unstale":"2024-04-16T17:04:21.500178+0000","last_undegraded":"2024-04-16T17:04:21.500178+0000","last_fullsized":"2024-04-16T17:04:21.500178+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:31:58.458641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"90'1","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:05:21.282044+0000","last_change":"2024-04-16T17:03:24.158816+0000","last_active":"2024-04-16T17:05:21.282044+0000","last_peered":"2024-04-16T17:05:21.282044+0000","last_clean":"2024-04-16T17:05:21.282044+0000","last_became_active":"2024-04-16T17:03:24.158518+0000","last_became_peered":"2024-04-16T17:03:24.158518+0000","last_unstale":"2024-04-16T17:05:21.282044+0000","last_undegraded":"2024-04-16T17:05:21.282044+0000","last_fullsized":"2024-04-16T17:05:21.282044+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:54:55.807975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"92'9","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T17:05:20.149239+0000","last_change":"2024-04-16T17:03:24.977190+0000","last_active":"2024-04-16T17:05:20.149239+0000","last_peered":"2024-04-16T17:05:20.149239+0000","last_clean":"2024-04-16T17:05:20.149239+0000","last_became_active":"2024-04-16T17:03:24.976840+0000","last_became_peered":"2024-04-16T17:03:24.976840+0000","last_unstale":"2024-04-16T17:05:20.149239+0000","last_undegraded":"2024-04-16T17:05:20.149239+0000","last_fullsized":"2024-04-16T17:05:20.149239+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:21:51.294150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T17:04:21.500534+0000","last_change":"2024-04-16T17:03:29.301423+0000","last_active":"2024-04-16T17:04:21.500534+0000","last_peered":"2024-04-16T17:04:21.500534+0000","last_clean":"2024-04-16T17:04:21.500534+0000","last_became_active":"2024-04-16T17:03:29.300888+0000","last_became_peered":"2024-04-16T17:03:29.300888+0000","last_unstale":"2024-04-16T17:04:21.500534+0000","last_undegraded":"2024-04-16T17:04:21.500534+0000","last_fullsized":"2024-04-16T17:04:21.500534+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:13:54.188090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"92'11","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.287648+0000","last_change":"2024-04-16T17:03:24.167643+0000","last_active":"2024-04-16T17:05:22.287648+0000","last_peered":"2024-04-16T17:05:22.287648+0000","last_clean":"2024-04-16T17:05:22.287648+0000","last_became_active":"2024-04-16T17:03:24.167103+0000","last_became_peered":"2024-04-16T17:03:24.167103+0000","last_unstale":"2024-04-16T17:05:22.287648+0000","last_undegraded":"2024-04-16T17:05:22.287648+0000","last_fullsized":"2024-04-16T17:05:22.287648+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:33:16.809371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T17:04:21.505980+0000","last_change":"2024-04-16T17:03:24.976981+0000","last_active":"2024-04-16T17:04:21.505980+0000","last_peered":"2024-04-16T17:04:21.505980+0000","last_clean":"2024-04-16T17:04:21.505980+0000","last_became_active":"2024-04-16T17:03:24.975385+0000","last_became_peered":"2024-04-16T17:03:24.975385+0000","last_unstale":"2024-04-16T17:04:21.505980+0000","last_undegraded":"2024-04-16T17:04:21.505980+0000","last_fullsized":"2024-04-16T17:04:21.505980+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:27:03.104838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T17:04:21.501741+0000","last_change":"2024-04-16T17:03:29.244767+0000","last_active":"2024-04-16T17:04:21.501741+0000","last_peered":"2024-04-16T17:04:21.501741+0000","last_clean":"2024-04-16T17:04:21.501741+0000","last_became_active":"2024-04-16T17:03:29.244205+0000","last_became_peered":"2024-04-16T17:03:29.244205+0000","last_unstale":"2024-04-16T17:04:21.501741+0000","last_undegraded":"2024-04-16T17:04:21.501741+0000","last_fullsized":"2024-04-16T17:04:21.501741+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:51:46.652387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"90'4","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:05:21.276373+0000","last_change":"2024-04-16T17:03:24.164916+0000","last_active":"2024-04-16T17:05:21.276373+0000","last_peered":"2024-04-16T17:05:21.276373+0000","last_clean":"2024-04-16T17:05:21.276373+0000","last_became_active":"2024-04-16T17:03:24.164648+0000","last_became_peered":"2024-04-16T17:03:24.164648+0000","last_unstale":"2024-04-16T17:05:21.276373+0000","last_undegraded":"2024-04-16T17:05:21.276373+0000","last_fullsized":"2024-04-16T17:05:21.276373+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:23:00.198785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"92'7","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:05:21.301427+0000","last_change":"2024-04-16T17:03:24.994393+0000","last_active":"2024-04-16T17:05:21.301427+0000","last_peered":"2024-04-16T17:05:21.301427+0000","last_clean":"2024-04-16T17:05:21.301427+0000","last_became_active":"2024-04-16T17:03:24.994002+0000","last_became_peered":"2024-04-16T17:03:24.994002+0000","last_unstale":"2024-04-16T17:05:21.301427+0000","last_undegraded":"2024-04-16T17:05:21.301427+0000","last_fullsized":"2024-04-16T17:05:21.301427+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:23:44.596781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":20,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:05:21.276259+0000","last_change":"2024-04-16T17:03:29.302009+0000","last_active":"2024-04-16T17:05:21.276259+0000","last_peered":"2024-04-16T17:05:21.276259+0000","last_clean":"2024-04-16T17:05:21.276259+0000","last_became_active":"2024-04-16T17:03:29.301183+0000","last_became_peered":"2024-04-16T17:03:29.301183+0000","last_unstale":"2024-04-16T17:05:21.276259+0000","last_undegraded":"2024-04-16T17:05:21.276259+0000","last_fullsized":"2024-04-16T17:05:21.276259+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:30:19.561875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"90'2","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.509857+0000","last_change":"2024-04-16T17:03:24.176829+0000","last_active":"2024-04-16T17:05:22.509857+0000","last_peered":"2024-04-16T17:05:22.509857+0000","last_clean":"2024-04-16T17:05:22.509857+0000","last_became_active":"2024-04-16T17:03:24.176555+0000","last_became_peered":"2024-04-16T17:03:24.176555+0000","last_unstale":"2024-04-16T17:05:22.509857+0000","last_undegraded":"2024-04-16T17:05:22.509857+0000","last_fullsized":"2024-04-16T17:05:22.509857+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:05:40.602150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"92'2","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:05:21.281441+0000","last_change":"2024-04-16T17:03:24.965746+0000","last_active":"2024-04-16T17:05:21.281441+0000","last_peered":"2024-04-16T17:05:21.281441+0000","last_clean":"2024-04-16T17:05:21.281441+0000","last_became_active":"2024-04-16T17:03:24.965241+0000","last_became_peered":"2024-04-16T17:03:24.965241+0000","last_unstale":"2024-04-16T17:05:21.281441+0000","last_undegraded":"2024-04-16T17:05:21.281441+0000","last_fullsized":"2024-04-16T17:05:21.281441+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:51:34.407565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.286457+0000","last_change":"2024-04-16T17:03:29.261441+0000","last_active":"2024-04-16T17:05:22.286457+0000","last_peered":"2024-04-16T17:05:22.286457+0000","last_clean":"2024-04-16T17:05:22.286457+0000","last_became_active":"2024-04-16T17:03:29.259538+0000","last_became_peered":"2024-04-16T17:03:29.259538+0000","last_unstale":"2024-04-16T17:05:22.286457+0000","last_undegraded":"2024-04-16T17:05:22.286457+0000","last_fullsized":"2024-04-16T17:05:22.286457+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:12:27.092053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"90'3","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T17:04:21.501265+0000","last_change":"2024-04-16T17:03:24.172053+0000","last_active":"2024-04-16T17:04:21.501265+0000","last_peered":"2024-04-16T17:04:21.501265+0000","last_clean":"2024-04-16T17:04:21.501265+0000","last_became_active":"2024-04-16T17:03:24.171709+0000","last_became_peered":"2024-04-16T17:03:24.171709+0000","last_unstale":"2024-04-16T17:04:21.501265+0000","last_undegraded":"2024-04-16T17:04:21.501265+0000","last_fullsized":"2024-04-16T17:04:21.501265+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:40:12.307756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"92'9","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T17:05:20.147345+0000","last_change":"2024-04-16T17:03:24.984267+0000","last_active":"2024-04-16T17:05:20.147345+0000","last_peered":"2024-04-16T17:05:20.147345+0000","last_clean":"2024-04-16T17:05:20.147345+0000","last_became_active":"2024-04-16T17:03:24.983888+0000","last_became_peered":"2024-04-16T17:03:24.983888+0000","last_unstale":"2024-04-16T17:05:20.147345+0000","last_undegraded":"2024-04-16T17:05:20.147345+0000","last_fullsized":"2024-04-16T17:05:20.147345+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:16:23.814917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"90'1","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:05:21.276083+0000","last_change":"2024-04-16T17:03:29.251716+0000","last_active":"2024-04-16T17:05:21.276083+0000","last_peered":"2024-04-16T17:05:21.276083+0000","last_clean":"2024-04-16T17:05:21.276083+0000","last_became_active":"2024-04-16T17:03:29.251317+0000","last_became_peered":"2024-04-16T17:03:29.251317+0000","last_unstale":"2024-04-16T17:05:21.276083+0000","last_undegraded":"2024-04-16T17:05:21.276083+0000","last_fullsized":"2024-04-16T17:05:21.276083+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:48:54.680195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"90'1","reported_seq":15,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T17:04:21.515765+0000","last_change":"2024-04-16T17:04:21.515306+0000","last_active":"2024-04-16T17:04:21.515765+0000","last_peered":"2024-04-16T17:04:21.515765+0000","last_clean":"2024-04-16T17:04:21.515765+0000","last_became_active":"2024-04-16T17:04:21.508377+0000","last_became_peered":"2024-04-16T17:04:21.508377+0000","last_unstale":"2024-04-16T17:04:21.515765+0000","last_undegraded":"2024-04-16T17:04:21.515765+0000","last_fullsized":"2024-04-16T17:04:21.515765+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:28:25.000406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T17:04:21.500332+0000","last_change":"2024-04-16T17:03:24.995547+0000","last_active":"2024-04-16T17:04:21.500332+0000","last_peered":"2024-04-16T17:04:21.500332+0000","last_clean":"2024-04-16T17:04:21.500332+0000","last_became_active":"2024-04-16T17:03:24.995096+0000","last_became_peered":"2024-04-16T17:03:24.995096+0000","last_unstale":"2024-04-16T17:04:21.500332+0000","last_undegraded":"2024-04-16T17:04:21.500332+0000","last_fullsized":"2024-04-16T17:04:21.500332+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:54:51.193117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":20,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:05:21.276420+0000","last_change":"2024-04-16T17:03:29.301526+0000","last_active":"2024-04-16T17:05:21.276420+0000","last_peered":"2024-04-16T17:05:21.276420+0000","last_clean":"2024-04-16T17:05:21.276420+0000","last_became_active":"2024-04-16T17:03:29.300808+0000","last_became_peered":"2024-04-16T17:03:29.300808+0000","last_unstale":"2024-04-16T17:05:21.276420+0000","last_undegraded":"2024-04-16T17:05:21.276420+0000","last_fullsized":"2024-04-16T17:05:21.276420+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:01:14.717548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"90'4","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:05:21.281992+0000","last_change":"2024-04-16T17:03:24.152645+0000","last_active":"2024-04-16T17:05:21.281992+0000","last_peered":"2024-04-16T17:05:21.281992+0000","last_clean":"2024-04-16T17:05:21.281992+0000","last_became_active":"2024-04-16T17:03:24.152281+0000","last_became_peered":"2024-04-16T17:03:24.152281+0000","last_unstale":"2024-04-16T17:05:21.281992+0000","last_undegraded":"2024-04-16T17:05:21.281992+0000","last_fullsized":"2024-04-16T17:05:21.281992+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:33:51.377421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"92'5","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.511176+0000","last_change":"2024-04-16T17:03:24.979827+0000","last_active":"2024-04-16T17:05:22.511176+0000","last_peered":"2024-04-16T17:05:22.511176+0000","last_clean":"2024-04-16T17:05:22.511176+0000","last_became_active":"2024-04-16T17:03:24.979452+0000","last_became_peered":"2024-04-16T17:03:24.979452+0000","last_unstale":"2024-04-16T17:05:22.511176+0000","last_undegraded":"2024-04-16T17:05:22.511176+0000","last_fullsized":"2024-04-16T17:05:22.511176+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:07:09.259366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.510047+0000","last_change":"2024-04-16T17:03:29.255519+0000","last_active":"2024-04-16T17:05:22.510047+0000","last_peered":"2024-04-16T17:05:22.510047+0000","last_clean":"2024-04-16T17:05:22.510047+0000","last_became_active":"2024-04-16T17:03:29.255104+0000","last_became_peered":"2024-04-16T17:03:29.255104+0000","last_unstale":"2024-04-16T17:05:22.510047+0000","last_undegraded":"2024-04-16T17:05:22.510047+0000","last_fullsized":"2024-04-16T17:05:22.510047+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:27:34.496199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"90'3","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T17:04:21.502127+0000","last_change":"2024-04-16T17:03:24.161145+0000","last_active":"2024-04-16T17:04:21.502127+0000","last_peered":"2024-04-16T17:04:21.502127+0000","last_clean":"2024-04-16T17:04:21.502127+0000","last_became_active":"2024-04-16T17:03:24.160824+0000","last_became_peered":"2024-04-16T17:03:24.160824+0000","last_unstale":"2024-04-16T17:04:21.502127+0000","last_undegraded":"2024-04-16T17:04:21.502127+0000","last_fullsized":"2024-04-16T17:04:21.502127+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:02:48.805638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"92'7","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.511418+0000","last_change":"2024-04-16T17:03:24.966513+0000","last_active":"2024-04-16T17:05:22.511418+0000","last_peered":"2024-04-16T17:05:22.511418+0000","last_clean":"2024-04-16T17:05:22.511418+0000","last_became_active":"2024-04-16T17:03:24.966074+0000","last_became_peered":"2024-04-16T17:03:24.966074+0000","last_unstale":"2024-04-16T17:05:22.511418+0000","last_undegraded":"2024-04-16T17:05:22.511418+0000","last_fullsized":"2024-04-16T17:05:22.511418+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:50:21.579901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:05:21.300897+0000","last_change":"2024-04-16T17:04:22.150786+0000","last_active":"2024-04-16T17:05:21.300897+0000","last_peered":"2024-04-16T17:05:21.300897+0000","last_clean":"2024-04-16T17:05:21.300897+0000","last_became_active":"2024-04-16T17:04:22.149344+0000","last_became_peered":"2024-04-16T17:04:22.149344+0000","last_unstale":"2024-04-16T17:05:21.300897+0000","last_undegraded":"2024-04-16T17:05:21.300897+0000","last_fullsized":"2024-04-16T17:05:21.300897+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:18:49.421095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,1,4],"acting":[0,8,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"85'1","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T17:04:21.505354+0000","last_change":"2024-04-16T17:03:24.158032+0000","last_active":"2024-04-16T17:04:21.505354+0000","last_peered":"2024-04-16T17:04:21.505354+0000","last_clean":"2024-04-16T17:04:21.505354+0000","last_became_active":"2024-04-16T17:03:24.157689+0000","last_became_peered":"2024-04-16T17:03:24.157689+0000","last_unstale":"2024-04-16T17:04:21.505354+0000","last_undegraded":"2024-04-16T17:04:21.505354+0000","last_fullsized":"2024-04-16T17:04:21.505354+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:38:18.240889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"92'6","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.511335+0000","last_change":"2024-04-16T17:04:21.519999+0000","last_active":"2024-04-16T17:05:22.511335+0000","last_peered":"2024-04-16T17:05:22.511335+0000","last_clean":"2024-04-16T17:05:22.511335+0000","last_became_active":"2024-04-16T17:04:21.519621+0000","last_became_peered":"2024-04-16T17:04:21.519621+0000","last_unstale":"2024-04-16T17:05:22.511335+0000","last_undegraded":"2024-04-16T17:05:22.511335+0000","last_fullsized":"2024-04-16T17:05:22.511335+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:50:13.266369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T17:04:21.504113+0000","last_change":"2024-04-16T17:03:29.245176+0000","last_active":"2024-04-16T17:04:21.504113+0000","last_peered":"2024-04-16T17:04:21.504113+0000","last_clean":"2024-04-16T17:04:21.504113+0000","last_became_active":"2024-04-16T17:03:29.244763+0000","last_became_peered":"2024-04-16T17:03:29.244763+0000","last_unstale":"2024-04-16T17:04:21.504113+0000","last_undegraded":"2024-04-16T17:04:21.504113+0000","last_fullsized":"2024-04-16T17:04:21.504113+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:52:30.381417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T17:04:22.199271+0000","last_change":"2024-04-16T17:03:24.168068+0000","last_active":"2024-04-16T17:04:22.199271+0000","last_peered":"2024-04-16T17:04:22.199271+0000","last_clean":"2024-04-16T17:04:22.199271+0000","last_became_active":"2024-04-16T17:03:24.167694+0000","last_became_peered":"2024-04-16T17:03:24.167694+0000","last_unstale":"2024-04-16T17:04:22.199271+0000","last_undegraded":"2024-04-16T17:04:22.199271+0000","last_fullsized":"2024-04-16T17:04:22.199271+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:13:35.692306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"92'9","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.510867+0000","last_change":"2024-04-16T17:03:24.966145+0000","last_active":"2024-04-16T17:05:22.510867+0000","last_peered":"2024-04-16T17:05:22.510867+0000","last_clean":"2024-04-16T17:05:22.510867+0000","last_became_active":"2024-04-16T17:03:24.965813+0000","last_became_peered":"2024-04-16T17:03:24.965813+0000","last_unstale":"2024-04-16T17:05:22.510867+0000","last_undegraded":"2024-04-16T17:05:22.510867+0000","last_fullsized":"2024-04-16T17:05:22.510867+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:43:43.224130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"92'9","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T17:05:20.145157+0000","last_change":"2024-04-16T17:03:24.970941+0000","last_active":"2024-04-16T17:05:20.145157+0000","last_peered":"2024-04-16T17:05:20.145157+0000","last_clean":"2024-04-16T17:05:20.145157+0000","last_became_active":"2024-04-16T17:03:24.970587+0000","last_became_peered":"2024-04-16T17:03:24.970587+0000","last_unstale":"2024-04-16T17:05:20.145157+0000","last_undegraded":"2024-04-16T17:05:20.145157+0000","last_fullsized":"2024-04-16T17:05:20.145157+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:14:13.930011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T17:04:21.502798+0000","last_change":"2024-04-16T17:03:24.967167+0000","last_active":"2024-04-16T17:04:21.502798+0000","last_peered":"2024-04-16T17:04:21.502798+0000","last_clean":"2024-04-16T17:04:21.502798+0000","last_became_active":"2024-04-16T17:03:24.966787+0000","last_became_peered":"2024-04-16T17:03:24.966787+0000","last_unstale":"2024-04-16T17:04:21.502798+0000","last_undegraded":"2024-04-16T17:04:21.502798+0000","last_fullsized":"2024-04-16T17:04:21.502798+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:51:39.510220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T17:04:21.502859+0000","last_change":"2024-04-16T17:03:24.981115+0000","last_active":"2024-04-16T17:04:21.502859+0000","last_peered":"2024-04-16T17:04:21.502859+0000","last_clean":"2024-04-16T17:04:21.502859+0000","last_became_active":"2024-04-16T17:03:24.980685+0000","last_became_peered":"2024-04-16T17:03:24.980685+0000","last_unstale":"2024-04-16T17:04:21.502859+0000","last_undegraded":"2024-04-16T17:04:21.502859+0000","last_fullsized":"2024-04-16T17:04:21.502859+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:20:44.066932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"92'7","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-16T17:05:21.301262+0000","last_change":"2024-04-16T17:03:24.990421+0000","last_active":"2024-04-16T17:05:21.301262+0000","last_peered":"2024-04-16T17:05:21.301262+0000","last_clean":"2024-04-16T17:05:21.301262+0000","last_became_active":"2024-04-16T17:03:24.989940+0000","last_became_peered":"2024-04-16T17:03:24.989940+0000","last_unstale":"2024-04-16T17:05:21.301262+0000","last_undegraded":"2024-04-16T17:05:21.301262+0000","last_fullsized":"2024-04-16T17:05:21.301262+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:19:57.548067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"92'4","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T17:05:20.157624+0000","last_change":"2024-04-16T17:03:24.991854+0000","last_active":"2024-04-16T17:05:20.157624+0000","last_peered":"2024-04-16T17:05:20.157624+0000","last_clean":"2024-04-16T17:05:20.157624+0000","last_became_active":"2024-04-16T17:03:24.990964+0000","last_became_peered":"2024-04-16T17:03:24.990964+0000","last_unstale":"2024-04-16T17:05:20.157624+0000","last_undegraded":"2024-04-16T17:05:20.157624+0000","last_fullsized":"2024-04-16T17:05:20.157624+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:10:10.618708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"92'6","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T17:05:20.141704+0000","last_change":"2024-04-16T17:04:21.517466+0000","last_active":"2024-04-16T17:05:20.141704+0000","last_peered":"2024-04-16T17:05:20.141704+0000","last_clean":"2024-04-16T17:05:20.141704+0000","last_became_active":"2024-04-16T17:04:21.517086+0000","last_became_peered":"2024-04-16T17:04:21.517086+0000","last_unstale":"2024-04-16T17:05:20.141704+0000","last_undegraded":"2024-04-16T17:05:20.141704+0000","last_fullsized":"2024-04-16T17:05:20.141704+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:54:33.417452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,9],"acting":[11,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T17:04:21.502644+0000","last_change":"2024-04-16T17:03:24.970377+0000","last_active":"2024-04-16T17:04:21.502644+0000","last_peered":"2024-04-16T17:04:21.502644+0000","last_clean":"2024-04-16T17:04:21.502644+0000","last_became_active":"2024-04-16T17:03:24.968856+0000","last_became_peered":"2024-04-16T17:03:24.968856+0000","last_unstale":"2024-04-16T17:04:21.502644+0000","last_undegraded":"2024-04-16T17:04:21.502644+0000","last_fullsized":"2024-04-16T17:04:21.502644+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:47:28.543120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T17:04:21.506256+0000","last_change":"2024-04-16T17:03:24.992877+0000","last_active":"2024-04-16T17:04:21.506256+0000","last_peered":"2024-04-16T17:04:21.506256+0000","last_clean":"2024-04-16T17:04:21.506256+0000","last_became_active":"2024-04-16T17:03:24.992484+0000","last_became_peered":"2024-04-16T17:03:24.992484+0000","last_unstale":"2024-04-16T17:04:21.506256+0000","last_undegraded":"2024-04-16T17:04:21.506256+0000","last_fullsized":"2024-04-16T17:04:21.506256+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:00:45.204782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"92'6","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T17:05:20.171924+0000","last_change":"2024-04-16T17:03:24.977794+0000","last_active":"2024-04-16T17:05:20.171924+0000","last_peered":"2024-04-16T17:05:20.171924+0000","last_clean":"2024-04-16T17:05:20.171924+0000","last_became_active":"2024-04-16T17:03:24.977434+0000","last_became_peered":"2024-04-16T17:03:24.977434+0000","last_unstale":"2024-04-16T17:05:20.171924+0000","last_undegraded":"2024-04-16T17:05:20.171924+0000","last_fullsized":"2024-04-16T17:05:20.171924+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:46:35.670357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T17:04:21.501616+0000","last_change":"2024-04-16T17:03:24.992225+0000","last_active":"2024-04-16T17:04:21.501616+0000","last_peered":"2024-04-16T17:04:21.501616+0000","last_clean":"2024-04-16T17:04:21.501616+0000","last_became_active":"2024-04-16T17:03:24.991904+0000","last_became_peered":"2024-04-16T17:03:24.991904+0000","last_unstale":"2024-04-16T17:04:21.501616+0000","last_undegraded":"2024-04-16T17:04:21.501616+0000","last_fullsized":"2024-04-16T17:04:21.501616+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:01:09.710698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"92'10","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.510323+0000","last_change":"2024-04-16T17:03:24.981322+0000","last_active":"2024-04-16T17:05:22.510323+0000","last_peered":"2024-04-16T17:05:22.510323+0000","last_clean":"2024-04-16T17:05:22.510323+0000","last_became_active":"2024-04-16T17:03:24.980920+0000","last_became_peered":"2024-04-16T17:03:24.980920+0000","last_unstale":"2024-04-16T17:05:22.510323+0000","last_undegraded":"2024-04-16T17:05:22.510323+0000","last_fullsized":"2024-04-16T17:05:22.510323+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:07:59.378512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"92'11","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T17:05:20.175123+0000","last_change":"2024-04-16T17:03:24.990117+0000","last_active":"2024-04-16T17:05:20.175123+0000","last_peered":"2024-04-16T17:05:20.175123+0000","last_clean":"2024-04-16T17:05:20.175123+0000","last_became_active":"2024-04-16T17:03:24.989734+0000","last_became_peered":"2024-04-16T17:03:24.989734+0000","last_unstale":"2024-04-16T17:05:20.175123+0000","last_undegraded":"2024-04-16T17:05:20.175123+0000","last_fullsized":"2024-04-16T17:05:20.175123+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:36:55.000702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"92'8","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-16T17:05:20.140582+0000","last_change":"2024-04-16T17:03:24.970032+0000","last_active":"2024-04-16T17:05:20.140582+0000","last_peered":"2024-04-16T17:05:20.140582+0000","last_clean":"2024-04-16T17:05:20.140582+0000","last_became_active":"2024-04-16T17:03:24.968608+0000","last_became_peered":"2024-04-16T17:03:24.968608+0000","last_unstale":"2024-04-16T17:05:20.140582+0000","last_undegraded":"2024-04-16T17:05:20.140582+0000","last_fullsized":"2024-04-16T17:05:20.140582+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:24:03.918418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":372,"ondisk_log_size":372,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":283858,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":68030,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":551,"num_read_kb":348,"num_write":171,"num_write_kb":121,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":616,"num_keys_recovered":20,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":204371,"data_compressed":4887,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":147,"ondisk_log_size":147,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351013,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":1007616,"data_stored":503892,"data_compressed":20691,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612652,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32300,"kb_used_data":1124,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712852,"statfs":{"total":95995035648,"available":95961960448,"internally_reserved":0,"allocated":1150976,"data_stored":1158969,"data_compressed":58418,"data_compressed_allocated":77824,"data_compressed_original":655360,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776177,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32204,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712948,"statfs":{"total":95995035648,"available":95962058752,"internally_reserved":0,"allocated":1044480,"data_stored":522868,"data_compressed":26684,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037816,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32184,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712968,"statfs":{"total":95995035648,"available":95962079232,"internally_reserved":0,"allocated":1028096,"data_stored":512840,"data_compressed":23684,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299454,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36412,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708740,"statfs":{"total":95995035648,"available":95957749760,"internally_reserved":0,"allocated":1093632,"data_stored":1119370,"data_compressed":62782,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462979,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32152,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713000,"statfs":{"total":95995035648,"available":95962112000,"internally_reserved":0,"allocated":999424,"data_stored":499270,"data_compressed":20691,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574944,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32236,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712916,"statfs":{"total":95995035648,"available":95962025984,"internally_reserved":0,"allocated":1081344,"data_stored":572775,"data_compressed":25313,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738468,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32192,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712960,"statfs":{"total":95995035648,"available":95962071040,"internally_reserved":0,"allocated":1036288,"data_stored":510066,"data_compressed":23684,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411417,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28076,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717076,"statfs":{"total":95995035648,"available":95966285824,"internally_reserved":0,"allocated":1085440,"data_stored":564243,"data_compressed":22320,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149779,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32168,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712984,"statfs":{"total":95995035648,"available":95962095616,"internally_reserved":0,"allocated":1011712,"data_stored":499464,"data_compressed":20691,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888142,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1011712,"data_stored":504452,"data_compressed":20691,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724617,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28068,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717084,"statfs":{"total":95995035648,"available":95966294016,"internally_reserved":0,"allocated":1077248,"data_stored":1095652,"data_compressed":56789,"data_compressed_allocated":73728,"data_compressed_original":647168,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34456,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34456,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34456,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36098,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36098,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36098,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36098,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36098,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":64187,"data_compressed":1629,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":65377,"data_compressed":1629,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":156,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":65325,"data_compressed":1629,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-16T17:05:26.881 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45 shell --fsid b1fecd38-fc11-11ee-bc90-c7b262605968 -- ceph pg dump --format=json 2024-04-16T17:05:27.388 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/mon.a/config 2024-04-16T17:05:27.555 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:05:27 smithi117 ceph-mon[36730]: from='client.14940 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-16T17:05:27.555 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:05:27 smithi117 ceph-mon[36730]: pgmap v387: 129 pgs: 1 peering, 128 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 895 B/s wr, 8 op/s 2024-04-16T17:05:27.556 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:05:27 smithi045 ceph-mon[31676]: from='client.14940 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-16T17:05:27.556 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:05:27 smithi045 ceph-mon[31676]: pgmap v387: 129 pgs: 1 peering, 128 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 895 B/s wr, 8 op/s 2024-04-16T17:05:27.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:05:27 smithi104 ceph-mon[36457]: from='client.14940 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-16T17:05:27.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:05:27 smithi104 ceph-mon[36457]: pgmap v387: 129 pgs: 1 peering, 128 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 895 B/s wr, 8 op/s 2024-04-16T17:05:28.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:05:28 smithi104 ceph-mon[36457]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-04-16T17:05:28.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:05:28 smithi104 ceph-mon[36457]: Cluster is now healthy 2024-04-16T17:05:28.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:05:28 smithi045 ceph-mon[31676]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-04-16T17:05:28.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:05:28 smithi045 ceph-mon[31676]: Cluster is now healthy 2024-04-16T17:05:28.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:05:28 smithi117 ceph-mon[36730]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-04-16T17:05:28.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:05:28 smithi117 ceph-mon[36730]: Cluster is now healthy 2024-04-16T17:05:29.478 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:05:29 smithi045 ceph-mon[31676]: pgmap v388: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 895 B/s wr, 7 op/s 2024-04-16T17:05:29.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:29.676+0000 7fdf3fc00700 1 -- 172.21.15.45:0/3611487464 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdf3810e460 msgr2=0x7fdf38116030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:29.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:29.676+0000 7fdf3fc00700 1 --2- 172.21.15.45:0/3611487464 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdf3810e460 0x7fdf38116030 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fdf300099c0 tx=0x7fdf30009cd0 comp rx=0 tx=0).stop 2024-04-16T17:05:29.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:29.676+0000 7fdf3fc00700 1 -- 172.21.15.45:0/3611487464 shutdown_connections 2024-04-16T17:05:29.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:29.676+0000 7fdf3fc00700 1 --2- 172.21.15.45:0/3611487464 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdf38109380 0x7fdf38109760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:29.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:29.676+0000 7fdf3fc00700 1 --2- 172.21.15.45:0/3611487464 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdf3810e460 0x7fdf38116030 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:29.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:29.676+0000 7fdf3fc00700 1 --2- 172.21.15.45:0/3611487464 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdf38109d30 0x7fdf3810dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:29.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:29.676+0000 7fdf3fc00700 1 -- 172.21.15.45:0/3611487464 >> 172.21.15.45:0/3611487464 conn(0x7fdf380762c0 msgr2=0x7fdf38107910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:05:29.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:29.676+0000 7fdf3fc00700 1 -- 172.21.15.45:0/3611487464 shutdown_connections 2024-04-16T17:05:29.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:29.676+0000 7fdf3fc00700 1 -- 172.21.15.45:0/3611487464 wait complete. 2024-04-16T17:05:29.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:29.678+0000 7fdf3fc00700 1 Processor -- start 2024-04-16T17:05:29.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:29.678+0000 7fdf3fc00700 1 -- start start 2024-04-16T17:05:29.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:29.678+0000 7fdf3fc00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdf38109380 0x7fdf381a5660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:29.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:29.678+0000 7fdf3fc00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdf38109d30 0x7fdf381a5ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:29.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:29.678+0000 7fdf3fc00700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdf3810e460 0x7fdf3819f780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:29.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:29.678+0000 7fdf3fc00700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fdf381194c0 con 0x7fdf38109d30 2024-04-16T17:05:29.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:29.678+0000 7fdf3fc00700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fdf38119340 con 0x7fdf3810e460 2024-04-16T17:05:29.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:29.678+0000 7fdf3fc00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fdf38119640 con 0x7fdf38109380 2024-04-16T17:05:29.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:29.679+0000 7fdf3d200700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdf38109380 0x7fdf381a5660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:29.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:29.679+0000 7fdf3d200700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdf38109380 0x7fdf381a5660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:56828/0 (socket says 172.21.15.45:56828) 2024-04-16T17:05:29.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:29.679+0000 7fdf37e00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdf38109d30 0x7fdf381a5ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:29.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:29.679+0000 7fdf3d200700 1 -- 172.21.15.45:0/3853017063 learned_addr learned my addr 172.21.15.45:0/3853017063 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:05:29.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:29.679+0000 7fdf3dc00700 1 --2- 172.21.15.45:0/3853017063 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdf3810e460 0x7fdf3819f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:29.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:29.679+0000 7fdf37e00700 1 -- 172.21.15.45:0/3853017063 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdf38109380 msgr2=0x7fdf381a5660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:29.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:29.679+0000 7fdf37e00700 1 --2- 172.21.15.45:0/3853017063 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdf38109380 0x7fdf381a5660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:29.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:29.679+0000 7fdf37e00700 1 -- 172.21.15.45:0/3853017063 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdf3810e460 msgr2=0x7fdf3819f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:29.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:29.679+0000 7fdf37e00700 1 --2- 172.21.15.45:0/3853017063 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdf3810e460 0x7fdf3819f780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:29.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:29.679+0000 7fdf37e00700 1 -- 172.21.15.45:0/3853017063 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf381a0060 con 0x7fdf38109d30 2024-04-16T17:05:29.683 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:05:29 smithi104 ceph-mon[36457]: pgmap v388: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 895 B/s wr, 7 op/s 2024-04-16T17:05:29.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:29.680+0000 7fdf37e00700 1 --2- 172.21.15.45:0/3853017063 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdf38109d30 0x7fdf381a5ba0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fdf2400e9c0 tx=0x7fdf2400ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:05:29.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:29.680+0000 7fdf35600700 1 -- 172.21.15.45:0/3853017063 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf2400cb70 con 0x7fdf38109d30 2024-04-16T17:05:29.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:29.680+0000 7fdf35600700 1 -- 172.21.15.45:0/3853017063 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdf24004500 con 0x7fdf38109d30 2024-04-16T17:05:29.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:29.680+0000 7fdf3fc00700 1 -- 172.21.15.45:0/3853017063 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf381a0350 con 0x7fdf38109d30 2024-04-16T17:05:29.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:29.680+0000 7fdf35600700 1 -- 172.21.15.45:0/3853017063 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf24003de0 con 0x7fdf38109d30 2024-04-16T17:05:29.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:29.680+0000 7fdf3fc00700 1 -- 172.21.15.45:0/3853017063 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf381ac5e0 con 0x7fdf38109d30 2024-04-16T17:05:29.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:29.681+0000 7fdf3fc00700 1 -- 172.21.15.45:0/3853017063 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf3810a860 con 0x7fdf38109d30 2024-04-16T17:05:29.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:29.683+0000 7fdf35600700 1 -- 172.21.15.45:0/3853017063 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fdf24003fc0 con 0x7fdf38109d30 2024-04-16T17:05:29.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:29.685+0000 7fdf35600700 1 --2- 172.21.15.45:0/3853017063 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fdf200a3c40 0x7fdf200a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:29.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:29.685+0000 7fdf35600700 1 -- 172.21.15.45:0/3853017063 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fdf240d08a0 con 0x7fdf38109d30 2024-04-16T17:05:29.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:29.685+0000 7fdf3d200700 1 --2- 172.21.15.45:0/3853017063 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fdf200a3c40 0x7fdf200a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:29.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:29.686+0000 7fdf3d200700 1 --2- 172.21.15.45:0/3853017063 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fdf200a3c40 0x7fdf200a6100 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fdf2c009c10 tx=0x7fdf2c00b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:05:29.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:29.690+0000 7fdf35600700 1 -- 172.21.15.45:0/3853017063 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fdf2400ee60 con 0x7fdf38109d30 2024-04-16T17:05:29.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:05:29 smithi117 ceph-mon[36730]: pgmap v388: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 895 B/s wr, 7 op/s 2024-04-16T17:05:29.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:29.917+0000 7fdf3fc00700 1 -- 172.21.15.45:0/3853017063 --> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fdf381a0c60 con 0x7fdf200a3c40 2024-04-16T17:05:29.947 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:29.946+0000 7fdf35600700 1 -- 172.21.15.45:0/3853017063 <== mgr.14152 v2:172.21.15.45:6800/268030699 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340878 (secure 0 0 0) 0x7fdf381a0c60 con 0x7fdf200a3c40 2024-04-16T17:05:29.955 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-16T17:05:29.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:29.955+0000 7fdf3fc00700 1 -- 172.21.15.45:0/3853017063 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fdf200a3c40 msgr2=0x7fdf200a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:29.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:29.955+0000 7fdf3fc00700 1 --2- 172.21.15.45:0/3853017063 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fdf200a3c40 0x7fdf200a6100 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fdf2c009c10 tx=0x7fdf2c00b040 comp rx=0 tx=0).stop 2024-04-16T17:05:29.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:29.955+0000 7fdf3fc00700 1 -- 172.21.15.45:0/3853017063 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdf38109d30 msgr2=0x7fdf381a5ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:29.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:29.955+0000 7fdf3fc00700 1 --2- 172.21.15.45:0/3853017063 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdf38109d30 0x7fdf381a5ba0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fdf2400e9c0 tx=0x7fdf2400ecd0 comp rx=0 tx=0).stop 2024-04-16T17:05:29.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:29.955+0000 7fdf3fc00700 1 -- 172.21.15.45:0/3853017063 shutdown_connections 2024-04-16T17:05:29.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:29.955+0000 7fdf3fc00700 1 --2- 172.21.15.45:0/3853017063 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fdf200a3c40 0x7fdf200a6100 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:29.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:29.956+0000 7fdf3fc00700 1 --2- 172.21.15.45:0/3853017063 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdf38109380 0x7fdf381a5660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:29.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:29.956+0000 7fdf3fc00700 1 --2- 172.21.15.45:0/3853017063 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdf3810e460 0x7fdf3819f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:29.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:29.956+0000 7fdf3fc00700 1 --2- 172.21.15.45:0/3853017063 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdf38109d30 0x7fdf381a5ba0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:29.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:29.956+0000 7fdf3fc00700 1 -- 172.21.15.45:0/3853017063 >> 172.21.15.45:0/3853017063 conn(0x7fdf380762c0 msgr2=0x7fdf38076880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:05:29.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:29.956+0000 7fdf3fc00700 1 -- 172.21.15.45:0/3853017063 shutdown_connections 2024-04-16T17:05:29.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:29.956+0000 7fdf3fc00700 1 -- 172.21.15.45:0/3853017063 wait complete. 2024-04-16T17:05:29.959 INFO:teuthology.orchestra.run.smithi045.stderr:dumped all 2024-04-16T17:05:30.855 INFO:teuthology.orchestra.run.smithi045.stdout:{"pg_ready":true,"pg_map":{"version":388,"stamp":"2024-04-16T17:05:28.155445+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":658881,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":719,"num_read_kb":484,"num_write":358,"num_write_kb":3065,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":1181469,"num_keys_recovered":20,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":790,"ondisk_log_size":790,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":374244,"kb_used_data":12536,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124567580,"statfs":{"total":1151940427776,"available":1151557201920,"internally_reserved":0,"allocated":12836864,"data_stored":8224157,"data_compressed":433389,"data_compressed_allocated":589824,"data_compressed_original":2678784,"omap_allocated":19143,"internal_metadata":370193721},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":531,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":48,"num_read_kb":23,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.004338"},"pg_stats":[{"pgid":"3.3f","version":"92'8","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.283564+0000","last_change":"2024-04-16T17:03:24.989508+0000","last_active":"2024-04-16T17:05:22.283564+0000","last_peered":"2024-04-16T17:05:22.283564+0000","last_clean":"2024-04-16T17:05:22.283564+0000","last_became_active":"2024-04-16T17:03:24.989178+0000","last_became_peered":"2024-04-16T17:03:24.989178+0000","last_unstale":"2024-04-16T17:05:22.283564+0000","last_undegraded":"2024-04-16T17:05:22.283564+0000","last_fullsized":"2024-04-16T17:05:22.283564+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:18:56.488358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"92'4","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.510490+0000","last_change":"2024-04-16T17:03:24.986378+0000","last_active":"2024-04-16T17:05:22.510490+0000","last_peered":"2024-04-16T17:05:22.510490+0000","last_clean":"2024-04-16T17:05:22.510490+0000","last_became_active":"2024-04-16T17:03:24.986030+0000","last_became_peered":"2024-04-16T17:03:24.986030+0000","last_unstale":"2024-04-16T17:05:22.510490+0000","last_undegraded":"2024-04-16T17:05:22.510490+0000","last_fullsized":"2024-04-16T17:05:22.510490+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:21:14.071109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"92'7","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.509305+0000","last_change":"2024-04-16T17:03:24.976232+0000","last_active":"2024-04-16T17:05:22.509305+0000","last_peered":"2024-04-16T17:05:22.509305+0000","last_clean":"2024-04-16T17:05:22.509305+0000","last_became_active":"2024-04-16T17:03:24.975873+0000","last_became_peered":"2024-04-16T17:03:24.975873+0000","last_unstale":"2024-04-16T17:05:22.509305+0000","last_undegraded":"2024-04-16T17:05:22.509305+0000","last_fullsized":"2024-04-16T17:05:22.509305+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:07:53.523101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"92'8","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.285223+0000","last_change":"2024-04-16T17:04:21.517830+0000","last_active":"2024-04-16T17:05:22.285223+0000","last_peered":"2024-04-16T17:05:22.285223+0000","last_clean":"2024-04-16T17:05:22.285223+0000","last_became_active":"2024-04-16T17:04:21.516749+0000","last_became_peered":"2024-04-16T17:04:21.516749+0000","last_unstale":"2024-04-16T17:05:22.285223+0000","last_undegraded":"2024-04-16T17:05:22.285223+0000","last_fullsized":"2024-04-16T17:05:22.285223+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:02:33.344034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"92'4","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.284780+0000","last_change":"2024-04-16T17:03:24.985609+0000","last_active":"2024-04-16T17:05:22.284780+0000","last_peered":"2024-04-16T17:05:22.284780+0000","last_clean":"2024-04-16T17:05:22.284780+0000","last_became_active":"2024-04-16T17:03:24.985239+0000","last_became_peered":"2024-04-16T17:03:24.985239+0000","last_unstale":"2024-04-16T17:05:22.284780+0000","last_undegraded":"2024-04-16T17:05:22.284780+0000","last_fullsized":"2024-04-16T17:05:22.284780+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:55:58.619210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"92'6","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.282909+0000","last_change":"2024-04-16T17:03:24.966862+0000","last_active":"2024-04-16T17:05:22.282909+0000","last_peered":"2024-04-16T17:05:22.282909+0000","last_clean":"2024-04-16T17:05:22.282909+0000","last_became_active":"2024-04-16T17:03:24.966234+0000","last_became_peered":"2024-04-16T17:03:24.966234+0000","last_unstale":"2024-04-16T17:05:22.282909+0000","last_undegraded":"2024-04-16T17:05:22.282909+0000","last_fullsized":"2024-04-16T17:05:22.282909+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:21:07.018281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"92'5","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.510598+0000","last_change":"2024-04-16T17:03:24.971219+0000","last_active":"2024-04-16T17:05:22.510598+0000","last_peered":"2024-04-16T17:05:22.510598+0000","last_clean":"2024-04-16T17:05:22.510598+0000","last_became_active":"2024-04-16T17:03:24.970876+0000","last_became_peered":"2024-04-16T17:03:24.970876+0000","last_unstale":"2024-04-16T17:05:22.510598+0000","last_undegraded":"2024-04-16T17:05:22.510598+0000","last_fullsized":"2024-04-16T17:05:22.510598+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:44:16.311167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"92'9","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.515605+0000","last_change":"2024-04-16T17:03:24.994694+0000","last_active":"2024-04-16T17:05:22.515605+0000","last_peered":"2024-04-16T17:05:22.515605+0000","last_clean":"2024-04-16T17:05:22.515605+0000","last_became_active":"2024-04-16T17:03:24.994315+0000","last_became_peered":"2024-04-16T17:03:24.994315+0000","last_unstale":"2024-04-16T17:05:22.515605+0000","last_undegraded":"2024-04-16T17:05:22.515605+0000","last_fullsized":"2024-04-16T17:05:22.515605+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:04:59.543816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"92'11","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.512218+0000","last_change":"2024-04-16T17:03:24.964654+0000","last_active":"2024-04-16T17:05:22.512218+0000","last_peered":"2024-04-16T17:05:22.512218+0000","last_clean":"2024-04-16T17:05:22.512218+0000","last_became_active":"2024-04-16T17:03:24.964349+0000","last_became_peered":"2024-04-16T17:03:24.964349+0000","last_unstale":"2024-04-16T17:05:22.512218+0000","last_undegraded":"2024-04-16T17:05:22.512218+0000","last_fullsized":"2024-04-16T17:05:22.512218+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:38:55.880948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"92'16","reported_seq":45,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.283246+0000","last_change":"2024-04-16T17:03:24.966364+0000","last_active":"2024-04-16T17:05:22.283246+0000","last_peered":"2024-04-16T17:05:22.283246+0000","last_clean":"2024-04-16T17:05:22.283246+0000","last_became_active":"2024-04-16T17:03:24.966061+0000","last_became_peered":"2024-04-16T17:03:24.966061+0000","last_unstale":"2024-04-16T17:05:22.283246+0000","last_undegraded":"2024-04-16T17:05:22.283246+0000","last_fullsized":"2024-04-16T17:05:22.283246+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:18:12.642602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"92'6","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.511278+0000","last_change":"2024-04-16T17:03:24.969874+0000","last_active":"2024-04-16T17:05:22.511278+0000","last_peered":"2024-04-16T17:05:22.511278+0000","last_clean":"2024-04-16T17:05:22.511278+0000","last_became_active":"2024-04-16T17:03:24.969523+0000","last_became_peered":"2024-04-16T17:03:24.969523+0000","last_unstale":"2024-04-16T17:05:22.511278+0000","last_undegraded":"2024-04-16T17:05:22.511278+0000","last_fullsized":"2024-04-16T17:05:22.511278+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:17:49.704855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"92'7","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.512621+0000","last_change":"2024-04-16T17:03:24.968660+0000","last_active":"2024-04-16T17:05:22.512621+0000","last_peered":"2024-04-16T17:05:22.512621+0000","last_clean":"2024-04-16T17:05:22.512621+0000","last_became_active":"2024-04-16T17:03:24.968348+0000","last_became_peered":"2024-04-16T17:03:24.968348+0000","last_unstale":"2024-04-16T17:05:22.512621+0000","last_undegraded":"2024-04-16T17:05:22.512621+0000","last_fullsized":"2024-04-16T17:05:22.512621+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:06:06.319391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"92'10","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.289436+0000","last_change":"2024-04-16T17:03:24.974741+0000","last_active":"2024-04-16T17:05:22.289436+0000","last_peered":"2024-04-16T17:05:22.289436+0000","last_clean":"2024-04-16T17:05:22.289436+0000","last_became_active":"2024-04-16T17:03:24.973568+0000","last_became_peered":"2024-04-16T17:03:24.973568+0000","last_unstale":"2024-04-16T17:05:22.289436+0000","last_undegraded":"2024-04-16T17:05:22.289436+0000","last_fullsized":"2024-04-16T17:05:22.289436+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:30:49.325469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"92'7","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.509052+0000","last_change":"2024-04-16T17:03:24.981629+0000","last_active":"2024-04-16T17:05:22.509052+0000","last_peered":"2024-04-16T17:05:22.509052+0000","last_clean":"2024-04-16T17:05:22.509052+0000","last_became_active":"2024-04-16T17:03:24.981111+0000","last_became_peered":"2024-04-16T17:03:24.981111+0000","last_unstale":"2024-04-16T17:05:22.509052+0000","last_undegraded":"2024-04-16T17:05:22.509052+0000","last_fullsized":"2024-04-16T17:05:22.509052+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:41:29.641610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"92'7","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.508949+0000","last_change":"2024-04-16T17:03:24.977890+0000","last_active":"2024-04-16T17:05:22.508949+0000","last_peered":"2024-04-16T17:05:22.508949+0000","last_clean":"2024-04-16T17:05:22.508949+0000","last_became_active":"2024-04-16T17:03:24.976500+0000","last_became_peered":"2024-04-16T17:03:24.976500+0000","last_unstale":"2024-04-16T17:05:22.508949+0000","last_undegraded":"2024-04-16T17:05:22.508949+0000","last_fullsized":"2024-04-16T17:05:22.508949+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:17:51.604701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"92'9","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.509773+0000","last_change":"2024-04-16T17:03:24.976085+0000","last_active":"2024-04-16T17:05:22.509773+0000","last_peered":"2024-04-16T17:05:22.509773+0000","last_clean":"2024-04-16T17:05:22.509773+0000","last_became_active":"2024-04-16T17:03:24.975804+0000","last_became_peered":"2024-04-16T17:03:24.975804+0000","last_unstale":"2024-04-16T17:05:22.509773+0000","last_undegraded":"2024-04-16T17:05:22.509773+0000","last_fullsized":"2024-04-16T17:05:22.509773+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:19:13.706889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"92'8","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.283256+0000","last_change":"2024-04-16T17:03:24.989456+0000","last_active":"2024-04-16T17:05:22.283256+0000","last_peered":"2024-04-16T17:05:22.283256+0000","last_clean":"2024-04-16T17:05:22.283256+0000","last_became_active":"2024-04-16T17:03:24.989161+0000","last_became_peered":"2024-04-16T17:03:24.989161+0000","last_unstale":"2024-04-16T17:05:22.283256+0000","last_undegraded":"2024-04-16T17:05:22.283256+0000","last_fullsized":"2024-04-16T17:05:22.283256+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:52:59.333625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"92'10","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.287957+0000","last_change":"2024-04-16T17:03:24.963428+0000","last_active":"2024-04-16T17:05:22.287957+0000","last_peered":"2024-04-16T17:05:22.287957+0000","last_clean":"2024-04-16T17:05:22.287957+0000","last_became_active":"2024-04-16T17:03:24.962523+0000","last_became_peered":"2024-04-16T17:03:24.962523+0000","last_unstale":"2024-04-16T17:05:22.287957+0000","last_undegraded":"2024-04-16T17:05:22.287957+0000","last_fullsized":"2024-04-16T17:05:22.287957+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:22:19.523901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"92'7","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.509759+0000","last_change":"2024-04-16T17:03:24.981122+0000","last_active":"2024-04-16T17:05:22.509759+0000","last_peered":"2024-04-16T17:05:22.509759+0000","last_clean":"2024-04-16T17:05:22.509759+0000","last_became_active":"2024-04-16T17:03:24.980533+0000","last_became_peered":"2024-04-16T17:03:24.980533+0000","last_unstale":"2024-04-16T17:05:22.509759+0000","last_undegraded":"2024-04-16T17:05:22.509759+0000","last_fullsized":"2024-04-16T17:05:22.509759+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:51:08.580965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.509563+0000","last_change":"2024-04-16T17:03:29.257239+0000","last_active":"2024-04-16T17:05:22.509563+0000","last_peered":"2024-04-16T17:05:22.509563+0000","last_clean":"2024-04-16T17:05:22.509563+0000","last_became_active":"2024-04-16T17:03:29.256827+0000","last_became_peered":"2024-04-16T17:03:29.256827+0000","last_unstale":"2024-04-16T17:05:22.509563+0000","last_undegraded":"2024-04-16T17:05:22.509563+0000","last_fullsized":"2024-04-16T17:05:22.509563+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:57:28.373386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"90'9","reported_seq":57,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.286649+0000","last_change":"2024-04-16T17:04:21.896665+0000","last_active":"2024-04-16T17:05:22.286649+0000","last_peered":"2024-04-16T17:05:22.286649+0000","last_clean":"2024-04-16T17:05:22.286649+0000","last_became_active":"2024-04-16T17:04:21.882573+0000","last_became_peered":"2024-04-16T17:04:21.882573+0000","last_unstale":"2024-04-16T17:05:22.286649+0000","last_undegraded":"2024-04-16T17:05:22.286649+0000","last_fullsized":"2024-04-16T17:05:22.286649+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:37:07.210368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,1],"acting":[9,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"92'5","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.516852+0000","last_change":"2024-04-16T17:03:24.994921+0000","last_active":"2024-04-16T17:05:22.516852+0000","last_peered":"2024-04-16T17:05:22.516852+0000","last_clean":"2024-04-16T17:05:22.516852+0000","last_became_active":"2024-04-16T17:03:24.994412+0000","last_became_peered":"2024-04-16T17:03:24.994412+0000","last_unstale":"2024-04-16T17:05:22.516852+0000","last_undegraded":"2024-04-16T17:05:22.516852+0000","last_fullsized":"2024-04-16T17:05:22.516852+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:11:40.748198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.511978+0000","last_change":"2024-04-16T17:03:29.259884+0000","last_active":"2024-04-16T17:05:22.511978+0000","last_peered":"2024-04-16T17:05:22.511978+0000","last_clean":"2024-04-16T17:05:22.511978+0000","last_became_active":"2024-04-16T17:03:29.259319+0000","last_became_peered":"2024-04-16T17:03:29.259319+0000","last_unstale":"2024-04-16T17:05:22.511978+0000","last_undegraded":"2024-04-16T17:05:22.511978+0000","last_fullsized":"2024-04-16T17:05:22.511978+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:05:18.244682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"90'2","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.519175+0000","last_change":"2024-04-16T17:03:24.179308+0000","last_active":"2024-04-16T17:05:22.519175+0000","last_peered":"2024-04-16T17:05:22.519175+0000","last_clean":"2024-04-16T17:05:22.519175+0000","last_became_active":"2024-04-16T17:03:24.178894+0000","last_became_peered":"2024-04-16T17:03:24.178894+0000","last_unstale":"2024-04-16T17:05:22.519175+0000","last_undegraded":"2024-04-16T17:05:22.519175+0000","last_fullsized":"2024-04-16T17:05:22.519175+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:17:46.831871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"92'6","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.511037+0000","last_change":"2024-04-16T17:03:24.982391+0000","last_active":"2024-04-16T17:05:22.511037+0000","last_peered":"2024-04-16T17:05:22.511037+0000","last_clean":"2024-04-16T17:05:22.511037+0000","last_became_active":"2024-04-16T17:03:24.981761+0000","last_became_peered":"2024-04-16T17:03:24.981761+0000","last_unstale":"2024-04-16T17:05:22.511037+0000","last_undegraded":"2024-04-16T17:05:22.511037+0000","last_fullsized":"2024-04-16T17:05:22.511037+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:40:57.657170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.512324+0000","last_change":"2024-04-16T17:03:29.259705+0000","last_active":"2024-04-16T17:05:22.512324+0000","last_peered":"2024-04-16T17:05:22.512324+0000","last_clean":"2024-04-16T17:05:22.512324+0000","last_became_active":"2024-04-16T17:03:29.259336+0000","last_became_peered":"2024-04-16T17:03:29.259336+0000","last_unstale":"2024-04-16T17:05:22.512324+0000","last_undegraded":"2024-04-16T17:05:22.512324+0000","last_fullsized":"2024-04-16T17:05:22.512324+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:53:29.042933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"90'4","reported_seq":47,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.285245+0000","last_change":"2024-04-16T17:04:22.153413+0000","last_active":"2024-04-16T17:05:22.285245+0000","last_peered":"2024-04-16T17:05:22.285245+0000","last_clean":"2024-04-16T17:05:22.285245+0000","last_became_active":"2024-04-16T17:04:22.148972+0000","last_became_peered":"2024-04-16T17:04:22.148972+0000","last_unstale":"2024-04-16T17:05:22.285245+0000","last_undegraded":"2024-04-16T17:05:22.285245+0000","last_fullsized":"2024-04-16T17:05:22.285245+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:05:57.513052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"92'9","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.510097+0000","last_change":"2024-04-16T17:03:24.973205+0000","last_active":"2024-04-16T17:05:22.510097+0000","last_peered":"2024-04-16T17:05:22.510097+0000","last_clean":"2024-04-16T17:05:22.510097+0000","last_became_active":"2024-04-16T17:03:24.972863+0000","last_became_peered":"2024-04-16T17:03:24.972863+0000","last_unstale":"2024-04-16T17:05:22.510097+0000","last_undegraded":"2024-04-16T17:05:22.510097+0000","last_fullsized":"2024-04-16T17:05:22.510097+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:28:29.005783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.512076+0000","last_change":"2024-04-16T17:03:29.262930+0000","last_active":"2024-04-16T17:05:22.512076+0000","last_peered":"2024-04-16T17:05:22.512076+0000","last_clean":"2024-04-16T17:05:22.512076+0000","last_became_active":"2024-04-16T17:03:29.262621+0000","last_became_peered":"2024-04-16T17:03:29.262621+0000","last_unstale":"2024-04-16T17:05:22.512076+0000","last_undegraded":"2024-04-16T17:05:22.512076+0000","last_fullsized":"2024-04-16T17:05:22.512076+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:22:31.555143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.285074+0000","last_change":"2024-04-16T17:03:24.150503+0000","last_active":"2024-04-16T17:05:22.285074+0000","last_peered":"2024-04-16T17:05:22.285074+0000","last_clean":"2024-04-16T17:05:22.285074+0000","last_became_active":"2024-04-16T17:03:24.150021+0000","last_became_peered":"2024-04-16T17:03:24.150021+0000","last_unstale":"2024-04-16T17:05:22.285074+0000","last_undegraded":"2024-04-16T17:05:22.285074+0000","last_fullsized":"2024-04-16T17:05:22.285074+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:21:36.119497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"92'15","reported_seq":44,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.513017+0000","last_change":"2024-04-16T17:03:24.965049+0000","last_active":"2024-04-16T17:05:22.513017+0000","last_peered":"2024-04-16T17:05:22.513017+0000","last_clean":"2024-04-16T17:05:22.513017+0000","last_became_active":"2024-04-16T17:03:24.963120+0000","last_became_peered":"2024-04-16T17:03:24.963120+0000","last_unstale":"2024-04-16T17:05:22.513017+0000","last_undegraded":"2024-04-16T17:05:22.513017+0000","last_fullsized":"2024-04-16T17:05:22.513017+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:17:49.725113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.509577+0000","last_change":"2024-04-16T17:03:29.247013+0000","last_active":"2024-04-16T17:05:22.509577+0000","last_peered":"2024-04-16T17:05:22.509577+0000","last_clean":"2024-04-16T17:05:22.509577+0000","last_became_active":"2024-04-16T17:03:29.246477+0000","last_became_peered":"2024-04-16T17:03:29.246477+0000","last_unstale":"2024-04-16T17:05:22.509577+0000","last_undegraded":"2024-04-16T17:05:22.509577+0000","last_fullsized":"2024-04-16T17:05:22.509577+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:52:19.525076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"92'7","reported_seq":134,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:23.519161+0000","last_change":"2024-04-16T17:03:24.155333+0000","last_active":"2024-04-16T17:05:23.519161+0000","last_peered":"2024-04-16T17:05:23.519161+0000","last_clean":"2024-04-16T17:05:23.519161+0000","last_became_active":"2024-04-16T17:03:24.154848+0000","last_became_peered":"2024-04-16T17:03:24.154848+0000","last_unstale":"2024-04-16T17:05:23.519161+0000","last_undegraded":"2024-04-16T17:05:23.519161+0000","last_fullsized":"2024-04-16T17:05:23.519161+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:03:34.769573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1154,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"92'7","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.285538+0000","last_change":"2024-04-16T17:03:24.967627+0000","last_active":"2024-04-16T17:05:22.285538+0000","last_peered":"2024-04-16T17:05:22.285538+0000","last_clean":"2024-04-16T17:05:22.285538+0000","last_became_active":"2024-04-16T17:03:24.967264+0000","last_became_peered":"2024-04-16T17:03:24.967264+0000","last_unstale":"2024-04-16T17:05:22.285538+0000","last_undegraded":"2024-04-16T17:05:22.285538+0000","last_fullsized":"2024-04-16T17:05:22.285538+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:35:29.182374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.509364+0000","last_change":"2024-04-16T17:03:29.264625+0000","last_active":"2024-04-16T17:05:22.509364+0000","last_peered":"2024-04-16T17:05:22.509364+0000","last_clean":"2024-04-16T17:05:22.509364+0000","last_became_active":"2024-04-16T17:03:29.264114+0000","last_became_peered":"2024-04-16T17:03:29.264114+0000","last_unstale":"2024-04-16T17:05:22.509364+0000","last_undegraded":"2024-04-16T17:05:22.509364+0000","last_fullsized":"2024-04-16T17:05:22.509364+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:14:49.449980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"90'3","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.282754+0000","last_change":"2024-04-16T17:03:24.160457+0000","last_active":"2024-04-16T17:05:22.282754+0000","last_peered":"2024-04-16T17:05:22.282754+0000","last_clean":"2024-04-16T17:05:22.282754+0000","last_became_active":"2024-04-16T17:03:24.160195+0000","last_became_peered":"2024-04-16T17:03:24.160195+0000","last_unstale":"2024-04-16T17:05:22.282754+0000","last_undegraded":"2024-04-16T17:05:22.282754+0000","last_fullsized":"2024-04-16T17:05:22.282754+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:23:07.064539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"92'8","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.512907+0000","last_change":"2024-04-16T17:03:24.991249+0000","last_active":"2024-04-16T17:05:22.512907+0000","last_peered":"2024-04-16T17:05:22.512907+0000","last_clean":"2024-04-16T17:05:22.512907+0000","last_became_active":"2024-04-16T17:03:24.990927+0000","last_became_peered":"2024-04-16T17:03:24.990927+0000","last_unstale":"2024-04-16T17:05:22.512907+0000","last_undegraded":"2024-04-16T17:05:22.512907+0000","last_fullsized":"2024-04-16T17:05:22.512907+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:36:39.975616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.283442+0000","last_change":"2024-04-16T17:03:29.253704+0000","last_active":"2024-04-16T17:05:22.283442+0000","last_peered":"2024-04-16T17:05:22.283442+0000","last_clean":"2024-04-16T17:05:22.283442+0000","last_became_active":"2024-04-16T17:03:29.253334+0000","last_became_peered":"2024-04-16T17:03:29.253334+0000","last_unstale":"2024-04-16T17:05:22.283442+0000","last_undegraded":"2024-04-16T17:05:22.283442+0000","last_fullsized":"2024-04-16T17:05:22.283442+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:19:12.611731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"90'2","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.288474+0000","last_change":"2024-04-16T17:03:24.158875+0000","last_active":"2024-04-16T17:05:22.288474+0000","last_peered":"2024-04-16T17:05:22.288474+0000","last_clean":"2024-04-16T17:05:22.288474+0000","last_became_active":"2024-04-16T17:03:24.158465+0000","last_became_peered":"2024-04-16T17:03:24.158465+0000","last_unstale":"2024-04-16T17:05:22.288474+0000","last_undegraded":"2024-04-16T17:05:22.288474+0000","last_fullsized":"2024-04-16T17:05:22.288474+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:21:29.882251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"92'8","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.516011+0000","last_change":"2024-04-16T17:03:24.994476+0000","last_active":"2024-04-16T17:05:22.516011+0000","last_peered":"2024-04-16T17:05:22.516011+0000","last_clean":"2024-04-16T17:05:22.516011+0000","last_became_active":"2024-04-16T17:03:24.993532+0000","last_became_peered":"2024-04-16T17:03:24.993532+0000","last_unstale":"2024-04-16T17:05:22.516011+0000","last_undegraded":"2024-04-16T17:05:22.516011+0000","last_fullsized":"2024-04-16T17:05:22.516011+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:04:44.062444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.510289+0000","last_change":"2024-04-16T17:03:29.255446+0000","last_active":"2024-04-16T17:05:22.510289+0000","last_peered":"2024-04-16T17:05:22.510289+0000","last_clean":"2024-04-16T17:05:22.510289+0000","last_became_active":"2024-04-16T17:03:29.255051+0000","last_became_peered":"2024-04-16T17:03:29.255051+0000","last_unstale":"2024-04-16T17:05:22.510289+0000","last_undegraded":"2024-04-16T17:05:22.510289+0000","last_fullsized":"2024-04-16T17:05:22.510289+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:24:04.946342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"90'1","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.512386+0000","last_change":"2024-04-16T17:03:24.154904+0000","last_active":"2024-04-16T17:05:22.512386+0000","last_peered":"2024-04-16T17:05:22.512386+0000","last_clean":"2024-04-16T17:05:22.512386+0000","last_became_active":"2024-04-16T17:03:24.154617+0000","last_became_peered":"2024-04-16T17:03:24.154617+0000","last_unstale":"2024-04-16T17:05:22.512386+0000","last_undegraded":"2024-04-16T17:05:22.512386+0000","last_fullsized":"2024-04-16T17:05:22.512386+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:35:55.498855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"92'11","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.284812+0000","last_change":"2024-04-16T17:03:24.979908+0000","last_active":"2024-04-16T17:05:22.284812+0000","last_peered":"2024-04-16T17:05:22.284812+0000","last_clean":"2024-04-16T17:05:22.284812+0000","last_became_active":"2024-04-16T17:03:24.979501+0000","last_became_peered":"2024-04-16T17:03:24.979501+0000","last_unstale":"2024-04-16T17:05:22.284812+0000","last_undegraded":"2024-04-16T17:05:22.284812+0000","last_fullsized":"2024-04-16T17:05:22.284812+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:51:05.421459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.516308+0000","last_change":"2024-04-16T17:03:29.258809+0000","last_active":"2024-04-16T17:05:22.516308+0000","last_peered":"2024-04-16T17:05:22.516308+0000","last_clean":"2024-04-16T17:05:22.516308+0000","last_became_active":"2024-04-16T17:03:29.256629+0000","last_became_peered":"2024-04-16T17:03:29.256629+0000","last_unstale":"2024-04-16T17:05:22.516308+0000","last_undegraded":"2024-04-16T17:05:22.516308+0000","last_fullsized":"2024-04-16T17:05:22.516308+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:46:51.634944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"92'4","reported_seq":126,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.517334+0000","last_change":"2024-04-16T17:03:24.153774+0000","last_active":"2024-04-16T17:05:22.517334+0000","last_peered":"2024-04-16T17:05:22.517334+0000","last_clean":"2024-04-16T17:05:22.517334+0000","last_became_active":"2024-04-16T17:03:24.153418+0000","last_became_peered":"2024-04-16T17:03:24.153418+0000","last_unstale":"2024-04-16T17:05:22.517334+0000","last_undegraded":"2024-04-16T17:05:22.517334+0000","last_fullsized":"2024-04-16T17:05:22.517334+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:58:36.093789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":91,"num_read_kb":91,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"92'10","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.511791+0000","last_change":"2024-04-16T17:03:24.962838+0000","last_active":"2024-04-16T17:05:22.511791+0000","last_peered":"2024-04-16T17:05:22.511791+0000","last_clean":"2024-04-16T17:05:22.511791+0000","last_became_active":"2024-04-16T17:03:24.960163+0000","last_became_peered":"2024-04-16T17:03:24.960163+0000","last_unstale":"2024-04-16T17:05:22.511791+0000","last_undegraded":"2024-04-16T17:05:22.511791+0000","last_fullsized":"2024-04-16T17:05:22.511791+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:48:18.723499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.284550+0000","last_change":"2024-04-16T17:03:29.251177+0000","last_active":"2024-04-16T17:05:22.284550+0000","last_peered":"2024-04-16T17:05:22.284550+0000","last_clean":"2024-04-16T17:05:22.284550+0000","last_became_active":"2024-04-16T17:03:29.250657+0000","last_became_peered":"2024-04-16T17:03:29.250657+0000","last_unstale":"2024-04-16T17:05:22.284550+0000","last_undegraded":"2024-04-16T17:05:22.284550+0000","last_fullsized":"2024-04-16T17:05:22.284550+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:16:47.217193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"90'2","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.510573+0000","last_change":"2024-04-16T17:03:24.154245+0000","last_active":"2024-04-16T17:05:22.510573+0000","last_peered":"2024-04-16T17:05:22.510573+0000","last_clean":"2024-04-16T17:05:22.510573+0000","last_became_active":"2024-04-16T17:03:24.153297+0000","last_became_peered":"2024-04-16T17:03:24.153297+0000","last_unstale":"2024-04-16T17:05:22.510573+0000","last_undegraded":"2024-04-16T17:05:22.510573+0000","last_fullsized":"2024-04-16T17:05:22.510573+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:23:49.490238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"92'9","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.511056+0000","last_change":"2024-04-16T17:03:24.984572+0000","last_active":"2024-04-16T17:05:22.511056+0000","last_peered":"2024-04-16T17:05:22.511056+0000","last_clean":"2024-04-16T17:05:22.511056+0000","last_became_active":"2024-04-16T17:03:24.983588+0000","last_became_peered":"2024-04-16T17:03:24.983588+0000","last_unstale":"2024-04-16T17:05:22.511056+0000","last_undegraded":"2024-04-16T17:05:22.511056+0000","last_fullsized":"2024-04-16T17:05:22.511056+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:41:24.704602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.512602+0000","last_change":"2024-04-16T17:03:29.245129+0000","last_active":"2024-04-16T17:05:22.512602+0000","last_peered":"2024-04-16T17:05:22.512602+0000","last_clean":"2024-04-16T17:05:22.512602+0000","last_became_active":"2024-04-16T17:03:29.244720+0000","last_became_peered":"2024-04-16T17:03:29.244720+0000","last_unstale":"2024-04-16T17:05:22.512602+0000","last_undegraded":"2024-04-16T17:05:22.512602+0000","last_fullsized":"2024-04-16T17:05:22.512602+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:55:26.612176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"90'4","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.512809+0000","last_change":"2024-04-16T17:03:24.165248+0000","last_active":"2024-04-16T17:05:22.512809+0000","last_peered":"2024-04-16T17:05:22.512809+0000","last_clean":"2024-04-16T17:05:22.512809+0000","last_became_active":"2024-04-16T17:03:24.164915+0000","last_became_peered":"2024-04-16T17:03:24.164915+0000","last_unstale":"2024-04-16T17:05:22.512809+0000","last_undegraded":"2024-04-16T17:05:22.512809+0000","last_fullsized":"2024-04-16T17:05:22.512809+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:04:47.207554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"92'11","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.510492+0000","last_change":"2024-04-16T17:03:24.967292+0000","last_active":"2024-04-16T17:05:22.510492+0000","last_peered":"2024-04-16T17:05:22.510492+0000","last_clean":"2024-04-16T17:05:22.510492+0000","last_became_active":"2024-04-16T17:03:24.966952+0000","last_became_peered":"2024-04-16T17:03:24.966952+0000","last_unstale":"2024-04-16T17:05:22.510492+0000","last_undegraded":"2024-04-16T17:05:22.510492+0000","last_fullsized":"2024-04-16T17:05:22.510492+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:42:54.969478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.285084+0000","last_change":"2024-04-16T17:03:29.257836+0000","last_active":"2024-04-16T17:05:22.285084+0000","last_peered":"2024-04-16T17:05:22.285084+0000","last_clean":"2024-04-16T17:05:22.285084+0000","last_became_active":"2024-04-16T17:03:29.257409+0000","last_became_peered":"2024-04-16T17:03:29.257409+0000","last_unstale":"2024-04-16T17:05:22.285084+0000","last_undegraded":"2024-04-16T17:05:22.285084+0000","last_fullsized":"2024-04-16T17:05:22.285084+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:25:21.288266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"92'36","reported_seq":259,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:23.518441+0000","last_change":"2024-04-16T17:03:24.150943+0000","last_active":"2024-04-16T17:05:23.518441+0000","last_peered":"2024-04-16T17:05:23.518441+0000","last_clean":"2024-04-16T17:05:23.518441+0000","last_became_active":"2024-04-16T17:03:24.150563+0000","last_became_peered":"2024-04-16T17:03:24.150563+0000","last_unstale":"2024-04-16T17:05:23.518441+0000","last_undegraded":"2024-04-16T17:05:23.518441+0000","last_fullsized":"2024-04-16T17:05:23.518441+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:05:07.380440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":63605,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":192,"num_read_kb":159,"num_write":37,"num_write_kb":80,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"92'6","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.285005+0000","last_change":"2024-04-16T17:03:24.966393+0000","last_active":"2024-04-16T17:05:22.285005+0000","last_peered":"2024-04-16T17:05:22.285005+0000","last_clean":"2024-04-16T17:05:22.285005+0000","last_became_active":"2024-04-16T17:03:24.966016+0000","last_became_peered":"2024-04-16T17:03:24.966016+0000","last_unstale":"2024-04-16T17:05:22.285005+0000","last_undegraded":"2024-04-16T17:05:22.285005+0000","last_fullsized":"2024-04-16T17:05:22.285005+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:30:25.203262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.513291+0000","last_change":"2024-04-16T17:03:29.265371+0000","last_active":"2024-04-16T17:05:22.513291+0000","last_peered":"2024-04-16T17:05:22.513291+0000","last_clean":"2024-04-16T17:05:22.513291+0000","last_became_active":"2024-04-16T17:03:29.264933+0000","last_became_peered":"2024-04-16T17:03:29.264933+0000","last_unstale":"2024-04-16T17:05:22.513291+0000","last_undegraded":"2024-04-16T17:05:22.513291+0000","last_fullsized":"2024-04-16T17:05:22.513291+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:50:18.203015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"90'6","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.282793+0000","last_change":"2024-04-16T17:04:21.526796+0000","last_active":"2024-04-16T17:05:22.282793+0000","last_peered":"2024-04-16T17:05:22.282793+0000","last_clean":"2024-04-16T17:05:22.282793+0000","last_became_active":"2024-04-16T17:04:21.508364+0000","last_became_peered":"2024-04-16T17:04:21.508364+0000","last_unstale":"2024-04-16T17:05:22.282793+0000","last_undegraded":"2024-04-16T17:05:22.282793+0000","last_fullsized":"2024-04-16T17:05:22.282793+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:45:42.726648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"92'9","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.289036+0000","last_change":"2024-04-16T17:03:24.986228+0000","last_active":"2024-04-16T17:05:22.289036+0000","last_peered":"2024-04-16T17:05:22.289036+0000","last_clean":"2024-04-16T17:05:22.289036+0000","last_became_active":"2024-04-16T17:03:24.985916+0000","last_became_peered":"2024-04-16T17:03:24.985916+0000","last_unstale":"2024-04-16T17:05:22.289036+0000","last_undegraded":"2024-04-16T17:05:22.289036+0000","last_fullsized":"2024-04-16T17:05:22.289036+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:23:54.309203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.282938+0000","last_change":"2024-04-16T17:03:29.264636+0000","last_active":"2024-04-16T17:05:22.282938+0000","last_peered":"2024-04-16T17:05:22.282938+0000","last_clean":"2024-04-16T17:05:22.282938+0000","last_became_active":"2024-04-16T17:03:29.264117+0000","last_became_peered":"2024-04-16T17:03:29.264117+0000","last_unstale":"2024-04-16T17:05:22.282938+0000","last_undegraded":"2024-04-16T17:05:22.282938+0000","last_fullsized":"2024-04-16T17:05:22.282938+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:42:36.123154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"90'3","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.512886+0000","last_change":"2024-04-16T17:03:24.166844+0000","last_active":"2024-04-16T17:05:22.512886+0000","last_peered":"2024-04-16T17:05:22.512886+0000","last_clean":"2024-04-16T17:05:22.512886+0000","last_became_active":"2024-04-16T17:03:24.166479+0000","last_became_peered":"2024-04-16T17:03:24.166479+0000","last_unstale":"2024-04-16T17:05:22.512886+0000","last_undegraded":"2024-04-16T17:05:22.512886+0000","last_fullsized":"2024-04-16T17:05:22.512886+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:02:20.139865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"92'15","reported_seq":44,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.510958+0000","last_change":"2024-04-16T17:03:24.973057+0000","last_active":"2024-04-16T17:05:22.510958+0000","last_peered":"2024-04-16T17:05:22.510958+0000","last_clean":"2024-04-16T17:05:22.510958+0000","last_became_active":"2024-04-16T17:03:24.972731+0000","last_became_peered":"2024-04-16T17:03:24.972731+0000","last_unstale":"2024-04-16T17:05:22.510958+0000","last_undegraded":"2024-04-16T17:05:22.510958+0000","last_fullsized":"2024-04-16T17:05:22.510958+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:37:38.702919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.286263+0000","last_change":"2024-04-16T17:03:29.259167+0000","last_active":"2024-04-16T17:05:22.286263+0000","last_peered":"2024-04-16T17:05:22.286263+0000","last_clean":"2024-04-16T17:05:22.286263+0000","last_became_active":"2024-04-16T17:03:29.258737+0000","last_became_peered":"2024-04-16T17:03:29.258737+0000","last_unstale":"2024-04-16T17:05:22.286263+0000","last_undegraded":"2024-04-16T17:05:22.286263+0000","last_fullsized":"2024-04-16T17:05:22.286263+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:01:13.774037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"92'5","reported_seq":133,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:24.600954+0000","last_change":"2024-04-16T17:03:24.151064+0000","last_active":"2024-04-16T17:05:24.600954+0000","last_peered":"2024-04-16T17:05:24.600954+0000","last_clean":"2024-04-16T17:05:24.600954+0000","last_became_active":"2024-04-16T17:03:24.150745+0000","last_became_peered":"2024-04-16T17:03:24.150745+0000","last_unstale":"2024-04-16T17:05:24.600954+0000","last_undegraded":"2024-04-16T17:05:24.600954+0000","last_fullsized":"2024-04-16T17:05:24.600954+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:32:26.271008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":98,"num_read_kb":5,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"92'6","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.510687+0000","last_change":"2024-04-16T17:03:24.971948+0000","last_active":"2024-04-16T17:05:22.510687+0000","last_peered":"2024-04-16T17:05:22.510687+0000","last_clean":"2024-04-16T17:05:22.510687+0000","last_became_active":"2024-04-16T17:03:24.971530+0000","last_became_peered":"2024-04-16T17:03:24.971530+0000","last_unstale":"2024-04-16T17:05:22.510687+0000","last_undegraded":"2024-04-16T17:05:22.510687+0000","last_fullsized":"2024-04-16T17:05:22.510687+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:03:29.286160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.290269+0000","last_change":"2024-04-16T17:03:29.264821+0000","last_active":"2024-04-16T17:05:22.290269+0000","last_peered":"2024-04-16T17:05:22.290269+0000","last_clean":"2024-04-16T17:05:22.290269+0000","last_became_active":"2024-04-16T17:03:29.264287+0000","last_became_peered":"2024-04-16T17:03:29.264287+0000","last_unstale":"2024-04-16T17:05:22.290269+0000","last_undegraded":"2024-04-16T17:05:22.290269+0000","last_fullsized":"2024-04-16T17:05:22.290269+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:33:28.400663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"90'15","reported_seq":47,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.284636+0000","last_change":"2024-04-16T17:03:24.158844+0000","last_active":"2024-04-16T17:05:22.284636+0000","last_peered":"2024-04-16T17:05:22.284636+0000","last_clean":"2024-04-16T17:05:22.284636+0000","last_became_active":"2024-04-16T17:03:24.158537+0000","last_became_peered":"2024-04-16T17:03:24.158537+0000","last_unstale":"2024-04-16T17:05:22.284636+0000","last_undegraded":"2024-04-16T17:05:22.284636+0000","last_fullsized":"2024-04-16T17:05:22.284636+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:42:22.772452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"92'6","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.512660+0000","last_change":"2024-04-16T17:03:24.962522+0000","last_active":"2024-04-16T17:05:22.512660+0000","last_peered":"2024-04-16T17:05:22.512660+0000","last_clean":"2024-04-16T17:05:22.512660+0000","last_became_active":"2024-04-16T17:03:24.962169+0000","last_became_peered":"2024-04-16T17:03:24.962169+0000","last_unstale":"2024-04-16T17:05:22.512660+0000","last_undegraded":"2024-04-16T17:05:22.512660+0000","last_fullsized":"2024-04-16T17:05:22.512660+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:28:17.147491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.512624+0000","last_change":"2024-04-16T17:03:29.259539+0000","last_active":"2024-04-16T17:05:22.512624+0000","last_peered":"2024-04-16T17:05:22.512624+0000","last_clean":"2024-04-16T17:05:22.512624+0000","last_became_active":"2024-04-16T17:03:29.259123+0000","last_became_peered":"2024-04-16T17:03:29.259123+0000","last_unstale":"2024-04-16T17:05:22.512624+0000","last_undegraded":"2024-04-16T17:05:22.512624+0000","last_fullsized":"2024-04-16T17:05:22.512624+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:15:17.142636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"90'2","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.285245+0000","last_change":"2024-04-16T17:03:24.147526+0000","last_active":"2024-04-16T17:05:22.285245+0000","last_peered":"2024-04-16T17:05:22.285245+0000","last_clean":"2024-04-16T17:05:22.285245+0000","last_became_active":"2024-04-16T17:03:24.147119+0000","last_became_peered":"2024-04-16T17:03:24.147119+0000","last_unstale":"2024-04-16T17:05:22.285245+0000","last_undegraded":"2024-04-16T17:05:22.285245+0000","last_fullsized":"2024-04-16T17:05:22.285245+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:30:38.742122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"92'5","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.283051+0000","last_change":"2024-04-16T17:04:21.507113+0000","last_active":"2024-04-16T17:05:22.283051+0000","last_peered":"2024-04-16T17:05:22.283051+0000","last_clean":"2024-04-16T17:05:22.283051+0000","last_became_active":"2024-04-16T17:04:21.506680+0000","last_became_peered":"2024-04-16T17:04:21.506680+0000","last_unstale":"2024-04-16T17:05:22.283051+0000","last_undegraded":"2024-04-16T17:05:22.283051+0000","last_fullsized":"2024-04-16T17:05:22.283051+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:08:47.971317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,8],"acting":[11,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.511966+0000","last_change":"2024-04-16T17:05:22.511966+0000","last_active":"2024-04-16T17:05:22.511966+0000","last_peered":"2024-04-16T17:05:22.511966+0000","last_clean":"2024-04-16T17:05:22.511966+0000","last_became_active":"2024-04-16T17:05:22.511512+0000","last_became_peered":"2024-04-16T17:05:22.511512+0000","last_unstale":"2024-04-16T17:05:22.511966+0000","last_undegraded":"2024-04-16T17:05:22.511966+0000","last_fullsized":"2024-04-16T17:05:22.511966+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:14:41.291499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,1],"acting":[8,10,11,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.285619+0000","last_change":"2024-04-16T17:03:24.162105+0000","last_active":"2024-04-16T17:05:22.285619+0000","last_peered":"2024-04-16T17:05:22.285619+0000","last_clean":"2024-04-16T17:05:22.285619+0000","last_became_active":"2024-04-16T17:03:24.161631+0000","last_became_peered":"2024-04-16T17:03:24.161631+0000","last_unstale":"2024-04-16T17:05:22.285619+0000","last_undegraded":"2024-04-16T17:05:22.285619+0000","last_fullsized":"2024-04-16T17:05:22.285619+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:22:33.677848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"92'4","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.283269+0000","last_change":"2024-04-16T17:03:24.978253+0000","last_active":"2024-04-16T17:05:22.283269+0000","last_peered":"2024-04-16T17:05:22.283269+0000","last_clean":"2024-04-16T17:05:22.283269+0000","last_became_active":"2024-04-16T17:03:24.977956+0000","last_became_peered":"2024-04-16T17:03:24.977956+0000","last_unstale":"2024-04-16T17:05:22.283269+0000","last_undegraded":"2024-04-16T17:05:22.283269+0000","last_fullsized":"2024-04-16T17:05:22.283269+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:59:45.102383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.289595+0000","last_change":"2024-04-16T17:03:29.265312+0000","last_active":"2024-04-16T17:05:22.289595+0000","last_peered":"2024-04-16T17:05:22.289595+0000","last_clean":"2024-04-16T17:05:22.289595+0000","last_became_active":"2024-04-16T17:03:29.264891+0000","last_became_peered":"2024-04-16T17:03:29.264891+0000","last_unstale":"2024-04-16T17:05:22.289595+0000","last_undegraded":"2024-04-16T17:05:22.289595+0000","last_fullsized":"2024-04-16T17:05:22.289595+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:26:40.088127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"90'1","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.510918+0000","last_change":"2024-04-16T17:03:24.170935+0000","last_active":"2024-04-16T17:05:22.510918+0000","last_peered":"2024-04-16T17:05:22.510918+0000","last_clean":"2024-04-16T17:05:22.510918+0000","last_became_active":"2024-04-16T17:03:24.170609+0000","last_became_peered":"2024-04-16T17:03:24.170609+0000","last_unstale":"2024-04-16T17:05:22.510918+0000","last_undegraded":"2024-04-16T17:05:22.510918+0000","last_fullsized":"2024-04-16T17:05:22.510918+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:18:33.516309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"92'7","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.283137+0000","last_change":"2024-04-16T17:03:24.985434+0000","last_active":"2024-04-16T17:05:22.283137+0000","last_peered":"2024-04-16T17:05:22.283137+0000","last_clean":"2024-04-16T17:05:22.283137+0000","last_became_active":"2024-04-16T17:03:24.984388+0000","last_became_peered":"2024-04-16T17:03:24.984388+0000","last_unstale":"2024-04-16T17:05:22.283137+0000","last_undegraded":"2024-04-16T17:05:22.283137+0000","last_fullsized":"2024-04-16T17:05:22.283137+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:43:26.355572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.516267+0000","last_change":"2024-04-16T17:05:22.516267+0000","last_active":"2024-04-16T17:05:22.516267+0000","last_peered":"2024-04-16T17:05:22.516267+0000","last_clean":"2024-04-16T17:05:22.516267+0000","last_became_active":"2024-04-16T17:05:22.513637+0000","last_became_peered":"2024-04-16T17:05:22.513637+0000","last_unstale":"2024-04-16T17:05:22.516267+0000","last_undegraded":"2024-04-16T17:05:22.516267+0000","last_fullsized":"2024-04-16T17:05:22.516267+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:39:47.975735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,8,10,0],"acting":[6,8,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"92'8","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.512569+0000","last_change":"2024-04-16T17:03:24.962786+0000","last_active":"2024-04-16T17:05:22.512569+0000","last_peered":"2024-04-16T17:05:22.512569+0000","last_clean":"2024-04-16T17:05:22.512569+0000","last_became_active":"2024-04-16T17:03:24.961704+0000","last_became_peered":"2024-04-16T17:03:24.961704+0000","last_unstale":"2024-04-16T17:05:22.512569+0000","last_undegraded":"2024-04-16T17:05:22.512569+0000","last_fullsized":"2024-04-16T17:05:22.512569+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:22:27.957607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"92'5","reported_seq":129,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:24.600122+0000","last_change":"2024-04-16T17:03:24.175617+0000","last_active":"2024-04-16T17:05:24.600122+0000","last_peered":"2024-04-16T17:05:24.600122+0000","last_clean":"2024-04-16T17:05:24.600122+0000","last_became_active":"2024-04-16T17:03:24.175272+0000","last_became_peered":"2024-04-16T17:03:24.175272+0000","last_unstale":"2024-04-16T17:05:24.600122+0000","last_undegraded":"2024-04-16T17:05:24.600122+0000","last_fullsized":"2024-04-16T17:05:24.600122+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:27:44.738030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":93,"num_read_kb":93,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"90'1","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.511030+0000","last_change":"2024-04-16T17:04:21.512033+0000","last_active":"2024-04-16T17:05:22.511030+0000","last_peered":"2024-04-16T17:05:22.511030+0000","last_clean":"2024-04-16T17:05:22.511030+0000","last_became_active":"2024-04-16T17:04:21.506718+0000","last_became_peered":"2024-04-16T17:04:21.506718+0000","last_unstale":"2024-04-16T17:05:22.511030+0000","last_undegraded":"2024-04-16T17:05:22.511030+0000","last_fullsized":"2024-04-16T17:05:22.511030+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:53:48.953887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,10],"acting":[7,5,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"92'8","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.517153+0000","last_change":"2024-04-16T17:03:24.983629+0000","last_active":"2024-04-16T17:05:22.517153+0000","last_peered":"2024-04-16T17:05:22.517153+0000","last_clean":"2024-04-16T17:05:22.517153+0000","last_became_active":"2024-04-16T17:03:24.979969+0000","last_became_peered":"2024-04-16T17:03:24.979969+0000","last_unstale":"2024-04-16T17:05:22.517153+0000","last_undegraded":"2024-04-16T17:05:22.517153+0000","last_fullsized":"2024-04-16T17:05:22.517153+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:01:24.391732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"90'3","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.512515+0000","last_change":"2024-04-16T17:03:24.152475+0000","last_active":"2024-04-16T17:05:22.512515+0000","last_peered":"2024-04-16T17:05:22.512515+0000","last_clean":"2024-04-16T17:05:22.512515+0000","last_became_active":"2024-04-16T17:03:24.152198+0000","last_became_peered":"2024-04-16T17:03:24.152198+0000","last_unstale":"2024-04-16T17:05:22.512515+0000","last_undegraded":"2024-04-16T17:05:22.512515+0000","last_fullsized":"2024-04-16T17:05:22.512515+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:49:33.914325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":364,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.510449+0000","last_change":"2024-04-16T17:02:02.111335+0000","last_active":"2024-04-16T17:05:22.510449+0000","last_peered":"2024-04-16T17:05:22.510449+0000","last_clean":"2024-04-16T17:05:22.510449+0000","last_became_active":"2024-04-16T17:02:02.110928+0000","last_became_peered":"2024-04-16T17:02:02.110928+0000","last_unstale":"2024-04-16T17:05:22.510449+0000","last_undegraded":"2024-04-16T17:05:22.510449+0000","last_fullsized":"2024-04-16T17:05:22.510449+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T16:58:07.081753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T16:58:07.081753+0000","last_clean_scrub_stamp":"2024-04-16T16:58:07.081753+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:27:02.865664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.513186+0000","last_change":"2024-04-16T17:03:29.301691+0000","last_active":"2024-04-16T17:05:22.513186+0000","last_peered":"2024-04-16T17:05:22.513186+0000","last_clean":"2024-04-16T17:05:22.513186+0000","last_became_active":"2024-04-16T17:03:29.301277+0000","last_became_peered":"2024-04-16T17:03:29.301277+0000","last_unstale":"2024-04-16T17:05:22.513186+0000","last_undegraded":"2024-04-16T17:05:22.513186+0000","last_fullsized":"2024-04-16T17:05:22.513186+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:45:16.727610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"92'5","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.287823+0000","last_change":"2024-04-16T17:03:24.970939+0000","last_active":"2024-04-16T17:05:22.287823+0000","last_peered":"2024-04-16T17:05:22.287823+0000","last_clean":"2024-04-16T17:05:22.287823+0000","last_became_active":"2024-04-16T17:03:24.970581+0000","last_became_peered":"2024-04-16T17:03:24.970581+0000","last_unstale":"2024-04-16T17:05:22.287823+0000","last_undegraded":"2024-04-16T17:05:22.287823+0000","last_fullsized":"2024-04-16T17:05:22.287823+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:33:07.448597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"90'3","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.510631+0000","last_change":"2024-04-16T17:03:24.155454+0000","last_active":"2024-04-16T17:05:22.510631+0000","last_peered":"2024-04-16T17:05:22.510631+0000","last_clean":"2024-04-16T17:05:22.510631+0000","last_became_active":"2024-04-16T17:03:24.155155+0000","last_became_peered":"2024-04-16T17:03:24.155155+0000","last_unstale":"2024-04-16T17:05:22.510631+0000","last_undegraded":"2024-04-16T17:05:22.510631+0000","last_fullsized":"2024-04-16T17:05:22.510631+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:34:08.618002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.282893+0000","last_change":"2024-04-16T17:03:29.255538+0000","last_active":"2024-04-16T17:05:22.282893+0000","last_peered":"2024-04-16T17:05:22.282893+0000","last_clean":"2024-04-16T17:05:22.282893+0000","last_became_active":"2024-04-16T17:03:29.255107+0000","last_became_peered":"2024-04-16T17:03:29.255107+0000","last_unstale":"2024-04-16T17:05:22.282893+0000","last_undegraded":"2024-04-16T17:05:22.282893+0000","last_fullsized":"2024-04-16T17:05:22.282893+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:31:58.458641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"90'1","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.285126+0000","last_change":"2024-04-16T17:03:24.158816+0000","last_active":"2024-04-16T17:05:22.285126+0000","last_peered":"2024-04-16T17:05:22.285126+0000","last_clean":"2024-04-16T17:05:22.285126+0000","last_became_active":"2024-04-16T17:03:24.158518+0000","last_became_peered":"2024-04-16T17:03:24.158518+0000","last_unstale":"2024-04-16T17:05:22.285126+0000","last_undegraded":"2024-04-16T17:05:22.285126+0000","last_fullsized":"2024-04-16T17:05:22.285126+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:54:55.807975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"92'9","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.289939+0000","last_change":"2024-04-16T17:03:24.977190+0000","last_active":"2024-04-16T17:05:22.289939+0000","last_peered":"2024-04-16T17:05:22.289939+0000","last_clean":"2024-04-16T17:05:22.289939+0000","last_became_active":"2024-04-16T17:03:24.976840+0000","last_became_peered":"2024-04-16T17:03:24.976840+0000","last_unstale":"2024-04-16T17:05:22.289939+0000","last_undegraded":"2024-04-16T17:05:22.289939+0000","last_fullsized":"2024-04-16T17:05:22.289939+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:21:51.294150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.515949+0000","last_change":"2024-04-16T17:03:29.301423+0000","last_active":"2024-04-16T17:05:22.515949+0000","last_peered":"2024-04-16T17:05:22.515949+0000","last_clean":"2024-04-16T17:05:22.515949+0000","last_became_active":"2024-04-16T17:03:29.300888+0000","last_became_peered":"2024-04-16T17:03:29.300888+0000","last_unstale":"2024-04-16T17:05:22.515949+0000","last_undegraded":"2024-04-16T17:05:22.515949+0000","last_fullsized":"2024-04-16T17:05:22.515949+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:13:54.188090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"92'11","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.287648+0000","last_change":"2024-04-16T17:03:24.167643+0000","last_active":"2024-04-16T17:05:22.287648+0000","last_peered":"2024-04-16T17:05:22.287648+0000","last_clean":"2024-04-16T17:05:22.287648+0000","last_became_active":"2024-04-16T17:03:24.167103+0000","last_became_peered":"2024-04-16T17:03:24.167103+0000","last_unstale":"2024-04-16T17:05:22.287648+0000","last_undegraded":"2024-04-16T17:05:22.287648+0000","last_fullsized":"2024-04-16T17:05:22.287648+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:33:16.809371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"92'5","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.519071+0000","last_change":"2024-04-16T17:03:24.976981+0000","last_active":"2024-04-16T17:05:22.519071+0000","last_peered":"2024-04-16T17:05:22.519071+0000","last_clean":"2024-04-16T17:05:22.519071+0000","last_became_active":"2024-04-16T17:03:24.975385+0000","last_became_peered":"2024-04-16T17:03:24.975385+0000","last_unstale":"2024-04-16T17:05:22.519071+0000","last_undegraded":"2024-04-16T17:05:22.519071+0000","last_fullsized":"2024-04-16T17:05:22.519071+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:27:03.104838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.509234+0000","last_change":"2024-04-16T17:03:29.244767+0000","last_active":"2024-04-16T17:05:22.509234+0000","last_peered":"2024-04-16T17:05:22.509234+0000","last_clean":"2024-04-16T17:05:22.509234+0000","last_became_active":"2024-04-16T17:03:29.244205+0000","last_became_peered":"2024-04-16T17:03:29.244205+0000","last_unstale":"2024-04-16T17:05:22.509234+0000","last_undegraded":"2024-04-16T17:05:22.509234+0000","last_fullsized":"2024-04-16T17:05:22.509234+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:51:46.652387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"90'4","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.513012+0000","last_change":"2024-04-16T17:03:24.164916+0000","last_active":"2024-04-16T17:05:22.513012+0000","last_peered":"2024-04-16T17:05:22.513012+0000","last_clean":"2024-04-16T17:05:22.513012+0000","last_became_active":"2024-04-16T17:03:24.164648+0000","last_became_peered":"2024-04-16T17:03:24.164648+0000","last_unstale":"2024-04-16T17:05:22.513012+0000","last_undegraded":"2024-04-16T17:05:22.513012+0000","last_fullsized":"2024-04-16T17:05:22.513012+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:23:00.198785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"92'7","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.510688+0000","last_change":"2024-04-16T17:03:24.994393+0000","last_active":"2024-04-16T17:05:22.510688+0000","last_peered":"2024-04-16T17:05:22.510688+0000","last_clean":"2024-04-16T17:05:22.510688+0000","last_became_active":"2024-04-16T17:03:24.994002+0000","last_became_peered":"2024-04-16T17:03:24.994002+0000","last_unstale":"2024-04-16T17:05:22.510688+0000","last_undegraded":"2024-04-16T17:05:22.510688+0000","last_fullsized":"2024-04-16T17:05:22.510688+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:23:44.596781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.512808+0000","last_change":"2024-04-16T17:03:29.302009+0000","last_active":"2024-04-16T17:05:22.512808+0000","last_peered":"2024-04-16T17:05:22.512808+0000","last_clean":"2024-04-16T17:05:22.512808+0000","last_became_active":"2024-04-16T17:03:29.301183+0000","last_became_peered":"2024-04-16T17:03:29.301183+0000","last_unstale":"2024-04-16T17:05:22.512808+0000","last_undegraded":"2024-04-16T17:05:22.512808+0000","last_fullsized":"2024-04-16T17:05:22.512808+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:30:19.561875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"90'2","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.509857+0000","last_change":"2024-04-16T17:03:24.176829+0000","last_active":"2024-04-16T17:05:22.509857+0000","last_peered":"2024-04-16T17:05:22.509857+0000","last_clean":"2024-04-16T17:05:22.509857+0000","last_became_active":"2024-04-16T17:03:24.176555+0000","last_became_peered":"2024-04-16T17:03:24.176555+0000","last_unstale":"2024-04-16T17:05:22.509857+0000","last_undegraded":"2024-04-16T17:05:22.509857+0000","last_fullsized":"2024-04-16T17:05:22.509857+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:05:40.602150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"92'2","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.284637+0000","last_change":"2024-04-16T17:03:24.965746+0000","last_active":"2024-04-16T17:05:22.284637+0000","last_peered":"2024-04-16T17:05:22.284637+0000","last_clean":"2024-04-16T17:05:22.284637+0000","last_became_active":"2024-04-16T17:03:24.965241+0000","last_became_peered":"2024-04-16T17:03:24.965241+0000","last_unstale":"2024-04-16T17:05:22.284637+0000","last_undegraded":"2024-04-16T17:05:22.284637+0000","last_fullsized":"2024-04-16T17:05:22.284637+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:51:34.407565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.286457+0000","last_change":"2024-04-16T17:03:29.261441+0000","last_active":"2024-04-16T17:05:22.286457+0000","last_peered":"2024-04-16T17:05:22.286457+0000","last_clean":"2024-04-16T17:05:22.286457+0000","last_became_active":"2024-04-16T17:03:29.259538+0000","last_became_peered":"2024-04-16T17:03:29.259538+0000","last_unstale":"2024-04-16T17:05:22.286457+0000","last_undegraded":"2024-04-16T17:05:22.286457+0000","last_fullsized":"2024-04-16T17:05:22.286457+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:12:27.092053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"90'3","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.516605+0000","last_change":"2024-04-16T17:03:24.172053+0000","last_active":"2024-04-16T17:05:22.516605+0000","last_peered":"2024-04-16T17:05:22.516605+0000","last_clean":"2024-04-16T17:05:22.516605+0000","last_became_active":"2024-04-16T17:03:24.171709+0000","last_became_peered":"2024-04-16T17:03:24.171709+0000","last_unstale":"2024-04-16T17:05:22.516605+0000","last_undegraded":"2024-04-16T17:05:22.516605+0000","last_fullsized":"2024-04-16T17:05:22.516605+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:40:12.307756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"92'9","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.282775+0000","last_change":"2024-04-16T17:03:24.984267+0000","last_active":"2024-04-16T17:05:22.282775+0000","last_peered":"2024-04-16T17:05:22.282775+0000","last_clean":"2024-04-16T17:05:22.282775+0000","last_became_active":"2024-04-16T17:03:24.983888+0000","last_became_peered":"2024-04-16T17:03:24.983888+0000","last_unstale":"2024-04-16T17:05:22.282775+0000","last_undegraded":"2024-04-16T17:05:22.282775+0000","last_fullsized":"2024-04-16T17:05:22.282775+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:16:23.814917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"90'1","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.512450+0000","last_change":"2024-04-16T17:03:29.251716+0000","last_active":"2024-04-16T17:05:22.512450+0000","last_peered":"2024-04-16T17:05:22.512450+0000","last_clean":"2024-04-16T17:05:22.512450+0000","last_became_active":"2024-04-16T17:03:29.251317+0000","last_became_peered":"2024-04-16T17:03:29.251317+0000","last_unstale":"2024-04-16T17:05:22.512450+0000","last_undegraded":"2024-04-16T17:05:22.512450+0000","last_fullsized":"2024-04-16T17:05:22.512450+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:48:54.680195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"90'1","reported_seq":19,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.282769+0000","last_change":"2024-04-16T17:04:21.515306+0000","last_active":"2024-04-16T17:05:22.282769+0000","last_peered":"2024-04-16T17:05:22.282769+0000","last_clean":"2024-04-16T17:05:22.282769+0000","last_became_active":"2024-04-16T17:04:21.508377+0000","last_became_peered":"2024-04-16T17:04:21.508377+0000","last_unstale":"2024-04-16T17:05:22.282769+0000","last_undegraded":"2024-04-16T17:05:22.282769+0000","last_fullsized":"2024-04-16T17:05:22.282769+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:28:25.000406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"92'8","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.518147+0000","last_change":"2024-04-16T17:03:24.995547+0000","last_active":"2024-04-16T17:05:22.518147+0000","last_peered":"2024-04-16T17:05:22.518147+0000","last_clean":"2024-04-16T17:05:22.518147+0000","last_became_active":"2024-04-16T17:03:24.995096+0000","last_became_peered":"2024-04-16T17:03:24.995096+0000","last_unstale":"2024-04-16T17:05:22.518147+0000","last_undegraded":"2024-04-16T17:05:22.518147+0000","last_fullsized":"2024-04-16T17:05:22.518147+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:54:51.193117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.512835+0000","last_change":"2024-04-16T17:03:29.301526+0000","last_active":"2024-04-16T17:05:22.512835+0000","last_peered":"2024-04-16T17:05:22.512835+0000","last_clean":"2024-04-16T17:05:22.512835+0000","last_became_active":"2024-04-16T17:03:29.300808+0000","last_became_peered":"2024-04-16T17:03:29.300808+0000","last_unstale":"2024-04-16T17:05:22.512835+0000","last_undegraded":"2024-04-16T17:05:22.512835+0000","last_fullsized":"2024-04-16T17:05:22.512835+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:01:14.717548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"90'4","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.284713+0000","last_change":"2024-04-16T17:03:24.152645+0000","last_active":"2024-04-16T17:05:22.284713+0000","last_peered":"2024-04-16T17:05:22.284713+0000","last_clean":"2024-04-16T17:05:22.284713+0000","last_became_active":"2024-04-16T17:03:24.152281+0000","last_became_peered":"2024-04-16T17:03:24.152281+0000","last_unstale":"2024-04-16T17:05:22.284713+0000","last_undegraded":"2024-04-16T17:05:22.284713+0000","last_fullsized":"2024-04-16T17:05:22.284713+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:33:51.377421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"92'5","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.511176+0000","last_change":"2024-04-16T17:03:24.979827+0000","last_active":"2024-04-16T17:05:22.511176+0000","last_peered":"2024-04-16T17:05:22.511176+0000","last_clean":"2024-04-16T17:05:22.511176+0000","last_became_active":"2024-04-16T17:03:24.979452+0000","last_became_peered":"2024-04-16T17:03:24.979452+0000","last_unstale":"2024-04-16T17:05:22.511176+0000","last_undegraded":"2024-04-16T17:05:22.511176+0000","last_fullsized":"2024-04-16T17:05:22.511176+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:07:09.259366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.510047+0000","last_change":"2024-04-16T17:03:29.255519+0000","last_active":"2024-04-16T17:05:22.510047+0000","last_peered":"2024-04-16T17:05:22.510047+0000","last_clean":"2024-04-16T17:05:22.510047+0000","last_became_active":"2024-04-16T17:03:29.255104+0000","last_became_peered":"2024-04-16T17:03:29.255104+0000","last_unstale":"2024-04-16T17:05:22.510047+0000","last_undegraded":"2024-04-16T17:05:22.510047+0000","last_fullsized":"2024-04-16T17:05:22.510047+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:27:34.496199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"90'3","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.509582+0000","last_change":"2024-04-16T17:03:24.161145+0000","last_active":"2024-04-16T17:05:22.509582+0000","last_peered":"2024-04-16T17:05:22.509582+0000","last_clean":"2024-04-16T17:05:22.509582+0000","last_became_active":"2024-04-16T17:03:24.160824+0000","last_became_peered":"2024-04-16T17:03:24.160824+0000","last_unstale":"2024-04-16T17:05:22.509582+0000","last_undegraded":"2024-04-16T17:05:22.509582+0000","last_fullsized":"2024-04-16T17:05:22.509582+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:02:48.805638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"92'7","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.511418+0000","last_change":"2024-04-16T17:03:24.966513+0000","last_active":"2024-04-16T17:05:22.511418+0000","last_peered":"2024-04-16T17:05:22.511418+0000","last_clean":"2024-04-16T17:05:22.511418+0000","last_became_active":"2024-04-16T17:03:24.966074+0000","last_became_peered":"2024-04-16T17:03:24.966074+0000","last_unstale":"2024-04-16T17:05:22.511418+0000","last_undegraded":"2024-04-16T17:05:22.511418+0000","last_fullsized":"2024-04-16T17:05:22.511418+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:50:21.579901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.509016+0000","last_change":"2024-04-16T17:04:22.150786+0000","last_active":"2024-04-16T17:05:22.509016+0000","last_peered":"2024-04-16T17:05:22.509016+0000","last_clean":"2024-04-16T17:05:22.509016+0000","last_became_active":"2024-04-16T17:04:22.149344+0000","last_became_peered":"2024-04-16T17:04:22.149344+0000","last_unstale":"2024-04-16T17:05:22.509016+0000","last_undegraded":"2024-04-16T17:05:22.509016+0000","last_fullsized":"2024-04-16T17:05:22.509016+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:18:49.421095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,1,4],"acting":[0,8,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"85'1","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.513202+0000","last_change":"2024-04-16T17:03:24.158032+0000","last_active":"2024-04-16T17:05:22.513202+0000","last_peered":"2024-04-16T17:05:22.513202+0000","last_clean":"2024-04-16T17:05:22.513202+0000","last_became_active":"2024-04-16T17:03:24.157689+0000","last_became_peered":"2024-04-16T17:03:24.157689+0000","last_unstale":"2024-04-16T17:05:22.513202+0000","last_undegraded":"2024-04-16T17:05:22.513202+0000","last_fullsized":"2024-04-16T17:05:22.513202+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:38:18.240889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"92'6","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.511335+0000","last_change":"2024-04-16T17:04:21.519999+0000","last_active":"2024-04-16T17:05:22.511335+0000","last_peered":"2024-04-16T17:05:22.511335+0000","last_clean":"2024-04-16T17:05:22.511335+0000","last_became_active":"2024-04-16T17:04:21.519621+0000","last_became_peered":"2024-04-16T17:04:21.519621+0000","last_unstale":"2024-04-16T17:05:22.511335+0000","last_undegraded":"2024-04-16T17:05:22.511335+0000","last_fullsized":"2024-04-16T17:05:22.511335+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:50:13.266369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.512149+0000","last_change":"2024-04-16T17:03:29.245176+0000","last_active":"2024-04-16T17:05:22.512149+0000","last_peered":"2024-04-16T17:05:22.512149+0000","last_clean":"2024-04-16T17:05:22.512149+0000","last_became_active":"2024-04-16T17:03:29.244763+0000","last_became_peered":"2024-04-16T17:03:29.244763+0000","last_unstale":"2024-04-16T17:05:22.512149+0000","last_undegraded":"2024-04-16T17:05:22.512149+0000","last_fullsized":"2024-04-16T17:05:22.512149+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:52:30.381417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.290160+0000","last_change":"2024-04-16T17:03:24.168068+0000","last_active":"2024-04-16T17:05:22.290160+0000","last_peered":"2024-04-16T17:05:22.290160+0000","last_clean":"2024-04-16T17:05:22.290160+0000","last_became_active":"2024-04-16T17:03:24.167694+0000","last_became_peered":"2024-04-16T17:03:24.167694+0000","last_unstale":"2024-04-16T17:05:22.290160+0000","last_undegraded":"2024-04-16T17:05:22.290160+0000","last_fullsized":"2024-04-16T17:05:22.290160+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:13:35.692306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"92'9","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.510867+0000","last_change":"2024-04-16T17:03:24.966145+0000","last_active":"2024-04-16T17:05:22.510867+0000","last_peered":"2024-04-16T17:05:22.510867+0000","last_clean":"2024-04-16T17:05:22.510867+0000","last_became_active":"2024-04-16T17:03:24.965813+0000","last_became_peered":"2024-04-16T17:03:24.965813+0000","last_unstale":"2024-04-16T17:05:22.510867+0000","last_undegraded":"2024-04-16T17:05:22.510867+0000","last_fullsized":"2024-04-16T17:05:22.510867+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:43:43.224130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"92'9","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.282938+0000","last_change":"2024-04-16T17:03:24.970941+0000","last_active":"2024-04-16T17:05:22.282938+0000","last_peered":"2024-04-16T17:05:22.282938+0000","last_clean":"2024-04-16T17:05:22.282938+0000","last_became_active":"2024-04-16T17:03:24.970587+0000","last_became_peered":"2024-04-16T17:03:24.970587+0000","last_unstale":"2024-04-16T17:05:22.282938+0000","last_undegraded":"2024-04-16T17:05:22.282938+0000","last_fullsized":"2024-04-16T17:05:22.282938+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:14:13.930011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"92'12","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.509727+0000","last_change":"2024-04-16T17:03:24.967167+0000","last_active":"2024-04-16T17:05:22.509727+0000","last_peered":"2024-04-16T17:05:22.509727+0000","last_clean":"2024-04-16T17:05:22.509727+0000","last_became_active":"2024-04-16T17:03:24.966787+0000","last_became_peered":"2024-04-16T17:03:24.966787+0000","last_unstale":"2024-04-16T17:05:22.509727+0000","last_undegraded":"2024-04-16T17:05:22.509727+0000","last_fullsized":"2024-04-16T17:05:22.509727+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:51:39.510220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"92'8","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.509320+0000","last_change":"2024-04-16T17:03:24.981115+0000","last_active":"2024-04-16T17:05:22.509320+0000","last_peered":"2024-04-16T17:05:22.509320+0000","last_clean":"2024-04-16T17:05:22.509320+0000","last_became_active":"2024-04-16T17:03:24.980685+0000","last_became_peered":"2024-04-16T17:03:24.980685+0000","last_unstale":"2024-04-16T17:05:22.509320+0000","last_undegraded":"2024-04-16T17:05:22.509320+0000","last_fullsized":"2024-04-16T17:05:22.509320+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:20:44.066932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"92'7","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.509388+0000","last_change":"2024-04-16T17:03:24.990421+0000","last_active":"2024-04-16T17:05:22.509388+0000","last_peered":"2024-04-16T17:05:22.509388+0000","last_clean":"2024-04-16T17:05:22.509388+0000","last_became_active":"2024-04-16T17:03:24.989940+0000","last_became_peered":"2024-04-16T17:03:24.989940+0000","last_unstale":"2024-04-16T17:05:22.509388+0000","last_undegraded":"2024-04-16T17:05:22.509388+0000","last_fullsized":"2024-04-16T17:05:22.509388+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:19:57.548067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"92'4","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.289982+0000","last_change":"2024-04-16T17:03:24.991854+0000","last_active":"2024-04-16T17:05:22.289982+0000","last_peered":"2024-04-16T17:05:22.289982+0000","last_clean":"2024-04-16T17:05:22.289982+0000","last_became_active":"2024-04-16T17:03:24.990964+0000","last_became_peered":"2024-04-16T17:03:24.990964+0000","last_unstale":"2024-04-16T17:05:22.289982+0000","last_undegraded":"2024-04-16T17:05:22.289982+0000","last_fullsized":"2024-04-16T17:05:22.289982+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:10:10.618708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"92'6","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.283320+0000","last_change":"2024-04-16T17:04:21.517466+0000","last_active":"2024-04-16T17:05:22.283320+0000","last_peered":"2024-04-16T17:05:22.283320+0000","last_clean":"2024-04-16T17:05:22.283320+0000","last_became_active":"2024-04-16T17:04:21.517086+0000","last_became_peered":"2024-04-16T17:04:21.517086+0000","last_unstale":"2024-04-16T17:05:22.283320+0000","last_undegraded":"2024-04-16T17:05:22.283320+0000","last_fullsized":"2024-04-16T17:05:22.283320+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:54:33.417452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,9],"acting":[11,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"92'15","reported_seq":44,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.509874+0000","last_change":"2024-04-16T17:03:24.970377+0000","last_active":"2024-04-16T17:05:22.509874+0000","last_peered":"2024-04-16T17:05:22.509874+0000","last_clean":"2024-04-16T17:05:22.509874+0000","last_became_active":"2024-04-16T17:03:24.968856+0000","last_became_peered":"2024-04-16T17:03:24.968856+0000","last_unstale":"2024-04-16T17:05:22.509874+0000","last_undegraded":"2024-04-16T17:05:22.509874+0000","last_fullsized":"2024-04-16T17:05:22.509874+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:47:28.543120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"92'2","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.519300+0000","last_change":"2024-04-16T17:03:24.992877+0000","last_active":"2024-04-16T17:05:22.519300+0000","last_peered":"2024-04-16T17:05:22.519300+0000","last_clean":"2024-04-16T17:05:22.519300+0000","last_became_active":"2024-04-16T17:03:24.992484+0000","last_became_peered":"2024-04-16T17:03:24.992484+0000","last_unstale":"2024-04-16T17:05:22.519300+0000","last_undegraded":"2024-04-16T17:05:22.519300+0000","last_fullsized":"2024-04-16T17:05:22.519300+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:00:45.204782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"92'6","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.512392+0000","last_change":"2024-04-16T17:03:24.977794+0000","last_active":"2024-04-16T17:05:22.512392+0000","last_peered":"2024-04-16T17:05:22.512392+0000","last_clean":"2024-04-16T17:05:22.512392+0000","last_became_active":"2024-04-16T17:03:24.977434+0000","last_became_peered":"2024-04-16T17:03:24.977434+0000","last_unstale":"2024-04-16T17:05:22.512392+0000","last_undegraded":"2024-04-16T17:05:22.512392+0000","last_fullsized":"2024-04-16T17:05:22.512392+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:46:35.670357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"92'8","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.516646+0000","last_change":"2024-04-16T17:03:24.992225+0000","last_active":"2024-04-16T17:05:22.516646+0000","last_peered":"2024-04-16T17:05:22.516646+0000","last_clean":"2024-04-16T17:05:22.516646+0000","last_became_active":"2024-04-16T17:03:24.991904+0000","last_became_peered":"2024-04-16T17:03:24.991904+0000","last_unstale":"2024-04-16T17:05:22.516646+0000","last_undegraded":"2024-04-16T17:05:22.516646+0000","last_fullsized":"2024-04-16T17:05:22.516646+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:01:09.710698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"92'10","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.510323+0000","last_change":"2024-04-16T17:03:24.981322+0000","last_active":"2024-04-16T17:05:22.510323+0000","last_peered":"2024-04-16T17:05:22.510323+0000","last_clean":"2024-04-16T17:05:22.510323+0000","last_became_active":"2024-04-16T17:03:24.980920+0000","last_became_peered":"2024-04-16T17:03:24.980920+0000","last_unstale":"2024-04-16T17:05:22.510323+0000","last_undegraded":"2024-04-16T17:05:22.510323+0000","last_fullsized":"2024-04-16T17:05:22.510323+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:07:59.378512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"92'11","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.289181+0000","last_change":"2024-04-16T17:03:24.990117+0000","last_active":"2024-04-16T17:05:22.289181+0000","last_peered":"2024-04-16T17:05:22.289181+0000","last_clean":"2024-04-16T17:05:22.289181+0000","last_became_active":"2024-04-16T17:03:24.989734+0000","last_became_peered":"2024-04-16T17:03:24.989734+0000","last_unstale":"2024-04-16T17:05:22.289181+0000","last_undegraded":"2024-04-16T17:05:22.289181+0000","last_fullsized":"2024-04-16T17:05:22.289181+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:36:55.000702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"92'8","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.283130+0000","last_change":"2024-04-16T17:03:24.970032+0000","last_active":"2024-04-16T17:05:22.283130+0000","last_peered":"2024-04-16T17:05:22.283130+0000","last_clean":"2024-04-16T17:05:22.283130+0000","last_became_active":"2024-04-16T17:03:24.968608+0000","last_became_peered":"2024-04-16T17:03:24.968608+0000","last_unstale":"2024-04-16T17:05:22.283130+0000","last_undegraded":"2024-04-16T17:05:22.283130+0000","last_fullsized":"2024-04-16T17:05:22.283130+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:24:03.918418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":283858,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":68279,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":583,"num_read_kb":365,"num_write":175,"num_write_kb":125,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":616,"num_keys_recovered":20,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":204837,"data_compressed":4887,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":151,"ondisk_log_size":151,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351014,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1032192,"data_stored":522666,"data_compressed":26684,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612653,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32332,"kb_used_data":1148,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712820,"statfs":{"total":95995035648,"available":95961927680,"internally_reserved":0,"allocated":1175552,"data_stored":1178133,"data_compressed":64411,"data_compressed_allocated":86016,"data_compressed_original":671744,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776178,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32204,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712948,"statfs":{"total":95995035648,"available":95962058752,"internally_reserved":0,"allocated":1044480,"data_stored":522868,"data_compressed":26684,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037817,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32200,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712952,"statfs":{"total":95995035648,"available":95962062848,"internally_reserved":0,"allocated":1040384,"data_stored":522206,"data_compressed":26684,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299455,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36412,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708740,"statfs":{"total":95995035648,"available":95957749760,"internally_reserved":0,"allocated":1093632,"data_stored":1119370,"data_compressed":62782,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462980,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32184,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712968,"statfs":{"total":95995035648,"available":95962079232,"internally_reserved":0,"allocated":1024000,"data_stored":518044,"data_compressed":26684,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574945,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32248,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712904,"statfs":{"total":95995035648,"available":95962013696,"internally_reserved":0,"allocated":1093632,"data_stored":582141,"data_compressed":28313,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738469,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32208,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712944,"statfs":{"total":95995035648,"available":95962054656,"internally_reserved":0,"allocated":1048576,"data_stored":519432,"data_compressed":26684,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411418,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28104,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717048,"statfs":{"total":95995035648,"available":95966257152,"internally_reserved":0,"allocated":1110016,"data_stored":583407,"data_compressed":28313,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149780,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32192,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712960,"statfs":{"total":95995035648,"available":95962071040,"internally_reserved":0,"allocated":1036288,"data_stored":518238,"data_compressed":26684,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888143,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1036288,"data_stored":523226,"data_compressed":26684,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724618,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28096,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717056,"statfs":{"total":95995035648,"available":95966265344,"internally_reserved":0,"allocated":1101824,"data_stored":1114426,"data_compressed":62782,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34456,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34456,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34456,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36098,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36098,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36098,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36098,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36098,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":64187,"data_compressed":1629,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":65453,"data_compressed":1629,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":156,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":65715,"data_compressed":1629,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-16T17:05:30.860 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45 shell --fsid b1fecd38-fc11-11ee-bc90-c7b262605968 -- ceph status --format=json 2024-04-16T17:05:31.364 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/mon.a/config 2024-04-16T17:05:31.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:05:31 smithi104 ceph-mon[36457]: from='client.14946 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-16T17:05:31.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:05:31 smithi104 ceph-mon[36457]: pgmap v389: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 576 B/s wr, 6 op/s 2024-04-16T17:05:31.698 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:05:31 smithi045 ceph-mon[31676]: from='client.14946 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-16T17:05:31.698 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:05:31 smithi045 ceph-mon[31676]: pgmap v389: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 576 B/s wr, 6 op/s 2024-04-16T17:05:31.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:05:31 smithi117 ceph-mon[36730]: from='client.14946 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-16T17:05:31.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:05:31 smithi117 ceph-mon[36730]: pgmap v389: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 576 B/s wr, 6 op/s 2024-04-16T17:05:33.441 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:05:33 smithi045 ceph-mon[31676]: pgmap v390: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 409 B/s wr, 6 op/s 2024-04-16T17:05:33.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:33.613+0000 7f3e2a400700 1 -- 172.21.15.45:0/369118568 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3e2410e460 msgr2=0x7f3e24116030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:33.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:33.613+0000 7f3e2a400700 1 --2- 172.21.15.45:0/369118568 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3e2410e460 0x7f3e24116030 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f3e1c00b0d0 tx=0x7f3e1c00b3e0 comp rx=0 tx=0).stop 2024-04-16T17:05:33.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:33.613+0000 7f3e2a400700 1 -- 172.21.15.45:0/369118568 shutdown_connections 2024-04-16T17:05:33.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:33.613+0000 7f3e2a400700 1 --2- 172.21.15.45:0/369118568 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3e24109380 0x7f3e24109760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:33.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:33.613+0000 7f3e2a400700 1 --2- 172.21.15.45:0/369118568 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3e2410e460 0x7f3e24116030 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:33.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:33.613+0000 7f3e2a400700 1 --2- 172.21.15.45:0/369118568 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3e24109d30 0x7f3e2410dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:33.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:33.613+0000 7f3e2a400700 1 -- 172.21.15.45:0/369118568 >> 172.21.15.45:0/369118568 conn(0x7f3e240762c0 msgr2=0x7f3e24107910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:05:33.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:33.613+0000 7f3e2a400700 1 -- 172.21.15.45:0/369118568 shutdown_connections 2024-04-16T17:05:33.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:33.613+0000 7f3e2a400700 1 -- 172.21.15.45:0/369118568 wait complete. 2024-04-16T17:05:33.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:33.615+0000 7f3e2a400700 1 Processor -- start 2024-04-16T17:05:33.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:33.615+0000 7f3e2a400700 1 -- start start 2024-04-16T17:05:33.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:33.616+0000 7f3e2a400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3e24109380 0x7f3e241a5660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:33.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:33.616+0000 7f3e2a400700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3e24109d30 0x7f3e241a5ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:33.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:33.616+0000 7f3e2a400700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3e2410e460 0x7f3e2419f780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:33.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:33.616+0000 7f3e2a400700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f3e241194c0 con 0x7f3e2410e460 2024-04-16T17:05:33.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:33.616+0000 7f3e2a400700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f3e24119340 con 0x7f3e24109d30 2024-04-16T17:05:33.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:33.616+0000 7f3e2a400700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f3e24119640 con 0x7f3e24109380 2024-04-16T17:05:33.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:33.616+0000 7f3e23e00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3e2410e460 0x7f3e2419f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:33.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:33.616+0000 7f3e23e00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3e2410e460 0x7f3e2419f780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:46666/0 (socket says 172.21.15.45:46666) 2024-04-16T17:05:33.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:33.616+0000 7f3e23e00700 1 -- 172.21.15.45:0/4274778033 learned_addr learned my addr 172.21.15.45:0/4274778033 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:05:33.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:33.616+0000 7f3e22a00700 1 --2- 172.21.15.45:0/4274778033 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3e24109d30 0x7f3e241a5ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:33.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:33.616+0000 7f3e23400700 1 --2- 172.21.15.45:0/4274778033 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3e24109380 0x7f3e241a5660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:33.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:33.616+0000 7f3e23e00700 1 -- 172.21.15.45:0/4274778033 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3e24109380 msgr2=0x7f3e241a5660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:33.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:33.616+0000 7f3e23e00700 1 --2- 172.21.15.45:0/4274778033 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3e24109380 0x7f3e241a5660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:33.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:33.616+0000 7f3e23e00700 1 -- 172.21.15.45:0/4274778033 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3e24109d30 msgr2=0x7f3e241a5ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:33.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:33.616+0000 7f3e23e00700 1 --2- 172.21.15.45:0/4274778033 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3e24109d30 0x7f3e241a5ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:33.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:33.616+0000 7f3e23e00700 1 -- 172.21.15.45:0/4274778033 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e241a0060 con 0x7f3e2410e460 2024-04-16T17:05:33.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:33.617+0000 7f3e23e00700 1 --2- 172.21.15.45:0/4274778033 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3e2410e460 0x7f3e2419f780 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f3e1c00d010 tx=0x7f3e1c0079a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:05:33.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:33.617+0000 7f3e1be00700 1 -- 172.21.15.45:0/4274778033 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e1c003c00 con 0x7f3e2410e460 2024-04-16T17:05:33.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:33.617+0000 7f3e1be00700 1 -- 172.21.15.45:0/4274778033 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3e1c007e10 con 0x7f3e2410e460 2024-04-16T17:05:33.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:33.617+0000 7f3e1be00700 1 -- 172.21.15.45:0/4274778033 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e1c01b8d0 con 0x7f3e2410e460 2024-04-16T17:05:33.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:33.617+0000 7f3e2a400700 1 -- 172.21.15.45:0/4274778033 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e241a02f0 con 0x7f3e2410e460 2024-04-16T17:05:33.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:33.617+0000 7f3e2a400700 1 -- 172.21.15.45:0/4274778033 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e241ac410 con 0x7f3e2410e460 2024-04-16T17:05:33.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:33.618+0000 7f3e2a400700 1 -- 172.21.15.45:0/4274778033 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e2410a860 con 0x7f3e2410e460 2024-04-16T17:05:33.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:33.621+0000 7f3e1be00700 1 -- 172.21.15.45:0/4274778033 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f3e1c009ba0 con 0x7f3e2410e460 2024-04-16T17:05:33.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:33.622+0000 7f3e1be00700 1 --2- 172.21.15.45:0/4274778033 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f3e080a3cc0 0x7f3e080a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:33.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:33.622+0000 7f3e1be00700 1 -- 172.21.15.45:0/4274778033 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f3e1c0d1b80 con 0x7f3e2410e460 2024-04-16T17:05:33.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:33.623+0000 7f3e23400700 1 --2- 172.21.15.45:0/4274778033 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f3e080a3cc0 0x7f3e080a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:33.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:33.623+0000 7f3e23400700 1 --2- 172.21.15.45:0/4274778033 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f3e080a3cc0 0x7f3e080a6180 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f3e14009c60 tx=0x7f3e14009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:05:33.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:33.627+0000 7f3e1be00700 1 -- 172.21.15.45:0/4274778033 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3e1c09e200 con 0x7f3e2410e460 2024-04-16T17:05:33.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:05:33 smithi104 ceph-mon[36457]: pgmap v390: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 409 B/s wr, 6 op/s 2024-04-16T17:05:33.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:05:33 smithi117 ceph-mon[36730]: pgmap v390: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 409 B/s wr, 6 op/s 2024-04-16T17:05:33.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:33.953+0000 7f3e2a400700 1 -- 172.21.15.45:0/4274778033 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f3e2402dfe0 con 0x7f3e2410e460 2024-04-16T17:05:33.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:33.954+0000 7f3e1be00700 1 -- 172.21.15.45:0/4274778033 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1495 (secure 0 0 0) 0x7f3e1c09db00 con 0x7f3e2410e460 2024-04-16T17:05:33.955 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-16T17:05:33.955 INFO:teuthology.orchestra.run.smithi045.stdout:{"fsid":"b1fecd38-fc11-11ee-bc90-c7b262605968","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":14,"quorum":[0,1,2],"quorum_names":["a","c","b"],"quorum_age":540,"monmap":{"epoch":3,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":94,"num_osds":12,"num_up_osds":12,"osd_up_since":1713286945,"num_in_osds":12,"osd_in_since":1713286924,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":129}],"num_pgs":129,"num_pools":4,"num_objects":64,"data_bytes":658881,"bytes_used":383225856,"bytes_avail":1151557201920,"bytes_total":1151940427776,"read_bytes_sec":3274,"write_bytes_sec":409,"read_op_per_sec":6,"write_op_per_sec":0},"fsmap":{"epoch":24,"id":1,"up":3,"in":3,"max":3,"by_rank":[{"filesystem_id":1,"rank":2,"name":"f","status":"up:standby-replay","gid":24499},{"filesystem_id":1,"rank":1,"name":"c","status":"up:standby-replay","gid":24511},{"filesystem_id":1,"rank":0,"name":"h","status":"up:standby-replay","gid":24524},{"filesystem_id":1,"rank":2,"name":"i","status":"up:active","gid":24532},{"filesystem_id":1,"rank":1,"name":"a","status":"up:active","gid":24569},{"filesystem_id":1,"rank":0,"name":"b","status":"up:active","gid":24581}],"up:standby":6},"mgrmap":{"available":true,"num_standbys":2,"modules":["cephadm","dashboard","iostat","nfs","restful"],"services":{"dashboard":"https://172.21.15.45:8443/"}},"servicemap":{"epoch":28,"modified":"2024-04-16T17:05:00.141085+0000","services":{}},"progress_events":{}} 2024-04-16T17:05:33.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:33.957+0000 7f3e2a400700 1 -- 172.21.15.45:0/4274778033 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f3e080a3cc0 msgr2=0x7f3e080a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:33.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:33.957+0000 7f3e2a400700 1 --2- 172.21.15.45:0/4274778033 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f3e080a3cc0 0x7f3e080a6180 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f3e14009c60 tx=0x7f3e14009260 comp rx=0 tx=0).stop 2024-04-16T17:05:33.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:33.957+0000 7f3e2a400700 1 -- 172.21.15.45:0/4274778033 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3e2410e460 msgr2=0x7f3e2419f780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:33.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:33.957+0000 7f3e2a400700 1 --2- 172.21.15.45:0/4274778033 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3e2410e460 0x7f3e2419f780 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f3e1c00d010 tx=0x7f3e1c0079a0 comp rx=0 tx=0).stop 2024-04-16T17:05:33.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:33.957+0000 7f3e2a400700 1 -- 172.21.15.45:0/4274778033 shutdown_connections 2024-04-16T17:05:33.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:33.957+0000 7f3e2a400700 1 --2- 172.21.15.45:0/4274778033 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f3e080a3cc0 0x7f3e080a6180 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:33.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:33.957+0000 7f3e2a400700 1 --2- 172.21.15.45:0/4274778033 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3e24109380 0x7f3e241a5660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:33.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:33.957+0000 7f3e2a400700 1 --2- 172.21.15.45:0/4274778033 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3e2410e460 0x7f3e2419f780 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:33.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:33.957+0000 7f3e2a400700 1 --2- 172.21.15.45:0/4274778033 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3e24109d30 0x7f3e241a5ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:33.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:33.957+0000 7f3e2a400700 1 -- 172.21.15.45:0/4274778033 >> 172.21.15.45:0/4274778033 conn(0x7f3e240762c0 msgr2=0x7f3e24076880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:05:33.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:33.957+0000 7f3e2a400700 1 -- 172.21.15.45:0/4274778033 shutdown_connections 2024-04-16T17:05:33.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:33.957+0000 7f3e2a400700 1 -- 172.21.15.45:0/4274778033 wait complete. 2024-04-16T17:05:34.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:05:34 smithi104 ceph-mon[36457]: from='client.? 172.21.15.45:0/4274778033' entity='client.admin' cmd=[{"prefix": "status", "format": "json"}]: dispatch 2024-04-16T17:05:34.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:05:34 smithi045 ceph-mon[31676]: from='client.? 172.21.15.45:0/4274778033' entity='client.admin' cmd=[{"prefix": "status", "format": "json"}]: dispatch 2024-04-16T17:05:34.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:05:34 smithi117 ceph-mon[36730]: from='client.? 172.21.15.45:0/4274778033' entity='client.admin' cmd=[{"prefix": "status", "format": "json"}]: dispatch 2024-04-16T17:05:34.835 INFO:tasks.cephadm.ceph_manager.ceph:no progress seen, keeping timeout for now 2024-04-16T17:05:34.836 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45 shell --fsid b1fecd38-fc11-11ee-bc90-c7b262605968 -- ceph pg dump --format=json 2024-04-16T17:05:35.318 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/mon.a/config 2024-04-16T17:05:35.625 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:05:35 smithi045 ceph-mon[31676]: pgmap v391: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 344 B/s wr, 5 op/s 2024-04-16T17:05:35.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:05:35 smithi104 ceph-mon[36457]: pgmap v391: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 344 B/s wr, 5 op/s 2024-04-16T17:05:35.707 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:05:35 smithi117 ceph-mon[36730]: pgmap v391: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 344 B/s wr, 5 op/s 2024-04-16T17:05:37.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:37.553+0000 7f3cbfe00700 1 -- 172.21.15.45:0/1106129448 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3cc0105470 msgr2=0x7f3cc01058f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:37.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:37.553+0000 7f3cbfe00700 1 --2- 172.21.15.45:0/1106129448 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3cc0105470 0x7f3cc01058f0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f3cb800b0a0 tx=0x7f3cb800b3b0 comp rx=0 tx=0).stop 2024-04-16T17:05:37.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:37.553+0000 7f3cbfe00700 1 -- 172.21.15.45:0/1106129448 shutdown_connections 2024-04-16T17:05:37.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:37.553+0000 7f3cbfe00700 1 --2- 172.21.15.45:0/1106129448 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3cc0105470 0x7f3cc01058f0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:37.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:37.553+0000 7f3cbfe00700 1 --2- 172.21.15.45:0/1106129448 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3cc01079d0 0x7f3cc006b060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:37.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:37.553+0000 7f3cbfe00700 1 --2- 172.21.15.45:0/1106129448 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3cc006b5a0 0x7f3cc006ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:37.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:37.553+0000 7f3cbfe00700 1 -- 172.21.15.45:0/1106129448 >> 172.21.15.45:0/1106129448 conn(0x7f3cc0078830 msgr2=0x7f3cc0078c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:05:37.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:37.553+0000 7f3cbfe00700 1 -- 172.21.15.45:0/1106129448 shutdown_connections 2024-04-16T17:05:37.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:37.553+0000 7f3cbfe00700 1 -- 172.21.15.45:0/1106129448 wait complete. 2024-04-16T17:05:37.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:37.555+0000 7f3cbfe00700 1 Processor -- start 2024-04-16T17:05:37.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:37.555+0000 7f3cbfe00700 1 -- start start 2024-04-16T17:05:37.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:37.555+0000 7f3cbfe00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3cc006b5a0 0x7f3cc019f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:37.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:37.555+0000 7f3cbfe00700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3cc0105470 0x7f3cc019fb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:37.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:37.555+0000 7f3cbfe00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3cc01079d0 0x7f3cc01a3e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:37.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:37.555+0000 7f3cbfe00700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f3cc0119280 con 0x7f3cc006b5a0 2024-04-16T17:05:37.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:37.555+0000 7f3cbfe00700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f3cc0119100 con 0x7f3cc0105470 2024-04-16T17:05:37.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:37.555+0000 7f3cbfe00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f3cc0119400 con 0x7f3cc01079d0 2024-04-16T17:05:37.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:37.556+0000 7f3cbea00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3cc006b5a0 0x7f3cc019f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:37.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:37.556+0000 7f3cbea00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3cc006b5a0 0x7f3cc019f5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:46672/0 (socket says 172.21.15.45:46672) 2024-04-16T17:05:37.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:37.556+0000 7f3cbea00700 1 -- 172.21.15.45:0/263249425 learned_addr learned my addr 172.21.15.45:0/263249425 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:05:37.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:37.556+0000 7f3cbe000700 1 --2- 172.21.15.45:0/263249425 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3cc0105470 0x7f3cc019fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:37.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:37.556+0000 7f3cbf400700 1 --2- 172.21.15.45:0/263249425 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3cc01079d0 0x7f3cc01a3e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:37.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:37.556+0000 7f3cbea00700 1 -- 172.21.15.45:0/263249425 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3cc01079d0 msgr2=0x7f3cc01a3e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:37.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:37.556+0000 7f3cbea00700 1 --2- 172.21.15.45:0/263249425 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3cc01079d0 0x7f3cc01a3e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:37.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:37.556+0000 7f3cbea00700 1 -- 172.21.15.45:0/263249425 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3cc0105470 msgr2=0x7f3cc019fb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:37.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:37.556+0000 7f3cbea00700 1 --2- 172.21.15.45:0/263249425 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3cc0105470 0x7f3cc019fb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:37.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:37.556+0000 7f3cbea00700 1 -- 172.21.15.45:0/263249425 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3cc01a45b0 con 0x7f3cc006b5a0 2024-04-16T17:05:37.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:37.557+0000 7f3cbea00700 1 --2- 172.21.15.45:0/263249425 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3cc006b5a0 0x7f3cc019f5f0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f3cb000e9a0 tx=0x7f3cb000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:05:37.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:37.557+0000 7f3cb7400700 1 -- 172.21.15.45:0/263249425 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cb000ca10 con 0x7f3cc006b5a0 2024-04-16T17:05:37.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:37.557+0000 7f3cb7400700 1 -- 172.21.15.45:0/263249425 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3cb000cbb0 con 0x7f3cc006b5a0 2024-04-16T17:05:37.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:37.557+0000 7f3cb7400700 1 -- 172.21.15.45:0/263249425 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cb0018680 con 0x7f3cc006b5a0 2024-04-16T17:05:37.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:37.557+0000 7f3cbfe00700 1 -- 172.21.15.45:0/263249425 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3cc01a48a0 con 0x7f3cc006b5a0 2024-04-16T17:05:37.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:37.557+0000 7f3cbfe00700 1 -- 172.21.15.45:0/263249425 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3cc01ac120 con 0x7f3cc006b5a0 2024-04-16T17:05:37.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:37.558+0000 7f3cbfe00700 1 -- 172.21.15.45:0/263249425 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3cc00697e0 con 0x7f3cc006b5a0 2024-04-16T17:05:37.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:37.561+0000 7f3cb7400700 1 -- 172.21.15.45:0/263249425 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f3cb0010430 con 0x7f3cc006b5a0 2024-04-16T17:05:37.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:37.562+0000 7f3cb7400700 1 --2- 172.21.15.45:0/263249425 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f3ca40a3cc0 0x7f3ca40a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:37.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:37.562+0000 7f3cb7400700 1 -- 172.21.15.45:0/263249425 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f3cb00d0e40 con 0x7f3cc006b5a0 2024-04-16T17:05:37.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:37.562+0000 7f3cbe000700 1 --2- 172.21.15.45:0/263249425 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f3ca40a3cc0 0x7f3ca40a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:37.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:37.563+0000 7f3cbe000700 1 --2- 172.21.15.45:0/263249425 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f3ca40a3cc0 0x7f3ca40a6180 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f3cc006bc10 tx=0x7f3cac009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:05:37.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:37.565+0000 7f3cb7400700 1 -- 172.21.15.45:0/263249425 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3cb009d430 con 0x7f3cc006b5a0 2024-04-16T17:05:37.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:05:37 smithi104 ceph-mon[36457]: pgmap v392: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 341 B/s wr, 7 op/s 2024-04-16T17:05:37.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:05:37 smithi045 ceph-mon[31676]: pgmap v392: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 341 B/s wr, 7 op/s 2024-04-16T17:05:37.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:37.792+0000 7f3cbfe00700 1 -- 172.21.15.45:0/263249425 --> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f3cc01012b0 con 0x7f3ca40a3cc0 2024-04-16T17:05:37.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:05:37 smithi117 ceph-mon[36730]: pgmap v392: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 341 B/s wr, 7 op/s 2024-04-16T17:05:37.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:37.808+0000 7f3cb7400700 1 -- 172.21.15.45:0/263249425 <== mgr.14152 v2:172.21.15.45:6800/268030699 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340817 (secure 0 0 0) 0x7f3cc01012b0 con 0x7f3ca40a3cc0 2024-04-16T17:05:37.814 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-16T17:05:37.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:37.814+0000 7f3cbfe00700 1 -- 172.21.15.45:0/263249425 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f3ca40a3cc0 msgr2=0x7f3ca40a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:37.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:37.814+0000 7f3cbfe00700 1 --2- 172.21.15.45:0/263249425 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f3ca40a3cc0 0x7f3ca40a6180 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f3cc006bc10 tx=0x7f3cac009260 comp rx=0 tx=0).stop 2024-04-16T17:05:37.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:37.814+0000 7f3cbfe00700 1 -- 172.21.15.45:0/263249425 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3cc006b5a0 msgr2=0x7f3cc019f5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:37.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:37.814+0000 7f3cbfe00700 1 --2- 172.21.15.45:0/263249425 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3cc006b5a0 0x7f3cc019f5f0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f3cb000e9a0 tx=0x7f3cb000ecb0 comp rx=0 tx=0).stop 2024-04-16T17:05:37.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:37.814+0000 7f3cbfe00700 1 -- 172.21.15.45:0/263249425 shutdown_connections 2024-04-16T17:05:37.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:37.814+0000 7f3cbfe00700 1 --2- 172.21.15.45:0/263249425 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f3ca40a3cc0 0x7f3ca40a6180 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:37.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:37.814+0000 7f3cbfe00700 1 --2- 172.21.15.45:0/263249425 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3cc01079d0 0x7f3cc01a3e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:37.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:37.814+0000 7f3cbfe00700 1 --2- 172.21.15.45:0/263249425 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3cc006b5a0 0x7f3cc019f5f0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:37.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:37.814+0000 7f3cbfe00700 1 --2- 172.21.15.45:0/263249425 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3cc0105470 0x7f3cc019fb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:37.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:37.814+0000 7f3cbfe00700 1 -- 172.21.15.45:0/263249425 >> 172.21.15.45:0/263249425 conn(0x7f3cc0078830 msgr2=0x7f3cc0100ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:05:37.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:37.815+0000 7f3cbfe00700 1 -- 172.21.15.45:0/263249425 shutdown_connections 2024-04-16T17:05:37.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:37.815+0000 7f3cbfe00700 1 -- 172.21.15.45:0/263249425 wait complete. 2024-04-16T17:05:37.818 INFO:teuthology.orchestra.run.smithi045.stderr:dumped all 2024-04-16T17:05:38.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:05:38 smithi104 ceph-mon[36457]: from='client.14958 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-16T17:05:38.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:05:38 smithi045 ceph-mon[31676]: from='client.14958 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-16T17:05:38.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:05:38 smithi117 ceph-mon[36730]: from='client.14958 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-16T17:05:38.820 INFO:teuthology.orchestra.run.smithi045.stdout:{"pg_ready":true,"pg_map":{"version":392,"stamp":"2024-04-16T17:05:36.159800+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":658881,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":774,"num_read_kb":514,"num_write":358,"num_write_kb":3065,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":1181469,"num_keys_recovered":20,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":790,"ondisk_log_size":790,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":374244,"kb_used_data":12536,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124567580,"statfs":{"total":1151940427776,"available":1151557201920,"internally_reserved":0,"allocated":12836864,"data_stored":8224157,"data_compressed":433389,"data_compressed_allocated":589824,"data_compressed_original":2678784,"omap_allocated":19143,"internal_metadata":370193721},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":249,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":87,"num_read_kb":47,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006817"},"pg_stats":[{"pgid":"3.3f","version":"92'8","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.283618+0000","last_change":"2024-04-16T17:03:24.989508+0000","last_active":"2024-04-16T17:05:22.283618+0000","last_peered":"2024-04-16T17:05:22.283618+0000","last_clean":"2024-04-16T17:05:22.283618+0000","last_became_active":"2024-04-16T17:03:24.989178+0000","last_became_peered":"2024-04-16T17:03:24.989178+0000","last_unstale":"2024-04-16T17:05:22.283618+0000","last_undegraded":"2024-04-16T17:05:22.283618+0000","last_fullsized":"2024-04-16T17:05:22.283618+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:18:56.488358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"92'4","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.510710+0000","last_change":"2024-04-16T17:03:24.986378+0000","last_active":"2024-04-16T17:05:22.510710+0000","last_peered":"2024-04-16T17:05:22.510710+0000","last_clean":"2024-04-16T17:05:22.510710+0000","last_became_active":"2024-04-16T17:03:24.986030+0000","last_became_peered":"2024-04-16T17:03:24.986030+0000","last_unstale":"2024-04-16T17:05:22.510710+0000","last_undegraded":"2024-04-16T17:05:22.510710+0000","last_fullsized":"2024-04-16T17:05:22.510710+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:21:14.071109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"92'7","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.509472+0000","last_change":"2024-04-16T17:03:24.976232+0000","last_active":"2024-04-16T17:05:22.509472+0000","last_peered":"2024-04-16T17:05:22.509472+0000","last_clean":"2024-04-16T17:05:22.509472+0000","last_became_active":"2024-04-16T17:03:24.975873+0000","last_became_peered":"2024-04-16T17:03:24.975873+0000","last_unstale":"2024-04-16T17:05:22.509472+0000","last_undegraded":"2024-04-16T17:05:22.509472+0000","last_fullsized":"2024-04-16T17:05:22.509472+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:07:53.523101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"92'8","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.285223+0000","last_change":"2024-04-16T17:04:21.517830+0000","last_active":"2024-04-16T17:05:22.285223+0000","last_peered":"2024-04-16T17:05:22.285223+0000","last_clean":"2024-04-16T17:05:22.285223+0000","last_became_active":"2024-04-16T17:04:21.516749+0000","last_became_peered":"2024-04-16T17:04:21.516749+0000","last_unstale":"2024-04-16T17:05:22.285223+0000","last_undegraded":"2024-04-16T17:05:22.285223+0000","last_fullsized":"2024-04-16T17:05:22.285223+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:02:33.344034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"92'4","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.284780+0000","last_change":"2024-04-16T17:03:24.985609+0000","last_active":"2024-04-16T17:05:22.284780+0000","last_peered":"2024-04-16T17:05:22.284780+0000","last_clean":"2024-04-16T17:05:22.284780+0000","last_became_active":"2024-04-16T17:03:24.985239+0000","last_became_peered":"2024-04-16T17:03:24.985239+0000","last_unstale":"2024-04-16T17:05:22.284780+0000","last_undegraded":"2024-04-16T17:05:22.284780+0000","last_fullsized":"2024-04-16T17:05:22.284780+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:55:58.619210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"92'6","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.282909+0000","last_change":"2024-04-16T17:03:24.966862+0000","last_active":"2024-04-16T17:05:22.282909+0000","last_peered":"2024-04-16T17:05:22.282909+0000","last_clean":"2024-04-16T17:05:22.282909+0000","last_became_active":"2024-04-16T17:03:24.966234+0000","last_became_peered":"2024-04-16T17:03:24.966234+0000","last_unstale":"2024-04-16T17:05:22.282909+0000","last_undegraded":"2024-04-16T17:05:22.282909+0000","last_fullsized":"2024-04-16T17:05:22.282909+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:21:07.018281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"92'5","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.510598+0000","last_change":"2024-04-16T17:03:24.971219+0000","last_active":"2024-04-16T17:05:22.510598+0000","last_peered":"2024-04-16T17:05:22.510598+0000","last_clean":"2024-04-16T17:05:22.510598+0000","last_became_active":"2024-04-16T17:03:24.970876+0000","last_became_peered":"2024-04-16T17:03:24.970876+0000","last_unstale":"2024-04-16T17:05:22.510598+0000","last_undegraded":"2024-04-16T17:05:22.510598+0000","last_fullsized":"2024-04-16T17:05:22.510598+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:44:16.311167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"92'9","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.515605+0000","last_change":"2024-04-16T17:03:24.994694+0000","last_active":"2024-04-16T17:05:22.515605+0000","last_peered":"2024-04-16T17:05:22.515605+0000","last_clean":"2024-04-16T17:05:22.515605+0000","last_became_active":"2024-04-16T17:03:24.994315+0000","last_became_peered":"2024-04-16T17:03:24.994315+0000","last_unstale":"2024-04-16T17:05:22.515605+0000","last_undegraded":"2024-04-16T17:05:22.515605+0000","last_fullsized":"2024-04-16T17:05:22.515605+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:04:59.543816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"92'11","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.512218+0000","last_change":"2024-04-16T17:03:24.964654+0000","last_active":"2024-04-16T17:05:22.512218+0000","last_peered":"2024-04-16T17:05:22.512218+0000","last_clean":"2024-04-16T17:05:22.512218+0000","last_became_active":"2024-04-16T17:03:24.964349+0000","last_became_peered":"2024-04-16T17:03:24.964349+0000","last_unstale":"2024-04-16T17:05:22.512218+0000","last_undegraded":"2024-04-16T17:05:22.512218+0000","last_fullsized":"2024-04-16T17:05:22.512218+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:38:55.880948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"92'16","reported_seq":45,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.283246+0000","last_change":"2024-04-16T17:03:24.966364+0000","last_active":"2024-04-16T17:05:22.283246+0000","last_peered":"2024-04-16T17:05:22.283246+0000","last_clean":"2024-04-16T17:05:22.283246+0000","last_became_active":"2024-04-16T17:03:24.966061+0000","last_became_peered":"2024-04-16T17:03:24.966061+0000","last_unstale":"2024-04-16T17:05:22.283246+0000","last_undegraded":"2024-04-16T17:05:22.283246+0000","last_fullsized":"2024-04-16T17:05:22.283246+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:18:12.642602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"92'6","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.511278+0000","last_change":"2024-04-16T17:03:24.969874+0000","last_active":"2024-04-16T17:05:22.511278+0000","last_peered":"2024-04-16T17:05:22.511278+0000","last_clean":"2024-04-16T17:05:22.511278+0000","last_became_active":"2024-04-16T17:03:24.969523+0000","last_became_peered":"2024-04-16T17:03:24.969523+0000","last_unstale":"2024-04-16T17:05:22.511278+0000","last_undegraded":"2024-04-16T17:05:22.511278+0000","last_fullsized":"2024-04-16T17:05:22.511278+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:17:49.704855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"92'7","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.512621+0000","last_change":"2024-04-16T17:03:24.968660+0000","last_active":"2024-04-16T17:05:22.512621+0000","last_peered":"2024-04-16T17:05:22.512621+0000","last_clean":"2024-04-16T17:05:22.512621+0000","last_became_active":"2024-04-16T17:03:24.968348+0000","last_became_peered":"2024-04-16T17:03:24.968348+0000","last_unstale":"2024-04-16T17:05:22.512621+0000","last_undegraded":"2024-04-16T17:05:22.512621+0000","last_fullsized":"2024-04-16T17:05:22.512621+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:06:06.319391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"92'10","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.289436+0000","last_change":"2024-04-16T17:03:24.974741+0000","last_active":"2024-04-16T17:05:22.289436+0000","last_peered":"2024-04-16T17:05:22.289436+0000","last_clean":"2024-04-16T17:05:22.289436+0000","last_became_active":"2024-04-16T17:03:24.973568+0000","last_became_peered":"2024-04-16T17:03:24.973568+0000","last_unstale":"2024-04-16T17:05:22.289436+0000","last_undegraded":"2024-04-16T17:05:22.289436+0000","last_fullsized":"2024-04-16T17:05:22.289436+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:30:49.325469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"92'7","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.509052+0000","last_change":"2024-04-16T17:03:24.981629+0000","last_active":"2024-04-16T17:05:22.509052+0000","last_peered":"2024-04-16T17:05:22.509052+0000","last_clean":"2024-04-16T17:05:22.509052+0000","last_became_active":"2024-04-16T17:03:24.981111+0000","last_became_peered":"2024-04-16T17:03:24.981111+0000","last_unstale":"2024-04-16T17:05:22.509052+0000","last_undegraded":"2024-04-16T17:05:22.509052+0000","last_fullsized":"2024-04-16T17:05:22.509052+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:41:29.641610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"92'7","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.508949+0000","last_change":"2024-04-16T17:03:24.977890+0000","last_active":"2024-04-16T17:05:22.508949+0000","last_peered":"2024-04-16T17:05:22.508949+0000","last_clean":"2024-04-16T17:05:22.508949+0000","last_became_active":"2024-04-16T17:03:24.976500+0000","last_became_peered":"2024-04-16T17:03:24.976500+0000","last_unstale":"2024-04-16T17:05:22.508949+0000","last_undegraded":"2024-04-16T17:05:22.508949+0000","last_fullsized":"2024-04-16T17:05:22.508949+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:17:51.604701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"92'9","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.509773+0000","last_change":"2024-04-16T17:03:24.976085+0000","last_active":"2024-04-16T17:05:22.509773+0000","last_peered":"2024-04-16T17:05:22.509773+0000","last_clean":"2024-04-16T17:05:22.509773+0000","last_became_active":"2024-04-16T17:03:24.975804+0000","last_became_peered":"2024-04-16T17:03:24.975804+0000","last_unstale":"2024-04-16T17:05:22.509773+0000","last_undegraded":"2024-04-16T17:05:22.509773+0000","last_fullsized":"2024-04-16T17:05:22.509773+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:19:13.706889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"92'8","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.283256+0000","last_change":"2024-04-16T17:03:24.989456+0000","last_active":"2024-04-16T17:05:22.283256+0000","last_peered":"2024-04-16T17:05:22.283256+0000","last_clean":"2024-04-16T17:05:22.283256+0000","last_became_active":"2024-04-16T17:03:24.989161+0000","last_became_peered":"2024-04-16T17:03:24.989161+0000","last_unstale":"2024-04-16T17:05:22.283256+0000","last_undegraded":"2024-04-16T17:05:22.283256+0000","last_fullsized":"2024-04-16T17:05:22.283256+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:52:59.333625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"92'10","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.287957+0000","last_change":"2024-04-16T17:03:24.963428+0000","last_active":"2024-04-16T17:05:22.287957+0000","last_peered":"2024-04-16T17:05:22.287957+0000","last_clean":"2024-04-16T17:05:22.287957+0000","last_became_active":"2024-04-16T17:03:24.962523+0000","last_became_peered":"2024-04-16T17:03:24.962523+0000","last_unstale":"2024-04-16T17:05:22.287957+0000","last_undegraded":"2024-04-16T17:05:22.287957+0000","last_fullsized":"2024-04-16T17:05:22.287957+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:22:19.523901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"92'7","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.509759+0000","last_change":"2024-04-16T17:03:24.981122+0000","last_active":"2024-04-16T17:05:22.509759+0000","last_peered":"2024-04-16T17:05:22.509759+0000","last_clean":"2024-04-16T17:05:22.509759+0000","last_became_active":"2024-04-16T17:03:24.980533+0000","last_became_peered":"2024-04-16T17:03:24.980533+0000","last_unstale":"2024-04-16T17:05:22.509759+0000","last_undegraded":"2024-04-16T17:05:22.509759+0000","last_fullsized":"2024-04-16T17:05:22.509759+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:51:08.580965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.509563+0000","last_change":"2024-04-16T17:03:29.257239+0000","last_active":"2024-04-16T17:05:22.509563+0000","last_peered":"2024-04-16T17:05:22.509563+0000","last_clean":"2024-04-16T17:05:22.509563+0000","last_became_active":"2024-04-16T17:03:29.256827+0000","last_became_peered":"2024-04-16T17:03:29.256827+0000","last_unstale":"2024-04-16T17:05:22.509563+0000","last_undegraded":"2024-04-16T17:05:22.509563+0000","last_fullsized":"2024-04-16T17:05:22.509563+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:57:28.373386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"90'9","reported_seq":58,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.287155+0000","last_change":"2024-04-16T17:04:21.896665+0000","last_active":"2024-04-16T17:05:22.287155+0000","last_peered":"2024-04-16T17:05:22.287155+0000","last_clean":"2024-04-16T17:05:22.287155+0000","last_became_active":"2024-04-16T17:04:21.882573+0000","last_became_peered":"2024-04-16T17:04:21.882573+0000","last_unstale":"2024-04-16T17:05:22.287155+0000","last_undegraded":"2024-04-16T17:05:22.287155+0000","last_fullsized":"2024-04-16T17:05:22.287155+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:37:07.210368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,1],"acting":[9,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"92'5","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.516852+0000","last_change":"2024-04-16T17:03:24.994921+0000","last_active":"2024-04-16T17:05:22.516852+0000","last_peered":"2024-04-16T17:05:22.516852+0000","last_clean":"2024-04-16T17:05:22.516852+0000","last_became_active":"2024-04-16T17:03:24.994412+0000","last_became_peered":"2024-04-16T17:03:24.994412+0000","last_unstale":"2024-04-16T17:05:22.516852+0000","last_undegraded":"2024-04-16T17:05:22.516852+0000","last_fullsized":"2024-04-16T17:05:22.516852+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:11:40.748198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.511978+0000","last_change":"2024-04-16T17:03:29.259884+0000","last_active":"2024-04-16T17:05:22.511978+0000","last_peered":"2024-04-16T17:05:22.511978+0000","last_clean":"2024-04-16T17:05:22.511978+0000","last_became_active":"2024-04-16T17:03:29.259319+0000","last_became_peered":"2024-04-16T17:03:29.259319+0000","last_unstale":"2024-04-16T17:05:22.511978+0000","last_undegraded":"2024-04-16T17:05:22.511978+0000","last_fullsized":"2024-04-16T17:05:22.511978+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:05:18.244682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"90'2","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.519175+0000","last_change":"2024-04-16T17:03:24.179308+0000","last_active":"2024-04-16T17:05:22.519175+0000","last_peered":"2024-04-16T17:05:22.519175+0000","last_clean":"2024-04-16T17:05:22.519175+0000","last_became_active":"2024-04-16T17:03:24.178894+0000","last_became_peered":"2024-04-16T17:03:24.178894+0000","last_unstale":"2024-04-16T17:05:22.519175+0000","last_undegraded":"2024-04-16T17:05:22.519175+0000","last_fullsized":"2024-04-16T17:05:22.519175+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:17:46.831871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"92'6","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.511197+0000","last_change":"2024-04-16T17:03:24.982391+0000","last_active":"2024-04-16T17:05:22.511197+0000","last_peered":"2024-04-16T17:05:22.511197+0000","last_clean":"2024-04-16T17:05:22.511197+0000","last_became_active":"2024-04-16T17:03:24.981761+0000","last_became_peered":"2024-04-16T17:03:24.981761+0000","last_unstale":"2024-04-16T17:05:22.511197+0000","last_undegraded":"2024-04-16T17:05:22.511197+0000","last_fullsized":"2024-04-16T17:05:22.511197+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:40:57.657170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.512324+0000","last_change":"2024-04-16T17:03:29.259705+0000","last_active":"2024-04-16T17:05:22.512324+0000","last_peered":"2024-04-16T17:05:22.512324+0000","last_clean":"2024-04-16T17:05:22.512324+0000","last_became_active":"2024-04-16T17:03:29.259336+0000","last_became_peered":"2024-04-16T17:03:29.259336+0000","last_unstale":"2024-04-16T17:05:22.512324+0000","last_undegraded":"2024-04-16T17:05:22.512324+0000","last_fullsized":"2024-04-16T17:05:22.512324+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:53:29.042933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"90'4","reported_seq":47,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.285245+0000","last_change":"2024-04-16T17:04:22.153413+0000","last_active":"2024-04-16T17:05:22.285245+0000","last_peered":"2024-04-16T17:05:22.285245+0000","last_clean":"2024-04-16T17:05:22.285245+0000","last_became_active":"2024-04-16T17:04:22.148972+0000","last_became_peered":"2024-04-16T17:04:22.148972+0000","last_unstale":"2024-04-16T17:05:22.285245+0000","last_undegraded":"2024-04-16T17:05:22.285245+0000","last_fullsized":"2024-04-16T17:05:22.285245+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:05:57.513052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"92'9","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.510097+0000","last_change":"2024-04-16T17:03:24.973205+0000","last_active":"2024-04-16T17:05:22.510097+0000","last_peered":"2024-04-16T17:05:22.510097+0000","last_clean":"2024-04-16T17:05:22.510097+0000","last_became_active":"2024-04-16T17:03:24.972863+0000","last_became_peered":"2024-04-16T17:03:24.972863+0000","last_unstale":"2024-04-16T17:05:22.510097+0000","last_undegraded":"2024-04-16T17:05:22.510097+0000","last_fullsized":"2024-04-16T17:05:22.510097+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:28:29.005783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.512076+0000","last_change":"2024-04-16T17:03:29.262930+0000","last_active":"2024-04-16T17:05:22.512076+0000","last_peered":"2024-04-16T17:05:22.512076+0000","last_clean":"2024-04-16T17:05:22.512076+0000","last_became_active":"2024-04-16T17:03:29.262621+0000","last_became_peered":"2024-04-16T17:03:29.262621+0000","last_unstale":"2024-04-16T17:05:22.512076+0000","last_undegraded":"2024-04-16T17:05:22.512076+0000","last_fullsized":"2024-04-16T17:05:22.512076+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:22:31.555143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.285074+0000","last_change":"2024-04-16T17:03:24.150503+0000","last_active":"2024-04-16T17:05:22.285074+0000","last_peered":"2024-04-16T17:05:22.285074+0000","last_clean":"2024-04-16T17:05:22.285074+0000","last_became_active":"2024-04-16T17:03:24.150021+0000","last_became_peered":"2024-04-16T17:03:24.150021+0000","last_unstale":"2024-04-16T17:05:22.285074+0000","last_undegraded":"2024-04-16T17:05:22.285074+0000","last_fullsized":"2024-04-16T17:05:22.285074+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:21:36.119497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"92'15","reported_seq":44,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.513017+0000","last_change":"2024-04-16T17:03:24.965049+0000","last_active":"2024-04-16T17:05:22.513017+0000","last_peered":"2024-04-16T17:05:22.513017+0000","last_clean":"2024-04-16T17:05:22.513017+0000","last_became_active":"2024-04-16T17:03:24.963120+0000","last_became_peered":"2024-04-16T17:03:24.963120+0000","last_unstale":"2024-04-16T17:05:22.513017+0000","last_undegraded":"2024-04-16T17:05:22.513017+0000","last_fullsized":"2024-04-16T17:05:22.513017+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:17:49.725113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.509577+0000","last_change":"2024-04-16T17:03:29.247013+0000","last_active":"2024-04-16T17:05:22.509577+0000","last_peered":"2024-04-16T17:05:22.509577+0000","last_clean":"2024-04-16T17:05:22.509577+0000","last_became_active":"2024-04-16T17:03:29.246477+0000","last_became_peered":"2024-04-16T17:03:29.246477+0000","last_unstale":"2024-04-16T17:05:22.509577+0000","last_undegraded":"2024-04-16T17:05:22.509577+0000","last_fullsized":"2024-04-16T17:05:22.509577+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:52:19.525076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"92'7","reported_seq":144,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:33.532914+0000","last_change":"2024-04-16T17:03:24.155333+0000","last_active":"2024-04-16T17:05:33.532914+0000","last_peered":"2024-04-16T17:05:33.532914+0000","last_clean":"2024-04-16T17:05:33.532914+0000","last_became_active":"2024-04-16T17:03:24.154848+0000","last_became_peered":"2024-04-16T17:03:24.154848+0000","last_unstale":"2024-04-16T17:05:33.532914+0000","last_undegraded":"2024-04-16T17:05:33.532914+0000","last_fullsized":"2024-04-16T17:05:33.532914+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:03:34.769573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1154,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"92'7","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.285538+0000","last_change":"2024-04-16T17:03:24.967627+0000","last_active":"2024-04-16T17:05:22.285538+0000","last_peered":"2024-04-16T17:05:22.285538+0000","last_clean":"2024-04-16T17:05:22.285538+0000","last_became_active":"2024-04-16T17:03:24.967264+0000","last_became_peered":"2024-04-16T17:03:24.967264+0000","last_unstale":"2024-04-16T17:05:22.285538+0000","last_undegraded":"2024-04-16T17:05:22.285538+0000","last_fullsized":"2024-04-16T17:05:22.285538+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:35:29.182374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.509364+0000","last_change":"2024-04-16T17:03:29.264625+0000","last_active":"2024-04-16T17:05:22.509364+0000","last_peered":"2024-04-16T17:05:22.509364+0000","last_clean":"2024-04-16T17:05:22.509364+0000","last_became_active":"2024-04-16T17:03:29.264114+0000","last_became_peered":"2024-04-16T17:03:29.264114+0000","last_unstale":"2024-04-16T17:05:22.509364+0000","last_undegraded":"2024-04-16T17:05:22.509364+0000","last_fullsized":"2024-04-16T17:05:22.509364+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:14:49.449980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"90'3","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.282754+0000","last_change":"2024-04-16T17:03:24.160457+0000","last_active":"2024-04-16T17:05:22.282754+0000","last_peered":"2024-04-16T17:05:22.282754+0000","last_clean":"2024-04-16T17:05:22.282754+0000","last_became_active":"2024-04-16T17:03:24.160195+0000","last_became_peered":"2024-04-16T17:03:24.160195+0000","last_unstale":"2024-04-16T17:05:22.282754+0000","last_undegraded":"2024-04-16T17:05:22.282754+0000","last_fullsized":"2024-04-16T17:05:22.282754+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:23:07.064539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"92'8","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.512907+0000","last_change":"2024-04-16T17:03:24.991249+0000","last_active":"2024-04-16T17:05:22.512907+0000","last_peered":"2024-04-16T17:05:22.512907+0000","last_clean":"2024-04-16T17:05:22.512907+0000","last_became_active":"2024-04-16T17:03:24.990927+0000","last_became_peered":"2024-04-16T17:03:24.990927+0000","last_unstale":"2024-04-16T17:05:22.512907+0000","last_undegraded":"2024-04-16T17:05:22.512907+0000","last_fullsized":"2024-04-16T17:05:22.512907+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:36:39.975616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.283496+0000","last_change":"2024-04-16T17:03:29.253704+0000","last_active":"2024-04-16T17:05:22.283496+0000","last_peered":"2024-04-16T17:05:22.283496+0000","last_clean":"2024-04-16T17:05:22.283496+0000","last_became_active":"2024-04-16T17:03:29.253334+0000","last_became_peered":"2024-04-16T17:03:29.253334+0000","last_unstale":"2024-04-16T17:05:22.283496+0000","last_undegraded":"2024-04-16T17:05:22.283496+0000","last_fullsized":"2024-04-16T17:05:22.283496+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:19:12.611731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"90'2","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.288474+0000","last_change":"2024-04-16T17:03:24.158875+0000","last_active":"2024-04-16T17:05:22.288474+0000","last_peered":"2024-04-16T17:05:22.288474+0000","last_clean":"2024-04-16T17:05:22.288474+0000","last_became_active":"2024-04-16T17:03:24.158465+0000","last_became_peered":"2024-04-16T17:03:24.158465+0000","last_unstale":"2024-04-16T17:05:22.288474+0000","last_undegraded":"2024-04-16T17:05:22.288474+0000","last_fullsized":"2024-04-16T17:05:22.288474+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:21:29.882251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"92'8","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.516011+0000","last_change":"2024-04-16T17:03:24.994476+0000","last_active":"2024-04-16T17:05:22.516011+0000","last_peered":"2024-04-16T17:05:22.516011+0000","last_clean":"2024-04-16T17:05:22.516011+0000","last_became_active":"2024-04-16T17:03:24.993532+0000","last_became_peered":"2024-04-16T17:03:24.993532+0000","last_unstale":"2024-04-16T17:05:22.516011+0000","last_undegraded":"2024-04-16T17:05:22.516011+0000","last_fullsized":"2024-04-16T17:05:22.516011+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:04:44.062444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.510439+0000","last_change":"2024-04-16T17:03:29.255446+0000","last_active":"2024-04-16T17:05:22.510439+0000","last_peered":"2024-04-16T17:05:22.510439+0000","last_clean":"2024-04-16T17:05:22.510439+0000","last_became_active":"2024-04-16T17:03:29.255051+0000","last_became_peered":"2024-04-16T17:03:29.255051+0000","last_unstale":"2024-04-16T17:05:22.510439+0000","last_undegraded":"2024-04-16T17:05:22.510439+0000","last_fullsized":"2024-04-16T17:05:22.510439+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:24:04.946342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"90'1","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.512386+0000","last_change":"2024-04-16T17:03:24.154904+0000","last_active":"2024-04-16T17:05:22.512386+0000","last_peered":"2024-04-16T17:05:22.512386+0000","last_clean":"2024-04-16T17:05:22.512386+0000","last_became_active":"2024-04-16T17:03:24.154617+0000","last_became_peered":"2024-04-16T17:03:24.154617+0000","last_unstale":"2024-04-16T17:05:22.512386+0000","last_undegraded":"2024-04-16T17:05:22.512386+0000","last_fullsized":"2024-04-16T17:05:22.512386+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:35:55.498855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"92'11","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.284812+0000","last_change":"2024-04-16T17:03:24.979908+0000","last_active":"2024-04-16T17:05:22.284812+0000","last_peered":"2024-04-16T17:05:22.284812+0000","last_clean":"2024-04-16T17:05:22.284812+0000","last_became_active":"2024-04-16T17:03:24.979501+0000","last_became_peered":"2024-04-16T17:03:24.979501+0000","last_unstale":"2024-04-16T17:05:22.284812+0000","last_undegraded":"2024-04-16T17:05:22.284812+0000","last_fullsized":"2024-04-16T17:05:22.284812+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:51:05.421459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.516308+0000","last_change":"2024-04-16T17:03:29.258809+0000","last_active":"2024-04-16T17:05:22.516308+0000","last_peered":"2024-04-16T17:05:22.516308+0000","last_clean":"2024-04-16T17:05:22.516308+0000","last_became_active":"2024-04-16T17:03:29.256629+0000","last_became_peered":"2024-04-16T17:03:29.256629+0000","last_unstale":"2024-04-16T17:05:22.516308+0000","last_undegraded":"2024-04-16T17:05:22.516308+0000","last_fullsized":"2024-04-16T17:05:22.516308+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:46:51.634944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"92'4","reported_seq":136,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:32.530909+0000","last_change":"2024-04-16T17:03:24.153774+0000","last_active":"2024-04-16T17:05:32.530909+0000","last_peered":"2024-04-16T17:05:32.530909+0000","last_clean":"2024-04-16T17:05:32.530909+0000","last_became_active":"2024-04-16T17:03:24.153418+0000","last_became_peered":"2024-04-16T17:03:24.153418+0000","last_unstale":"2024-04-16T17:05:32.530909+0000","last_undegraded":"2024-04-16T17:05:32.530909+0000","last_fullsized":"2024-04-16T17:05:32.530909+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:58:36.093789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":101,"num_read_kb":101,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"92'10","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.511791+0000","last_change":"2024-04-16T17:03:24.962838+0000","last_active":"2024-04-16T17:05:22.511791+0000","last_peered":"2024-04-16T17:05:22.511791+0000","last_clean":"2024-04-16T17:05:22.511791+0000","last_became_active":"2024-04-16T17:03:24.960163+0000","last_became_peered":"2024-04-16T17:03:24.960163+0000","last_unstale":"2024-04-16T17:05:22.511791+0000","last_undegraded":"2024-04-16T17:05:22.511791+0000","last_fullsized":"2024-04-16T17:05:22.511791+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:48:18.723499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.284550+0000","last_change":"2024-04-16T17:03:29.251177+0000","last_active":"2024-04-16T17:05:22.284550+0000","last_peered":"2024-04-16T17:05:22.284550+0000","last_clean":"2024-04-16T17:05:22.284550+0000","last_became_active":"2024-04-16T17:03:29.250657+0000","last_became_peered":"2024-04-16T17:03:29.250657+0000","last_unstale":"2024-04-16T17:05:22.284550+0000","last_undegraded":"2024-04-16T17:05:22.284550+0000","last_fullsized":"2024-04-16T17:05:22.284550+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:16:47.217193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"90'2","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.510573+0000","last_change":"2024-04-16T17:03:24.154245+0000","last_active":"2024-04-16T17:05:22.510573+0000","last_peered":"2024-04-16T17:05:22.510573+0000","last_clean":"2024-04-16T17:05:22.510573+0000","last_became_active":"2024-04-16T17:03:24.153297+0000","last_became_peered":"2024-04-16T17:03:24.153297+0000","last_unstale":"2024-04-16T17:05:22.510573+0000","last_undegraded":"2024-04-16T17:05:22.510573+0000","last_fullsized":"2024-04-16T17:05:22.510573+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:23:49.490238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"92'9","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.511056+0000","last_change":"2024-04-16T17:03:24.984572+0000","last_active":"2024-04-16T17:05:22.511056+0000","last_peered":"2024-04-16T17:05:22.511056+0000","last_clean":"2024-04-16T17:05:22.511056+0000","last_became_active":"2024-04-16T17:03:24.983588+0000","last_became_peered":"2024-04-16T17:03:24.983588+0000","last_unstale":"2024-04-16T17:05:22.511056+0000","last_undegraded":"2024-04-16T17:05:22.511056+0000","last_fullsized":"2024-04-16T17:05:22.511056+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:41:24.704602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.512602+0000","last_change":"2024-04-16T17:03:29.245129+0000","last_active":"2024-04-16T17:05:22.512602+0000","last_peered":"2024-04-16T17:05:22.512602+0000","last_clean":"2024-04-16T17:05:22.512602+0000","last_became_active":"2024-04-16T17:03:29.244720+0000","last_became_peered":"2024-04-16T17:03:29.244720+0000","last_unstale":"2024-04-16T17:05:22.512602+0000","last_undegraded":"2024-04-16T17:05:22.512602+0000","last_fullsized":"2024-04-16T17:05:22.512602+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:55:26.612176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"90'4","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.512809+0000","last_change":"2024-04-16T17:03:24.165248+0000","last_active":"2024-04-16T17:05:22.512809+0000","last_peered":"2024-04-16T17:05:22.512809+0000","last_clean":"2024-04-16T17:05:22.512809+0000","last_became_active":"2024-04-16T17:03:24.164915+0000","last_became_peered":"2024-04-16T17:03:24.164915+0000","last_unstale":"2024-04-16T17:05:22.512809+0000","last_undegraded":"2024-04-16T17:05:22.512809+0000","last_fullsized":"2024-04-16T17:05:22.512809+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:04:47.207554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"92'11","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.510492+0000","last_change":"2024-04-16T17:03:24.967292+0000","last_active":"2024-04-16T17:05:22.510492+0000","last_peered":"2024-04-16T17:05:22.510492+0000","last_clean":"2024-04-16T17:05:22.510492+0000","last_became_active":"2024-04-16T17:03:24.966952+0000","last_became_peered":"2024-04-16T17:03:24.966952+0000","last_unstale":"2024-04-16T17:05:22.510492+0000","last_undegraded":"2024-04-16T17:05:22.510492+0000","last_fullsized":"2024-04-16T17:05:22.510492+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:42:54.969478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.285084+0000","last_change":"2024-04-16T17:03:29.257836+0000","last_active":"2024-04-16T17:05:22.285084+0000","last_peered":"2024-04-16T17:05:22.285084+0000","last_clean":"2024-04-16T17:05:22.285084+0000","last_became_active":"2024-04-16T17:03:29.257409+0000","last_became_peered":"2024-04-16T17:03:29.257409+0000","last_unstale":"2024-04-16T17:05:22.285084+0000","last_undegraded":"2024-04-16T17:05:22.285084+0000","last_fullsized":"2024-04-16T17:05:22.285084+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:25:21.288266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"92'36","reported_seq":279,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:33.532111+0000","last_change":"2024-04-16T17:03:24.150943+0000","last_active":"2024-04-16T17:05:33.532111+0000","last_peered":"2024-04-16T17:05:33.532111+0000","last_clean":"2024-04-16T17:05:33.532111+0000","last_became_active":"2024-04-16T17:03:24.150563+0000","last_became_peered":"2024-04-16T17:03:24.150563+0000","last_unstale":"2024-04-16T17:05:33.532111+0000","last_undegraded":"2024-04-16T17:05:33.532111+0000","last_fullsized":"2024-04-16T17:05:33.532111+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:05:07.380440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":63605,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":212,"num_read_kb":169,"num_write":37,"num_write_kb":80,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"92'6","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.285005+0000","last_change":"2024-04-16T17:03:24.966393+0000","last_active":"2024-04-16T17:05:22.285005+0000","last_peered":"2024-04-16T17:05:22.285005+0000","last_clean":"2024-04-16T17:05:22.285005+0000","last_became_active":"2024-04-16T17:03:24.966016+0000","last_became_peered":"2024-04-16T17:03:24.966016+0000","last_unstale":"2024-04-16T17:05:22.285005+0000","last_undegraded":"2024-04-16T17:05:22.285005+0000","last_fullsized":"2024-04-16T17:05:22.285005+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:30:25.203262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.513291+0000","last_change":"2024-04-16T17:03:29.265371+0000","last_active":"2024-04-16T17:05:22.513291+0000","last_peered":"2024-04-16T17:05:22.513291+0000","last_clean":"2024-04-16T17:05:22.513291+0000","last_became_active":"2024-04-16T17:03:29.264933+0000","last_became_peered":"2024-04-16T17:03:29.264933+0000","last_unstale":"2024-04-16T17:05:22.513291+0000","last_undegraded":"2024-04-16T17:05:22.513291+0000","last_fullsized":"2024-04-16T17:05:22.513291+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:50:18.203015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"90'6","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.282793+0000","last_change":"2024-04-16T17:04:21.526796+0000","last_active":"2024-04-16T17:05:22.282793+0000","last_peered":"2024-04-16T17:05:22.282793+0000","last_clean":"2024-04-16T17:05:22.282793+0000","last_became_active":"2024-04-16T17:04:21.508364+0000","last_became_peered":"2024-04-16T17:04:21.508364+0000","last_unstale":"2024-04-16T17:05:22.282793+0000","last_undegraded":"2024-04-16T17:05:22.282793+0000","last_fullsized":"2024-04-16T17:05:22.282793+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:45:42.726648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"92'9","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.289036+0000","last_change":"2024-04-16T17:03:24.986228+0000","last_active":"2024-04-16T17:05:22.289036+0000","last_peered":"2024-04-16T17:05:22.289036+0000","last_clean":"2024-04-16T17:05:22.289036+0000","last_became_active":"2024-04-16T17:03:24.985916+0000","last_became_peered":"2024-04-16T17:03:24.985916+0000","last_unstale":"2024-04-16T17:05:22.289036+0000","last_undegraded":"2024-04-16T17:05:22.289036+0000","last_fullsized":"2024-04-16T17:05:22.289036+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:23:54.309203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.282938+0000","last_change":"2024-04-16T17:03:29.264636+0000","last_active":"2024-04-16T17:05:22.282938+0000","last_peered":"2024-04-16T17:05:22.282938+0000","last_clean":"2024-04-16T17:05:22.282938+0000","last_became_active":"2024-04-16T17:03:29.264117+0000","last_became_peered":"2024-04-16T17:03:29.264117+0000","last_unstale":"2024-04-16T17:05:22.282938+0000","last_undegraded":"2024-04-16T17:05:22.282938+0000","last_fullsized":"2024-04-16T17:05:22.282938+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:42:36.123154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"90'3","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.512886+0000","last_change":"2024-04-16T17:03:24.166844+0000","last_active":"2024-04-16T17:05:22.512886+0000","last_peered":"2024-04-16T17:05:22.512886+0000","last_clean":"2024-04-16T17:05:22.512886+0000","last_became_active":"2024-04-16T17:03:24.166479+0000","last_became_peered":"2024-04-16T17:03:24.166479+0000","last_unstale":"2024-04-16T17:05:22.512886+0000","last_undegraded":"2024-04-16T17:05:22.512886+0000","last_fullsized":"2024-04-16T17:05:22.512886+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:02:20.139865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"92'15","reported_seq":44,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.510958+0000","last_change":"2024-04-16T17:03:24.973057+0000","last_active":"2024-04-16T17:05:22.510958+0000","last_peered":"2024-04-16T17:05:22.510958+0000","last_clean":"2024-04-16T17:05:22.510958+0000","last_became_active":"2024-04-16T17:03:24.972731+0000","last_became_peered":"2024-04-16T17:03:24.972731+0000","last_unstale":"2024-04-16T17:05:22.510958+0000","last_undegraded":"2024-04-16T17:05:22.510958+0000","last_fullsized":"2024-04-16T17:05:22.510958+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:37:38.702919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.286263+0000","last_change":"2024-04-16T17:03:29.259167+0000","last_active":"2024-04-16T17:05:22.286263+0000","last_peered":"2024-04-16T17:05:22.286263+0000","last_clean":"2024-04-16T17:05:22.286263+0000","last_became_active":"2024-04-16T17:03:29.258737+0000","last_became_peered":"2024-04-16T17:03:29.258737+0000","last_unstale":"2024-04-16T17:05:22.286263+0000","last_undegraded":"2024-04-16T17:05:22.286263+0000","last_fullsized":"2024-04-16T17:05:22.286263+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:01:13.774037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"92'5","reported_seq":138,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:29.613818+0000","last_change":"2024-04-16T17:03:24.151064+0000","last_active":"2024-04-16T17:05:29.613818+0000","last_peered":"2024-04-16T17:05:29.613818+0000","last_clean":"2024-04-16T17:05:29.613818+0000","last_became_active":"2024-04-16T17:03:24.150745+0000","last_became_peered":"2024-04-16T17:03:24.150745+0000","last_unstale":"2024-04-16T17:05:29.613818+0000","last_undegraded":"2024-04-16T17:05:29.613818+0000","last_fullsized":"2024-04-16T17:05:29.613818+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:32:26.271008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":103,"num_read_kb":5,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"92'6","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.510687+0000","last_change":"2024-04-16T17:03:24.971948+0000","last_active":"2024-04-16T17:05:22.510687+0000","last_peered":"2024-04-16T17:05:22.510687+0000","last_clean":"2024-04-16T17:05:22.510687+0000","last_became_active":"2024-04-16T17:03:24.971530+0000","last_became_peered":"2024-04-16T17:03:24.971530+0000","last_unstale":"2024-04-16T17:05:22.510687+0000","last_undegraded":"2024-04-16T17:05:22.510687+0000","last_fullsized":"2024-04-16T17:05:22.510687+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:03:29.286160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.290269+0000","last_change":"2024-04-16T17:03:29.264821+0000","last_active":"2024-04-16T17:05:22.290269+0000","last_peered":"2024-04-16T17:05:22.290269+0000","last_clean":"2024-04-16T17:05:22.290269+0000","last_became_active":"2024-04-16T17:03:29.264287+0000","last_became_peered":"2024-04-16T17:03:29.264287+0000","last_unstale":"2024-04-16T17:05:22.290269+0000","last_undegraded":"2024-04-16T17:05:22.290269+0000","last_fullsized":"2024-04-16T17:05:22.290269+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:33:28.400663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"90'15","reported_seq":47,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.284636+0000","last_change":"2024-04-16T17:03:24.158844+0000","last_active":"2024-04-16T17:05:22.284636+0000","last_peered":"2024-04-16T17:05:22.284636+0000","last_clean":"2024-04-16T17:05:22.284636+0000","last_became_active":"2024-04-16T17:03:24.158537+0000","last_became_peered":"2024-04-16T17:03:24.158537+0000","last_unstale":"2024-04-16T17:05:22.284636+0000","last_undegraded":"2024-04-16T17:05:22.284636+0000","last_fullsized":"2024-04-16T17:05:22.284636+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:42:22.772452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"92'6","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.512660+0000","last_change":"2024-04-16T17:03:24.962522+0000","last_active":"2024-04-16T17:05:22.512660+0000","last_peered":"2024-04-16T17:05:22.512660+0000","last_clean":"2024-04-16T17:05:22.512660+0000","last_became_active":"2024-04-16T17:03:24.962169+0000","last_became_peered":"2024-04-16T17:03:24.962169+0000","last_unstale":"2024-04-16T17:05:22.512660+0000","last_undegraded":"2024-04-16T17:05:22.512660+0000","last_fullsized":"2024-04-16T17:05:22.512660+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:28:17.147491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.512624+0000","last_change":"2024-04-16T17:03:29.259539+0000","last_active":"2024-04-16T17:05:22.512624+0000","last_peered":"2024-04-16T17:05:22.512624+0000","last_clean":"2024-04-16T17:05:22.512624+0000","last_became_active":"2024-04-16T17:03:29.259123+0000","last_became_peered":"2024-04-16T17:03:29.259123+0000","last_unstale":"2024-04-16T17:05:22.512624+0000","last_undegraded":"2024-04-16T17:05:22.512624+0000","last_fullsized":"2024-04-16T17:05:22.512624+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:15:17.142636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"90'2","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.285245+0000","last_change":"2024-04-16T17:03:24.147526+0000","last_active":"2024-04-16T17:05:22.285245+0000","last_peered":"2024-04-16T17:05:22.285245+0000","last_clean":"2024-04-16T17:05:22.285245+0000","last_became_active":"2024-04-16T17:03:24.147119+0000","last_became_peered":"2024-04-16T17:03:24.147119+0000","last_unstale":"2024-04-16T17:05:22.285245+0000","last_undegraded":"2024-04-16T17:05:22.285245+0000","last_fullsized":"2024-04-16T17:05:22.285245+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:30:38.742122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"92'5","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.283051+0000","last_change":"2024-04-16T17:04:21.507113+0000","last_active":"2024-04-16T17:05:22.283051+0000","last_peered":"2024-04-16T17:05:22.283051+0000","last_clean":"2024-04-16T17:05:22.283051+0000","last_became_active":"2024-04-16T17:04:21.506680+0000","last_became_peered":"2024-04-16T17:04:21.506680+0000","last_unstale":"2024-04-16T17:05:22.283051+0000","last_undegraded":"2024-04-16T17:05:22.283051+0000","last_fullsized":"2024-04-16T17:05:22.283051+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:08:47.971317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,8],"acting":[11,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.511966+0000","last_change":"2024-04-16T17:05:22.511966+0000","last_active":"2024-04-16T17:05:22.511966+0000","last_peered":"2024-04-16T17:05:22.511966+0000","last_clean":"2024-04-16T17:05:22.511966+0000","last_became_active":"2024-04-16T17:05:22.511512+0000","last_became_peered":"2024-04-16T17:05:22.511512+0000","last_unstale":"2024-04-16T17:05:22.511966+0000","last_undegraded":"2024-04-16T17:05:22.511966+0000","last_fullsized":"2024-04-16T17:05:22.511966+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:14:41.291499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,1],"acting":[8,10,11,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.285749+0000","last_change":"2024-04-16T17:03:24.162105+0000","last_active":"2024-04-16T17:05:22.285749+0000","last_peered":"2024-04-16T17:05:22.285749+0000","last_clean":"2024-04-16T17:05:22.285749+0000","last_became_active":"2024-04-16T17:03:24.161631+0000","last_became_peered":"2024-04-16T17:03:24.161631+0000","last_unstale":"2024-04-16T17:05:22.285749+0000","last_undegraded":"2024-04-16T17:05:22.285749+0000","last_fullsized":"2024-04-16T17:05:22.285749+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:22:33.677848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"92'4","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.283327+0000","last_change":"2024-04-16T17:03:24.978253+0000","last_active":"2024-04-16T17:05:22.283327+0000","last_peered":"2024-04-16T17:05:22.283327+0000","last_clean":"2024-04-16T17:05:22.283327+0000","last_became_active":"2024-04-16T17:03:24.977956+0000","last_became_peered":"2024-04-16T17:03:24.977956+0000","last_unstale":"2024-04-16T17:05:22.283327+0000","last_undegraded":"2024-04-16T17:05:22.283327+0000","last_fullsized":"2024-04-16T17:05:22.283327+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:59:45.102383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.289715+0000","last_change":"2024-04-16T17:03:29.265312+0000","last_active":"2024-04-16T17:05:22.289715+0000","last_peered":"2024-04-16T17:05:22.289715+0000","last_clean":"2024-04-16T17:05:22.289715+0000","last_became_active":"2024-04-16T17:03:29.264891+0000","last_became_peered":"2024-04-16T17:03:29.264891+0000","last_unstale":"2024-04-16T17:05:22.289715+0000","last_undegraded":"2024-04-16T17:05:22.289715+0000","last_fullsized":"2024-04-16T17:05:22.289715+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:26:40.088127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"90'1","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.510989+0000","last_change":"2024-04-16T17:03:24.170935+0000","last_active":"2024-04-16T17:05:22.510989+0000","last_peered":"2024-04-16T17:05:22.510989+0000","last_clean":"2024-04-16T17:05:22.510989+0000","last_became_active":"2024-04-16T17:03:24.170609+0000","last_became_peered":"2024-04-16T17:03:24.170609+0000","last_unstale":"2024-04-16T17:05:22.510989+0000","last_undegraded":"2024-04-16T17:05:22.510989+0000","last_fullsized":"2024-04-16T17:05:22.510989+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:18:33.516309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"92'7","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.283209+0000","last_change":"2024-04-16T17:03:24.985434+0000","last_active":"2024-04-16T17:05:22.283209+0000","last_peered":"2024-04-16T17:05:22.283209+0000","last_clean":"2024-04-16T17:05:22.283209+0000","last_became_active":"2024-04-16T17:03:24.984388+0000","last_became_peered":"2024-04-16T17:03:24.984388+0000","last_unstale":"2024-04-16T17:05:22.283209+0000","last_undegraded":"2024-04-16T17:05:22.283209+0000","last_fullsized":"2024-04-16T17:05:22.283209+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:43:26.355572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.516267+0000","last_change":"2024-04-16T17:05:22.516267+0000","last_active":"2024-04-16T17:05:22.516267+0000","last_peered":"2024-04-16T17:05:22.516267+0000","last_clean":"2024-04-16T17:05:22.516267+0000","last_became_active":"2024-04-16T17:05:22.513637+0000","last_became_peered":"2024-04-16T17:05:22.513637+0000","last_unstale":"2024-04-16T17:05:22.516267+0000","last_undegraded":"2024-04-16T17:05:22.516267+0000","last_fullsized":"2024-04-16T17:05:22.516267+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:39:47.975735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,8,10,0],"acting":[6,8,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"92'8","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.512569+0000","last_change":"2024-04-16T17:03:24.962786+0000","last_active":"2024-04-16T17:05:22.512569+0000","last_peered":"2024-04-16T17:05:22.512569+0000","last_clean":"2024-04-16T17:05:22.512569+0000","last_became_active":"2024-04-16T17:03:24.961704+0000","last_became_peered":"2024-04-16T17:03:24.961704+0000","last_unstale":"2024-04-16T17:05:22.512569+0000","last_undegraded":"2024-04-16T17:05:22.512569+0000","last_fullsized":"2024-04-16T17:05:22.512569+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:22:27.957607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"92'5","reported_seq":140,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:34.624930+0000","last_change":"2024-04-16T17:03:24.175617+0000","last_active":"2024-04-16T17:05:34.624930+0000","last_peered":"2024-04-16T17:05:34.624930+0000","last_clean":"2024-04-16T17:05:34.624930+0000","last_became_active":"2024-04-16T17:03:24.175272+0000","last_became_peered":"2024-04-16T17:03:24.175272+0000","last_unstale":"2024-04-16T17:05:34.624930+0000","last_undegraded":"2024-04-16T17:05:34.624930+0000","last_fullsized":"2024-04-16T17:05:34.624930+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:27:44.738030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":103,"num_read_kb":103,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"90'1","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.511030+0000","last_change":"2024-04-16T17:04:21.512033+0000","last_active":"2024-04-16T17:05:22.511030+0000","last_peered":"2024-04-16T17:05:22.511030+0000","last_clean":"2024-04-16T17:05:22.511030+0000","last_became_active":"2024-04-16T17:04:21.506718+0000","last_became_peered":"2024-04-16T17:04:21.506718+0000","last_unstale":"2024-04-16T17:05:22.511030+0000","last_undegraded":"2024-04-16T17:05:22.511030+0000","last_fullsized":"2024-04-16T17:05:22.511030+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:53:48.953887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,10],"acting":[7,5,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"92'8","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.517270+0000","last_change":"2024-04-16T17:03:24.983629+0000","last_active":"2024-04-16T17:05:22.517270+0000","last_peered":"2024-04-16T17:05:22.517270+0000","last_clean":"2024-04-16T17:05:22.517270+0000","last_became_active":"2024-04-16T17:03:24.979969+0000","last_became_peered":"2024-04-16T17:03:24.979969+0000","last_unstale":"2024-04-16T17:05:22.517270+0000","last_undegraded":"2024-04-16T17:05:22.517270+0000","last_fullsized":"2024-04-16T17:05:22.517270+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:01:24.391732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"90'3","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.512560+0000","last_change":"2024-04-16T17:03:24.152475+0000","last_active":"2024-04-16T17:05:22.512560+0000","last_peered":"2024-04-16T17:05:22.512560+0000","last_clean":"2024-04-16T17:05:22.512560+0000","last_became_active":"2024-04-16T17:03:24.152198+0000","last_became_peered":"2024-04-16T17:03:24.152198+0000","last_unstale":"2024-04-16T17:05:22.512560+0000","last_undegraded":"2024-04-16T17:05:22.512560+0000","last_fullsized":"2024-04-16T17:05:22.512560+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:49:33.914325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":364,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.510449+0000","last_change":"2024-04-16T17:02:02.111335+0000","last_active":"2024-04-16T17:05:22.510449+0000","last_peered":"2024-04-16T17:05:22.510449+0000","last_clean":"2024-04-16T17:05:22.510449+0000","last_became_active":"2024-04-16T17:02:02.110928+0000","last_became_peered":"2024-04-16T17:02:02.110928+0000","last_unstale":"2024-04-16T17:05:22.510449+0000","last_undegraded":"2024-04-16T17:05:22.510449+0000","last_fullsized":"2024-04-16T17:05:22.510449+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T16:58:07.081753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T16:58:07.081753+0000","last_clean_scrub_stamp":"2024-04-16T16:58:07.081753+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:27:02.865664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.513186+0000","last_change":"2024-04-16T17:03:29.301691+0000","last_active":"2024-04-16T17:05:22.513186+0000","last_peered":"2024-04-16T17:05:22.513186+0000","last_clean":"2024-04-16T17:05:22.513186+0000","last_became_active":"2024-04-16T17:03:29.301277+0000","last_became_peered":"2024-04-16T17:03:29.301277+0000","last_unstale":"2024-04-16T17:05:22.513186+0000","last_undegraded":"2024-04-16T17:05:22.513186+0000","last_fullsized":"2024-04-16T17:05:22.513186+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:45:16.727610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"92'5","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.287991+0000","last_change":"2024-04-16T17:03:24.970939+0000","last_active":"2024-04-16T17:05:22.287991+0000","last_peered":"2024-04-16T17:05:22.287991+0000","last_clean":"2024-04-16T17:05:22.287991+0000","last_became_active":"2024-04-16T17:03:24.970581+0000","last_became_peered":"2024-04-16T17:03:24.970581+0000","last_unstale":"2024-04-16T17:05:22.287991+0000","last_undegraded":"2024-04-16T17:05:22.287991+0000","last_fullsized":"2024-04-16T17:05:22.287991+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:33:07.448597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"90'3","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.510631+0000","last_change":"2024-04-16T17:03:24.155454+0000","last_active":"2024-04-16T17:05:22.510631+0000","last_peered":"2024-04-16T17:05:22.510631+0000","last_clean":"2024-04-16T17:05:22.510631+0000","last_became_active":"2024-04-16T17:03:24.155155+0000","last_became_peered":"2024-04-16T17:03:24.155155+0000","last_unstale":"2024-04-16T17:05:22.510631+0000","last_undegraded":"2024-04-16T17:05:22.510631+0000","last_fullsized":"2024-04-16T17:05:22.510631+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:34:08.618002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.282951+0000","last_change":"2024-04-16T17:03:29.255538+0000","last_active":"2024-04-16T17:05:22.282951+0000","last_peered":"2024-04-16T17:05:22.282951+0000","last_clean":"2024-04-16T17:05:22.282951+0000","last_became_active":"2024-04-16T17:03:29.255107+0000","last_became_peered":"2024-04-16T17:03:29.255107+0000","last_unstale":"2024-04-16T17:05:22.282951+0000","last_undegraded":"2024-04-16T17:05:22.282951+0000","last_fullsized":"2024-04-16T17:05:22.282951+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:31:58.458641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"90'1","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.285190+0000","last_change":"2024-04-16T17:03:24.158816+0000","last_active":"2024-04-16T17:05:22.285190+0000","last_peered":"2024-04-16T17:05:22.285190+0000","last_clean":"2024-04-16T17:05:22.285190+0000","last_became_active":"2024-04-16T17:03:24.158518+0000","last_became_peered":"2024-04-16T17:03:24.158518+0000","last_unstale":"2024-04-16T17:05:22.285190+0000","last_undegraded":"2024-04-16T17:05:22.285190+0000","last_fullsized":"2024-04-16T17:05:22.285190+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:54:55.807975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"92'9","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.290107+0000","last_change":"2024-04-16T17:03:24.977190+0000","last_active":"2024-04-16T17:05:22.290107+0000","last_peered":"2024-04-16T17:05:22.290107+0000","last_clean":"2024-04-16T17:05:22.290107+0000","last_became_active":"2024-04-16T17:03:24.976840+0000","last_became_peered":"2024-04-16T17:03:24.976840+0000","last_unstale":"2024-04-16T17:05:22.290107+0000","last_undegraded":"2024-04-16T17:05:22.290107+0000","last_fullsized":"2024-04-16T17:05:22.290107+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:21:51.294150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.516089+0000","last_change":"2024-04-16T17:03:29.301423+0000","last_active":"2024-04-16T17:05:22.516089+0000","last_peered":"2024-04-16T17:05:22.516089+0000","last_clean":"2024-04-16T17:05:22.516089+0000","last_became_active":"2024-04-16T17:03:29.300888+0000","last_became_peered":"2024-04-16T17:03:29.300888+0000","last_unstale":"2024-04-16T17:05:22.516089+0000","last_undegraded":"2024-04-16T17:05:22.516089+0000","last_fullsized":"2024-04-16T17:05:22.516089+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:13:54.188090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"92'11","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.287648+0000","last_change":"2024-04-16T17:03:24.167643+0000","last_active":"2024-04-16T17:05:22.287648+0000","last_peered":"2024-04-16T17:05:22.287648+0000","last_clean":"2024-04-16T17:05:22.287648+0000","last_became_active":"2024-04-16T17:03:24.167103+0000","last_became_peered":"2024-04-16T17:03:24.167103+0000","last_unstale":"2024-04-16T17:05:22.287648+0000","last_undegraded":"2024-04-16T17:05:22.287648+0000","last_fullsized":"2024-04-16T17:05:22.287648+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:33:16.809371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"92'5","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.519200+0000","last_change":"2024-04-16T17:03:24.976981+0000","last_active":"2024-04-16T17:05:22.519200+0000","last_peered":"2024-04-16T17:05:22.519200+0000","last_clean":"2024-04-16T17:05:22.519200+0000","last_became_active":"2024-04-16T17:03:24.975385+0000","last_became_peered":"2024-04-16T17:03:24.975385+0000","last_unstale":"2024-04-16T17:05:22.519200+0000","last_undegraded":"2024-04-16T17:05:22.519200+0000","last_fullsized":"2024-04-16T17:05:22.519200+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:27:03.104838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.509345+0000","last_change":"2024-04-16T17:03:29.244767+0000","last_active":"2024-04-16T17:05:22.509345+0000","last_peered":"2024-04-16T17:05:22.509345+0000","last_clean":"2024-04-16T17:05:22.509345+0000","last_became_active":"2024-04-16T17:03:29.244205+0000","last_became_peered":"2024-04-16T17:03:29.244205+0000","last_unstale":"2024-04-16T17:05:22.509345+0000","last_undegraded":"2024-04-16T17:05:22.509345+0000","last_fullsized":"2024-04-16T17:05:22.509345+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:51:46.652387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"90'4","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.513012+0000","last_change":"2024-04-16T17:03:24.164916+0000","last_active":"2024-04-16T17:05:22.513012+0000","last_peered":"2024-04-16T17:05:22.513012+0000","last_clean":"2024-04-16T17:05:22.513012+0000","last_became_active":"2024-04-16T17:03:24.164648+0000","last_became_peered":"2024-04-16T17:03:24.164648+0000","last_unstale":"2024-04-16T17:05:22.513012+0000","last_undegraded":"2024-04-16T17:05:22.513012+0000","last_fullsized":"2024-04-16T17:05:22.513012+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:23:00.198785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"92'7","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.510688+0000","last_change":"2024-04-16T17:03:24.994393+0000","last_active":"2024-04-16T17:05:22.510688+0000","last_peered":"2024-04-16T17:05:22.510688+0000","last_clean":"2024-04-16T17:05:22.510688+0000","last_became_active":"2024-04-16T17:03:24.994002+0000","last_became_peered":"2024-04-16T17:03:24.994002+0000","last_unstale":"2024-04-16T17:05:22.510688+0000","last_undegraded":"2024-04-16T17:05:22.510688+0000","last_fullsized":"2024-04-16T17:05:22.510688+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:23:44.596781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.512808+0000","last_change":"2024-04-16T17:03:29.302009+0000","last_active":"2024-04-16T17:05:22.512808+0000","last_peered":"2024-04-16T17:05:22.512808+0000","last_clean":"2024-04-16T17:05:22.512808+0000","last_became_active":"2024-04-16T17:03:29.301183+0000","last_became_peered":"2024-04-16T17:03:29.301183+0000","last_unstale":"2024-04-16T17:05:22.512808+0000","last_undegraded":"2024-04-16T17:05:22.512808+0000","last_fullsized":"2024-04-16T17:05:22.512808+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:30:19.561875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"90'2","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.509857+0000","last_change":"2024-04-16T17:03:24.176829+0000","last_active":"2024-04-16T17:05:22.509857+0000","last_peered":"2024-04-16T17:05:22.509857+0000","last_clean":"2024-04-16T17:05:22.509857+0000","last_became_active":"2024-04-16T17:03:24.176555+0000","last_became_peered":"2024-04-16T17:03:24.176555+0000","last_unstale":"2024-04-16T17:05:22.509857+0000","last_undegraded":"2024-04-16T17:05:22.509857+0000","last_fullsized":"2024-04-16T17:05:22.509857+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:05:40.602150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"92'2","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.284637+0000","last_change":"2024-04-16T17:03:24.965746+0000","last_active":"2024-04-16T17:05:22.284637+0000","last_peered":"2024-04-16T17:05:22.284637+0000","last_clean":"2024-04-16T17:05:22.284637+0000","last_became_active":"2024-04-16T17:03:24.965241+0000","last_became_peered":"2024-04-16T17:03:24.965241+0000","last_unstale":"2024-04-16T17:05:22.284637+0000","last_undegraded":"2024-04-16T17:05:22.284637+0000","last_fullsized":"2024-04-16T17:05:22.284637+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:51:34.407565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.286457+0000","last_change":"2024-04-16T17:03:29.261441+0000","last_active":"2024-04-16T17:05:22.286457+0000","last_peered":"2024-04-16T17:05:22.286457+0000","last_clean":"2024-04-16T17:05:22.286457+0000","last_became_active":"2024-04-16T17:03:29.259538+0000","last_became_peered":"2024-04-16T17:03:29.259538+0000","last_unstale":"2024-04-16T17:05:22.286457+0000","last_undegraded":"2024-04-16T17:05:22.286457+0000","last_fullsized":"2024-04-16T17:05:22.286457+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:12:27.092053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"90'3","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.516764+0000","last_change":"2024-04-16T17:03:24.172053+0000","last_active":"2024-04-16T17:05:22.516764+0000","last_peered":"2024-04-16T17:05:22.516764+0000","last_clean":"2024-04-16T17:05:22.516764+0000","last_became_active":"2024-04-16T17:03:24.171709+0000","last_became_peered":"2024-04-16T17:03:24.171709+0000","last_unstale":"2024-04-16T17:05:22.516764+0000","last_undegraded":"2024-04-16T17:05:22.516764+0000","last_fullsized":"2024-04-16T17:05:22.516764+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:40:12.307756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"92'9","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.282872+0000","last_change":"2024-04-16T17:03:24.984267+0000","last_active":"2024-04-16T17:05:22.282872+0000","last_peered":"2024-04-16T17:05:22.282872+0000","last_clean":"2024-04-16T17:05:22.282872+0000","last_became_active":"2024-04-16T17:03:24.983888+0000","last_became_peered":"2024-04-16T17:03:24.983888+0000","last_unstale":"2024-04-16T17:05:22.282872+0000","last_undegraded":"2024-04-16T17:05:22.282872+0000","last_fullsized":"2024-04-16T17:05:22.282872+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:16:23.814917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"90'1","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.512450+0000","last_change":"2024-04-16T17:03:29.251716+0000","last_active":"2024-04-16T17:05:22.512450+0000","last_peered":"2024-04-16T17:05:22.512450+0000","last_clean":"2024-04-16T17:05:22.512450+0000","last_became_active":"2024-04-16T17:03:29.251317+0000","last_became_peered":"2024-04-16T17:03:29.251317+0000","last_unstale":"2024-04-16T17:05:22.512450+0000","last_undegraded":"2024-04-16T17:05:22.512450+0000","last_fullsized":"2024-04-16T17:05:22.512450+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:48:54.680195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"90'1","reported_seq":19,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.282769+0000","last_change":"2024-04-16T17:04:21.515306+0000","last_active":"2024-04-16T17:05:22.282769+0000","last_peered":"2024-04-16T17:05:22.282769+0000","last_clean":"2024-04-16T17:05:22.282769+0000","last_became_active":"2024-04-16T17:04:21.508377+0000","last_became_peered":"2024-04-16T17:04:21.508377+0000","last_unstale":"2024-04-16T17:05:22.282769+0000","last_undegraded":"2024-04-16T17:05:22.282769+0000","last_fullsized":"2024-04-16T17:05:22.282769+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:28:25.000406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"92'8","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.518147+0000","last_change":"2024-04-16T17:03:24.995547+0000","last_active":"2024-04-16T17:05:22.518147+0000","last_peered":"2024-04-16T17:05:22.518147+0000","last_clean":"2024-04-16T17:05:22.518147+0000","last_became_active":"2024-04-16T17:03:24.995096+0000","last_became_peered":"2024-04-16T17:03:24.995096+0000","last_unstale":"2024-04-16T17:05:22.518147+0000","last_undegraded":"2024-04-16T17:05:22.518147+0000","last_fullsized":"2024-04-16T17:05:22.518147+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:54:51.193117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.512835+0000","last_change":"2024-04-16T17:03:29.301526+0000","last_active":"2024-04-16T17:05:22.512835+0000","last_peered":"2024-04-16T17:05:22.512835+0000","last_clean":"2024-04-16T17:05:22.512835+0000","last_became_active":"2024-04-16T17:03:29.300808+0000","last_became_peered":"2024-04-16T17:03:29.300808+0000","last_unstale":"2024-04-16T17:05:22.512835+0000","last_undegraded":"2024-04-16T17:05:22.512835+0000","last_fullsized":"2024-04-16T17:05:22.512835+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:01:14.717548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"90'4","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.284713+0000","last_change":"2024-04-16T17:03:24.152645+0000","last_active":"2024-04-16T17:05:22.284713+0000","last_peered":"2024-04-16T17:05:22.284713+0000","last_clean":"2024-04-16T17:05:22.284713+0000","last_became_active":"2024-04-16T17:03:24.152281+0000","last_became_peered":"2024-04-16T17:03:24.152281+0000","last_unstale":"2024-04-16T17:05:22.284713+0000","last_undegraded":"2024-04-16T17:05:22.284713+0000","last_fullsized":"2024-04-16T17:05:22.284713+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:33:51.377421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"92'5","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.511176+0000","last_change":"2024-04-16T17:03:24.979827+0000","last_active":"2024-04-16T17:05:22.511176+0000","last_peered":"2024-04-16T17:05:22.511176+0000","last_clean":"2024-04-16T17:05:22.511176+0000","last_became_active":"2024-04-16T17:03:24.979452+0000","last_became_peered":"2024-04-16T17:03:24.979452+0000","last_unstale":"2024-04-16T17:05:22.511176+0000","last_undegraded":"2024-04-16T17:05:22.511176+0000","last_fullsized":"2024-04-16T17:05:22.511176+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:07:09.259366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.510047+0000","last_change":"2024-04-16T17:03:29.255519+0000","last_active":"2024-04-16T17:05:22.510047+0000","last_peered":"2024-04-16T17:05:22.510047+0000","last_clean":"2024-04-16T17:05:22.510047+0000","last_became_active":"2024-04-16T17:03:29.255104+0000","last_became_peered":"2024-04-16T17:03:29.255104+0000","last_unstale":"2024-04-16T17:05:22.510047+0000","last_undegraded":"2024-04-16T17:05:22.510047+0000","last_fullsized":"2024-04-16T17:05:22.510047+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:27:34.496199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"90'3","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.509582+0000","last_change":"2024-04-16T17:03:24.161145+0000","last_active":"2024-04-16T17:05:22.509582+0000","last_peered":"2024-04-16T17:05:22.509582+0000","last_clean":"2024-04-16T17:05:22.509582+0000","last_became_active":"2024-04-16T17:03:24.160824+0000","last_became_peered":"2024-04-16T17:03:24.160824+0000","last_unstale":"2024-04-16T17:05:22.509582+0000","last_undegraded":"2024-04-16T17:05:22.509582+0000","last_fullsized":"2024-04-16T17:05:22.509582+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:02:48.805638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"92'7","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.511418+0000","last_change":"2024-04-16T17:03:24.966513+0000","last_active":"2024-04-16T17:05:22.511418+0000","last_peered":"2024-04-16T17:05:22.511418+0000","last_clean":"2024-04-16T17:05:22.511418+0000","last_became_active":"2024-04-16T17:03:24.966074+0000","last_became_peered":"2024-04-16T17:03:24.966074+0000","last_unstale":"2024-04-16T17:05:22.511418+0000","last_undegraded":"2024-04-16T17:05:22.511418+0000","last_fullsized":"2024-04-16T17:05:22.511418+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:50:21.579901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.509016+0000","last_change":"2024-04-16T17:04:22.150786+0000","last_active":"2024-04-16T17:05:22.509016+0000","last_peered":"2024-04-16T17:05:22.509016+0000","last_clean":"2024-04-16T17:05:22.509016+0000","last_became_active":"2024-04-16T17:04:22.149344+0000","last_became_peered":"2024-04-16T17:04:22.149344+0000","last_unstale":"2024-04-16T17:05:22.509016+0000","last_undegraded":"2024-04-16T17:05:22.509016+0000","last_fullsized":"2024-04-16T17:05:22.509016+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:18:49.421095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,1,4],"acting":[0,8,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"85'1","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.513202+0000","last_change":"2024-04-16T17:03:24.158032+0000","last_active":"2024-04-16T17:05:22.513202+0000","last_peered":"2024-04-16T17:05:22.513202+0000","last_clean":"2024-04-16T17:05:22.513202+0000","last_became_active":"2024-04-16T17:03:24.157689+0000","last_became_peered":"2024-04-16T17:03:24.157689+0000","last_unstale":"2024-04-16T17:05:22.513202+0000","last_undegraded":"2024-04-16T17:05:22.513202+0000","last_fullsized":"2024-04-16T17:05:22.513202+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:38:18.240889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"92'6","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.511335+0000","last_change":"2024-04-16T17:04:21.519999+0000","last_active":"2024-04-16T17:05:22.511335+0000","last_peered":"2024-04-16T17:05:22.511335+0000","last_clean":"2024-04-16T17:05:22.511335+0000","last_became_active":"2024-04-16T17:04:21.519621+0000","last_became_peered":"2024-04-16T17:04:21.519621+0000","last_unstale":"2024-04-16T17:05:22.511335+0000","last_undegraded":"2024-04-16T17:05:22.511335+0000","last_fullsized":"2024-04-16T17:05:22.511335+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:50:13.266369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.512149+0000","last_change":"2024-04-16T17:03:29.245176+0000","last_active":"2024-04-16T17:05:22.512149+0000","last_peered":"2024-04-16T17:05:22.512149+0000","last_clean":"2024-04-16T17:05:22.512149+0000","last_became_active":"2024-04-16T17:03:29.244763+0000","last_became_peered":"2024-04-16T17:03:29.244763+0000","last_unstale":"2024-04-16T17:05:22.512149+0000","last_undegraded":"2024-04-16T17:05:22.512149+0000","last_fullsized":"2024-04-16T17:05:22.512149+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:52:30.381417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.290160+0000","last_change":"2024-04-16T17:03:24.168068+0000","last_active":"2024-04-16T17:05:22.290160+0000","last_peered":"2024-04-16T17:05:22.290160+0000","last_clean":"2024-04-16T17:05:22.290160+0000","last_became_active":"2024-04-16T17:03:24.167694+0000","last_became_peered":"2024-04-16T17:03:24.167694+0000","last_unstale":"2024-04-16T17:05:22.290160+0000","last_undegraded":"2024-04-16T17:05:22.290160+0000","last_fullsized":"2024-04-16T17:05:22.290160+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:13:35.692306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"92'9","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.510867+0000","last_change":"2024-04-16T17:03:24.966145+0000","last_active":"2024-04-16T17:05:22.510867+0000","last_peered":"2024-04-16T17:05:22.510867+0000","last_clean":"2024-04-16T17:05:22.510867+0000","last_became_active":"2024-04-16T17:03:24.965813+0000","last_became_peered":"2024-04-16T17:03:24.965813+0000","last_unstale":"2024-04-16T17:05:22.510867+0000","last_undegraded":"2024-04-16T17:05:22.510867+0000","last_fullsized":"2024-04-16T17:05:22.510867+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:43:43.224130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"92'9","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.283010+0000","last_change":"2024-04-16T17:03:24.970941+0000","last_active":"2024-04-16T17:05:22.283010+0000","last_peered":"2024-04-16T17:05:22.283010+0000","last_clean":"2024-04-16T17:05:22.283010+0000","last_became_active":"2024-04-16T17:03:24.970587+0000","last_became_peered":"2024-04-16T17:03:24.970587+0000","last_unstale":"2024-04-16T17:05:22.283010+0000","last_undegraded":"2024-04-16T17:05:22.283010+0000","last_fullsized":"2024-04-16T17:05:22.283010+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:14:13.930011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"92'12","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.509789+0000","last_change":"2024-04-16T17:03:24.967167+0000","last_active":"2024-04-16T17:05:22.509789+0000","last_peered":"2024-04-16T17:05:22.509789+0000","last_clean":"2024-04-16T17:05:22.509789+0000","last_became_active":"2024-04-16T17:03:24.966787+0000","last_became_peered":"2024-04-16T17:03:24.966787+0000","last_unstale":"2024-04-16T17:05:22.509789+0000","last_undegraded":"2024-04-16T17:05:22.509789+0000","last_fullsized":"2024-04-16T17:05:22.509789+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:51:39.510220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"92'8","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.509416+0000","last_change":"2024-04-16T17:03:24.981115+0000","last_active":"2024-04-16T17:05:22.509416+0000","last_peered":"2024-04-16T17:05:22.509416+0000","last_clean":"2024-04-16T17:05:22.509416+0000","last_became_active":"2024-04-16T17:03:24.980685+0000","last_became_peered":"2024-04-16T17:03:24.980685+0000","last_unstale":"2024-04-16T17:05:22.509416+0000","last_undegraded":"2024-04-16T17:05:22.509416+0000","last_fullsized":"2024-04-16T17:05:22.509416+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:20:44.066932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"92'7","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.509572+0000","last_change":"2024-04-16T17:03:24.990421+0000","last_active":"2024-04-16T17:05:22.509572+0000","last_peered":"2024-04-16T17:05:22.509572+0000","last_clean":"2024-04-16T17:05:22.509572+0000","last_became_active":"2024-04-16T17:03:24.989940+0000","last_became_peered":"2024-04-16T17:03:24.989940+0000","last_unstale":"2024-04-16T17:05:22.509572+0000","last_undegraded":"2024-04-16T17:05:22.509572+0000","last_fullsized":"2024-04-16T17:05:22.509572+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:19:57.548067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"92'4","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.290057+0000","last_change":"2024-04-16T17:03:24.991854+0000","last_active":"2024-04-16T17:05:22.290057+0000","last_peered":"2024-04-16T17:05:22.290057+0000","last_clean":"2024-04-16T17:05:22.290057+0000","last_became_active":"2024-04-16T17:03:24.990964+0000","last_became_peered":"2024-04-16T17:03:24.990964+0000","last_unstale":"2024-04-16T17:05:22.290057+0000","last_undegraded":"2024-04-16T17:05:22.290057+0000","last_fullsized":"2024-04-16T17:05:22.290057+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:10:10.618708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"92'6","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.283375+0000","last_change":"2024-04-16T17:04:21.517466+0000","last_active":"2024-04-16T17:05:22.283375+0000","last_peered":"2024-04-16T17:05:22.283375+0000","last_clean":"2024-04-16T17:05:22.283375+0000","last_became_active":"2024-04-16T17:04:21.517086+0000","last_became_peered":"2024-04-16T17:04:21.517086+0000","last_unstale":"2024-04-16T17:05:22.283375+0000","last_undegraded":"2024-04-16T17:05:22.283375+0000","last_fullsized":"2024-04-16T17:05:22.283375+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:54:33.417452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,9],"acting":[11,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"92'15","reported_seq":45,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.509933+0000","last_change":"2024-04-16T17:03:24.970377+0000","last_active":"2024-04-16T17:05:22.509933+0000","last_peered":"2024-04-16T17:05:22.509933+0000","last_clean":"2024-04-16T17:05:22.509933+0000","last_became_active":"2024-04-16T17:03:24.968856+0000","last_became_peered":"2024-04-16T17:03:24.968856+0000","last_unstale":"2024-04-16T17:05:22.509933+0000","last_undegraded":"2024-04-16T17:05:22.509933+0000","last_fullsized":"2024-04-16T17:05:22.509933+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:47:28.543120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"92'2","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.519372+0000","last_change":"2024-04-16T17:03:24.992877+0000","last_active":"2024-04-16T17:05:22.519372+0000","last_peered":"2024-04-16T17:05:22.519372+0000","last_clean":"2024-04-16T17:05:22.519372+0000","last_became_active":"2024-04-16T17:03:24.992484+0000","last_became_peered":"2024-04-16T17:03:24.992484+0000","last_unstale":"2024-04-16T17:05:22.519372+0000","last_undegraded":"2024-04-16T17:05:22.519372+0000","last_fullsized":"2024-04-16T17:05:22.519372+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:00:45.204782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"92'6","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.512442+0000","last_change":"2024-04-16T17:03:24.977794+0000","last_active":"2024-04-16T17:05:22.512442+0000","last_peered":"2024-04-16T17:05:22.512442+0000","last_clean":"2024-04-16T17:05:22.512442+0000","last_became_active":"2024-04-16T17:03:24.977434+0000","last_became_peered":"2024-04-16T17:03:24.977434+0000","last_unstale":"2024-04-16T17:05:22.512442+0000","last_undegraded":"2024-04-16T17:05:22.512442+0000","last_fullsized":"2024-04-16T17:05:22.512442+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:46:35.670357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"92'8","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.516646+0000","last_change":"2024-04-16T17:03:24.992225+0000","last_active":"2024-04-16T17:05:22.516646+0000","last_peered":"2024-04-16T17:05:22.516646+0000","last_clean":"2024-04-16T17:05:22.516646+0000","last_became_active":"2024-04-16T17:03:24.991904+0000","last_became_peered":"2024-04-16T17:03:24.991904+0000","last_unstale":"2024-04-16T17:05:22.516646+0000","last_undegraded":"2024-04-16T17:05:22.516646+0000","last_fullsized":"2024-04-16T17:05:22.516646+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:01:09.710698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"92'10","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.510323+0000","last_change":"2024-04-16T17:03:24.981322+0000","last_active":"2024-04-16T17:05:22.510323+0000","last_peered":"2024-04-16T17:05:22.510323+0000","last_clean":"2024-04-16T17:05:22.510323+0000","last_became_active":"2024-04-16T17:03:24.980920+0000","last_became_peered":"2024-04-16T17:03:24.980920+0000","last_unstale":"2024-04-16T17:05:22.510323+0000","last_undegraded":"2024-04-16T17:05:22.510323+0000","last_fullsized":"2024-04-16T17:05:22.510323+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:07:59.378512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"92'11","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.289181+0000","last_change":"2024-04-16T17:03:24.990117+0000","last_active":"2024-04-16T17:05:22.289181+0000","last_peered":"2024-04-16T17:05:22.289181+0000","last_clean":"2024-04-16T17:05:22.289181+0000","last_became_active":"2024-04-16T17:03:24.989734+0000","last_became_peered":"2024-04-16T17:03:24.989734+0000","last_unstale":"2024-04-16T17:05:22.289181+0000","last_undegraded":"2024-04-16T17:05:22.289181+0000","last_fullsized":"2024-04-16T17:05:22.289181+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:36:55.000702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"92'8","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.283130+0000","last_change":"2024-04-16T17:03:24.970032+0000","last_active":"2024-04-16T17:05:22.283130+0000","last_peered":"2024-04-16T17:05:22.283130+0000","last_clean":"2024-04-16T17:05:22.283130+0000","last_became_active":"2024-04-16T17:03:24.968608+0000","last_became_peered":"2024-04-16T17:03:24.968608+0000","last_unstale":"2024-04-16T17:05:22.283130+0000","last_undegraded":"2024-04-16T17:05:22.283130+0000","last_fullsized":"2024-04-16T17:05:22.283130+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:24:03.918418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":283858,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":68279,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":638,"num_read_kb":395,"num_write":175,"num_write_kb":125,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":616,"num_keys_recovered":20,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":204837,"data_compressed":4887,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":151,"ondisk_log_size":151,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351016,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1032192,"data_stored":522666,"data_compressed":26684,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612655,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32332,"kb_used_data":1148,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712820,"statfs":{"total":95995035648,"available":95961927680,"internally_reserved":0,"allocated":1175552,"data_stored":1178133,"data_compressed":64411,"data_compressed_allocated":86016,"data_compressed_original":671744,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776179,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32204,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712948,"statfs":{"total":95995035648,"available":95962058752,"internally_reserved":0,"allocated":1044480,"data_stored":522868,"data_compressed":26684,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037818,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32200,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712952,"statfs":{"total":95995035648,"available":95962062848,"internally_reserved":0,"allocated":1040384,"data_stored":522206,"data_compressed":26684,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299456,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36412,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708740,"statfs":{"total":95995035648,"available":95957749760,"internally_reserved":0,"allocated":1093632,"data_stored":1119370,"data_compressed":62782,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462982,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32184,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712968,"statfs":{"total":95995035648,"available":95962079232,"internally_reserved":0,"allocated":1024000,"data_stored":518044,"data_compressed":26684,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574946,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32248,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712904,"statfs":{"total":95995035648,"available":95962013696,"internally_reserved":0,"allocated":1093632,"data_stored":582141,"data_compressed":28313,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738470,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32208,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712944,"statfs":{"total":95995035648,"available":95962054656,"internally_reserved":0,"allocated":1048576,"data_stored":519432,"data_compressed":26684,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411420,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28104,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717048,"statfs":{"total":95995035648,"available":95966257152,"internally_reserved":0,"allocated":1110016,"data_stored":583407,"data_compressed":28313,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149782,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32192,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712960,"statfs":{"total":95995035648,"available":95962071040,"internally_reserved":0,"allocated":1036288,"data_stored":518238,"data_compressed":26684,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888144,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1036288,"data_stored":523226,"data_compressed":26684,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724620,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28096,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717056,"statfs":{"total":95995035648,"available":95966265344,"internally_reserved":0,"allocated":1101824,"data_stored":1114426,"data_compressed":62782,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34456,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34456,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34456,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36098,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36098,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36098,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36098,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36098,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":64187,"data_compressed":1629,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":65453,"data_compressed":1629,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":156,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":65715,"data_compressed":1629,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-16T17:05:39.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:05:39 smithi104 ceph-mon[36457]: pgmap v393: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-16T17:05:39.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:05:39 smithi045 ceph-mon[31676]: pgmap v393: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-16T17:05:39.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:05:39 smithi117 ceph-mon[36730]: pgmap v393: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-16T17:05:41.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:05:41 smithi104 ceph-mon[36457]: pgmap v394: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-16T17:05:41.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:05:41 smithi045 ceph-mon[31676]: pgmap v394: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-16T17:05:41.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:05:41 smithi117 ceph-mon[36730]: pgmap v394: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-16T17:05:41.826 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45 shell --fsid b1fecd38-fc11-11ee-bc90-c7b262605968 -- ceph pg dump --format=json 2024-04-16T17:05:42.309 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/mon.a/config 2024-04-16T17:05:43.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:05:43 smithi104 ceph-mon[36457]: pgmap v395: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-16T17:05:43.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:05:43 smithi045 ceph-mon[31676]: pgmap v395: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-16T17:05:43.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:05:43 smithi117 ceph-mon[36730]: pgmap v395: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-16T17:05:44.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:44.713+0000 7fb21b200700 1 -- 172.21.15.45:0/2902848908 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb214109380 msgr2=0x7fb214109760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:44.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:44.713+0000 7fb21b200700 1 --2- 172.21.15.45:0/2902848908 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb214109380 0x7fb214109760 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fb204009920 tx=0x7fb204009c30 comp rx=0 tx=0).stop 2024-04-16T17:05:44.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:44.713+0000 7fb21b200700 1 -- 172.21.15.45:0/2902848908 shutdown_connections 2024-04-16T17:05:44.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:44.713+0000 7fb21b200700 1 --2- 172.21.15.45:0/2902848908 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb214109380 0x7fb214109760 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:44.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:44.713+0000 7fb21b200700 1 --2- 172.21.15.45:0/2902848908 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb21410e460 0x7fb214116030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:44.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:44.713+0000 7fb21b200700 1 --2- 172.21.15.45:0/2902848908 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb214109d30 0x7fb21410dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:44.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:44.713+0000 7fb21b200700 1 -- 172.21.15.45:0/2902848908 >> 172.21.15.45:0/2902848908 conn(0x7fb2140762c0 msgr2=0x7fb214107910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:05:44.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:44.713+0000 7fb21b200700 1 -- 172.21.15.45:0/2902848908 shutdown_connections 2024-04-16T17:05:44.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:44.713+0000 7fb21b200700 1 -- 172.21.15.45:0/2902848908 wait complete. 2024-04-16T17:05:44.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:44.714+0000 7fb21b200700 1 Processor -- start 2024-04-16T17:05:44.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:44.714+0000 7fb21b200700 1 -- start start 2024-04-16T17:05:44.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:44.715+0000 7fb21b200700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb214109380 0x7fb2141a5690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:44.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:44.715+0000 7fb21b200700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb214109d30 0x7fb2141a5bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:44.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:44.715+0000 7fb21b200700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb21410e460 0x7fb21419f760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:44.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:44.715+0000 7fb21b200700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fb214119440 con 0x7fb21410e460 2024-04-16T17:05:44.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:44.715+0000 7fb21b200700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fb2141192c0 con 0x7fb214109380 2024-04-16T17:05:44.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:44.715+0000 7fb21b200700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb2141195c0 con 0x7fb214109d30 2024-04-16T17:05:44.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:44.715+0000 7fb219200700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb21410e460 0x7fb21419f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:44.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:44.715+0000 7fb213e00700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb214109380 0x7fb2141a5690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:44.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:44.715+0000 7fb219200700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb21410e460 0x7fb21419f760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39570/0 (socket says 172.21.15.45:39570) 2024-04-16T17:05:44.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:44.715+0000 7fb219200700 1 -- 172.21.15.45:0/2937752884 learned_addr learned my addr 172.21.15.45:0/2937752884 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:05:44.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:44.715+0000 7fb213400700 1 --2- 172.21.15.45:0/2937752884 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb214109d30 0x7fb2141a5bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:44.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:44.716+0000 7fb213e00700 1 -- 172.21.15.45:0/2937752884 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb214109d30 msgr2=0x7fb2141a5bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:44.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:44.716+0000 7fb213e00700 1 --2- 172.21.15.45:0/2937752884 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb214109d30 0x7fb2141a5bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:44.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:44.716+0000 7fb213e00700 1 -- 172.21.15.45:0/2937752884 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb21410e460 msgr2=0x7fb21419f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:44.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:44.716+0000 7fb213e00700 1 --2- 172.21.15.45:0/2937752884 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb21410e460 0x7fb21419f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:44.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:44.716+0000 7fb213e00700 1 -- 172.21.15.45:0/2937752884 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2141a0040 con 0x7fb214109380 2024-04-16T17:05:44.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:44.716+0000 7fb219200700 1 --2- 172.21.15.45:0/2937752884 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb21410e460 0x7fb21419f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:05:44.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:44.716+0000 7fb213e00700 1 --2- 172.21.15.45:0/2937752884 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb214109380 0x7fb2141a5690 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fb204000c00 tx=0x7fb20400fa00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:05:44.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:44.716+0000 7fb210c00700 1 -- 172.21.15.45:0/2937752884 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb204027070 con 0x7fb214109380 2024-04-16T17:05:44.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:44.716+0000 7fb210c00700 1 -- 172.21.15.45:0/2937752884 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb204005620 con 0x7fb214109380 2024-04-16T17:05:44.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:44.716+0000 7fb21b200700 1 -- 172.21.15.45:0/2937752884 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2141a02d0 con 0x7fb214109380 2024-04-16T17:05:44.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:44.716+0000 7fb210c00700 1 -- 172.21.15.45:0/2937752884 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb204004930 con 0x7fb214109380 2024-04-16T17:05:44.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:44.716+0000 7fb21b200700 1 -- 172.21.15.45:0/2937752884 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2141ac440 con 0x7fb214109380 2024-04-16T17:05:44.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:44.717+0000 7fb21b200700 1 -- 172.21.15.45:0/2937752884 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb21410a860 con 0x7fb214109380 2024-04-16T17:05:44.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:44.720+0000 7fb210c00700 1 -- 172.21.15.45:0/2937752884 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fb20402a070 con 0x7fb214109380 2024-04-16T17:05:44.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:44.721+0000 7fb210c00700 1 --2- 172.21.15.45:0/2937752884 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fb1f80a3c40 0x7fb1f80a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:44.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:44.721+0000 7fb210c00700 1 -- 172.21.15.45:0/2937752884 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fb2040d1340 con 0x7fb214109380 2024-04-16T17:05:44.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:44.721+0000 7fb213400700 1 --2- 172.21.15.45:0/2937752884 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fb1f80a3c40 0x7fb1f80a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:44.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:44.722+0000 7fb213400700 1 --2- 172.21.15.45:0/2937752884 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fb1f80a3c40 0x7fb1f80a6100 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fb2141a0f20 tx=0x7fb200009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:05:44.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:44.723+0000 7fb210c00700 1 -- 172.21.15.45:0/2937752884 <== mon.2 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb204031080 con 0x7fb214109380 2024-04-16T17:05:44.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:44.952+0000 7fb21b200700 1 -- 172.21.15.45:0/2937752884 --> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb2141a0c90 con 0x7fb1f80a3c40 2024-04-16T17:05:44.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:44.969+0000 7fb210c00700 1 -- 172.21.15.45:0/2937752884 <== mgr.14152 v2:172.21.15.45:6800/268030699 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340815 (secure 0 0 0) 0x7fb2141a0c90 con 0x7fb1f80a3c40 2024-04-16T17:05:44.973 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-16T17:05:44.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:44.977+0000 7fb21b200700 1 -- 172.21.15.45:0/2937752884 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fb1f80a3c40 msgr2=0x7fb1f80a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:44.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:44.977+0000 7fb21b200700 1 --2- 172.21.15.45:0/2937752884 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fb1f80a3c40 0x7fb1f80a6100 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fb2141a0f20 tx=0x7fb200009310 comp rx=0 tx=0).stop 2024-04-16T17:05:44.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:44.977+0000 7fb21b200700 1 -- 172.21.15.45:0/2937752884 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb214109380 msgr2=0x7fb2141a5690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:44.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:44.977+0000 7fb21b200700 1 --2- 172.21.15.45:0/2937752884 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb214109380 0x7fb2141a5690 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fb204000c00 tx=0x7fb20400fa00 comp rx=0 tx=0).stop 2024-04-16T17:05:44.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:44.978+0000 7fb21b200700 1 -- 172.21.15.45:0/2937752884 shutdown_connections 2024-04-16T17:05:44.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:44.978+0000 7fb21b200700 1 --2- 172.21.15.45:0/2937752884 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb21410e460 0x7fb21419f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:44.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:44.978+0000 7fb21b200700 1 --2- 172.21.15.45:0/2937752884 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb214109380 0x7fb2141a5690 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:44.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:44.978+0000 7fb21b200700 1 --2- 172.21.15.45:0/2937752884 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fb1f80a3c40 0x7fb1f80a6100 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:44.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:44.978+0000 7fb21b200700 1 --2- 172.21.15.45:0/2937752884 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb214109d30 0x7fb2141a5bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:44.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:44.978+0000 7fb21b200700 1 -- 172.21.15.45:0/2937752884 >> 172.21.15.45:0/2937752884 conn(0x7fb2140762c0 msgr2=0x7fb214076940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:05:44.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:44.978+0000 7fb21b200700 1 -- 172.21.15.45:0/2937752884 shutdown_connections 2024-04-16T17:05:44.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:44.978+0000 7fb21b200700 1 -- 172.21.15.45:0/2937752884 wait complete. 2024-04-16T17:05:44.980 INFO:teuthology.orchestra.run.smithi045.stderr:dumped all 2024-04-16T17:05:45.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:05:45 smithi104 ceph-mon[36457]: pgmap v396: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T17:05:45.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:05:45 smithi045 ceph-mon[31676]: pgmap v396: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T17:05:45.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:05:45 smithi117 ceph-mon[36730]: pgmap v396: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T17:05:45.882 INFO:teuthology.orchestra.run.smithi045.stdout:{"pg_ready":true,"pg_map":{"version":396,"stamp":"2024-04-16T17:05:44.163839+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":658881,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":809,"num_read_kb":529,"num_write":358,"num_write_kb":3065,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":1181469,"num_keys_recovered":20,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":790,"ondisk_log_size":790,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":374244,"kb_used_data":12536,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124567580,"statfs":{"total":1151940427776,"available":1151557201920,"internally_reserved":0,"allocated":12836864,"data_stored":8224157,"data_compressed":433389,"data_compressed_allocated":589824,"data_compressed_original":2678784,"omap_allocated":19143,"internal_metadata":370193721},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":60,"num_read_kb":30,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006080"},"pg_stats":[{"pgid":"3.3f","version":"92'8","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.283618+0000","last_change":"2024-04-16T17:03:24.989508+0000","last_active":"2024-04-16T17:05:22.283618+0000","last_peered":"2024-04-16T17:05:22.283618+0000","last_clean":"2024-04-16T17:05:22.283618+0000","last_became_active":"2024-04-16T17:03:24.989178+0000","last_became_peered":"2024-04-16T17:03:24.989178+0000","last_unstale":"2024-04-16T17:05:22.283618+0000","last_undegraded":"2024-04-16T17:05:22.283618+0000","last_fullsized":"2024-04-16T17:05:22.283618+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:18:56.488358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"92'4","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.510710+0000","last_change":"2024-04-16T17:03:24.986378+0000","last_active":"2024-04-16T17:05:22.510710+0000","last_peered":"2024-04-16T17:05:22.510710+0000","last_clean":"2024-04-16T17:05:22.510710+0000","last_became_active":"2024-04-16T17:03:24.986030+0000","last_became_peered":"2024-04-16T17:03:24.986030+0000","last_unstale":"2024-04-16T17:05:22.510710+0000","last_undegraded":"2024-04-16T17:05:22.510710+0000","last_fullsized":"2024-04-16T17:05:22.510710+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:21:14.071109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"92'7","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.509472+0000","last_change":"2024-04-16T17:03:24.976232+0000","last_active":"2024-04-16T17:05:22.509472+0000","last_peered":"2024-04-16T17:05:22.509472+0000","last_clean":"2024-04-16T17:05:22.509472+0000","last_became_active":"2024-04-16T17:03:24.975873+0000","last_became_peered":"2024-04-16T17:03:24.975873+0000","last_unstale":"2024-04-16T17:05:22.509472+0000","last_undegraded":"2024-04-16T17:05:22.509472+0000","last_fullsized":"2024-04-16T17:05:22.509472+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:07:53.523101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"92'8","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.285223+0000","last_change":"2024-04-16T17:04:21.517830+0000","last_active":"2024-04-16T17:05:22.285223+0000","last_peered":"2024-04-16T17:05:22.285223+0000","last_clean":"2024-04-16T17:05:22.285223+0000","last_became_active":"2024-04-16T17:04:21.516749+0000","last_became_peered":"2024-04-16T17:04:21.516749+0000","last_unstale":"2024-04-16T17:05:22.285223+0000","last_undegraded":"2024-04-16T17:05:22.285223+0000","last_fullsized":"2024-04-16T17:05:22.285223+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:02:33.344034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"92'4","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.284780+0000","last_change":"2024-04-16T17:03:24.985609+0000","last_active":"2024-04-16T17:05:22.284780+0000","last_peered":"2024-04-16T17:05:22.284780+0000","last_clean":"2024-04-16T17:05:22.284780+0000","last_became_active":"2024-04-16T17:03:24.985239+0000","last_became_peered":"2024-04-16T17:03:24.985239+0000","last_unstale":"2024-04-16T17:05:22.284780+0000","last_undegraded":"2024-04-16T17:05:22.284780+0000","last_fullsized":"2024-04-16T17:05:22.284780+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:55:58.619210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"92'6","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.282909+0000","last_change":"2024-04-16T17:03:24.966862+0000","last_active":"2024-04-16T17:05:22.282909+0000","last_peered":"2024-04-16T17:05:22.282909+0000","last_clean":"2024-04-16T17:05:22.282909+0000","last_became_active":"2024-04-16T17:03:24.966234+0000","last_became_peered":"2024-04-16T17:03:24.966234+0000","last_unstale":"2024-04-16T17:05:22.282909+0000","last_undegraded":"2024-04-16T17:05:22.282909+0000","last_fullsized":"2024-04-16T17:05:22.282909+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:21:07.018281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"92'5","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.510598+0000","last_change":"2024-04-16T17:03:24.971219+0000","last_active":"2024-04-16T17:05:22.510598+0000","last_peered":"2024-04-16T17:05:22.510598+0000","last_clean":"2024-04-16T17:05:22.510598+0000","last_became_active":"2024-04-16T17:03:24.970876+0000","last_became_peered":"2024-04-16T17:03:24.970876+0000","last_unstale":"2024-04-16T17:05:22.510598+0000","last_undegraded":"2024-04-16T17:05:22.510598+0000","last_fullsized":"2024-04-16T17:05:22.510598+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:44:16.311167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"92'9","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.515605+0000","last_change":"2024-04-16T17:03:24.994694+0000","last_active":"2024-04-16T17:05:22.515605+0000","last_peered":"2024-04-16T17:05:22.515605+0000","last_clean":"2024-04-16T17:05:22.515605+0000","last_became_active":"2024-04-16T17:03:24.994315+0000","last_became_peered":"2024-04-16T17:03:24.994315+0000","last_unstale":"2024-04-16T17:05:22.515605+0000","last_undegraded":"2024-04-16T17:05:22.515605+0000","last_fullsized":"2024-04-16T17:05:22.515605+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:04:59.543816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"92'11","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.512218+0000","last_change":"2024-04-16T17:03:24.964654+0000","last_active":"2024-04-16T17:05:22.512218+0000","last_peered":"2024-04-16T17:05:22.512218+0000","last_clean":"2024-04-16T17:05:22.512218+0000","last_became_active":"2024-04-16T17:03:24.964349+0000","last_became_peered":"2024-04-16T17:03:24.964349+0000","last_unstale":"2024-04-16T17:05:22.512218+0000","last_undegraded":"2024-04-16T17:05:22.512218+0000","last_fullsized":"2024-04-16T17:05:22.512218+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:38:55.880948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"92'16","reported_seq":45,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.283246+0000","last_change":"2024-04-16T17:03:24.966364+0000","last_active":"2024-04-16T17:05:22.283246+0000","last_peered":"2024-04-16T17:05:22.283246+0000","last_clean":"2024-04-16T17:05:22.283246+0000","last_became_active":"2024-04-16T17:03:24.966061+0000","last_became_peered":"2024-04-16T17:03:24.966061+0000","last_unstale":"2024-04-16T17:05:22.283246+0000","last_undegraded":"2024-04-16T17:05:22.283246+0000","last_fullsized":"2024-04-16T17:05:22.283246+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:18:12.642602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"92'6","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.511278+0000","last_change":"2024-04-16T17:03:24.969874+0000","last_active":"2024-04-16T17:05:22.511278+0000","last_peered":"2024-04-16T17:05:22.511278+0000","last_clean":"2024-04-16T17:05:22.511278+0000","last_became_active":"2024-04-16T17:03:24.969523+0000","last_became_peered":"2024-04-16T17:03:24.969523+0000","last_unstale":"2024-04-16T17:05:22.511278+0000","last_undegraded":"2024-04-16T17:05:22.511278+0000","last_fullsized":"2024-04-16T17:05:22.511278+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:17:49.704855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"92'7","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.512621+0000","last_change":"2024-04-16T17:03:24.968660+0000","last_active":"2024-04-16T17:05:22.512621+0000","last_peered":"2024-04-16T17:05:22.512621+0000","last_clean":"2024-04-16T17:05:22.512621+0000","last_became_active":"2024-04-16T17:03:24.968348+0000","last_became_peered":"2024-04-16T17:03:24.968348+0000","last_unstale":"2024-04-16T17:05:22.512621+0000","last_undegraded":"2024-04-16T17:05:22.512621+0000","last_fullsized":"2024-04-16T17:05:22.512621+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:06:06.319391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"92'10","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.289436+0000","last_change":"2024-04-16T17:03:24.974741+0000","last_active":"2024-04-16T17:05:22.289436+0000","last_peered":"2024-04-16T17:05:22.289436+0000","last_clean":"2024-04-16T17:05:22.289436+0000","last_became_active":"2024-04-16T17:03:24.973568+0000","last_became_peered":"2024-04-16T17:03:24.973568+0000","last_unstale":"2024-04-16T17:05:22.289436+0000","last_undegraded":"2024-04-16T17:05:22.289436+0000","last_fullsized":"2024-04-16T17:05:22.289436+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:30:49.325469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"92'7","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.509052+0000","last_change":"2024-04-16T17:03:24.981629+0000","last_active":"2024-04-16T17:05:22.509052+0000","last_peered":"2024-04-16T17:05:22.509052+0000","last_clean":"2024-04-16T17:05:22.509052+0000","last_became_active":"2024-04-16T17:03:24.981111+0000","last_became_peered":"2024-04-16T17:03:24.981111+0000","last_unstale":"2024-04-16T17:05:22.509052+0000","last_undegraded":"2024-04-16T17:05:22.509052+0000","last_fullsized":"2024-04-16T17:05:22.509052+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:41:29.641610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"92'7","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.508949+0000","last_change":"2024-04-16T17:03:24.977890+0000","last_active":"2024-04-16T17:05:22.508949+0000","last_peered":"2024-04-16T17:05:22.508949+0000","last_clean":"2024-04-16T17:05:22.508949+0000","last_became_active":"2024-04-16T17:03:24.976500+0000","last_became_peered":"2024-04-16T17:03:24.976500+0000","last_unstale":"2024-04-16T17:05:22.508949+0000","last_undegraded":"2024-04-16T17:05:22.508949+0000","last_fullsized":"2024-04-16T17:05:22.508949+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:17:51.604701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"92'9","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.509773+0000","last_change":"2024-04-16T17:03:24.976085+0000","last_active":"2024-04-16T17:05:22.509773+0000","last_peered":"2024-04-16T17:05:22.509773+0000","last_clean":"2024-04-16T17:05:22.509773+0000","last_became_active":"2024-04-16T17:03:24.975804+0000","last_became_peered":"2024-04-16T17:03:24.975804+0000","last_unstale":"2024-04-16T17:05:22.509773+0000","last_undegraded":"2024-04-16T17:05:22.509773+0000","last_fullsized":"2024-04-16T17:05:22.509773+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:19:13.706889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"92'8","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.283256+0000","last_change":"2024-04-16T17:03:24.989456+0000","last_active":"2024-04-16T17:05:22.283256+0000","last_peered":"2024-04-16T17:05:22.283256+0000","last_clean":"2024-04-16T17:05:22.283256+0000","last_became_active":"2024-04-16T17:03:24.989161+0000","last_became_peered":"2024-04-16T17:03:24.989161+0000","last_unstale":"2024-04-16T17:05:22.283256+0000","last_undegraded":"2024-04-16T17:05:22.283256+0000","last_fullsized":"2024-04-16T17:05:22.283256+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:52:59.333625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"92'10","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.287957+0000","last_change":"2024-04-16T17:03:24.963428+0000","last_active":"2024-04-16T17:05:22.287957+0000","last_peered":"2024-04-16T17:05:22.287957+0000","last_clean":"2024-04-16T17:05:22.287957+0000","last_became_active":"2024-04-16T17:03:24.962523+0000","last_became_peered":"2024-04-16T17:03:24.962523+0000","last_unstale":"2024-04-16T17:05:22.287957+0000","last_undegraded":"2024-04-16T17:05:22.287957+0000","last_fullsized":"2024-04-16T17:05:22.287957+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:22:19.523901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"92'7","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.509802+0000","last_change":"2024-04-16T17:03:24.981122+0000","last_active":"2024-04-16T17:05:22.509802+0000","last_peered":"2024-04-16T17:05:22.509802+0000","last_clean":"2024-04-16T17:05:22.509802+0000","last_became_active":"2024-04-16T17:03:24.980533+0000","last_became_peered":"2024-04-16T17:03:24.980533+0000","last_unstale":"2024-04-16T17:05:22.509802+0000","last_undegraded":"2024-04-16T17:05:22.509802+0000","last_fullsized":"2024-04-16T17:05:22.509802+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:51:08.580965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.509563+0000","last_change":"2024-04-16T17:03:29.257239+0000","last_active":"2024-04-16T17:05:22.509563+0000","last_peered":"2024-04-16T17:05:22.509563+0000","last_clean":"2024-04-16T17:05:22.509563+0000","last_became_active":"2024-04-16T17:03:29.256827+0000","last_became_peered":"2024-04-16T17:03:29.256827+0000","last_unstale":"2024-04-16T17:05:22.509563+0000","last_undegraded":"2024-04-16T17:05:22.509563+0000","last_fullsized":"2024-04-16T17:05:22.509563+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:57:28.373386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"90'9","reported_seq":58,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.287155+0000","last_change":"2024-04-16T17:04:21.896665+0000","last_active":"2024-04-16T17:05:22.287155+0000","last_peered":"2024-04-16T17:05:22.287155+0000","last_clean":"2024-04-16T17:05:22.287155+0000","last_became_active":"2024-04-16T17:04:21.882573+0000","last_became_peered":"2024-04-16T17:04:21.882573+0000","last_unstale":"2024-04-16T17:05:22.287155+0000","last_undegraded":"2024-04-16T17:05:22.287155+0000","last_fullsized":"2024-04-16T17:05:22.287155+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:37:07.210368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,1],"acting":[9,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"92'5","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.516852+0000","last_change":"2024-04-16T17:03:24.994921+0000","last_active":"2024-04-16T17:05:22.516852+0000","last_peered":"2024-04-16T17:05:22.516852+0000","last_clean":"2024-04-16T17:05:22.516852+0000","last_became_active":"2024-04-16T17:03:24.994412+0000","last_became_peered":"2024-04-16T17:03:24.994412+0000","last_unstale":"2024-04-16T17:05:22.516852+0000","last_undegraded":"2024-04-16T17:05:22.516852+0000","last_fullsized":"2024-04-16T17:05:22.516852+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:11:40.748198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.511978+0000","last_change":"2024-04-16T17:03:29.259884+0000","last_active":"2024-04-16T17:05:22.511978+0000","last_peered":"2024-04-16T17:05:22.511978+0000","last_clean":"2024-04-16T17:05:22.511978+0000","last_became_active":"2024-04-16T17:03:29.259319+0000","last_became_peered":"2024-04-16T17:03:29.259319+0000","last_unstale":"2024-04-16T17:05:22.511978+0000","last_undegraded":"2024-04-16T17:05:22.511978+0000","last_fullsized":"2024-04-16T17:05:22.511978+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:05:18.244682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"90'2","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.519175+0000","last_change":"2024-04-16T17:03:24.179308+0000","last_active":"2024-04-16T17:05:22.519175+0000","last_peered":"2024-04-16T17:05:22.519175+0000","last_clean":"2024-04-16T17:05:22.519175+0000","last_became_active":"2024-04-16T17:03:24.178894+0000","last_became_peered":"2024-04-16T17:03:24.178894+0000","last_unstale":"2024-04-16T17:05:22.519175+0000","last_undegraded":"2024-04-16T17:05:22.519175+0000","last_fullsized":"2024-04-16T17:05:22.519175+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:17:46.831871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"92'6","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.511197+0000","last_change":"2024-04-16T17:03:24.982391+0000","last_active":"2024-04-16T17:05:22.511197+0000","last_peered":"2024-04-16T17:05:22.511197+0000","last_clean":"2024-04-16T17:05:22.511197+0000","last_became_active":"2024-04-16T17:03:24.981761+0000","last_became_peered":"2024-04-16T17:03:24.981761+0000","last_unstale":"2024-04-16T17:05:22.511197+0000","last_undegraded":"2024-04-16T17:05:22.511197+0000","last_fullsized":"2024-04-16T17:05:22.511197+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:40:57.657170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.512324+0000","last_change":"2024-04-16T17:03:29.259705+0000","last_active":"2024-04-16T17:05:22.512324+0000","last_peered":"2024-04-16T17:05:22.512324+0000","last_clean":"2024-04-16T17:05:22.512324+0000","last_became_active":"2024-04-16T17:03:29.259336+0000","last_became_peered":"2024-04-16T17:03:29.259336+0000","last_unstale":"2024-04-16T17:05:22.512324+0000","last_undegraded":"2024-04-16T17:05:22.512324+0000","last_fullsized":"2024-04-16T17:05:22.512324+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:53:29.042933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"90'4","reported_seq":47,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.285245+0000","last_change":"2024-04-16T17:04:22.153413+0000","last_active":"2024-04-16T17:05:22.285245+0000","last_peered":"2024-04-16T17:05:22.285245+0000","last_clean":"2024-04-16T17:05:22.285245+0000","last_became_active":"2024-04-16T17:04:22.148972+0000","last_became_peered":"2024-04-16T17:04:22.148972+0000","last_unstale":"2024-04-16T17:05:22.285245+0000","last_undegraded":"2024-04-16T17:05:22.285245+0000","last_fullsized":"2024-04-16T17:05:22.285245+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:05:57.513052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"92'9","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.510097+0000","last_change":"2024-04-16T17:03:24.973205+0000","last_active":"2024-04-16T17:05:22.510097+0000","last_peered":"2024-04-16T17:05:22.510097+0000","last_clean":"2024-04-16T17:05:22.510097+0000","last_became_active":"2024-04-16T17:03:24.972863+0000","last_became_peered":"2024-04-16T17:03:24.972863+0000","last_unstale":"2024-04-16T17:05:22.510097+0000","last_undegraded":"2024-04-16T17:05:22.510097+0000","last_fullsized":"2024-04-16T17:05:22.510097+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:28:29.005783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.512076+0000","last_change":"2024-04-16T17:03:29.262930+0000","last_active":"2024-04-16T17:05:22.512076+0000","last_peered":"2024-04-16T17:05:22.512076+0000","last_clean":"2024-04-16T17:05:22.512076+0000","last_became_active":"2024-04-16T17:03:29.262621+0000","last_became_peered":"2024-04-16T17:03:29.262621+0000","last_unstale":"2024-04-16T17:05:22.512076+0000","last_undegraded":"2024-04-16T17:05:22.512076+0000","last_fullsized":"2024-04-16T17:05:22.512076+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:22:31.555143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.285074+0000","last_change":"2024-04-16T17:03:24.150503+0000","last_active":"2024-04-16T17:05:22.285074+0000","last_peered":"2024-04-16T17:05:22.285074+0000","last_clean":"2024-04-16T17:05:22.285074+0000","last_became_active":"2024-04-16T17:03:24.150021+0000","last_became_peered":"2024-04-16T17:03:24.150021+0000","last_unstale":"2024-04-16T17:05:22.285074+0000","last_undegraded":"2024-04-16T17:05:22.285074+0000","last_fullsized":"2024-04-16T17:05:22.285074+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:21:36.119497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"92'15","reported_seq":44,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.513017+0000","last_change":"2024-04-16T17:03:24.965049+0000","last_active":"2024-04-16T17:05:22.513017+0000","last_peered":"2024-04-16T17:05:22.513017+0000","last_clean":"2024-04-16T17:05:22.513017+0000","last_became_active":"2024-04-16T17:03:24.963120+0000","last_became_peered":"2024-04-16T17:03:24.963120+0000","last_unstale":"2024-04-16T17:05:22.513017+0000","last_undegraded":"2024-04-16T17:05:22.513017+0000","last_fullsized":"2024-04-16T17:05:22.513017+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:17:49.725113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.509577+0000","last_change":"2024-04-16T17:03:29.247013+0000","last_active":"2024-04-16T17:05:22.509577+0000","last_peered":"2024-04-16T17:05:22.509577+0000","last_clean":"2024-04-16T17:05:22.509577+0000","last_became_active":"2024-04-16T17:03:29.246477+0000","last_became_peered":"2024-04-16T17:03:29.246477+0000","last_unstale":"2024-04-16T17:05:22.509577+0000","last_undegraded":"2024-04-16T17:05:22.509577+0000","last_fullsized":"2024-04-16T17:05:22.509577+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:52:19.525076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"92'7","reported_seq":149,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:38.540255+0000","last_change":"2024-04-16T17:03:24.155333+0000","last_active":"2024-04-16T17:05:38.540255+0000","last_peered":"2024-04-16T17:05:38.540255+0000","last_clean":"2024-04-16T17:05:38.540255+0000","last_became_active":"2024-04-16T17:03:24.154848+0000","last_became_peered":"2024-04-16T17:03:24.154848+0000","last_unstale":"2024-04-16T17:05:38.540255+0000","last_undegraded":"2024-04-16T17:05:38.540255+0000","last_fullsized":"2024-04-16T17:05:38.540255+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:03:34.769573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1154,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":111,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"92'7","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.285538+0000","last_change":"2024-04-16T17:03:24.967627+0000","last_active":"2024-04-16T17:05:22.285538+0000","last_peered":"2024-04-16T17:05:22.285538+0000","last_clean":"2024-04-16T17:05:22.285538+0000","last_became_active":"2024-04-16T17:03:24.967264+0000","last_became_peered":"2024-04-16T17:03:24.967264+0000","last_unstale":"2024-04-16T17:05:22.285538+0000","last_undegraded":"2024-04-16T17:05:22.285538+0000","last_fullsized":"2024-04-16T17:05:22.285538+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:35:29.182374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.509364+0000","last_change":"2024-04-16T17:03:29.264625+0000","last_active":"2024-04-16T17:05:22.509364+0000","last_peered":"2024-04-16T17:05:22.509364+0000","last_clean":"2024-04-16T17:05:22.509364+0000","last_became_active":"2024-04-16T17:03:29.264114+0000","last_became_peered":"2024-04-16T17:03:29.264114+0000","last_unstale":"2024-04-16T17:05:22.509364+0000","last_undegraded":"2024-04-16T17:05:22.509364+0000","last_fullsized":"2024-04-16T17:05:22.509364+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:14:49.449980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"90'3","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.282754+0000","last_change":"2024-04-16T17:03:24.160457+0000","last_active":"2024-04-16T17:05:22.282754+0000","last_peered":"2024-04-16T17:05:22.282754+0000","last_clean":"2024-04-16T17:05:22.282754+0000","last_became_active":"2024-04-16T17:03:24.160195+0000","last_became_peered":"2024-04-16T17:03:24.160195+0000","last_unstale":"2024-04-16T17:05:22.282754+0000","last_undegraded":"2024-04-16T17:05:22.282754+0000","last_fullsized":"2024-04-16T17:05:22.282754+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:23:07.064539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"92'8","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.512907+0000","last_change":"2024-04-16T17:03:24.991249+0000","last_active":"2024-04-16T17:05:22.512907+0000","last_peered":"2024-04-16T17:05:22.512907+0000","last_clean":"2024-04-16T17:05:22.512907+0000","last_became_active":"2024-04-16T17:03:24.990927+0000","last_became_peered":"2024-04-16T17:03:24.990927+0000","last_unstale":"2024-04-16T17:05:22.512907+0000","last_undegraded":"2024-04-16T17:05:22.512907+0000","last_fullsized":"2024-04-16T17:05:22.512907+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:36:39.975616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.283496+0000","last_change":"2024-04-16T17:03:29.253704+0000","last_active":"2024-04-16T17:05:22.283496+0000","last_peered":"2024-04-16T17:05:22.283496+0000","last_clean":"2024-04-16T17:05:22.283496+0000","last_became_active":"2024-04-16T17:03:29.253334+0000","last_became_peered":"2024-04-16T17:03:29.253334+0000","last_unstale":"2024-04-16T17:05:22.283496+0000","last_undegraded":"2024-04-16T17:05:22.283496+0000","last_fullsized":"2024-04-16T17:05:22.283496+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:19:12.611731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"90'2","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.288474+0000","last_change":"2024-04-16T17:03:24.158875+0000","last_active":"2024-04-16T17:05:22.288474+0000","last_peered":"2024-04-16T17:05:22.288474+0000","last_clean":"2024-04-16T17:05:22.288474+0000","last_became_active":"2024-04-16T17:03:24.158465+0000","last_became_peered":"2024-04-16T17:03:24.158465+0000","last_unstale":"2024-04-16T17:05:22.288474+0000","last_undegraded":"2024-04-16T17:05:22.288474+0000","last_fullsized":"2024-04-16T17:05:22.288474+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:21:29.882251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"92'8","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.516011+0000","last_change":"2024-04-16T17:03:24.994476+0000","last_active":"2024-04-16T17:05:22.516011+0000","last_peered":"2024-04-16T17:05:22.516011+0000","last_clean":"2024-04-16T17:05:22.516011+0000","last_became_active":"2024-04-16T17:03:24.993532+0000","last_became_peered":"2024-04-16T17:03:24.993532+0000","last_unstale":"2024-04-16T17:05:22.516011+0000","last_undegraded":"2024-04-16T17:05:22.516011+0000","last_fullsized":"2024-04-16T17:05:22.516011+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:04:44.062444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.510439+0000","last_change":"2024-04-16T17:03:29.255446+0000","last_active":"2024-04-16T17:05:22.510439+0000","last_peered":"2024-04-16T17:05:22.510439+0000","last_clean":"2024-04-16T17:05:22.510439+0000","last_became_active":"2024-04-16T17:03:29.255051+0000","last_became_peered":"2024-04-16T17:03:29.255051+0000","last_unstale":"2024-04-16T17:05:22.510439+0000","last_undegraded":"2024-04-16T17:05:22.510439+0000","last_fullsized":"2024-04-16T17:05:22.510439+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:24:04.946342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"90'1","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.512386+0000","last_change":"2024-04-16T17:03:24.154904+0000","last_active":"2024-04-16T17:05:22.512386+0000","last_peered":"2024-04-16T17:05:22.512386+0000","last_clean":"2024-04-16T17:05:22.512386+0000","last_became_active":"2024-04-16T17:03:24.154617+0000","last_became_peered":"2024-04-16T17:03:24.154617+0000","last_unstale":"2024-04-16T17:05:22.512386+0000","last_undegraded":"2024-04-16T17:05:22.512386+0000","last_fullsized":"2024-04-16T17:05:22.512386+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:35:55.498855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"92'11","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.284812+0000","last_change":"2024-04-16T17:03:24.979908+0000","last_active":"2024-04-16T17:05:22.284812+0000","last_peered":"2024-04-16T17:05:22.284812+0000","last_clean":"2024-04-16T17:05:22.284812+0000","last_became_active":"2024-04-16T17:03:24.979501+0000","last_became_peered":"2024-04-16T17:03:24.979501+0000","last_unstale":"2024-04-16T17:05:22.284812+0000","last_undegraded":"2024-04-16T17:05:22.284812+0000","last_fullsized":"2024-04-16T17:05:22.284812+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:51:05.421459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.516308+0000","last_change":"2024-04-16T17:03:29.258809+0000","last_active":"2024-04-16T17:05:22.516308+0000","last_peered":"2024-04-16T17:05:22.516308+0000","last_clean":"2024-04-16T17:05:22.516308+0000","last_became_active":"2024-04-16T17:03:29.256629+0000","last_became_peered":"2024-04-16T17:03:29.256629+0000","last_unstale":"2024-04-16T17:05:22.516308+0000","last_undegraded":"2024-04-16T17:05:22.516308+0000","last_fullsized":"2024-04-16T17:05:22.516308+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:46:51.634944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"92'4","reported_seq":141,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:37.538079+0000","last_change":"2024-04-16T17:03:24.153774+0000","last_active":"2024-04-16T17:05:37.538079+0000","last_peered":"2024-04-16T17:05:37.538079+0000","last_clean":"2024-04-16T17:05:37.538079+0000","last_became_active":"2024-04-16T17:03:24.153418+0000","last_became_peered":"2024-04-16T17:03:24.153418+0000","last_unstale":"2024-04-16T17:05:37.538079+0000","last_undegraded":"2024-04-16T17:05:37.538079+0000","last_fullsized":"2024-04-16T17:05:37.538079+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:58:36.093789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":106,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"92'10","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.511791+0000","last_change":"2024-04-16T17:03:24.962838+0000","last_active":"2024-04-16T17:05:22.511791+0000","last_peered":"2024-04-16T17:05:22.511791+0000","last_clean":"2024-04-16T17:05:22.511791+0000","last_became_active":"2024-04-16T17:03:24.960163+0000","last_became_peered":"2024-04-16T17:03:24.960163+0000","last_unstale":"2024-04-16T17:05:22.511791+0000","last_undegraded":"2024-04-16T17:05:22.511791+0000","last_fullsized":"2024-04-16T17:05:22.511791+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:48:18.723499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.284550+0000","last_change":"2024-04-16T17:03:29.251177+0000","last_active":"2024-04-16T17:05:22.284550+0000","last_peered":"2024-04-16T17:05:22.284550+0000","last_clean":"2024-04-16T17:05:22.284550+0000","last_became_active":"2024-04-16T17:03:29.250657+0000","last_became_peered":"2024-04-16T17:03:29.250657+0000","last_unstale":"2024-04-16T17:05:22.284550+0000","last_undegraded":"2024-04-16T17:05:22.284550+0000","last_fullsized":"2024-04-16T17:05:22.284550+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:16:47.217193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"90'2","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.510573+0000","last_change":"2024-04-16T17:03:24.154245+0000","last_active":"2024-04-16T17:05:22.510573+0000","last_peered":"2024-04-16T17:05:22.510573+0000","last_clean":"2024-04-16T17:05:22.510573+0000","last_became_active":"2024-04-16T17:03:24.153297+0000","last_became_peered":"2024-04-16T17:03:24.153297+0000","last_unstale":"2024-04-16T17:05:22.510573+0000","last_undegraded":"2024-04-16T17:05:22.510573+0000","last_fullsized":"2024-04-16T17:05:22.510573+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:23:49.490238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"92'9","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.511056+0000","last_change":"2024-04-16T17:03:24.984572+0000","last_active":"2024-04-16T17:05:22.511056+0000","last_peered":"2024-04-16T17:05:22.511056+0000","last_clean":"2024-04-16T17:05:22.511056+0000","last_became_active":"2024-04-16T17:03:24.983588+0000","last_became_peered":"2024-04-16T17:03:24.983588+0000","last_unstale":"2024-04-16T17:05:22.511056+0000","last_undegraded":"2024-04-16T17:05:22.511056+0000","last_fullsized":"2024-04-16T17:05:22.511056+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:41:24.704602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.512602+0000","last_change":"2024-04-16T17:03:29.245129+0000","last_active":"2024-04-16T17:05:22.512602+0000","last_peered":"2024-04-16T17:05:22.512602+0000","last_clean":"2024-04-16T17:05:22.512602+0000","last_became_active":"2024-04-16T17:03:29.244720+0000","last_became_peered":"2024-04-16T17:03:29.244720+0000","last_unstale":"2024-04-16T17:05:22.512602+0000","last_undegraded":"2024-04-16T17:05:22.512602+0000","last_fullsized":"2024-04-16T17:05:22.512602+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:55:26.612176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"90'4","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.512809+0000","last_change":"2024-04-16T17:03:24.165248+0000","last_active":"2024-04-16T17:05:22.512809+0000","last_peered":"2024-04-16T17:05:22.512809+0000","last_clean":"2024-04-16T17:05:22.512809+0000","last_became_active":"2024-04-16T17:03:24.164915+0000","last_became_peered":"2024-04-16T17:03:24.164915+0000","last_unstale":"2024-04-16T17:05:22.512809+0000","last_undegraded":"2024-04-16T17:05:22.512809+0000","last_fullsized":"2024-04-16T17:05:22.512809+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:04:47.207554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"92'11","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.510492+0000","last_change":"2024-04-16T17:03:24.967292+0000","last_active":"2024-04-16T17:05:22.510492+0000","last_peered":"2024-04-16T17:05:22.510492+0000","last_clean":"2024-04-16T17:05:22.510492+0000","last_became_active":"2024-04-16T17:03:24.966952+0000","last_became_peered":"2024-04-16T17:03:24.966952+0000","last_unstale":"2024-04-16T17:05:22.510492+0000","last_undegraded":"2024-04-16T17:05:22.510492+0000","last_fullsized":"2024-04-16T17:05:22.510492+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:42:54.969478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.285084+0000","last_change":"2024-04-16T17:03:29.257836+0000","last_active":"2024-04-16T17:05:22.285084+0000","last_peered":"2024-04-16T17:05:22.285084+0000","last_clean":"2024-04-16T17:05:22.285084+0000","last_became_active":"2024-04-16T17:03:29.257409+0000","last_became_peered":"2024-04-16T17:03:29.257409+0000","last_unstale":"2024-04-16T17:05:22.285084+0000","last_undegraded":"2024-04-16T17:05:22.285084+0000","last_fullsized":"2024-04-16T17:05:22.285084+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:25:21.288266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"92'36","reported_seq":289,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:38.539565+0000","last_change":"2024-04-16T17:03:24.150943+0000","last_active":"2024-04-16T17:05:38.539565+0000","last_peered":"2024-04-16T17:05:38.539565+0000","last_clean":"2024-04-16T17:05:38.539565+0000","last_became_active":"2024-04-16T17:03:24.150563+0000","last_became_peered":"2024-04-16T17:03:24.150563+0000","last_unstale":"2024-04-16T17:05:38.539565+0000","last_undegraded":"2024-04-16T17:05:38.539565+0000","last_fullsized":"2024-04-16T17:05:38.539565+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:05:07.380440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":63605,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":222,"num_read_kb":174,"num_write":37,"num_write_kb":80,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"92'6","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.285005+0000","last_change":"2024-04-16T17:03:24.966393+0000","last_active":"2024-04-16T17:05:22.285005+0000","last_peered":"2024-04-16T17:05:22.285005+0000","last_clean":"2024-04-16T17:05:22.285005+0000","last_became_active":"2024-04-16T17:03:24.966016+0000","last_became_peered":"2024-04-16T17:03:24.966016+0000","last_unstale":"2024-04-16T17:05:22.285005+0000","last_undegraded":"2024-04-16T17:05:22.285005+0000","last_fullsized":"2024-04-16T17:05:22.285005+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:30:25.203262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.513291+0000","last_change":"2024-04-16T17:03:29.265371+0000","last_active":"2024-04-16T17:05:22.513291+0000","last_peered":"2024-04-16T17:05:22.513291+0000","last_clean":"2024-04-16T17:05:22.513291+0000","last_became_active":"2024-04-16T17:03:29.264933+0000","last_became_peered":"2024-04-16T17:03:29.264933+0000","last_unstale":"2024-04-16T17:05:22.513291+0000","last_undegraded":"2024-04-16T17:05:22.513291+0000","last_fullsized":"2024-04-16T17:05:22.513291+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:50:18.203015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"90'6","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.282793+0000","last_change":"2024-04-16T17:04:21.526796+0000","last_active":"2024-04-16T17:05:22.282793+0000","last_peered":"2024-04-16T17:05:22.282793+0000","last_clean":"2024-04-16T17:05:22.282793+0000","last_became_active":"2024-04-16T17:04:21.508364+0000","last_became_peered":"2024-04-16T17:04:21.508364+0000","last_unstale":"2024-04-16T17:05:22.282793+0000","last_undegraded":"2024-04-16T17:05:22.282793+0000","last_fullsized":"2024-04-16T17:05:22.282793+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:45:42.726648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"92'9","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.289036+0000","last_change":"2024-04-16T17:03:24.986228+0000","last_active":"2024-04-16T17:05:22.289036+0000","last_peered":"2024-04-16T17:05:22.289036+0000","last_clean":"2024-04-16T17:05:22.289036+0000","last_became_active":"2024-04-16T17:03:24.985916+0000","last_became_peered":"2024-04-16T17:03:24.985916+0000","last_unstale":"2024-04-16T17:05:22.289036+0000","last_undegraded":"2024-04-16T17:05:22.289036+0000","last_fullsized":"2024-04-16T17:05:22.289036+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:23:54.309203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.282938+0000","last_change":"2024-04-16T17:03:29.264636+0000","last_active":"2024-04-16T17:05:22.282938+0000","last_peered":"2024-04-16T17:05:22.282938+0000","last_clean":"2024-04-16T17:05:22.282938+0000","last_became_active":"2024-04-16T17:03:29.264117+0000","last_became_peered":"2024-04-16T17:03:29.264117+0000","last_unstale":"2024-04-16T17:05:22.282938+0000","last_undegraded":"2024-04-16T17:05:22.282938+0000","last_fullsized":"2024-04-16T17:05:22.282938+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:42:36.123154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"90'3","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.512886+0000","last_change":"2024-04-16T17:03:24.166844+0000","last_active":"2024-04-16T17:05:22.512886+0000","last_peered":"2024-04-16T17:05:22.512886+0000","last_clean":"2024-04-16T17:05:22.512886+0000","last_became_active":"2024-04-16T17:03:24.166479+0000","last_became_peered":"2024-04-16T17:03:24.166479+0000","last_unstale":"2024-04-16T17:05:22.512886+0000","last_undegraded":"2024-04-16T17:05:22.512886+0000","last_fullsized":"2024-04-16T17:05:22.512886+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:02:20.139865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"92'15","reported_seq":44,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.510958+0000","last_change":"2024-04-16T17:03:24.973057+0000","last_active":"2024-04-16T17:05:22.510958+0000","last_peered":"2024-04-16T17:05:22.510958+0000","last_clean":"2024-04-16T17:05:22.510958+0000","last_became_active":"2024-04-16T17:03:24.972731+0000","last_became_peered":"2024-04-16T17:03:24.972731+0000","last_unstale":"2024-04-16T17:05:22.510958+0000","last_undegraded":"2024-04-16T17:05:22.510958+0000","last_fullsized":"2024-04-16T17:05:22.510958+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:37:38.702919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.286263+0000","last_change":"2024-04-16T17:03:29.259167+0000","last_active":"2024-04-16T17:05:22.286263+0000","last_peered":"2024-04-16T17:05:22.286263+0000","last_clean":"2024-04-16T17:05:22.286263+0000","last_became_active":"2024-04-16T17:03:29.258737+0000","last_became_peered":"2024-04-16T17:03:29.258737+0000","last_unstale":"2024-04-16T17:05:22.286263+0000","last_undegraded":"2024-04-16T17:05:22.286263+0000","last_fullsized":"2024-04-16T17:05:22.286263+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:01:13.774037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"92'5","reported_seq":148,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:39.637849+0000","last_change":"2024-04-16T17:03:24.151064+0000","last_active":"2024-04-16T17:05:39.637849+0000","last_peered":"2024-04-16T17:05:39.637849+0000","last_clean":"2024-04-16T17:05:39.637849+0000","last_became_active":"2024-04-16T17:03:24.150745+0000","last_became_peered":"2024-04-16T17:03:24.150745+0000","last_unstale":"2024-04-16T17:05:39.637849+0000","last_undegraded":"2024-04-16T17:05:39.637849+0000","last_fullsized":"2024-04-16T17:05:39.637849+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:32:26.271008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":113,"num_read_kb":5,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"92'6","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.510687+0000","last_change":"2024-04-16T17:03:24.971948+0000","last_active":"2024-04-16T17:05:22.510687+0000","last_peered":"2024-04-16T17:05:22.510687+0000","last_clean":"2024-04-16T17:05:22.510687+0000","last_became_active":"2024-04-16T17:03:24.971530+0000","last_became_peered":"2024-04-16T17:03:24.971530+0000","last_unstale":"2024-04-16T17:05:22.510687+0000","last_undegraded":"2024-04-16T17:05:22.510687+0000","last_fullsized":"2024-04-16T17:05:22.510687+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:03:29.286160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.290269+0000","last_change":"2024-04-16T17:03:29.264821+0000","last_active":"2024-04-16T17:05:22.290269+0000","last_peered":"2024-04-16T17:05:22.290269+0000","last_clean":"2024-04-16T17:05:22.290269+0000","last_became_active":"2024-04-16T17:03:29.264287+0000","last_became_peered":"2024-04-16T17:03:29.264287+0000","last_unstale":"2024-04-16T17:05:22.290269+0000","last_undegraded":"2024-04-16T17:05:22.290269+0000","last_fullsized":"2024-04-16T17:05:22.290269+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:33:28.400663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"90'15","reported_seq":47,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.284636+0000","last_change":"2024-04-16T17:03:24.158844+0000","last_active":"2024-04-16T17:05:22.284636+0000","last_peered":"2024-04-16T17:05:22.284636+0000","last_clean":"2024-04-16T17:05:22.284636+0000","last_became_active":"2024-04-16T17:03:24.158537+0000","last_became_peered":"2024-04-16T17:03:24.158537+0000","last_unstale":"2024-04-16T17:05:22.284636+0000","last_undegraded":"2024-04-16T17:05:22.284636+0000","last_fullsized":"2024-04-16T17:05:22.284636+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:42:22.772452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"92'6","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.512660+0000","last_change":"2024-04-16T17:03:24.962522+0000","last_active":"2024-04-16T17:05:22.512660+0000","last_peered":"2024-04-16T17:05:22.512660+0000","last_clean":"2024-04-16T17:05:22.512660+0000","last_became_active":"2024-04-16T17:03:24.962169+0000","last_became_peered":"2024-04-16T17:03:24.962169+0000","last_unstale":"2024-04-16T17:05:22.512660+0000","last_undegraded":"2024-04-16T17:05:22.512660+0000","last_fullsized":"2024-04-16T17:05:22.512660+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:28:17.147491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.512683+0000","last_change":"2024-04-16T17:03:29.259539+0000","last_active":"2024-04-16T17:05:22.512683+0000","last_peered":"2024-04-16T17:05:22.512683+0000","last_clean":"2024-04-16T17:05:22.512683+0000","last_became_active":"2024-04-16T17:03:29.259123+0000","last_became_peered":"2024-04-16T17:03:29.259123+0000","last_unstale":"2024-04-16T17:05:22.512683+0000","last_undegraded":"2024-04-16T17:05:22.512683+0000","last_fullsized":"2024-04-16T17:05:22.512683+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:15:17.142636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"90'2","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.285316+0000","last_change":"2024-04-16T17:03:24.147526+0000","last_active":"2024-04-16T17:05:22.285316+0000","last_peered":"2024-04-16T17:05:22.285316+0000","last_clean":"2024-04-16T17:05:22.285316+0000","last_became_active":"2024-04-16T17:03:24.147119+0000","last_became_peered":"2024-04-16T17:03:24.147119+0000","last_unstale":"2024-04-16T17:05:22.285316+0000","last_undegraded":"2024-04-16T17:05:22.285316+0000","last_fullsized":"2024-04-16T17:05:22.285316+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:30:38.742122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"92'5","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.283120+0000","last_change":"2024-04-16T17:04:21.507113+0000","last_active":"2024-04-16T17:05:22.283120+0000","last_peered":"2024-04-16T17:05:22.283120+0000","last_clean":"2024-04-16T17:05:22.283120+0000","last_became_active":"2024-04-16T17:04:21.506680+0000","last_became_peered":"2024-04-16T17:04:21.506680+0000","last_unstale":"2024-04-16T17:05:22.283120+0000","last_undegraded":"2024-04-16T17:05:22.283120+0000","last_fullsized":"2024-04-16T17:05:22.283120+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:08:47.971317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,8],"acting":[11,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":30,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.512982+0000","last_change":"2024-04-16T17:05:22.511966+0000","last_active":"2024-04-16T17:05:22.512982+0000","last_peered":"2024-04-16T17:05:22.512982+0000","last_clean":"2024-04-16T17:05:22.512982+0000","last_became_active":"2024-04-16T17:05:22.511512+0000","last_became_peered":"2024-04-16T17:05:22.511512+0000","last_unstale":"2024-04-16T17:05:22.512982+0000","last_undegraded":"2024-04-16T17:05:22.512982+0000","last_fullsized":"2024-04-16T17:05:22.512982+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:14:41.291499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,1],"acting":[8,10,11,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.285749+0000","last_change":"2024-04-16T17:03:24.162105+0000","last_active":"2024-04-16T17:05:22.285749+0000","last_peered":"2024-04-16T17:05:22.285749+0000","last_clean":"2024-04-16T17:05:22.285749+0000","last_became_active":"2024-04-16T17:03:24.161631+0000","last_became_peered":"2024-04-16T17:03:24.161631+0000","last_unstale":"2024-04-16T17:05:22.285749+0000","last_undegraded":"2024-04-16T17:05:22.285749+0000","last_fullsized":"2024-04-16T17:05:22.285749+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:22:33.677848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"92'4","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.283327+0000","last_change":"2024-04-16T17:03:24.978253+0000","last_active":"2024-04-16T17:05:22.283327+0000","last_peered":"2024-04-16T17:05:22.283327+0000","last_clean":"2024-04-16T17:05:22.283327+0000","last_became_active":"2024-04-16T17:03:24.977956+0000","last_became_peered":"2024-04-16T17:03:24.977956+0000","last_unstale":"2024-04-16T17:05:22.283327+0000","last_undegraded":"2024-04-16T17:05:22.283327+0000","last_fullsized":"2024-04-16T17:05:22.283327+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:59:45.102383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.289715+0000","last_change":"2024-04-16T17:03:29.265312+0000","last_active":"2024-04-16T17:05:22.289715+0000","last_peered":"2024-04-16T17:05:22.289715+0000","last_clean":"2024-04-16T17:05:22.289715+0000","last_became_active":"2024-04-16T17:03:29.264891+0000","last_became_peered":"2024-04-16T17:03:29.264891+0000","last_unstale":"2024-04-16T17:05:22.289715+0000","last_undegraded":"2024-04-16T17:05:22.289715+0000","last_fullsized":"2024-04-16T17:05:22.289715+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:26:40.088127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"90'1","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.510989+0000","last_change":"2024-04-16T17:03:24.170935+0000","last_active":"2024-04-16T17:05:22.510989+0000","last_peered":"2024-04-16T17:05:22.510989+0000","last_clean":"2024-04-16T17:05:22.510989+0000","last_became_active":"2024-04-16T17:03:24.170609+0000","last_became_peered":"2024-04-16T17:03:24.170609+0000","last_unstale":"2024-04-16T17:05:22.510989+0000","last_undegraded":"2024-04-16T17:05:22.510989+0000","last_fullsized":"2024-04-16T17:05:22.510989+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:18:33.516309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"92'7","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.283209+0000","last_change":"2024-04-16T17:03:24.985434+0000","last_active":"2024-04-16T17:05:22.283209+0000","last_peered":"2024-04-16T17:05:22.283209+0000","last_clean":"2024-04-16T17:05:22.283209+0000","last_became_active":"2024-04-16T17:03:24.984388+0000","last_became_peered":"2024-04-16T17:03:24.984388+0000","last_unstale":"2024-04-16T17:05:22.283209+0000","last_undegraded":"2024-04-16T17:05:22.283209+0000","last_fullsized":"2024-04-16T17:05:22.283209+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:43:26.355572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":30,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.518211+0000","last_change":"2024-04-16T17:05:22.516267+0000","last_active":"2024-04-16T17:05:22.518211+0000","last_peered":"2024-04-16T17:05:22.518211+0000","last_clean":"2024-04-16T17:05:22.518211+0000","last_became_active":"2024-04-16T17:05:22.513637+0000","last_became_peered":"2024-04-16T17:05:22.513637+0000","last_unstale":"2024-04-16T17:05:22.518211+0000","last_undegraded":"2024-04-16T17:05:22.518211+0000","last_fullsized":"2024-04-16T17:05:22.518211+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:39:47.975735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,8,10,0],"acting":[6,8,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"92'8","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.512569+0000","last_change":"2024-04-16T17:03:24.962786+0000","last_active":"2024-04-16T17:05:22.512569+0000","last_peered":"2024-04-16T17:05:22.512569+0000","last_clean":"2024-04-16T17:05:22.512569+0000","last_became_active":"2024-04-16T17:03:24.961704+0000","last_became_peered":"2024-04-16T17:03:24.961704+0000","last_unstale":"2024-04-16T17:05:22.512569+0000","last_undegraded":"2024-04-16T17:05:22.512569+0000","last_fullsized":"2024-04-16T17:05:22.512569+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:22:27.957607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"92'5","reported_seq":145,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:39.637020+0000","last_change":"2024-04-16T17:03:24.175617+0000","last_active":"2024-04-16T17:05:39.637020+0000","last_peered":"2024-04-16T17:05:39.637020+0000","last_clean":"2024-04-16T17:05:39.637020+0000","last_became_active":"2024-04-16T17:03:24.175272+0000","last_became_peered":"2024-04-16T17:03:24.175272+0000","last_unstale":"2024-04-16T17:05:39.637020+0000","last_undegraded":"2024-04-16T17:05:39.637020+0000","last_fullsized":"2024-04-16T17:05:39.637020+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:27:44.738030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":108,"num_read_kb":108,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"90'1","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.511066+0000","last_change":"2024-04-16T17:04:21.512033+0000","last_active":"2024-04-16T17:05:22.511066+0000","last_peered":"2024-04-16T17:05:22.511066+0000","last_clean":"2024-04-16T17:05:22.511066+0000","last_became_active":"2024-04-16T17:04:21.506718+0000","last_became_peered":"2024-04-16T17:04:21.506718+0000","last_unstale":"2024-04-16T17:05:22.511066+0000","last_undegraded":"2024-04-16T17:05:22.511066+0000","last_fullsized":"2024-04-16T17:05:22.511066+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:53:48.953887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,10],"acting":[7,5,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"92'8","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.517270+0000","last_change":"2024-04-16T17:03:24.983629+0000","last_active":"2024-04-16T17:05:22.517270+0000","last_peered":"2024-04-16T17:05:22.517270+0000","last_clean":"2024-04-16T17:05:22.517270+0000","last_became_active":"2024-04-16T17:03:24.979969+0000","last_became_peered":"2024-04-16T17:03:24.979969+0000","last_unstale":"2024-04-16T17:05:22.517270+0000","last_undegraded":"2024-04-16T17:05:22.517270+0000","last_fullsized":"2024-04-16T17:05:22.517270+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:01:24.391732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"90'3","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.512560+0000","last_change":"2024-04-16T17:03:24.152475+0000","last_active":"2024-04-16T17:05:22.512560+0000","last_peered":"2024-04-16T17:05:22.512560+0000","last_clean":"2024-04-16T17:05:22.512560+0000","last_became_active":"2024-04-16T17:03:24.152198+0000","last_became_peered":"2024-04-16T17:03:24.152198+0000","last_unstale":"2024-04-16T17:05:22.512560+0000","last_undegraded":"2024-04-16T17:05:22.512560+0000","last_fullsized":"2024-04-16T17:05:22.512560+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:49:33.914325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":364,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.510449+0000","last_change":"2024-04-16T17:02:02.111335+0000","last_active":"2024-04-16T17:05:22.510449+0000","last_peered":"2024-04-16T17:05:22.510449+0000","last_clean":"2024-04-16T17:05:22.510449+0000","last_became_active":"2024-04-16T17:02:02.110928+0000","last_became_peered":"2024-04-16T17:02:02.110928+0000","last_unstale":"2024-04-16T17:05:22.510449+0000","last_undegraded":"2024-04-16T17:05:22.510449+0000","last_fullsized":"2024-04-16T17:05:22.510449+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T16:58:07.081753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T16:58:07.081753+0000","last_clean_scrub_stamp":"2024-04-16T16:58:07.081753+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:27:02.865664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.513248+0000","last_change":"2024-04-16T17:03:29.301691+0000","last_active":"2024-04-16T17:05:22.513248+0000","last_peered":"2024-04-16T17:05:22.513248+0000","last_clean":"2024-04-16T17:05:22.513248+0000","last_became_active":"2024-04-16T17:03:29.301277+0000","last_became_peered":"2024-04-16T17:03:29.301277+0000","last_unstale":"2024-04-16T17:05:22.513248+0000","last_undegraded":"2024-04-16T17:05:22.513248+0000","last_fullsized":"2024-04-16T17:05:22.513248+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:45:16.727610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"92'5","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.287991+0000","last_change":"2024-04-16T17:03:24.970939+0000","last_active":"2024-04-16T17:05:22.287991+0000","last_peered":"2024-04-16T17:05:22.287991+0000","last_clean":"2024-04-16T17:05:22.287991+0000","last_became_active":"2024-04-16T17:03:24.970581+0000","last_became_peered":"2024-04-16T17:03:24.970581+0000","last_unstale":"2024-04-16T17:05:22.287991+0000","last_undegraded":"2024-04-16T17:05:22.287991+0000","last_fullsized":"2024-04-16T17:05:22.287991+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:33:07.448597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"90'3","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.510631+0000","last_change":"2024-04-16T17:03:24.155454+0000","last_active":"2024-04-16T17:05:22.510631+0000","last_peered":"2024-04-16T17:05:22.510631+0000","last_clean":"2024-04-16T17:05:22.510631+0000","last_became_active":"2024-04-16T17:03:24.155155+0000","last_became_peered":"2024-04-16T17:03:24.155155+0000","last_unstale":"2024-04-16T17:05:22.510631+0000","last_undegraded":"2024-04-16T17:05:22.510631+0000","last_fullsized":"2024-04-16T17:05:22.510631+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:34:08.618002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.282951+0000","last_change":"2024-04-16T17:03:29.255538+0000","last_active":"2024-04-16T17:05:22.282951+0000","last_peered":"2024-04-16T17:05:22.282951+0000","last_clean":"2024-04-16T17:05:22.282951+0000","last_became_active":"2024-04-16T17:03:29.255107+0000","last_became_peered":"2024-04-16T17:03:29.255107+0000","last_unstale":"2024-04-16T17:05:22.282951+0000","last_undegraded":"2024-04-16T17:05:22.282951+0000","last_fullsized":"2024-04-16T17:05:22.282951+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:31:58.458641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"90'1","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.285190+0000","last_change":"2024-04-16T17:03:24.158816+0000","last_active":"2024-04-16T17:05:22.285190+0000","last_peered":"2024-04-16T17:05:22.285190+0000","last_clean":"2024-04-16T17:05:22.285190+0000","last_became_active":"2024-04-16T17:03:24.158518+0000","last_became_peered":"2024-04-16T17:03:24.158518+0000","last_unstale":"2024-04-16T17:05:22.285190+0000","last_undegraded":"2024-04-16T17:05:22.285190+0000","last_fullsized":"2024-04-16T17:05:22.285190+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:54:55.807975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"92'9","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.290107+0000","last_change":"2024-04-16T17:03:24.977190+0000","last_active":"2024-04-16T17:05:22.290107+0000","last_peered":"2024-04-16T17:05:22.290107+0000","last_clean":"2024-04-16T17:05:22.290107+0000","last_became_active":"2024-04-16T17:03:24.976840+0000","last_became_peered":"2024-04-16T17:03:24.976840+0000","last_unstale":"2024-04-16T17:05:22.290107+0000","last_undegraded":"2024-04-16T17:05:22.290107+0000","last_fullsized":"2024-04-16T17:05:22.290107+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:21:51.294150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.516089+0000","last_change":"2024-04-16T17:03:29.301423+0000","last_active":"2024-04-16T17:05:22.516089+0000","last_peered":"2024-04-16T17:05:22.516089+0000","last_clean":"2024-04-16T17:05:22.516089+0000","last_became_active":"2024-04-16T17:03:29.300888+0000","last_became_peered":"2024-04-16T17:03:29.300888+0000","last_unstale":"2024-04-16T17:05:22.516089+0000","last_undegraded":"2024-04-16T17:05:22.516089+0000","last_fullsized":"2024-04-16T17:05:22.516089+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:13:54.188090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"92'11","reported_seq":43,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.287802+0000","last_change":"2024-04-16T17:03:24.167643+0000","last_active":"2024-04-16T17:05:22.287802+0000","last_peered":"2024-04-16T17:05:22.287802+0000","last_clean":"2024-04-16T17:05:22.287802+0000","last_became_active":"2024-04-16T17:03:24.167103+0000","last_became_peered":"2024-04-16T17:03:24.167103+0000","last_unstale":"2024-04-16T17:05:22.287802+0000","last_undegraded":"2024-04-16T17:05:22.287802+0000","last_fullsized":"2024-04-16T17:05:22.287802+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:33:16.809371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"92'5","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.519200+0000","last_change":"2024-04-16T17:03:24.976981+0000","last_active":"2024-04-16T17:05:22.519200+0000","last_peered":"2024-04-16T17:05:22.519200+0000","last_clean":"2024-04-16T17:05:22.519200+0000","last_became_active":"2024-04-16T17:03:24.975385+0000","last_became_peered":"2024-04-16T17:03:24.975385+0000","last_unstale":"2024-04-16T17:05:22.519200+0000","last_undegraded":"2024-04-16T17:05:22.519200+0000","last_fullsized":"2024-04-16T17:05:22.519200+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:27:03.104838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.509345+0000","last_change":"2024-04-16T17:03:29.244767+0000","last_active":"2024-04-16T17:05:22.509345+0000","last_peered":"2024-04-16T17:05:22.509345+0000","last_clean":"2024-04-16T17:05:22.509345+0000","last_became_active":"2024-04-16T17:03:29.244205+0000","last_became_peered":"2024-04-16T17:03:29.244205+0000","last_unstale":"2024-04-16T17:05:22.509345+0000","last_undegraded":"2024-04-16T17:05:22.509345+0000","last_fullsized":"2024-04-16T17:05:22.509345+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:51:46.652387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"90'4","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.513054+0000","last_change":"2024-04-16T17:03:24.164916+0000","last_active":"2024-04-16T17:05:22.513054+0000","last_peered":"2024-04-16T17:05:22.513054+0000","last_clean":"2024-04-16T17:05:22.513054+0000","last_became_active":"2024-04-16T17:03:24.164648+0000","last_became_peered":"2024-04-16T17:03:24.164648+0000","last_unstale":"2024-04-16T17:05:22.513054+0000","last_undegraded":"2024-04-16T17:05:22.513054+0000","last_fullsized":"2024-04-16T17:05:22.513054+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:23:00.198785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"92'7","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.510837+0000","last_change":"2024-04-16T17:03:24.994393+0000","last_active":"2024-04-16T17:05:22.510837+0000","last_peered":"2024-04-16T17:05:22.510837+0000","last_clean":"2024-04-16T17:05:22.510837+0000","last_became_active":"2024-04-16T17:03:24.994002+0000","last_became_peered":"2024-04-16T17:03:24.994002+0000","last_unstale":"2024-04-16T17:05:22.510837+0000","last_undegraded":"2024-04-16T17:05:22.510837+0000","last_fullsized":"2024-04-16T17:05:22.510837+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:23:44.596781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.512808+0000","last_change":"2024-04-16T17:03:29.302009+0000","last_active":"2024-04-16T17:05:22.512808+0000","last_peered":"2024-04-16T17:05:22.512808+0000","last_clean":"2024-04-16T17:05:22.512808+0000","last_became_active":"2024-04-16T17:03:29.301183+0000","last_became_peered":"2024-04-16T17:03:29.301183+0000","last_unstale":"2024-04-16T17:05:22.512808+0000","last_undegraded":"2024-04-16T17:05:22.512808+0000","last_fullsized":"2024-04-16T17:05:22.512808+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:30:19.561875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"90'2","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.509922+0000","last_change":"2024-04-16T17:03:24.176829+0000","last_active":"2024-04-16T17:05:22.509922+0000","last_peered":"2024-04-16T17:05:22.509922+0000","last_clean":"2024-04-16T17:05:22.509922+0000","last_became_active":"2024-04-16T17:03:24.176555+0000","last_became_peered":"2024-04-16T17:03:24.176555+0000","last_unstale":"2024-04-16T17:05:22.509922+0000","last_undegraded":"2024-04-16T17:05:22.509922+0000","last_fullsized":"2024-04-16T17:05:22.509922+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:05:40.602150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"92'2","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.284722+0000","last_change":"2024-04-16T17:03:24.965746+0000","last_active":"2024-04-16T17:05:22.284722+0000","last_peered":"2024-04-16T17:05:22.284722+0000","last_clean":"2024-04-16T17:05:22.284722+0000","last_became_active":"2024-04-16T17:03:24.965241+0000","last_became_peered":"2024-04-16T17:03:24.965241+0000","last_unstale":"2024-04-16T17:05:22.284722+0000","last_undegraded":"2024-04-16T17:05:22.284722+0000","last_fullsized":"2024-04-16T17:05:22.284722+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:51:34.407565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.286457+0000","last_change":"2024-04-16T17:03:29.261441+0000","last_active":"2024-04-16T17:05:22.286457+0000","last_peered":"2024-04-16T17:05:22.286457+0000","last_clean":"2024-04-16T17:05:22.286457+0000","last_became_active":"2024-04-16T17:03:29.259538+0000","last_became_peered":"2024-04-16T17:03:29.259538+0000","last_unstale":"2024-04-16T17:05:22.286457+0000","last_undegraded":"2024-04-16T17:05:22.286457+0000","last_fullsized":"2024-04-16T17:05:22.286457+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:12:27.092053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"90'3","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.516764+0000","last_change":"2024-04-16T17:03:24.172053+0000","last_active":"2024-04-16T17:05:22.516764+0000","last_peered":"2024-04-16T17:05:22.516764+0000","last_clean":"2024-04-16T17:05:22.516764+0000","last_became_active":"2024-04-16T17:03:24.171709+0000","last_became_peered":"2024-04-16T17:03:24.171709+0000","last_unstale":"2024-04-16T17:05:22.516764+0000","last_undegraded":"2024-04-16T17:05:22.516764+0000","last_fullsized":"2024-04-16T17:05:22.516764+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:40:12.307756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"92'9","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.282872+0000","last_change":"2024-04-16T17:03:24.984267+0000","last_active":"2024-04-16T17:05:22.282872+0000","last_peered":"2024-04-16T17:05:22.282872+0000","last_clean":"2024-04-16T17:05:22.282872+0000","last_became_active":"2024-04-16T17:03:24.983888+0000","last_became_peered":"2024-04-16T17:03:24.983888+0000","last_unstale":"2024-04-16T17:05:22.282872+0000","last_undegraded":"2024-04-16T17:05:22.282872+0000","last_fullsized":"2024-04-16T17:05:22.282872+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:16:23.814917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"90'1","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.512450+0000","last_change":"2024-04-16T17:03:29.251716+0000","last_active":"2024-04-16T17:05:22.512450+0000","last_peered":"2024-04-16T17:05:22.512450+0000","last_clean":"2024-04-16T17:05:22.512450+0000","last_became_active":"2024-04-16T17:03:29.251317+0000","last_became_peered":"2024-04-16T17:03:29.251317+0000","last_unstale":"2024-04-16T17:05:22.512450+0000","last_undegraded":"2024-04-16T17:05:22.512450+0000","last_fullsized":"2024-04-16T17:05:22.512450+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:48:54.680195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"90'1","reported_seq":19,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.282769+0000","last_change":"2024-04-16T17:04:21.515306+0000","last_active":"2024-04-16T17:05:22.282769+0000","last_peered":"2024-04-16T17:05:22.282769+0000","last_clean":"2024-04-16T17:05:22.282769+0000","last_became_active":"2024-04-16T17:04:21.508377+0000","last_became_peered":"2024-04-16T17:04:21.508377+0000","last_unstale":"2024-04-16T17:05:22.282769+0000","last_undegraded":"2024-04-16T17:05:22.282769+0000","last_fullsized":"2024-04-16T17:05:22.282769+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:28:25.000406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"92'8","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.518294+0000","last_change":"2024-04-16T17:03:24.995547+0000","last_active":"2024-04-16T17:05:22.518294+0000","last_peered":"2024-04-16T17:05:22.518294+0000","last_clean":"2024-04-16T17:05:22.518294+0000","last_became_active":"2024-04-16T17:03:24.995096+0000","last_became_peered":"2024-04-16T17:03:24.995096+0000","last_unstale":"2024-04-16T17:05:22.518294+0000","last_undegraded":"2024-04-16T17:05:22.518294+0000","last_fullsized":"2024-04-16T17:05:22.518294+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:54:51.193117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.512835+0000","last_change":"2024-04-16T17:03:29.301526+0000","last_active":"2024-04-16T17:05:22.512835+0000","last_peered":"2024-04-16T17:05:22.512835+0000","last_clean":"2024-04-16T17:05:22.512835+0000","last_became_active":"2024-04-16T17:03:29.300808+0000","last_became_peered":"2024-04-16T17:03:29.300808+0000","last_unstale":"2024-04-16T17:05:22.512835+0000","last_undegraded":"2024-04-16T17:05:22.512835+0000","last_fullsized":"2024-04-16T17:05:22.512835+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:01:14.717548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"90'4","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.284776+0000","last_change":"2024-04-16T17:03:24.152645+0000","last_active":"2024-04-16T17:05:22.284776+0000","last_peered":"2024-04-16T17:05:22.284776+0000","last_clean":"2024-04-16T17:05:22.284776+0000","last_became_active":"2024-04-16T17:03:24.152281+0000","last_became_peered":"2024-04-16T17:03:24.152281+0000","last_unstale":"2024-04-16T17:05:22.284776+0000","last_undegraded":"2024-04-16T17:05:22.284776+0000","last_fullsized":"2024-04-16T17:05:22.284776+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:33:51.377421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"92'5","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.511231+0000","last_change":"2024-04-16T17:03:24.979827+0000","last_active":"2024-04-16T17:05:22.511231+0000","last_peered":"2024-04-16T17:05:22.511231+0000","last_clean":"2024-04-16T17:05:22.511231+0000","last_became_active":"2024-04-16T17:03:24.979452+0000","last_became_peered":"2024-04-16T17:03:24.979452+0000","last_unstale":"2024-04-16T17:05:22.511231+0000","last_undegraded":"2024-04-16T17:05:22.511231+0000","last_fullsized":"2024-04-16T17:05:22.511231+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:07:09.259366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.510109+0000","last_change":"2024-04-16T17:03:29.255519+0000","last_active":"2024-04-16T17:05:22.510109+0000","last_peered":"2024-04-16T17:05:22.510109+0000","last_clean":"2024-04-16T17:05:22.510109+0000","last_became_active":"2024-04-16T17:03:29.255104+0000","last_became_peered":"2024-04-16T17:03:29.255104+0000","last_unstale":"2024-04-16T17:05:22.510109+0000","last_undegraded":"2024-04-16T17:05:22.510109+0000","last_fullsized":"2024-04-16T17:05:22.510109+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:27:34.496199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"90'3","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.509653+0000","last_change":"2024-04-16T17:03:24.161145+0000","last_active":"2024-04-16T17:05:22.509653+0000","last_peered":"2024-04-16T17:05:22.509653+0000","last_clean":"2024-04-16T17:05:22.509653+0000","last_became_active":"2024-04-16T17:03:24.160824+0000","last_became_peered":"2024-04-16T17:03:24.160824+0000","last_unstale":"2024-04-16T17:05:22.509653+0000","last_undegraded":"2024-04-16T17:05:22.509653+0000","last_fullsized":"2024-04-16T17:05:22.509653+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:02:48.805638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"92'7","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.511461+0000","last_change":"2024-04-16T17:03:24.966513+0000","last_active":"2024-04-16T17:05:22.511461+0000","last_peered":"2024-04-16T17:05:22.511461+0000","last_clean":"2024-04-16T17:05:22.511461+0000","last_became_active":"2024-04-16T17:03:24.966074+0000","last_became_peered":"2024-04-16T17:03:24.966074+0000","last_unstale":"2024-04-16T17:05:22.511461+0000","last_undegraded":"2024-04-16T17:05:22.511461+0000","last_fullsized":"2024-04-16T17:05:22.511461+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:50:21.579901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.509016+0000","last_change":"2024-04-16T17:04:22.150786+0000","last_active":"2024-04-16T17:05:22.509016+0000","last_peered":"2024-04-16T17:05:22.509016+0000","last_clean":"2024-04-16T17:05:22.509016+0000","last_became_active":"2024-04-16T17:04:22.149344+0000","last_became_peered":"2024-04-16T17:04:22.149344+0000","last_unstale":"2024-04-16T17:05:22.509016+0000","last_undegraded":"2024-04-16T17:05:22.509016+0000","last_fullsized":"2024-04-16T17:05:22.509016+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:18:49.421095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,1,4],"acting":[0,8,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"85'1","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.513257+0000","last_change":"2024-04-16T17:03:24.158032+0000","last_active":"2024-04-16T17:05:22.513257+0000","last_peered":"2024-04-16T17:05:22.513257+0000","last_clean":"2024-04-16T17:05:22.513257+0000","last_became_active":"2024-04-16T17:03:24.157689+0000","last_became_peered":"2024-04-16T17:03:24.157689+0000","last_unstale":"2024-04-16T17:05:22.513257+0000","last_undegraded":"2024-04-16T17:05:22.513257+0000","last_fullsized":"2024-04-16T17:05:22.513257+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:38:18.240889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"92'6","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.511404+0000","last_change":"2024-04-16T17:04:21.519999+0000","last_active":"2024-04-16T17:05:22.511404+0000","last_peered":"2024-04-16T17:05:22.511404+0000","last_clean":"2024-04-16T17:05:22.511404+0000","last_became_active":"2024-04-16T17:04:21.519621+0000","last_became_peered":"2024-04-16T17:04:21.519621+0000","last_unstale":"2024-04-16T17:05:22.511404+0000","last_undegraded":"2024-04-16T17:05:22.511404+0000","last_fullsized":"2024-04-16T17:05:22.511404+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:50:13.266369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.512211+0000","last_change":"2024-04-16T17:03:29.245176+0000","last_active":"2024-04-16T17:05:22.512211+0000","last_peered":"2024-04-16T17:05:22.512211+0000","last_clean":"2024-04-16T17:05:22.512211+0000","last_became_active":"2024-04-16T17:03:29.244763+0000","last_became_peered":"2024-04-16T17:03:29.244763+0000","last_unstale":"2024-04-16T17:05:22.512211+0000","last_undegraded":"2024-04-16T17:05:22.512211+0000","last_fullsized":"2024-04-16T17:05:22.512211+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:52:30.381417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.290262+0000","last_change":"2024-04-16T17:03:24.168068+0000","last_active":"2024-04-16T17:05:22.290262+0000","last_peered":"2024-04-16T17:05:22.290262+0000","last_clean":"2024-04-16T17:05:22.290262+0000","last_became_active":"2024-04-16T17:03:24.167694+0000","last_became_peered":"2024-04-16T17:03:24.167694+0000","last_unstale":"2024-04-16T17:05:22.290262+0000","last_undegraded":"2024-04-16T17:05:22.290262+0000","last_fullsized":"2024-04-16T17:05:22.290262+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:13:35.692306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"92'9","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.510920+0000","last_change":"2024-04-16T17:03:24.966145+0000","last_active":"2024-04-16T17:05:22.510920+0000","last_peered":"2024-04-16T17:05:22.510920+0000","last_clean":"2024-04-16T17:05:22.510920+0000","last_became_active":"2024-04-16T17:03:24.965813+0000","last_became_peered":"2024-04-16T17:03:24.965813+0000","last_unstale":"2024-04-16T17:05:22.510920+0000","last_undegraded":"2024-04-16T17:05:22.510920+0000","last_fullsized":"2024-04-16T17:05:22.510920+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:43:43.224130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"92'9","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.283010+0000","last_change":"2024-04-16T17:03:24.970941+0000","last_active":"2024-04-16T17:05:22.283010+0000","last_peered":"2024-04-16T17:05:22.283010+0000","last_clean":"2024-04-16T17:05:22.283010+0000","last_became_active":"2024-04-16T17:03:24.970587+0000","last_became_peered":"2024-04-16T17:03:24.970587+0000","last_unstale":"2024-04-16T17:05:22.283010+0000","last_undegraded":"2024-04-16T17:05:22.283010+0000","last_fullsized":"2024-04-16T17:05:22.283010+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:14:13.930011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"92'12","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.509789+0000","last_change":"2024-04-16T17:03:24.967167+0000","last_active":"2024-04-16T17:05:22.509789+0000","last_peered":"2024-04-16T17:05:22.509789+0000","last_clean":"2024-04-16T17:05:22.509789+0000","last_became_active":"2024-04-16T17:03:24.966787+0000","last_became_peered":"2024-04-16T17:03:24.966787+0000","last_unstale":"2024-04-16T17:05:22.509789+0000","last_undegraded":"2024-04-16T17:05:22.509789+0000","last_fullsized":"2024-04-16T17:05:22.509789+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:51:39.510220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"92'8","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.509416+0000","last_change":"2024-04-16T17:03:24.981115+0000","last_active":"2024-04-16T17:05:22.509416+0000","last_peered":"2024-04-16T17:05:22.509416+0000","last_clean":"2024-04-16T17:05:22.509416+0000","last_became_active":"2024-04-16T17:03:24.980685+0000","last_became_peered":"2024-04-16T17:03:24.980685+0000","last_unstale":"2024-04-16T17:05:22.509416+0000","last_undegraded":"2024-04-16T17:05:22.509416+0000","last_fullsized":"2024-04-16T17:05:22.509416+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:20:44.066932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"92'7","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.509572+0000","last_change":"2024-04-16T17:03:24.990421+0000","last_active":"2024-04-16T17:05:22.509572+0000","last_peered":"2024-04-16T17:05:22.509572+0000","last_clean":"2024-04-16T17:05:22.509572+0000","last_became_active":"2024-04-16T17:03:24.989940+0000","last_became_peered":"2024-04-16T17:03:24.989940+0000","last_unstale":"2024-04-16T17:05:22.509572+0000","last_undegraded":"2024-04-16T17:05:22.509572+0000","last_fullsized":"2024-04-16T17:05:22.509572+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:19:57.548067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"92'4","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.290057+0000","last_change":"2024-04-16T17:03:24.991854+0000","last_active":"2024-04-16T17:05:22.290057+0000","last_peered":"2024-04-16T17:05:22.290057+0000","last_clean":"2024-04-16T17:05:22.290057+0000","last_became_active":"2024-04-16T17:03:24.990964+0000","last_became_peered":"2024-04-16T17:03:24.990964+0000","last_unstale":"2024-04-16T17:05:22.290057+0000","last_undegraded":"2024-04-16T17:05:22.290057+0000","last_fullsized":"2024-04-16T17:05:22.290057+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:10:10.618708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"92'6","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.283375+0000","last_change":"2024-04-16T17:04:21.517466+0000","last_active":"2024-04-16T17:05:22.283375+0000","last_peered":"2024-04-16T17:05:22.283375+0000","last_clean":"2024-04-16T17:05:22.283375+0000","last_became_active":"2024-04-16T17:04:21.517086+0000","last_became_peered":"2024-04-16T17:04:21.517086+0000","last_unstale":"2024-04-16T17:05:22.283375+0000","last_undegraded":"2024-04-16T17:05:22.283375+0000","last_fullsized":"2024-04-16T17:05:22.283375+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:54:33.417452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,9],"acting":[11,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"92'15","reported_seq":45,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.509933+0000","last_change":"2024-04-16T17:03:24.970377+0000","last_active":"2024-04-16T17:05:22.509933+0000","last_peered":"2024-04-16T17:05:22.509933+0000","last_clean":"2024-04-16T17:05:22.509933+0000","last_became_active":"2024-04-16T17:03:24.968856+0000","last_became_peered":"2024-04-16T17:03:24.968856+0000","last_unstale":"2024-04-16T17:05:22.509933+0000","last_undegraded":"2024-04-16T17:05:22.509933+0000","last_fullsized":"2024-04-16T17:05:22.509933+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:47:28.543120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"92'2","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.519372+0000","last_change":"2024-04-16T17:03:24.992877+0000","last_active":"2024-04-16T17:05:22.519372+0000","last_peered":"2024-04-16T17:05:22.519372+0000","last_clean":"2024-04-16T17:05:22.519372+0000","last_became_active":"2024-04-16T17:03:24.992484+0000","last_became_peered":"2024-04-16T17:03:24.992484+0000","last_unstale":"2024-04-16T17:05:22.519372+0000","last_undegraded":"2024-04-16T17:05:22.519372+0000","last_fullsized":"2024-04-16T17:05:22.519372+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:00:45.204782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"92'6","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.512442+0000","last_change":"2024-04-16T17:03:24.977794+0000","last_active":"2024-04-16T17:05:22.512442+0000","last_peered":"2024-04-16T17:05:22.512442+0000","last_clean":"2024-04-16T17:05:22.512442+0000","last_became_active":"2024-04-16T17:03:24.977434+0000","last_became_peered":"2024-04-16T17:03:24.977434+0000","last_unstale":"2024-04-16T17:05:22.512442+0000","last_undegraded":"2024-04-16T17:05:22.512442+0000","last_fullsized":"2024-04-16T17:05:22.512442+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:46:35.670357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"92'8","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.517929+0000","last_change":"2024-04-16T17:03:24.992225+0000","last_active":"2024-04-16T17:05:22.517929+0000","last_peered":"2024-04-16T17:05:22.517929+0000","last_clean":"2024-04-16T17:05:22.517929+0000","last_became_active":"2024-04-16T17:03:24.991904+0000","last_became_peered":"2024-04-16T17:03:24.991904+0000","last_unstale":"2024-04-16T17:05:22.517929+0000","last_undegraded":"2024-04-16T17:05:22.517929+0000","last_fullsized":"2024-04-16T17:05:22.517929+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:01:09.710698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"92'10","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.510379+0000","last_change":"2024-04-16T17:03:24.981322+0000","last_active":"2024-04-16T17:05:22.510379+0000","last_peered":"2024-04-16T17:05:22.510379+0000","last_clean":"2024-04-16T17:05:22.510379+0000","last_became_active":"2024-04-16T17:03:24.980920+0000","last_became_peered":"2024-04-16T17:03:24.980920+0000","last_unstale":"2024-04-16T17:05:22.510379+0000","last_undegraded":"2024-04-16T17:05:22.510379+0000","last_fullsized":"2024-04-16T17:05:22.510379+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:07:59.378512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"92'11","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.289306+0000","last_change":"2024-04-16T17:03:24.990117+0000","last_active":"2024-04-16T17:05:22.289306+0000","last_peered":"2024-04-16T17:05:22.289306+0000","last_clean":"2024-04-16T17:05:22.289306+0000","last_became_active":"2024-04-16T17:03:24.989734+0000","last_became_peered":"2024-04-16T17:03:24.989734+0000","last_unstale":"2024-04-16T17:05:22.289306+0000","last_undegraded":"2024-04-16T17:05:22.289306+0000","last_fullsized":"2024-04-16T17:05:22.289306+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:36:55.000702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"92'8","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-16T17:05:22.283254+0000","last_change":"2024-04-16T17:03:24.970032+0000","last_active":"2024-04-16T17:05:22.283254+0000","last_peered":"2024-04-16T17:05:22.283254+0000","last_clean":"2024-04-16T17:05:22.283254+0000","last_became_active":"2024-04-16T17:03:24.968608+0000","last_became_peered":"2024-04-16T17:03:24.968608+0000","last_unstale":"2024-04-16T17:05:22.283254+0000","last_undegraded":"2024-04-16T17:05:22.283254+0000","last_fullsized":"2024-04-16T17:05:22.283254+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:24:03.918418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":283858,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":68279,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":673,"num_read_kb":410,"num_write":175,"num_write_kb":125,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":616,"num_keys_recovered":20,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":204837,"data_compressed":4887,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":151,"ondisk_log_size":151,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351017,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1032192,"data_stored":522666,"data_compressed":26684,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612656,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32332,"kb_used_data":1148,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712820,"statfs":{"total":95995035648,"available":95961927680,"internally_reserved":0,"allocated":1175552,"data_stored":1178133,"data_compressed":64411,"data_compressed_allocated":86016,"data_compressed_original":671744,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776181,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32204,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712948,"statfs":{"total":95995035648,"available":95962058752,"internally_reserved":0,"allocated":1044480,"data_stored":522868,"data_compressed":26684,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037820,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32200,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712952,"statfs":{"total":95995035648,"available":95962062848,"internally_reserved":0,"allocated":1040384,"data_stored":522206,"data_compressed":26684,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299458,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36412,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708740,"statfs":{"total":95995035648,"available":95957749760,"internally_reserved":0,"allocated":1093632,"data_stored":1119370,"data_compressed":62782,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462983,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32184,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712968,"statfs":{"total":95995035648,"available":95962079232,"internally_reserved":0,"allocated":1024000,"data_stored":518044,"data_compressed":26684,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574948,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32248,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712904,"statfs":{"total":95995035648,"available":95962013696,"internally_reserved":0,"allocated":1093632,"data_stored":582141,"data_compressed":28313,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738472,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32208,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712944,"statfs":{"total":95995035648,"available":95962054656,"internally_reserved":0,"allocated":1048576,"data_stored":519432,"data_compressed":26684,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411421,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28104,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717048,"statfs":{"total":95995035648,"available":95966257152,"internally_reserved":0,"allocated":1110016,"data_stored":583407,"data_compressed":28313,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149783,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32192,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712960,"statfs":{"total":95995035648,"available":95962071040,"internally_reserved":0,"allocated":1036288,"data_stored":518238,"data_compressed":26684,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888146,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1036288,"data_stored":523226,"data_compressed":26684,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724621,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28096,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717056,"statfs":{"total":95995035648,"available":95966265344,"internally_reserved":0,"allocated":1101824,"data_stored":1114426,"data_compressed":62782,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34456,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34456,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34456,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36098,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36098,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36098,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36098,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36098,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":64187,"data_compressed":1629,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":65453,"data_compressed":1629,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":156,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":65715,"data_compressed":1629,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-16T17:05:45.886 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-04-16T17:05:45.886 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-04-16T17:05:45.887 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-04-16T17:05:45.887 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45 shell --fsid b1fecd38-fc11-11ee-bc90-c7b262605968 -- ceph health --format=json 2024-04-16T17:05:46.426 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/mon.a/config 2024-04-16T17:05:46.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:05:46 smithi104 ceph-mon[36457]: from='client.25076 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-16T17:05:46.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:05:46 smithi045 ceph-mon[31676]: from='client.25076 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-16T17:05:46.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:05:46 smithi117 ceph-mon[36730]: from='client.25076 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-16T17:05:47.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:05:47 smithi104 ceph-mon[36457]: pgmap v397: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-16T17:05:47.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:05:47 smithi045 ceph-mon[31676]: pgmap v397: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-16T17:05:47.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:05:47 smithi117 ceph-mon[36730]: pgmap v397: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-16T17:05:48.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:48.869+0000 7fd517400700 1 -- 172.21.15.45:0/1089052736 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd510108b70 msgr2=0x7fd510108f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:48.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:48.869+0000 7fd517400700 1 --2- 172.21.15.45:0/1089052736 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd510108b70 0x7fd510108f50 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fd504009990 tx=0x7fd504009ca0 comp rx=0 tx=0).stop 2024-04-16T17:05:48.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:48.869+0000 7fd517400700 1 -- 172.21.15.45:0/1089052736 shutdown_connections 2024-04-16T17:05:48.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:48.869+0000 7fd517400700 1 --2- 172.21.15.45:0/1089052736 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd51006b1a0 0x7fd51010f9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:48.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:48.869+0000 7fd517400700 1 --2- 172.21.15.45:0/1089052736 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd510108b70 0x7fd510108f50 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:48.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:48.869+0000 7fd517400700 1 --2- 172.21.15.45:0/1089052736 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd51006a800 0x7fd51006ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:48.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:48.869+0000 7fd517400700 1 -- 172.21.15.45:0/1089052736 >> 172.21.15.45:0/1089052736 conn(0x7fd5100789a0 msgr2=0x7fd510078db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:05:48.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:48.869+0000 7fd517400700 1 -- 172.21.15.45:0/1089052736 shutdown_connections 2024-04-16T17:05:48.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:48.869+0000 7fd517400700 1 -- 172.21.15.45:0/1089052736 wait complete. 2024-04-16T17:05:48.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:48.870+0000 7fd517400700 1 Processor -- start 2024-04-16T17:05:48.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:48.871+0000 7fd517400700 1 -- start start 2024-04-16T17:05:48.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:48.871+0000 7fd517400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd51006a800 0x7fd51019b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:48.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:48.871+0000 7fd517400700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd51006b1a0 0x7fd51019ba10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:48.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:48.871+0000 7fd517400700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd510108b70 0x7fd51019fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:48.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:48.871+0000 7fd517400700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fd510102bb0 con 0x7fd510108b70 2024-04-16T17:05:48.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:48.871+0000 7fd517400700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fd510102a30 con 0x7fd51006b1a0 2024-04-16T17:05:48.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:48.871+0000 7fd517400700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fd510102d30 con 0x7fd51006a800 2024-04-16T17:05:48.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:48.871+0000 7fd514a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd51006a800 0x7fd51019b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:48.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:48.871+0000 7fd514a00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd51006a800 0x7fd51019b4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:49652/0 (socket says 172.21.15.45:49652) 2024-04-16T17:05:48.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:48.871+0000 7fd514a00700 1 -- 172.21.15.45:0/4164036367 learned_addr learned my addr 172.21.15.45:0/4164036367 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:05:48.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:48.872+0000 7fd50fe00700 1 --2- 172.21.15.45:0/4164036367 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd51006b1a0 0x7fd51019ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:48.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:48.872+0000 7fd515400700 1 --2- 172.21.15.45:0/4164036367 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd510108b70 0x7fd51019fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:48.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:48.872+0000 7fd514a00700 1 -- 172.21.15.45:0/4164036367 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd51006b1a0 msgr2=0x7fd51019ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:48.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:48.872+0000 7fd514a00700 1 --2- 172.21.15.45:0/4164036367 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd51006b1a0 0x7fd51019ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:48.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:48.872+0000 7fd514a00700 1 -- 172.21.15.45:0/4164036367 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd510108b70 msgr2=0x7fd51019fd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:48.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:48.872+0000 7fd514a00700 1 --2- 172.21.15.45:0/4164036367 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd510108b70 0x7fd51019fd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:48.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:48.872+0000 7fd514a00700 1 -- 172.21.15.45:0/4164036367 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5101a0490 con 0x7fd51006a800 2024-04-16T17:05:48.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:48.872+0000 7fd515400700 1 --2- 172.21.15.45:0/4164036367 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd510108b70 0x7fd51019fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:05:48.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:48.872+0000 7fd50fe00700 1 --2- 172.21.15.45:0/4164036367 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd51006b1a0 0x7fd51019ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:05:48.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:48.872+0000 7fd514a00700 1 --2- 172.21.15.45:0/4164036367 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd51006a800 0x7fd51019b4d0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fd504000c00 tx=0x7fd50400ba90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:05:48.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:48.873+0000 7fd50d600700 1 -- 172.21.15.45:0/4164036367 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd504027070 con 0x7fd51006a800 2024-04-16T17:05:48.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:48.873+0000 7fd517400700 1 -- 172.21.15.45:0/4164036367 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5101a0720 con 0x7fd51006a800 2024-04-16T17:05:48.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:48.873+0000 7fd50d600700 1 -- 172.21.15.45:0/4164036367 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd504003d70 con 0x7fd51006a800 2024-04-16T17:05:48.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:48.873+0000 7fd50d600700 1 -- 172.21.15.45:0/4164036367 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd50400f460 con 0x7fd51006a800 2024-04-16T17:05:48.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:48.873+0000 7fd517400700 1 -- 172.21.15.45:0/4164036367 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5101a8000 con 0x7fd51006a800 2024-04-16T17:05:48.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:48.874+0000 7fd517400700 1 -- 172.21.15.45:0/4164036367 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5101041e0 con 0x7fd51006a800 2024-04-16T17:05:48.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:48.878+0000 7fd50d600700 1 -- 172.21.15.45:0/4164036367 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fd50402a070 con 0x7fd51006a800 2024-04-16T17:05:48.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:48.879+0000 7fd50d600700 1 --2- 172.21.15.45:0/4164036367 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fd4f80a3c40 0x7fd4f80a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:48.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:48.880+0000 7fd50d600700 1 -- 172.21.15.45:0/4164036367 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fd5040d19d0 con 0x7fd51006a800 2024-04-16T17:05:48.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:48.880+0000 7fd50fe00700 1 --2- 172.21.15.45:0/4164036367 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fd4f80a3c40 0x7fd4f80a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:48.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:48.881+0000 7fd50d600700 1 -- 172.21.15.45:0/4164036367 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd504013070 con 0x7fd51006a800 2024-04-16T17:05:48.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:48.881+0000 7fd50fe00700 1 --2- 172.21.15.45:0/4164036367 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fd4f80a3c40 0x7fd4f80a6100 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fd510078f30 tx=0x7fd4fc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:05:49.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:49.172+0000 7fd517400700 1 -- 172.21.15.45:0/4164036367 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fd5100697e0 con 0x7fd51006a800 2024-04-16T17:05:49.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:49.173+0000 7fd50d600700 1 -- 172.21.15.45:0/4164036367 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7fd50409e050 con 0x7fd51006a800 2024-04-16T17:05:49.174 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-16T17:05:49.174 INFO:teuthology.orchestra.run.smithi045.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-16T17:05:49.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:49.176+0000 7fd517400700 1 -- 172.21.15.45:0/4164036367 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fd4f80a3c40 msgr2=0x7fd4f80a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:49.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:49.176+0000 7fd517400700 1 --2- 172.21.15.45:0/4164036367 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fd4f80a3c40 0x7fd4f80a6100 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fd510078f30 tx=0x7fd4fc009260 comp rx=0 tx=0).stop 2024-04-16T17:05:49.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:49.176+0000 7fd517400700 1 -- 172.21.15.45:0/4164036367 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd51006a800 msgr2=0x7fd51019b4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:49.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:49.176+0000 7fd517400700 1 --2- 172.21.15.45:0/4164036367 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd51006a800 0x7fd51019b4d0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fd504000c00 tx=0x7fd50400ba90 comp rx=0 tx=0).stop 2024-04-16T17:05:49.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:49.176+0000 7fd517400700 1 -- 172.21.15.45:0/4164036367 shutdown_connections 2024-04-16T17:05:49.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:49.176+0000 7fd517400700 1 --2- 172.21.15.45:0/4164036367 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fd4f80a3c40 0x7fd4f80a6100 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:49.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:49.176+0000 7fd517400700 1 --2- 172.21.15.45:0/4164036367 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd51006a800 0x7fd51019b4d0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:49.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:49.176+0000 7fd517400700 1 --2- 172.21.15.45:0/4164036367 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd510108b70 0x7fd51019fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:49.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:49.176+0000 7fd517400700 1 --2- 172.21.15.45:0/4164036367 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd51006b1a0 0x7fd51019ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:49.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:49.176+0000 7fd517400700 1 -- 172.21.15.45:0/4164036367 >> 172.21.15.45:0/4164036367 conn(0x7fd5100789a0 msgr2=0x7fd510106680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:05:49.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:49.176+0000 7fd517400700 1 -- 172.21.15.45:0/4164036367 shutdown_connections 2024-04-16T17:05:49.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:49.176+0000 7fd517400700 1 -- 172.21.15.45:0/4164036367 wait complete. 2024-04-16T17:05:49.477 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:05:49 smithi045 ceph-mon[31676]: pgmap v398: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-16T17:05:49.478 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:05:49 smithi045 ceph-mon[31676]: from='client.? 172.21.15.45:0/4164036367' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-16T17:05:49.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:05:49 smithi104 ceph-mon[36457]: pgmap v398: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-16T17:05:49.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:05:49 smithi104 ceph-mon[36457]: from='client.? 172.21.15.45:0/4164036367' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-16T17:05:49.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:05:49 smithi117 ceph-mon[36730]: pgmap v398: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-16T17:05:49.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:05:49 smithi117 ceph-mon[36730]: from='client.? 172.21.15.45:0/4164036367' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-16T17:05:50.104 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-04-16T17:05:50.105 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-16T17:05:50.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:50.346+0000 7f3eeb200640 1 -- 172.21.15.45:0/3895471820 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3eec105270 msgr2=0x7f3eec1056f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:50.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:50.346+0000 7f3eeb200640 1 --2- 172.21.15.45:0/3895471820 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3eec105270 0x7f3eec1056f0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f3ed8009ca0 tx=0x7f3ed802c0f0 comp rx=0 tx=0).stop 2024-04-16T17:05:50.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:50.346+0000 7f3eeb200640 1 -- 172.21.15.45:0/3895471820 shutdown_connections 2024-04-16T17:05:50.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:50.346+0000 7f3eeb200640 1 --2- 172.21.15.45:0/3895471820 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3eec105c30 0x7f3eec10c4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:50.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:50.346+0000 7f3eeb200640 1 --2- 172.21.15.45:0/3895471820 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3eec105270 0x7f3eec1056f0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:50.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:50.346+0000 7f3eeb200640 1 --2- 172.21.15.45:0/3895471820 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3eec104080 0x7f3eec1044a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:50.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:50.346+0000 7f3eeb200640 1 -- 172.21.15.45:0/3895471820 >> 172.21.15.45:0/3895471820 conn(0x7f3eec0ff610 msgr2=0x7f3eec101a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:05:50.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:50.346+0000 7f3eeb200640 1 -- 172.21.15.45:0/3895471820 shutdown_connections 2024-04-16T17:05:50.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:50.346+0000 7f3eeb200640 1 -- 172.21.15.45:0/3895471820 wait complete. 2024-04-16T17:05:50.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:50.347+0000 7f3eeb200640 1 Processor -- start 2024-04-16T17:05:50.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:50.347+0000 7f3eeb200640 1 -- start start 2024-04-16T17:05:50.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:50.348+0000 7f3eeb200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3eec104080 0x7f3eec19a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:50.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:50.348+0000 7f3eeb200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3eec105270 0x7f3eec19a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:50.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:50.348+0000 7f3eeb200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3eec105c30 0x7f3eec1a1900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:50.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:50.348+0000 7f3eeb200640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f3eec10fd10 con 0x7f3eec105c30 2024-04-16T17:05:50.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:50.348+0000 7f3eeb200640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f3eec10fb90 con 0x7f3eec104080 2024-04-16T17:05:50.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:50.348+0000 7f3eeb200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f3eec10fe90 con 0x7f3eec105270 2024-04-16T17:05:50.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:50.348+0000 7f3eea800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3eec105c30 0x7f3eec1a1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:50.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:50.348+0000 7f3eea800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3eec105c30 0x7f3eec1a1900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39252/0 (socket says 172.21.15.45:39252) 2024-04-16T17:05:50.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:50.348+0000 7f3eea800640 1 -- 172.21.15.45:0/255852379 learned_addr learned my addr 172.21.15.45:0/255852379 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:05:50.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:50.348+0000 7f3ee9e00640 1 --2- 172.21.15.45:0/255852379 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3eec104080 0x7f3eec19a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:50.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:50.348+0000 7f3ee9400640 1 --2- 172.21.15.45:0/255852379 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3eec105270 0x7f3eec19a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:50.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:50.349+0000 7f3ee9e00640 1 -- 172.21.15.45:0/255852379 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3eec105270 msgr2=0x7f3eec19a8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:50.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:50.349+0000 7f3ee9e00640 1 --2- 172.21.15.45:0/255852379 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3eec105270 0x7f3eec19a8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:50.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:50.349+0000 7f3ee9e00640 1 -- 172.21.15.45:0/255852379 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3eec105c30 msgr2=0x7f3eec1a1900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:50.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:50.349+0000 7f3ee9e00640 1 --2- 172.21.15.45:0/255852379 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3eec105c30 0x7f3eec1a1900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:50.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:50.349+0000 7f3ee9e00640 1 -- 172.21.15.45:0/255852379 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3eec1a1f70 con 0x7f3eec104080 2024-04-16T17:05:50.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:50.349+0000 7f3eea800640 1 --2- 172.21.15.45:0/255852379 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3eec105c30 0x7f3eec1a1900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:05:50.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:50.349+0000 7f3ee9400640 1 --2- 172.21.15.45:0/255852379 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3eec105270 0x7f3eec19a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:05:50.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:50.349+0000 7f3ee9e00640 1 --2- 172.21.15.45:0/255852379 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3eec104080 0x7f3eec19a3a0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f3edc00e9c0 tx=0x7f3edc00ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:05:50.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:50.349+0000 7f3ee2a00640 1 -- 172.21.15.45:0/255852379 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3edc00cd80 con 0x7f3eec104080 2024-04-16T17:05:50.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:50.349+0000 7f3eeb200640 1 -- 172.21.15.45:0/255852379 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3eec1a2260 con 0x7f3eec104080 2024-04-16T17:05:50.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:50.349+0000 7f3ee2a00640 1 -- 172.21.15.45:0/255852379 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3edc010910 con 0x7f3eec104080 2024-04-16T17:05:50.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:50.349+0000 7f3ee2a00640 1 -- 172.21.15.45:0/255852379 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3edc018980 con 0x7f3eec104080 2024-04-16T17:05:50.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:50.349+0000 7f3eeb200640 1 -- 172.21.15.45:0/255852379 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3eec1a27d0 con 0x7f3eec104080 2024-04-16T17:05:50.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:50.350+0000 7f3eeb200640 1 -- 172.21.15.45:0/255852379 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3eec10b3a0 con 0x7f3eec104080 2024-04-16T17:05:50.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:50.353+0000 7f3ee2a00640 1 -- 172.21.15.45:0/255852379 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f3edc01f020 con 0x7f3eec104080 2024-04-16T17:05:50.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:50.354+0000 7f3ee2a00640 1 --2- 172.21.15.45:0/255852379 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f3ed00a3c30 0x7f3ed00a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:50.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:50.354+0000 7f3ee2a00640 1 -- 172.21.15.45:0/255852379 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f3edc0d0700 con 0x7f3eec104080 2024-04-16T17:05:50.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:50.354+0000 7f3ee9400640 1 --2- 172.21.15.45:0/255852379 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f3ed00a3c30 0x7f3ed00a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:50.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:50.355+0000 7f3ee9400640 1 --2- 172.21.15.45:0/255852379 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f3ed00a3c30 0x7f3ed00a60f0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f3ed802c700 tx=0x7f3ed8005ef0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:05:50.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:50.359+0000 7f3ee2a00640 1 -- 172.21.15.45:0/255852379 <== mon.2 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3edc09ccf0 con 0x7f3eec104080 2024-04-16T17:05:50.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:50.564+0000 7f3eeb200640 1 -- 172.21.15.45:0/255852379 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3eec062c00 con 0x7f3eec104080 2024-04-16T17:05:50.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:50.566+0000 7f3ee2a00640 1 -- 172.21.15.45:0/255852379 <== mon.2 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) ==== 74+0+24726 (secure 0 0 0) 0x7f3edc09c5f0 con 0x7f3eec104080 2024-04-16T17:05:50.567 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-16T17:05:50.568 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":94,"fsid":"b1fecd38-fc11-11ee-bc90-c7b262605968","created":"2024-04-16T16:54:02.385906+0000","modified":"2024-04-16T17:05:22.272335+0000","last_up_change":"2024-04-16T17:02:25.162624+0000","last_in_change":"2024-04-16T17:02:04.903274+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-16T16:58:06.166659+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-16T17:03:22.304198+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-16T17:03:23.616450+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-16T17:03:27.226289+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b655de2c-59f8-4fd4-ac98-0cc222229f56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":2290170881},{"type":"v1","addr":"172.21.15.45:6803","nonce":2290170881}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":2290170881},{"type":"v1","addr":"172.21.15.45:6805","nonce":2290170881}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":2290170881},{"type":"v1","addr":"172.21.15.45:6809","nonce":2290170881}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":2290170881},{"type":"v1","addr":"172.21.15.45:6807","nonce":2290170881}]},"public_addr":"172.21.15.45:6803/2290170881","cluster_addr":"172.21.15.45:6805/2290170881","heartbeat_back_addr":"172.21.15.45:6809/2290170881","heartbeat_front_addr":"172.21.15.45:6807/2290170881","state":["exists","up"]},{"osd":1,"uuid":"ebd450ec-08fd-4c3d-8b24-4e10634ddb7e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6800","nonce":1462942067},{"type":"v1","addr":"172.21.15.104:6801","nonce":1462942067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":1462942067},{"type":"v1","addr":"172.21.15.104:6803","nonce":1462942067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":1462942067},{"type":"v1","addr":"172.21.15.104:6807","nonce":1462942067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":1462942067},{"type":"v1","addr":"172.21.15.104:6805","nonce":1462942067}]},"public_addr":"172.21.15.104:6801/1462942067","cluster_addr":"172.21.15.104:6803/1462942067","heartbeat_back_addr":"172.21.15.104:6807/1462942067","heartbeat_front_addr":"172.21.15.104:6805/1462942067","state":["exists","up"]},{"osd":2,"uuid":"b318a512-4190-42a2-8104-dcc4026ad8ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":2829260735},{"type":"v1","addr":"172.21.15.117:6801","nonce":2829260735}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":2829260735},{"type":"v1","addr":"172.21.15.117:6803","nonce":2829260735}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":2829260735},{"type":"v1","addr":"172.21.15.117:6807","nonce":2829260735}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":2829260735},{"type":"v1","addr":"172.21.15.117:6805","nonce":2829260735}]},"public_addr":"172.21.15.117:6801/2829260735","cluster_addr":"172.21.15.117:6803/2829260735","heartbeat_back_addr":"172.21.15.117:6807/2829260735","heartbeat_front_addr":"172.21.15.117:6805/2829260735","state":["exists","up"]},{"osd":3,"uuid":"ec6490cb-a715-4c12-a54a-225aae26fcb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":2823712137},{"type":"v1","addr":"172.21.15.45:6811","nonce":2823712137}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":2823712137},{"type":"v1","addr":"172.21.15.45:6813","nonce":2823712137}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":2823712137},{"type":"v1","addr":"172.21.15.45:6817","nonce":2823712137}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":2823712137},{"type":"v1","addr":"172.21.15.45:6815","nonce":2823712137}]},"public_addr":"172.21.15.45:6811/2823712137","cluster_addr":"172.21.15.45:6813/2823712137","heartbeat_back_addr":"172.21.15.45:6817/2823712137","heartbeat_front_addr":"172.21.15.45:6815/2823712137","state":["exists","up"]},{"osd":4,"uuid":"5d91d568-0250-4aae-8dc2-a0e079f92062","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":1396278127},{"type":"v1","addr":"172.21.15.104:6809","nonce":1396278127}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":1396278127},{"type":"v1","addr":"172.21.15.104:6811","nonce":1396278127}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":1396278127},{"type":"v1","addr":"172.21.15.104:6815","nonce":1396278127}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":1396278127},{"type":"v1","addr":"172.21.15.104:6813","nonce":1396278127}]},"public_addr":"172.21.15.104:6809/1396278127","cluster_addr":"172.21.15.104:6811/1396278127","heartbeat_back_addr":"172.21.15.104:6815/1396278127","heartbeat_front_addr":"172.21.15.104:6813/1396278127","state":["exists","up"]},{"osd":5,"uuid":"ac510b43-6fb2-4d0f-bc5f-16ac080d9e9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":4145720047},{"type":"v1","addr":"172.21.15.117:6809","nonce":4145720047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":4145720047},{"type":"v1","addr":"172.21.15.117:6811","nonce":4145720047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":4145720047},{"type":"v1","addr":"172.21.15.117:6815","nonce":4145720047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":4145720047},{"type":"v1","addr":"172.21.15.117:6813","nonce":4145720047}]},"public_addr":"172.21.15.117:6809/4145720047","cluster_addr":"172.21.15.117:6811/4145720047","heartbeat_back_addr":"172.21.15.117:6815/4145720047","heartbeat_front_addr":"172.21.15.117:6813/4145720047","state":["exists","up"]},{"osd":6,"uuid":"a41adc18-d88a-419e-b2c2-b089c1f32472","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":3254229176},{"type":"v1","addr":"172.21.15.45:6819","nonce":3254229176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":3254229176},{"type":"v1","addr":"172.21.15.45:6821","nonce":3254229176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":3254229176},{"type":"v1","addr":"172.21.15.45:6825","nonce":3254229176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":3254229176},{"type":"v1","addr":"172.21.15.45:6823","nonce":3254229176}]},"public_addr":"172.21.15.45:6819/3254229176","cluster_addr":"172.21.15.45:6821/3254229176","heartbeat_back_addr":"172.21.15.45:6825/3254229176","heartbeat_front_addr":"172.21.15.45:6823/3254229176","state":["exists","up"]},{"osd":7,"uuid":"b71712a4-7cf6-454d-9fd1-565e3df21c1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":4151077186},{"type":"v1","addr":"172.21.15.104:6817","nonce":4151077186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":4151077186},{"type":"v1","addr":"172.21.15.104:6819","nonce":4151077186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":4151077186},{"type":"v1","addr":"172.21.15.104:6823","nonce":4151077186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":4151077186},{"type":"v1","addr":"172.21.15.104:6821","nonce":4151077186}]},"public_addr":"172.21.15.104:6817/4151077186","cluster_addr":"172.21.15.104:6819/4151077186","heartbeat_back_addr":"172.21.15.104:6823/4151077186","heartbeat_front_addr":"172.21.15.104:6821/4151077186","state":["exists","up"]},{"osd":8,"uuid":"9e572aef-59a8-450e-a92e-df9dff4d57a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":1500286830},{"type":"v1","addr":"172.21.15.117:6817","nonce":1500286830}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":1500286830},{"type":"v1","addr":"172.21.15.117:6819","nonce":1500286830}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":1500286830},{"type":"v1","addr":"172.21.15.117:6823","nonce":1500286830}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":1500286830},{"type":"v1","addr":"172.21.15.117:6821","nonce":1500286830}]},"public_addr":"172.21.15.117:6817/1500286830","cluster_addr":"172.21.15.117:6819/1500286830","heartbeat_back_addr":"172.21.15.117:6823/1500286830","heartbeat_front_addr":"172.21.15.117:6821/1500286830","state":["exists","up"]},{"osd":9,"uuid":"014e9598-c413-4cc4-9bb8-61c6e8f59d30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":3106734074},{"type":"v1","addr":"172.21.15.45:6827","nonce":3106734074}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":3106734074},{"type":"v1","addr":"172.21.15.45:6829","nonce":3106734074}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":3106734074},{"type":"v1","addr":"172.21.15.45:6833","nonce":3106734074}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":3106734074},{"type":"v1","addr":"172.21.15.45:6831","nonce":3106734074}]},"public_addr":"172.21.15.45:6827/3106734074","cluster_addr":"172.21.15.45:6829/3106734074","heartbeat_back_addr":"172.21.15.45:6833/3106734074","heartbeat_front_addr":"172.21.15.45:6831/3106734074","state":["exists","up"]},{"osd":10,"uuid":"2deba364-3e39-47be-8a1f-8256df5b82d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":1404794652},{"type":"v1","addr":"172.21.15.104:6825","nonce":1404794652}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":1404794652},{"type":"v1","addr":"172.21.15.104:6827","nonce":1404794652}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":1404794652},{"type":"v1","addr":"172.21.15.104:6831","nonce":1404794652}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":1404794652},{"type":"v1","addr":"172.21.15.104:6829","nonce":1404794652}]},"public_addr":"172.21.15.104:6825/1404794652","cluster_addr":"172.21.15.104:6827/1404794652","heartbeat_back_addr":"172.21.15.104:6831/1404794652","heartbeat_front_addr":"172.21.15.104:6829/1404794652","state":["exists","up"]},{"osd":11,"uuid":"3fda4fea-59ad-4d1a-9fe6-fcb30e31fd7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":3384003243},{"type":"v1","addr":"172.21.15.117:6825","nonce":3384003243}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":3384003243},{"type":"v1","addr":"172.21.15.117:6827","nonce":3384003243}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":3384003243},{"type":"v1","addr":"172.21.15.117:6831","nonce":3384003243}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":3384003243},{"type":"v1","addr":"172.21.15.117:6829","nonce":3384003243}]},"public_addr":"172.21.15.117:6825/3384003243","cluster_addr":"172.21.15.117:6827/3384003243","heartbeat_back_addr":"172.21.15.117:6831/3384003243","heartbeat_front_addr":"172.21.15.117:6829/3384003243","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:57:14.799608+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:57:39.884668+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:58:03.327057+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:58:37.314531+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:59:03.667650+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:59:28.462823+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T17:00:02.968574+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T17:00:29.559190+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T17:00:55.291007+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T17:01:29.815977+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T17:01:57.015580+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T17:02:23.233151+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.5","mappings":[{"from":9,"to":10}]},{"pgid":"4.7","mappings":[{"from":5,"to":8}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.45:0/4074982598":"2024-04-17T16:55:19.948001+0000","172.21.15.45:0/2443572576":"2024-04-17T16:55:19.948001+0000","172.21.15.45:6801/1983630136":"2024-04-17T16:55:19.948001+0000","172.21.15.45:6800/1983630136":"2024-04-17T16:55:19.948001+0000","172.21.15.45:0/3150465248":"2024-04-17T16:54:43.692466+0000","172.21.15.45:0/556738623":"2024-04-17T16:54:43.692466+0000","172.21.15.45:0/445676297":"2024-04-17T16:55:19.948001+0000","172.21.15.45:6800/2553659622":"2024-04-17T16:54:43.692466+0000","172.21.15.45:6801/2553659622":"2024-04-17T16:54:43.692466+0000","172.21.15.45:0/1184615408":"2024-04-17T16:54:43.692466+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-16T17:05:50.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:50.569+0000 7f3eeb200640 1 -- 172.21.15.45:0/255852379 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f3ed00a3c30 msgr2=0x7f3ed00a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:50.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:50.569+0000 7f3eeb200640 1 --2- 172.21.15.45:0/255852379 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f3ed00a3c30 0x7f3ed00a60f0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f3ed802c700 tx=0x7f3ed8005ef0 comp rx=0 tx=0).stop 2024-04-16T17:05:50.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:50.569+0000 7f3eeb200640 1 -- 172.21.15.45:0/255852379 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3eec104080 msgr2=0x7f3eec19a3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:50.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:50.569+0000 7f3eeb200640 1 --2- 172.21.15.45:0/255852379 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3eec104080 0x7f3eec19a3a0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f3edc00e9c0 tx=0x7f3edc00ee80 comp rx=0 tx=0).stop 2024-04-16T17:05:50.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:50.569+0000 7f3eeb200640 1 -- 172.21.15.45:0/255852379 shutdown_connections 2024-04-16T17:05:50.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:50.569+0000 7f3eeb200640 1 --2- 172.21.15.45:0/255852379 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f3ed00a3c30 0x7f3ed00a60f0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:50.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:50.569+0000 7f3eeb200640 1 --2- 172.21.15.45:0/255852379 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3eec105270 0x7f3eec19a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:50.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:50.569+0000 7f3eeb200640 1 --2- 172.21.15.45:0/255852379 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3eec105c30 0x7f3eec1a1900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:50.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:50.569+0000 7f3eeb200640 1 --2- 172.21.15.45:0/255852379 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3eec104080 0x7f3eec19a3a0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:50.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:50.569+0000 7f3eeb200640 1 -- 172.21.15.45:0/255852379 >> 172.21.15.45:0/255852379 conn(0x7f3eec0ff610 msgr2=0x7f3eec108570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:05:50.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:50.569+0000 7f3eeb200640 1 -- 172.21.15.45:0/255852379 shutdown_connections 2024-04-16T17:05:50.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:50.569+0000 7f3eeb200640 1 -- 172.21.15.45:0/255852379 wait complete. 2024-04-16T17:05:50.588 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-16T16:58:06.166659+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-16T17:03:22.304198+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-16T17:03:23.616450+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-16T17:03:27.226289+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-16T17:05:50.588 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-16T17:05:50.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:50.811+0000 7f9586000640 1 -- 172.21.15.45:0/2021029204 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9580032990 con 0x7f95880785a0 2024-04-16T17:05:50.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:50.812+0000 7f958ca00640 1 -- 172.21.15.45:0/2021029204 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f95880785a0 msgr2=0x7f9588078e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:50.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:50.812+0000 7f958ca00640 1 --2- 172.21.15.45:0/2021029204 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f95880785a0 0x7f9588078e40 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f9580009ca0 tx=0x7f958002c100 comp rx=0 tx=0).stop 2024-04-16T17:05:50.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:50.812+0000 7f958ca00640 1 -- 172.21.15.45:0/2021029204 shutdown_connections 2024-04-16T17:05:50.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:50.812+0000 7f958ca00640 1 --2- 172.21.15.45:0/2021029204 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9588077c00 0x7f9588078060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:50.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:50.812+0000 7f958ca00640 1 --2- 172.21.15.45:0/2021029204 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f95880785a0 0x7f9588078e40 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:50.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:50.812+0000 7f958ca00640 1 --2- 172.21.15.45:0/2021029204 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f95880769c0 0x7f9588076de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:50.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:50.812+0000 7f958ca00640 1 -- 172.21.15.45:0/2021029204 >> 172.21.15.45:0/2021029204 conn(0x7f95880ff5d0 msgr2=0x7f95881019f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:05:50.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:50.812+0000 7f958ca00640 1 -- 172.21.15.45:0/2021029204 shutdown_connections 2024-04-16T17:05:50.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:50.812+0000 7f958ca00640 1 -- 172.21.15.45:0/2021029204 wait complete. 2024-04-16T17:05:50.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:50.813+0000 7f958ca00640 1 Processor -- start 2024-04-16T17:05:50.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:50.813+0000 7f958ca00640 1 -- start start 2024-04-16T17:05:50.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:50.813+0000 7f958ca00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f95880769c0 0x7f958819e740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:50.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:50.813+0000 7f958ca00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9588077c00 0x7f958819ec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:50.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:50.813+0000 7f958ca00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f95880785a0 0x7f95881a5ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:50.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:50.813+0000 7f958ca00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f95881140c0 con 0x7f9588077c00 2024-04-16T17:05:50.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:50.813+0000 7f958ca00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f9588113f40 con 0x7f95880769c0 2024-04-16T17:05:50.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:50.813+0000 7f958ca00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f9588114240 con 0x7f95880785a0 2024-04-16T17:05:50.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:50.814+0000 7f9586a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9588077c00 0x7f958819ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:50.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:50.814+0000 7f9586a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9588077c00 0x7f958819ec80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39266/0 (socket says 172.21.15.45:39266) 2024-04-16T17:05:50.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:50.814+0000 7f9586a00640 1 -- 172.21.15.45:0/1164651242 learned_addr learned my addr 172.21.15.45:0/1164651242 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:05:50.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:50.814+0000 7f9587e00640 1 --2- 172.21.15.45:0/1164651242 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f95880785a0 0x7f95881a5ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:50.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:50.814+0000 7f9587400640 1 --2- 172.21.15.45:0/1164651242 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f95880769c0 0x7f958819e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:50.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:50.814+0000 7f9586a00640 1 -- 172.21.15.45:0/1164651242 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f95880785a0 msgr2=0x7f95881a5ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:50.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:50.814+0000 7f9586a00640 1 --2- 172.21.15.45:0/1164651242 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f95880785a0 0x7f95881a5ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:50.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:50.814+0000 7f9586a00640 1 -- 172.21.15.45:0/1164651242 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f95880769c0 msgr2=0x7f958819e740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:50.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:50.814+0000 7f9586a00640 1 --2- 172.21.15.45:0/1164651242 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f95880769c0 0x7f958819e740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:50.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:50.814+0000 7f9586a00640 1 -- 172.21.15.45:0/1164651242 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95881a6310 con 0x7f9588077c00 2024-04-16T17:05:50.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:50.814+0000 7f9587400640 1 --2- 172.21.15.45:0/1164651242 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f95880769c0 0x7f958819e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:05:50.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:50.815+0000 7f9586a00640 1 --2- 172.21.15.45:0/1164651242 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9588077c00 0x7f958819ec80 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f957400e9f0 tx=0x7f957400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:05:50.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:50.815+0000 7f957fe00640 1 -- 172.21.15.45:0/1164651242 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f957400cd80 con 0x7f9588077c00 2024-04-16T17:05:50.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:50.815+0000 7f957fe00640 1 -- 172.21.15.45:0/1164651242 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9574005500 con 0x7f9588077c00 2024-04-16T17:05:50.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:50.815+0000 7f958ca00640 1 -- 172.21.15.45:0/1164651242 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95881a6600 con 0x7f9588077c00 2024-04-16T17:05:50.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:50.815+0000 7f957fe00640 1 -- 172.21.15.45:0/1164651242 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9574010690 con 0x7f9588077c00 2024-04-16T17:05:50.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:50.815+0000 7f958ca00640 1 -- 172.21.15.45:0/1164651242 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95881a6b50 con 0x7f9588077c00 2024-04-16T17:05:50.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:50.817+0000 7f958ca00640 1 -- 172.21.15.45:0/1164651242 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9588069190 con 0x7f9588077c00 2024-04-16T17:05:50.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:50.818+0000 7f957fe00640 1 -- 172.21.15.45:0/1164651242 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f95740046c0 con 0x7f9588077c00 2024-04-16T17:05:50.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:50.820+0000 7f957fe00640 1 --2- 172.21.15.45:0/1164651242 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f956c0a3c30 0x7f956c0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:50.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:50.820+0000 7f957fe00640 1 -- 172.21.15.45:0/1164651242 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f95740d0c20 con 0x7f9588077c00 2024-04-16T17:05:50.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:50.820+0000 7f9587400640 1 --2- 172.21.15.45:0/1164651242 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f956c0a3c30 0x7f956c0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:50.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:50.821+0000 7f9587400640 1 --2- 172.21.15.45:0/1164651242 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f956c0a3c30 0x7f956c0a60f0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f9578009c20 tx=0x7f9578005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:05:50.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:50.824+0000 7f957fe00640 1 -- 172.21.15.45:0/1164651242 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f957409d210 con 0x7f9588077c00 2024-04-16T17:05:51.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:51.026+0000 7f958ca00640 1 -- 172.21.15.45:0/1164651242 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f9588062c00 con 0x7f9588077c00 2024-04-16T17:05:51.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:51.027+0000 7f957fe00640 1 -- 172.21.15.45:0/1164651242 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v94) ==== 93+0+10 (secure 0 0 0) 0x7f957409cb10 con 0x7f9588077c00 2024-04-16T17:05:51.028 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 1 2024-04-16T17:05:51.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:51.030+0000 7f958ca00640 1 -- 172.21.15.45:0/1164651242 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f956c0a3c30 msgr2=0x7f956c0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:51.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:51.030+0000 7f958ca00640 1 --2- 172.21.15.45:0/1164651242 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f956c0a3c30 0x7f956c0a60f0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f9578009c20 tx=0x7f9578005f90 comp rx=0 tx=0).stop 2024-04-16T17:05:51.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:51.030+0000 7f958ca00640 1 -- 172.21.15.45:0/1164651242 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9588077c00 msgr2=0x7f958819ec80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:51.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:51.030+0000 7f958ca00640 1 --2- 172.21.15.45:0/1164651242 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9588077c00 0x7f958819ec80 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f957400e9f0 tx=0x7f957400edf0 comp rx=0 tx=0).stop 2024-04-16T17:05:51.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:51.030+0000 7f958ca00640 1 -- 172.21.15.45:0/1164651242 shutdown_connections 2024-04-16T17:05:51.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:51.030+0000 7f958ca00640 1 --2- 172.21.15.45:0/1164651242 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f956c0a3c30 0x7f956c0a60f0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:51.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:51.030+0000 7f958ca00640 1 --2- 172.21.15.45:0/1164651242 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f95880785a0 0x7f95881a5ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:51.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:51.030+0000 7f958ca00640 1 --2- 172.21.15.45:0/1164651242 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9588077c00 0x7f958819ec80 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:51.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:51.030+0000 7f958ca00640 1 --2- 172.21.15.45:0/1164651242 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f95880769c0 0x7f958819e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:51.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:51.030+0000 7f958ca00640 1 -- 172.21.15.45:0/1164651242 >> 172.21.15.45:0/1164651242 conn(0x7f95880ff5d0 msgr2=0x7f95881010d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:05:51.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:51.030+0000 7f958ca00640 1 -- 172.21.15.45:0/1164651242 shutdown_connections 2024-04-16T17:05:51.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:51.030+0000 7f958ca00640 1 -- 172.21.15.45:0/1164651242 wait complete. 2024-04-16T17:05:51.050 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-16T17:05:51.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:51.274+0000 7f91a3200640 1 -- 172.21.15.45:0/2836668865 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f91a4074a60 msgr2=0x7f91a4074ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:51.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:51.274+0000 7f91a3200640 1 --2- 172.21.15.45:0/2836668865 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f91a4074a60 0x7f91a4074ee0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f9194009900 tx=0x7f919402bc40 comp rx=0 tx=0).stop 2024-04-16T17:05:51.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:51.275+0000 7f91a3200640 1 -- 172.21.15.45:0/2836668865 shutdown_connections 2024-04-16T17:05:51.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:51.275+0000 7f91a3200640 1 --2- 172.21.15.45:0/2836668865 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f91a4109e50 0x7f91a410c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:51.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:51.275+0000 7f91a3200640 1 --2- 172.21.15.45:0/2836668865 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f91a4074a60 0x7f91a4074ee0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:51.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:51.275+0000 7f91a3200640 1 --2- 172.21.15.45:0/2836668865 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f91a4076040 0x7f91a4074490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:51.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:51.275+0000 7f91a3200640 1 -- 172.21.15.45:0/2836668865 >> 172.21.15.45:0/2836668865 conn(0x7f91a40fd500 msgr2=0x7f91a40ff940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:05:51.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:51.275+0000 7f91a3200640 1 -- 172.21.15.45:0/2836668865 shutdown_connections 2024-04-16T17:05:51.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:51.275+0000 7f91a3200640 1 -- 172.21.15.45:0/2836668865 wait complete. 2024-04-16T17:05:51.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:51.276+0000 7f91a3200640 1 Processor -- start 2024-04-16T17:05:51.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:51.276+0000 7f91a3200640 1 -- start start 2024-04-16T17:05:51.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:51.276+0000 7f91a3200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f91a4074a60 0x7f91a419a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:51.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:51.277+0000 7f91a3200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f91a4076040 0x7f91a419a960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:51.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:51.277+0000 7f91a3200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f91a4109e50 0x7f91a41a19d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:51.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:51.277+0000 7f91a3200640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f91a410fdf0 con 0x7f91a4076040 2024-04-16T17:05:51.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:51.277+0000 7f91a3200640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f91a410fc70 con 0x7f91a4074a60 2024-04-16T17:05:51.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:51.277+0000 7f91a3200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f91a410ff70 con 0x7f91a4109e50 2024-04-16T17:05:51.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:51.277+0000 7f91a1400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f91a4076040 0x7f91a419a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:51.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:51.277+0000 7f91a1e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f91a4074a60 0x7f91a419a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:51.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:51.277+0000 7f91a1e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f91a4074a60 0x7f91a419a420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:51672/0 (socket says 172.21.15.45:51672) 2024-04-16T17:05:51.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:51.277+0000 7f91a1e00640 1 -- 172.21.15.45:0/2931023939 learned_addr learned my addr 172.21.15.45:0/2931023939 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:05:51.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:51.277+0000 7f91a2800640 1 --2- 172.21.15.45:0/2931023939 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f91a4109e50 0x7f91a41a19d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:51.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:51.277+0000 7f91a1e00640 1 -- 172.21.15.45:0/2931023939 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f91a4109e50 msgr2=0x7f91a41a19d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:51.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:51.277+0000 7f91a1e00640 1 --2- 172.21.15.45:0/2931023939 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f91a4109e50 0x7f91a41a19d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:51.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:51.277+0000 7f91a1e00640 1 -- 172.21.15.45:0/2931023939 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f91a4076040 msgr2=0x7f91a419a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:51.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:51.277+0000 7f91a1e00640 1 --2- 172.21.15.45:0/2931023939 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f91a4076040 0x7f91a419a960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:51.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:51.277+0000 7f91a1e00640 1 -- 172.21.15.45:0/2931023939 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91a41a2040 con 0x7f91a4074a60 2024-04-16T17:05:51.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:51.278+0000 7f91a1e00640 1 --2- 172.21.15.45:0/2931023939 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f91a4074a60 0x7f91a419a420 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f918c00e9f0 tx=0x7f918c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:05:51.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:51.278+0000 7f919aa00640 1 -- 172.21.15.45:0/2931023939 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f918c00cd80 con 0x7f91a4074a60 2024-04-16T17:05:51.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:51.278+0000 7f919aa00640 1 -- 172.21.15.45:0/2931023939 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f918c010910 con 0x7f91a4074a60 2024-04-16T17:05:51.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:51.278+0000 7f91a3200640 1 -- 172.21.15.45:0/2931023939 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91a41a2330 con 0x7f91a4074a60 2024-04-16T17:05:51.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:51.278+0000 7f919aa00640 1 -- 172.21.15.45:0/2931023939 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f918c018990 con 0x7f91a4074a60 2024-04-16T17:05:51.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:51.278+0000 7f91a3200640 1 -- 172.21.15.45:0/2931023939 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91a41a2850 con 0x7f91a4074a60 2024-04-16T17:05:51.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:51.279+0000 7f91a3200640 1 -- 172.21.15.45:0/2931023939 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91a4108b80 con 0x7f91a4074a60 2024-04-16T17:05:51.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:51.281+0000 7f919aa00640 1 -- 172.21.15.45:0/2931023939 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f918c010ac0 con 0x7f91a4074a60 2024-04-16T17:05:51.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:51.282+0000 7f919aa00640 1 --2- 172.21.15.45:0/2931023939 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f91880a3bb0 0x7f91880a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:51.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:51.282+0000 7f919aa00640 1 -- 172.21.15.45:0/2931023939 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f918c0d0ab0 con 0x7f91a4074a60 2024-04-16T17:05:51.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:51.283+0000 7f91a1400640 1 --2- 172.21.15.45:0/2931023939 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f91880a3bb0 0x7f91880a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:51.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:51.283+0000 7f91a1400640 1 --2- 172.21.15.45:0/2931023939 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f91880a3bb0 0x7f91880a6070 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f919402c250 tx=0x7f919402cf90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:05:51.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:51.287+0000 7f919aa00640 1 -- 172.21.15.45:0/2931023939 <== mon.2 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f918c09cab0 con 0x7f91a4074a60 2024-04-16T17:05:51.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:51.491+0000 7f91a3200640 1 -- 172.21.15.45:0/2931023939 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f91a4062c00 con 0x7f91a4074a60 2024-04-16T17:05:51.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:51.492+0000 7f919aa00640 1 -- 172.21.15.45:0/2931023939 <== mon.2 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v94) ==== 104+0+11 (secure 0 0 0) 0x7f918c0a0db0 con 0x7f91a4074a60 2024-04-16T17:05:51.492 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 32 2024-04-16T17:05:51.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:51.494+0000 7f91a3200640 1 -- 172.21.15.45:0/2931023939 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f91880a3bb0 msgr2=0x7f91880a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:51.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:51.494+0000 7f91a3200640 1 --2- 172.21.15.45:0/2931023939 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f91880a3bb0 0x7f91880a6070 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f919402c250 tx=0x7f919402cf90 comp rx=0 tx=0).stop 2024-04-16T17:05:51.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:51.494+0000 7f91a3200640 1 -- 172.21.15.45:0/2931023939 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f91a4074a60 msgr2=0x7f91a419a420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:51.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:51.494+0000 7f91a3200640 1 --2- 172.21.15.45:0/2931023939 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f91a4074a60 0x7f91a419a420 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f918c00e9f0 tx=0x7f918c00eeb0 comp rx=0 tx=0).stop 2024-04-16T17:05:51.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:51.494+0000 7f91a3200640 1 -- 172.21.15.45:0/2931023939 shutdown_connections 2024-04-16T17:05:51.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:51.494+0000 7f91a3200640 1 --2- 172.21.15.45:0/2931023939 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f91880a3bb0 0x7f91880a6070 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:51.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:51.494+0000 7f91a3200640 1 --2- 172.21.15.45:0/2931023939 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f91a4109e50 0x7f91a41a19d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:51.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:51.494+0000 7f91a3200640 1 --2- 172.21.15.45:0/2931023939 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f91a4076040 0x7f91a419a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:51.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:51.494+0000 7f91a3200640 1 --2- 172.21.15.45:0/2931023939 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f91a4074a60 0x7f91a419a420 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:51.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:51.494+0000 7f91a3200640 1 -- 172.21.15.45:0/2931023939 >> 172.21.15.45:0/2931023939 conn(0x7f91a40fd500 msgr2=0x7f91a4105d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:05:51.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:51.494+0000 7f91a3200640 1 -- 172.21.15.45:0/2931023939 shutdown_connections 2024-04-16T17:05:51.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:51.494+0000 7f91a3200640 1 -- 172.21.15.45:0/2931023939 wait complete. 2024-04-16T17:05:51.512 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-16T17:05:51.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:05:51 smithi104 ceph-mon[36457]: pgmap v399: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-16T17:05:51.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:05:51 smithi104 ceph-mon[36457]: from='client.? 172.21.15.45:0/255852379' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T17:05:51.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:05:51 smithi104 ceph-mon[36457]: from='client.? 172.21.15.45:0/1164651242' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-16T17:05:51.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:05:51 smithi045 ceph-mon[31676]: pgmap v399: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-16T17:05:51.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:05:51 smithi045 ceph-mon[31676]: from='client.? 172.21.15.45:0/255852379' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T17:05:51.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:05:51 smithi045 ceph-mon[31676]: from='client.? 172.21.15.45:0/1164651242' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-16T17:05:51.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:51.741+0000 7f9ea4a00640 1 -- 172.21.15.45:0/4036031753 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9ea0104140 msgr2=0x7f9ea0104560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:51.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:51.741+0000 7f9ea4a00640 1 --2- 172.21.15.45:0/4036031753 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9ea0104140 0x7f9ea0104560 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f9e90009860 tx=0x7f9e9002bb70 comp rx=0 tx=0).stop 2024-04-16T17:05:51.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:51.741+0000 7f9ea4a00640 1 -- 172.21.15.45:0/4036031753 shutdown_connections 2024-04-16T17:05:51.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:51.741+0000 7f9ea4a00640 1 --2- 172.21.15.45:0/4036031753 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9ea0105330 0x7f9ea01057b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:51.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:51.741+0000 7f9ea4a00640 1 --2- 172.21.15.45:0/4036031753 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9ea0105cf0 0x7f9ea010c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:51.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:51.741+0000 7f9ea4a00640 1 --2- 172.21.15.45:0/4036031753 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9ea0104140 0x7f9ea0104560 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:51.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:51.741+0000 7f9ea4a00640 1 -- 172.21.15.45:0/4036031753 >> 172.21.15.45:0/4036031753 conn(0x7f9ea00ff6b0 msgr2=0x7f9ea0101af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:05:51.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:51.741+0000 7f9ea4a00640 1 -- 172.21.15.45:0/4036031753 shutdown_connections 2024-04-16T17:05:51.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:51.741+0000 7f9ea4a00640 1 -- 172.21.15.45:0/4036031753 wait complete. 2024-04-16T17:05:51.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:51.742+0000 7f9ea4a00640 1 Processor -- start 2024-04-16T17:05:51.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:51.742+0000 7f9ea4a00640 1 -- start start 2024-04-16T17:05:51.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:51.743+0000 7f9ea4a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9ea0104140 0x7f9ea019a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:51.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:51.743+0000 7f9ea4a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9ea0105330 0x7f9ea019a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:51.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:51.743+0000 7f9ea4a00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9ea0105cf0 0x7f9ea01a1a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:51.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:51.743+0000 7f9ea4a00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f9ea010fda0 con 0x7f9ea0104140 2024-04-16T17:05:51.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:51.743+0000 7f9ea4a00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f9ea010fc20 con 0x7f9ea0105cf0 2024-04-16T17:05:51.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:51.743+0000 7f9ea4a00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f9ea010ff20 con 0x7f9ea0105330 2024-04-16T17:05:51.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:51.743+0000 7f9e9e800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9ea0104140 0x7f9ea019a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:51.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:51.743+0000 7f9e9e800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9ea0104140 0x7f9ea019a450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39292/0 (socket says 172.21.15.45:39292) 2024-04-16T17:05:51.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:51.743+0000 7f9e9e800640 1 -- 172.21.15.45:0/2298037689 learned_addr learned my addr 172.21.15.45:0/2298037689 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:05:51.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:51.743+0000 7f9e9f200640 1 --2- 172.21.15.45:0/2298037689 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9ea0105cf0 0x7f9ea01a1a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:51.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:51.743+0000 7f9e9de00640 1 --2- 172.21.15.45:0/2298037689 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9ea0105330 0x7f9ea019a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:51.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:51.744+0000 7f9e9e800640 1 -- 172.21.15.45:0/2298037689 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9ea0105330 msgr2=0x7f9ea019a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:51.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:51.744+0000 7f9e9e800640 1 --2- 172.21.15.45:0/2298037689 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9ea0105330 0x7f9ea019a990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:51.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:51.744+0000 7f9e9e800640 1 -- 172.21.15.45:0/2298037689 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9ea0105cf0 msgr2=0x7f9ea01a1a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:51.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:51.744+0000 7f9e9e800640 1 --2- 172.21.15.45:0/2298037689 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9ea0105cf0 0x7f9ea01a1a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:51.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:51.744+0000 7f9e9e800640 1 -- 172.21.15.45:0/2298037689 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ea01a2070 con 0x7f9ea0104140 2024-04-16T17:05:51.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:51.744+0000 7f9e9f200640 1 --2- 172.21.15.45:0/2298037689 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9ea0105cf0 0x7f9ea01a1a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:05:51.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:51.744+0000 7f9e9e800640 1 --2- 172.21.15.45:0/2298037689 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9ea0104140 0x7f9ea019a450 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f9e9002c0b0 tx=0x7f9e90032430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:05:51.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:51.744+0000 7f9e97400640 1 -- 172.21.15.45:0/2298037689 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e90041020 con 0x7f9ea0104140 2024-04-16T17:05:51.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:51.744+0000 7f9ea4a00640 1 -- 172.21.15.45:0/2298037689 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ea01a2300 con 0x7f9ea0104140 2024-04-16T17:05:51.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:51.744+0000 7f9e97400640 1 -- 172.21.15.45:0/2298037689 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9e9002c9f0 con 0x7f9ea0104140 2024-04-16T17:05:51.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:51.744+0000 7f9e97400640 1 -- 172.21.15.45:0/2298037689 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e90039040 con 0x7f9ea0104140 2024-04-16T17:05:51.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:51.745+0000 7f9ea4a00640 1 -- 172.21.15.45:0/2298037689 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ea01a2840 con 0x7f9ea0104140 2024-04-16T17:05:51.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:51.746+0000 7f9ea4a00640 1 -- 172.21.15.45:0/2298037689 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ea010b480 con 0x7f9ea0104140 2024-04-16T17:05:51.748 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:51.748+0000 7f9e97400640 1 -- 172.21.15.45:0/2298037689 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f9e90044020 con 0x7f9ea0104140 2024-04-16T17:05:51.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:51.749+0000 7f9e97400640 1 --2- 172.21.15.45:0/2298037689 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f9e840a3c30 0x7f9e840a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:51.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:51.749+0000 7f9e97400640 1 -- 172.21.15.45:0/2298037689 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f9e900f1fe0 con 0x7f9ea0104140 2024-04-16T17:05:51.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:51.749+0000 7f9e9de00640 1 --2- 172.21.15.45:0/2298037689 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f9e840a3c30 0x7f9e840a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:51.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:51.750+0000 7f9e9de00640 1 --2- 172.21.15.45:0/2298037689 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f9e840a3c30 0x7f9e840a60f0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f9ea019b960 tx=0x7f9e8c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:05:51.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:51.754+0000 7f9e97400640 1 -- 172.21.15.45:0/2298037689 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9e9004b020 con 0x7f9ea0104140 2024-04-16T17:05:51.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:05:51 smithi117 ceph-mon[36730]: pgmap v399: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-16T17:05:51.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:05:51 smithi117 ceph-mon[36730]: from='client.? 172.21.15.45:0/255852379' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T17:05:51.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:05:51 smithi117 ceph-mon[36730]: from='client.? 172.21.15.45:0/1164651242' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-16T17:05:51.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:51.961+0000 7f9ea4a00640 1 -- 172.21.15.45:0/2298037689 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f9ea0062c00 con 0x7f9ea0104140 2024-04-16T17:05:51.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:51.963+0000 7f9e97400640 1 -- 172.21.15.45:0/2298037689 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v94) ==== 100+0+11 (secure 0 0 0) 0x7f9e900be630 con 0x7f9ea0104140 2024-04-16T17:05:51.963 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 64 2024-04-16T17:05:51.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:51.965+0000 7f9ea4a00640 1 -- 172.21.15.45:0/2298037689 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f9e840a3c30 msgr2=0x7f9e840a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:51.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:51.965+0000 7f9ea4a00640 1 --2- 172.21.15.45:0/2298037689 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f9e840a3c30 0x7f9e840a60f0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f9ea019b960 tx=0x7f9e8c005f90 comp rx=0 tx=0).stop 2024-04-16T17:05:51.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:51.965+0000 7f9ea4a00640 1 -- 172.21.15.45:0/2298037689 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9ea0104140 msgr2=0x7f9ea019a450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:51.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:51.965+0000 7f9ea4a00640 1 --2- 172.21.15.45:0/2298037689 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9ea0104140 0x7f9ea019a450 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f9e9002c0b0 tx=0x7f9e90032430 comp rx=0 tx=0).stop 2024-04-16T17:05:51.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:51.965+0000 7f9ea4a00640 1 -- 172.21.15.45:0/2298037689 shutdown_connections 2024-04-16T17:05:51.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:51.965+0000 7f9ea4a00640 1 --2- 172.21.15.45:0/2298037689 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f9e840a3c30 0x7f9e840a60f0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:51.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:51.965+0000 7f9ea4a00640 1 --2- 172.21.15.45:0/2298037689 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9ea0105330 0x7f9ea019a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:51.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:51.965+0000 7f9ea4a00640 1 --2- 172.21.15.45:0/2298037689 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9ea0105cf0 0x7f9ea01a1a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:51.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:51.965+0000 7f9ea4a00640 1 --2- 172.21.15.45:0/2298037689 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9ea0104140 0x7f9ea019a450 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:51.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:51.965+0000 7f9ea4a00640 1 -- 172.21.15.45:0/2298037689 >> 172.21.15.45:0/2298037689 conn(0x7f9ea00ff6b0 msgr2=0x7f9ea0108650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:05:51.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:51.966+0000 7f9ea4a00640 1 -- 172.21.15.45:0/2298037689 shutdown_connections 2024-04-16T17:05:51.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:51.966+0000 7f9ea4a00640 1 -- 172.21.15.45:0/2298037689 wait complete. 2024-04-16T17:05:51.985 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-16T17:05:52.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:52.213+0000 7f81ebe00640 1 -- 172.21.15.45:0/3088140383 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f81e4103bf0 msgr2=0x7f81e410a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:52.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:52.213+0000 7f81ebe00640 1 --2- 172.21.15.45:0/3088140383 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f81e4103bf0 0x7f81e410a480 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f81dc009ca0 tx=0x7f81dc02c0f0 comp rx=0 tx=0).stop 2024-04-16T17:05:52.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:52.214+0000 7f81ebe00640 1 -- 172.21.15.45:0/3088140383 shutdown_connections 2024-04-16T17:05:52.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:52.214+0000 7f81ebe00640 1 --2- 172.21.15.45:0/3088140383 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f81e4103bf0 0x7f81e410a480 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:52.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:52.214+0000 7f81ebe00640 1 --2- 172.21.15.45:0/3088140383 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f81e4103230 0x7f81e41036b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:52.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:52.214+0000 7f81ebe00640 1 --2- 172.21.15.45:0/3088140383 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f81e4102040 0x7f81e4102460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:52.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:52.214+0000 7f81ebe00640 1 -- 172.21.15.45:0/3088140383 >> 172.21.15.45:0/3088140383 conn(0x7f81e40fd5b0 msgr2=0x7f81e40ff9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:05:52.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:52.214+0000 7f81ebe00640 1 -- 172.21.15.45:0/3088140383 shutdown_connections 2024-04-16T17:05:52.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:52.214+0000 7f81ebe00640 1 -- 172.21.15.45:0/3088140383 wait complete. 2024-04-16T17:05:52.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:52.215+0000 7f81ebe00640 1 Processor -- start 2024-04-16T17:05:52.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:52.215+0000 7f81ebe00640 1 -- start start 2024-04-16T17:05:52.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:52.215+0000 7f81ebe00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f81e4102040 0x7f81e419a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:52.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:52.215+0000 7f81ebe00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f81e4103230 0x7f81e419aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:52.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:52.215+0000 7f81ebe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f81e4103bf0 0x7f81e41a1aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:52.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:52.215+0000 7f81ebe00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f81e410dcd0 con 0x7f81e4103bf0 2024-04-16T17:05:52.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:52.215+0000 7f81ebe00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f81e410db50 con 0x7f81e4103230 2024-04-16T17:05:52.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:52.215+0000 7f81ebe00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f81e410de50 con 0x7f81e4102040 2024-04-16T17:05:52.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:52.216+0000 7f81e8a00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f81e4103230 0x7f81e419aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:52.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:52.216+0000 7f81e9e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f81e4103bf0 0x7f81e41a1aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:52.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:52.216+0000 7f81e8a00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f81e4103230 0x7f81e419aa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:51720/0 (socket says 172.21.15.45:51720) 2024-04-16T17:05:52.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:52.216+0000 7f81e8a00640 1 -- 172.21.15.45:0/3749782368 learned_addr learned my addr 172.21.15.45:0/3749782368 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:05:52.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:52.216+0000 7f81e9400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f81e4102040 0x7f81e419a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:52.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:52.216+0000 7f81e8a00640 1 -- 172.21.15.45:0/3749782368 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f81e4102040 msgr2=0x7f81e419a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:52.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:52.216+0000 7f81e8a00640 1 --2- 172.21.15.45:0/3749782368 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f81e4102040 0x7f81e419a4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:52.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:52.216+0000 7f81e8a00640 1 -- 172.21.15.45:0/3749782368 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f81e4103bf0 msgr2=0x7f81e41a1aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:52.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:52.216+0000 7f81e8a00640 1 --2- 172.21.15.45:0/3749782368 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f81e4103bf0 0x7f81e41a1aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:52.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:52.216+0000 7f81e8a00640 1 -- 172.21.15.45:0/3749782368 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81e41a2110 con 0x7f81e4103230 2024-04-16T17:05:52.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:52.216+0000 7f81e9e00640 1 --2- 172.21.15.45:0/3749782368 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f81e4103bf0 0x7f81e41a1aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:05:52.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:52.216+0000 7f81e8a00640 1 --2- 172.21.15.45:0/3749782368 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f81e4103230 0x7f81e419aa30 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f81d000e9f0 tx=0x7f81d000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:05:52.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:52.217+0000 7f81e2000640 1 -- 172.21.15.45:0/3749782368 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81d000cd80 con 0x7f81e4103230 2024-04-16T17:05:52.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:52.217+0000 7f81ebe00640 1 -- 172.21.15.45:0/3749782368 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81e41a2400 con 0x7f81e4103230 2024-04-16T17:05:52.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:52.217+0000 7f81e2000640 1 -- 172.21.15.45:0/3749782368 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f81d0010910 con 0x7f81e4103230 2024-04-16T17:05:52.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:52.217+0000 7f81e2000640 1 -- 172.21.15.45:0/3749782368 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81d0018990 con 0x7f81e4103230 2024-04-16T17:05:52.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:52.217+0000 7f81ebe00640 1 -- 172.21.15.45:0/3749782368 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81e41a2950 con 0x7f81e4103230 2024-04-16T17:05:52.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:52.218+0000 7f81ebe00640 1 -- 172.21.15.45:0/3749782368 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81e41a2bb0 con 0x7f81e4103230 2024-04-16T17:05:52.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:52.221+0000 7f81e2000640 1 -- 172.21.15.45:0/3749782368 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f81d0010ac0 con 0x7f81e4103230 2024-04-16T17:05:52.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:52.222+0000 7f81e2000640 1 --2- 172.21.15.45:0/3749782368 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f81cc0a3bb0 0x7f81cc0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:52.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:52.222+0000 7f81e2000640 1 -- 172.21.15.45:0/3749782368 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f81d00d0a60 con 0x7f81e4103230 2024-04-16T17:05:52.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:52.222+0000 7f81e9400640 1 --2- 172.21.15.45:0/3749782368 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f81cc0a3bb0 0x7f81cc0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:52.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:52.223+0000 7f81e9400640 1 --2- 172.21.15.45:0/3749782368 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f81cc0a3bb0 0x7f81cc0a6070 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f81d80095f0 tx=0x7f81d8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:05:52.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:52.226+0000 7f81e2000640 1 -- 172.21.15.45:0/3749782368 <== mon.2 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f81d009ca60 con 0x7f81e4103230 2024-04-16T17:05:52.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:52.437+0000 7f81ebe00640 1 -- 172.21.15.45:0/3749782368 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f81e41a33e0 con 0x7f81e4103230 2024-04-16T17:05:52.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:52.438+0000 7f81e2000640 1 -- 172.21.15.45:0/3749782368 <== mon.2 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v94) ==== 103+0+11 (secure 0 0 0) 0x7f81d00a0d60 con 0x7f81e4103230 2024-04-16T17:05:52.439 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 32 2024-04-16T17:05:52.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:52.441+0000 7f81ebe00640 1 -- 172.21.15.45:0/3749782368 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f81cc0a3bb0 msgr2=0x7f81cc0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:52.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:52.441+0000 7f81ebe00640 1 --2- 172.21.15.45:0/3749782368 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f81cc0a3bb0 0x7f81cc0a6070 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f81d80095f0 tx=0x7f81d8005f90 comp rx=0 tx=0).stop 2024-04-16T17:05:52.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:52.441+0000 7f81ebe00640 1 -- 172.21.15.45:0/3749782368 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f81e4103230 msgr2=0x7f81e419aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:52.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:52.441+0000 7f81ebe00640 1 --2- 172.21.15.45:0/3749782368 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f81e4103230 0x7f81e419aa30 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f81d000e9f0 tx=0x7f81d000eeb0 comp rx=0 tx=0).stop 2024-04-16T17:05:52.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:52.441+0000 7f81ebe00640 1 -- 172.21.15.45:0/3749782368 shutdown_connections 2024-04-16T17:05:52.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:52.441+0000 7f81ebe00640 1 --2- 172.21.15.45:0/3749782368 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f81cc0a3bb0 0x7f81cc0a6070 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:52.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:52.441+0000 7f81ebe00640 1 --2- 172.21.15.45:0/3749782368 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f81e4103bf0 0x7f81e41a1aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:52.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:52.441+0000 7f81ebe00640 1 --2- 172.21.15.45:0/3749782368 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f81e4103230 0x7f81e419aa30 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:52.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:52.441+0000 7f81ebe00640 1 --2- 172.21.15.45:0/3749782368 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f81e4102040 0x7f81e419a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:52.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:52.441+0000 7f81ebe00640 1 -- 172.21.15.45:0/3749782368 >> 172.21.15.45:0/3749782368 conn(0x7f81e40fd5b0 msgr2=0x7f81e4074230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:05:52.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:52.441+0000 7f81ebe00640 1 -- 172.21.15.45:0/3749782368 shutdown_connections 2024-04-16T17:05:52.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:52.442+0000 7f81ebe00640 1 -- 172.21.15.45:0/3749782368 wait complete. 2024-04-16T17:05:52.463 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-16T17:05:52.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:05:52 smithi104 ceph-mon[36457]: from='client.? 172.21.15.45:0/2931023939' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-16T17:05:52.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:05:52 smithi104 ceph-mon[36457]: from='client.? 172.21.15.45:0/2298037689' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-16T17:05:52.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:52.693+0000 7fb7cb200640 1 -- 172.21.15.45:0/2302767577 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb7c40785a0 msgr2=0x7fb7c4078e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:52.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:52.693+0000 7fb7cb200640 1 --2- 172.21.15.45:0/2302767577 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb7c40785a0 0x7fb7c4078e40 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fb7bc009ca0 tx=0x7fb7bc02c100 comp rx=0 tx=0).stop 2024-04-16T17:05:52.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:52.693+0000 7fb7cb200640 1 -- 172.21.15.45:0/2302767577 shutdown_connections 2024-04-16T17:05:52.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:52.693+0000 7fb7cb200640 1 --2- 172.21.15.45:0/2302767577 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb7c40785a0 0x7fb7c4078e40 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:52.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:52.693+0000 7fb7cb200640 1 --2- 172.21.15.45:0/2302767577 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb7c4077c00 0x7fb7c4078060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:52.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:52.693+0000 7fb7cb200640 1 --2- 172.21.15.45:0/2302767577 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb7c40769c0 0x7fb7c4076de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:52.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:52.693+0000 7fb7cb200640 1 -- 172.21.15.45:0/2302767577 >> 172.21.15.45:0/2302767577 conn(0x7fb7c40ff7f0 msgr2=0x7fb7c4101c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:05:52.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:52.693+0000 7fb7cb200640 1 -- 172.21.15.45:0/2302767577 shutdown_connections 2024-04-16T17:05:52.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:52.693+0000 7fb7cb200640 1 -- 172.21.15.45:0/2302767577 wait complete. 2024-04-16T17:05:52.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:52.694+0000 7fb7cb200640 1 Processor -- start 2024-04-16T17:05:52.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:52.694+0000 7fb7cb200640 1 -- start start 2024-04-16T17:05:52.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:52.694+0000 7fb7cb200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb7c40769c0 0x7fb7c419e910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:52.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:52.694+0000 7fb7cb200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb7c4077c00 0x7fb7c419ee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:52.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:52.694+0000 7fb7cb200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb7c40785a0 0x7fb7c41a5ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:52.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:52.694+0000 7fb7cb200640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fb7c4114310 con 0x7fb7c40769c0 2024-04-16T17:05:52.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:52.694+0000 7fb7cb200640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fb7c4114190 con 0x7fb7c40785a0 2024-04-16T17:05:52.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:52.694+0000 7fb7cb200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb7c4114490 con 0x7fb7c4077c00 2024-04-16T17:05:52.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:52.695+0000 7fb7c3e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb7c40769c0 0x7fb7c419e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:52.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:52.695+0000 7fb7c3e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb7c40769c0 0x7fb7c419e910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39330/0 (socket says 172.21.15.45:39330) 2024-04-16T17:05:52.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:52.695+0000 7fb7c3e00640 1 -- 172.21.15.45:0/1342781054 learned_addr learned my addr 172.21.15.45:0/1342781054 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:05:52.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:52.695+0000 7fb7c9200640 1 --2- 172.21.15.45:0/1342781054 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb7c40785a0 0x7fb7c41a5ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:52.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:52.695+0000 7fb7c3400640 1 --2- 172.21.15.45:0/1342781054 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb7c4077c00 0x7fb7c419ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:52.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:52.695+0000 7fb7c3e00640 1 -- 172.21.15.45:0/1342781054 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb7c4077c00 msgr2=0x7fb7c419ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:52.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:52.695+0000 7fb7c3e00640 1 --2- 172.21.15.45:0/1342781054 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb7c4077c00 0x7fb7c419ee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:52.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:52.695+0000 7fb7c3e00640 1 -- 172.21.15.45:0/1342781054 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb7c40785a0 msgr2=0x7fb7c41a5ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:52.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:52.695+0000 7fb7c3e00640 1 --2- 172.21.15.45:0/1342781054 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb7c40785a0 0x7fb7c41a5ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:52.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:52.695+0000 7fb7c3e00640 1 -- 172.21.15.45:0/1342781054 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7c41a6530 con 0x7fb7c40769c0 2024-04-16T17:05:52.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:52.695+0000 7fb7c9200640 1 --2- 172.21.15.45:0/1342781054 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb7c40785a0 0x7fb7c41a5ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:05:52.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:52.696+0000 7fb7c3e00640 1 --2- 172.21.15.45:0/1342781054 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb7c40769c0 0x7fb7c419e910 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fb7b400ea60 tx=0x7fb7b400ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:05:52.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:52.696+0000 7fb7c0c00640 1 -- 172.21.15.45:0/1342781054 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7b400cde0 con 0x7fb7c40769c0 2024-04-16T17:05:52.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:52.696+0000 7fb7cb200640 1 -- 172.21.15.45:0/1342781054 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7c41a6820 con 0x7fb7c40769c0 2024-04-16T17:05:52.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:52.696+0000 7fb7c0c00640 1 -- 172.21.15.45:0/1342781054 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb7b40035f0 con 0x7fb7c40769c0 2024-04-16T17:05:52.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:52.696+0000 7fb7c0c00640 1 -- 172.21.15.45:0/1342781054 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7b4010670 con 0x7fb7c40769c0 2024-04-16T17:05:52.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:52.696+0000 7fb7cb200640 1 -- 172.21.15.45:0/1342781054 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7c41a6dc0 con 0x7fb7c40769c0 2024-04-16T17:05:52.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:52.697+0000 7fb7cb200640 1 -- 172.21.15.45:0/1342781054 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb7c4110d40 con 0x7fb7c40769c0 2024-04-16T17:05:52.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:52.699+0000 7fb7c0c00640 1 -- 172.21.15.45:0/1342781054 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fb7b40037a0 con 0x7fb7c40769c0 2024-04-16T17:05:52.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:52.701+0000 7fb7c0c00640 1 --2- 172.21.15.45:0/1342781054 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fb7a80a3bb0 0x7fb7a80a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:52.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:52.701+0000 7fb7c0c00640 1 -- 172.21.15.45:0/1342781054 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fb7b40d0aa0 con 0x7fb7c40769c0 2024-04-16T17:05:52.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:52.701+0000 7fb7c3400640 1 --2- 172.21.15.45:0/1342781054 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fb7a80a3bb0 0x7fb7a80a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:52.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:52.702+0000 7fb7c3400640 1 --2- 172.21.15.45:0/1342781054 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fb7a80a3bb0 0x7fb7a80a6070 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fb7c419fe20 tx=0x7fb7b0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:05:52.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:52.705+0000 7fb7c0c00640 1 -- 172.21.15.45:0/1342781054 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb7b409d090 con 0x7fb7c40769c0 2024-04-16T17:05:52.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:05:52 smithi045 ceph-mon[31676]: from='client.? 172.21.15.45:0/2931023939' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-16T17:05:52.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:05:52 smithi045 ceph-mon[31676]: from='client.? 172.21.15.45:0/2298037689' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-16T17:05:52.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:05:52 smithi117 ceph-mon[36730]: from='client.? 172.21.15.45:0/2931023939' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-16T17:05:52.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:05:52 smithi117 ceph-mon[36730]: from='client.? 172.21.15.45:0/2298037689' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-16T17:05:52.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:52.958+0000 7fb7cb200640 1 -- 172.21.15.45:0/1342781054 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb7c402d620 con 0x7fb7c40769c0 2024-04-16T17:05:52.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:52.960+0000 7fb7c0c00640 1 -- 172.21.15.45:0/1342781054 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11887 (secure 0 0 0) 0x7fb7c402d620 con 0x7fb7c40769c0 2024-04-16T17:05:52.960 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-16T17:05:52.961 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/1748842443","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":1748842443},{"type":"v1","addr":"172.21.15.45:6835","nonce":1748842443}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6839/1308627051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":1308627051},{"type":"v1","addr":"172.21.15.45:6839","nonce":1308627051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/1768242489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1768242489},{"type":"v1","addr":"172.21.15.117:6833","nonce":1768242489}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6833/3558902585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":3558902585},{"type":"v1","addr":"172.21.15.104:6833","nonce":3558902585}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24478,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/1308142903","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":1308142903},{"type":"v1","addr":"172.21.15.45:6837","nonce":1308142903}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24491,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6835/3430679937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":3430679937},{"type":"v1","addr":"172.21.15.104:6835","nonce":3430679937}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-16T17:03:24.663611+0000","modified":"2024-04-16T17:03:53.381209+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24581,"mds_1":24569,"mds_2":24532},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24499":{"gid":24499,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/2224123841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2224123841},{"type":"v1","addr":"172.21.15.117:6835","nonce":2224123841}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6837/2471356815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":2471356815},{"type":"v1","addr":"172.21.15.117:6837","nonce":2471356815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24524":{"gid":24524,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6837/2647756509","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":2647756509},{"type":"v1","addr":"172.21.15.104:6837","nonce":2647756509}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24532":{"gid":24532,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":13,"addr":"172.21.15.117:6839/2378675949","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":2378675949},{"type":"v1","addr":"172.21.15.117:6839","nonce":2378675949}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24569":{"gid":24569,"name":"a","rank":1,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.45:6841/3755310822","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6840","nonce":3755310822},{"type":"v1","addr":"172.21.15.45:6841","nonce":3755310822}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24581":{"gid":24581,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.104:6839/3660070592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6838","nonce":3660070592},{"type":"v1","addr":"172.21.15.104:6839","nonce":3660070592}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24581,"qdb_cluster":[24569,24532,24581]},"id":1}]} 2024-04-16T17:05:52.961 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 24 2024-04-16T17:05:52.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:52.962+0000 7fb7cb200640 1 -- 172.21.15.45:0/1342781054 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fb7a80a3bb0 msgr2=0x7fb7a80a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:52.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:52.962+0000 7fb7cb200640 1 --2- 172.21.15.45:0/1342781054 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fb7a80a3bb0 0x7fb7a80a6070 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fb7c419fe20 tx=0x7fb7b0005f90 comp rx=0 tx=0).stop 2024-04-16T17:05:52.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:52.962+0000 7fb7cb200640 1 -- 172.21.15.45:0/1342781054 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb7c40769c0 msgr2=0x7fb7c419e910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:52.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:52.962+0000 7fb7cb200640 1 --2- 172.21.15.45:0/1342781054 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb7c40769c0 0x7fb7c419e910 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fb7b400ea60 tx=0x7fb7b400ef20 comp rx=0 tx=0).stop 2024-04-16T17:05:52.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:52.963+0000 7fb7cb200640 1 -- 172.21.15.45:0/1342781054 shutdown_connections 2024-04-16T17:05:52.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:52.963+0000 7fb7cb200640 1 --2- 172.21.15.45:0/1342781054 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fb7a80a3bb0 0x7fb7a80a6070 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:52.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:52.963+0000 7fb7cb200640 1 --2- 172.21.15.45:0/1342781054 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb7c4077c00 0x7fb7c419ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:52.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:52.963+0000 7fb7cb200640 1 --2- 172.21.15.45:0/1342781054 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb7c40785a0 0x7fb7c41a5ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:52.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:52.963+0000 7fb7cb200640 1 --2- 172.21.15.45:0/1342781054 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb7c40769c0 0x7fb7c419e910 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:52.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:52.963+0000 7fb7cb200640 1 -- 172.21.15.45:0/1342781054 >> 172.21.15.45:0/1342781054 conn(0x7fb7c40ff7f0 msgr2=0x7fb7c4100ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:05:52.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:52.963+0000 7fb7cb200640 1 -- 172.21.15.45:0/1342781054 shutdown_connections 2024-04-16T17:05:52.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:52.963+0000 7fb7cb200640 1 -- 172.21.15.45:0/1342781054 wait complete. 2024-04-16T17:05:52.982 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-16T17:05:53.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:53.209+0000 7efc68a00640 1 -- 172.21.15.45:0/95662356 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efc64105c30 msgr2=0x7efc6410c4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:53.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:53.209+0000 7efc68a00640 1 --2- 172.21.15.45:0/95662356 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efc64105c30 0x7efc6410c4c0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7efc5c009ca0 tx=0x7efc5c02c0f0 comp rx=0 tx=0).stop 2024-04-16T17:05:53.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:53.209+0000 7efc68a00640 1 -- 172.21.15.45:0/95662356 shutdown_connections 2024-04-16T17:05:53.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:53.209+0000 7efc68a00640 1 --2- 172.21.15.45:0/95662356 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efc64105270 0x7efc641056f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:53.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:53.209+0000 7efc68a00640 1 --2- 172.21.15.45:0/95662356 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efc64105c30 0x7efc6410c4c0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:53.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:53.209+0000 7efc68a00640 1 --2- 172.21.15.45:0/95662356 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efc64104080 0x7efc641044a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:53.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:53.209+0000 7efc68a00640 1 -- 172.21.15.45:0/95662356 >> 172.21.15.45:0/95662356 conn(0x7efc640ff610 msgr2=0x7efc64101a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:05:53.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:53.209+0000 7efc68a00640 1 -- 172.21.15.45:0/95662356 shutdown_connections 2024-04-16T17:05:53.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:53.209+0000 7efc68a00640 1 -- 172.21.15.45:0/95662356 wait complete. 2024-04-16T17:05:53.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:53.210+0000 7efc68a00640 1 Processor -- start 2024-04-16T17:05:53.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:53.211+0000 7efc68a00640 1 -- start start 2024-04-16T17:05:53.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:53.211+0000 7efc68a00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efc64104080 0x7efc6419a360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:53.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:53.211+0000 7efc68a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efc64105270 0x7efc6419a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:53.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:53.211+0000 7efc68a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efc64105c30 0x7efc641a1910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:53.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:53.211+0000 7efc68a00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7efc6410fd10 con 0x7efc64105270 2024-04-16T17:05:53.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:53.211+0000 7efc68a00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7efc6410fb90 con 0x7efc64104080 2024-04-16T17:05:53.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:53.211+0000 7efc68a00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7efc6410fe90 con 0x7efc64105c30 2024-04-16T17:05:53.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:53.211+0000 7efc62a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efc64105270 0x7efc6419a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:53.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:53.211+0000 7efc62a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efc64105270 0x7efc6419a8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39352/0 (socket says 172.21.15.45:39352) 2024-04-16T17:05:53.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:53.211+0000 7efc62a00640 1 -- 172.21.15.45:0/1227739277 learned_addr learned my addr 172.21.15.45:0/1227739277 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:05:53.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:53.211+0000 7efc63400640 1 --2- 172.21.15.45:0/1227739277 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efc64104080 0x7efc6419a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:53.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:53.211+0000 7efc63e00640 1 --2- 172.21.15.45:0/1227739277 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efc64105c30 0x7efc641a1910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:53.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:53.212+0000 7efc62a00640 1 -- 172.21.15.45:0/1227739277 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efc64105c30 msgr2=0x7efc641a1910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:53.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:53.212+0000 7efc62a00640 1 --2- 172.21.15.45:0/1227739277 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efc64105c30 0x7efc641a1910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:53.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:53.212+0000 7efc62a00640 1 -- 172.21.15.45:0/1227739277 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efc64104080 msgr2=0x7efc6419a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:53.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:53.212+0000 7efc62a00640 1 --2- 172.21.15.45:0/1227739277 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efc64104080 0x7efc6419a360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:53.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:53.212+0000 7efc62a00640 1 -- 172.21.15.45:0/1227739277 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc641a1f80 con 0x7efc64105270 2024-04-16T17:05:53.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:53.212+0000 7efc63400640 1 --2- 172.21.15.45:0/1227739277 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efc64104080 0x7efc6419a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:05:53.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:53.212+0000 7efc62a00640 1 --2- 172.21.15.45:0/1227739277 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efc64105270 0x7efc6419a8a0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7efc5000e9f0 tx=0x7efc5000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:05:53.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:53.212+0000 7efc5be00640 1 -- 172.21.15.45:0/1227739277 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc5000cd80 con 0x7efc64105270 2024-04-16T17:05:53.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:53.212+0000 7efc5be00640 1 -- 172.21.15.45:0/1227739277 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efc50005500 con 0x7efc64105270 2024-04-16T17:05:53.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:53.213+0000 7efc5be00640 1 -- 172.21.15.45:0/1227739277 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc50010690 con 0x7efc64105270 2024-04-16T17:05:53.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:53.213+0000 7efc68a00640 1 -- 172.21.15.45:0/1227739277 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc641a2270 con 0x7efc64105270 2024-04-16T17:05:53.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:53.213+0000 7efc68a00640 1 -- 172.21.15.45:0/1227739277 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc641a2770 con 0x7efc64105270 2024-04-16T17:05:53.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:53.214+0000 7efc68a00640 1 -- 172.21.15.45:0/1227739277 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc6410b3a0 con 0x7efc64105270 2024-04-16T17:05:53.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:53.217+0000 7efc5be00640 1 -- 172.21.15.45:0/1227739277 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7efc500046c0 con 0x7efc64105270 2024-04-16T17:05:53.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:53.218+0000 7efc5be00640 1 --2- 172.21.15.45:0/1227739277 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7efc480a3bb0 0x7efc480a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:53.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:53.218+0000 7efc5be00640 1 -- 172.21.15.45:0/1227739277 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7efc500d0bf0 con 0x7efc64105270 2024-04-16T17:05:53.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:53.218+0000 7efc63400640 1 --2- 172.21.15.45:0/1227739277 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7efc480a3bb0 0x7efc480a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:53.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:53.219+0000 7efc63400640 1 --2- 172.21.15.45:0/1227739277 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7efc480a3bb0 0x7efc480a6070 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7efc641050d0 tx=0x7efc54005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:05:53.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:53.223+0000 7efc5be00640 1 -- 172.21.15.45:0/1227739277 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7efc5009d1e0 con 0x7efc64105270 2024-04-16T17:05:53.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:53.440+0000 7efc68a00640 1 -- 172.21.15.45:0/1227739277 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7efc64062c00 con 0x7efc64105270 2024-04-16T17:05:53.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:53.442+0000 7efc5be00640 1 -- 172.21.15.45:0/1227739277 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) ==== 74+0+24726 (secure 0 0 0) 0x7efc5009cae0 con 0x7efc64105270 2024-04-16T17:05:53.443 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-16T17:05:53.444 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":94,"fsid":"b1fecd38-fc11-11ee-bc90-c7b262605968","created":"2024-04-16T16:54:02.385906+0000","modified":"2024-04-16T17:05:22.272335+0000","last_up_change":"2024-04-16T17:02:25.162624+0000","last_in_change":"2024-04-16T17:02:04.903274+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-16T16:58:06.166659+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-16T17:03:22.304198+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-16T17:03:23.616450+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-16T17:03:27.226289+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b655de2c-59f8-4fd4-ac98-0cc222229f56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":2290170881},{"type":"v1","addr":"172.21.15.45:6803","nonce":2290170881}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":2290170881},{"type":"v1","addr":"172.21.15.45:6805","nonce":2290170881}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":2290170881},{"type":"v1","addr":"172.21.15.45:6809","nonce":2290170881}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":2290170881},{"type":"v1","addr":"172.21.15.45:6807","nonce":2290170881}]},"public_addr":"172.21.15.45:6803/2290170881","cluster_addr":"172.21.15.45:6805/2290170881","heartbeat_back_addr":"172.21.15.45:6809/2290170881","heartbeat_front_addr":"172.21.15.45:6807/2290170881","state":["exists","up"]},{"osd":1,"uuid":"ebd450ec-08fd-4c3d-8b24-4e10634ddb7e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6800","nonce":1462942067},{"type":"v1","addr":"172.21.15.104:6801","nonce":1462942067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":1462942067},{"type":"v1","addr":"172.21.15.104:6803","nonce":1462942067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":1462942067},{"type":"v1","addr":"172.21.15.104:6807","nonce":1462942067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":1462942067},{"type":"v1","addr":"172.21.15.104:6805","nonce":1462942067}]},"public_addr":"172.21.15.104:6801/1462942067","cluster_addr":"172.21.15.104:6803/1462942067","heartbeat_back_addr":"172.21.15.104:6807/1462942067","heartbeat_front_addr":"172.21.15.104:6805/1462942067","state":["exists","up"]},{"osd":2,"uuid":"b318a512-4190-42a2-8104-dcc4026ad8ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":2829260735},{"type":"v1","addr":"172.21.15.117:6801","nonce":2829260735}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":2829260735},{"type":"v1","addr":"172.21.15.117:6803","nonce":2829260735}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":2829260735},{"type":"v1","addr":"172.21.15.117:6807","nonce":2829260735}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":2829260735},{"type":"v1","addr":"172.21.15.117:6805","nonce":2829260735}]},"public_addr":"172.21.15.117:6801/2829260735","cluster_addr":"172.21.15.117:6803/2829260735","heartbeat_back_addr":"172.21.15.117:6807/2829260735","heartbeat_front_addr":"172.21.15.117:6805/2829260735","state":["exists","up"]},{"osd":3,"uuid":"ec6490cb-a715-4c12-a54a-225aae26fcb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":2823712137},{"type":"v1","addr":"172.21.15.45:6811","nonce":2823712137}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":2823712137},{"type":"v1","addr":"172.21.15.45:6813","nonce":2823712137}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":2823712137},{"type":"v1","addr":"172.21.15.45:6817","nonce":2823712137}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":2823712137},{"type":"v1","addr":"172.21.15.45:6815","nonce":2823712137}]},"public_addr":"172.21.15.45:6811/2823712137","cluster_addr":"172.21.15.45:6813/2823712137","heartbeat_back_addr":"172.21.15.45:6817/2823712137","heartbeat_front_addr":"172.21.15.45:6815/2823712137","state":["exists","up"]},{"osd":4,"uuid":"5d91d568-0250-4aae-8dc2-a0e079f92062","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":1396278127},{"type":"v1","addr":"172.21.15.104:6809","nonce":1396278127}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":1396278127},{"type":"v1","addr":"172.21.15.104:6811","nonce":1396278127}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":1396278127},{"type":"v1","addr":"172.21.15.104:6815","nonce":1396278127}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":1396278127},{"type":"v1","addr":"172.21.15.104:6813","nonce":1396278127}]},"public_addr":"172.21.15.104:6809/1396278127","cluster_addr":"172.21.15.104:6811/1396278127","heartbeat_back_addr":"172.21.15.104:6815/1396278127","heartbeat_front_addr":"172.21.15.104:6813/1396278127","state":["exists","up"]},{"osd":5,"uuid":"ac510b43-6fb2-4d0f-bc5f-16ac080d9e9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":4145720047},{"type":"v1","addr":"172.21.15.117:6809","nonce":4145720047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":4145720047},{"type":"v1","addr":"172.21.15.117:6811","nonce":4145720047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":4145720047},{"type":"v1","addr":"172.21.15.117:6815","nonce":4145720047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":4145720047},{"type":"v1","addr":"172.21.15.117:6813","nonce":4145720047}]},"public_addr":"172.21.15.117:6809/4145720047","cluster_addr":"172.21.15.117:6811/4145720047","heartbeat_back_addr":"172.21.15.117:6815/4145720047","heartbeat_front_addr":"172.21.15.117:6813/4145720047","state":["exists","up"]},{"osd":6,"uuid":"a41adc18-d88a-419e-b2c2-b089c1f32472","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":3254229176},{"type":"v1","addr":"172.21.15.45:6819","nonce":3254229176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":3254229176},{"type":"v1","addr":"172.21.15.45:6821","nonce":3254229176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":3254229176},{"type":"v1","addr":"172.21.15.45:6825","nonce":3254229176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":3254229176},{"type":"v1","addr":"172.21.15.45:6823","nonce":3254229176}]},"public_addr":"172.21.15.45:6819/3254229176","cluster_addr":"172.21.15.45:6821/3254229176","heartbeat_back_addr":"172.21.15.45:6825/3254229176","heartbeat_front_addr":"172.21.15.45:6823/3254229176","state":["exists","up"]},{"osd":7,"uuid":"b71712a4-7cf6-454d-9fd1-565e3df21c1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":4151077186},{"type":"v1","addr":"172.21.15.104:6817","nonce":4151077186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":4151077186},{"type":"v1","addr":"172.21.15.104:6819","nonce":4151077186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":4151077186},{"type":"v1","addr":"172.21.15.104:6823","nonce":4151077186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":4151077186},{"type":"v1","addr":"172.21.15.104:6821","nonce":4151077186}]},"public_addr":"172.21.15.104:6817/4151077186","cluster_addr":"172.21.15.104:6819/4151077186","heartbeat_back_addr":"172.21.15.104:6823/4151077186","heartbeat_front_addr":"172.21.15.104:6821/4151077186","state":["exists","up"]},{"osd":8,"uuid":"9e572aef-59a8-450e-a92e-df9dff4d57a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":1500286830},{"type":"v1","addr":"172.21.15.117:6817","nonce":1500286830}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":1500286830},{"type":"v1","addr":"172.21.15.117:6819","nonce":1500286830}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":1500286830},{"type":"v1","addr":"172.21.15.117:6823","nonce":1500286830}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":1500286830},{"type":"v1","addr":"172.21.15.117:6821","nonce":1500286830}]},"public_addr":"172.21.15.117:6817/1500286830","cluster_addr":"172.21.15.117:6819/1500286830","heartbeat_back_addr":"172.21.15.117:6823/1500286830","heartbeat_front_addr":"172.21.15.117:6821/1500286830","state":["exists","up"]},{"osd":9,"uuid":"014e9598-c413-4cc4-9bb8-61c6e8f59d30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":3106734074},{"type":"v1","addr":"172.21.15.45:6827","nonce":3106734074}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":3106734074},{"type":"v1","addr":"172.21.15.45:6829","nonce":3106734074}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":3106734074},{"type":"v1","addr":"172.21.15.45:6833","nonce":3106734074}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":3106734074},{"type":"v1","addr":"172.21.15.45:6831","nonce":3106734074}]},"public_addr":"172.21.15.45:6827/3106734074","cluster_addr":"172.21.15.45:6829/3106734074","heartbeat_back_addr":"172.21.15.45:6833/3106734074","heartbeat_front_addr":"172.21.15.45:6831/3106734074","state":["exists","up"]},{"osd":10,"uuid":"2deba364-3e39-47be-8a1f-8256df5b82d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":1404794652},{"type":"v1","addr":"172.21.15.104:6825","nonce":1404794652}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":1404794652},{"type":"v1","addr":"172.21.15.104:6827","nonce":1404794652}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":1404794652},{"type":"v1","addr":"172.21.15.104:6831","nonce":1404794652}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":1404794652},{"type":"v1","addr":"172.21.15.104:6829","nonce":1404794652}]},"public_addr":"172.21.15.104:6825/1404794652","cluster_addr":"172.21.15.104:6827/1404794652","heartbeat_back_addr":"172.21.15.104:6831/1404794652","heartbeat_front_addr":"172.21.15.104:6829/1404794652","state":["exists","up"]},{"osd":11,"uuid":"3fda4fea-59ad-4d1a-9fe6-fcb30e31fd7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":3384003243},{"type":"v1","addr":"172.21.15.117:6825","nonce":3384003243}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":3384003243},{"type":"v1","addr":"172.21.15.117:6827","nonce":3384003243}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":3384003243},{"type":"v1","addr":"172.21.15.117:6831","nonce":3384003243}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":3384003243},{"type":"v1","addr":"172.21.15.117:6829","nonce":3384003243}]},"public_addr":"172.21.15.117:6825/3384003243","cluster_addr":"172.21.15.117:6827/3384003243","heartbeat_back_addr":"172.21.15.117:6831/3384003243","heartbeat_front_addr":"172.21.15.117:6829/3384003243","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:57:14.799608+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:57:39.884668+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:58:03.327057+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:58:37.314531+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:59:03.667650+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:59:28.462823+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T17:00:02.968574+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T17:00:29.559190+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T17:00:55.291007+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T17:01:29.815977+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T17:01:57.015580+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T17:02:23.233151+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.5","mappings":[{"from":9,"to":10}]},{"pgid":"4.7","mappings":[{"from":5,"to":8}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.45:0/4074982598":"2024-04-17T16:55:19.948001+0000","172.21.15.45:0/2443572576":"2024-04-17T16:55:19.948001+0000","172.21.15.45:6801/1983630136":"2024-04-17T16:55:19.948001+0000","172.21.15.45:6800/1983630136":"2024-04-17T16:55:19.948001+0000","172.21.15.45:0/3150465248":"2024-04-17T16:54:43.692466+0000","172.21.15.45:0/556738623":"2024-04-17T16:54:43.692466+0000","172.21.15.45:0/445676297":"2024-04-17T16:55:19.948001+0000","172.21.15.45:6800/2553659622":"2024-04-17T16:54:43.692466+0000","172.21.15.45:6801/2553659622":"2024-04-17T16:54:43.692466+0000","172.21.15.45:0/1184615408":"2024-04-17T16:54:43.692466+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-16T17:05:53.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:53.445+0000 7efc68a00640 1 -- 172.21.15.45:0/1227739277 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7efc480a3bb0 msgr2=0x7efc480a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:53.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:53.445+0000 7efc68a00640 1 --2- 172.21.15.45:0/1227739277 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7efc480a3bb0 0x7efc480a6070 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7efc641050d0 tx=0x7efc54005f90 comp rx=0 tx=0).stop 2024-04-16T17:05:53.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:53.446+0000 7efc68a00640 1 -- 172.21.15.45:0/1227739277 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efc64105270 msgr2=0x7efc6419a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:53.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:53.446+0000 7efc68a00640 1 --2- 172.21.15.45:0/1227739277 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efc64105270 0x7efc6419a8a0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7efc5000e9f0 tx=0x7efc5000edf0 comp rx=0 tx=0).stop 2024-04-16T17:05:53.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:53.446+0000 7efc68a00640 1 -- 172.21.15.45:0/1227739277 shutdown_connections 2024-04-16T17:05:53.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:53.446+0000 7efc68a00640 1 --2- 172.21.15.45:0/1227739277 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7efc480a3bb0 0x7efc480a6070 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:53.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:53.446+0000 7efc68a00640 1 --2- 172.21.15.45:0/1227739277 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efc64105c30 0x7efc641a1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:53.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:53.446+0000 7efc68a00640 1 --2- 172.21.15.45:0/1227739277 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efc64105270 0x7efc6419a8a0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:53.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:53.446+0000 7efc68a00640 1 --2- 172.21.15.45:0/1227739277 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efc64104080 0x7efc6419a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:53.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:53.446+0000 7efc68a00640 1 -- 172.21.15.45:0/1227739277 >> 172.21.15.45:0/1227739277 conn(0x7efc640ff610 msgr2=0x7efc64108570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:05:53.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:53.446+0000 7efc68a00640 1 -- 172.21.15.45:0/1227739277 shutdown_connections 2024-04-16T17:05:53.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:53.446+0000 7efc68a00640 1 -- 172.21.15.45:0/1227739277 wait complete. 2024-04-16T17:05:53.467 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-16T16:58:06.166659+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-16T17:03:22.304198+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-16T17:03:23.616450+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-16T17:03:27.226289+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-16T17:05:53.467 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-16T17:05:53.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:05:53 smithi104 ceph-mon[36457]: pgmap v400: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-16T17:05:53.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:05:53 smithi104 ceph-mon[36457]: from='client.? 172.21.15.45:0/3749782368' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-16T17:05:53.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:05:53 smithi104 ceph-mon[36457]: from='client.? 172.21.15.45:0/1342781054' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:05:53.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:53.699+0000 7f6d8bc00640 1 -- 172.21.15.45:0/245670597 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6d84102aa0 msgr2=0x7f6d84104e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:53.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:53.699+0000 7f6d8bc00640 1 --2- 172.21.15.45:0/245670597 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6d84102aa0 0x7f6d84104e90 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f6d74009ca0 tx=0x7f6d7402c100 comp rx=0 tx=0).stop 2024-04-16T17:05:53.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:53.700+0000 7f6d8bc00640 1 -- 172.21.15.45:0/245670597 shutdown_connections 2024-04-16T17:05:53.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:53.700+0000 7f6d8bc00640 1 --2- 172.21.15.45:0/245670597 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6d84107d00 0x7f6d8410a150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:53.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:53.700+0000 7f6d8bc00640 1 --2- 172.21.15.45:0/245670597 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6d841053d0 0x7f6d841077c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:53.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:53.700+0000 7f6d8bc00640 1 --2- 172.21.15.45:0/245670597 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6d84102aa0 0x7f6d84104e90 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:53.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:53.700+0000 7f6d8bc00640 1 -- 172.21.15.45:0/245670597 >> 172.21.15.45:0/245670597 conn(0x7f6d840fc490 msgr2=0x7f6d840fe8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:05:53.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:53.700+0000 7f6d8bc00640 1 -- 172.21.15.45:0/245670597 shutdown_connections 2024-04-16T17:05:53.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:53.700+0000 7f6d8bc00640 1 -- 172.21.15.45:0/245670597 wait complete. 2024-04-16T17:05:53.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:53.701+0000 7f6d8bc00640 1 Processor -- start 2024-04-16T17:05:53.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:53.701+0000 7f6d8bc00640 1 -- start start 2024-04-16T17:05:53.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:53.701+0000 7f6d8bc00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6d84102aa0 0x7f6d8419a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:53.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:53.701+0000 7f6d8bc00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6d841053d0 0x7f6d8419aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:53.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:53.701+0000 7f6d8bc00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6d84107d00 0x7f6d841a1b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:53.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:53.701+0000 7f6d8bc00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f6d8410dcb0 con 0x7f6d84102aa0 2024-04-16T17:05:53.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:53.701+0000 7f6d8bc00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f6d8410db30 con 0x7f6d84107d00 2024-04-16T17:05:53.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:53.701+0000 7f6d8bc00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f6d8410de30 con 0x7f6d841053d0 2024-04-16T17:05:53.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:53.702+0000 7f6d89c00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6d84107d00 0x7f6d841a1b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:53.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:53.702+0000 7f6d83e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6d841053d0 0x7f6d8419aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:53.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:53.702+0000 7f6d89200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6d84102aa0 0x7f6d8419a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:53.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:53.702+0000 7f6d89c00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6d84107d00 0x7f6d841a1b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:51762/0 (socket says 172.21.15.45:51762) 2024-04-16T17:05:53.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:53.702+0000 7f6d89c00640 1 -- 172.21.15.45:0/46292568 learned_addr learned my addr 172.21.15.45:0/46292568 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:05:53.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:53.702+0000 7f6d89c00640 1 -- 172.21.15.45:0/46292568 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6d841053d0 msgr2=0x7f6d8419aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:53.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:53.702+0000 7f6d89c00640 1 --2- 172.21.15.45:0/46292568 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6d841053d0 0x7f6d8419aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:53.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:53.702+0000 7f6d89c00640 1 -- 172.21.15.45:0/46292568 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6d84102aa0 msgr2=0x7f6d8419a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:53.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:53.702+0000 7f6d89c00640 1 --2- 172.21.15.45:0/46292568 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6d84102aa0 0x7f6d8419a560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:53.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:53.702+0000 7f6d89c00640 1 -- 172.21.15.45:0/46292568 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d841a2180 con 0x7f6d84107d00 2024-04-16T17:05:53.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:53.702+0000 7f6d83e00640 1 --2- 172.21.15.45:0/46292568 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6d841053d0 0x7f6d8419aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:05:53.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:53.702+0000 7f6d89c00640 1 --2- 172.21.15.45:0/46292568 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6d84107d00 0x7f6d841a1b10 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f6d7c00d750 tx=0x7f6d7c00dc10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:05:53.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:53.703+0000 7f6d81600640 1 -- 172.21.15.45:0/46292568 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d7c017070 con 0x7f6d84107d00 2024-04-16T17:05:53.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:53.703+0000 7f6d81600640 1 -- 172.21.15.45:0/46292568 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6d7c005500 con 0x7f6d84107d00 2024-04-16T17:05:53.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:53.703+0000 7f6d81600640 1 -- 172.21.15.45:0/46292568 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d7c004d90 con 0x7f6d84107d00 2024-04-16T17:05:53.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:53.703+0000 7f6d8bc00640 1 -- 172.21.15.45:0/46292568 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d841a2470 con 0x7f6d84107d00 2024-04-16T17:05:53.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:53.703+0000 7f6d8bc00640 1 -- 172.21.15.45:0/46292568 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d841a2970 con 0x7f6d84107d00 2024-04-16T17:05:53.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:53.704+0000 7f6d8bc00640 1 -- 172.21.15.45:0/46292568 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d84109850 con 0x7f6d84107d00 2024-04-16T17:05:53.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:53.706+0000 7f6d81600640 1 -- 172.21.15.45:0/46292568 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f6d7c004f30 con 0x7f6d84107d00 2024-04-16T17:05:53.708 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:53.707+0000 7f6d81600640 1 --2- 172.21.15.45:0/46292568 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f6d680a3c30 0x7f6d680a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:53.708 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:53.707+0000 7f6d81600640 1 -- 172.21.15.45:0/46292568 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f6d7c0d00c0 con 0x7f6d84107d00 2024-04-16T17:05:53.708 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:53.708+0000 7f6d89200640 1 --2- 172.21.15.45:0/46292568 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f6d680a3c30 0x7f6d680a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:53.708 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:53.708+0000 7f6d89200640 1 --2- 172.21.15.45:0/46292568 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f6d680a3c30 0x7f6d680a60f0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f6d7402c710 tx=0x7f6d740323c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:05:53.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:53.712+0000 7f6d81600640 1 -- 172.21.15.45:0/46292568 <== mon.2 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6d7c09c060 con 0x7f6d84107d00 2024-04-16T17:05:53.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:05:53 smithi045 ceph-mon[31676]: pgmap v400: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-16T17:05:53.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:05:53 smithi045 ceph-mon[31676]: from='client.? 172.21.15.45:0/3749782368' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-16T17:05:53.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:05:53 smithi045 ceph-mon[31676]: from='client.? 172.21.15.45:0/1342781054' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:05:53.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:05:53 smithi117 ceph-mon[36730]: pgmap v400: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-16T17:05:53.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:05:53 smithi117 ceph-mon[36730]: from='client.? 172.21.15.45:0/3749782368' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-16T17:05:53.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:05:53 smithi117 ceph-mon[36730]: from='client.? 172.21.15.45:0/1342781054' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:05:53.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:53.917+0000 7f6d8bc00640 1 -- 172.21.15.45:0/46292568 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f6d84062c00 con 0x7f6d84107d00 2024-04-16T17:05:53.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:53.918+0000 7f6d81600640 1 -- 172.21.15.45:0/46292568 <== mon.2 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v94) ==== 93+0+10 (secure 0 0 0) 0x7f6d7c0a0360 con 0x7f6d84107d00 2024-04-16T17:05:53.919 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 1 2024-04-16T17:05:53.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:53.921+0000 7f6d8bc00640 1 -- 172.21.15.45:0/46292568 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f6d680a3c30 msgr2=0x7f6d680a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:53.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:53.921+0000 7f6d8bc00640 1 --2- 172.21.15.45:0/46292568 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f6d680a3c30 0x7f6d680a60f0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f6d7402c710 tx=0x7f6d740323c0 comp rx=0 tx=0).stop 2024-04-16T17:05:53.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:53.921+0000 7f6d8bc00640 1 -- 172.21.15.45:0/46292568 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6d84107d00 msgr2=0x7f6d841a1b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:53.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:53.921+0000 7f6d8bc00640 1 --2- 172.21.15.45:0/46292568 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6d84107d00 0x7f6d841a1b10 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f6d7c00d750 tx=0x7f6d7c00dc10 comp rx=0 tx=0).stop 2024-04-16T17:05:53.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:53.921+0000 7f6d8bc00640 1 -- 172.21.15.45:0/46292568 shutdown_connections 2024-04-16T17:05:53.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:53.921+0000 7f6d8bc00640 1 --2- 172.21.15.45:0/46292568 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f6d680a3c30 0x7f6d680a60f0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:53.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:53.921+0000 7f6d8bc00640 1 --2- 172.21.15.45:0/46292568 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6d841053d0 0x7f6d8419aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:53.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:53.921+0000 7f6d8bc00640 1 --2- 172.21.15.45:0/46292568 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6d84107d00 0x7f6d841a1b10 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:53.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:53.921+0000 7f6d8bc00640 1 --2- 172.21.15.45:0/46292568 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6d84102aa0 0x7f6d8419a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:53.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:53.921+0000 7f6d8bc00640 1 -- 172.21.15.45:0/46292568 >> 172.21.15.45:0/46292568 conn(0x7f6d840fc490 msgr2=0x7f6d84102220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:05:53.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:53.921+0000 7f6d8bc00640 1 -- 172.21.15.45:0/46292568 shutdown_connections 2024-04-16T17:05:53.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:53.921+0000 7f6d8bc00640 1 -- 172.21.15.45:0/46292568 wait complete. 2024-04-16T17:05:53.941 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-16T17:05:54.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:54.173+0000 7fe05ca00640 1 -- 172.21.15.45:0/3723109038 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe058109730 msgr2=0x7fe058109bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:54.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:54.173+0000 7fe05ca00640 1 --2- 172.21.15.45:0/3723109038 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe058109730 0x7fe058109bb0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fe044009900 tx=0x7fe04402bc30 comp rx=0 tx=0).stop 2024-04-16T17:05:54.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:54.173+0000 7fe05ca00640 1 -- 172.21.15.45:0/3723109038 shutdown_connections 2024-04-16T17:05:54.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:54.173+0000 7fe05ca00640 1 --2- 172.21.15.45:0/3723109038 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe05810a0f0 0x7fe058110980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:54.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:54.173+0000 7fe05ca00640 1 --2- 172.21.15.45:0/3723109038 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe058109730 0x7fe058109bb0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:54.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:54.173+0000 7fe05ca00640 1 --2- 172.21.15.45:0/3723109038 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe058108540 0x7fe058108960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:54.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:54.173+0000 7fe05ca00640 1 -- 172.21.15.45:0/3723109038 >> 172.21.15.45:0/3723109038 conn(0x7fe0580774d0 msgr2=0x7fe058079910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:05:54.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:54.173+0000 7fe05ca00640 1 -- 172.21.15.45:0/3723109038 shutdown_connections 2024-04-16T17:05:54.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:54.173+0000 7fe05ca00640 1 -- 172.21.15.45:0/3723109038 wait complete. 2024-04-16T17:05:54.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:54.174+0000 7fe05ca00640 1 Processor -- start 2024-04-16T17:05:54.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:54.174+0000 7fe05ca00640 1 -- start start 2024-04-16T17:05:54.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:54.174+0000 7fe05ca00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe058108540 0x7fe05819e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:54.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:54.175+0000 7fe05ca00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe058109730 0x7fe05819ed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:54.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:54.175+0000 7fe05ca00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe05810a0f0 0x7fe0581a5d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:54.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:54.175+0000 7fe05ca00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fe058114220 con 0x7fe05810a0f0 2024-04-16T17:05:54.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:54.175+0000 7fe05ca00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fe0581140a0 con 0x7fe058108540 2024-04-16T17:05:54.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:54.175+0000 7fe05ca00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fe0581143a0 con 0x7fe058109730 2024-04-16T17:05:54.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:54.175+0000 7fe055e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe058109730 0x7fe05819ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:54.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:54.175+0000 7fe055e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe058109730 0x7fe05819ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:59988/0 (socket says 172.21.15.45:59988) 2024-04-16T17:05:54.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:54.175+0000 7fe055e00640 1 -- 172.21.15.45:0/3536380936 learned_addr learned my addr 172.21.15.45:0/3536380936 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:05:54.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:54.175+0000 7fe056800640 1 --2- 172.21.15.45:0/3536380936 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe058108540 0x7fe05819e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:54.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:54.175+0000 7fe057200640 1 --2- 172.21.15.45:0/3536380936 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe05810a0f0 0x7fe0581a5d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:54.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:54.175+0000 7fe055e00640 1 -- 172.21.15.45:0/3536380936 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe058108540 msgr2=0x7fe05819e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:54.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:54.175+0000 7fe055e00640 1 --2- 172.21.15.45:0/3536380936 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe058108540 0x7fe05819e7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:54.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:54.175+0000 7fe055e00640 1 -- 172.21.15.45:0/3536380936 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe05810a0f0 msgr2=0x7fe0581a5d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:54.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:54.175+0000 7fe055e00640 1 --2- 172.21.15.45:0/3536380936 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe05810a0f0 0x7fe0581a5d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:54.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:54.175+0000 7fe055e00640 1 -- 172.21.15.45:0/3536380936 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0581a63e0 con 0x7fe058109730 2024-04-16T17:05:54.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:54.176+0000 7fe055e00640 1 --2- 172.21.15.45:0/3536380936 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe058109730 0x7fe05819ed00 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fe044000c00 tx=0x7fe04402c650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:05:54.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:54.176+0000 7fe04f400640 1 -- 172.21.15.45:0/3536380936 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe044043020 con 0x7fe058109730 2024-04-16T17:05:54.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:54.176+0000 7fe04f400640 1 -- 172.21.15.45:0/3536380936 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe0440054e0 con 0x7fe058109730 2024-04-16T17:05:54.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:54.176+0000 7fe05ca00640 1 -- 172.21.15.45:0/3536380936 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0581a6670 con 0x7fe058109730 2024-04-16T17:05:54.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:54.176+0000 7fe04f400640 1 -- 172.21.15.45:0/3536380936 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe044003c40 con 0x7fe058109730 2024-04-16T17:05:54.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:54.176+0000 7fe05ca00640 1 -- 172.21.15.45:0/3536380936 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0581a6c10 con 0x7fe058109730 2024-04-16T17:05:54.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:54.177+0000 7fe05ca00640 1 -- 172.21.15.45:0/3536380936 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe05810f800 con 0x7fe058109730 2024-04-16T17:05:54.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:54.179+0000 7fe04f400640 1 -- 172.21.15.45:0/3536380936 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fe044046020 con 0x7fe058109730 2024-04-16T17:05:54.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:54.181+0000 7fe04f400640 1 --2- 172.21.15.45:0/3536380936 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fe03c0a3c30 0x7fe03c0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:54.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:54.181+0000 7fe04f400640 1 -- 172.21.15.45:0/3536380936 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fe0440f2bc0 con 0x7fe058109730 2024-04-16T17:05:54.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:54.181+0000 7fe056800640 1 --2- 172.21.15.45:0/3536380936 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fe03c0a3c30 0x7fe03c0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:54.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:54.182+0000 7fe056800640 1 --2- 172.21.15.45:0/3536380936 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fe03c0a3c30 0x7fe03c0a60f0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fe0480095f0 tx=0x7fe048005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:05:54.184 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:54.184+0000 7fe04f400640 1 -- 172.21.15.45:0/3536380936 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe04404d020 con 0x7fe058109730 2024-04-16T17:05:54.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:54.390+0000 7fe05ca00640 1 -- 172.21.15.45:0/3536380936 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fe05802d870 con 0x7fe058109730 2024-04-16T17:05:54.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:54.391+0000 7fe04f400640 1 -- 172.21.15.45:0/3536380936 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v94) ==== 104+0+11 (secure 0 0 0) 0x7fe0440bf1b0 con 0x7fe058109730 2024-04-16T17:05:54.391 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 32 2024-04-16T17:05:54.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:54.393+0000 7fe05ca00640 1 -- 172.21.15.45:0/3536380936 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fe03c0a3c30 msgr2=0x7fe03c0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:54.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:54.393+0000 7fe05ca00640 1 --2- 172.21.15.45:0/3536380936 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fe03c0a3c30 0x7fe03c0a60f0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fe0480095f0 tx=0x7fe048005f90 comp rx=0 tx=0).stop 2024-04-16T17:05:54.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:54.393+0000 7fe05ca00640 1 -- 172.21.15.45:0/3536380936 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe058109730 msgr2=0x7fe05819ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:54.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:54.393+0000 7fe05ca00640 1 --2- 172.21.15.45:0/3536380936 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe058109730 0x7fe05819ed00 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fe044000c00 tx=0x7fe04402c650 comp rx=0 tx=0).stop 2024-04-16T17:05:54.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:54.393+0000 7fe05ca00640 1 -- 172.21.15.45:0/3536380936 shutdown_connections 2024-04-16T17:05:54.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:54.393+0000 7fe05ca00640 1 --2- 172.21.15.45:0/3536380936 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fe03c0a3c30 0x7fe03c0a60f0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:54.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:54.393+0000 7fe05ca00640 1 --2- 172.21.15.45:0/3536380936 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe058109730 0x7fe05819ed00 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:54.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:54.393+0000 7fe05ca00640 1 --2- 172.21.15.45:0/3536380936 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe05810a0f0 0x7fe0581a5d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:54.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:54.394+0000 7fe05ca00640 1 --2- 172.21.15.45:0/3536380936 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe058108540 0x7fe05819e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:54.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:54.394+0000 7fe05ca00640 1 -- 172.21.15.45:0/3536380936 >> 172.21.15.45:0/3536380936 conn(0x7fe0580774d0 msgr2=0x7fe05810c9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:05:54.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:54.394+0000 7fe05ca00640 1 -- 172.21.15.45:0/3536380936 shutdown_connections 2024-04-16T17:05:54.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:54.394+0000 7fe05ca00640 1 -- 172.21.15.45:0/3536380936 wait complete. 2024-04-16T17:05:54.413 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-16T17:05:54.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:54.632+0000 7f2866600640 1 -- 172.21.15.45:0/750409913 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f28601030f0 msgr2=0x7f2860105560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:54.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:54.632+0000 7f2866600640 1 --2- 172.21.15.45:0/750409913 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f28601030f0 0x7f2860105560 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f2848009860 tx=0x7f284802bb70 comp rx=0 tx=0).stop 2024-04-16T17:05:54.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:54.632+0000 7f2866600640 1 -- 172.21.15.45:0/750409913 shutdown_connections 2024-04-16T17:05:54.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:54.632+0000 7f2866600640 1 --2- 172.21.15.45:0/750409913 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f28601030f0 0x7f2860105560 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:54.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:54.632+0000 7f2866600640 1 --2- 172.21.15.45:0/750409913 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2860105aa0 0x7f2860105f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:54.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:54.632+0000 7f2866600640 1 --2- 172.21.15.45:0/750409913 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2860100780 0x7f2860102bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:54.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:54.632+0000 7f2866600640 1 -- 172.21.15.45:0/750409913 >> 172.21.15.45:0/750409913 conn(0x7f28600fc3a0 msgr2=0x7f28600fe7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:05:54.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:54.633+0000 7f2866600640 1 -- 172.21.15.45:0/750409913 shutdown_connections 2024-04-16T17:05:54.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:54.633+0000 7f2866600640 1 -- 172.21.15.45:0/750409913 wait complete. 2024-04-16T17:05:54.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:54.634+0000 7f2866600640 1 Processor -- start 2024-04-16T17:05:54.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:54.634+0000 7f2866600640 1 -- start start 2024-04-16T17:05:54.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:54.634+0000 7f2866600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2860100780 0x7f286019e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:54.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:54.634+0000 7f2866600640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f28601030f0 0x7f286019ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:54.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:54.634+0000 7f2866600640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2860105aa0 0x7f28601a5da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:54.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:54.634+0000 7f2866600640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f28601141b0 con 0x7f2860100780 2024-04-16T17:05:54.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:54.634+0000 7f2866600640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f2860114030 con 0x7f28601030f0 2024-04-16T17:05:54.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:54.634+0000 7f2866600640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f2860114330 con 0x7f2860105aa0 2024-04-16T17:05:54.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:54.634+0000 7f2865200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2860100780 0x7f286019e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:54.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:54.634+0000 7f2857e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f28601030f0 0x7f286019ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:54.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:54.635+0000 7f2865200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2860100780 0x7f286019e840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39398/0 (socket says 172.21.15.45:39398) 2024-04-16T17:05:54.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:54.635+0000 7f2865200640 1 -- 172.21.15.45:0/3391953023 learned_addr learned my addr 172.21.15.45:0/3391953023 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:05:54.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:54.635+0000 7f2865c00640 1 --2- 172.21.15.45:0/3391953023 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2860105aa0 0x7f28601a5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:54.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:54.635+0000 7f2857e00640 1 -- 172.21.15.45:0/3391953023 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2860105aa0 msgr2=0x7f28601a5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:54.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:54.635+0000 7f2857e00640 1 --2- 172.21.15.45:0/3391953023 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2860105aa0 0x7f28601a5da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:54.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:54.635+0000 7f2857e00640 1 -- 172.21.15.45:0/3391953023 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2860100780 msgr2=0x7f286019e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:54.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:54.635+0000 7f2857e00640 1 --2- 172.21.15.45:0/3391953023 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2860100780 0x7f286019e840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:54.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:54.635+0000 7f2857e00640 1 -- 172.21.15.45:0/3391953023 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28601a6410 con 0x7f28601030f0 2024-04-16T17:05:54.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:54.635+0000 7f2865200640 1 --2- 172.21.15.45:0/3391953023 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2860100780 0x7f286019e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:05:54.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:54.635+0000 7f2857e00640 1 --2- 172.21.15.45:0/3391953023 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f28601030f0 0x7f286019ed80 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f284802c0b0 tx=0x7f2848032430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:05:54.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:54.636+0000 7f285e000640 1 -- 172.21.15.45:0/3391953023 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2848041020 con 0x7f28601030f0 2024-04-16T17:05:54.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:54.636+0000 7f285e000640 1 -- 172.21.15.45:0/3391953023 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f284802c9f0 con 0x7f28601030f0 2024-04-16T17:05:54.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:54.636+0000 7f285e000640 1 -- 172.21.15.45:0/3391953023 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2848039040 con 0x7f28601030f0 2024-04-16T17:05:54.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:54.636+0000 7f2866600640 1 -- 172.21.15.45:0/3391953023 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28601a66a0 con 0x7f28601030f0 2024-04-16T17:05:54.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:54.636+0000 7f2866600640 1 -- 172.21.15.45:0/3391953023 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28601a6b60 con 0x7f28601030f0 2024-04-16T17:05:54.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:54.637+0000 7f2866600640 1 -- 172.21.15.45:0/3391953023 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28600fde90 con 0x7f28601030f0 2024-04-16T17:05:54.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:54.639+0000 7f285e000640 1 -- 172.21.15.45:0/3391953023 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f2848044020 con 0x7f28601030f0 2024-04-16T17:05:54.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:54.640+0000 7f285e000640 1 --2- 172.21.15.45:0/3391953023 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f28440a3bb0 0x7f28440a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:54.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:54.640+0000 7f285e000640 1 -- 172.21.15.45:0/3391953023 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f28480f1fe0 con 0x7f28601030f0 2024-04-16T17:05:54.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:54.641+0000 7f2865200640 1 --2- 172.21.15.45:0/3391953023 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f28440a3bb0 0x7f28440a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:54.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:54.641+0000 7f2865200640 1 --2- 172.21.15.45:0/3391953023 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f28440a3bb0 0x7f28440a6070 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f28500058c0 tx=0x7f2850009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:05:54.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:54.644+0000 7f285e000640 1 -- 172.21.15.45:0/3391953023 <== mon.2 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f284804b020 con 0x7f28601030f0 2024-04-16T17:05:54.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:05:54 smithi045 ceph-mon[31676]: from='client.? 172.21.15.45:0/1227739277' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T17:05:54.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:05:54 smithi045 ceph-mon[31676]: from='client.? 172.21.15.45:0/46292568' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-16T17:05:54.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:05:54 smithi045 ceph-mon[31676]: from='client.? 172.21.15.45:0/3536380936' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-16T17:05:54.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:05:54 smithi117 ceph-mon[36730]: from='client.? 172.21.15.45:0/1227739277' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T17:05:54.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:05:54 smithi117 ceph-mon[36730]: from='client.? 172.21.15.45:0/46292568' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-16T17:05:54.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:05:54 smithi117 ceph-mon[36730]: from='client.? 172.21.15.45:0/3536380936' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-16T17:05:54.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:54.853+0000 7f2866600640 1 -- 172.21.15.45:0/3391953023 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f2860062c00 con 0x7f28601030f0 2024-04-16T17:05:54.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:54.855+0000 7f285e000640 1 -- 172.21.15.45:0/3391953023 <== mon.2 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v94) ==== 100+0+11 (secure 0 0 0) 0x7f2848032e50 con 0x7f28601030f0 2024-04-16T17:05:54.855 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 64 2024-04-16T17:05:54.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:54.857+0000 7f2866600640 1 -- 172.21.15.45:0/3391953023 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f28440a3bb0 msgr2=0x7f28440a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:54.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:54.857+0000 7f2866600640 1 --2- 172.21.15.45:0/3391953023 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f28440a3bb0 0x7f28440a6070 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f28500058c0 tx=0x7f2850009260 comp rx=0 tx=0).stop 2024-04-16T17:05:54.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:54.857+0000 7f2866600640 1 -- 172.21.15.45:0/3391953023 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f28601030f0 msgr2=0x7f286019ed80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:54.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:54.857+0000 7f2866600640 1 --2- 172.21.15.45:0/3391953023 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f28601030f0 0x7f286019ed80 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f284802c0b0 tx=0x7f2848032430 comp rx=0 tx=0).stop 2024-04-16T17:05:54.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:54.857+0000 7f2866600640 1 -- 172.21.15.45:0/3391953023 shutdown_connections 2024-04-16T17:05:54.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:54.857+0000 7f2866600640 1 --2- 172.21.15.45:0/3391953023 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f28440a3bb0 0x7f28440a6070 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:54.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:54.857+0000 7f2866600640 1 --2- 172.21.15.45:0/3391953023 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2860105aa0 0x7f28601a5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:54.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:54.857+0000 7f2866600640 1 --2- 172.21.15.45:0/3391953023 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f28601030f0 0x7f286019ed80 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:54.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:54.857+0000 7f2866600640 1 --2- 172.21.15.45:0/3391953023 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2860100780 0x7f286019e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:54.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:54.857+0000 7f2866600640 1 -- 172.21.15.45:0/3391953023 >> 172.21.15.45:0/3391953023 conn(0x7f28600fc3a0 msgr2=0x7f28601014d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:05:54.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:54.858+0000 7f2866600640 1 -- 172.21.15.45:0/3391953023 shutdown_connections 2024-04-16T17:05:54.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:54.858+0000 7f2866600640 1 -- 172.21.15.45:0/3391953023 wait complete. 2024-04-16T17:05:54.876 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-16T17:05:54.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:05:54 smithi104 ceph-mon[36457]: from='client.? 172.21.15.45:0/1227739277' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T17:05:54.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:05:54 smithi104 ceph-mon[36457]: from='client.? 172.21.15.45:0/46292568' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-16T17:05:54.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:05:54 smithi104 ceph-mon[36457]: from='client.? 172.21.15.45:0/3536380936' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-16T17:05:55.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:55.099+0000 7f1d21200640 1 -- 172.21.15.45:0/1923031671 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1d1c104140 msgr2=0x7f1d1c104560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:55.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:55.099+0000 7f1d21200640 1 --2- 172.21.15.45:0/1923031671 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1d1c104140 0x7f1d1c104560 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f1d0c009900 tx=0x7f1d0c02bc30 comp rx=0 tx=0).stop 2024-04-16T17:05:55.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:55.099+0000 7f1d21200640 1 -- 172.21.15.45:0/1923031671 shutdown_connections 2024-04-16T17:05:55.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:55.099+0000 7f1d21200640 1 --2- 172.21.15.45:0/1923031671 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1d1c105cf0 0x7f1d1c10c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:55.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:55.099+0000 7f1d21200640 1 --2- 172.21.15.45:0/1923031671 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1d1c105330 0x7f1d1c1057b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:55.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:55.099+0000 7f1d21200640 1 --2- 172.21.15.45:0/1923031671 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1d1c104140 0x7f1d1c104560 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:55.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:55.099+0000 7f1d21200640 1 -- 172.21.15.45:0/1923031671 >> 172.21.15.45:0/1923031671 conn(0x7f1d1c0ff6b0 msgr2=0x7f1d1c101af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:05:55.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:55.099+0000 7f1d21200640 1 -- 172.21.15.45:0/1923031671 shutdown_connections 2024-04-16T17:05:55.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:55.099+0000 7f1d21200640 1 -- 172.21.15.45:0/1923031671 wait complete. 2024-04-16T17:05:55.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:55.100+0000 7f1d21200640 1 Processor -- start 2024-04-16T17:05:55.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:55.101+0000 7f1d21200640 1 -- start start 2024-04-16T17:05:55.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:55.101+0000 7f1d21200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1d1c104140 0x7f1d1c19a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:55.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:55.101+0000 7f1d21200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1d1c105330 0x7f1d1c19a960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:55.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:55.101+0000 7f1d21200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1d1c105cf0 0x7f1d1c1a19d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:55.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:55.101+0000 7f1d21200640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f1d1c10fe20 con 0x7f1d1c104140 2024-04-16T17:05:55.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:55.101+0000 7f1d21200640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f1d1c10fca0 con 0x7f1d1c105330 2024-04-16T17:05:55.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:55.101+0000 7f1d21200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f1d1c10ffa0 con 0x7f1d1c105cf0 2024-04-16T17:05:55.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:55.101+0000 7f1d1b400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1d1c104140 0x7f1d1c19a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:55.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:55.101+0000 7f1d1aa00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1d1c105330 0x7f1d1c19a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:55.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:55.101+0000 7f1d1b400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1d1c104140 0x7f1d1c19a420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39414/0 (socket says 172.21.15.45:39414) 2024-04-16T17:05:55.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:55.101+0000 7f1d1b400640 1 -- 172.21.15.45:0/2197151308 learned_addr learned my addr 172.21.15.45:0/2197151308 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:05:55.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:55.101+0000 7f1d1be00640 1 --2- 172.21.15.45:0/2197151308 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1d1c105cf0 0x7f1d1c1a19d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:55.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:55.102+0000 7f1d1b400640 1 -- 172.21.15.45:0/2197151308 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1d1c105cf0 msgr2=0x7f1d1c1a19d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:55.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:55.102+0000 7f1d1b400640 1 --2- 172.21.15.45:0/2197151308 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1d1c105cf0 0x7f1d1c1a19d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:55.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:55.102+0000 7f1d1b400640 1 -- 172.21.15.45:0/2197151308 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1d1c105330 msgr2=0x7f1d1c19a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:55.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:55.102+0000 7f1d1b400640 1 --2- 172.21.15.45:0/2197151308 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1d1c105330 0x7f1d1c19a960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:55.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:55.102+0000 7f1d1b400640 1 -- 172.21.15.45:0/2197151308 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d1c1a2040 con 0x7f1d1c104140 2024-04-16T17:05:55.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:55.102+0000 7f1d1b400640 1 --2- 172.21.15.45:0/2197151308 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1d1c104140 0x7f1d1c19a420 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f1d0c000c00 tx=0x7f1d0c02c650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:05:55.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:55.102+0000 7f1d13e00640 1 -- 172.21.15.45:0/2197151308 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d0c043020 con 0x7f1d1c104140 2024-04-16T17:05:55.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:55.102+0000 7f1d13e00640 1 -- 172.21.15.45:0/2197151308 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1d0c0054e0 con 0x7f1d1c104140 2024-04-16T17:05:55.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:55.102+0000 7f1d21200640 1 -- 172.21.15.45:0/2197151308 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d1c1a22d0 con 0x7f1d1c104140 2024-04-16T17:05:55.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:55.102+0000 7f1d13e00640 1 -- 172.21.15.45:0/2197151308 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d0c003c40 con 0x7f1d1c104140 2024-04-16T17:05:55.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:55.103+0000 7f1d21200640 1 -- 172.21.15.45:0/2197151308 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d1c1a2820 con 0x7f1d1c104140 2024-04-16T17:05:55.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:55.104+0000 7f1d21200640 1 -- 172.21.15.45:0/2197151308 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d1c10b400 con 0x7f1d1c104140 2024-04-16T17:05:55.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:55.106+0000 7f1d13e00640 1 -- 172.21.15.45:0/2197151308 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f1d0c046020 con 0x7f1d1c104140 2024-04-16T17:05:55.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:55.107+0000 7f1d13e00640 1 --2- 172.21.15.45:0/2197151308 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f1d000a3bb0 0x7f1d000a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:55.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:55.108+0000 7f1d13e00640 1 -- 172.21.15.45:0/2197151308 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f1d0c0f27c0 con 0x7f1d1c104140 2024-04-16T17:05:55.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:55.108+0000 7f1d1aa00640 1 --2- 172.21.15.45:0/2197151308 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f1d000a3bb0 0x7f1d000a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:55.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:55.108+0000 7f1d1aa00640 1 --2- 172.21.15.45:0/2197151308 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f1d000a3bb0 0x7f1d000a6070 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f1d1c19b930 tx=0x7f1d04005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:05:55.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:55.112+0000 7f1d13e00640 1 -- 172.21.15.45:0/2197151308 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1d0c04d020 con 0x7f1d1c104140 2024-04-16T17:05:55.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:55.317+0000 7f1d21200640 1 -- 172.21.15.45:0/2197151308 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f1d1c02d840 con 0x7f1d1c104140 2024-04-16T17:05:55.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:55.318+0000 7f1d13e00640 1 -- 172.21.15.45:0/2197151308 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v94) ==== 103+0+11 (secure 0 0 0) 0x7f1d0c0bedb0 con 0x7f1d1c104140 2024-04-16T17:05:55.319 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 32 2024-04-16T17:05:55.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:55.321+0000 7f1d21200640 1 -- 172.21.15.45:0/2197151308 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f1d000a3bb0 msgr2=0x7f1d000a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:55.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:55.321+0000 7f1d21200640 1 --2- 172.21.15.45:0/2197151308 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f1d000a3bb0 0x7f1d000a6070 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f1d1c19b930 tx=0x7f1d04005f90 comp rx=0 tx=0).stop 2024-04-16T17:05:55.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:55.321+0000 7f1d21200640 1 -- 172.21.15.45:0/2197151308 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1d1c104140 msgr2=0x7f1d1c19a420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:55.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:55.321+0000 7f1d21200640 1 --2- 172.21.15.45:0/2197151308 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1d1c104140 0x7f1d1c19a420 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f1d0c000c00 tx=0x7f1d0c02c650 comp rx=0 tx=0).stop 2024-04-16T17:05:55.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:55.321+0000 7f1d21200640 1 -- 172.21.15.45:0/2197151308 shutdown_connections 2024-04-16T17:05:55.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:55.321+0000 7f1d21200640 1 --2- 172.21.15.45:0/2197151308 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f1d000a3bb0 0x7f1d000a6070 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:55.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:55.321+0000 7f1d21200640 1 --2- 172.21.15.45:0/2197151308 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1d1c105cf0 0x7f1d1c1a19d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:55.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:55.321+0000 7f1d21200640 1 --2- 172.21.15.45:0/2197151308 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1d1c105330 0x7f1d1c19a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:55.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:55.321+0000 7f1d21200640 1 --2- 172.21.15.45:0/2197151308 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1d1c104140 0x7f1d1c19a420 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:55.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:55.321+0000 7f1d21200640 1 -- 172.21.15.45:0/2197151308 >> 172.21.15.45:0/2197151308 conn(0x7f1d1c0ff6b0 msgr2=0x7f1d1c1085d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:05:55.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:55.321+0000 7f1d21200640 1 -- 172.21.15.45:0/2197151308 shutdown_connections 2024-04-16T17:05:55.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:55.321+0000 7f1d21200640 1 -- 172.21.15.45:0/2197151308 wait complete. 2024-04-16T17:05:55.343 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-16T17:05:55.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:55.555+0000 7fe512a00640 1 -- 172.21.15.45:0/2558916644 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe50c108660 msgr2=0x7fe50c108a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:55.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:55.555+0000 7fe512a00640 1 --2- 172.21.15.45:0/2558916644 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe50c108660 0x7fe50c108a80 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fe4fc009900 tx=0x7fe4fc02bc40 comp rx=0 tx=0).stop 2024-04-16T17:05:55.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:55.556+0000 7fe512a00640 1 -- 172.21.15.45:0/2558916644 shutdown_connections 2024-04-16T17:05:55.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:55.556+0000 7fe512a00640 1 --2- 172.21.15.45:0/2558916644 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe50c10a210 0x7fe50c110aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:55.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:55.556+0000 7fe512a00640 1 --2- 172.21.15.45:0/2558916644 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe50c109850 0x7fe50c109cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:55.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:55.556+0000 7fe512a00640 1 --2- 172.21.15.45:0/2558916644 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe50c108660 0x7fe50c108a80 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:55.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:55.556+0000 7fe512a00640 1 -- 172.21.15.45:0/2558916644 >> 172.21.15.45:0/2558916644 conn(0x7fe50c077620 msgr2=0x7fe50c079a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:05:55.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:55.556+0000 7fe512a00640 1 -- 172.21.15.45:0/2558916644 shutdown_connections 2024-04-16T17:05:55.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:55.556+0000 7fe512a00640 1 -- 172.21.15.45:0/2558916644 wait complete. 2024-04-16T17:05:55.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:55.557+0000 7fe512a00640 1 Processor -- start 2024-04-16T17:05:55.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:55.557+0000 7fe512a00640 1 -- start start 2024-04-16T17:05:55.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:55.557+0000 7fe512a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe50c108660 0x7fe50c19e970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:55.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:55.557+0000 7fe512a00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe50c109850 0x7fe50c19eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:55.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:55.557+0000 7fe512a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe50c10a210 0x7fe50c1a5f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:55.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:55.557+0000 7fe512a00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fe50c114330 con 0x7fe50c10a210 2024-04-16T17:05:55.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:55.557+0000 7fe512a00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fe50c1141b0 con 0x7fe50c109850 2024-04-16T17:05:55.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:55.557+0000 7fe512a00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fe50c1144b0 con 0x7fe50c108660 2024-04-16T17:05:55.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:55.558+0000 7fe510a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe50c10a210 0x7fe50c1a5f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:55.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:55.558+0000 7fe50b400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe50c109850 0x7fe50c19eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:55.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:55.558+0000 7fe510a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe50c10a210 0x7fe50c1a5f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39438/0 (socket says 172.21.15.45:39438) 2024-04-16T17:05:55.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:55.558+0000 7fe510a00640 1 -- 172.21.15.45:0/271366420 learned_addr learned my addr 172.21.15.45:0/271366420 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:05:55.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:55.558+0000 7fe50be00640 1 --2- 172.21.15.45:0/271366420 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe50c108660 0x7fe50c19e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:55.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:55.558+0000 7fe510a00640 1 -- 172.21.15.45:0/271366420 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe50c108660 msgr2=0x7fe50c19e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:55.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:55.558+0000 7fe510a00640 1 --2- 172.21.15.45:0/271366420 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe50c108660 0x7fe50c19e970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:55.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:55.558+0000 7fe510a00640 1 -- 172.21.15.45:0/271366420 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe50c109850 msgr2=0x7fe50c19eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:55.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:55.558+0000 7fe510a00640 1 --2- 172.21.15.45:0/271366420 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe50c109850 0x7fe50c19eeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:55.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:55.558+0000 7fe510a00640 1 -- 172.21.15.45:0/271366420 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe50c1a6590 con 0x7fe50c10a210 2024-04-16T17:05:55.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:55.558+0000 7fe50b400640 1 --2- 172.21.15.45:0/271366420 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe50c109850 0x7fe50c19eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:05:55.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:55.559+0000 7fe510a00640 1 --2- 172.21.15.45:0/271366420 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe50c10a210 0x7fe50c1a5f20 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fe50400eb10 tx=0x7fe50400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:05:55.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:55.559+0000 7fe508c00640 1 -- 172.21.15.45:0/271366420 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe504004b20 con 0x7fe50c10a210 2024-04-16T17:05:55.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:55.559+0000 7fe508c00640 1 -- 172.21.15.45:0/271366420 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe504004cc0 con 0x7fe50c10a210 2024-04-16T17:05:55.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:55.559+0000 7fe512a00640 1 -- 172.21.15.45:0/271366420 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe50c1a6880 con 0x7fe50c10a210 2024-04-16T17:05:55.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:55.559+0000 7fe508c00640 1 -- 172.21.15.45:0/271366420 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe504010760 con 0x7fe50c10a210 2024-04-16T17:05:55.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:55.559+0000 7fe512a00640 1 -- 172.21.15.45:0/271366420 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe50c1a6dd0 con 0x7fe50c10a210 2024-04-16T17:05:55.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:55.560+0000 7fe512a00640 1 -- 172.21.15.45:0/271366420 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe50c10f980 con 0x7fe50c10a210 2024-04-16T17:05:55.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:55.562+0000 7fe508c00640 1 -- 172.21.15.45:0/271366420 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fe504005020 con 0x7fe50c10a210 2024-04-16T17:05:55.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:55.563+0000 7fe508c00640 1 --2- 172.21.15.45:0/271366420 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fe4f00a3bb0 0x7fe4f00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:55.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:55.564+0000 7fe508c00640 1 -- 172.21.15.45:0/271366420 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fe5040d0e80 con 0x7fe50c10a210 2024-04-16T17:05:55.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:55.564+0000 7fe50be00640 1 --2- 172.21.15.45:0/271366420 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fe4f00a3bb0 0x7fe4f00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:55.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:55.564+0000 7fe50be00640 1 --2- 172.21.15.45:0/271366420 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fe4f00a3bb0 0x7fe4f00a6070 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fe4fc02c180 tx=0x7fe4fc02cf90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:05:55.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:55.568+0000 7fe508c00640 1 -- 172.21.15.45:0/271366420 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe50409d470 con 0x7fe50c10a210 2024-04-16T17:05:55.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:05:55 smithi045 ceph-mon[31676]: pgmap v401: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T17:05:55.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:05:55 smithi045 ceph-mon[31676]: from='client.? 172.21.15.45:0/3391953023' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-16T17:05:55.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:05:55 smithi045 ceph-mon[31676]: from='client.? 172.21.15.45:0/2197151308' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-16T17:05:55.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:05:55 smithi117 ceph-mon[36730]: pgmap v401: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T17:05:55.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:05:55 smithi117 ceph-mon[36730]: from='client.? 172.21.15.45:0/3391953023' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-16T17:05:55.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:05:55 smithi117 ceph-mon[36730]: from='client.? 172.21.15.45:0/2197151308' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-16T17:05:55.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:55.816+0000 7fe512a00640 1 -- 172.21.15.45:0/271366420 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe50c062c00 con 0x7fe50c10a210 2024-04-16T17:05:55.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:55.817+0000 7fe508c00640 1 -- 172.21.15.45:0/271366420 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11887 (secure 0 0 0) 0x7fe50409cd70 con 0x7fe50c10a210 2024-04-16T17:05:55.818 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-16T17:05:55.818 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/1748842443","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":1748842443},{"type":"v1","addr":"172.21.15.45:6835","nonce":1748842443}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6839/1308627051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":1308627051},{"type":"v1","addr":"172.21.15.45:6839","nonce":1308627051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/1768242489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1768242489},{"type":"v1","addr":"172.21.15.117:6833","nonce":1768242489}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6833/3558902585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":3558902585},{"type":"v1","addr":"172.21.15.104:6833","nonce":3558902585}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24478,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/1308142903","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":1308142903},{"type":"v1","addr":"172.21.15.45:6837","nonce":1308142903}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24491,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6835/3430679937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":3430679937},{"type":"v1","addr":"172.21.15.104:6835","nonce":3430679937}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-16T17:03:24.663611+0000","modified":"2024-04-16T17:03:53.381209+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24581,"mds_1":24569,"mds_2":24532},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24499":{"gid":24499,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/2224123841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2224123841},{"type":"v1","addr":"172.21.15.117:6835","nonce":2224123841}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6837/2471356815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":2471356815},{"type":"v1","addr":"172.21.15.117:6837","nonce":2471356815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24524":{"gid":24524,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6837/2647756509","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":2647756509},{"type":"v1","addr":"172.21.15.104:6837","nonce":2647756509}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24532":{"gid":24532,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":13,"addr":"172.21.15.117:6839/2378675949","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":2378675949},{"type":"v1","addr":"172.21.15.117:6839","nonce":2378675949}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24569":{"gid":24569,"name":"a","rank":1,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.45:6841/3755310822","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6840","nonce":3755310822},{"type":"v1","addr":"172.21.15.45:6841","nonce":3755310822}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24581":{"gid":24581,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.104:6839/3660070592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6838","nonce":3660070592},{"type":"v1","addr":"172.21.15.104:6839","nonce":3660070592}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24581,"qdb_cluster":[24569,24532,24581]},"id":1}]} 2024-04-16T17:05:55.819 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 24 2024-04-16T17:05:55.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:55.820+0000 7fe512a00640 1 -- 172.21.15.45:0/271366420 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fe4f00a3bb0 msgr2=0x7fe4f00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:55.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:55.820+0000 7fe512a00640 1 --2- 172.21.15.45:0/271366420 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fe4f00a3bb0 0x7fe4f00a6070 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fe4fc02c180 tx=0x7fe4fc02cf90 comp rx=0 tx=0).stop 2024-04-16T17:05:55.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:55.820+0000 7fe512a00640 1 -- 172.21.15.45:0/271366420 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe50c10a210 msgr2=0x7fe50c1a5f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:55.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:55.820+0000 7fe512a00640 1 --2- 172.21.15.45:0/271366420 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe50c10a210 0x7fe50c1a5f20 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fe50400eb10 tx=0x7fe50400c2d0 comp rx=0 tx=0).stop 2024-04-16T17:05:55.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:55.820+0000 7fe512a00640 1 -- 172.21.15.45:0/271366420 shutdown_connections 2024-04-16T17:05:55.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:55.820+0000 7fe512a00640 1 --2- 172.21.15.45:0/271366420 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fe4f00a3bb0 0x7fe4f00a6070 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:55.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:55.820+0000 7fe512a00640 1 --2- 172.21.15.45:0/271366420 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe50c10a210 0x7fe50c1a5f20 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:55.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:55.820+0000 7fe512a00640 1 --2- 172.21.15.45:0/271366420 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe50c109850 0x7fe50c19eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:55.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:55.820+0000 7fe512a00640 1 --2- 172.21.15.45:0/271366420 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe50c108660 0x7fe50c19e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:55.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:55.820+0000 7fe512a00640 1 -- 172.21.15.45:0/271366420 >> 172.21.15.45:0/271366420 conn(0x7fe50c077620 msgr2=0x7fe50c10cb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:05:55.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:55.820+0000 7fe512a00640 1 -- 172.21.15.45:0/271366420 shutdown_connections 2024-04-16T17:05:55.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:55.820+0000 7fe512a00640 1 -- 172.21.15.45:0/271366420 wait complete. 2024-04-16T17:05:55.841 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-16T17:05:55.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:05:55 smithi104 ceph-mon[36457]: pgmap v401: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T17:05:55.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:05:55 smithi104 ceph-mon[36457]: from='client.? 172.21.15.45:0/3391953023' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-16T17:05:55.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:05:55 smithi104 ceph-mon[36457]: from='client.? 172.21.15.45:0/2197151308' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-16T17:05:56.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:56.049+0000 7f60bbe00640 1 -- 172.21.15.45:0/413642531 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f60bc109670 msgr2=0x7f60bc109af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:56.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:56.049+0000 7f60bbe00640 1 --2- 172.21.15.45:0/413642531 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f60bc109670 0x7f60bc109af0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f60a4009900 tx=0x7f60a402bc30 comp rx=0 tx=0).stop 2024-04-16T17:05:56.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:56.050+0000 7f60bbe00640 1 -- 172.21.15.45:0/413642531 shutdown_connections 2024-04-16T17:05:56.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:56.050+0000 7f60bbe00640 1 --2- 172.21.15.45:0/413642531 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f60bc10a030 0x7f60bc1108c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:56.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:56.050+0000 7f60bbe00640 1 --2- 172.21.15.45:0/413642531 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f60bc109670 0x7f60bc109af0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:56.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:56.050+0000 7f60bbe00640 1 --2- 172.21.15.45:0/413642531 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f60bc108480 0x7f60bc1088a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:56.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:56.050+0000 7f60bbe00640 1 -- 172.21.15.45:0/413642531 >> 172.21.15.45:0/413642531 conn(0x7f60bc077440 msgr2=0x7f60bc079860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:05:56.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:56.050+0000 7f60bbe00640 1 -- 172.21.15.45:0/413642531 shutdown_connections 2024-04-16T17:05:56.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:56.050+0000 7f60bbe00640 1 -- 172.21.15.45:0/413642531 wait complete. 2024-04-16T17:05:56.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:56.051+0000 7f60bbe00640 1 Processor -- start 2024-04-16T17:05:56.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:56.051+0000 7f60bbe00640 1 -- start start 2024-04-16T17:05:56.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:56.051+0000 7f60bbe00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f60bc108480 0x7f60bc19e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:56.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:56.051+0000 7f60bbe00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f60bc109670 0x7f60bc19ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:56.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:56.051+0000 7f60bbe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f60bc10a030 0x7f60bc1a5d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:56.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:56.051+0000 7f60bbe00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f60bc114160 con 0x7f60bc10a030 2024-04-16T17:05:56.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:56.051+0000 7f60bbe00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f60bc113fe0 con 0x7f60bc108480 2024-04-16T17:05:56.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:56.051+0000 7f60bbe00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f60bc1142e0 con 0x7f60bc109670 2024-04-16T17:05:56.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:56.051+0000 7f60bb400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f60bc10a030 0x7f60bc1a5d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:56.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:56.051+0000 7f60bb400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f60bc10a030 0x7f60bc1a5d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39454/0 (socket says 172.21.15.45:39454) 2024-04-16T17:05:56.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:56.051+0000 7f60bb400640 1 -- 172.21.15.45:0/807866360 learned_addr learned my addr 172.21.15.45:0/807866360 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:05:56.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:56.051+0000 7f60b3e00640 1 --2- 172.21.15.45:0/807866360 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f60bc109670 0x7f60bc19ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:56.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:56.052+0000 7f60baa00640 1 --2- 172.21.15.45:0/807866360 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f60bc108480 0x7f60bc19e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:56.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:56.052+0000 7f60bb400640 1 -- 172.21.15.45:0/807866360 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f60bc109670 msgr2=0x7f60bc19ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:56.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:56.052+0000 7f60bb400640 1 --2- 172.21.15.45:0/807866360 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f60bc109670 0x7f60bc19ecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:56.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:56.052+0000 7f60bb400640 1 -- 172.21.15.45:0/807866360 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f60bc108480 msgr2=0x7f60bc19e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:56.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:56.052+0000 7f60bb400640 1 --2- 172.21.15.45:0/807866360 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f60bc108480 0x7f60bc19e790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:56.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:56.052+0000 7f60bb400640 1 -- 172.21.15.45:0/807866360 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60bc1a63b0 con 0x7f60bc10a030 2024-04-16T17:05:56.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:56.052+0000 7f60bb400640 1 --2- 172.21.15.45:0/807866360 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f60bc10a030 0x7f60bc1a5d40 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f60b400efd0 tx=0x7f60b400c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:05:56.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:56.052+0000 7f60c0a00640 1 -- 172.21.15.45:0/807866360 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60b4003b70 con 0x7f60bc10a030 2024-04-16T17:05:56.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:56.052+0000 7f60c0a00640 1 -- 172.21.15.45:0/807866360 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f60b4003d10 con 0x7f60bc10a030 2024-04-16T17:05:56.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:56.052+0000 7f60bbe00640 1 -- 172.21.15.45:0/807866360 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60bc1a66a0 con 0x7f60bc10a030 2024-04-16T17:05:56.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:56.053+0000 7f60c0a00640 1 -- 172.21.15.45:0/807866360 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60b40109f0 con 0x7f60bc10a030 2024-04-16T17:05:56.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:56.053+0000 7f60bbe00640 1 -- 172.21.15.45:0/807866360 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60bc1a6b70 con 0x7f60bc10a030 2024-04-16T17:05:56.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:56.054+0000 7f60bbe00640 1 -- 172.21.15.45:0/807866360 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60bc10f7a0 con 0x7f60bc10a030 2024-04-16T17:05:56.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:56.056+0000 7f60c0a00640 1 -- 172.21.15.45:0/807866360 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f60b4004020 con 0x7f60bc10a030 2024-04-16T17:05:56.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:56.057+0000 7f60c0a00640 1 --2- 172.21.15.45:0/807866360 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f60a00a3bb0 0x7f60a00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:56.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:56.057+0000 7f60c0a00640 1 -- 172.21.15.45:0/807866360 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f60b40d0c40 con 0x7f60bc10a030 2024-04-16T17:05:56.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:56.058+0000 7f60baa00640 1 --2- 172.21.15.45:0/807866360 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f60a00a3bb0 0x7f60a00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:56.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:56.058+0000 7f60baa00640 1 --2- 172.21.15.45:0/807866360 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f60a00a3bb0 0x7f60a00a6070 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f60bc1094d0 tx=0x7f60ac005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:05:56.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:56.062+0000 7f60c0a00640 1 -- 172.21.15.45:0/807866360 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f60b409d230 con 0x7f60bc10a030 2024-04-16T17:05:56.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:56.273+0000 7f60bbe00640 1 -- 172.21.15.45:0/807866360 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f60bc062c00 con 0x7f60bc10a030 2024-04-16T17:05:56.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:56.276+0000 7f60c0a00640 1 -- 172.21.15.45:0/807866360 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) ==== 74+0+24726 (secure 0 0 0) 0x7f60b409cb30 con 0x7f60bc10a030 2024-04-16T17:05:56.277 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-16T17:05:56.277 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":94,"fsid":"b1fecd38-fc11-11ee-bc90-c7b262605968","created":"2024-04-16T16:54:02.385906+0000","modified":"2024-04-16T17:05:22.272335+0000","last_up_change":"2024-04-16T17:02:25.162624+0000","last_in_change":"2024-04-16T17:02:04.903274+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-16T16:58:06.166659+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-16T17:03:22.304198+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-16T17:03:23.616450+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-16T17:03:27.226289+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b655de2c-59f8-4fd4-ac98-0cc222229f56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":2290170881},{"type":"v1","addr":"172.21.15.45:6803","nonce":2290170881}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":2290170881},{"type":"v1","addr":"172.21.15.45:6805","nonce":2290170881}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":2290170881},{"type":"v1","addr":"172.21.15.45:6809","nonce":2290170881}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":2290170881},{"type":"v1","addr":"172.21.15.45:6807","nonce":2290170881}]},"public_addr":"172.21.15.45:6803/2290170881","cluster_addr":"172.21.15.45:6805/2290170881","heartbeat_back_addr":"172.21.15.45:6809/2290170881","heartbeat_front_addr":"172.21.15.45:6807/2290170881","state":["exists","up"]},{"osd":1,"uuid":"ebd450ec-08fd-4c3d-8b24-4e10634ddb7e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6800","nonce":1462942067},{"type":"v1","addr":"172.21.15.104:6801","nonce":1462942067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":1462942067},{"type":"v1","addr":"172.21.15.104:6803","nonce":1462942067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":1462942067},{"type":"v1","addr":"172.21.15.104:6807","nonce":1462942067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":1462942067},{"type":"v1","addr":"172.21.15.104:6805","nonce":1462942067}]},"public_addr":"172.21.15.104:6801/1462942067","cluster_addr":"172.21.15.104:6803/1462942067","heartbeat_back_addr":"172.21.15.104:6807/1462942067","heartbeat_front_addr":"172.21.15.104:6805/1462942067","state":["exists","up"]},{"osd":2,"uuid":"b318a512-4190-42a2-8104-dcc4026ad8ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":2829260735},{"type":"v1","addr":"172.21.15.117:6801","nonce":2829260735}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":2829260735},{"type":"v1","addr":"172.21.15.117:6803","nonce":2829260735}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":2829260735},{"type":"v1","addr":"172.21.15.117:6807","nonce":2829260735}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":2829260735},{"type":"v1","addr":"172.21.15.117:6805","nonce":2829260735}]},"public_addr":"172.21.15.117:6801/2829260735","cluster_addr":"172.21.15.117:6803/2829260735","heartbeat_back_addr":"172.21.15.117:6807/2829260735","heartbeat_front_addr":"172.21.15.117:6805/2829260735","state":["exists","up"]},{"osd":3,"uuid":"ec6490cb-a715-4c12-a54a-225aae26fcb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":2823712137},{"type":"v1","addr":"172.21.15.45:6811","nonce":2823712137}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":2823712137},{"type":"v1","addr":"172.21.15.45:6813","nonce":2823712137}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":2823712137},{"type":"v1","addr":"172.21.15.45:6817","nonce":2823712137}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":2823712137},{"type":"v1","addr":"172.21.15.45:6815","nonce":2823712137}]},"public_addr":"172.21.15.45:6811/2823712137","cluster_addr":"172.21.15.45:6813/2823712137","heartbeat_back_addr":"172.21.15.45:6817/2823712137","heartbeat_front_addr":"172.21.15.45:6815/2823712137","state":["exists","up"]},{"osd":4,"uuid":"5d91d568-0250-4aae-8dc2-a0e079f92062","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":1396278127},{"type":"v1","addr":"172.21.15.104:6809","nonce":1396278127}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":1396278127},{"type":"v1","addr":"172.21.15.104:6811","nonce":1396278127}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":1396278127},{"type":"v1","addr":"172.21.15.104:6815","nonce":1396278127}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":1396278127},{"type":"v1","addr":"172.21.15.104:6813","nonce":1396278127}]},"public_addr":"172.21.15.104:6809/1396278127","cluster_addr":"172.21.15.104:6811/1396278127","heartbeat_back_addr":"172.21.15.104:6815/1396278127","heartbeat_front_addr":"172.21.15.104:6813/1396278127","state":["exists","up"]},{"osd":5,"uuid":"ac510b43-6fb2-4d0f-bc5f-16ac080d9e9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":4145720047},{"type":"v1","addr":"172.21.15.117:6809","nonce":4145720047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":4145720047},{"type":"v1","addr":"172.21.15.117:6811","nonce":4145720047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":4145720047},{"type":"v1","addr":"172.21.15.117:6815","nonce":4145720047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":4145720047},{"type":"v1","addr":"172.21.15.117:6813","nonce":4145720047}]},"public_addr":"172.21.15.117:6809/4145720047","cluster_addr":"172.21.15.117:6811/4145720047","heartbeat_back_addr":"172.21.15.117:6815/4145720047","heartbeat_front_addr":"172.21.15.117:6813/4145720047","state":["exists","up"]},{"osd":6,"uuid":"a41adc18-d88a-419e-b2c2-b089c1f32472","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":3254229176},{"type":"v1","addr":"172.21.15.45:6819","nonce":3254229176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":3254229176},{"type":"v1","addr":"172.21.15.45:6821","nonce":3254229176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":3254229176},{"type":"v1","addr":"172.21.15.45:6825","nonce":3254229176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":3254229176},{"type":"v1","addr":"172.21.15.45:6823","nonce":3254229176}]},"public_addr":"172.21.15.45:6819/3254229176","cluster_addr":"172.21.15.45:6821/3254229176","heartbeat_back_addr":"172.21.15.45:6825/3254229176","heartbeat_front_addr":"172.21.15.45:6823/3254229176","state":["exists","up"]},{"osd":7,"uuid":"b71712a4-7cf6-454d-9fd1-565e3df21c1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":4151077186},{"type":"v1","addr":"172.21.15.104:6817","nonce":4151077186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":4151077186},{"type":"v1","addr":"172.21.15.104:6819","nonce":4151077186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":4151077186},{"type":"v1","addr":"172.21.15.104:6823","nonce":4151077186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":4151077186},{"type":"v1","addr":"172.21.15.104:6821","nonce":4151077186}]},"public_addr":"172.21.15.104:6817/4151077186","cluster_addr":"172.21.15.104:6819/4151077186","heartbeat_back_addr":"172.21.15.104:6823/4151077186","heartbeat_front_addr":"172.21.15.104:6821/4151077186","state":["exists","up"]},{"osd":8,"uuid":"9e572aef-59a8-450e-a92e-df9dff4d57a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":1500286830},{"type":"v1","addr":"172.21.15.117:6817","nonce":1500286830}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":1500286830},{"type":"v1","addr":"172.21.15.117:6819","nonce":1500286830}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":1500286830},{"type":"v1","addr":"172.21.15.117:6823","nonce":1500286830}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":1500286830},{"type":"v1","addr":"172.21.15.117:6821","nonce":1500286830}]},"public_addr":"172.21.15.117:6817/1500286830","cluster_addr":"172.21.15.117:6819/1500286830","heartbeat_back_addr":"172.21.15.117:6823/1500286830","heartbeat_front_addr":"172.21.15.117:6821/1500286830","state":["exists","up"]},{"osd":9,"uuid":"014e9598-c413-4cc4-9bb8-61c6e8f59d30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":3106734074},{"type":"v1","addr":"172.21.15.45:6827","nonce":3106734074}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":3106734074},{"type":"v1","addr":"172.21.15.45:6829","nonce":3106734074}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":3106734074},{"type":"v1","addr":"172.21.15.45:6833","nonce":3106734074}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":3106734074},{"type":"v1","addr":"172.21.15.45:6831","nonce":3106734074}]},"public_addr":"172.21.15.45:6827/3106734074","cluster_addr":"172.21.15.45:6829/3106734074","heartbeat_back_addr":"172.21.15.45:6833/3106734074","heartbeat_front_addr":"172.21.15.45:6831/3106734074","state":["exists","up"]},{"osd":10,"uuid":"2deba364-3e39-47be-8a1f-8256df5b82d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":1404794652},{"type":"v1","addr":"172.21.15.104:6825","nonce":1404794652}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":1404794652},{"type":"v1","addr":"172.21.15.104:6827","nonce":1404794652}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":1404794652},{"type":"v1","addr":"172.21.15.104:6831","nonce":1404794652}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":1404794652},{"type":"v1","addr":"172.21.15.104:6829","nonce":1404794652}]},"public_addr":"172.21.15.104:6825/1404794652","cluster_addr":"172.21.15.104:6827/1404794652","heartbeat_back_addr":"172.21.15.104:6831/1404794652","heartbeat_front_addr":"172.21.15.104:6829/1404794652","state":["exists","up"]},{"osd":11,"uuid":"3fda4fea-59ad-4d1a-9fe6-fcb30e31fd7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":3384003243},{"type":"v1","addr":"172.21.15.117:6825","nonce":3384003243}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":3384003243},{"type":"v1","addr":"172.21.15.117:6827","nonce":3384003243}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":3384003243},{"type":"v1","addr":"172.21.15.117:6831","nonce":3384003243}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":3384003243},{"type":"v1","addr":"172.21.15.117:6829","nonce":3384003243}]},"public_addr":"172.21.15.117:6825/3384003243","cluster_addr":"172.21.15.117:6827/3384003243","heartbeat_back_addr":"172.21.15.117:6831/3384003243","heartbeat_front_addr":"172.21.15.117:6829/3384003243","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:57:14.799608+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:57:39.884668+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:58:03.327057+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:58:37.314531+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:59:03.667650+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:59:28.462823+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T17:00:02.968574+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T17:00:29.559190+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T17:00:55.291007+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T17:01:29.815977+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T17:01:57.015580+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T17:02:23.233151+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.5","mappings":[{"from":9,"to":10}]},{"pgid":"4.7","mappings":[{"from":5,"to":8}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.45:0/4074982598":"2024-04-17T16:55:19.948001+0000","172.21.15.45:0/2443572576":"2024-04-17T16:55:19.948001+0000","172.21.15.45:6801/1983630136":"2024-04-17T16:55:19.948001+0000","172.21.15.45:6800/1983630136":"2024-04-17T16:55:19.948001+0000","172.21.15.45:0/3150465248":"2024-04-17T16:54:43.692466+0000","172.21.15.45:0/556738623":"2024-04-17T16:54:43.692466+0000","172.21.15.45:0/445676297":"2024-04-17T16:55:19.948001+0000","172.21.15.45:6800/2553659622":"2024-04-17T16:54:43.692466+0000","172.21.15.45:6801/2553659622":"2024-04-17T16:54:43.692466+0000","172.21.15.45:0/1184615408":"2024-04-17T16:54:43.692466+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-16T17:05:56.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:56.278+0000 7f60bbe00640 1 -- 172.21.15.45:0/807866360 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f60a00a3bb0 msgr2=0x7f60a00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:56.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:56.278+0000 7f60bbe00640 1 --2- 172.21.15.45:0/807866360 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f60a00a3bb0 0x7f60a00a6070 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f60bc1094d0 tx=0x7f60ac005f90 comp rx=0 tx=0).stop 2024-04-16T17:05:56.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:56.278+0000 7f60bbe00640 1 -- 172.21.15.45:0/807866360 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f60bc10a030 msgr2=0x7f60bc1a5d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:56.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:56.278+0000 7f60bbe00640 1 --2- 172.21.15.45:0/807866360 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f60bc10a030 0x7f60bc1a5d40 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f60b400efd0 tx=0x7f60b400c6a0 comp rx=0 tx=0).stop 2024-04-16T17:05:56.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:56.278+0000 7f60bbe00640 1 -- 172.21.15.45:0/807866360 shutdown_connections 2024-04-16T17:05:56.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:56.278+0000 7f60bbe00640 1 --2- 172.21.15.45:0/807866360 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f60a00a3bb0 0x7f60a00a6070 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:56.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:56.278+0000 7f60bbe00640 1 --2- 172.21.15.45:0/807866360 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f60bc109670 0x7f60bc19ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:56.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:56.278+0000 7f60bbe00640 1 --2- 172.21.15.45:0/807866360 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f60bc10a030 0x7f60bc1a5d40 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:56.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:56.278+0000 7f60bbe00640 1 --2- 172.21.15.45:0/807866360 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f60bc108480 0x7f60bc19e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:56.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:56.278+0000 7f60bbe00640 1 -- 172.21.15.45:0/807866360 >> 172.21.15.45:0/807866360 conn(0x7f60bc077440 msgr2=0x7f60bc10c970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:05:56.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:56.279+0000 7f60bbe00640 1 -- 172.21.15.45:0/807866360 shutdown_connections 2024-04-16T17:05:56.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:56.279+0000 7f60bbe00640 1 -- 172.21.15.45:0/807866360 wait complete. 2024-04-16T17:05:56.299 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-16T17:05:56.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:56.528+0000 7f4ca3400640 1 -- 172.21.15.45:0/1596304750 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4c9c105400 msgr2=0x7f4c9c1077f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:56.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:56.528+0000 7f4ca3400640 1 --2- 172.21.15.45:0/1596304750 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4c9c105400 0x7f4c9c1077f0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f4c88009900 tx=0x7f4c8802bc40 comp rx=0 tx=0).stop 2024-04-16T17:05:56.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:56.528+0000 7f4ca3400640 1 -- 172.21.15.45:0/1596304750 shutdown_connections 2024-04-16T17:05:56.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:56.528+0000 7f4ca3400640 1 --2- 172.21.15.45:0/1596304750 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4c9c107d30 0x7f4c9c10a180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:56.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:56.528+0000 7f4ca3400640 1 --2- 172.21.15.45:0/1596304750 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4c9c105400 0x7f4c9c1077f0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:56.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:56.528+0000 7f4ca3400640 1 --2- 172.21.15.45:0/1596304750 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4c9c102ad0 0x7f4c9c104ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:56.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:56.528+0000 7f4ca3400640 1 -- 172.21.15.45:0/1596304750 >> 172.21.15.45:0/1596304750 conn(0x7f4c9c0fc4e0 msgr2=0x7f4c9c0fe900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:05:56.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:56.528+0000 7f4ca3400640 1 -- 172.21.15.45:0/1596304750 shutdown_connections 2024-04-16T17:05:56.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:56.528+0000 7f4ca3400640 1 -- 172.21.15.45:0/1596304750 wait complete. 2024-04-16T17:05:56.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:56.530+0000 7f4ca3400640 1 Processor -- start 2024-04-16T17:05:56.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:56.530+0000 7f4ca3400640 1 -- start start 2024-04-16T17:05:56.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:56.530+0000 7f4ca3400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4c9c102ad0 0x7f4c9c1983a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:56.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:56.530+0000 7f4ca3400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4c9c105400 0x7f4c9c1988e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:56.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:56.530+0000 7f4ca3400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4c9c107d30 0x7f4c9c19f950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:56.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:56.530+0000 7f4ca3400640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f4c9c10dca0 con 0x7f4c9c102ad0 2024-04-16T17:05:56.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:56.530+0000 7f4ca3400640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f4c9c10db20 con 0x7f4c9c105400 2024-04-16T17:05:56.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:56.530+0000 7f4ca3400640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f4c9c10de20 con 0x7f4c9c107d30 2024-04-16T17:05:56.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:56.531+0000 7f4ca0a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4c9c102ad0 0x7f4c9c1983a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:56.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:56.531+0000 7f4ca0a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4c9c102ad0 0x7f4c9c1983a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39472/0 (socket says 172.21.15.45:39472) 2024-04-16T17:05:56.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:56.531+0000 7f4ca0a00640 1 -- 172.21.15.45:0/3283115974 learned_addr learned my addr 172.21.15.45:0/3283115974 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:05:56.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:56.531+0000 7f4c9be00640 1 --2- 172.21.15.45:0/3283115974 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4c9c105400 0x7f4c9c1988e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:56.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:56.531+0000 7f4ca1400640 1 --2- 172.21.15.45:0/3283115974 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4c9c107d30 0x7f4c9c19f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:56.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:56.531+0000 7f4c9be00640 1 -- 172.21.15.45:0/3283115974 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4c9c107d30 msgr2=0x7f4c9c19f950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:56.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:56.531+0000 7f4c9be00640 1 --2- 172.21.15.45:0/3283115974 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4c9c107d30 0x7f4c9c19f950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:56.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:56.531+0000 7f4c9be00640 1 -- 172.21.15.45:0/3283115974 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4c9c102ad0 msgr2=0x7f4c9c1983a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:56.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:56.531+0000 7f4c9be00640 1 --2- 172.21.15.45:0/3283115974 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4c9c102ad0 0x7f4c9c1983a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:56.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:56.531+0000 7f4c9be00640 1 -- 172.21.15.45:0/3283115974 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c9c19ffc0 con 0x7f4c9c105400 2024-04-16T17:05:56.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:56.531+0000 7f4ca0a00640 1 --2- 172.21.15.45:0/3283115974 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4c9c102ad0 0x7f4c9c1983a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:05:56.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:56.532+0000 7f4c9be00640 1 --2- 172.21.15.45:0/3283115974 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4c9c105400 0x7f4c9c1988e0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f4c8802c180 tx=0x7f4c8802c660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:05:56.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:56.532+0000 7f4c99600640 1 -- 172.21.15.45:0/3283115974 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c88041070 con 0x7f4c9c105400 2024-04-16T17:05:56.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:56.532+0000 7f4c99600640 1 -- 172.21.15.45:0/3283115974 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4c88004450 con 0x7f4c9c105400 2024-04-16T17:05:56.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:56.532+0000 7f4c99600640 1 -- 172.21.15.45:0/3283115974 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c88003e50 con 0x7f4c9c105400 2024-04-16T17:05:56.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:56.532+0000 7f4ca3400640 1 -- 172.21.15.45:0/3283115974 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c9c1a0250 con 0x7f4c9c105400 2024-04-16T17:05:56.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:56.532+0000 7f4ca3400640 1 -- 172.21.15.45:0/3283115974 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c9c1a06c0 con 0x7f4c9c105400 2024-04-16T17:05:56.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:56.533+0000 7f4ca3400640 1 -- 172.21.15.45:0/3283115974 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c9c0fdfd0 con 0x7f4c9c105400 2024-04-16T17:05:56.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:56.535+0000 7f4c99600640 1 -- 172.21.15.45:0/3283115974 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f4c88044070 con 0x7f4c9c105400 2024-04-16T17:05:56.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:56.537+0000 7f4c99600640 1 --2- 172.21.15.45:0/3283115974 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f4c840a3d50 0x7f4c840a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:56.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:56.537+0000 7f4c99600640 1 -- 172.21.15.45:0/3283115974 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f4c880f22a0 con 0x7f4c9c105400 2024-04-16T17:05:56.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:56.537+0000 7f4ca0a00640 1 --2- 172.21.15.45:0/3283115974 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f4c840a3d50 0x7f4c840a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:56.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:56.538+0000 7f4ca0a00640 1 --2- 172.21.15.45:0/3283115974 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f4c840a3d50 0x7f4c840a6210 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f4c90011550 tx=0x7f4c90005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:05:56.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:56.541+0000 7f4c99600640 1 -- 172.21.15.45:0/3283115974 <== mon.2 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4c880be890 con 0x7f4c9c105400 2024-04-16T17:05:56.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:05:56 smithi045 ceph-mon[31676]: from='client.? 172.21.15.45:0/271366420' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:05:56.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:05:56 smithi045 ceph-mon[31676]: from='client.? 172.21.15.45:0/807866360' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T17:05:56.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:56.788+0000 7f4ca3400640 1 -- 172.21.15.45:0/3283115974 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4c9c062c00 con 0x7f4c9c105400 2024-04-16T17:05:56.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:56.789+0000 7f4c99600640 1 -- 172.21.15.45:0/3283115974 <== mon.2 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11887 (secure 0 0 0) 0x7f4c880be190 con 0x7f4c9c105400 2024-04-16T17:05:56.790 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-16T17:05:56.790 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/1748842443","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":1748842443},{"type":"v1","addr":"172.21.15.45:6835","nonce":1748842443}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6839/1308627051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":1308627051},{"type":"v1","addr":"172.21.15.45:6839","nonce":1308627051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/1768242489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1768242489},{"type":"v1","addr":"172.21.15.117:6833","nonce":1768242489}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6833/3558902585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":3558902585},{"type":"v1","addr":"172.21.15.104:6833","nonce":3558902585}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24478,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/1308142903","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":1308142903},{"type":"v1","addr":"172.21.15.45:6837","nonce":1308142903}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24491,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6835/3430679937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":3430679937},{"type":"v1","addr":"172.21.15.104:6835","nonce":3430679937}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-16T17:03:24.663611+0000","modified":"2024-04-16T17:03:53.381209+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24581,"mds_1":24569,"mds_2":24532},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24499":{"gid":24499,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/2224123841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2224123841},{"type":"v1","addr":"172.21.15.117:6835","nonce":2224123841}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6837/2471356815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":2471356815},{"type":"v1","addr":"172.21.15.117:6837","nonce":2471356815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24524":{"gid":24524,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6837/2647756509","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":2647756509},{"type":"v1","addr":"172.21.15.104:6837","nonce":2647756509}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24532":{"gid":24532,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":13,"addr":"172.21.15.117:6839/2378675949","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":2378675949},{"type":"v1","addr":"172.21.15.117:6839","nonce":2378675949}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24569":{"gid":24569,"name":"a","rank":1,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.45:6841/3755310822","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6840","nonce":3755310822},{"type":"v1","addr":"172.21.15.45:6841","nonce":3755310822}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24581":{"gid":24581,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.104:6839/3660070592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6838","nonce":3660070592},{"type":"v1","addr":"172.21.15.104:6839","nonce":3660070592}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24581,"qdb_cluster":[24569,24532,24581]},"id":1}]} 2024-04-16T17:05:56.790 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 24 2024-04-16T17:05:56.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:56.792+0000 7f4ca3400640 1 -- 172.21.15.45:0/3283115974 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f4c840a3d50 msgr2=0x7f4c840a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:56.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:56.792+0000 7f4ca3400640 1 --2- 172.21.15.45:0/3283115974 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f4c840a3d50 0x7f4c840a6210 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f4c90011550 tx=0x7f4c90005f90 comp rx=0 tx=0).stop 2024-04-16T17:05:56.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:56.792+0000 7f4ca3400640 1 -- 172.21.15.45:0/3283115974 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4c9c105400 msgr2=0x7f4c9c1988e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:56.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:56.792+0000 7f4ca3400640 1 --2- 172.21.15.45:0/3283115974 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4c9c105400 0x7f4c9c1988e0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f4c8802c180 tx=0x7f4c8802c660 comp rx=0 tx=0).stop 2024-04-16T17:05:56.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:56.792+0000 7f4ca3400640 1 -- 172.21.15.45:0/3283115974 shutdown_connections 2024-04-16T17:05:56.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:56.792+0000 7f4ca3400640 1 --2- 172.21.15.45:0/3283115974 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f4c840a3d50 0x7f4c840a6210 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:56.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:56.792+0000 7f4ca3400640 1 --2- 172.21.15.45:0/3283115974 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4c9c107d30 0x7f4c9c19f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:56.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:56.792+0000 7f4ca3400640 1 --2- 172.21.15.45:0/3283115974 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4c9c105400 0x7f4c9c1988e0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:56.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:56.792+0000 7f4ca3400640 1 --2- 172.21.15.45:0/3283115974 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4c9c102ad0 0x7f4c9c1983a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:56.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:56.792+0000 7f4ca3400640 1 -- 172.21.15.45:0/3283115974 >> 172.21.15.45:0/3283115974 conn(0x7f4c9c0fc4e0 msgr2=0x7f4c9c103410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:05:56.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:56.792+0000 7f4ca3400640 1 -- 172.21.15.45:0/3283115974 shutdown_connections 2024-04-16T17:05:56.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:56.792+0000 7f4ca3400640 1 -- 172.21.15.45:0/3283115974 wait complete. 2024-04-16T17:05:56.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:05:56 smithi117 ceph-mon[36730]: from='client.? 172.21.15.45:0/271366420' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:05:56.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:05:56 smithi117 ceph-mon[36730]: from='client.? 172.21.15.45:0/807866360' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T17:05:56.812 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 24, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-16T17:03:24.663611+0000', 'modified': '2024-04-16T17:03:53.381209+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24581, 'mds_1': 24569, 'mds_2': 24532}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24499': {'gid': 24499, 'name': 'f', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.117:6835/2224123841', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6834', 'nonce': 2224123841}, {'type': 'v1', 'addr': '172.21.15.117:6835', 'nonce': 2224123841}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24511': {'gid': 24511, 'name': 'c', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.117:6837/2471356815', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6836', 'nonce': 2471356815}, {'type': 'v1', 'addr': '172.21.15.117:6837', 'nonce': 2471356815}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24524': {'gid': 24524, 'name': 'h', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.104:6837/2647756509', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.104:6836', 'nonce': 2647756509}, {'type': 'v1', 'addr': '172.21.15.104:6837', 'nonce': 2647756509}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24532': {'gid': 24532, 'name': 'i', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 13, 'addr': '172.21.15.117:6839/2378675949', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6838', 'nonce': 2378675949}, {'type': 'v1', 'addr': '172.21.15.117:6839', 'nonce': 2378675949}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24569': {'gid': 24569, 'name': 'a', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 10, 'addr': '172.21.15.45:6841/3755310822', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.45:6840', 'nonce': 3755310822}, {'type': 'v1', 'addr': '172.21.15.45:6841', 'nonce': 3755310822}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24581': {'gid': 24581, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.104:6839/3660070592', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.104:6838', 'nonce': 3660070592}, {'type': 'v1', 'addr': '172.21.15.104:6839', 'nonce': 3660070592}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24581, 'qdb_cluster': [24569, 24532, 24581]} 2024-04-16T17:05:56.813 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-04-16T17:05:56.813 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-04-16T17:05:56.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:05:56 smithi104 ceph-mon[36457]: from='client.? 172.21.15.45:0/271366420' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:05:56.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:05:56 smithi104 ceph-mon[36457]: from='client.? 172.21.15.45:0/807866360' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T17:05:57.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.041+0000 7f519ac00640 1 -- 172.21.15.45:0/1032883239 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5194103a20 msgr2=0x7f519410a340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:57.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.041+0000 7f519ac00640 1 --2- 172.21.15.45:0/1032883239 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5194103a20 0x7f519410a340 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f518c009ca0 tx=0x7f518c02c100 comp rx=0 tx=0).stop 2024-04-16T17:05:57.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.041+0000 7f519ac00640 1 -- 172.21.15.45:0/1032883239 shutdown_connections 2024-04-16T17:05:57.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.041+0000 7f519ac00640 1 --2- 172.21.15.45:0/1032883239 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5194103060 0x7f51941034e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:57.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.041+0000 7f519ac00640 1 --2- 172.21.15.45:0/1032883239 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5194103a20 0x7f519410a340 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:57.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.041+0000 7f519ac00640 1 --2- 172.21.15.45:0/1032883239 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5194101f10 0x7f5194102330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:57.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.041+0000 7f519ac00640 1 -- 172.21.15.45:0/1032883239 >> 172.21.15.45:0/1032883239 conn(0x7f51940fd4a0 msgr2=0x7f51940ff8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:05:57.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.041+0000 7f519ac00640 1 -- 172.21.15.45:0/1032883239 shutdown_connections 2024-04-16T17:05:57.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.042+0000 7f519ac00640 1 -- 172.21.15.45:0/1032883239 wait complete. 2024-04-16T17:05:57.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.043+0000 7f519ac00640 1 Processor -- start 2024-04-16T17:05:57.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.043+0000 7f519ac00640 1 -- start start 2024-04-16T17:05:57.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.043+0000 7f519ac00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5194101f10 0x7f519419a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:57.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.043+0000 7f519ac00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5194103060 0x7f519419a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:57.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.043+0000 7f519ac00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5194103a20 0x7f51941a1970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:57.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.043+0000 7f519ac00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f519410dbc0 con 0x7f5194101f10 2024-04-16T17:05:57.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.043+0000 7f519ac00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f519410da40 con 0x7f5194103060 2024-04-16T17:05:57.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.043+0000 7f519ac00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f519410dd40 con 0x7f5194103a20 2024-04-16T17:05:57.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.044+0000 7f5199800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5194101f10 0x7f519419a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:57.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.044+0000 7f5198e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5194103060 0x7f519419a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:57.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.044+0000 7f5198e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5194103060 0x7f519419a900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:51882/0 (socket says 172.21.15.45:51882) 2024-04-16T17:05:57.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.044+0000 7f5199800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5194101f10 0x7f519419a3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39486/0 (socket says 172.21.15.45:39486) 2024-04-16T17:05:57.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.044+0000 7f5198e00640 1 -- 172.21.15.45:0/3852578398 learned_addr learned my addr 172.21.15.45:0/3852578398 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:05:57.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.044+0000 7f519a200640 1 --2- 172.21.15.45:0/3852578398 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5194103a20 0x7f51941a1970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:57.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.044+0000 7f5198e00640 1 -- 172.21.15.45:0/3852578398 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5194103a20 msgr2=0x7f51941a1970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:57.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.044+0000 7f5198e00640 1 --2- 172.21.15.45:0/3852578398 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5194103a20 0x7f51941a1970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:57.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.044+0000 7f5198e00640 1 -- 172.21.15.45:0/3852578398 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5194101f10 msgr2=0x7f519419a3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:57.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.044+0000 7f5198e00640 1 --2- 172.21.15.45:0/3852578398 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5194101f10 0x7f519419a3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:57.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.044+0000 7f5198e00640 1 -- 172.21.15.45:0/3852578398 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51941a1fe0 con 0x7f5194103060 2024-04-16T17:05:57.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.044+0000 7f519a200640 1 --2- 172.21.15.45:0/3852578398 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5194103a20 0x7f51941a1970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:05:57.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.044+0000 7f5198e00640 1 --2- 172.21.15.45:0/3852578398 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5194103060 0x7f519419a900 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f518400e9c0 tx=0x7f518400ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:05:57.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.045+0000 7f5192000640 1 -- 172.21.15.45:0/3852578398 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f518400cd80 con 0x7f5194103060 2024-04-16T17:05:57.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.045+0000 7f519ac00640 1 -- 172.21.15.45:0/3852578398 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51941a22d0 con 0x7f5194103060 2024-04-16T17:05:57.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.045+0000 7f5192000640 1 -- 172.21.15.45:0/3852578398 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f51840035f0 con 0x7f5194103060 2024-04-16T17:05:57.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.045+0000 7f5192000640 1 -- 172.21.15.45:0/3852578398 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5184010690 con 0x7f5194103060 2024-04-16T17:05:57.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.045+0000 7f519ac00640 1 -- 172.21.15.45:0/3852578398 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51941a2820 con 0x7f5194103060 2024-04-16T17:05:57.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.048+0000 7f519ef89740 1 Processor -- start 2024-04-16T17:05:57.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.048+0000 7f519ef89740 1 -- start start 2024-04-16T17:05:57.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.048+0000 7f519ef89740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c56faa74a0 0x55c56faa78c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:57.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.048+0000 7f519ef89740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c56faa7e90 0x55c56faac990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:57.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.048+0000 7f519ef89740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c56faad000 0x55c56faaf490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:57.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.049+0000 7f519ef89740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55c56fa0d980 con 0x55c56faa7e90 2024-04-16T17:05:57.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.049+0000 7f5192000640 1 -- 172.21.15.45:0/3852578398 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f51840037a0 con 0x7f5194103060 2024-04-16T17:05:57.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.049+0000 7f519ef89740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55c56f9b4220 con 0x55c56faa74a0 2024-04-16T17:05:57.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.049+0000 7f519ef89740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55c56fa09940 con 0x55c56faad000 2024-04-16T17:05:57.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.049+0000 7f5199800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c56faa7e90 0x55c56faac990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:57.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.049+0000 7f519a200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c56faa74a0 0x55c56faa78c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:57.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.049+0000 7f5199800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c56faa7e90 0x55c56faac990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39502/0 (socket says 172.21.15.45:39502) 2024-04-16T17:05:57.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.049+0000 7f5199800640 1 -- 172.21.15.45:0/2322925508 learned_addr learned my addr 172.21.15.45:0/2322925508 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:05:57.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.049+0000 7f519a200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c56faa74a0 0x55c56faa78c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:51898/0 (socket says 172.21.15.45:51898) 2024-04-16T17:05:57.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.049+0000 7f5198e00640 1 --2- 172.21.15.45:0/2322925508 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c56faad000 0x55c56faaf490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:57.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.049+0000 7f519a200640 1 -- 172.21.15.45:0/2322925508 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c56faad000 msgr2=0x55c56faaf490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:05:57.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.049+0000 7f519a200640 1 --2- 172.21.15.45:0/2322925508 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c56faad000 0x55c56faaf490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:57.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.049+0000 7f519a200640 1 -- 172.21.15.45:0/2322925508 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c56faa7e90 msgr2=0x55c56faac990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:05:57.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.049+0000 7f519a200640 1 --2- 172.21.15.45:0/2322925508 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c56faa7e90 0x55c56faac990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:57.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.049+0000 7f519a200640 1 -- 172.21.15.45:0/2322925508 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c56f7202d0 con 0x55c56faa74a0 2024-04-16T17:05:57.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.049+0000 7f5198e00640 1 --2- 172.21.15.45:0/2322925508 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c56faad000 0x55c56faaf490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:05:57.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.050+0000 7f519a200640 1 --2- 172.21.15.45:0/2322925508 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c56faa74a0 0x55c56faa78c0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f518c037040 tx=0x7f518c005ef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f260eeb4aa2f286e server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:05:57.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.050+0000 7f5182a00640 1 -- 172.21.15.45:0/2322925508 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f518c043070 con 0x55c56faa74a0 2024-04-16T17:05:57.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.050+0000 7f5182a00640 1 -- 172.21.15.45:0/2322925508 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f518c003970 con 0x55c56faa74a0 2024-04-16T17:05:57.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.050+0000 7f5182a00640 1 -- 172.21.15.45:0/2322925508 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f518c039040 con 0x55c56faa74a0 2024-04-16T17:05:57.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.050+0000 7f5192000640 1 --2- 172.21.15.45:0/3852578398 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f517c0a3bb0 0x7f517c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:57.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.051+0000 7f5192000640 1 -- 172.21.15.45:0/3852578398 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f51840d0af0 con 0x7f5194103060 2024-04-16T17:05:57.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.051+0000 7f5199800640 1 --2- 172.21.15.45:0/3852578398 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f517c0a3bb0 0x7f517c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:57.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.051+0000 7f519ef89740 1 -- 172.21.15.45:0/2322925508 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c56faa74a0 msgr2=0x55c56faa78c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:57.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.051+0000 7f519ef89740 1 --2- 172.21.15.45:0/2322925508 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c56faa74a0 0x55c56faa78c0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f518c037040 tx=0x7f518c005ef0 comp rx=0 tx=0).stop 2024-04-16T17:05:57.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.051+0000 7f519ef89740 1 -- 172.21.15.45:0/2322925508 shutdown_connections 2024-04-16T17:05:57.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.051+0000 7f519ef89740 1 --2- 172.21.15.45:0/2322925508 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c56faad000 0x55c56faaf490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:57.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.051+0000 7f519ef89740 1 --2- 172.21.15.45:0/2322925508 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c56faa7e90 0x55c56faac990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:57.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.051+0000 7f519ef89740 1 --2- 172.21.15.45:0/2322925508 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c56faa74a0 0x55c56faa78c0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:57.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.051+0000 7f519ef89740 1 -- 172.21.15.45:0/2322925508 >> 172.21.15.45:0/2322925508 conn(0x55c56faa2a10 msgr2=0x55c56faa2e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:05:57.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.051+0000 7f519ef89740 1 -- 172.21.15.45:0/2322925508 shutdown_connections 2024-04-16T17:05:57.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.051+0000 7f519ef89740 1 -- 172.21.15.45:0/2322925508 wait complete. 2024-04-16T17:05:57.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.052+0000 7f519ef89740 1 -- start start 2024-04-16T17:05:57.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.052+0000 7f5199800640 1 --2- 172.21.15.45:0/3852578398 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f517c0a3bb0 0x7f517c0a6070 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x55c56faa3980 tx=0x7f5188005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:05:57.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.052+0000 7f519ef89740 1 Processor -- start 2024-04-16T17:05:57.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.052+0000 7f519ef89740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c56faa74a0 0x55c56fab1930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:57.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.052+0000 7f519ef89740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c56faa7e90 0x55c56fab1e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:57.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.052+0000 7f519ef89740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c56faad000 0x55c56fab8eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:57.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.052+0000 7f519ef89740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55c56faaff50 con 0x55c56faa7e90 2024-04-16T17:05:57.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.052+0000 7f519ef89740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55c56f9c7590 con 0x55c56faad000 2024-04-16T17:05:57.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.052+0000 7f519ef89740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55c56fab00d0 con 0x55c56faa74a0 2024-04-16T17:05:57.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.052+0000 7f519ef89740 10 client.0 _ll_register_callbacks cb 0x55c56f9cf050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:05:57.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.053+0000 7f519a200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c56faad000 0x55c56fab8eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:57.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.053+0000 7f5198e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c56faa7e90 0x55c56fab1e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:57.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.053+0000 7f5199800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c56faa74a0 0x55c56fab1930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:57.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.053+0000 7f519a200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c56faad000 0x55c56fab8eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:51912/0 (socket says 172.21.15.45:51912) 2024-04-16T17:05:57.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.053+0000 7f519a200640 1 -- 172.21.15.45:0/3378681985 learned_addr learned my addr 172.21.15.45:0/3378681985 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:05:57.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.053+0000 7f5198e00640 1 -- 172.21.15.45:0/3378681985 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c56faa74a0 msgr2=0x55c56fab1930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:05:57.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.053+0000 7f5198e00640 1 --2- 172.21.15.45:0/3378681985 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c56faa74a0 0x55c56fab1930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:57.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.053+0000 7f5198e00640 1 -- 172.21.15.45:0/3378681985 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c56faad000 msgr2=0x55c56fab8eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:05:57.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.053+0000 7f5198e00640 1 --2- 172.21.15.45:0/3378681985 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c56faad000 0x55c56fab8eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:57.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.053+0000 7f5198e00640 1 -- 172.21.15.45:0/3378681985 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c56f7ea6b0 con 0x55c56faa7e90 2024-04-16T17:05:57.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.053+0000 7f5199800640 1 --2- 172.21.15.45:0/3378681985 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c56faa74a0 0x55c56fab1930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:05:57.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.053+0000 7f519a200640 1 --2- 172.21.15.45:0/3378681985 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c56faad000 0x55c56fab8eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:05:57.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.053+0000 7f5198e00640 1 --2- 172.21.15.45:0/3378681985 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c56faa7e90 0x55c56fab1e70 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f51840ce2b0 tx=0x7f518409c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf04a137b5d96561 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:05:57.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.053+0000 7f5177400640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-16T17:05:57.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.054+0000 7f5177400640 1 -- 172.21.15.45:0/3378681985 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51840d8040 con 0x55c56faa7e90 2024-04-16T17:05:57.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.054+0000 7f5177400640 1 -- 172.21.15.45:0/3378681985 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f51840a5220 con 0x55c56faa7e90 2024-04-16T17:05:57.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.054+0000 7f519ef89740 1 -- 172.21.15.45:0/3378681985 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f518c003970 con 0x55c56faa7e90 2024-04-16T17:05:57.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.054+0000 7f5177400640 1 -- 172.21.15.45:0/3378681985 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f518409ca40 con 0x55c56faa7e90 2024-04-16T17:05:57.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.054+0000 7f519ef89740 10 client.15066 fetch_fsmap learned FSMap version 24 2024-04-16T17:05:57.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.054+0000 7f519ef89740 1 -- 172.21.15.45:0/3378681985 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55c56fabc2e0 con 0x55c56faa7e90 2024-04-16T17:05:57.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.054+0000 7f5177400640 1 -- 172.21.15.45:0/3378681985 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f518c003970 con 0x55c56faa7e90 2024-04-16T17:05:57.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.054+0000 7f5177400640 1 -- 172.21.15.45:0/3378681985 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f518409be90 con 0x55c56faa7e90 2024-04-16T17:05:57.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.055+0000 7f519ef89740 10 client.15066 fetch_fsmap finished waiting for FSMap version 24 2024-04-16T17:05:57.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.055+0000 7f519ef89740 10 client.15066 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-16T17:05:57.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.055+0000 7f519ef89740 20 client.15066 populate_metadata read hostname 'smithi045' 2024-04-16T17:05:57.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.055+0000 7f519ef89740 1 --2- 172.21.15.45:0/3378681985 >> [v2:172.21.15.117:6838/2378675949,v1:172.21.15.117:6839/2378675949] conn(0x55c56fabe8f0 0x55c56fabfd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:57.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.055+0000 7f519ef89740 4 client.15066 mds_command: new command op to 24532 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:05:57.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.055+0000 7f519ef89740 1 -- 172.21.15.45:0/3378681985 --> [v2:172.21.15.117:6838/2378675949,v1:172.21.15.117:6839/2378675949] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c56fa1b3a0 con 0x55c56fabe8f0 2024-04-16T17:05:57.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.055+0000 7f5199800640 1 --2- 172.21.15.45:0/3378681985 >> [v2:172.21.15.117:6838/2378675949,v1:172.21.15.117:6839/2378675949] conn(0x55c56fabe8f0 0x55c56fabfd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:57.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.056+0000 7f5199800640 1 --2- 172.21.15.45:0/3378681985 >> [v2:172.21.15.117:6838/2378675949,v1:172.21.15.117:6839/2378675949] conn(0x55c56fabe8f0 0x55c56fabfd60 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a4e309245750c6ba server_cookie=b656d9752664830b in_seq=0 out_seq=0 2024-04-16T17:05:57.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.056+0000 7f5177400640 10 client.15066 ms_handle_connect on v2:172.21.15.117:6838/2378675949 2024-04-16T17:05:57.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.057+0000 7f5177400640 1 -- 172.21.15.45:0/3378681985 <== mds.2 v2:172.21.15.117:6838/2378675949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c56fa1b3a0 con 0x55c56fabe8f0 2024-04-16T17:05:57.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.057+0000 7f5177400640 10 client.15066 handle_command_reply: tid=0 2024-04-16T17:05:57.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.057+0000 7f519ef89740 1 client.15066 shutdown 2024-04-16T17:05:57.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.057+0000 7f519ef89740 1 -- 172.21.15.45:0/3378681985 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c56faa7e90 msgr2=0x55c56fab1e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:57.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.057+0000 7f519ef89740 1 --2- 172.21.15.45:0/3378681985 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c56faa7e90 0x55c56fab1e70 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f51840ce2b0 tx=0x7f518409c8e0 comp rx=0 tx=0).stop 2024-04-16T17:05:57.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.057+0000 7f519ef89740 1 -- 172.21.15.45:0/3378681985 shutdown_connections 2024-04-16T17:05:57.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.057+0000 7f519ef89740 1 --2- 172.21.15.45:0/3378681985 >> [v2:172.21.15.117:6838/2378675949,v1:172.21.15.117:6839/2378675949] conn(0x55c56fabe8f0 0x55c56fabfd60 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:57.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.057+0000 7f519ef89740 1 --2- 172.21.15.45:0/3378681985 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c56faad000 0x55c56fab8eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:57.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.057+0000 7f519ef89740 1 --2- 172.21.15.45:0/3378681985 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c56faa7e90 0x55c56fab1e70 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:57.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.057+0000 7f519ef89740 1 --2- 172.21.15.45:0/3378681985 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c56faa74a0 0x55c56fab1930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:57.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.057+0000 7f5177400640 0 client.15066 ms_handle_reset on v2:172.21.15.117:6838/2378675949 2024-04-16T17:05:57.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.057+0000 7f519ef89740 1 -- 172.21.15.45:0/3378681985 >> 172.21.15.45:0/3378681985 conn(0x55c56faa2a10 msgr2=0x55c56faa1c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:05:57.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.057+0000 7f519ef89740 1 -- 172.21.15.45:0/3378681985 shutdown_connections 2024-04-16T17:05:57.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.057+0000 7f519ef89740 1 -- 172.21.15.45:0/3378681985 wait complete. 2024-04-16T17:05:57.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.057+0000 7f519ef89740 20 client.15066 trim_cache size 0 max 16384 2024-04-16T17:05:57.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.077+0000 7f519ef89740 1 Processor -- start 2024-04-16T17:05:57.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.078+0000 7f519ef89740 1 -- start start 2024-04-16T17:05:57.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.078+0000 7f519ef89740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c56faa74a0 0x55c56faaf3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:57.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.078+0000 7f519ef89740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c56faa7e90 0x55c56fab8e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:57.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.078+0000 7f519ef89740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c56faad000 0x55c56faa9350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:57.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.078+0000 7f519ef89740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55c56fabe0a0 con 0x55c56faad000 2024-04-16T17:05:57.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.078+0000 7f519ef89740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55c56fabe770 con 0x55c56faa74a0 2024-04-16T17:05:57.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.078+0000 7f519ef89740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55c56fabee40 con 0x55c56faa7e90 2024-04-16T17:05:57.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.078+0000 7f519a200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c56faad000 0x55c56faa9350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:57.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.078+0000 7f519a200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c56faad000 0x55c56faa9350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39518/0 (socket says 172.21.15.45:39518) 2024-04-16T17:05:57.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.078+0000 7f519a200640 1 -- 172.21.15.45:0/1821190974 learned_addr learned my addr 172.21.15.45:0/1821190974 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:05:57.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.078+0000 7f5199800640 1 --2- 172.21.15.45:0/1821190974 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c56faa74a0 0x55c56faaf3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:57.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.078+0000 7f5198e00640 1 --2- 172.21.15.45:0/1821190974 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c56faa7e90 0x55c56fab8e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:57.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.079+0000 7f5198e00640 1 -- 172.21.15.45:0/1821190974 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c56faa74a0 msgr2=0x55c56faaf3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:05:57.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.079+0000 7f5198e00640 1 --2- 172.21.15.45:0/1821190974 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c56faa74a0 0x55c56faaf3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:57.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.079+0000 7f5198e00640 1 -- 172.21.15.45:0/1821190974 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c56faad000 msgr2=0x55c56faa9350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:05:57.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.079+0000 7f5198e00640 1 --2- 172.21.15.45:0/1821190974 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c56faad000 0x55c56faa9350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:57.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.079+0000 7f5198e00640 1 -- 172.21.15.45:0/1821190974 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c56faa3780 con 0x55c56faa7e90 2024-04-16T17:05:57.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.079+0000 7f519a200640 1 --2- 172.21.15.45:0/1821190974 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c56faad000 0x55c56faa9350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:05:57.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.079+0000 7f5198e00640 1 --2- 172.21.15.45:0/1821190974 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c56faa7e90 0x55c56fab8e70 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f518400c6a0 tx=0x7f518409af50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3faa9596b1dc4741 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:05:57.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.080+0000 7f5182000640 1 -- 172.21.15.45:0/1821190974 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f518409c3a0 con 0x55c56faa7e90 2024-04-16T17:05:57.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.080+0000 7f5182000640 1 -- 172.21.15.45:0/1821190974 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f51840bd0b0 con 0x55c56faa7e90 2024-04-16T17:05:57.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.080+0000 7f5182000640 1 -- 172.21.15.45:0/1821190974 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51840a5220 con 0x55c56faa7e90 2024-04-16T17:05:57.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.080+0000 7f519ef89740 1 -- 172.21.15.45:0/1821190974 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c56faa7e90 msgr2=0x55c56fab8e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:57.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.080+0000 7f519ef89740 1 --2- 172.21.15.45:0/1821190974 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c56faa7e90 0x55c56fab8e70 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f518400c6a0 tx=0x7f518409af50 comp rx=0 tx=0).stop 2024-04-16T17:05:57.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.080+0000 7f519ef89740 1 -- 172.21.15.45:0/1821190974 shutdown_connections 2024-04-16T17:05:57.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.080+0000 7f519ef89740 1 --2- 172.21.15.45:0/1821190974 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c56faa7e90 0x55c56fab8e70 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:57.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.080+0000 7f519ef89740 1 --2- 172.21.15.45:0/1821190974 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c56faad000 0x55c56faa9350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:57.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.080+0000 7f519ef89740 1 --2- 172.21.15.45:0/1821190974 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c56faa74a0 0x55c56faaf3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:57.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.080+0000 7f519ef89740 1 -- 172.21.15.45:0/1821190974 >> 172.21.15.45:0/1821190974 conn(0x55c56faa2a10 msgr2=0x55c56fa9da70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:05:57.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.081+0000 7f519ef89740 1 -- 172.21.15.45:0/1821190974 shutdown_connections 2024-04-16T17:05:57.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.081+0000 7f519ef89740 1 -- 172.21.15.45:0/1821190974 wait complete. 2024-04-16T17:05:57.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.081+0000 7f519ef89740 1 -- start start 2024-04-16T17:05:57.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.081+0000 7f519ef89740 1 Processor -- start 2024-04-16T17:05:57.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.082+0000 7f519ef89740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c56faa74a0 0x55c56fab4050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:57.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.082+0000 7f519ef89740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c56faa7e90 0x55c56fab4590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:57.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.082+0000 7f519ef89740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c56faad000 0x55c56fab75b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:57.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.082+0000 7f519ef89740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55c56fabfbe0 con 0x55c56faa7e90 2024-04-16T17:05:57.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.082+0000 7f519ef89740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55c56fabf510 con 0x55c56faa74a0 2024-04-16T17:05:57.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.082+0000 7f519ef89740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55c56fac02b0 con 0x55c56faad000 2024-04-16T17:05:57.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.082+0000 7f519ef89740 10 client.0 _ll_register_callbacks cb 0x7f5184003e10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:05:57.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.082+0000 7f5198e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c56faa7e90 0x55c56fab4590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:57.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.082+0000 7f5199800640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c56faa74a0 0x55c56fab4050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:57.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.082+0000 7f5198e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c56faa7e90 0x55c56fab4590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39532/0 (socket says 172.21.15.45:39532) 2024-04-16T17:05:57.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.082+0000 7f5198e00640 1 -- 172.21.15.45:0/904392167 learned_addr learned my addr 172.21.15.45:0/904392167 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:05:57.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.082+0000 7f519a200640 1 --2- 172.21.15.45:0/904392167 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c56faad000 0x55c56fab75b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:57.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.082+0000 7f5199800640 1 -- 172.21.15.45:0/904392167 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c56faad000 msgr2=0x55c56fab75b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:05:57.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.082+0000 7f5199800640 1 --2- 172.21.15.45:0/904392167 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c56faad000 0x55c56fab75b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:57.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.082+0000 7f5199800640 1 -- 172.21.15.45:0/904392167 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c56faa7e90 msgr2=0x55c56fab4590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:05:57.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.082+0000 7f5199800640 1 --2- 172.21.15.45:0/904392167 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c56faa7e90 0x55c56fab4590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:57.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.082+0000 7f5199800640 1 -- 172.21.15.45:0/904392167 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c56fab7ba0 con 0x55c56faa74a0 2024-04-16T17:05:57.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.082+0000 7f5198e00640 1 --2- 172.21.15.45:0/904392167 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c56faa7e90 0x55c56fab4590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:05:57.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.083+0000 7f5199800640 1 --2- 172.21.15.45:0/904392167 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c56faa74a0 0x55c56fab4050 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x55c56fac06f0 tx=0x7f51880046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f59ba4e5a465797 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:05:57.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.083+0000 7f5177400640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-04-16T17:05:57.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.083+0000 7f5177400640 1 -- 172.21.15.45:0/904392167 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f518801ee00 con 0x55c56faa74a0 2024-04-16T17:05:57.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.083+0000 7f5177400640 1 -- 172.21.15.45:0/904392167 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f518802e0f0 con 0x55c56faa74a0 2024-04-16T17:05:57.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.083+0000 7f519ef89740 1 -- 172.21.15.45:0/904392167 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f51840bd0b0 con 0x55c56faa74a0 2024-04-16T17:05:57.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.083+0000 7f5177400640 1 -- 172.21.15.45:0/904392167 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f518802e2f0 con 0x55c56faa74a0 2024-04-16T17:05:57.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.083+0000 7f519ef89740 10 client.25184 fetch_fsmap learned FSMap version 24 2024-04-16T17:05:57.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.083+0000 7f519ef89740 1 -- 172.21.15.45:0/904392167 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55c56fab8f10 con 0x55c56faa74a0 2024-04-16T17:05:57.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.083+0000 7f5177400640 1 -- 172.21.15.45:0/904392167 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f51840bd0b0 con 0x55c56faa74a0 2024-04-16T17:05:57.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.084+0000 7f5177400640 1 -- 172.21.15.45:0/904392167 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f5188037b70 con 0x55c56faa74a0 2024-04-16T17:05:57.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.084+0000 7f519ef89740 10 client.25184 fetch_fsmap finished waiting for FSMap version 24 2024-04-16T17:05:57.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.084+0000 7f519ef89740 10 client.25184 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-16T17:05:57.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.084+0000 7f519ef89740 20 client.25184 populate_metadata read hostname 'smithi045' 2024-04-16T17:05:57.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.084+0000 7f519ef89740 1 --2- 172.21.15.45:0/904392167 >> [v2:172.21.15.117:6838/2378675949,v1:172.21.15.117:6839/2378675949] conn(0x55c56fb31770 0x55c56fb30480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:57.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.084+0000 7f519ef89740 4 client.25184 mds_command: new command op to 24532 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-16T17:05:57.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.084+0000 7f519ef89740 1 -- 172.21.15.45:0/904392167 --> [v2:172.21.15.117:6838/2378675949,v1:172.21.15.117:6839/2378675949] -- command(tid 0: {"prefix": "status"}) -- 0x55c56fa15210 con 0x55c56fb31770 2024-04-16T17:05:57.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.084+0000 7f5198e00640 1 --2- 172.21.15.45:0/904392167 >> [v2:172.21.15.117:6838/2378675949,v1:172.21.15.117:6839/2378675949] conn(0x55c56fb31770 0x55c56fb30480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:57.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.085+0000 7f5198e00640 1 --2- 172.21.15.45:0/904392167 >> [v2:172.21.15.117:6838/2378675949,v1:172.21.15.117:6839/2378675949] conn(0x55c56fb31770 0x55c56fb30480 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f0290842eb1a74d9 server_cookie=9016d11553984cd in_seq=0 out_seq=0 2024-04-16T17:05:57.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.085+0000 7f5177400640 10 client.25184 ms_handle_connect on v2:172.21.15.117:6838/2378675949 2024-04-16T17:05:57.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.085+0000 7f5177400640 1 -- 172.21.15.45:0/904392167 <== mds.2 v2:172.21.15.117:6838/2378675949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x55c56fa15210 con 0x55c56fb31770 2024-04-16T17:05:57.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.085+0000 7f5177400640 10 client.25184 handle_command_reply: tid=0 2024-04-16T17:05:57.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.085+0000 7f519ef89740 1 client.25184 shutdown 2024-04-16T17:05:57.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.085+0000 7f519ef89740 1 -- 172.21.15.45:0/904392167 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c56faa74a0 msgr2=0x55c56fab4050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:57.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.085+0000 7f519ef89740 1 --2- 172.21.15.45:0/904392167 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c56faa74a0 0x55c56fab4050 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x55c56fac06f0 tx=0x7f51880046c0 comp rx=0 tx=0).stop 2024-04-16T17:05:57.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.085+0000 7f519ef89740 1 -- 172.21.15.45:0/904392167 shutdown_connections 2024-04-16T17:05:57.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.085+0000 7f519ef89740 1 --2- 172.21.15.45:0/904392167 >> [v2:172.21.15.117:6838/2378675949,v1:172.21.15.117:6839/2378675949] conn(0x55c56fb31770 0x55c56fb30480 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:57.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.085+0000 7f519ef89740 1 --2- 172.21.15.45:0/904392167 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c56faad000 0x55c56fab75b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:57.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.085+0000 7f519ef89740 1 --2- 172.21.15.45:0/904392167 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c56faa7e90 0x55c56fab4590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:57.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.085+0000 7f5177400640 0 client.25184 ms_handle_reset on v2:172.21.15.117:6838/2378675949 2024-04-16T17:05:57.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.085+0000 7f519ef89740 1 --2- 172.21.15.45:0/904392167 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c56faa74a0 0x55c56fab4050 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:57.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.085+0000 7f519ef89740 1 -- 172.21.15.45:0/904392167 >> 172.21.15.45:0/904392167 conn(0x55c56faa2a10 msgr2=0x55c56faa9350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:05:57.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.085+0000 7f519ef89740 1 -- 172.21.15.45:0/904392167 shutdown_connections 2024-04-16T17:05:57.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.085+0000 7f519ef89740 1 -- 172.21.15.45:0/904392167 wait complete. 2024-04-16T17:05:57.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.086+0000 7f519ef89740 20 client.25184 trim_cache size 0 max 16384 2024-04-16T17:05:57.090 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-16T17:05:57.090 INFO:teuthology.orchestra.run.smithi045.stdout: "cluster_fsid": "b1fecd38-fc11-11ee-bc90-c7b262605968", 2024-04-16T17:05:57.090 INFO:teuthology.orchestra.run.smithi045.stdout: "whoami": 2, 2024-04-16T17:05:57.090 INFO:teuthology.orchestra.run.smithi045.stdout: "id": 24532, 2024-04-16T17:05:57.090 INFO:teuthology.orchestra.run.smithi045.stdout: "want_state": "up:active", 2024-04-16T17:05:57.090 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "up:active", 2024-04-16T17:05:57.090 INFO:teuthology.orchestra.run.smithi045.stdout: "fs_name": "cephfs", 2024-04-16T17:05:57.090 INFO:teuthology.orchestra.run.smithi045.stdout: "rank_uptime": 124.714816015, 2024-04-16T17:05:57.090 INFO:teuthology.orchestra.run.smithi045.stdout: "mdsmap_epoch": 24, 2024-04-16T17:05:57.090 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_epoch": 94, 2024-04-16T17:05:57.091 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_epoch_barrier": 90, 2024-04-16T17:05:57.091 INFO:teuthology.orchestra.run.smithi045.stdout: "uptime": 169.839859937 2024-04-16T17:05:57.091 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-16T17:05:57.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.086+0000 7f519ac00640 1 -- 172.21.15.45:0/3852578398 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f517c0a3bb0 msgr2=0x7f517c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:57.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.086+0000 7f519ac00640 1 --2- 172.21.15.45:0/3852578398 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f517c0a3bb0 0x7f517c0a6070 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x55c56faa3980 tx=0x7f5188005f90 comp rx=0 tx=0).stop 2024-04-16T17:05:57.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.086+0000 7f519ac00640 1 -- 172.21.15.45:0/3852578398 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5194103060 msgr2=0x7f519419a900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:57.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.086+0000 7f519ac00640 1 --2- 172.21.15.45:0/3852578398 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5194103060 0x7f519419a900 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f518400e9c0 tx=0x7f518400ee80 comp rx=0 tx=0).stop 2024-04-16T17:05:57.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.086+0000 7f519ac00640 1 -- 172.21.15.45:0/3852578398 shutdown_connections 2024-04-16T17:05:57.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.086+0000 7f519ac00640 1 --2- 172.21.15.45:0/3852578398 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f517c0a3bb0 0x7f517c0a6070 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:57.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.086+0000 7f519ac00640 1 --2- 172.21.15.45:0/3852578398 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5194103a20 0x7f51941a1970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:57.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.086+0000 7f519ac00640 1 --2- 172.21.15.45:0/3852578398 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5194103060 0x7f519419a900 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:57.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.086+0000 7f519ac00640 1 --2- 172.21.15.45:0/3852578398 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5194101f10 0x7f519419a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:57.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.086+0000 7f519ac00640 1 -- 172.21.15.45:0/3852578398 >> 172.21.15.45:0/3852578398 conn(0x7f51940fd4a0 msgr2=0x7f5194074230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:05:57.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.086+0000 7f519ac00640 1 -- 172.21.15.45:0/3852578398 shutdown_connections 2024-04-16T17:05:57.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.087+0000 7f519ac00640 1 -- 172.21.15.45:0/3852578398 wait complete. 2024-04-16T17:05:57.106 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-16T17:05:57.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.324+0000 7fd0a3200640 1 -- 172.21.15.45:0/2428407311 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd0a4104140 msgr2=0x7fd0a4104560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:57.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.324+0000 7fd0a3200640 1 --2- 172.21.15.45:0/2428407311 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd0a4104140 0x7fd0a4104560 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fd094009ca0 tx=0x7fd09402c100 comp rx=0 tx=0).stop 2024-04-16T17:05:57.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.324+0000 7fd0a3200640 1 -- 172.21.15.45:0/2428407311 shutdown_connections 2024-04-16T17:05:57.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.324+0000 7fd0a3200640 1 --2- 172.21.15.45:0/2428407311 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd0a4105330 0x7fd0a41057b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:57.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.324+0000 7fd0a3200640 1 --2- 172.21.15.45:0/2428407311 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd0a4105cf0 0x7fd0a410c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:57.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.324+0000 7fd0a3200640 1 --2- 172.21.15.45:0/2428407311 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd0a4104140 0x7fd0a4104560 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:57.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.324+0000 7fd0a3200640 1 -- 172.21.15.45:0/2428407311 >> 172.21.15.45:0/2428407311 conn(0x7fd0a40ff6b0 msgr2=0x7fd0a4101af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:05:57.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.324+0000 7fd0a3200640 1 -- 172.21.15.45:0/2428407311 shutdown_connections 2024-04-16T17:05:57.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.324+0000 7fd0a3200640 1 -- 172.21.15.45:0/2428407311 wait complete. 2024-04-16T17:05:57.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.325+0000 7fd0a3200640 1 Processor -- start 2024-04-16T17:05:57.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.325+0000 7fd0a3200640 1 -- start start 2024-04-16T17:05:57.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.326+0000 7fd0a3200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd0a4104140 0x7fd0a419a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:57.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.326+0000 7fd0a3200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd0a4105330 0x7fd0a419a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:57.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.326+0000 7fd0a3200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd0a4105cf0 0x7fd0a41a1a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:57.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.326+0000 7fd0a3200640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fd0a410fdd0 con 0x7fd0a4105cf0 2024-04-16T17:05:57.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.326+0000 7fd0a3200640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fd0a410fc50 con 0x7fd0a4104140 2024-04-16T17:05:57.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.326+0000 7fd0a3200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fd0a410ff50 con 0x7fd0a4105330 2024-04-16T17:05:57.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.326+0000 7fd0a1e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd0a4104140 0x7fd0a419a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:57.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.326+0000 7fd0a1e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd0a4104140 0x7fd0a419a460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:51942/0 (socket says 172.21.15.45:51942) 2024-04-16T17:05:57.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.326+0000 7fd0a1e00640 1 -- 172.21.15.45:0/2298784065 learned_addr learned my addr 172.21.15.45:0/2298784065 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:05:57.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.326+0000 7fd0a2800640 1 --2- 172.21.15.45:0/2298784065 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd0a4105cf0 0x7fd0a41a1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:57.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.326+0000 7fd0a1400640 1 --2- 172.21.15.45:0/2298784065 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd0a4105330 0x7fd0a419a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:57.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.327+0000 7fd0a1e00640 1 -- 172.21.15.45:0/2298784065 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd0a4105330 msgr2=0x7fd0a419a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:57.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.327+0000 7fd0a1e00640 1 --2- 172.21.15.45:0/2298784065 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd0a4105330 0x7fd0a419a9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:57.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.327+0000 7fd0a1e00640 1 -- 172.21.15.45:0/2298784065 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd0a4105cf0 msgr2=0x7fd0a41a1a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:57.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.327+0000 7fd0a1e00640 1 --2- 172.21.15.45:0/2298784065 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd0a4105cf0 0x7fd0a41a1a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:57.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.327+0000 7fd0a1e00640 1 -- 172.21.15.45:0/2298784065 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0a41a2080 con 0x7fd0a4104140 2024-04-16T17:05:57.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.327+0000 7fd0a1e00640 1 --2- 172.21.15.45:0/2298784065 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd0a4104140 0x7fd0a419a460 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fd094000f80 tx=0x7fd094035d10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:05:57.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.327+0000 7fd09aa00640 1 -- 172.21.15.45:0/2298784065 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd094039040 con 0x7fd0a4104140 2024-04-16T17:05:57.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.327+0000 7fd09aa00640 1 -- 172.21.15.45:0/2298784065 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd094044070 con 0x7fd0a4104140 2024-04-16T17:05:57.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.327+0000 7fd0a3200640 1 -- 172.21.15.45:0/2298784065 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0a41a2310 con 0x7fd0a4104140 2024-04-16T17:05:57.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.327+0000 7fd09aa00640 1 -- 172.21.15.45:0/2298784065 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0940044a0 con 0x7fd0a4104140 2024-04-16T17:05:57.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.328+0000 7fd0a3200640 1 -- 172.21.15.45:0/2298784065 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0a41a2820 con 0x7fd0a4104140 2024-04-16T17:05:57.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.331+0000 7fd09aa00640 1 -- 172.21.15.45:0/2298784065 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fd094004640 con 0x7fd0a4104140 2024-04-16T17:05:57.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.331+0000 7fd0abb99740 1 Processor -- start 2024-04-16T17:05:57.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.331+0000 7fd0abb99740 1 -- start start 2024-04-16T17:05:57.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.331+0000 7fd0abb99740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d17f73f4a0 0x55d17f73f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:57.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.331+0000 7fd0abb99740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d17f73fe90 0x55d17f744990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:57.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.331+0000 7fd0abb99740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d17f745000 0x55d17f747490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:57.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.331+0000 7fd0abb99740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55d17f6a5980 con 0x55d17f745000 2024-04-16T17:05:57.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.331+0000 7fd0abb99740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55d17f64c220 con 0x55d17f73f4a0 2024-04-16T17:05:57.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.331+0000 7fd0abb99740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55d17f6a1940 con 0x55d17f73fe90 2024-04-16T17:05:57.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.332+0000 7fd0a2800640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d17f73f4a0 0x55d17f73f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:57.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.332+0000 7fd0a2800640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d17f73f4a0 0x55d17f73f8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:51946/0 (socket says 172.21.15.45:51946) 2024-04-16T17:05:57.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.332+0000 7fd0a2800640 1 -- 172.21.15.45:0/1846003498 learned_addr learned my addr 172.21.15.45:0/1846003498 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:05:57.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.332+0000 7fd0a1e00640 1 --2- 172.21.15.45:0/1846003498 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d17f73fe90 0x55d17f744990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:57.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.332+0000 7fd0a1400640 1 --2- 172.21.15.45:0/1846003498 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d17f745000 0x55d17f747490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:57.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.332+0000 7fd0a2800640 1 -- 172.21.15.45:0/1846003498 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d17f73fe90 msgr2=0x55d17f744990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:05:57.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.332+0000 7fd0a2800640 1 --2- 172.21.15.45:0/1846003498 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d17f73fe90 0x55d17f744990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:57.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.332+0000 7fd0a2800640 1 -- 172.21.15.45:0/1846003498 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d17f745000 msgr2=0x55d17f747490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:05:57.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.332+0000 7fd0a2800640 1 --2- 172.21.15.45:0/1846003498 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d17f745000 0x55d17f747490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:57.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.332+0000 7fd0a2800640 1 -- 172.21.15.45:0/1846003498 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d17f3b82d0 con 0x55d17f73f4a0 2024-04-16T17:05:57.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.332+0000 7fd0a1e00640 1 --2- 172.21.15.45:0/1846003498 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d17f73fe90 0x55d17f744990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:05:57.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.332+0000 7fd0a2800640 1 --2- 172.21.15.45:0/1846003498 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d17f73f4a0 0x55d17f73f8c0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fd09c00b6b0 tx=0x7fd09c00bb70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ac3eb23edf588115 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:05:57.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.333+0000 7fd09aa00640 1 --2- 172.21.15.45:0/2298784065 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fd0880a3c30 0x7fd0880a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:57.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.333+0000 7fd08f400640 1 -- 172.21.15.45:0/1846003498 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd09c003bd0 con 0x55d17f73f4a0 2024-04-16T17:05:57.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.333+0000 7fd08f400640 1 -- 172.21.15.45:0/1846003498 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd09c015070 con 0x55d17f73f4a0 2024-04-16T17:05:57.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.333+0000 7fd08f400640 1 -- 172.21.15.45:0/1846003498 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd09c011680 con 0x55d17f73f4a0 2024-04-16T17:05:57.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.333+0000 7fd09aa00640 1 -- 172.21.15.45:0/2298784065 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fd0940f2d40 con 0x7fd0a4104140 2024-04-16T17:05:57.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.333+0000 7fd0a1e00640 1 --2- 172.21.15.45:0/2298784065 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fd0880a3c30 0x7fd0880a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:57.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.334+0000 7fd0abb99740 1 -- 172.21.15.45:0/1846003498 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d17f73f4a0 msgr2=0x55d17f73f8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:57.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.334+0000 7fd0abb99740 1 --2- 172.21.15.45:0/1846003498 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d17f73f4a0 0x55d17f73f8c0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fd09c00b6b0 tx=0x7fd09c00bb70 comp rx=0 tx=0).stop 2024-04-16T17:05:57.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.334+0000 7fd0a1e00640 1 --2- 172.21.15.45:0/2298784065 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fd0880a3c30 0x7fd0880a60f0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fd094009bf0 tx=0x7fd0940f6780 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:05:57.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.334+0000 7fd0abb99740 1 -- 172.21.15.45:0/1846003498 shutdown_connections 2024-04-16T17:05:57.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.334+0000 7fd0abb99740 1 --2- 172.21.15.45:0/1846003498 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d17f73fe90 0x55d17f744990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:57.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.334+0000 7fd0abb99740 1 --2- 172.21.15.45:0/1846003498 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d17f745000 0x55d17f747490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:57.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.334+0000 7fd0abb99740 1 --2- 172.21.15.45:0/1846003498 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d17f73f4a0 0x55d17f73f8c0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:57.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.334+0000 7fd0abb99740 1 -- 172.21.15.45:0/1846003498 >> 172.21.15.45:0/1846003498 conn(0x55d17f73aa10 msgr2=0x55d17f73ae10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:05:57.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.334+0000 7fd0abb99740 1 -- 172.21.15.45:0/1846003498 shutdown_connections 2024-04-16T17:05:57.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.334+0000 7fd0abb99740 1 -- 172.21.15.45:0/1846003498 wait complete. 2024-04-16T17:05:57.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.335+0000 7fd0abb99740 1 -- start start 2024-04-16T17:05:57.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.335+0000 7fd0abb99740 1 Processor -- start 2024-04-16T17:05:57.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.335+0000 7fd0abb99740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d17f73f4a0 0x55d17f7499f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:57.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.335+0000 7fd0abb99740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d17f73fe90 0x55d17f749f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:57.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.335+0000 7fd0abb99740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d17f745000 0x55d17f750f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:57.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.335+0000 7fd0abb99740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55d17f747fb0 con 0x55d17f73f4a0 2024-04-16T17:05:57.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.335+0000 7fd0abb99740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55d17f65f590 con 0x55d17f73fe90 2024-04-16T17:05:57.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.335+0000 7fd0abb99740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55d17f748130 con 0x55d17f745000 2024-04-16T17:05:57.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.335+0000 7fd0abb99740 10 client.0 _ll_register_callbacks cb 0x55d17f667050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:05:57.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.336+0000 7fd0a2800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d17f73f4a0 0x55d17f7499f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:57.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.336+0000 7fd0a1400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d17f73fe90 0x55d17f749f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:57.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.336+0000 7fd0a1400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d17f73fe90 0x55d17f749f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:51950/0 (socket says 172.21.15.45:51950) 2024-04-16T17:05:57.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.336+0000 7fd0a1400640 1 -- 172.21.15.45:0/3366803039 learned_addr learned my addr 172.21.15.45:0/3366803039 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:05:57.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.336+0000 7fd0a2800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d17f73f4a0 0x55d17f7499f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39562/0 (socket says 172.21.15.45:39562) 2024-04-16T17:05:57.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.336+0000 7fd0a2800640 1 --2- 172.21.15.45:0/3366803039 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d17f745000 0x55d17f750f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:57.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.336+0000 7fd0a1400640 1 -- 172.21.15.45:0/3366803039 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d17f745000 msgr2=0x55d17f750f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:05:57.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.336+0000 7fd0a1400640 1 --2- 172.21.15.45:0/3366803039 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d17f745000 0x55d17f750f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:57.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.336+0000 7fd0a1400640 1 -- 172.21.15.45:0/3366803039 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d17f73f4a0 msgr2=0x55d17f7499f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:05:57.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.336+0000 7fd0a1400640 1 --2- 172.21.15.45:0/3366803039 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d17f73f4a0 0x55d17f7499f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:57.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.336+0000 7fd0a1400640 1 -- 172.21.15.45:0/3366803039 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d17f4826b0 con 0x55d17f73fe90 2024-04-16T17:05:57.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.336+0000 7fd0a2800640 1 --2- 172.21.15.45:0/3366803039 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d17f73f4a0 0x55d17f7499f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:05:57.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.336+0000 7fd0a1400640 1 --2- 172.21.15.45:0/3366803039 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d17f73fe90 0x55d17f749f30 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fd090012700 tx=0x7fd090012bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7e15172dbcf706df server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:05:57.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.336+0000 7fd083e00640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-04-16T17:05:57.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.337+0000 7fd083e00640 1 -- 172.21.15.45:0/3366803039 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd09000f9a0 con 0x55d17f73fe90 2024-04-16T17:05:57.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.337+0000 7fd0abb99740 1 -- 172.21.15.45:0/3366803039 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd09c015070 con 0x55d17f73fe90 2024-04-16T17:05:57.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.337+0000 7fd083e00640 1 -- 172.21.15.45:0/3366803039 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fd09000fb40 con 0x55d17f73fe90 2024-04-16T17:05:57.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.337+0000 7fd083e00640 1 -- 172.21.15.45:0/3366803039 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fd09000fd40 con 0x55d17f73fe90 2024-04-16T17:05:57.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.337+0000 7fd0abb99740 10 client.25196 fetch_fsmap learned FSMap version 24 2024-04-16T17:05:57.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.337+0000 7fd0abb99740 1 -- 172.21.15.45:0/3366803039 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55d17f7543a0 con 0x55d17f73fe90 2024-04-16T17:05:57.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.337+0000 7fd083e00640 1 -- 172.21.15.45:0/3366803039 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd09c015070 con 0x55d17f73fe90 2024-04-16T17:05:57.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.338+0000 7fd083e00640 1 -- 172.21.15.45:0/3366803039 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7fd09001c460 con 0x55d17f73fe90 2024-04-16T17:05:57.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.338+0000 7fd0abb99740 10 client.25196 fetch_fsmap finished waiting for FSMap version 24 2024-04-16T17:05:57.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.338+0000 7fd0abb99740 10 client.25196 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-16T17:05:57.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.338+0000 7fd0abb99740 20 client.25196 populate_metadata read hostname 'smithi045' 2024-04-16T17:05:57.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.338+0000 7fd0abb99740 1 --2- 172.21.15.45:0/3366803039 >> [v2:172.21.15.45:6840/3755310822,v1:172.21.15.45:6841/3755310822] conn(0x55d17f7569c0 0x55d17f757e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:57.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.338+0000 7fd0abb99740 4 client.25196 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:05:57.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.338+0000 7fd0abb99740 1 -- 172.21.15.45:0/3366803039 --> [v2:172.21.15.45:6840/3755310822,v1:172.21.15.45:6841/3755310822] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d17f6b33a0 con 0x55d17f7569c0 2024-04-16T17:05:57.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.338+0000 7fd0a2800640 1 --2- 172.21.15.45:0/3366803039 >> [v2:172.21.15.45:6840/3755310822,v1:172.21.15.45:6841/3755310822] conn(0x55d17f7569c0 0x55d17f757e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:57.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.339+0000 7fd0a2800640 1 --2- 172.21.15.45:0/3366803039 >> [v2:172.21.15.45:6840/3755310822,v1:172.21.15.45:6841/3755310822] conn(0x55d17f7569c0 0x55d17f757e30 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5c1142ac721cf066 server_cookie=77eaa28abb01e06e in_seq=0 out_seq=0 2024-04-16T17:05:57.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.339+0000 7fd083e00640 10 client.25196 ms_handle_connect on v2:172.21.15.45:6840/3755310822 2024-04-16T17:05:57.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.340+0000 7fd083e00640 1 -- 172.21.15.45:0/3366803039 <== mds.1 v2:172.21.15.45:6840/3755310822 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d17f6b33a0 con 0x55d17f7569c0 2024-04-16T17:05:57.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.340+0000 7fd083e00640 10 client.25196 handle_command_reply: tid=0 2024-04-16T17:05:57.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.340+0000 7fd0abb99740 1 client.25196 shutdown 2024-04-16T17:05:57.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.340+0000 7fd0abb99740 1 -- 172.21.15.45:0/3366803039 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d17f73fe90 msgr2=0x55d17f749f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:57.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.340+0000 7fd0abb99740 1 --2- 172.21.15.45:0/3366803039 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d17f73fe90 0x55d17f749f30 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fd090012700 tx=0x7fd090012bc0 comp rx=0 tx=0).stop 2024-04-16T17:05:57.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.340+0000 7fd0abb99740 1 -- 172.21.15.45:0/3366803039 shutdown_connections 2024-04-16T17:05:57.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.340+0000 7fd0abb99740 1 --2- 172.21.15.45:0/3366803039 >> [v2:172.21.15.45:6840/3755310822,v1:172.21.15.45:6841/3755310822] conn(0x55d17f7569c0 0x55d17f757e30 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:57.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.340+0000 7fd0abb99740 1 --2- 172.21.15.45:0/3366803039 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d17f745000 0x55d17f750f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:57.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.340+0000 7fd0abb99740 1 --2- 172.21.15.45:0/3366803039 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d17f73fe90 0x55d17f749f30 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:57.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.340+0000 7fd0abb99740 1 --2- 172.21.15.45:0/3366803039 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d17f73f4a0 0x55d17f7499f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:57.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.340+0000 7fd083e00640 0 client.25196 ms_handle_reset on v2:172.21.15.45:6840/3755310822 2024-04-16T17:05:57.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.340+0000 7fd0abb99740 1 -- 172.21.15.45:0/3366803039 >> 172.21.15.45:0/3366803039 conn(0x55d17f73aa10 msgr2=0x55d17f739c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:05:57.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.340+0000 7fd0abb99740 1 -- 172.21.15.45:0/3366803039 shutdown_connections 2024-04-16T17:05:57.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.340+0000 7fd0abb99740 1 -- 172.21.15.45:0/3366803039 wait complete. 2024-04-16T17:05:57.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.340+0000 7fd0abb99740 20 client.25196 trim_cache size 0 max 16384 2024-04-16T17:05:57.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.361+0000 7fd0abb99740 1 Processor -- start 2024-04-16T17:05:57.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.361+0000 7fd0abb99740 1 -- start start 2024-04-16T17:05:57.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.361+0000 7fd0abb99740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d17f73f4a0 0x55d17f7449c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:57.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.361+0000 7fd0abb99740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d17f73fe90 0x55d17f750f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:57.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.361+0000 7fd0abb99740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d17f745000 0x55d17f741350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:57.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.361+0000 7fd0abb99740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55d17f756170 con 0x55d17f745000 2024-04-16T17:05:57.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.361+0000 7fd0abb99740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55d17f756840 con 0x55d17f73f4a0 2024-04-16T17:05:57.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.361+0000 7fd0abb99740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55d17f756f10 con 0x55d17f73fe90 2024-04-16T17:05:57.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.362+0000 7fd0a2800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d17f745000 0x55d17f741350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:57.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.362+0000 7fd0a2800640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d17f73f4a0 0x55d17f7449c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:57.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.362+0000 7fd0a2800640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d17f73f4a0 0x55d17f7449c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:51962/0 (socket says 172.21.15.45:51962) 2024-04-16T17:05:57.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.362+0000 7fd0a2800640 1 -- 172.21.15.45:0/3161820736 learned_addr learned my addr 172.21.15.45:0/3161820736 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:05:57.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.362+0000 7fd0a1400640 1 --2- 172.21.15.45:0/3161820736 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d17f73fe90 0x55d17f750f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:57.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.362+0000 7fd0a2800640 1 -- 172.21.15.45:0/3161820736 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d17f73fe90 msgr2=0x55d17f750f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:05:57.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.362+0000 7fd0a2800640 1 --2- 172.21.15.45:0/3161820736 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d17f73fe90 0x55d17f750f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:57.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.362+0000 7fd0a2800640 1 -- 172.21.15.45:0/3161820736 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d17f745000 msgr2=0x55d17f741350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:05:57.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.362+0000 7fd0a2800640 1 --2- 172.21.15.45:0/3161820736 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d17f745000 0x55d17f741350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:57.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.362+0000 7fd0a2800640 1 -- 172.21.15.45:0/3161820736 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d17f740720 con 0x55d17f73f4a0 2024-04-16T17:05:57.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.362+0000 7fd0a1400640 1 --2- 172.21.15.45:0/3161820736 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d17f73fe90 0x55d17f750f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:05:57.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.362+0000 7fd0a2800640 1 --2- 172.21.15.45:0/3161820736 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d17f73f4a0 0x55d17f7449c0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fd09c00b6b0 tx=0x7fd09c005cc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9a846611ea83213e server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:05:57.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.362+0000 7fd08ea00640 1 -- 172.21.15.45:0/3161820736 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd09c004120 con 0x55d17f73f4a0 2024-04-16T17:05:57.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.362+0000 7fd08ea00640 1 -- 172.21.15.45:0/3161820736 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd09c015040 con 0x55d17f73f4a0 2024-04-16T17:05:57.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.362+0000 7fd08ea00640 1 -- 172.21.15.45:0/3161820736 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd09c005390 con 0x55d17f73f4a0 2024-04-16T17:05:57.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.363+0000 7fd0abb99740 1 -- 172.21.15.45:0/3161820736 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d17f73f4a0 msgr2=0x55d17f7449c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:57.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.363+0000 7fd0abb99740 1 --2- 172.21.15.45:0/3161820736 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d17f73f4a0 0x55d17f7449c0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fd09c00b6b0 tx=0x7fd09c005cc0 comp rx=0 tx=0).stop 2024-04-16T17:05:57.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.363+0000 7fd0abb99740 1 -- 172.21.15.45:0/3161820736 shutdown_connections 2024-04-16T17:05:57.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.363+0000 7fd0abb99740 1 --2- 172.21.15.45:0/3161820736 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d17f73fe90 0x55d17f750f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:57.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.363+0000 7fd0abb99740 1 --2- 172.21.15.45:0/3161820736 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d17f745000 0x55d17f741350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:57.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.363+0000 7fd0abb99740 1 --2- 172.21.15.45:0/3161820736 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d17f73f4a0 0x55d17f7449c0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:57.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.363+0000 7fd0abb99740 1 -- 172.21.15.45:0/3161820736 >> 172.21.15.45:0/3161820736 conn(0x55d17f73aa10 msgr2=0x55d17f7557b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:05:57.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.363+0000 7fd0abb99740 1 -- 172.21.15.45:0/3161820736 shutdown_connections 2024-04-16T17:05:57.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.363+0000 7fd0abb99740 1 -- 172.21.15.45:0/3161820736 wait complete. 2024-04-16T17:05:57.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.364+0000 7fd0abb99740 1 -- start start 2024-04-16T17:05:57.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.364+0000 7fd0abb99740 1 Processor -- start 2024-04-16T17:05:57.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.364+0000 7fd0abb99740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d17f73f4a0 0x55d17f74c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:57.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.364+0000 7fd0abb99740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d17f73fe90 0x55d17f74c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:57.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.364+0000 7fd0abb99740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d17f745000 0x55d17f74fa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:57.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.364+0000 7fd0abb99740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55d17f757cb0 con 0x55d17f73fe90 2024-04-16T17:05:57.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.364+0000 7fd0abb99740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55d17f7575e0 con 0x55d17f745000 2024-04-16T17:05:57.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.364+0000 7fd0abb99740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55d17f758380 con 0x55d17f73f4a0 2024-04-16T17:05:57.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.364+0000 7fd0abb99740 10 client.0 _ll_register_callbacks cb 0x7fd090009910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:05:57.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.365+0000 7fd0a1400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d17f73fe90 0x55d17f74c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:57.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.365+0000 7fd0a2800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d17f73f4a0 0x55d17f74c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:57.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.365+0000 7fd0a1400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d17f73fe90 0x55d17f74c9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39576/0 (socket says 172.21.15.45:39576) 2024-04-16T17:05:57.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.365+0000 7fd0a1400640 1 -- 172.21.15.45:0/2830436658 learned_addr learned my addr 172.21.15.45:0/2830436658 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:05:57.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.365+0000 7fd0a2800640 1 --2- 172.21.15.45:0/2830436658 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d17f745000 0x55d17f74fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:57.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.365+0000 7fd0a1400640 1 -- 172.21.15.45:0/2830436658 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d17f73f4a0 msgr2=0x55d17f74c4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:05:57.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.365+0000 7fd0a1400640 1 --2- 172.21.15.45:0/2830436658 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d17f73f4a0 0x55d17f74c4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:57.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.365+0000 7fd0a1400640 1 -- 172.21.15.45:0/2830436658 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d17f745000 msgr2=0x55d17f74fa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:05:57.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.365+0000 7fd0a1400640 1 --2- 172.21.15.45:0/2830436658 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d17f745000 0x55d17f74fa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:57.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.365+0000 7fd0a1400640 1 -- 172.21.15.45:0/2830436658 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d17f74fff0 con 0x55d17f73fe90 2024-04-16T17:05:57.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.365+0000 7fd0a1400640 1 --2- 172.21.15.45:0/2830436658 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d17f73fe90 0x55d17f74c9e0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x55d17f74aed0 tx=0x7fd09000fcb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5b7fdbfae0c67af server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:05:57.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.365+0000 7fd083e00640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-16T17:05:57.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.366+0000 7fd083e00640 1 -- 172.21.15.45:0/2830436658 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd090030060 con 0x55d17f73fe90 2024-04-16T17:05:57.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.366+0000 7fd083e00640 1 -- 172.21.15.45:0/2830436658 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fd09001c420 con 0x55d17f73fe90 2024-04-16T17:05:57.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.366+0000 7fd0abb99740 1 -- 172.21.15.45:0/2830436658 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd09c015040 con 0x55d17f73fe90 2024-04-16T17:05:57.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.366+0000 7fd083e00640 1 -- 172.21.15.45:0/2830436658 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fd090037050 con 0x55d17f73fe90 2024-04-16T17:05:57.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.366+0000 7fd0abb99740 10 client.15090 fetch_fsmap learned FSMap version 24 2024-04-16T17:05:57.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.366+0000 7fd0abb99740 1 -- 172.21.15.45:0/2830436658 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55d17f7bc970 con 0x55d17f73fe90 2024-04-16T17:05:57.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.366+0000 7fd083e00640 1 -- 172.21.15.45:0/2830436658 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd09c015040 con 0x55d17f73fe90 2024-04-16T17:05:57.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.367+0000 7fd083e00640 1 -- 172.21.15.45:0/2830436658 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7fd090003a10 con 0x55d17f73fe90 2024-04-16T17:05:57.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.367+0000 7fd0abb99740 10 client.15090 fetch_fsmap finished waiting for FSMap version 24 2024-04-16T17:05:57.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.367+0000 7fd0abb99740 10 client.15090 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-16T17:05:57.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.367+0000 7fd0abb99740 20 client.15090 populate_metadata read hostname 'smithi045' 2024-04-16T17:05:57.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.367+0000 7fd0abb99740 1 --2- 172.21.15.45:0/2830436658 >> [v2:172.21.15.45:6840/3755310822,v1:172.21.15.45:6841/3755310822] conn(0x55d17f7c1420 0x55d17f7c0130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:57.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.367+0000 7fd0abb99740 4 client.15090 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-16T17:05:57.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.367+0000 7fd0abb99740 1 -- 172.21.15.45:0/2830436658 --> [v2:172.21.15.45:6840/3755310822,v1:172.21.15.45:6841/3755310822] -- command(tid 0: {"prefix": "status"}) -- 0x55d17f6ad210 con 0x55d17f7c1420 2024-04-16T17:05:57.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.367+0000 7fd0a2800640 1 --2- 172.21.15.45:0/2830436658 >> [v2:172.21.15.45:6840/3755310822,v1:172.21.15.45:6841/3755310822] conn(0x55d17f7c1420 0x55d17f7c0130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:57.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.368+0000 7fd0a2800640 1 --2- 172.21.15.45:0/2830436658 >> [v2:172.21.15.45:6840/3755310822,v1:172.21.15.45:6841/3755310822] conn(0x55d17f7c1420 0x55d17f7c0130 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=39d456cd5fccfe85 server_cookie=68dd693c1cdbd327 in_seq=0 out_seq=0 2024-04-16T17:05:57.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.368+0000 7fd083e00640 10 client.15090 ms_handle_connect on v2:172.21.15.45:6840/3755310822 2024-04-16T17:05:57.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.368+0000 7fd083e00640 1 -- 172.21.15.45:0/2830436658 <== mds.1 v2:172.21.15.45:6840/3755310822 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55d17f6ad210 con 0x55d17f7c1420 2024-04-16T17:05:57.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.368+0000 7fd083e00640 10 client.15090 handle_command_reply: tid=0 2024-04-16T17:05:57.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.368+0000 7fd0abb99740 1 client.15090 shutdown 2024-04-16T17:05:57.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.368+0000 7fd0abb99740 1 -- 172.21.15.45:0/2830436658 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d17f73fe90 msgr2=0x55d17f74c9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:57.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.368+0000 7fd0abb99740 1 --2- 172.21.15.45:0/2830436658 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d17f73fe90 0x55d17f74c9e0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x55d17f74aed0 tx=0x7fd09000fcb0 comp rx=0 tx=0).stop 2024-04-16T17:05:57.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.368+0000 7fd0abb99740 1 -- 172.21.15.45:0/2830436658 shutdown_connections 2024-04-16T17:05:57.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.368+0000 7fd0abb99740 1 --2- 172.21.15.45:0/2830436658 >> [v2:172.21.15.45:6840/3755310822,v1:172.21.15.45:6841/3755310822] conn(0x55d17f7c1420 0x55d17f7c0130 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:57.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.368+0000 7fd0abb99740 1 --2- 172.21.15.45:0/2830436658 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d17f745000 0x55d17f74fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:57.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.368+0000 7fd0abb99740 1 --2- 172.21.15.45:0/2830436658 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d17f73fe90 0x55d17f74c9e0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:57.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.368+0000 7fd0abb99740 1 --2- 172.21.15.45:0/2830436658 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d17f73f4a0 0x55d17f74c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:57.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.368+0000 7fd083e00640 0 client.15090 ms_handle_reset on v2:172.21.15.45:6840/3755310822 2024-04-16T17:05:57.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.368+0000 7fd0abb99740 1 -- 172.21.15.45:0/2830436658 >> 172.21.15.45:0/2830436658 conn(0x55d17f73aa10 msgr2=0x55d17f741350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:05:57.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.368+0000 7fd0abb99740 1 -- 172.21.15.45:0/2830436658 shutdown_connections 2024-04-16T17:05:57.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.369+0000 7fd0abb99740 1 -- 172.21.15.45:0/2830436658 wait complete. 2024-04-16T17:05:57.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.369+0000 7fd0abb99740 20 client.15090 trim_cache size 0 max 16384 2024-04-16T17:05:57.372 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-16T17:05:57.372 INFO:teuthology.orchestra.run.smithi045.stdout: "cluster_fsid": "b1fecd38-fc11-11ee-bc90-c7b262605968", 2024-04-16T17:05:57.372 INFO:teuthology.orchestra.run.smithi045.stdout: "whoami": 1, 2024-04-16T17:05:57.372 INFO:teuthology.orchestra.run.smithi045.stdout: "id": 24569, 2024-04-16T17:05:57.372 INFO:teuthology.orchestra.run.smithi045.stdout: "want_state": "up:active", 2024-04-16T17:05:57.372 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "up:active", 2024-04-16T17:05:57.372 INFO:teuthology.orchestra.run.smithi045.stdout: "fs_name": "cephfs", 2024-04-16T17:05:57.373 INFO:teuthology.orchestra.run.smithi045.stdout: "rank_uptime": 126.008977917, 2024-04-16T17:05:57.373 INFO:teuthology.orchestra.run.smithi045.stdout: "mdsmap_epoch": 24, 2024-04-16T17:05:57.373 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_epoch": 94, 2024-04-16T17:05:57.373 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_epoch_barrier": 90, 2024-04-16T17:05:57.373 INFO:teuthology.orchestra.run.smithi045.stdout: "uptime": 161.66930283900001 2024-04-16T17:05:57.373 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-16T17:05:57.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.369+0000 7fd0a3200640 1 -- 172.21.15.45:0/2298784065 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fd0880a3c30 msgr2=0x7fd0880a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:57.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.369+0000 7fd0a3200640 1 --2- 172.21.15.45:0/2298784065 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fd0880a3c30 0x7fd0880a60f0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fd094009bf0 tx=0x7fd0940f6780 comp rx=0 tx=0).stop 2024-04-16T17:05:57.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.369+0000 7fd0a3200640 1 -- 172.21.15.45:0/2298784065 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd0a4104140 msgr2=0x7fd0a419a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:57.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.369+0000 7fd0a3200640 1 --2- 172.21.15.45:0/2298784065 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd0a4104140 0x7fd0a419a460 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fd094000f80 tx=0x7fd094035d10 comp rx=0 tx=0).stop 2024-04-16T17:05:57.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.370+0000 7fd0a3200640 1 -- 172.21.15.45:0/2298784065 shutdown_connections 2024-04-16T17:05:57.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.370+0000 7fd0a3200640 1 --2- 172.21.15.45:0/2298784065 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fd0880a3c30 0x7fd0880a60f0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:57.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.370+0000 7fd0a3200640 1 --2- 172.21.15.45:0/2298784065 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd0a4105330 0x7fd0a419a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:57.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.370+0000 7fd0a3200640 1 --2- 172.21.15.45:0/2298784065 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd0a4105cf0 0x7fd0a41a1a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:57.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.370+0000 7fd0a3200640 1 --2- 172.21.15.45:0/2298784065 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd0a4104140 0x7fd0a419a460 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:57.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.370+0000 7fd0a3200640 1 -- 172.21.15.45:0/2298784065 >> 172.21.15.45:0/2298784065 conn(0x7fd0a40ff6b0 msgr2=0x7fd0a4108630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:05:57.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.370+0000 7fd0a3200640 1 -- 172.21.15.45:0/2298784065 shutdown_connections 2024-04-16T17:05:57.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.370+0000 7fd0a3200640 1 -- 172.21.15.45:0/2298784065 wait complete. 2024-04-16T17:05:57.389 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-16T17:05:57.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.616+0000 7f03ec000640 1 -- 172.21.15.45:0/2307902343 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f03e4077c00 msgr2=0x7f03e4078060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:57.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.616+0000 7f03ec000640 1 --2- 172.21.15.45:0/2307902343 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f03e4077c00 0x7f03e4078060 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f03d0009900 tx=0x7f03d002bc30 comp rx=0 tx=0).stop 2024-04-16T17:05:57.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.616+0000 7f03ec000640 1 -- 172.21.15.45:0/2307902343 shutdown_connections 2024-04-16T17:05:57.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.616+0000 7f03ec000640 1 --2- 172.21.15.45:0/2307902343 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f03e40785a0 0x7f03e4078e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:57.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.616+0000 7f03ec000640 1 --2- 172.21.15.45:0/2307902343 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f03e4077c00 0x7f03e4078060 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:57.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.616+0000 7f03ec000640 1 --2- 172.21.15.45:0/2307902343 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f03e40769c0 0x7f03e4076de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:57.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.616+0000 7f03ec000640 1 -- 172.21.15.45:0/2307902343 >> 172.21.15.45:0/2307902343 conn(0x7f03e40ff7b0 msgr2=0x7f03e4101c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:05:57.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.617+0000 7f03ec000640 1 -- 172.21.15.45:0/2307902343 shutdown_connections 2024-04-16T17:05:57.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.617+0000 7f03ec000640 1 -- 172.21.15.45:0/2307902343 wait complete. 2024-04-16T17:05:57.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.618+0000 7f03ec000640 1 Processor -- start 2024-04-16T17:05:57.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.618+0000 7f03ec000640 1 -- start start 2024-04-16T17:05:57.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.618+0000 7f03ec000640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f03e40769c0 0x7f03e419e980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:57.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.618+0000 7f03ec000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f03e4077c00 0x7f03e419eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:57.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.618+0000 7f03ec000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f03e40785a0 0x7f03e41a5f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:57.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.618+0000 7f03ec000640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f03e41142f0 con 0x7f03e4077c00 2024-04-16T17:05:57.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.618+0000 7f03ec000640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f03e4114170 con 0x7f03e40769c0 2024-04-16T17:05:57.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.618+0000 7f03ec000640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f03e4114470 con 0x7f03e40785a0 2024-04-16T17:05:57.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.619+0000 7f03e8c00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f03e4077c00 0x7f03e419eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:57.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.619+0000 7f03e8c00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f03e4077c00 0x7f03e419eec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39602/0 (socket says 172.21.15.45:39602) 2024-04-16T17:05:57.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.619+0000 7f03e8c00640 1 -- 172.21.15.45:0/926957293 learned_addr learned my addr 172.21.15.45:0/926957293 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:05:57.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.619+0000 7f03ea000640 1 --2- 172.21.15.45:0/926957293 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f03e40785a0 0x7f03e41a5f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:57.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.619+0000 7f03e9600640 1 --2- 172.21.15.45:0/926957293 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f03e40769c0 0x7f03e419e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:57.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.619+0000 7f03e8c00640 1 -- 172.21.15.45:0/926957293 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f03e40785a0 msgr2=0x7f03e41a5f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:57.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.619+0000 7f03e8c00640 1 --2- 172.21.15.45:0/926957293 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f03e40785a0 0x7f03e41a5f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:57.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.619+0000 7f03e8c00640 1 -- 172.21.15.45:0/926957293 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f03e40769c0 msgr2=0x7f03e419e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:57.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.619+0000 7f03e8c00640 1 --2- 172.21.15.45:0/926957293 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f03e40769c0 0x7f03e419e980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:57.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.619+0000 7f03e8c00640 1 -- 172.21.15.45:0/926957293 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03e41a65a0 con 0x7f03e4077c00 2024-04-16T17:05:57.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.620+0000 7f03e8c00640 1 --2- 172.21.15.45:0/926957293 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f03e4077c00 0x7f03e419eec0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f03d0009230 tx=0x7f03d0005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:05:57.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.620+0000 7f03e2000640 1 -- 172.21.15.45:0/926957293 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03d0041070 con 0x7f03e4077c00 2024-04-16T17:05:57.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.620+0000 7f03e2000640 1 -- 172.21.15.45:0/926957293 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f03d00044d0 con 0x7f03e4077c00 2024-04-16T17:05:57.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.620+0000 7f03ec000640 1 -- 172.21.15.45:0/926957293 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03e41a6830 con 0x7f03e4077c00 2024-04-16T17:05:57.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.620+0000 7f03e2000640 1 -- 172.21.15.45:0/926957293 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03d0034440 con 0x7f03e4077c00 2024-04-16T17:05:57.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.620+0000 7f03ec000640 1 -- 172.21.15.45:0/926957293 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03e41a6d20 con 0x7f03e4077c00 2024-04-16T17:05:57.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.624+0000 7f03eedd0740 1 Processor -- start 2024-04-16T17:05:57.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.624+0000 7f03e2000640 1 -- 172.21.15.45:0/926957293 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f03d0044070 con 0x7f03e4077c00 2024-04-16T17:05:57.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.624+0000 7f03eedd0740 1 -- start start 2024-04-16T17:05:57.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.624+0000 7f03eedd0740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a617ea02d0 0x55a617ea06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:57.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.624+0000 7f03eedd0740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a617ea0cc0 0x55a617ea57c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:57.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.624+0000 7f03eedd0740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a617ea5e30 0x55a617ea82c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:57.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.624+0000 7f03eedd0740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55a617e06980 con 0x55a617ea5e30 2024-04-16T17:05:57.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.624+0000 7f03eedd0740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55a617dad220 con 0x55a617ea02d0 2024-04-16T17:05:57.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.624+0000 7f03eedd0740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55a617e02940 con 0x55a617ea0cc0 2024-04-16T17:05:57.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.624+0000 7f03ea000640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a617ea02d0 0x55a617ea06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:57.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.624+0000 7f03ea000640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a617ea02d0 0x55a617ea06f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:52002/0 (socket says 172.21.15.45:52002) 2024-04-16T17:05:57.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.624+0000 7f03ea000640 1 -- 172.21.15.45:0/1929027481 learned_addr learned my addr 172.21.15.45:0/1929027481 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:05:57.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.624+0000 7f03e9600640 1 --2- 172.21.15.45:0/1929027481 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a617ea0cc0 0x55a617ea57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:57.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.625+0000 7f03e8c00640 1 --2- 172.21.15.45:0/1929027481 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a617ea5e30 0x55a617ea82c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:57.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.625+0000 7f03ea000640 1 -- 172.21.15.45:0/1929027481 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a617ea0cc0 msgr2=0x55a617ea57c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:05:57.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.625+0000 7f03ea000640 1 --2- 172.21.15.45:0/1929027481 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a617ea0cc0 0x55a617ea57c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:57.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.625+0000 7f03ea000640 1 -- 172.21.15.45:0/1929027481 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a617ea5e30 msgr2=0x55a617ea82c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:05:57.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.625+0000 7f03ea000640 1 --2- 172.21.15.45:0/1929027481 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a617ea5e30 0x55a617ea82c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:57.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.625+0000 7f03ea000640 1 -- 172.21.15.45:0/1929027481 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a617b192d0 con 0x55a617ea02d0 2024-04-16T17:05:57.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.625+0000 7f03ea000640 1 --2- 172.21.15.45:0/1929027481 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a617ea02d0 0x55a617ea06f0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f03dc009fd0 tx=0x7f03dc0118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7b04288a2ff45df7 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:05:57.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.625+0000 7f03d6600640 1 -- 172.21.15.45:0/1929027481 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03dc004d20 con 0x55a617ea02d0 2024-04-16T17:05:57.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.625+0000 7f03e2000640 1 --2- 172.21.15.45:0/926957293 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f03cc0a3bb0 0x7f03cc0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:57.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.626+0000 7f03d6600640 1 -- 172.21.15.45:0/1929027481 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f03dc004050 con 0x55a617ea02d0 2024-04-16T17:05:57.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.626+0000 7f03d6600640 1 -- 172.21.15.45:0/1929027481 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03dc005260 con 0x55a617ea02d0 2024-04-16T17:05:57.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.626+0000 7f03e2000640 1 -- 172.21.15.45:0/926957293 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f03d00f2350 con 0x7f03e4077c00 2024-04-16T17:05:57.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.626+0000 7f03e9600640 1 --2- 172.21.15.45:0/926957293 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f03cc0a3bb0 0x7f03cc0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:57.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.626+0000 7f03eedd0740 1 -- 172.21.15.45:0/1929027481 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a617ea02d0 msgr2=0x55a617ea06f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:57.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.626+0000 7f03eedd0740 1 --2- 172.21.15.45:0/1929027481 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a617ea02d0 0x55a617ea06f0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f03dc009fd0 tx=0x7f03dc0118d0 comp rx=0 tx=0).stop 2024-04-16T17:05:57.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.627+0000 7f03eedd0740 1 -- 172.21.15.45:0/1929027481 shutdown_connections 2024-04-16T17:05:57.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.627+0000 7f03eedd0740 1 --2- 172.21.15.45:0/1929027481 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a617ea0cc0 0x55a617ea57c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:57.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.627+0000 7f03eedd0740 1 --2- 172.21.15.45:0/1929027481 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a617ea5e30 0x55a617ea82c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:57.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.627+0000 7f03eedd0740 1 --2- 172.21.15.45:0/1929027481 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a617ea02d0 0x55a617ea06f0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:57.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.627+0000 7f03eedd0740 1 -- 172.21.15.45:0/1929027481 >> 172.21.15.45:0/1929027481 conn(0x55a617e9b840 msgr2=0x55a617e9bc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:05:57.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.627+0000 7f03e9600640 1 --2- 172.21.15.45:0/926957293 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f03cc0a3bb0 0x7f03cc0a6070 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x55a617ea10c0 tx=0x7f03d800b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:05:57.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.627+0000 7f03eedd0740 1 -- 172.21.15.45:0/1929027481 shutdown_connections 2024-04-16T17:05:57.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.627+0000 7f03eedd0740 1 -- 172.21.15.45:0/1929027481 wait complete. 2024-04-16T17:05:57.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.627+0000 7f03eedd0740 1 -- start start 2024-04-16T17:05:57.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.627+0000 7f03eedd0740 1 Processor -- start 2024-04-16T17:05:57.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.628+0000 7f03eedd0740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a617ea02d0 0x55a617eaa8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:57.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.628+0000 7f03eedd0740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a617ea0cc0 0x55a617eaade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:57.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.628+0000 7f03eedd0740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a617ea5e30 0x55a617eb1e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:57.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.628+0000 7f03eedd0740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55a617ea8de0 con 0x55a617ea02d0 2024-04-16T17:05:57.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.628+0000 7f03eedd0740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55a617dc0590 con 0x55a617ea0cc0 2024-04-16T17:05:57.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.628+0000 7f03eedd0740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55a617ea8f60 con 0x55a617ea5e30 2024-04-16T17:05:57.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.628+0000 7f03eedd0740 10 client.0 _ll_register_callbacks cb 0x55a617dc8050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:05:57.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.628+0000 7f03e9600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a617ea02d0 0x55a617eaa8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:57.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.628+0000 7f03e9600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a617ea02d0 0x55a617eaa8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39614/0 (socket says 172.21.15.45:39614) 2024-04-16T17:05:57.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.628+0000 7f03e9600640 1 -- 172.21.15.45:0/80020341 learned_addr learned my addr 172.21.15.45:0/80020341 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:05:57.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.628+0000 7f03e8c00640 1 --2- 172.21.15.45:0/80020341 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a617ea0cc0 0x55a617eaade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:57.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.628+0000 7f03ea000640 1 --2- 172.21.15.45:0/80020341 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a617ea5e30 0x55a617eb1e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:57.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.629+0000 7f03e9600640 1 -- 172.21.15.45:0/80020341 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a617ea5e30 msgr2=0x55a617eb1e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:05:57.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.629+0000 7f03e9600640 1 --2- 172.21.15.45:0/80020341 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a617ea5e30 0x55a617eb1e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:57.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.629+0000 7f03e9600640 1 -- 172.21.15.45:0/80020341 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a617ea0cc0 msgr2=0x55a617eaade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:05:57.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.629+0000 7f03e9600640 1 --2- 172.21.15.45:0/80020341 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a617ea0cc0 0x55a617eaade0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:57.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.629+0000 7f03e9600640 1 -- 172.21.15.45:0/80020341 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a617be36b0 con 0x55a617ea02d0 2024-04-16T17:05:57.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.629+0000 7f03e8c00640 1 --2- 172.21.15.45:0/80020341 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a617ea0cc0 0x55a617eaade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:05:57.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.629+0000 7f03e9600640 1 --2- 172.21.15.45:0/80020341 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a617ea02d0 0x55a617eaa8a0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f03d8004650 tx=0x7f03d8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f97855353066d3ff server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:05:57.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.629+0000 7f03caa00640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-16T17:05:57.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.629+0000 7f03caa00640 1 -- 172.21.15.45:0/80020341 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03d802c060 con 0x55a617ea02d0 2024-04-16T17:05:57.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.629+0000 7f03caa00640 1 -- 172.21.15.45:0/80020341 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f03d8017030 con 0x55a617ea02d0 2024-04-16T17:05:57.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.629+0000 7f03eedd0740 1 -- 172.21.15.45:0/80020341 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f03dc004050 con 0x55a617ea02d0 2024-04-16T17:05:57.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.630+0000 7f03caa00640 1 -- 172.21.15.45:0/80020341 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f03d8033050 con 0x55a617ea02d0 2024-04-16T17:05:57.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.630+0000 7f03eedd0740 10 client.15102 fetch_fsmap learned FSMap version 24 2024-04-16T17:05:57.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.630+0000 7f03eedd0740 1 -- 172.21.15.45:0/80020341 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55a617eb5260 con 0x55a617ea02d0 2024-04-16T17:05:57.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.630+0000 7f03caa00640 1 -- 172.21.15.45:0/80020341 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f03dc004050 con 0x55a617ea02d0 2024-04-16T17:05:57.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.630+0000 7f03caa00640 1 -- 172.21.15.45:0/80020341 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f03d801c3f0 con 0x55a617ea02d0 2024-04-16T17:05:57.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.630+0000 7f03eedd0740 10 client.15102 fetch_fsmap finished waiting for FSMap version 24 2024-04-16T17:05:57.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.630+0000 7f03eedd0740 10 client.15102 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-16T17:05:57.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.630+0000 7f03eedd0740 20 client.15102 populate_metadata read hostname 'smithi045' 2024-04-16T17:05:57.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.630+0000 7f03eedd0740 1 --2- 172.21.15.45:0/80020341 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55a617eb78f0 0x55a617eb8d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:57.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.630+0000 7f03eedd0740 4 client.15102 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:05:57.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.630+0000 7f03eedd0740 1 -- 172.21.15.45:0/80020341 --> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a617e143a0 con 0x55a617eb78f0 2024-04-16T17:05:57.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.631+0000 7f03e8c00640 1 --2- 172.21.15.45:0/80020341 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55a617eb78f0 0x55a617eb8d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:57.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.631+0000 7f03e8c00640 1 --2- 172.21.15.45:0/80020341 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55a617eb78f0 0x55a617eb8d60 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f6a5c3361ed52da server_cookie=5ba9778ab72a13cd in_seq=0 out_seq=0 2024-04-16T17:05:57.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.631+0000 7f03caa00640 10 client.15102 ms_handle_connect on v2:172.21.15.104:6838/3660070592 2024-04-16T17:05:57.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.632+0000 7f03caa00640 1 -- 172.21.15.45:0/80020341 <== mds.0 v2:172.21.15.104:6838/3660070592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a617e143a0 con 0x55a617eb78f0 2024-04-16T17:05:57.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.632+0000 7f03caa00640 10 client.15102 handle_command_reply: tid=0 2024-04-16T17:05:57.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.632+0000 7f03eedd0740 1 client.15102 shutdown 2024-04-16T17:05:57.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.632+0000 7f03eedd0740 1 -- 172.21.15.45:0/80020341 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a617ea02d0 msgr2=0x55a617eaa8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:57.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.632+0000 7f03eedd0740 1 --2- 172.21.15.45:0/80020341 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a617ea02d0 0x55a617eaa8a0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f03d8004650 tx=0x7f03d8004730 comp rx=0 tx=0).stop 2024-04-16T17:05:57.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.632+0000 7f03eedd0740 1 -- 172.21.15.45:0/80020341 shutdown_connections 2024-04-16T17:05:57.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.632+0000 7f03eedd0740 1 --2- 172.21.15.45:0/80020341 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55a617eb78f0 0x55a617eb8d60 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:57.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.632+0000 7f03eedd0740 1 --2- 172.21.15.45:0/80020341 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a617ea5e30 0x55a617eb1e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:57.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.632+0000 7f03eedd0740 1 --2- 172.21.15.45:0/80020341 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a617ea0cc0 0x55a617eaade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:57.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.632+0000 7f03eedd0740 1 --2- 172.21.15.45:0/80020341 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a617ea02d0 0x55a617eaa8a0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:57.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.632+0000 7f03eedd0740 1 -- 172.21.15.45:0/80020341 >> 172.21.15.45:0/80020341 conn(0x55a617e9b840 msgr2=0x55a617e9aa80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:05:57.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.632+0000 7f03caa00640 0 client.15102 ms_handle_reset on v2:172.21.15.104:6838/3660070592 2024-04-16T17:05:57.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.633+0000 7f03eedd0740 1 -- 172.21.15.45:0/80020341 shutdown_connections 2024-04-16T17:05:57.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.633+0000 7f03eedd0740 1 -- 172.21.15.45:0/80020341 wait complete. 2024-04-16T17:05:57.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.633+0000 7f03eedd0740 20 client.15102 trim_cache size 0 max 16384 2024-04-16T17:05:57.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.653+0000 7f03eedd0740 1 Processor -- start 2024-04-16T17:05:57.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.653+0000 7f03eedd0740 1 -- start start 2024-04-16T17:05:57.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.653+0000 7f03eedd0740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a617e9b840 0x55a617eb8690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:57.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.653+0000 7f03eedd0740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a617ea0cc0 0x55a617eb1db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:57.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.653+0000 7f03eedd0740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a617ea5e30 0x55a617ea21c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:57.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.653+0000 7f03eedd0740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55a617eb70a0 con 0x55a617e9b840 2024-04-16T17:05:57.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.653+0000 7f03eedd0740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55a617eb7770 con 0x55a617ea5e30 2024-04-16T17:05:57.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.653+0000 7f03eedd0740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55a617eb7e40 con 0x55a617ea0cc0 2024-04-16T17:05:57.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.654+0000 7f03e8c00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a617ea0cc0 0x55a617eb1db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:57.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.654+0000 7f03e8c00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a617ea0cc0 0x55a617eb1db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:60266/0 (socket says 172.21.15.45:60266) 2024-04-16T17:05:57.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.654+0000 7f03e8c00640 1 -- 172.21.15.45:0/2120442989 learned_addr learned my addr 172.21.15.45:0/2120442989 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:05:57.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.654+0000 7f03ea000640 1 --2- 172.21.15.45:0/2120442989 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a617ea5e30 0x55a617ea21c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:57.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.654+0000 7f03e9600640 1 --2- 172.21.15.45:0/2120442989 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a617e9b840 0x55a617eb8690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:57.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.654+0000 7f03e8c00640 1 -- 172.21.15.45:0/2120442989 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a617ea5e30 msgr2=0x55a617ea21c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:05:57.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.654+0000 7f03e8c00640 1 --2- 172.21.15.45:0/2120442989 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a617ea5e30 0x55a617ea21c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:57.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.654+0000 7f03e8c00640 1 -- 172.21.15.45:0/2120442989 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a617e9b840 msgr2=0x55a617eb8690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:05:57.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.654+0000 7f03e8c00640 1 --2- 172.21.15.45:0/2120442989 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a617e9b840 0x55a617eb8690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:57.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.654+0000 7f03e8c00640 1 -- 172.21.15.45:0/2120442989 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a617eb32a0 con 0x55a617ea0cc0 2024-04-16T17:05:57.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.654+0000 7f03e8c00640 1 --2- 172.21.15.45:0/2120442989 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a617ea0cc0 0x55a617eb1db0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f03d004c000 tx=0x7f03d002ce80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ba34681b246ef6a4 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:05:57.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.655+0000 7f03d5c00640 1 -- 172.21.15.45:0/2120442989 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03d004b040 con 0x55a617ea0cc0 2024-04-16T17:05:57.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.655+0000 7f03d5c00640 1 -- 172.21.15.45:0/2120442989 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f03d00be350 con 0x55a617ea0cc0 2024-04-16T17:05:57.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.655+0000 7f03d5c00640 1 -- 172.21.15.45:0/2120442989 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03d00be5b0 con 0x55a617ea0cc0 2024-04-16T17:05:57.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.655+0000 7f03eedd0740 1 -- 172.21.15.45:0/2120442989 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a617ea0cc0 msgr2=0x55a617eb1db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:57.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.655+0000 7f03eedd0740 1 --2- 172.21.15.45:0/2120442989 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a617ea0cc0 0x55a617eb1db0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f03d004c000 tx=0x7f03d002ce80 comp rx=0 tx=0).stop 2024-04-16T17:05:57.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.655+0000 7f03eedd0740 1 -- 172.21.15.45:0/2120442989 shutdown_connections 2024-04-16T17:05:57.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.655+0000 7f03eedd0740 1 --2- 172.21.15.45:0/2120442989 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a617ea0cc0 0x55a617eb1db0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:57.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.655+0000 7f03eedd0740 1 --2- 172.21.15.45:0/2120442989 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a617ea5e30 0x55a617ea21c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:57.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.655+0000 7f03eedd0740 1 --2- 172.21.15.45:0/2120442989 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a617e9b840 0x55a617eb8690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:57.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.655+0000 7f03eedd0740 1 -- 172.21.15.45:0/2120442989 >> 172.21.15.45:0/2120442989 conn(0x55a617ea1aa0 msgr2=0x55a617ea9390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:05:57.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.656+0000 7f03eedd0740 1 -- 172.21.15.45:0/2120442989 shutdown_connections 2024-04-16T17:05:57.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.656+0000 7f03eedd0740 1 -- 172.21.15.45:0/2120442989 wait complete. 2024-04-16T17:05:57.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.656+0000 7f03eedd0740 1 -- start start 2024-04-16T17:05:57.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.656+0000 7f03eedd0740 1 Processor -- start 2024-04-16T17:05:57.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.656+0000 7f03eedd0740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a617e9b840 0x55a617eacf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:57.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.656+0000 7f03eedd0740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a617ea0cc0 0x55a617ead4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:57.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.657+0000 7f03eedd0740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a617ea5e30 0x55a617eb04c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:57.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.657+0000 7f03eedd0740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55a617eb8be0 con 0x55a617ea5e30 2024-04-16T17:05:57.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.657+0000 7f03eedd0740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55a617eb8510 con 0x55a617e9b840 2024-04-16T17:05:57.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.657+0000 7f03eedd0740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55a617eb92b0 con 0x55a617ea0cc0 2024-04-16T17:05:57.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.657+0000 7f03eedd0740 10 client.0 _ll_register_callbacks cb 0x7f03d8005cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:05:57.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.657+0000 7f03ea000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a617ea5e30 0x55a617eb04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:57.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.657+0000 7f03e9600640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a617e9b840 0x55a617eacf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:57.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.657+0000 7f03e9600640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a617e9b840 0x55a617eacf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:52026/0 (socket says 172.21.15.45:52026) 2024-04-16T17:05:57.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.657+0000 7f03e9600640 1 -- 172.21.15.45:0/3361809341 learned_addr learned my addr 172.21.15.45:0/3361809341 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:05:57.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.657+0000 7f03e8c00640 1 --2- 172.21.15.45:0/3361809341 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a617ea0cc0 0x55a617ead4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:57.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.657+0000 7f03e9600640 1 -- 172.21.15.45:0/3361809341 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a617ea0cc0 msgr2=0x55a617ead4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:05:57.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.657+0000 7f03e9600640 1 --2- 172.21.15.45:0/3361809341 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a617ea0cc0 0x55a617ead4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:57.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.657+0000 7f03e9600640 1 -- 172.21.15.45:0/3361809341 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a617ea5e30 msgr2=0x55a617eb04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:05:57.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.657+0000 7f03e9600640 1 --2- 172.21.15.45:0/3361809341 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a617ea5e30 0x55a617eb04c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:57.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.657+0000 7f03e9600640 1 -- 172.21.15.45:0/3361809341 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a617eb0ab0 con 0x55a617e9b840 2024-04-16T17:05:57.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.657+0000 7f03e9600640 1 --2- 172.21.15.45:0/3361809341 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a617e9b840 0x55a617eacf60 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f03d8005d80 tx=0x7f03d801cd30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7985e18d2320604d server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:05:57.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.658+0000 7f03caa00640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-04-16T17:05:57.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.658+0000 7f03caa00640 1 -- 172.21.15.45:0/3361809341 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03d801fdd0 con 0x55a617e9b840 2024-04-16T17:05:57.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.658+0000 7f03caa00640 1 -- 172.21.15.45:0/3361809341 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f03d8005040 con 0x55a617e9b840 2024-04-16T17:05:57.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.658+0000 7f03eedd0740 1 -- 172.21.15.45:0/3361809341 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f03d00be350 con 0x55a617e9b840 2024-04-16T17:05:57.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.658+0000 7f03caa00640 1 -- 172.21.15.45:0/3361809341 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f03d8003bc0 con 0x55a617e9b840 2024-04-16T17:05:57.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.658+0000 7f03eedd0740 10 client.25220 fetch_fsmap learned FSMap version 24 2024-04-16T17:05:57.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.658+0000 7f03eedd0740 1 -- 172.21.15.45:0/3361809341 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55a617eb1e20 con 0x55a617e9b840 2024-04-16T17:05:57.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.659+0000 7f03caa00640 1 -- 172.21.15.45:0/3361809341 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f03d00be350 con 0x55a617e9b840 2024-04-16T17:05:57.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.659+0000 7f03caa00640 1 -- 172.21.15.45:0/3361809341 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f03d8004be0 con 0x55a617e9b840 2024-04-16T17:05:57.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.659+0000 7f03eedd0740 10 client.25220 fetch_fsmap finished waiting for FSMap version 24 2024-04-16T17:05:57.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.659+0000 7f03eedd0740 10 client.25220 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-16T17:05:57.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.659+0000 7f03eedd0740 20 client.25220 populate_metadata read hostname 'smithi045' 2024-04-16T17:05:57.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.659+0000 7f03eedd0740 1 --2- 172.21.15.45:0/3361809341 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55a617f21f90 0x55a617f20ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:05:57.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.659+0000 7f03eedd0740 4 client.25220 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-16T17:05:57.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.659+0000 7f03eedd0740 1 -- 172.21.15.45:0/3361809341 --> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] -- command(tid 0: {"prefix": "status"}) -- 0x55a617e0e210 con 0x55a617f21f90 2024-04-16T17:05:57.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.659+0000 7f03e8c00640 1 --2- 172.21.15.45:0/3361809341 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55a617f21f90 0x55a617f20ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:05:57.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.660+0000 7f03e8c00640 1 --2- 172.21.15.45:0/3361809341 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55a617f21f90 0x55a617f20ca0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1a1d21ae74a86461 server_cookie=b757226115ca5c4a in_seq=0 out_seq=0 2024-04-16T17:05:57.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.660+0000 7f03caa00640 10 client.25220 ms_handle_connect on v2:172.21.15.104:6838/3660070592 2024-04-16T17:05:57.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.660+0000 7f03caa00640 1 -- 172.21.15.45:0/3361809341 <== mds.0 v2:172.21.15.104:6838/3660070592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55a617e0e210 con 0x55a617f21f90 2024-04-16T17:05:57.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.660+0000 7f03caa00640 10 client.25220 handle_command_reply: tid=0 2024-04-16T17:05:57.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.660+0000 7f03eedd0740 1 client.25220 shutdown 2024-04-16T17:05:57.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.660+0000 7f03eedd0740 1 -- 172.21.15.45:0/3361809341 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a617e9b840 msgr2=0x55a617eacf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:57.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.660+0000 7f03eedd0740 1 --2- 172.21.15.45:0/3361809341 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a617e9b840 0x55a617eacf60 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f03d8005d80 tx=0x7f03d801cd30 comp rx=0 tx=0).stop 2024-04-16T17:05:57.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.660+0000 7f03eedd0740 1 -- 172.21.15.45:0/3361809341 shutdown_connections 2024-04-16T17:05:57.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.660+0000 7f03eedd0740 1 --2- 172.21.15.45:0/3361809341 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55a617f21f90 0x55a617f20ca0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:57.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.660+0000 7f03eedd0740 1 --2- 172.21.15.45:0/3361809341 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a617ea0cc0 0x55a617ead4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:57.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.660+0000 7f03eedd0740 1 --2- 172.21.15.45:0/3361809341 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a617ea5e30 0x55a617eb04c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:57.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.660+0000 7f03eedd0740 1 --2- 172.21.15.45:0/3361809341 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a617e9b840 0x55a617eacf60 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:57.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.660+0000 7f03eedd0740 1 -- 172.21.15.45:0/3361809341 >> 172.21.15.45:0/3361809341 conn(0x55a617ea1aa0 msgr2=0x55a617ea21c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:05:57.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.660+0000 7f03caa00640 0 client.25220 ms_handle_reset on v2:172.21.15.104:6838/3660070592 2024-04-16T17:05:57.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.660+0000 7f03eedd0740 1 -- 172.21.15.45:0/3361809341 shutdown_connections 2024-04-16T17:05:57.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.661+0000 7f03eedd0740 1 -- 172.21.15.45:0/3361809341 wait complete. 2024-04-16T17:05:57.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.661+0000 7f03eedd0740 20 client.25220 trim_cache size 0 max 16384 2024-04-16T17:05:57.665 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-16T17:05:57.665 INFO:teuthology.orchestra.run.smithi045.stdout: "cluster_fsid": "b1fecd38-fc11-11ee-bc90-c7b262605968", 2024-04-16T17:05:57.665 INFO:teuthology.orchestra.run.smithi045.stdout: "whoami": 0, 2024-04-16T17:05:57.665 INFO:teuthology.orchestra.run.smithi045.stdout: "id": 24581, 2024-04-16T17:05:57.665 INFO:teuthology.orchestra.run.smithi045.stdout: "want_state": "up:active", 2024-04-16T17:05:57.665 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "up:active", 2024-04-16T17:05:57.665 INFO:teuthology.orchestra.run.smithi045.stdout: "fs_name": "cephfs", 2024-04-16T17:05:57.665 INFO:teuthology.orchestra.run.smithi045.stdout: "rank_uptime": 152.49603347499999, 2024-04-16T17:05:57.665 INFO:teuthology.orchestra.run.smithi045.stdout: "mdsmap_epoch": 24, 2024-04-16T17:05:57.665 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_epoch": 94, 2024-04-16T17:05:57.665 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_epoch_barrier": 90, 2024-04-16T17:05:57.666 INFO:teuthology.orchestra.run.smithi045.stdout: "uptime": 159.161129781 2024-04-16T17:05:57.666 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-16T17:05:57.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.661+0000 7f03ec000640 1 -- 172.21.15.45:0/926957293 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f03cc0a3bb0 msgr2=0x7f03cc0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:57.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.661+0000 7f03ec000640 1 --2- 172.21.15.45:0/926957293 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f03cc0a3bb0 0x7f03cc0a6070 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x55a617ea10c0 tx=0x7f03d800b390 comp rx=0 tx=0).stop 2024-04-16T17:05:57.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.661+0000 7f03ec000640 1 -- 172.21.15.45:0/926957293 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f03e4077c00 msgr2=0x7f03e419eec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:05:57.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.661+0000 7f03ec000640 1 --2- 172.21.15.45:0/926957293 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f03e4077c00 0x7f03e419eec0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f03d0009230 tx=0x7f03d0005d50 comp rx=0 tx=0).stop 2024-04-16T17:05:57.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.661+0000 7f03ec000640 1 -- 172.21.15.45:0/926957293 shutdown_connections 2024-04-16T17:05:57.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.661+0000 7f03ec000640 1 --2- 172.21.15.45:0/926957293 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f03cc0a3bb0 0x7f03cc0a6070 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:57.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.661+0000 7f03ec000640 1 --2- 172.21.15.45:0/926957293 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f03e40785a0 0x7f03e41a5f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:57.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.661+0000 7f03ec000640 1 --2- 172.21.15.45:0/926957293 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f03e4077c00 0x7f03e419eec0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:57.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.661+0000 7f03ec000640 1 --2- 172.21.15.45:0/926957293 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f03e40769c0 0x7f03e419e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:05:57.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.661+0000 7f03ec000640 1 -- 172.21.15.45:0/926957293 >> 172.21.15.45:0/926957293 conn(0x7f03e40ff7b0 msgr2=0x7f03e4100f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:05:57.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.662+0000 7f03ec000640 1 -- 172.21.15.45:0/926957293 shutdown_connections 2024-04-16T17:05:57.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:05:57.662+0000 7f03ec000640 1 -- 172.21.15.45:0/926957293 wait complete. 2024-04-16T17:05:57.678 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:05:57 smithi045 ceph-mon[31676]: pgmap v402: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-16T17:05:57.678 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:05:57 smithi045 ceph-mon[31676]: from='client.? 172.21.15.45:0/3283115974' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:05:57.682 INFO:tasks.cephadm:Setup complete, yielding 2024-04-16T17:05:57.682 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-16T17:05:57.691 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi045.front.sepia.ceph.com 2024-04-16T17:05:57.691 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b1fecd38-fc11-11ee-bc90-c7b262605968 -- bash -c 'ceph orch status' 2024-04-16T17:05:57.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:05:57 smithi117 ceph-mon[36730]: pgmap v402: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-16T17:05:57.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:05:57 smithi117 ceph-mon[36730]: from='client.? 172.21.15.45:0/3283115974' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:05:57.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:05:57 smithi104 ceph-mon[36457]: pgmap v402: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-16T17:05:57.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:05:57 smithi104 ceph-mon[36457]: from='client.? 172.21.15.45:0/3283115974' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:05:58.170 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/mon.a/config 2024-04-16T17:05:59.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:05:59 smithi045 ceph-mon[31676]: pgmap v403: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-16T17:05:59.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:05:59 smithi117 ceph-mon[36730]: pgmap v403: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-16T17:05:59.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:05:59 smithi104 ceph-mon[36457]: pgmap v403: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-16T17:06:00.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:00.689+0000 7f2508400700 1 -- 172.21.15.45:0/2959293070 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2500108840 msgr2=0x7f250010acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:00.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:00.690+0000 7f2508400700 1 --2- 172.21.15.45:0/2959293070 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2500108840 0x7f250010acb0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f24f800b0a0 tx=0x7f24f800b3b0 comp rx=0 tx=0).stop 2024-04-16T17:06:00.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:00.690+0000 7f2508400700 1 -- 172.21.15.45:0/2959293070 shutdown_connections 2024-04-16T17:06:00.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:00.690+0000 7f2508400700 1 --2- 172.21.15.45:0/2959293070 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f25001035e0 0x7f25001059d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:00.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:00.690+0000 7f2508400700 1 --2- 172.21.15.45:0/2959293070 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2500108840 0x7f250010acb0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:00.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:00.690+0000 7f2508400700 1 --2- 172.21.15.45:0/2959293070 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2500105f10 0x7f2500108300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:00.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:00.690+0000 7f2508400700 1 -- 172.21.15.45:0/2959293070 >> 172.21.15.45:0/2959293070 conn(0x7f25000fcff0 msgr2=0x7f25000ff430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:06:00.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:00.690+0000 7f2508400700 1 -- 172.21.15.45:0/2959293070 shutdown_connections 2024-04-16T17:06:00.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:00.690+0000 7f2508400700 1 -- 172.21.15.45:0/2959293070 wait complete. 2024-04-16T17:06:00.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:00.691+0000 7f2508400700 1 Processor -- start 2024-04-16T17:06:00.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:00.691+0000 7f2508400700 1 -- start start 2024-04-16T17:06:00.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:00.691+0000 7f2508400700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f25001035e0 0x7f250019afa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:00.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:00.691+0000 7f2508400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2500105f10 0x7f250019b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:00.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:00.691+0000 7f2508400700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2500108840 0x7f25001a2560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:00.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:00.691+0000 7f2508400700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f250010e700 con 0x7f2500108840 2024-04-16T17:06:00.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:00.691+0000 7f2508400700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f250010e580 con 0x7f25001035e0 2024-04-16T17:06:00.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:00.691+0000 7f2508400700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f250010e880 con 0x7f2500105f10 2024-04-16T17:06:00.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:00.692+0000 7f2506400700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2500108840 0x7f25001a2560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:00.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:00.692+0000 7f2505a00700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f25001035e0 0x7f250019afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:00.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:00.692+0000 7f2505000700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2500105f10 0x7f250019b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:00.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:00.692+0000 7f2506400700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2500108840 0x7f25001a2560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42232/0 (socket says 172.21.15.45:42232) 2024-04-16T17:06:00.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:00.692+0000 7f2506400700 1 -- 172.21.15.45:0/628781471 learned_addr learned my addr 172.21.15.45:0/628781471 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:06:00.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:00.692+0000 7f2505a00700 1 -- 172.21.15.45:0/628781471 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2500105f10 msgr2=0x7f250019b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:00.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:00.692+0000 7f2505a00700 1 --2- 172.21.15.45:0/628781471 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2500105f10 0x7f250019b4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:00.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:00.692+0000 7f2505a00700 1 -- 172.21.15.45:0/628781471 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2500108840 msgr2=0x7f25001a2560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:00.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:00.692+0000 7f2505a00700 1 --2- 172.21.15.45:0/628781471 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2500108840 0x7f25001a2560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:00.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:00.692+0000 7f2505a00700 1 -- 172.21.15.45:0/628781471 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25001a2c00 con 0x7f25001035e0 2024-04-16T17:06:00.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:00.692+0000 7f2505000700 1 --2- 172.21.15.45:0/628781471 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2500105f10 0x7f250019b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:06:00.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:00.693+0000 7f2505a00700 1 --2- 172.21.15.45:0/628781471 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f25001035e0 0x7f250019afa0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f24f400e9f0 tx=0x7f24f400ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:06:00.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:00.693+0000 7f24fe000700 1 -- 172.21.15.45:0/628781471 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24f400c9c0 con 0x7f25001035e0 2024-04-16T17:06:00.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:00.693+0000 7f24fe000700 1 -- 172.21.15.45:0/628781471 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f24f400cb60 con 0x7f25001035e0 2024-04-16T17:06:00.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:00.693+0000 7f2508400700 1 -- 172.21.15.45:0/628781471 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25001a2ef0 con 0x7f25001035e0 2024-04-16T17:06:00.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:00.693+0000 7f24fe000700 1 -- 172.21.15.45:0/628781471 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24f4018680 con 0x7f25001035e0 2024-04-16T17:06:00.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:00.693+0000 7f2508400700 1 -- 172.21.15.45:0/628781471 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25001a3350 con 0x7f25001035e0 2024-04-16T17:06:00.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:00.695+0000 7f2508400700 1 -- 172.21.15.45:0/628781471 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25000697e0 con 0x7f25001035e0 2024-04-16T17:06:00.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:00.696+0000 7f24fe000700 1 -- 172.21.15.45:0/628781471 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f24f4010430 con 0x7f25001035e0 2024-04-16T17:06:00.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:00.698+0000 7f24fe000700 1 --2- 172.21.15.45:0/628781471 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f24e80a3cc0 0x7f24e80a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:00.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:00.698+0000 7f24fe000700 1 -- 172.21.15.45:0/628781471 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f24f40d0780 con 0x7f25001035e0 2024-04-16T17:06:00.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:00.698+0000 7f2505000700 1 --2- 172.21.15.45:0/628781471 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f24e80a3cc0 0x7f24e80a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:00.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:00.699+0000 7f2505000700 1 --2- 172.21.15.45:0/628781471 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f24e80a3cc0 0x7f24e80a6180 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f24ec011490 tx=0x7f24ec009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:06:00.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:00.702+0000 7f24fe000700 1 -- 172.21.15.45:0/628781471 <== mon.2 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f24f409cdd0 con 0x7f25001035e0 2024-04-16T17:06:00.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:00.954+0000 7f2508400700 1 -- 172.21.15.45:0/628781471 --> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7f2500062f30 con 0x7f24e80a3cc0 2024-04-16T17:06:00.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:00.957+0000 7f24fe000700 1 -- 172.21.15.45:0/628781471 <== mgr.14152 v2:172.21.15.45:6800/268030699 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7f2500062f30 con 0x7f24e80a3cc0 2024-04-16T17:06:00.959 INFO:teuthology.orchestra.run.smithi045.stdout:Backend: cephadm 2024-04-16T17:06:00.959 INFO:teuthology.orchestra.run.smithi045.stdout:Available: Yes 2024-04-16T17:06:00.959 INFO:teuthology.orchestra.run.smithi045.stdout:Paused: No 2024-04-16T17:06:00.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:00.960+0000 7f2508400700 1 -- 172.21.15.45:0/628781471 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f24e80a3cc0 msgr2=0x7f24e80a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:00.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:00.960+0000 7f2508400700 1 --2- 172.21.15.45:0/628781471 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f24e80a3cc0 0x7f24e80a6180 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f24ec011490 tx=0x7f24ec009260 comp rx=0 tx=0).stop 2024-04-16T17:06:00.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:00.960+0000 7f2508400700 1 -- 172.21.15.45:0/628781471 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f25001035e0 msgr2=0x7f250019afa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:00.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:00.960+0000 7f2508400700 1 --2- 172.21.15.45:0/628781471 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f25001035e0 0x7f250019afa0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f24f400e9f0 tx=0x7f24f400ed00 comp rx=0 tx=0).stop 2024-04-16T17:06:00.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:00.960+0000 7f2508400700 1 -- 172.21.15.45:0/628781471 shutdown_connections 2024-04-16T17:06:00.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:00.960+0000 7f2508400700 1 --2- 172.21.15.45:0/628781471 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2500108840 0x7f25001a2560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:00.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:00.960+0000 7f2508400700 1 --2- 172.21.15.45:0/628781471 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f25001035e0 0x7f250019afa0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:00.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:00.960+0000 7f2508400700 1 --2- 172.21.15.45:0/628781471 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f24e80a3cc0 0x7f24e80a6180 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:00.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:00.960+0000 7f2508400700 1 --2- 172.21.15.45:0/628781471 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2500105f10 0x7f250019b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:00.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:00.960+0000 7f2508400700 1 -- 172.21.15.45:0/628781471 >> 172.21.15.45:0/628781471 conn(0x7f25000fcff0 msgr2=0x7f2500102d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:06:00.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:00.960+0000 7f2508400700 1 -- 172.21.15.45:0/628781471 shutdown_connections 2024-04-16T17:06:00.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:00.961+0000 7f2508400700 1 -- 172.21.15.45:0/628781471 wait complete. 2024-04-16T17:06:01.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:06:01 smithi045 ceph-mon[31676]: pgmap v404: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-16T17:06:01.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:06:01 smithi117 ceph-mon[36730]: pgmap v404: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-16T17:06:01.858 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b1fecd38-fc11-11ee-bc90-c7b262605968 -- bash -c 'ceph orch ps' 2024-04-16T17:06:01.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:06:01 smithi104 ceph-mon[36457]: pgmap v404: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-16T17:06:02.357 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/mon.a/config 2024-04-16T17:06:02.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:06:02 smithi045 ceph-mon[31676]: from='client.25226 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T17:06:02.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:06:02 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T17:06:02.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:06:02 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:06:02.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:06:02 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:06:02.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:06:02 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:06:02.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:06:02 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:06:02.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:06:02 smithi117 ceph-mon[36730]: from='client.25226 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T17:06:02.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:06:02 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T17:06:02.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:06:02 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:06:02.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:06:02 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:06:02.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:06:02 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:06:02.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:06:02 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:06:02.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:06:02 smithi104 ceph-mon[36457]: from='client.25226 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T17:06:02.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:06:02 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T17:06:02.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:06:02 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:06:02.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:06:02 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:06:02.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:06:02 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:06:02.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:06:02 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:06:03.555 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:06:03 smithi117 ceph-mon[36730]: pgmap v405: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-16T17:06:03.555 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:06:03 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:06:03.555 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:06:03 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:06:03.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:06:03 smithi045 ceph-mon[31676]: pgmap v405: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-16T17:06:03.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:06:03 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:06:03.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:06:03 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:06:03.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:06:03 smithi104 ceph-mon[36457]: pgmap v405: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-16T17:06:03.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:06:03 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:06:03.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:06:03 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:06:04.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:04.750+0000 7f50c7000700 1 -- 172.21.15.45:0/4248153914 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f50c0103d90 msgr2=0x7f50c01041f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:04.753 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:04.750+0000 7f50c7000700 1 --2- 172.21.15.45:0/4248153914 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f50c0103d90 0x7f50c01041f0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f50ac009990 tx=0x7f50ac009ca0 comp rx=0 tx=0).stop 2024-04-16T17:06:04.753 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:04.750+0000 7f50c7000700 1 -- 172.21.15.45:0/4248153914 shutdown_connections 2024-04-16T17:06:04.753 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:04.750+0000 7f50c7000700 1 --2- 172.21.15.45:0/4248153914 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f50c0104730 0x7f50c010afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:04.753 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:04.750+0000 7f50c7000700 1 --2- 172.21.15.45:0/4248153914 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f50c0102b90 0x7f50c0102fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:04.753 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:04.750+0000 7f50c7000700 1 --2- 172.21.15.45:0/4248153914 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f50c0103d90 0x7f50c01041f0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:04.753 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:04.751+0000 7f50c7000700 1 -- 172.21.15.45:0/4248153914 >> 172.21.15.45:0/4248153914 conn(0x7f50c00fe130 msgr2=0x7f50c0100570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:06:04.753 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:04.751+0000 7f50c7000700 1 -- 172.21.15.45:0/4248153914 shutdown_connections 2024-04-16T17:06:04.753 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:04.751+0000 7f50c7000700 1 -- 172.21.15.45:0/4248153914 wait complete. 2024-04-16T17:06:04.753 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:04.752+0000 7f50c7000700 1 Processor -- start 2024-04-16T17:06:04.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:04.753+0000 7f50c7000700 1 -- start start 2024-04-16T17:06:04.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:04.753+0000 7f50c7000700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f50c0102b90 0x7f50c019aff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:04.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:04.753+0000 7f50c7000700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f50c0103d90 0x7f50c019b530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:04.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:04.753+0000 7f50c7000700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f50c0104730 0x7f50c01a2560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:04.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:04.753+0000 7f50c7000700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f50c010e7a0 con 0x7f50c0102b90 2024-04-16T17:06:04.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:04.753+0000 7f50c7000700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f50c010e620 con 0x7f50c0103d90 2024-04-16T17:06:04.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:04.753+0000 7f50c7000700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f50c010e920 con 0x7f50c0104730 2024-04-16T17:06:04.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:04.754+0000 7f50bfe00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f50c0102b90 0x7f50c019aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:04.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:04.754+0000 7f50c5000700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f50c0104730 0x7f50c01a2560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:04.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:04.754+0000 7f50bf400700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f50c0103d90 0x7f50c019b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:04.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:04.754+0000 7f50bfe00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f50c0102b90 0x7f50c019aff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42252/0 (socket says 172.21.15.45:42252) 2024-04-16T17:06:04.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:04.754+0000 7f50bfe00700 1 -- 172.21.15.45:0/4031296893 learned_addr learned my addr 172.21.15.45:0/4031296893 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:06:04.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:04.754+0000 7f50c5000700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f50c0104730 0x7f50c01a2560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:37194/0 (socket says 172.21.15.45:37194) 2024-04-16T17:06:04.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:04.754+0000 7f50bf400700 1 -- 172.21.15.45:0/4031296893 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f50c0104730 msgr2=0x7f50c01a2560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:04.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:04.754+0000 7f50bf400700 1 --2- 172.21.15.45:0/4031296893 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f50c0104730 0x7f50c01a2560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:04.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:04.754+0000 7f50bf400700 1 -- 172.21.15.45:0/4031296893 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f50c0102b90 msgr2=0x7f50c019aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:04.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:04.754+0000 7f50bf400700 1 --2- 172.21.15.45:0/4031296893 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f50c0102b90 0x7f50c019aff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:04.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:04.754+0000 7f50bf400700 1 -- 172.21.15.45:0/4031296893 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50c01a2c00 con 0x7f50c0103d90 2024-04-16T17:06:04.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:04.754+0000 7f50c5000700 1 --2- 172.21.15.45:0/4031296893 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f50c0104730 0x7f50c01a2560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:06:04.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:04.754+0000 7f50bf400700 1 --2- 172.21.15.45:0/4031296893 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f50c0103d90 0x7f50c019b530 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f50ac000c00 tx=0x7f50ac009580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:06:04.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:04.755+0000 7f50bcc00700 1 -- 172.21.15.45:0/4031296893 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50ac026070 con 0x7f50c0103d90 2024-04-16T17:06:04.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:04.755+0000 7f50bcc00700 1 -- 172.21.15.45:0/4031296893 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f50ac00fd00 con 0x7f50c0103d90 2024-04-16T17:06:04.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:04.755+0000 7f50c7000700 1 -- 172.21.15.45:0/4031296893 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50c01a2e90 con 0x7f50c0103d90 2024-04-16T17:06:04.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:04.755+0000 7f50bcc00700 1 -- 172.21.15.45:0/4031296893 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50ac021e60 con 0x7f50c0103d90 2024-04-16T17:06:04.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:04.755+0000 7f50c7000700 1 -- 172.21.15.45:0/4031296893 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50c01a3280 con 0x7f50c0103d90 2024-04-16T17:06:04.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:04.756+0000 7f50c7000700 1 -- 172.21.15.45:0/4031296893 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f50c0069810 con 0x7f50c0103d90 2024-04-16T17:06:04.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:04.758+0000 7f50bcc00700 1 -- 172.21.15.45:0/4031296893 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f50ac029070 con 0x7f50c0103d90 2024-04-16T17:06:04.761 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:04.759+0000 7f50bcc00700 1 --2- 172.21.15.45:0/4031296893 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f50a40a3c40 0x7f50a40a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:04.761 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:04.760+0000 7f50bcc00700 1 -- 172.21.15.45:0/4031296893 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f50ac0d14b0 con 0x7f50c0103d90 2024-04-16T17:06:04.761 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:04.760+0000 7f50bfe00700 1 --2- 172.21.15.45:0/4031296893 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f50a40a3c40 0x7f50a40a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:04.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:04.761+0000 7f50bfe00700 1 --2- 172.21.15.45:0/4031296893 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f50a40a3c40 0x7f50a40a6100 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f50b0005950 tx=0x7f50b0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:06:04.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:04.761+0000 7f50bcc00700 1 -- 172.21.15.45:0/4031296893 <== mon.2 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f50c0069810 con 0x7f50c0103d90 2024-04-16T17:06:05.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:05.014+0000 7f50c7000700 1 -- 172.21.15.45:0/4031296893 --> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f50c0076440 con 0x7f50a40a3c40 2024-04-16T17:06:05.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:05.066+0000 7f50bcc00700 1 -- 172.21.15.45:0/4031296893 <== mgr.14152 v2:172.21.15.45:6800/268030699 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4123 (secure 0 0 0) 0x7f50c0076440 con 0x7f50a40a3c40 2024-04-16T17:06:05.068 INFO:teuthology.orchestra.run.smithi045.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-16T17:06:05.068 INFO:teuthology.orchestra.run.smithi045.stdout:mds.a smithi045 running (2m) 65s ago 2m 16.3M - 19.0.0-3101-g9b42bfbe e5ce10bdffbe 070e1db7eaf3 2024-04-16T17:06:05.068 INFO:teuthology.orchestra.run.smithi045.stdout:mds.b smithi104 running (2m) 1s ago 2m 28.2M - 19.0.0-3101-g9b42bfbe e5ce10bdffbe 7f17776ddeed 2024-04-16T17:06:05.068 INFO:teuthology.orchestra.run.smithi045.stdout:mds.c smithi117 running (3m) 1s ago 3m 20.8M - 19.0.0-3101-g9b42bfbe e5ce10bdffbe 13fd9e10d581 2024-04-16T17:06:05.068 INFO:teuthology.orchestra.run.smithi045.stdout:mds.d smithi045 running (3m) 65s ago 3m 12.5M - 19.0.0-3101-g9b42bfbe e5ce10bdffbe 37490c52e388 2024-04-16T17:06:05.068 INFO:teuthology.orchestra.run.smithi045.stdout:mds.e smithi104 running (3m) 1s ago 3m 13.9M - 19.0.0-3101-g9b42bfbe e5ce10bdffbe 27e929370e37 2024-04-16T17:06:05.068 INFO:teuthology.orchestra.run.smithi045.stdout:mds.f smithi117 running (3m) 1s ago 3m 22.9M - 19.0.0-3101-g9b42bfbe e5ce10bdffbe a9eb34cce0d4 2024-04-16T17:06:05.068 INFO:teuthology.orchestra.run.smithi045.stdout:mds.g smithi045 running (2m) 65s ago 2m 12.4M - 19.0.0-3101-g9b42bfbe e5ce10bdffbe 0cf613ae6b13 2024-04-16T17:06:05.068 INFO:teuthology.orchestra.run.smithi045.stdout:mds.h smithi104 running (3m) 1s ago 3m 21.1M - 19.0.0-3101-g9b42bfbe e5ce10bdffbe 664cad82bac4 2024-04-16T17:06:05.068 INFO:teuthology.orchestra.run.smithi045.stdout:mds.i smithi117 running (2m) 1s ago 2m 20.2M - 19.0.0-3101-g9b42bfbe e5ce10bdffbe 02de05e9e9c2 2024-04-16T17:06:05.068 INFO:teuthology.orchestra.run.smithi045.stdout:mds.j smithi045 running (3m) 65s ago 3m 12.5M - 19.0.0-3101-g9b42bfbe e5ce10bdffbe f9a5b8751be2 2024-04-16T17:06:05.069 INFO:teuthology.orchestra.run.smithi045.stdout:mds.k smithi104 running (3m) 1s ago 3m 14.7M - 19.0.0-3101-g9b42bfbe e5ce10bdffbe c06d93d410ff 2024-04-16T17:06:05.069 INFO:teuthology.orchestra.run.smithi045.stdout:mds.l smithi117 running (3m) 1s ago 3m 16.2M - 19.0.0-3101-g9b42bfbe e5ce10bdffbe 888d9071f176 2024-04-16T17:06:05.069 INFO:teuthology.orchestra.run.smithi045.stdout:mgr.x smithi045 *:9283,8765 running (11m) 65s ago 11m 515M - 19.0.0-3101-g9b42bfbe e5ce10bdffbe 9b59d149dd29 2024-04-16T17:06:05.069 INFO:teuthology.orchestra.run.smithi045.stdout:mgr.y smithi104 *:8443,8765 running (9m) 1s ago 9m 455M - 19.0.0-3101-g9b42bfbe e5ce10bdffbe ae9050b2d6c5 2024-04-16T17:06:05.069 INFO:teuthology.orchestra.run.smithi045.stdout:mgr.z smithi117 *:8443,8765 running (9m) 1s ago 9m 453M - 19.0.0-3101-g9b42bfbe e5ce10bdffbe 08195d3a5a95 2024-04-16T17:06:05.069 INFO:teuthology.orchestra.run.smithi045.stdout:mon.a smithi045 running (11m) 65s ago 12m 48.2M 2048M 19.0.0-3101-g9b42bfbe e5ce10bdffbe 5e03aa10df48 2024-04-16T17:06:05.069 INFO:teuthology.orchestra.run.smithi045.stdout:mon.b smithi104 running (9m) 1s ago 9m 47.0M 2048M 19.0.0-3101-g9b42bfbe e5ce10bdffbe 68a342e84dea 2024-04-16T17:06:05.069 INFO:teuthology.orchestra.run.smithi045.stdout:mon.c smithi117 running (9m) 1s ago 9m 45.2M 2048M 19.0.0-3101-g9b42bfbe e5ce10bdffbe a26181e132ad 2024-04-16T17:06:05.069 INFO:teuthology.orchestra.run.smithi045.stdout:osd.0 smithi045 running (8m) 65s ago 8m 54.9M 4273M 19.0.0-3101-g9b42bfbe e5ce10bdffbe e52617cc8fdb 2024-04-16T17:06:05.069 INFO:teuthology.orchestra.run.smithi045.stdout:osd.1 smithi104 running (8m) 1s ago 8m 57.5M 4270M 19.0.0-3101-g9b42bfbe e5ce10bdffbe 04c09ef53b42 2024-04-16T17:06:05.069 INFO:teuthology.orchestra.run.smithi045.stdout:osd.2 smithi117 running (8m) 1s ago 8m 48.4M 4270M 19.0.0-3101-g9b42bfbe e5ce10bdffbe 36470e2c5246 2024-04-16T17:06:05.069 INFO:teuthology.orchestra.run.smithi045.stdout:osd.3 smithi045 running (7m) 65s ago 7m 54.0M 4273M 19.0.0-3101-g9b42bfbe e5ce10bdffbe cf9586aa6178 2024-04-16T17:06:05.069 INFO:teuthology.orchestra.run.smithi045.stdout:osd.4 smithi104 running (7m) 1s ago 7m 44.8M 4270M 19.0.0-3101-g9b42bfbe e5ce10bdffbe 90c38fbb7da6 2024-04-16T17:06:05.069 INFO:teuthology.orchestra.run.smithi045.stdout:osd.5 smithi117 running (6m) 1s ago 6m 47.0M 4270M 19.0.0-3101-g9b42bfbe e5ce10bdffbe 5b060c3476ff 2024-04-16T17:06:05.069 INFO:teuthology.orchestra.run.smithi045.stdout:osd.6 smithi045 running (6m) 65s ago 6m 52.4M 4273M 19.0.0-3101-g9b42bfbe e5ce10bdffbe 717404df8e8c 2024-04-16T17:06:05.069 INFO:teuthology.orchestra.run.smithi045.stdout:osd.7 smithi104 running (5m) 1s ago 5m 60.4M 4270M 19.0.0-3101-g9b42bfbe e5ce10bdffbe 8eb3a9b475d1 2024-04-16T17:06:05.070 INFO:teuthology.orchestra.run.smithi045.stdout:osd.8 smithi117 running (5m) 1s ago 5m 54.9M 4270M 19.0.0-3101-g9b42bfbe e5ce10bdffbe f23150d642f1 2024-04-16T17:06:05.070 INFO:teuthology.orchestra.run.smithi045.stdout:osd.9 smithi045 running (4m) 65s ago 4m 52.7M 4273M 19.0.0-3101-g9b42bfbe e5ce10bdffbe 4c51f92b25f2 2024-04-16T17:06:05.070 INFO:teuthology.orchestra.run.smithi045.stdout:osd.10 smithi104 running (4m) 1s ago 4m 55.4M 4270M 19.0.0-3101-g9b42bfbe e5ce10bdffbe 3e58719018ae 2024-04-16T17:06:05.070 INFO:teuthology.orchestra.run.smithi045.stdout:osd.11 smithi117 running (3m) 1s ago 3m 46.1M 4270M 19.0.0-3101-g9b42bfbe e5ce10bdffbe 6ef904b3d002 2024-04-16T17:06:05.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:05.069+0000 7f50c7000700 1 -- 172.21.15.45:0/4031296893 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f50a40a3c40 msgr2=0x7f50a40a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:05.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:05.069+0000 7f50c7000700 1 --2- 172.21.15.45:0/4031296893 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f50a40a3c40 0x7f50a40a6100 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f50b0005950 tx=0x7f50b0009260 comp rx=0 tx=0).stop 2024-04-16T17:06:05.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:05.069+0000 7f50c7000700 1 -- 172.21.15.45:0/4031296893 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f50c0103d90 msgr2=0x7f50c019b530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:05.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:05.069+0000 7f50c7000700 1 --2- 172.21.15.45:0/4031296893 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f50c0103d90 0x7f50c019b530 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f50ac000c00 tx=0x7f50ac009580 comp rx=0 tx=0).stop 2024-04-16T17:06:05.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:05.069+0000 7f50c7000700 1 -- 172.21.15.45:0/4031296893 shutdown_connections 2024-04-16T17:06:05.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:05.069+0000 7f50c7000700 1 --2- 172.21.15.45:0/4031296893 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f50a40a3c40 0x7f50a40a6100 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:05.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:05.069+0000 7f50c7000700 1 --2- 172.21.15.45:0/4031296893 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f50c0104730 0x7f50c01a2560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:05.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:05.069+0000 7f50c7000700 1 --2- 172.21.15.45:0/4031296893 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f50c0102b90 0x7f50c019aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:05.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:05.069+0000 7f50c7000700 1 --2- 172.21.15.45:0/4031296893 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f50c0103d90 0x7f50c019b530 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:05.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:05.069+0000 7f50c7000700 1 -- 172.21.15.45:0/4031296893 >> 172.21.15.45:0/4031296893 conn(0x7f50c00fe130 msgr2=0x7f50c0074d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:06:05.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:05.070+0000 7f50c7000700 1 -- 172.21.15.45:0/4031296893 shutdown_connections 2024-04-16T17:06:05.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:05.070+0000 7f50c7000700 1 -- 172.21.15.45:0/4031296893 wait complete. 2024-04-16T17:06:05.178 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:06:04 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:06:05.178 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:06:04 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:06:05.178 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:06:04 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:06:05.178 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:06:04 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:06:05.221 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:06:04 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:06:05.221 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:06:04 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:06:05.221 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:06:04 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:06:05.221 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:06:04 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:06:05.305 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:06:04 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:06:05.305 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:06:04 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:06:05.305 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:06:04 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:06:05.305 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:06:04 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:06:06.178 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:06:05 smithi104 ceph-mon[36457]: pgmap v406: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T17:06:06.227 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:06:05 smithi045 ceph-mon[31676]: pgmap v406: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T17:06:06.305 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:06:05 smithi117 ceph-mon[36730]: pgmap v406: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T17:06:06.460 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b1fecd38-fc11-11ee-bc90-c7b262605968 -- bash -c 'ceph orch ls' 2024-04-16T17:06:06.926 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/mon.a/config 2024-04-16T17:06:07.178 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:06:06 smithi104 ceph-mon[36457]: from='client.25232 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T17:06:07.228 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:06:06 smithi045 ceph-mon[31676]: from='client.25232 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T17:06:07.305 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:06:06 smithi117 ceph-mon[36730]: from='client.25232 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T17:06:08.178 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:06:07 smithi104 ceph-mon[36457]: pgmap v407: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-16T17:06:08.227 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:06:07 smithi045 ceph-mon[31676]: pgmap v407: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-16T17:06:08.305 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:06:07 smithi117 ceph-mon[36730]: pgmap v407: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-16T17:06:08.966 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:06:08 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:06:08.967 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:06:08 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:06:08.967 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:06:08 smithi045 ceph-mon[31676]: pgmap v408: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-16T17:06:09.178 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:06:08 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:06:09.178 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:06:08 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:06:09.178 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:06:08 smithi104 ceph-mon[36457]: pgmap v408: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-16T17:06:09.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:09.255+0000 7f6738400700 1 -- 172.21.15.45:0/163242238 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6730102b60 msgr2=0x7f6730102f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:09.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:09.255+0000 7f6738400700 1 --2- 172.21.15.45:0/163242238 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6730102b60 0x7f6730102f80 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f6724009920 tx=0x7f6724009c30 comp rx=0 tx=0).stop 2024-04-16T17:06:09.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:09.255+0000 7f6738400700 1 -- 172.21.15.45:0/163242238 shutdown_connections 2024-04-16T17:06:09.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:09.255+0000 7f6738400700 1 --2- 172.21.15.45:0/163242238 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6730104700 0x7f673010afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:09.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:09.255+0000 7f6738400700 1 --2- 172.21.15.45:0/163242238 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6730102b60 0x7f6730102f80 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:09.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:09.255+0000 7f6738400700 1 --2- 172.21.15.45:0/163242238 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6730103d60 0x7f67301041c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:09.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:09.255+0000 7f6738400700 1 -- 172.21.15.45:0/163242238 >> 172.21.15.45:0/163242238 conn(0x7f67300fe100 msgr2=0x7f6730100540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:06:09.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:09.255+0000 7f6738400700 1 -- 172.21.15.45:0/163242238 shutdown_connections 2024-04-16T17:06:09.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:09.255+0000 7f6738400700 1 -- 172.21.15.45:0/163242238 wait complete. 2024-04-16T17:06:09.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:09.257+0000 7f6738400700 1 Processor -- start 2024-04-16T17:06:09.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:09.257+0000 7f6738400700 1 -- start start 2024-04-16T17:06:09.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:09.257+0000 7f6738400700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6730102b60 0x7f6730076890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:09.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:09.257+0000 7f6738400700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6730103d60 0x7f6730074ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:09.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:09.257+0000 7f6738400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6730104700 0x7f6730075420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:09.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:09.257+0000 7f6738400700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f673010e720 con 0x7f6730102b60 2024-04-16T17:06:09.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:09.257+0000 7f6738400700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f673010e5a0 con 0x7f6730103d60 2024-04-16T17:06:09.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:09.257+0000 7f6738400700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f673010e8a0 con 0x7f6730104700 2024-04-16T17:06:09.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:09.258+0000 7f6735a00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6730102b60 0x7f6730076890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:09.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:09.258+0000 7f6736400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6730104700 0x7f6730075420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:09.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:09.258+0000 7f6735a00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6730102b60 0x7f6730076890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42270/0 (socket says 172.21.15.45:42270) 2024-04-16T17:06:09.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:09.258+0000 7f6735a00700 1 -- 172.21.15.45:0/561252995 learned_addr learned my addr 172.21.15.45:0/561252995 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:06:09.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:09.258+0000 7f6735000700 1 --2- 172.21.15.45:0/561252995 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6730103d60 0x7f6730074ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:09.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:09.258+0000 7f6735a00700 1 -- 172.21.15.45:0/561252995 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6730104700 msgr2=0x7f6730075420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:09.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:09.258+0000 7f6735a00700 1 --2- 172.21.15.45:0/561252995 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6730104700 0x7f6730075420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:09.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:09.258+0000 7f6735a00700 1 -- 172.21.15.45:0/561252995 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6730103d60 msgr2=0x7f6730074ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:09.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:09.258+0000 7f6735a00700 1 --2- 172.21.15.45:0/561252995 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6730103d60 0x7f6730074ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:09.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:09.258+0000 7f6735a00700 1 -- 172.21.15.45:0/561252995 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6730075cd0 con 0x7f6730102b60 2024-04-16T17:06:09.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:09.259+0000 7f6735a00700 1 --2- 172.21.15.45:0/561252995 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6730102b60 0x7f6730076890 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f672400a010 tx=0x7f6724005250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:06:09.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:09.259+0000 7f672e000700 1 -- 172.21.15.45:0/561252995 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6724027070 con 0x7f6730102b60 2024-04-16T17:06:09.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:09.259+0000 7f672e000700 1 -- 172.21.15.45:0/561252995 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f672400f950 con 0x7f6730102b60 2024-04-16T17:06:09.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:09.259+0000 7f672e000700 1 -- 172.21.15.45:0/561252995 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67240049c0 con 0x7f6730102b60 2024-04-16T17:06:09.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:09.260+0000 7f6738400700 1 -- 172.21.15.45:0/561252995 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67301a2ef0 con 0x7f6730102b60 2024-04-16T17:06:09.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:09.260+0000 7f6738400700 1 -- 172.21.15.45:0/561252995 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67301a3290 con 0x7f6730102b60 2024-04-16T17:06:09.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:09.261+0000 7f6738400700 1 -- 172.21.15.45:0/561252995 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f67300697e0 con 0x7f6730102b60 2024-04-16T17:06:09.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:09.265+0000 7f672e000700 1 -- 172.21.15.45:0/561252995 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f672402a070 con 0x7f6730102b60 2024-04-16T17:06:09.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:09.267+0000 7f672e000700 1 --2- 172.21.15.45:0/561252995 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f67180a3cc0 0x7f67180a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:09.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:09.267+0000 7f672e000700 1 -- 172.21.15.45:0/561252995 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f67240d1370 con 0x7f6730102b60 2024-04-16T17:06:09.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:09.268+0000 7f6735000700 1 --2- 172.21.15.45:0/561252995 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f67180a3cc0 0x7f67180a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:09.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:09.268+0000 7f6735000700 1 --2- 172.21.15.45:0/561252995 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f67180a3cc0 0x7f67180a6180 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f671c009680 tx=0x7f671c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:06:09.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:09.270+0000 7f672e000700 1 -- 172.21.15.45:0/561252995 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6724031080 con 0x7f6730102b60 2024-04-16T17:06:09.305 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:06:08 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:06:09.305 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:06:08 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:06:09.306 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:06:08 smithi117 ceph-mon[36730]: pgmap v408: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-16T17:06:09.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:09.530+0000 7f6738400700 1 -- 172.21.15.45:0/561252995 --> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7f6730062f30 con 0x7f67180a3cc0 2024-04-16T17:06:09.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:09.539+0000 7f672e000700 1 -- 172.21.15.45:0/561252995 <== mgr.14152 v2:172.21.15.45:6800/268030699 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7f6730062f30 con 0x7f67180a3cc0 2024-04-16T17:06:09.541 INFO:teuthology.orchestra.run.smithi045.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-04-16T17:06:09.541 INFO:teuthology.orchestra.run.smithi045.stdout:mds.all 12/12 5s ago 3m smithi045=a;smithi045=d;smithi045=g;smithi045=j;smithi104=b;smithi104=e;smithi104=h;smithi104=k;smithi117=c;smithi117=f;smithi117=i;smithi117=l;count:12 2024-04-16T17:06:09.541 INFO:teuthology.orchestra.run.smithi045.stdout:mgr 3/3 5s ago 9m smithi045=x;smithi104=y;smithi117=z;count:3 2024-04-16T17:06:09.541 INFO:teuthology.orchestra.run.smithi045.stdout:mon 3/3 5s ago 9m smithi045:172.21.15.45=a;smithi104:172.21.15.104=b;smithi117:172.21.15.117=c;count:3 2024-04-16T17:06:09.542 INFO:teuthology.orchestra.run.smithi045.stdout:osd 12 5s ago - 2024-04-16T17:06:09.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:09.542+0000 7f6738400700 1 -- 172.21.15.45:0/561252995 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f67180a3cc0 msgr2=0x7f67180a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:09.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:09.542+0000 7f6738400700 1 --2- 172.21.15.45:0/561252995 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f67180a3cc0 0x7f67180a6180 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f671c009680 tx=0x7f671c009260 comp rx=0 tx=0).stop 2024-04-16T17:06:09.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:09.542+0000 7f6738400700 1 -- 172.21.15.45:0/561252995 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6730102b60 msgr2=0x7f6730076890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:09.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:09.542+0000 7f6738400700 1 --2- 172.21.15.45:0/561252995 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6730102b60 0x7f6730076890 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f672400a010 tx=0x7f6724005250 comp rx=0 tx=0).stop 2024-04-16T17:06:09.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:09.542+0000 7f6738400700 1 -- 172.21.15.45:0/561252995 shutdown_connections 2024-04-16T17:06:09.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:09.542+0000 7f6738400700 1 --2- 172.21.15.45:0/561252995 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f67180a3cc0 0x7f67180a6180 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:09.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:09.542+0000 7f6738400700 1 --2- 172.21.15.45:0/561252995 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6730104700 0x7f6730075420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:09.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:09.542+0000 7f6738400700 1 --2- 172.21.15.45:0/561252995 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6730102b60 0x7f6730076890 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:09.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:09.542+0000 7f6738400700 1 --2- 172.21.15.45:0/561252995 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6730103d60 0x7f6730074ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:09.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:09.542+0000 7f6738400700 1 -- 172.21.15.45:0/561252995 >> 172.21.15.45:0/561252995 conn(0x7f67300fe100 msgr2=0x7f67301070b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:06:09.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:09.543+0000 7f6738400700 1 -- 172.21.15.45:0/561252995 shutdown_connections 2024-04-16T17:06:09.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:09.543+0000 7f6738400700 1 -- 172.21.15.45:0/561252995 wait complete. 2024-04-16T17:06:10.228 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:06:09 smithi045 ceph-mon[31676]: from='client.15126 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T17:06:10.228 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:06:09 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:06:10.228 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:06:09 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T17:06:10.228 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:06:09 smithi045 ceph-mon[31676]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:06:10.305 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:06:09 smithi117 ceph-mon[36730]: from='client.15126 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T17:06:10.305 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:06:09 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:06:10.305 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:06:09 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T17:06:10.305 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:06:09 smithi117 ceph-mon[36730]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:06:10.428 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:06:09 smithi104 ceph-mon[36457]: from='client.15126 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T17:06:10.428 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:06:09 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:06:10.428 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:06:09 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T17:06:10.428 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:06:09 smithi104 ceph-mon[36457]: from='mgr.14152 172.21.15.45:0/34261514' entity='mgr.x' 2024-04-16T17:06:10.509 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b1fecd38-fc11-11ee-bc90-c7b262605968 -- bash -c 'ceph orch host ls' 2024-04-16T17:06:11.002 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/mon.a/config 2024-04-16T17:06:11.192 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:06:10 smithi045 ceph-mon[31676]: pgmap v409: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-16T17:06:11.305 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:06:10 smithi117 ceph-mon[36730]: pgmap v409: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-16T17:06:11.427 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:06:10 smithi104 ceph-mon[36457]: pgmap v409: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-16T17:06:13.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:13.332+0000 7f0870e00700 1 -- 172.21.15.45:0/2685422924 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f086c104c70 msgr2=0x7f086c105090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:13.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:13.332+0000 7f0870e00700 1 --2- 172.21.15.45:0/2685422924 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f086c104c70 0x7f086c105090 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f085c0099c0 tx=0x7f085c009cd0 comp rx=0 tx=0).stop 2024-04-16T17:06:13.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:13.332+0000 7f0870e00700 1 -- 172.21.15.45:0/2685422924 shutdown_connections 2024-04-16T17:06:13.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:13.332+0000 7f0870e00700 1 --2- 172.21.15.45:0/2685422924 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f086c106810 0x7f086c10d0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:13.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:13.332+0000 7f0870e00700 1 --2- 172.21.15.45:0/2685422924 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f086c104c70 0x7f086c105090 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:13.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:13.332+0000 7f0870e00700 1 --2- 172.21.15.45:0/2685422924 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f086c105e70 0x7f086c1062d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:13.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:13.332+0000 7f0870e00700 1 -- 172.21.15.45:0/2685422924 >> 172.21.15.45:0/2685422924 conn(0x7f086c100210 msgr2=0x7f086c102650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:06:13.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:13.333+0000 7f0870e00700 1 -- 172.21.15.45:0/2685422924 shutdown_connections 2024-04-16T17:06:13.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:13.333+0000 7f0870e00700 1 -- 172.21.15.45:0/2685422924 wait complete. 2024-04-16T17:06:13.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:13.334+0000 7f0870e00700 1 Processor -- start 2024-04-16T17:06:13.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:13.334+0000 7f0870e00700 1 -- start start 2024-04-16T17:06:13.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:13.334+0000 7f0870e00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f086c104c70 0x7f086c19ae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:13.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:13.335+0000 7f0870e00700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f086c105e70 0x7f086c19b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:13.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:13.335+0000 7f0870e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f086c106810 0x7f086c1a2420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:13.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:13.335+0000 7f0870e00700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f086c110880 con 0x7f086c104c70 2024-04-16T17:06:13.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:13.335+0000 7f0870e00700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f086c110700 con 0x7f086c105e70 2024-04-16T17:06:13.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:13.335+0000 7f0870e00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f086c110a00 con 0x7f086c106810 2024-04-16T17:06:13.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:13.335+0000 7f086b400700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f086c104c70 0x7f086c19ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:13.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:13.335+0000 7f086b400700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f086c104c70 0x7f086c19ae60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57172/0 (socket says 172.21.15.45:57172) 2024-04-16T17:06:13.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:13.335+0000 7f086b400700 1 -- 172.21.15.45:0/2973982244 learned_addr learned my addr 172.21.15.45:0/2973982244 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:06:13.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:13.335+0000 7f086aa00700 1 --2- 172.21.15.45:0/2973982244 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f086c105e70 0x7f086c19b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:13.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:13.335+0000 7f086be00700 1 --2- 172.21.15.45:0/2973982244 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f086c106810 0x7f086c1a2420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:13.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:13.335+0000 7f086b400700 1 -- 172.21.15.45:0/2973982244 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f086c106810 msgr2=0x7f086c1a2420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:13.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:13.335+0000 7f086b400700 1 --2- 172.21.15.45:0/2973982244 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f086c106810 0x7f086c1a2420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:13.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:13.335+0000 7f086b400700 1 -- 172.21.15.45:0/2973982244 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f086c105e70 msgr2=0x7f086c19b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:13.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:13.335+0000 7f086b400700 1 --2- 172.21.15.45:0/2973982244 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f086c105e70 0x7f086c19b3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:13.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:13.335+0000 7f086b400700 1 -- 172.21.15.45:0/2973982244 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f086c1a2ac0 con 0x7f086c104c70 2024-04-16T17:06:13.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:13.335+0000 7f086aa00700 1 --2- 172.21.15.45:0/2973982244 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f086c105e70 0x7f086c19b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:06:13.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:13.336+0000 7f086b400700 1 --2- 172.21.15.45:0/2973982244 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f086c104c70 0x7f086c19ae60 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f085c00bee0 tx=0x7f085c00bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:06:13.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:13.336+0000 7f0863e00700 1 -- 172.21.15.45:0/2973982244 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f085c003e30 con 0x7f086c104c70 2024-04-16T17:06:13.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:13.336+0000 7f0863e00700 1 -- 172.21.15.45:0/2973982244 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f085c00f460 con 0x7f086c104c70 2024-04-16T17:06:13.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:13.336+0000 7f0870e00700 1 -- 172.21.15.45:0/2973982244 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f086c1a2d50 con 0x7f086c104c70 2024-04-16T17:06:13.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:13.336+0000 7f0863e00700 1 -- 172.21.15.45:0/2973982244 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f085c017650 con 0x7f086c104c70 2024-04-16T17:06:13.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:13.336+0000 7f0870e00700 1 -- 172.21.15.45:0/2973982244 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f086c1a31a0 con 0x7f086c104c70 2024-04-16T17:06:13.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:13.337+0000 7f0870e00700 1 -- 172.21.15.45:0/2973982244 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f086c0697e0 con 0x7f086c104c70 2024-04-16T17:06:13.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:13.340+0000 7f0863e00700 1 -- 172.21.15.45:0/2973982244 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f085c00f610 con 0x7f086c104c70 2024-04-16T17:06:13.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:13.341+0000 7f0863e00700 1 --2- 172.21.15.45:0/2973982244 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f08500a3c40 0x7f08500a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:13.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:13.341+0000 7f0863e00700 1 -- 172.21.15.45:0/2973982244 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f085c0d15d0 con 0x7f086c104c70 2024-04-16T17:06:13.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:13.341+0000 7f086aa00700 1 --2- 172.21.15.45:0/2973982244 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f08500a3c40 0x7f08500a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:13.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:13.342+0000 7f086aa00700 1 --2- 172.21.15.45:0/2973982244 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f08500a3c40 0x7f08500a6100 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f0858011490 tx=0x7f0858009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:06:13.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:13.346+0000 7f0863e00700 1 -- 172.21.15.45:0/2973982244 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f085c013070 con 0x7f086c104c70 2024-04-16T17:06:13.555 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:06:13 smithi117 ceph-mon[36730]: pgmap v410: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-16T17:06:13.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:13.583+0000 7f0870e00700 1 -- 172.21.15.45:0/2973982244 --> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7f086c062f30 con 0x7f08500a3c40 2024-04-16T17:06:13.585 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:06:13 smithi045 ceph-mon[31676]: pgmap v410: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-16T17:06:13.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:13.586+0000 7f0863e00700 1 -- 172.21.15.45:0/2973982244 <== mgr.14152 v2:172.21.15.45:6800/268030699 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7f086c062f30 con 0x7f08500a3c40 2024-04-16T17:06:13.588 INFO:teuthology.orchestra.run.smithi045.stdout:HOST ADDR LABELS STATUS 2024-04-16T17:06:13.588 INFO:teuthology.orchestra.run.smithi045.stdout:smithi045 172.21.15.45 2024-04-16T17:06:13.588 INFO:teuthology.orchestra.run.smithi045.stdout:smithi104 172.21.15.104 2024-04-16T17:06:13.588 INFO:teuthology.orchestra.run.smithi045.stdout:smithi117 172.21.15.117 2024-04-16T17:06:13.588 INFO:teuthology.orchestra.run.smithi045.stdout:3 hosts in cluster 2024-04-16T17:06:13.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:13.590+0000 7f0870e00700 1 -- 172.21.15.45:0/2973982244 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f08500a3c40 msgr2=0x7f08500a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:13.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:13.590+0000 7f0870e00700 1 --2- 172.21.15.45:0/2973982244 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f08500a3c40 0x7f08500a6100 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f0858011490 tx=0x7f0858009260 comp rx=0 tx=0).stop 2024-04-16T17:06:13.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:13.590+0000 7f0870e00700 1 -- 172.21.15.45:0/2973982244 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f086c104c70 msgr2=0x7f086c19ae60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:13.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:13.590+0000 7f0870e00700 1 --2- 172.21.15.45:0/2973982244 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f086c104c70 0x7f086c19ae60 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f085c00bee0 tx=0x7f085c00bf10 comp rx=0 tx=0).stop 2024-04-16T17:06:13.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:13.590+0000 7f0870e00700 1 -- 172.21.15.45:0/2973982244 shutdown_connections 2024-04-16T17:06:13.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:13.590+0000 7f0870e00700 1 --2- 172.21.15.45:0/2973982244 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f08500a3c40 0x7f08500a6100 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:13.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:13.590+0000 7f0870e00700 1 --2- 172.21.15.45:0/2973982244 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f086c106810 0x7f086c1a2420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:13.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:13.590+0000 7f0870e00700 1 --2- 172.21.15.45:0/2973982244 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f086c104c70 0x7f086c19ae60 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:13.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:13.590+0000 7f0870e00700 1 --2- 172.21.15.45:0/2973982244 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f086c105e70 0x7f086c19b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:13.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:13.590+0000 7f0870e00700 1 -- 172.21.15.45:0/2973982244 >> 172.21.15.45:0/2973982244 conn(0x7f086c100210 msgr2=0x7f086c109110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:06:13.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:13.591+0000 7f0870e00700 1 -- 172.21.15.45:0/2973982244 shutdown_connections 2024-04-16T17:06:13.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:13.591+0000 7f0870e00700 1 -- 172.21.15.45:0/2973982244 wait complete. 2024-04-16T17:06:13.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:06:13 smithi104 ceph-mon[36457]: pgmap v410: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-16T17:06:14.515 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:06:14 smithi045 ceph-mon[31676]: from='client.15132 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T17:06:14.519 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b1fecd38-fc11-11ee-bc90-c7b262605968 -- bash -c 'ceph orch device ls' 2024-04-16T17:06:14.555 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:06:14 smithi117 ceph-mon[36730]: from='client.15132 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T17:06:14.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:06:14 smithi104 ceph-mon[36457]: from='client.15132 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T17:06:15.005 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/mon.a/config 2024-04-16T17:06:15.477 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:06:15 smithi045 ceph-mon[31676]: pgmap v411: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T17:06:15.555 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:06:15 smithi117 ceph-mon[36730]: pgmap v411: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T17:06:15.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:06:15 smithi104 ceph-mon[36457]: pgmap v411: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T17:06:17.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:17.435+0000 7f8471c00700 1 -- 172.21.15.45:0/3710481560 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f846c076b20 msgr2=0x7f846c074f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:17.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:17.435+0000 7f8471c00700 1 --2- 172.21.15.45:0/3710481560 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f846c076b20 0x7f846c074f80 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f845c009990 tx=0x7f845c009ca0 comp rx=0 tx=0).stop 2024-04-16T17:06:17.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:17.435+0000 7f8471c00700 1 -- 172.21.15.45:0/3710481560 shutdown_connections 2024-04-16T17:06:17.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:17.435+0000 7f8471c00700 1 --2- 172.21.15.45:0/3710481560 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f846c10a950 0x7f846c10cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:17.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:17.435+0000 7f8471c00700 1 --2- 172.21.15.45:0/3710481560 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f846c076b20 0x7f846c074f80 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:17.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:17.435+0000 7f8471c00700 1 --2- 172.21.15.45:0/3710481560 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f846c075550 0x7f846c0759d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:17.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:17.435+0000 7f8471c00700 1 -- 172.21.15.45:0/3710481560 >> 172.21.15.45:0/3710481560 conn(0x7f846c0fe020 msgr2=0x7f846c100460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:06:17.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:17.435+0000 7f8471c00700 1 -- 172.21.15.45:0/3710481560 shutdown_connections 2024-04-16T17:06:17.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:17.436+0000 7f8471c00700 1 -- 172.21.15.45:0/3710481560 wait complete. 2024-04-16T17:06:17.438 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:06:17 smithi045 ceph-mon[31676]: pgmap v412: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-16T17:06:17.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:17.437+0000 7f8471c00700 1 Processor -- start 2024-04-16T17:06:17.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:17.437+0000 7f8471c00700 1 -- start start 2024-04-16T17:06:17.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:17.438+0000 7f8471c00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f846c075550 0x7f846c19ae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:17.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:17.438+0000 7f8471c00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f846c076b20 0x7f846c19b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:17.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:17.438+0000 7f8471c00700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f846c10a950 0x7f846c1a23e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:17.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:17.438+0000 7f8471c00700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f846c110840 con 0x7f846c076b20 2024-04-16T17:06:17.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:17.438+0000 7f8471c00700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f846c1106c0 con 0x7f846c10a950 2024-04-16T17:06:17.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:17.438+0000 7f8471c00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f846c1109c0 con 0x7f846c075550 2024-04-16T17:06:17.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:17.438+0000 7f846be00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f846c075550 0x7f846c19ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:17.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:17.438+0000 7f846b400700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f846c076b20 0x7f846c19b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:17.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:17.438+0000 7f846be00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f846c075550 0x7f846c19ae20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:59576/0 (socket says 172.21.15.45:59576) 2024-04-16T17:06:17.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:17.438+0000 7f846be00700 1 -- 172.21.15.45:0/2858023646 learned_addr learned my addr 172.21.15.45:0/2858023646 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:06:17.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:17.438+0000 7f8471200700 1 --2- 172.21.15.45:0/2858023646 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f846c10a950 0x7f846c1a23e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:17.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:17.439+0000 7f846be00700 1 -- 172.21.15.45:0/2858023646 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f846c10a950 msgr2=0x7f846c1a23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:17.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:17.439+0000 7f846be00700 1 --2- 172.21.15.45:0/2858023646 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f846c10a950 0x7f846c1a23e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:17.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:17.439+0000 7f846be00700 1 -- 172.21.15.45:0/2858023646 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f846c076b20 msgr2=0x7f846c19b360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:17.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:17.439+0000 7f846be00700 1 --2- 172.21.15.45:0/2858023646 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f846c076b20 0x7f846c19b360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:17.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:17.439+0000 7f846be00700 1 -- 172.21.15.45:0/2858023646 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f846c1a2a80 con 0x7f846c075550 2024-04-16T17:06:17.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:17.439+0000 7f846be00700 1 --2- 172.21.15.45:0/2858023646 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f846c075550 0x7f846c19ae20 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f845c005f50 tx=0x7f845c00bbb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:06:17.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:17.439+0000 7f8468c00700 1 -- 172.21.15.45:0/2858023646 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f845c026070 con 0x7f846c075550 2024-04-16T17:06:17.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:17.439+0000 7f8468c00700 1 -- 172.21.15.45:0/2858023646 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f845c00f460 con 0x7f846c075550 2024-04-16T17:06:17.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:17.439+0000 7f8468c00700 1 -- 172.21.15.45:0/2858023646 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f845c021670 con 0x7f846c075550 2024-04-16T17:06:17.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:17.439+0000 7f8471c00700 1 -- 172.21.15.45:0/2858023646 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f846c1a2d10 con 0x7f846c075550 2024-04-16T17:06:17.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:17.440+0000 7f8471c00700 1 -- 172.21.15.45:0/2858023646 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f846c1a3110 con 0x7f846c075550 2024-04-16T17:06:17.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:17.441+0000 7f8471c00700 1 -- 172.21.15.45:0/2858023646 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f846c0697e0 con 0x7f846c075550 2024-04-16T17:06:17.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:17.443+0000 7f8468c00700 1 -- 172.21.15.45:0/2858023646 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f845c029070 con 0x7f846c075550 2024-04-16T17:06:17.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:17.444+0000 7f8468c00700 1 --2- 172.21.15.45:0/2858023646 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f84500a3cc0 0x7f84500a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:17.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:17.444+0000 7f8468c00700 1 -- 172.21.15.45:0/2858023646 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f845c0d15b0 con 0x7f846c075550 2024-04-16T17:06:17.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:17.445+0000 7f846b400700 1 --2- 172.21.15.45:0/2858023646 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f84500a3cc0 0x7f84500a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:17.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:17.445+0000 7f846b400700 1 --2- 172.21.15.45:0/2858023646 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f84500a3cc0 0x7f84500a6180 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f846c076850 tx=0x7f8458009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:06:17.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:17.449+0000 7f8468c00700 1 -- 172.21.15.45:0/2858023646 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f845c05dff0 con 0x7f846c075550 2024-04-16T17:06:17.555 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:06:17 smithi117 ceph-mon[36730]: pgmap v412: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-16T17:06:17.677 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:06:17 smithi104 ceph-mon[36457]: pgmap v412: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-16T17:06:17.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:17.688+0000 7f8471c00700 1 -- 172.21.15.45:0/2858023646 --> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7f846c062f30 con 0x7f84500a3cc0 2024-04-16T17:06:17.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:17.692+0000 7f8468c00700 1 -- 172.21.15.45:0/2858023646 <== mgr.14152 v2:172.21.15.45:6800/268030699 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7f846c062f30 con 0x7f84500a3cc0 2024-04-16T17:06:17.693 INFO:teuthology.orchestra.run.smithi045.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-04-16T17:06:17.694 INFO:teuthology.orchestra.run.smithi045.stdout:smithi045 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB75120098480DGN 447G No 4m ago Has a FileSystem, LVM detected 2024-04-16T17:06:17.694 INFO:teuthology.orchestra.run.smithi045.stdout:smithi104 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400NW400BGN 372G No 4m ago Has a FileSystem, LVM detected 2024-04-16T17:06:17.694 INFO:teuthology.orchestra.run.smithi045.stdout:smithi117 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400V6400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-04-16T17:06:17.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:17.695+0000 7f8471c00700 1 -- 172.21.15.45:0/2858023646 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f84500a3cc0 msgr2=0x7f84500a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:17.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:17.695+0000 7f8471c00700 1 --2- 172.21.15.45:0/2858023646 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f84500a3cc0 0x7f84500a6180 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f846c076850 tx=0x7f8458009260 comp rx=0 tx=0).stop 2024-04-16T17:06:17.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:17.695+0000 7f8471c00700 1 -- 172.21.15.45:0/2858023646 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f846c075550 msgr2=0x7f846c19ae20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:17.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:17.695+0000 7f8471c00700 1 --2- 172.21.15.45:0/2858023646 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f846c075550 0x7f846c19ae20 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f845c005f50 tx=0x7f845c00bbb0 comp rx=0 tx=0).stop 2024-04-16T17:06:17.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:17.695+0000 7f8471c00700 1 -- 172.21.15.45:0/2858023646 shutdown_connections 2024-04-16T17:06:17.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:17.695+0000 7f8471c00700 1 --2- 172.21.15.45:0/2858023646 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f84500a3cc0 0x7f84500a6180 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:17.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:17.695+0000 7f8471c00700 1 --2- 172.21.15.45:0/2858023646 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f846c075550 0x7f846c19ae20 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:17.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:17.695+0000 7f8471c00700 1 --2- 172.21.15.45:0/2858023646 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f846c10a950 0x7f846c1a23e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:17.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:17.696+0000 7f8471c00700 1 --2- 172.21.15.45:0/2858023646 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f846c076b20 0x7f846c19b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:17.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:17.696+0000 7f8471c00700 1 -- 172.21.15.45:0/2858023646 >> 172.21.15.45:0/2858023646 conn(0x7f846c0fe020 msgr2=0x7f846c106850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:06:17.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:17.696+0000 7f8471c00700 1 -- 172.21.15.45:0/2858023646 shutdown_connections 2024-04-16T17:06:17.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:17.696+0000 7f8471c00700 1 -- 172.21.15.45:0/2858023646 wait complete. 2024-04-16T17:06:18.477 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:06:18 smithi045 ceph-mon[31676]: from='client.25246 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T17:06:18.555 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:06:18 smithi117 ceph-mon[36730]: from='client.25246 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T17:06:18.646 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-16T17:06:18.658 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi045.front.sepia.ceph.com 2024-04-16T17:06:18.658 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b1fecd38-fc11-11ee-bc90-c7b262605968 -- bash -c 'ceph fs dump' 2024-04-16T17:06:18.677 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:06:18 smithi104 ceph-mon[36457]: from='client.25246 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T17:06:19.119 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/mon.a/config 2024-04-16T17:06:19.519 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:06:19 smithi045 ceph-mon[31676]: pgmap v413: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-16T17:06:19.555 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:06:19 smithi117 ceph-mon[36730]: pgmap v413: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-16T17:06:19.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:06:19 smithi104 ceph-mon[36457]: pgmap v413: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-16T17:06:21.477 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:06:21 smithi045 ceph-mon[31676]: pgmap v414: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-16T17:06:21.555 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:06:21 smithi117 ceph-mon[36730]: pgmap v414: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-16T17:06:21.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:21.606+0000 7f6277e00700 1 -- 172.21.15.45:0/2301579614 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6278076ac0 msgr2=0x7f6278074f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:21.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:21.606+0000 7f6277e00700 1 --2- 172.21.15.45:0/2301579614 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6278076ac0 0x7f6278074f20 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f6268009990 tx=0x7f6268009ca0 comp rx=0 tx=0).stop 2024-04-16T17:06:21.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:21.606+0000 7f6277e00700 1 -- 172.21.15.45:0/2301579614 shutdown_connections 2024-04-16T17:06:21.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:21.606+0000 7f6277e00700 1 --2- 172.21.15.45:0/2301579614 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f627810a960 0x7f627810cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:21.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:21.606+0000 7f6277e00700 1 --2- 172.21.15.45:0/2301579614 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6278076ac0 0x7f6278074f20 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:21.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:21.606+0000 7f6277e00700 1 --2- 172.21.15.45:0/2301579614 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f62780754f0 0x7f6278075970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:21.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:21.606+0000 7f6277e00700 1 -- 172.21.15.45:0/2301579614 >> 172.21.15.45:0/2301579614 conn(0x7f62780fe020 msgr2=0x7f6278100460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:06:21.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:21.606+0000 7f6277e00700 1 -- 172.21.15.45:0/2301579614 shutdown_connections 2024-04-16T17:06:21.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:21.606+0000 7f6277e00700 1 -- 172.21.15.45:0/2301579614 wait complete. 2024-04-16T17:06:21.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:21.607+0000 7f6277e00700 1 Processor -- start 2024-04-16T17:06:21.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:21.608+0000 7f6277e00700 1 -- start start 2024-04-16T17:06:21.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:21.608+0000 7f6277e00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f62780754f0 0x7f627819ae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:21.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:21.608+0000 7f6277e00700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6278076ac0 0x7f627819b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:21.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:21.608+0000 7f6277e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f627810a960 0x7f62781a23d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:21.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:21.608+0000 7f6277e00700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f6278110870 con 0x7f62780754f0 2024-04-16T17:06:21.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:21.608+0000 7f6277e00700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f62781106f0 con 0x7f6278076ac0 2024-04-16T17:06:21.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:21.608+0000 7f6277e00700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f62781109f0 con 0x7f627810a960 2024-04-16T17:06:21.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:21.608+0000 7f6276a00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f62780754f0 0x7f627819ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:21.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:21.608+0000 7f6276a00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f62780754f0 0x7f627819ae60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:32816/0 (socket says 172.21.15.45:32816) 2024-04-16T17:06:21.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:21.608+0000 7f6276a00700 1 -- 172.21.15.45:0/3810725162 learned_addr learned my addr 172.21.15.45:0/3810725162 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:06:21.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:21.608+0000 7f626fe00700 1 --2- 172.21.15.45:0/3810725162 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6278076ac0 0x7f627819b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:21.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:21.608+0000 7f6277400700 1 --2- 172.21.15.45:0/3810725162 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f627810a960 0x7f62781a23d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:21.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:21.609+0000 7f626fe00700 1 -- 172.21.15.45:0/3810725162 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f627810a960 msgr2=0x7f62781a23d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:21.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:21.609+0000 7f626fe00700 1 --2- 172.21.15.45:0/3810725162 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f627810a960 0x7f62781a23d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:21.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:21.609+0000 7f626fe00700 1 -- 172.21.15.45:0/3810725162 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f62780754f0 msgr2=0x7f627819ae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:21.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:21.609+0000 7f626fe00700 1 --2- 172.21.15.45:0/3810725162 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f62780754f0 0x7f627819ae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:21.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:21.609+0000 7f626fe00700 1 -- 172.21.15.45:0/3810725162 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62781a2a70 con 0x7f6278076ac0 2024-04-16T17:06:21.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:21.609+0000 7f6276a00700 1 --2- 172.21.15.45:0/3810725162 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f62780754f0 0x7f627819ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:06:21.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:21.609+0000 7f626fe00700 1 --2- 172.21.15.45:0/3810725162 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6278076ac0 0x7f627819b3a0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f626000d750 tx=0x7f626000da60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:06:21.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:21.610+0000 7f626f400700 1 -- 172.21.15.45:0/3810725162 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6260014030 con 0x7f6278076ac0 2024-04-16T17:06:21.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:21.610+0000 7f626f400700 1 -- 172.21.15.45:0/3810725162 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6260004500 con 0x7f6278076ac0 2024-04-16T17:06:21.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:21.610+0000 7f6277e00700 1 -- 172.21.15.45:0/3810725162 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62781a2d60 con 0x7f6278076ac0 2024-04-16T17:06:21.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:21.610+0000 7f626f400700 1 -- 172.21.15.45:0/3810725162 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62600054b0 con 0x7f6278076ac0 2024-04-16T17:06:21.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:21.610+0000 7f6277e00700 1 -- 172.21.15.45:0/3810725162 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62781a3260 con 0x7f6278076ac0 2024-04-16T17:06:21.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:21.611+0000 7f6277e00700 1 -- 172.21.15.45:0/3810725162 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f62780697e0 con 0x7f6278076ac0 2024-04-16T17:06:21.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:21.613+0000 7f626f400700 1 -- 172.21.15.45:0/3810725162 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f6260004750 con 0x7f6278076ac0 2024-04-16T17:06:21.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:21.615+0000 7f626f400700 1 --2- 172.21.15.45:0/3810725162 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f625c0a3cc0 0x7f625c0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:21.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:21.615+0000 7f626f400700 1 -- 172.21.15.45:0/3810725162 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f62600cffe0 con 0x7f6278076ac0 2024-04-16T17:06:21.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:21.615+0000 7f6276a00700 1 --2- 172.21.15.45:0/3810725162 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f625c0a3cc0 0x7f625c0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:21.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:21.616+0000 7f6276a00700 1 --2- 172.21.15.45:0/3810725162 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f625c0a3cc0 0x7f625c0a6180 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f626800b5c0 tx=0x7f6268003960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:06:21.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:21.619+0000 7f626f400700 1 -- 172.21.15.45:0/3810725162 <== mon.2 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f626009c5f0 con 0x7f6278076ac0 2024-04-16T17:06:21.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:06:21 smithi104 ceph-mon[36457]: pgmap v414: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-16T17:06:21.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:21.900+0000 7f6277e00700 1 -- 172.21.15.45:0/3810725162 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f627802dfe0 con 0x7f6278076ac0 2024-04-16T17:06:21.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:21.901+0000 7f626f400700 1 -- 172.21.15.45:0/3810725162 <== mon.2 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 24 v24) ==== 76+0+3062 (secure 0 0 0) 0x7f626009bef0 con 0x7f6278076ac0 2024-04-16T17:06:21.903 INFO:teuthology.orchestra.run.smithi045.stdout:e24 2024-04-16T17:06:21.903 INFO:teuthology.orchestra.run.smithi045.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-16T17:06:21.903 INFO:teuthology.orchestra.run.smithi045.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-16T17:06:21.903 INFO:teuthology.orchestra.run.smithi045.stdout:legacy client fscid: 1 2024-04-16T17:06:21.903 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-16T17:06:21.903 INFO:teuthology.orchestra.run.smithi045.stdout:Filesystem 'cephfs' (1) 2024-04-16T17:06:21.903 INFO:teuthology.orchestra.run.smithi045.stdout:fs_name cephfs 2024-04-16T17:06:21.903 INFO:teuthology.orchestra.run.smithi045.stdout:epoch 24 2024-04-16T17:06:21.903 INFO:teuthology.orchestra.run.smithi045.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-04-16T17:06:21.904 INFO:teuthology.orchestra.run.smithi045.stdout:created 2024-04-16T17:03:24.663611+0000 2024-04-16T17:06:21.904 INFO:teuthology.orchestra.run.smithi045.stdout:modified 2024-04-16T17:03:53.381209+0000 2024-04-16T17:06:21.904 INFO:teuthology.orchestra.run.smithi045.stdout:tableserver 0 2024-04-16T17:06:21.904 INFO:teuthology.orchestra.run.smithi045.stdout:root 0 2024-04-16T17:06:21.904 INFO:teuthology.orchestra.run.smithi045.stdout:session_timeout 300 2024-04-16T17:06:21.904 INFO:teuthology.orchestra.run.smithi045.stdout:session_autoclose 300 2024-04-16T17:06:21.904 INFO:teuthology.orchestra.run.smithi045.stdout:max_file_size 1099511627776 2024-04-16T17:06:21.904 INFO:teuthology.orchestra.run.smithi045.stdout:max_xattr_size 65536 2024-04-16T17:06:21.904 INFO:teuthology.orchestra.run.smithi045.stdout:required_client_features {} 2024-04-16T17:06:21.904 INFO:teuthology.orchestra.run.smithi045.stdout:last_failure 0 2024-04-16T17:06:21.904 INFO:teuthology.orchestra.run.smithi045.stdout:last_failure_osd_epoch 0 2024-04-16T17:06:21.904 INFO:teuthology.orchestra.run.smithi045.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-16T17:06:21.904 INFO:teuthology.orchestra.run.smithi045.stdout:max_mds 3 2024-04-16T17:06:21.904 INFO:teuthology.orchestra.run.smithi045.stdout:in 0,1,2 2024-04-16T17:06:21.904 INFO:teuthology.orchestra.run.smithi045.stdout:up {0=24581,1=24569,2=24532} 2024-04-16T17:06:21.905 INFO:teuthology.orchestra.run.smithi045.stdout:failed 2024-04-16T17:06:21.905 INFO:teuthology.orchestra.run.smithi045.stdout:damaged 2024-04-16T17:06:21.905 INFO:teuthology.orchestra.run.smithi045.stdout:stopped 2024-04-16T17:06:21.905 INFO:teuthology.orchestra.run.smithi045.stdout:data_pools [3,4] 2024-04-16T17:06:21.905 INFO:teuthology.orchestra.run.smithi045.stdout:metadata_pool 2 2024-04-16T17:06:21.905 INFO:teuthology.orchestra.run.smithi045.stdout:inline_data disabled 2024-04-16T17:06:21.905 INFO:teuthology.orchestra.run.smithi045.stdout:balancer 2024-04-16T17:06:21.905 INFO:teuthology.orchestra.run.smithi045.stdout:bal_rank_mask -1 2024-04-16T17:06:21.905 INFO:teuthology.orchestra.run.smithi045.stdout:standby_count_wanted 0 2024-04-16T17:06:21.905 INFO:teuthology.orchestra.run.smithi045.stdout:qdb_cluster leader: 24581 members: 24569,24532,24581 2024-04-16T17:06:21.905 INFO:teuthology.orchestra.run.smithi045.stdout:[mds.b{0:24581} state up:active seq 3 addr [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] compat {c=[1],r=[1],i=[1fff]}] 2024-04-16T17:06:21.905 INFO:teuthology.orchestra.run.smithi045.stdout:[mds.h{0:24524} state up:standby-replay seq 1 addr [v2:172.21.15.104:6836/2647756509,v1:172.21.15.104:6837/2647756509] compat {c=[1],r=[1],i=[1fff]}] 2024-04-16T17:06:21.905 INFO:teuthology.orchestra.run.smithi045.stdout:[mds.a{1:24569} state up:active seq 10 addr [v2:172.21.15.45:6840/3755310822,v1:172.21.15.45:6841/3755310822] compat {c=[1],r=[1],i=[1fff]}] 2024-04-16T17:06:21.906 INFO:teuthology.orchestra.run.smithi045.stdout:[mds.c{1:24511} state up:standby-replay seq 1 addr [v2:172.21.15.117:6836/2471356815,v1:172.21.15.117:6837/2471356815] compat {c=[1],r=[1],i=[1fff]}] 2024-04-16T17:06:21.906 INFO:teuthology.orchestra.run.smithi045.stdout:[mds.i{2:24532} state up:active seq 13 addr [v2:172.21.15.117:6838/2378675949,v1:172.21.15.117:6839/2378675949] compat {c=[1],r=[1],i=[1fff]}] 2024-04-16T17:06:21.906 INFO:teuthology.orchestra.run.smithi045.stdout:[mds.f{2:24499} state up:standby-replay seq 1 addr [v2:172.21.15.117:6834/2224123841,v1:172.21.15.117:6835/2224123841] compat {c=[1],r=[1],i=[1fff]}] 2024-04-16T17:06:21.906 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-16T17:06:21.906 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-16T17:06:21.906 INFO:teuthology.orchestra.run.smithi045.stdout:Standby daemons: 2024-04-16T17:06:21.906 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-16T17:06:21.906 INFO:teuthology.orchestra.run.smithi045.stdout:[mds.j{-1:14382} state up:standby seq 1 addr [v2:172.21.15.45:6834/1748842443,v1:172.21.15.45:6835/1748842443] compat {c=[1],r=[1],i=[1fff]}] 2024-04-16T17:06:21.906 INFO:teuthology.orchestra.run.smithi045.stdout:[mds.g{-1:14448} state up:standby seq 1 addr [v2:172.21.15.45:6838/1308627051,v1:172.21.15.45:6839/1308627051] compat {c=[1],r=[1],i=[1fff]}] 2024-04-16T17:06:21.906 INFO:teuthology.orchestra.run.smithi045.stdout:[mds.l{-1:24430} state up:standby seq 1 addr [v2:172.21.15.117:6832/1768242489,v1:172.21.15.117:6833/1768242489] compat {c=[1],r=[1],i=[1fff]}] 2024-04-16T17:06:21.906 INFO:teuthology.orchestra.run.smithi045.stdout:[mds.k{-1:24443} state up:standby seq 1 addr [v2:172.21.15.104:6832/3558902585,v1:172.21.15.104:6833/3558902585] compat {c=[1],r=[1],i=[1fff]}] 2024-04-16T17:06:21.906 INFO:teuthology.orchestra.run.smithi045.stdout:[mds.d{-1:24478} state up:standby seq 1 addr [v2:172.21.15.45:6836/1308142903,v1:172.21.15.45:6837/1308142903] compat {c=[1],r=[1],i=[1fff]}] 2024-04-16T17:06:21.906 INFO:teuthology.orchestra.run.smithi045.stdout:[mds.e{-1:24491} state up:standby seq 1 addr [v2:172.21.15.104:6834/3430679937,v1:172.21.15.104:6835/3430679937] compat {c=[1],r=[1],i=[1fff]}] 2024-04-16T17:06:21.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:21.904+0000 7f6277e00700 1 -- 172.21.15.45:0/3810725162 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f625c0a3cc0 msgr2=0x7f625c0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:21.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:21.904+0000 7f6277e00700 1 --2- 172.21.15.45:0/3810725162 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f625c0a3cc0 0x7f625c0a6180 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f626800b5c0 tx=0x7f6268003960 comp rx=0 tx=0).stop 2024-04-16T17:06:21.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:21.904+0000 7f6277e00700 1 -- 172.21.15.45:0/3810725162 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6278076ac0 msgr2=0x7f627819b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:21.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:21.904+0000 7f6277e00700 1 --2- 172.21.15.45:0/3810725162 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6278076ac0 0x7f627819b3a0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f626000d750 tx=0x7f626000da60 comp rx=0 tx=0).stop 2024-04-16T17:06:21.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:21.904+0000 7f6277e00700 1 -- 172.21.15.45:0/3810725162 shutdown_connections 2024-04-16T17:06:21.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:21.904+0000 7f6277e00700 1 --2- 172.21.15.45:0/3810725162 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f625c0a3cc0 0x7f625c0a6180 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:21.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:21.904+0000 7f6277e00700 1 --2- 172.21.15.45:0/3810725162 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f627810a960 0x7f62781a23d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:21.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:21.904+0000 7f6277e00700 1 --2- 172.21.15.45:0/3810725162 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f62780754f0 0x7f627819ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:21.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:21.904+0000 7f6277e00700 1 --2- 172.21.15.45:0/3810725162 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6278076ac0 0x7f627819b3a0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:21.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:21.904+0000 7f6277e00700 1 -- 172.21.15.45:0/3810725162 >> 172.21.15.45:0/3810725162 conn(0x7f62780fe020 msgr2=0x7f62781068b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:06:21.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:21.904+0000 7f6277e00700 1 -- 172.21.15.45:0/3810725162 shutdown_connections 2024-04-16T17:06:21.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:21.905+0000 7f6277e00700 1 -- 172.21.15.45:0/3810725162 wait complete. 2024-04-16T17:06:21.908 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 24 2024-04-16T17:06:22.555 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:06:22 smithi117 ceph-mon[36730]: from='client.? 172.21.15.45:0/3810725162' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-16T17:06:22.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:06:22 smithi104 ceph-mon[36457]: from='client.? 172.21.15.45:0/3810725162' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-16T17:06:22.687 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:06:22 smithi045 ceph-mon[31676]: from='client.? 172.21.15.45:0/3810725162' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-16T17:06:22.764 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b1fecd38-fc11-11ee-bc90-c7b262605968 -- bash -c 'ceph osd dump' 2024-04-16T17:06:23.258 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/mon.a/config 2024-04-16T17:06:23.555 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:06:23 smithi117 ceph-mon[36730]: pgmap v415: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-16T17:06:23.598 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:06:23 smithi045 ceph-mon[31676]: pgmap v415: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-16T17:06:23.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:06:23 smithi104 ceph-mon[36457]: pgmap v415: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-16T17:06:25.432 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:06:25 smithi045 ceph-mon[31676]: pgmap v416: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T17:06:25.555 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:06:25 smithi117 ceph-mon[36730]: pgmap v416: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T17:06:25.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:25.612+0000 7ff099200700 1 -- 172.21.15.45:0/1746680425 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff094102b60 msgr2=0x7ff094102f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:25.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:25.612+0000 7ff099200700 1 --2- 172.21.15.45:0/1746680425 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff094102b60 0x7ff094102f80 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7ff084009920 tx=0x7ff084009c30 comp rx=0 tx=0).stop 2024-04-16T17:06:25.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:25.613+0000 7ff099200700 1 -- 172.21.15.45:0/1746680425 shutdown_connections 2024-04-16T17:06:25.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:25.613+0000 7ff099200700 1 --2- 172.21.15.45:0/1746680425 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff094104700 0x7ff09410afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:25.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:25.613+0000 7ff099200700 1 --2- 172.21.15.45:0/1746680425 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff094102b60 0x7ff094102f80 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:25.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:25.613+0000 7ff099200700 1 --2- 172.21.15.45:0/1746680425 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff094103d60 0x7ff0941041c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:25.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:25.613+0000 7ff099200700 1 -- 172.21.15.45:0/1746680425 >> 172.21.15.45:0/1746680425 conn(0x7ff0940fe0c0 msgr2=0x7ff094100540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:06:25.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:25.613+0000 7ff099200700 1 -- 172.21.15.45:0/1746680425 shutdown_connections 2024-04-16T17:06:25.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:25.613+0000 7ff099200700 1 -- 172.21.15.45:0/1746680425 wait complete. 2024-04-16T17:06:25.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:25.614+0000 7ff099200700 1 Processor -- start 2024-04-16T17:06:25.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:25.614+0000 7ff099200700 1 -- start start 2024-04-16T17:06:25.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:25.614+0000 7ff099200700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff094102b60 0x7ff09419b050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:25.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:25.614+0000 7ff099200700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff094103d60 0x7ff09419b590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:25.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:25.614+0000 7ff099200700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff094104700 0x7ff0941a2610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:25.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:25.614+0000 7ff099200700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7ff09410e770 con 0x7ff094104700 2024-04-16T17:06:25.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:25.614+0000 7ff099200700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7ff09410e5f0 con 0x7ff094103d60 2024-04-16T17:06:25.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:25.614+0000 7ff099200700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7ff09410e8f0 con 0x7ff094102b60 2024-04-16T17:06:25.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:25.615+0000 7ff091e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff094102b60 0x7ff09419b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:25.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:25.615+0000 7ff092800700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff094104700 0x7ff0941a2610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:25.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:25.615+0000 7ff091400700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff094103d60 0x7ff09419b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:25.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:25.615+0000 7ff091e00700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff094102b60 0x7ff09419b050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:56352/0 (socket says 172.21.15.45:56352) 2024-04-16T17:06:25.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:25.615+0000 7ff091e00700 1 -- 172.21.15.45:0/3801584309 learned_addr learned my addr 172.21.15.45:0/3801584309 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:06:25.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:25.615+0000 7ff092800700 1 -- 172.21.15.45:0/3801584309 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff094102b60 msgr2=0x7ff09419b050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:25.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:25.615+0000 7ff092800700 1 --2- 172.21.15.45:0/3801584309 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff094102b60 0x7ff09419b050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:25.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:25.615+0000 7ff092800700 1 -- 172.21.15.45:0/3801584309 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff094103d60 msgr2=0x7ff09419b590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:25.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:25.615+0000 7ff092800700 1 --2- 172.21.15.45:0/3801584309 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff094103d60 0x7ff09419b590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:25.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:25.615+0000 7ff092800700 1 -- 172.21.15.45:0/3801584309 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0941a2cb0 con 0x7ff094104700 2024-04-16T17:06:25.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:25.615+0000 7ff091400700 1 --2- 172.21.15.45:0/3801584309 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff094103d60 0x7ff09419b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:06:25.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:25.616+0000 7ff092800700 1 --2- 172.21.15.45:0/3801584309 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff094104700 0x7ff0941a2610 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7ff08c009f60 tx=0x7ff08c009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:06:25.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:25.616+0000 7ff08aa00700 1 -- 172.21.15.45:0/3801584309 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff08c0092e0 con 0x7ff094104700 2024-04-16T17:06:25.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:25.616+0000 7ff08aa00700 1 -- 172.21.15.45:0/3801584309 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff08c004bb0 con 0x7ff094104700 2024-04-16T17:06:25.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:25.616+0000 7ff099200700 1 -- 172.21.15.45:0/3801584309 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0941a2f40 con 0x7ff094104700 2024-04-16T17:06:25.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:25.616+0000 7ff08aa00700 1 -- 172.21.15.45:0/3801584309 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff08c004280 con 0x7ff094104700 2024-04-16T17:06:25.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:25.616+0000 7ff099200700 1 -- 172.21.15.45:0/3801584309 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0941a33a0 con 0x7ff094104700 2024-04-16T17:06:25.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:25.617+0000 7ff099200700 1 -- 172.21.15.45:0/3801584309 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff094069810 con 0x7ff094104700 2024-04-16T17:06:25.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:25.619+0000 7ff08aa00700 1 -- 172.21.15.45:0/3801584309 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7ff08c01b030 con 0x7ff094104700 2024-04-16T17:06:25.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:25.621+0000 7ff08aa00700 1 --2- 172.21.15.45:0/3801584309 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7ff0780a3c40 0x7ff0780a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:25.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:25.621+0000 7ff08aa00700 1 -- 172.21.15.45:0/3801584309 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7ff08c0d01d0 con 0x7ff094104700 2024-04-16T17:06:25.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:25.621+0000 7ff091e00700 1 --2- 172.21.15.45:0/3801584309 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7ff0780a3c40 0x7ff0780a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:25.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:25.622+0000 7ff091e00700 1 --2- 172.21.15.45:0/3801584309 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7ff0780a3c40 0x7ff0780a6100 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7ff08400fca0 tx=0x7ff084019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:06:25.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:25.626+0000 7ff08aa00700 1 -- 172.21.15.45:0/3801584309 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff094069810 con 0x7ff094104700 2024-04-16T17:06:25.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:06:25 smithi104 ceph-mon[36457]: pgmap v416: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T17:06:25.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:25.858+0000 7ff099200700 1 -- 172.21.15.45:0/3801584309 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7ff0941a3620 con 0x7ff094104700 2024-04-16T17:06:25.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:25.861+0000 7ff08aa00700 1 -- 172.21.15.45:0/3801584309 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v94) ==== 56+0+5663 (secure 0 0 0) 0x7ff08c022020 con 0x7ff094104700 2024-04-16T17:06:25.863 INFO:teuthology.orchestra.run.smithi045.stdout:epoch 94 2024-04-16T17:06:25.863 INFO:teuthology.orchestra.run.smithi045.stdout:fsid b1fecd38-fc11-11ee-bc90-c7b262605968 2024-04-16T17:06:25.863 INFO:teuthology.orchestra.run.smithi045.stdout:created 2024-04-16T16:54:02.385906+0000 2024-04-16T17:06:25.863 INFO:teuthology.orchestra.run.smithi045.stdout:modified 2024-04-16T17:05:22.272335+0000 2024-04-16T17:06:25.863 INFO:teuthology.orchestra.run.smithi045.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-04-16T17:06:25.863 INFO:teuthology.orchestra.run.smithi045.stdout:crush_version 27 2024-04-16T17:06:25.863 INFO:teuthology.orchestra.run.smithi045.stdout:full_ratio 0.95 2024-04-16T17:06:25.863 INFO:teuthology.orchestra.run.smithi045.stdout:backfillfull_ratio 0.9 2024-04-16T17:06:25.863 INFO:teuthology.orchestra.run.smithi045.stdout:nearfull_ratio 0.85 2024-04-16T17:06:25.863 INFO:teuthology.orchestra.run.smithi045.stdout:require_min_compat_client luminous 2024-04-16T17:06:25.863 INFO:teuthology.orchestra.run.smithi045.stdout:min_compat_client luminous 2024-04-16T17:06:25.863 INFO:teuthology.orchestra.run.smithi045.stdout:require_osd_release squid 2024-04-16T17:06:25.863 INFO:teuthology.orchestra.run.smithi045.stdout:stretch_mode_enabled false 2024-04-16T17:06:25.864 INFO:teuthology.orchestra.run.smithi045.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 22 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-04-16T17:06:25.864 INFO:teuthology.orchestra.run.smithi045.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 85 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 2.25 2024-04-16T17:06:25.864 INFO:teuthology.orchestra.run.smithi045.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 86 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.05 application cephfs read_balance_score 1.50 2024-04-16T17:06:25.864 INFO:teuthology.orchestra.run.smithi045.stdout:pool 4 'cephfs_data_ec' erasure profile cephfs_data_ec size 4 min_size 3 crush_rule 1 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 90 flags hashpspool,ec_overwrites stripe_width 8192 pg_num_min 64 target_size_ratio 0.9 application cephfs 2024-04-16T17:06:25.864 INFO:teuthology.orchestra.run.smithi045.stdout:max_osd 12 2024-04-16T17:06:25.864 INFO:teuthology.orchestra.run.smithi045.stdout:osd.0 up in weight 1 up_from 8 up_thru 91 down_at 0 last_clean_interval [0,0) [v2:172.21.15.45:6802/2290170881,v1:172.21.15.45:6803/2290170881] [v2:172.21.15.45:6804/2290170881,v1:172.21.15.45:6805/2290170881] exists,up b655de2c-59f8-4fd4-ac98-0cc222229f56 2024-04-16T17:06:25.864 INFO:teuthology.orchestra.run.smithi045.stdout:osd.1 up in weight 1 up_from 13 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.104:6800/1462942067,v1:172.21.15.104:6801/1462942067] [v2:172.21.15.104:6802/1462942067,v1:172.21.15.104:6803/1462942067] exists,up ebd450ec-08fd-4c3d-8b24-4e10634ddb7e 2024-04-16T17:06:25.864 INFO:teuthology.orchestra.run.smithi045.stdout:osd.2 up in weight 1 up_from 18 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.117:6800/2829260735,v1:172.21.15.117:6801/2829260735] [v2:172.21.15.117:6802/2829260735,v1:172.21.15.117:6803/2829260735] exists,up b318a512-4190-42a2-8104-dcc4026ad8ad 2024-04-16T17:06:25.864 INFO:teuthology.orchestra.run.smithi045.stdout:osd.3 up in weight 1 up_from 26 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.45:6810/2823712137,v1:172.21.15.45:6811/2823712137] [v2:172.21.15.45:6812/2823712137,v1:172.21.15.45:6813/2823712137] exists,up ec6490cb-a715-4c12-a54a-225aae26fcb9 2024-04-16T17:06:25.864 INFO:teuthology.orchestra.run.smithi045.stdout:osd.4 up in weight 1 up_from 34 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.104:6808/1396278127,v1:172.21.15.104:6809/1396278127] [v2:172.21.15.104:6810/1396278127,v1:172.21.15.104:6811/1396278127] exists,up 5d91d568-0250-4aae-8dc2-a0e079f92062 2024-04-16T17:06:25.864 INFO:teuthology.orchestra.run.smithi045.stdout:osd.5 up in weight 1 up_from 39 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.117:6808/4145720047,v1:172.21.15.117:6809/4145720047] [v2:172.21.15.117:6810/4145720047,v1:172.21.15.117:6811/4145720047] exists,up ac510b43-6fb2-4d0f-bc5f-16ac080d9e9d 2024-04-16T17:06:25.864 INFO:teuthology.orchestra.run.smithi045.stdout:osd.6 up in weight 1 up_from 47 up_thru 93 down_at 0 last_clean_interval [0,0) [v2:172.21.15.45:6818/3254229176,v1:172.21.15.45:6819/3254229176] [v2:172.21.15.45:6820/3254229176,v1:172.21.15.45:6821/3254229176] exists,up a41adc18-d88a-419e-b2c2-b089c1f32472 2024-04-16T17:06:25.864 INFO:teuthology.orchestra.run.smithi045.stdout:osd.7 up in weight 1 up_from 52 up_thru 91 down_at 0 last_clean_interval [0,0) [v2:172.21.15.104:6816/4151077186,v1:172.21.15.104:6817/4151077186] [v2:172.21.15.104:6818/4151077186,v1:172.21.15.104:6819/4151077186] exists,up b71712a4-7cf6-454d-9fd1-565e3df21c1b 2024-04-16T17:06:25.864 INFO:teuthology.orchestra.run.smithi045.stdout:osd.8 up in weight 1 up_from 60 up_thru 93 down_at 0 last_clean_interval [0,0) [v2:172.21.15.117:6816/1500286830,v1:172.21.15.117:6817/1500286830] [v2:172.21.15.117:6818/1500286830,v1:172.21.15.117:6819/1500286830] exists,up 9e572aef-59a8-450e-a92e-df9dff4d57a4 2024-04-16T17:06:25.865 INFO:teuthology.orchestra.run.smithi045.stdout:osd.9 up in weight 1 up_from 68 up_thru 91 down_at 0 last_clean_interval [0,0) [v2:172.21.15.45:6826/3106734074,v1:172.21.15.45:6827/3106734074] [v2:172.21.15.45:6828/3106734074,v1:172.21.15.45:6829/3106734074] exists,up 014e9598-c413-4cc4-9bb8-61c6e8f59d30 2024-04-16T17:06:25.865 INFO:teuthology.orchestra.run.smithi045.stdout:osd.10 up in weight 1 up_from 73 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.104:6824/1404794652,v1:172.21.15.104:6825/1404794652] [v2:172.21.15.104:6826/1404794652,v1:172.21.15.104:6827/1404794652] exists,up 2deba364-3e39-47be-8a1f-8256df5b82d6 2024-04-16T17:06:25.865 INFO:teuthology.orchestra.run.smithi045.stdout:osd.11 up in weight 1 up_from 81 up_thru 91 down_at 0 last_clean_interval [0,0) [v2:172.21.15.117:6824/3384003243,v1:172.21.15.117:6825/3384003243] [v2:172.21.15.117:6826/3384003243,v1:172.21.15.117:6827/3384003243] exists,up 3fda4fea-59ad-4d1a-9fe6-fcb30e31fd7d 2024-04-16T17:06:25.865 INFO:teuthology.orchestra.run.smithi045.stdout:pg_upmap_items 2.a [2,11] 2024-04-16T17:06:25.865 INFO:teuthology.orchestra.run.smithi045.stdout:pg_upmap_items 2.13 [2,11] 2024-04-16T17:06:25.865 INFO:teuthology.orchestra.run.smithi045.stdout:pg_upmap_items 2.1d [2,4] 2024-04-16T17:06:25.865 INFO:teuthology.orchestra.run.smithi045.stdout:pg_upmap_items 2.1f [8,1] 2024-04-16T17:06:25.865 INFO:teuthology.orchestra.run.smithi045.stdout:pg_upmap_items 3.c [3,9] 2024-04-16T17:06:25.865 INFO:teuthology.orchestra.run.smithi045.stdout:pg_upmap_items 3.e [4,8] 2024-04-16T17:06:25.865 INFO:teuthology.orchestra.run.smithi045.stdout:pg_upmap_items 3.25 [5,9] 2024-04-16T17:06:25.865 INFO:teuthology.orchestra.run.smithi045.stdout:pg_upmap_items 3.3c [4,9] 2024-04-16T17:06:25.866 INFO:teuthology.orchestra.run.smithi045.stdout:pg_upmap_items 4.2 [9,1] 2024-04-16T17:06:25.866 INFO:teuthology.orchestra.run.smithi045.stdout:pg_upmap_items 4.5 [9,10] 2024-04-16T17:06:25.866 INFO:teuthology.orchestra.run.smithi045.stdout:pg_upmap_items 4.7 [5,8] 2024-04-16T17:06:25.866 INFO:teuthology.orchestra.run.smithi045.stdout:pg_upmap_items 4.b [9,8] 2024-04-16T17:06:25.866 INFO:teuthology.orchestra.run.smithi045.stdout:blocklist 172.21.15.45:0/4074982598 expires 2024-04-17T16:55:19.948001+0000 2024-04-16T17:06:25.866 INFO:teuthology.orchestra.run.smithi045.stdout:blocklist 172.21.15.45:0/2443572576 expires 2024-04-17T16:55:19.948001+0000 2024-04-16T17:06:25.866 INFO:teuthology.orchestra.run.smithi045.stdout:blocklist 172.21.15.45:6801/1983630136 expires 2024-04-17T16:55:19.948001+0000 2024-04-16T17:06:25.866 INFO:teuthology.orchestra.run.smithi045.stdout:blocklist 172.21.15.45:6800/1983630136 expires 2024-04-17T16:55:19.948001+0000 2024-04-16T17:06:25.866 INFO:teuthology.orchestra.run.smithi045.stdout:blocklist 172.21.15.45:0/3150465248 expires 2024-04-17T16:54:43.692466+0000 2024-04-16T17:06:25.866 INFO:teuthology.orchestra.run.smithi045.stdout:blocklist 172.21.15.45:0/556738623 expires 2024-04-17T16:54:43.692466+0000 2024-04-16T17:06:25.866 INFO:teuthology.orchestra.run.smithi045.stdout:blocklist 172.21.15.45:0/445676297 expires 2024-04-17T16:55:19.948001+0000 2024-04-16T17:06:25.866 INFO:teuthology.orchestra.run.smithi045.stdout:blocklist 172.21.15.45:6800/2553659622 expires 2024-04-17T16:54:43.692466+0000 2024-04-16T17:06:25.866 INFO:teuthology.orchestra.run.smithi045.stdout:blocklist 172.21.15.45:6801/2553659622 expires 2024-04-17T16:54:43.692466+0000 2024-04-16T17:06:25.866 INFO:teuthology.orchestra.run.smithi045.stdout:blocklist 172.21.15.45:0/1184615408 expires 2024-04-17T16:54:43.692466+0000 2024-04-16T17:06:25.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:25.864+0000 7ff099200700 1 -- 172.21.15.45:0/3801584309 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7ff0780a3c40 msgr2=0x7ff0780a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:25.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:25.864+0000 7ff099200700 1 --2- 172.21.15.45:0/3801584309 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7ff0780a3c40 0x7ff0780a6100 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7ff08400fca0 tx=0x7ff084019040 comp rx=0 tx=0).stop 2024-04-16T17:06:25.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:25.864+0000 7ff099200700 1 -- 172.21.15.45:0/3801584309 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff094104700 msgr2=0x7ff0941a2610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:25.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:25.864+0000 7ff099200700 1 --2- 172.21.15.45:0/3801584309 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff094104700 0x7ff0941a2610 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7ff08c009f60 tx=0x7ff08c009f90 comp rx=0 tx=0).stop 2024-04-16T17:06:25.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:25.864+0000 7ff099200700 1 -- 172.21.15.45:0/3801584309 shutdown_connections 2024-04-16T17:06:25.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:25.864+0000 7ff099200700 1 --2- 172.21.15.45:0/3801584309 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7ff0780a3c40 0x7ff0780a6100 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:25.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:25.864+0000 7ff099200700 1 --2- 172.21.15.45:0/3801584309 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff094102b60 0x7ff09419b050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:25.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:25.864+0000 7ff099200700 1 --2- 172.21.15.45:0/3801584309 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff094104700 0x7ff0941a2610 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:25.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:25.864+0000 7ff099200700 1 --2- 172.21.15.45:0/3801584309 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff094103d60 0x7ff09419b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:25.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:25.864+0000 7ff099200700 1 -- 172.21.15.45:0/3801584309 >> 172.21.15.45:0/3801584309 conn(0x7ff0940fe0c0 msgr2=0x7ff094074d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:06:25.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:25.865+0000 7ff099200700 1 -- 172.21.15.45:0/3801584309 shutdown_connections 2024-04-16T17:06:25.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:25.865+0000 7ff099200700 1 -- 172.21.15.45:0/3801584309 wait complete. 2024-04-16T17:06:26.555 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:06:26 smithi117 ceph-mon[36730]: from='client.? 172.21.15.45:0/3801584309' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-16T17:06:26.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:06:26 smithi104 ceph-mon[36457]: from='client.? 172.21.15.45:0/3801584309' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-16T17:06:26.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:06:26 smithi045 ceph-mon[31676]: from='client.? 172.21.15.45:0/3801584309' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-16T17:06:27.005 INFO:teuthology.run_tasks:Running task fs.ready... 2024-04-16T17:06:27.017 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-16T17:06:27.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:27.242+0000 7fda52000640 1 -- 172.21.15.45:0/4289295211 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fda4c102ff0 msgr2=0x7fda4c103470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:27.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:27.242+0000 7fda52000640 1 --2- 172.21.15.45:0/4289295211 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fda4c102ff0 0x7fda4c103470 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fda38009860 tx=0x7fda3802bb70 comp rx=0 tx=0).stop 2024-04-16T17:06:27.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:27.242+0000 7fda52000640 1 -- 172.21.15.45:0/4289295211 shutdown_connections 2024-04-16T17:06:27.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:27.242+0000 7fda52000640 1 --2- 172.21.15.45:0/4289295211 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fda4c1039b0 0x7fda4c10a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:27.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:27.242+0000 7fda52000640 1 --2- 172.21.15.45:0/4289295211 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fda4c102ff0 0x7fda4c103470 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:27.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:27.242+0000 7fda52000640 1 --2- 172.21.15.45:0/4289295211 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fda4c101ea0 0x7fda4c1022c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:27.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:27.242+0000 7fda52000640 1 -- 172.21.15.45:0/4289295211 >> 172.21.15.45:0/4289295211 conn(0x7fda4c0fd410 msgr2=0x7fda4c0ff850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:06:27.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:27.242+0000 7fda52000640 1 -- 172.21.15.45:0/4289295211 shutdown_connections 2024-04-16T17:06:27.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:27.242+0000 7fda52000640 1 -- 172.21.15.45:0/4289295211 wait complete. 2024-04-16T17:06:27.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:27.243+0000 7fda52000640 1 Processor -- start 2024-04-16T17:06:27.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:27.243+0000 7fda52000640 1 -- start start 2024-04-16T17:06:27.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:27.243+0000 7fda52000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fda4c101ea0 0x7fda4c1981c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:27.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:27.243+0000 7fda52000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fda4c102ff0 0x7fda4c198700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:27.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:27.243+0000 7fda52000640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fda4c1039b0 0x7fda4c19f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:27.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:27.243+0000 7fda52000640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fda4c10db40 con 0x7fda4c101ea0 2024-04-16T17:06:27.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:27.243+0000 7fda52000640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fda4c10d9c0 con 0x7fda4c1039b0 2024-04-16T17:06:27.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:27.243+0000 7fda52000640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fda4c10dcc0 con 0x7fda4c102ff0 2024-04-16T17:06:27.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:27.244+0000 7fda50c00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fda4c101ea0 0x7fda4c1981c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:27.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:27.244+0000 7fda50c00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fda4c101ea0 0x7fda4c1981c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:32840/0 (socket says 172.21.15.45:32840) 2024-04-16T17:06:27.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:27.244+0000 7fda50c00640 1 -- 172.21.15.45:0/1209437453 learned_addr learned my addr 172.21.15.45:0/1209437453 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:06:27.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:27.244+0000 7fda4be00640 1 --2- 172.21.15.45:0/1209437453 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fda4c102ff0 0x7fda4c198700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:27.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:27.244+0000 7fda51600640 1 --2- 172.21.15.45:0/1209437453 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fda4c1039b0 0x7fda4c19f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:27.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:27.244+0000 7fda50c00640 1 -- 172.21.15.45:0/1209437453 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fda4c102ff0 msgr2=0x7fda4c198700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:27.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:27.244+0000 7fda50c00640 1 --2- 172.21.15.45:0/1209437453 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fda4c102ff0 0x7fda4c198700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:27.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:27.244+0000 7fda50c00640 1 -- 172.21.15.45:0/1209437453 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fda4c1039b0 msgr2=0x7fda4c19f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:27.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:27.244+0000 7fda50c00640 1 --2- 172.21.15.45:0/1209437453 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fda4c1039b0 0x7fda4c19f720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:27.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:27.244+0000 7fda50c00640 1 -- 172.21.15.45:0/1209437453 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda4c19fd90 con 0x7fda4c101ea0 2024-04-16T17:06:27.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:27.244+0000 7fda4be00640 1 --2- 172.21.15.45:0/1209437453 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fda4c102ff0 0x7fda4c198700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:06:27.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:27.244+0000 7fda51600640 1 --2- 172.21.15.45:0/1209437453 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fda4c1039b0 0x7fda4c19f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:06:27.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:27.245+0000 7fda50c00640 1 --2- 172.21.15.45:0/1209437453 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fda4c101ea0 0x7fda4c1981c0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fda3c00e9f0 tx=0x7fda3c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:06:27.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:27.245+0000 7fda49600640 1 -- 172.21.15.45:0/1209437453 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda3c00cd80 con 0x7fda4c101ea0 2024-04-16T17:06:27.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:27.245+0000 7fda49600640 1 -- 172.21.15.45:0/1209437453 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fda3c010910 con 0x7fda4c101ea0 2024-04-16T17:06:27.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:27.245+0000 7fda52000640 1 -- 172.21.15.45:0/1209437453 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda4c1a0080 con 0x7fda4c101ea0 2024-04-16T17:06:27.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:27.245+0000 7fda49600640 1 -- 172.21.15.45:0/1209437453 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda3c018990 con 0x7fda4c101ea0 2024-04-16T17:06:27.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:27.245+0000 7fda52000640 1 -- 172.21.15.45:0/1209437453 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda4c1a05a0 con 0x7fda4c101ea0 2024-04-16T17:06:27.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:27.246+0000 7fda52000640 1 -- 172.21.15.45:0/1209437453 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda4c1092d0 con 0x7fda4c101ea0 2024-04-16T17:06:27.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:27.249+0000 7fda49600640 1 -- 172.21.15.45:0/1209437453 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fda3c010ac0 con 0x7fda4c101ea0 2024-04-16T17:06:27.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:27.250+0000 7fda49600640 1 --2- 172.21.15.45:0/1209437453 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fda300a3bb0 0x7fda300a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:27.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:27.250+0000 7fda49600640 1 -- 172.21.15.45:0/1209437453 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fda3c0d0b10 con 0x7fda4c101ea0 2024-04-16T17:06:27.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:27.250+0000 7fda4be00640 1 --2- 172.21.15.45:0/1209437453 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fda300a3bb0 0x7fda300a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:27.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:27.251+0000 7fda4be00640 1 --2- 172.21.15.45:0/1209437453 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fda300a3bb0 0x7fda300a6070 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fda38009830 tx=0x7fda3802e040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:06:27.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:27.255+0000 7fda49600640 1 -- 172.21.15.45:0/1209437453 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fda3c09cab0 con 0x7fda4c101ea0 2024-04-16T17:06:27.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:27.467+0000 7fda52000640 1 -- 172.21.15.45:0/1209437453 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fda4c062c00 con 0x7fda4c101ea0 2024-04-16T17:06:27.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:27.471+0000 7fda49600640 1 -- 172.21.15.45:0/1209437453 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) ==== 74+0+24726 (secure 0 0 0) 0x7fda3c0a0db0 con 0x7fda4c101ea0 2024-04-16T17:06:27.473 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-16T17:06:27.473 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":94,"fsid":"b1fecd38-fc11-11ee-bc90-c7b262605968","created":"2024-04-16T16:54:02.385906+0000","modified":"2024-04-16T17:05:22.272335+0000","last_up_change":"2024-04-16T17:02:25.162624+0000","last_in_change":"2024-04-16T17:02:04.903274+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-16T16:58:06.166659+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-16T17:03:22.304198+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-16T17:03:23.616450+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-16T17:03:27.226289+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b655de2c-59f8-4fd4-ac98-0cc222229f56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":2290170881},{"type":"v1","addr":"172.21.15.45:6803","nonce":2290170881}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":2290170881},{"type":"v1","addr":"172.21.15.45:6805","nonce":2290170881}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":2290170881},{"type":"v1","addr":"172.21.15.45:6809","nonce":2290170881}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":2290170881},{"type":"v1","addr":"172.21.15.45:6807","nonce":2290170881}]},"public_addr":"172.21.15.45:6803/2290170881","cluster_addr":"172.21.15.45:6805/2290170881","heartbeat_back_addr":"172.21.15.45:6809/2290170881","heartbeat_front_addr":"172.21.15.45:6807/2290170881","state":["exists","up"]},{"osd":1,"uuid":"ebd450ec-08fd-4c3d-8b24-4e10634ddb7e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6800","nonce":1462942067},{"type":"v1","addr":"172.21.15.104:6801","nonce":1462942067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":1462942067},{"type":"v1","addr":"172.21.15.104:6803","nonce":1462942067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":1462942067},{"type":"v1","addr":"172.21.15.104:6807","nonce":1462942067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":1462942067},{"type":"v1","addr":"172.21.15.104:6805","nonce":1462942067}]},"public_addr":"172.21.15.104:6801/1462942067","cluster_addr":"172.21.15.104:6803/1462942067","heartbeat_back_addr":"172.21.15.104:6807/1462942067","heartbeat_front_addr":"172.21.15.104:6805/1462942067","state":["exists","up"]},{"osd":2,"uuid":"b318a512-4190-42a2-8104-dcc4026ad8ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":2829260735},{"type":"v1","addr":"172.21.15.117:6801","nonce":2829260735}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":2829260735},{"type":"v1","addr":"172.21.15.117:6803","nonce":2829260735}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":2829260735},{"type":"v1","addr":"172.21.15.117:6807","nonce":2829260735}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":2829260735},{"type":"v1","addr":"172.21.15.117:6805","nonce":2829260735}]},"public_addr":"172.21.15.117:6801/2829260735","cluster_addr":"172.21.15.117:6803/2829260735","heartbeat_back_addr":"172.21.15.117:6807/2829260735","heartbeat_front_addr":"172.21.15.117:6805/2829260735","state":["exists","up"]},{"osd":3,"uuid":"ec6490cb-a715-4c12-a54a-225aae26fcb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":2823712137},{"type":"v1","addr":"172.21.15.45:6811","nonce":2823712137}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":2823712137},{"type":"v1","addr":"172.21.15.45:6813","nonce":2823712137}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":2823712137},{"type":"v1","addr":"172.21.15.45:6817","nonce":2823712137}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":2823712137},{"type":"v1","addr":"172.21.15.45:6815","nonce":2823712137}]},"public_addr":"172.21.15.45:6811/2823712137","cluster_addr":"172.21.15.45:6813/2823712137","heartbeat_back_addr":"172.21.15.45:6817/2823712137","heartbeat_front_addr":"172.21.15.45:6815/2823712137","state":["exists","up"]},{"osd":4,"uuid":"5d91d568-0250-4aae-8dc2-a0e079f92062","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":1396278127},{"type":"v1","addr":"172.21.15.104:6809","nonce":1396278127}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":1396278127},{"type":"v1","addr":"172.21.15.104:6811","nonce":1396278127}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":1396278127},{"type":"v1","addr":"172.21.15.104:6815","nonce":1396278127}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":1396278127},{"type":"v1","addr":"172.21.15.104:6813","nonce":1396278127}]},"public_addr":"172.21.15.104:6809/1396278127","cluster_addr":"172.21.15.104:6811/1396278127","heartbeat_back_addr":"172.21.15.104:6815/1396278127","heartbeat_front_addr":"172.21.15.104:6813/1396278127","state":["exists","up"]},{"osd":5,"uuid":"ac510b43-6fb2-4d0f-bc5f-16ac080d9e9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":4145720047},{"type":"v1","addr":"172.21.15.117:6809","nonce":4145720047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":4145720047},{"type":"v1","addr":"172.21.15.117:6811","nonce":4145720047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":4145720047},{"type":"v1","addr":"172.21.15.117:6815","nonce":4145720047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":4145720047},{"type":"v1","addr":"172.21.15.117:6813","nonce":4145720047}]},"public_addr":"172.21.15.117:6809/4145720047","cluster_addr":"172.21.15.117:6811/4145720047","heartbeat_back_addr":"172.21.15.117:6815/4145720047","heartbeat_front_addr":"172.21.15.117:6813/4145720047","state":["exists","up"]},{"osd":6,"uuid":"a41adc18-d88a-419e-b2c2-b089c1f32472","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":3254229176},{"type":"v1","addr":"172.21.15.45:6819","nonce":3254229176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":3254229176},{"type":"v1","addr":"172.21.15.45:6821","nonce":3254229176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":3254229176},{"type":"v1","addr":"172.21.15.45:6825","nonce":3254229176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":3254229176},{"type":"v1","addr":"172.21.15.45:6823","nonce":3254229176}]},"public_addr":"172.21.15.45:6819/3254229176","cluster_addr":"172.21.15.45:6821/3254229176","heartbeat_back_addr":"172.21.15.45:6825/3254229176","heartbeat_front_addr":"172.21.15.45:6823/3254229176","state":["exists","up"]},{"osd":7,"uuid":"b71712a4-7cf6-454d-9fd1-565e3df21c1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":4151077186},{"type":"v1","addr":"172.21.15.104:6817","nonce":4151077186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":4151077186},{"type":"v1","addr":"172.21.15.104:6819","nonce":4151077186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":4151077186},{"type":"v1","addr":"172.21.15.104:6823","nonce":4151077186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":4151077186},{"type":"v1","addr":"172.21.15.104:6821","nonce":4151077186}]},"public_addr":"172.21.15.104:6817/4151077186","cluster_addr":"172.21.15.104:6819/4151077186","heartbeat_back_addr":"172.21.15.104:6823/4151077186","heartbeat_front_addr":"172.21.15.104:6821/4151077186","state":["exists","up"]},{"osd":8,"uuid":"9e572aef-59a8-450e-a92e-df9dff4d57a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":1500286830},{"type":"v1","addr":"172.21.15.117:6817","nonce":1500286830}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":1500286830},{"type":"v1","addr":"172.21.15.117:6819","nonce":1500286830}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":1500286830},{"type":"v1","addr":"172.21.15.117:6823","nonce":1500286830}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":1500286830},{"type":"v1","addr":"172.21.15.117:6821","nonce":1500286830}]},"public_addr":"172.21.15.117:6817/1500286830","cluster_addr":"172.21.15.117:6819/1500286830","heartbeat_back_addr":"172.21.15.117:6823/1500286830","heartbeat_front_addr":"172.21.15.117:6821/1500286830","state":["exists","up"]},{"osd":9,"uuid":"014e9598-c413-4cc4-9bb8-61c6e8f59d30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":3106734074},{"type":"v1","addr":"172.21.15.45:6827","nonce":3106734074}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":3106734074},{"type":"v1","addr":"172.21.15.45:6829","nonce":3106734074}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":3106734074},{"type":"v1","addr":"172.21.15.45:6833","nonce":3106734074}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":3106734074},{"type":"v1","addr":"172.21.15.45:6831","nonce":3106734074}]},"public_addr":"172.21.15.45:6827/3106734074","cluster_addr":"172.21.15.45:6829/3106734074","heartbeat_back_addr":"172.21.15.45:6833/3106734074","heartbeat_front_addr":"172.21.15.45:6831/3106734074","state":["exists","up"]},{"osd":10,"uuid":"2deba364-3e39-47be-8a1f-8256df5b82d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":1404794652},{"type":"v1","addr":"172.21.15.104:6825","nonce":1404794652}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":1404794652},{"type":"v1","addr":"172.21.15.104:6827","nonce":1404794652}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":1404794652},{"type":"v1","addr":"172.21.15.104:6831","nonce":1404794652}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":1404794652},{"type":"v1","addr":"172.21.15.104:6829","nonce":1404794652}]},"public_addr":"172.21.15.104:6825/1404794652","cluster_addr":"172.21.15.104:6827/1404794652","heartbeat_back_addr":"172.21.15.104:6831/1404794652","heartbeat_front_addr":"172.21.15.104:6829/1404794652","state":["exists","up"]},{"osd":11,"uuid":"3fda4fea-59ad-4d1a-9fe6-fcb30e31fd7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":3384003243},{"type":"v1","addr":"172.21.15.117:6825","nonce":3384003243}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":3384003243},{"type":"v1","addr":"172.21.15.117:6827","nonce":3384003243}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":3384003243},{"type":"v1","addr":"172.21.15.117:6831","nonce":3384003243}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":3384003243},{"type":"v1","addr":"172.21.15.117:6829","nonce":3384003243}]},"public_addr":"172.21.15.117:6825/3384003243","cluster_addr":"172.21.15.117:6827/3384003243","heartbeat_back_addr":"172.21.15.117:6831/3384003243","heartbeat_front_addr":"172.21.15.117:6829/3384003243","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:57:14.799608+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:57:39.884668+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:58:03.327057+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:58:37.314531+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:59:03.667650+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:59:28.462823+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T17:00:02.968574+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T17:00:29.559190+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T17:00:55.291007+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T17:01:29.815977+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T17:01:57.015580+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T17:02:23.233151+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.5","mappings":[{"from":9,"to":10}]},{"pgid":"4.7","mappings":[{"from":5,"to":8}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.45:0/4074982598":"2024-04-17T16:55:19.948001+0000","172.21.15.45:0/2443572576":"2024-04-17T16:55:19.948001+0000","172.21.15.45:6801/1983630136":"2024-04-17T16:55:19.948001+0000","172.21.15.45:6800/1983630136":"2024-04-17T16:55:19.948001+0000","172.21.15.45:0/3150465248":"2024-04-17T16:54:43.692466+0000","172.21.15.45:0/556738623":"2024-04-17T16:54:43.692466+0000","172.21.15.45:0/445676297":"2024-04-17T16:55:19.948001+0000","172.21.15.45:6800/2553659622":"2024-04-17T16:54:43.692466+0000","172.21.15.45:6801/2553659622":"2024-04-17T16:54:43.692466+0000","172.21.15.45:0/1184615408":"2024-04-17T16:54:43.692466+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-16T17:06:27.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:27.473+0000 7fda52000640 1 -- 172.21.15.45:0/1209437453 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fda300a3bb0 msgr2=0x7fda300a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:27.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:27.473+0000 7fda52000640 1 --2- 172.21.15.45:0/1209437453 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fda300a3bb0 0x7fda300a6070 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fda38009830 tx=0x7fda3802e040 comp rx=0 tx=0).stop 2024-04-16T17:06:27.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:27.473+0000 7fda52000640 1 -- 172.21.15.45:0/1209437453 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fda4c101ea0 msgr2=0x7fda4c1981c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:27.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:27.473+0000 7fda52000640 1 --2- 172.21.15.45:0/1209437453 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fda4c101ea0 0x7fda4c1981c0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fda3c00e9f0 tx=0x7fda3c00eeb0 comp rx=0 tx=0).stop 2024-04-16T17:06:27.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:27.474+0000 7fda52000640 1 -- 172.21.15.45:0/1209437453 shutdown_connections 2024-04-16T17:06:27.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:27.474+0000 7fda52000640 1 --2- 172.21.15.45:0/1209437453 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fda300a3bb0 0x7fda300a6070 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:27.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:27.474+0000 7fda52000640 1 --2- 172.21.15.45:0/1209437453 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fda4c102ff0 0x7fda4c198700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:27.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:27.474+0000 7fda52000640 1 --2- 172.21.15.45:0/1209437453 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fda4c1039b0 0x7fda4c19f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:27.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:27.474+0000 7fda52000640 1 --2- 172.21.15.45:0/1209437453 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fda4c101ea0 0x7fda4c1981c0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:27.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:27.474+0000 7fda52000640 1 -- 172.21.15.45:0/1209437453 >> 172.21.15.45:0/1209437453 conn(0x7fda4c0fd410 msgr2=0x7fda4c1064a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:06:27.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:27.474+0000 7fda52000640 1 -- 172.21.15.45:0/1209437453 shutdown_connections 2024-04-16T17:06:27.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:27.474+0000 7fda52000640 1 -- 172.21.15.45:0/1209437453 wait complete. 2024-04-16T17:06:27.494 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-16T16:58:06.166659+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-16T17:03:22.304198+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-16T17:03:23.616450+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-16T17:03:27.226289+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-16T17:06:27.495 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-16T17:06:27.555 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:06:27 smithi117 ceph-mon[36730]: pgmap v417: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-16T17:06:27.570 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:06:27 smithi045 ceph-mon[31676]: pgmap v417: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-16T17:06:27.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:06:27 smithi104 ceph-mon[36457]: pgmap v417: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-16T17:06:27.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:27.713+0000 7f724b200640 1 -- 172.21.15.45:0/2855326804 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f724c107b00 msgr2=0x7f724c109f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:27.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:27.713+0000 7f724b200640 1 --2- 172.21.15.45:0/2855326804 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f724c107b00 0x7f724c109f50 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f7244009860 tx=0x7f724402bb70 comp rx=0 tx=0).stop 2024-04-16T17:06:27.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:27.713+0000 7f724b200640 1 -- 172.21.15.45:0/2855326804 shutdown_connections 2024-04-16T17:06:27.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:27.713+0000 7f724b200640 1 --2- 172.21.15.45:0/2855326804 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f724c107b00 0x7f724c109f50 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:27.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:27.713+0000 7f724b200640 1 --2- 172.21.15.45:0/2855326804 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f724c1051d0 0x7f724c1075c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:27.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:27.713+0000 7f724b200640 1 --2- 172.21.15.45:0/2855326804 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f724c1028a0 0x7f724c104c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:27.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:27.713+0000 7f724b200640 1 -- 172.21.15.45:0/2855326804 >> 172.21.15.45:0/2855326804 conn(0x7f724c0fc2d0 msgr2=0x7f724c0fe6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:06:27.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:27.714+0000 7f724b200640 1 -- 172.21.15.45:0/2855326804 shutdown_connections 2024-04-16T17:06:27.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:27.714+0000 7f724b200640 1 -- 172.21.15.45:0/2855326804 wait complete. 2024-04-16T17:06:27.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:27.715+0000 7f724b200640 1 Processor -- start 2024-04-16T17:06:27.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:27.715+0000 7f724b200640 1 -- start start 2024-04-16T17:06:27.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:27.715+0000 7f724b200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f724c1028a0 0x7f724c19a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:27.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:27.715+0000 7f724b200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f724c1051d0 0x7f724c19a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:27.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:27.715+0000 7f724b200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f724c107b00 0x7f724c1a18a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:27.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:27.715+0000 7f724b200640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f724c10daa0 con 0x7f724c1028a0 2024-04-16T17:06:27.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:27.715+0000 7f724b200640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f724c10d920 con 0x7f724c1051d0 2024-04-16T17:06:27.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:27.715+0000 7f724b200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f724c10dc20 con 0x7f724c107b00 2024-04-16T17:06:27.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:27.715+0000 7f7249400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f724c1051d0 0x7f724c19a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:27.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:27.715+0000 7f7249400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f724c1051d0 0x7f724c19a830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:52594/0 (socket says 172.21.15.45:52594) 2024-04-16T17:06:27.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:27.715+0000 7f7249400640 1 -- 172.21.15.45:0/2426785339 learned_addr learned my addr 172.21.15.45:0/2426785339 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:06:27.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:27.716+0000 7f724a800640 1 --2- 172.21.15.45:0/2426785339 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f724c107b00 0x7f724c1a18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:27.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:27.716+0000 7f7249e00640 1 --2- 172.21.15.45:0/2426785339 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f724c1028a0 0x7f724c19a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:27.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:27.716+0000 7f7249400640 1 -- 172.21.15.45:0/2426785339 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f724c107b00 msgr2=0x7f724c1a18a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:27.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:27.716+0000 7f7249400640 1 --2- 172.21.15.45:0/2426785339 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f724c107b00 0x7f724c1a18a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:27.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:27.716+0000 7f7249400640 1 -- 172.21.15.45:0/2426785339 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f724c1028a0 msgr2=0x7f724c19a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:27.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:27.716+0000 7f7249400640 1 --2- 172.21.15.45:0/2426785339 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f724c1028a0 0x7f724c19a2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:27.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:27.716+0000 7f7249400640 1 -- 172.21.15.45:0/2426785339 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f724c1a1f10 con 0x7f724c1051d0 2024-04-16T17:06:27.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:27.716+0000 7f724a800640 1 --2- 172.21.15.45:0/2426785339 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f724c107b00 0x7f724c1a18a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:06:27.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:27.716+0000 7f7249400640 1 --2- 172.21.15.45:0/2426785339 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f724c1051d0 0x7f724c19a830 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f723800e9c0 tx=0x7f723800ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:06:27.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:27.716+0000 7f7242a00640 1 -- 172.21.15.45:0/2426785339 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f723800cd80 con 0x7f724c1051d0 2024-04-16T17:06:27.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:27.717+0000 7f724b200640 1 -- 172.21.15.45:0/2426785339 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f724c1a2200 con 0x7f724c1051d0 2024-04-16T17:06:27.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:27.717+0000 7f7242a00640 1 -- 172.21.15.45:0/2426785339 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7238005500 con 0x7f724c1051d0 2024-04-16T17:06:27.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:27.717+0000 7f7242a00640 1 -- 172.21.15.45:0/2426785339 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7238010690 con 0x7f724c1051d0 2024-04-16T17:06:27.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:27.717+0000 7f724b200640 1 -- 172.21.15.45:0/2426785339 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f724c1a27a0 con 0x7f724c1051d0 2024-04-16T17:06:27.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:27.718+0000 7f724b200640 1 -- 172.21.15.45:0/2426785339 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f724c1095f0 con 0x7f724c1051d0 2024-04-16T17:06:27.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:27.720+0000 7f7242a00640 1 -- 172.21.15.45:0/2426785339 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f72380046c0 con 0x7f724c1051d0 2024-04-16T17:06:27.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:27.721+0000 7f7242a00640 1 --2- 172.21.15.45:0/2426785339 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f72300a3c30 0x7f72300a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:27.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:27.721+0000 7f7242a00640 1 -- 172.21.15.45:0/2426785339 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f72380d0ad0 con 0x7f724c1051d0 2024-04-16T17:06:27.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:27.721+0000 7f7249e00640 1 --2- 172.21.15.45:0/2426785339 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f72300a3c30 0x7f72300a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:27.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:27.722+0000 7f7249e00640 1 --2- 172.21.15.45:0/2426785339 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f72300a3c30 0x7f72300a60f0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f723c005fd0 tx=0x7f723c010040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:06:27.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:27.725+0000 7f7242a00640 1 -- 172.21.15.45:0/2426785339 <== mon.2 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f723809d0c0 con 0x7f724c1051d0 2024-04-16T17:06:27.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:27.940+0000 7f724b200640 1 -- 172.21.15.45:0/2426785339 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f724c062c00 con 0x7f724c1051d0 2024-04-16T17:06:27.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:27.941+0000 7f7242a00640 1 -- 172.21.15.45:0/2426785339 <== mon.2 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v94) ==== 93+0+10 (secure 0 0 0) 0x7f723809c9c0 con 0x7f724c1051d0 2024-04-16T17:06:27.942 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 1 2024-04-16T17:06:27.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:27.943+0000 7f724b200640 1 -- 172.21.15.45:0/2426785339 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f72300a3c30 msgr2=0x7f72300a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:27.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:27.943+0000 7f724b200640 1 --2- 172.21.15.45:0/2426785339 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f72300a3c30 0x7f72300a60f0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f723c005fd0 tx=0x7f723c010040 comp rx=0 tx=0).stop 2024-04-16T17:06:27.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:27.943+0000 7f724b200640 1 -- 172.21.15.45:0/2426785339 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f724c1051d0 msgr2=0x7f724c19a830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:27.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:27.943+0000 7f724b200640 1 --2- 172.21.15.45:0/2426785339 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f724c1051d0 0x7f724c19a830 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f723800e9c0 tx=0x7f723800ee80 comp rx=0 tx=0).stop 2024-04-16T17:06:27.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:27.944+0000 7f724b200640 1 -- 172.21.15.45:0/2426785339 shutdown_connections 2024-04-16T17:06:27.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:27.944+0000 7f724b200640 1 --2- 172.21.15.45:0/2426785339 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f72300a3c30 0x7f72300a60f0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:27.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:27.944+0000 7f724b200640 1 --2- 172.21.15.45:0/2426785339 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f724c107b00 0x7f724c1a18a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:27.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:27.944+0000 7f724b200640 1 --2- 172.21.15.45:0/2426785339 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f724c1051d0 0x7f724c19a830 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:27.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:27.944+0000 7f724b200640 1 --2- 172.21.15.45:0/2426785339 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f724c1028a0 0x7f724c19a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:27.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:27.944+0000 7f724b200640 1 -- 172.21.15.45:0/2426785339 >> 172.21.15.45:0/2426785339 conn(0x7f724c0fc2d0 msgr2=0x7f724c101f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:06:27.947 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:27.944+0000 7f724b200640 1 -- 172.21.15.45:0/2426785339 shutdown_connections 2024-04-16T17:06:27.947 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:27.944+0000 7f724b200640 1 -- 172.21.15.45:0/2426785339 wait complete. 2024-04-16T17:06:27.965 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-16T17:06:28.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:28.195+0000 7fd464a00640 1 -- 172.21.15.45:0/665447657 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd4600785a0 msgr2=0x7fd460078e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:28.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:28.195+0000 7fd464a00640 1 --2- 172.21.15.45:0/665447657 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd4600785a0 0x7fd460078e40 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fd458009ca0 tx=0x7fd45802c0f0 comp rx=0 tx=0).stop 2024-04-16T17:06:28.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:28.195+0000 7fd464a00640 1 -- 172.21.15.45:0/665447657 shutdown_connections 2024-04-16T17:06:28.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:28.195+0000 7fd464a00640 1 --2- 172.21.15.45:0/665447657 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd4600785a0 0x7fd460078e40 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:28.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:28.195+0000 7fd464a00640 1 --2- 172.21.15.45:0/665447657 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd460077c00 0x7fd460078060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:28.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:28.195+0000 7fd464a00640 1 --2- 172.21.15.45:0/665447657 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd4600769c0 0x7fd460076de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:28.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:28.195+0000 7fd464a00640 1 -- 172.21.15.45:0/665447657 >> 172.21.15.45:0/665447657 conn(0x7fd4600ff6b0 msgr2=0x7fd460101af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:06:28.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:28.195+0000 7fd464a00640 1 -- 172.21.15.45:0/665447657 shutdown_connections 2024-04-16T17:06:28.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:28.195+0000 7fd464a00640 1 -- 172.21.15.45:0/665447657 wait complete. 2024-04-16T17:06:28.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:28.196+0000 7fd464a00640 1 Processor -- start 2024-04-16T17:06:28.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:28.196+0000 7fd464a00640 1 -- start start 2024-04-16T17:06:28.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:28.196+0000 7fd464a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd4600769c0 0x7fd46019e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:28.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:28.196+0000 7fd464a00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd460077c00 0x7fd46019ed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:28.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:28.196+0000 7fd464a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd4600785a0 0x7fd4601a5d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:28.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:28.196+0000 7fd464a00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fd4601141d0 con 0x7fd4600785a0 2024-04-16T17:06:28.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:28.196+0000 7fd464a00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fd460114050 con 0x7fd460077c00 2024-04-16T17:06:28.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:28.196+0000 7fd464a00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fd460114350 con 0x7fd4600769c0 2024-04-16T17:06:28.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:28.197+0000 7fd45fe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd4600785a0 0x7fd4601a5d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:28.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:28.197+0000 7fd45f400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd4600769c0 0x7fd46019e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:28.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:28.197+0000 7fd45f400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd4600769c0 0x7fd46019e7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:56406/0 (socket says 172.21.15.45:56406) 2024-04-16T17:06:28.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:28.197+0000 7fd45fe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd4600785a0 0x7fd4601a5d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:32884/0 (socket says 172.21.15.45:32884) 2024-04-16T17:06:28.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:28.197+0000 7fd45f400640 1 -- 172.21.15.45:0/866615806 learned_addr learned my addr 172.21.15.45:0/866615806 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:06:28.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:28.197+0000 7fd45ea00640 1 --2- 172.21.15.45:0/866615806 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd460077c00 0x7fd46019ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:28.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:28.197+0000 7fd45f400640 1 -- 172.21.15.45:0/866615806 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd460077c00 msgr2=0x7fd46019ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:28.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:28.197+0000 7fd45f400640 1 --2- 172.21.15.45:0/866615806 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd460077c00 0x7fd46019ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:28.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:28.197+0000 7fd45f400640 1 -- 172.21.15.45:0/866615806 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd4600785a0 msgr2=0x7fd4601a5d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:28.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:28.197+0000 7fd45f400640 1 --2- 172.21.15.45:0/866615806 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd4600785a0 0x7fd4601a5d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:28.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:28.197+0000 7fd45f400640 1 -- 172.21.15.45:0/866615806 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4601a63e0 con 0x7fd4600769c0 2024-04-16T17:06:28.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:28.197+0000 7fd45ea00640 1 --2- 172.21.15.45:0/866615806 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd460077c00 0x7fd46019ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:06:28.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:28.197+0000 7fd45f400640 1 --2- 172.21.15.45:0/866615806 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd4600769c0 0x7fd46019e7c0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fd45000e9f0 tx=0x7fd45000edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:06:28.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:28.198+0000 7fd457e00640 1 -- 172.21.15.45:0/866615806 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd45000cd80 con 0x7fd4600769c0 2024-04-16T17:06:28.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:28.198+0000 7fd457e00640 1 -- 172.21.15.45:0/866615806 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd450005500 con 0x7fd4600769c0 2024-04-16T17:06:28.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:28.198+0000 7fd457e00640 1 -- 172.21.15.45:0/866615806 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd450010690 con 0x7fd4600769c0 2024-04-16T17:06:28.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:28.198+0000 7fd464a00640 1 -- 172.21.15.45:0/866615806 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4601a66d0 con 0x7fd4600769c0 2024-04-16T17:06:28.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:28.198+0000 7fd464a00640 1 -- 172.21.15.45:0/866615806 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4601a6bd0 con 0x7fd4600769c0 2024-04-16T17:06:28.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:28.199+0000 7fd464a00640 1 -- 172.21.15.45:0/866615806 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd460110ab0 con 0x7fd4600769c0 2024-04-16T17:06:28.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:28.201+0000 7fd457e00640 1 -- 172.21.15.45:0/866615806 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fd4500046c0 con 0x7fd4600769c0 2024-04-16T17:06:28.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:28.202+0000 7fd457e00640 1 --2- 172.21.15.45:0/866615806 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fd4440a3c30 0x7fd4440a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:28.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:28.203+0000 7fd457e00640 1 -- 172.21.15.45:0/866615806 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fd4500d0d70 con 0x7fd4600769c0 2024-04-16T17:06:28.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:28.203+0000 7fd45ea00640 1 --2- 172.21.15.45:0/866615806 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fd4440a3c30 0x7fd4440a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:28.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:28.204+0000 7fd45ea00640 1 --2- 172.21.15.45:0/866615806 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fd4440a3c30 0x7fd4440a60f0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fd46019fcd0 tx=0x7fd44c00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:06:28.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:28.205+0000 7fd457e00640 1 -- 172.21.15.45:0/866615806 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd450053ff0 con 0x7fd4600769c0 2024-04-16T17:06:28.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:28.411+0000 7fd464a00640 1 -- 172.21.15.45:0/866615806 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fd46002d8b0 con 0x7fd4600769c0 2024-04-16T17:06:28.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:28.412+0000 7fd457e00640 1 -- 172.21.15.45:0/866615806 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v94) ==== 104+0+11 (secure 0 0 0) 0x7fd4500bbe20 con 0x7fd4600769c0 2024-04-16T17:06:28.413 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 32 2024-04-16T17:06:28.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:28.415+0000 7fd464a00640 1 -- 172.21.15.45:0/866615806 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fd4440a3c30 msgr2=0x7fd4440a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:28.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:28.415+0000 7fd464a00640 1 --2- 172.21.15.45:0/866615806 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fd4440a3c30 0x7fd4440a60f0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fd46019fcd0 tx=0x7fd44c00b410 comp rx=0 tx=0).stop 2024-04-16T17:06:28.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:28.415+0000 7fd464a00640 1 -- 172.21.15.45:0/866615806 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd4600769c0 msgr2=0x7fd46019e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:28.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:28.415+0000 7fd464a00640 1 --2- 172.21.15.45:0/866615806 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd4600769c0 0x7fd46019e7c0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fd45000e9f0 tx=0x7fd45000edf0 comp rx=0 tx=0).stop 2024-04-16T17:06:28.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:28.415+0000 7fd464a00640 1 -- 172.21.15.45:0/866615806 shutdown_connections 2024-04-16T17:06:28.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:28.415+0000 7fd464a00640 1 --2- 172.21.15.45:0/866615806 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fd4440a3c30 0x7fd4440a60f0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:28.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:28.415+0000 7fd464a00640 1 --2- 172.21.15.45:0/866615806 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd4600785a0 0x7fd4601a5d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:28.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:28.415+0000 7fd464a00640 1 --2- 172.21.15.45:0/866615806 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd460077c00 0x7fd46019ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:28.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:28.415+0000 7fd464a00640 1 --2- 172.21.15.45:0/866615806 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd4600769c0 0x7fd46019e7c0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:28.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:28.415+0000 7fd464a00640 1 -- 172.21.15.45:0/866615806 >> 172.21.15.45:0/866615806 conn(0x7fd4600ff6b0 msgr2=0x7fd4601011b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:06:28.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:28.415+0000 7fd464a00640 1 -- 172.21.15.45:0/866615806 shutdown_connections 2024-04-16T17:06:28.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:28.415+0000 7fd464a00640 1 -- 172.21.15.45:0/866615806 wait complete. 2024-04-16T17:06:28.436 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-16T17:06:28.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:28.664+0000 7f8a53000640 1 -- 172.21.15.45:0/346462793 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8a4c104140 msgr2=0x7f8a4c104560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:28.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:28.664+0000 7f8a53000640 1 --2- 172.21.15.45:0/346462793 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8a4c104140 0x7f8a4c104560 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f8a40009900 tx=0x7f8a4002bc30 comp rx=0 tx=0).stop 2024-04-16T17:06:28.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:28.664+0000 7f8a53000640 1 -- 172.21.15.45:0/346462793 shutdown_connections 2024-04-16T17:06:28.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:28.664+0000 7f8a53000640 1 --2- 172.21.15.45:0/346462793 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8a4c105330 0x7f8a4c1057b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:28.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:28.664+0000 7f8a53000640 1 --2- 172.21.15.45:0/346462793 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8a4c105cf0 0x7f8a4c10c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:28.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:28.664+0000 7f8a53000640 1 --2- 172.21.15.45:0/346462793 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8a4c104140 0x7f8a4c104560 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:28.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:28.664+0000 7f8a53000640 1 -- 172.21.15.45:0/346462793 >> 172.21.15.45:0/346462793 conn(0x7f8a4c0ff6b0 msgr2=0x7f8a4c101af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:06:28.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:28.664+0000 7f8a53000640 1 -- 172.21.15.45:0/346462793 shutdown_connections 2024-04-16T17:06:28.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:28.664+0000 7f8a53000640 1 -- 172.21.15.45:0/346462793 wait complete. 2024-04-16T17:06:28.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:28.665+0000 7f8a53000640 1 Processor -- start 2024-04-16T17:06:28.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:28.666+0000 7f8a53000640 1 -- start start 2024-04-16T17:06:28.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:28.666+0000 7f8a53000640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8a4c104140 0x7f8a4c19a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:28.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:28.666+0000 7f8a53000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8a4c105330 0x7f8a4c19a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:28.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:28.666+0000 7f8a53000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8a4c105cf0 0x7f8a4c1a19e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:28.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:28.666+0000 7f8a53000640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f8a4c10fe20 con 0x7f8a4c105cf0 2024-04-16T17:06:28.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:28.666+0000 7f8a53000640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f8a4c10fca0 con 0x7f8a4c104140 2024-04-16T17:06:28.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:28.666+0000 7f8a53000640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f8a4c10ffa0 con 0x7f8a4c105330 2024-04-16T17:06:28.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:28.666+0000 7f8a51c00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8a4c104140 0x7f8a4c19a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:28.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:28.666+0000 7f8a52600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8a4c105cf0 0x7f8a4c1a19e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:28.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:28.666+0000 7f8a51c00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8a4c104140 0x7f8a4c19a430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:52638/0 (socket says 172.21.15.45:52638) 2024-04-16T17:06:28.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:28.666+0000 7f8a51c00640 1 -- 172.21.15.45:0/1202477042 learned_addr learned my addr 172.21.15.45:0/1202477042 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:06:28.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:28.666+0000 7f8a51200640 1 --2- 172.21.15.45:0/1202477042 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8a4c105330 0x7f8a4c19a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:28.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:28.667+0000 7f8a51c00640 1 -- 172.21.15.45:0/1202477042 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8a4c105330 msgr2=0x7f8a4c19a970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:28.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:28.667+0000 7f8a51c00640 1 --2- 172.21.15.45:0/1202477042 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8a4c105330 0x7f8a4c19a970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:28.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:28.667+0000 7f8a51c00640 1 -- 172.21.15.45:0/1202477042 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8a4c105cf0 msgr2=0x7f8a4c1a19e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:28.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:28.667+0000 7f8a51c00640 1 --2- 172.21.15.45:0/1202477042 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8a4c105cf0 0x7f8a4c1a19e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:28.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:28.667+0000 7f8a51c00640 1 -- 172.21.15.45:0/1202477042 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a4c1a2050 con 0x7f8a4c104140 2024-04-16T17:06:28.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:28.667+0000 7f8a51200640 1 --2- 172.21.15.45:0/1202477042 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8a4c105330 0x7f8a4c19a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:06:28.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:28.667+0000 7f8a51c00640 1 --2- 172.21.15.45:0/1202477042 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8a4c104140 0x7f8a4c19a430 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f8a40000c00 tx=0x7f8a4002c6c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:06:28.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:28.667+0000 7f8a4a000640 1 -- 172.21.15.45:0/1202477042 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a40043020 con 0x7f8a4c104140 2024-04-16T17:06:28.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:28.667+0000 7f8a4a000640 1 -- 172.21.15.45:0/1202477042 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8a400054e0 con 0x7f8a4c104140 2024-04-16T17:06:28.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:28.667+0000 7f8a4a000640 1 -- 172.21.15.45:0/1202477042 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a40003ce0 con 0x7f8a4c104140 2024-04-16T17:06:28.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:28.667+0000 7f8a53000640 1 -- 172.21.15.45:0/1202477042 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a4c1a22e0 con 0x7f8a4c104140 2024-04-16T17:06:28.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:28.668+0000 7f8a53000640 1 -- 172.21.15.45:0/1202477042 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a4c1a2800 con 0x7f8a4c104140 2024-04-16T17:06:28.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:28.669+0000 7f8a53000640 1 -- 172.21.15.45:0/1202477042 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a4c10b460 con 0x7f8a4c104140 2024-04-16T17:06:28.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:28.671+0000 7f8a4a000640 1 -- 172.21.15.45:0/1202477042 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f8a40046020 con 0x7f8a4c104140 2024-04-16T17:06:28.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:28.672+0000 7f8a4a000640 1 --2- 172.21.15.45:0/1202477042 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f8a340a3bb0 0x7f8a340a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:28.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:28.672+0000 7f8a4a000640 1 -- 172.21.15.45:0/1202477042 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f8a400f2fd0 con 0x7f8a4c104140 2024-04-16T17:06:28.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:28.672+0000 7f8a51200640 1 --2- 172.21.15.45:0/1202477042 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f8a340a3bb0 0x7f8a340a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:28.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:28.673+0000 7f8a51200640 1 --2- 172.21.15.45:0/1202477042 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f8a340a3bb0 0x7f8a340a6070 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f8a38005d00 tx=0x7f8a38005c70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:06:28.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:28.675+0000 7f8a4a000640 1 -- 172.21.15.45:0/1202477042 <== mon.2 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8a4004d020 con 0x7f8a4c104140 2024-04-16T17:06:28.677 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:06:28 smithi104 ceph-mon[36457]: from='client.? 172.21.15.45:0/1209437453' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T17:06:28.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:06:28 smithi104 ceph-mon[36457]: from='client.? 172.21.15.45:0/2426785339' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-16T17:06:28.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:06:28 smithi045 ceph-mon[31676]: from='client.? 172.21.15.45:0/1209437453' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T17:06:28.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:06:28 smithi045 ceph-mon[31676]: from='client.? 172.21.15.45:0/2426785339' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-16T17:06:28.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:06:28 smithi117 ceph-mon[36730]: from='client.? 172.21.15.45:0/1209437453' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T17:06:28.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:06:28 smithi117 ceph-mon[36730]: from='client.? 172.21.15.45:0/2426785339' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-16T17:06:28.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:28.883+0000 7f8a53000640 1 -- 172.21.15.45:0/1202477042 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f8a4c02d7b0 con 0x7f8a4c104140 2024-04-16T17:06:28.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:28.885+0000 7f8a4a000640 1 -- 172.21.15.45:0/1202477042 <== mon.2 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v94) ==== 100+0+11 (secure 0 0 0) 0x7f8a400bf5c0 con 0x7f8a4c104140 2024-04-16T17:06:28.885 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 64 2024-04-16T17:06:28.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:28.887+0000 7f8a53000640 1 -- 172.21.15.45:0/1202477042 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f8a340a3bb0 msgr2=0x7f8a340a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:28.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:28.887+0000 7f8a53000640 1 --2- 172.21.15.45:0/1202477042 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f8a340a3bb0 0x7f8a340a6070 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f8a38005d00 tx=0x7f8a38005c70 comp rx=0 tx=0).stop 2024-04-16T17:06:28.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:28.887+0000 7f8a53000640 1 -- 172.21.15.45:0/1202477042 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8a4c104140 msgr2=0x7f8a4c19a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:28.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:28.887+0000 7f8a53000640 1 --2- 172.21.15.45:0/1202477042 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8a4c104140 0x7f8a4c19a430 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f8a40000c00 tx=0x7f8a4002c6c0 comp rx=0 tx=0).stop 2024-04-16T17:06:28.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:28.887+0000 7f8a53000640 1 -- 172.21.15.45:0/1202477042 shutdown_connections 2024-04-16T17:06:28.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:28.887+0000 7f8a53000640 1 --2- 172.21.15.45:0/1202477042 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f8a340a3bb0 0x7f8a340a6070 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:28.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:28.887+0000 7f8a53000640 1 --2- 172.21.15.45:0/1202477042 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8a4c105330 0x7f8a4c19a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:28.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:28.887+0000 7f8a53000640 1 --2- 172.21.15.45:0/1202477042 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8a4c105cf0 0x7f8a4c1a19e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:28.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:28.887+0000 7f8a53000640 1 --2- 172.21.15.45:0/1202477042 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8a4c104140 0x7f8a4c19a430 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:28.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:28.887+0000 7f8a53000640 1 -- 172.21.15.45:0/1202477042 >> 172.21.15.45:0/1202477042 conn(0x7f8a4c0ff6b0 msgr2=0x7f8a4c108630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:06:28.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:28.887+0000 7f8a53000640 1 -- 172.21.15.45:0/1202477042 shutdown_connections 2024-04-16T17:06:28.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:28.887+0000 7f8a53000640 1 -- 172.21.15.45:0/1202477042 wait complete. 2024-04-16T17:06:28.909 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-16T17:06:29.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:29.156+0000 7f1c27e00640 1 -- 172.21.15.45:0/2147710459 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1c28100780 msgr2=0x7f1c28102bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:29.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:29.156+0000 7f1c27e00640 1 --2- 172.21.15.45:0/2147710459 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1c28100780 0x7f1c28102bb0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f1c18009be0 tx=0x7f1c1802bef0 comp rx=0 tx=0).stop 2024-04-16T17:06:29.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:29.157+0000 7f1c27e00640 1 -- 172.21.15.45:0/2147710459 shutdown_connections 2024-04-16T17:06:29.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:29.157+0000 7f1c27e00640 1 --2- 172.21.15.45:0/2147710459 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1c28105aa0 0x7f1c28105f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:29.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:29.157+0000 7f1c27e00640 1 --2- 172.21.15.45:0/2147710459 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1c281030f0 0x7f1c28105560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:29.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:29.157+0000 7f1c27e00640 1 --2- 172.21.15.45:0/2147710459 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1c28100780 0x7f1c28102bb0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:29.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:29.157+0000 7f1c27e00640 1 -- 172.21.15.45:0/2147710459 >> 172.21.15.45:0/2147710459 conn(0x7f1c280fc3a0 msgr2=0x7f1c280fe7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:06:29.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:29.157+0000 7f1c27e00640 1 -- 172.21.15.45:0/2147710459 shutdown_connections 2024-04-16T17:06:29.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:29.157+0000 7f1c27e00640 1 -- 172.21.15.45:0/2147710459 wait complete. 2024-04-16T17:06:29.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:29.158+0000 7f1c27e00640 1 Processor -- start 2024-04-16T17:06:29.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:29.158+0000 7f1c27e00640 1 -- start start 2024-04-16T17:06:29.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:29.158+0000 7f1c27e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1c28100780 0x7f1c2819e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:29.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:29.158+0000 7f1c27e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1c281030f0 0x7f1c2819ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:29.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:29.158+0000 7f1c27e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1c28105aa0 0x7f1c281a5d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:29.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:29.158+0000 7f1c27e00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f1c281141b0 con 0x7f1c28105aa0 2024-04-16T17:06:29.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:29.158+0000 7f1c27e00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f1c28114030 con 0x7f1c28100780 2024-04-16T17:06:29.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:29.158+0000 7f1c27e00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f1c28114330 con 0x7f1c281030f0 2024-04-16T17:06:29.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:29.159+0000 7f1c26000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1c281030f0 0x7f1c2819ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:29.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:29.159+0000 7f1c26a00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1c28100780 0x7f1c2819e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:29.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:29.159+0000 7f1c27400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1c28105aa0 0x7f1c281a5d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:29.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:29.159+0000 7f1c26a00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1c28100780 0x7f1c2819e810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:52660/0 (socket says 172.21.15.45:52660) 2024-04-16T17:06:29.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:29.159+0000 7f1c26a00640 1 -- 172.21.15.45:0/1810601579 learned_addr learned my addr 172.21.15.45:0/1810601579 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:06:29.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:29.159+0000 7f1c26000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1c281030f0 0x7f1c2819ed50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:56434/0 (socket says 172.21.15.45:56434) 2024-04-16T17:06:29.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:29.159+0000 7f1c26000640 1 -- 172.21.15.45:0/1810601579 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1c28100780 msgr2=0x7f1c2819e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:29.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:29.159+0000 7f1c26000640 1 --2- 172.21.15.45:0/1810601579 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1c28100780 0x7f1c2819e810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:29.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:29.159+0000 7f1c26000640 1 -- 172.21.15.45:0/1810601579 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1c28105aa0 msgr2=0x7f1c281a5d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:29.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:29.159+0000 7f1c26000640 1 --2- 172.21.15.45:0/1810601579 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1c28105aa0 0x7f1c281a5d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:29.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:29.159+0000 7f1c26000640 1 -- 172.21.15.45:0/1810601579 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c281a63e0 con 0x7f1c281030f0 2024-04-16T17:06:29.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:29.160+0000 7f1c26000640 1 --2- 172.21.15.45:0/1810601579 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1c281030f0 0x7f1c2819ed50 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f1c10005fd0 tx=0x7f1c1000ee50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:06:29.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:29.160+0000 7f1c2ca00640 1 -- 172.21.15.45:0/1810601579 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c1000cd00 con 0x7f1c281030f0 2024-04-16T17:06:29.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:29.160+0000 7f1c27e00640 1 -- 172.21.15.45:0/1810601579 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c281a66d0 con 0x7f1c281030f0 2024-04-16T17:06:29.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:29.160+0000 7f1c2ca00640 1 -- 172.21.15.45:0/1810601579 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1c100035f0 con 0x7f1c281030f0 2024-04-16T17:06:29.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:29.160+0000 7f1c2ca00640 1 -- 172.21.15.45:0/1810601579 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c10010670 con 0x7f1c281030f0 2024-04-16T17:06:29.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:29.160+0000 7f1c27e00640 1 -- 172.21.15.45:0/1810601579 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c281a6c70 con 0x7f1c281030f0 2024-04-16T17:06:29.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:29.161+0000 7f1c27e00640 1 -- 172.21.15.45:0/1810601579 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c280fde90 con 0x7f1c281030f0 2024-04-16T17:06:29.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:29.163+0000 7f1c2ca00640 1 -- 172.21.15.45:0/1810601579 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f1c100037a0 con 0x7f1c281030f0 2024-04-16T17:06:29.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:29.165+0000 7f1c2ca00640 1 --2- 172.21.15.45:0/1810601579 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f1c0c0a3c30 0x7f1c0c0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:29.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:29.165+0000 7f1c2ca00640 1 -- 172.21.15.45:0/1810601579 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f1c100d09e0 con 0x7f1c281030f0 2024-04-16T17:06:29.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:29.166+0000 7f1c26a00640 1 --2- 172.21.15.45:0/1810601579 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f1c0c0a3c30 0x7f1c0c0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:29.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:29.166+0000 7f1c26a00640 1 --2- 172.21.15.45:0/1810601579 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f1c0c0a3c30 0x7f1c0c0a60f0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f1c18004af0 tx=0x7f1c18004a80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:06:29.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:29.167+0000 7f1c2ca00640 1 -- 172.21.15.45:0/1810601579 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1c10009510 con 0x7f1c281030f0 2024-04-16T17:06:29.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:29.375+0000 7f1c27e00640 1 -- 172.21.15.45:0/1810601579 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f1c2802d870 con 0x7f1c281030f0 2024-04-16T17:06:29.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:29.377+0000 7f1c2ca00640 1 -- 172.21.15.45:0/1810601579 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v94) ==== 103+0+11 (secure 0 0 0) 0x7f1c1009d060 con 0x7f1c281030f0 2024-04-16T17:06:29.378 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 32 2024-04-16T17:06:29.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:29.378+0000 7f1c27e00640 1 -- 172.21.15.45:0/1810601579 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f1c0c0a3c30 msgr2=0x7f1c0c0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:29.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:29.378+0000 7f1c27e00640 1 --2- 172.21.15.45:0/1810601579 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f1c0c0a3c30 0x7f1c0c0a60f0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f1c18004af0 tx=0x7f1c18004a80 comp rx=0 tx=0).stop 2024-04-16T17:06:29.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:29.378+0000 7f1c27e00640 1 -- 172.21.15.45:0/1810601579 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1c281030f0 msgr2=0x7f1c2819ed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:29.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:29.378+0000 7f1c27e00640 1 --2- 172.21.15.45:0/1810601579 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1c281030f0 0x7f1c2819ed50 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f1c10005fd0 tx=0x7f1c1000ee50 comp rx=0 tx=0).stop 2024-04-16T17:06:29.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:29.378+0000 7f1c27e00640 1 -- 172.21.15.45:0/1810601579 shutdown_connections 2024-04-16T17:06:29.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:29.378+0000 7f1c27e00640 1 --2- 172.21.15.45:0/1810601579 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f1c0c0a3c30 0x7f1c0c0a60f0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:29.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:29.378+0000 7f1c27e00640 1 --2- 172.21.15.45:0/1810601579 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1c281030f0 0x7f1c2819ed50 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:29.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:29.378+0000 7f1c27e00640 1 --2- 172.21.15.45:0/1810601579 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1c28105aa0 0x7f1c281a5d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:29.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:29.378+0000 7f1c27e00640 1 --2- 172.21.15.45:0/1810601579 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1c28100780 0x7f1c2819e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:29.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:29.378+0000 7f1c27e00640 1 -- 172.21.15.45:0/1810601579 >> 172.21.15.45:0/1810601579 conn(0x7f1c280fc3a0 msgr2=0x7f1c28103dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:06:29.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:29.379+0000 7f1c27e00640 1 -- 172.21.15.45:0/1810601579 shutdown_connections 2024-04-16T17:06:29.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:29.379+0000 7f1c27e00640 1 -- 172.21.15.45:0/1810601579 wait complete. 2024-04-16T17:06:29.401 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-16T17:06:29.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:29.627+0000 7f2029400640 1 -- 172.21.15.45:0/2172090836 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2024105290 msgr2=0x7f2024105710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:29.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:29.627+0000 7f2029400640 1 --2- 172.21.15.45:0/2172090836 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2024105290 0x7f2024105710 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f2010009860 tx=0x7f201002bb70 comp rx=0 tx=0).stop 2024-04-16T17:06:29.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:29.627+0000 7f2029400640 1 -- 172.21.15.45:0/2172090836 shutdown_connections 2024-04-16T17:06:29.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:29.627+0000 7f2029400640 1 --2- 172.21.15.45:0/2172090836 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2024105290 0x7f2024105710 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:29.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:29.627+0000 7f2029400640 1 --2- 172.21.15.45:0/2172090836 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2024105c50 0x7f202410c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:29.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:29.627+0000 7f2029400640 1 --2- 172.21.15.45:0/2172090836 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f20241040a0 0x7f20241044c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:29.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:29.627+0000 7f2029400640 1 -- 172.21.15.45:0/2172090836 >> 172.21.15.45:0/2172090836 conn(0x7f20240ff610 msgr2=0x7f2024101a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:06:29.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:29.627+0000 7f2029400640 1 -- 172.21.15.45:0/2172090836 shutdown_connections 2024-04-16T17:06:29.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:29.627+0000 7f2029400640 1 -- 172.21.15.45:0/2172090836 wait complete. 2024-04-16T17:06:29.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:29.628+0000 7f2029400640 1 Processor -- start 2024-04-16T17:06:29.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:29.629+0000 7f2029400640 1 -- start start 2024-04-16T17:06:29.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:29.629+0000 7f2029400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f20241040a0 0x7f202419a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:29.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:29.629+0000 7f2029400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2024105290 0x7f202419a950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:29.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:29.629+0000 7f2029400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2024105c50 0x7f20241a19c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:29.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:29.629+0000 7f2029400640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f202410fd80 con 0x7f2024105290 2024-04-16T17:06:29.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:29.629+0000 7f2029400640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f202410fc00 con 0x7f20241040a0 2024-04-16T17:06:29.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:29.629+0000 7f2029400640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f202410ff00 con 0x7f2024105c50 2024-04-16T17:06:29.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:29.629+0000 7f2023400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2024105290 0x7f202419a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:29.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:29.629+0000 7f2023e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f20241040a0 0x7f202419a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:29.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:29.629+0000 7f2028a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2024105c50 0x7f20241a19c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:29.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:29.630+0000 7f2023e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f20241040a0 0x7f202419a410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:52674/0 (socket says 172.21.15.45:52674) 2024-04-16T17:06:29.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:29.630+0000 7f2023e00640 1 -- 172.21.15.45:0/3138436626 learned_addr learned my addr 172.21.15.45:0/3138436626 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:06:29.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:29.630+0000 7f2023e00640 1 -- 172.21.15.45:0/3138436626 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2024105c50 msgr2=0x7f20241a19c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:29.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:29.630+0000 7f2023e00640 1 --2- 172.21.15.45:0/3138436626 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2024105c50 0x7f20241a19c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:29.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:29.630+0000 7f2023e00640 1 -- 172.21.15.45:0/3138436626 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2024105290 msgr2=0x7f202419a950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:29.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:29.630+0000 7f2023e00640 1 --2- 172.21.15.45:0/3138436626 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2024105290 0x7f202419a950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:29.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:29.630+0000 7f2023e00640 1 -- 172.21.15.45:0/3138436626 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20241a2030 con 0x7f20241040a0 2024-04-16T17:06:29.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:29.630+0000 7f2023400640 1 --2- 172.21.15.45:0/3138436626 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2024105290 0x7f202419a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:06:29.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:29.630+0000 7f2023e00640 1 --2- 172.21.15.45:0/3138436626 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f20241040a0 0x7f202419a410 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f201400e9c0 tx=0x7f201400ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:06:29.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:29.630+0000 7f2020c00640 1 -- 172.21.15.45:0/3138436626 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f201400cd80 con 0x7f20241040a0 2024-04-16T17:06:29.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:29.631+0000 7f2020c00640 1 -- 172.21.15.45:0/3138436626 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f20140035f0 con 0x7f20241040a0 2024-04-16T17:06:29.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:29.631+0000 7f2020c00640 1 -- 172.21.15.45:0/3138436626 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2014010690 con 0x7f20241040a0 2024-04-16T17:06:29.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:29.631+0000 7f2029400640 1 -- 172.21.15.45:0/3138436626 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20241a2320 con 0x7f20241040a0 2024-04-16T17:06:29.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:29.631+0000 7f2029400640 1 -- 172.21.15.45:0/3138436626 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20241a27f0 con 0x7f20241040a0 2024-04-16T17:06:29.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:29.632+0000 7f2029400640 1 -- 172.21.15.45:0/3138436626 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f202410b3e0 con 0x7f20241040a0 2024-04-16T17:06:29.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:29.634+0000 7f2020c00640 1 -- 172.21.15.45:0/3138436626 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f20140037a0 con 0x7f20241040a0 2024-04-16T17:06:29.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:29.635+0000 7f2020c00640 1 --2- 172.21.15.45:0/3138436626 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f20080a3c00 0x7f20080a60c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:29.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:29.636+0000 7f2020c00640 1 -- 172.21.15.45:0/3138436626 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f20140d0c90 con 0x7f20241040a0 2024-04-16T17:06:29.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:29.636+0000 7f2023400640 1 --2- 172.21.15.45:0/3138436626 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f20080a3c00 0x7f20080a60c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:29.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:29.637+0000 7f2023400640 1 --2- 172.21.15.45:0/3138436626 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f20080a3c00 0x7f20080a60c0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f201002c180 tx=0x7f201002e040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:06:29.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:29.637+0000 7f2020c00640 1 -- 172.21.15.45:0/3138436626 <== mon.2 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f201409d310 con 0x7f20241040a0 2024-04-16T17:06:29.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:06:29 smithi104 ceph-mon[36457]: pgmap v418: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-16T17:06:29.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:06:29 smithi104 ceph-mon[36457]: from='client.? 172.21.15.45:0/866615806' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-16T17:06:29.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:06:29 smithi104 ceph-mon[36457]: from='client.? 172.21.15.45:0/1202477042' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-16T17:06:29.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:06:29 smithi045 ceph-mon[31676]: pgmap v418: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-16T17:06:29.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:06:29 smithi045 ceph-mon[31676]: from='client.? 172.21.15.45:0/866615806' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-16T17:06:29.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:06:29 smithi045 ceph-mon[31676]: from='client.? 172.21.15.45:0/1202477042' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-16T17:06:29.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:06:29 smithi117 ceph-mon[36730]: pgmap v418: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-16T17:06:29.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:06:29 smithi117 ceph-mon[36730]: from='client.? 172.21.15.45:0/866615806' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-16T17:06:29.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:06:29 smithi117 ceph-mon[36730]: from='client.? 172.21.15.45:0/1202477042' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-16T17:06:29.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:29.895+0000 7f2029400640 1 -- 172.21.15.45:0/3138436626 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f202402d8d0 con 0x7f20241040a0 2024-04-16T17:06:29.897 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:29.896+0000 7f2020c00640 1 -- 172.21.15.45:0/3138436626 <== mon.2 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11887 (secure 0 0 0) 0x7f201409cc10 con 0x7f20241040a0 2024-04-16T17:06:29.898 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-16T17:06:29.898 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/1748842443","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":1748842443},{"type":"v1","addr":"172.21.15.45:6835","nonce":1748842443}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6839/1308627051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":1308627051},{"type":"v1","addr":"172.21.15.45:6839","nonce":1308627051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/1768242489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1768242489},{"type":"v1","addr":"172.21.15.117:6833","nonce":1768242489}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6833/3558902585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":3558902585},{"type":"v1","addr":"172.21.15.104:6833","nonce":3558902585}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24478,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/1308142903","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":1308142903},{"type":"v1","addr":"172.21.15.45:6837","nonce":1308142903}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24491,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6835/3430679937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":3430679937},{"type":"v1","addr":"172.21.15.104:6835","nonce":3430679937}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-16T17:03:24.663611+0000","modified":"2024-04-16T17:03:53.381209+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24581,"mds_1":24569,"mds_2":24532},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24499":{"gid":24499,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/2224123841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2224123841},{"type":"v1","addr":"172.21.15.117:6835","nonce":2224123841}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6837/2471356815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":2471356815},{"type":"v1","addr":"172.21.15.117:6837","nonce":2471356815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24524":{"gid":24524,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6837/2647756509","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":2647756509},{"type":"v1","addr":"172.21.15.104:6837","nonce":2647756509}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24532":{"gid":24532,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":13,"addr":"172.21.15.117:6839/2378675949","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":2378675949},{"type":"v1","addr":"172.21.15.117:6839","nonce":2378675949}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24569":{"gid":24569,"name":"a","rank":1,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.45:6841/3755310822","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6840","nonce":3755310822},{"type":"v1","addr":"172.21.15.45:6841","nonce":3755310822}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24581":{"gid":24581,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.104:6839/3660070592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6838","nonce":3660070592},{"type":"v1","addr":"172.21.15.104:6839","nonce":3660070592}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24581,"qdb_cluster":[24569,24532,24581]},"id":1}]} 2024-04-16T17:06:29.898 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 24 2024-04-16T17:06:29.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:29.899+0000 7f2029400640 1 -- 172.21.15.45:0/3138436626 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f20080a3c00 msgr2=0x7f20080a60c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:29.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:29.899+0000 7f2029400640 1 --2- 172.21.15.45:0/3138436626 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f20080a3c00 0x7f20080a60c0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f201002c180 tx=0x7f201002e040 comp rx=0 tx=0).stop 2024-04-16T17:06:29.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:29.899+0000 7f2029400640 1 -- 172.21.15.45:0/3138436626 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f20241040a0 msgr2=0x7f202419a410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:29.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:29.899+0000 7f2029400640 1 --2- 172.21.15.45:0/3138436626 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f20241040a0 0x7f202419a410 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f201400e9c0 tx=0x7f201400ee80 comp rx=0 tx=0).stop 2024-04-16T17:06:29.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:29.899+0000 7f2029400640 1 -- 172.21.15.45:0/3138436626 shutdown_connections 2024-04-16T17:06:29.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:29.899+0000 7f2029400640 1 --2- 172.21.15.45:0/3138436626 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f20080a3c00 0x7f20080a60c0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:29.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:29.899+0000 7f2029400640 1 --2- 172.21.15.45:0/3138436626 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2024105c50 0x7f20241a19c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:29.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:29.899+0000 7f2029400640 1 --2- 172.21.15.45:0/3138436626 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2024105290 0x7f202419a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:29.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:29.899+0000 7f2029400640 1 --2- 172.21.15.45:0/3138436626 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f20241040a0 0x7f202419a410 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:29.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:29.899+0000 7f2029400640 1 -- 172.21.15.45:0/3138436626 >> 172.21.15.45:0/3138436626 conn(0x7f20240ff610 msgr2=0x7f20241085b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:06:29.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:29.899+0000 7f2029400640 1 -- 172.21.15.45:0/3138436626 shutdown_connections 2024-04-16T17:06:29.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:29.899+0000 7f2029400640 1 -- 172.21.15.45:0/3138436626 wait complete. 2024-04-16T17:06:29.921 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-16T17:06:30.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:30.148+0000 7f7fb4a00640 1 -- 172.21.15.45:0/2419868485 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7fb0108480 msgr2=0x7f7fb01088a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:30.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:30.148+0000 7f7fb4a00640 1 --2- 172.21.15.45:0/2419868485 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7fb0108480 0x7f7fb01088a0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f7fa0009900 tx=0x7f7fa002bc50 comp rx=0 tx=0).stop 2024-04-16T17:06:30.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:30.148+0000 7f7fb4a00640 1 -- 172.21.15.45:0/2419868485 shutdown_connections 2024-04-16T17:06:30.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:30.148+0000 7f7fb4a00640 1 --2- 172.21.15.45:0/2419868485 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7fb0109670 0x7f7fb0109af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:30.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:30.149+0000 7f7fb4a00640 1 --2- 172.21.15.45:0/2419868485 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7fb010a030 0x7f7fb01108c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:30.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:30.149+0000 7f7fb4a00640 1 --2- 172.21.15.45:0/2419868485 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7fb0108480 0x7f7fb01088a0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:30.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:30.149+0000 7f7fb4a00640 1 -- 172.21.15.45:0/2419868485 >> 172.21.15.45:0/2419868485 conn(0x7f7fb0077440 msgr2=0x7f7fb0079860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:06:30.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:30.149+0000 7f7fb4a00640 1 -- 172.21.15.45:0/2419868485 shutdown_connections 2024-04-16T17:06:30.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:30.149+0000 7f7fb4a00640 1 -- 172.21.15.45:0/2419868485 wait complete. 2024-04-16T17:06:30.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:30.150+0000 7f7fb4a00640 1 Processor -- start 2024-04-16T17:06:30.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:30.150+0000 7f7fb4a00640 1 -- start start 2024-04-16T17:06:30.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:30.150+0000 7f7fb4a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7fb0108480 0x7f7fb019e830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:30.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:30.150+0000 7f7fb4a00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7fb0109670 0x7f7fb019ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:30.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:30.150+0000 7f7fb4a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7fb010a030 0x7f7fb01a5d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:30.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:30.150+0000 7f7fb4a00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f7fb0114160 con 0x7f7fb0108480 2024-04-16T17:06:30.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:30.150+0000 7f7fb4a00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f7fb0113fe0 con 0x7f7fb0109670 2024-04-16T17:06:30.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:30.150+0000 7f7fb4a00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f7fb01142e0 con 0x7f7fb010a030 2024-04-16T17:06:30.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:30.150+0000 7f7faf400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7fb0108480 0x7f7fb019e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:30.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:30.150+0000 7f7fa7e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7fb0109670 0x7f7fb019ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:30.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:30.150+0000 7f7fa7e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7fb0109670 0x7f7fb019ed70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:48594/0 (socket says 172.21.15.45:48594) 2024-04-16T17:06:30.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:30.150+0000 7f7faf400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7fb0108480 0x7f7fb019e830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55418/0 (socket says 172.21.15.45:55418) 2024-04-16T17:06:30.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:30.150+0000 7f7fa7e00640 1 -- 172.21.15.45:0/3649771771 learned_addr learned my addr 172.21.15.45:0/3649771771 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:06:30.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:30.151+0000 7f7fafe00640 1 --2- 172.21.15.45:0/3649771771 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7fb010a030 0x7f7fb01a5d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:30.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:30.151+0000 7f7fa7e00640 1 -- 172.21.15.45:0/3649771771 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7fb010a030 msgr2=0x7f7fb01a5d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:30.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:30.151+0000 7f7fa7e00640 1 --2- 172.21.15.45:0/3649771771 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7fb010a030 0x7f7fb01a5d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:30.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:30.151+0000 7f7fa7e00640 1 -- 172.21.15.45:0/3649771771 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7fb0108480 msgr2=0x7f7fb019e830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:30.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:30.151+0000 7f7fa7e00640 1 --2- 172.21.15.45:0/3649771771 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7fb0108480 0x7f7fb019e830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:30.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:30.151+0000 7f7fa7e00640 1 -- 172.21.15.45:0/3649771771 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fb01a6400 con 0x7f7fb0109670 2024-04-16T17:06:30.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:30.151+0000 7f7faf400640 1 --2- 172.21.15.45:0/3649771771 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7fb0108480 0x7f7fb019e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:06:30.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:30.151+0000 7f7fa7e00640 1 --2- 172.21.15.45:0/3649771771 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7fb0109670 0x7f7fb019ed70 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f7f9800e9f0 tx=0x7f7f9800edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:06:30.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:30.151+0000 7f7facc00640 1 -- 172.21.15.45:0/3649771771 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f9800ccc0 con 0x7f7fb0109670 2024-04-16T17:06:30.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:30.152+0000 7f7fb4a00640 1 -- 172.21.15.45:0/3649771771 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7fb01a66f0 con 0x7f7fb0109670 2024-04-16T17:06:30.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:30.152+0000 7f7facc00640 1 -- 172.21.15.45:0/3649771771 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f9800ce60 con 0x7f7fb0109670 2024-04-16T17:06:30.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:30.152+0000 7f7facc00640 1 -- 172.21.15.45:0/3649771771 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f98010690 con 0x7f7fb0109670 2024-04-16T17:06:30.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:30.152+0000 7f7fb4a00640 1 -- 172.21.15.45:0/3649771771 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7fb01a6bf0 con 0x7f7fb0109670 2024-04-16T17:06:30.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:30.153+0000 7f7fb4a00640 1 -- 172.21.15.45:0/3649771771 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7fb010f7a0 con 0x7f7fb0109670 2024-04-16T17:06:30.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:30.155+0000 7f7facc00640 1 -- 172.21.15.45:0/3649771771 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f7f980046c0 con 0x7f7fb0109670 2024-04-16T17:06:30.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:30.156+0000 7f7facc00640 1 --2- 172.21.15.45:0/3649771771 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f7f940a3bb0 0x7f7f940a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:30.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:30.157+0000 7f7facc00640 1 -- 172.21.15.45:0/3649771771 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f7f980d0ba0 con 0x7f7fb0109670 2024-04-16T17:06:30.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:30.157+0000 7f7faf400640 1 --2- 172.21.15.45:0/3649771771 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f7f940a3bb0 0x7f7f940a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:30.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:30.159+0000 7f7faf400640 1 --2- 172.21.15.45:0/3649771771 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f7f940a3bb0 0x7f7f940a6070 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f7fa002c190 tx=0x7f7fa0000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:06:30.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:30.159+0000 7f7facc00640 1 -- 172.21.15.45:0/3649771771 <== mon.2 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7f980d0e20 con 0x7f7fb0109670 2024-04-16T17:06:30.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:30.371+0000 7f7fb4a00640 1 -- 172.21.15.45:0/3649771771 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7fb002d870 con 0x7f7fb0109670 2024-04-16T17:06:30.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:30.374+0000 7f7facc00640 1 -- 172.21.15.45:0/3649771771 <== mon.2 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) ==== 74+0+24726 (secure 0 0 0) 0x7f7f9809d190 con 0x7f7fb0109670 2024-04-16T17:06:30.375 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-16T17:06:30.375 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":94,"fsid":"b1fecd38-fc11-11ee-bc90-c7b262605968","created":"2024-04-16T16:54:02.385906+0000","modified":"2024-04-16T17:05:22.272335+0000","last_up_change":"2024-04-16T17:02:25.162624+0000","last_in_change":"2024-04-16T17:02:04.903274+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-16T16:58:06.166659+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-16T17:03:22.304198+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-16T17:03:23.616450+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-16T17:03:27.226289+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b655de2c-59f8-4fd4-ac98-0cc222229f56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":2290170881},{"type":"v1","addr":"172.21.15.45:6803","nonce":2290170881}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":2290170881},{"type":"v1","addr":"172.21.15.45:6805","nonce":2290170881}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":2290170881},{"type":"v1","addr":"172.21.15.45:6809","nonce":2290170881}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":2290170881},{"type":"v1","addr":"172.21.15.45:6807","nonce":2290170881}]},"public_addr":"172.21.15.45:6803/2290170881","cluster_addr":"172.21.15.45:6805/2290170881","heartbeat_back_addr":"172.21.15.45:6809/2290170881","heartbeat_front_addr":"172.21.15.45:6807/2290170881","state":["exists","up"]},{"osd":1,"uuid":"ebd450ec-08fd-4c3d-8b24-4e10634ddb7e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6800","nonce":1462942067},{"type":"v1","addr":"172.21.15.104:6801","nonce":1462942067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":1462942067},{"type":"v1","addr":"172.21.15.104:6803","nonce":1462942067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":1462942067},{"type":"v1","addr":"172.21.15.104:6807","nonce":1462942067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":1462942067},{"type":"v1","addr":"172.21.15.104:6805","nonce":1462942067}]},"public_addr":"172.21.15.104:6801/1462942067","cluster_addr":"172.21.15.104:6803/1462942067","heartbeat_back_addr":"172.21.15.104:6807/1462942067","heartbeat_front_addr":"172.21.15.104:6805/1462942067","state":["exists","up"]},{"osd":2,"uuid":"b318a512-4190-42a2-8104-dcc4026ad8ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":2829260735},{"type":"v1","addr":"172.21.15.117:6801","nonce":2829260735}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":2829260735},{"type":"v1","addr":"172.21.15.117:6803","nonce":2829260735}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":2829260735},{"type":"v1","addr":"172.21.15.117:6807","nonce":2829260735}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":2829260735},{"type":"v1","addr":"172.21.15.117:6805","nonce":2829260735}]},"public_addr":"172.21.15.117:6801/2829260735","cluster_addr":"172.21.15.117:6803/2829260735","heartbeat_back_addr":"172.21.15.117:6807/2829260735","heartbeat_front_addr":"172.21.15.117:6805/2829260735","state":["exists","up"]},{"osd":3,"uuid":"ec6490cb-a715-4c12-a54a-225aae26fcb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":2823712137},{"type":"v1","addr":"172.21.15.45:6811","nonce":2823712137}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":2823712137},{"type":"v1","addr":"172.21.15.45:6813","nonce":2823712137}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":2823712137},{"type":"v1","addr":"172.21.15.45:6817","nonce":2823712137}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":2823712137},{"type":"v1","addr":"172.21.15.45:6815","nonce":2823712137}]},"public_addr":"172.21.15.45:6811/2823712137","cluster_addr":"172.21.15.45:6813/2823712137","heartbeat_back_addr":"172.21.15.45:6817/2823712137","heartbeat_front_addr":"172.21.15.45:6815/2823712137","state":["exists","up"]},{"osd":4,"uuid":"5d91d568-0250-4aae-8dc2-a0e079f92062","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":1396278127},{"type":"v1","addr":"172.21.15.104:6809","nonce":1396278127}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":1396278127},{"type":"v1","addr":"172.21.15.104:6811","nonce":1396278127}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":1396278127},{"type":"v1","addr":"172.21.15.104:6815","nonce":1396278127}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":1396278127},{"type":"v1","addr":"172.21.15.104:6813","nonce":1396278127}]},"public_addr":"172.21.15.104:6809/1396278127","cluster_addr":"172.21.15.104:6811/1396278127","heartbeat_back_addr":"172.21.15.104:6815/1396278127","heartbeat_front_addr":"172.21.15.104:6813/1396278127","state":["exists","up"]},{"osd":5,"uuid":"ac510b43-6fb2-4d0f-bc5f-16ac080d9e9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":4145720047},{"type":"v1","addr":"172.21.15.117:6809","nonce":4145720047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":4145720047},{"type":"v1","addr":"172.21.15.117:6811","nonce":4145720047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":4145720047},{"type":"v1","addr":"172.21.15.117:6815","nonce":4145720047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":4145720047},{"type":"v1","addr":"172.21.15.117:6813","nonce":4145720047}]},"public_addr":"172.21.15.117:6809/4145720047","cluster_addr":"172.21.15.117:6811/4145720047","heartbeat_back_addr":"172.21.15.117:6815/4145720047","heartbeat_front_addr":"172.21.15.117:6813/4145720047","state":["exists","up"]},{"osd":6,"uuid":"a41adc18-d88a-419e-b2c2-b089c1f32472","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":3254229176},{"type":"v1","addr":"172.21.15.45:6819","nonce":3254229176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":3254229176},{"type":"v1","addr":"172.21.15.45:6821","nonce":3254229176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":3254229176},{"type":"v1","addr":"172.21.15.45:6825","nonce":3254229176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":3254229176},{"type":"v1","addr":"172.21.15.45:6823","nonce":3254229176}]},"public_addr":"172.21.15.45:6819/3254229176","cluster_addr":"172.21.15.45:6821/3254229176","heartbeat_back_addr":"172.21.15.45:6825/3254229176","heartbeat_front_addr":"172.21.15.45:6823/3254229176","state":["exists","up"]},{"osd":7,"uuid":"b71712a4-7cf6-454d-9fd1-565e3df21c1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":4151077186},{"type":"v1","addr":"172.21.15.104:6817","nonce":4151077186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":4151077186},{"type":"v1","addr":"172.21.15.104:6819","nonce":4151077186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":4151077186},{"type":"v1","addr":"172.21.15.104:6823","nonce":4151077186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":4151077186},{"type":"v1","addr":"172.21.15.104:6821","nonce":4151077186}]},"public_addr":"172.21.15.104:6817/4151077186","cluster_addr":"172.21.15.104:6819/4151077186","heartbeat_back_addr":"172.21.15.104:6823/4151077186","heartbeat_front_addr":"172.21.15.104:6821/4151077186","state":["exists","up"]},{"osd":8,"uuid":"9e572aef-59a8-450e-a92e-df9dff4d57a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":1500286830},{"type":"v1","addr":"172.21.15.117:6817","nonce":1500286830}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":1500286830},{"type":"v1","addr":"172.21.15.117:6819","nonce":1500286830}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":1500286830},{"type":"v1","addr":"172.21.15.117:6823","nonce":1500286830}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":1500286830},{"type":"v1","addr":"172.21.15.117:6821","nonce":1500286830}]},"public_addr":"172.21.15.117:6817/1500286830","cluster_addr":"172.21.15.117:6819/1500286830","heartbeat_back_addr":"172.21.15.117:6823/1500286830","heartbeat_front_addr":"172.21.15.117:6821/1500286830","state":["exists","up"]},{"osd":9,"uuid":"014e9598-c413-4cc4-9bb8-61c6e8f59d30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":3106734074},{"type":"v1","addr":"172.21.15.45:6827","nonce":3106734074}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":3106734074},{"type":"v1","addr":"172.21.15.45:6829","nonce":3106734074}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":3106734074},{"type":"v1","addr":"172.21.15.45:6833","nonce":3106734074}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":3106734074},{"type":"v1","addr":"172.21.15.45:6831","nonce":3106734074}]},"public_addr":"172.21.15.45:6827/3106734074","cluster_addr":"172.21.15.45:6829/3106734074","heartbeat_back_addr":"172.21.15.45:6833/3106734074","heartbeat_front_addr":"172.21.15.45:6831/3106734074","state":["exists","up"]},{"osd":10,"uuid":"2deba364-3e39-47be-8a1f-8256df5b82d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":1404794652},{"type":"v1","addr":"172.21.15.104:6825","nonce":1404794652}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":1404794652},{"type":"v1","addr":"172.21.15.104:6827","nonce":1404794652}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":1404794652},{"type":"v1","addr":"172.21.15.104:6831","nonce":1404794652}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":1404794652},{"type":"v1","addr":"172.21.15.104:6829","nonce":1404794652}]},"public_addr":"172.21.15.104:6825/1404794652","cluster_addr":"172.21.15.104:6827/1404794652","heartbeat_back_addr":"172.21.15.104:6831/1404794652","heartbeat_front_addr":"172.21.15.104:6829/1404794652","state":["exists","up"]},{"osd":11,"uuid":"3fda4fea-59ad-4d1a-9fe6-fcb30e31fd7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":3384003243},{"type":"v1","addr":"172.21.15.117:6825","nonce":3384003243}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":3384003243},{"type":"v1","addr":"172.21.15.117:6827","nonce":3384003243}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":3384003243},{"type":"v1","addr":"172.21.15.117:6831","nonce":3384003243}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":3384003243},{"type":"v1","addr":"172.21.15.117:6829","nonce":3384003243}]},"public_addr":"172.21.15.117:6825/3384003243","cluster_addr":"172.21.15.117:6827/3384003243","heartbeat_back_addr":"172.21.15.117:6831/3384003243","heartbeat_front_addr":"172.21.15.117:6829/3384003243","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:57:14.799608+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:57:39.884668+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:58:03.327057+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:58:37.314531+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:59:03.667650+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:59:28.462823+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T17:00:02.968574+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T17:00:29.559190+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T17:00:55.291007+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T17:01:29.815977+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T17:01:57.015580+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T17:02:23.233151+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.5","mappings":[{"from":9,"to":10}]},{"pgid":"4.7","mappings":[{"from":5,"to":8}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.45:0/4074982598":"2024-04-17T16:55:19.948001+0000","172.21.15.45:0/2443572576":"2024-04-17T16:55:19.948001+0000","172.21.15.45:6801/1983630136":"2024-04-17T16:55:19.948001+0000","172.21.15.45:6800/1983630136":"2024-04-17T16:55:19.948001+0000","172.21.15.45:0/3150465248":"2024-04-17T16:54:43.692466+0000","172.21.15.45:0/556738623":"2024-04-17T16:54:43.692466+0000","172.21.15.45:0/445676297":"2024-04-17T16:55:19.948001+0000","172.21.15.45:6800/2553659622":"2024-04-17T16:54:43.692466+0000","172.21.15.45:6801/2553659622":"2024-04-17T16:54:43.692466+0000","172.21.15.45:0/1184615408":"2024-04-17T16:54:43.692466+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-16T17:06:30.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:30.376+0000 7f7fb4a00640 1 -- 172.21.15.45:0/3649771771 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f7f940a3bb0 msgr2=0x7f7f940a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:30.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:30.376+0000 7f7fb4a00640 1 --2- 172.21.15.45:0/3649771771 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f7f940a3bb0 0x7f7f940a6070 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f7fa002c190 tx=0x7f7fa0000bc0 comp rx=0 tx=0).stop 2024-04-16T17:06:30.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:30.376+0000 7f7fb4a00640 1 -- 172.21.15.45:0/3649771771 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7fb0109670 msgr2=0x7f7fb019ed70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:30.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:30.376+0000 7f7fb4a00640 1 --2- 172.21.15.45:0/3649771771 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7fb0109670 0x7f7fb019ed70 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f7f9800e9f0 tx=0x7f7f9800edf0 comp rx=0 tx=0).stop 2024-04-16T17:06:30.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:30.376+0000 7f7fb4a00640 1 -- 172.21.15.45:0/3649771771 shutdown_connections 2024-04-16T17:06:30.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:30.376+0000 7f7fb4a00640 1 --2- 172.21.15.45:0/3649771771 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f7f940a3bb0 0x7f7f940a6070 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:30.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:30.376+0000 7f7fb4a00640 1 --2- 172.21.15.45:0/3649771771 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7fb010a030 0x7f7fb01a5d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:30.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:30.376+0000 7f7fb4a00640 1 --2- 172.21.15.45:0/3649771771 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7fb0109670 0x7f7fb019ed70 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:30.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:30.376+0000 7f7fb4a00640 1 --2- 172.21.15.45:0/3649771771 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7fb0108480 0x7f7fb019e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:30.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:30.376+0000 7f7fb4a00640 1 -- 172.21.15.45:0/3649771771 >> 172.21.15.45:0/3649771771 conn(0x7f7fb0077440 msgr2=0x7f7fb010c970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:06:30.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:30.376+0000 7f7fb4a00640 1 -- 172.21.15.45:0/3649771771 shutdown_connections 2024-04-16T17:06:30.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:30.376+0000 7f7fb4a00640 1 -- 172.21.15.45:0/3649771771 wait complete. 2024-04-16T17:06:30.398 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-16T16:58:06.166659+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-16T17:03:22.304198+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-16T17:03:23.616450+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-16T17:03:27.226289+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-16T17:06:30.398 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-16T17:06:30.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:30.624+0000 7f9277200640 1 -- 172.21.15.45:0/2975697623 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9278077c00 msgr2=0x7f9278078060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:30.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:30.624+0000 7f9277200640 1 --2- 172.21.15.45:0/2975697623 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9278077c00 0x7f9278078060 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f9260009900 tx=0x7f926002bc30 comp rx=0 tx=0).stop 2024-04-16T17:06:30.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:30.625+0000 7f9277200640 1 -- 172.21.15.45:0/2975697623 shutdown_connections 2024-04-16T17:06:30.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:30.625+0000 7f9277200640 1 --2- 172.21.15.45:0/2975697623 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f92780785a0 0x7f9278078e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:30.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:30.625+0000 7f9277200640 1 --2- 172.21.15.45:0/2975697623 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9278077c00 0x7f9278078060 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:30.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:30.625+0000 7f9277200640 1 --2- 172.21.15.45:0/2975697623 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f92780769c0 0x7f9278076de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:30.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:30.625+0000 7f9277200640 1 -- 172.21.15.45:0/2975697623 >> 172.21.15.45:0/2975697623 conn(0x7f92780ff5d0 msgr2=0x7f9278101a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:06:30.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:30.625+0000 7f9277200640 1 -- 172.21.15.45:0/2975697623 shutdown_connections 2024-04-16T17:06:30.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:30.625+0000 7f9277200640 1 -- 172.21.15.45:0/2975697623 wait complete. 2024-04-16T17:06:30.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:30.626+0000 7f9277200640 1 Processor -- start 2024-04-16T17:06:30.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:30.626+0000 7f9277200640 1 -- start start 2024-04-16T17:06:30.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:30.626+0000 7f9277200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f92780769c0 0x7f927819e750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:30.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:30.626+0000 7f9277200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9278077c00 0x7f927819ec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:30.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:30.626+0000 7f9277200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f92780785a0 0x7f92781a5d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:30.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:30.626+0000 7f9277200640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f9278114130 con 0x7f92780785a0 2024-04-16T17:06:30.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:30.626+0000 7f9277200640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f9278113fb0 con 0x7f9278077c00 2024-04-16T17:06:30.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:30.626+0000 7f9277200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f92781142b0 con 0x7f92780769c0 2024-04-16T17:06:30.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:30.627+0000 7f9275e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f92780769c0 0x7f927819e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:30.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:30.627+0000 7f926fe00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9278077c00 0x7f927819ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:30.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:30.627+0000 7f9275e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f92780769c0 0x7f927819e750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:39296/0 (socket says 172.21.15.45:39296) 2024-04-16T17:06:30.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:30.627+0000 7f9276800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f92780785a0 0x7f92781a5d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:30.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:30.627+0000 7f9275e00640 1 -- 172.21.15.45:0/1820700391 learned_addr learned my addr 172.21.15.45:0/1820700391 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:06:30.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:30.627+0000 7f9275e00640 1 -- 172.21.15.45:0/1820700391 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9278077c00 msgr2=0x7f927819ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:30.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:30.627+0000 7f9275e00640 1 --2- 172.21.15.45:0/1820700391 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9278077c00 0x7f927819ec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:30.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:30.627+0000 7f9275e00640 1 -- 172.21.15.45:0/1820700391 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f92780785a0 msgr2=0x7f92781a5d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:30.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:30.627+0000 7f9275e00640 1 --2- 172.21.15.45:0/1820700391 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f92780785a0 0x7f92781a5d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:30.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:30.627+0000 7f9275e00640 1 -- 172.21.15.45:0/1820700391 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92781a6370 con 0x7f92780769c0 2024-04-16T17:06:30.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:30.627+0000 7f9275e00640 1 --2- 172.21.15.45:0/1820700391 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f92780769c0 0x7f927819e750 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f926800e9f0 tx=0x7f926800edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:06:30.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:30.628+0000 7f926ea00640 1 -- 172.21.15.45:0/1820700391 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f926800cd80 con 0x7f92780769c0 2024-04-16T17:06:30.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:30.628+0000 7f926ea00640 1 -- 172.21.15.45:0/1820700391 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9268005500 con 0x7f92780769c0 2024-04-16T17:06:30.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:30.628+0000 7f926ea00640 1 -- 172.21.15.45:0/1820700391 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9268010690 con 0x7f92780769c0 2024-04-16T17:06:30.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:30.628+0000 7f9277200640 1 -- 172.21.15.45:0/1820700391 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92781a6660 con 0x7f92780769c0 2024-04-16T17:06:30.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:30.628+0000 7f9277200640 1 -- 172.21.15.45:0/1820700391 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92781a6b30 con 0x7f92780769c0 2024-04-16T17:06:30.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:30.629+0000 7f9277200640 1 -- 172.21.15.45:0/1820700391 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f92781109d0 con 0x7f92780769c0 2024-04-16T17:06:30.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:30.632+0000 7f926ea00640 1 -- 172.21.15.45:0/1820700391 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f92680046c0 con 0x7f92780769c0 2024-04-16T17:06:30.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:30.633+0000 7f926ea00640 1 --2- 172.21.15.45:0/1820700391 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f925c0a3bb0 0x7f925c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:30.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:30.633+0000 7f926ea00640 1 -- 172.21.15.45:0/1820700391 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f92680d0bf0 con 0x7f92780769c0 2024-04-16T17:06:30.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:30.633+0000 7f926fe00640 1 --2- 172.21.15.45:0/1820700391 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f925c0a3bb0 0x7f925c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:30.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:30.633+0000 7f926fe00640 1 --2- 172.21.15.45:0/1820700391 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f925c0a3bb0 0x7f925c0a6070 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f926002c240 tx=0x7f926002df90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:06:30.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:30.638+0000 7f926ea00640 1 -- 172.21.15.45:0/1820700391 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f926809d1e0 con 0x7f92780769c0 2024-04-16T17:06:30.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:06:30 smithi104 ceph-mon[36457]: from='client.? 172.21.15.45:0/1810601579' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-16T17:06:30.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:06:30 smithi104 ceph-mon[36457]: from='client.? 172.21.15.45:0/3138436626' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:06:30.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:06:30 smithi045 ceph-mon[31676]: from='client.? 172.21.15.45:0/1810601579' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-16T17:06:30.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:06:30 smithi045 ceph-mon[31676]: from='client.? 172.21.15.45:0/3138436626' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:06:30.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:06:30 smithi117 ceph-mon[36730]: from='client.? 172.21.15.45:0/1810601579' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-16T17:06:30.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:06:30 smithi117 ceph-mon[36730]: from='client.? 172.21.15.45:0/3138436626' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:06:30.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:30.844+0000 7f9277200640 1 -- 172.21.15.45:0/1820700391 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f9278062c00 con 0x7f92780769c0 2024-04-16T17:06:30.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:30.845+0000 7f926ea00640 1 -- 172.21.15.45:0/1820700391 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v94) ==== 93+0+10 (secure 0 0 0) 0x7f926809cae0 con 0x7f92780769c0 2024-04-16T17:06:30.846 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 1 2024-04-16T17:06:30.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:30.847+0000 7f9277200640 1 -- 172.21.15.45:0/1820700391 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f925c0a3bb0 msgr2=0x7f925c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:30.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:30.847+0000 7f9277200640 1 --2- 172.21.15.45:0/1820700391 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f925c0a3bb0 0x7f925c0a6070 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f926002c240 tx=0x7f926002df90 comp rx=0 tx=0).stop 2024-04-16T17:06:30.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:30.847+0000 7f9277200640 1 -- 172.21.15.45:0/1820700391 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f92780769c0 msgr2=0x7f927819e750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:30.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:30.847+0000 7f9277200640 1 --2- 172.21.15.45:0/1820700391 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f92780769c0 0x7f927819e750 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f926800e9f0 tx=0x7f926800edf0 comp rx=0 tx=0).stop 2024-04-16T17:06:30.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:30.847+0000 7f9277200640 1 -- 172.21.15.45:0/1820700391 shutdown_connections 2024-04-16T17:06:30.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:30.847+0000 7f9277200640 1 --2- 172.21.15.45:0/1820700391 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f925c0a3bb0 0x7f925c0a6070 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:30.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:30.847+0000 7f9277200640 1 --2- 172.21.15.45:0/1820700391 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f92780785a0 0x7f92781a5d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:30.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:30.847+0000 7f9277200640 1 --2- 172.21.15.45:0/1820700391 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9278077c00 0x7f927819ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:30.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:30.847+0000 7f9277200640 1 --2- 172.21.15.45:0/1820700391 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f92780769c0 0x7f927819e750 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:30.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:30.848+0000 7f9277200640 1 -- 172.21.15.45:0/1820700391 >> 172.21.15.45:0/1820700391 conn(0x7f92780ff5d0 msgr2=0x7f92781010d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:06:30.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:30.848+0000 7f9277200640 1 -- 172.21.15.45:0/1820700391 shutdown_connections 2024-04-16T17:06:30.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:30.848+0000 7f9277200640 1 -- 172.21.15.45:0/1820700391 wait complete. 2024-04-16T17:06:30.869 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-16T17:06:31.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:31.099+0000 7fa459c00640 1 -- 172.21.15.45:0/1815736371 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa454103c00 msgr2=0x7fa45410a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:31.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:31.099+0000 7fa459c00640 1 --2- 172.21.15.45:0/1815736371 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa454103c00 0x7fa45410a490 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fa44c009be0 tx=0x7fa44c02bef0 comp rx=0 tx=0).stop 2024-04-16T17:06:31.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:31.099+0000 7fa459c00640 1 -- 172.21.15.45:0/1815736371 shutdown_connections 2024-04-16T17:06:31.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:31.099+0000 7fa459c00640 1 --2- 172.21.15.45:0/1815736371 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa454103c00 0x7fa45410a490 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:31.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:31.099+0000 7fa459c00640 1 --2- 172.21.15.45:0/1815736371 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa454103240 0x7fa4541036c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:31.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:31.099+0000 7fa459c00640 1 --2- 172.21.15.45:0/1815736371 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa454102050 0x7fa454102470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:31.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:31.099+0000 7fa459c00640 1 -- 172.21.15.45:0/1815736371 >> 172.21.15.45:0/1815736371 conn(0x7fa4540fd5e0 msgr2=0x7fa4540ffa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:06:31.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:31.099+0000 7fa459c00640 1 -- 172.21.15.45:0/1815736371 shutdown_connections 2024-04-16T17:06:31.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:31.099+0000 7fa459c00640 1 -- 172.21.15.45:0/1815736371 wait complete. 2024-04-16T17:06:31.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:31.100+0000 7fa459c00640 1 Processor -- start 2024-04-16T17:06:31.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:31.100+0000 7fa459c00640 1 -- start start 2024-04-16T17:06:31.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:31.101+0000 7fa459c00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa454102050 0x7fa45419a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:31.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:31.101+0000 7fa459c00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa454103240 0x7fa45419aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:31.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:31.101+0000 7fa459c00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa454103c00 0x7fa4541a1aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:31.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:31.101+0000 7fa459c00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fa45410dd30 con 0x7fa454103240 2024-04-16T17:06:31.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:31.101+0000 7fa459c00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fa45410dbb0 con 0x7fa454102050 2024-04-16T17:06:31.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:31.101+0000 7fa459c00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa45410deb0 con 0x7fa454103c00 2024-04-16T17:06:31.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:31.101+0000 7fa452000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa454103240 0x7fa45419aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:31.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:31.101+0000 7fa452000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa454103240 0x7fa45419aa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55454/0 (socket says 172.21.15.45:55454) 2024-04-16T17:06:31.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:31.101+0000 7fa452a00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa454102050 0x7fa45419a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:31.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:31.101+0000 7fa452000640 1 -- 172.21.15.45:0/781425113 learned_addr learned my addr 172.21.15.45:0/781425113 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:06:31.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:31.101+0000 7fa453400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa454103c00 0x7fa4541a1aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:31.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:31.101+0000 7fa452000640 1 -- 172.21.15.45:0/781425113 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa454103c00 msgr2=0x7fa4541a1aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:31.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:31.101+0000 7fa452000640 1 --2- 172.21.15.45:0/781425113 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa454103c00 0x7fa4541a1aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:31.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:31.101+0000 7fa452000640 1 -- 172.21.15.45:0/781425113 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa454102050 msgr2=0x7fa45419a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:31.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:31.101+0000 7fa452000640 1 --2- 172.21.15.45:0/781425113 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa454102050 0x7fa45419a4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:31.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:31.101+0000 7fa452000640 1 -- 172.21.15.45:0/781425113 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4541a2110 con 0x7fa454103240 2024-04-16T17:06:31.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:31.102+0000 7fa453400640 1 --2- 172.21.15.45:0/781425113 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa454103c00 0x7fa4541a1aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:06:31.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:31.102+0000 7fa452a00640 1 --2- 172.21.15.45:0/781425113 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa454102050 0x7fa45419a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:06:31.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:31.102+0000 7fa452000640 1 --2- 172.21.15.45:0/781425113 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa454103240 0x7fa45419aa30 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fa44400e9f0 tx=0x7fa44400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:06:31.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:31.102+0000 7fa44b400640 1 -- 172.21.15.45:0/781425113 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa44400ccc0 con 0x7fa454103240 2024-04-16T17:06:31.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:31.102+0000 7fa459c00640 1 -- 172.21.15.45:0/781425113 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4541a2400 con 0x7fa454103240 2024-04-16T17:06:31.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:31.102+0000 7fa44b400640 1 -- 172.21.15.45:0/781425113 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa444010910 con 0x7fa454103240 2024-04-16T17:06:31.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:31.102+0000 7fa44b400640 1 -- 172.21.15.45:0/781425113 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4440188a0 con 0x7fa454103240 2024-04-16T17:06:31.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:31.102+0000 7fa459c00640 1 -- 172.21.15.45:0/781425113 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4541a29a0 con 0x7fa454103240 2024-04-16T17:06:31.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:31.104+0000 7fa459c00640 1 -- 172.21.15.45:0/781425113 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4541a2c00 con 0x7fa454103240 2024-04-16T17:06:31.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:31.107+0000 7fa44b400640 1 -- 172.21.15.45:0/781425113 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fa444010ac0 con 0x7fa454103240 2024-04-16T17:06:31.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:31.108+0000 7fa44b400640 1 --2- 172.21.15.45:0/781425113 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fa4380a3bb0 0x7fa4380a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:31.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:31.108+0000 7fa44b400640 1 -- 172.21.15.45:0/781425113 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fa4440d0c20 con 0x7fa454103240 2024-04-16T17:06:31.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:31.108+0000 7fa452a00640 1 --2- 172.21.15.45:0/781425113 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fa4380a3bb0 0x7fa4380a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:31.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:31.108+0000 7fa452a00640 1 --2- 172.21.15.45:0/781425113 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fa4380a3bb0 0x7fa4380a6070 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fa4541030a0 tx=0x7fa43c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:06:31.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:31.113+0000 7fa44b400640 1 -- 172.21.15.45:0/781425113 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa44409d210 con 0x7fa454103240 2024-04-16T17:06:31.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:31.323+0000 7fa459c00640 1 -- 172.21.15.45:0/781425113 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fa4541a2dd0 con 0x7fa454103240 2024-04-16T17:06:31.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:31.328+0000 7fa44b400640 1 -- 172.21.15.45:0/781425113 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v94) ==== 104+0+11 (secure 0 0 0) 0x7fa44409cb10 con 0x7fa454103240 2024-04-16T17:06:31.328 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 32 2024-04-16T17:06:31.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:31.330+0000 7fa459c00640 1 -- 172.21.15.45:0/781425113 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fa4380a3bb0 msgr2=0x7fa4380a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:31.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:31.330+0000 7fa459c00640 1 --2- 172.21.15.45:0/781425113 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fa4380a3bb0 0x7fa4380a6070 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fa4541030a0 tx=0x7fa43c009260 comp rx=0 tx=0).stop 2024-04-16T17:06:31.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:31.330+0000 7fa459c00640 1 -- 172.21.15.45:0/781425113 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa454103240 msgr2=0x7fa45419aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:31.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:31.330+0000 7fa459c00640 1 --2- 172.21.15.45:0/781425113 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa454103240 0x7fa45419aa30 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fa44400e9f0 tx=0x7fa44400edf0 comp rx=0 tx=0).stop 2024-04-16T17:06:31.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:31.330+0000 7fa459c00640 1 -- 172.21.15.45:0/781425113 shutdown_connections 2024-04-16T17:06:31.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:31.330+0000 7fa459c00640 1 --2- 172.21.15.45:0/781425113 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fa4380a3bb0 0x7fa4380a6070 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:31.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:31.330+0000 7fa459c00640 1 --2- 172.21.15.45:0/781425113 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa454103c00 0x7fa4541a1aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:31.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:31.330+0000 7fa459c00640 1 --2- 172.21.15.45:0/781425113 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa454103240 0x7fa45419aa30 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:31.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:31.330+0000 7fa459c00640 1 --2- 172.21.15.45:0/781425113 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa454102050 0x7fa45419a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:31.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:31.330+0000 7fa459c00640 1 -- 172.21.15.45:0/781425113 >> 172.21.15.45:0/781425113 conn(0x7fa4540fd5e0 msgr2=0x7fa454074230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:06:31.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:31.331+0000 7fa459c00640 1 -- 172.21.15.45:0/781425113 shutdown_connections 2024-04-16T17:06:31.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:31.331+0000 7fa459c00640 1 -- 172.21.15.45:0/781425113 wait complete. 2024-04-16T17:06:31.350 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-16T17:06:31.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:31.581+0000 7f13f3e00640 1 -- 172.21.15.45:0/3895929845 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f13f40684b0 msgr2=0x7f13f4101340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:31.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:31.581+0000 7f13f3e00640 1 --2- 172.21.15.45:0/3895929845 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f13f40684b0 0x7f13f4101340 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f13e4009930 tx=0x7f13e402bc20 comp rx=0 tx=0).stop 2024-04-16T17:06:31.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:31.581+0000 7f13f3e00640 1 -- 172.21.15.45:0/3895929845 shutdown_connections 2024-04-16T17:06:31.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:31.581+0000 7f13f3e00640 1 --2- 172.21.15.45:0/3895929845 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f13f4106580 0x7f13f4108970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:31.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:31.581+0000 7f13f3e00640 1 --2- 172.21.15.45:0/3895929845 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f13f4101880 0x7f13f4101d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:31.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:31.581+0000 7f13f3e00640 1 --2- 172.21.15.45:0/3895929845 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f13f40684b0 0x7f13f4101340 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:31.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:31.581+0000 7f13f3e00640 1 -- 172.21.15.45:0/3895929845 >> 172.21.15.45:0/3895929845 conn(0x7f13f40fb6c0 msgr2=0x7f13f40fdb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:06:31.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:31.581+0000 7f13f3e00640 1 -- 172.21.15.45:0/3895929845 shutdown_connections 2024-04-16T17:06:31.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:31.581+0000 7f13f3e00640 1 -- 172.21.15.45:0/3895929845 wait complete. 2024-04-16T17:06:31.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:31.582+0000 7f13f3e00640 1 Processor -- start 2024-04-16T17:06:31.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:31.582+0000 7f13f3e00640 1 -- start start 2024-04-16T17:06:31.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:31.582+0000 7f13f3e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f13f40684b0 0x7f13f419a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:31.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:31.583+0000 7f13f3e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f13f4101880 0x7f13f419aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:31.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:31.583+0000 7f13f3e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f13f4106580 0x7f13f41a1b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:31.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:31.583+0000 7f13f3e00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f13f410b9b0 con 0x7f13f4101880 2024-04-16T17:06:31.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:31.583+0000 7f13f3e00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f13f410b830 con 0x7f13f4106580 2024-04-16T17:06:31.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:31.583+0000 7f13f3e00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f13f410bb30 con 0x7f13f40684b0 2024-04-16T17:06:31.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:31.583+0000 7f13ebe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f13f4101880 0x7f13f419aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:31.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:31.583+0000 7f13ebe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f13f4101880 0x7f13f419aaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55468/0 (socket says 172.21.15.45:55468) 2024-04-16T17:06:31.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:31.583+0000 7f13ebe00640 1 -- 172.21.15.45:0/3250549147 learned_addr learned my addr 172.21.15.45:0/3250549147 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:06:31.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:31.583+0000 7f13f2a00640 1 --2- 172.21.15.45:0/3250549147 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f13f40684b0 0x7f13f419a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:31.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:31.583+0000 7f13f3400640 1 --2- 172.21.15.45:0/3250549147 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f13f4106580 0x7f13f41a1b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:31.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:31.583+0000 7f13ebe00640 1 -- 172.21.15.45:0/3250549147 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f13f40684b0 msgr2=0x7f13f419a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:31.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:31.583+0000 7f13ebe00640 1 --2- 172.21.15.45:0/3250549147 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f13f40684b0 0x7f13f419a560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:31.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:31.583+0000 7f13ebe00640 1 -- 172.21.15.45:0/3250549147 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f13f4106580 msgr2=0x7f13f41a1b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:31.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:31.583+0000 7f13ebe00640 1 --2- 172.21.15.45:0/3250549147 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f13f4106580 0x7f13f41a1b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:31.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:31.583+0000 7f13ebe00640 1 -- 172.21.15.45:0/3250549147 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13f41a2180 con 0x7f13f4101880 2024-04-16T17:06:31.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:31.583+0000 7f13f2a00640 1 --2- 172.21.15.45:0/3250549147 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f13f40684b0 0x7f13f419a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:06:31.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:31.584+0000 7f13ebe00640 1 --2- 172.21.15.45:0/3250549147 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f13f4101880 0x7f13f419aaa0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f13dc00e9f0 tx=0x7f13dc00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:06:31.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:31.584+0000 7f13eb400640 1 -- 172.21.15.45:0/3250549147 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13dc00cce0 con 0x7f13f4101880 2024-04-16T17:06:31.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:31.584+0000 7f13eb400640 1 -- 172.21.15.45:0/3250549147 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f13dc0035f0 con 0x7f13f4101880 2024-04-16T17:06:31.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:31.584+0000 7f13f3e00640 1 -- 172.21.15.45:0/3250549147 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13f41a2470 con 0x7f13f4101880 2024-04-16T17:06:31.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:31.584+0000 7f13eb400640 1 -- 172.21.15.45:0/3250549147 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13dc016670 con 0x7f13f4101880 2024-04-16T17:06:31.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:31.584+0000 7f13f3e00640 1 -- 172.21.15.45:0/3250549147 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13f41a29c0 con 0x7f13f4101880 2024-04-16T17:06:31.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:31.585+0000 7f13f3e00640 1 -- 172.21.15.45:0/3250549147 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f13f4105390 con 0x7f13f4101880 2024-04-16T17:06:31.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:31.587+0000 7f13eb400640 1 -- 172.21.15.45:0/3250549147 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f13dc01d020 con 0x7f13f4101880 2024-04-16T17:06:31.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:31.589+0000 7f13eb400640 1 --2- 172.21.15.45:0/3250549147 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f13d80a3bb0 0x7f13d80a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:31.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:31.589+0000 7f13eb400640 1 -- 172.21.15.45:0/3250549147 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f13dc0d02f0 con 0x7f13f4101880 2024-04-16T17:06:31.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:31.589+0000 7f13f2a00640 1 --2- 172.21.15.45:0/3250549147 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f13d80a3bb0 0x7f13d80a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:31.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:31.590+0000 7f13f2a00640 1 --2- 172.21.15.45:0/3250549147 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f13d80a3bb0 0x7f13d80a6070 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f13e4009900 tx=0x7f13e4000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:06:31.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:31.593+0000 7f13eb400640 1 -- 172.21.15.45:0/3250549147 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f13dc09c8e0 con 0x7f13f4101880 2024-04-16T17:06:31.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:06:31 smithi104 ceph-mon[36457]: pgmap v419: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-16T17:06:31.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:06:31 smithi104 ceph-mon[36457]: from='client.? 172.21.15.45:0/3649771771' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T17:06:31.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:06:31 smithi104 ceph-mon[36457]: from='client.? 172.21.15.45:0/1820700391' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-16T17:06:31.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:06:31 smithi045 ceph-mon[31676]: pgmap v419: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-16T17:06:31.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:06:31 smithi045 ceph-mon[31676]: from='client.? 172.21.15.45:0/3649771771' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T17:06:31.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:06:31 smithi045 ceph-mon[31676]: from='client.? 172.21.15.45:0/1820700391' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-16T17:06:31.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:31.801+0000 7f13f3e00640 1 -- 172.21.15.45:0/3250549147 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f13f4062c00 con 0x7f13f4101880 2024-04-16T17:06:31.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:31.802+0000 7f13eb400640 1 -- 172.21.15.45:0/3250549147 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v94) ==== 100+0+11 (secure 0 0 0) 0x7f13dc09c1e0 con 0x7f13f4101880 2024-04-16T17:06:31.803 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 64 2024-04-16T17:06:31.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:31.805+0000 7f13f3e00640 1 -- 172.21.15.45:0/3250549147 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f13d80a3bb0 msgr2=0x7f13d80a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:31.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:31.805+0000 7f13f3e00640 1 --2- 172.21.15.45:0/3250549147 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f13d80a3bb0 0x7f13d80a6070 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f13e4009900 tx=0x7f13e4000bc0 comp rx=0 tx=0).stop 2024-04-16T17:06:31.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:31.805+0000 7f13f3e00640 1 -- 172.21.15.45:0/3250549147 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f13f4101880 msgr2=0x7f13f419aaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:31.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:31.805+0000 7f13f3e00640 1 --2- 172.21.15.45:0/3250549147 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f13f4101880 0x7f13f419aaa0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f13dc00e9f0 tx=0x7f13dc00eeb0 comp rx=0 tx=0).stop 2024-04-16T17:06:31.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:31.805+0000 7f13f3e00640 1 -- 172.21.15.45:0/3250549147 shutdown_connections 2024-04-16T17:06:31.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:31.805+0000 7f13f3e00640 1 --2- 172.21.15.45:0/3250549147 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f13d80a3bb0 0x7f13d80a6070 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:31.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:31.805+0000 7f13f3e00640 1 --2- 172.21.15.45:0/3250549147 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f13f4106580 0x7f13f41a1b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:31.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:31.805+0000 7f13f3e00640 1 --2- 172.21.15.45:0/3250549147 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f13f4101880 0x7f13f419aaa0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:31.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:31.805+0000 7f13f3e00640 1 --2- 172.21.15.45:0/3250549147 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f13f40684b0 0x7f13f419a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:31.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:31.805+0000 7f13f3e00640 1 -- 172.21.15.45:0/3250549147 >> 172.21.15.45:0/3250549147 conn(0x7f13f40fb6c0 msgr2=0x7f13f4102560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:06:31.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:31.805+0000 7f13f3e00640 1 -- 172.21.15.45:0/3250549147 shutdown_connections 2024-04-16T17:06:31.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:31.805+0000 7f13f3e00640 1 -- 172.21.15.45:0/3250549147 wait complete. 2024-04-16T17:06:31.807 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:06:31 smithi117 ceph-mon[36730]: pgmap v419: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-16T17:06:31.807 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:06:31 smithi117 ceph-mon[36730]: from='client.? 172.21.15.45:0/3649771771' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T17:06:31.807 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:06:31 smithi117 ceph-mon[36730]: from='client.? 172.21.15.45:0/1820700391' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-16T17:06:31.827 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-16T17:06:32.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:32.065+0000 7f891a400640 1 -- 172.21.15.45:0/2161661233 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8914109dd0 msgr2=0x7f891410c260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:32.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:32.065+0000 7f891a400640 1 --2- 172.21.15.45:0/2161661233 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8914109dd0 0x7f891410c260 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f890c009ca0 tx=0x7f890c02c0f0 comp rx=0 tx=0).stop 2024-04-16T17:06:32.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:32.065+0000 7f891a400640 1 -- 172.21.15.45:0/2161661233 shutdown_connections 2024-04-16T17:06:32.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:32.065+0000 7f891a400640 1 --2- 172.21.15.45:0/2161661233 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8914074a60 0x7f8914074ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:32.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:32.065+0000 7f891a400640 1 --2- 172.21.15.45:0/2161661233 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8914109dd0 0x7f891410c260 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:32.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:32.065+0000 7f891a400640 1 --2- 172.21.15.45:0/2161661233 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8914076040 0x7f8914074490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:32.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:32.065+0000 7f891a400640 1 -- 172.21.15.45:0/2161661233 >> 172.21.15.45:0/2161661233 conn(0x7f89140fd4a0 msgr2=0x7f89140ff8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:06:32.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:32.065+0000 7f891a400640 1 -- 172.21.15.45:0/2161661233 shutdown_connections 2024-04-16T17:06:32.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:32.065+0000 7f891a400640 1 -- 172.21.15.45:0/2161661233 wait complete. 2024-04-16T17:06:32.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:32.066+0000 7f891a400640 1 Processor -- start 2024-04-16T17:06:32.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:32.066+0000 7f891a400640 1 -- start start 2024-04-16T17:06:32.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:32.067+0000 7f891a400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8914074a60 0x7f891419a360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:32.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:32.067+0000 7f891a400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8914076040 0x7f891419a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:32.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:32.067+0000 7f891a400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8914109dd0 0x7f89141a1910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:32.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:32.067+0000 7f891a400640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f891410fd90 con 0x7f8914074a60 2024-04-16T17:06:32.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:32.067+0000 7f891a400640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f891410fc10 con 0x7f8914076040 2024-04-16T17:06:32.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:32.067+0000 7f891a400640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f891410ff10 con 0x7f8914109dd0 2024-04-16T17:06:32.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:32.067+0000 7f8919000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8914074a60 0x7f891419a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:32.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:32.067+0000 7f8919000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8914074a60 0x7f891419a360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55494/0 (socket says 172.21.15.45:55494) 2024-04-16T17:06:32.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:32.067+0000 7f8919000640 1 -- 172.21.15.45:0/2513977137 learned_addr learned my addr 172.21.15.45:0/2513977137 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:06:32.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:32.067+0000 7f8913e00640 1 --2- 172.21.15.45:0/2513977137 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8914076040 0x7f891419a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:32.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:32.067+0000 7f8919a00640 1 --2- 172.21.15.45:0/2513977137 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8914109dd0 0x7f89141a1910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:32.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:32.067+0000 7f8919000640 1 -- 172.21.15.45:0/2513977137 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8914109dd0 msgr2=0x7f89141a1910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:32.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:32.067+0000 7f8919000640 1 --2- 172.21.15.45:0/2513977137 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8914109dd0 0x7f89141a1910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:32.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:32.067+0000 7f8919000640 1 -- 172.21.15.45:0/2513977137 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8914076040 msgr2=0x7f891419a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:32.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:32.067+0000 7f8919000640 1 --2- 172.21.15.45:0/2513977137 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8914076040 0x7f891419a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:32.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:32.067+0000 7f8919000640 1 -- 172.21.15.45:0/2513977137 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89141a1f80 con 0x7f8914074a60 2024-04-16T17:06:32.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:32.068+0000 7f8919000640 1 --2- 172.21.15.45:0/2513977137 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8914074a60 0x7f891419a360 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f890800e9f0 tx=0x7f890800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:06:32.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:32.068+0000 7f8911600640 1 -- 172.21.15.45:0/2513977137 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f890800cd80 con 0x7f8914074a60 2024-04-16T17:06:32.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:32.068+0000 7f8911600640 1 -- 172.21.15.45:0/2513977137 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8908010910 con 0x7f8914074a60 2024-04-16T17:06:32.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:32.068+0000 7f8911600640 1 -- 172.21.15.45:0/2513977137 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8908018990 con 0x7f8914074a60 2024-04-16T17:06:32.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:32.068+0000 7f891a400640 1 -- 172.21.15.45:0/2513977137 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89141a2270 con 0x7f8914074a60 2024-04-16T17:06:32.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:32.068+0000 7f891a400640 1 -- 172.21.15.45:0/2513977137 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89141a2770 con 0x7f8914074a60 2024-04-16T17:06:32.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:32.070+0000 7f891a400640 1 -- 172.21.15.45:0/2513977137 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8914108b00 con 0x7f8914074a60 2024-04-16T17:06:32.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:32.072+0000 7f8911600640 1 -- 172.21.15.45:0/2513977137 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f8908010ac0 con 0x7f8914074a60 2024-04-16T17:06:32.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:32.074+0000 7f8911600640 1 --2- 172.21.15.45:0/2513977137 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f88fc0a3bb0 0x7f88fc0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:32.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:32.074+0000 7f8911600640 1 -- 172.21.15.45:0/2513977137 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f89080d0b10 con 0x7f8914074a60 2024-04-16T17:06:32.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:32.074+0000 7f8913e00640 1 --2- 172.21.15.45:0/2513977137 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f88fc0a3bb0 0x7f88fc0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:32.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:32.075+0000 7f8913e00640 1 --2- 172.21.15.45:0/2513977137 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f88fc0a3bb0 0x7f88fc0a6070 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f891419b870 tx=0x7f890400b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:06:32.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:32.078+0000 7f8911600640 1 -- 172.21.15.45:0/2513977137 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f890809cab0 con 0x7f8914074a60 2024-04-16T17:06:32.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:32.285+0000 7f891a400640 1 -- 172.21.15.45:0/2513977137 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f8914062c00 con 0x7f8914074a60 2024-04-16T17:06:32.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:32.286+0000 7f8911600640 1 -- 172.21.15.45:0/2513977137 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v94) ==== 103+0+11 (secure 0 0 0) 0x7f89080a0db0 con 0x7f8914074a60 2024-04-16T17:06:32.286 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 32 2024-04-16T17:06:32.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:32.289+0000 7f891a400640 1 -- 172.21.15.45:0/2513977137 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f88fc0a3bb0 msgr2=0x7f88fc0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:32.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:32.289+0000 7f891a400640 1 --2- 172.21.15.45:0/2513977137 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f88fc0a3bb0 0x7f88fc0a6070 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f891419b870 tx=0x7f890400b390 comp rx=0 tx=0).stop 2024-04-16T17:06:32.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:32.289+0000 7f891a400640 1 -- 172.21.15.45:0/2513977137 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8914074a60 msgr2=0x7f891419a360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:32.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:32.289+0000 7f891a400640 1 --2- 172.21.15.45:0/2513977137 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8914074a60 0x7f891419a360 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f890800e9f0 tx=0x7f890800eeb0 comp rx=0 tx=0).stop 2024-04-16T17:06:32.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:32.289+0000 7f891a400640 1 -- 172.21.15.45:0/2513977137 shutdown_connections 2024-04-16T17:06:32.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:32.289+0000 7f891a400640 1 --2- 172.21.15.45:0/2513977137 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f88fc0a3bb0 0x7f88fc0a6070 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:32.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:32.289+0000 7f891a400640 1 --2- 172.21.15.45:0/2513977137 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8914109dd0 0x7f89141a1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:32.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:32.289+0000 7f891a400640 1 --2- 172.21.15.45:0/2513977137 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8914076040 0x7f891419a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:32.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:32.289+0000 7f891a400640 1 --2- 172.21.15.45:0/2513977137 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8914074a60 0x7f891419a360 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:32.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:32.289+0000 7f891a400640 1 -- 172.21.15.45:0/2513977137 >> 172.21.15.45:0/2513977137 conn(0x7f89140fd4a0 msgr2=0x7f8914105cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:06:32.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:32.289+0000 7f891a400640 1 -- 172.21.15.45:0/2513977137 shutdown_connections 2024-04-16T17:06:32.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:32.289+0000 7f891a400640 1 -- 172.21.15.45:0/2513977137 wait complete. 2024-04-16T17:06:32.308 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-16T17:06:32.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:32.539+0000 7f2f98600640 1 -- 172.21.15.45:0/1625146441 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2f90109ee0 msgr2=0x7f2f9010c380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:32.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:32.539+0000 7f2f98600640 1 --2- 172.21.15.45:0/1625146441 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2f90109ee0 0x7f2f9010c380 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f2f88009ca0 tx=0x7f2f8802c190 comp rx=0 tx=0).stop 2024-04-16T17:06:32.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:32.539+0000 7f2f98600640 1 -- 172.21.15.45:0/1625146441 shutdown_connections 2024-04-16T17:06:32.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:32.539+0000 7f2f98600640 1 --2- 172.21.15.45:0/1625146441 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2f90074a00 0x7f2f90074e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:32.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:32.539+0000 7f2f98600640 1 --2- 172.21.15.45:0/1625146441 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2f90109ee0 0x7f2f9010c380 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:32.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:32.539+0000 7f2f98600640 1 --2- 172.21.15.45:0/1625146441 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2f90075fe0 0x7f2f90074430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:32.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:32.539+0000 7f2f98600640 1 -- 172.21.15.45:0/1625146441 >> 172.21.15.45:0/1625146441 conn(0x7f2f900fd600 msgr2=0x7f2f900ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:06:32.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:32.539+0000 7f2f98600640 1 -- 172.21.15.45:0/1625146441 shutdown_connections 2024-04-16T17:06:32.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:32.539+0000 7f2f98600640 1 -- 172.21.15.45:0/1625146441 wait complete. 2024-04-16T17:06:32.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:32.541+0000 7f2f98600640 1 Processor -- start 2024-04-16T17:06:32.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:32.541+0000 7f2f98600640 1 -- start start 2024-04-16T17:06:32.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:32.541+0000 7f2f98600640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2f90074a00 0x7f2f9019a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:32.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:32.541+0000 7f2f98600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2f90075fe0 0x7f2f9019aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:32.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:32.541+0000 7f2f98600640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2f90109ee0 0x7f2f901a1ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:32.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:32.541+0000 7f2f98600640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f2f9010fee0 con 0x7f2f90075fe0 2024-04-16T17:06:32.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:32.542+0000 7f2f98600640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f2f9010fd60 con 0x7f2f90074a00 2024-04-16T17:06:32.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:32.542+0000 7f2f98600640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f2f90110060 con 0x7f2f90109ee0 2024-04-16T17:06:32.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:32.542+0000 7f2f95200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2f90075fe0 0x7f2f9019aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:32.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:32.542+0000 7f2f95200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2f90075fe0 0x7f2f9019aa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55516/0 (socket says 172.21.15.45:55516) 2024-04-16T17:06:32.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:32.542+0000 7f2f95200640 1 -- 172.21.15.45:0/1152120360 learned_addr learned my addr 172.21.15.45:0/1152120360 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:06:32.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:32.542+0000 7f2f95c00640 1 --2- 172.21.15.45:0/1152120360 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2f90074a00 0x7f2f9019a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:32.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:32.542+0000 7f2f96600640 1 --2- 172.21.15.45:0/1152120360 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2f90109ee0 0x7f2f901a1ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:32.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:32.542+0000 7f2f95200640 1 -- 172.21.15.45:0/1152120360 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2f90109ee0 msgr2=0x7f2f901a1ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:32.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:32.542+0000 7f2f95200640 1 --2- 172.21.15.45:0/1152120360 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2f90109ee0 0x7f2f901a1ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:32.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:32.542+0000 7f2f95200640 1 -- 172.21.15.45:0/1152120360 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2f90074a00 msgr2=0x7f2f9019a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:32.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:32.542+0000 7f2f95200640 1 --2- 172.21.15.45:0/1152120360 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2f90074a00 0x7f2f9019a510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:32.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:32.542+0000 7f2f95200640 1 -- 172.21.15.45:0/1152120360 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f901a2130 con 0x7f2f90075fe0 2024-04-16T17:06:32.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:32.542+0000 7f2f96600640 1 --2- 172.21.15.45:0/1152120360 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2f90109ee0 0x7f2f901a1ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:06:32.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:32.543+0000 7f2f95200640 1 --2- 172.21.15.45:0/1152120360 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2f90075fe0 0x7f2f9019aa50 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f2f7c00e9f0 tx=0x7f2f7c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:06:32.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:32.543+0000 7f2f8e000640 1 -- 172.21.15.45:0/1152120360 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f7c00cd80 con 0x7f2f90075fe0 2024-04-16T17:06:32.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:32.543+0000 7f2f8e000640 1 -- 172.21.15.45:0/1152120360 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2f7c005500 con 0x7f2f90075fe0 2024-04-16T17:06:32.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:32.543+0000 7f2f98600640 1 -- 172.21.15.45:0/1152120360 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f901a2420 con 0x7f2f90075fe0 2024-04-16T17:06:32.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:32.543+0000 7f2f8e000640 1 -- 172.21.15.45:0/1152120360 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f7c010690 con 0x7f2f90075fe0 2024-04-16T17:06:32.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:32.543+0000 7f2f98600640 1 -- 172.21.15.45:0/1152120360 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f901a2970 con 0x7f2f90075fe0 2024-04-16T17:06:32.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:32.544+0000 7f2f98600640 1 -- 172.21.15.45:0/1152120360 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f90108ca0 con 0x7f2f90075fe0 2024-04-16T17:06:32.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:32.546+0000 7f2f8e000640 1 -- 172.21.15.45:0/1152120360 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f2f7c0046c0 con 0x7f2f90075fe0 2024-04-16T17:06:32.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:32.548+0000 7f2f8e000640 1 --2- 172.21.15.45:0/1152120360 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f2f780a3c30 0x7f2f780a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:32.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:32.548+0000 7f2f8e000640 1 -- 172.21.15.45:0/1152120360 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f2f7c0d0c20 con 0x7f2f90075fe0 2024-04-16T17:06:32.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:32.548+0000 7f2f95c00640 1 --2- 172.21.15.45:0/1152120360 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f2f780a3c30 0x7f2f780a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:32.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:32.549+0000 7f2f95c00640 1 --2- 172.21.15.45:0/1152120360 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f2f780a3c30 0x7f2f780a60f0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f2f84009bd0 tx=0x7f2f84005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:06:32.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:32.552+0000 7f2f8e000640 1 -- 172.21.15.45:0/1152120360 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2f7c09d210 con 0x7f2f90075fe0 2024-04-16T17:06:32.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:06:32 smithi104 ceph-mon[36457]: from='client.? 172.21.15.45:0/781425113' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-16T17:06:32.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:06:32 smithi104 ceph-mon[36457]: from='client.? 172.21.15.45:0/3250549147' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-16T17:06:32.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:06:32 smithi104 ceph-mon[36457]: from='client.? 172.21.15.45:0/2513977137' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-16T17:06:32.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:06:32 smithi045 ceph-mon[31676]: from='client.? 172.21.15.45:0/781425113' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-16T17:06:32.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:06:32 smithi045 ceph-mon[31676]: from='client.? 172.21.15.45:0/3250549147' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-16T17:06:32.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:06:32 smithi045 ceph-mon[31676]: from='client.? 172.21.15.45:0/2513977137' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-16T17:06:32.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:06:32 smithi117 ceph-mon[36730]: from='client.? 172.21.15.45:0/781425113' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-16T17:06:32.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:06:32 smithi117 ceph-mon[36730]: from='client.? 172.21.15.45:0/3250549147' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-16T17:06:32.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:06:32 smithi117 ceph-mon[36730]: from='client.? 172.21.15.45:0/2513977137' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-16T17:06:32.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:32.808+0000 7f2f98600640 1 -- 172.21.15.45:0/1152120360 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2f9002d620 con 0x7f2f90075fe0 2024-04-16T17:06:32.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:32.810+0000 7f2f8e000640 1 -- 172.21.15.45:0/1152120360 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11887 (secure 0 0 0) 0x7f2f7c09cb10 con 0x7f2f90075fe0 2024-04-16T17:06:32.811 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-16T17:06:32.811 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/1748842443","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":1748842443},{"type":"v1","addr":"172.21.15.45:6835","nonce":1748842443}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6839/1308627051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":1308627051},{"type":"v1","addr":"172.21.15.45:6839","nonce":1308627051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/1768242489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1768242489},{"type":"v1","addr":"172.21.15.117:6833","nonce":1768242489}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6833/3558902585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":3558902585},{"type":"v1","addr":"172.21.15.104:6833","nonce":3558902585}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24478,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/1308142903","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":1308142903},{"type":"v1","addr":"172.21.15.45:6837","nonce":1308142903}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24491,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6835/3430679937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":3430679937},{"type":"v1","addr":"172.21.15.104:6835","nonce":3430679937}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-16T17:03:24.663611+0000","modified":"2024-04-16T17:03:53.381209+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24581,"mds_1":24569,"mds_2":24532},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24499":{"gid":24499,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/2224123841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2224123841},{"type":"v1","addr":"172.21.15.117:6835","nonce":2224123841}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6837/2471356815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":2471356815},{"type":"v1","addr":"172.21.15.117:6837","nonce":2471356815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24524":{"gid":24524,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6837/2647756509","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":2647756509},{"type":"v1","addr":"172.21.15.104:6837","nonce":2647756509}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24532":{"gid":24532,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":13,"addr":"172.21.15.117:6839/2378675949","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":2378675949},{"type":"v1","addr":"172.21.15.117:6839","nonce":2378675949}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24569":{"gid":24569,"name":"a","rank":1,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.45:6841/3755310822","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6840","nonce":3755310822},{"type":"v1","addr":"172.21.15.45:6841","nonce":3755310822}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24581":{"gid":24581,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.104:6839/3660070592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6838","nonce":3660070592},{"type":"v1","addr":"172.21.15.104:6839","nonce":3660070592}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24581,"qdb_cluster":[24569,24532,24581]},"id":1}]} 2024-04-16T17:06:32.811 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 24 2024-04-16T17:06:32.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:32.813+0000 7f2f98600640 1 -- 172.21.15.45:0/1152120360 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f2f780a3c30 msgr2=0x7f2f780a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:32.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:32.813+0000 7f2f98600640 1 --2- 172.21.15.45:0/1152120360 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f2f780a3c30 0x7f2f780a60f0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f2f84009bd0 tx=0x7f2f84005f90 comp rx=0 tx=0).stop 2024-04-16T17:06:32.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:32.813+0000 7f2f98600640 1 -- 172.21.15.45:0/1152120360 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2f90075fe0 msgr2=0x7f2f9019aa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:32.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:32.813+0000 7f2f98600640 1 --2- 172.21.15.45:0/1152120360 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2f90075fe0 0x7f2f9019aa50 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f2f7c00e9f0 tx=0x7f2f7c00edf0 comp rx=0 tx=0).stop 2024-04-16T17:06:32.814 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:32.813+0000 7f2f98600640 1 -- 172.21.15.45:0/1152120360 shutdown_connections 2024-04-16T17:06:32.814 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:32.813+0000 7f2f98600640 1 --2- 172.21.15.45:0/1152120360 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f2f780a3c30 0x7f2f780a60f0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:32.814 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:32.813+0000 7f2f98600640 1 --2- 172.21.15.45:0/1152120360 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2f90109ee0 0x7f2f901a1ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:32.814 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:32.813+0000 7f2f98600640 1 --2- 172.21.15.45:0/1152120360 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2f90075fe0 0x7f2f9019aa50 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:32.814 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:32.813+0000 7f2f98600640 1 --2- 172.21.15.45:0/1152120360 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2f90074a00 0x7f2f9019a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:32.814 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:32.813+0000 7f2f98600640 1 -- 172.21.15.45:0/1152120360 >> 172.21.15.45:0/1152120360 conn(0x7f2f900fd600 msgr2=0x7f2f90105e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:06:32.814 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:32.814+0000 7f2f98600640 1 -- 172.21.15.45:0/1152120360 shutdown_connections 2024-04-16T17:06:32.814 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:32.814+0000 7f2f98600640 1 -- 172.21.15.45:0/1152120360 wait complete. 2024-04-16T17:06:32.834 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-16T17:06:33.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.061+0000 7f9c72400640 1 -- 172.21.15.45:0/565028 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9c6c108480 msgr2=0x7f9c6c1088a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:33.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.061+0000 7f9c72400640 1 --2- 172.21.15.45:0/565028 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9c6c108480 0x7f9c6c1088a0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f9c5c009900 tx=0x7f9c5c02bc50 comp rx=0 tx=0).stop 2024-04-16T17:06:33.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.061+0000 7f9c72400640 1 -- 172.21.15.45:0/565028 shutdown_connections 2024-04-16T17:06:33.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.061+0000 7f9c72400640 1 --2- 172.21.15.45:0/565028 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9c6c109670 0x7f9c6c109af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:33.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.061+0000 7f9c72400640 1 --2- 172.21.15.45:0/565028 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9c6c10a030 0x7f9c6c1108c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:33.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.061+0000 7f9c72400640 1 --2- 172.21.15.45:0/565028 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9c6c108480 0x7f9c6c1088a0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:33.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.061+0000 7f9c72400640 1 -- 172.21.15.45:0/565028 >> 172.21.15.45:0/565028 conn(0x7f9c6c077440 msgr2=0x7f9c6c079860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:06:33.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.062+0000 7f9c72400640 1 -- 172.21.15.45:0/565028 shutdown_connections 2024-04-16T17:06:33.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.062+0000 7f9c72400640 1 -- 172.21.15.45:0/565028 wait complete. 2024-04-16T17:06:33.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.063+0000 7f9c72400640 1 Processor -- start 2024-04-16T17:06:33.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.063+0000 7f9c72400640 1 -- start start 2024-04-16T17:06:33.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.063+0000 7f9c72400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9c6c108480 0x7f9c6c19e850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:33.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.063+0000 7f9c72400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9c6c109670 0x7f9c6c19ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:33.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.063+0000 7f9c72400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9c6c10a030 0x7f9c6c1a5e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:33.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.063+0000 7f9c72400640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f9c6c114110 con 0x7f9c6c108480 2024-04-16T17:06:33.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.063+0000 7f9c72400640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f9c6c113f90 con 0x7f9c6c109670 2024-04-16T17:06:33.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.063+0000 7f9c72400640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f9c6c114290 con 0x7f9c6c10a030 2024-04-16T17:06:33.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.063+0000 7f9c71000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9c6c108480 0x7f9c6c19e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:33.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.063+0000 7f9c71000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9c6c108480 0x7f9c6c19e850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55528/0 (socket says 172.21.15.45:55528) 2024-04-16T17:06:33.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.063+0000 7f9c71000640 1 -- 172.21.15.45:0/3453016261 learned_addr learned my addr 172.21.15.45:0/3453016261 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:06:33.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.063+0000 7f9c6be00640 1 --2- 172.21.15.45:0/3453016261 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9c6c109670 0x7f9c6c19ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:33.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.064+0000 7f9c71a00640 1 --2- 172.21.15.45:0/3453016261 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9c6c10a030 0x7f9c6c1a5e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:33.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.064+0000 7f9c71000640 1 -- 172.21.15.45:0/3453016261 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9c6c10a030 msgr2=0x7f9c6c1a5e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:33.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.064+0000 7f9c71000640 1 --2- 172.21.15.45:0/3453016261 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9c6c10a030 0x7f9c6c1a5e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:33.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.064+0000 7f9c71000640 1 -- 172.21.15.45:0/3453016261 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9c6c109670 msgr2=0x7f9c6c19ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:33.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.064+0000 7f9c71000640 1 --2- 172.21.15.45:0/3453016261 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9c6c109670 0x7f9c6c19ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:33.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.064+0000 7f9c71000640 1 -- 172.21.15.45:0/3453016261 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c6c1a6470 con 0x7f9c6c108480 2024-04-16T17:06:33.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.064+0000 7f9c71000640 1 --2- 172.21.15.45:0/3453016261 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9c6c108480 0x7f9c6c19e850 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f9c5c000c00 tx=0x7f9c5c02c660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:06:33.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.065+0000 7f9c69600640 1 -- 172.21.15.45:0/3453016261 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c5c042020 con 0x7f9c6c108480 2024-04-16T17:06:33.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.065+0000 7f9c69600640 1 -- 172.21.15.45:0/3453016261 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9c5c004450 con 0x7f9c6c108480 2024-04-16T17:06:33.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.065+0000 7f9c69600640 1 -- 172.21.15.45:0/3453016261 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c5c003df0 con 0x7f9c6c108480 2024-04-16T17:06:33.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.065+0000 7f9c72400640 1 -- 172.21.15.45:0/3453016261 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c6c1a6700 con 0x7f9c6c108480 2024-04-16T17:06:33.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.065+0000 7f9c72400640 1 -- 172.21.15.45:0/3453016261 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c6c1a6c50 con 0x7f9c6c108480 2024-04-16T17:06:33.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.066+0000 7f9c72400640 1 -- 172.21.15.45:0/3453016261 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c6c10f7a0 con 0x7f9c6c108480 2024-04-16T17:06:33.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.068+0000 7f9c69600640 1 -- 172.21.15.45:0/3453016261 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f9c5c045020 con 0x7f9c6c108480 2024-04-16T17:06:33.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.069+0000 7f9c69600640 1 --2- 172.21.15.45:0/3453016261 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f9c540a3bb0 0x7f9c540a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:33.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.070+0000 7f9c69600640 1 -- 172.21.15.45:0/3453016261 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f9c5c0f2730 con 0x7f9c6c108480 2024-04-16T17:06:33.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.070+0000 7f9c6be00640 1 --2- 172.21.15.45:0/3453016261 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f9c540a3bb0 0x7f9c540a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:33.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.070+0000 7f9c6be00640 1 --2- 172.21.15.45:0/3453016261 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f9c540a3bb0 0x7f9c540a6070 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f9c6c19fd60 tx=0x7f9c60005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:06:33.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.074+0000 7f9c69600640 1 -- 172.21.15.45:0/3453016261 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9c5c04c020 con 0x7f9c6c108480 2024-04-16T17:06:33.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.289+0000 7f9c72400640 1 -- 172.21.15.45:0/3453016261 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9c6c062c00 con 0x7f9c6c108480 2024-04-16T17:06:33.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.292+0000 7f9c69600640 1 -- 172.21.15.45:0/3453016261 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) ==== 74+0+24726 (secure 0 0 0) 0x7f9c5c0bedb0 con 0x7f9c6c108480 2024-04-16T17:06:33.292 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-16T17:06:33.293 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":94,"fsid":"b1fecd38-fc11-11ee-bc90-c7b262605968","created":"2024-04-16T16:54:02.385906+0000","modified":"2024-04-16T17:05:22.272335+0000","last_up_change":"2024-04-16T17:02:25.162624+0000","last_in_change":"2024-04-16T17:02:04.903274+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-16T16:58:06.166659+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-16T17:03:22.304198+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-16T17:03:23.616450+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-16T17:03:27.226289+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b655de2c-59f8-4fd4-ac98-0cc222229f56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":2290170881},{"type":"v1","addr":"172.21.15.45:6803","nonce":2290170881}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":2290170881},{"type":"v1","addr":"172.21.15.45:6805","nonce":2290170881}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":2290170881},{"type":"v1","addr":"172.21.15.45:6809","nonce":2290170881}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":2290170881},{"type":"v1","addr":"172.21.15.45:6807","nonce":2290170881}]},"public_addr":"172.21.15.45:6803/2290170881","cluster_addr":"172.21.15.45:6805/2290170881","heartbeat_back_addr":"172.21.15.45:6809/2290170881","heartbeat_front_addr":"172.21.15.45:6807/2290170881","state":["exists","up"]},{"osd":1,"uuid":"ebd450ec-08fd-4c3d-8b24-4e10634ddb7e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6800","nonce":1462942067},{"type":"v1","addr":"172.21.15.104:6801","nonce":1462942067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":1462942067},{"type":"v1","addr":"172.21.15.104:6803","nonce":1462942067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":1462942067},{"type":"v1","addr":"172.21.15.104:6807","nonce":1462942067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":1462942067},{"type":"v1","addr":"172.21.15.104:6805","nonce":1462942067}]},"public_addr":"172.21.15.104:6801/1462942067","cluster_addr":"172.21.15.104:6803/1462942067","heartbeat_back_addr":"172.21.15.104:6807/1462942067","heartbeat_front_addr":"172.21.15.104:6805/1462942067","state":["exists","up"]},{"osd":2,"uuid":"b318a512-4190-42a2-8104-dcc4026ad8ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":2829260735},{"type":"v1","addr":"172.21.15.117:6801","nonce":2829260735}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":2829260735},{"type":"v1","addr":"172.21.15.117:6803","nonce":2829260735}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":2829260735},{"type":"v1","addr":"172.21.15.117:6807","nonce":2829260735}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":2829260735},{"type":"v1","addr":"172.21.15.117:6805","nonce":2829260735}]},"public_addr":"172.21.15.117:6801/2829260735","cluster_addr":"172.21.15.117:6803/2829260735","heartbeat_back_addr":"172.21.15.117:6807/2829260735","heartbeat_front_addr":"172.21.15.117:6805/2829260735","state":["exists","up"]},{"osd":3,"uuid":"ec6490cb-a715-4c12-a54a-225aae26fcb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":2823712137},{"type":"v1","addr":"172.21.15.45:6811","nonce":2823712137}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":2823712137},{"type":"v1","addr":"172.21.15.45:6813","nonce":2823712137}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":2823712137},{"type":"v1","addr":"172.21.15.45:6817","nonce":2823712137}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":2823712137},{"type":"v1","addr":"172.21.15.45:6815","nonce":2823712137}]},"public_addr":"172.21.15.45:6811/2823712137","cluster_addr":"172.21.15.45:6813/2823712137","heartbeat_back_addr":"172.21.15.45:6817/2823712137","heartbeat_front_addr":"172.21.15.45:6815/2823712137","state":["exists","up"]},{"osd":4,"uuid":"5d91d568-0250-4aae-8dc2-a0e079f92062","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":1396278127},{"type":"v1","addr":"172.21.15.104:6809","nonce":1396278127}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":1396278127},{"type":"v1","addr":"172.21.15.104:6811","nonce":1396278127}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":1396278127},{"type":"v1","addr":"172.21.15.104:6815","nonce":1396278127}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":1396278127},{"type":"v1","addr":"172.21.15.104:6813","nonce":1396278127}]},"public_addr":"172.21.15.104:6809/1396278127","cluster_addr":"172.21.15.104:6811/1396278127","heartbeat_back_addr":"172.21.15.104:6815/1396278127","heartbeat_front_addr":"172.21.15.104:6813/1396278127","state":["exists","up"]},{"osd":5,"uuid":"ac510b43-6fb2-4d0f-bc5f-16ac080d9e9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":4145720047},{"type":"v1","addr":"172.21.15.117:6809","nonce":4145720047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":4145720047},{"type":"v1","addr":"172.21.15.117:6811","nonce":4145720047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":4145720047},{"type":"v1","addr":"172.21.15.117:6815","nonce":4145720047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":4145720047},{"type":"v1","addr":"172.21.15.117:6813","nonce":4145720047}]},"public_addr":"172.21.15.117:6809/4145720047","cluster_addr":"172.21.15.117:6811/4145720047","heartbeat_back_addr":"172.21.15.117:6815/4145720047","heartbeat_front_addr":"172.21.15.117:6813/4145720047","state":["exists","up"]},{"osd":6,"uuid":"a41adc18-d88a-419e-b2c2-b089c1f32472","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":3254229176},{"type":"v1","addr":"172.21.15.45:6819","nonce":3254229176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":3254229176},{"type":"v1","addr":"172.21.15.45:6821","nonce":3254229176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":3254229176},{"type":"v1","addr":"172.21.15.45:6825","nonce":3254229176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":3254229176},{"type":"v1","addr":"172.21.15.45:6823","nonce":3254229176}]},"public_addr":"172.21.15.45:6819/3254229176","cluster_addr":"172.21.15.45:6821/3254229176","heartbeat_back_addr":"172.21.15.45:6825/3254229176","heartbeat_front_addr":"172.21.15.45:6823/3254229176","state":["exists","up"]},{"osd":7,"uuid":"b71712a4-7cf6-454d-9fd1-565e3df21c1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":4151077186},{"type":"v1","addr":"172.21.15.104:6817","nonce":4151077186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":4151077186},{"type":"v1","addr":"172.21.15.104:6819","nonce":4151077186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":4151077186},{"type":"v1","addr":"172.21.15.104:6823","nonce":4151077186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":4151077186},{"type":"v1","addr":"172.21.15.104:6821","nonce":4151077186}]},"public_addr":"172.21.15.104:6817/4151077186","cluster_addr":"172.21.15.104:6819/4151077186","heartbeat_back_addr":"172.21.15.104:6823/4151077186","heartbeat_front_addr":"172.21.15.104:6821/4151077186","state":["exists","up"]},{"osd":8,"uuid":"9e572aef-59a8-450e-a92e-df9dff4d57a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":1500286830},{"type":"v1","addr":"172.21.15.117:6817","nonce":1500286830}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":1500286830},{"type":"v1","addr":"172.21.15.117:6819","nonce":1500286830}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":1500286830},{"type":"v1","addr":"172.21.15.117:6823","nonce":1500286830}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":1500286830},{"type":"v1","addr":"172.21.15.117:6821","nonce":1500286830}]},"public_addr":"172.21.15.117:6817/1500286830","cluster_addr":"172.21.15.117:6819/1500286830","heartbeat_back_addr":"172.21.15.117:6823/1500286830","heartbeat_front_addr":"172.21.15.117:6821/1500286830","state":["exists","up"]},{"osd":9,"uuid":"014e9598-c413-4cc4-9bb8-61c6e8f59d30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":3106734074},{"type":"v1","addr":"172.21.15.45:6827","nonce":3106734074}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":3106734074},{"type":"v1","addr":"172.21.15.45:6829","nonce":3106734074}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":3106734074},{"type":"v1","addr":"172.21.15.45:6833","nonce":3106734074}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":3106734074},{"type":"v1","addr":"172.21.15.45:6831","nonce":3106734074}]},"public_addr":"172.21.15.45:6827/3106734074","cluster_addr":"172.21.15.45:6829/3106734074","heartbeat_back_addr":"172.21.15.45:6833/3106734074","heartbeat_front_addr":"172.21.15.45:6831/3106734074","state":["exists","up"]},{"osd":10,"uuid":"2deba364-3e39-47be-8a1f-8256df5b82d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":1404794652},{"type":"v1","addr":"172.21.15.104:6825","nonce":1404794652}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":1404794652},{"type":"v1","addr":"172.21.15.104:6827","nonce":1404794652}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":1404794652},{"type":"v1","addr":"172.21.15.104:6831","nonce":1404794652}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":1404794652},{"type":"v1","addr":"172.21.15.104:6829","nonce":1404794652}]},"public_addr":"172.21.15.104:6825/1404794652","cluster_addr":"172.21.15.104:6827/1404794652","heartbeat_back_addr":"172.21.15.104:6831/1404794652","heartbeat_front_addr":"172.21.15.104:6829/1404794652","state":["exists","up"]},{"osd":11,"uuid":"3fda4fea-59ad-4d1a-9fe6-fcb30e31fd7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":3384003243},{"type":"v1","addr":"172.21.15.117:6825","nonce":3384003243}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":3384003243},{"type":"v1","addr":"172.21.15.117:6827","nonce":3384003243}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":3384003243},{"type":"v1","addr":"172.21.15.117:6831","nonce":3384003243}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":3384003243},{"type":"v1","addr":"172.21.15.117:6829","nonce":3384003243}]},"public_addr":"172.21.15.117:6825/3384003243","cluster_addr":"172.21.15.117:6827/3384003243","heartbeat_back_addr":"172.21.15.117:6831/3384003243","heartbeat_front_addr":"172.21.15.117:6829/3384003243","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:57:14.799608+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:57:39.884668+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:58:03.327057+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:58:37.314531+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:59:03.667650+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:59:28.462823+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T17:00:02.968574+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T17:00:29.559190+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T17:00:55.291007+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T17:01:29.815977+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T17:01:57.015580+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T17:02:23.233151+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.5","mappings":[{"from":9,"to":10}]},{"pgid":"4.7","mappings":[{"from":5,"to":8}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.45:0/4074982598":"2024-04-17T16:55:19.948001+0000","172.21.15.45:0/2443572576":"2024-04-17T16:55:19.948001+0000","172.21.15.45:6801/1983630136":"2024-04-17T16:55:19.948001+0000","172.21.15.45:6800/1983630136":"2024-04-17T16:55:19.948001+0000","172.21.15.45:0/3150465248":"2024-04-17T16:54:43.692466+0000","172.21.15.45:0/556738623":"2024-04-17T16:54:43.692466+0000","172.21.15.45:0/445676297":"2024-04-17T16:55:19.948001+0000","172.21.15.45:6800/2553659622":"2024-04-17T16:54:43.692466+0000","172.21.15.45:6801/2553659622":"2024-04-17T16:54:43.692466+0000","172.21.15.45:0/1184615408":"2024-04-17T16:54:43.692466+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-16T17:06:33.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.295+0000 7f9c72400640 1 -- 172.21.15.45:0/3453016261 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f9c540a3bb0 msgr2=0x7f9c540a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:33.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.295+0000 7f9c72400640 1 --2- 172.21.15.45:0/3453016261 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f9c540a3bb0 0x7f9c540a6070 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f9c6c19fd60 tx=0x7f9c60005f90 comp rx=0 tx=0).stop 2024-04-16T17:06:33.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.295+0000 7f9c72400640 1 -- 172.21.15.45:0/3453016261 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9c6c108480 msgr2=0x7f9c6c19e850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:33.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.295+0000 7f9c72400640 1 --2- 172.21.15.45:0/3453016261 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9c6c108480 0x7f9c6c19e850 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f9c5c000c00 tx=0x7f9c5c02c660 comp rx=0 tx=0).stop 2024-04-16T17:06:33.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.295+0000 7f9c72400640 1 -- 172.21.15.45:0/3453016261 shutdown_connections 2024-04-16T17:06:33.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.295+0000 7f9c72400640 1 --2- 172.21.15.45:0/3453016261 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f9c540a3bb0 0x7f9c540a6070 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:33.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.295+0000 7f9c72400640 1 --2- 172.21.15.45:0/3453016261 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9c6c10a030 0x7f9c6c1a5e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:33.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.295+0000 7f9c72400640 1 --2- 172.21.15.45:0/3453016261 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9c6c109670 0x7f9c6c19ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:33.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.295+0000 7f9c72400640 1 --2- 172.21.15.45:0/3453016261 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9c6c108480 0x7f9c6c19e850 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:33.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.295+0000 7f9c72400640 1 -- 172.21.15.45:0/3453016261 >> 172.21.15.45:0/3453016261 conn(0x7f9c6c077440 msgr2=0x7f9c6c10c970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:06:33.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.295+0000 7f9c72400640 1 -- 172.21.15.45:0/3453016261 shutdown_connections 2024-04-16T17:06:33.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.295+0000 7f9c72400640 1 -- 172.21.15.45:0/3453016261 wait complete. 2024-04-16T17:06:33.317 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 24, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-16T17:03:24.663611+0000', 'modified': '2024-04-16T17:03:53.381209+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24581, 'mds_1': 24569, 'mds_2': 24532}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24499': {'gid': 24499, 'name': 'f', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.117:6835/2224123841', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6834', 'nonce': 2224123841}, {'type': 'v1', 'addr': '172.21.15.117:6835', 'nonce': 2224123841}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24511': {'gid': 24511, 'name': 'c', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.117:6837/2471356815', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6836', 'nonce': 2471356815}, {'type': 'v1', 'addr': '172.21.15.117:6837', 'nonce': 2471356815}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24524': {'gid': 24524, 'name': 'h', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.104:6837/2647756509', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.104:6836', 'nonce': 2647756509}, {'type': 'v1', 'addr': '172.21.15.104:6837', 'nonce': 2647756509}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24532': {'gid': 24532, 'name': 'i', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 13, 'addr': '172.21.15.117:6839/2378675949', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6838', 'nonce': 2378675949}, {'type': 'v1', 'addr': '172.21.15.117:6839', 'nonce': 2378675949}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24569': {'gid': 24569, 'name': 'a', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 10, 'addr': '172.21.15.45:6841/3755310822', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.45:6840', 'nonce': 3755310822}, {'type': 'v1', 'addr': '172.21.15.45:6841', 'nonce': 3755310822}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24581': {'gid': 24581, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.104:6839/3660070592', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.104:6838', 'nonce': 3660070592}, {'type': 'v1', 'addr': '172.21.15.104:6839', 'nonce': 3660070592}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24581, 'qdb_cluster': [24569, 24532, 24581]} 2024-04-16T17:06:33.317 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-04-16T17:06:33.317 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-04-16T17:06:33.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.537+0000 7f1bda800640 1 -- 172.21.15.45:0/3663349487 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1bd4076040 msgr2=0x7f1bd4074490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:33.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.537+0000 7f1bda800640 1 --2- 172.21.15.45:0/3663349487 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1bd4076040 0x7f1bd4074490 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f1bc8009c10 tx=0x7f1bc802bee0 comp rx=0 tx=0).stop 2024-04-16T17:06:33.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.537+0000 7f1bda800640 1 -- 172.21.15.45:0/3663349487 shutdown_connections 2024-04-16T17:06:33.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.537+0000 7f1bda800640 1 --2- 172.21.15.45:0/3663349487 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1bd410e4e0 0x7f1bd41108d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:33.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.537+0000 7f1bda800640 1 --2- 172.21.15.45:0/3663349487 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1bd4074a60 0x7f1bd4074ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:33.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.537+0000 7f1bda800640 1 --2- 172.21.15.45:0/3663349487 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1bd4076040 0x7f1bd4074490 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:33.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.537+0000 7f1bda800640 1 -- 172.21.15.45:0/3663349487 >> 172.21.15.45:0/3663349487 conn(0x7f1bd40fd4a0 msgr2=0x7f1bd40ff8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:06:33.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.538+0000 7f1bda800640 1 -- 172.21.15.45:0/3663349487 shutdown_connections 2024-04-16T17:06:33.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.538+0000 7f1bda800640 1 -- 172.21.15.45:0/3663349487 wait complete. 2024-04-16T17:06:33.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.539+0000 7f1bda800640 1 Processor -- start 2024-04-16T17:06:33.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.539+0000 7f1bda800640 1 -- start start 2024-04-16T17:06:33.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.539+0000 7f1bda800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1bd4074a60 0x7f1bd419e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:33.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.539+0000 7f1bda800640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1bd4076040 0x7f1bd419ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:33.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.539+0000 7f1bda800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1bd410e4e0 0x7f1bd41a5df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:33.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.539+0000 7f1bda800640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f1bd41141f0 con 0x7f1bd4074a60 2024-04-16T17:06:33.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.539+0000 7f1bda800640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f1bd4114070 con 0x7f1bd4076040 2024-04-16T17:06:33.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.539+0000 7f1bda800640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f1bd4114370 con 0x7f1bd410e4e0 2024-04-16T17:06:33.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.540+0000 7f1bd9400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1bd4074a60 0x7f1bd419e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:33.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.540+0000 7f1bd8a00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1bd4076040 0x7f1bd419ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:33.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.540+0000 7f1bd9400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1bd4074a60 0x7f1bd419e840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55550/0 (socket says 172.21.15.45:55550) 2024-04-16T17:06:33.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.540+0000 7f1bd9400640 1 -- 172.21.15.45:0/4136985619 learned_addr learned my addr 172.21.15.45:0/4136985619 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:06:33.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.540+0000 7f1bd9e00640 1 --2- 172.21.15.45:0/4136985619 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1bd410e4e0 0x7f1bd41a5df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:33.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.540+0000 7f1bd8a00640 1 -- 172.21.15.45:0/4136985619 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1bd410e4e0 msgr2=0x7f1bd41a5df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:33.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.540+0000 7f1bd8a00640 1 --2- 172.21.15.45:0/4136985619 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1bd410e4e0 0x7f1bd41a5df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:33.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.540+0000 7f1bd8a00640 1 -- 172.21.15.45:0/4136985619 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1bd4074a60 msgr2=0x7f1bd419e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:33.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.540+0000 7f1bd8a00640 1 --2- 172.21.15.45:0/4136985619 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1bd4074a60 0x7f1bd419e840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:33.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.540+0000 7f1bd8a00640 1 -- 172.21.15.45:0/4136985619 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1bd41a6460 con 0x7f1bd4076040 2024-04-16T17:06:33.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.541+0000 7f1bd8a00640 1 --2- 172.21.15.45:0/4136985619 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1bd4076040 0x7f1bd419ed80 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f1bc0005fd0 tx=0x7f1bc000ee50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:06:33.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.541+0000 7f1bd2000640 1 -- 172.21.15.45:0/4136985619 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bc000cd00 con 0x7f1bd4076040 2024-04-16T17:06:33.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.541+0000 7f1bda800640 1 -- 172.21.15.45:0/4136985619 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1bd41a6750 con 0x7f1bd4076040 2024-04-16T17:06:33.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.541+0000 7f1bd2000640 1 -- 172.21.15.45:0/4136985619 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1bc00035f0 con 0x7f1bd4076040 2024-04-16T17:06:33.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.541+0000 7f1bd2000640 1 -- 172.21.15.45:0/4136985619 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bc0010690 con 0x7f1bd4076040 2024-04-16T17:06:33.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.541+0000 7f1bda800640 1 -- 172.21.15.45:0/4136985619 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1bd41a6ca0 con 0x7f1bd4076040 2024-04-16T17:06:33.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.545+0000 7f1bdeb7f740 1 Processor -- start 2024-04-16T17:06:33.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.546+0000 7f1bdeb7f740 1 -- start start 2024-04-16T17:06:33.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.546+0000 7f1bdeb7f740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558e5e2a04a0 0x558e5e2a08c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:33.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.546+0000 7f1bdeb7f740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558e5e2a0e90 0x558e5e2a5990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:33.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.546+0000 7f1bdeb7f740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558e5e2a6000 0x558e5e2a8490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:33.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.546+0000 7f1bdeb7f740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x558e5e206980 con 0x558e5e2a6000 2024-04-16T17:06:33.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.546+0000 7f1bdeb7f740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x558e5e1ad220 con 0x558e5e2a0e90 2024-04-16T17:06:33.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.546+0000 7f1bdeb7f740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x558e5e202940 con 0x558e5e2a04a0 2024-04-16T17:06:33.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.546+0000 7f1bd2000640 1 -- 172.21.15.45:0/4136985619 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f1bc00037a0 con 0x7f1bd4076040 2024-04-16T17:06:33.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.546+0000 7f1bd9400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558e5e2a0e90 0x558e5e2a5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:33.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.546+0000 7f1bd9e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558e5e2a04a0 0x558e5e2a08c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:33.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.546+0000 7f1bd9400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558e5e2a0e90 0x558e5e2a5990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:48746/0 (socket says 172.21.15.45:48746) 2024-04-16T17:06:33.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.546+0000 7f1bd9400640 1 -- 172.21.15.45:0/1182547320 learned_addr learned my addr 172.21.15.45:0/1182547320 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:06:33.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.547+0000 7f1bd8a00640 1 --2- 172.21.15.45:0/1182547320 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558e5e2a6000 0x558e5e2a8490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:33.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.547+0000 7f1bd9e00640 1 -- 172.21.15.45:0/1182547320 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558e5e2a0e90 msgr2=0x558e5e2a5990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:06:33.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.547+0000 7f1bd9e00640 1 --2- 172.21.15.45:0/1182547320 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558e5e2a0e90 0x558e5e2a5990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:33.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.547+0000 7f1bd9e00640 1 -- 172.21.15.45:0/1182547320 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558e5e2a6000 msgr2=0x558e5e2a8490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:06:33.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.547+0000 7f1bd9e00640 1 --2- 172.21.15.45:0/1182547320 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558e5e2a6000 0x558e5e2a8490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:33.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.547+0000 7f1bd9e00640 1 -- 172.21.15.45:0/1182547320 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558e5df192d0 con 0x558e5e2a04a0 2024-04-16T17:06:33.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.547+0000 7f1bd9e00640 1 --2- 172.21.15.45:0/1182547320 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558e5e2a04a0 0x558e5e2a08c0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f1bcc009b80 tx=0x7f1bcc011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=58914a9c732f082f server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:06:33.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.547+0000 7f1bc6600640 1 -- 172.21.15.45:0/1182547320 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bcc004030 con 0x558e5e2a04a0 2024-04-16T17:06:33.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.548+0000 7f1bc6600640 1 -- 172.21.15.45:0/1182547320 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1bcc005500 con 0x558e5e2a04a0 2024-04-16T17:06:33.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.548+0000 7f1bc6600640 1 -- 172.21.15.45:0/1182547320 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bcc004dd0 con 0x558e5e2a04a0 2024-04-16T17:06:33.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.548+0000 7f1bd2000640 1 --2- 172.21.15.45:0/4136985619 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f1bbc0a3bb0 0x7f1bbc0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:33.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.548+0000 7f1bdeb7f740 1 -- 172.21.15.45:0/1182547320 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558e5e2a04a0 msgr2=0x558e5e2a08c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:33.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.548+0000 7f1bdeb7f740 1 --2- 172.21.15.45:0/1182547320 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558e5e2a04a0 0x558e5e2a08c0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f1bcc009b80 tx=0x7f1bcc011810 comp rx=0 tx=0).stop 2024-04-16T17:06:33.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.548+0000 7f1bd2000640 1 -- 172.21.15.45:0/4136985619 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f1bc00d09a0 con 0x7f1bd4076040 2024-04-16T17:06:33.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.548+0000 7f1bdeb7f740 1 -- 172.21.15.45:0/1182547320 shutdown_connections 2024-04-16T17:06:33.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.548+0000 7f1bdeb7f740 1 --2- 172.21.15.45:0/1182547320 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558e5e2a6000 0x558e5e2a8490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:33.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.548+0000 7f1bdeb7f740 1 --2- 172.21.15.45:0/1182547320 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558e5e2a0e90 0x558e5e2a5990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:33.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.548+0000 7f1bd9400640 1 --2- 172.21.15.45:0/4136985619 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f1bbc0a3bb0 0x7f1bbc0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:33.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.548+0000 7f1bdeb7f740 1 --2- 172.21.15.45:0/1182547320 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558e5e2a04a0 0x558e5e2a08c0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:33.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.548+0000 7f1bdeb7f740 1 -- 172.21.15.45:0/1182547320 >> 172.21.15.45:0/1182547320 conn(0x558e5e29ba10 msgr2=0x558e5e29be10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:06:33.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.548+0000 7f1bdeb7f740 1 -- 172.21.15.45:0/1182547320 shutdown_connections 2024-04-16T17:06:33.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.548+0000 7f1bdeb7f740 1 -- 172.21.15.45:0/1182547320 wait complete. 2024-04-16T17:06:33.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.549+0000 7f1bdeb7f740 1 -- start start 2024-04-16T17:06:33.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.549+0000 7f1bdeb7f740 1 Processor -- start 2024-04-16T17:06:33.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.549+0000 7f1bd9400640 1 --2- 172.21.15.45:0/4136985619 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f1bbc0a3bb0 0x7f1bbc0a6070 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f1bc8009be0 tx=0x7f1bc8004a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:06:33.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.550+0000 7f1bdeb7f740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558e5e2a04a0 0x558e5e2aaa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:33.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.550+0000 7f1bdeb7f740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558e5e2a0e90 0x558e5e2aaf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:33.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.550+0000 7f1bdeb7f740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558e5e2a6000 0x558e5e2b1f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:33.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.550+0000 7f1bdeb7f740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x558e5e2a8fb0 con 0x558e5e2a0e90 2024-04-16T17:06:33.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.550+0000 7f1bdeb7f740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x558e5e1c0590 con 0x558e5e2a6000 2024-04-16T17:06:33.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.550+0000 7f1bdeb7f740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x558e5e2a9130 con 0x558e5e2a04a0 2024-04-16T17:06:33.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.550+0000 7f1bdeb7f740 10 client.0 _ll_register_callbacks cb 0x558e5e1c8050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:06:33.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.550+0000 7f1bd9400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558e5e2a04a0 0x558e5e2aaa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:33.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.550+0000 7f1bd9400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558e5e2a04a0 0x558e5e2aaa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:39388/0 (socket says 172.21.15.45:39388) 2024-04-16T17:06:33.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.550+0000 7f1bd9400640 1 -- 172.21.15.45:0/3162507029 learned_addr learned my addr 172.21.15.45:0/3162507029 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:06:33.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.550+0000 7f1bd8a00640 1 --2- 172.21.15.45:0/3162507029 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558e5e2a0e90 0x558e5e2aaf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:33.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.550+0000 7f1bd9e00640 1 --2- 172.21.15.45:0/3162507029 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558e5e2a6000 0x558e5e2b1f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:33.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.550+0000 7f1bd9400640 1 -- 172.21.15.45:0/3162507029 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558e5e2a6000 msgr2=0x558e5e2b1f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:06:33.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.550+0000 7f1bd9400640 1 --2- 172.21.15.45:0/3162507029 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558e5e2a6000 0x558e5e2b1f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:33.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.550+0000 7f1bd9400640 1 -- 172.21.15.45:0/3162507029 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558e5e2a0e90 msgr2=0x558e5e2aaf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:06:33.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.550+0000 7f1bd9400640 1 --2- 172.21.15.45:0/3162507029 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558e5e2a0e90 0x558e5e2aaf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:33.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.550+0000 7f1bd9400640 1 -- 172.21.15.45:0/3162507029 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558e5dfe36b0 con 0x558e5e2a04a0 2024-04-16T17:06:33.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.550+0000 7f1bd9e00640 1 --2- 172.21.15.45:0/3162507029 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558e5e2a6000 0x558e5e2b1f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:06:33.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.551+0000 7f1bd9400640 1 --2- 172.21.15.45:0/3162507029 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558e5e2a04a0 0x558e5e2aaa10 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f1bc8036040 tx=0x7f1bc8009500 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9f66a8b41c0390ba server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:06:33.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.551+0000 7f1bbaa00640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-04-16T17:06:33.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.551+0000 7f1bbaa00640 1 -- 172.21.15.45:0/3162507029 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bc8051070 con 0x558e5e2a04a0 2024-04-16T17:06:33.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.551+0000 7f1bbaa00640 1 -- 172.21.15.45:0/3162507029 <== mon.1 v2:172.21.15.117:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f1bc8003f80 con 0x558e5e2a04a0 2024-04-16T17:06:33.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.551+0000 7f1bdeb7f740 1 -- 172.21.15.45:0/3162507029 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1bcc005500 con 0x558e5e2a04a0 2024-04-16T17:06:33.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.552+0000 7f1bbaa00640 1 -- 172.21.15.45:0/3162507029 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f1bc805d050 con 0x558e5e2a04a0 2024-04-16T17:06:33.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.552+0000 7f1bdeb7f740 10 client.25348 fetch_fsmap learned FSMap version 24 2024-04-16T17:06:33.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.552+0000 7f1bdeb7f740 1 -- 172.21.15.45:0/3162507029 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x558e5e2b5370 con 0x558e5e2a04a0 2024-04-16T17:06:33.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.552+0000 7f1bbaa00640 1 -- 172.21.15.45:0/3162507029 <== mon.1 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1bcc005500 con 0x558e5e2a04a0 2024-04-16T17:06:33.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.552+0000 7f1bbaa00640 1 -- 172.21.15.45:0/3162507029 <== mon.1 v2:172.21.15.117:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f1bc8034890 con 0x558e5e2a04a0 2024-04-16T17:06:33.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.552+0000 7f1bdeb7f740 10 client.25348 fetch_fsmap finished waiting for FSMap version 24 2024-04-16T17:06:33.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.552+0000 7f1bdeb7f740 10 client.25348 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-16T17:06:33.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.552+0000 7f1bdeb7f740 20 client.25348 populate_metadata read hostname 'smithi045' 2024-04-16T17:06:33.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.552+0000 7f1bdeb7f740 1 --2- 172.21.15.45:0/3162507029 >> [v2:172.21.15.117:6838/2378675949,v1:172.21.15.117:6839/2378675949] conn(0x558e5e2b79f0 0x558e5e2b8e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:33.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.552+0000 7f1bdeb7f740 4 client.25348 mds_command: new command op to 24532 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:06:33.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.552+0000 7f1bdeb7f740 1 -- 172.21.15.45:0/3162507029 --> [v2:172.21.15.117:6838/2378675949,v1:172.21.15.117:6839/2378675949] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558e5e2143a0 con 0x558e5e2b79f0 2024-04-16T17:06:33.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.553+0000 7f1bd8a00640 1 --2- 172.21.15.45:0/3162507029 >> [v2:172.21.15.117:6838/2378675949,v1:172.21.15.117:6839/2378675949] conn(0x558e5e2b79f0 0x558e5e2b8e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:33.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.554+0000 7f1bd8a00640 1 --2- 172.21.15.45:0/3162507029 >> [v2:172.21.15.117:6838/2378675949,v1:172.21.15.117:6839/2378675949] conn(0x558e5e2b79f0 0x558e5e2b8e60 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=dbd6f5010b85c747 server_cookie=65cdb3d2b07c6304 in_seq=0 out_seq=0 2024-04-16T17:06:33.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.554+0000 7f1bbaa00640 10 client.25348 ms_handle_connect on v2:172.21.15.117:6838/2378675949 2024-04-16T17:06:33.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.554+0000 7f1bbaa00640 1 -- 172.21.15.45:0/3162507029 <== mds.2 v2:172.21.15.117:6838/2378675949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558e5e2143a0 con 0x558e5e2b79f0 2024-04-16T17:06:33.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.554+0000 7f1bbaa00640 10 client.25348 handle_command_reply: tid=0 2024-04-16T17:06:33.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.555+0000 7f1bdeb7f740 1 client.25348 shutdown 2024-04-16T17:06:33.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.555+0000 7f1bdeb7f740 1 -- 172.21.15.45:0/3162507029 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558e5e2a04a0 msgr2=0x558e5e2aaa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:33.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.555+0000 7f1bdeb7f740 1 --2- 172.21.15.45:0/3162507029 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558e5e2a04a0 0x558e5e2aaa10 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f1bc8036040 tx=0x7f1bc8009500 comp rx=0 tx=0).stop 2024-04-16T17:06:33.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.555+0000 7f1bdeb7f740 1 -- 172.21.15.45:0/3162507029 shutdown_connections 2024-04-16T17:06:33.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.555+0000 7f1bdeb7f740 1 --2- 172.21.15.45:0/3162507029 >> [v2:172.21.15.117:6838/2378675949,v1:172.21.15.117:6839/2378675949] conn(0x558e5e2b79f0 0x558e5e2b8e60 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:33.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.555+0000 7f1bdeb7f740 1 --2- 172.21.15.45:0/3162507029 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558e5e2a6000 0x558e5e2b1f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:33.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.555+0000 7f1bdeb7f740 1 --2- 172.21.15.45:0/3162507029 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558e5e2a0e90 0x558e5e2aaf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:33.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.555+0000 7f1bdeb7f740 1 --2- 172.21.15.45:0/3162507029 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558e5e2a04a0 0x558e5e2aaa10 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:33.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.555+0000 7f1bbaa00640 0 client.25348 ms_handle_reset on v2:172.21.15.117:6838/2378675949 2024-04-16T17:06:33.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.555+0000 7f1bdeb7f740 1 -- 172.21.15.45:0/3162507029 >> 172.21.15.45:0/3162507029 conn(0x558e5e29ba10 msgr2=0x558e5e29ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:06:33.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.555+0000 7f1bdeb7f740 1 -- 172.21.15.45:0/3162507029 shutdown_connections 2024-04-16T17:06:33.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.555+0000 7f1bdeb7f740 1 -- 172.21.15.45:0/3162507029 wait complete. 2024-04-16T17:06:33.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.555+0000 7f1bdeb7f740 20 client.25348 trim_cache size 0 max 16384 2024-04-16T17:06:33.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.575+0000 7f1bdeb7f740 1 Processor -- start 2024-04-16T17:06:33.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.575+0000 7f1bdeb7f740 1 -- start start 2024-04-16T17:06:33.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.576+0000 7f1bdeb7f740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558e5e2a04a0 0x558e5e29bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:33.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.576+0000 7f1bdeb7f740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558e5e2a0e90 0x558e5e2b1f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:33.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.576+0000 7f1bdeb7f740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558e5e2a6000 0x558e5e2b67e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:33.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.576+0000 7f1bdeb7f740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x558e5e2b71a0 con 0x558e5e2a04a0 2024-04-16T17:06:33.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.576+0000 7f1bdeb7f740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x558e5e2b7870 con 0x558e5e2a6000 2024-04-16T17:06:33.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.576+0000 7f1bdeb7f740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x558e5e2b7f40 con 0x558e5e2a0e90 2024-04-16T17:06:33.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.576+0000 7f1bd9400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558e5e2a04a0 0x558e5e29bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:33.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.576+0000 7f1bd9400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558e5e2a04a0 0x558e5e29bdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55578/0 (socket says 172.21.15.45:55578) 2024-04-16T17:06:33.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.576+0000 7f1bd9400640 1 -- 172.21.15.45:0/2844956968 learned_addr learned my addr 172.21.15.45:0/2844956968 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:06:33.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.576+0000 7f1bd8a00640 1 --2- 172.21.15.45:0/2844956968 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558e5e2a0e90 0x558e5e2b1f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:33.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.576+0000 7f1bd9e00640 1 --2- 172.21.15.45:0/2844956968 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558e5e2a6000 0x558e5e2b67e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:33.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.576+0000 7f1bd9400640 1 -- 172.21.15.45:0/2844956968 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558e5e2a0e90 msgr2=0x558e5e2b1f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:06:33.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.576+0000 7f1bd9400640 1 --2- 172.21.15.45:0/2844956968 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558e5e2a0e90 0x558e5e2b1f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:33.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.576+0000 7f1bd9400640 1 -- 172.21.15.45:0/2844956968 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558e5e2a6000 msgr2=0x558e5e2b67e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:06:33.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.576+0000 7f1bd9400640 1 --2- 172.21.15.45:0/2844956968 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558e5e2a6000 0x558e5e2b67e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:33.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.576+0000 7f1bd9400640 1 -- 172.21.15.45:0/2844956968 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558e5e29c780 con 0x558e5e2a04a0 2024-04-16T17:06:33.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.576+0000 7f1bd8a00640 1 --2- 172.21.15.45:0/2844956968 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558e5e2a0e90 0x558e5e2b1f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:06:33.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.577+0000 7f1bd9400640 1 --2- 172.21.15.45:0/2844956968 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558e5e2a04a0 0x558e5e29bdf0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f1bc80099a0 tx=0x7f1bc8009300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b177de8fa3a5672 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:06:33.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.577+0000 7f1bc5c00640 1 -- 172.21.15.45:0/2844956968 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bc8051070 con 0x558e5e2a04a0 2024-04-16T17:06:33.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.577+0000 7f1bc5c00640 1 -- 172.21.15.45:0/2844956968 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1bc8033920 con 0x558e5e2a04a0 2024-04-16T17:06:33.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.577+0000 7f1bc5c00640 1 -- 172.21.15.45:0/2844956968 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bc8004860 con 0x558e5e2a04a0 2024-04-16T17:06:33.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.578+0000 7f1bdeb7f740 1 -- 172.21.15.45:0/2844956968 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558e5e2a04a0 msgr2=0x558e5e29bdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:33.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.578+0000 7f1bdeb7f740 1 --2- 172.21.15.45:0/2844956968 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558e5e2a04a0 0x558e5e29bdf0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f1bc80099a0 tx=0x7f1bc8009300 comp rx=0 tx=0).stop 2024-04-16T17:06:33.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.578+0000 7f1bdeb7f740 1 -- 172.21.15.45:0/2844956968 shutdown_connections 2024-04-16T17:06:33.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.578+0000 7f1bdeb7f740 1 --2- 172.21.15.45:0/2844956968 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558e5e2a0e90 0x558e5e2b1f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:33.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.578+0000 7f1bdeb7f740 1 --2- 172.21.15.45:0/2844956968 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558e5e2a6000 0x558e5e2b67e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:33.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.578+0000 7f1bdeb7f740 1 --2- 172.21.15.45:0/2844956968 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558e5e2a04a0 0x558e5e29bdf0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:33.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.578+0000 7f1bdeb7f740 1 -- 172.21.15.45:0/2844956968 >> 172.21.15.45:0/2844956968 conn(0x558e5e29ba10 msgr2=0x558e5e2a94d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:06:33.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.578+0000 7f1bdeb7f740 1 -- 172.21.15.45:0/2844956968 shutdown_connections 2024-04-16T17:06:33.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.578+0000 7f1bdeb7f740 1 -- 172.21.15.45:0/2844956968 wait complete. 2024-04-16T17:06:33.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.578+0000 7f1bdeb7f740 1 -- start start 2024-04-16T17:06:33.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.579+0000 7f1bdeb7f740 1 Processor -- start 2024-04-16T17:06:33.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.579+0000 7f1bdeb7f740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558e5e2a04a0 0x558e5e2ad620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:33.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.579+0000 7f1bdeb7f740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558e5e2a0e90 0x558e5e2adb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:33.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.579+0000 7f1bdeb7f740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558e5e2a6000 0x558e5e2b0b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:33.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.579+0000 7f1bdeb7f740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x558e5e2b8ce0 con 0x558e5e2a04a0 2024-04-16T17:06:33.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.579+0000 7f1bdeb7f740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x558e5e2b8610 con 0x558e5e2a6000 2024-04-16T17:06:33.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.579+0000 7f1bdeb7f740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x558e5e2b93b0 con 0x558e5e2a0e90 2024-04-16T17:06:33.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.579+0000 7f1bdeb7f740 10 client.0 _ll_register_callbacks cb 0x7f1bc8009a60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:06:33.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.579+0000 7f1bd9400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558e5e2a04a0 0x558e5e2ad620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:33.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.579+0000 7f1bd9400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558e5e2a04a0 0x558e5e2ad620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55588/0 (socket says 172.21.15.45:55588) 2024-04-16T17:06:33.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.579+0000 7f1bd9400640 1 -- 172.21.15.45:0/364704727 learned_addr learned my addr 172.21.15.45:0/364704727 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:06:33.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.579+0000 7f1bd8a00640 1 --2- 172.21.15.45:0/364704727 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558e5e2a0e90 0x558e5e2adb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:33.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.579+0000 7f1bd9e00640 1 --2- 172.21.15.45:0/364704727 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558e5e2a6000 0x558e5e2b0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:33.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.580+0000 7f1bd9400640 1 -- 172.21.15.45:0/364704727 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558e5e2a0e90 msgr2=0x558e5e2adb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:06:33.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.580+0000 7f1bd9400640 1 --2- 172.21.15.45:0/364704727 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558e5e2a0e90 0x558e5e2adb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:33.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.580+0000 7f1bd9400640 1 -- 172.21.15.45:0/364704727 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558e5e2a6000 msgr2=0x558e5e2b0b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:06:33.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.580+0000 7f1bd9400640 1 --2- 172.21.15.45:0/364704727 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558e5e2a6000 0x558e5e2b0b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:33.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.580+0000 7f1bd9400640 1 -- 172.21.15.45:0/364704727 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558e5e2b1170 con 0x558e5e2a04a0 2024-04-16T17:06:33.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.580+0000 7f1bd9400640 1 --2- 172.21.15.45:0/364704727 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558e5e2a04a0 0x558e5e2ad620 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f1bc8063040 tx=0x7f1bc80047d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=788ccaa964691565 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:06:33.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.580+0000 7f1bbaa00640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-16T17:06:33.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.580+0000 7f1bbaa00640 1 -- 172.21.15.45:0/364704727 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bc8051040 con 0x558e5e2a04a0 2024-04-16T17:06:33.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.580+0000 7f1bbaa00640 1 -- 172.21.15.45:0/364704727 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f1bc8004190 con 0x558e5e2a04a0 2024-04-16T17:06:33.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.580+0000 7f1bdeb7f740 1 -- 172.21.15.45:0/364704727 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x558e5e20e210 con 0x558e5e2a04a0 2024-04-16T17:06:33.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.581+0000 7f1bbaa00640 1 -- 172.21.15.45:0/364704727 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f1bc8076050 con 0x558e5e2a04a0 2024-04-16T17:06:33.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.581+0000 7f1bdeb7f740 10 client.15246 fetch_fsmap learned FSMap version 24 2024-04-16T17:06:33.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.581+0000 7f1bdeb7f740 1 -- 172.21.15.45:0/364704727 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x558e5e31d9a0 con 0x558e5e2a04a0 2024-04-16T17:06:33.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.581+0000 7f1bbaa00640 1 -- 172.21.15.45:0/364704727 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x558e5e20e210 con 0x558e5e2a04a0 2024-04-16T17:06:33.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.581+0000 7f1bbaa00640 1 -- 172.21.15.45:0/364704727 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f1bc806a070 con 0x558e5e2a04a0 2024-04-16T17:06:33.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.581+0000 7f1bdeb7f740 10 client.15246 fetch_fsmap finished waiting for FSMap version 24 2024-04-16T17:06:33.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.581+0000 7f1bdeb7f740 10 client.15246 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-16T17:06:33.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.581+0000 7f1bdeb7f740 20 client.15246 populate_metadata read hostname 'smithi045' 2024-04-16T17:06:33.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.581+0000 7f1bdeb7f740 1 --2- 172.21.15.45:0/364704727 >> [v2:172.21.15.117:6838/2378675949,v1:172.21.15.117:6839/2378675949] conn(0x558e5e322310 0x558e5e321020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:33.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.581+0000 7f1bdeb7f740 4 client.15246 mds_command: new command op to 24532 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-16T17:06:33.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.581+0000 7f1bdeb7f740 1 -- 172.21.15.45:0/364704727 --> [v2:172.21.15.117:6838/2378675949,v1:172.21.15.117:6839/2378675949] -- command(tid 0: {"prefix": "status"}) -- 0x558e5e18a7d0 con 0x558e5e322310 2024-04-16T17:06:33.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.582+0000 7f1bd8a00640 1 --2- 172.21.15.45:0/364704727 >> [v2:172.21.15.117:6838/2378675949,v1:172.21.15.117:6839/2378675949] conn(0x558e5e322310 0x558e5e321020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:33.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.582+0000 7f1bd8a00640 1 --2- 172.21.15.45:0/364704727 >> [v2:172.21.15.117:6838/2378675949,v1:172.21.15.117:6839/2378675949] conn(0x558e5e322310 0x558e5e321020 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=31f02461d20960da server_cookie=ba5fe7f7f36fc20d in_seq=0 out_seq=0 2024-04-16T17:06:33.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.582+0000 7f1bbaa00640 10 client.15246 ms_handle_connect on v2:172.21.15.117:6838/2378675949 2024-04-16T17:06:33.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.583+0000 7f1bbaa00640 1 -- 172.21.15.45:0/364704727 <== mds.2 v2:172.21.15.117:6838/2378675949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x558e5e18a7d0 con 0x558e5e322310 2024-04-16T17:06:33.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.583+0000 7f1bbaa00640 10 client.15246 handle_command_reply: tid=0 2024-04-16T17:06:33.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.583+0000 7f1bdeb7f740 1 client.15246 shutdown 2024-04-16T17:06:33.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.583+0000 7f1bdeb7f740 1 -- 172.21.15.45:0/364704727 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558e5e2a04a0 msgr2=0x558e5e2ad620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:33.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.583+0000 7f1bdeb7f740 1 --2- 172.21.15.45:0/364704727 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558e5e2a04a0 0x558e5e2ad620 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f1bc8063040 tx=0x7f1bc80047d0 comp rx=0 tx=0).stop 2024-04-16T17:06:33.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.583+0000 7f1bdeb7f740 1 -- 172.21.15.45:0/364704727 shutdown_connections 2024-04-16T17:06:33.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.583+0000 7f1bdeb7f740 1 --2- 172.21.15.45:0/364704727 >> [v2:172.21.15.117:6838/2378675949,v1:172.21.15.117:6839/2378675949] conn(0x558e5e322310 0x558e5e321020 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:33.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.583+0000 7f1bdeb7f740 1 --2- 172.21.15.45:0/364704727 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558e5e2a0e90 0x558e5e2adb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:33.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.583+0000 7f1bdeb7f740 1 --2- 172.21.15.45:0/364704727 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558e5e2a6000 0x558e5e2b0b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:33.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.583+0000 7f1bdeb7f740 1 --2- 172.21.15.45:0/364704727 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558e5e2a04a0 0x558e5e2ad620 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:33.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.583+0000 7f1bbaa00640 0 client.15246 ms_handle_reset on v2:172.21.15.117:6838/2378675949 2024-04-16T17:06:33.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.583+0000 7f1bdeb7f740 1 -- 172.21.15.45:0/364704727 >> 172.21.15.45:0/364704727 conn(0x558e5e29ba10 msgr2=0x558e5e2b67e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:06:33.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.583+0000 7f1bdeb7f740 1 -- 172.21.15.45:0/364704727 shutdown_connections 2024-04-16T17:06:33.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.583+0000 7f1bdeb7f740 1 -- 172.21.15.45:0/364704727 wait complete. 2024-04-16T17:06:33.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.583+0000 7f1bdeb7f740 20 client.15246 trim_cache size 0 max 16384 2024-04-16T17:06:33.589 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-16T17:06:33.589 INFO:teuthology.orchestra.run.smithi045.stdout: "cluster_fsid": "b1fecd38-fc11-11ee-bc90-c7b262605968", 2024-04-16T17:06:33.589 INFO:teuthology.orchestra.run.smithi045.stdout: "whoami": 2, 2024-04-16T17:06:33.589 INFO:teuthology.orchestra.run.smithi045.stdout: "id": 24532, 2024-04-16T17:06:33.589 INFO:teuthology.orchestra.run.smithi045.stdout: "want_state": "up:active", 2024-04-16T17:06:33.589 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "up:active", 2024-04-16T17:06:33.589 INFO:teuthology.orchestra.run.smithi045.stdout: "fs_name": "cephfs", 2024-04-16T17:06:33.589 INFO:teuthology.orchestra.run.smithi045.stdout: "rank_uptime": 161.212471943, 2024-04-16T17:06:33.589 INFO:teuthology.orchestra.run.smithi045.stdout: "mdsmap_epoch": 24, 2024-04-16T17:06:33.589 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_epoch": 94, 2024-04-16T17:06:33.589 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_epoch_barrier": 90, 2024-04-16T17:06:33.589 INFO:teuthology.orchestra.run.smithi045.stdout: "uptime": 206.33751565099999 2024-04-16T17:06:33.590 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-16T17:06:33.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.584+0000 7f1bda800640 1 -- 172.21.15.45:0/4136985619 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f1bbc0a3bb0 msgr2=0x7f1bbc0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:33.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.584+0000 7f1bda800640 1 --2- 172.21.15.45:0/4136985619 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f1bbc0a3bb0 0x7f1bbc0a6070 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f1bc8009be0 tx=0x7f1bc8004a40 comp rx=0 tx=0).stop 2024-04-16T17:06:33.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.584+0000 7f1bda800640 1 -- 172.21.15.45:0/4136985619 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1bd4076040 msgr2=0x7f1bd419ed80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:33.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.584+0000 7f1bda800640 1 --2- 172.21.15.45:0/4136985619 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1bd4076040 0x7f1bd419ed80 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f1bc0005fd0 tx=0x7f1bc000ee50 comp rx=0 tx=0).stop 2024-04-16T17:06:33.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.584+0000 7f1bda800640 1 -- 172.21.15.45:0/4136985619 shutdown_connections 2024-04-16T17:06:33.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.584+0000 7f1bda800640 1 --2- 172.21.15.45:0/4136985619 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f1bbc0a3bb0 0x7f1bbc0a6070 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:33.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.584+0000 7f1bda800640 1 --2- 172.21.15.45:0/4136985619 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1bd410e4e0 0x7f1bd41a5df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:33.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.584+0000 7f1bda800640 1 --2- 172.21.15.45:0/4136985619 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1bd4076040 0x7f1bd419ed80 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:33.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.584+0000 7f1bda800640 1 --2- 172.21.15.45:0/4136985619 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1bd4074a60 0x7f1bd419e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:33.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.584+0000 7f1bda800640 1 -- 172.21.15.45:0/4136985619 >> 172.21.15.45:0/4136985619 conn(0x7f1bd40fd4a0 msgr2=0x7f1bd4101880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:06:33.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.584+0000 7f1bda800640 1 -- 172.21.15.45:0/4136985619 shutdown_connections 2024-04-16T17:06:33.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.584+0000 7f1bda800640 1 -- 172.21.15.45:0/4136985619 wait complete. 2024-04-16T17:06:33.599 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:06:33 smithi045 ceph-mon[31676]: pgmap v420: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-16T17:06:33.599 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:06:33 smithi045 ceph-mon[31676]: from='client.? 172.21.15.45:0/1152120360' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:06:33.599 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:06:33 smithi045 ceph-mon[31676]: from='client.? 172.21.15.45:0/3453016261' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T17:06:33.602 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-16T17:06:33.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:06:33 smithi104 ceph-mon[36457]: pgmap v420: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-16T17:06:33.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:06:33 smithi104 ceph-mon[36457]: from='client.? 172.21.15.45:0/1152120360' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:06:33.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:06:33 smithi104 ceph-mon[36457]: from='client.? 172.21.15.45:0/3453016261' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T17:06:33.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:06:33 smithi117 ceph-mon[36730]: pgmap v420: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-16T17:06:33.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:06:33 smithi117 ceph-mon[36730]: from='client.? 172.21.15.45:0/1152120360' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:06:33.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:06:33 smithi117 ceph-mon[36730]: from='client.? 172.21.15.45:0/3453016261' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T17:06:33.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.831+0000 7f3397a00640 1 -- 172.21.15.45:0/330784372 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3390105e10 msgr2=0x7f339010c6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:33.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.831+0000 7f3397a00640 1 --2- 172.21.15.45:0/330784372 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3390105e10 0x7f339010c6a0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f3388009900 tx=0x7f338802bc50 comp rx=0 tx=0).stop 2024-04-16T17:06:33.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.831+0000 7f3397a00640 1 -- 172.21.15.45:0/330784372 shutdown_connections 2024-04-16T17:06:33.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.831+0000 7f3397a00640 1 --2- 172.21.15.45:0/330784372 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3390105450 0x7f33901058d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:33.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.831+0000 7f3397a00640 1 --2- 172.21.15.45:0/330784372 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3390105e10 0x7f339010c6a0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:33.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.831+0000 7f3397a00640 1 --2- 172.21.15.45:0/330784372 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3390104260 0x7f3390104680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:33.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.831+0000 7f3397a00640 1 -- 172.21.15.45:0/330784372 >> 172.21.15.45:0/330784372 conn(0x7f33900ff7b0 msgr2=0x7f3390101c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:06:33.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.831+0000 7f3397a00640 1 -- 172.21.15.45:0/330784372 shutdown_connections 2024-04-16T17:06:33.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.831+0000 7f3397a00640 1 -- 172.21.15.45:0/330784372 wait complete. 2024-04-16T17:06:33.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.832+0000 7f3397a00640 1 Processor -- start 2024-04-16T17:06:33.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.832+0000 7f3397a00640 1 -- start start 2024-04-16T17:06:33.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.832+0000 7f3397a00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3390104260 0x7f339019a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:33.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.832+0000 7f3397a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3390105450 0x7f339019aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:33.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.833+0000 7f3397a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3390105e10 0x7f33901a1b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:33.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.833+0000 7f3397a00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f339010fef0 con 0x7f3390105450 2024-04-16T17:06:33.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.833+0000 7f3397a00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f339010fd70 con 0x7f3390104260 2024-04-16T17:06:33.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.833+0000 7f3397a00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f3390110070 con 0x7f3390105e10 2024-04-16T17:06:33.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.833+0000 7f3395000640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3390104260 0x7f339019a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:33.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.833+0000 7f3395000640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3390104260 0x7f339019a590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:48780/0 (socket says 172.21.15.45:48780) 2024-04-16T17:06:33.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.833+0000 7f3395000640 1 -- 172.21.15.45:0/2606093465 learned_addr learned my addr 172.21.15.45:0/2606093465 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:06:33.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.833+0000 7f338fe00640 1 --2- 172.21.15.45:0/2606093465 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3390105450 0x7f339019aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:33.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.833+0000 7f3395a00640 1 --2- 172.21.15.45:0/2606093465 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3390105e10 0x7f33901a1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:33.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.833+0000 7f3395000640 1 -- 172.21.15.45:0/2606093465 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3390105e10 msgr2=0x7f33901a1b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:33.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.833+0000 7f3395000640 1 --2- 172.21.15.45:0/2606093465 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3390105e10 0x7f33901a1b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:33.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.833+0000 7f3395000640 1 -- 172.21.15.45:0/2606093465 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3390105450 msgr2=0x7f339019aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:33.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.833+0000 7f3395000640 1 --2- 172.21.15.45:0/2606093465 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3390105450 0x7f339019aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:33.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.833+0000 7f3395000640 1 -- 172.21.15.45:0/2606093465 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33901a21b0 con 0x7f3390104260 2024-04-16T17:06:33.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.834+0000 7f3395000640 1 --2- 172.21.15.45:0/2606093465 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3390104260 0x7f339019a590 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f338000e9c0 tx=0x7f338000ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:06:33.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.834+0000 7f338d600640 1 -- 172.21.15.45:0/2606093465 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f338000cde0 con 0x7f3390104260 2024-04-16T17:06:33.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.834+0000 7f338d600640 1 -- 172.21.15.45:0/2606093465 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3380010910 con 0x7f3390104260 2024-04-16T17:06:33.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.834+0000 7f338d600640 1 -- 172.21.15.45:0/2606093465 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3380018a00 con 0x7f3390104260 2024-04-16T17:06:33.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.834+0000 7f3397a00640 1 -- 172.21.15.45:0/2606093465 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33901a24a0 con 0x7f3390104260 2024-04-16T17:06:33.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.834+0000 7f3397a00640 1 -- 172.21.15.45:0/2606093465 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33901a29a0 con 0x7f3390104260 2024-04-16T17:06:33.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.837+0000 7f339a7da740 1 Processor -- start 2024-04-16T17:06:33.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.837+0000 7f338d600640 1 -- 172.21.15.45:0/2606093465 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f338001f020 con 0x7f3390104260 2024-04-16T17:06:33.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.837+0000 7f339a7da740 1 -- start start 2024-04-16T17:06:33.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.838+0000 7f339a7da740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55988af872d0 0x55988af876f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:33.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.838+0000 7f339a7da740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55988af87cc0 0x55988af8c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:33.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.838+0000 7f339a7da740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55988af8ce30 0x55988af8f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:33.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.838+0000 7f339a7da740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55988aeed980 con 0x55988af872d0 2024-04-16T17:06:33.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.838+0000 7f339a7da740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55988ae94220 con 0x55988af8ce30 2024-04-16T17:06:33.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.838+0000 7f339a7da740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55988aee9940 con 0x55988af87cc0 2024-04-16T17:06:33.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.838+0000 7f3395a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55988af872d0 0x55988af876f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:33.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.838+0000 7f3395a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55988af872d0 0x55988af876f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55604/0 (socket says 172.21.15.45:55604) 2024-04-16T17:06:33.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.838+0000 7f3395a00640 1 -- 172.21.15.45:0/617679853 learned_addr learned my addr 172.21.15.45:0/617679853 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:06:33.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.838+0000 7f338fe00640 1 --2- 172.21.15.45:0/617679853 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55988af8ce30 0x55988af8f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:33.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.838+0000 7f3395000640 1 --2- 172.21.15.45:0/617679853 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55988af87cc0 0x55988af8c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:33.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.839+0000 7f3395a00640 1 -- 172.21.15.45:0/617679853 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55988af87cc0 msgr2=0x55988af8c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:06:33.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.839+0000 7f3395a00640 1 --2- 172.21.15.45:0/617679853 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55988af87cc0 0x55988af8c7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:33.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.839+0000 7f3395a00640 1 -- 172.21.15.45:0/617679853 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55988af8ce30 msgr2=0x55988af8f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:06:33.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.839+0000 7f3395a00640 1 --2- 172.21.15.45:0/617679853 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55988af8ce30 0x55988af8f2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:33.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.839+0000 7f3395a00640 1 -- 172.21.15.45:0/617679853 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55988ac002d0 con 0x55988af872d0 2024-04-16T17:06:33.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.839+0000 7f338fe00640 1 --2- 172.21.15.45:0/617679853 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55988af8ce30 0x55988af8f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:06:33.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.839+0000 7f3395a00640 1 --2- 172.21.15.45:0/617679853 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55988af872d0 0x55988af876f0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f3388009230 tx=0x7f3388009560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9045835b467ec95 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:06:33.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.839+0000 7f338d600640 1 --2- 172.21.15.45:0/2606093465 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f33740a3c30 0x7f33740a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:33.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.839+0000 7f3386000640 1 -- 172.21.15.45:0/617679853 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f338802cba0 con 0x55988af872d0 2024-04-16T17:06:33.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.839+0000 7f3386000640 1 -- 172.21.15.45:0/617679853 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3388005dd0 con 0x55988af872d0 2024-04-16T17:06:33.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.839+0000 7f3386000640 1 -- 172.21.15.45:0/617679853 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3388005150 con 0x55988af872d0 2024-04-16T17:06:33.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.839+0000 7f338d600640 1 -- 172.21.15.45:0/2606093465 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f33800d08e0 con 0x7f3390104260 2024-04-16T17:06:33.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.839+0000 7f3395000640 1 --2- 172.21.15.45:0/2606093465 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f33740a3c30 0x7f33740a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:33.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.840+0000 7f339a7da740 1 -- 172.21.15.45:0/617679853 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55988af872d0 msgr2=0x55988af876f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:33.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.840+0000 7f3395000640 1 --2- 172.21.15.45:0/2606093465 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f33740a3c30 0x7f33740a60f0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x55988af880c0 tx=0x7f338000efb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:06:33.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.840+0000 7f339a7da740 1 --2- 172.21.15.45:0/617679853 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55988af872d0 0x55988af876f0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f3388009230 tx=0x7f3388009560 comp rx=0 tx=0).stop 2024-04-16T17:06:33.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.840+0000 7f339a7da740 1 -- 172.21.15.45:0/617679853 shutdown_connections 2024-04-16T17:06:33.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.840+0000 7f339a7da740 1 --2- 172.21.15.45:0/617679853 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55988af87cc0 0x55988af8c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:33.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.840+0000 7f339a7da740 1 --2- 172.21.15.45:0/617679853 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55988af8ce30 0x55988af8f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:33.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.840+0000 7f339a7da740 1 --2- 172.21.15.45:0/617679853 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55988af872d0 0x55988af876f0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:33.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.840+0000 7f339a7da740 1 -- 172.21.15.45:0/617679853 >> 172.21.15.45:0/617679853 conn(0x55988af82840 msgr2=0x55988af82c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:06:33.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.840+0000 7f339a7da740 1 -- 172.21.15.45:0/617679853 shutdown_connections 2024-04-16T17:06:33.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.840+0000 7f339a7da740 1 -- 172.21.15.45:0/617679853 wait complete. 2024-04-16T17:06:33.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.841+0000 7f339a7da740 1 -- start start 2024-04-16T17:06:33.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.841+0000 7f339a7da740 1 Processor -- start 2024-04-16T17:06:33.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.842+0000 7f339a7da740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55988af872d0 0x55988af917e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:33.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.842+0000 7f339a7da740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55988af87cc0 0x55988af91d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:33.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.842+0000 7f339a7da740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55988af8ce30 0x55988af98d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:33.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.842+0000 7f339a7da740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55988af8fd80 con 0x55988af8ce30 2024-04-16T17:06:33.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.842+0000 7f339a7da740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55988aea7590 con 0x55988af87cc0 2024-04-16T17:06:33.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.842+0000 7f339a7da740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55988af8ff00 con 0x55988af872d0 2024-04-16T17:06:33.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.842+0000 7f339a7da740 10 client.0 _ll_register_callbacks cb 0x55988aeaf050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:06:33.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.842+0000 7f3395a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55988af8ce30 0x55988af98d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:33.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.842+0000 7f338fe00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55988af87cc0 0x55988af91d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:33.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.842+0000 7f3395a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55988af8ce30 0x55988af98d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55606/0 (socket says 172.21.15.45:55606) 2024-04-16T17:06:33.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.842+0000 7f3395a00640 1 -- 172.21.15.45:0/3781656977 learned_addr learned my addr 172.21.15.45:0/3781656977 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:06:33.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.842+0000 7f3395a00640 1 --2- 172.21.15.45:0/3781656977 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55988af872d0 0x55988af917e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:33.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.842+0000 7f338fe00640 1 -- 172.21.15.45:0/3781656977 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55988af872d0 msgr2=0x55988af917e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:06:33.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.842+0000 7f338fe00640 1 --2- 172.21.15.45:0/3781656977 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55988af872d0 0x55988af917e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:33.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.842+0000 7f338fe00640 1 -- 172.21.15.45:0/3781656977 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55988af8ce30 msgr2=0x55988af98d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:06:33.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.842+0000 7f338fe00640 1 --2- 172.21.15.45:0/3781656977 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55988af8ce30 0x55988af98d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:33.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.842+0000 7f338fe00640 1 -- 172.21.15.45:0/3781656977 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55988acca6b0 con 0x55988af87cc0 2024-04-16T17:06:33.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.843+0000 7f338fe00640 1 --2- 172.21.15.45:0/3781656977 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55988af87cc0 0x55988af91d20 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f339019baa0 tx=0x7f337c00f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=910e856adc7ab244 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:06:33.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.843+0000 7f337aa00640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-04-16T17:06:33.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.843+0000 7f337aa00640 1 -- 172.21.15.45:0/3781656977 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f337c003970 con 0x55988af87cc0 2024-04-16T17:06:33.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.843+0000 7f337aa00640 1 -- 172.21.15.45:0/3781656977 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f337c003b10 con 0x55988af87cc0 2024-04-16T17:06:33.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.843+0000 7f339a7da740 1 -- 172.21.15.45:0/3781656977 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3388005dd0 con 0x55988af87cc0 2024-04-16T17:06:33.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.843+0000 7f337aa00640 1 -- 172.21.15.45:0/3781656977 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f337c003d10 con 0x55988af87cc0 2024-04-16T17:06:33.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.843+0000 7f339a7da740 10 client.25370 fetch_fsmap learned FSMap version 24 2024-04-16T17:06:33.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.843+0000 7f339a7da740 1 -- 172.21.15.45:0/3781656977 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55988af9c140 con 0x55988af87cc0 2024-04-16T17:06:33.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.844+0000 7f337aa00640 1 -- 172.21.15.45:0/3781656977 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3388005dd0 con 0x55988af87cc0 2024-04-16T17:06:33.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.844+0000 7f337aa00640 1 -- 172.21.15.45:0/3781656977 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f337c005020 con 0x55988af87cc0 2024-04-16T17:06:33.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.844+0000 7f339a7da740 10 client.25370 fetch_fsmap finished waiting for FSMap version 24 2024-04-16T17:06:33.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.844+0000 7f339a7da740 10 client.25370 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-16T17:06:33.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.844+0000 7f339a7da740 20 client.25370 populate_metadata read hostname 'smithi045' 2024-04-16T17:06:33.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.844+0000 7f339a7da740 1 --2- 172.21.15.45:0/3781656977 >> [v2:172.21.15.45:6840/3755310822,v1:172.21.15.45:6841/3755310822] conn(0x55988af9e790 0x55988af9fc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:33.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.844+0000 7f339a7da740 4 client.25370 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:06:33.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.844+0000 7f339a7da740 1 -- 172.21.15.45:0/3781656977 --> [v2:172.21.15.45:6840/3755310822,v1:172.21.15.45:6841/3755310822] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55988aefb3a0 con 0x55988af9e790 2024-04-16T17:06:33.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.844+0000 7f3395a00640 1 --2- 172.21.15.45:0/3781656977 >> [v2:172.21.15.45:6840/3755310822,v1:172.21.15.45:6841/3755310822] conn(0x55988af9e790 0x55988af9fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:33.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.845+0000 7f3395a00640 1 --2- 172.21.15.45:0/3781656977 >> [v2:172.21.15.45:6840/3755310822,v1:172.21.15.45:6841/3755310822] conn(0x55988af9e790 0x55988af9fc00 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a4e725817304d246 server_cookie=614e3d0ea9ddbdbf in_seq=0 out_seq=0 2024-04-16T17:06:33.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.845+0000 7f337aa00640 10 client.25370 ms_handle_connect on v2:172.21.15.45:6840/3755310822 2024-04-16T17:06:33.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.846+0000 7f337aa00640 1 -- 172.21.15.45:0/3781656977 <== mds.1 v2:172.21.15.45:6840/3755310822 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55988aefb3a0 con 0x55988af9e790 2024-04-16T17:06:33.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.846+0000 7f337aa00640 10 client.25370 handle_command_reply: tid=0 2024-04-16T17:06:33.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.846+0000 7f339a7da740 1 client.25370 shutdown 2024-04-16T17:06:33.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.846+0000 7f339a7da740 1 -- 172.21.15.45:0/3781656977 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55988af87cc0 msgr2=0x55988af91d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:33.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.846+0000 7f339a7da740 1 --2- 172.21.15.45:0/3781656977 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55988af87cc0 0x55988af91d20 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f339019baa0 tx=0x7f337c00f790 comp rx=0 tx=0).stop 2024-04-16T17:06:33.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.846+0000 7f339a7da740 1 -- 172.21.15.45:0/3781656977 shutdown_connections 2024-04-16T17:06:33.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.846+0000 7f339a7da740 1 --2- 172.21.15.45:0/3781656977 >> [v2:172.21.15.45:6840/3755310822,v1:172.21.15.45:6841/3755310822] conn(0x55988af9e790 0x55988af9fc00 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:33.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.846+0000 7f339a7da740 1 --2- 172.21.15.45:0/3781656977 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55988af8ce30 0x55988af98d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:33.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.846+0000 7f339a7da740 1 --2- 172.21.15.45:0/3781656977 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55988af87cc0 0x55988af91d20 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:33.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.846+0000 7f339a7da740 1 --2- 172.21.15.45:0/3781656977 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55988af872d0 0x55988af917e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:33.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.846+0000 7f339a7da740 1 -- 172.21.15.45:0/3781656977 >> 172.21.15.45:0/3781656977 conn(0x55988af82840 msgr2=0x55988af81a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:06:33.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.846+0000 7f337aa00640 0 client.25370 ms_handle_reset on v2:172.21.15.45:6840/3755310822 2024-04-16T17:06:33.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.846+0000 7f339a7da740 1 -- 172.21.15.45:0/3781656977 shutdown_connections 2024-04-16T17:06:33.852 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.846+0000 7f339a7da740 1 -- 172.21.15.45:0/3781656977 wait complete. 2024-04-16T17:06:33.852 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.847+0000 7f339a7da740 20 client.25370 trim_cache size 0 max 16384 2024-04-16T17:06:33.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.867+0000 7f339a7da740 1 Processor -- start 2024-04-16T17:06:33.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.867+0000 7f339a7da740 1 -- start start 2024-04-16T17:06:33.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.867+0000 7f339a7da740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55988af872d0 0x55988af89180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:33.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.867+0000 7f339a7da740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55988af87cc0 0x55988af896c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:33.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.867+0000 7f339a7da740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55988af8ce30 0x55988af98d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:33.869 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.867+0000 7f339a7da740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55988af9df40 con 0x55988af8ce30 2024-04-16T17:06:33.869 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.867+0000 7f339a7da740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55988af9e610 con 0x55988af87cc0 2024-04-16T17:06:33.870 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.867+0000 7f339a7da740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55988af9ece0 con 0x55988af872d0 2024-04-16T17:06:33.870 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.867+0000 7f3395a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55988af8ce30 0x55988af98d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:33.870 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.867+0000 7f3395a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55988af8ce30 0x55988af98d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55614/0 (socket says 172.21.15.45:55614) 2024-04-16T17:06:33.870 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.867+0000 7f3395a00640 1 -- 172.21.15.45:0/2058805140 learned_addr learned my addr 172.21.15.45:0/2058805140 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:06:33.870 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.867+0000 7f338fe00640 1 --2- 172.21.15.45:0/2058805140 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55988af87cc0 0x55988af896c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:33.870 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.868+0000 7f3395a00640 1 --2- 172.21.15.45:0/2058805140 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55988af872d0 0x55988af89180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:33.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.868+0000 7f338fe00640 1 -- 172.21.15.45:0/2058805140 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55988af872d0 msgr2=0x55988af89180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:06:33.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.868+0000 7f338fe00640 1 --2- 172.21.15.45:0/2058805140 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55988af872d0 0x55988af89180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:33.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.868+0000 7f338fe00640 1 -- 172.21.15.45:0/2058805140 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55988af8ce30 msgr2=0x55988af98d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:06:33.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.868+0000 7f338fe00640 1 --2- 172.21.15.45:0/2058805140 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55988af8ce30 0x55988af98d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:33.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.868+0000 7f338fe00640 1 -- 172.21.15.45:0/2058805140 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55988af8c450 con 0x55988af87cc0 2024-04-16T17:06:33.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.868+0000 7f338fe00640 1 --2- 172.21.15.45:0/2058805140 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55988af87cc0 0x55988af896c0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f337c005c40 tx=0x7f337c003c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cc6b1710e3c5c016 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:06:33.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.868+0000 7f3385600640 1 -- 172.21.15.45:0/2058805140 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f337c005070 con 0x55988af87cc0 2024-04-16T17:06:33.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.868+0000 7f3385600640 1 -- 172.21.15.45:0/2058805140 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f337c009570 con 0x55988af87cc0 2024-04-16T17:06:33.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.868+0000 7f3385600640 1 -- 172.21.15.45:0/2058805140 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f337c029b90 con 0x55988af87cc0 2024-04-16T17:06:33.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.869+0000 7f339a7da740 1 -- 172.21.15.45:0/2058805140 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55988af87cc0 msgr2=0x55988af896c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:33.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.869+0000 7f339a7da740 1 --2- 172.21.15.45:0/2058805140 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55988af87cc0 0x55988af896c0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f337c005c40 tx=0x7f337c003c00 comp rx=0 tx=0).stop 2024-04-16T17:06:33.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.869+0000 7f339a7da740 1 -- 172.21.15.45:0/2058805140 shutdown_connections 2024-04-16T17:06:33.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.869+0000 7f339a7da740 1 --2- 172.21.15.45:0/2058805140 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55988af8ce30 0x55988af98d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:33.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.869+0000 7f339a7da740 1 --2- 172.21.15.45:0/2058805140 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55988af87cc0 0x55988af896c0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:33.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.869+0000 7f339a7da740 1 --2- 172.21.15.45:0/2058805140 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55988af872d0 0x55988af89180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:33.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.869+0000 7f339a7da740 1 -- 172.21.15.45:0/2058805140 >> 172.21.15.45:0/2058805140 conn(0x55988af82840 msgr2=0x55988af82c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:06:33.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.869+0000 7f339a7da740 1 -- 172.21.15.45:0/2058805140 shutdown_connections 2024-04-16T17:06:33.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.869+0000 7f339a7da740 1 -- 172.21.15.45:0/2058805140 wait complete. 2024-04-16T17:06:33.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.870+0000 7f339a7da740 1 -- start start 2024-04-16T17:06:33.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.870+0000 7f339a7da740 1 Processor -- start 2024-04-16T17:06:33.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.870+0000 7f339a7da740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55988af872d0 0x55988af94630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:33.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.870+0000 7f339a7da740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55988af87cc0 0x55988af94b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:33.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.870+0000 7f339a7da740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55988af8ce30 0x55988af97b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:33.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.870+0000 7f339a7da740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55988af9fa80 con 0x55988af8ce30 2024-04-16T17:06:33.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.870+0000 7f339a7da740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55988af9f3b0 con 0x55988af87cc0 2024-04-16T17:06:33.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.870+0000 7f339a7da740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55988afa0150 con 0x55988af872d0 2024-04-16T17:06:33.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.870+0000 7f339a7da740 10 client.0 _ll_register_callbacks cb 0x7f337c009cc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:06:33.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.871+0000 7f338fe00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55988af87cc0 0x55988af94b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:33.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.871+0000 7f3395a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55988af872d0 0x55988af94630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:33.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.871+0000 7f338fe00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55988af87cc0 0x55988af94b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:48806/0 (socket says 172.21.15.45:48806) 2024-04-16T17:06:33.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.871+0000 7f338fe00640 1 -- 172.21.15.45:0/1031011015 learned_addr learned my addr 172.21.15.45:0/1031011015 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:06:33.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.871+0000 7f3395a00640 1 --2- 172.21.15.45:0/1031011015 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55988af8ce30 0x55988af97b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:33.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.871+0000 7f3395a00640 1 -- 172.21.15.45:0/1031011015 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55988af87cc0 msgr2=0x55988af94b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:06:33.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.871+0000 7f3395a00640 1 --2- 172.21.15.45:0/1031011015 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55988af87cc0 0x55988af94b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:33.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.871+0000 7f3395a00640 1 -- 172.21.15.45:0/1031011015 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55988af8ce30 msgr2=0x55988af97b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:06:33.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.871+0000 7f3395a00640 1 --2- 172.21.15.45:0/1031011015 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55988af8ce30 0x55988af97b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:33.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.871+0000 7f3395a00640 1 -- 172.21.15.45:0/1031011015 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55988af98180 con 0x55988af872d0 2024-04-16T17:06:33.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.871+0000 7f3395a00640 1 --2- 172.21.15.45:0/1031011015 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55988af872d0 0x55988af94630 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f338802c190 tx=0x7f3388009560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=148f5e41cead5bd2 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:06:33.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.871+0000 7f337aa00640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-04-16T17:06:33.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.872+0000 7f337aa00640 1 -- 172.21.15.45:0/1031011015 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f338802c9a0 con 0x55988af872d0 2024-04-16T17:06:33.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.872+0000 7f337aa00640 1 -- 172.21.15.45:0/1031011015 <== mon.1 v2:172.21.15.117:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f338802cb40 con 0x55988af872d0 2024-04-16T17:06:33.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.872+0000 7f339a7da740 1 -- 172.21.15.45:0/1031011015 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f337c009570 con 0x55988af872d0 2024-04-16T17:06:33.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.872+0000 7f337aa00640 1 -- 172.21.15.45:0/1031011015 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f338802cd40 con 0x55988af872d0 2024-04-16T17:06:33.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.872+0000 7f339a7da740 10 client.25372 fetch_fsmap learned FSMap version 24 2024-04-16T17:06:33.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.872+0000 7f339a7da740 1 -- 172.21.15.45:0/1031011015 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55988b00d180 con 0x55988af872d0 2024-04-16T17:06:33.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.872+0000 7f337aa00640 1 -- 172.21.15.45:0/1031011015 <== mon.1 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f337c009570 con 0x55988af872d0 2024-04-16T17:06:33.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.873+0000 7f337aa00640 1 -- 172.21.15.45:0/1031011015 <== mon.1 v2:172.21.15.117:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f3388005060 con 0x55988af872d0 2024-04-16T17:06:33.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.873+0000 7f339a7da740 10 client.25372 fetch_fsmap finished waiting for FSMap version 24 2024-04-16T17:06:33.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.873+0000 7f339a7da740 10 client.25372 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-16T17:06:33.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.873+0000 7f339a7da740 20 client.25372 populate_metadata read hostname 'smithi045' 2024-04-16T17:06:33.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.873+0000 7f339a7da740 1 --2- 172.21.15.45:0/1031011015 >> [v2:172.21.15.45:6840/3755310822,v1:172.21.15.45:6841/3755310822] conn(0x55988b011950 0x55988b00dd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:33.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.873+0000 7f339a7da740 4 client.25372 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-16T17:06:33.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.873+0000 7f339a7da740 1 -- 172.21.15.45:0/1031011015 --> [v2:172.21.15.45:6840/3755310822,v1:172.21.15.45:6841/3755310822] -- command(tid 0: {"prefix": "status"}) -- 0x55988aef5210 con 0x55988b011950 2024-04-16T17:06:33.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.873+0000 7f338fe00640 1 --2- 172.21.15.45:0/1031011015 >> [v2:172.21.15.45:6840/3755310822,v1:172.21.15.45:6841/3755310822] conn(0x55988b011950 0x55988b00dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:33.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.874+0000 7f338fe00640 1 --2- 172.21.15.45:0/1031011015 >> [v2:172.21.15.45:6840/3755310822,v1:172.21.15.45:6841/3755310822] conn(0x55988b011950 0x55988b00dd90 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2bbbc928bc388f5d server_cookie=7f7a85838bbeef2a in_seq=0 out_seq=0 2024-04-16T17:06:33.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.874+0000 7f337aa00640 10 client.25372 ms_handle_connect on v2:172.21.15.45:6840/3755310822 2024-04-16T17:06:33.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.874+0000 7f337aa00640 1 -- 172.21.15.45:0/1031011015 <== mds.1 v2:172.21.15.45:6840/3755310822 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55988aef5210 con 0x55988b011950 2024-04-16T17:06:33.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.874+0000 7f337aa00640 10 client.25372 handle_command_reply: tid=0 2024-04-16T17:06:33.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.874+0000 7f339a7da740 1 client.25372 shutdown 2024-04-16T17:06:33.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.874+0000 7f339a7da740 1 -- 172.21.15.45:0/1031011015 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55988af872d0 msgr2=0x55988af94630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:33.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.874+0000 7f339a7da740 1 --2- 172.21.15.45:0/1031011015 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55988af872d0 0x55988af94630 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f338802c190 tx=0x7f3388009560 comp rx=0 tx=0).stop 2024-04-16T17:06:33.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.874+0000 7f339a7da740 1 -- 172.21.15.45:0/1031011015 shutdown_connections 2024-04-16T17:06:33.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.874+0000 7f339a7da740 1 --2- 172.21.15.45:0/1031011015 >> [v2:172.21.15.45:6840/3755310822,v1:172.21.15.45:6841/3755310822] conn(0x55988b011950 0x55988b00dd90 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:33.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.874+0000 7f339a7da740 1 --2- 172.21.15.45:0/1031011015 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55988af8ce30 0x55988af97b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:33.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.874+0000 7f339a7da740 1 --2- 172.21.15.45:0/1031011015 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55988af87cc0 0x55988af94b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:33.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.874+0000 7f339a7da740 1 --2- 172.21.15.45:0/1031011015 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55988af872d0 0x55988af94630 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:33.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.874+0000 7f339a7da740 1 -- 172.21.15.45:0/1031011015 >> 172.21.15.45:0/1031011015 conn(0x55988af82840 msgr2=0x55988af89180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:06:33.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.874+0000 7f337aa00640 0 client.25372 ms_handle_reset on v2:172.21.15.45:6840/3755310822 2024-04-16T17:06:33.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.875+0000 7f339a7da740 1 -- 172.21.15.45:0/1031011015 shutdown_connections 2024-04-16T17:06:33.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.875+0000 7f339a7da740 1 -- 172.21.15.45:0/1031011015 wait complete. 2024-04-16T17:06:33.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.875+0000 7f339a7da740 20 client.25372 trim_cache size 0 max 16384 2024-04-16T17:06:33.881 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-16T17:06:33.881 INFO:teuthology.orchestra.run.smithi045.stdout: "cluster_fsid": "b1fecd38-fc11-11ee-bc90-c7b262605968", 2024-04-16T17:06:33.881 INFO:teuthology.orchestra.run.smithi045.stdout: "whoami": 1, 2024-04-16T17:06:33.881 INFO:teuthology.orchestra.run.smithi045.stdout: "id": 24569, 2024-04-16T17:06:33.881 INFO:teuthology.orchestra.run.smithi045.stdout: "want_state": "up:active", 2024-04-16T17:06:33.882 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "up:active", 2024-04-16T17:06:33.882 INFO:teuthology.orchestra.run.smithi045.stdout: "fs_name": "cephfs", 2024-04-16T17:06:33.882 INFO:teuthology.orchestra.run.smithi045.stdout: "rank_uptime": 162.515061745, 2024-04-16T17:06:33.882 INFO:teuthology.orchestra.run.smithi045.stdout: "mdsmap_epoch": 24, 2024-04-16T17:06:33.882 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_epoch": 94, 2024-04-16T17:06:33.882 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_epoch_barrier": 90, 2024-04-16T17:06:33.882 INFO:teuthology.orchestra.run.smithi045.stdout: "uptime": 198.17538664200001 2024-04-16T17:06:33.882 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-16T17:06:33.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.875+0000 7f3397a00640 1 -- 172.21.15.45:0/2606093465 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f33740a3c30 msgr2=0x7f33740a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:33.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.875+0000 7f3397a00640 1 --2- 172.21.15.45:0/2606093465 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f33740a3c30 0x7f33740a60f0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x55988af880c0 tx=0x7f338000efb0 comp rx=0 tx=0).stop 2024-04-16T17:06:33.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.875+0000 7f3397a00640 1 -- 172.21.15.45:0/2606093465 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3390104260 msgr2=0x7f339019a590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:33.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.875+0000 7f3397a00640 1 --2- 172.21.15.45:0/2606093465 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3390104260 0x7f339019a590 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f338000e9c0 tx=0x7f338000ee80 comp rx=0 tx=0).stop 2024-04-16T17:06:33.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.876+0000 7f3397a00640 1 -- 172.21.15.45:0/2606093465 shutdown_connections 2024-04-16T17:06:33.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.876+0000 7f3397a00640 1 --2- 172.21.15.45:0/2606093465 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f33740a3c30 0x7f33740a60f0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:33.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.876+0000 7f3397a00640 1 --2- 172.21.15.45:0/2606093465 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3390105e10 0x7f33901a1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:33.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.876+0000 7f3397a00640 1 --2- 172.21.15.45:0/2606093465 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3390105450 0x7f339019aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:33.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.876+0000 7f3397a00640 1 --2- 172.21.15.45:0/2606093465 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3390104260 0x7f339019a590 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:33.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.876+0000 7f3397a00640 1 -- 172.21.15.45:0/2606093465 >> 172.21.15.45:0/2606093465 conn(0x7f33900ff7b0 msgr2=0x7f3390108750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:06:33.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.876+0000 7f3397a00640 1 -- 172.21.15.45:0/2606093465 shutdown_connections 2024-04-16T17:06:33.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:33.876+0000 7f3397a00640 1 -- 172.21.15.45:0/2606093465 wait complete. 2024-04-16T17:06:33.897 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-16T17:06:34.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.114+0000 7f5c8ac00640 1 -- 172.21.15.45:0/4112539298 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5c84105cf0 msgr2=0x7f5c8410c580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:34.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.114+0000 7f5c8ac00640 1 --2- 172.21.15.45:0/4112539298 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5c84105cf0 0x7f5c8410c580 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f5c7c009ca0 tx=0x7f5c7c02c100 comp rx=0 tx=0).stop 2024-04-16T17:06:34.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.114+0000 7f5c8ac00640 1 -- 172.21.15.45:0/4112539298 shutdown_connections 2024-04-16T17:06:34.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.114+0000 7f5c8ac00640 1 --2- 172.21.15.45:0/4112539298 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5c84105330 0x7f5c841057b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:34.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.114+0000 7f5c8ac00640 1 --2- 172.21.15.45:0/4112539298 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5c84105cf0 0x7f5c8410c580 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:34.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.114+0000 7f5c8ac00640 1 --2- 172.21.15.45:0/4112539298 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5c84104140 0x7f5c84104560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:34.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.114+0000 7f5c8ac00640 1 -- 172.21.15.45:0/4112539298 >> 172.21.15.45:0/4112539298 conn(0x7f5c840ff6b0 msgr2=0x7f5c84101af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:06:34.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.114+0000 7f5c8ac00640 1 -- 172.21.15.45:0/4112539298 shutdown_connections 2024-04-16T17:06:34.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.114+0000 7f5c8ac00640 1 -- 172.21.15.45:0/4112539298 wait complete. 2024-04-16T17:06:34.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.115+0000 7f5c8ac00640 1 Processor -- start 2024-04-16T17:06:34.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.115+0000 7f5c8ac00640 1 -- start start 2024-04-16T17:06:34.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.116+0000 7f5c8ac00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5c84104140 0x7f5c8419a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:34.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.116+0000 7f5c8ac00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5c84105330 0x7f5c8419a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:34.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.116+0000 7f5c8ac00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5c84105cf0 0x7f5c841a1960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:34.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.116+0000 7f5c8ac00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f5c8410fda0 con 0x7f5c84104140 2024-04-16T17:06:34.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.116+0000 7f5c8ac00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f5c8410fc20 con 0x7f5c84105330 2024-04-16T17:06:34.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.116+0000 7f5c8ac00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f5c8410ff20 con 0x7f5c84105cf0 2024-04-16T17:06:34.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.116+0000 7f5c89800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5c84104140 0x7f5c8419a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:34.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.116+0000 7f5c88e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5c84105330 0x7f5c8419a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:34.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.116+0000 7f5c8a200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5c84105cf0 0x7f5c841a1960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:34.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.116+0000 7f5c88e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5c84105330 0x7f5c8419a8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:48816/0 (socket says 172.21.15.45:48816) 2024-04-16T17:06:34.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.116+0000 7f5c89800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5c84104140 0x7f5c8419a3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55648/0 (socket says 172.21.15.45:55648) 2024-04-16T17:06:34.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.116+0000 7f5c88e00640 1 -- 172.21.15.45:0/3334898624 learned_addr learned my addr 172.21.15.45:0/3334898624 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:06:34.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.117+0000 7f5c8a200640 1 -- 172.21.15.45:0/3334898624 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5c84105330 msgr2=0x7f5c8419a8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:34.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.117+0000 7f5c8a200640 1 --2- 172.21.15.45:0/3334898624 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5c84105330 0x7f5c8419a8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:34.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.117+0000 7f5c8a200640 1 -- 172.21.15.45:0/3334898624 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5c84104140 msgr2=0x7f5c8419a3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:34.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.117+0000 7f5c8a200640 1 --2- 172.21.15.45:0/3334898624 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5c84104140 0x7f5c8419a3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:34.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.117+0000 7f5c8a200640 1 -- 172.21.15.45:0/3334898624 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c841a1fd0 con 0x7f5c84105cf0 2024-04-16T17:06:34.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.117+0000 7f5c89800640 1 --2- 172.21.15.45:0/3334898624 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5c84104140 0x7f5c8419a3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:06:34.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.117+0000 7f5c8a200640 1 --2- 172.21.15.45:0/3334898624 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5c84105cf0 0x7f5c841a1960 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f5c7c02c640 tx=0x7f5c7c035d90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:06:34.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.117+0000 7f5c82000640 1 -- 172.21.15.45:0/3334898624 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c7c039040 con 0x7f5c84105cf0 2024-04-16T17:06:34.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.117+0000 7f5c8ac00640 1 -- 172.21.15.45:0/3334898624 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c841a2260 con 0x7f5c84105cf0 2024-04-16T17:06:34.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.118+0000 7f5c82000640 1 -- 172.21.15.45:0/3334898624 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5c7c044070 con 0x7f5c84105cf0 2024-04-16T17:06:34.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.118+0000 7f5c82000640 1 -- 172.21.15.45:0/3334898624 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c7c0043e0 con 0x7f5c84105cf0 2024-04-16T17:06:34.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.118+0000 7f5c8ac00640 1 -- 172.21.15.45:0/3334898624 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c841a2770 con 0x7f5c84105cf0 2024-04-16T17:06:34.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.121+0000 7f5c82000640 1 -- 172.21.15.45:0/3334898624 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f5c7c004580 con 0x7f5c84105cf0 2024-04-16T17:06:34.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.121+0000 7f5c8effa740 1 Processor -- start 2024-04-16T17:06:34.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.121+0000 7f5c8effa740 1 -- start start 2024-04-16T17:06:34.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.121+0000 7f5c8effa740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559a062044a0 0x559a062048c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:34.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.121+0000 7f5c8effa740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559a06204e90 0x559a06209990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:34.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.121+0000 7f5c8effa740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559a0620a000 0x559a0620c490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:34.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.121+0000 7f5c8effa740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x559a0616a980 con 0x559a062044a0 2024-04-16T17:06:34.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.121+0000 7f5c8effa740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x559a06111220 con 0x559a0620a000 2024-04-16T17:06:34.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.121+0000 7f5c8effa740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x559a06166940 con 0x559a06204e90 2024-04-16T17:06:34.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.122+0000 7f5c88e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559a062044a0 0x559a062048c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:34.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.122+0000 7f5c88e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559a062044a0 0x559a062048c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55662/0 (socket says 172.21.15.45:55662) 2024-04-16T17:06:34.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.122+0000 7f5c88e00640 1 -- 172.21.15.45:0/1907290499 learned_addr learned my addr 172.21.15.45:0/1907290499 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:06:34.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.122+0000 7f5c89800640 1 --2- 172.21.15.45:0/1907290499 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559a06204e90 0x559a06209990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:34.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.122+0000 7f5c88e00640 1 --2- 172.21.15.45:0/1907290499 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559a0620a000 0x559a0620c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:34.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.122+0000 7f5c88e00640 1 -- 172.21.15.45:0/1907290499 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559a06204e90 msgr2=0x559a06209990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:06:34.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.122+0000 7f5c88e00640 1 --2- 172.21.15.45:0/1907290499 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559a06204e90 0x559a06209990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:34.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.122+0000 7f5c88e00640 1 -- 172.21.15.45:0/1907290499 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559a0620a000 msgr2=0x559a0620c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:06:34.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.122+0000 7f5c88e00640 1 --2- 172.21.15.45:0/1907290499 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559a0620a000 0x559a0620c490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:34.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.122+0000 7f5c88e00640 1 -- 172.21.15.45:0/1907290499 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559a05e7d2d0 con 0x559a062044a0 2024-04-16T17:06:34.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.123+0000 7f5c82000640 1 --2- 172.21.15.45:0/3334898624 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f5c6c0a3bb0 0x7f5c6c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:34.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.123+0000 7f5c88e00640 1 --2- 172.21.15.45:0/1907290499 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559a062044a0 0x559a062048c0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f5c70013910 tx=0x7f5c700094f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fe37bb9601274f4 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:06:34.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.123+0000 7f5c82000640 1 -- 172.21.15.45:0/3334898624 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f5c7c0f2db0 con 0x7f5c84105cf0 2024-04-16T17:06:34.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.123+0000 7f5c89800640 1 --2- 172.21.15.45:0/3334898624 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f5c6c0a3bb0 0x7f5c6c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:34.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.123+0000 7f5c76a00640 1 -- 172.21.15.45:0/1907290499 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c70013170 con 0x559a062044a0 2024-04-16T17:06:34.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.123+0000 7f5c76a00640 1 -- 172.21.15.45:0/1907290499 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5c7000fca0 con 0x559a062044a0 2024-04-16T17:06:34.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.123+0000 7f5c76a00640 1 -- 172.21.15.45:0/1907290499 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c70010930 con 0x559a062044a0 2024-04-16T17:06:34.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.124+0000 7f5c8effa740 1 -- 172.21.15.45:0/1907290499 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559a062044a0 msgr2=0x559a062048c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:34.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.124+0000 7f5c8effa740 1 --2- 172.21.15.45:0/1907290499 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559a062044a0 0x559a062048c0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f5c70013910 tx=0x7f5c700094f0 comp rx=0 tx=0).stop 2024-04-16T17:06:34.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.124+0000 7f5c89800640 1 --2- 172.21.15.45:0/3334898624 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f5c6c0a3bb0 0x7f5c6c0a6070 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x559a06205290 tx=0x7f5c78005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:06:34.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.124+0000 7f5c8effa740 1 -- 172.21.15.45:0/1907290499 shutdown_connections 2024-04-16T17:06:34.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.124+0000 7f5c8effa740 1 --2- 172.21.15.45:0/1907290499 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559a06204e90 0x559a06209990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:34.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.124+0000 7f5c8effa740 1 --2- 172.21.15.45:0/1907290499 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559a0620a000 0x559a0620c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:34.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.124+0000 7f5c8effa740 1 --2- 172.21.15.45:0/1907290499 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559a062044a0 0x559a062048c0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:34.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.124+0000 7f5c8effa740 1 -- 172.21.15.45:0/1907290499 >> 172.21.15.45:0/1907290499 conn(0x559a061ffa10 msgr2=0x559a061ffe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:06:34.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.124+0000 7f5c8effa740 1 -- 172.21.15.45:0/1907290499 shutdown_connections 2024-04-16T17:06:34.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.124+0000 7f5c8effa740 1 -- 172.21.15.45:0/1907290499 wait complete. 2024-04-16T17:06:34.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.124+0000 7f5c8effa740 1 -- start start 2024-04-16T17:06:34.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.125+0000 7f5c8effa740 1 Processor -- start 2024-04-16T17:06:34.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.125+0000 7f5c8effa740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559a062044a0 0x559a0620ea50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:34.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.125+0000 7f5c8effa740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559a06204e90 0x559a0620ef90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:34.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.125+0000 7f5c8effa740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559a0620a000 0x559a06215fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:34.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.125+0000 7f5c8effa740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x559a0620cfb0 con 0x559a06204e90 2024-04-16T17:06:34.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.125+0000 7f5c8effa740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x559a06124590 con 0x559a062044a0 2024-04-16T17:06:34.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.125+0000 7f5c8effa740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x559a0620d130 con 0x559a0620a000 2024-04-16T17:06:34.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.125+0000 7f5c8effa740 10 client.0 _ll_register_callbacks cb 0x559a0612c050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:06:34.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.125+0000 7f5c8a200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559a0620a000 0x559a06215fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:34.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.125+0000 7f5c89800640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559a062044a0 0x559a0620ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:34.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.125+0000 7f5c8a200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559a0620a000 0x559a06215fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:39476/0 (socket says 172.21.15.45:39476) 2024-04-16T17:06:34.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.125+0000 7f5c8a200640 1 -- 172.21.15.45:0/3232413986 learned_addr learned my addr 172.21.15.45:0/3232413986 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:06:34.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.125+0000 7f5c88e00640 1 --2- 172.21.15.45:0/3232413986 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559a06204e90 0x559a0620ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:34.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.126+0000 7f5c8a200640 1 -- 172.21.15.45:0/3232413986 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559a062044a0 msgr2=0x559a0620ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:06:34.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.126+0000 7f5c8a200640 1 --2- 172.21.15.45:0/3232413986 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559a062044a0 0x559a0620ea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:34.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.126+0000 7f5c8a200640 1 -- 172.21.15.45:0/3232413986 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559a06204e90 msgr2=0x559a0620ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:06:34.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.126+0000 7f5c8a200640 1 --2- 172.21.15.45:0/3232413986 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559a06204e90 0x559a0620ef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:34.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.126+0000 7f5c8a200640 1 -- 172.21.15.45:0/3232413986 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559a05f476b0 con 0x559a0620a000 2024-04-16T17:06:34.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.126+0000 7f5c8a200640 1 --2- 172.21.15.45:0/3232413986 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559a0620a000 0x559a06215fd0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f5c7c0c7440 tx=0x7f5c7c0bb060 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ab820aed1f30a96 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:06:34.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.126+0000 7f5c6b400640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-04-16T17:06:34.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.126+0000 7f5c6b400640 1 -- 172.21.15.45:0/3232413986 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c7c0bf160 con 0x559a0620a000 2024-04-16T17:06:34.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.126+0000 7f5c6b400640 1 -- 172.21.15.45:0/3232413986 <== mon.1 v2:172.21.15.117:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f5c7c0df0f0 con 0x559a0620a000 2024-04-16T17:06:34.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.126+0000 7f5c8effa740 1 -- 172.21.15.45:0/3232413986 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5c7000fca0 con 0x559a0620a000 2024-04-16T17:06:34.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.127+0000 7f5c6b400640 1 -- 172.21.15.45:0/3232413986 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f5c7c032890 con 0x559a0620a000 2024-04-16T17:06:34.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.127+0000 7f5c8effa740 10 client.25384 fetch_fsmap learned FSMap version 24 2024-04-16T17:06:34.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.127+0000 7f5c8effa740 1 -- 172.21.15.45:0/3232413986 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x559a0620d020 con 0x559a0620a000 2024-04-16T17:06:34.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.127+0000 7f5c6b400640 1 -- 172.21.15.45:0/3232413986 <== mon.1 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5c7000fca0 con 0x559a0620a000 2024-04-16T17:06:34.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.127+0000 7f5c6b400640 1 -- 172.21.15.45:0/3232413986 <== mon.1 v2:172.21.15.117:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f5c7c03c030 con 0x559a0620a000 2024-04-16T17:06:34.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.127+0000 7f5c8effa740 10 client.25384 fetch_fsmap finished waiting for FSMap version 24 2024-04-16T17:06:34.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.127+0000 7f5c8effa740 10 client.25384 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-16T17:06:34.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.127+0000 7f5c8effa740 20 client.25384 populate_metadata read hostname 'smithi045' 2024-04-16T17:06:34.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.127+0000 7f5c8effa740 1 --2- 172.21.15.45:0/3232413986 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x559a0621b9b0 0x559a0621ce20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:34.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.127+0000 7f5c8effa740 4 client.25384 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:06:34.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.127+0000 7f5c8effa740 1 -- 172.21.15.45:0/3232413986 --> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559a061783a0 con 0x559a0621b9b0 2024-04-16T17:06:34.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.128+0000 7f5c89800640 1 --2- 172.21.15.45:0/3232413986 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x559a0621b9b0 0x559a0621ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:34.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.128+0000 7f5c89800640 1 --2- 172.21.15.45:0/3232413986 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x559a0621b9b0 0x559a0621ce20 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e4a625369d1561ed server_cookie=eed82abd285ed8c3 in_seq=0 out_seq=0 2024-04-16T17:06:34.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.128+0000 7f5c6b400640 10 client.25384 ms_handle_connect on v2:172.21.15.104:6838/3660070592 2024-04-16T17:06:34.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.129+0000 7f5c6b400640 1 -- 172.21.15.45:0/3232413986 <== mds.0 v2:172.21.15.104:6838/3660070592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559a061783a0 con 0x559a0621b9b0 2024-04-16T17:06:34.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.129+0000 7f5c6b400640 10 client.25384 handle_command_reply: tid=0 2024-04-16T17:06:34.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.129+0000 7f5c8effa740 1 client.25384 shutdown 2024-04-16T17:06:34.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.129+0000 7f5c8effa740 1 -- 172.21.15.45:0/3232413986 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559a0620a000 msgr2=0x559a06215fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:34.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.129+0000 7f5c8effa740 1 --2- 172.21.15.45:0/3232413986 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559a0620a000 0x559a06215fd0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f5c7c0c7440 tx=0x7f5c7c0bb060 comp rx=0 tx=0).stop 2024-04-16T17:06:34.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.130+0000 7f5c8effa740 1 -- 172.21.15.45:0/3232413986 shutdown_connections 2024-04-16T17:06:34.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.130+0000 7f5c8effa740 1 --2- 172.21.15.45:0/3232413986 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x559a0621b9b0 0x559a0621ce20 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:34.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.130+0000 7f5c8effa740 1 --2- 172.21.15.45:0/3232413986 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559a0620a000 0x559a06215fd0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:34.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.130+0000 7f5c8effa740 1 --2- 172.21.15.45:0/3232413986 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559a06204e90 0x559a0620ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:34.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.130+0000 7f5c8effa740 1 --2- 172.21.15.45:0/3232413986 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559a062044a0 0x559a0620ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:34.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.130+0000 7f5c8effa740 1 -- 172.21.15.45:0/3232413986 >> 172.21.15.45:0/3232413986 conn(0x559a061ffa10 msgr2=0x559a061fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:06:34.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.130+0000 7f5c6b400640 0 client.25384 ms_handle_reset on v2:172.21.15.104:6838/3660070592 2024-04-16T17:06:34.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.130+0000 7f5c8effa740 1 -- 172.21.15.45:0/3232413986 shutdown_connections 2024-04-16T17:06:34.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.130+0000 7f5c8effa740 1 -- 172.21.15.45:0/3232413986 wait complete. 2024-04-16T17:06:34.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.130+0000 7f5c8effa740 20 client.25384 trim_cache size 0 max 16384 2024-04-16T17:06:34.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.150+0000 7f5c8effa740 1 Processor -- start 2024-04-16T17:06:34.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.150+0000 7f5c8effa740 1 -- start start 2024-04-16T17:06:34.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.150+0000 7f5c8effa740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559a061ffa10 0x559a0621a570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:34.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.150+0000 7f5c8effa740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559a062044a0 0x559a06215f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:34.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.150+0000 7f5c8effa740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559a06204e90 0x559a0621aab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:34.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.150+0000 7f5c8effa740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x559a0621b160 con 0x559a061ffa10 2024-04-16T17:06:34.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.151+0000 7f5c8effa740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x559a0621b830 con 0x559a06204e90 2024-04-16T17:06:34.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.151+0000 7f5c8effa740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x559a0621bf00 con 0x559a062044a0 2024-04-16T17:06:34.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.151+0000 7f5c89800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559a061ffa10 0x559a0621a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:34.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.151+0000 7f5c89800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559a061ffa10 0x559a0621a570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55682/0 (socket says 172.21.15.45:55682) 2024-04-16T17:06:34.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.151+0000 7f5c89800640 1 -- 172.21.15.45:0/675209773 learned_addr learned my addr 172.21.15.45:0/675209773 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:06:34.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.151+0000 7f5c8a200640 1 --2- 172.21.15.45:0/675209773 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559a06204e90 0x559a0621aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:34.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.151+0000 7f5c88e00640 1 --2- 172.21.15.45:0/675209773 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559a062044a0 0x559a06215f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:34.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.151+0000 7f5c89800640 1 -- 172.21.15.45:0/675209773 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559a062044a0 msgr2=0x559a06215f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:06:34.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.151+0000 7f5c89800640 1 --2- 172.21.15.45:0/675209773 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559a062044a0 0x559a06215f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:34.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.151+0000 7f5c89800640 1 -- 172.21.15.45:0/675209773 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559a06204e90 msgr2=0x559a0621aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:06:34.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.151+0000 7f5c89800640 1 --2- 172.21.15.45:0/675209773 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559a06204e90 0x559a0621aab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:34.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.151+0000 7f5c89800640 1 -- 172.21.15.45:0/675209773 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559a06200780 con 0x559a061ffa10 2024-04-16T17:06:34.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.151+0000 7f5c8a200640 1 --2- 172.21.15.45:0/675209773 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559a06204e90 0x559a0621aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:06:34.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.152+0000 7f5c89800640 1 --2- 172.21.15.45:0/675209773 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559a061ffa10 0x559a0621a570 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x559a06200980 tx=0x7f5c7801b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fefac597879dbd9f server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:06:34.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.152+0000 7f5c76000640 1 -- 172.21.15.45:0/675209773 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c78030070 con 0x559a061ffa10 2024-04-16T17:06:34.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.152+0000 7f5c76000640 1 -- 172.21.15.45:0/675209773 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5c7800f620 con 0x559a061ffa10 2024-04-16T17:06:34.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.152+0000 7f5c76000640 1 -- 172.21.15.45:0/675209773 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c7801f660 con 0x559a061ffa10 2024-04-16T17:06:34.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.153+0000 7f5c8effa740 1 -- 172.21.15.45:0/675209773 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559a061ffa10 msgr2=0x559a0621a570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:34.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.153+0000 7f5c8effa740 1 --2- 172.21.15.45:0/675209773 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559a061ffa10 0x559a0621a570 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x559a06200980 tx=0x7f5c7801b000 comp rx=0 tx=0).stop 2024-04-16T17:06:34.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.153+0000 7f5c8effa740 1 -- 172.21.15.45:0/675209773 shutdown_connections 2024-04-16T17:06:34.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.153+0000 7f5c8effa740 1 --2- 172.21.15.45:0/675209773 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559a062044a0 0x559a06215f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:34.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.153+0000 7f5c8effa740 1 --2- 172.21.15.45:0/675209773 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559a06204e90 0x559a0621aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:34.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.153+0000 7f5c8effa740 1 --2- 172.21.15.45:0/675209773 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559a061ffa10 0x559a0621a570 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:34.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.153+0000 7f5c8effa740 1 -- 172.21.15.45:0/675209773 >> 172.21.15.45:0/675209773 conn(0x559a06200160 msgr2=0x559a0620ea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:06:34.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.153+0000 7f5c8effa740 1 -- 172.21.15.45:0/675209773 shutdown_connections 2024-04-16T17:06:34.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.153+0000 7f5c8effa740 1 -- 172.21.15.45:0/675209773 wait complete. 2024-04-16T17:06:34.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.153+0000 7f5c8effa740 1 -- start start 2024-04-16T17:06:34.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.153+0000 7f5c8effa740 1 Processor -- start 2024-04-16T17:06:34.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.154+0000 7f5c8effa740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559a061ffa10 0x559a062117f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:34.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.154+0000 7f5c8effa740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559a062044a0 0x559a06211d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:34.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.154+0000 7f5c8effa740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559a06204e90 0x559a06214d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:34.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.154+0000 7f5c8effa740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x559a0621cca0 con 0x559a06204e90 2024-04-16T17:06:34.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.154+0000 7f5c8effa740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x559a0621c5d0 con 0x559a062044a0 2024-04-16T17:06:34.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.154+0000 7f5c8effa740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x559a0621d370 con 0x559a061ffa10 2024-04-16T17:06:34.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.154+0000 7f5c8effa740 10 client.0 _ll_register_callbacks cb 0x7f5c7c009c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:06:34.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.154+0000 7f5c8a200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559a06204e90 0x559a06214d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:34.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.154+0000 7f5c8a200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559a06204e90 0x559a06214d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55698/0 (socket says 172.21.15.45:55698) 2024-04-16T17:06:34.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.154+0000 7f5c8a200640 1 -- 172.21.15.45:0/2096301738 learned_addr learned my addr 172.21.15.45:0/2096301738 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:06:34.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.154+0000 7f5c88e00640 1 --2- 172.21.15.45:0/2096301738 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559a062044a0 0x559a06211d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:34.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.154+0000 7f5c89800640 1 --2- 172.21.15.45:0/2096301738 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559a061ffa10 0x559a062117f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:34.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.154+0000 7f5c8a200640 1 -- 172.21.15.45:0/2096301738 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559a061ffa10 msgr2=0x559a062117f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:06:34.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.154+0000 7f5c8a200640 1 --2- 172.21.15.45:0/2096301738 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559a061ffa10 0x559a062117f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:34.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.154+0000 7f5c8a200640 1 -- 172.21.15.45:0/2096301738 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559a062044a0 msgr2=0x559a06211d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:06:34.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.154+0000 7f5c8a200640 1 --2- 172.21.15.45:0/2096301738 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559a062044a0 0x559a06211d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:34.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.155+0000 7f5c8a200640 1 -- 172.21.15.45:0/2096301738 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559a06215340 con 0x559a06204e90 2024-04-16T17:06:34.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.155+0000 7f5c88e00640 1 --2- 172.21.15.45:0/2096301738 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559a062044a0 0x559a06211d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:06:34.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.155+0000 7f5c8a200640 1 --2- 172.21.15.45:0/2096301738 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559a06204e90 0x559a06214d50 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f5c7c0bcfb0 tx=0x7f5c7c02c100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f395c23e8bb3ed8f server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:06:34.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.155+0000 7f5c6b400640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-16T17:06:34.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.155+0000 7f5c6b400640 1 -- 172.21.15.45:0/2096301738 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c7c0be840 con 0x559a06204e90 2024-04-16T17:06:34.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.155+0000 7f5c6b400640 1 -- 172.21.15.45:0/2096301738 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f5c7c09db70 con 0x559a06204e90 2024-04-16T17:06:34.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.155+0000 7f5c8effa740 1 -- 172.21.15.45:0/2096301738 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5c7800f620 con 0x559a06204e90 2024-04-16T17:06:34.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.155+0000 7f5c6b400640 1 -- 172.21.15.45:0/2096301738 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f5c7c0d8260 con 0x559a06204e90 2024-04-16T17:06:34.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.155+0000 7f5c8effa740 10 client.15282 fetch_fsmap learned FSMap version 24 2024-04-16T17:06:34.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.155+0000 7f5c8effa740 1 -- 172.21.15.45:0/2096301738 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x559a06281b50 con 0x559a06204e90 2024-04-16T17:06:34.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.156+0000 7f5c6b400640 1 -- 172.21.15.45:0/2096301738 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5c7800f620 con 0x559a06204e90 2024-04-16T17:06:34.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.156+0000 7f5c6b400640 1 -- 172.21.15.45:0/2096301738 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f5c7c004b40 con 0x559a06204e90 2024-04-16T17:06:34.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.156+0000 7f5c8effa740 10 client.15282 fetch_fsmap finished waiting for FSMap version 24 2024-04-16T17:06:34.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.156+0000 7f5c8effa740 10 client.15282 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-16T17:06:34.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.156+0000 7f5c8effa740 20 client.15282 populate_metadata read hostname 'smithi045' 2024-04-16T17:06:34.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.156+0000 7f5c8effa740 1 --2- 172.21.15.45:0/2096301738 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x559a0620a000 0x559a062851f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:34.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.156+0000 7f5c8effa740 4 client.15282 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-16T17:06:34.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.156+0000 7f5c8effa740 1 -- 172.21.15.45:0/2096301738 --> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] -- command(tid 0: {"prefix": "status"}) -- 0x559a06172210 con 0x559a0620a000 2024-04-16T17:06:34.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.156+0000 7f5c89800640 1 --2- 172.21.15.45:0/2096301738 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x559a0620a000 0x559a062851f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:34.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.157+0000 7f5c89800640 1 --2- 172.21.15.45:0/2096301738 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x559a0620a000 0x559a062851f0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d3b804d2ddacc7a7 server_cookie=67794882f92bb140 in_seq=0 out_seq=0 2024-04-16T17:06:34.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.157+0000 7f5c6b400640 10 client.15282 ms_handle_connect on v2:172.21.15.104:6838/3660070592 2024-04-16T17:06:34.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.157+0000 7f5c6b400640 1 -- 172.21.15.45:0/2096301738 <== mds.0 v2:172.21.15.104:6838/3660070592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x559a06172210 con 0x559a0620a000 2024-04-16T17:06:34.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.157+0000 7f5c6b400640 10 client.15282 handle_command_reply: tid=0 2024-04-16T17:06:34.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.157+0000 7f5c8effa740 1 client.15282 shutdown 2024-04-16T17:06:34.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.157+0000 7f5c8effa740 1 -- 172.21.15.45:0/2096301738 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559a06204e90 msgr2=0x559a06214d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:34.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.157+0000 7f5c8effa740 1 --2- 172.21.15.45:0/2096301738 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559a06204e90 0x559a06214d50 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f5c7c0bcfb0 tx=0x7f5c7c02c100 comp rx=0 tx=0).stop 2024-04-16T17:06:34.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.158+0000 7f5c8effa740 1 -- 172.21.15.45:0/2096301738 shutdown_connections 2024-04-16T17:06:34.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.158+0000 7f5c8effa740 1 --2- 172.21.15.45:0/2096301738 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559a06204e90 0x559a06214d50 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:34.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.158+0000 7f5c8effa740 1 --2- 172.21.15.45:0/2096301738 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559a062044a0 0x559a06211d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:34.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.158+0000 7f5c8effa740 1 --2- 172.21.15.45:0/2096301738 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x559a0620a000 0x559a062851f0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:34.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.158+0000 7f5c8effa740 1 --2- 172.21.15.45:0/2096301738 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559a061ffa10 0x559a062117f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:34.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.158+0000 7f5c8effa740 1 -- 172.21.15.45:0/2096301738 >> 172.21.15.45:0/2096301738 conn(0x559a06200160 msgr2=0x559a0620ea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:06:34.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.158+0000 7f5c6b400640 0 client.15282 ms_handle_reset on v2:172.21.15.104:6838/3660070592 2024-04-16T17:06:34.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.158+0000 7f5c8effa740 1 -- 172.21.15.45:0/2096301738 shutdown_connections 2024-04-16T17:06:34.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.158+0000 7f5c8effa740 1 -- 172.21.15.45:0/2096301738 wait complete. 2024-04-16T17:06:34.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.158+0000 7f5c8effa740 20 client.15282 trim_cache size 0 max 16384 2024-04-16T17:06:34.164 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-16T17:06:34.165 INFO:teuthology.orchestra.run.smithi045.stdout: "cluster_fsid": "b1fecd38-fc11-11ee-bc90-c7b262605968", 2024-04-16T17:06:34.165 INFO:teuthology.orchestra.run.smithi045.stdout: "whoami": 0, 2024-04-16T17:06:34.165 INFO:teuthology.orchestra.run.smithi045.stdout: "id": 24581, 2024-04-16T17:06:34.165 INFO:teuthology.orchestra.run.smithi045.stdout: "want_state": "up:active", 2024-04-16T17:06:34.165 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "up:active", 2024-04-16T17:06:34.165 INFO:teuthology.orchestra.run.smithi045.stdout: "fs_name": "cephfs", 2024-04-16T17:06:34.165 INFO:teuthology.orchestra.run.smithi045.stdout: "rank_uptime": 188.99326086799999, 2024-04-16T17:06:34.165 INFO:teuthology.orchestra.run.smithi045.stdout: "mdsmap_epoch": 24, 2024-04-16T17:06:34.165 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_epoch": 94, 2024-04-16T17:06:34.165 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_epoch_barrier": 90, 2024-04-16T17:06:34.165 INFO:teuthology.orchestra.run.smithi045.stdout: "uptime": 195.65835752199999 2024-04-16T17:06:34.165 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-16T17:06:34.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.158+0000 7f5c8ac00640 1 -- 172.21.15.45:0/3334898624 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f5c6c0a3bb0 msgr2=0x7f5c6c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:34.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.158+0000 7f5c8ac00640 1 --2- 172.21.15.45:0/3334898624 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f5c6c0a3bb0 0x7f5c6c0a6070 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x559a06205290 tx=0x7f5c78005f90 comp rx=0 tx=0).stop 2024-04-16T17:06:34.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.158+0000 7f5c8ac00640 1 -- 172.21.15.45:0/3334898624 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5c84105cf0 msgr2=0x7f5c841a1960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:34.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.158+0000 7f5c8ac00640 1 --2- 172.21.15.45:0/3334898624 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5c84105cf0 0x7f5c841a1960 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f5c7c02c640 tx=0x7f5c7c035d90 comp rx=0 tx=0).stop 2024-04-16T17:06:34.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.159+0000 7f5c8ac00640 1 -- 172.21.15.45:0/3334898624 shutdown_connections 2024-04-16T17:06:34.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.159+0000 7f5c8ac00640 1 --2- 172.21.15.45:0/3334898624 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f5c6c0a3bb0 0x7f5c6c0a6070 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:34.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.159+0000 7f5c8ac00640 1 --2- 172.21.15.45:0/3334898624 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5c84105cf0 0x7f5c841a1960 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:34.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.159+0000 7f5c8ac00640 1 --2- 172.21.15.45:0/3334898624 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5c84105330 0x7f5c8419a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:34.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.159+0000 7f5c8ac00640 1 --2- 172.21.15.45:0/3334898624 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5c84104140 0x7f5c8419a3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:34.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.159+0000 7f5c8ac00640 1 -- 172.21.15.45:0/3334898624 >> 172.21.15.45:0/3334898624 conn(0x7f5c840ff6b0 msgr2=0x7f5c841085d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:06:34.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.159+0000 7f5c8ac00640 1 -- 172.21.15.45:0/3334898624 shutdown_connections 2024-04-16T17:06:34.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.159+0000 7f5c8ac00640 1 -- 172.21.15.45:0/3334898624 wait complete. 2024-04-16T17:06:34.177 INFO:teuthology.run_tasks:Running task sequential... 2024-04-16T17:06:34.188 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-04-16T17:06:34.188 INFO:teuthology.task.sequential:In sequential, running task print... 2024-04-16T17:06:34.188 INFO:teuthology.task.print:Enabling mgr modules 2024-04-16T17:06:34.188 INFO:teuthology.task.sequential:In sequential, running task exec... 2024-04-16T17:06:34.189 INFO:teuthology.task.exec:Executing custom commands... 2024-04-16T17:06:34.189 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi045.front.sepia.ceph.com 2024-04-16T17:06:34.190 DEBUG:teuthology.orchestra.run.smithi045:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph mgr module enable snap_schedule' 2024-04-16T17:06:34.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.395+0000 7f6c16c00640 1 -- 172.21.15.45:0/952103275 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6c101012c0 msgr2=0x7f6c10101740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:34.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.395+0000 7f6c16c00640 1 --2- 172.21.15.45:0/952103275 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6c101012c0 0x7f6c10101740 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f6bfc009860 tx=0x7f6bfc02bb70 comp rx=0 tx=0).stop 2024-04-16T17:06:34.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.395+0000 7f6c16c00640 1 -- 172.21.15.45:0/952103275 shutdown_connections 2024-04-16T17:06:34.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.395+0000 7f6c16c00640 1 --2- 172.21.15.45:0/952103275 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6c10101c80 0x7f6c10108510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:34.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.395+0000 7f6c16c00640 1 --2- 172.21.15.45:0/952103275 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6c101012c0 0x7f6c10101740 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:34.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.395+0000 7f6c16c00640 1 --2- 172.21.15.45:0/952103275 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6c101000d0 0x7f6c101004f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:34.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.395+0000 7f6c16c00640 1 -- 172.21.15.45:0/952103275 >> 172.21.15.45:0/952103275 conn(0x7f6c100fb660 msgr2=0x7f6c100fda80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:06:34.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.395+0000 7f6c16c00640 1 -- 172.21.15.45:0/952103275 shutdown_connections 2024-04-16T17:06:34.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.395+0000 7f6c16c00640 1 -- 172.21.15.45:0/952103275 wait complete. 2024-04-16T17:06:34.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.397+0000 7f6c16c00640 1 Processor -- start 2024-04-16T17:06:34.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.397+0000 7f6c16c00640 1 -- start start 2024-04-16T17:06:34.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.397+0000 7f6c16c00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6c101000d0 0x7f6c1019a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:34.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.397+0000 7f6c16c00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6c101012c0 0x7f6c1019a930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:34.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.397+0000 7f6c16c00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6c10101c80 0x7f6c101a19a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:34.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.397+0000 7f6c16c00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f6c1010ca40 con 0x7f6c101012c0 2024-04-16T17:06:34.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.397+0000 7f6c16c00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f6c1010c8c0 con 0x7f6c101000d0 2024-04-16T17:06:34.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.397+0000 7f6c16c00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f6c1010cbc0 con 0x7f6c10101c80 2024-04-16T17:06:34.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.397+0000 7f6c15800640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6c101000d0 0x7f6c1019a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:34.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.397+0000 7f6c15800640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6c101000d0 0x7f6c1019a3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:48876/0 (socket says 172.21.15.45:48876) 2024-04-16T17:06:34.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.397+0000 7f6c16200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6c10101c80 0x7f6c101a19a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:34.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.398+0000 7f6c15800640 1 -- 172.21.15.45:0/2772072438 learned_addr learned my addr 172.21.15.45:0/2772072438 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:06:34.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.398+0000 7f6c14e00640 1 --2- 172.21.15.45:0/2772072438 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6c101012c0 0x7f6c1019a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:34.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.398+0000 7f6c15800640 1 -- 172.21.15.45:0/2772072438 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6c10101c80 msgr2=0x7f6c101a19a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:34.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.398+0000 7f6c15800640 1 --2- 172.21.15.45:0/2772072438 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6c10101c80 0x7f6c101a19a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:34.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.398+0000 7f6c15800640 1 -- 172.21.15.45:0/2772072438 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6c101012c0 msgr2=0x7f6c1019a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:34.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.398+0000 7f6c15800640 1 --2- 172.21.15.45:0/2772072438 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6c101012c0 0x7f6c1019a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:34.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.398+0000 7f6c15800640 1 -- 172.21.15.45:0/2772072438 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c101a2010 con 0x7f6c101000d0 2024-04-16T17:06:34.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.398+0000 7f6c15800640 1 --2- 172.21.15.45:0/2772072438 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6c101000d0 0x7f6c1019a3f0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f6c0400e9f0 tx=0x7f6c0400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:06:34.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.399+0000 7f6c0e000640 1 -- 172.21.15.45:0/2772072438 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c0400ccc0 con 0x7f6c101000d0 2024-04-16T17:06:34.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.399+0000 7f6c0e000640 1 -- 172.21.15.45:0/2772072438 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c040035f0 con 0x7f6c101000d0 2024-04-16T17:06:34.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.399+0000 7f6c16c00640 1 -- 172.21.15.45:0/2772072438 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c101a2300 con 0x7f6c101000d0 2024-04-16T17:06:34.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.399+0000 7f6c0e000640 1 -- 172.21.15.45:0/2772072438 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c0400ce60 con 0x7f6c101000d0 2024-04-16T17:06:34.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.399+0000 7f6c16c00640 1 -- 172.21.15.45:0/2772072438 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c101a2820 con 0x7f6c101000d0 2024-04-16T17:06:34.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.400+0000 7f6c16c00640 1 -- 172.21.15.45:0/2772072438 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6c101a2ae0 con 0x7f6c101000d0 2024-04-16T17:06:34.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.402+0000 7f6c0e000640 1 -- 172.21.15.45:0/2772072438 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f6c040037a0 con 0x7f6c101000d0 2024-04-16T17:06:34.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.403+0000 7f6c0e000640 1 --2- 172.21.15.45:0/2772072438 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f6bf80a3ba0 0x7f6bf80a6060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:34.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.403+0000 7f6c0e000640 1 -- 172.21.15.45:0/2772072438 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f6c040d0a20 con 0x7f6c101000d0 2024-04-16T17:06:34.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.404+0000 7f6c14e00640 1 --2- 172.21.15.45:0/2772072438 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f6bf80a3ba0 0x7f6bf80a6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:34.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.404+0000 7f6c14e00640 1 --2- 172.21.15.45:0/2772072438 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f6bf80a3ba0 0x7f6bf80a6060 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f6bfc009830 tx=0x7f6bfc034ab0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:06:34.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.408+0000 7f6c0e000640 1 -- 172.21.15.45:0/2772072438 <== mon.2 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6c0409d010 con 0x7f6c101000d0 2024-04-16T17:06:34.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:34.670+0000 7f6c16c00640 1 -- 172.21.15.45:0/2772072438 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) -- 0x7f6c101a2cb0 con 0x7f6c101000d0 2024-04-16T17:06:35.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:35.367+0000 7f6c0e000640 1 -- 172.21.15.45:0/2772072438 <== mon.2 v2:172.21.15.104:3300/0 7 ==== mgrmap(e 19) ==== 138663+0+0 (secure 0 0 0) 0x7f6c04098f80 con 0x7f6c101000d0 2024-04-16T17:06:35.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:35.368+0000 7f6c0e000640 1 -- 172.21.15.45:0/2772072438 <== mon.2 v2:172.21.15.104:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v19) ==== 92+0+0 (secure 0 0 0) 0x7f6c04078320 con 0x7f6c101000d0 2024-04-16T17:06:35.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:35.376+0000 7f6c16c00640 1 -- 172.21.15.45:0/2772072438 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f6bf80a3ba0 msgr2=0x7f6bf80a6060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:35.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:35.376+0000 7f6c16c00640 1 --2- 172.21.15.45:0/2772072438 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f6bf80a3ba0 0x7f6bf80a6060 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f6bfc009830 tx=0x7f6bfc034ab0 comp rx=0 tx=0).stop 2024-04-16T17:06:35.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:35.376+0000 7f6c16c00640 1 -- 172.21.15.45:0/2772072438 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6c101000d0 msgr2=0x7f6c1019a3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:35.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:35.376+0000 7f6c16c00640 1 --2- 172.21.15.45:0/2772072438 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6c101000d0 0x7f6c1019a3f0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f6c0400e9f0 tx=0x7f6c0400eeb0 comp rx=0 tx=0).stop 2024-04-16T17:06:35.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:35.376+0000 7f6c16c00640 1 -- 172.21.15.45:0/2772072438 shutdown_connections 2024-04-16T17:06:35.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:35.376+0000 7f6c16c00640 1 --2- 172.21.15.45:0/2772072438 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f6bf80a3ba0 0x7f6bf80a6060 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:35.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:35.376+0000 7f6c16c00640 1 --2- 172.21.15.45:0/2772072438 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6c10101c80 0x7f6c101a19a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:35.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:35.376+0000 7f6c16c00640 1 --2- 172.21.15.45:0/2772072438 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6c101012c0 0x7f6c1019a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:35.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:35.376+0000 7f6c16c00640 1 --2- 172.21.15.45:0/2772072438 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6c101000d0 0x7f6c1019a3f0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:35.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:35.376+0000 7f6c16c00640 1 -- 172.21.15.45:0/2772072438 >> 172.21.15.45:0/2772072438 conn(0x7f6c100fb660 msgr2=0x7f6c100fd120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:06:35.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:35.376+0000 7f6c16c00640 1 -- 172.21.15.45:0/2772072438 shutdown_connections 2024-04-16T17:06:35.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:35.376+0000 7f6c16c00640 1 -- 172.21.15.45:0/2772072438 wait complete. 2024-04-16T17:06:35.401 DEBUG:teuthology.orchestra.run.smithi045:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/allow_m_granularity true' 2024-04-16T17:06:35.607 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:06:35 smithi117 ceph-mon[36730]: pgmap v421: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T17:06:35.607 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:06:35 smithi117 ceph-mon[36730]: from='client.? 172.21.15.45:0/2772072438' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-04-16T17:06:35.607 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:06:35 smithi117 ceph-mon[36730]: from='client.? ' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-04-16T17:06:35.607 INFO:journalctl@ceph.mgr.z.smithi117.stdout:Apr 16 17:06:35 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-z[38592]: ignoring --setuser ceph since I am not root 2024-04-16T17:06:35.607 INFO:journalctl@ceph.mgr.z.smithi117.stdout:Apr 16 17:06:35 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-z[38592]: ignoring --setgroup ceph since I am not root 2024-04-16T17:06:35.607 INFO:journalctl@ceph.mgr.z.smithi117.stdout:Apr 16 17:06:35 smithi117 ceph-mgr[38596]: -- 172.21.15.117:0/2533386197 <== mon.1 v2:172.21.15.117:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5556072a85a0 con 0x55560727c800 2024-04-16T17:06:35.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:35.610+0000 7f6e34200640 1 -- 172.21.15.45:0/3280952306 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6e2c10a240 msgr2=0x7f6e2c110ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:35.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:35.610+0000 7f6e34200640 1 --2- 172.21.15.45:0/3280952306 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6e2c10a240 0x7f6e2c110ad0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f6e24009ca0 tx=0x7f6e2402c0f0 comp rx=0 tx=0).stop 2024-04-16T17:06:35.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:35.610+0000 7f6e34200640 1 -- 172.21.15.45:0/3280952306 shutdown_connections 2024-04-16T17:06:35.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:35.610+0000 7f6e34200640 1 --2- 172.21.15.45:0/3280952306 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6e2c10a240 0x7f6e2c110ad0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:35.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:35.610+0000 7f6e34200640 1 --2- 172.21.15.45:0/3280952306 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6e2c109880 0x7f6e2c109d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:35.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:35.610+0000 7f6e34200640 1 --2- 172.21.15.45:0/3280952306 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6e2c108690 0x7f6e2c108ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:35.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:35.610+0000 7f6e34200640 1 -- 172.21.15.45:0/3280952306 >> 172.21.15.45:0/3280952306 conn(0x7f6e2c077650 msgr2=0x7f6e2c079a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:06:35.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:35.611+0000 7f6e34200640 1 -- 172.21.15.45:0/3280952306 shutdown_connections 2024-04-16T17:06:35.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:35.611+0000 7f6e34200640 1 -- 172.21.15.45:0/3280952306 wait complete. 2024-04-16T17:06:35.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:35.611+0000 7f6e34200640 1 Processor -- start 2024-04-16T17:06:35.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:35.612+0000 7f6e34200640 1 -- start start 2024-04-16T17:06:35.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:35.612+0000 7f6e34200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6e2c108690 0x7f6e2c19e850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:35.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:35.612+0000 7f6e34200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6e2c109880 0x7f6e2c19ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:35.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:35.612+0000 7f6e34200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6e2c10a240 0x7f6e2c1a5e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:35.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:35.612+0000 7f6e34200640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f6e2c1142f0 con 0x7f6e2c109880 2024-04-16T17:06:35.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:35.612+0000 7f6e34200640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f6e2c114170 con 0x7f6e2c108690 2024-04-16T17:06:35.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:35.612+0000 7f6e34200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f6e2c114470 con 0x7f6e2c10a240 2024-04-16T17:06:35.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:35.612+0000 7f6e31800640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6e2c108690 0x7f6e2c19e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:35.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:35.612+0000 7f6e32200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6e2c10a240 0x7f6e2c1a5e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:35.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:35.612+0000 7f6e31800640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6e2c108690 0x7f6e2c19e850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:48906/0 (socket says 172.21.15.45:48906) 2024-04-16T17:06:35.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:35.612+0000 7f6e31800640 1 -- 172.21.15.45:0/2131227188 learned_addr learned my addr 172.21.15.45:0/2131227188 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:06:35.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:35.612+0000 7f6e30e00640 1 --2- 172.21.15.45:0/2131227188 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6e2c109880 0x7f6e2c19ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:35.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:35.613+0000 7f6e32200640 1 -- 172.21.15.45:0/2131227188 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6e2c108690 msgr2=0x7f6e2c19e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:35.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:35.613+0000 7f6e32200640 1 --2- 172.21.15.45:0/2131227188 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6e2c108690 0x7f6e2c19e850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:35.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:35.613+0000 7f6e32200640 1 -- 172.21.15.45:0/2131227188 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6e2c109880 msgr2=0x7f6e2c19ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:35.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:35.613+0000 7f6e32200640 1 --2- 172.21.15.45:0/2131227188 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6e2c109880 0x7f6e2c19ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:35.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:35.613+0000 7f6e32200640 1 -- 172.21.15.45:0/2131227188 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e2c1a6470 con 0x7f6e2c10a240 2024-04-16T17:06:35.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:35.613+0000 7f6e30e00640 1 --2- 172.21.15.45:0/2131227188 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6e2c109880 0x7f6e2c19ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:06:35.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:35.613+0000 7f6e32200640 1 --2- 172.21.15.45:0/2131227188 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6e2c10a240 0x7f6e2c1a5e00 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f6e24037040 tx=0x7f6e24032ea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:06:35.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:35.613+0000 7f6e2a000640 1 -- 172.21.15.45:0/2131227188 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e24041020 con 0x7f6e2c10a240 2024-04-16T17:06:35.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:35.613+0000 7f6e2a000640 1 -- 172.21.15.45:0/2131227188 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6e24034cd0 con 0x7f6e2c10a240 2024-04-16T17:06:35.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:35.613+0000 7f6e2a000640 1 -- 172.21.15.45:0/2131227188 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e24039040 con 0x7f6e2c10a240 2024-04-16T17:06:35.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:35.613+0000 7f6e34200640 1 -- 172.21.15.45:0/2131227188 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e2c1a6700 con 0x7f6e2c10a240 2024-04-16T17:06:35.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:35.614+0000 7f6e34200640 1 -- 172.21.15.45:0/2131227188 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e2c1a6b70 con 0x7f6e2c10a240 2024-04-16T17:06:35.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:35.615+0000 7f6e34200640 1 -- 172.21.15.45:0/2131227188 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e2c10f950 con 0x7f6e2c10a240 2024-04-16T17:06:35.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:35.616+0000 7f6e2a000640 1 -- 172.21.15.45:0/2131227188 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 19) ==== 138663+0+0 (secure 0 0 0) 0x7f6e24044020 con 0x7f6e2c10a240 2024-04-16T17:06:35.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:35.618+0000 7f6e2a000640 1 --2- 172.21.15.45:0/2131227188 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f6e140a3d20 0x7f6e140a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:35.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:35.618+0000 7f6e31800640 1 -- 172.21.15.45:0/2131227188 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f6e140a3d20 msgr2=0x7f6e140a61e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.45:6800/268030699 2024-04-16T17:06:35.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:35.618+0000 7f6e2a000640 1 -- 172.21.15.45:0/2131227188 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f6e240f26a0 con 0x7f6e2c10a240 2024-04-16T17:06:35.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:35.618+0000 7f6e31800640 1 --2- 172.21.15.45:0/2131227188 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f6e140a3d20 0x7f6e140a61e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-16T17:06:35.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:35.622+0000 7f6e2a000640 1 -- 172.21.15.45:0/2131227188 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6e2404b020 con 0x7f6e2c10a240 2024-04-16T17:06:35.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:06:35 smithi104 ceph-mon[36457]: pgmap v421: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T17:06:35.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:06:35 smithi104 ceph-mon[36457]: from='client.? 172.21.15.45:0/2772072438' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-04-16T17:06:35.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:06:35 smithi104 ceph-mon[36457]: from='client.? ' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-04-16T17:06:35.678 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Apr 16 17:06:35 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-y[37841]: ignoring --setuser ceph since I am not root 2024-04-16T17:06:35.678 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Apr 16 17:06:35 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-y[37841]: ignoring --setgroup ceph since I am not root 2024-04-16T17:06:35.678 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Apr 16 17:06:35 smithi104 ceph-mgr[37845]: -- 172.21.15.104:0/3836294961 <== mon.2 v2:172.21.15.104:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55f3126305a0 con 0x55f312604c00 2024-04-16T17:06:35.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:06:35 smithi045 ceph-mon[31676]: pgmap v421: 129 pgs: 129 active+clean; 643 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T17:06:35.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:06:35 smithi045 ceph-mon[31676]: from='client.? 172.21.15.45:0/2772072438' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-04-16T17:06:35.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:06:35 smithi045 ceph-mon[31676]: from='client.? ' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-04-16T17:06:35.728 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 16 17:06:35 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-x[31905]: ignoring --setuser ceph since I am not root 2024-04-16T17:06:35.728 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 16 17:06:35 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-x[31905]: ignoring --setgroup ceph since I am not root 2024-04-16T17:06:35.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:35.819+0000 7f6e31800640 1 -- 172.21.15.45:0/2131227188 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f6e140a3d20 msgr2=0x7f6e140a61e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.45:6800/268030699 2024-04-16T17:06:35.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:35.819+0000 7f6e31800640 1 --2- 172.21.15.45:0/2131227188 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f6e140a3d20 0x7f6e140a61e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-16T17:06:35.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:35.832+0000 7f6e34200640 1 -- 172.21.15.45:0/2131227188 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) -- 0x7f6e2c062c00 con 0x7f6e2c10a240 2024-04-16T17:06:35.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:35.838+0000 7f6e2a000640 1 -- 172.21.15.45:0/2131227188 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v31)=0 v31) ==== 138+0+0 (secure 0 0 0) 0x7f6e240bec90 con 0x7f6e2c10a240 2024-04-16T17:06:35.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:35.841+0000 7f6e34200640 1 -- 172.21.15.45:0/2131227188 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f6e140a3d20 msgr2=0x7f6e140a61e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-16T17:06:35.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:35.841+0000 7f6e34200640 1 --2- 172.21.15.45:0/2131227188 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f6e140a3d20 0x7f6e140a61e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:35.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:35.841+0000 7f6e34200640 1 -- 172.21.15.45:0/2131227188 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6e2c10a240 msgr2=0x7f6e2c1a5e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:35.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:35.841+0000 7f6e34200640 1 --2- 172.21.15.45:0/2131227188 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6e2c10a240 0x7f6e2c1a5e00 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f6e24037040 tx=0x7f6e24032ea0 comp rx=0 tx=0).stop 2024-04-16T17:06:35.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:35.841+0000 7f6e34200640 1 -- 172.21.15.45:0/2131227188 shutdown_connections 2024-04-16T17:06:35.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:35.841+0000 7f6e34200640 1 --2- 172.21.15.45:0/2131227188 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f6e140a3d20 0x7f6e140a61e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:35.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:35.841+0000 7f6e34200640 1 --2- 172.21.15.45:0/2131227188 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6e2c10a240 0x7f6e2c1a5e00 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:35.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:35.841+0000 7f6e34200640 1 --2- 172.21.15.45:0/2131227188 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6e2c109880 0x7f6e2c19ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:35.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:35.841+0000 7f6e34200640 1 --2- 172.21.15.45:0/2131227188 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6e2c108690 0x7f6e2c19e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:35.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:35.841+0000 7f6e34200640 1 -- 172.21.15.45:0/2131227188 >> 172.21.15.45:0/2131227188 conn(0x7f6e2c077650 msgr2=0x7f6e2c10cb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:06:35.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:35.841+0000 7f6e34200640 1 -- 172.21.15.45:0/2131227188 shutdown_connections 2024-04-16T17:06:35.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:35.841+0000 7f6e34200640 1 -- 172.21.15.45:0/2131227188 wait complete. 2024-04-16T17:06:35.862 DEBUG:teuthology.orchestra.run.smithi045:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/dump_on_update true' 2024-04-16T17:06:36.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:36.070+0000 7f44fb200640 1 -- 172.21.15.45:0/3714754081 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f44fc076040 msgr2=0x7f44fc074490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:36.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:36.070+0000 7f44fb200640 1 --2- 172.21.15.45:0/3714754081 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f44fc076040 0x7f44fc074490 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f44ec009900 tx=0x7f44ec02bc50 comp rx=0 tx=0).stop 2024-04-16T17:06:36.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:36.070+0000 7f44fb200640 1 -- 172.21.15.45:0/3714754081 shutdown_connections 2024-04-16T17:06:36.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:36.070+0000 7f44fb200640 1 --2- 172.21.15.45:0/3714754081 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f44fc10e460 0x7f44fc110850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:36.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:36.070+0000 7f44fb200640 1 --2- 172.21.15.45:0/3714754081 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f44fc074a60 0x7f44fc074ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:36.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:36.070+0000 7f44fb200640 1 --2- 172.21.15.45:0/3714754081 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f44fc076040 0x7f44fc074490 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:36.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:36.070+0000 7f44fb200640 1 -- 172.21.15.45:0/3714754081 >> 172.21.15.45:0/3714754081 conn(0x7f44fc0fd400 msgr2=0x7f44fc0ff840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:06:36.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:36.070+0000 7f44fb200640 1 -- 172.21.15.45:0/3714754081 shutdown_connections 2024-04-16T17:06:36.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:36.070+0000 7f44fb200640 1 -- 172.21.15.45:0/3714754081 wait complete. 2024-04-16T17:06:36.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:36.071+0000 7f44fb200640 1 Processor -- start 2024-04-16T17:06:36.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:36.072+0000 7f44fb200640 1 -- start start 2024-04-16T17:06:36.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:36.072+0000 7f44fb200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f44fc074a60 0x7f44fc19e820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:36.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:36.072+0000 7f44fb200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f44fc076040 0x7f44fc19ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:36.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:36.072+0000 7f44fb200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f44fc10e460 0x7f44fc1a5d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:36.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:36.072+0000 7f44fb200640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f44fc114170 con 0x7f44fc076040 2024-04-16T17:06:36.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:36.072+0000 7f44fb200640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f44fc113ff0 con 0x7f44fc10e460 2024-04-16T17:06:36.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:36.072+0000 7f44fb200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f44fc1142f0 con 0x7f44fc074a60 2024-04-16T17:06:36.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:36.072+0000 7f44f9e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f44fc074a60 0x7f44fc19e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:36.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:36.072+0000 7f44fa800640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f44fc10e460 0x7f44fc1a5d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:36.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:36.072+0000 7f44f9e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f44fc074a60 0x7f44fc19e820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:39540/0 (socket says 172.21.15.45:39540) 2024-04-16T17:06:36.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:36.072+0000 7f44f9e00640 1 -- 172.21.15.45:0/2490295920 learned_addr learned my addr 172.21.15.45:0/2490295920 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:06:36.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:36.072+0000 7f44f3e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f44fc076040 0x7f44fc19ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:36.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:36.073+0000 7f44fa800640 1 -- 172.21.15.45:0/2490295920 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f44fc074a60 msgr2=0x7f44fc19e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:36.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:36.073+0000 7f44fa800640 1 --2- 172.21.15.45:0/2490295920 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f44fc074a60 0x7f44fc19e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:36.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:36.073+0000 7f44fa800640 1 -- 172.21.15.45:0/2490295920 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f44fc076040 msgr2=0x7f44fc19ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:36.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:36.073+0000 7f44fa800640 1 --2- 172.21.15.45:0/2490295920 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f44fc076040 0x7f44fc19ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:36.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:36.073+0000 7f44fa800640 1 -- 172.21.15.45:0/2490295920 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44fc1a63f0 con 0x7f44fc10e460 2024-04-16T17:06:36.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:36.073+0000 7f44f9e00640 1 --2- 172.21.15.45:0/2490295920 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f44fc074a60 0x7f44fc19e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:06:36.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:36.073+0000 7f44fa800640 1 --2- 172.21.15.45:0/2490295920 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f44fc10e460 0x7f44fc1a5d80 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f44f4009fd0 tx=0x7f44f400ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:06:36.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:36.073+0000 7f44f2a00640 1 -- 172.21.15.45:0/2490295920 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44f400ce20 con 0x7f44fc10e460 2024-04-16T17:06:36.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:36.073+0000 7f44f2a00640 1 -- 172.21.15.45:0/2490295920 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f44f4005020 con 0x7f44fc10e460 2024-04-16T17:06:36.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:36.073+0000 7f44fb200640 1 -- 172.21.15.45:0/2490295920 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44fc1a66e0 con 0x7f44fc10e460 2024-04-16T17:06:36.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:36.073+0000 7f44f2a00640 1 -- 172.21.15.45:0/2490295920 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44f4010720 con 0x7f44fc10e460 2024-04-16T17:06:36.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:36.073+0000 7f44fb200640 1 -- 172.21.15.45:0/2490295920 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44fc1a6c30 con 0x7f44fc10e460 2024-04-16T17:06:36.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:36.075+0000 7f44fb200640 1 -- 172.21.15.45:0/2490295920 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44fc104630 con 0x7f44fc10e460 2024-04-16T17:06:36.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:36.076+0000 7f44f2a00640 1 -- 172.21.15.45:0/2490295920 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 19) ==== 138663+0+0 (secure 0 0 0) 0x7f44f4005540 con 0x7f44fc10e460 2024-04-16T17:06:36.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:36.077+0000 7f44f2a00640 1 --2- 172.21.15.45:0/2490295920 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f44e00a3c70 0x7f44e00a6130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:36.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:36.078+0000 7f44f2a00640 1 -- 172.21.15.45:0/2490295920 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f44f40d0ed0 con 0x7f44fc10e460 2024-04-16T17:06:36.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:36.078+0000 7f44f9e00640 1 -- 172.21.15.45:0/2490295920 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f44e00a3c70 msgr2=0x7f44e00a6130 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.45:6800/268030699 2024-04-16T17:06:36.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:36.078+0000 7f44f9e00640 1 --2- 172.21.15.45:0/2490295920 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f44e00a3c70 0x7f44e00a6130 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-16T17:06:36.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:36.082+0000 7f44f2a00640 1 -- 172.21.15.45:0/2490295920 <== mon.2 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f44f409d4c0 con 0x7f44fc10e460 2024-04-16T17:06:36.178 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Apr 16 17:06:35 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-y[37841]: 2024-04-16T17:06:35.915+0000 7ff4cc180200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-16T17:06:36.178 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Apr 16 17:06:36 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-y[37841]: 2024-04-16T17:06:36.007+0000 7ff4cc180200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-16T17:06:36.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:36.279+0000 7f44f9e00640 1 -- 172.21.15.45:0/2490295920 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f44e00a3c70 msgr2=0x7f44e00a6130 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.45:6800/268030699 2024-04-16T17:06:36.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:36.279+0000 7f44f9e00640 1 --2- 172.21.15.45:0/2490295920 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f44e00a3c70 0x7f44e00a6130 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-16T17:06:36.298 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:36.297+0000 7f44fb200640 1 -- 172.21.15.45:0/2490295920 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) -- 0x7f44fc062c70 con 0x7f44fc10e460 2024-04-16T17:06:36.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:36.304+0000 7f44f2a00640 1 -- 172.21.15.45:0/2490295920 <== mon.2 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v32)=0 v32) ==== 133+0+0 (secure 0 0 0) 0x7f44f409cdc0 con 0x7f44fc10e460 2024-04-16T17:06:36.305 INFO:journalctl@ceph.mgr.z.smithi117.stdout:Apr 16 17:06:35 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-z[38592]: 2024-04-16T17:06:35.919+0000 7f408e266200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-16T17:06:36.305 INFO:journalctl@ceph.mgr.z.smithi117.stdout:Apr 16 17:06:36 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-z[38592]: 2024-04-16T17:06:36.012+0000 7f408e266200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-16T17:06:36.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:36.307+0000 7f44fb200640 1 -- 172.21.15.45:0/2490295920 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f44e00a3c70 msgr2=0x7f44e00a6130 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-16T17:06:36.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:36.307+0000 7f44fb200640 1 --2- 172.21.15.45:0/2490295920 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f44e00a3c70 0x7f44e00a6130 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:36.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:36.307+0000 7f44fb200640 1 -- 172.21.15.45:0/2490295920 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f44fc10e460 msgr2=0x7f44fc1a5d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:36.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:36.307+0000 7f44fb200640 1 --2- 172.21.15.45:0/2490295920 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f44fc10e460 0x7f44fc1a5d80 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f44f4009fd0 tx=0x7f44f400ef10 comp rx=0 tx=0).stop 2024-04-16T17:06:36.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:36.307+0000 7f44fb200640 1 -- 172.21.15.45:0/2490295920 shutdown_connections 2024-04-16T17:06:36.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:36.307+0000 7f44fb200640 1 --2- 172.21.15.45:0/2490295920 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f44e00a3c70 0x7f44e00a6130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:36.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:36.308+0000 7f44fb200640 1 --2- 172.21.15.45:0/2490295920 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f44fc10e460 0x7f44fc1a5d80 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:36.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:36.308+0000 7f44fb200640 1 --2- 172.21.15.45:0/2490295920 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f44fc076040 0x7f44fc19ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:36.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:36.308+0000 7f44fb200640 1 --2- 172.21.15.45:0/2490295920 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f44fc074a60 0x7f44fc19e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:36.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:36.308+0000 7f44fb200640 1 -- 172.21.15.45:0/2490295920 >> 172.21.15.45:0/2490295920 conn(0x7f44fc0fd400 msgr2=0x7f44fc101800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:06:36.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:36.308+0000 7f44fb200640 1 -- 172.21.15.45:0/2490295920 shutdown_connections 2024-04-16T17:06:36.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:36.308+0000 7f44fb200640 1 -- 172.21.15.45:0/2490295920 wait complete. 2024-04-16T17:06:36.327 INFO:teuthology.run_tasks:Running task kclient... 2024-04-16T17:06:36.341 INFO:tasks.kclient:Mounting kernel clients... 2024-04-16T17:06:36.341 INFO:tasks.kclient:config is {'client.0': {}} 2024-04-16T17:06:36.341 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T17:06:36.341 DEBUG:teuthology.orchestra.run.smithi045:> ip netns list 2024-04-16T17:06:36.387 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2024-04-16T17:06:36.388 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T17:06:36.389 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-16T17:06:36.523 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T17:06:36.523 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip link delete ceph-brx 2024-04-16T17:06:36.547 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:06:36 smithi045 ceph-mon[31676]: from='client.? ' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-04-16T17:06:36.547 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:06:36 smithi045 ceph-mon[31676]: mgrmap e19: x(active, since 11m), standbys: z, y 2024-04-16T17:06:36.547 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:06:36 smithi045 ceph-mon[31676]: from='client.? ' entity='client.admin' 2024-04-16T17:06:36.547 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:06:36 smithi045 ceph-mon[31676]: from='client.? ' entity='client.admin' 2024-04-16T17:06:36.548 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 16 17:06:36 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-x[31905]: 2024-04-16T17:06:36.295+0000 7f6d5eb50200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-16T17:06:36.548 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 16 17:06:36 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-x[31905]: 2024-04-16T17:06:36.428+0000 7f6d5eb50200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-16T17:06:36.621 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:06:36 smithi117 ceph-mon[36730]: from='client.? ' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-04-16T17:06:36.621 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:06:36 smithi117 ceph-mon[36730]: mgrmap e19: x(active, since 11m), standbys: z, y 2024-04-16T17:06:36.621 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:06:36 smithi117 ceph-mon[36730]: from='client.? ' entity='client.admin' 2024-04-16T17:06:36.621 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:06:36 smithi117 ceph-mon[36730]: from='client.? ' entity='client.admin' 2024-04-16T17:06:36.631 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=crc', 'wsync'], 'syntax': 'v2', 'mount_subvol_num': 1} 2024-04-16T17:06:36.631 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-16T17:06:36.631 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-16T17:06:36.631 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45 shell --fsid b1fecd38-fc11-11ee-bc90-c7b262605968 -- ceph fs ls 2024-04-16T17:06:36.649 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:06:36 smithi104 ceph-mon[36457]: from='client.? ' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-04-16T17:06:36.649 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:06:36 smithi104 ceph-mon[36457]: mgrmap e19: x(active, since 11m), standbys: z, y 2024-04-16T17:06:36.649 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:06:36 smithi104 ceph-mon[36457]: from='client.? ' entity='client.admin' 2024-04-16T17:06:36.650 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:06:36 smithi104 ceph-mon[36457]: from='client.? ' entity='client.admin' 2024-04-16T17:06:36.883 INFO:journalctl@ceph.mgr.z.smithi117.stdout:Apr 16 17:06:36 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-z[38592]: 2024-04-16T17:06:36.650+0000 7f408e266200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-16T17:06:36.883 INFO:journalctl@ceph.mgr.z.smithi117.stdout:Apr 16 17:06:36 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-z[38592]: 2024-04-16T17:06:36.746+0000 7f408e266200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-16T17:06:36.930 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Apr 16 17:06:36 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-y[37841]: 2024-04-16T17:06:36.648+0000 7ff4cc180200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-16T17:06:36.930 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Apr 16 17:06:36 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-y[37841]: 2024-04-16T17:06:36.741+0000 7ff4cc180200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-16T17:06:36.930 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Apr 16 17:06:36 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-y[37841]: 2024-04-16T17:06:36.882+0000 7ff4cc180200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-16T17:06:37.089 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/mon.a/config 2024-04-16T17:06:37.306 INFO:journalctl@ceph.mgr.z.smithi117.stdout:Apr 16 17:06:36 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-z[38592]: 2024-04-16T17:06:36.893+0000 7f408e266200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-16T17:06:37.306 INFO:journalctl@ceph.mgr.z.smithi117.stdout:Apr 16 17:06:36 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-z[38592]: 2024-04-16T17:06:36.995+0000 7f408e266200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-16T17:06:37.306 INFO:journalctl@ceph.mgr.z.smithi117.stdout:Apr 16 17:06:37 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-z[38592]: 2024-04-16T17:06:37.082+0000 7f408e266200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-16T17:06:37.326 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Apr 16 17:06:36 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-y[37841]: 2024-04-16T17:06:36.979+0000 7ff4cc180200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-16T17:06:37.326 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Apr 16 17:06:37 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-y[37841]: 2024-04-16T17:06:37.063+0000 7ff4cc180200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-16T17:06:37.566 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 16 17:06:37 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-x[31905]: 2024-04-16T17:06:37.295+0000 7f6d5eb50200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-16T17:06:37.566 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 16 17:06:37 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-x[31905]: 2024-04-16T17:06:37.426+0000 7f6d5eb50200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-16T17:06:37.613 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Apr 16 17:06:37 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-y[37841]: 2024-04-16T17:06:37.325+0000 7ff4cc180200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-16T17:06:37.613 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Apr 16 17:06:37 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-y[37841]: 2024-04-16T17:06:37.433+0000 7ff4cc180200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-16T17:06:37.613 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Apr 16 17:06:37 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-y[37841]: 2024-04-16T17:06:37.518+0000 7ff4cc180200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-16T17:06:37.614 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Apr 16 17:06:37 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-y[37841]: 2024-04-16T17:06:37.612+0000 7ff4cc180200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-16T17:06:37.647 INFO:journalctl@ceph.mgr.z.smithi117.stdout:Apr 16 17:06:37 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-z[38592]: 2024-04-16T17:06:37.354+0000 7f408e266200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-16T17:06:37.647 INFO:journalctl@ceph.mgr.z.smithi117.stdout:Apr 16 17:06:37 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-z[38592]: 2024-04-16T17:06:37.465+0000 7f408e266200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-16T17:06:37.648 INFO:journalctl@ceph.mgr.z.smithi117.stdout:Apr 16 17:06:37 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-z[38592]: 2024-04-16T17:06:37.552+0000 7f408e266200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-16T17:06:37.648 INFO:journalctl@ceph.mgr.z.smithi117.stdout:Apr 16 17:06:37 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-z[38592]: 2024-04-16T17:06:37.647+0000 7f408e266200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-16T17:06:37.883 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 16 17:06:37 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-x[31905]: 2024-04-16T17:06:37.629+0000 7f6d5eb50200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-16T17:06:37.883 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 16 17:06:37 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-x[31905]: 2024-04-16T17:06:37.765+0000 7f6d5eb50200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-16T17:06:37.884 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 16 17:06:37 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-x[31905]: 2024-04-16T17:06:37.882+0000 7f6d5eb50200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-16T17:06:37.927 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Apr 16 17:06:37 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-y[37841]: 2024-04-16T17:06:37.787+0000 7ff4cc180200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-16T17:06:38.055 INFO:journalctl@ceph.mgr.z.smithi117.stdout:Apr 16 17:06:37 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-z[38592]: 2024-04-16T17:06:37.823+0000 7f408e266200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-16T17:06:38.510 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 16 17:06:38 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-x[31905]: 2024-04-16T17:06:38.237+0000 7f6d5eb50200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-16T17:06:38.510 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 16 17:06:38 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-x[31905]: 2024-04-16T17:06:38.386+0000 7f6d5eb50200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-16T17:06:38.510 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 16 17:06:38 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-x[31905]: 2024-04-16T17:06:38.508+0000 7f6d5eb50200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-16T17:06:38.871 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 16 17:06:38 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-x[31905]: 2024-04-16T17:06:38.642+0000 7f6d5eb50200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-16T17:06:39.055 INFO:journalctl@ceph.mgr.z.smithi117.stdout:Apr 16 17:06:38 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-z[38592]: 2024-04-16T17:06:38.773+0000 7f408e266200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-16T17:06:39.055 INFO:journalctl@ceph.mgr.z.smithi117.stdout:Apr 16 17:06:38 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-z[38592]: 2024-04-16T17:06:38.901+0000 7f408e266200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-16T17:06:39.139 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 16 17:06:38 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-x[31905]: 2024-04-16T17:06:38.870+0000 7f6d5eb50200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-16T17:06:39.178 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Apr 16 17:06:38 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-y[37841]: 2024-04-16T17:06:38.740+0000 7ff4cc180200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-16T17:06:39.178 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Apr 16 17:06:38 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-y[37841]: 2024-04-16T17:06:38.867+0000 7ff4cc180200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-16T17:06:39.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:39.444+0000 7fb21d400700 1 -- 172.21.15.45:0/1050089494 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb2100a4de0 msgr2=0x7fb2100a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:39.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:39.444+0000 7fb21d400700 1 --2- 172.21.15.45:0/1050089494 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb2100a4de0 0x7fb2100a5260 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fb204009990 tx=0x7fb204009ca0 comp rx=0 tx=0).stop 2024-04-16T17:06:39.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:39.444+0000 7fb21d400700 1 -- 172.21.15.45:0/1050089494 shutdown_connections 2024-04-16T17:06:39.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:39.444+0000 7fb21d400700 1 --2- 172.21.15.45:0/1050089494 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb2100abd60 0x7fb2100ac140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:39.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:39.444+0000 7fb21d400700 1 --2- 172.21.15.45:0/1050089494 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb2100a57a0 0x7fb2100b3c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:39.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:39.444+0000 7fb21d400700 1 --2- 172.21.15.45:0/1050089494 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb2100a4de0 0x7fb2100a5260 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:39.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:39.444+0000 7fb21d400700 1 -- 172.21.15.45:0/1050089494 >> 172.21.15.45:0/1050089494 conn(0x7fb21001a6a0 msgr2=0x7fb21001aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:06:39.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:39.445+0000 7fb21d400700 1 -- 172.21.15.45:0/1050089494 shutdown_connections 2024-04-16T17:06:39.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:39.445+0000 7fb21d400700 1 -- 172.21.15.45:0/1050089494 wait complete. 2024-04-16T17:06:39.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:39.446+0000 7fb21d400700 1 Processor -- start 2024-04-16T17:06:39.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:39.446+0000 7fb21d400700 1 -- start start 2024-04-16T17:06:39.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:39.447+0000 7fb21d400700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb2100a4de0 0x7fb21013f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:39.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:39.447+0000 7fb21d400700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb2100a57a0 0x7fb21013fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:39.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:39.447+0000 7fb21d400700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb2100abd60 0x7fb210143f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:39.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:39.447+0000 7fb21d400700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fb2100b7050 con 0x7fb2100abd60 2024-04-16T17:06:39.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:39.447+0000 7fb21d400700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fb2100b6ed0 con 0x7fb2100a4de0 2024-04-16T17:06:39.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:39.447+0000 7fb21d400700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb2100b71d0 con 0x7fb2100a57a0 2024-04-16T17:06:39.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:39.447+0000 7fb217e00700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb2100a4de0 0x7fb21013f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:39.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:39.447+0000 7fb217e00700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb2100a4de0 0x7fb21013f6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:48940/0 (socket says 172.21.15.45:48940) 2024-04-16T17:06:39.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:39.447+0000 7fb217e00700 1 -- 172.21.15.45:0/2010643350 learned_addr learned my addr 172.21.15.45:0/2010643350 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:06:39.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:39.447+0000 7fb217400700 1 --2- 172.21.15.45:0/2010643350 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb2100a57a0 0x7fb21013fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:39.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:39.447+0000 7fb21ca00700 1 --2- 172.21.15.45:0/2010643350 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb2100abd60 0x7fb210143f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:39.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:39.447+0000 7fb217400700 1 -- 172.21.15.45:0/2010643350 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb2100a4de0 msgr2=0x7fb21013f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:39.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:39.447+0000 7fb217400700 1 --2- 172.21.15.45:0/2010643350 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb2100a4de0 0x7fb21013f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:39.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:39.447+0000 7fb217400700 1 -- 172.21.15.45:0/2010643350 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb2100abd60 msgr2=0x7fb210143f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:39.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:39.447+0000 7fb217400700 1 --2- 172.21.15.45:0/2010643350 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb2100abd60 0x7fb210143f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:39.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:39.447+0000 7fb217400700 1 -- 172.21.15.45:0/2010643350 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2101446b0 con 0x7fb2100a57a0 2024-04-16T17:06:39.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:39.448+0000 7fb217400700 1 --2- 172.21.15.45:0/2010643350 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb2100a57a0 0x7fb21013fbe0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fb204000c00 tx=0x7fb20400bab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:06:39.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:39.448+0000 7fb214c00700 1 -- 172.21.15.45:0/2010643350 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb204026070 con 0x7fb2100a57a0 2024-04-16T17:06:39.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:39.448+0000 7fb214c00700 1 -- 172.21.15.45:0/2010643350 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb204003d70 con 0x7fb2100a57a0 2024-04-16T17:06:39.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:39.448+0000 7fb214c00700 1 -- 172.21.15.45:0/2010643350 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb20400fe50 con 0x7fb2100a57a0 2024-04-16T17:06:39.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:39.448+0000 7fb21d400700 1 -- 172.21.15.45:0/2010643350 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb210144940 con 0x7fb2100a57a0 2024-04-16T17:06:39.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:39.448+0000 7fb21d400700 1 -- 172.21.15.45:0/2010643350 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb21014c220 con 0x7fb2100a57a0 2024-04-16T17:06:39.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:39.450+0000 7fb21d400700 1 -- 172.21.15.45:0/2010643350 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb2100a6260 con 0x7fb2100a57a0 2024-04-16T17:06:39.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:39.451+0000 7fb214c00700 1 -- 172.21.15.45:0/2010643350 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 19) ==== 138663+0+0 (secure 0 0 0) 0x7fb204029070 con 0x7fb2100a57a0 2024-04-16T17:06:39.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:39.453+0000 7fb214c00700 1 --2- 172.21.15.45:0/2010643350 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fb2000a3eb0 0x7fb2000a6370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:39.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:39.453+0000 7fb217e00700 1 -- 172.21.15.45:0/2010643350 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fb2000a3eb0 msgr2=0x7fb2000a6370 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.45:6800/268030699 2024-04-16T17:06:39.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:39.453+0000 7fb214c00700 1 -- 172.21.15.45:0/2010643350 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7fb2040d14f0 con 0x7fb2100a57a0 2024-04-16T17:06:39.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:39.453+0000 7fb217e00700 1 --2- 172.21.15.45:0/2010643350 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fb2000a3eb0 0x7fb2000a6370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-16T17:06:39.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:39.457+0000 7fb214c00700 1 -- 172.21.15.45:0/2010643350 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb20405dff0 con 0x7fb2100a57a0 2024-04-16T17:06:39.479 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Apr 16 17:06:39 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-y[37841]: 2024-04-16T17:06:39.196+0000 7ff4cc180200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-16T17:06:39.479 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Apr 16 17:06:39 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-y[37841]: 2024-04-16T17:06:39.284+0000 7ff4cc180200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-16T17:06:39.514 INFO:journalctl@ceph.mgr.z.smithi117.stdout:Apr 16 17:06:39 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-z[38592]: 2024-04-16T17:06:39.230+0000 7f408e266200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-16T17:06:39.514 INFO:journalctl@ceph.mgr.z.smithi117.stdout:Apr 16 17:06:39 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-z[38592]: 2024-04-16T17:06:39.317+0000 7f408e266200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-16T17:06:39.514 INFO:journalctl@ceph.mgr.z.smithi117.stdout:Apr 16 17:06:39 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-z[38592]: 2024-04-16T17:06:39.513+0000 7f408e266200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-16T17:06:39.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:39.654+0000 7fb217e00700 1 -- 172.21.15.45:0/2010643350 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fb2000a3eb0 msgr2=0x7fb2000a6370 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.45:6800/268030699 2024-04-16T17:06:39.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:39.654+0000 7fb217e00700 1 --2- 172.21.15.45:0/2010643350 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fb2000a3eb0 0x7fb2000a6370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-16T17:06:39.748 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:39.747+0000 7fb21d400700 1 -- 172.21.15.45:0/2010643350 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fb210140a00 con 0x7fb2100a57a0 2024-04-16T17:06:39.748 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:39.748+0000 7fb214c00700 1 -- 172.21.15.45:0/2010643350 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v24) ==== 53+0+88 (secure 0 0 0) 0x7fb204013070 con 0x7fb2100a57a0 2024-04-16T17:06:39.748 INFO:teuthology.orchestra.run.smithi045.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-04-16T17:06:39.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:39.751+0000 7fb21d400700 1 -- 172.21.15.45:0/2010643350 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fb2000a3eb0 msgr2=0x7fb2000a6370 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-16T17:06:39.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:39.751+0000 7fb21d400700 1 --2- 172.21.15.45:0/2010643350 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fb2000a3eb0 0x7fb2000a6370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:39.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:39.751+0000 7fb21d400700 1 -- 172.21.15.45:0/2010643350 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb2100a57a0 msgr2=0x7fb21013fbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:39.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:39.751+0000 7fb21d400700 1 --2- 172.21.15.45:0/2010643350 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb2100a57a0 0x7fb21013fbe0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fb204000c00 tx=0x7fb20400bab0 comp rx=0 tx=0).stop 2024-04-16T17:06:39.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:39.751+0000 7fb21d400700 1 -- 172.21.15.45:0/2010643350 shutdown_connections 2024-04-16T17:06:39.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:39.751+0000 7fb21d400700 1 --2- 172.21.15.45:0/2010643350 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb2100abd60 0x7fb210143f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:39.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:39.751+0000 7fb21d400700 1 --2- 172.21.15.45:0/2010643350 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb2100a4de0 0x7fb21013f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:39.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:39.751+0000 7fb21d400700 1 --2- 172.21.15.45:0/2010643350 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7fb2000a3eb0 0x7fb2000a6370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:39.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:39.751+0000 7fb21d400700 1 --2- 172.21.15.45:0/2010643350 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb2100a57a0 0x7fb21013fbe0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:39.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:39.751+0000 7fb21d400700 1 -- 172.21.15.45:0/2010643350 >> 172.21.15.45:0/2010643350 conn(0x7fb21001a6a0 msgr2=0x7fb2100b2080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:06:39.753 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:39.751+0000 7fb21d400700 1 -- 172.21.15.45:0/2010643350 shutdown_connections 2024-04-16T17:06:39.753 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:39.751+0000 7fb21d400700 1 -- 172.21.15.45:0/2010643350 wait complete. 2024-04-16T17:06:39.801 INFO:journalctl@ceph.mgr.z.smithi117.stdout:Apr 16 17:06:39 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-z[38592]: 2024-04-16T17:06:39.696+0000 7f408e266200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-16T17:06:39.801 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Apr 16 17:06:39 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-y[37841]: 2024-04-16T17:06:39.478+0000 7ff4cc180200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-16T17:06:39.801 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Apr 16 17:06:39 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-y[37841]: 2024-04-16T17:06:39.662+0000 7ff4cc180200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-16T17:06:40.055 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:06:39 smithi117 ceph-mon[36730]: from='client.? 172.21.15.45:0/2010643350' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-16T17:06:40.056 INFO:journalctl@ceph.mgr.z.smithi117.stdout:Apr 16 17:06:39 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-z[38592]: 2024-04-16T17:06:39.977+0000 7f408e266200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-16T17:06:40.178 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:06:39 smithi104 ceph-mon[36457]: from='client.? 172.21.15.45:0/2010643350' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-16T17:06:40.178 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Apr 16 17:06:39 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-y[37841]: 2024-04-16T17:06:39.944+0000 7ff4cc180200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-16T17:06:40.213 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:06:39 smithi045 ceph-mon[31676]: from='client.? 172.21.15.45:0/2010643350' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-16T17:06:40.478 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 16 17:06:40 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-x[31905]: 2024-04-16T17:06:40.327+0000 7f6d5eb50200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-16T17:06:40.555 INFO:journalctl@ceph.mgr.z.smithi117.stdout:Apr 16 17:06:40 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-z[38592]: 2024-04-16T17:06:40.232+0000 7f408e266200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-16T17:06:40.555 INFO:journalctl@ceph.mgr.z.smithi117.stdout:Apr 16 17:06:40 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-z[38592]: 2024-04-16T17:06:40.467+0000 7f408e266200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-16T17:06:40.678 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Apr 16 17:06:40 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-y[37841]: 2024-04-16T17:06:40.199+0000 7ff4cc180200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-16T17:06:40.678 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Apr 16 17:06:40 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-y[37841]: 2024-04-16T17:06:40.433+0000 7ff4cc180200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-16T17:06:40.681 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-16T17:06:40.681 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-16T17:06:40.681 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi045.front.sepia.ceph.com 2024-04-16T17:06:40.682 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-16T17:06:40.682 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-16T17:06:40.682 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-16T17:06:40.682 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-16T17:06:40.682 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-16T17:06:40.682 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-16T17:06:40.682 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T17:06:40.683 DEBUG:teuthology.orchestra.run.smithi045:> ip addr 2024-04-16T17:06:40.738 INFO:teuthology.orchestra.run.smithi045.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-16T17:06:40.738 INFO:teuthology.orchestra.run.smithi045.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-16T17:06:40.738 INFO:teuthology.orchestra.run.smithi045.stdout: inet 127.0.0.1/8 scope host lo 2024-04-16T17:06:40.739 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-04-16T17:06:40.739 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 ::1/128 scope host 2024-04-16T17:06:40.739 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-04-16T17:06:40.739 INFO:teuthology.orchestra.run.smithi045.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-16T17:06:40.739 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:6b:fd:a6 brd ff:ff:ff:ff:ff:ff 2024-04-16T17:06:40.739 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp7s0f0 2024-04-16T17:06:40.739 INFO:teuthology.orchestra.run.smithi045.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-16T17:06:40.739 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:6b:fd:a7 brd ff:ff:ff:ff:ff:ff 2024-04-16T17:06:40.739 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp7s0f1 2024-04-16T17:06:40.739 INFO:teuthology.orchestra.run.smithi045.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-16T17:06:40.740 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:bd:12:be brd ff:ff:ff:ff:ff:ff 2024-04-16T17:06:40.740 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp1s0f0 2024-04-16T17:06:40.740 INFO:teuthology.orchestra.run.smithi045.stdout: inet 172.21.15.45/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-16T17:06:40.740 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft 41774sec preferred_lft 41774sec 2024-04-16T17:06:40.740 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::ec4:7aff:febd:12be/64 scope link noprefixroute 2024-04-16T17:06:40.740 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-04-16T17:06:40.740 INFO:teuthology.orchestra.run.smithi045.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-16T17:06:40.740 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:bd:12:bf brd ff:ff:ff:ff:ff:ff 2024-04-16T17:06:40.740 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp1s0f1 2024-04-16T17:06:40.741 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-16T17:06:40.741 DEBUG:teuthology.orchestra.run.smithi045:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-16T17:06:40.741 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-04-16T17:06:40.741 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip link add name ceph-brx type bridge 2024-04-16T17:06:40.741 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip addr flush dev ceph-brx 2024-04-16T17:06:40.741 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip link set ceph-brx up 2024-04-16T17:06:40.741 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-16T17:06:40.741 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-04-16T17:06:40.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:40 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-16T17:06:40.882 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 16 17:06:40 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-x[31905]: 2024-04-16T17:06:40.513+0000 7f6d5eb50200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-16T17:06:41.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:41 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-16T17:06:41.020 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T17:06:41.020 DEBUG:teuthology.orchestra.run.smithi045:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-16T17:06:41.094 INFO:teuthology.orchestra.run.smithi045.stdout:1 2024-04-16T17:06:41.099 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T17:06:41.099 DEBUG:teuthology.orchestra.run.smithi045:> route 2024-04-16T17:06:41.147 INFO:teuthology.orchestra.run.smithi045.stdout:Kernel IP routing table 2024-04-16T17:06:41.148 INFO:teuthology.orchestra.run.smithi045.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-16T17:06:41.148 INFO:teuthology.orchestra.run.smithi045.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-16T17:06:41.148 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-16T17:06:41.148 INFO:teuthology.orchestra.run.smithi045.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-16T17:06:41.148 DEBUG:teuthology.orchestra.run.smithi045:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-16T17:06:41.148 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-04-16T17:06:41.149 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-16T17:06:41.149 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-16T17:06:41.149 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-16T17:06:41.149 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-04-16T17:06:41.227 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 16 17:06:40 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-x[31905]: 2024-04-16T17:06:40.963+0000 7f6d5eb50200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-16T17:06:41.227 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 16 17:06:41 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-x[31905]: 2024-04-16T17:06:41.096+0000 7f6d5eb50200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-16T17:06:41.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:41 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-16T17:06:41.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:41 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-16T17:06:41.433 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T17:06:41.434 DEBUG:teuthology.orchestra.run.smithi045:> ip netns list 2024-04-16T17:06:41.530 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T17:06:41.530 DEBUG:teuthology.orchestra.run.smithi045:> ip netns list-id 2024-04-16T17:06:41.555 INFO:journalctl@ceph.mgr.z.smithi117.stdout:Apr 16 17:06:41 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-z[38592]: 2024-04-16T17:06:41.259+0000 7f408e266200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-16T17:06:41.581 DEBUG:teuthology.orchestra.run.smithi045:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-16T17:06:41.581 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-04-16T17:06:41.581 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-16T17:06:41.581 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-04-16T17:06:41.581 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-04-16T17:06:41.646 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 16 17:06:41 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-x[31905]: 2024-04-16T17:06:41.379+0000 7f6d5eb50200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-16T17:06:41.677 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Apr 16 17:06:41 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-y[37841]: 2024-04-16T17:06:41.229+0000 7ff4cc180200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-16T17:06:41.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:41 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-16T17:06:41.768 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:41 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-16T17:06:41.777 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-04-16T17:06:41.777 DEBUG:teuthology.orchestra.run.smithi045:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-16T17:06:41.777 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-04-16T17:06:41.777 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-04-16T17:06:41.777 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-16T17:06:41.777 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-04-16T17:06:41.777 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-04-16T17:06:41.778 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-04-16T17:06:41.778 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-04-16T17:06:41.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:41 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-16T17:06:41.903 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 16 17:06:41 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-x[31905]: 2024-04-16T17:06:41.645+0000 7f6d5eb50200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-16T17:06:42.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-16T17:06:42.080 DEBUG:teuthology.orchestra.run.smithi045:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-16T17:06:42.080 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-04-16T17:06:42.080 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip link set brx.0 up 2024-04-16T17:06:42.080 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip link set dev brx.0 master ceph-brx 2024-04-16T17:06:42.081 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-04-16T17:06:42.104 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Apr 16 17:06:41 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-y[37841]: 2024-04-16T17:06:41.795+0000 7ff4cc180200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-16T17:06:42.104 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Apr 16 17:06:41 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-y[37841]: 2024-04-16T17:06:41.889+0000 7ff4cc180200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-16T17:06:42.104 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Apr 16 17:06:42 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-y[37841]: 2024-04-16T17:06:42.103+0000 7ff4cc180200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-16T17:06:42.132 INFO:journalctl@ceph.mgr.z.smithi117.stdout:Apr 16 17:06:41 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-z[38592]: 2024-04-16T17:06:41.824+0000 7f408e266200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-16T17:06:42.132 INFO:journalctl@ceph.mgr.z.smithi117.stdout:Apr 16 17:06:41 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-z[38592]: 2024-04-16T17:06:41.919+0000 7f408e266200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-16T17:06:42.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:42 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-16T17:06:42.185 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 16 17:06:42 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-x[31905]: 2024-04-16T17:06:42.072+0000 7f6d5eb50200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-16T17:06:42.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-16T17:06:42.295 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-16T17:06:42.295 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-16T17:06:42.337 INFO:teuthology.orchestra.run.smithi045.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-16T17:06:42.337 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-16T17:06:42.337 DEBUG:teuthology.orchestra.run.smithi045:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-16T17:06:42.426 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-04-16T17:06:42.426 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-04-16T17:06:42.426 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/volumes/qa/sv_1/584ee6b0-43a5-48e1-b843-9914b8e65254 2024-04-16T17:06:42.426 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T17:06:42.426 DEBUG:teuthology.orchestra.run.smithi045:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/volumes/qa/sv_1/584ee6b0-43a5-48e1-b843-9914b8e65254 /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=crc,wsync,nofallback 2024-04-16T17:06:42.427 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:06:42 smithi104 ceph-mon[36457]: Standby manager daemon y restarted 2024-04-16T17:06:42.427 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:06:42 smithi104 ceph-mon[36457]: Standby manager daemon y started 2024-04-16T17:06:42.428 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:06:42 smithi104 ceph-mon[36457]: from='mgr.? 172.21.15.104:0/2044194580' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-16T17:06:42.428 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:06:42 smithi104 ceph-mon[36457]: from='mgr.? 172.21.15.104:0/2044194580' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-16T17:06:42.428 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:06:42 smithi104 ceph-mon[36457]: from='mgr.? 172.21.15.104:0/2044194580' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-16T17:06:42.428 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:06:42 smithi104 ceph-mon[36457]: from='mgr.? 172.21.15.104:0/2044194580' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-16T17:06:42.428 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:06:42 smithi104 ceph-mon[36457]: Standby manager daemon z restarted 2024-04-16T17:06:42.428 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:06:42 smithi104 ceph-mon[36457]: Standby manager daemon z started 2024-04-16T17:06:42.428 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:06:42 smithi104 ceph-mon[36457]: from='mgr.? 172.21.15.117:0/2351501017' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-16T17:06:42.428 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:06:42 smithi104 ceph-mon[36457]: from='mgr.? 172.21.15.117:0/2351501017' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-16T17:06:42.428 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:06:42 smithi104 ceph-mon[36457]: from='mgr.? 172.21.15.117:0/2351501017' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-16T17:06:42.428 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:06:42 smithi104 ceph-mon[36457]: from='mgr.? 172.21.15.117:0/2351501017' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-16T17:06:42.477 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 16 17:06:42 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-x[31905]: 2024-04-16T17:06:42.413+0000 7f6d5eb50200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-16T17:06:42.477 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:06:42 smithi045 ceph-mon[31676]: Standby manager daemon y restarted 2024-04-16T17:06:42.478 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:06:42 smithi045 ceph-mon[31676]: Standby manager daemon y started 2024-04-16T17:06:42.478 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:06:42 smithi045 ceph-mon[31676]: from='mgr.? 172.21.15.104:0/2044194580' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-16T17:06:42.478 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:06:42 smithi045 ceph-mon[31676]: from='mgr.? 172.21.15.104:0/2044194580' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-16T17:06:42.478 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:06:42 smithi045 ceph-mon[31676]: from='mgr.? 172.21.15.104:0/2044194580' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-16T17:06:42.478 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:06:42 smithi045 ceph-mon[31676]: from='mgr.? 172.21.15.104:0/2044194580' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-16T17:06:42.478 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:06:42 smithi045 ceph-mon[31676]: Standby manager daemon z restarted 2024-04-16T17:06:42.478 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:06:42 smithi045 ceph-mon[31676]: Standby manager daemon z started 2024-04-16T17:06:42.478 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:06:42 smithi045 ceph-mon[31676]: from='mgr.? 172.21.15.117:0/2351501017' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-16T17:06:42.478 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:06:42 smithi045 ceph-mon[31676]: from='mgr.? 172.21.15.117:0/2351501017' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-16T17:06:42.478 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:06:42 smithi045 ceph-mon[31676]: from='mgr.? 172.21.15.117:0/2351501017' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-16T17:06:42.478 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:06:42 smithi045 ceph-mon[31676]: from='mgr.? 172.21.15.117:0/2351501017' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-16T17:06:42.549 INFO:teuthology.orchestra.run.smithi045.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=crc,wsync,nofallback 2024-04-16T17:06:42.549 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=crc,wsync". 2024-04-16T17:06:42.555 INFO:journalctl@ceph.mgr.z.smithi117.stdout:Apr 16 17:06:42 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-z[38592]: 2024-04-16T17:06:42.131+0000 7f408e266200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-16T17:06:42.555 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:06:42 smithi117 ceph-mon[36730]: Standby manager daemon y restarted 2024-04-16T17:06:42.555 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:06:42 smithi117 ceph-mon[36730]: Standby manager daemon y started 2024-04-16T17:06:42.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:06:42 smithi117 ceph-mon[36730]: from='mgr.? 172.21.15.104:0/2044194580' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-16T17:06:42.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:06:42 smithi117 ceph-mon[36730]: from='mgr.? 172.21.15.104:0/2044194580' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-16T17:06:42.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:06:42 smithi117 ceph-mon[36730]: from='mgr.? 172.21.15.104:0/2044194580' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-16T17:06:42.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:06:42 smithi117 ceph-mon[36730]: from='mgr.? 172.21.15.104:0/2044194580' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-16T17:06:42.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:06:42 smithi117 ceph-mon[36730]: Standby manager daemon z restarted 2024-04-16T17:06:42.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:06:42 smithi117 ceph-mon[36730]: Standby manager daemon z started 2024-04-16T17:06:42.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:06:42 smithi117 ceph-mon[36730]: from='mgr.? 172.21.15.117:0/2351501017' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-16T17:06:42.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:06:42 smithi117 ceph-mon[36730]: from='mgr.? 172.21.15.117:0/2351501017' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-16T17:06:42.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:06:42 smithi117 ceph-mon[36730]: from='mgr.? 172.21.15.117:0/2351501017' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-16T17:06:42.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:06:42 smithi117 ceph-mon[36730]: from='mgr.? 172.21.15.117:0/2351501017' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-16T17:06:42.841 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 16 17:06:42 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-x[31905]: 2024-04-16T17:06:42.745+0000 7f6d5eb50200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-16T17:06:43.305 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:06:43 smithi045 ceph-mon[31676]: mgrmap e20: x(active, since 11m), standbys: z, y 2024-04-16T17:06:43.329 INFO:teuthology.orchestra.run.smithi045.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=crc,wsync,nofallback 2024-04-16T17:06:43.329 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=crc,wsync". 2024-04-16T17:06:43.329 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: resolved to: "172.21.15.45:3300,172.21.15.104:3300,172.21.15.117:3300" 2024-04-16T17:06:43.330 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: trying mount with new device syntax: 0@b1fecd38-fc11-11ee-bc90-c7b262605968.cephfs=/volumes/qa/sv_1/584ee6b0-43a5-48e1-b843-9914b8e65254 2024-04-16T17:06:43.330 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=crc,wsync,name=0,key=0,mon_addr=172.21.15.45:3300/172.21.15.104:3300/172.21.15.117:3300" will pass to kernel 2024-04-16T17:06:43.333 INFO:teuthology.orchestra.run.smithi045.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-16T17:06:43.333 INFO:teuthology.orchestra.run.smithi045.stdout: You just mounted a file system that supports labels which does not 2024-04-16T17:06:43.333 INFO:teuthology.orchestra.run.smithi045.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-16T17:06:43.333 INFO:teuthology.orchestra.run.smithi045.stdout: applications will generate AVC messages and not be allowed access to 2024-04-16T17:06:43.333 INFO:teuthology.orchestra.run.smithi045.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-16T17:06:43.337 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-16T17:06:43.337 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T17:06:43.337 DEBUG:teuthology.orchestra.run.smithi045:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-16T17:06:43.425 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-16T17:06:43.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:43 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-16T17:06:43.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-16T17:06:43.542 INFO:teuthology.orchestra.run.smithi045.stdout:client25436 2024-04-16T17:06:43.543 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-04-16T17:06:43.543 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-16T17:06:43.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:06:43 smithi117 ceph-mon[36730]: mgrmap e20: x(active, since 11m), standbys: z, y 2024-04-16T17:06:43.590 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-16T17:06:43.612 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:06:43 smithi104 ceph-mon[36457]: mgrmap e20: x(active, since 11m), standbys: z, y 2024-04-16T17:06:43.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:43 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-16T17:06:43.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-16T17:06:43.750 INFO:teuthology.orchestra.run.smithi045.stdout:client25436 2024-04-16T17:06:43.750 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-16T17:06:43.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:43 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-16T17:06:43.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-16T17:06:43.860 INFO:teuthology.orchestra.run.smithi045.stdout:client25436 2024-04-16T17:06:43.860 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968.client25436/status') 2024-04-16T17:06:43.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:43 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-16T17:06:44.002 INFO:teuthology.orchestra.run.smithi045.stdout:instance: client.25436 (3)172.21.15.45:0/2349481026 2024-04-16T17:06:44.002 INFO:teuthology.orchestra.run.smithi045.stdout:blocklisted: false 2024-04-16T17:06:44.002 INFO:teuthology.orchestra.run.smithi045.stderr:0+1 records in 2024-04-16T17:06:44.002 INFO:teuthology.orchestra.run.smithi045.stderr:0+1 records out 2024-04-16T17:06:44.002 INFO:teuthology.orchestra.run.smithi045.stderr:71 bytes copied, 8.5592e-05 s, 830 kB/s 2024-04-16T17:06:44.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-16T17:06:44.015 INFO:teuthology.run_tasks:Running task exec... 2024-04-16T17:06:44.027 INFO:teuthology.task.exec:Executing custom commands... 2024-04-16T17:06:44.027 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi045.front.sepia.ceph.com 2024-04-16T17:06:44.027 DEBUG:teuthology.orchestra.run.smithi045:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate true' 2024-04-16T17:06:44.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:44.225+0000 7f2b91800640 1 -- 172.21.15.45:0/1553431827 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2b8c1054c0 msgr2=0x7f2b8c1078b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:44.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:44.225+0000 7f2b91800640 1 --2- 172.21.15.45:0/1553431827 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2b8c1054c0 0x7f2b8c1078b0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f2b78009900 tx=0x7f2b7802bc30 comp rx=0 tx=0).stop 2024-04-16T17:06:44.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:44.225+0000 7f2b91800640 1 -- 172.21.15.45:0/1553431827 shutdown_connections 2024-04-16T17:06:44.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:44.225+0000 7f2b91800640 1 --2- 172.21.15.45:0/1553431827 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2b8c107df0 0x7f2b8c10a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:44.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:44.225+0000 7f2b91800640 1 --2- 172.21.15.45:0/1553431827 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2b8c1054c0 0x7f2b8c1078b0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:44.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:44.225+0000 7f2b91800640 1 --2- 172.21.15.45:0/1553431827 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2b8c102b90 0x7f2b8c104f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:44.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:44.225+0000 7f2b91800640 1 -- 172.21.15.45:0/1553431827 >> 172.21.15.45:0/1553431827 conn(0x7f2b8c0fc580 msgr2=0x7f2b8c0fe9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:06:44.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:44.226+0000 7f2b91800640 1 -- 172.21.15.45:0/1553431827 shutdown_connections 2024-04-16T17:06:44.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:44.226+0000 7f2b91800640 1 -- 172.21.15.45:0/1553431827 wait complete. 2024-04-16T17:06:44.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:44.227+0000 7f2b91800640 1 Processor -- start 2024-04-16T17:06:44.228 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 16 17:06:43 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-x[31905]: 2024-04-16T17:06:43.867+0000 7f6d5eb50200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-16T17:06:44.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:44.227+0000 7f2b91800640 1 -- start start 2024-04-16T17:06:44.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:44.227+0000 7f2b91800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2b8c102b90 0x7f2b8c19a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:44.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:44.227+0000 7f2b91800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2b8c1054c0 0x7f2b8c19aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:44.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:44.227+0000 7f2b91800640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2b8c107df0 0x7f2b8c1a1a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:44.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:44.227+0000 7f2b91800640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f2b8c10dd30 con 0x7f2b8c1054c0 2024-04-16T17:06:44.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:44.227+0000 7f2b91800640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f2b8c10dbb0 con 0x7f2b8c107df0 2024-04-16T17:06:44.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:44.227+0000 7f2b91800640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f2b8c10deb0 con 0x7f2b8c102b90 2024-04-16T17:06:44.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:44.228+0000 7f2b8aa00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2b8c102b90 0x7f2b8c19a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:44.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:44.228+0000 7f2b8b400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2b8c107df0 0x7f2b8c1a1a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:44.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:44.228+0000 7f2b8aa00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2b8c102b90 0x7f2b8c19a4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:50908/0 (socket says 172.21.15.45:50908) 2024-04-16T17:06:44.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:44.228+0000 7f2b8aa00640 1 -- 172.21.15.45:0/48305014 learned_addr learned my addr 172.21.15.45:0/48305014 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:06:44.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:44.228+0000 7f2b8a000640 1 --2- 172.21.15.45:0/48305014 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2b8c1054c0 0x7f2b8c19aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:44.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:44.228+0000 7f2b8b400640 1 -- 172.21.15.45:0/48305014 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2b8c102b90 msgr2=0x7f2b8c19a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:44.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:44.228+0000 7f2b8b400640 1 --2- 172.21.15.45:0/48305014 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2b8c102b90 0x7f2b8c19a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:44.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:44.228+0000 7f2b8b400640 1 -- 172.21.15.45:0/48305014 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2b8c1054c0 msgr2=0x7f2b8c19aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:44.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:44.228+0000 7f2b8b400640 1 --2- 172.21.15.45:0/48305014 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2b8c1054c0 0x7f2b8c19aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:44.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:44.228+0000 7f2b8b400640 1 -- 172.21.15.45:0/48305014 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b8c1a20e0 con 0x7f2b8c107df0 2024-04-16T17:06:44.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:44.228+0000 7f2b8aa00640 1 --2- 172.21.15.45:0/48305014 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2b8c102b90 0x7f2b8c19a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:06:44.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:44.228+0000 7f2b8b400640 1 --2- 172.21.15.45:0/48305014 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2b8c107df0 0x7f2b8c1a1a70 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f2b84009fd0 tx=0x7f2b8400ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:06:44.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:44.229+0000 7f2b83400640 1 -- 172.21.15.45:0/48305014 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b8400ce20 con 0x7f2b8c107df0 2024-04-16T17:06:44.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:44.229+0000 7f2b83400640 1 -- 172.21.15.45:0/48305014 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2b84005020 con 0x7f2b8c107df0 2024-04-16T17:06:44.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:44.229+0000 7f2b83400640 1 -- 172.21.15.45:0/48305014 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b84010720 con 0x7f2b8c107df0 2024-04-16T17:06:44.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:44.229+0000 7f2b91800640 1 -- 172.21.15.45:0/48305014 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b8c1a23d0 con 0x7f2b8c107df0 2024-04-16T17:06:44.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:44.229+0000 7f2b91800640 1 -- 172.21.15.45:0/48305014 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b8c1a28f0 con 0x7f2b8c107df0 2024-04-16T17:06:44.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:44.230+0000 7f2b91800640 1 -- 172.21.15.45:0/48305014 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b8c109940 con 0x7f2b8c107df0 2024-04-16T17:06:44.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:44.232+0000 7f2b83400640 1 -- 172.21.15.45:0/48305014 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 20) ==== 138663+0+0 (secure 0 0 0) 0x7f2b84005540 con 0x7f2b8c107df0 2024-04-16T17:06:44.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:44.234+0000 7f2b83400640 1 --2- 172.21.15.45:0/48305014 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f2b700a3d20 0x7f2b700a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:44.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:44.234+0000 7f2b8aa00640 1 -- 172.21.15.45:0/48305014 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f2b700a3d20 msgr2=0x7f2b700a61e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.45:6800/268030699 2024-04-16T17:06:44.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:44.234+0000 7f2b83400640 1 -- 172.21.15.45:0/48305014 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f2b840d0e70 con 0x7f2b8c107df0 2024-04-16T17:06:44.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:44.234+0000 7f2b8aa00640 1 --2- 172.21.15.45:0/48305014 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f2b700a3d20 0x7f2b700a61e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-16T17:06:44.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:44.237+0000 7f2b83400640 1 -- 172.21.15.45:0/48305014 <== mon.2 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2b8409d4c0 con 0x7f2b8c107df0 2024-04-16T17:06:44.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:44.435+0000 7f2b8aa00640 1 -- 172.21.15.45:0/48305014 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f2b700a3d20 msgr2=0x7f2b700a61e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.45:6800/268030699 2024-04-16T17:06:44.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:44.435+0000 7f2b8aa00640 1 --2- 172.21.15.45:0/48305014 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f2b700a3d20 0x7f2b700a61e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-16T17:06:44.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:44.483+0000 7f2b91800640 1 -- 172.21.15.45:0/48305014 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"} v 0) -- 0x7f2b8c062c00 con 0x7f2b8c107df0 2024-04-16T17:06:44.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:44.542+0000 7f2b83400640 1 -- 172.21.15.45:0/48305014 <== mon.2 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]=0 v25) ==== 117+0+0 (secure 0 0 0) 0x7f2b8409cdc0 con 0x7f2b8c107df0 2024-04-16T17:06:44.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:44.545+0000 7f2b91800640 1 -- 172.21.15.45:0/48305014 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f2b700a3d20 msgr2=0x7f2b700a61e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-16T17:06:44.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:44.545+0000 7f2b91800640 1 --2- 172.21.15.45:0/48305014 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f2b700a3d20 0x7f2b700a61e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:44.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:44.546+0000 7f2b91800640 1 -- 172.21.15.45:0/48305014 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2b8c107df0 msgr2=0x7f2b8c1a1a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:44.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:44.546+0000 7f2b91800640 1 --2- 172.21.15.45:0/48305014 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2b8c107df0 0x7f2b8c1a1a70 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f2b84009fd0 tx=0x7f2b8400ef10 comp rx=0 tx=0).stop 2024-04-16T17:06:44.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:44.546+0000 7f2b91800640 1 -- 172.21.15.45:0/48305014 shutdown_connections 2024-04-16T17:06:44.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:44.546+0000 7f2b91800640 1 --2- 172.21.15.45:0/48305014 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f2b700a3d20 0x7f2b700a61e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:44.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:44.546+0000 7f2b91800640 1 --2- 172.21.15.45:0/48305014 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2b8c107df0 0x7f2b8c1a1a70 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:44.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:44.546+0000 7f2b91800640 1 --2- 172.21.15.45:0/48305014 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2b8c1054c0 0x7f2b8c19aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:44.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:44.546+0000 7f2b91800640 1 --2- 172.21.15.45:0/48305014 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2b8c102b90 0x7f2b8c19a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:44.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:44.546+0000 7f2b91800640 1 -- 172.21.15.45:0/48305014 >> 172.21.15.45:0/48305014 conn(0x7f2b8c0fc580 msgr2=0x7f2b8c1021c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:06:44.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:44.546+0000 7f2b91800640 1 -- 172.21.15.45:0/48305014 shutdown_connections 2024-04-16T17:06:44.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:44.546+0000 7f2b91800640 1 -- 172.21.15.45:0/48305014 wait complete. 2024-04-16T17:06:44.572 INFO:teuthology.run_tasks:Running task check-counter... 2024-04-16T17:06:44.583 INFO:teuthology.run_tasks:Running task fwd_scrub... 2024-04-16T17:06:44.594 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-16T17:06:44.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:06:44 smithi117 ceph-mon[36730]: from='client.? 172.21.15.45:0/48305014' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-04-16T17:06:44.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:06:44 smithi117 ceph-mon[36730]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-04-16T17:06:44.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:44.806+0000 7f3c7a600640 1 -- 172.21.15.45:0/2486833983 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3c74104280 msgr2=0x7f3c741046a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:44.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:44.806+0000 7f3c7a600640 1 --2- 172.21.15.45:0/2486833983 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3c74104280 0x7f3c741046a0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f3c64009900 tx=0x7f3c6402bc50 comp rx=0 tx=0).stop 2024-04-16T17:06:44.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:44.806+0000 7f3c7a600640 1 -- 172.21.15.45:0/2486833983 shutdown_connections 2024-04-16T17:06:44.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:44.806+0000 7f3c7a600640 1 --2- 172.21.15.45:0/2486833983 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3c74105470 0x7f3c741058f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:44.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:44.806+0000 7f3c7a600640 1 --2- 172.21.15.45:0/2486833983 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3c74105e30 0x7f3c7410c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:44.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:44.806+0000 7f3c7a600640 1 --2- 172.21.15.45:0/2486833983 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3c74104280 0x7f3c741046a0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:44.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:44.806+0000 7f3c7a600640 1 -- 172.21.15.45:0/2486833983 >> 172.21.15.45:0/2486833983 conn(0x7f3c740ff7f0 msgr2=0x7f3c74101c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:06:44.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:44.806+0000 7f3c7a600640 1 -- 172.21.15.45:0/2486833983 shutdown_connections 2024-04-16T17:06:44.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:44.806+0000 7f3c7a600640 1 -- 172.21.15.45:0/2486833983 wait complete. 2024-04-16T17:06:44.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:44.808+0000 7f3c7a600640 1 Processor -- start 2024-04-16T17:06:44.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:44.808+0000 7f3c7a600640 1 -- start start 2024-04-16T17:06:44.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:44.808+0000 7f3c7a600640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3c74104280 0x7f3c7419a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:44.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:44.808+0000 7f3c7a600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3c74105470 0x7f3c7419aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:44.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:44.809+0000 7f3c7a600640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3c74105e30 0x7f3c741a1b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:44.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:44.809+0000 7f3c7a600640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f3c7410ff60 con 0x7f3c74105470 2024-04-16T17:06:44.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:44.809+0000 7f3c7a600640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f3c7410fde0 con 0x7f3c74105e30 2024-04-16T17:06:44.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:44.809+0000 7f3c7a600640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f3c741100e0 con 0x7f3c74104280 2024-04-16T17:06:44.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:44.809+0000 7f3c73400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3c74104280 0x7f3c7419a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:44.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:44.809+0000 7f3c73400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3c74104280 0x7f3c7419a590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:50926/0 (socket says 172.21.15.45:50926) 2024-04-16T17:06:44.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:44.809+0000 7f3c73400640 1 -- 172.21.15.45:0/937679098 learned_addr learned my addr 172.21.15.45:0/937679098 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:06:44.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:44.809+0000 7f3c73e00640 1 --2- 172.21.15.45:0/937679098 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3c74105e30 0x7f3c741a1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:44.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:44.809+0000 7f3c72a00640 1 --2- 172.21.15.45:0/937679098 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3c74105470 0x7f3c7419aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:44.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:44.809+0000 7f3c73e00640 1 -- 172.21.15.45:0/937679098 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3c74104280 msgr2=0x7f3c7419a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:44.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:44.809+0000 7f3c73e00640 1 --2- 172.21.15.45:0/937679098 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3c74104280 0x7f3c7419a590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:44.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:44.809+0000 7f3c73e00640 1 -- 172.21.15.45:0/937679098 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3c74105470 msgr2=0x7f3c7419aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:44.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:44.809+0000 7f3c73e00640 1 --2- 172.21.15.45:0/937679098 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3c74105470 0x7f3c7419aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:44.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:44.809+0000 7f3c73e00640 1 -- 172.21.15.45:0/937679098 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c741a21b0 con 0x7f3c74105e30 2024-04-16T17:06:44.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:44.809+0000 7f3c73400640 1 --2- 172.21.15.45:0/937679098 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3c74104280 0x7f3c7419a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:06:44.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:44.809+0000 7f3c72a00640 1 --2- 172.21.15.45:0/937679098 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3c74105470 0x7f3c7419aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:06:44.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:44.810+0000 7f3c73e00640 1 --2- 172.21.15.45:0/937679098 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3c74105e30 0x7f3c741a1b40 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f3c6c009fd0 tx=0x7f3c6c00ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:06:44.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:44.810+0000 7f3c6be00640 1 -- 172.21.15.45:0/937679098 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c6c00ce20 con 0x7f3c74105e30 2024-04-16T17:06:44.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:44.810+0000 7f3c7a600640 1 -- 172.21.15.45:0/937679098 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c741a24a0 con 0x7f3c74105e30 2024-04-16T17:06:44.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:44.810+0000 7f3c6be00640 1 -- 172.21.15.45:0/937679098 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3c6c005020 con 0x7f3c74105e30 2024-04-16T17:06:44.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:44.810+0000 7f3c6be00640 1 -- 172.21.15.45:0/937679098 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c6c010720 con 0x7f3c74105e30 2024-04-16T17:06:44.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:44.810+0000 7f3c7a600640 1 -- 172.21.15.45:0/937679098 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c741a29a0 con 0x7f3c74105e30 2024-04-16T17:06:44.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:44.811+0000 7f3c7a600640 1 -- 172.21.15.45:0/937679098 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c7410b5a0 con 0x7f3c74105e30 2024-04-16T17:06:44.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:44.813+0000 7f3c6be00640 1 -- 172.21.15.45:0/937679098 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 20) ==== 138663+0+0 (secure 0 0 0) 0x7f3c6c005540 con 0x7f3c74105e30 2024-04-16T17:06:44.814 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:44.814+0000 7f3c6be00640 1 --2- 172.21.15.45:0/937679098 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f3c580a3c00 0x7f3c580a60c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:44.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:44.814+0000 7f3c73400640 1 -- 172.21.15.45:0/937679098 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f3c580a3c00 msgr2=0x7f3c580a60c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.45:6800/268030699 2024-04-16T17:06:44.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:44.815+0000 7f3c6be00640 1 -- 172.21.15.45:0/937679098 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9150+0+0 (secure 0 0 0) 0x7f3c6c0d13e0 con 0x7f3c74105e30 2024-04-16T17:06:44.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:44.815+0000 7f3c73400640 1 --2- 172.21.15.45:0/937679098 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f3c580a3c00 0x7f3c580a60c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-16T17:06:44.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:44.821+0000 7f3c6be00640 1 -- 172.21.15.45:0/937679098 <== mon.2 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3c6c09d9d0 con 0x7f3c74105e30 2024-04-16T17:06:44.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:06:44 smithi104 ceph-mon[36457]: from='client.? 172.21.15.45:0/48305014' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-04-16T17:06:44.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:06:44 smithi104 ceph-mon[36457]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-04-16T17:06:44.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:06:44 smithi045 ceph-mon[31676]: from='client.? 172.21.15.45:0/48305014' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-04-16T17:06:44.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:06:44 smithi045 ceph-mon[31676]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-04-16T17:06:44.977 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 16 17:06:44 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-x[31905]: 2024-04-16T17:06:44.634+0000 7f6d5eb50200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-16T17:06:44.978 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 16 17:06:44 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-x[31905]: 2024-04-16T17:06:44.776+0000 7f6d5eb50200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-16T17:06:45.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:45.016+0000 7f3c73400640 1 -- 172.21.15.45:0/937679098 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f3c580a3c00 msgr2=0x7f3c580a60c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.45:6800/268030699 2024-04-16T17:06:45.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:45.016+0000 7f3c73400640 1 --2- 172.21.15.45:0/937679098 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f3c580a3c00 0x7f3c580a60c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-16T17:06:45.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:45.033+0000 7f3c7a600640 1 -- 172.21.15.45:0/937679098 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3c74062c00 con 0x7f3c74105e30 2024-04-16T17:06:45.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:45.036+0000 7f3c6be00640 1 -- 172.21.15.45:0/937679098 <== mon.2 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) ==== 74+0+24726 (secure 0 0 0) 0x7f3c6c09d2d0 con 0x7f3c74105e30 2024-04-16T17:06:45.036 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-16T17:06:45.036 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":94,"fsid":"b1fecd38-fc11-11ee-bc90-c7b262605968","created":"2024-04-16T16:54:02.385906+0000","modified":"2024-04-16T17:05:22.272335+0000","last_up_change":"2024-04-16T17:02:25.162624+0000","last_in_change":"2024-04-16T17:02:04.903274+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-16T16:58:06.166659+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-16T17:03:22.304198+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-16T17:03:23.616450+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-16T17:03:27.226289+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b655de2c-59f8-4fd4-ac98-0cc222229f56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":2290170881},{"type":"v1","addr":"172.21.15.45:6803","nonce":2290170881}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":2290170881},{"type":"v1","addr":"172.21.15.45:6805","nonce":2290170881}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":2290170881},{"type":"v1","addr":"172.21.15.45:6809","nonce":2290170881}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":2290170881},{"type":"v1","addr":"172.21.15.45:6807","nonce":2290170881}]},"public_addr":"172.21.15.45:6803/2290170881","cluster_addr":"172.21.15.45:6805/2290170881","heartbeat_back_addr":"172.21.15.45:6809/2290170881","heartbeat_front_addr":"172.21.15.45:6807/2290170881","state":["exists","up"]},{"osd":1,"uuid":"ebd450ec-08fd-4c3d-8b24-4e10634ddb7e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6800","nonce":1462942067},{"type":"v1","addr":"172.21.15.104:6801","nonce":1462942067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":1462942067},{"type":"v1","addr":"172.21.15.104:6803","nonce":1462942067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":1462942067},{"type":"v1","addr":"172.21.15.104:6807","nonce":1462942067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":1462942067},{"type":"v1","addr":"172.21.15.104:6805","nonce":1462942067}]},"public_addr":"172.21.15.104:6801/1462942067","cluster_addr":"172.21.15.104:6803/1462942067","heartbeat_back_addr":"172.21.15.104:6807/1462942067","heartbeat_front_addr":"172.21.15.104:6805/1462942067","state":["exists","up"]},{"osd":2,"uuid":"b318a512-4190-42a2-8104-dcc4026ad8ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":2829260735},{"type":"v1","addr":"172.21.15.117:6801","nonce":2829260735}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":2829260735},{"type":"v1","addr":"172.21.15.117:6803","nonce":2829260735}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":2829260735},{"type":"v1","addr":"172.21.15.117:6807","nonce":2829260735}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":2829260735},{"type":"v1","addr":"172.21.15.117:6805","nonce":2829260735}]},"public_addr":"172.21.15.117:6801/2829260735","cluster_addr":"172.21.15.117:6803/2829260735","heartbeat_back_addr":"172.21.15.117:6807/2829260735","heartbeat_front_addr":"172.21.15.117:6805/2829260735","state":["exists","up"]},{"osd":3,"uuid":"ec6490cb-a715-4c12-a54a-225aae26fcb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":2823712137},{"type":"v1","addr":"172.21.15.45:6811","nonce":2823712137}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":2823712137},{"type":"v1","addr":"172.21.15.45:6813","nonce":2823712137}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":2823712137},{"type":"v1","addr":"172.21.15.45:6817","nonce":2823712137}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":2823712137},{"type":"v1","addr":"172.21.15.45:6815","nonce":2823712137}]},"public_addr":"172.21.15.45:6811/2823712137","cluster_addr":"172.21.15.45:6813/2823712137","heartbeat_back_addr":"172.21.15.45:6817/2823712137","heartbeat_front_addr":"172.21.15.45:6815/2823712137","state":["exists","up"]},{"osd":4,"uuid":"5d91d568-0250-4aae-8dc2-a0e079f92062","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":1396278127},{"type":"v1","addr":"172.21.15.104:6809","nonce":1396278127}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":1396278127},{"type":"v1","addr":"172.21.15.104:6811","nonce":1396278127}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":1396278127},{"type":"v1","addr":"172.21.15.104:6815","nonce":1396278127}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":1396278127},{"type":"v1","addr":"172.21.15.104:6813","nonce":1396278127}]},"public_addr":"172.21.15.104:6809/1396278127","cluster_addr":"172.21.15.104:6811/1396278127","heartbeat_back_addr":"172.21.15.104:6815/1396278127","heartbeat_front_addr":"172.21.15.104:6813/1396278127","state":["exists","up"]},{"osd":5,"uuid":"ac510b43-6fb2-4d0f-bc5f-16ac080d9e9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":4145720047},{"type":"v1","addr":"172.21.15.117:6809","nonce":4145720047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":4145720047},{"type":"v1","addr":"172.21.15.117:6811","nonce":4145720047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":4145720047},{"type":"v1","addr":"172.21.15.117:6815","nonce":4145720047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":4145720047},{"type":"v1","addr":"172.21.15.117:6813","nonce":4145720047}]},"public_addr":"172.21.15.117:6809/4145720047","cluster_addr":"172.21.15.117:6811/4145720047","heartbeat_back_addr":"172.21.15.117:6815/4145720047","heartbeat_front_addr":"172.21.15.117:6813/4145720047","state":["exists","up"]},{"osd":6,"uuid":"a41adc18-d88a-419e-b2c2-b089c1f32472","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":3254229176},{"type":"v1","addr":"172.21.15.45:6819","nonce":3254229176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":3254229176},{"type":"v1","addr":"172.21.15.45:6821","nonce":3254229176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":3254229176},{"type":"v1","addr":"172.21.15.45:6825","nonce":3254229176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":3254229176},{"type":"v1","addr":"172.21.15.45:6823","nonce":3254229176}]},"public_addr":"172.21.15.45:6819/3254229176","cluster_addr":"172.21.15.45:6821/3254229176","heartbeat_back_addr":"172.21.15.45:6825/3254229176","heartbeat_front_addr":"172.21.15.45:6823/3254229176","state":["exists","up"]},{"osd":7,"uuid":"b71712a4-7cf6-454d-9fd1-565e3df21c1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":4151077186},{"type":"v1","addr":"172.21.15.104:6817","nonce":4151077186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":4151077186},{"type":"v1","addr":"172.21.15.104:6819","nonce":4151077186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":4151077186},{"type":"v1","addr":"172.21.15.104:6823","nonce":4151077186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":4151077186},{"type":"v1","addr":"172.21.15.104:6821","nonce":4151077186}]},"public_addr":"172.21.15.104:6817/4151077186","cluster_addr":"172.21.15.104:6819/4151077186","heartbeat_back_addr":"172.21.15.104:6823/4151077186","heartbeat_front_addr":"172.21.15.104:6821/4151077186","state":["exists","up"]},{"osd":8,"uuid":"9e572aef-59a8-450e-a92e-df9dff4d57a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":1500286830},{"type":"v1","addr":"172.21.15.117:6817","nonce":1500286830}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":1500286830},{"type":"v1","addr":"172.21.15.117:6819","nonce":1500286830}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":1500286830},{"type":"v1","addr":"172.21.15.117:6823","nonce":1500286830}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":1500286830},{"type":"v1","addr":"172.21.15.117:6821","nonce":1500286830}]},"public_addr":"172.21.15.117:6817/1500286830","cluster_addr":"172.21.15.117:6819/1500286830","heartbeat_back_addr":"172.21.15.117:6823/1500286830","heartbeat_front_addr":"172.21.15.117:6821/1500286830","state":["exists","up"]},{"osd":9,"uuid":"014e9598-c413-4cc4-9bb8-61c6e8f59d30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":3106734074},{"type":"v1","addr":"172.21.15.45:6827","nonce":3106734074}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":3106734074},{"type":"v1","addr":"172.21.15.45:6829","nonce":3106734074}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":3106734074},{"type":"v1","addr":"172.21.15.45:6833","nonce":3106734074}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":3106734074},{"type":"v1","addr":"172.21.15.45:6831","nonce":3106734074}]},"public_addr":"172.21.15.45:6827/3106734074","cluster_addr":"172.21.15.45:6829/3106734074","heartbeat_back_addr":"172.21.15.45:6833/3106734074","heartbeat_front_addr":"172.21.15.45:6831/3106734074","state":["exists","up"]},{"osd":10,"uuid":"2deba364-3e39-47be-8a1f-8256df5b82d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":1404794652},{"type":"v1","addr":"172.21.15.104:6825","nonce":1404794652}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":1404794652},{"type":"v1","addr":"172.21.15.104:6827","nonce":1404794652}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":1404794652},{"type":"v1","addr":"172.21.15.104:6831","nonce":1404794652}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":1404794652},{"type":"v1","addr":"172.21.15.104:6829","nonce":1404794652}]},"public_addr":"172.21.15.104:6825/1404794652","cluster_addr":"172.21.15.104:6827/1404794652","heartbeat_back_addr":"172.21.15.104:6831/1404794652","heartbeat_front_addr":"172.21.15.104:6829/1404794652","state":["exists","up"]},{"osd":11,"uuid":"3fda4fea-59ad-4d1a-9fe6-fcb30e31fd7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":3384003243},{"type":"v1","addr":"172.21.15.117:6825","nonce":3384003243}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":3384003243},{"type":"v1","addr":"172.21.15.117:6827","nonce":3384003243}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":3384003243},{"type":"v1","addr":"172.21.15.117:6831","nonce":3384003243}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":3384003243},{"type":"v1","addr":"172.21.15.117:6829","nonce":3384003243}]},"public_addr":"172.21.15.117:6825/3384003243","cluster_addr":"172.21.15.117:6827/3384003243","heartbeat_back_addr":"172.21.15.117:6831/3384003243","heartbeat_front_addr":"172.21.15.117:6829/3384003243","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:57:14.799608+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:57:39.884668+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:58:03.327057+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:58:37.314531+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:59:03.667650+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:59:28.462823+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T17:00:02.968574+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T17:00:29.559190+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T17:00:55.291007+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T17:01:29.815977+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T17:01:57.015580+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T17:02:23.233151+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.5","mappings":[{"from":9,"to":10}]},{"pgid":"4.7","mappings":[{"from":5,"to":8}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.45:0/4074982598":"2024-04-17T16:55:19.948001+0000","172.21.15.45:0/2443572576":"2024-04-17T16:55:19.948001+0000","172.21.15.45:6801/1983630136":"2024-04-17T16:55:19.948001+0000","172.21.15.45:6800/1983630136":"2024-04-17T16:55:19.948001+0000","172.21.15.45:0/3150465248":"2024-04-17T16:54:43.692466+0000","172.21.15.45:0/556738623":"2024-04-17T16:54:43.692466+0000","172.21.15.45:0/445676297":"2024-04-17T16:55:19.948001+0000","172.21.15.45:6800/2553659622":"2024-04-17T16:54:43.692466+0000","172.21.15.45:6801/2553659622":"2024-04-17T16:54:43.692466+0000","172.21.15.45:0/1184615408":"2024-04-17T16:54:43.692466+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-16T17:06:45.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:45.038+0000 7f3c7a600640 1 -- 172.21.15.45:0/937679098 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f3c580a3c00 msgr2=0x7f3c580a60c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-16T17:06:45.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:45.038+0000 7f3c7a600640 1 --2- 172.21.15.45:0/937679098 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f3c580a3c00 0x7f3c580a60c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:45.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:45.038+0000 7f3c7a600640 1 -- 172.21.15.45:0/937679098 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3c74105e30 msgr2=0x7f3c741a1b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:45.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:45.038+0000 7f3c7a600640 1 --2- 172.21.15.45:0/937679098 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3c74105e30 0x7f3c741a1b40 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f3c6c009fd0 tx=0x7f3c6c00ef10 comp rx=0 tx=0).stop 2024-04-16T17:06:45.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:45.038+0000 7f3c7a600640 1 -- 172.21.15.45:0/937679098 shutdown_connections 2024-04-16T17:06:45.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:45.038+0000 7f3c7a600640 1 --2- 172.21.15.45:0/937679098 >> [v2:172.21.15.45:6800/268030699,v1:172.21.15.45:6801/268030699] conn(0x7f3c580a3c00 0x7f3c580a60c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:45.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:45.038+0000 7f3c7a600640 1 --2- 172.21.15.45:0/937679098 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3c74105e30 0x7f3c741a1b40 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:45.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:45.038+0000 7f3c7a600640 1 --2- 172.21.15.45:0/937679098 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3c74105470 0x7f3c7419aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:45.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:45.038+0000 7f3c7a600640 1 --2- 172.21.15.45:0/937679098 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3c74104280 0x7f3c7419a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:45.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:45.038+0000 7f3c7a600640 1 -- 172.21.15.45:0/937679098 >> 172.21.15.45:0/937679098 conn(0x7f3c740ff7f0 msgr2=0x7f3c74108770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:06:45.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:45.038+0000 7f3c7a600640 1 -- 172.21.15.45:0/937679098 shutdown_connections 2024-04-16T17:06:45.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:45.039+0000 7f3c7a600640 1 -- 172.21.15.45:0/937679098 wait complete. 2024-04-16T17:06:45.058 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-16T16:58:06.166659+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-16T17:03:22.304198+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-16T17:03:23.616450+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-16T17:03:27.226289+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-16T17:06:45.058 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-16T17:06:45.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:45.274+0000 7fa1ac200640 1 -- 172.21.15.45:0/2753722885 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa1a4105450 msgr2=0x7fa1a41058d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:45.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:45.274+0000 7fa1ac200640 1 --2- 172.21.15.45:0/2753722885 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa1a4105450 0x7fa1a41058d0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fa190009900 tx=0x7fa19002bc30 comp rx=0 tx=0).stop 2024-04-16T17:06:45.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:45.274+0000 7fa1ac200640 1 -- 172.21.15.45:0/2753722885 shutdown_connections 2024-04-16T17:06:45.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:45.274+0000 7fa1ac200640 1 --2- 172.21.15.45:0/2753722885 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa1a4105e10 0x7fa1a410c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:45.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:45.274+0000 7fa1ac200640 1 --2- 172.21.15.45:0/2753722885 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa1a4105450 0x7fa1a41058d0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:45.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:45.274+0000 7fa1ac200640 1 --2- 172.21.15.45:0/2753722885 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa1a4104260 0x7fa1a4104680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:45.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:45.274+0000 7fa1ac200640 1 -- 172.21.15.45:0/2753722885 >> 172.21.15.45:0/2753722885 conn(0x7fa1a40ff7f0 msgr2=0x7fa1a4101c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:06:45.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:45.274+0000 7fa1ac200640 1 -- 172.21.15.45:0/2753722885 shutdown_connections 2024-04-16T17:06:45.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:45.274+0000 7fa1ac200640 1 -- 172.21.15.45:0/2753722885 wait complete. 2024-04-16T17:06:45.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:45.275+0000 7fa1ac200640 1 Processor -- start 2024-04-16T17:06:45.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:45.275+0000 7fa1ac200640 1 -- start start 2024-04-16T17:06:45.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:45.276+0000 7fa1ac200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa1a4104260 0x7fa1a4079db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:45.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:45.276+0000 7fa1ac200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa1a4105450 0x7fa1a407a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:45.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:45.276+0000 7fa1ac200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa1a4105e10 0x7fa1a4076920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:45.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:45.276+0000 7fa1ac200640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fa1a410ff40 con 0x7fa1a4104260 2024-04-16T17:06:45.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:45.276+0000 7fa1ac200640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fa1a410fdc0 con 0x7fa1a4105e10 2024-04-16T17:06:45.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:45.276+0000 7fa1ac200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa1a41100c0 con 0x7fa1a4105450 2024-04-16T17:06:45.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:45.276+0000 7fa1aa200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa1a4105e10 0x7fa1a4076920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:45.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:45.276+0000 7fa1aa200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa1a4105e10 0x7fa1a4076920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:56226/0 (socket says 172.21.15.45:56226) 2024-04-16T17:06:45.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:45.276+0000 7fa1aa200640 1 -- 172.21.15.45:0/1952169184 learned_addr learned my addr 172.21.15.45:0/1952169184 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:06:45.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:45.276+0000 7fa1a8e00640 1 --2- 172.21.15.45:0/1952169184 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa1a4105450 0x7fa1a407a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:45.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:45.276+0000 7fa1a9800640 1 --2- 172.21.15.45:0/1952169184 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa1a4104260 0x7fa1a4079db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:45.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:45.276+0000 7fa1aa200640 1 -- 172.21.15.45:0/1952169184 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa1a4105450 msgr2=0x7fa1a407a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:45.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:45.277+0000 7fa1aa200640 1 --2- 172.21.15.45:0/1952169184 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa1a4105450 0x7fa1a407a2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:45.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:45.277+0000 7fa1aa200640 1 -- 172.21.15.45:0/1952169184 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa1a4104260 msgr2=0x7fa1a4079db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:45.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:45.277+0000 7fa1aa200640 1 --2- 172.21.15.45:0/1952169184 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa1a4104260 0x7fa1a4079db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:45.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:45.277+0000 7fa1aa200640 1 -- 172.21.15.45:0/1952169184 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1a40771e0 con 0x7fa1a4105e10 2024-04-16T17:06:45.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:45.277+0000 7fa1a9800640 1 --2- 172.21.15.45:0/1952169184 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa1a4104260 0x7fa1a4079db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:06:45.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:45.277+0000 7fa1aa200640 1 --2- 172.21.15.45:0/1952169184 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa1a4105e10 0x7fa1a4076920 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fa19c00eb10 tx=0x7fa19c00c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:06:45.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:45.277+0000 7fa1a2000640 1 -- 172.21.15.45:0/1952169184 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa19c004b20 con 0x7fa1a4105e10 2024-04-16T17:06:45.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:45.277+0000 7fa1a2000640 1 -- 172.21.15.45:0/1952169184 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa19c004cc0 con 0x7fa1a4105e10 2024-04-16T17:06:45.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:45.277+0000 7fa1ac200640 1 -- 172.21.15.45:0/1952169184 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1a41a6960 con 0x7fa1a4105e10 2024-04-16T17:06:45.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:45.277+0000 7fa1a2000640 1 -- 172.21.15.45:0/1952169184 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa19c010760 con 0x7fa1a4105e10 2024-04-16T17:06:45.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:45.278+0000 7fa1ac200640 1 -- 172.21.15.45:0/1952169184 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1a41a6dc0 con 0x7fa1a4105e10 2024-04-16T17:06:45.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:45.279+0000 7fa1ac200640 1 -- 172.21.15.45:0/1952169184 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1a410b520 con 0x7fa1a4105e10 2024-04-16T17:06:45.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:45.282+0000 7fa1a2000640 1 -- 172.21.15.45:0/1952169184 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 21) ==== 138331+0+0 (secure 0 0 0) 0x7fa19c010900 con 0x7fa1a4105e10 2024-04-16T17:06:45.284 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:45.284+0000 7fa1a2000640 1 -- 172.21.15.45:0/1952169184 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fa19c0d0dd0 con 0x7fa1a4105e10 2024-04-16T17:06:45.284 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:45.284+0000 7fa1a2000640 1 -- 172.21.15.45:0/1952169184 <== mon.2 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa19c026080 con 0x7fa1a4105e10 2024-04-16T17:06:45.338 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 16 17:06:45 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-x[31905]: 2024-04-16T17:06:45.089+0000 7f6d5eb50200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-16T17:06:45.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:45.534+0000 7fa1ac200640 1 -- 172.21.15.45:0/1952169184 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fa1a402d620 con 0x7fa1a4105e10 2024-04-16T17:06:45.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:45.535+0000 7fa1a2000640 1 -- 172.21.15.45:0/1952169184 <== mon.2 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v95) ==== 93+0+10 (secure 0 0 0) 0x7fa19c09d6e0 con 0x7fa1a4105e10 2024-04-16T17:06:45.535 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 1 2024-04-16T17:06:45.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:45.538+0000 7fa1ac200640 1 -- 172.21.15.45:0/1952169184 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa1a4105e10 msgr2=0x7fa1a4076920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:45.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:45.538+0000 7fa1ac200640 1 --2- 172.21.15.45:0/1952169184 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa1a4105e10 0x7fa1a4076920 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fa19c00eb10 tx=0x7fa19c00c2d0 comp rx=0 tx=0).stop 2024-04-16T17:06:45.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:45.538+0000 7fa1ac200640 1 -- 172.21.15.45:0/1952169184 shutdown_connections 2024-04-16T17:06:45.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:45.538+0000 7fa1ac200640 1 --2- 172.21.15.45:0/1952169184 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa1a4105450 0x7fa1a407a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:45.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:45.538+0000 7fa1ac200640 1 --2- 172.21.15.45:0/1952169184 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa1a4105e10 0x7fa1a4076920 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:45.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:45.538+0000 7fa1ac200640 1 --2- 172.21.15.45:0/1952169184 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa1a4104260 0x7fa1a4079db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:45.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:45.538+0000 7fa1ac200640 1 -- 172.21.15.45:0/1952169184 >> 172.21.15.45:0/1952169184 conn(0x7fa1a40ff7f0 msgr2=0x7fa1a41086f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:06:45.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:45.538+0000 7fa1ac200640 1 -- 172.21.15.45:0/1952169184 shutdown_connections 2024-04-16T17:06:45.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:45.538+0000 7fa1ac200640 1 -- 172.21.15.45:0/1952169184 wait complete. 2024-04-16T17:06:45.560 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-16T17:06:45.626 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 16 17:06:45 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-x[31905]: 2024-04-16T17:06:45.433+0000 7f6cb1400700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-16T17:06:45.626 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 16 17:06:45 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-x[31905]: 2024-04-16T17:06:45.433+0000 7f6cb1400700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-16T17:06:45.626 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 16 17:06:45 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-x[31905]: 2024-04-16T17:06:45.433+0000 7f6cb1400700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-16T17:06:45.626 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 16 17:06:45 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-x[31905]: 2024-04-16T17:06:45.433+0000 7f6cb1400700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-16T17:06:45.626 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 16 17:06:45 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-x[31905]: 2024-04-16T17:06:45.433+0000 7f6cb1400700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-16T17:06:45.626 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 16 17:06:45 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-x[31905]: 2024-04-16T17:06:45.434+0000 7f6cb4600700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-16T17:06:45.626 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 16 17:06:45 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-x[31905]: 2024-04-16T17:06:45.434+0000 7f6cb4600700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-16T17:06:45.626 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 16 17:06:45 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-x[31905]: 2024-04-16T17:06:45.434+0000 7f6cb4600700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-16T17:06:45.626 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 16 17:06:45 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-x[31905]: 2024-04-16T17:06:45.434+0000 7f6cb4600700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-16T17:06:45.626 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 16 17:06:45 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-x[31905]: 2024-04-16T17:06:45.434+0000 7f6cb4600700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-16T17:06:45.627 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:06:45 smithi045 ceph-mon[31676]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-04-16T17:06:45.627 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:06:45 smithi045 ceph-mon[31676]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-16T17:06:45.627 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:06:45 smithi045 ceph-mon[31676]: from='client.? 172.21.15.45:0/937679098' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T17:06:45.627 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:06:45 smithi045 ceph-mon[31676]: Active manager daemon x restarted 2024-04-16T17:06:45.627 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:06:45 smithi045 ceph-mon[31676]: Activating manager daemon x 2024-04-16T17:06:45.627 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:06:45 smithi045 ceph-mon[31676]: osdmap e95: 12 total, 12 up, 12 in 2024-04-16T17:06:45.627 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:06:45 smithi045 ceph-mon[31676]: mgrmap e21: x(active, starting, since 0.013651s), standbys: z, y 2024-04-16T17:06:45.627 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:06:45 smithi045 ceph-mon[31676]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-16T17:06:45.627 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:06:45 smithi045 ceph-mon[31676]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-16T17:06:45.627 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:06:45 smithi045 ceph-mon[31676]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-16T17:06:45.627 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:06:45 smithi045 ceph-mon[31676]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-16T17:06:45.628 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:06:45 smithi045 ceph-mon[31676]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-16T17:06:45.628 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:06:45 smithi045 ceph-mon[31676]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-16T17:06:45.628 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:06:45 smithi045 ceph-mon[31676]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-16T17:06:45.628 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:06:45 smithi045 ceph-mon[31676]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-16T17:06:45.628 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:06:45 smithi045 ceph-mon[31676]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-16T17:06:45.628 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:06:45 smithi045 ceph-mon[31676]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-16T17:06:45.628 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:06:45 smithi045 ceph-mon[31676]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-16T17:06:45.628 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:06:45 smithi045 ceph-mon[31676]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-16T17:06:45.628 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:06:45 smithi045 ceph-mon[31676]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-16T17:06:45.628 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:06:45 smithi045 ceph-mon[31676]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-16T17:06:45.628 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:06:45 smithi045 ceph-mon[31676]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-16T17:06:45.628 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:06:45 smithi045 ceph-mon[31676]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-16T17:06:45.628 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:06:45 smithi045 ceph-mon[31676]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-16T17:06:45.628 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:06:45 smithi045 ceph-mon[31676]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-16T17:06:45.628 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:06:45 smithi045 ceph-mon[31676]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-16T17:06:45.629 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:06:45 smithi045 ceph-mon[31676]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-16T17:06:45.629 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:06:45 smithi045 ceph-mon[31676]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-16T17:06:45.629 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:06:45 smithi045 ceph-mon[31676]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-16T17:06:45.629 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:06:45 smithi045 ceph-mon[31676]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-16T17:06:45.629 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:06:45 smithi045 ceph-mon[31676]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-16T17:06:45.629 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:06:45 smithi045 ceph-mon[31676]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-16T17:06:45.629 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:06:45 smithi045 ceph-mon[31676]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-16T17:06:45.629 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:06:45 smithi045 ceph-mon[31676]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-16T17:06:45.629 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:06:45 smithi045 ceph-mon[31676]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-16T17:06:45.629 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:06:45 smithi045 ceph-mon[31676]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-16T17:06:45.629 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:06:45 smithi045 ceph-mon[31676]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-16T17:06:45.629 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:06:45 smithi045 ceph-mon[31676]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-16T17:06:45.629 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:06:45 smithi045 ceph-mon[31676]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-16T17:06:45.629 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:06:45 smithi045 ceph-mon[31676]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-16T17:06:45.629 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:06:45 smithi045 ceph-mon[31676]: Manager daemon x is now available 2024-04-16T17:06:45.630 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:06:45 smithi045 ceph-mon[31676]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T17:06:45.630 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:06:45 smithi045 ceph-mon[31676]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-16T17:06:45.630 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:06:45 smithi045 ceph-mon[31676]: from='mgr.25411 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-16T17:06:45.630 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:06:45 smithi045 ceph-mon[31676]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-16T17:06:45.630 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:06:45 smithi045 ceph-mon[31676]: from='mgr.25411 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-16T17:06:45.630 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:06:45 smithi045 ceph-mon[31676]: from='client.? 172.21.15.45:0/1952169184' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-16T17:06:45.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:06:45 smithi104 ceph-mon[36457]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-04-16T17:06:45.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:06:45 smithi104 ceph-mon[36457]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-16T17:06:45.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:06:45 smithi104 ceph-mon[36457]: from='client.? 172.21.15.45:0/937679098' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T17:06:45.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:06:45 smithi104 ceph-mon[36457]: Active manager daemon x restarted 2024-04-16T17:06:45.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:06:45 smithi104 ceph-mon[36457]: Activating manager daemon x 2024-04-16T17:06:45.679 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:06:45 smithi104 ceph-mon[36457]: osdmap e95: 12 total, 12 up, 12 in 2024-04-16T17:06:45.679 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:06:45 smithi104 ceph-mon[36457]: mgrmap e21: x(active, starting, since 0.013651s), standbys: z, y 2024-04-16T17:06:45.679 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:06:45 smithi104 ceph-mon[36457]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-16T17:06:45.679 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:06:45 smithi104 ceph-mon[36457]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-16T17:06:45.679 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:06:45 smithi104 ceph-mon[36457]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-16T17:06:45.679 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:06:45 smithi104 ceph-mon[36457]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-16T17:06:45.679 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:06:45 smithi104 ceph-mon[36457]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-16T17:06:45.679 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:06:45 smithi104 ceph-mon[36457]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-16T17:06:45.679 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:06:45 smithi104 ceph-mon[36457]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-16T17:06:45.679 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:06:45 smithi104 ceph-mon[36457]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-16T17:06:45.679 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:06:45 smithi104 ceph-mon[36457]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-16T17:06:45.679 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:06:45 smithi104 ceph-mon[36457]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-16T17:06:45.680 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:06:45 smithi104 ceph-mon[36457]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-16T17:06:45.680 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:06:45 smithi104 ceph-mon[36457]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-16T17:06:45.680 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:06:45 smithi104 ceph-mon[36457]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-16T17:06:45.680 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:06:45 smithi104 ceph-mon[36457]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-16T17:06:45.680 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:06:45 smithi104 ceph-mon[36457]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-16T17:06:45.680 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:06:45 smithi104 ceph-mon[36457]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-16T17:06:45.680 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:06:45 smithi104 ceph-mon[36457]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-16T17:06:45.680 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:06:45 smithi104 ceph-mon[36457]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-16T17:06:45.680 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:06:45 smithi104 ceph-mon[36457]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-16T17:06:45.680 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:06:45 smithi104 ceph-mon[36457]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-16T17:06:45.680 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:06:45 smithi104 ceph-mon[36457]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-16T17:06:45.681 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:06:45 smithi104 ceph-mon[36457]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-16T17:06:45.681 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:06:45 smithi104 ceph-mon[36457]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-16T17:06:45.681 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:06:45 smithi104 ceph-mon[36457]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-16T17:06:45.681 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:06:45 smithi104 ceph-mon[36457]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-16T17:06:45.681 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:06:45 smithi104 ceph-mon[36457]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-16T17:06:45.681 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:06:45 smithi104 ceph-mon[36457]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-16T17:06:45.681 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:06:45 smithi104 ceph-mon[36457]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-16T17:06:45.681 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:06:45 smithi104 ceph-mon[36457]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-16T17:06:45.681 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:06:45 smithi104 ceph-mon[36457]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-16T17:06:45.681 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:06:45 smithi104 ceph-mon[36457]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-16T17:06:45.681 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:06:45 smithi104 ceph-mon[36457]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-16T17:06:45.681 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:06:45 smithi104 ceph-mon[36457]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-16T17:06:45.681 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:06:45 smithi104 ceph-mon[36457]: Manager daemon x is now available 2024-04-16T17:06:45.681 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:06:45 smithi104 ceph-mon[36457]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T17:06:45.682 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:06:45 smithi104 ceph-mon[36457]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-16T17:06:45.682 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:06:45 smithi104 ceph-mon[36457]: from='mgr.25411 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-16T17:06:45.682 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:06:45 smithi104 ceph-mon[36457]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-16T17:06:45.682 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:06:45 smithi104 ceph-mon[36457]: from='mgr.25411 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-16T17:06:45.682 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:06:45 smithi104 ceph-mon[36457]: from='client.? 172.21.15.45:0/1952169184' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-16T17:06:45.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:45.757+0000 7fa5d2800640 1 -- 172.21.15.45:0/2875350651 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa5cc1052c0 msgr2=0x7fa5cc1076b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:45.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:45.757+0000 7fa5d2800640 1 --2- 172.21.15.45:0/2875350651 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa5cc1052c0 0x7fa5cc1076b0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fa5b8009900 tx=0x7fa5b802bc30 comp rx=0 tx=0).stop 2024-04-16T17:06:45.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:45.757+0000 7fa5d2800640 1 -- 172.21.15.45:0/2875350651 shutdown_connections 2024-04-16T17:06:45.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:45.757+0000 7fa5d2800640 1 --2- 172.21.15.45:0/2875350651 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa5cc107bf0 0x7fa5cc10a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:45.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:45.757+0000 7fa5d2800640 1 --2- 172.21.15.45:0/2875350651 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa5cc1052c0 0x7fa5cc1076b0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:45.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:45.757+0000 7fa5d2800640 1 --2- 172.21.15.45:0/2875350651 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa5cc102990 0x7fa5cc104d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:45.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:45.757+0000 7fa5d2800640 1 -- 172.21.15.45:0/2875350651 >> 172.21.15.45:0/2875350651 conn(0x7fa5cc0fc3a0 msgr2=0x7fa5cc0fe7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:06:45.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:45.757+0000 7fa5d2800640 1 -- 172.21.15.45:0/2875350651 shutdown_connections 2024-04-16T17:06:45.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:45.757+0000 7fa5d2800640 1 -- 172.21.15.45:0/2875350651 wait complete. 2024-04-16T17:06:45.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:45.758+0000 7fa5d2800640 1 Processor -- start 2024-04-16T17:06:45.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:45.758+0000 7fa5d2800640 1 -- start start 2024-04-16T17:06:45.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:45.758+0000 7fa5d2800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa5cc102990 0x7fa5cc19a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:45.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:45.758+0000 7fa5d2800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa5cc1052c0 0x7fa5cc19a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:45.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:45.758+0000 7fa5d2800640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa5cc107bf0 0x7fa5cc1a1a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:45.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:45.758+0000 7fa5d2800640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fa5cc10dbc0 con 0x7fa5cc102990 2024-04-16T17:06:45.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:45.758+0000 7fa5d2800640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fa5cc10da40 con 0x7fa5cc107bf0 2024-04-16T17:06:45.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:45.758+0000 7fa5d2800640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa5cc10dd40 con 0x7fa5cc1052c0 2024-04-16T17:06:45.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:45.759+0000 7fa5d0a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa5cc1052c0 0x7fa5cc19a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:45.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:45.759+0000 7fa5d0a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa5cc1052c0 0x7fa5cc19a990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:51076/0 (socket says 172.21.15.45:51076) 2024-04-16T17:06:45.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:45.759+0000 7fa5d0a00640 1 -- 172.21.15.45:0/1205474696 learned_addr learned my addr 172.21.15.45:0/1205474696 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:06:45.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:45.759+0000 7fa5d1e00640 1 --2- 172.21.15.45:0/1205474696 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa5cc107bf0 0x7fa5cc1a1a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:45.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:45.759+0000 7fa5d1400640 1 --2- 172.21.15.45:0/1205474696 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa5cc102990 0x7fa5cc19a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:45.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:45.759+0000 7fa5d0a00640 1 -- 172.21.15.45:0/1205474696 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa5cc107bf0 msgr2=0x7fa5cc1a1a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:45.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:45.759+0000 7fa5d0a00640 1 --2- 172.21.15.45:0/1205474696 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa5cc107bf0 0x7fa5cc1a1a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:45.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:45.759+0000 7fa5d0a00640 1 -- 172.21.15.45:0/1205474696 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa5cc102990 msgr2=0x7fa5cc19a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:45.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:45.759+0000 7fa5d0a00640 1 --2- 172.21.15.45:0/1205474696 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa5cc102990 0x7fa5cc19a450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:45.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:45.759+0000 7fa5d0a00640 1 -- 172.21.15.45:0/1205474696 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5cc1a2070 con 0x7fa5cc1052c0 2024-04-16T17:06:45.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:45.759+0000 7fa5d1400640 1 --2- 172.21.15.45:0/1205474696 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa5cc102990 0x7fa5cc19a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:06:45.761 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:45.759+0000 7fa5d0a00640 1 --2- 172.21.15.45:0/1205474696 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa5cc1052c0 0x7fa5cc19a990 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fa5b8000c00 tx=0x7fa5b802c650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:06:45.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:45.760+0000 7fa5ca000640 1 -- 172.21.15.45:0/1205474696 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5b8043020 con 0x7fa5cc1052c0 2024-04-16T17:06:45.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:45.760+0000 7fa5ca000640 1 -- 172.21.15.45:0/1205474696 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa5b80054e0 con 0x7fa5cc1052c0 2024-04-16T17:06:45.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:45.760+0000 7fa5ca000640 1 -- 172.21.15.45:0/1205474696 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5b8003bd0 con 0x7fa5cc1052c0 2024-04-16T17:06:45.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:45.760+0000 7fa5d2800640 1 -- 172.21.15.45:0/1205474696 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5cc1a2300 con 0x7fa5cc1052c0 2024-04-16T17:06:45.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:45.760+0000 7fa5d2800640 1 -- 172.21.15.45:0/1205474696 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5cc1a2800 con 0x7fa5cc1052c0 2024-04-16T17:06:45.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:45.761+0000 7fa5d2800640 1 -- 172.21.15.45:0/1205474696 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5cc109740 con 0x7fa5cc1052c0 2024-04-16T17:06:45.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:45.763+0000 7fa5ca000640 1 -- 172.21.15.45:0/1205474696 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 21) ==== 138331+0+0 (secure 0 0 0) 0x7fa5b8046020 con 0x7fa5cc1052c0 2024-04-16T17:06:45.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:45.765+0000 7fa5ca000640 1 -- 172.21.15.45:0/1205474696 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fa5b80f2340 con 0x7fa5cc1052c0 2024-04-16T17:06:45.768 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:45.768+0000 7fa5ca000640 1 -- 172.21.15.45:0/1205474696 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa5b804d020 con 0x7fa5cc1052c0 2024-04-16T17:06:45.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:06:45 smithi117 ceph-mon[36730]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-04-16T17:06:45.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:06:45 smithi117 ceph-mon[36730]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-16T17:06:45.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:06:45 smithi117 ceph-mon[36730]: from='client.? 172.21.15.45:0/937679098' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T17:06:45.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:06:45 smithi117 ceph-mon[36730]: Active manager daemon x restarted 2024-04-16T17:06:45.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:06:45 smithi117 ceph-mon[36730]: Activating manager daemon x 2024-04-16T17:06:45.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:06:45 smithi117 ceph-mon[36730]: osdmap e95: 12 total, 12 up, 12 in 2024-04-16T17:06:45.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:06:45 smithi117 ceph-mon[36730]: mgrmap e21: x(active, starting, since 0.013651s), standbys: z, y 2024-04-16T17:06:45.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:06:45 smithi117 ceph-mon[36730]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-16T17:06:45.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:06:45 smithi117 ceph-mon[36730]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-16T17:06:45.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:06:45 smithi117 ceph-mon[36730]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-16T17:06:45.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:06:45 smithi117 ceph-mon[36730]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-16T17:06:45.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:06:45 smithi117 ceph-mon[36730]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-16T17:06:45.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:06:45 smithi117 ceph-mon[36730]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-16T17:06:45.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:06:45 smithi117 ceph-mon[36730]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-16T17:06:45.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:06:45 smithi117 ceph-mon[36730]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-16T17:06:45.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:06:45 smithi117 ceph-mon[36730]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-16T17:06:45.807 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:06:45 smithi117 ceph-mon[36730]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-16T17:06:45.807 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:06:45 smithi117 ceph-mon[36730]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-16T17:06:45.807 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:06:45 smithi117 ceph-mon[36730]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-16T17:06:45.807 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:06:45 smithi117 ceph-mon[36730]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-16T17:06:45.807 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:06:45 smithi117 ceph-mon[36730]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-16T17:06:45.807 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:06:45 smithi117 ceph-mon[36730]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-16T17:06:45.807 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:06:45 smithi117 ceph-mon[36730]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-16T17:06:45.807 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:06:45 smithi117 ceph-mon[36730]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-16T17:06:45.807 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:06:45 smithi117 ceph-mon[36730]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-16T17:06:45.807 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:06:45 smithi117 ceph-mon[36730]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-16T17:06:45.807 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:06:45 smithi117 ceph-mon[36730]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-16T17:06:45.807 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:06:45 smithi117 ceph-mon[36730]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-16T17:06:45.807 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:06:45 smithi117 ceph-mon[36730]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-16T17:06:45.807 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:06:45 smithi117 ceph-mon[36730]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-16T17:06:45.807 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:06:45 smithi117 ceph-mon[36730]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-16T17:06:45.808 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:06:45 smithi117 ceph-mon[36730]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-16T17:06:45.808 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:06:45 smithi117 ceph-mon[36730]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-16T17:06:45.808 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:06:45 smithi117 ceph-mon[36730]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-16T17:06:45.808 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:06:45 smithi117 ceph-mon[36730]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-16T17:06:45.808 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:06:45 smithi117 ceph-mon[36730]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-16T17:06:45.808 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:06:45 smithi117 ceph-mon[36730]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-16T17:06:45.808 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:06:45 smithi117 ceph-mon[36730]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-16T17:06:45.808 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:06:45 smithi117 ceph-mon[36730]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-16T17:06:45.808 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:06:45 smithi117 ceph-mon[36730]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-16T17:06:45.808 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:06:45 smithi117 ceph-mon[36730]: Manager daemon x is now available 2024-04-16T17:06:45.808 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:06:45 smithi117 ceph-mon[36730]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T17:06:45.808 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:06:45 smithi117 ceph-mon[36730]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-16T17:06:45.808 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:06:45 smithi117 ceph-mon[36730]: from='mgr.25411 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-16T17:06:45.808 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:06:45 smithi117 ceph-mon[36730]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-16T17:06:45.808 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:06:45 smithi117 ceph-mon[36730]: from='mgr.25411 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-16T17:06:45.809 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:06:45 smithi117 ceph-mon[36730]: from='client.? 172.21.15.45:0/1952169184' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-16T17:06:45.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:45.970+0000 7fa5d2800640 1 -- 172.21.15.45:0/1205474696 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fa5cc062c00 con 0x7fa5cc1052c0 2024-04-16T17:06:45.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:45.971+0000 7fa5ca000640 1 -- 172.21.15.45:0/1205474696 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v95) ==== 104+0+11 (secure 0 0 0) 0x7fa5b80bec50 con 0x7fa5cc1052c0 2024-04-16T17:06:45.971 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 32 2024-04-16T17:06:45.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:45.973+0000 7fa5d2800640 1 -- 172.21.15.45:0/1205474696 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa5cc1052c0 msgr2=0x7fa5cc19a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:45.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:45.973+0000 7fa5d2800640 1 --2- 172.21.15.45:0/1205474696 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa5cc1052c0 0x7fa5cc19a990 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fa5b8000c00 tx=0x7fa5b802c650 comp rx=0 tx=0).stop 2024-04-16T17:06:45.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:45.973+0000 7fa5d2800640 1 -- 172.21.15.45:0/1205474696 shutdown_connections 2024-04-16T17:06:45.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:45.973+0000 7fa5d2800640 1 --2- 172.21.15.45:0/1205474696 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa5cc1052c0 0x7fa5cc19a990 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:45.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:45.973+0000 7fa5d2800640 1 --2- 172.21.15.45:0/1205474696 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa5cc107bf0 0x7fa5cc1a1a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:45.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:45.973+0000 7fa5d2800640 1 --2- 172.21.15.45:0/1205474696 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa5cc102990 0x7fa5cc19a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:45.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:45.973+0000 7fa5d2800640 1 -- 172.21.15.45:0/1205474696 >> 172.21.15.45:0/1205474696 conn(0x7fa5cc0fc3a0 msgr2=0x7fa5cc102090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:06:45.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:45.973+0000 7fa5d2800640 1 -- 172.21.15.45:0/1205474696 shutdown_connections 2024-04-16T17:06:45.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:45.973+0000 7fa5d2800640 1 -- 172.21.15.45:0/1205474696 wait complete. 2024-04-16T17:06:45.991 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-16T17:06:46.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:46.188+0000 7fc11f800640 1 -- 172.21.15.45:0/2068084884 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc118105470 msgr2=0x7fc1181058f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:46.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:46.188+0000 7fc11f800640 1 --2- 172.21.15.45:0/2068084884 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc118105470 0x7fc1181058f0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fc104009900 tx=0x7fc10402bc30 comp rx=0 tx=0).stop 2024-04-16T17:06:46.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:46.188+0000 7fc11f800640 1 -- 172.21.15.45:0/2068084884 shutdown_connections 2024-04-16T17:06:46.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:46.188+0000 7fc11f800640 1 --2- 172.21.15.45:0/2068084884 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc118105470 0x7fc1181058f0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:46.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:46.188+0000 7fc11f800640 1 --2- 172.21.15.45:0/2068084884 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc118105e30 0x7fc11810c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:46.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:46.188+0000 7fc11f800640 1 --2- 172.21.15.45:0/2068084884 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc118104280 0x7fc1181046a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:46.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:46.188+0000 7fc11f800640 1 -- 172.21.15.45:0/2068084884 >> 172.21.15.45:0/2068084884 conn(0x7fc1180ff7f0 msgr2=0x7fc118101c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:06:46.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:46.188+0000 7fc11f800640 1 -- 172.21.15.45:0/2068084884 shutdown_connections 2024-04-16T17:06:46.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:46.188+0000 7fc11f800640 1 -- 172.21.15.45:0/2068084884 wait complete. 2024-04-16T17:06:46.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:46.189+0000 7fc11f800640 1 Processor -- start 2024-04-16T17:06:46.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:46.189+0000 7fc11f800640 1 -- start start 2024-04-16T17:06:46.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:46.189+0000 7fc11f800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc118104280 0x7fc11819a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:46.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:46.189+0000 7fc11f800640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc118105470 0x7fc11819aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:46.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:46.189+0000 7fc11f800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc118105e30 0x7fc1181a1b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:46.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:46.189+0000 7fc11f800640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fc11810ff60 con 0x7fc118104280 2024-04-16T17:06:46.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:46.189+0000 7fc11f800640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fc11810fde0 con 0x7fc118105470 2024-04-16T17:06:46.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:46.189+0000 7fc11f800640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fc1181100e0 con 0x7fc118105e30 2024-04-16T17:06:46.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:46.190+0000 7fc11ce00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc118104280 0x7fc11819a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:46.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:46.190+0000 7fc117e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc118105470 0x7fc11819aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:46.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:46.190+0000 7fc117e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc118105470 0x7fc11819aac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:56326/0 (socket says 172.21.15.45:56326) 2024-04-16T17:06:46.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:46.190+0000 7fc117e00640 1 -- 172.21.15.45:0/3795501444 learned_addr learned my addr 172.21.15.45:0/3795501444 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:06:46.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:46.190+0000 7fc11d800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc118105e30 0x7fc1181a1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:46.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:46.190+0000 7fc117e00640 1 -- 172.21.15.45:0/3795501444 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc118105e30 msgr2=0x7fc1181a1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:46.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:46.190+0000 7fc117e00640 1 --2- 172.21.15.45:0/3795501444 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc118105e30 0x7fc1181a1b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:46.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:46.190+0000 7fc117e00640 1 -- 172.21.15.45:0/3795501444 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc118104280 msgr2=0x7fc11819a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:46.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:46.190+0000 7fc117e00640 1 --2- 172.21.15.45:0/3795501444 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc118104280 0x7fc11819a580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:46.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:46.190+0000 7fc117e00640 1 -- 172.21.15.45:0/3795501444 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1181a21a0 con 0x7fc118105470 2024-04-16T17:06:46.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:46.190+0000 7fc11ce00640 1 --2- 172.21.15.45:0/3795501444 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc118104280 0x7fc11819a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:06:46.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:46.190+0000 7fc117e00640 1 --2- 172.21.15.45:0/3795501444 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc118105470 0x7fc11819aac0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fc104000c00 tx=0x7fc10402c650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:06:46.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:46.191+0000 7fc115600640 1 -- 172.21.15.45:0/3795501444 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc104043020 con 0x7fc118105470 2024-04-16T17:06:46.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:46.191+0000 7fc11f800640 1 -- 172.21.15.45:0/3795501444 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1181a2430 con 0x7fc118105470 2024-04-16T17:06:46.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:46.191+0000 7fc115600640 1 -- 172.21.15.45:0/3795501444 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc1040054e0 con 0x7fc118105470 2024-04-16T17:06:46.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:46.191+0000 7fc115600640 1 -- 172.21.15.45:0/3795501444 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc104003cd0 con 0x7fc118105470 2024-04-16T17:06:46.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:46.191+0000 7fc11f800640 1 -- 172.21.15.45:0/3795501444 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1181a29d0 con 0x7fc118105470 2024-04-16T17:06:46.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:46.192+0000 7fc11f800640 1 -- 172.21.15.45:0/3795501444 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc11810b620 con 0x7fc118105470 2024-04-16T17:06:46.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:46.194+0000 7fc115600640 1 -- 172.21.15.45:0/3795501444 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 22) ==== 138458+0+0 (secure 0 0 0) 0x7fc104046020 con 0x7fc118105470 2024-04-16T17:06:46.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:46.195+0000 7fc115600640 1 --2- 172.21.15.45:0/3795501444 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fc1000a39b0 0x7fc1000a5e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:46.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:46.195+0000 7fc115600640 1 -- 172.21.15.45:0/3795501444 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fc1040f24f0 con 0x7fc118105470 2024-04-16T17:06:46.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:46.195+0000 7fc11ce00640 1 --2- 172.21.15.45:0/3795501444 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fc1000a39b0 0x7fc1000a5e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:46.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:46.196+0000 7fc11ce00640 1 --2- 172.21.15.45:0/3795501444 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fc1000a39b0 0x7fc1000a5e70 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fc1181052d0 tx=0x7fc10c009500 comp rx=0 tx=0).ready entity=mgr.25411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:06:46.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:46.200+0000 7fc115600640 1 -- 172.21.15.45:0/3795501444 <== mon.2 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc10404d020 con 0x7fc118105470 2024-04-16T17:06:46.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:46.411+0000 7fc11f800640 1 -- 172.21.15.45:0/3795501444 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fc118062c00 con 0x7fc118105470 2024-04-16T17:06:46.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:46.412+0000 7fc115600640 1 -- 172.21.15.45:0/3795501444 <== mon.2 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v95) ==== 100+0+11 (secure 0 0 0) 0x7fc1040bed30 con 0x7fc118105470 2024-04-16T17:06:46.412 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 64 2024-04-16T17:06:46.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:46.414+0000 7fc11f800640 1 -- 172.21.15.45:0/3795501444 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fc1000a39b0 msgr2=0x7fc1000a5e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:46.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:46.414+0000 7fc11f800640 1 --2- 172.21.15.45:0/3795501444 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fc1000a39b0 0x7fc1000a5e70 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fc1181052d0 tx=0x7fc10c009500 comp rx=0 tx=0).stop 2024-04-16T17:06:46.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:46.414+0000 7fc11f800640 1 -- 172.21.15.45:0/3795501444 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc118105470 msgr2=0x7fc11819aac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:46.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:46.414+0000 7fc11f800640 1 --2- 172.21.15.45:0/3795501444 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc118105470 0x7fc11819aac0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fc104000c00 tx=0x7fc10402c650 comp rx=0 tx=0).stop 2024-04-16T17:06:46.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:46.414+0000 7fc11f800640 1 -- 172.21.15.45:0/3795501444 shutdown_connections 2024-04-16T17:06:46.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:46.414+0000 7fc11f800640 1 --2- 172.21.15.45:0/3795501444 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fc1000a39b0 0x7fc1000a5e70 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:46.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:46.414+0000 7fc11f800640 1 --2- 172.21.15.45:0/3795501444 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc118105e30 0x7fc1181a1b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:46.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:46.414+0000 7fc11f800640 1 --2- 172.21.15.45:0/3795501444 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc118105470 0x7fc11819aac0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:46.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:46.414+0000 7fc11f800640 1 --2- 172.21.15.45:0/3795501444 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc118104280 0x7fc11819a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:46.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:46.414+0000 7fc11f800640 1 -- 172.21.15.45:0/3795501444 >> 172.21.15.45:0/3795501444 conn(0x7fc1180ff7f0 msgr2=0x7fc1181087f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:06:46.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:46.414+0000 7fc11f800640 1 -- 172.21.15.45:0/3795501444 shutdown_connections 2024-04-16T17:06:46.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:46.414+0000 7fc11f800640 1 -- 172.21.15.45:0/3795501444 wait complete. 2024-04-16T17:06:46.432 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-16T17:06:46.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:46.647+0000 7f0af7000640 1 -- 172.21.15.45:0/491474117 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0af0105470 msgr2=0x7f0af01058f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:46.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:46.647+0000 7f0af7000640 1 --2- 172.21.15.45:0/491474117 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0af0105470 0x7f0af01058f0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f0ad8009900 tx=0x7f0ad802bc30 comp rx=0 tx=0).stop 2024-04-16T17:06:46.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:46.648+0000 7f0af7000640 1 -- 172.21.15.45:0/491474117 shutdown_connections 2024-04-16T17:06:46.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:46.648+0000 7f0af7000640 1 --2- 172.21.15.45:0/491474117 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0af0105e30 0x7f0af010c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:46.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:46.648+0000 7f0af7000640 1 --2- 172.21.15.45:0/491474117 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0af0105470 0x7f0af01058f0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:46.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:46.648+0000 7f0af7000640 1 --2- 172.21.15.45:0/491474117 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0af0104280 0x7f0af01046a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:46.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:46.648+0000 7f0af7000640 1 -- 172.21.15.45:0/491474117 >> 172.21.15.45:0/491474117 conn(0x7f0af00ff7f0 msgr2=0x7f0af0101c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:06:46.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:46.648+0000 7f0af7000640 1 -- 172.21.15.45:0/491474117 shutdown_connections 2024-04-16T17:06:46.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:46.648+0000 7f0af7000640 1 -- 172.21.15.45:0/491474117 wait complete. 2024-04-16T17:06:46.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:46.649+0000 7f0af7000640 1 Processor -- start 2024-04-16T17:06:46.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:46.649+0000 7f0af7000640 1 -- start start 2024-04-16T17:06:46.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:46.649+0000 7f0af7000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0af0104280 0x7f0af019a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:46.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:46.649+0000 7f0af7000640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0af0105470 0x7f0af019aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:46.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:46.649+0000 7f0af7000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0af0105e30 0x7f0af01a1b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:46.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:46.649+0000 7f0af7000640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f0af010ff60 con 0x7f0af0105e30 2024-04-16T17:06:46.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:46.649+0000 7f0af7000640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f0af010fde0 con 0x7f0af0105470 2024-04-16T17:06:46.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:46.649+0000 7f0af7000640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f0af01100e0 con 0x7f0af0104280 2024-04-16T17:06:46.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:46.649+0000 7f0aefe00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0af0104280 0x7f0af019a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:46.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:46.649+0000 7f0aefe00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0af0104280 0x7f0af019a550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:51120/0 (socket says 172.21.15.45:51120) 2024-04-16T17:06:46.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:46.649+0000 7f0aefe00640 1 -- 172.21.15.45:0/4279231666 learned_addr learned my addr 172.21.15.45:0/4279231666 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:06:46.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:46.649+0000 7f0ae7e00640 1 --2- 172.21.15.45:0/4279231666 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0af0105470 0x7f0af019aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:46.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:46.650+0000 7f0af5000640 1 --2- 172.21.15.45:0/4279231666 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0af0105e30 0x7f0af01a1b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:46.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:46.650+0000 7f0aefe00640 1 -- 172.21.15.45:0/4279231666 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0af0105470 msgr2=0x7f0af019aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:46.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:46.650+0000 7f0aefe00640 1 --2- 172.21.15.45:0/4279231666 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0af0105470 0x7f0af019aa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:46.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:46.650+0000 7f0aefe00640 1 -- 172.21.15.45:0/4279231666 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0af0105e30 msgr2=0x7f0af01a1b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:46.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:46.650+0000 7f0aefe00640 1 --2- 172.21.15.45:0/4279231666 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0af0105e30 0x7f0af01a1b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:46.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:46.650+0000 7f0aefe00640 1 -- 172.21.15.45:0/4279231666 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0af01a2170 con 0x7f0af0104280 2024-04-16T17:06:46.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:46.650+0000 7f0aefe00640 1 --2- 172.21.15.45:0/4279231666 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0af0104280 0x7f0af019a550 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f0ae000e9c0 tx=0x7f0ae000ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:06:46.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:46.650+0000 7f0aed600640 1 -- 172.21.15.45:0/4279231666 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ae000cd80 con 0x7f0af0104280 2024-04-16T17:06:46.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:46.650+0000 7f0aed600640 1 -- 172.21.15.45:0/4279231666 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0ae00035f0 con 0x7f0af0104280 2024-04-16T17:06:46.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:46.650+0000 7f0af7000640 1 -- 172.21.15.45:0/4279231666 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0af01a2460 con 0x7f0af0104280 2024-04-16T17:06:46.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:46.650+0000 7f0aed600640 1 -- 172.21.15.45:0/4279231666 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ae0010690 con 0x7f0af0104280 2024-04-16T17:06:46.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:46.651+0000 7f0af7000640 1 -- 172.21.15.45:0/4279231666 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0af01a2980 con 0x7f0af0104280 2024-04-16T17:06:46.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:46.652+0000 7f0af7000640 1 -- 172.21.15.45:0/4279231666 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0af010b5a0 con 0x7f0af0104280 2024-04-16T17:06:46.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:46.653+0000 7f0aed600640 1 -- 172.21.15.45:0/4279231666 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 22) ==== 138458+0+0 (secure 0 0 0) 0x7f0ae00037a0 con 0x7f0af0104280 2024-04-16T17:06:46.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:46.655+0000 7f0aed600640 1 --2- 172.21.15.45:0/4279231666 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f0ad40a3a30 0x7f0ad40a5ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:46.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:46.655+0000 7f0aed600640 1 -- 172.21.15.45:0/4279231666 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f0ae00d0810 con 0x7f0af0104280 2024-04-16T17:06:46.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:46.655+0000 7f0ae7e00640 1 --2- 172.21.15.45:0/4279231666 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f0ad40a3a30 0x7f0ad40a5ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:46.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:46.656+0000 7f0ae7e00640 1 --2- 172.21.15.45:0/4279231666 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f0ad40a3a30 0x7f0ad40a5ef0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f0ad802c240 tx=0x7f0ad802df90 comp rx=0 tx=0).ready entity=mgr.25411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:06:46.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:46.659+0000 7f0aed600640 1 -- 172.21.15.45:0/4279231666 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0ae009d030 con 0x7f0af0104280 2024-04-16T17:06:46.721 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:06:46 smithi117 ceph-mon[36730]: [16/Apr/2024:17:06:45] ENGINE Bus STARTING 2024-04-16T17:06:46.721 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:06:46 smithi117 ceph-mon[36730]: from='client.? 172.21.15.45:0/1205474696' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-16T17:06:46.721 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:06:46 smithi117 ceph-mon[36730]: [16/Apr/2024:17:06:46] ENGINE Serving on https://172.21.15.45:7150 2024-04-16T17:06:46.721 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:06:46 smithi117 ceph-mon[36730]: mgrmap e22: x(active, since 1.02389s), standbys: z, y 2024-04-16T17:06:46.722 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:06:46 smithi117 ceph-mon[36730]: pgmap v3: 129 pgs: 129 active+clean; 1.1 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T17:06:46.722 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:06:46 smithi117 ceph-mon[36730]: [16/Apr/2024:17:06:46] ENGINE Serving on http://172.21.15.45:8765 2024-04-16T17:06:46.722 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:06:46 smithi117 ceph-mon[36730]: [16/Apr/2024:17:06:46] ENGINE Bus STARTED 2024-04-16T17:06:46.722 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:06:46 smithi117 ceph-mon[36730]: from='client.? 172.21.15.45:0/3795501444' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-16T17:06:46.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:06:46 smithi045 ceph-mon[31676]: [16/Apr/2024:17:06:45] ENGINE Bus STARTING 2024-04-16T17:06:46.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:06:46 smithi045 ceph-mon[31676]: from='client.? 172.21.15.45:0/1205474696' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-16T17:06:46.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:06:46 smithi045 ceph-mon[31676]: [16/Apr/2024:17:06:46] ENGINE Serving on https://172.21.15.45:7150 2024-04-16T17:06:46.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:06:46 smithi045 ceph-mon[31676]: mgrmap e22: x(active, since 1.02389s), standbys: z, y 2024-04-16T17:06:46.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:06:46 smithi045 ceph-mon[31676]: pgmap v3: 129 pgs: 129 active+clean; 1.1 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T17:06:46.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:06:46 smithi045 ceph-mon[31676]: [16/Apr/2024:17:06:46] ENGINE Serving on http://172.21.15.45:8765 2024-04-16T17:06:46.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:06:46 smithi045 ceph-mon[31676]: [16/Apr/2024:17:06:46] ENGINE Bus STARTED 2024-04-16T17:06:46.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:06:46 smithi045 ceph-mon[31676]: from='client.? 172.21.15.45:0/3795501444' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-16T17:06:46.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:46.877+0000 7f0af7000640 1 -- 172.21.15.45:0/4279231666 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f0af0062c00 con 0x7f0af0104280 2024-04-16T17:06:46.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:46.878+0000 7f0aed600640 1 -- 172.21.15.45:0/4279231666 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v95) ==== 103+0+11 (secure 0 0 0) 0x7f0ae009c930 con 0x7f0af0104280 2024-04-16T17:06:46.879 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 32 2024-04-16T17:06:46.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:46.881+0000 7f0af7000640 1 -- 172.21.15.45:0/4279231666 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f0ad40a3a30 msgr2=0x7f0ad40a5ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:46.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:46.881+0000 7f0af7000640 1 --2- 172.21.15.45:0/4279231666 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f0ad40a3a30 0x7f0ad40a5ef0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f0ad802c240 tx=0x7f0ad802df90 comp rx=0 tx=0).stop 2024-04-16T17:06:46.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:46.881+0000 7f0af7000640 1 -- 172.21.15.45:0/4279231666 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0af0104280 msgr2=0x7f0af019a550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:46.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:46.881+0000 7f0af7000640 1 --2- 172.21.15.45:0/4279231666 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0af0104280 0x7f0af019a550 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f0ae000e9c0 tx=0x7f0ae000ee80 comp rx=0 tx=0).stop 2024-04-16T17:06:46.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:46.881+0000 7f0af7000640 1 -- 172.21.15.45:0/4279231666 shutdown_connections 2024-04-16T17:06:46.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:46.881+0000 7f0af7000640 1 --2- 172.21.15.45:0/4279231666 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f0ad40a3a30 0x7f0ad40a5ef0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:46.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:46.881+0000 7f0af7000640 1 --2- 172.21.15.45:0/4279231666 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0af0105e30 0x7f0af01a1b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:46.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:46.881+0000 7f0af7000640 1 --2- 172.21.15.45:0/4279231666 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0af0105470 0x7f0af019aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:46.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:46.881+0000 7f0af7000640 1 --2- 172.21.15.45:0/4279231666 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0af0104280 0x7f0af019a550 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:46.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:46.881+0000 7f0af7000640 1 -- 172.21.15.45:0/4279231666 >> 172.21.15.45:0/4279231666 conn(0x7f0af00ff7f0 msgr2=0x7f0af0108770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:06:46.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:46.881+0000 7f0af7000640 1 -- 172.21.15.45:0/4279231666 shutdown_connections 2024-04-16T17:06:46.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:46.881+0000 7f0af7000640 1 -- 172.21.15.45:0/4279231666 wait complete. 2024-04-16T17:06:46.902 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-16T17:06:46.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:06:46 smithi104 ceph-mon[36457]: [16/Apr/2024:17:06:45] ENGINE Bus STARTING 2024-04-16T17:06:46.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:06:46 smithi104 ceph-mon[36457]: from='client.? 172.21.15.45:0/1205474696' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-16T17:06:46.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:06:46 smithi104 ceph-mon[36457]: [16/Apr/2024:17:06:46] ENGINE Serving on https://172.21.15.45:7150 2024-04-16T17:06:46.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:06:46 smithi104 ceph-mon[36457]: mgrmap e22: x(active, since 1.02389s), standbys: z, y 2024-04-16T17:06:46.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:06:46 smithi104 ceph-mon[36457]: pgmap v3: 129 pgs: 129 active+clean; 1.1 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T17:06:46.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:06:46 smithi104 ceph-mon[36457]: [16/Apr/2024:17:06:46] ENGINE Serving on http://172.21.15.45:8765 2024-04-16T17:06:46.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:06:46 smithi104 ceph-mon[36457]: [16/Apr/2024:17:06:46] ENGINE Bus STARTED 2024-04-16T17:06:46.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:06:46 smithi104 ceph-mon[36457]: from='client.? 172.21.15.45:0/3795501444' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-16T17:06:47.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:47.117+0000 7f1b53200640 1 -- 172.21.15.45:0/1190274797 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1b54101ea0 msgr2=0x7f1b541022c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:47.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:47.117+0000 7f1b53200640 1 --2- 172.21.15.45:0/1190274797 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1b54101ea0 0x7f1b541022c0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f1b44009900 tx=0x7f1b4402bc40 comp rx=0 tx=0).stop 2024-04-16T17:06:47.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:47.117+0000 7f1b53200640 1 -- 172.21.15.45:0/1190274797 shutdown_connections 2024-04-16T17:06:47.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:47.117+0000 7f1b53200640 1 --2- 172.21.15.45:0/1190274797 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1b541039b0 0x7f1b5410a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:47.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:47.117+0000 7f1b53200640 1 --2- 172.21.15.45:0/1190274797 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1b54102ff0 0x7f1b54103470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:47.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:47.117+0000 7f1b53200640 1 --2- 172.21.15.45:0/1190274797 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1b54101ea0 0x7f1b541022c0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:47.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:47.117+0000 7f1b53200640 1 -- 172.21.15.45:0/1190274797 >> 172.21.15.45:0/1190274797 conn(0x7f1b540fd410 msgr2=0x7f1b540ff850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:06:47.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:47.117+0000 7f1b53200640 1 -- 172.21.15.45:0/1190274797 shutdown_connections 2024-04-16T17:06:47.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:47.117+0000 7f1b53200640 1 -- 172.21.15.45:0/1190274797 wait complete. 2024-04-16T17:06:47.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:47.118+0000 7f1b53200640 1 Processor -- start 2024-04-16T17:06:47.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:47.119+0000 7f1b53200640 1 -- start start 2024-04-16T17:06:47.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:47.119+0000 7f1b53200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1b54101ea0 0x7f1b541981d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:47.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:47.119+0000 7f1b53200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1b54102ff0 0x7f1b54198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:47.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:47.119+0000 7f1b53200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1b541039b0 0x7f1b5419f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:47.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:47.119+0000 7f1b53200640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f1b5410db90 con 0x7f1b54102ff0 2024-04-16T17:06:47.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:47.119+0000 7f1b53200640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f1b5410da10 con 0x7f1b541039b0 2024-04-16T17:06:47.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:47.119+0000 7f1b53200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f1b5410dd10 con 0x7f1b54101ea0 2024-04-16T17:06:47.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:47.119+0000 7f1b51e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1b54101ea0 0x7f1b541981d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:47.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:47.119+0000 7f1b51e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1b54101ea0 0x7f1b541981d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:51140/0 (socket says 172.21.15.45:51140) 2024-04-16T17:06:47.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:47.119+0000 7f1b51e00640 1 -- 172.21.15.45:0/2097237099 learned_addr learned my addr 172.21.15.45:0/2097237099 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:06:47.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:47.119+0000 7f1b52800640 1 --2- 172.21.15.45:0/2097237099 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1b541039b0 0x7f1b5419f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:47.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:47.119+0000 7f1b51400640 1 --2- 172.21.15.45:0/2097237099 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1b54102ff0 0x7f1b54198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:47.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:47.120+0000 7f1b51e00640 1 -- 172.21.15.45:0/2097237099 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1b541039b0 msgr2=0x7f1b5419f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:47.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:47.120+0000 7f1b51e00640 1 --2- 172.21.15.45:0/2097237099 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1b541039b0 0x7f1b5419f730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:47.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:47.120+0000 7f1b51e00640 1 -- 172.21.15.45:0/2097237099 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1b54102ff0 msgr2=0x7f1b54198710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:47.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:47.120+0000 7f1b51e00640 1 --2- 172.21.15.45:0/2097237099 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1b54102ff0 0x7f1b54198710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:47.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:47.120+0000 7f1b51e00640 1 -- 172.21.15.45:0/2097237099 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b5419fda0 con 0x7f1b54101ea0 2024-04-16T17:06:47.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:47.120+0000 7f1b52800640 1 --2- 172.21.15.45:0/2097237099 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1b541039b0 0x7f1b5419f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:06:47.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:47.120+0000 7f1b51e00640 1 --2- 172.21.15.45:0/2097237099 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1b54101ea0 0x7f1b541981d0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f1b4402c8f0 tx=0x7f1b4402c660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:06:47.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:47.120+0000 7f1b4aa00640 1 -- 172.21.15.45:0/2097237099 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b44041070 con 0x7f1b54101ea0 2024-04-16T17:06:47.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:47.120+0000 7f1b53200640 1 -- 172.21.15.45:0/2097237099 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b541a0030 con 0x7f1b54101ea0 2024-04-16T17:06:47.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:47.120+0000 7f1b4aa00640 1 -- 172.21.15.45:0/2097237099 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1b44004450 con 0x7f1b54101ea0 2024-04-16T17:06:47.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:47.120+0000 7f1b4aa00640 1 -- 172.21.15.45:0/2097237099 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b44034440 con 0x7f1b54101ea0 2024-04-16T17:06:47.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:47.120+0000 7f1b53200640 1 -- 172.21.15.45:0/2097237099 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b541a0570 con 0x7f1b54101ea0 2024-04-16T17:06:47.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:47.122+0000 7f1b53200640 1 -- 172.21.15.45:0/2097237099 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b541091b0 con 0x7f1b54101ea0 2024-04-16T17:06:47.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:47.123+0000 7f1b4aa00640 1 -- 172.21.15.45:0/2097237099 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 22) ==== 138458+0+0 (secure 0 0 0) 0x7f1b44044070 con 0x7f1b54101ea0 2024-04-16T17:06:47.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:47.125+0000 7f1b4aa00640 1 --2- 172.21.15.45:0/2097237099 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f1b380a39b0 0x7f1b380a5e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:47.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:47.125+0000 7f1b4aa00640 1 -- 172.21.15.45:0/2097237099 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f1b440f1fd0 con 0x7f1b54101ea0 2024-04-16T17:06:47.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:47.125+0000 7f1b51400640 1 --2- 172.21.15.45:0/2097237099 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f1b380a39b0 0x7f1b380a5e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:47.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:47.126+0000 7f1b51400640 1 --2- 172.21.15.45:0/2097237099 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f1b380a39b0 0x7f1b380a5e70 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f1b54199690 tx=0x7f1b3c005f90 comp rx=0 tx=0).ready entity=mgr.25411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:06:47.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:47.129+0000 7f1b4aa00640 1 -- 172.21.15.45:0/2097237099 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1b440be1a0 con 0x7f1b54101ea0 2024-04-16T17:06:47.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:47.347+0000 7f1b53200640 1 -- 172.21.15.45:0/2097237099 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1b54062c00 con 0x7f1b54101ea0 2024-04-16T17:06:47.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:47.349+0000 7f1b4aa00640 1 -- 172.21.15.45:0/2097237099 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v95) ==== 74+0+25099 (secure 0 0 0) 0x7f1b440c24a0 con 0x7f1b54101ea0 2024-04-16T17:06:47.350 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-16T17:06:47.350 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":95,"fsid":"b1fecd38-fc11-11ee-bc90-c7b262605968","created":"2024-04-16T16:54:02.385906+0000","modified":"2024-04-16T17:06:45.093262+0000","last_up_change":"2024-04-16T17:02:25.162624+0000","last_in_change":"2024-04-16T17:02:04.903274+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-16T16:58:06.166659+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-16T17:03:22.304198+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-16T17:03:23.616450+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-16T17:03:27.226289+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b655de2c-59f8-4fd4-ac98-0cc222229f56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":2290170881},{"type":"v1","addr":"172.21.15.45:6803","nonce":2290170881}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":2290170881},{"type":"v1","addr":"172.21.15.45:6805","nonce":2290170881}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":2290170881},{"type":"v1","addr":"172.21.15.45:6809","nonce":2290170881}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":2290170881},{"type":"v1","addr":"172.21.15.45:6807","nonce":2290170881}]},"public_addr":"172.21.15.45:6803/2290170881","cluster_addr":"172.21.15.45:6805/2290170881","heartbeat_back_addr":"172.21.15.45:6809/2290170881","heartbeat_front_addr":"172.21.15.45:6807/2290170881","state":["exists","up"]},{"osd":1,"uuid":"ebd450ec-08fd-4c3d-8b24-4e10634ddb7e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6800","nonce":1462942067},{"type":"v1","addr":"172.21.15.104:6801","nonce":1462942067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":1462942067},{"type":"v1","addr":"172.21.15.104:6803","nonce":1462942067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":1462942067},{"type":"v1","addr":"172.21.15.104:6807","nonce":1462942067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":1462942067},{"type":"v1","addr":"172.21.15.104:6805","nonce":1462942067}]},"public_addr":"172.21.15.104:6801/1462942067","cluster_addr":"172.21.15.104:6803/1462942067","heartbeat_back_addr":"172.21.15.104:6807/1462942067","heartbeat_front_addr":"172.21.15.104:6805/1462942067","state":["exists","up"]},{"osd":2,"uuid":"b318a512-4190-42a2-8104-dcc4026ad8ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":2829260735},{"type":"v1","addr":"172.21.15.117:6801","nonce":2829260735}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":2829260735},{"type":"v1","addr":"172.21.15.117:6803","nonce":2829260735}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":2829260735},{"type":"v1","addr":"172.21.15.117:6807","nonce":2829260735}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":2829260735},{"type":"v1","addr":"172.21.15.117:6805","nonce":2829260735}]},"public_addr":"172.21.15.117:6801/2829260735","cluster_addr":"172.21.15.117:6803/2829260735","heartbeat_back_addr":"172.21.15.117:6807/2829260735","heartbeat_front_addr":"172.21.15.117:6805/2829260735","state":["exists","up"]},{"osd":3,"uuid":"ec6490cb-a715-4c12-a54a-225aae26fcb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":2823712137},{"type":"v1","addr":"172.21.15.45:6811","nonce":2823712137}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":2823712137},{"type":"v1","addr":"172.21.15.45:6813","nonce":2823712137}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":2823712137},{"type":"v1","addr":"172.21.15.45:6817","nonce":2823712137}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":2823712137},{"type":"v1","addr":"172.21.15.45:6815","nonce":2823712137}]},"public_addr":"172.21.15.45:6811/2823712137","cluster_addr":"172.21.15.45:6813/2823712137","heartbeat_back_addr":"172.21.15.45:6817/2823712137","heartbeat_front_addr":"172.21.15.45:6815/2823712137","state":["exists","up"]},{"osd":4,"uuid":"5d91d568-0250-4aae-8dc2-a0e079f92062","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":1396278127},{"type":"v1","addr":"172.21.15.104:6809","nonce":1396278127}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":1396278127},{"type":"v1","addr":"172.21.15.104:6811","nonce":1396278127}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":1396278127},{"type":"v1","addr":"172.21.15.104:6815","nonce":1396278127}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":1396278127},{"type":"v1","addr":"172.21.15.104:6813","nonce":1396278127}]},"public_addr":"172.21.15.104:6809/1396278127","cluster_addr":"172.21.15.104:6811/1396278127","heartbeat_back_addr":"172.21.15.104:6815/1396278127","heartbeat_front_addr":"172.21.15.104:6813/1396278127","state":["exists","up"]},{"osd":5,"uuid":"ac510b43-6fb2-4d0f-bc5f-16ac080d9e9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":4145720047},{"type":"v1","addr":"172.21.15.117:6809","nonce":4145720047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":4145720047},{"type":"v1","addr":"172.21.15.117:6811","nonce":4145720047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":4145720047},{"type":"v1","addr":"172.21.15.117:6815","nonce":4145720047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":4145720047},{"type":"v1","addr":"172.21.15.117:6813","nonce":4145720047}]},"public_addr":"172.21.15.117:6809/4145720047","cluster_addr":"172.21.15.117:6811/4145720047","heartbeat_back_addr":"172.21.15.117:6815/4145720047","heartbeat_front_addr":"172.21.15.117:6813/4145720047","state":["exists","up"]},{"osd":6,"uuid":"a41adc18-d88a-419e-b2c2-b089c1f32472","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":3254229176},{"type":"v1","addr":"172.21.15.45:6819","nonce":3254229176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":3254229176},{"type":"v1","addr":"172.21.15.45:6821","nonce":3254229176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":3254229176},{"type":"v1","addr":"172.21.15.45:6825","nonce":3254229176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":3254229176},{"type":"v1","addr":"172.21.15.45:6823","nonce":3254229176}]},"public_addr":"172.21.15.45:6819/3254229176","cluster_addr":"172.21.15.45:6821/3254229176","heartbeat_back_addr":"172.21.15.45:6825/3254229176","heartbeat_front_addr":"172.21.15.45:6823/3254229176","state":["exists","up"]},{"osd":7,"uuid":"b71712a4-7cf6-454d-9fd1-565e3df21c1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":4151077186},{"type":"v1","addr":"172.21.15.104:6817","nonce":4151077186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":4151077186},{"type":"v1","addr":"172.21.15.104:6819","nonce":4151077186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":4151077186},{"type":"v1","addr":"172.21.15.104:6823","nonce":4151077186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":4151077186},{"type":"v1","addr":"172.21.15.104:6821","nonce":4151077186}]},"public_addr":"172.21.15.104:6817/4151077186","cluster_addr":"172.21.15.104:6819/4151077186","heartbeat_back_addr":"172.21.15.104:6823/4151077186","heartbeat_front_addr":"172.21.15.104:6821/4151077186","state":["exists","up"]},{"osd":8,"uuid":"9e572aef-59a8-450e-a92e-df9dff4d57a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":1500286830},{"type":"v1","addr":"172.21.15.117:6817","nonce":1500286830}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":1500286830},{"type":"v1","addr":"172.21.15.117:6819","nonce":1500286830}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":1500286830},{"type":"v1","addr":"172.21.15.117:6823","nonce":1500286830}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":1500286830},{"type":"v1","addr":"172.21.15.117:6821","nonce":1500286830}]},"public_addr":"172.21.15.117:6817/1500286830","cluster_addr":"172.21.15.117:6819/1500286830","heartbeat_back_addr":"172.21.15.117:6823/1500286830","heartbeat_front_addr":"172.21.15.117:6821/1500286830","state":["exists","up"]},{"osd":9,"uuid":"014e9598-c413-4cc4-9bb8-61c6e8f59d30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":3106734074},{"type":"v1","addr":"172.21.15.45:6827","nonce":3106734074}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":3106734074},{"type":"v1","addr":"172.21.15.45:6829","nonce":3106734074}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":3106734074},{"type":"v1","addr":"172.21.15.45:6833","nonce":3106734074}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":3106734074},{"type":"v1","addr":"172.21.15.45:6831","nonce":3106734074}]},"public_addr":"172.21.15.45:6827/3106734074","cluster_addr":"172.21.15.45:6829/3106734074","heartbeat_back_addr":"172.21.15.45:6833/3106734074","heartbeat_front_addr":"172.21.15.45:6831/3106734074","state":["exists","up"]},{"osd":10,"uuid":"2deba364-3e39-47be-8a1f-8256df5b82d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":1404794652},{"type":"v1","addr":"172.21.15.104:6825","nonce":1404794652}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":1404794652},{"type":"v1","addr":"172.21.15.104:6827","nonce":1404794652}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":1404794652},{"type":"v1","addr":"172.21.15.104:6831","nonce":1404794652}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":1404794652},{"type":"v1","addr":"172.21.15.104:6829","nonce":1404794652}]},"public_addr":"172.21.15.104:6825/1404794652","cluster_addr":"172.21.15.104:6827/1404794652","heartbeat_back_addr":"172.21.15.104:6831/1404794652","heartbeat_front_addr":"172.21.15.104:6829/1404794652","state":["exists","up"]},{"osd":11,"uuid":"3fda4fea-59ad-4d1a-9fe6-fcb30e31fd7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":3384003243},{"type":"v1","addr":"172.21.15.117:6825","nonce":3384003243}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":3384003243},{"type":"v1","addr":"172.21.15.117:6827","nonce":3384003243}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":3384003243},{"type":"v1","addr":"172.21.15.117:6831","nonce":3384003243}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":3384003243},{"type":"v1","addr":"172.21.15.117:6829","nonce":3384003243}]},"public_addr":"172.21.15.117:6825/3384003243","cluster_addr":"172.21.15.117:6827/3384003243","heartbeat_back_addr":"172.21.15.117:6831/3384003243","heartbeat_front_addr":"172.21.15.117:6829/3384003243","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:57:14.799608+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:57:39.884668+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:58:03.327057+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:58:37.314531+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:59:03.667650+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:59:28.462823+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T17:00:02.968574+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T17:00:29.559190+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T17:00:55.291007+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T17:01:29.815977+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T17:01:57.015580+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T17:02:23.233151+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.5","mappings":[{"from":9,"to":10}]},{"pgid":"4.7","mappings":[{"from":5,"to":8}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.45:6800/268030699":"2024-04-17T17:06:45.093215+0000","172.21.15.45:6801/268030699":"2024-04-17T17:06:45.093215+0000","172.21.15.45:0/4074982598":"2024-04-17T16:55:19.948001+0000","172.21.15.45:0/1915281176":"2024-04-17T17:06:45.093215+0000","172.21.15.45:0/2443572576":"2024-04-17T16:55:19.948001+0000","172.21.15.45:0/1500147666":"2024-04-17T17:06:45.093215+0000","172.21.15.45:6801/1983630136":"2024-04-17T16:55:19.948001+0000","172.21.15.45:6800/1983630136":"2024-04-17T16:55:19.948001+0000","172.21.15.45:0/12001708":"2024-04-17T17:06:45.093215+0000","172.21.15.45:0/3150465248":"2024-04-17T16:54:43.692466+0000","172.21.15.45:0/556738623":"2024-04-17T16:54:43.692466+0000","172.21.15.45:0/445676297":"2024-04-17T16:55:19.948001+0000","172.21.15.45:6800/2553659622":"2024-04-17T16:54:43.692466+0000","172.21.15.45:6801/2553659622":"2024-04-17T16:54:43.692466+0000","172.21.15.45:0/265066921":"2024-04-17T17:06:45.093215+0000","172.21.15.45:0/1184615408":"2024-04-17T16:54:43.692466+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-16T17:06:47.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:47.352+0000 7f1b53200640 1 -- 172.21.15.45:0/2097237099 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f1b380a39b0 msgr2=0x7f1b380a5e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:47.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:47.352+0000 7f1b53200640 1 --2- 172.21.15.45:0/2097237099 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f1b380a39b0 0x7f1b380a5e70 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f1b54199690 tx=0x7f1b3c005f90 comp rx=0 tx=0).stop 2024-04-16T17:06:47.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:47.352+0000 7f1b53200640 1 -- 172.21.15.45:0/2097237099 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1b54101ea0 msgr2=0x7f1b541981d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:47.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:47.352+0000 7f1b53200640 1 --2- 172.21.15.45:0/2097237099 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1b54101ea0 0x7f1b541981d0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f1b4402c8f0 tx=0x7f1b4402c660 comp rx=0 tx=0).stop 2024-04-16T17:06:47.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:47.352+0000 7f1b53200640 1 -- 172.21.15.45:0/2097237099 shutdown_connections 2024-04-16T17:06:47.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:47.352+0000 7f1b53200640 1 --2- 172.21.15.45:0/2097237099 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f1b380a39b0 0x7f1b380a5e70 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:47.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:47.353+0000 7f1b53200640 1 --2- 172.21.15.45:0/2097237099 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1b541039b0 0x7f1b5419f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:47.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:47.353+0000 7f1b53200640 1 --2- 172.21.15.45:0/2097237099 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1b54102ff0 0x7f1b54198710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:47.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:47.353+0000 7f1b53200640 1 --2- 172.21.15.45:0/2097237099 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1b54101ea0 0x7f1b541981d0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:47.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:47.353+0000 7f1b53200640 1 -- 172.21.15.45:0/2097237099 >> 172.21.15.45:0/2097237099 conn(0x7f1b540fd410 msgr2=0x7f1b54106380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:06:47.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:47.353+0000 7f1b53200640 1 -- 172.21.15.45:0/2097237099 shutdown_connections 2024-04-16T17:06:47.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:47.353+0000 7f1b53200640 1 -- 172.21.15.45:0/2097237099 wait complete. 2024-04-16T17:06:47.373 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-16T16:58:06.166659+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-16T17:03:22.304198+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-16T17:03:23.616450+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-16T17:03:27.226289+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-16T17:06:47.373 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-16T17:06:47.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:47.592+0000 7fbf88a00640 1 -- 172.21.15.45:0/274134762 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbf841051d0 msgr2=0x7fbf841075c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:47.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:47.592+0000 7fbf88a00640 1 --2- 172.21.15.45:0/274134762 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbf841051d0 0x7fbf841075c0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fbf70009900 tx=0x7fbf7002bc30 comp rx=0 tx=0).stop 2024-04-16T17:06:47.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:47.592+0000 7fbf88a00640 1 -- 172.21.15.45:0/274134762 shutdown_connections 2024-04-16T17:06:47.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:47.592+0000 7fbf88a00640 1 --2- 172.21.15.45:0/274134762 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbf84107b00 0x7fbf84109f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:47.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:47.592+0000 7fbf88a00640 1 --2- 172.21.15.45:0/274134762 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbf841051d0 0x7fbf841075c0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:47.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:47.592+0000 7fbf88a00640 1 --2- 172.21.15.45:0/274134762 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbf841028a0 0x7fbf84104c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:47.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:47.592+0000 7fbf88a00640 1 -- 172.21.15.45:0/274134762 >> 172.21.15.45:0/274134762 conn(0x7fbf840fc2d0 msgr2=0x7fbf840fe6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:06:47.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:47.592+0000 7fbf88a00640 1 -- 172.21.15.45:0/274134762 shutdown_connections 2024-04-16T17:06:47.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:47.592+0000 7fbf88a00640 1 -- 172.21.15.45:0/274134762 wait complete. 2024-04-16T17:06:47.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:47.593+0000 7fbf88a00640 1 Processor -- start 2024-04-16T17:06:47.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:47.593+0000 7fbf88a00640 1 -- start start 2024-04-16T17:06:47.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:47.593+0000 7fbf88a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbf841028a0 0x7fbf8419a330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:47.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:47.593+0000 7fbf88a00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbf841051d0 0x7fbf8419a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:47.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:47.593+0000 7fbf88a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbf84107b00 0x7fbf841a18e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:47.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:47.593+0000 7fbf88a00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fbf8410db00 con 0x7fbf84107b00 2024-04-16T17:06:47.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:47.593+0000 7fbf88a00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fbf8410d980 con 0x7fbf841051d0 2024-04-16T17:06:47.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:47.593+0000 7fbf88a00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fbf8410dc80 con 0x7fbf841028a0 2024-04-16T17:06:47.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:47.594+0000 7fbf82a00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbf841051d0 0x7fbf8419a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:47.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:47.594+0000 7fbf83400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbf841028a0 0x7fbf8419a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:47.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:47.594+0000 7fbf82a00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbf841051d0 0x7fbf8419a870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:56396/0 (socket says 172.21.15.45:56396) 2024-04-16T17:06:47.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:47.594+0000 7fbf82a00640 1 -- 172.21.15.45:0/3408903184 learned_addr learned my addr 172.21.15.45:0/3408903184 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:06:47.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:47.594+0000 7fbf83e00640 1 --2- 172.21.15.45:0/3408903184 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbf84107b00 0x7fbf841a18e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:47.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:47.594+0000 7fbf82a00640 1 -- 172.21.15.45:0/3408903184 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbf841028a0 msgr2=0x7fbf8419a330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:47.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:47.594+0000 7fbf82a00640 1 --2- 172.21.15.45:0/3408903184 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbf841028a0 0x7fbf8419a330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:47.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:47.594+0000 7fbf82a00640 1 -- 172.21.15.45:0/3408903184 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbf84107b00 msgr2=0x7fbf841a18e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:47.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:47.594+0000 7fbf82a00640 1 --2- 172.21.15.45:0/3408903184 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbf84107b00 0x7fbf841a18e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:47.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:47.594+0000 7fbf82a00640 1 -- 172.21.15.45:0/3408903184 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf841a1f50 con 0x7fbf841051d0 2024-04-16T17:06:47.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:47.594+0000 7fbf83e00640 1 --2- 172.21.15.45:0/3408903184 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbf84107b00 0x7fbf841a18e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:06:47.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:47.594+0000 7fbf83400640 1 --2- 172.21.15.45:0/3408903184 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbf841028a0 0x7fbf8419a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:06:47.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:47.595+0000 7fbf82a00640 1 --2- 172.21.15.45:0/3408903184 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbf841051d0 0x7fbf8419a870 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fbf70000c00 tx=0x7fbf7002c650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:06:47.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:47.595+0000 7fbf7be00640 1 -- 172.21.15.45:0/3408903184 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf70043020 con 0x7fbf841051d0 2024-04-16T17:06:47.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:47.595+0000 7fbf7be00640 1 -- 172.21.15.45:0/3408903184 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbf700054e0 con 0x7fbf841051d0 2024-04-16T17:06:47.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:47.595+0000 7fbf7be00640 1 -- 172.21.15.45:0/3408903184 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf70003c40 con 0x7fbf841051d0 2024-04-16T17:06:47.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:47.595+0000 7fbf88a00640 1 -- 172.21.15.45:0/3408903184 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf841a21e0 con 0x7fbf841051d0 2024-04-16T17:06:47.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:47.595+0000 7fbf88a00640 1 -- 172.21.15.45:0/3408903184 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf841a26e0 con 0x7fbf841051d0 2024-04-16T17:06:47.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:47.597+0000 7fbf88a00640 1 -- 172.21.15.45:0/3408903184 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf84109650 con 0x7fbf841051d0 2024-04-16T17:06:47.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:47.598+0000 7fbf7be00640 1 -- 172.21.15.45:0/3408903184 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 23) ==== 138766+0+0 (secure 0 0 0) 0x7fbf70046020 con 0x7fbf841051d0 2024-04-16T17:06:47.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:47.599+0000 7fbf7be00640 1 --2- 172.21.15.45:0/3408903184 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fbf680a3da0 0x7fbf680a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:47.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:47.600+0000 7fbf7be00640 1 -- 172.21.15.45:0/3408903184 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fbf700f2a70 con 0x7fbf841051d0 2024-04-16T17:06:47.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:47.600+0000 7fbf83400640 1 --2- 172.21.15.45:0/3408903184 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fbf680a3da0 0x7fbf680a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:47.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:47.601+0000 7fbf83400640 1 --2- 172.21.15.45:0/3408903184 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fbf680a3da0 0x7fbf680a6260 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fbf74009fc0 tx=0x7fbf74005f90 comp rx=0 tx=0).ready entity=mgr.25411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:06:47.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:47.603+0000 7fbf7be00640 1 -- 172.21.15.45:0/3408903184 <== mon.2 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fbf7004d020 con 0x7fbf841051d0 2024-04-16T17:06:47.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:06:47 smithi045 ceph-mon[31676]: from='client.? 172.21.15.45:0/4279231666' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-16T17:06:47.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:06:47 smithi045 ceph-mon[31676]: pgmap v4: 129 pgs: 129 active+clean; 1.1 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T17:06:47.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:06:47 smithi045 ceph-mon[31676]: from='mgr.25411 ' entity='mgr.x' 2024-04-16T17:06:47.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:06:47 smithi045 ceph-mon[31676]: from='mgr.25411 ' entity='mgr.x' 2024-04-16T17:06:47.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:06:47 smithi045 ceph-mon[31676]: from='client.? 172.21.15.45:0/2097237099' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T17:06:47.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:06:47 smithi045 ceph-mon[31676]: from='mgr.25411 ' entity='mgr.x' 2024-04-16T17:06:47.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:06:47 smithi045 ceph-mon[31676]: from='mgr.25411 ' entity='mgr.x' 2024-04-16T17:06:47.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:06:47 smithi117 ceph-mon[36730]: from='client.? 172.21.15.45:0/4279231666' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-16T17:06:47.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:06:47 smithi117 ceph-mon[36730]: pgmap v4: 129 pgs: 129 active+clean; 1.1 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T17:06:47.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:06:47 smithi117 ceph-mon[36730]: from='mgr.25411 ' entity='mgr.x' 2024-04-16T17:06:47.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:06:47 smithi117 ceph-mon[36730]: from='mgr.25411 ' entity='mgr.x' 2024-04-16T17:06:47.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:06:47 smithi117 ceph-mon[36730]: from='client.? 172.21.15.45:0/2097237099' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T17:06:47.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:06:47 smithi117 ceph-mon[36730]: from='mgr.25411 ' entity='mgr.x' 2024-04-16T17:06:47.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:06:47 smithi117 ceph-mon[36730]: from='mgr.25411 ' entity='mgr.x' 2024-04-16T17:06:47.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:47.816+0000 7fbf88a00640 1 -- 172.21.15.45:0/3408903184 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fbf8402d870 con 0x7fbf841051d0 2024-04-16T17:06:47.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:47.817+0000 7fbf7be00640 1 -- 172.21.15.45:0/3408903184 <== mon.2 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v95) ==== 93+0+10 (secure 0 0 0) 0x7fbf700bee70 con 0x7fbf841051d0 2024-04-16T17:06:47.818 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 1 2024-04-16T17:06:47.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:47.820+0000 7fbf88a00640 1 -- 172.21.15.45:0/3408903184 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fbf680a3da0 msgr2=0x7fbf680a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:47.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:47.820+0000 7fbf88a00640 1 --2- 172.21.15.45:0/3408903184 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fbf680a3da0 0x7fbf680a6260 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fbf74009fc0 tx=0x7fbf74005f90 comp rx=0 tx=0).stop 2024-04-16T17:06:47.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:47.820+0000 7fbf88a00640 1 -- 172.21.15.45:0/3408903184 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbf841051d0 msgr2=0x7fbf8419a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:47.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:47.820+0000 7fbf88a00640 1 --2- 172.21.15.45:0/3408903184 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbf841051d0 0x7fbf8419a870 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fbf70000c00 tx=0x7fbf7002c650 comp rx=0 tx=0).stop 2024-04-16T17:06:47.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:47.820+0000 7fbf88a00640 1 -- 172.21.15.45:0/3408903184 shutdown_connections 2024-04-16T17:06:47.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:47.820+0000 7fbf88a00640 1 --2- 172.21.15.45:0/3408903184 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fbf680a3da0 0x7fbf680a6260 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:47.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:47.820+0000 7fbf88a00640 1 --2- 172.21.15.45:0/3408903184 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbf84107b00 0x7fbf841a18e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:47.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:47.820+0000 7fbf88a00640 1 --2- 172.21.15.45:0/3408903184 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbf841051d0 0x7fbf8419a870 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:47.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:47.820+0000 7fbf88a00640 1 --2- 172.21.15.45:0/3408903184 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbf841028a0 0x7fbf8419a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:47.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:47.820+0000 7fbf88a00640 1 -- 172.21.15.45:0/3408903184 >> 172.21.15.45:0/3408903184 conn(0x7fbf840fc2d0 msgr2=0x7fbf84101fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:06:47.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:47.820+0000 7fbf88a00640 1 -- 172.21.15.45:0/3408903184 shutdown_connections 2024-04-16T17:06:47.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:47.820+0000 7fbf88a00640 1 -- 172.21.15.45:0/3408903184 wait complete. 2024-04-16T17:06:47.840 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-16T17:06:47.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:06:47 smithi104 ceph-mon[36457]: from='client.? 172.21.15.45:0/4279231666' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-16T17:06:47.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:06:47 smithi104 ceph-mon[36457]: pgmap v4: 129 pgs: 129 active+clean; 1.1 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T17:06:47.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:06:47 smithi104 ceph-mon[36457]: from='mgr.25411 ' entity='mgr.x' 2024-04-16T17:06:47.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:06:47 smithi104 ceph-mon[36457]: from='mgr.25411 ' entity='mgr.x' 2024-04-16T17:06:47.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:06:47 smithi104 ceph-mon[36457]: from='client.? 172.21.15.45:0/2097237099' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T17:06:47.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:06:47 smithi104 ceph-mon[36457]: from='mgr.25411 ' entity='mgr.x' 2024-04-16T17:06:47.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:06:47 smithi104 ceph-mon[36457]: from='mgr.25411 ' entity='mgr.x' 2024-04-16T17:06:48.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:48.049+0000 7fcd4d400640 1 -- 172.21.15.45:0/1373314597 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcd48077c00 msgr2=0x7fcd48078060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:48.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:48.049+0000 7fcd4d400640 1 --2- 172.21.15.45:0/1373314597 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcd48077c00 0x7fcd48078060 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fcd34009900 tx=0x7fcd3402bc40 comp rx=0 tx=0).stop 2024-04-16T17:06:48.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:48.049+0000 7fcd4d400640 1 -- 172.21.15.45:0/1373314597 shutdown_connections 2024-04-16T17:06:48.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:48.049+0000 7fcd4d400640 1 --2- 172.21.15.45:0/1373314597 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcd48077c00 0x7fcd48078060 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:48.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:48.049+0000 7fcd4d400640 1 --2- 172.21.15.45:0/1373314597 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcd480785a0 0x7fcd48078e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:48.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:48.049+0000 7fcd4d400640 1 --2- 172.21.15.45:0/1373314597 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcd480769c0 0x7fcd48076de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:48.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:48.049+0000 7fcd4d400640 1 -- 172.21.15.45:0/1373314597 >> 172.21.15.45:0/1373314597 conn(0x7fcd480ff7b0 msgr2=0x7fcd48101c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:06:48.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:48.050+0000 7fcd4d400640 1 -- 172.21.15.45:0/1373314597 shutdown_connections 2024-04-16T17:06:48.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:48.050+0000 7fcd4d400640 1 -- 172.21.15.45:0/1373314597 wait complete. 2024-04-16T17:06:48.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:48.051+0000 7fcd4d400640 1 Processor -- start 2024-04-16T17:06:48.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:48.051+0000 7fcd4d400640 1 -- start start 2024-04-16T17:06:48.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:48.051+0000 7fcd4d400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcd480769c0 0x7fcd4819e960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:48.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:48.051+0000 7fcd4d400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcd48077c00 0x7fcd4819eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:48.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:48.051+0000 7fcd4d400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcd480785a0 0x7fcd481a5f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:48.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:48.051+0000 7fcd4d400640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fcd481142f0 con 0x7fcd48077c00 2024-04-16T17:06:48.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:48.051+0000 7fcd4d400640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fcd48114170 con 0x7fcd480769c0 2024-04-16T17:06:48.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:48.051+0000 7fcd4d400640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fcd48114470 con 0x7fcd480785a0 2024-04-16T17:06:48.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:48.052+0000 7fcd45e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcd48077c00 0x7fcd4819eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:48.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:48.052+0000 7fcd46800640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcd480769c0 0x7fcd4819e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:48.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:48.052+0000 7fcd46800640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcd480769c0 0x7fcd4819e960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:56408/0 (socket says 172.21.15.45:56408) 2024-04-16T17:06:48.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:48.052+0000 7fcd47200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcd480785a0 0x7fcd481a5f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:48.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:48.052+0000 7fcd46800640 1 -- 172.21.15.45:0/2743754627 learned_addr learned my addr 172.21.15.45:0/2743754627 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:06:48.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:48.052+0000 7fcd45e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcd48077c00 0x7fcd4819eea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36760/0 (socket says 172.21.15.45:36760) 2024-04-16T17:06:48.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:48.052+0000 7fcd47200640 1 -- 172.21.15.45:0/2743754627 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcd480769c0 msgr2=0x7fcd4819e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:48.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:48.052+0000 7fcd47200640 1 --2- 172.21.15.45:0/2743754627 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcd480769c0 0x7fcd4819e960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:48.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:48.052+0000 7fcd47200640 1 -- 172.21.15.45:0/2743754627 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcd48077c00 msgr2=0x7fcd4819eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:48.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:48.052+0000 7fcd47200640 1 --2- 172.21.15.45:0/2743754627 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcd48077c00 0x7fcd4819eea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:48.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:48.052+0000 7fcd47200640 1 -- 172.21.15.45:0/2743754627 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd481a6580 con 0x7fcd480785a0 2024-04-16T17:06:48.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:48.053+0000 7fcd47200640 1 --2- 172.21.15.45:0/2743754627 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcd480785a0 0x7fcd481a5f10 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fcd40009fd0 tx=0x7fcd4000ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:06:48.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:48.053+0000 7fcd3f400640 1 -- 172.21.15.45:0/2743754627 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd4000ce20 con 0x7fcd480785a0 2024-04-16T17:06:48.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:48.053+0000 7fcd3f400640 1 -- 172.21.15.45:0/2743754627 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcd40005020 con 0x7fcd480785a0 2024-04-16T17:06:48.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:48.053+0000 7fcd4d400640 1 -- 172.21.15.45:0/2743754627 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd481a6870 con 0x7fcd480785a0 2024-04-16T17:06:48.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:48.053+0000 7fcd3f400640 1 -- 172.21.15.45:0/2743754627 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd40010720 con 0x7fcd480785a0 2024-04-16T17:06:48.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:48.053+0000 7fcd4d400640 1 -- 172.21.15.45:0/2743754627 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd481a6d90 con 0x7fcd480785a0 2024-04-16T17:06:48.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:48.054+0000 7fcd4d400640 1 -- 172.21.15.45:0/2743754627 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd48110d70 con 0x7fcd480785a0 2024-04-16T17:06:48.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:48.057+0000 7fcd3f400640 1 -- 172.21.15.45:0/2743754627 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 23) ==== 138766+0+0 (secure 0 0 0) 0x7fcd40005540 con 0x7fcd480785a0 2024-04-16T17:06:48.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:48.058+0000 7fcd3f400640 1 --2- 172.21.15.45:0/2743754627 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fcd2c0a3de0 0x7fcd2c0a62a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:48.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:48.058+0000 7fcd3f400640 1 -- 172.21.15.45:0/2743754627 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fcd400d1720 con 0x7fcd480785a0 2024-04-16T17:06:48.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:48.059+0000 7fcd46800640 1 --2- 172.21.15.45:0/2743754627 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fcd2c0a3de0 0x7fcd2c0a62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:48.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:48.060+0000 7fcd46800640 1 --2- 172.21.15.45:0/2743754627 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fcd2c0a3de0 0x7fcd2c0a62a0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fcd38009c90 tx=0x7fcd38005f90 comp rx=0 tx=0).ready entity=mgr.25411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:06:48.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:48.061+0000 7fcd3f400640 1 -- 172.21.15.45:0/2743754627 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fcd4009da10 con 0x7fcd480785a0 2024-04-16T17:06:48.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:48.278+0000 7fcd4d400640 1 -- 172.21.15.45:0/2743754627 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fcd48062c00 con 0x7fcd480785a0 2024-04-16T17:06:48.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:48.278+0000 7fcd3f400640 1 -- 172.21.15.45:0/2743754627 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v95) ==== 104+0+11 (secure 0 0 0) 0x7fcd4009d310 con 0x7fcd480785a0 2024-04-16T17:06:48.279 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 32 2024-04-16T17:06:48.282 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:48.281+0000 7fcd4d400640 1 -- 172.21.15.45:0/2743754627 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fcd2c0a3de0 msgr2=0x7fcd2c0a62a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:48.282 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:48.281+0000 7fcd4d400640 1 --2- 172.21.15.45:0/2743754627 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fcd2c0a3de0 0x7fcd2c0a62a0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fcd38009c90 tx=0x7fcd38005f90 comp rx=0 tx=0).stop 2024-04-16T17:06:48.282 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:48.281+0000 7fcd4d400640 1 -- 172.21.15.45:0/2743754627 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcd480785a0 msgr2=0x7fcd481a5f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:48.282 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:48.281+0000 7fcd4d400640 1 --2- 172.21.15.45:0/2743754627 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcd480785a0 0x7fcd481a5f10 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fcd40009fd0 tx=0x7fcd4000ef10 comp rx=0 tx=0).stop 2024-04-16T17:06:48.282 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:48.281+0000 7fcd4d400640 1 -- 172.21.15.45:0/2743754627 shutdown_connections 2024-04-16T17:06:48.282 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:48.281+0000 7fcd4d400640 1 --2- 172.21.15.45:0/2743754627 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fcd2c0a3de0 0x7fcd2c0a62a0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:48.282 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:48.281+0000 7fcd4d400640 1 --2- 172.21.15.45:0/2743754627 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcd480785a0 0x7fcd481a5f10 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:48.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:48.281+0000 7fcd4d400640 1 --2- 172.21.15.45:0/2743754627 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcd48077c00 0x7fcd4819eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:48.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:48.281+0000 7fcd4d400640 1 --2- 172.21.15.45:0/2743754627 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcd480769c0 0x7fcd4819e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:48.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:48.281+0000 7fcd4d400640 1 -- 172.21.15.45:0/2743754627 >> 172.21.15.45:0/2743754627 conn(0x7fcd480ff7b0 msgr2=0x7fcd48100f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:06:48.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:48.281+0000 7fcd4d400640 1 -- 172.21.15.45:0/2743754627 shutdown_connections 2024-04-16T17:06:48.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:48.281+0000 7fcd4d400640 1 -- 172.21.15.45:0/2743754627 wait complete. 2024-04-16T17:06:48.300 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-16T17:06:48.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:48.510+0000 7f25ca600640 1 -- 172.21.15.45:0/4105325867 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f25c4102050 msgr2=0x7f25c4102470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:48.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:48.510+0000 7f25ca600640 1 --2- 172.21.15.45:0/4105325867 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f25c4102050 0x7f25c4102470 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f25b4009900 tx=0x7f25b402bc40 comp rx=0 tx=0).stop 2024-04-16T17:06:48.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:48.510+0000 7f25ca600640 1 -- 172.21.15.45:0/4105325867 shutdown_connections 2024-04-16T17:06:48.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:48.510+0000 7f25ca600640 1 --2- 172.21.15.45:0/4105325867 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f25c4103c00 0x7f25c410a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:48.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:48.510+0000 7f25ca600640 1 --2- 172.21.15.45:0/4105325867 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f25c4103240 0x7f25c41036c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:48.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:48.510+0000 7f25ca600640 1 --2- 172.21.15.45:0/4105325867 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f25c4102050 0x7f25c4102470 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:48.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:48.511+0000 7f25ca600640 1 -- 172.21.15.45:0/4105325867 >> 172.21.15.45:0/4105325867 conn(0x7f25c40fd5e0 msgr2=0x7f25c40ffa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:06:48.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:48.511+0000 7f25ca600640 1 -- 172.21.15.45:0/4105325867 shutdown_connections 2024-04-16T17:06:48.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:48.511+0000 7f25ca600640 1 -- 172.21.15.45:0/4105325867 wait complete. 2024-04-16T17:06:48.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:48.512+0000 7f25ca600640 1 Processor -- start 2024-04-16T17:06:48.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:48.512+0000 7f25ca600640 1 -- start start 2024-04-16T17:06:48.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:48.512+0000 7f25ca600640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f25c4102050 0x7f25c419a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:48.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:48.512+0000 7f25ca600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f25c4103240 0x7f25c419ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:48.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:48.512+0000 7f25ca600640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f25c4103c00 0x7f25c41a1bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:48.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:48.512+0000 7f25ca600640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f25c410dd30 con 0x7f25c4103240 2024-04-16T17:06:48.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:48.512+0000 7f25ca600640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f25c410dbb0 con 0x7f25c4103c00 2024-04-16T17:06:48.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:48.512+0000 7f25ca600640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f25c410deb0 con 0x7f25c4102050 2024-04-16T17:06:48.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:48.513+0000 7f25c3400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f25c4102050 0x7f25c419a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:48.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:48.513+0000 7f25c3e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f25c4103c00 0x7f25c41a1bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:48.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:48.513+0000 7f25c2a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f25c4103240 0x7f25c419ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:48.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:48.513+0000 7f25c3400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f25c4102050 0x7f25c419a610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:51224/0 (socket says 172.21.15.45:51224) 2024-04-16T17:06:48.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:48.513+0000 7f25c3400640 1 -- 172.21.15.45:0/2662894184 learned_addr learned my addr 172.21.15.45:0/2662894184 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:06:48.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:48.513+0000 7f25c3400640 1 -- 172.21.15.45:0/2662894184 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f25c4103c00 msgr2=0x7f25c41a1bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:48.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:48.513+0000 7f25c3400640 1 --2- 172.21.15.45:0/2662894184 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f25c4103c00 0x7f25c41a1bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:48.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:48.513+0000 7f25c3400640 1 -- 172.21.15.45:0/2662894184 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f25c4103240 msgr2=0x7f25c419ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:48.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:48.513+0000 7f25c3400640 1 --2- 172.21.15.45:0/2662894184 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f25c4103240 0x7f25c419ab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:48.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:48.513+0000 7f25c3400640 1 -- 172.21.15.45:0/2662894184 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25c41a2230 con 0x7f25c4102050 2024-04-16T17:06:48.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:48.514+0000 7f25c3400640 1 --2- 172.21.15.45:0/2662894184 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f25c4102050 0x7f25c419a610 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f25b402efd0 tx=0x7f25b4005e40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:06:48.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:48.514+0000 7f25bbe00640 1 -- 172.21.15.45:0/2662894184 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25b4041070 con 0x7f25c4102050 2024-04-16T17:06:48.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:48.514+0000 7f25bbe00640 1 -- 172.21.15.45:0/2662894184 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f25b40054e0 con 0x7f25c4102050 2024-04-16T17:06:48.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:48.514+0000 7f25bbe00640 1 -- 172.21.15.45:0/2662894184 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25b4034440 con 0x7f25c4102050 2024-04-16T17:06:48.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:48.514+0000 7f25ca600640 1 -- 172.21.15.45:0/2662894184 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25c41a24c0 con 0x7f25c4102050 2024-04-16T17:06:48.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:48.514+0000 7f25ca600640 1 -- 172.21.15.45:0/2662894184 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25c41a29d0 con 0x7f25c4102050 2024-04-16T17:06:48.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:48.515+0000 7f25ca600640 1 -- 172.21.15.45:0/2662894184 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25c41a2c30 con 0x7f25c4102050 2024-04-16T17:06:48.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:48.517+0000 7f25bbe00640 1 -- 172.21.15.45:0/2662894184 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 23) ==== 138766+0+0 (secure 0 0 0) 0x7f25b4044070 con 0x7f25c4102050 2024-04-16T17:06:48.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:48.518+0000 7f25bbe00640 1 --2- 172.21.15.45:0/2662894184 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f25a80a3da0 0x7f25a80a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:48.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:48.519+0000 7f25bbe00640 1 -- 172.21.15.45:0/2662894184 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f25b40f25b0 con 0x7f25c4102050 2024-04-16T17:06:48.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:48.519+0000 7f25c2a00640 1 --2- 172.21.15.45:0/2662894184 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f25a80a3da0 0x7f25a80a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:48.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:48.519+0000 7f25c2a00640 1 --2- 172.21.15.45:0/2662894184 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f25a80a3da0 0x7f25a80a6260 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f25c419bb20 tx=0x7f25b000b390 comp rx=0 tx=0).ready entity=mgr.25411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:06:48.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:48.522+0000 7f25bbe00640 1 -- 172.21.15.45:0/2662894184 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f25b40be3a0 con 0x7f25c4102050 2024-04-16T17:06:48.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:48.750+0000 7f25ca600640 1 -- 172.21.15.45:0/2662894184 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f25c41a2e00 con 0x7f25c4102050 2024-04-16T17:06:48.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:48.751+0000 7f25bbe00640 1 -- 172.21.15.45:0/2662894184 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v95) ==== 100+0+11 (secure 0 0 0) 0x7f25b40c26a0 con 0x7f25c4102050 2024-04-16T17:06:48.751 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 64 2024-04-16T17:06:48.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:48.753+0000 7f25ca600640 1 -- 172.21.15.45:0/2662894184 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f25a80a3da0 msgr2=0x7f25a80a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:48.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:48.753+0000 7f25ca600640 1 --2- 172.21.15.45:0/2662894184 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f25a80a3da0 0x7f25a80a6260 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f25c419bb20 tx=0x7f25b000b390 comp rx=0 tx=0).stop 2024-04-16T17:06:48.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:48.753+0000 7f25ca600640 1 -- 172.21.15.45:0/2662894184 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f25c4102050 msgr2=0x7f25c419a610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:48.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:48.753+0000 7f25ca600640 1 --2- 172.21.15.45:0/2662894184 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f25c4102050 0x7f25c419a610 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f25b402efd0 tx=0x7f25b4005e40 comp rx=0 tx=0).stop 2024-04-16T17:06:48.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:48.753+0000 7f25ca600640 1 -- 172.21.15.45:0/2662894184 shutdown_connections 2024-04-16T17:06:48.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:48.753+0000 7f25ca600640 1 --2- 172.21.15.45:0/2662894184 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f25a80a3da0 0x7f25a80a6260 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:48.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:48.753+0000 7f25ca600640 1 --2- 172.21.15.45:0/2662894184 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f25c4103c00 0x7f25c41a1bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:48.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:48.753+0000 7f25ca600640 1 --2- 172.21.15.45:0/2662894184 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f25c4103240 0x7f25c419ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:48.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:48.753+0000 7f25ca600640 1 --2- 172.21.15.45:0/2662894184 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f25c4102050 0x7f25c419a610 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:48.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:48.753+0000 7f25ca600640 1 -- 172.21.15.45:0/2662894184 >> 172.21.15.45:0/2662894184 conn(0x7f25c40fd5e0 msgr2=0x7f25c4074230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:06:48.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:48.754+0000 7f25ca600640 1 -- 172.21.15.45:0/2662894184 shutdown_connections 2024-04-16T17:06:48.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:48.754+0000 7f25ca600640 1 -- 172.21.15.45:0/2662894184 wait complete. 2024-04-16T17:06:48.775 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-16T17:06:48.840 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:06:48 smithi045 ceph-mon[31676]: mgrmap e23: x(active, since 2s), standbys: z, y 2024-04-16T17:06:48.840 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:06:48 smithi045 ceph-mon[31676]: from='client.? 172.21.15.45:0/3408903184' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-16T17:06:48.840 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:06:48 smithi045 ceph-mon[31676]: from='client.? 172.21.15.45:0/2743754627' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-16T17:06:48.840 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:06:48 smithi045 ceph-mon[31676]: from='mgr.25411 ' entity='mgr.x' 2024-04-16T17:06:48.840 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:06:48 smithi045 ceph-mon[31676]: from='mgr.25411 ' entity='mgr.x' 2024-04-16T17:06:48.840 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:06:48 smithi045 ceph-mon[31676]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:06:48.840 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:06:48 smithi045 ceph-mon[31676]: from='mgr.25411 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:06:48.840 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:06:48 smithi045 ceph-mon[31676]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:06:48.840 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:06:48 smithi045 ceph-mon[31676]: from='mgr.25411 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:06:48.841 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:06:48 smithi045 ceph-mon[31676]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:06:48.841 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:06:48 smithi045 ceph-mon[31676]: from='mgr.25411 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:06:48.841 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:06:48 smithi045 ceph-mon[31676]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:06:48.841 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:06:48 smithi045 ceph-mon[31676]: from='mgr.25411 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:06:48.841 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:06:48 smithi045 ceph-mon[31676]: Adjusting osd_memory_target on smithi117 to 174.1M 2024-04-16T17:06:48.841 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:06:48 smithi045 ceph-mon[31676]: Unable to set osd_memory_target on smithi117 to 182632038: error parsing value: Value '182632038' is below minimum 939524096 2024-04-16T17:06:48.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:06:48 smithi104 ceph-mon[36457]: mgrmap e23: x(active, since 2s), standbys: z, y 2024-04-16T17:06:48.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:06:48 smithi104 ceph-mon[36457]: from='client.? 172.21.15.45:0/3408903184' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-16T17:06:48.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:06:48 smithi104 ceph-mon[36457]: from='client.? 172.21.15.45:0/2743754627' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-16T17:06:48.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:06:48 smithi104 ceph-mon[36457]: from='mgr.25411 ' entity='mgr.x' 2024-04-16T17:06:48.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:06:48 smithi104 ceph-mon[36457]: from='mgr.25411 ' entity='mgr.x' 2024-04-16T17:06:48.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:06:48 smithi104 ceph-mon[36457]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:06:48.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:06:48 smithi104 ceph-mon[36457]: from='mgr.25411 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:06:48.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:06:48 smithi104 ceph-mon[36457]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:06:48.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:06:48 smithi104 ceph-mon[36457]: from='mgr.25411 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:06:48.929 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:06:48 smithi104 ceph-mon[36457]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:06:48.929 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:06:48 smithi104 ceph-mon[36457]: from='mgr.25411 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:06:48.929 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:06:48 smithi104 ceph-mon[36457]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:06:48.929 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:06:48 smithi104 ceph-mon[36457]: from='mgr.25411 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:06:48.929 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:06:48 smithi104 ceph-mon[36457]: Adjusting osd_memory_target on smithi117 to 174.1M 2024-04-16T17:06:48.929 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:06:48 smithi104 ceph-mon[36457]: Unable to set osd_memory_target on smithi117 to 182632038: error parsing value: Value '182632038' is below minimum 939524096 2024-04-16T17:06:48.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:48.981+0000 7f14d2a00640 1 -- 172.21.15.45:0/745912741 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f14cc107bf0 msgr2=0x7f14cc10a040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:48.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:48.981+0000 7f14d2a00640 1 --2- 172.21.15.45:0/745912741 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f14cc107bf0 0x7f14cc10a040 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f14c4009ca0 tx=0x7f14c402c0f0 comp rx=0 tx=0).stop 2024-04-16T17:06:48.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:48.981+0000 7f14d2a00640 1 -- 172.21.15.45:0/745912741 shutdown_connections 2024-04-16T17:06:48.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:48.981+0000 7f14d2a00640 1 --2- 172.21.15.45:0/745912741 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f14cc107bf0 0x7f14cc10a040 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:48.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:48.981+0000 7f14d2a00640 1 --2- 172.21.15.45:0/745912741 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f14cc1052c0 0x7f14cc1076b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:48.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:48.981+0000 7f14d2a00640 1 --2- 172.21.15.45:0/745912741 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f14cc102990 0x7f14cc104d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:48.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:48.981+0000 7f14d2a00640 1 -- 172.21.15.45:0/745912741 >> 172.21.15.45:0/745912741 conn(0x7f14cc0fc3a0 msgr2=0x7f14cc0fe7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:06:48.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:48.981+0000 7f14d2a00640 1 -- 172.21.15.45:0/745912741 shutdown_connections 2024-04-16T17:06:48.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:48.981+0000 7f14d2a00640 1 -- 172.21.15.45:0/745912741 wait complete. 2024-04-16T17:06:48.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:48.982+0000 7f14d2a00640 1 Processor -- start 2024-04-16T17:06:48.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:48.982+0000 7f14d2a00640 1 -- start start 2024-04-16T17:06:48.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:48.983+0000 7f14d2a00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f14cc102990 0x7f14cc19a390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:48.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:48.983+0000 7f14d2a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f14cc1052c0 0x7f14cc19a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:48.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:48.983+0000 7f14d2a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f14cc107bf0 0x7f14cc1a1940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:48.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:48.983+0000 7f14d2a00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f14cc10dba0 con 0x7f14cc1052c0 2024-04-16T17:06:48.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:48.983+0000 7f14d2a00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f14cc10da20 con 0x7f14cc102990 2024-04-16T17:06:48.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:48.983+0000 7f14d2a00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f14cc10dd20 con 0x7f14cc107bf0 2024-04-16T17:06:48.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:48.983+0000 7f14d0c00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f14cc1052c0 0x7f14cc19a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:48.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:48.983+0000 7f14d2000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f14cc107bf0 0x7f14cc1a1940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:48.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:48.983+0000 7f14d0c00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f14cc1052c0 0x7f14cc19a8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36778/0 (socket says 172.21.15.45:36778) 2024-04-16T17:06:48.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:48.983+0000 7f14d0c00640 1 -- 172.21.15.45:0/379042610 learned_addr learned my addr 172.21.15.45:0/379042610 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:06:48.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:48.983+0000 7f14d2000640 1 --2- 172.21.15.45:0/379042610 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f14cc107bf0 0x7f14cc1a1940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:51246/0 (socket says 172.21.15.45:51246) 2024-04-16T17:06:48.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:48.983+0000 7f14d1600640 1 --2- 172.21.15.45:0/379042610 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f14cc102990 0x7f14cc19a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:48.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:48.983+0000 7f14d2000640 1 -- 172.21.15.45:0/379042610 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f14cc102990 msgr2=0x7f14cc19a390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:48.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:48.983+0000 7f14d2000640 1 --2- 172.21.15.45:0/379042610 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f14cc102990 0x7f14cc19a390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:48.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:48.983+0000 7f14d2000640 1 -- 172.21.15.45:0/379042610 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f14cc1052c0 msgr2=0x7f14cc19a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:48.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:48.983+0000 7f14d2000640 1 --2- 172.21.15.45:0/379042610 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f14cc1052c0 0x7f14cc19a8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:48.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:48.983+0000 7f14d2000640 1 -- 172.21.15.45:0/379042610 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14cc1a1fb0 con 0x7f14cc107bf0 2024-04-16T17:06:48.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:48.984+0000 7f14d2000640 1 --2- 172.21.15.45:0/379042610 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f14cc107bf0 0x7f14cc1a1940 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f14c4037040 tx=0x7f14c4032ea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:06:48.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:48.984+0000 7f14ca000640 1 -- 172.21.15.45:0/379042610 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14c4041020 con 0x7f14cc107bf0 2024-04-16T17:06:48.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:48.984+0000 7f14d2a00640 1 -- 172.21.15.45:0/379042610 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14cc1a2240 con 0x7f14cc107bf0 2024-04-16T17:06:48.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:48.984+0000 7f14ca000640 1 -- 172.21.15.45:0/379042610 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f14c4034cd0 con 0x7f14cc107bf0 2024-04-16T17:06:48.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:48.984+0000 7f14ca000640 1 -- 172.21.15.45:0/379042610 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14c4039040 con 0x7f14cc107bf0 2024-04-16T17:06:48.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:48.984+0000 7f14d2a00640 1 -- 172.21.15.45:0/379042610 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14cc1a2700 con 0x7f14cc107bf0 2024-04-16T17:06:48.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:48.985+0000 7f14d2a00640 1 -- 172.21.15.45:0/379042610 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14cc1096e0 con 0x7f14cc107bf0 2024-04-16T17:06:48.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:48.987+0000 7f14ca000640 1 -- 172.21.15.45:0/379042610 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 23) ==== 138766+0+0 (secure 0 0 0) 0x7f14c4044020 con 0x7f14cc107bf0 2024-04-16T17:06:48.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:48.989+0000 7f14ca000640 1 --2- 172.21.15.45:0/379042610 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f14b40a3d20 0x7f14b40a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:48.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:48.989+0000 7f14ca000640 1 -- 172.21.15.45:0/379042610 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f14c40f2830 con 0x7f14cc107bf0 2024-04-16T17:06:48.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:48.989+0000 7f14d1600640 1 --2- 172.21.15.45:0/379042610 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f14b40a3d20 0x7f14b40a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:48.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:48.990+0000 7f14d1600640 1 --2- 172.21.15.45:0/379042610 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f14b40a3d20 0x7f14b40a61e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f14c0009bd0 tx=0x7f14c0005f90 comp rx=0 tx=0).ready entity=mgr.25411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:06:48.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:48.993+0000 7f14ca000640 1 -- 172.21.15.45:0/379042610 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f14c404b020 con 0x7f14cc107bf0 2024-04-16T17:06:49.055 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:06:48 smithi117 ceph-mon[36730]: mgrmap e23: x(active, since 2s), standbys: z, y 2024-04-16T17:06:49.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:06:48 smithi117 ceph-mon[36730]: from='client.? 172.21.15.45:0/3408903184' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-16T17:06:49.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:06:48 smithi117 ceph-mon[36730]: from='client.? 172.21.15.45:0/2743754627' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-16T17:06:49.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:06:48 smithi117 ceph-mon[36730]: from='mgr.25411 ' entity='mgr.x' 2024-04-16T17:06:49.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:06:48 smithi117 ceph-mon[36730]: from='mgr.25411 ' entity='mgr.x' 2024-04-16T17:06:49.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:06:48 smithi117 ceph-mon[36730]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:06:49.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:06:48 smithi117 ceph-mon[36730]: from='mgr.25411 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:06:49.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:06:48 smithi117 ceph-mon[36730]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:06:49.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:06:48 smithi117 ceph-mon[36730]: from='mgr.25411 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:06:49.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:06:48 smithi117 ceph-mon[36730]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:06:49.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:06:48 smithi117 ceph-mon[36730]: from='mgr.25411 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:06:49.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:06:48 smithi117 ceph-mon[36730]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:06:49.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:06:48 smithi117 ceph-mon[36730]: from='mgr.25411 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:06:49.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:06:48 smithi117 ceph-mon[36730]: Adjusting osd_memory_target on smithi117 to 174.1M 2024-04-16T17:06:49.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:06:48 smithi117 ceph-mon[36730]: Unable to set osd_memory_target on smithi117 to 182632038: error parsing value: Value '182632038' is below minimum 939524096 2024-04-16T17:06:49.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:49.199+0000 7f14d2a00640 1 -- 172.21.15.45:0/379042610 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f14cc062c00 con 0x7f14cc107bf0 2024-04-16T17:06:49.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:49.200+0000 7f14ca000640 1 -- 172.21.15.45:0/379042610 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v95) ==== 103+0+11 (secure 0 0 0) 0x7f14c40becc0 con 0x7f14cc107bf0 2024-04-16T17:06:49.200 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 32 2024-04-16T17:06:49.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:49.202+0000 7f14d2a00640 1 -- 172.21.15.45:0/379042610 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f14b40a3d20 msgr2=0x7f14b40a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:49.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:49.202+0000 7f14d2a00640 1 --2- 172.21.15.45:0/379042610 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f14b40a3d20 0x7f14b40a61e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f14c0009bd0 tx=0x7f14c0005f90 comp rx=0 tx=0).stop 2024-04-16T17:06:49.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:49.202+0000 7f14d2a00640 1 -- 172.21.15.45:0/379042610 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f14cc107bf0 msgr2=0x7f14cc1a1940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:49.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:49.202+0000 7f14d2a00640 1 --2- 172.21.15.45:0/379042610 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f14cc107bf0 0x7f14cc1a1940 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f14c4037040 tx=0x7f14c4032ea0 comp rx=0 tx=0).stop 2024-04-16T17:06:49.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:49.202+0000 7f14d2a00640 1 -- 172.21.15.45:0/379042610 shutdown_connections 2024-04-16T17:06:49.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:49.202+0000 7f14d2a00640 1 --2- 172.21.15.45:0/379042610 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f14b40a3d20 0x7f14b40a61e0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:49.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:49.202+0000 7f14d2a00640 1 --2- 172.21.15.45:0/379042610 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f14cc107bf0 0x7f14cc1a1940 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:49.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:49.202+0000 7f14d2a00640 1 --2- 172.21.15.45:0/379042610 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f14cc1052c0 0x7f14cc19a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:49.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:49.202+0000 7f14d2a00640 1 --2- 172.21.15.45:0/379042610 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f14cc102990 0x7f14cc19a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:49.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:49.202+0000 7f14d2a00640 1 -- 172.21.15.45:0/379042610 >> 172.21.15.45:0/379042610 conn(0x7f14cc0fc3a0 msgr2=0x7f14cc101fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:06:49.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:49.203+0000 7f14d2a00640 1 -- 172.21.15.45:0/379042610 shutdown_connections 2024-04-16T17:06:49.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:49.203+0000 7f14d2a00640 1 -- 172.21.15.45:0/379042610 wait complete. 2024-04-16T17:06:49.221 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2024-04-16T17:06:49.221 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-16T17:06:49.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:49.419+0000 7faad7e00640 1 -- 172.21.15.45:0/394933055 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7faad8105230 msgr2=0x7faad8107620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:49.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:49.419+0000 7faad7e00640 1 --2- 172.21.15.45:0/394933055 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7faad8105230 0x7faad8107620 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7faac4009900 tx=0x7faac402bc40 comp rx=0 tx=0).stop 2024-04-16T17:06:49.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:49.419+0000 7faad7e00640 1 -- 172.21.15.45:0/394933055 shutdown_connections 2024-04-16T17:06:49.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:49.419+0000 7faad7e00640 1 --2- 172.21.15.45:0/394933055 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faad8107b60 0x7faad8109fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:49.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:49.419+0000 7faad7e00640 1 --2- 172.21.15.45:0/394933055 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7faad8105230 0x7faad8107620 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:49.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:49.419+0000 7faad7e00640 1 --2- 172.21.15.45:0/394933055 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faad8102900 0x7faad8104cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:49.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:49.419+0000 7faad7e00640 1 -- 172.21.15.45:0/394933055 >> 172.21.15.45:0/394933055 conn(0x7faad80fc2d0 msgr2=0x7faad80fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:06:49.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:49.419+0000 7faad7e00640 1 -- 172.21.15.45:0/394933055 shutdown_connections 2024-04-16T17:06:49.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:49.419+0000 7faad7e00640 1 -- 172.21.15.45:0/394933055 wait complete. 2024-04-16T17:06:49.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:49.421+0000 7faad7e00640 1 Processor -- start 2024-04-16T17:06:49.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:49.421+0000 7faad7e00640 1 -- start start 2024-04-16T17:06:49.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:49.422+0000 7faad7e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faad8102900 0x7faad8102270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:49.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:49.422+0000 7faad7e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7faad8105230 0x7faad81008b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:49.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:49.422+0000 7faad7e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faad8107b60 0x7faad8100df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:49.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:49.422+0000 7faad7e00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7faad810db40 con 0x7faad8102900 2024-04-16T17:06:49.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:49.422+0000 7faad7e00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7faad810d9c0 con 0x7faad8105230 2024-04-16T17:06:49.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:49.422+0000 7faad7e00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7faad810dcc0 con 0x7faad8107b60 2024-04-16T17:06:49.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:49.422+0000 7faad6a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faad8102900 0x7faad8102270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:49.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:49.422+0000 7faad6a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faad8102900 0x7faad8102270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36800/0 (socket says 172.21.15.45:36800) 2024-04-16T17:06:49.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:49.422+0000 7faad6a00640 1 -- 172.21.15.45:0/3249440094 learned_addr learned my addr 172.21.15.45:0/3249440094 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:06:49.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:49.422+0000 7faad7400640 1 --2- 172.21.15.45:0/3249440094 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faad8107b60 0x7faad8100df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:49.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:49.422+0000 7faad6000640 1 --2- 172.21.15.45:0/3249440094 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7faad8105230 0x7faad81008b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:49.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:49.423+0000 7faad6a00640 1 -- 172.21.15.45:0/3249440094 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faad8107b60 msgr2=0x7faad8100df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:49.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:49.423+0000 7faad6a00640 1 --2- 172.21.15.45:0/3249440094 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faad8107b60 0x7faad8100df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:49.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:49.423+0000 7faad6a00640 1 -- 172.21.15.45:0/3249440094 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7faad8105230 msgr2=0x7faad81008b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:49.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:49.423+0000 7faad6a00640 1 --2- 172.21.15.45:0/3249440094 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7faad8105230 0x7faad81008b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:49.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:49.423+0000 7faad6a00640 1 -- 172.21.15.45:0/3249440094 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faad8101680 con 0x7faad8102900 2024-04-16T17:06:49.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:49.423+0000 7faad6a00640 1 --2- 172.21.15.45:0/3249440094 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faad8102900 0x7faad8102270 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7faac800ea60 tx=0x7faac800ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:06:49.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:49.423+0000 7faacf400640 1 -- 172.21.15.45:0/3249440094 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faac800cd20 con 0x7faad8102900 2024-04-16T17:06:49.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:49.423+0000 7faad7e00640 1 -- 172.21.15.45:0/3249440094 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faad81a22c0 con 0x7faad8102900 2024-04-16T17:06:49.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:49.423+0000 7faacf400640 1 -- 172.21.15.45:0/3249440094 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faac80035f0 con 0x7faad8102900 2024-04-16T17:06:49.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:49.424+0000 7faacf400640 1 -- 172.21.15.45:0/3249440094 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faac8005220 con 0x7faad8102900 2024-04-16T17:06:49.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:49.424+0000 7faad7e00640 1 -- 172.21.15.45:0/3249440094 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faad81a27e0 con 0x7faad8102900 2024-04-16T17:06:49.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:49.425+0000 7faad7e00640 1 -- 172.21.15.45:0/3249440094 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faad8068d40 con 0x7faad8102900 2024-04-16T17:06:49.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:49.427+0000 7faacf400640 1 -- 172.21.15.45:0/3249440094 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 23) ==== 138766+0+0 (secure 0 0 0) 0x7faac80037a0 con 0x7faad8102900 2024-04-16T17:06:49.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:49.428+0000 7faacf400640 1 --2- 172.21.15.45:0/3249440094 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7faabc0a3d20 0x7faabc0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:49.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:49.428+0000 7faacf400640 1 -- 172.21.15.45:0/3249440094 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7faac80d0e60 con 0x7faad8102900 2024-04-16T17:06:49.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:49.429+0000 7faad6000640 1 --2- 172.21.15.45:0/3249440094 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7faabc0a3d20 0x7faabc0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:49.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:49.429+0000 7faad6000640 1 --2- 172.21.15.45:0/3249440094 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7faabc0a3d20 0x7faabc0a61e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7faac402c180 tx=0x7faac402cf90 comp rx=0 tx=0).ready entity=mgr.25411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:06:49.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:49.434+0000 7faacf400640 1 -- 172.21.15.45:0/3249440094 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7faac809d150 con 0x7faad8102900 2024-04-16T17:06:49.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:49.640+0000 7faacf400640 1 -- 172.21.15.45:0/3249440094 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mgrmap(e 24) ==== 138822+0+0 (secure 0 0 0) 0x7faac80990c0 con 0x7faad8102900 2024-04-16T17:06:49.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:49.709+0000 7faad7e00640 1 -- 172.21.15.45:0/3249440094 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7faad80fddc0 con 0x7faad8102900 2024-04-16T17:06:49.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:49.711+0000 7faacf400640 1 -- 172.21.15.45:0/3249440094 <== mon.0 v2:172.21.15.45:3300/0 8 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11887 (secure 0 0 0) 0x7faac8051de0 con 0x7faad8102900 2024-04-16T17:06:49.712 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-16T17:06:49.712 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/1748842443","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":1748842443},{"type":"v1","addr":"172.21.15.45:6835","nonce":1748842443}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6839/1308627051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":1308627051},{"type":"v1","addr":"172.21.15.45:6839","nonce":1308627051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/1768242489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1768242489},{"type":"v1","addr":"172.21.15.117:6833","nonce":1768242489}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6833/3558902585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":3558902585},{"type":"v1","addr":"172.21.15.104:6833","nonce":3558902585}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24478,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/1308142903","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":1308142903},{"type":"v1","addr":"172.21.15.45:6837","nonce":1308142903}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24491,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6835/3430679937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":3430679937},{"type":"v1","addr":"172.21.15.104:6835","nonce":3430679937}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-16T17:03:24.663611+0000","modified":"2024-04-16T17:06:44.485317+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24581,"mds_1":24569,"mds_2":24532},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24499":{"gid":24499,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/2224123841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2224123841},{"type":"v1","addr":"172.21.15.117:6835","nonce":2224123841}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6837/2471356815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":2471356815},{"type":"v1","addr":"172.21.15.117:6837","nonce":2471356815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24524":{"gid":24524,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6837/2647756509","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":2647756509},{"type":"v1","addr":"172.21.15.104:6837","nonce":2647756509}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24532":{"gid":24532,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":13,"addr":"172.21.15.117:6839/2378675949","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":2378675949},{"type":"v1","addr":"172.21.15.117:6839","nonce":2378675949}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24569":{"gid":24569,"name":"a","rank":1,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.45:6841/3755310822","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6840","nonce":3755310822},{"type":"v1","addr":"172.21.15.45:6841","nonce":3755310822}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24581":{"gid":24581,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.104:6839/3660070592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6838","nonce":3660070592},{"type":"v1","addr":"172.21.15.104:6839","nonce":3660070592}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24581,"qdb_cluster":[24569,24581,24532]},"id":1}]} 2024-04-16T17:06:49.712 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 25 2024-04-16T17:06:49.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:49.713+0000 7faad7e00640 1 -- 172.21.15.45:0/3249440094 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7faabc0a3d20 msgr2=0x7faabc0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:49.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:49.713+0000 7faad7e00640 1 --2- 172.21.15.45:0/3249440094 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7faabc0a3d20 0x7faabc0a61e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7faac402c180 tx=0x7faac402cf90 comp rx=0 tx=0).stop 2024-04-16T17:06:49.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:49.713+0000 7faad7e00640 1 -- 172.21.15.45:0/3249440094 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faad8102900 msgr2=0x7faad8102270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:49.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:49.713+0000 7faad7e00640 1 --2- 172.21.15.45:0/3249440094 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faad8102900 0x7faad8102270 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7faac800ea60 tx=0x7faac800ee60 comp rx=0 tx=0).stop 2024-04-16T17:06:49.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:49.713+0000 7faad7e00640 1 -- 172.21.15.45:0/3249440094 shutdown_connections 2024-04-16T17:06:49.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:49.713+0000 7faad7e00640 1 --2- 172.21.15.45:0/3249440094 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7faabc0a3d20 0x7faabc0a61e0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:49.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:49.713+0000 7faad7e00640 1 --2- 172.21.15.45:0/3249440094 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faad8107b60 0x7faad8100df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:49.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:49.713+0000 7faad7e00640 1 --2- 172.21.15.45:0/3249440094 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7faad8105230 0x7faad81008b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:49.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:49.713+0000 7faad7e00640 1 --2- 172.21.15.45:0/3249440094 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faad8102900 0x7faad8102270 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:49.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:49.713+0000 7faad7e00640 1 -- 172.21.15.45:0/3249440094 >> 172.21.15.45:0/3249440094 conn(0x7faad80fc2d0 msgr2=0x7faad8103240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:06:49.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:49.714+0000 7faad7e00640 1 -- 172.21.15.45:0/3249440094 shutdown_connections 2024-04-16T17:06:49.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:49.714+0000 7faad7e00640 1 -- 172.21.15.45:0/3249440094 wait complete. 2024-04-16T17:06:49.733 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2024-04-16T17:06:49.733 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2024-04-16T17:06:49.733 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-16T17:06:49.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:06:49 smithi104 ceph-mon[36457]: from='mgr.25411 ' entity='mgr.x' 2024-04-16T17:06:49.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:06:49 smithi104 ceph-mon[36457]: from='mgr.25411 ' entity='mgr.x' 2024-04-16T17:06:49.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:06:49 smithi104 ceph-mon[36457]: from='mgr.25411 ' entity='mgr.x' 2024-04-16T17:06:49.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:06:49 smithi104 ceph-mon[36457]: from='mgr.25411 ' entity='mgr.x' 2024-04-16T17:06:49.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:06:49 smithi104 ceph-mon[36457]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:06:49.929 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:06:49 smithi104 ceph-mon[36457]: from='mgr.25411 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:06:49.929 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:06:49 smithi104 ceph-mon[36457]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:06:49.929 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:06:49 smithi104 ceph-mon[36457]: from='mgr.25411 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:06:49.929 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:06:49 smithi104 ceph-mon[36457]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:06:49.929 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:06:49 smithi104 ceph-mon[36457]: from='mgr.25411 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:06:49.929 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:06:49 smithi104 ceph-mon[36457]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:06:49.929 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:06:49 smithi104 ceph-mon[36457]: from='mgr.25411 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:06:49.929 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:06:49 smithi104 ceph-mon[36457]: Adjusting osd_memory_target on smithi104 to 174.1M 2024-04-16T17:06:49.929 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:06:49 smithi104 ceph-mon[36457]: Unable to set osd_memory_target on smithi104 to 182633472: error parsing value: Value '182633472' is below minimum 939524096 2024-04-16T17:06:49.929 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:06:49 smithi104 ceph-mon[36457]: from='client.? 172.21.15.45:0/2662894184' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-16T17:06:49.929 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:06:49 smithi104 ceph-mon[36457]: pgmap v5: 129 pgs: 129 active+clean; 1.1 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T17:06:49.930 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:06:49 smithi104 ceph-mon[36457]: from='client.? 172.21.15.45:0/379042610' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-16T17:06:49.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:49.943+0000 7f7aea400640 1 -- 172.21.15.45:0/2832050337 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7ae4102900 msgr2=0x7f7ae4104cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:49.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:49.943+0000 7f7aea400640 1 --2- 172.21.15.45:0/2832050337 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7ae4102900 0x7f7ae4104cf0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f7ad8009860 tx=0x7f7ad802bb70 comp rx=0 tx=0).stop 2024-04-16T17:06:49.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:49.943+0000 7f7aea400640 1 -- 172.21.15.45:0/2832050337 shutdown_connections 2024-04-16T17:06:49.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:49.943+0000 7f7aea400640 1 --2- 172.21.15.45:0/2832050337 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7ae4107b60 0x7f7ae4109fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:49.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:49.943+0000 7f7aea400640 1 --2- 172.21.15.45:0/2832050337 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7ae4105230 0x7f7ae4107620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:49.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:49.943+0000 7f7aea400640 1 --2- 172.21.15.45:0/2832050337 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7ae4102900 0x7f7ae4104cf0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:49.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:49.943+0000 7f7aea400640 1 -- 172.21.15.45:0/2832050337 >> 172.21.15.45:0/2832050337 conn(0x7f7ae40fc2d0 msgr2=0x7f7ae40fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:06:49.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:49.943+0000 7f7aea400640 1 -- 172.21.15.45:0/2832050337 shutdown_connections 2024-04-16T17:06:49.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:49.943+0000 7f7aea400640 1 -- 172.21.15.45:0/2832050337 wait complete. 2024-04-16T17:06:49.947 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:49.945+0000 7f7aea400640 1 Processor -- start 2024-04-16T17:06:49.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:49.945+0000 7f7aea400640 1 -- start start 2024-04-16T17:06:49.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:49.945+0000 7f7aea400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7ae4102900 0x7f7ae419a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:49.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:49.945+0000 7f7aea400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7ae4105230 0x7f7ae419a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:49.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:49.945+0000 7f7aea400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7ae4107b60 0x7f7ae41a19b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:49.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:49.945+0000 7f7aea400640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f7ae410db00 con 0x7f7ae4102900 2024-04-16T17:06:49.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:49.945+0000 7f7aea400640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f7ae410d980 con 0x7f7ae4105230 2024-04-16T17:06:49.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:49.945+0000 7f7aea400640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f7ae410dc80 con 0x7f7ae4107b60 2024-04-16T17:06:49.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:49.946+0000 7f7ae9000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7ae4102900 0x7f7ae419a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:49.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:49.946+0000 7f7ae9000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7ae4102900 0x7f7ae419a400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55256/0 (socket says 172.21.15.45:55256) 2024-04-16T17:06:49.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:49.946+0000 7f7ae3e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7ae4105230 0x7f7ae419a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:49.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:49.946+0000 7f7ae9000640 1 -- 172.21.15.45:0/2206233644 learned_addr learned my addr 172.21.15.45:0/2206233644 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:06:49.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:49.946+0000 7f7ae9a00640 1 --2- 172.21.15.45:0/2206233644 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7ae4107b60 0x7f7ae41a19b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:49.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:49.946+0000 7f7ae3e00640 1 -- 172.21.15.45:0/2206233644 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7ae4107b60 msgr2=0x7f7ae41a19b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:49.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:49.946+0000 7f7ae3e00640 1 --2- 172.21.15.45:0/2206233644 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7ae4107b60 0x7f7ae41a19b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:49.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:49.946+0000 7f7ae3e00640 1 -- 172.21.15.45:0/2206233644 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7ae4102900 msgr2=0x7f7ae419a400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:49.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:49.946+0000 7f7ae3e00640 1 --2- 172.21.15.45:0/2206233644 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7ae4102900 0x7f7ae419a400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:49.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:49.946+0000 7f7ae3e00640 1 -- 172.21.15.45:0/2206233644 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ae41a2020 con 0x7f7ae4105230 2024-04-16T17:06:49.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:49.946+0000 7f7ae9a00640 1 --2- 172.21.15.45:0/2206233644 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7ae4107b60 0x7f7ae41a19b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:06:49.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:49.946+0000 7f7ae3e00640 1 --2- 172.21.15.45:0/2206233644 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7ae4105230 0x7f7ae419a940 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f7ad0005fd0 tx=0x7f7ad000eb60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:06:49.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:49.947+0000 7f7ae1600640 1 -- 172.21.15.45:0/2206233644 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ad000cab0 con 0x7f7ae4105230 2024-04-16T17:06:49.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:49.947+0000 7f7ae1600640 1 -- 172.21.15.45:0/2206233644 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7ad000cc50 con 0x7f7ae4105230 2024-04-16T17:06:49.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:49.947+0000 7f7aea400640 1 -- 172.21.15.45:0/2206233644 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ae41a2310 con 0x7f7ae4105230 2024-04-16T17:06:49.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:49.947+0000 7f7ae1600640 1 -- 172.21.15.45:0/2206233644 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ad0018680 con 0x7f7ae4105230 2024-04-16T17:06:49.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:49.947+0000 7f7aea400640 1 -- 172.21.15.45:0/2206233644 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ae41a2860 con 0x7f7ae4105230 2024-04-16T17:06:49.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:49.948+0000 7f7aea400640 1 -- 172.21.15.45:0/2206233644 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ae41096b0 con 0x7f7ae4105230 2024-04-16T17:06:49.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:49.951+0000 7f7ae1600640 1 -- 172.21.15.45:0/2206233644 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 24) ==== 138822+0+0 (secure 0 0 0) 0x7f7ad0010430 con 0x7f7ae4105230 2024-04-16T17:06:49.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:49.952+0000 7f7ae1600640 1 --2- 172.21.15.45:0/2206233644 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f7acc0a3db0 0x7f7acc0a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:49.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:49.953+0000 7f7ae1600640 1 -- 172.21.15.45:0/2206233644 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f7ad00d0fa0 con 0x7f7ae4105230 2024-04-16T17:06:49.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:49.953+0000 7f7ae9000640 1 --2- 172.21.15.45:0/2206233644 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f7acc0a3db0 0x7f7acc0a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:49.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:49.954+0000 7f7ae9000640 1 --2- 172.21.15.45:0/2206233644 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f7acc0a3db0 0x7f7acc0a6270 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f7ad80095f0 tx=0x7f7ad80058b0 comp rx=0 tx=0).ready entity=mgr.25411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:06:49.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:49.957+0000 7f7ae1600640 1 -- 172.21.15.45:0/2206233644 <== mon.2 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7ad009d120 con 0x7f7ae4105230 2024-04-16T17:06:49.978 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:06:49 smithi045 ceph-mon[31676]: from='mgr.25411 ' entity='mgr.x' 2024-04-16T17:06:49.978 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:06:49 smithi045 ceph-mon[31676]: from='mgr.25411 ' entity='mgr.x' 2024-04-16T17:06:49.978 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:06:49 smithi045 ceph-mon[31676]: from='mgr.25411 ' entity='mgr.x' 2024-04-16T17:06:49.978 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:06:49 smithi045 ceph-mon[31676]: from='mgr.25411 ' entity='mgr.x' 2024-04-16T17:06:49.978 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:06:49 smithi045 ceph-mon[31676]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:06:49.978 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:06:49 smithi045 ceph-mon[31676]: from='mgr.25411 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:06:49.978 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:06:49 smithi045 ceph-mon[31676]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:06:49.978 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:06:49 smithi045 ceph-mon[31676]: from='mgr.25411 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:06:49.978 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:06:49 smithi045 ceph-mon[31676]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:06:49.979 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:06:49 smithi045 ceph-mon[31676]: from='mgr.25411 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:06:49.979 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:06:49 smithi045 ceph-mon[31676]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:06:49.979 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:06:49 smithi045 ceph-mon[31676]: from='mgr.25411 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:06:49.979 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:06:49 smithi045 ceph-mon[31676]: Adjusting osd_memory_target on smithi104 to 174.1M 2024-04-16T17:06:49.979 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:06:49 smithi045 ceph-mon[31676]: Unable to set osd_memory_target on smithi104 to 182633472: error parsing value: Value '182633472' is below minimum 939524096 2024-04-16T17:06:49.979 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:06:49 smithi045 ceph-mon[31676]: from='client.? 172.21.15.45:0/2662894184' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-16T17:06:49.979 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:06:49 smithi045 ceph-mon[31676]: pgmap v5: 129 pgs: 129 active+clean; 1.1 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T17:06:49.979 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:06:49 smithi045 ceph-mon[31676]: from='client.? 172.21.15.45:0/379042610' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-16T17:06:50.055 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:06:49 smithi117 ceph-mon[36730]: from='mgr.25411 ' entity='mgr.x' 2024-04-16T17:06:50.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:06:49 smithi117 ceph-mon[36730]: from='mgr.25411 ' entity='mgr.x' 2024-04-16T17:06:50.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:06:49 smithi117 ceph-mon[36730]: from='mgr.25411 ' entity='mgr.x' 2024-04-16T17:06:50.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:06:49 smithi117 ceph-mon[36730]: from='mgr.25411 ' entity='mgr.x' 2024-04-16T17:06:50.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:06:49 smithi117 ceph-mon[36730]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:06:50.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:06:49 smithi117 ceph-mon[36730]: from='mgr.25411 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:06:50.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:06:49 smithi117 ceph-mon[36730]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:06:50.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:06:49 smithi117 ceph-mon[36730]: from='mgr.25411 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:06:50.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:06:49 smithi117 ceph-mon[36730]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:06:50.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:06:49 smithi117 ceph-mon[36730]: from='mgr.25411 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:06:50.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:06:49 smithi117 ceph-mon[36730]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:06:50.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:06:49 smithi117 ceph-mon[36730]: from='mgr.25411 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:06:50.057 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:06:49 smithi117 ceph-mon[36730]: Adjusting osd_memory_target on smithi104 to 174.1M 2024-04-16T17:06:50.057 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:06:49 smithi117 ceph-mon[36730]: Unable to set osd_memory_target on smithi104 to 182633472: error parsing value: Value '182633472' is below minimum 939524096 2024-04-16T17:06:50.057 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:06:49 smithi117 ceph-mon[36730]: from='client.? 172.21.15.45:0/2662894184' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-16T17:06:50.057 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:06:49 smithi117 ceph-mon[36730]: pgmap v5: 129 pgs: 129 active+clean; 1.1 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-16T17:06:50.057 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:06:49 smithi117 ceph-mon[36730]: from='client.? 172.21.15.45:0/379042610' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-16T17:06:50.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:50.166+0000 7f7aea400640 1 -- 172.21.15.45:0/2206233644 --> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f7ae4062440 con 0x7f7acc0a3db0 2024-04-16T17:06:50.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:50.180+0000 7f7ae1600640 1 -- 172.21.15.45:0/2206233644 <== mgr.25411 v2:172.21.15.45:6800/751128195 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+339706 (secure 0 0 0) 0x7f7ae4062440 con 0x7f7acc0a3db0 2024-04-16T17:06:50.189 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-16T17:06:50.192 INFO:teuthology.orchestra.run.smithi045.stderr:dumped all 2024-04-16T17:06:50.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:50.189+0000 7f7aea400640 1 -- 172.21.15.45:0/2206233644 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f7acc0a3db0 msgr2=0x7f7acc0a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:50.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:50.189+0000 7f7aea400640 1 --2- 172.21.15.45:0/2206233644 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f7acc0a3db0 0x7f7acc0a6270 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f7ad80095f0 tx=0x7f7ad80058b0 comp rx=0 tx=0).stop 2024-04-16T17:06:50.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:50.189+0000 7f7aea400640 1 -- 172.21.15.45:0/2206233644 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7ae4105230 msgr2=0x7f7ae419a940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:50.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:50.189+0000 7f7aea400640 1 --2- 172.21.15.45:0/2206233644 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7ae4105230 0x7f7ae419a940 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f7ad0005fd0 tx=0x7f7ad000eb60 comp rx=0 tx=0).stop 2024-04-16T17:06:50.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:50.189+0000 7f7aea400640 1 -- 172.21.15.45:0/2206233644 shutdown_connections 2024-04-16T17:06:50.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:50.189+0000 7f7aea400640 1 --2- 172.21.15.45:0/2206233644 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f7acc0a3db0 0x7f7acc0a6270 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:50.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:50.189+0000 7f7aea400640 1 --2- 172.21.15.45:0/2206233644 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7ae4107b60 0x7f7ae41a19b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:50.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:50.189+0000 7f7aea400640 1 --2- 172.21.15.45:0/2206233644 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7ae4105230 0x7f7ae419a940 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:50.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:50.189+0000 7f7aea400640 1 --2- 172.21.15.45:0/2206233644 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7ae4102900 0x7f7ae419a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:50.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:50.189+0000 7f7aea400640 1 -- 172.21.15.45:0/2206233644 >> 172.21.15.45:0/2206233644 conn(0x7f7ae40fc2d0 msgr2=0x7f7ae4102060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:06:50.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:50.189+0000 7f7aea400640 1 -- 172.21.15.45:0/2206233644 shutdown_connections 2024-04-16T17:06:50.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:50.189+0000 7f7aea400640 1 -- 172.21.15.45:0/2206233644 wait complete. 2024-04-16T17:06:50.215 INFO:teuthology.orchestra.run.smithi045.stdout:{"pg_ready":true,"pg_map":{"version":5,"stamp":"2024-04-16T17:06:49.124283+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1190586,"num_objects":66,"num_object_clones":0,"num_object_copies":200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":1303,"num_read_kb":816,"num_write":481,"num_write_kb":4388,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":1181469,"num_keys_recovered":20,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":875,"ondisk_log_size":875,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":374476,"kb_used_data":12800,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124567348,"statfs":{"total":1151940427776,"available":1151556964352,"internally_reserved":0,"allocated":13107200,"data_stored":9917932,"data_compressed":539415,"data_compressed_allocated":729088,"data_compressed_original":4308992,"omap_allocated":19143,"internal_metadata":370193721},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.986206"},"pg_stats":[{"pgid":"3.3f","version":"92'8","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.557723+0000","last_change":"2024-04-16T17:03:24.989508+0000","last_active":"2024-04-16T17:06:45.557723+0000","last_peered":"2024-04-16T17:06:45.557723+0000","last_clean":"2024-04-16T17:06:45.557723+0000","last_became_active":"2024-04-16T17:03:24.989178+0000","last_became_peered":"2024-04-16T17:03:24.989178+0000","last_unstale":"2024-04-16T17:06:45.557723+0000","last_undegraded":"2024-04-16T17:06:45.557723+0000","last_fullsized":"2024-04-16T17:06:45.557723+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:18:56.488358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"92'4","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.577296+0000","last_change":"2024-04-16T17:03:24.986378+0000","last_active":"2024-04-16T17:06:45.577296+0000","last_peered":"2024-04-16T17:06:45.577296+0000","last_clean":"2024-04-16T17:06:45.577296+0000","last_became_active":"2024-04-16T17:03:24.986030+0000","last_became_peered":"2024-04-16T17:03:24.986030+0000","last_unstale":"2024-04-16T17:06:45.577296+0000","last_undegraded":"2024-04-16T17:06:45.577296+0000","last_fullsized":"2024-04-16T17:06:45.577296+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:21:14.071109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"92'7","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.577316+0000","last_change":"2024-04-16T17:03:24.976232+0000","last_active":"2024-04-16T17:06:45.577316+0000","last_peered":"2024-04-16T17:06:45.577316+0000","last_clean":"2024-04-16T17:06:45.577316+0000","last_became_active":"2024-04-16T17:03:24.975873+0000","last_became_peered":"2024-04-16T17:03:24.975873+0000","last_unstale":"2024-04-16T17:06:45.577316+0000","last_undegraded":"2024-04-16T17:06:45.577316+0000","last_fullsized":"2024-04-16T17:06:45.577316+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:07:53.523101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"92'8","reported_seq":26,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.102010+0000","last_change":"2024-04-16T17:04:21.517830+0000","last_active":"2024-04-16T17:06:45.102010+0000","last_peered":"2024-04-16T17:06:45.102010+0000","last_clean":"2024-04-16T17:06:45.102010+0000","last_became_active":"2024-04-16T17:04:21.516749+0000","last_became_peered":"2024-04-16T17:04:21.516749+0000","last_unstale":"2024-04-16T17:06:45.102010+0000","last_undegraded":"2024-04-16T17:06:45.102010+0000","last_fullsized":"2024-04-16T17:06:45.102010+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:02:33.344034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"92'4","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.239444+0000","last_change":"2024-04-16T17:03:24.985609+0000","last_active":"2024-04-16T17:06:45.239444+0000","last_peered":"2024-04-16T17:06:45.239444+0000","last_clean":"2024-04-16T17:06:45.239444+0000","last_became_active":"2024-04-16T17:03:24.985239+0000","last_became_peered":"2024-04-16T17:03:24.985239+0000","last_unstale":"2024-04-16T17:06:45.239444+0000","last_undegraded":"2024-04-16T17:06:45.239444+0000","last_fullsized":"2024-04-16T17:06:45.239444+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:55:58.619210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"92'6","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.227011+0000","last_change":"2024-04-16T17:03:24.966862+0000","last_active":"2024-04-16T17:06:45.227011+0000","last_peered":"2024-04-16T17:06:45.227011+0000","last_clean":"2024-04-16T17:06:45.227011+0000","last_became_active":"2024-04-16T17:03:24.966234+0000","last_became_peered":"2024-04-16T17:03:24.966234+0000","last_unstale":"2024-04-16T17:06:45.227011+0000","last_undegraded":"2024-04-16T17:06:45.227011+0000","last_fullsized":"2024-04-16T17:06:45.227011+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:21:07.018281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"92'5","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.442957+0000","last_change":"2024-04-16T17:03:24.971219+0000","last_active":"2024-04-16T17:06:45.442957+0000","last_peered":"2024-04-16T17:06:45.442957+0000","last_clean":"2024-04-16T17:06:45.442957+0000","last_became_active":"2024-04-16T17:03:24.970876+0000","last_became_peered":"2024-04-16T17:03:24.970876+0000","last_unstale":"2024-04-16T17:06:45.442957+0000","last_undegraded":"2024-04-16T17:06:45.442957+0000","last_fullsized":"2024-04-16T17:06:45.442957+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:44:16.311167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"92'9","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.239719+0000","last_change":"2024-04-16T17:03:24.994694+0000","last_active":"2024-04-16T17:06:45.239719+0000","last_peered":"2024-04-16T17:06:45.239719+0000","last_clean":"2024-04-16T17:06:45.239719+0000","last_became_active":"2024-04-16T17:03:24.994315+0000","last_became_peered":"2024-04-16T17:03:24.994315+0000","last_unstale":"2024-04-16T17:06:45.239719+0000","last_undegraded":"2024-04-16T17:06:45.239719+0000","last_fullsized":"2024-04-16T17:06:45.239719+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:04:59.543816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"92'11","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.451196+0000","last_change":"2024-04-16T17:03:24.964654+0000","last_active":"2024-04-16T17:06:45.451196+0000","last_peered":"2024-04-16T17:06:45.451196+0000","last_clean":"2024-04-16T17:06:45.451196+0000","last_became_active":"2024-04-16T17:03:24.964349+0000","last_became_peered":"2024-04-16T17:03:24.964349+0000","last_unstale":"2024-04-16T17:06:45.451196+0000","last_undegraded":"2024-04-16T17:06:45.451196+0000","last_fullsized":"2024-04-16T17:06:45.451196+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:38:55.880948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"92'16","reported_seq":48,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.226821+0000","last_change":"2024-04-16T17:03:24.966364+0000","last_active":"2024-04-16T17:06:45.226821+0000","last_peered":"2024-04-16T17:06:45.226821+0000","last_clean":"2024-04-16T17:06:45.226821+0000","last_became_active":"2024-04-16T17:03:24.966061+0000","last_became_peered":"2024-04-16T17:03:24.966061+0000","last_unstale":"2024-04-16T17:06:45.226821+0000","last_undegraded":"2024-04-16T17:06:45.226821+0000","last_fullsized":"2024-04-16T17:06:45.226821+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:18:12.642602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"92'6","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.443443+0000","last_change":"2024-04-16T17:03:24.969874+0000","last_active":"2024-04-16T17:06:45.443443+0000","last_peered":"2024-04-16T17:06:45.443443+0000","last_clean":"2024-04-16T17:06:45.443443+0000","last_became_active":"2024-04-16T17:03:24.969523+0000","last_became_peered":"2024-04-16T17:03:24.969523+0000","last_unstale":"2024-04-16T17:06:45.443443+0000","last_undegraded":"2024-04-16T17:06:45.443443+0000","last_fullsized":"2024-04-16T17:06:45.443443+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:17:49.704855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"92'7","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.105227+0000","last_change":"2024-04-16T17:03:24.968660+0000","last_active":"2024-04-16T17:06:45.105227+0000","last_peered":"2024-04-16T17:06:45.105227+0000","last_clean":"2024-04-16T17:06:45.105227+0000","last_became_active":"2024-04-16T17:03:24.968348+0000","last_became_peered":"2024-04-16T17:03:24.968348+0000","last_unstale":"2024-04-16T17:06:45.105227+0000","last_undegraded":"2024-04-16T17:06:45.105227+0000","last_fullsized":"2024-04-16T17:06:45.105227+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:06:06.319391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"92'10","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.444115+0000","last_change":"2024-04-16T17:03:24.974741+0000","last_active":"2024-04-16T17:06:45.444115+0000","last_peered":"2024-04-16T17:06:45.444115+0000","last_clean":"2024-04-16T17:06:45.444115+0000","last_became_active":"2024-04-16T17:03:24.973568+0000","last_became_peered":"2024-04-16T17:03:24.973568+0000","last_unstale":"2024-04-16T17:06:45.444115+0000","last_undegraded":"2024-04-16T17:06:45.444115+0000","last_fullsized":"2024-04-16T17:06:45.444115+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:30:49.325469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"92'7","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.576655+0000","last_change":"2024-04-16T17:03:24.981629+0000","last_active":"2024-04-16T17:06:45.576655+0000","last_peered":"2024-04-16T17:06:45.576655+0000","last_clean":"2024-04-16T17:06:45.576655+0000","last_became_active":"2024-04-16T17:03:24.981111+0000","last_became_peered":"2024-04-16T17:03:24.981111+0000","last_unstale":"2024-04-16T17:06:45.576655+0000","last_undegraded":"2024-04-16T17:06:45.576655+0000","last_fullsized":"2024-04-16T17:06:45.576655+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:41:29.641610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"92'7","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.576489+0000","last_change":"2024-04-16T17:03:24.977890+0000","last_active":"2024-04-16T17:06:45.576489+0000","last_peered":"2024-04-16T17:06:45.576489+0000","last_clean":"2024-04-16T17:06:45.576489+0000","last_became_active":"2024-04-16T17:03:24.976500+0000","last_became_peered":"2024-04-16T17:03:24.976500+0000","last_unstale":"2024-04-16T17:06:45.576489+0000","last_undegraded":"2024-04-16T17:06:45.576489+0000","last_fullsized":"2024-04-16T17:06:45.576489+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:17:51.604701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"92'9","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.450479+0000","last_change":"2024-04-16T17:03:24.976085+0000","last_active":"2024-04-16T17:06:45.450479+0000","last_peered":"2024-04-16T17:06:45.450479+0000","last_clean":"2024-04-16T17:06:45.450479+0000","last_became_active":"2024-04-16T17:03:24.975804+0000","last_became_peered":"2024-04-16T17:03:24.975804+0000","last_unstale":"2024-04-16T17:06:45.450479+0000","last_undegraded":"2024-04-16T17:06:45.450479+0000","last_fullsized":"2024-04-16T17:06:45.450479+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:19:13.706889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"92'8","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.557548+0000","last_change":"2024-04-16T17:03:24.989456+0000","last_active":"2024-04-16T17:06:45.557548+0000","last_peered":"2024-04-16T17:06:45.557548+0000","last_clean":"2024-04-16T17:06:45.557548+0000","last_became_active":"2024-04-16T17:03:24.989161+0000","last_became_peered":"2024-04-16T17:03:24.989161+0000","last_unstale":"2024-04-16T17:06:45.557548+0000","last_undegraded":"2024-04-16T17:06:45.557548+0000","last_fullsized":"2024-04-16T17:06:45.557548+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:52:59.333625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"92'10","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.102702+0000","last_change":"2024-04-16T17:03:24.963428+0000","last_active":"2024-04-16T17:06:45.102702+0000","last_peered":"2024-04-16T17:06:45.102702+0000","last_clean":"2024-04-16T17:06:45.102702+0000","last_became_active":"2024-04-16T17:03:24.962523+0000","last_became_peered":"2024-04-16T17:03:24.962523+0000","last_unstale":"2024-04-16T17:06:45.102702+0000","last_undegraded":"2024-04-16T17:06:45.102702+0000","last_fullsized":"2024-04-16T17:06:45.102702+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:22:19.523901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"92'7","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.105614+0000","last_change":"2024-04-16T17:03:24.981122+0000","last_active":"2024-04-16T17:06:45.105614+0000","last_peered":"2024-04-16T17:06:45.105614+0000","last_clean":"2024-04-16T17:06:45.105614+0000","last_became_active":"2024-04-16T17:03:24.980533+0000","last_became_peered":"2024-04-16T17:03:24.980533+0000","last_unstale":"2024-04-16T17:06:45.105614+0000","last_undegraded":"2024-04-16T17:06:45.105614+0000","last_fullsized":"2024-04-16T17:06:45.105614+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:51:08.580965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.105345+0000","last_change":"2024-04-16T17:03:29.257239+0000","last_active":"2024-04-16T17:06:45.105345+0000","last_peered":"2024-04-16T17:06:45.105345+0000","last_clean":"2024-04-16T17:06:45.105345+0000","last_became_active":"2024-04-16T17:03:29.256827+0000","last_became_peered":"2024-04-16T17:03:29.256827+0000","last_unstale":"2024-04-16T17:06:45.105345+0000","last_undegraded":"2024-04-16T17:06:45.105345+0000","last_fullsized":"2024-04-16T17:06:45.105345+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:57:28.373386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"90'9","reported_seq":60,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.102601+0000","last_change":"2024-04-16T17:04:21.896665+0000","last_active":"2024-04-16T17:06:45.102601+0000","last_peered":"2024-04-16T17:06:45.102601+0000","last_clean":"2024-04-16T17:06:45.102601+0000","last_became_active":"2024-04-16T17:04:21.882573+0000","last_became_peered":"2024-04-16T17:04:21.882573+0000","last_unstale":"2024-04-16T17:06:45.102601+0000","last_undegraded":"2024-04-16T17:06:45.102601+0000","last_fullsized":"2024-04-16T17:06:45.102601+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:37:07.210368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,1],"acting":[9,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"92'5","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.239912+0000","last_change":"2024-04-16T17:03:24.994921+0000","last_active":"2024-04-16T17:06:45.239912+0000","last_peered":"2024-04-16T17:06:45.239912+0000","last_clean":"2024-04-16T17:06:45.239912+0000","last_became_active":"2024-04-16T17:03:24.994412+0000","last_became_peered":"2024-04-16T17:03:24.994412+0000","last_unstale":"2024-04-16T17:06:45.239912+0000","last_undegraded":"2024-04-16T17:06:45.239912+0000","last_fullsized":"2024-04-16T17:06:45.239912+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:11:40.748198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.104669+0000","last_change":"2024-04-16T17:03:29.259884+0000","last_active":"2024-04-16T17:06:45.104669+0000","last_peered":"2024-04-16T17:06:45.104669+0000","last_clean":"2024-04-16T17:06:45.104669+0000","last_became_active":"2024-04-16T17:03:29.259319+0000","last_became_peered":"2024-04-16T17:03:29.259319+0000","last_unstale":"2024-04-16T17:06:45.104669+0000","last_undegraded":"2024-04-16T17:06:45.104669+0000","last_fullsized":"2024-04-16T17:06:45.104669+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:05:18.244682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"90'2","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.241406+0000","last_change":"2024-04-16T17:03:24.179308+0000","last_active":"2024-04-16T17:06:45.241406+0000","last_peered":"2024-04-16T17:06:45.241406+0000","last_clean":"2024-04-16T17:06:45.241406+0000","last_became_active":"2024-04-16T17:03:24.178894+0000","last_became_peered":"2024-04-16T17:03:24.178894+0000","last_unstale":"2024-04-16T17:06:45.241406+0000","last_undegraded":"2024-04-16T17:06:45.241406+0000","last_fullsized":"2024-04-16T17:06:45.241406+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:17:46.831871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"92'6","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.577690+0000","last_change":"2024-04-16T17:03:24.982391+0000","last_active":"2024-04-16T17:06:45.577690+0000","last_peered":"2024-04-16T17:06:45.577690+0000","last_clean":"2024-04-16T17:06:45.577690+0000","last_became_active":"2024-04-16T17:03:24.981761+0000","last_became_peered":"2024-04-16T17:03:24.981761+0000","last_unstale":"2024-04-16T17:06:45.577690+0000","last_undegraded":"2024-04-16T17:06:45.577690+0000","last_fullsized":"2024-04-16T17:06:45.577690+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:40:57.657170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.104794+0000","last_change":"2024-04-16T17:03:29.259705+0000","last_active":"2024-04-16T17:06:45.104794+0000","last_peered":"2024-04-16T17:06:45.104794+0000","last_clean":"2024-04-16T17:06:45.104794+0000","last_became_active":"2024-04-16T17:03:29.259336+0000","last_became_peered":"2024-04-16T17:03:29.259336+0000","last_unstale":"2024-04-16T17:06:45.104794+0000","last_undegraded":"2024-04-16T17:06:45.104794+0000","last_fullsized":"2024-04-16T17:06:45.104794+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:53:29.042933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"90'4","reported_seq":50,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.239845+0000","last_change":"2024-04-16T17:04:22.153413+0000","last_active":"2024-04-16T17:06:45.239845+0000","last_peered":"2024-04-16T17:06:45.239845+0000","last_clean":"2024-04-16T17:06:45.239845+0000","last_became_active":"2024-04-16T17:04:22.148972+0000","last_became_peered":"2024-04-16T17:04:22.148972+0000","last_unstale":"2024-04-16T17:06:45.239845+0000","last_undegraded":"2024-04-16T17:06:45.239845+0000","last_fullsized":"2024-04-16T17:06:45.239845+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:05:57.513052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"92'9","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.450605+0000","last_change":"2024-04-16T17:03:24.973205+0000","last_active":"2024-04-16T17:06:45.450605+0000","last_peered":"2024-04-16T17:06:45.450605+0000","last_clean":"2024-04-16T17:06:45.450605+0000","last_became_active":"2024-04-16T17:03:24.972863+0000","last_became_peered":"2024-04-16T17:03:24.972863+0000","last_unstale":"2024-04-16T17:06:45.450605+0000","last_undegraded":"2024-04-16T17:06:45.450605+0000","last_fullsized":"2024-04-16T17:06:45.450605+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:28:29.005783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.104818+0000","last_change":"2024-04-16T17:03:29.262930+0000","last_active":"2024-04-16T17:06:45.104818+0000","last_peered":"2024-04-16T17:06:45.104818+0000","last_clean":"2024-04-16T17:06:45.104818+0000","last_became_active":"2024-04-16T17:03:29.262621+0000","last_became_peered":"2024-04-16T17:03:29.262621+0000","last_unstale":"2024-04-16T17:06:45.104818+0000","last_undegraded":"2024-04-16T17:06:45.104818+0000","last_fullsized":"2024-04-16T17:06:45.104818+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:22:31.555143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.239656+0000","last_change":"2024-04-16T17:03:24.150503+0000","last_active":"2024-04-16T17:06:45.239656+0000","last_peered":"2024-04-16T17:06:45.239656+0000","last_clean":"2024-04-16T17:06:45.239656+0000","last_became_active":"2024-04-16T17:03:24.150021+0000","last_became_peered":"2024-04-16T17:03:24.150021+0000","last_unstale":"2024-04-16T17:06:45.239656+0000","last_undegraded":"2024-04-16T17:06:45.239656+0000","last_fullsized":"2024-04-16T17:06:45.239656+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:21:36.119497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"92'15","reported_seq":47,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.251429+0000","last_change":"2024-04-16T17:03:24.965049+0000","last_active":"2024-04-16T17:06:45.251429+0000","last_peered":"2024-04-16T17:06:45.251429+0000","last_clean":"2024-04-16T17:06:45.251429+0000","last_became_active":"2024-04-16T17:03:24.963120+0000","last_became_peered":"2024-04-16T17:03:24.963120+0000","last_unstale":"2024-04-16T17:06:45.251429+0000","last_undegraded":"2024-04-16T17:06:45.251429+0000","last_fullsized":"2024-04-16T17:06:45.251429+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:17:49.725113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.105640+0000","last_change":"2024-04-16T17:03:29.247013+0000","last_active":"2024-04-16T17:06:45.105640+0000","last_peered":"2024-04-16T17:06:45.105640+0000","last_clean":"2024-04-16T17:06:45.105640+0000","last_became_active":"2024-04-16T17:03:29.246477+0000","last_became_peered":"2024-04-16T17:03:29.246477+0000","last_unstale":"2024-04-16T17:06:45.105640+0000","last_undegraded":"2024-04-16T17:06:45.105640+0000","last_fullsized":"2024-04-16T17:06:45.105640+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:52:19.525076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"95'8","reported_seq":220,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.629994+0000","last_change":"2024-04-16T17:03:24.155333+0000","last_active":"2024-04-16T17:06:45.629994+0000","last_peered":"2024-04-16T17:06:45.629994+0000","last_clean":"2024-04-16T17:06:45.629994+0000","last_became_active":"2024-04-16T17:03:24.154848+0000","last_became_peered":"2024-04-16T17:03:24.154848+0000","last_unstale":"2024-04-16T17:06:45.629994+0000","last_undegraded":"2024-04-16T17:06:45.629994+0000","last_fullsized":"2024-04-16T17:06:45.629994+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:03:34.769573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1545,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":178,"num_read_kb":5,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"92'7","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.239446+0000","last_change":"2024-04-16T17:03:24.967627+0000","last_active":"2024-04-16T17:06:45.239446+0000","last_peered":"2024-04-16T17:06:45.239446+0000","last_clean":"2024-04-16T17:06:45.239446+0000","last_became_active":"2024-04-16T17:03:24.967264+0000","last_became_peered":"2024-04-16T17:03:24.967264+0000","last_unstale":"2024-04-16T17:06:45.239446+0000","last_undegraded":"2024-04-16T17:06:45.239446+0000","last_fullsized":"2024-04-16T17:06:45.239446+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:35:29.182374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.105252+0000","last_change":"2024-04-16T17:03:29.264625+0000","last_active":"2024-04-16T17:06:45.105252+0000","last_peered":"2024-04-16T17:06:45.105252+0000","last_clean":"2024-04-16T17:06:45.105252+0000","last_became_active":"2024-04-16T17:03:29.264114+0000","last_became_peered":"2024-04-16T17:03:29.264114+0000","last_unstale":"2024-04-16T17:06:45.105252+0000","last_undegraded":"2024-04-16T17:06:45.105252+0000","last_fullsized":"2024-04-16T17:06:45.105252+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:14:49.449980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"90'3","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.226892+0000","last_change":"2024-04-16T17:03:24.160457+0000","last_active":"2024-04-16T17:06:45.226892+0000","last_peered":"2024-04-16T17:06:45.226892+0000","last_clean":"2024-04-16T17:06:45.226892+0000","last_became_active":"2024-04-16T17:03:24.160195+0000","last_became_peered":"2024-04-16T17:03:24.160195+0000","last_unstale":"2024-04-16T17:06:45.226892+0000","last_undegraded":"2024-04-16T17:06:45.226892+0000","last_fullsized":"2024-04-16T17:06:45.226892+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:23:07.064539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"92'8","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.251287+0000","last_change":"2024-04-16T17:03:24.991249+0000","last_active":"2024-04-16T17:06:45.251287+0000","last_peered":"2024-04-16T17:06:45.251287+0000","last_clean":"2024-04-16T17:06:45.251287+0000","last_became_active":"2024-04-16T17:03:24.990927+0000","last_became_peered":"2024-04-16T17:03:24.990927+0000","last_unstale":"2024-04-16T17:06:45.251287+0000","last_undegraded":"2024-04-16T17:06:45.251287+0000","last_fullsized":"2024-04-16T17:06:45.251287+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:36:39.975616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.557483+0000","last_change":"2024-04-16T17:03:29.253704+0000","last_active":"2024-04-16T17:06:45.557483+0000","last_peered":"2024-04-16T17:06:45.557483+0000","last_clean":"2024-04-16T17:06:45.557483+0000","last_became_active":"2024-04-16T17:03:29.253334+0000","last_became_peered":"2024-04-16T17:03:29.253334+0000","last_unstale":"2024-04-16T17:06:45.557483+0000","last_undegraded":"2024-04-16T17:06:45.557483+0000","last_fullsized":"2024-04-16T17:06:45.557483+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:19:12.611731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"90'2","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.444031+0000","last_change":"2024-04-16T17:03:24.158875+0000","last_active":"2024-04-16T17:06:45.444031+0000","last_peered":"2024-04-16T17:06:45.444031+0000","last_clean":"2024-04-16T17:06:45.444031+0000","last_became_active":"2024-04-16T17:03:24.158465+0000","last_became_peered":"2024-04-16T17:03:24.158465+0000","last_unstale":"2024-04-16T17:06:45.444031+0000","last_undegraded":"2024-04-16T17:06:45.444031+0000","last_fullsized":"2024-04-16T17:06:45.444031+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:21:29.882251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"92'8","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.239904+0000","last_change":"2024-04-16T17:03:24.994476+0000","last_active":"2024-04-16T17:06:45.239904+0000","last_peered":"2024-04-16T17:06:45.239904+0000","last_clean":"2024-04-16T17:06:45.239904+0000","last_became_active":"2024-04-16T17:03:24.993532+0000","last_became_peered":"2024-04-16T17:03:24.993532+0000","last_unstale":"2024-04-16T17:06:45.239904+0000","last_undegraded":"2024-04-16T17:06:45.239904+0000","last_fullsized":"2024-04-16T17:06:45.239904+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:04:44.062444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.577184+0000","last_change":"2024-04-16T17:03:29.255446+0000","last_active":"2024-04-16T17:06:45.577184+0000","last_peered":"2024-04-16T17:06:45.577184+0000","last_clean":"2024-04-16T17:06:45.577184+0000","last_became_active":"2024-04-16T17:03:29.255051+0000","last_became_peered":"2024-04-16T17:03:29.255051+0000","last_unstale":"2024-04-16T17:06:45.577184+0000","last_undegraded":"2024-04-16T17:06:45.577184+0000","last_fullsized":"2024-04-16T17:06:45.577184+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:24:04.946342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"90'1","reported_seq":35,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.250870+0000","last_change":"2024-04-16T17:03:24.154904+0000","last_active":"2024-04-16T17:06:45.250870+0000","last_peered":"2024-04-16T17:06:45.250870+0000","last_clean":"2024-04-16T17:06:45.250870+0000","last_became_active":"2024-04-16T17:03:24.154617+0000","last_became_peered":"2024-04-16T17:03:24.154617+0000","last_unstale":"2024-04-16T17:06:45.250870+0000","last_undegraded":"2024-04-16T17:06:45.250870+0000","last_fullsized":"2024-04-16T17:06:45.250870+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:35:55.498855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"92'11","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.101961+0000","last_change":"2024-04-16T17:03:24.979908+0000","last_active":"2024-04-16T17:06:45.101961+0000","last_peered":"2024-04-16T17:06:45.101961+0000","last_clean":"2024-04-16T17:06:45.101961+0000","last_became_active":"2024-04-16T17:03:24.979501+0000","last_became_peered":"2024-04-16T17:03:24.979501+0000","last_unstale":"2024-04-16T17:06:45.101961+0000","last_undegraded":"2024-04-16T17:06:45.101961+0000","last_fullsized":"2024-04-16T17:06:45.101961+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:51:05.421459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.240099+0000","last_change":"2024-04-16T17:03:29.258809+0000","last_active":"2024-04-16T17:06:45.240099+0000","last_peered":"2024-04-16T17:06:45.240099+0000","last_clean":"2024-04-16T17:06:45.240099+0000","last_became_active":"2024-04-16T17:03:29.256629+0000","last_became_peered":"2024-04-16T17:03:29.256629+0000","last_unstale":"2024-04-16T17:06:45.240099+0000","last_undegraded":"2024-04-16T17:06:45.240099+0000","last_fullsized":"2024-04-16T17:06:45.240099+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:46:51.634944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"95'5","reported_seq":212,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.445694+0000","last_change":"2024-04-16T17:03:24.153774+0000","last_active":"2024-04-16T17:06:45.445694+0000","last_peered":"2024-04-16T17:06:45.445694+0000","last_clean":"2024-04-16T17:06:45.445694+0000","last_became_active":"2024-04-16T17:03:24.153418+0000","last_became_peered":"2024-04-16T17:03:24.153418+0000","last_unstale":"2024-04-16T17:06:45.445694+0000","last_undegraded":"2024-04-16T17:06:45.445694+0000","last_fullsized":"2024-04-16T17:06:45.445694+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:58:36.093789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":173,"num_read_kb":173,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"92'10","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.451116+0000","last_change":"2024-04-16T17:03:24.962838+0000","last_active":"2024-04-16T17:06:45.451116+0000","last_peered":"2024-04-16T17:06:45.451116+0000","last_clean":"2024-04-16T17:06:45.451116+0000","last_became_active":"2024-04-16T17:03:24.960163+0000","last_became_peered":"2024-04-16T17:03:24.960163+0000","last_unstale":"2024-04-16T17:06:45.451116+0000","last_undegraded":"2024-04-16T17:06:45.451116+0000","last_fullsized":"2024-04-16T17:06:45.451116+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:48:18.723499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.101798+0000","last_change":"2024-04-16T17:03:29.251177+0000","last_active":"2024-04-16T17:06:45.101798+0000","last_peered":"2024-04-16T17:06:45.101798+0000","last_clean":"2024-04-16T17:06:45.101798+0000","last_became_active":"2024-04-16T17:03:29.250657+0000","last_became_peered":"2024-04-16T17:03:29.250657+0000","last_unstale":"2024-04-16T17:06:45.101798+0000","last_undegraded":"2024-04-16T17:06:45.101798+0000","last_fullsized":"2024-04-16T17:06:45.101798+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:16:47.217193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"90'2","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.443359+0000","last_change":"2024-04-16T17:03:24.154245+0000","last_active":"2024-04-16T17:06:45.443359+0000","last_peered":"2024-04-16T17:06:45.443359+0000","last_clean":"2024-04-16T17:06:45.443359+0000","last_became_active":"2024-04-16T17:03:24.153297+0000","last_became_peered":"2024-04-16T17:03:24.153297+0000","last_unstale":"2024-04-16T17:06:45.443359+0000","last_undegraded":"2024-04-16T17:06:45.443359+0000","last_fullsized":"2024-04-16T17:06:45.443359+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:23:49.490238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"92'9","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.577695+0000","last_change":"2024-04-16T17:03:24.984572+0000","last_active":"2024-04-16T17:06:45.577695+0000","last_peered":"2024-04-16T17:06:45.577695+0000","last_clean":"2024-04-16T17:06:45.577695+0000","last_became_active":"2024-04-16T17:03:24.983588+0000","last_became_peered":"2024-04-16T17:03:24.983588+0000","last_unstale":"2024-04-16T17:06:45.577695+0000","last_undegraded":"2024-04-16T17:06:45.577695+0000","last_fullsized":"2024-04-16T17:06:45.577695+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:41:24.704602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.105052+0000","last_change":"2024-04-16T17:03:29.245129+0000","last_active":"2024-04-16T17:06:45.105052+0000","last_peered":"2024-04-16T17:06:45.105052+0000","last_clean":"2024-04-16T17:06:45.105052+0000","last_became_active":"2024-04-16T17:03:29.244720+0000","last_became_peered":"2024-04-16T17:03:29.244720+0000","last_unstale":"2024-04-16T17:06:45.105052+0000","last_undegraded":"2024-04-16T17:06:45.105052+0000","last_fullsized":"2024-04-16T17:06:45.105052+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:55:26.612176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"90'4","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.251165+0000","last_change":"2024-04-16T17:03:24.165248+0000","last_active":"2024-04-16T17:06:45.251165+0000","last_peered":"2024-04-16T17:06:45.251165+0000","last_clean":"2024-04-16T17:06:45.251165+0000","last_became_active":"2024-04-16T17:03:24.164915+0000","last_became_peered":"2024-04-16T17:03:24.164915+0000","last_unstale":"2024-04-16T17:06:45.251165+0000","last_undegraded":"2024-04-16T17:06:45.251165+0000","last_fullsized":"2024-04-16T17:06:45.251165+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:04:47.207554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"92'11","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.450807+0000","last_change":"2024-04-16T17:03:24.967292+0000","last_active":"2024-04-16T17:06:45.450807+0000","last_peered":"2024-04-16T17:06:45.450807+0000","last_clean":"2024-04-16T17:06:45.450807+0000","last_became_active":"2024-04-16T17:03:24.966952+0000","last_became_peered":"2024-04-16T17:03:24.966952+0000","last_unstale":"2024-04-16T17:06:45.450807+0000","last_undegraded":"2024-04-16T17:06:45.450807+0000","last_fullsized":"2024-04-16T17:06:45.450807+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:42:54.969478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.101807+0000","last_change":"2024-04-16T17:03:29.257836+0000","last_active":"2024-04-16T17:06:45.101807+0000","last_peered":"2024-04-16T17:06:45.101807+0000","last_clean":"2024-04-16T17:06:45.101807+0000","last_became_active":"2024-04-16T17:03:29.257409+0000","last_became_peered":"2024-04-16T17:03:29.257409+0000","last_unstale":"2024-04-16T17:06:45.101807+0000","last_undegraded":"2024-04-16T17:06:45.101807+0000","last_fullsized":"2024-04-16T17:06:45.101807+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:25:21.288266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"95'41","reported_seq":433,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.629791+0000","last_change":"2024-04-16T17:03:24.150943+0000","last_active":"2024-04-16T17:06:45.629791+0000","last_peered":"2024-04-16T17:06:45.629791+0000","last_clean":"2024-04-16T17:06:45.629791+0000","last_became_active":"2024-04-16T17:03:24.150563+0000","last_became_peered":"2024-04-16T17:03:24.150563+0000","last_unstale":"2024-04-16T17:06:45.629791+0000","last_undegraded":"2024-04-16T17:06:45.629791+0000","last_fullsized":"2024-04-16T17:06:45.629791+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:05:07.380440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":69011,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":358,"num_read_kb":247,"num_write":42,"num_write_kb":89,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"92'6","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.239659+0000","last_change":"2024-04-16T17:03:24.966393+0000","last_active":"2024-04-16T17:06:45.239659+0000","last_peered":"2024-04-16T17:06:45.239659+0000","last_clean":"2024-04-16T17:06:45.239659+0000","last_became_active":"2024-04-16T17:03:24.966016+0000","last_became_peered":"2024-04-16T17:03:24.966016+0000","last_unstale":"2024-04-16T17:06:45.239659+0000","last_undegraded":"2024-04-16T17:06:45.239659+0000","last_fullsized":"2024-04-16T17:06:45.239659+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:30:25.203262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.251684+0000","last_change":"2024-04-16T17:03:29.265371+0000","last_active":"2024-04-16T17:06:45.251684+0000","last_peered":"2024-04-16T17:06:45.251684+0000","last_clean":"2024-04-16T17:06:45.251684+0000","last_became_active":"2024-04-16T17:03:29.264933+0000","last_became_peered":"2024-04-16T17:03:29.264933+0000","last_unstale":"2024-04-16T17:06:45.251684+0000","last_undegraded":"2024-04-16T17:06:45.251684+0000","last_fullsized":"2024-04-16T17:06:45.251684+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:50:18.203015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"90'6","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.557238+0000","last_change":"2024-04-16T17:04:21.526796+0000","last_active":"2024-04-16T17:06:45.557238+0000","last_peered":"2024-04-16T17:06:45.557238+0000","last_clean":"2024-04-16T17:06:45.557238+0000","last_became_active":"2024-04-16T17:04:21.508364+0000","last_became_peered":"2024-04-16T17:04:21.508364+0000","last_unstale":"2024-04-16T17:06:45.557238+0000","last_undegraded":"2024-04-16T17:06:45.557238+0000","last_fullsized":"2024-04-16T17:06:45.557238+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:45:42.726648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"92'9","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.443893+0000","last_change":"2024-04-16T17:03:24.986228+0000","last_active":"2024-04-16T17:06:45.443893+0000","last_peered":"2024-04-16T17:06:45.443893+0000","last_clean":"2024-04-16T17:06:45.443893+0000","last_became_active":"2024-04-16T17:03:24.985916+0000","last_became_peered":"2024-04-16T17:03:24.985916+0000","last_unstale":"2024-04-16T17:06:45.443893+0000","last_undegraded":"2024-04-16T17:06:45.443893+0000","last_fullsized":"2024-04-16T17:06:45.443893+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:23:54.309203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.557348+0000","last_change":"2024-04-16T17:03:29.264636+0000","last_active":"2024-04-16T17:06:45.557348+0000","last_peered":"2024-04-16T17:06:45.557348+0000","last_clean":"2024-04-16T17:06:45.557348+0000","last_became_active":"2024-04-16T17:03:29.264117+0000","last_became_peered":"2024-04-16T17:03:29.264117+0000","last_unstale":"2024-04-16T17:06:45.557348+0000","last_undegraded":"2024-04-16T17:06:45.557348+0000","last_fullsized":"2024-04-16T17:06:45.557348+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:42:36.123154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"90'3","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.251233+0000","last_change":"2024-04-16T17:03:24.166844+0000","last_active":"2024-04-16T17:06:45.251233+0000","last_peered":"2024-04-16T17:06:45.251233+0000","last_clean":"2024-04-16T17:06:45.251233+0000","last_became_active":"2024-04-16T17:03:24.166479+0000","last_became_peered":"2024-04-16T17:03:24.166479+0000","last_unstale":"2024-04-16T17:06:45.251233+0000","last_undegraded":"2024-04-16T17:06:45.251233+0000","last_fullsized":"2024-04-16T17:06:45.251233+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:02:20.139865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"92'15","reported_seq":47,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.443198+0000","last_change":"2024-04-16T17:03:24.973057+0000","last_active":"2024-04-16T17:06:45.443198+0000","last_peered":"2024-04-16T17:06:45.443198+0000","last_clean":"2024-04-16T17:06:45.443198+0000","last_became_active":"2024-04-16T17:03:24.972731+0000","last_became_peered":"2024-04-16T17:03:24.972731+0000","last_unstale":"2024-04-16T17:06:45.443198+0000","last_undegraded":"2024-04-16T17:06:45.443198+0000","last_fullsized":"2024-04-16T17:06:45.443198+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:37:38.702919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.102363+0000","last_change":"2024-04-16T17:03:29.259167+0000","last_active":"2024-04-16T17:06:45.102363+0000","last_peered":"2024-04-16T17:06:45.102363+0000","last_clean":"2024-04-16T17:06:45.102363+0000","last_became_active":"2024-04-16T17:03:29.258737+0000","last_became_peered":"2024-04-16T17:03:29.258737+0000","last_unstale":"2024-04-16T17:06:45.102363+0000","last_undegraded":"2024-04-16T17:06:45.102363+0000","last_fullsized":"2024-04-16T17:06:45.102363+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:01:13.774037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"95'8","reported_seq":221,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.788731+0000","last_change":"2024-04-16T17:03:24.151064+0000","last_active":"2024-04-16T17:06:45.788731+0000","last_peered":"2024-04-16T17:06:45.788731+0000","last_clean":"2024-04-16T17:06:45.788731+0000","last_became_active":"2024-04-16T17:03:24.150745+0000","last_became_peered":"2024-04-16T17:03:24.150745+0000","last_unstale":"2024-04-16T17:06:45.788731+0000","last_undegraded":"2024-04-16T17:06:45.788731+0000","last_fullsized":"2024-04-16T17:06:45.788731+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:32:26.271008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2212,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":180,"num_read_kb":7,"num_write":7,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"92'6","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.442895+0000","last_change":"2024-04-16T17:03:24.971948+0000","last_active":"2024-04-16T17:06:45.442895+0000","last_peered":"2024-04-16T17:06:45.442895+0000","last_clean":"2024-04-16T17:06:45.442895+0000","last_became_active":"2024-04-16T17:03:24.971530+0000","last_became_peered":"2024-04-16T17:03:24.971530+0000","last_unstale":"2024-04-16T17:06:45.442895+0000","last_undegraded":"2024-04-16T17:06:45.442895+0000","last_fullsized":"2024-04-16T17:06:45.442895+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:03:29.286160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.445918+0000","last_change":"2024-04-16T17:03:29.264821+0000","last_active":"2024-04-16T17:06:45.445918+0000","last_peered":"2024-04-16T17:06:45.445918+0000","last_clean":"2024-04-16T17:06:45.445918+0000","last_became_active":"2024-04-16T17:03:29.264287+0000","last_became_peered":"2024-04-16T17:03:29.264287+0000","last_unstale":"2024-04-16T17:06:45.445918+0000","last_undegraded":"2024-04-16T17:06:45.445918+0000","last_fullsized":"2024-04-16T17:06:45.445918+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:33:28.400663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"90'15","reported_seq":50,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.239188+0000","last_change":"2024-04-16T17:03:24.158844+0000","last_active":"2024-04-16T17:06:45.239188+0000","last_peered":"2024-04-16T17:06:45.239188+0000","last_clean":"2024-04-16T17:06:45.239188+0000","last_became_active":"2024-04-16T17:03:24.158537+0000","last_became_peered":"2024-04-16T17:03:24.158537+0000","last_unstale":"2024-04-16T17:06:45.239188+0000","last_undegraded":"2024-04-16T17:06:45.239188+0000","last_fullsized":"2024-04-16T17:06:45.239188+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:42:22.772452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"92'6","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.251080+0000","last_change":"2024-04-16T17:03:24.962522+0000","last_active":"2024-04-16T17:06:45.251080+0000","last_peered":"2024-04-16T17:06:45.251080+0000","last_clean":"2024-04-16T17:06:45.251080+0000","last_became_active":"2024-04-16T17:03:24.962169+0000","last_became_peered":"2024-04-16T17:03:24.962169+0000","last_unstale":"2024-04-16T17:06:45.251080+0000","last_undegraded":"2024-04-16T17:06:45.251080+0000","last_fullsized":"2024-04-16T17:06:45.251080+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:28:17.147491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.250937+0000","last_change":"2024-04-16T17:03:29.259539+0000","last_active":"2024-04-16T17:06:45.250937+0000","last_peered":"2024-04-16T17:06:45.250937+0000","last_clean":"2024-04-16T17:06:45.250937+0000","last_became_active":"2024-04-16T17:03:29.259123+0000","last_became_peered":"2024-04-16T17:03:29.259123+0000","last_unstale":"2024-04-16T17:06:45.250937+0000","last_undegraded":"2024-04-16T17:06:45.250937+0000","last_fullsized":"2024-04-16T17:06:45.250937+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:15:17.142636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"94'3","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.239865+0000","last_change":"2024-04-16T17:03:24.147526+0000","last_active":"2024-04-16T17:06:45.239865+0000","last_peered":"2024-04-16T17:06:45.239865+0000","last_clean":"2024-04-16T17:06:45.239865+0000","last_became_active":"2024-04-16T17:03:24.147119+0000","last_became_peered":"2024-04-16T17:03:24.147119+0000","last_unstale":"2024-04-16T17:06:45.239865+0000","last_undegraded":"2024-04-16T17:06:45.239865+0000","last_fullsized":"2024-04-16T17:06:45.239865+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:30:38.742122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"92'5","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.557187+0000","last_change":"2024-04-16T17:04:21.507113+0000","last_active":"2024-04-16T17:06:45.557187+0000","last_peered":"2024-04-16T17:06:45.557187+0000","last_clean":"2024-04-16T17:06:45.557187+0000","last_became_active":"2024-04-16T17:04:21.506680+0000","last_became_peered":"2024-04-16T17:04:21.506680+0000","last_unstale":"2024-04-16T17:06:45.557187+0000","last_undegraded":"2024-04-16T17:06:45.557187+0000","last_fullsized":"2024-04-16T17:06:45.557187+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:08:47.971317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,8],"acting":[11,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":32,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.239441+0000","last_change":"2024-04-16T17:05:22.511966+0000","last_active":"2024-04-16T17:06:45.239441+0000","last_peered":"2024-04-16T17:06:45.239441+0000","last_clean":"2024-04-16T17:06:45.239441+0000","last_became_active":"2024-04-16T17:05:22.511512+0000","last_became_peered":"2024-04-16T17:05:22.511512+0000","last_unstale":"2024-04-16T17:06:45.239441+0000","last_undegraded":"2024-04-16T17:06:45.239441+0000","last_fullsized":"2024-04-16T17:06:45.239441+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:14:41.291499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,1],"acting":[8,10,11,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.102181+0000","last_change":"2024-04-16T17:03:24.162105+0000","last_active":"2024-04-16T17:06:45.102181+0000","last_peered":"2024-04-16T17:06:45.102181+0000","last_clean":"2024-04-16T17:06:45.102181+0000","last_became_active":"2024-04-16T17:03:24.161631+0000","last_became_peered":"2024-04-16T17:03:24.161631+0000","last_unstale":"2024-04-16T17:06:45.102181+0000","last_undegraded":"2024-04-16T17:06:45.102181+0000","last_fullsized":"2024-04-16T17:06:45.102181+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:22:33.677848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"92'4","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.226784+0000","last_change":"2024-04-16T17:03:24.978253+0000","last_active":"2024-04-16T17:06:45.226784+0000","last_peered":"2024-04-16T17:06:45.226784+0000","last_clean":"2024-04-16T17:06:45.226784+0000","last_became_active":"2024-04-16T17:03:24.977956+0000","last_became_peered":"2024-04-16T17:03:24.977956+0000","last_unstale":"2024-04-16T17:06:45.226784+0000","last_undegraded":"2024-04-16T17:06:45.226784+0000","last_fullsized":"2024-04-16T17:06:45.226784+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:59:45.102383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.444635+0000","last_change":"2024-04-16T17:03:29.265312+0000","last_active":"2024-04-16T17:06:45.444635+0000","last_peered":"2024-04-16T17:06:45.444635+0000","last_clean":"2024-04-16T17:06:45.444635+0000","last_became_active":"2024-04-16T17:03:29.264891+0000","last_became_peered":"2024-04-16T17:03:29.264891+0000","last_unstale":"2024-04-16T17:06:45.444635+0000","last_undegraded":"2024-04-16T17:06:45.444635+0000","last_fullsized":"2024-04-16T17:06:45.444635+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:26:40.088127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"90'1","reported_seq":35,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.442929+0000","last_change":"2024-04-16T17:03:24.170935+0000","last_active":"2024-04-16T17:06:45.442929+0000","last_peered":"2024-04-16T17:06:45.442929+0000","last_clean":"2024-04-16T17:06:45.442929+0000","last_became_active":"2024-04-16T17:03:24.170609+0000","last_became_peered":"2024-04-16T17:03:24.170609+0000","last_unstale":"2024-04-16T17:06:45.442929+0000","last_undegraded":"2024-04-16T17:06:45.442929+0000","last_fullsized":"2024-04-16T17:06:45.442929+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:18:33.516309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"92'7","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.226731+0000","last_change":"2024-04-16T17:03:24.985434+0000","last_active":"2024-04-16T17:06:45.226731+0000","last_peered":"2024-04-16T17:06:45.226731+0000","last_clean":"2024-04-16T17:06:45.226731+0000","last_became_active":"2024-04-16T17:03:24.984388+0000","last_became_peered":"2024-04-16T17:03:24.984388+0000","last_unstale":"2024-04-16T17:06:45.226731+0000","last_undegraded":"2024-04-16T17:06:45.226731+0000","last_fullsized":"2024-04-16T17:06:45.226731+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:43:26.355572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":32,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.445935+0000","last_change":"2024-04-16T17:05:22.516267+0000","last_active":"2024-04-16T17:06:45.445935+0000","last_peered":"2024-04-16T17:06:45.445935+0000","last_clean":"2024-04-16T17:06:45.445935+0000","last_became_active":"2024-04-16T17:05:22.513637+0000","last_became_peered":"2024-04-16T17:05:22.513637+0000","last_unstale":"2024-04-16T17:06:45.445935+0000","last_undegraded":"2024-04-16T17:06:45.445935+0000","last_fullsized":"2024-04-16T17:06:45.445935+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:39:47.975735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,8,10,0],"acting":[6,8,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"92'8","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.251371+0000","last_change":"2024-04-16T17:03:24.962786+0000","last_active":"2024-04-16T17:06:45.251371+0000","last_peered":"2024-04-16T17:06:45.251371+0000","last_clean":"2024-04-16T17:06:45.251371+0000","last_became_active":"2024-04-16T17:03:24.961704+0000","last_became_peered":"2024-04-16T17:03:24.961704+0000","last_unstale":"2024-04-16T17:06:45.251371+0000","last_undegraded":"2024-04-16T17:06:45.251371+0000","last_fullsized":"2024-04-16T17:06:45.251371+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:22:27.957607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"94'6","reported_seq":213,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.443944+0000","last_change":"2024-04-16T17:03:24.175617+0000","last_active":"2024-04-16T17:06:45.443944+0000","last_peered":"2024-04-16T17:06:45.443944+0000","last_clean":"2024-04-16T17:06:45.443944+0000","last_became_active":"2024-04-16T17:03:24.175272+0000","last_became_peered":"2024-04-16T17:03:24.175272+0000","last_unstale":"2024-04-16T17:06:45.443944+0000","last_undegraded":"2024-04-16T17:06:45.443944+0000","last_fullsized":"2024-04-16T17:06:45.443944+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:27:44.738030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":173,"num_read_kb":173,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"90'1","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.443256+0000","last_change":"2024-04-16T17:04:21.512033+0000","last_active":"2024-04-16T17:06:45.443256+0000","last_peered":"2024-04-16T17:06:45.443256+0000","last_clean":"2024-04-16T17:06:45.443256+0000","last_became_active":"2024-04-16T17:04:21.506718+0000","last_became_peered":"2024-04-16T17:04:21.506718+0000","last_unstale":"2024-04-16T17:06:45.443256+0000","last_undegraded":"2024-04-16T17:06:45.443256+0000","last_fullsized":"2024-04-16T17:06:45.443256+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:53:48.953887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,10],"acting":[7,5,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"92'8","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.240468+0000","last_change":"2024-04-16T17:03:24.983629+0000","last_active":"2024-04-16T17:06:45.240468+0000","last_peered":"2024-04-16T17:06:45.240468+0000","last_clean":"2024-04-16T17:06:45.240468+0000","last_became_active":"2024-04-16T17:03:24.979969+0000","last_became_peered":"2024-04-16T17:03:24.979969+0000","last_unstale":"2024-04-16T17:06:45.240468+0000","last_undegraded":"2024-04-16T17:06:45.240468+0000","last_fullsized":"2024-04-16T17:06:45.240468+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:01:24.391732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"90'3","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.104956+0000","last_change":"2024-04-16T17:03:24.152475+0000","last_active":"2024-04-16T17:06:45.104956+0000","last_peered":"2024-04-16T17:06:45.104956+0000","last_clean":"2024-04-16T17:06:45.104956+0000","last_became_active":"2024-04-16T17:03:24.152198+0000","last_became_peered":"2024-04-16T17:03:24.152198+0000","last_unstale":"2024-04-16T17:06:45.104956+0000","last_undegraded":"2024-04-16T17:06:45.104956+0000","last_fullsized":"2024-04-16T17:06:45.104956+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:49:33.914325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":367,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.442836+0000","last_change":"2024-04-16T17:02:02.111335+0000","last_active":"2024-04-16T17:06:45.442836+0000","last_peered":"2024-04-16T17:06:45.442836+0000","last_clean":"2024-04-16T17:06:45.442836+0000","last_became_active":"2024-04-16T17:02:02.110928+0000","last_became_peered":"2024-04-16T17:02:02.110928+0000","last_unstale":"2024-04-16T17:06:45.442836+0000","last_undegraded":"2024-04-16T17:06:45.442836+0000","last_fullsized":"2024-04-16T17:06:45.442836+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T16:58:07.081753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T16:58:07.081753+0000","last_clean_scrub_stamp":"2024-04-16T16:58:07.081753+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:27:02.865664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.251060+0000","last_change":"2024-04-16T17:03:29.301691+0000","last_active":"2024-04-16T17:06:45.251060+0000","last_peered":"2024-04-16T17:06:45.251060+0000","last_clean":"2024-04-16T17:06:45.251060+0000","last_became_active":"2024-04-16T17:03:29.301277+0000","last_became_peered":"2024-04-16T17:03:29.301277+0000","last_unstale":"2024-04-16T17:06:45.251060+0000","last_undegraded":"2024-04-16T17:06:45.251060+0000","last_fullsized":"2024-04-16T17:06:45.251060+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:45:16.727610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"92'5","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.443705+0000","last_change":"2024-04-16T17:03:24.970939+0000","last_active":"2024-04-16T17:06:45.443705+0000","last_peered":"2024-04-16T17:06:45.443705+0000","last_clean":"2024-04-16T17:06:45.443705+0000","last_became_active":"2024-04-16T17:03:24.970581+0000","last_became_peered":"2024-04-16T17:03:24.970581+0000","last_unstale":"2024-04-16T17:06:45.443705+0000","last_undegraded":"2024-04-16T17:06:45.443705+0000","last_fullsized":"2024-04-16T17:06:45.443705+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:33:07.448597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"90'3","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.442815+0000","last_change":"2024-04-16T17:03:24.155454+0000","last_active":"2024-04-16T17:06:45.442815+0000","last_peered":"2024-04-16T17:06:45.442815+0000","last_clean":"2024-04-16T17:06:45.442815+0000","last_became_active":"2024-04-16T17:03:24.155155+0000","last_became_peered":"2024-04-16T17:03:24.155155+0000","last_unstale":"2024-04-16T17:06:45.442815+0000","last_undegraded":"2024-04-16T17:06:45.442815+0000","last_fullsized":"2024-04-16T17:06:45.442815+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:34:08.618002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.557582+0000","last_change":"2024-04-16T17:03:29.255538+0000","last_active":"2024-04-16T17:06:45.557582+0000","last_peered":"2024-04-16T17:06:45.557582+0000","last_clean":"2024-04-16T17:06:45.557582+0000","last_became_active":"2024-04-16T17:03:29.255107+0000","last_became_peered":"2024-04-16T17:03:29.255107+0000","last_unstale":"2024-04-16T17:06:45.557582+0000","last_undegraded":"2024-04-16T17:06:45.557582+0000","last_fullsized":"2024-04-16T17:06:45.557582+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:31:58.458641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"90'1","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.239669+0000","last_change":"2024-04-16T17:03:24.158816+0000","last_active":"2024-04-16T17:06:45.239669+0000","last_peered":"2024-04-16T17:06:45.239669+0000","last_clean":"2024-04-16T17:06:45.239669+0000","last_became_active":"2024-04-16T17:03:24.158518+0000","last_became_peered":"2024-04-16T17:03:24.158518+0000","last_unstale":"2024-04-16T17:06:45.239669+0000","last_undegraded":"2024-04-16T17:06:45.239669+0000","last_fullsized":"2024-04-16T17:06:45.239669+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:54:55.807975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"92'9","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.445158+0000","last_change":"2024-04-16T17:03:24.977190+0000","last_active":"2024-04-16T17:06:45.445158+0000","last_peered":"2024-04-16T17:06:45.445158+0000","last_clean":"2024-04-16T17:06:45.445158+0000","last_became_active":"2024-04-16T17:03:24.976840+0000","last_became_peered":"2024-04-16T17:03:24.976840+0000","last_unstale":"2024-04-16T17:06:45.445158+0000","last_undegraded":"2024-04-16T17:06:45.445158+0000","last_fullsized":"2024-04-16T17:06:45.445158+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:21:51.294150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.239834+0000","last_change":"2024-04-16T17:03:29.301423+0000","last_active":"2024-04-16T17:06:45.239834+0000","last_peered":"2024-04-16T17:06:45.239834+0000","last_clean":"2024-04-16T17:06:45.239834+0000","last_became_active":"2024-04-16T17:03:29.300888+0000","last_became_peered":"2024-04-16T17:03:29.300888+0000","last_unstale":"2024-04-16T17:06:45.239834+0000","last_undegraded":"2024-04-16T17:06:45.239834+0000","last_fullsized":"2024-04-16T17:06:45.239834+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:13:54.188090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"95'46","reported_seq":103,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.390333+0000","last_change":"2024-04-16T17:03:24.167643+0000","last_active":"2024-04-16T17:06:45.390333+0000","last_peered":"2024-04-16T17:06:45.390333+0000","last_clean":"2024-04-16T17:06:45.390333+0000","last_became_active":"2024-04-16T17:03:24.167103+0000","last_became_peered":"2024-04-16T17:03:24.167103+0000","last_unstale":"2024-04-16T17:06:45.390333+0000","last_undegraded":"2024-04-16T17:06:45.390333+0000","last_fullsized":"2024-04-16T17:06:45.390333+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:33:16.809371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":82,"num_write_kb":837,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"92'5","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.241641+0000","last_change":"2024-04-16T17:03:24.976981+0000","last_active":"2024-04-16T17:06:45.241641+0000","last_peered":"2024-04-16T17:06:45.241641+0000","last_clean":"2024-04-16T17:06:45.241641+0000","last_became_active":"2024-04-16T17:03:24.975385+0000","last_became_peered":"2024-04-16T17:03:24.975385+0000","last_unstale":"2024-04-16T17:06:45.241641+0000","last_undegraded":"2024-04-16T17:06:45.241641+0000","last_fullsized":"2024-04-16T17:06:45.241641+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:27:03.104838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.105056+0000","last_change":"2024-04-16T17:03:29.244767+0000","last_active":"2024-04-16T17:06:45.105056+0000","last_peered":"2024-04-16T17:06:45.105056+0000","last_clean":"2024-04-16T17:06:45.105056+0000","last_became_active":"2024-04-16T17:03:29.244205+0000","last_became_peered":"2024-04-16T17:03:29.244205+0000","last_unstale":"2024-04-16T17:06:45.105056+0000","last_undegraded":"2024-04-16T17:06:45.105056+0000","last_fullsized":"2024-04-16T17:06:45.105056+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:51:46.652387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"90'4","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.105248+0000","last_change":"2024-04-16T17:03:24.164916+0000","last_active":"2024-04-16T17:06:45.105248+0000","last_peered":"2024-04-16T17:06:45.105248+0000","last_clean":"2024-04-16T17:06:45.105248+0000","last_became_active":"2024-04-16T17:03:24.164648+0000","last_became_peered":"2024-04-16T17:03:24.164648+0000","last_unstale":"2024-04-16T17:06:45.105248+0000","last_undegraded":"2024-04-16T17:06:45.105248+0000","last_fullsized":"2024-04-16T17:06:45.105248+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:23:00.198785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"92'7","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.577426+0000","last_change":"2024-04-16T17:03:24.994393+0000","last_active":"2024-04-16T17:06:45.577426+0000","last_peered":"2024-04-16T17:06:45.577426+0000","last_clean":"2024-04-16T17:06:45.577426+0000","last_became_active":"2024-04-16T17:03:24.994002+0000","last_became_peered":"2024-04-16T17:03:24.994002+0000","last_unstale":"2024-04-16T17:06:45.577426+0000","last_undegraded":"2024-04-16T17:06:45.577426+0000","last_fullsized":"2024-04-16T17:06:45.577426+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:23:44.596781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.104795+0000","last_change":"2024-04-16T17:03:29.302009+0000","last_active":"2024-04-16T17:06:45.104795+0000","last_peered":"2024-04-16T17:06:45.104795+0000","last_clean":"2024-04-16T17:06:45.104795+0000","last_became_active":"2024-04-16T17:03:29.301183+0000","last_became_peered":"2024-04-16T17:03:29.301183+0000","last_unstale":"2024-04-16T17:06:45.104795+0000","last_undegraded":"2024-04-16T17:06:45.104795+0000","last_fullsized":"2024-04-16T17:06:45.104795+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:30:19.561875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"90'2","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.450559+0000","last_change":"2024-04-16T17:03:24.176829+0000","last_active":"2024-04-16T17:06:45.450559+0000","last_peered":"2024-04-16T17:06:45.450559+0000","last_clean":"2024-04-16T17:06:45.450559+0000","last_became_active":"2024-04-16T17:03:24.176555+0000","last_became_peered":"2024-04-16T17:03:24.176555+0000","last_unstale":"2024-04-16T17:06:45.450559+0000","last_undegraded":"2024-04-16T17:06:45.450559+0000","last_fullsized":"2024-04-16T17:06:45.450559+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:05:40.602150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"92'2","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.239184+0000","last_change":"2024-04-16T17:03:24.965746+0000","last_active":"2024-04-16T17:06:45.239184+0000","last_peered":"2024-04-16T17:06:45.239184+0000","last_clean":"2024-04-16T17:06:45.239184+0000","last_became_active":"2024-04-16T17:03:24.965241+0000","last_became_peered":"2024-04-16T17:03:24.965241+0000","last_unstale":"2024-04-16T17:06:45.239184+0000","last_undegraded":"2024-04-16T17:06:45.239184+0000","last_fullsized":"2024-04-16T17:06:45.239184+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:51:34.407565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.102475+0000","last_change":"2024-04-16T17:03:29.261441+0000","last_active":"2024-04-16T17:06:45.102475+0000","last_peered":"2024-04-16T17:06:45.102475+0000","last_clean":"2024-04-16T17:06:45.102475+0000","last_became_active":"2024-04-16T17:03:29.259538+0000","last_became_peered":"2024-04-16T17:03:29.259538+0000","last_unstale":"2024-04-16T17:06:45.102475+0000","last_undegraded":"2024-04-16T17:06:45.102475+0000","last_fullsized":"2024-04-16T17:06:45.102475+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:12:27.092053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"90'3","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.240172+0000","last_change":"2024-04-16T17:03:24.172053+0000","last_active":"2024-04-16T17:06:45.240172+0000","last_peered":"2024-04-16T17:06:45.240172+0000","last_clean":"2024-04-16T17:06:45.240172+0000","last_became_active":"2024-04-16T17:03:24.171709+0000","last_became_peered":"2024-04-16T17:03:24.171709+0000","last_unstale":"2024-04-16T17:06:45.240172+0000","last_undegraded":"2024-04-16T17:06:45.240172+0000","last_fullsized":"2024-04-16T17:06:45.240172+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:40:12.307756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"92'9","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.557252+0000","last_change":"2024-04-16T17:03:24.984267+0000","last_active":"2024-04-16T17:06:45.557252+0000","last_peered":"2024-04-16T17:06:45.557252+0000","last_clean":"2024-04-16T17:06:45.557252+0000","last_became_active":"2024-04-16T17:03:24.983888+0000","last_became_peered":"2024-04-16T17:03:24.983888+0000","last_unstale":"2024-04-16T17:06:45.557252+0000","last_undegraded":"2024-04-16T17:06:45.557252+0000","last_fullsized":"2024-04-16T17:06:45.557252+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:16:23.814917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"90'1","reported_seq":26,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.105091+0000","last_change":"2024-04-16T17:03:29.251716+0000","last_active":"2024-04-16T17:06:45.105091+0000","last_peered":"2024-04-16T17:06:45.105091+0000","last_clean":"2024-04-16T17:06:45.105091+0000","last_became_active":"2024-04-16T17:03:29.251317+0000","last_became_peered":"2024-04-16T17:03:29.251317+0000","last_unstale":"2024-04-16T17:06:45.105091+0000","last_undegraded":"2024-04-16T17:06:45.105091+0000","last_fullsized":"2024-04-16T17:06:45.105091+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:48:54.680195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"90'1","reported_seq":22,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.557164+0000","last_change":"2024-04-16T17:04:21.515306+0000","last_active":"2024-04-16T17:06:45.557164+0000","last_peered":"2024-04-16T17:06:45.557164+0000","last_clean":"2024-04-16T17:06:45.557164+0000","last_became_active":"2024-04-16T17:04:21.508377+0000","last_became_peered":"2024-04-16T17:04:21.508377+0000","last_unstale":"2024-04-16T17:06:45.557164+0000","last_undegraded":"2024-04-16T17:06:45.557164+0000","last_fullsized":"2024-04-16T17:06:45.557164+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:28:25.000406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"92'8","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.240125+0000","last_change":"2024-04-16T17:03:24.995547+0000","last_active":"2024-04-16T17:06:45.240125+0000","last_peered":"2024-04-16T17:06:45.240125+0000","last_clean":"2024-04-16T17:06:45.240125+0000","last_became_active":"2024-04-16T17:03:24.995096+0000","last_became_peered":"2024-04-16T17:03:24.995096+0000","last_unstale":"2024-04-16T17:06:45.240125+0000","last_undegraded":"2024-04-16T17:06:45.240125+0000","last_fullsized":"2024-04-16T17:06:45.240125+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:54:51.193117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.104926+0000","last_change":"2024-04-16T17:03:29.301526+0000","last_active":"2024-04-16T17:06:45.104926+0000","last_peered":"2024-04-16T17:06:45.104926+0000","last_clean":"2024-04-16T17:06:45.104926+0000","last_became_active":"2024-04-16T17:03:29.300808+0000","last_became_peered":"2024-04-16T17:03:29.300808+0000","last_unstale":"2024-04-16T17:06:45.104926+0000","last_undegraded":"2024-04-16T17:06:45.104926+0000","last_fullsized":"2024-04-16T17:06:45.104926+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:01:14.717548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"90'4","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.239357+0000","last_change":"2024-04-16T17:03:24.152645+0000","last_active":"2024-04-16T17:06:45.239357+0000","last_peered":"2024-04-16T17:06:45.239357+0000","last_clean":"2024-04-16T17:06:45.239357+0000","last_became_active":"2024-04-16T17:03:24.152281+0000","last_became_peered":"2024-04-16T17:03:24.152281+0000","last_unstale":"2024-04-16T17:06:45.239357+0000","last_undegraded":"2024-04-16T17:06:45.239357+0000","last_fullsized":"2024-04-16T17:06:45.239357+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:33:51.377421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"92'5","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.443067+0000","last_change":"2024-04-16T17:03:24.979827+0000","last_active":"2024-04-16T17:06:45.443067+0000","last_peered":"2024-04-16T17:06:45.443067+0000","last_clean":"2024-04-16T17:06:45.443067+0000","last_became_active":"2024-04-16T17:03:24.979452+0000","last_became_peered":"2024-04-16T17:03:24.979452+0000","last_unstale":"2024-04-16T17:06:45.443067+0000","last_undegraded":"2024-04-16T17:06:45.443067+0000","last_fullsized":"2024-04-16T17:06:45.443067+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:07:09.259366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.450802+0000","last_change":"2024-04-16T17:03:29.255519+0000","last_active":"2024-04-16T17:06:45.450802+0000","last_peered":"2024-04-16T17:06:45.450802+0000","last_clean":"2024-04-16T17:06:45.450802+0000","last_became_active":"2024-04-16T17:03:29.255104+0000","last_became_peered":"2024-04-16T17:03:29.255104+0000","last_unstale":"2024-04-16T17:06:45.450802+0000","last_undegraded":"2024-04-16T17:06:45.450802+0000","last_fullsized":"2024-04-16T17:06:45.450802+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:27:34.496199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"90'3","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.105533+0000","last_change":"2024-04-16T17:03:24.161145+0000","last_active":"2024-04-16T17:06:45.105533+0000","last_peered":"2024-04-16T17:06:45.105533+0000","last_clean":"2024-04-16T17:06:45.105533+0000","last_became_active":"2024-04-16T17:03:24.160824+0000","last_became_peered":"2024-04-16T17:03:24.160824+0000","last_unstale":"2024-04-16T17:06:45.105533+0000","last_undegraded":"2024-04-16T17:06:45.105533+0000","last_fullsized":"2024-04-16T17:06:45.105533+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:02:48.805638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"92'7","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.443517+0000","last_change":"2024-04-16T17:03:24.966513+0000","last_active":"2024-04-16T17:06:45.443517+0000","last_peered":"2024-04-16T17:06:45.443517+0000","last_clean":"2024-04-16T17:06:45.443517+0000","last_became_active":"2024-04-16T17:03:24.966074+0000","last_became_peered":"2024-04-16T17:03:24.966074+0000","last_unstale":"2024-04-16T17:06:45.443517+0000","last_undegraded":"2024-04-16T17:06:45.443517+0000","last_fullsized":"2024-04-16T17:06:45.443517+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:50:21.579901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":32,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.576687+0000","last_change":"2024-04-16T17:04:22.150786+0000","last_active":"2024-04-16T17:06:45.576687+0000","last_peered":"2024-04-16T17:06:45.576687+0000","last_clean":"2024-04-16T17:06:45.576687+0000","last_became_active":"2024-04-16T17:04:22.149344+0000","last_became_peered":"2024-04-16T17:04:22.149344+0000","last_unstale":"2024-04-16T17:06:45.576687+0000","last_undegraded":"2024-04-16T17:06:45.576687+0000","last_fullsized":"2024-04-16T17:06:45.576687+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:18:49.421095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,1,4],"acting":[0,8,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"95'39","reported_seq":81,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.386141+0000","last_change":"2024-04-16T17:03:24.158032+0000","last_active":"2024-04-16T17:06:45.386141+0000","last_peered":"2024-04-16T17:06:45.386141+0000","last_clean":"2024-04-16T17:06:45.386141+0000","last_became_active":"2024-04-16T17:03:24.157689+0000","last_became_peered":"2024-04-16T17:03:24.157689+0000","last_unstale":"2024-04-16T17:06:45.386141+0000","last_undegraded":"2024-04-16T17:06:45.386141+0000","last_fullsized":"2024-04-16T17:06:45.386141+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:38:18.240889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"92'6","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.443058+0000","last_change":"2024-04-16T17:04:21.519999+0000","last_active":"2024-04-16T17:06:45.443058+0000","last_peered":"2024-04-16T17:06:45.443058+0000","last_clean":"2024-04-16T17:06:45.443058+0000","last_became_active":"2024-04-16T17:04:21.519621+0000","last_became_peered":"2024-04-16T17:04:21.519621+0000","last_unstale":"2024-04-16T17:06:45.443058+0000","last_undegraded":"2024-04-16T17:06:45.443058+0000","last_fullsized":"2024-04-16T17:06:45.443058+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:50:13.266369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.250659+0000","last_change":"2024-04-16T17:03:29.245176+0000","last_active":"2024-04-16T17:06:45.250659+0000","last_peered":"2024-04-16T17:06:45.250659+0000","last_clean":"2024-04-16T17:06:45.250659+0000","last_became_active":"2024-04-16T17:03:29.244763+0000","last_became_peered":"2024-04-16T17:03:29.244763+0000","last_unstale":"2024-04-16T17:06:45.250659+0000","last_undegraded":"2024-04-16T17:06:45.250659+0000","last_fullsized":"2024-04-16T17:06:45.250659+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:52:30.381417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.445799+0000","last_change":"2024-04-16T17:03:24.168068+0000","last_active":"2024-04-16T17:06:45.445799+0000","last_peered":"2024-04-16T17:06:45.445799+0000","last_clean":"2024-04-16T17:06:45.445799+0000","last_became_active":"2024-04-16T17:03:24.167694+0000","last_became_peered":"2024-04-16T17:03:24.167694+0000","last_unstale":"2024-04-16T17:06:45.445799+0000","last_undegraded":"2024-04-16T17:06:45.445799+0000","last_fullsized":"2024-04-16T17:06:45.445799+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:13:35.692306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"92'9","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.443511+0000","last_change":"2024-04-16T17:03:24.966145+0000","last_active":"2024-04-16T17:06:45.443511+0000","last_peered":"2024-04-16T17:06:45.443511+0000","last_clean":"2024-04-16T17:06:45.443511+0000","last_became_active":"2024-04-16T17:03:24.965813+0000","last_became_peered":"2024-04-16T17:03:24.965813+0000","last_unstale":"2024-04-16T17:06:45.443511+0000","last_undegraded":"2024-04-16T17:06:45.443511+0000","last_fullsized":"2024-04-16T17:06:45.443511+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:43:43.224130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"92'9","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.557305+0000","last_change":"2024-04-16T17:03:24.970941+0000","last_active":"2024-04-16T17:06:45.557305+0000","last_peered":"2024-04-16T17:06:45.557305+0000","last_clean":"2024-04-16T17:06:45.557305+0000","last_became_active":"2024-04-16T17:03:24.970587+0000","last_became_peered":"2024-04-16T17:03:24.970587+0000","last_unstale":"2024-04-16T17:06:45.557305+0000","last_undegraded":"2024-04-16T17:06:45.557305+0000","last_fullsized":"2024-04-16T17:06:45.557305+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:14:13.930011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"92'12","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.105601+0000","last_change":"2024-04-16T17:03:24.967167+0000","last_active":"2024-04-16T17:06:45.105601+0000","last_peered":"2024-04-16T17:06:45.105601+0000","last_clean":"2024-04-16T17:06:45.105601+0000","last_became_active":"2024-04-16T17:03:24.966787+0000","last_became_peered":"2024-04-16T17:03:24.966787+0000","last_unstale":"2024-04-16T17:06:45.105601+0000","last_undegraded":"2024-04-16T17:06:45.105601+0000","last_fullsized":"2024-04-16T17:06:45.105601+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:51:39.510220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"92'8","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.105196+0000","last_change":"2024-04-16T17:03:24.981115+0000","last_active":"2024-04-16T17:06:45.105196+0000","last_peered":"2024-04-16T17:06:45.105196+0000","last_clean":"2024-04-16T17:06:45.105196+0000","last_became_active":"2024-04-16T17:03:24.980685+0000","last_became_peered":"2024-04-16T17:03:24.980685+0000","last_unstale":"2024-04-16T17:06:45.105196+0000","last_undegraded":"2024-04-16T17:06:45.105196+0000","last_fullsized":"2024-04-16T17:06:45.105196+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:20:44.066932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"92'7","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.576960+0000","last_change":"2024-04-16T17:03:24.990421+0000","last_active":"2024-04-16T17:06:45.576960+0000","last_peered":"2024-04-16T17:06:45.576960+0000","last_clean":"2024-04-16T17:06:45.576960+0000","last_became_active":"2024-04-16T17:03:24.989940+0000","last_became_peered":"2024-04-16T17:03:24.989940+0000","last_unstale":"2024-04-16T17:06:45.576960+0000","last_undegraded":"2024-04-16T17:06:45.576960+0000","last_fullsized":"2024-04-16T17:06:45.576960+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:19:57.548067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"92'4","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.445217+0000","last_change":"2024-04-16T17:03:24.991854+0000","last_active":"2024-04-16T17:06:45.445217+0000","last_peered":"2024-04-16T17:06:45.445217+0000","last_clean":"2024-04-16T17:06:45.445217+0000","last_became_active":"2024-04-16T17:03:24.990964+0000","last_became_peered":"2024-04-16T17:03:24.990964+0000","last_unstale":"2024-04-16T17:06:45.445217+0000","last_undegraded":"2024-04-16T17:06:45.445217+0000","last_fullsized":"2024-04-16T17:06:45.445217+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:10:10.618708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"92'6","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.557167+0000","last_change":"2024-04-16T17:04:21.517466+0000","last_active":"2024-04-16T17:06:45.557167+0000","last_peered":"2024-04-16T17:06:45.557167+0000","last_clean":"2024-04-16T17:06:45.557167+0000","last_became_active":"2024-04-16T17:04:21.517086+0000","last_became_peered":"2024-04-16T17:04:21.517086+0000","last_unstale":"2024-04-16T17:06:45.557167+0000","last_undegraded":"2024-04-16T17:06:45.557167+0000","last_fullsized":"2024-04-16T17:06:45.557167+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:54:33.417452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,9],"acting":[11,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"92'15","reported_seq":47,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.105712+0000","last_change":"2024-04-16T17:03:24.970377+0000","last_active":"2024-04-16T17:06:45.105712+0000","last_peered":"2024-04-16T17:06:45.105712+0000","last_clean":"2024-04-16T17:06:45.105712+0000","last_became_active":"2024-04-16T17:03:24.968856+0000","last_became_peered":"2024-04-16T17:03:24.968856+0000","last_unstale":"2024-04-16T17:06:45.105712+0000","last_undegraded":"2024-04-16T17:06:45.105712+0000","last_fullsized":"2024-04-16T17:06:45.105712+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:47:28.543120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"92'2","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.241849+0000","last_change":"2024-04-16T17:03:24.992877+0000","last_active":"2024-04-16T17:06:45.241849+0000","last_peered":"2024-04-16T17:06:45.241849+0000","last_clean":"2024-04-16T17:06:45.241849+0000","last_became_active":"2024-04-16T17:03:24.992484+0000","last_became_peered":"2024-04-16T17:03:24.992484+0000","last_unstale":"2024-04-16T17:06:45.241849+0000","last_undegraded":"2024-04-16T17:06:45.241849+0000","last_fullsized":"2024-04-16T17:06:45.241849+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:00:45.204782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"92'6","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.250911+0000","last_change":"2024-04-16T17:03:24.977794+0000","last_active":"2024-04-16T17:06:45.250911+0000","last_peered":"2024-04-16T17:06:45.250911+0000","last_clean":"2024-04-16T17:06:45.250911+0000","last_became_active":"2024-04-16T17:03:24.977434+0000","last_became_peered":"2024-04-16T17:03:24.977434+0000","last_unstale":"2024-04-16T17:06:45.250911+0000","last_undegraded":"2024-04-16T17:06:45.250911+0000","last_fullsized":"2024-04-16T17:06:45.250911+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:46:35.670357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"92'8","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.240206+0000","last_change":"2024-04-16T17:03:24.992225+0000","last_active":"2024-04-16T17:06:45.240206+0000","last_peered":"2024-04-16T17:06:45.240206+0000","last_clean":"2024-04-16T17:06:45.240206+0000","last_became_active":"2024-04-16T17:03:24.991904+0000","last_became_peered":"2024-04-16T17:03:24.991904+0000","last_unstale":"2024-04-16T17:06:45.240206+0000","last_undegraded":"2024-04-16T17:06:45.240206+0000","last_fullsized":"2024-04-16T17:06:45.240206+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:01:09.710698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"92'10","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.450979+0000","last_change":"2024-04-16T17:03:24.981322+0000","last_active":"2024-04-16T17:06:45.450979+0000","last_peered":"2024-04-16T17:06:45.450979+0000","last_clean":"2024-04-16T17:06:45.450979+0000","last_became_active":"2024-04-16T17:03:24.980920+0000","last_became_peered":"2024-04-16T17:03:24.980920+0000","last_unstale":"2024-04-16T17:06:45.450979+0000","last_undegraded":"2024-04-16T17:06:45.450979+0000","last_fullsized":"2024-04-16T17:06:45.450979+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:07:59.378512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"92'11","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.443916+0000","last_change":"2024-04-16T17:03:24.990117+0000","last_active":"2024-04-16T17:06:45.443916+0000","last_peered":"2024-04-16T17:06:45.443916+0000","last_clean":"2024-04-16T17:06:45.443916+0000","last_became_active":"2024-04-16T17:03:24.989734+0000","last_became_peered":"2024-04-16T17:03:24.989734+0000","last_unstale":"2024-04-16T17:06:45.443916+0000","last_undegraded":"2024-04-16T17:06:45.443916+0000","last_fullsized":"2024-04-16T17:06:45.443916+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:36:55.000702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"92'8","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.557602+0000","last_change":"2024-04-16T17:03:24.970032+0000","last_active":"2024-04-16T17:06:45.557602+0000","last_peered":"2024-04-16T17:06:45.557602+0000","last_clean":"2024-04-16T17:06:45.557602+0000","last_became_active":"2024-04-16T17:03:24.968608+0000","last_became_peered":"2024-04-16T17:03:24.968608+0000","last_unstale":"2024-04-16T17:06:45.557602+0000","last_undegraded":"2024-04-16T17:06:45.557602+0000","last_fullsized":"2024-04-16T17:06:45.557602+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:24:03.918418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":1771104,"data_compressed":108294,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":599984,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":1167,"num_read_kb":697,"num_write":298,"num_write_kb":1448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":616,"num_keys_recovered":20,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":1799952,"data_compressed":80793,"data_compressed_allocated":110592,"data_compressed_original":1572864,"omap_allocated":0,"internal_metadata":0},"log_size":236,"ondisk_log_size":236,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351029,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1032192,"data_stored":522666,"data_compressed":26684,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612668,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32360,"kb_used_data":1184,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712792,"statfs":{"total":95995035648,"available":95961899008,"internally_reserved":0,"allocated":1212416,"data_stored":1326097,"data_compressed":73579,"data_compressed_allocated":98304,"data_compressed_original":802816,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776192,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32236,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712916,"statfs":{"total":95995035648,"available":95962025984,"internally_reserved":0,"allocated":1081344,"data_stored":925950,"data_compressed":48842,"data_compressed_allocated":65536,"data_compressed_original":475136,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037831,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32232,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712920,"statfs":{"total":95995035648,"available":95962030080,"internally_reserved":0,"allocated":1077248,"data_stored":925288,"data_compressed":48842,"data_compressed_allocated":65536,"data_compressed_original":475136,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299469,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36420,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708732,"statfs":{"total":95995035648,"available":95957741568,"internally_reserved":0,"allocated":1105920,"data_stored":1130719,"data_compressed":65794,"data_compressed_allocated":86016,"data_compressed_original":671744,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462994,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32192,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712960,"statfs":{"total":95995035648,"available":95962071040,"internally_reserved":0,"allocated":1036288,"data_stored":527910,"data_compressed":29696,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574959,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32264,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712888,"statfs":{"total":95995035648,"available":95961997312,"internally_reserved":0,"allocated":1110016,"data_stored":597413,"data_compressed":31325,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738483,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32208,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712944,"statfs":{"total":95995035648,"available":95962054656,"internally_reserved":0,"allocated":1048576,"data_stored":519432,"data_compressed":26684,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411433,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28120,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717032,"statfs":{"total":95995035648,"available":95966240768,"internally_reserved":0,"allocated":1126400,"data_stored":599070,"data_compressed":31325,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149795,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32244,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712908,"statfs":{"total":95995035648,"available":95962017792,"internally_reserved":0,"allocated":1093632,"data_stored":1052920,"data_compressed":54998,"data_compressed_allocated":73728,"data_compressed_original":598016,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888157,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1048576,"data_stored":534184,"data_compressed":29696,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724633,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28124,"kb_used_data":1108,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717028,"statfs":{"total":95995035648,"available":95966236672,"internally_reserved":0,"allocated":1134592,"data_stored":1256283,"data_compressed":71950,"data_compressed_allocated":94208,"data_compressed_original":794624,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36098,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36098,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36098,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":69593,"data_compressed":1629,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":71250,"data_compressed":1629,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":525100,"data_compressed":25302,"data_compressed_allocated":32768,"data_compressed_original":516096,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":2268,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":133999,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":4339,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":393372,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":394034,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":126976,"data_stored":203813,"data_compressed":7785,"data_compressed_allocated":12288,"data_compressed_original":131072,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-16T17:06:50.221 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-16T17:06:50.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:50.473+0000 7f4d43000640 1 -- 172.21.15.45:0/53336226 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4d3c102a90 msgr2=0x7f4d3c1093b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:50.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:50.473+0000 7f4d43000640 1 --2- 172.21.15.45:0/53336226 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4d3c102a90 0x7f4d3c1093b0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f4d34009900 tx=0x7f4d3402bc30 comp rx=0 tx=0).stop 2024-04-16T17:06:50.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:50.474+0000 7f4d43000640 1 -- 172.21.15.45:0/53336226 shutdown_connections 2024-04-16T17:06:50.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:50.474+0000 7f4d43000640 1 --2- 172.21.15.45:0/53336226 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4d3c1020d0 0x7f4d3c102550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:50.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:50.474+0000 7f4d43000640 1 --2- 172.21.15.45:0/53336226 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4d3c102a90 0x7f4d3c1093b0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:50.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:50.474+0000 7f4d43000640 1 --2- 172.21.15.45:0/53336226 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4d3c100f00 0x7f4d3c101340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:50.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:50.474+0000 7f4d43000640 1 -- 172.21.15.45:0/53336226 >> 172.21.15.45:0/53336226 conn(0x7f4d3c0fc4e0 msgr2=0x7f4d3c0fe900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:06:50.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:50.474+0000 7f4d43000640 1 -- 172.21.15.45:0/53336226 shutdown_connections 2024-04-16T17:06:50.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:50.474+0000 7f4d43000640 1 -- 172.21.15.45:0/53336226 wait complete. 2024-04-16T17:06:50.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:50.475+0000 7f4d43000640 1 Processor -- start 2024-04-16T17:06:50.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:50.475+0000 7f4d43000640 1 -- start start 2024-04-16T17:06:50.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:50.475+0000 7f4d43000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4d3c100f00 0x7f4d3c19a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:50.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:50.475+0000 7f4d43000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4d3c1020d0 0x7f4d3c19aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:50.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:50.476+0000 7f4d43000640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4d3c102a90 0x7f4d3c1a1ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:50.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:50.476+0000 7f4d43000640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f4d3c10cc20 con 0x7f4d3c1020d0 2024-04-16T17:06:50.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:50.476+0000 7f4d43000640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f4d3c10caa0 con 0x7f4d3c102a90 2024-04-16T17:06:50.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:50.476+0000 7f4d43000640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f4d3c10cda0 con 0x7f4d3c100f00 2024-04-16T17:06:50.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:50.476+0000 7f4d3b400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4d3c1020d0 0x7f4d3c19aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:50.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:50.476+0000 7f4d3b400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4d3c1020d0 0x7f4d3c19aa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55270/0 (socket says 172.21.15.45:55270) 2024-04-16T17:06:50.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:50.476+0000 7f4d3b400640 1 -- 172.21.15.45:0/3024790688 learned_addr learned my addr 172.21.15.45:0/3024790688 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:06:50.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:50.476+0000 7f4d3be00640 1 --2- 172.21.15.45:0/3024790688 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4d3c100f00 0x7f4d3c19a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:50.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:50.476+0000 7f4d41000640 1 --2- 172.21.15.45:0/3024790688 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4d3c102a90 0x7f4d3c1a1ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:50.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:50.476+0000 7f4d3b400640 1 -- 172.21.15.45:0/3024790688 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4d3c100f00 msgr2=0x7f4d3c19a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:50.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:50.476+0000 7f4d3b400640 1 --2- 172.21.15.45:0/3024790688 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4d3c100f00 0x7f4d3c19a510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:50.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:50.476+0000 7f4d3b400640 1 -- 172.21.15.45:0/3024790688 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4d3c102a90 msgr2=0x7f4d3c1a1ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:50.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:50.476+0000 7f4d3b400640 1 --2- 172.21.15.45:0/3024790688 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4d3c102a90 0x7f4d3c1a1ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:50.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:50.476+0000 7f4d3b400640 1 -- 172.21.15.45:0/3024790688 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d3c1a2130 con 0x7f4d3c1020d0 2024-04-16T17:06:50.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:50.477+0000 7f4d3b400640 1 --2- 172.21.15.45:0/3024790688 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4d3c1020d0 0x7f4d3c19aa50 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f4d2400e9f0 tx=0x7f4d2400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:06:50.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:50.477+0000 7f4d38c00640 1 -- 172.21.15.45:0/3024790688 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d2400cd80 con 0x7f4d3c1020d0 2024-04-16T17:06:50.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:50.477+0000 7f4d38c00640 1 -- 172.21.15.45:0/3024790688 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d24005500 con 0x7f4d3c1020d0 2024-04-16T17:06:50.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:50.477+0000 7f4d43000640 1 -- 172.21.15.45:0/3024790688 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d3c1a2420 con 0x7f4d3c1020d0 2024-04-16T17:06:50.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:50.477+0000 7f4d38c00640 1 -- 172.21.15.45:0/3024790688 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d24010690 con 0x7f4d3c1020d0 2024-04-16T17:06:50.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:50.477+0000 7f4d43000640 1 -- 172.21.15.45:0/3024790688 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d3c1a2920 con 0x7f4d3c1020d0 2024-04-16T17:06:50.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:50.479+0000 7f4d43000640 1 -- 172.21.15.45:0/3024790688 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d3c1a2be0 con 0x7f4d3c1020d0 2024-04-16T17:06:50.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:50.481+0000 7f4d38c00640 1 -- 172.21.15.45:0/3024790688 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 24) ==== 138822+0+0 (secure 0 0 0) 0x7f4d240046c0 con 0x7f4d3c1020d0 2024-04-16T17:06:50.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:50.483+0000 7f4d38c00640 1 --2- 172.21.15.45:0/3024790688 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f4d200a3db0 0x7f4d200a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:50.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:50.483+0000 7f4d38c00640 1 -- 172.21.15.45:0/3024790688 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f4d240d10e0 con 0x7f4d3c1020d0 2024-04-16T17:06:50.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:50.484+0000 7f4d3be00640 1 --2- 172.21.15.45:0/3024790688 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f4d200a3db0 0x7f4d200a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:50.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:50.484+0000 7f4d3be00640 1 --2- 172.21.15.45:0/3024790688 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f4d200a3db0 0x7f4d200a6270 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f4d3c100c20 tx=0x7f4d2c009720 comp rx=0 tx=0).ready entity=mgr.25411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:06:50.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:50.487+0000 7f4d38c00640 1 -- 172.21.15.45:0/3024790688 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4d2409d260 con 0x7f4d3c1020d0 2024-04-16T17:06:50.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:50.698+0000 7f4d43000640 1 -- 172.21.15.45:0/3024790688 --> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f4d3c0fe5f0 con 0x7f4d200a3db0 2024-04-16T17:06:50.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:50.712+0000 7f4d38c00640 1 -- 172.21.15.45:0/3024790688 <== mgr.25411 v2:172.21.15.45:6800/751128195 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+339706 (secure 0 0 0) 0x7f4d3c0fe5f0 con 0x7f4d200a3db0 2024-04-16T17:06:50.719 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-16T17:06:50.722 INFO:teuthology.orchestra.run.smithi045.stderr:dumped all 2024-04-16T17:06:50.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:50.718+0000 7f4d43000640 1 -- 172.21.15.45:0/3024790688 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f4d200a3db0 msgr2=0x7f4d200a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:50.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:50.718+0000 7f4d43000640 1 --2- 172.21.15.45:0/3024790688 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f4d200a3db0 0x7f4d200a6270 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f4d3c100c20 tx=0x7f4d2c009720 comp rx=0 tx=0).stop 2024-04-16T17:06:50.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:50.718+0000 7f4d43000640 1 -- 172.21.15.45:0/3024790688 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4d3c1020d0 msgr2=0x7f4d3c19aa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:50.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:50.718+0000 7f4d43000640 1 --2- 172.21.15.45:0/3024790688 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4d3c1020d0 0x7f4d3c19aa50 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f4d2400e9f0 tx=0x7f4d2400edf0 comp rx=0 tx=0).stop 2024-04-16T17:06:50.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:50.718+0000 7f4d43000640 1 -- 172.21.15.45:0/3024790688 shutdown_connections 2024-04-16T17:06:50.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:50.718+0000 7f4d43000640 1 --2- 172.21.15.45:0/3024790688 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f4d200a3db0 0x7f4d200a6270 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:50.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:50.718+0000 7f4d43000640 1 --2- 172.21.15.45:0/3024790688 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4d3c102a90 0x7f4d3c1a1ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:50.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:50.718+0000 7f4d43000640 1 --2- 172.21.15.45:0/3024790688 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4d3c1020d0 0x7f4d3c19aa50 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:50.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:50.718+0000 7f4d43000640 1 --2- 172.21.15.45:0/3024790688 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4d3c100f00 0x7f4d3c19a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:50.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:50.718+0000 7f4d43000640 1 -- 172.21.15.45:0/3024790688 >> 172.21.15.45:0/3024790688 conn(0x7f4d3c0fc4e0 msgr2=0x7f4d3c0fdee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:06:50.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:50.718+0000 7f4d43000640 1 -- 172.21.15.45:0/3024790688 shutdown_connections 2024-04-16T17:06:50.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:50.718+0000 7f4d43000640 1 -- 172.21.15.45:0/3024790688 wait complete. 2024-04-16T17:06:50.737 INFO:teuthology.orchestra.run.smithi045.stdout:{"pg_ready":true,"pg_map":{"version":5,"stamp":"2024-04-16T17:06:49.124283+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1190586,"num_objects":66,"num_object_clones":0,"num_object_copies":200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":1303,"num_read_kb":816,"num_write":481,"num_write_kb":4388,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":1181469,"num_keys_recovered":20,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":875,"ondisk_log_size":875,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":374476,"kb_used_data":12800,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124567348,"statfs":{"total":1151940427776,"available":1151556964352,"internally_reserved":0,"allocated":13107200,"data_stored":9917932,"data_compressed":539415,"data_compressed_allocated":729088,"data_compressed_original":4308992,"omap_allocated":19143,"internal_metadata":370193721},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.986206"},"pg_stats":[{"pgid":"3.3f","version":"92'8","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.557723+0000","last_change":"2024-04-16T17:03:24.989508+0000","last_active":"2024-04-16T17:06:45.557723+0000","last_peered":"2024-04-16T17:06:45.557723+0000","last_clean":"2024-04-16T17:06:45.557723+0000","last_became_active":"2024-04-16T17:03:24.989178+0000","last_became_peered":"2024-04-16T17:03:24.989178+0000","last_unstale":"2024-04-16T17:06:45.557723+0000","last_undegraded":"2024-04-16T17:06:45.557723+0000","last_fullsized":"2024-04-16T17:06:45.557723+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:18:56.488358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"92'4","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.577296+0000","last_change":"2024-04-16T17:03:24.986378+0000","last_active":"2024-04-16T17:06:45.577296+0000","last_peered":"2024-04-16T17:06:45.577296+0000","last_clean":"2024-04-16T17:06:45.577296+0000","last_became_active":"2024-04-16T17:03:24.986030+0000","last_became_peered":"2024-04-16T17:03:24.986030+0000","last_unstale":"2024-04-16T17:06:45.577296+0000","last_undegraded":"2024-04-16T17:06:45.577296+0000","last_fullsized":"2024-04-16T17:06:45.577296+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:21:14.071109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"92'7","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.577316+0000","last_change":"2024-04-16T17:03:24.976232+0000","last_active":"2024-04-16T17:06:45.577316+0000","last_peered":"2024-04-16T17:06:45.577316+0000","last_clean":"2024-04-16T17:06:45.577316+0000","last_became_active":"2024-04-16T17:03:24.975873+0000","last_became_peered":"2024-04-16T17:03:24.975873+0000","last_unstale":"2024-04-16T17:06:45.577316+0000","last_undegraded":"2024-04-16T17:06:45.577316+0000","last_fullsized":"2024-04-16T17:06:45.577316+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:07:53.523101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"92'8","reported_seq":26,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.102010+0000","last_change":"2024-04-16T17:04:21.517830+0000","last_active":"2024-04-16T17:06:45.102010+0000","last_peered":"2024-04-16T17:06:45.102010+0000","last_clean":"2024-04-16T17:06:45.102010+0000","last_became_active":"2024-04-16T17:04:21.516749+0000","last_became_peered":"2024-04-16T17:04:21.516749+0000","last_unstale":"2024-04-16T17:06:45.102010+0000","last_undegraded":"2024-04-16T17:06:45.102010+0000","last_fullsized":"2024-04-16T17:06:45.102010+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:02:33.344034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"92'4","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.239444+0000","last_change":"2024-04-16T17:03:24.985609+0000","last_active":"2024-04-16T17:06:45.239444+0000","last_peered":"2024-04-16T17:06:45.239444+0000","last_clean":"2024-04-16T17:06:45.239444+0000","last_became_active":"2024-04-16T17:03:24.985239+0000","last_became_peered":"2024-04-16T17:03:24.985239+0000","last_unstale":"2024-04-16T17:06:45.239444+0000","last_undegraded":"2024-04-16T17:06:45.239444+0000","last_fullsized":"2024-04-16T17:06:45.239444+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:55:58.619210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"92'6","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.227011+0000","last_change":"2024-04-16T17:03:24.966862+0000","last_active":"2024-04-16T17:06:45.227011+0000","last_peered":"2024-04-16T17:06:45.227011+0000","last_clean":"2024-04-16T17:06:45.227011+0000","last_became_active":"2024-04-16T17:03:24.966234+0000","last_became_peered":"2024-04-16T17:03:24.966234+0000","last_unstale":"2024-04-16T17:06:45.227011+0000","last_undegraded":"2024-04-16T17:06:45.227011+0000","last_fullsized":"2024-04-16T17:06:45.227011+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:21:07.018281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"92'5","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.442957+0000","last_change":"2024-04-16T17:03:24.971219+0000","last_active":"2024-04-16T17:06:45.442957+0000","last_peered":"2024-04-16T17:06:45.442957+0000","last_clean":"2024-04-16T17:06:45.442957+0000","last_became_active":"2024-04-16T17:03:24.970876+0000","last_became_peered":"2024-04-16T17:03:24.970876+0000","last_unstale":"2024-04-16T17:06:45.442957+0000","last_undegraded":"2024-04-16T17:06:45.442957+0000","last_fullsized":"2024-04-16T17:06:45.442957+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:44:16.311167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"92'9","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.239719+0000","last_change":"2024-04-16T17:03:24.994694+0000","last_active":"2024-04-16T17:06:45.239719+0000","last_peered":"2024-04-16T17:06:45.239719+0000","last_clean":"2024-04-16T17:06:45.239719+0000","last_became_active":"2024-04-16T17:03:24.994315+0000","last_became_peered":"2024-04-16T17:03:24.994315+0000","last_unstale":"2024-04-16T17:06:45.239719+0000","last_undegraded":"2024-04-16T17:06:45.239719+0000","last_fullsized":"2024-04-16T17:06:45.239719+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:04:59.543816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"92'11","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.451196+0000","last_change":"2024-04-16T17:03:24.964654+0000","last_active":"2024-04-16T17:06:45.451196+0000","last_peered":"2024-04-16T17:06:45.451196+0000","last_clean":"2024-04-16T17:06:45.451196+0000","last_became_active":"2024-04-16T17:03:24.964349+0000","last_became_peered":"2024-04-16T17:03:24.964349+0000","last_unstale":"2024-04-16T17:06:45.451196+0000","last_undegraded":"2024-04-16T17:06:45.451196+0000","last_fullsized":"2024-04-16T17:06:45.451196+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:38:55.880948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"92'16","reported_seq":48,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.226821+0000","last_change":"2024-04-16T17:03:24.966364+0000","last_active":"2024-04-16T17:06:45.226821+0000","last_peered":"2024-04-16T17:06:45.226821+0000","last_clean":"2024-04-16T17:06:45.226821+0000","last_became_active":"2024-04-16T17:03:24.966061+0000","last_became_peered":"2024-04-16T17:03:24.966061+0000","last_unstale":"2024-04-16T17:06:45.226821+0000","last_undegraded":"2024-04-16T17:06:45.226821+0000","last_fullsized":"2024-04-16T17:06:45.226821+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:18:12.642602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"92'6","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.443443+0000","last_change":"2024-04-16T17:03:24.969874+0000","last_active":"2024-04-16T17:06:45.443443+0000","last_peered":"2024-04-16T17:06:45.443443+0000","last_clean":"2024-04-16T17:06:45.443443+0000","last_became_active":"2024-04-16T17:03:24.969523+0000","last_became_peered":"2024-04-16T17:03:24.969523+0000","last_unstale":"2024-04-16T17:06:45.443443+0000","last_undegraded":"2024-04-16T17:06:45.443443+0000","last_fullsized":"2024-04-16T17:06:45.443443+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:17:49.704855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"92'7","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.105227+0000","last_change":"2024-04-16T17:03:24.968660+0000","last_active":"2024-04-16T17:06:45.105227+0000","last_peered":"2024-04-16T17:06:45.105227+0000","last_clean":"2024-04-16T17:06:45.105227+0000","last_became_active":"2024-04-16T17:03:24.968348+0000","last_became_peered":"2024-04-16T17:03:24.968348+0000","last_unstale":"2024-04-16T17:06:45.105227+0000","last_undegraded":"2024-04-16T17:06:45.105227+0000","last_fullsized":"2024-04-16T17:06:45.105227+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:06:06.319391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"92'10","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.444115+0000","last_change":"2024-04-16T17:03:24.974741+0000","last_active":"2024-04-16T17:06:45.444115+0000","last_peered":"2024-04-16T17:06:45.444115+0000","last_clean":"2024-04-16T17:06:45.444115+0000","last_became_active":"2024-04-16T17:03:24.973568+0000","last_became_peered":"2024-04-16T17:03:24.973568+0000","last_unstale":"2024-04-16T17:06:45.444115+0000","last_undegraded":"2024-04-16T17:06:45.444115+0000","last_fullsized":"2024-04-16T17:06:45.444115+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:30:49.325469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"92'7","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.576655+0000","last_change":"2024-04-16T17:03:24.981629+0000","last_active":"2024-04-16T17:06:45.576655+0000","last_peered":"2024-04-16T17:06:45.576655+0000","last_clean":"2024-04-16T17:06:45.576655+0000","last_became_active":"2024-04-16T17:03:24.981111+0000","last_became_peered":"2024-04-16T17:03:24.981111+0000","last_unstale":"2024-04-16T17:06:45.576655+0000","last_undegraded":"2024-04-16T17:06:45.576655+0000","last_fullsized":"2024-04-16T17:06:45.576655+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:41:29.641610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"92'7","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.576489+0000","last_change":"2024-04-16T17:03:24.977890+0000","last_active":"2024-04-16T17:06:45.576489+0000","last_peered":"2024-04-16T17:06:45.576489+0000","last_clean":"2024-04-16T17:06:45.576489+0000","last_became_active":"2024-04-16T17:03:24.976500+0000","last_became_peered":"2024-04-16T17:03:24.976500+0000","last_unstale":"2024-04-16T17:06:45.576489+0000","last_undegraded":"2024-04-16T17:06:45.576489+0000","last_fullsized":"2024-04-16T17:06:45.576489+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:17:51.604701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"92'9","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.450479+0000","last_change":"2024-04-16T17:03:24.976085+0000","last_active":"2024-04-16T17:06:45.450479+0000","last_peered":"2024-04-16T17:06:45.450479+0000","last_clean":"2024-04-16T17:06:45.450479+0000","last_became_active":"2024-04-16T17:03:24.975804+0000","last_became_peered":"2024-04-16T17:03:24.975804+0000","last_unstale":"2024-04-16T17:06:45.450479+0000","last_undegraded":"2024-04-16T17:06:45.450479+0000","last_fullsized":"2024-04-16T17:06:45.450479+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:19:13.706889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"92'8","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.557548+0000","last_change":"2024-04-16T17:03:24.989456+0000","last_active":"2024-04-16T17:06:45.557548+0000","last_peered":"2024-04-16T17:06:45.557548+0000","last_clean":"2024-04-16T17:06:45.557548+0000","last_became_active":"2024-04-16T17:03:24.989161+0000","last_became_peered":"2024-04-16T17:03:24.989161+0000","last_unstale":"2024-04-16T17:06:45.557548+0000","last_undegraded":"2024-04-16T17:06:45.557548+0000","last_fullsized":"2024-04-16T17:06:45.557548+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:52:59.333625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"92'10","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.102702+0000","last_change":"2024-04-16T17:03:24.963428+0000","last_active":"2024-04-16T17:06:45.102702+0000","last_peered":"2024-04-16T17:06:45.102702+0000","last_clean":"2024-04-16T17:06:45.102702+0000","last_became_active":"2024-04-16T17:03:24.962523+0000","last_became_peered":"2024-04-16T17:03:24.962523+0000","last_unstale":"2024-04-16T17:06:45.102702+0000","last_undegraded":"2024-04-16T17:06:45.102702+0000","last_fullsized":"2024-04-16T17:06:45.102702+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:22:19.523901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"92'7","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.105614+0000","last_change":"2024-04-16T17:03:24.981122+0000","last_active":"2024-04-16T17:06:45.105614+0000","last_peered":"2024-04-16T17:06:45.105614+0000","last_clean":"2024-04-16T17:06:45.105614+0000","last_became_active":"2024-04-16T17:03:24.980533+0000","last_became_peered":"2024-04-16T17:03:24.980533+0000","last_unstale":"2024-04-16T17:06:45.105614+0000","last_undegraded":"2024-04-16T17:06:45.105614+0000","last_fullsized":"2024-04-16T17:06:45.105614+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:51:08.580965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.105345+0000","last_change":"2024-04-16T17:03:29.257239+0000","last_active":"2024-04-16T17:06:45.105345+0000","last_peered":"2024-04-16T17:06:45.105345+0000","last_clean":"2024-04-16T17:06:45.105345+0000","last_became_active":"2024-04-16T17:03:29.256827+0000","last_became_peered":"2024-04-16T17:03:29.256827+0000","last_unstale":"2024-04-16T17:06:45.105345+0000","last_undegraded":"2024-04-16T17:06:45.105345+0000","last_fullsized":"2024-04-16T17:06:45.105345+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:57:28.373386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"90'9","reported_seq":60,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.102601+0000","last_change":"2024-04-16T17:04:21.896665+0000","last_active":"2024-04-16T17:06:45.102601+0000","last_peered":"2024-04-16T17:06:45.102601+0000","last_clean":"2024-04-16T17:06:45.102601+0000","last_became_active":"2024-04-16T17:04:21.882573+0000","last_became_peered":"2024-04-16T17:04:21.882573+0000","last_unstale":"2024-04-16T17:06:45.102601+0000","last_undegraded":"2024-04-16T17:06:45.102601+0000","last_fullsized":"2024-04-16T17:06:45.102601+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:37:07.210368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,1],"acting":[9,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"92'5","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.239912+0000","last_change":"2024-04-16T17:03:24.994921+0000","last_active":"2024-04-16T17:06:45.239912+0000","last_peered":"2024-04-16T17:06:45.239912+0000","last_clean":"2024-04-16T17:06:45.239912+0000","last_became_active":"2024-04-16T17:03:24.994412+0000","last_became_peered":"2024-04-16T17:03:24.994412+0000","last_unstale":"2024-04-16T17:06:45.239912+0000","last_undegraded":"2024-04-16T17:06:45.239912+0000","last_fullsized":"2024-04-16T17:06:45.239912+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:11:40.748198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.104669+0000","last_change":"2024-04-16T17:03:29.259884+0000","last_active":"2024-04-16T17:06:45.104669+0000","last_peered":"2024-04-16T17:06:45.104669+0000","last_clean":"2024-04-16T17:06:45.104669+0000","last_became_active":"2024-04-16T17:03:29.259319+0000","last_became_peered":"2024-04-16T17:03:29.259319+0000","last_unstale":"2024-04-16T17:06:45.104669+0000","last_undegraded":"2024-04-16T17:06:45.104669+0000","last_fullsized":"2024-04-16T17:06:45.104669+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:05:18.244682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"90'2","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.241406+0000","last_change":"2024-04-16T17:03:24.179308+0000","last_active":"2024-04-16T17:06:45.241406+0000","last_peered":"2024-04-16T17:06:45.241406+0000","last_clean":"2024-04-16T17:06:45.241406+0000","last_became_active":"2024-04-16T17:03:24.178894+0000","last_became_peered":"2024-04-16T17:03:24.178894+0000","last_unstale":"2024-04-16T17:06:45.241406+0000","last_undegraded":"2024-04-16T17:06:45.241406+0000","last_fullsized":"2024-04-16T17:06:45.241406+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:17:46.831871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"92'6","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.577690+0000","last_change":"2024-04-16T17:03:24.982391+0000","last_active":"2024-04-16T17:06:45.577690+0000","last_peered":"2024-04-16T17:06:45.577690+0000","last_clean":"2024-04-16T17:06:45.577690+0000","last_became_active":"2024-04-16T17:03:24.981761+0000","last_became_peered":"2024-04-16T17:03:24.981761+0000","last_unstale":"2024-04-16T17:06:45.577690+0000","last_undegraded":"2024-04-16T17:06:45.577690+0000","last_fullsized":"2024-04-16T17:06:45.577690+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:40:57.657170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.104794+0000","last_change":"2024-04-16T17:03:29.259705+0000","last_active":"2024-04-16T17:06:45.104794+0000","last_peered":"2024-04-16T17:06:45.104794+0000","last_clean":"2024-04-16T17:06:45.104794+0000","last_became_active":"2024-04-16T17:03:29.259336+0000","last_became_peered":"2024-04-16T17:03:29.259336+0000","last_unstale":"2024-04-16T17:06:45.104794+0000","last_undegraded":"2024-04-16T17:06:45.104794+0000","last_fullsized":"2024-04-16T17:06:45.104794+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:53:29.042933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"90'4","reported_seq":50,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.239845+0000","last_change":"2024-04-16T17:04:22.153413+0000","last_active":"2024-04-16T17:06:45.239845+0000","last_peered":"2024-04-16T17:06:45.239845+0000","last_clean":"2024-04-16T17:06:45.239845+0000","last_became_active":"2024-04-16T17:04:22.148972+0000","last_became_peered":"2024-04-16T17:04:22.148972+0000","last_unstale":"2024-04-16T17:06:45.239845+0000","last_undegraded":"2024-04-16T17:06:45.239845+0000","last_fullsized":"2024-04-16T17:06:45.239845+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:05:57.513052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"92'9","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.450605+0000","last_change":"2024-04-16T17:03:24.973205+0000","last_active":"2024-04-16T17:06:45.450605+0000","last_peered":"2024-04-16T17:06:45.450605+0000","last_clean":"2024-04-16T17:06:45.450605+0000","last_became_active":"2024-04-16T17:03:24.972863+0000","last_became_peered":"2024-04-16T17:03:24.972863+0000","last_unstale":"2024-04-16T17:06:45.450605+0000","last_undegraded":"2024-04-16T17:06:45.450605+0000","last_fullsized":"2024-04-16T17:06:45.450605+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:28:29.005783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.104818+0000","last_change":"2024-04-16T17:03:29.262930+0000","last_active":"2024-04-16T17:06:45.104818+0000","last_peered":"2024-04-16T17:06:45.104818+0000","last_clean":"2024-04-16T17:06:45.104818+0000","last_became_active":"2024-04-16T17:03:29.262621+0000","last_became_peered":"2024-04-16T17:03:29.262621+0000","last_unstale":"2024-04-16T17:06:45.104818+0000","last_undegraded":"2024-04-16T17:06:45.104818+0000","last_fullsized":"2024-04-16T17:06:45.104818+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:22:31.555143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.239656+0000","last_change":"2024-04-16T17:03:24.150503+0000","last_active":"2024-04-16T17:06:45.239656+0000","last_peered":"2024-04-16T17:06:45.239656+0000","last_clean":"2024-04-16T17:06:45.239656+0000","last_became_active":"2024-04-16T17:03:24.150021+0000","last_became_peered":"2024-04-16T17:03:24.150021+0000","last_unstale":"2024-04-16T17:06:45.239656+0000","last_undegraded":"2024-04-16T17:06:45.239656+0000","last_fullsized":"2024-04-16T17:06:45.239656+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:21:36.119497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"92'15","reported_seq":47,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.251429+0000","last_change":"2024-04-16T17:03:24.965049+0000","last_active":"2024-04-16T17:06:45.251429+0000","last_peered":"2024-04-16T17:06:45.251429+0000","last_clean":"2024-04-16T17:06:45.251429+0000","last_became_active":"2024-04-16T17:03:24.963120+0000","last_became_peered":"2024-04-16T17:03:24.963120+0000","last_unstale":"2024-04-16T17:06:45.251429+0000","last_undegraded":"2024-04-16T17:06:45.251429+0000","last_fullsized":"2024-04-16T17:06:45.251429+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:17:49.725113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.105640+0000","last_change":"2024-04-16T17:03:29.247013+0000","last_active":"2024-04-16T17:06:45.105640+0000","last_peered":"2024-04-16T17:06:45.105640+0000","last_clean":"2024-04-16T17:06:45.105640+0000","last_became_active":"2024-04-16T17:03:29.246477+0000","last_became_peered":"2024-04-16T17:03:29.246477+0000","last_unstale":"2024-04-16T17:06:45.105640+0000","last_undegraded":"2024-04-16T17:06:45.105640+0000","last_fullsized":"2024-04-16T17:06:45.105640+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:52:19.525076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"95'8","reported_seq":220,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.629994+0000","last_change":"2024-04-16T17:03:24.155333+0000","last_active":"2024-04-16T17:06:45.629994+0000","last_peered":"2024-04-16T17:06:45.629994+0000","last_clean":"2024-04-16T17:06:45.629994+0000","last_became_active":"2024-04-16T17:03:24.154848+0000","last_became_peered":"2024-04-16T17:03:24.154848+0000","last_unstale":"2024-04-16T17:06:45.629994+0000","last_undegraded":"2024-04-16T17:06:45.629994+0000","last_fullsized":"2024-04-16T17:06:45.629994+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:03:34.769573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1545,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":178,"num_read_kb":5,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"92'7","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.239446+0000","last_change":"2024-04-16T17:03:24.967627+0000","last_active":"2024-04-16T17:06:45.239446+0000","last_peered":"2024-04-16T17:06:45.239446+0000","last_clean":"2024-04-16T17:06:45.239446+0000","last_became_active":"2024-04-16T17:03:24.967264+0000","last_became_peered":"2024-04-16T17:03:24.967264+0000","last_unstale":"2024-04-16T17:06:45.239446+0000","last_undegraded":"2024-04-16T17:06:45.239446+0000","last_fullsized":"2024-04-16T17:06:45.239446+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:35:29.182374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.105252+0000","last_change":"2024-04-16T17:03:29.264625+0000","last_active":"2024-04-16T17:06:45.105252+0000","last_peered":"2024-04-16T17:06:45.105252+0000","last_clean":"2024-04-16T17:06:45.105252+0000","last_became_active":"2024-04-16T17:03:29.264114+0000","last_became_peered":"2024-04-16T17:03:29.264114+0000","last_unstale":"2024-04-16T17:06:45.105252+0000","last_undegraded":"2024-04-16T17:06:45.105252+0000","last_fullsized":"2024-04-16T17:06:45.105252+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:14:49.449980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"90'3","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.226892+0000","last_change":"2024-04-16T17:03:24.160457+0000","last_active":"2024-04-16T17:06:45.226892+0000","last_peered":"2024-04-16T17:06:45.226892+0000","last_clean":"2024-04-16T17:06:45.226892+0000","last_became_active":"2024-04-16T17:03:24.160195+0000","last_became_peered":"2024-04-16T17:03:24.160195+0000","last_unstale":"2024-04-16T17:06:45.226892+0000","last_undegraded":"2024-04-16T17:06:45.226892+0000","last_fullsized":"2024-04-16T17:06:45.226892+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:23:07.064539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"92'8","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.251287+0000","last_change":"2024-04-16T17:03:24.991249+0000","last_active":"2024-04-16T17:06:45.251287+0000","last_peered":"2024-04-16T17:06:45.251287+0000","last_clean":"2024-04-16T17:06:45.251287+0000","last_became_active":"2024-04-16T17:03:24.990927+0000","last_became_peered":"2024-04-16T17:03:24.990927+0000","last_unstale":"2024-04-16T17:06:45.251287+0000","last_undegraded":"2024-04-16T17:06:45.251287+0000","last_fullsized":"2024-04-16T17:06:45.251287+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:36:39.975616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.557483+0000","last_change":"2024-04-16T17:03:29.253704+0000","last_active":"2024-04-16T17:06:45.557483+0000","last_peered":"2024-04-16T17:06:45.557483+0000","last_clean":"2024-04-16T17:06:45.557483+0000","last_became_active":"2024-04-16T17:03:29.253334+0000","last_became_peered":"2024-04-16T17:03:29.253334+0000","last_unstale":"2024-04-16T17:06:45.557483+0000","last_undegraded":"2024-04-16T17:06:45.557483+0000","last_fullsized":"2024-04-16T17:06:45.557483+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:19:12.611731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"90'2","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.444031+0000","last_change":"2024-04-16T17:03:24.158875+0000","last_active":"2024-04-16T17:06:45.444031+0000","last_peered":"2024-04-16T17:06:45.444031+0000","last_clean":"2024-04-16T17:06:45.444031+0000","last_became_active":"2024-04-16T17:03:24.158465+0000","last_became_peered":"2024-04-16T17:03:24.158465+0000","last_unstale":"2024-04-16T17:06:45.444031+0000","last_undegraded":"2024-04-16T17:06:45.444031+0000","last_fullsized":"2024-04-16T17:06:45.444031+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:21:29.882251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"92'8","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.239904+0000","last_change":"2024-04-16T17:03:24.994476+0000","last_active":"2024-04-16T17:06:45.239904+0000","last_peered":"2024-04-16T17:06:45.239904+0000","last_clean":"2024-04-16T17:06:45.239904+0000","last_became_active":"2024-04-16T17:03:24.993532+0000","last_became_peered":"2024-04-16T17:03:24.993532+0000","last_unstale":"2024-04-16T17:06:45.239904+0000","last_undegraded":"2024-04-16T17:06:45.239904+0000","last_fullsized":"2024-04-16T17:06:45.239904+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:04:44.062444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.577184+0000","last_change":"2024-04-16T17:03:29.255446+0000","last_active":"2024-04-16T17:06:45.577184+0000","last_peered":"2024-04-16T17:06:45.577184+0000","last_clean":"2024-04-16T17:06:45.577184+0000","last_became_active":"2024-04-16T17:03:29.255051+0000","last_became_peered":"2024-04-16T17:03:29.255051+0000","last_unstale":"2024-04-16T17:06:45.577184+0000","last_undegraded":"2024-04-16T17:06:45.577184+0000","last_fullsized":"2024-04-16T17:06:45.577184+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:24:04.946342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"90'1","reported_seq":35,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.250870+0000","last_change":"2024-04-16T17:03:24.154904+0000","last_active":"2024-04-16T17:06:45.250870+0000","last_peered":"2024-04-16T17:06:45.250870+0000","last_clean":"2024-04-16T17:06:45.250870+0000","last_became_active":"2024-04-16T17:03:24.154617+0000","last_became_peered":"2024-04-16T17:03:24.154617+0000","last_unstale":"2024-04-16T17:06:45.250870+0000","last_undegraded":"2024-04-16T17:06:45.250870+0000","last_fullsized":"2024-04-16T17:06:45.250870+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:35:55.498855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"92'11","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.101961+0000","last_change":"2024-04-16T17:03:24.979908+0000","last_active":"2024-04-16T17:06:45.101961+0000","last_peered":"2024-04-16T17:06:45.101961+0000","last_clean":"2024-04-16T17:06:45.101961+0000","last_became_active":"2024-04-16T17:03:24.979501+0000","last_became_peered":"2024-04-16T17:03:24.979501+0000","last_unstale":"2024-04-16T17:06:45.101961+0000","last_undegraded":"2024-04-16T17:06:45.101961+0000","last_fullsized":"2024-04-16T17:06:45.101961+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:51:05.421459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.240099+0000","last_change":"2024-04-16T17:03:29.258809+0000","last_active":"2024-04-16T17:06:45.240099+0000","last_peered":"2024-04-16T17:06:45.240099+0000","last_clean":"2024-04-16T17:06:45.240099+0000","last_became_active":"2024-04-16T17:03:29.256629+0000","last_became_peered":"2024-04-16T17:03:29.256629+0000","last_unstale":"2024-04-16T17:06:45.240099+0000","last_undegraded":"2024-04-16T17:06:45.240099+0000","last_fullsized":"2024-04-16T17:06:45.240099+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:46:51.634944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"95'5","reported_seq":212,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.445694+0000","last_change":"2024-04-16T17:03:24.153774+0000","last_active":"2024-04-16T17:06:45.445694+0000","last_peered":"2024-04-16T17:06:45.445694+0000","last_clean":"2024-04-16T17:06:45.445694+0000","last_became_active":"2024-04-16T17:03:24.153418+0000","last_became_peered":"2024-04-16T17:03:24.153418+0000","last_unstale":"2024-04-16T17:06:45.445694+0000","last_undegraded":"2024-04-16T17:06:45.445694+0000","last_fullsized":"2024-04-16T17:06:45.445694+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:58:36.093789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":173,"num_read_kb":173,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"92'10","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.451116+0000","last_change":"2024-04-16T17:03:24.962838+0000","last_active":"2024-04-16T17:06:45.451116+0000","last_peered":"2024-04-16T17:06:45.451116+0000","last_clean":"2024-04-16T17:06:45.451116+0000","last_became_active":"2024-04-16T17:03:24.960163+0000","last_became_peered":"2024-04-16T17:03:24.960163+0000","last_unstale":"2024-04-16T17:06:45.451116+0000","last_undegraded":"2024-04-16T17:06:45.451116+0000","last_fullsized":"2024-04-16T17:06:45.451116+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:48:18.723499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.101798+0000","last_change":"2024-04-16T17:03:29.251177+0000","last_active":"2024-04-16T17:06:45.101798+0000","last_peered":"2024-04-16T17:06:45.101798+0000","last_clean":"2024-04-16T17:06:45.101798+0000","last_became_active":"2024-04-16T17:03:29.250657+0000","last_became_peered":"2024-04-16T17:03:29.250657+0000","last_unstale":"2024-04-16T17:06:45.101798+0000","last_undegraded":"2024-04-16T17:06:45.101798+0000","last_fullsized":"2024-04-16T17:06:45.101798+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:16:47.217193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"90'2","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.443359+0000","last_change":"2024-04-16T17:03:24.154245+0000","last_active":"2024-04-16T17:06:45.443359+0000","last_peered":"2024-04-16T17:06:45.443359+0000","last_clean":"2024-04-16T17:06:45.443359+0000","last_became_active":"2024-04-16T17:03:24.153297+0000","last_became_peered":"2024-04-16T17:03:24.153297+0000","last_unstale":"2024-04-16T17:06:45.443359+0000","last_undegraded":"2024-04-16T17:06:45.443359+0000","last_fullsized":"2024-04-16T17:06:45.443359+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:23:49.490238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"92'9","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.577695+0000","last_change":"2024-04-16T17:03:24.984572+0000","last_active":"2024-04-16T17:06:45.577695+0000","last_peered":"2024-04-16T17:06:45.577695+0000","last_clean":"2024-04-16T17:06:45.577695+0000","last_became_active":"2024-04-16T17:03:24.983588+0000","last_became_peered":"2024-04-16T17:03:24.983588+0000","last_unstale":"2024-04-16T17:06:45.577695+0000","last_undegraded":"2024-04-16T17:06:45.577695+0000","last_fullsized":"2024-04-16T17:06:45.577695+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:41:24.704602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.105052+0000","last_change":"2024-04-16T17:03:29.245129+0000","last_active":"2024-04-16T17:06:45.105052+0000","last_peered":"2024-04-16T17:06:45.105052+0000","last_clean":"2024-04-16T17:06:45.105052+0000","last_became_active":"2024-04-16T17:03:29.244720+0000","last_became_peered":"2024-04-16T17:03:29.244720+0000","last_unstale":"2024-04-16T17:06:45.105052+0000","last_undegraded":"2024-04-16T17:06:45.105052+0000","last_fullsized":"2024-04-16T17:06:45.105052+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:55:26.612176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"90'4","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.251165+0000","last_change":"2024-04-16T17:03:24.165248+0000","last_active":"2024-04-16T17:06:45.251165+0000","last_peered":"2024-04-16T17:06:45.251165+0000","last_clean":"2024-04-16T17:06:45.251165+0000","last_became_active":"2024-04-16T17:03:24.164915+0000","last_became_peered":"2024-04-16T17:03:24.164915+0000","last_unstale":"2024-04-16T17:06:45.251165+0000","last_undegraded":"2024-04-16T17:06:45.251165+0000","last_fullsized":"2024-04-16T17:06:45.251165+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:04:47.207554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"92'11","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.450807+0000","last_change":"2024-04-16T17:03:24.967292+0000","last_active":"2024-04-16T17:06:45.450807+0000","last_peered":"2024-04-16T17:06:45.450807+0000","last_clean":"2024-04-16T17:06:45.450807+0000","last_became_active":"2024-04-16T17:03:24.966952+0000","last_became_peered":"2024-04-16T17:03:24.966952+0000","last_unstale":"2024-04-16T17:06:45.450807+0000","last_undegraded":"2024-04-16T17:06:45.450807+0000","last_fullsized":"2024-04-16T17:06:45.450807+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:42:54.969478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.101807+0000","last_change":"2024-04-16T17:03:29.257836+0000","last_active":"2024-04-16T17:06:45.101807+0000","last_peered":"2024-04-16T17:06:45.101807+0000","last_clean":"2024-04-16T17:06:45.101807+0000","last_became_active":"2024-04-16T17:03:29.257409+0000","last_became_peered":"2024-04-16T17:03:29.257409+0000","last_unstale":"2024-04-16T17:06:45.101807+0000","last_undegraded":"2024-04-16T17:06:45.101807+0000","last_fullsized":"2024-04-16T17:06:45.101807+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:25:21.288266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"95'41","reported_seq":433,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.629791+0000","last_change":"2024-04-16T17:03:24.150943+0000","last_active":"2024-04-16T17:06:45.629791+0000","last_peered":"2024-04-16T17:06:45.629791+0000","last_clean":"2024-04-16T17:06:45.629791+0000","last_became_active":"2024-04-16T17:03:24.150563+0000","last_became_peered":"2024-04-16T17:03:24.150563+0000","last_unstale":"2024-04-16T17:06:45.629791+0000","last_undegraded":"2024-04-16T17:06:45.629791+0000","last_fullsized":"2024-04-16T17:06:45.629791+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:05:07.380440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":69011,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":358,"num_read_kb":247,"num_write":42,"num_write_kb":89,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"92'6","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.239659+0000","last_change":"2024-04-16T17:03:24.966393+0000","last_active":"2024-04-16T17:06:45.239659+0000","last_peered":"2024-04-16T17:06:45.239659+0000","last_clean":"2024-04-16T17:06:45.239659+0000","last_became_active":"2024-04-16T17:03:24.966016+0000","last_became_peered":"2024-04-16T17:03:24.966016+0000","last_unstale":"2024-04-16T17:06:45.239659+0000","last_undegraded":"2024-04-16T17:06:45.239659+0000","last_fullsized":"2024-04-16T17:06:45.239659+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:30:25.203262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.251684+0000","last_change":"2024-04-16T17:03:29.265371+0000","last_active":"2024-04-16T17:06:45.251684+0000","last_peered":"2024-04-16T17:06:45.251684+0000","last_clean":"2024-04-16T17:06:45.251684+0000","last_became_active":"2024-04-16T17:03:29.264933+0000","last_became_peered":"2024-04-16T17:03:29.264933+0000","last_unstale":"2024-04-16T17:06:45.251684+0000","last_undegraded":"2024-04-16T17:06:45.251684+0000","last_fullsized":"2024-04-16T17:06:45.251684+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:50:18.203015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"90'6","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.557238+0000","last_change":"2024-04-16T17:04:21.526796+0000","last_active":"2024-04-16T17:06:45.557238+0000","last_peered":"2024-04-16T17:06:45.557238+0000","last_clean":"2024-04-16T17:06:45.557238+0000","last_became_active":"2024-04-16T17:04:21.508364+0000","last_became_peered":"2024-04-16T17:04:21.508364+0000","last_unstale":"2024-04-16T17:06:45.557238+0000","last_undegraded":"2024-04-16T17:06:45.557238+0000","last_fullsized":"2024-04-16T17:06:45.557238+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:45:42.726648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"92'9","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.443893+0000","last_change":"2024-04-16T17:03:24.986228+0000","last_active":"2024-04-16T17:06:45.443893+0000","last_peered":"2024-04-16T17:06:45.443893+0000","last_clean":"2024-04-16T17:06:45.443893+0000","last_became_active":"2024-04-16T17:03:24.985916+0000","last_became_peered":"2024-04-16T17:03:24.985916+0000","last_unstale":"2024-04-16T17:06:45.443893+0000","last_undegraded":"2024-04-16T17:06:45.443893+0000","last_fullsized":"2024-04-16T17:06:45.443893+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:23:54.309203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.557348+0000","last_change":"2024-04-16T17:03:29.264636+0000","last_active":"2024-04-16T17:06:45.557348+0000","last_peered":"2024-04-16T17:06:45.557348+0000","last_clean":"2024-04-16T17:06:45.557348+0000","last_became_active":"2024-04-16T17:03:29.264117+0000","last_became_peered":"2024-04-16T17:03:29.264117+0000","last_unstale":"2024-04-16T17:06:45.557348+0000","last_undegraded":"2024-04-16T17:06:45.557348+0000","last_fullsized":"2024-04-16T17:06:45.557348+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:42:36.123154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"90'3","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.251233+0000","last_change":"2024-04-16T17:03:24.166844+0000","last_active":"2024-04-16T17:06:45.251233+0000","last_peered":"2024-04-16T17:06:45.251233+0000","last_clean":"2024-04-16T17:06:45.251233+0000","last_became_active":"2024-04-16T17:03:24.166479+0000","last_became_peered":"2024-04-16T17:03:24.166479+0000","last_unstale":"2024-04-16T17:06:45.251233+0000","last_undegraded":"2024-04-16T17:06:45.251233+0000","last_fullsized":"2024-04-16T17:06:45.251233+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:02:20.139865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"92'15","reported_seq":47,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.443198+0000","last_change":"2024-04-16T17:03:24.973057+0000","last_active":"2024-04-16T17:06:45.443198+0000","last_peered":"2024-04-16T17:06:45.443198+0000","last_clean":"2024-04-16T17:06:45.443198+0000","last_became_active":"2024-04-16T17:03:24.972731+0000","last_became_peered":"2024-04-16T17:03:24.972731+0000","last_unstale":"2024-04-16T17:06:45.443198+0000","last_undegraded":"2024-04-16T17:06:45.443198+0000","last_fullsized":"2024-04-16T17:06:45.443198+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:37:38.702919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.102363+0000","last_change":"2024-04-16T17:03:29.259167+0000","last_active":"2024-04-16T17:06:45.102363+0000","last_peered":"2024-04-16T17:06:45.102363+0000","last_clean":"2024-04-16T17:06:45.102363+0000","last_became_active":"2024-04-16T17:03:29.258737+0000","last_became_peered":"2024-04-16T17:03:29.258737+0000","last_unstale":"2024-04-16T17:06:45.102363+0000","last_undegraded":"2024-04-16T17:06:45.102363+0000","last_fullsized":"2024-04-16T17:06:45.102363+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:01:13.774037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"95'8","reported_seq":221,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.788731+0000","last_change":"2024-04-16T17:03:24.151064+0000","last_active":"2024-04-16T17:06:45.788731+0000","last_peered":"2024-04-16T17:06:45.788731+0000","last_clean":"2024-04-16T17:06:45.788731+0000","last_became_active":"2024-04-16T17:03:24.150745+0000","last_became_peered":"2024-04-16T17:03:24.150745+0000","last_unstale":"2024-04-16T17:06:45.788731+0000","last_undegraded":"2024-04-16T17:06:45.788731+0000","last_fullsized":"2024-04-16T17:06:45.788731+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:32:26.271008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2212,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":180,"num_read_kb":7,"num_write":7,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"92'6","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.442895+0000","last_change":"2024-04-16T17:03:24.971948+0000","last_active":"2024-04-16T17:06:45.442895+0000","last_peered":"2024-04-16T17:06:45.442895+0000","last_clean":"2024-04-16T17:06:45.442895+0000","last_became_active":"2024-04-16T17:03:24.971530+0000","last_became_peered":"2024-04-16T17:03:24.971530+0000","last_unstale":"2024-04-16T17:06:45.442895+0000","last_undegraded":"2024-04-16T17:06:45.442895+0000","last_fullsized":"2024-04-16T17:06:45.442895+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:03:29.286160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.445918+0000","last_change":"2024-04-16T17:03:29.264821+0000","last_active":"2024-04-16T17:06:45.445918+0000","last_peered":"2024-04-16T17:06:45.445918+0000","last_clean":"2024-04-16T17:06:45.445918+0000","last_became_active":"2024-04-16T17:03:29.264287+0000","last_became_peered":"2024-04-16T17:03:29.264287+0000","last_unstale":"2024-04-16T17:06:45.445918+0000","last_undegraded":"2024-04-16T17:06:45.445918+0000","last_fullsized":"2024-04-16T17:06:45.445918+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:33:28.400663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"90'15","reported_seq":50,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.239188+0000","last_change":"2024-04-16T17:03:24.158844+0000","last_active":"2024-04-16T17:06:45.239188+0000","last_peered":"2024-04-16T17:06:45.239188+0000","last_clean":"2024-04-16T17:06:45.239188+0000","last_became_active":"2024-04-16T17:03:24.158537+0000","last_became_peered":"2024-04-16T17:03:24.158537+0000","last_unstale":"2024-04-16T17:06:45.239188+0000","last_undegraded":"2024-04-16T17:06:45.239188+0000","last_fullsized":"2024-04-16T17:06:45.239188+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:42:22.772452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"92'6","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.251080+0000","last_change":"2024-04-16T17:03:24.962522+0000","last_active":"2024-04-16T17:06:45.251080+0000","last_peered":"2024-04-16T17:06:45.251080+0000","last_clean":"2024-04-16T17:06:45.251080+0000","last_became_active":"2024-04-16T17:03:24.962169+0000","last_became_peered":"2024-04-16T17:03:24.962169+0000","last_unstale":"2024-04-16T17:06:45.251080+0000","last_undegraded":"2024-04-16T17:06:45.251080+0000","last_fullsized":"2024-04-16T17:06:45.251080+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:28:17.147491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.250937+0000","last_change":"2024-04-16T17:03:29.259539+0000","last_active":"2024-04-16T17:06:45.250937+0000","last_peered":"2024-04-16T17:06:45.250937+0000","last_clean":"2024-04-16T17:06:45.250937+0000","last_became_active":"2024-04-16T17:03:29.259123+0000","last_became_peered":"2024-04-16T17:03:29.259123+0000","last_unstale":"2024-04-16T17:06:45.250937+0000","last_undegraded":"2024-04-16T17:06:45.250937+0000","last_fullsized":"2024-04-16T17:06:45.250937+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:15:17.142636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"94'3","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.239865+0000","last_change":"2024-04-16T17:03:24.147526+0000","last_active":"2024-04-16T17:06:45.239865+0000","last_peered":"2024-04-16T17:06:45.239865+0000","last_clean":"2024-04-16T17:06:45.239865+0000","last_became_active":"2024-04-16T17:03:24.147119+0000","last_became_peered":"2024-04-16T17:03:24.147119+0000","last_unstale":"2024-04-16T17:06:45.239865+0000","last_undegraded":"2024-04-16T17:06:45.239865+0000","last_fullsized":"2024-04-16T17:06:45.239865+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:30:38.742122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"92'5","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.557187+0000","last_change":"2024-04-16T17:04:21.507113+0000","last_active":"2024-04-16T17:06:45.557187+0000","last_peered":"2024-04-16T17:06:45.557187+0000","last_clean":"2024-04-16T17:06:45.557187+0000","last_became_active":"2024-04-16T17:04:21.506680+0000","last_became_peered":"2024-04-16T17:04:21.506680+0000","last_unstale":"2024-04-16T17:06:45.557187+0000","last_undegraded":"2024-04-16T17:06:45.557187+0000","last_fullsized":"2024-04-16T17:06:45.557187+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:08:47.971317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,8],"acting":[11,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":32,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.239441+0000","last_change":"2024-04-16T17:05:22.511966+0000","last_active":"2024-04-16T17:06:45.239441+0000","last_peered":"2024-04-16T17:06:45.239441+0000","last_clean":"2024-04-16T17:06:45.239441+0000","last_became_active":"2024-04-16T17:05:22.511512+0000","last_became_peered":"2024-04-16T17:05:22.511512+0000","last_unstale":"2024-04-16T17:06:45.239441+0000","last_undegraded":"2024-04-16T17:06:45.239441+0000","last_fullsized":"2024-04-16T17:06:45.239441+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:14:41.291499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,1],"acting":[8,10,11,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.102181+0000","last_change":"2024-04-16T17:03:24.162105+0000","last_active":"2024-04-16T17:06:45.102181+0000","last_peered":"2024-04-16T17:06:45.102181+0000","last_clean":"2024-04-16T17:06:45.102181+0000","last_became_active":"2024-04-16T17:03:24.161631+0000","last_became_peered":"2024-04-16T17:03:24.161631+0000","last_unstale":"2024-04-16T17:06:45.102181+0000","last_undegraded":"2024-04-16T17:06:45.102181+0000","last_fullsized":"2024-04-16T17:06:45.102181+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:22:33.677848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"92'4","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.226784+0000","last_change":"2024-04-16T17:03:24.978253+0000","last_active":"2024-04-16T17:06:45.226784+0000","last_peered":"2024-04-16T17:06:45.226784+0000","last_clean":"2024-04-16T17:06:45.226784+0000","last_became_active":"2024-04-16T17:03:24.977956+0000","last_became_peered":"2024-04-16T17:03:24.977956+0000","last_unstale":"2024-04-16T17:06:45.226784+0000","last_undegraded":"2024-04-16T17:06:45.226784+0000","last_fullsized":"2024-04-16T17:06:45.226784+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:59:45.102383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.444635+0000","last_change":"2024-04-16T17:03:29.265312+0000","last_active":"2024-04-16T17:06:45.444635+0000","last_peered":"2024-04-16T17:06:45.444635+0000","last_clean":"2024-04-16T17:06:45.444635+0000","last_became_active":"2024-04-16T17:03:29.264891+0000","last_became_peered":"2024-04-16T17:03:29.264891+0000","last_unstale":"2024-04-16T17:06:45.444635+0000","last_undegraded":"2024-04-16T17:06:45.444635+0000","last_fullsized":"2024-04-16T17:06:45.444635+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:26:40.088127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"90'1","reported_seq":35,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.442929+0000","last_change":"2024-04-16T17:03:24.170935+0000","last_active":"2024-04-16T17:06:45.442929+0000","last_peered":"2024-04-16T17:06:45.442929+0000","last_clean":"2024-04-16T17:06:45.442929+0000","last_became_active":"2024-04-16T17:03:24.170609+0000","last_became_peered":"2024-04-16T17:03:24.170609+0000","last_unstale":"2024-04-16T17:06:45.442929+0000","last_undegraded":"2024-04-16T17:06:45.442929+0000","last_fullsized":"2024-04-16T17:06:45.442929+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:18:33.516309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"92'7","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.226731+0000","last_change":"2024-04-16T17:03:24.985434+0000","last_active":"2024-04-16T17:06:45.226731+0000","last_peered":"2024-04-16T17:06:45.226731+0000","last_clean":"2024-04-16T17:06:45.226731+0000","last_became_active":"2024-04-16T17:03:24.984388+0000","last_became_peered":"2024-04-16T17:03:24.984388+0000","last_unstale":"2024-04-16T17:06:45.226731+0000","last_undegraded":"2024-04-16T17:06:45.226731+0000","last_fullsized":"2024-04-16T17:06:45.226731+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:43:26.355572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":32,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.445935+0000","last_change":"2024-04-16T17:05:22.516267+0000","last_active":"2024-04-16T17:06:45.445935+0000","last_peered":"2024-04-16T17:06:45.445935+0000","last_clean":"2024-04-16T17:06:45.445935+0000","last_became_active":"2024-04-16T17:05:22.513637+0000","last_became_peered":"2024-04-16T17:05:22.513637+0000","last_unstale":"2024-04-16T17:06:45.445935+0000","last_undegraded":"2024-04-16T17:06:45.445935+0000","last_fullsized":"2024-04-16T17:06:45.445935+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:39:47.975735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,8,10,0],"acting":[6,8,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"92'8","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.251371+0000","last_change":"2024-04-16T17:03:24.962786+0000","last_active":"2024-04-16T17:06:45.251371+0000","last_peered":"2024-04-16T17:06:45.251371+0000","last_clean":"2024-04-16T17:06:45.251371+0000","last_became_active":"2024-04-16T17:03:24.961704+0000","last_became_peered":"2024-04-16T17:03:24.961704+0000","last_unstale":"2024-04-16T17:06:45.251371+0000","last_undegraded":"2024-04-16T17:06:45.251371+0000","last_fullsized":"2024-04-16T17:06:45.251371+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:22:27.957607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"94'6","reported_seq":213,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.443944+0000","last_change":"2024-04-16T17:03:24.175617+0000","last_active":"2024-04-16T17:06:45.443944+0000","last_peered":"2024-04-16T17:06:45.443944+0000","last_clean":"2024-04-16T17:06:45.443944+0000","last_became_active":"2024-04-16T17:03:24.175272+0000","last_became_peered":"2024-04-16T17:03:24.175272+0000","last_unstale":"2024-04-16T17:06:45.443944+0000","last_undegraded":"2024-04-16T17:06:45.443944+0000","last_fullsized":"2024-04-16T17:06:45.443944+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:27:44.738030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":173,"num_read_kb":173,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"90'1","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.443256+0000","last_change":"2024-04-16T17:04:21.512033+0000","last_active":"2024-04-16T17:06:45.443256+0000","last_peered":"2024-04-16T17:06:45.443256+0000","last_clean":"2024-04-16T17:06:45.443256+0000","last_became_active":"2024-04-16T17:04:21.506718+0000","last_became_peered":"2024-04-16T17:04:21.506718+0000","last_unstale":"2024-04-16T17:06:45.443256+0000","last_undegraded":"2024-04-16T17:06:45.443256+0000","last_fullsized":"2024-04-16T17:06:45.443256+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:53:48.953887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,10],"acting":[7,5,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"92'8","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.240468+0000","last_change":"2024-04-16T17:03:24.983629+0000","last_active":"2024-04-16T17:06:45.240468+0000","last_peered":"2024-04-16T17:06:45.240468+0000","last_clean":"2024-04-16T17:06:45.240468+0000","last_became_active":"2024-04-16T17:03:24.979969+0000","last_became_peered":"2024-04-16T17:03:24.979969+0000","last_unstale":"2024-04-16T17:06:45.240468+0000","last_undegraded":"2024-04-16T17:06:45.240468+0000","last_fullsized":"2024-04-16T17:06:45.240468+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:01:24.391732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"90'3","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.104956+0000","last_change":"2024-04-16T17:03:24.152475+0000","last_active":"2024-04-16T17:06:45.104956+0000","last_peered":"2024-04-16T17:06:45.104956+0000","last_clean":"2024-04-16T17:06:45.104956+0000","last_became_active":"2024-04-16T17:03:24.152198+0000","last_became_peered":"2024-04-16T17:03:24.152198+0000","last_unstale":"2024-04-16T17:06:45.104956+0000","last_undegraded":"2024-04-16T17:06:45.104956+0000","last_fullsized":"2024-04-16T17:06:45.104956+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:49:33.914325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":367,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.442836+0000","last_change":"2024-04-16T17:02:02.111335+0000","last_active":"2024-04-16T17:06:45.442836+0000","last_peered":"2024-04-16T17:06:45.442836+0000","last_clean":"2024-04-16T17:06:45.442836+0000","last_became_active":"2024-04-16T17:02:02.110928+0000","last_became_peered":"2024-04-16T17:02:02.110928+0000","last_unstale":"2024-04-16T17:06:45.442836+0000","last_undegraded":"2024-04-16T17:06:45.442836+0000","last_fullsized":"2024-04-16T17:06:45.442836+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T16:58:07.081753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T16:58:07.081753+0000","last_clean_scrub_stamp":"2024-04-16T16:58:07.081753+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:27:02.865664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.251060+0000","last_change":"2024-04-16T17:03:29.301691+0000","last_active":"2024-04-16T17:06:45.251060+0000","last_peered":"2024-04-16T17:06:45.251060+0000","last_clean":"2024-04-16T17:06:45.251060+0000","last_became_active":"2024-04-16T17:03:29.301277+0000","last_became_peered":"2024-04-16T17:03:29.301277+0000","last_unstale":"2024-04-16T17:06:45.251060+0000","last_undegraded":"2024-04-16T17:06:45.251060+0000","last_fullsized":"2024-04-16T17:06:45.251060+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:45:16.727610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"92'5","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.443705+0000","last_change":"2024-04-16T17:03:24.970939+0000","last_active":"2024-04-16T17:06:45.443705+0000","last_peered":"2024-04-16T17:06:45.443705+0000","last_clean":"2024-04-16T17:06:45.443705+0000","last_became_active":"2024-04-16T17:03:24.970581+0000","last_became_peered":"2024-04-16T17:03:24.970581+0000","last_unstale":"2024-04-16T17:06:45.443705+0000","last_undegraded":"2024-04-16T17:06:45.443705+0000","last_fullsized":"2024-04-16T17:06:45.443705+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:33:07.448597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"90'3","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.442815+0000","last_change":"2024-04-16T17:03:24.155454+0000","last_active":"2024-04-16T17:06:45.442815+0000","last_peered":"2024-04-16T17:06:45.442815+0000","last_clean":"2024-04-16T17:06:45.442815+0000","last_became_active":"2024-04-16T17:03:24.155155+0000","last_became_peered":"2024-04-16T17:03:24.155155+0000","last_unstale":"2024-04-16T17:06:45.442815+0000","last_undegraded":"2024-04-16T17:06:45.442815+0000","last_fullsized":"2024-04-16T17:06:45.442815+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:34:08.618002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.557582+0000","last_change":"2024-04-16T17:03:29.255538+0000","last_active":"2024-04-16T17:06:45.557582+0000","last_peered":"2024-04-16T17:06:45.557582+0000","last_clean":"2024-04-16T17:06:45.557582+0000","last_became_active":"2024-04-16T17:03:29.255107+0000","last_became_peered":"2024-04-16T17:03:29.255107+0000","last_unstale":"2024-04-16T17:06:45.557582+0000","last_undegraded":"2024-04-16T17:06:45.557582+0000","last_fullsized":"2024-04-16T17:06:45.557582+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:31:58.458641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"90'1","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.239669+0000","last_change":"2024-04-16T17:03:24.158816+0000","last_active":"2024-04-16T17:06:45.239669+0000","last_peered":"2024-04-16T17:06:45.239669+0000","last_clean":"2024-04-16T17:06:45.239669+0000","last_became_active":"2024-04-16T17:03:24.158518+0000","last_became_peered":"2024-04-16T17:03:24.158518+0000","last_unstale":"2024-04-16T17:06:45.239669+0000","last_undegraded":"2024-04-16T17:06:45.239669+0000","last_fullsized":"2024-04-16T17:06:45.239669+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:54:55.807975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"92'9","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.445158+0000","last_change":"2024-04-16T17:03:24.977190+0000","last_active":"2024-04-16T17:06:45.445158+0000","last_peered":"2024-04-16T17:06:45.445158+0000","last_clean":"2024-04-16T17:06:45.445158+0000","last_became_active":"2024-04-16T17:03:24.976840+0000","last_became_peered":"2024-04-16T17:03:24.976840+0000","last_unstale":"2024-04-16T17:06:45.445158+0000","last_undegraded":"2024-04-16T17:06:45.445158+0000","last_fullsized":"2024-04-16T17:06:45.445158+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:21:51.294150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.239834+0000","last_change":"2024-04-16T17:03:29.301423+0000","last_active":"2024-04-16T17:06:45.239834+0000","last_peered":"2024-04-16T17:06:45.239834+0000","last_clean":"2024-04-16T17:06:45.239834+0000","last_became_active":"2024-04-16T17:03:29.300888+0000","last_became_peered":"2024-04-16T17:03:29.300888+0000","last_unstale":"2024-04-16T17:06:45.239834+0000","last_undegraded":"2024-04-16T17:06:45.239834+0000","last_fullsized":"2024-04-16T17:06:45.239834+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:13:54.188090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"95'46","reported_seq":103,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.390333+0000","last_change":"2024-04-16T17:03:24.167643+0000","last_active":"2024-04-16T17:06:45.390333+0000","last_peered":"2024-04-16T17:06:45.390333+0000","last_clean":"2024-04-16T17:06:45.390333+0000","last_became_active":"2024-04-16T17:03:24.167103+0000","last_became_peered":"2024-04-16T17:03:24.167103+0000","last_unstale":"2024-04-16T17:06:45.390333+0000","last_undegraded":"2024-04-16T17:06:45.390333+0000","last_fullsized":"2024-04-16T17:06:45.390333+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:33:16.809371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":82,"num_write_kb":837,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"92'5","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.241641+0000","last_change":"2024-04-16T17:03:24.976981+0000","last_active":"2024-04-16T17:06:45.241641+0000","last_peered":"2024-04-16T17:06:45.241641+0000","last_clean":"2024-04-16T17:06:45.241641+0000","last_became_active":"2024-04-16T17:03:24.975385+0000","last_became_peered":"2024-04-16T17:03:24.975385+0000","last_unstale":"2024-04-16T17:06:45.241641+0000","last_undegraded":"2024-04-16T17:06:45.241641+0000","last_fullsized":"2024-04-16T17:06:45.241641+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:27:03.104838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.105056+0000","last_change":"2024-04-16T17:03:29.244767+0000","last_active":"2024-04-16T17:06:45.105056+0000","last_peered":"2024-04-16T17:06:45.105056+0000","last_clean":"2024-04-16T17:06:45.105056+0000","last_became_active":"2024-04-16T17:03:29.244205+0000","last_became_peered":"2024-04-16T17:03:29.244205+0000","last_unstale":"2024-04-16T17:06:45.105056+0000","last_undegraded":"2024-04-16T17:06:45.105056+0000","last_fullsized":"2024-04-16T17:06:45.105056+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:51:46.652387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"90'4","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.105248+0000","last_change":"2024-04-16T17:03:24.164916+0000","last_active":"2024-04-16T17:06:45.105248+0000","last_peered":"2024-04-16T17:06:45.105248+0000","last_clean":"2024-04-16T17:06:45.105248+0000","last_became_active":"2024-04-16T17:03:24.164648+0000","last_became_peered":"2024-04-16T17:03:24.164648+0000","last_unstale":"2024-04-16T17:06:45.105248+0000","last_undegraded":"2024-04-16T17:06:45.105248+0000","last_fullsized":"2024-04-16T17:06:45.105248+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:23:00.198785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"92'7","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.577426+0000","last_change":"2024-04-16T17:03:24.994393+0000","last_active":"2024-04-16T17:06:45.577426+0000","last_peered":"2024-04-16T17:06:45.577426+0000","last_clean":"2024-04-16T17:06:45.577426+0000","last_became_active":"2024-04-16T17:03:24.994002+0000","last_became_peered":"2024-04-16T17:03:24.994002+0000","last_unstale":"2024-04-16T17:06:45.577426+0000","last_undegraded":"2024-04-16T17:06:45.577426+0000","last_fullsized":"2024-04-16T17:06:45.577426+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:23:44.596781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.104795+0000","last_change":"2024-04-16T17:03:29.302009+0000","last_active":"2024-04-16T17:06:45.104795+0000","last_peered":"2024-04-16T17:06:45.104795+0000","last_clean":"2024-04-16T17:06:45.104795+0000","last_became_active":"2024-04-16T17:03:29.301183+0000","last_became_peered":"2024-04-16T17:03:29.301183+0000","last_unstale":"2024-04-16T17:06:45.104795+0000","last_undegraded":"2024-04-16T17:06:45.104795+0000","last_fullsized":"2024-04-16T17:06:45.104795+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:30:19.561875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"90'2","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.450559+0000","last_change":"2024-04-16T17:03:24.176829+0000","last_active":"2024-04-16T17:06:45.450559+0000","last_peered":"2024-04-16T17:06:45.450559+0000","last_clean":"2024-04-16T17:06:45.450559+0000","last_became_active":"2024-04-16T17:03:24.176555+0000","last_became_peered":"2024-04-16T17:03:24.176555+0000","last_unstale":"2024-04-16T17:06:45.450559+0000","last_undegraded":"2024-04-16T17:06:45.450559+0000","last_fullsized":"2024-04-16T17:06:45.450559+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:05:40.602150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"92'2","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.239184+0000","last_change":"2024-04-16T17:03:24.965746+0000","last_active":"2024-04-16T17:06:45.239184+0000","last_peered":"2024-04-16T17:06:45.239184+0000","last_clean":"2024-04-16T17:06:45.239184+0000","last_became_active":"2024-04-16T17:03:24.965241+0000","last_became_peered":"2024-04-16T17:03:24.965241+0000","last_unstale":"2024-04-16T17:06:45.239184+0000","last_undegraded":"2024-04-16T17:06:45.239184+0000","last_fullsized":"2024-04-16T17:06:45.239184+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:51:34.407565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.102475+0000","last_change":"2024-04-16T17:03:29.261441+0000","last_active":"2024-04-16T17:06:45.102475+0000","last_peered":"2024-04-16T17:06:45.102475+0000","last_clean":"2024-04-16T17:06:45.102475+0000","last_became_active":"2024-04-16T17:03:29.259538+0000","last_became_peered":"2024-04-16T17:03:29.259538+0000","last_unstale":"2024-04-16T17:06:45.102475+0000","last_undegraded":"2024-04-16T17:06:45.102475+0000","last_fullsized":"2024-04-16T17:06:45.102475+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:12:27.092053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"90'3","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.240172+0000","last_change":"2024-04-16T17:03:24.172053+0000","last_active":"2024-04-16T17:06:45.240172+0000","last_peered":"2024-04-16T17:06:45.240172+0000","last_clean":"2024-04-16T17:06:45.240172+0000","last_became_active":"2024-04-16T17:03:24.171709+0000","last_became_peered":"2024-04-16T17:03:24.171709+0000","last_unstale":"2024-04-16T17:06:45.240172+0000","last_undegraded":"2024-04-16T17:06:45.240172+0000","last_fullsized":"2024-04-16T17:06:45.240172+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:40:12.307756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"92'9","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.557252+0000","last_change":"2024-04-16T17:03:24.984267+0000","last_active":"2024-04-16T17:06:45.557252+0000","last_peered":"2024-04-16T17:06:45.557252+0000","last_clean":"2024-04-16T17:06:45.557252+0000","last_became_active":"2024-04-16T17:03:24.983888+0000","last_became_peered":"2024-04-16T17:03:24.983888+0000","last_unstale":"2024-04-16T17:06:45.557252+0000","last_undegraded":"2024-04-16T17:06:45.557252+0000","last_fullsized":"2024-04-16T17:06:45.557252+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:16:23.814917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"90'1","reported_seq":26,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.105091+0000","last_change":"2024-04-16T17:03:29.251716+0000","last_active":"2024-04-16T17:06:45.105091+0000","last_peered":"2024-04-16T17:06:45.105091+0000","last_clean":"2024-04-16T17:06:45.105091+0000","last_became_active":"2024-04-16T17:03:29.251317+0000","last_became_peered":"2024-04-16T17:03:29.251317+0000","last_unstale":"2024-04-16T17:06:45.105091+0000","last_undegraded":"2024-04-16T17:06:45.105091+0000","last_fullsized":"2024-04-16T17:06:45.105091+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:48:54.680195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"90'1","reported_seq":22,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.557164+0000","last_change":"2024-04-16T17:04:21.515306+0000","last_active":"2024-04-16T17:06:45.557164+0000","last_peered":"2024-04-16T17:06:45.557164+0000","last_clean":"2024-04-16T17:06:45.557164+0000","last_became_active":"2024-04-16T17:04:21.508377+0000","last_became_peered":"2024-04-16T17:04:21.508377+0000","last_unstale":"2024-04-16T17:06:45.557164+0000","last_undegraded":"2024-04-16T17:06:45.557164+0000","last_fullsized":"2024-04-16T17:06:45.557164+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:28:25.000406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"92'8","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.240125+0000","last_change":"2024-04-16T17:03:24.995547+0000","last_active":"2024-04-16T17:06:45.240125+0000","last_peered":"2024-04-16T17:06:45.240125+0000","last_clean":"2024-04-16T17:06:45.240125+0000","last_became_active":"2024-04-16T17:03:24.995096+0000","last_became_peered":"2024-04-16T17:03:24.995096+0000","last_unstale":"2024-04-16T17:06:45.240125+0000","last_undegraded":"2024-04-16T17:06:45.240125+0000","last_fullsized":"2024-04-16T17:06:45.240125+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:54:51.193117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.104926+0000","last_change":"2024-04-16T17:03:29.301526+0000","last_active":"2024-04-16T17:06:45.104926+0000","last_peered":"2024-04-16T17:06:45.104926+0000","last_clean":"2024-04-16T17:06:45.104926+0000","last_became_active":"2024-04-16T17:03:29.300808+0000","last_became_peered":"2024-04-16T17:03:29.300808+0000","last_unstale":"2024-04-16T17:06:45.104926+0000","last_undegraded":"2024-04-16T17:06:45.104926+0000","last_fullsized":"2024-04-16T17:06:45.104926+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:01:14.717548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"90'4","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.239357+0000","last_change":"2024-04-16T17:03:24.152645+0000","last_active":"2024-04-16T17:06:45.239357+0000","last_peered":"2024-04-16T17:06:45.239357+0000","last_clean":"2024-04-16T17:06:45.239357+0000","last_became_active":"2024-04-16T17:03:24.152281+0000","last_became_peered":"2024-04-16T17:03:24.152281+0000","last_unstale":"2024-04-16T17:06:45.239357+0000","last_undegraded":"2024-04-16T17:06:45.239357+0000","last_fullsized":"2024-04-16T17:06:45.239357+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:33:51.377421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"92'5","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.443067+0000","last_change":"2024-04-16T17:03:24.979827+0000","last_active":"2024-04-16T17:06:45.443067+0000","last_peered":"2024-04-16T17:06:45.443067+0000","last_clean":"2024-04-16T17:06:45.443067+0000","last_became_active":"2024-04-16T17:03:24.979452+0000","last_became_peered":"2024-04-16T17:03:24.979452+0000","last_unstale":"2024-04-16T17:06:45.443067+0000","last_undegraded":"2024-04-16T17:06:45.443067+0000","last_fullsized":"2024-04-16T17:06:45.443067+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:07:09.259366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.450802+0000","last_change":"2024-04-16T17:03:29.255519+0000","last_active":"2024-04-16T17:06:45.450802+0000","last_peered":"2024-04-16T17:06:45.450802+0000","last_clean":"2024-04-16T17:06:45.450802+0000","last_became_active":"2024-04-16T17:03:29.255104+0000","last_became_peered":"2024-04-16T17:03:29.255104+0000","last_unstale":"2024-04-16T17:06:45.450802+0000","last_undegraded":"2024-04-16T17:06:45.450802+0000","last_fullsized":"2024-04-16T17:06:45.450802+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:27:34.496199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"90'3","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.105533+0000","last_change":"2024-04-16T17:03:24.161145+0000","last_active":"2024-04-16T17:06:45.105533+0000","last_peered":"2024-04-16T17:06:45.105533+0000","last_clean":"2024-04-16T17:06:45.105533+0000","last_became_active":"2024-04-16T17:03:24.160824+0000","last_became_peered":"2024-04-16T17:03:24.160824+0000","last_unstale":"2024-04-16T17:06:45.105533+0000","last_undegraded":"2024-04-16T17:06:45.105533+0000","last_fullsized":"2024-04-16T17:06:45.105533+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:02:48.805638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"92'7","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.443517+0000","last_change":"2024-04-16T17:03:24.966513+0000","last_active":"2024-04-16T17:06:45.443517+0000","last_peered":"2024-04-16T17:06:45.443517+0000","last_clean":"2024-04-16T17:06:45.443517+0000","last_became_active":"2024-04-16T17:03:24.966074+0000","last_became_peered":"2024-04-16T17:03:24.966074+0000","last_unstale":"2024-04-16T17:06:45.443517+0000","last_undegraded":"2024-04-16T17:06:45.443517+0000","last_fullsized":"2024-04-16T17:06:45.443517+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:50:21.579901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":32,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.576687+0000","last_change":"2024-04-16T17:04:22.150786+0000","last_active":"2024-04-16T17:06:45.576687+0000","last_peered":"2024-04-16T17:06:45.576687+0000","last_clean":"2024-04-16T17:06:45.576687+0000","last_became_active":"2024-04-16T17:04:22.149344+0000","last_became_peered":"2024-04-16T17:04:22.149344+0000","last_unstale":"2024-04-16T17:06:45.576687+0000","last_undegraded":"2024-04-16T17:06:45.576687+0000","last_fullsized":"2024-04-16T17:06:45.576687+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:18:49.421095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,1,4],"acting":[0,8,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"95'39","reported_seq":81,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.386141+0000","last_change":"2024-04-16T17:03:24.158032+0000","last_active":"2024-04-16T17:06:45.386141+0000","last_peered":"2024-04-16T17:06:45.386141+0000","last_clean":"2024-04-16T17:06:45.386141+0000","last_became_active":"2024-04-16T17:03:24.157689+0000","last_became_peered":"2024-04-16T17:03:24.157689+0000","last_unstale":"2024-04-16T17:06:45.386141+0000","last_undegraded":"2024-04-16T17:06:45.386141+0000","last_fullsized":"2024-04-16T17:06:45.386141+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:38:18.240889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"92'6","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.443058+0000","last_change":"2024-04-16T17:04:21.519999+0000","last_active":"2024-04-16T17:06:45.443058+0000","last_peered":"2024-04-16T17:06:45.443058+0000","last_clean":"2024-04-16T17:06:45.443058+0000","last_became_active":"2024-04-16T17:04:21.519621+0000","last_became_peered":"2024-04-16T17:04:21.519621+0000","last_unstale":"2024-04-16T17:06:45.443058+0000","last_undegraded":"2024-04-16T17:06:45.443058+0000","last_fullsized":"2024-04-16T17:06:45.443058+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:50:13.266369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.250659+0000","last_change":"2024-04-16T17:03:29.245176+0000","last_active":"2024-04-16T17:06:45.250659+0000","last_peered":"2024-04-16T17:06:45.250659+0000","last_clean":"2024-04-16T17:06:45.250659+0000","last_became_active":"2024-04-16T17:03:29.244763+0000","last_became_peered":"2024-04-16T17:03:29.244763+0000","last_unstale":"2024-04-16T17:06:45.250659+0000","last_undegraded":"2024-04-16T17:06:45.250659+0000","last_fullsized":"2024-04-16T17:06:45.250659+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:52:30.381417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.445799+0000","last_change":"2024-04-16T17:03:24.168068+0000","last_active":"2024-04-16T17:06:45.445799+0000","last_peered":"2024-04-16T17:06:45.445799+0000","last_clean":"2024-04-16T17:06:45.445799+0000","last_became_active":"2024-04-16T17:03:24.167694+0000","last_became_peered":"2024-04-16T17:03:24.167694+0000","last_unstale":"2024-04-16T17:06:45.445799+0000","last_undegraded":"2024-04-16T17:06:45.445799+0000","last_fullsized":"2024-04-16T17:06:45.445799+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:13:35.692306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"92'9","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.443511+0000","last_change":"2024-04-16T17:03:24.966145+0000","last_active":"2024-04-16T17:06:45.443511+0000","last_peered":"2024-04-16T17:06:45.443511+0000","last_clean":"2024-04-16T17:06:45.443511+0000","last_became_active":"2024-04-16T17:03:24.965813+0000","last_became_peered":"2024-04-16T17:03:24.965813+0000","last_unstale":"2024-04-16T17:06:45.443511+0000","last_undegraded":"2024-04-16T17:06:45.443511+0000","last_fullsized":"2024-04-16T17:06:45.443511+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:43:43.224130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"92'9","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.557305+0000","last_change":"2024-04-16T17:03:24.970941+0000","last_active":"2024-04-16T17:06:45.557305+0000","last_peered":"2024-04-16T17:06:45.557305+0000","last_clean":"2024-04-16T17:06:45.557305+0000","last_became_active":"2024-04-16T17:03:24.970587+0000","last_became_peered":"2024-04-16T17:03:24.970587+0000","last_unstale":"2024-04-16T17:06:45.557305+0000","last_undegraded":"2024-04-16T17:06:45.557305+0000","last_fullsized":"2024-04-16T17:06:45.557305+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:14:13.930011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"92'12","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.105601+0000","last_change":"2024-04-16T17:03:24.967167+0000","last_active":"2024-04-16T17:06:45.105601+0000","last_peered":"2024-04-16T17:06:45.105601+0000","last_clean":"2024-04-16T17:06:45.105601+0000","last_became_active":"2024-04-16T17:03:24.966787+0000","last_became_peered":"2024-04-16T17:03:24.966787+0000","last_unstale":"2024-04-16T17:06:45.105601+0000","last_undegraded":"2024-04-16T17:06:45.105601+0000","last_fullsized":"2024-04-16T17:06:45.105601+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:51:39.510220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"92'8","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.105196+0000","last_change":"2024-04-16T17:03:24.981115+0000","last_active":"2024-04-16T17:06:45.105196+0000","last_peered":"2024-04-16T17:06:45.105196+0000","last_clean":"2024-04-16T17:06:45.105196+0000","last_became_active":"2024-04-16T17:03:24.980685+0000","last_became_peered":"2024-04-16T17:03:24.980685+0000","last_unstale":"2024-04-16T17:06:45.105196+0000","last_undegraded":"2024-04-16T17:06:45.105196+0000","last_fullsized":"2024-04-16T17:06:45.105196+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:20:44.066932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"92'7","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.576960+0000","last_change":"2024-04-16T17:03:24.990421+0000","last_active":"2024-04-16T17:06:45.576960+0000","last_peered":"2024-04-16T17:06:45.576960+0000","last_clean":"2024-04-16T17:06:45.576960+0000","last_became_active":"2024-04-16T17:03:24.989940+0000","last_became_peered":"2024-04-16T17:03:24.989940+0000","last_unstale":"2024-04-16T17:06:45.576960+0000","last_undegraded":"2024-04-16T17:06:45.576960+0000","last_fullsized":"2024-04-16T17:06:45.576960+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:19:57.548067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"92'4","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.445217+0000","last_change":"2024-04-16T17:03:24.991854+0000","last_active":"2024-04-16T17:06:45.445217+0000","last_peered":"2024-04-16T17:06:45.445217+0000","last_clean":"2024-04-16T17:06:45.445217+0000","last_became_active":"2024-04-16T17:03:24.990964+0000","last_became_peered":"2024-04-16T17:03:24.990964+0000","last_unstale":"2024-04-16T17:06:45.445217+0000","last_undegraded":"2024-04-16T17:06:45.445217+0000","last_fullsized":"2024-04-16T17:06:45.445217+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:10:10.618708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"92'6","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.557167+0000","last_change":"2024-04-16T17:04:21.517466+0000","last_active":"2024-04-16T17:06:45.557167+0000","last_peered":"2024-04-16T17:06:45.557167+0000","last_clean":"2024-04-16T17:06:45.557167+0000","last_became_active":"2024-04-16T17:04:21.517086+0000","last_became_peered":"2024-04-16T17:04:21.517086+0000","last_unstale":"2024-04-16T17:06:45.557167+0000","last_undegraded":"2024-04-16T17:06:45.557167+0000","last_fullsized":"2024-04-16T17:06:45.557167+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:54:33.417452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,9],"acting":[11,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"92'15","reported_seq":47,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.105712+0000","last_change":"2024-04-16T17:03:24.970377+0000","last_active":"2024-04-16T17:06:45.105712+0000","last_peered":"2024-04-16T17:06:45.105712+0000","last_clean":"2024-04-16T17:06:45.105712+0000","last_became_active":"2024-04-16T17:03:24.968856+0000","last_became_peered":"2024-04-16T17:03:24.968856+0000","last_unstale":"2024-04-16T17:06:45.105712+0000","last_undegraded":"2024-04-16T17:06:45.105712+0000","last_fullsized":"2024-04-16T17:06:45.105712+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:47:28.543120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"92'2","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.241849+0000","last_change":"2024-04-16T17:03:24.992877+0000","last_active":"2024-04-16T17:06:45.241849+0000","last_peered":"2024-04-16T17:06:45.241849+0000","last_clean":"2024-04-16T17:06:45.241849+0000","last_became_active":"2024-04-16T17:03:24.992484+0000","last_became_peered":"2024-04-16T17:03:24.992484+0000","last_unstale":"2024-04-16T17:06:45.241849+0000","last_undegraded":"2024-04-16T17:06:45.241849+0000","last_fullsized":"2024-04-16T17:06:45.241849+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:00:45.204782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"92'6","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.250911+0000","last_change":"2024-04-16T17:03:24.977794+0000","last_active":"2024-04-16T17:06:45.250911+0000","last_peered":"2024-04-16T17:06:45.250911+0000","last_clean":"2024-04-16T17:06:45.250911+0000","last_became_active":"2024-04-16T17:03:24.977434+0000","last_became_peered":"2024-04-16T17:03:24.977434+0000","last_unstale":"2024-04-16T17:06:45.250911+0000","last_undegraded":"2024-04-16T17:06:45.250911+0000","last_fullsized":"2024-04-16T17:06:45.250911+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:46:35.670357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"92'8","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.240206+0000","last_change":"2024-04-16T17:03:24.992225+0000","last_active":"2024-04-16T17:06:45.240206+0000","last_peered":"2024-04-16T17:06:45.240206+0000","last_clean":"2024-04-16T17:06:45.240206+0000","last_became_active":"2024-04-16T17:03:24.991904+0000","last_became_peered":"2024-04-16T17:03:24.991904+0000","last_unstale":"2024-04-16T17:06:45.240206+0000","last_undegraded":"2024-04-16T17:06:45.240206+0000","last_fullsized":"2024-04-16T17:06:45.240206+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:01:09.710698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"92'10","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.450979+0000","last_change":"2024-04-16T17:03:24.981322+0000","last_active":"2024-04-16T17:06:45.450979+0000","last_peered":"2024-04-16T17:06:45.450979+0000","last_clean":"2024-04-16T17:06:45.450979+0000","last_became_active":"2024-04-16T17:03:24.980920+0000","last_became_peered":"2024-04-16T17:03:24.980920+0000","last_unstale":"2024-04-16T17:06:45.450979+0000","last_undegraded":"2024-04-16T17:06:45.450979+0000","last_fullsized":"2024-04-16T17:06:45.450979+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:07:59.378512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"92'11","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.443916+0000","last_change":"2024-04-16T17:03:24.990117+0000","last_active":"2024-04-16T17:06:45.443916+0000","last_peered":"2024-04-16T17:06:45.443916+0000","last_clean":"2024-04-16T17:06:45.443916+0000","last_became_active":"2024-04-16T17:03:24.989734+0000","last_became_peered":"2024-04-16T17:03:24.989734+0000","last_unstale":"2024-04-16T17:06:45.443916+0000","last_undegraded":"2024-04-16T17:06:45.443916+0000","last_fullsized":"2024-04-16T17:06:45.443916+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:36:55.000702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"92'8","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.557602+0000","last_change":"2024-04-16T17:03:24.970032+0000","last_active":"2024-04-16T17:06:45.557602+0000","last_peered":"2024-04-16T17:06:45.557602+0000","last_clean":"2024-04-16T17:06:45.557602+0000","last_became_active":"2024-04-16T17:03:24.968608+0000","last_became_peered":"2024-04-16T17:03:24.968608+0000","last_unstale":"2024-04-16T17:06:45.557602+0000","last_undegraded":"2024-04-16T17:06:45.557602+0000","last_fullsized":"2024-04-16T17:06:45.557602+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:24:03.918418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":1771104,"data_compressed":108294,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":599984,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":1167,"num_read_kb":697,"num_write":298,"num_write_kb":1448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":616,"num_keys_recovered":20,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":1799952,"data_compressed":80793,"data_compressed_allocated":110592,"data_compressed_original":1572864,"omap_allocated":0,"internal_metadata":0},"log_size":236,"ondisk_log_size":236,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351029,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1032192,"data_stored":522666,"data_compressed":26684,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612668,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32360,"kb_used_data":1184,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712792,"statfs":{"total":95995035648,"available":95961899008,"internally_reserved":0,"allocated":1212416,"data_stored":1326097,"data_compressed":73579,"data_compressed_allocated":98304,"data_compressed_original":802816,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776192,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32236,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712916,"statfs":{"total":95995035648,"available":95962025984,"internally_reserved":0,"allocated":1081344,"data_stored":925950,"data_compressed":48842,"data_compressed_allocated":65536,"data_compressed_original":475136,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037831,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32232,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712920,"statfs":{"total":95995035648,"available":95962030080,"internally_reserved":0,"allocated":1077248,"data_stored":925288,"data_compressed":48842,"data_compressed_allocated":65536,"data_compressed_original":475136,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299469,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36420,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708732,"statfs":{"total":95995035648,"available":95957741568,"internally_reserved":0,"allocated":1105920,"data_stored":1130719,"data_compressed":65794,"data_compressed_allocated":86016,"data_compressed_original":671744,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462994,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32192,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712960,"statfs":{"total":95995035648,"available":95962071040,"internally_reserved":0,"allocated":1036288,"data_stored":527910,"data_compressed":29696,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574959,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32264,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712888,"statfs":{"total":95995035648,"available":95961997312,"internally_reserved":0,"allocated":1110016,"data_stored":597413,"data_compressed":31325,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738483,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32208,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712944,"statfs":{"total":95995035648,"available":95962054656,"internally_reserved":0,"allocated":1048576,"data_stored":519432,"data_compressed":26684,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411433,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28120,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717032,"statfs":{"total":95995035648,"available":95966240768,"internally_reserved":0,"allocated":1126400,"data_stored":599070,"data_compressed":31325,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149795,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32244,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712908,"statfs":{"total":95995035648,"available":95962017792,"internally_reserved":0,"allocated":1093632,"data_stored":1052920,"data_compressed":54998,"data_compressed_allocated":73728,"data_compressed_original":598016,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888157,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1048576,"data_stored":534184,"data_compressed":29696,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724633,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28124,"kb_used_data":1108,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717028,"statfs":{"total":95995035648,"available":95966236672,"internally_reserved":0,"allocated":1134592,"data_stored":1256283,"data_compressed":71950,"data_compressed_allocated":94208,"data_compressed_original":794624,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36098,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36098,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36098,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":69593,"data_compressed":1629,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":71250,"data_compressed":1629,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":525100,"data_compressed":25302,"data_compressed_allocated":32768,"data_compressed_original":516096,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":2268,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":133999,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":4339,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":393372,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":394034,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":126976,"data_stored":203813,"data_compressed":7785,"data_compressed_allocated":12288,"data_compressed_original":131072,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-16T17:06:50.742 INFO:tasks.fwd_scrub.ceph_manager:clean! 2024-04-16T17:06:50.742 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-16T17:06:50.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:06:50 smithi104 ceph-mon[36457]: mgrmap e24: x(active, since 4s), standbys: z, y 2024-04-16T17:06:50.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:06:50 smithi104 ceph-mon[36457]: from='client.? 172.21.15.45:0/3249440094' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:06:50.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:06:50 smithi104 ceph-mon[36457]: from='client.25553 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-16T17:06:50.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:06:50 smithi104 ceph-mon[36457]: from='mgr.25411 ' entity='mgr.x' 2024-04-16T17:06:50.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:06:50 smithi104 ceph-mon[36457]: from='mgr.25411 ' entity='mgr.x' 2024-04-16T17:06:50.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:06:50 smithi045 ceph-mon[31676]: mgrmap e24: x(active, since 4s), standbys: z, y 2024-04-16T17:06:50.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:06:50 smithi045 ceph-mon[31676]: from='client.? 172.21.15.45:0/3249440094' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:06:50.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:06:50 smithi045 ceph-mon[31676]: from='client.25553 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-16T17:06:50.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:06:50 smithi045 ceph-mon[31676]: from='mgr.25411 ' entity='mgr.x' 2024-04-16T17:06:50.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:06:50 smithi045 ceph-mon[31676]: from='mgr.25411 ' entity='mgr.x' 2024-04-16T17:06:50.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:50.988+0000 7f1760000640 1 -- 172.21.15.45:0/1420401903 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1758105470 msgr2=0x7f17581058f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:50.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:50.988+0000 7f1760000640 1 --2- 172.21.15.45:0/1420401903 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1758105470 0x7f17581058f0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f1744009900 tx=0x7f174402bc30 comp rx=0 tx=0).stop 2024-04-16T17:06:50.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:50.988+0000 7f1760000640 1 -- 172.21.15.45:0/1420401903 shutdown_connections 2024-04-16T17:06:50.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:50.988+0000 7f1760000640 1 --2- 172.21.15.45:0/1420401903 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1758105e30 0x7f175810c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:50.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:50.988+0000 7f1760000640 1 --2- 172.21.15.45:0/1420401903 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1758105470 0x7f17581058f0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:50.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:50.988+0000 7f1760000640 1 --2- 172.21.15.45:0/1420401903 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1758104280 0x7f17581046a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:50.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:50.988+0000 7f1760000640 1 -- 172.21.15.45:0/1420401903 >> 172.21.15.45:0/1420401903 conn(0x7f17580ff7f0 msgr2=0x7f1758101c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:06:50.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:50.988+0000 7f1760000640 1 -- 172.21.15.45:0/1420401903 shutdown_connections 2024-04-16T17:06:50.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:50.988+0000 7f1760000640 1 -- 172.21.15.45:0/1420401903 wait complete. 2024-04-16T17:06:50.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:50.989+0000 7f1760000640 1 Processor -- start 2024-04-16T17:06:50.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:50.989+0000 7f1760000640 1 -- start start 2024-04-16T17:06:50.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:50.990+0000 7f1760000640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1758104280 0x7f175819a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:50.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:50.990+0000 7f1760000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1758105470 0x7f175819aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:50.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:50.990+0000 7f1760000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1758105e30 0x7f17581a1b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:50.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:50.990+0000 7f1760000640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f175810ff60 con 0x7f1758105e30 2024-04-16T17:06:50.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:50.990+0000 7f1760000640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f175810fde0 con 0x7f1758104280 2024-04-16T17:06:50.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:50.990+0000 7f1760000640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f17581100e0 con 0x7f1758105470 2024-04-16T17:06:50.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:50.990+0000 7f175d600640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1758104280 0x7f175819a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:50.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:50.990+0000 7f175d600640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1758104280 0x7f175819a560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:33364/0 (socket says 172.21.15.45:33364) 2024-04-16T17:06:50.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:50.990+0000 7f175d600640 1 -- 172.21.15.45:0/2258170731 learned_addr learned my addr 172.21.15.45:0/2258170731 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:06:50.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:50.990+0000 7f175cc00640 1 --2- 172.21.15.45:0/2258170731 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1758105470 0x7f175819aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:50.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:50.990+0000 7f175e000640 1 --2- 172.21.15.45:0/2258170731 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1758105e30 0x7f17581a1b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:50.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:50.990+0000 7f175d600640 1 -- 172.21.15.45:0/2258170731 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1758105470 msgr2=0x7f175819aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:50.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:50.990+0000 7f175d600640 1 --2- 172.21.15.45:0/2258170731 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1758105470 0x7f175819aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:50.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:50.990+0000 7f175d600640 1 -- 172.21.15.45:0/2258170731 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1758105e30 msgr2=0x7f17581a1b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:50.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:50.990+0000 7f175d600640 1 --2- 172.21.15.45:0/2258170731 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1758105e30 0x7f17581a1b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:50.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:50.990+0000 7f175d600640 1 -- 172.21.15.45:0/2258170731 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17581a2180 con 0x7f1758104280 2024-04-16T17:06:50.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:50.991+0000 7f175d600640 1 --2- 172.21.15.45:0/2258170731 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1758104280 0x7f175819a560 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f174c00e9c0 tx=0x7f174c00ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:06:50.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:50.991+0000 7f1756000640 1 -- 172.21.15.45:0/2258170731 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f174c00cde0 con 0x7f1758104280 2024-04-16T17:06:50.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:50.991+0000 7f1760000640 1 -- 172.21.15.45:0/2258170731 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17581a2470 con 0x7f1758104280 2024-04-16T17:06:50.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:50.991+0000 7f1756000640 1 -- 172.21.15.45:0/2258170731 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f174c010910 con 0x7f1758104280 2024-04-16T17:06:50.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:50.991+0000 7f1756000640 1 -- 172.21.15.45:0/2258170731 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f174c018a00 con 0x7f1758104280 2024-04-16T17:06:50.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:50.991+0000 7f1760000640 1 -- 172.21.15.45:0/2258170731 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17581a2990 con 0x7f1758104280 2024-04-16T17:06:50.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:50.992+0000 7f1760000640 1 -- 172.21.15.45:0/2258170731 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f175810b5a0 con 0x7f1758104280 2024-04-16T17:06:50.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:50.994+0000 7f1756000640 1 -- 172.21.15.45:0/2258170731 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 24) ==== 138822+0+0 (secure 0 0 0) 0x7f174c01f020 con 0x7f1758104280 2024-04-16T17:06:50.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:50.996+0000 7f1756000640 1 --2- 172.21.15.45:0/2258170731 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f17400a3db0 0x7f17400a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:50.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:50.996+0000 7f1756000640 1 -- 172.21.15.45:0/2258170731 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f174c0d0f30 con 0x7f1758104280 2024-04-16T17:06:50.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:50.996+0000 7f175cc00640 1 --2- 172.21.15.45:0/2258170731 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f17400a3db0 0x7f17400a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:50.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:50.997+0000 7f175cc00640 1 --2- 172.21.15.45:0/2258170731 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f17400a3db0 0x7f17400a6270 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f174402c240 tx=0x7f174402df90 comp rx=0 tx=0).ready entity=mgr.25411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:06:51.000 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:51.000+0000 7f1756000640 1 -- 172.21.15.45:0/2258170731 <== mon.2 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f174c09d0b0 con 0x7f1758104280 2024-04-16T17:06:51.055 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:06:50 smithi117 ceph-mon[36730]: mgrmap e24: x(active, since 4s), standbys: z, y 2024-04-16T17:06:51.055 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:06:50 smithi117 ceph-mon[36730]: from='client.? 172.21.15.45:0/3249440094' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:06:51.055 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:06:50 smithi117 ceph-mon[36730]: from='client.25553 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-16T17:06:51.055 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:06:50 smithi117 ceph-mon[36730]: from='mgr.25411 ' entity='mgr.x' 2024-04-16T17:06:51.055 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:06:50 smithi117 ceph-mon[36730]: from='mgr.25411 ' entity='mgr.x' 2024-04-16T17:06:51.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:51.226+0000 7f1760000640 1 -- 172.21.15.45:0/2258170731 --> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f1758062440 con 0x7f17400a3db0 2024-04-16T17:06:51.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:51.238+0000 7f1756000640 1 -- 172.21.15.45:0/2258170731 <== mgr.25411 v2:172.21.15.45:6800/751128195 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+339706 (secure 0 0 0) 0x7f1758062440 con 0x7f17400a3db0 2024-04-16T17:06:51.243 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-16T17:06:51.244 INFO:teuthology.orchestra.run.smithi045.stderr:dumped all 2024-04-16T17:06:51.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:51.243+0000 7f1760000640 1 -- 172.21.15.45:0/2258170731 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f17400a3db0 msgr2=0x7f17400a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:51.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:51.243+0000 7f1760000640 1 --2- 172.21.15.45:0/2258170731 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f17400a3db0 0x7f17400a6270 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f174402c240 tx=0x7f174402df90 comp rx=0 tx=0).stop 2024-04-16T17:06:51.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:51.243+0000 7f1760000640 1 -- 172.21.15.45:0/2258170731 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1758104280 msgr2=0x7f175819a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:51.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:51.243+0000 7f1760000640 1 --2- 172.21.15.45:0/2258170731 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1758104280 0x7f175819a560 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f174c00e9c0 tx=0x7f174c00ee80 comp rx=0 tx=0).stop 2024-04-16T17:06:51.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:51.243+0000 7f1760000640 1 -- 172.21.15.45:0/2258170731 shutdown_connections 2024-04-16T17:06:51.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:51.243+0000 7f1760000640 1 --2- 172.21.15.45:0/2258170731 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f17400a3db0 0x7f17400a6270 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:51.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:51.243+0000 7f1760000640 1 --2- 172.21.15.45:0/2258170731 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1758105470 0x7f175819aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:51.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:51.243+0000 7f1760000640 1 --2- 172.21.15.45:0/2258170731 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1758105e30 0x7f17581a1b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:51.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:51.243+0000 7f1760000640 1 --2- 172.21.15.45:0/2258170731 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1758104280 0x7f175819a560 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:51.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:51.243+0000 7f1760000640 1 -- 172.21.15.45:0/2258170731 >> 172.21.15.45:0/2258170731 conn(0x7f17580ff7f0 msgr2=0x7f1758108770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:06:51.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:51.244+0000 7f1760000640 1 -- 172.21.15.45:0/2258170731 shutdown_connections 2024-04-16T17:06:51.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:51.244+0000 7f1760000640 1 -- 172.21.15.45:0/2258170731 wait complete. 2024-04-16T17:06:51.264 INFO:teuthology.orchestra.run.smithi045.stdout:{"pg_ready":true,"pg_map":{"version":6,"stamp":"2024-04-16T17:06:51.124870+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1190586,"num_objects":66,"num_object_clones":0,"num_object_copies":200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":1308,"num_read_kb":816,"num_write":483,"num_write_kb":4389,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":1181469,"num_keys_recovered":20,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":876,"ondisk_log_size":876,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":374484,"kb_used_data":12812,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124567340,"statfs":{"total":1151940427776,"available":1151556956160,"internally_reserved":0,"allocated":13119488,"data_stored":9927798,"data_compressed":542427,"data_compressed_allocated":733184,"data_compressed_original":4317184,"omap_allocated":19143,"internal_metadata":370193721},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.986793"},"pg_stats":[{"pgid":"3.3f","version":"92'8","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.557723+0000","last_change":"2024-04-16T17:03:24.989508+0000","last_active":"2024-04-16T17:06:45.557723+0000","last_peered":"2024-04-16T17:06:45.557723+0000","last_clean":"2024-04-16T17:06:45.557723+0000","last_became_active":"2024-04-16T17:03:24.989178+0000","last_became_peered":"2024-04-16T17:03:24.989178+0000","last_unstale":"2024-04-16T17:06:45.557723+0000","last_undegraded":"2024-04-16T17:06:45.557723+0000","last_fullsized":"2024-04-16T17:06:45.557723+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:18:56.488358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"92'4","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.577296+0000","last_change":"2024-04-16T17:03:24.986378+0000","last_active":"2024-04-16T17:06:45.577296+0000","last_peered":"2024-04-16T17:06:45.577296+0000","last_clean":"2024-04-16T17:06:45.577296+0000","last_became_active":"2024-04-16T17:03:24.986030+0000","last_became_peered":"2024-04-16T17:03:24.986030+0000","last_unstale":"2024-04-16T17:06:45.577296+0000","last_undegraded":"2024-04-16T17:06:45.577296+0000","last_fullsized":"2024-04-16T17:06:45.577296+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:21:14.071109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"92'7","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.577316+0000","last_change":"2024-04-16T17:03:24.976232+0000","last_active":"2024-04-16T17:06:45.577316+0000","last_peered":"2024-04-16T17:06:45.577316+0000","last_clean":"2024-04-16T17:06:45.577316+0000","last_became_active":"2024-04-16T17:03:24.975873+0000","last_became_peered":"2024-04-16T17:03:24.975873+0000","last_unstale":"2024-04-16T17:06:45.577316+0000","last_undegraded":"2024-04-16T17:06:45.577316+0000","last_fullsized":"2024-04-16T17:06:45.577316+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:07:53.523101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"92'8","reported_seq":26,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.102010+0000","last_change":"2024-04-16T17:04:21.517830+0000","last_active":"2024-04-16T17:06:45.102010+0000","last_peered":"2024-04-16T17:06:45.102010+0000","last_clean":"2024-04-16T17:06:45.102010+0000","last_became_active":"2024-04-16T17:04:21.516749+0000","last_became_peered":"2024-04-16T17:04:21.516749+0000","last_unstale":"2024-04-16T17:06:45.102010+0000","last_undegraded":"2024-04-16T17:06:45.102010+0000","last_fullsized":"2024-04-16T17:06:45.102010+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:02:33.344034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"92'4","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.239444+0000","last_change":"2024-04-16T17:03:24.985609+0000","last_active":"2024-04-16T17:06:45.239444+0000","last_peered":"2024-04-16T17:06:45.239444+0000","last_clean":"2024-04-16T17:06:45.239444+0000","last_became_active":"2024-04-16T17:03:24.985239+0000","last_became_peered":"2024-04-16T17:03:24.985239+0000","last_unstale":"2024-04-16T17:06:45.239444+0000","last_undegraded":"2024-04-16T17:06:45.239444+0000","last_fullsized":"2024-04-16T17:06:45.239444+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:55:58.619210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"92'6","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.227011+0000","last_change":"2024-04-16T17:03:24.966862+0000","last_active":"2024-04-16T17:06:45.227011+0000","last_peered":"2024-04-16T17:06:45.227011+0000","last_clean":"2024-04-16T17:06:45.227011+0000","last_became_active":"2024-04-16T17:03:24.966234+0000","last_became_peered":"2024-04-16T17:03:24.966234+0000","last_unstale":"2024-04-16T17:06:45.227011+0000","last_undegraded":"2024-04-16T17:06:45.227011+0000","last_fullsized":"2024-04-16T17:06:45.227011+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:21:07.018281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"92'5","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.442957+0000","last_change":"2024-04-16T17:03:24.971219+0000","last_active":"2024-04-16T17:06:45.442957+0000","last_peered":"2024-04-16T17:06:45.442957+0000","last_clean":"2024-04-16T17:06:45.442957+0000","last_became_active":"2024-04-16T17:03:24.970876+0000","last_became_peered":"2024-04-16T17:03:24.970876+0000","last_unstale":"2024-04-16T17:06:45.442957+0000","last_undegraded":"2024-04-16T17:06:45.442957+0000","last_fullsized":"2024-04-16T17:06:45.442957+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:44:16.311167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"92'9","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.239719+0000","last_change":"2024-04-16T17:03:24.994694+0000","last_active":"2024-04-16T17:06:45.239719+0000","last_peered":"2024-04-16T17:06:45.239719+0000","last_clean":"2024-04-16T17:06:45.239719+0000","last_became_active":"2024-04-16T17:03:24.994315+0000","last_became_peered":"2024-04-16T17:03:24.994315+0000","last_unstale":"2024-04-16T17:06:45.239719+0000","last_undegraded":"2024-04-16T17:06:45.239719+0000","last_fullsized":"2024-04-16T17:06:45.239719+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:04:59.543816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"92'11","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.451196+0000","last_change":"2024-04-16T17:03:24.964654+0000","last_active":"2024-04-16T17:06:45.451196+0000","last_peered":"2024-04-16T17:06:45.451196+0000","last_clean":"2024-04-16T17:06:45.451196+0000","last_became_active":"2024-04-16T17:03:24.964349+0000","last_became_peered":"2024-04-16T17:03:24.964349+0000","last_unstale":"2024-04-16T17:06:45.451196+0000","last_undegraded":"2024-04-16T17:06:45.451196+0000","last_fullsized":"2024-04-16T17:06:45.451196+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:38:55.880948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"92'16","reported_seq":48,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.226821+0000","last_change":"2024-04-16T17:03:24.966364+0000","last_active":"2024-04-16T17:06:45.226821+0000","last_peered":"2024-04-16T17:06:45.226821+0000","last_clean":"2024-04-16T17:06:45.226821+0000","last_became_active":"2024-04-16T17:03:24.966061+0000","last_became_peered":"2024-04-16T17:03:24.966061+0000","last_unstale":"2024-04-16T17:06:45.226821+0000","last_undegraded":"2024-04-16T17:06:45.226821+0000","last_fullsized":"2024-04-16T17:06:45.226821+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:18:12.642602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"92'6","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.443443+0000","last_change":"2024-04-16T17:03:24.969874+0000","last_active":"2024-04-16T17:06:45.443443+0000","last_peered":"2024-04-16T17:06:45.443443+0000","last_clean":"2024-04-16T17:06:45.443443+0000","last_became_active":"2024-04-16T17:03:24.969523+0000","last_became_peered":"2024-04-16T17:03:24.969523+0000","last_unstale":"2024-04-16T17:06:45.443443+0000","last_undegraded":"2024-04-16T17:06:45.443443+0000","last_fullsized":"2024-04-16T17:06:45.443443+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:17:49.704855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"92'7","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.105227+0000","last_change":"2024-04-16T17:03:24.968660+0000","last_active":"2024-04-16T17:06:45.105227+0000","last_peered":"2024-04-16T17:06:45.105227+0000","last_clean":"2024-04-16T17:06:45.105227+0000","last_became_active":"2024-04-16T17:03:24.968348+0000","last_became_peered":"2024-04-16T17:03:24.968348+0000","last_unstale":"2024-04-16T17:06:45.105227+0000","last_undegraded":"2024-04-16T17:06:45.105227+0000","last_fullsized":"2024-04-16T17:06:45.105227+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:06:06.319391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"92'10","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.444115+0000","last_change":"2024-04-16T17:03:24.974741+0000","last_active":"2024-04-16T17:06:45.444115+0000","last_peered":"2024-04-16T17:06:45.444115+0000","last_clean":"2024-04-16T17:06:45.444115+0000","last_became_active":"2024-04-16T17:03:24.973568+0000","last_became_peered":"2024-04-16T17:03:24.973568+0000","last_unstale":"2024-04-16T17:06:45.444115+0000","last_undegraded":"2024-04-16T17:06:45.444115+0000","last_fullsized":"2024-04-16T17:06:45.444115+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:30:49.325469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"92'7","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.576655+0000","last_change":"2024-04-16T17:03:24.981629+0000","last_active":"2024-04-16T17:06:45.576655+0000","last_peered":"2024-04-16T17:06:45.576655+0000","last_clean":"2024-04-16T17:06:45.576655+0000","last_became_active":"2024-04-16T17:03:24.981111+0000","last_became_peered":"2024-04-16T17:03:24.981111+0000","last_unstale":"2024-04-16T17:06:45.576655+0000","last_undegraded":"2024-04-16T17:06:45.576655+0000","last_fullsized":"2024-04-16T17:06:45.576655+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:41:29.641610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"92'7","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.576489+0000","last_change":"2024-04-16T17:03:24.977890+0000","last_active":"2024-04-16T17:06:45.576489+0000","last_peered":"2024-04-16T17:06:45.576489+0000","last_clean":"2024-04-16T17:06:45.576489+0000","last_became_active":"2024-04-16T17:03:24.976500+0000","last_became_peered":"2024-04-16T17:03:24.976500+0000","last_unstale":"2024-04-16T17:06:45.576489+0000","last_undegraded":"2024-04-16T17:06:45.576489+0000","last_fullsized":"2024-04-16T17:06:45.576489+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:17:51.604701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"92'9","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.450479+0000","last_change":"2024-04-16T17:03:24.976085+0000","last_active":"2024-04-16T17:06:45.450479+0000","last_peered":"2024-04-16T17:06:45.450479+0000","last_clean":"2024-04-16T17:06:45.450479+0000","last_became_active":"2024-04-16T17:03:24.975804+0000","last_became_peered":"2024-04-16T17:03:24.975804+0000","last_unstale":"2024-04-16T17:06:45.450479+0000","last_undegraded":"2024-04-16T17:06:45.450479+0000","last_fullsized":"2024-04-16T17:06:45.450479+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:19:13.706889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"92'8","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.557548+0000","last_change":"2024-04-16T17:03:24.989456+0000","last_active":"2024-04-16T17:06:45.557548+0000","last_peered":"2024-04-16T17:06:45.557548+0000","last_clean":"2024-04-16T17:06:45.557548+0000","last_became_active":"2024-04-16T17:03:24.989161+0000","last_became_peered":"2024-04-16T17:03:24.989161+0000","last_unstale":"2024-04-16T17:06:45.557548+0000","last_undegraded":"2024-04-16T17:06:45.557548+0000","last_fullsized":"2024-04-16T17:06:45.557548+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:52:59.333625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"92'10","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.102702+0000","last_change":"2024-04-16T17:03:24.963428+0000","last_active":"2024-04-16T17:06:45.102702+0000","last_peered":"2024-04-16T17:06:45.102702+0000","last_clean":"2024-04-16T17:06:45.102702+0000","last_became_active":"2024-04-16T17:03:24.962523+0000","last_became_peered":"2024-04-16T17:03:24.962523+0000","last_unstale":"2024-04-16T17:06:45.102702+0000","last_undegraded":"2024-04-16T17:06:45.102702+0000","last_fullsized":"2024-04-16T17:06:45.102702+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:22:19.523901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"92'7","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.105614+0000","last_change":"2024-04-16T17:03:24.981122+0000","last_active":"2024-04-16T17:06:45.105614+0000","last_peered":"2024-04-16T17:06:45.105614+0000","last_clean":"2024-04-16T17:06:45.105614+0000","last_became_active":"2024-04-16T17:03:24.980533+0000","last_became_peered":"2024-04-16T17:03:24.980533+0000","last_unstale":"2024-04-16T17:06:45.105614+0000","last_undegraded":"2024-04-16T17:06:45.105614+0000","last_fullsized":"2024-04-16T17:06:45.105614+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:51:08.580965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.105345+0000","last_change":"2024-04-16T17:03:29.257239+0000","last_active":"2024-04-16T17:06:45.105345+0000","last_peered":"2024-04-16T17:06:45.105345+0000","last_clean":"2024-04-16T17:06:45.105345+0000","last_became_active":"2024-04-16T17:03:29.256827+0000","last_became_peered":"2024-04-16T17:03:29.256827+0000","last_unstale":"2024-04-16T17:06:45.105345+0000","last_undegraded":"2024-04-16T17:06:45.105345+0000","last_fullsized":"2024-04-16T17:06:45.105345+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:57:28.373386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"90'9","reported_seq":60,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.102601+0000","last_change":"2024-04-16T17:04:21.896665+0000","last_active":"2024-04-16T17:06:45.102601+0000","last_peered":"2024-04-16T17:06:45.102601+0000","last_clean":"2024-04-16T17:06:45.102601+0000","last_became_active":"2024-04-16T17:04:21.882573+0000","last_became_peered":"2024-04-16T17:04:21.882573+0000","last_unstale":"2024-04-16T17:06:45.102601+0000","last_undegraded":"2024-04-16T17:06:45.102601+0000","last_fullsized":"2024-04-16T17:06:45.102601+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:37:07.210368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,1],"acting":[9,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"92'5","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.239912+0000","last_change":"2024-04-16T17:03:24.994921+0000","last_active":"2024-04-16T17:06:45.239912+0000","last_peered":"2024-04-16T17:06:45.239912+0000","last_clean":"2024-04-16T17:06:45.239912+0000","last_became_active":"2024-04-16T17:03:24.994412+0000","last_became_peered":"2024-04-16T17:03:24.994412+0000","last_unstale":"2024-04-16T17:06:45.239912+0000","last_undegraded":"2024-04-16T17:06:45.239912+0000","last_fullsized":"2024-04-16T17:06:45.239912+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:11:40.748198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.104669+0000","last_change":"2024-04-16T17:03:29.259884+0000","last_active":"2024-04-16T17:06:45.104669+0000","last_peered":"2024-04-16T17:06:45.104669+0000","last_clean":"2024-04-16T17:06:45.104669+0000","last_became_active":"2024-04-16T17:03:29.259319+0000","last_became_peered":"2024-04-16T17:03:29.259319+0000","last_unstale":"2024-04-16T17:06:45.104669+0000","last_undegraded":"2024-04-16T17:06:45.104669+0000","last_fullsized":"2024-04-16T17:06:45.104669+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:05:18.244682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"90'2","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.241406+0000","last_change":"2024-04-16T17:03:24.179308+0000","last_active":"2024-04-16T17:06:45.241406+0000","last_peered":"2024-04-16T17:06:45.241406+0000","last_clean":"2024-04-16T17:06:45.241406+0000","last_became_active":"2024-04-16T17:03:24.178894+0000","last_became_peered":"2024-04-16T17:03:24.178894+0000","last_unstale":"2024-04-16T17:06:45.241406+0000","last_undegraded":"2024-04-16T17:06:45.241406+0000","last_fullsized":"2024-04-16T17:06:45.241406+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:17:46.831871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"92'6","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.577690+0000","last_change":"2024-04-16T17:03:24.982391+0000","last_active":"2024-04-16T17:06:45.577690+0000","last_peered":"2024-04-16T17:06:45.577690+0000","last_clean":"2024-04-16T17:06:45.577690+0000","last_became_active":"2024-04-16T17:03:24.981761+0000","last_became_peered":"2024-04-16T17:03:24.981761+0000","last_unstale":"2024-04-16T17:06:45.577690+0000","last_undegraded":"2024-04-16T17:06:45.577690+0000","last_fullsized":"2024-04-16T17:06:45.577690+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:40:57.657170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.104794+0000","last_change":"2024-04-16T17:03:29.259705+0000","last_active":"2024-04-16T17:06:45.104794+0000","last_peered":"2024-04-16T17:06:45.104794+0000","last_clean":"2024-04-16T17:06:45.104794+0000","last_became_active":"2024-04-16T17:03:29.259336+0000","last_became_peered":"2024-04-16T17:03:29.259336+0000","last_unstale":"2024-04-16T17:06:45.104794+0000","last_undegraded":"2024-04-16T17:06:45.104794+0000","last_fullsized":"2024-04-16T17:06:45.104794+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:53:29.042933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"90'4","reported_seq":50,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.239845+0000","last_change":"2024-04-16T17:04:22.153413+0000","last_active":"2024-04-16T17:06:45.239845+0000","last_peered":"2024-04-16T17:06:45.239845+0000","last_clean":"2024-04-16T17:06:45.239845+0000","last_became_active":"2024-04-16T17:04:22.148972+0000","last_became_peered":"2024-04-16T17:04:22.148972+0000","last_unstale":"2024-04-16T17:06:45.239845+0000","last_undegraded":"2024-04-16T17:06:45.239845+0000","last_fullsized":"2024-04-16T17:06:45.239845+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:05:57.513052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"92'9","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.450605+0000","last_change":"2024-04-16T17:03:24.973205+0000","last_active":"2024-04-16T17:06:45.450605+0000","last_peered":"2024-04-16T17:06:45.450605+0000","last_clean":"2024-04-16T17:06:45.450605+0000","last_became_active":"2024-04-16T17:03:24.972863+0000","last_became_peered":"2024-04-16T17:03:24.972863+0000","last_unstale":"2024-04-16T17:06:45.450605+0000","last_undegraded":"2024-04-16T17:06:45.450605+0000","last_fullsized":"2024-04-16T17:06:45.450605+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:28:29.005783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.104818+0000","last_change":"2024-04-16T17:03:29.262930+0000","last_active":"2024-04-16T17:06:45.104818+0000","last_peered":"2024-04-16T17:06:45.104818+0000","last_clean":"2024-04-16T17:06:45.104818+0000","last_became_active":"2024-04-16T17:03:29.262621+0000","last_became_peered":"2024-04-16T17:03:29.262621+0000","last_unstale":"2024-04-16T17:06:45.104818+0000","last_undegraded":"2024-04-16T17:06:45.104818+0000","last_fullsized":"2024-04-16T17:06:45.104818+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:22:31.555143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.239656+0000","last_change":"2024-04-16T17:03:24.150503+0000","last_active":"2024-04-16T17:06:45.239656+0000","last_peered":"2024-04-16T17:06:45.239656+0000","last_clean":"2024-04-16T17:06:45.239656+0000","last_became_active":"2024-04-16T17:03:24.150021+0000","last_became_peered":"2024-04-16T17:03:24.150021+0000","last_unstale":"2024-04-16T17:06:45.239656+0000","last_undegraded":"2024-04-16T17:06:45.239656+0000","last_fullsized":"2024-04-16T17:06:45.239656+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:21:36.119497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"92'15","reported_seq":47,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.251429+0000","last_change":"2024-04-16T17:03:24.965049+0000","last_active":"2024-04-16T17:06:45.251429+0000","last_peered":"2024-04-16T17:06:45.251429+0000","last_clean":"2024-04-16T17:06:45.251429+0000","last_became_active":"2024-04-16T17:03:24.963120+0000","last_became_peered":"2024-04-16T17:03:24.963120+0000","last_unstale":"2024-04-16T17:06:45.251429+0000","last_undegraded":"2024-04-16T17:06:45.251429+0000","last_fullsized":"2024-04-16T17:06:45.251429+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:17:49.725113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.105640+0000","last_change":"2024-04-16T17:03:29.247013+0000","last_active":"2024-04-16T17:06:45.105640+0000","last_peered":"2024-04-16T17:06:45.105640+0000","last_clean":"2024-04-16T17:06:45.105640+0000","last_became_active":"2024-04-16T17:03:29.246477+0000","last_became_peered":"2024-04-16T17:03:29.246477+0000","last_unstale":"2024-04-16T17:06:45.105640+0000","last_undegraded":"2024-04-16T17:06:45.105640+0000","last_fullsized":"2024-04-16T17:06:45.105640+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:52:19.525076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"95'8","reported_seq":220,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.629994+0000","last_change":"2024-04-16T17:03:24.155333+0000","last_active":"2024-04-16T17:06:45.629994+0000","last_peered":"2024-04-16T17:06:45.629994+0000","last_clean":"2024-04-16T17:06:45.629994+0000","last_became_active":"2024-04-16T17:03:24.154848+0000","last_became_peered":"2024-04-16T17:03:24.154848+0000","last_unstale":"2024-04-16T17:06:45.629994+0000","last_undegraded":"2024-04-16T17:06:45.629994+0000","last_fullsized":"2024-04-16T17:06:45.629994+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:03:34.769573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1545,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":178,"num_read_kb":5,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"92'7","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.239446+0000","last_change":"2024-04-16T17:03:24.967627+0000","last_active":"2024-04-16T17:06:45.239446+0000","last_peered":"2024-04-16T17:06:45.239446+0000","last_clean":"2024-04-16T17:06:45.239446+0000","last_became_active":"2024-04-16T17:03:24.967264+0000","last_became_peered":"2024-04-16T17:03:24.967264+0000","last_unstale":"2024-04-16T17:06:45.239446+0000","last_undegraded":"2024-04-16T17:06:45.239446+0000","last_fullsized":"2024-04-16T17:06:45.239446+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:35:29.182374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.105252+0000","last_change":"2024-04-16T17:03:29.264625+0000","last_active":"2024-04-16T17:06:45.105252+0000","last_peered":"2024-04-16T17:06:45.105252+0000","last_clean":"2024-04-16T17:06:45.105252+0000","last_became_active":"2024-04-16T17:03:29.264114+0000","last_became_peered":"2024-04-16T17:03:29.264114+0000","last_unstale":"2024-04-16T17:06:45.105252+0000","last_undegraded":"2024-04-16T17:06:45.105252+0000","last_fullsized":"2024-04-16T17:06:45.105252+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:14:49.449980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"90'3","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.226892+0000","last_change":"2024-04-16T17:03:24.160457+0000","last_active":"2024-04-16T17:06:45.226892+0000","last_peered":"2024-04-16T17:06:45.226892+0000","last_clean":"2024-04-16T17:06:45.226892+0000","last_became_active":"2024-04-16T17:03:24.160195+0000","last_became_peered":"2024-04-16T17:03:24.160195+0000","last_unstale":"2024-04-16T17:06:45.226892+0000","last_undegraded":"2024-04-16T17:06:45.226892+0000","last_fullsized":"2024-04-16T17:06:45.226892+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:23:07.064539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"92'8","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.251287+0000","last_change":"2024-04-16T17:03:24.991249+0000","last_active":"2024-04-16T17:06:45.251287+0000","last_peered":"2024-04-16T17:06:45.251287+0000","last_clean":"2024-04-16T17:06:45.251287+0000","last_became_active":"2024-04-16T17:03:24.990927+0000","last_became_peered":"2024-04-16T17:03:24.990927+0000","last_unstale":"2024-04-16T17:06:45.251287+0000","last_undegraded":"2024-04-16T17:06:45.251287+0000","last_fullsized":"2024-04-16T17:06:45.251287+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:36:39.975616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.557483+0000","last_change":"2024-04-16T17:03:29.253704+0000","last_active":"2024-04-16T17:06:45.557483+0000","last_peered":"2024-04-16T17:06:45.557483+0000","last_clean":"2024-04-16T17:06:45.557483+0000","last_became_active":"2024-04-16T17:03:29.253334+0000","last_became_peered":"2024-04-16T17:03:29.253334+0000","last_unstale":"2024-04-16T17:06:45.557483+0000","last_undegraded":"2024-04-16T17:06:45.557483+0000","last_fullsized":"2024-04-16T17:06:45.557483+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:19:12.611731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"90'2","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.444031+0000","last_change":"2024-04-16T17:03:24.158875+0000","last_active":"2024-04-16T17:06:45.444031+0000","last_peered":"2024-04-16T17:06:45.444031+0000","last_clean":"2024-04-16T17:06:45.444031+0000","last_became_active":"2024-04-16T17:03:24.158465+0000","last_became_peered":"2024-04-16T17:03:24.158465+0000","last_unstale":"2024-04-16T17:06:45.444031+0000","last_undegraded":"2024-04-16T17:06:45.444031+0000","last_fullsized":"2024-04-16T17:06:45.444031+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:21:29.882251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"92'8","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.239904+0000","last_change":"2024-04-16T17:03:24.994476+0000","last_active":"2024-04-16T17:06:45.239904+0000","last_peered":"2024-04-16T17:06:45.239904+0000","last_clean":"2024-04-16T17:06:45.239904+0000","last_became_active":"2024-04-16T17:03:24.993532+0000","last_became_peered":"2024-04-16T17:03:24.993532+0000","last_unstale":"2024-04-16T17:06:45.239904+0000","last_undegraded":"2024-04-16T17:06:45.239904+0000","last_fullsized":"2024-04-16T17:06:45.239904+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:04:44.062444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.577184+0000","last_change":"2024-04-16T17:03:29.255446+0000","last_active":"2024-04-16T17:06:45.577184+0000","last_peered":"2024-04-16T17:06:45.577184+0000","last_clean":"2024-04-16T17:06:45.577184+0000","last_became_active":"2024-04-16T17:03:29.255051+0000","last_became_peered":"2024-04-16T17:03:29.255051+0000","last_unstale":"2024-04-16T17:06:45.577184+0000","last_undegraded":"2024-04-16T17:06:45.577184+0000","last_fullsized":"2024-04-16T17:06:45.577184+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:24:04.946342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"90'1","reported_seq":35,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.250870+0000","last_change":"2024-04-16T17:03:24.154904+0000","last_active":"2024-04-16T17:06:45.250870+0000","last_peered":"2024-04-16T17:06:45.250870+0000","last_clean":"2024-04-16T17:06:45.250870+0000","last_became_active":"2024-04-16T17:03:24.154617+0000","last_became_peered":"2024-04-16T17:03:24.154617+0000","last_unstale":"2024-04-16T17:06:45.250870+0000","last_undegraded":"2024-04-16T17:06:45.250870+0000","last_fullsized":"2024-04-16T17:06:45.250870+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:35:55.498855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"92'11","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.101961+0000","last_change":"2024-04-16T17:03:24.979908+0000","last_active":"2024-04-16T17:06:45.101961+0000","last_peered":"2024-04-16T17:06:45.101961+0000","last_clean":"2024-04-16T17:06:45.101961+0000","last_became_active":"2024-04-16T17:03:24.979501+0000","last_became_peered":"2024-04-16T17:03:24.979501+0000","last_unstale":"2024-04-16T17:06:45.101961+0000","last_undegraded":"2024-04-16T17:06:45.101961+0000","last_fullsized":"2024-04-16T17:06:45.101961+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:51:05.421459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.240099+0000","last_change":"2024-04-16T17:03:29.258809+0000","last_active":"2024-04-16T17:06:45.240099+0000","last_peered":"2024-04-16T17:06:45.240099+0000","last_clean":"2024-04-16T17:06:45.240099+0000","last_became_active":"2024-04-16T17:03:29.256629+0000","last_became_peered":"2024-04-16T17:03:29.256629+0000","last_unstale":"2024-04-16T17:06:45.240099+0000","last_undegraded":"2024-04-16T17:06:45.240099+0000","last_fullsized":"2024-04-16T17:06:45.240099+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:46:51.634944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"95'5","reported_seq":212,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.445694+0000","last_change":"2024-04-16T17:03:24.153774+0000","last_active":"2024-04-16T17:06:45.445694+0000","last_peered":"2024-04-16T17:06:45.445694+0000","last_clean":"2024-04-16T17:06:45.445694+0000","last_became_active":"2024-04-16T17:03:24.153418+0000","last_became_peered":"2024-04-16T17:03:24.153418+0000","last_unstale":"2024-04-16T17:06:45.445694+0000","last_undegraded":"2024-04-16T17:06:45.445694+0000","last_fullsized":"2024-04-16T17:06:45.445694+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:58:36.093789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":173,"num_read_kb":173,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"92'10","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.451116+0000","last_change":"2024-04-16T17:03:24.962838+0000","last_active":"2024-04-16T17:06:45.451116+0000","last_peered":"2024-04-16T17:06:45.451116+0000","last_clean":"2024-04-16T17:06:45.451116+0000","last_became_active":"2024-04-16T17:03:24.960163+0000","last_became_peered":"2024-04-16T17:03:24.960163+0000","last_unstale":"2024-04-16T17:06:45.451116+0000","last_undegraded":"2024-04-16T17:06:45.451116+0000","last_fullsized":"2024-04-16T17:06:45.451116+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:48:18.723499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.101798+0000","last_change":"2024-04-16T17:03:29.251177+0000","last_active":"2024-04-16T17:06:45.101798+0000","last_peered":"2024-04-16T17:06:45.101798+0000","last_clean":"2024-04-16T17:06:45.101798+0000","last_became_active":"2024-04-16T17:03:29.250657+0000","last_became_peered":"2024-04-16T17:03:29.250657+0000","last_unstale":"2024-04-16T17:06:45.101798+0000","last_undegraded":"2024-04-16T17:06:45.101798+0000","last_fullsized":"2024-04-16T17:06:45.101798+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:16:47.217193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"90'2","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.443359+0000","last_change":"2024-04-16T17:03:24.154245+0000","last_active":"2024-04-16T17:06:45.443359+0000","last_peered":"2024-04-16T17:06:45.443359+0000","last_clean":"2024-04-16T17:06:45.443359+0000","last_became_active":"2024-04-16T17:03:24.153297+0000","last_became_peered":"2024-04-16T17:03:24.153297+0000","last_unstale":"2024-04-16T17:06:45.443359+0000","last_undegraded":"2024-04-16T17:06:45.443359+0000","last_fullsized":"2024-04-16T17:06:45.443359+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:23:49.490238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"92'9","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.577695+0000","last_change":"2024-04-16T17:03:24.984572+0000","last_active":"2024-04-16T17:06:45.577695+0000","last_peered":"2024-04-16T17:06:45.577695+0000","last_clean":"2024-04-16T17:06:45.577695+0000","last_became_active":"2024-04-16T17:03:24.983588+0000","last_became_peered":"2024-04-16T17:03:24.983588+0000","last_unstale":"2024-04-16T17:06:45.577695+0000","last_undegraded":"2024-04-16T17:06:45.577695+0000","last_fullsized":"2024-04-16T17:06:45.577695+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:41:24.704602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.105052+0000","last_change":"2024-04-16T17:03:29.245129+0000","last_active":"2024-04-16T17:06:45.105052+0000","last_peered":"2024-04-16T17:06:45.105052+0000","last_clean":"2024-04-16T17:06:45.105052+0000","last_became_active":"2024-04-16T17:03:29.244720+0000","last_became_peered":"2024-04-16T17:03:29.244720+0000","last_unstale":"2024-04-16T17:06:45.105052+0000","last_undegraded":"2024-04-16T17:06:45.105052+0000","last_fullsized":"2024-04-16T17:06:45.105052+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:55:26.612176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"90'4","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.251165+0000","last_change":"2024-04-16T17:03:24.165248+0000","last_active":"2024-04-16T17:06:45.251165+0000","last_peered":"2024-04-16T17:06:45.251165+0000","last_clean":"2024-04-16T17:06:45.251165+0000","last_became_active":"2024-04-16T17:03:24.164915+0000","last_became_peered":"2024-04-16T17:03:24.164915+0000","last_unstale":"2024-04-16T17:06:45.251165+0000","last_undegraded":"2024-04-16T17:06:45.251165+0000","last_fullsized":"2024-04-16T17:06:45.251165+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:04:47.207554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"92'11","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.450807+0000","last_change":"2024-04-16T17:03:24.967292+0000","last_active":"2024-04-16T17:06:45.450807+0000","last_peered":"2024-04-16T17:06:45.450807+0000","last_clean":"2024-04-16T17:06:45.450807+0000","last_became_active":"2024-04-16T17:03:24.966952+0000","last_became_peered":"2024-04-16T17:03:24.966952+0000","last_unstale":"2024-04-16T17:06:45.450807+0000","last_undegraded":"2024-04-16T17:06:45.450807+0000","last_fullsized":"2024-04-16T17:06:45.450807+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:42:54.969478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.101807+0000","last_change":"2024-04-16T17:03:29.257836+0000","last_active":"2024-04-16T17:06:45.101807+0000","last_peered":"2024-04-16T17:06:45.101807+0000","last_clean":"2024-04-16T17:06:45.101807+0000","last_became_active":"2024-04-16T17:03:29.257409+0000","last_became_peered":"2024-04-16T17:03:29.257409+0000","last_unstale":"2024-04-16T17:06:45.101807+0000","last_undegraded":"2024-04-16T17:06:45.101807+0000","last_fullsized":"2024-04-16T17:06:45.101807+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:25:21.288266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"95'41","reported_seq":433,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.629791+0000","last_change":"2024-04-16T17:03:24.150943+0000","last_active":"2024-04-16T17:06:45.629791+0000","last_peered":"2024-04-16T17:06:45.629791+0000","last_clean":"2024-04-16T17:06:45.629791+0000","last_became_active":"2024-04-16T17:03:24.150563+0000","last_became_peered":"2024-04-16T17:03:24.150563+0000","last_unstale":"2024-04-16T17:06:45.629791+0000","last_undegraded":"2024-04-16T17:06:45.629791+0000","last_fullsized":"2024-04-16T17:06:45.629791+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:05:07.380440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":69011,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":358,"num_read_kb":247,"num_write":42,"num_write_kb":89,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"92'6","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.239659+0000","last_change":"2024-04-16T17:03:24.966393+0000","last_active":"2024-04-16T17:06:45.239659+0000","last_peered":"2024-04-16T17:06:45.239659+0000","last_clean":"2024-04-16T17:06:45.239659+0000","last_became_active":"2024-04-16T17:03:24.966016+0000","last_became_peered":"2024-04-16T17:03:24.966016+0000","last_unstale":"2024-04-16T17:06:45.239659+0000","last_undegraded":"2024-04-16T17:06:45.239659+0000","last_fullsized":"2024-04-16T17:06:45.239659+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:30:25.203262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.251684+0000","last_change":"2024-04-16T17:03:29.265371+0000","last_active":"2024-04-16T17:06:45.251684+0000","last_peered":"2024-04-16T17:06:45.251684+0000","last_clean":"2024-04-16T17:06:45.251684+0000","last_became_active":"2024-04-16T17:03:29.264933+0000","last_became_peered":"2024-04-16T17:03:29.264933+0000","last_unstale":"2024-04-16T17:06:45.251684+0000","last_undegraded":"2024-04-16T17:06:45.251684+0000","last_fullsized":"2024-04-16T17:06:45.251684+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:50:18.203015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"90'6","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.557238+0000","last_change":"2024-04-16T17:04:21.526796+0000","last_active":"2024-04-16T17:06:45.557238+0000","last_peered":"2024-04-16T17:06:45.557238+0000","last_clean":"2024-04-16T17:06:45.557238+0000","last_became_active":"2024-04-16T17:04:21.508364+0000","last_became_peered":"2024-04-16T17:04:21.508364+0000","last_unstale":"2024-04-16T17:06:45.557238+0000","last_undegraded":"2024-04-16T17:06:45.557238+0000","last_fullsized":"2024-04-16T17:06:45.557238+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:45:42.726648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"92'9","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.443893+0000","last_change":"2024-04-16T17:03:24.986228+0000","last_active":"2024-04-16T17:06:45.443893+0000","last_peered":"2024-04-16T17:06:45.443893+0000","last_clean":"2024-04-16T17:06:45.443893+0000","last_became_active":"2024-04-16T17:03:24.985916+0000","last_became_peered":"2024-04-16T17:03:24.985916+0000","last_unstale":"2024-04-16T17:06:45.443893+0000","last_undegraded":"2024-04-16T17:06:45.443893+0000","last_fullsized":"2024-04-16T17:06:45.443893+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:23:54.309203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.557348+0000","last_change":"2024-04-16T17:03:29.264636+0000","last_active":"2024-04-16T17:06:45.557348+0000","last_peered":"2024-04-16T17:06:45.557348+0000","last_clean":"2024-04-16T17:06:45.557348+0000","last_became_active":"2024-04-16T17:03:29.264117+0000","last_became_peered":"2024-04-16T17:03:29.264117+0000","last_unstale":"2024-04-16T17:06:45.557348+0000","last_undegraded":"2024-04-16T17:06:45.557348+0000","last_fullsized":"2024-04-16T17:06:45.557348+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:42:36.123154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"90'3","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.251233+0000","last_change":"2024-04-16T17:03:24.166844+0000","last_active":"2024-04-16T17:06:45.251233+0000","last_peered":"2024-04-16T17:06:45.251233+0000","last_clean":"2024-04-16T17:06:45.251233+0000","last_became_active":"2024-04-16T17:03:24.166479+0000","last_became_peered":"2024-04-16T17:03:24.166479+0000","last_unstale":"2024-04-16T17:06:45.251233+0000","last_undegraded":"2024-04-16T17:06:45.251233+0000","last_fullsized":"2024-04-16T17:06:45.251233+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:02:20.139865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"92'15","reported_seq":47,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.443198+0000","last_change":"2024-04-16T17:03:24.973057+0000","last_active":"2024-04-16T17:06:45.443198+0000","last_peered":"2024-04-16T17:06:45.443198+0000","last_clean":"2024-04-16T17:06:45.443198+0000","last_became_active":"2024-04-16T17:03:24.972731+0000","last_became_peered":"2024-04-16T17:03:24.972731+0000","last_unstale":"2024-04-16T17:06:45.443198+0000","last_undegraded":"2024-04-16T17:06:45.443198+0000","last_fullsized":"2024-04-16T17:06:45.443198+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:37:38.702919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.102363+0000","last_change":"2024-04-16T17:03:29.259167+0000","last_active":"2024-04-16T17:06:45.102363+0000","last_peered":"2024-04-16T17:06:45.102363+0000","last_clean":"2024-04-16T17:06:45.102363+0000","last_became_active":"2024-04-16T17:03:29.258737+0000","last_became_peered":"2024-04-16T17:03:29.258737+0000","last_unstale":"2024-04-16T17:06:45.102363+0000","last_undegraded":"2024-04-16T17:06:45.102363+0000","last_fullsized":"2024-04-16T17:06:45.102363+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:01:13.774037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"95'8","reported_seq":226,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:49.796559+0000","last_change":"2024-04-16T17:03:24.151064+0000","last_active":"2024-04-16T17:06:49.796559+0000","last_peered":"2024-04-16T17:06:49.796559+0000","last_clean":"2024-04-16T17:06:49.796559+0000","last_became_active":"2024-04-16T17:03:24.150745+0000","last_became_peered":"2024-04-16T17:03:24.150745+0000","last_unstale":"2024-04-16T17:06:49.796559+0000","last_undegraded":"2024-04-16T17:06:49.796559+0000","last_fullsized":"2024-04-16T17:06:49.796559+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:32:26.271008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2212,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":185,"num_read_kb":7,"num_write":7,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"92'6","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.442895+0000","last_change":"2024-04-16T17:03:24.971948+0000","last_active":"2024-04-16T17:06:45.442895+0000","last_peered":"2024-04-16T17:06:45.442895+0000","last_clean":"2024-04-16T17:06:45.442895+0000","last_became_active":"2024-04-16T17:03:24.971530+0000","last_became_peered":"2024-04-16T17:03:24.971530+0000","last_unstale":"2024-04-16T17:06:45.442895+0000","last_undegraded":"2024-04-16T17:06:45.442895+0000","last_fullsized":"2024-04-16T17:06:45.442895+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:03:29.286160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.445918+0000","last_change":"2024-04-16T17:03:29.264821+0000","last_active":"2024-04-16T17:06:45.445918+0000","last_peered":"2024-04-16T17:06:45.445918+0000","last_clean":"2024-04-16T17:06:45.445918+0000","last_became_active":"2024-04-16T17:03:29.264287+0000","last_became_peered":"2024-04-16T17:03:29.264287+0000","last_unstale":"2024-04-16T17:06:45.445918+0000","last_undegraded":"2024-04-16T17:06:45.445918+0000","last_fullsized":"2024-04-16T17:06:45.445918+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:33:28.400663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"90'15","reported_seq":50,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.239188+0000","last_change":"2024-04-16T17:03:24.158844+0000","last_active":"2024-04-16T17:06:45.239188+0000","last_peered":"2024-04-16T17:06:45.239188+0000","last_clean":"2024-04-16T17:06:45.239188+0000","last_became_active":"2024-04-16T17:03:24.158537+0000","last_became_peered":"2024-04-16T17:03:24.158537+0000","last_unstale":"2024-04-16T17:06:45.239188+0000","last_undegraded":"2024-04-16T17:06:45.239188+0000","last_fullsized":"2024-04-16T17:06:45.239188+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:42:22.772452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"92'6","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.251080+0000","last_change":"2024-04-16T17:03:24.962522+0000","last_active":"2024-04-16T17:06:45.251080+0000","last_peered":"2024-04-16T17:06:45.251080+0000","last_clean":"2024-04-16T17:06:45.251080+0000","last_became_active":"2024-04-16T17:03:24.962169+0000","last_became_peered":"2024-04-16T17:03:24.962169+0000","last_unstale":"2024-04-16T17:06:45.251080+0000","last_undegraded":"2024-04-16T17:06:45.251080+0000","last_fullsized":"2024-04-16T17:06:45.251080+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:28:17.147491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.250937+0000","last_change":"2024-04-16T17:03:29.259539+0000","last_active":"2024-04-16T17:06:45.250937+0000","last_peered":"2024-04-16T17:06:45.250937+0000","last_clean":"2024-04-16T17:06:45.250937+0000","last_became_active":"2024-04-16T17:03:29.259123+0000","last_became_peered":"2024-04-16T17:03:29.259123+0000","last_unstale":"2024-04-16T17:06:45.250937+0000","last_undegraded":"2024-04-16T17:06:45.250937+0000","last_fullsized":"2024-04-16T17:06:45.250937+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:15:17.142636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"95'4","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.239908+0000","last_change":"2024-04-16T17:03:24.147526+0000","last_active":"2024-04-16T17:06:45.239908+0000","last_peered":"2024-04-16T17:06:45.239908+0000","last_clean":"2024-04-16T17:06:45.239908+0000","last_became_active":"2024-04-16T17:03:24.147119+0000","last_became_peered":"2024-04-16T17:03:24.147119+0000","last_unstale":"2024-04-16T17:06:45.239908+0000","last_undegraded":"2024-04-16T17:06:45.239908+0000","last_fullsized":"2024-04-16T17:06:45.239908+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:30:38.742122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"92'5","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.557187+0000","last_change":"2024-04-16T17:04:21.507113+0000","last_active":"2024-04-16T17:06:45.557187+0000","last_peered":"2024-04-16T17:06:45.557187+0000","last_clean":"2024-04-16T17:06:45.557187+0000","last_became_active":"2024-04-16T17:04:21.506680+0000","last_became_peered":"2024-04-16T17:04:21.506680+0000","last_unstale":"2024-04-16T17:06:45.557187+0000","last_undegraded":"2024-04-16T17:06:45.557187+0000","last_fullsized":"2024-04-16T17:06:45.557187+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:08:47.971317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,8],"acting":[11,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":32,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.239441+0000","last_change":"2024-04-16T17:05:22.511966+0000","last_active":"2024-04-16T17:06:45.239441+0000","last_peered":"2024-04-16T17:06:45.239441+0000","last_clean":"2024-04-16T17:06:45.239441+0000","last_became_active":"2024-04-16T17:05:22.511512+0000","last_became_peered":"2024-04-16T17:05:22.511512+0000","last_unstale":"2024-04-16T17:06:45.239441+0000","last_undegraded":"2024-04-16T17:06:45.239441+0000","last_fullsized":"2024-04-16T17:06:45.239441+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:14:41.291499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,1],"acting":[8,10,11,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.102181+0000","last_change":"2024-04-16T17:03:24.162105+0000","last_active":"2024-04-16T17:06:45.102181+0000","last_peered":"2024-04-16T17:06:45.102181+0000","last_clean":"2024-04-16T17:06:45.102181+0000","last_became_active":"2024-04-16T17:03:24.161631+0000","last_became_peered":"2024-04-16T17:03:24.161631+0000","last_unstale":"2024-04-16T17:06:45.102181+0000","last_undegraded":"2024-04-16T17:06:45.102181+0000","last_fullsized":"2024-04-16T17:06:45.102181+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:22:33.677848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"92'4","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.226784+0000","last_change":"2024-04-16T17:03:24.978253+0000","last_active":"2024-04-16T17:06:45.226784+0000","last_peered":"2024-04-16T17:06:45.226784+0000","last_clean":"2024-04-16T17:06:45.226784+0000","last_became_active":"2024-04-16T17:03:24.977956+0000","last_became_peered":"2024-04-16T17:03:24.977956+0000","last_unstale":"2024-04-16T17:06:45.226784+0000","last_undegraded":"2024-04-16T17:06:45.226784+0000","last_fullsized":"2024-04-16T17:06:45.226784+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T17:59:45.102383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.444635+0000","last_change":"2024-04-16T17:03:29.265312+0000","last_active":"2024-04-16T17:06:45.444635+0000","last_peered":"2024-04-16T17:06:45.444635+0000","last_clean":"2024-04-16T17:06:45.444635+0000","last_became_active":"2024-04-16T17:03:29.264891+0000","last_became_peered":"2024-04-16T17:03:29.264891+0000","last_unstale":"2024-04-16T17:06:45.444635+0000","last_undegraded":"2024-04-16T17:06:45.444635+0000","last_fullsized":"2024-04-16T17:06:45.444635+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:26:40.088127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"90'1","reported_seq":35,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.442929+0000","last_change":"2024-04-16T17:03:24.170935+0000","last_active":"2024-04-16T17:06:45.442929+0000","last_peered":"2024-04-16T17:06:45.442929+0000","last_clean":"2024-04-16T17:06:45.442929+0000","last_became_active":"2024-04-16T17:03:24.170609+0000","last_became_peered":"2024-04-16T17:03:24.170609+0000","last_unstale":"2024-04-16T17:06:45.442929+0000","last_undegraded":"2024-04-16T17:06:45.442929+0000","last_fullsized":"2024-04-16T17:06:45.442929+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:18:33.516309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"92'7","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.226731+0000","last_change":"2024-04-16T17:03:24.985434+0000","last_active":"2024-04-16T17:06:45.226731+0000","last_peered":"2024-04-16T17:06:45.226731+0000","last_clean":"2024-04-16T17:06:45.226731+0000","last_became_active":"2024-04-16T17:03:24.984388+0000","last_became_peered":"2024-04-16T17:03:24.984388+0000","last_unstale":"2024-04-16T17:06:45.226731+0000","last_undegraded":"2024-04-16T17:06:45.226731+0000","last_fullsized":"2024-04-16T17:06:45.226731+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:43:26.355572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":32,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.445935+0000","last_change":"2024-04-16T17:05:22.516267+0000","last_active":"2024-04-16T17:06:45.445935+0000","last_peered":"2024-04-16T17:06:45.445935+0000","last_clean":"2024-04-16T17:06:45.445935+0000","last_became_active":"2024-04-16T17:05:22.513637+0000","last_became_peered":"2024-04-16T17:05:22.513637+0000","last_unstale":"2024-04-16T17:06:45.445935+0000","last_undegraded":"2024-04-16T17:06:45.445935+0000","last_fullsized":"2024-04-16T17:06:45.445935+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:39:47.975735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,8,10,0],"acting":[6,8,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"92'8","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.251371+0000","last_change":"2024-04-16T17:03:24.962786+0000","last_active":"2024-04-16T17:06:45.251371+0000","last_peered":"2024-04-16T17:06:45.251371+0000","last_clean":"2024-04-16T17:06:45.251371+0000","last_became_active":"2024-04-16T17:03:24.961704+0000","last_became_peered":"2024-04-16T17:03:24.961704+0000","last_unstale":"2024-04-16T17:06:45.251371+0000","last_undegraded":"2024-04-16T17:06:45.251371+0000","last_fullsized":"2024-04-16T17:06:45.251371+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:22:27.957607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"94'6","reported_seq":213,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.443944+0000","last_change":"2024-04-16T17:03:24.175617+0000","last_active":"2024-04-16T17:06:45.443944+0000","last_peered":"2024-04-16T17:06:45.443944+0000","last_clean":"2024-04-16T17:06:45.443944+0000","last_became_active":"2024-04-16T17:03:24.175272+0000","last_became_peered":"2024-04-16T17:03:24.175272+0000","last_unstale":"2024-04-16T17:06:45.443944+0000","last_undegraded":"2024-04-16T17:06:45.443944+0000","last_fullsized":"2024-04-16T17:06:45.443944+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:27:44.738030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":173,"num_read_kb":173,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"90'1","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.443256+0000","last_change":"2024-04-16T17:04:21.512033+0000","last_active":"2024-04-16T17:06:45.443256+0000","last_peered":"2024-04-16T17:06:45.443256+0000","last_clean":"2024-04-16T17:06:45.443256+0000","last_became_active":"2024-04-16T17:04:21.506718+0000","last_became_peered":"2024-04-16T17:04:21.506718+0000","last_unstale":"2024-04-16T17:06:45.443256+0000","last_undegraded":"2024-04-16T17:06:45.443256+0000","last_fullsized":"2024-04-16T17:06:45.443256+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:53:48.953887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,10],"acting":[7,5,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"92'8","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.240468+0000","last_change":"2024-04-16T17:03:24.983629+0000","last_active":"2024-04-16T17:06:45.240468+0000","last_peered":"2024-04-16T17:06:45.240468+0000","last_clean":"2024-04-16T17:06:45.240468+0000","last_became_active":"2024-04-16T17:03:24.979969+0000","last_became_peered":"2024-04-16T17:03:24.979969+0000","last_unstale":"2024-04-16T17:06:45.240468+0000","last_undegraded":"2024-04-16T17:06:45.240468+0000","last_fullsized":"2024-04-16T17:06:45.240468+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:01:24.391732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"90'3","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.104956+0000","last_change":"2024-04-16T17:03:24.152475+0000","last_active":"2024-04-16T17:06:45.104956+0000","last_peered":"2024-04-16T17:06:45.104956+0000","last_clean":"2024-04-16T17:06:45.104956+0000","last_became_active":"2024-04-16T17:03:24.152198+0000","last_became_peered":"2024-04-16T17:03:24.152198+0000","last_unstale":"2024-04-16T17:06:45.104956+0000","last_undegraded":"2024-04-16T17:06:45.104956+0000","last_fullsized":"2024-04-16T17:06:45.104956+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:49:33.914325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":367,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.442836+0000","last_change":"2024-04-16T17:02:02.111335+0000","last_active":"2024-04-16T17:06:45.442836+0000","last_peered":"2024-04-16T17:06:45.442836+0000","last_clean":"2024-04-16T17:06:45.442836+0000","last_became_active":"2024-04-16T17:02:02.110928+0000","last_became_peered":"2024-04-16T17:02:02.110928+0000","last_unstale":"2024-04-16T17:06:45.442836+0000","last_undegraded":"2024-04-16T17:06:45.442836+0000","last_fullsized":"2024-04-16T17:06:45.442836+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T16:58:07.081753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T16:58:07.081753+0000","last_clean_scrub_stamp":"2024-04-16T16:58:07.081753+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:27:02.865664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.251060+0000","last_change":"2024-04-16T17:03:29.301691+0000","last_active":"2024-04-16T17:06:45.251060+0000","last_peered":"2024-04-16T17:06:45.251060+0000","last_clean":"2024-04-16T17:06:45.251060+0000","last_became_active":"2024-04-16T17:03:29.301277+0000","last_became_peered":"2024-04-16T17:03:29.301277+0000","last_unstale":"2024-04-16T17:06:45.251060+0000","last_undegraded":"2024-04-16T17:06:45.251060+0000","last_fullsized":"2024-04-16T17:06:45.251060+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:45:16.727610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"92'5","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.443705+0000","last_change":"2024-04-16T17:03:24.970939+0000","last_active":"2024-04-16T17:06:45.443705+0000","last_peered":"2024-04-16T17:06:45.443705+0000","last_clean":"2024-04-16T17:06:45.443705+0000","last_became_active":"2024-04-16T17:03:24.970581+0000","last_became_peered":"2024-04-16T17:03:24.970581+0000","last_unstale":"2024-04-16T17:06:45.443705+0000","last_undegraded":"2024-04-16T17:06:45.443705+0000","last_fullsized":"2024-04-16T17:06:45.443705+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:33:07.448597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"90'3","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.442815+0000","last_change":"2024-04-16T17:03:24.155454+0000","last_active":"2024-04-16T17:06:45.442815+0000","last_peered":"2024-04-16T17:06:45.442815+0000","last_clean":"2024-04-16T17:06:45.442815+0000","last_became_active":"2024-04-16T17:03:24.155155+0000","last_became_peered":"2024-04-16T17:03:24.155155+0000","last_unstale":"2024-04-16T17:06:45.442815+0000","last_undegraded":"2024-04-16T17:06:45.442815+0000","last_fullsized":"2024-04-16T17:06:45.442815+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:34:08.618002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.557582+0000","last_change":"2024-04-16T17:03:29.255538+0000","last_active":"2024-04-16T17:06:45.557582+0000","last_peered":"2024-04-16T17:06:45.557582+0000","last_clean":"2024-04-16T17:06:45.557582+0000","last_became_active":"2024-04-16T17:03:29.255107+0000","last_became_peered":"2024-04-16T17:03:29.255107+0000","last_unstale":"2024-04-16T17:06:45.557582+0000","last_undegraded":"2024-04-16T17:06:45.557582+0000","last_fullsized":"2024-04-16T17:06:45.557582+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:31:58.458641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"90'1","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.239669+0000","last_change":"2024-04-16T17:03:24.158816+0000","last_active":"2024-04-16T17:06:45.239669+0000","last_peered":"2024-04-16T17:06:45.239669+0000","last_clean":"2024-04-16T17:06:45.239669+0000","last_became_active":"2024-04-16T17:03:24.158518+0000","last_became_peered":"2024-04-16T17:03:24.158518+0000","last_unstale":"2024-04-16T17:06:45.239669+0000","last_undegraded":"2024-04-16T17:06:45.239669+0000","last_fullsized":"2024-04-16T17:06:45.239669+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:54:55.807975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"92'9","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.445158+0000","last_change":"2024-04-16T17:03:24.977190+0000","last_active":"2024-04-16T17:06:45.445158+0000","last_peered":"2024-04-16T17:06:45.445158+0000","last_clean":"2024-04-16T17:06:45.445158+0000","last_became_active":"2024-04-16T17:03:24.976840+0000","last_became_peered":"2024-04-16T17:03:24.976840+0000","last_unstale":"2024-04-16T17:06:45.445158+0000","last_undegraded":"2024-04-16T17:06:45.445158+0000","last_fullsized":"2024-04-16T17:06:45.445158+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:21:51.294150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.239834+0000","last_change":"2024-04-16T17:03:29.301423+0000","last_active":"2024-04-16T17:06:45.239834+0000","last_peered":"2024-04-16T17:06:45.239834+0000","last_clean":"2024-04-16T17:06:45.239834+0000","last_became_active":"2024-04-16T17:03:29.300888+0000","last_became_peered":"2024-04-16T17:03:29.300888+0000","last_unstale":"2024-04-16T17:06:45.239834+0000","last_undegraded":"2024-04-16T17:06:45.239834+0000","last_fullsized":"2024-04-16T17:06:45.239834+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:13:54.188090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"95'46","reported_seq":103,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.390333+0000","last_change":"2024-04-16T17:03:24.167643+0000","last_active":"2024-04-16T17:06:45.390333+0000","last_peered":"2024-04-16T17:06:45.390333+0000","last_clean":"2024-04-16T17:06:45.390333+0000","last_became_active":"2024-04-16T17:03:24.167103+0000","last_became_peered":"2024-04-16T17:03:24.167103+0000","last_unstale":"2024-04-16T17:06:45.390333+0000","last_undegraded":"2024-04-16T17:06:45.390333+0000","last_fullsized":"2024-04-16T17:06:45.390333+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:33:16.809371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":82,"num_write_kb":837,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"92'5","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.241641+0000","last_change":"2024-04-16T17:03:24.976981+0000","last_active":"2024-04-16T17:06:45.241641+0000","last_peered":"2024-04-16T17:06:45.241641+0000","last_clean":"2024-04-16T17:06:45.241641+0000","last_became_active":"2024-04-16T17:03:24.975385+0000","last_became_peered":"2024-04-16T17:03:24.975385+0000","last_unstale":"2024-04-16T17:06:45.241641+0000","last_undegraded":"2024-04-16T17:06:45.241641+0000","last_fullsized":"2024-04-16T17:06:45.241641+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:27:03.104838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.105056+0000","last_change":"2024-04-16T17:03:29.244767+0000","last_active":"2024-04-16T17:06:45.105056+0000","last_peered":"2024-04-16T17:06:45.105056+0000","last_clean":"2024-04-16T17:06:45.105056+0000","last_became_active":"2024-04-16T17:03:29.244205+0000","last_became_peered":"2024-04-16T17:03:29.244205+0000","last_unstale":"2024-04-16T17:06:45.105056+0000","last_undegraded":"2024-04-16T17:06:45.105056+0000","last_fullsized":"2024-04-16T17:06:45.105056+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:51:46.652387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"90'4","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.105248+0000","last_change":"2024-04-16T17:03:24.164916+0000","last_active":"2024-04-16T17:06:45.105248+0000","last_peered":"2024-04-16T17:06:45.105248+0000","last_clean":"2024-04-16T17:06:45.105248+0000","last_became_active":"2024-04-16T17:03:24.164648+0000","last_became_peered":"2024-04-16T17:03:24.164648+0000","last_unstale":"2024-04-16T17:06:45.105248+0000","last_undegraded":"2024-04-16T17:06:45.105248+0000","last_fullsized":"2024-04-16T17:06:45.105248+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:23:00.198785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"92'7","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.577426+0000","last_change":"2024-04-16T17:03:24.994393+0000","last_active":"2024-04-16T17:06:45.577426+0000","last_peered":"2024-04-16T17:06:45.577426+0000","last_clean":"2024-04-16T17:06:45.577426+0000","last_became_active":"2024-04-16T17:03:24.994002+0000","last_became_peered":"2024-04-16T17:03:24.994002+0000","last_unstale":"2024-04-16T17:06:45.577426+0000","last_undegraded":"2024-04-16T17:06:45.577426+0000","last_fullsized":"2024-04-16T17:06:45.577426+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:23:44.596781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.104795+0000","last_change":"2024-04-16T17:03:29.302009+0000","last_active":"2024-04-16T17:06:45.104795+0000","last_peered":"2024-04-16T17:06:45.104795+0000","last_clean":"2024-04-16T17:06:45.104795+0000","last_became_active":"2024-04-16T17:03:29.301183+0000","last_became_peered":"2024-04-16T17:03:29.301183+0000","last_unstale":"2024-04-16T17:06:45.104795+0000","last_undegraded":"2024-04-16T17:06:45.104795+0000","last_fullsized":"2024-04-16T17:06:45.104795+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:30:19.561875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"90'2","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.450559+0000","last_change":"2024-04-16T17:03:24.176829+0000","last_active":"2024-04-16T17:06:45.450559+0000","last_peered":"2024-04-16T17:06:45.450559+0000","last_clean":"2024-04-16T17:06:45.450559+0000","last_became_active":"2024-04-16T17:03:24.176555+0000","last_became_peered":"2024-04-16T17:03:24.176555+0000","last_unstale":"2024-04-16T17:06:45.450559+0000","last_undegraded":"2024-04-16T17:06:45.450559+0000","last_fullsized":"2024-04-16T17:06:45.450559+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:05:40.602150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"92'2","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.239184+0000","last_change":"2024-04-16T17:03:24.965746+0000","last_active":"2024-04-16T17:06:45.239184+0000","last_peered":"2024-04-16T17:06:45.239184+0000","last_clean":"2024-04-16T17:06:45.239184+0000","last_became_active":"2024-04-16T17:03:24.965241+0000","last_became_peered":"2024-04-16T17:03:24.965241+0000","last_unstale":"2024-04-16T17:06:45.239184+0000","last_undegraded":"2024-04-16T17:06:45.239184+0000","last_fullsized":"2024-04-16T17:06:45.239184+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:51:34.407565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.102475+0000","last_change":"2024-04-16T17:03:29.261441+0000","last_active":"2024-04-16T17:06:45.102475+0000","last_peered":"2024-04-16T17:06:45.102475+0000","last_clean":"2024-04-16T17:06:45.102475+0000","last_became_active":"2024-04-16T17:03:29.259538+0000","last_became_peered":"2024-04-16T17:03:29.259538+0000","last_unstale":"2024-04-16T17:06:45.102475+0000","last_undegraded":"2024-04-16T17:06:45.102475+0000","last_fullsized":"2024-04-16T17:06:45.102475+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:12:27.092053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"90'3","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.240172+0000","last_change":"2024-04-16T17:03:24.172053+0000","last_active":"2024-04-16T17:06:45.240172+0000","last_peered":"2024-04-16T17:06:45.240172+0000","last_clean":"2024-04-16T17:06:45.240172+0000","last_became_active":"2024-04-16T17:03:24.171709+0000","last_became_peered":"2024-04-16T17:03:24.171709+0000","last_unstale":"2024-04-16T17:06:45.240172+0000","last_undegraded":"2024-04-16T17:06:45.240172+0000","last_fullsized":"2024-04-16T17:06:45.240172+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T01:40:12.307756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"92'9","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.557252+0000","last_change":"2024-04-16T17:03:24.984267+0000","last_active":"2024-04-16T17:06:45.557252+0000","last_peered":"2024-04-16T17:06:45.557252+0000","last_clean":"2024-04-16T17:06:45.557252+0000","last_became_active":"2024-04-16T17:03:24.983888+0000","last_became_peered":"2024-04-16T17:03:24.983888+0000","last_unstale":"2024-04-16T17:06:45.557252+0000","last_undegraded":"2024-04-16T17:06:45.557252+0000","last_fullsized":"2024-04-16T17:06:45.557252+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:16:23.814917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"90'1","reported_seq":26,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.105091+0000","last_change":"2024-04-16T17:03:29.251716+0000","last_active":"2024-04-16T17:06:45.105091+0000","last_peered":"2024-04-16T17:06:45.105091+0000","last_clean":"2024-04-16T17:06:45.105091+0000","last_became_active":"2024-04-16T17:03:29.251317+0000","last_became_peered":"2024-04-16T17:03:29.251317+0000","last_unstale":"2024-04-16T17:06:45.105091+0000","last_undegraded":"2024-04-16T17:06:45.105091+0000","last_fullsized":"2024-04-16T17:06:45.105091+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:48:54.680195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"90'1","reported_seq":22,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.557164+0000","last_change":"2024-04-16T17:04:21.515306+0000","last_active":"2024-04-16T17:06:45.557164+0000","last_peered":"2024-04-16T17:06:45.557164+0000","last_clean":"2024-04-16T17:06:45.557164+0000","last_became_active":"2024-04-16T17:04:21.508377+0000","last_became_peered":"2024-04-16T17:04:21.508377+0000","last_unstale":"2024-04-16T17:06:45.557164+0000","last_undegraded":"2024-04-16T17:06:45.557164+0000","last_fullsized":"2024-04-16T17:06:45.557164+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:28:25.000406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"92'8","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.240125+0000","last_change":"2024-04-16T17:03:24.995547+0000","last_active":"2024-04-16T17:06:45.240125+0000","last_peered":"2024-04-16T17:06:45.240125+0000","last_clean":"2024-04-16T17:06:45.240125+0000","last_became_active":"2024-04-16T17:03:24.995096+0000","last_became_peered":"2024-04-16T17:03:24.995096+0000","last_unstale":"2024-04-16T17:06:45.240125+0000","last_undegraded":"2024-04-16T17:06:45.240125+0000","last_fullsized":"2024-04-16T17:06:45.240125+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:54:51.193117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.104926+0000","last_change":"2024-04-16T17:03:29.301526+0000","last_active":"2024-04-16T17:06:45.104926+0000","last_peered":"2024-04-16T17:06:45.104926+0000","last_clean":"2024-04-16T17:06:45.104926+0000","last_became_active":"2024-04-16T17:03:29.300808+0000","last_became_peered":"2024-04-16T17:03:29.300808+0000","last_unstale":"2024-04-16T17:06:45.104926+0000","last_undegraded":"2024-04-16T17:06:45.104926+0000","last_fullsized":"2024-04-16T17:06:45.104926+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T18:01:14.717548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"90'4","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.239357+0000","last_change":"2024-04-16T17:03:24.152645+0000","last_active":"2024-04-16T17:06:45.239357+0000","last_peered":"2024-04-16T17:06:45.239357+0000","last_clean":"2024-04-16T17:06:45.239357+0000","last_became_active":"2024-04-16T17:03:24.152281+0000","last_became_peered":"2024-04-16T17:03:24.152281+0000","last_unstale":"2024-04-16T17:06:45.239357+0000","last_undegraded":"2024-04-16T17:06:45.239357+0000","last_fullsized":"2024-04-16T17:06:45.239357+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:33:51.377421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"92'5","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.443067+0000","last_change":"2024-04-16T17:03:24.979827+0000","last_active":"2024-04-16T17:06:45.443067+0000","last_peered":"2024-04-16T17:06:45.443067+0000","last_clean":"2024-04-16T17:06:45.443067+0000","last_became_active":"2024-04-16T17:03:24.979452+0000","last_became_peered":"2024-04-16T17:03:24.979452+0000","last_unstale":"2024-04-16T17:06:45.443067+0000","last_undegraded":"2024-04-16T17:06:45.443067+0000","last_fullsized":"2024-04-16T17:06:45.443067+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:07:09.259366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.450802+0000","last_change":"2024-04-16T17:03:29.255519+0000","last_active":"2024-04-16T17:06:45.450802+0000","last_peered":"2024-04-16T17:06:45.450802+0000","last_clean":"2024-04-16T17:06:45.450802+0000","last_became_active":"2024-04-16T17:03:29.255104+0000","last_became_peered":"2024-04-16T17:03:29.255104+0000","last_unstale":"2024-04-16T17:06:45.450802+0000","last_undegraded":"2024-04-16T17:06:45.450802+0000","last_fullsized":"2024-04-16T17:06:45.450802+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:27:34.496199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"90'3","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.105533+0000","last_change":"2024-04-16T17:03:24.161145+0000","last_active":"2024-04-16T17:06:45.105533+0000","last_peered":"2024-04-16T17:06:45.105533+0000","last_clean":"2024-04-16T17:06:45.105533+0000","last_became_active":"2024-04-16T17:03:24.160824+0000","last_became_peered":"2024-04-16T17:03:24.160824+0000","last_unstale":"2024-04-16T17:06:45.105533+0000","last_undegraded":"2024-04-16T17:06:45.105533+0000","last_fullsized":"2024-04-16T17:06:45.105533+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:02:48.805638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"92'7","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.443517+0000","last_change":"2024-04-16T17:03:24.966513+0000","last_active":"2024-04-16T17:06:45.443517+0000","last_peered":"2024-04-16T17:06:45.443517+0000","last_clean":"2024-04-16T17:06:45.443517+0000","last_became_active":"2024-04-16T17:03:24.966074+0000","last_became_peered":"2024-04-16T17:03:24.966074+0000","last_unstale":"2024-04-16T17:06:45.443517+0000","last_undegraded":"2024-04-16T17:06:45.443517+0000","last_fullsized":"2024-04-16T17:06:45.443517+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:50:21.579901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":32,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.576687+0000","last_change":"2024-04-16T17:04:22.150786+0000","last_active":"2024-04-16T17:06:45.576687+0000","last_peered":"2024-04-16T17:06:45.576687+0000","last_clean":"2024-04-16T17:06:45.576687+0000","last_became_active":"2024-04-16T17:04:22.149344+0000","last_became_peered":"2024-04-16T17:04:22.149344+0000","last_unstale":"2024-04-16T17:06:45.576687+0000","last_undegraded":"2024-04-16T17:06:45.576687+0000","last_fullsized":"2024-04-16T17:06:45.576687+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:18:49.421095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,1,4],"acting":[0,8,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"95'39","reported_seq":81,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.386141+0000","last_change":"2024-04-16T17:03:24.158032+0000","last_active":"2024-04-16T17:06:45.386141+0000","last_peered":"2024-04-16T17:06:45.386141+0000","last_clean":"2024-04-16T17:06:45.386141+0000","last_became_active":"2024-04-16T17:03:24.157689+0000","last_became_peered":"2024-04-16T17:03:24.157689+0000","last_unstale":"2024-04-16T17:06:45.386141+0000","last_undegraded":"2024-04-16T17:06:45.386141+0000","last_fullsized":"2024-04-16T17:06:45.386141+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:38:18.240889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"92'6","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.443058+0000","last_change":"2024-04-16T17:04:21.519999+0000","last_active":"2024-04-16T17:06:45.443058+0000","last_peered":"2024-04-16T17:06:45.443058+0000","last_clean":"2024-04-16T17:06:45.443058+0000","last_became_active":"2024-04-16T17:04:21.519621+0000","last_became_peered":"2024-04-16T17:04:21.519621+0000","last_unstale":"2024-04-16T17:06:45.443058+0000","last_undegraded":"2024-04-16T17:06:45.443058+0000","last_fullsized":"2024-04-16T17:06:45.443058+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:50:13.266369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.250659+0000","last_change":"2024-04-16T17:03:29.245176+0000","last_active":"2024-04-16T17:06:45.250659+0000","last_peered":"2024-04-16T17:06:45.250659+0000","last_clean":"2024-04-16T17:06:45.250659+0000","last_became_active":"2024-04-16T17:03:29.244763+0000","last_became_peered":"2024-04-16T17:03:29.244763+0000","last_unstale":"2024-04-16T17:06:45.250659+0000","last_undegraded":"2024-04-16T17:06:45.250659+0000","last_fullsized":"2024-04-16T17:06:45.250659+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:28.203961+0000","last_clean_scrub_stamp":"2024-04-16T17:03:28.203961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:52:30.381417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.445799+0000","last_change":"2024-04-16T17:03:24.168068+0000","last_active":"2024-04-16T17:06:45.445799+0000","last_peered":"2024-04-16T17:06:45.445799+0000","last_clean":"2024-04-16T17:06:45.445799+0000","last_became_active":"2024-04-16T17:03:24.167694+0000","last_became_peered":"2024-04-16T17:03:24.167694+0000","last_unstale":"2024-04-16T17:06:45.445799+0000","last_undegraded":"2024-04-16T17:06:45.445799+0000","last_fullsized":"2024-04-16T17:06:45.445799+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:23.127979+0000","last_clean_scrub_stamp":"2024-04-16T17:03:23.127979+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:13:35.692306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"92'9","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.443511+0000","last_change":"2024-04-16T17:03:24.966145+0000","last_active":"2024-04-16T17:06:45.443511+0000","last_peered":"2024-04-16T17:06:45.443511+0000","last_clean":"2024-04-16T17:06:45.443511+0000","last_became_active":"2024-04-16T17:03:24.965813+0000","last_became_peered":"2024-04-16T17:03:24.965813+0000","last_unstale":"2024-04-16T17:06:45.443511+0000","last_undegraded":"2024-04-16T17:06:45.443511+0000","last_fullsized":"2024-04-16T17:06:45.443511+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T02:43:43.224130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"92'9","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.557305+0000","last_change":"2024-04-16T17:03:24.970941+0000","last_active":"2024-04-16T17:06:45.557305+0000","last_peered":"2024-04-16T17:06:45.557305+0000","last_clean":"2024-04-16T17:06:45.557305+0000","last_became_active":"2024-04-16T17:03:24.970587+0000","last_became_peered":"2024-04-16T17:03:24.970587+0000","last_unstale":"2024-04-16T17:06:45.557305+0000","last_undegraded":"2024-04-16T17:06:45.557305+0000","last_fullsized":"2024-04-16T17:06:45.557305+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:14:13.930011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"92'12","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.105601+0000","last_change":"2024-04-16T17:03:24.967167+0000","last_active":"2024-04-16T17:06:45.105601+0000","last_peered":"2024-04-16T17:06:45.105601+0000","last_clean":"2024-04-16T17:06:45.105601+0000","last_became_active":"2024-04-16T17:03:24.966787+0000","last_became_peered":"2024-04-16T17:03:24.966787+0000","last_unstale":"2024-04-16T17:06:45.105601+0000","last_undegraded":"2024-04-16T17:06:45.105601+0000","last_fullsized":"2024-04-16T17:06:45.105601+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T21:51:39.510220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"92'8","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.105196+0000","last_change":"2024-04-16T17:03:24.981115+0000","last_active":"2024-04-16T17:06:45.105196+0000","last_peered":"2024-04-16T17:06:45.105196+0000","last_clean":"2024-04-16T17:06:45.105196+0000","last_became_active":"2024-04-16T17:03:24.980685+0000","last_became_peered":"2024-04-16T17:03:24.980685+0000","last_unstale":"2024-04-16T17:06:45.105196+0000","last_undegraded":"2024-04-16T17:06:45.105196+0000","last_fullsized":"2024-04-16T17:06:45.105196+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:20:44.066932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"92'7","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.576960+0000","last_change":"2024-04-16T17:03:24.990421+0000","last_active":"2024-04-16T17:06:45.576960+0000","last_peered":"2024-04-16T17:06:45.576960+0000","last_clean":"2024-04-16T17:06:45.576960+0000","last_became_active":"2024-04-16T17:03:24.989940+0000","last_became_peered":"2024-04-16T17:03:24.989940+0000","last_unstale":"2024-04-16T17:06:45.576960+0000","last_undegraded":"2024-04-16T17:06:45.576960+0000","last_fullsized":"2024-04-16T17:06:45.576960+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T20:19:57.548067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"92'4","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.445217+0000","last_change":"2024-04-16T17:03:24.991854+0000","last_active":"2024-04-16T17:06:45.445217+0000","last_peered":"2024-04-16T17:06:45.445217+0000","last_clean":"2024-04-16T17:06:45.445217+0000","last_became_active":"2024-04-16T17:03:24.990964+0000","last_became_peered":"2024-04-16T17:03:24.990964+0000","last_unstale":"2024-04-16T17:06:45.445217+0000","last_undegraded":"2024-04-16T17:06:45.445217+0000","last_fullsized":"2024-04-16T17:06:45.445217+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:10:10.618708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"92'6","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.557167+0000","last_change":"2024-04-16T17:04:21.517466+0000","last_active":"2024-04-16T17:06:45.557167+0000","last_peered":"2024-04-16T17:06:45.557167+0000","last_clean":"2024-04-16T17:06:45.557167+0000","last_became_active":"2024-04-16T17:04:21.517086+0000","last_became_peered":"2024-04-16T17:04:21.517086+0000","last_unstale":"2024-04-16T17:06:45.557167+0000","last_undegraded":"2024-04-16T17:06:45.557167+0000","last_fullsized":"2024-04-16T17:06:45.557167+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:54:33.417452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,9],"acting":[11,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"92'15","reported_seq":47,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.105712+0000","last_change":"2024-04-16T17:03:24.970377+0000","last_active":"2024-04-16T17:06:45.105712+0000","last_peered":"2024-04-16T17:06:45.105712+0000","last_clean":"2024-04-16T17:06:45.105712+0000","last_became_active":"2024-04-16T17:03:24.968856+0000","last_became_peered":"2024-04-16T17:03:24.968856+0000","last_unstale":"2024-04-16T17:06:45.105712+0000","last_undegraded":"2024-04-16T17:06:45.105712+0000","last_fullsized":"2024-04-16T17:06:45.105712+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T23:47:28.543120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"92'2","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.241849+0000","last_change":"2024-04-16T17:03:24.992877+0000","last_active":"2024-04-16T17:06:45.241849+0000","last_peered":"2024-04-16T17:06:45.241849+0000","last_clean":"2024-04-16T17:06:45.241849+0000","last_became_active":"2024-04-16T17:03:24.992484+0000","last_became_peered":"2024-04-16T17:03:24.992484+0000","last_unstale":"2024-04-16T17:06:45.241849+0000","last_undegraded":"2024-04-16T17:06:45.241849+0000","last_fullsized":"2024-04-16T17:06:45.241849+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T00:00:45.204782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"92'6","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.250911+0000","last_change":"2024-04-16T17:03:24.977794+0000","last_active":"2024-04-16T17:06:45.250911+0000","last_peered":"2024-04-16T17:06:45.250911+0000","last_clean":"2024-04-16T17:06:45.250911+0000","last_became_active":"2024-04-16T17:03:24.977434+0000","last_became_peered":"2024-04-16T17:03:24.977434+0000","last_unstale":"2024-04-16T17:06:45.250911+0000","last_undegraded":"2024-04-16T17:06:45.250911+0000","last_fullsized":"2024-04-16T17:06:45.250911+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T03:46:35.670357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"92'8","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.240206+0000","last_change":"2024-04-16T17:03:24.992225+0000","last_active":"2024-04-16T17:06:45.240206+0000","last_peered":"2024-04-16T17:06:45.240206+0000","last_clean":"2024-04-16T17:06:45.240206+0000","last_became_active":"2024-04-16T17:03:24.991904+0000","last_became_peered":"2024-04-16T17:03:24.991904+0000","last_unstale":"2024-04-16T17:06:45.240206+0000","last_undegraded":"2024-04-16T17:06:45.240206+0000","last_fullsized":"2024-04-16T17:06:45.240206+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T04:01:09.710698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"92'10","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.450979+0000","last_change":"2024-04-16T17:03:24.981322+0000","last_active":"2024-04-16T17:06:45.450979+0000","last_peered":"2024-04-16T17:06:45.450979+0000","last_clean":"2024-04-16T17:06:45.450979+0000","last_became_active":"2024-04-16T17:03:24.980920+0000","last_became_peered":"2024-04-16T17:03:24.980920+0000","last_unstale":"2024-04-16T17:06:45.450979+0000","last_undegraded":"2024-04-16T17:06:45.450979+0000","last_fullsized":"2024-04-16T17:06:45.450979+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:07:59.378512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"92'11","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.443916+0000","last_change":"2024-04-16T17:03:24.990117+0000","last_active":"2024-04-16T17:06:45.443916+0000","last_peered":"2024-04-16T17:06:45.443916+0000","last_clean":"2024-04-16T17:06:45.443916+0000","last_became_active":"2024-04-16T17:03:24.989734+0000","last_became_peered":"2024-04-16T17:03:24.989734+0000","last_unstale":"2024-04-16T17:06:45.443916+0000","last_undegraded":"2024-04-16T17:06:45.443916+0000","last_fullsized":"2024-04-16T17:06:45.443916+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T22:36:55.000702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"92'8","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-16T17:06:45.557602+0000","last_change":"2024-04-16T17:03:24.970032+0000","last_active":"2024-04-16T17:06:45.557602+0000","last_peered":"2024-04-16T17:06:45.557602+0000","last_clean":"2024-04-16T17:06:45.557602+0000","last_became_active":"2024-04-16T17:03:24.968608+0000","last_became_peered":"2024-04-16T17:03:24.968608+0000","last_unstale":"2024-04-16T17:06:45.557602+0000","last_undegraded":"2024-04-16T17:06:45.557602+0000","last_fullsized":"2024-04-16T17:06:45.557602+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-16T17:03:24.134841+0000","last_clean_scrub_stamp":"2024-04-16T17:03:24.134841+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-17T19:24:03.918418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":1771104,"data_compressed":108294,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":599984,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":1172,"num_read_kb":697,"num_write":300,"num_write_kb":1449,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":616,"num_keys_recovered":20,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":1799952,"data_compressed":80793,"data_compressed_allocated":110592,"data_compressed_original":1572864,"omap_allocated":0,"internal_metadata":0},"log_size":237,"ondisk_log_size":237,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351030,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28040,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717112,"statfs":{"total":95995035648,"available":95966322688,"internally_reserved":0,"allocated":1044480,"data_stored":532532,"data_compressed":29696,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612668,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32360,"kb_used_data":1184,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712792,"statfs":{"total":95995035648,"available":95961899008,"internally_reserved":0,"allocated":1212416,"data_stored":1326097,"data_compressed":73579,"data_compressed_allocated":98304,"data_compressed_original":802816,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776192,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32236,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712916,"statfs":{"total":95995035648,"available":95962025984,"internally_reserved":0,"allocated":1081344,"data_stored":925950,"data_compressed":48842,"data_compressed_allocated":65536,"data_compressed_original":475136,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037832,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32232,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712920,"statfs":{"total":95995035648,"available":95962030080,"internally_reserved":0,"allocated":1077248,"data_stored":925288,"data_compressed":48842,"data_compressed_allocated":65536,"data_compressed_original":475136,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299469,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36420,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708732,"statfs":{"total":95995035648,"available":95957741568,"internally_reserved":0,"allocated":1105920,"data_stored":1130719,"data_compressed":65794,"data_compressed_allocated":86016,"data_compressed_original":671744,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462994,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32192,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712960,"statfs":{"total":95995035648,"available":95962071040,"internally_reserved":0,"allocated":1036288,"data_stored":527910,"data_compressed":29696,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574959,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32264,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712888,"statfs":{"total":95995035648,"available":95961997312,"internally_reserved":0,"allocated":1110016,"data_stored":597413,"data_compressed":31325,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738483,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32208,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712944,"statfs":{"total":95995035648,"available":95962054656,"internally_reserved":0,"allocated":1048576,"data_stored":519432,"data_compressed":26684,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411433,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28120,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717032,"statfs":{"total":95995035648,"available":95966240768,"internally_reserved":0,"allocated":1126400,"data_stored":599070,"data_compressed":31325,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149795,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32244,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712908,"statfs":{"total":95995035648,"available":95962017792,"internally_reserved":0,"allocated":1093632,"data_stored":1052920,"data_compressed":54998,"data_compressed_allocated":73728,"data_compressed_original":598016,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888158,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1048576,"data_stored":534184,"data_compressed":29696,"data_compressed_allocated":40960,"data_compressed_original":81920,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724633,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28124,"kb_used_data":1108,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717028,"statfs":{"total":95995035648,"available":95966236672,"internally_reserved":0,"allocated":1134592,"data_stored":1256283,"data_compressed":71950,"data_compressed_allocated":94208,"data_compressed_original":794624,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36098,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36098,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36098,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":69593,"data_compressed":1629,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":71250,"data_compressed":1629,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":525100,"data_compressed":25302,"data_compressed_allocated":32768,"data_compressed_original":516096,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":2268,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":133999,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":4339,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":393372,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":394034,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":126976,"data_stored":203813,"data_compressed":7785,"data_compressed_allocated":12288,"data_compressed_original":131072,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-16T17:06:51.270 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-16T17:06:51.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:51.503+0000 7f6cd6a00640 1 -- 172.21.15.45:0/3720549357 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6cd0105400 msgr2=0x7f6cd01077f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:51.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:51.503+0000 7f6cd6a00640 1 --2- 172.21.15.45:0/3720549357 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6cd0105400 0x7f6cd01077f0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f6cbc009900 tx=0x7f6cbc02bc40 comp rx=0 tx=0).stop 2024-04-16T17:06:51.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:51.503+0000 7f6cd6a00640 1 -- 172.21.15.45:0/3720549357 shutdown_connections 2024-04-16T17:06:51.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:51.503+0000 7f6cd6a00640 1 --2- 172.21.15.45:0/3720549357 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6cd0107d30 0x7f6cd010a180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:51.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:51.503+0000 7f6cd6a00640 1 --2- 172.21.15.45:0/3720549357 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6cd0105400 0x7f6cd01077f0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:51.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:51.503+0000 7f6cd6a00640 1 --2- 172.21.15.45:0/3720549357 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6cd0102ad0 0x7f6cd0104ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:51.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:51.503+0000 7f6cd6a00640 1 -- 172.21.15.45:0/3720549357 >> 172.21.15.45:0/3720549357 conn(0x7f6cd00fc4e0 msgr2=0x7f6cd00fe900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:06:51.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:51.503+0000 7f6cd6a00640 1 -- 172.21.15.45:0/3720549357 shutdown_connections 2024-04-16T17:06:51.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:51.503+0000 7f6cd6a00640 1 -- 172.21.15.45:0/3720549357 wait complete. 2024-04-16T17:06:51.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:51.504+0000 7f6cd6a00640 1 Processor -- start 2024-04-16T17:06:51.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:51.505+0000 7f6cd6a00640 1 -- start start 2024-04-16T17:06:51.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:51.505+0000 7f6cd6a00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6cd0102ad0 0x7f6cd019a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:51.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:51.505+0000 7f6cd6a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6cd0105400 0x7f6cd019ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:51.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:51.505+0000 7f6cd6a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6cd0107d30 0x7f6cd01a1b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:51.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:51.505+0000 7f6cd6a00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f6cd010dcf0 con 0x7f6cd0105400 2024-04-16T17:06:51.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:51.505+0000 7f6cd6a00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f6cd010db70 con 0x7f6cd0102ad0 2024-04-16T17:06:51.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:51.505+0000 7f6cd6a00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f6cd010de70 con 0x7f6cd0107d30 2024-04-16T17:06:51.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:51.505+0000 7f6ccfe00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6cd0102ad0 0x7f6cd019a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:51.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:51.505+0000 7f6ccfe00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6cd0102ad0 0x7f6cd019a5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:33386/0 (socket says 172.21.15.45:33386) 2024-04-16T17:06:51.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:51.505+0000 7f6ccfe00640 1 -- 172.21.15.45:0/2945687729 learned_addr learned my addr 172.21.15.45:0/2945687729 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:06:51.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:51.505+0000 7f6ccf400640 1 --2- 172.21.15.45:0/2945687729 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6cd0105400 0x7f6cd019ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:51.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:51.505+0000 7f6cd4a00640 1 --2- 172.21.15.45:0/2945687729 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6cd0107d30 0x7f6cd01a1b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:51.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:51.506+0000 7f6ccfe00640 1 -- 172.21.15.45:0/2945687729 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6cd0107d30 msgr2=0x7f6cd01a1b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:51.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:51.506+0000 7f6ccfe00640 1 --2- 172.21.15.45:0/2945687729 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6cd0107d30 0x7f6cd01a1b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:51.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:51.506+0000 7f6ccfe00640 1 -- 172.21.15.45:0/2945687729 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6cd0105400 msgr2=0x7f6cd019ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:51.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:51.506+0000 7f6ccfe00640 1 --2- 172.21.15.45:0/2945687729 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6cd0105400 0x7f6cd019ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:51.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:51.506+0000 7f6ccfe00640 1 -- 172.21.15.45:0/2945687729 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6cd01a21e0 con 0x7f6cd0102ad0 2024-04-16T17:06:51.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:51.506+0000 7f6ccfe00640 1 --2- 172.21.15.45:0/2945687729 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6cd0102ad0 0x7f6cd019a5c0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f6cc000e9f0 tx=0x7f6cc000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:06:51.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:51.506+0000 7f6cccc00640 1 -- 172.21.15.45:0/2945687729 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cc000cd80 con 0x7f6cd0102ad0 2024-04-16T17:06:51.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:51.506+0000 7f6cccc00640 1 -- 172.21.15.45:0/2945687729 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6cc0010910 con 0x7f6cd0102ad0 2024-04-16T17:06:51.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:51.506+0000 7f6cccc00640 1 -- 172.21.15.45:0/2945687729 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cc0018990 con 0x7f6cd0102ad0 2024-04-16T17:06:51.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:51.506+0000 7f6cd6a00640 1 -- 172.21.15.45:0/2945687729 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6cd01a24d0 con 0x7f6cd0102ad0 2024-04-16T17:06:51.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:51.506+0000 7f6cd6a00640 1 -- 172.21.15.45:0/2945687729 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6cd01a29f0 con 0x7f6cd0102ad0 2024-04-16T17:06:51.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:51.508+0000 7f6cd6a00640 1 -- 172.21.15.45:0/2945687729 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6cd0109880 con 0x7f6cd0102ad0 2024-04-16T17:06:51.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:51.509+0000 7f6cccc00640 1 -- 172.21.15.45:0/2945687729 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 24) ==== 138822+0+0 (secure 0 0 0) 0x7f6cc0010ac0 con 0x7f6cd0102ad0 2024-04-16T17:06:51.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:51.511+0000 7f6cccc00640 1 --2- 172.21.15.45:0/2945687729 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f6cb40a3db0 0x7f6cb40a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:51.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:51.511+0000 7f6cccc00640 1 -- 172.21.15.45:0/2945687729 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f6cc00d1040 con 0x7f6cd0102ad0 2024-04-16T17:06:51.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:51.511+0000 7f6ccf400640 1 --2- 172.21.15.45:0/2945687729 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f6cb40a3db0 0x7f6cb40a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:51.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:51.512+0000 7f6ccf400640 1 --2- 172.21.15.45:0/2945687729 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f6cb40a3db0 0x7f6cb40a6270 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f6cbc02c180 tx=0x7f6cbc02cf90 comp rx=0 tx=0).ready entity=mgr.25411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:06:51.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:51.515+0000 7f6cccc00640 1 -- 172.21.15.45:0/2945687729 <== mon.2 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6cc009cbb0 con 0x7f6cd0102ad0 2024-04-16T17:06:51.753 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:51.753+0000 7f6cd6a00640 1 -- 172.21.15.45:0/2945687729 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6cd002d620 con 0x7f6cd0102ad0 2024-04-16T17:06:51.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:51.755+0000 7f6cccc00640 1 -- 172.21.15.45:0/2945687729 <== mon.2 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v95) ==== 74+0+25099 (secure 0 0 0) 0x7f6cc00a0eb0 con 0x7f6cd0102ad0 2024-04-16T17:06:51.756 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-16T17:06:51.756 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":95,"fsid":"b1fecd38-fc11-11ee-bc90-c7b262605968","created":"2024-04-16T16:54:02.385906+0000","modified":"2024-04-16T17:06:45.093262+0000","last_up_change":"2024-04-16T17:02:25.162624+0000","last_in_change":"2024-04-16T17:02:04.903274+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-16T16:58:06.166659+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-16T17:03:22.304198+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-16T17:03:23.616450+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-16T17:03:27.226289+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b655de2c-59f8-4fd4-ac98-0cc222229f56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":2290170881},{"type":"v1","addr":"172.21.15.45:6803","nonce":2290170881}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":2290170881},{"type":"v1","addr":"172.21.15.45:6805","nonce":2290170881}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":2290170881},{"type":"v1","addr":"172.21.15.45:6809","nonce":2290170881}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":2290170881},{"type":"v1","addr":"172.21.15.45:6807","nonce":2290170881}]},"public_addr":"172.21.15.45:6803/2290170881","cluster_addr":"172.21.15.45:6805/2290170881","heartbeat_back_addr":"172.21.15.45:6809/2290170881","heartbeat_front_addr":"172.21.15.45:6807/2290170881","state":["exists","up"]},{"osd":1,"uuid":"ebd450ec-08fd-4c3d-8b24-4e10634ddb7e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6800","nonce":1462942067},{"type":"v1","addr":"172.21.15.104:6801","nonce":1462942067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":1462942067},{"type":"v1","addr":"172.21.15.104:6803","nonce":1462942067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":1462942067},{"type":"v1","addr":"172.21.15.104:6807","nonce":1462942067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":1462942067},{"type":"v1","addr":"172.21.15.104:6805","nonce":1462942067}]},"public_addr":"172.21.15.104:6801/1462942067","cluster_addr":"172.21.15.104:6803/1462942067","heartbeat_back_addr":"172.21.15.104:6807/1462942067","heartbeat_front_addr":"172.21.15.104:6805/1462942067","state":["exists","up"]},{"osd":2,"uuid":"b318a512-4190-42a2-8104-dcc4026ad8ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":2829260735},{"type":"v1","addr":"172.21.15.117:6801","nonce":2829260735}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":2829260735},{"type":"v1","addr":"172.21.15.117:6803","nonce":2829260735}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":2829260735},{"type":"v1","addr":"172.21.15.117:6807","nonce":2829260735}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":2829260735},{"type":"v1","addr":"172.21.15.117:6805","nonce":2829260735}]},"public_addr":"172.21.15.117:6801/2829260735","cluster_addr":"172.21.15.117:6803/2829260735","heartbeat_back_addr":"172.21.15.117:6807/2829260735","heartbeat_front_addr":"172.21.15.117:6805/2829260735","state":["exists","up"]},{"osd":3,"uuid":"ec6490cb-a715-4c12-a54a-225aae26fcb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":2823712137},{"type":"v1","addr":"172.21.15.45:6811","nonce":2823712137}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":2823712137},{"type":"v1","addr":"172.21.15.45:6813","nonce":2823712137}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":2823712137},{"type":"v1","addr":"172.21.15.45:6817","nonce":2823712137}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":2823712137},{"type":"v1","addr":"172.21.15.45:6815","nonce":2823712137}]},"public_addr":"172.21.15.45:6811/2823712137","cluster_addr":"172.21.15.45:6813/2823712137","heartbeat_back_addr":"172.21.15.45:6817/2823712137","heartbeat_front_addr":"172.21.15.45:6815/2823712137","state":["exists","up"]},{"osd":4,"uuid":"5d91d568-0250-4aae-8dc2-a0e079f92062","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":1396278127},{"type":"v1","addr":"172.21.15.104:6809","nonce":1396278127}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":1396278127},{"type":"v1","addr":"172.21.15.104:6811","nonce":1396278127}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":1396278127},{"type":"v1","addr":"172.21.15.104:6815","nonce":1396278127}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":1396278127},{"type":"v1","addr":"172.21.15.104:6813","nonce":1396278127}]},"public_addr":"172.21.15.104:6809/1396278127","cluster_addr":"172.21.15.104:6811/1396278127","heartbeat_back_addr":"172.21.15.104:6815/1396278127","heartbeat_front_addr":"172.21.15.104:6813/1396278127","state":["exists","up"]},{"osd":5,"uuid":"ac510b43-6fb2-4d0f-bc5f-16ac080d9e9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":4145720047},{"type":"v1","addr":"172.21.15.117:6809","nonce":4145720047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":4145720047},{"type":"v1","addr":"172.21.15.117:6811","nonce":4145720047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":4145720047},{"type":"v1","addr":"172.21.15.117:6815","nonce":4145720047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":4145720047},{"type":"v1","addr":"172.21.15.117:6813","nonce":4145720047}]},"public_addr":"172.21.15.117:6809/4145720047","cluster_addr":"172.21.15.117:6811/4145720047","heartbeat_back_addr":"172.21.15.117:6815/4145720047","heartbeat_front_addr":"172.21.15.117:6813/4145720047","state":["exists","up"]},{"osd":6,"uuid":"a41adc18-d88a-419e-b2c2-b089c1f32472","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":3254229176},{"type":"v1","addr":"172.21.15.45:6819","nonce":3254229176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":3254229176},{"type":"v1","addr":"172.21.15.45:6821","nonce":3254229176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":3254229176},{"type":"v1","addr":"172.21.15.45:6825","nonce":3254229176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":3254229176},{"type":"v1","addr":"172.21.15.45:6823","nonce":3254229176}]},"public_addr":"172.21.15.45:6819/3254229176","cluster_addr":"172.21.15.45:6821/3254229176","heartbeat_back_addr":"172.21.15.45:6825/3254229176","heartbeat_front_addr":"172.21.15.45:6823/3254229176","state":["exists","up"]},{"osd":7,"uuid":"b71712a4-7cf6-454d-9fd1-565e3df21c1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":4151077186},{"type":"v1","addr":"172.21.15.104:6817","nonce":4151077186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":4151077186},{"type":"v1","addr":"172.21.15.104:6819","nonce":4151077186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":4151077186},{"type":"v1","addr":"172.21.15.104:6823","nonce":4151077186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":4151077186},{"type":"v1","addr":"172.21.15.104:6821","nonce":4151077186}]},"public_addr":"172.21.15.104:6817/4151077186","cluster_addr":"172.21.15.104:6819/4151077186","heartbeat_back_addr":"172.21.15.104:6823/4151077186","heartbeat_front_addr":"172.21.15.104:6821/4151077186","state":["exists","up"]},{"osd":8,"uuid":"9e572aef-59a8-450e-a92e-df9dff4d57a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":1500286830},{"type":"v1","addr":"172.21.15.117:6817","nonce":1500286830}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":1500286830},{"type":"v1","addr":"172.21.15.117:6819","nonce":1500286830}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":1500286830},{"type":"v1","addr":"172.21.15.117:6823","nonce":1500286830}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":1500286830},{"type":"v1","addr":"172.21.15.117:6821","nonce":1500286830}]},"public_addr":"172.21.15.117:6817/1500286830","cluster_addr":"172.21.15.117:6819/1500286830","heartbeat_back_addr":"172.21.15.117:6823/1500286830","heartbeat_front_addr":"172.21.15.117:6821/1500286830","state":["exists","up"]},{"osd":9,"uuid":"014e9598-c413-4cc4-9bb8-61c6e8f59d30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":3106734074},{"type":"v1","addr":"172.21.15.45:6827","nonce":3106734074}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":3106734074},{"type":"v1","addr":"172.21.15.45:6829","nonce":3106734074}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":3106734074},{"type":"v1","addr":"172.21.15.45:6833","nonce":3106734074}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":3106734074},{"type":"v1","addr":"172.21.15.45:6831","nonce":3106734074}]},"public_addr":"172.21.15.45:6827/3106734074","cluster_addr":"172.21.15.45:6829/3106734074","heartbeat_back_addr":"172.21.15.45:6833/3106734074","heartbeat_front_addr":"172.21.15.45:6831/3106734074","state":["exists","up"]},{"osd":10,"uuid":"2deba364-3e39-47be-8a1f-8256df5b82d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":1404794652},{"type":"v1","addr":"172.21.15.104:6825","nonce":1404794652}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":1404794652},{"type":"v1","addr":"172.21.15.104:6827","nonce":1404794652}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":1404794652},{"type":"v1","addr":"172.21.15.104:6831","nonce":1404794652}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":1404794652},{"type":"v1","addr":"172.21.15.104:6829","nonce":1404794652}]},"public_addr":"172.21.15.104:6825/1404794652","cluster_addr":"172.21.15.104:6827/1404794652","heartbeat_back_addr":"172.21.15.104:6831/1404794652","heartbeat_front_addr":"172.21.15.104:6829/1404794652","state":["exists","up"]},{"osd":11,"uuid":"3fda4fea-59ad-4d1a-9fe6-fcb30e31fd7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":3384003243},{"type":"v1","addr":"172.21.15.117:6825","nonce":3384003243}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":3384003243},{"type":"v1","addr":"172.21.15.117:6827","nonce":3384003243}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":3384003243},{"type":"v1","addr":"172.21.15.117:6831","nonce":3384003243}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":3384003243},{"type":"v1","addr":"172.21.15.117:6829","nonce":3384003243}]},"public_addr":"172.21.15.117:6825/3384003243","cluster_addr":"172.21.15.117:6827/3384003243","heartbeat_back_addr":"172.21.15.117:6831/3384003243","heartbeat_front_addr":"172.21.15.117:6829/3384003243","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:57:14.799608+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:57:39.884668+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:58:03.327057+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:58:37.314531+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:59:03.667650+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:59:28.462823+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T17:00:02.968574+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T17:00:29.559190+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T17:00:55.291007+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T17:01:29.815977+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T17:01:57.015580+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T17:02:23.233151+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.5","mappings":[{"from":9,"to":10}]},{"pgid":"4.7","mappings":[{"from":5,"to":8}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.45:6800/268030699":"2024-04-17T17:06:45.093215+0000","172.21.15.45:6801/268030699":"2024-04-17T17:06:45.093215+0000","172.21.15.45:0/4074982598":"2024-04-17T16:55:19.948001+0000","172.21.15.45:0/1915281176":"2024-04-17T17:06:45.093215+0000","172.21.15.45:0/2443572576":"2024-04-17T16:55:19.948001+0000","172.21.15.45:0/1500147666":"2024-04-17T17:06:45.093215+0000","172.21.15.45:6801/1983630136":"2024-04-17T16:55:19.948001+0000","172.21.15.45:6800/1983630136":"2024-04-17T16:55:19.948001+0000","172.21.15.45:0/12001708":"2024-04-17T17:06:45.093215+0000","172.21.15.45:0/3150465248":"2024-04-17T16:54:43.692466+0000","172.21.15.45:0/556738623":"2024-04-17T16:54:43.692466+0000","172.21.15.45:0/445676297":"2024-04-17T16:55:19.948001+0000","172.21.15.45:6800/2553659622":"2024-04-17T16:54:43.692466+0000","172.21.15.45:6801/2553659622":"2024-04-17T16:54:43.692466+0000","172.21.15.45:0/265066921":"2024-04-17T17:06:45.093215+0000","172.21.15.45:0/1184615408":"2024-04-17T16:54:43.692466+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-16T17:06:51.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:51.757+0000 7f6cd6a00640 1 -- 172.21.15.45:0/2945687729 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f6cb40a3db0 msgr2=0x7f6cb40a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:51.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:51.757+0000 7f6cd6a00640 1 --2- 172.21.15.45:0/2945687729 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f6cb40a3db0 0x7f6cb40a6270 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f6cbc02c180 tx=0x7f6cbc02cf90 comp rx=0 tx=0).stop 2024-04-16T17:06:51.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:51.758+0000 7f6cd6a00640 1 -- 172.21.15.45:0/2945687729 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6cd0102ad0 msgr2=0x7f6cd019a5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:51.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:51.758+0000 7f6cd6a00640 1 --2- 172.21.15.45:0/2945687729 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6cd0102ad0 0x7f6cd019a5c0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f6cc000e9f0 tx=0x7f6cc000eeb0 comp rx=0 tx=0).stop 2024-04-16T17:06:51.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:51.758+0000 7f6cd6a00640 1 -- 172.21.15.45:0/2945687729 shutdown_connections 2024-04-16T17:06:51.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:51.758+0000 7f6cd6a00640 1 --2- 172.21.15.45:0/2945687729 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f6cb40a3db0 0x7f6cb40a6270 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:51.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:51.758+0000 7f6cd6a00640 1 --2- 172.21.15.45:0/2945687729 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6cd0107d30 0x7f6cd01a1b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:51.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:51.758+0000 7f6cd6a00640 1 --2- 172.21.15.45:0/2945687729 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6cd0105400 0x7f6cd019ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:51.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:51.758+0000 7f6cd6a00640 1 --2- 172.21.15.45:0/2945687729 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6cd0102ad0 0x7f6cd019a5c0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:51.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:51.758+0000 7f6cd6a00640 1 -- 172.21.15.45:0/2945687729 >> 172.21.15.45:0/2945687729 conn(0x7f6cd00fc4e0 msgr2=0x7f6cd01022b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:06:51.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:51.758+0000 7f6cd6a00640 1 -- 172.21.15.45:0/2945687729 shutdown_connections 2024-04-16T17:06:51.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:51.758+0000 7f6cd6a00640 1 -- 172.21.15.45:0/2945687729 wait complete. 2024-04-16T17:06:51.777 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-16T16:58:06.166659+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-16T17:03:22.304198+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-16T17:03:23.616450+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-16T17:03:27.226289+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-16T17:06:51.777 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-16T17:06:51.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:06:51 smithi104 ceph-mon[36457]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:06:51.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:06:51 smithi104 ceph-mon[36457]: from='mgr.25411 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:06:51.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:06:51 smithi104 ceph-mon[36457]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:06:51.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:06:51 smithi104 ceph-mon[36457]: from='mgr.25411 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:06:51.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:06:51 smithi104 ceph-mon[36457]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:06:51.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:06:51 smithi104 ceph-mon[36457]: from='mgr.25411 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:06:51.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:06:51 smithi104 ceph-mon[36457]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:06:51.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:06:51 smithi104 ceph-mon[36457]: from='mgr.25411 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:06:51.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:06:51 smithi104 ceph-mon[36457]: Adjusting osd_memory_target on smithi045 to 177.5M 2024-04-16T17:06:51.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:06:51 smithi104 ceph-mon[36457]: Unable to set osd_memory_target on smithi045 to 186173030: error parsing value: Value '186173030' is below minimum 939524096 2024-04-16T17:06:51.929 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:06:51 smithi104 ceph-mon[36457]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:06:51.929 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:06:51 smithi104 ceph-mon[36457]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T17:06:51.929 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:06:51 smithi104 ceph-mon[36457]: Updating smithi045:/etc/ceph/ceph.conf 2024-04-16T17:06:51.929 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:06:51 smithi104 ceph-mon[36457]: Updating smithi104:/etc/ceph/ceph.conf 2024-04-16T17:06:51.929 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:06:51 smithi104 ceph-mon[36457]: Updating smithi117:/etc/ceph/ceph.conf 2024-04-16T17:06:51.929 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:06:51 smithi104 ceph-mon[36457]: from='client.15432 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-16T17:06:51.929 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:06:51 smithi104 ceph-mon[36457]: Updating smithi104:/var/lib/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/config/ceph.conf 2024-04-16T17:06:51.929 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:06:51 smithi104 ceph-mon[36457]: Updating smithi117:/var/lib/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/config/ceph.conf 2024-04-16T17:06:51.929 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:06:51 smithi104 ceph-mon[36457]: pgmap v6: 129 pgs: 129 active+clean; 1.1 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s rd, 205 B/s wr, 1 op/s 2024-04-16T17:06:51.929 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:06:51 smithi104 ceph-mon[36457]: Updating smithi104:/etc/ceph/ceph.client.admin.keyring 2024-04-16T17:06:51.929 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:06:51 smithi104 ceph-mon[36457]: from='client.25565 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-16T17:06:51.929 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:06:51 smithi104 ceph-mon[36457]: Updating smithi117:/etc/ceph/ceph.client.admin.keyring 2024-04-16T17:06:51.929 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:06:51 smithi104 ceph-mon[36457]: Updating smithi045:/var/lib/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/config/ceph.conf 2024-04-16T17:06:51.929 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:06:51 smithi104 ceph-mon[36457]: Updating smithi104:/var/lib/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/config/ceph.client.admin.keyring 2024-04-16T17:06:51.929 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:06:51 smithi104 ceph-mon[36457]: Updating smithi117:/var/lib/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/config/ceph.client.admin.keyring 2024-04-16T17:06:51.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:51.969+0000 7f1a37e00640 1 -- 172.21.15.45:0/2221977076 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1a30105430 msgr2=0x7f1a301058b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:51.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:51.969+0000 7f1a37e00640 1 --2- 172.21.15.45:0/2221977076 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1a30105430 0x7f1a301058b0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f1a1c009860 tx=0x7f1a1c02bb70 comp rx=0 tx=0).stop 2024-04-16T17:06:51.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:51.969+0000 7f1a37e00640 1 -- 172.21.15.45:0/2221977076 shutdown_connections 2024-04-16T17:06:51.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:51.969+0000 7f1a37e00640 1 --2- 172.21.15.45:0/2221977076 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1a30105df0 0x7f1a3010c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:51.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:51.969+0000 7f1a37e00640 1 --2- 172.21.15.45:0/2221977076 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1a30105430 0x7f1a301058b0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:51.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:51.969+0000 7f1a37e00640 1 --2- 172.21.15.45:0/2221977076 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1a30104240 0x7f1a30104660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:51.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:51.969+0000 7f1a37e00640 1 -- 172.21.15.45:0/2221977076 >> 172.21.15.45:0/2221977076 conn(0x7f1a300ff7f0 msgr2=0x7f1a30101c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:06:51.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:51.969+0000 7f1a37e00640 1 -- 172.21.15.45:0/2221977076 shutdown_connections 2024-04-16T17:06:51.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:51.969+0000 7f1a37e00640 1 -- 172.21.15.45:0/2221977076 wait complete. 2024-04-16T17:06:51.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:51.970+0000 7f1a37e00640 1 Processor -- start 2024-04-16T17:06:51.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:51.970+0000 7f1a37e00640 1 -- start start 2024-04-16T17:06:51.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:51.971+0000 7f1a37e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1a30104240 0x7f1a3019a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:51.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:51.971+0000 7f1a37e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1a30105430 0x7f1a3019aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:51.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:51.971+0000 7f1a37e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1a30105df0 0x7f1a301a1ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:51.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:51.971+0000 7f1a37e00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f1a3010fed0 con 0x7f1a30105df0 2024-04-16T17:06:51.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:51.971+0000 7f1a37e00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f1a3010fd50 con 0x7f1a30104240 2024-04-16T17:06:51.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:51.971+0000 7f1a37e00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f1a30110050 con 0x7f1a30105430 2024-04-16T17:06:51.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:51.971+0000 7f1a35e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1a30105df0 0x7f1a301a1ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:51.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:51.971+0000 7f1a34a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1a30105430 0x7f1a3019aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:51.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:51.971+0000 7f1a35400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1a30104240 0x7f1a3019a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:51.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:51.971+0000 7f1a34a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1a30105430 0x7f1a3019aa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:35656/0 (socket says 172.21.15.45:35656) 2024-04-16T17:06:51.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:51.971+0000 7f1a34a00640 1 -- 172.21.15.45:0/234718447 learned_addr learned my addr 172.21.15.45:0/234718447 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:06:51.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:51.971+0000 7f1a34a00640 1 -- 172.21.15.45:0/234718447 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1a30104240 msgr2=0x7f1a3019a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:51.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:51.971+0000 7f1a34a00640 1 --2- 172.21.15.45:0/234718447 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1a30104240 0x7f1a3019a530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:51.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:51.971+0000 7f1a34a00640 1 -- 172.21.15.45:0/234718447 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1a30105df0 msgr2=0x7f1a301a1ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:51.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:51.971+0000 7f1a34a00640 1 --2- 172.21.15.45:0/234718447 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1a30105df0 0x7f1a301a1ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:51.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:51.971+0000 7f1a34a00640 1 -- 172.21.15.45:0/234718447 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a301a2150 con 0x7f1a30105430 2024-04-16T17:06:51.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:51.972+0000 7f1a34a00640 1 --2- 172.21.15.45:0/234718447 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1a30105430 0x7f1a3019aa70 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f1a1c005940 tx=0x7f1a1c032470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:06:51.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:51.972+0000 7f1a2e000640 1 -- 172.21.15.45:0/234718447 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a1c041020 con 0x7f1a30105430 2024-04-16T17:06:51.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:51.972+0000 7f1a2e000640 1 -- 172.21.15.45:0/234718447 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1a1c02c9f0 con 0x7f1a30105430 2024-04-16T17:06:51.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:51.972+0000 7f1a2e000640 1 -- 172.21.15.45:0/234718447 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a1c039040 con 0x7f1a30105430 2024-04-16T17:06:51.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:51.972+0000 7f1a37e00640 1 -- 172.21.15.45:0/234718447 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a301a23e0 con 0x7f1a30105430 2024-04-16T17:06:51.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:51.972+0000 7f1a37e00640 1 -- 172.21.15.45:0/234718447 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a301a28d0 con 0x7f1a30105430 2024-04-16T17:06:51.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:51.973+0000 7f1a37e00640 1 -- 172.21.15.45:0/234718447 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a3010b560 con 0x7f1a30105430 2024-04-16T17:06:51.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:51.976+0000 7f1a2e000640 1 -- 172.21.15.45:0/234718447 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 24) ==== 138822+0+0 (secure 0 0 0) 0x7f1a1c044020 con 0x7f1a30105430 2024-04-16T17:06:51.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:06:51 smithi045 ceph-mon[31676]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:06:51.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:06:51 smithi045 ceph-mon[31676]: from='mgr.25411 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:06:51.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:06:51 smithi045 ceph-mon[31676]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:06:51.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:06:51 smithi045 ceph-mon[31676]: from='mgr.25411 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:06:51.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:06:51 smithi045 ceph-mon[31676]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:06:51.978 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:06:51 smithi045 ceph-mon[31676]: from='mgr.25411 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:06:51.978 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:06:51 smithi045 ceph-mon[31676]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:06:51.978 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:06:51 smithi045 ceph-mon[31676]: from='mgr.25411 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:06:51.978 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:06:51 smithi045 ceph-mon[31676]: Adjusting osd_memory_target on smithi045 to 177.5M 2024-04-16T17:06:51.978 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:06:51 smithi045 ceph-mon[31676]: Unable to set osd_memory_target on smithi045 to 186173030: error parsing value: Value '186173030' is below minimum 939524096 2024-04-16T17:06:51.978 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:06:51 smithi045 ceph-mon[31676]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:06:51.978 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:06:51 smithi045 ceph-mon[31676]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T17:06:51.978 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:06:51 smithi045 ceph-mon[31676]: Updating smithi045:/etc/ceph/ceph.conf 2024-04-16T17:06:51.978 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:06:51 smithi045 ceph-mon[31676]: Updating smithi104:/etc/ceph/ceph.conf 2024-04-16T17:06:51.978 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:06:51 smithi045 ceph-mon[31676]: Updating smithi117:/etc/ceph/ceph.conf 2024-04-16T17:06:51.978 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:06:51 smithi045 ceph-mon[31676]: from='client.15432 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-16T17:06:51.978 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:06:51 smithi045 ceph-mon[31676]: Updating smithi104:/var/lib/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/config/ceph.conf 2024-04-16T17:06:51.978 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:06:51 smithi045 ceph-mon[31676]: Updating smithi117:/var/lib/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/config/ceph.conf 2024-04-16T17:06:51.979 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:06:51 smithi045 ceph-mon[31676]: pgmap v6: 129 pgs: 129 active+clean; 1.1 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s rd, 205 B/s wr, 1 op/s 2024-04-16T17:06:51.979 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:06:51 smithi045 ceph-mon[31676]: Updating smithi104:/etc/ceph/ceph.client.admin.keyring 2024-04-16T17:06:51.979 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:06:51 smithi045 ceph-mon[31676]: from='client.25565 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-16T17:06:51.979 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:06:51 smithi045 ceph-mon[31676]: Updating smithi117:/etc/ceph/ceph.client.admin.keyring 2024-04-16T17:06:51.979 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:06:51 smithi045 ceph-mon[31676]: Updating smithi045:/var/lib/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/config/ceph.conf 2024-04-16T17:06:51.979 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:06:51 smithi045 ceph-mon[31676]: Updating smithi104:/var/lib/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/config/ceph.client.admin.keyring 2024-04-16T17:06:51.979 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:06:51 smithi045 ceph-mon[31676]: Updating smithi117:/var/lib/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/config/ceph.client.admin.keyring 2024-04-16T17:06:51.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:51.977+0000 7f1a2e000640 1 --2- 172.21.15.45:0/234718447 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f1a140a3db0 0x7f1a140a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:51.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:51.978+0000 7f1a2e000640 1 -- 172.21.15.45:0/234718447 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f1a1c0f2580 con 0x7f1a30105430 2024-04-16T17:06:51.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:51.978+0000 7f1a35400640 1 --2- 172.21.15.45:0/234718447 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f1a140a3db0 0x7f1a140a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:51.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:51.978+0000 7f1a35400640 1 --2- 172.21.15.45:0/234718447 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f1a140a3db0 0x7f1a140a6270 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f1a30105290 tx=0x7f1a28009500 comp rx=0 tx=0).ready entity=mgr.25411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:06:51.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:51.982+0000 7f1a2e000640 1 -- 172.21.15.45:0/234718447 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1a1c04b020 con 0x7f1a30105430 2024-04-16T17:06:52.055 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:06:51 smithi117 ceph-mon[36730]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:06:52.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:06:51 smithi117 ceph-mon[36730]: from='mgr.25411 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:06:52.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:06:51 smithi117 ceph-mon[36730]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:06:52.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:06:51 smithi117 ceph-mon[36730]: from='mgr.25411 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:06:52.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:06:51 smithi117 ceph-mon[36730]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:06:52.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:06:51 smithi117 ceph-mon[36730]: from='mgr.25411 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:06:52.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:06:51 smithi117 ceph-mon[36730]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:06:52.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:06:51 smithi117 ceph-mon[36730]: from='mgr.25411 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:06:52.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:06:51 smithi117 ceph-mon[36730]: Adjusting osd_memory_target on smithi045 to 177.5M 2024-04-16T17:06:52.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:06:51 smithi117 ceph-mon[36730]: Unable to set osd_memory_target on smithi045 to 186173030: error parsing value: Value '186173030' is below minimum 939524096 2024-04-16T17:06:52.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:06:51 smithi117 ceph-mon[36730]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:06:52.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:06:51 smithi117 ceph-mon[36730]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T17:06:52.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:06:51 smithi117 ceph-mon[36730]: Updating smithi045:/etc/ceph/ceph.conf 2024-04-16T17:06:52.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:06:51 smithi117 ceph-mon[36730]: Updating smithi104:/etc/ceph/ceph.conf 2024-04-16T17:06:52.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:06:51 smithi117 ceph-mon[36730]: Updating smithi117:/etc/ceph/ceph.conf 2024-04-16T17:06:52.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:06:51 smithi117 ceph-mon[36730]: from='client.15432 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-16T17:06:52.057 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:06:51 smithi117 ceph-mon[36730]: Updating smithi104:/var/lib/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/config/ceph.conf 2024-04-16T17:06:52.057 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:06:51 smithi117 ceph-mon[36730]: Updating smithi117:/var/lib/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/config/ceph.conf 2024-04-16T17:06:52.057 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:06:51 smithi117 ceph-mon[36730]: pgmap v6: 129 pgs: 129 active+clean; 1.1 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s rd, 205 B/s wr, 1 op/s 2024-04-16T17:06:52.057 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:06:51 smithi117 ceph-mon[36730]: Updating smithi104:/etc/ceph/ceph.client.admin.keyring 2024-04-16T17:06:52.057 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:06:51 smithi117 ceph-mon[36730]: from='client.25565 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-16T17:06:52.057 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:06:51 smithi117 ceph-mon[36730]: Updating smithi117:/etc/ceph/ceph.client.admin.keyring 2024-04-16T17:06:52.057 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:06:51 smithi117 ceph-mon[36730]: Updating smithi045:/var/lib/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/config/ceph.conf 2024-04-16T17:06:52.057 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:06:51 smithi117 ceph-mon[36730]: Updating smithi104:/var/lib/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/config/ceph.client.admin.keyring 2024-04-16T17:06:52.057 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:06:51 smithi117 ceph-mon[36730]: Updating smithi117:/var/lib/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/config/ceph.client.admin.keyring 2024-04-16T17:06:52.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:52.206+0000 7f1a37e00640 1 -- 172.21.15.45:0/234718447 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f1a3002d870 con 0x7f1a30105430 2024-04-16T17:06:52.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:52.207+0000 7f1a2e000640 1 -- 172.21.15.45:0/234718447 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v95) ==== 93+0+10 (secure 0 0 0) 0x7f1a1c0be810 con 0x7f1a30105430 2024-04-16T17:06:52.207 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 1 2024-04-16T17:06:52.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:52.209+0000 7f1a37e00640 1 -- 172.21.15.45:0/234718447 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f1a140a3db0 msgr2=0x7f1a140a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:52.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:52.209+0000 7f1a37e00640 1 --2- 172.21.15.45:0/234718447 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f1a140a3db0 0x7f1a140a6270 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f1a30105290 tx=0x7f1a28009500 comp rx=0 tx=0).stop 2024-04-16T17:06:52.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:52.209+0000 7f1a37e00640 1 -- 172.21.15.45:0/234718447 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1a30105430 msgr2=0x7f1a3019aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:52.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:52.209+0000 7f1a37e00640 1 --2- 172.21.15.45:0/234718447 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1a30105430 0x7f1a3019aa70 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f1a1c005940 tx=0x7f1a1c032470 comp rx=0 tx=0).stop 2024-04-16T17:06:52.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:52.209+0000 7f1a37e00640 1 -- 172.21.15.45:0/234718447 shutdown_connections 2024-04-16T17:06:52.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:52.209+0000 7f1a37e00640 1 --2- 172.21.15.45:0/234718447 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f1a140a3db0 0x7f1a140a6270 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:52.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:52.209+0000 7f1a37e00640 1 --2- 172.21.15.45:0/234718447 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1a30105430 0x7f1a3019aa70 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:52.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:52.209+0000 7f1a37e00640 1 --2- 172.21.15.45:0/234718447 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1a30105df0 0x7f1a301a1ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:52.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:52.209+0000 7f1a37e00640 1 --2- 172.21.15.45:0/234718447 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1a30104240 0x7f1a3019a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:52.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:52.209+0000 7f1a37e00640 1 -- 172.21.15.45:0/234718447 >> 172.21.15.45:0/234718447 conn(0x7f1a300ff7f0 msgr2=0x7f1a30108730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:06:52.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:52.209+0000 7f1a37e00640 1 -- 172.21.15.45:0/234718447 shutdown_connections 2024-04-16T17:06:52.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:52.209+0000 7f1a37e00640 1 -- 172.21.15.45:0/234718447 wait complete. 2024-04-16T17:06:52.228 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-16T17:06:52.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:52.421+0000 7f079a400640 1 -- 172.21.15.45:0/3503151763 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f07941008b0 msgr2=0x7f0794102ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:52.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:52.421+0000 7f079a400640 1 --2- 172.21.15.45:0/3503151763 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f07941008b0 0x7f0794102ce0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f0784009860 tx=0x7f078402bb70 comp rx=0 tx=0).stop 2024-04-16T17:06:52.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:52.422+0000 7f079a400640 1 -- 172.21.15.45:0/3503151763 shutdown_connections 2024-04-16T17:06:52.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:52.422+0000 7f079a400640 1 --2- 172.21.15.45:0/3503151763 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0794103220 0x7f0794105690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:52.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:52.422+0000 7f079a400640 1 --2- 172.21.15.45:0/3503151763 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0794105bd0 0x7f0794106050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:52.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:52.422+0000 7f079a400640 1 --2- 172.21.15.45:0/3503151763 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f07941008b0 0x7f0794102ce0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:52.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:52.422+0000 7f079a400640 1 -- 172.21.15.45:0/3503151763 >> 172.21.15.45:0/3503151763 conn(0x7f07940fc4b0 msgr2=0x7f07940fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:06:52.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:52.422+0000 7f079a400640 1 -- 172.21.15.45:0/3503151763 shutdown_connections 2024-04-16T17:06:52.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:52.422+0000 7f079a400640 1 -- 172.21.15.45:0/3503151763 wait complete. 2024-04-16T17:06:52.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:52.423+0000 7f079a400640 1 Processor -- start 2024-04-16T17:06:52.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:52.423+0000 7f079a400640 1 -- start start 2024-04-16T17:06:52.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:52.423+0000 7f079a400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f07941008b0 0x7f079419e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:52.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:52.423+0000 7f079a400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0794103220 0x7f079419ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:52.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:52.423+0000 7f079a400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0794105bd0 0x7f07941a5f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:52.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:52.423+0000 7f079a400640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f07941142a0 con 0x7f0794103220 2024-04-16T17:06:52.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:52.423+0000 7f079a400640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f0794114120 con 0x7f0794105bd0 2024-04-16T17:06:52.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:52.423+0000 7f079a400640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f0794114420 con 0x7f07941008b0 2024-04-16T17:06:52.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:52.423+0000 7f0793400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f07941008b0 0x7f079419e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:52.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:52.423+0000 7f078be00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0794103220 0x7f079419ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:52.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:52.423+0000 7f0793e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0794105bd0 0x7f07941a5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:52.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:52.423+0000 7f0793400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f07941008b0 0x7f079419e9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:35682/0 (socket says 172.21.15.45:35682) 2024-04-16T17:06:52.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:52.423+0000 7f0793400640 1 -- 172.21.15.45:0/575644021 learned_addr learned my addr 172.21.15.45:0/575644021 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:06:52.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:52.424+0000 7f0793e00640 1 -- 172.21.15.45:0/575644021 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f07941008b0 msgr2=0x7f079419e9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:52.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:52.424+0000 7f0793e00640 1 --2- 172.21.15.45:0/575644021 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f07941008b0 0x7f079419e9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:52.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:52.424+0000 7f0793e00640 1 -- 172.21.15.45:0/575644021 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0794103220 msgr2=0x7f079419ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:52.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:52.424+0000 7f0793e00640 1 --2- 172.21.15.45:0/575644021 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0794103220 0x7f079419ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:52.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:52.424+0000 7f0793e00640 1 -- 172.21.15.45:0/575644021 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07941a65b0 con 0x7f0794105bd0 2024-04-16T17:06:52.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:52.424+0000 7f0793400640 1 --2- 172.21.15.45:0/575644021 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f07941008b0 0x7f079419e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:06:52.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:52.424+0000 7f0793e00640 1 --2- 172.21.15.45:0/575644021 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0794105bd0 0x7f07941a5f40 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f078c00efc0 tx=0x7f078c00c800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:06:52.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:52.424+0000 7f0790c00640 1 -- 172.21.15.45:0/575644021 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f078c003970 con 0x7f0794105bd0 2024-04-16T17:06:52.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:52.424+0000 7f0790c00640 1 -- 172.21.15.45:0/575644021 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f078c004020 con 0x7f0794105bd0 2024-04-16T17:06:52.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:52.424+0000 7f0790c00640 1 -- 172.21.15.45:0/575644021 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f078c00b9f0 con 0x7f0794105bd0 2024-04-16T17:06:52.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:52.425+0000 7f079a400640 1 -- 172.21.15.45:0/575644021 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07941a68a0 con 0x7f0794105bd0 2024-04-16T17:06:52.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:52.425+0000 7f079a400640 1 -- 172.21.15.45:0/575644021 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07941a6df0 con 0x7f0794105bd0 2024-04-16T17:06:52.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:52.426+0000 7f079a400640 1 -- 172.21.15.45:0/575644021 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f07940fdfa0 con 0x7f0794105bd0 2024-04-16T17:06:52.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:52.428+0000 7f0790c00640 1 -- 172.21.15.45:0/575644021 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 24) ==== 138822+0+0 (secure 0 0 0) 0x7f078c01d020 con 0x7f0794105bd0 2024-04-16T17:06:52.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:52.430+0000 7f0790c00640 1 --2- 172.21.15.45:0/575644021 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f07780a3e30 0x7f07780a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:52.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:52.430+0000 7f0790c00640 1 -- 172.21.15.45:0/575644021 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f078c0d0b90 con 0x7f0794105bd0 2024-04-16T17:06:52.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:52.430+0000 7f0793400640 1 --2- 172.21.15.45:0/575644021 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f07780a3e30 0x7f07780a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:52.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:52.431+0000 7f0793400640 1 --2- 172.21.15.45:0/575644021 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f07780a3e30 0x7f07780a62f0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f0784035fd0 tx=0x7f078402e040 comp rx=0 tx=0).ready entity=mgr.25411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:06:52.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:52.434+0000 7f0790c00640 1 -- 172.21.15.45:0/575644021 <== mon.2 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f078c09cd10 con 0x7f0794105bd0 2024-04-16T17:06:52.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:52.672+0000 7f079a400640 1 -- 172.21.15.45:0/575644021 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f0794062c00 con 0x7f0794105bd0 2024-04-16T17:06:52.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:52.673+0000 7f0790c00640 1 -- 172.21.15.45:0/575644021 <== mon.2 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v95) ==== 104+0+11 (secure 0 0 0) 0x7f078c09c610 con 0x7f0794105bd0 2024-04-16T17:06:52.673 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 32 2024-04-16T17:06:52.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:52.676+0000 7f079a400640 1 -- 172.21.15.45:0/575644021 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f07780a3e30 msgr2=0x7f07780a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:52.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:52.676+0000 7f079a400640 1 --2- 172.21.15.45:0/575644021 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f07780a3e30 0x7f07780a62f0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f0784035fd0 tx=0x7f078402e040 comp rx=0 tx=0).stop 2024-04-16T17:06:52.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:52.676+0000 7f079a400640 1 -- 172.21.15.45:0/575644021 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0794105bd0 msgr2=0x7f07941a5f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:52.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:52.676+0000 7f079a400640 1 --2- 172.21.15.45:0/575644021 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0794105bd0 0x7f07941a5f40 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f078c00efc0 tx=0x7f078c00c800 comp rx=0 tx=0).stop 2024-04-16T17:06:52.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:52.676+0000 7f079a400640 1 -- 172.21.15.45:0/575644021 shutdown_connections 2024-04-16T17:06:52.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:52.676+0000 7f079a400640 1 --2- 172.21.15.45:0/575644021 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f07780a3e30 0x7f07780a62f0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:52.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:52.676+0000 7f079a400640 1 --2- 172.21.15.45:0/575644021 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0794105bd0 0x7f07941a5f40 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:52.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:52.676+0000 7f079a400640 1 --2- 172.21.15.45:0/575644021 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0794103220 0x7f079419ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:52.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:52.676+0000 7f079a400640 1 --2- 172.21.15.45:0/575644021 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f07941008b0 0x7f079419e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:52.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:52.676+0000 7f079a400640 1 -- 172.21.15.45:0/575644021 >> 172.21.15.45:0/575644021 conn(0x7f07940fc4b0 msgr2=0x7f0794103b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:06:52.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:52.676+0000 7f079a400640 1 -- 172.21.15.45:0/575644021 shutdown_connections 2024-04-16T17:06:52.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:52.676+0000 7f079a400640 1 -- 172.21.15.45:0/575644021 wait complete. 2024-04-16T17:06:52.695 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-16T17:06:52.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:52.892+0000 7f1390000640 1 -- 172.21.15.45:0/1317049456 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f13881044b0 msgr2=0x7f1388104890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:52.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:52.892+0000 7f1390000640 1 --2- 172.21.15.45:0/1317049456 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f13881044b0 0x7f1388104890 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f137c009860 tx=0x7f137c02bb70 comp rx=0 tx=0).stop 2024-04-16T17:06:52.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:52.892+0000 7f1390000640 1 -- 172.21.15.45:0/1317049456 shutdown_connections 2024-04-16T17:06:52.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:52.893+0000 7f1390000640 1 --2- 172.21.15.45:0/1317049456 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1388104e60 0x7f1388108ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:52.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:52.893+0000 7f1390000640 1 --2- 172.21.15.45:0/1317049456 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1388109580 0x7f1388111150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:52.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:52.893+0000 7f1390000640 1 --2- 172.21.15.45:0/1317049456 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f13881044b0 0x7f1388104890 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:52.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:52.893+0000 7f1390000640 1 -- 172.21.15.45:0/1317049456 >> 172.21.15.45:0/1317049456 conn(0x7f13880ffd90 msgr2=0x7f13881021b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:06:52.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:52.893+0000 7f1390000640 1 -- 172.21.15.45:0/1317049456 shutdown_connections 2024-04-16T17:06:52.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:52.893+0000 7f1390000640 1 -- 172.21.15.45:0/1317049456 wait complete. 2024-04-16T17:06:52.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:52.894+0000 7f1390000640 1 Processor -- start 2024-04-16T17:06:52.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:52.894+0000 7f1390000640 1 -- start start 2024-04-16T17:06:52.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:52.894+0000 7f1390000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f13881044b0 0x7f13881a07b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:52.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:52.894+0000 7f1390000640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1388109580 0x7f13881a0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:52.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:52.894+0000 7f1390000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f138819a9b0 0x7f138819ae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:52.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:52.894+0000 7f1390000640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f1388114610 con 0x7f13881044b0 2024-04-16T17:06:52.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:52.894+0000 7f1390000640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f1388114490 con 0x7f1388109580 2024-04-16T17:06:52.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:52.894+0000 7f1390000640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f1388114790 con 0x7f138819a9b0 2024-04-16T17:06:52.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:52.895+0000 7f138cc00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1388109580 0x7f13881a0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:52.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:52.895+0000 7f138d600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f13881044b0 0x7f13881a07b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:52.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:52.895+0000 7f138cc00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1388109580 0x7f13881a0cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:33438/0 (socket says 172.21.15.45:33438) 2024-04-16T17:06:52.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:52.895+0000 7f138cc00640 1 -- 172.21.15.45:0/603136398 learned_addr learned my addr 172.21.15.45:0/603136398 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:06:52.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:52.895+0000 7f138e000640 1 --2- 172.21.15.45:0/603136398 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f138819a9b0 0x7f138819ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:52.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:52.895+0000 7f138cc00640 1 -- 172.21.15.45:0/603136398 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f138819a9b0 msgr2=0x7f138819ae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:52.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:52.895+0000 7f138cc00640 1 --2- 172.21.15.45:0/603136398 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f138819a9b0 0x7f138819ae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:52.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:52.895+0000 7f138cc00640 1 -- 172.21.15.45:0/603136398 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f13881044b0 msgr2=0x7f13881a07b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:52.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:52.895+0000 7f138cc00640 1 --2- 172.21.15.45:0/603136398 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f13881044b0 0x7f13881a07b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:52.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:52.895+0000 7f138cc00640 1 -- 172.21.15.45:0/603136398 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f138819b4d0 con 0x7f1388109580 2024-04-16T17:06:52.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:52.895+0000 7f138d600640 1 --2- 172.21.15.45:0/603136398 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f13881044b0 0x7f13881a07b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:06:52.897 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:52.896+0000 7f138cc00640 1 --2- 172.21.15.45:0/603136398 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1388109580 0x7f13881a0cf0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f137400e9c0 tx=0x7f137400ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:06:52.897 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:52.896+0000 7f1386000640 1 -- 172.21.15.45:0/603136398 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f137400cd80 con 0x7f1388109580 2024-04-16T17:06:52.897 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:52.896+0000 7f1386000640 1 -- 172.21.15.45:0/603136398 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1374005500 con 0x7f1388109580 2024-04-16T17:06:52.897 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:52.896+0000 7f1386000640 1 -- 172.21.15.45:0/603136398 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1374010690 con 0x7f1388109580 2024-04-16T17:06:52.897 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:52.896+0000 7f1390000640 1 -- 172.21.15.45:0/603136398 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f138819b7c0 con 0x7f1388109580 2024-04-16T17:06:52.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:52.896+0000 7f1390000640 1 -- 172.21.15.45:0/603136398 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13881a7970 con 0x7f1388109580 2024-04-16T17:06:52.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:52.898+0000 7f1390000640 1 -- 172.21.15.45:0/603136398 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1388069270 con 0x7f1388109580 2024-04-16T17:06:52.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:52.900+0000 7f1386000640 1 -- 172.21.15.45:0/603136398 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 24) ==== 138822+0+0 (secure 0 0 0) 0x7f13740046c0 con 0x7f1388109580 2024-04-16T17:06:52.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:52.901+0000 7f1386000640 1 --2- 172.21.15.45:0/603136398 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f13700a3e30 0x7f13700a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:52.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:52.902+0000 7f1386000640 1 -- 172.21.15.45:0/603136398 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f13740d1040 con 0x7f1388109580 2024-04-16T17:06:52.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:52.902+0000 7f138d600640 1 --2- 172.21.15.45:0/603136398 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f13700a3e30 0x7f13700a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:52.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:52.903+0000 7f138d600640 1 --2- 172.21.15.45:0/603136398 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f13700a3e30 0x7f13700a62f0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f137c009830 tx=0x7f137c02e040 comp rx=0 tx=0).ready entity=mgr.25411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:06:52.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:52.905+0000 7f1386000640 1 -- 172.21.15.45:0/603136398 <== mon.2 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f137409d1c0 con 0x7f1388109580 2024-04-16T17:06:52.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:06:52 smithi045 ceph-mon[31676]: from='mgr.25411 ' entity='mgr.x' 2024-04-16T17:06:52.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:06:52 smithi045 ceph-mon[31676]: from='mgr.25411 ' entity='mgr.x' 2024-04-16T17:06:52.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:06:52 smithi045 ceph-mon[31676]: from='client.? 172.21.15.45:0/2945687729' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T17:06:52.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:06:52 smithi045 ceph-mon[31676]: from='mgr.25411 ' entity='mgr.x' 2024-04-16T17:06:52.978 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:06:52 smithi045 ceph-mon[31676]: from='mgr.25411 ' entity='mgr.x' 2024-04-16T17:06:52.978 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:06:52 smithi045 ceph-mon[31676]: Updating smithi045:/etc/ceph/ceph.client.admin.keyring 2024-04-16T17:06:52.978 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:06:52 smithi045 ceph-mon[31676]: from='client.? 172.21.15.45:0/234718447' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-16T17:06:52.978 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:06:52 smithi045 ceph-mon[31676]: from='client.? 172.21.15.45:0/575644021' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-16T17:06:53.055 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:06:52 smithi117 ceph-mon[36730]: from='mgr.25411 ' entity='mgr.x' 2024-04-16T17:06:53.055 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:06:52 smithi117 ceph-mon[36730]: from='mgr.25411 ' entity='mgr.x' 2024-04-16T17:06:53.055 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:06:52 smithi117 ceph-mon[36730]: from='client.? 172.21.15.45:0/2945687729' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T17:06:53.055 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:06:52 smithi117 ceph-mon[36730]: from='mgr.25411 ' entity='mgr.x' 2024-04-16T17:06:53.055 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:06:52 smithi117 ceph-mon[36730]: from='mgr.25411 ' entity='mgr.x' 2024-04-16T17:06:53.055 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:06:52 smithi117 ceph-mon[36730]: Updating smithi045:/etc/ceph/ceph.client.admin.keyring 2024-04-16T17:06:53.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:06:52 smithi117 ceph-mon[36730]: from='client.? 172.21.15.45:0/234718447' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-16T17:06:53.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:06:52 smithi117 ceph-mon[36730]: from='client.? 172.21.15.45:0/575644021' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-16T17:06:53.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:53.126+0000 7f1390000640 1 -- 172.21.15.45:0/603136398 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f138810c7b0 con 0x7f1388109580 2024-04-16T17:06:53.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:53.126+0000 7f1386000640 1 -- 172.21.15.45:0/603136398 <== mon.2 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v95) ==== 100+0+11 (secure 0 0 0) 0x7f137409cac0 con 0x7f1388109580 2024-04-16T17:06:53.127 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 64 2024-04-16T17:06:53.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:53.129+0000 7f1390000640 1 -- 172.21.15.45:0/603136398 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f13700a3e30 msgr2=0x7f13700a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:53.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:53.129+0000 7f1390000640 1 --2- 172.21.15.45:0/603136398 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f13700a3e30 0x7f13700a62f0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f137c009830 tx=0x7f137c02e040 comp rx=0 tx=0).stop 2024-04-16T17:06:53.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:53.129+0000 7f1390000640 1 -- 172.21.15.45:0/603136398 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1388109580 msgr2=0x7f13881a0cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:53.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:53.129+0000 7f1390000640 1 --2- 172.21.15.45:0/603136398 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1388109580 0x7f13881a0cf0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f137400e9c0 tx=0x7f137400ee80 comp rx=0 tx=0).stop 2024-04-16T17:06:53.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:53.129+0000 7f1390000640 1 -- 172.21.15.45:0/603136398 shutdown_connections 2024-04-16T17:06:53.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:53.129+0000 7f1390000640 1 --2- 172.21.15.45:0/603136398 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f13700a3e30 0x7f13700a62f0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:53.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:53.129+0000 7f1390000640 1 --2- 172.21.15.45:0/603136398 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f138819a9b0 0x7f138819ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:53.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:53.129+0000 7f1390000640 1 --2- 172.21.15.45:0/603136398 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1388109580 0x7f13881a0cf0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:53.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:53.129+0000 7f1390000640 1 --2- 172.21.15.45:0/603136398 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f13881044b0 0x7f13881a07b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:53.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:53.129+0000 7f1390000640 1 -- 172.21.15.45:0/603136398 >> 172.21.15.45:0/603136398 conn(0x7f13880ffd90 msgr2=0x7f1388101100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:06:53.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:53.129+0000 7f1390000640 1 -- 172.21.15.45:0/603136398 shutdown_connections 2024-04-16T17:06:53.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:53.129+0000 7f1390000640 1 -- 172.21.15.45:0/603136398 wait complete. 2024-04-16T17:06:53.148 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-16T17:06:53.178 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:06:52 smithi104 ceph-mon[36457]: from='mgr.25411 ' entity='mgr.x' 2024-04-16T17:06:53.178 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:06:52 smithi104 ceph-mon[36457]: from='mgr.25411 ' entity='mgr.x' 2024-04-16T17:06:53.178 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:06:52 smithi104 ceph-mon[36457]: from='client.? 172.21.15.45:0/2945687729' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T17:06:53.178 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:06:52 smithi104 ceph-mon[36457]: from='mgr.25411 ' entity='mgr.x' 2024-04-16T17:06:53.178 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:06:52 smithi104 ceph-mon[36457]: from='mgr.25411 ' entity='mgr.x' 2024-04-16T17:06:53.178 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:06:52 smithi104 ceph-mon[36457]: Updating smithi045:/etc/ceph/ceph.client.admin.keyring 2024-04-16T17:06:53.178 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:06:52 smithi104 ceph-mon[36457]: from='client.? 172.21.15.45:0/234718447' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-16T17:06:53.178 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:06:52 smithi104 ceph-mon[36457]: from='client.? 172.21.15.45:0/575644021' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-16T17:06:53.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:53.346+0000 7f347ba00640 1 -- 172.21.15.45:0/3184840455 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3474102240 msgr2=0x7f3474102620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:53.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:53.346+0000 7f347ba00640 1 --2- 172.21.15.45:0/3184840455 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3474102240 0x7f3474102620 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f3458009900 tx=0x7f345802bc30 comp rx=0 tx=0).stop 2024-04-16T17:06:53.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:53.346+0000 7f347ba00640 1 -- 172.21.15.45:0/3184840455 shutdown_connections 2024-04-16T17:06:53.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:53.346+0000 7f347ba00640 1 --2- 172.21.15.45:0/3184840455 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3474107310 0x7f347410eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:53.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:53.346+0000 7f347ba00640 1 --2- 172.21.15.45:0/3184840455 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3474102bf0 0x7f3474106c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:53.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:53.346+0000 7f347ba00640 1 --2- 172.21.15.45:0/3184840455 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3474102240 0x7f3474102620 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:53.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:53.346+0000 7f347ba00640 1 -- 172.21.15.45:0/3184840455 >> 172.21.15.45:0/3184840455 conn(0x7f3474079450 msgr2=0x7f3474079860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:06:53.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:53.346+0000 7f347ba00640 1 -- 172.21.15.45:0/3184840455 shutdown_connections 2024-04-16T17:06:53.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:53.346+0000 7f347ba00640 1 -- 172.21.15.45:0/3184840455 wait complete. 2024-04-16T17:06:53.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:53.347+0000 7f347ba00640 1 Processor -- start 2024-04-16T17:06:53.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:53.348+0000 7f347ba00640 1 -- start start 2024-04-16T17:06:53.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:53.348+0000 7f347ba00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3474102240 0x7f34741a0770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:53.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:53.348+0000 7f347ba00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3474107310 0x7f34741a0cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:53.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:53.348+0000 7f347ba00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f347419a920 0x7f347419add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:53.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:53.348+0000 7f347ba00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f3474112420 con 0x7f3474102240 2024-04-16T17:06:53.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:53.348+0000 7f347ba00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f34741122a0 con 0x7f347419a920 2024-04-16T17:06:53.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:53.348+0000 7f347ba00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f34741125a0 con 0x7f3474107310 2024-04-16T17:06:53.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:53.348+0000 7f3479000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3474102240 0x7f34741a0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:53.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:53.348+0000 7f3479a00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f347419a920 0x7f347419add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:53.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:53.348+0000 7f3479000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3474102240 0x7f34741a0770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55382/0 (socket says 172.21.15.45:55382) 2024-04-16T17:06:53.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:53.348+0000 7f3479000640 1 -- 172.21.15.45:0/2599667110 learned_addr learned my addr 172.21.15.45:0/2599667110 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:06:53.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:53.348+0000 7f346be00640 1 --2- 172.21.15.45:0/2599667110 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3474107310 0x7f34741a0cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:53.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:53.349+0000 7f3479000640 1 -- 172.21.15.45:0/2599667110 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3474107310 msgr2=0x7f34741a0cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:53.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:53.349+0000 7f3479000640 1 --2- 172.21.15.45:0/2599667110 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3474107310 0x7f34741a0cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:53.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:53.349+0000 7f3479000640 1 -- 172.21.15.45:0/2599667110 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f347419a920 msgr2=0x7f347419add0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:53.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:53.349+0000 7f3479000640 1 --2- 172.21.15.45:0/2599667110 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f347419a920 0x7f347419add0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:53.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:53.349+0000 7f3479000640 1 -- 172.21.15.45:0/2599667110 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f347419b440 con 0x7f3474102240 2024-04-16T17:06:53.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:53.349+0000 7f3479000640 1 --2- 172.21.15.45:0/2599667110 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3474102240 0x7f34741a0770 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f345802c240 tx=0x7f34580042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:06:53.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:53.350+0000 7f3472000640 1 -- 172.21.15.45:0/2599667110 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3458043020 con 0x7f3474102240 2024-04-16T17:06:53.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:53.350+0000 7f347ba00640 1 -- 172.21.15.45:0/2599667110 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f347419b6d0 con 0x7f3474102240 2024-04-16T17:06:53.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:53.350+0000 7f347ba00640 1 -- 172.21.15.45:0/2599667110 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34741a7970 con 0x7f3474102240 2024-04-16T17:06:53.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:53.350+0000 7f3472000640 1 -- 172.21.15.45:0/2599667110 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3458004c20 con 0x7f3474102240 2024-04-16T17:06:53.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:53.350+0000 7f3472000640 1 -- 172.21.15.45:0/2599667110 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34580035f0 con 0x7f3474102240 2024-04-16T17:06:53.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:53.351+0000 7f347ba00640 1 -- 172.21.15.45:0/2599667110 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3448005290 con 0x7f3474102240 2024-04-16T17:06:53.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:53.353+0000 7f3472000640 1 -- 172.21.15.45:0/2599667110 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 24) ==== 138822+0+0 (secure 0 0 0) 0x7f3458046020 con 0x7f3474102240 2024-04-16T17:06:53.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:53.355+0000 7f3472000640 1 --2- 172.21.15.45:0/2599667110 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f34500a3e30 0x7f34500a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:53.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:53.356+0000 7f3472000640 1 -- 172.21.15.45:0/2599667110 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f34580f23f0 con 0x7f3474102240 2024-04-16T17:06:53.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:53.356+0000 7f346be00640 1 --2- 172.21.15.45:0/2599667110 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f34500a3e30 0x7f34500a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:53.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:53.357+0000 7f346be00640 1 --2- 172.21.15.45:0/2599667110 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f34500a3e30 0x7f34500a62f0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f34600095f0 tx=0x7f3460005f90 comp rx=0 tx=0).ready entity=mgr.25411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:06:53.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:53.360+0000 7f3472000640 1 -- 172.21.15.45:0/2599667110 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f345804d030 con 0x7f3474102240 2024-04-16T17:06:53.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:53.573+0000 7f347ba00640 1 -- 172.21.15.45:0/2599667110 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f3448005850 con 0x7f3474102240 2024-04-16T17:06:53.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:53.574+0000 7f3472000640 1 -- 172.21.15.45:0/2599667110 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v95) ==== 103+0+11 (secure 0 0 0) 0x7f34580be680 con 0x7f3474102240 2024-04-16T17:06:53.574 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 32 2024-04-16T17:06:53.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:53.577+0000 7f347ba00640 1 -- 172.21.15.45:0/2599667110 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f34500a3e30 msgr2=0x7f34500a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:53.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:53.577+0000 7f347ba00640 1 --2- 172.21.15.45:0/2599667110 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f34500a3e30 0x7f34500a62f0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f34600095f0 tx=0x7f3460005f90 comp rx=0 tx=0).stop 2024-04-16T17:06:53.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:53.577+0000 7f347ba00640 1 -- 172.21.15.45:0/2599667110 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3474102240 msgr2=0x7f34741a0770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:53.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:53.577+0000 7f347ba00640 1 --2- 172.21.15.45:0/2599667110 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3474102240 0x7f34741a0770 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f345802c240 tx=0x7f34580042f0 comp rx=0 tx=0).stop 2024-04-16T17:06:53.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:53.577+0000 7f347ba00640 1 -- 172.21.15.45:0/2599667110 shutdown_connections 2024-04-16T17:06:53.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:53.577+0000 7f347ba00640 1 --2- 172.21.15.45:0/2599667110 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f34500a3e30 0x7f34500a62f0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:53.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:53.577+0000 7f347ba00640 1 --2- 172.21.15.45:0/2599667110 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3474107310 0x7f34741a0cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:53.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:53.577+0000 7f347ba00640 1 --2- 172.21.15.45:0/2599667110 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f347419a920 0x7f347419add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:53.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:53.577+0000 7f347ba00640 1 --2- 172.21.15.45:0/2599667110 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3474102240 0x7f34741a0770 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:53.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:53.577+0000 7f347ba00640 1 -- 172.21.15.45:0/2599667110 >> 172.21.15.45:0/2599667110 conn(0x7f3474079450 msgr2=0x7f3474100100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:06:53.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:53.578+0000 7f347ba00640 1 -- 172.21.15.45:0/2599667110 shutdown_connections 2024-04-16T17:06:53.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:53.578+0000 7f347ba00640 1 -- 172.21.15.45:0/2599667110 wait complete. 2024-04-16T17:06:53.599 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-16T17:06:53.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:53.805+0000 7f33a2c00640 1 -- 172.21.15.45:0/2571943356 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f339c109330 msgr2=0x7f339c110f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:53.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:53.805+0000 7f33a2c00640 1 --2- 172.21.15.45:0/2571943356 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f339c109330 0x7f339c110f00 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f3394009900 tx=0x7f339402bc40 comp rx=0 tx=0).stop 2024-04-16T17:06:53.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:53.805+0000 7f33a2c00640 1 -- 172.21.15.45:0/2571943356 shutdown_connections 2024-04-16T17:06:53.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:53.805+0000 7f33a2c00640 1 --2- 172.21.15.45:0/2571943356 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f339c109330 0x7f339c110f00 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:53.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:53.805+0000 7f33a2c00640 1 --2- 172.21.15.45:0/2571943356 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f339c104c10 0x7f339c108c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:53.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:53.805+0000 7f33a2c00640 1 --2- 172.21.15.45:0/2571943356 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f339c104260 0x7f339c104640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:53.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:53.805+0000 7f33a2c00640 1 -- 172.21.15.45:0/2571943356 >> 172.21.15.45:0/2571943356 conn(0x7f339c0ffaa0 msgr2=0x7f339c101ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:06:53.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:53.806+0000 7f33a2c00640 1 -- 172.21.15.45:0/2571943356 shutdown_connections 2024-04-16T17:06:53.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:53.806+0000 7f33a2c00640 1 -- 172.21.15.45:0/2571943356 wait complete. 2024-04-16T17:06:53.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:53.807+0000 7f33a2c00640 1 Processor -- start 2024-04-16T17:06:53.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:53.807+0000 7f33a2c00640 1 -- start start 2024-04-16T17:06:53.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:53.807+0000 7f33a2c00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f339c104260 0x7f339c1a05e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:53.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:53.807+0000 7f33a2c00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f339c109330 0x7f339c1a0b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:53.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:53.807+0000 7f33a2c00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f339c19a6f0 0x7f339c19ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:53.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:53.807+0000 7f33a2c00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f339c1143d0 con 0x7f339c109330 2024-04-16T17:06:53.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:53.807+0000 7f33a2c00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f339c114250 con 0x7f339c19a6f0 2024-04-16T17:06:53.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:53.807+0000 7f33a2c00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f339c114550 con 0x7f339c104260 2024-04-16T17:06:53.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:53.808+0000 7f33a0e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f339c109330 0x7f339c1a0b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:53.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:53.808+0000 7f33a1800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f339c104260 0x7f339c1a05e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:53.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:53.808+0000 7f33a0e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f339c109330 0x7f339c1a0b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55406/0 (socket says 172.21.15.45:55406) 2024-04-16T17:06:53.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:53.808+0000 7f33a0e00640 1 -- 172.21.15.45:0/4055561081 learned_addr learned my addr 172.21.15.45:0/4055561081 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:06:53.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:53.808+0000 7f33a2200640 1 --2- 172.21.15.45:0/4055561081 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f339c19a6f0 0x7f339c19ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:53.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:53.808+0000 7f33a1800640 1 -- 172.21.15.45:0/4055561081 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f339c19a6f0 msgr2=0x7f339c19ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:53.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:53.808+0000 7f33a1800640 1 --2- 172.21.15.45:0/4055561081 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f339c19a6f0 0x7f339c19ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:53.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:53.808+0000 7f33a1800640 1 -- 172.21.15.45:0/4055561081 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f339c109330 msgr2=0x7f339c1a0b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:53.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:53.808+0000 7f33a1800640 1 --2- 172.21.15.45:0/4055561081 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f339c109330 0x7f339c1a0b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:53.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:53.808+0000 7f33a1800640 1 -- 172.21.15.45:0/4055561081 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f339c19b2f0 con 0x7f339c104260 2024-04-16T17:06:53.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:53.808+0000 7f33a0e00640 1 --2- 172.21.15.45:0/4055561081 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f339c109330 0x7f339c1a0b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:06:53.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:53.809+0000 7f33a1800640 1 --2- 172.21.15.45:0/4055561081 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f339c104260 0x7f339c1a05e0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f339000ec80 tx=0x7f339000c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:06:53.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:53.809+0000 7f339a000640 1 -- 172.21.15.45:0/4055561081 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3390003b70 con 0x7f339c104260 2024-04-16T17:06:53.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:53.809+0000 7f339a000640 1 -- 172.21.15.45:0/4055561081 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3390003d10 con 0x7f339c104260 2024-04-16T17:06:53.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:53.809+0000 7f33a2c00640 1 -- 172.21.15.45:0/4055561081 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f339c1a7380 con 0x7f339c104260 2024-04-16T17:06:53.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:53.809+0000 7f339a000640 1 -- 172.21.15.45:0/4055561081 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3390010a30 con 0x7f339c104260 2024-04-16T17:06:53.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:53.809+0000 7f33a2c00640 1 -- 172.21.15.45:0/4055561081 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f339c1a77e0 con 0x7f339c104260 2024-04-16T17:06:53.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:53.810+0000 7f33a2c00640 1 -- 172.21.15.45:0/4055561081 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f339c105730 con 0x7f339c104260 2024-04-16T17:06:53.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:53.812+0000 7f339a000640 1 -- 172.21.15.45:0/4055561081 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 24) ==== 138822+0+0 (secure 0 0 0) 0x7f3390003ec0 con 0x7f339c104260 2024-04-16T17:06:53.814 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:53.814+0000 7f339a000640 1 --2- 172.21.15.45:0/4055561081 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f33840a3e30 0x7f33840a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:53.814 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:53.814+0000 7f339a000640 1 -- 172.21.15.45:0/4055561081 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f33900d13d0 con 0x7f339c104260 2024-04-16T17:06:53.814 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:53.814+0000 7f33a0e00640 1 --2- 172.21.15.45:0/4055561081 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f33840a3e30 0x7f33840a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:53.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:53.815+0000 7f33a0e00640 1 --2- 172.21.15.45:0/4055561081 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f33840a3e30 0x7f33840a62f0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f33880095f0 tx=0x7f3388005f90 comp rx=0 tx=0).ready entity=mgr.25411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:06:53.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:53.819+0000 7f339a000640 1 -- 172.21.15.45:0/4055561081 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f339009d550 con 0x7f339c104260 2024-04-16T17:06:53.978 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:06:53 smithi045 ceph-mon[31676]: Updating smithi045:/var/lib/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/config/ceph.client.admin.keyring 2024-04-16T17:06:53.978 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:06:53 smithi045 ceph-mon[31676]: pgmap v7: 129 pgs: 129 active+clean; 1.1 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 146 B/s wr, 21 op/s 2024-04-16T17:06:53.978 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:06:53 smithi045 ceph-mon[31676]: from='client.? 172.21.15.45:0/603136398' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-16T17:06:53.978 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:06:53 smithi045 ceph-mon[31676]: from='mgr.25411 ' entity='mgr.x' 2024-04-16T17:06:53.978 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:06:53 smithi045 ceph-mon[31676]: from='mgr.25411 ' entity='mgr.x' 2024-04-16T17:06:53.978 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:06:53 smithi045 ceph-mon[31676]: from='mgr.25411 ' entity='mgr.x' 2024-04-16T17:06:53.979 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:06:53 smithi045 ceph-mon[31676]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T17:06:53.979 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:06:53 smithi045 ceph-mon[31676]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:06:53.979 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:06:53 smithi045 ceph-mon[31676]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T17:06:53.979 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:06:53 smithi045 ceph-mon[31676]: from='client.? 172.21.15.45:0/2599667110' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-16T17:06:53.979 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:06:53 smithi045 ceph-mon[31676]: from='mgr.25411 ' entity='mgr.x' 2024-04-16T17:06:54.042 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:06:53 smithi104 ceph-mon[36457]: Updating smithi045:/var/lib/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/config/ceph.client.admin.keyring 2024-04-16T17:06:54.042 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:06:53 smithi104 ceph-mon[36457]: pgmap v7: 129 pgs: 129 active+clean; 1.1 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 146 B/s wr, 21 op/s 2024-04-16T17:06:54.042 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:06:53 smithi104 ceph-mon[36457]: from='client.? 172.21.15.45:0/603136398' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-16T17:06:54.042 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:06:53 smithi104 ceph-mon[36457]: from='mgr.25411 ' entity='mgr.x' 2024-04-16T17:06:54.042 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:06:53 smithi104 ceph-mon[36457]: from='mgr.25411 ' entity='mgr.x' 2024-04-16T17:06:54.042 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:06:53 smithi104 ceph-mon[36457]: from='mgr.25411 ' entity='mgr.x' 2024-04-16T17:06:54.042 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:06:53 smithi104 ceph-mon[36457]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T17:06:54.042 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:06:53 smithi104 ceph-mon[36457]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:06:54.042 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:06:53 smithi104 ceph-mon[36457]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T17:06:54.043 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:06:53 smithi104 ceph-mon[36457]: from='client.? 172.21.15.45:0/2599667110' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-16T17:06:54.043 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:06:53 smithi104 ceph-mon[36457]: from='mgr.25411 ' entity='mgr.x' 2024-04-16T17:06:54.055 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:06:53 smithi117 ceph-mon[36730]: Updating smithi045:/var/lib/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/config/ceph.client.admin.keyring 2024-04-16T17:06:54.055 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:06:53 smithi117 ceph-mon[36730]: pgmap v7: 129 pgs: 129 active+clean; 1.1 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 146 B/s wr, 21 op/s 2024-04-16T17:06:54.055 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:06:53 smithi117 ceph-mon[36730]: from='client.? 172.21.15.45:0/603136398' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-16T17:06:54.055 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:06:53 smithi117 ceph-mon[36730]: from='mgr.25411 ' entity='mgr.x' 2024-04-16T17:06:54.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:06:53 smithi117 ceph-mon[36730]: from='mgr.25411 ' entity='mgr.x' 2024-04-16T17:06:54.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:06:53 smithi117 ceph-mon[36730]: from='mgr.25411 ' entity='mgr.x' 2024-04-16T17:06:54.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:06:53 smithi117 ceph-mon[36730]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T17:06:54.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:06:53 smithi117 ceph-mon[36730]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:06:54.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:06:53 smithi117 ceph-mon[36730]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T17:06:54.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:06:53 smithi117 ceph-mon[36730]: from='client.? 172.21.15.45:0/2599667110' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-16T17:06:54.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:06:53 smithi117 ceph-mon[36730]: from='mgr.25411 ' entity='mgr.x' 2024-04-16T17:06:54.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.077+0000 7f33a2c00640 1 -- 172.21.15.45:0/4055561081 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f339c19bbd0 con 0x7f339c104260 2024-04-16T17:06:54.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.078+0000 7f339a000640 1 -- 172.21.15.45:0/4055561081 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11887 (secure 0 0 0) 0x7f339009ce50 con 0x7f339c104260 2024-04-16T17:06:54.079 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-16T17:06:54.079 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/1748842443","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":1748842443},{"type":"v1","addr":"172.21.15.45:6835","nonce":1748842443}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6839/1308627051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":1308627051},{"type":"v1","addr":"172.21.15.45:6839","nonce":1308627051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/1768242489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1768242489},{"type":"v1","addr":"172.21.15.117:6833","nonce":1768242489}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6833/3558902585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":3558902585},{"type":"v1","addr":"172.21.15.104:6833","nonce":3558902585}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24478,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/1308142903","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":1308142903},{"type":"v1","addr":"172.21.15.45:6837","nonce":1308142903}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24491,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6835/3430679937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":3430679937},{"type":"v1","addr":"172.21.15.104:6835","nonce":3430679937}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-16T17:03:24.663611+0000","modified":"2024-04-16T17:06:44.485317+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24581,"mds_1":24569,"mds_2":24532},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24499":{"gid":24499,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/2224123841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2224123841},{"type":"v1","addr":"172.21.15.117:6835","nonce":2224123841}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6837/2471356815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":2471356815},{"type":"v1","addr":"172.21.15.117:6837","nonce":2471356815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24524":{"gid":24524,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6837/2647756509","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":2647756509},{"type":"v1","addr":"172.21.15.104:6837","nonce":2647756509}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24532":{"gid":24532,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":13,"addr":"172.21.15.117:6839/2378675949","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":2378675949},{"type":"v1","addr":"172.21.15.117:6839","nonce":2378675949}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24569":{"gid":24569,"name":"a","rank":1,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.45:6841/3755310822","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6840","nonce":3755310822},{"type":"v1","addr":"172.21.15.45:6841","nonce":3755310822}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24581":{"gid":24581,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.104:6839/3660070592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6838","nonce":3660070592},{"type":"v1","addr":"172.21.15.104:6839","nonce":3660070592}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24581,"qdb_cluster":[24569,24581,24532]},"id":1}]} 2024-04-16T17:06:54.079 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 25 2024-04-16T17:06:54.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.081+0000 7f33a2c00640 1 -- 172.21.15.45:0/4055561081 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f33840a3e30 msgr2=0x7f33840a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:54.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.081+0000 7f33a2c00640 1 --2- 172.21.15.45:0/4055561081 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f33840a3e30 0x7f33840a62f0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f33880095f0 tx=0x7f3388005f90 comp rx=0 tx=0).stop 2024-04-16T17:06:54.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.081+0000 7f33a2c00640 1 -- 172.21.15.45:0/4055561081 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f339c104260 msgr2=0x7f339c1a05e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:54.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.081+0000 7f33a2c00640 1 --2- 172.21.15.45:0/4055561081 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f339c104260 0x7f339c1a05e0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f339000ec80 tx=0x7f339000c6a0 comp rx=0 tx=0).stop 2024-04-16T17:06:54.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.081+0000 7f33a2c00640 1 -- 172.21.15.45:0/4055561081 shutdown_connections 2024-04-16T17:06:54.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.081+0000 7f33a2c00640 1 --2- 172.21.15.45:0/4055561081 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f33840a3e30 0x7f33840a62f0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:54.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.081+0000 7f33a2c00640 1 --2- 172.21.15.45:0/4055561081 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f339c19a6f0 0x7f339c19ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:54.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.081+0000 7f33a2c00640 1 --2- 172.21.15.45:0/4055561081 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f339c109330 0x7f339c1a0b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:54.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.081+0000 7f33a2c00640 1 --2- 172.21.15.45:0/4055561081 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f339c104260 0x7f339c1a05e0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:54.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.081+0000 7f33a2c00640 1 -- 172.21.15.45:0/4055561081 >> 172.21.15.45:0/4055561081 conn(0x7f339c0ffaa0 msgr2=0x7f339c101d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:06:54.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.081+0000 7f33a2c00640 1 -- 172.21.15.45:0/4055561081 shutdown_connections 2024-04-16T17:06:54.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.081+0000 7f33a2c00640 1 -- 172.21.15.45:0/4055561081 wait complete. 2024-04-16T17:06:54.102 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-16T17:06:54.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.329+0000 7f4886200640 1 -- 172.21.15.45:0/1315139274 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4880102090 msgr2=0x7f4880102470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:54.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.329+0000 7f4886200640 1 --2- 172.21.15.45:0/1315139274 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4880102090 0x7f4880102470 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f4870009860 tx=0x7f487002bb70 comp rx=0 tx=0).stop 2024-04-16T17:06:54.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.329+0000 7f4886200640 1 -- 172.21.15.45:0/1315139274 shutdown_connections 2024-04-16T17:06:54.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.329+0000 7f4886200640 1 --2- 172.21.15.45:0/1315139274 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4880107160 0x7f488010ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:54.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.329+0000 7f4886200640 1 --2- 172.21.15.45:0/1315139274 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4880102a40 0x7f4880106a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:54.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.329+0000 7f4886200640 1 --2- 172.21.15.45:0/1315139274 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4880102090 0x7f4880102470 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:54.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.329+0000 7f4886200640 1 -- 172.21.15.45:0/1315139274 >> 172.21.15.45:0/1315139274 conn(0x7f48800fd930 msgr2=0x7f48800ffd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:06:54.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.329+0000 7f4886200640 1 -- 172.21.15.45:0/1315139274 shutdown_connections 2024-04-16T17:06:54.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.329+0000 7f4886200640 1 -- 172.21.15.45:0/1315139274 wait complete. 2024-04-16T17:06:54.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.330+0000 7f4886200640 1 Processor -- start 2024-04-16T17:06:54.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.331+0000 7f4886200640 1 -- start start 2024-04-16T17:06:54.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.331+0000 7f4886200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4880102090 0x7f4880074300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:54.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.331+0000 7f4886200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4880107160 0x7f4880074840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:54.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.331+0000 7f4886200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4880074d80 0x7f48801a69f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:54.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.331+0000 7f4886200640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f4880112220 con 0x7f4880107160 2024-04-16T17:06:54.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.331+0000 7f4886200640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f48801120a0 con 0x7f4880074d80 2024-04-16T17:06:54.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.331+0000 7f4886200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f48801123a0 con 0x7f4880102090 2024-04-16T17:06:54.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.331+0000 7f487fe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4880107160 0x7f4880074840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:54.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.332+0000 7f487fe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4880107160 0x7f4880074840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55424/0 (socket says 172.21.15.45:55424) 2024-04-16T17:06:54.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.332+0000 7f487fe00640 1 -- 172.21.15.45:0/2279906483 learned_addr learned my addr 172.21.15.45:0/2279906483 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:06:54.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.332+0000 7f4884e00640 1 --2- 172.21.15.45:0/2279906483 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4880102090 0x7f4880074300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:54.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.332+0000 7f4885800640 1 --2- 172.21.15.45:0/2279906483 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4880074d80 0x7f48801a69f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:54.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.332+0000 7f4884e00640 1 -- 172.21.15.45:0/2279906483 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4880074d80 msgr2=0x7f48801a69f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:54.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.332+0000 7f4884e00640 1 --2- 172.21.15.45:0/2279906483 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4880074d80 0x7f48801a69f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:54.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.332+0000 7f4884e00640 1 -- 172.21.15.45:0/2279906483 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4880107160 msgr2=0x7f4880074840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:54.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.332+0000 7f4884e00640 1 --2- 172.21.15.45:0/2279906483 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4880107160 0x7f4880074840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:54.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.332+0000 7f4884e00640 1 -- 172.21.15.45:0/2279906483 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48801a7090 con 0x7f4880102090 2024-04-16T17:06:54.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.332+0000 7f487fe00640 1 --2- 172.21.15.45:0/2279906483 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4880107160 0x7f4880074840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:06:54.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.332+0000 7f4885800640 1 --2- 172.21.15.45:0/2279906483 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4880074d80 0x7f48801a69f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:06:54.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.332+0000 7f4884e00640 1 --2- 172.21.15.45:0/2279906483 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4880102090 0x7f4880074300 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f487002c0b0 tx=0x7f48700045b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:06:54.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.333+0000 7f487d600640 1 -- 172.21.15.45:0/2279906483 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4870041020 con 0x7f4880102090 2024-04-16T17:06:54.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.333+0000 7f4886200640 1 -- 172.21.15.45:0/2279906483 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48801a7320 con 0x7f4880102090 2024-04-16T17:06:54.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.333+0000 7f487d600640 1 -- 172.21.15.45:0/2279906483 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f487002c9f0 con 0x7f4880102090 2024-04-16T17:06:54.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.333+0000 7f487d600640 1 -- 172.21.15.45:0/2279906483 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4870039040 con 0x7f4880102090 2024-04-16T17:06:54.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.333+0000 7f4886200640 1 -- 172.21.15.45:0/2279906483 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48801a7810 con 0x7f4880102090 2024-04-16T17:06:54.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.334+0000 7f4886200640 1 -- 172.21.15.45:0/2279906483 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4880068d40 con 0x7f4880102090 2024-04-16T17:06:54.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.336+0000 7f487d600640 1 -- 172.21.15.45:0/2279906483 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 24) ==== 138822+0+0 (secure 0 0 0) 0x7f4870044020 con 0x7f4880102090 2024-04-16T17:06:54.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.338+0000 7f487d600640 1 --2- 172.21.15.45:0/2279906483 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f48600a3db0 0x7f48600a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:54.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.338+0000 7f487d600640 1 -- 172.21.15.45:0/2279906483 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f48700f2750 con 0x7f4880102090 2024-04-16T17:06:54.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.338+0000 7f487fe00640 1 --2- 172.21.15.45:0/2279906483 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f48600a3db0 0x7f48600a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:54.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.339+0000 7f487fe00640 1 --2- 172.21.15.45:0/2279906483 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f48600a3db0 0x7f48600a6270 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f4880075d40 tx=0x7f48680095c0 comp rx=0 tx=0).ready entity=mgr.25411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:06:54.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.340+0000 7f487d600640 1 -- 172.21.15.45:0/2279906483 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f487004b020 con 0x7f4880102090 2024-04-16T17:06:54.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.547+0000 7f4886200640 1 -- 172.21.15.45:0/2279906483 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f488010a390 con 0x7f4880102090 2024-04-16T17:06:54.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.549+0000 7f487d600640 1 -- 172.21.15.45:0/2279906483 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v95) ==== 74+0+25099 (secure 0 0 0) 0x7f48700be9e0 con 0x7f4880102090 2024-04-16T17:06:54.550 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-16T17:06:54.550 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":95,"fsid":"b1fecd38-fc11-11ee-bc90-c7b262605968","created":"2024-04-16T16:54:02.385906+0000","modified":"2024-04-16T17:06:45.093262+0000","last_up_change":"2024-04-16T17:02:25.162624+0000","last_in_change":"2024-04-16T17:02:04.903274+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-16T16:58:06.166659+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-16T17:03:22.304198+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-16T17:03:23.616450+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-16T17:03:27.226289+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b655de2c-59f8-4fd4-ac98-0cc222229f56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":2290170881},{"type":"v1","addr":"172.21.15.45:6803","nonce":2290170881}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":2290170881},{"type":"v1","addr":"172.21.15.45:6805","nonce":2290170881}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":2290170881},{"type":"v1","addr":"172.21.15.45:6809","nonce":2290170881}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":2290170881},{"type":"v1","addr":"172.21.15.45:6807","nonce":2290170881}]},"public_addr":"172.21.15.45:6803/2290170881","cluster_addr":"172.21.15.45:6805/2290170881","heartbeat_back_addr":"172.21.15.45:6809/2290170881","heartbeat_front_addr":"172.21.15.45:6807/2290170881","state":["exists","up"]},{"osd":1,"uuid":"ebd450ec-08fd-4c3d-8b24-4e10634ddb7e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6800","nonce":1462942067},{"type":"v1","addr":"172.21.15.104:6801","nonce":1462942067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":1462942067},{"type":"v1","addr":"172.21.15.104:6803","nonce":1462942067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":1462942067},{"type":"v1","addr":"172.21.15.104:6807","nonce":1462942067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":1462942067},{"type":"v1","addr":"172.21.15.104:6805","nonce":1462942067}]},"public_addr":"172.21.15.104:6801/1462942067","cluster_addr":"172.21.15.104:6803/1462942067","heartbeat_back_addr":"172.21.15.104:6807/1462942067","heartbeat_front_addr":"172.21.15.104:6805/1462942067","state":["exists","up"]},{"osd":2,"uuid":"b318a512-4190-42a2-8104-dcc4026ad8ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":2829260735},{"type":"v1","addr":"172.21.15.117:6801","nonce":2829260735}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":2829260735},{"type":"v1","addr":"172.21.15.117:6803","nonce":2829260735}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":2829260735},{"type":"v1","addr":"172.21.15.117:6807","nonce":2829260735}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":2829260735},{"type":"v1","addr":"172.21.15.117:6805","nonce":2829260735}]},"public_addr":"172.21.15.117:6801/2829260735","cluster_addr":"172.21.15.117:6803/2829260735","heartbeat_back_addr":"172.21.15.117:6807/2829260735","heartbeat_front_addr":"172.21.15.117:6805/2829260735","state":["exists","up"]},{"osd":3,"uuid":"ec6490cb-a715-4c12-a54a-225aae26fcb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":2823712137},{"type":"v1","addr":"172.21.15.45:6811","nonce":2823712137}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":2823712137},{"type":"v1","addr":"172.21.15.45:6813","nonce":2823712137}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":2823712137},{"type":"v1","addr":"172.21.15.45:6817","nonce":2823712137}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":2823712137},{"type":"v1","addr":"172.21.15.45:6815","nonce":2823712137}]},"public_addr":"172.21.15.45:6811/2823712137","cluster_addr":"172.21.15.45:6813/2823712137","heartbeat_back_addr":"172.21.15.45:6817/2823712137","heartbeat_front_addr":"172.21.15.45:6815/2823712137","state":["exists","up"]},{"osd":4,"uuid":"5d91d568-0250-4aae-8dc2-a0e079f92062","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":1396278127},{"type":"v1","addr":"172.21.15.104:6809","nonce":1396278127}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":1396278127},{"type":"v1","addr":"172.21.15.104:6811","nonce":1396278127}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":1396278127},{"type":"v1","addr":"172.21.15.104:6815","nonce":1396278127}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":1396278127},{"type":"v1","addr":"172.21.15.104:6813","nonce":1396278127}]},"public_addr":"172.21.15.104:6809/1396278127","cluster_addr":"172.21.15.104:6811/1396278127","heartbeat_back_addr":"172.21.15.104:6815/1396278127","heartbeat_front_addr":"172.21.15.104:6813/1396278127","state":["exists","up"]},{"osd":5,"uuid":"ac510b43-6fb2-4d0f-bc5f-16ac080d9e9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":4145720047},{"type":"v1","addr":"172.21.15.117:6809","nonce":4145720047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":4145720047},{"type":"v1","addr":"172.21.15.117:6811","nonce":4145720047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":4145720047},{"type":"v1","addr":"172.21.15.117:6815","nonce":4145720047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":4145720047},{"type":"v1","addr":"172.21.15.117:6813","nonce":4145720047}]},"public_addr":"172.21.15.117:6809/4145720047","cluster_addr":"172.21.15.117:6811/4145720047","heartbeat_back_addr":"172.21.15.117:6815/4145720047","heartbeat_front_addr":"172.21.15.117:6813/4145720047","state":["exists","up"]},{"osd":6,"uuid":"a41adc18-d88a-419e-b2c2-b089c1f32472","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":3254229176},{"type":"v1","addr":"172.21.15.45:6819","nonce":3254229176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":3254229176},{"type":"v1","addr":"172.21.15.45:6821","nonce":3254229176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":3254229176},{"type":"v1","addr":"172.21.15.45:6825","nonce":3254229176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":3254229176},{"type":"v1","addr":"172.21.15.45:6823","nonce":3254229176}]},"public_addr":"172.21.15.45:6819/3254229176","cluster_addr":"172.21.15.45:6821/3254229176","heartbeat_back_addr":"172.21.15.45:6825/3254229176","heartbeat_front_addr":"172.21.15.45:6823/3254229176","state":["exists","up"]},{"osd":7,"uuid":"b71712a4-7cf6-454d-9fd1-565e3df21c1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":4151077186},{"type":"v1","addr":"172.21.15.104:6817","nonce":4151077186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":4151077186},{"type":"v1","addr":"172.21.15.104:6819","nonce":4151077186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":4151077186},{"type":"v1","addr":"172.21.15.104:6823","nonce":4151077186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":4151077186},{"type":"v1","addr":"172.21.15.104:6821","nonce":4151077186}]},"public_addr":"172.21.15.104:6817/4151077186","cluster_addr":"172.21.15.104:6819/4151077186","heartbeat_back_addr":"172.21.15.104:6823/4151077186","heartbeat_front_addr":"172.21.15.104:6821/4151077186","state":["exists","up"]},{"osd":8,"uuid":"9e572aef-59a8-450e-a92e-df9dff4d57a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":1500286830},{"type":"v1","addr":"172.21.15.117:6817","nonce":1500286830}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":1500286830},{"type":"v1","addr":"172.21.15.117:6819","nonce":1500286830}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":1500286830},{"type":"v1","addr":"172.21.15.117:6823","nonce":1500286830}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":1500286830},{"type":"v1","addr":"172.21.15.117:6821","nonce":1500286830}]},"public_addr":"172.21.15.117:6817/1500286830","cluster_addr":"172.21.15.117:6819/1500286830","heartbeat_back_addr":"172.21.15.117:6823/1500286830","heartbeat_front_addr":"172.21.15.117:6821/1500286830","state":["exists","up"]},{"osd":9,"uuid":"014e9598-c413-4cc4-9bb8-61c6e8f59d30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":3106734074},{"type":"v1","addr":"172.21.15.45:6827","nonce":3106734074}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":3106734074},{"type":"v1","addr":"172.21.15.45:6829","nonce":3106734074}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":3106734074},{"type":"v1","addr":"172.21.15.45:6833","nonce":3106734074}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":3106734074},{"type":"v1","addr":"172.21.15.45:6831","nonce":3106734074}]},"public_addr":"172.21.15.45:6827/3106734074","cluster_addr":"172.21.15.45:6829/3106734074","heartbeat_back_addr":"172.21.15.45:6833/3106734074","heartbeat_front_addr":"172.21.15.45:6831/3106734074","state":["exists","up"]},{"osd":10,"uuid":"2deba364-3e39-47be-8a1f-8256df5b82d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":1404794652},{"type":"v1","addr":"172.21.15.104:6825","nonce":1404794652}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":1404794652},{"type":"v1","addr":"172.21.15.104:6827","nonce":1404794652}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":1404794652},{"type":"v1","addr":"172.21.15.104:6831","nonce":1404794652}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":1404794652},{"type":"v1","addr":"172.21.15.104:6829","nonce":1404794652}]},"public_addr":"172.21.15.104:6825/1404794652","cluster_addr":"172.21.15.104:6827/1404794652","heartbeat_back_addr":"172.21.15.104:6831/1404794652","heartbeat_front_addr":"172.21.15.104:6829/1404794652","state":["exists","up"]},{"osd":11,"uuid":"3fda4fea-59ad-4d1a-9fe6-fcb30e31fd7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":3384003243},{"type":"v1","addr":"172.21.15.117:6825","nonce":3384003243}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":3384003243},{"type":"v1","addr":"172.21.15.117:6827","nonce":3384003243}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":3384003243},{"type":"v1","addr":"172.21.15.117:6831","nonce":3384003243}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":3384003243},{"type":"v1","addr":"172.21.15.117:6829","nonce":3384003243}]},"public_addr":"172.21.15.117:6825/3384003243","cluster_addr":"172.21.15.117:6827/3384003243","heartbeat_back_addr":"172.21.15.117:6831/3384003243","heartbeat_front_addr":"172.21.15.117:6829/3384003243","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:57:14.799608+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:57:39.884668+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:58:03.327057+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:58:37.314531+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:59:03.667650+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:59:28.462823+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T17:00:02.968574+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T17:00:29.559190+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T17:00:55.291007+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T17:01:29.815977+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T17:01:57.015580+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T17:02:23.233151+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.5","mappings":[{"from":9,"to":10}]},{"pgid":"4.7","mappings":[{"from":5,"to":8}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.45:6800/268030699":"2024-04-17T17:06:45.093215+0000","172.21.15.45:6801/268030699":"2024-04-17T17:06:45.093215+0000","172.21.15.45:0/4074982598":"2024-04-17T16:55:19.948001+0000","172.21.15.45:0/1915281176":"2024-04-17T17:06:45.093215+0000","172.21.15.45:0/2443572576":"2024-04-17T16:55:19.948001+0000","172.21.15.45:0/1500147666":"2024-04-17T17:06:45.093215+0000","172.21.15.45:6801/1983630136":"2024-04-17T16:55:19.948001+0000","172.21.15.45:6800/1983630136":"2024-04-17T16:55:19.948001+0000","172.21.15.45:0/12001708":"2024-04-17T17:06:45.093215+0000","172.21.15.45:0/3150465248":"2024-04-17T16:54:43.692466+0000","172.21.15.45:0/556738623":"2024-04-17T16:54:43.692466+0000","172.21.15.45:0/445676297":"2024-04-17T16:55:19.948001+0000","172.21.15.45:6800/2553659622":"2024-04-17T16:54:43.692466+0000","172.21.15.45:6801/2553659622":"2024-04-17T16:54:43.692466+0000","172.21.15.45:0/265066921":"2024-04-17T17:06:45.093215+0000","172.21.15.45:0/1184615408":"2024-04-17T16:54:43.692466+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-16T17:06:54.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.552+0000 7f4886200640 1 -- 172.21.15.45:0/2279906483 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f48600a3db0 msgr2=0x7f48600a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:54.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.552+0000 7f4886200640 1 --2- 172.21.15.45:0/2279906483 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f48600a3db0 0x7f48600a6270 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f4880075d40 tx=0x7f48680095c0 comp rx=0 tx=0).stop 2024-04-16T17:06:54.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.552+0000 7f4886200640 1 -- 172.21.15.45:0/2279906483 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4880102090 msgr2=0x7f4880074300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:54.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.552+0000 7f4886200640 1 --2- 172.21.15.45:0/2279906483 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4880102090 0x7f4880074300 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f487002c0b0 tx=0x7f48700045b0 comp rx=0 tx=0).stop 2024-04-16T17:06:54.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.552+0000 7f4886200640 1 -- 172.21.15.45:0/2279906483 shutdown_connections 2024-04-16T17:06:54.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.552+0000 7f4886200640 1 --2- 172.21.15.45:0/2279906483 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f48600a3db0 0x7f48600a6270 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:54.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.552+0000 7f4886200640 1 --2- 172.21.15.45:0/2279906483 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4880074d80 0x7f48801a69f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:54.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.552+0000 7f4886200640 1 --2- 172.21.15.45:0/2279906483 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4880107160 0x7f4880074840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:54.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.552+0000 7f4886200640 1 --2- 172.21.15.45:0/2279906483 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4880102090 0x7f4880074300 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:54.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.552+0000 7f4886200640 1 -- 172.21.15.45:0/2279906483 >> 172.21.15.45:0/2279906483 conn(0x7f48800fd930 msgr2=0x7f48800fec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:06:54.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.552+0000 7f4886200640 1 -- 172.21.15.45:0/2279906483 shutdown_connections 2024-04-16T17:06:54.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.552+0000 7f4886200640 1 -- 172.21.15.45:0/2279906483 wait complete. 2024-04-16T17:06:54.574 DEBUG:tasks.fwd_scrub:Yielding 2024-04-16T17:06:54.574 INFO:teuthology.run_tasks:Running task exec... 2024-04-16T17:06:54.584 INFO:teuthology.task.exec:Executing custom commands... 2024-04-16T17:06:54.584 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi045.front.sepia.ceph.com 2024-04-16T17:06:54.584 DEBUG:teuthology.orchestra.run.smithi045:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m' 2024-04-16T17:06:54.585 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2024-04-16T17:06:54.585 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-16T17:06:54.585 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T17:06:54.585 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-16T17:06:54.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.796+0000 7f4ebfe00640 1 -- 172.21.15.45:0/2655675293 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4ec01042b0 msgr2=0x7f4ec0104690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:54.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.796+0000 7f4ebfe00640 1 --2- 172.21.15.45:0/2655675293 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4ec01042b0 0x7f4ec0104690 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f4eb0009860 tx=0x7f4eb002bb70 comp rx=0 tx=0).stop 2024-04-16T17:06:54.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.797+0000 7f4ebfe00640 1 -- 172.21.15.45:0/2655675293 shutdown_connections 2024-04-16T17:06:54.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.797+0000 7f4ebfe00640 1 --2- 172.21.15.45:0/2655675293 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4ec0109380 0x7f4ec0110f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:54.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.797+0000 7f4ebfe00640 1 --2- 172.21.15.45:0/2655675293 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4ec0104c60 0x7f4ec0108ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:54.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.797+0000 7f4ebfe00640 1 --2- 172.21.15.45:0/2655675293 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4ec01042b0 0x7f4ec0104690 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:54.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.797+0000 7f4ebfe00640 1 -- 172.21.15.45:0/2655675293 >> 172.21.15.45:0/2655675293 conn(0x7f4ec00ffb10 msgr2=0x7f4ec0101f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:06:54.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.797+0000 7f4ebfe00640 1 -- 172.21.15.45:0/2655675293 shutdown_connections 2024-04-16T17:06:54.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.797+0000 7f4ebfe00640 1 -- 172.21.15.45:0/2655675293 wait complete. 2024-04-16T17:06:54.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.798+0000 7f4ebfe00640 1 Processor -- start 2024-04-16T17:06:54.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.798+0000 7f4ebfe00640 1 -- start start 2024-04-16T17:06:54.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.798+0000 7f4ebfe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4ec01042b0 0x7f4ec01a06b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:54.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.798+0000 7f4ebfe00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4ec0109380 0x7f4ec01a0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:54.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.798+0000 7f4ebfe00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4ec019a770 0x7f4ec019abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:54.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.798+0000 7f4ebfe00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f4ec0114410 con 0x7f4ec01042b0 2024-04-16T17:06:54.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.798+0000 7f4ebfe00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f4ec0114290 con 0x7f4ec019a770 2024-04-16T17:06:54.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.798+0000 7f4ebfe00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f4ec0114590 con 0x7f4ec0109380 2024-04-16T17:06:54.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.799+0000 7f4ebe000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4ec0109380 0x7f4ec01a0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:54.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.799+0000 7f4ebf400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4ec019a770 0x7f4ec019abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:54.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.799+0000 7f4ebe000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4ec0109380 0x7f4ec01a0bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:35772/0 (socket says 172.21.15.45:35772) 2024-04-16T17:06:54.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.799+0000 7f4ebe000640 1 -- 172.21.15.45:0/3987064813 learned_addr learned my addr 172.21.15.45:0/3987064813 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:06:54.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.799+0000 7f4ebea00640 1 --2- 172.21.15.45:0/3987064813 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4ec01042b0 0x7f4ec01a06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:54.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.799+0000 7f4ebe000640 1 -- 172.21.15.45:0/3987064813 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4ec019a770 msgr2=0x7f4ec019abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:54.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.799+0000 7f4ebe000640 1 --2- 172.21.15.45:0/3987064813 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4ec019a770 0x7f4ec019abf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:54.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.799+0000 7f4ebe000640 1 -- 172.21.15.45:0/3987064813 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4ec01042b0 msgr2=0x7f4ec01a06b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:54.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.799+0000 7f4ebe000640 1 --2- 172.21.15.45:0/3987064813 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4ec01042b0 0x7f4ec01a06b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:54.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.799+0000 7f4ebe000640 1 -- 172.21.15.45:0/3987064813 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ec019b370 con 0x7f4ec0109380 2024-04-16T17:06:54.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.799+0000 7f4ebf400640 1 --2- 172.21.15.45:0/3987064813 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4ec019a770 0x7f4ec019abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:06:54.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.799+0000 7f4ebea00640 1 --2- 172.21.15.45:0/3987064813 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4ec01042b0 0x7f4ec01a06b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:06:54.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.799+0000 7f4ebe000640 1 --2- 172.21.15.45:0/3987064813 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4ec0109380 0x7f4ec01a0bf0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f4eac00e9f0 tx=0x7f4eac00edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:06:54.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.800+0000 7f4eb7400640 1 -- 172.21.15.45:0/3987064813 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4eac00ccc0 con 0x7f4ec0109380 2024-04-16T17:06:54.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.800+0000 7f4eb7400640 1 -- 172.21.15.45:0/3987064813 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4eac010910 con 0x7f4ec0109380 2024-04-16T17:06:54.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.800+0000 7f4ebfe00640 1 -- 172.21.15.45:0/3987064813 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ec01a7450 con 0x7f4ec0109380 2024-04-16T17:06:54.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.800+0000 7f4eb7400640 1 -- 172.21.15.45:0/3987064813 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4eac0188c0 con 0x7f4ec0109380 2024-04-16T17:06:54.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.800+0000 7f4ebfe00640 1 -- 172.21.15.45:0/3987064813 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ec01a7850 con 0x7f4ec0109380 2024-04-16T17:06:54.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.801+0000 7f4ebfe00640 1 -- 172.21.15.45:0/3987064813 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ec0105780 con 0x7f4ec0109380 2024-04-16T17:06:54.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.804+0000 7f4eb7400640 1 -- 172.21.15.45:0/3987064813 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 24) ==== 138822+0+0 (secure 0 0 0) 0x7f4eac010ac0 con 0x7f4ec0109380 2024-04-16T17:06:54.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.804+0000 7fc502e00640 1 -- 172.21.15.45:0/3985170714 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc4fc10e6f0 msgr2=0x7fc4fc110b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:54.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.804+0000 7fc502e00640 1 --2- 172.21.15.45:0/3985170714 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc4fc10e6f0 0x7fc4fc110b40 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fc4f4009ca0 tx=0x7fc4f402c180 comp rx=0 tx=0).stop 2024-04-16T17:06:54.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.804+0000 7fc502e00640 1 -- 172.21.15.45:0/3985170714 shutdown_connections 2024-04-16T17:06:54.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.804+0000 7fc502e00640 1 --2- 172.21.15.45:0/3985170714 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc4fc10e6f0 0x7fc4fc110b40 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:54.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.804+0000 7fc502e00640 1 --2- 172.21.15.45:0/3985170714 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc4fc074d50 0x7fc4fc10e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:54.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.804+0000 7fc502e00640 1 --2- 172.21.15.45:0/3985170714 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc4fc074430 0x7fc4fc074810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:54.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.804+0000 7fc502e00640 1 -- 172.21.15.45:0/3985170714 >> 172.21.15.45:0/3985170714 conn(0x7fc4fc0fd9a0 msgr2=0x7fc4fc0ffdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:06:54.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.804+0000 7fc502e00640 1 -- 172.21.15.45:0/3985170714 shutdown_connections 2024-04-16T17:06:54.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.804+0000 7fc502e00640 1 -- 172.21.15.45:0/3985170714 wait complete. 2024-04-16T17:06:54.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.805+0000 7f4eb7400640 1 --2- 172.21.15.45:0/3987064813 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f4ea40a3da0 0x7f4ea40a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:54.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.805+0000 7f4eb7400640 1 -- 172.21.15.45:0/3987064813 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f4eac0d10b0 con 0x7f4ec0109380 2024-04-16T17:06:54.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.805+0000 7fc502e00640 1 Processor -- start 2024-04-16T17:06:54.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.805+0000 7f4ebea00640 1 --2- 172.21.15.45:0/3987064813 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f4ea40a3da0 0x7f4ea40a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:54.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.806+0000 7fc502e00640 1 -- start start 2024-04-16T17:06:54.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.806+0000 7fc502e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc4fc074430 0x7fc4fc1a0630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:54.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.806+0000 7fc502e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc4fc074d50 0x7fc4fc1a0b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:54.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.806+0000 7fc502e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc4fc10e6f0 0x7fc4fc19a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:54.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.806+0000 7fc502e00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fc4fc114470 con 0x7fc4fc074d50 2024-04-16T17:06:54.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.806+0000 7fc502e00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fc4fc1142f0 con 0x7fc4fc074430 2024-04-16T17:06:54.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.806+0000 7fc502e00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fc4fc1145f0 con 0x7fc4fc10e6f0 2024-04-16T17:06:54.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.806+0000 7fc501a00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc4fc074430 0x7fc4fc1a0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:54.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.806+0000 7fc501a00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc4fc074430 0x7fc4fc1a0630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:33504/0 (socket says 172.21.15.45:33504) 2024-04-16T17:06:54.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.806+0000 7fc501a00640 1 -- 172.21.15.45:0/1117668780 learned_addr learned my addr 172.21.15.45:0/1117668780 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:06:54.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.806+0000 7f4ebea00640 1 --2- 172.21.15.45:0/3987064813 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f4ea40a3da0 0x7f4ea40a6260 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f4eb002c180 tx=0x7f4eb0005a90 comp rx=0 tx=0).ready entity=mgr.25411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:06:54.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.807+0000 7fc502400640 1 --2- 172.21.15.45:0/1117668780 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc4fc10e6f0 0x7fc4fc19a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:54.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.807+0000 7fc501000640 1 --2- 172.21.15.45:0/1117668780 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc4fc074d50 0x7fc4fc1a0b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:54.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.807+0000 7fc501a00640 1 -- 172.21.15.45:0/1117668780 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc4fc10e6f0 msgr2=0x7fc4fc19a7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:54.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.807+0000 7fc501a00640 1 --2- 172.21.15.45:0/1117668780 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc4fc10e6f0 0x7fc4fc19a7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:54.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.807+0000 7fc501a00640 1 -- 172.21.15.45:0/1117668780 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc4fc074d50 msgr2=0x7fc4fc1a0b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:54.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.807+0000 7fc501a00640 1 --2- 172.21.15.45:0/1117668780 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc4fc074d50 0x7fc4fc1a0b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:54.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.807+0000 7fc501a00640 1 -- 172.21.15.45:0/1117668780 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4fc19afc0 con 0x7fc4fc074430 2024-04-16T17:06:54.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.807+0000 7fc502400640 1 --2- 172.21.15.45:0/1117668780 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc4fc10e6f0 0x7fc4fc19a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:06:54.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.807+0000 7fc501a00640 1 --2- 172.21.15.45:0/1117668780 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc4fc074430 0x7fc4fc1a0630 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fc4f000e9f0 tx=0x7fc4f000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:06:54.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.808+0000 7fc4fa000640 1 -- 172.21.15.45:0/1117668780 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4f000cd80 con 0x7fc4fc074430 2024-04-16T17:06:54.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.808+0000 7fc4fa000640 1 -- 172.21.15.45:0/1117668780 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc4f0010910 con 0x7fc4fc074430 2024-04-16T17:06:54.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.808+0000 7fc502e00640 1 -- 172.21.15.45:0/1117668780 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4fc19b2b0 con 0x7fc4fc074430 2024-04-16T17:06:54.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.808+0000 7fc4fa000640 1 -- 172.21.15.45:0/1117668780 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4f0018990 con 0x7fc4fc074430 2024-04-16T17:06:54.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.808+0000 7fc502e00640 1 -- 172.21.15.45:0/1117668780 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4fc1a7480 con 0x7fc4fc074430 2024-04-16T17:06:54.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.810+0000 7f4eb7400640 1 -- 172.21.15.45:0/3987064813 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4eac09d230 con 0x7f4ec0109380 2024-04-16T17:06:54.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.812+0000 7fc4fa000640 1 -- 172.21.15.45:0/1117668780 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 24) ==== 138822+0+0 (secure 0 0 0) 0x7fc4f0010ac0 con 0x7fc4fc074430 2024-04-16T17:06:54.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.812+0000 7fc507046740 1 Processor -- start 2024-04-16T17:06:54.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.812+0000 7fc507046740 1 -- start start 2024-04-16T17:06:54.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.812+0000 7fc507046740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a40d745e90 0x55a40d746270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:54.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.812+0000 7fc507046740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a40d7467b0 0x55a40d74ee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:54.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.813+0000 7fc507046740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a40d74f390 0x55a40d751780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:54.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.813+0000 7fc507046740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55a40d6aacb0 con 0x55a40d7467b0 2024-04-16T17:06:54.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.813+0000 7fc507046740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55a40d651370 con 0x55a40d74f390 2024-04-16T17:06:54.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.813+0000 7fc507046740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55a40d6a6c70 con 0x55a40d745e90 2024-04-16T17:06:54.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.813+0000 7fc502400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a40d745e90 0x55a40d746270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:54.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.813+0000 7fc501a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a40d7467b0 0x55a40d74ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:54.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.813+0000 7fc502400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a40d745e90 0x55a40d746270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:35802/0 (socket says 172.21.15.45:35802) 2024-04-16T17:06:54.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.813+0000 7fc502400640 1 -- 172.21.15.45:0/437224637 learned_addr learned my addr 172.21.15.45:0/437224637 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:06:54.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.813+0000 7fc501000640 1 --2- 172.21.15.45:0/437224637 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a40d74f390 0x55a40d751780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:54.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.813+0000 7fc4fa000640 1 --2- 172.21.15.45:0/1117668780 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fc4e40a3e30 0x7fc4e40a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:54.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.813+0000 7fc501000640 1 -- 172.21.15.45:0/437224637 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a40d745e90 msgr2=0x55a40d746270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:06:54.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.814+0000 7fc501000640 1 --2- 172.21.15.45:0/437224637 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a40d745e90 0x55a40d746270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:54.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.814+0000 7fc501000640 1 -- 172.21.15.45:0/437224637 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a40d7467b0 msgr2=0x55a40d74ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:06:54.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.814+0000 7fc501000640 1 --2- 172.21.15.45:0/437224637 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a40d7467b0 0x55a40d74ee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:54.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.814+0000 7fc501000640 1 -- 172.21.15.45:0/437224637 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a40d670910 con 0x55a40d74f390 2024-04-16T17:06:54.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.814+0000 7fc4fa000640 1 -- 172.21.15.45:0/1117668780 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fc4f00d0fd0 con 0x7fc4fc074430 2024-04-16T17:06:54.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.814+0000 7fc502400640 1 --2- 172.21.15.45:0/1117668780 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fc4e40a3e30 0x7fc4e40a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:54.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.814+0000 7fc501000640 1 --2- 172.21.15.45:0/437224637 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a40d74f390 0x55a40d751780 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fc4fc19baf0 tx=0x7fc4e8011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3068769cf2e55556 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:06:54.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.814+0000 7fc4eea00640 1 -- 172.21.15.45:0/437224637 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4e8009d50 con 0x55a40d74f390 2024-04-16T17:06:54.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.814+0000 7fc4eea00640 1 -- 172.21.15.45:0/437224637 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc4e8005020 con 0x55a40d74f390 2024-04-16T17:06:54.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.814+0000 7fc4eea00640 1 -- 172.21.15.45:0/437224637 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4e8004b20 con 0x55a40d74f390 2024-04-16T17:06:54.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.815+0000 7fc507046740 1 -- 172.21.15.45:0/437224637 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a40d74f390 msgr2=0x55a40d751780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:54.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.815+0000 7fc507046740 1 --2- 172.21.15.45:0/437224637 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a40d74f390 0x55a40d751780 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fc4fc19baf0 tx=0x7fc4e8011810 comp rx=0 tx=0).stop 2024-04-16T17:06:54.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.815+0000 7fc502400640 1 --2- 172.21.15.45:0/1117668780 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fc4e40a3e30 0x7fc4e40a62f0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fc4f4009a60 tx=0x7fc4f402e040 comp rx=0 tx=0).ready entity=mgr.25411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:06:54.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.815+0000 7fc507046740 1 -- 172.21.15.45:0/437224637 shutdown_connections 2024-04-16T17:06:54.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.815+0000 7fc507046740 1 --2- 172.21.15.45:0/437224637 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a40d74f390 0x55a40d751780 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:54.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.815+0000 7fc507046740 1 --2- 172.21.15.45:0/437224637 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a40d7467b0 0x55a40d74ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:54.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.815+0000 7fc507046740 1 --2- 172.21.15.45:0/437224637 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a40d745e90 0x55a40d746270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:54.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.815+0000 7fc507046740 1 -- 172.21.15.45:0/437224637 >> 172.21.15.45:0/437224637 conn(0x55a40d7405c0 msgr2=0x55a40d7409a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:06:54.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.815+0000 7fc507046740 1 -- 172.21.15.45:0/437224637 shutdown_connections 2024-04-16T17:06:54.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.816+0000 7fc507046740 1 -- 172.21.15.45:0/437224637 wait complete. 2024-04-16T17:06:54.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.816+0000 7fc507046740 1 -- start start 2024-04-16T17:06:54.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.816+0000 7fc507046740 1 Processor -- start 2024-04-16T17:06:54.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.817+0000 7fc507046740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a40d745e90 0x55a40d74c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:54.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.817+0000 7fc507046740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a40d7467b0 0x55a40d74c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:54.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.817+0000 7fc507046740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a40d74f390 0x55a40d74ce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:54.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.817+0000 7fc507046740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55a40d752180 con 0x55a40d745e90 2024-04-16T17:06:54.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.817+0000 7fc507046740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55a40d664730 con 0x55a40d7467b0 2024-04-16T17:06:54.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.817+0000 7fc507046740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55a40d752300 con 0x55a40d74f390 2024-04-16T17:06:54.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.817+0000 7fc507046740 10 client.0 _ll_register_callbacks cb 0x55a40d6331e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:06:54.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.817+0000 7fc501a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a40d745e90 0x55a40d74c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:54.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.817+0000 7fc501a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a40d745e90 0x55a40d74c3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55480/0 (socket says 172.21.15.45:55480) 2024-04-16T17:06:54.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.817+0000 7fc501a00640 1 -- 172.21.15.45:0/3273381806 learned_addr learned my addr 172.21.15.45:0/3273381806 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:06:54.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.817+0000 7fc501000640 1 --2- 172.21.15.45:0/3273381806 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a40d7467b0 0x55a40d74c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:54.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.817+0000 7fc502400640 1 --2- 172.21.15.45:0/3273381806 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a40d74f390 0x55a40d74ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:54.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.818+0000 7fc501a00640 1 -- 172.21.15.45:0/3273381806 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a40d74f390 msgr2=0x55a40d74ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:06:54.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.818+0000 7fc501a00640 1 --2- 172.21.15.45:0/3273381806 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a40d74f390 0x55a40d74ce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:54.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.818+0000 7fc501a00640 1 -- 172.21.15.45:0/3273381806 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a40d7467b0 msgr2=0x55a40d74c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:06:54.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.818+0000 7fc501a00640 1 --2- 172.21.15.45:0/3273381806 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a40d7467b0 0x55a40d74c8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:54.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.818+0000 7fc501a00640 1 -- 172.21.15.45:0/3273381806 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a40d4857e0 con 0x55a40d745e90 2024-04-16T17:06:54.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.818+0000 7fc501000640 1 --2- 172.21.15.45:0/3273381806 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a40d7467b0 0x55a40d74c8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:06:54.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.818+0000 7fc501a00640 1 --2- 172.21.15.45:0/3273381806 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a40d745e90 0x55a40d74c3b0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fc4f000e9c0 tx=0x7fc4f00d45b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d1566957bafdfda server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:06:54.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.818+0000 7fc4e3400640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-16T17:06:54.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.818+0000 7fc4e3400640 1 -- 172.21.15.45:0/3273381806 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4f00bd0b0 con 0x55a40d745e90 2024-04-16T17:06:54.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.818+0000 7fc4e3400640 1 -- 172.21.15.45:0/3273381806 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fc4f0018b30 con 0x55a40d745e90 2024-04-16T17:06:54.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.818+0000 7fc507046740 1 -- 172.21.15.45:0/3273381806 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc4e8005020 con 0x55a40d745e90 2024-04-16T17:06:54.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.819+0000 7fc4e3400640 1 -- 172.21.15.45:0/3273381806 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fc4f00a56c0 con 0x55a40d745e90 2024-04-16T17:06:54.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.819+0000 7fc507046740 10 client.15498 fetch_fsmap learned FSMap version 25 2024-04-16T17:06:54.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.819+0000 7fc507046740 1 -- 172.21.15.45:0/3273381806 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55a40d4e5f70 con 0x55a40d745e90 2024-04-16T17:06:54.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.819+0000 7fc4e3400640 1 -- 172.21.15.45:0/3273381806 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc4e8005020 con 0x55a40d745e90 2024-04-16T17:06:54.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.819+0000 7fc4e3400640 1 -- 172.21.15.45:0/3273381806 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fc4f0098970 con 0x55a40d745e90 2024-04-16T17:06:54.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.819+0000 7fc507046740 10 client.15498 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T17:06:54.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.819+0000 7fc507046740 10 client.15498 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:06:54.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.819+0000 7fc507046740 20 client.15498 populate_metadata read hostname 'smithi045' 2024-04-16T17:06:54.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.819+0000 7fc507046740 1 --2- 172.21.15.45:0/3273381806 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55a40d75f6f0 0x55a40d761b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:54.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.819+0000 7fc507046740 4 client.15498 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:06:54.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.819+0000 7fc507046740 1 -- 172.21.15.45:0/3273381806 --> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a40d6b86d0 con 0x55a40d75f6f0 2024-04-16T17:06:54.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.820+0000 7fc501000640 1 --2- 172.21.15.45:0/3273381806 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55a40d75f6f0 0x55a40d761b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:54.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.820+0000 7fc501000640 1 --2- 172.21.15.45:0/3273381806 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55a40d75f6f0 0x55a40d761b60 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=20eca9049ee6cddf server_cookie=9115e9d6a700bbde in_seq=0 out_seq=0 2024-04-16T17:06:54.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.820+0000 7fc4e3400640 10 client.15498 ms_handle_connect on v2:172.21.15.104:6838/3660070592 2024-04-16T17:06:54.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.821+0000 7fc4e3400640 1 -- 172.21.15.45:0/3273381806 <== mds.0 v2:172.21.15.104:6838/3660070592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a40d6b86d0 con 0x55a40d75f6f0 2024-04-16T17:06:54.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.821+0000 7fc4e3400640 10 client.15498 handle_command_reply: tid=0 2024-04-16T17:06:54.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.821+0000 7fc507046740 1 client.15498 shutdown 2024-04-16T17:06:54.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.821+0000 7fc507046740 1 -- 172.21.15.45:0/3273381806 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a40d745e90 msgr2=0x55a40d74c3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:54.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.821+0000 7fc507046740 1 --2- 172.21.15.45:0/3273381806 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a40d745e90 0x55a40d74c3b0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fc4f000e9c0 tx=0x7fc4f00d45b0 comp rx=0 tx=0).stop 2024-04-16T17:06:54.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.821+0000 7fc507046740 1 -- 172.21.15.45:0/3273381806 shutdown_connections 2024-04-16T17:06:54.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.821+0000 7fc507046740 1 --2- 172.21.15.45:0/3273381806 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55a40d75f6f0 0x55a40d761b60 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:54.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.821+0000 7fc507046740 1 --2- 172.21.15.45:0/3273381806 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a40d74f390 0x55a40d74ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:54.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.821+0000 7fc507046740 1 --2- 172.21.15.45:0/3273381806 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a40d7467b0 0x55a40d74c8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:54.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.821+0000 7fc507046740 1 --2- 172.21.15.45:0/3273381806 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a40d745e90 0x55a40d74c3b0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:54.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.821+0000 7fc507046740 1 -- 172.21.15.45:0/3273381806 >> 172.21.15.45:0/3273381806 conn(0x55a40d7405c0 msgr2=0x55a40d742310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:06:54.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.822+0000 7fc4e3400640 0 client.15498 ms_handle_reset on v2:172.21.15.104:6838/3660070592 2024-04-16T17:06:54.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.822+0000 7fc507046740 1 -- 172.21.15.45:0/3273381806 shutdown_connections 2024-04-16T17:06:54.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.822+0000 7fc507046740 1 -- 172.21.15.45:0/3273381806 wait complete. 2024-04-16T17:06:54.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.822+0000 7fc507046740 20 client.15498 trim_cache size 0 max 16384 2024-04-16T17:06:54.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.856+0000 7fc507046740 1 Processor -- start 2024-04-16T17:06:54.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.856+0000 7fc507046740 1 -- start start 2024-04-16T17:06:54.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.856+0000 7fc507046740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a40d745e90 0x55a40d75b040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:54.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.856+0000 7fc507046740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a40d7467b0 0x55a40d74bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:54.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.856+0000 7fc507046740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a40d74f390 0x55a40d767400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:54.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.856+0000 7fc507046740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fc4fc1b3e40 con 0x55a40d7467b0 2024-04-16T17:06:54.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.856+0000 7fc507046740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fc4fc1b1710 con 0x55a40d745e90 2024-04-16T17:06:54.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.856+0000 7fc507046740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fc4fc1af540 con 0x55a40d74f390 2024-04-16T17:06:54.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.857+0000 7fc501000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a40d7467b0 0x55a40d74bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:54.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.857+0000 7fc501a00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a40d745e90 0x55a40d75b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:54.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.857+0000 7fc502400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a40d74f390 0x55a40d767400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:54.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.857+0000 7fc501000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a40d7467b0 0x55a40d74bcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55482/0 (socket says 172.21.15.45:55482) 2024-04-16T17:06:54.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.857+0000 7fc501000640 1 -- 172.21.15.45:0/4081834902 learned_addr learned my addr 172.21.15.45:0/4081834902 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:06:54.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.857+0000 7fc501000640 1 -- 172.21.15.45:0/4081834902 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a40d74f390 msgr2=0x55a40d767400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:06:54.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.857+0000 7fc501000640 1 --2- 172.21.15.45:0/4081834902 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a40d74f390 0x55a40d767400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:54.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.857+0000 7fc501000640 1 -- 172.21.15.45:0/4081834902 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a40d745e90 msgr2=0x55a40d75b040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:06:54.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.857+0000 7fc501000640 1 --2- 172.21.15.45:0/4081834902 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a40d745e90 0x55a40d75b040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:54.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.857+0000 7fc501000640 1 -- 172.21.15.45:0/4081834902 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a40d37d170 con 0x55a40d7467b0 2024-04-16T17:06:54.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.857+0000 7fc501a00640 1 --2- 172.21.15.45:0/4081834902 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a40d745e90 0x55a40d75b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:06:54.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.858+0000 7fc501000640 1 --2- 172.21.15.45:0/4081834902 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a40d7467b0 0x55a40d74bcb0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fc4e8000c00 tx=0x7fc4e8011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=182943a3198a508d server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:06:54.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.858+0000 7fc4ee000640 1 -- 172.21.15.45:0/4081834902 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4e8031070 con 0x55a40d7467b0 2024-04-16T17:06:54.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.858+0000 7fc4ee000640 1 -- 172.21.15.45:0/4081834902 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc4e802b870 con 0x55a40d7467b0 2024-04-16T17:06:54.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.858+0000 7fc4ee000640 1 -- 172.21.15.45:0/4081834902 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4e8003850 con 0x55a40d7467b0 2024-04-16T17:06:54.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.859+0000 7fc507046740 1 -- 172.21.15.45:0/4081834902 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a40d7467b0 msgr2=0x55a40d74bcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:54.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.859+0000 7fc507046740 1 --2- 172.21.15.45:0/4081834902 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a40d7467b0 0x55a40d74bcb0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fc4e8000c00 tx=0x7fc4e8011810 comp rx=0 tx=0).stop 2024-04-16T17:06:54.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.859+0000 7fc507046740 1 -- 172.21.15.45:0/4081834902 shutdown_connections 2024-04-16T17:06:54.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.859+0000 7fc507046740 1 --2- 172.21.15.45:0/4081834902 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a40d74f390 0x55a40d767400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:54.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.859+0000 7fc507046740 1 --2- 172.21.15.45:0/4081834902 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a40d7467b0 0x55a40d74bcb0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:54.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.859+0000 7fc507046740 1 --2- 172.21.15.45:0/4081834902 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a40d745e90 0x55a40d75b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:54.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.859+0000 7fc507046740 1 -- 172.21.15.45:0/4081834902 >> 172.21.15.45:0/4081834902 conn(0x55a40d7405c0 msgr2=0x55a40d742080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:06:54.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.859+0000 7fc507046740 1 -- 172.21.15.45:0/4081834902 shutdown_connections 2024-04-16T17:06:54.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.859+0000 7fc507046740 1 -- 172.21.15.45:0/4081834902 wait complete. 2024-04-16T17:06:54.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.859+0000 7fc507046740 1 -- start start 2024-04-16T17:06:54.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.859+0000 7fc507046740 1 Processor -- start 2024-04-16T17:06:54.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.860+0000 7fc507046740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a40d745e90 0x55a40d7490d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:54.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.860+0000 7fc507046740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a40d7467b0 0x55a40d749610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:54.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.860+0000 7fc507046740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a40d74f390 0x55a40d749b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:54.869 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.860+0000 7fc507046740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fc4fc1afbd0 con 0x55a40d74f390 2024-04-16T17:06:54.869 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.860+0000 7fc507046740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fc4fc1b1040 con 0x55a40d7467b0 2024-04-16T17:06:54.869 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.860+0000 7fc507046740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fc4fc1b02a0 con 0x55a40d745e90 2024-04-16T17:06:54.869 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.860+0000 7fc507046740 10 client.0 _ll_register_callbacks cb 0x7fc4f00d4770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:06:54.869 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.860+0000 7fc502400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a40d74f390 0x55a40d749b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:54.869 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.860+0000 7fc501000640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a40d7467b0 0x55a40d749610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:54.869 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.860+0000 7fc502400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a40d74f390 0x55a40d749b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55490/0 (socket says 172.21.15.45:55490) 2024-04-16T17:06:54.869 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.860+0000 7fc502400640 1 -- 172.21.15.45:0/537344629 learned_addr learned my addr 172.21.15.45:0/537344629 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:06:54.869 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.860+0000 7fc501a00640 1 --2- 172.21.15.45:0/537344629 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a40d745e90 0x55a40d7490d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:54.870 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.861+0000 7fc501000640 1 -- 172.21.15.45:0/537344629 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a40d745e90 msgr2=0x55a40d7490d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:06:54.870 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.861+0000 7fc501000640 1 --2- 172.21.15.45:0/537344629 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a40d745e90 0x55a40d7490d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:54.870 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.861+0000 7fc501000640 1 -- 172.21.15.45:0/537344629 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a40d74f390 msgr2=0x55a40d749b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:06:54.870 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.861+0000 7fc501000640 1 --2- 172.21.15.45:0/537344629 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a40d74f390 0x55a40d749b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:54.870 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.861+0000 7fc501000640 1 -- 172.21.15.45:0/537344629 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a40d7cd4b0 con 0x55a40d7467b0 2024-04-16T17:06:54.870 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.861+0000 7fc501000640 1 --2- 172.21.15.45:0/537344629 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a40d7467b0 0x55a40d749610 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x55a40d73baf0 tx=0x7fc4e8009420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c0f564bcd97eea67 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:06:54.870 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.861+0000 7fc4e3400640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-04-16T17:06:54.870 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.861+0000 7fc4e3400640 1 -- 172.21.15.45:0/537344629 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4e803c060 con 0x55a40d7467b0 2024-04-16T17:06:54.870 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.861+0000 7fc4e3400640 1 -- 172.21.15.45:0/537344629 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fc4e80054e0 con 0x55a40d7467b0 2024-04-16T17:06:54.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.861+0000 7fc507046740 1 -- 172.21.15.45:0/537344629 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc4e802b870 con 0x55a40d7467b0 2024-04-16T17:06:54.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.862+0000 7fc4e3400640 1 -- 172.21.15.45:0/537344629 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fc4e8031070 con 0x55a40d7467b0 2024-04-16T17:06:54.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.862+0000 7fc507046740 10 client.25631 fetch_fsmap learned FSMap version 25 2024-04-16T17:06:54.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.862+0000 7fc507046740 1 -- 172.21.15.45:0/537344629 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55a40d7ce7f0 con 0x55a40d7467b0 2024-04-16T17:06:54.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.862+0000 7fc4e3400640 1 -- 172.21.15.45:0/537344629 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc4e802b870 con 0x55a40d7467b0 2024-04-16T17:06:54.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.862+0000 7fc4e3400640 1 -- 172.21.15.45:0/537344629 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fc4e802b070 con 0x55a40d7467b0 2024-04-16T17:06:54.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.862+0000 7fc507046740 10 client.25631 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T17:06:54.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.862+0000 7fc507046740 10 client.25631 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:06:54.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.862+0000 7fc507046740 20 client.25631 populate_metadata read hostname 'smithi045' 2024-04-16T17:06:54.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.862+0000 7fc507046740 1 --2- 172.21.15.45:0/537344629 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55a40d7ceda0 0x55a40d7d11c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:54.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.862+0000 7fc507046740 4 client.25631 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-16T17:06:54.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.862+0000 7fc507046740 1 -- 172.21.15.45:0/537344629 --> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55a40d6b2540 con 0x55a40d7ceda0 2024-04-16T17:06:54.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.863+0000 7fc501a00640 1 --2- 172.21.15.45:0/537344629 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55a40d7ceda0 0x55a40d7d11c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:54.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.863+0000 7fc501a00640 1 --2- 172.21.15.45:0/537344629 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55a40d7ceda0 0x55a40d7d11c0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b6d16198fd7a49f1 server_cookie=4f5195522cff77a9 in_seq=0 out_seq=0 2024-04-16T17:06:54.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.863+0000 7fc4e3400640 10 client.25631 ms_handle_connect on v2:172.21.15.104:6838/3660070592 2024-04-16T17:06:54.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.864+0000 7fc4e3400640 1 -- 172.21.15.45:0/537344629 <== mds.0 v2:172.21.15.104:6838/3660070592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55a40d6b2540 con 0x55a40d7ceda0 2024-04-16T17:06:54.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.864+0000 7fc4e3400640 10 client.25631 handle_command_reply: tid=0 2024-04-16T17:06:54.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.864+0000 7fc507046740 1 client.25631 shutdown 2024-04-16T17:06:54.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.865+0000 7fc507046740 1 -- 172.21.15.45:0/537344629 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a40d7467b0 msgr2=0x55a40d749610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:54.873 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.865+0000 7fc507046740 1 --2- 172.21.15.45:0/537344629 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a40d7467b0 0x55a40d749610 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x55a40d73baf0 tx=0x7fc4e8009420 comp rx=0 tx=0).stop 2024-04-16T17:06:54.873 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.865+0000 7fc507046740 1 -- 172.21.15.45:0/537344629 shutdown_connections 2024-04-16T17:06:54.873 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.865+0000 7fc507046740 1 --2- 172.21.15.45:0/537344629 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a40d74f390 0x55a40d749b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:54.873 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.865+0000 7fc507046740 1 --2- 172.21.15.45:0/537344629 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a40d7467b0 0x55a40d749610 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:54.873 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.865+0000 7fc507046740 1 --2- 172.21.15.45:0/537344629 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55a40d7ceda0 0x55a40d7d11c0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:54.873 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.865+0000 7fc507046740 1 --2- 172.21.15.45:0/537344629 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a40d745e90 0x55a40d7490d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:54.873 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.865+0000 7fc507046740 1 -- 172.21.15.45:0/537344629 >> 172.21.15.45:0/537344629 conn(0x55a40d7405c0 msgr2=0x55a40d741640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:06:54.873 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.865+0000 7fc4e3400640 0 client.25631 ms_handle_reset on v2:172.21.15.104:6838/3660070592 2024-04-16T17:06:54.873 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.865+0000 7fc507046740 1 -- 172.21.15.45:0/537344629 shutdown_connections 2024-04-16T17:06:54.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.865+0000 7fc507046740 1 -- 172.21.15.45:0/537344629 wait complete. 2024-04-16T17:06:54.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.865+0000 7fc507046740 20 client.25631 trim_cache size 0 max 16384 2024-04-16T17:06:54.874 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-16T17:06:54.874 INFO:teuthology.orchestra.run.smithi045.stdout: "return_code": 0, 2024-04-16T17:06:54.874 INFO:teuthology.orchestra.run.smithi045.stdout: "scrub_tag": "69d01c66-ef71-4d0f-a59b-3ef7c610a99c", 2024-04-16T17:06:54.874 INFO:teuthology.orchestra.run.smithi045.stdout: "mode": "asynchronous" 2024-04-16T17:06:54.875 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-16T17:06:54.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.866+0000 7fc502e00640 1 -- 172.21.15.45:0/1117668780 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fc4e40a3e30 msgr2=0x7fc4e40a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:54.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.866+0000 7fc502e00640 1 --2- 172.21.15.45:0/1117668780 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fc4e40a3e30 0x7fc4e40a62f0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fc4f4009a60 tx=0x7fc4f402e040 comp rx=0 tx=0).stop 2024-04-16T17:06:54.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.866+0000 7fc502e00640 1 -- 172.21.15.45:0/1117668780 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc4fc074430 msgr2=0x7fc4fc1a0630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:54.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.866+0000 7fc502e00640 1 --2- 172.21.15.45:0/1117668780 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc4fc074430 0x7fc4fc1a0630 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fc4f000e9f0 tx=0x7fc4f000eeb0 comp rx=0 tx=0).stop 2024-04-16T17:06:54.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.866+0000 7fc502e00640 1 -- 172.21.15.45:0/1117668780 shutdown_connections 2024-04-16T17:06:54.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.866+0000 7fc502e00640 1 --2- 172.21.15.45:0/1117668780 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fc4e40a3e30 0x7fc4e40a62f0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:54.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.866+0000 7fc502e00640 1 --2- 172.21.15.45:0/1117668780 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc4fc10e6f0 0x7fc4fc19a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:54.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.866+0000 7fc502e00640 1 --2- 172.21.15.45:0/1117668780 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc4fc074d50 0x7fc4fc1a0b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:54.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.866+0000 7fc502e00640 1 --2- 172.21.15.45:0/1117668780 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc4fc074430 0x7fc4fc1a0630 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:54.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.866+0000 7fc502e00640 1 -- 172.21.15.45:0/1117668780 >> 172.21.15.45:0/1117668780 conn(0x7fc4fc0fd9a0 msgr2=0x7fc4fc0ff1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:06:54.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.866+0000 7fc502e00640 1 -- 172.21.15.45:0/1117668780 shutdown_connections 2024-04-16T17:06:54.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:54.866+0000 7fc502e00640 1 -- 172.21.15.45:0/1117668780 wait complete. 2024-04-16T17:06:54.881 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:06:54 smithi045 ceph-mon[31676]: from='client.? 172.21.15.45:0/4055561081' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:06:54.881 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:06:54 smithi045 ceph-mon[31676]: from='client.? 172.21.15.45:0/2279906483' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T17:06:54.884 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-16T17:06:54.884 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-16T17:06:54.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:06:54 smithi104 ceph-mon[36457]: from='client.? 172.21.15.45:0/4055561081' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:06:54.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:06:54 smithi104 ceph-mon[36457]: from='client.? 172.21.15.45:0/2279906483' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T17:06:55.055 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:06:54 smithi117 ceph-mon[36730]: from='client.? 172.21.15.45:0/4055561081' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:06:55.055 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:06:54 smithi117 ceph-mon[36730]: from='client.? 172.21.15.45:0/2279906483' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T17:06:55.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.066+0000 7f4ebfe00640 1 -- 172.21.15.45:0/3987064813 --> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) -- 0x7f4ec0104c60 con 0x7f4ea40a3da0 2024-04-16T17:06:55.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.095+0000 7f1673200640 1 -- 172.21.15.45:0/1561228820 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1674102110 msgr2=0x7f16741024f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:55.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.095+0000 7f1673200640 1 --2- 172.21.15.45:0/1561228820 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1674102110 0x7f16741024f0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f1664009900 tx=0x7f166402bc40 comp rx=0 tx=0).stop 2024-04-16T17:06:55.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.095+0000 7f1673200640 1 -- 172.21.15.45:0/1561228820 shutdown_connections 2024-04-16T17:06:55.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.095+0000 7f1673200640 1 --2- 172.21.15.45:0/1561228820 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f16741071e0 0x7f167410edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:55.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.095+0000 7f1673200640 1 --2- 172.21.15.45:0/1561228820 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1674102ac0 0x7f1674106b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:55.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.095+0000 7f1673200640 1 --2- 172.21.15.45:0/1561228820 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1674102110 0x7f16741024f0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:55.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.095+0000 7f1673200640 1 -- 172.21.15.45:0/1561228820 >> 172.21.15.45:0/1561228820 conn(0x7f16740fd9d0 msgr2=0x7f16740ffdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:06:55.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.096+0000 7f1673200640 1 -- 172.21.15.45:0/1561228820 shutdown_connections 2024-04-16T17:06:55.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.096+0000 7f1673200640 1 -- 172.21.15.45:0/1561228820 wait complete. 2024-04-16T17:06:55.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.097+0000 7f1673200640 1 Processor -- start 2024-04-16T17:06:55.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.097+0000 7f1673200640 1 -- start start 2024-04-16T17:06:55.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.098+0000 7f1673200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1674102110 0x7f16741a0690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:55.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.098+0000 7f1673200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f16741071e0 0x7f16741a0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:55.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.098+0000 7f1673200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f167419a800 0x7f167419acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:55.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.098+0000 7f1673200640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f16741122c0 con 0x7f1674102110 2024-04-16T17:06:55.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.098+0000 7f1673200640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f1674112140 con 0x7f16741071e0 2024-04-16T17:06:55.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.098+0000 7f1673200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f1674112440 con 0x7f167419a800 2024-04-16T17:06:55.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.098+0000 7f1671400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f16741071e0 0x7f16741a0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:55.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.098+0000 7f1671400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f16741071e0 0x7f16741a0bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:33572/0 (socket says 172.21.15.45:33572) 2024-04-16T17:06:55.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.098+0000 7f1671400640 1 -- 172.21.15.45:0/3818751910 learned_addr learned my addr 172.21.15.45:0/3818751910 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:06:55.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.098+0000 7f1671e00640 1 --2- 172.21.15.45:0/3818751910 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1674102110 0x7f16741a0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:55.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.098+0000 7f1672800640 1 --2- 172.21.15.45:0/3818751910 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f167419a800 0x7f167419acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:55.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.098+0000 7f1671400640 1 -- 172.21.15.45:0/3818751910 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f167419a800 msgr2=0x7f167419acb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:55.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.098+0000 7f1671400640 1 --2- 172.21.15.45:0/3818751910 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f167419a800 0x7f167419acb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:55.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.099+0000 7f1671400640 1 -- 172.21.15.45:0/3818751910 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1674102110 msgr2=0x7f16741a0690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:55.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.099+0000 7f1671400640 1 --2- 172.21.15.45:0/3818751910 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1674102110 0x7f16741a0690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:55.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.099+0000 7f1671400640 1 -- 172.21.15.45:0/3818751910 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f167419b390 con 0x7f16741071e0 2024-04-16T17:06:55.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.099+0000 7f1671e00640 1 --2- 172.21.15.45:0/3818751910 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1674102110 0x7f16741a0690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:06:55.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.099+0000 7f1671400640 1 --2- 172.21.15.45:0/3818751910 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f16741071e0 0x7f16741a0bd0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f166000e9a0 tx=0x7f166000ee60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:06:55.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.099+0000 7f166aa00640 1 -- 172.21.15.45:0/3818751910 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f166000cd80 con 0x7f16741071e0 2024-04-16T17:06:55.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.099+0000 7f1673200640 1 -- 172.21.15.45:0/3818751910 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f167419b680 con 0x7f16741071e0 2024-04-16T17:06:55.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.099+0000 7f166aa00640 1 -- 172.21.15.45:0/3818751910 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f16600035f0 con 0x7f16741071e0 2024-04-16T17:06:55.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.099+0000 7f166aa00640 1 -- 172.21.15.45:0/3818751910 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1660010670 con 0x7f16741071e0 2024-04-16T17:06:55.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.099+0000 7f1673200640 1 -- 172.21.15.45:0/3818751910 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16741a78b0 con 0x7f16741071e0 2024-04-16T17:06:55.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.103+0000 7f166aa00640 1 -- 172.21.15.45:0/3818751910 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 24) ==== 138822+0+0 (secure 0 0 0) 0x7f16600037a0 con 0x7f16741071e0 2024-04-16T17:06:55.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.103+0000 7f167b785740 1 Processor -- start 2024-04-16T17:06:55.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.104+0000 7f167b785740 1 -- start start 2024-04-16T17:06:55.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.104+0000 7f167b785740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55726b0f9390 0x55726b0f9770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:55.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.104+0000 7f167b785740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55726b0f9cb0 0x55726b102350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:55.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.104+0000 7f167b785740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55726b102890 0x55726b104c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:55.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.104+0000 7f167b785740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55726b05f120 con 0x55726b102890 2024-04-16T17:06:55.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.104+0000 7f167b785740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55726b005340 con 0x55726b0f9cb0 2024-04-16T17:06:55.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.104+0000 7f167b785740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55726b05b0e0 con 0x55726b0f9390 2024-04-16T17:06:55.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.105+0000 7f1672800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55726b0f9390 0x55726b0f9770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:55.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.105+0000 7f1671e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55726b0f9cb0 0x55726b102350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:55.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.105+0000 7f1672800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55726b0f9390 0x55726b0f9770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:35870/0 (socket says 172.21.15.45:35870) 2024-04-16T17:06:55.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.105+0000 7f1672800640 1 -- 172.21.15.45:0/1922093856 learned_addr learned my addr 172.21.15.45:0/1922093856 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:06:55.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.105+0000 7f1671400640 1 --2- 172.21.15.45:0/1922093856 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55726b102890 0x55726b104c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:55.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.105+0000 7f1672800640 1 -- 172.21.15.45:0/1922093856 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55726b0f9cb0 msgr2=0x55726b102350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:06:55.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.105+0000 7f1672800640 1 --2- 172.21.15.45:0/1922093856 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55726b0f9cb0 0x55726b102350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:55.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.105+0000 7f1672800640 1 -- 172.21.15.45:0/1922093856 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55726b102890 msgr2=0x55726b104c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:06:55.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.105+0000 7f1672800640 1 --2- 172.21.15.45:0/1922093856 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55726b102890 0x55726b104c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:55.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.105+0000 7f1672800640 1 -- 172.21.15.45:0/1922093856 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55726ae3b7d0 con 0x55726b0f9390 2024-04-16T17:06:55.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.105+0000 7f166aa00640 1 --2- 172.21.15.45:0/3818751910 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f16580a3db0 0x7f16580a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:55.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.105+0000 7f166aa00640 1 -- 172.21.15.45:0/3818751910 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f16600d10f0 con 0x7f16741071e0 2024-04-16T17:06:55.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.105+0000 7f1672800640 1 --2- 172.21.15.45:0/1922093856 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55726b0f9390 0x55726b0f9770 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f166c009f40 tx=0x7f166c010860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c8c5559583d99c25 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:06:55.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.106+0000 7f1671e00640 1 --2- 172.21.15.45:0/3818751910 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f16580a3db0 0x7f16580a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:55.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.106+0000 7f165f000640 1 -- 172.21.15.45:0/1922093856 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f166c003b70 con 0x55726b0f9390 2024-04-16T17:06:55.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.106+0000 7f165f000640 1 -- 172.21.15.45:0/1922093856 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f166c003d10 con 0x55726b0f9390 2024-04-16T17:06:55.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.106+0000 7f165f000640 1 -- 172.21.15.45:0/1922093856 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f166c011a90 con 0x55726b0f9390 2024-04-16T17:06:55.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.107+0000 7f167b785740 1 -- 172.21.15.45:0/1922093856 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55726b0f9390 msgr2=0x55726b0f9770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:55.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.107+0000 7f167b785740 1 --2- 172.21.15.45:0/1922093856 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55726b0f9390 0x55726b0f9770 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f166c009f40 tx=0x7f166c010860 comp rx=0 tx=0).stop 2024-04-16T17:06:55.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.107+0000 7f1671e00640 1 --2- 172.21.15.45:0/3818751910 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f16580a3db0 0x7f16580a6270 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f1664006010 tx=0x7f166402cf90 comp rx=0 tx=0).ready entity=mgr.25411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:06:55.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.107+0000 7f167b785740 1 -- 172.21.15.45:0/1922093856 shutdown_connections 2024-04-16T17:06:55.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.107+0000 7f167b785740 1 --2- 172.21.15.45:0/1922093856 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55726b102890 0x55726b104c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:55.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.107+0000 7f167b785740 1 --2- 172.21.15.45:0/1922093856 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55726b0f9cb0 0x55726b102350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:55.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.107+0000 7f167b785740 1 --2- 172.21.15.45:0/1922093856 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55726b0f9390 0x55726b0f9770 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:55.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.107+0000 7f167b785740 1 -- 172.21.15.45:0/1922093856 >> 172.21.15.45:0/1922093856 conn(0x55726b0f3ac0 msgr2=0x55726b0f3ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:06:55.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.107+0000 7f167b785740 1 -- 172.21.15.45:0/1922093856 shutdown_connections 2024-04-16T17:06:55.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.107+0000 7f167b785740 1 -- 172.21.15.45:0/1922093856 wait complete. 2024-04-16T17:06:55.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.108+0000 7f167b785740 1 -- start start 2024-04-16T17:06:55.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.108+0000 7f167b785740 1 Processor -- start 2024-04-16T17:06:55.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.108+0000 7f167b785740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55726b0f9390 0x55726b0ff930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:55.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.108+0000 7f167b785740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55726b0f9cb0 0x55726b0ffe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:55.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.108+0000 7f167b785740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55726b102890 0x55726b1003b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:55.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.108+0000 7f167b785740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55726b105680 con 0x55726b0f9390 2024-04-16T17:06:55.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.108+0000 7f167b785740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55726b018710 con 0x55726b102890 2024-04-16T17:06:55.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.108+0000 7f167b785740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55726b105800 con 0x55726b0f9cb0 2024-04-16T17:06:55.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.109+0000 7f167b785740 10 client.0 _ll_register_callbacks cb 0x55726b013af0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:06:55.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.109+0000 7f1671400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55726b0f9cb0 0x55726b0ffe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:55.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.109+0000 7f1671400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55726b0f9cb0 0x55726b0ffe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:35882/0 (socket says 172.21.15.45:35882) 2024-04-16T17:06:55.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.109+0000 7f1671400640 1 -- 172.21.15.45:0/134725708 learned_addr learned my addr 172.21.15.45:0/134725708 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:06:55.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.109+0000 7f1672800640 1 --2- 172.21.15.45:0/134725708 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55726b102890 0x55726b1003b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:55.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.109+0000 7f1671e00640 1 --2- 172.21.15.45:0/134725708 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55726b0f9390 0x55726b0ff930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:55.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.109+0000 7f1671400640 1 -- 172.21.15.45:0/134725708 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55726b102890 msgr2=0x55726b1003b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:06:55.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.109+0000 7f1671400640 1 --2- 172.21.15.45:0/134725708 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55726b102890 0x55726b1003b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:55.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.109+0000 7f1671400640 1 -- 172.21.15.45:0/134725708 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55726b0f9390 msgr2=0x55726b0ff930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:06:55.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.109+0000 7f1671400640 1 --2- 172.21.15.45:0/134725708 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55726b0f9390 0x55726b0ff930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:55.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.109+0000 7f1671400640 1 -- 172.21.15.45:0/134725708 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55726ad713f0 con 0x55726b0f9cb0 2024-04-16T17:06:55.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.109+0000 7f1672800640 1 --2- 172.21.15.45:0/134725708 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55726b102890 0x55726b1003b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:06:55.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.109+0000 7f1671e00640 1 --2- 172.21.15.45:0/134725708 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55726b0f9390 0x55726b0ff930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:06:55.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.109+0000 7f1671400640 1 --2- 172.21.15.45:0/134725708 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55726b0f9cb0 0x55726b0ffe70 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f166000e970 tx=0x7f166009ca00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9f79720cbe7a0de2 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:06:55.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.109+0000 7f1653400640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-04-16T17:06:55.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.110+0000 7f1653400640 1 -- 172.21.15.45:0/134725708 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f166009cb70 con 0x55726b0f9cb0 2024-04-16T17:06:55.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.110+0000 7f1653400640 1 -- 172.21.15.45:0/134725708 <== mon.1 v2:172.21.15.117:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f16600bd070 con 0x55726b0f9cb0 2024-04-16T17:06:55.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.110+0000 7f167b785740 1 -- 172.21.15.45:0/134725708 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f166c003d10 con 0x55726b0f9cb0 2024-04-16T17:06:55.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.110+0000 7f1653400640 1 -- 172.21.15.45:0/134725708 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f16600a0f20 con 0x55726b0f9cb0 2024-04-16T17:06:55.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.110+0000 7f167b785740 10 client.25639 fetch_fsmap learned FSMap version 25 2024-04-16T17:06:55.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.110+0000 7f167b785740 1 -- 172.21.15.45:0/134725708 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55726ad310f0 con 0x55726b0f9cb0 2024-04-16T17:06:55.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.110+0000 7f1653400640 1 -- 172.21.15.45:0/134725708 <== mon.1 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f166c003d10 con 0x55726b0f9cb0 2024-04-16T17:06:55.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.110+0000 7f1653400640 1 -- 172.21.15.45:0/134725708 <== mon.1 v2:172.21.15.117:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f16600b7020 con 0x55726b0f9cb0 2024-04-16T17:06:55.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.111+0000 7f167b785740 10 client.25639 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T17:06:55.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.111+0000 7f167b785740 10 client.25639 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:06:55.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.111+0000 7f167b785740 20 client.25639 populate_metadata read hostname 'smithi045' 2024-04-16T17:06:55.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.111+0000 7f167b785740 1 --2- 172.21.15.45:0/134725708 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55726b112c70 0x55726b115080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:55.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.111+0000 7f167b785740 4 client.25639 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:06:55.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.111+0000 7f167b785740 1 -- 172.21.15.45:0/134725708 --> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55726b06cb40 con 0x55726b112c70 2024-04-16T17:06:55.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.111+0000 7f1671e00640 1 --2- 172.21.15.45:0/134725708 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55726b112c70 0x55726b115080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:55.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.112+0000 7f1671e00640 1 --2- 172.21.15.45:0/134725708 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55726b112c70 0x55726b115080 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b076533f8ed29ff server_cookie=77705f2907318e8 in_seq=0 out_seq=0 2024-04-16T17:06:55.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.112+0000 7f1653400640 10 client.25639 ms_handle_connect on v2:172.21.15.104:6838/3660070592 2024-04-16T17:06:55.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.113+0000 7f1653400640 1 -- 172.21.15.45:0/134725708 <== mds.0 v2:172.21.15.104:6838/3660070592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55726b06cb40 con 0x55726b112c70 2024-04-16T17:06:55.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.113+0000 7f1653400640 10 client.25639 handle_command_reply: tid=0 2024-04-16T17:06:55.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.113+0000 7f167b785740 1 client.25639 shutdown 2024-04-16T17:06:55.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.113+0000 7f167b785740 1 -- 172.21.15.45:0/134725708 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55726b0f9cb0 msgr2=0x55726b0ffe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:55.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.113+0000 7f167b785740 1 --2- 172.21.15.45:0/134725708 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55726b0f9cb0 0x55726b0ffe70 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f166000e970 tx=0x7f166009ca00 comp rx=0 tx=0).stop 2024-04-16T17:06:55.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.113+0000 7f167b785740 1 -- 172.21.15.45:0/134725708 shutdown_connections 2024-04-16T17:06:55.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.113+0000 7f167b785740 1 --2- 172.21.15.45:0/134725708 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55726b112c70 0x55726b115080 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:55.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.113+0000 7f167b785740 1 --2- 172.21.15.45:0/134725708 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55726b0f9cb0 0x55726b0ffe70 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:55.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.113+0000 7f167b785740 1 --2- 172.21.15.45:0/134725708 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55726b102890 0x55726b1003b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:55.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.113+0000 7f1653400640 0 client.25639 ms_handle_reset on v2:172.21.15.104:6838/3660070592 2024-04-16T17:06:55.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.113+0000 7f167b785740 1 --2- 172.21.15.45:0/134725708 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55726b0f9390 0x55726b0ff930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:55.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.113+0000 7f167b785740 1 -- 172.21.15.45:0/134725708 >> 172.21.15.45:0/134725708 conn(0x55726b0f3ac0 msgr2=0x55726b0f5810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:06:55.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.113+0000 7f167b785740 1 -- 172.21.15.45:0/134725708 shutdown_connections 2024-04-16T17:06:55.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.113+0000 7f167b785740 1 -- 172.21.15.45:0/134725708 wait complete. 2024-04-16T17:06:55.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.114+0000 7f167b785740 20 client.25639 trim_cache size 0 max 16384 2024-04-16T17:06:55.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.138+0000 7f167b785740 1 Processor -- start 2024-04-16T17:06:55.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.138+0000 7f167b785740 1 -- start start 2024-04-16T17:06:55.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.138+0000 7f167b785740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55726b0f9390 0x55726b102060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:55.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.138+0000 7f167b785740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55726b0f9cb0 0x55726b10e540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:55.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.138+0000 7f167b785740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55726b102890 0x55726b0ee9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:55.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.138+0000 7f167b785740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f16741b3e70 con 0x55726b102890 2024-04-16T17:06:55.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.138+0000 7f167b785740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f16741b1740 con 0x55726b0f9cb0 2024-04-16T17:06:55.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.138+0000 7f167b785740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f16741af570 con 0x55726b0f9390 2024-04-16T17:06:55.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.139+0000 7f1671e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55726b0f9390 0x55726b102060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:55.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.139+0000 7f1671e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55726b0f9390 0x55726b102060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:35886/0 (socket says 172.21.15.45:35886) 2024-04-16T17:06:55.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.139+0000 7f1671e00640 1 -- 172.21.15.45:0/2130007088 learned_addr learned my addr 172.21.15.45:0/2130007088 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:06:55.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.139+0000 7f1671400640 1 --2- 172.21.15.45:0/2130007088 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55726b0f9cb0 0x55726b10e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:55.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.139+0000 7f1672800640 1 --2- 172.21.15.45:0/2130007088 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55726b102890 0x55726b0ee9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:55.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.139+0000 7f1671e00640 1 -- 172.21.15.45:0/2130007088 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55726b0f9cb0 msgr2=0x55726b10e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:06:55.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.139+0000 7f1671e00640 1 --2- 172.21.15.45:0/2130007088 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55726b0f9cb0 0x55726b10e540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:55.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.139+0000 7f1671e00640 1 -- 172.21.15.45:0/2130007088 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55726b102890 msgr2=0x55726b0ee9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:06:55.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.139+0000 7f1671e00640 1 --2- 172.21.15.45:0/2130007088 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55726b102890 0x55726b0ee9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:55.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.139+0000 7f1671e00640 1 -- 172.21.15.45:0/2130007088 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55726b0fe8e0 con 0x55726b0f9390 2024-04-16T17:06:55.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.139+0000 7f1671e00640 1 --2- 172.21.15.45:0/2130007088 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55726b0f9390 0x55726b102060 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f1664004650 tx=0x7f1664040070 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a65876a8da4d3e06 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:06:55.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.140+0000 7f165e600640 1 -- 172.21.15.45:0/2130007088 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f166402cb60 con 0x55726b0f9390 2024-04-16T17:06:55.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.140+0000 7f165e600640 1 -- 172.21.15.45:0/2130007088 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f166402cd00 con 0x55726b0f9390 2024-04-16T17:06:55.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.140+0000 7f165e600640 1 -- 172.21.15.45:0/2130007088 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1664004920 con 0x55726b0f9390 2024-04-16T17:06:55.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.141+0000 7f167b785740 1 -- 172.21.15.45:0/2130007088 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55726b0f9390 msgr2=0x55726b102060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:55.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.141+0000 7f167b785740 1 --2- 172.21.15.45:0/2130007088 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55726b0f9390 0x55726b102060 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f1664004650 tx=0x7f1664040070 comp rx=0 tx=0).stop 2024-04-16T17:06:55.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.141+0000 7f167b785740 1 -- 172.21.15.45:0/2130007088 shutdown_connections 2024-04-16T17:06:55.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.141+0000 7f167b785740 1 --2- 172.21.15.45:0/2130007088 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55726b102890 0x55726b0ee9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:55.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.141+0000 7f167b785740 1 --2- 172.21.15.45:0/2130007088 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55726b0f9cb0 0x55726b10e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:55.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.141+0000 7f167b785740 1 --2- 172.21.15.45:0/2130007088 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55726b0f9390 0x55726b102060 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:55.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.141+0000 7f167b785740 1 -- 172.21.15.45:0/2130007088 >> 172.21.15.45:0/2130007088 conn(0x55726b0f3ac0 msgr2=0x55726b0f4ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:06:55.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.141+0000 7f167b785740 1 -- 172.21.15.45:0/2130007088 shutdown_connections 2024-04-16T17:06:55.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.141+0000 7f167b785740 1 -- 172.21.15.45:0/2130007088 wait complete. 2024-04-16T17:06:55.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.141+0000 7f167b785740 1 -- start start 2024-04-16T17:06:55.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.142+0000 7f167b785740 1 Processor -- start 2024-04-16T17:06:55.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.142+0000 7f167b785740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55726b0f9390 0x55726b10af50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:55.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.142+0000 7f167b785740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55726b0f9cb0 0x55726b1062c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:55.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.142+0000 7f167b785740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55726b102890 0x55726b106a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:55.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.142+0000 7f167b785740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f16741afc00 con 0x55726b102890 2024-04-16T17:06:55.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.142+0000 7f167b785740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f16741b1070 con 0x55726b0f9390 2024-04-16T17:06:55.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.142+0000 7f167b785740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f16741b02d0 con 0x55726b0f9cb0 2024-04-16T17:06:55.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.142+0000 7f167b785740 10 client.0 _ll_register_callbacks cb 0x7f16600ce260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:06:55.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.142+0000 7f1671400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55726b0f9cb0 0x55726b1062c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:55.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.142+0000 7f1671400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55726b0f9cb0 0x55726b1062c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:35902/0 (socket says 172.21.15.45:35902) 2024-04-16T17:06:55.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.143+0000 7f1671400640 1 -- 172.21.15.45:0/3209916499 learned_addr learned my addr 172.21.15.45:0/3209916499 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:06:55.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.143+0000 7f1671e00640 1 --2- 172.21.15.45:0/3209916499 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55726b0f9390 0x55726b10af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:55.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.143+0000 7f1672800640 1 --2- 172.21.15.45:0/3209916499 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55726b102890 0x55726b106a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:55.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.143+0000 7f1671400640 1 -- 172.21.15.45:0/3209916499 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55726b0f9390 msgr2=0x55726b10af50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:06:55.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.143+0000 7f1671400640 1 --2- 172.21.15.45:0/3209916499 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55726b0f9390 0x55726b10af50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:55.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.143+0000 7f1671400640 1 -- 172.21.15.45:0/3209916499 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55726b102890 msgr2=0x55726b106a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:06:55.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.143+0000 7f1671400640 1 --2- 172.21.15.45:0/3209916499 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55726b102890 0x55726b106a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:55.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.143+0000 7f1671400640 1 -- 172.21.15.45:0/3209916499 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55726b178170 con 0x55726b0f9cb0 2024-04-16T17:06:55.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.143+0000 7f1671e00640 1 --2- 172.21.15.45:0/3209916499 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55726b0f9390 0x55726b10af50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:06:55.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.143+0000 7f1672800640 1 --2- 172.21.15.45:0/3209916499 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55726b102890 0x55726b106a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:06:55.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.143+0000 7f1671400640 1 --2- 172.21.15.45:0/3209916499 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55726b0f9cb0 0x55726b1062c0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f1660098fd0 tx=0x7f16600d3150 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5f51451a23dbf188 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:06:55.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.143+0000 7f1653400640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-04-16T17:06:55.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.143+0000 7f1653400640 1 -- 172.21.15.45:0/3209916499 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1660098e10 con 0x55726b0f9cb0 2024-04-16T17:06:55.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.143+0000 7f1653400640 1 -- 172.21.15.45:0/3209916499 <== mon.1 v2:172.21.15.117:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f16600ce680 con 0x55726b0f9cb0 2024-04-16T17:06:55.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.143+0000 7f167b785740 1 -- 172.21.15.45:0/3209916499 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f166402cd00 con 0x55726b0f9cb0 2024-04-16T17:06:55.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.144+0000 7f1653400640 1 -- 172.21.15.45:0/3209916499 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f166009c4d0 con 0x55726b0f9cb0 2024-04-16T17:06:55.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.144+0000 7f167b785740 10 client.25645 fetch_fsmap learned FSMap version 25 2024-04-16T17:06:55.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.144+0000 7f167b785740 1 -- 172.21.15.45:0/3209916499 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55726b1792b0 con 0x55726b0f9cb0 2024-04-16T17:06:55.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.144+0000 7f1653400640 1 -- 172.21.15.45:0/3209916499 <== mon.1 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f166402cd00 con 0x55726b0f9cb0 2024-04-16T17:06:55.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.144+0000 7f1653400640 1 -- 172.21.15.45:0/3209916499 <== mon.1 v2:172.21.15.117:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f16600b7070 con 0x55726b0f9cb0 2024-04-16T17:06:55.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.144+0000 7f167b785740 10 client.25645 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T17:06:55.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.144+0000 7f167b785740 10 client.25645 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:06:55.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.144+0000 7f167b785740 20 client.25645 populate_metadata read hostname 'smithi045' 2024-04-16T17:06:55.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.145+0000 7f167b785740 1 --2- 172.21.15.45:0/3209916499 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55726b179a10 0x55726b17be30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:55.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.145+0000 7f167b785740 4 client.25645 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-16T17:06:55.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.145+0000 7f167b785740 1 -- 172.21.15.45:0/3209916499 --> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55726b0ef180 con 0x55726b179a10 2024-04-16T17:06:55.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.145+0000 7f1671e00640 1 --2- 172.21.15.45:0/3209916499 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55726b179a10 0x55726b17be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:55.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.146+0000 7f1671e00640 1 --2- 172.21.15.45:0/3209916499 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55726b179a10 0x55726b17be30 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=50a9af3125b41080 server_cookie=f23c49a8df229076 in_seq=0 out_seq=0 2024-04-16T17:06:55.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.146+0000 7f1653400640 10 client.25645 ms_handle_connect on v2:172.21.15.104:6838/3660070592 2024-04-16T17:06:55.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.146+0000 7f1653400640 1 -- 172.21.15.45:0/3209916499 <== mds.0 v2:172.21.15.104:6838/3660070592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55726b0ef180 con 0x55726b179a10 2024-04-16T17:06:55.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.146+0000 7f1653400640 10 client.25645 handle_command_reply: tid=0 2024-04-16T17:06:55.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.146+0000 7f167b785740 1 client.25645 shutdown 2024-04-16T17:06:55.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.146+0000 7f167b785740 1 -- 172.21.15.45:0/3209916499 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55726b0f9cb0 msgr2=0x55726b1062c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:55.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.146+0000 7f167b785740 1 --2- 172.21.15.45:0/3209916499 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55726b0f9cb0 0x55726b1062c0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f1660098fd0 tx=0x7f16600d3150 comp rx=0 tx=0).stop 2024-04-16T17:06:55.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.147+0000 7f167b785740 1 -- 172.21.15.45:0/3209916499 shutdown_connections 2024-04-16T17:06:55.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.147+0000 7f167b785740 1 --2- 172.21.15.45:0/3209916499 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55726b179a10 0x55726b17be30 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:55.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.147+0000 7f167b785740 1 --2- 172.21.15.45:0/3209916499 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55726b0f9cb0 0x55726b1062c0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:55.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.147+0000 7f167b785740 1 --2- 172.21.15.45:0/3209916499 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55726b102890 0x55726b106a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:55.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.147+0000 7f167b785740 1 --2- 172.21.15.45:0/3209916499 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55726b0f9390 0x55726b10af50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:55.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.147+0000 7f1653400640 0 client.25645 ms_handle_reset on v2:172.21.15.104:6838/3660070592 2024-04-16T17:06:55.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.147+0000 7f167b785740 1 -- 172.21.15.45:0/3209916499 >> 172.21.15.45:0/3209916499 conn(0x55726b0f3ac0 msgr2=0x55726b0f5780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:06:55.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.147+0000 7f167b785740 1 -- 172.21.15.45:0/3209916499 shutdown_connections 2024-04-16T17:06:55.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.147+0000 7f167b785740 1 -- 172.21.15.45:0/3209916499 wait complete. 2024-04-16T17:06:55.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.147+0000 7f167b785740 20 client.25645 trim_cache size 0 max 16384 2024-04-16T17:06:55.155 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-16T17:06:55.155 INFO:teuthology.orchestra.run.smithi045.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-16T17:06:55.155 INFO:teuthology.orchestra.run.smithi045.stdout: "scrubs": { 2024-04-16T17:06:55.155 INFO:teuthology.orchestra.run.smithi045.stdout: "69d01c66-ef71-4d0f-a59b-3ef7c610a99c": { 2024-04-16T17:06:55.155 INFO:teuthology.orchestra.run.smithi045.stdout: "path": "/", 2024-04-16T17:06:55.156 INFO:teuthology.orchestra.run.smithi045.stdout: "tag": "69d01c66-ef71-4d0f-a59b-3ef7c610a99c", 2024-04-16T17:06:55.156 INFO:teuthology.orchestra.run.smithi045.stdout: "options": "recursive,force" 2024-04-16T17:06:55.156 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:06:55.156 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:06:55.156 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-16T17:06:55.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.148+0000 7f1673200640 1 -- 172.21.15.45:0/3818751910 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f16580a3db0 msgr2=0x7f16580a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:55.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.148+0000 7f1673200640 1 --2- 172.21.15.45:0/3818751910 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f16580a3db0 0x7f16580a6270 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f1664006010 tx=0x7f166402cf90 comp rx=0 tx=0).stop 2024-04-16T17:06:55.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.148+0000 7f1673200640 1 -- 172.21.15.45:0/3818751910 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f16741071e0 msgr2=0x7f16741a0bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:55.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.148+0000 7f1673200640 1 --2- 172.21.15.45:0/3818751910 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f16741071e0 0x7f16741a0bd0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f166000e9a0 tx=0x7f166000ee60 comp rx=0 tx=0).stop 2024-04-16T17:06:55.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.148+0000 7f1673200640 1 -- 172.21.15.45:0/3818751910 shutdown_connections 2024-04-16T17:06:55.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.148+0000 7f1673200640 1 --2- 172.21.15.45:0/3818751910 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f16580a3db0 0x7f16580a6270 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:55.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.148+0000 7f1673200640 1 --2- 172.21.15.45:0/3818751910 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f167419a800 0x7f167419acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:55.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.148+0000 7f1673200640 1 --2- 172.21.15.45:0/3818751910 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f16741071e0 0x7f16741a0bd0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:55.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.148+0000 7f1673200640 1 --2- 172.21.15.45:0/3818751910 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1674102110 0x7f16741a0690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:55.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.148+0000 7f1673200640 1 -- 172.21.15.45:0/3818751910 >> 172.21.15.45:0/3818751910 conn(0x7f16740fd9d0 msgr2=0x7f16740fec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:06:55.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.148+0000 7f1673200640 1 -- 172.21.15.45:0/3818751910 shutdown_connections 2024-04-16T17:06:55.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.148+0000 7f1673200640 1 -- 172.21.15.45:0/3818751910 wait complete. 2024-04-16T17:06:55.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.152+0000 7f4eb7400640 1 -- 172.21.15.45:0/3987064813 <== mgr.25411 v2:172.21.15.45:6800/751128195 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f4ec0104c60 con 0x7f4ea40a3da0 2024-04-16T17:06:55.159 INFO:teuthology.orchestra.run.smithi045.stdout:Schedule set for path / 2024-04-16T17:06:55.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.154+0000 7f4ebfe00640 1 -- 172.21.15.45:0/3987064813 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f4ea40a3da0 msgr2=0x7f4ea40a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:55.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.154+0000 7f4ebfe00640 1 --2- 172.21.15.45:0/3987064813 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f4ea40a3da0 0x7f4ea40a6260 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f4eb002c180 tx=0x7f4eb0005a90 comp rx=0 tx=0).stop 2024-04-16T17:06:55.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.154+0000 7f4ebfe00640 1 -- 172.21.15.45:0/3987064813 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4ec0109380 msgr2=0x7f4ec01a0bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:55.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.154+0000 7f4ebfe00640 1 --2- 172.21.15.45:0/3987064813 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4ec0109380 0x7f4ec01a0bf0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f4eac00e9f0 tx=0x7f4eac00edf0 comp rx=0 tx=0).stop 2024-04-16T17:06:55.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.155+0000 7f4ebfe00640 1 -- 172.21.15.45:0/3987064813 shutdown_connections 2024-04-16T17:06:55.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.155+0000 7f4ebfe00640 1 --2- 172.21.15.45:0/3987064813 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f4ea40a3da0 0x7f4ea40a6260 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:55.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.155+0000 7f4ebfe00640 1 --2- 172.21.15.45:0/3987064813 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4ec0109380 0x7f4ec01a0bf0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:55.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.155+0000 7f4ebfe00640 1 --2- 172.21.15.45:0/3987064813 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4ec019a770 0x7f4ec019abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:55.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.155+0000 7f4ebfe00640 1 --2- 172.21.15.45:0/3987064813 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4ec01042b0 0x7f4ec01a06b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:55.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.155+0000 7f4ebfe00640 1 -- 172.21.15.45:0/3987064813 >> 172.21.15.45:0/3987064813 conn(0x7f4ec00ffb10 msgr2=0x7f4ec0101e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:06:55.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.155+0000 7f4ebfe00640 1 -- 172.21.15.45:0/3987064813 shutdown_connections 2024-04-16T17:06:55.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.155+0000 7f4ebfe00640 1 -- 172.21.15.45:0/3987064813 wait complete. 2024-04-16T17:06:55.169 INFO:tasks.cephfs.filesystem:scrub status for tag:69d01c66-ef71-4d0f-a59b-3ef7c610a99c - {'path': '/', 'tag': '69d01c66-ef71-4d0f-a59b-3ef7c610a99c', 'options': 'recursive,force'} 2024-04-16T17:06:55.176 DEBUG:teuthology.orchestra.run.smithi045:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h' 2024-04-16T17:06:55.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.425+0000 7f80e3200640 1 -- 172.21.15.45:0/3550992976 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f80e4074cd0 msgr2=0x7f80e410e1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:55.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.425+0000 7f80e3200640 1 --2- 172.21.15.45:0/3550992976 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f80e4074cd0 0x7f80e410e1a0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f80cc009860 tx=0x7f80cc02bc00 comp rx=0 tx=0).stop 2024-04-16T17:06:55.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.425+0000 7f80e3200640 1 -- 172.21.15.45:0/3550992976 shutdown_connections 2024-04-16T17:06:55.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.425+0000 7f80e3200640 1 --2- 172.21.15.45:0/3550992976 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f80e410e6e0 0x7f80e4110ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:55.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.425+0000 7f80e3200640 1 --2- 172.21.15.45:0/3550992976 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f80e4074cd0 0x7f80e410e1a0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:55.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.425+0000 7f80e3200640 1 --2- 172.21.15.45:0/3550992976 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f80e40743b0 0x7f80e4074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:55.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.425+0000 7f80e3200640 1 -- 172.21.15.45:0/3550992976 >> 172.21.15.45:0/3550992976 conn(0x7f80e40fd950 msgr2=0x7f80e40ffd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:06:55.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.425+0000 7f80e3200640 1 -- 172.21.15.45:0/3550992976 shutdown_connections 2024-04-16T17:06:55.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.425+0000 7f80e3200640 1 -- 172.21.15.45:0/3550992976 wait complete. 2024-04-16T17:06:55.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.426+0000 7f80e3200640 1 Processor -- start 2024-04-16T17:06:55.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.426+0000 7f80e3200640 1 -- start start 2024-04-16T17:06:55.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.427+0000 7f80e3200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f80e40743b0 0x7f80e4101e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:55.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.427+0000 7f80e3200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f80e4074cd0 0x7f80e4102340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:55.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.427+0000 7f80e3200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f80e410e6e0 0x7f80e4102880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:55.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.427+0000 7f80e3200640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f80e41143f0 con 0x7f80e40743b0 2024-04-16T17:06:55.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.427+0000 7f80e3200640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f80e4114270 con 0x7f80e4074cd0 2024-04-16T17:06:55.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.427+0000 7f80e3200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f80e4114570 con 0x7f80e410e6e0 2024-04-16T17:06:55.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.427+0000 7f80e1e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f80e40743b0 0x7f80e4101e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:55.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.427+0000 7f80e1e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f80e40743b0 0x7f80e4101e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55534/0 (socket says 172.21.15.45:55534) 2024-04-16T17:06:55.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.427+0000 7f80e1e00640 1 -- 172.21.15.45:0/925826051 learned_addr learned my addr 172.21.15.45:0/925826051 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:06:55.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.427+0000 7f80d9400640 1 --2- 172.21.15.45:0/925826051 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f80e4074cd0 0x7f80e4102340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:55.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.427+0000 7f80e2800640 1 --2- 172.21.15.45:0/925826051 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f80e410e6e0 0x7f80e4102880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:55.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.428+0000 7f80e1e00640 1 -- 172.21.15.45:0/925826051 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f80e410e6e0 msgr2=0x7f80e4102880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:55.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.428+0000 7f80e1e00640 1 --2- 172.21.15.45:0/925826051 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f80e410e6e0 0x7f80e4102880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:55.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.428+0000 7f80e1e00640 1 -- 172.21.15.45:0/925826051 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f80e4074cd0 msgr2=0x7f80e4102340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:55.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.428+0000 7f80e1e00640 1 --2- 172.21.15.45:0/925826051 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f80e4074cd0 0x7f80e4102340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:55.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.428+0000 7f80e1e00640 1 -- 172.21.15.45:0/925826051 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80e41aafe0 con 0x7f80e40743b0 2024-04-16T17:06:55.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.428+0000 7f80d9400640 1 --2- 172.21.15.45:0/925826051 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f80e4074cd0 0x7f80e4102340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:06:55.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.428+0000 7f80e1e00640 1 --2- 172.21.15.45:0/925826051 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f80e40743b0 0x7f80e4101e00 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f80d400e9f0 tx=0x7f80d400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:06:55.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.428+0000 7f80db400640 1 -- 172.21.15.45:0/925826051 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80d400ccc0 con 0x7f80e40743b0 2024-04-16T17:06:55.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.428+0000 7f80db400640 1 -- 172.21.15.45:0/925826051 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f80d400ce60 con 0x7f80e40743b0 2024-04-16T17:06:55.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.428+0000 7f80e3200640 1 -- 172.21.15.45:0/925826051 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80e41ab2d0 con 0x7f80e40743b0 2024-04-16T17:06:55.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.428+0000 7f80db400640 1 -- 172.21.15.45:0/925826051 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80d4010690 con 0x7f80e40743b0 2024-04-16T17:06:55.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.429+0000 7f80e3200640 1 -- 172.21.15.45:0/925826051 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80e41ab7f0 con 0x7f80e40743b0 2024-04-16T17:06:55.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.430+0000 7f80e3200640 1 -- 172.21.15.45:0/925826051 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80e4075a60 con 0x7f80e40743b0 2024-04-16T17:06:55.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.432+0000 7f80db400640 1 -- 172.21.15.45:0/925826051 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 24) ==== 138822+0+0 (secure 0 0 0) 0x7f80d40046c0 con 0x7f80e40743b0 2024-04-16T17:06:55.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.433+0000 7f80db400640 1 --2- 172.21.15.45:0/925826051 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f80c80a3da0 0x7f80c80a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:55.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.433+0000 7f80db400640 1 -- 172.21.15.45:0/925826051 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f80d40d0f60 con 0x7f80e40743b0 2024-04-16T17:06:55.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.433+0000 7f80d9400640 1 --2- 172.21.15.45:0/925826051 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f80c80a3da0 0x7f80c80a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:55.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.434+0000 7f80d9400640 1 --2- 172.21.15.45:0/925826051 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f80c80a3da0 0x7f80c80a6260 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f80cc02c210 tx=0x7f80cc02e040 comp rx=0 tx=0).ready entity=mgr.25411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:06:55.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.438+0000 7f80db400640 1 -- 172.21.15.45:0/925826051 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f80d409d120 con 0x7f80e40743b0 2024-04-16T17:06:55.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.672+0000 7f80e3200640 1 -- 172.21.15.45:0/925826051 --> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}) -- 0x7f80e4062440 con 0x7f80c80a3da0 2024-04-16T17:06:55.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.783+0000 7f80db400640 1 -- 172.21.15.45:0/925826051 <== mgr.25411 v2:172.21.15.45:6800/751128195 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+26 (secure 0 0 0) 0x7f80e4062440 con 0x7f80c80a3da0 2024-04-16T17:06:55.784 INFO:teuthology.orchestra.run.smithi045.stdout:Retention added to path / 2024-04-16T17:06:55.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.786+0000 7f80e3200640 1 -- 172.21.15.45:0/925826051 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f80c80a3da0 msgr2=0x7f80c80a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:55.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.786+0000 7f80e3200640 1 --2- 172.21.15.45:0/925826051 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f80c80a3da0 0x7f80c80a6260 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f80cc02c210 tx=0x7f80cc02e040 comp rx=0 tx=0).stop 2024-04-16T17:06:55.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.786+0000 7f80e3200640 1 -- 172.21.15.45:0/925826051 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f80e40743b0 msgr2=0x7f80e4101e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:55.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.786+0000 7f80e3200640 1 --2- 172.21.15.45:0/925826051 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f80e40743b0 0x7f80e4101e00 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f80d400e9f0 tx=0x7f80d400eeb0 comp rx=0 tx=0).stop 2024-04-16T17:06:55.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.787+0000 7f80e3200640 1 -- 172.21.15.45:0/925826051 shutdown_connections 2024-04-16T17:06:55.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.787+0000 7f80e3200640 1 --2- 172.21.15.45:0/925826051 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f80c80a3da0 0x7f80c80a6260 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:55.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.787+0000 7f80e3200640 1 --2- 172.21.15.45:0/925826051 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f80e410e6e0 0x7f80e4102880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:55.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.787+0000 7f80e3200640 1 --2- 172.21.15.45:0/925826051 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f80e4074cd0 0x7f80e4102340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:55.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.787+0000 7f80e3200640 1 --2- 172.21.15.45:0/925826051 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f80e40743b0 0x7f80e4101e00 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:55.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.787+0000 7f80e3200640 1 -- 172.21.15.45:0/925826051 >> 172.21.15.45:0/925826051 conn(0x7f80e40fd950 msgr2=0x7f80e40fe660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:06:55.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.787+0000 7f80e3200640 1 -- 172.21.15.45:0/925826051 shutdown_connections 2024-04-16T17:06:55.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:55.787+0000 7f80e3200640 1 -- 172.21.15.45:0/925826051 wait complete. 2024-04-16T17:06:55.803 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:06:55 smithi045 ceph-mon[31676]: scrub queued for path: / 2024-04-16T17:06:55.803 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:06:55 smithi045 ceph-mon[31676]: scrub summary: idle+waiting paths [/] 2024-04-16T17:06:55.803 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:06:55 smithi045 ceph-mon[31676]: scrub summary: active paths [/] 2024-04-16T17:06:55.803 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:06:55 smithi045 ceph-mon[31676]: scrub summary: idle+waiting paths [/] 2024-04-16T17:06:55.803 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:06:55 smithi045 ceph-mon[31676]: from='client.25609 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T17:06:55.803 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:06:55 smithi045 ceph-mon[31676]: pgmap v8: 129 pgs: 129 active+clean; 1.1 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 113 B/s wr, 17 op/s 2024-04-16T17:06:55.806 DEBUG:teuthology.orchestra.run.smithi045:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-04-16T17:06:56.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:56.005+0000 7f600da00640 1 -- 172.21.15.45:0/4243010828 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6008102090 msgr2=0x7f6008102470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:56.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:56.005+0000 7f600da00640 1 --2- 172.21.15.45:0/4243010828 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6008102090 0x7f6008102470 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f5ff8009900 tx=0x7f5ff802bc30 comp rx=0 tx=0).stop 2024-04-16T17:06:56.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:56.006+0000 7f600da00640 1 -- 172.21.15.45:0/4243010828 shutdown_connections 2024-04-16T17:06:56.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:56.006+0000 7f600da00640 1 --2- 172.21.15.45:0/4243010828 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6008107160 0x7f600810ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:56.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:56.006+0000 7f600da00640 1 --2- 172.21.15.45:0/4243010828 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6008102a40 0x7f6008106a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:56.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:56.006+0000 7f600da00640 1 --2- 172.21.15.45:0/4243010828 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6008102090 0x7f6008102470 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:56.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:56.006+0000 7f600da00640 1 -- 172.21.15.45:0/4243010828 >> 172.21.15.45:0/4243010828 conn(0x7f60080fd950 msgr2=0x7f60080ffd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:06:56.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:56.006+0000 7f600da00640 1 -- 172.21.15.45:0/4243010828 shutdown_connections 2024-04-16T17:06:56.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:56.006+0000 7f600da00640 1 -- 172.21.15.45:0/4243010828 wait complete. 2024-04-16T17:06:56.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:56.007+0000 7f600da00640 1 Processor -- start 2024-04-16T17:06:56.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:56.007+0000 7f600da00640 1 -- start start 2024-04-16T17:06:56.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:56.007+0000 7f600da00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6008102090 0x7f60081a05c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:56.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:56.007+0000 7f600da00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6008107160 0x7f60081a0b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:56.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:56.007+0000 7f600da00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f600819a7d0 0x7f600819ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:56.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:56.007+0000 7f600da00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f6008112270 con 0x7f600819a7d0 2024-04-16T17:06:56.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:56.007+0000 7f600da00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f60081120f0 con 0x7f6008107160 2024-04-16T17:06:56.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:56.007+0000 7f600da00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f60081123f0 con 0x7f6008102090 2024-04-16T17:06:56.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:56.008+0000 7f6007e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6008102090 0x7f60081a05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:56.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:56.008+0000 7f600d000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f600819a7d0 0x7f600819ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:56.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:56.008+0000 7f5fffe00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6008107160 0x7f60081a0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:56.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:56.008+0000 7f6007e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6008102090 0x7f60081a05c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:35950/0 (socket says 172.21.15.45:35950) 2024-04-16T17:06:56.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:56.008+0000 7f6007e00640 1 -- 172.21.15.45:0/1429052046 learned_addr learned my addr 172.21.15.45:0/1429052046 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:06:56.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:56.008+0000 7f5fffe00640 1 -- 172.21.15.45:0/1429052046 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6008102090 msgr2=0x7f60081a05c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:56.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:56.008+0000 7f5fffe00640 1 --2- 172.21.15.45:0/1429052046 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6008102090 0x7f60081a05c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:56.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:56.008+0000 7f5fffe00640 1 -- 172.21.15.45:0/1429052046 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f600819a7d0 msgr2=0x7f600819ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:56.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:56.008+0000 7f5fffe00640 1 --2- 172.21.15.45:0/1429052046 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f600819a7d0 0x7f600819ac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:56.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:56.008+0000 7f5fffe00640 1 -- 172.21.15.45:0/1429052046 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f600819b360 con 0x7f6008107160 2024-04-16T17:06:56.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:56.008+0000 7f600d000640 1 --2- 172.21.15.45:0/1429052046 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f600819a7d0 0x7f600819ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:06:56.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:56.009+0000 7f5fffe00640 1 --2- 172.21.15.45:0/1429052046 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6008107160 0x7f60081a0b00 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f5ff000e9c0 tx=0x7f5ff000ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:06:56.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:56.009+0000 7f6005600640 1 -- 172.21.15.45:0/1429052046 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ff000cd80 con 0x7f6008107160 2024-04-16T17:06:56.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:56.009+0000 7f6005600640 1 -- 172.21.15.45:0/1429052046 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5ff00035f0 con 0x7f6008107160 2024-04-16T17:06:56.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:56.009+0000 7f600da00640 1 -- 172.21.15.45:0/1429052046 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f600819b650 con 0x7f6008107160 2024-04-16T17:06:56.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:56.009+0000 7f6005600640 1 -- 172.21.15.45:0/1429052046 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ff0010670 con 0x7f6008107160 2024-04-16T17:06:56.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:56.009+0000 7f600da00640 1 -- 172.21.15.45:0/1429052046 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60081a7790 con 0x7f6008107160 2024-04-16T17:06:56.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:56.010+0000 7f600da00640 1 -- 172.21.15.45:0/1429052046 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6008103560 con 0x7f6008107160 2024-04-16T17:06:56.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:56.012+0000 7f6005600640 1 -- 172.21.15.45:0/1429052046 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 24) ==== 138822+0+0 (secure 0 0 0) 0x7f5ff00037a0 con 0x7f6008107160 2024-04-16T17:06:56.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:56.013+0000 7f6005600640 1 --2- 172.21.15.45:0/1429052046 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f5fec0a3da0 0x7f5fec0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:56.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:56.013+0000 7f6005600640 1 -- 172.21.15.45:0/1429052046 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f5ff00d0ff0 con 0x7f6008107160 2024-04-16T17:06:56.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:56.014+0000 7f6007e00640 1 --2- 172.21.15.45:0/1429052046 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f5fec0a3da0 0x7f5fec0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:56.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:56.014+0000 7f6007e00640 1 --2- 172.21.15.45:0/1429052046 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f5fec0a3da0 0x7f5fec0a6260 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f5ff80096c0 tx=0x7f5ff802df90 comp rx=0 tx=0).ready entity=mgr.25411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:06:56.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:56.017+0000 7f6005600640 1 -- 172.21.15.45:0/1429052046 <== mon.2 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f600819b650 con 0x7f6008107160 2024-04-16T17:06:56.055 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:06:55 smithi117 ceph-mon[36730]: scrub queued for path: / 2024-04-16T17:06:56.055 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:06:55 smithi117 ceph-mon[36730]: scrub summary: idle+waiting paths [/] 2024-04-16T17:06:56.055 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:06:55 smithi117 ceph-mon[36730]: scrub summary: active paths [/] 2024-04-16T17:06:56.055 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:06:55 smithi117 ceph-mon[36730]: scrub summary: idle+waiting paths [/] 2024-04-16T17:06:56.055 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:06:55 smithi117 ceph-mon[36730]: from='client.25609 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T17:06:56.055 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:06:55 smithi117 ceph-mon[36730]: pgmap v8: 129 pgs: 129 active+clean; 1.1 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 113 B/s wr, 17 op/s 2024-04-16T17:06:56.178 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:06:55 smithi104 ceph-mon[36457]: scrub queued for path: / 2024-04-16T17:06:56.178 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:06:55 smithi104 ceph-mon[36457]: scrub summary: idle+waiting paths [/] 2024-04-16T17:06:56.178 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:06:55 smithi104 ceph-mon[36457]: scrub summary: active paths [/] 2024-04-16T17:06:56.178 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:06:55 smithi104 ceph-mon[36457]: scrub summary: idle+waiting paths [/] 2024-04-16T17:06:56.178 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:06:55 smithi104 ceph-mon[36457]: from='client.25609 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T17:06:56.178 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:06:55 smithi104 ceph-mon[36457]: pgmap v8: 129 pgs: 129 active+clean; 1.1 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 113 B/s wr, 17 op/s 2024-04-16T17:06:56.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:56.252+0000 7f600da00640 1 -- 172.21.15.45:0/1429052046 --> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7f6008102b20 con 0x7f5fec0a3da0 2024-04-16T17:06:56.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:56.268+0000 7f6005600640 1 -- 172.21.15.45:0/1429052046 <== mgr.25411 v2:172.21.15.45:6800/751128195 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+298 (secure 0 0 0) 0x7f6008102b20 con 0x7f5fec0a3da0 2024-04-16T17:06:56.268 INFO:teuthology.orchestra.run.smithi045.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-04-16T00:00:00", "created": "2024-04-16T17:06:55", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true} 2024-04-16T17:06:56.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:56.270+0000 7f600da00640 1 -- 172.21.15.45:0/1429052046 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f5fec0a3da0 msgr2=0x7f5fec0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:56.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:56.270+0000 7f600da00640 1 --2- 172.21.15.45:0/1429052046 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f5fec0a3da0 0x7f5fec0a6260 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f5ff80096c0 tx=0x7f5ff802df90 comp rx=0 tx=0).stop 2024-04-16T17:06:56.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:56.270+0000 7f600da00640 1 -- 172.21.15.45:0/1429052046 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6008107160 msgr2=0x7f60081a0b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:56.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:56.270+0000 7f600da00640 1 --2- 172.21.15.45:0/1429052046 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6008107160 0x7f60081a0b00 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f5ff000e9c0 tx=0x7f5ff000ee80 comp rx=0 tx=0).stop 2024-04-16T17:06:56.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:56.271+0000 7f600da00640 1 -- 172.21.15.45:0/1429052046 shutdown_connections 2024-04-16T17:06:56.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:56.271+0000 7f600da00640 1 --2- 172.21.15.45:0/1429052046 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f5fec0a3da0 0x7f5fec0a6260 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:56.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:56.271+0000 7f600da00640 1 --2- 172.21.15.45:0/1429052046 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f600819a7d0 0x7f600819ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:56.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:56.271+0000 7f600da00640 1 --2- 172.21.15.45:0/1429052046 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6008107160 0x7f60081a0b00 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:56.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:56.271+0000 7f600da00640 1 --2- 172.21.15.45:0/1429052046 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6008102090 0x7f60081a05c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:56.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:56.271+0000 7f600da00640 1 -- 172.21.15.45:0/1429052046 >> 172.21.15.45:0/1429052046 conn(0x7f60080fd950 msgr2=0x7f60080fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:06:56.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:56.271+0000 7f600da00640 1 -- 172.21.15.45:0/1429052046 shutdown_connections 2024-04-16T17:06:56.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:56.271+0000 7f600da00640 1 -- 172.21.15.45:0/1429052046 wait complete. 2024-04-16T17:06:56.290 DEBUG:teuthology.orchestra.run.smithi045:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-04-16T17:06:56.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:56.498+0000 7f24ce400640 1 -- 172.21.15.45:0/2100570747 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f24c81038e0 msgr2=0x7f24c810bdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:56.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:56.498+0000 7f24ce400640 1 --2- 172.21.15.45:0/2100570747 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f24c81038e0 0x7f24c810bdc0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f24b4009900 tx=0x7f24b402bcc0 comp rx=0 tx=0).stop 2024-04-16T17:06:56.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:56.498+0000 7f24ce400640 1 -- 172.21.15.45:0/2100570747 shutdown_connections 2024-04-16T17:06:56.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:56.498+0000 7f24ce400640 1 --2- 172.21.15.45:0/2100570747 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f24c810c3e0 0x7f24c810e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:56.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:56.498+0000 7f24ce400640 1 --2- 172.21.15.45:0/2100570747 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f24c81038e0 0x7f24c810bdc0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:56.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:56.498+0000 7f24ce400640 1 --2- 172.21.15.45:0/2100570747 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f24c8102fc0 0x7f24c81033a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:56.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:56.498+0000 7f24ce400640 1 -- 172.21.15.45:0/2100570747 >> 172.21.15.45:0/2100570747 conn(0x7f24c80fc820 msgr2=0x7f24c80fec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:06:56.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:56.498+0000 7f24ce400640 1 -- 172.21.15.45:0/2100570747 shutdown_connections 2024-04-16T17:06:56.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:56.498+0000 7f24ce400640 1 -- 172.21.15.45:0/2100570747 wait complete. 2024-04-16T17:06:56.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:56.499+0000 7f24ce400640 1 Processor -- start 2024-04-16T17:06:56.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:56.499+0000 7f24ce400640 1 -- start start 2024-04-16T17:06:56.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:56.499+0000 7f24ce400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f24c8102fc0 0x7f24c81a0670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:56.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:56.499+0000 7f24ce400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f24c81038e0 0x7f24c81a0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:56.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:56.499+0000 7f24ce400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f24c810c3e0 0x7f24c819a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:56.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:56.499+0000 7f24ce400640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f24c8112210 con 0x7f24c810c3e0 2024-04-16T17:06:56.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:56.499+0000 7f24ce400640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f24c8112090 con 0x7f24c81038e0 2024-04-16T17:06:56.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:56.499+0000 7f24ce400640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f24c8112390 con 0x7f24c8102fc0 2024-04-16T17:06:56.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:56.500+0000 7f24cd000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f24c8102fc0 0x7f24c81a0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:56.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:56.500+0000 7f24c7e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f24c81038e0 0x7f24c81a0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:56.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:56.500+0000 7f24c7e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f24c81038e0 0x7f24c81a0bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:33644/0 (socket says 172.21.15.45:33644) 2024-04-16T17:06:56.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:56.500+0000 7f24cd000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f24c8102fc0 0x7f24c81a0670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:35978/0 (socket says 172.21.15.45:35978) 2024-04-16T17:06:56.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:56.500+0000 7f24c7e00640 1 -- 172.21.15.45:0/1602703114 learned_addr learned my addr 172.21.15.45:0/1602703114 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:06:56.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:56.500+0000 7f24cda00640 1 --2- 172.21.15.45:0/1602703114 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f24c810c3e0 0x7f24c819a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:56.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:56.500+0000 7f24c7e00640 1 -- 172.21.15.45:0/1602703114 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f24c8102fc0 msgr2=0x7f24c81a0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:56.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:56.500+0000 7f24c7e00640 1 --2- 172.21.15.45:0/1602703114 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f24c8102fc0 0x7f24c81a0670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:56.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:56.500+0000 7f24c7e00640 1 -- 172.21.15.45:0/1602703114 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f24c810c3e0 msgr2=0x7f24c819a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:56.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:56.500+0000 7f24c7e00640 1 --2- 172.21.15.45:0/1602703114 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f24c810c3e0 0x7f24c819a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:56.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:56.500+0000 7f24c7e00640 1 -- 172.21.15.45:0/1602703114 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24c819aff0 con 0x7f24c81038e0 2024-04-16T17:06:56.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:56.501+0000 7f24c7e00640 1 --2- 172.21.15.45:0/1602703114 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f24c81038e0 0x7f24c81a0bb0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f24b4000c00 tx=0x7f24b40042f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:06:56.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:56.501+0000 7f24c5600640 1 -- 172.21.15.45:0/1602703114 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24b4042020 con 0x7f24c81038e0 2024-04-16T17:06:56.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:56.501+0000 7f24ce400640 1 -- 172.21.15.45:0/1602703114 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24c819b280 con 0x7f24c81038e0 2024-04-16T17:06:56.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:56.501+0000 7f24c5600640 1 -- 172.21.15.45:0/1602703114 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f24b4005000 con 0x7f24c81038e0 2024-04-16T17:06:56.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:56.501+0000 7f24c5600640 1 -- 172.21.15.45:0/1602703114 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24b4003a10 con 0x7f24c81038e0 2024-04-16T17:06:56.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:56.501+0000 7f24ce400640 1 -- 172.21.15.45:0/1602703114 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24c81a75f0 con 0x7f24c81038e0 2024-04-16T17:06:56.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:56.502+0000 7f24ce400640 1 -- 172.21.15.45:0/1602703114 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f24c819b5f0 con 0x7f24c81038e0 2024-04-16T17:06:56.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:56.504+0000 7f24c5600640 1 -- 172.21.15.45:0/1602703114 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 24) ==== 138822+0+0 (secure 0 0 0) 0x7f24b4045020 con 0x7f24c81038e0 2024-04-16T17:06:56.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:56.506+0000 7f24c5600640 1 --2- 172.21.15.45:0/1602703114 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f24b00a3da0 0x7f24b00a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:06:56.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:56.506+0000 7f24c5600640 1 -- 172.21.15.45:0/1602703114 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f24b40f2ae0 con 0x7f24c81038e0 2024-04-16T17:06:56.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:56.507+0000 7f24cd000640 1 --2- 172.21.15.45:0/1602703114 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f24b00a3da0 0x7f24b00a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:06:56.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:56.507+0000 7f24cd000640 1 --2- 172.21.15.45:0/1602703114 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f24b00a3da0 0x7f24b00a6260 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f24bc005fd0 tx=0x7f24bc009500 comp rx=0 tx=0).ready entity=mgr.25411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:06:56.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:56.508+0000 7f24c5600640 1 -- 172.21.15.45:0/1602703114 <== mon.2 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f24b404c020 con 0x7f24c81038e0 2024-04-16T17:06:56.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:56.736+0000 7f24ce400640 1 -- 172.21.15.45:0/1602703114 --> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7f24c8102260 con 0x7f24b00a3da0 2024-04-16T17:06:56.748 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:56.748+0000 7f24c5600640 1 -- 172.21.15.45:0/1602703114 <== mgr.25411 v2:172.21.15.45:6800/751128195 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7f24c8102260 con 0x7f24b00a3da0 2024-04-16T17:06:56.748 INFO:teuthology.orchestra.run.smithi045.stdout:/ 1m 6m3h 2024-04-16T17:06:56.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:56.750+0000 7f24ce400640 1 -- 172.21.15.45:0/1602703114 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f24b00a3da0 msgr2=0x7f24b00a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:56.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:56.750+0000 7f24ce400640 1 --2- 172.21.15.45:0/1602703114 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f24b00a3da0 0x7f24b00a6260 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f24bc005fd0 tx=0x7f24bc009500 comp rx=0 tx=0).stop 2024-04-16T17:06:56.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:56.750+0000 7f24ce400640 1 -- 172.21.15.45:0/1602703114 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f24c81038e0 msgr2=0x7f24c81a0bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:06:56.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:56.750+0000 7f24ce400640 1 --2- 172.21.15.45:0/1602703114 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f24c81038e0 0x7f24c81a0bb0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f24b4000c00 tx=0x7f24b40042f0 comp rx=0 tx=0).stop 2024-04-16T17:06:56.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:56.750+0000 7f24ce400640 1 -- 172.21.15.45:0/1602703114 shutdown_connections 2024-04-16T17:06:56.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:56.750+0000 7f24ce400640 1 --2- 172.21.15.45:0/1602703114 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f24b00a3da0 0x7f24b00a6260 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:56.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:56.750+0000 7f24ce400640 1 --2- 172.21.15.45:0/1602703114 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f24c810c3e0 0x7f24c819a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:56.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:56.750+0000 7f24ce400640 1 --2- 172.21.15.45:0/1602703114 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f24c81038e0 0x7f24c81a0bb0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:56.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:56.750+0000 7f24ce400640 1 --2- 172.21.15.45:0/1602703114 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f24c8102fc0 0x7f24c81a0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:06:56.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:56.750+0000 7f24ce400640 1 -- 172.21.15.45:0/1602703114 >> 172.21.15.45:0/1602703114 conn(0x7f24c80fc820 msgr2=0x7f24c8100b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:06:56.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:56.750+0000 7f24ce400640 1 -- 172.21.15.45:0/1602703114 shutdown_connections 2024-04-16T17:06:56.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:06:56.750+0000 7f24ce400640 1 -- 172.21.15.45:0/1602703114 wait complete. 2024-04-16T17:06:56.768 DEBUG:teuthology.orchestra.run.smithi045:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > START_TIME' 2024-04-16T17:06:56.855 INFO:teuthology.run_tasks:Running task full_sequential_finally... 2024-04-16T17:06:56.866 INFO:teuthology.run_tasks:Running task background_exec... 2024-04-16T17:06:56.901 INFO:teuthology.task.background_exec:Running background command on role mon.a host ubuntu@smithi045.front.sepia.ceph.com 2024-04-16T17:06:56.901 DEBUG:teuthology.orchestra.run.smithi045:> sudo TESTDIR=/home/ubuntu/cephtest daemon-helper kill --kill-group bash -c 'while sleep 13; do ceph tell mds.cephfs:0 flush journal; done' 2024-04-16T17:06:56.908 INFO:teuthology.run_tasks:Running task workunit... 2024-04-16T17:06:56.921 INFO:tasks.workunit:Pulling workunits from ref b137e4a675398e295567f9652210865a4d573fac 2024-04-16T17:06:56.921 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-04-16T17:06:56.921 INFO:tasks.workunit:timeout=3h 2024-04-16T17:06:56.921 INFO:tasks.workunit:cleanup=True 2024-04-16T17:06:56.921 DEBUG:teuthology.orchestra.run.smithi045:> stat -- /home/ubuntu/cephtest/mnt.0 2024-04-16T17:06:56.966 INFO:teuthology.orchestra.run.smithi045.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-04-16T17:06:56.966 INFO:teuthology.orchestra.run.smithi045.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-04-16T17:06:56.966 INFO:teuthology.orchestra.run.smithi045.stdout:Device: 77h/119d Inode: 1099511628283 Links: 2 2024-04-16T17:06:56.966 INFO:teuthology.orchestra.run.smithi045.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-16T17:06:56.966 INFO:teuthology.orchestra.run.smithi045.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-04-16T17:06:56.966 INFO:teuthology.orchestra.run.smithi045.stdout:Access: 2024-04-16 17:03:55.438823102 +0000 2024-04-16T17:06:56.967 INFO:teuthology.orchestra.run.smithi045.stdout:Modify: 2024-04-16 17:03:55.438823102 +0000 2024-04-16T17:06:56.967 INFO:teuthology.orchestra.run.smithi045.stdout:Change: 2024-04-16 17:06:43.415323111 +0000 2024-04-16T17:06:56.967 INFO:teuthology.orchestra.run.smithi045.stdout: Birth: 2024-04-16 17:03:55.438823102 +0000 2024-04-16T17:06:56.967 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-04-16T17:06:56.967 DEBUG:teuthology.orchestra.run.smithi045:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-04-16T17:06:57.055 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:06:56 smithi117 ceph-mon[36730]: from='client.15522 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T17:06:57.055 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:06:56 smithi117 ceph-mon[36730]: from='client.25658 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T17:06:57.079 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:06:56 smithi045 ceph-mon[31676]: from='client.15522 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T17:06:57.080 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:06:56 smithi045 ceph-mon[31676]: from='client.25658 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T17:06:57.101 DEBUG:teuthology.orchestra.run.smithi045:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://github.com/mchangir/ceph.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout b137e4a675398e295567f9652210865a4d573fac 2024-04-16T17:06:57.149 INFO:tasks.workunit.client.0.smithi045.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-04-16T17:06:57.177 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:06:56 smithi104 ceph-mon[36457]: from='client.15522 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T17:06:57.178 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:06:56 smithi104 ceph-mon[36457]: from='client.25658 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T17:06:58.055 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:06:57 smithi117 ceph-mon[36730]: from='client.25664 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T17:06:58.055 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:06:57 smithi117 ceph-mon[36730]: pgmap v9: 129 pgs: 129 active+clean; 1.3 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 83 KiB/s wr, 48 op/s 2024-04-16T17:06:58.177 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:06:57 smithi104 ceph-mon[36457]: from='client.25664 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T17:06:58.178 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:06:57 smithi104 ceph-mon[36457]: pgmap v9: 129 pgs: 129 active+clean; 1.3 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 83 KiB/s wr, 48 op/s 2024-04-16T17:06:58.227 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:06:57 smithi045 ceph-mon[31676]: from='client.25664 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T17:06:58.228 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:06:57 smithi045 ceph-mon[31676]: pgmap v9: 129 pgs: 129 active+clean; 1.3 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 83 KiB/s wr, 48 op/s 2024-04-16T17:07:00.477 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:07:00 smithi045 ceph-mon[31676]: pgmap v10: 129 pgs: 129 active+clean; 1.3 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 76 KiB/s wr, 44 op/s 2024-04-16T17:07:00.555 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:07:00 smithi117 ceph-mon[36730]: pgmap v10: 129 pgs: 129 active+clean; 1.3 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 76 KiB/s wr, 44 op/s 2024-04-16T17:07:00.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:07:00 smithi104 ceph-mon[36457]: pgmap v10: 129 pgs: 129 active+clean; 1.3 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 76 KiB/s wr, 44 op/s 2024-04-16T17:07:01.228 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 16 17:07:00 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-x[31905]: 2024-04-16T17:07:00.806+0000 7f6c85800700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-16T17:07:01.228 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 16 17:07:00 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-x[31905]: 2024-04-16T17:07:00.806+0000 7f6c85800700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-16T17:07:01.228 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 16 17:07:00 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-x[31905]: 2024-04-16T17:07:00.806+0000 7f6c85800700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-16T17:07:01.228 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 16 17:07:00 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-x[31905]: 2024-04-16T17:07:00.806+0000 7f6c85800700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-16T17:07:01.228 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 16 17:07:00 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-x[31905]: 2024-04-16T17:07:00.806+0000 7f6c85800700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-16T17:07:02.477 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:07:02 smithi045 ceph-mon[31676]: pgmap v11: 129 pgs: 129 active+clean; 1.3 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 76 KiB/s wr, 45 op/s 2024-04-16T17:07:02.477 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:07:02 smithi045 ceph-mon[31676]: mgrmap e25: x(active, since 16s), standbys: z, y 2024-04-16T17:07:02.555 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:07:02 smithi117 ceph-mon[36730]: pgmap v11: 129 pgs: 129 active+clean; 1.3 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 76 KiB/s wr, 45 op/s 2024-04-16T17:07:02.555 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:07:02 smithi117 ceph-mon[36730]: mgrmap e25: x(active, since 16s), standbys: z, y 2024-04-16T17:07:02.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:07:02 smithi104 ceph-mon[36457]: pgmap v11: 129 pgs: 129 active+clean; 1.3 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 76 KiB/s wr, 45 op/s 2024-04-16T17:07:02.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:07:02 smithi104 ceph-mon[36457]: mgrmap e25: x(active, since 16s), standbys: z, y 2024-04-16T17:07:04.477 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:07:04 smithi045 ceph-mon[31676]: pgmap v12: 129 pgs: 129 active+clean; 1.3 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 99 KiB/s wr, 64 op/s 2024-04-16T17:07:04.555 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:07:04 smithi117 ceph-mon[36730]: pgmap v12: 129 pgs: 129 active+clean; 1.3 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 99 KiB/s wr, 64 op/s 2024-04-16T17:07:04.677 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:07:04 smithi104 ceph-mon[36457]: pgmap v12: 129 pgs: 129 active+clean; 1.3 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 99 KiB/s wr, 64 op/s 2024-04-16T17:07:06.477 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:07:06 smithi045 ceph-mon[31676]: pgmap v13: 129 pgs: 129 active+clean; 1.3 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 99 KiB/s wr, 52 op/s 2024-04-16T17:07:06.555 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:07:06 smithi117 ceph-mon[36730]: pgmap v13: 129 pgs: 129 active+clean; 1.3 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 99 KiB/s wr, 52 op/s 2024-04-16T17:07:06.677 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:07:06 smithi104 ceph-mon[36457]: pgmap v13: 129 pgs: 129 active+clean; 1.3 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 99 KiB/s wr, 52 op/s 2024-04-16T17:07:08.478 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:07:08 smithi045 ceph-mon[31676]: pgmap v14: 129 pgs: 129 active+clean; 1.3 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 100 KiB/s wr, 55 op/s 2024-04-16T17:07:08.555 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:07:08 smithi117 ceph-mon[36730]: pgmap v14: 129 pgs: 129 active+clean; 1.3 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 100 KiB/s wr, 55 op/s 2024-04-16T17:07:08.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:07:08 smithi104 ceph-mon[36457]: pgmap v14: 129 pgs: 129 active+clean; 1.3 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 100 KiB/s wr, 55 op/s 2024-04-16T17:07:09.477 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:07:09 smithi045 ceph-mon[31676]: scrub summary: idle 2024-04-16T17:07:09.555 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:07:09 smithi117 ceph-mon[36730]: scrub summary: idle 2024-04-16T17:07:09.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:07:09 smithi104 ceph-mon[36457]: scrub summary: idle 2024-04-16T17:07:10.175 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:10.173+0000 7f4d4ee00640 1 -- 172.21.15.45:0/1067015514 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4d48109520 msgr2=0x7f4d481110f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:10.175 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:10.173+0000 7f4d4ee00640 1 --2- 172.21.15.45:0/1067015514 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4d48109520 0x7f4d481110f0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f4d40009be0 tx=0x7f4d4002bef0 comp rx=0 tx=0).stop 2024-04-16T17:07:10.175 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:10.173+0000 7f4d4ee00640 1 -- 172.21.15.45:0/1067015514 shutdown_connections 2024-04-16T17:07:10.175 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:10.173+0000 7f4d4ee00640 1 --2- 172.21.15.45:0/1067015514 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4d48104e00 0x7f4d48108e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:10.175 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:10.173+0000 7f4d4ee00640 1 --2- 172.21.15.45:0/1067015514 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4d48109520 0x7f4d481110f0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:10.175 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:10.173+0000 7f4d4ee00640 1 --2- 172.21.15.45:0/1067015514 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4d48104450 0x7f4d48104830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:10.175 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:10.173+0000 7f4d4ee00640 1 -- 172.21.15.45:0/1067015514 >> 172.21.15.45:0/1067015514 conn(0x7f4d480ffcf0 msgr2=0x7f4d48102110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:07:10.175 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:10.173+0000 7f4d4ee00640 1 -- 172.21.15.45:0/1067015514 shutdown_connections 2024-04-16T17:07:10.176 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:10.174+0000 7f4d4ee00640 1 -- 172.21.15.45:0/1067015514 wait complete. 2024-04-16T17:07:10.176 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:10.175+0000 7f4d4ee00640 1 Processor -- start 2024-04-16T17:07:10.176 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:10.175+0000 7f4d4ee00640 1 -- start start 2024-04-16T17:07:10.178 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:10.176+0000 7f4d4ee00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4d48104450 0x7f4d481a0670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:10.178 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:10.176+0000 7f4d4ee00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4d48109520 0x7f4d481a0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:10.178 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:10.176+0000 7f4d4ee00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4d4819a7e0 0x7f4d4819ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:10.178 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:10.176+0000 7f4d4ee00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f4d481145c0 con 0x7f4d48104450 2024-04-16T17:07:10.178 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:10.176+0000 7f4d4ee00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f4d48114440 con 0x7f4d4819a7e0 2024-04-16T17:07:10.178 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:10.176+0000 7f4d4ee00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f4d48114740 con 0x7f4d48109520 2024-04-16T17:07:10.178 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:10.176+0000 7f4d47e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4d48104450 0x7f4d481a0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:10.178 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:10.176+0000 7f4d47e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4d48104450 0x7f4d481a0670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53140/0 (socket says 172.21.15.45:53140) 2024-04-16T17:07:10.178 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:10.176+0000 7f4d47e00640 1 -- 172.21.15.45:0/3719137481 learned_addr learned my addr 172.21.15.45:0/3719137481 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:07:10.178 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:10.176+0000 7f4d4ce00640 1 --2- 172.21.15.45:0/3719137481 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4d4819a7e0 0x7f4d4819ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:10.178 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:10.176+0000 7f4d47400640 1 --2- 172.21.15.45:0/3719137481 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4d48109520 0x7f4d481a0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:10.178 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:10.177+0000 7f4d47e00640 1 -- 172.21.15.45:0/3719137481 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4d48109520 msgr2=0x7f4d481a0bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:10.178 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:10.177+0000 7f4d47e00640 1 --2- 172.21.15.45:0/3719137481 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4d48109520 0x7f4d481a0bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:10.179 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:10.177+0000 7f4d47e00640 1 -- 172.21.15.45:0/3719137481 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4d4819a7e0 msgr2=0x7f4d4819ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:10.179 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:10.177+0000 7f4d47e00640 1 --2- 172.21.15.45:0/3719137481 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4d4819a7e0 0x7f4d4819ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:10.179 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:10.177+0000 7f4d47e00640 1 -- 172.21.15.45:0/3719137481 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d4819b370 con 0x7f4d48104450 2024-04-16T17:07:10.179 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:10.177+0000 7f4d47e00640 1 --2- 172.21.15.45:0/3719137481 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4d48104450 0x7f4d481a0670 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f4d3800e9f0 tx=0x7f4d3800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:07:10.179 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:10.177+0000 7f4d44c00640 1 -- 172.21.15.45:0/3719137481 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d3800cd80 con 0x7f4d48104450 2024-04-16T17:07:10.180 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:10.178+0000 7f4d4ee00640 1 -- 172.21.15.45:0/3719137481 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d4819b660 con 0x7f4d48104450 2024-04-16T17:07:10.180 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:10.178+0000 7f4d44c00640 1 -- 172.21.15.45:0/3719137481 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d38010910 con 0x7f4d48104450 2024-04-16T17:07:10.180 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:10.178+0000 7f4d44c00640 1 -- 172.21.15.45:0/3719137481 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d38018990 con 0x7f4d48104450 2024-04-16T17:07:10.180 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:10.178+0000 7f4d4ee00640 1 -- 172.21.15.45:0/3719137481 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d481a7840 con 0x7f4d48104450 2024-04-16T17:07:10.183 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:10.182+0000 7f4d51c37740 1 Processor -- start 2024-04-16T17:07:10.183 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:10.182+0000 7f4d51c37740 1 -- start start 2024-04-16T17:07:10.183 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:10.182+0000 7f4d51c37740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f578796610 0x55f578795360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:10.183 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:10.182+0000 7f4d51c37740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f5787958a0 0x55f578795d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:10.184 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:10.182+0000 7f4d51c37740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f57879ee20 0x55f5787a1210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:10.184 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:10.182+0000 7f4d51c37740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55f5786fa780 con 0x55f5787958a0 2024-04-16T17:07:10.184 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:10.182+0000 7f4d51c37740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55f5786a0f40 con 0x55f57879ee20 2024-04-16T17:07:10.184 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:10.182+0000 7f4d51c37740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55f5786f6740 con 0x55f578796610 2024-04-16T17:07:10.184 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:10.183+0000 7f4d4ce00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f578796610 0x55f578795360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:10.184 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:10.183+0000 7f4d4ce00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f578796610 0x55f578795360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:32818/0 (socket says 172.21.15.45:32818) 2024-04-16T17:07:10.185 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:10.183+0000 7f4d4ce00640 1 -- 172.21.15.45:0/9696850 learned_addr learned my addr 172.21.15.45:0/9696850 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:07:10.185 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:10.183+0000 7f4d47400640 1 --2- 172.21.15.45:0/9696850 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f57879ee20 0x55f5787a1210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:10.185 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:10.183+0000 7f4d4ce00640 1 -- 172.21.15.45:0/9696850 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f57879ee20 msgr2=0x55f5787a1210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:07:10.185 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:10.183+0000 7f4d4ce00640 1 --2- 172.21.15.45:0/9696850 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f57879ee20 0x55f5787a1210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:10.185 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:10.183+0000 7f4d4ce00640 1 -- 172.21.15.45:0/9696850 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f5787958a0 msgr2=0x55f578795d20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-16T17:07:10.185 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:10.183+0000 7f4d4ce00640 1 --2- 172.21.15.45:0/9696850 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f5787958a0 0x55f578795d20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:10.185 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:10.183+0000 7f4d4ce00640 1 -- 172.21.15.45:0/9696850 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f57840d210 con 0x55f578796610 2024-04-16T17:07:10.186 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:10.184+0000 7f4d4ce00640 1 --2- 172.21.15.45:0/9696850 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f578796610 0x55f578795360 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f4d40009be0 tx=0x7f4d40005c60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bd0dba45ac4c7959 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:07:10.186 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:10.184+0000 7f4d3d600640 1 -- 172.21.15.45:0/9696850 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d40043030 con 0x55f578796610 2024-04-16T17:07:10.186 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:10.184+0000 7f4d3d600640 1 -- 172.21.15.45:0/9696850 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d40033c50 con 0x55f578796610 2024-04-16T17:07:10.186 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:10.184+0000 7f4d3d600640 1 -- 172.21.15.45:0/9696850 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d4002c890 con 0x55f578796610 2024-04-16T17:07:10.186 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:10.185+0000 7f4d44c00640 1 -- 172.21.15.45:0/3719137481 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) ==== 138879+0+0 (secure 0 0 0) 0x7f4d38010ac0 con 0x7f4d48104450 2024-04-16T17:07:10.186 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:10.185+0000 7f4d51c37740 1 -- 172.21.15.45:0/9696850 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f578796610 msgr2=0x55f578795360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:10.186 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:10.185+0000 7f4d51c37740 1 --2- 172.21.15.45:0/9696850 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f578796610 0x55f578795360 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f4d40009be0 tx=0x7f4d40005c60 comp rx=0 tx=0).stop 2024-04-16T17:07:10.186 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:10.185+0000 7f4d51c37740 1 -- 172.21.15.45:0/9696850 shutdown_connections 2024-04-16T17:07:10.187 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:10.185+0000 7f4d51c37740 1 --2- 172.21.15.45:0/9696850 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f57879ee20 0x55f5787a1210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:10.187 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:10.185+0000 7f4d51c37740 1 --2- 172.21.15.45:0/9696850 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f5787958a0 0x55f578795d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:10.187 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:10.185+0000 7f4d51c37740 1 --2- 172.21.15.45:0/9696850 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f578796610 0x55f578795360 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:10.187 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:10.185+0000 7f4d51c37740 1 -- 172.21.15.45:0/9696850 >> 172.21.15.45:0/9696850 conn(0x55f5787305d0 msgr2=0x55f578790590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:07:10.187 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:10.185+0000 7f4d51c37740 1 -- 172.21.15.45:0/9696850 shutdown_connections 2024-04-16T17:07:10.187 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:10.185+0000 7f4d51c37740 1 -- 172.21.15.45:0/9696850 wait complete. 2024-04-16T17:07:10.190 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:10.185+0000 7f4d51c37740 1 -- start start 2024-04-16T17:07:10.190 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:10.186+0000 7f4d51c37740 1 Processor -- start 2024-04-16T17:07:10.190 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:10.186+0000 7f4d44c00640 1 --2- 172.21.15.45:0/3719137481 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f4d2c0a3df0 0x7f4d2c0a62b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:10.190 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:10.186+0000 7f4d51c37740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f5787958a0 0x55f57879e870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:10.190 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:10.186+0000 7f4d51c37740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f578796610 0x55f57879bc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:10.190 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:10.186+0000 7f4d51c37740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f57879ee20 0x55f57879c340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:10.190 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:10.186+0000 7f4d51c37740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55f5787a1af0 con 0x55f578796610 2024-04-16T17:07:10.191 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:10.186+0000 7f4d51c37740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55f5786b4390 con 0x55f5787958a0 2024-04-16T17:07:10.191 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:10.186+0000 7f4d51c37740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55f5787a1c70 con 0x55f57879ee20 2024-04-16T17:07:10.191 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:10.186+0000 7f4d44c00640 1 -- 172.21.15.45:0/3719137481 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f4d380d1210 con 0x7f4d48104450 2024-04-16T17:07:10.191 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:10.186+0000 7f4d51c37740 10 client.0 _ll_register_callbacks cb 0x55f57869df60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:07:10.191 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:10.187+0000 7f4d47e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f5787958a0 0x55f57879e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:10.191 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:10.187+0000 7f4d4ce00640 1 --2- 172.21.15.45:0/3719137481 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f4d2c0a3df0 0x7f4d2c0a62b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:10.191 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:10.187+0000 7f4d47e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f5787958a0 0x55f57879e870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:48614/0 (socket says 172.21.15.45:48614) 2024-04-16T17:07:10.191 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:10.187+0000 7f4d47e00640 1 -- 172.21.15.45:0/4203304680 learned_addr learned my addr 172.21.15.45:0/4203304680 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:07:10.191 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:10.187+0000 7f4d47400640 1 --2- 172.21.15.45:0/4203304680 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f578796610 0x55f57879bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:10.191 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:10.187+0000 7f4d4ce00640 1 --2- 172.21.15.45:0/4203304680 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f57879ee20 0x55f57879c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:10.191 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:10.187+0000 7f4d47e00640 1 -- 172.21.15.45:0/4203304680 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f57879ee20 msgr2=0x55f57879c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:07:10.191 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:10.187+0000 7f4d47e00640 1 --2- 172.21.15.45:0/4203304680 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f57879ee20 0x55f57879c340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:10.191 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:10.187+0000 7f4d47e00640 1 -- 172.21.15.45:0/4203304680 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f578796610 msgr2=0x55f57879bc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:07:10.191 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:10.187+0000 7f4d47e00640 1 --2- 172.21.15.45:0/4203304680 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f578796610 0x55f57879bc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:10.191 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:10.187+0000 7f4d47e00640 1 -- 172.21.15.45:0/4203304680 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f5784d9620 con 0x55f5787958a0 2024-04-16T17:07:10.192 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:10.187+0000 7f4d47400640 1 --2- 172.21.15.45:0/4203304680 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f578796610 0x55f57879bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:07:10.192 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:10.187+0000 7f4d47e00640 1 --2- 172.21.15.45:0/4203304680 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f5787958a0 0x55f57879e870 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f4d3800c6a0 tx=0x7f4d3809c610 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b42110ec7d3aafd2 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:07:10.192 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:10.187+0000 7f4d32000640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-04-16T17:07:10.192 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:10.188+0000 7f4d32000640 1 -- 172.21.15.45:0/4203304680 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d380a0fa0 con 0x55f5787958a0 2024-04-16T17:07:10.192 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:10.188+0000 7f4d32000640 1 -- 172.21.15.45:0/4203304680 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f4d380ce750 con 0x55f5787958a0 2024-04-16T17:07:10.192 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:10.188+0000 7f4d51c37740 1 -- 172.21.15.45:0/4203304680 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4d40033c50 con 0x55f5787958a0 2024-04-16T17:07:10.192 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:10.188+0000 7f4d4ce00640 1 --2- 172.21.15.45:0/3719137481 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f4d2c0a3df0 0x7f4d2c0a62b0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f4d40009b30 tx=0x7f4d40003db0 comp rx=0 tx=0).ready entity=mgr.25411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:07:10.192 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:10.188+0000 7f4d32000640 1 -- 172.21.15.45:0/4203304680 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f4d380a5860 con 0x55f5787958a0 2024-04-16T17:07:10.192 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:10.188+0000 7f4d51c37740 10 client.25679 fetch_fsmap learned FSMap version 25 2024-04-16T17:07:10.192 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:10.188+0000 7f4d51c37740 1 -- 172.21.15.45:0/4203304680 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55f5787b63c0 con 0x55f5787958a0 2024-04-16T17:07:10.192 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:10.188+0000 7f4d32000640 1 -- 172.21.15.45:0/4203304680 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d40033c50 con 0x55f5787958a0 2024-04-16T17:07:10.192 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:10.189+0000 7f4d32000640 1 -- 172.21.15.45:0/4203304680 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f4d3809c040 con 0x55f5787958a0 2024-04-16T17:07:10.192 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:10.189+0000 7f4d51c37740 10 client.25679 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T17:07:10.192 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:10.189+0000 7f4d51c37740 10 client.25679 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T17:07:10.192 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:10.189+0000 7f4d51c37740 20 client.25679 populate_metadata read hostname 'smithi045' 2024-04-16T17:07:10.192 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:10.189+0000 7f4d51c37740 1 --2- 172.21.15.45:0/4203304680 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55f5787b7830 0x55f5787b9c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:10.193 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:10.189+0000 7f4d51c37740 4 client.25679 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:07:10.193 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:10.189+0000 7f4d51c37740 1 -- 172.21.15.45:0/4203304680 --> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f5787081a0 con 0x55f5787b7830 2024-04-16T17:07:10.193 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:10.189+0000 7f4d47400640 1 --2- 172.21.15.45:0/4203304680 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55f5787b7830 0x55f5787b9c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:10.194 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:10.190+0000 7f4d47400640 1 --2- 172.21.15.45:0/4203304680 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55f5787b7830 0x55f5787b9c20 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=20d80537891d8d88 server_cookie=59a2908918f54524 in_seq=0 out_seq=0 2024-04-16T17:07:10.194 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:10.190+0000 7f4d32000640 10 client.25679 ms_handle_connect on v2:172.21.15.104:6838/3660070592 2024-04-16T17:07:10.194 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:10.191+0000 7f4d32000640 1 -- 172.21.15.45:0/4203304680 <== mds.0 v2:172.21.15.104:6838/3660070592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f5787081a0 con 0x55f5787b7830 2024-04-16T17:07:10.194 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:10.191+0000 7f4d32000640 10 client.25679 handle_command_reply: tid=0 2024-04-16T17:07:10.194 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:10.191+0000 7f4d51c37740 1 client.25679 shutdown 2024-04-16T17:07:10.194 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:10.191+0000 7f4d51c37740 1 -- 172.21.15.45:0/4203304680 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f5787958a0 msgr2=0x55f57879e870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:10.194 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:10.191+0000 7f4d51c37740 1 --2- 172.21.15.45:0/4203304680 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f5787958a0 0x55f57879e870 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f4d3800c6a0 tx=0x7f4d3809c610 comp rx=0 tx=0).stop 2024-04-16T17:07:10.194 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:10.191+0000 7f4d51c37740 1 -- 172.21.15.45:0/4203304680 shutdown_connections 2024-04-16T17:07:10.194 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:10.191+0000 7f4d51c37740 1 --2- 172.21.15.45:0/4203304680 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55f5787b7830 0x55f5787b9c20 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:10.194 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:10.191+0000 7f4d51c37740 1 --2- 172.21.15.45:0/4203304680 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f57879ee20 0x55f57879c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:10.194 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:10.191+0000 7f4d51c37740 1 --2- 172.21.15.45:0/4203304680 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f578796610 0x55f57879bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:10.194 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:10.191+0000 7f4d51c37740 1 --2- 172.21.15.45:0/4203304680 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f5787958a0 0x55f57879e870 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:10.195 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:10.191+0000 7f4d51c37740 1 -- 172.21.15.45:0/4203304680 >> 172.21.15.45:0/4203304680 conn(0x55f5787305d0 msgr2=0x55f5787922e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:07:10.195 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:10.191+0000 7f4d32000640 0 client.25679 ms_handle_reset on v2:172.21.15.104:6838/3660070592 2024-04-16T17:07:10.195 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:10.191+0000 7f4d51c37740 1 -- 172.21.15.45:0/4203304680 shutdown_connections 2024-04-16T17:07:10.195 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:10.191+0000 7f4d51c37740 1 -- 172.21.15.45:0/4203304680 wait complete. 2024-04-16T17:07:10.195 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:10.191+0000 7f4d51c37740 20 client.25679 trim_cache size 0 max 16384 2024-04-16T17:07:10.219 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:10.210+0000 7f4d51c37740 1 Processor -- start 2024-04-16T17:07:10.219 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:10.210+0000 7f4d51c37740 1 -- start start 2024-04-16T17:07:10.219 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:10.211+0000 7f4d51c37740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f5787958a0 0x55f5787aaa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:10.219 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:10.211+0000 7f4d51c37740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f578796610 0x55f57879b850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:10.219 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:10.211+0000 7f4d51c37740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f57879ee20 0x55f5787bf440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:10.219 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:10.211+0000 7f4d51c37740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f4d481afa90 con 0x55f57879ee20 2024-04-16T17:07:10.219 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:10.211+0000 7f4d51c37740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f4d481ad360 con 0x55f5787958a0 2024-04-16T17:07:10.219 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:10.211+0000 7f4d51c37740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f4d481ab1b0 con 0x55f578796610 2024-04-16T17:07:10.219 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:10.211+0000 7f4d47e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f5787958a0 0x55f5787aaa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:10.219 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:10.211+0000 7f4d47e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f5787958a0 0x55f5787aaa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:48618/0 (socket says 172.21.15.45:48618) 2024-04-16T17:07:10.219 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:10.211+0000 7f4d47e00640 1 -- 172.21.15.45:0/2035872859 learned_addr learned my addr 172.21.15.45:0/2035872859 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:07:10.220 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:10.211+0000 7f4d4ce00640 1 --2- 172.21.15.45:0/2035872859 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f57879ee20 0x55f5787bf440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:10.220 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:10.211+0000 7f4d47400640 1 --2- 172.21.15.45:0/2035872859 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f578796610 0x55f57879b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:10.220 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:10.211+0000 7f4d47e00640 1 -- 172.21.15.45:0/2035872859 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f578796610 msgr2=0x55f57879b850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:07:10.220 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:10.211+0000 7f4d47e00640 1 --2- 172.21.15.45:0/2035872859 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f578796610 0x55f57879b850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:10.220 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:10.211+0000 7f4d47e00640 1 -- 172.21.15.45:0/2035872859 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f57879ee20 msgr2=0x55f5787bf440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:07:10.220 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:10.211+0000 7f4d47e00640 1 --2- 172.21.15.45:0/2035872859 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f57879ee20 0x55f5787bf440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:10.220 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:10.211+0000 7f4d47e00640 1 -- 172.21.15.45:0/2035872859 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f5787907a0 con 0x55f5787958a0 2024-04-16T17:07:10.220 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:10.212+0000 7f4d47e00640 1 --2- 172.21.15.45:0/2035872859 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f5787958a0 0x55f5787aaa20 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f4d38085040 tx=0x7f4d38009770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bd55ea01b19779fb server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:07:10.220 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:10.212+0000 7f4d3cc00640 1 -- 172.21.15.45:0/2035872859 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d380ce3b0 con 0x55f5787958a0 2024-04-16T17:07:10.220 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:10.212+0000 7f4d3cc00640 1 -- 172.21.15.45:0/2035872859 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d3809c040 con 0x55f5787958a0 2024-04-16T17:07:10.220 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:10.212+0000 7f4d3cc00640 1 -- 172.21.15.45:0/2035872859 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d380873c0 con 0x55f5787958a0 2024-04-16T17:07:10.220 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:10.213+0000 7f4d51c37740 1 -- 172.21.15.45:0/2035872859 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f5787958a0 msgr2=0x55f5787aaa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:10.220 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:10.213+0000 7f4d51c37740 1 --2- 172.21.15.45:0/2035872859 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f5787958a0 0x55f5787aaa20 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f4d38085040 tx=0x7f4d38009770 comp rx=0 tx=0).stop 2024-04-16T17:07:10.220 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:10.213+0000 7f4d51c37740 1 -- 172.21.15.45:0/2035872859 shutdown_connections 2024-04-16T17:07:10.221 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:10.213+0000 7f4d51c37740 1 --2- 172.21.15.45:0/2035872859 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f578796610 0x55f57879b850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:10.221 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:10.213+0000 7f4d51c37740 1 --2- 172.21.15.45:0/2035872859 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f57879ee20 0x55f5787bf440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:10.221 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:10.213+0000 7f4d51c37740 1 --2- 172.21.15.45:0/2035872859 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f5787958a0 0x55f5787aaa20 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:10.221 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:10.213+0000 7f4d51c37740 1 -- 172.21.15.45:0/2035872859 >> 172.21.15.45:0/2035872859 conn(0x55f5787305d0 msgr2=0x55f57879b140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:07:10.221 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:10.213+0000 7f4d51c37740 1 -- 172.21.15.45:0/2035872859 shutdown_connections 2024-04-16T17:07:10.221 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:10.213+0000 7f4d51c37740 1 -- 172.21.15.45:0/2035872859 wait complete. 2024-04-16T17:07:10.221 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:10.213+0000 7f4d51c37740 1 -- start start 2024-04-16T17:07:10.221 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:10.214+0000 7f4d51c37740 1 Processor -- start 2024-04-16T17:07:10.221 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:10.214+0000 7f4d51c37740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f5787958a0 0x55f5787a2a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:10.221 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:10.214+0000 7f4d51c37740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f578796610 0x55f5787a2f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:10.221 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:10.214+0000 7f4d51c37740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f57879ee20 0x55f5787a3490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:10.221 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:10.214+0000 7f4d51c37740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f4d481ab820 con 0x55f57879ee20 2024-04-16T17:07:10.221 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:10.214+0000 7f4d51c37740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f4d481acc90 con 0x55f5787958a0 2024-04-16T17:07:10.221 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:10.214+0000 7f4d51c37740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f4d481abef0 con 0x55f578796610 2024-04-16T17:07:10.222 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:10.214+0000 7f4d51c37740 10 client.0 _ll_register_callbacks cb 0x7f4d3800e940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:07:10.222 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:10.214+0000 7f4d47e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f5787958a0 0x55f5787a2a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:10.222 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:10.214+0000 7f4d47400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f578796610 0x55f5787a2f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:10.222 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:10.214+0000 7f4d47e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f5787958a0 0x55f5787a2a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:48634/0 (socket says 172.21.15.45:48634) 2024-04-16T17:07:10.222 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:10.214+0000 7f4d47400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f578796610 0x55f5787a2f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:32856/0 (socket says 172.21.15.45:32856) 2024-04-16T17:07:10.222 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:10.214+0000 7f4d47e00640 1 -- 172.21.15.45:0/2135337355 learned_addr learned my addr 172.21.15.45:0/2135337355 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:07:10.222 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:10.215+0000 7f4d4ce00640 1 --2- 172.21.15.45:0/2135337355 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f57879ee20 0x55f5787a3490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:10.222 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:10.215+0000 7f4d47400640 1 -- 172.21.15.45:0/2135337355 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f5787958a0 msgr2=0x55f5787a2a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:07:10.222 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:10.215+0000 7f4d47400640 1 --2- 172.21.15.45:0/2135337355 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f5787958a0 0x55f5787a2a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:10.222 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:10.215+0000 7f4d47400640 1 -- 172.21.15.45:0/2135337355 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f57879ee20 msgr2=0x55f5787a3490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:07:10.222 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:10.215+0000 7f4d47400640 1 --2- 172.21.15.45:0/2135337355 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f57879ee20 0x55f5787a3490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:10.222 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:10.215+0000 7f4d47400640 1 -- 172.21.15.45:0/2135337355 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f57881cd80 con 0x55f578796610 2024-04-16T17:07:10.223 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:10.215+0000 7f4d47400640 1 --2- 172.21.15.45:0/2135337355 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f578796610 0x55f5787a2f50 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x55f578796340 tx=0x7f4d3401f860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c76fc6ffd927a16a server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:07:10.223 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:10.215+0000 7f4d32000640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-04-16T17:07:10.223 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:10.215+0000 7f4d32000640 1 -- 172.21.15.45:0/2135337355 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d340037f0 con 0x55f578796610 2024-04-16T17:07:10.223 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:10.216+0000 7f4d51c37740 1 -- 172.21.15.45:0/2135337355 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4d3809c040 con 0x55f578796610 2024-04-16T17:07:10.223 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:10.216+0000 7f4d32000640 1 -- 172.21.15.45:0/2135337355 <== mon.1 v2:172.21.15.117:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f4d34003990 con 0x55f578796610 2024-04-16T17:07:10.223 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:10.216+0000 7f4d32000640 1 -- 172.21.15.45:0/2135337355 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f4d34003b90 con 0x55f578796610 2024-04-16T17:07:10.223 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:10.216+0000 7f4d51c37740 10 client.25687 fetch_fsmap learned FSMap version 25 2024-04-16T17:07:10.223 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:10.216+0000 7f4d51c37740 1 -- 172.21.15.45:0/2135337355 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55f57881e0d0 con 0x55f578796610 2024-04-16T17:07:10.223 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:10.216+0000 7f4d32000640 1 -- 172.21.15.45:0/2135337355 <== mon.1 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d3809c040 con 0x55f578796610 2024-04-16T17:07:10.223 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:10.216+0000 7f4d32000640 1 -- 172.21.15.45:0/2135337355 <== mon.1 v2:172.21.15.117:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f4d3401e510 con 0x55f578796610 2024-04-16T17:07:10.223 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:10.216+0000 7f4d51c37740 10 client.25687 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T17:07:10.223 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:10.216+0000 7f4d51c37740 10 client.25687 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T17:07:10.223 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:10.216+0000 7f4d51c37740 20 client.25687 populate_metadata read hostname 'smithi045' 2024-04-16T17:07:10.223 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:10.216+0000 7f4d51c37740 1 --2- 172.21.15.45:0/2135337355 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55f57881e800 0x55f578820c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:10.223 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:10.216+0000 7f4d51c37740 4 client.25687 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-16T17:07:10.223 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:10.216+0000 7f4d51c37740 1 -- 172.21.15.45:0/2135337355 --> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f578702010 con 0x55f57881e800 2024-04-16T17:07:10.224 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:10.217+0000 7f4d47e00640 1 --2- 172.21.15.45:0/2135337355 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55f57881e800 0x55f578820c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:10.224 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:10.217+0000 7f4d47e00640 1 --2- 172.21.15.45:0/2135337355 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55f57881e800 0x55f578820c20 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6200df83c6a29348 server_cookie=2b0175afada582a4 in_seq=0 out_seq=0 2024-04-16T17:07:10.224 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:10.217+0000 7f4d32000640 10 client.25687 ms_handle_connect on v2:172.21.15.104:6838/3660070592 2024-04-16T17:07:10.233 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:10.232+0000 7f4d32000640 1 -- 172.21.15.45:0/2135337355 <== mds.0 v2:172.21.15.104:6838/3660070592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f578702010 con 0x55f57881e800 2024-04-16T17:07:10.233 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:10.232+0000 7f4d32000640 10 client.25687 handle_command_reply: tid=0 2024-04-16T17:07:10.233 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:10.232+0000 7f4d51c37740 1 client.25687 shutdown 2024-04-16T17:07:10.233 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:10.232+0000 7f4d51c37740 1 -- 172.21.15.45:0/2135337355 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f578796610 msgr2=0x55f5787a2f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:10.233 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:10.232+0000 7f4d51c37740 1 --2- 172.21.15.45:0/2135337355 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f578796610 0x55f5787a2f50 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x55f578796340 tx=0x7f4d3401f860 comp rx=0 tx=0).stop 2024-04-16T17:07:10.233 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:10.232+0000 7f4d51c37740 1 -- 172.21.15.45:0/2135337355 shutdown_connections 2024-04-16T17:07:10.234 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:10.232+0000 7f4d51c37740 1 --2- 172.21.15.45:0/2135337355 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55f57881e800 0x55f578820c20 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:10.234 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:10.232+0000 7f4d51c37740 1 --2- 172.21.15.45:0/2135337355 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f578796610 0x55f5787a2f50 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:10.234 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:10.232+0000 7f4d51c37740 1 --2- 172.21.15.45:0/2135337355 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f57879ee20 0x55f5787a3490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:10.234 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:10.232+0000 7f4d32000640 0 client.25687 ms_handle_reset on v2:172.21.15.104:6838/3660070592 2024-04-16T17:07:10.234 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:10.232+0000 7f4d51c37740 1 --2- 172.21.15.45:0/2135337355 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f5787958a0 0x55f5787a2a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:10.234 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:10.232+0000 7f4d51c37740 1 -- 172.21.15.45:0/2135337355 >> 172.21.15.45:0/2135337355 conn(0x55f5787305d0 msgr2=0x55f57879b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:07:10.235 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:10.232+0000 7f4d51c37740 1 -- 172.21.15.45:0/2135337355 shutdown_connections 2024-04-16T17:07:10.235 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:10.232+0000 7f4d51c37740 1 -- 172.21.15.45:0/2135337355 wait complete. 2024-04-16T17:07:10.235 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:10.233+0000 7f4d51c37740 20 client.25687 trim_cache size 0 max 16384 2024-04-16T17:07:10.235 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-04-16T17:07:10.235 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-04-16T17:07:10.235 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-04-16T17:07:10.235 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-04-16T17:07:10.236 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:10.233+0000 7f4d4ee00640 1 -- 172.21.15.45:0/3719137481 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f4d2c0a3df0 msgr2=0x7f4d2c0a62b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:10.236 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:10.233+0000 7f4d4ee00640 1 --2- 172.21.15.45:0/3719137481 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f4d2c0a3df0 0x7f4d2c0a62b0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f4d40009b30 tx=0x7f4d40003db0 comp rx=0 tx=0).stop 2024-04-16T17:07:10.236 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:10.233+0000 7f4d4ee00640 1 -- 172.21.15.45:0/3719137481 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4d48104450 msgr2=0x7f4d481a0670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:10.236 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:10.233+0000 7f4d4ee00640 1 --2- 172.21.15.45:0/3719137481 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4d48104450 0x7f4d481a0670 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f4d3800e9f0 tx=0x7f4d3800eeb0 comp rx=0 tx=0).stop 2024-04-16T17:07:10.236 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:10.233+0000 7f4d4ee00640 1 -- 172.21.15.45:0/3719137481 shutdown_connections 2024-04-16T17:07:10.236 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:10.233+0000 7f4d4ee00640 1 --2- 172.21.15.45:0/3719137481 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f4d2c0a3df0 0x7f4d2c0a62b0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:10.236 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:10.233+0000 7f4d4ee00640 1 --2- 172.21.15.45:0/3719137481 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4d48109520 0x7f4d481a0bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:10.236 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:10.233+0000 7f4d4ee00640 1 --2- 172.21.15.45:0/3719137481 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4d4819a7e0 0x7f4d4819ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:10.236 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:10.233+0000 7f4d4ee00640 1 --2- 172.21.15.45:0/3719137481 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4d48104450 0x7f4d481a0670 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:10.236 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:10.234+0000 7f4d4ee00640 1 -- 172.21.15.45:0/3719137481 >> 172.21.15.45:0/3719137481 conn(0x7f4d480ffcf0 msgr2=0x7f4d481006f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:07:10.237 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:10.234+0000 7f4d4ee00640 1 -- 172.21.15.45:0/3719137481 shutdown_connections 2024-04-16T17:07:10.237 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:10.234+0000 7f4d4ee00640 1 -- 172.21.15.45:0/3719137481 wait complete. 2024-04-16T17:07:10.477 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:07:10 smithi045 ceph-mon[31676]: pgmap v15: 129 pgs: 129 active+clean; 1.3 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 24 KiB/s wr, 23 op/s 2024-04-16T17:07:10.555 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:07:10 smithi117 ceph-mon[36730]: pgmap v15: 129 pgs: 129 active+clean; 1.3 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 24 KiB/s wr, 23 op/s 2024-04-16T17:07:10.677 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:07:10 smithi104 ceph-mon[36457]: pgmap v15: 129 pgs: 129 active+clean; 1.3 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 24 KiB/s wr, 23 op/s 2024-04-16T17:07:12.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:07:12 smithi117 ceph-mon[36730]: pgmap v16: 129 pgs: 129 active+clean; 1.3 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 25 KiB/s wr, 26 op/s 2024-04-16T17:07:12.677 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:07:12 smithi104 ceph-mon[36457]: pgmap v16: 129 pgs: 129 active+clean; 1.3 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 25 KiB/s wr, 26 op/s 2024-04-16T17:07:12.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:07:12 smithi045 ceph-mon[31676]: pgmap v16: 129 pgs: 129 active+clean; 1.3 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 25 KiB/s wr, 26 op/s 2024-04-16T17:07:14.500 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:07:14 smithi104 ceph-mon[36457]: pgmap v17: 129 pgs: 129 active+clean; 1.3 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 25 KiB/s wr, 30 op/s 2024-04-16T17:07:14.555 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:07:14 smithi117 ceph-mon[36730]: pgmap v17: 129 pgs: 129 active+clean; 1.3 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 25 KiB/s wr, 30 op/s 2024-04-16T17:07:14.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:07:14 smithi045 ceph-mon[31676]: pgmap v17: 129 pgs: 129 active+clean; 1.3 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 25 KiB/s wr, 30 op/s 2024-04-16T17:07:16.555 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:07:16 smithi117 ceph-mon[36730]: pgmap v18: 129 pgs: 129 active+clean; 1.3 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 2.5 KiB/s wr, 10 op/s 2024-04-16T17:07:16.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:07:16 smithi104 ceph-mon[36457]: pgmap v18: 129 pgs: 129 active+clean; 1.3 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 2.5 KiB/s wr, 10 op/s 2024-04-16T17:07:16.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:07:16 smithi045 ceph-mon[31676]: pgmap v18: 129 pgs: 129 active+clean; 1.3 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 2.5 KiB/s wr, 10 op/s 2024-04-16T17:07:18.555 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:07:18 smithi117 ceph-mon[36730]: pgmap v19: 129 pgs: 129 active+clean; 1.3 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 2.5 KiB/s wr, 12 op/s 2024-04-16T17:07:18.677 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:07:18 smithi104 ceph-mon[36457]: pgmap v19: 129 pgs: 129 active+clean; 1.3 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 2.5 KiB/s wr, 12 op/s 2024-04-16T17:07:18.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:07:18 smithi045 ceph-mon[31676]: pgmap v19: 129 pgs: 129 active+clean; 1.3 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 2.5 KiB/s wr, 12 op/s 2024-04-16T17:07:20.555 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:07:20 smithi117 ceph-mon[36730]: pgmap v20: 129 pgs: 129 active+clean; 1.3 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1.8 KiB/s wr, 9 op/s 2024-04-16T17:07:20.677 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:07:20 smithi104 ceph-mon[36457]: pgmap v20: 129 pgs: 129 active+clean; 1.3 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1.8 KiB/s wr, 9 op/s 2024-04-16T17:07:20.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:07:20 smithi045 ceph-mon[31676]: pgmap v20: 129 pgs: 129 active+clean; 1.3 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1.8 KiB/s wr, 9 op/s 2024-04-16T17:07:22.555 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:07:22 smithi117 ceph-mon[36730]: pgmap v21: 129 pgs: 129 active+clean; 1.3 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 1.8 KiB/s wr, 11 op/s 2024-04-16T17:07:22.677 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:07:22 smithi104 ceph-mon[36457]: pgmap v21: 129 pgs: 129 active+clean; 1.3 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 1.8 KiB/s wr, 11 op/s 2024-04-16T17:07:22.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:07:22 smithi045 ceph-mon[31676]: pgmap v21: 129 pgs: 129 active+clean; 1.3 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 1.8 KiB/s wr, 11 op/s 2024-04-16T17:07:23.385 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:23.383+0000 7f6d27600640 1 -- 172.21.15.45:0/1943922904 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6d20101080 msgr2=0x7f6d20101460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:23.385 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:23.383+0000 7f6d27600640 1 --2- 172.21.15.45:0/1943922904 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6d20101080 0x7f6d20101460 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f6d14009900 tx=0x7f6d1402bd60 comp rx=0 tx=0).stop 2024-04-16T17:07:23.385 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:23.384+0000 7f6d27600640 1 -- 172.21.15.45:0/1943922904 shutdown_connections 2024-04-16T17:07:23.385 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:23.384+0000 7f6d27600640 1 --2- 172.21.15.45:0/1943922904 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6d201019a0 0x7f6d20105e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:23.385 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:23.384+0000 7f6d27600640 1 --2- 172.21.15.45:0/1943922904 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6d201063a0 0x7f6d20106830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:23.385 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:23.384+0000 7f6d27600640 1 --2- 172.21.15.45:0/1943922904 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6d20101080 0x7f6d20101460 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:23.386 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:23.384+0000 7f6d27600640 1 -- 172.21.15.45:0/1943922904 >> 172.21.15.45:0/1943922904 conn(0x7f6d20078080 msgr2=0x7f6d20078490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:07:23.386 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:23.384+0000 7f6d27600640 1 -- 172.21.15.45:0/1943922904 shutdown_connections 2024-04-16T17:07:23.386 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:23.384+0000 7f6d27600640 1 -- 172.21.15.45:0/1943922904 wait complete. 2024-04-16T17:07:23.387 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:23.385+0000 7f6d27600640 1 Processor -- start 2024-04-16T17:07:23.387 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:23.385+0000 7f6d27600640 1 -- start start 2024-04-16T17:07:23.387 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:23.385+0000 7f6d27600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6d20101080 0x7f6d2019ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:23.387 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:23.385+0000 7f6d27600640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6d201019a0 0x7f6d2019f440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:23.387 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:23.385+0000 7f6d27600640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6d201063a0 0x7f6d201a37c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:23.387 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:23.385+0000 7f6d27600640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f6d20118a20 con 0x7f6d20101080 2024-04-16T17:07:23.388 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:23.385+0000 7f6d27600640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f6d201188a0 con 0x7f6d201019a0 2024-04-16T17:07:23.388 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:23.385+0000 7f6d27600640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f6d20118ba0 con 0x7f6d201063a0 2024-04-16T17:07:23.388 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:23.386+0000 7f6d24c00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6d20101080 0x7f6d2019ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:23.388 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:23.386+0000 7f6d24c00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6d20101080 0x7f6d2019ef00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45062/0 (socket says 172.21.15.45:45062) 2024-04-16T17:07:23.388 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:23.386+0000 7f6d24c00640 1 -- 172.21.15.45:0/3624982477 learned_addr learned my addr 172.21.15.45:0/3624982477 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:07:23.388 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:23.386+0000 7f6d25600640 1 --2- 172.21.15.45:0/3624982477 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6d201063a0 0x7f6d201a37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:23.388 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:23.386+0000 7f6d1fe00640 1 --2- 172.21.15.45:0/3624982477 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6d201019a0 0x7f6d2019f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:23.388 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:23.386+0000 7f6d25600640 1 -- 172.21.15.45:0/3624982477 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6d201019a0 msgr2=0x7f6d2019f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:23.388 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:23.386+0000 7f6d25600640 1 --2- 172.21.15.45:0/3624982477 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6d201019a0 0x7f6d2019f440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:23.388 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:23.386+0000 7f6d25600640 1 -- 172.21.15.45:0/3624982477 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6d20101080 msgr2=0x7f6d2019ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:23.388 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:23.386+0000 7f6d25600640 1 --2- 172.21.15.45:0/3624982477 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6d20101080 0x7f6d2019ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:23.388 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:23.386+0000 7f6d25600640 1 -- 172.21.15.45:0/3624982477 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d201a3ef0 con 0x7f6d201063a0 2024-04-16T17:07:23.389 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:23.387+0000 7f6d25600640 1 --2- 172.21.15.45:0/3624982477 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6d201063a0 0x7f6d201a37c0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f6d1800da10 tx=0x7f6d1800ded0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:07:23.389 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:23.387+0000 7f6d1d600640 1 -- 172.21.15.45:0/3624982477 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d1800bd70 con 0x7f6d201063a0 2024-04-16T17:07:23.389 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:23.387+0000 7f6d1d600640 1 -- 172.21.15.45:0/3624982477 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6d18003e50 con 0x7f6d201063a0 2024-04-16T17:07:23.389 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:23.387+0000 7f6d1d600640 1 -- 172.21.15.45:0/3624982477 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d180106a0 con 0x7f6d201063a0 2024-04-16T17:07:23.389 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:23.387+0000 7f6d27600640 1 -- 172.21.15.45:0/3624982477 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d201a41e0 con 0x7f6d201063a0 2024-04-16T17:07:23.389 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:23.387+0000 7f6d27600640 1 -- 172.21.15.45:0/3624982477 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d201aba70 con 0x7f6d201063a0 2024-04-16T17:07:23.391 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:23.390+0000 7f6d1d600640 1 -- 172.21.15.45:0/3624982477 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 25) ==== 138879+0+0 (secure 0 0 0) 0x7f6d18003970 con 0x7f6d201063a0 2024-04-16T17:07:23.392 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:23.391+0000 7f6d2a268740 1 Processor -- start 2024-04-16T17:07:23.392 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:23.391+0000 7f6d2a268740 1 -- start start 2024-04-16T17:07:23.393 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:23.391+0000 7f6d2a268740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bcbbac9610 0x55bcbbac8360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:23.393 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:23.392+0000 7f6d2a268740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bcbbac88a0 0x55bcbbac8d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:23.393 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:23.392+0000 7f6d2a268740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bcbbad1e20 0x55bcbbad4210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:23.393 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:23.392+0000 7f6d2a268740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55bcbba2d780 con 0x55bcbbad1e20 2024-04-16T17:07:23.393 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:23.392+0000 7f6d2a268740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55bcbb9d3f40 con 0x55bcbbac88a0 2024-04-16T17:07:23.393 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:23.392+0000 7f6d2a268740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55bcbba29740 con 0x55bcbbac9610 2024-04-16T17:07:23.393 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:23.392+0000 7f6d24c00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bcbbac88a0 0x55bcbbac8d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:23.394 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:23.392+0000 7f6d1fe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bcbbad1e20 0x55bcbbad4210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:23.394 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:23.392+0000 7f6d24c00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bcbbac88a0 0x55bcbbac8d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:35236/0 (socket says 172.21.15.45:35236) 2024-04-16T17:07:23.394 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:23.392+0000 7f6d24c00640 1 -- 172.21.15.45:0/1931565691 learned_addr learned my addr 172.21.15.45:0/1931565691 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:07:23.394 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:23.392+0000 7f6d1fe00640 1 --2- 172.21.15.45:0/1931565691 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bcbbac9610 0x55bcbbac8360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:23.395 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:23.392+0000 7f6d24c00640 1 -- 172.21.15.45:0/1931565691 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bcbbac9610 msgr2=0x55bcbbac8360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:07:23.395 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:23.392+0000 7f6d24c00640 1 --2- 172.21.15.45:0/1931565691 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bcbbac9610 0x55bcbbac8360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:23.395 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:23.392+0000 7f6d24c00640 1 -- 172.21.15.45:0/1931565691 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bcbbad1e20 msgr2=0x55bcbbad4210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:07:23.395 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:23.392+0000 7f6d24c00640 1 --2- 172.21.15.45:0/1931565691 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bcbbad1e20 0x55bcbbad4210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:23.395 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:23.392+0000 7f6d24c00640 1 -- 172.21.15.45:0/1931565691 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bcbb740210 con 0x55bcbbac88a0 2024-04-16T17:07:23.395 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:23.393+0000 7f6d1d600640 1 --2- 172.21.15.45:0/3624982477 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f6d080a3ee0 0x7f6d080a63a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:23.395 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:23.393+0000 7f6d1d600640 1 -- 172.21.15.45:0/3624982477 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f6d180d1150 con 0x7f6d201063a0 2024-04-16T17:07:23.395 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:23.393+0000 7f6d24c00640 1 --2- 172.21.15.45:0/1931565691 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bcbbac88a0 0x55bcbbac8d20 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f6d14036040 tx=0x7f6d14009610 comp rx=0 tx=0).ready entity=mon.2 client_cookie=483b55173db2ae56 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:07:23.395 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:23.393+0000 7f6d1fe00640 1 --2- 172.21.15.45:0/3624982477 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f6d080a3ee0 0x7f6d080a63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:23.395 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:23.393+0000 7f6d11c00640 1 -- 172.21.15.45:0/1931565691 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d14043070 con 0x55bcbbac88a0 2024-04-16T17:07:23.395 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:23.393+0000 7f6d11c00640 1 -- 172.21.15.45:0/1931565691 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6d1402c9b0 con 0x55bcbbac88a0 2024-04-16T17:07:23.395 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:23.393+0000 7f6d11c00640 1 -- 172.21.15.45:0/1931565691 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d14004c20 con 0x55bcbbac88a0 2024-04-16T17:07:23.396 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:23.394+0000 7f6d2a268740 1 -- 172.21.15.45:0/1931565691 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bcbbac88a0 msgr2=0x55bcbbac8d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:23.396 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:23.394+0000 7f6d2a268740 1 --2- 172.21.15.45:0/1931565691 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bcbbac88a0 0x55bcbbac8d20 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f6d14036040 tx=0x7f6d14009610 comp rx=0 tx=0).stop 2024-04-16T17:07:23.396 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:23.394+0000 7f6d1fe00640 1 --2- 172.21.15.45:0/3624982477 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f6d080a3ee0 0x7f6d080a63a0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x55bcbbac4720 tx=0x7f6d0c004730 comp rx=0 tx=0).ready entity=mgr.25411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:07:23.396 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:23.394+0000 7f6d2a268740 1 -- 172.21.15.45:0/1931565691 shutdown_connections 2024-04-16T17:07:23.396 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:23.394+0000 7f6d2a268740 1 --2- 172.21.15.45:0/1931565691 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bcbbad1e20 0x55bcbbad4210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:23.396 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:23.394+0000 7f6d2a268740 1 --2- 172.21.15.45:0/1931565691 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bcbbac88a0 0x55bcbbac8d20 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:23.396 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:23.394+0000 7f6d2a268740 1 --2- 172.21.15.45:0/1931565691 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bcbbac9610 0x55bcbbac8360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:23.397 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:23.394+0000 7f6d2a268740 1 -- 172.21.15.45:0/1931565691 >> 172.21.15.45:0/1931565691 conn(0x55bcbba635d0 msgr2=0x55bcbbac3590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:07:23.397 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:23.395+0000 7f6d2a268740 1 -- 172.21.15.45:0/1931565691 shutdown_connections 2024-04-16T17:07:23.397 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:23.395+0000 7f6d2a268740 1 -- 172.21.15.45:0/1931565691 wait complete. 2024-04-16T17:07:23.397 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:23.395+0000 7f6d2a268740 1 -- start start 2024-04-16T17:07:23.397 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:23.395+0000 7f6d2a268740 1 Processor -- start 2024-04-16T17:07:23.397 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:23.396+0000 7f6d2a268740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bcbbac88a0 0x55bcbbacec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:23.398 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:23.396+0000 7f6d2a268740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bcbbac9610 0x55bcbbacf1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:23.398 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:23.396+0000 7f6d2a268740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bcbbad1e20 0x55bcbbacf710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:23.398 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:23.396+0000 7f6d2a268740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55bcbbad4af0 con 0x55bcbbac88a0 2024-04-16T17:07:23.398 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:23.396+0000 7f6d2a268740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55bcbb9e7390 con 0x55bcbbac9610 2024-04-16T17:07:23.398 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:23.396+0000 7f6d2a268740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55bcbbad4c70 con 0x55bcbbad1e20 2024-04-16T17:07:23.398 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:23.396+0000 7f6d24c00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bcbbac88a0 0x55bcbbacec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:23.398 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:23.396+0000 7f6d25600640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bcbbad1e20 0x55bcbbacf710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:23.398 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:23.396+0000 7f6d2a268740 10 client.0 _ll_register_callbacks cb 0x55bcbb9d0f60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:07:23.398 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:23.396+0000 7f6d25600640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bcbbad1e20 0x55bcbbacf710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:58124/0 (socket says 172.21.15.45:58124) 2024-04-16T17:07:23.398 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:23.396+0000 7f6d25600640 1 -- 172.21.15.45:0/2098668018 learned_addr learned my addr 172.21.15.45:0/2098668018 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:07:23.398 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:23.396+0000 7f6d1fe00640 1 --2- 172.21.15.45:0/2098668018 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bcbbac9610 0x55bcbbacf1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:23.400 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:23.397+0000 7f6d25600640 1 -- 172.21.15.45:0/2098668018 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bcbbac9610 msgr2=0x55bcbbacf1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:07:23.400 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:23.397+0000 7f6d25600640 1 --2- 172.21.15.45:0/2098668018 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bcbbac9610 0x55bcbbacf1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:23.400 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:23.397+0000 7f6d25600640 1 -- 172.21.15.45:0/2098668018 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bcbbac88a0 msgr2=0x55bcbbacec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:07:23.400 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:23.397+0000 7f6d25600640 1 --2- 172.21.15.45:0/2098668018 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bcbbac88a0 0x55bcbbacec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:23.400 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:23.397+0000 7f6d25600640 1 -- 172.21.15.45:0/2098668018 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bcbb80c620 con 0x55bcbbad1e20 2024-04-16T17:07:23.400 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:23.397+0000 7f6d25600640 1 --2- 172.21.15.45:0/2098668018 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bcbbad1e20 0x55bcbbacf710 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f6d1809ca80 tx=0x7f6d1809d2b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f442ea61d94bb304 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:07:23.400 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:23.397+0000 7f6d06000640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-04-16T17:07:23.400 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:23.398+0000 7f6d06000640 1 -- 172.21.15.45:0/2098668018 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d180a5760 con 0x55bcbbad1e20 2024-04-16T17:07:23.400 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:23.398+0000 7f6d06000640 1 -- 172.21.15.45:0/2098668018 <== mon.1 v2:172.21.15.117:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f6d180042e0 con 0x55bcbbad1e20 2024-04-16T17:07:23.400 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:23.398+0000 7f6d2a268740 1 -- 172.21.15.45:0/2098668018 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6d1402c9b0 con 0x55bcbbad1e20 2024-04-16T17:07:23.400 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:23.398+0000 7f6d06000640 1 -- 172.21.15.45:0/2098668018 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f6d18003c60 con 0x55bcbbad1e20 2024-04-16T17:07:23.401 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:23.398+0000 7f6d2a268740 10 client.25699 fetch_fsmap learned FSMap version 25 2024-04-16T17:07:23.401 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:23.398+0000 7f6d2a268740 1 -- 172.21.15.45:0/2098668018 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55bcbbae0c80 con 0x55bcbbad1e20 2024-04-16T17:07:23.401 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:23.398+0000 7f6d06000640 1 -- 172.21.15.45:0/2098668018 <== mon.1 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6d1402c9b0 con 0x55bcbbad1e20 2024-04-16T17:07:23.401 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:23.399+0000 7f6d06000640 1 -- 172.21.15.45:0/2098668018 <== mon.1 v2:172.21.15.117:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f6d18024060 con 0x55bcbbad1e20 2024-04-16T17:07:23.401 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:23.399+0000 7f6d2a268740 10 client.25699 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T17:07:23.401 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:23.399+0000 7f6d2a268740 10 client.25699 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T17:07:23.401 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:23.399+0000 7f6d2a268740 20 client.25699 populate_metadata read hostname 'smithi045' 2024-04-16T17:07:23.401 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:23.399+0000 7f6d2a268740 1 --2- 172.21.15.45:0/2098668018 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55bcbbae2140 0x55bcbbae4570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:23.401 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:23.399+0000 7f6d2a268740 4 client.25699 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:07:23.401 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:23.399+0000 7f6d2a268740 1 -- 172.21.15.45:0/2098668018 --> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bcbba3b1a0 con 0x55bcbbae2140 2024-04-16T17:07:23.402 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:23.399+0000 7f6d24c00640 1 --2- 172.21.15.45:0/2098668018 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55bcbbae2140 0x55bcbbae4570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:23.402 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:23.400+0000 7f6d24c00640 1 --2- 172.21.15.45:0/2098668018 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55bcbbae2140 0x55bcbbae4570 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ff389e5bbf6ab286 server_cookie=20f9a70948a488b9 in_seq=0 out_seq=0 2024-04-16T17:07:23.403 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:23.400+0000 7f6d06000640 10 client.25699 ms_handle_connect on v2:172.21.15.104:6838/3660070592 2024-04-16T17:07:23.403 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:23.401+0000 7f6d06000640 1 -- 172.21.15.45:0/2098668018 <== mds.0 v2:172.21.15.104:6838/3660070592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55bcbba3b1a0 con 0x55bcbbae2140 2024-04-16T17:07:23.403 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:23.401+0000 7f6d06000640 10 client.25699 handle_command_reply: tid=0 2024-04-16T17:07:23.403 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:23.401+0000 7f6d2a268740 1 client.25699 shutdown 2024-04-16T17:07:23.403 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:23.401+0000 7f6d2a268740 1 -- 172.21.15.45:0/2098668018 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bcbbad1e20 msgr2=0x55bcbbacf710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:23.403 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:23.401+0000 7f6d2a268740 1 --2- 172.21.15.45:0/2098668018 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bcbbad1e20 0x55bcbbacf710 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f6d1809ca80 tx=0x7f6d1809d2b0 comp rx=0 tx=0).stop 2024-04-16T17:07:23.403 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:23.401+0000 7f6d2a268740 1 -- 172.21.15.45:0/2098668018 shutdown_connections 2024-04-16T17:07:23.403 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:23.401+0000 7f6d2a268740 1 --2- 172.21.15.45:0/2098668018 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55bcbbae2140 0x55bcbbae4570 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:23.403 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:23.401+0000 7f6d2a268740 1 --2- 172.21.15.45:0/2098668018 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bcbbad1e20 0x55bcbbacf710 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:23.403 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:23.401+0000 7f6d2a268740 1 --2- 172.21.15.45:0/2098668018 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bcbbac9610 0x55bcbbacf1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:23.403 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:23.401+0000 7f6d2a268740 1 --2- 172.21.15.45:0/2098668018 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bcbbac88a0 0x55bcbbacec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:23.403 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:23.401+0000 7f6d06000640 0 client.25699 ms_handle_reset on v2:172.21.15.104:6838/3660070592 2024-04-16T17:07:23.403 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:23.401+0000 7f6d2a268740 1 -- 172.21.15.45:0/2098668018 >> 172.21.15.45:0/2098668018 conn(0x55bcbba635d0 msgr2=0x55bcbbac52e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:07:23.403 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:23.401+0000 7f6d2a268740 1 -- 172.21.15.45:0/2098668018 shutdown_connections 2024-04-16T17:07:23.403 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:23.401+0000 7f6d2a268740 1 -- 172.21.15.45:0/2098668018 wait complete. 2024-04-16T17:07:23.403 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:23.401+0000 7f6d2a268740 20 client.25699 trim_cache size 0 max 16384 2024-04-16T17:07:23.423 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:23.422+0000 7f6d2a268740 1 Processor -- start 2024-04-16T17:07:23.424 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:23.423+0000 7f6d2a268740 1 -- start start 2024-04-16T17:07:23.425 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:23.423+0000 7f6d2a268740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bcbbac88a0 0x55bcbbad13c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:23.425 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:23.423+0000 7f6d2a268740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bcbbac9610 0x55bcbbadd9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:23.425 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:23.423+0000 7f6d2a268740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bcbbad1e20 0x55bcbbacddf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:23.425 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:23.423+0000 7f6d2a268740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f6d201b8200 con 0x55bcbbad1e20 2024-04-16T17:07:23.425 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:23.423+0000 7f6d2a268740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f6d201b5ad0 con 0x55bcbbac88a0 2024-04-16T17:07:23.425 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:23.423+0000 7f6d2a268740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f6d201b3900 con 0x55bcbbac9610 2024-04-16T17:07:23.425 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:23.423+0000 7f6d24c00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bcbbac88a0 0x55bcbbad13c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:23.425 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:23.423+0000 7f6d24c00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bcbbac88a0 0x55bcbbad13c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:35258/0 (socket says 172.21.15.45:35258) 2024-04-16T17:07:23.425 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:23.423+0000 7f6d24c00640 1 -- 172.21.15.45:0/3504831914 learned_addr learned my addr 172.21.15.45:0/3504831914 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:07:23.425 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:23.424+0000 7f6d1fe00640 1 --2- 172.21.15.45:0/3504831914 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bcbbac9610 0x55bcbbadd9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:23.425 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:23.424+0000 7f6d25600640 1 --2- 172.21.15.45:0/3504831914 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bcbbad1e20 0x55bcbbacddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:23.426 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:23.424+0000 7f6d24c00640 1 -- 172.21.15.45:0/3504831914 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bcbbac9610 msgr2=0x55bcbbadd9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:07:23.426 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:23.424+0000 7f6d24c00640 1 --2- 172.21.15.45:0/3504831914 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bcbbac9610 0x55bcbbadd9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:23.426 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:23.424+0000 7f6d24c00640 1 -- 172.21.15.45:0/3504831914 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bcbbad1e20 msgr2=0x55bcbbacddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:07:23.426 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:23.424+0000 7f6d24c00640 1 --2- 172.21.15.45:0/3504831914 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bcbbad1e20 0x55bcbbacddf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:23.426 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:23.424+0000 7f6d24c00640 1 -- 172.21.15.45:0/3504831914 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bcbbac4750 con 0x55bcbbac88a0 2024-04-16T17:07:23.426 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:23.424+0000 7f6d25600640 1 --2- 172.21.15.45:0/3504831914 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bcbbad1e20 0x55bcbbacddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:07:23.427 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:23.424+0000 7f6d24c00640 1 --2- 172.21.15.45:0/3504831914 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bcbbac88a0 0x55bcbbad13c0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f6d14004730 tx=0x7f6d14009680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d338cbd91ebe60e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:07:23.427 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:23.425+0000 7f6d11200640 1 -- 172.21.15.45:0/3504831914 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d14043040 con 0x55bcbbac88a0 2024-04-16T17:07:23.427 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:23.425+0000 7f6d11200640 1 -- 172.21.15.45:0/3504831914 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6d14004040 con 0x55bcbbac88a0 2024-04-16T17:07:23.427 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:23.425+0000 7f6d11200640 1 -- 172.21.15.45:0/3504831914 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d14005480 con 0x55bcbbac88a0 2024-04-16T17:07:23.427 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:23.426+0000 7f6d2a268740 1 -- 172.21.15.45:0/3504831914 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bcbbac88a0 msgr2=0x55bcbbad13c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:23.427 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:23.426+0000 7f6d2a268740 1 --2- 172.21.15.45:0/3504831914 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bcbbac88a0 0x55bcbbad13c0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f6d14004730 tx=0x7f6d14009680 comp rx=0 tx=0).stop 2024-04-16T17:07:23.427 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:23.426+0000 7f6d2a268740 1 -- 172.21.15.45:0/3504831914 shutdown_connections 2024-04-16T17:07:23.427 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:23.426+0000 7f6d2a268740 1 --2- 172.21.15.45:0/3504831914 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bcbbac9610 0x55bcbbadd9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:23.427 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:23.426+0000 7f6d2a268740 1 --2- 172.21.15.45:0/3504831914 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bcbbad1e20 0x55bcbbacddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:23.428 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:23.426+0000 7f6d2a268740 1 --2- 172.21.15.45:0/3504831914 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bcbbac88a0 0x55bcbbad13c0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:23.428 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:23.426+0000 7f6d2a268740 1 -- 172.21.15.45:0/3504831914 >> 172.21.15.45:0/3504831914 conn(0x55bcbba635d0 msgr2=0x55bcbbac4960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:07:23.428 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:23.426+0000 7f6d2a268740 1 -- 172.21.15.45:0/3504831914 shutdown_connections 2024-04-16T17:07:23.428 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:23.426+0000 7f6d2a268740 1 -- 172.21.15.45:0/3504831914 wait complete. 2024-04-16T17:07:23.428 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:23.426+0000 7f6d2a268740 1 -- start start 2024-04-16T17:07:23.430 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:23.427+0000 7f6d2a268740 1 Processor -- start 2024-04-16T17:07:23.430 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:23.427+0000 7f6d2a268740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bcbbac88a0 0x55bcbbad9f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:23.430 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:23.427+0000 7f6d2a268740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bcbbac9610 0x55bcbbada440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:23.430 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:23.427+0000 7f6d2a268740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bcbbad1e20 0x55bcbbad2d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:23.430 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:23.427+0000 7f6d2a268740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f6d201b3f90 con 0x55bcbbad1e20 2024-04-16T17:07:23.430 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:23.427+0000 7f6d2a268740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f6d201b5400 con 0x55bcbbac88a0 2024-04-16T17:07:23.430 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:23.427+0000 7f6d2a268740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f6d201b4660 con 0x55bcbbac9610 2024-04-16T17:07:23.430 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:23.427+0000 7f6d2a268740 10 client.0 _ll_register_callbacks cb 0x55bcbbad1d70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:07:23.430 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:23.427+0000 7f6d24c00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bcbbac88a0 0x55bcbbad9f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:23.431 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:23.427+0000 7f6d24c00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bcbbac88a0 0x55bcbbad9f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:35260/0 (socket says 172.21.15.45:35260) 2024-04-16T17:07:23.431 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:23.427+0000 7f6d24c00640 1 -- 172.21.15.45:0/131751921 learned_addr learned my addr 172.21.15.45:0/131751921 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:07:23.431 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:23.427+0000 7f6d25600640 1 --2- 172.21.15.45:0/131751921 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bcbbad1e20 0x55bcbbad2d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:23.431 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:23.427+0000 7f6d1fe00640 1 --2- 172.21.15.45:0/131751921 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bcbbac9610 0x55bcbbada440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:23.431 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:23.428+0000 7f6d25600640 1 -- 172.21.15.45:0/131751921 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bcbbac9610 msgr2=0x55bcbbada440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:07:23.431 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:23.428+0000 7f6d25600640 1 --2- 172.21.15.45:0/131751921 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bcbbac9610 0x55bcbbada440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:23.431 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:23.428+0000 7f6d25600640 1 -- 172.21.15.45:0/131751921 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bcbbac88a0 msgr2=0x55bcbbad9f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:07:23.431 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:23.428+0000 7f6d25600640 1 --2- 172.21.15.45:0/131751921 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bcbbac88a0 0x55bcbbad9f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:23.431 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:23.428+0000 7f6d25600640 1 -- 172.21.15.45:0/131751921 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bcbbac7b70 con 0x55bcbbad1e20 2024-04-16T17:07:23.431 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:23.428+0000 7f6d24c00640 1 --2- 172.21.15.45:0/131751921 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bcbbac88a0 0x55bcbbad9f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:07:23.431 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:23.428+0000 7f6d25600640 1 --2- 172.21.15.45:0/131751921 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bcbbad1e20 0x55bcbbad2d90 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f6d1800b630 tx=0x7f6d18004230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e09be780b48c6bd server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:07:23.431 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:23.428+0000 7f6d06000640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-16T17:07:23.431 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:23.429+0000 7f6d06000640 1 -- 172.21.15.45:0/131751921 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d1809c630 con 0x55bcbbad1e20 2024-04-16T17:07:23.431 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:23.429+0000 7f6d06000640 1 -- 172.21.15.45:0/131751921 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f6d1809bf40 con 0x55bcbbad1e20 2024-04-16T17:07:23.431 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:23.429+0000 7f6d2a268740 1 -- 172.21.15.45:0/131751921 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6d14004040 con 0x55bcbbad1e20 2024-04-16T17:07:23.431 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:23.429+0000 7f6d06000640 1 -- 172.21.15.45:0/131751921 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f6d1809c450 con 0x55bcbbad1e20 2024-04-16T17:07:23.432 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:23.429+0000 7f6d2a268740 10 client.15570 fetch_fsmap learned FSMap version 25 2024-04-16T17:07:23.432 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:23.429+0000 7f6d2a268740 1 -- 172.21.15.45:0/131751921 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55bcbbb48d30 con 0x55bcbbad1e20 2024-04-16T17:07:23.432 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:23.429+0000 7f6d06000640 1 -- 172.21.15.45:0/131751921 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6d14004040 con 0x55bcbbad1e20 2024-04-16T17:07:23.432 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:23.429+0000 7f6d06000640 1 -- 172.21.15.45:0/131751921 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f6d18082020 con 0x55bcbbad1e20 2024-04-16T17:07:23.432 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:23.429+0000 7f6d2a268740 10 client.15570 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T17:07:23.432 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:23.429+0000 7f6d2a268740 10 client.15570 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T17:07:23.433 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:23.429+0000 7f6d2a268740 20 client.15570 populate_metadata read hostname 'smithi045' 2024-04-16T17:07:23.433 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:23.429+0000 7f6d2a268740 1 --2- 172.21.15.45:0/131751921 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55bcbbb49370 0x55bcbbb4b790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:23.433 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:23.429+0000 7f6d2a268740 4 client.15570 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-16T17:07:23.433 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:23.429+0000 7f6d2a268740 1 -- 172.21.15.45:0/131751921 --> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55bcbba35010 con 0x55bcbbb49370 2024-04-16T17:07:23.433 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:23.430+0000 7f6d24c00640 1 --2- 172.21.15.45:0/131751921 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55bcbbb49370 0x55bcbbb4b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:23.433 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:23.430+0000 7f6d24c00640 1 --2- 172.21.15.45:0/131751921 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55bcbbb49370 0x55bcbbb4b790 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1f6e0db866d5c2b9 server_cookie=b0e9de30504fce2e in_seq=0 out_seq=0 2024-04-16T17:07:23.433 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:23.430+0000 7f6d06000640 10 client.15570 ms_handle_connect on v2:172.21.15.104:6838/3660070592 2024-04-16T17:07:23.436 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:23.434+0000 7f6d06000640 1 -- 172.21.15.45:0/131751921 <== mds.0 v2:172.21.15.104:6838/3660070592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55bcbba35010 con 0x55bcbbb49370 2024-04-16T17:07:23.436 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:23.434+0000 7f6d06000640 10 client.15570 handle_command_reply: tid=0 2024-04-16T17:07:23.436 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:23.434+0000 7f6d2a268740 1 client.15570 shutdown 2024-04-16T17:07:23.436 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:23.434+0000 7f6d2a268740 1 -- 172.21.15.45:0/131751921 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bcbbad1e20 msgr2=0x55bcbbad2d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:23.436 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:23.434+0000 7f6d2a268740 1 --2- 172.21.15.45:0/131751921 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bcbbad1e20 0x55bcbbad2d90 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f6d1800b630 tx=0x7f6d18004230 comp rx=0 tx=0).stop 2024-04-16T17:07:23.436 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:23.434+0000 7f6d2a268740 1 -- 172.21.15.45:0/131751921 shutdown_connections 2024-04-16T17:07:23.436 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:23.434+0000 7f6d2a268740 1 --2- 172.21.15.45:0/131751921 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55bcbbb49370 0x55bcbbb4b790 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:23.436 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:23.434+0000 7f6d2a268740 1 --2- 172.21.15.45:0/131751921 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bcbbac9610 0x55bcbbada440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:23.436 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:23.434+0000 7f6d2a268740 1 --2- 172.21.15.45:0/131751921 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bcbbad1e20 0x55bcbbad2d90 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:23.436 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:23.434+0000 7f6d2a268740 1 --2- 172.21.15.45:0/131751921 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bcbbac88a0 0x55bcbbad9f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:23.436 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:23.434+0000 7f6d2a268740 1 -- 172.21.15.45:0/131751921 >> 172.21.15.45:0/131751921 conn(0x55bcbba635d0 msgr2=0x55bcbbac51f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:07:23.436 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:23.434+0000 7f6d06000640 0 client.15570 ms_handle_reset on v2:172.21.15.104:6838/3660070592 2024-04-16T17:07:23.436 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:23.435+0000 7f6d2a268740 1 -- 172.21.15.45:0/131751921 shutdown_connections 2024-04-16T17:07:23.436 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:23.435+0000 7f6d2a268740 1 -- 172.21.15.45:0/131751921 wait complete. 2024-04-16T17:07:23.436 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:23.435+0000 7f6d2a268740 20 client.15570 trim_cache size 0 max 16384 2024-04-16T17:07:23.437 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-04-16T17:07:23.437 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-04-16T17:07:23.437 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-04-16T17:07:23.437 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-04-16T17:07:23.438 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:23.435+0000 7f6d27600640 1 -- 172.21.15.45:0/3624982477 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f6d080a3ee0 msgr2=0x7f6d080a63a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:23.438 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:23.435+0000 7f6d27600640 1 --2- 172.21.15.45:0/3624982477 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f6d080a3ee0 0x7f6d080a63a0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x55bcbbac4720 tx=0x7f6d0c004730 comp rx=0 tx=0).stop 2024-04-16T17:07:23.438 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:23.435+0000 7f6d27600640 1 -- 172.21.15.45:0/3624982477 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6d201063a0 msgr2=0x7f6d201a37c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:23.438 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:23.435+0000 7f6d27600640 1 --2- 172.21.15.45:0/3624982477 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6d201063a0 0x7f6d201a37c0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f6d1800da10 tx=0x7f6d1800ded0 comp rx=0 tx=0).stop 2024-04-16T17:07:23.438 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:23.435+0000 7f6d27600640 1 -- 172.21.15.45:0/3624982477 shutdown_connections 2024-04-16T17:07:23.438 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:23.435+0000 7f6d27600640 1 --2- 172.21.15.45:0/3624982477 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f6d080a3ee0 0x7f6d080a63a0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:23.438 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:23.435+0000 7f6d27600640 1 --2- 172.21.15.45:0/3624982477 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6d201063a0 0x7f6d201a37c0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:23.438 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:23.435+0000 7f6d27600640 1 --2- 172.21.15.45:0/3624982477 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6d201019a0 0x7f6d2019f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:23.438 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:23.436+0000 7f6d27600640 1 --2- 172.21.15.45:0/3624982477 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6d20101080 0x7f6d2019ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:23.438 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:23.436+0000 7f6d27600640 1 -- 172.21.15.45:0/3624982477 >> 172.21.15.45:0/3624982477 conn(0x7f6d20078080 msgr2=0x7f6d20103e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:07:23.438 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:23.436+0000 7f6d27600640 1 -- 172.21.15.45:0/3624982477 shutdown_connections 2024-04-16T17:07:23.438 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:23.436+0000 7f6d27600640 1 -- 172.21.15.45:0/3624982477 wait complete. 2024-04-16T17:07:24.555 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:07:24 smithi117 ceph-mon[36730]: pgmap v22: 129 pgs: 129 active+clean; 1.3 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 852 B/s wr, 9 op/s 2024-04-16T17:07:24.677 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:07:24 smithi104 ceph-mon[36457]: pgmap v22: 129 pgs: 129 active+clean; 1.3 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 852 B/s wr, 9 op/s 2024-04-16T17:07:24.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:07:24 smithi045 ceph-mon[31676]: pgmap v22: 129 pgs: 129 active+clean; 1.3 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 852 B/s wr, 9 op/s 2024-04-16T17:07:25.170 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-16T17:07:25.170 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-16T17:07:25.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.427+0000 7f381be00640 1 -- 172.21.15.45:0/2277045372 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f381c076530 msgr2=0x7f381c076910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:25.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.427+0000 7f381be00640 1 --2- 172.21.15.45:0/2277045372 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f381c076530 0x7f381c076910 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f380c009900 tx=0x7f380c02bc40 comp rx=0 tx=0).stop 2024-04-16T17:07:25.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.427+0000 7f381be00640 1 -- 172.21.15.45:0/2277045372 shutdown_connections 2024-04-16T17:07:25.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.427+0000 7f381be00640 1 --2- 172.21.15.45:0/2277045372 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f381c112d50 0x7f381c115140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:25.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.427+0000 7f381be00640 1 --2- 172.21.15.45:0/2277045372 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f381c076e50 0x7f381c112810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:25.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.427+0000 7f381be00640 1 --2- 172.21.15.45:0/2277045372 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f381c076530 0x7f381c076910 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:25.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.427+0000 7f381be00640 1 -- 172.21.15.45:0/2277045372 >> 172.21.15.45:0/2277045372 conn(0x7f381c0ffb40 msgr2=0x7f381c101f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:07:25.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.427+0000 7f381be00640 1 -- 172.21.15.45:0/2277045372 shutdown_connections 2024-04-16T17:07:25.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.427+0000 7f381be00640 1 -- 172.21.15.45:0/2277045372 wait complete. 2024-04-16T17:07:25.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.429+0000 7f381be00640 1 Processor -- start 2024-04-16T17:07:25.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.429+0000 7f381be00640 1 -- start start 2024-04-16T17:07:25.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.429+0000 7f381be00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f381c076530 0x7f381c19eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:25.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.429+0000 7f381be00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f381c076e50 0x7f381c19f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:25.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.429+0000 7f381be00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f381c112d50 0x7f381c1a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:25.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.429+0000 7f381be00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f381c1188a0 con 0x7f381c076530 2024-04-16T17:07:25.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.429+0000 7f381be00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f381c118720 con 0x7f381c076e50 2024-04-16T17:07:25.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.429+0000 7f381be00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f381c118a20 con 0x7f381c112d50 2024-04-16T17:07:25.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.430+0000 7f381aa00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f381c076530 0x7f381c19eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:25.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.430+0000 7f381aa00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f381c076530 0x7f381c19eda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45124/0 (socket says 172.21.15.45:45124) 2024-04-16T17:07:25.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.430+0000 7f381aa00640 1 -- 172.21.15.45:0/1494548399 learned_addr learned my addr 172.21.15.45:0/1494548399 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:07:25.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.430+0000 7f381b400640 1 --2- 172.21.15.45:0/1494548399 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f381c112d50 0x7f381c1a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:25.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.430+0000 7f3813e00640 1 --2- 172.21.15.45:0/1494548399 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f381c076e50 0x7f381c19f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:25.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.430+0000 7f381aa00640 1 -- 172.21.15.45:0/1494548399 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f381c112d50 msgr2=0x7f381c1a3660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:25.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.430+0000 7f381aa00640 1 --2- 172.21.15.45:0/1494548399 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f381c112d50 0x7f381c1a3660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:25.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.430+0000 7f381aa00640 1 -- 172.21.15.45:0/1494548399 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f381c076e50 msgr2=0x7f381c19f2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:25.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.430+0000 7f381aa00640 1 --2- 172.21.15.45:0/1494548399 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f381c076e50 0x7f381c19f2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:25.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.430+0000 7f381aa00640 1 -- 172.21.15.45:0/1494548399 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f381c1a3d90 con 0x7f381c076530 2024-04-16T17:07:25.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.430+0000 7f381b400640 1 --2- 172.21.15.45:0/1494548399 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f381c112d50 0x7f381c1a3660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:07:25.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.430+0000 7f3813e00640 1 --2- 172.21.15.45:0/1494548399 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f381c076e50 0x7f381c19f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:07:25.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.431+0000 7f381aa00640 1 --2- 172.21.15.45:0/1494548399 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f381c076530 0x7f381c19eda0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f380c006010 tx=0x7f380c0042d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:07:25.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.431+0000 7f3813400640 1 -- 172.21.15.45:0/1494548399 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f380c041070 con 0x7f381c076530 2024-04-16T17:07:25.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.431+0000 7f3813400640 1 -- 172.21.15.45:0/1494548399 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f380c02ccd0 con 0x7f381c076530 2024-04-16T17:07:25.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.431+0000 7f3813400640 1 -- 172.21.15.45:0/1494548399 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f380c003d70 con 0x7f381c076530 2024-04-16T17:07:25.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.431+0000 7f381be00640 1 -- 172.21.15.45:0/1494548399 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f381c1a4020 con 0x7f381c076530 2024-04-16T17:07:25.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.431+0000 7f381be00640 1 -- 172.21.15.45:0/1494548399 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f381c1ab910 con 0x7f381c076530 2024-04-16T17:07:25.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.435+0000 7f3824573740 1 Processor -- start 2024-04-16T17:07:25.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.435+0000 7f3824573740 1 -- start start 2024-04-16T17:07:25.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.435+0000 7f3813400640 1 -- 172.21.15.45:0/1494548399 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) ==== 138879+0+0 (secure 0 0 0) 0x7f380c044070 con 0x7f381c076530 2024-04-16T17:07:25.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.435+0000 7f3824573740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559c01e81390 0x559c01e81770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:25.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.435+0000 7f3824573740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559c01e81cb0 0x559c01e8a350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:25.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.435+0000 7f3824573740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559c01e8a890 0x559c01e8cc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:25.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.435+0000 7f3824573740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x559c01de7120 con 0x559c01e81cb0 2024-04-16T17:07:25.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.435+0000 7f3824573740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x559c01d8d340 con 0x559c01e8a890 2024-04-16T17:07:25.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.435+0000 7f3824573740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x559c01de30e0 con 0x559c01e81390 2024-04-16T17:07:25.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.436+0000 7f381b400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559c01e81390 0x559c01e81770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:25.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.436+0000 7f381aa00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559c01e81cb0 0x559c01e8a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:25.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.436+0000 7f381b400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559c01e81390 0x559c01e81770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:58158/0 (socket says 172.21.15.45:58158) 2024-04-16T17:07:25.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.436+0000 7f381b400640 1 -- 172.21.15.45:0/2455625974 learned_addr learned my addr 172.21.15.45:0/2455625974 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:07:25.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.436+0000 7f3813e00640 1 --2- 172.21.15.45:0/2455625974 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559c01e8a890 0x559c01e8cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:25.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.436+0000 7f381b400640 1 -- 172.21.15.45:0/2455625974 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559c01e8a890 msgr2=0x559c01e8cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:07:25.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.436+0000 7f381b400640 1 --2- 172.21.15.45:0/2455625974 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559c01e8a890 0x559c01e8cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:25.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.436+0000 7f381b400640 1 -- 172.21.15.45:0/2455625974 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559c01e81cb0 msgr2=0x559c01e8a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:07:25.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.436+0000 7f381b400640 1 --2- 172.21.15.45:0/2455625974 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559c01e81cb0 0x559c01e8a350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:25.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.436+0000 7f381b400640 1 -- 172.21.15.45:0/2455625974 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559c01bc37d0 con 0x559c01e81390 2024-04-16T17:07:25.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.437+0000 7f381b400640 1 --2- 172.21.15.45:0/2455625974 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559c01e81390 0x559c01e81770 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f38140106c0 tx=0x7f3814010b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=713b41880dab45b4 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:07:25.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.437+0000 7f380be00640 1 -- 172.21.15.45:0/2455625974 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f381400e8d0 con 0x559c01e81390 2024-04-16T17:07:25.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.437+0000 7f380be00640 1 -- 172.21.15.45:0/2455625974 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3814012450 con 0x559c01e81390 2024-04-16T17:07:25.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.437+0000 7f380be00640 1 -- 172.21.15.45:0/2455625974 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f381401c630 con 0x559c01e81390 2024-04-16T17:07:25.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.438+0000 7f3813400640 1 --2- 172.21.15.45:0/1494548399 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f38000a3ec0 0x7f38000a6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:25.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.438+0000 7f3824573740 1 -- 172.21.15.45:0/2455625974 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559c01e81390 msgr2=0x559c01e81770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:25.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.438+0000 7f3824573740 1 --2- 172.21.15.45:0/2455625974 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559c01e81390 0x559c01e81770 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f38140106c0 tx=0x7f3814010b80 comp rx=0 tx=0).stop 2024-04-16T17:07:25.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.438+0000 7f3813400640 1 -- 172.21.15.45:0/1494548399 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f380c0f2790 con 0x7f381c076530 2024-04-16T17:07:25.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.438+0000 7f381aa00640 1 --2- 172.21.15.45:0/1494548399 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f38000a3ec0 0x7f38000a6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:25.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.438+0000 7f3824573740 1 -- 172.21.15.45:0/2455625974 shutdown_connections 2024-04-16T17:07:25.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.438+0000 7f3824573740 1 --2- 172.21.15.45:0/2455625974 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559c01e8a890 0x559c01e8cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:25.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.438+0000 7f3824573740 1 --2- 172.21.15.45:0/2455625974 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559c01e81cb0 0x559c01e8a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:25.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.438+0000 7f3824573740 1 --2- 172.21.15.45:0/2455625974 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559c01e81390 0x559c01e81770 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:25.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.438+0000 7f3824573740 1 -- 172.21.15.45:0/2455625974 >> 172.21.15.45:0/2455625974 conn(0x559c01e7bac0 msgr2=0x559c01e7bea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:07:25.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.438+0000 7f3824573740 1 -- 172.21.15.45:0/2455625974 shutdown_connections 2024-04-16T17:07:25.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.438+0000 7f3824573740 1 -- 172.21.15.45:0/2455625974 wait complete. 2024-04-16T17:07:25.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.439+0000 7f381aa00640 1 --2- 172.21.15.45:0/1494548399 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f38000a3ec0 0x7f38000a6380 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f380c0efb20 tx=0x7f380c0f8530 comp rx=0 tx=0).ready entity=mgr.25411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:07:25.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.439+0000 7f3824573740 1 -- start start 2024-04-16T17:07:25.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.439+0000 7f3824573740 1 Processor -- start 2024-04-16T17:07:25.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.440+0000 7f3824573740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559c01e81390 0x559c01e87900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:25.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.440+0000 7f3824573740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559c01e81cb0 0x559c01e87e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:25.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.440+0000 7f3824573740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559c01e8a890 0x559c01e88380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:25.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.440+0000 7f3824573740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x559c01e8d680 con 0x559c01e81390 2024-04-16T17:07:25.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.440+0000 7f3824573740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x559c01da0710 con 0x559c01e81cb0 2024-04-16T17:07:25.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.440+0000 7f3824573740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x559c01e8d800 con 0x559c01e8a890 2024-04-16T17:07:25.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.440+0000 7f3824573740 10 client.0 _ll_register_callbacks cb 0x559c01d9baf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:07:25.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.440+0000 7f381b400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559c01e81390 0x559c01e87900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:25.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.440+0000 7f381b400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559c01e81390 0x559c01e87900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45142/0 (socket says 172.21.15.45:45142) 2024-04-16T17:07:25.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.440+0000 7f381b400640 1 -- 172.21.15.45:0/4215733188 learned_addr learned my addr 172.21.15.45:0/4215733188 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:07:25.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.440+0000 7f3813e00640 1 --2- 172.21.15.45:0/4215733188 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559c01e81cb0 0x559c01e87e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:25.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.440+0000 7f381b400640 1 --2- 172.21.15.45:0/4215733188 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559c01e8a890 0x559c01e88380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:25.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.441+0000 7f3813e00640 1 -- 172.21.15.45:0/4215733188 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559c01e8a890 msgr2=0x559c01e88380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:07:25.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.441+0000 7f3813e00640 1 --2- 172.21.15.45:0/4215733188 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559c01e8a890 0x559c01e88380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:25.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.441+0000 7f3813e00640 1 -- 172.21.15.45:0/4215733188 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559c01e81390 msgr2=0x559c01e87900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:07:25.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.441+0000 7f3813e00640 1 --2- 172.21.15.45:0/4215733188 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559c01e81390 0x559c01e87900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:25.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.441+0000 7f3813e00640 1 -- 172.21.15.45:0/4215733188 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559c01af93f0 con 0x559c01e81cb0 2024-04-16T17:07:25.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.441+0000 7f3813e00640 1 --2- 172.21.15.45:0/4215733188 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559c01e81cb0 0x559c01e87e40 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f3804012cf0 tx=0x7f380400f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4b89e15ae42e0963 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:07:25.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.441+0000 7f3808c00640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-04-16T17:07:25.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.442+0000 7f3808c00640 1 -- 172.21.15.45:0/4215733188 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3804012110 con 0x559c01e81cb0 2024-04-16T17:07:25.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.442+0000 7f3824573740 1 -- 172.21.15.45:0/4215733188 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3814012450 con 0x559c01e81cb0 2024-04-16T17:07:25.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.442+0000 7f3808c00640 1 -- 172.21.15.45:0/4215733188 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f3804004050 con 0x559c01e81cb0 2024-04-16T17:07:25.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.442+0000 7f3808c00640 1 -- 172.21.15.45:0/4215733188 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f3804025050 con 0x559c01e81cb0 2024-04-16T17:07:25.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.442+0000 7f3824573740 10 client.25715 fetch_fsmap learned FSMap version 25 2024-04-16T17:07:25.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.442+0000 7f3824573740 1 -- 172.21.15.45:0/4215733188 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x559c01ab90f0 con 0x559c01e81cb0 2024-04-16T17:07:25.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.442+0000 7f3808c00640 1 -- 172.21.15.45:0/4215733188 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3814012450 con 0x559c01e81cb0 2024-04-16T17:07:25.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.442+0000 7f3808c00640 1 -- 172.21.15.45:0/4215733188 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f3804010430 con 0x559c01e81cb0 2024-04-16T17:07:25.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.443+0000 7f3824573740 10 client.25715 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T17:07:25.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.443+0000 7f3824573740 10 client.25715 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:07:25.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.443+0000 7f3824573740 20 client.25715 populate_metadata read hostname 'smithi045' 2024-04-16T17:07:25.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.443+0000 7f3824573740 1 --2- 172.21.15.45:0/4215733188 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x559c01e9ac90 0x559c01e9d080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:25.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.443+0000 7f3824573740 4 client.25715 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:07:25.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.443+0000 7f3824573740 1 -- 172.21.15.45:0/4215733188 --> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559c01df4b40 con 0x559c01e9ac90 2024-04-16T17:07:25.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.443+0000 7f381b400640 1 --2- 172.21.15.45:0/4215733188 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x559c01e9ac90 0x559c01e9d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:25.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.443+0000 7f381b400640 1 --2- 172.21.15.45:0/4215733188 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x559c01e9ac90 0x559c01e9d080 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8ded0511d01e0a54 server_cookie=e4bd64638921ac07 in_seq=0 out_seq=0 2024-04-16T17:07:25.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.443+0000 7f3808c00640 10 client.25715 ms_handle_connect on v2:172.21.15.104:6838/3660070592 2024-04-16T17:07:25.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.444+0000 7f3808c00640 1 -- 172.21.15.45:0/4215733188 <== mds.0 v2:172.21.15.104:6838/3660070592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559c01df4b40 con 0x559c01e9ac90 2024-04-16T17:07:25.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.444+0000 7f3808c00640 10 client.25715 handle_command_reply: tid=0 2024-04-16T17:07:25.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.444+0000 7f3824573740 1 client.25715 shutdown 2024-04-16T17:07:25.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.445+0000 7f3824573740 1 -- 172.21.15.45:0/4215733188 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559c01e81cb0 msgr2=0x559c01e87e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:25.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.445+0000 7f3824573740 1 --2- 172.21.15.45:0/4215733188 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559c01e81cb0 0x559c01e87e40 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f3804012cf0 tx=0x7f380400f790 comp rx=0 tx=0).stop 2024-04-16T17:07:25.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.445+0000 7f3824573740 1 -- 172.21.15.45:0/4215733188 shutdown_connections 2024-04-16T17:07:25.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.445+0000 7f3824573740 1 --2- 172.21.15.45:0/4215733188 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x559c01e9ac90 0x559c01e9d080 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:25.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.445+0000 7f3824573740 1 --2- 172.21.15.45:0/4215733188 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559c01e8a890 0x559c01e88380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:25.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.445+0000 7f3824573740 1 --2- 172.21.15.45:0/4215733188 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559c01e81cb0 0x559c01e87e40 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:25.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.445+0000 7f3824573740 1 --2- 172.21.15.45:0/4215733188 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559c01e81390 0x559c01e87900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:25.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.445+0000 7f3824573740 1 -- 172.21.15.45:0/4215733188 >> 172.21.15.45:0/4215733188 conn(0x559c01e7bac0 msgr2=0x559c01e7d810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:07:25.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.445+0000 7f3808c00640 0 client.25715 ms_handle_reset on v2:172.21.15.104:6838/3660070592 2024-04-16T17:07:25.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.445+0000 7f3824573740 1 -- 172.21.15.45:0/4215733188 shutdown_connections 2024-04-16T17:07:25.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.445+0000 7f3824573740 1 -- 172.21.15.45:0/4215733188 wait complete. 2024-04-16T17:07:25.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.445+0000 7f3824573740 20 client.25715 trim_cache size 0 max 16384 2024-04-16T17:07:25.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.470+0000 7f3824573740 1 Processor -- start 2024-04-16T17:07:25.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.471+0000 7f3824573740 1 -- start start 2024-04-16T17:07:25.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.471+0000 7f3824573740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559c01e81390 0x559c01e8a030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:25.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.471+0000 7f3824573740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559c01e81cb0 0x559c01e86840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:25.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.471+0000 7f3824573740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559c01e8a890 0x559c01e96570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:25.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.471+0000 7f3824573740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f381c1b8140 con 0x559c01e81390 2024-04-16T17:07:25.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.471+0000 7f3824573740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f381c1b5a10 con 0x559c01e81cb0 2024-04-16T17:07:25.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.471+0000 7f3824573740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f381c1b3840 con 0x559c01e8a890 2024-04-16T17:07:25.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.471+0000 7f3813e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559c01e81cb0 0x559c01e86840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:25.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.471+0000 7f3813e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559c01e81cb0 0x559c01e86840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:35310/0 (socket says 172.21.15.45:35310) 2024-04-16T17:07:25.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.471+0000 7f3813e00640 1 -- 172.21.15.45:0/1684944224 learned_addr learned my addr 172.21.15.45:0/1684944224 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:07:25.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.471+0000 7f381b400640 1 --2- 172.21.15.45:0/1684944224 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559c01e81390 0x559c01e8a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:25.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.471+0000 7f381b400640 1 --2- 172.21.15.45:0/1684944224 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559c01e8a890 0x559c01e96570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:25.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.472+0000 7f3813e00640 1 -- 172.21.15.45:0/1684944224 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559c01e8a890 msgr2=0x559c01e96570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:07:25.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.472+0000 7f3813e00640 1 --2- 172.21.15.45:0/1684944224 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559c01e8a890 0x559c01e96570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:25.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.472+0000 7f3813e00640 1 -- 172.21.15.45:0/1684944224 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559c01e81390 msgr2=0x559c01e8a030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:07:25.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.472+0000 7f3813e00640 1 --2- 172.21.15.45:0/1684944224 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559c01e81390 0x559c01e8a030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:25.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.472+0000 7f3813e00640 1 -- 172.21.15.45:0/1684944224 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559c01e9d700 con 0x559c01e81cb0 2024-04-16T17:07:25.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.472+0000 7f381b400640 1 --2- 172.21.15.45:0/1684944224 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559c01e8a890 0x559c01e96570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:07:25.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.472+0000 7f3813e00640 1 --2- 172.21.15.45:0/1684944224 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559c01e81cb0 0x559c01e86840 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f3804012320 tx=0x7f3804012350 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f25977805607175f server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:07:25.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.472+0000 7f380b400640 1 -- 172.21.15.45:0/1684944224 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f380402e070 con 0x559c01e81cb0 2024-04-16T17:07:25.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.472+0000 7f380b400640 1 -- 172.21.15.45:0/1684944224 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3804005590 con 0x559c01e81cb0 2024-04-16T17:07:25.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.472+0000 7f380b400640 1 -- 172.21.15.45:0/1684944224 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3804003850 con 0x559c01e81cb0 2024-04-16T17:07:25.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.473+0000 7f3824573740 1 -- 172.21.15.45:0/1684944224 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559c01e81cb0 msgr2=0x559c01e86840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:25.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.473+0000 7f3824573740 1 --2- 172.21.15.45:0/1684944224 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559c01e81cb0 0x559c01e86840 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f3804012320 tx=0x7f3804012350 comp rx=0 tx=0).stop 2024-04-16T17:07:25.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.473+0000 7f3824573740 1 -- 172.21.15.45:0/1684944224 shutdown_connections 2024-04-16T17:07:25.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.473+0000 7f3824573740 1 --2- 172.21.15.45:0/1684944224 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559c01e8a890 0x559c01e96570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:25.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.473+0000 7f3824573740 1 --2- 172.21.15.45:0/1684944224 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559c01e81cb0 0x559c01e86840 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:25.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.473+0000 7f3824573740 1 --2- 172.21.15.45:0/1684944224 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559c01e81390 0x559c01e8a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:25.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.473+0000 7f3824573740 1 -- 172.21.15.45:0/1684944224 >> 172.21.15.45:0/1684944224 conn(0x559c01e7bac0 msgr2=0x559c01e7d710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:07:25.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.473+0000 7f3824573740 1 -- 172.21.15.45:0/1684944224 shutdown_connections 2024-04-16T17:07:25.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.474+0000 7f3824573740 1 -- 172.21.15.45:0/1684944224 wait complete. 2024-04-16T17:07:25.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.475+0000 7f3824573740 1 -- start start 2024-04-16T17:07:25.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.475+0000 7f3824573740 1 Processor -- start 2024-04-16T17:07:25.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.475+0000 7f3824573740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559c01e81390 0x559c01e92ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:25.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.475+0000 7f3824573740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559c01e8a890 0x559c01e93000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:25.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.475+0000 7f3824573740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559c01e8ef50 0x559c01e8f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:25.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.475+0000 7f3824573740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f381c1b3ed0 con 0x559c01e8a890 2024-04-16T17:07:25.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.475+0000 7f3824573740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f381c1b5340 con 0x559c01e8ef50 2024-04-16T17:07:25.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.475+0000 7f3824573740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f381c1b45a0 con 0x559c01e81390 2024-04-16T17:07:25.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.475+0000 7f3824573740 10 client.0 _ll_register_callbacks cb 0x7f3804009ce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:07:25.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.476+0000 7f3813e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559c01e8a890 0x559c01e93000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:25.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.476+0000 7f3813e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559c01e8a890 0x559c01e93000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45162/0 (socket says 172.21.15.45:45162) 2024-04-16T17:07:25.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.476+0000 7f381b400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559c01e81390 0x559c01e92ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:25.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.476+0000 7f3813e00640 1 -- 172.21.15.45:0/3747293681 learned_addr learned my addr 172.21.15.45:0/3747293681 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:07:25.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.476+0000 7f381b400640 1 --2- 172.21.15.45:0/3747293681 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559c01e8ef50 0x559c01e8f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:25.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.476+0000 7f3813e00640 1 -- 172.21.15.45:0/3747293681 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559c01e81390 msgr2=0x559c01e92ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:07:25.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.476+0000 7f3813e00640 1 --2- 172.21.15.45:0/3747293681 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559c01e81390 0x559c01e92ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:25.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.476+0000 7f3813e00640 1 -- 172.21.15.45:0/3747293681 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559c01e8ef50 msgr2=0x559c01e8f3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:07:25.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.476+0000 7f3813e00640 1 --2- 172.21.15.45:0/3747293681 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559c01e8ef50 0x559c01e8f3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:25.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.476+0000 7f3813e00640 1 -- 172.21.15.45:0/3747293681 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559c01e8cc80 con 0x559c01e8a890 2024-04-16T17:07:25.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.476+0000 7f381b400640 1 --2- 172.21.15.45:0/3747293681 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559c01e81390 0x559c01e92ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:07:25.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.477+0000 7f3813e00640 1 --2- 172.21.15.45:0/3747293681 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559c01e8a890 0x559c01e93000 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f38040107d0 tx=0x7f380400fd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad5c103938ec9a7d server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:07:25.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.477+0000 7f3808c00640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-16T17:07:25.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.477+0000 7f3808c00640 1 -- 172.21.15.45:0/3747293681 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3804037060 con 0x559c01e8a890 2024-04-16T17:07:25.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.477+0000 7f3808c00640 1 -- 172.21.15.45:0/3747293681 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f3804005040 con 0x559c01e8a890 2024-04-16T17:07:25.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.477+0000 7f3824573740 1 -- 172.21.15.45:0/3747293681 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3804005590 con 0x559c01e8a890 2024-04-16T17:07:25.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.477+0000 7f3808c00640 1 -- 172.21.15.45:0/3747293681 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f380403e050 con 0x559c01e8a890 2024-04-16T17:07:25.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.477+0000 7f3824573740 10 client.15588 fetch_fsmap learned FSMap version 25 2024-04-16T17:07:25.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.477+0000 7f3824573740 1 -- 172.21.15.45:0/3747293681 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x559c01e998e0 con 0x559c01e8a890 2024-04-16T17:07:25.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.478+0000 7f3808c00640 1 -- 172.21.15.45:0/3747293681 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3804005590 con 0x559c01e8a890 2024-04-16T17:07:25.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.478+0000 7f3808c00640 1 -- 172.21.15.45:0/3747293681 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f3804003b60 con 0x559c01e8a890 2024-04-16T17:07:25.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.478+0000 7f3824573740 10 client.15588 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T17:07:25.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.478+0000 7f3824573740 10 client.15588 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:07:25.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.478+0000 7f3824573740 20 client.15588 populate_metadata read hostname 'smithi045' 2024-04-16T17:07:25.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.478+0000 7f3824573740 1 --2- 172.21.15.45:0/3747293681 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x559c01f024b0 0x559c01f048d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:25.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.478+0000 7f3824573740 4 client.15588 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-16T17:07:25.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.478+0000 7f3824573740 1 -- 172.21.15.45:0/3747293681 --> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] -- command(tid 0: {"prefix": "scrub status"}) -- 0x559c01dee9b0 con 0x559c01f024b0 2024-04-16T17:07:25.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.478+0000 7f381b400640 1 --2- 172.21.15.45:0/3747293681 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x559c01f024b0 0x559c01f048d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:25.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.479+0000 7f381b400640 1 --2- 172.21.15.45:0/3747293681 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x559c01f024b0 0x559c01f048d0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=46b92302ae6fb55d server_cookie=b7e5eff35243c0b6 in_seq=0 out_seq=0 2024-04-16T17:07:25.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.479+0000 7f3808c00640 10 client.15588 ms_handle_connect on v2:172.21.15.104:6838/3660070592 2024-04-16T17:07:25.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.479+0000 7f3808c00640 1 -- 172.21.15.45:0/3747293681 <== mds.0 v2:172.21.15.104:6838/3660070592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x559c01dee9b0 con 0x559c01f024b0 2024-04-16T17:07:25.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.479+0000 7f3808c00640 10 client.15588 handle_command_reply: tid=0 2024-04-16T17:07:25.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.479+0000 7f3824573740 1 client.15588 shutdown 2024-04-16T17:07:25.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.479+0000 7f3824573740 1 -- 172.21.15.45:0/3747293681 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559c01e8a890 msgr2=0x559c01e93000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:25.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.479+0000 7f3824573740 1 --2- 172.21.15.45:0/3747293681 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559c01e8a890 0x559c01e93000 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f38040107d0 tx=0x7f380400fd50 comp rx=0 tx=0).stop 2024-04-16T17:07:25.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.479+0000 7f3824573740 1 -- 172.21.15.45:0/3747293681 shutdown_connections 2024-04-16T17:07:25.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.479+0000 7f3824573740 1 --2- 172.21.15.45:0/3747293681 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559c01e8ef50 0x559c01e8f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:25.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.479+0000 7f3824573740 1 --2- 172.21.15.45:0/3747293681 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559c01e8a890 0x559c01e93000 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:25.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.479+0000 7f3824573740 1 --2- 172.21.15.45:0/3747293681 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x559c01f024b0 0x559c01f048d0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:25.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.479+0000 7f3824573740 1 --2- 172.21.15.45:0/3747293681 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559c01e81390 0x559c01e92ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:25.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.479+0000 7f3824573740 1 -- 172.21.15.45:0/3747293681 >> 172.21.15.45:0/3747293681 conn(0x559c01e7bac0 msgr2=0x559c01e878c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:07:25.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.479+0000 7f3808c00640 0 client.15588 ms_handle_reset on v2:172.21.15.104:6838/3660070592 2024-04-16T17:07:25.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.479+0000 7f3824573740 1 -- 172.21.15.45:0/3747293681 shutdown_connections 2024-04-16T17:07:25.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.479+0000 7f3824573740 1 -- 172.21.15.45:0/3747293681 wait complete. 2024-04-16T17:07:25.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.480+0000 7f3824573740 20 client.15588 trim_cache size 0 max 16384 2024-04-16T17:07:25.484 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-16T17:07:25.484 INFO:teuthology.orchestra.run.smithi045.stdout: "status": "no active scrubs running", 2024-04-16T17:07:25.484 INFO:teuthology.orchestra.run.smithi045.stdout: "scrubs": {} 2024-04-16T17:07:25.484 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-16T17:07:25.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.480+0000 7f381be00640 1 -- 172.21.15.45:0/1494548399 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f38000a3ec0 msgr2=0x7f38000a6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:25.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.480+0000 7f381be00640 1 --2- 172.21.15.45:0/1494548399 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f38000a3ec0 0x7f38000a6380 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f380c0efb20 tx=0x7f380c0f8530 comp rx=0 tx=0).stop 2024-04-16T17:07:25.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.480+0000 7f381be00640 1 -- 172.21.15.45:0/1494548399 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f381c076530 msgr2=0x7f381c19eda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:25.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.480+0000 7f381be00640 1 --2- 172.21.15.45:0/1494548399 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f381c076530 0x7f381c19eda0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f380c006010 tx=0x7f380c0042d0 comp rx=0 tx=0).stop 2024-04-16T17:07:25.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.480+0000 7f381be00640 1 -- 172.21.15.45:0/1494548399 shutdown_connections 2024-04-16T17:07:25.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.480+0000 7f381be00640 1 --2- 172.21.15.45:0/1494548399 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f38000a3ec0 0x7f38000a6380 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:25.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.480+0000 7f381be00640 1 --2- 172.21.15.45:0/1494548399 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f381c112d50 0x7f381c1a3660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:25.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.480+0000 7f381be00640 1 --2- 172.21.15.45:0/1494548399 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f381c076e50 0x7f381c19f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:25.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.480+0000 7f381be00640 1 --2- 172.21.15.45:0/1494548399 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f381c076530 0x7f381c19eda0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:25.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.480+0000 7f381be00640 1 -- 172.21.15.45:0/1494548399 >> 172.21.15.45:0/1494548399 conn(0x7f381c0ffb40 msgr2=0x7f381c101f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:07:25.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.481+0000 7f381be00640 1 -- 172.21.15.45:0/1494548399 shutdown_connections 2024-04-16T17:07:25.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.481+0000 7f381be00640 1 -- 172.21.15.45:0/1494548399 wait complete. 2024-04-16T17:07:25.500 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-16T17:07:25.500 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-16T17:07:25.775 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.773+0000 7f7343e00640 1 -- 172.21.15.45:0/540323830 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7344074cf0 msgr2=0x7f734410e180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:25.775 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.773+0000 7f7343e00640 1 --2- 172.21.15.45:0/540323830 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7344074cf0 0x7f734410e180 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f7330009900 tx=0x7f733002bcd0 comp rx=0 tx=0).stop 2024-04-16T17:07:25.775 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.774+0000 7f7343e00640 1 -- 172.21.15.45:0/540323830 shutdown_connections 2024-04-16T17:07:25.775 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.774+0000 7f7343e00640 1 --2- 172.21.15.45:0/540323830 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f734410e6c0 0x7f7344110ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:25.775 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.774+0000 7f7343e00640 1 --2- 172.21.15.45:0/540323830 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7344074cf0 0x7f734410e180 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:25.775 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.774+0000 7f7343e00640 1 --2- 172.21.15.45:0/540323830 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f73440743d0 0x7f73440747b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:25.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.774+0000 7f7343e00640 1 -- 172.21.15.45:0/540323830 >> 172.21.15.45:0/540323830 conn(0x7f73440fd930 msgr2=0x7f73440ffd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:07:25.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.774+0000 7f7343e00640 1 -- 172.21.15.45:0/540323830 shutdown_connections 2024-04-16T17:07:25.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.774+0000 7f7343e00640 1 -- 172.21.15.45:0/540323830 wait complete. 2024-04-16T17:07:25.777 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.775+0000 7f7343e00640 1 Processor -- start 2024-04-16T17:07:25.777 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.775+0000 7f7343e00640 1 -- start start 2024-04-16T17:07:25.777 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.775+0000 7f7343e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f73440743d0 0x7f73441a0680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:25.777 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.775+0000 7f7343e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7344074cf0 0x7f73441a0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:25.777 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.775+0000 7f7343e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f734410e6c0 0x7f734419a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:25.777 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.775+0000 7f7343e00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f7344114440 con 0x7f7344074cf0 2024-04-16T17:07:25.777 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.775+0000 7f7343e00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f73441142c0 con 0x7f73440743d0 2024-04-16T17:07:25.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.775+0000 7f7343e00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f73441145c0 con 0x7f734410e6c0 2024-04-16T17:07:25.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.776+0000 7f7342000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7344074cf0 0x7f73441a0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:25.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.776+0000 7f7343400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f734410e6c0 0x7f734419a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:25.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.776+0000 7f7342a00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f73440743d0 0x7f73441a0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:25.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.776+0000 7f7343400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f734410e6c0 0x7f734419a840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:58214/0 (socket says 172.21.15.45:58214) 2024-04-16T17:07:25.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.776+0000 7f7342000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7344074cf0 0x7f73441a0bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45184/0 (socket says 172.21.15.45:45184) 2024-04-16T17:07:25.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.776+0000 7f7343400640 1 -- 172.21.15.45:0/1790862155 learned_addr learned my addr 172.21.15.45:0/1790862155 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:07:25.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.776+0000 7f7342a00640 1 -- 172.21.15.45:0/1790862155 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f734410e6c0 msgr2=0x7f734419a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:25.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.776+0000 7f7342a00640 1 --2- 172.21.15.45:0/1790862155 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f734410e6c0 0x7f734419a840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:25.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.776+0000 7f7342a00640 1 -- 172.21.15.45:0/1790862155 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7344074cf0 msgr2=0x7f73441a0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:25.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.776+0000 7f7342a00640 1 --2- 172.21.15.45:0/1790862155 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7344074cf0 0x7f73441a0bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:25.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.776+0000 7f7342a00640 1 -- 172.21.15.45:0/1790862155 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f734419b010 con 0x7f73440743d0 2024-04-16T17:07:25.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.777+0000 7f7342a00640 1 --2- 172.21.15.45:0/1790862155 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f73440743d0 0x7f73441a0680 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f733400e9f0 tx=0x7f733400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:07:25.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.777+0000 7f733b400640 1 -- 172.21.15.45:0/1790862155 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f733400ccc0 con 0x7f73440743d0 2024-04-16T17:07:25.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.777+0000 7f7343e00640 1 -- 172.21.15.45:0/1790862155 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f734419b300 con 0x7f73440743d0 2024-04-16T17:07:25.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.777+0000 7f733b400640 1 -- 172.21.15.45:0/1790862155 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7334005020 con 0x7f73440743d0 2024-04-16T17:07:25.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.777+0000 7f7343e00640 1 -- 172.21.15.45:0/1790862155 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73441a74d0 con 0x7f73440743d0 2024-04-16T17:07:25.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.777+0000 7f733b400640 1 -- 172.21.15.45:0/1790862155 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73340049e0 con 0x7f73440743d0 2024-04-16T17:07:25.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.778+0000 7f7343e00640 1 -- 172.21.15.45:0/1790862155 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7344075a80 con 0x7f73440743d0 2024-04-16T17:07:25.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.781+0000 7f733b400640 1 -- 172.21.15.45:0/1790862155 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 25) ==== 138879+0+0 (secure 0 0 0) 0x7f7334005560 con 0x7f73440743d0 2024-04-16T17:07:25.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.782+0000 7f733b400640 1 --2- 172.21.15.45:0/1790862155 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f73280a3e40 0x7f73280a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:25.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.782+0000 7f733b400640 1 -- 172.21.15.45:0/1790862155 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f73340d0de0 con 0x7f73440743d0 2024-04-16T17:07:25.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.782+0000 7f7342000640 1 --2- 172.21.15.45:0/1790862155 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f73280a3e40 0x7f73280a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:25.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.783+0000 7f7342000640 1 --2- 172.21.15.45:0/1790862155 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f73280a3e40 0x7f73280a6300 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f733002efd0 tx=0x7f733002cf90 comp rx=0 tx=0).ready entity=mgr.25411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:07:25.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:25.786+0000 7f733b400640 1 -- 172.21.15.45:0/1790862155 <== mon.2 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f73340051d0 con 0x7f73440743d0 2024-04-16T17:07:26.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.085+0000 7f7343e00640 1 -- 172.21.15.45:0/1790862155 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7344109840 con 0x7f73440743d0 2024-04-16T17:07:26.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.087+0000 7f733b400640 1 -- 172.21.15.45:0/1790862155 <== mon.2 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11887 (secure 0 0 0) 0x7f733409ce80 con 0x7f73440743d0 2024-04-16T17:07:26.088 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-16T17:07:26.088 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/1748842443","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":1748842443},{"type":"v1","addr":"172.21.15.45:6835","nonce":1748842443}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6839/1308627051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":1308627051},{"type":"v1","addr":"172.21.15.45:6839","nonce":1308627051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/1768242489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1768242489},{"type":"v1","addr":"172.21.15.117:6833","nonce":1768242489}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6833/3558902585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":3558902585},{"type":"v1","addr":"172.21.15.104:6833","nonce":3558902585}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24478,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/1308142903","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":1308142903},{"type":"v1","addr":"172.21.15.45:6837","nonce":1308142903}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24491,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6835/3430679937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":3430679937},{"type":"v1","addr":"172.21.15.104:6835","nonce":3430679937}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-16T17:03:24.663611+0000","modified":"2024-04-16T17:06:44.485317+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24581,"mds_1":24569,"mds_2":24532},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24499":{"gid":24499,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/2224123841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2224123841},{"type":"v1","addr":"172.21.15.117:6835","nonce":2224123841}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6837/2471356815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":2471356815},{"type":"v1","addr":"172.21.15.117:6837","nonce":2471356815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24524":{"gid":24524,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6837/2647756509","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":2647756509},{"type":"v1","addr":"172.21.15.104:6837","nonce":2647756509}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24532":{"gid":24532,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":13,"addr":"172.21.15.117:6839/2378675949","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":2378675949},{"type":"v1","addr":"172.21.15.117:6839","nonce":2378675949}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24569":{"gid":24569,"name":"a","rank":1,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.45:6841/3755310822","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6840","nonce":3755310822},{"type":"v1","addr":"172.21.15.45:6841","nonce":3755310822}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24581":{"gid":24581,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.104:6839/3660070592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6838","nonce":3660070592},{"type":"v1","addr":"172.21.15.104:6839","nonce":3660070592}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24581,"qdb_cluster":[24569,24581,24532]},"id":1}]} 2024-04-16T17:07:26.088 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 25 2024-04-16T17:07:26.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.089+0000 7f7343e00640 1 -- 172.21.15.45:0/1790862155 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f73280a3e40 msgr2=0x7f73280a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:26.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.089+0000 7f7343e00640 1 --2- 172.21.15.45:0/1790862155 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f73280a3e40 0x7f73280a6300 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f733002efd0 tx=0x7f733002cf90 comp rx=0 tx=0).stop 2024-04-16T17:07:26.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.090+0000 7f7343e00640 1 -- 172.21.15.45:0/1790862155 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f73440743d0 msgr2=0x7f73441a0680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:26.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.090+0000 7f7343e00640 1 --2- 172.21.15.45:0/1790862155 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f73440743d0 0x7f73441a0680 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f733400e9f0 tx=0x7f733400eeb0 comp rx=0 tx=0).stop 2024-04-16T17:07:26.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.090+0000 7f7343e00640 1 -- 172.21.15.45:0/1790862155 shutdown_connections 2024-04-16T17:07:26.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.090+0000 7f7343e00640 1 --2- 172.21.15.45:0/1790862155 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f73280a3e40 0x7f73280a6300 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:26.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.090+0000 7f7343e00640 1 --2- 172.21.15.45:0/1790862155 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f734410e6c0 0x7f734419a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:26.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.090+0000 7f7343e00640 1 --2- 172.21.15.45:0/1790862155 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7344074cf0 0x7f73441a0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:26.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.090+0000 7f7343e00640 1 --2- 172.21.15.45:0/1790862155 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f73440743d0 0x7f73441a0680 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:26.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.090+0000 7f7343e00640 1 -- 172.21.15.45:0/1790862155 >> 172.21.15.45:0/1790862155 conn(0x7f73440fd930 msgr2=0x7f73440fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:07:26.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.090+0000 7f7343e00640 1 -- 172.21.15.45:0/1790862155 shutdown_connections 2024-04-16T17:07:26.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.090+0000 7f7343e00640 1 -- 172.21.15.45:0/1790862155 wait complete. 2024-04-16T17:07:26.109 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-16T17:07:26.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.323+0000 7f90f5800640 1 -- 172.21.15.45:0/3549513976 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f90f0074d50 msgr2=0x7f90f010e1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:26.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.324+0000 7f90f5800640 1 --2- 172.21.15.45:0/3549513976 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f90f0074d50 0x7f90f010e1a0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f90dc009900 tx=0x7f90dc02bcd0 comp rx=0 tx=0).stop 2024-04-16T17:07:26.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.324+0000 7f90f5800640 1 -- 172.21.15.45:0/3549513976 shutdown_connections 2024-04-16T17:07:26.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.324+0000 7f90f5800640 1 --2- 172.21.15.45:0/3549513976 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f90f010e6e0 0x7f90f0110b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:26.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.324+0000 7f90f5800640 1 --2- 172.21.15.45:0/3549513976 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f90f0074d50 0x7f90f010e1a0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:26.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.324+0000 7f90f5800640 1 --2- 172.21.15.45:0/3549513976 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f90f0074430 0x7f90f0074810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:26.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.324+0000 7f90f5800640 1 -- 172.21.15.45:0/3549513976 >> 172.21.15.45:0/3549513976 conn(0x7f90f00fd9d0 msgr2=0x7f90f00ffdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:07:26.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.324+0000 7f90f5800640 1 -- 172.21.15.45:0/3549513976 shutdown_connections 2024-04-16T17:07:26.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.324+0000 7f90f5800640 1 -- 172.21.15.45:0/3549513976 wait complete. 2024-04-16T17:07:26.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.325+0000 7f90f5800640 1 Processor -- start 2024-04-16T17:07:26.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.326+0000 7f90f5800640 1 -- start start 2024-04-16T17:07:26.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.326+0000 7f90f5800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f90f0074430 0x7f90f01a0670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:26.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.326+0000 7f90f5800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f90f0074d50 0x7f90f01a0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:26.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.326+0000 7f90f5800640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f90f010e6e0 0x7f90f019a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:26.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.326+0000 7f90f5800640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f90f01144a0 con 0x7f90f0074d50 2024-04-16T17:07:26.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.326+0000 7f90f5800640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f90f0114320 con 0x7f90f010e6e0 2024-04-16T17:07:26.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.326+0000 7f90f5800640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f90f0114620 con 0x7f90f0074430 2024-04-16T17:07:26.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.327+0000 7f90efe00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f90f0074430 0x7f90f01a0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:26.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.327+0000 7f90ef400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f90f0074d50 0x7f90f01a0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:26.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.327+0000 7f90efe00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f90f0074430 0x7f90f01a0670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:58234/0 (socket says 172.21.15.45:58234) 2024-04-16T17:07:26.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.327+0000 7f90efe00640 1 -- 172.21.15.45:0/2490211085 learned_addr learned my addr 172.21.15.45:0/2490211085 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:07:26.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.327+0000 7f90f4e00640 1 --2- 172.21.15.45:0/2490211085 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f90f010e6e0 0x7f90f019a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:26.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.327+0000 7f90efe00640 1 -- 172.21.15.45:0/2490211085 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f90f010e6e0 msgr2=0x7f90f019a7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:26.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.327+0000 7f90efe00640 1 --2- 172.21.15.45:0/2490211085 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f90f010e6e0 0x7f90f019a7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:26.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.327+0000 7f90efe00640 1 -- 172.21.15.45:0/2490211085 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f90f0074d50 msgr2=0x7f90f01a0bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:26.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.327+0000 7f90efe00640 1 --2- 172.21.15.45:0/2490211085 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f90f0074d50 0x7f90f01a0bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:26.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.327+0000 7f90efe00640 1 -- 172.21.15.45:0/2490211085 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90f019afb0 con 0x7f90f0074430 2024-04-16T17:07:26.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.327+0000 7f90efe00640 1 --2- 172.21.15.45:0/2490211085 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f90f0074430 0x7f90f01a0670 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f90e000ea60 tx=0x7f90e000ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:07:26.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.328+0000 7f90ecc00640 1 -- 172.21.15.45:0/2490211085 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90e000cde0 con 0x7f90f0074430 2024-04-16T17:07:26.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.328+0000 7f90ecc00640 1 -- 172.21.15.45:0/2490211085 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f90e00035f0 con 0x7f90f0074430 2024-04-16T17:07:26.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.328+0000 7f90f5800640 1 -- 172.21.15.45:0/2490211085 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90f019b2a0 con 0x7f90f0074430 2024-04-16T17:07:26.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.328+0000 7f90ecc00640 1 -- 172.21.15.45:0/2490211085 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90e0010670 con 0x7f90f0074430 2024-04-16T17:07:26.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.328+0000 7f90f5800640 1 -- 172.21.15.45:0/2490211085 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90f01a74c0 con 0x7f90f0074430 2024-04-16T17:07:26.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.332+0000 7f90ecc00640 1 -- 172.21.15.45:0/2490211085 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 25) ==== 138879+0+0 (secure 0 0 0) 0x7f90e00037a0 con 0x7f90f0074430 2024-04-16T17:07:26.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.333+0000 7f90f9b93740 1 Processor -- start 2024-04-16T17:07:26.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.333+0000 7f90f9b93740 1 -- start start 2024-04-16T17:07:26.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.333+0000 7f90f9b93740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556ba98bc720 0x556ba98bcb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:26.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.334+0000 7f90f9b93740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556ba98bd040 0x556ba98c56e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:26.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.334+0000 7f90f9b93740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556ba98c5c20 0x556ba98c8010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:26.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.334+0000 7f90f9b93740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x556ba9819020 con 0x556ba98bd040 2024-04-16T17:07:26.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.334+0000 7f90f9b93740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x556ba97bf460 con 0x556ba98bc720 2024-04-16T17:07:26.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.334+0000 7f90f9b93740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x556ba9814fe0 con 0x556ba98c5c20 2024-04-16T17:07:26.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.334+0000 7f90ecc00640 1 --2- 172.21.15.45:0/2490211085 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f90d40a3ec0 0x7f90d40a6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:26.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.334+0000 7f90efe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556ba98bd040 0x556ba98c56e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:26.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.334+0000 7f90f4e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556ba98bc720 0x556ba98bcb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:26.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.334+0000 7f90f4e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556ba98bc720 0x556ba98bcb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:35360/0 (socket says 172.21.15.45:35360) 2024-04-16T17:07:26.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.334+0000 7f90efe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556ba98bd040 0x556ba98c56e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45194/0 (socket says 172.21.15.45:45194) 2024-04-16T17:07:26.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.334+0000 7f90f4e00640 1 -- 172.21.15.45:0/1580641182 learned_addr learned my addr 172.21.15.45:0/1580641182 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:07:26.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.334+0000 7f90ef400640 1 --2- 172.21.15.45:0/1580641182 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556ba98c5c20 0x556ba98c8010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:26.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.334+0000 7f90ecc00640 1 -- 172.21.15.45:0/2490211085 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f90e00d1100 con 0x7f90f0074430 2024-04-16T17:07:26.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.334+0000 7f90f4e00640 1 --2- 172.21.15.45:0/2490211085 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f90d40a3ec0 0x7f90d40a6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:26.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.335+0000 7f90ef400640 1 -- 172.21.15.45:0/1580641182 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556ba98bc720 msgr2=0x556ba98bcb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:07:26.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.335+0000 7f90ef400640 1 --2- 172.21.15.45:0/1580641182 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556ba98bc720 0x556ba98bcb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:26.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.335+0000 7f90ef400640 1 -- 172.21.15.45:0/1580641182 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556ba98bd040 msgr2=0x556ba98c56e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:07:26.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.335+0000 7f90ef400640 1 --2- 172.21.15.45:0/1580641182 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556ba98bd040 0x556ba98c56e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:26.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.335+0000 7f90ef400640 1 -- 172.21.15.45:0/1580641182 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556ba97c7040 con 0x556ba98c5c20 2024-04-16T17:07:26.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.335+0000 7f90f4e00640 1 --2- 172.21.15.45:0/1580641182 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556ba98bc720 0x556ba98bcb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:07:26.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.335+0000 7f90ef400640 1 --2- 172.21.15.45:0/1580641182 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556ba98c5c20 0x556ba98c8010 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f90dc02c210 tx=0x7f90dc009560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=427d519658faa228 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:07:26.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.335+0000 7f90f4e00640 1 --2- 172.21.15.45:0/2490211085 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f90d40a3ec0 0x7f90d40a6380 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f90e80049d0 tx=0x7f90e8004ab0 comp rx=0 tx=0).ready entity=mgr.25411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:07:26.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.336+0000 7f90e5600640 1 -- 172.21.15.45:0/1580641182 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90dc043070 con 0x556ba98c5c20 2024-04-16T17:07:26.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.336+0000 7f90e5600640 1 -- 172.21.15.45:0/1580641182 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f90dc02c6f0 con 0x556ba98c5c20 2024-04-16T17:07:26.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.336+0000 7f90e5600640 1 -- 172.21.15.45:0/1580641182 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90dc004ae0 con 0x556ba98c5c20 2024-04-16T17:07:26.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.336+0000 7f90f9b93740 1 -- 172.21.15.45:0/1580641182 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556ba98c5c20 msgr2=0x556ba98c8010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:26.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.336+0000 7f90f9b93740 1 --2- 172.21.15.45:0/1580641182 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556ba98c5c20 0x556ba98c8010 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f90dc02c210 tx=0x7f90dc009560 comp rx=0 tx=0).stop 2024-04-16T17:07:26.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.337+0000 7f90f9b93740 1 -- 172.21.15.45:0/1580641182 shutdown_connections 2024-04-16T17:07:26.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.337+0000 7f90f9b93740 1 --2- 172.21.15.45:0/1580641182 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556ba98c5c20 0x556ba98c8010 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:26.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.337+0000 7f90f9b93740 1 --2- 172.21.15.45:0/1580641182 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556ba98bd040 0x556ba98c56e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:26.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.337+0000 7f90f9b93740 1 --2- 172.21.15.45:0/1580641182 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556ba98bc720 0x556ba98bcb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:26.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.337+0000 7f90f9b93740 1 -- 172.21.15.45:0/1580641182 >> 172.21.15.45:0/1580641182 conn(0x556ba98b6e50 msgr2=0x556ba98b7230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:07:26.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.337+0000 7f90f9b93740 1 -- 172.21.15.45:0/1580641182 shutdown_connections 2024-04-16T17:07:26.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.337+0000 7f90f9b93740 1 -- 172.21.15.45:0/1580641182 wait complete. 2024-04-16T17:07:26.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.337+0000 7f90f9b93740 1 -- start start 2024-04-16T17:07:26.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.338+0000 7f90f9b93740 1 Processor -- start 2024-04-16T17:07:26.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.338+0000 7f90f9b93740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556ba98bc720 0x556ba98c2d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:26.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.338+0000 7f90f9b93740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556ba98bd040 0x556ba98c3280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:26.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.338+0000 7f90f9b93740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556ba98c5c20 0x556ba98c37c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:26.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.338+0000 7f90f9b93740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x556ba98c89b0 con 0x556ba98c5c20 2024-04-16T17:07:26.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.339+0000 7f90f9b93740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x556ba97d2830 con 0x556ba98bc720 2024-04-16T17:07:26.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.339+0000 7f90f9b93740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x556ba98c8b30 con 0x556ba98bd040 2024-04-16T17:07:26.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.339+0000 7f90f9b93740 10 client.0 _ll_register_callbacks cb 0x556ba97da150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:07:26.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.339+0000 7f90ef400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556ba98bd040 0x556ba98c3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:26.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.339+0000 7f90ef400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556ba98bd040 0x556ba98c3280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:58244/0 (socket says 172.21.15.45:58244) 2024-04-16T17:07:26.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.339+0000 7f90ef400640 1 -- 172.21.15.45:0/2718216774 learned_addr learned my addr 172.21.15.45:0/2718216774 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:07:26.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.339+0000 7f90f4e00640 1 --2- 172.21.15.45:0/2718216774 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556ba98c5c20 0x556ba98c37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:26.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.339+0000 7f90ef400640 1 -- 172.21.15.45:0/2718216774 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556ba98bc720 msgr2=0x556ba98c2d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:07:26.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.339+0000 7f90ef400640 1 --2- 172.21.15.45:0/2718216774 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556ba98bc720 0x556ba98c2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:26.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.339+0000 7f90ef400640 1 -- 172.21.15.45:0/2718216774 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556ba98c5c20 msgr2=0x556ba98c37c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:07:26.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.339+0000 7f90ef400640 1 --2- 172.21.15.45:0/2718216774 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556ba98c5c20 0x556ba98c37c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:26.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.339+0000 7f90ef400640 1 -- 172.21.15.45:0/2718216774 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556ba97d6920 con 0x556ba98bd040 2024-04-16T17:07:26.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.339+0000 7f90f4e00640 1 --2- 172.21.15.45:0/2718216774 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556ba98c5c20 0x556ba98c37c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:07:26.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.340+0000 7f90ef400640 1 --2- 172.21.15.45:0/2718216774 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556ba98bd040 0x556ba98c3280 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f90dc005ab0 tx=0x7f90dc02bcd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1a5079064931ac06 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:07:26.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.340+0000 7f90da000640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-04-16T17:07:26.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.340+0000 7f90da000640 1 -- 172.21.15.45:0/2718216774 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90dc043040 con 0x556ba98bd040 2024-04-16T17:07:26.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.340+0000 7f90da000640 1 -- 172.21.15.45:0/2718216774 <== mon.1 v2:172.21.15.117:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f90dc02c920 con 0x556ba98bd040 2024-04-16T17:07:26.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.340+0000 7f90f9b93740 1 -- 172.21.15.45:0/2718216774 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f90dc02c6f0 con 0x556ba98bd040 2024-04-16T17:07:26.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.340+0000 7f90da000640 1 -- 172.21.15.45:0/2718216774 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f90dc057050 con 0x556ba98bd040 2024-04-16T17:07:26.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.340+0000 7f90f9b93740 10 client.25738 fetch_fsmap learned FSMap version 25 2024-04-16T17:07:26.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.340+0000 7f90f9b93740 1 -- 172.21.15.45:0/2718216774 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x556ba97a47f0 con 0x556ba98bd040 2024-04-16T17:07:26.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.341+0000 7f90da000640 1 -- 172.21.15.45:0/2718216774 <== mon.1 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f90dc02c6f0 con 0x556ba98bd040 2024-04-16T17:07:26.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.341+0000 7f90da000640 1 -- 172.21.15.45:0/2718216774 <== mon.1 v2:172.21.15.117:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f90dc003680 con 0x556ba98bd040 2024-04-16T17:07:26.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.341+0000 7f90f9b93740 10 client.25738 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T17:07:26.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.341+0000 7f90f9b93740 10 client.25738 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-16T17:07:26.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.341+0000 7f90f9b93740 20 client.25738 populate_metadata read hostname 'smithi045' 2024-04-16T17:07:26.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.341+0000 7f90f9b93740 1 --2- 172.21.15.45:0/2718216774 >> [v2:172.21.15.117:6838/2378675949,v1:172.21.15.117:6839/2378675949] conn(0x556ba98d5ec0 0x556ba98d82b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:26.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.341+0000 7f90f9b93740 4 client.25738 mds_command: new command op to 24532 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:07:26.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.341+0000 7f90f9b93740 1 -- 172.21.15.45:0/2718216774 --> [v2:172.21.15.117:6838/2378675949,v1:172.21.15.117:6839/2378675949] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556ba9826a40 con 0x556ba98d5ec0 2024-04-16T17:07:26.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.342+0000 7f90efe00640 1 --2- 172.21.15.45:0/2718216774 >> [v2:172.21.15.117:6838/2378675949,v1:172.21.15.117:6839/2378675949] conn(0x556ba98d5ec0 0x556ba98d82b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:26.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.343+0000 7f90efe00640 1 --2- 172.21.15.45:0/2718216774 >> [v2:172.21.15.117:6838/2378675949,v1:172.21.15.117:6839/2378675949] conn(0x556ba98d5ec0 0x556ba98d82b0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=199962f763fb9df4 server_cookie=f541f600a5f72b5a in_seq=0 out_seq=0 2024-04-16T17:07:26.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.343+0000 7f90da000640 10 client.25738 ms_handle_connect on v2:172.21.15.117:6838/2378675949 2024-04-16T17:07:26.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.344+0000 7f90da000640 1 -- 172.21.15.45:0/2718216774 <== mds.2 v2:172.21.15.117:6838/2378675949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556ba9826a40 con 0x556ba98d5ec0 2024-04-16T17:07:26.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.344+0000 7f90da000640 10 client.25738 handle_command_reply: tid=0 2024-04-16T17:07:26.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.344+0000 7f90f9b93740 1 client.25738 shutdown 2024-04-16T17:07:26.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.344+0000 7f90f9b93740 1 -- 172.21.15.45:0/2718216774 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556ba98bd040 msgr2=0x556ba98c3280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:26.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.344+0000 7f90f9b93740 1 --2- 172.21.15.45:0/2718216774 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556ba98bd040 0x556ba98c3280 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f90dc005ab0 tx=0x7f90dc02bcd0 comp rx=0 tx=0).stop 2024-04-16T17:07:26.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.344+0000 7f90f9b93740 1 -- 172.21.15.45:0/2718216774 shutdown_connections 2024-04-16T17:07:26.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.344+0000 7f90f9b93740 1 --2- 172.21.15.45:0/2718216774 >> [v2:172.21.15.117:6838/2378675949,v1:172.21.15.117:6839/2378675949] conn(0x556ba98d5ec0 0x556ba98d82b0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:26.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.344+0000 7f90f9b93740 1 --2- 172.21.15.45:0/2718216774 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556ba98bd040 0x556ba98c3280 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:26.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.344+0000 7f90f9b93740 1 --2- 172.21.15.45:0/2718216774 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556ba98c5c20 0x556ba98c37c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:26.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.344+0000 7f90f9b93740 1 --2- 172.21.15.45:0/2718216774 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556ba98bc720 0x556ba98c2d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:26.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.344+0000 7f90f9b93740 1 -- 172.21.15.45:0/2718216774 >> 172.21.15.45:0/2718216774 conn(0x556ba98b6e50 msgr2=0x556ba98b8ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:07:26.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.344+0000 7f90da000640 0 client.25738 ms_handle_reset on v2:172.21.15.117:6838/2378675949 2024-04-16T17:07:26.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.344+0000 7f90f9b93740 1 -- 172.21.15.45:0/2718216774 shutdown_connections 2024-04-16T17:07:26.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.344+0000 7f90f9b93740 1 -- 172.21.15.45:0/2718216774 wait complete. 2024-04-16T17:07:26.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.344+0000 7f90f9b93740 20 client.25738 trim_cache size 0 max 16384 2024-04-16T17:07:26.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.361+0000 7f90f9b93740 1 Processor -- start 2024-04-16T17:07:26.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.361+0000 7f90f9b93740 1 -- start start 2024-04-16T17:07:26.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.361+0000 7f90f9b93740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556ba98bc720 0x556ba98c5470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:26.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.361+0000 7f90f9b93740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556ba98bd040 0x556ba98d1870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:26.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.361+0000 7f90f9b93740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556ba98c5c20 0x556ba98c1ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:26.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.361+0000 7f90f9b93740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f90f01af990 con 0x556ba98bd040 2024-04-16T17:07:26.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.361+0000 7f90f9b93740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f90f01ad260 con 0x556ba98bc720 2024-04-16T17:07:26.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.361+0000 7f90f9b93740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f90f01ab090 con 0x556ba98c5c20 2024-04-16T17:07:26.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.362+0000 7f90ef400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556ba98bd040 0x556ba98d1870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:26.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.362+0000 7f90efe00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556ba98bc720 0x556ba98c5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:26.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.362+0000 7f90ef400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556ba98bd040 0x556ba98d1870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45220/0 (socket says 172.21.15.45:45220) 2024-04-16T17:07:26.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.362+0000 7f90ef400640 1 -- 172.21.15.45:0/3871919842 learned_addr learned my addr 172.21.15.45:0/3871919842 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:07:26.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.362+0000 7f90f4e00640 1 --2- 172.21.15.45:0/3871919842 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556ba98c5c20 0x556ba98c1ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:26.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.362+0000 7f90efe00640 1 -- 172.21.15.45:0/3871919842 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556ba98c5c20 msgr2=0x556ba98c1ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:07:26.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.362+0000 7f90efe00640 1 --2- 172.21.15.45:0/3871919842 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556ba98c5c20 0x556ba98c1ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:26.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.362+0000 7f90efe00640 1 -- 172.21.15.45:0/3871919842 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556ba98bd040 msgr2=0x556ba98d1870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:07:26.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.362+0000 7f90efe00640 1 --2- 172.21.15.45:0/3871919842 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556ba98bd040 0x556ba98d1870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:26.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.362+0000 7f90efe00640 1 -- 172.21.15.45:0/3871919842 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556ba98d27f0 con 0x556ba98bc720 2024-04-16T17:07:26.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.363+0000 7f90efe00640 1 --2- 172.21.15.45:0/3871919842 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556ba98bc720 0x556ba98c5470 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f90e00ce320 tx=0x7f90e00d3550 comp rx=0 tx=0).ready entity=mon.2 client_cookie=15876e57869d0387 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:07:26.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.363+0000 7f90e4c00640 1 -- 172.21.15.45:0/3871919842 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90e009caa0 con 0x556ba98bc720 2024-04-16T17:07:26.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.363+0000 7f90e4c00640 1 -- 172.21.15.45:0/3871919842 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f90e00a0e50 con 0x556ba98bc720 2024-04-16T17:07:26.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.363+0000 7f90e4c00640 1 -- 172.21.15.45:0/3871919842 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90e00d30e0 con 0x556ba98bc720 2024-04-16T17:07:26.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.364+0000 7f90f9b93740 1 -- 172.21.15.45:0/3871919842 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556ba98bc720 msgr2=0x556ba98c5470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:26.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.364+0000 7f90f9b93740 1 --2- 172.21.15.45:0/3871919842 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556ba98bc720 0x556ba98c5470 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f90e00ce320 tx=0x7f90e00d3550 comp rx=0 tx=0).stop 2024-04-16T17:07:26.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.364+0000 7f90f9b93740 1 -- 172.21.15.45:0/3871919842 shutdown_connections 2024-04-16T17:07:26.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.364+0000 7f90f9b93740 1 --2- 172.21.15.45:0/3871919842 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556ba98c5c20 0x556ba98c1ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:26.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.364+0000 7f90f9b93740 1 --2- 172.21.15.45:0/3871919842 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556ba98bd040 0x556ba98d1870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:26.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.364+0000 7f90f9b93740 1 --2- 172.21.15.45:0/3871919842 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556ba98bc720 0x556ba98c5470 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:26.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.364+0000 7f90f9b93740 1 -- 172.21.15.45:0/3871919842 >> 172.21.15.45:0/3871919842 conn(0x556ba98b6e50 msgr2=0x556ba98b8280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:07:26.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.364+0000 7f90f9b93740 1 -- 172.21.15.45:0/3871919842 shutdown_connections 2024-04-16T17:07:26.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.364+0000 7f90f9b93740 1 -- 172.21.15.45:0/3871919842 wait complete. 2024-04-16T17:07:26.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.364+0000 7f90f9b93740 1 -- start start 2024-04-16T17:07:26.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.365+0000 7f90f9b93740 1 Processor -- start 2024-04-16T17:07:26.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.365+0000 7f90f9b93740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556ba98bc720 0x556ba98bfac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:26.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.365+0000 7f90f9b93740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556ba98bd040 0x556ba98c0000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:26.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.365+0000 7f90f9b93740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556ba98c5c20 0x556ba98c0540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:26.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.365+0000 7f90f9b93740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f90f01ab720 con 0x556ba98bd040 2024-04-16T17:07:26.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.365+0000 7f90f9b93740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f90f01acb90 con 0x556ba98bc720 2024-04-16T17:07:26.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.365+0000 7f90f9b93740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f90f01abdf0 con 0x556ba98c5c20 2024-04-16T17:07:26.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.365+0000 7f90f9b93740 10 client.0 _ll_register_callbacks cb 0x7f90e0003e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:07:26.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.365+0000 7f90ef400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556ba98bd040 0x556ba98c0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:26.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.365+0000 7f90efe00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556ba98bc720 0x556ba98bfac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:26.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.365+0000 7f90efe00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556ba98bc720 0x556ba98bfac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:35390/0 (socket says 172.21.15.45:35390) 2024-04-16T17:07:26.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.365+0000 7f90efe00640 1 -- 172.21.15.45:0/3565871435 learned_addr learned my addr 172.21.15.45:0/3565871435 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:07:26.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.365+0000 7f90f4e00640 1 --2- 172.21.15.45:0/3565871435 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556ba98c5c20 0x556ba98c0540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:26.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.366+0000 7f90efe00640 1 -- 172.21.15.45:0/3565871435 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556ba98c5c20 msgr2=0x556ba98c0540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:07:26.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.366+0000 7f90efe00640 1 --2- 172.21.15.45:0/3565871435 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556ba98c5c20 0x556ba98c0540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:26.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.366+0000 7f90efe00640 1 -- 172.21.15.45:0/3565871435 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556ba98bd040 msgr2=0x556ba98c0000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:07:26.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.366+0000 7f90efe00640 1 --2- 172.21.15.45:0/3565871435 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556ba98bd040 0x556ba98c0000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:26.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.366+0000 7f90efe00640 1 -- 172.21.15.45:0/3565871435 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556ba952b3f0 con 0x556ba98bc720 2024-04-16T17:07:26.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.366+0000 7f90ef400640 1 --2- 172.21.15.45:0/3565871435 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556ba98bd040 0x556ba98c0000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:07:26.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.366+0000 7f90efe00640 1 --2- 172.21.15.45:0/3565871435 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556ba98bc720 0x556ba98bfac0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f90e0000c00 tx=0x7f90e009d260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3296176b6d5f7ec3 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:07:26.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.366+0000 7f90da000640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-04-16T17:07:26.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.366+0000 7f90da000640 1 -- 172.21.15.45:0/3565871435 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90e009ab40 con 0x556ba98bc720 2024-04-16T17:07:26.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.367+0000 7f90da000640 1 -- 172.21.15.45:0/3565871435 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f90e0099110 con 0x556ba98bc720 2024-04-16T17:07:26.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.367+0000 7f90f9b93740 1 -- 172.21.15.45:0/3565871435 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f90e00a0e50 con 0x556ba98bc720 2024-04-16T17:07:26.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.367+0000 7f90da000640 1 -- 172.21.15.45:0/3565871435 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f90e009caa0 con 0x556ba98bc720 2024-04-16T17:07:26.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.367+0000 7f90f9b93740 10 client.25742 fetch_fsmap learned FSMap version 25 2024-04-16T17:07:26.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.367+0000 7f90f9b93740 1 -- 172.21.15.45:0/3565871435 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x556ba95f57d0 con 0x556ba98bc720 2024-04-16T17:07:26.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.367+0000 7f90da000640 1 -- 172.21.15.45:0/3565871435 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f90e00a0e50 con 0x556ba98bc720 2024-04-16T17:07:26.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.367+0000 7f90da000640 1 -- 172.21.15.45:0/3565871435 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f90e00ce770 con 0x556ba98bc720 2024-04-16T17:07:26.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.367+0000 7f90f9b93740 10 client.25742 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T17:07:26.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.367+0000 7f90f9b93740 10 client.25742 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-16T17:07:26.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.367+0000 7f90f9b93740 20 client.25742 populate_metadata read hostname 'smithi045' 2024-04-16T17:07:26.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.367+0000 7f90f9b93740 1 --2- 172.21.15.45:0/3565871435 >> [v2:172.21.15.117:6838/2378675949,v1:172.21.15.117:6839/2378675949] conn(0x556ba993c770 0x556ba993eb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:26.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.367+0000 7f90f9b93740 4 client.25742 mds_command: new command op to 24532 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T17:07:26.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.367+0000 7f90f9b93740 1 -- 172.21.15.45:0/3565871435 --> [v2:172.21.15.117:6838/2378675949,v1:172.21.15.117:6839/2378675949] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556ba98208b0 con 0x556ba993c770 2024-04-16T17:07:26.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.368+0000 7f90ef400640 1 --2- 172.21.15.45:0/3565871435 >> [v2:172.21.15.117:6838/2378675949,v1:172.21.15.117:6839/2378675949] conn(0x556ba993c770 0x556ba993eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:26.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.368+0000 7f90ef400640 1 --2- 172.21.15.45:0/3565871435 >> [v2:172.21.15.117:6838/2378675949,v1:172.21.15.117:6839/2378675949] conn(0x556ba993c770 0x556ba993eb90 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6f5577aceaf3d974 server_cookie=642d869ac10ba729 in_seq=0 out_seq=0 2024-04-16T17:07:26.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.368+0000 7f90da000640 10 client.25742 ms_handle_connect on v2:172.21.15.117:6838/2378675949 2024-04-16T17:07:26.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.369+0000 7f90da000640 1 -- 172.21.15.45:0/3565871435 <== mds.2 v2:172.21.15.117:6838/2378675949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556ba98208b0 con 0x556ba993c770 2024-04-16T17:07:26.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.369+0000 7f90da000640 10 client.25742 handle_command_reply: tid=0 2024-04-16T17:07:26.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.369+0000 7f90f9b93740 1 client.25742 shutdown 2024-04-16T17:07:26.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.369+0000 7f90f9b93740 1 -- 172.21.15.45:0/3565871435 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556ba98bc720 msgr2=0x556ba98bfac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:26.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.369+0000 7f90f9b93740 1 --2- 172.21.15.45:0/3565871435 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556ba98bc720 0x556ba98bfac0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f90e0000c00 tx=0x7f90e009d260 comp rx=0 tx=0).stop 2024-04-16T17:07:26.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.369+0000 7f90f9b93740 1 -- 172.21.15.45:0/3565871435 shutdown_connections 2024-04-16T17:07:26.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.369+0000 7f90f9b93740 1 --2- 172.21.15.45:0/3565871435 >> [v2:172.21.15.117:6838/2378675949,v1:172.21.15.117:6839/2378675949] conn(0x556ba993c770 0x556ba993eb90 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:26.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.369+0000 7f90f9b93740 1 --2- 172.21.15.45:0/3565871435 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556ba98c5c20 0x556ba98c0540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:26.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.369+0000 7f90f9b93740 1 --2- 172.21.15.45:0/3565871435 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556ba98bd040 0x556ba98c0000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:26.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.369+0000 7f90f9b93740 1 --2- 172.21.15.45:0/3565871435 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556ba98bc720 0x556ba98bfac0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:26.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.369+0000 7f90da000640 0 client.25742 ms_handle_reset on v2:172.21.15.117:6838/2378675949 2024-04-16T17:07:26.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.369+0000 7f90f9b93740 1 -- 172.21.15.45:0/3565871435 >> 172.21.15.45:0/3565871435 conn(0x556ba98b6e50 msgr2=0x556ba98b81e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:07:26.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.369+0000 7f90f9b93740 1 -- 172.21.15.45:0/3565871435 shutdown_connections 2024-04-16T17:07:26.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.369+0000 7f90f9b93740 1 -- 172.21.15.45:0/3565871435 wait complete. 2024-04-16T17:07:26.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.369+0000 7f90f9b93740 20 client.25742 trim_cache size 0 max 16384 2024-04-16T17:07:26.374 INFO:teuthology.orchestra.run.smithi045.stdout:[] 2024-04-16T17:07:26.374 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:07:26 smithi045 ceph-mon[31676]: pgmap v23: 129 pgs: 129 active+clean; 1.3 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T17:07:26.374 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:07:26 smithi045 ceph-mon[31676]: from='client.? 172.21.15.45:0/1790862155' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:07:26.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.371+0000 7f90f5800640 1 -- 172.21.15.45:0/2490211085 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f90d40a3ec0 msgr2=0x7f90d40a6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:26.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.371+0000 7f90f5800640 1 --2- 172.21.15.45:0/2490211085 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f90d40a3ec0 0x7f90d40a6380 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f90e80049d0 tx=0x7f90e8004ab0 comp rx=0 tx=0).stop 2024-04-16T17:07:26.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.371+0000 7f90f5800640 1 -- 172.21.15.45:0/2490211085 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f90f0074430 msgr2=0x7f90f01a0670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:26.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.371+0000 7f90f5800640 1 --2- 172.21.15.45:0/2490211085 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f90f0074430 0x7f90f01a0670 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f90e000ea60 tx=0x7f90e000ef20 comp rx=0 tx=0).stop 2024-04-16T17:07:26.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.371+0000 7f90f5800640 1 -- 172.21.15.45:0/2490211085 shutdown_connections 2024-04-16T17:07:26.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.371+0000 7f90f5800640 1 --2- 172.21.15.45:0/2490211085 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f90d40a3ec0 0x7f90d40a6380 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:26.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.371+0000 7f90f5800640 1 --2- 172.21.15.45:0/2490211085 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f90f010e6e0 0x7f90f019a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:26.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.371+0000 7f90f5800640 1 --2- 172.21.15.45:0/2490211085 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f90f0074d50 0x7f90f01a0bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:26.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.371+0000 7f90f5800640 1 --2- 172.21.15.45:0/2490211085 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f90f0074430 0x7f90f01a0670 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:26.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.371+0000 7f90f5800640 1 -- 172.21.15.45:0/2490211085 >> 172.21.15.45:0/2490211085 conn(0x7f90f00fd9d0 msgr2=0x7f90f00fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:07:26.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.371+0000 7f90f5800640 1 -- 172.21.15.45:0/2490211085 shutdown_connections 2024-04-16T17:07:26.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.371+0000 7f90f5800640 1 -- 172.21.15.45:0/2490211085 wait complete. 2024-04-16T17:07:26.391 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-16T17:07:26.555 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:07:26 smithi117 ceph-mon[36730]: pgmap v23: 129 pgs: 129 active+clean; 1.3 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T17:07:26.555 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:07:26 smithi117 ceph-mon[36730]: from='client.? 172.21.15.45:0/1790862155' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:07:26.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.622+0000 7fe5cae00640 1 -- 172.21.15.45:0/3436743552 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe5c4102120 msgr2=0x7fe5c4102500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:26.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.622+0000 7fe5cae00640 1 --2- 172.21.15.45:0/3436743552 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe5c4102120 0x7fe5c4102500 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fe5b8009900 tx=0x7fe5b802bc50 comp rx=0 tx=0).stop 2024-04-16T17:07:26.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.622+0000 7fe5cae00640 1 -- 172.21.15.45:0/3436743552 shutdown_connections 2024-04-16T17:07:26.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.622+0000 7fe5cae00640 1 --2- 172.21.15.45:0/3436743552 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe5c41071f0 0x7fe5c410edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:26.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.622+0000 7fe5cae00640 1 --2- 172.21.15.45:0/3436743552 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe5c4102ad0 0x7fe5c4106b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:26.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.622+0000 7fe5cae00640 1 --2- 172.21.15.45:0/3436743552 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe5c4102120 0x7fe5c4102500 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:26.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.622+0000 7fe5cae00640 1 -- 172.21.15.45:0/3436743552 >> 172.21.15.45:0/3436743552 conn(0x7fe5c40fd9a0 msgr2=0x7fe5c40ffdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:07:26.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.622+0000 7fe5cae00640 1 -- 172.21.15.45:0/3436743552 shutdown_connections 2024-04-16T17:07:26.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.622+0000 7fe5cae00640 1 -- 172.21.15.45:0/3436743552 wait complete. 2024-04-16T17:07:26.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.624+0000 7fe5cae00640 1 Processor -- start 2024-04-16T17:07:26.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.624+0000 7fe5cae00640 1 -- start start 2024-04-16T17:07:26.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.624+0000 7fe5cae00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe5c4102120 0x7fe5c419e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:26.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.624+0000 7fe5cae00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe5c41071f0 0x7fe5c419eb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:26.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.624+0000 7fe5cae00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe5c41986a0 0x7fe5c4198b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:26.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.624+0000 7fe5cae00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fe5c4112300 con 0x7fe5c41071f0 2024-04-16T17:07:26.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.624+0000 7fe5cae00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fe5c4112180 con 0x7fe5c4102120 2024-04-16T17:07:26.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.624+0000 7fe5cae00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fe5c4112480 con 0x7fe5c41986a0 2024-04-16T17:07:26.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.624+0000 7fe5c9a00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe5c4102120 0x7fe5c419e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:26.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.625+0000 7fe5c9000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe5c41071f0 0x7fe5c419eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:26.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.625+0000 7fe5c9a00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe5c4102120 0x7fe5c419e5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:35398/0 (socket says 172.21.15.45:35398) 2024-04-16T17:07:26.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.625+0000 7fe5c9a00640 1 -- 172.21.15.45:0/3910520009 learned_addr learned my addr 172.21.15.45:0/3910520009 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:07:26.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.625+0000 7fe5ca400640 1 --2- 172.21.15.45:0/3910520009 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe5c41986a0 0x7fe5c4198b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:26.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.625+0000 7fe5c9a00640 1 -- 172.21.15.45:0/3910520009 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe5c41986a0 msgr2=0x7fe5c4198b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:26.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.625+0000 7fe5c9a00640 1 --2- 172.21.15.45:0/3910520009 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe5c41986a0 0x7fe5c4198b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:26.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.625+0000 7fe5c9a00640 1 -- 172.21.15.45:0/3910520009 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe5c41071f0 msgr2=0x7fe5c419eb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:26.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.625+0000 7fe5c9a00640 1 --2- 172.21.15.45:0/3910520009 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe5c41071f0 0x7fe5c419eb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:26.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.625+0000 7fe5c9a00640 1 -- 172.21.15.45:0/3910520009 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5c41992a0 con 0x7fe5c4102120 2024-04-16T17:07:26.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.626+0000 7fe5c9a00640 1 --2- 172.21.15.45:0/3910520009 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe5c4102120 0x7fe5c419e5e0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fe5b8005ab0 tx=0x7fe5b80042d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:07:26.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.626+0000 7fe5c2000640 1 -- 172.21.15.45:0/3910520009 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5b8041070 con 0x7fe5c4102120 2024-04-16T17:07:26.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.626+0000 7fe5c2000640 1 -- 172.21.15.45:0/3910520009 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe5b80044c0 con 0x7fe5c4102120 2024-04-16T17:07:26.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.626+0000 7fe5cae00640 1 -- 172.21.15.45:0/3910520009 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5c4199530 con 0x7fe5c4102120 2024-04-16T17:07:26.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.626+0000 7fe5c2000640 1 -- 172.21.15.45:0/3910520009 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5b8003d80 con 0x7fe5c4102120 2024-04-16T17:07:26.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.626+0000 7fe5cae00640 1 -- 172.21.15.45:0/3910520009 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5c41a5720 con 0x7fe5c4102120 2024-04-16T17:07:26.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.630+0000 7fe5cf05d740 1 Processor -- start 2024-04-16T17:07:26.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.630+0000 7fe5cf05d740 1 -- start start 2024-04-16T17:07:26.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.630+0000 7fe5cf05d740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eb37850f10 0x55eb378512f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:26.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.630+0000 7fe5cf05d740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55eb37851830 0x55eb37859ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:26.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.630+0000 7fe5cf05d740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55eb3785a410 0x55eb3785c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:26.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.630+0000 7fe5cf05d740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55eb377b6020 con 0x55eb37851830 2024-04-16T17:07:26.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.630+0000 7fe5cf05d740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55eb3775c460 con 0x55eb3785a410 2024-04-16T17:07:26.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.630+0000 7fe5cf05d740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55eb377b1fe0 con 0x55eb37850f10 2024-04-16T17:07:26.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.630+0000 7fe5ca400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eb37850f10 0x55eb378512f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:26.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.631+0000 7fe5ca400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eb37850f10 0x55eb378512f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:58294/0 (socket says 172.21.15.45:58294) 2024-04-16T17:07:26.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.631+0000 7fe5ca400640 1 -- 172.21.15.45:0/2666059389 learned_addr learned my addr 172.21.15.45:0/2666059389 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:07:26.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.631+0000 7fe5c9000640 1 --2- 172.21.15.45:0/2666059389 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55eb3785a410 0x55eb3785c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:26.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.631+0000 7fe5ca400640 1 -- 172.21.15.45:0/2666059389 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55eb3785a410 msgr2=0x55eb3785c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:07:26.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.631+0000 7fe5ca400640 1 --2- 172.21.15.45:0/2666059389 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55eb3785a410 0x55eb3785c800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:26.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.631+0000 7fe5ca400640 1 -- 172.21.15.45:0/2666059389 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55eb37851830 msgr2=0x55eb37859ed0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-16T17:07:26.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.631+0000 7fe5ca400640 1 --2- 172.21.15.45:0/2666059389 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55eb37851830 0x55eb37859ed0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:26.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.631+0000 7fe5ca400640 1 -- 172.21.15.45:0/2666059389 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eb37764040 con 0x55eb37850f10 2024-04-16T17:07:26.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.631+0000 7fe5c2000640 1 -- 172.21.15.45:0/3910520009 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 25) ==== 138879+0+0 (secure 0 0 0) 0x7fe5b8044070 con 0x7fe5c4102120 2024-04-16T17:07:26.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.631+0000 7fe5ca400640 1 --2- 172.21.15.45:0/2666059389 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eb37850f10 0x55eb378512f0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fe5bc009f50 tx=0x7fe5bc0118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c07d7fb4dca39d9c server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:07:26.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.632+0000 7fe5b6a00640 1 -- 172.21.15.45:0/2666059389 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5bc004d20 con 0x55eb37850f10 2024-04-16T17:07:26.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.632+0000 7fe5b6a00640 1 -- 172.21.15.45:0/2666059389 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe5bc004050 con 0x55eb37850f10 2024-04-16T17:07:26.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.632+0000 7fe5b6a00640 1 -- 172.21.15.45:0/2666059389 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5bc005280 con 0x55eb37850f10 2024-04-16T17:07:26.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.633+0000 7fe5cf05d740 1 -- 172.21.15.45:0/2666059389 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eb37850f10 msgr2=0x55eb378512f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:26.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.633+0000 7fe5cf05d740 1 --2- 172.21.15.45:0/2666059389 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eb37850f10 0x55eb378512f0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fe5bc009f50 tx=0x7fe5bc0118d0 comp rx=0 tx=0).stop 2024-04-16T17:07:26.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.633+0000 7fe5cf05d740 1 -- 172.21.15.45:0/2666059389 shutdown_connections 2024-04-16T17:07:26.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.633+0000 7fe5cf05d740 1 --2- 172.21.15.45:0/2666059389 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55eb3785a410 0x55eb3785c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:26.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.633+0000 7fe5cf05d740 1 --2- 172.21.15.45:0/2666059389 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55eb37851830 0x55eb37859ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:26.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.633+0000 7fe5cf05d740 1 --2- 172.21.15.45:0/2666059389 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eb37850f10 0x55eb378512f0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:26.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.633+0000 7fe5cf05d740 1 -- 172.21.15.45:0/2666059389 >> 172.21.15.45:0/2666059389 conn(0x55eb3784b640 msgr2=0x55eb3784ba20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:07:26.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.633+0000 7fe5cf05d740 1 -- 172.21.15.45:0/2666059389 shutdown_connections 2024-04-16T17:07:26.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.633+0000 7fe5cf05d740 1 -- 172.21.15.45:0/2666059389 wait complete. 2024-04-16T17:07:26.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.634+0000 7fe5cf05d740 1 -- start start 2024-04-16T17:07:26.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.634+0000 7fe5c2000640 1 --2- 172.21.15.45:0/3910520009 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fe5ac0a3dd0 0x7fe5ac0a6290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:26.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.634+0000 7fe5c2000640 1 -- 172.21.15.45:0/3910520009 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fe5b80f25b0 con 0x7fe5c4102120 2024-04-16T17:07:26.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.634+0000 7fe5cf05d740 1 Processor -- start 2024-04-16T17:07:26.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.634+0000 7fe5ca400640 1 --2- 172.21.15.45:0/3910520009 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fe5ac0a3dd0 0x7fe5ac0a6290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:26.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.635+0000 7fe5cf05d740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55eb37850f10 0x55eb37857560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:26.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.635+0000 7fe5cf05d740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55eb37851830 0x55eb37857aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:26.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.635+0000 7fe5cf05d740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eb3785a410 0x55eb37857fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:26.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.635+0000 7fe5cf05d740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55eb3785d200 con 0x55eb37851830 2024-04-16T17:07:26.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.635+0000 7fe5cf05d740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55eb3776f830 con 0x55eb37850f10 2024-04-16T17:07:26.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.635+0000 7fe5cf05d740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55eb3785d380 con 0x55eb3785a410 2024-04-16T17:07:26.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.635+0000 7fe5cf05d740 10 client.0 _ll_register_callbacks cb 0x55eb37777150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:07:26.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.635+0000 7fe5c9000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55eb37851830 0x55eb37857aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:26.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.635+0000 7fe5c9000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55eb37851830 0x55eb37857aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45254/0 (socket says 172.21.15.45:45254) 2024-04-16T17:07:26.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.635+0000 7fe5c9000640 1 -- 172.21.15.45:0/81526890 learned_addr learned my addr 172.21.15.45:0/81526890 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:07:26.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.635+0000 7fe5c9a00640 1 --2- 172.21.15.45:0/81526890 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55eb37850f10 0x55eb37857560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:26.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.635+0000 7fe5ca400640 1 --2- 172.21.15.45:0/3910520009 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fe5ac0a3dd0 0x7fe5ac0a6290 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fe5bc009d60 tx=0x7fe5bc004700 comp rx=0 tx=0).ready entity=mgr.25411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:07:26.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.635+0000 7fe5ca400640 1 --2- 172.21.15.45:0/81526890 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eb3785a410 0x55eb37857fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:26.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.636+0000 7fe5c9000640 1 -- 172.21.15.45:0/81526890 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eb3785a410 msgr2=0x55eb37857fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:07:26.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.636+0000 7fe5c9000640 1 --2- 172.21.15.45:0/81526890 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eb3785a410 0x55eb37857fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:26.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.636+0000 7fe5c9000640 1 -- 172.21.15.45:0/81526890 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55eb37850f10 msgr2=0x55eb37857560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:07:26.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.636+0000 7fe5c9000640 1 --2- 172.21.15.45:0/81526890 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55eb37850f10 0x55eb37857560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:26.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.636+0000 7fe5c9000640 1 -- 172.21.15.45:0/81526890 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eb37773920 con 0x55eb37851830 2024-04-16T17:07:26.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.636+0000 7fe5c9a00640 1 --2- 172.21.15.45:0/81526890 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55eb37850f10 0x55eb37857560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:07:26.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.636+0000 7fe5c9000640 1 --2- 172.21.15.45:0/81526890 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55eb37851830 0x55eb37857aa0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fe5b0012700 tx=0x7fe5b0012b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8538ac160273548e server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:07:26.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.636+0000 7fe5ab400640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-16T17:07:26.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.636+0000 7fe5ab400640 1 -- 172.21.15.45:0/81526890 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5b000f5e0 con 0x55eb37851830 2024-04-16T17:07:26.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.636+0000 7fe5cf05d740 1 -- 172.21.15.45:0/81526890 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe5bc004050 con 0x55eb37851830 2024-04-16T17:07:26.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.636+0000 7fe5ab400640 1 -- 172.21.15.45:0/81526890 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fe5b000f780 con 0x55eb37851830 2024-04-16T17:07:26.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.637+0000 7fe5ab400640 1 -- 172.21.15.45:0/81526890 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fe5b000f980 con 0x55eb37851830 2024-04-16T17:07:26.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.637+0000 7fe5cf05d740 10 client.15621 fetch_fsmap learned FSMap version 25 2024-04-16T17:07:26.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.637+0000 7fe5cf05d740 1 -- 172.21.15.45:0/81526890 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55eb377417f0 con 0x55eb37851830 2024-04-16T17:07:26.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.637+0000 7fe5ab400640 1 -- 172.21.15.45:0/81526890 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe5bc004050 con 0x55eb37851830 2024-04-16T17:07:26.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.637+0000 7fe5ab400640 1 -- 172.21.15.45:0/81526890 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fe5b0010c60 con 0x55eb37851830 2024-04-16T17:07:26.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.638+0000 7fe5cf05d740 10 client.15621 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T17:07:26.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.638+0000 7fe5cf05d740 10 client.15621 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-04-16T17:07:26.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.638+0000 7fe5cf05d740 20 client.15621 populate_metadata read hostname 'smithi045' 2024-04-16T17:07:26.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.638+0000 7fe5cf05d740 1 --2- 172.21.15.45:0/81526890 >> [v2:172.21.15.45:6840/3755310822,v1:172.21.15.45:6841/3755310822] conn(0x55eb3786ed10 0x55eb37871120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:26.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.638+0000 7fe5cf05d740 4 client.15621 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:07:26.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.638+0000 7fe5cf05d740 1 -- 172.21.15.45:0/81526890 --> [v2:172.21.15.45:6840/3755310822,v1:172.21.15.45:6841/3755310822] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55eb377c3a40 con 0x55eb3786ed10 2024-04-16T17:07:26.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.638+0000 7fe5c9a00640 1 --2- 172.21.15.45:0/81526890 >> [v2:172.21.15.45:6840/3755310822,v1:172.21.15.45:6841/3755310822] conn(0x55eb3786ed10 0x55eb37871120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:26.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.638+0000 7fe5c9a00640 1 --2- 172.21.15.45:0/81526890 >> [v2:172.21.15.45:6840/3755310822,v1:172.21.15.45:6841/3755310822] conn(0x55eb3786ed10 0x55eb37871120 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4a08192f8b3f3e4 server_cookie=a07d7ae69a2c2bf1 in_seq=0 out_seq=0 2024-04-16T17:07:26.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.638+0000 7fe5ab400640 10 client.15621 ms_handle_connect on v2:172.21.15.45:6840/3755310822 2024-04-16T17:07:26.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.639+0000 7fe5ab400640 1 -- 172.21.15.45:0/81526890 <== mds.1 v2:172.21.15.45:6840/3755310822 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55eb377c3a40 con 0x55eb3786ed10 2024-04-16T17:07:26.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.639+0000 7fe5ab400640 10 client.15621 handle_command_reply: tid=0 2024-04-16T17:07:26.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.639+0000 7fe5cf05d740 1 client.15621 shutdown 2024-04-16T17:07:26.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.639+0000 7fe5cf05d740 1 -- 172.21.15.45:0/81526890 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55eb37851830 msgr2=0x55eb37857aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:26.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.639+0000 7fe5cf05d740 1 --2- 172.21.15.45:0/81526890 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55eb37851830 0x55eb37857aa0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fe5b0012700 tx=0x7fe5b0012b00 comp rx=0 tx=0).stop 2024-04-16T17:07:26.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.640+0000 7fe5cf05d740 1 -- 172.21.15.45:0/81526890 shutdown_connections 2024-04-16T17:07:26.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.640+0000 7fe5cf05d740 1 --2- 172.21.15.45:0/81526890 >> [v2:172.21.15.45:6840/3755310822,v1:172.21.15.45:6841/3755310822] conn(0x55eb3786ed10 0x55eb37871120 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:26.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.640+0000 7fe5cf05d740 1 --2- 172.21.15.45:0/81526890 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eb3785a410 0x55eb37857fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:26.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.640+0000 7fe5cf05d740 1 --2- 172.21.15.45:0/81526890 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55eb37851830 0x55eb37857aa0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:26.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.640+0000 7fe5cf05d740 1 --2- 172.21.15.45:0/81526890 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55eb37850f10 0x55eb37857560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:26.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.640+0000 7fe5ab400640 0 client.15621 ms_handle_reset on v2:172.21.15.45:6840/3755310822 2024-04-16T17:07:26.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.640+0000 7fe5cf05d740 1 -- 172.21.15.45:0/81526890 >> 172.21.15.45:0/81526890 conn(0x55eb3784b640 msgr2=0x55eb3784d390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:07:26.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.640+0000 7fe5cf05d740 1 -- 172.21.15.45:0/81526890 shutdown_connections 2024-04-16T17:07:26.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.640+0000 7fe5cf05d740 1 -- 172.21.15.45:0/81526890 wait complete. 2024-04-16T17:07:26.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.640+0000 7fe5cf05d740 20 client.15621 trim_cache size 0 max 16384 2024-04-16T17:07:26.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.666+0000 7fe5cf05d740 1 Processor -- start 2024-04-16T17:07:26.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.666+0000 7fe5cf05d740 1 -- start start 2024-04-16T17:07:26.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.667+0000 7fe5cf05d740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55eb37850f10 0x55eb3786a580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:26.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.667+0000 7fe5cf05d740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eb37851830 0x55eb378769f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:26.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.667+0000 7fe5cf05d740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55eb3785a410 0x55eb37876f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:26.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.667+0000 7fe5cf05d740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fe5c41afa00 con 0x55eb37850f10 2024-04-16T17:07:26.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.667+0000 7fe5cf05d740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fe5c41ad2d0 con 0x55eb3785a410 2024-04-16T17:07:26.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.667+0000 7fe5cf05d740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fe5c41ab100 con 0x55eb37851830 2024-04-16T17:07:26.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.667+0000 7fe5c9a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55eb37850f10 0x55eb3786a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:26.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.667+0000 7fe5ca400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55eb3785a410 0x55eb37876f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:26.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.667+0000 7fe5c9000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eb37851830 0x55eb378769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:26.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.667+0000 7fe5c9a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55eb37850f10 0x55eb3786a580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45262/0 (socket says 172.21.15.45:45262) 2024-04-16T17:07:26.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.667+0000 7fe5c9a00640 1 -- 172.21.15.45:0/971472432 learned_addr learned my addr 172.21.15.45:0/971472432 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:07:26.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.668+0000 7fe5c9a00640 1 -- 172.21.15.45:0/971472432 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eb37851830 msgr2=0x55eb378769f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:07:26.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.668+0000 7fe5c9a00640 1 --2- 172.21.15.45:0/971472432 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eb37851830 0x55eb378769f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:26.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.668+0000 7fe5c9a00640 1 -- 172.21.15.45:0/971472432 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55eb3785a410 msgr2=0x55eb37876f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:07:26.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.668+0000 7fe5c9a00640 1 --2- 172.21.15.45:0/971472432 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55eb3785a410 0x55eb37876f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:26.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.668+0000 7fe5c9a00640 1 -- 172.21.15.45:0/971472432 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eb374c83f0 con 0x55eb37850f10 2024-04-16T17:07:26.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.668+0000 7fe5ca400640 1 --2- 172.21.15.45:0/971472432 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55eb3785a410 0x55eb37876f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:07:26.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.668+0000 7fe5c9000640 1 --2- 172.21.15.45:0/971472432 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eb37851830 0x55eb378769f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:07:26.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.668+0000 7fe5c9a00640 1 --2- 172.21.15.45:0/971472432 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55eb37850f10 0x55eb3786a580 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fe5b80be0e0 tx=0x7fe5b8049000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=783531ff77f7f5e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:07:26.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.668+0000 7fe5b6000640 1 -- 172.21.15.45:0/971472432 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5b804b040 con 0x55eb37850f10 2024-04-16T17:07:26.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.668+0000 7fe5b6000640 1 -- 172.21.15.45:0/971472432 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe5b8004c80 con 0x55eb37850f10 2024-04-16T17:07:26.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.668+0000 7fe5b6000640 1 -- 172.21.15.45:0/971472432 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5b80bc4e0 con 0x55eb37850f10 2024-04-16T17:07:26.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.669+0000 7fe5cf05d740 1 -- 172.21.15.45:0/971472432 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55eb37850f10 msgr2=0x55eb3786a580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:26.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.669+0000 7fe5cf05d740 1 --2- 172.21.15.45:0/971472432 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55eb37850f10 0x55eb3786a580 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fe5b80be0e0 tx=0x7fe5b8049000 comp rx=0 tx=0).stop 2024-04-16T17:07:26.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.669+0000 7fe5cf05d740 1 -- 172.21.15.45:0/971472432 shutdown_connections 2024-04-16T17:07:26.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.669+0000 7fe5cf05d740 1 --2- 172.21.15.45:0/971472432 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eb37851830 0x55eb378769f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:26.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.669+0000 7fe5cf05d740 1 --2- 172.21.15.45:0/971472432 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55eb3785a410 0x55eb37876f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:26.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.669+0000 7fe5cf05d740 1 --2- 172.21.15.45:0/971472432 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55eb37850f10 0x55eb3786a580 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:26.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.669+0000 7fe5cf05d740 1 -- 172.21.15.45:0/971472432 >> 172.21.15.45:0/971472432 conn(0x55eb3784b640 msgr2=0x55eb378527d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:07:26.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.669+0000 7fe5cf05d740 1 -- 172.21.15.45:0/971472432 shutdown_connections 2024-04-16T17:07:26.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.670+0000 7fe5cf05d740 1 -- 172.21.15.45:0/971472432 wait complete. 2024-04-16T17:07:26.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.670+0000 7fe5cf05d740 1 -- start start 2024-04-16T17:07:26.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.670+0000 7fe5cf05d740 1 Processor -- start 2024-04-16T17:07:26.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.671+0000 7fe5cf05d740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55eb37850f10 0x55eb37866860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:26.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.671+0000 7fe5cf05d740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55eb37851830 0x55eb37866da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:26.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.671+0000 7fe5cf05d740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eb3785a410 0x55eb37861e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:26.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.671+0000 7fe5cf05d740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fe5c41ab790 con 0x55eb37850f10 2024-04-16T17:07:26.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.671+0000 7fe5cf05d740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fe5c41acc00 con 0x55eb37851830 2024-04-16T17:07:26.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.671+0000 7fe5cf05d740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fe5c41abe60 con 0x55eb3785a410 2024-04-16T17:07:26.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.671+0000 7fe5cf05d740 10 client.0 _ll_register_callbacks cb 0x7fe5b0009850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:07:26.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.671+0000 7fe5c9a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55eb37850f10 0x55eb37866860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:26.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.671+0000 7fe5c9a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55eb37850f10 0x55eb37866860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45266/0 (socket says 172.21.15.45:45266) 2024-04-16T17:07:26.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.671+0000 7fe5c9a00640 1 -- 172.21.15.45:0/2543704286 learned_addr learned my addr 172.21.15.45:0/2543704286 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:07:26.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.671+0000 7fe5c9000640 1 --2- 172.21.15.45:0/2543704286 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55eb37851830 0x55eb37866da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:26.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.671+0000 7fe5ca400640 1 --2- 172.21.15.45:0/2543704286 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eb3785a410 0x55eb37861e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:26.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.672+0000 7fe5c9a00640 1 -- 172.21.15.45:0/2543704286 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eb3785a410 msgr2=0x55eb37861e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:07:26.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.672+0000 7fe5c9a00640 1 --2- 172.21.15.45:0/2543704286 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eb3785a410 0x55eb37861e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:26.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.672+0000 7fe5c9a00640 1 -- 172.21.15.45:0/2543704286 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55eb37851830 msgr2=0x55eb37866da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:07:26.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.672+0000 7fe5c9a00640 1 --2- 172.21.15.45:0/2543704286 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55eb37851830 0x55eb37866da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:26.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.672+0000 7fe5c9a00640 1 -- 172.21.15.45:0/2543704286 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eb375927d0 con 0x55eb37850f10 2024-04-16T17:07:26.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.672+0000 7fe5c9000640 1 --2- 172.21.15.45:0/2543704286 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55eb37851830 0x55eb37866da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:07:26.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.672+0000 7fe5c9a00640 1 --2- 172.21.15.45:0/2543704286 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55eb37850f10 0x55eb37866860 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fe5b80c67e0 tx=0x7fe5b80bdbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cca2ca1c749329b server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:07:26.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.672+0000 7fe5ab400640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-16T17:07:26.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.672+0000 7fe5ab400640 1 -- 172.21.15.45:0/2543704286 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5b804b040 con 0x55eb37850f10 2024-04-16T17:07:26.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.672+0000 7fe5ab400640 1 -- 172.21.15.45:0/2543704286 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fe5b80c20b0 con 0x55eb37850f10 2024-04-16T17:07:26.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.672+0000 7fe5cf05d740 1 -- 172.21.15.45:0/2543704286 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe5b8004c80 con 0x55eb37850f10 2024-04-16T17:07:26.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.672+0000 7fe5ab400640 1 -- 172.21.15.45:0/2543704286 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fe5b80c2450 con 0x55eb37850f10 2024-04-16T17:07:26.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.672+0000 7fe5cf05d740 10 client.15627 fetch_fsmap learned FSMap version 25 2024-04-16T17:07:26.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.672+0000 7fe5cf05d740 1 -- 172.21.15.45:0/2543704286 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55eb378d4b90 con 0x55eb37850f10 2024-04-16T17:07:26.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.673+0000 7fe5ab400640 1 -- 172.21.15.45:0/2543704286 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe5b8004c80 con 0x55eb37850f10 2024-04-16T17:07:26.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.673+0000 7fe5ab400640 1 -- 172.21.15.45:0/2543704286 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fe5b80c6880 con 0x55eb37850f10 2024-04-16T17:07:26.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.673+0000 7fe5cf05d740 10 client.15627 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T17:07:26.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.673+0000 7fe5cf05d740 10 client.15627 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-04-16T17:07:26.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.673+0000 7fe5cf05d740 20 client.15627 populate_metadata read hostname 'smithi045' 2024-04-16T17:07:26.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.673+0000 7fe5cf05d740 1 --2- 172.21.15.45:0/2543704286 >> [v2:172.21.15.45:6840/3755310822,v1:172.21.15.45:6841/3755310822] conn(0x55eb378d5060 0x55eb378d7450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:26.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.673+0000 7fe5cf05d740 4 client.15627 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T17:07:26.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.673+0000 7fe5cf05d740 1 -- 172.21.15.45:0/2543704286 --> [v2:172.21.15.45:6840/3755310822,v1:172.21.15.45:6841/3755310822] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55eb377bd8b0 con 0x55eb378d5060 2024-04-16T17:07:26.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.673+0000 7fe5c9000640 1 --2- 172.21.15.45:0/2543704286 >> [v2:172.21.15.45:6840/3755310822,v1:172.21.15.45:6841/3755310822] conn(0x55eb378d5060 0x55eb378d7450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:26.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:07:26 smithi104 ceph-mon[36457]: pgmap v23: 129 pgs: 129 active+clean; 1.3 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T17:07:26.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:07:26 smithi104 ceph-mon[36457]: from='client.? 172.21.15.45:0/1790862155' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:07:26.679 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.674+0000 7fe5c9000640 1 --2- 172.21.15.45:0/2543704286 >> [v2:172.21.15.45:6840/3755310822,v1:172.21.15.45:6841/3755310822] conn(0x55eb378d5060 0x55eb378d7450 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d216794e3a97f31 server_cookie=1fd8a71492206d64 in_seq=0 out_seq=0 2024-04-16T17:07:26.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.674+0000 7fe5ab400640 10 client.15627 ms_handle_connect on v2:172.21.15.45:6840/3755310822 2024-04-16T17:07:26.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.674+0000 7fe5ab400640 1 -- 172.21.15.45:0/2543704286 <== mds.1 v2:172.21.15.45:6840/3755310822 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55eb377bd8b0 con 0x55eb378d5060 2024-04-16T17:07:26.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.674+0000 7fe5ab400640 10 client.15627 handle_command_reply: tid=0 2024-04-16T17:07:26.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.674+0000 7fe5cf05d740 1 client.15627 shutdown 2024-04-16T17:07:26.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.674+0000 7fe5cf05d740 1 -- 172.21.15.45:0/2543704286 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55eb37850f10 msgr2=0x55eb37866860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:26.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.674+0000 7fe5cf05d740 1 --2- 172.21.15.45:0/2543704286 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55eb37850f10 0x55eb37866860 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fe5b80c67e0 tx=0x7fe5b80bdbe0 comp rx=0 tx=0).stop 2024-04-16T17:07:26.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.674+0000 7fe5cf05d740 1 -- 172.21.15.45:0/2543704286 shutdown_connections 2024-04-16T17:07:26.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.674+0000 7fe5cf05d740 1 --2- 172.21.15.45:0/2543704286 >> [v2:172.21.15.45:6840/3755310822,v1:172.21.15.45:6841/3755310822] conn(0x55eb378d5060 0x55eb378d7450 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:26.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.675+0000 7fe5cf05d740 1 --2- 172.21.15.45:0/2543704286 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eb3785a410 0x55eb37861e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:26.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.675+0000 7fe5ab400640 0 client.15627 ms_handle_reset on v2:172.21.15.45:6840/3755310822 2024-04-16T17:07:26.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.675+0000 7fe5cf05d740 1 --2- 172.21.15.45:0/2543704286 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55eb37851830 0x55eb37866da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:26.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.675+0000 7fe5cf05d740 1 --2- 172.21.15.45:0/2543704286 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55eb37850f10 0x55eb37866860 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:26.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.675+0000 7fe5cf05d740 1 -- 172.21.15.45:0/2543704286 >> 172.21.15.45:0/2543704286 conn(0x55eb3784b640 msgr2=0x55eb37857560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:07:26.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.675+0000 7fe5cf05d740 1 -- 172.21.15.45:0/2543704286 shutdown_connections 2024-04-16T17:07:26.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.675+0000 7fe5cf05d740 1 -- 172.21.15.45:0/2543704286 wait complete. 2024-04-16T17:07:26.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.675+0000 7fe5cf05d740 20 client.15627 trim_cache size 0 max 16384 2024-04-16T17:07:26.681 INFO:teuthology.orchestra.run.smithi045.stdout:[] 2024-04-16T17:07:26.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.676+0000 7fe5cae00640 1 -- 172.21.15.45:0/3910520009 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fe5ac0a3dd0 msgr2=0x7fe5ac0a6290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:26.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.676+0000 7fe5cae00640 1 --2- 172.21.15.45:0/3910520009 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fe5ac0a3dd0 0x7fe5ac0a6290 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fe5bc009d60 tx=0x7fe5bc004700 comp rx=0 tx=0).stop 2024-04-16T17:07:26.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.676+0000 7fe5cae00640 1 -- 172.21.15.45:0/3910520009 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe5c4102120 msgr2=0x7fe5c419e5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:26.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.676+0000 7fe5cae00640 1 --2- 172.21.15.45:0/3910520009 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe5c4102120 0x7fe5c419e5e0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fe5b8005ab0 tx=0x7fe5b80042d0 comp rx=0 tx=0).stop 2024-04-16T17:07:26.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.676+0000 7fe5cae00640 1 -- 172.21.15.45:0/3910520009 shutdown_connections 2024-04-16T17:07:26.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.676+0000 7fe5cae00640 1 --2- 172.21.15.45:0/3910520009 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fe5ac0a3dd0 0x7fe5ac0a6290 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:26.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.676+0000 7fe5cae00640 1 --2- 172.21.15.45:0/3910520009 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe5c41986a0 0x7fe5c4198b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:26.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.676+0000 7fe5cae00640 1 --2- 172.21.15.45:0/3910520009 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe5c41071f0 0x7fe5c419eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:26.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.676+0000 7fe5cae00640 1 --2- 172.21.15.45:0/3910520009 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe5c4102120 0x7fe5c419e5e0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:26.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.676+0000 7fe5cae00640 1 -- 172.21.15.45:0/3910520009 >> 172.21.15.45:0/3910520009 conn(0x7fe5c40fd9a0 msgr2=0x7fe5c40ffcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:07:26.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.676+0000 7fe5cae00640 1 -- 172.21.15.45:0/3910520009 shutdown_connections 2024-04-16T17:07:26.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.676+0000 7fe5cae00640 1 -- 172.21.15.45:0/3910520009 wait complete. 2024-04-16T17:07:26.704 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-16T17:07:26.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.912+0000 7f7b9ec00640 1 -- 172.21.15.45:0/3433732824 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7b9810c490 msgr2=0x7f7b9810e910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:26.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.912+0000 7f7b9ec00640 1 --2- 172.21.15.45:0/3433732824 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7b9810c490 0x7f7b9810e910 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f7b90009ca0 tx=0x7f7b9002c190 comp rx=0 tx=0).stop 2024-04-16T17:07:26.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.913+0000 7f7b9ec00640 1 -- 172.21.15.45:0/3433732824 shutdown_connections 2024-04-16T17:07:26.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.913+0000 7f7b9ec00640 1 --2- 172.21.15.45:0/3433732824 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7b981039e0 0x7f7b9810bec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:26.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.913+0000 7f7b9ec00640 1 --2- 172.21.15.45:0/3433732824 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7b9810c490 0x7f7b9810e910 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:26.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.913+0000 7f7b9ec00640 1 --2- 172.21.15.45:0/3433732824 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7b981030c0 0x7f7b981034a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:26.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.913+0000 7f7b9ec00640 1 -- 172.21.15.45:0/3433732824 >> 172.21.15.45:0/3433732824 conn(0x7f7b980fc880 msgr2=0x7f7b980feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:07:26.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.913+0000 7f7b9ec00640 1 -- 172.21.15.45:0/3433732824 shutdown_connections 2024-04-16T17:07:26.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.913+0000 7f7b9ec00640 1 -- 172.21.15.45:0/3433732824 wait complete. 2024-04-16T17:07:26.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.914+0000 7f7b9ec00640 1 Processor -- start 2024-04-16T17:07:26.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.914+0000 7f7b9ec00640 1 -- start start 2024-04-16T17:07:26.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.914+0000 7f7b9ec00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7b981030c0 0x7f7b981a0680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:26.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.914+0000 7f7b9ec00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7b981039e0 0x7f7b981a0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:26.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.914+0000 7f7b9ec00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7b9810c490 0x7f7b9819a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:26.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.914+0000 7f7b9ec00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f7b98112230 con 0x7f7b981030c0 2024-04-16T17:07:26.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.914+0000 7f7b9ec00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f7b981120b0 con 0x7f7b981039e0 2024-04-16T17:07:26.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.914+0000 7f7b9ec00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f7b981123b0 con 0x7f7b9810c490 2024-04-16T17:07:26.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.915+0000 7f7b9d800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7b981030c0 0x7f7b981a0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:26.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.915+0000 7f7b9ce00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7b981039e0 0x7f7b981a0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:26.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.915+0000 7f7b9ce00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7b981039e0 0x7f7b981a0bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:35446/0 (socket says 172.21.15.45:35446) 2024-04-16T17:07:26.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.915+0000 7f7b9d800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7b981030c0 0x7f7b981a0680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45282/0 (socket says 172.21.15.45:45282) 2024-04-16T17:07:26.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.915+0000 7f7b9ce00640 1 -- 172.21.15.45:0/545735953 learned_addr learned my addr 172.21.15.45:0/545735953 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:07:26.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.915+0000 7f7b9e200640 1 --2- 172.21.15.45:0/545735953 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7b9810c490 0x7f7b9819a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:26.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.915+0000 7f7b9ce00640 1 -- 172.21.15.45:0/545735953 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7b9810c490 msgr2=0x7f7b9819a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:26.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.915+0000 7f7b9ce00640 1 --2- 172.21.15.45:0/545735953 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7b9810c490 0x7f7b9819a740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:26.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.915+0000 7f7b9ce00640 1 -- 172.21.15.45:0/545735953 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7b981030c0 msgr2=0x7f7b981a0680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:26.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.915+0000 7f7b9ce00640 1 --2- 172.21.15.45:0/545735953 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7b981030c0 0x7f7b981a0680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:26.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.915+0000 7f7b9ce00640 1 -- 172.21.15.45:0/545735953 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b9819b000 con 0x7f7b981039e0 2024-04-16T17:07:26.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.915+0000 7f7b9d800640 1 --2- 172.21.15.45:0/545735953 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7b981030c0 0x7f7b981a0680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:07:26.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.916+0000 7f7b9ce00640 1 --2- 172.21.15.45:0/545735953 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7b981039e0 0x7f7b981a0bc0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f7b8400e9f0 tx=0x7f7b8400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:07:26.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.916+0000 7f7b96000640 1 -- 172.21.15.45:0/545735953 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b8400cd80 con 0x7f7b981039e0 2024-04-16T17:07:26.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.916+0000 7f7b96000640 1 -- 172.21.15.45:0/545735953 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b84010910 con 0x7f7b981039e0 2024-04-16T17:07:26.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.916+0000 7f7b9ec00640 1 -- 172.21.15.45:0/545735953 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b9819b2f0 con 0x7f7b981039e0 2024-04-16T17:07:26.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.916+0000 7f7b96000640 1 -- 172.21.15.45:0/545735953 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b84018990 con 0x7f7b981039e0 2024-04-16T17:07:26.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.916+0000 7f7b9ec00640 1 -- 172.21.15.45:0/545735953 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b981a7600 con 0x7f7b981039e0 2024-04-16T17:07:26.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.919+0000 7f7ba2fc6740 1 Processor -- start 2024-04-16T17:07:26.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.919+0000 7f7ba2fc6740 1 -- start start 2024-04-16T17:07:26.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.920+0000 7f7ba2fc6740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5652ba5e4f10 0x5652ba5e52f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:26.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.920+0000 7f7ba2fc6740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5652ba5e5830 0x5652ba5eded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:26.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.920+0000 7f7ba2fc6740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5652ba5ee410 0x5652ba5f0800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:26.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.920+0000 7f7ba2fc6740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5652ba54a020 con 0x5652ba5e5830 2024-04-16T17:07:26.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.920+0000 7f7ba2fc6740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5652ba4f0460 con 0x5652ba5ee410 2024-04-16T17:07:26.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.920+0000 7f7ba2fc6740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5652ba545fe0 con 0x5652ba5e4f10 2024-04-16T17:07:26.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.920+0000 7f7b9e200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5652ba5e4f10 0x5652ba5e52f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:26.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.920+0000 7f7b9d800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5652ba5e5830 0x5652ba5eded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:26.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.920+0000 7f7b9e200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5652ba5e4f10 0x5652ba5e52f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:58332/0 (socket says 172.21.15.45:58332) 2024-04-16T17:07:26.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.920+0000 7f7b9e200640 1 -- 172.21.15.45:0/1205387999 learned_addr learned my addr 172.21.15.45:0/1205387999 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:07:26.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.920+0000 7f7b96000640 1 -- 172.21.15.45:0/545735953 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 25) ==== 138879+0+0 (secure 0 0 0) 0x7f7b84010ac0 con 0x7f7b981039e0 2024-04-16T17:07:26.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.921+0000 7f7b9e200640 1 -- 172.21.15.45:0/1205387999 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5652ba5ee410 msgr2=0x5652ba5f0800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:07:26.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.921+0000 7f7b9e200640 1 --2- 172.21.15.45:0/1205387999 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5652ba5ee410 0x5652ba5f0800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:26.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.921+0000 7f7b9e200640 1 -- 172.21.15.45:0/1205387999 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5652ba5e5830 msgr2=0x5652ba5eded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:07:26.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.921+0000 7f7b9e200640 1 --2- 172.21.15.45:0/1205387999 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5652ba5e5830 0x5652ba5eded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:26.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.921+0000 7f7b9e200640 1 -- 172.21.15.45:0/1205387999 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5652ba4f8040 con 0x5652ba5e4f10 2024-04-16T17:07:26.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.921+0000 7f7b9e200640 1 --2- 172.21.15.45:0/1205387999 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5652ba5e4f10 0x5652ba5e52f0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f7b90009ca0 tx=0x7f7b90009900 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e1ddfe7ee36a05e2 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:07:26.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.921+0000 7f7b8aa00640 1 -- 172.21.15.45:0/1205387999 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b90043070 con 0x5652ba5e4f10 2024-04-16T17:07:26.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.921+0000 7f7b8aa00640 1 -- 172.21.15.45:0/1205387999 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b90003e50 con 0x5652ba5e4f10 2024-04-16T17:07:26.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.921+0000 7f7b8aa00640 1 -- 172.21.15.45:0/1205387999 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b90039040 con 0x5652ba5e4f10 2024-04-16T17:07:26.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.922+0000 7f7b96000640 1 --2- 172.21.15.45:0/545735953 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f7b800a3f40 0x7f7b800a6400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:26.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.922+0000 7f7ba2fc6740 1 -- 172.21.15.45:0/1205387999 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5652ba5e4f10 msgr2=0x5652ba5e52f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:26.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.922+0000 7f7ba2fc6740 1 --2- 172.21.15.45:0/1205387999 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5652ba5e4f10 0x5652ba5e52f0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f7b90009ca0 tx=0x7f7b90009900 comp rx=0 tx=0).stop 2024-04-16T17:07:26.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.922+0000 7f7b96000640 1 -- 172.21.15.45:0/545735953 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f7b840d1040 con 0x7f7b981039e0 2024-04-16T17:07:26.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.923+0000 7f7ba2fc6740 1 -- 172.21.15.45:0/1205387999 shutdown_connections 2024-04-16T17:07:26.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.923+0000 7f7b9d800640 1 --2- 172.21.15.45:0/545735953 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f7b800a3f40 0x7f7b800a6400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:26.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.923+0000 7f7ba2fc6740 1 --2- 172.21.15.45:0/1205387999 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5652ba5ee410 0x5652ba5f0800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:26.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.923+0000 7f7ba2fc6740 1 --2- 172.21.15.45:0/1205387999 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5652ba5e5830 0x5652ba5eded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:26.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.923+0000 7f7ba2fc6740 1 --2- 172.21.15.45:0/1205387999 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5652ba5e4f10 0x5652ba5e52f0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:26.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.923+0000 7f7ba2fc6740 1 -- 172.21.15.45:0/1205387999 >> 172.21.15.45:0/1205387999 conn(0x5652ba5df640 msgr2=0x5652ba5dfa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:07:26.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.923+0000 7f7ba2fc6740 1 -- 172.21.15.45:0/1205387999 shutdown_connections 2024-04-16T17:07:26.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.923+0000 7f7ba2fc6740 1 -- 172.21.15.45:0/1205387999 wait complete. 2024-04-16T17:07:26.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.923+0000 7f7b9d800640 1 --2- 172.21.15.45:0/545735953 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f7b800a3f40 0x7f7b800a6400 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x5652ba5e49e0 tx=0x7f7b8c005f90 comp rx=0 tx=0).ready entity=mgr.25411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:07:26.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.923+0000 7f7ba2fc6740 1 -- start start 2024-04-16T17:07:26.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.924+0000 7f7ba2fc6740 1 Processor -- start 2024-04-16T17:07:26.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.924+0000 7f7ba2fc6740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5652ba5e4f10 0x5652ba5eb490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:26.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.924+0000 7f7ba2fc6740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5652ba5e5830 0x5652ba5eb9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:26.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.924+0000 7f7ba2fc6740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5652ba5ee410 0x5652ba5ebf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:26.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.924+0000 7f7ba2fc6740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5652ba5f11a0 con 0x5652ba5ee410 2024-04-16T17:07:26.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.925+0000 7f7ba2fc6740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5652ba503830 con 0x5652ba5e5830 2024-04-16T17:07:26.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.925+0000 7f7ba2fc6740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5652ba5f1320 con 0x5652ba5e4f10 2024-04-16T17:07:26.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.925+0000 7f7ba2fc6740 10 client.0 _ll_register_callbacks cb 0x5652ba50b150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:07:26.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.925+0000 7f7b9d800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5652ba5e4f10 0x5652ba5eb490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:26.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.925+0000 7f7b9ce00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5652ba5e5830 0x5652ba5eb9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:26.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.925+0000 7f7b9d800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5652ba5e4f10 0x5652ba5eb490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:58338/0 (socket says 172.21.15.45:58338) 2024-04-16T17:07:26.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.925+0000 7f7b9d800640 1 -- 172.21.15.45:0/3910115598 learned_addr learned my addr 172.21.15.45:0/3910115598 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:07:26.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.925+0000 7f7b9d800640 1 -- 172.21.15.45:0/3910115598 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5652ba5e5830 msgr2=0x5652ba5eb9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:07:26.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.925+0000 7f7b9d800640 1 --2- 172.21.15.45:0/3910115598 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5652ba5e5830 0x5652ba5eb9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:26.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.925+0000 7f7b9e200640 1 --2- 172.21.15.45:0/3910115598 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5652ba5ee410 0x5652ba5ebf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:26.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.925+0000 7f7b9d800640 1 -- 172.21.15.45:0/3910115598 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5652ba5ee410 msgr2=0x5652ba5ebf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:07:26.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.926+0000 7f7b9d800640 1 --2- 172.21.15.45:0/3910115598 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5652ba5ee410 0x5652ba5ebf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:26.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.926+0000 7f7b9d800640 1 -- 172.21.15.45:0/3910115598 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5652ba507920 con 0x5652ba5e4f10 2024-04-16T17:07:26.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.926+0000 7f7b9e200640 1 --2- 172.21.15.45:0/3910115598 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5652ba5ee410 0x5652ba5ebf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:07:26.930 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.926+0000 7f7b9d800640 1 --2- 172.21.15.45:0/3910115598 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5652ba5e4f10 0x5652ba5eb490 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f7b8c003670 tx=0x7f7b8c004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f1d0f2fd52c1bff5 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:07:26.930 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.926+0000 7f7b7f400640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-04-16T17:07:26.930 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.926+0000 7f7b7f400640 1 -- 172.21.15.45:0/3910115598 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b8c00f910 con 0x5652ba5e4f10 2024-04-16T17:07:26.930 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.926+0000 7f7b7f400640 1 -- 172.21.15.45:0/3910115598 <== mon.1 v2:172.21.15.117:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f7b8c00fab0 con 0x5652ba5e4f10 2024-04-16T17:07:26.930 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.926+0000 7f7ba2fc6740 1 -- 172.21.15.45:0/3910115598 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7b90003e50 con 0x5652ba5e4f10 2024-04-16T17:07:26.930 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.927+0000 7f7b7f400640 1 -- 172.21.15.45:0/3910115598 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f7b8c032050 con 0x5652ba5e4f10 2024-04-16T17:07:26.930 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.927+0000 7f7ba2fc6740 10 client.25774 fetch_fsmap learned FSMap version 25 2024-04-16T17:07:26.930 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.927+0000 7f7ba2fc6740 1 -- 172.21.15.45:0/3910115598 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5652ba4d57f0 con 0x5652ba5e4f10 2024-04-16T17:07:26.931 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.927+0000 7f7b7f400640 1 -- 172.21.15.45:0/3910115598 <== mon.1 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b90003e50 con 0x5652ba5e4f10 2024-04-16T17:07:26.931 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.927+0000 7f7b7f400640 1 -- 172.21.15.45:0/3910115598 <== mon.1 v2:172.21.15.117:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f7b8c027020 con 0x5652ba5e4f10 2024-04-16T17:07:26.931 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.928+0000 7f7ba2fc6740 10 client.25774 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T17:07:26.931 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.928+0000 7f7ba2fc6740 10 client.25774 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:07:26.931 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.928+0000 7f7ba2fc6740 20 client.25774 populate_metadata read hostname 'smithi045' 2024-04-16T17:07:26.931 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.928+0000 7f7ba2fc6740 1 --2- 172.21.15.45:0/3910115598 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x5652ba606ff0 0x5652ba609440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:26.931 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.928+0000 7f7ba2fc6740 4 client.25774 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:07:26.931 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.928+0000 7f7ba2fc6740 1 -- 172.21.15.45:0/3910115598 --> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5652ba557a40 con 0x5652ba606ff0 2024-04-16T17:07:26.931 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.928+0000 7f7b9ce00640 1 --2- 172.21.15.45:0/3910115598 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x5652ba606ff0 0x5652ba609440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:26.931 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.929+0000 7f7b9ce00640 1 --2- 172.21.15.45:0/3910115598 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x5652ba606ff0 0x5652ba609440 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d0f22d99535a21b0 server_cookie=840389d3788f6498 in_seq=0 out_seq=0 2024-04-16T17:07:26.931 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.929+0000 7f7b7f400640 10 client.25774 ms_handle_connect on v2:172.21.15.104:6838/3660070592 2024-04-16T17:07:26.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.929+0000 7f7b7f400640 1 -- 172.21.15.45:0/3910115598 <== mds.0 v2:172.21.15.104:6838/3660070592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5652ba557a40 con 0x5652ba606ff0 2024-04-16T17:07:26.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.929+0000 7f7b7f400640 10 client.25774 handle_command_reply: tid=0 2024-04-16T17:07:26.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.930+0000 7f7ba2fc6740 1 client.25774 shutdown 2024-04-16T17:07:26.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.930+0000 7f7ba2fc6740 1 -- 172.21.15.45:0/3910115598 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5652ba5e4f10 msgr2=0x5652ba5eb490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:26.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.930+0000 7f7ba2fc6740 1 --2- 172.21.15.45:0/3910115598 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5652ba5e4f10 0x5652ba5eb490 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f7b8c003670 tx=0x7f7b8c004730 comp rx=0 tx=0).stop 2024-04-16T17:07:26.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.930+0000 7f7ba2fc6740 1 -- 172.21.15.45:0/3910115598 shutdown_connections 2024-04-16T17:07:26.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.930+0000 7f7ba2fc6740 1 --2- 172.21.15.45:0/3910115598 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5652ba5ee410 0x5652ba5ebf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:26.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.930+0000 7f7ba2fc6740 1 --2- 172.21.15.45:0/3910115598 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5652ba5e5830 0x5652ba5eb9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:26.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.930+0000 7f7ba2fc6740 1 --2- 172.21.15.45:0/3910115598 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x5652ba606ff0 0x5652ba609440 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:26.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.930+0000 7f7ba2fc6740 1 --2- 172.21.15.45:0/3910115598 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5652ba5e4f10 0x5652ba5eb490 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:26.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.930+0000 7f7ba2fc6740 1 -- 172.21.15.45:0/3910115598 >> 172.21.15.45:0/3910115598 conn(0x5652ba5df640 msgr2=0x5652ba5e1390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:07:26.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.930+0000 7f7b7f400640 0 client.25774 ms_handle_reset on v2:172.21.15.104:6838/3660070592 2024-04-16T17:07:26.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.930+0000 7f7ba2fc6740 1 -- 172.21.15.45:0/3910115598 shutdown_connections 2024-04-16T17:07:26.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.930+0000 7f7ba2fc6740 1 -- 172.21.15.45:0/3910115598 wait complete. 2024-04-16T17:07:26.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.930+0000 7f7ba2fc6740 20 client.25774 trim_cache size 0 max 16384 2024-04-16T17:07:26.947 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.945+0000 7f7ba2fc6740 1 Processor -- start 2024-04-16T17:07:26.947 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.946+0000 7f7ba2fc6740 1 -- start start 2024-04-16T17:07:26.947 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.946+0000 7f7ba2fc6740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5652ba5e4f10 0x5652ba5edbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:26.947 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.946+0000 7f7ba2fc6740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5652ba5e5830 0x5652ba5ead70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:26.947 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.946+0000 7f7ba2fc6740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5652ba5ee410 0x5652ba5dde70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:26.947 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.946+0000 7f7ba2fc6740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f7b981af930 con 0x5652ba5e4f10 2024-04-16T17:07:26.947 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.946+0000 7f7ba2fc6740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f7b981ad200 con 0x5652ba5ee410 2024-04-16T17:07:26.947 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.946+0000 7f7ba2fc6740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f7b981ab090 con 0x5652ba5e5830 2024-04-16T17:07:26.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.946+0000 7f7b9d800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5652ba5e4f10 0x5652ba5edbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:26.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.946+0000 7f7b9d800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5652ba5e4f10 0x5652ba5edbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45310/0 (socket says 172.21.15.45:45310) 2024-04-16T17:07:26.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.946+0000 7f7b9d800640 1 -- 172.21.15.45:0/2484487976 learned_addr learned my addr 172.21.15.45:0/2484487976 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:07:26.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.946+0000 7f7b9e200640 1 --2- 172.21.15.45:0/2484487976 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5652ba5ee410 0x5652ba5dde70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:26.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.947+0000 7f7b9ce00640 1 --2- 172.21.15.45:0/2484487976 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5652ba5e5830 0x5652ba5ead70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:26.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.947+0000 7f7b9e200640 1 -- 172.21.15.45:0/2484487976 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5652ba5e5830 msgr2=0x5652ba5ead70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:07:26.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.947+0000 7f7b9e200640 1 --2- 172.21.15.45:0/2484487976 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5652ba5e5830 0x5652ba5ead70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:26.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.947+0000 7f7b9e200640 1 -- 172.21.15.45:0/2484487976 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5652ba5e4f10 msgr2=0x5652ba5edbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:07:26.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.947+0000 7f7b9e200640 1 --2- 172.21.15.45:0/2484487976 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5652ba5e4f10 0x5652ba5edbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:26.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.947+0000 7f7b9e200640 1 -- 172.21.15.45:0/2484487976 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5652ba5e5660 con 0x5652ba5ee410 2024-04-16T17:07:26.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.947+0000 7f7b9d800640 1 --2- 172.21.15.45:0/2484487976 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5652ba5e4f10 0x5652ba5edbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:07:26.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.947+0000 7f7b9e200640 1 --2- 172.21.15.45:0/2484487976 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5652ba5ee410 0x5652ba5dde70 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f7b9002c6d0 tx=0x7f7b90000f40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=98b7e73f4080bf0d server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:07:26.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.947+0000 7f7b8a000640 1 -- 172.21.15.45:0/2484487976 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b90039070 con 0x5652ba5ee410 2024-04-16T17:07:26.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.948+0000 7f7b8a000640 1 -- 172.21.15.45:0/2484487976 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b9004d070 con 0x5652ba5ee410 2024-04-16T17:07:26.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.948+0000 7f7b8a000640 1 -- 172.21.15.45:0/2484487976 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b90043050 con 0x5652ba5ee410 2024-04-16T17:07:26.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.948+0000 7f7ba2fc6740 1 -- 172.21.15.45:0/2484487976 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5652ba5ee410 msgr2=0x5652ba5dde70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:26.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.948+0000 7f7ba2fc6740 1 --2- 172.21.15.45:0/2484487976 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5652ba5ee410 0x5652ba5dde70 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f7b9002c6d0 tx=0x7f7b90000f40 comp rx=0 tx=0).stop 2024-04-16T17:07:26.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.948+0000 7f7ba2fc6740 1 -- 172.21.15.45:0/2484487976 shutdown_connections 2024-04-16T17:07:26.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.948+0000 7f7ba2fc6740 1 --2- 172.21.15.45:0/2484487976 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5652ba5e5830 0x5652ba5ead70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:26.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.948+0000 7f7ba2fc6740 1 --2- 172.21.15.45:0/2484487976 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5652ba5ee410 0x5652ba5dde70 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:26.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.948+0000 7f7ba2fc6740 1 --2- 172.21.15.45:0/2484487976 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5652ba5e4f10 0x5652ba5edbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:26.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.948+0000 7f7ba2fc6740 1 -- 172.21.15.45:0/2484487976 >> 172.21.15.45:0/2484487976 conn(0x5652ba5df640 msgr2=0x5652ba5e12b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:07:26.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.949+0000 7f7ba2fc6740 1 -- 172.21.15.45:0/2484487976 shutdown_connections 2024-04-16T17:07:26.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.949+0000 7f7ba2fc6740 1 -- 172.21.15.45:0/2484487976 wait complete. 2024-04-16T17:07:26.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.949+0000 7f7ba2fc6740 1 -- start start 2024-04-16T17:07:26.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.949+0000 7f7ba2fc6740 1 Processor -- start 2024-04-16T17:07:26.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.950+0000 7f7ba2fc6740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5652ba5e4f10 0x5652ba5fa840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:26.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.950+0000 7f7ba2fc6740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5652ba5e5830 0x5652ba5fad80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:26.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.950+0000 7f7ba2fc6740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5652ba5ee410 0x5652ba5fb2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:26.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.950+0000 7f7ba2fc6740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f7b981ab6e0 con 0x5652ba5e4f10 2024-04-16T17:07:26.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.950+0000 7f7ba2fc6740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f7b981acb30 con 0x5652ba5ee410 2024-04-16T17:07:26.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.950+0000 7f7ba2fc6740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f7b981abd90 con 0x5652ba5e5830 2024-04-16T17:07:26.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.950+0000 7f7ba2fc6740 10 client.0 _ll_register_callbacks cb 0x7f7b8c004040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:07:26.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.950+0000 7f7b9d800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5652ba5e4f10 0x5652ba5fa840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:26.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.950+0000 7f7b9d800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5652ba5e4f10 0x5652ba5fa840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45326/0 (socket says 172.21.15.45:45326) 2024-04-16T17:07:26.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.950+0000 7f7b9d800640 1 -- 172.21.15.45:0/2756311237 learned_addr learned my addr 172.21.15.45:0/2756311237 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:07:26.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.950+0000 7f7b9ce00640 1 --2- 172.21.15.45:0/2756311237 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5652ba5e5830 0x5652ba5fad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:26.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.950+0000 7f7b9e200640 1 --2- 172.21.15.45:0/2756311237 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5652ba5ee410 0x5652ba5fb2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:26.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.951+0000 7f7b9d800640 1 -- 172.21.15.45:0/2756311237 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5652ba5e5830 msgr2=0x5652ba5fad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:07:26.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.951+0000 7f7b9d800640 1 --2- 172.21.15.45:0/2756311237 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5652ba5e5830 0x5652ba5fad80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:26.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.951+0000 7f7b9d800640 1 -- 172.21.15.45:0/2756311237 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5652ba5ee410 msgr2=0x5652ba5fb2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:07:26.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.951+0000 7f7b9d800640 1 --2- 172.21.15.45:0/2756311237 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5652ba5ee410 0x5652ba5fb2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:26.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.951+0000 7f7b9d800640 1 -- 172.21.15.45:0/2756311237 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5652ba25c3f0 con 0x5652ba5e4f10 2024-04-16T17:07:26.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.951+0000 7f7b9d800640 1 --2- 172.21.15.45:0/2756311237 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5652ba5e4f10 0x5652ba5fa840 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f7b8c003670 tx=0x7f7b8c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54caf146dc227119 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:07:26.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.951+0000 7f7b7f400640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-16T17:07:26.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.951+0000 7f7b7f400640 1 -- 172.21.15.45:0/2756311237 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b8c03c030 con 0x5652ba5e4f10 2024-04-16T17:07:26.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.951+0000 7f7b7f400640 1 -- 172.21.15.45:0/2756311237 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f7b8c02f070 con 0x5652ba5e4f10 2024-04-16T17:07:26.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.951+0000 7f7ba2fc6740 1 -- 172.21.15.45:0/2756311237 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7b9004d070 con 0x5652ba5e4f10 2024-04-16T17:07:26.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.952+0000 7f7b7f400640 1 -- 172.21.15.45:0/2756311237 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f7b8c01f710 con 0x5652ba5e4f10 2024-04-16T17:07:26.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.952+0000 7f7ba2fc6740 10 client.15642 fetch_fsmap learned FSMap version 25 2024-04-16T17:07:26.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.952+0000 7f7ba2fc6740 1 -- 172.21.15.45:0/2756311237 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5652ba3267d0 con 0x5652ba5e4f10 2024-04-16T17:07:26.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.952+0000 7f7b7f400640 1 -- 172.21.15.45:0/2756311237 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b9004d070 con 0x5652ba5e4f10 2024-04-16T17:07:26.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.952+0000 7f7b7f400640 1 -- 172.21.15.45:0/2756311237 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f7b8c027040 con 0x5652ba5e4f10 2024-04-16T17:07:26.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.952+0000 7f7ba2fc6740 10 client.15642 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T17:07:26.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.952+0000 7f7ba2fc6740 10 client.15642 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:07:26.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.952+0000 7f7ba2fc6740 20 client.15642 populate_metadata read hostname 'smithi045' 2024-04-16T17:07:26.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.952+0000 7f7ba2fc6740 1 --2- 172.21.15.45:0/2756311237 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x5652ba66dc70 0x5652ba670090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:26.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.952+0000 7f7ba2fc6740 4 client.15642 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T17:07:26.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.952+0000 7f7ba2fc6740 1 -- 172.21.15.45:0/2756311237 --> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5652ba5ded20 con 0x5652ba66dc70 2024-04-16T17:07:26.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.953+0000 7f7b9ce00640 1 --2- 172.21.15.45:0/2756311237 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x5652ba66dc70 0x5652ba670090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:26.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.953+0000 7f7b9ce00640 1 --2- 172.21.15.45:0/2756311237 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x5652ba66dc70 0x5652ba670090 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2d21ca495648c757 server_cookie=540d61f224fe82c4 in_seq=0 out_seq=0 2024-04-16T17:07:26.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.953+0000 7f7b7f400640 10 client.15642 ms_handle_connect on v2:172.21.15.104:6838/3660070592 2024-04-16T17:07:26.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.953+0000 7f7b7f400640 1 -- 172.21.15.45:0/2756311237 <== mds.0 v2:172.21.15.104:6838/3660070592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5652ba5ded20 con 0x5652ba66dc70 2024-04-16T17:07:26.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.953+0000 7f7b7f400640 10 client.15642 handle_command_reply: tid=0 2024-04-16T17:07:26.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.953+0000 7f7ba2fc6740 1 client.15642 shutdown 2024-04-16T17:07:26.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.954+0000 7f7ba2fc6740 1 -- 172.21.15.45:0/2756311237 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5652ba5e4f10 msgr2=0x5652ba5fa840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:26.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.954+0000 7f7ba2fc6740 1 --2- 172.21.15.45:0/2756311237 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5652ba5e4f10 0x5652ba5fa840 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f7b8c003670 tx=0x7f7b8c017000 comp rx=0 tx=0).stop 2024-04-16T17:07:26.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.954+0000 7f7ba2fc6740 1 -- 172.21.15.45:0/2756311237 shutdown_connections 2024-04-16T17:07:26.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.954+0000 7f7ba2fc6740 1 --2- 172.21.15.45:0/2756311237 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x5652ba66dc70 0x5652ba670090 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:26.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.954+0000 7f7ba2fc6740 1 --2- 172.21.15.45:0/2756311237 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5652ba5e5830 0x5652ba5fad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:26.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.954+0000 7f7ba2fc6740 1 --2- 172.21.15.45:0/2756311237 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5652ba5ee410 0x5652ba5fb2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:26.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.954+0000 7f7ba2fc6740 1 --2- 172.21.15.45:0/2756311237 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5652ba5e4f10 0x5652ba5fa840 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:26.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.954+0000 7f7ba2fc6740 1 -- 172.21.15.45:0/2756311237 >> 172.21.15.45:0/2756311237 conn(0x5652ba5df640 msgr2=0x5652ba5e0a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:07:26.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.954+0000 7f7b7f400640 0 client.15642 ms_handle_reset on v2:172.21.15.104:6838/3660070592 2024-04-16T17:07:26.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.954+0000 7f7ba2fc6740 1 -- 172.21.15.45:0/2756311237 shutdown_connections 2024-04-16T17:07:26.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.954+0000 7f7ba2fc6740 1 -- 172.21.15.45:0/2756311237 wait complete. 2024-04-16T17:07:26.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.954+0000 7f7ba2fc6740 20 client.15642 trim_cache size 0 max 16384 2024-04-16T17:07:26.959 INFO:teuthology.orchestra.run.smithi045.stdout:[] 2024-04-16T17:07:26.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.955+0000 7f7b9ec00640 1 -- 172.21.15.45:0/545735953 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f7b800a3f40 msgr2=0x7f7b800a6400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:26.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.955+0000 7f7b9ec00640 1 --2- 172.21.15.45:0/545735953 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f7b800a3f40 0x7f7b800a6400 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x5652ba5e49e0 tx=0x7f7b8c005f90 comp rx=0 tx=0).stop 2024-04-16T17:07:26.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.955+0000 7f7b9ec00640 1 -- 172.21.15.45:0/545735953 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7b981039e0 msgr2=0x7f7b981a0bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:26.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.955+0000 7f7b9ec00640 1 --2- 172.21.15.45:0/545735953 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7b981039e0 0x7f7b981a0bc0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f7b8400e9f0 tx=0x7f7b8400eeb0 comp rx=0 tx=0).stop 2024-04-16T17:07:26.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.955+0000 7f7b9ec00640 1 -- 172.21.15.45:0/545735953 shutdown_connections 2024-04-16T17:07:26.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.955+0000 7f7b9ec00640 1 --2- 172.21.15.45:0/545735953 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f7b800a3f40 0x7f7b800a6400 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:26.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.955+0000 7f7b9ec00640 1 --2- 172.21.15.45:0/545735953 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7b9810c490 0x7f7b9819a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:26.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.955+0000 7f7b9ec00640 1 --2- 172.21.15.45:0/545735953 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7b981039e0 0x7f7b981a0bc0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:26.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.955+0000 7f7b9ec00640 1 --2- 172.21.15.45:0/545735953 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7b981030c0 0x7f7b981a0680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:26.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.955+0000 7f7b9ec00640 1 -- 172.21.15.45:0/545735953 >> 172.21.15.45:0/545735953 conn(0x7f7b980fc880 msgr2=0x7f7b98100c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:07:26.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.955+0000 7f7b9ec00640 1 -- 172.21.15.45:0/545735953 shutdown_connections 2024-04-16T17:07:26.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:26.955+0000 7f7b9ec00640 1 -- 172.21.15.45:0/545735953 wait complete. 2024-04-16T17:07:27.974 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-16T17:07:27.974 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T17:07:27.974 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-16T17:07:28.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.174+0000 7f2dafe00640 1 -- 172.21.15.45:0/546980714 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2db010e820 msgr2=0x7f2db0110c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:28.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.174+0000 7f2dafe00640 1 --2- 172.21.15.45:0/546980714 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2db010e820 0x7f2db0110c10 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f2da8009ca0 tx=0x7f2da802c100 comp rx=0 tx=0).stop 2024-04-16T17:07:28.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.175+0000 7f2dafe00640 1 -- 172.21.15.45:0/546980714 shutdown_connections 2024-04-16T17:07:28.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.175+0000 7f2dafe00640 1 --2- 172.21.15.45:0/546980714 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2db010e820 0x7f2db0110c10 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:28.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.175+0000 7f2dafe00640 1 --2- 172.21.15.45:0/546980714 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2db0101620 0x7f2db010e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:28.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.175+0000 7f2dafe00640 1 --2- 172.21.15.45:0/546980714 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2db0100c70 0x7f2db0101050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:28.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.175+0000 7f2dafe00640 1 -- 172.21.15.45:0/546980714 >> 172.21.15.45:0/546980714 conn(0x7f2db00fc880 msgr2=0x7f2db00feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:07:28.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.175+0000 7f2dafe00640 1 -- 172.21.15.45:0/546980714 shutdown_connections 2024-04-16T17:07:28.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.175+0000 7f2dafe00640 1 -- 172.21.15.45:0/546980714 wait complete. 2024-04-16T17:07:28.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.176+0000 7f2dafe00640 1 Processor -- start 2024-04-16T17:07:28.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.176+0000 7f2dafe00640 1 -- start start 2024-04-16T17:07:28.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.177+0000 7f2dafe00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2db0100c70 0x7f2db01a0670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:28.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.177+0000 7f2dafe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2db0101620 0x7f2db01a0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:28.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.177+0000 7f2dafe00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2db010e820 0x7f2db019a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:28.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.177+0000 7f2dafe00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f2db0114450 con 0x7f2db0101620 2024-04-16T17:07:28.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.177+0000 7f2dafe00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f2db01142d0 con 0x7f2db010e820 2024-04-16T17:07:28.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.177+0000 7f2dafe00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f2db01145d0 con 0x7f2db0100c70 2024-04-16T17:07:28.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.177+0000 7f2daea00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2db0100c70 0x7f2db01a0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:28.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.177+0000 7f2daf400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2db010e820 0x7f2db019a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:28.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.177+0000 7f2daea00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2db0100c70 0x7f2db01a0670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:58366/0 (socket says 172.21.15.45:58366) 2024-04-16T17:07:28.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.177+0000 7f2daea00640 1 -- 172.21.15.45:0/1747128572 learned_addr learned my addr 172.21.15.45:0/1747128572 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:07:28.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.177+0000 7f2dae000640 1 --2- 172.21.15.45:0/1747128572 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2db0101620 0x7f2db01a0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:28.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.178+0000 7f2daf400640 1 -- 172.21.15.45:0/1747128572 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2db0100c70 msgr2=0x7f2db01a0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:28.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.178+0000 7f2daf400640 1 --2- 172.21.15.45:0/1747128572 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2db0100c70 0x7f2db01a0670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:28.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.178+0000 7f2daf400640 1 -- 172.21.15.45:0/1747128572 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2db0101620 msgr2=0x7f2db01a0bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:28.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.178+0000 7f2daf400640 1 --2- 172.21.15.45:0/1747128572 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2db0101620 0x7f2db01a0bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:28.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.178+0000 7f2daf400640 1 -- 172.21.15.45:0/1747128572 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2db019b000 con 0x7f2db010e820 2024-04-16T17:07:28.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.179+0000 7f2daf400640 1 --2- 172.21.15.45:0/1747128572 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2db010e820 0x7f2db019a830 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f2da802c710 tx=0x7f2da8032f30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:07:28.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.179+0000 7f2da7400640 1 -- 172.21.15.45:0/1747128572 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2da8039040 con 0x7f2db010e820 2024-04-16T17:07:28.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.179+0000 7f2da7400640 1 -- 172.21.15.45:0/1747128572 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2da8044070 con 0x7f2db010e820 2024-04-16T17:07:28.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.179+0000 7f2da7400640 1 -- 172.21.15.45:0/1747128572 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2da80042a0 con 0x7f2db010e820 2024-04-16T17:07:28.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.179+0000 7f2dafe00640 1 -- 172.21.15.45:0/1747128572 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2db019b290 con 0x7f2db010e820 2024-04-16T17:07:28.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.179+0000 7f2dafe00640 1 -- 172.21.15.45:0/1747128572 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2db01a74c0 con 0x7f2db010e820 2024-04-16T17:07:28.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.184+0000 7f2db8278740 1 Processor -- start 2024-04-16T17:07:28.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.184+0000 7f2da7400640 1 -- 172.21.15.45:0/1747128572 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 25) ==== 138879+0+0 (secure 0 0 0) 0x7f2da8004440 con 0x7f2db010e820 2024-04-16T17:07:28.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.185+0000 7f2db8278740 1 -- start start 2024-04-16T17:07:28.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.185+0000 7f2db8278740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560ae4b5ae90 0x560ae4b5b270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:28.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.185+0000 7f2db8278740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560ae4b5b7b0 0x560ae4b63e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:28.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.185+0000 7f2db8278740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560ae4b64390 0x560ae4b66780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:28.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.185+0000 7f2db8278740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x560ae4abfcb0 con 0x560ae4b5ae90 2024-04-16T17:07:28.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.185+0000 7f2db8278740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x560ae4a66370 con 0x560ae4b64390 2024-04-16T17:07:28.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.185+0000 7f2db8278740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x560ae4abbc70 con 0x560ae4b5b7b0 2024-04-16T17:07:28.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.186+0000 7f2daea00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560ae4b5b7b0 0x560ae4b63e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:28.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.186+0000 7f2daea00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560ae4b5b7b0 0x560ae4b63e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:58372/0 (socket says 172.21.15.45:58372) 2024-04-16T17:07:28.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.186+0000 7f2daea00640 1 -- 172.21.15.45:0/2381595536 learned_addr learned my addr 172.21.15.45:0/2381595536 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:07:28.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.186+0000 7f2dae000640 1 --2- 172.21.15.45:0/2381595536 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560ae4b5ae90 0x560ae4b5b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:28.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.186+0000 7f2daea00640 1 -- 172.21.15.45:0/2381595536 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560ae4b64390 msgr2=0x560ae4b66780 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-16T17:07:28.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.186+0000 7f2dae000640 1 --2- 172.21.15.45:0/2381595536 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560ae4b64390 0x560ae4b66780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:28.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.186+0000 7f2daea00640 1 --2- 172.21.15.45:0/2381595536 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560ae4b64390 0x560ae4b66780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:28.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.186+0000 7f2daea00640 1 -- 172.21.15.45:0/2381595536 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560ae4b5ae90 msgr2=0x560ae4b5b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:07:28.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.186+0000 7f2daea00640 1 --2- 172.21.15.45:0/2381595536 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560ae4b5ae90 0x560ae4b5b270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:28.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.186+0000 7f2daea00640 1 -- 172.21.15.45:0/2381595536 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560ae4a85910 con 0x560ae4b5b7b0 2024-04-16T17:07:28.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.187+0000 7f2da7400640 1 --2- 172.21.15.45:0/1747128572 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f2d940a3e40 0x7f2d940a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:28.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.187+0000 7f2da7400640 1 -- 172.21.15.45:0/1747128572 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f2da80f31b0 con 0x7f2db010e820 2024-04-16T17:07:28.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.187+0000 7f2dae000640 1 --2- 172.21.15.45:0/1747128572 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f2d940a3e40 0x7f2d940a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:28.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.187+0000 7f2daea00640 1 --2- 172.21.15.45:0/2381595536 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560ae4b5b7b0 0x560ae4b63e50 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f2da0009e10 tx=0x7f2da0011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c8edc59240d830cb server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:07:28.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.188+0000 7f2d9b400640 1 -- 172.21.15.45:0/2381595536 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2da001e070 con 0x560ae4b5b7b0 2024-04-16T17:07:28.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.188+0000 7f2d9b400640 1 -- 172.21.15.45:0/2381595536 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2da0019b10 con 0x560ae4b5b7b0 2024-04-16T17:07:28.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.188+0000 7f2d9b400640 1 -- 172.21.15.45:0/2381595536 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2da000fc30 con 0x560ae4b5b7b0 2024-04-16T17:07:28.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.189+0000 7f2db8278740 1 -- 172.21.15.45:0/2381595536 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560ae4b5b7b0 msgr2=0x560ae4b63e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:28.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.189+0000 7f2db8278740 1 --2- 172.21.15.45:0/2381595536 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560ae4b5b7b0 0x560ae4b63e50 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f2da0009e10 tx=0x7f2da0011810 comp rx=0 tx=0).stop 2024-04-16T17:07:28.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.189+0000 7f2dae000640 1 --2- 172.21.15.45:0/1747128572 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f2d940a3e40 0x7f2d940a6300 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x560ae4b56750 tx=0x7f2d9c005f90 comp rx=0 tx=0).ready entity=mgr.25411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:07:28.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.189+0000 7f2db8278740 1 -- 172.21.15.45:0/2381595536 shutdown_connections 2024-04-16T17:07:28.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.189+0000 7f2db8278740 1 --2- 172.21.15.45:0/2381595536 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560ae4b5b7b0 0x560ae4b63e50 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:28.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.189+0000 7f2db8278740 1 --2- 172.21.15.45:0/2381595536 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560ae4b64390 0x560ae4b66780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:28.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.189+0000 7f2db8278740 1 --2- 172.21.15.45:0/2381595536 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560ae4b5ae90 0x560ae4b5b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:28.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.189+0000 7f2db8278740 1 -- 172.21.15.45:0/2381595536 >> 172.21.15.45:0/2381595536 conn(0x560ae4b555c0 msgr2=0x560ae4b559a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:07:28.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.190+0000 7f2db8278740 1 -- 172.21.15.45:0/2381595536 shutdown_connections 2024-04-16T17:07:28.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.190+0000 7f2db8278740 1 -- 172.21.15.45:0/2381595536 wait complete. 2024-04-16T17:07:28.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.190+0000 7f2db8278740 1 -- start start 2024-04-16T17:07:28.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.190+0000 7f2db8278740 1 Processor -- start 2024-04-16T17:07:28.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.191+0000 7f2db8278740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560ae4b5ae90 0x560ae4b61420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:28.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.191+0000 7f2db8278740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560ae4b5b7b0 0x560ae4b61960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:28.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.192+0000 7f2db8278740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560ae4b64390 0x560ae4b61ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:28.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.192+0000 7f2db8278740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x560ae4b67180 con 0x560ae4b64390 2024-04-16T17:07:28.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.192+0000 7f2daea00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560ae4b5ae90 0x560ae4b61420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:28.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.192+0000 7f2dae000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560ae4b5b7b0 0x560ae4b61960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:28.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.192+0000 7f2dae000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560ae4b5b7b0 0x560ae4b61960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:58380/0 (socket says 172.21.15.45:58380) 2024-04-16T17:07:28.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.192+0000 7f2dae000640 1 -- 172.21.15.45:0/2208247995 learned_addr learned my addr 172.21.15.45:0/2208247995 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:07:28.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.192+0000 7f2db8278740 1 -- 172.21.15.45:0/2208247995 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x560ae4a79730 con 0x560ae4b5ae90 2024-04-16T17:07:28.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.192+0000 7f2db8278740 1 -- 172.21.15.45:0/2208247995 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x560ae4b67300 con 0x560ae4b5b7b0 2024-04-16T17:07:28.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.192+0000 7f2db8278740 10 client.0 _ll_register_callbacks cb 0x560ae4a481e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:07:28.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.192+0000 7f2daf400640 1 --2- 172.21.15.45:0/2208247995 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560ae4b64390 0x560ae4b61ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:28.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.192+0000 7f2daea00640 1 -- 172.21.15.45:0/2208247995 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560ae4b5b7b0 msgr2=0x560ae4b61960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:07:28.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.192+0000 7f2daea00640 1 --2- 172.21.15.45:0/2208247995 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560ae4b5b7b0 0x560ae4b61960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:28.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.192+0000 7f2daea00640 1 -- 172.21.15.45:0/2208247995 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560ae4b64390 msgr2=0x560ae4b61ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:07:28.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.193+0000 7f2daea00640 1 --2- 172.21.15.45:0/2208247995 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560ae4b64390 0x560ae4b61ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:28.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.193+0000 7f2daea00640 1 -- 172.21.15.45:0/2208247995 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560ae489a7e0 con 0x560ae4b5ae90 2024-04-16T17:07:28.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.193+0000 7f2daf400640 1 --2- 172.21.15.45:0/2208247995 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560ae4b64390 0x560ae4b61ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:07:28.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.193+0000 7f2daea00640 1 --2- 172.21.15.45:0/2208247995 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560ae4b5ae90 0x560ae4b61420 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f2da000f810 tx=0x7f2da000f450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ba56a4a1b34d4928 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:07:28.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.193+0000 7f2d8fe00640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-04-16T17:07:28.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.193+0000 7f2d8fe00640 1 -- 172.21.15.45:0/2208247995 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2da001e040 con 0x560ae4b5ae90 2024-04-16T17:07:28.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.193+0000 7f2d8fe00640 1 -- 172.21.15.45:0/2208247995 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f2da000fd80 con 0x560ae4b5ae90 2024-04-16T17:07:28.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.194+0000 7f2db8278740 1 -- 172.21.15.45:0/2208247995 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2da0019b10 con 0x560ae4b5ae90 2024-04-16T17:07:28.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.194+0000 7f2d8fe00640 1 -- 172.21.15.45:0/2208247995 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f2da0032050 con 0x560ae4b5ae90 2024-04-16T17:07:28.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.194+0000 7f2db8278740 10 client.25784 fetch_fsmap learned FSMap version 25 2024-04-16T17:07:28.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.194+0000 7f2db8278740 1 -- 172.21.15.45:0/2208247995 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x560ae48faf70 con 0x560ae4b5ae90 2024-04-16T17:07:28.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.194+0000 7f2d8fe00640 1 -- 172.21.15.45:0/2208247995 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2da0019910 con 0x560ae4b5ae90 2024-04-16T17:07:28.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.195+0000 7f2d8fe00640 1 -- 172.21.15.45:0/2208247995 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f2da00046c0 con 0x560ae4b5ae90 2024-04-16T17:07:28.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.195+0000 7f2db8278740 10 client.25784 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T17:07:28.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.195+0000 7f2db8278740 10 client.25784 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:07:28.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.195+0000 7f2db8278740 20 client.25784 populate_metadata read hostname 'smithi045' 2024-04-16T17:07:28.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.196+0000 7f2db8278740 1 --2- 172.21.15.45:0/2208247995 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x560ae4b74710 0x560ae4b76b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:28.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.196+0000 7f2db8278740 4 client.25784 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:07:28.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.196+0000 7f2db8278740 1 -- 172.21.15.45:0/2208247995 --> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f2da0019b10 con 0x560ae4b74710 2024-04-16T17:07:28.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.196+0000 7f2dae000640 1 --2- 172.21.15.45:0/2208247995 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x560ae4b74710 0x560ae4b76b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:28.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.197+0000 7f2dae000640 1 --2- 172.21.15.45:0/2208247995 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x560ae4b74710 0x560ae4b76b00 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=271d71ccdfbf3608 server_cookie=42169debad5a72f6 in_seq=0 out_seq=0 2024-04-16T17:07:28.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.197+0000 7f2d8fe00640 10 client.25784 ms_handle_connect on v2:172.21.15.104:6838/3660070592 2024-04-16T17:07:28.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.197+0000 7f2d8fe00640 1 -- 172.21.15.45:0/2208247995 <== mds.0 v2:172.21.15.104:6838/3660070592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x7f2da0019b10 con 0x560ae4b74710 2024-04-16T17:07:28.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.197+0000 7f2d8fe00640 10 client.25784 handle_command_reply: tid=0 2024-04-16T17:07:28.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.198+0000 7f2db8278740 1 client.25784 shutdown 2024-04-16T17:07:28.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.198+0000 7f2db8278740 1 -- 172.21.15.45:0/2208247995 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560ae4b5ae90 msgr2=0x560ae4b61420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:28.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.198+0000 7f2db8278740 1 --2- 172.21.15.45:0/2208247995 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560ae4b5ae90 0x560ae4b61420 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f2da000f810 tx=0x7f2da000f450 comp rx=0 tx=0).stop 2024-04-16T17:07:28.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.198+0000 7f2db8278740 1 -- 172.21.15.45:0/2208247995 shutdown_connections 2024-04-16T17:07:28.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.198+0000 7f2db8278740 1 --2- 172.21.15.45:0/2208247995 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x560ae4b74710 0x560ae4b76b00 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:28.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.198+0000 7f2db8278740 1 --2- 172.21.15.45:0/2208247995 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560ae4b5b7b0 0x560ae4b61960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:28.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.198+0000 7f2db8278740 1 --2- 172.21.15.45:0/2208247995 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560ae4b64390 0x560ae4b61ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:28.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.198+0000 7f2db8278740 1 --2- 172.21.15.45:0/2208247995 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560ae4b5ae90 0x560ae4b61420 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:28.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.198+0000 7f2db8278740 1 -- 172.21.15.45:0/2208247995 >> 172.21.15.45:0/2208247995 conn(0x560ae4b555c0 msgr2=0x560ae4b57310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:07:28.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.198+0000 7f2d8fe00640 0 client.25784 ms_handle_reset on v2:172.21.15.104:6838/3660070592 2024-04-16T17:07:28.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.198+0000 7f2db8278740 1 -- 172.21.15.45:0/2208247995 shutdown_connections 2024-04-16T17:07:28.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.198+0000 7f2db8278740 1 -- 172.21.15.45:0/2208247995 wait complete. 2024-04-16T17:07:28.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.199+0000 7f2db8278740 20 client.25784 trim_cache size 0 max 16384 2024-04-16T17:07:28.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.237+0000 7f2db8278740 1 Processor -- start 2024-04-16T17:07:28.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.237+0000 7f2db8278740 1 -- start start 2024-04-16T17:07:28.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.238+0000 7f2db8278740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560ae4b5ae90 0x560ae4b63b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:28.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.238+0000 7f2db8278740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560ae4b5b7b0 0x560ae4b70040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:28.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.238+0000 7f2db8278740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560ae4b64390 0x560ae4b61af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:28.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.238+0000 7f2db8278740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f2db01b3da0 con 0x560ae4b5b7b0 2024-04-16T17:07:28.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.238+0000 7f2db8278740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f2db01b1670 con 0x560ae4b5ae90 2024-04-16T17:07:28.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.238+0000 7f2db8278740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f2db01af4a0 con 0x560ae4b64390 2024-04-16T17:07:28.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.238+0000 7f2daea00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560ae4b5ae90 0x560ae4b63b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:28.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.238+0000 7f2daf400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560ae4b64390 0x560ae4b61af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:28.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.238+0000 7f2daea00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560ae4b5ae90 0x560ae4b63b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:35522/0 (socket says 172.21.15.45:35522) 2024-04-16T17:07:28.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.238+0000 7f2daea00640 1 -- 172.21.15.45:0/2560783856 learned_addr learned my addr 172.21.15.45:0/2560783856 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:07:28.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.238+0000 7f2dae000640 1 --2- 172.21.15.45:0/2560783856 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560ae4b5b7b0 0x560ae4b70040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:28.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.239+0000 7f2daea00640 1 -- 172.21.15.45:0/2560783856 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560ae4b64390 msgr2=0x560ae4b61af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:07:28.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.239+0000 7f2daea00640 1 --2- 172.21.15.45:0/2560783856 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560ae4b64390 0x560ae4b61af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:28.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.239+0000 7f2daea00640 1 -- 172.21.15.45:0/2560783856 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560ae4b5b7b0 msgr2=0x560ae4b70040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:07:28.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.239+0000 7f2daea00640 1 --2- 172.21.15.45:0/2560783856 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560ae4b5b7b0 0x560ae4b70040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:28.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.239+0000 7f2daea00640 1 -- 172.21.15.45:0/2560783856 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560ae4792170 con 0x560ae4b5ae90 2024-04-16T17:07:28.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.239+0000 7f2dae000640 1 --2- 172.21.15.45:0/2560783856 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560ae4b5b7b0 0x560ae4b70040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:07:28.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.239+0000 7f2daea00640 1 --2- 172.21.15.45:0/2560783856 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560ae4b5ae90 0x560ae4b63b50 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f2da0000c00 tx=0x7f2da000fe00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4896ec40b6869d33 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:07:28.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.239+0000 7f2d9aa00640 1 -- 172.21.15.45:0/2560783856 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2da001e040 con 0x560ae4b5ae90 2024-04-16T17:07:28.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.239+0000 7f2d9aa00640 1 -- 172.21.15.45:0/2560783856 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2da002e2e0 con 0x560ae4b5ae90 2024-04-16T17:07:28.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.239+0000 7f2d9aa00640 1 -- 172.21.15.45:0/2560783856 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2da0004780 con 0x560ae4b5ae90 2024-04-16T17:07:28.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.240+0000 7f2db8278740 1 -- 172.21.15.45:0/2560783856 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560ae4b5ae90 msgr2=0x560ae4b63b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:28.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.240+0000 7f2db8278740 1 --2- 172.21.15.45:0/2560783856 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560ae4b5ae90 0x560ae4b63b50 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f2da0000c00 tx=0x7f2da000fe00 comp rx=0 tx=0).stop 2024-04-16T17:07:28.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.241+0000 7f2db8278740 1 -- 172.21.15.45:0/2560783856 shutdown_connections 2024-04-16T17:07:28.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.241+0000 7f2db8278740 1 --2- 172.21.15.45:0/2560783856 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560ae4b64390 0x560ae4b61af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:28.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.241+0000 7f2db8278740 1 --2- 172.21.15.45:0/2560783856 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560ae4b5b7b0 0x560ae4b70040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:28.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.241+0000 7f2db8278740 1 --2- 172.21.15.45:0/2560783856 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560ae4b5ae90 0x560ae4b63b50 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:28.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.241+0000 7f2db8278740 1 -- 172.21.15.45:0/2560783856 >> 172.21.15.45:0/2560783856 conn(0x560ae4b555c0 msgr2=0x560ae4b613e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:07:28.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.241+0000 7f2db8278740 1 -- 172.21.15.45:0/2560783856 shutdown_connections 2024-04-16T17:07:28.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.241+0000 7f2db8278740 1 -- 172.21.15.45:0/2560783856 wait complete. 2024-04-16T17:07:28.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.241+0000 7f2db8278740 1 -- start start 2024-04-16T17:07:28.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.242+0000 7f2db8278740 1 Processor -- start 2024-04-16T17:07:28.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.242+0000 7f2db8278740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560ae4b5ae90 0x560ae4b6c570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:28.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.242+0000 7f2db8278740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560ae4b5b7b0 0x560ae4b6cab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:28.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.242+0000 7f2db8278740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560ae4b64390 0x560ae4b67d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:28.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.242+0000 7f2db8278740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f2db01afb30 con 0x560ae4b5ae90 2024-04-16T17:07:28.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.242+0000 7f2db8278740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f2db01b0fa0 con 0x560ae4b5b7b0 2024-04-16T17:07:28.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.242+0000 7f2db8278740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f2db01b0200 con 0x560ae4b64390 2024-04-16T17:07:28.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.242+0000 7f2db8278740 10 client.0 _ll_register_callbacks cb 0x7f2da000f620 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:07:28.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.243+0000 7f2daea00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560ae4b5ae90 0x560ae4b6c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:28.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.243+0000 7f2daea00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560ae4b5ae90 0x560ae4b6c570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45376/0 (socket says 172.21.15.45:45376) 2024-04-16T17:07:28.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.243+0000 7f2daea00640 1 -- 172.21.15.45:0/13411175 learned_addr learned my addr 172.21.15.45:0/13411175 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:07:28.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.243+0000 7f2dae000640 1 --2- 172.21.15.45:0/13411175 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560ae4b5b7b0 0x560ae4b6cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:28.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.243+0000 7f2daea00640 1 -- 172.21.15.45:0/13411175 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560ae4b64390 msgr2=0x560ae4b67d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:07:28.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.243+0000 7f2daea00640 1 --2- 172.21.15.45:0/13411175 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560ae4b64390 0x560ae4b67d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:28.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.243+0000 7f2daea00640 1 -- 172.21.15.45:0/13411175 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560ae4b5b7b0 msgr2=0x560ae4b6cab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:07:28.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.243+0000 7f2daea00640 1 --2- 172.21.15.45:0/13411175 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560ae4b5b7b0 0x560ae4b6cab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:28.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.243+0000 7f2daea00640 1 -- 172.21.15.45:0/13411175 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560ae4b684e0 con 0x560ae4b5ae90 2024-04-16T17:07:28.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.243+0000 7f2dae000640 1 --2- 172.21.15.45:0/13411175 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560ae4b5b7b0 0x560ae4b6cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:07:28.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.243+0000 7f2daea00640 1 --2- 172.21.15.45:0/13411175 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560ae4b5ae90 0x560ae4b6c570 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f2da00094f0 tx=0x7f2da000f900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39096d744fbcfefb server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:07:28.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.243+0000 7f2d8fe00640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-16T17:07:28.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.244+0000 7f2d8fe00640 1 -- 172.21.15.45:0/13411175 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2da003a020 con 0x560ae4b5ae90 2024-04-16T17:07:28.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.244+0000 7f2d8fe00640 1 -- 172.21.15.45:0/13411175 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f2da0004240 con 0x560ae4b5ae90 2024-04-16T17:07:28.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.244+0000 7f2db8278740 1 -- 172.21.15.45:0/13411175 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2da002e2e0 con 0x560ae4b5ae90 2024-04-16T17:07:28.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.244+0000 7f2d8fe00640 1 -- 172.21.15.45:0/13411175 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f2da001e070 con 0x560ae4b5ae90 2024-04-16T17:07:28.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.244+0000 7f2db8278740 10 client.15660 fetch_fsmap learned FSMap version 25 2024-04-16T17:07:28.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.244+0000 7f2db8278740 1 -- 172.21.15.45:0/13411175 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x560ae4beb870 con 0x560ae4b5ae90 2024-04-16T17:07:28.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.245+0000 7f2d8fe00640 1 -- 172.21.15.45:0/13411175 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2da002e2e0 con 0x560ae4b5ae90 2024-04-16T17:07:28.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.245+0000 7f2d8fe00640 1 -- 172.21.15.45:0/13411175 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f2da0003b40 con 0x560ae4b5ae90 2024-04-16T17:07:28.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.245+0000 7f2db8278740 10 client.15660 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T17:07:28.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.245+0000 7f2db8278740 10 client.15660 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:07:28.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.245+0000 7f2db8278740 20 client.15660 populate_metadata read hostname 'smithi045' 2024-04-16T17:07:28.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.245+0000 7f2db8278740 1 --2- 172.21.15.45:0/13411175 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x560ae4bebe50 0x560ae4bee270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:28.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.245+0000 7f2db8278740 4 client.15660 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-16T17:07:28.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.245+0000 7f2db8278740 1 -- 172.21.15.45:0/13411175 --> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x560ae4acd6d0 con 0x560ae4bebe50 2024-04-16T17:07:28.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.246+0000 7f2dae000640 1 --2- 172.21.15.45:0/13411175 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x560ae4bebe50 0x560ae4bee270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:28.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.246+0000 7f2dae000640 1 --2- 172.21.15.45:0/13411175 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x560ae4bebe50 0x560ae4bee270 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9c7a06a4acaec6bf server_cookie=b86e6b6850e45014 in_seq=0 out_seq=0 2024-04-16T17:07:28.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.246+0000 7f2d8fe00640 10 client.15660 ms_handle_connect on v2:172.21.15.104:6838/3660070592 2024-04-16T17:07:28.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.248+0000 7f2d8fe00640 1 -- 172.21.15.45:0/13411175 <== mds.0 v2:172.21.15.104:6838/3660070592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x560ae4acd6d0 con 0x560ae4bebe50 2024-04-16T17:07:28.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.248+0000 7f2d8fe00640 10 client.15660 handle_command_reply: tid=0 2024-04-16T17:07:28.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.248+0000 7f2db8278740 1 client.15660 shutdown 2024-04-16T17:07:28.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.248+0000 7f2db8278740 1 -- 172.21.15.45:0/13411175 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560ae4b5ae90 msgr2=0x560ae4b6c570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:28.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.248+0000 7f2db8278740 1 --2- 172.21.15.45:0/13411175 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560ae4b5ae90 0x560ae4b6c570 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f2da00094f0 tx=0x7f2da000f900 comp rx=0 tx=0).stop 2024-04-16T17:07:28.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.248+0000 7f2db8278740 1 -- 172.21.15.45:0/13411175 shutdown_connections 2024-04-16T17:07:28.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.249+0000 7f2db8278740 1 --2- 172.21.15.45:0/13411175 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x560ae4bebe50 0x560ae4bee270 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:28.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.249+0000 7f2d8fe00640 0 client.15660 ms_handle_reset on v2:172.21.15.104:6838/3660070592 2024-04-16T17:07:28.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.249+0000 7f2db8278740 1 --2- 172.21.15.45:0/13411175 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560ae4b64390 0x560ae4b67d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:28.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.249+0000 7f2db8278740 1 --2- 172.21.15.45:0/13411175 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560ae4b5b7b0 0x560ae4b6cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:28.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.249+0000 7f2db8278740 1 --2- 172.21.15.45:0/13411175 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560ae4b5ae90 0x560ae4b6c570 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:28.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.249+0000 7f2db8278740 1 -- 172.21.15.45:0/13411175 >> 172.21.15.45:0/13411175 conn(0x560ae4b555c0 msgr2=0x560ae4b61c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:07:28.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.249+0000 7f2db8278740 1 -- 172.21.15.45:0/13411175 shutdown_connections 2024-04-16T17:07:28.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.249+0000 7f2db8278740 1 -- 172.21.15.45:0/13411175 wait complete. 2024-04-16T17:07:28.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.249+0000 7f2db8278740 20 client.15660 trim_cache size 0 max 16384 2024-04-16T17:07:28.251 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-16T17:07:28.251 INFO:teuthology.orchestra.run.smithi045.stdout: "return_code": 0, 2024-04-16T17:07:28.251 INFO:teuthology.orchestra.run.smithi045.stdout: "scrub_tag": "79a59bf4-4bb7-4c3f-9141-e93e93671a28", 2024-04-16T17:07:28.251 INFO:teuthology.orchestra.run.smithi045.stdout: "mode": "asynchronous" 2024-04-16T17:07:28.252 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-16T17:07:28.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.250+0000 7f2dafe00640 1 -- 172.21.15.45:0/1747128572 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f2d940a3e40 msgr2=0x7f2d940a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:28.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.250+0000 7f2dafe00640 1 --2- 172.21.15.45:0/1747128572 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f2d940a3e40 0x7f2d940a6300 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x560ae4b56750 tx=0x7f2d9c005f90 comp rx=0 tx=0).stop 2024-04-16T17:07:28.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.250+0000 7f2dafe00640 1 -- 172.21.15.45:0/1747128572 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2db010e820 msgr2=0x7f2db019a830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:28.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.250+0000 7f2dafe00640 1 --2- 172.21.15.45:0/1747128572 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2db010e820 0x7f2db019a830 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f2da802c710 tx=0x7f2da8032f30 comp rx=0 tx=0).stop 2024-04-16T17:07:28.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.250+0000 7f2dafe00640 1 -- 172.21.15.45:0/1747128572 shutdown_connections 2024-04-16T17:07:28.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.250+0000 7f2dafe00640 1 --2- 172.21.15.45:0/1747128572 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f2d940a3e40 0x7f2d940a6300 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:28.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.250+0000 7f2dafe00640 1 --2- 172.21.15.45:0/1747128572 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2db010e820 0x7f2db019a830 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:28.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.250+0000 7f2dafe00640 1 --2- 172.21.15.45:0/1747128572 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2db0101620 0x7f2db01a0bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:28.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.250+0000 7f2dafe00640 1 --2- 172.21.15.45:0/1747128572 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2db0100c70 0x7f2db01a0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:28.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.250+0000 7f2dafe00640 1 -- 172.21.15.45:0/1747128572 >> 172.21.15.45:0/1747128572 conn(0x7f2db00fc880 msgr2=0x7f2db00fec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:07:28.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.250+0000 7f2dafe00640 1 -- 172.21.15.45:0/1747128572 shutdown_connections 2024-04-16T17:07:28.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.250+0000 7f2dafe00640 1 -- 172.21.15.45:0/1747128572 wait complete. 2024-04-16T17:07:28.267 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-16T17:07:28.268 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-16T17:07:28.363 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:07:28 smithi045 ceph-mon[31676]: pgmap v24: 129 pgs: 129 active+clean; 1.3 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-16T17:07:28.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.552+0000 7fefbae00640 1 -- 172.21.15.45:0/4148754869 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fefb4102c40 msgr2=0x7fefb4106c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:28.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.552+0000 7fefbae00640 1 --2- 172.21.15.45:0/4148754869 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fefb4102c40 0x7fefb4106c80 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fefa0009860 tx=0x7fefa002bb70 comp rx=0 tx=0).stop 2024-04-16T17:07:28.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.552+0000 7fefbae00640 1 -- 172.21.15.45:0/4148754869 shutdown_connections 2024-04-16T17:07:28.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.552+0000 7fefbae00640 1 --2- 172.21.15.45:0/4148754869 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fefb4107360 0x7fefb410ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:28.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.552+0000 7fefbae00640 1 --2- 172.21.15.45:0/4148754869 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fefb4102c40 0x7fefb4106c80 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:28.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.552+0000 7fefbae00640 1 --2- 172.21.15.45:0/4148754869 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fefb4102290 0x7fefb4102670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:28.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.552+0000 7fefbae00640 1 -- 172.21.15.45:0/4148754869 >> 172.21.15.45:0/4148754869 conn(0x7fefb40fdb10 msgr2=0x7fefb40fff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:07:28.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.552+0000 7fefbae00640 1 -- 172.21.15.45:0/4148754869 shutdown_connections 2024-04-16T17:07:28.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.553+0000 7fefbae00640 1 -- 172.21.15.45:0/4148754869 wait complete. 2024-04-16T17:07:28.555 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:07:28 smithi117 ceph-mon[36730]: pgmap v24: 129 pgs: 129 active+clean; 1.3 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-16T17:07:28.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.554+0000 7fefbae00640 1 Processor -- start 2024-04-16T17:07:28.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.554+0000 7fefbae00640 1 -- start start 2024-04-16T17:07:28.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.555+0000 7fefbae00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fefb4102290 0x7fefb41a07e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:28.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.555+0000 7fefbae00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fefb4107360 0x7fefb41a0d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:28.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.555+0000 7fefbae00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fefb419a990 0x7fefb419ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:28.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.555+0000 7fefbae00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fefb41123d0 con 0x7fefb419a990 2024-04-16T17:07:28.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.555+0000 7fefbae00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fefb4112250 con 0x7fefb4107360 2024-04-16T17:07:28.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.555+0000 7fefbae00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fefb4112550 con 0x7fefb4102290 2024-04-16T17:07:28.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.555+0000 7fefb3e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fefb4102290 0x7fefb41a07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:28.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.555+0000 7fefb8e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fefb419a990 0x7fefb419ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:28.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.555+0000 7fefb3e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fefb4102290 0x7fefb41a07e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:58422/0 (socket says 172.21.15.45:58422) 2024-04-16T17:07:28.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.555+0000 7fefb3e00640 1 -- 172.21.15.45:0/3074539466 learned_addr learned my addr 172.21.15.45:0/3074539466 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:07:28.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.555+0000 7fefb3400640 1 --2- 172.21.15.45:0/3074539466 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fefb4107360 0x7fefb41a0d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:28.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.556+0000 7fefb8e00640 1 -- 172.21.15.45:0/3074539466 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fefb4102290 msgr2=0x7fefb41a07e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:28.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.556+0000 7fefb8e00640 1 --2- 172.21.15.45:0/3074539466 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fefb4102290 0x7fefb41a07e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:28.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.556+0000 7fefb8e00640 1 -- 172.21.15.45:0/3074539466 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fefb4107360 msgr2=0x7fefb41a0d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:28.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.556+0000 7fefb8e00640 1 --2- 172.21.15.45:0/3074539466 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fefb4107360 0x7fefb41a0d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:28.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.556+0000 7fefb8e00640 1 -- 172.21.15.45:0/3074539466 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefb419b4b0 con 0x7fefb419a990 2024-04-16T17:07:28.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.556+0000 7fefb3400640 1 --2- 172.21.15.45:0/3074539466 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fefb4107360 0x7fefb41a0d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:07:28.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.556+0000 7fefb8e00640 1 --2- 172.21.15.45:0/3074539466 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fefb419a990 0x7fefb419ae40 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fefac009fd0 tx=0x7fefac00ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:07:28.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.557+0000 7fefb0c00640 1 -- 172.21.15.45:0/3074539466 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefac00ce20 con 0x7fefb419a990 2024-04-16T17:07:28.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.557+0000 7fefbae00640 1 -- 172.21.15.45:0/3074539466 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fefb419b7a0 con 0x7fefb419a990 2024-04-16T17:07:28.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.557+0000 7fefb0c00640 1 -- 172.21.15.45:0/3074539466 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fefac005020 con 0x7fefb419a990 2024-04-16T17:07:28.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.557+0000 7fefb0c00640 1 -- 172.21.15.45:0/3074539466 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefac010720 con 0x7fefb419a990 2024-04-16T17:07:28.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.557+0000 7fefbae00640 1 -- 172.21.15.45:0/3074539466 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fefb41a79f0 con 0x7fefb419a990 2024-04-16T17:07:28.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.560+0000 7fefb0c00640 1 -- 172.21.15.45:0/3074539466 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) ==== 138879+0+0 (secure 0 0 0) 0x7fefac005540 con 0x7fefb419a990 2024-04-16T17:07:28.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.562+0000 7fefbdb89740 1 Processor -- start 2024-04-16T17:07:28.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.563+0000 7fefbdb89740 1 -- start start 2024-04-16T17:07:28.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.563+0000 7fefb0c00640 1 --2- 172.21.15.45:0/3074539466 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fef980a3e40 0x7fef980a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:28.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.563+0000 7fefbdb89740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ce3353a1b0 0x55ce3353a590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:28.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.563+0000 7fefb0c00640 1 -- 172.21.15.45:0/3074539466 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fefac0d14d0 con 0x7fefb419a990 2024-04-16T17:07:28.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.563+0000 7fefbdb89740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ce3353aad0 0x55ce33543170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:28.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.563+0000 7fefbdb89740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ce335436b0 0x55ce33545aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:28.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.563+0000 7fefbdb89740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55ce334a0120 con 0x55ce3353aad0 2024-04-16T17:07:28.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.563+0000 7fefbdb89740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55ce33446340 con 0x55ce335436b0 2024-04-16T17:07:28.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.563+0000 7fefbdb89740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55ce3349c0e0 con 0x55ce3353a1b0 2024-04-16T17:07:28.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.563+0000 7fefb8e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ce335436b0 0x55ce33545aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:28.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.563+0000 7fefb8e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ce335436b0 0x55ce33545aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:35558/0 (socket says 172.21.15.45:35558) 2024-04-16T17:07:28.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.563+0000 7fefb8e00640 1 -- 172.21.15.45:0/2092038758 learned_addr learned my addr 172.21.15.45:0/2092038758 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:07:28.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.563+0000 7fefb3e00640 1 --2- 172.21.15.45:0/2092038758 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ce3353a1b0 0x55ce3353a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:28.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.564+0000 7fefb3e00640 1 -- 172.21.15.45:0/2092038758 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ce335436b0 msgr2=0x55ce33545aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:07:28.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.564+0000 7fefb3e00640 1 --2- 172.21.15.45:0/2092038758 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ce335436b0 0x55ce33545aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:28.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.564+0000 7fefb3e00640 1 -- 172.21.15.45:0/2092038758 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ce3353aad0 msgr2=0x55ce33543170 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-16T17:07:28.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.564+0000 7fefb3e00640 1 --2- 172.21.15.45:0/2092038758 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ce3353aad0 0x55ce33543170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:28.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.564+0000 7fefb3e00640 1 -- 172.21.15.45:0/2092038758 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ce3327c7d0 con 0x55ce3353a1b0 2024-04-16T17:07:28.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.564+0000 7fefb3400640 1 --2- 172.21.15.45:0/3074539466 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fef980a3e40 0x7fef980a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:28.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.565+0000 7fefb3e00640 1 --2- 172.21.15.45:0/2092038758 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ce3353a1b0 0x55ce3353a590 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fefa4005c80 tx=0x7fefa4011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8e255542b16e0a12 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:07:28.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.566+0000 7fefa9600640 1 -- 172.21.15.45:0/2092038758 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefa4004230 con 0x55ce3353a1b0 2024-04-16T17:07:28.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.566+0000 7fefa9600640 1 -- 172.21.15.45:0/2092038758 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fefa4009e40 con 0x55ce3353a1b0 2024-04-16T17:07:28.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.566+0000 7fefa9600640 1 -- 172.21.15.45:0/2092038758 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefa4004de0 con 0x55ce3353a1b0 2024-04-16T17:07:28.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.566+0000 7fefbdb89740 1 -- 172.21.15.45:0/2092038758 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ce3353a1b0 msgr2=0x55ce3353a590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:28.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.566+0000 7fefbdb89740 1 --2- 172.21.15.45:0/2092038758 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ce3353a1b0 0x55ce3353a590 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fefa4005c80 tx=0x7fefa4011810 comp rx=0 tx=0).stop 2024-04-16T17:07:28.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.566+0000 7fefb3400640 1 --2- 172.21.15.45:0/3074539466 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fef980a3e40 0x7fef980a6300 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fefa0009830 tx=0x7fefa0000bc0 comp rx=0 tx=0).ready entity=mgr.25411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:07:28.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.567+0000 7fefbdb89740 1 -- 172.21.15.45:0/2092038758 shutdown_connections 2024-04-16T17:07:28.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.567+0000 7fefbdb89740 1 --2- 172.21.15.45:0/2092038758 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ce335436b0 0x55ce33545aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:28.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.567+0000 7fefbdb89740 1 --2- 172.21.15.45:0/2092038758 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ce3353aad0 0x55ce33543170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:28.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.567+0000 7fefbdb89740 1 --2- 172.21.15.45:0/2092038758 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ce3353a1b0 0x55ce3353a590 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:28.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.567+0000 7fefbdb89740 1 -- 172.21.15.45:0/2092038758 >> 172.21.15.45:0/2092038758 conn(0x55ce335348e0 msgr2=0x55ce33534cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:07:28.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.567+0000 7fefbdb89740 1 -- 172.21.15.45:0/2092038758 shutdown_connections 2024-04-16T17:07:28.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.567+0000 7fefbdb89740 1 -- 172.21.15.45:0/2092038758 wait complete. 2024-04-16T17:07:28.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.567+0000 7fefbdb89740 1 -- start start 2024-04-16T17:07:28.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.567+0000 7fefbdb89740 1 Processor -- start 2024-04-16T17:07:28.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.568+0000 7fefbdb89740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ce3353a1b0 0x55ce335407f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:28.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.568+0000 7fefbdb89740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ce3353aad0 0x55ce33540d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:28.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.568+0000 7fefbdb89740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ce335436b0 0x55ce33541270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:28.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.568+0000 7fefbdb89740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55ce335464a0 con 0x55ce335436b0 2024-04-16T17:07:28.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.568+0000 7fefbdb89740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55ce33459710 con 0x55ce3353a1b0 2024-04-16T17:07:28.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.568+0000 7fefbdb89740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55ce33546620 con 0x55ce3353aad0 2024-04-16T17:07:28.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.568+0000 7fefbdb89740 10 client.0 _ll_register_callbacks cb 0x55ce33454af0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:07:28.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.568+0000 7fefb3400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ce3353aad0 0x55ce33540d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:28.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.568+0000 7fefb3400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ce3353aad0 0x55ce33540d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:58438/0 (socket says 172.21.15.45:58438) 2024-04-16T17:07:28.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.569+0000 7fefb3e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ce3353a1b0 0x55ce335407f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:28.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.569+0000 7fefb3400640 1 -- 172.21.15.45:0/3421609657 learned_addr learned my addr 172.21.15.45:0/3421609657 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:07:28.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.569+0000 7fefb8e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ce335436b0 0x55ce33541270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:28.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.569+0000 7fefb3e00640 1 -- 172.21.15.45:0/3421609657 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ce3353aad0 msgr2=0x55ce33540d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:07:28.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.569+0000 7fefb3e00640 1 --2- 172.21.15.45:0/3421609657 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ce3353aad0 0x55ce33540d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:28.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.569+0000 7fefb3e00640 1 -- 172.21.15.45:0/3421609657 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ce335436b0 msgr2=0x55ce33541270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:07:28.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.569+0000 7fefb3e00640 1 --2- 172.21.15.45:0/3421609657 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ce335436b0 0x55ce33541270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:28.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.569+0000 7fefb3e00640 1 -- 172.21.15.45:0/3421609657 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ce331b23f0 con 0x55ce3353a1b0 2024-04-16T17:07:28.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.569+0000 7fefb8e00640 1 --2- 172.21.15.45:0/3421609657 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ce335436b0 0x55ce33541270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:07:28.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.570+0000 7fefb3e00640 1 --2- 172.21.15.45:0/3421609657 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ce3353a1b0 0x55ce335407f0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fefa4000c00 tx=0x7fefa40045f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5e8c020a79855e1d server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:07:28.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.570+0000 7fef9e000640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-04-16T17:07:28.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.570+0000 7fef9e000640 1 -- 172.21.15.45:0/3421609657 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefa401e040 con 0x55ce3353a1b0 2024-04-16T17:07:28.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.570+0000 7fef9e000640 1 -- 172.21.15.45:0/3421609657 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fefa40037f0 con 0x55ce3353a1b0 2024-04-16T17:07:28.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.570+0000 7fefbdb89740 1 -- 172.21.15.45:0/3421609657 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55ce334adb40 con 0x55ce3353a1b0 2024-04-16T17:07:28.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.571+0000 7fef9e000640 1 -- 172.21.15.45:0/3421609657 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fefa4031050 con 0x55ce3353a1b0 2024-04-16T17:07:28.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.571+0000 7fefbdb89740 10 client.25802 fetch_fsmap learned FSMap version 25 2024-04-16T17:07:28.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.571+0000 7fefbdb89740 1 -- 172.21.15.45:0/3421609657 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55ce331720f0 con 0x55ce3353a1b0 2024-04-16T17:07:28.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.571+0000 7fef9e000640 1 -- 172.21.15.45:0/3421609657 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55ce334adb40 con 0x55ce3353a1b0 2024-04-16T17:07:28.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.571+0000 7fef9e000640 1 -- 172.21.15.45:0/3421609657 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fefa4018070 con 0x55ce3353a1b0 2024-04-16T17:07:28.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.571+0000 7fefbdb89740 10 client.25802 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T17:07:28.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.571+0000 7fefbdb89740 10 client.25802 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:07:28.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.571+0000 7fefbdb89740 20 client.25802 populate_metadata read hostname 'smithi045' 2024-04-16T17:07:28.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.571+0000 7fefbdb89740 1 --2- 172.21.15.45:0/3421609657 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55ce3355c2f0 0x55ce3355e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:28.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.571+0000 7fefbdb89740 4 client.25802 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:07:28.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.571+0000 7fefbdb89740 1 -- 172.21.15.45:0/3421609657 --> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ce334a79b0 con 0x55ce3355c2f0 2024-04-16T17:07:28.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.572+0000 7fefb3400640 1 --2- 172.21.15.45:0/3421609657 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55ce3355c2f0 0x55ce3355e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:28.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.572+0000 7fefb3400640 1 --2- 172.21.15.45:0/3421609657 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55ce3355c2f0 0x55ce3355e6e0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=294acf73b7c8bf8a server_cookie=4bb0ec72ea24867b in_seq=0 out_seq=0 2024-04-16T17:07:28.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.572+0000 7fef9e000640 10 client.25802 ms_handle_connect on v2:172.21.15.104:6838/3660070592 2024-04-16T17:07:28.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.573+0000 7fef9e000640 1 -- 172.21.15.45:0/3421609657 <== mds.0 v2:172.21.15.104:6838/3660070592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ce334a79b0 con 0x55ce3355c2f0 2024-04-16T17:07:28.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.573+0000 7fef9e000640 10 client.25802 handle_command_reply: tid=0 2024-04-16T17:07:28.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.573+0000 7fefbdb89740 1 client.25802 shutdown 2024-04-16T17:07:28.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.573+0000 7fefbdb89740 1 -- 172.21.15.45:0/3421609657 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ce3353a1b0 msgr2=0x55ce335407f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:28.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.573+0000 7fefbdb89740 1 --2- 172.21.15.45:0/3421609657 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ce3353a1b0 0x55ce335407f0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fefa4000c00 tx=0x7fefa40045f0 comp rx=0 tx=0).stop 2024-04-16T17:07:28.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.574+0000 7fefbdb89740 1 -- 172.21.15.45:0/3421609657 shutdown_connections 2024-04-16T17:07:28.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.574+0000 7fefbdb89740 1 --2- 172.21.15.45:0/3421609657 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55ce3355c2f0 0x55ce3355e6e0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:28.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.574+0000 7fefbdb89740 1 --2- 172.21.15.45:0/3421609657 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ce3353aad0 0x55ce33540d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:28.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.574+0000 7fefbdb89740 1 --2- 172.21.15.45:0/3421609657 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ce335436b0 0x55ce33541270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:28.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.574+0000 7fefbdb89740 1 --2- 172.21.15.45:0/3421609657 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ce3353a1b0 0x55ce335407f0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:28.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.574+0000 7fefbdb89740 1 -- 172.21.15.45:0/3421609657 >> 172.21.15.45:0/3421609657 conn(0x55ce335348e0 msgr2=0x55ce33536630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:07:28.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.574+0000 7fef9e000640 0 client.25802 ms_handle_reset on v2:172.21.15.104:6838/3660070592 2024-04-16T17:07:28.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.574+0000 7fefbdb89740 1 -- 172.21.15.45:0/3421609657 shutdown_connections 2024-04-16T17:07:28.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.574+0000 7fefbdb89740 1 -- 172.21.15.45:0/3421609657 wait complete. 2024-04-16T17:07:28.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.574+0000 7fefbdb89740 20 client.25802 trim_cache size 0 max 16384 2024-04-16T17:07:28.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.592+0000 7fefbdb89740 1 Processor -- start 2024-04-16T17:07:28.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.593+0000 7fefbdb89740 1 -- start start 2024-04-16T17:07:28.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.593+0000 7fefbdb89740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ce3353a1b0 0x55ce33542f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:28.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.593+0000 7fefbdb89740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ce3353aad0 0x55ce33563e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:28.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.593+0000 7fefbdb89740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ce335436b0 0x55ce335643b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:28.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.593+0000 7fefbdb89740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fefb41afb60 con 0x55ce3353a1b0 2024-04-16T17:07:28.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.593+0000 7fefbdb89740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fefb41ad430 con 0x55ce3353aad0 2024-04-16T17:07:28.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.593+0000 7fefbdb89740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fefb41ab2c0 con 0x55ce335436b0 2024-04-16T17:07:28.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.593+0000 7fefb3e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ce3353a1b0 0x55ce33542f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:28.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.593+0000 7fefb3400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ce3353aad0 0x55ce33563e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:28.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.593+0000 7fefb3400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ce3353aad0 0x55ce33563e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:35572/0 (socket says 172.21.15.45:35572) 2024-04-16T17:07:28.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.593+0000 7fefb3400640 1 -- 172.21.15.45:0/1652467624 learned_addr learned my addr 172.21.15.45:0/1652467624 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:07:28.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.593+0000 7fefb8e00640 1 --2- 172.21.15.45:0/1652467624 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ce335436b0 0x55ce335643b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:28.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.594+0000 7fefb3400640 1 -- 172.21.15.45:0/1652467624 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ce335436b0 msgr2=0x55ce335643b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:07:28.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.594+0000 7fefb3400640 1 --2- 172.21.15.45:0/1652467624 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ce335436b0 0x55ce335643b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:28.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.594+0000 7fefb3400640 1 -- 172.21.15.45:0/1652467624 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ce3353a1b0 msgr2=0x55ce33542f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:07:28.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.594+0000 7fefb3400640 1 --2- 172.21.15.45:0/1652467624 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ce3353a1b0 0x55ce33542f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:28.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.594+0000 7fefb3400640 1 -- 172.21.15.45:0/1652467624 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ce33534410 con 0x55ce3353aad0 2024-04-16T17:07:28.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.594+0000 7fefb3400640 1 --2- 172.21.15.45:0/1652467624 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ce3353aad0 0x55ce33563e70 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fefa00057c0 tx=0x7fefa003f000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dae0db74554f7e2e server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:07:28.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.594+0000 7fefa8c00640 1 -- 172.21.15.45:0/1652467624 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefa0059070 con 0x55ce3353aad0 2024-04-16T17:07:28.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.594+0000 7fefa8c00640 1 -- 172.21.15.45:0/1652467624 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fefa0045040 con 0x55ce3353aad0 2024-04-16T17:07:28.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.594+0000 7fefa8c00640 1 -- 172.21.15.45:0/1652467624 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefa0039040 con 0x55ce3353aad0 2024-04-16T17:07:28.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.595+0000 7fefbdb89740 1 -- 172.21.15.45:0/1652467624 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ce3353aad0 msgr2=0x55ce33563e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:28.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.595+0000 7fefbdb89740 1 --2- 172.21.15.45:0/1652467624 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ce3353aad0 0x55ce33563e70 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fefa00057c0 tx=0x7fefa003f000 comp rx=0 tx=0).stop 2024-04-16T17:07:28.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.596+0000 7fefbdb89740 1 -- 172.21.15.45:0/1652467624 shutdown_connections 2024-04-16T17:07:28.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.596+0000 7fefbdb89740 1 --2- 172.21.15.45:0/1652467624 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ce335436b0 0x55ce335643b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:28.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.596+0000 7fefbdb89740 1 --2- 172.21.15.45:0/1652467624 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ce3353aad0 0x55ce33563e70 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:28.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.596+0000 7fefbdb89740 1 --2- 172.21.15.45:0/1652467624 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ce3353a1b0 0x55ce33542f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:28.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.596+0000 7fefbdb89740 1 -- 172.21.15.45:0/1652467624 >> 172.21.15.45:0/1652467624 conn(0x55ce335348e0 msgr2=0x55ce3353f820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:07:28.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.596+0000 7fefbdb89740 1 -- 172.21.15.45:0/1652467624 shutdown_connections 2024-04-16T17:07:28.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.596+0000 7fefbdb89740 1 -- 172.21.15.45:0/1652467624 wait complete. 2024-04-16T17:07:28.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.596+0000 7fefbdb89740 1 -- start start 2024-04-16T17:07:28.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.597+0000 7fefbdb89740 1 Processor -- start 2024-04-16T17:07:28.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.597+0000 7fefbdb89740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ce3353a1b0 0x55ce3354f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:28.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.597+0000 7fefbdb89740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ce3353aad0 0x55ce3354f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:28.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.597+0000 7fefbdb89740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ce335436b0 0x55ce335ba710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:28.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.597+0000 7fefbdb89740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fefb41ab910 con 0x55ce3353aad0 2024-04-16T17:07:28.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.597+0000 7fefbdb89740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fefb41acd60 con 0x55ce335436b0 2024-04-16T17:07:28.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.597+0000 7fefbdb89740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fefb41abfc0 con 0x55ce3353a1b0 2024-04-16T17:07:28.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.597+0000 7fefbdb89740 10 client.0 _ll_register_callbacks cb 0x7fefa4009b40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:07:28.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.598+0000 7fefb3400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ce3353aad0 0x55ce3354f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:28.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.598+0000 7fefb3400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ce3353aad0 0x55ce3354f8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45422/0 (socket says 172.21.15.45:45422) 2024-04-16T17:07:28.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.598+0000 7fefb3400640 1 -- 172.21.15.45:0/247607431 learned_addr learned my addr 172.21.15.45:0/247607431 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:07:28.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.598+0000 7fefb8e00640 1 --2- 172.21.15.45:0/247607431 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ce335436b0 0x55ce335ba710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:28.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.598+0000 7fefb3400640 1 -- 172.21.15.45:0/247607431 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ce3353a1b0 msgr2=0x55ce3354f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:07:28.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.598+0000 7fefb3e00640 1 --2- 172.21.15.45:0/247607431 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ce3353a1b0 0x55ce3354f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:28.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.598+0000 7fefb3400640 1 --2- 172.21.15.45:0/247607431 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ce3353a1b0 0x55ce3354f3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:28.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.598+0000 7fefb3400640 1 -- 172.21.15.45:0/247607431 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ce335436b0 msgr2=0x55ce335ba710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:07:28.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.598+0000 7fefb3400640 1 --2- 172.21.15.45:0/247607431 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ce335436b0 0x55ce335ba710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:28.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.598+0000 7fefb3400640 1 -- 172.21.15.45:0/247607431 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ce335bad50 con 0x55ce3353aad0 2024-04-16T17:07:28.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.598+0000 7fefb8e00640 1 --2- 172.21.15.45:0/247607431 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ce335436b0 0x55ce335ba710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:07:28.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.599+0000 7fefb3400640 1 --2- 172.21.15.45:0/247607431 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ce3353aad0 0x55ce3354f8f0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fefa0032570 tx=0x7fefa0035e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9956d69142e8e364 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:07:28.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.599+0000 7fef9e000640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-16T17:07:28.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.599+0000 7fef9e000640 1 -- 172.21.15.45:0/247607431 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefa0059070 con 0x55ce3353aad0 2024-04-16T17:07:28.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.599+0000 7fefbdb89740 1 -- 172.21.15.45:0/247607431 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fefa0045040 con 0x55ce3353aad0 2024-04-16T17:07:28.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.599+0000 7fef9e000640 1 -- 172.21.15.45:0/247607431 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fefa002c830 con 0x55ce3353aad0 2024-04-16T17:07:28.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.600+0000 7fef9e000640 1 -- 172.21.15.45:0/247607431 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fefa0067050 con 0x55ce3353aad0 2024-04-16T17:07:28.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.600+0000 7fefbdb89740 10 client.15675 fetch_fsmap learned FSMap version 25 2024-04-16T17:07:28.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.600+0000 7fefbdb89740 1 -- 172.21.15.45:0/247607431 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55ce335c2110 con 0x55ce3353aad0 2024-04-16T17:07:28.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.600+0000 7fef9e000640 1 -- 172.21.15.45:0/247607431 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fefa0045040 con 0x55ce3353aad0 2024-04-16T17:07:28.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.601+0000 7fef9e000640 1 -- 172.21.15.45:0/247607431 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fefa004a040 con 0x55ce3353aad0 2024-04-16T17:07:28.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.601+0000 7fefbdb89740 10 client.15675 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T17:07:28.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.601+0000 7fefbdb89740 10 client.15675 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:07:28.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.601+0000 7fefbdb89740 20 client.15675 populate_metadata read hostname 'smithi045' 2024-04-16T17:07:28.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.601+0000 7fefbdb89740 1 --2- 172.21.15.45:0/247607431 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55ce335c27c0 0x55ce335c4be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:28.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.601+0000 7fefbdb89740 4 client.15675 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-16T17:07:28.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.601+0000 7fefbdb89740 1 -- 172.21.15.45:0/247607431 --> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55ce33423420 con 0x55ce335c27c0 2024-04-16T17:07:28.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.601+0000 7fefb3e00640 1 --2- 172.21.15.45:0/247607431 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55ce335c27c0 0x55ce335c4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:28.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.602+0000 7fefb3e00640 1 --2- 172.21.15.45:0/247607431 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55ce335c27c0 0x55ce335c4be0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=161eef60396febbd server_cookie=a8ab59a7f06b9cc4 in_seq=0 out_seq=0 2024-04-16T17:07:28.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.602+0000 7fef9e000640 10 client.15675 ms_handle_connect on v2:172.21.15.104:6838/3660070592 2024-04-16T17:07:28.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.602+0000 7fef9e000640 1 -- 172.21.15.45:0/247607431 <== mds.0 v2:172.21.15.104:6838/3660070592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55ce33423420 con 0x55ce335c27c0 2024-04-16T17:07:28.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.602+0000 7fef9e000640 10 client.15675 handle_command_reply: tid=0 2024-04-16T17:07:28.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.602+0000 7fefbdb89740 1 client.15675 shutdown 2024-04-16T17:07:28.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.602+0000 7fefbdb89740 1 -- 172.21.15.45:0/247607431 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ce3353aad0 msgr2=0x55ce3354f8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:28.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.602+0000 7fefbdb89740 1 --2- 172.21.15.45:0/247607431 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ce3353aad0 0x55ce3354f8f0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fefa0032570 tx=0x7fefa0035e70 comp rx=0 tx=0).stop 2024-04-16T17:07:28.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.603+0000 7fefbdb89740 1 -- 172.21.15.45:0/247607431 shutdown_connections 2024-04-16T17:07:28.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.603+0000 7fefbdb89740 1 --2- 172.21.15.45:0/247607431 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ce335436b0 0x55ce335ba710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:28.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.603+0000 7fefbdb89740 1 --2- 172.21.15.45:0/247607431 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ce3353aad0 0x55ce3354f8f0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:28.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.603+0000 7fefbdb89740 1 --2- 172.21.15.45:0/247607431 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55ce335c27c0 0x55ce335c4be0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:28.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.603+0000 7fefbdb89740 1 --2- 172.21.15.45:0/247607431 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ce3353a1b0 0x55ce3354f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:28.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.603+0000 7fefbdb89740 1 -- 172.21.15.45:0/247607431 >> 172.21.15.45:0/247607431 conn(0x55ce335348e0 msgr2=0x55ce33533040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:07:28.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.603+0000 7fef9e000640 0 client.15675 ms_handle_reset on v2:172.21.15.104:6838/3660070592 2024-04-16T17:07:28.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.603+0000 7fefbdb89740 1 -- 172.21.15.45:0/247607431 shutdown_connections 2024-04-16T17:07:28.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.603+0000 7fefbdb89740 1 -- 172.21.15.45:0/247607431 wait complete. 2024-04-16T17:07:28.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.603+0000 7fefbdb89740 20 client.15675 trim_cache size 0 max 16384 2024-04-16T17:07:28.607 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-16T17:07:28.607 INFO:teuthology.orchestra.run.smithi045.stdout: "status": "no active scrubs running", 2024-04-16T17:07:28.607 INFO:teuthology.orchestra.run.smithi045.stdout: "scrubs": {} 2024-04-16T17:07:28.607 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-16T17:07:28.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.604+0000 7fefbae00640 1 -- 172.21.15.45:0/3074539466 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fef980a3e40 msgr2=0x7fef980a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:28.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.604+0000 7fefbae00640 1 --2- 172.21.15.45:0/3074539466 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fef980a3e40 0x7fef980a6300 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fefa0009830 tx=0x7fefa0000bc0 comp rx=0 tx=0).stop 2024-04-16T17:07:28.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.604+0000 7fefbae00640 1 -- 172.21.15.45:0/3074539466 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fefb419a990 msgr2=0x7fefb419ae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:28.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.604+0000 7fefbae00640 1 --2- 172.21.15.45:0/3074539466 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fefb419a990 0x7fefb419ae40 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fefac009fd0 tx=0x7fefac00ef10 comp rx=0 tx=0).stop 2024-04-16T17:07:28.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.604+0000 7fefbae00640 1 -- 172.21.15.45:0/3074539466 shutdown_connections 2024-04-16T17:07:28.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.604+0000 7fefbae00640 1 --2- 172.21.15.45:0/3074539466 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fef980a3e40 0x7fef980a6300 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:28.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.604+0000 7fefbae00640 1 --2- 172.21.15.45:0/3074539466 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fefb419a990 0x7fefb419ae40 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:28.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.604+0000 7fefbae00640 1 --2- 172.21.15.45:0/3074539466 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fefb4107360 0x7fefb41a0d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:28.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.604+0000 7fefbae00640 1 --2- 172.21.15.45:0/3074539466 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fefb4102290 0x7fefb41a07e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:28.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.604+0000 7fefbae00640 1 -- 172.21.15.45:0/3074539466 >> 172.21.15.45:0/3074539466 conn(0x7fefb40fdb10 msgr2=0x7fefb40fede0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:07:28.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.604+0000 7fefbae00640 1 -- 172.21.15.45:0/3074539466 shutdown_connections 2024-04-16T17:07:28.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.604+0000 7fefbae00640 1 -- 172.21.15.45:0/3074539466 wait complete. 2024-04-16T17:07:28.628 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-16T17:07:28.628 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-16T17:07:28.677 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:07:28 smithi104 ceph-mon[36457]: pgmap v24: 129 pgs: 129 active+clean; 1.3 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-16T17:07:28.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.823+0000 7f26a7200640 1 -- 172.21.15.45:0/3574422655 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f26a8069db0 msgr2=0x7f26a8106a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:28.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.823+0000 7f26a7200640 1 --2- 172.21.15.45:0/3574422655 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f26a8069db0 0x7f26a8106a70 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f269400fb60 tx=0x7f269402fcf0 comp rx=0 tx=0).stop 2024-04-16T17:07:28.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.823+0000 7f26a7200640 1 -- 172.21.15.45:0/3574422655 shutdown_connections 2024-04-16T17:07:28.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.823+0000 7f26a7200640 1 --2- 172.21.15.45:0/3574422655 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f26a8069db0 0x7f26a8106a70 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:28.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.823+0000 7f26a7200640 1 --2- 172.21.15.45:0/3574422655 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f26a81071a0 0x7f26a810ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:28.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.823+0000 7f26a7200640 1 --2- 172.21.15.45:0/3574422655 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f26a8069400 0x7f26a80697e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:28.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.823+0000 7f26a7200640 1 -- 172.21.15.45:0/3574422655 >> 172.21.15.45:0/3574422655 conn(0x7f26a8076840 msgr2=0x7f26a8076c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:07:28.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.823+0000 7f26a7200640 1 -- 172.21.15.45:0/3574422655 shutdown_connections 2024-04-16T17:07:28.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.823+0000 7f26a7200640 1 -- 172.21.15.45:0/3574422655 wait complete. 2024-04-16T17:07:28.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.824+0000 7f26a7200640 1 Processor -- start 2024-04-16T17:07:28.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.824+0000 7f26a7200640 1 -- start start 2024-04-16T17:07:28.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.825+0000 7f26a7200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f26a8069400 0x7f26a8196530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:28.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.825+0000 7f26a7200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f26a81071a0 0x7f26a8196a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:28.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.825+0000 7f26a7200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f26a819adf0 0x7f26a819b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:28.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.825+0000 7f26a7200640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f26a8110010 con 0x7f26a819adf0 2024-04-16T17:07:28.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.825+0000 7f26a7200640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f26a810fe90 con 0x7f26a81071a0 2024-04-16T17:07:28.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.825+0000 7f26a7200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f26a8110190 con 0x7f26a8069400 2024-04-16T17:07:28.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.825+0000 7f26a5e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f26a8069400 0x7f26a8196530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:28.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.825+0000 7f26a5400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f26a81071a0 0x7f26a8196a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:28.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.825+0000 7f26a5e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f26a8069400 0x7f26a8196530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:58468/0 (socket says 172.21.15.45:58468) 2024-04-16T17:07:28.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.825+0000 7f26a5400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f26a81071a0 0x7f26a8196a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:35604/0 (socket says 172.21.15.45:35604) 2024-04-16T17:07:28.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.825+0000 7f26a5e00640 1 -- 172.21.15.45:0/1199200536 learned_addr learned my addr 172.21.15.45:0/1199200536 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:07:28.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.825+0000 7f26a6800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f26a819adf0 0x7f26a819b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:28.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.826+0000 7f26a5e00640 1 -- 172.21.15.45:0/1199200536 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f26a81071a0 msgr2=0x7f26a8196a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:28.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.826+0000 7f26a5e00640 1 --2- 172.21.15.45:0/1199200536 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f26a81071a0 0x7f26a8196a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:28.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.826+0000 7f26a5e00640 1 -- 172.21.15.45:0/1199200536 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f26a819adf0 msgr2=0x7f26a819b2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:28.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.826+0000 7f26a5e00640 1 --2- 172.21.15.45:0/1199200536 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f26a819adf0 0x7f26a819b2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:28.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.826+0000 7f26a5e00640 1 -- 172.21.15.45:0/1199200536 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26a819b8e0 con 0x7f26a8069400 2024-04-16T17:07:28.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.826+0000 7f26a6800640 1 --2- 172.21.15.45:0/1199200536 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f26a819adf0 0x7f26a819b2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:07:28.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.826+0000 7f26a5e00640 1 --2- 172.21.15.45:0/1199200536 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f26a8069400 0x7f26a8196530 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f269800e9c0 tx=0x7f269800ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:07:28.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.826+0000 7f269ea00640 1 -- 172.21.15.45:0/1199200536 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f269800cde0 con 0x7f26a8069400 2024-04-16T17:07:28.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.826+0000 7f269ea00640 1 -- 172.21.15.45:0/1199200536 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f26980035f0 con 0x7f26a8069400 2024-04-16T17:07:28.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.826+0000 7f269ea00640 1 -- 172.21.15.45:0/1199200536 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2698010690 con 0x7f26a8069400 2024-04-16T17:07:28.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.826+0000 7f26a7200640 1 -- 172.21.15.45:0/1199200536 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26a819bbd0 con 0x7f26a8069400 2024-04-16T17:07:28.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.827+0000 7f26a7200640 1 -- 172.21.15.45:0/1199200536 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26a81a3340 con 0x7f26a8069400 2024-04-16T17:07:28.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.828+0000 7f26a7200640 1 -- 172.21.15.45:0/1199200536 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26a8068d40 con 0x7f26a8069400 2024-04-16T17:07:28.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.831+0000 7f269ea00640 1 -- 172.21.15.45:0/1199200536 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 25) ==== 138879+0+0 (secure 0 0 0) 0x7f26980037a0 con 0x7f26a8069400 2024-04-16T17:07:28.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.833+0000 7f269ea00640 1 --2- 172.21.15.45:0/1199200536 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f268c0a3e40 0x7f268c0a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:28.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.833+0000 7f269ea00640 1 -- 172.21.15.45:0/1199200536 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f26980d1230 con 0x7f26a8069400 2024-04-16T17:07:28.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.833+0000 7f26a5400640 1 --2- 172.21.15.45:0/1199200536 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f268c0a3e40 0x7f268c0a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:28.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.834+0000 7f26a5400640 1 --2- 172.21.15.45:0/1199200536 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f268c0a3e40 0x7f268c0a6300 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f269403b040 tx=0x7f26940056d0 comp rx=0 tx=0).ready entity=mgr.25411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:07:28.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:28.837+0000 7f269ea00640 1 -- 172.21.15.45:0/1199200536 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f269809d2d0 con 0x7f26a8069400 2024-04-16T17:07:29.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.260+0000 7f26a7200640 1 -- 172.21.15.45:0/1199200536 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f26a8069db0 con 0x7f26a8069400 2024-04-16T17:07:29.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.261+0000 7f269ea00640 1 -- 172.21.15.45:0/1199200536 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11887 (secure 0 0 0) 0x7f269809cbd0 con 0x7f26a8069400 2024-04-16T17:07:29.263 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-16T17:07:29.263 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/1748842443","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":1748842443},{"type":"v1","addr":"172.21.15.45:6835","nonce":1748842443}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6839/1308627051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":1308627051},{"type":"v1","addr":"172.21.15.45:6839","nonce":1308627051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/1768242489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1768242489},{"type":"v1","addr":"172.21.15.117:6833","nonce":1768242489}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6833/3558902585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":3558902585},{"type":"v1","addr":"172.21.15.104:6833","nonce":3558902585}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24478,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/1308142903","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":1308142903},{"type":"v1","addr":"172.21.15.45:6837","nonce":1308142903}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24491,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6835/3430679937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":3430679937},{"type":"v1","addr":"172.21.15.104:6835","nonce":3430679937}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-16T17:03:24.663611+0000","modified":"2024-04-16T17:06:44.485317+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24581,"mds_1":24569,"mds_2":24532},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24499":{"gid":24499,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/2224123841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2224123841},{"type":"v1","addr":"172.21.15.117:6835","nonce":2224123841}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6837/2471356815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":2471356815},{"type":"v1","addr":"172.21.15.117:6837","nonce":2471356815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24524":{"gid":24524,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6837/2647756509","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":2647756509},{"type":"v1","addr":"172.21.15.104:6837","nonce":2647756509}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24532":{"gid":24532,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":13,"addr":"172.21.15.117:6839/2378675949","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":2378675949},{"type":"v1","addr":"172.21.15.117:6839","nonce":2378675949}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24569":{"gid":24569,"name":"a","rank":1,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.45:6841/3755310822","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6840","nonce":3755310822},{"type":"v1","addr":"172.21.15.45:6841","nonce":3755310822}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24581":{"gid":24581,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.104:6839/3660070592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6838","nonce":3660070592},{"type":"v1","addr":"172.21.15.104:6839","nonce":3660070592}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24581,"qdb_cluster":[24569,24581,24532]},"id":1}]} 2024-04-16T17:07:29.263 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 25 2024-04-16T17:07:29.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.265+0000 7f26a7200640 1 -- 172.21.15.45:0/1199200536 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f268c0a3e40 msgr2=0x7f268c0a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:29.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.265+0000 7f26a7200640 1 --2- 172.21.15.45:0/1199200536 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f268c0a3e40 0x7f268c0a6300 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f269403b040 tx=0x7f26940056d0 comp rx=0 tx=0).stop 2024-04-16T17:07:29.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.265+0000 7f26a7200640 1 -- 172.21.15.45:0/1199200536 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f26a8069400 msgr2=0x7f26a8196530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:29.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.265+0000 7f26a7200640 1 --2- 172.21.15.45:0/1199200536 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f26a8069400 0x7f26a8196530 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f269800e9c0 tx=0x7f269800ee80 comp rx=0 tx=0).stop 2024-04-16T17:07:29.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.266+0000 7f26a7200640 1 -- 172.21.15.45:0/1199200536 shutdown_connections 2024-04-16T17:07:29.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.266+0000 7f26a7200640 1 --2- 172.21.15.45:0/1199200536 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f268c0a3e40 0x7f268c0a6300 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:29.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.266+0000 7f26a7200640 1 --2- 172.21.15.45:0/1199200536 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f26a819adf0 0x7f26a819b2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:29.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.266+0000 7f26a7200640 1 --2- 172.21.15.45:0/1199200536 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f26a81071a0 0x7f26a8196a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:29.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.266+0000 7f26a7200640 1 --2- 172.21.15.45:0/1199200536 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f26a8069400 0x7f26a8196530 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:29.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.266+0000 7f26a7200640 1 -- 172.21.15.45:0/1199200536 >> 172.21.15.45:0/1199200536 conn(0x7f26a8076840 msgr2=0x7f26a81082d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:07:29.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.266+0000 7f26a7200640 1 -- 172.21.15.45:0/1199200536 shutdown_connections 2024-04-16T17:07:29.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.266+0000 7f26a7200640 1 -- 172.21.15.45:0/1199200536 wait complete. 2024-04-16T17:07:29.291 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-16T17:07:29.555 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:07:29 smithi117 ceph-mon[36730]: scrub queued for path: / 2024-04-16T17:07:29.555 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:07:29 smithi117 ceph-mon[36730]: scrub summary: idle+waiting paths [/] 2024-04-16T17:07:29.555 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:07:29 smithi117 ceph-mon[36730]: scrub summary: active paths [/] 2024-04-16T17:07:29.555 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:07:29 smithi117 ceph-mon[36730]: scrub summary: idle+waiting paths [/] 2024-04-16T17:07:29.555 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:07:29 smithi117 ceph-mon[36730]: scrub summary: idle 2024-04-16T17:07:29.555 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:07:29 smithi117 ceph-mon[36730]: from='client.? 172.21.15.45:0/1199200536' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:07:29.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.558+0000 7f3293200640 1 -- 172.21.15.45:0/1685362493 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f328c074cc0 msgr2=0x7f328c112970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:29.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.558+0000 7f3293200640 1 --2- 172.21.15.45:0/1685362493 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f328c074cc0 0x7f328c112970 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f327c009860 tx=0x7f327c02bc10 comp rx=0 tx=0).stop 2024-04-16T17:07:29.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.558+0000 7f3293200640 1 -- 172.21.15.45:0/1685362493 shutdown_connections 2024-04-16T17:07:29.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.558+0000 7f3293200640 1 --2- 172.21.15.45:0/1685362493 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f328c074cc0 0x7f328c112970 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:29.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.558+0000 7f3293200640 1 --2- 172.21.15.45:0/1685362493 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f328c112eb0 0x7f328c1152a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:29.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.558+0000 7f3293200640 1 --2- 172.21.15.45:0/1685362493 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f328c0743a0 0x7f328c074780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:29.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.558+0000 7f3293200640 1 -- 172.21.15.45:0/1685362493 >> 172.21.15.45:0/1685362493 conn(0x7f328c0fd9d0 msgr2=0x7f328c0ffdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:07:29.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.558+0000 7f3293200640 1 -- 172.21.15.45:0/1685362493 shutdown_connections 2024-04-16T17:07:29.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.558+0000 7f3293200640 1 -- 172.21.15.45:0/1685362493 wait complete. 2024-04-16T17:07:29.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.559+0000 7f3293200640 1 Processor -- start 2024-04-16T17:07:29.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.560+0000 7f3293200640 1 -- start start 2024-04-16T17:07:29.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.560+0000 7f3293200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f328c0743a0 0x7f328c19ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:29.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.560+0000 7f3293200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f328c074cc0 0x7f328c19f380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:29.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.560+0000 7f3293200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f328c112eb0 0x7f328c1a36b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:29.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.560+0000 7f3293200640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f328c118880 con 0x7f328c0743a0 2024-04-16T17:07:29.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.560+0000 7f3293200640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f328c118700 con 0x7f328c112eb0 2024-04-16T17:07:29.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.560+0000 7f3293200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f328c118a00 con 0x7f328c074cc0 2024-04-16T17:07:29.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.560+0000 7f3291e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f328c0743a0 0x7f328c19ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:29.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.560+0000 7f3291e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f328c0743a0 0x7f328c19ee40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45458/0 (socket says 172.21.15.45:45458) 2024-04-16T17:07:29.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.560+0000 7f3291e00640 1 -- 172.21.15.45:0/2206493904 learned_addr learned my addr 172.21.15.45:0/2206493904 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:07:29.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.561+0000 7f3291400640 1 --2- 172.21.15.45:0/2206493904 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f328c074cc0 0x7f328c19f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:29.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.561+0000 7f3291e00640 1 -- 172.21.15.45:0/2206493904 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f328c074cc0 msgr2=0x7f328c19f380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:29.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.561+0000 7f3291e00640 1 --2- 172.21.15.45:0/2206493904 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f328c074cc0 0x7f328c19f380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:29.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.561+0000 7f3291e00640 1 -- 172.21.15.45:0/2206493904 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f328c112eb0 msgr2=0x7f328c1a36b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:29.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.561+0000 7f3291e00640 1 --2- 172.21.15.45:0/2206493904 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f328c112eb0 0x7f328c1a36b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:29.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.561+0000 7f3291e00640 1 -- 172.21.15.45:0/2206493904 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f328c1a3de0 con 0x7f328c0743a0 2024-04-16T17:07:29.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.561+0000 7f3291e00640 1 --2- 172.21.15.45:0/2206493904 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f328c0743a0 0x7f328c19ee40 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f328000e9c0 tx=0x7f328000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:07:29.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.561+0000 7f3286a00640 1 -- 172.21.15.45:0/2206493904 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f328000ccc0 con 0x7f328c0743a0 2024-04-16T17:07:29.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.561+0000 7f3293200640 1 -- 172.21.15.45:0/2206493904 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f328c1a40d0 con 0x7f328c0743a0 2024-04-16T17:07:29.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.562+0000 7f3286a00640 1 -- 172.21.15.45:0/2206493904 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3280005500 con 0x7f328c0743a0 2024-04-16T17:07:29.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.562+0000 7f3286a00640 1 -- 172.21.15.45:0/2206493904 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3280010430 con 0x7f328c0743a0 2024-04-16T17:07:29.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.562+0000 7f3293200640 1 -- 172.21.15.45:0/2206493904 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f328c1ab960 con 0x7f328c0743a0 2024-04-16T17:07:29.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.565+0000 7f3297590740 1 Processor -- start 2024-04-16T17:07:29.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.566+0000 7f3297590740 1 -- start start 2024-04-16T17:07:29.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.566+0000 7f3297590740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5623bc2fff10 0x5623bc3002f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:29.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.566+0000 7f3297590740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5623bc300830 0x5623bc308ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:29.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.566+0000 7f3297590740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5623bc309410 0x5623bc30b800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:29.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.566+0000 7f3297590740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5623bc265020 con 0x5623bc309410 2024-04-16T17:07:29.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.566+0000 7f3297590740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5623bc20b460 con 0x5623bc2fff10 2024-04-16T17:07:29.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.566+0000 7f3297590740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5623bc260fe0 con 0x5623bc300830 2024-04-16T17:07:29.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.566+0000 7f3292800640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5623bc2fff10 0x5623bc3002f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:29.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.567+0000 7f3292800640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5623bc2fff10 0x5623bc3002f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:35628/0 (socket says 172.21.15.45:35628) 2024-04-16T17:07:29.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.567+0000 7f3292800640 1 -- 172.21.15.45:0/1388334478 learned_addr learned my addr 172.21.15.45:0/1388334478 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:07:29.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.567+0000 7f3291400640 1 --2- 172.21.15.45:0/1388334478 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5623bc309410 0x5623bc30b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:29.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.567+0000 7f3286a00640 1 -- 172.21.15.45:0/2206493904 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) ==== 138879+0+0 (secure 0 0 0) 0x7f32800046c0 con 0x7f328c0743a0 2024-04-16T17:07:29.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.567+0000 7f3292800640 1 -- 172.21.15.45:0/1388334478 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5623bc300830 msgr2=0x5623bc308ed0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-16T17:07:29.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.567+0000 7f3292800640 1 --2- 172.21.15.45:0/1388334478 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5623bc300830 0x5623bc308ed0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:29.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.567+0000 7f3292800640 1 -- 172.21.15.45:0/1388334478 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5623bc309410 msgr2=0x5623bc30b800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:07:29.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.567+0000 7f3292800640 1 --2- 172.21.15.45:0/1388334478 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5623bc309410 0x5623bc30b800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:29.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.567+0000 7f3292800640 1 -- 172.21.15.45:0/1388334478 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5623bc213040 con 0x5623bc2fff10 2024-04-16T17:07:29.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.567+0000 7f3291400640 1 --2- 172.21.15.45:0/1388334478 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5623bc309410 0x5623bc30b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:07:29.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.567+0000 7f3292800640 1 --2- 172.21.15.45:0/1388334478 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5623bc2fff10 0x5623bc3002f0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f3288009d20 tx=0x7f328800d6b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=779f028ff2483092 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:07:29.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.568+0000 7f327b000640 1 -- 172.21.15.45:0/1388334478 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3288016070 con 0x5623bc2fff10 2024-04-16T17:07:29.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.568+0000 7f327b000640 1 -- 172.21.15.45:0/1388334478 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3288003970 con 0x5623bc2fff10 2024-04-16T17:07:29.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.568+0000 7f327b000640 1 -- 172.21.15.45:0/1388334478 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3288004630 con 0x5623bc2fff10 2024-04-16T17:07:29.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.568+0000 7f3297590740 1 -- 172.21.15.45:0/1388334478 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5623bc2fff10 msgr2=0x5623bc3002f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:29.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.568+0000 7f3297590740 1 --2- 172.21.15.45:0/1388334478 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5623bc2fff10 0x5623bc3002f0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f3288009d20 tx=0x7f328800d6b0 comp rx=0 tx=0).stop 2024-04-16T17:07:29.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.569+0000 7f3297590740 1 -- 172.21.15.45:0/1388334478 shutdown_connections 2024-04-16T17:07:29.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.569+0000 7f3297590740 1 --2- 172.21.15.45:0/1388334478 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5623bc300830 0x5623bc308ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:29.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.569+0000 7f3297590740 1 --2- 172.21.15.45:0/1388334478 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5623bc309410 0x5623bc30b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:29.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.569+0000 7f3297590740 1 --2- 172.21.15.45:0/1388334478 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5623bc2fff10 0x5623bc3002f0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:29.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.569+0000 7f3297590740 1 -- 172.21.15.45:0/1388334478 >> 172.21.15.45:0/1388334478 conn(0x5623bc2fa640 msgr2=0x5623bc2faa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:07:29.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.569+0000 7f3297590740 1 -- 172.21.15.45:0/1388334478 shutdown_connections 2024-04-16T17:07:29.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.569+0000 7f3297590740 1 -- 172.21.15.45:0/1388334478 wait complete. 2024-04-16T17:07:29.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.569+0000 7f3286a00640 1 --2- 172.21.15.45:0/2206493904 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f32740a3d50 0x7f32740a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:29.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.569+0000 7f3286a00640 1 -- 172.21.15.45:0/2206493904 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f32800d1260 con 0x7f328c0743a0 2024-04-16T17:07:29.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.570+0000 7f3297590740 1 -- start start 2024-04-16T17:07:29.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.570+0000 7f3297590740 1 Processor -- start 2024-04-16T17:07:29.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.570+0000 7f3291400640 1 --2- 172.21.15.45:0/2206493904 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f32740a3d50 0x7f32740a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:29.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.570+0000 7f3297590740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5623bc2fff10 0x5623bc306530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:29.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.570+0000 7f3297590740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5623bc300830 0x5623bc306a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:29.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.570+0000 7f3297590740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5623bc309410 0x5623bc306fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:29.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.570+0000 7f3297590740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5623bc30c200 con 0x5623bc300830 2024-04-16T17:07:29.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.570+0000 7f3297590740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5623bc21e830 con 0x5623bc309410 2024-04-16T17:07:29.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.570+0000 7f3297590740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5623bc30c380 con 0x5623bc2fff10 2024-04-16T17:07:29.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.570+0000 7f3297590740 10 client.0 _ll_register_callbacks cb 0x5623bc226150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:07:29.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.571+0000 7f3291e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5623bc2fff10 0x5623bc306530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:29.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.571+0000 7f3291e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5623bc2fff10 0x5623bc306530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:58504/0 (socket says 172.21.15.45:58504) 2024-04-16T17:07:29.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.571+0000 7f3291e00640 1 -- 172.21.15.45:0/4263160403 learned_addr learned my addr 172.21.15.45:0/4263160403 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:07:29.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.571+0000 7f3292800640 1 --2- 172.21.15.45:0/4263160403 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5623bc309410 0x5623bc306fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:29.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.571+0000 7f3291e00640 1 -- 172.21.15.45:0/4263160403 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5623bc309410 msgr2=0x5623bc306fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:07:29.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.571+0000 7f3291e00640 1 --2- 172.21.15.45:0/4263160403 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5623bc309410 0x5623bc306fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:29.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.571+0000 7f3291e00640 1 -- 172.21.15.45:0/4263160403 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5623bc300830 msgr2=0x5623bc306a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:07:29.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.571+0000 7f3291e00640 1 --2- 172.21.15.45:0/4263160403 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5623bc300830 0x5623bc306a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:29.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.571+0000 7f3291e00640 1 -- 172.21.15.45:0/4263160403 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5623bc222920 con 0x5623bc2fff10 2024-04-16T17:07:29.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.571+0000 7f3291e00640 1 --2- 172.21.15.45:0/4263160403 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5623bc2fff10 0x5623bc306530 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f328009c900 tx=0x7f328009c690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c9fd4c498145096d server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:07:29.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.571+0000 7f3291400640 1 --2- 172.21.15.45:0/2206493904 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f32740a3d50 0x7f32740a6210 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f327c000c00 tx=0x7f327c035e40 comp rx=0 tx=0).ready entity=mgr.25411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:07:29.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.571+0000 7f326f400640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-04-16T17:07:29.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.571+0000 7f326f400640 1 -- 172.21.15.45:0/4263160403 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32800b6060 con 0x5623bc2fff10 2024-04-16T17:07:29.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.572+0000 7f326f400640 1 -- 172.21.15.45:0/4263160403 <== mon.1 v2:172.21.15.117:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f32800a53e0 con 0x5623bc2fff10 2024-04-16T17:07:29.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.572+0000 7f3297590740 1 -- 172.21.15.45:0/4263160403 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3288003970 con 0x5623bc2fff10 2024-04-16T17:07:29.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.572+0000 7f326f400640 1 -- 172.21.15.45:0/4263160403 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f32800a0fb0 con 0x5623bc2fff10 2024-04-16T17:07:29.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.572+0000 7f3297590740 10 client.25825 fetch_fsmap learned FSMap version 25 2024-04-16T17:07:29.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.572+0000 7f3297590740 1 -- 172.21.15.45:0/4263160403 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5623bc1f07f0 con 0x5623bc2fff10 2024-04-16T17:07:29.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.572+0000 7f326f400640 1 -- 172.21.15.45:0/4263160403 <== mon.1 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3288003970 con 0x5623bc2fff10 2024-04-16T17:07:29.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.573+0000 7f326f400640 1 -- 172.21.15.45:0/4263160403 <== mon.1 v2:172.21.15.117:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f32800ce880 con 0x5623bc2fff10 2024-04-16T17:07:29.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.573+0000 7f3297590740 10 client.25825 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T17:07:29.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.573+0000 7f3297590740 10 client.25825 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-16T17:07:29.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.573+0000 7f3297590740 20 client.25825 populate_metadata read hostname 'smithi045' 2024-04-16T17:07:29.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.573+0000 7f3297590740 1 --2- 172.21.15.45:0/4263160403 >> [v2:172.21.15.117:6838/2378675949,v1:172.21.15.117:6839/2378675949] conn(0x5623bc3198a0 0x5623bc31bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:29.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.573+0000 7f3297590740 4 client.25825 mds_command: new command op to 24532 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:07:29.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.573+0000 7f3297590740 1 -- 172.21.15.45:0/4263160403 --> [v2:172.21.15.117:6838/2378675949,v1:172.21.15.117:6839/2378675949] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5623bc272a40 con 0x5623bc3198a0 2024-04-16T17:07:29.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.573+0000 7f3291400640 1 --2- 172.21.15.45:0/4263160403 >> [v2:172.21.15.117:6838/2378675949,v1:172.21.15.117:6839/2378675949] conn(0x5623bc3198a0 0x5623bc31bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:29.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.574+0000 7f3291400640 1 --2- 172.21.15.45:0/4263160403 >> [v2:172.21.15.117:6838/2378675949,v1:172.21.15.117:6839/2378675949] conn(0x5623bc3198a0 0x5623bc31bcb0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b1b331809023d41c server_cookie=26f57c67e19f97ad in_seq=0 out_seq=0 2024-04-16T17:07:29.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.574+0000 7f326f400640 10 client.25825 ms_handle_connect on v2:172.21.15.117:6838/2378675949 2024-04-16T17:07:29.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.574+0000 7f326f400640 1 -- 172.21.15.45:0/4263160403 <== mds.2 v2:172.21.15.117:6838/2378675949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5623bc272a40 con 0x5623bc3198a0 2024-04-16T17:07:29.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.574+0000 7f326f400640 10 client.25825 handle_command_reply: tid=0 2024-04-16T17:07:29.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.575+0000 7f3297590740 1 client.25825 shutdown 2024-04-16T17:07:29.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.575+0000 7f3297590740 1 -- 172.21.15.45:0/4263160403 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5623bc2fff10 msgr2=0x5623bc306530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:29.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.575+0000 7f3297590740 1 --2- 172.21.15.45:0/4263160403 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5623bc2fff10 0x5623bc306530 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f328009c900 tx=0x7f328009c690 comp rx=0 tx=0).stop 2024-04-16T17:07:29.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.575+0000 7f3297590740 1 -- 172.21.15.45:0/4263160403 shutdown_connections 2024-04-16T17:07:29.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.575+0000 7f3297590740 1 --2- 172.21.15.45:0/4263160403 >> [v2:172.21.15.117:6838/2378675949,v1:172.21.15.117:6839/2378675949] conn(0x5623bc3198a0 0x5623bc31bcb0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:29.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.575+0000 7f3297590740 1 --2- 172.21.15.45:0/4263160403 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5623bc309410 0x5623bc306fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:29.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.575+0000 7f3297590740 1 --2- 172.21.15.45:0/4263160403 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5623bc300830 0x5623bc306a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:29.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.575+0000 7f3297590740 1 --2- 172.21.15.45:0/4263160403 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5623bc2fff10 0x5623bc306530 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:29.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.575+0000 7f326f400640 0 client.25825 ms_handle_reset on v2:172.21.15.117:6838/2378675949 2024-04-16T17:07:29.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.575+0000 7f3297590740 1 -- 172.21.15.45:0/4263160403 >> 172.21.15.45:0/4263160403 conn(0x5623bc2fa640 msgr2=0x5623bc2fc390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:07:29.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.575+0000 7f3297590740 1 -- 172.21.15.45:0/4263160403 shutdown_connections 2024-04-16T17:07:29.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.575+0000 7f3297590740 1 -- 172.21.15.45:0/4263160403 wait complete. 2024-04-16T17:07:29.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.575+0000 7f3297590740 20 client.25825 trim_cache size 0 max 16384 2024-04-16T17:07:29.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.590+0000 7f3297590740 1 Processor -- start 2024-04-16T17:07:29.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.590+0000 7f3297590740 1 -- start start 2024-04-16T17:07:29.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.590+0000 7f3297590740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5623bc2fff10 0x5623bc3150c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:29.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.590+0000 7f3297590740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5623bc300830 0x5623bc2f8e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:29.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.590+0000 7f3297590740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5623bc309410 0x5623bc306dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:29.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.590+0000 7f3297590740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f328c1b8220 con 0x5623bc300830 2024-04-16T17:07:29.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.590+0000 7f3297590740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f328c1b5af0 con 0x5623bc2fff10 2024-04-16T17:07:29.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.590+0000 7f3297590740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f328c1b3920 con 0x5623bc309410 2024-04-16T17:07:29.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.590+0000 7f3291e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5623bc2fff10 0x5623bc3150c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:29.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.590+0000 7f3291e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5623bc2fff10 0x5623bc3150c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:35650/0 (socket says 172.21.15.45:35650) 2024-04-16T17:07:29.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.590+0000 7f3291e00640 1 -- 172.21.15.45:0/3411676014 learned_addr learned my addr 172.21.15.45:0/3411676014 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:07:29.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.590+0000 7f3291400640 1 --2- 172.21.15.45:0/3411676014 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5623bc300830 0x5623bc2f8e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:29.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.591+0000 7f3292800640 1 --2- 172.21.15.45:0/3411676014 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5623bc309410 0x5623bc306dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:29.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.591+0000 7f3291e00640 1 -- 172.21.15.45:0/3411676014 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5623bc309410 msgr2=0x5623bc306dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:07:29.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.591+0000 7f3291e00640 1 --2- 172.21.15.45:0/3411676014 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5623bc309410 0x5623bc306dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:29.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.591+0000 7f3291e00640 1 -- 172.21.15.45:0/3411676014 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5623bc300830 msgr2=0x5623bc2f8e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:07:29.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.591+0000 7f3291e00640 1 --2- 172.21.15.45:0/3411676014 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5623bc300830 0x5623bc2f8e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:29.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.591+0000 7f3291e00640 1 -- 172.21.15.45:0/3411676014 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5623bc316040 con 0x5623bc2fff10 2024-04-16T17:07:29.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.591+0000 7f3291e00640 1 --2- 172.21.15.45:0/3411676014 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5623bc2fff10 0x5623bc3150c0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f32800a1170 tx=0x7f3280009f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cefb023646b63af server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:07:29.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.591+0000 7f327a600640 1 -- 172.21.15.45:0/3411676014 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32800056b0 con 0x5623bc2fff10 2024-04-16T17:07:29.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.591+0000 7f327a600640 1 -- 172.21.15.45:0/3411676014 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f32800ce3c0 con 0x5623bc2fff10 2024-04-16T17:07:29.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.591+0000 7f327a600640 1 -- 172.21.15.45:0/3411676014 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32800a53e0 con 0x5623bc2fff10 2024-04-16T17:07:29.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.592+0000 7f3297590740 1 -- 172.21.15.45:0/3411676014 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5623bc2fff10 msgr2=0x5623bc3150c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:29.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.592+0000 7f3297590740 1 --2- 172.21.15.45:0/3411676014 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5623bc2fff10 0x5623bc3150c0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f32800a1170 tx=0x7f3280009f90 comp rx=0 tx=0).stop 2024-04-16T17:07:29.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.592+0000 7f3297590740 1 -- 172.21.15.45:0/3411676014 shutdown_connections 2024-04-16T17:07:29.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.592+0000 7f3297590740 1 --2- 172.21.15.45:0/3411676014 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5623bc309410 0x5623bc306dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:29.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.592+0000 7f3297590740 1 --2- 172.21.15.45:0/3411676014 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5623bc300830 0x5623bc2f8e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:29.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.592+0000 7f3297590740 1 --2- 172.21.15.45:0/3411676014 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5623bc2fff10 0x5623bc3150c0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:29.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.592+0000 7f3297590740 1 -- 172.21.15.45:0/3411676014 >> 172.21.15.45:0/3411676014 conn(0x5623bc2fa640 msgr2=0x5623bc305560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:07:29.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.593+0000 7f3297590740 1 -- 172.21.15.45:0/3411676014 shutdown_connections 2024-04-16T17:07:29.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.593+0000 7f3297590740 1 -- 172.21.15.45:0/3411676014 wait complete. 2024-04-16T17:07:29.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.593+0000 7f3297590740 1 -- start start 2024-04-16T17:07:29.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.593+0000 7f3297590740 1 Processor -- start 2024-04-16T17:07:29.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.594+0000 7f3297590740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5623bc2fff10 0x5623bc3034a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:29.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.594+0000 7f3297590740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5623bc300830 0x5623bc3039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:29.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.594+0000 7f3297590740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5623bc309410 0x5623bc303f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:29.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.594+0000 7f3297590740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f328c1b3fb0 con 0x5623bc2fff10 2024-04-16T17:07:29.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.594+0000 7f3297590740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f328c1b5420 con 0x5623bc300830 2024-04-16T17:07:29.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.594+0000 7f3297590740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f328c1b4680 con 0x5623bc309410 2024-04-16T17:07:29.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.594+0000 7f3297590740 10 client.0 _ll_register_callbacks cb 0x7f3280004a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:07:29.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.595+0000 7f3291e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5623bc2fff10 0x5623bc3034a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:29.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.595+0000 7f3292800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5623bc309410 0x5623bc303f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:29.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.595+0000 7f3291400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5623bc300830 0x5623bc3039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:29.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.595+0000 7f3291400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5623bc300830 0x5623bc3039e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:35658/0 (socket says 172.21.15.45:35658) 2024-04-16T17:07:29.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.595+0000 7f3291400640 1 -- 172.21.15.45:0/3208983609 learned_addr learned my addr 172.21.15.45:0/3208983609 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:07:29.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.595+0000 7f3292800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5623bc309410 0x5623bc303f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:58530/0 (socket says 172.21.15.45:58530) 2024-04-16T17:07:29.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.595+0000 7f3291e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5623bc2fff10 0x5623bc3034a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45502/0 (socket says 172.21.15.45:45502) 2024-04-16T17:07:29.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.595+0000 7f3291400640 1 -- 172.21.15.45:0/3208983609 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5623bc309410 msgr2=0x5623bc303f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:07:29.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.595+0000 7f3291400640 1 --2- 172.21.15.45:0/3208983609 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5623bc309410 0x5623bc303f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:29.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.595+0000 7f3291400640 1 -- 172.21.15.45:0/3208983609 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5623bc2fff10 msgr2=0x5623bc3034a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:07:29.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.595+0000 7f3291400640 1 --2- 172.21.15.45:0/3208983609 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5623bc2fff10 0x5623bc3034a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:29.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.595+0000 7f3291400640 1 -- 172.21.15.45:0/3208983609 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5623bbf773f0 con 0x5623bc300830 2024-04-16T17:07:29.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.595+0000 7f3292800640 1 --2- 172.21.15.45:0/3208983609 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5623bc309410 0x5623bc303f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:07:29.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.595+0000 7f3291400640 1 --2- 172.21.15.45:0/3208983609 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5623bc300830 0x5623bc3039e0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f327c035fd0 tx=0x7f327c02bc10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a478ca3b5a695bf3 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:07:29.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.595+0000 7f326f400640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-04-16T17:07:29.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.596+0000 7f326f400640 1 -- 172.21.15.45:0/3208983609 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f327c039040 con 0x5623bc300830 2024-04-16T17:07:29.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.596+0000 7f3297590740 1 -- 172.21.15.45:0/3208983609 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f32800ce3c0 con 0x5623bc300830 2024-04-16T17:07:29.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.596+0000 7f326f400640 1 -- 172.21.15.45:0/3208983609 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f327c034c10 con 0x5623bc300830 2024-04-16T17:07:29.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.596+0000 7f326f400640 1 -- 172.21.15.45:0/3208983609 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f327c060050 con 0x5623bc300830 2024-04-16T17:07:29.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.596+0000 7f3297590740 10 client.25829 fetch_fsmap learned FSMap version 25 2024-04-16T17:07:29.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.596+0000 7f3297590740 1 -- 172.21.15.45:0/3208983609 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5623bc0417d0 con 0x5623bc300830 2024-04-16T17:07:29.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.597+0000 7f326f400640 1 -- 172.21.15.45:0/3208983609 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f32800ce3c0 con 0x5623bc300830 2024-04-16T17:07:29.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.597+0000 7f326f400640 1 -- 172.21.15.45:0/3208983609 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f327c055020 con 0x5623bc300830 2024-04-16T17:07:29.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.597+0000 7f3297590740 10 client.25829 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T17:07:29.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.597+0000 7f3297590740 10 client.25829 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-16T17:07:29.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.597+0000 7f3297590740 20 client.25829 populate_metadata read hostname 'smithi045' 2024-04-16T17:07:29.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.597+0000 7f3297590740 1 --2- 172.21.15.45:0/3208983609 >> [v2:172.21.15.117:6838/2378675949,v1:172.21.15.117:6839/2378675949] conn(0x5623bc380770 0x5623bc382b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:29.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.597+0000 7f3297590740 4 client.25829 mds_command: new command op to 24532 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T17:07:29.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.597+0000 7f3297590740 1 -- 172.21.15.45:0/3208983609 --> [v2:172.21.15.117:6838/2378675949,v1:172.21.15.117:6839/2378675949] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5623bc26c8b0 con 0x5623bc380770 2024-04-16T17:07:29.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.598+0000 7f3291e00640 1 --2- 172.21.15.45:0/3208983609 >> [v2:172.21.15.117:6838/2378675949,v1:172.21.15.117:6839/2378675949] conn(0x5623bc380770 0x5623bc382b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:29.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.598+0000 7f3291e00640 1 --2- 172.21.15.45:0/3208983609 >> [v2:172.21.15.117:6838/2378675949,v1:172.21.15.117:6839/2378675949] conn(0x5623bc380770 0x5623bc382b90 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f3fe37dc9e15c7a6 server_cookie=71d02f736e778ee0 in_seq=0 out_seq=0 2024-04-16T17:07:29.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.598+0000 7f326f400640 10 client.25829 ms_handle_connect on v2:172.21.15.117:6838/2378675949 2024-04-16T17:07:29.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.599+0000 7f326f400640 1 -- 172.21.15.45:0/3208983609 <== mds.2 v2:172.21.15.117:6838/2378675949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5623bc26c8b0 con 0x5623bc380770 2024-04-16T17:07:29.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.599+0000 7f326f400640 10 client.25829 handle_command_reply: tid=0 2024-04-16T17:07:29.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.599+0000 7f3297590740 1 client.25829 shutdown 2024-04-16T17:07:29.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.599+0000 7f3297590740 1 -- 172.21.15.45:0/3208983609 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5623bc300830 msgr2=0x5623bc3039e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:29.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.599+0000 7f3297590740 1 --2- 172.21.15.45:0/3208983609 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5623bc300830 0x5623bc3039e0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f327c035fd0 tx=0x7f327c02bc10 comp rx=0 tx=0).stop 2024-04-16T17:07:29.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.599+0000 7f3297590740 1 -- 172.21.15.45:0/3208983609 shutdown_connections 2024-04-16T17:07:29.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.599+0000 7f3297590740 1 --2- 172.21.15.45:0/3208983609 >> [v2:172.21.15.117:6838/2378675949,v1:172.21.15.117:6839/2378675949] conn(0x5623bc380770 0x5623bc382b90 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:29.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.599+0000 7f3297590740 1 --2- 172.21.15.45:0/3208983609 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5623bc309410 0x5623bc303f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:29.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.599+0000 7f3297590740 1 --2- 172.21.15.45:0/3208983609 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5623bc300830 0x5623bc3039e0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:29.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.599+0000 7f3297590740 1 --2- 172.21.15.45:0/3208983609 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5623bc2fff10 0x5623bc3034a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:29.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.599+0000 7f3297590740 1 -- 172.21.15.45:0/3208983609 >> 172.21.15.45:0/3208983609 conn(0x5623bc2fa640 msgr2=0x5623bc2fb940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:07:29.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.599+0000 7f326f400640 0 client.25829 ms_handle_reset on v2:172.21.15.117:6838/2378675949 2024-04-16T17:07:29.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.599+0000 7f3297590740 1 -- 172.21.15.45:0/3208983609 shutdown_connections 2024-04-16T17:07:29.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.599+0000 7f3297590740 1 -- 172.21.15.45:0/3208983609 wait complete. 2024-04-16T17:07:29.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.600+0000 7f3297590740 20 client.25829 trim_cache size 0 max 16384 2024-04-16T17:07:29.603 INFO:teuthology.orchestra.run.smithi045.stdout:[] 2024-04-16T17:07:29.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.600+0000 7f3293200640 1 -- 172.21.15.45:0/2206493904 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f32740a3d50 msgr2=0x7f32740a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:29.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.600+0000 7f3293200640 1 --2- 172.21.15.45:0/2206493904 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f32740a3d50 0x7f32740a6210 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f327c000c00 tx=0x7f327c035e40 comp rx=0 tx=0).stop 2024-04-16T17:07:29.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.600+0000 7f3293200640 1 -- 172.21.15.45:0/2206493904 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f328c0743a0 msgr2=0x7f328c19ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:29.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.600+0000 7f3293200640 1 --2- 172.21.15.45:0/2206493904 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f328c0743a0 0x7f328c19ee40 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f328000e9c0 tx=0x7f328000ee80 comp rx=0 tx=0).stop 2024-04-16T17:07:29.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.601+0000 7f3293200640 1 -- 172.21.15.45:0/2206493904 shutdown_connections 2024-04-16T17:07:29.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.601+0000 7f3293200640 1 --2- 172.21.15.45:0/2206493904 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f32740a3d50 0x7f32740a6210 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:29.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.601+0000 7f3293200640 1 --2- 172.21.15.45:0/2206493904 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f328c074cc0 0x7f328c19f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:29.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.601+0000 7f3293200640 1 --2- 172.21.15.45:0/2206493904 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f328c112eb0 0x7f328c1a36b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:29.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.601+0000 7f3293200640 1 --2- 172.21.15.45:0/2206493904 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f328c0743a0 0x7f328c19ee40 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:29.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.601+0000 7f3293200640 1 -- 172.21.15.45:0/2206493904 >> 172.21.15.45:0/2206493904 conn(0x7f328c0fd9d0 msgr2=0x7f328c0ff2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:07:29.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.601+0000 7f3293200640 1 -- 172.21.15.45:0/2206493904 shutdown_connections 2024-04-16T17:07:29.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.601+0000 7f3293200640 1 -- 172.21.15.45:0/2206493904 wait complete. 2024-04-16T17:07:29.621 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:07:29 smithi045 ceph-mon[31676]: scrub queued for path: / 2024-04-16T17:07:29.621 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:07:29 smithi045 ceph-mon[31676]: scrub summary: idle+waiting paths [/] 2024-04-16T17:07:29.621 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:07:29 smithi045 ceph-mon[31676]: scrub summary: active paths [/] 2024-04-16T17:07:29.621 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:07:29 smithi045 ceph-mon[31676]: scrub summary: idle+waiting paths [/] 2024-04-16T17:07:29.621 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:07:29 smithi045 ceph-mon[31676]: scrub summary: idle 2024-04-16T17:07:29.622 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:07:29 smithi045 ceph-mon[31676]: from='client.? 172.21.15.45:0/1199200536' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:07:29.625 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-16T17:07:29.677 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:07:29 smithi104 ceph-mon[36457]: scrub queued for path: / 2024-04-16T17:07:29.677 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:07:29 smithi104 ceph-mon[36457]: scrub summary: idle+waiting paths [/] 2024-04-16T17:07:29.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:07:29 smithi104 ceph-mon[36457]: scrub summary: active paths [/] 2024-04-16T17:07:29.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:07:29 smithi104 ceph-mon[36457]: scrub summary: idle+waiting paths [/] 2024-04-16T17:07:29.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:07:29 smithi104 ceph-mon[36457]: scrub summary: idle 2024-04-16T17:07:29.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:07:29 smithi104 ceph-mon[36457]: from='client.? 172.21.15.45:0/1199200536' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:07:29.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.810+0000 7f23ee400640 1 -- 172.21.15.45:0/3826322007 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f23e81039e0 msgr2=0x7f23e810bec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:29.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.810+0000 7f23ee400640 1 --2- 172.21.15.45:0/3826322007 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f23e81039e0 0x7f23e810bec0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f23d4009900 tx=0x7f23d402bcd0 comp rx=0 tx=0).stop 2024-04-16T17:07:29.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.810+0000 7f23ee400640 1 -- 172.21.15.45:0/3826322007 shutdown_connections 2024-04-16T17:07:29.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.810+0000 7f23ee400640 1 --2- 172.21.15.45:0/3826322007 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f23e81039e0 0x7f23e810bec0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:29.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.810+0000 7f23ee400640 1 --2- 172.21.15.45:0/3826322007 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f23e810c490 0x7f23e810e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:29.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.810+0000 7f23ee400640 1 --2- 172.21.15.45:0/3826322007 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f23e81030c0 0x7f23e81034a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:29.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.810+0000 7f23ee400640 1 -- 172.21.15.45:0/3826322007 >> 172.21.15.45:0/3826322007 conn(0x7f23e80fc880 msgr2=0x7f23e80feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:07:29.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.810+0000 7f23ee400640 1 -- 172.21.15.45:0/3826322007 shutdown_connections 2024-04-16T17:07:29.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.810+0000 7f23ee400640 1 -- 172.21.15.45:0/3826322007 wait complete. 2024-04-16T17:07:29.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.812+0000 7f23ee400640 1 Processor -- start 2024-04-16T17:07:29.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.812+0000 7f23ee400640 1 -- start start 2024-04-16T17:07:29.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.812+0000 7f23ee400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f23e81030c0 0x7f23e8100d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:29.814 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.812+0000 7f23ee400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f23e81039e0 0x7f23e8101250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:29.814 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.812+0000 7f23ee400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f23e810c490 0x7f23e8101790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:29.814 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.812+0000 7f23ee400640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f23e8112290 con 0x7f23e81030c0 2024-04-16T17:07:29.814 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.812+0000 7f23ee400640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f23e8112110 con 0x7f23e810c490 2024-04-16T17:07:29.814 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.812+0000 7f23ee400640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f23e8112410 con 0x7f23e81039e0 2024-04-16T17:07:29.814 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.813+0000 7f23ed000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f23e81030c0 0x7f23e8100d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:29.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.813+0000 7f23ed000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f23e81030c0 0x7f23e8100d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45522/0 (socket says 172.21.15.45:45522) 2024-04-16T17:07:29.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.813+0000 7f23ed000640 1 -- 172.21.15.45:0/3408274202 learned_addr learned my addr 172.21.15.45:0/3408274202 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:07:29.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.813+0000 7f23e7e00640 1 --2- 172.21.15.45:0/3408274202 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f23e81039e0 0x7f23e8101250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:29.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.813+0000 7f23eda00640 1 --2- 172.21.15.45:0/3408274202 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f23e810c490 0x7f23e8101790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:29.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.814+0000 7f23eda00640 1 -- 172.21.15.45:0/3408274202 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f23e81039e0 msgr2=0x7f23e8101250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:29.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.814+0000 7f23eda00640 1 --2- 172.21.15.45:0/3408274202 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f23e81039e0 0x7f23e8101250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:29.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.814+0000 7f23eda00640 1 -- 172.21.15.45:0/3408274202 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f23e81030c0 msgr2=0x7f23e8100d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:29.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.814+0000 7f23eda00640 1 --2- 172.21.15.45:0/3408274202 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f23e81030c0 0x7f23e8100d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:29.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.814+0000 7f23eda00640 1 -- 172.21.15.45:0/3408274202 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23e81a6c90 con 0x7f23e810c490 2024-04-16T17:07:29.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.814+0000 7f23e7e00640 1 --2- 172.21.15.45:0/3408274202 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f23e81039e0 0x7f23e8101250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:07:29.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.814+0000 7f23eda00640 1 --2- 172.21.15.45:0/3408274202 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f23e810c490 0x7f23e8101790 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f23e0009fd0 tx=0x7f23e000ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:07:29.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.814+0000 7f23e5600640 1 -- 172.21.15.45:0/3408274202 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23e000ce20 con 0x7f23e810c490 2024-04-16T17:07:29.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.814+0000 7f23ee400640 1 -- 172.21.15.45:0/3408274202 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23e81a6f80 con 0x7f23e810c490 2024-04-16T17:07:29.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.815+0000 7f23e5600640 1 -- 172.21.15.45:0/3408274202 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f23e0005020 con 0x7f23e810c490 2024-04-16T17:07:29.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.815+0000 7f23e5600640 1 -- 172.21.15.45:0/3408274202 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23e0010720 con 0x7f23e810c490 2024-04-16T17:07:29.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.815+0000 7f23ee400640 1 -- 172.21.15.45:0/3408274202 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23e81a7450 con 0x7f23e810c490 2024-04-16T17:07:29.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.819+0000 7f23f2651740 1 Processor -- start 2024-04-16T17:07:29.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.819+0000 7f23e5600640 1 -- 172.21.15.45:0/3408274202 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 25) ==== 138879+0+0 (secure 0 0 0) 0x7f23e00109c0 con 0x7f23e810c490 2024-04-16T17:07:29.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.820+0000 7f23f2651740 1 -- start start 2024-04-16T17:07:29.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.820+0000 7f23f2651740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cfb3766f10 0x55cfb37672f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:29.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.820+0000 7f23f2651740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55cfb3767830 0x55cfb376fed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:29.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.820+0000 7f23f2651740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cfb3770410 0x55cfb3772800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:29.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.820+0000 7f23f2651740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55cfb36cc020 con 0x55cfb3770410 2024-04-16T17:07:29.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.820+0000 7f23f2651740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55cfb3672460 con 0x55cfb3767830 2024-04-16T17:07:29.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.820+0000 7f23f2651740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55cfb36c7fe0 con 0x55cfb3766f10 2024-04-16T17:07:29.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.820+0000 7f23ed000640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55cfb3767830 0x55cfb376fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:29.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.820+0000 7f23ed000640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55cfb3767830 0x55cfb376fed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:35688/0 (socket says 172.21.15.45:35688) 2024-04-16T17:07:29.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.820+0000 7f23ed000640 1 -- 172.21.15.45:0/4092308508 learned_addr learned my addr 172.21.15.45:0/4092308508 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:07:29.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.821+0000 7f23ed000640 1 -- 172.21.15.45:0/4092308508 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cfb3766f10 msgr2=0x55cfb37672f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-16T17:07:29.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.821+0000 7f23ed000640 1 --2- 172.21.15.45:0/4092308508 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cfb3766f10 0x55cfb37672f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:29.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.821+0000 7f23ed000640 1 -- 172.21.15.45:0/4092308508 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cfb3770410 msgr2=0x55cfb3772800 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-16T17:07:29.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.821+0000 7f23ed000640 1 --2- 172.21.15.45:0/4092308508 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cfb3770410 0x55cfb3772800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:29.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.821+0000 7f23ed000640 1 -- 172.21.15.45:0/4092308508 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cfb367a040 con 0x55cfb3767830 2024-04-16T17:07:29.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.821+0000 7f23ed000640 1 --2- 172.21.15.45:0/4092308508 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55cfb3767830 0x55cfb376fed0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f23dc009dc0 tx=0x7f23dc011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=85a6c0057678a6ba server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:07:29.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.821+0000 7f23da000640 1 -- 172.21.15.45:0/4092308508 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23dc005220 con 0x55cfb3767830 2024-04-16T17:07:29.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.822+0000 7f23da000640 1 -- 172.21.15.45:0/4092308508 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f23dc004c80 con 0x55cfb3767830 2024-04-16T17:07:29.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.822+0000 7f23da000640 1 -- 172.21.15.45:0/4092308508 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23dc019970 con 0x55cfb3767830 2024-04-16T17:07:29.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.822+0000 7f23e5600640 1 --2- 172.21.15.45:0/3408274202 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f23d00a3ec0 0x7f23d00a6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:29.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.822+0000 7f23f2651740 1 -- 172.21.15.45:0/4092308508 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55cfb3767830 msgr2=0x55cfb376fed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:29.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.822+0000 7f23f2651740 1 --2- 172.21.15.45:0/4092308508 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55cfb3767830 0x55cfb376fed0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f23dc009dc0 tx=0x7f23dc011810 comp rx=0 tx=0).stop 2024-04-16T17:07:29.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.823+0000 7f23e5600640 1 -- 172.21.15.45:0/3408274202 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f23e00d1410 con 0x7f23e810c490 2024-04-16T17:07:29.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.823+0000 7f23f2651740 1 -- 172.21.15.45:0/4092308508 shutdown_connections 2024-04-16T17:07:29.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.823+0000 7f23e7e00640 1 --2- 172.21.15.45:0/3408274202 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f23d00a3ec0 0x7f23d00a6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:29.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.823+0000 7f23f2651740 1 --2- 172.21.15.45:0/4092308508 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cfb3770410 0x55cfb3772800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:29.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.823+0000 7f23f2651740 1 --2- 172.21.15.45:0/4092308508 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55cfb3767830 0x55cfb376fed0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:29.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.823+0000 7f23f2651740 1 --2- 172.21.15.45:0/4092308508 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cfb3766f10 0x55cfb37672f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:29.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.823+0000 7f23f2651740 1 -- 172.21.15.45:0/4092308508 >> 172.21.15.45:0/4092308508 conn(0x55cfb3761640 msgr2=0x55cfb3761a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:07:29.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.824+0000 7f23e7e00640 1 --2- 172.21.15.45:0/3408274202 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f23d00a3ec0 0x7f23d00a6380 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f23d4005ab0 tx=0x7f23d4005a20 comp rx=0 tx=0).ready entity=mgr.25411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:07:29.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.824+0000 7f23f2651740 1 -- 172.21.15.45:0/4092308508 shutdown_connections 2024-04-16T17:07:29.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.824+0000 7f23f2651740 1 -- 172.21.15.45:0/4092308508 wait complete. 2024-04-16T17:07:29.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.825+0000 7f23f2651740 1 -- start start 2024-04-16T17:07:29.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.825+0000 7f23f2651740 1 Processor -- start 2024-04-16T17:07:29.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.825+0000 7f23f2651740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55cfb3766f10 0x55cfb376d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:29.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.825+0000 7f23f2651740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cfb3767830 0x55cfb376d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:29.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.825+0000 7f23f2651740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cfb3770410 0x55cfb376de20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:29.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.825+0000 7f23f2651740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55cfb3784210 con 0x55cfb3767830 2024-04-16T17:07:29.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.825+0000 7f23f2651740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55cfb3685830 con 0x55cfb3766f10 2024-04-16T17:07:29.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.825+0000 7f23f2651740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55cfb3784390 con 0x55cfb3770410 2024-04-16T17:07:29.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.826+0000 7f23f2651740 10 client.0 _ll_register_callbacks cb 0x55cfb368d150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:07:29.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.826+0000 7f23ed000640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55cfb3766f10 0x55cfb376d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:29.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.826+0000 7f23ed000640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55cfb3766f10 0x55cfb376d3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:35690/0 (socket says 172.21.15.45:35690) 2024-04-16T17:07:29.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.826+0000 7f23ed000640 1 -- 172.21.15.45:0/3048521522 learned_addr learned my addr 172.21.15.45:0/3048521522 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:07:29.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.826+0000 7f23e7e00640 1 --2- 172.21.15.45:0/3048521522 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cfb3767830 0x55cfb376d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:29.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.826+0000 7f23eda00640 1 --2- 172.21.15.45:0/3048521522 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cfb3770410 0x55cfb376de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:29.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.826+0000 7f23ed000640 1 -- 172.21.15.45:0/3048521522 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cfb3770410 msgr2=0x55cfb376de20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:07:29.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.826+0000 7f23ed000640 1 --2- 172.21.15.45:0/3048521522 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cfb3770410 0x55cfb376de20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:29.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.826+0000 7f23ed000640 1 -- 172.21.15.45:0/3048521522 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cfb3767830 msgr2=0x55cfb376d8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:07:29.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.826+0000 7f23ed000640 1 --2- 172.21.15.45:0/3048521522 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cfb3767830 0x55cfb376d8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:29.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.826+0000 7f23ed000640 1 -- 172.21.15.45:0/3048521522 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cfb3689920 con 0x55cfb3766f10 2024-04-16T17:07:29.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.827+0000 7f23ed000640 1 --2- 172.21.15.45:0/3048521522 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55cfb3766f10 0x55cfb376d3a0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f23dc009350 tx=0x7f23dc004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3fd82f6014bcd82f server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:07:29.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.827+0000 7f23cea00640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-04-16T17:07:29.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.827+0000 7f23cea00640 1 -- 172.21.15.45:0/3048521522 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23dc005240 con 0x55cfb3766f10 2024-04-16T17:07:29.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.827+0000 7f23f2651740 1 -- 172.21.15.45:0/3048521522 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f23dc004c80 con 0x55cfb3766f10 2024-04-16T17:07:29.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.827+0000 7f23cea00640 1 -- 172.21.15.45:0/3048521522 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f23dc019670 con 0x55cfb3766f10 2024-04-16T17:07:29.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.827+0000 7f23cea00640 1 -- 172.21.15.45:0/3048521522 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f23dc032050 con 0x55cfb3766f10 2024-04-16T17:07:29.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.827+0000 7f23f2651740 10 client.25841 fetch_fsmap learned FSMap version 25 2024-04-16T17:07:29.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.827+0000 7f23f2651740 1 -- 172.21.15.45:0/3048521522 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55cfb36577f0 con 0x55cfb3766f10 2024-04-16T17:07:29.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.828+0000 7f23cea00640 1 -- 172.21.15.45:0/3048521522 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f23dc004c80 con 0x55cfb3766f10 2024-04-16T17:07:29.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.828+0000 7f23cea00640 1 -- 172.21.15.45:0/3048521522 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f23dc0035f0 con 0x55cfb3766f10 2024-04-16T17:07:29.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.828+0000 7f23f2651740 10 client.25841 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T17:07:29.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.828+0000 7f23f2651740 10 client.25841 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-04-16T17:07:29.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.828+0000 7f23f2651740 20 client.25841 populate_metadata read hostname 'smithi045' 2024-04-16T17:07:29.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.828+0000 7f23f2651740 1 --2- 172.21.15.45:0/3048521522 >> [v2:172.21.15.45:6840/3755310822,v1:172.21.15.45:6841/3755310822] conn(0x55cfb37916d0 0x55cfb3793ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:29.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.828+0000 7f23f2651740 4 client.25841 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:07:29.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.828+0000 7f23f2651740 1 -- 172.21.15.45:0/3048521522 --> [v2:172.21.15.45:6840/3755310822,v1:172.21.15.45:6841/3755310822] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cfb36d9a40 con 0x55cfb37916d0 2024-04-16T17:07:29.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.828+0000 7f23e7e00640 1 --2- 172.21.15.45:0/3048521522 >> [v2:172.21.15.45:6840/3755310822,v1:172.21.15.45:6841/3755310822] conn(0x55cfb37916d0 0x55cfb3793ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:29.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.829+0000 7f23e7e00640 1 --2- 172.21.15.45:0/3048521522 >> [v2:172.21.15.45:6840/3755310822,v1:172.21.15.45:6841/3755310822] conn(0x55cfb37916d0 0x55cfb3793ac0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=65ba7ba3d52e185a server_cookie=6756c346c503381f in_seq=0 out_seq=0 2024-04-16T17:07:29.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.829+0000 7f23cea00640 10 client.25841 ms_handle_connect on v2:172.21.15.45:6840/3755310822 2024-04-16T17:07:29.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.830+0000 7f23cea00640 1 -- 172.21.15.45:0/3048521522 <== mds.1 v2:172.21.15.45:6840/3755310822 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55cfb36d9a40 con 0x55cfb37916d0 2024-04-16T17:07:29.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.830+0000 7f23cea00640 10 client.25841 handle_command_reply: tid=0 2024-04-16T17:07:29.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.830+0000 7f23f2651740 1 client.25841 shutdown 2024-04-16T17:07:29.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.830+0000 7f23f2651740 1 -- 172.21.15.45:0/3048521522 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55cfb3766f10 msgr2=0x55cfb376d3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:29.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.830+0000 7f23f2651740 1 --2- 172.21.15.45:0/3048521522 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55cfb3766f10 0x55cfb376d3a0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f23dc009350 tx=0x7f23dc004270 comp rx=0 tx=0).stop 2024-04-16T17:07:29.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.830+0000 7f23f2651740 1 -- 172.21.15.45:0/3048521522 shutdown_connections 2024-04-16T17:07:29.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.830+0000 7f23f2651740 1 --2- 172.21.15.45:0/3048521522 >> [v2:172.21.15.45:6840/3755310822,v1:172.21.15.45:6841/3755310822] conn(0x55cfb37916d0 0x55cfb3793ac0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:29.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.830+0000 7f23f2651740 1 --2- 172.21.15.45:0/3048521522 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cfb3770410 0x55cfb376de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:29.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.830+0000 7f23f2651740 1 --2- 172.21.15.45:0/3048521522 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cfb3767830 0x55cfb376d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:29.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.830+0000 7f23f2651740 1 --2- 172.21.15.45:0/3048521522 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55cfb3766f10 0x55cfb376d3a0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:29.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.830+0000 7f23cea00640 0 client.25841 ms_handle_reset on v2:172.21.15.45:6840/3755310822 2024-04-16T17:07:29.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.830+0000 7f23f2651740 1 -- 172.21.15.45:0/3048521522 >> 172.21.15.45:0/3048521522 conn(0x55cfb3761640 msgr2=0x55cfb3763390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:07:29.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.831+0000 7f23f2651740 1 -- 172.21.15.45:0/3048521522 shutdown_connections 2024-04-16T17:07:29.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.831+0000 7f23f2651740 1 -- 172.21.15.45:0/3048521522 wait complete. 2024-04-16T17:07:29.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.831+0000 7f23f2651740 20 client.25841 trim_cache size 0 max 16384 2024-04-16T17:07:29.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.845+0000 7f23f2651740 1 Processor -- start 2024-04-16T17:07:29.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.845+0000 7f23f2651740 1 -- start start 2024-04-16T17:07:29.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.845+0000 7f23f2651740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cfb3766f10 0x55cfb378d100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:29.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.845+0000 7f23f2651740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cfb3767830 0x55cfb376dc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:29.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.845+0000 7f23f2651740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55cfb3770410 0x55cfb3799250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:29.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.845+0000 7f23f2651740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f23e81ab530 con 0x55cfb3767830 2024-04-16T17:07:29.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.845+0000 7f23f2651740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f23e81a8e00 con 0x55cfb3770410 2024-04-16T17:07:29.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.845+0000 7f23f2651740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f23e8073770 con 0x55cfb3766f10 2024-04-16T17:07:29.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.846+0000 7f23e7e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cfb3767830 0x55cfb376dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:29.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.846+0000 7f23ed000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cfb3766f10 0x55cfb378d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:29.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.846+0000 7f23ed000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cfb3766f10 0x55cfb378d100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:52004/0 (socket says 172.21.15.45:52004) 2024-04-16T17:07:29.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.846+0000 7f23ed000640 1 -- 172.21.15.45:0/3398365200 learned_addr learned my addr 172.21.15.45:0/3398365200 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:07:29.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.846+0000 7f23eda00640 1 --2- 172.21.15.45:0/3398365200 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55cfb3770410 0x55cfb3799250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:29.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.846+0000 7f23ed000640 1 -- 172.21.15.45:0/3398365200 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55cfb3770410 msgr2=0x55cfb3799250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:07:29.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.846+0000 7f23ed000640 1 --2- 172.21.15.45:0/3398365200 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55cfb3770410 0x55cfb3799250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:29.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.846+0000 7f23ed000640 1 -- 172.21.15.45:0/3398365200 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cfb3767830 msgr2=0x55cfb376dc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:07:29.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.846+0000 7f23ed000640 1 --2- 172.21.15.45:0/3398365200 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cfb3767830 0x55cfb376dc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:29.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.847+0000 7f23ed000640 1 -- 172.21.15.45:0/3398365200 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cfb33de3f0 con 0x55cfb3766f10 2024-04-16T17:07:29.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.847+0000 7f23eda00640 1 --2- 172.21.15.45:0/3398365200 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55cfb3770410 0x55cfb3799250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:07:29.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.847+0000 7f23e7e00640 1 --2- 172.21.15.45:0/3398365200 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cfb3767830 0x55cfb376dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:07:29.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.847+0000 7f23ed000640 1 --2- 172.21.15.45:0/3398365200 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cfb3766f10 0x55cfb378d100 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f23dc009380 tx=0x7f23dc011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4d81c0c8cd29eb34 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:07:29.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.847+0000 7f23d9600640 1 -- 172.21.15.45:0/3398365200 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23dc004390 con 0x55cfb3766f10 2024-04-16T17:07:29.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.847+0000 7f23d9600640 1 -- 172.21.15.45:0/3398365200 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f23dc005520 con 0x55cfb3766f10 2024-04-16T17:07:29.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.847+0000 7f23d9600640 1 -- 172.21.15.45:0/3398365200 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23dc019540 con 0x55cfb3766f10 2024-04-16T17:07:29.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.848+0000 7f23f2651740 1 -- 172.21.15.45:0/3398365200 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cfb3766f10 msgr2=0x55cfb378d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:29.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.848+0000 7f23f2651740 1 --2- 172.21.15.45:0/3398365200 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cfb3766f10 0x55cfb378d100 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f23dc009380 tx=0x7f23dc011810 comp rx=0 tx=0).stop 2024-04-16T17:07:29.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.848+0000 7f23f2651740 1 -- 172.21.15.45:0/3398365200 shutdown_connections 2024-04-16T17:07:29.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.848+0000 7f23f2651740 1 --2- 172.21.15.45:0/3398365200 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55cfb3770410 0x55cfb3799250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:29.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.848+0000 7f23f2651740 1 --2- 172.21.15.45:0/3398365200 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cfb3767830 0x55cfb376dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:29.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.848+0000 7f23f2651740 1 --2- 172.21.15.45:0/3398365200 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cfb3766f10 0x55cfb378d100 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:29.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.848+0000 7f23f2651740 1 -- 172.21.15.45:0/3398365200 >> 172.21.15.45:0/3398365200 conn(0x55cfb3761640 msgr2=0x55cfb3760700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:07:29.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.848+0000 7f23f2651740 1 -- 172.21.15.45:0/3398365200 shutdown_connections 2024-04-16T17:07:29.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.848+0000 7f23f2651740 1 -- 172.21.15.45:0/3398365200 wait complete. 2024-04-16T17:07:29.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.848+0000 7f23f2651740 1 -- start start 2024-04-16T17:07:29.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.849+0000 7f23f2651740 1 Processor -- start 2024-04-16T17:07:29.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.849+0000 7f23f2651740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cfb3766f10 0x55cfb3789610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:29.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.849+0000 7f23f2651740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55cfb3767830 0x55cfb3789b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:29.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.849+0000 7f23f2651740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cfb3770410 0x55cfb3769630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:29.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.849+0000 7f23f2651740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f23e8073e00 con 0x55cfb3766f10 2024-04-16T17:07:29.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.849+0000 7f23f2651740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f23e81a8730 con 0x55cfb3767830 2024-04-16T17:07:29.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.849+0000 7f23f2651740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f23e81a7990 con 0x55cfb3770410 2024-04-16T17:07:29.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.849+0000 7f23f2651740 10 client.0 _ll_register_callbacks cb 0x7f23dc009c10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:07:29.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.850+0000 7f23ed000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cfb3766f10 0x55cfb3789610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:29.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.850+0000 7f23e7e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55cfb3767830 0x55cfb3789b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:29.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.850+0000 7f23e7e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55cfb3767830 0x55cfb3789b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:34816/0 (socket says 172.21.15.45:34816) 2024-04-16T17:07:29.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.850+0000 7f23e7e00640 1 -- 172.21.15.45:0/652650404 learned_addr learned my addr 172.21.15.45:0/652650404 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:07:29.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.850+0000 7f23eda00640 1 --2- 172.21.15.45:0/652650404 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cfb3770410 0x55cfb3769630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:29.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.850+0000 7f23e7e00640 1 -- 172.21.15.45:0/652650404 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cfb3770410 msgr2=0x55cfb3769630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:07:29.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.850+0000 7f23e7e00640 1 --2- 172.21.15.45:0/652650404 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cfb3770410 0x55cfb3769630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:29.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.850+0000 7f23e7e00640 1 -- 172.21.15.45:0/652650404 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cfb3766f10 msgr2=0x55cfb3789610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:07:29.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.850+0000 7f23e7e00640 1 --2- 172.21.15.45:0/652650404 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cfb3766f10 0x55cfb3789610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:29.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.850+0000 7f23e7e00640 1 -- 172.21.15.45:0/652650404 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cfb34a87d0 con 0x55cfb3767830 2024-04-16T17:07:29.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.850+0000 7f23e7e00640 1 --2- 172.21.15.45:0/652650404 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55cfb3767830 0x55cfb3789b50 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f23d4009230 tx=0x7f23d4009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=842e6717f42d6811 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:07:29.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.850+0000 7f23cea00640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-04-16T17:07:29.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.851+0000 7f23cea00640 1 -- 172.21.15.45:0/652650404 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23d40043d0 con 0x55cfb3767830 2024-04-16T17:07:29.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.851+0000 7f23f2651740 1 -- 172.21.15.45:0/652650404 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f23dc005520 con 0x55cfb3767830 2024-04-16T17:07:29.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.851+0000 7f23cea00640 1 -- 172.21.15.45:0/652650404 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f23d403a040 con 0x55cfb3767830 2024-04-16T17:07:29.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.851+0000 7f23cea00640 1 -- 172.21.15.45:0/652650404 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f23d405f050 con 0x55cfb3767830 2024-04-16T17:07:29.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.851+0000 7f23f2651740 10 client.25847 fetch_fsmap learned FSMap version 25 2024-04-16T17:07:29.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.851+0000 7f23f2651740 1 -- 172.21.15.45:0/652650404 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55cfb37f7260 con 0x55cfb3767830 2024-04-16T17:07:29.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.851+0000 7f23cea00640 1 -- 172.21.15.45:0/652650404 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f23dc005520 con 0x55cfb3767830 2024-04-16T17:07:29.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.852+0000 7f23cea00640 1 -- 172.21.15.45:0/652650404 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f23d404a040 con 0x55cfb3767830 2024-04-16T17:07:29.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.852+0000 7f23f2651740 10 client.25847 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T17:07:29.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.852+0000 7f23f2651740 10 client.25847 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-04-16T17:07:29.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.852+0000 7f23f2651740 20 client.25847 populate_metadata read hostname 'smithi045' 2024-04-16T17:07:29.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.852+0000 7f23f2651740 1 --2- 172.21.15.45:0/652650404 >> [v2:172.21.15.45:6840/3755310822,v1:172.21.15.45:6841/3755310822] conn(0x55cfb37f79b0 0x55cfb37f9dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:29.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.852+0000 7f23f2651740 4 client.25847 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T17:07:29.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.852+0000 7f23f2651740 1 -- 172.21.15.45:0/652650404 --> [v2:172.21.15.45:6840/3755310822,v1:172.21.15.45:6841/3755310822] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55cfb36d38b0 con 0x55cfb37f79b0 2024-04-16T17:07:29.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.852+0000 7f23ed000640 1 --2- 172.21.15.45:0/652650404 >> [v2:172.21.15.45:6840/3755310822,v1:172.21.15.45:6841/3755310822] conn(0x55cfb37f79b0 0x55cfb37f9dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:29.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.853+0000 7f23ed000640 1 --2- 172.21.15.45:0/652650404 >> [v2:172.21.15.45:6840/3755310822,v1:172.21.15.45:6841/3755310822] conn(0x55cfb37f79b0 0x55cfb37f9dd0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=55460550faf75a7b server_cookie=6b04f0ebf6b81e3d in_seq=0 out_seq=0 2024-04-16T17:07:29.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.853+0000 7f23cea00640 10 client.25847 ms_handle_connect on v2:172.21.15.45:6840/3755310822 2024-04-16T17:07:29.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.853+0000 7f23cea00640 1 -- 172.21.15.45:0/652650404 <== mds.1 v2:172.21.15.45:6840/3755310822 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55cfb36d38b0 con 0x55cfb37f79b0 2024-04-16T17:07:29.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.853+0000 7f23cea00640 10 client.25847 handle_command_reply: tid=0 2024-04-16T17:07:29.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.853+0000 7f23f2651740 1 client.25847 shutdown 2024-04-16T17:07:29.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.854+0000 7f23f2651740 1 -- 172.21.15.45:0/652650404 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55cfb3767830 msgr2=0x55cfb3789b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:29.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.854+0000 7f23f2651740 1 --2- 172.21.15.45:0/652650404 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55cfb3767830 0x55cfb3789b50 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f23d4009230 tx=0x7f23d4009260 comp rx=0 tx=0).stop 2024-04-16T17:07:29.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.854+0000 7f23f2651740 1 -- 172.21.15.45:0/652650404 shutdown_connections 2024-04-16T17:07:29.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.854+0000 7f23f2651740 1 --2- 172.21.15.45:0/652650404 >> [v2:172.21.15.45:6840/3755310822,v1:172.21.15.45:6841/3755310822] conn(0x55cfb37f79b0 0x55cfb37f9dd0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:29.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.854+0000 7f23f2651740 1 --2- 172.21.15.45:0/652650404 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cfb3770410 0x55cfb3769630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:29.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.854+0000 7f23f2651740 1 --2- 172.21.15.45:0/652650404 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55cfb3767830 0x55cfb3789b50 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:29.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.854+0000 7f23f2651740 1 --2- 172.21.15.45:0/652650404 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cfb3766f10 0x55cfb3789610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:29.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.854+0000 7f23f2651740 1 -- 172.21.15.45:0/652650404 >> 172.21.15.45:0/652650404 conn(0x55cfb3761640 msgr2=0x55cfb3762640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:07:29.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.854+0000 7f23cea00640 0 client.25847 ms_handle_reset on v2:172.21.15.45:6840/3755310822 2024-04-16T17:07:29.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.854+0000 7f23f2651740 1 -- 172.21.15.45:0/652650404 shutdown_connections 2024-04-16T17:07:29.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.854+0000 7f23f2651740 1 -- 172.21.15.45:0/652650404 wait complete. 2024-04-16T17:07:29.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.854+0000 7f23f2651740 20 client.25847 trim_cache size 0 max 16384 2024-04-16T17:07:29.859 INFO:teuthology.orchestra.run.smithi045.stdout:[] 2024-04-16T17:07:29.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.855+0000 7f23ee400640 1 -- 172.21.15.45:0/3408274202 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f23d00a3ec0 msgr2=0x7f23d00a6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:29.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.855+0000 7f23ee400640 1 --2- 172.21.15.45:0/3408274202 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f23d00a3ec0 0x7f23d00a6380 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f23d4005ab0 tx=0x7f23d4005a20 comp rx=0 tx=0).stop 2024-04-16T17:07:29.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.855+0000 7f23ee400640 1 -- 172.21.15.45:0/3408274202 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f23e810c490 msgr2=0x7f23e8101790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:29.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.855+0000 7f23ee400640 1 --2- 172.21.15.45:0/3408274202 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f23e810c490 0x7f23e8101790 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f23e0009fd0 tx=0x7f23e000ef10 comp rx=0 tx=0).stop 2024-04-16T17:07:29.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.855+0000 7f23ee400640 1 -- 172.21.15.45:0/3408274202 shutdown_connections 2024-04-16T17:07:29.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.855+0000 7f23ee400640 1 --2- 172.21.15.45:0/3408274202 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f23d00a3ec0 0x7f23d00a6380 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:29.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.855+0000 7f23ee400640 1 --2- 172.21.15.45:0/3408274202 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f23e81039e0 0x7f23e8101250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:29.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.855+0000 7f23ee400640 1 --2- 172.21.15.45:0/3408274202 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f23e810c490 0x7f23e8101790 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:29.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.855+0000 7f23ee400640 1 --2- 172.21.15.45:0/3408274202 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f23e81030c0 0x7f23e8100d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:29.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.855+0000 7f23ee400640 1 -- 172.21.15.45:0/3408274202 >> 172.21.15.45:0/3408274202 conn(0x7f23e80fc880 msgr2=0x7f23e80fdfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:07:29.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.855+0000 7f23ee400640 1 -- 172.21.15.45:0/3408274202 shutdown_connections 2024-04-16T17:07:29.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:29.856+0000 7f23ee400640 1 -- 172.21.15.45:0/3408274202 wait complete. 2024-04-16T17:07:29.873 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-16T17:07:30.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:30.068+0000 7ff9e6a00640 1 -- 172.21.15.45:0/1827499209 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff9e0102110 msgr2=0x7ff9e01024f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:30.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:30.068+0000 7ff9e6a00640 1 --2- 172.21.15.45:0/1827499209 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff9e0102110 0x7ff9e01024f0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7ff9d4009900 tx=0x7ff9d402bc40 comp rx=0 tx=0).stop 2024-04-16T17:07:30.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:30.069+0000 7ff9e6a00640 1 -- 172.21.15.45:0/1827499209 shutdown_connections 2024-04-16T17:07:30.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:30.069+0000 7ff9e6a00640 1 --2- 172.21.15.45:0/1827499209 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff9e0102ac0 0x7ff9e0106b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:30.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:30.069+0000 7ff9e6a00640 1 --2- 172.21.15.45:0/1827499209 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff9e01071e0 0x7ff9e010edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:30.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:30.069+0000 7ff9e6a00640 1 --2- 172.21.15.45:0/1827499209 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff9e0102110 0x7ff9e01024f0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:30.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:30.069+0000 7ff9e6a00640 1 -- 172.21.15.45:0/1827499209 >> 172.21.15.45:0/1827499209 conn(0x7ff9e00fd9d0 msgr2=0x7ff9e00ffdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:07:30.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:30.069+0000 7ff9e6a00640 1 -- 172.21.15.45:0/1827499209 shutdown_connections 2024-04-16T17:07:30.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:30.069+0000 7ff9e6a00640 1 -- 172.21.15.45:0/1827499209 wait complete. 2024-04-16T17:07:30.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:30.070+0000 7ff9e6a00640 1 Processor -- start 2024-04-16T17:07:30.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:30.070+0000 7ff9e6a00640 1 -- start start 2024-04-16T17:07:30.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:30.070+0000 7ff9e6a00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff9e0102110 0x7ff9e01a06c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:30.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:30.070+0000 7ff9e6a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff9e01071e0 0x7ff9e01a0c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:30.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:30.070+0000 7ff9e6a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff9e019a830 0x7ff9e019ace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:30.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:30.071+0000 7ff9e6a00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7ff9e01122c0 con 0x7ff9e019a830 2024-04-16T17:07:30.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:30.071+0000 7ff9e6a00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7ff9e0112140 con 0x7ff9e0102110 2024-04-16T17:07:30.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:30.071+0000 7ff9e6a00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7ff9e0112440 con 0x7ff9e01071e0 2024-04-16T17:07:30.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:30.071+0000 7ff9e5600640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff9e0102110 0x7ff9e01a06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:30.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:30.071+0000 7ff9e5600640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff9e0102110 0x7ff9e01a06c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:34846/0 (socket says 172.21.15.45:34846) 2024-04-16T17:07:30.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:30.071+0000 7ff9e5600640 1 -- 172.21.15.45:0/2301157893 learned_addr learned my addr 172.21.15.45:0/2301157893 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:07:30.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:30.071+0000 7ff9e4c00640 1 --2- 172.21.15.45:0/2301157893 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff9e01071e0 0x7ff9e01a0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:30.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:30.071+0000 7ff9e5600640 1 -- 172.21.15.45:0/2301157893 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff9e01071e0 msgr2=0x7ff9e01a0c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:30.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:30.071+0000 7ff9e5600640 1 --2- 172.21.15.45:0/2301157893 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff9e01071e0 0x7ff9e01a0c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:30.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:30.071+0000 7ff9e5600640 1 -- 172.21.15.45:0/2301157893 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff9e019a830 msgr2=0x7ff9e019ace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:30.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:30.071+0000 7ff9e6000640 1 --2- 172.21.15.45:0/2301157893 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff9e019a830 0x7ff9e019ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:30.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:30.071+0000 7ff9e5600640 1 --2- 172.21.15.45:0/2301157893 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff9e019a830 0x7ff9e019ace0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:30.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:30.071+0000 7ff9e5600640 1 -- 172.21.15.45:0/2301157893 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9e019b3c0 con 0x7ff9e0102110 2024-04-16T17:07:30.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:30.072+0000 7ff9e5600640 1 --2- 172.21.15.45:0/2301157893 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff9e0102110 0x7ff9e01a06c0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7ff9d4005ab0 tx=0x7ff9d40042f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:07:30.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:30.072+0000 7ff9de000640 1 -- 172.21.15.45:0/2301157893 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9d4041070 con 0x7ff9e0102110 2024-04-16T17:07:30.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:30.072+0000 7ff9de000640 1 -- 172.21.15.45:0/2301157893 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff9d402cca0 con 0x7ff9e0102110 2024-04-16T17:07:30.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:30.072+0000 7ff9de000640 1 -- 172.21.15.45:0/2301157893 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9d4003d90 con 0x7ff9e0102110 2024-04-16T17:07:30.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:30.072+0000 7ff9e6a00640 1 -- 172.21.15.45:0/2301157893 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9e019b650 con 0x7ff9e0102110 2024-04-16T17:07:30.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:30.072+0000 7ff9e6a00640 1 -- 172.21.15.45:0/2301157893 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9e01a77b0 con 0x7ff9e0102110 2024-04-16T17:07:30.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:30.076+0000 7ff9de000640 1 -- 172.21.15.45:0/2301157893 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 25) ==== 138879+0+0 (secure 0 0 0) 0x7ff9d4044070 con 0x7ff9e0102110 2024-04-16T17:07:30.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:30.076+0000 7ff9eadd5740 1 Processor -- start 2024-04-16T17:07:30.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:30.077+0000 7ff9eadd5740 1 -- start start 2024-04-16T17:07:30.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:30.077+0000 7ff9eadd5740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b39093ef10 0x55b39093f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:30.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:30.077+0000 7ff9eadd5740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b39093f830 0x55b390947ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:30.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:30.077+0000 7ff9eadd5740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b390948410 0x55b39094a800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:30.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:30.077+0000 7ff9eadd5740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55b3908a4020 con 0x55b390948410 2024-04-16T17:07:30.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:30.077+0000 7ff9eadd5740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55b39084a460 con 0x55b39093ef10 2024-04-16T17:07:30.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:30.077+0000 7ff9eadd5740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55b39089ffe0 con 0x55b39093f830 2024-04-16T17:07:30.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:30.078+0000 7ff9e6000640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b39093ef10 0x55b39093f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:30.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:30.078+0000 7ff9e5600640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b39093f830 0x55b390947ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:30.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:30.078+0000 7ff9e6000640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b39093ef10 0x55b39093f2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:34858/0 (socket says 172.21.15.45:34858) 2024-04-16T17:07:30.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:30.078+0000 7ff9e6000640 1 -- 172.21.15.45:0/2287789898 learned_addr learned my addr 172.21.15.45:0/2287789898 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:07:30.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:30.078+0000 7ff9e4c00640 1 --2- 172.21.15.45:0/2287789898 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b390948410 0x55b39094a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:30.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:30.078+0000 7ff9de000640 1 --2- 172.21.15.45:0/2301157893 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7ff9c80a3ec0 0x7ff9c80a6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:30.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:30.078+0000 7ff9e5600640 1 -- 172.21.15.45:0/2287789898 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b39093ef10 msgr2=0x55b39093f2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:07:30.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:30.078+0000 7ff9e5600640 1 --2- 172.21.15.45:0/2287789898 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b39093ef10 0x55b39093f2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:30.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:30.078+0000 7ff9e5600640 1 -- 172.21.15.45:0/2287789898 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b390948410 msgr2=0x55b39094a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:07:30.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:30.078+0000 7ff9e5600640 1 --2- 172.21.15.45:0/2287789898 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b390948410 0x55b39094a800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:30.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:30.078+0000 7ff9de000640 1 -- 172.21.15.45:0/2301157893 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7ff9d40f21c0 con 0x7ff9e0102110 2024-04-16T17:07:30.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:30.078+0000 7ff9e6000640 1 --2- 172.21.15.45:0/2301157893 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7ff9c80a3ec0 0x7ff9c80a6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:30.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:30.078+0000 7ff9e5600640 1 -- 172.21.15.45:0/2287789898 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b390852040 con 0x55b39093f830 2024-04-16T17:07:30.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:30.079+0000 7ff9e4c00640 1 --2- 172.21.15.45:0/2287789898 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b390948410 0x55b39094a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:07:30.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:30.079+0000 7ff9e6000640 1 --2- 172.21.15.45:0/2301157893 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7ff9c80a3ec0 0x7ff9c80a6380 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7ff9d8005f40 tx=0x7ff9d8004a40 comp rx=0 tx=0).ready entity=mgr.25411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:07:30.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:30.079+0000 7ff9e5600640 1 --2- 172.21.15.45:0/2287789898 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b39093f830 0x55b390947ed0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7ff9d40ef4f0 tx=0x7ff9d4000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eb4577158e08302e server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:07:30.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:30.080+0000 7ff9d2600640 1 -- 172.21.15.45:0/2287789898 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9d40bda40 con 0x55b39093f830 2024-04-16T17:07:30.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:30.080+0000 7ff9d2600640 1 -- 172.21.15.45:0/2287789898 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff9d4004c80 con 0x55b39093f830 2024-04-16T17:07:30.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:30.080+0000 7ff9d2600640 1 -- 172.21.15.45:0/2287789898 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9d4086830 con 0x55b39093f830 2024-04-16T17:07:30.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:30.080+0000 7ff9eadd5740 1 -- 172.21.15.45:0/2287789898 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b39093f830 msgr2=0x55b390947ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:30.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:30.080+0000 7ff9eadd5740 1 --2- 172.21.15.45:0/2287789898 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b39093f830 0x55b390947ed0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7ff9d40ef4f0 tx=0x7ff9d4000bc0 comp rx=0 tx=0).stop 2024-04-16T17:07:30.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:30.081+0000 7ff9eadd5740 1 -- 172.21.15.45:0/2287789898 shutdown_connections 2024-04-16T17:07:30.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:30.081+0000 7ff9eadd5740 1 --2- 172.21.15.45:0/2287789898 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b39093f830 0x55b390947ed0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:30.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:30.081+0000 7ff9eadd5740 1 --2- 172.21.15.45:0/2287789898 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b390948410 0x55b39094a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:30.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:30.081+0000 7ff9eadd5740 1 --2- 172.21.15.45:0/2287789898 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b39093ef10 0x55b39093f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:30.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:30.081+0000 7ff9eadd5740 1 -- 172.21.15.45:0/2287789898 >> 172.21.15.45:0/2287789898 conn(0x55b390939640 msgr2=0x55b390939a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:07:30.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:30.081+0000 7ff9eadd5740 1 -- 172.21.15.45:0/2287789898 shutdown_connections 2024-04-16T17:07:30.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:30.081+0000 7ff9eadd5740 1 -- 172.21.15.45:0/2287789898 wait complete. 2024-04-16T17:07:30.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:30.081+0000 7ff9eadd5740 1 -- start start 2024-04-16T17:07:30.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:30.082+0000 7ff9eadd5740 1 Processor -- start 2024-04-16T17:07:30.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:30.082+0000 7ff9eadd5740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b39093ef10 0x55b390945430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:30.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:30.082+0000 7ff9eadd5740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b39093f830 0x55b390945970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:30.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:30.082+0000 7ff9eadd5740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b390948410 0x55b390945eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:30.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:30.082+0000 7ff9eadd5740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55b39094afc0 con 0x55b39093f830 2024-04-16T17:07:30.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:30.082+0000 7ff9eadd5740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55b39085d830 con 0x55b390948410 2024-04-16T17:07:30.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:30.082+0000 7ff9eadd5740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55b39094b140 con 0x55b39093ef10 2024-04-16T17:07:30.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:30.082+0000 7ff9eadd5740 10 client.0 _ll_register_callbacks cb 0x55b390865150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:07:30.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:30.082+0000 7ff9e6000640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b390948410 0x55b390945eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:30.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:30.082+0000 7ff9e5600640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b39093ef10 0x55b390945430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:30.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:30.082+0000 7ff9e6000640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b390948410 0x55b390945eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:34860/0 (socket says 172.21.15.45:34860) 2024-04-16T17:07:30.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:30.082+0000 7ff9e6000640 1 -- 172.21.15.45:0/16230144 learned_addr learned my addr 172.21.15.45:0/16230144 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:07:30.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:30.083+0000 7ff9e4c00640 1 --2- 172.21.15.45:0/16230144 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b39093f830 0x55b390945970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:30.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:30.083+0000 7ff9e6000640 1 -- 172.21.15.45:0/16230144 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b39093ef10 msgr2=0x55b390945430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:07:30.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:30.083+0000 7ff9e6000640 1 --2- 172.21.15.45:0/16230144 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b39093ef10 0x55b390945430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:30.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:30.083+0000 7ff9e6000640 1 -- 172.21.15.45:0/16230144 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b39093f830 msgr2=0x55b390945970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:07:30.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:30.083+0000 7ff9e6000640 1 --2- 172.21.15.45:0/16230144 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b39093f830 0x55b390945970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:30.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:30.083+0000 7ff9e6000640 1 -- 172.21.15.45:0/16230144 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b390861920 con 0x55b390948410 2024-04-16T17:07:30.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:30.083+0000 7ff9e4c00640 1 --2- 172.21.15.45:0/16230144 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b39093f830 0x55b390945970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:07:30.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:30.083+0000 7ff9e6000640 1 --2- 172.21.15.45:0/16230144 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b390948410 0x55b390945eb0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7ff9d8005f10 tx=0x7ff9d8004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=73c1a4e88d5e63b0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:07:30.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:30.083+0000 7ff9c6a00640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-04-16T17:07:30.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:30.083+0000 7ff9c6a00640 1 -- 172.21.15.45:0/16230144 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9d801ede0 con 0x55b390948410 2024-04-16T17:07:30.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:30.083+0000 7ff9c6a00640 1 -- 172.21.15.45:0/16230144 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7ff9d801b410 con 0x55b390948410 2024-04-16T17:07:30.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:30.083+0000 7ff9eadd5740 1 -- 172.21.15.45:0/16230144 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff9d4004c80 con 0x55b390948410 2024-04-16T17:07:30.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:30.084+0000 7ff9c6a00640 1 -- 172.21.15.45:0/16230144 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7ff9d801b610 con 0x55b390948410 2024-04-16T17:07:30.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:30.084+0000 7ff9eadd5740 10 client.25859 fetch_fsmap learned FSMap version 25 2024-04-16T17:07:30.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:30.084+0000 7ff9eadd5740 1 -- 172.21.15.45:0/16230144 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55b39082f7f0 con 0x55b390948410 2024-04-16T17:07:30.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:30.084+0000 7ff9c6a00640 1 -- 172.21.15.45:0/16230144 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff9d4004c80 con 0x55b390948410 2024-04-16T17:07:30.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:30.085+0000 7ff9c6a00640 1 -- 172.21.15.45:0/16230144 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7ff9d8025070 con 0x55b390948410 2024-04-16T17:07:30.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:30.085+0000 7ff9eadd5740 10 client.25859 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T17:07:30.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:30.085+0000 7ff9eadd5740 10 client.25859 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:07:30.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:30.085+0000 7ff9eadd5740 20 client.25859 populate_metadata read hostname 'smithi045' 2024-04-16T17:07:30.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:30.085+0000 7ff9eadd5740 1 --2- 172.21.15.45:0/16230144 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55b390960d20 0x55b390963180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:30.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:30.085+0000 7ff9eadd5740 4 client.25859 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:07:30.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:30.085+0000 7ff9eadd5740 1 -- 172.21.15.45:0/16230144 --> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b3908b1a40 con 0x55b390960d20 2024-04-16T17:07:30.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:30.087+0000 7ff9e5600640 1 --2- 172.21.15.45:0/16230144 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55b390960d20 0x55b390963180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:30.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:30.088+0000 7ff9e5600640 1 --2- 172.21.15.45:0/16230144 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55b390960d20 0x55b390963180 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4fface3a7608e50c server_cookie=f2e51ec4f4fd7f3b in_seq=0 out_seq=0 2024-04-16T17:07:30.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:30.088+0000 7ff9c6a00640 10 client.25859 ms_handle_connect on v2:172.21.15.104:6838/3660070592 2024-04-16T17:07:30.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:30.089+0000 7ff9c6a00640 1 -- 172.21.15.45:0/16230144 <== mds.0 v2:172.21.15.104:6838/3660070592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b3908b1a40 con 0x55b390960d20 2024-04-16T17:07:30.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:30.089+0000 7ff9c6a00640 10 client.25859 handle_command_reply: tid=0 2024-04-16T17:07:30.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:30.089+0000 7ff9eadd5740 1 client.25859 shutdown 2024-04-16T17:07:30.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:30.089+0000 7ff9eadd5740 1 -- 172.21.15.45:0/16230144 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b390948410 msgr2=0x55b390945eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:30.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:30.089+0000 7ff9eadd5740 1 --2- 172.21.15.45:0/16230144 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b390948410 0x55b390945eb0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7ff9d8005f10 tx=0x7ff9d8004ab0 comp rx=0 tx=0).stop 2024-04-16T17:07:30.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:30.089+0000 7ff9eadd5740 1 -- 172.21.15.45:0/16230144 shutdown_connections 2024-04-16T17:07:30.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:30.089+0000 7ff9eadd5740 1 --2- 172.21.15.45:0/16230144 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b390948410 0x55b390945eb0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:30.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:30.089+0000 7ff9eadd5740 1 --2- 172.21.15.45:0/16230144 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b39093f830 0x55b390945970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:30.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:30.089+0000 7ff9eadd5740 1 --2- 172.21.15.45:0/16230144 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55b390960d20 0x55b390963180 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:30.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:30.089+0000 7ff9eadd5740 1 --2- 172.21.15.45:0/16230144 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b39093ef10 0x55b390945430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:30.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:30.089+0000 7ff9eadd5740 1 -- 172.21.15.45:0/16230144 >> 172.21.15.45:0/16230144 conn(0x55b390939640 msgr2=0x55b39093b390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:07:30.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:30.090+0000 7ff9c6a00640 0 client.25859 ms_handle_reset on v2:172.21.15.104:6838/3660070592 2024-04-16T17:07:30.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:30.090+0000 7ff9eadd5740 1 -- 172.21.15.45:0/16230144 shutdown_connections 2024-04-16T17:07:30.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:30.090+0000 7ff9eadd5740 1 -- 172.21.15.45:0/16230144 wait complete. 2024-04-16T17:07:30.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:30.090+0000 7ff9eadd5740 20 client.25859 trim_cache size 0 max 16384 2024-04-16T17:07:30.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:30.118+0000 7ff9eadd5740 1 Processor -- start 2024-04-16T17:07:30.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:30.118+0000 7ff9eadd5740 1 -- start start 2024-04-16T17:07:30.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:30.119+0000 7ff9eadd5740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b39093ef10 0x55b390947b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:30.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:30.119+0000 7ff9eadd5740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b39093f830 0x55b390945cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:30.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:30.119+0000 7ff9eadd5740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b390948410 0x55b390938700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:30.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:30.119+0000 7ff9eadd5740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7ff9e01af9a0 con 0x55b39093ef10 2024-04-16T17:07:30.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:30.119+0000 7ff9eadd5740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7ff9e01ad270 con 0x55b390948410 2024-04-16T17:07:30.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:30.119+0000 7ff9eadd5740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7ff9e01ab0e0 con 0x55b39093f830 2024-04-16T17:07:30.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:30.119+0000 7ff9e5600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b39093ef10 0x55b390947b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:30.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:30.119+0000 7ff9e5600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b39093ef10 0x55b390947b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50458/0 (socket says 172.21.15.45:50458) 2024-04-16T17:07:30.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:30.119+0000 7ff9e5600640 1 -- 172.21.15.45:0/1305492416 learned_addr learned my addr 172.21.15.45:0/1305492416 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:07:30.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:30.119+0000 7ff9e6000640 1 --2- 172.21.15.45:0/1305492416 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b390948410 0x55b390938700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:30.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:30.119+0000 7ff9e4c00640 1 --2- 172.21.15.45:0/1305492416 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b39093f830 0x55b390945cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:30.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:30.120+0000 7ff9e5600640 1 -- 172.21.15.45:0/1305492416 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b39093f830 msgr2=0x55b390945cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:07:30.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:30.120+0000 7ff9e5600640 1 --2- 172.21.15.45:0/1305492416 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b39093f830 0x55b390945cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:30.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:30.120+0000 7ff9e5600640 1 -- 172.21.15.45:0/1305492416 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b390948410 msgr2=0x55b390938700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:07:30.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:30.120+0000 7ff9e5600640 1 --2- 172.21.15.45:0/1305492416 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b390948410 0x55b390938700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:30.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:30.120+0000 7ff9e5600640 1 -- 172.21.15.45:0/1305492416 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b3909443e0 con 0x55b39093ef10 2024-04-16T17:07:30.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:30.120+0000 7ff9e5600640 1 --2- 172.21.15.45:0/1305492416 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b39093ef10 0x55b390947b60 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7ff9d402c830 tx=0x7ff9d4005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=947ec9d24def7236 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:07:30.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:30.120+0000 7ff9d1c00640 1 -- 172.21.15.45:0/1305492416 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9d40bd5d0 con 0x55b39093ef10 2024-04-16T17:07:30.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:30.120+0000 7ff9d1c00640 1 -- 172.21.15.45:0/1305492416 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff9d4004c80 con 0x55b39093ef10 2024-04-16T17:07:30.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:30.120+0000 7ff9d1c00640 1 -- 172.21.15.45:0/1305492416 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9d40bd7b0 con 0x55b39093ef10 2024-04-16T17:07:30.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:30.121+0000 7ff9eadd5740 1 -- 172.21.15.45:0/1305492416 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b39093ef10 msgr2=0x55b390947b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:30.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:30.121+0000 7ff9eadd5740 1 --2- 172.21.15.45:0/1305492416 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b39093ef10 0x55b390947b60 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7ff9d402c830 tx=0x7ff9d4005fd0 comp rx=0 tx=0).stop 2024-04-16T17:07:30.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:30.121+0000 7ff9eadd5740 1 -- 172.21.15.45:0/1305492416 shutdown_connections 2024-04-16T17:07:30.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:30.121+0000 7ff9eadd5740 1 --2- 172.21.15.45:0/1305492416 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b39093f830 0x55b390945cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:30.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:30.121+0000 7ff9eadd5740 1 --2- 172.21.15.45:0/1305492416 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b390948410 0x55b390938700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:30.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:30.121+0000 7ff9eadd5740 1 --2- 172.21.15.45:0/1305492416 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b39093ef10 0x55b390947b60 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:30.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:30.121+0000 7ff9eadd5740 1 -- 172.21.15.45:0/1305492416 >> 172.21.15.45:0/1305492416 conn(0x55b390939640 msgr2=0x55b390939a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:07:30.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:30.121+0000 7ff9eadd5740 1 -- 172.21.15.45:0/1305492416 shutdown_connections 2024-04-16T17:07:30.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:30.121+0000 7ff9eadd5740 1 -- 172.21.15.45:0/1305492416 wait complete. 2024-04-16T17:07:30.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:30.122+0000 7ff9eadd5740 1 -- start start 2024-04-16T17:07:30.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:30.122+0000 7ff9eadd5740 1 Processor -- start 2024-04-16T17:07:30.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:30.123+0000 7ff9eadd5740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b39093ef10 0x55b390942300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:30.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:30.123+0000 7ff9eadd5740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b39093f830 0x55b390942840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:30.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:30.123+0000 7ff9eadd5740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b390948410 0x55b390942d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:30.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:30.123+0000 7ff9eadd5740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7ff9e01ab730 con 0x55b39093ef10 2024-04-16T17:07:30.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:30.123+0000 7ff9eadd5740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7ff9e01acba0 con 0x55b390948410 2024-04-16T17:07:30.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:30.123+0000 7ff9eadd5740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7ff9e01abe00 con 0x55b39093f830 2024-04-16T17:07:30.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:30.123+0000 7ff9eadd5740 10 client.0 _ll_register_callbacks cb 0x7ff9d8009d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:07:30.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:30.123+0000 7ff9e5600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b39093ef10 0x55b390942300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:30.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:30.123+0000 7ff9e4c00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b39093f830 0x55b390942840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:30.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:30.123+0000 7ff9e4c00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b39093f830 0x55b390942840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:52084/0 (socket says 172.21.15.45:52084) 2024-04-16T17:07:30.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:30.123+0000 7ff9e4c00640 1 -- 172.21.15.45:0/2273753825 learned_addr learned my addr 172.21.15.45:0/2273753825 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:07:30.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:30.123+0000 7ff9e5600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b39093ef10 0x55b390942300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50472/0 (socket says 172.21.15.45:50472) 2024-04-16T17:07:30.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:30.123+0000 7ff9e6000640 1 --2- 172.21.15.45:0/2273753825 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b390948410 0x55b390942d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:30.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:30.124+0000 7ff9e4c00640 1 -- 172.21.15.45:0/2273753825 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b390948410 msgr2=0x55b390942d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:07:30.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:30.124+0000 7ff9e4c00640 1 --2- 172.21.15.45:0/2273753825 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b390948410 0x55b390942d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:30.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:30.124+0000 7ff9e4c00640 1 -- 172.21.15.45:0/2273753825 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b39093ef10 msgr2=0x55b390942300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:07:30.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:30.124+0000 7ff9e4c00640 1 --2- 172.21.15.45:0/2273753825 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b39093ef10 0x55b390942300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:30.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:30.124+0000 7ff9e4c00640 1 -- 172.21.15.45:0/2273753825 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b3905b63f0 con 0x55b39093f830 2024-04-16T17:07:30.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:30.124+0000 7ff9e5600640 1 --2- 172.21.15.45:0/2273753825 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b39093ef10 0x55b390942300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:07:30.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:30.124+0000 7ff9e4c00640 1 --2- 172.21.15.45:0/2273753825 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b39093f830 0x55b390942840 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7ff9e019bb80 tx=0x7ff9cc010b40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=86fde7b2f2f7e8c1 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:07:30.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:30.124+0000 7ff9c6a00640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-04-16T17:07:30.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:30.124+0000 7ff9c6a00640 1 -- 172.21.15.45:0/2273753825 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9cc00fa80 con 0x55b39093f830 2024-04-16T17:07:30.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:30.124+0000 7ff9c6a00640 1 -- 172.21.15.45:0/2273753825 <== mon.1 v2:172.21.15.117:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7ff9cc00fc20 con 0x55b39093f830 2024-04-16T17:07:30.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:30.124+0000 7ff9eadd5740 1 -- 172.21.15.45:0/2273753825 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff9d4004c80 con 0x55b39093f830 2024-04-16T17:07:30.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:30.125+0000 7ff9c6a00640 1 -- 172.21.15.45:0/2273753825 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7ff9cc029050 con 0x55b39093f830 2024-04-16T17:07:30.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:30.125+0000 7ff9eadd5740 10 client.25864 fetch_fsmap learned FSMap version 25 2024-04-16T17:07:30.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:30.125+0000 7ff9eadd5740 1 -- 172.21.15.45:0/2273753825 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55b3906807d0 con 0x55b39093f830 2024-04-16T17:07:30.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:30.125+0000 7ff9c6a00640 1 -- 172.21.15.45:0/2273753825 <== mon.1 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff9d4004c80 con 0x55b39093f830 2024-04-16T17:07:30.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:30.126+0000 7ff9c6a00640 1 -- 172.21.15.45:0/2273753825 <== mon.1 v2:172.21.15.117:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7ff9cc024030 con 0x55b39093f830 2024-04-16T17:07:30.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:30.126+0000 7ff9eadd5740 10 client.25864 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T17:07:30.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:30.126+0000 7ff9eadd5740 10 client.25864 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:07:30.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:30.126+0000 7ff9eadd5740 20 client.25864 populate_metadata read hostname 'smithi045' 2024-04-16T17:07:30.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:30.126+0000 7ff9eadd5740 1 --2- 172.21.15.45:0/2273753825 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55b3909c7c90 0x55b3909ca0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:30.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:30.126+0000 7ff9eadd5740 4 client.25864 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T17:07:30.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:30.126+0000 7ff9eadd5740 1 -- 172.21.15.45:0/2273753825 --> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b3908ab8b0 con 0x55b3909c7c90 2024-04-16T17:07:30.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:30.126+0000 7ff9e5600640 1 --2- 172.21.15.45:0/2273753825 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55b3909c7c90 0x55b3909ca0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:30.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:30.127+0000 7ff9c6a00640 10 client.25864 ms_handle_connect on v2:172.21.15.104:6838/3660070592 2024-04-16T17:07:30.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:30.127+0000 7ff9e5600640 1 --2- 172.21.15.45:0/2273753825 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55b3909c7c90 0x55b3909ca0b0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7e759b14bc5c5d08 server_cookie=7f310a4ddc74af42 in_seq=0 out_seq=0 2024-04-16T17:07:30.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:30.127+0000 7ff9c6a00640 1 -- 172.21.15.45:0/2273753825 <== mds.0 v2:172.21.15.104:6838/3660070592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b3908ab8b0 con 0x55b3909c7c90 2024-04-16T17:07:30.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:30.127+0000 7ff9c6a00640 10 client.25864 handle_command_reply: tid=0 2024-04-16T17:07:30.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:30.127+0000 7ff9eadd5740 1 client.25864 shutdown 2024-04-16T17:07:30.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:30.127+0000 7ff9eadd5740 1 -- 172.21.15.45:0/2273753825 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b39093f830 msgr2=0x55b390942840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:30.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:30.127+0000 7ff9eadd5740 1 --2- 172.21.15.45:0/2273753825 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b39093f830 0x55b390942840 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7ff9e019bb80 tx=0x7ff9cc010b40 comp rx=0 tx=0).stop 2024-04-16T17:07:30.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:30.127+0000 7ff9eadd5740 1 -- 172.21.15.45:0/2273753825 shutdown_connections 2024-04-16T17:07:30.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:30.127+0000 7ff9eadd5740 1 --2- 172.21.15.45:0/2273753825 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55b3909c7c90 0x55b3909ca0b0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:30.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:30.127+0000 7ff9eadd5740 1 --2- 172.21.15.45:0/2273753825 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b39093f830 0x55b390942840 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:30.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:30.127+0000 7ff9eadd5740 1 --2- 172.21.15.45:0/2273753825 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b390948410 0x55b390942d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:30.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:30.127+0000 7ff9eadd5740 1 --2- 172.21.15.45:0/2273753825 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b39093ef10 0x55b390942300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:30.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:30.127+0000 7ff9c6a00640 0 client.25864 ms_handle_reset on v2:172.21.15.104:6838/3660070592 2024-04-16T17:07:30.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:30.127+0000 7ff9eadd5740 1 -- 172.21.15.45:0/2273753825 >> 172.21.15.45:0/2273753825 conn(0x55b390939640 msgr2=0x55b390938700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:07:30.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:30.128+0000 7ff9eadd5740 1 -- 172.21.15.45:0/2273753825 shutdown_connections 2024-04-16T17:07:30.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:30.128+0000 7ff9eadd5740 1 -- 172.21.15.45:0/2273753825 wait complete. 2024-04-16T17:07:30.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:30.128+0000 7ff9eadd5740 20 client.25864 trim_cache size 0 max 16384 2024-04-16T17:07:30.131 INFO:teuthology.orchestra.run.smithi045.stdout:[] 2024-04-16T17:07:30.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:30.128+0000 7ff9e6a00640 1 -- 172.21.15.45:0/2301157893 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7ff9c80a3ec0 msgr2=0x7ff9c80a6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:30.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:30.128+0000 7ff9e6a00640 1 --2- 172.21.15.45:0/2301157893 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7ff9c80a3ec0 0x7ff9c80a6380 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7ff9d8005f40 tx=0x7ff9d8004a40 comp rx=0 tx=0).stop 2024-04-16T17:07:30.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:30.128+0000 7ff9e6a00640 1 -- 172.21.15.45:0/2301157893 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff9e0102110 msgr2=0x7ff9e01a06c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:30.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:30.128+0000 7ff9e6a00640 1 --2- 172.21.15.45:0/2301157893 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff9e0102110 0x7ff9e01a06c0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7ff9d4005ab0 tx=0x7ff9d40042f0 comp rx=0 tx=0).stop 2024-04-16T17:07:30.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:30.129+0000 7ff9e6a00640 1 -- 172.21.15.45:0/2301157893 shutdown_connections 2024-04-16T17:07:30.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:30.129+0000 7ff9e6a00640 1 --2- 172.21.15.45:0/2301157893 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7ff9c80a3ec0 0x7ff9c80a6380 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:30.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:30.129+0000 7ff9e6a00640 1 --2- 172.21.15.45:0/2301157893 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff9e01071e0 0x7ff9e01a0c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:30.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:30.129+0000 7ff9e6a00640 1 --2- 172.21.15.45:0/2301157893 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff9e019a830 0x7ff9e019ace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:30.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:30.129+0000 7ff9e6a00640 1 --2- 172.21.15.45:0/2301157893 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff9e0102110 0x7ff9e01a06c0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:30.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:30.129+0000 7ff9e6a00640 1 -- 172.21.15.45:0/2301157893 >> 172.21.15.45:0/2301157893 conn(0x7ff9e00fd9d0 msgr2=0x7ff9e00fec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:07:30.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:30.129+0000 7ff9e6a00640 1 -- 172.21.15.45:0/2301157893 shutdown_connections 2024-04-16T17:07:30.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:30.129+0000 7ff9e6a00640 1 -- 172.21.15.45:0/2301157893 wait complete. 2024-04-16T17:07:30.555 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:07:30 smithi117 ceph-mon[36730]: pgmap v25: 129 pgs: 129 active+clean; 1.3 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-16T17:07:30.677 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:07:30 smithi104 ceph-mon[36457]: pgmap v25: 129 pgs: 129 active+clean; 1.3 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-16T17:07:30.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:07:30 smithi045 ceph-mon[31676]: pgmap v25: 129 pgs: 129 active+clean; 1.3 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-16T17:07:31.154 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-16T17:07:31.154 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T17:07:31.154 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-16T17:07:31.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.346+0000 7f276a000640 1 -- 172.21.15.45:0/479520280 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2764102110 msgr2=0x7f27641024f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:31.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.346+0000 7f276a000640 1 --2- 172.21.15.45:0/479520280 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2764102110 0x7f27641024f0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f2758009900 tx=0x7f275802bc30 comp rx=0 tx=0).stop 2024-04-16T17:07:31.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.346+0000 7f276a000640 1 -- 172.21.15.45:0/479520280 shutdown_connections 2024-04-16T17:07:31.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.346+0000 7f276a000640 1 --2- 172.21.15.45:0/479520280 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2764102ac0 0x7f2764106b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:31.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.346+0000 7f276a000640 1 --2- 172.21.15.45:0/479520280 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f27641071e0 0x7f276410edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:31.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.346+0000 7f276a000640 1 --2- 172.21.15.45:0/479520280 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2764102110 0x7f27641024f0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:31.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.346+0000 7f276a000640 1 -- 172.21.15.45:0/479520280 >> 172.21.15.45:0/479520280 conn(0x7f27640fd9d0 msgr2=0x7f27640ffdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:07:31.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.347+0000 7f276a000640 1 -- 172.21.15.45:0/479520280 shutdown_connections 2024-04-16T17:07:31.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.347+0000 7f276a000640 1 -- 172.21.15.45:0/479520280 wait complete. 2024-04-16T17:07:31.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.348+0000 7f276a000640 1 Processor -- start 2024-04-16T17:07:31.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.348+0000 7f276a000640 1 -- start start 2024-04-16T17:07:31.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.348+0000 7f276a000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f27641071e0 0x7f27641a0850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:31.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.348+0000 7f276a000640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f27641a0d90 0x7f276419aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:31.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.348+0000 7f276a000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f276419af40 0x7f276419b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:31.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.348+0000 7f276a000640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f27641122e0 con 0x7f276419af40 2024-04-16T17:07:31.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.348+0000 7f276a000640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f2764112160 con 0x7f27641a0d90 2024-04-16T17:07:31.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.348+0000 7f276a000640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f2764112460 con 0x7f27641071e0 2024-04-16T17:07:31.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.349+0000 7f2769600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f276419af40 0x7f276419b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:31.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.349+0000 7f2768c00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f27641071e0 0x7f27641a0850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:31.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.349+0000 7f2768c00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f27641071e0 0x7f27641a0850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:52104/0 (socket says 172.21.15.45:52104) 2024-04-16T17:07:31.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.349+0000 7f2763e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f27641a0d90 0x7f276419aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:31.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.349+0000 7f2768c00640 1 -- 172.21.15.45:0/1960324495 learned_addr learned my addr 172.21.15.45:0/1960324495 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:07:31.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.349+0000 7f2768c00640 1 -- 172.21.15.45:0/1960324495 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f27641a0d90 msgr2=0x7f276419aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:31.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.349+0000 7f2768c00640 1 --2- 172.21.15.45:0/1960324495 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f27641a0d90 0x7f276419aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:31.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.349+0000 7f2768c00640 1 -- 172.21.15.45:0/1960324495 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f276419af40 msgr2=0x7f276419b3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:31.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.349+0000 7f2768c00640 1 --2- 172.21.15.45:0/1960324495 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f276419af40 0x7f276419b3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:31.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.349+0000 7f2768c00640 1 -- 172.21.15.45:0/1960324495 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27641a7710 con 0x7f27641071e0 2024-04-16T17:07:31.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.349+0000 7f2769600640 1 --2- 172.21.15.45:0/1960324495 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f276419af40 0x7f276419b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:07:31.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.350+0000 7f2768c00640 1 --2- 172.21.15.45:0/1960324495 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f27641071e0 0x7f27641a0850 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f275802c240 tx=0x7f27580042f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:07:31.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.350+0000 7f2761600640 1 -- 172.21.15.45:0/1960324495 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2758043020 con 0x7f27641071e0 2024-04-16T17:07:31.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.350+0000 7f276a000640 1 -- 172.21.15.45:0/1960324495 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2764108d60 con 0x7f27641071e0 2024-04-16T17:07:31.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.350+0000 7f276a000640 1 -- 172.21.15.45:0/1960324495 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27641092b0 con 0x7f27641071e0 2024-04-16T17:07:31.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.350+0000 7f2761600640 1 -- 172.21.15.45:0/1960324495 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f27580054e0 con 0x7f27641071e0 2024-04-16T17:07:31.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.350+0000 7f2761600640 1 -- 172.21.15.45:0/1960324495 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2758003a80 con 0x7f27641071e0 2024-04-16T17:07:31.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.353+0000 7f276e3bf740 1 Processor -- start 2024-04-16T17:07:31.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.354+0000 7f276e3bf740 1 -- start start 2024-04-16T17:07:31.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.354+0000 7f276e3bf740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557a9be2de90 0x557a9be2e270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:31.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.354+0000 7f276e3bf740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557a9be2e7b0 0x557a9be36e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:31.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.354+0000 7f276e3bf740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557a9be37390 0x557a9be39780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:31.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.354+0000 7f276e3bf740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x557a9bd92cb0 con 0x557a9be2e7b0 2024-04-16T17:07:31.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.354+0000 7f276e3bf740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x557a9bd39370 con 0x557a9be2de90 2024-04-16T17:07:31.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.354+0000 7f276e3bf740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x557a9bd8ec70 con 0x557a9be37390 2024-04-16T17:07:31.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.354+0000 7f2769600640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557a9be2de90 0x557a9be2e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:31.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.354+0000 7f2763e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557a9be37390 0x557a9be39780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:31.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.354+0000 7f2769600640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557a9be2de90 0x557a9be2e270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:34896/0 (socket says 172.21.15.45:34896) 2024-04-16T17:07:31.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.354+0000 7f2769600640 1 -- 172.21.15.45:0/3258730815 learned_addr learned my addr 172.21.15.45:0/3258730815 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:07:31.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.355+0000 7f2763e00640 1 -- 172.21.15.45:0/3258730815 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557a9be2de90 msgr2=0x557a9be2e270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:07:31.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.355+0000 7f2763e00640 1 --2- 172.21.15.45:0/3258730815 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557a9be2de90 0x557a9be2e270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:31.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.355+0000 7f2763e00640 1 -- 172.21.15.45:0/3258730815 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557a9be2e7b0 msgr2=0x557a9be36e50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-16T17:07:31.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.355+0000 7f2763e00640 1 --2- 172.21.15.45:0/3258730815 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557a9be2e7b0 0x557a9be36e50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:31.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.355+0000 7f2763e00640 1 -- 172.21.15.45:0/3258730815 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557a9bd58910 con 0x557a9be37390 2024-04-16T17:07:31.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.355+0000 7f2769600640 1 --2- 172.21.15.45:0/3258730815 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557a9be2de90 0x557a9be2e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:07:31.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.355+0000 7f2763e00640 1 --2- 172.21.15.45:0/3258730815 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557a9be37390 0x557a9be39780 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f2750009630 tx=0x7f2750005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b22a458321cce4cf server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:07:31.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.355+0000 7f2761600640 1 -- 172.21.15.45:0/1960324495 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 25) ==== 138879+0+0 (secure 0 0 0) 0x7f2758046020 con 0x7f27641071e0 2024-04-16T17:07:31.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.356+0000 7f2755c00640 1 -- 172.21.15.45:0/3258730815 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2750004310 con 0x557a9be37390 2024-04-16T17:07:31.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.356+0000 7f2755c00640 1 -- 172.21.15.45:0/3258730815 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2750023070 con 0x557a9be37390 2024-04-16T17:07:31.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.356+0000 7f2755c00640 1 -- 172.21.15.45:0/3258730815 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f275000ba90 con 0x557a9be37390 2024-04-16T17:07:31.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.357+0000 7f276e3bf740 1 -- 172.21.15.45:0/3258730815 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557a9be37390 msgr2=0x557a9be39780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:31.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.357+0000 7f276e3bf740 1 --2- 172.21.15.45:0/3258730815 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557a9be37390 0x557a9be39780 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f2750009630 tx=0x7f2750005f90 comp rx=0 tx=0).stop 2024-04-16T17:07:31.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.357+0000 7f276e3bf740 1 -- 172.21.15.45:0/3258730815 shutdown_connections 2024-04-16T17:07:31.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.357+0000 7f276e3bf740 1 --2- 172.21.15.45:0/3258730815 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557a9be37390 0x557a9be39780 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:31.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.357+0000 7f276e3bf740 1 --2- 172.21.15.45:0/3258730815 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557a9be2e7b0 0x557a9be36e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:31.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.357+0000 7f276e3bf740 1 --2- 172.21.15.45:0/3258730815 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557a9be2de90 0x557a9be2e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:31.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.357+0000 7f276e3bf740 1 -- 172.21.15.45:0/3258730815 >> 172.21.15.45:0/3258730815 conn(0x557a9be285c0 msgr2=0x557a9be289a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:07:31.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.358+0000 7f276e3bf740 1 -- 172.21.15.45:0/3258730815 shutdown_connections 2024-04-16T17:07:31.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.358+0000 7f276e3bf740 1 -- 172.21.15.45:0/3258730815 wait complete. 2024-04-16T17:07:31.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.358+0000 7f2761600640 1 --2- 172.21.15.45:0/1960324495 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f274c0a3e50 0x7f274c0a6310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:31.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.358+0000 7f2761600640 1 -- 172.21.15.45:0/1960324495 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f27580f2920 con 0x7f27641071e0 2024-04-16T17:07:31.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.358+0000 7f276e3bf740 1 -- start start 2024-04-16T17:07:31.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.359+0000 7f2763e00640 1 --2- 172.21.15.45:0/1960324495 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f274c0a3e50 0x7f274c0a6310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:31.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.359+0000 7f276e3bf740 1 Processor -- start 2024-04-16T17:07:31.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.359+0000 7f276e3bf740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557a9be2de90 0x557a9be34490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:31.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.359+0000 7f276e3bf740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557a9be2e7b0 0x557a9be349d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:31.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.359+0000 7f276e3bf740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557a9be37390 0x557a9be34f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:31.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.359+0000 7f276e3bf740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x557a9be3a180 con 0x557a9be37390 2024-04-16T17:07:31.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.360+0000 7f276e3bf740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x557a9bd4c730 con 0x557a9be2e7b0 2024-04-16T17:07:31.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.360+0000 7f276e3bf740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x557a9be3a300 con 0x557a9be2de90 2024-04-16T17:07:31.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.360+0000 7f276e3bf740 10 client.0 _ll_register_callbacks cb 0x557a9bd1b1e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:07:31.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.360+0000 7f2763e00640 1 --2- 172.21.15.45:0/1960324495 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f274c0a3e50 0x7f274c0a6310 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f2750016010 tx=0x7f2750005850 comp rx=0 tx=0).ready entity=mgr.25411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:07:31.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.360+0000 7f2768c00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557a9be2de90 0x557a9be34490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:31.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.360+0000 7f2768c00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557a9be2de90 0x557a9be34490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:52116/0 (socket says 172.21.15.45:52116) 2024-04-16T17:07:31.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.360+0000 7f2768c00640 1 -- 172.21.15.45:0/2465616718 learned_addr learned my addr 172.21.15.45:0/2465616718 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:07:31.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.360+0000 7f2769600640 1 --2- 172.21.15.45:0/2465616718 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557a9be37390 0x557a9be34f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:31.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.360+0000 7f2763e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557a9be2e7b0 0x557a9be349d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:31.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.360+0000 7f2768c00640 1 -- 172.21.15.45:0/2465616718 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557a9be2e7b0 msgr2=0x557a9be349d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:07:31.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.360+0000 7f2768c00640 1 --2- 172.21.15.45:0/2465616718 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557a9be2e7b0 0x557a9be349d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:31.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.360+0000 7f2768c00640 1 -- 172.21.15.45:0/2465616718 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557a9be37390 msgr2=0x557a9be34f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:07:31.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.360+0000 7f2768c00640 1 --2- 172.21.15.45:0/2465616718 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557a9be37390 0x557a9be34f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:31.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.360+0000 7f2768c00640 1 -- 172.21.15.45:0/2465616718 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557a9bb6d7e0 con 0x557a9be2de90 2024-04-16T17:07:31.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.360+0000 7f2763e00640 1 --2- 172.21.15.45:0/2465616718 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557a9be2e7b0 0x557a9be349d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:07:31.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.360+0000 7f2769600640 1 --2- 172.21.15.45:0/2465616718 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557a9be37390 0x557a9be34f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:07:31.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.361+0000 7f2768c00640 1 --2- 172.21.15.45:0/2465616718 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557a9be2de90 0x557a9be34490 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f2758006010 tx=0x7f27580be030 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a5af47fb96833857 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:07:31.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.361+0000 7f274a000640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-04-16T17:07:31.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.361+0000 7f274a000640 1 -- 172.21.15.45:0/2465616718 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27580be950 con 0x557a9be2de90 2024-04-16T17:07:31.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.361+0000 7f274a000640 1 -- 172.21.15.45:0/2465616718 <== mon.1 v2:172.21.15.117:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f27580c6cd0 con 0x557a9be2de90 2024-04-16T17:07:31.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.361+0000 7f276e3bf740 1 -- 172.21.15.45:0/2465616718 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2750023070 con 0x557a9be2de90 2024-04-16T17:07:31.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.361+0000 7f274a000640 1 -- 172.21.15.45:0/2465616718 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f27580bebf0 con 0x557a9be2de90 2024-04-16T17:07:31.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.361+0000 7f276e3bf740 10 client.25876 fetch_fsmap learned FSMap version 25 2024-04-16T17:07:31.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.361+0000 7f276e3bf740 1 -- 172.21.15.45:0/2465616718 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x557a9bbcdf70 con 0x557a9be2de90 2024-04-16T17:07:31.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.362+0000 7f274a000640 1 -- 172.21.15.45:0/2465616718 <== mon.1 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2750023070 con 0x557a9be2de90 2024-04-16T17:07:31.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.362+0000 7f274a000640 1 -- 172.21.15.45:0/2465616718 <== mon.1 v2:172.21.15.117:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f27580dd270 con 0x557a9be2de90 2024-04-16T17:07:31.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.362+0000 7f276e3bf740 10 client.25876 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T17:07:31.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.362+0000 7f276e3bf740 10 client.25876 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:07:31.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.362+0000 7f276e3bf740 20 client.25876 populate_metadata read hostname 'smithi045' 2024-04-16T17:07:31.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.362+0000 7f276e3bf740 1 --2- 172.21.15.45:0/2465616718 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x557a9be477e0 0x557a9be49c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:31.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.362+0000 7f276e3bf740 4 client.25876 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:07:31.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.362+0000 7f276e3bf740 1 -- 172.21.15.45:0/2465616718 --> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557a9bda06d0 con 0x557a9be477e0 2024-04-16T17:07:31.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.363+0000 7f2763e00640 1 --2- 172.21.15.45:0/2465616718 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x557a9be477e0 0x557a9be49c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:31.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.363+0000 7f2763e00640 1 --2- 172.21.15.45:0/2465616718 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x557a9be477e0 0x557a9be49c00 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3aed78190d1a41f8 server_cookie=73269fd48cf5af49 in_seq=0 out_seq=0 2024-04-16T17:07:31.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.363+0000 7f274a000640 10 client.25876 ms_handle_connect on v2:172.21.15.104:6838/3660070592 2024-04-16T17:07:31.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.364+0000 7f274a000640 1 -- 172.21.15.45:0/2465616718 <== mds.0 v2:172.21.15.104:6838/3660070592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557a9bda06d0 con 0x557a9be477e0 2024-04-16T17:07:31.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.364+0000 7f274a000640 10 client.25876 handle_command_reply: tid=0 2024-04-16T17:07:31.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.364+0000 7f276e3bf740 1 client.25876 shutdown 2024-04-16T17:07:31.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.365+0000 7f276e3bf740 1 -- 172.21.15.45:0/2465616718 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557a9be2de90 msgr2=0x557a9be34490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:31.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.365+0000 7f276e3bf740 1 --2- 172.21.15.45:0/2465616718 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557a9be2de90 0x557a9be34490 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f2758006010 tx=0x7f27580be030 comp rx=0 tx=0).stop 2024-04-16T17:07:31.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.365+0000 7f276e3bf740 1 -- 172.21.15.45:0/2465616718 shutdown_connections 2024-04-16T17:07:31.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.365+0000 7f276e3bf740 1 --2- 172.21.15.45:0/2465616718 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557a9be37390 0x557a9be34f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:31.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.365+0000 7f276e3bf740 1 --2- 172.21.15.45:0/2465616718 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557a9be2e7b0 0x557a9be349d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:31.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.365+0000 7f276e3bf740 1 --2- 172.21.15.45:0/2465616718 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x557a9be477e0 0x557a9be49c00 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:31.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.365+0000 7f274a000640 0 client.25876 ms_handle_reset on v2:172.21.15.104:6838/3660070592 2024-04-16T17:07:31.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.365+0000 7f276e3bf740 1 --2- 172.21.15.45:0/2465616718 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557a9be2de90 0x557a9be34490 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:31.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.365+0000 7f276e3bf740 1 -- 172.21.15.45:0/2465616718 >> 172.21.15.45:0/2465616718 conn(0x557a9be285c0 msgr2=0x557a9be2a310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:07:31.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.365+0000 7f276e3bf740 1 -- 172.21.15.45:0/2465616718 shutdown_connections 2024-04-16T17:07:31.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.365+0000 7f276e3bf740 1 -- 172.21.15.45:0/2465616718 wait complete. 2024-04-16T17:07:31.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.366+0000 7f276e3bf740 20 client.25876 trim_cache size 0 max 16384 2024-04-16T17:07:31.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.385+0000 7f276e3bf740 1 Processor -- start 2024-04-16T17:07:31.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.385+0000 7f276e3bf740 1 -- start start 2024-04-16T17:07:31.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.385+0000 7f276e3bf740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557a9be2de90 0x557a9be36bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:31.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.385+0000 7f276e3bf740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557a9be2e7b0 0x557a9be33430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:31.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.385+0000 7f276e3bf740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557a9be37390 0x557a9be430a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:31.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.385+0000 7f276e3bf740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f2744012750 con 0x557a9be2e7b0 2024-04-16T17:07:31.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.385+0000 7f276e3bf740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f2744010020 con 0x557a9be2de90 2024-04-16T17:07:31.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.385+0000 7f276e3bf740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f274400de50 con 0x557a9be37390 2024-04-16T17:07:31.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.386+0000 7f2763e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557a9be2e7b0 0x557a9be33430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:31.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.386+0000 7f2768c00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557a9be2de90 0x557a9be36bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:31.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.386+0000 7f2768c00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557a9be2de90 0x557a9be36bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:34904/0 (socket says 172.21.15.45:34904) 2024-04-16T17:07:31.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.386+0000 7f2768c00640 1 -- 172.21.15.45:0/124118461 learned_addr learned my addr 172.21.15.45:0/124118461 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:07:31.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.386+0000 7f2769600640 1 --2- 172.21.15.45:0/124118461 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557a9be37390 0x557a9be430a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:31.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.386+0000 7f2763e00640 1 -- 172.21.15.45:0/124118461 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557a9be37390 msgr2=0x557a9be430a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:07:31.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.386+0000 7f2763e00640 1 --2- 172.21.15.45:0/124118461 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557a9be37390 0x557a9be430a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:31.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.386+0000 7f2763e00640 1 -- 172.21.15.45:0/124118461 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557a9be2de90 msgr2=0x557a9be36bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:07:31.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.386+0000 7f2763e00640 1 --2- 172.21.15.45:0/124118461 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557a9be2de90 0x557a9be36bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:31.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.386+0000 7f2763e00640 1 -- 172.21.15.45:0/124118461 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557a9ba65170 con 0x557a9be2e7b0 2024-04-16T17:07:31.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.386+0000 7f2768c00640 1 --2- 172.21.15.45:0/124118461 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557a9be2de90 0x557a9be36bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:07:31.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.387+0000 7f2763e00640 1 --2- 172.21.15.45:0/124118461 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557a9be2e7b0 0x557a9be33430 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f2750003f70 tx=0x7f275002b070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f74f1268734a9b85 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:07:31.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.387+0000 7f2755200640 1 -- 172.21.15.45:0/124118461 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2750046070 con 0x557a9be2e7b0 2024-04-16T17:07:31.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.387+0000 7f2755200640 1 -- 172.21.15.45:0/124118461 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2750023070 con 0x557a9be2e7b0 2024-04-16T17:07:31.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.387+0000 7f2755200640 1 -- 172.21.15.45:0/124118461 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27500039f0 con 0x557a9be2e7b0 2024-04-16T17:07:31.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.388+0000 7f276e3bf740 1 -- 172.21.15.45:0/124118461 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557a9be2e7b0 msgr2=0x557a9be33430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:31.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.388+0000 7f276e3bf740 1 --2- 172.21.15.45:0/124118461 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557a9be2e7b0 0x557a9be33430 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f2750003f70 tx=0x7f275002b070 comp rx=0 tx=0).stop 2024-04-16T17:07:31.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.388+0000 7f276e3bf740 1 -- 172.21.15.45:0/124118461 shutdown_connections 2024-04-16T17:07:31.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.388+0000 7f276e3bf740 1 --2- 172.21.15.45:0/124118461 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557a9be37390 0x557a9be430a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:31.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.388+0000 7f276e3bf740 1 --2- 172.21.15.45:0/124118461 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557a9be2e7b0 0x557a9be33430 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:31.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.388+0000 7f276e3bf740 1 --2- 172.21.15.45:0/124118461 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557a9be2de90 0x557a9be36bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:31.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.388+0000 7f276e3bf740 1 -- 172.21.15.45:0/124118461 >> 172.21.15.45:0/124118461 conn(0x557a9be285c0 msgr2=0x557a9be2a280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:07:31.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.388+0000 7f276e3bf740 1 -- 172.21.15.45:0/124118461 shutdown_connections 2024-04-16T17:07:31.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.388+0000 7f276e3bf740 1 -- 172.21.15.45:0/124118461 wait complete. 2024-04-16T17:07:31.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.388+0000 7f276e3bf740 1 -- start start 2024-04-16T17:07:31.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.389+0000 7f276e3bf740 1 Processor -- start 2024-04-16T17:07:31.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.389+0000 7f276e3bf740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557a9be2de90 0x557a9be317e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:31.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.389+0000 7f276e3bf740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557a9be2e7b0 0x557a9be31d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:31.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.389+0000 7f276e3bf740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557a9be37390 0x557a9beacd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:31.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.389+0000 7f276e3bf740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f274400e4e0 con 0x557a9be37390 2024-04-16T17:07:31.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.389+0000 7f276e3bf740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f274400f950 con 0x557a9be2de90 2024-04-16T17:07:31.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.389+0000 7f276e3bf740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f274400ebb0 con 0x557a9be2e7b0 2024-04-16T17:07:31.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.389+0000 7f276e3bf740 10 client.0 _ll_register_callbacks cb 0x7f27580095a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:07:31.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.390+0000 7f2768c00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557a9be2de90 0x557a9be317e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:31.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.390+0000 7f2768c00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557a9be2de90 0x557a9be317e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:34908/0 (socket says 172.21.15.45:34908) 2024-04-16T17:07:31.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.390+0000 7f2768c00640 1 -- 172.21.15.45:0/2968287650 learned_addr learned my addr 172.21.15.45:0/2968287650 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:07:31.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.390+0000 7f2763e00640 1 --2- 172.21.15.45:0/2968287650 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557a9be2e7b0 0x557a9be31d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:31.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.390+0000 7f2769600640 1 --2- 172.21.15.45:0/2968287650 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557a9be37390 0x557a9beacd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:31.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.390+0000 7f2768c00640 1 -- 172.21.15.45:0/2968287650 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557a9be2e7b0 msgr2=0x557a9be31d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:07:31.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.390+0000 7f2768c00640 1 --2- 172.21.15.45:0/2968287650 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557a9be2e7b0 0x557a9be31d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:31.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.390+0000 7f2768c00640 1 -- 172.21.15.45:0/2968287650 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557a9be37390 msgr2=0x557a9beacd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:07:31.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.390+0000 7f2768c00640 1 --2- 172.21.15.45:0/2968287650 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557a9be37390 0x557a9beacd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:31.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.390+0000 7f2768c00640 1 -- 172.21.15.45:0/2968287650 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557a9bead2a0 con 0x557a9be2de90 2024-04-16T17:07:31.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.390+0000 7f2769600640 1 --2- 172.21.15.45:0/2968287650 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557a9be37390 0x557a9beacd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:07:31.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.390+0000 7f2763e00640 1 --2- 172.21.15.45:0/2968287650 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557a9be2e7b0 0x557a9be31d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:07:31.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.390+0000 7f2768c00640 1 --2- 172.21.15.45:0/2968287650 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557a9be2de90 0x557a9be317e0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f27580bc7a0 tx=0x7f2758005bf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=49d1cce73280c217 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:07:31.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.390+0000 7f274a000640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-04-16T17:07:31.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.391+0000 7f274a000640 1 -- 172.21.15.45:0/2968287650 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27580bdb40 con 0x557a9be2de90 2024-04-16T17:07:31.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.391+0000 7f274a000640 1 -- 172.21.15.45:0/2968287650 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f2758004ee0 con 0x557a9be2de90 2024-04-16T17:07:31.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.391+0000 7f276e3bf740 1 -- 172.21.15.45:0/2968287650 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2750023070 con 0x557a9be2de90 2024-04-16T17:07:31.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.391+0000 7f274a000640 1 -- 172.21.15.45:0/2968287650 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f27580c28a0 con 0x557a9be2de90 2024-04-16T17:07:31.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.391+0000 7f276e3bf740 10 client.25883 fetch_fsmap learned FSMap version 25 2024-04-16T17:07:31.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.391+0000 7f276e3bf740 1 -- 172.21.15.45:0/2968287650 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x557a9beae4e0 con 0x557a9be2de90 2024-04-16T17:07:31.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.391+0000 7f274a000640 1 -- 172.21.15.45:0/2968287650 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2750023070 con 0x557a9be2de90 2024-04-16T17:07:31.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.391+0000 7f274a000640 1 -- 172.21.15.45:0/2968287650 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f27580a7b50 con 0x557a9be2de90 2024-04-16T17:07:31.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.392+0000 7f276e3bf740 10 client.25883 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T17:07:31.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.392+0000 7f276e3bf740 10 client.25883 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:07:31.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.392+0000 7f276e3bf740 20 client.25883 populate_metadata read hostname 'smithi045' 2024-04-16T17:07:31.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.392+0000 7f276e3bf740 1 --2- 172.21.15.45:0/2968287650 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x557a9beaeaf0 0x557a9beb0f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:31.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.392+0000 7f276e3bf740 4 client.25883 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-16T17:07:31.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.392+0000 7f276e3bf740 1 -- 172.21.15.45:0/2968287650 --> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x557a9bd9a540 con 0x557a9beaeaf0 2024-04-16T17:07:31.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.392+0000 7f2763e00640 1 --2- 172.21.15.45:0/2968287650 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x557a9beaeaf0 0x557a9beb0f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:31.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.393+0000 7f2763e00640 1 --2- 172.21.15.45:0/2968287650 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x557a9beaeaf0 0x557a9beb0f10 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a9e8b60eb270c72a server_cookie=8d525d81ffac326 in_seq=0 out_seq=0 2024-04-16T17:07:31.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.393+0000 7f274a000640 10 client.25883 ms_handle_connect on v2:172.21.15.104:6838/3660070592 2024-04-16T17:07:31.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.394+0000 7f274a000640 1 -- 172.21.15.45:0/2968287650 <== mds.0 v2:172.21.15.104:6838/3660070592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x557a9bd9a540 con 0x557a9beaeaf0 2024-04-16T17:07:31.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.394+0000 7f274a000640 10 client.25883 handle_command_reply: tid=0 2024-04-16T17:07:31.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.394+0000 7f276e3bf740 1 client.25883 shutdown 2024-04-16T17:07:31.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.394+0000 7f276e3bf740 1 -- 172.21.15.45:0/2968287650 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557a9be2de90 msgr2=0x557a9be317e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:31.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.394+0000 7f276e3bf740 1 --2- 172.21.15.45:0/2968287650 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557a9be2de90 0x557a9be317e0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f27580bc7a0 tx=0x7f2758005bf0 comp rx=0 tx=0).stop 2024-04-16T17:07:31.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.394+0000 7f276e3bf740 1 -- 172.21.15.45:0/2968287650 shutdown_connections 2024-04-16T17:07:31.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.394+0000 7f276e3bf740 1 --2- 172.21.15.45:0/2968287650 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x557a9beaeaf0 0x557a9beb0f10 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:31.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.394+0000 7f276e3bf740 1 --2- 172.21.15.45:0/2968287650 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557a9be2e7b0 0x557a9be31d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:31.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.394+0000 7f276e3bf740 1 --2- 172.21.15.45:0/2968287650 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557a9be37390 0x557a9beacd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:31.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.394+0000 7f276e3bf740 1 --2- 172.21.15.45:0/2968287650 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557a9be2de90 0x557a9be317e0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:31.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.394+0000 7f276e3bf740 1 -- 172.21.15.45:0/2968287650 >> 172.21.15.45:0/2968287650 conn(0x557a9be285c0 msgr2=0x557a9be34310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:07:31.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.395+0000 7f274a000640 0 client.25883 ms_handle_reset on v2:172.21.15.104:6838/3660070592 2024-04-16T17:07:31.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.395+0000 7f276e3bf740 1 -- 172.21.15.45:0/2968287650 shutdown_connections 2024-04-16T17:07:31.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.395+0000 7f276e3bf740 1 -- 172.21.15.45:0/2968287650 wait complete. 2024-04-16T17:07:31.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.395+0000 7f276e3bf740 20 client.25883 trim_cache size 0 max 16384 2024-04-16T17:07:31.400 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-16T17:07:31.400 INFO:teuthology.orchestra.run.smithi045.stdout: "return_code": 0, 2024-04-16T17:07:31.400 INFO:teuthology.orchestra.run.smithi045.stdout: "scrub_tag": "fc32d7cb-79dd-43af-bbdf-5271ffe24e04", 2024-04-16T17:07:31.400 INFO:teuthology.orchestra.run.smithi045.stdout: "mode": "asynchronous" 2024-04-16T17:07:31.400 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-16T17:07:31.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.396+0000 7f276a000640 1 -- 172.21.15.45:0/1960324495 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f274c0a3e50 msgr2=0x7f274c0a6310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:31.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.396+0000 7f276a000640 1 --2- 172.21.15.45:0/1960324495 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f274c0a3e50 0x7f274c0a6310 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f2750016010 tx=0x7f2750005850 comp rx=0 tx=0).stop 2024-04-16T17:07:31.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.396+0000 7f276a000640 1 -- 172.21.15.45:0/1960324495 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f27641071e0 msgr2=0x7f27641a0850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:31.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.396+0000 7f276a000640 1 --2- 172.21.15.45:0/1960324495 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f27641071e0 0x7f27641a0850 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f275802c240 tx=0x7f27580042f0 comp rx=0 tx=0).stop 2024-04-16T17:07:31.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.396+0000 7f276a000640 1 -- 172.21.15.45:0/1960324495 shutdown_connections 2024-04-16T17:07:31.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.396+0000 7f276a000640 1 --2- 172.21.15.45:0/1960324495 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f274c0a3e50 0x7f274c0a6310 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:31.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.396+0000 7f276a000640 1 --2- 172.21.15.45:0/1960324495 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f276419af40 0x7f276419b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:31.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.396+0000 7f276a000640 1 --2- 172.21.15.45:0/1960324495 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f27641a0d90 0x7f276419aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:31.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.396+0000 7f276a000640 1 --2- 172.21.15.45:0/1960324495 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f27641071e0 0x7f27641a0850 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:31.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.396+0000 7f276a000640 1 -- 172.21.15.45:0/1960324495 >> 172.21.15.45:0/1960324495 conn(0x7f27640fd9d0 msgr2=0x7f27640fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:07:31.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.396+0000 7f276a000640 1 -- 172.21.15.45:0/1960324495 shutdown_connections 2024-04-16T17:07:31.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.397+0000 7f276a000640 1 -- 172.21.15.45:0/1960324495 wait complete. 2024-04-16T17:07:31.421 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-16T17:07:31.421 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-16T17:07:31.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.603+0000 7f5999600640 1 -- 172.21.15.45:0/224845920 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f59940743b0 msgr2=0x7f5994074790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:31.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.603+0000 7f5999600640 1 --2- 172.21.15.45:0/224845920 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f59940743b0 0x7f5994074790 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f5984009900 tx=0x7f598402bcc0 comp rx=0 tx=0).stop 2024-04-16T17:07:31.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.604+0000 7f5999600640 1 -- 172.21.15.45:0/224845920 shutdown_connections 2024-04-16T17:07:31.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.604+0000 7f5999600640 1 --2- 172.21.15.45:0/224845920 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5994074cd0 0x7f599410e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:31.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.604+0000 7f5999600640 1 --2- 172.21.15.45:0/224845920 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f599410e760 0x7f5994110b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:31.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.604+0000 7f5999600640 1 --2- 172.21.15.45:0/224845920 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f59940743b0 0x7f5994074790 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:31.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.604+0000 7f5999600640 1 -- 172.21.15.45:0/224845920 >> 172.21.15.45:0/224845920 conn(0x7f59940fd9d0 msgr2=0x7f59940ffdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:07:31.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.604+0000 7f5999600640 1 -- 172.21.15.45:0/224845920 shutdown_connections 2024-04-16T17:07:31.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.604+0000 7f5999600640 1 -- 172.21.15.45:0/224845920 wait complete. 2024-04-16T17:07:31.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.605+0000 7f5999600640 1 Processor -- start 2024-04-16T17:07:31.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.606+0000 7f5999600640 1 -- start start 2024-04-16T17:07:31.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.606+0000 7f5999600640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f59940743b0 0x7f59941a0680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:31.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.606+0000 7f5999600640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5994074cd0 0x7f59941a0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:31.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.606+0000 7f5999600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f599410e760 0x7f599419a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:31.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.606+0000 7f5999600640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f59941144c0 con 0x7f599410e760 2024-04-16T17:07:31.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.606+0000 7f5999600640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f5994114340 con 0x7f59940743b0 2024-04-16T17:07:31.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.606+0000 7f5999600640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f5994114640 con 0x7f5994074cd0 2024-04-16T17:07:31.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.606+0000 7f5993400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5994074cd0 0x7f59941a0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:31.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.607+0000 7f5993e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f59940743b0 0x7f59941a0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:31.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.607+0000 7f5993400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5994074cd0 0x7f59941a0bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:52148/0 (socket says 172.21.15.45:52148) 2024-04-16T17:07:31.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.607+0000 7f5993e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f59940743b0 0x7f59941a0680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:34932/0 (socket says 172.21.15.45:34932) 2024-04-16T17:07:31.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.607+0000 7f5993400640 1 -- 172.21.15.45:0/3543968742 learned_addr learned my addr 172.21.15.45:0/3543968742 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:07:31.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.607+0000 7f5998c00640 1 --2- 172.21.15.45:0/3543968742 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f599410e760 0x7f599419a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:31.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.607+0000 7f5993400640 1 -- 172.21.15.45:0/3543968742 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f59940743b0 msgr2=0x7f59941a0680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:31.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.607+0000 7f5993400640 1 --2- 172.21.15.45:0/3543968742 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f59940743b0 0x7f59941a0680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:31.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.607+0000 7f5993400640 1 -- 172.21.15.45:0/3543968742 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f599410e760 msgr2=0x7f599419a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:31.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.607+0000 7f5993400640 1 --2- 172.21.15.45:0/3543968742 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f599410e760 0x7f599419a840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:31.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.607+0000 7f5993400640 1 -- 172.21.15.45:0/3543968742 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f599419b010 con 0x7f5994074cd0 2024-04-16T17:07:31.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.607+0000 7f5993400640 1 --2- 172.21.15.45:0/3543968742 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5994074cd0 0x7f59941a0bc0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f598000e9f0 tx=0x7f598000edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:07:31.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.608+0000 7f5990c00640 1 -- 172.21.15.45:0/3543968742 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f598000cd80 con 0x7f5994074cd0 2024-04-16T17:07:31.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.608+0000 7f5990c00640 1 -- 172.21.15.45:0/3543968742 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5980005500 con 0x7f5994074cd0 2024-04-16T17:07:31.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.608+0000 7f5990c00640 1 -- 172.21.15.45:0/3543968742 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5980010690 con 0x7f5994074cd0 2024-04-16T17:07:31.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.608+0000 7f5999600640 1 -- 172.21.15.45:0/3543968742 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f599419b300 con 0x7f5994074cd0 2024-04-16T17:07:31.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.608+0000 7f5999600640 1 -- 172.21.15.45:0/3543968742 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59941a74d0 con 0x7f5994074cd0 2024-04-16T17:07:31.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.612+0000 7f5990c00640 1 -- 172.21.15.45:0/3543968742 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 25) ==== 138879+0+0 (secure 0 0 0) 0x7f59800046c0 con 0x7f5994074cd0 2024-04-16T17:07:31.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.612+0000 7f599d87b740 1 Processor -- start 2024-04-16T17:07:31.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.613+0000 7f599d87b740 1 -- start start 2024-04-16T17:07:31.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.613+0000 7f599d87b740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5566e641b390 0x5566e641b770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:31.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.613+0000 7f599d87b740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5566e641bcb0 0x5566e6424350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:31.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.613+0000 7f599d87b740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5566e6424890 0x5566e6426c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:31.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.613+0000 7f599d87b740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5566e6381120 con 0x5566e6424890 2024-04-16T17:07:31.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.613+0000 7f599d87b740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5566e6327340 con 0x5566e641bcb0 2024-04-16T17:07:31.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.613+0000 7f599d87b740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5566e637d0e0 con 0x5566e641b390 2024-04-16T17:07:31.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.614+0000 7f5990c00640 1 --2- 172.21.15.45:0/3543968742 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f59780a3f40 0x7f59780a6400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:31.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.614+0000 7f5998c00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5566e641b390 0x5566e641b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:31.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.614+0000 7f5998c00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5566e641b390 0x5566e641b770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:52154/0 (socket says 172.21.15.45:52154) 2024-04-16T17:07:31.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.614+0000 7f5998c00640 1 -- 172.21.15.45:0/2454530988 learned_addr learned my addr 172.21.15.45:0/2454530988 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:07:31.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.614+0000 7f5990c00640 1 -- 172.21.15.45:0/3543968742 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f59800d1220 con 0x7f5994074cd0 2024-04-16T17:07:31.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.614+0000 7f5993400640 1 --2- 172.21.15.45:0/2454530988 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5566e6424890 0x5566e6426c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:31.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.614+0000 7f5993e00640 1 --2- 172.21.15.45:0/2454530988 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5566e641bcb0 0x5566e6424350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:31.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.614+0000 7f5998c00640 1 -- 172.21.15.45:0/2454530988 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5566e641bcb0 msgr2=0x5566e6424350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:07:31.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.614+0000 7f5998c00640 1 --2- 172.21.15.45:0/2454530988 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5566e641bcb0 0x5566e6424350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:31.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.614+0000 7f5998c00640 1 -- 172.21.15.45:0/2454530988 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5566e6424890 msgr2=0x5566e6426c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:07:31.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.614+0000 7f5998c00640 1 --2- 172.21.15.45:0/2454530988 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5566e6424890 0x5566e6426c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:31.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.615+0000 7f5998c00640 1 -- 172.21.15.45:0/2454530988 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5566e615d7d0 con 0x5566e641b390 2024-04-16T17:07:31.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.615+0000 7f5998c00640 1 --2- 172.21.15.45:0/3543968742 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f59780a3f40 0x7f59780a6400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:31.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.615+0000 7f5998c00640 1 --2- 172.21.15.45:0/2454530988 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5566e641b390 0x5566e641b770 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f598c012410 tx=0x7f598c012810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=98a9829d5ed8c22a server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:07:31.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.615+0000 7f5989200640 1 -- 172.21.15.45:0/2454530988 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f598c022070 con 0x5566e641b390 2024-04-16T17:07:31.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.616+0000 7f5989200640 1 -- 172.21.15.45:0/2454530988 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f598c01b3c0 con 0x5566e641b390 2024-04-16T17:07:31.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.616+0000 7f5989200640 1 -- 172.21.15.45:0/2454530988 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f598c01a5c0 con 0x5566e641b390 2024-04-16T17:07:31.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.616+0000 7f5998c00640 1 --2- 172.21.15.45:0/3543968742 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f59780a3f40 0x7f59780a6400 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f598c01f000 tx=0x7f598c01c000 comp rx=0 tx=0).ready entity=mgr.25411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:07:31.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.617+0000 7f599d87b740 1 -- 172.21.15.45:0/2454530988 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5566e641b390 msgr2=0x5566e641b770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:31.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.617+0000 7f599d87b740 1 --2- 172.21.15.45:0/2454530988 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5566e641b390 0x5566e641b770 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f598c012410 tx=0x7f598c012810 comp rx=0 tx=0).stop 2024-04-16T17:07:31.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.617+0000 7f599d87b740 1 -- 172.21.15.45:0/2454530988 shutdown_connections 2024-04-16T17:07:31.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.617+0000 7f599d87b740 1 --2- 172.21.15.45:0/2454530988 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5566e6424890 0x5566e6426c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:31.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.617+0000 7f599d87b740 1 --2- 172.21.15.45:0/2454530988 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5566e641bcb0 0x5566e6424350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:31.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.617+0000 7f599d87b740 1 --2- 172.21.15.45:0/2454530988 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5566e641b390 0x5566e641b770 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:31.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.617+0000 7f599d87b740 1 -- 172.21.15.45:0/2454530988 >> 172.21.15.45:0/2454530988 conn(0x5566e6415ac0 msgr2=0x5566e6415ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:07:31.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.617+0000 7f599d87b740 1 -- 172.21.15.45:0/2454530988 shutdown_connections 2024-04-16T17:07:31.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.617+0000 7f599d87b740 1 -- 172.21.15.45:0/2454530988 wait complete. 2024-04-16T17:07:31.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.617+0000 7f599d87b740 1 -- start start 2024-04-16T17:07:31.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.618+0000 7f599d87b740 1 Processor -- start 2024-04-16T17:07:31.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.618+0000 7f599d87b740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5566e641b390 0x5566e6421930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:31.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.618+0000 7f599d87b740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5566e641bcb0 0x5566e6421e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:31.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.618+0000 7f599d87b740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5566e6424890 0x5566e64223b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:31.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.618+0000 7f599d87b740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5566e6427680 con 0x5566e641b390 2024-04-16T17:07:31.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.618+0000 7f599d87b740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5566e633a710 con 0x5566e6424890 2024-04-16T17:07:31.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.618+0000 7f599d87b740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5566e6427800 con 0x5566e641bcb0 2024-04-16T17:07:31.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.618+0000 7f599d87b740 10 client.0 _ll_register_callbacks cb 0x5566e6335af0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:07:31.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.619+0000 7f5993e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5566e641b390 0x5566e6421930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:31.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.619+0000 7f5998c00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5566e6424890 0x5566e64223b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:31.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.619+0000 7f5998c00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5566e6424890 0x5566e64223b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:34944/0 (socket says 172.21.15.45:34944) 2024-04-16T17:07:31.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.619+0000 7f5993e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5566e641b390 0x5566e6421930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50556/0 (socket says 172.21.15.45:50556) 2024-04-16T17:07:31.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.619+0000 7f5993e00640 1 -- 172.21.15.45:0/2492328632 learned_addr learned my addr 172.21.15.45:0/2492328632 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:07:31.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.619+0000 7f5993400640 1 --2- 172.21.15.45:0/2492328632 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5566e641bcb0 0x5566e6421e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:31.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.619+0000 7f5998c00640 1 -- 172.21.15.45:0/2492328632 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5566e641bcb0 msgr2=0x5566e6421e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:07:31.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.619+0000 7f5998c00640 1 --2- 172.21.15.45:0/2492328632 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5566e641bcb0 0x5566e6421e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:31.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.619+0000 7f5998c00640 1 -- 172.21.15.45:0/2492328632 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5566e641b390 msgr2=0x5566e6421930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:07:31.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.619+0000 7f5998c00640 1 --2- 172.21.15.45:0/2492328632 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5566e641b390 0x5566e6421930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:31.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.619+0000 7f5998c00640 1 -- 172.21.15.45:0/2492328632 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5566e60933f0 con 0x5566e6424890 2024-04-16T17:07:31.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.620+0000 7f5998c00640 1 --2- 172.21.15.45:0/2492328632 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5566e6424890 0x5566e64223b0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f598c009640 tx=0x7f598c013a10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c56f53ab0a020b70 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:07:31.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.620+0000 7f597d600640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-04-16T17:07:31.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.620+0000 7f597d600640 1 -- 172.21.15.45:0/2492328632 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f598c022030 con 0x5566e6424890 2024-04-16T17:07:31.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.620+0000 7f599d87b740 1 -- 172.21.15.45:0/2492328632 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f598c01b3c0 con 0x5566e6424890 2024-04-16T17:07:31.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.620+0000 7f597d600640 1 -- 172.21.15.45:0/2492328632 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f598c01b570 con 0x5566e6424890 2024-04-16T17:07:31.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.621+0000 7f597d600640 1 -- 172.21.15.45:0/2492328632 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f598c004980 con 0x5566e6424890 2024-04-16T17:07:31.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.621+0000 7f599d87b740 10 client.25895 fetch_fsmap learned FSMap version 25 2024-04-16T17:07:31.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.621+0000 7f599d87b740 1 -- 172.21.15.45:0/2492328632 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5566e60530f0 con 0x5566e6424890 2024-04-16T17:07:31.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.621+0000 7f597d600640 1 -- 172.21.15.45:0/2492328632 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f598c01b3c0 con 0x5566e6424890 2024-04-16T17:07:31.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.622+0000 7f597d600640 1 -- 172.21.15.45:0/2492328632 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f598c02e020 con 0x5566e6424890 2024-04-16T17:07:31.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.622+0000 7f599d87b740 10 client.25895 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T17:07:31.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.622+0000 7f599d87b740 10 client.25895 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:07:31.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.622+0000 7f599d87b740 20 client.25895 populate_metadata read hostname 'smithi045' 2024-04-16T17:07:31.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.622+0000 7f599d87b740 1 --2- 172.21.15.45:0/2492328632 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x5566e6434b50 0x5566e6436f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:31.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.622+0000 7f599d87b740 4 client.25895 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:07:31.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.622+0000 7f599d87b740 1 -- 172.21.15.45:0/2492328632 --> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5566e638eb40 con 0x5566e6434b50 2024-04-16T17:07:31.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.622+0000 7f5993e00640 1 --2- 172.21.15.45:0/2492328632 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x5566e6434b50 0x5566e6436f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:31.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.623+0000 7f5993e00640 1 --2- 172.21.15.45:0/2492328632 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x5566e6434b50 0x5566e6436f40 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=328e74d582f7bd5f server_cookie=368a938ead918f95 in_seq=0 out_seq=0 2024-04-16T17:07:31.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.623+0000 7f597d600640 10 client.25895 ms_handle_connect on v2:172.21.15.104:6838/3660070592 2024-04-16T17:07:31.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.623+0000 7f597d600640 1 -- 172.21.15.45:0/2492328632 <== mds.0 v2:172.21.15.104:6838/3660070592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5566e638eb40 con 0x5566e6434b50 2024-04-16T17:07:31.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.623+0000 7f597d600640 10 client.25895 handle_command_reply: tid=0 2024-04-16T17:07:31.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.623+0000 7f599d87b740 1 client.25895 shutdown 2024-04-16T17:07:31.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.624+0000 7f599d87b740 1 -- 172.21.15.45:0/2492328632 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5566e6424890 msgr2=0x5566e64223b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:31.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.624+0000 7f599d87b740 1 --2- 172.21.15.45:0/2492328632 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5566e6424890 0x5566e64223b0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f598c009640 tx=0x7f598c013a10 comp rx=0 tx=0).stop 2024-04-16T17:07:31.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.624+0000 7f599d87b740 1 -- 172.21.15.45:0/2492328632 shutdown_connections 2024-04-16T17:07:31.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.624+0000 7f599d87b740 1 --2- 172.21.15.45:0/2492328632 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x5566e6434b50 0x5566e6436f40 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:31.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.624+0000 7f599d87b740 1 --2- 172.21.15.45:0/2492328632 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5566e641bcb0 0x5566e6421e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:31.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.624+0000 7f599d87b740 1 --2- 172.21.15.45:0/2492328632 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5566e6424890 0x5566e64223b0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:31.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.624+0000 7f599d87b740 1 --2- 172.21.15.45:0/2492328632 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5566e641b390 0x5566e6421930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:31.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.624+0000 7f599d87b740 1 -- 172.21.15.45:0/2492328632 >> 172.21.15.45:0/2492328632 conn(0x5566e6415ac0 msgr2=0x5566e6417810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:07:31.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.624+0000 7f597d600640 0 client.25895 ms_handle_reset on v2:172.21.15.104:6838/3660070592 2024-04-16T17:07:31.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.624+0000 7f599d87b740 1 -- 172.21.15.45:0/2492328632 shutdown_connections 2024-04-16T17:07:31.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.624+0000 7f599d87b740 1 -- 172.21.15.45:0/2492328632 wait complete. 2024-04-16T17:07:31.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.624+0000 7f599d87b740 20 client.25895 trim_cache size 0 max 16384 2024-04-16T17:07:31.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.643+0000 7f599d87b740 1 Processor -- start 2024-04-16T17:07:31.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.643+0000 7f599d87b740 1 -- start start 2024-04-16T17:07:31.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.643+0000 7f599d87b740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5566e641b390 0x5566e6423ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:31.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.643+0000 7f599d87b740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5566e641bcb0 0x5566e6430590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:31.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.643+0000 7f599d87b740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5566e6424890 0x5566e6422180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:31.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.643+0000 7f599d87b740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f59941b3e40 con 0x5566e641bcb0 2024-04-16T17:07:31.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.643+0000 7f599d87b740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f59941b1710 con 0x5566e6424890 2024-04-16T17:07:31.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.643+0000 7f599d87b740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f59941af540 con 0x5566e641b390 2024-04-16T17:07:31.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.644+0000 7f5993e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5566e641b390 0x5566e6423ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:31.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.644+0000 7f5993e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5566e641b390 0x5566e6423ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:52182/0 (socket says 172.21.15.45:52182) 2024-04-16T17:07:31.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.644+0000 7f5993e00640 1 -- 172.21.15.45:0/3670112153 learned_addr learned my addr 172.21.15.45:0/3670112153 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:07:31.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.644+0000 7f5993400640 1 --2- 172.21.15.45:0/3670112153 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5566e641bcb0 0x5566e6430590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:31.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.644+0000 7f5998c00640 1 --2- 172.21.15.45:0/3670112153 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5566e6424890 0x5566e6422180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:31.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.644+0000 7f5998c00640 1 -- 172.21.15.45:0/3670112153 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5566e641b390 msgr2=0x5566e6423ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:07:31.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.644+0000 7f5998c00640 1 --2- 172.21.15.45:0/3670112153 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5566e641b390 0x5566e6423ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:31.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.644+0000 7f5998c00640 1 -- 172.21.15.45:0/3670112153 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5566e641bcb0 msgr2=0x5566e6430590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:07:31.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.644+0000 7f5998c00640 1 --2- 172.21.15.45:0/3670112153 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5566e641bcb0 0x5566e6430590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:31.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.644+0000 7f5998c00640 1 -- 172.21.15.45:0/3670112153 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5566e64372b0 con 0x5566e6424890 2024-04-16T17:07:31.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.644+0000 7f5993e00640 1 --2- 172.21.15.45:0/3670112153 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5566e641b390 0x5566e6423ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:07:31.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.645+0000 7f5998c00640 1 --2- 172.21.15.45:0/3670112153 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5566e6424890 0x5566e6422180 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f598c030440 tx=0x7f598c013a80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2d982e7b8dc75444 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:07:31.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.645+0000 7f597fe00640 1 -- 172.21.15.45:0/3670112153 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f598c022070 con 0x5566e6424890 2024-04-16T17:07:31.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.645+0000 7f597fe00640 1 -- 172.21.15.45:0/3670112153 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f598c026de0 con 0x5566e6424890 2024-04-16T17:07:31.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.645+0000 7f597fe00640 1 -- 172.21.15.45:0/3670112153 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f598c030a50 con 0x5566e6424890 2024-04-16T17:07:31.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.646+0000 7f599d87b740 1 -- 172.21.15.45:0/3670112153 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5566e6424890 msgr2=0x5566e6422180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:31.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.646+0000 7f599d87b740 1 --2- 172.21.15.45:0/3670112153 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5566e6424890 0x5566e6422180 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f598c030440 tx=0x7f598c013a80 comp rx=0 tx=0).stop 2024-04-16T17:07:31.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.646+0000 7f599d87b740 1 -- 172.21.15.45:0/3670112153 shutdown_connections 2024-04-16T17:07:31.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.646+0000 7f599d87b740 1 --2- 172.21.15.45:0/3670112153 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5566e6424890 0x5566e6422180 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:31.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.646+0000 7f599d87b740 1 --2- 172.21.15.45:0/3670112153 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5566e641bcb0 0x5566e6430590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:31.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.646+0000 7f599d87b740 1 --2- 172.21.15.45:0/3670112153 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5566e641b390 0x5566e6423ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:31.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.646+0000 7f599d87b740 1 -- 172.21.15.45:0/3670112153 >> 172.21.15.45:0/3670112153 conn(0x5566e6415ac0 msgr2=0x5566e6420870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:07:31.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.646+0000 7f599d87b740 1 -- 172.21.15.45:0/3670112153 shutdown_connections 2024-04-16T17:07:31.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.646+0000 7f599d87b740 1 -- 172.21.15.45:0/3670112153 wait complete. 2024-04-16T17:07:31.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.646+0000 7f599d87b740 1 -- start start 2024-04-16T17:07:31.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.647+0000 7f599d87b740 1 Processor -- start 2024-04-16T17:07:31.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.647+0000 7f599d87b740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5566e641b390 0x5566e642cce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:31.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.647+0000 7f599d87b740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5566e641bcb0 0x5566e64280a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:31.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.647+0000 7f599d87b740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5566e6424890 0x5566e64285e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:31.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.647+0000 7f599d87b740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f59941afbd0 con 0x5566e641bcb0 2024-04-16T17:07:31.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.647+0000 7f599d87b740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f59941b1040 con 0x5566e641b390 2024-04-16T17:07:31.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.647+0000 7f599d87b740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f59941b02a0 con 0x5566e6424890 2024-04-16T17:07:31.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.647+0000 7f599d87b740 10 client.0 _ll_register_callbacks cb 0x7f598c020070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:07:31.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.647+0000 7f5993e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5566e641b390 0x5566e642cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:31.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.647+0000 7f5993e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5566e641b390 0x5566e642cce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:34962/0 (socket says 172.21.15.45:34962) 2024-04-16T17:07:31.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.647+0000 7f5993e00640 1 -- 172.21.15.45:0/758988761 learned_addr learned my addr 172.21.15.45:0/758988761 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:07:31.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.647+0000 7f5993400640 1 --2- 172.21.15.45:0/758988761 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5566e641bcb0 0x5566e64280a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:31.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.647+0000 7f5998c00640 1 --2- 172.21.15.45:0/758988761 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5566e6424890 0x5566e64285e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:31.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.648+0000 7f5998c00640 1 -- 172.21.15.45:0/758988761 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5566e641b390 msgr2=0x5566e642cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:07:31.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.648+0000 7f5998c00640 1 --2- 172.21.15.45:0/758988761 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5566e641b390 0x5566e642cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:31.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.648+0000 7f5998c00640 1 -- 172.21.15.45:0/758988761 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5566e641bcb0 msgr2=0x5566e64280a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:07:31.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.648+0000 7f5998c00640 1 --2- 172.21.15.45:0/758988761 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5566e641bcb0 0x5566e64280a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:31.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.648+0000 7f5998c00640 1 -- 172.21.15.45:0/758988761 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5566e6428c50 con 0x5566e6424890 2024-04-16T17:07:31.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.648+0000 7f5998c00640 1 --2- 172.21.15.45:0/758988761 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5566e6424890 0x5566e64285e0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f598c020040 tx=0x7f598c0306a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3561117e9a03841e server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:07:31.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.648+0000 7f597d600640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-04-16T17:07:31.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.649+0000 7f597d600640 1 -- 172.21.15.45:0/758988761 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f598c041020 con 0x5566e6424890 2024-04-16T17:07:31.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.649+0000 7f599d87b740 1 -- 172.21.15.45:0/758988761 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f598c026de0 con 0x5566e6424890 2024-04-16T17:07:31.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.649+0000 7f597d600640 1 -- 172.21.15.45:0/758988761 <== mon.1 v2:172.21.15.117:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f598c0309e0 con 0x5566e6424890 2024-04-16T17:07:31.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.649+0000 7f597d600640 1 -- 172.21.15.45:0/758988761 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f598c04c050 con 0x5566e6424890 2024-04-16T17:07:31.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.649+0000 7f599d87b740 10 client.25900 fetch_fsmap learned FSMap version 25 2024-04-16T17:07:31.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.649+0000 7f599d87b740 1 -- 172.21.15.45:0/758988761 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5566e649ae20 con 0x5566e6424890 2024-04-16T17:07:31.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.650+0000 7f597d600640 1 -- 172.21.15.45:0/758988761 <== mon.1 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f598c026de0 con 0x5566e6424890 2024-04-16T17:07:31.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.650+0000 7f597d600640 1 -- 172.21.15.45:0/758988761 <== mon.1 v2:172.21.15.117:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f598c02e070 con 0x5566e6424890 2024-04-16T17:07:31.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.650+0000 7f599d87b740 10 client.25900 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T17:07:31.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.650+0000 7f599d87b740 10 client.25900 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:07:31.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.650+0000 7f599d87b740 20 client.25900 populate_metadata read hostname 'smithi045' 2024-04-16T17:07:31.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.650+0000 7f599d87b740 1 --2- 172.21.15.45:0/758988761 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x5566e649b4e0 0x5566e649d900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:31.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.650+0000 7f599d87b740 4 client.25900 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-16T17:07:31.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.650+0000 7f599d87b740 1 -- 172.21.15.45:0/758988761 --> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5566e63889b0 con 0x5566e649b4e0 2024-04-16T17:07:31.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.650+0000 7f5993e00640 1 --2- 172.21.15.45:0/758988761 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x5566e649b4e0 0x5566e649d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:31.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.651+0000 7f5993e00640 1 --2- 172.21.15.45:0/758988761 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x5566e649b4e0 0x5566e649d900 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bc36c0baec76354e server_cookie=bbc2077c8fedad45 in_seq=0 out_seq=0 2024-04-16T17:07:31.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.651+0000 7f597d600640 10 client.25900 ms_handle_connect on v2:172.21.15.104:6838/3660070592 2024-04-16T17:07:31.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.651+0000 7f597d600640 1 -- 172.21.15.45:0/758988761 <== mds.0 v2:172.21.15.104:6838/3660070592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5566e63889b0 con 0x5566e649b4e0 2024-04-16T17:07:31.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.651+0000 7f597d600640 10 client.25900 handle_command_reply: tid=0 2024-04-16T17:07:31.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.651+0000 7f599d87b740 1 client.25900 shutdown 2024-04-16T17:07:31.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.651+0000 7f599d87b740 1 -- 172.21.15.45:0/758988761 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5566e6424890 msgr2=0x5566e64285e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:31.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.651+0000 7f599d87b740 1 --2- 172.21.15.45:0/758988761 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5566e6424890 0x5566e64285e0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f598c020040 tx=0x7f598c0306a0 comp rx=0 tx=0).stop 2024-04-16T17:07:31.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.651+0000 7f599d87b740 1 -- 172.21.15.45:0/758988761 shutdown_connections 2024-04-16T17:07:31.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.651+0000 7f599d87b740 1 --2- 172.21.15.45:0/758988761 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x5566e649b4e0 0x5566e649d900 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:31.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.651+0000 7f599d87b740 1 --2- 172.21.15.45:0/758988761 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5566e6424890 0x5566e64285e0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:31.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.651+0000 7f599d87b740 1 --2- 172.21.15.45:0/758988761 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5566e641bcb0 0x5566e64280a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:31.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.651+0000 7f599d87b740 1 --2- 172.21.15.45:0/758988761 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5566e641b390 0x5566e642cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:31.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.651+0000 7f597d600640 0 client.25900 ms_handle_reset on v2:172.21.15.104:6838/3660070592 2024-04-16T17:07:31.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.651+0000 7f599d87b740 1 -- 172.21.15.45:0/758988761 >> 172.21.15.45:0/758988761 conn(0x5566e6415ac0 msgr2=0x5566e6414b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:07:31.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.652+0000 7f599d87b740 1 -- 172.21.15.45:0/758988761 shutdown_connections 2024-04-16T17:07:31.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.652+0000 7f599d87b740 1 -- 172.21.15.45:0/758988761 wait complete. 2024-04-16T17:07:31.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.652+0000 7f599d87b740 20 client.25900 trim_cache size 0 max 16384 2024-04-16T17:07:31.656 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-16T17:07:31.656 INFO:teuthology.orchestra.run.smithi045.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-16T17:07:31.656 INFO:teuthology.orchestra.run.smithi045.stdout: "scrubs": { 2024-04-16T17:07:31.656 INFO:teuthology.orchestra.run.smithi045.stdout: "fc32d7cb-79dd-43af-bbdf-5271ffe24e04": { 2024-04-16T17:07:31.656 INFO:teuthology.orchestra.run.smithi045.stdout: "path": "/", 2024-04-16T17:07:31.656 INFO:teuthology.orchestra.run.smithi045.stdout: "tag": "fc32d7cb-79dd-43af-bbdf-5271ffe24e04", 2024-04-16T17:07:31.656 INFO:teuthology.orchestra.run.smithi045.stdout: "options": "recursive,force" 2024-04-16T17:07:31.656 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:07:31.656 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:07:31.656 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-16T17:07:31.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.652+0000 7f5999600640 1 -- 172.21.15.45:0/3543968742 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f59780a3f40 msgr2=0x7f59780a6400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:31.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.652+0000 7f5999600640 1 --2- 172.21.15.45:0/3543968742 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f59780a3f40 0x7f59780a6400 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f598c01f000 tx=0x7f598c01c000 comp rx=0 tx=0).stop 2024-04-16T17:07:31.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.652+0000 7f5999600640 1 -- 172.21.15.45:0/3543968742 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5994074cd0 msgr2=0x7f59941a0bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:31.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.652+0000 7f5999600640 1 --2- 172.21.15.45:0/3543968742 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5994074cd0 0x7f59941a0bc0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f598000e9f0 tx=0x7f598000edf0 comp rx=0 tx=0).stop 2024-04-16T17:07:31.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.653+0000 7f5999600640 1 -- 172.21.15.45:0/3543968742 shutdown_connections 2024-04-16T17:07:31.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.653+0000 7f5999600640 1 --2- 172.21.15.45:0/3543968742 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f59780a3f40 0x7f59780a6400 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:31.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.653+0000 7f5999600640 1 --2- 172.21.15.45:0/3543968742 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5994074cd0 0x7f59941a0bc0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:31.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.653+0000 7f5999600640 1 --2- 172.21.15.45:0/3543968742 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f599410e760 0x7f599419a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:31.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.653+0000 7f5999600640 1 --2- 172.21.15.45:0/3543968742 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f59940743b0 0x7f59941a0680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:31.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.653+0000 7f5999600640 1 -- 172.21.15.45:0/3543968742 >> 172.21.15.45:0/3543968742 conn(0x7f59940fd9d0 msgr2=0x7f59940fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:07:31.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.653+0000 7f5999600640 1 -- 172.21.15.45:0/3543968742 shutdown_connections 2024-04-16T17:07:31.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:07:31.653+0000 7f5999600640 1 -- 172.21.15.45:0/3543968742 wait complete. 2024-04-16T17:07:31.670 INFO:tasks.cephfs.filesystem:scrub status for tag:fc32d7cb-79dd-43af-bbdf-5271ffe24e04 - {'path': '/', 'tag': 'fc32d7cb-79dd-43af-bbdf-5271ffe24e04', 'options': 'recursive,force'} 2024-04-16T17:07:32.555 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:07:32 smithi117 ceph-mon[36730]: pgmap v26: 129 pgs: 129 active+clean; 1.4 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1.6 KiB/s wr, 9 op/s 2024-04-16T17:07:32.555 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:07:32 smithi117 ceph-mon[36730]: scrub queued for path: / 2024-04-16T17:07:32.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:07:32 smithi117 ceph-mon[36730]: scrub summary: idle+waiting paths [/] 2024-04-16T17:07:32.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:07:32 smithi117 ceph-mon[36730]: scrub summary: active paths [/] 2024-04-16T17:07:32.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:07:32 smithi117 ceph-mon[36730]: scrub summary: idle+waiting paths [/] 2024-04-16T17:07:32.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:07:32 smithi104 ceph-mon[36457]: pgmap v26: 129 pgs: 129 active+clean; 1.4 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1.6 KiB/s wr, 9 op/s 2024-04-16T17:07:32.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:07:32 smithi104 ceph-mon[36457]: scrub queued for path: / 2024-04-16T17:07:32.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:07:32 smithi104 ceph-mon[36457]: scrub summary: idle+waiting paths [/] 2024-04-16T17:07:32.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:07:32 smithi104 ceph-mon[36457]: scrub summary: active paths [/] 2024-04-16T17:07:32.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:07:32 smithi104 ceph-mon[36457]: scrub summary: idle+waiting paths [/] 2024-04-16T17:07:32.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:07:32 smithi045 ceph-mon[31676]: pgmap v26: 129 pgs: 129 active+clean; 1.4 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1.6 KiB/s wr, 9 op/s 2024-04-16T17:07:32.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:07:32 smithi045 ceph-mon[31676]: scrub queued for path: / 2024-04-16T17:07:32.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:07:32 smithi045 ceph-mon[31676]: scrub summary: idle+waiting paths [/] 2024-04-16T17:07:32.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:07:32 smithi045 ceph-mon[31676]: scrub summary: active paths [/] 2024-04-16T17:07:32.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:07:32 smithi045 ceph-mon[31676]: scrub summary: idle+waiting paths [/] 2024-04-16T17:07:34.555 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:07:34 smithi117 ceph-mon[36730]: pgmap v27: 129 pgs: 129 active+clean; 1.4 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.6 KiB/s wr, 7 op/s 2024-04-16T17:07:34.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:07:34 smithi117 ceph-mon[36730]: scrub summary: idle 2024-04-16T17:07:34.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:07:34 smithi104 ceph-mon[36457]: pgmap v27: 129 pgs: 129 active+clean; 1.4 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.6 KiB/s wr, 7 op/s 2024-04-16T17:07:34.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:07:34 smithi104 ceph-mon[36457]: scrub summary: idle 2024-04-16T17:07:34.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:07:34 smithi045 ceph-mon[31676]: pgmap v27: 129 pgs: 129 active+clean; 1.4 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.6 KiB/s wr, 7 op/s 2024-04-16T17:07:34.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:07:34 smithi045 ceph-mon[31676]: scrub summary: idle 2024-04-16T17:07:36.555 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:07:36 smithi117 ceph-mon[36730]: pgmap v28: 129 pgs: 129 active+clean; 1.4 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.6 KiB/s wr, 6 op/s 2024-04-16T17:07:36.561 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:36.561+0000 7f6961000640 1 -- 172.21.15.45:0/1197459604 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f695c074cd0 msgr2=0x7f695c10e1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:36.562 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:36.561+0000 7f6961000640 1 --2- 172.21.15.45:0/1197459604 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f695c074cd0 0x7f695c10e1a0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f6948009860 tx=0x7f694802bc20 comp rx=0 tx=0).stop 2024-04-16T17:07:36.562 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:36.561+0000 7f6961000640 1 -- 172.21.15.45:0/1197459604 shutdown_connections 2024-04-16T17:07:36.562 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:36.561+0000 7f6961000640 1 --2- 172.21.15.45:0/1197459604 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f695c10e6e0 0x7f695c110ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:36.562 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:36.561+0000 7f6961000640 1 --2- 172.21.15.45:0/1197459604 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f695c074cd0 0x7f695c10e1a0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:36.562 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:36.561+0000 7f6961000640 1 --2- 172.21.15.45:0/1197459604 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f695c0743b0 0x7f695c074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:36.562 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:36.561+0000 7f6961000640 1 -- 172.21.15.45:0/1197459604 >> 172.21.15.45:0/1197459604 conn(0x7f695c0fd950 msgr2=0x7f695c0ffd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:07:36.562 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:36.561+0000 7f6961000640 1 -- 172.21.15.45:0/1197459604 shutdown_connections 2024-04-16T17:07:36.562 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:36.562+0000 7f6961000640 1 -- 172.21.15.45:0/1197459604 wait complete. 2024-04-16T17:07:36.564 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:36.562+0000 7f6961000640 1 Processor -- start 2024-04-16T17:07:36.564 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:36.563+0000 7f6961000640 1 -- start start 2024-04-16T17:07:36.565 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:36.563+0000 7f6961000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f695c0743b0 0x7f695c1a0620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:36.565 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:36.563+0000 7f6961000640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f695c074cd0 0x7f695c1a0b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:36.565 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:36.563+0000 7f6961000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f695c10e6e0 0x7f695c19a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:36.565 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:36.563+0000 7f6961000640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f695c1143f0 con 0x7f695c10e6e0 2024-04-16T17:07:36.565 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:36.563+0000 7f6961000640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f695c114270 con 0x7f695c074cd0 2024-04-16T17:07:36.565 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:36.563+0000 7f6961000640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f695c114570 con 0x7f695c0743b0 2024-04-16T17:07:36.565 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:36.563+0000 7f695be00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f695c10e6e0 0x7f695c19a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:36.565 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:36.563+0000 7f695aa00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f695c074cd0 0x7f695c1a0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:36.565 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:36.563+0000 7f695aa00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f695c074cd0 0x7f695c1a0b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:34982/0 (socket says 172.21.15.45:34982) 2024-04-16T17:07:36.565 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:36.563+0000 7f695aa00640 1 -- 172.21.15.45:0/2400737014 learned_addr learned my addr 172.21.15.45:0/2400737014 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:07:36.565 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:36.563+0000 7f695be00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f695c10e6e0 0x7f695c19a790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50576/0 (socket says 172.21.15.45:50576) 2024-04-16T17:07:36.565 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:36.563+0000 7f695b400640 1 --2- 172.21.15.45:0/2400737014 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f695c0743b0 0x7f695c1a0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:36.565 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:36.564+0000 7f695aa00640 1 -- 172.21.15.45:0/2400737014 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f695c0743b0 msgr2=0x7f695c1a0620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:36.565 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:36.564+0000 7f695aa00640 1 --2- 172.21.15.45:0/2400737014 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f695c0743b0 0x7f695c1a0620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:36.565 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:36.564+0000 7f695aa00640 1 -- 172.21.15.45:0/2400737014 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f695c10e6e0 msgr2=0x7f695c19a790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:36.566 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:36.564+0000 7f695aa00640 1 --2- 172.21.15.45:0/2400737014 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f695c10e6e0 0x7f695c19a790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:36.566 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:36.564+0000 7f695aa00640 1 -- 172.21.15.45:0/2400737014 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f695c19af60 con 0x7f695c074cd0 2024-04-16T17:07:36.566 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:36.564+0000 7f695b400640 1 --2- 172.21.15.45:0/2400737014 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f695c0743b0 0x7f695c1a0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:07:36.566 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:36.564+0000 7f695be00640 1 --2- 172.21.15.45:0/2400737014 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f695c10e6e0 0x7f695c19a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:07:36.566 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:36.565+0000 7f695aa00640 1 --2- 172.21.15.45:0/2400737014 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f695c074cd0 0x7f695c1a0b60 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f6948037040 tx=0x7f69480344c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:07:36.567 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:36.565+0000 7f6953e00640 1 -- 172.21.15.45:0/2400737014 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6948041020 con 0x7f695c074cd0 2024-04-16T17:07:36.567 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:36.565+0000 7f6953e00640 1 -- 172.21.15.45:0/2400737014 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f694802ce50 con 0x7f695c074cd0 2024-04-16T17:07:36.567 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:36.565+0000 7f6953e00640 1 -- 172.21.15.45:0/2400737014 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6948039040 con 0x7f695c074cd0 2024-04-16T17:07:36.567 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:36.565+0000 7f6961000640 1 -- 172.21.15.45:0/2400737014 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f695c19b1f0 con 0x7f695c074cd0 2024-04-16T17:07:36.567 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:36.565+0000 7f6961000640 1 -- 172.21.15.45:0/2400737014 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f695c1a7470 con 0x7f695c074cd0 2024-04-16T17:07:36.570 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:36.570+0000 7f69653e5740 1 Processor -- start 2024-04-16T17:07:36.570 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:36.570+0000 7f69653e5740 1 -- start start 2024-04-16T17:07:36.570 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:36.570+0000 7f6953e00640 1 -- 172.21.15.45:0/2400737014 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 25) ==== 138879+0+0 (secure 0 0 0) 0x7f6948044020 con 0x7f695c074cd0 2024-04-16T17:07:36.571 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:36.570+0000 7f69653e5740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564a0c8a97f0 0x564a0c8a8540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:36.571 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:36.571+0000 7f69653e5740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564a0c8a8a80 0x564a0c8a8f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:36.571 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:36.571+0000 7f69653e5740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564a0c8b2000 0x564a0c8b43f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:36.571 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:36.571+0000 7f69653e5740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x564a0c80d780 con 0x564a0c8b2000 2024-04-16T17:07:36.571 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:36.571+0000 7f69653e5740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x564a0c7b3f40 con 0x564a0c8a8a80 2024-04-16T17:07:36.571 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:36.571+0000 7f69653e5740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x564a0c809740 con 0x564a0c8a97f0 2024-04-16T17:07:36.572 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:36.571+0000 7f695be00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564a0c8a97f0 0x564a0c8a8540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:36.572 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:36.571+0000 7f695b400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564a0c8a8a80 0x564a0c8a8f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:36.572 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:36.571+0000 7f695b400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564a0c8a8a80 0x564a0c8a8f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:34986/0 (socket says 172.21.15.45:34986) 2024-04-16T17:07:36.572 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:36.571+0000 7f695b400640 1 -- 172.21.15.45:0/4289726157 learned_addr learned my addr 172.21.15.45:0/4289726157 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:07:36.572 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:36.571+0000 7f695aa00640 1 --2- 172.21.15.45:0/4289726157 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564a0c8b2000 0x564a0c8b43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:36.573 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:36.572+0000 7f695b400640 1 -- 172.21.15.45:0/4289726157 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564a0c8a97f0 msgr2=0x564a0c8a8540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:07:36.573 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:36.572+0000 7f695b400640 1 --2- 172.21.15.45:0/4289726157 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564a0c8a97f0 0x564a0c8a8540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:36.573 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:36.572+0000 7f695b400640 1 -- 172.21.15.45:0/4289726157 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564a0c8b2000 msgr2=0x564a0c8b43f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:07:36.573 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:36.572+0000 7f695b400640 1 --2- 172.21.15.45:0/4289726157 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564a0c8b2000 0x564a0c8b43f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:36.573 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:36.572+0000 7f695b400640 1 -- 172.21.15.45:0/4289726157 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564a0c520210 con 0x564a0c8a8a80 2024-04-16T17:07:36.573 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:36.572+0000 7f695aa00640 1 --2- 172.21.15.45:0/4289726157 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564a0c8b2000 0x564a0c8b43f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:07:36.573 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:36.572+0000 7f6953e00640 1 --2- 172.21.15.45:0/2400737014 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f69400a3e30 0x7f69400a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:36.573 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:36.572+0000 7f695b400640 1 --2- 172.21.15.45:0/4289726157 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564a0c8a8a80 0x564a0c8a8f00 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f694c0115c0 tx=0x7f694c011a80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d33c522ced80312c server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:07:36.573 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:36.572+0000 7f6953e00640 1 -- 172.21.15.45:0/2400737014 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f69480f2200 con 0x7f695c074cd0 2024-04-16T17:07:36.573 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:36.573+0000 7f6950c00640 1 -- 172.21.15.45:0/4289726157 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f694c00f640 con 0x564a0c8a8a80 2024-04-16T17:07:36.573 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:36.573+0000 7f6950c00640 1 -- 172.21.15.45:0/4289726157 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f694c00fcc0 con 0x564a0c8a8a80 2024-04-16T17:07:36.574 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:36.573+0000 7f6950c00640 1 -- 172.21.15.45:0/4289726157 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f694c019dc0 con 0x564a0c8a8a80 2024-04-16T17:07:36.574 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:36.573+0000 7f695be00640 1 --2- 172.21.15.45:0/2400737014 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f69400a3e30 0x7f69400a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:36.574 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:36.574+0000 7f69653e5740 1 -- 172.21.15.45:0/4289726157 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564a0c8a8a80 msgr2=0x564a0c8a8f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:36.574 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:36.574+0000 7f69653e5740 1 --2- 172.21.15.45:0/4289726157 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564a0c8a8a80 0x564a0c8a8f00 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f694c0115c0 tx=0x7f694c011a80 comp rx=0 tx=0).stop 2024-04-16T17:07:36.574 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:36.574+0000 7f695be00640 1 --2- 172.21.15.45:0/2400737014 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f69400a3e30 0x7f69400a62f0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f6954005fd0 tx=0x7f69540095c0 comp rx=0 tx=0).ready entity=mgr.25411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:07:36.574 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:36.574+0000 7f69653e5740 1 -- 172.21.15.45:0/4289726157 shutdown_connections 2024-04-16T17:07:36.575 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:36.574+0000 7f69653e5740 1 --2- 172.21.15.45:0/4289726157 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564a0c8b2000 0x564a0c8b43f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:36.575 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:36.574+0000 7f69653e5740 1 --2- 172.21.15.45:0/4289726157 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564a0c8a8a80 0x564a0c8a8f00 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:36.575 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:36.574+0000 7f69653e5740 1 --2- 172.21.15.45:0/4289726157 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564a0c8a97f0 0x564a0c8a8540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:36.575 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:36.574+0000 7f69653e5740 1 -- 172.21.15.45:0/4289726157 >> 172.21.15.45:0/4289726157 conn(0x564a0c8435d0 msgr2=0x564a0c8a3770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:07:36.575 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:36.574+0000 7f69653e5740 1 -- 172.21.15.45:0/4289726157 shutdown_connections 2024-04-16T17:07:36.575 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:36.574+0000 7f69653e5740 1 -- 172.21.15.45:0/4289726157 wait complete. 2024-04-16T17:07:36.575 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:36.575+0000 7f69653e5740 1 -- start start 2024-04-16T17:07:36.575 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:36.575+0000 7f69653e5740 1 Processor -- start 2024-04-16T17:07:36.576 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:36.576+0000 7f69653e5740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564a0c8a8a80 0x564a0c8aeed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:36.576 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:36.576+0000 7f69653e5740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564a0c8a97f0 0x564a0c8af410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:36.576 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:36.576+0000 7f69653e5740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564a0c8b2000 0x564a0c8af950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:36.576 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:36.576+0000 7f69653e5740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x564a0c8b4d30 con 0x564a0c8b2000 2024-04-16T17:07:36.576 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:36.576+0000 7f69653e5740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x564a0c7c7390 con 0x564a0c8a97f0 2024-04-16T17:07:36.576 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:36.576+0000 7f69653e5740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x564a0c8b4eb0 con 0x564a0c8a8a80 2024-04-16T17:07:36.576 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:36.576+0000 7f69653e5740 10 client.0 _ll_register_callbacks cb 0x564a0c7b0f60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:07:36.577 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:36.576+0000 7f695be00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564a0c8b2000 0x564a0c8af950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:36.577 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:36.576+0000 7f695be00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564a0c8b2000 0x564a0c8af950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50594/0 (socket says 172.21.15.45:50594) 2024-04-16T17:07:36.577 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:36.576+0000 7f695be00640 1 -- 172.21.15.45:0/2074523759 learned_addr learned my addr 172.21.15.45:0/2074523759 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:07:36.577 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:36.577+0000 7f695aa00640 1 --2- 172.21.15.45:0/2074523759 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564a0c8a97f0 0x564a0c8af410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:36.577 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:36.577+0000 7f695b400640 1 --2- 172.21.15.45:0/2074523759 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564a0c8a8a80 0x564a0c8aeed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:36.577 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:36.577+0000 7f695be00640 1 -- 172.21.15.45:0/2074523759 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564a0c8a8a80 msgr2=0x564a0c8aeed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:07:36.578 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:36.577+0000 7f695be00640 1 --2- 172.21.15.45:0/2074523759 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564a0c8a8a80 0x564a0c8aeed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:36.578 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:36.577+0000 7f695be00640 1 -- 172.21.15.45:0/2074523759 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564a0c8a97f0 msgr2=0x564a0c8af410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:07:36.578 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:36.577+0000 7f695be00640 1 --2- 172.21.15.45:0/2074523759 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564a0c8a97f0 0x564a0c8af410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:36.578 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:36.577+0000 7f695be00640 1 -- 172.21.15.45:0/2074523759 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564a0c5ec620 con 0x564a0c8b2000 2024-04-16T17:07:36.578 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:36.577+0000 7f695aa00640 1 --2- 172.21.15.45:0/2074523759 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564a0c8a97f0 0x564a0c8af410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:07:36.579 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:36.578+0000 7f695be00640 1 --2- 172.21.15.45:0/2074523759 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564a0c8b2000 0x564a0c8af950 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f695401b450 tx=0x7f6954004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59caeeac4ccca611 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:07:36.579 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:36.578+0000 7f6945600640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-16T17:07:36.579 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:36.578+0000 7f6945600640 1 -- 172.21.15.45:0/2074523759 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69540197e0 con 0x564a0c8b2000 2024-04-16T17:07:36.579 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:36.578+0000 7f6945600640 1 -- 172.21.15.45:0/2074523759 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f6954019980 con 0x564a0c8b2000 2024-04-16T17:07:36.579 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:36.578+0000 7f69653e5740 1 -- 172.21.15.45:0/2074523759 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f694c00fcc0 con 0x564a0c8b2000 2024-04-16T17:07:36.579 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:36.579+0000 7f6945600640 1 -- 172.21.15.45:0/2074523759 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f6954019b80 con 0x564a0c8b2000 2024-04-16T17:07:36.579 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:36.579+0000 7f69653e5740 10 client.15765 fetch_fsmap learned FSMap version 25 2024-04-16T17:07:36.579 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:36.579+0000 7f69653e5740 1 -- 172.21.15.45:0/2074523759 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x564a0c8c0f00 con 0x564a0c8b2000 2024-04-16T17:07:36.580 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:36.580+0000 7f6945600640 1 -- 172.21.15.45:0/2074523759 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f694c00fcc0 con 0x564a0c8b2000 2024-04-16T17:07:36.580 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:36.580+0000 7f6945600640 1 -- 172.21.15.45:0/2074523759 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f695400b040 con 0x564a0c8b2000 2024-04-16T17:07:36.580 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:36.580+0000 7f69653e5740 10 client.15765 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T17:07:36.580 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:36.580+0000 7f69653e5740 10 client.15765 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T17:07:36.580 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:36.580+0000 7f69653e5740 20 client.15765 populate_metadata read hostname 'smithi045' 2024-04-16T17:07:36.580 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:36.580+0000 7f69653e5740 1 --2- 172.21.15.45:0/2074523759 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x564a0c8c2410 0x564a0c8c4800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:36.580 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:36.580+0000 7f69653e5740 4 client.15765 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:07:36.580 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:36.580+0000 7f69653e5740 1 -- 172.21.15.45:0/2074523759 --> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564a0c81b1a0 con 0x564a0c8c2410 2024-04-16T17:07:36.581 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:36.580+0000 7f695b400640 1 --2- 172.21.15.45:0/2074523759 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x564a0c8c2410 0x564a0c8c4800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:36.581 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:36.581+0000 7f6945600640 10 client.15765 ms_handle_connect on v2:172.21.15.104:6838/3660070592 2024-04-16T17:07:36.581 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:36.581+0000 7f695b400640 1 --2- 172.21.15.45:0/2074523759 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x564a0c8c2410 0x564a0c8c4800 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=16ddd13e9e31d0ac server_cookie=9f3d247ad159eb70 in_seq=0 out_seq=0 2024-04-16T17:07:36.582 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:36.582+0000 7f6945600640 1 -- 172.21.15.45:0/2074523759 <== mds.0 v2:172.21.15.104:6838/3660070592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564a0c81b1a0 con 0x564a0c8c2410 2024-04-16T17:07:36.582 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:36.582+0000 7f6945600640 10 client.15765 handle_command_reply: tid=0 2024-04-16T17:07:36.582 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:36.582+0000 7f69653e5740 1 client.15765 shutdown 2024-04-16T17:07:36.582 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:36.583+0000 7f69653e5740 1 -- 172.21.15.45:0/2074523759 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564a0c8b2000 msgr2=0x564a0c8af950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:36.583 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:36.583+0000 7f69653e5740 1 --2- 172.21.15.45:0/2074523759 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564a0c8b2000 0x564a0c8af950 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f695401b450 tx=0x7f6954004ab0 comp rx=0 tx=0).stop 2024-04-16T17:07:36.583 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:36.583+0000 7f69653e5740 1 -- 172.21.15.45:0/2074523759 shutdown_connections 2024-04-16T17:07:36.583 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:36.583+0000 7f69653e5740 1 --2- 172.21.15.45:0/2074523759 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564a0c8b2000 0x564a0c8af950 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:36.584 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:36.583+0000 7f69653e5740 1 --2- 172.21.15.45:0/2074523759 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564a0c8a97f0 0x564a0c8af410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:36.584 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:36.583+0000 7f69653e5740 1 --2- 172.21.15.45:0/2074523759 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x564a0c8c2410 0x564a0c8c4800 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:36.584 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:36.583+0000 7f69653e5740 1 --2- 172.21.15.45:0/2074523759 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564a0c8a8a80 0x564a0c8aeed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:36.584 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:36.583+0000 7f69653e5740 1 -- 172.21.15.45:0/2074523759 >> 172.21.15.45:0/2074523759 conn(0x564a0c8435d0 msgr2=0x564a0c8a54c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:07:36.584 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:36.583+0000 7f6945600640 0 client.15765 ms_handle_reset on v2:172.21.15.104:6838/3660070592 2024-04-16T17:07:36.584 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:36.583+0000 7f69653e5740 1 -- 172.21.15.45:0/2074523759 shutdown_connections 2024-04-16T17:07:36.584 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:36.583+0000 7f69653e5740 1 -- 172.21.15.45:0/2074523759 wait complete. 2024-04-16T17:07:36.584 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:36.584+0000 7f69653e5740 20 client.15765 trim_cache size 0 max 16384 2024-04-16T17:07:36.613 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:36.613+0000 7f69653e5740 1 Processor -- start 2024-04-16T17:07:36.613 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:36.613+0000 7f69653e5740 1 -- start start 2024-04-16T17:07:36.613 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:36.613+0000 7f69653e5740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564a0c8a8a80 0x564a0c8b1600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:36.613 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:36.613+0000 7f69653e5740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564a0c8a97f0 0x564a0c8bdc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:36.614 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:36.613+0000 7f69653e5740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564a0c8b2000 0x564a0c8ca0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:36.614 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:36.613+0000 7f69653e5740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f695c1b3d90 con 0x564a0c8a97f0 2024-04-16T17:07:36.615 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:36.614+0000 7f69653e5740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f695c1b1660 con 0x564a0c8a8a80 2024-04-16T17:07:36.615 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:36.614+0000 7f69653e5740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f695c1af490 con 0x564a0c8b2000 2024-04-16T17:07:36.615 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:36.614+0000 7f695be00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564a0c8b2000 0x564a0c8ca0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:36.615 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:36.614+0000 7f695aa00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564a0c8a97f0 0x564a0c8bdc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:36.615 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:36.614+0000 7f695be00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564a0c8b2000 0x564a0c8ca0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:52216/0 (socket says 172.21.15.45:52216) 2024-04-16T17:07:36.615 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:36.614+0000 7f695be00640 1 -- 172.21.15.45:0/2214504909 learned_addr learned my addr 172.21.15.45:0/2214504909 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:07:36.615 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:36.614+0000 7f695b400640 1 --2- 172.21.15.45:0/2214504909 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564a0c8a8a80 0x564a0c8b1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:36.615 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:36.614+0000 7f695aa00640 1 -- 172.21.15.45:0/2214504909 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564a0c8b2000 msgr2=0x564a0c8ca0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:07:36.615 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:36.615+0000 7f695aa00640 1 --2- 172.21.15.45:0/2214504909 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564a0c8b2000 0x564a0c8ca0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:36.615 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:36.615+0000 7f695aa00640 1 -- 172.21.15.45:0/2214504909 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564a0c8a8a80 msgr2=0x564a0c8b1600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:07:36.615 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:36.615+0000 7f695aa00640 1 --2- 172.21.15.45:0/2214504909 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564a0c8a8a80 0x564a0c8b1600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:36.615 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:36.615+0000 7f695aa00640 1 -- 172.21.15.45:0/2214504909 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564a0c8a7d50 con 0x564a0c8a97f0 2024-04-16T17:07:36.616 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:36.615+0000 7f695aa00640 1 --2- 172.21.15.45:0/2214504909 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564a0c8a97f0 0x564a0c8bdc30 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f69480ef570 tx=0x7f6948005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e6ff04fd459034 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:07:36.616 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:36.615+0000 7f6947e00640 1 -- 172.21.15.45:0/2214504909 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69480bd830 con 0x564a0c8a97f0 2024-04-16T17:07:36.616 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:36.615+0000 7f6947e00640 1 -- 172.21.15.45:0/2214504909 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6948034940 con 0x564a0c8a97f0 2024-04-16T17:07:36.617 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:36.615+0000 7f6947e00640 1 -- 172.21.15.45:0/2214504909 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69480bd650 con 0x564a0c8a97f0 2024-04-16T17:07:36.617 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:36.616+0000 7f69653e5740 1 -- 172.21.15.45:0/2214504909 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564a0c8a97f0 msgr2=0x564a0c8bdc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:36.617 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:36.616+0000 7f69653e5740 1 --2- 172.21.15.45:0/2214504909 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564a0c8a97f0 0x564a0c8bdc30 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f69480ef570 tx=0x7f6948005f90 comp rx=0 tx=0).stop 2024-04-16T17:07:36.617 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:36.616+0000 7f69653e5740 1 -- 172.21.15.45:0/2214504909 shutdown_connections 2024-04-16T17:07:36.617 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:36.616+0000 7f69653e5740 1 --2- 172.21.15.45:0/2214504909 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564a0c8b2000 0x564a0c8ca0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:36.617 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:36.616+0000 7f69653e5740 1 --2- 172.21.15.45:0/2214504909 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564a0c8a97f0 0x564a0c8bdc30 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:36.617 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:36.616+0000 7f69653e5740 1 --2- 172.21.15.45:0/2214504909 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564a0c8a8a80 0x564a0c8b1600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:36.617 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:36.616+0000 7f69653e5740 1 -- 172.21.15.45:0/2214504909 >> 172.21.15.45:0/2214504909 conn(0x564a0c8435d0 msgr2=0x564a0c8a32a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:07:36.617 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:36.617+0000 7f69653e5740 1 -- 172.21.15.45:0/2214504909 shutdown_connections 2024-04-16T17:07:36.617 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:36.617+0000 7f69653e5740 1 -- 172.21.15.45:0/2214504909 wait complete. 2024-04-16T17:07:36.618 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:36.617+0000 7f69653e5740 1 -- start start 2024-04-16T17:07:36.618 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:36.617+0000 7f69653e5740 1 Processor -- start 2024-04-16T17:07:36.619 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:36.618+0000 7f69653e5740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564a0c8a8a80 0x564a0c8ba540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:36.619 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:36.618+0000 7f69653e5740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564a0c8a97f0 0x564a0c8b2710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:36.619 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:36.618+0000 7f69653e5740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564a0c8b2000 0x564a0c8b2c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:36.619 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:36.618+0000 7f69653e5740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f695c1afb20 con 0x564a0c8b2000 2024-04-16T17:07:36.619 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:36.618+0000 7f69653e5740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f695c1b0f90 con 0x564a0c8a97f0 2024-04-16T17:07:36.619 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:36.618+0000 7f69653e5740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f695c1b01f0 con 0x564a0c8a8a80 2024-04-16T17:07:36.620 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:36.618+0000 7f69653e5740 10 client.0 _ll_register_callbacks cb 0x7f6954009dc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:07:36.620 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:36.618+0000 7f695b400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564a0c8a8a80 0x564a0c8ba540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:36.620 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:36.619+0000 7f695b400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564a0c8a8a80 0x564a0c8ba540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:52218/0 (socket says 172.21.15.45:52218) 2024-04-16T17:07:36.620 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:36.619+0000 7f695b400640 1 -- 172.21.15.45:0/3771447502 learned_addr learned my addr 172.21.15.45:0/3771447502 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:07:36.620 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:36.619+0000 7f695be00640 1 --2- 172.21.15.45:0/3771447502 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564a0c8b2000 0x564a0c8b2c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:36.620 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:36.619+0000 7f695aa00640 1 --2- 172.21.15.45:0/3771447502 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564a0c8a97f0 0x564a0c8b2710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:36.620 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:36.619+0000 7f695b400640 1 -- 172.21.15.45:0/3771447502 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564a0c8a97f0 msgr2=0x564a0c8b2710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:07:36.620 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:36.619+0000 7f695b400640 1 --2- 172.21.15.45:0/3771447502 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564a0c8a97f0 0x564a0c8b2710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:36.620 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:36.619+0000 7f695b400640 1 -- 172.21.15.45:0/3771447502 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564a0c8b2000 msgr2=0x564a0c8b2c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:07:36.620 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:36.619+0000 7f695b400640 1 --2- 172.21.15.45:0/3771447502 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564a0c8b2000 0x564a0c8b2c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:36.620 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:36.619+0000 7f695b400640 1 -- 172.21.15.45:0/3771447502 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564a0c8aaab0 con 0x564a0c8a8a80 2024-04-16T17:07:36.620 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:36.619+0000 7f695be00640 1 --2- 172.21.15.45:0/3771447502 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564a0c8b2000 0x564a0c8b2c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:07:36.620 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:36.619+0000 7f695b400640 1 --2- 172.21.15.45:0/3771447502 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564a0c8a8a80 0x564a0c8ba540 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f694c019930 tx=0x7f694c0096a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8dcdce9b4e39a873 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:07:36.620 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:36.619+0000 7f6945600640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-04-16T17:07:36.621 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:36.620+0000 7f6945600640 1 -- 172.21.15.45:0/3771447502 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f694c019550 con 0x564a0c8a8a80 2024-04-16T17:07:36.621 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:36.620+0000 7f6945600640 1 -- 172.21.15.45:0/3771447502 <== mon.1 v2:172.21.15.117:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f694c019a30 con 0x564a0c8a8a80 2024-04-16T17:07:36.621 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:36.620+0000 7f69653e5740 1 -- 172.21.15.45:0/3771447502 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x564a0c815010 con 0x564a0c8a8a80 2024-04-16T17:07:36.621 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:36.620+0000 7f6945600640 1 -- 172.21.15.45:0/3771447502 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f694c0196f0 con 0x564a0c8a8a80 2024-04-16T17:07:36.622 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:36.620+0000 7f69653e5740 10 client.25918 fetch_fsmap learned FSMap version 25 2024-04-16T17:07:36.622 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:36.620+0000 7f69653e5740 1 -- 172.21.15.45:0/3771447502 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x564a0c922930 con 0x564a0c8a8a80 2024-04-16T17:07:36.622 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:36.621+0000 7f6945600640 1 -- 172.21.15.45:0/3771447502 <== mon.1 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x564a0c815010 con 0x564a0c8a8a80 2024-04-16T17:07:36.622 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:36.621+0000 7f6945600640 1 -- 172.21.15.45:0/3771447502 <== mon.1 v2:172.21.15.117:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f694c00f7d0 con 0x564a0c8a8a80 2024-04-16T17:07:36.622 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:36.621+0000 7f69653e5740 10 client.25918 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T17:07:36.622 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:36.621+0000 7f69653e5740 10 client.25918 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T17:07:36.622 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:36.621+0000 7f69653e5740 20 client.25918 populate_metadata read hostname 'smithi045' 2024-04-16T17:07:36.622 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:36.621+0000 7f69653e5740 1 --2- 172.21.15.45:0/3771447502 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x564a0c922d30 0x564a0c925150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:36.622 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:36.621+0000 7f69653e5740 4 client.25918 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-16T17:07:36.622 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:36.621+0000 7f69653e5740 1 -- 172.21.15.45:0/3771447502 --> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564a0c791530 con 0x564a0c922d30 2024-04-16T17:07:36.622 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:36.621+0000 7f695aa00640 1 --2- 172.21.15.45:0/3771447502 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x564a0c922d30 0x564a0c925150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:36.623 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:36.622+0000 7f695aa00640 1 --2- 172.21.15.45:0/3771447502 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x564a0c922d30 0x564a0c925150 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=490cf3d7be871452 server_cookie=e18c90c420b9194 in_seq=0 out_seq=0 2024-04-16T17:07:36.623 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:36.622+0000 7f6945600640 10 client.25918 ms_handle_connect on v2:172.21.15.104:6838/3660070592 2024-04-16T17:07:36.632 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:36.631+0000 7f6945600640 1 -- 172.21.15.45:0/3771447502 <== mds.0 v2:172.21.15.104:6838/3660070592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x564a0c791530 con 0x564a0c922d30 2024-04-16T17:07:36.632 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:36.631+0000 7f6945600640 10 client.25918 handle_command_reply: tid=0 2024-04-16T17:07:36.632 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:36.631+0000 7f69653e5740 1 client.25918 shutdown 2024-04-16T17:07:36.632 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:36.631+0000 7f69653e5740 1 -- 172.21.15.45:0/3771447502 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564a0c8a8a80 msgr2=0x564a0c8ba540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:36.632 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:36.631+0000 7f69653e5740 1 --2- 172.21.15.45:0/3771447502 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564a0c8a8a80 0x564a0c8ba540 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f694c019930 tx=0x7f694c0096a0 comp rx=0 tx=0).stop 2024-04-16T17:07:36.632 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:36.632+0000 7f69653e5740 1 -- 172.21.15.45:0/3771447502 shutdown_connections 2024-04-16T17:07:36.632 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:36.632+0000 7f69653e5740 1 --2- 172.21.15.45:0/3771447502 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564a0c8b2000 0x564a0c8b2c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:36.632 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:36.632+0000 7f69653e5740 1 --2- 172.21.15.45:0/3771447502 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564a0c8a97f0 0x564a0c8b2710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:36.632 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:36.632+0000 7f69653e5740 1 --2- 172.21.15.45:0/3771447502 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x564a0c922d30 0x564a0c925150 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:36.633 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:36.632+0000 7f69653e5740 1 --2- 172.21.15.45:0/3771447502 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564a0c8a8a80 0x564a0c8ba540 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:36.633 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:36.632+0000 7f6945600640 0 client.25918 ms_handle_reset on v2:172.21.15.104:6838/3660070592 2024-04-16T17:07:36.633 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:36.632+0000 7f69653e5740 1 -- 172.21.15.45:0/3771447502 >> 172.21.15.45:0/3771447502 conn(0x564a0c8435d0 msgr2=0x564a0c8aee90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:07:36.633 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:36.632+0000 7f69653e5740 1 -- 172.21.15.45:0/3771447502 shutdown_connections 2024-04-16T17:07:36.633 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:36.632+0000 7f69653e5740 1 -- 172.21.15.45:0/3771447502 wait complete. 2024-04-16T17:07:36.633 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:36.632+0000 7f69653e5740 20 client.25918 trim_cache size 0 max 16384 2024-04-16T17:07:36.633 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-04-16T17:07:36.633 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-04-16T17:07:36.633 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-04-16T17:07:36.634 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-04-16T17:07:36.634 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:36.633+0000 7f6961000640 1 -- 172.21.15.45:0/2400737014 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f69400a3e30 msgr2=0x7f69400a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:36.634 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:36.633+0000 7f6961000640 1 --2- 172.21.15.45:0/2400737014 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f69400a3e30 0x7f69400a62f0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f6954005fd0 tx=0x7f69540095c0 comp rx=0 tx=0).stop 2024-04-16T17:07:36.634 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:36.633+0000 7f6961000640 1 -- 172.21.15.45:0/2400737014 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f695c074cd0 msgr2=0x7f695c1a0b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:36.634 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:36.633+0000 7f6961000640 1 --2- 172.21.15.45:0/2400737014 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f695c074cd0 0x7f695c1a0b60 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f6948037040 tx=0x7f69480344c0 comp rx=0 tx=0).stop 2024-04-16T17:07:36.634 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:36.633+0000 7f6961000640 1 -- 172.21.15.45:0/2400737014 shutdown_connections 2024-04-16T17:07:36.635 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:36.633+0000 7f6961000640 1 --2- 172.21.15.45:0/2400737014 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f69400a3e30 0x7f69400a62f0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:36.635 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:36.633+0000 7f6961000640 1 --2- 172.21.15.45:0/2400737014 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f695c10e6e0 0x7f695c19a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:36.635 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:36.633+0000 7f6961000640 1 --2- 172.21.15.45:0/2400737014 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f695c074cd0 0x7f695c1a0b60 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:36.635 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:36.633+0000 7f6961000640 1 --2- 172.21.15.45:0/2400737014 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f695c0743b0 0x7f695c1a0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:36.635 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:36.633+0000 7f6961000640 1 -- 172.21.15.45:0/2400737014 >> 172.21.15.45:0/2400737014 conn(0x7f695c0fd950 msgr2=0x7f695c0fe760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:07:36.635 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:36.634+0000 7f6961000640 1 -- 172.21.15.45:0/2400737014 shutdown_connections 2024-04-16T17:07:36.635 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:36.634+0000 7f6961000640 1 -- 172.21.15.45:0/2400737014 wait complete. 2024-04-16T17:07:36.677 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:07:36 smithi104 ceph-mon[36457]: pgmap v28: 129 pgs: 129 active+clean; 1.4 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.6 KiB/s wr, 6 op/s 2024-04-16T17:07:36.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:07:36 smithi045 ceph-mon[31676]: pgmap v28: 129 pgs: 129 active+clean; 1.4 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.6 KiB/s wr, 6 op/s 2024-04-16T17:07:38.555 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:07:38 smithi117 ceph-mon[36730]: pgmap v29: 129 pgs: 129 active+clean; 1.4 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 1.6 KiB/s wr, 10 op/s 2024-04-16T17:07:38.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:07:38 smithi104 ceph-mon[36457]: pgmap v29: 129 pgs: 129 active+clean; 1.4 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 1.6 KiB/s wr, 10 op/s 2024-04-16T17:07:38.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:07:38 smithi045 ceph-mon[31676]: pgmap v29: 129 pgs: 129 active+clean; 1.4 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 1.6 KiB/s wr, 10 op/s 2024-04-16T17:07:40.555 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:07:40 smithi117 ceph-mon[36730]: pgmap v30: 129 pgs: 129 active+clean; 1.4 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 1.3 KiB/s wr, 7 op/s 2024-04-16T17:07:40.677 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:07:40 smithi104 ceph-mon[36457]: pgmap v30: 129 pgs: 129 active+clean; 1.4 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 1.3 KiB/s wr, 7 op/s 2024-04-16T17:07:40.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:07:40 smithi045 ceph-mon[31676]: pgmap v30: 129 pgs: 129 active+clean; 1.4 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 1.3 KiB/s wr, 7 op/s 2024-04-16T17:07:42.555 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:07:42 smithi117 ceph-mon[36730]: pgmap v31: 129 pgs: 129 active+clean; 1.4 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 2.5 KiB/s wr, 11 op/s 2024-04-16T17:07:42.677 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:07:42 smithi104 ceph-mon[36457]: pgmap v31: 129 pgs: 129 active+clean; 1.4 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 2.5 KiB/s wr, 11 op/s 2024-04-16T17:07:42.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:07:42 smithi045 ceph-mon[31676]: pgmap v31: 129 pgs: 129 active+clean; 1.4 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 2.5 KiB/s wr, 11 op/s 2024-04-16T17:07:44.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:07:44 smithi104 ceph-mon[36457]: pgmap v32: 129 pgs: 129 active+clean; 1.4 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-04-16T17:07:44.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:07:44 smithi045 ceph-mon[31676]: pgmap v32: 129 pgs: 129 active+clean; 1.4 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-04-16T17:07:44.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:07:44 smithi117 ceph-mon[36730]: pgmap v32: 129 pgs: 129 active+clean; 1.4 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-04-16T17:07:46.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:07:46 smithi104 ceph-mon[36457]: pgmap v33: 129 pgs: 129 active+clean; 1.4 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-04-16T17:07:46.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:07:46 smithi045 ceph-mon[31676]: pgmap v33: 129 pgs: 129 active+clean; 1.4 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-04-16T17:07:46.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:07:46 smithi117 ceph-mon[36730]: pgmap v33: 129 pgs: 129 active+clean; 1.4 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-04-16T17:07:48.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:07:48 smithi104 ceph-mon[36457]: pgmap v34: 129 pgs: 129 active+clean; 1.4 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-04-16T17:07:48.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:07:48 smithi045 ceph-mon[31676]: pgmap v34: 129 pgs: 129 active+clean; 1.4 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-04-16T17:07:48.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:07:48 smithi117 ceph-mon[36730]: pgmap v34: 129 pgs: 129 active+clean; 1.4 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-04-16T17:07:49.772 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:49.772+0000 7fdaa7200640 1 -- 172.21.15.45:0/3099788559 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdaa810e6e0 msgr2=0x7fdaa8110ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:49.772 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:49.772+0000 7fdaa7200640 1 --2- 172.21.15.45:0/3099788559 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdaa810e6e0 0x7fdaa8110ad0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fdaa0009be0 tx=0x7fdaa002bf80 comp rx=0 tx=0).stop 2024-04-16T17:07:49.773 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:49.772+0000 7fdaa7200640 1 -- 172.21.15.45:0/3099788559 shutdown_connections 2024-04-16T17:07:49.773 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:49.772+0000 7fdaa7200640 1 --2- 172.21.15.45:0/3099788559 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdaa8074cd0 0x7fdaa810e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:49.773 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:49.772+0000 7fdaa7200640 1 --2- 172.21.15.45:0/3099788559 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdaa810e6e0 0x7fdaa8110ad0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:49.773 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:49.772+0000 7fdaa7200640 1 --2- 172.21.15.45:0/3099788559 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdaa80743b0 0x7fdaa8074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:49.773 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:49.772+0000 7fdaa7200640 1 -- 172.21.15.45:0/3099788559 >> 172.21.15.45:0/3099788559 conn(0x7fdaa80fd950 msgr2=0x7fdaa80ffd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:07:49.773 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:49.772+0000 7fdaa7200640 1 -- 172.21.15.45:0/3099788559 shutdown_connections 2024-04-16T17:07:49.773 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:49.772+0000 7fdaa7200640 1 -- 172.21.15.45:0/3099788559 wait complete. 2024-04-16T17:07:49.774 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:49.774+0000 7fdaa7200640 1 Processor -- start 2024-04-16T17:07:49.774 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:49.774+0000 7fdaa7200640 1 -- start start 2024-04-16T17:07:49.774 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:49.775+0000 7fdaa7200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdaa80743b0 0x7fdaa81a0650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:49.775 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:49.775+0000 7fdaa7200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdaa8074cd0 0x7fdaa81a0b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:49.776 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:49.775+0000 7fdaa7200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdaa810e6e0 0x7fdaa819a760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:49.777 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:49.775+0000 7fdaa7200640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fdaa81143c0 con 0x7fdaa8074cd0 2024-04-16T17:07:49.777 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:49.775+0000 7fdaa7200640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fdaa8114240 con 0x7fdaa80743b0 2024-04-16T17:07:49.777 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:49.775+0000 7fdaa7200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fdaa8114540 con 0x7fdaa810e6e0 2024-04-16T17:07:49.777 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:49.775+0000 7fdaa5e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdaa80743b0 0x7fdaa81a0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:49.777 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:49.775+0000 7fdaa5e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdaa80743b0 0x7fdaa81a0650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:33820/0 (socket says 172.21.15.45:33820) 2024-04-16T17:07:49.777 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:49.775+0000 7fdaa5e00640 1 -- 172.21.15.45:0/3324335229 learned_addr learned my addr 172.21.15.45:0/3324335229 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:07:49.777 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:49.775+0000 7fdaa6800640 1 --2- 172.21.15.45:0/3324335229 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdaa810e6e0 0x7fdaa819a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:49.777 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:49.775+0000 7fdaa5400640 1 --2- 172.21.15.45:0/3324335229 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdaa8074cd0 0x7fdaa81a0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:49.777 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:49.776+0000 7fdaa5e00640 1 -- 172.21.15.45:0/3324335229 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdaa810e6e0 msgr2=0x7fdaa819a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:49.777 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:49.776+0000 7fdaa5e00640 1 --2- 172.21.15.45:0/3324335229 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdaa810e6e0 0x7fdaa819a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:49.777 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:49.776+0000 7fdaa5e00640 1 -- 172.21.15.45:0/3324335229 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdaa8074cd0 msgr2=0x7fdaa81a0b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:49.777 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:49.776+0000 7fdaa5e00640 1 --2- 172.21.15.45:0/3324335229 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdaa8074cd0 0x7fdaa81a0b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:49.777 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:49.776+0000 7fdaa5e00640 1 -- 172.21.15.45:0/3324335229 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdaa819b020 con 0x7fdaa80743b0 2024-04-16T17:07:49.777 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:49.776+0000 7fdaa5400640 1 --2- 172.21.15.45:0/3324335229 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdaa8074cd0 0x7fdaa81a0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:07:49.778 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:49.776+0000 7fdaa5e00640 1 --2- 172.21.15.45:0/3324335229 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdaa80743b0 0x7fdaa81a0650 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fda9800d950 tx=0x7fda9800de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:07:49.778 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:49.776+0000 7fda9ea00640 1 -- 172.21.15.45:0/3324335229 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda98017070 con 0x7fdaa80743b0 2024-04-16T17:07:49.778 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:49.776+0000 7fda9ea00640 1 -- 172.21.15.45:0/3324335229 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fda98009c30 con 0x7fdaa80743b0 2024-04-16T17:07:49.778 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:49.776+0000 7fda9ea00640 1 -- 172.21.15.45:0/3324335229 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda9800f690 con 0x7fdaa80743b0 2024-04-16T17:07:49.778 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:49.776+0000 7fdaa7200640 1 -- 172.21.15.45:0/3324335229 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdaa819b310 con 0x7fdaa80743b0 2024-04-16T17:07:49.778 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:49.777+0000 7fdaa7200640 1 -- 172.21.15.45:0/3324335229 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdaa81a75d0 con 0x7fdaa80743b0 2024-04-16T17:07:49.780 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:49.780+0000 7fda9ea00640 1 -- 172.21.15.45:0/3324335229 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 25) ==== 138879+0+0 (secure 0 0 0) 0x7fda980109d0 con 0x7fdaa80743b0 2024-04-16T17:07:49.781 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:49.781+0000 7fdaaf61e740 1 Processor -- start 2024-04-16T17:07:49.781 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:49.781+0000 7fda9ea00640 1 --2- 172.21.15.45:0/3324335229 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fda8c0a3eb0 0x7fda8c0a6370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:49.782 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:49.781+0000 7fdaaf61e740 1 -- start start 2024-04-16T17:07:49.782 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:49.782+0000 7fda9ea00640 1 -- 172.21.15.45:0/3324335229 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fda980d09f0 con 0x7fdaa80743b0 2024-04-16T17:07:49.782 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:49.782+0000 7fdaa6800640 1 --2- 172.21.15.45:0/3324335229 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fda8c0a3eb0 0x7fda8c0a6370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:49.782 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:49.782+0000 7fdaaf61e740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561a249d07f0 0x561a249cf540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:49.782 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:49.782+0000 7fdaaf61e740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561a249cfa80 0x561a249cff00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:49.782 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:49.782+0000 7fdaaf61e740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561a249d9000 0x561a249db3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:49.783 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:49.782+0000 7fdaaf61e740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x561a24934780 con 0x561a249d9000 2024-04-16T17:07:49.783 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:49.782+0000 7fdaaf61e740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x561a248daf40 con 0x561a249d07f0 2024-04-16T17:07:49.783 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:49.782+0000 7fdaaf61e740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x561a24930740 con 0x561a249cfa80 2024-04-16T17:07:49.784 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:49.782+0000 7fdaa6800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561a249d9000 0x561a249db3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:49.784 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:49.782+0000 7fdaa5e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561a249d07f0 0x561a249cf540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:49.784 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:49.782+0000 7fdaa6800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561a249d9000 0x561a249db3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34220/0 (socket says 172.21.15.45:34220) 2024-04-16T17:07:49.784 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:49.782+0000 7fdaa6800640 1 -- 172.21.15.45:0/3697253128 learned_addr learned my addr 172.21.15.45:0/3697253128 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:07:49.784 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:49.783+0000 7fdaa5400640 1 --2- 172.21.15.45:0/3697253128 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561a249cfa80 0x561a249cff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:49.784 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:49.783+0000 7fdaa5e00640 1 -- 172.21.15.45:0/3697253128 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561a249cfa80 msgr2=0x561a249cff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:07:49.784 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:49.783+0000 7fdaa5e00640 1 --2- 172.21.15.45:0/3697253128 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561a249cfa80 0x561a249cff00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:49.785 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:49.783+0000 7fdaa5e00640 1 -- 172.21.15.45:0/3697253128 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561a249d9000 msgr2=0x561a249db3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:07:49.785 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:49.783+0000 7fdaa5e00640 1 --2- 172.21.15.45:0/3697253128 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561a249d9000 0x561a249db3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:49.785 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:49.783+0000 7fdaa5e00640 1 -- 172.21.15.45:0/3697253128 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561a24647210 con 0x561a249d07f0 2024-04-16T17:07:49.785 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:49.783+0000 7fdaa6800640 1 --2- 172.21.15.45:0/3697253128 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561a249d9000 0x561a249db3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:07:49.785 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:49.784+0000 7fdaa5e00640 1 --2- 172.21.15.45:0/3697253128 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561a249d07f0 0x561a249cf540 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fda980cdc10 tx=0x7fda9809bfe0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ca534dcc9ce782a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:07:49.785 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:49.784+0000 7fdaa6800640 1 --2- 172.21.15.45:0/3324335229 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fda8c0a3eb0 0x7fda8c0a6370 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fdaa0036040 tx=0x7fdaa0003c80 comp rx=0 tx=0).ready entity=mgr.25411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:07:49.785 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:49.784+0000 7fda93000640 1 -- 172.21.15.45:0/3697253128 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda98020070 con 0x561a249d07f0 2024-04-16T17:07:49.785 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:49.784+0000 7fda93000640 1 -- 172.21.15.45:0/3697253128 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fda9809a690 con 0x561a249d07f0 2024-04-16T17:07:49.785 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:49.784+0000 7fda93000640 1 -- 172.21.15.45:0/3697253128 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda9809ba40 con 0x561a249d07f0 2024-04-16T17:07:49.787 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:49.785+0000 7fdaaf61e740 1 -- 172.21.15.45:0/3697253128 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561a249d07f0 msgr2=0x561a249cf540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:49.788 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:49.785+0000 7fdaaf61e740 1 --2- 172.21.15.45:0/3697253128 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561a249d07f0 0x561a249cf540 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fda980cdc10 tx=0x7fda9809bfe0 comp rx=0 tx=0).stop 2024-04-16T17:07:49.788 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:49.785+0000 7fdaaf61e740 1 -- 172.21.15.45:0/3697253128 shutdown_connections 2024-04-16T17:07:49.788 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:49.785+0000 7fdaaf61e740 1 --2- 172.21.15.45:0/3697253128 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561a249cfa80 0x561a249cff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:49.788 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:49.785+0000 7fdaaf61e740 1 --2- 172.21.15.45:0/3697253128 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561a249d9000 0x561a249db3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:49.788 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:49.785+0000 7fdaaf61e740 1 --2- 172.21.15.45:0/3697253128 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561a249d07f0 0x561a249cf540 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:49.788 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:49.785+0000 7fdaaf61e740 1 -- 172.21.15.45:0/3697253128 >> 172.21.15.45:0/3697253128 conn(0x561a2496a5d0 msgr2=0x561a249ca770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:07:49.788 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:49.785+0000 7fdaaf61e740 1 -- 172.21.15.45:0/3697253128 shutdown_connections 2024-04-16T17:07:49.788 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:49.785+0000 7fdaaf61e740 1 -- 172.21.15.45:0/3697253128 wait complete. 2024-04-16T17:07:49.788 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:49.786+0000 7fdaaf61e740 1 -- start start 2024-04-16T17:07:49.788 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:49.786+0000 7fdaaf61e740 1 Processor -- start 2024-04-16T17:07:49.788 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:49.786+0000 7fdaaf61e740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561a249cfa80 0x561a249d5ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:49.788 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:49.787+0000 7fdaaf61e740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561a249d9000 0x561a249d6530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:49.788 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:49.787+0000 7fdaaf61e740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561a249d6a70 0x561a249e5120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:49.788 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:49.787+0000 7fdaaf61e740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x561a249dbc70 con 0x561a249cfa80 2024-04-16T17:07:49.788 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:49.787+0000 7fdaaf61e740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x561a248ee390 con 0x561a249d6a70 2024-04-16T17:07:49.789 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:49.787+0000 7fdaaf61e740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x561a249dbdf0 con 0x561a249d9000 2024-04-16T17:07:49.789 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:49.787+0000 7fdaaf61e740 10 client.0 _ll_register_callbacks cb 0x561a248d7f60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:07:49.789 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:49.787+0000 7fdaa5e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561a249cfa80 0x561a249d5ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:49.789 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:49.787+0000 7fdaa5e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561a249cfa80 0x561a249d5ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34234/0 (socket says 172.21.15.45:34234) 2024-04-16T17:07:49.789 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:49.787+0000 7fdaa5e00640 1 -- 172.21.15.45:0/2767442022 learned_addr learned my addr 172.21.15.45:0/2767442022 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:07:49.789 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:49.787+0000 7fdaa5400640 1 --2- 172.21.15.45:0/2767442022 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561a249d9000 0x561a249d6530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:49.789 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:49.787+0000 7fdaa6800640 1 --2- 172.21.15.45:0/2767442022 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561a249d6a70 0x561a249e5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:49.789 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:49.788+0000 7fdaa5e00640 1 -- 172.21.15.45:0/2767442022 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561a249d9000 msgr2=0x561a249d6530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:07:49.789 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:49.788+0000 7fdaa5e00640 1 --2- 172.21.15.45:0/2767442022 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561a249d9000 0x561a249d6530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:49.789 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:49.788+0000 7fdaa5e00640 1 -- 172.21.15.45:0/2767442022 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561a249d6a70 msgr2=0x561a249e5120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:07:49.789 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:49.788+0000 7fdaa5e00640 1 --2- 172.21.15.45:0/2767442022 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561a249d6a70 0x561a249e5120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:49.789 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:49.788+0000 7fdaa5e00640 1 -- 172.21.15.45:0/2767442022 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561a24713620 con 0x561a249cfa80 2024-04-16T17:07:49.791 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:49.788+0000 7fdaa5e00640 1 --2- 172.21.15.45:0/2767442022 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561a249cfa80 0x561a249d5ff0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fda9809c090 tx=0x7fda98098c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a8f416a9be3173d server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:07:49.791 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:49.788+0000 7fda87400640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-16T17:07:49.791 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:49.788+0000 7fda87400640 1 -- 172.21.15.45:0/2767442022 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda980b5060 con 0x561a249cfa80 2024-04-16T17:07:49.791 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:49.788+0000 7fda87400640 1 -- 172.21.15.45:0/2767442022 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fda9809b9e0 con 0x561a249cfa80 2024-04-16T17:07:49.792 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:49.788+0000 7fdaaf61e740 1 -- 172.21.15.45:0/2767442022 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fda9809a690 con 0x561a249cfa80 2024-04-16T17:07:49.792 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:49.789+0000 7fda87400640 1 -- 172.21.15.45:0/2767442022 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fda98020070 con 0x561a249cfa80 2024-04-16T17:07:49.792 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:49.789+0000 7fdaaf61e740 10 client.15783 fetch_fsmap learned FSMap version 25 2024-04-16T17:07:49.792 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:49.789+0000 7fdaaf61e740 1 -- 172.21.15.45:0/2767442022 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x561a249e8390 con 0x561a249cfa80 2024-04-16T17:07:49.792 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:49.789+0000 7fda87400640 1 -- 172.21.15.45:0/2767442022 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fda9809a690 con 0x561a249cfa80 2024-04-16T17:07:49.792 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:49.789+0000 7fda87400640 1 -- 172.21.15.45:0/2767442022 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fda9809b250 con 0x561a249cfa80 2024-04-16T17:07:49.792 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:49.789+0000 7fdaaf61e740 10 client.15783 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T17:07:49.792 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:49.789+0000 7fdaaf61e740 10 client.15783 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T17:07:49.792 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:49.789+0000 7fdaaf61e740 20 client.15783 populate_metadata read hostname 'smithi045' 2024-04-16T17:07:49.792 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:49.790+0000 7fdaaf61e740 1 --2- 172.21.15.45:0/2767442022 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x561a249e96e0 0x561a249ebad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:49.792 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:49.790+0000 7fdaaf61e740 4 client.15783 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:07:49.792 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:49.790+0000 7fdaaf61e740 1 -- 172.21.15.45:0/2767442022 --> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561a249421a0 con 0x561a249e96e0 2024-04-16T17:07:49.792 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:49.790+0000 7fdaa5400640 1 --2- 172.21.15.45:0/2767442022 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x561a249e96e0 0x561a249ebad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:49.793 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:49.790+0000 7fdaa5400640 1 --2- 172.21.15.45:0/2767442022 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x561a249e96e0 0x561a249ebad0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dbf505cf76d04e97 server_cookie=d0d6d9154f34276c in_seq=0 out_seq=0 2024-04-16T17:07:49.793 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:49.790+0000 7fda87400640 10 client.15783 ms_handle_connect on v2:172.21.15.104:6838/3660070592 2024-04-16T17:07:49.793 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:49.791+0000 7fda87400640 1 -- 172.21.15.45:0/2767442022 <== mds.0 v2:172.21.15.104:6838/3660070592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561a249421a0 con 0x561a249e96e0 2024-04-16T17:07:49.793 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:49.791+0000 7fda87400640 10 client.15783 handle_command_reply: tid=0 2024-04-16T17:07:49.793 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:49.791+0000 7fdaaf61e740 1 client.15783 shutdown 2024-04-16T17:07:49.794 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:49.792+0000 7fdaaf61e740 1 -- 172.21.15.45:0/2767442022 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561a249cfa80 msgr2=0x561a249d5ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:49.794 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:49.792+0000 7fdaaf61e740 1 --2- 172.21.15.45:0/2767442022 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561a249cfa80 0x561a249d5ff0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fda9809c090 tx=0x7fda98098c30 comp rx=0 tx=0).stop 2024-04-16T17:07:49.794 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:49.792+0000 7fdaaf61e740 1 -- 172.21.15.45:0/2767442022 shutdown_connections 2024-04-16T17:07:49.794 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:49.792+0000 7fdaaf61e740 1 --2- 172.21.15.45:0/2767442022 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x561a249e96e0 0x561a249ebad0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:49.794 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:49.792+0000 7fdaaf61e740 1 --2- 172.21.15.45:0/2767442022 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561a249d9000 0x561a249d6530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:49.794 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:49.792+0000 7fdaaf61e740 1 --2- 172.21.15.45:0/2767442022 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561a249d6a70 0x561a249e5120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:49.794 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:49.792+0000 7fdaaf61e740 1 --2- 172.21.15.45:0/2767442022 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561a249cfa80 0x561a249d5ff0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:49.794 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:49.792+0000 7fdaaf61e740 1 -- 172.21.15.45:0/2767442022 >> 172.21.15.45:0/2767442022 conn(0x561a2496a5d0 msgr2=0x561a249cc4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:07:49.794 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:49.792+0000 7fda87400640 0 client.15783 ms_handle_reset on v2:172.21.15.104:6838/3660070592 2024-04-16T17:07:49.794 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:49.792+0000 7fdaaf61e740 1 -- 172.21.15.45:0/2767442022 shutdown_connections 2024-04-16T17:07:49.794 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:49.792+0000 7fdaaf61e740 1 -- 172.21.15.45:0/2767442022 wait complete. 2024-04-16T17:07:49.795 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:49.792+0000 7fdaaf61e740 20 client.15783 trim_cache size 0 max 16384 2024-04-16T17:07:49.808 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:49.808+0000 7fdaaf61e740 1 Processor -- start 2024-04-16T17:07:49.808 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:49.808+0000 7fdaaf61e740 1 -- start start 2024-04-16T17:07:49.808 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:49.808+0000 7fdaaf61e740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561a249cfa80 0x561a249d5150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:49.809 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:49.808+0000 7fdaaf61e740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561a249d6a70 0x561a249d7f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:49.810 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:49.809+0000 7fdaaf61e740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561a249d9000 0x561a249f1090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:49.810 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:49.809+0000 7fdaaf61e740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fdaa81b3e40 con 0x561a249d6a70 2024-04-16T17:07:49.810 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:49.809+0000 7fdaaf61e740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fdaa81b1710 con 0x561a249d9000 2024-04-16T17:07:49.810 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:49.809+0000 7fdaaf61e740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fdaa81af540 con 0x561a249cfa80 2024-04-16T17:07:49.810 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:49.809+0000 7fdaa5400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561a249d6a70 0x561a249d7f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:49.810 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:49.809+0000 7fdaa5400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561a249d6a70 0x561a249d7f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34244/0 (socket says 172.21.15.45:34244) 2024-04-16T17:07:49.810 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:49.809+0000 7fdaa5400640 1 -- 172.21.15.45:0/2530108529 learned_addr learned my addr 172.21.15.45:0/2530108529 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:07:49.811 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:49.809+0000 7fdaa5e00640 1 --2- 172.21.15.45:0/2530108529 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561a249cfa80 0x561a249d5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:49.811 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:49.809+0000 7fdaa6800640 1 --2- 172.21.15.45:0/2530108529 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561a249d9000 0x561a249f1090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:49.811 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:49.809+0000 7fdaa5400640 1 -- 172.21.15.45:0/2530108529 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561a249cfa80 msgr2=0x561a249d5150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:07:49.811 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:49.809+0000 7fdaa5400640 1 --2- 172.21.15.45:0/2530108529 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561a249cfa80 0x561a249d5150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:49.811 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:49.809+0000 7fdaa5400640 1 -- 172.21.15.45:0/2530108529 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561a249d9000 msgr2=0x561a249f1090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:07:49.811 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:49.809+0000 7fdaa5400640 1 --2- 172.21.15.45:0/2530108529 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561a249d9000 0x561a249f1090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:49.811 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:49.809+0000 7fdaa5400640 1 -- 172.21.15.45:0/2530108529 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561a249ced50 con 0x561a249d6a70 2024-04-16T17:07:49.811 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:49.810+0000 7fdaa5e00640 1 --2- 172.21.15.45:0/2530108529 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561a249cfa80 0x561a249d5150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:07:49.811 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:49.810+0000 7fdaa5400640 1 --2- 172.21.15.45:0/2530108529 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561a249d6a70 0x561a249d7f30 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fda94000c00 tx=0x7fda94014840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f87e3b786d11850 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:07:49.811 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:49.810+0000 7fda92600640 1 -- 172.21.15.45:0/2530108529 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda94024070 con 0x561a249d6a70 2024-04-16T17:07:49.811 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:49.810+0000 7fda92600640 1 -- 172.21.15.45:0/2530108529 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fda9401e470 con 0x561a249d6a70 2024-04-16T17:07:49.811 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:49.810+0000 7fda92600640 1 -- 172.21.15.45:0/2530108529 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda94003930 con 0x561a249d6a70 2024-04-16T17:07:49.812 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:49.811+0000 7fdaaf61e740 1 -- 172.21.15.45:0/2530108529 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561a249d6a70 msgr2=0x561a249d7f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:49.812 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:49.811+0000 7fdaaf61e740 1 --2- 172.21.15.45:0/2530108529 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561a249d6a70 0x561a249d7f30 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fda94000c00 tx=0x7fda94014840 comp rx=0 tx=0).stop 2024-04-16T17:07:49.812 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:49.811+0000 7fdaaf61e740 1 -- 172.21.15.45:0/2530108529 shutdown_connections 2024-04-16T17:07:49.813 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:49.811+0000 7fdaaf61e740 1 --2- 172.21.15.45:0/2530108529 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561a249d9000 0x561a249f1090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:49.813 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:49.811+0000 7fdaaf61e740 1 --2- 172.21.15.45:0/2530108529 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561a249d6a70 0x561a249d7f30 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:49.813 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:49.811+0000 7fdaaf61e740 1 --2- 172.21.15.45:0/2530108529 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561a249cfa80 0x561a249d5150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:49.813 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:49.811+0000 7fdaaf61e740 1 -- 172.21.15.45:0/2530108529 >> 172.21.15.45:0/2530108529 conn(0x561a2496a5d0 msgr2=0x561a249d7820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:07:49.813 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:49.811+0000 7fdaaf61e740 1 -- 172.21.15.45:0/2530108529 shutdown_connections 2024-04-16T17:07:49.813 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:49.811+0000 7fdaaf61e740 1 -- 172.21.15.45:0/2530108529 wait complete. 2024-04-16T17:07:49.813 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:49.812+0000 7fdaaf61e740 1 -- start start 2024-04-16T17:07:49.813 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:49.812+0000 7fdaaf61e740 1 Processor -- start 2024-04-16T17:07:49.815 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:49.813+0000 7fdaaf61e740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561a249cfa80 0x561a249e12e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:49.815 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:49.813+0000 7fdaaf61e740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561a249d6a70 0x561a249d2820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:49.815 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:49.813+0000 7fdaaf61e740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561a249d9000 0x561a249d2d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:49.815 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:49.813+0000 7fdaaf61e740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fdaa81afbd0 con 0x561a249d6a70 2024-04-16T17:07:49.815 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:49.813+0000 7fdaaf61e740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fdaa81b1040 con 0x561a249cfa80 2024-04-16T17:07:49.815 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:49.813+0000 7fdaaf61e740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fdaa81b02a0 con 0x561a249d9000 2024-04-16T17:07:49.815 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:49.813+0000 7fdaaf61e740 10 client.0 _ll_register_callbacks cb 0x7fdaa0009a60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:07:49.815 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:49.813+0000 7fdaa5400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561a249d6a70 0x561a249d2820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:49.815 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:49.813+0000 7fdaa5400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561a249d6a70 0x561a249d2820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34258/0 (socket says 172.21.15.45:34258) 2024-04-16T17:07:49.815 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:49.813+0000 7fdaa5e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561a249cfa80 0x561a249e12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:49.815 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:49.813+0000 7fdaa5400640 1 -- 172.21.15.45:0/1952850209 learned_addr learned my addr 172.21.15.45:0/1952850209 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:07:49.815 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:49.813+0000 7fdaa6800640 1 --2- 172.21.15.45:0/1952850209 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561a249d9000 0x561a249d2d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:49.815 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:49.814+0000 7fdaa5400640 1 -- 172.21.15.45:0/1952850209 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561a249d9000 msgr2=0x561a249d2d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:07:49.816 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:49.814+0000 7fdaa5400640 1 --2- 172.21.15.45:0/1952850209 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561a249d9000 0x561a249d2d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:49.816 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:49.814+0000 7fdaa5400640 1 -- 172.21.15.45:0/1952850209 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561a249cfa80 msgr2=0x561a249e12e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:07:49.816 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:49.814+0000 7fdaa5400640 1 --2- 172.21.15.45:0/1952850209 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561a249cfa80 0x561a249e12e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:49.816 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:49.814+0000 7fdaa5400640 1 -- 172.21.15.45:0/1952850209 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561a249d33d0 con 0x561a249d6a70 2024-04-16T17:07:49.816 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:49.814+0000 7fdaa5400640 1 --2- 172.21.15.45:0/1952850209 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561a249d6a70 0x561a249d2820 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fda94005780 tx=0x7fda94014190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43e691991d28ad79 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:07:49.816 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:49.814+0000 7fda87400640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-16T17:07:49.816 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:49.814+0000 7fda87400640 1 -- 172.21.15.45:0/1952850209 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda94024040 con 0x561a249d6a70 2024-04-16T17:07:49.816 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:49.814+0000 7fdaaf61e740 1 -- 172.21.15.45:0/1952850209 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fda9401e470 con 0x561a249d6a70 2024-04-16T17:07:49.816 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:49.815+0000 7fda87400640 1 -- 172.21.15.45:0/1952850209 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fda9401e820 con 0x561a249d6a70 2024-04-16T17:07:49.816 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:49.815+0000 7fda87400640 1 -- 172.21.15.45:0/1952850209 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fda94036050 con 0x561a249d6a70 2024-04-16T17:07:49.816 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:49.815+0000 7fdaaf61e740 10 client.15789 fetch_fsmap learned FSMap version 25 2024-04-16T17:07:49.816 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:49.815+0000 7fdaaf61e740 1 -- 172.21.15.45:0/1952850209 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x561a24a57f90 con 0x561a249d6a70 2024-04-16T17:07:49.817 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:49.815+0000 7fda87400640 1 -- 172.21.15.45:0/1952850209 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fda9401e470 con 0x561a249d6a70 2024-04-16T17:07:49.817 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:49.815+0000 7fda87400640 1 -- 172.21.15.45:0/1952850209 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fda94003c20 con 0x561a249d6a70 2024-04-16T17:07:49.817 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:49.815+0000 7fdaaf61e740 10 client.15789 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T17:07:49.817 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:49.815+0000 7fdaaf61e740 10 client.15789 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T17:07:49.818 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:49.815+0000 7fdaaf61e740 20 client.15789 populate_metadata read hostname 'smithi045' 2024-04-16T17:07:49.818 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:49.815+0000 7fdaaf61e740 1 --2- 172.21.15.45:0/1952850209 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x561a24a585e0 0x561a24a5aa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:07:49.818 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:49.815+0000 7fdaaf61e740 4 client.15789 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-16T17:07:49.818 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:49.815+0000 7fdaaf61e740 1 -- 172.21.15.45:0/1952850209 --> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561a2493c010 con 0x561a24a585e0 2024-04-16T17:07:49.818 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:49.816+0000 7fdaa5e00640 1 --2- 172.21.15.45:0/1952850209 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x561a24a585e0 0x561a24a5aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:07:49.818 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:49.816+0000 7fdaa5e00640 1 --2- 172.21.15.45:0/1952850209 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x561a24a585e0 0x561a24a5aa00 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=374774398b24935 server_cookie=1fba2577ecfb9ee2 in_seq=0 out_seq=0 2024-04-16T17:07:49.818 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:49.816+0000 7fda87400640 10 client.15789 ms_handle_connect on v2:172.21.15.104:6838/3660070592 2024-04-16T17:07:49.821 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:49.820+0000 7fda87400640 1 -- 172.21.15.45:0/1952850209 <== mds.0 v2:172.21.15.104:6838/3660070592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x561a2493c010 con 0x561a24a585e0 2024-04-16T17:07:49.821 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:49.820+0000 7fda87400640 10 client.15789 handle_command_reply: tid=0 2024-04-16T17:07:49.821 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:49.820+0000 7fdaaf61e740 1 client.15789 shutdown 2024-04-16T17:07:49.821 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:49.821+0000 7fdaaf61e740 1 -- 172.21.15.45:0/1952850209 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561a249d6a70 msgr2=0x561a249d2820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:49.821 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:49.821+0000 7fdaaf61e740 1 --2- 172.21.15.45:0/1952850209 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561a249d6a70 0x561a249d2820 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fda94005780 tx=0x7fda94014190 comp rx=0 tx=0).stop 2024-04-16T17:07:49.821 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:49.821+0000 7fdaaf61e740 1 -- 172.21.15.45:0/1952850209 shutdown_connections 2024-04-16T17:07:49.821 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:49.821+0000 7fdaaf61e740 1 --2- 172.21.15.45:0/1952850209 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x561a24a585e0 0x561a24a5aa00 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:49.821 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:49.821+0000 7fdaaf61e740 1 --2- 172.21.15.45:0/1952850209 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561a249d9000 0x561a249d2d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:49.822 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:49.821+0000 7fdaaf61e740 1 --2- 172.21.15.45:0/1952850209 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561a249d6a70 0x561a249d2820 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:49.822 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:49.821+0000 7fdaaf61e740 1 --2- 172.21.15.45:0/1952850209 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561a249cfa80 0x561a249e12e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:49.822 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:49.821+0000 7fda87400640 0 client.15789 ms_handle_reset on v2:172.21.15.104:6838/3660070592 2024-04-16T17:07:49.822 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:49.821+0000 7fdaaf61e740 1 -- 172.21.15.45:0/1952850209 >> 172.21.15.45:0/1952850209 conn(0x561a2496a5d0 msgr2=0x561a249d80b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:07:49.822 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:49.821+0000 7fdaaf61e740 1 -- 172.21.15.45:0/1952850209 shutdown_connections 2024-04-16T17:07:49.822 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:49.821+0000 7fdaaf61e740 1 -- 172.21.15.45:0/1952850209 wait complete. 2024-04-16T17:07:49.822 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:49.821+0000 7fdaaf61e740 20 client.15789 trim_cache size 0 max 16384 2024-04-16T17:07:49.822 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-04-16T17:07:49.823 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-04-16T17:07:49.823 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-04-16T17:07:49.823 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-04-16T17:07:49.823 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:49.822+0000 7fdaa7200640 1 -- 172.21.15.45:0/3324335229 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fda8c0a3eb0 msgr2=0x7fda8c0a6370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:49.823 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:49.822+0000 7fdaa7200640 1 --2- 172.21.15.45:0/3324335229 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fda8c0a3eb0 0x7fda8c0a6370 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fdaa0036040 tx=0x7fdaa0003c80 comp rx=0 tx=0).stop 2024-04-16T17:07:49.823 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:49.822+0000 7fdaa7200640 1 -- 172.21.15.45:0/3324335229 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdaa80743b0 msgr2=0x7fdaa81a0650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:07:49.823 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:49.822+0000 7fdaa7200640 1 --2- 172.21.15.45:0/3324335229 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdaa80743b0 0x7fdaa81a0650 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fda9800d950 tx=0x7fda9800de10 comp rx=0 tx=0).stop 2024-04-16T17:07:49.824 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:49.822+0000 7fdaa7200640 1 -- 172.21.15.45:0/3324335229 shutdown_connections 2024-04-16T17:07:49.824 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:49.822+0000 7fdaa7200640 1 --2- 172.21.15.45:0/3324335229 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fda8c0a3eb0 0x7fda8c0a6370 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:49.824 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:49.822+0000 7fdaa7200640 1 --2- 172.21.15.45:0/3324335229 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdaa810e6e0 0x7fdaa819a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:49.824 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:49.822+0000 7fdaa7200640 1 --2- 172.21.15.45:0/3324335229 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdaa8074cd0 0x7fdaa81a0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:49.824 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:49.822+0000 7fdaa7200640 1 --2- 172.21.15.45:0/3324335229 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdaa80743b0 0x7fdaa81a0650 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:07:49.824 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:49.822+0000 7fdaa7200640 1 -- 172.21.15.45:0/3324335229 >> 172.21.15.45:0/3324335229 conn(0x7fdaa80fd950 msgr2=0x7fdaa80fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:07:49.824 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:49.822+0000 7fdaa7200640 1 -- 172.21.15.45:0/3324335229 shutdown_connections 2024-04-16T17:07:49.824 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:07:49.822+0000 7fdaa7200640 1 -- 172.21.15.45:0/3324335229 wait complete. 2024-04-16T17:07:50.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:07:50 smithi104 ceph-mon[36457]: pgmap v35: 129 pgs: 129 active+clean; 1.4 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-04-16T17:07:50.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:07:50 smithi045 ceph-mon[31676]: pgmap v35: 129 pgs: 129 active+clean; 1.4 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-04-16T17:07:50.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:07:50 smithi117 ceph-mon[36730]: pgmap v35: 129 pgs: 129 active+clean; 1.4 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-04-16T17:07:52.677 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:07:52 smithi104 ceph-mon[36457]: pgmap v36: 129 pgs: 129 active+clean; 1.4 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.4 KiB/s wr, 9 op/s 2024-04-16T17:07:52.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:07:52 smithi045 ceph-mon[31676]: pgmap v36: 129 pgs: 129 active+clean; 1.4 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.4 KiB/s wr, 9 op/s 2024-04-16T17:07:52.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:07:52 smithi117 ceph-mon[36730]: pgmap v36: 129 pgs: 129 active+clean; 1.4 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.4 KiB/s wr, 9 op/s 2024-04-16T17:07:54.677 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:07:54 smithi104 ceph-mon[36457]: pgmap v37: 129 pgs: 129 active+clean; 1.4 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-16T17:07:54.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:07:54 smithi104 ceph-mon[36457]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T17:07:54.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:07:54 smithi045 ceph-mon[31676]: pgmap v37: 129 pgs: 129 active+clean; 1.4 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-16T17:07:54.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:07:54 smithi045 ceph-mon[31676]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T17:07:54.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:07:54 smithi117 ceph-mon[36730]: pgmap v37: 129 pgs: 129 active+clean; 1.4 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-16T17:07:54.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:07:54 smithi117 ceph-mon[36730]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T17:07:55.629 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:07:55 smithi045 ceph-mon[31676]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:07:55.629 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:07:55 smithi045 ceph-mon[31676]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T17:07:55.629 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:07:55 smithi045 ceph-mon[31676]: from='mgr.25411 ' entity='mgr.x' 2024-04-16T17:07:55.677 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:07:55 smithi104 ceph-mon[36457]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:07:55.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:07:55 smithi104 ceph-mon[36457]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T17:07:55.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:07:55 smithi104 ceph-mon[36457]: from='mgr.25411 ' entity='mgr.x' 2024-04-16T17:07:55.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:07:55 smithi117 ceph-mon[36730]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:07:55.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:07:55 smithi117 ceph-mon[36730]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T17:07:55.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:07:55 smithi117 ceph-mon[36730]: from='mgr.25411 ' entity='mgr.x' 2024-04-16T17:07:56.248 INFO:tasks.workunit.client.0.smithi045.stderr:Updating files: 61% (7734/12482) Updating files: 62% (7739/12482) Updating files: 63% (7864/12482) Updating files: 64% (7989/12482) Updating files: 65% (8114/12482) Updating files: 66% (8239/12482) Updating files: 67% (8363/12482) Updating files: 68% (8488/12482) Updating files: 69% (8613/12482) Updating files: 70% (8738/12482) Updating files: 71% (8863/12482) Updating files: 72% (8988/12482) Updating files: 73% (9112/12482) Updating files: 74% (9237/12482) Updating files: 75% (9362/12482) Updating files: 76% (9487/12482) Updating files: 77% (9612/12482) Updating files: 78% (9736/12482) Updating files: 79% (9861/12482) Updating files: 80% (9986/12482) Updating files: 81% (10111/12482) Updating files: 82% (10236/12482) Updating files: 83% (10361/12482) Updating files: 84% (10485/12482) Updating files: 85% (10610/12482) Updating files: 86% (10735/12482) Updating files: 87% (10860/12482) Updating files: 88% (10985/12482) Updating files: 89% (11109/12482) Updating files: 90% (11234/12482) Updating files: 91% (11359/12482) Updating files: 92% (11484/12482) Updating files: 93% (11609/12482) Updating files: 94% (11734/12482) Updating files: 95% (11858/12482) Updating files: 96% (11983/12482) Updating files: 97% (12108/12482) Updating files: 98% (12233/12482) Updating files: 98% (12301/12482) Updating files: 99% (12358/12482) Updating files: 100% (12482/12482) Updating files: 100% (12482/12482), done. 2024-04-16T17:07:56.447 INFO:tasks.workunit.client.0.smithi045.stderr:Note: switching to 'b137e4a675398e295567f9652210865a4d573fac'. 2024-04-16T17:07:56.447 INFO:tasks.workunit.client.0.smithi045.stderr: 2024-04-16T17:07:56.447 INFO:tasks.workunit.client.0.smithi045.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-04-16T17:07:56.448 INFO:tasks.workunit.client.0.smithi045.stderr:changes and commit them, and you can discard any commits you make in this 2024-04-16T17:07:56.448 INFO:tasks.workunit.client.0.smithi045.stderr:state without impacting any branches by switching back to a branch. 2024-04-16T17:07:56.448 INFO:tasks.workunit.client.0.smithi045.stderr: 2024-04-16T17:07:56.448 INFO:tasks.workunit.client.0.smithi045.stderr:If you want to create a new branch to retain commits you create, you may 2024-04-16T17:07:56.448 INFO:tasks.workunit.client.0.smithi045.stderr:do so (now or later) by using -c with the switch command. Example: 2024-04-16T17:07:56.448 INFO:tasks.workunit.client.0.smithi045.stderr: 2024-04-16T17:07:56.448 INFO:tasks.workunit.client.0.smithi045.stderr: git switch -c 2024-04-16T17:07:56.448 INFO:tasks.workunit.client.0.smithi045.stderr: 2024-04-16T17:07:56.448 INFO:tasks.workunit.client.0.smithi045.stderr:Or undo this operation with: 2024-04-16T17:07:56.448 INFO:tasks.workunit.client.0.smithi045.stderr: 2024-04-16T17:07:56.448 INFO:tasks.workunit.client.0.smithi045.stderr: git switch - 2024-04-16T17:07:56.448 INFO:tasks.workunit.client.0.smithi045.stderr: 2024-04-16T17:07:56.448 INFO:tasks.workunit.client.0.smithi045.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-04-16T17:07:56.448 INFO:tasks.workunit.client.0.smithi045.stderr: 2024-04-16T17:07:56.448 INFO:tasks.workunit.client.0.smithi045.stderr:HEAD is now at b137e4a6753 qa: enable dynamic debug for dbench workunit 2024-04-16T17:07:56.451 DEBUG:teuthology.orchestra.run.smithi045:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-04-16T17:07:56.512 INFO:tasks.workunit.client.0.smithi045.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-04-16T17:07:56.521 INFO:tasks.workunit.client.0.smithi045.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-04-16T17:07:56.522 INFO:tasks.workunit.client.0.smithi045.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-04-16T17:07:56.677 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:07:56 smithi104 ceph-mon[36457]: pgmap v38: 129 pgs: 129 active+clean; 1.4 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-16T17:07:56.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:07:56 smithi045 ceph-mon[31676]: pgmap v38: 129 pgs: 129 active+clean; 1.4 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-16T17:07:56.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:07:56 smithi117 ceph-mon[36730]: pgmap v38: 129 pgs: 129 active+clean; 1.4 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-16T17:07:57.457 INFO:tasks.workunit.client.0.smithi045.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-04-16T17:07:57.611 INFO:tasks.workunit.client.0.smithi045.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-04-16T17:07:57.703 INFO:tasks.workunit.client.0.smithi045.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-04-16T17:07:57.709 INFO:tasks.workunit.client.0.smithi045.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-04-16T17:07:57.710 INFO:tasks.workunit.client.0.smithi045.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-04-16T17:07:57.799 INFO:tasks.workunit.client.0.smithi045.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-04-16T17:07:57.810 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-04-16T17:07:57.810 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-04-16T17:07:57.866 INFO:tasks.workunit:Running workunits matching suites/dbench.sh on client.0... 2024-04-16T17:07:57.867 INFO:tasks.workunit:Running workunit suites/dbench.sh... 2024-04-16T17:07:57.868 DEBUG:teuthology.orchestra.run.smithi045:workunit test suites/dbench.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=b137e4a675398e295567f9652210865a4d573fac TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh 2024-04-16T17:07:57.946 INFO:tasks.workunit.client.0.smithi045.stderr:+ sudo -s eval 'echo -n '\''module ceph +mp'\'' > /sys/kernel/debug/dynamic_debug/control' 2024-04-16T17:07:58.014 INFO:tasks.workunit.client.0.smithi045.stderr:+ dbench 1 2024-04-16T17:07:58.098 INFO:tasks.workunit.client.0.smithi045.stdout:dbench version 4.00 - Copyright Andrew Tridgell 1999-2004 2024-04-16T17:07:58.098 INFO:tasks.workunit.client.0.smithi045.stdout: 2024-04-16T17:07:58.098 INFO:tasks.workunit.client.0.smithi045.stdout:Running for 600 seconds with load '/usr/share/dbench/client.txt' and minimum warmup 120 secs 2024-04-16T17:07:58.102 INFO:tasks.workunit.client.0.smithi045.stdout:0 of 1 processes prepared for launch 0 sec 2024-04-16T17:07:58.202 INFO:tasks.workunit.client.0.smithi045.stdout:1 of 1 processes prepared for launch 0 sec 2024-04-16T17:07:58.202 INFO:tasks.workunit.client.0.smithi045.stdout:releasing clients 2024-04-16T17:07:58.677 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:07:58 smithi104 ceph-mon[36457]: pgmap v39: 129 pgs: 129 active+clean; 1.4 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-16T17:07:58.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:07:58 smithi117 ceph-mon[36730]: pgmap v39: 129 pgs: 129 active+clean; 1.4 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-16T17:07:59.202 INFO:tasks.workunit.client.0.smithi045.stdout: 1 245 9.34 MB/sec warmup 1 sec latency 73.876 ms 2024-04-16T17:08:00.202 INFO:tasks.workunit.client.0.smithi045.stdout: 1 477 8.34 MB/sec warmup 2 sec latency 65.095 ms 2024-04-16T17:08:00.677 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:08:00 smithi104 ceph-mon[36457]: pgmap v40: 129 pgs: 129 active+clean; 1.4 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-16T17:08:00.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:08:00 smithi117 ceph-mon[36730]: pgmap v40: 129 pgs: 129 active+clean; 1.4 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-16T17:08:01.202 INFO:tasks.workunit.client.0.smithi045.stdout: 1 1691 8.74 MB/sec warmup 3 sec latency 16.004 ms 2024-04-16T17:08:01.671 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-16T17:08:01.671 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-16T17:08:01.818 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:07:58 smithi045 ceph-mon[31676]: pgmap v39: 129 pgs: 129 active+clean; 1.4 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-16T17:08:01.818 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:08:00 smithi045 ceph-mon[31676]: pgmap v40: 129 pgs: 129 active+clean; 1.4 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-16T17:08:01.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:01.893+0000 7fa18a600640 1 -- 172.21.15.45:0/60169109 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa184101250 msgr2=0x7fa1841016b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:01.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:01.893+0000 7fa18a600640 1 --2- 172.21.15.45:0/60169109 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa184101250 0x7fa1841016b0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fa16c009930 tx=0x7fa16c02bc20 comp rx=0 tx=0).stop 2024-04-16T17:08:01.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:01.893+0000 7fa18a600640 1 -- 172.21.15.45:0/60169109 shutdown_connections 2024-04-16T17:08:01.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:01.893+0000 7fa18a600640 1 --2- 172.21.15.45:0/60169109 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa184101bf0 0x7fa18410dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:01.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:01.893+0000 7fa18a600640 1 --2- 172.21.15.45:0/60169109 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa184101250 0x7fa1841016b0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:01.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:01.893+0000 7fa18a600640 1 --2- 172.21.15.45:0/60169109 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa1841070e0 0x7fa1841074c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:01.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:01.893+0000 7fa18a600640 1 -- 172.21.15.45:0/60169109 >> 172.21.15.45:0/60169109 conn(0x7fa184076d00 msgr2=0x7fa184077110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:08:01.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:01.893+0000 7fa18a600640 1 -- 172.21.15.45:0/60169109 shutdown_connections 2024-04-16T17:08:01.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:01.894+0000 7fa18a600640 1 -- 172.21.15.45:0/60169109 wait complete. 2024-04-16T17:08:01.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:01.894+0000 7fa18a600640 1 Processor -- start 2024-04-16T17:08:01.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:01.895+0000 7fa18a600640 1 -- start start 2024-04-16T17:08:01.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:01.895+0000 7fa18a600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa184101250 0x7fa1841a4bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:01.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:01.895+0000 7fa18a600640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa184101bf0 0x7fa1841a5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:01.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:01.895+0000 7fa18a600640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa1841070e0 0x7fa18410cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:01.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:01.895+0000 7fa18a600640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fa184112450 con 0x7fa184101250 2024-04-16T17:08:01.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:01.895+0000 7fa18a600640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fa1841122d0 con 0x7fa1841070e0 2024-04-16T17:08:01.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:01.895+0000 7fa18a600640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa1841125d0 con 0x7fa184101bf0 2024-04-16T17:08:01.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:01.895+0000 7fa17aa00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa184101bf0 0x7fa1841a5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:01.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:01.895+0000 7fa17aa00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa184101bf0 0x7fa1841a5100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:46682/0 (socket says 172.21.15.45:46682) 2024-04-16T17:08:01.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:01.895+0000 7fa17aa00640 1 -- 172.21.15.45:0/1032773061 learned_addr learned my addr 172.21.15.45:0/1032773061 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:08:01.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:01.895+0000 7fa183400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa184101250 0x7fa1841a4bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:01.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:01.895+0000 7fa183e00640 1 --2- 172.21.15.45:0/1032773061 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa1841070e0 0x7fa18410cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:01.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:01.896+0000 7fa17aa00640 1 -- 172.21.15.45:0/1032773061 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa1841070e0 msgr2=0x7fa18410cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:01.897 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:01.896+0000 7fa17aa00640 1 --2- 172.21.15.45:0/1032773061 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa1841070e0 0x7fa18410cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:01.897 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:01.896+0000 7fa17aa00640 1 -- 172.21.15.45:0/1032773061 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa184101250 msgr2=0x7fa1841a4bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:01.897 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:01.896+0000 7fa17aa00640 1 --2- 172.21.15.45:0/1032773061 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa184101250 0x7fa1841a4bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:01.897 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:01.896+0000 7fa17aa00640 1 -- 172.21.15.45:0/1032773061 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa18410d680 con 0x7fa184101bf0 2024-04-16T17:08:01.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:01.896+0000 7fa17aa00640 1 --2- 172.21.15.45:0/1032773061 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa184101bf0 0x7fa1841a5100 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fa16c005ab0 tx=0x7fa16c0043b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:08:01.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:01.896+0000 7fa180c00640 1 -- 172.21.15.45:0/1032773061 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa16c043020 con 0x7fa184101bf0 2024-04-16T17:08:01.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:01.897+0000 7fa180c00640 1 -- 172.21.15.45:0/1032773061 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa16c0054e0 con 0x7fa184101bf0 2024-04-16T17:08:01.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:01.897+0000 7fa18a600640 1 -- 172.21.15.45:0/1032773061 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa18410d910 con 0x7fa184101bf0 2024-04-16T17:08:01.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:01.897+0000 7fa180c00640 1 -- 172.21.15.45:0/1032773061 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa16c003c50 con 0x7fa184101bf0 2024-04-16T17:08:01.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:01.897+0000 7fa18a600640 1 -- 172.21.15.45:0/1032773061 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1841aba10 con 0x7fa184101bf0 2024-04-16T17:08:01.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:01.900+0000 7fa18d1e0740 1 Processor -- start 2024-04-16T17:08:01.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:01.900+0000 7fa18d1e0740 1 -- start start 2024-04-16T17:08:01.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:01.900+0000 7fa18d1e0740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55af6ade11b0 0x55af6ade1590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:01.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:01.900+0000 7fa18d1e0740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55af6ade1ad0 0x55af6adea170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:01.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:01.900+0000 7fa18d1e0740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55af6adea6b0 0x55af6adecaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:01.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:01.900+0000 7fa18d1e0740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55af6ad47120 con 0x55af6adea6b0 2024-04-16T17:08:01.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:01.900+0000 7fa18d1e0740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55af6aced340 con 0x55af6ade1ad0 2024-04-16T17:08:01.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:01.900+0000 7fa18d1e0740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55af6ad430e0 con 0x55af6ade11b0 2024-04-16T17:08:01.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:01.900+0000 7fa180c00640 1 -- 172.21.15.45:0/1032773061 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 25) ==== 138879+0+0 (secure 0 0 0) 0x7fa16c046020 con 0x7fa184101bf0 2024-04-16T17:08:01.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:01.900+0000 7fa183400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55af6ade1ad0 0x55af6adea170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:01.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:01.900+0000 7fa183e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55af6ade11b0 0x55af6ade1590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:01.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:01.900+0000 7fa183400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55af6ade1ad0 0x55af6adea170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:47438/0 (socket says 172.21.15.45:47438) 2024-04-16T17:08:01.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:01.901+0000 7fa183400640 1 -- 172.21.15.45:0/146894193 learned_addr learned my addr 172.21.15.45:0/146894193 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:08:01.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:01.901+0000 7fa183400640 1 -- 172.21.15.45:0/146894193 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55af6ade11b0 msgr2=0x55af6ade1590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:08:01.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:01.901+0000 7fa17aa00640 1 --2- 172.21.15.45:0/146894193 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55af6adea6b0 0x55af6adecaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:01.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:01.901+0000 7fa183400640 1 --2- 172.21.15.45:0/146894193 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55af6ade11b0 0x55af6ade1590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:01.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:01.901+0000 7fa183400640 1 -- 172.21.15.45:0/146894193 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55af6adea6b0 msgr2=0x55af6adecaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:08:01.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:01.901+0000 7fa183400640 1 --2- 172.21.15.45:0/146894193 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55af6adea6b0 0x55af6adecaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:01.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:01.901+0000 7fa183400640 1 -- 172.21.15.45:0/146894193 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55af6ab237d0 con 0x55af6ade1ad0 2024-04-16T17:08:01.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:01.901+0000 7fa17aa00640 1 --2- 172.21.15.45:0/146894193 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55af6adea6b0 0x55af6adecaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:08:01.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:01.901+0000 7fa183400640 1 --2- 172.21.15.45:0/146894193 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55af6ade1ad0 0x55af6adea170 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fa17400bc90 tx=0x7fa1740107a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cd6d577f5564ec90 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:08:01.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:01.902+0000 7fa178c00640 1 -- 172.21.15.45:0/146894193 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa17401c480 con 0x55af6ade1ad0 2024-04-16T17:08:01.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:01.902+0000 7fa178c00640 1 -- 172.21.15.45:0/146894193 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa17401cb00 con 0x55af6ade1ad0 2024-04-16T17:08:01.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:01.902+0000 7fa178c00640 1 -- 172.21.15.45:0/146894193 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa174011bb0 con 0x55af6ade1ad0 2024-04-16T17:08:01.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:01.902+0000 7fa180c00640 1 --2- 172.21.15.45:0/1032773061 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fa1680a3ec0 0x7fa1680a6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:01.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:01.902+0000 7fa180c00640 1 -- 172.21.15.45:0/1032773061 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fa16c0f2ce0 con 0x7fa184101bf0 2024-04-16T17:08:01.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:01.902+0000 7fa18d1e0740 1 -- 172.21.15.45:0/146894193 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55af6ade1ad0 msgr2=0x55af6adea170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:01.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:01.902+0000 7fa18d1e0740 1 --2- 172.21.15.45:0/146894193 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55af6ade1ad0 0x55af6adea170 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fa17400bc90 tx=0x7fa1740107a0 comp rx=0 tx=0).stop 2024-04-16T17:08:01.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:01.902+0000 7fa183e00640 1 --2- 172.21.15.45:0/1032773061 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fa1680a3ec0 0x7fa1680a6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:01.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:01.903+0000 7fa18d1e0740 1 -- 172.21.15.45:0/146894193 shutdown_connections 2024-04-16T17:08:01.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:01.903+0000 7fa18d1e0740 1 --2- 172.21.15.45:0/146894193 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55af6adea6b0 0x55af6adecaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:01.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:01.903+0000 7fa18d1e0740 1 --2- 172.21.15.45:0/146894193 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55af6ade1ad0 0x55af6adea170 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:01.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:01.903+0000 7fa18d1e0740 1 --2- 172.21.15.45:0/146894193 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55af6ade11b0 0x55af6ade1590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:01.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:01.903+0000 7fa18d1e0740 1 -- 172.21.15.45:0/146894193 >> 172.21.15.45:0/146894193 conn(0x55af6addb8e0 msgr2=0x55af6addbcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:08:01.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:01.903+0000 7fa18d1e0740 1 -- 172.21.15.45:0/146894193 shutdown_connections 2024-04-16T17:08:01.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:01.903+0000 7fa18d1e0740 1 -- 172.21.15.45:0/146894193 wait complete. 2024-04-16T17:08:01.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:01.903+0000 7fa183e00640 1 --2- 172.21.15.45:0/1032773061 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fa1680a3ec0 0x7fa1680a6380 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fa17c005fd0 tx=0x7fa17c0096a0 comp rx=0 tx=0).ready entity=mgr.25411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:08:01.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:01.903+0000 7fa18d1e0740 1 -- start start 2024-04-16T17:08:01.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:01.904+0000 7fa18d1e0740 1 Processor -- start 2024-04-16T17:08:01.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:01.904+0000 7fa18d1e0740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55af6ade11b0 0x55af6ade76a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:01.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:01.904+0000 7fa18d1e0740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55af6ade1ad0 0x55af6ade7be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:01.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:01.904+0000 7fa18d1e0740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55af6adea6b0 0x55af6ade8120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:01.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:01.904+0000 7fa18d1e0740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55af6aded4a0 con 0x55af6ade11b0 2024-04-16T17:08:01.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:01.904+0000 7fa18d1e0740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55af6ad00710 con 0x55af6ade1ad0 2024-04-16T17:08:01.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:01.904+0000 7fa18d1e0740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55af6aded620 con 0x55af6adea6b0 2024-04-16T17:08:01.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:01.904+0000 7fa18d1e0740 10 client.0 _ll_register_callbacks cb 0x55af6acfbaf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:08:01.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:01.904+0000 7fa183e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55af6adea6b0 0x55af6ade8120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:01.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:01.904+0000 7fa17aa00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55af6ade1ad0 0x55af6ade7be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:01.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:01.904+0000 7fa183e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55af6adea6b0 0x55af6ade8120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:46712/0 (socket says 172.21.15.45:46712) 2024-04-16T17:08:01.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:01.904+0000 7fa183e00640 1 -- 172.21.15.45:0/1591515477 learned_addr learned my addr 172.21.15.45:0/1591515477 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:08:01.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:01.904+0000 7fa183400640 1 --2- 172.21.15.45:0/1591515477 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55af6ade11b0 0x55af6ade76a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:01.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:01.905+0000 7fa17aa00640 1 -- 172.21.15.45:0/1591515477 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55af6adea6b0 msgr2=0x55af6ade8120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:08:01.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:01.905+0000 7fa17aa00640 1 --2- 172.21.15.45:0/1591515477 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55af6adea6b0 0x55af6ade8120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:01.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:01.905+0000 7fa17aa00640 1 -- 172.21.15.45:0/1591515477 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55af6ade11b0 msgr2=0x55af6ade76a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:08:01.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:01.905+0000 7fa17aa00640 1 --2- 172.21.15.45:0/1591515477 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55af6ade11b0 0x55af6ade76a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:01.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:01.905+0000 7fa17aa00640 1 -- 172.21.15.45:0/1591515477 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55af6aa593f0 con 0x55af6ade1ad0 2024-04-16T17:08:01.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:01.905+0000 7fa17aa00640 1 --2- 172.21.15.45:0/1591515477 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55af6ade1ad0 0x55af6ade7be0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fa16c0f0010 tx=0x7fa16c009680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e25ca63fd7de472e server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:08:01.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:01.905+0000 7fa171600640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-04-16T17:08:01.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:01.905+0000 7fa171600640 1 -- 172.21.15.45:0/1591515477 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa16c02cbf0 con 0x55af6ade1ad0 2024-04-16T17:08:01.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:01.905+0000 7fa18d1e0740 1 -- 172.21.15.45:0/1591515477 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa17401cb00 con 0x55af6ade1ad0 2024-04-16T17:08:01.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:01.905+0000 7fa171600640 1 -- 172.21.15.45:0/1591515477 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fa16c0bfde0 con 0x55af6ade1ad0 2024-04-16T17:08:01.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:01.905+0000 7fa171600640 1 -- 172.21.15.45:0/1591515477 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fa16c0a2050 con 0x55af6ade1ad0 2024-04-16T17:08:01.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:01.906+0000 7fa18d1e0740 10 client.25949 fetch_fsmap learned FSMap version 25 2024-04-16T17:08:01.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:01.906+0000 7fa18d1e0740 1 -- 172.21.15.45:0/1591515477 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55af6aa190f0 con 0x55af6ade1ad0 2024-04-16T17:08:01.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:01.906+0000 7fa171600640 1 -- 172.21.15.45:0/1591515477 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa17401cb00 con 0x55af6ade1ad0 2024-04-16T17:08:01.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:01.906+0000 7fa171600640 1 -- 172.21.15.45:0/1591515477 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fa16c0c7090 con 0x55af6ade1ad0 2024-04-16T17:08:01.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:01.906+0000 7fa18d1e0740 10 client.25949 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T17:08:01.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:01.906+0000 7fa18d1e0740 10 client.25949 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:08:01.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:01.906+0000 7fa18d1e0740 20 client.25949 populate_metadata read hostname 'smithi045' 2024-04-16T17:08:01.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:01.906+0000 7fa18d1e0740 1 --2- 172.21.15.45:0/1591515477 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55af6adfa830 0x55af6adfccc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:01.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:01.906+0000 7fa18d1e0740 4 client.25949 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:08:01.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:01.906+0000 7fa18d1e0740 1 -- 172.21.15.45:0/1591515477 --> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55af6ad54b40 con 0x55af6adfa830 2024-04-16T17:08:01.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:01.907+0000 7fa183400640 1 --2- 172.21.15.45:0/1591515477 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55af6adfa830 0x55af6adfccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:01.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:01.907+0000 7fa171600640 10 client.25949 ms_handle_connect on v2:172.21.15.104:6838/3660070592 2024-04-16T17:08:01.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:01.907+0000 7fa183400640 1 --2- 172.21.15.45:0/1591515477 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55af6adfa830 0x55af6adfccc0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5ae489a3b880193d server_cookie=c7135c31f3c99925 in_seq=0 out_seq=0 2024-04-16T17:08:01.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:01.908+0000 7fa171600640 1 -- 172.21.15.45:0/1591515477 <== mds.0 v2:172.21.15.104:6838/3660070592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55af6ad54b40 con 0x55af6adfa830 2024-04-16T17:08:01.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:01.908+0000 7fa171600640 10 client.25949 handle_command_reply: tid=0 2024-04-16T17:08:01.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:01.908+0000 7fa18d1e0740 1 client.25949 shutdown 2024-04-16T17:08:01.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:01.908+0000 7fa18d1e0740 1 -- 172.21.15.45:0/1591515477 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55af6ade1ad0 msgr2=0x55af6ade7be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:01.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:01.908+0000 7fa18d1e0740 1 --2- 172.21.15.45:0/1591515477 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55af6ade1ad0 0x55af6ade7be0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fa16c0f0010 tx=0x7fa16c009680 comp rx=0 tx=0).stop 2024-04-16T17:08:01.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:01.908+0000 7fa18d1e0740 1 -- 172.21.15.45:0/1591515477 shutdown_connections 2024-04-16T17:08:01.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:01.908+0000 7fa18d1e0740 1 --2- 172.21.15.45:0/1591515477 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55af6adfa830 0x55af6adfccc0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:01.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:01.908+0000 7fa18d1e0740 1 --2- 172.21.15.45:0/1591515477 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55af6adea6b0 0x55af6ade8120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:01.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:01.908+0000 7fa18d1e0740 1 --2- 172.21.15.45:0/1591515477 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55af6ade1ad0 0x55af6ade7be0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:01.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:01.908+0000 7fa18d1e0740 1 --2- 172.21.15.45:0/1591515477 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55af6ade11b0 0x55af6ade76a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:01.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:01.908+0000 7fa171600640 0 client.25949 ms_handle_reset on v2:172.21.15.104:6838/3660070592 2024-04-16T17:08:01.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:01.908+0000 7fa18d1e0740 1 -- 172.21.15.45:0/1591515477 >> 172.21.15.45:0/1591515477 conn(0x55af6addb8e0 msgr2=0x55af6addd630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:08:01.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:01.908+0000 7fa18d1e0740 1 -- 172.21.15.45:0/1591515477 shutdown_connections 2024-04-16T17:08:01.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:01.908+0000 7fa18d1e0740 1 -- 172.21.15.45:0/1591515477 wait complete. 2024-04-16T17:08:01.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:01.909+0000 7fa18d1e0740 20 client.25949 trim_cache size 0 max 16384 2024-04-16T17:08:01.929 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:01.929+0000 7fa18d1e0740 1 Processor -- start 2024-04-16T17:08:01.929 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:01.929+0000 7fa18d1e0740 1 -- start start 2024-04-16T17:08:01.929 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:01.929+0000 7fa18d1e0740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55af6ade11b0 0x55af6ade9dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:01.929 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:01.929+0000 7fa18d1e0740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55af6ade1ad0 0x55af6ade65e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:01.930 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:01.929+0000 7fa18d1e0740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55af6adea6b0 0x55af6adf6360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:01.930 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:01.929+0000 7fa18d1e0740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fa1841b8310 con 0x55af6ade1ad0 2024-04-16T17:08:01.930 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:01.929+0000 7fa18d1e0740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fa1841b5be0 con 0x55af6ade11b0 2024-04-16T17:08:01.930 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:01.929+0000 7fa18d1e0740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa1841b3a10 con 0x55af6adea6b0 2024-04-16T17:08:01.930 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:01.929+0000 7fa183400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55af6ade11b0 0x55af6ade9dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:01.930 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:01.929+0000 7fa183400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55af6ade11b0 0x55af6ade9dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:47448/0 (socket says 172.21.15.45:47448) 2024-04-16T17:08:01.930 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:01.929+0000 7fa183400640 1 -- 172.21.15.45:0/3098576584 learned_addr learned my addr 172.21.15.45:0/3098576584 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:08:01.930 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:01.929+0000 7fa183e00640 1 --2- 172.21.15.45:0/3098576584 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55af6adea6b0 0x55af6adf6360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:01.930 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:01.929+0000 7fa17aa00640 1 --2- 172.21.15.45:0/3098576584 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55af6ade1ad0 0x55af6ade65e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:01.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:01.930+0000 7fa183400640 1 -- 172.21.15.45:0/3098576584 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55af6adea6b0 msgr2=0x55af6adf6360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:08:01.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:01.930+0000 7fa183400640 1 --2- 172.21.15.45:0/3098576584 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55af6adea6b0 0x55af6adf6360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:01.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:01.930+0000 7fa183400640 1 -- 172.21.15.45:0/3098576584 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55af6ade1ad0 msgr2=0x55af6ade65e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:08:01.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:01.930+0000 7fa183400640 1 --2- 172.21.15.45:0/3098576584 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55af6ade1ad0 0x55af6ade65e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:01.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:01.930+0000 7fa183400640 1 -- 172.21.15.45:0/3098576584 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55af6aded380 con 0x55af6ade11b0 2024-04-16T17:08:01.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:01.930+0000 7fa183400640 1 --2- 172.21.15.45:0/3098576584 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55af6ade11b0 0x55af6ade9dd0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fa17400ba20 tx=0x7fa174011790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d58dc1d7452eba56 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:08:01.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:01.930+0000 7fa173e00640 1 -- 172.21.15.45:0/3098576584 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa174004020 con 0x55af6ade11b0 2024-04-16T17:08:01.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:01.930+0000 7fa173e00640 1 -- 172.21.15.45:0/3098576584 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa174005520 con 0x55af6ade11b0 2024-04-16T17:08:01.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:01.930+0000 7fa173e00640 1 -- 172.21.15.45:0/3098576584 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa174011ad0 con 0x55af6ade11b0 2024-04-16T17:08:01.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:01.931+0000 7fa18d1e0740 1 -- 172.21.15.45:0/3098576584 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55af6ade11b0 msgr2=0x55af6ade9dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:01.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:01.931+0000 7fa18d1e0740 1 --2- 172.21.15.45:0/3098576584 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55af6ade11b0 0x55af6ade9dd0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fa17400ba20 tx=0x7fa174011790 comp rx=0 tx=0).stop 2024-04-16T17:08:01.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:01.931+0000 7fa18d1e0740 1 -- 172.21.15.45:0/3098576584 shutdown_connections 2024-04-16T17:08:01.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:01.931+0000 7fa18d1e0740 1 --2- 172.21.15.45:0/3098576584 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55af6adea6b0 0x55af6adf6360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:01.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:01.931+0000 7fa18d1e0740 1 --2- 172.21.15.45:0/3098576584 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55af6ade1ad0 0x55af6ade65e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:01.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:01.931+0000 7fa18d1e0740 1 --2- 172.21.15.45:0/3098576584 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55af6ade11b0 0x55af6ade9dd0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:01.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:01.931+0000 7fa18d1e0740 1 -- 172.21.15.45:0/3098576584 >> 172.21.15.45:0/3098576584 conn(0x55af6addb8e0 msgr2=0x55af6adda8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:08:01.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:01.931+0000 7fa18d1e0740 1 -- 172.21.15.45:0/3098576584 shutdown_connections 2024-04-16T17:08:01.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:01.931+0000 7fa18d1e0740 1 -- 172.21.15.45:0/3098576584 wait complete. 2024-04-16T17:08:01.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:01.932+0000 7fa18d1e0740 1 -- start start 2024-04-16T17:08:01.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:01.932+0000 7fa18d1e0740 1 Processor -- start 2024-04-16T17:08:01.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:01.932+0000 7fa18d1e0740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55af6ade11b0 0x55af6adee990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:01.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:01.933+0000 7fa18d1e0740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55af6ade1ad0 0x55af6adeeed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:01.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:01.933+0000 7fa18d1e0740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55af6adea6b0 0x55af6ae5fe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:01.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:01.933+0000 7fa18d1e0740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fa1841b40a0 con 0x55af6adea6b0 2024-04-16T17:08:01.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:01.933+0000 7fa18d1e0740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fa1841b5510 con 0x55af6ade1ad0 2024-04-16T17:08:01.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:01.933+0000 7fa18d1e0740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa1841b4770 con 0x55af6ade11b0 2024-04-16T17:08:01.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:01.933+0000 7fa18d1e0740 10 client.0 _ll_register_callbacks cb 0x7fa16c009880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:08:01.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:01.933+0000 7fa17aa00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55af6ade1ad0 0x55af6adeeed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:01.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:01.933+0000 7fa183400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55af6ade11b0 0x55af6adee990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:01.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:01.933+0000 7fa17aa00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55af6ade1ad0 0x55af6adeeed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:47450/0 (socket says 172.21.15.45:47450) 2024-04-16T17:08:01.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:01.933+0000 7fa17aa00640 1 -- 172.21.15.45:0/3410407616 learned_addr learned my addr 172.21.15.45:0/3410407616 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:08:01.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:01.933+0000 7fa183e00640 1 --2- 172.21.15.45:0/3410407616 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55af6adea6b0 0x55af6ae5fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:01.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:01.933+0000 7fa17aa00640 1 -- 172.21.15.45:0/3410407616 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55af6ade11b0 msgr2=0x55af6adee990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:08:01.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:01.933+0000 7fa17aa00640 1 --2- 172.21.15.45:0/3410407616 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55af6ade11b0 0x55af6adee990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:01.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:01.933+0000 7fa17aa00640 1 -- 172.21.15.45:0/3410407616 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55af6adea6b0 msgr2=0x55af6ae5fe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:08:01.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:01.933+0000 7fa17aa00640 1 --2- 172.21.15.45:0/3410407616 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55af6adea6b0 0x55af6ae5fe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:01.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:01.933+0000 7fa17aa00640 1 -- 172.21.15.45:0/3410407616 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55af6ae60340 con 0x55af6ade1ad0 2024-04-16T17:08:01.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:01.934+0000 7fa17aa00640 1 --2- 172.21.15.45:0/3410407616 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55af6ade1ad0 0x55af6adeeed0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fa16c0bad10 tx=0x7fa16c0c7340 comp rx=0 tx=0).ready entity=mon.2 client_cookie=61270b83902f4a69 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:08:01.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:01.934+0000 7fa171600640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-04-16T17:08:01.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:01.934+0000 7fa171600640 1 -- 172.21.15.45:0/3410407616 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa16c0be3f0 con 0x55af6ade1ad0 2024-04-16T17:08:01.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:01.934+0000 7fa171600640 1 -- 172.21.15.45:0/3410407616 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fa16c0f0230 con 0x55af6ade1ad0 2024-04-16T17:08:01.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:01.934+0000 7fa18d1e0740 1 -- 172.21.15.45:0/3410407616 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa174005520 con 0x55af6ade1ad0 2024-04-16T17:08:01.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:01.934+0000 7fa171600640 1 -- 172.21.15.45:0/3410407616 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fa16c004c80 con 0x55af6ade1ad0 2024-04-16T17:08:01.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:01.934+0000 7fa18d1e0740 10 client.25955 fetch_fsmap learned FSMap version 25 2024-04-16T17:08:01.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:01.934+0000 7fa18d1e0740 1 -- 172.21.15.45:0/3410407616 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55af6ae61650 con 0x55af6ade1ad0 2024-04-16T17:08:01.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:01.934+0000 7fa171600640 1 -- 172.21.15.45:0/3410407616 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa174005520 con 0x55af6ade1ad0 2024-04-16T17:08:01.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:01.935+0000 7fa171600640 1 -- 172.21.15.45:0/3410407616 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fa16c0bdd00 con 0x55af6ade1ad0 2024-04-16T17:08:01.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:01.935+0000 7fa18d1e0740 10 client.25955 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T17:08:01.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:01.935+0000 7fa18d1e0740 10 client.25955 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:08:01.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:01.935+0000 7fa18d1e0740 20 client.25955 populate_metadata read hostname 'smithi045' 2024-04-16T17:08:01.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:01.935+0000 7fa18d1e0740 1 --2- 172.21.15.45:0/3410407616 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55af6ae61c70 0x55af6ae64090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:01.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:01.935+0000 7fa18d1e0740 4 client.25955 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-16T17:08:01.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:01.935+0000 7fa18d1e0740 1 -- 172.21.15.45:0/3410407616 --> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55af6ad4e9b0 con 0x55af6ae61c70 2024-04-16T17:08:01.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:01.935+0000 7fa183400640 1 --2- 172.21.15.45:0/3410407616 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55af6ae61c70 0x55af6ae64090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:01.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:01.936+0000 7fa183400640 1 --2- 172.21.15.45:0/3410407616 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55af6ae61c70 0x55af6ae64090 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e5c2ab4d87f1eab7 server_cookie=e31b4e439ef4e51a in_seq=0 out_seq=0 2024-04-16T17:08:01.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:01.936+0000 7fa171600640 10 client.25955 ms_handle_connect on v2:172.21.15.104:6838/3660070592 2024-04-16T17:08:01.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:01.936+0000 7fa171600640 1 -- 172.21.15.45:0/3410407616 <== mds.0 v2:172.21.15.104:6838/3660070592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55af6ad4e9b0 con 0x55af6ae61c70 2024-04-16T17:08:01.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:01.936+0000 7fa171600640 10 client.25955 handle_command_reply: tid=0 2024-04-16T17:08:01.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:01.936+0000 7fa18d1e0740 1 client.25955 shutdown 2024-04-16T17:08:01.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:01.936+0000 7fa18d1e0740 1 -- 172.21.15.45:0/3410407616 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55af6ade1ad0 msgr2=0x55af6adeeed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:01.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:01.936+0000 7fa18d1e0740 1 --2- 172.21.15.45:0/3410407616 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55af6ade1ad0 0x55af6adeeed0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fa16c0bad10 tx=0x7fa16c0c7340 comp rx=0 tx=0).stop 2024-04-16T17:08:01.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:01.936+0000 7fa18d1e0740 1 -- 172.21.15.45:0/3410407616 shutdown_connections 2024-04-16T17:08:01.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:01.936+0000 7fa18d1e0740 1 --2- 172.21.15.45:0/3410407616 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55af6adea6b0 0x55af6ae5fe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:01.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:01.936+0000 7fa18d1e0740 1 --2- 172.21.15.45:0/3410407616 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55af6ade1ad0 0x55af6adeeed0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:01.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:01.936+0000 7fa18d1e0740 1 --2- 172.21.15.45:0/3410407616 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55af6ae61c70 0x55af6ae64090 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:01.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:01.936+0000 7fa18d1e0740 1 --2- 172.21.15.45:0/3410407616 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55af6ade11b0 0x55af6adee990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:01.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:01.936+0000 7fa171600640 0 client.25955 ms_handle_reset on v2:172.21.15.104:6838/3660070592 2024-04-16T17:08:01.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:01.936+0000 7fa18d1e0740 1 -- 172.21.15.45:0/3410407616 >> 172.21.15.45:0/3410407616 conn(0x55af6addb8e0 msgr2=0x55af6ade7640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:08:01.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:01.936+0000 7fa18d1e0740 1 -- 172.21.15.45:0/3410407616 shutdown_connections 2024-04-16T17:08:01.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:01.936+0000 7fa18d1e0740 1 -- 172.21.15.45:0/3410407616 wait complete. 2024-04-16T17:08:01.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:01.936+0000 7fa18d1e0740 20 client.25955 trim_cache size 0 max 16384 2024-04-16T17:08:01.940 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-16T17:08:01.940 INFO:teuthology.orchestra.run.smithi045.stdout: "status": "no active scrubs running", 2024-04-16T17:08:01.940 INFO:teuthology.orchestra.run.smithi045.stdout: "scrubs": {} 2024-04-16T17:08:01.940 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-16T17:08:01.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:01.937+0000 7fa18a600640 1 -- 172.21.15.45:0/1032773061 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fa1680a3ec0 msgr2=0x7fa1680a6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:01.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:01.937+0000 7fa18a600640 1 --2- 172.21.15.45:0/1032773061 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fa1680a3ec0 0x7fa1680a6380 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fa17c005fd0 tx=0x7fa17c0096a0 comp rx=0 tx=0).stop 2024-04-16T17:08:01.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:01.937+0000 7fa18a600640 1 -- 172.21.15.45:0/1032773061 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa184101bf0 msgr2=0x7fa1841a5100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:01.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:01.937+0000 7fa18a600640 1 --2- 172.21.15.45:0/1032773061 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa184101bf0 0x7fa1841a5100 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fa16c005ab0 tx=0x7fa16c0043b0 comp rx=0 tx=0).stop 2024-04-16T17:08:01.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:01.937+0000 7fa18a600640 1 -- 172.21.15.45:0/1032773061 shutdown_connections 2024-04-16T17:08:01.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:01.937+0000 7fa18a600640 1 --2- 172.21.15.45:0/1032773061 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fa1680a3ec0 0x7fa1680a6380 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:01.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:01.937+0000 7fa18a600640 1 --2- 172.21.15.45:0/1032773061 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa184101bf0 0x7fa1841a5100 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:01.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:01.937+0000 7fa18a600640 1 --2- 172.21.15.45:0/1032773061 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa1841070e0 0x7fa18410cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:01.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:01.937+0000 7fa18a600640 1 --2- 172.21.15.45:0/1032773061 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa184101250 0x7fa1841a4bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:01.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:01.937+0000 7fa18a600640 1 -- 172.21.15.45:0/1032773061 >> 172.21.15.45:0/1032773061 conn(0x7fa184076d00 msgr2=0x7fa1840ffc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:08:01.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:01.937+0000 7fa18a600640 1 -- 172.21.15.45:0/1032773061 shutdown_connections 2024-04-16T17:08:01.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:01.937+0000 7fa18a600640 1 -- 172.21.15.45:0/1032773061 wait complete. 2024-04-16T17:08:01.954 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-16T17:08:01.955 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-16T17:08:02.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.155+0000 7f6647200640 1 -- 172.21.15.45:0/1077562595 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6648107160 msgr2=0x7f664810ed30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:02.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.155+0000 7f6647200640 1 --2- 172.21.15.45:0/1077562595 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6648107160 0x7f664810ed30 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f6640009ca0 tx=0x7f664002c100 comp rx=0 tx=0).stop 2024-04-16T17:08:02.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.156+0000 7f6647200640 1 -- 172.21.15.45:0/1077562595 shutdown_connections 2024-04-16T17:08:02.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.156+0000 7f6647200640 1 --2- 172.21.15.45:0/1077562595 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6648107160 0x7f664810ed30 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:02.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.156+0000 7f6647200640 1 --2- 172.21.15.45:0/1077562595 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6648102a40 0x7f6648106a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:02.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.156+0000 7f6647200640 1 --2- 172.21.15.45:0/1077562595 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6648102090 0x7f6648102470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:02.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.156+0000 7f6647200640 1 -- 172.21.15.45:0/1077562595 >> 172.21.15.45:0/1077562595 conn(0x7f66480fd930 msgr2=0x7f66480ffd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:08:02.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.156+0000 7f6647200640 1 -- 172.21.15.45:0/1077562595 shutdown_connections 2024-04-16T17:08:02.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.156+0000 7f6647200640 1 -- 172.21.15.45:0/1077562595 wait complete. 2024-04-16T17:08:02.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.157+0000 7f6647200640 1 Processor -- start 2024-04-16T17:08:02.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.157+0000 7f6647200640 1 -- start start 2024-04-16T17:08:02.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.157+0000 7f6647200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6648102090 0x7f66481a06a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:02.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.157+0000 7f6647200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6648107160 0x7f66481a0be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:02.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.157+0000 7f6647200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f664819a760 0x7f664819abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:02.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.157+0000 7f6647200640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f6648112220 con 0x7f664819a760 2024-04-16T17:08:02.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.157+0000 7f6647200640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f66481120a0 con 0x7f6648107160 2024-04-16T17:08:02.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.157+0000 7f6647200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f66481123a0 con 0x7f6648102090 2024-04-16T17:08:02.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.158+0000 7f6645e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6648102090 0x7f66481a06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:02.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.158+0000 7f6645e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6648102090 0x7f66481a06a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:46750/0 (socket says 172.21.15.45:46750) 2024-04-16T17:08:02.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.158+0000 7f6645e00640 1 -- 172.21.15.45:0/2580881476 learned_addr learned my addr 172.21.15.45:0/2580881476 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:08:02.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.158+0000 7f6645400640 1 --2- 172.21.15.45:0/2580881476 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6648107160 0x7f66481a0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:02.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.158+0000 7f6646800640 1 --2- 172.21.15.45:0/2580881476 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f664819a760 0x7f664819abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:02.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.158+0000 7f6645400640 1 -- 172.21.15.45:0/2580881476 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6648102090 msgr2=0x7f66481a06a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:02.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.158+0000 7f6645400640 1 --2- 172.21.15.45:0/2580881476 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6648102090 0x7f66481a06a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:02.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.158+0000 7f6645400640 1 -- 172.21.15.45:0/2580881476 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f664819a760 msgr2=0x7f664819abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:02.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.158+0000 7f6645400640 1 --2- 172.21.15.45:0/2580881476 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f664819a760 0x7f664819abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:02.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.158+0000 7f6645400640 1 -- 172.21.15.45:0/2580881476 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f664819b360 con 0x7f6648107160 2024-04-16T17:08:02.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.159+0000 7f6645400640 1 --2- 172.21.15.45:0/2580881476 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6648107160 0x7f66481a0be0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f663400e9c0 tx=0x7f663400ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:08:02.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.159+0000 7f663ea00640 1 -- 172.21.15.45:0/2580881476 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f663400ccc0 con 0x7f6648107160 2024-04-16T17:08:02.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.159+0000 7f663ea00640 1 -- 172.21.15.45:0/2580881476 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6634005020 con 0x7f6648107160 2024-04-16T17:08:02.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.159+0000 7f6647200640 1 -- 172.21.15.45:0/2580881476 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66481a7440 con 0x7f6648107160 2024-04-16T17:08:02.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.159+0000 7f663ea00640 1 -- 172.21.15.45:0/2580881476 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6634004a00 con 0x7f6648107160 2024-04-16T17:08:02.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.159+0000 7f6647200640 1 -- 172.21.15.45:0/2580881476 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66481a7840 con 0x7f6648107160 2024-04-16T17:08:02.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.160+0000 7f6647200640 1 -- 172.21.15.45:0/2580881476 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6648103560 con 0x7f6648107160 2024-04-16T17:08:02.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.165+0000 7f663ea00640 1 -- 172.21.15.45:0/2580881476 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 25) ==== 138879+0+0 (secure 0 0 0) 0x7f66340051d0 con 0x7f6648107160 2024-04-16T17:08:02.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.168+0000 7f663ea00640 1 --2- 172.21.15.45:0/2580881476 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f662c0a3ec0 0x7f662c0a6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:02.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.168+0000 7f663ea00640 1 -- 172.21.15.45:0/2580881476 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f66340d0cb0 con 0x7f6648107160 2024-04-16T17:08:02.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.168+0000 7f6645e00640 1 --2- 172.21.15.45:0/2580881476 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f662c0a3ec0 0x7f662c0a6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:02.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.169+0000 7f6645e00640 1 --2- 172.21.15.45:0/2580881476 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f662c0a3ec0 0x7f662c0a6380 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f6638005fd0 tx=0x7f663800b4a0 comp rx=0 tx=0).ready entity=mgr.25411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:08:02.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.171+0000 7f663ea00640 1 -- 172.21.15.45:0/2580881476 <== mon.2 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f663409cd50 con 0x7f6648107160 2024-04-16T17:08:02.202 INFO:tasks.workunit.client.0.smithi045.stdout: 1 2362 6.97 MB/sec warmup 4 sec latency 50.312 ms 2024-04-16T17:08:02.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.451+0000 7f6647200640 1 -- 172.21.15.45:0/2580881476 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6648102a40 con 0x7f6648107160 2024-04-16T17:08:02.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.452+0000 7f663ea00640 1 -- 172.21.15.45:0/2580881476 <== mon.2 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11887 (secure 0 0 0) 0x7f663409c650 con 0x7f6648107160 2024-04-16T17:08:02.452 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-16T17:08:02.452 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/1748842443","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":1748842443},{"type":"v1","addr":"172.21.15.45:6835","nonce":1748842443}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6839/1308627051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":1308627051},{"type":"v1","addr":"172.21.15.45:6839","nonce":1308627051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/1768242489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1768242489},{"type":"v1","addr":"172.21.15.117:6833","nonce":1768242489}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6833/3558902585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":3558902585},{"type":"v1","addr":"172.21.15.104:6833","nonce":3558902585}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24478,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/1308142903","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":1308142903},{"type":"v1","addr":"172.21.15.45:6837","nonce":1308142903}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24491,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6835/3430679937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":3430679937},{"type":"v1","addr":"172.21.15.104:6835","nonce":3430679937}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-16T17:03:24.663611+0000","modified":"2024-04-16T17:06:44.485317+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24581,"mds_1":24569,"mds_2":24532},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24499":{"gid":24499,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/2224123841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2224123841},{"type":"v1","addr":"172.21.15.117:6835","nonce":2224123841}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6837/2471356815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":2471356815},{"type":"v1","addr":"172.21.15.117:6837","nonce":2471356815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24524":{"gid":24524,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6837/2647756509","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":2647756509},{"type":"v1","addr":"172.21.15.104:6837","nonce":2647756509}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24532":{"gid":24532,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":13,"addr":"172.21.15.117:6839/2378675949","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":2378675949},{"type":"v1","addr":"172.21.15.117:6839","nonce":2378675949}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24569":{"gid":24569,"name":"a","rank":1,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.45:6841/3755310822","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6840","nonce":3755310822},{"type":"v1","addr":"172.21.15.45:6841","nonce":3755310822}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24581":{"gid":24581,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.104:6839/3660070592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6838","nonce":3660070592},{"type":"v1","addr":"172.21.15.104:6839","nonce":3660070592}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24581,"qdb_cluster":[24569,24581,24532]},"id":1}]} 2024-04-16T17:08:02.453 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 25 2024-04-16T17:08:02.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.454+0000 7f6647200640 1 -- 172.21.15.45:0/2580881476 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f662c0a3ec0 msgr2=0x7f662c0a6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:02.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.454+0000 7f6647200640 1 --2- 172.21.15.45:0/2580881476 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f662c0a3ec0 0x7f662c0a6380 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f6638005fd0 tx=0x7f663800b4a0 comp rx=0 tx=0).stop 2024-04-16T17:08:02.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.454+0000 7f6647200640 1 -- 172.21.15.45:0/2580881476 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6648107160 msgr2=0x7f66481a0be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:02.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.454+0000 7f6647200640 1 --2- 172.21.15.45:0/2580881476 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6648107160 0x7f66481a0be0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f663400e9c0 tx=0x7f663400ee80 comp rx=0 tx=0).stop 2024-04-16T17:08:02.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.455+0000 7f6647200640 1 -- 172.21.15.45:0/2580881476 shutdown_connections 2024-04-16T17:08:02.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.455+0000 7f6647200640 1 --2- 172.21.15.45:0/2580881476 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f662c0a3ec0 0x7f662c0a6380 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:02.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.455+0000 7f6647200640 1 --2- 172.21.15.45:0/2580881476 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f664819a760 0x7f664819abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:02.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.455+0000 7f6647200640 1 --2- 172.21.15.45:0/2580881476 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6648107160 0x7f66481a0be0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:02.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.455+0000 7f6647200640 1 --2- 172.21.15.45:0/2580881476 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6648102090 0x7f66481a06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:02.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.455+0000 7f6647200640 1 -- 172.21.15.45:0/2580881476 >> 172.21.15.45:0/2580881476 conn(0x7f66480fd930 msgr2=0x7f664810ea40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:08:02.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.455+0000 7f6647200640 1 -- 172.21.15.45:0/2580881476 shutdown_connections 2024-04-16T17:08:02.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.455+0000 7f6647200640 1 -- 172.21.15.45:0/2580881476 wait complete. 2024-04-16T17:08:02.476 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-16T17:08:02.677 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:08:02 smithi104 ceph-mon[36457]: pgmap v41: 129 pgs: 129 active+clean; 2.2 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 111 KiB/s wr, 34 op/s 2024-04-16T17:08:02.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.680+0000 7fd0c5200640 1 -- 172.21.15.45:0/613251900 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd0c0102110 msgr2=0x7fd0c01024f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:02.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.680+0000 7fd0c5200640 1 --2- 172.21.15.45:0/613251900 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd0c0102110 0x7fd0c01024f0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fd0b0009900 tx=0x7fd0b002bc30 comp rx=0 tx=0).stop 2024-04-16T17:08:02.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.680+0000 7fd0c5200640 1 -- 172.21.15.45:0/613251900 shutdown_connections 2024-04-16T17:08:02.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.680+0000 7fd0c5200640 1 --2- 172.21.15.45:0/613251900 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd0c01071e0 0x7fd0c010edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:02.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.680+0000 7fd0c5200640 1 --2- 172.21.15.45:0/613251900 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd0c0102ac0 0x7fd0c0106b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:02.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.680+0000 7fd0c5200640 1 --2- 172.21.15.45:0/613251900 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd0c0102110 0x7fd0c01024f0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:02.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.680+0000 7fd0c5200640 1 -- 172.21.15.45:0/613251900 >> 172.21.15.45:0/613251900 conn(0x7fd0c00fd9d0 msgr2=0x7fd0c00ffdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:08:02.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.680+0000 7fd0c5200640 1 -- 172.21.15.45:0/613251900 shutdown_connections 2024-04-16T17:08:02.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.680+0000 7fd0c5200640 1 -- 172.21.15.45:0/613251900 wait complete. 2024-04-16T17:08:02.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.681+0000 7fd0c5200640 1 Processor -- start 2024-04-16T17:08:02.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.681+0000 7fd0c5200640 1 -- start start 2024-04-16T17:08:02.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.682+0000 7fd0c5200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd0c0102110 0x7fd0c0074300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:02.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.682+0000 7fd0c5200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd0c01071e0 0x7fd0c0074840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:02.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.682+0000 7fd0c5200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd0c0074d80 0x7fd0c01a6a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:02.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.682+0000 7fd0c5200640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fd0c0112270 con 0x7fd0c0102110 2024-04-16T17:08:02.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.682+0000 7fd0c5200640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fd0c01120f0 con 0x7fd0c0074d80 2024-04-16T17:08:02.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.682+0000 7fd0c5200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fd0c01123f0 con 0x7fd0c01071e0 2024-04-16T17:08:02.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.682+0000 7fd0bf400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd0c0102110 0x7fd0c0074300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:02.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.682+0000 7fd0bfe00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd0c0074d80 0x7fd0c01a6a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:02.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.682+0000 7fd0bfe00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd0c0074d80 0x7fd0c01a6a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:47488/0 (socket says 172.21.15.45:47488) 2024-04-16T17:08:02.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.682+0000 7fd0bfe00640 1 -- 172.21.15.45:0/1589834225 learned_addr learned my addr 172.21.15.45:0/1589834225 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:08:02.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.683+0000 7fd0bea00640 1 --2- 172.21.15.45:0/1589834225 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd0c01071e0 0x7fd0c0074840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:02.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.683+0000 7fd0bfe00640 1 -- 172.21.15.45:0/1589834225 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd0c01071e0 msgr2=0x7fd0c0074840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:02.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.683+0000 7fd0bfe00640 1 --2- 172.21.15.45:0/1589834225 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd0c01071e0 0x7fd0c0074840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:02.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.683+0000 7fd0bfe00640 1 -- 172.21.15.45:0/1589834225 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd0c0102110 msgr2=0x7fd0c0074300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:02.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.683+0000 7fd0bfe00640 1 --2- 172.21.15.45:0/1589834225 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd0c0102110 0x7fd0c0074300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:02.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.683+0000 7fd0bfe00640 1 -- 172.21.15.45:0/1589834225 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0c01a7120 con 0x7fd0c0074d80 2024-04-16T17:08:02.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.683+0000 7fd0bfe00640 1 --2- 172.21.15.45:0/1589834225 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd0c0074d80 0x7fd0c01a6a20 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fd0b800efd0 tx=0x7fd0b800c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:08:02.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.683+0000 7fd0b7e00640 1 -- 172.21.15.45:0/1589834225 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0b8003b70 con 0x7fd0c0074d80 2024-04-16T17:08:02.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.683+0000 7fd0c5200640 1 -- 172.21.15.45:0/1589834225 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0c01a7410 con 0x7fd0c0074d80 2024-04-16T17:08:02.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.684+0000 7fd0b7e00640 1 -- 172.21.15.45:0/1589834225 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd0b8003d10 con 0x7fd0c0074d80 2024-04-16T17:08:02.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.684+0000 7fd0b7e00640 1 -- 172.21.15.45:0/1589834225 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0b80109f0 con 0x7fd0c0074d80 2024-04-16T17:08:02.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.684+0000 7fd0c5200640 1 -- 172.21.15.45:0/1589834225 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0c01a7960 con 0x7fd0c0074d80 2024-04-16T17:08:02.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.687+0000 7fd0c95ba740 1 Processor -- start 2024-04-16T17:08:02.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.689+0000 7fd0b7e00640 1 -- 172.21.15.45:0/1589834225 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 25) ==== 138879+0+0 (secure 0 0 0) 0x7fd0b8004020 con 0x7fd0c0074d80 2024-04-16T17:08:02.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.689+0000 7fd0c95ba740 1 -- start start 2024-04-16T17:08:02.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.689+0000 7fd0c95ba740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55869696ef10 0x55869696f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:02.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.689+0000 7fd0c95ba740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55869696f830 0x558696977ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:02.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.689+0000 7fd0c95ba740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558696978410 0x55869697a800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:02.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.689+0000 7fd0c95ba740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5586968d4020 con 0x558696978410 2024-04-16T17:08:02.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.689+0000 7fd0c95ba740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55869687a460 con 0x55869696ef10 2024-04-16T17:08:02.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.689+0000 7fd0c95ba740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5586968cffe0 con 0x55869696f830 2024-04-16T17:08:02.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.690+0000 7fd0bea00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55869696ef10 0x55869696f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:02.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.690+0000 7fd0bea00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55869696ef10 0x55869696f2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:47504/0 (socket says 172.21.15.45:47504) 2024-04-16T17:08:02.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.690+0000 7fd0bea00640 1 -- 172.21.15.45:0/457154150 learned_addr learned my addr 172.21.15.45:0/457154150 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:08:02.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.690+0000 7fd0bf400640 1 --2- 172.21.15.45:0/457154150 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55869696f830 0x558696977ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:02.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.690+0000 7fd0bea00640 1 --2- 172.21.15.45:0/457154150 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558696978410 0x55869697a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:02.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.690+0000 7fd0bea00640 1 -- 172.21.15.45:0/457154150 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55869696f830 msgr2=0x558696977ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:08:02.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.690+0000 7fd0bea00640 1 --2- 172.21.15.45:0/457154150 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55869696f830 0x558696977ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:02.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.690+0000 7fd0bea00640 1 -- 172.21.15.45:0/457154150 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558696978410 msgr2=0x55869697a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:08:02.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.690+0000 7fd0bea00640 1 --2- 172.21.15.45:0/457154150 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558696978410 0x55869697a800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:02.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.690+0000 7fd0bea00640 1 -- 172.21.15.45:0/457154150 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558696882040 con 0x55869696ef10 2024-04-16T17:08:02.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.690+0000 7fd0b7e00640 1 --2- 172.21.15.45:0/1589834225 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fd0a40a3e70 0x7fd0a40a6330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:02.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.690+0000 7fd0bf400640 1 --2- 172.21.15.45:0/457154150 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55869696f830 0x558696977ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:08:02.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.691+0000 7fd0b7e00640 1 -- 172.21.15.45:0/1589834225 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fd0b80d1260 con 0x7fd0c0074d80 2024-04-16T17:08:02.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.691+0000 7fd0bea00640 1 --2- 172.21.15.45:0/457154150 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55869696ef10 0x55869696f2f0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fd0ac009fa0 tx=0x7fd0ac0096c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=60dd59f045037afa server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:08:02.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.691+0000 7fd0bf400640 1 --2- 172.21.15.45:0/1589834225 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fd0a40a3e70 0x7fd0a40a6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:02.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.691+0000 7fd0b4c00640 1 -- 172.21.15.45:0/457154150 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0ac019030 con 0x55869696ef10 2024-04-16T17:08:02.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.691+0000 7fd0b4c00640 1 -- 172.21.15.45:0/457154150 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd0ac011930 con 0x55869696ef10 2024-04-16T17:08:02.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.691+0000 7fd0b4c00640 1 -- 172.21.15.45:0/457154150 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0ac00faa0 con 0x55869696ef10 2024-04-16T17:08:02.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.692+0000 7fd0c95ba740 1 -- 172.21.15.45:0/457154150 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55869696ef10 msgr2=0x55869696f2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:02.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.692+0000 7fd0c95ba740 1 --2- 172.21.15.45:0/457154150 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55869696ef10 0x55869696f2f0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fd0ac009fa0 tx=0x7fd0ac0096c0 comp rx=0 tx=0).stop 2024-04-16T17:08:02.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.692+0000 7fd0c95ba740 1 -- 172.21.15.45:0/457154150 shutdown_connections 2024-04-16T17:08:02.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.692+0000 7fd0c95ba740 1 --2- 172.21.15.45:0/457154150 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55869696f830 0x558696977ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:02.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.692+0000 7fd0c95ba740 1 --2- 172.21.15.45:0/457154150 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558696978410 0x55869697a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:02.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.692+0000 7fd0c95ba740 1 --2- 172.21.15.45:0/457154150 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55869696ef10 0x55869696f2f0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:02.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.692+0000 7fd0c95ba740 1 -- 172.21.15.45:0/457154150 >> 172.21.15.45:0/457154150 conn(0x558696969640 msgr2=0x558696969a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:08:02.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.692+0000 7fd0bf400640 1 --2- 172.21.15.45:0/1589834225 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fd0a40a3e70 0x7fd0a40a6330 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fd0b0006010 tx=0x7fd0b0005dd0 comp rx=0 tx=0).ready entity=mgr.25411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:08:02.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.692+0000 7fd0c95ba740 1 -- 172.21.15.45:0/457154150 shutdown_connections 2024-04-16T17:08:02.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.692+0000 7fd0c95ba740 1 -- 172.21.15.45:0/457154150 wait complete. 2024-04-16T17:08:02.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.693+0000 7fd0c95ba740 1 -- start start 2024-04-16T17:08:02.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.693+0000 7fd0c95ba740 1 Processor -- start 2024-04-16T17:08:02.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.694+0000 7fd0c95ba740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55869696ef10 0x5586969754a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:02.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.694+0000 7fd0c95ba740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55869696f830 0x5586969759e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:02.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.694+0000 7fd0c95ba740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558696978410 0x558696975f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:02.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.694+0000 7fd0c95ba740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55869697b200 con 0x55869696f830 2024-04-16T17:08:02.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.694+0000 7fd0c95ba740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55869688d830 con 0x55869696ef10 2024-04-16T17:08:02.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.694+0000 7fd0c95ba740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55869697b380 con 0x558696978410 2024-04-16T17:08:02.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.694+0000 7fd0c95ba740 10 client.0 _ll_register_callbacks cb 0x558696895150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:08:02.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.694+0000 7fd0bf400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55869696ef10 0x5586969754a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:02.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.694+0000 7fd0bf400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55869696ef10 0x5586969754a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:47508/0 (socket says 172.21.15.45:47508) 2024-04-16T17:08:02.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.694+0000 7fd0bf400640 1 -- 172.21.15.45:0/54803656 learned_addr learned my addr 172.21.15.45:0/54803656 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:08:02.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.694+0000 7fd0bea00640 1 --2- 172.21.15.45:0/54803656 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55869696f830 0x5586969759e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:02.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.694+0000 7fd0bfe00640 1 --2- 172.21.15.45:0/54803656 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558696978410 0x558696975f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:02.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.694+0000 7fd0bf400640 1 -- 172.21.15.45:0/54803656 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558696978410 msgr2=0x558696975f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:08:02.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.694+0000 7fd0bf400640 1 --2- 172.21.15.45:0/54803656 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558696978410 0x558696975f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:02.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.694+0000 7fd0bf400640 1 -- 172.21.15.45:0/54803656 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55869696f830 msgr2=0x5586969759e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:08:02.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.694+0000 7fd0bf400640 1 --2- 172.21.15.45:0/54803656 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55869696f830 0x5586969759e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:02.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.694+0000 7fd0bf400640 1 -- 172.21.15.45:0/54803656 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558696891920 con 0x55869696ef10 2024-04-16T17:08:02.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.694+0000 7fd0bea00640 1 --2- 172.21.15.45:0/54803656 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55869696f830 0x5586969759e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:08:02.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.695+0000 7fd0bf400640 1 --2- 172.21.15.45:0/54803656 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55869696ef10 0x5586969754a0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fd0b0009900 tx=0x7fd0b0009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6099561ae3641e7d server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:08:02.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.695+0000 7fd0a9600640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-04-16T17:08:02.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.695+0000 7fd0a9600640 1 -- 172.21.15.45:0/54803656 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0b004f070 con 0x55869696ef10 2024-04-16T17:08:02.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.695+0000 7fd0a9600640 1 -- 172.21.15.45:0/54803656 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fd0b0004ec0 con 0x55869696ef10 2024-04-16T17:08:02.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.695+0000 7fd0c95ba740 1 -- 172.21.15.45:0/54803656 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd0ac011930 con 0x55869696ef10 2024-04-16T17:08:02.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.695+0000 7fd0a9600640 1 -- 172.21.15.45:0/54803656 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fd0b005b050 con 0x55869696ef10 2024-04-16T17:08:02.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.695+0000 7fd0c95ba740 10 client.25973 fetch_fsmap learned FSMap version 25 2024-04-16T17:08:02.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.695+0000 7fd0c95ba740 1 -- 172.21.15.45:0/54803656 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55869685f7f0 con 0x55869696ef10 2024-04-16T17:08:02.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.696+0000 7fd0a9600640 1 -- 172.21.15.45:0/54803656 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd0ac011930 con 0x55869696ef10 2024-04-16T17:08:02.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.696+0000 7fd0a9600640 1 -- 172.21.15.45:0/54803656 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fd0b0003c60 con 0x55869696ef10 2024-04-16T17:08:02.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.696+0000 7fd0c95ba740 10 client.25973 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T17:08:02.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.696+0000 7fd0c95ba740 10 client.25973 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-16T17:08:02.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.696+0000 7fd0c95ba740 20 client.25973 populate_metadata read hostname 'smithi045' 2024-04-16T17:08:02.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.696+0000 7fd0c95ba740 1 --2- 172.21.15.45:0/54803656 >> [v2:172.21.15.117:6838/2378675949,v1:172.21.15.117:6839/2378675949] conn(0x558696988740 0x55869698ac00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:02.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.696+0000 7fd0c95ba740 4 client.25973 mds_command: new command op to 24532 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:08:02.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.696+0000 7fd0c95ba740 1 -- 172.21.15.45:0/54803656 --> [v2:172.21.15.117:6838/2378675949,v1:172.21.15.117:6839/2378675949] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5586968e1a40 con 0x558696988740 2024-04-16T17:08:02.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.696+0000 7fd0bea00640 1 --2- 172.21.15.45:0/54803656 >> [v2:172.21.15.117:6838/2378675949,v1:172.21.15.117:6839/2378675949] conn(0x558696988740 0x55869698ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:02.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.697+0000 7fd0bea00640 1 --2- 172.21.15.45:0/54803656 >> [v2:172.21.15.117:6838/2378675949,v1:172.21.15.117:6839/2378675949] conn(0x558696988740 0x55869698ac00 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b7f37c8c4520affe server_cookie=dcb4f7ece7316463 in_seq=0 out_seq=0 2024-04-16T17:08:02.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.698+0000 7fd0a9600640 10 client.25973 ms_handle_connect on v2:172.21.15.117:6838/2378675949 2024-04-16T17:08:02.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.698+0000 7fd0a9600640 1 -- 172.21.15.45:0/54803656 <== mds.2 v2:172.21.15.117:6838/2378675949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5586968e1a40 con 0x558696988740 2024-04-16T17:08:02.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.698+0000 7fd0a9600640 10 client.25973 handle_command_reply: tid=0 2024-04-16T17:08:02.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.698+0000 7fd0c95ba740 1 client.25973 shutdown 2024-04-16T17:08:02.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.698+0000 7fd0c95ba740 1 -- 172.21.15.45:0/54803656 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55869696ef10 msgr2=0x5586969754a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:02.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.698+0000 7fd0c95ba740 1 --2- 172.21.15.45:0/54803656 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55869696ef10 0x5586969754a0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fd0b0009900 tx=0x7fd0b0009260 comp rx=0 tx=0).stop 2024-04-16T17:08:02.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.698+0000 7fd0c95ba740 1 -- 172.21.15.45:0/54803656 shutdown_connections 2024-04-16T17:08:02.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.698+0000 7fd0c95ba740 1 --2- 172.21.15.45:0/54803656 >> [v2:172.21.15.117:6838/2378675949,v1:172.21.15.117:6839/2378675949] conn(0x558696988740 0x55869698ac00 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:02.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.698+0000 7fd0c95ba740 1 --2- 172.21.15.45:0/54803656 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558696978410 0x558696975f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:02.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.698+0000 7fd0c95ba740 1 --2- 172.21.15.45:0/54803656 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55869696f830 0x5586969759e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:02.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.698+0000 7fd0c95ba740 1 --2- 172.21.15.45:0/54803656 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55869696ef10 0x5586969754a0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:02.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.698+0000 7fd0a9600640 0 client.25973 ms_handle_reset on v2:172.21.15.117:6838/2378675949 2024-04-16T17:08:02.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.698+0000 7fd0c95ba740 1 -- 172.21.15.45:0/54803656 >> 172.21.15.45:0/54803656 conn(0x558696969640 msgr2=0x55869696b390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:08:02.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.698+0000 7fd0c95ba740 1 -- 172.21.15.45:0/54803656 shutdown_connections 2024-04-16T17:08:02.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.698+0000 7fd0c95ba740 1 -- 172.21.15.45:0/54803656 wait complete. 2024-04-16T17:08:02.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.699+0000 7fd0c95ba740 20 client.25973 trim_cache size 0 max 16384 2024-04-16T17:08:02.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.713+0000 7fd0c95ba740 1 Processor -- start 2024-04-16T17:08:02.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.713+0000 7fd0c95ba740 1 -- start start 2024-04-16T17:08:02.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.713+0000 7fd0c95ba740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55869696ef10 0x558696977bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:02.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.713+0000 7fd0c95ba740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55869696f830 0x5586969840c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:02.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.714+0000 7fd0c95ba740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558696978410 0x5586969904d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:02.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.714+0000 7fd0c95ba740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fd0c01b3c60 con 0x558696978410 2024-04-16T17:08:02.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.714+0000 7fd0c95ba740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fd0c01b1530 con 0x55869696ef10 2024-04-16T17:08:02.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.714+0000 7fd0c95ba740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fd0c01af360 con 0x55869696f830 2024-04-16T17:08:02.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.714+0000 7fd0bf400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55869696ef10 0x558696977bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:02.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.714+0000 7fd0bea00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55869696f830 0x5586969840c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:02.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.714+0000 7fd0bf400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55869696ef10 0x558696977bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:47512/0 (socket says 172.21.15.45:47512) 2024-04-16T17:08:02.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.714+0000 7fd0bf400640 1 -- 172.21.15.45:0/3469719044 learned_addr learned my addr 172.21.15.45:0/3469719044 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:08:02.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.714+0000 7fd0bfe00640 1 --2- 172.21.15.45:0/3469719044 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558696978410 0x5586969904d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:02.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.714+0000 7fd0bf400640 1 -- 172.21.15.45:0/3469719044 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55869696f830 msgr2=0x5586969840c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:08:02.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.714+0000 7fd0bf400640 1 --2- 172.21.15.45:0/3469719044 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55869696f830 0x5586969840c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:02.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.714+0000 7fd0bf400640 1 -- 172.21.15.45:0/3469719044 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558696978410 msgr2=0x5586969904d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:08:02.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.714+0000 7fd0bf400640 1 --2- 172.21.15.45:0/3469719044 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558696978410 0x5586969904d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:02.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.714+0000 7fd0bf400640 1 -- 172.21.15.45:0/3469719044 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558696974470 con 0x55869696ef10 2024-04-16T17:08:02.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.714+0000 7fd0bf400640 1 --2- 172.21.15.45:0/3469719044 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55869696ef10 0x558696977bd0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fd0b0059040 tx=0x7fd0b002bc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=50f4afc81caf8173 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:08:02.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.715+0000 7fd0abe00640 1 -- 172.21.15.45:0/3469719044 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0b0063070 con 0x55869696ef10 2024-04-16T17:08:02.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.715+0000 7fd0abe00640 1 -- 172.21.15.45:0/3469719044 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd0b004b860 con 0x55869696ef10 2024-04-16T17:08:02.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.715+0000 7fd0abe00640 1 -- 172.21.15.45:0/3469719044 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0b004f040 con 0x55869696ef10 2024-04-16T17:08:02.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.715+0000 7fd0c95ba740 1 -- 172.21.15.45:0/3469719044 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55869696ef10 msgr2=0x558696977bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:02.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.715+0000 7fd0c95ba740 1 --2- 172.21.15.45:0/3469719044 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55869696ef10 0x558696977bd0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fd0b0059040 tx=0x7fd0b002bc30 comp rx=0 tx=0).stop 2024-04-16T17:08:02.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.716+0000 7fd0c95ba740 1 -- 172.21.15.45:0/3469719044 shutdown_connections 2024-04-16T17:08:02.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.716+0000 7fd0c95ba740 1 --2- 172.21.15.45:0/3469719044 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55869696f830 0x5586969840c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:02.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.716+0000 7fd0c95ba740 1 --2- 172.21.15.45:0/3469719044 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558696978410 0x5586969904d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:02.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.716+0000 7fd0c95ba740 1 --2- 172.21.15.45:0/3469719044 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55869696ef10 0x558696977bd0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:02.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.716+0000 7fd0c95ba740 1 -- 172.21.15.45:0/3469719044 >> 172.21.15.45:0/3469719044 conn(0x558696969640 msgr2=0x558696975cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:08:02.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.716+0000 7fd0c95ba740 1 -- 172.21.15.45:0/3469719044 shutdown_connections 2024-04-16T17:08:02.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.716+0000 7fd0c95ba740 1 -- 172.21.15.45:0/3469719044 wait complete. 2024-04-16T17:08:02.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.716+0000 7fd0c95ba740 1 -- start start 2024-04-16T17:08:02.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.716+0000 7fd0c95ba740 1 Processor -- start 2024-04-16T17:08:02.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.717+0000 7fd0c95ba740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55869696ef10 0x5586969809e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:02.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.717+0000 7fd0c95ba740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55869696f830 0x55869697d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:02.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.717+0000 7fd0c95ba740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558696978410 0x55869697dae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:02.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.717+0000 7fd0c95ba740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fd0c01af9f0 con 0x558696978410 2024-04-16T17:08:02.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.717+0000 7fd0c95ba740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fd0c01b0e60 con 0x55869696ef10 2024-04-16T17:08:02.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.717+0000 7fd0c95ba740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fd0c01b00c0 con 0x55869696f830 2024-04-16T17:08:02.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.717+0000 7fd0c95ba740 10 client.0 _ll_register_callbacks cb 0x7fd0b80093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:08:02.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.717+0000 7fd0bf400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55869696ef10 0x5586969809e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:02.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.717+0000 7fd0bf400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55869696ef10 0x5586969809e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:47516/0 (socket says 172.21.15.45:47516) 2024-04-16T17:08:02.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.717+0000 7fd0bf400640 1 -- 172.21.15.45:0/3461334449 learned_addr learned my addr 172.21.15.45:0/3461334449 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:08:02.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.717+0000 7fd0bfe00640 1 --2- 172.21.15.45:0/3461334449 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558696978410 0x55869697dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:02.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.717+0000 7fd0bea00640 1 --2- 172.21.15.45:0/3461334449 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55869696f830 0x55869697d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:02.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.718+0000 7fd0bf400640 1 -- 172.21.15.45:0/3461334449 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55869696f830 msgr2=0x55869697d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:08:02.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.718+0000 7fd0bf400640 1 --2- 172.21.15.45:0/3461334449 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55869696f830 0x55869697d5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:02.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.718+0000 7fd0bf400640 1 -- 172.21.15.45:0/3461334449 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558696978410 msgr2=0x55869697dae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:08:02.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.718+0000 7fd0bf400640 1 --2- 172.21.15.45:0/3461334449 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558696978410 0x55869697dae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:02.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.718+0000 7fd0bf400640 1 -- 172.21.15.45:0/3461334449 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55869697ab70 con 0x55869696ef10 2024-04-16T17:08:02.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.718+0000 7fd0bea00640 1 --2- 172.21.15.45:0/3461334449 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55869696f830 0x55869697d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:08:02.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.718+0000 7fd0bf400640 1 --2- 172.21.15.45:0/3461334449 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55869696ef10 0x5586969809e0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fd0b0003bc0 tx=0x7fd0b002c7e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3b6b3f688e738df4 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:08:02.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.718+0000 7fd0a9600640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-04-16T17:08:02.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.718+0000 7fd0a9600640 1 -- 172.21.15.45:0/3461334449 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0b004f070 con 0x55869696ef10 2024-04-16T17:08:02.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.718+0000 7fd0a9600640 1 -- 172.21.15.45:0/3461334449 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fd0b004bc10 con 0x55869696ef10 2024-04-16T17:08:02.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.719+0000 7fd0c95ba740 1 -- 172.21.15.45:0/3461334449 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd0b004b860 con 0x55869696ef10 2024-04-16T17:08:02.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.719+0000 7fd0a9600640 1 -- 172.21.15.45:0/3461334449 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fd0b0063040 con 0x55869696ef10 2024-04-16T17:08:02.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.719+0000 7fd0c95ba740 10 client.25979 fetch_fsmap learned FSMap version 25 2024-04-16T17:08:02.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.719+0000 7fd0c95ba740 1 -- 172.21.15.45:0/3461334449 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5586965e63f0 con 0x55869696ef10 2024-04-16T17:08:02.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.719+0000 7fd0a9600640 1 -- 172.21.15.45:0/3461334449 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd0b004b860 con 0x55869696ef10 2024-04-16T17:08:02.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.719+0000 7fd0a9600640 1 -- 172.21.15.45:0/3461334449 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fd0b0054cc0 con 0x55869696ef10 2024-04-16T17:08:02.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.720+0000 7fd0c95ba740 10 client.25979 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T17:08:02.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.720+0000 7fd0c95ba740 10 client.25979 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-16T17:08:02.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.720+0000 7fd0c95ba740 20 client.25979 populate_metadata read hostname 'smithi045' 2024-04-16T17:08:02.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.720+0000 7fd0c95ba740 1 --2- 172.21.15.45:0/3461334449 >> [v2:172.21.15.117:6838/2378675949,v1:172.21.15.117:6839/2378675949] conn(0x5586969f0020 0x5586969f2410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:02.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.720+0000 7fd0c95ba740 4 client.25979 mds_command: new command op to 24532 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T17:08:02.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.720+0000 7fd0c95ba740 1 -- 172.21.15.45:0/3461334449 --> [v2:172.21.15.117:6838/2378675949,v1:172.21.15.117:6839/2378675949] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5586968db8b0 con 0x5586969f0020 2024-04-16T17:08:02.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.720+0000 7fd0bea00640 1 --2- 172.21.15.45:0/3461334449 >> [v2:172.21.15.117:6838/2378675949,v1:172.21.15.117:6839/2378675949] conn(0x5586969f0020 0x5586969f2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:02.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.721+0000 7fd0bea00640 1 --2- 172.21.15.45:0/3461334449 >> [v2:172.21.15.117:6838/2378675949,v1:172.21.15.117:6839/2378675949] conn(0x5586969f0020 0x5586969f2410 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1db6b52c9586c624 server_cookie=10bf32dc2e4b9a6c in_seq=0 out_seq=0 2024-04-16T17:08:02.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.721+0000 7fd0a9600640 10 client.25979 ms_handle_connect on v2:172.21.15.117:6838/2378675949 2024-04-16T17:08:02.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.721+0000 7fd0a9600640 1 -- 172.21.15.45:0/3461334449 <== mds.2 v2:172.21.15.117:6838/2378675949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5586968db8b0 con 0x5586969f0020 2024-04-16T17:08:02.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.721+0000 7fd0a9600640 10 client.25979 handle_command_reply: tid=0 2024-04-16T17:08:02.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.721+0000 7fd0c95ba740 1 client.25979 shutdown 2024-04-16T17:08:02.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.721+0000 7fd0c95ba740 1 -- 172.21.15.45:0/3461334449 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55869696ef10 msgr2=0x5586969809e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:02.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.721+0000 7fd0c95ba740 1 --2- 172.21.15.45:0/3461334449 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55869696ef10 0x5586969809e0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fd0b0003bc0 tx=0x7fd0b002c7e0 comp rx=0 tx=0).stop 2024-04-16T17:08:02.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.721+0000 7fd0c95ba740 1 -- 172.21.15.45:0/3461334449 shutdown_connections 2024-04-16T17:08:02.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.721+0000 7fd0c95ba740 1 --2- 172.21.15.45:0/3461334449 >> [v2:172.21.15.117:6838/2378675949,v1:172.21.15.117:6839/2378675949] conn(0x5586969f0020 0x5586969f2410 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:02.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.721+0000 7fd0c95ba740 1 --2- 172.21.15.45:0/3461334449 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55869696f830 0x55869697d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:02.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.721+0000 7fd0c95ba740 1 --2- 172.21.15.45:0/3461334449 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558696978410 0x55869697dae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:02.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.721+0000 7fd0c95ba740 1 --2- 172.21.15.45:0/3461334449 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55869696ef10 0x5586969809e0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:02.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.721+0000 7fd0c95ba740 1 -- 172.21.15.45:0/3461334449 >> 172.21.15.45:0/3461334449 conn(0x558696969640 msgr2=0x558696975460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:08:02.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.721+0000 7fd0a9600640 0 client.25979 ms_handle_reset on v2:172.21.15.117:6838/2378675949 2024-04-16T17:08:02.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.722+0000 7fd0c95ba740 1 -- 172.21.15.45:0/3461334449 shutdown_connections 2024-04-16T17:08:02.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.722+0000 7fd0c95ba740 1 -- 172.21.15.45:0/3461334449 wait complete. 2024-04-16T17:08:02.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.722+0000 7fd0c95ba740 20 client.25979 trim_cache size 0 max 16384 2024-04-16T17:08:02.725 INFO:teuthology.orchestra.run.smithi045.stdout:[] 2024-04-16T17:08:02.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.723+0000 7fd0c5200640 1 -- 172.21.15.45:0/1589834225 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fd0a40a3e70 msgr2=0x7fd0a40a6330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:02.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.723+0000 7fd0c5200640 1 --2- 172.21.15.45:0/1589834225 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fd0a40a3e70 0x7fd0a40a6330 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fd0b0006010 tx=0x7fd0b0005dd0 comp rx=0 tx=0).stop 2024-04-16T17:08:02.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.723+0000 7fd0c5200640 1 -- 172.21.15.45:0/1589834225 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd0c0074d80 msgr2=0x7fd0c01a6a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:02.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.723+0000 7fd0c5200640 1 --2- 172.21.15.45:0/1589834225 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd0c0074d80 0x7fd0c01a6a20 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fd0b800efd0 tx=0x7fd0b800c6a0 comp rx=0 tx=0).stop 2024-04-16T17:08:02.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.723+0000 7fd0c5200640 1 -- 172.21.15.45:0/1589834225 shutdown_connections 2024-04-16T17:08:02.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.723+0000 7fd0c5200640 1 --2- 172.21.15.45:0/1589834225 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fd0a40a3e70 0x7fd0a40a6330 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:02.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.723+0000 7fd0c5200640 1 --2- 172.21.15.45:0/1589834225 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd0c01071e0 0x7fd0c0074840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:02.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.723+0000 7fd0c5200640 1 --2- 172.21.15.45:0/1589834225 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd0c0074d80 0x7fd0c01a6a20 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:02.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.723+0000 7fd0c5200640 1 --2- 172.21.15.45:0/1589834225 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd0c0102110 0x7fd0c0074300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:02.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.723+0000 7fd0c5200640 1 -- 172.21.15.45:0/1589834225 >> 172.21.15.45:0/1589834225 conn(0x7fd0c00fd9d0 msgr2=0x7fd0c00ffd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:08:02.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.723+0000 7fd0c5200640 1 -- 172.21.15.45:0/1589834225 shutdown_connections 2024-04-16T17:08:02.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.723+0000 7fd0c5200640 1 -- 172.21.15.45:0/1589834225 wait complete. 2024-04-16T17:08:02.740 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-16T17:08:02.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:08:02 smithi117 ceph-mon[36730]: pgmap v41: 129 pgs: 129 active+clean; 2.2 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 111 KiB/s wr, 34 op/s 2024-04-16T17:08:02.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.945+0000 7f728b200640 1 -- 172.21.15.45:0/2598166871 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f728c101770 msgr2=0x7f728c105c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:02.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.945+0000 7f728b200640 1 --2- 172.21.15.45:0/2598166871 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f728c101770 0x7f728c105c30 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f7278009900 tx=0x7f727802bcc0 comp rx=0 tx=0).stop 2024-04-16T17:08:02.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.945+0000 7f728b200640 1 -- 172.21.15.45:0/2598166871 shutdown_connections 2024-04-16T17:08:02.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.945+0000 7f728b200640 1 --2- 172.21.15.45:0/2598166871 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f728c106250 0x7f728c1066e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:02.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.945+0000 7f728b200640 1 --2- 172.21.15.45:0/2598166871 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f728c101770 0x7f728c105c30 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:02.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.945+0000 7f728b200640 1 --2- 172.21.15.45:0/2598166871 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f728c100e50 0x7f728c101230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:02.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.945+0000 7f728b200640 1 -- 172.21.15.45:0/2598166871 >> 172.21.15.45:0/2598166871 conn(0x7f728c0fc880 msgr2=0x7f728c0feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:08:02.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.945+0000 7f728b200640 1 -- 172.21.15.45:0/2598166871 shutdown_connections 2024-04-16T17:08:02.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.945+0000 7f728b200640 1 -- 172.21.15.45:0/2598166871 wait complete. 2024-04-16T17:08:02.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.946+0000 7f728b200640 1 Processor -- start 2024-04-16T17:08:02.947 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.947+0000 7f728b200640 1 -- start start 2024-04-16T17:08:02.947 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.947+0000 7f728b200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f728c100e50 0x7f728c1a4950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:02.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.947+0000 7f728b200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f728c101770 0x7f728c1a4e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:02.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.947+0000 7f728b200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f728c106250 0x7f728c19eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:02.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.947+0000 7f728b200640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f728c1187e0 con 0x7f728c106250 2024-04-16T17:08:02.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.947+0000 7f728b200640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f728c118660 con 0x7f728c101770 2024-04-16T17:08:02.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.947+0000 7f728b200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f728c118960 con 0x7f728c100e50 2024-04-16T17:08:02.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.947+0000 7f7289e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f728c100e50 0x7f728c1a4950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:02.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.947+0000 7f7289e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f728c100e50 0x7f728c1a4950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:46824/0 (socket says 172.21.15.45:46824) 2024-04-16T17:08:02.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.948+0000 7f7289e00640 1 -- 172.21.15.45:0/883905054 learned_addr learned my addr 172.21.15.45:0/883905054 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:08:02.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.948+0000 7f7289400640 1 --2- 172.21.15.45:0/883905054 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f728c101770 0x7f728c1a4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:02.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.948+0000 7f728a800640 1 --2- 172.21.15.45:0/883905054 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f728c106250 0x7f728c19eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:02.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.948+0000 7f7289e00640 1 -- 172.21.15.45:0/883905054 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f728c101770 msgr2=0x7f728c1a4e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:02.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.948+0000 7f7289e00640 1 --2- 172.21.15.45:0/883905054 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f728c101770 0x7f728c1a4e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:02.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.948+0000 7f7289e00640 1 -- 172.21.15.45:0/883905054 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f728c106250 msgr2=0x7f728c19eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:02.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.948+0000 7f7289e00640 1 --2- 172.21.15.45:0/883905054 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f728c106250 0x7f728c19eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:02.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.948+0000 7f7289e00640 1 -- 172.21.15.45:0/883905054 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f728c19f280 con 0x7f728c100e50 2024-04-16T17:08:02.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.948+0000 7f7289e00640 1 --2- 172.21.15.45:0/883905054 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f728c100e50 0x7f728c1a4950 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f727c00d950 tx=0x7f727c00de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:08:02.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.949+0000 7f7282a00640 1 -- 172.21.15.45:0/883905054 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f727c017030 con 0x7f728c100e50 2024-04-16T17:08:02.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.949+0000 7f7282a00640 1 -- 172.21.15.45:0/883905054 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f727c004020 con 0x7f728c100e50 2024-04-16T17:08:02.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.949+0000 7f7282a00640 1 -- 172.21.15.45:0/883905054 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f727c0047a0 con 0x7f728c100e50 2024-04-16T17:08:02.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.949+0000 7f728b200640 1 -- 172.21.15.45:0/883905054 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f728c19f570 con 0x7f728c100e50 2024-04-16T17:08:02.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.949+0000 7f728b200640 1 -- 172.21.15.45:0/883905054 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f728c1ab810 con 0x7f728c100e50 2024-04-16T17:08:02.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.953+0000 7f7282a00640 1 -- 172.21.15.45:0/883905054 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 25) ==== 138879+0+0 (secure 0 0 0) 0x7f727c009a30 con 0x7f728c100e50 2024-04-16T17:08:02.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.954+0000 7f7293f7b740 1 Processor -- start 2024-04-16T17:08:02.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.954+0000 7f7293f7b740 1 -- start start 2024-04-16T17:08:02.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.954+0000 7f7293f7b740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bdff410f10 0x55bdff4112f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:02.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.954+0000 7f7293f7b740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bdff411830 0x55bdff419ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:02.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.955+0000 7f7282a00640 1 --2- 172.21.15.45:0/883905054 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f72700a3ec0 0x7f72700a6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:02.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.955+0000 7f7293f7b740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bdff41a410 0x55bdff41c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:02.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.955+0000 7f7293f7b740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55bdff376020 con 0x55bdff411830 2024-04-16T17:08:02.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.955+0000 7f7293f7b740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55bdff31c460 con 0x55bdff410f10 2024-04-16T17:08:02.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.955+0000 7f7293f7b740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55bdff371fe0 con 0x55bdff41a410 2024-04-16T17:08:02.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.955+0000 7f7282a00640 1 -- 172.21.15.45:0/883905054 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f727c0d0530 con 0x7f728c100e50 2024-04-16T17:08:02.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.955+0000 7f7289400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bdff41a410 0x55bdff41c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:02.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.955+0000 7f728a800640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bdff410f10 0x55bdff4112f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:02.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.955+0000 7f728a800640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bdff410f10 0x55bdff4112f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:47546/0 (socket says 172.21.15.45:47546) 2024-04-16T17:08:02.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.955+0000 7f728a800640 1 -- 172.21.15.45:0/4074396729 learned_addr learned my addr 172.21.15.45:0/4074396729 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:08:02.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.955+0000 7f7289e00640 1 --2- 172.21.15.45:0/4074396729 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bdff411830 0x55bdff419ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:02.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.956+0000 7f728a800640 1 --2- 172.21.15.45:0/883905054 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f72700a3ec0 0x7f72700a6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:02.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.956+0000 7f7289400640 1 -- 172.21.15.45:0/4074396729 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bdff410f10 msgr2=0x55bdff4112f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:08:02.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.956+0000 7f7289400640 1 --2- 172.21.15.45:0/4074396729 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bdff410f10 0x55bdff4112f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:02.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.956+0000 7f7289400640 1 -- 172.21.15.45:0/4074396729 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bdff411830 msgr2=0x55bdff419ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:08:02.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.956+0000 7f7289400640 1 --2- 172.21.15.45:0/4074396729 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bdff411830 0x55bdff419ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:02.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.956+0000 7f7289400640 1 -- 172.21.15.45:0/4074396729 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bdff324040 con 0x55bdff41a410 2024-04-16T17:08:02.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.956+0000 7f7289400640 1 --2- 172.21.15.45:0/4074396729 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bdff41a410 0x55bdff41c800 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f7278009230 tx=0x7f7278005d50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fa976403e7f4037 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:08:02.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.957+0000 7f728a800640 1 --2- 172.21.15.45:0/883905054 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f72700a3ec0 0x7f72700a6380 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f72840127d0 tx=0x7f7284005bd0 comp rx=0 tx=0).ready entity=mgr.25411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:08:02.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.957+0000 7f7277400640 1 -- 172.21.15.45:0/4074396729 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7278043070 con 0x55bdff41a410 2024-04-16T17:08:02.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.957+0000 7f7277400640 1 -- 172.21.15.45:0/4074396729 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7278004040 con 0x55bdff41a410 2024-04-16T17:08:02.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.957+0000 7f7277400640 1 -- 172.21.15.45:0/4074396729 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7278004bc0 con 0x55bdff41a410 2024-04-16T17:08:02.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.958+0000 7f7293f7b740 1 -- 172.21.15.45:0/4074396729 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bdff41a410 msgr2=0x55bdff41c800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:02.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.958+0000 7f7293f7b740 1 --2- 172.21.15.45:0/4074396729 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bdff41a410 0x55bdff41c800 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f7278009230 tx=0x7f7278005d50 comp rx=0 tx=0).stop 2024-04-16T17:08:02.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.958+0000 7f7293f7b740 1 -- 172.21.15.45:0/4074396729 shutdown_connections 2024-04-16T17:08:02.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.958+0000 7f7293f7b740 1 --2- 172.21.15.45:0/4074396729 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bdff41a410 0x55bdff41c800 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:02.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.958+0000 7f7293f7b740 1 --2- 172.21.15.45:0/4074396729 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bdff411830 0x55bdff419ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:02.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.958+0000 7f7293f7b740 1 --2- 172.21.15.45:0/4074396729 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bdff410f10 0x55bdff4112f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:02.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.958+0000 7f7293f7b740 1 -- 172.21.15.45:0/4074396729 >> 172.21.15.45:0/4074396729 conn(0x55bdff40b640 msgr2=0x55bdff40ba20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:08:02.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.960+0000 7f7293f7b740 1 -- 172.21.15.45:0/4074396729 shutdown_connections 2024-04-16T17:08:02.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.960+0000 7f7293f7b740 1 -- 172.21.15.45:0/4074396729 wait complete. 2024-04-16T17:08:02.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.961+0000 7f7293f7b740 1 -- start start 2024-04-16T17:08:02.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.961+0000 7f7293f7b740 1 Processor -- start 2024-04-16T17:08:02.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.962+0000 7f7293f7b740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bdff410f10 0x55bdff417490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:02.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.962+0000 7f7293f7b740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bdff411830 0x55bdff4179d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:02.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.962+0000 7f7293f7b740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bdff41a410 0x55bdff417f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:02.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.962+0000 7f7293f7b740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55bdff41d1a0 con 0x55bdff411830 2024-04-16T17:08:02.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.962+0000 7f7293f7b740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55bdff32f830 con 0x55bdff41a410 2024-04-16T17:08:02.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.962+0000 7f7293f7b740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55bdff41d320 con 0x55bdff410f10 2024-04-16T17:08:02.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.962+0000 7f7293f7b740 10 client.0 _ll_register_callbacks cb 0x55bdff337150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:08:02.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.962+0000 7f728a800640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bdff41a410 0x55bdff417f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:02.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.962+0000 7f7289e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bdff410f10 0x55bdff417490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:02.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.962+0000 7f7289e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bdff410f10 0x55bdff417490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:46838/0 (socket says 172.21.15.45:46838) 2024-04-16T17:08:02.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.962+0000 7f7289e00640 1 -- 172.21.15.45:0/1787468001 learned_addr learned my addr 172.21.15.45:0/1787468001 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:08:02.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.963+0000 7f7289400640 1 --2- 172.21.15.45:0/1787468001 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bdff411830 0x55bdff4179d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:02.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.963+0000 7f7289e00640 1 -- 172.21.15.45:0/1787468001 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bdff41a410 msgr2=0x55bdff417f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:08:02.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.963+0000 7f7289e00640 1 --2- 172.21.15.45:0/1787468001 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bdff41a410 0x55bdff417f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:02.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.963+0000 7f7289e00640 1 -- 172.21.15.45:0/1787468001 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bdff411830 msgr2=0x55bdff4179d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:08:02.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.963+0000 7f7289e00640 1 --2- 172.21.15.45:0/1787468001 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bdff411830 0x55bdff4179d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:02.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.963+0000 7f7289e00640 1 -- 172.21.15.45:0/1787468001 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bdff333920 con 0x55bdff410f10 2024-04-16T17:08:02.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.963+0000 7f7289e00640 1 --2- 172.21.15.45:0/1787468001 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bdff410f10 0x55bdff417490 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f727c09bc10 tx=0x7f727c000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4ed32a14d7944086 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:08:02.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.963+0000 7f726be00640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-04-16T17:08:02.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.963+0000 7f726be00640 1 -- 172.21.15.45:0/1787468001 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f727c09c610 con 0x55bdff410f10 2024-04-16T17:08:02.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.963+0000 7f726be00640 1 -- 172.21.15.45:0/1787468001 <== mon.1 v2:172.21.15.117:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f727c0783e0 con 0x55bdff410f10 2024-04-16T17:08:02.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.964+0000 7f7293f7b740 1 -- 172.21.15.45:0/1787468001 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7278004040 con 0x55bdff410f10 2024-04-16T17:08:02.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.964+0000 7f726be00640 1 -- 172.21.15.45:0/1787468001 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f727c09bf10 con 0x55bdff410f10 2024-04-16T17:08:02.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.964+0000 7f7293f7b740 10 client.25987 fetch_fsmap learned FSMap version 25 2024-04-16T17:08:02.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.964+0000 7f7293f7b740 1 -- 172.21.15.45:0/1787468001 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55bdff3017f0 con 0x55bdff410f10 2024-04-16T17:08:02.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.964+0000 7f726be00640 1 -- 172.21.15.45:0/1787468001 <== mon.1 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7278004040 con 0x55bdff410f10 2024-04-16T17:08:02.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.964+0000 7f726be00640 1 -- 172.21.15.45:0/1787468001 <== mon.1 v2:172.21.15.117:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f727c0d0180 con 0x55bdff410f10 2024-04-16T17:08:02.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.965+0000 7f7293f7b740 10 client.25987 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T17:08:02.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.965+0000 7f7293f7b740 10 client.25987 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-04-16T17:08:02.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.965+0000 7f7293f7b740 20 client.25987 populate_metadata read hostname 'smithi045' 2024-04-16T17:08:02.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.965+0000 7f7293f7b740 1 --2- 172.21.15.45:0/1787468001 >> [v2:172.21.15.45:6840/3755310822,v1:172.21.15.45:6841/3755310822] conn(0x55bdff42a6d0 0x55bdff42cb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:02.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.965+0000 7f7293f7b740 4 client.25987 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:08:02.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.965+0000 7f7293f7b740 1 -- 172.21.15.45:0/1787468001 --> [v2:172.21.15.45:6840/3755310822,v1:172.21.15.45:6841/3755310822] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bdff383a40 con 0x55bdff42a6d0 2024-04-16T17:08:02.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.965+0000 7f7289400640 1 --2- 172.21.15.45:0/1787468001 >> [v2:172.21.15.45:6840/3755310822,v1:172.21.15.45:6841/3755310822] conn(0x55bdff42a6d0 0x55bdff42cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:02.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.966+0000 7f7289400640 1 --2- 172.21.15.45:0/1787468001 >> [v2:172.21.15.45:6840/3755310822,v1:172.21.15.45:6841/3755310822] conn(0x55bdff42a6d0 0x55bdff42cb20 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2609a4c8ee1d276b server_cookie=916787e9827c26b0 in_seq=0 out_seq=0 2024-04-16T17:08:02.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.966+0000 7f726be00640 10 client.25987 ms_handle_connect on v2:172.21.15.45:6840/3755310822 2024-04-16T17:08:02.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.967+0000 7f726be00640 1 -- 172.21.15.45:0/1787468001 <== mds.1 v2:172.21.15.45:6840/3755310822 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55bdff383a40 con 0x55bdff42a6d0 2024-04-16T17:08:02.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.967+0000 7f726be00640 10 client.25987 handle_command_reply: tid=0 2024-04-16T17:08:02.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.967+0000 7f7293f7b740 1 client.25987 shutdown 2024-04-16T17:08:02.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.967+0000 7f7293f7b740 1 -- 172.21.15.45:0/1787468001 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bdff410f10 msgr2=0x55bdff417490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:02.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.967+0000 7f7293f7b740 1 --2- 172.21.15.45:0/1787468001 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bdff410f10 0x55bdff417490 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f727c09bc10 tx=0x7f727c000bc0 comp rx=0 tx=0).stop 2024-04-16T17:08:02.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.967+0000 7f7293f7b740 1 -- 172.21.15.45:0/1787468001 shutdown_connections 2024-04-16T17:08:02.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.967+0000 7f7293f7b740 1 --2- 172.21.15.45:0/1787468001 >> [v2:172.21.15.45:6840/3755310822,v1:172.21.15.45:6841/3755310822] conn(0x55bdff42a6d0 0x55bdff42cb20 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:02.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.967+0000 7f7293f7b740 1 --2- 172.21.15.45:0/1787468001 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bdff41a410 0x55bdff417f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:02.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.967+0000 7f7293f7b740 1 --2- 172.21.15.45:0/1787468001 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bdff411830 0x55bdff4179d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:02.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.967+0000 7f7293f7b740 1 --2- 172.21.15.45:0/1787468001 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bdff410f10 0x55bdff417490 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:02.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.967+0000 7f726be00640 0 client.25987 ms_handle_reset on v2:172.21.15.45:6840/3755310822 2024-04-16T17:08:02.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.967+0000 7f7293f7b740 1 -- 172.21.15.45:0/1787468001 >> 172.21.15.45:0/1787468001 conn(0x55bdff40b640 msgr2=0x55bdff40d390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:08:02.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.967+0000 7f7293f7b740 1 -- 172.21.15.45:0/1787468001 shutdown_connections 2024-04-16T17:08:02.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.968+0000 7f7293f7b740 1 -- 172.21.15.45:0/1787468001 wait complete. 2024-04-16T17:08:02.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.968+0000 7f7293f7b740 20 client.25987 trim_cache size 0 max 16384 2024-04-16T17:08:02.979 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:02.979+0000 7ff22b200640 1 -- 172.21.15.45:0/1093915358 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff224107320 msgr2=0x7ff22410eef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:02.979 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:02.979+0000 7ff22b200640 1 --2- 172.21.15.45:0/1093915358 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff224107320 0x7ff22410eef0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7ff21c009be0 tx=0x7ff21c02bef0 comp rx=0 tx=0).stop 2024-04-16T17:08:02.979 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:02.979+0000 7ff22b200640 1 -- 172.21.15.45:0/1093915358 shutdown_connections 2024-04-16T17:08:02.980 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:02.979+0000 7ff22b200640 1 --2- 172.21.15.45:0/1093915358 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff224107320 0x7ff22410eef0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:02.980 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:02.979+0000 7ff22b200640 1 --2- 172.21.15.45:0/1093915358 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff224102c00 0x7ff224106c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:02.980 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:02.979+0000 7ff22b200640 1 --2- 172.21.15.45:0/1093915358 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff224102250 0x7ff224102630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:02.980 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:02.979+0000 7ff22b200640 1 -- 172.21.15.45:0/1093915358 >> 172.21.15.45:0/1093915358 conn(0x7ff224079490 msgr2=0x7ff2240798a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:08:02.980 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:02.979+0000 7ff22b200640 1 -- 172.21.15.45:0/1093915358 shutdown_connections 2024-04-16T17:08:02.980 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:02.979+0000 7ff22b200640 1 -- 172.21.15.45:0/1093915358 wait complete. 2024-04-16T17:08:02.981 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:02.981+0000 7ff22b200640 1 Processor -- start 2024-04-16T17:08:02.981 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:02.981+0000 7ff22b200640 1 -- start start 2024-04-16T17:08:02.982 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:02.981+0000 7ff22b200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff224102250 0x7ff22419e450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:02.982 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:02.981+0000 7ff22b200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff224107320 0x7ff22419e990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:02.982 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:02.981+0000 7ff22b200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff224198650 0x7ff224198b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:02.982 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:02.981+0000 7ff22b200640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7ff224112410 con 0x7ff224198650 2024-04-16T17:08:02.982 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:02.981+0000 7ff22b200640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7ff224112290 con 0x7ff224107320 2024-04-16T17:08:02.983 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:02.981+0000 7ff22b200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7ff224112590 con 0x7ff224102250 2024-04-16T17:08:02.983 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:02.982+0000 7ff223e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff224102250 0x7ff22419e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:02.983 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:02.982+0000 7ff223e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff224102250 0x7ff22419e450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:46860/0 (socket says 172.21.15.45:46860) 2024-04-16T17:08:02.983 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:02.982+0000 7ff223e00640 1 -- 172.21.15.45:0/577392108 learned_addr learned my addr 172.21.15.45:0/577392108 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:08:02.983 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:02.982+0000 7ff223400640 1 --2- 172.21.15.45:0/577392108 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff224107320 0x7ff22419e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:02.983 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:02.982+0000 7ff223e00640 1 -- 172.21.15.45:0/577392108 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff224107320 msgr2=0x7ff22419e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:02.983 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:02.982+0000 7ff223e00640 1 --2- 172.21.15.45:0/577392108 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff224107320 0x7ff22419e990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:02.983 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:02.982+0000 7ff223e00640 1 -- 172.21.15.45:0/577392108 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff224198650 msgr2=0x7ff224198b00 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-16T17:08:02.984 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:02.983+0000 7ff229200640 1 --2- 172.21.15.45:0/577392108 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff224198650 0x7ff224198b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:02.984 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:02.983+0000 7ff223e00640 1 --2- 172.21.15.45:0/577392108 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff224198650 0x7ff224198b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:02.984 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:02.983+0000 7ff223e00640 1 -- 172.21.15.45:0/577392108 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff224199170 con 0x7ff224102250 2024-04-16T17:08:02.984 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:02.983+0000 7ff229200640 1 --2- 172.21.15.45:0/577392108 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff224198650 0x7ff224198b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:08:02.984 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:02.984+0000 7ff223e00640 1 --2- 172.21.15.45:0/577392108 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff224102250 0x7ff22419e450 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7ff21400d950 tx=0x7ff21400de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:08:02.984 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:02.984+0000 7ff220c00640 1 -- 172.21.15.45:0/577392108 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff214017070 con 0x7ff224102250 2024-04-16T17:08:02.984 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:02.984+0000 7ff220c00640 1 -- 172.21.15.45:0/577392108 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff214009c30 con 0x7ff224102250 2024-04-16T17:08:02.984 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:02.984+0000 7ff220c00640 1 -- 172.21.15.45:0/577392108 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff21400f6b0 con 0x7ff224102250 2024-04-16T17:08:02.985 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:02.984+0000 7ff22b200640 1 -- 172.21.15.45:0/577392108 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff224199460 con 0x7ff224102250 2024-04-16T17:08:02.985 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:02.984+0000 7ff22b200640 1 -- 172.21.15.45:0/577392108 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2241a56b0 con 0x7ff224102250 2024-04-16T17:08:02.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.988+0000 7f7293f7b740 1 Processor -- start 2024-04-16T17:08:02.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.988+0000 7f7293f7b740 1 -- start start 2024-04-16T17:08:02.988 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:02.988+0000 7ff220c00640 1 -- 172.21.15.45:0/577392108 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 25) ==== 138879+0+0 (secure 0 0 0) 0x7ff2140109d0 con 0x7ff224102250 2024-04-16T17:08:02.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.989+0000 7f7293f7b740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bdff410f10 0x55bdff419bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:02.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.989+0000 7f7293f7b740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bdff411830 0x55bdff426060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:02.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.989+0000 7f7293f7b740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bdff41a410 0x55bdff40a580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:02.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.989+0000 7f7293f7b740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f728c1b8170 con 0x55bdff41a410 2024-04-16T17:08:02.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.989+0000 7f7293f7b740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f728c1b5a40 con 0x55bdff410f10 2024-04-16T17:08:02.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.989+0000 7f7293f7b740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f728c1b3870 con 0x55bdff411830 2024-04-16T17:08:02.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.989+0000 7f7289e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bdff410f10 0x55bdff419bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:02.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.989+0000 7f7289e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bdff410f10 0x55bdff419bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:47578/0 (socket says 172.21.15.45:47578) 2024-04-16T17:08:02.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.989+0000 7f7289e00640 1 -- 172.21.15.45:0/3909390924 learned_addr learned my addr 172.21.15.45:0/3909390924 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:08:02.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.989+0000 7f728a800640 1 --2- 172.21.15.45:0/3909390924 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bdff41a410 0x55bdff40a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:02.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.989+0000 7f7289e00640 1 -- 172.21.15.45:0/3909390924 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bdff411830 msgr2=0x55bdff426060 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-16T17:08:02.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.989+0000 7f7289e00640 1 --2- 172.21.15.45:0/3909390924 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bdff411830 0x55bdff426060 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:02.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.989+0000 7f7289e00640 1 -- 172.21.15.45:0/3909390924 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bdff41a410 msgr2=0x55bdff40a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:08:02.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.989+0000 7f7289e00640 1 --2- 172.21.15.45:0/3909390924 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bdff41a410 0x55bdff40a580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:02.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.989+0000 7f7289e00640 1 -- 172.21.15.45:0/3909390924 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bdff0883f0 con 0x55bdff410f10 2024-04-16T17:08:02.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.990+0000 7f7289e00640 1 --2- 172.21.15.45:0/3909390924 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bdff410f10 0x55bdff419bc0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f727c0a4650 tx=0x7f727c0981b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=38147e08b26bc66d server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:08:02.991 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:02.990+0000 7ff22df80740 1 Processor -- start 2024-04-16T17:08:02.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.990+0000 7f7276a00640 1 -- 172.21.15.45:0/3909390924 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f727c0b9070 con 0x55bdff410f10 2024-04-16T17:08:02.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.990+0000 7f7276a00640 1 -- 172.21.15.45:0/3909390924 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f727c0cd6d0 con 0x55bdff410f10 2024-04-16T17:08:02.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.990+0000 7f7276a00640 1 -- 172.21.15.45:0/3909390924 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f727c0a46f0 con 0x55bdff410f10 2024-04-16T17:08:02.993 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:02.990+0000 7ff22df80740 1 -- start start 2024-04-16T17:08:02.993 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:02.990+0000 7ff22df80740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55924312ba20 0x55924312a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:02.993 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:02.990+0000 7ff22df80740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55924312acb0 0x55924312b130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:02.993 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:02.991+0000 7ff22df80740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559243134230 0x559243136620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:02.993 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:02.991+0000 7ff22df80740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55924308b780 con 0x55924312ba20 2024-04-16T17:08:02.993 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:02.991+0000 7ff22df80740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x559243031f40 con 0x55924312acb0 2024-04-16T17:08:02.993 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:02.991+0000 7ff22df80740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x559243087740 con 0x559243134230 2024-04-16T17:08:02.993 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:02.991+0000 7ff220c00640 1 --2- 172.21.15.45:0/577392108 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7ff2080a3fe0 0x7ff2080a64a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:02.993 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:02.991+0000 7ff223e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55924312acb0 0x55924312b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:02.993 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:02.991+0000 7ff223400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559243134230 0x559243136620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:02.993 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:02.991+0000 7ff220c00640 1 -- 172.21.15.45:0/577392108 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7ff2140d06e0 con 0x7ff224102250 2024-04-16T17:08:02.994 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:02.991+0000 7ff223400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559243134230 0x559243136620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:46868/0 (socket says 172.21.15.45:46868) 2024-04-16T17:08:02.994 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:02.991+0000 7ff223400640 1 -- 172.21.15.45:0/2576303143 learned_addr learned my addr 172.21.15.45:0/2576303143 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:08:02.994 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:02.991+0000 7ff229200640 1 --2- 172.21.15.45:0/2576303143 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55924312ba20 0x55924312a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:02.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.991+0000 7f7293f7b740 1 -- 172.21.15.45:0/3909390924 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bdff410f10 msgr2=0x55bdff419bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:02.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.991+0000 7f7293f7b740 1 --2- 172.21.15.45:0/3909390924 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bdff410f10 0x55bdff419bc0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f727c0a4650 tx=0x7f727c0981b0 comp rx=0 tx=0).stop 2024-04-16T17:08:02.995 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:02.991+0000 7ff223400640 1 -- 172.21.15.45:0/2576303143 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55924312acb0 msgr2=0x55924312b130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:08:02.995 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:02.991+0000 7ff223400640 1 --2- 172.21.15.45:0/2576303143 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55924312acb0 0x55924312b130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:02.995 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:02.991+0000 7ff223400640 1 -- 172.21.15.45:0/2576303143 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55924312ba20 msgr2=0x55924312a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:08:02.995 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:02.991+0000 7ff223400640 1 --2- 172.21.15.45:0/2576303143 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55924312ba20 0x55924312a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:02.995 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:02.991+0000 7ff229200640 1 --2- 172.21.15.45:0/577392108 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7ff2080a3fe0 0x7ff2080a64a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:02.995 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:02.991+0000 7ff223400640 1 -- 172.21.15.45:0/2576303143 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559242d9e210 con 0x559243134230 2024-04-16T17:08:02.995 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:02.992+0000 7ff223400640 1 --2- 172.21.15.45:0/2576303143 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559243134230 0x559243136620 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7ff210009c30 tx=0x7ff210011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f6b7e5326b74e74e server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:08:02.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.992+0000 7f7293f7b740 1 -- 172.21.15.45:0/3909390924 shutdown_connections 2024-04-16T17:08:02.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.992+0000 7f7293f7b740 1 --2- 172.21.15.45:0/3909390924 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bdff411830 0x55bdff426060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:02.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.992+0000 7f7293f7b740 1 --2- 172.21.15.45:0/3909390924 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bdff41a410 0x55bdff40a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:02.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.992+0000 7f7293f7b740 1 --2- 172.21.15.45:0/3909390924 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bdff410f10 0x55bdff419bc0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:02.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.992+0000 7f7293f7b740 1 -- 172.21.15.45:0/3909390924 >> 172.21.15.45:0/3909390924 conn(0x55bdff40b640 msgr2=0x55bdff409e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:08:02.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.992+0000 7f7293f7b740 1 -- 172.21.15.45:0/3909390924 shutdown_connections 2024-04-16T17:08:02.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.992+0000 7f7293f7b740 1 -- 172.21.15.45:0/3909390924 wait complete. 2024-04-16T17:08:02.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.992+0000 7f7293f7b740 1 -- start start 2024-04-16T17:08:02.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.993+0000 7f7293f7b740 1 Processor -- start 2024-04-16T17:08:02.997 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:02.992+0000 7ff219600640 1 -- 172.21.15.45:0/2576303143 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff21001e070 con 0x559243134230 2024-04-16T17:08:02.997 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:02.992+0000 7ff219600640 1 -- 172.21.15.45:0/2576303143 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff210009de0 con 0x559243134230 2024-04-16T17:08:02.997 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:02.992+0000 7ff219600640 1 -- 172.21.15.45:0/2576303143 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff210018820 con 0x559243134230 2024-04-16T17:08:02.997 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:02.993+0000 7ff22df80740 1 -- 172.21.15.45:0/2576303143 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559243134230 msgr2=0x559243136620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:02.997 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:02.993+0000 7ff229200640 1 --2- 172.21.15.45:0/577392108 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7ff2080a3fe0 0x7ff2080a64a0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7ff21c02c430 tx=0x7ff21c005fd0 comp rx=0 tx=0).ready entity=mgr.25411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:08:02.997 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:02.993+0000 7ff22df80740 1 --2- 172.21.15.45:0/2576303143 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559243134230 0x559243136620 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7ff210009c30 tx=0x7ff210011810 comp rx=0 tx=0).stop 2024-04-16T17:08:02.998 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:02.993+0000 7ff22df80740 1 -- 172.21.15.45:0/2576303143 shutdown_connections 2024-04-16T17:08:02.998 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:02.993+0000 7ff22df80740 1 --2- 172.21.15.45:0/2576303143 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559243134230 0x559243136620 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:02.998 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:02.993+0000 7ff22df80740 1 --2- 172.21.15.45:0/2576303143 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55924312acb0 0x55924312b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:02.998 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:02.993+0000 7ff22df80740 1 --2- 172.21.15.45:0/2576303143 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55924312ba20 0x55924312a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:02.998 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:02.993+0000 7ff22df80740 1 -- 172.21.15.45:0/2576303143 >> 172.21.15.45:0/2576303143 conn(0x5592430c15d0 msgr2=0x5592431259a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:08:02.998 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:02.993+0000 7ff22df80740 1 -- 172.21.15.45:0/2576303143 shutdown_connections 2024-04-16T17:08:02.998 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:02.993+0000 7ff22df80740 1 -- 172.21.15.45:0/2576303143 wait complete. 2024-04-16T17:08:02.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.994+0000 7f7293f7b740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bdff410f10 0x55bdff422980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:02.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.994+0000 7f7293f7b740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bdff411830 0x55bdff413e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:02.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.994+0000 7f7293f7b740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bdff41a410 0x55bdff4143a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:02.999 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:02.994+0000 7ff22df80740 1 -- start start 2024-04-16T17:08:02.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.994+0000 7f7293f7b740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f728c1b3f00 con 0x55bdff411830 2024-04-16T17:08:03.000 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.994+0000 7f7293f7b740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f728c1b5370 con 0x55bdff410f10 2024-04-16T17:08:03.000 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.994+0000 7f7289e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bdff410f10 0x55bdff422980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:03.000 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.994+0000 7f7293f7b740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f728c1b45d0 con 0x55bdff41a410 2024-04-16T17:08:03.000 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:02.995+0000 7ff22df80740 1 Processor -- start 2024-04-16T17:08:03.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.994+0000 7f728a800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bdff41a410 0x55bdff4143a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:03.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.994+0000 7f7289e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bdff410f10 0x55bdff422980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:47602/0 (socket says 172.21.15.45:47602) 2024-04-16T17:08:03.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.994+0000 7f7289e00640 1 -- 172.21.15.45:0/3539256626 learned_addr learned my addr 172.21.15.45:0/3539256626 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:08:03.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.994+0000 7f7293f7b740 10 client.0 _ll_register_callbacks cb 0x7f727c098280 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:08:03.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.995+0000 7f7289400640 1 --2- 172.21.15.45:0/3539256626 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bdff411830 0x55bdff413e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:03.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.995+0000 7f728a800640 1 -- 172.21.15.45:0/3539256626 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bdff410f10 msgr2=0x55bdff422980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:08:03.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.995+0000 7f728a800640 1 --2- 172.21.15.45:0/3539256626 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bdff410f10 0x55bdff422980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:03.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.995+0000 7f728a800640 1 -- 172.21.15.45:0/3539256626 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bdff411830 msgr2=0x55bdff413e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:08:03.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.995+0000 7f728a800640 1 --2- 172.21.15.45:0/3539256626 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bdff411830 0x55bdff413e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:03.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.995+0000 7f728a800640 1 -- 172.21.15.45:0/3539256626 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bdff1527d0 con 0x55bdff41a410 2024-04-16T17:08:03.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.995+0000 7f728a800640 1 --2- 172.21.15.45:0/3539256626 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bdff41a410 0x55bdff4143a0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f7284005fd0 tx=0x7f7284004ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8c77b29cdb025403 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:08:03.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.995+0000 7f726be00640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-04-16T17:08:03.002 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:02.995+0000 7ff22df80740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55924312acb0 0x559243133cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:03.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.995+0000 7f726be00640 1 -- 172.21.15.45:0/3539256626 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7284004b20 con 0x55bdff41a410 2024-04-16T17:08:03.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.995+0000 7f726be00640 1 -- 172.21.15.45:0/3539256626 <== mon.1 v2:172.21.15.117:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f7284004cc0 con 0x55bdff41a410 2024-04-16T17:08:03.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.996+0000 7f7293f7b740 1 -- 172.21.15.45:0/3539256626 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f727c0cd6d0 con 0x55bdff41a410 2024-04-16T17:08:03.003 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:02.995+0000 7ff22df80740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55924312ba20 0x559243131040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:03.003 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:02.995+0000 7ff22df80740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559243134230 0x559243131780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:03.003 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:02.995+0000 7ff22df80740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x559243136f60 con 0x559243134230 2024-04-16T17:08:03.003 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:02.995+0000 7ff22df80740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x559243045390 con 0x55924312acb0 2024-04-16T17:08:03.003 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:02.995+0000 7ff22df80740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5592431370e0 con 0x55924312ba20 2024-04-16T17:08:03.003 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:02.995+0000 7ff22df80740 10 client.0 _ll_register_callbacks cb 0x55924302ef60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:08:03.003 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:02.996+0000 7ff223e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55924312acb0 0x559243133cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:03.003 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:02.996+0000 7ff223e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55924312acb0 0x559243133cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:47608/0 (socket says 172.21.15.45:47608) 2024-04-16T17:08:03.004 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:02.996+0000 7ff223e00640 1 -- 172.21.15.45:0/2505821954 learned_addr learned my addr 172.21.15.45:0/2505821954 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:08:03.004 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:02.996+0000 7ff223400640 1 --2- 172.21.15.45:0/2505821954 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55924312ba20 0x559243131040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:03.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.996+0000 7f726be00640 1 -- 172.21.15.45:0/3539256626 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f72840206e0 con 0x55bdff41a410 2024-04-16T17:08:03.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.996+0000 7f7293f7b740 10 client.25999 fetch_fsmap learned FSMap version 25 2024-04-16T17:08:03.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.996+0000 7f7293f7b740 1 -- 172.21.15.45:0/3539256626 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55bdff490b40 con 0x55bdff41a410 2024-04-16T17:08:03.005 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:02.996+0000 7ff229200640 1 --2- 172.21.15.45:0/2505821954 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559243134230 0x559243131780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:03.005 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:02.996+0000 7ff223e00640 1 -- 172.21.15.45:0/2505821954 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55924312ba20 msgr2=0x559243131040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:08:03.005 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:02.996+0000 7ff223e00640 1 --2- 172.21.15.45:0/2505821954 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55924312ba20 0x559243131040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:03.005 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:02.996+0000 7ff223e00640 1 -- 172.21.15.45:0/2505821954 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559243134230 msgr2=0x559243131780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:08:03.005 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:02.996+0000 7ff223e00640 1 --2- 172.21.15.45:0/2505821954 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559243134230 0x559243131780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:03.005 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:02.996+0000 7ff223e00640 1 -- 172.21.15.45:0/2505821954 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559242e6a620 con 0x55924312acb0 2024-04-16T17:08:03.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.997+0000 7f726be00640 1 -- 172.21.15.45:0/3539256626 <== mon.1 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f727c0cd6d0 con 0x55bdff41a410 2024-04-16T17:08:03.006 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:02.997+0000 7ff223e00640 1 --2- 172.21.15.45:0/2505821954 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55924312acb0 0x559243133cb0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7ff2140984e0 tx=0x7ff21409c840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5aee3ec332fd92e2 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:08:03.006 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:02.997+0000 7ff20e000640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-04-16T17:08:03.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.997+0000 7f726be00640 1 -- 172.21.15.45:0/3539256626 <== mon.1 v2:172.21.15.117:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f728400c050 con 0x55bdff41a410 2024-04-16T17:08:03.006 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:02.997+0000 7ff20e000640 1 -- 172.21.15.45:0/2505821954 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff214020040 con 0x55924312acb0 2024-04-16T17:08:03.007 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:02.997+0000 7ff20e000640 1 -- 172.21.15.45:0/2505821954 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7ff2140a00d0 con 0x55924312acb0 2024-04-16T17:08:03.007 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:02.997+0000 7ff22df80740 1 -- 172.21.15.45:0/2505821954 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff210009de0 con 0x55924312acb0 2024-04-16T17:08:03.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.997+0000 7f7293f7b740 10 client.25999 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T17:08:03.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.997+0000 7f7293f7b740 10 client.25999 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-04-16T17:08:03.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.997+0000 7f7293f7b740 20 client.25999 populate_metadata read hostname 'smithi045' 2024-04-16T17:08:03.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.997+0000 7f7293f7b740 1 --2- 172.21.15.45:0/3539256626 >> [v2:172.21.15.45:6840/3755310822,v1:172.21.15.45:6841/3755310822] conn(0x55bdff491210 0x55bdff493630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:03.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.997+0000 7f7293f7b740 4 client.25999 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T17:08:03.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.997+0000 7f7293f7b740 1 -- 172.21.15.45:0/3539256626 --> [v2:172.21.15.45:6840/3755310822,v1:172.21.15.45:6841/3755310822] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55bdff37d8b0 con 0x55bdff491210 2024-04-16T17:08:03.008 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:02.997+0000 7ff20e000640 1 -- 172.21.15.45:0/2505821954 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7ff2140a0430 con 0x55924312acb0 2024-04-16T17:08:03.008 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:02.997+0000 7ff22df80740 10 client.26009 fetch_fsmap learned FSMap version 25 2024-04-16T17:08:03.008 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:02.997+0000 7ff22df80740 1 -- 172.21.15.45:0/2505821954 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x559243143080 con 0x55924312acb0 2024-04-16T17:08:03.008 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:02.997+0000 7ff20e000640 1 -- 172.21.15.45:0/2505821954 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff210009de0 con 0x55924312acb0 2024-04-16T17:08:03.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.997+0000 7f7289e00640 1 --2- 172.21.15.45:0/3539256626 >> [v2:172.21.15.45:6840/3755310822,v1:172.21.15.45:6841/3755310822] conn(0x55bdff491210 0x55bdff493630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:03.009 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:02.998+0000 7ff20e000640 1 -- 172.21.15.45:0/2505821954 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7ff2140cdc70 con 0x55924312acb0 2024-04-16T17:08:03.009 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:02.998+0000 7ff22df80740 10 client.26009 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T17:08:03.009 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:02.998+0000 7ff22df80740 10 client.26009 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T17:08:03.009 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:02.998+0000 7ff22df80740 20 client.26009 populate_metadata read hostname 'smithi045' 2024-04-16T17:08:03.009 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:02.998+0000 7ff22df80740 1 --2- 172.21.15.45:0/2505821954 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x559243144900 0x559243146cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:03.009 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:02.998+0000 7ff22df80740 4 client.26009 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:08:03.009 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:02.998+0000 7ff22df80740 1 -- 172.21.15.45:0/2505821954 --> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5592430991a0 con 0x559243144900 2024-04-16T17:08:03.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.998+0000 7f7289e00640 1 --2- 172.21.15.45:0/3539256626 >> [v2:172.21.15.45:6840/3755310822,v1:172.21.15.45:6841/3755310822] conn(0x55bdff491210 0x55bdff493630 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9dc785a035102f48 server_cookie=885ec030e7bef897 in_seq=0 out_seq=0 2024-04-16T17:08:03.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.998+0000 7f726be00640 10 client.25999 ms_handle_connect on v2:172.21.15.45:6840/3755310822 2024-04-16T17:08:03.010 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:02.998+0000 7ff223400640 1 --2- 172.21.15.45:0/2505821954 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x559243144900 0x559243146cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:03.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.999+0000 7f726be00640 1 -- 172.21.15.45:0/3539256626 <== mds.1 v2:172.21.15.45:6840/3755310822 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55bdff37d8b0 con 0x55bdff491210 2024-04-16T17:08:03.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.999+0000 7f726be00640 10 client.25999 handle_command_reply: tid=0 2024-04-16T17:08:03.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.999+0000 7f7293f7b740 1 client.25999 shutdown 2024-04-16T17:08:03.010 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:02.999+0000 7ff223400640 1 --2- 172.21.15.45:0/2505821954 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x559243144900 0x559243146cf0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e3234354b3ac82e server_cookie=9c87c64bb9baa213 in_seq=0 out_seq=0 2024-04-16T17:08:03.010 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:02.999+0000 7ff20e000640 10 client.26009 ms_handle_connect on v2:172.21.15.104:6838/3660070592 2024-04-16T17:08:03.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.999+0000 7f7293f7b740 1 -- 172.21.15.45:0/3539256626 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bdff41a410 msgr2=0x55bdff4143a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:03.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.999+0000 7f7293f7b740 1 --2- 172.21.15.45:0/3539256626 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bdff41a410 0x55bdff4143a0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f7284005fd0 tx=0x7f7284004ab0 comp rx=0 tx=0).stop 2024-04-16T17:08:03.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.999+0000 7f7293f7b740 1 -- 172.21.15.45:0/3539256626 shutdown_connections 2024-04-16T17:08:03.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.999+0000 7f7293f7b740 1 --2- 172.21.15.45:0/3539256626 >> [v2:172.21.15.45:6840/3755310822,v1:172.21.15.45:6841/3755310822] conn(0x55bdff491210 0x55bdff493630 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:03.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.999+0000 7f7293f7b740 1 --2- 172.21.15.45:0/3539256626 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bdff41a410 0x55bdff4143a0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:03.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.999+0000 7f726be00640 0 client.25999 ms_handle_reset on v2:172.21.15.45:6840/3755310822 2024-04-16T17:08:03.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.999+0000 7f7293f7b740 1 --2- 172.21.15.45:0/3539256626 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bdff411830 0x55bdff413e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:03.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.999+0000 7f7293f7b740 1 --2- 172.21.15.45:0/3539256626 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bdff410f10 0x55bdff422980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:03.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:02.999+0000 7f7293f7b740 1 -- 172.21.15.45:0/3539256626 >> 172.21.15.45:0/3539256626 conn(0x55bdff40b640 msgr2=0x55bdff40d300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:08:03.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:03.000+0000 7f7293f7b740 1 -- 172.21.15.45:0/3539256626 shutdown_connections 2024-04-16T17:08:03.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:03.000+0000 7f7293f7b740 1 -- 172.21.15.45:0/3539256626 wait complete. 2024-04-16T17:08:03.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:03.000+0000 7f7293f7b740 20 client.25999 trim_cache size 0 max 16384 2024-04-16T17:08:03.012 INFO:teuthology.orchestra.run.smithi045.stdout:[] 2024-04-16T17:08:03.013 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:03.000+0000 7ff20e000640 1 -- 172.21.15.45:0/2505821954 <== mds.0 v2:172.21.15.104:6838/3660070592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5592430991a0 con 0x559243144900 2024-04-16T17:08:03.013 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:03.000+0000 7ff20e000640 10 client.26009 handle_command_reply: tid=0 2024-04-16T17:08:03.013 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:03.000+0000 7ff22df80740 1 client.26009 shutdown 2024-04-16T17:08:03.013 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:03.000+0000 7ff22df80740 1 -- 172.21.15.45:0/2505821954 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55924312acb0 msgr2=0x559243133cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:03.013 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:03.000+0000 7ff22df80740 1 --2- 172.21.15.45:0/2505821954 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55924312acb0 0x559243133cb0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7ff2140984e0 tx=0x7ff21409c840 comp rx=0 tx=0).stop 2024-04-16T17:08:03.013 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:03.001+0000 7ff22df80740 1 -- 172.21.15.45:0/2505821954 shutdown_connections 2024-04-16T17:08:03.013 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:03.001+0000 7ff22df80740 1 --2- 172.21.15.45:0/2505821954 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x559243144900 0x559243146cf0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:03.013 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:03.001+0000 7ff22df80740 1 --2- 172.21.15.45:0/2505821954 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55924312ba20 0x559243131040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:03.014 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:03.001+0000 7ff22df80740 1 --2- 172.21.15.45:0/2505821954 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559243134230 0x559243131780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:03.014 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:03.001+0000 7ff22df80740 1 --2- 172.21.15.45:0/2505821954 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55924312acb0 0x559243133cb0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:03.014 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:03.001+0000 7ff22df80740 1 -- 172.21.15.45:0/2505821954 >> 172.21.15.45:0/2505821954 conn(0x5592430c15d0 msgr2=0x5592431206b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:08:03.014 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:03.001+0000 7ff20e000640 0 client.26009 ms_handle_reset on v2:172.21.15.104:6838/3660070592 2024-04-16T17:08:03.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:03.001+0000 7f728b200640 1 -- 172.21.15.45:0/883905054 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f72700a3ec0 msgr2=0x7f72700a6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:03.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:03.001+0000 7f728b200640 1 --2- 172.21.15.45:0/883905054 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f72700a3ec0 0x7f72700a6380 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f72840127d0 tx=0x7f7284005bd0 comp rx=0 tx=0).stop 2024-04-16T17:08:03.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:03.001+0000 7f728b200640 1 -- 172.21.15.45:0/883905054 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f728c100e50 msgr2=0x7f728c1a4950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:03.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:03.001+0000 7f728b200640 1 --2- 172.21.15.45:0/883905054 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f728c100e50 0x7f728c1a4950 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f727c00d950 tx=0x7f727c00de10 comp rx=0 tx=0).stop 2024-04-16T17:08:03.015 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:03.001+0000 7ff22df80740 1 -- 172.21.15.45:0/2505821954 shutdown_connections 2024-04-16T17:08:03.015 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:03.001+0000 7ff22df80740 1 -- 172.21.15.45:0/2505821954 wait complete. 2024-04-16T17:08:03.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:03.001+0000 7f728b200640 1 -- 172.21.15.45:0/883905054 shutdown_connections 2024-04-16T17:08:03.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:03.001+0000 7f728b200640 1 --2- 172.21.15.45:0/883905054 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f72700a3ec0 0x7f72700a6380 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:03.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:03.001+0000 7f728b200640 1 --2- 172.21.15.45:0/883905054 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f728c106250 0x7f728c19eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:03.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:03.001+0000 7f728b200640 1 --2- 172.21.15.45:0/883905054 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f728c101770 0x7f728c1a4e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:03.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:03.001+0000 7f728b200640 1 --2- 172.21.15.45:0/883905054 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f728c100e50 0x7f728c1a4950 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:03.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:03.001+0000 7f728b200640 1 -- 172.21.15.45:0/883905054 >> 172.21.15.45:0/883905054 conn(0x7f728c0fc880 msgr2=0x7f728c0fd900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:08:03.016 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:03.001+0000 7ff22df80740 20 client.26009 trim_cache size 0 max 16384 2024-04-16T17:08:03.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:03.001+0000 7f728b200640 1 -- 172.21.15.45:0/883905054 shutdown_connections 2024-04-16T17:08:03.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:03.001+0000 7f728b200640 1 -- 172.21.15.45:0/883905054 wait complete. 2024-04-16T17:08:03.018 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-16T17:08:03.020 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:03.018+0000 7ff22df80740 1 Processor -- start 2024-04-16T17:08:03.020 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:03.018+0000 7ff22df80740 1 -- start start 2024-04-16T17:08:03.021 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:03.019+0000 7ff22df80740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55924312acb0 0x559243130fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:03.021 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:03.019+0000 7ff22df80740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55924312ba20 0x559243133770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:03.021 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:03.019+0000 7ff22df80740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559243134230 0x559243133cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:03.021 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:03.019+0000 7ff22df80740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7ff2241af9f0 con 0x55924312ba20 2024-04-16T17:08:03.021 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:03.019+0000 7ff22df80740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7ff2241ad2c0 con 0x559243134230 2024-04-16T17:08:03.021 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:03.019+0000 7ff22df80740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7ff2241ab0f0 con 0x55924312acb0 2024-04-16T17:08:03.021 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:03.019+0000 7ff229200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559243134230 0x559243133cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:03.021 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:03.019+0000 7ff223400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55924312ba20 0x559243133770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:03.021 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:03.019+0000 7ff229200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559243134230 0x559243133cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:47610/0 (socket says 172.21.15.45:47610) 2024-04-16T17:08:03.021 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:03.019+0000 7ff229200640 1 -- 172.21.15.45:0/2024858237 learned_addr learned my addr 172.21.15.45:0/2024858237 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:08:03.021 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:03.019+0000 7ff223e00640 1 --2- 172.21.15.45:0/2024858237 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55924312acb0 0x559243130fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:03.021 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:03.020+0000 7ff229200640 1 -- 172.21.15.45:0/2024858237 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55924312acb0 msgr2=0x559243130fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:08:03.021 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:03.020+0000 7ff229200640 1 --2- 172.21.15.45:0/2024858237 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55924312acb0 0x559243130fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:03.021 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:03.020+0000 7ff229200640 1 -- 172.21.15.45:0/2024858237 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55924312ba20 msgr2=0x559243133770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:08:03.022 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:03.020+0000 7ff229200640 1 --2- 172.21.15.45:0/2024858237 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55924312ba20 0x559243133770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:03.022 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:03.020+0000 7ff229200640 1 -- 172.21.15.45:0/2024858237 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559243125d60 con 0x559243134230 2024-04-16T17:08:03.022 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:03.020+0000 7ff223e00640 1 --2- 172.21.15.45:0/2024858237 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55924312acb0 0x559243130fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:08:03.022 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:03.020+0000 7ff223400640 1 --2- 172.21.15.45:0/2024858237 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55924312ba20 0x559243133770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:08:03.022 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:03.020+0000 7ff229200640 1 --2- 172.21.15.45:0/2024858237 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559243134230 0x559243133cb0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7ff21c009980 tx=0x7ff21c03a000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=25bbd20b04c7022f server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:08:03.022 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:03.020+0000 7ff218c00640 1 -- 172.21.15.45:0/2024858237 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff21c02c890 con 0x559243134230 2024-04-16T17:08:03.022 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:03.020+0000 7ff218c00640 1 -- 172.21.15.45:0/2024858237 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff21c052070 con 0x559243134230 2024-04-16T17:08:03.022 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:03.020+0000 7ff218c00640 1 -- 172.21.15.45:0/2024858237 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff21c034690 con 0x559243134230 2024-04-16T17:08:03.025 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:03.021+0000 7ff22df80740 1 -- 172.21.15.45:0/2024858237 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559243134230 msgr2=0x559243133cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:03.025 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:03.021+0000 7ff22df80740 1 --2- 172.21.15.45:0/2024858237 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559243134230 0x559243133cb0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7ff21c009980 tx=0x7ff21c03a000 comp rx=0 tx=0).stop 2024-04-16T17:08:03.025 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:03.021+0000 7ff22df80740 1 -- 172.21.15.45:0/2024858237 shutdown_connections 2024-04-16T17:08:03.025 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:03.021+0000 7ff22df80740 1 --2- 172.21.15.45:0/2024858237 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559243134230 0x559243133cb0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:03.025 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:03.021+0000 7ff22df80740 1 --2- 172.21.15.45:0/2024858237 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55924312ba20 0x559243133770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:03.025 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:03.021+0000 7ff22df80740 1 --2- 172.21.15.45:0/2024858237 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55924312acb0 0x559243130fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:03.025 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:03.021+0000 7ff22df80740 1 -- 172.21.15.45:0/2024858237 >> 172.21.15.45:0/2024858237 conn(0x5592430c15d0 msgr2=0x5592431308d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:08:03.026 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:03.021+0000 7ff22df80740 1 -- 172.21.15.45:0/2024858237 shutdown_connections 2024-04-16T17:08:03.026 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:03.021+0000 7ff22df80740 1 -- 172.21.15.45:0/2024858237 wait complete. 2024-04-16T17:08:03.026 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:03.022+0000 7ff22df80740 1 -- start start 2024-04-16T17:08:03.026 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:03.022+0000 7ff22df80740 1 Processor -- start 2024-04-16T17:08:03.026 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:03.022+0000 7ff22df80740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55924312acb0 0x55924314a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:03.026 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:03.022+0000 7ff22df80740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55924312ba20 0x55924312d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:03.026 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:03.022+0000 7ff22df80740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559243134230 0x55924312df30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:03.026 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:03.022+0000 7ff22df80740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7ff2241ab780 con 0x55924312ba20 2024-04-16T17:08:03.026 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:03.022+0000 7ff22df80740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7ff2241acbf0 con 0x559243134230 2024-04-16T17:08:03.026 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:03.022+0000 7ff22df80740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7ff2241abe50 con 0x55924312acb0 2024-04-16T17:08:03.026 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:03.022+0000 7ff22df80740 10 client.0 _ll_register_callbacks cb 0x7ff21409b9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:08:03.026 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:03.023+0000 7ff223e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55924312acb0 0x55924314a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:03.026 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:03.023+0000 7ff229200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559243134230 0x55924312df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:03.027 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:03.023+0000 7ff223400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55924312ba20 0x55924312d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:03.027 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:03.023+0000 7ff223e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55924312acb0 0x55924314a0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:46904/0 (socket says 172.21.15.45:46904) 2024-04-16T17:08:03.027 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:03.023+0000 7ff229200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559243134230 0x55924312df30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:47616/0 (socket says 172.21.15.45:47616) 2024-04-16T17:08:03.027 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:03.023+0000 7ff223e00640 1 -- 172.21.15.45:0/597891006 learned_addr learned my addr 172.21.15.45:0/597891006 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:08:03.027 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:03.023+0000 7ff223e00640 1 -- 172.21.15.45:0/597891006 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559243134230 msgr2=0x55924312df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:08:03.027 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:03.023+0000 7ff223e00640 1 --2- 172.21.15.45:0/597891006 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559243134230 0x55924312df30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:03.027 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:03.023+0000 7ff223e00640 1 -- 172.21.15.45:0/597891006 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55924312ba20 msgr2=0x55924312d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:08:03.027 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:03.023+0000 7ff223e00640 1 --2- 172.21.15.45:0/597891006 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55924312ba20 0x55924312d9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:03.027 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:03.023+0000 7ff223e00640 1 -- 172.21.15.45:0/597891006 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55924312e760 con 0x55924312acb0 2024-04-16T17:08:03.027 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:03.023+0000 7ff229200640 1 --2- 172.21.15.45:0/597891006 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559243134230 0x55924312df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:08:03.027 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:03.023+0000 7ff223e00640 1 --2- 172.21.15.45:0/597891006 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55924312acb0 0x55924314a0f0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7ff214010fd0 tx=0x7ff214098320 comp rx=0 tx=0).ready entity=mon.1 client_cookie=761e043880d7e33c server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:08:03.027 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:03.023+0000 7ff20e000640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-04-16T17:08:03.027 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:03.023+0000 7ff20e000640 1 -- 172.21.15.45:0/597891006 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff214020040 con 0x55924312acb0 2024-04-16T17:08:03.027 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:03.024+0000 7ff22df80740 1 -- 172.21.15.45:0/597891006 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff21c052070 con 0x55924312acb0 2024-04-16T17:08:03.027 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:03.024+0000 7ff20e000640 1 -- 172.21.15.45:0/597891006 <== mon.1 v2:172.21.15.117:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7ff2140d02b0 con 0x55924312acb0 2024-04-16T17:08:03.028 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:03.024+0000 7ff20e000640 1 -- 172.21.15.45:0/597891006 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7ff2140a4cf0 con 0x55924312acb0 2024-04-16T17:08:03.028 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:03.024+0000 7ff22df80740 10 client.26008 fetch_fsmap learned FSMap version 25 2024-04-16T17:08:03.028 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:03.024+0000 7ff22df80740 1 -- 172.21.15.45:0/597891006 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5592431aaac0 con 0x55924312acb0 2024-04-16T17:08:03.028 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:03.024+0000 7ff20e000640 1 -- 172.21.15.45:0/597891006 <== mon.1 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff21c052070 con 0x55924312acb0 2024-04-16T17:08:03.028 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:03.024+0000 7ff20e000640 1 -- 172.21.15.45:0/597891006 <== mon.1 v2:172.21.15.117:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7ff214085080 con 0x55924312acb0 2024-04-16T17:08:03.028 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:03.024+0000 7ff22df80740 10 client.26008 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T17:08:03.028 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:03.024+0000 7ff22df80740 10 client.26008 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T17:08:03.028 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:03.024+0000 7ff22df80740 20 client.26008 populate_metadata read hostname 'smithi045' 2024-04-16T17:08:03.028 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:03.024+0000 7ff22df80740 1 --2- 172.21.15.45:0/597891006 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x5592431ab1d0 0x5592431ad5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:03.028 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:03.024+0000 7ff22df80740 4 client.26008 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-16T17:08:03.028 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:03.024+0000 7ff22df80740 1 -- 172.21.15.45:0/597891006 --> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559243093010 con 0x5592431ab1d0 2024-04-16T17:08:03.028 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:03.025+0000 7ff223400640 1 --2- 172.21.15.45:0/597891006 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x5592431ab1d0 0x5592431ad5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:03.028 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:03.025+0000 7ff223400640 1 --2- 172.21.15.45:0/597891006 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x5592431ab1d0 0x5592431ad5c0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d4e4c9939d4c2a00 server_cookie=f16f7595d3678926 in_seq=0 out_seq=0 2024-04-16T17:08:03.028 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:03.025+0000 7ff20e000640 10 client.26008 ms_handle_connect on v2:172.21.15.104:6838/3660070592 2024-04-16T17:08:03.128 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:03.128+0000 7ff20e000640 1 -- 172.21.15.45:0/597891006 <== mds.0 v2:172.21.15.104:6838/3660070592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x559243093010 con 0x5592431ab1d0 2024-04-16T17:08:03.128 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:03.128+0000 7ff20e000640 10 client.26008 handle_command_reply: tid=0 2024-04-16T17:08:03.129 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:03.128+0000 7ff22df80740 1 client.26008 shutdown 2024-04-16T17:08:03.129 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:03.128+0000 7ff22df80740 1 -- 172.21.15.45:0/597891006 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55924312acb0 msgr2=0x55924314a0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:03.129 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:03.128+0000 7ff22df80740 1 --2- 172.21.15.45:0/597891006 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55924312acb0 0x55924314a0f0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7ff214010fd0 tx=0x7ff214098320 comp rx=0 tx=0).stop 2024-04-16T17:08:03.129 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:03.128+0000 7ff22df80740 1 -- 172.21.15.45:0/597891006 shutdown_connections 2024-04-16T17:08:03.129 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:03.128+0000 7ff22df80740 1 --2- 172.21.15.45:0/597891006 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559243134230 0x55924312df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:03.129 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:03.128+0000 7ff22df80740 1 --2- 172.21.15.45:0/597891006 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55924312ba20 0x55924312d9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:03.129 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:03.128+0000 7ff22df80740 1 --2- 172.21.15.45:0/597891006 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x5592431ab1d0 0x5592431ad5c0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:03.129 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:03.128+0000 7ff22df80740 1 --2- 172.21.15.45:0/597891006 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55924312acb0 0x55924314a0f0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:03.129 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:03.128+0000 7ff20e000640 0 client.26008 ms_handle_reset on v2:172.21.15.104:6838/3660070592 2024-04-16T17:08:03.129 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:03.128+0000 7ff22df80740 1 -- 172.21.15.45:0/597891006 >> 172.21.15.45:0/597891006 conn(0x5592430c15d0 msgr2=0x5592431206b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:08:03.129 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:03.128+0000 7ff22df80740 1 -- 172.21.15.45:0/597891006 shutdown_connections 2024-04-16T17:08:03.129 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:03.128+0000 7ff22df80740 1 -- 172.21.15.45:0/597891006 wait complete. 2024-04-16T17:08:03.129 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:03.128+0000 7ff22df80740 20 client.26008 trim_cache size 0 max 16384 2024-04-16T17:08:03.130 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-04-16T17:08:03.130 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-04-16T17:08:03.130 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-04-16T17:08:03.130 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-04-16T17:08:03.131 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:03.129+0000 7ff22b200640 1 -- 172.21.15.45:0/577392108 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7ff2080a3fe0 msgr2=0x7ff2080a64a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:03.131 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:03.129+0000 7ff22b200640 1 --2- 172.21.15.45:0/577392108 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7ff2080a3fe0 0x7ff2080a64a0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7ff21c02c430 tx=0x7ff21c005fd0 comp rx=0 tx=0).stop 2024-04-16T17:08:03.131 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:03.129+0000 7ff22b200640 1 -- 172.21.15.45:0/577392108 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff224102250 msgr2=0x7ff22419e450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:03.131 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:03.129+0000 7ff22b200640 1 --2- 172.21.15.45:0/577392108 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff224102250 0x7ff22419e450 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7ff21400d950 tx=0x7ff21400de10 comp rx=0 tx=0).stop 2024-04-16T17:08:03.131 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:03.130+0000 7ff22b200640 1 -- 172.21.15.45:0/577392108 shutdown_connections 2024-04-16T17:08:03.131 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:03.130+0000 7ff22b200640 1 --2- 172.21.15.45:0/577392108 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7ff2080a3fe0 0x7ff2080a64a0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:03.131 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:03.130+0000 7ff22b200640 1 --2- 172.21.15.45:0/577392108 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff224198650 0x7ff224198b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:03.131 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:03.130+0000 7ff22b200640 1 --2- 172.21.15.45:0/577392108 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff224107320 0x7ff22419e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:03.131 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:03.130+0000 7ff22b200640 1 --2- 172.21.15.45:0/577392108 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff224102250 0x7ff22419e450 secure :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7ff21400d950 tx=0x7ff21400de10 comp rx=0 tx=0).stop 2024-04-16T17:08:03.131 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:03.130+0000 7ff22b200640 1 -- 172.21.15.45:0/577392108 >> 172.21.15.45:0/577392108 conn(0x7ff224079490 msgr2=0x7ff2240ffc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:08:03.132 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:03.131+0000 7ff22b200640 1 -- 172.21.15.45:0/577392108 shutdown_connections 2024-04-16T17:08:03.132 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:03.131+0000 7ff22b200640 1 -- 172.21.15.45:0/577392108 wait complete. 2024-04-16T17:08:03.202 INFO:tasks.workunit.client.0.smithi045.stdout: 1 3794 7.95 MB/sec warmup 5 sec latency 29.300 ms 2024-04-16T17:08:03.254 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:03.254+0000 7f6c48000640 1 -- 172.21.15.45:0/4079100706 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6c40107310 msgr2=0x7f6c4010eee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:03.254 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:03.254+0000 7f6c48000640 1 --2- 172.21.15.45:0/4079100706 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6c40107310 0x7f6c4010eee0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f6c38009be0 tx=0x7f6c3802bef0 comp rx=0 tx=0).stop 2024-04-16T17:08:03.254 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:03.254+0000 7f6c48000640 1 -- 172.21.15.45:0/4079100706 shutdown_connections 2024-04-16T17:08:03.254 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:03.254+0000 7f6c48000640 1 --2- 172.21.15.45:0/4079100706 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6c40102bf0 0x7f6c40106c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:03.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:03.254+0000 7f6c48000640 1 --2- 172.21.15.45:0/4079100706 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6c40107310 0x7f6c4010eee0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:03.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:03.254+0000 7f6c48000640 1 --2- 172.21.15.45:0/4079100706 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6c40102240 0x7f6c40102620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:03.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:03.254+0000 7f6c48000640 1 -- 172.21.15.45:0/4079100706 >> 172.21.15.45:0/4079100706 conn(0x7f6c40079450 msgr2=0x7f6c40079860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:08:03.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:03.254+0000 7f6c48000640 1 -- 172.21.15.45:0/4079100706 shutdown_connections 2024-04-16T17:08:03.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:03.254+0000 7f6c48000640 1 -- 172.21.15.45:0/4079100706 wait complete. 2024-04-16T17:08:03.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:03.255+0000 7f6c48000640 1 Processor -- start 2024-04-16T17:08:03.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:03.255+0000 7f6c48000640 1 -- start start 2024-04-16T17:08:03.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:03.256+0000 7f6c48000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6c40102240 0x7f6c4019e630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:03.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:03.256+0000 7f6c48000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6c40107310 0x7f6c4019eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:03.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:03.256+0000 7f6c48000640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6c40198740 0x7f6c40198bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:03.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:03.256+0000 7f6c48000640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f6c40112420 con 0x7f6c40102240 2024-04-16T17:08:03.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:03.256+0000 7f6c48000640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f6c401122a0 con 0x7f6c40198740 2024-04-16T17:08:03.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:03.256+0000 7f6c48000640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f6c401125a0 con 0x7f6c40107310 2024-04-16T17:08:03.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:03.256+0000 7f6c45600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6c40102240 0x7f6c4019e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:03.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:03.256+0000 7f6c45600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6c40102240 0x7f6c4019e630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35196/0 (socket says 172.21.15.45:35196) 2024-04-16T17:08:03.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:03.256+0000 7f6c45600640 1 -- 172.21.15.45:0/334693158 learned_addr learned my addr 172.21.15.45:0/334693158 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:08:03.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:03.256+0000 7f6c44c00640 1 --2- 172.21.15.45:0/334693158 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6c40107310 0x7f6c4019eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:03.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:03.256+0000 7f6c46000640 1 --2- 172.21.15.45:0/334693158 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6c40198740 0x7f6c40198bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:03.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:03.257+0000 7f6c45600640 1 -- 172.21.15.45:0/334693158 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6c40107310 msgr2=0x7f6c4019eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:03.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:03.257+0000 7f6c45600640 1 --2- 172.21.15.45:0/334693158 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6c40107310 0x7f6c4019eb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:03.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:03.257+0000 7f6c45600640 1 -- 172.21.15.45:0/334693158 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6c40198740 msgr2=0x7f6c40198bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:03.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:03.257+0000 7f6c45600640 1 --2- 172.21.15.45:0/334693158 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6c40198740 0x7f6c40198bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:03.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:03.257+0000 7f6c45600640 1 -- 172.21.15.45:0/334693158 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c40199340 con 0x7f6c40102240 2024-04-16T17:08:03.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:03.257+0000 7f6c44c00640 1 --2- 172.21.15.45:0/334693158 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6c40107310 0x7f6c4019eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:08:03.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:03.257+0000 7f6c46000640 1 --2- 172.21.15.45:0/334693158 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6c40198740 0x7f6c40198bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:08:03.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:03.257+0000 7f6c45600640 1 --2- 172.21.15.45:0/334693158 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6c40102240 0x7f6c4019e630 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f6c3400e9c0 tx=0x7f6c3400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:08:03.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:03.257+0000 7f6c3e000640 1 -- 172.21.15.45:0/334693158 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c3400cde0 con 0x7f6c40102240 2024-04-16T17:08:03.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:03.257+0000 7f6c3e000640 1 -- 172.21.15.45:0/334693158 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c340035f0 con 0x7f6c40102240 2024-04-16T17:08:03.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:03.257+0000 7f6c3e000640 1 -- 172.21.15.45:0/334693158 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c34010690 con 0x7f6c40102240 2024-04-16T17:08:03.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:03.257+0000 7f6c48000640 1 -- 172.21.15.45:0/334693158 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c40074230 con 0x7f6c40102240 2024-04-16T17:08:03.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:03.258+0000 7f6c48000640 1 -- 172.21.15.45:0/334693158 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c40074730 con 0x7f6c40102240 2024-04-16T17:08:03.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:03.261+0000 7f6c4ada8740 1 Processor -- start 2024-04-16T17:08:03.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:03.262+0000 7f6c4ada8740 1 -- start start 2024-04-16T17:08:03.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:03.262+0000 7f6c4ada8740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5579c5689d50 0x5579c568a130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:03.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:03.262+0000 7f6c4ada8740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5579c568a670 0x5579c5692d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:03.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:03.262+0000 7f6c4ada8740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5579c5693250 0x5579c5695640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:03.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:03.262+0000 7f6c4ada8740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5579c55ef020 con 0x5579c5689d50 2024-04-16T17:08:03.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:03.262+0000 7f6c4ada8740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5579c5595460 con 0x5579c5693250 2024-04-16T17:08:03.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:03.262+0000 7f6c4ada8740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5579c55eafe0 con 0x5579c568a670 2024-04-16T17:08:03.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:03.262+0000 7f6c44c00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5579c5693250 0x5579c5695640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:03.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:03.262+0000 7f6c44c00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5579c5693250 0x5579c5695640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:47632/0 (socket says 172.21.15.45:47632) 2024-04-16T17:08:03.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:03.262+0000 7f6c44c00640 1 -- 172.21.15.45:0/3115872942 learned_addr learned my addr 172.21.15.45:0/3115872942 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:08:03.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:03.263+0000 7f6c46000640 1 --2- 172.21.15.45:0/3115872942 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5579c5689d50 0x5579c568a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:03.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:03.263+0000 7f6c3e000640 1 -- 172.21.15.45:0/334693158 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) ==== 138879+0+0 (secure 0 0 0) 0x7f6c340037a0 con 0x7f6c40102240 2024-04-16T17:08:03.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:03.263+0000 7f6c44c00640 1 -- 172.21.15.45:0/3115872942 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5579c568a670 msgr2=0x5579c5692d10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-16T17:08:03.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:03.263+0000 7f6c44c00640 1 --2- 172.21.15.45:0/3115872942 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5579c568a670 0x5579c5692d10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:03.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:03.263+0000 7f6c44c00640 1 -- 172.21.15.45:0/3115872942 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5579c5689d50 msgr2=0x5579c568a130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:08:03.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:03.263+0000 7f6c44c00640 1 --2- 172.21.15.45:0/3115872942 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5579c5689d50 0x5579c568a130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:03.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:03.263+0000 7f6c44c00640 1 -- 172.21.15.45:0/3115872942 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5579c559d040 con 0x5579c5693250 2024-04-16T17:08:03.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:03.263+0000 7f6c46000640 1 --2- 172.21.15.45:0/3115872942 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5579c5689d50 0x5579c568a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:08:03.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:03.263+0000 7f6c44c00640 1 --2- 172.21.15.45:0/3115872942 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5579c5693250 0x5579c5695640 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f6c2c009fd0 tx=0x7f6c2c0118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cbabbea6c237b108 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:08:03.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:03.264+0000 7f6c32600640 1 -- 172.21.15.45:0/3115872942 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c2c004310 con 0x5579c5693250 2024-04-16T17:08:03.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:03.264+0000 7f6c32600640 1 -- 172.21.15.45:0/3115872942 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c2c0044b0 con 0x5579c5693250 2024-04-16T17:08:03.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:03.264+0000 7f6c32600640 1 -- 172.21.15.45:0/3115872942 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c2c0035f0 con 0x5579c5693250 2024-04-16T17:08:03.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:03.264+0000 7f6c3e000640 1 --2- 172.21.15.45:0/334693158 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f6c280a3ec0 0x7f6c280a6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:03.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:03.264+0000 7f6c3e000640 1 -- 172.21.15.45:0/334693158 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f6c340d10e0 con 0x7f6c40102240 2024-04-16T17:08:03.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:03.265+0000 7f6c4ada8740 1 -- 172.21.15.45:0/3115872942 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5579c5693250 msgr2=0x5579c5695640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:03.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:03.265+0000 7f6c4ada8740 1 --2- 172.21.15.45:0/3115872942 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5579c5693250 0x5579c5695640 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f6c2c009fd0 tx=0x7f6c2c0118d0 comp rx=0 tx=0).stop 2024-04-16T17:08:03.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:03.265+0000 7f6c46000640 1 --2- 172.21.15.45:0/334693158 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f6c280a3ec0 0x7f6c280a6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:03.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:03.265+0000 7f6c4ada8740 1 -- 172.21.15.45:0/3115872942 shutdown_connections 2024-04-16T17:08:03.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:03.265+0000 7f6c4ada8740 1 --2- 172.21.15.45:0/3115872942 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5579c568a670 0x5579c5692d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:03.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:03.265+0000 7f6c4ada8740 1 --2- 172.21.15.45:0/3115872942 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5579c5693250 0x5579c5695640 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:03.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:03.265+0000 7f6c4ada8740 1 --2- 172.21.15.45:0/3115872942 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5579c5689d50 0x5579c568a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:03.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:03.265+0000 7f6c4ada8740 1 -- 172.21.15.45:0/3115872942 >> 172.21.15.45:0/3115872942 conn(0x5579c5684480 msgr2=0x5579c5684860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:08:03.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:03.265+0000 7f6c4ada8740 1 -- 172.21.15.45:0/3115872942 shutdown_connections 2024-04-16T17:08:03.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:03.265+0000 7f6c4ada8740 1 -- 172.21.15.45:0/3115872942 wait complete. 2024-04-16T17:08:03.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:03.266+0000 7f6c4ada8740 1 -- start start 2024-04-16T17:08:03.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:03.266+0000 7f6c46000640 1 --2- 172.21.15.45:0/334693158 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f6c280a3ec0 0x7f6c280a6380 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f6c38004af0 tx=0x7f6c3803f040 comp rx=0 tx=0).ready entity=mgr.25411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:08:03.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:03.266+0000 7f6c4ada8740 1 Processor -- start 2024-04-16T17:08:03.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:03.266+0000 7f6c4ada8740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5579c5689d50 0x5579c5690310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:03.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:03.266+0000 7f6c4ada8740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5579c568a670 0x5579c5690850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:03.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:03.266+0000 7f6c4ada8740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5579c5693250 0x5579c5690d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:03.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:03.266+0000 7f6c4ada8740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5579c5696040 con 0x5579c5689d50 2024-04-16T17:08:03.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:03.266+0000 7f6c4ada8740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5579c55a8830 con 0x5579c5693250 2024-04-16T17:08:03.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:03.266+0000 7f6c4ada8740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5579c56961c0 con 0x5579c568a670 2024-04-16T17:08:03.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:03.266+0000 7f6c4ada8740 10 client.0 _ll_register_callbacks cb 0x5579c55b0150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:08:03.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:03.267+0000 7f6c44c00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5579c568a670 0x5579c5690850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:03.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:03.267+0000 7f6c45600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5579c5689d50 0x5579c5690310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:03.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:03.267+0000 7f6c46000640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5579c5693250 0x5579c5690d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:03.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:03.267+0000 7f6c44c00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5579c568a670 0x5579c5690850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:46940/0 (socket says 172.21.15.45:46940) 2024-04-16T17:08:03.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:03.267+0000 7f6c44c00640 1 -- 172.21.15.45:0/2081896982 learned_addr learned my addr 172.21.15.45:0/2081896982 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:08:03.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:03.267+0000 7f6c44c00640 1 -- 172.21.15.45:0/2081896982 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5579c5693250 msgr2=0x5579c5690d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:08:03.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:03.267+0000 7f6c44c00640 1 --2- 172.21.15.45:0/2081896982 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5579c5693250 0x5579c5690d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:03.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:03.267+0000 7f6c44c00640 1 -- 172.21.15.45:0/2081896982 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5579c5689d50 msgr2=0x5579c5690310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:08:03.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:03.267+0000 7f6c44c00640 1 --2- 172.21.15.45:0/2081896982 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5579c5689d50 0x5579c5690310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:03.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:03.267+0000 7f6c44c00640 1 -- 172.21.15.45:0/2081896982 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5579c55ac920 con 0x5579c568a670 2024-04-16T17:08:03.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:03.268+0000 7f6c44c00640 1 --2- 172.21.15.45:0/2081896982 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5579c568a670 0x5579c5690850 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f6c2c009350 tx=0x7f6c2c0049c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c5287ad73492e1df server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:08:03.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:03.268+0000 7f6c26a00640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-04-16T17:08:03.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:03.268+0000 7f6c26a00640 1 -- 172.21.15.45:0/2081896982 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c2c029070 con 0x5579c568a670 2024-04-16T17:08:03.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:03.268+0000 7f6c26a00640 1 -- 172.21.15.45:0/2081896982 <== mon.1 v2:172.21.15.117:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f6c2c003a80 con 0x5579c568a670 2024-04-16T17:08:03.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:03.268+0000 7f6c4ada8740 1 -- 172.21.15.45:0/2081896982 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6c2c0044b0 con 0x5579c568a670 2024-04-16T17:08:03.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:03.268+0000 7f6c26a00640 1 -- 172.21.15.45:0/2081896982 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f6c2c035050 con 0x5579c568a670 2024-04-16T17:08:03.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:03.268+0000 7f6c4ada8740 10 client.26017 fetch_fsmap learned FSMap version 25 2024-04-16T17:08:03.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:03.268+0000 7f6c4ada8740 1 -- 172.21.15.45:0/2081896982 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5579c557a7f0 con 0x5579c568a670 2024-04-16T17:08:03.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:03.269+0000 7f6c26a00640 1 -- 172.21.15.45:0/2081896982 <== mon.1 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c2c0044b0 con 0x5579c568a670 2024-04-16T17:08:03.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:03.269+0000 7f6c26a00640 1 -- 172.21.15.45:0/2081896982 <== mon.1 v2:172.21.15.117:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f6c2c00f570 con 0x5579c568a670 2024-04-16T17:08:03.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:03.269+0000 7f6c4ada8740 10 client.26017 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T17:08:03.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:03.269+0000 7f6c4ada8740 10 client.26017 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:08:03.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:03.269+0000 7f6c4ada8740 20 client.26017 populate_metadata read hostname 'smithi045' 2024-04-16T17:08:03.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:03.269+0000 7f6c4ada8740 1 --2- 172.21.15.45:0/2081896982 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x5579c56a34f0 0x5579c56a58e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:03.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:03.269+0000 7f6c4ada8740 4 client.26017 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:08:03.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:03.269+0000 7f6c4ada8740 1 -- 172.21.15.45:0/2081896982 --> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5579c55fca40 con 0x5579c56a34f0 2024-04-16T17:08:03.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:03.269+0000 7f6c45600640 1 --2- 172.21.15.45:0/2081896982 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x5579c56a34f0 0x5579c56a58e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:03.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:03.270+0000 7f6c45600640 1 --2- 172.21.15.45:0/2081896982 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x5579c56a34f0 0x5579c56a58e0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=63568b5a0056cfc server_cookie=e315b65fc09d512c in_seq=0 out_seq=0 2024-04-16T17:08:03.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:03.270+0000 7f6c26a00640 10 client.26017 ms_handle_connect on v2:172.21.15.104:6838/3660070592 2024-04-16T17:08:03.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:03.271+0000 7f6c26a00640 1 -- 172.21.15.45:0/2081896982 <== mds.0 v2:172.21.15.104:6838/3660070592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5579c55fca40 con 0x5579c56a34f0 2024-04-16T17:08:03.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:03.271+0000 7f6c26a00640 10 client.26017 handle_command_reply: tid=0 2024-04-16T17:08:03.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:03.271+0000 7f6c4ada8740 1 client.26017 shutdown 2024-04-16T17:08:03.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:03.271+0000 7f6c4ada8740 1 -- 172.21.15.45:0/2081896982 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5579c568a670 msgr2=0x5579c5690850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:03.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:03.271+0000 7f6c4ada8740 1 --2- 172.21.15.45:0/2081896982 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5579c568a670 0x5579c5690850 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f6c2c009350 tx=0x7f6c2c0049c0 comp rx=0 tx=0).stop 2024-04-16T17:08:03.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:03.271+0000 7f6c4ada8740 1 -- 172.21.15.45:0/2081896982 shutdown_connections 2024-04-16T17:08:03.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:03.271+0000 7f6c4ada8740 1 --2- 172.21.15.45:0/2081896982 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x5579c56a34f0 0x5579c56a58e0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:03.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:03.271+0000 7f6c4ada8740 1 --2- 172.21.15.45:0/2081896982 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5579c568a670 0x5579c5690850 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:03.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:03.271+0000 7f6c4ada8740 1 --2- 172.21.15.45:0/2081896982 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5579c5693250 0x5579c5690d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:03.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:03.271+0000 7f6c4ada8740 1 --2- 172.21.15.45:0/2081896982 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5579c5689d50 0x5579c5690310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:03.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:03.271+0000 7f6c4ada8740 1 -- 172.21.15.45:0/2081896982 >> 172.21.15.45:0/2081896982 conn(0x5579c5684480 msgr2=0x5579c56861d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:08:03.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:03.271+0000 7f6c26a00640 0 client.26017 ms_handle_reset on v2:172.21.15.104:6838/3660070592 2024-04-16T17:08:03.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:03.271+0000 7f6c4ada8740 1 -- 172.21.15.45:0/2081896982 shutdown_connections 2024-04-16T17:08:03.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:03.271+0000 7f6c4ada8740 1 -- 172.21.15.45:0/2081896982 wait complete. 2024-04-16T17:08:03.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:03.272+0000 7f6c4ada8740 20 client.26017 trim_cache size 0 max 16384 2024-04-16T17:08:03.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:03.286+0000 7f6c4ada8740 1 Processor -- start 2024-04-16T17:08:03.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:03.286+0000 7f6c4ada8740 1 -- start start 2024-04-16T17:08:03.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:03.286+0000 7f6c4ada8740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5579c5689d50 0x5579c5692a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:03.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:03.286+0000 7f6c4ada8740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5579c568a670 0x5579c569ef00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:03.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:03.286+0000 7f6c4ada8740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5579c5693250 0x5579c56aaf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:03.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:03.287+0000 7f6c4ada8740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f6c401afb60 con 0x5579c568a670 2024-04-16T17:08:03.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:03.287+0000 7f6c4ada8740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f6c401ad430 con 0x5579c5689d50 2024-04-16T17:08:03.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:03.287+0000 7f6c4ada8740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f6c401ab260 con 0x5579c5693250 2024-04-16T17:08:03.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:03.287+0000 7f6c45600640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5579c5689d50 0x5579c5692a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:03.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:03.287+0000 7f6c45600640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5579c5689d50 0x5579c5692a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:47636/0 (socket says 172.21.15.45:47636) 2024-04-16T17:08:03.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:03.287+0000 7f6c45600640 1 -- 172.21.15.45:0/1179653036 learned_addr learned my addr 172.21.15.45:0/1179653036 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:08:03.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:03.287+0000 7f6c44c00640 1 --2- 172.21.15.45:0/1179653036 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5579c568a670 0x5579c569ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:03.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:03.287+0000 7f6c46000640 1 --2- 172.21.15.45:0/1179653036 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5579c5693250 0x5579c56aaf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:03.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:03.287+0000 7f6c45600640 1 -- 172.21.15.45:0/1179653036 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5579c5693250 msgr2=0x5579c56aaf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:08:03.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:03.287+0000 7f6c45600640 1 --2- 172.21.15.45:0/1179653036 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5579c5693250 0x5579c56aaf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:03.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:03.287+0000 7f6c45600640 1 -- 172.21.15.45:0/1179653036 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5579c568a670 msgr2=0x5579c569ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:08:03.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:03.287+0000 7f6c45600640 1 --2- 172.21.15.45:0/1179653036 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5579c568a670 0x5579c569ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:03.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:03.287+0000 7f6c45600640 1 -- 172.21.15.45:0/1179653036 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5579c568a4a0 con 0x5579c5689d50 2024-04-16T17:08:03.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:03.287+0000 7f6c45600640 1 --2- 172.21.15.45:0/1179653036 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5579c5689d50 0x5579c5692a40 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f6c3409c780 tx=0x7f6c34005c80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dcb148f10395db87 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:08:03.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:03.287+0000 7f6c31c00640 1 -- 172.21.15.45:0/1179653036 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c340a0e30 con 0x5579c5689d50 2024-04-16T17:08:03.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:03.287+0000 7f6c31c00640 1 -- 172.21.15.45:0/1179653036 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c340a56f0 con 0x5579c5689d50 2024-04-16T17:08:03.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:03.287+0000 7f6c31c00640 1 -- 172.21.15.45:0/1179653036 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c340a5260 con 0x5579c5689d50 2024-04-16T17:08:03.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:03.287+0000 7f6c4ada8740 1 -- 172.21.15.45:0/1179653036 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5579c5689d50 msgr2=0x5579c5692a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:03.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:03.287+0000 7f6c4ada8740 1 --2- 172.21.15.45:0/1179653036 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5579c5689d50 0x5579c5692a40 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f6c3409c780 tx=0x7f6c34005c80 comp rx=0 tx=0).stop 2024-04-16T17:08:03.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:03.287+0000 7f6c4ada8740 1 -- 172.21.15.45:0/1179653036 shutdown_connections 2024-04-16T17:08:03.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:03.287+0000 7f6c4ada8740 1 --2- 172.21.15.45:0/1179653036 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5579c5693250 0x5579c56aaf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:03.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:03.287+0000 7f6c4ada8740 1 --2- 172.21.15.45:0/1179653036 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5579c568a670 0x5579c569ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:03.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:03.287+0000 7f6c4ada8740 1 --2- 172.21.15.45:0/1179653036 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5579c5689d50 0x5579c5692a40 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:03.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:03.287+0000 7f6c4ada8740 1 -- 172.21.15.45:0/1179653036 >> 172.21.15.45:0/1179653036 conn(0x5579c5684480 msgr2=0x5579c568b5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:08:03.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:03.287+0000 7f6c4ada8740 1 -- 172.21.15.45:0/1179653036 shutdown_connections 2024-04-16T17:08:03.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:03.287+0000 7f6c4ada8740 1 -- 172.21.15.45:0/1179653036 wait complete. 2024-04-16T17:08:03.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:03.287+0000 7f6c4ada8740 1 -- start start 2024-04-16T17:08:03.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:03.287+0000 7f6c4ada8740 1 Processor -- start 2024-04-16T17:08:03.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:03.290+0000 7f6c4ada8740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5579c5689d50 0x5579c569b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:03.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:03.290+0000 7f6c4ada8740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5579c568a670 0x5579c569b920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:03.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:03.290+0000 7f6c4ada8740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5579c5693250 0x5579c5696b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:03.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:03.290+0000 7f6c4ada8740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f6c401ab8f0 con 0x5579c5689d50 2024-04-16T17:08:03.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:03.290+0000 7f6c4ada8740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f6c401acd60 con 0x5579c5693250 2024-04-16T17:08:03.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:03.290+0000 7f6c4ada8740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f6c401abfc0 con 0x5579c568a670 2024-04-16T17:08:03.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:03.290+0000 7f6c4ada8740 10 client.0 _ll_register_callbacks cb 0x7f6c2c009e20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:08:03.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:03.290+0000 7f6c45600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5579c5689d50 0x5579c569b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:03.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:03.290+0000 7f6c45600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5579c5689d50 0x5579c569b3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35236/0 (socket says 172.21.15.45:35236) 2024-04-16T17:08:03.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:03.290+0000 7f6c45600640 1 -- 172.21.15.45:0/1694527612 learned_addr learned my addr 172.21.15.45:0/1694527612 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:08:03.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:03.290+0000 7f6c44c00640 1 --2- 172.21.15.45:0/1694527612 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5579c568a670 0x5579c569b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:03.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:03.290+0000 7f6c46000640 1 --2- 172.21.15.45:0/1694527612 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5579c5693250 0x5579c5696b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:03.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:03.290+0000 7f6c44c00640 1 -- 172.21.15.45:0/1694527612 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5579c5693250 msgr2=0x5579c5696b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:08:03.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:03.290+0000 7f6c44c00640 1 --2- 172.21.15.45:0/1694527612 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5579c5693250 0x5579c5696b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:03.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:03.290+0000 7f6c44c00640 1 -- 172.21.15.45:0/1694527612 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5579c5689d50 msgr2=0x5579c569b3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:08:03.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:03.290+0000 7f6c44c00640 1 --2- 172.21.15.45:0/1694527612 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5579c5689d50 0x5579c569b3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:03.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:03.290+0000 7f6c44c00640 1 -- 172.21.15.45:0/1694527612 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5579c53013f0 con 0x5579c568a670 2024-04-16T17:08:03.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:03.290+0000 7f6c45600640 1 --2- 172.21.15.45:0/1694527612 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5579c5689d50 0x5579c569b3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:08:03.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:03.290+0000 7f6c46000640 1 --2- 172.21.15.45:0/1694527612 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5579c5693250 0x5579c5696b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:08:03.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:03.290+0000 7f6c44c00640 1 --2- 172.21.15.45:0/1694527612 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5579c568a670 0x5579c569b920 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f6c2c011dd0 tx=0x7f6c2c004630 comp rx=0 tx=0).ready entity=mon.1 client_cookie=94e851c16d000ab8 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:08:03.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:03.290+0000 7f6c26a00640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-04-16T17:08:03.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:03.290+0000 7f6c26a00640 1 -- 172.21.15.45:0/1694527612 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c2c029070 con 0x5579c568a670 2024-04-16T17:08:03.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:03.290+0000 7f6c4ada8740 1 -- 172.21.15.45:0/1694527612 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6c340a56f0 con 0x5579c568a670 2024-04-16T17:08:03.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:03.290+0000 7f6c26a00640 1 -- 172.21.15.45:0/1694527612 <== mon.1 v2:172.21.15.117:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f6c2c00f4e0 con 0x5579c568a670 2024-04-16T17:08:03.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:03.290+0000 7f6c26a00640 1 -- 172.21.15.45:0/1694527612 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f6c2c040050 con 0x5579c568a670 2024-04-16T17:08:03.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:03.290+0000 7f6c4ada8740 10 client.26023 fetch_fsmap learned FSMap version 25 2024-04-16T17:08:03.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:03.290+0000 7f6c4ada8740 1 -- 172.21.15.45:0/1694527612 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5579c53cb7d0 con 0x5579c568a670 2024-04-16T17:08:03.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:03.290+0000 7f6c26a00640 1 -- 172.21.15.45:0/1694527612 <== mon.1 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c340a56f0 con 0x5579c568a670 2024-04-16T17:08:03.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:03.290+0000 7f6c26a00640 1 -- 172.21.15.45:0/1694527612 <== mon.1 v2:172.21.15.117:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f6c2c02e440 con 0x5579c568a670 2024-04-16T17:08:03.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:03.290+0000 7f6c4ada8740 10 client.26023 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T17:08:03.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:03.290+0000 7f6c4ada8740 10 client.26023 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:08:03.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:03.290+0000 7f6c4ada8740 20 client.26023 populate_metadata read hostname 'smithi045' 2024-04-16T17:08:03.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:03.290+0000 7f6c4ada8740 1 --2- 172.21.15.45:0/1694527612 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x5579c5709860 0x5579c570bc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:03.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:03.290+0000 7f6c4ada8740 4 client.26023 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T17:08:03.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:03.290+0000 7f6c4ada8740 1 -- 172.21.15.45:0/1694527612 --> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5579c55f68b0 con 0x5579c5709860 2024-04-16T17:08:03.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:03.290+0000 7f6c45600640 1 --2- 172.21.15.45:0/1694527612 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x5579c5709860 0x5579c570bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:03.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:03.290+0000 7f6c45600640 1 --2- 172.21.15.45:0/1694527612 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x5579c5709860 0x5579c570bc80 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=70b915a8cb9779b9 server_cookie=41bc9c188466f29c in_seq=0 out_seq=0 2024-04-16T17:08:03.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:03.290+0000 7f6c26a00640 10 client.26023 ms_handle_connect on v2:172.21.15.104:6838/3660070592 2024-04-16T17:08:03.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:03.290+0000 7f6c26a00640 1 -- 172.21.15.45:0/1694527612 <== mds.0 v2:172.21.15.104:6838/3660070592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5579c55f68b0 con 0x5579c5709860 2024-04-16T17:08:03.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:03.290+0000 7f6c26a00640 10 client.26023 handle_command_reply: tid=0 2024-04-16T17:08:03.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:03.290+0000 7f6c4ada8740 1 client.26023 shutdown 2024-04-16T17:08:03.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:03.290+0000 7f6c4ada8740 1 -- 172.21.15.45:0/1694527612 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5579c568a670 msgr2=0x5579c569b920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:03.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:03.290+0000 7f6c4ada8740 1 --2- 172.21.15.45:0/1694527612 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5579c568a670 0x5579c569b920 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f6c2c011dd0 tx=0x7f6c2c004630 comp rx=0 tx=0).stop 2024-04-16T17:08:03.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:03.290+0000 7f6c4ada8740 1 -- 172.21.15.45:0/1694527612 shutdown_connections 2024-04-16T17:08:03.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:03.290+0000 7f6c4ada8740 1 --2- 172.21.15.45:0/1694527612 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x5579c5709860 0x5579c570bc80 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:03.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:03.290+0000 7f6c4ada8740 1 --2- 172.21.15.45:0/1694527612 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5579c568a670 0x5579c569b920 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:03.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:03.290+0000 7f6c4ada8740 1 --2- 172.21.15.45:0/1694527612 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5579c5693250 0x5579c5696b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:03.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:03.290+0000 7f6c4ada8740 1 --2- 172.21.15.45:0/1694527612 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5579c5689d50 0x5579c569b3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:03.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:03.290+0000 7f6c4ada8740 1 -- 172.21.15.45:0/1694527612 >> 172.21.15.45:0/1694527612 conn(0x5579c5684480 msgr2=0x5579c5690230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:08:03.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:03.290+0000 7f6c26a00640 0 client.26023 ms_handle_reset on v2:172.21.15.104:6838/3660070592 2024-04-16T17:08:03.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:03.290+0000 7f6c4ada8740 1 -- 172.21.15.45:0/1694527612 shutdown_connections 2024-04-16T17:08:03.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:03.290+0000 7f6c4ada8740 1 -- 172.21.15.45:0/1694527612 wait complete. 2024-04-16T17:08:03.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:03.290+0000 7f6c4ada8740 20 client.26023 trim_cache size 0 max 16384 2024-04-16T17:08:03.298 INFO:teuthology.orchestra.run.smithi045.stdout:[] 2024-04-16T17:08:03.298 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:03.290+0000 7f6c48000640 1 -- 172.21.15.45:0/334693158 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f6c280a3ec0 msgr2=0x7f6c280a6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:03.299 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:03.290+0000 7f6c48000640 1 --2- 172.21.15.45:0/334693158 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f6c280a3ec0 0x7f6c280a6380 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f6c38004af0 tx=0x7f6c3803f040 comp rx=0 tx=0).stop 2024-04-16T17:08:03.299 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:03.290+0000 7f6c48000640 1 -- 172.21.15.45:0/334693158 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6c40102240 msgr2=0x7f6c4019e630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:03.299 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:03.290+0000 7f6c48000640 1 --2- 172.21.15.45:0/334693158 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6c40102240 0x7f6c4019e630 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f6c3400e9c0 tx=0x7f6c3400ee80 comp rx=0 tx=0).stop 2024-04-16T17:08:03.299 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:03.290+0000 7f6c48000640 1 -- 172.21.15.45:0/334693158 shutdown_connections 2024-04-16T17:08:03.299 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:03.290+0000 7f6c48000640 1 --2- 172.21.15.45:0/334693158 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f6c280a3ec0 0x7f6c280a6380 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:03.299 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:03.290+0000 7f6c48000640 1 --2- 172.21.15.45:0/334693158 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6c40107310 0x7f6c4019eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:03.299 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:03.290+0000 7f6c48000640 1 --2- 172.21.15.45:0/334693158 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6c40198740 0x7f6c40198bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:03.299 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:03.290+0000 7f6c48000640 1 --2- 172.21.15.45:0/334693158 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6c40102240 0x7f6c4019e630 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:03.299 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:03.290+0000 7f6c48000640 1 -- 172.21.15.45:0/334693158 >> 172.21.15.45:0/334693158 conn(0x7f6c40079450 msgr2=0x7f6c400ffe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:08:03.299 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:03.295+0000 7f6c48000640 1 -- 172.21.15.45:0/334693158 shutdown_connections 2024-04-16T17:08:03.299 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:03.295+0000 7f6c48000640 1 -- 172.21.15.45:0/334693158 wait complete. 2024-04-16T17:08:03.677 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:08:03 smithi104 ceph-mon[36457]: from='client.? 172.21.15.45:0/2580881476' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:08:03.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:08:03 smithi117 ceph-mon[36730]: from='client.? 172.21.15.45:0/2580881476' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:08:04.203 INFO:tasks.workunit.client.0.smithi045.stdout: 1 3998 6.68 MB/sec warmup 6 sec latency 98.784 ms 2024-04-16T17:08:04.314 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-16T17:08:04.314 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T17:08:04.314 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-16T17:08:04.527 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.527+0000 7fbe15c00640 1 -- 172.21.15.45:0/1473304667 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbe101071e0 msgr2=0x7fbe1010edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:04.527 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.527+0000 7fbe15c00640 1 --2- 172.21.15.45:0/1473304667 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbe101071e0 0x7fbe1010edb0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fbe08009ca0 tx=0x7fbe0802c100 comp rx=0 tx=0).stop 2024-04-16T17:08:04.527 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.527+0000 7fbe15c00640 1 -- 172.21.15.45:0/1473304667 shutdown_connections 2024-04-16T17:08:04.527 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.527+0000 7fbe15c00640 1 --2- 172.21.15.45:0/1473304667 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbe101071e0 0x7fbe1010edb0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:04.527 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.527+0000 7fbe15c00640 1 --2- 172.21.15.45:0/1473304667 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbe10102ac0 0x7fbe10106b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:04.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.527+0000 7fbe15c00640 1 --2- 172.21.15.45:0/1473304667 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbe10102110 0x7fbe101024f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:04.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.527+0000 7fbe15c00640 1 -- 172.21.15.45:0/1473304667 >> 172.21.15.45:0/1473304667 conn(0x7fbe100fd9d0 msgr2=0x7fbe100ffdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:08:04.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.527+0000 7fbe15c00640 1 -- 172.21.15.45:0/1473304667 shutdown_connections 2024-04-16T17:08:04.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.527+0000 7fbe15c00640 1 -- 172.21.15.45:0/1473304667 wait complete. 2024-04-16T17:08:04.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.528+0000 7fbe15c00640 1 Processor -- start 2024-04-16T17:08:04.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.528+0000 7fbe15c00640 1 -- start start 2024-04-16T17:08:04.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.529+0000 7fbe15c00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbe10102110 0x7fbe101a0660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:04.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.529+0000 7fbe15c00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbe101071e0 0x7fbe101a0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:04.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.529+0000 7fbe15c00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbe1019a820 0x7fbe1019acd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:04.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.529+0000 7fbe15c00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fbe10112280 con 0x7fbe10102110 2024-04-16T17:08:04.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.529+0000 7fbe15c00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fbe10112100 con 0x7fbe101071e0 2024-04-16T17:08:04.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.529+0000 7fbe15c00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fbe10112400 con 0x7fbe1019a820 2024-04-16T17:08:04.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.529+0000 7fbe0fe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbe10102110 0x7fbe101a0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:04.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.529+0000 7fbe0fe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbe10102110 0x7fbe101a0660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35246/0 (socket says 172.21.15.45:35246) 2024-04-16T17:08:04.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.529+0000 7fbe0fe00640 1 -- 172.21.15.45:0/190633189 learned_addr learned my addr 172.21.15.45:0/190633189 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:08:04.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.529+0000 7fbe0f400640 1 --2- 172.21.15.45:0/190633189 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbe101071e0 0x7fbe101a0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:04.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.529+0000 7fbe15200640 1 --2- 172.21.15.45:0/190633189 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbe1019a820 0x7fbe1019acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:04.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.529+0000 7fbe0fe00640 1 -- 172.21.15.45:0/190633189 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbe1019a820 msgr2=0x7fbe1019acd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:04.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.529+0000 7fbe0fe00640 1 --2- 172.21.15.45:0/190633189 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbe1019a820 0x7fbe1019acd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:04.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.529+0000 7fbe0fe00640 1 -- 172.21.15.45:0/190633189 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbe101071e0 msgr2=0x7fbe101a0ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:04.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.529+0000 7fbe0fe00640 1 --2- 172.21.15.45:0/190633189 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbe101071e0 0x7fbe101a0ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:04.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.529+0000 7fbe0fe00640 1 -- 172.21.15.45:0/190633189 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe1019b3b0 con 0x7fbe10102110 2024-04-16T17:08:04.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.530+0000 7fbe0fe00640 1 --2- 172.21.15.45:0/190633189 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbe10102110 0x7fbe101a0660 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fbe0000ea60 tx=0x7fbe0000ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:08:04.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.530+0000 7fbe0cc00640 1 -- 172.21.15.45:0/190633189 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe0000cde0 con 0x7fbe10102110 2024-04-16T17:08:04.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.530+0000 7fbe0cc00640 1 -- 172.21.15.45:0/190633189 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbe000035f0 con 0x7fbe10102110 2024-04-16T17:08:04.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.530+0000 7fbe0cc00640 1 -- 172.21.15.45:0/190633189 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe00010670 con 0x7fbe10102110 2024-04-16T17:08:04.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.530+0000 7fbe15c00640 1 -- 172.21.15.45:0/190633189 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe1019b6a0 con 0x7fbe10102110 2024-04-16T17:08:04.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.530+0000 7fbe15c00640 1 -- 172.21.15.45:0/190633189 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe101a7880 con 0x7fbe10102110 2024-04-16T17:08:04.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.534+0000 7fbe1a019740 1 Processor -- start 2024-04-16T17:08:04.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.534+0000 7fbe1a019740 1 -- start start 2024-04-16T17:08:04.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.534+0000 7fbe1a019740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5617caddde90 0x5617cadde270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:04.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.535+0000 7fbe1a019740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5617cadde7b0 0x5617cade6e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:04.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.535+0000 7fbe1a019740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5617cade7390 0x5617cade9780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:04.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.535+0000 7fbe1a019740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5617cad42cb0 con 0x5617cade7390 2024-04-16T17:08:04.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.535+0000 7fbe1a019740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5617cace9370 con 0x5617caddde90 2024-04-16T17:08:04.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.535+0000 7fbe1a019740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5617cad3ec70 con 0x5617cadde7b0 2024-04-16T17:08:04.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.535+0000 7fbe15200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5617caddde90 0x5617cadde270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:04.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.535+0000 7fbe15200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5617caddde90 0x5617cadde270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:47676/0 (socket says 172.21.15.45:47676) 2024-04-16T17:08:04.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.535+0000 7fbe15200640 1 -- 172.21.15.45:0/667318571 learned_addr learned my addr 172.21.15.45:0/667318571 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:08:04.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.535+0000 7fbe15200640 1 -- 172.21.15.45:0/667318571 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5617cadde7b0 msgr2=0x5617cade6e50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-16T17:08:04.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.535+0000 7fbe15200640 1 --2- 172.21.15.45:0/667318571 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5617cadde7b0 0x5617cade6e50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:04.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.535+0000 7fbe15200640 1 -- 172.21.15.45:0/667318571 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5617cade7390 msgr2=0x5617cade9780 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-16T17:08:04.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.535+0000 7fbe15200640 1 --2- 172.21.15.45:0/667318571 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5617cade7390 0x5617cade9780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:04.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.535+0000 7fbe15200640 1 -- 172.21.15.45:0/667318571 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5617cad08910 con 0x5617caddde90 2024-04-16T17:08:04.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.536+0000 7fbe15200640 1 --2- 172.21.15.45:0/667318571 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5617caddde90 0x5617cadde270 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fbe08009ca0 tx=0x7fbe080093f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8b646657ab66b651 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:08:04.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.536+0000 7fbe05600640 1 -- 172.21.15.45:0/667318571 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe08043020 con 0x5617caddde90 2024-04-16T17:08:04.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.536+0000 7fbe05600640 1 -- 172.21.15.45:0/667318571 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbe08032c90 con 0x5617caddde90 2024-04-16T17:08:04.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.536+0000 7fbe05600640 1 -- 172.21.15.45:0/667318571 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe08039040 con 0x5617caddde90 2024-04-16T17:08:04.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.536+0000 7fbe0cc00640 1 -- 172.21.15.45:0/190633189 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) ==== 138879+0+0 (secure 0 0 0) 0x7fbe000037a0 con 0x7fbe10102110 2024-04-16T17:08:04.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.537+0000 7fbe1a019740 1 -- 172.21.15.45:0/667318571 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5617caddde90 msgr2=0x5617cadde270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:04.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.537+0000 7fbe1a019740 1 --2- 172.21.15.45:0/667318571 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5617caddde90 0x5617cadde270 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fbe08009ca0 tx=0x7fbe080093f0 comp rx=0 tx=0).stop 2024-04-16T17:08:04.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.537+0000 7fbe1a019740 1 -- 172.21.15.45:0/667318571 shutdown_connections 2024-04-16T17:08:04.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.537+0000 7fbe1a019740 1 --2- 172.21.15.45:0/667318571 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5617cadde7b0 0x5617cade6e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:04.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.537+0000 7fbe1a019740 1 --2- 172.21.15.45:0/667318571 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5617cade7390 0x5617cade9780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:04.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.537+0000 7fbe1a019740 1 --2- 172.21.15.45:0/667318571 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5617caddde90 0x5617cadde270 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:04.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.537+0000 7fbe1a019740 1 -- 172.21.15.45:0/667318571 >> 172.21.15.45:0/667318571 conn(0x5617cadd85c0 msgr2=0x5617cadd89a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:08:04.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.537+0000 7fbe1a019740 1 -- 172.21.15.45:0/667318571 shutdown_connections 2024-04-16T17:08:04.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.537+0000 7fbe1a019740 1 -- 172.21.15.45:0/667318571 wait complete. 2024-04-16T17:08:04.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.537+0000 7fbe0cc00640 1 --2- 172.21.15.45:0/190633189 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fbdf40a3dd0 0x7fbdf40a6290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:04.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.537+0000 7fbe1a019740 1 -- start start 2024-04-16T17:08:04.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.537+0000 7fbe0cc00640 1 -- 172.21.15.45:0/190633189 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fbe000d11f0 con 0x7fbe10102110 2024-04-16T17:08:04.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.538+0000 7fbe1a019740 1 Processor -- start 2024-04-16T17:08:04.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.538+0000 7fbe15200640 1 --2- 172.21.15.45:0/190633189 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fbdf40a3dd0 0x7fbdf40a6290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:04.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.539+0000 7fbe1a019740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5617caddde90 0x5617cade4510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:04.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.539+0000 7fbe1a019740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5617cadde7b0 0x5617cade4a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:04.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.539+0000 7fbe1a019740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5617cade7390 0x5617cade4f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:04.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.539+0000 7fbe1a019740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5617cadea120 con 0x5617cadde7b0 2024-04-16T17:08:04.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.539+0000 7fbe1a019740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5617cacfc730 con 0x5617cade7390 2024-04-16T17:08:04.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.539+0000 7fbe1a019740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5617cadea2a0 con 0x5617caddde90 2024-04-16T17:08:04.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.539+0000 7fbe15200640 1 --2- 172.21.15.45:0/190633189 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fbdf40a3dd0 0x7fbdf40a6290 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fbe0802c640 tx=0x7fbe08005df0 comp rx=0 tx=0).ready entity=mgr.25411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:08:04.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.539+0000 7fbe1a019740 10 client.0 _ll_register_callbacks cb 0x5617caccb1e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:08:04.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.539+0000 7fbe0fe00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5617caddde90 0x5617cade4510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:04.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.539+0000 7fbe0fe00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5617caddde90 0x5617cade4510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:46994/0 (socket says 172.21.15.45:46994) 2024-04-16T17:08:04.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.539+0000 7fbe0fe00640 1 -- 172.21.15.45:0/2247849000 learned_addr learned my addr 172.21.15.45:0/2247849000 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:08:04.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.539+0000 7fbe15200640 1 --2- 172.21.15.45:0/2247849000 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5617cade7390 0x5617cade4f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:04.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.539+0000 7fbe0f400640 1 --2- 172.21.15.45:0/2247849000 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5617cadde7b0 0x5617cade4a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:04.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.539+0000 7fbe0fe00640 1 -- 172.21.15.45:0/2247849000 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5617cade7390 msgr2=0x5617cade4f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:08:04.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.539+0000 7fbe0fe00640 1 --2- 172.21.15.45:0/2247849000 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5617cade7390 0x5617cade4f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:04.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.539+0000 7fbe0fe00640 1 -- 172.21.15.45:0/2247849000 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5617cadde7b0 msgr2=0x5617cade4a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:08:04.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.539+0000 7fbe0fe00640 1 --2- 172.21.15.45:0/2247849000 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5617cadde7b0 0x5617cade4a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:04.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.539+0000 7fbe0fe00640 1 -- 172.21.15.45:0/2247849000 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5617cab1d7e0 con 0x5617caddde90 2024-04-16T17:08:04.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.540+0000 7fbe0fe00640 1 --2- 172.21.15.45:0/2247849000 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5617caddde90 0x5617cade4510 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fbe0009cb60 tx=0x7fbe0009c810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c84bb1c121da82bb server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:08:04.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.540+0000 7fbdfa000640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-04-16T17:08:04.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.540+0000 7fbdfa000640 1 -- 172.21.15.45:0/2247849000 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe000b6060 con 0x5617caddde90 2024-04-16T17:08:04.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.540+0000 7fbe1a019740 1 -- 172.21.15.45:0/2247849000 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbe08032c90 con 0x5617caddde90 2024-04-16T17:08:04.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.540+0000 7fbdfa000640 1 -- 172.21.15.45:0/2247849000 <== mon.1 v2:172.21.15.117:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fbe000bd0d0 con 0x5617caddde90 2024-04-16T17:08:04.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.540+0000 7fbdfa000640 1 -- 172.21.15.45:0/2247849000 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fbe000a0f80 con 0x5617caddde90 2024-04-16T17:08:04.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.540+0000 7fbe1a019740 10 client.26032 fetch_fsmap learned FSMap version 25 2024-04-16T17:08:04.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.540+0000 7fbe1a019740 1 -- 172.21.15.45:0/2247849000 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5617cab7df70 con 0x5617caddde90 2024-04-16T17:08:04.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.541+0000 7fbdfa000640 1 -- 172.21.15.45:0/2247849000 <== mon.1 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbe08032c90 con 0x5617caddde90 2024-04-16T17:08:04.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.541+0000 7fbdfa000640 1 -- 172.21.15.45:0/2247849000 <== mon.1 v2:172.21.15.117:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fbe000ce7c0 con 0x5617caddde90 2024-04-16T17:08:04.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.541+0000 7fbe1a019740 10 client.26032 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T17:08:04.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.541+0000 7fbe1a019740 10 client.26032 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:08:04.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.541+0000 7fbe1a019740 20 client.26032 populate_metadata read hostname 'smithi045' 2024-04-16T17:08:04.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.541+0000 7fbe1a019740 1 --2- 172.21.15.45:0/2247849000 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x5617cae000a0 0x5617cae02490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:04.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.541+0000 7fbe1a019740 4 client.26032 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:08:04.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.541+0000 7fbe1a019740 1 -- 172.21.15.45:0/2247849000 --> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5617cad506d0 con 0x5617cae000a0 2024-04-16T17:08:04.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.541+0000 7fbe0f400640 1 --2- 172.21.15.45:0/2247849000 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x5617cae000a0 0x5617cae02490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:04.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.542+0000 7fbe0f400640 1 --2- 172.21.15.45:0/2247849000 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x5617cae000a0 0x5617cae02490 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e205370b80b085cd server_cookie=2a5f69f592cdf03a in_seq=0 out_seq=0 2024-04-16T17:08:04.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.542+0000 7fbdfa000640 10 client.26032 ms_handle_connect on v2:172.21.15.104:6838/3660070592 2024-04-16T17:08:04.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.543+0000 7fbdfa000640 1 -- 172.21.15.45:0/2247849000 <== mds.0 v2:172.21.15.104:6838/3660070592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5617cad506d0 con 0x5617cae000a0 2024-04-16T17:08:04.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.543+0000 7fbdfa000640 10 client.26032 handle_command_reply: tid=0 2024-04-16T17:08:04.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.543+0000 7fbe1a019740 1 client.26032 shutdown 2024-04-16T17:08:04.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.543+0000 7fbe1a019740 1 -- 172.21.15.45:0/2247849000 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5617caddde90 msgr2=0x5617cade4510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:04.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.543+0000 7fbe1a019740 1 --2- 172.21.15.45:0/2247849000 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5617caddde90 0x5617cade4510 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fbe0009cb60 tx=0x7fbe0009c810 comp rx=0 tx=0).stop 2024-04-16T17:08:04.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.543+0000 7fbe1a019740 1 -- 172.21.15.45:0/2247849000 shutdown_connections 2024-04-16T17:08:04.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.543+0000 7fbe1a019740 1 --2- 172.21.15.45:0/2247849000 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5617cade7390 0x5617cade4f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:04.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.543+0000 7fbe1a019740 1 --2- 172.21.15.45:0/2247849000 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5617cadde7b0 0x5617cade4a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:04.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.543+0000 7fbe1a019740 1 --2- 172.21.15.45:0/2247849000 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x5617cae000a0 0x5617cae02490 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:04.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.543+0000 7fbe1a019740 1 --2- 172.21.15.45:0/2247849000 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5617caddde90 0x5617cade4510 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:04.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.543+0000 7fbe1a019740 1 -- 172.21.15.45:0/2247849000 >> 172.21.15.45:0/2247849000 conn(0x5617cadd85c0 msgr2=0x5617cadda310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:08:04.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.543+0000 7fbdfa000640 0 client.26032 ms_handle_reset on v2:172.21.15.104:6838/3660070592 2024-04-16T17:08:04.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.543+0000 7fbe1a019740 1 -- 172.21.15.45:0/2247849000 shutdown_connections 2024-04-16T17:08:04.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.543+0000 7fbe1a019740 1 -- 172.21.15.45:0/2247849000 wait complete. 2024-04-16T17:08:04.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.543+0000 7fbe1a019740 20 client.26032 trim_cache size 0 max 16384 2024-04-16T17:08:04.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.562+0000 7fbe1a019740 1 Processor -- start 2024-04-16T17:08:04.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.562+0000 7fbe1a019740 1 -- start start 2024-04-16T17:08:04.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.563+0000 7fbe1a019740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5617caddde90 0x5617cade34f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:04.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.563+0000 7fbe1a019740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5617cadde7b0 0x5617cae07cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:04.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.563+0000 7fbe1a019740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5617cade7390 0x5617cae08230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:04.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.563+0000 7fbe1a019740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fbe101af970 con 0x5617cadde7b0 2024-04-16T17:08:04.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.563+0000 7fbe1a019740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fbe101ad240 con 0x5617cade7390 2024-04-16T17:08:04.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.563+0000 7fbe1a019740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fbe101ab0b0 con 0x5617caddde90 2024-04-16T17:08:04.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.563+0000 7fbe0f400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5617cadde7b0 0x5617cae07cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:04.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.563+0000 7fbe0fe00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5617caddde90 0x5617cade34f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:04.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.563+0000 7fbe0fe00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5617caddde90 0x5617cade34f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:47000/0 (socket says 172.21.15.45:47000) 2024-04-16T17:08:04.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.563+0000 7fbe0fe00640 1 -- 172.21.15.45:0/766976602 learned_addr learned my addr 172.21.15.45:0/766976602 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:08:04.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.563+0000 7fbe15200640 1 --2- 172.21.15.45:0/766976602 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5617cade7390 0x5617cae08230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:04.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.564+0000 7fbe0fe00640 1 -- 172.21.15.45:0/766976602 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5617cade7390 msgr2=0x5617cae08230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:08:04.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.564+0000 7fbe0fe00640 1 --2- 172.21.15.45:0/766976602 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5617cade7390 0x5617cae08230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:04.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.564+0000 7fbe0fe00640 1 -- 172.21.15.45:0/766976602 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5617cadde7b0 msgr2=0x5617cae07cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:08:04.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.564+0000 7fbe0fe00640 1 --2- 172.21.15.45:0/766976602 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5617cadde7b0 0x5617cae07cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:04.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.564+0000 7fbe0fe00640 1 -- 172.21.15.45:0/766976602 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5617caa15170 con 0x5617caddde90 2024-04-16T17:08:04.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.564+0000 7fbe0fe00640 1 --2- 172.21.15.45:0/766976602 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5617caddde90 0x5617cade34f0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fbe00099030 tx=0x7fbe0009b0e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=af04c774897505e4 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:08:04.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.564+0000 7fbe04c00640 1 -- 172.21.15.45:0/766976602 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe0009d2d0 con 0x5617caddde90 2024-04-16T17:08:04.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.564+0000 7fbe04c00640 1 -- 172.21.15.45:0/766976602 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbe000ce390 con 0x5617caddde90 2024-04-16T17:08:04.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.564+0000 7fbe04c00640 1 -- 172.21.15.45:0/766976602 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe000ce730 con 0x5617caddde90 2024-04-16T17:08:04.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.565+0000 7fbe1a019740 1 -- 172.21.15.45:0/766976602 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5617caddde90 msgr2=0x5617cade34f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:04.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.565+0000 7fbe1a019740 1 --2- 172.21.15.45:0/766976602 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5617caddde90 0x5617cade34f0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fbe00099030 tx=0x7fbe0009b0e0 comp rx=0 tx=0).stop 2024-04-16T17:08:04.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.565+0000 7fbe1a019740 1 -- 172.21.15.45:0/766976602 shutdown_connections 2024-04-16T17:08:04.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.565+0000 7fbe1a019740 1 --2- 172.21.15.45:0/766976602 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5617cade7390 0x5617cae08230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:04.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.565+0000 7fbe1a019740 1 --2- 172.21.15.45:0/766976602 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5617cadde7b0 0x5617cae07cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:04.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.565+0000 7fbe1a019740 1 --2- 172.21.15.45:0/766976602 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5617caddde90 0x5617cade34f0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:04.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.565+0000 7fbe1a019740 1 -- 172.21.15.45:0/766976602 >> 172.21.15.45:0/766976602 conn(0x5617cadd85c0 msgr2=0x5617cade4d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:08:04.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.565+0000 7fbe1a019740 1 -- 172.21.15.45:0/766976602 shutdown_connections 2024-04-16T17:08:04.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.565+0000 7fbe1a019740 1 -- 172.21.15.45:0/766976602 wait complete. 2024-04-16T17:08:04.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.566+0000 7fbe1a019740 1 -- start start 2024-04-16T17:08:04.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.566+0000 7fbe1a019740 1 Processor -- start 2024-04-16T17:08:04.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.566+0000 7fbe1a019740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5617caddde90 0x5617cade18f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:04.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.566+0000 7fbe1a019740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5617cadde7b0 0x5617cade1e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:04.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.567+0000 7fbe1a019740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5617cade7390 0x5617cae65560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:04.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.567+0000 7fbe1a019740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fbe101ab700 con 0x5617cadde7b0 2024-04-16T17:08:04.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.567+0000 7fbe1a019740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fbe101acb70 con 0x5617cade7390 2024-04-16T17:08:04.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.567+0000 7fbe1a019740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fbe101abdd0 con 0x5617caddde90 2024-04-16T17:08:04.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.567+0000 7fbe1a019740 10 client.0 _ll_register_callbacks cb 0x7fbe0009c9a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:08:04.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.567+0000 7fbe0f400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5617cadde7b0 0x5617cade1e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:04.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.567+0000 7fbe0f400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5617cadde7b0 0x5617cade1e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35264/0 (socket says 172.21.15.45:35264) 2024-04-16T17:08:04.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.567+0000 7fbe0f400640 1 -- 172.21.15.45:0/3682196687 learned_addr learned my addr 172.21.15.45:0/3682196687 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:08:04.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.567+0000 7fbe0fe00640 1 --2- 172.21.15.45:0/3682196687 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5617caddde90 0x5617cade18f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:04.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.567+0000 7fbe15200640 1 --2- 172.21.15.45:0/3682196687 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5617cade7390 0x5617cae65560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:04.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.567+0000 7fbe0fe00640 1 -- 172.21.15.45:0/3682196687 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5617cade7390 msgr2=0x5617cae65560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:08:04.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.567+0000 7fbe0fe00640 1 --2- 172.21.15.45:0/3682196687 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5617cade7390 0x5617cae65560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:04.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.567+0000 7fbe0fe00640 1 -- 172.21.15.45:0/3682196687 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5617cadde7b0 msgr2=0x5617cade1e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:08:04.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.567+0000 7fbe0fe00640 1 --2- 172.21.15.45:0/3682196687 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5617cadde7b0 0x5617cade1e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:04.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.567+0000 7fbe0fe00640 1 -- 172.21.15.45:0/3682196687 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5617cae65aa0 con 0x5617caddde90 2024-04-16T17:08:04.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.567+0000 7fbe0f400640 1 --2- 172.21.15.45:0/3682196687 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5617cadde7b0 0x5617cade1e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:08:04.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.568+0000 7fbe0fe00640 1 --2- 172.21.15.45:0/3682196687 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5617caddde90 0x5617cade18f0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fbe000a5310 tx=0x7fbe0000e9b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=725c084826d8fa5c server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:08:04.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.568+0000 7fbdfa000640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-04-16T17:08:04.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.568+0000 7fbdfa000640 1 -- 172.21.15.45:0/3682196687 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe0009d2d0 con 0x5617caddde90 2024-04-16T17:08:04.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.568+0000 7fbdfa000640 1 -- 172.21.15.45:0/3682196687 <== mon.1 v2:172.21.15.117:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fbe00099240 con 0x5617caddde90 2024-04-16T17:08:04.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.568+0000 7fbe1a019740 1 -- 172.21.15.45:0/3682196687 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbe000ce390 con 0x5617caddde90 2024-04-16T17:08:04.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.568+0000 7fbdfa000640 1 -- 172.21.15.45:0/3682196687 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fbe000ce730 con 0x5617caddde90 2024-04-16T17:08:04.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.568+0000 7fbe1a019740 10 client.26038 fetch_fsmap learned FSMap version 25 2024-04-16T17:08:04.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.568+0000 7fbe1a019740 1 -- 172.21.15.45:0/3682196687 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5617cae66ce0 con 0x5617caddde90 2024-04-16T17:08:04.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.568+0000 7fbdfa000640 1 -- 172.21.15.45:0/3682196687 <== mon.1 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbe000ce390 con 0x5617caddde90 2024-04-16T17:08:04.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.569+0000 7fbdfa000640 1 -- 172.21.15.45:0/3682196687 <== mon.1 v2:172.21.15.117:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fbe00098920 con 0x5617caddde90 2024-04-16T17:08:04.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.569+0000 7fbe1a019740 10 client.26038 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T17:08:04.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.569+0000 7fbe1a019740 10 client.26038 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:08:04.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.569+0000 7fbe1a019740 20 client.26038 populate_metadata read hostname 'smithi045' 2024-04-16T17:08:04.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.569+0000 7fbe1a019740 1 --2- 172.21.15.45:0/3682196687 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x5617cae67110 0x5617cae69500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:04.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.569+0000 7fbe1a019740 4 client.26038 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-16T17:08:04.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.569+0000 7fbe1a019740 1 -- 172.21.15.45:0/3682196687 --> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5617cad4a540 con 0x5617cae67110 2024-04-16T17:08:04.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.569+0000 7fbe0f400640 1 --2- 172.21.15.45:0/3682196687 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x5617cae67110 0x5617cae69500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:04.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.570+0000 7fbe0f400640 1 --2- 172.21.15.45:0/3682196687 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x5617cae67110 0x5617cae69500 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=312572ccde92cbf8 server_cookie=154c4d0c337c029d in_seq=0 out_seq=0 2024-04-16T17:08:04.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.570+0000 7fbdfa000640 10 client.26038 ms_handle_connect on v2:172.21.15.104:6838/3660070592 2024-04-16T17:08:04.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.571+0000 7fbdfa000640 1 -- 172.21.15.45:0/3682196687 <== mds.0 v2:172.21.15.104:6838/3660070592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5617cad4a540 con 0x5617cae67110 2024-04-16T17:08:04.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.571+0000 7fbdfa000640 10 client.26038 handle_command_reply: tid=0 2024-04-16T17:08:04.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.571+0000 7fbe1a019740 1 client.26038 shutdown 2024-04-16T17:08:04.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.571+0000 7fbe1a019740 1 -- 172.21.15.45:0/3682196687 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5617caddde90 msgr2=0x5617cade18f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:04.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.571+0000 7fbe1a019740 1 --2- 172.21.15.45:0/3682196687 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5617caddde90 0x5617cade18f0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fbe000a5310 tx=0x7fbe0000e9b0 comp rx=0 tx=0).stop 2024-04-16T17:08:04.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.571+0000 7fbe1a019740 1 -- 172.21.15.45:0/3682196687 shutdown_connections 2024-04-16T17:08:04.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.571+0000 7fbe1a019740 1 --2- 172.21.15.45:0/3682196687 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5617cade7390 0x5617cae65560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:04.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.571+0000 7fbe1a019740 1 --2- 172.21.15.45:0/3682196687 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5617cadde7b0 0x5617cade1e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:04.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.571+0000 7fbe1a019740 1 --2- 172.21.15.45:0/3682196687 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x5617cae67110 0x5617cae69500 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:04.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.571+0000 7fbe1a019740 1 --2- 172.21.15.45:0/3682196687 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5617caddde90 0x5617cade18f0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:04.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.571+0000 7fbe1a019740 1 -- 172.21.15.45:0/3682196687 >> 172.21.15.45:0/3682196687 conn(0x5617cadd85c0 msgr2=0x5617cade4480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:08:04.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.571+0000 7fbdfa000640 0 client.26038 ms_handle_reset on v2:172.21.15.104:6838/3660070592 2024-04-16T17:08:04.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.571+0000 7fbe1a019740 1 -- 172.21.15.45:0/3682196687 shutdown_connections 2024-04-16T17:08:04.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.571+0000 7fbe1a019740 1 -- 172.21.15.45:0/3682196687 wait complete. 2024-04-16T17:08:04.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.572+0000 7fbe1a019740 20 client.26038 trim_cache size 0 max 16384 2024-04-16T17:08:04.574 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-16T17:08:04.574 INFO:teuthology.orchestra.run.smithi045.stdout: "return_code": 0, 2024-04-16T17:08:04.575 INFO:teuthology.orchestra.run.smithi045.stdout: "scrub_tag": "487544a3-4979-4ac9-b77c-ca618a488309", 2024-04-16T17:08:04.575 INFO:teuthology.orchestra.run.smithi045.stdout: "mode": "asynchronous" 2024-04-16T17:08:04.575 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-16T17:08:04.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.572+0000 7fbe15c00640 1 -- 172.21.15.45:0/190633189 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fbdf40a3dd0 msgr2=0x7fbdf40a6290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:04.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.572+0000 7fbe15c00640 1 --2- 172.21.15.45:0/190633189 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fbdf40a3dd0 0x7fbdf40a6290 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fbe0802c640 tx=0x7fbe08005df0 comp rx=0 tx=0).stop 2024-04-16T17:08:04.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.572+0000 7fbe15c00640 1 -- 172.21.15.45:0/190633189 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbe10102110 msgr2=0x7fbe101a0660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:04.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.572+0000 7fbe15c00640 1 --2- 172.21.15.45:0/190633189 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbe10102110 0x7fbe101a0660 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fbe0000ea60 tx=0x7fbe0000ef20 comp rx=0 tx=0).stop 2024-04-16T17:08:04.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.573+0000 7fbe15c00640 1 -- 172.21.15.45:0/190633189 shutdown_connections 2024-04-16T17:08:04.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.573+0000 7fbe15c00640 1 --2- 172.21.15.45:0/190633189 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fbdf40a3dd0 0x7fbdf40a6290 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:04.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.573+0000 7fbe15c00640 1 --2- 172.21.15.45:0/190633189 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbe1019a820 0x7fbe1019acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:04.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.573+0000 7fbe15c00640 1 --2- 172.21.15.45:0/190633189 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbe101071e0 0x7fbe101a0ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:04.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.573+0000 7fbe15c00640 1 --2- 172.21.15.45:0/190633189 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbe10102110 0x7fbe101a0660 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:04.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.573+0000 7fbe15c00640 1 -- 172.21.15.45:0/190633189 >> 172.21.15.45:0/190633189 conn(0x7fbe100fd9d0 msgr2=0x7fbe100fed10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:08:04.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.573+0000 7fbe15c00640 1 -- 172.21.15.45:0/190633189 shutdown_connections 2024-04-16T17:08:04.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.573+0000 7fbe15c00640 1 -- 172.21.15.45:0/190633189 wait complete. 2024-04-16T17:08:04.592 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-16T17:08:04.592 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-16T17:08:04.677 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:08:04 smithi104 ceph-mon[36457]: pgmap v42: 129 pgs: 129 active+clean; 2.2 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 111 KiB/s wr, 31 op/s 2024-04-16T17:08:04.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.771+0000 7f1004600640 1 -- 172.21.15.45:0/488529821 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0ffc1044b0 msgr2=0x7f0ffc104890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:04.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.771+0000 7f1004600640 1 --2- 172.21.15.45:0/488529821 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0ffc1044b0 0x7f0ffc104890 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f0ff0009900 tx=0x7f0ff002bc30 comp rx=0 tx=0).stop 2024-04-16T17:08:04.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.771+0000 7f1004600640 1 -- 172.21.15.45:0/488529821 shutdown_connections 2024-04-16T17:08:04.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.771+0000 7f1004600640 1 --2- 172.21.15.45:0/488529821 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0ffc104e60 0x7f0ffc108ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:04.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.771+0000 7f1004600640 1 --2- 172.21.15.45:0/488529821 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0ffc109580 0x7f0ffc111150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:04.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.771+0000 7f1004600640 1 --2- 172.21.15.45:0/488529821 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0ffc1044b0 0x7f0ffc104890 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:04.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.771+0000 7f1004600640 1 -- 172.21.15.45:0/488529821 >> 172.21.15.45:0/488529821 conn(0x7f0ffc0ffd90 msgr2=0x7f0ffc1021b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:08:04.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.772+0000 7f1004600640 1 -- 172.21.15.45:0/488529821 shutdown_connections 2024-04-16T17:08:04.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.772+0000 7f1004600640 1 -- 172.21.15.45:0/488529821 wait complete. 2024-04-16T17:08:04.774 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.772+0000 7f1004600640 1 Processor -- start 2024-04-16T17:08:04.775 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.773+0000 7f1004600640 1 -- start start 2024-04-16T17:08:04.775 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.773+0000 7f1004600640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0ffc1044b0 0x7f0ffc1a07d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:04.775 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.773+0000 7f1004600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0ffc109580 0x7f0ffc1a0d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:04.775 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.773+0000 7f1004600640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0ffc19a980 0x7f0ffc19ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:04.775 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.773+0000 7f1004600640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f0ffc114660 con 0x7f0ffc109580 2024-04-16T17:08:04.775 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.773+0000 7f1004600640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f0ffc1144e0 con 0x7f0ffc1044b0 2024-04-16T17:08:04.775 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.773+0000 7f1004600640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f0ffc1147e0 con 0x7f0ffc19a980 2024-04-16T17:08:04.775 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.773+0000 7f1001c00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0ffc1044b0 0x7f0ffc1a07d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:04.775 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.773+0000 7f1001c00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0ffc1044b0 0x7f0ffc1a07d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:47740/0 (socket says 172.21.15.45:47740) 2024-04-16T17:08:04.775 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.773+0000 7f1001c00640 1 -- 172.21.15.45:0/568986481 learned_addr learned my addr 172.21.15.45:0/568986481 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:08:04.775 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.773+0000 7f1001200640 1 --2- 172.21.15.45:0/568986481 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0ffc109580 0x7f0ffc1a0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:04.775 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.773+0000 7f1002600640 1 --2- 172.21.15.45:0/568986481 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0ffc19a980 0x7f0ffc19ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:04.775 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.774+0000 7f1001c00640 1 -- 172.21.15.45:0/568986481 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0ffc19a980 msgr2=0x7f0ffc19ae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:04.775 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.774+0000 7f1001c00640 1 --2- 172.21.15.45:0/568986481 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0ffc19a980 0x7f0ffc19ae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:04.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.774+0000 7f1001c00640 1 -- 172.21.15.45:0/568986481 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0ffc109580 msgr2=0x7f0ffc1a0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:04.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.774+0000 7f1001c00640 1 --2- 172.21.15.45:0/568986481 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0ffc109580 0x7f0ffc1a0d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:04.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.774+0000 7f1001c00640 1 -- 172.21.15.45:0/568986481 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ffc19b4a0 con 0x7f0ffc1044b0 2024-04-16T17:08:04.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.774+0000 7f1001c00640 1 --2- 172.21.15.45:0/568986481 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0ffc1044b0 0x7f0ffc1a07d0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f0ff002c240 tx=0x7f0ff0004390 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:08:04.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.774+0000 7f0ffa000640 1 -- 172.21.15.45:0/568986481 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ff0043020 con 0x7f0ffc1044b0 2024-04-16T17:08:04.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.774+0000 7f0ffa000640 1 -- 172.21.15.45:0/568986481 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0ff00054e0 con 0x7f0ffc1044b0 2024-04-16T17:08:04.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.774+0000 7f1004600640 1 -- 172.21.15.45:0/568986481 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ffc19b730 con 0x7f0ffc1044b0 2024-04-16T17:08:04.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.774+0000 7f0ffa000640 1 -- 172.21.15.45:0/568986481 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ff0003c00 con 0x7f0ffc1044b0 2024-04-16T17:08:04.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.775+0000 7f1004600640 1 -- 172.21.15.45:0/568986481 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ffc1a79d0 con 0x7f0ffc1044b0 2024-04-16T17:08:04.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.778+0000 7f100741e740 1 Processor -- start 2024-04-16T17:08:04.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.778+0000 7f0ffa000640 1 -- 172.21.15.45:0/568986481 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 25) ==== 138879+0+0 (secure 0 0 0) 0x7f0ff0046020 con 0x7f0ffc1044b0 2024-04-16T17:08:04.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.778+0000 7f100741e740 1 -- start start 2024-04-16T17:08:04.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.778+0000 7f100741e740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56182a3951b0 0x56182a395590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:04.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.778+0000 7f100741e740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56182a395ad0 0x56182a39e170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:04.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.778+0000 7f100741e740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56182a39e6b0 0x56182a3a0aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:04.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.778+0000 7f100741e740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x56182a2fb120 con 0x56182a39e6b0 2024-04-16T17:08:04.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.778+0000 7f100741e740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x56182a2a1340 con 0x56182a395ad0 2024-04-16T17:08:04.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.778+0000 7f100741e740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x56182a2f70e0 con 0x56182a3951b0 2024-04-16T17:08:04.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.779+0000 7f1001c00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56182a395ad0 0x56182a39e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:04.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.779+0000 7f1001200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56182a39e6b0 0x56182a3a0aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:04.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.779+0000 7f1001c00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56182a395ad0 0x56182a39e170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:47752/0 (socket says 172.21.15.45:47752) 2024-04-16T17:08:04.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.779+0000 7f1001c00640 1 -- 172.21.15.45:0/1654339661 learned_addr learned my addr 172.21.15.45:0/1654339661 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:08:04.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.779+0000 7f1002600640 1 --2- 172.21.15.45:0/1654339661 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56182a3951b0 0x56182a395590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:04.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.779+0000 7f1001200640 1 -- 172.21.15.45:0/1654339661 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56182a3951b0 msgr2=0x56182a395590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:08:04.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.779+0000 7f1001200640 1 --2- 172.21.15.45:0/1654339661 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56182a3951b0 0x56182a395590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:04.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.779+0000 7f1001200640 1 -- 172.21.15.45:0/1654339661 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56182a395ad0 msgr2=0x56182a39e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:08:04.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.779+0000 7f1001200640 1 --2- 172.21.15.45:0/1654339661 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56182a395ad0 0x56182a39e170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:04.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.779+0000 7f1001200640 1 -- 172.21.15.45:0/1654339661 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56182a0d77d0 con 0x56182a39e6b0 2024-04-16T17:08:04.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.779+0000 7f1001c00640 1 --2- 172.21.15.45:0/1654339661 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56182a395ad0 0x56182a39e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:08:04.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.780+0000 7f1001200640 1 --2- 172.21.15.45:0/1654339661 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56182a39e6b0 0x56182a3a0aa0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f0ffc19c170 tx=0x7f0fe80107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d5a6e0da2760e58 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:08:04.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.780+0000 7f0ffa000640 1 --2- 172.21.15.45:0/568986481 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f0fe40a3f10 0x7f0fe40a63d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:04.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.780+0000 7f0feea00640 1 -- 172.21.15.45:0/1654339661 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fe8004030 con 0x56182a39e6b0 2024-04-16T17:08:04.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.780+0000 7f0feea00640 1 -- 172.21.15.45:0/1654339661 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0fe80041d0 con 0x56182a39e6b0 2024-04-16T17:08:04.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.780+0000 7f0feea00640 1 -- 172.21.15.45:0/1654339661 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fe80116a0 con 0x56182a39e6b0 2024-04-16T17:08:04.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.780+0000 7f0ffa000640 1 -- 172.21.15.45:0/568986481 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f0ff00f2e40 con 0x7f0ffc1044b0 2024-04-16T17:08:04.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.781+0000 7f1002600640 1 --2- 172.21.15.45:0/568986481 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f0fe40a3f10 0x7f0fe40a63d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:04.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.781+0000 7f100741e740 1 -- 172.21.15.45:0/1654339661 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56182a39e6b0 msgr2=0x56182a3a0aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:04.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.781+0000 7f100741e740 1 --2- 172.21.15.45:0/1654339661 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56182a39e6b0 0x56182a3a0aa0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f0ffc19c170 tx=0x7f0fe80107a0 comp rx=0 tx=0).stop 2024-04-16T17:08:04.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.781+0000 7f100741e740 1 -- 172.21.15.45:0/1654339661 shutdown_connections 2024-04-16T17:08:04.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.781+0000 7f100741e740 1 --2- 172.21.15.45:0/1654339661 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56182a39e6b0 0x56182a3a0aa0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:04.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.781+0000 7f100741e740 1 --2- 172.21.15.45:0/1654339661 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56182a395ad0 0x56182a39e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:04.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.781+0000 7f100741e740 1 --2- 172.21.15.45:0/1654339661 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56182a3951b0 0x56182a395590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:04.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.781+0000 7f100741e740 1 -- 172.21.15.45:0/1654339661 >> 172.21.15.45:0/1654339661 conn(0x56182a38f8e0 msgr2=0x56182a38fcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:08:04.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.781+0000 7f100741e740 1 -- 172.21.15.45:0/1654339661 shutdown_connections 2024-04-16T17:08:04.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.781+0000 7f100741e740 1 -- 172.21.15.45:0/1654339661 wait complete. 2024-04-16T17:08:04.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.781+0000 7f1002600640 1 --2- 172.21.15.45:0/568986481 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f0fe40a3f10 0x7f0fe40a63d0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f0ff4005fd0 tx=0x7f0ff4009600 comp rx=0 tx=0).ready entity=mgr.25411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:08:04.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.782+0000 7f100741e740 1 -- start start 2024-04-16T17:08:04.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.782+0000 7f100741e740 1 Processor -- start 2024-04-16T17:08:04.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.782+0000 7f100741e740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56182a3951b0 0x56182a39b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:04.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.782+0000 7f100741e740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56182a395ad0 0x56182a39bc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:04.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.782+0000 7f100741e740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56182a39e6b0 0x56182a39c170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:04.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.782+0000 7f100741e740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x56182a3a14a0 con 0x56182a39e6b0 2024-04-16T17:08:04.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.782+0000 7f100741e740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x56182a2b4710 con 0x56182a395ad0 2024-04-16T17:08:04.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.782+0000 7f100741e740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x56182a3a1620 con 0x56182a3951b0 2024-04-16T17:08:04.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.782+0000 7f100741e740 10 client.0 _ll_register_callbacks cb 0x56182a2afaf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:08:04.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.783+0000 7f1001200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56182a395ad0 0x56182a39bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:04.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.783+0000 7f1002600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56182a39e6b0 0x56182a39c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:04.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.783+0000 7f1001200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56182a395ad0 0x56182a39bc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:47766/0 (socket says 172.21.15.45:47766) 2024-04-16T17:08:04.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.783+0000 7f1001200640 1 -- 172.21.15.45:0/435904576 learned_addr learned my addr 172.21.15.45:0/435904576 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:08:04.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.783+0000 7f1001c00640 1 --2- 172.21.15.45:0/435904576 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56182a3951b0 0x56182a39b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:04.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.783+0000 7f1001200640 1 -- 172.21.15.45:0/435904576 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56182a3951b0 msgr2=0x56182a39b6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:08:04.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.783+0000 7f1001200640 1 --2- 172.21.15.45:0/435904576 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56182a3951b0 0x56182a39b6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:04.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.783+0000 7f1001200640 1 -- 172.21.15.45:0/435904576 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56182a39e6b0 msgr2=0x56182a39c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:08:04.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.783+0000 7f1001200640 1 --2- 172.21.15.45:0/435904576 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56182a39e6b0 0x56182a39c170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:04.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.783+0000 7f1001200640 1 -- 172.21.15.45:0/435904576 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56182a00d3f0 con 0x56182a395ad0 2024-04-16T17:08:04.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.783+0000 7f1002600640 1 --2- 172.21.15.45:0/435904576 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56182a39e6b0 0x56182a39c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:08:04.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.783+0000 7f1001200640 1 --2- 172.21.15.45:0/435904576 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56182a395ad0 0x56182a39bc30 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f0fe8010fe0 tx=0x7f0fe8003ef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7ac1ba349e4afa3 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:08:04.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.783+0000 7f0fe3400640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-04-16T17:08:04.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.784+0000 7f0fe3400640 1 -- 172.21.15.45:0/435904576 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fe802a060 con 0x56182a395ad0 2024-04-16T17:08:04.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.784+0000 7f100741e740 1 -- 172.21.15.45:0/435904576 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0fe80041d0 con 0x56182a395ad0 2024-04-16T17:08:04.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.784+0000 7f0fe3400640 1 -- 172.21.15.45:0/435904576 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f0fe80035f0 con 0x56182a395ad0 2024-04-16T17:08:04.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.784+0000 7f0fe3400640 1 -- 172.21.15.45:0/435904576 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f0fe8031050 con 0x56182a395ad0 2024-04-16T17:08:04.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.784+0000 7f100741e740 10 client.26063 fetch_fsmap learned FSMap version 25 2024-04-16T17:08:04.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.784+0000 7f100741e740 1 -- 172.21.15.45:0/435904576 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x561829fcd0f0 con 0x56182a395ad0 2024-04-16T17:08:04.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.784+0000 7f0fe3400640 1 -- 172.21.15.45:0/435904576 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0fe80041d0 con 0x56182a395ad0 2024-04-16T17:08:04.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.785+0000 7f0fe3400640 1 -- 172.21.15.45:0/435904576 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f0fe80046c0 con 0x56182a395ad0 2024-04-16T17:08:04.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.785+0000 7f100741e740 10 client.26063 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T17:08:04.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.785+0000 7f100741e740 10 client.26063 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:08:04.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.785+0000 7f100741e740 20 client.26063 populate_metadata read hostname 'smithi045' 2024-04-16T17:08:04.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.785+0000 7f100741e740 1 --2- 172.21.15.45:0/435904576 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x56182a3ae9f0 0x56182a3b0de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:04.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.785+0000 7f100741e740 4 client.26063 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:08:04.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.785+0000 7f100741e740 1 -- 172.21.15.45:0/435904576 --> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56182a308b40 con 0x56182a3ae9f0 2024-04-16T17:08:04.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.785+0000 7f1001c00640 1 --2- 172.21.15.45:0/435904576 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x56182a3ae9f0 0x56182a3b0de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:04.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.785+0000 7f1001c00640 1 --2- 172.21.15.45:0/435904576 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x56182a3ae9f0 0x56182a3b0de0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b1262141380ca2b0 server_cookie=55d4d5a4b8da8f56 in_seq=0 out_seq=0 2024-04-16T17:08:04.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.785+0000 7f0fe3400640 10 client.26063 ms_handle_connect on v2:172.21.15.104:6838/3660070592 2024-04-16T17:08:04.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.786+0000 7f0fe3400640 1 -- 172.21.15.45:0/435904576 <== mds.0 v2:172.21.15.104:6838/3660070592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56182a308b40 con 0x56182a3ae9f0 2024-04-16T17:08:04.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.786+0000 7f0fe3400640 10 client.26063 handle_command_reply: tid=0 2024-04-16T17:08:04.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.786+0000 7f100741e740 1 client.26063 shutdown 2024-04-16T17:08:04.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.786+0000 7f100741e740 1 -- 172.21.15.45:0/435904576 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56182a395ad0 msgr2=0x56182a39bc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:04.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.786+0000 7f100741e740 1 --2- 172.21.15.45:0/435904576 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56182a395ad0 0x56182a39bc30 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f0fe8010fe0 tx=0x7f0fe8003ef0 comp rx=0 tx=0).stop 2024-04-16T17:08:04.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.786+0000 7f100741e740 1 -- 172.21.15.45:0/435904576 shutdown_connections 2024-04-16T17:08:04.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.786+0000 7f100741e740 1 --2- 172.21.15.45:0/435904576 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56182a39e6b0 0x56182a39c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:04.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.786+0000 7f100741e740 1 --2- 172.21.15.45:0/435904576 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56182a395ad0 0x56182a39bc30 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:04.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.786+0000 7f100741e740 1 --2- 172.21.15.45:0/435904576 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x56182a3ae9f0 0x56182a3b0de0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:04.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.786+0000 7f100741e740 1 --2- 172.21.15.45:0/435904576 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56182a3951b0 0x56182a39b6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:04.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.786+0000 7f100741e740 1 -- 172.21.15.45:0/435904576 >> 172.21.15.45:0/435904576 conn(0x56182a38f8e0 msgr2=0x56182a391630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:08:04.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.787+0000 7f0fe3400640 0 client.26063 ms_handle_reset on v2:172.21.15.104:6838/3660070592 2024-04-16T17:08:04.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.787+0000 7f100741e740 1 -- 172.21.15.45:0/435904576 shutdown_connections 2024-04-16T17:08:04.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.787+0000 7f100741e740 1 -- 172.21.15.45:0/435904576 wait complete. 2024-04-16T17:08:04.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.787+0000 7f100741e740 20 client.26063 trim_cache size 0 max 16384 2024-04-16T17:08:04.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:08:04 smithi117 ceph-mon[36730]: pgmap v42: 129 pgs: 129 active+clean; 2.2 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 111 KiB/s wr, 31 op/s 2024-04-16T17:08:04.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.806+0000 7f100741e740 1 Processor -- start 2024-04-16T17:08:04.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.806+0000 7f100741e740 1 -- start start 2024-04-16T17:08:04.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.806+0000 7f100741e740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56182a3951b0 0x56182a39de20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:04.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.806+0000 7f100741e740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56182a395ad0 0x56182a39a630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:04.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.806+0000 7f100741e740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56182a39e6b0 0x56182a3aa3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:04.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.806+0000 7f100741e740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f0ffc1b3ec0 con 0x56182a395ad0 2024-04-16T17:08:04.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.806+0000 7f100741e740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f0ffc1b1790 con 0x56182a39e6b0 2024-04-16T17:08:04.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.806+0000 7f100741e740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f0ffc1af5c0 con 0x56182a3951b0 2024-04-16T17:08:04.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.806+0000 7f1001200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56182a395ad0 0x56182a39a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:04.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.807+0000 7f1001200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56182a395ad0 0x56182a39a630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35324/0 (socket says 172.21.15.45:35324) 2024-04-16T17:08:04.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.807+0000 7f1001200640 1 -- 172.21.15.45:0/2681853218 learned_addr learned my addr 172.21.15.45:0/2681853218 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:08:04.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.807+0000 7f1002600640 1 --2- 172.21.15.45:0/2681853218 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56182a39e6b0 0x56182a3aa3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:04.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.807+0000 7f1001c00640 1 --2- 172.21.15.45:0/2681853218 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56182a3951b0 0x56182a39de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:04.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.807+0000 7f1001200640 1 -- 172.21.15.45:0/2681853218 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56182a3951b0 msgr2=0x56182a39de20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:08:04.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.807+0000 7f1001200640 1 --2- 172.21.15.45:0/2681853218 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56182a3951b0 0x56182a39de20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:04.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.807+0000 7f1001200640 1 -- 172.21.15.45:0/2681853218 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56182a39e6b0 msgr2=0x56182a3aa3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:08:04.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.807+0000 7f1001200640 1 --2- 172.21.15.45:0/2681853218 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56182a39e6b0 0x56182a3aa3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:04.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.807+0000 7f1001200640 1 -- 172.21.15.45:0/2681853218 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56182a3a0aa0 con 0x56182a395ad0 2024-04-16T17:08:04.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.807+0000 7f1001200640 1 --2- 172.21.15.45:0/2681853218 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56182a395ad0 0x56182a39a630 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f0fe80107e0 tx=0x7f0fe8001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d781e1ef422c4b47 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:08:04.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.808+0000 7f0fee000640 1 -- 172.21.15.45:0/2681853218 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fe802a070 con 0x56182a395ad0 2024-04-16T17:08:04.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.808+0000 7f0fee000640 1 -- 172.21.15.45:0/2681853218 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0fe8005bf0 con 0x56182a395ad0 2024-04-16T17:08:04.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.808+0000 7f0fee000640 1 -- 172.21.15.45:0/2681853218 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fe8003f80 con 0x56182a395ad0 2024-04-16T17:08:04.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.809+0000 7f100741e740 1 -- 172.21.15.45:0/2681853218 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56182a395ad0 msgr2=0x56182a39a630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:04.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.809+0000 7f100741e740 1 --2- 172.21.15.45:0/2681853218 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56182a395ad0 0x56182a39a630 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f0fe80107e0 tx=0x7f0fe8001e50 comp rx=0 tx=0).stop 2024-04-16T17:08:04.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.809+0000 7f100741e740 1 -- 172.21.15.45:0/2681853218 shutdown_connections 2024-04-16T17:08:04.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.809+0000 7f100741e740 1 --2- 172.21.15.45:0/2681853218 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56182a39e6b0 0x56182a3aa3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:04.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.809+0000 7f100741e740 1 --2- 172.21.15.45:0/2681853218 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56182a395ad0 0x56182a39a630 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:04.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.809+0000 7f100741e740 1 --2- 172.21.15.45:0/2681853218 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56182a3951b0 0x56182a39de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:04.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.809+0000 7f100741e740 1 -- 172.21.15.45:0/2681853218 >> 172.21.15.45:0/2681853218 conn(0x56182a38f8e0 msgr2=0x56182a38e8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:08:04.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.809+0000 7f100741e740 1 -- 172.21.15.45:0/2681853218 shutdown_connections 2024-04-16T17:08:04.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.809+0000 7f100741e740 1 -- 172.21.15.45:0/2681853218 wait complete. 2024-04-16T17:08:04.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.810+0000 7f100741e740 1 -- start start 2024-04-16T17:08:04.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.810+0000 7f100741e740 1 Processor -- start 2024-04-16T17:08:04.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.810+0000 7f100741e740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56182a3951b0 0x56182a3a6bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:04.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.810+0000 7f100741e740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56182a395ad0 0x56182a3a1f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:04.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.810+0000 7f100741e740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56182a39e6b0 0x56182a3a2470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:04.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.810+0000 7f100741e740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f0ffc1afc50 con 0x56182a3951b0 2024-04-16T17:08:04.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.810+0000 7f100741e740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f0ffc1b10c0 con 0x56182a395ad0 2024-04-16T17:08:04.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.810+0000 7f100741e740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f0ffc1b0320 con 0x56182a39e6b0 2024-04-16T17:08:04.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.810+0000 7f100741e740 10 client.0 _ll_register_callbacks cb 0x7f0ff00f67a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:08:04.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.811+0000 7f1001c00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56182a3951b0 0x56182a3a6bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:04.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.811+0000 7f1001200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56182a395ad0 0x56182a3a1f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:04.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.811+0000 7f1001200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56182a395ad0 0x56182a3a1f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:47784/0 (socket says 172.21.15.45:47784) 2024-04-16T17:08:04.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.811+0000 7f1001200640 1 -- 172.21.15.45:0/1764875311 learned_addr learned my addr 172.21.15.45:0/1764875311 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:08:04.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.811+0000 7f1002600640 1 --2- 172.21.15.45:0/1764875311 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56182a39e6b0 0x56182a3a2470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:04.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.811+0000 7f1001200640 1 -- 172.21.15.45:0/1764875311 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56182a39e6b0 msgr2=0x56182a3a2470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:08:04.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.811+0000 7f1001200640 1 --2- 172.21.15.45:0/1764875311 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56182a39e6b0 0x56182a3a2470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:04.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.811+0000 7f1001200640 1 -- 172.21.15.45:0/1764875311 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56182a3951b0 msgr2=0x56182a3a6bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:08:04.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.811+0000 7f1001200640 1 --2- 172.21.15.45:0/1764875311 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56182a3951b0 0x56182a3a6bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:04.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.811+0000 7f1001200640 1 -- 172.21.15.45:0/1764875311 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56182a3a2bb0 con 0x56182a395ad0 2024-04-16T17:08:04.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.811+0000 7f1001200640 1 --2- 172.21.15.45:0/1764875311 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56182a395ad0 0x56182a3a1f30 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f0fe8004ac0 tx=0x7f0fe8010d90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f0cdafdadc75af72 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:08:04.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.811+0000 7f0fe3400640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-04-16T17:08:04.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.812+0000 7f0fe3400640 1 -- 172.21.15.45:0/1764875311 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fe8042060 con 0x56182a395ad0 2024-04-16T17:08:04.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.812+0000 7f0fe3400640 1 -- 172.21.15.45:0/1764875311 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f0fe8003800 con 0x56182a395ad0 2024-04-16T17:08:04.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.812+0000 7f100741e740 1 -- 172.21.15.45:0/1764875311 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56182a3029b0 con 0x56182a395ad0 2024-04-16T17:08:04.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.812+0000 7f0fe3400640 1 -- 172.21.15.45:0/1764875311 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f0fe8049050 con 0x56182a395ad0 2024-04-16T17:08:04.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.812+0000 7f100741e740 10 client.26069 fetch_fsmap learned FSMap version 25 2024-04-16T17:08:04.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.812+0000 7f100741e740 1 -- 172.21.15.45:0/1764875311 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56182a414bf0 con 0x56182a395ad0 2024-04-16T17:08:04.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.812+0000 7f0fe3400640 1 -- 172.21.15.45:0/1764875311 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x56182a3029b0 con 0x56182a395ad0 2024-04-16T17:08:04.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.812+0000 7f0fe3400640 1 -- 172.21.15.45:0/1764875311 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f0fe8003a40 con 0x56182a395ad0 2024-04-16T17:08:04.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.812+0000 7f100741e740 10 client.26069 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T17:08:04.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.812+0000 7f100741e740 10 client.26069 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:08:04.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.812+0000 7f100741e740 20 client.26069 populate_metadata read hostname 'smithi045' 2024-04-16T17:08:04.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.813+0000 7f100741e740 1 --2- 172.21.15.45:0/1764875311 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x56182a415290 0x56182a4176b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:04.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.813+0000 7f100741e740 4 client.26069 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-16T17:08:04.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.813+0000 7f100741e740 1 -- 172.21.15.45:0/1764875311 --> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56182a27e420 con 0x56182a415290 2024-04-16T17:08:04.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.813+0000 7f1001c00640 1 --2- 172.21.15.45:0/1764875311 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x56182a415290 0x56182a4176b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:04.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.813+0000 7f1001c00640 1 --2- 172.21.15.45:0/1764875311 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x56182a415290 0x56182a4176b0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=17eb8e3573dcd627 server_cookie=9c3118f8c5fb0ebc in_seq=0 out_seq=0 2024-04-16T17:08:04.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.813+0000 7f0fe3400640 10 client.26069 ms_handle_connect on v2:172.21.15.104:6838/3660070592 2024-04-16T17:08:04.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.814+0000 7f0fe3400640 1 -- 172.21.15.45:0/1764875311 <== mds.0 v2:172.21.15.104:6838/3660070592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x56182a27e420 con 0x56182a415290 2024-04-16T17:08:04.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.814+0000 7f0fe3400640 10 client.26069 handle_command_reply: tid=0 2024-04-16T17:08:04.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.814+0000 7f100741e740 1 client.26069 shutdown 2024-04-16T17:08:04.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.814+0000 7f100741e740 1 -- 172.21.15.45:0/1764875311 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56182a395ad0 msgr2=0x56182a3a1f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:04.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.814+0000 7f100741e740 1 --2- 172.21.15.45:0/1764875311 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56182a395ad0 0x56182a3a1f30 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f0fe8004ac0 tx=0x7f0fe8010d90 comp rx=0 tx=0).stop 2024-04-16T17:08:04.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.814+0000 7f100741e740 1 -- 172.21.15.45:0/1764875311 shutdown_connections 2024-04-16T17:08:04.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.814+0000 7f100741e740 1 --2- 172.21.15.45:0/1764875311 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x56182a415290 0x56182a4176b0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:04.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.814+0000 7f100741e740 1 --2- 172.21.15.45:0/1764875311 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56182a39e6b0 0x56182a3a2470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:04.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.814+0000 7f0fe3400640 0 client.26069 ms_handle_reset on v2:172.21.15.104:6838/3660070592 2024-04-16T17:08:04.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.814+0000 7f100741e740 1 --2- 172.21.15.45:0/1764875311 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56182a395ad0 0x56182a3a1f30 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:04.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.814+0000 7f100741e740 1 --2- 172.21.15.45:0/1764875311 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56182a3951b0 0x56182a3a6bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:04.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.814+0000 7f100741e740 1 -- 172.21.15.45:0/1764875311 >> 172.21.15.45:0/1764875311 conn(0x56182a38f8e0 msgr2=0x56182a39b4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:08:04.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.814+0000 7f100741e740 1 -- 172.21.15.45:0/1764875311 shutdown_connections 2024-04-16T17:08:04.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.814+0000 7f100741e740 1 -- 172.21.15.45:0/1764875311 wait complete. 2024-04-16T17:08:04.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.814+0000 7f100741e740 20 client.26069 trim_cache size 0 max 16384 2024-04-16T17:08:04.818 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-16T17:08:04.818 INFO:teuthology.orchestra.run.smithi045.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-16T17:08:04.818 INFO:teuthology.orchestra.run.smithi045.stdout: "scrubs": { 2024-04-16T17:08:04.818 INFO:teuthology.orchestra.run.smithi045.stdout: "487544a3-4979-4ac9-b77c-ca618a488309": { 2024-04-16T17:08:04.818 INFO:teuthology.orchestra.run.smithi045.stdout: "path": "/", 2024-04-16T17:08:04.818 INFO:teuthology.orchestra.run.smithi045.stdout: "tag": "487544a3-4979-4ac9-b77c-ca618a488309", 2024-04-16T17:08:04.818 INFO:teuthology.orchestra.run.smithi045.stdout: "options": "recursive,force" 2024-04-16T17:08:04.818 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:08:04.819 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:08:04.819 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-16T17:08:04.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.815+0000 7f1004600640 1 -- 172.21.15.45:0/568986481 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f0fe40a3f10 msgr2=0x7f0fe40a63d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:04.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.815+0000 7f1004600640 1 --2- 172.21.15.45:0/568986481 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f0fe40a3f10 0x7f0fe40a63d0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f0ff4005fd0 tx=0x7f0ff4009600 comp rx=0 tx=0).stop 2024-04-16T17:08:04.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.815+0000 7f1004600640 1 -- 172.21.15.45:0/568986481 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0ffc1044b0 msgr2=0x7f0ffc1a07d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:04.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.815+0000 7f1004600640 1 --2- 172.21.15.45:0/568986481 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0ffc1044b0 0x7f0ffc1a07d0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f0ff002c240 tx=0x7f0ff0004390 comp rx=0 tx=0).stop 2024-04-16T17:08:04.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.815+0000 7f1004600640 1 -- 172.21.15.45:0/568986481 shutdown_connections 2024-04-16T17:08:04.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.815+0000 7f1004600640 1 --2- 172.21.15.45:0/568986481 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f0fe40a3f10 0x7f0fe40a63d0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:04.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.815+0000 7f1004600640 1 --2- 172.21.15.45:0/568986481 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0ffc19a980 0x7f0ffc19ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:04.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.815+0000 7f1004600640 1 --2- 172.21.15.45:0/568986481 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0ffc109580 0x7f0ffc1a0d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:04.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.815+0000 7f1004600640 1 --2- 172.21.15.45:0/568986481 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0ffc1044b0 0x7f0ffc1a07d0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:04.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.815+0000 7f1004600640 1 -- 172.21.15.45:0/568986481 >> 172.21.15.45:0/568986481 conn(0x7f0ffc0ffd90 msgr2=0x7f0ffc1020f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:08:04.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.815+0000 7f1004600640 1 -- 172.21.15.45:0/568986481 shutdown_connections 2024-04-16T17:08:04.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:04.815+0000 7f1004600640 1 -- 172.21.15.45:0/568986481 wait complete. 2024-04-16T17:08:04.836 INFO:tasks.cephfs.filesystem:scrub status for tag:487544a3-4979-4ac9-b77c-ca618a488309 - {'path': '/', 'tag': '487544a3-4979-4ac9-b77c-ca618a488309', 'options': 'recursive,force'} 2024-04-16T17:08:05.203 INFO:tasks.workunit.client.0.smithi045.stdout: 1 4292 5.77 MB/sec warmup 7 sec latency 52.688 ms 2024-04-16T17:08:05.989 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:08:02 smithi045 ceph-mon[31676]: pgmap v41: 129 pgs: 129 active+clean; 2.2 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 111 KiB/s wr, 34 op/s 2024-04-16T17:08:05.990 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:08:03 smithi045 ceph-mon[31676]: from='client.? 172.21.15.45:0/2580881476' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:08:05.990 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:08:04 smithi045 ceph-mon[31676]: pgmap v42: 129 pgs: 129 active+clean; 2.2 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 111 KiB/s wr, 31 op/s 2024-04-16T17:08:06.203 INFO:tasks.workunit.client.0.smithi045.stdout: 1 5135 5.66 MB/sec warmup 8 sec latency 39.331 ms 2024-04-16T17:08:06.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:08:06 smithi104 ceph-mon[36457]: scrub queued for path: / 2024-04-16T17:08:06.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:08:06 smithi104 ceph-mon[36457]: scrub summary: idle+waiting paths [/] 2024-04-16T17:08:06.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:08:06 smithi104 ceph-mon[36457]: scrub summary: active paths [/] 2024-04-16T17:08:06.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:08:06 smithi104 ceph-mon[36457]: scrub summary: idle+waiting paths [/] 2024-04-16T17:08:06.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:08:06 smithi104 ceph-mon[36457]: pgmap v43: 129 pgs: 129 active+clean; 2.2 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 111 KiB/s wr, 31 op/s 2024-04-16T17:08:06.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:08:06 smithi117 ceph-mon[36730]: scrub queued for path: / 2024-04-16T17:08:06.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:08:06 smithi117 ceph-mon[36730]: scrub summary: idle+waiting paths [/] 2024-04-16T17:08:06.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:08:06 smithi117 ceph-mon[36730]: scrub summary: active paths [/] 2024-04-16T17:08:06.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:08:06 smithi117 ceph-mon[36730]: scrub summary: idle+waiting paths [/] 2024-04-16T17:08:06.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:08:06 smithi117 ceph-mon[36730]: pgmap v43: 129 pgs: 129 active+clean; 2.2 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 111 KiB/s wr, 31 op/s 2024-04-16T17:08:07.203 INFO:tasks.workunit.client.0.smithi045.stdout: 1 6489 5.84 MB/sec warmup 9 sec latency 14.388 ms 2024-04-16T17:08:08.203 INFO:tasks.workunit.client.0.smithi045.stdout: 1 7491 5.92 MB/sec warmup 10 sec latency 29.514 ms 2024-04-16T17:08:08.467 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:08:06 smithi045 ceph-mon[31676]: scrub queued for path: / 2024-04-16T17:08:08.467 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:08:06 smithi045 ceph-mon[31676]: scrub summary: idle+waiting paths [/] 2024-04-16T17:08:08.467 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:08:06 smithi045 ceph-mon[31676]: scrub summary: active paths [/] 2024-04-16T17:08:08.467 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:08:06 smithi045 ceph-mon[31676]: scrub summary: idle+waiting paths [/] 2024-04-16T17:08:08.467 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:08:06 smithi045 ceph-mon[31676]: pgmap v43: 129 pgs: 129 active+clean; 2.2 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 111 KiB/s wr, 31 op/s 2024-04-16T17:08:08.677 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:08:08 smithi104 ceph-mon[36457]: pgmap v44: 129 pgs: 129 active+clean; 5.5 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 416 KiB/s wr, 124 op/s 2024-04-16T17:08:08.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:08:08 smithi117 ceph-mon[36730]: pgmap v44: 129 pgs: 129 active+clean; 5.5 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 416 KiB/s wr, 124 op/s 2024-04-16T17:08:09.205 INFO:tasks.workunit.client.0.smithi045.stdout: 1 7642 5.38 MB/sec warmup 11 sec latency 82.034 ms 2024-04-16T17:08:09.677 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:08:09 smithi104 ceph-mon[36457]: scrub summary: idle 2024-04-16T17:08:09.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:08:09 smithi117 ceph-mon[36730]: scrub summary: idle 2024-04-16T17:08:10.205 INFO:tasks.workunit.client.0.smithi045.stdout: 1 8025 5.00 MB/sec warmup 12 sec latency 84.242 ms 2024-04-16T17:08:10.570 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:08:08 smithi045 ceph-mon[31676]: pgmap v44: 129 pgs: 129 active+clean; 5.5 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 416 KiB/s wr, 124 op/s 2024-04-16T17:08:10.571 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:08:09 smithi045 ceph-mon[31676]: scrub summary: idle 2024-04-16T17:08:10.677 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:08:10 smithi104 ceph-mon[36457]: pgmap v45: 129 pgs: 129 active+clean; 5.5 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 416 KiB/s wr, 122 op/s 2024-04-16T17:08:10.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:08:10 smithi117 ceph-mon[36730]: pgmap v45: 129 pgs: 129 active+clean; 5.5 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 416 KiB/s wr, 122 op/s 2024-04-16T17:08:11.206 INFO:tasks.workunit.client.0.smithi045.stdout: 1 8993 4.97 MB/sec warmup 13 sec latency 48.887 ms 2024-04-16T17:08:12.206 INFO:tasks.workunit.client.0.smithi045.stdout: 1 10268 5.17 MB/sec warmup 14 sec latency 13.981 ms 2024-04-16T17:08:12.677 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:08:12 smithi104 ceph-mon[36457]: pgmap v46: 129 pgs: 129 active+clean; 7.6 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 283 KiB/s rd, 620 KiB/s wr, 152 op/s 2024-04-16T17:08:12.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:08:12 smithi117 ceph-mon[36730]: pgmap v46: 129 pgs: 129 active+clean; 7.6 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 283 KiB/s rd, 620 KiB/s wr, 152 op/s 2024-04-16T17:08:13.206 INFO:tasks.workunit.client.0.smithi045.stdout: 1 11103 5.22 MB/sec warmup 15 sec latency 11.667 ms 2024-04-16T17:08:13.833 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:08:10 smithi045 ceph-mon[31676]: pgmap v45: 129 pgs: 129 active+clean; 5.5 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 416 KiB/s wr, 122 op/s 2024-04-16T17:08:13.833 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:08:12 smithi045 ceph-mon[31676]: pgmap v46: 129 pgs: 129 active+clean; 7.6 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 283 KiB/s rd, 620 KiB/s wr, 152 op/s 2024-04-16T17:08:14.206 INFO:tasks.workunit.client.0.smithi045.stdout: 1 11461 4.93 MB/sec warmup 16 sec latency 98.810 ms 2024-04-16T17:08:14.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:08:14 smithi104 ceph-mon[36457]: pgmap v47: 129 pgs: 129 active+clean; 7.6 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 247 KiB/s rd, 509 KiB/s wr, 123 op/s 2024-04-16T17:08:14.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:08:14 smithi117 ceph-mon[36730]: pgmap v47: 129 pgs: 129 active+clean; 7.6 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 247 KiB/s rd, 509 KiB/s wr, 123 op/s 2024-04-16T17:08:15.206 INFO:tasks.workunit.client.0.smithi045.stdout: 1 11777 4.72 MB/sec warmup 17 sec latency 82.098 ms 2024-04-16T17:08:16.206 INFO:tasks.workunit.client.0.smithi045.stdout: 1 13095 4.75 MB/sec warmup 18 sec latency 66.929 ms 2024-04-16T17:08:16.277 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:16.275+0000 7f7d61c00640 1 -- 172.21.15.45:0/3907309363 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7d5c107160 msgr2=0x7f7d5c10ed30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:16.277 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:16.275+0000 7f7d61c00640 1 --2- 172.21.15.45:0/3907309363 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7d5c107160 0x7f7d5c10ed30 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f7d54009ca0 tx=0x7f7d5402c100 comp rx=0 tx=0).stop 2024-04-16T17:08:16.277 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:16.275+0000 7f7d61c00640 1 -- 172.21.15.45:0/3907309363 shutdown_connections 2024-04-16T17:08:16.277 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:16.275+0000 7f7d61c00640 1 --2- 172.21.15.45:0/3907309363 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7d5c107160 0x7f7d5c10ed30 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:16.277 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:16.275+0000 7f7d61c00640 1 --2- 172.21.15.45:0/3907309363 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7d5c102a40 0x7f7d5c106a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:16.277 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:16.275+0000 7f7d61c00640 1 --2- 172.21.15.45:0/3907309363 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7d5c102090 0x7f7d5c102470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:16.277 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:16.275+0000 7f7d61c00640 1 -- 172.21.15.45:0/3907309363 >> 172.21.15.45:0/3907309363 conn(0x7f7d5c0fd950 msgr2=0x7f7d5c0ffd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:08:16.277 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:16.275+0000 7f7d61c00640 1 -- 172.21.15.45:0/3907309363 shutdown_connections 2024-04-16T17:08:16.277 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:16.275+0000 7f7d61c00640 1 -- 172.21.15.45:0/3907309363 wait complete. 2024-04-16T17:08:16.279 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:16.276+0000 7f7d61c00640 1 Processor -- start 2024-04-16T17:08:16.280 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:16.277+0000 7f7d61c00640 1 -- start start 2024-04-16T17:08:16.280 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:16.277+0000 7f7d61c00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7d5c102090 0x7f7d5c198740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:16.280 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:16.277+0000 7f7d61c00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7d5c107160 0x7f7d5c198c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:16.280 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:16.277+0000 7f7d61c00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7d5c19d000 0x7f7d5c19d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:16.280 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:16.277+0000 7f7d61c00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f7d5c112220 con 0x7f7d5c102090 2024-04-16T17:08:16.280 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:16.277+0000 7f7d61c00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f7d5c1120a0 con 0x7f7d5c107160 2024-04-16T17:08:16.280 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:16.277+0000 7f7d61c00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f7d5c1123a0 con 0x7f7d5c19d000 2024-04-16T17:08:16.280 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:16.277+0000 7f7d5be00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7d5c102090 0x7f7d5c198740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:16.280 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:16.277+0000 7f7d5be00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7d5c102090 0x7f7d5c198740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33692/0 (socket says 172.21.15.45:33692) 2024-04-16T17:08:16.280 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:16.277+0000 7f7d5be00640 1 -- 172.21.15.45:0/4033981625 learned_addr learned my addr 172.21.15.45:0/4033981625 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:08:16.280 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:16.277+0000 7f7d5b400640 1 --2- 172.21.15.45:0/4033981625 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7d5c107160 0x7f7d5c198c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:16.280 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:16.277+0000 7f7d61200640 1 --2- 172.21.15.45:0/4033981625 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7d5c19d000 0x7f7d5c19d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:16.280 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:16.278+0000 7f7d5b400640 1 -- 172.21.15.45:0/4033981625 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7d5c19d000 msgr2=0x7f7d5c19d4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:16.281 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:16.278+0000 7f7d5b400640 1 --2- 172.21.15.45:0/4033981625 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7d5c19d000 0x7f7d5c19d4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:16.281 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:16.278+0000 7f7d5b400640 1 -- 172.21.15.45:0/4033981625 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7d5c102090 msgr2=0x7f7d5c198740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:16.281 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:16.278+0000 7f7d5b400640 1 --2- 172.21.15.45:0/4033981625 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7d5c102090 0x7f7d5c198740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:16.281 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:16.278+0000 7f7d5b400640 1 -- 172.21.15.45:0/4033981625 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d5c19daf0 con 0x7f7d5c107160 2024-04-16T17:08:16.281 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:16.278+0000 7f7d61200640 1 --2- 172.21.15.45:0/4033981625 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7d5c19d000 0x7f7d5c19d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:08:16.281 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:16.278+0000 7f7d5b400640 1 --2- 172.21.15.45:0/4033981625 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7d5c107160 0x7f7d5c198c80 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f7d4800e9c0 tx=0x7f7d4800ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:08:16.281 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:16.279+0000 7f7d58c00640 1 -- 172.21.15.45:0/4033981625 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d4800ccc0 con 0x7f7d5c107160 2024-04-16T17:08:16.281 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:16.279+0000 7f7d58c00640 1 -- 172.21.15.45:0/4033981625 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d48010910 con 0x7f7d5c107160 2024-04-16T17:08:16.281 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:16.279+0000 7f7d58c00640 1 -- 172.21.15.45:0/4033981625 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d480188c0 con 0x7f7d5c107160 2024-04-16T17:08:16.281 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:16.279+0000 7f7d61c00640 1 -- 172.21.15.45:0/4033981625 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d5c19dde0 con 0x7f7d5c107160 2024-04-16T17:08:16.281 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:16.279+0000 7f7d61c00640 1 -- 172.21.15.45:0/4033981625 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d5c1a5580 con 0x7f7d5c107160 2024-04-16T17:08:16.283 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:16.282+0000 7f7d65e5d740 1 Processor -- start 2024-04-16T17:08:16.283 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:16.282+0000 7f7d65e5d740 1 -- start start 2024-04-16T17:08:16.284 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:16.283+0000 7f7d65e5d740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55eeed3c27f0 0x55eeed3c1540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:16.284 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:16.283+0000 7f7d65e5d740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eeed3c1a80 0x55eeed3c1f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:16.284 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:16.283+0000 7f7d65e5d740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55eeed3cb000 0x55eeed3cd3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:16.285 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:16.283+0000 7f7d65e5d740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55eeed326780 con 0x55eeed3cb000 2024-04-16T17:08:16.285 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:16.283+0000 7f7d65e5d740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55eeed2ccf40 con 0x55eeed3c27f0 2024-04-16T17:08:16.285 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:16.283+0000 7f7d65e5d740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55eeed322740 con 0x55eeed3c1a80 2024-04-16T17:08:16.285 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:16.283+0000 7f7d61200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55eeed3c27f0 0x55eeed3c1540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:16.285 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:16.283+0000 7f7d61200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55eeed3c27f0 0x55eeed3c1540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:55174/0 (socket says 172.21.15.45:55174) 2024-04-16T17:08:16.285 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:16.283+0000 7f7d61200640 1 -- 172.21.15.45:0/591922253 learned_addr learned my addr 172.21.15.45:0/591922253 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:08:16.285 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:16.283+0000 7f7d5be00640 1 --2- 172.21.15.45:0/591922253 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eeed3c1a80 0x55eeed3c1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:16.285 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:16.283+0000 7f7d61200640 1 -- 172.21.15.45:0/591922253 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eeed3c1a80 msgr2=0x55eeed3c1f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:08:16.285 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:16.283+0000 7f7d61200640 1 --2- 172.21.15.45:0/591922253 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eeed3c1a80 0x55eeed3c1f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:16.285 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:16.283+0000 7f7d61200640 1 -- 172.21.15.45:0/591922253 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55eeed3cb000 msgr2=0x55eeed3cd3f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-16T17:08:16.285 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:16.283+0000 7f7d61200640 1 --2- 172.21.15.45:0/591922253 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55eeed3cb000 0x55eeed3cd3f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:16.285 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:16.283+0000 7f7d61200640 1 -- 172.21.15.45:0/591922253 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eeed039210 con 0x55eeed3c27f0 2024-04-16T17:08:16.285 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:16.284+0000 7f7d61200640 1 --2- 172.21.15.45:0/591922253 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55eeed3c27f0 0x55eeed3c1540 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f7d54009ca0 tx=0x7f7d54009900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2f2bc5158881ac66 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:08:16.286 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:16.284+0000 7f7d51600640 1 -- 172.21.15.45:0/591922253 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d54043070 con 0x55eeed3c27f0 2024-04-16T17:08:16.286 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:16.284+0000 7f7d58c00640 1 -- 172.21.15.45:0/4033981625 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 25) ==== 138879+0+0 (secure 0 0 0) 0x7f7d48010ac0 con 0x7f7d5c107160 2024-04-16T17:08:16.287 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:16.284+0000 7f7d51600640 1 -- 172.21.15.45:0/591922253 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d54003970 con 0x55eeed3c27f0 2024-04-16T17:08:16.287 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:16.284+0000 7f7d51600640 1 -- 172.21.15.45:0/591922253 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d54039040 con 0x55eeed3c27f0 2024-04-16T17:08:16.287 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:16.285+0000 7f7d65e5d740 1 -- 172.21.15.45:0/591922253 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55eeed3c27f0 msgr2=0x55eeed3c1540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:16.287 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:16.285+0000 7f7d65e5d740 1 --2- 172.21.15.45:0/591922253 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55eeed3c27f0 0x55eeed3c1540 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f7d54009ca0 tx=0x7f7d54009900 comp rx=0 tx=0).stop 2024-04-16T17:08:16.287 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:16.285+0000 7f7d65e5d740 1 -- 172.21.15.45:0/591922253 shutdown_connections 2024-04-16T17:08:16.287 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:16.285+0000 7f7d65e5d740 1 --2- 172.21.15.45:0/591922253 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eeed3c1a80 0x55eeed3c1f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:16.287 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:16.285+0000 7f7d65e5d740 1 --2- 172.21.15.45:0/591922253 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55eeed3cb000 0x55eeed3cd3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:16.287 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:16.285+0000 7f7d65e5d740 1 --2- 172.21.15.45:0/591922253 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55eeed3c27f0 0x55eeed3c1540 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:16.287 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:16.285+0000 7f7d65e5d740 1 -- 172.21.15.45:0/591922253 >> 172.21.15.45:0/591922253 conn(0x55eeed35c5d0 msgr2=0x55eeed3bc770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:08:16.287 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:16.285+0000 7f7d65e5d740 1 -- 172.21.15.45:0/591922253 shutdown_connections 2024-04-16T17:08:16.287 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:16.285+0000 7f7d65e5d740 1 -- 172.21.15.45:0/591922253 wait complete. 2024-04-16T17:08:16.287 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:16.286+0000 7f7d65e5d740 1 -- start start 2024-04-16T17:08:16.290 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:16.286+0000 7f7d65e5d740 1 Processor -- start 2024-04-16T17:08:16.290 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:16.286+0000 7f7d58c00640 1 --2- 172.21.15.45:0/4033981625 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f7d400a3dc0 0x7f7d400a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:16.290 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:16.286+0000 7f7d58c00640 1 -- 172.21.15.45:0/4033981625 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f7d480d10e0 con 0x7f7d5c107160 2024-04-16T17:08:16.290 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:16.287+0000 7f7d61200640 1 --2- 172.21.15.45:0/4033981625 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f7d400a3dc0 0x7f7d400a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:16.290 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:16.287+0000 7f7d65e5d740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55eeed3c1a80 0x55eeed3caac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:16.290 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:16.287+0000 7f7d65e5d740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eeed3c27f0 0x55eeed3c7e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:16.290 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:16.287+0000 7f7d65e5d740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55eeed3cb000 0x55eeed3c8570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:16.290 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:16.287+0000 7f7d65e5d740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55eeed3cdcd0 con 0x55eeed3c1a80 2024-04-16T17:08:16.290 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:16.287+0000 7f7d65e5d740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55eeed2e0390 con 0x55eeed3cb000 2024-04-16T17:08:16.290 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:16.287+0000 7f7d65e5d740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55eeed3cde50 con 0x55eeed3c27f0 2024-04-16T17:08:16.291 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:16.287+0000 7f7d65e5d740 10 client.0 _ll_register_callbacks cb 0x55eeed2c9f60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:08:16.291 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:16.287+0000 7f7d5be00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55eeed3c1a80 0x55eeed3caac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:16.291 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:16.287+0000 7f7d5be00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55eeed3c1a80 0x55eeed3caac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33706/0 (socket says 172.21.15.45:33706) 2024-04-16T17:08:16.291 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:16.287+0000 7f7d5be00640 1 -- 172.21.15.45:0/2503804285 learned_addr learned my addr 172.21.15.45:0/2503804285 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:08:16.291 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:16.288+0000 7f7d61200640 1 --2- 172.21.15.45:0/2503804285 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55eeed3cb000 0x55eeed3c8570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:16.291 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:16.288+0000 7f7d5be00640 1 -- 172.21.15.45:0/2503804285 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eeed3c27f0 msgr2=0x55eeed3c7e50 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-16T17:08:16.291 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:16.288+0000 7f7d5be00640 1 --2- 172.21.15.45:0/2503804285 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eeed3c27f0 0x55eeed3c7e50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:16.291 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:16.288+0000 7f7d5be00640 1 -- 172.21.15.45:0/2503804285 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55eeed3cb000 msgr2=0x55eeed3c8570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:08:16.291 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:16.288+0000 7f7d5be00640 1 --2- 172.21.15.45:0/2503804285 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55eeed3cb000 0x55eeed3c8570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:16.291 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:16.288+0000 7f7d5be00640 1 -- 172.21.15.45:0/2503804285 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eeed105620 con 0x55eeed3c1a80 2024-04-16T17:08:16.291 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:16.288+0000 7f7d61200640 1 --2- 172.21.15.45:0/2503804285 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55eeed3cb000 0x55eeed3c8570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:08:16.291 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:16.288+0000 7f7d5be00640 1 --2- 172.21.15.45:0/2503804285 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55eeed3c1a80 0x55eeed3caac0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f7d4c012770 tx=0x7f7d4c012c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87c903801ce7d9d3 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:08:16.291 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:16.288+0000 7f7d61200640 1 --2- 172.21.15.45:0/4033981625 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f7d400a3dc0 0x7f7d400a6280 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f7d54009bf0 tx=0x7f7d5402c100 comp rx=0 tx=0).ready entity=mgr.25411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:08:16.291 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:16.288+0000 7f7d46000640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-16T17:08:16.292 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:16.289+0000 7f7d46000640 1 -- 172.21.15.45:0/2503804285 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d4c00f920 con 0x55eeed3c1a80 2024-04-16T17:08:16.292 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:16.289+0000 7f7d46000640 1 -- 172.21.15.45:0/2503804285 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f7d4c00fac0 con 0x55eeed3c1a80 2024-04-16T17:08:16.292 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:16.289+0000 7f7d65e5d740 1 -- 172.21.15.45:0/2503804285 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7d54003970 con 0x55eeed3c1a80 2024-04-16T17:08:16.292 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:16.289+0000 7f7d46000640 1 -- 172.21.15.45:0/2503804285 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f7d4c00fcc0 con 0x55eeed3c1a80 2024-04-16T17:08:16.292 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:16.289+0000 7f7d65e5d740 10 client.15921 fetch_fsmap learned FSMap version 25 2024-04-16T17:08:16.292 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:16.289+0000 7f7d65e5d740 1 -- 172.21.15.45:0/2503804285 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55eeed3d9e00 con 0x55eeed3c1a80 2024-04-16T17:08:16.294 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:16.289+0000 7f7d46000640 1 -- 172.21.15.45:0/2503804285 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d54003970 con 0x55eeed3c1a80 2024-04-16T17:08:16.294 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:16.290+0000 7f7d46000640 1 -- 172.21.15.45:0/2503804285 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f7d4c01c420 con 0x55eeed3c1a80 2024-04-16T17:08:16.294 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:16.290+0000 7f7d65e5d740 10 client.15921 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T17:08:16.294 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:16.290+0000 7f7d65e5d740 10 client.15921 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T17:08:16.294 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:16.290+0000 7f7d65e5d740 20 client.15921 populate_metadata read hostname 'smithi045' 2024-04-16T17:08:16.294 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:16.290+0000 7f7d65e5d740 1 --2- 172.21.15.45:0/2503804285 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55eeed3db310 0x55eeed3dd700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:16.294 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:16.290+0000 7f7d65e5d740 4 client.15921 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:08:16.294 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:16.290+0000 7f7d65e5d740 1 -- 172.21.15.45:0/2503804285 --> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55eeed3341a0 con 0x55eeed3db310 2024-04-16T17:08:16.294 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:16.290+0000 7f7d5b400640 1 --2- 172.21.15.45:0/2503804285 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55eeed3db310 0x55eeed3dd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:16.295 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:16.291+0000 7f7d5b400640 1 --2- 172.21.15.45:0/2503804285 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55eeed3db310 0x55eeed3dd700 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a8f94952e8976582 server_cookie=e4ec47dc1516168c in_seq=0 out_seq=0 2024-04-16T17:08:16.295 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:16.291+0000 7f7d46000640 10 client.15921 ms_handle_connect on v2:172.21.15.104:6838/3660070592 2024-04-16T17:08:16.295 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:16.292+0000 7f7d46000640 1 -- 172.21.15.45:0/2503804285 <== mds.0 v2:172.21.15.104:6838/3660070592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55eeed3341a0 con 0x55eeed3db310 2024-04-16T17:08:16.295 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:16.292+0000 7f7d46000640 10 client.15921 handle_command_reply: tid=0 2024-04-16T17:08:16.295 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:16.292+0000 7f7d65e5d740 1 client.15921 shutdown 2024-04-16T17:08:16.295 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:16.292+0000 7f7d65e5d740 1 -- 172.21.15.45:0/2503804285 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55eeed3c1a80 msgr2=0x55eeed3caac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:16.295 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:16.292+0000 7f7d65e5d740 1 --2- 172.21.15.45:0/2503804285 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55eeed3c1a80 0x55eeed3caac0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f7d4c012770 tx=0x7f7d4c012c30 comp rx=0 tx=0).stop 2024-04-16T17:08:16.295 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:16.292+0000 7f7d65e5d740 1 -- 172.21.15.45:0/2503804285 shutdown_connections 2024-04-16T17:08:16.295 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:16.292+0000 7f7d65e5d740 1 --2- 172.21.15.45:0/2503804285 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55eeed3db310 0x55eeed3dd700 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:16.296 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:16.292+0000 7f7d65e5d740 1 --2- 172.21.15.45:0/2503804285 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eeed3c27f0 0x55eeed3c7e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:16.296 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:16.292+0000 7f7d65e5d740 1 --2- 172.21.15.45:0/2503804285 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55eeed3cb000 0x55eeed3c8570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:16.296 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:16.292+0000 7f7d65e5d740 1 --2- 172.21.15.45:0/2503804285 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55eeed3c1a80 0x55eeed3caac0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:16.296 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:16.292+0000 7f7d46000640 0 client.15921 ms_handle_reset on v2:172.21.15.104:6838/3660070592 2024-04-16T17:08:16.296 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:16.292+0000 7f7d65e5d740 1 -- 172.21.15.45:0/2503804285 >> 172.21.15.45:0/2503804285 conn(0x55eeed35c5d0 msgr2=0x55eeed3be4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:08:16.296 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:16.292+0000 7f7d65e5d740 1 -- 172.21.15.45:0/2503804285 shutdown_connections 2024-04-16T17:08:16.296 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:16.292+0000 7f7d65e5d740 1 -- 172.21.15.45:0/2503804285 wait complete. 2024-04-16T17:08:16.296 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:16.293+0000 7f7d65e5d740 20 client.15921 trim_cache size 0 max 16384 2024-04-16T17:08:16.325 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:16.324+0000 7f7d65e5d740 1 Processor -- start 2024-04-16T17:08:16.325 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:16.324+0000 7f7d65e5d740 1 -- start start 2024-04-16T17:08:16.326 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:16.325+0000 7f7d65e5d740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55eeed3c1a80 0x55eeed3c7df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:16.326 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:16.325+0000 7f7d65e5d740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55eeed3c27f0 0x55eeed3d6bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:16.326 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:16.325+0000 7f7d65e5d740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eeed3cb000 0x55eeed3ca580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:16.326 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:16.325+0000 7f7d65e5d740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f7d5c1b1b90 con 0x55eeed3c1a80 2024-04-16T17:08:16.326 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:16.325+0000 7f7d65e5d740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f7d5c1af460 con 0x55eeed3c27f0 2024-04-16T17:08:16.326 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:16.325+0000 7f7d65e5d740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f7d5c1ad290 con 0x55eeed3cb000 2024-04-16T17:08:16.326 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:16.325+0000 7f7d5b400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55eeed3c27f0 0x55eeed3d6bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:16.327 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:16.325+0000 7f7d5be00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55eeed3c1a80 0x55eeed3c7df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:16.327 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:16.325+0000 7f7d5b400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55eeed3c27f0 0x55eeed3d6bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:55194/0 (socket says 172.21.15.45:55194) 2024-04-16T17:08:16.327 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:16.325+0000 7f7d5b400640 1 -- 172.21.15.45:0/1937578490 learned_addr learned my addr 172.21.15.45:0/1937578490 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:08:16.327 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:16.325+0000 7f7d61200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eeed3cb000 0x55eeed3ca580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:16.328 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:16.326+0000 7f7d5b400640 1 -- 172.21.15.45:0/1937578490 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eeed3cb000 msgr2=0x55eeed3ca580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:08:16.328 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:16.326+0000 7f7d5b400640 1 --2- 172.21.15.45:0/1937578490 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eeed3cb000 0x55eeed3ca580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:16.328 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:16.326+0000 7f7d5b400640 1 -- 172.21.15.45:0/1937578490 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55eeed3c1a80 msgr2=0x55eeed3c7df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:08:16.328 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:16.326+0000 7f7d5b400640 1 --2- 172.21.15.45:0/1937578490 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55eeed3c1a80 0x55eeed3c7df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:16.328 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:16.326+0000 7f7d5b400640 1 -- 172.21.15.45:0/1937578490 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eeed3dda70 con 0x55eeed3c27f0 2024-04-16T17:08:16.329 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:16.326+0000 7f7d5b400640 1 --2- 172.21.15.45:0/1937578490 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55eeed3c27f0 0x55eeed3d6bd0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f7d4800c6a0 tx=0x7f7d4801a040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2c3ef68202587843 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:08:16.329 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:16.326+0000 7f7d50c00640 1 -- 172.21.15.45:0/1937578490 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d4809abd0 con 0x55eeed3c27f0 2024-04-16T17:08:16.329 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:16.326+0000 7f7d50c00640 1 -- 172.21.15.45:0/1937578490 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d4809ae30 con 0x55eeed3c27f0 2024-04-16T17:08:16.329 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:16.326+0000 7f7d50c00640 1 -- 172.21.15.45:0/1937578490 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d480ce840 con 0x55eeed3c27f0 2024-04-16T17:08:16.329 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:16.327+0000 7f7d65e5d740 1 -- 172.21.15.45:0/1937578490 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55eeed3c27f0 msgr2=0x55eeed3d6bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:16.329 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:16.327+0000 7f7d65e5d740 1 --2- 172.21.15.45:0/1937578490 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55eeed3c27f0 0x55eeed3d6bd0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f7d4800c6a0 tx=0x7f7d4801a040 comp rx=0 tx=0).stop 2024-04-16T17:08:16.329 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:16.327+0000 7f7d65e5d740 1 -- 172.21.15.45:0/1937578490 shutdown_connections 2024-04-16T17:08:16.329 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:16.327+0000 7f7d65e5d740 1 --2- 172.21.15.45:0/1937578490 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eeed3cb000 0x55eeed3ca580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:16.329 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:16.327+0000 7f7d65e5d740 1 --2- 172.21.15.45:0/1937578490 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55eeed3c27f0 0x55eeed3d6bd0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:16.329 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:16.327+0000 7f7d65e5d740 1 --2- 172.21.15.45:0/1937578490 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55eeed3c1a80 0x55eeed3c7df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:16.329 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:16.327+0000 7f7d65e5d740 1 -- 172.21.15.45:0/1937578490 >> 172.21.15.45:0/1937578490 conn(0x55eeed35c5d0 msgr2=0x55eeed3c76e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:08:16.329 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:16.327+0000 7f7d65e5d740 1 -- 172.21.15.45:0/1937578490 shutdown_connections 2024-04-16T17:08:16.329 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:16.327+0000 7f7d65e5d740 1 -- 172.21.15.45:0/1937578490 wait complete. 2024-04-16T17:08:16.332 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:16.328+0000 7f7d65e5d740 1 -- start start 2024-04-16T17:08:16.332 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:16.328+0000 7f7d65e5d740 1 Processor -- start 2024-04-16T17:08:16.332 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:16.328+0000 7f7d65e5d740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55eeed3c1a80 0x55eeed3d30e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:16.332 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:16.328+0000 7f7d65e5d740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eeed3c27f0 0x55eeed3d3620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:16.333 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:16.328+0000 7f7d65e5d740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55eeed3cb000 0x55eeed3d00b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:16.333 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:16.328+0000 7f7d65e5d740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f7d5c1ad920 con 0x55eeed3c1a80 2024-04-16T17:08:16.333 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:16.328+0000 7f7d65e5d740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f7d5c1aed90 con 0x55eeed3cb000 2024-04-16T17:08:16.333 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:16.328+0000 7f7d65e5d740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f7d5c1adff0 con 0x55eeed3c27f0 2024-04-16T17:08:16.333 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:16.329+0000 7f7d65e5d740 10 client.0 _ll_register_callbacks cb 0x7f7d4c009880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:08:16.333 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:16.329+0000 7f7d61200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55eeed3cb000 0x55eeed3d00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:16.333 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:16.329+0000 7f7d61200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55eeed3cb000 0x55eeed3d00b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:55200/0 (socket says 172.21.15.45:55200) 2024-04-16T17:08:16.333 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:16.329+0000 7f7d61200640 1 -- 172.21.15.45:0/4086569951 learned_addr learned my addr 172.21.15.45:0/4086569951 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:08:16.333 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:16.329+0000 7f7d5be00640 1 --2- 172.21.15.45:0/4086569951 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55eeed3c1a80 0x55eeed3d30e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:16.333 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:16.329+0000 7f7d5b400640 1 --2- 172.21.15.45:0/4086569951 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eeed3c27f0 0x55eeed3d3620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:16.333 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:16.329+0000 7f7d61200640 1 -- 172.21.15.45:0/4086569951 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eeed3c27f0 msgr2=0x55eeed3d3620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:08:16.333 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:16.329+0000 7f7d61200640 1 --2- 172.21.15.45:0/4086569951 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eeed3c27f0 0x55eeed3d3620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:16.333 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:16.329+0000 7f7d61200640 1 -- 172.21.15.45:0/4086569951 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55eeed3c1a80 msgr2=0x55eeed3d30e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:08:16.334 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:16.329+0000 7f7d61200640 1 --2- 172.21.15.45:0/4086569951 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55eeed3c1a80 0x55eeed3d30e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:16.334 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:16.329+0000 7f7d61200640 1 -- 172.21.15.45:0/4086569951 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eeed3d07a0 con 0x55eeed3cb000 2024-04-16T17:08:16.334 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:16.329+0000 7f7d5b400640 1 --2- 172.21.15.45:0/4086569951 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eeed3c27f0 0x55eeed3d3620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:08:16.334 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:16.330+0000 7f7d61200640 1 --2- 172.21.15.45:0/4086569951 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55eeed3cb000 0x55eeed3d00b0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f7d54000f80 tx=0x7f7d54005c50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=650a3d4e72569414 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:08:16.334 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:16.330+0000 7f7d46000640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-04-16T17:08:16.334 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:16.330+0000 7f7d46000640 1 -- 172.21.15.45:0/4086569951 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d54043050 con 0x55eeed3cb000 2024-04-16T17:08:16.334 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:16.330+0000 7f7d46000640 1 -- 172.21.15.45:0/4086569951 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f7d540349f0 con 0x55eeed3cb000 2024-04-16T17:08:16.334 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:16.330+0000 7f7d65e5d740 1 -- 172.21.15.45:0/4086569951 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7d4809ae30 con 0x55eeed3cb000 2024-04-16T17:08:16.334 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:16.331+0000 7f7d46000640 1 -- 172.21.15.45:0/4086569951 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f7d54067050 con 0x55eeed3cb000 2024-04-16T17:08:16.334 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:16.331+0000 7f7d65e5d740 10 client.26084 fetch_fsmap learned FSMap version 25 2024-04-16T17:08:16.334 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:16.331+0000 7f7d65e5d740 1 -- 172.21.15.45:0/4086569951 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55eeed446670 con 0x55eeed3cb000 2024-04-16T17:08:16.334 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:16.331+0000 7f7d46000640 1 -- 172.21.15.45:0/4086569951 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d4809ae30 con 0x55eeed3cb000 2024-04-16T17:08:16.334 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:16.331+0000 7f7d46000640 1 -- 172.21.15.45:0/4086569951 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f7d54052040 con 0x55eeed3cb000 2024-04-16T17:08:16.334 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:16.331+0000 7f7d65e5d740 10 client.26084 fetch_fsmap finished waiting for FSMap version 25 2024-04-16T17:08:16.335 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:16.331+0000 7f7d65e5d740 10 client.26084 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T17:08:16.335 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:16.331+0000 7f7d65e5d740 20 client.26084 populate_metadata read hostname 'smithi045' 2024-04-16T17:08:16.335 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:16.331+0000 7f7d65e5d740 1 --2- 172.21.15.45:0/4086569951 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55eeed4469f0 0x55eeed448de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:16.335 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:16.331+0000 7f7d65e5d740 4 client.26084 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-16T17:08:16.335 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:16.331+0000 7f7d65e5d740 1 -- 172.21.15.45:0/4086569951 --> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55eeed32e010 con 0x55eeed4469f0 2024-04-16T17:08:16.335 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:16.332+0000 7f7d5be00640 1 --2- 172.21.15.45:0/4086569951 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55eeed4469f0 0x55eeed448de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:16.335 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:16.332+0000 7f7d5be00640 1 --2- 172.21.15.45:0/4086569951 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55eeed4469f0 0x55eeed448de0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a65015d40e548311 server_cookie=6bc0a4fd3a1edee7 in_seq=0 out_seq=0 2024-04-16T17:08:16.335 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:16.332+0000 7f7d46000640 10 client.26084 ms_handle_connect on v2:172.21.15.104:6838/3660070592 2024-04-16T17:08:16.487 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:16.485+0000 7f7d46000640 1 -- 172.21.15.45:0/4086569951 <== mds.0 v2:172.21.15.104:6838/3660070592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55eeed32e010 con 0x55eeed4469f0 2024-04-16T17:08:16.487 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:16.485+0000 7f7d46000640 10 client.26084 handle_command_reply: tid=0 2024-04-16T17:08:16.487 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:16.485+0000 7f7d65e5d740 1 client.26084 shutdown 2024-04-16T17:08:16.487 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:16.486+0000 7f7d65e5d740 1 -- 172.21.15.45:0/4086569951 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55eeed3cb000 msgr2=0x55eeed3d00b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:16.487 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:16.486+0000 7f7d65e5d740 1 --2- 172.21.15.45:0/4086569951 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55eeed3cb000 0x55eeed3d00b0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f7d54000f80 tx=0x7f7d54005c50 comp rx=0 tx=0).stop 2024-04-16T17:08:16.487 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:16.486+0000 7f7d65e5d740 1 -- 172.21.15.45:0/4086569951 shutdown_connections 2024-04-16T17:08:16.487 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:16.486+0000 7f7d65e5d740 1 --2- 172.21.15.45:0/4086569951 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55eeed4469f0 0x55eeed448de0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:16.487 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:16.486+0000 7f7d65e5d740 1 --2- 172.21.15.45:0/4086569951 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eeed3c27f0 0x55eeed3d3620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:16.487 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:16.486+0000 7f7d65e5d740 1 --2- 172.21.15.45:0/4086569951 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55eeed3cb000 0x55eeed3d00b0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:16.487 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:16.486+0000 7f7d65e5d740 1 --2- 172.21.15.45:0/4086569951 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55eeed3c1a80 0x55eeed3d30e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:16.488 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:16.486+0000 7f7d65e5d740 1 -- 172.21.15.45:0/4086569951 >> 172.21.15.45:0/4086569951 conn(0x55eeed35c5d0 msgr2=0x55eeed3baf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:08:16.488 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:16.486+0000 7f7d46000640 0 client.26084 ms_handle_reset on v2:172.21.15.104:6838/3660070592 2024-04-16T17:08:16.488 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:16.486+0000 7f7d65e5d740 1 -- 172.21.15.45:0/4086569951 shutdown_connections 2024-04-16T17:08:16.488 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:16.486+0000 7f7d65e5d740 1 -- 172.21.15.45:0/4086569951 wait complete. 2024-04-16T17:08:16.488 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:16.486+0000 7f7d65e5d740 20 client.26084 trim_cache size 0 max 16384 2024-04-16T17:08:16.489 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-04-16T17:08:16.489 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-04-16T17:08:16.489 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-04-16T17:08:16.489 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-04-16T17:08:16.490 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:16.487+0000 7f7d61c00640 1 -- 172.21.15.45:0/4033981625 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f7d400a3dc0 msgr2=0x7f7d400a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:16.490 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:16.487+0000 7f7d61c00640 1 --2- 172.21.15.45:0/4033981625 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f7d400a3dc0 0x7f7d400a6280 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f7d54009bf0 tx=0x7f7d5402c100 comp rx=0 tx=0).stop 2024-04-16T17:08:16.490 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:16.487+0000 7f7d61c00640 1 -- 172.21.15.45:0/4033981625 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7d5c107160 msgr2=0x7f7d5c198c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:16.490 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:16.487+0000 7f7d61c00640 1 --2- 172.21.15.45:0/4033981625 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7d5c107160 0x7f7d5c198c80 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f7d4800e9c0 tx=0x7f7d4800ee80 comp rx=0 tx=0).stop 2024-04-16T17:08:16.490 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:16.487+0000 7f7d61c00640 1 -- 172.21.15.45:0/4033981625 shutdown_connections 2024-04-16T17:08:16.490 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:16.487+0000 7f7d61c00640 1 --2- 172.21.15.45:0/4033981625 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f7d400a3dc0 0x7f7d400a6280 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:16.490 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:16.487+0000 7f7d61c00640 1 --2- 172.21.15.45:0/4033981625 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7d5c19d000 0x7f7d5c19d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:16.490 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:16.487+0000 7f7d61c00640 1 --2- 172.21.15.45:0/4033981625 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7d5c107160 0x7f7d5c198c80 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:16.490 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:16.487+0000 7f7d61c00640 1 --2- 172.21.15.45:0/4033981625 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7d5c102090 0x7f7d5c198740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:16.490 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:16.487+0000 7f7d61c00640 1 -- 172.21.15.45:0/4033981625 >> 172.21.15.45:0/4033981625 conn(0x7f7d5c0fd950 msgr2=0x7f7d5c0ffc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:08:16.490 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:16.488+0000 7f7d61c00640 1 -- 172.21.15.45:0/4033981625 shutdown_connections 2024-04-16T17:08:16.490 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:16.488+0000 7f7d61c00640 1 -- 172.21.15.45:0/4033981625 wait complete. 2024-04-16T17:08:16.665 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:08:14 smithi045 ceph-mon[31676]: pgmap v47: 129 pgs: 129 active+clean; 7.6 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 247 KiB/s rd, 509 KiB/s wr, 123 op/s 2024-04-16T17:08:16.677 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:08:16 smithi104 ceph-mon[36457]: pgmap v48: 129 pgs: 129 active+clean; 7.6 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 247 KiB/s rd, 509 KiB/s wr, 123 op/s 2024-04-16T17:08:16.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:08:16 smithi117 ceph-mon[36730]: pgmap v48: 129 pgs: 129 active+clean; 7.6 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 247 KiB/s rd, 509 KiB/s wr, 123 op/s 2024-04-16T17:08:17.206 INFO:tasks.workunit.client.0.smithi045.stdout: 1 14422 5.11 MB/sec warmup 19 sec latency 56.711 ms 2024-04-16T17:08:18.206 INFO:tasks.workunit.client.0.smithi045.stdout: 1 15046 4.90 MB/sec warmup 20 sec latency 15.955 ms 2024-04-16T17:08:18.632 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:08:16 smithi045 ceph-mon[31676]: pgmap v48: 129 pgs: 129 active+clean; 7.6 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 247 KiB/s rd, 509 KiB/s wr, 123 op/s 2024-04-16T17:08:18.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:08:18 smithi117 ceph-mon[36730]: pgmap v49: 129 pgs: 129 active+clean; 15 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 369 KiB/s rd, 1.1 MiB/s wr, 150 op/s 2024-04-16T17:08:18.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:08:18 smithi117 ceph-mon[36730]: mgrmap e26: x(active, since 92s), standbys: z, y 2024-04-16T17:08:18.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:08:18 smithi104 ceph-mon[36457]: pgmap v49: 129 pgs: 129 active+clean; 15 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 369 KiB/s rd, 1.1 MiB/s wr, 150 op/s 2024-04-16T17:08:18.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:08:18 smithi104 ceph-mon[36457]: mgrmap e26: x(active, since 92s), standbys: z, y 2024-04-16T17:08:19.210 INFO:tasks.workunit.client.0.smithi045.stdout: 1 15412 4.74 MB/sec warmup 21 sec latency 75.150 ms 2024-04-16T17:08:20.211 INFO:tasks.workunit.client.0.smithi045.stdout: 1 16286 4.69 MB/sec warmup 22 sec latency 99.216 ms 2024-04-16T17:08:20.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:08:20 smithi117 ceph-mon[36730]: pgmap v50: 129 pgs: 129 active+clean; 15 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 245 KiB/s rd, 846 KiB/s wr, 57 op/s 2024-04-16T17:08:20.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:08:20 smithi104 ceph-mon[36457]: pgmap v50: 129 pgs: 129 active+clean; 15 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 245 KiB/s rd, 846 KiB/s wr, 57 op/s 2024-04-16T17:08:21.211 INFO:tasks.workunit.client.0.smithi045.stdout: 1 17558 4.86 MB/sec warmup 23 sec latency 25.166 ms 2024-04-16T17:08:22.211 INFO:tasks.workunit.client.0.smithi045.stdout: 1 18250 4.85 MB/sec warmup 24 sec latency 11.802 ms 2024-04-16T17:08:22.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:08:22 smithi117 ceph-mon[36730]: pgmap v51: 129 pgs: 129 active+clean; 26 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 443 KiB/s rd, 3.0 MiB/s wr, 204 op/s 2024-04-16T17:08:22.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:08:22 smithi104 ceph-mon[36457]: pgmap v51: 129 pgs: 129 active+clean; 26 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 443 KiB/s rd, 3.0 MiB/s wr, 204 op/s 2024-04-16T17:08:23.179 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:08:18 smithi045 ceph-mon[31676]: pgmap v49: 129 pgs: 129 active+clean; 15 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 369 KiB/s rd, 1.1 MiB/s wr, 150 op/s 2024-04-16T17:08:23.179 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:08:18 smithi045 ceph-mon[31676]: mgrmap e26: x(active, since 92s), standbys: z, y 2024-04-16T17:08:23.179 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:08:20 smithi045 ceph-mon[31676]: pgmap v50: 129 pgs: 129 active+clean; 15 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 245 KiB/s rd, 846 KiB/s wr, 57 op/s 2024-04-16T17:08:23.179 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:08:22 smithi045 ceph-mon[31676]: pgmap v51: 129 pgs: 129 active+clean; 26 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 443 KiB/s rd, 3.0 MiB/s wr, 204 op/s 2024-04-16T17:08:23.211 INFO:tasks.workunit.client.0.smithi045.stdout: 1 19186 4.86 MB/sec warmup 25 sec latency 29.151 ms 2024-04-16T17:08:24.211 INFO:tasks.workunit.client.0.smithi045.stdout: 1 19533 4.69 MB/sec warmup 26 sec latency 68.339 ms 2024-04-16T17:08:24.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:08:24 smithi117 ceph-mon[36730]: pgmap v52: 129 pgs: 129 active+clean; 26 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 320 KiB/s rd, 2.8 MiB/s wr, 173 op/s 2024-04-16T17:08:24.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:08:24 smithi104 ceph-mon[36457]: pgmap v52: 129 pgs: 129 active+clean; 26 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 320 KiB/s rd, 2.8 MiB/s wr, 173 op/s 2024-04-16T17:08:25.211 INFO:tasks.workunit.client.0.smithi045.stdout: 1 19804 4.53 MB/sec warmup 27 sec latency 81.697 ms 2024-04-16T17:08:26.211 INFO:tasks.workunit.client.0.smithi045.stdout: 1 21131 4.68 MB/sec warmup 28 sec latency 14.718 ms 2024-04-16T17:08:26.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:08:26 smithi117 ceph-mon[36730]: pgmap v53: 129 pgs: 129 active+clean; 26 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 320 KiB/s rd, 2.8 MiB/s wr, 173 op/s 2024-04-16T17:08:26.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:08:26 smithi104 ceph-mon[36457]: pgmap v53: 129 pgs: 129 active+clean; 26 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 320 KiB/s rd, 2.8 MiB/s wr, 173 op/s 2024-04-16T17:08:27.211 INFO:tasks.workunit.client.0.smithi045.stdout: 1 21778 4.68 MB/sec warmup 29 sec latency 11.944 ms 2024-04-16T17:08:28.211 INFO:tasks.workunit.client.0.smithi045.stdout: 1 22562 4.59 MB/sec warmup 30 sec latency 17.641 ms 2024-04-16T17:08:28.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:08:28 smithi117 ceph-mon[36730]: pgmap v54: 129 pgs: 129 active+clean; 28 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 435 KiB/s rd, 3.0 MiB/s wr, 198 op/s 2024-04-16T17:08:28.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:08:28 smithi104 ceph-mon[36457]: pgmap v54: 129 pgs: 129 active+clean; 28 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 435 KiB/s rd, 3.0 MiB/s wr, 198 op/s 2024-04-16T17:08:29.218 INFO:tasks.workunit.client.0.smithi045.stdout: 1 22918 4.54 MB/sec warmup 31 sec latency 66.475 ms 2024-04-16T17:08:29.627 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:29.625+0000 7f090ca00640 1 -- 172.21.15.45:0/3618236989 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0908109520 msgr2=0x7f09081110f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:29.627 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:29.625+0000 7f090ca00640 1 --2- 172.21.15.45:0/3618236989 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0908109520 0x7f09081110f0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f0900009ca0 tx=0x7f090002c100 comp rx=0 tx=0).stop 2024-04-16T17:08:29.627 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:29.625+0000 7f090ca00640 1 -- 172.21.15.45:0/3618236989 shutdown_connections 2024-04-16T17:08:29.627 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:29.625+0000 7f090ca00640 1 --2- 172.21.15.45:0/3618236989 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0908104e00 0x7f0908108e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:29.627 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:29.625+0000 7f090ca00640 1 --2- 172.21.15.45:0/3618236989 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0908109520 0x7f09081110f0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:29.627 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:29.625+0000 7f090ca00640 1 --2- 172.21.15.45:0/3618236989 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0908104450 0x7f0908104830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:29.627 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:29.625+0000 7f090ca00640 1 -- 172.21.15.45:0/3618236989 >> 172.21.15.45:0/3618236989 conn(0x7f09080ffcf0 msgr2=0x7f0908102110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:08:29.628 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:29.625+0000 7f090ca00640 1 -- 172.21.15.45:0/3618236989 shutdown_connections 2024-04-16T17:08:29.628 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:29.625+0000 7f090ca00640 1 -- 172.21.15.45:0/3618236989 wait complete. 2024-04-16T17:08:29.629 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:29.625+0000 7f090ca00640 1 Processor -- start 2024-04-16T17:08:29.629 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:29.625+0000 7f090ca00640 1 -- start start 2024-04-16T17:08:29.629 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:29.625+0000 7f090ca00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0908104450 0x7f09081a0670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:29.629 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:29.625+0000 7f090ca00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0908109520 0x7f09081a0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:29.629 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:29.625+0000 7f090ca00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f090819a7e0 0x7f090819ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:29.630 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:29.625+0000 7f090ca00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f09081145c0 con 0x7f0908104450 2024-04-16T17:08:29.630 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:29.625+0000 7f090ca00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f0908114440 con 0x7f0908109520 2024-04-16T17:08:29.630 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:29.625+0000 7f090ca00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f0908114740 con 0x7f090819a7e0 2024-04-16T17:08:29.630 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:29.625+0000 7f0905e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0908104450 0x7f09081a0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:29.630 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:29.625+0000 7f0905400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0908109520 0x7f09081a0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:29.630 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:29.625+0000 7f0905e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0908104450 0x7f09081a0670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42034/0 (socket says 172.21.15.45:42034) 2024-04-16T17:08:29.630 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:29.625+0000 7f0905e00640 1 -- 172.21.15.45:0/585956780 learned_addr learned my addr 172.21.15.45:0/585956780 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:08:29.630 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:29.625+0000 7f0906800640 1 --2- 172.21.15.45:0/585956780 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f090819a7e0 0x7f090819ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:29.630 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:29.625+0000 7f0906800640 1 -- 172.21.15.45:0/585956780 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0908109520 msgr2=0x7f09081a0bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:29.630 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:29.625+0000 7f0906800640 1 --2- 172.21.15.45:0/585956780 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0908109520 0x7f09081a0bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:29.630 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:29.625+0000 7f0906800640 1 -- 172.21.15.45:0/585956780 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0908104450 msgr2=0x7f09081a0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:29.630 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:29.625+0000 7f0906800640 1 --2- 172.21.15.45:0/585956780 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0908104450 0x7f09081a0670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:29.630 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:29.625+0000 7f0906800640 1 -- 172.21.15.45:0/585956780 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f090819b370 con 0x7f090819a7e0 2024-04-16T17:08:29.630 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:29.625+0000 7f0905e00640 1 --2- 172.21.15.45:0/585956780 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0908104450 0x7f09081a0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:08:29.631 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:29.625+0000 7f0906800640 1 --2- 172.21.15.45:0/585956780 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f090819a7e0 0x7f090819ac90 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f0900009bf0 tx=0x7f0900035d10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:08:29.631 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:29.625+0000 7f08fea00640 1 -- 172.21.15.45:0/585956780 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0900039040 con 0x7f090819a7e0 2024-04-16T17:08:29.631 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:29.625+0000 7f08fea00640 1 -- 172.21.15.45:0/585956780 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0900044070 con 0x7f090819a7e0 2024-04-16T17:08:29.631 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:29.625+0000 7f08fea00640 1 -- 172.21.15.45:0/585956780 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09000042a0 con 0x7f090819a7e0 2024-04-16T17:08:29.632 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:29.625+0000 7f090ca00640 1 -- 172.21.15.45:0/585956780 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f090819b600 con 0x7f090819a7e0 2024-04-16T17:08:29.632 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:29.625+0000 7f090ca00640 1 -- 172.21.15.45:0/585956780 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09081a7700 con 0x7f090819a7e0 2024-04-16T17:08:29.633 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:29.633+0000 7f08fea00640 1 -- 172.21.15.45:0/585956780 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f0900004440 con 0x7f090819a7e0 2024-04-16T17:08:29.634 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:29.633+0000 7f090f80c740 1 Processor -- start 2024-04-16T17:08:29.634 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:29.633+0000 7f090f80c740 1 -- start start 2024-04-16T17:08:29.635 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:29.633+0000 7f090f80c740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b70c8ff610 0x55b70c8fe360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:29.635 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:29.633+0000 7f090f80c740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b70c8fe8a0 0x55b70c8fed20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:29.635 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:29.633+0000 7f090f80c740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b70c907e20 0x55b70c90a210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:29.635 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:29.633+0000 7f090f80c740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55b70c863780 con 0x55b70c8fe8a0 2024-04-16T17:08:29.635 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:29.633+0000 7f090f80c740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55b70c809f40 con 0x55b70c8ff610 2024-04-16T17:08:29.635 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:29.633+0000 7f090f80c740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55b70c85f740 con 0x55b70c907e20 2024-04-16T17:08:29.635 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:29.633+0000 7f08fea00640 1 --2- 172.21.15.45:0/585956780 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f08ec0a3e40 0x7f08ec0a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:29.635 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:29.633+0000 7f0905400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b70c8ff610 0x55b70c8fe360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:29.635 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:29.633+0000 7f0905400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b70c8ff610 0x55b70c8fe360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:48908/0 (socket says 172.21.15.45:48908) 2024-04-16T17:08:29.636 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:29.633+0000 7f0905400640 1 -- 172.21.15.45:0/3065019750 learned_addr learned my addr 172.21.15.45:0/3065019750 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:08:29.636 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:29.633+0000 7f0905e00640 1 --2- 172.21.15.45:0/3065019750 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b70c8fe8a0 0x55b70c8fed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:29.636 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:29.633+0000 7f08fea00640 1 -- 172.21.15.45:0/585956780 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f09000f2ec0 con 0x7f090819a7e0 2024-04-16T17:08:29.636 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:29.633+0000 7f0905400640 1 --2- 172.21.15.45:0/3065019750 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b70c907e20 0x55b70c90a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:29.637 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:29.633+0000 7f0906800640 1 --2- 172.21.15.45:0/585956780 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f08ec0a3e40 0x7f08ec0a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:29.638 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:29.633+0000 7f0905400640 1 -- 172.21.15.45:0/3065019750 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b70c8ff610 msgr2=0x55b70c8fe360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:08:29.638 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:29.633+0000 7f0905400640 1 --2- 172.21.15.45:0/3065019750 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b70c8ff610 0x55b70c8fe360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:29.638 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:29.633+0000 7f0905400640 1 -- 172.21.15.45:0/3065019750 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b70c8fe8a0 msgr2=0x55b70c8fed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:08:29.638 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:29.633+0000 7f0905400640 1 --2- 172.21.15.45:0/3065019750 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b70c8fe8a0 0x55b70c8fed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:29.638 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:29.633+0000 7f0905400640 1 -- 172.21.15.45:0/3065019750 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b70c576210 con 0x55b70c907e20 2024-04-16T17:08:29.638 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:29.633+0000 7f0905400640 1 --2- 172.21.15.45:0/3065019750 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b70c907e20 0x55b70c90a210 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f08f0011730 tx=0x7f08f0005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ee9dad93a96a6e08 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:08:29.638 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:29.633+0000 7f0906800640 1 --2- 172.21.15.45:0/585956780 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f08ec0a3e40 0x7f08ec0a6300 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f09000bb590 tx=0x7f09000095c0 comp rx=0 tx=0).ready entity=mgr.25411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:08:29.638 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:29.633+0000 7f08f7000640 1 -- 172.21.15.45:0/3065019750 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08f0011e30 con 0x55b70c907e20 2024-04-16T17:08:29.638 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:29.633+0000 7f08f7000640 1 -- 172.21.15.45:0/3065019750 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f08f000fca0 con 0x55b70c907e20 2024-04-16T17:08:29.638 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:29.633+0000 7f08f7000640 1 -- 172.21.15.45:0/3065019750 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08f001fc00 con 0x55b70c907e20 2024-04-16T17:08:29.638 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:29.633+0000 7f090f80c740 1 -- 172.21.15.45:0/3065019750 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b70c907e20 msgr2=0x55b70c90a210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:29.638 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:29.633+0000 7f090f80c740 1 --2- 172.21.15.45:0/3065019750 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b70c907e20 0x55b70c90a210 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f08f0011730 tx=0x7f08f0005f90 comp rx=0 tx=0).stop 2024-04-16T17:08:29.639 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:29.633+0000 7f090f80c740 1 -- 172.21.15.45:0/3065019750 shutdown_connections 2024-04-16T17:08:29.639 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:29.633+0000 7f090f80c740 1 --2- 172.21.15.45:0/3065019750 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b70c907e20 0x55b70c90a210 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:29.639 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:29.633+0000 7f090f80c740 1 --2- 172.21.15.45:0/3065019750 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b70c8fe8a0 0x55b70c8fed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:29.639 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:29.633+0000 7f090f80c740 1 --2- 172.21.15.45:0/3065019750 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b70c8ff610 0x55b70c8fe360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:29.639 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:29.633+0000 7f090f80c740 1 -- 172.21.15.45:0/3065019750 >> 172.21.15.45:0/3065019750 conn(0x55b70c8995d0 msgr2=0x55b70c8f9590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:08:29.639 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:29.633+0000 7f090f80c740 1 -- 172.21.15.45:0/3065019750 shutdown_connections 2024-04-16T17:08:29.639 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:29.633+0000 7f090f80c740 1 -- 172.21.15.45:0/3065019750 wait complete. 2024-04-16T17:08:29.639 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:29.633+0000 7f090f80c740 1 -- start start 2024-04-16T17:08:29.639 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:29.633+0000 7f090f80c740 1 Processor -- start 2024-04-16T17:08:29.642 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:29.633+0000 7f090f80c740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b70c8fe8a0 0x55b70c904cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:29.642 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:29.633+0000 7f090f80c740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b70c8ff610 0x55b70c905200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:29.642 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:29.633+0000 7f090f80c740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b70c907e20 0x55b70c905740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:29.642 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:29.633+0000 7f090f80c740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55b70c90ab50 con 0x55b70c8ff610 2024-04-16T17:08:29.642 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:29.633+0000 7f090f80c740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55b70c81d390 con 0x55b70c8fe8a0 2024-04-16T17:08:29.642 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:29.633+0000 7f090f80c740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55b70c90acd0 con 0x55b70c907e20 2024-04-16T17:08:29.642 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:29.638+0000 7f090f80c740 10 client.0 _ll_register_callbacks cb 0x55b70c806f60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:08:29.642 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:29.638+0000 7f0905400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b70c8fe8a0 0x55b70c904cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:29.642 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:29.638+0000 7f0905400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b70c8fe8a0 0x55b70c904cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:48916/0 (socket says 172.21.15.45:48916) 2024-04-16T17:08:29.642 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:29.638+0000 7f0905400640 1 -- 172.21.15.45:0/952737444 learned_addr learned my addr 172.21.15.45:0/952737444 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:08:29.643 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:29.638+0000 7f0905e00640 1 --2- 172.21.15.45:0/952737444 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b70c8ff610 0x55b70c905200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:29.643 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:29.638+0000 7f0905400640 1 --2- 172.21.15.45:0/952737444 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b70c907e20 0x55b70c905740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:29.643 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:29.638+0000 7f0905e00640 1 -- 172.21.15.45:0/952737444 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b70c907e20 msgr2=0x55b70c905740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:08:29.643 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:29.638+0000 7f0905e00640 1 --2- 172.21.15.45:0/952737444 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b70c907e20 0x55b70c905740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:29.643 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:29.638+0000 7f0905e00640 1 -- 172.21.15.45:0/952737444 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b70c8fe8a0 msgr2=0x55b70c904cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:08:29.643 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:29.638+0000 7f0905e00640 1 --2- 172.21.15.45:0/952737444 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b70c8fe8a0 0x55b70c904cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:29.643 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:29.638+0000 7f0905e00640 1 -- 172.21.15.45:0/952737444 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b70c642620 con 0x55b70c8ff610 2024-04-16T17:08:29.643 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:29.638+0000 7f0905e00640 1 --2- 172.21.15.45:0/952737444 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b70c8ff610 0x55b70c905200 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x55b70c8ffd70 tx=0x7f08f800f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=522bc644a71db82f server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:08:29.643 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:29.638+0000 7f08eb400640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-16T17:08:29.643 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:29.638+0000 7f08eb400640 1 -- 172.21.15.45:0/952737444 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08f8004030 con 0x55b70c8ff610 2024-04-16T17:08:29.643 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:29.638+0000 7f08eb400640 1 -- 172.21.15.45:0/952737444 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f08f80041d0 con 0x55b70c8ff610 2024-04-16T17:08:29.643 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:29.638+0000 7f090f80c740 1 -- 172.21.15.45:0/952737444 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f08f000fca0 con 0x55b70c8ff610 2024-04-16T17:08:29.643 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:29.638+0000 7f08eb400640 1 -- 172.21.15.45:0/952737444 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f08f8025050 con 0x55b70c8ff610 2024-04-16T17:08:29.643 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:29.638+0000 7f090f80c740 10 client.15939 fetch_fsmap learned FSMap version 26 2024-04-16T17:08:29.643 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:29.638+0000 7f090f80c740 1 -- 172.21.15.45:0/952737444 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55b70c916d00 con 0x55b70c8ff610 2024-04-16T17:08:29.643 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:29.638+0000 7f08eb400640 1 -- 172.21.15.45:0/952737444 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f08f000fca0 con 0x55b70c8ff610 2024-04-16T17:08:29.644 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:29.640+0000 7f08eb400640 1 -- 172.21.15.45:0/952737444 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f08f8010430 con 0x55b70c8ff610 2024-04-16T17:08:29.644 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:29.640+0000 7f090f80c740 10 client.15939 fetch_fsmap finished waiting for FSMap version 26 2024-04-16T17:08:29.644 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:29.640+0000 7f090f80c740 10 client.15939 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T17:08:29.644 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:29.640+0000 7f090f80c740 20 client.15939 populate_metadata read hostname 'smithi045' 2024-04-16T17:08:29.644 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:29.640+0000 7f090f80c740 1 --2- 172.21.15.45:0/952737444 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55b70c918210 0x55b70c91a640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:29.644 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:29.640+0000 7f090f80c740 4 client.15939 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:08:29.644 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:29.640+0000 7f090f80c740 1 -- 172.21.15.45:0/952737444 --> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b70c8711a0 con 0x55b70c918210 2024-04-16T17:08:29.644 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:29.640+0000 7f0905400640 1 --2- 172.21.15.45:0/952737444 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55b70c918210 0x55b70c91a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:29.644 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:29.641+0000 7f0905400640 1 --2- 172.21.15.45:0/952737444 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55b70c918210 0x55b70c91a640 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b675ce1c0688b071 server_cookie=193721407ff3f9a9 in_seq=0 out_seq=0 2024-04-16T17:08:29.644 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:29.641+0000 7f08eb400640 10 client.15939 ms_handle_connect on v2:172.21.15.104:6838/3660070592 2024-04-16T17:08:29.645 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:29.641+0000 7f08eb400640 1 -- 172.21.15.45:0/952737444 <== mds.0 v2:172.21.15.104:6838/3660070592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b70c8711a0 con 0x55b70c918210 2024-04-16T17:08:29.645 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:29.641+0000 7f08eb400640 10 client.15939 handle_command_reply: tid=0 2024-04-16T17:08:29.645 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:29.641+0000 7f090f80c740 1 client.15939 shutdown 2024-04-16T17:08:29.645 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:29.642+0000 7f090f80c740 1 -- 172.21.15.45:0/952737444 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b70c8ff610 msgr2=0x55b70c905200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:29.645 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:29.642+0000 7f090f80c740 1 --2- 172.21.15.45:0/952737444 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b70c8ff610 0x55b70c905200 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x55b70c8ffd70 tx=0x7f08f800f790 comp rx=0 tx=0).stop 2024-04-16T17:08:29.646 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:29.642+0000 7f090f80c740 1 -- 172.21.15.45:0/952737444 shutdown_connections 2024-04-16T17:08:29.646 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:29.642+0000 7f090f80c740 1 --2- 172.21.15.45:0/952737444 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55b70c918210 0x55b70c91a640 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:29.646 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:29.642+0000 7f090f80c740 1 --2- 172.21.15.45:0/952737444 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b70c907e20 0x55b70c905740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:29.646 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:29.642+0000 7f090f80c740 1 --2- 172.21.15.45:0/952737444 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b70c8ff610 0x55b70c905200 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:29.646 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:29.642+0000 7f090f80c740 1 --2- 172.21.15.45:0/952737444 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b70c8fe8a0 0x55b70c904cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:29.646 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:29.642+0000 7f090f80c740 1 -- 172.21.15.45:0/952737444 >> 172.21.15.45:0/952737444 conn(0x55b70c8995d0 msgr2=0x55b70c8fb2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:08:29.646 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:29.642+0000 7f08eb400640 0 client.15939 ms_handle_reset on v2:172.21.15.104:6838/3660070592 2024-04-16T17:08:29.646 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:29.642+0000 7f090f80c740 1 -- 172.21.15.45:0/952737444 shutdown_connections 2024-04-16T17:08:29.646 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:29.642+0000 7f090f80c740 1 -- 172.21.15.45:0/952737444 wait complete. 2024-04-16T17:08:29.646 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:29.642+0000 7f090f80c740 20 client.15939 trim_cache size 0 max 16384 2024-04-16T17:08:29.660 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:29.659+0000 7f090f80c740 1 Processor -- start 2024-04-16T17:08:29.660 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:29.659+0000 7f090f80c740 1 -- start start 2024-04-16T17:08:29.662 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:29.660+0000 7f090f80c740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b70c8fe8a0 0x55b70c9073f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:29.662 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:29.660+0000 7f090f80c740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b70c8ff610 0x55b70c913a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:29.662 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:29.660+0000 7f090f80c740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b70c907e20 0x55b70c903ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:29.662 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:29.660+0000 7f090f80c740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f09081b3e00 con 0x55b70c907e20 2024-04-16T17:08:29.662 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:29.660+0000 7f090f80c740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f09081b16d0 con 0x55b70c8fe8a0 2024-04-16T17:08:29.662 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:29.660+0000 7f090f80c740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f09081af500 con 0x55b70c8ff610 2024-04-16T17:08:29.662 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:29.660+0000 7f0905400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b70c8fe8a0 0x55b70c9073f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:29.662 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:29.660+0000 7f0905400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b70c8fe8a0 0x55b70c9073f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:48920/0 (socket says 172.21.15.45:48920) 2024-04-16T17:08:29.662 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:29.660+0000 7f0905400640 1 -- 172.21.15.45:0/2162113587 learned_addr learned my addr 172.21.15.45:0/2162113587 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:08:29.662 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:29.660+0000 7f0905400640 1 --2- 172.21.15.45:0/2162113587 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b70c907e20 0x55b70c903ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:29.662 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:29.660+0000 7f0905e00640 1 --2- 172.21.15.45:0/2162113587 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b70c8ff610 0x55b70c913a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:29.662 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:29.660+0000 7f0905400640 1 -- 172.21.15.45:0/2162113587 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b70c8ff610 msgr2=0x55b70c913a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:08:29.663 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:29.660+0000 7f0905400640 1 --2- 172.21.15.45:0/2162113587 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b70c8ff610 0x55b70c913a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:29.663 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:29.660+0000 7f0905400640 1 -- 172.21.15.45:0/2162113587 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b70c907e20 msgr2=0x55b70c903ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:08:29.663 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:29.660+0000 7f0905400640 1 --2- 172.21.15.45:0/2162113587 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b70c907e20 0x55b70c903ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:29.663 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:29.660+0000 7f0905400640 1 -- 172.21.15.45:0/2162113587 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b70c8fe6d0 con 0x55b70c8fe8a0 2024-04-16T17:08:29.663 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:29.661+0000 7f0905400640 1 --2- 172.21.15.45:0/2162113587 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b70c8fe8a0 0x55b70c9073f0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f08f00058c0 tx=0x7f08f0005850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=650eb1a9bb2ac766 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:08:29.663 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:29.661+0000 7f08f6600640 1 -- 172.21.15.45:0/2162113587 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08f001f4c0 con 0x55b70c8fe8a0 2024-04-16T17:08:29.663 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:29.661+0000 7f08f6600640 1 -- 172.21.15.45:0/2162113587 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f08f001f660 con 0x55b70c8fe8a0 2024-04-16T17:08:29.663 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:29.661+0000 7f08f6600640 1 -- 172.21.15.45:0/2162113587 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08f000f9f0 con 0x55b70c8fe8a0 2024-04-16T17:08:29.666 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:29.662+0000 7f090f80c740 1 -- 172.21.15.45:0/2162113587 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b70c8fe8a0 msgr2=0x55b70c9073f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:29.666 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:29.662+0000 7f090f80c740 1 --2- 172.21.15.45:0/2162113587 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b70c8fe8a0 0x55b70c9073f0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f08f00058c0 tx=0x7f08f0005850 comp rx=0 tx=0).stop 2024-04-16T17:08:29.666 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:29.662+0000 7f090f80c740 1 -- 172.21.15.45:0/2162113587 shutdown_connections 2024-04-16T17:08:29.666 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:29.662+0000 7f090f80c740 1 --2- 172.21.15.45:0/2162113587 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b70c8ff610 0x55b70c913a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:29.666 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:29.662+0000 7f090f80c740 1 --2- 172.21.15.45:0/2162113587 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b70c907e20 0x55b70c903ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:29.666 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:29.662+0000 7f090f80c740 1 --2- 172.21.15.45:0/2162113587 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b70c8fe8a0 0x55b70c9073f0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:29.666 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:29.662+0000 7f090f80c740 1 -- 172.21.15.45:0/2162113587 >> 172.21.15.45:0/2162113587 conn(0x55b70c8995d0 msgr2=0x55b70c903590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:08:29.666 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:29.662+0000 7f090f80c740 1 -- 172.21.15.45:0/2162113587 shutdown_connections 2024-04-16T17:08:29.667 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:29.662+0000 7f090f80c740 1 -- 172.21.15.45:0/2162113587 wait complete. 2024-04-16T17:08:29.667 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:29.662+0000 7f090f80c740 1 -- start start 2024-04-16T17:08:29.667 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:29.663+0000 7f090f80c740 1 Processor -- start 2024-04-16T17:08:29.667 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:29.663+0000 7f090f80c740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b70c8fe8a0 0x55b70c9083f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:29.667 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:29.663+0000 7f090f80c740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b70c8ff610 0x55b70c908930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:29.667 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:29.663+0000 7f090f80c740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b70c907e20 0x55b70c90ef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:29.667 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:29.663+0000 7f090f80c740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f09081afb90 con 0x55b70c8ff610 2024-04-16T17:08:29.667 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:29.663+0000 7f090f80c740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f09081b1000 con 0x55b70c8fe8a0 2024-04-16T17:08:29.667 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:29.663+0000 7f090f80c740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f09081b0260 con 0x55b70c907e20 2024-04-16T17:08:29.667 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:29.663+0000 7f090f80c740 10 client.0 _ll_register_callbacks cb 0x7f08f8009cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:08:29.667 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:29.663+0000 7f0905e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b70c8ff610 0x55b70c908930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:29.667 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:29.663+0000 7f0905400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b70c8fe8a0 0x55b70c9083f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:29.667 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:29.663+0000 7f0905e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b70c8ff610 0x55b70c908930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42060/0 (socket says 172.21.15.45:42060) 2024-04-16T17:08:29.667 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:29.663+0000 7f0905e00640 1 -- 172.21.15.45:0/511437730 learned_addr learned my addr 172.21.15.45:0/511437730 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:08:29.667 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:29.663+0000 7f0905400640 1 --2- 172.21.15.45:0/511437730 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b70c907e20 0x55b70c90ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:29.667 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:29.664+0000 7f0905400640 1 -- 172.21.15.45:0/511437730 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b70c907e20 msgr2=0x55b70c90ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:08:29.668 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:29.664+0000 7f0905400640 1 --2- 172.21.15.45:0/511437730 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b70c907e20 0x55b70c90ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:29.668 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:29.664+0000 7f0905400640 1 -- 172.21.15.45:0/511437730 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b70c8ff610 msgr2=0x55b70c908930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:08:29.668 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:29.664+0000 7f0905400640 1 --2- 172.21.15.45:0/511437730 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b70c8ff610 0x55b70c908930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:29.668 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:29.664+0000 7f0905400640 1 -- 172.21.15.45:0/511437730 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b70c90f470 con 0x55b70c8fe8a0 2024-04-16T17:08:29.668 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:29.664+0000 7f0905e00640 1 --2- 172.21.15.45:0/511437730 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b70c8ff610 0x55b70c908930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:08:29.668 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:29.664+0000 7f0905400640 1 --2- 172.21.15.45:0/511437730 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b70c8fe8a0 0x55b70c9083f0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f08f0011730 tx=0x7f08f0009770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=11c115236fa19aa0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:08:29.668 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:29.664+0000 7f08eb400640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-04-16T17:08:29.668 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:29.664+0000 7f08eb400640 1 -- 172.21.15.45:0/511437730 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08f0005500 con 0x55b70c8fe8a0 2024-04-16T17:08:29.668 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:29.664+0000 7f090f80c740 1 -- 172.21.15.45:0/511437730 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f08f001f660 con 0x55b70c8fe8a0 2024-04-16T17:08:29.668 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:29.664+0000 7f08eb400640 1 -- 172.21.15.45:0/511437730 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f08f000f790 con 0x55b70c8fe8a0 2024-04-16T17:08:29.668 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:29.664+0000 7f08eb400640 1 -- 172.21.15.45:0/511437730 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f08f000f990 con 0x55b70c8fe8a0 2024-04-16T17:08:29.668 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:29.664+0000 7f090f80c740 10 client.26102 fetch_fsmap learned FSMap version 26 2024-04-16T17:08:29.668 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:29.664+0000 7f090f80c740 1 -- 172.21.15.45:0/511437730 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55b70c9106f0 con 0x55b70c8fe8a0 2024-04-16T17:08:29.668 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:29.664+0000 7f08eb400640 1 -- 172.21.15.45:0/511437730 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f08f001f660 con 0x55b70c8fe8a0 2024-04-16T17:08:29.669 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:29.664+0000 7f08eb400640 1 -- 172.21.15.45:0/511437730 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f08f001d020 con 0x55b70c8fe8a0 2024-04-16T17:08:29.669 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:29.666+0000 7f090f80c740 10 client.26102 fetch_fsmap finished waiting for FSMap version 26 2024-04-16T17:08:29.669 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:29.666+0000 7f090f80c740 10 client.26102 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T17:08:29.669 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:29.666+0000 7f090f80c740 20 client.26102 populate_metadata read hostname 'smithi045' 2024-04-16T17:08:29.669 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:29.666+0000 7f090f80c740 1 --2- 172.21.15.45:0/511437730 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55b70c97e930 0x55b70c980d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:29.669 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:29.666+0000 7f090f80c740 4 client.26102 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-16T17:08:29.669 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:29.666+0000 7f090f80c740 1 -- 172.21.15.45:0/511437730 --> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b70c86b010 con 0x55b70c97e930 2024-04-16T17:08:29.680 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:29.679+0000 7f0905e00640 1 --2- 172.21.15.45:0/511437730 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55b70c97e930 0x55b70c980d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:29.710 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:29.709+0000 7f0905e00640 1 --2- 172.21.15.45:0/511437730 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55b70c97e930 0x55b70c980d50 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c0c5e370fc99a63e server_cookie=a29763e2f7e2bc96 in_seq=0 out_seq=0 2024-04-16T17:08:29.710 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:29.709+0000 7f08eb400640 10 client.26102 ms_handle_connect on v2:172.21.15.104:6838/3660070592 2024-04-16T17:08:29.818 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:29.816+0000 7f08eb400640 1 -- 172.21.15.45:0/511437730 <== mds.0 v2:172.21.15.104:6838/3660070592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55b70c86b010 con 0x55b70c97e930 2024-04-16T17:08:29.818 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:29.816+0000 7f08eb400640 10 client.26102 handle_command_reply: tid=0 2024-04-16T17:08:29.818 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:29.816+0000 7f090f80c740 1 client.26102 shutdown 2024-04-16T17:08:29.818 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:29.817+0000 7f090f80c740 1 -- 172.21.15.45:0/511437730 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b70c8fe8a0 msgr2=0x55b70c9083f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:29.818 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:29.817+0000 7f090f80c740 1 --2- 172.21.15.45:0/511437730 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b70c8fe8a0 0x55b70c9083f0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f08f0011730 tx=0x7f08f0009770 comp rx=0 tx=0).stop 2024-04-16T17:08:29.818 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:29.817+0000 7f090f80c740 1 -- 172.21.15.45:0/511437730 shutdown_connections 2024-04-16T17:08:29.818 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:29.817+0000 7f090f80c740 1 --2- 172.21.15.45:0/511437730 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55b70c97e930 0x55b70c980d50 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:29.818 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:29.817+0000 7f090f80c740 1 --2- 172.21.15.45:0/511437730 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b70c907e20 0x55b70c90ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:29.819 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:29.817+0000 7f090f80c740 1 --2- 172.21.15.45:0/511437730 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b70c8ff610 0x55b70c908930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:29.819 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:29.817+0000 7f090f80c740 1 --2- 172.21.15.45:0/511437730 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b70c8fe8a0 0x55b70c9083f0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:29.819 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:29.817+0000 7f090f80c740 1 -- 172.21.15.45:0/511437730 >> 172.21.15.45:0/511437730 conn(0x55b70c8995d0 msgr2=0x55b70c904c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:08:29.819 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:29.817+0000 7f08eb400640 0 client.26102 ms_handle_reset on v2:172.21.15.104:6838/3660070592 2024-04-16T17:08:29.819 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:29.817+0000 7f090f80c740 1 -- 172.21.15.45:0/511437730 shutdown_connections 2024-04-16T17:08:29.819 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:29.817+0000 7f090f80c740 1 -- 172.21.15.45:0/511437730 wait complete. 2024-04-16T17:08:29.819 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:29.817+0000 7f090f80c740 20 client.26102 trim_cache size 0 max 16384 2024-04-16T17:08:29.819 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-04-16T17:08:29.819 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-04-16T17:08:29.819 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-04-16T17:08:29.820 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-04-16T17:08:29.820 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:29.818+0000 7f090ca00640 1 -- 172.21.15.45:0/585956780 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f08ec0a3e40 msgr2=0x7f08ec0a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:29.820 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:29.818+0000 7f090ca00640 1 --2- 172.21.15.45:0/585956780 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f08ec0a3e40 0x7f08ec0a6300 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f09000bb590 tx=0x7f09000095c0 comp rx=0 tx=0).stop 2024-04-16T17:08:29.820 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:29.818+0000 7f090ca00640 1 -- 172.21.15.45:0/585956780 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f090819a7e0 msgr2=0x7f090819ac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:29.820 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:29.818+0000 7f090ca00640 1 --2- 172.21.15.45:0/585956780 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f090819a7e0 0x7f090819ac90 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f0900009bf0 tx=0x7f0900035d10 comp rx=0 tx=0).stop 2024-04-16T17:08:29.820 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:29.818+0000 7f090ca00640 1 -- 172.21.15.45:0/585956780 shutdown_connections 2024-04-16T17:08:29.820 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:29.818+0000 7f090ca00640 1 --2- 172.21.15.45:0/585956780 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f08ec0a3e40 0x7f08ec0a6300 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:29.820 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:29.818+0000 7f090ca00640 1 --2- 172.21.15.45:0/585956780 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f090819a7e0 0x7f090819ac90 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:29.820 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:29.818+0000 7f090ca00640 1 --2- 172.21.15.45:0/585956780 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0908109520 0x7f09081a0bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:29.821 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:29.818+0000 7f090ca00640 1 --2- 172.21.15.45:0/585956780 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0908104450 0x7f09081a0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:29.821 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:29.818+0000 7f090ca00640 1 -- 172.21.15.45:0/585956780 >> 172.21.15.45:0/585956780 conn(0x7f09080ffcf0 msgr2=0x7f0908100700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:08:29.821 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:29.818+0000 7f090ca00640 1 -- 172.21.15.45:0/585956780 shutdown_connections 2024-04-16T17:08:29.821 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:29.818+0000 7f090ca00640 1 -- 172.21.15.45:0/585956780 wait complete. 2024-04-16T17:08:30.218 INFO:tasks.workunit.client.0.smithi045.stdout: 1 23185 4.41 MB/sec warmup 32 sec latency 97.041 ms 2024-04-16T17:08:30.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:08:30 smithi117 ceph-mon[36730]: pgmap v55: 129 pgs: 129 active+clean; 28 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 313 KiB/s rd, 2.3 MiB/s wr, 171 op/s 2024-04-16T17:08:30.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:08:30 smithi117 ceph-mon[36730]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-16T17:08:30.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:08:30 smithi104 ceph-mon[36457]: pgmap v55: 129 pgs: 129 active+clean; 28 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 313 KiB/s rd, 2.3 MiB/s wr, 171 op/s 2024-04-16T17:08:30.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:08:30 smithi104 ceph-mon[36457]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-16T17:08:31.218 INFO:tasks.workunit.client.0.smithi045.stdout: 1 24219 4.47 MB/sec warmup 33 sec latency 46.821 ms 2024-04-16T17:08:32.218 INFO:tasks.workunit.client.0.smithi045.stdout: 1 25112 4.54 MB/sec warmup 34 sec latency 21.630 ms 2024-04-16T17:08:32.424 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:08:24 smithi045 ceph-mon[31676]: pgmap v52: 129 pgs: 129 active+clean; 26 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 320 KiB/s rd, 2.8 MiB/s wr, 173 op/s 2024-04-16T17:08:32.424 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:08:26 smithi045 ceph-mon[31676]: pgmap v53: 129 pgs: 129 active+clean; 26 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 320 KiB/s rd, 2.8 MiB/s wr, 173 op/s 2024-04-16T17:08:32.424 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:08:28 smithi045 ceph-mon[31676]: pgmap v54: 129 pgs: 129 active+clean; 28 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 435 KiB/s rd, 3.0 MiB/s wr, 198 op/s 2024-04-16T17:08:32.424 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:08:30 smithi045 ceph-mon[31676]: pgmap v55: 129 pgs: 129 active+clean; 28 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 313 KiB/s rd, 2.3 MiB/s wr, 171 op/s 2024-04-16T17:08:32.424 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:08:30 smithi045 ceph-mon[31676]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-16T17:08:32.482 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:08:32 smithi045 ceph-mon[31676]: pgmap v56: 129 pgs: 129 active+clean; 26 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 662 KiB/s rd, 2.6 MiB/s wr, 240 op/s 2024-04-16T17:08:32.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:08:32 smithi117 ceph-mon[36730]: pgmap v56: 129 pgs: 129 active+clean; 26 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 662 KiB/s rd, 2.6 MiB/s wr, 240 op/s 2024-04-16T17:08:32.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:08:32 smithi104 ceph-mon[36457]: pgmap v56: 129 pgs: 129 active+clean; 26 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 662 KiB/s rd, 2.6 MiB/s wr, 240 op/s 2024-04-16T17:08:33.218 INFO:tasks.workunit.client.0.smithi045.stdout: 1 25199 4.41 MB/sec warmup 35 sec latency 763.895 ms 2024-04-16T17:08:34.218 INFO:tasks.workunit.client.0.smithi045.stdout: 1 25199 4.29 MB/sec warmup 36 sec latency 1764.015 ms 2024-04-16T17:08:34.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:08:34 smithi045 ceph-mon[31676]: pgmap v57: 129 pgs: 129 active+clean; 26 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 464 KiB/s rd, 418 KiB/s wr, 93 op/s 2024-04-16T17:08:34.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:08:34 smithi117 ceph-mon[36730]: pgmap v57: 129 pgs: 129 active+clean; 26 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 464 KiB/s rd, 418 KiB/s wr, 93 op/s 2024-04-16T17:08:34.836 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-16T17:08:34.837 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-16T17:08:34.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:08:34 smithi104 ceph-mon[36457]: pgmap v57: 129 pgs: 129 active+clean; 26 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 464 KiB/s rd, 418 KiB/s wr, 93 op/s 2024-04-16T17:08:35.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.050+0000 7f6a85800640 1 -- 172.21.15.45:0/975952134 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6a80101000 msgr2=0x7f6a801013e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:35.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.050+0000 7f6a85800640 1 --2- 172.21.15.45:0/975952134 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6a80101000 0x7f6a801013e0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f6a70009900 tx=0x7f6a7002bc30 comp rx=0 tx=0).stop 2024-04-16T17:08:35.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.050+0000 7f6a85800640 1 -- 172.21.15.45:0/975952134 shutdown_connections 2024-04-16T17:08:35.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.050+0000 7f6a85800640 1 --2- 172.21.15.45:0/975952134 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6a801019b0 0x7f6a801059f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:35.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.050+0000 7f6a85800640 1 --2- 172.21.15.45:0/975952134 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6a801060d0 0x7f6a8010dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:35.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.050+0000 7f6a85800640 1 --2- 172.21.15.45:0/975952134 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6a80101000 0x7f6a801013e0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:35.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.050+0000 7f6a85800640 1 -- 172.21.15.45:0/975952134 >> 172.21.15.45:0/975952134 conn(0x7f6a800fc880 msgr2=0x7f6a800feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:08:35.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.050+0000 7f6a85800640 1 -- 172.21.15.45:0/975952134 shutdown_connections 2024-04-16T17:08:35.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.050+0000 7f6a85800640 1 -- 172.21.15.45:0/975952134 wait complete. 2024-04-16T17:08:35.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.051+0000 7f6a85800640 1 Processor -- start 2024-04-16T17:08:35.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.051+0000 7f6a85800640 1 -- start start 2024-04-16T17:08:35.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.052+0000 7f6a85800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6a80101000 0x7f6a801a0650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:35.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.052+0000 7f6a85800640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6a801060d0 0x7f6a801a0b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:35.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.052+0000 7f6a85800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6a8019a7c0 0x7f6a8019ac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:35.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.052+0000 7f6a85800640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f6a801111d0 con 0x7f6a8019a7c0 2024-04-16T17:08:35.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.052+0000 7f6a85800640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f6a80111050 con 0x7f6a801060d0 2024-04-16T17:08:35.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.052+0000 7f6a85800640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f6a80111350 con 0x7f6a80101000 2024-04-16T17:08:35.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.052+0000 7f6a84e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6a8019a7c0 0x7f6a8019ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:35.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.052+0000 7f6a84e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6a8019a7c0 0x7f6a8019ac70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:40632/0 (socket says 172.21.15.45:40632) 2024-04-16T17:08:35.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.052+0000 7f6a7fe00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6a80101000 0x7f6a801a0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:35.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.052+0000 7f6a84e00640 1 -- 172.21.15.45:0/267078919 learned_addr learned my addr 172.21.15.45:0/267078919 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:08:35.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.052+0000 7f6a77400640 1 --2- 172.21.15.45:0/267078919 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6a801060d0 0x7f6a801a0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:35.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.053+0000 7f6a7fe00640 1 -- 172.21.15.45:0/267078919 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6a801060d0 msgr2=0x7f6a801a0b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:35.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.053+0000 7f6a7fe00640 1 --2- 172.21.15.45:0/267078919 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6a801060d0 0x7f6a801a0b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:35.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.053+0000 7f6a7fe00640 1 -- 172.21.15.45:0/267078919 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6a8019a7c0 msgr2=0x7f6a8019ac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:35.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.053+0000 7f6a7fe00640 1 --2- 172.21.15.45:0/267078919 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6a8019a7c0 0x7f6a8019ac70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:35.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.053+0000 7f6a7fe00640 1 -- 172.21.15.45:0/267078919 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a8019b350 con 0x7f6a80101000 2024-04-16T17:08:35.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.053+0000 7f6a77400640 1 --2- 172.21.15.45:0/267078919 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6a801060d0 0x7f6a801a0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:08:35.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.053+0000 7f6a7fe00640 1 --2- 172.21.15.45:0/267078919 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6a80101000 0x7f6a801a0650 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f6a7002c170 tx=0x7f6a70004360 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:08:35.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.053+0000 7f6a7d600640 1 -- 172.21.15.45:0/267078919 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a70043020 con 0x7f6a80101000 2024-04-16T17:08:35.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.053+0000 7f6a85800640 1 -- 172.21.15.45:0/267078919 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a8019b5e0 con 0x7f6a80101000 2024-04-16T17:08:35.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.053+0000 7f6a7d600640 1 -- 172.21.15.45:0/267078919 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6a700054e0 con 0x7f6a80101000 2024-04-16T17:08:35.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.053+0000 7f6a7d600640 1 -- 172.21.15.45:0/267078919 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a70003b80 con 0x7f6a80101000 2024-04-16T17:08:35.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.053+0000 7f6a85800640 1 -- 172.21.15.45:0/267078919 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a801a77e0 con 0x7f6a80101000 2024-04-16T17:08:35.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.056+0000 7f6a89ba5740 1 Processor -- start 2024-04-16T17:08:35.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.057+0000 7f6a7d600640 1 -- 172.21.15.45:0/267078919 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f6a70046020 con 0x7f6a80101000 2024-04-16T17:08:35.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.057+0000 7f6a89ba5740 1 -- start start 2024-04-16T17:08:35.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.057+0000 7f6a89ba5740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5640d53ee390 0x5640d53ee770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:35.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.057+0000 7f6a89ba5740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5640d53eecb0 0x5640d53f7350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:35.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.057+0000 7f6a89ba5740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5640d53f7890 0x5640d53f9c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:35.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.057+0000 7f6a89ba5740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5640d5354120 con 0x5640d53ee390 2024-04-16T17:08:35.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.057+0000 7f6a89ba5740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5640d52fa340 con 0x5640d53eecb0 2024-04-16T17:08:35.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.057+0000 7f6a89ba5740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5640d53500e0 con 0x5640d53f7890 2024-04-16T17:08:35.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.057+0000 7f6a84e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5640d53ee390 0x5640d53ee770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:35.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.057+0000 7f6a84e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5640d53ee390 0x5640d53ee770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:40634/0 (socket says 172.21.15.45:40634) 2024-04-16T17:08:35.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.057+0000 7f6a84e00640 1 -- 172.21.15.45:0/2803404445 learned_addr learned my addr 172.21.15.45:0/2803404445 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:08:35.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.057+0000 7f6a7fe00640 1 --2- 172.21.15.45:0/2803404445 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5640d53eecb0 0x5640d53f7350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:35.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.057+0000 7f6a77400640 1 --2- 172.21.15.45:0/2803404445 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5640d53f7890 0x5640d53f9c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:35.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.058+0000 7f6a84e00640 1 -- 172.21.15.45:0/2803404445 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5640d53f7890 msgr2=0x5640d53f9c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:08:35.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.058+0000 7f6a84e00640 1 --2- 172.21.15.45:0/2803404445 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5640d53f7890 0x5640d53f9c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:35.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.058+0000 7f6a84e00640 1 -- 172.21.15.45:0/2803404445 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5640d53eecb0 msgr2=0x5640d53f7350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:08:35.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.058+0000 7f6a84e00640 1 --2- 172.21.15.45:0/2803404445 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5640d53eecb0 0x5640d53f7350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:35.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.058+0000 7f6a84e00640 1 -- 172.21.15.45:0/2803404445 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5640d51307d0 con 0x5640d53ee390 2024-04-16T17:08:35.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.058+0000 7f6a84e00640 1 --2- 172.21.15.45:0/2803404445 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5640d53ee390 0x5640d53ee770 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f6a78011470 tx=0x7f6a78011930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3360ad8fd2c393a0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:08:35.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.058+0000 7f6a7d600640 1 --2- 172.21.15.45:0/267078919 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f6a640a3d20 0x7f6a640a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:35.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.058+0000 7f6a75600640 1 -- 172.21.15.45:0/2803404445 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a78004b20 con 0x5640d53ee390 2024-04-16T17:08:35.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.059+0000 7f6a7d600640 1 -- 172.21.15.45:0/267078919 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f6a700f29d0 con 0x7f6a80101000 2024-04-16T17:08:35.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.059+0000 7f6a75600640 1 -- 172.21.15.45:0/2803404445 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6a78004cc0 con 0x5640d53ee390 2024-04-16T17:08:35.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.059+0000 7f6a75600640 1 -- 172.21.15.45:0/2803404445 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a780053e0 con 0x5640d53ee390 2024-04-16T17:08:35.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.059+0000 7f6a7fe00640 1 --2- 172.21.15.45:0/267078919 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f6a640a3d20 0x7f6a640a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:35.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.059+0000 7f6a89ba5740 1 -- 172.21.15.45:0/2803404445 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5640d53ee390 msgr2=0x5640d53ee770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:35.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.059+0000 7f6a89ba5740 1 --2- 172.21.15.45:0/2803404445 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5640d53ee390 0x5640d53ee770 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f6a78011470 tx=0x7f6a78011930 comp rx=0 tx=0).stop 2024-04-16T17:08:35.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.059+0000 7f6a89ba5740 1 -- 172.21.15.45:0/2803404445 shutdown_connections 2024-04-16T17:08:35.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.059+0000 7f6a89ba5740 1 --2- 172.21.15.45:0/2803404445 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5640d53f7890 0x5640d53f9c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:35.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.059+0000 7f6a89ba5740 1 --2- 172.21.15.45:0/2803404445 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5640d53eecb0 0x5640d53f7350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:35.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.059+0000 7f6a89ba5740 1 --2- 172.21.15.45:0/2803404445 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5640d53ee390 0x5640d53ee770 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:35.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.059+0000 7f6a89ba5740 1 -- 172.21.15.45:0/2803404445 >> 172.21.15.45:0/2803404445 conn(0x5640d53e8ac0 msgr2=0x5640d53e8ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:08:35.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.059+0000 7f6a7fe00640 1 --2- 172.21.15.45:0/267078919 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f6a640a3d20 0x7f6a640a61e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x5640d53ede60 tx=0x7f6a700f8530 comp rx=0 tx=0).ready entity=mgr.25411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:08:35.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.060+0000 7f6a89ba5740 1 -- 172.21.15.45:0/2803404445 shutdown_connections 2024-04-16T17:08:35.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.060+0000 7f6a89ba5740 1 -- 172.21.15.45:0/2803404445 wait complete. 2024-04-16T17:08:35.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.060+0000 7f6a89ba5740 1 -- start start 2024-04-16T17:08:35.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.060+0000 7f6a89ba5740 1 Processor -- start 2024-04-16T17:08:35.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.061+0000 7f6a89ba5740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5640d53ee390 0x5640d53f4850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:35.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.061+0000 7f6a89ba5740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5640d53eecb0 0x5640d53f4d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:35.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.061+0000 7f6a89ba5740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5640d53f7890 0x5640d53f52d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:35.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.061+0000 7f6a89ba5740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5640d53fa680 con 0x5640d53ee390 2024-04-16T17:08:35.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.061+0000 7f6a89ba5740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5640d530d710 con 0x5640d53eecb0 2024-04-16T17:08:35.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.061+0000 7f6a89ba5740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5640d53fa800 con 0x5640d53f7890 2024-04-16T17:08:35.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.061+0000 7f6a89ba5740 10 client.0 _ll_register_callbacks cb 0x5640d5308af0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:08:35.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.061+0000 7f6a84e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5640d53ee390 0x5640d53f4850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:35.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.061+0000 7f6a84e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5640d53ee390 0x5640d53f4850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:40636/0 (socket says 172.21.15.45:40636) 2024-04-16T17:08:35.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.061+0000 7f6a84e00640 1 -- 172.21.15.45:0/1327093470 learned_addr learned my addr 172.21.15.45:0/1327093470 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:08:35.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.061+0000 7f6a77400640 1 --2- 172.21.15.45:0/1327093470 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5640d53eecb0 0x5640d53f4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:35.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.061+0000 7f6a84e00640 1 --2- 172.21.15.45:0/1327093470 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5640d53f7890 0x5640d53f52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:35.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.062+0000 7f6a84e00640 1 -- 172.21.15.45:0/1327093470 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5640d53f7890 msgr2=0x5640d53f52d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:08:35.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.062+0000 7f6a84e00640 1 --2- 172.21.15.45:0/1327093470 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5640d53f7890 0x5640d53f52d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:35.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.062+0000 7f6a84e00640 1 -- 172.21.15.45:0/1327093470 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5640d53eecb0 msgr2=0x5640d53f4d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:08:35.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.062+0000 7f6a84e00640 1 --2- 172.21.15.45:0/1327093470 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5640d53eecb0 0x5640d53f4d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:35.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.062+0000 7f6a84e00640 1 -- 172.21.15.45:0/1327093470 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5640d50663f0 con 0x5640d53ee390 2024-04-16T17:08:35.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.062+0000 7f6a77400640 1 --2- 172.21.15.45:0/1327093470 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5640d53eecb0 0x5640d53f4d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:08:35.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.062+0000 7f6a84e00640 1 --2- 172.21.15.45:0/1327093470 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5640d53ee390 0x5640d53f4850 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f6a78011e30 tx=0x7f6a78005000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b9c327328f87055 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:08:35.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.062+0000 7f6a6e000640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-16T17:08:35.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.062+0000 7f6a6e000640 1 -- 172.21.15.45:0/1327093470 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a7802b020 con 0x5640d53ee390 2024-04-16T17:08:35.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.062+0000 7f6a89ba5740 1 -- 172.21.15.45:0/1327093470 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6a78004cc0 con 0x5640d53ee390 2024-04-16T17:08:35.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.062+0000 7f6a6e000640 1 -- 172.21.15.45:0/1327093470 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f6a78005330 con 0x5640d53ee390 2024-04-16T17:08:35.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.063+0000 7f6a6e000640 1 -- 172.21.15.45:0/1327093470 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f6a78036050 con 0x5640d53ee390 2024-04-16T17:08:35.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.063+0000 7f6a89ba5740 10 client.15957 fetch_fsmap learned FSMap version 26 2024-04-16T17:08:35.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.063+0000 7f6a89ba5740 1 -- 172.21.15.45:0/1327093470 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5640d50260f0 con 0x5640d53ee390 2024-04-16T17:08:35.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.063+0000 7f6a6e000640 1 -- 172.21.15.45:0/1327093470 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6a78004cc0 con 0x5640d53ee390 2024-04-16T17:08:35.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.063+0000 7f6a6e000640 1 -- 172.21.15.45:0/1327093470 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f6a78027810 con 0x5640d53ee390 2024-04-16T17:08:35.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.063+0000 7f6a89ba5740 10 client.15957 fetch_fsmap finished waiting for FSMap version 26 2024-04-16T17:08:35.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.063+0000 7f6a89ba5740 10 client.15957 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:08:35.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.063+0000 7f6a89ba5740 20 client.15957 populate_metadata read hostname 'smithi045' 2024-04-16T17:08:35.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.063+0000 7f6a89ba5740 1 --2- 172.21.15.45:0/1327093470 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x5640d5407af0 0x5640d5409ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:35.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.063+0000 7f6a89ba5740 4 client.15957 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:08:35.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.063+0000 7f6a89ba5740 1 -- 172.21.15.45:0/1327093470 --> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5640d5361b40 con 0x5640d5407af0 2024-04-16T17:08:35.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.064+0000 7f6a77400640 1 --2- 172.21.15.45:0/1327093470 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x5640d5407af0 0x5640d5409ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:35.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.064+0000 7f6a77400640 1 --2- 172.21.15.45:0/1327093470 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x5640d5407af0 0x5640d5409ee0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2b4c2cbafee94ec4 server_cookie=8e6ebdb8571a9e7a in_seq=0 out_seq=0 2024-04-16T17:08:35.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.064+0000 7f6a6e000640 10 client.15957 ms_handle_connect on v2:172.21.15.104:6838/3660070592 2024-04-16T17:08:35.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.065+0000 7f6a6e000640 1 -- 172.21.15.45:0/1327093470 <== mds.0 v2:172.21.15.104:6838/3660070592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5640d5361b40 con 0x5640d5407af0 2024-04-16T17:08:35.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.065+0000 7f6a6e000640 10 client.15957 handle_command_reply: tid=0 2024-04-16T17:08:35.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.065+0000 7f6a89ba5740 1 client.15957 shutdown 2024-04-16T17:08:35.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.065+0000 7f6a89ba5740 1 -- 172.21.15.45:0/1327093470 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5640d53ee390 msgr2=0x5640d53f4850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:35.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.065+0000 7f6a89ba5740 1 --2- 172.21.15.45:0/1327093470 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5640d53ee390 0x5640d53f4850 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f6a78011e30 tx=0x7f6a78005000 comp rx=0 tx=0).stop 2024-04-16T17:08:35.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.066+0000 7f6a89ba5740 1 -- 172.21.15.45:0/1327093470 shutdown_connections 2024-04-16T17:08:35.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.066+0000 7f6a89ba5740 1 --2- 172.21.15.45:0/1327093470 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x5640d5407af0 0x5640d5409ee0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:35.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.066+0000 7f6a89ba5740 1 --2- 172.21.15.45:0/1327093470 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5640d53f7890 0x5640d53f52d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:35.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.066+0000 7f6a89ba5740 1 --2- 172.21.15.45:0/1327093470 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5640d53eecb0 0x5640d53f4d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:35.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.066+0000 7f6a89ba5740 1 --2- 172.21.15.45:0/1327093470 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5640d53ee390 0x5640d53f4850 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:35.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.066+0000 7f6a6e000640 0 client.15957 ms_handle_reset on v2:172.21.15.104:6838/3660070592 2024-04-16T17:08:35.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.066+0000 7f6a89ba5740 1 -- 172.21.15.45:0/1327093470 >> 172.21.15.45:0/1327093470 conn(0x5640d53e8ac0 msgr2=0x5640d53ea810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:08:35.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.066+0000 7f6a89ba5740 1 -- 172.21.15.45:0/1327093470 shutdown_connections 2024-04-16T17:08:35.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.066+0000 7f6a89ba5740 1 -- 172.21.15.45:0/1327093470 wait complete. 2024-04-16T17:08:35.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.066+0000 7f6a89ba5740 20 client.15957 trim_cache size 0 max 16384 2024-04-16T17:08:35.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.086+0000 7f6a89ba5740 1 Processor -- start 2024-04-16T17:08:35.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.086+0000 7f6a89ba5740 1 -- start start 2024-04-16T17:08:35.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.086+0000 7f6a89ba5740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5640d53ee390 0x5640d53f6f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:35.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.086+0000 7f6a89ba5740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5640d53eecb0 0x5640d5403540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:35.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.086+0000 7f6a89ba5740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5640d53f7890 0x5640d53f3790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:35.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.086+0000 7f6a89ba5740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f6a801b3d10 con 0x5640d53f7890 2024-04-16T17:08:35.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.086+0000 7f6a89ba5740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f6a801b15e0 con 0x5640d53ee390 2024-04-16T17:08:35.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.086+0000 7f6a89ba5740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f6a801af410 con 0x5640d53eecb0 2024-04-16T17:08:35.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.087+0000 7f6a84e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5640d53ee390 0x5640d53f6f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:35.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.087+0000 7f6a84e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5640d53ee390 0x5640d53f6f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:43980/0 (socket says 172.21.15.45:43980) 2024-04-16T17:08:35.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.087+0000 7f6a84e00640 1 -- 172.21.15.45:0/3635770160 learned_addr learned my addr 172.21.15.45:0/3635770160 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:08:35.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.087+0000 7f6a84e00640 1 --2- 172.21.15.45:0/3635770160 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5640d53f7890 0x5640d53f3790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:35.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.087+0000 7f6a77400640 1 --2- 172.21.15.45:0/3635770160 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5640d53eecb0 0x5640d5403540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:35.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.087+0000 7f6a84e00640 1 -- 172.21.15.45:0/3635770160 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5640d53eecb0 msgr2=0x5640d5403540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:08:35.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.087+0000 7f6a84e00640 1 --2- 172.21.15.45:0/3635770160 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5640d53eecb0 0x5640d5403540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:35.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.087+0000 7f6a84e00640 1 -- 172.21.15.45:0/3635770160 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5640d53f7890 msgr2=0x5640d53f3790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:08:35.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.087+0000 7f6a84e00640 1 --2- 172.21.15.45:0/3635770160 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5640d53f7890 0x5640d53f3790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:35.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.087+0000 7f6a84e00640 1 -- 172.21.15.45:0/3635770160 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5640d53ede90 con 0x5640d53ee390 2024-04-16T17:08:35.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.087+0000 7f6a77400640 1 --2- 172.21.15.45:0/3635770160 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5640d53eecb0 0x5640d5403540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:08:35.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.088+0000 7f6a84e00640 1 --2- 172.21.15.45:0/3635770160 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5640d53ee390 0x5640d53f6f80 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f6a780050f0 tx=0x7f6a78011930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5c1272630ddf1ea8 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:08:35.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.088+0000 7f6a74c00640 1 -- 172.21.15.45:0/3635770160 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a7802b070 con 0x5640d53ee390 2024-04-16T17:08:35.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.088+0000 7f6a74c00640 1 -- 172.21.15.45:0/3635770160 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6a78011020 con 0x5640d53ee390 2024-04-16T17:08:35.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.088+0000 7f6a74c00640 1 -- 172.21.15.45:0/3635770160 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a78005330 con 0x5640d53ee390 2024-04-16T17:08:35.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.088+0000 7f6a89ba5740 1 -- 172.21.15.45:0/3635770160 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5640d53ee390 msgr2=0x5640d53f6f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:35.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.088+0000 7f6a89ba5740 1 --2- 172.21.15.45:0/3635770160 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5640d53ee390 0x5640d53f6f80 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f6a780050f0 tx=0x7f6a78011930 comp rx=0 tx=0).stop 2024-04-16T17:08:35.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.088+0000 7f6a89ba5740 1 -- 172.21.15.45:0/3635770160 shutdown_connections 2024-04-16T17:08:35.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.088+0000 7f6a89ba5740 1 --2- 172.21.15.45:0/3635770160 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5640d53eecb0 0x5640d5403540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:35.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.088+0000 7f6a89ba5740 1 --2- 172.21.15.45:0/3635770160 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5640d53f7890 0x5640d53f3790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:35.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.088+0000 7f6a89ba5740 1 --2- 172.21.15.45:0/3635770160 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5640d53ee390 0x5640d53f6f80 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:35.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.088+0000 7f6a89ba5740 1 -- 172.21.15.45:0/3635770160 >> 172.21.15.45:0/3635770160 conn(0x5640d53e8ac0 msgr2=0x5640d53f50a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:08:35.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.089+0000 7f6a89ba5740 1 -- 172.21.15.45:0/3635770160 shutdown_connections 2024-04-16T17:08:35.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.089+0000 7f6a89ba5740 1 -- 172.21.15.45:0/3635770160 wait complete. 2024-04-16T17:08:35.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.089+0000 7f6a89ba5740 1 -- start start 2024-04-16T17:08:35.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.089+0000 7f6a89ba5740 1 Processor -- start 2024-04-16T17:08:35.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.090+0000 7f6a89ba5740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5640d53ee390 0x5640d53ffa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:35.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.090+0000 7f6a89ba5740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5640d53eecb0 0x5640d53fff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:35.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.090+0000 7f6a89ba5740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5640d53f7890 0x5640d53f0b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:35.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.090+0000 7f6a89ba5740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f6a801afaa0 con 0x5640d53eecb0 2024-04-16T17:08:35.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.090+0000 7f6a89ba5740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f6a801b0f10 con 0x5640d53f7890 2024-04-16T17:08:35.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.090+0000 7f6a89ba5740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f6a801b0170 con 0x5640d53ee390 2024-04-16T17:08:35.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.090+0000 7f6a89ba5740 10 client.0 _ll_register_callbacks cb 0x7f6a78009ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:08:35.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.090+0000 7f6a77400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5640d53eecb0 0x5640d53fff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:35.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.090+0000 7f6a84e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5640d53ee390 0x5640d53ffa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:35.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.090+0000 7f6a77400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5640d53eecb0 0x5640d53fff60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:40650/0 (socket says 172.21.15.45:40650) 2024-04-16T17:08:35.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.090+0000 7f6a77400640 1 -- 172.21.15.45:0/3950956907 learned_addr learned my addr 172.21.15.45:0/3950956907 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:08:35.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.090+0000 7f6a84e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5640d53ee390 0x5640d53ffa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:54146/0 (socket says 172.21.15.45:54146) 2024-04-16T17:08:35.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.090+0000 7f6a84e00640 1 --2- 172.21.15.45:0/3950956907 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5640d53f7890 0x5640d53f0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:35.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.090+0000 7f6a77400640 1 -- 172.21.15.45:0/3950956907 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5640d53ee390 msgr2=0x5640d53ffa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:08:35.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.090+0000 7f6a77400640 1 --2- 172.21.15.45:0/3950956907 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5640d53ee390 0x5640d53ffa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:35.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.090+0000 7f6a77400640 1 -- 172.21.15.45:0/3950956907 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5640d53f7890 msgr2=0x5640d53f0b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:08:35.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.090+0000 7f6a77400640 1 --2- 172.21.15.45:0/3950956907 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5640d53f7890 0x5640d53f0b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:35.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.090+0000 7f6a77400640 1 -- 172.21.15.45:0/3950956907 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5640d53f1070 con 0x5640d53eecb0 2024-04-16T17:08:35.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.091+0000 7f6a77400640 1 --2- 172.21.15.45:0/3950956907 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5640d53eecb0 0x5640d53fff60 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f6a68004700 tx=0x7f6a680207d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd03a9d5a2fabaff server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:08:35.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.091+0000 7f6a6e000640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-16T17:08:35.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.091+0000 7f6a6e000640 1 -- 172.21.15.45:0/3950956907 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a68004230 con 0x5640d53eecb0 2024-04-16T17:08:35.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.091+0000 7f6a6e000640 1 -- 172.21.15.45:0/3950956907 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f6a680043d0 con 0x5640d53eecb0 2024-04-16T17:08:35.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.091+0000 7f6a89ba5740 1 -- 172.21.15.45:0/3950956907 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6a78011020 con 0x5640d53eecb0 2024-04-16T17:08:35.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.091+0000 7f6a6e000640 1 -- 172.21.15.45:0/3950956907 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f6a68012e10 con 0x5640d53eecb0 2024-04-16T17:08:35.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.091+0000 7f6a89ba5740 10 client.15963 fetch_fsmap learned FSMap version 26 2024-04-16T17:08:35.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.091+0000 7f6a89ba5740 1 -- 172.21.15.45:0/3950956907 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5640d546d8c0 con 0x5640d53eecb0 2024-04-16T17:08:35.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.092+0000 7f6a6e000640 1 -- 172.21.15.45:0/3950956907 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6a78011020 con 0x5640d53eecb0 2024-04-16T17:08:35.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.092+0000 7f6a6e000640 1 -- 172.21.15.45:0/3950956907 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f6a68004bf0 con 0x5640d53eecb0 2024-04-16T17:08:35.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.092+0000 7f6a89ba5740 10 client.15963 fetch_fsmap finished waiting for FSMap version 26 2024-04-16T17:08:35.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.092+0000 7f6a89ba5740 10 client.15963 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:08:35.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.092+0000 7f6a89ba5740 20 client.15963 populate_metadata read hostname 'smithi045' 2024-04-16T17:08:35.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.092+0000 7f6a89ba5740 1 --2- 172.21.15.45:0/3950956907 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x5640d546e020 0x5640d5470440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:35.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.092+0000 7f6a89ba5740 4 client.15963 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-16T17:08:35.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.092+0000 7f6a89ba5740 1 -- 172.21.15.45:0/3950956907 --> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5640d535b9b0 con 0x5640d546e020 2024-04-16T17:08:35.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.092+0000 7f6a84e00640 1 --2- 172.21.15.45:0/3950956907 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x5640d546e020 0x5640d5470440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:35.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.093+0000 7f6a84e00640 1 --2- 172.21.15.45:0/3950956907 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x5640d546e020 0x5640d5470440 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3e3ecd4da13705b2 server_cookie=b05c954ddb2cd4a4 in_seq=0 out_seq=0 2024-04-16T17:08:35.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.093+0000 7f6a6e000640 10 client.15963 ms_handle_connect on v2:172.21.15.104:6838/3660070592 2024-04-16T17:08:35.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.093+0000 7f6a6e000640 1 -- 172.21.15.45:0/3950956907 <== mds.0 v2:172.21.15.104:6838/3660070592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5640d535b9b0 con 0x5640d546e020 2024-04-16T17:08:35.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.093+0000 7f6a6e000640 10 client.15963 handle_command_reply: tid=0 2024-04-16T17:08:35.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.093+0000 7f6a89ba5740 1 client.15963 shutdown 2024-04-16T17:08:35.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.093+0000 7f6a89ba5740 1 -- 172.21.15.45:0/3950956907 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5640d53eecb0 msgr2=0x5640d53fff60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:35.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.093+0000 7f6a89ba5740 1 --2- 172.21.15.45:0/3950956907 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5640d53eecb0 0x5640d53fff60 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f6a68004700 tx=0x7f6a680207d0 comp rx=0 tx=0).stop 2024-04-16T17:08:35.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.093+0000 7f6a89ba5740 1 -- 172.21.15.45:0/3950956907 shutdown_connections 2024-04-16T17:08:35.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.093+0000 7f6a89ba5740 1 --2- 172.21.15.45:0/3950956907 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5640d53f7890 0x5640d53f0b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:35.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.093+0000 7f6a89ba5740 1 --2- 172.21.15.45:0/3950956907 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5640d53eecb0 0x5640d53fff60 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:35.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.093+0000 7f6a89ba5740 1 --2- 172.21.15.45:0/3950956907 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x5640d546e020 0x5640d5470440 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:35.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.093+0000 7f6a89ba5740 1 --2- 172.21.15.45:0/3950956907 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5640d53ee390 0x5640d53ffa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:35.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.093+0000 7f6a89ba5740 1 -- 172.21.15.45:0/3950956907 >> 172.21.15.45:0/3950956907 conn(0x5640d53e8ac0 msgr2=0x5640d53e7290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:08:35.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.093+0000 7f6a6e000640 0 client.15963 ms_handle_reset on v2:172.21.15.104:6838/3660070592 2024-04-16T17:08:35.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.094+0000 7f6a89ba5740 1 -- 172.21.15.45:0/3950956907 shutdown_connections 2024-04-16T17:08:35.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.094+0000 7f6a89ba5740 1 -- 172.21.15.45:0/3950956907 wait complete. 2024-04-16T17:08:35.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.094+0000 7f6a89ba5740 20 client.15963 trim_cache size 0 max 16384 2024-04-16T17:08:35.099 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-16T17:08:35.099 INFO:teuthology.orchestra.run.smithi045.stdout: "status": "no active scrubs running", 2024-04-16T17:08:35.099 INFO:teuthology.orchestra.run.smithi045.stdout: "scrubs": {} 2024-04-16T17:08:35.099 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-16T17:08:35.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.094+0000 7f6a85800640 1 -- 172.21.15.45:0/267078919 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f6a640a3d20 msgr2=0x7f6a640a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:35.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.094+0000 7f6a85800640 1 --2- 172.21.15.45:0/267078919 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f6a640a3d20 0x7f6a640a61e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x5640d53ede60 tx=0x7f6a700f8530 comp rx=0 tx=0).stop 2024-04-16T17:08:35.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.094+0000 7f6a85800640 1 -- 172.21.15.45:0/267078919 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6a80101000 msgr2=0x7f6a801a0650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:35.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.094+0000 7f6a85800640 1 --2- 172.21.15.45:0/267078919 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6a80101000 0x7f6a801a0650 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f6a7002c170 tx=0x7f6a70004360 comp rx=0 tx=0).stop 2024-04-16T17:08:35.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.095+0000 7f6a85800640 1 -- 172.21.15.45:0/267078919 shutdown_connections 2024-04-16T17:08:35.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.095+0000 7f6a85800640 1 --2- 172.21.15.45:0/267078919 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f6a640a3d20 0x7f6a640a61e0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:35.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.095+0000 7f6a85800640 1 --2- 172.21.15.45:0/267078919 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6a8019a7c0 0x7f6a8019ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:35.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.095+0000 7f6a85800640 1 --2- 172.21.15.45:0/267078919 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6a801060d0 0x7f6a801a0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:35.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.095+0000 7f6a85800640 1 --2- 172.21.15.45:0/267078919 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6a80101000 0x7f6a801a0650 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:35.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.095+0000 7f6a85800640 1 -- 172.21.15.45:0/267078919 >> 172.21.15.45:0/267078919 conn(0x7f6a800fc880 msgr2=0x7f6a800fdb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:08:35.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.095+0000 7f6a85800640 1 -- 172.21.15.45:0/267078919 shutdown_connections 2024-04-16T17:08:35.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.095+0000 7f6a85800640 1 -- 172.21.15.45:0/267078919 wait complete. 2024-04-16T17:08:35.120 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-16T17:08:35.120 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-16T17:08:35.218 INFO:tasks.workunit.client.0.smithi045.stdout: 1 25199 4.18 MB/sec warmup 37 sec latency 2764.090 ms 2024-04-16T17:08:35.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.331+0000 7f4b9a600640 1 -- 172.21.15.45:0/1665283430 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4b940743d0 msgr2=0x7f4b940747b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:35.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.331+0000 7f4b9a600640 1 --2- 172.21.15.45:0/1665283430 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4b940743d0 0x7f4b940747b0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f4b84009900 tx=0x7f4b8402bcc0 comp rx=0 tx=0).stop 2024-04-16T17:08:35.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.331+0000 7f4b9a600640 1 -- 172.21.15.45:0/1665283430 shutdown_connections 2024-04-16T17:08:35.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.331+0000 7f4b9a600640 1 --2- 172.21.15.45:0/1665283430 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4b94074cf0 0x7f4b9410e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:35.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.331+0000 7f4b9a600640 1 --2- 172.21.15.45:0/1665283430 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4b9410e6c0 0x7f4b94110ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:35.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.331+0000 7f4b9a600640 1 --2- 172.21.15.45:0/1665283430 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4b940743d0 0x7f4b940747b0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:35.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.331+0000 7f4b9a600640 1 -- 172.21.15.45:0/1665283430 >> 172.21.15.45:0/1665283430 conn(0x7f4b940fd930 msgr2=0x7f4b940ffd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:08:35.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.331+0000 7f4b9a600640 1 -- 172.21.15.45:0/1665283430 shutdown_connections 2024-04-16T17:08:35.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.331+0000 7f4b9a600640 1 -- 172.21.15.45:0/1665283430 wait complete. 2024-04-16T17:08:35.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.332+0000 7f4b9a600640 1 Processor -- start 2024-04-16T17:08:35.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.332+0000 7f4b9a600640 1 -- start start 2024-04-16T17:08:35.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.332+0000 7f4b9a600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4b940743d0 0x7f4b941a0620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:35.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.332+0000 7f4b9a600640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4b94074cf0 0x7f4b941a0b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:35.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.333+0000 7f4b9a600640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4b9410e6c0 0x7f4b9419a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:35.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.333+0000 7f4b9a600640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f4b94114450 con 0x7f4b940743d0 2024-04-16T17:08:35.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.333+0000 7f4b9a600640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f4b941142d0 con 0x7f4b94074cf0 2024-04-16T17:08:35.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.333+0000 7f4b9a600640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f4b941145d0 con 0x7f4b9410e6c0 2024-04-16T17:08:35.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.333+0000 7f4b99200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4b940743d0 0x7f4b941a0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:35.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.333+0000 7f4b99200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4b940743d0 0x7f4b941a0620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:40664/0 (socket says 172.21.15.45:40664) 2024-04-16T17:08:35.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.333+0000 7f4b99200640 1 -- 172.21.15.45:0/247275965 learned_addr learned my addr 172.21.15.45:0/247275965 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:08:35.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.333+0000 7f4b8be00640 1 --2- 172.21.15.45:0/247275965 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4b94074cf0 0x7f4b941a0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:35.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.333+0000 7f4b99c00640 1 --2- 172.21.15.45:0/247275965 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4b9410e6c0 0x7f4b9419a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:35.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.333+0000 7f4b99200640 1 -- 172.21.15.45:0/247275965 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4b9410e6c0 msgr2=0x7f4b9419a790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:35.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.333+0000 7f4b99200640 1 --2- 172.21.15.45:0/247275965 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4b9410e6c0 0x7f4b9419a790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:35.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.333+0000 7f4b99200640 1 -- 172.21.15.45:0/247275965 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4b94074cf0 msgr2=0x7f4b941a0b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:35.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.333+0000 7f4b99200640 1 --2- 172.21.15.45:0/247275965 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4b94074cf0 0x7f4b941a0b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:35.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.333+0000 7f4b99200640 1 -- 172.21.15.45:0/247275965 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b9419af60 con 0x7f4b940743d0 2024-04-16T17:08:35.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.333+0000 7f4b8be00640 1 --2- 172.21.15.45:0/247275965 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4b94074cf0 0x7f4b941a0b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:08:35.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.334+0000 7f4b99200640 1 --2- 172.21.15.45:0/247275965 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4b940743d0 0x7f4b941a0620 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f4b84036040 tx=0x7f4b8402cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:08:35.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.334+0000 7f4b92000640 1 -- 172.21.15.45:0/247275965 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b84043020 con 0x7f4b940743d0 2024-04-16T17:08:35.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.334+0000 7f4b92000640 1 -- 172.21.15.45:0/247275965 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4b840054e0 con 0x7f4b940743d0 2024-04-16T17:08:35.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.334+0000 7f4b92000640 1 -- 172.21.15.45:0/247275965 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b84003bb0 con 0x7f4b940743d0 2024-04-16T17:08:35.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.334+0000 7f4b9a600640 1 -- 172.21.15.45:0/247275965 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b9419b1f0 con 0x7f4b940743d0 2024-04-16T17:08:35.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.334+0000 7f4b9a600640 1 -- 172.21.15.45:0/247275965 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b941a7470 con 0x7f4b940743d0 2024-04-16T17:08:35.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.335+0000 7f4b9a600640 1 -- 172.21.15.45:0/247275965 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b94075a80 con 0x7f4b940743d0 2024-04-16T17:08:35.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.337+0000 7f4b92000640 1 -- 172.21.15.45:0/247275965 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f4b84046020 con 0x7f4b940743d0 2024-04-16T17:08:35.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.339+0000 7f4b92000640 1 --2- 172.21.15.45:0/247275965 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f4b780a3d20 0x7f4b780a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:35.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.339+0000 7f4b92000640 1 -- 172.21.15.45:0/247275965 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f4b840f2b00 con 0x7f4b940743d0 2024-04-16T17:08:35.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.339+0000 7f4b8be00640 1 --2- 172.21.15.45:0/247275965 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f4b780a3d20 0x7f4b780a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:35.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.340+0000 7f4b8be00640 1 --2- 172.21.15.45:0/247275965 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f4b780a3d20 0x7f4b780a61e0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f4b94101530 tx=0x7f4b7c005f90 comp rx=0 tx=0).ready entity=mgr.25411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:08:35.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.343+0000 7f4b92000640 1 -- 172.21.15.45:0/247275965 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4b8404d020 con 0x7f4b940743d0 2024-04-16T17:08:35.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.598+0000 7f4b9a600640 1 -- 172.21.15.45:0/247275965 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4b94068d40 con 0x7f4b940743d0 2024-04-16T17:08:35.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.600+0000 7f4b92000640 1 -- 172.21.15.45:0/247275965 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11890 (secure 0 0 0) 0x7f4b840bef00 con 0x7f4b940743d0 2024-04-16T17:08:35.601 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-16T17:08:35.601 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/1748842443","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":1748842443},{"type":"v1","addr":"172.21.15.45:6835","nonce":1748842443}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6839/1308627051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":1308627051},{"type":"v1","addr":"172.21.15.45:6839","nonce":1308627051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/1768242489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1768242489},{"type":"v1","addr":"172.21.15.117:6833","nonce":1768242489}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6833/3558902585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":3558902585},{"type":"v1","addr":"172.21.15.104:6833","nonce":3558902585}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24478,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/1308142903","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":1308142903},{"type":"v1","addr":"172.21.15.45:6837","nonce":1308142903}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24491,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6835/3430679937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":3430679937},{"type":"v1","addr":"172.21.15.104:6835","nonce":3430679937}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":26,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-16T17:03:24.663611+0000","modified":"2024-04-16T17:08:28.508390+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24581,"mds_1":24569,"mds_2":24532},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24499":{"gid":24499,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/2224123841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2224123841},{"type":"v1","addr":"172.21.15.117:6835","nonce":2224123841}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6837/2471356815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":2471356815},{"type":"v1","addr":"172.21.15.117:6837","nonce":2471356815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24524":{"gid":24524,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6837/2647756509","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":2647756509},{"type":"v1","addr":"172.21.15.104:6837","nonce":2647756509}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24532":{"gid":24532,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":13,"addr":"172.21.15.117:6839/2378675949","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":2378675949},{"type":"v1","addr":"172.21.15.117:6839","nonce":2378675949}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24569":{"gid":24569,"name":"a","rank":1,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.45:6841/3755310822","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6840","nonce":3755310822},{"type":"v1","addr":"172.21.15.45:6841","nonce":3755310822}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24581":{"gid":24581,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.104:6839/3660070592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6838","nonce":3660070592},{"type":"v1","addr":"172.21.15.104:6839","nonce":3660070592}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24581,"qdb_cluster":[24569,24532,24581]},"id":1}]} 2024-04-16T17:08:35.601 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 26 2024-04-16T17:08:35.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.602+0000 7f4b9a600640 1 -- 172.21.15.45:0/247275965 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f4b780a3d20 msgr2=0x7f4b780a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:35.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.603+0000 7f4b9a600640 1 --2- 172.21.15.45:0/247275965 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f4b780a3d20 0x7f4b780a61e0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f4b94101530 tx=0x7f4b7c005f90 comp rx=0 tx=0).stop 2024-04-16T17:08:35.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.603+0000 7f4b9a600640 1 -- 172.21.15.45:0/247275965 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4b940743d0 msgr2=0x7f4b941a0620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:35.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.603+0000 7f4b9a600640 1 --2- 172.21.15.45:0/247275965 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4b940743d0 0x7f4b941a0620 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f4b84036040 tx=0x7f4b8402cbc0 comp rx=0 tx=0).stop 2024-04-16T17:08:35.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.603+0000 7f4b9a600640 1 -- 172.21.15.45:0/247275965 shutdown_connections 2024-04-16T17:08:35.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.603+0000 7f4b9a600640 1 --2- 172.21.15.45:0/247275965 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f4b780a3d20 0x7f4b780a61e0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:35.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.603+0000 7f4b9a600640 1 --2- 172.21.15.45:0/247275965 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4b9410e6c0 0x7f4b9419a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:35.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.603+0000 7f4b9a600640 1 --2- 172.21.15.45:0/247275965 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4b94074cf0 0x7f4b941a0b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:35.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.603+0000 7f4b9a600640 1 --2- 172.21.15.45:0/247275965 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4b940743d0 0x7f4b941a0620 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:35.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.603+0000 7f4b9a600640 1 -- 172.21.15.45:0/247275965 >> 172.21.15.45:0/247275965 conn(0x7f4b940fd930 msgr2=0x7f4b940fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:08:35.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.603+0000 7f4b9a600640 1 -- 172.21.15.45:0/247275965 shutdown_connections 2024-04-16T17:08:35.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.603+0000 7f4b9a600640 1 -- 172.21.15.45:0/247275965 wait complete. 2024-04-16T17:08:35.621 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-16T17:08:35.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.841+0000 7ff31cc00640 1 -- 172.21.15.45:0/1658835098 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff318101400 msgr2=0x7ff318106b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:35.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.841+0000 7ff31cc00640 1 --2- 172.21.15.45:0/1658835098 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff318101400 0x7ff318106b30 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7ff304009900 tx=0x7ff30402bc40 comp rx=0 tx=0).stop 2024-04-16T17:08:35.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.841+0000 7ff31cc00640 1 -- 172.21.15.45:0/1658835098 shutdown_connections 2024-04-16T17:08:35.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.841+0000 7ff31cc00640 1 --2- 172.21.15.45:0/1658835098 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff318107260 0x7ff31810ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:35.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.841+0000 7ff31cc00640 1 --2- 172.21.15.45:0/1658835098 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff318101400 0x7ff318106b30 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:35.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.841+0000 7ff31cc00640 1 --2- 172.21.15.45:0/1658835098 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff318100a50 0x7ff318100e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:35.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.841+0000 7ff31cc00640 1 -- 172.21.15.45:0/1658835098 >> 172.21.15.45:0/1658835098 conn(0x7ff318076090 msgr2=0x7ff3180764a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:08:35.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.841+0000 7ff31cc00640 1 -- 172.21.15.45:0/1658835098 shutdown_connections 2024-04-16T17:08:35.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.841+0000 7ff31cc00640 1 -- 172.21.15.45:0/1658835098 wait complete. 2024-04-16T17:08:35.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.842+0000 7ff31cc00640 1 Processor -- start 2024-04-16T17:08:35.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.842+0000 7ff31cc00640 1 -- start start 2024-04-16T17:08:35.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.843+0000 7ff31cc00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff318100a50 0x7ff3181a06d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:35.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.843+0000 7ff31cc00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff318107260 0x7ff3181a0c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:35.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.843+0000 7ff31cc00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff31819a840 0x7ff31819acf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:35.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.843+0000 7ff31cc00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7ff3181122d0 con 0x7ff31819a840 2024-04-16T17:08:35.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.843+0000 7ff31cc00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7ff318112150 con 0x7ff318107260 2024-04-16T17:08:35.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.843+0000 7ff31cc00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7ff318112450 con 0x7ff318100a50 2024-04-16T17:08:35.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.843+0000 7ff317e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff31819a840 0x7ff31819acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:35.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.843+0000 7ff316a00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff318107260 0x7ff3181a0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:35.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.843+0000 7ff317e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff31819a840 0x7ff31819acf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:40682/0 (socket says 172.21.15.45:40682) 2024-04-16T17:08:35.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.843+0000 7ff317e00640 1 -- 172.21.15.45:0/293488646 learned_addr learned my addr 172.21.15.45:0/293488646 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:08:35.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.843+0000 7ff316a00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff318107260 0x7ff3181a0c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:44010/0 (socket says 172.21.15.45:44010) 2024-04-16T17:08:35.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.843+0000 7ff317400640 1 --2- 172.21.15.45:0/293488646 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff318100a50 0x7ff3181a06d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:35.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.843+0000 7ff316a00640 1 -- 172.21.15.45:0/293488646 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff318100a50 msgr2=0x7ff3181a06d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:35.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.843+0000 7ff316a00640 1 --2- 172.21.15.45:0/293488646 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff318100a50 0x7ff3181a06d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:35.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.843+0000 7ff316a00640 1 -- 172.21.15.45:0/293488646 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff31819a840 msgr2=0x7ff31819acf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:35.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.844+0000 7ff316a00640 1 --2- 172.21.15.45:0/293488646 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff31819a840 0x7ff31819acf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:35.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.844+0000 7ff316a00640 1 -- 172.21.15.45:0/293488646 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff31819b3d0 con 0x7ff318107260 2024-04-16T17:08:35.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.844+0000 7ff317e00640 1 --2- 172.21.15.45:0/293488646 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff31819a840 0x7ff31819acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:08:35.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.844+0000 7ff316a00640 1 --2- 172.21.15.45:0/293488646 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff318107260 0x7ff3181a0c10 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7ff304009230 tx=0x7ff3040042f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:08:35.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.844+0000 7ff30fe00640 1 -- 172.21.15.45:0/293488646 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff304041070 con 0x7ff318107260 2024-04-16T17:08:35.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.844+0000 7ff30fe00640 1 -- 172.21.15.45:0/293488646 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff30402cca0 con 0x7ff318107260 2024-04-16T17:08:35.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.844+0000 7ff30fe00640 1 -- 172.21.15.45:0/293488646 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff304003df0 con 0x7ff318107260 2024-04-16T17:08:35.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.844+0000 7ff31cc00640 1 -- 172.21.15.45:0/293488646 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff31819b660 con 0x7ff318107260 2024-04-16T17:08:35.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.845+0000 7ff31cc00640 1 -- 172.21.15.45:0/293488646 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3181a77b0 con 0x7ff318107260 2024-04-16T17:08:35.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.848+0000 7ff320f6c740 1 Processor -- start 2024-04-16T17:08:35.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.848+0000 7ff320f6c740 1 -- start start 2024-04-16T17:08:35.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.848+0000 7ff320f6c740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559ee71dff10 0x559ee71e02f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:35.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.848+0000 7ff320f6c740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559ee71e0830 0x559ee71e8ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:35.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.848+0000 7ff320f6c740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559ee71e9410 0x559ee71eb800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:35.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.848+0000 7ff320f6c740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x559ee7145020 con 0x559ee71e0830 2024-04-16T17:08:35.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.848+0000 7ff320f6c740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x559ee70eb460 con 0x559ee71e9410 2024-04-16T17:08:35.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.848+0000 7ff320f6c740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x559ee7140fe0 con 0x559ee71dff10 2024-04-16T17:08:35.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.848+0000 7ff317400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559ee71e0830 0x559ee71e8ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:35.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.848+0000 7ff317400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559ee71e0830 0x559ee71e8ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:40696/0 (socket says 172.21.15.45:40696) 2024-04-16T17:08:35.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.848+0000 7ff317400640 1 -- 172.21.15.45:0/500865814 learned_addr learned my addr 172.21.15.45:0/500865814 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:08:35.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.848+0000 7ff317e00640 1 --2- 172.21.15.45:0/500865814 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559ee71dff10 0x559ee71e02f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:35.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.849+0000 7ff317400640 1 -- 172.21.15.45:0/500865814 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559ee71dff10 msgr2=0x559ee71e02f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:08:35.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.849+0000 7ff317400640 1 --2- 172.21.15.45:0/500865814 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559ee71dff10 0x559ee71e02f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:35.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.849+0000 7ff317400640 1 -- 172.21.15.45:0/500865814 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559ee71e9410 msgr2=0x559ee71eb800 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-16T17:08:35.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.849+0000 7ff317400640 1 --2- 172.21.15.45:0/500865814 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559ee71e9410 0x559ee71eb800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:35.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.849+0000 7ff317400640 1 -- 172.21.15.45:0/500865814 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559ee70f3040 con 0x559ee71e0830 2024-04-16T17:08:35.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.849+0000 7ff317400640 1 --2- 172.21.15.45:0/500865814 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559ee71e0830 0x559ee71e8ed0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7ff308009d20 tx=0x7ff308011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1613273eae9eed82 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:08:35.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.849+0000 7ff30fe00640 1 -- 172.21.15.45:0/293488646 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7ff304044070 con 0x7ff318107260 2024-04-16T17:08:35.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.849+0000 7ff303e00640 1 -- 172.21.15.45:0/500865814 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff30801e070 con 0x559ee71e0830 2024-04-16T17:08:35.852 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.849+0000 7ff303e00640 1 -- 172.21.15.45:0/500865814 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff308009e10 con 0x559ee71e0830 2024-04-16T17:08:35.852 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.849+0000 7ff303e00640 1 -- 172.21.15.45:0/500865814 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff30800e9a0 con 0x559ee71e0830 2024-04-16T17:08:35.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.850+0000 7ff320f6c740 1 -- 172.21.15.45:0/500865814 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559ee71e0830 msgr2=0x559ee71e8ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:35.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.850+0000 7ff320f6c740 1 --2- 172.21.15.45:0/500865814 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559ee71e0830 0x559ee71e8ed0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7ff308009d20 tx=0x7ff308011810 comp rx=0 tx=0).stop 2024-04-16T17:08:35.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.850+0000 7ff320f6c740 1 -- 172.21.15.45:0/500865814 shutdown_connections 2024-04-16T17:08:35.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.850+0000 7ff320f6c740 1 --2- 172.21.15.45:0/500865814 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559ee71e9410 0x559ee71eb800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:35.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.850+0000 7ff320f6c740 1 --2- 172.21.15.45:0/500865814 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559ee71e0830 0x559ee71e8ed0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:35.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.850+0000 7ff320f6c740 1 --2- 172.21.15.45:0/500865814 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559ee71dff10 0x559ee71e02f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:35.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.850+0000 7ff320f6c740 1 -- 172.21.15.45:0/500865814 >> 172.21.15.45:0/500865814 conn(0x559ee71da640 msgr2=0x559ee71daa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:08:35.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.850+0000 7ff320f6c740 1 -- 172.21.15.45:0/500865814 shutdown_connections 2024-04-16T17:08:35.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.850+0000 7ff320f6c740 1 -- 172.21.15.45:0/500865814 wait complete. 2024-04-16T17:08:35.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.851+0000 7ff30fe00640 1 --2- 172.21.15.45:0/293488646 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7ff2fc0a3cb0 0x7ff2fc0a6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:35.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.851+0000 7ff320f6c740 1 -- start start 2024-04-16T17:08:35.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.851+0000 7ff30fe00640 1 -- 172.21.15.45:0/293488646 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7ff3040f26a0 con 0x7ff318107260 2024-04-16T17:08:35.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.851+0000 7ff317e00640 1 --2- 172.21.15.45:0/293488646 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7ff2fc0a3cb0 0x7ff2fc0a6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:35.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.851+0000 7ff320f6c740 1 Processor -- start 2024-04-16T17:08:35.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.852+0000 7ff317e00640 1 --2- 172.21.15.45:0/293488646 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7ff2fc0a3cb0 0x7ff2fc0a6170 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7ff3100049d0 tx=0x7ff310009500 comp rx=0 tx=0).ready entity=mgr.25411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:08:35.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.852+0000 7ff320f6c740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559ee71dff10 0x559ee71e6440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:35.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.852+0000 7ff320f6c740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559ee71e0830 0x559ee71e6980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:35.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.852+0000 7ff320f6c740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559ee71e9410 0x559ee71e6ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:35.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.852+0000 7ff320f6c740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x559ee71ec200 con 0x559ee71e9410 2024-04-16T17:08:35.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.852+0000 7ff320f6c740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x559ee70fe830 con 0x559ee71dff10 2024-04-16T17:08:35.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.852+0000 7ff320f6c740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x559ee71ec380 con 0x559ee71e0830 2024-04-16T17:08:35.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.852+0000 7ff320f6c740 10 client.0 _ll_register_callbacks cb 0x559ee7106150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:08:35.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.852+0000 7ff317e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559ee71e9410 0x559ee71e6ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:35.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.852+0000 7ff317e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559ee71e9410 0x559ee71e6ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:40700/0 (socket says 172.21.15.45:40700) 2024-04-16T17:08:35.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.852+0000 7ff317e00640 1 -- 172.21.15.45:0/2642082752 learned_addr learned my addr 172.21.15.45:0/2642082752 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:08:35.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.852+0000 7ff317400640 1 --2- 172.21.15.45:0/2642082752 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559ee71dff10 0x559ee71e6440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:35.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.852+0000 7ff316a00640 1 --2- 172.21.15.45:0/2642082752 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559ee71e0830 0x559ee71e6980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:35.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.853+0000 7ff317400640 1 -- 172.21.15.45:0/2642082752 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559ee71e0830 msgr2=0x559ee71e6980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:08:35.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.853+0000 7ff317400640 1 --2- 172.21.15.45:0/2642082752 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559ee71e0830 0x559ee71e6980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:35.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.853+0000 7ff317400640 1 -- 172.21.15.45:0/2642082752 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559ee71e9410 msgr2=0x559ee71e6ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:08:35.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.853+0000 7ff317400640 1 --2- 172.21.15.45:0/2642082752 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559ee71e9410 0x559ee71e6ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:35.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.853+0000 7ff317400640 1 -- 172.21.15.45:0/2642082752 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559ee7102920 con 0x559ee71dff10 2024-04-16T17:08:35.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.853+0000 7ff317e00640 1 --2- 172.21.15.45:0/2642082752 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559ee71e9410 0x559ee71e6ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:08:35.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.853+0000 7ff317400640 1 --2- 172.21.15.45:0/2642082752 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559ee71dff10 0x559ee71e6440 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7ff30801c040 tx=0x7ff308005720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=70b51bcf4196f456 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:08:35.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.853+0000 7ff300c00640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-04-16T17:08:35.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.853+0000 7ff300c00640 1 -- 172.21.15.45:0/2642082752 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff308027030 con 0x559ee71dff10 2024-04-16T17:08:35.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.853+0000 7ff320f6c740 1 -- 172.21.15.45:0/2642082752 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff308009e10 con 0x559ee71dff10 2024-04-16T17:08:35.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.853+0000 7ff300c00640 1 -- 172.21.15.45:0/2642082752 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7ff308004020 con 0x559ee71dff10 2024-04-16T17:08:35.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.854+0000 7ff300c00640 1 -- 172.21.15.45:0/2642082752 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ff308031050 con 0x559ee71dff10 2024-04-16T17:08:35.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.854+0000 7ff320f6c740 10 client.26135 fetch_fsmap learned FSMap version 26 2024-04-16T17:08:35.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.854+0000 7ff320f6c740 1 -- 172.21.15.45:0/2642082752 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x559ee70d07f0 con 0x559ee71dff10 2024-04-16T17:08:35.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.854+0000 7ff300c00640 1 -- 172.21.15.45:0/2642082752 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff308009e10 con 0x559ee71dff10 2024-04-16T17:08:35.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.854+0000 7ff300c00640 1 -- 172.21.15.45:0/2642082752 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7ff308004790 con 0x559ee71dff10 2024-04-16T17:08:35.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.854+0000 7ff320f6c740 10 client.26135 fetch_fsmap finished waiting for FSMap version 26 2024-04-16T17:08:35.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.854+0000 7ff320f6c740 10 client.26135 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-16T17:08:35.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.854+0000 7ff320f6c740 20 client.26135 populate_metadata read hostname 'smithi045' 2024-04-16T17:08:35.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.854+0000 7ff320f6c740 1 --2- 172.21.15.45:0/2642082752 >> [v2:172.21.15.117:6838/2378675949,v1:172.21.15.117:6839/2378675949] conn(0x559ee71f96b0 0x559ee71fbaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:35.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.854+0000 7ff320f6c740 4 client.26135 mds_command: new command op to 24532 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:08:35.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.854+0000 7ff320f6c740 1 -- 172.21.15.45:0/2642082752 --> [v2:172.21.15.117:6838/2378675949,v1:172.21.15.117:6839/2378675949] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559ee7152a40 con 0x559ee71f96b0 2024-04-16T17:08:35.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.855+0000 7ff316a00640 1 --2- 172.21.15.45:0/2642082752 >> [v2:172.21.15.117:6838/2378675949,v1:172.21.15.117:6839/2378675949] conn(0x559ee71f96b0 0x559ee71fbaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:35.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.855+0000 7ff316a00640 1 --2- 172.21.15.45:0/2642082752 >> [v2:172.21.15.117:6838/2378675949,v1:172.21.15.117:6839/2378675949] conn(0x559ee71f96b0 0x559ee71fbaa0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e97c93c43da3c44a server_cookie=18856602d9c360ac in_seq=0 out_seq=0 2024-04-16T17:08:35.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.855+0000 7ff300c00640 10 client.26135 ms_handle_connect on v2:172.21.15.117:6838/2378675949 2024-04-16T17:08:35.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.856+0000 7ff300c00640 1 -- 172.21.15.45:0/2642082752 <== mds.2 v2:172.21.15.117:6838/2378675949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559ee7152a40 con 0x559ee71f96b0 2024-04-16T17:08:35.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.856+0000 7ff300c00640 10 client.26135 handle_command_reply: tid=0 2024-04-16T17:08:35.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.856+0000 7ff320f6c740 1 client.26135 shutdown 2024-04-16T17:08:35.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.856+0000 7ff320f6c740 1 -- 172.21.15.45:0/2642082752 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559ee71dff10 msgr2=0x559ee71e6440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:35.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.856+0000 7ff320f6c740 1 --2- 172.21.15.45:0/2642082752 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559ee71dff10 0x559ee71e6440 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7ff30801c040 tx=0x7ff308005720 comp rx=0 tx=0).stop 2024-04-16T17:08:35.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.856+0000 7ff320f6c740 1 -- 172.21.15.45:0/2642082752 shutdown_connections 2024-04-16T17:08:35.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.856+0000 7ff320f6c740 1 --2- 172.21.15.45:0/2642082752 >> [v2:172.21.15.117:6838/2378675949,v1:172.21.15.117:6839/2378675949] conn(0x559ee71f96b0 0x559ee71fbaa0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:35.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.856+0000 7ff320f6c740 1 --2- 172.21.15.45:0/2642082752 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559ee71e0830 0x559ee71e6980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:35.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.856+0000 7ff320f6c740 1 --2- 172.21.15.45:0/2642082752 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559ee71e9410 0x559ee71e6ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:35.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.856+0000 7ff320f6c740 1 --2- 172.21.15.45:0/2642082752 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559ee71dff10 0x559ee71e6440 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:35.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.856+0000 7ff320f6c740 1 -- 172.21.15.45:0/2642082752 >> 172.21.15.45:0/2642082752 conn(0x559ee71da640 msgr2=0x559ee71dc390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:08:35.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.856+0000 7ff300c00640 0 client.26135 ms_handle_reset on v2:172.21.15.117:6838/2378675949 2024-04-16T17:08:35.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.856+0000 7ff320f6c740 1 -- 172.21.15.45:0/2642082752 shutdown_connections 2024-04-16T17:08:35.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.856+0000 7ff320f6c740 1 -- 172.21.15.45:0/2642082752 wait complete. 2024-04-16T17:08:35.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.857+0000 7ff320f6c740 20 client.26135 trim_cache size 0 max 16384 2024-04-16T17:08:35.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.871+0000 7ff320f6c740 1 Processor -- start 2024-04-16T17:08:35.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.871+0000 7ff320f6c740 1 -- start start 2024-04-16T17:08:35.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.871+0000 7ff320f6c740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559ee71dff10 0x559ee71e8b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:35.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.871+0000 7ff320f6c740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559ee71e0830 0x559ee71f50f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:35.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.871+0000 7ff320f6c740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559ee71e9410 0x559ee71e0c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:35.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.871+0000 7ff320f6c740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7ff3181b3c70 con 0x559ee71dff10 2024-04-16T17:08:35.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.871+0000 7ff320f6c740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7ff3181b1540 con 0x559ee71e0830 2024-04-16T17:08:35.873 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.871+0000 7ff320f6c740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7ff3181af370 con 0x559ee71e9410 2024-04-16T17:08:35.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.872+0000 7ff317400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559ee71dff10 0x559ee71e8b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:35.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.872+0000 7ff316a00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559ee71e0830 0x559ee71f50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:35.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.872+0000 7ff316a00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559ee71e0830 0x559ee71f50f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:44038/0 (socket says 172.21.15.45:44038) 2024-04-16T17:08:35.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.872+0000 7ff317400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559ee71dff10 0x559ee71e8b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:40704/0 (socket says 172.21.15.45:40704) 2024-04-16T17:08:35.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.872+0000 7ff316a00640 1 -- 172.21.15.45:0/1673408447 learned_addr learned my addr 172.21.15.45:0/1673408447 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:08:35.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.872+0000 7ff317e00640 1 --2- 172.21.15.45:0/1673408447 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559ee71e9410 0x559ee71e0c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:35.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.872+0000 7ff317400640 1 -- 172.21.15.45:0/1673408447 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559ee71e9410 msgr2=0x559ee71e0c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:08:35.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.872+0000 7ff317400640 1 --2- 172.21.15.45:0/1673408447 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559ee71e9410 0x559ee71e0c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:35.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.872+0000 7ff317400640 1 -- 172.21.15.45:0/1673408447 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559ee71e0830 msgr2=0x559ee71f50f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:08:35.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.872+0000 7ff317400640 1 --2- 172.21.15.45:0/1673408447 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559ee71e0830 0x559ee71f50f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:35.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.872+0000 7ff317400640 1 -- 172.21.15.45:0/1673408447 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559ee71e0660 con 0x559ee71dff10 2024-04-16T17:08:35.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.872+0000 7ff316a00640 1 --2- 172.21.15.45:0/1673408447 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559ee71e0830 0x559ee71f50f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:08:35.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.872+0000 7ff317400640 1 --2- 172.21.15.45:0/1673408447 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559ee71dff10 0x559ee71e8b70 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7ff308004b60 tx=0x7ff308011dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ec837461a29d194 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:08:35.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.873+0000 7ff303400640 1 -- 172.21.15.45:0/1673408447 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff308027040 con 0x559ee71dff10 2024-04-16T17:08:35.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.873+0000 7ff303400640 1 -- 172.21.15.45:0/1673408447 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff30801f040 con 0x559ee71dff10 2024-04-16T17:08:35.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.873+0000 7ff303400640 1 -- 172.21.15.45:0/1673408447 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff30801e070 con 0x559ee71dff10 2024-04-16T17:08:35.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.873+0000 7ff320f6c740 1 -- 172.21.15.45:0/1673408447 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559ee71dff10 msgr2=0x559ee71e8b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:35.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.873+0000 7ff320f6c740 1 --2- 172.21.15.45:0/1673408447 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559ee71dff10 0x559ee71e8b70 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7ff308004b60 tx=0x7ff308011dc0 comp rx=0 tx=0).stop 2024-04-16T17:08:35.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.873+0000 7ff320f6c740 1 -- 172.21.15.45:0/1673408447 shutdown_connections 2024-04-16T17:08:35.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.873+0000 7ff320f6c740 1 --2- 172.21.15.45:0/1673408447 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559ee71e9410 0x559ee71e0c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:35.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.873+0000 7ff320f6c740 1 --2- 172.21.15.45:0/1673408447 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559ee71e0830 0x559ee71f50f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:35.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.873+0000 7ff320f6c740 1 --2- 172.21.15.45:0/1673408447 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559ee71dff10 0x559ee71e8b70 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:35.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.873+0000 7ff320f6c740 1 -- 172.21.15.45:0/1673408447 >> 172.21.15.45:0/1673408447 conn(0x559ee71da640 msgr2=0x559ee71dc2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:08:35.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.873+0000 7ff320f6c740 1 -- 172.21.15.45:0/1673408447 shutdown_connections 2024-04-16T17:08:35.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.874+0000 7ff320f6c740 1 -- 172.21.15.45:0/1673408447 wait complete. 2024-04-16T17:08:35.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.874+0000 7ff320f6c740 1 -- start start 2024-04-16T17:08:35.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.874+0000 7ff320f6c740 1 Processor -- start 2024-04-16T17:08:35.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.875+0000 7ff320f6c740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559ee71dff10 0x559ee71f1b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:35.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.875+0000 7ff320f6c740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559ee71e0830 0x559ee71e2f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:35.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.875+0000 7ff320f6c740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559ee71e9410 0x559ee71e3670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:35.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.875+0000 7ff320f6c740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7ff3181afa00 con 0x559ee71e9410 2024-04-16T17:08:35.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.875+0000 7ff320f6c740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7ff3181b0e70 con 0x559ee71e0830 2024-04-16T17:08:35.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.875+0000 7ff320f6c740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7ff3181b00d0 con 0x559ee71dff10 2024-04-16T17:08:35.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.875+0000 7ff320f6c740 10 client.0 _ll_register_callbacks cb 0x7ff30801c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:08:35.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.875+0000 7ff317e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559ee71e9410 0x559ee71e3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:35.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.875+0000 7ff317e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559ee71e9410 0x559ee71e3670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:40716/0 (socket says 172.21.15.45:40716) 2024-04-16T17:08:35.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.875+0000 7ff317e00640 1 -- 172.21.15.45:0/388373864 learned_addr learned my addr 172.21.15.45:0/388373864 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:08:35.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.875+0000 7ff317400640 1 --2- 172.21.15.45:0/388373864 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559ee71dff10 0x559ee71f1b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:35.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.875+0000 7ff316a00640 1 --2- 172.21.15.45:0/388373864 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559ee71e0830 0x559ee71e2f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:35.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.876+0000 7ff317e00640 1 -- 172.21.15.45:0/388373864 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559ee71dff10 msgr2=0x559ee71f1b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:08:35.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.876+0000 7ff317e00640 1 --2- 172.21.15.45:0/388373864 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559ee71dff10 0x559ee71f1b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:35.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.876+0000 7ff317e00640 1 -- 172.21.15.45:0/388373864 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559ee71e0830 msgr2=0x559ee71e2f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:08:35.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.876+0000 7ff317e00640 1 --2- 172.21.15.45:0/388373864 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559ee71e0830 0x559ee71e2f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:35.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.876+0000 7ff317e00640 1 -- 172.21.15.45:0/388373864 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559ee6e573f0 con 0x559ee71e9410 2024-04-16T17:08:35.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.876+0000 7ff317400640 1 --2- 172.21.15.45:0/388373864 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559ee71dff10 0x559ee71f1b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:08:35.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.876+0000 7ff317e00640 1 --2- 172.21.15.45:0/388373864 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559ee71e9410 0x559ee71e3670 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7ff310012800 tx=0x7ff310004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97245df3cc996b1c server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:08:35.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.876+0000 7ff300c00640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-16T17:08:35.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.876+0000 7ff300c00640 1 -- 172.21.15.45:0/388373864 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff31002f060 con 0x559ee71e9410 2024-04-16T17:08:35.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.876+0000 7ff300c00640 1 -- 172.21.15.45:0/388373864 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7ff310011d50 con 0x559ee71e9410 2024-04-16T17:08:35.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.876+0000 7ff320f6c740 1 -- 172.21.15.45:0/388373864 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff30801f040 con 0x559ee71e9410 2024-04-16T17:08:35.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.877+0000 7ff300c00640 1 -- 172.21.15.45:0/388373864 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ff310036050 con 0x559ee71e9410 2024-04-16T17:08:35.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.877+0000 7ff320f6c740 10 client.15987 fetch_fsmap learned FSMap version 26 2024-04-16T17:08:35.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.877+0000 7ff320f6c740 1 -- 172.21.15.45:0/388373864 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x559ee6f217d0 con 0x559ee71e9410 2024-04-16T17:08:35.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.877+0000 7ff300c00640 1 -- 172.21.15.45:0/388373864 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff30801f040 con 0x559ee71e9410 2024-04-16T17:08:35.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.877+0000 7ff300c00640 1 -- 172.21.15.45:0/388373864 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7ff31001e530 con 0x559ee71e9410 2024-04-16T17:08:35.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.877+0000 7ff320f6c740 10 client.15987 fetch_fsmap finished waiting for FSMap version 26 2024-04-16T17:08:35.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.877+0000 7ff320f6c740 10 client.15987 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-16T17:08:35.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.877+0000 7ff320f6c740 20 client.15987 populate_metadata read hostname 'smithi045' 2024-04-16T17:08:35.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.877+0000 7ff320f6c740 1 --2- 172.21.15.45:0/388373864 >> [v2:172.21.15.117:6838/2378675949,v1:172.21.15.117:6839/2378675949] conn(0x559ee7260000 0x559ee7262420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:35.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.877+0000 7ff320f6c740 4 client.15987 mds_command: new command op to 24532 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T17:08:35.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.877+0000 7ff320f6c740 1 -- 172.21.15.45:0/388373864 --> [v2:172.21.15.117:6838/2378675949,v1:172.21.15.117:6839/2378675949] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559ee714c8b0 con 0x559ee7260000 2024-04-16T17:08:35.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.878+0000 7ff317400640 1 --2- 172.21.15.45:0/388373864 >> [v2:172.21.15.117:6838/2378675949,v1:172.21.15.117:6839/2378675949] conn(0x559ee7260000 0x559ee7262420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:35.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.878+0000 7ff317400640 1 --2- 172.21.15.45:0/388373864 >> [v2:172.21.15.117:6838/2378675949,v1:172.21.15.117:6839/2378675949] conn(0x559ee7260000 0x559ee7262420 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=372cb8ad556ee68c server_cookie=78a58f4b9830d252 in_seq=0 out_seq=0 2024-04-16T17:08:35.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.878+0000 7ff300c00640 10 client.15987 ms_handle_connect on v2:172.21.15.117:6838/2378675949 2024-04-16T17:08:35.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.879+0000 7ff300c00640 1 -- 172.21.15.45:0/388373864 <== mds.2 v2:172.21.15.117:6838/2378675949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559ee714c8b0 con 0x559ee7260000 2024-04-16T17:08:35.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.879+0000 7ff300c00640 10 client.15987 handle_command_reply: tid=0 2024-04-16T17:08:35.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.879+0000 7ff320f6c740 1 client.15987 shutdown 2024-04-16T17:08:35.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.879+0000 7ff320f6c740 1 -- 172.21.15.45:0/388373864 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559ee71e9410 msgr2=0x559ee71e3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:35.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.879+0000 7ff320f6c740 1 --2- 172.21.15.45:0/388373864 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559ee71e9410 0x559ee71e3670 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7ff310012800 tx=0x7ff310004a40 comp rx=0 tx=0).stop 2024-04-16T17:08:35.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.879+0000 7ff320f6c740 1 -- 172.21.15.45:0/388373864 shutdown_connections 2024-04-16T17:08:35.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.879+0000 7ff320f6c740 1 --2- 172.21.15.45:0/388373864 >> [v2:172.21.15.117:6838/2378675949,v1:172.21.15.117:6839/2378675949] conn(0x559ee7260000 0x559ee7262420 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:35.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.879+0000 7ff320f6c740 1 --2- 172.21.15.45:0/388373864 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559ee71e9410 0x559ee71e3670 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:35.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.879+0000 7ff320f6c740 1 --2- 172.21.15.45:0/388373864 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559ee71e0830 0x559ee71e2f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:35.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.879+0000 7ff320f6c740 1 --2- 172.21.15.45:0/388373864 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559ee71dff10 0x559ee71f1b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:35.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.879+0000 7ff300c00640 0 client.15987 ms_handle_reset on v2:172.21.15.117:6838/2378675949 2024-04-16T17:08:35.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.879+0000 7ff320f6c740 1 -- 172.21.15.45:0/388373864 >> 172.21.15.45:0/388373864 conn(0x559ee71da640 msgr2=0x559ee71d8e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:08:35.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.879+0000 7ff320f6c740 1 -- 172.21.15.45:0/388373864 shutdown_connections 2024-04-16T17:08:35.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.879+0000 7ff320f6c740 1 -- 172.21.15.45:0/388373864 wait complete. 2024-04-16T17:08:35.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.879+0000 7ff320f6c740 20 client.15987 trim_cache size 0 max 16384 2024-04-16T17:08:35.884 INFO:teuthology.orchestra.run.smithi045.stdout:[] 2024-04-16T17:08:35.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.880+0000 7ff31cc00640 1 -- 172.21.15.45:0/293488646 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7ff2fc0a3cb0 msgr2=0x7ff2fc0a6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:35.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.880+0000 7ff31cc00640 1 --2- 172.21.15.45:0/293488646 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7ff2fc0a3cb0 0x7ff2fc0a6170 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7ff3100049d0 tx=0x7ff310009500 comp rx=0 tx=0).stop 2024-04-16T17:08:35.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.880+0000 7ff31cc00640 1 -- 172.21.15.45:0/293488646 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff318107260 msgr2=0x7ff3181a0c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:35.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.880+0000 7ff31cc00640 1 --2- 172.21.15.45:0/293488646 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff318107260 0x7ff3181a0c10 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7ff304009230 tx=0x7ff3040042f0 comp rx=0 tx=0).stop 2024-04-16T17:08:35.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.880+0000 7ff31cc00640 1 -- 172.21.15.45:0/293488646 shutdown_connections 2024-04-16T17:08:35.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.880+0000 7ff31cc00640 1 --2- 172.21.15.45:0/293488646 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7ff2fc0a3cb0 0x7ff2fc0a6170 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:35.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.880+0000 7ff31cc00640 1 --2- 172.21.15.45:0/293488646 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff31819a840 0x7ff31819acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:35.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.880+0000 7ff31cc00640 1 --2- 172.21.15.45:0/293488646 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff318107260 0x7ff3181a0c10 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:35.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.880+0000 7ff31cc00640 1 --2- 172.21.15.45:0/293488646 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff318100a50 0x7ff3181a06d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:35.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.880+0000 7ff31cc00640 1 -- 172.21.15.45:0/293488646 >> 172.21.15.45:0/293488646 conn(0x7ff318076090 msgr2=0x7ff3180fe7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:08:35.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.880+0000 7ff31cc00640 1 -- 172.21.15.45:0/293488646 shutdown_connections 2024-04-16T17:08:35.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:35.880+0000 7ff31cc00640 1 -- 172.21.15.45:0/293488646 wait complete. 2024-04-16T17:08:35.897 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-16T17:08:36.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.109+0000 7fc697e00640 1 -- 172.21.15.45:0/1566266063 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc698102120 msgr2=0x7fc698102500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:36.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.109+0000 7fc697e00640 1 --2- 172.21.15.45:0/1566266063 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc698102120 0x7fc698102500 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fc688009860 tx=0x7fc68802bb70 comp rx=0 tx=0).stop 2024-04-16T17:08:36.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.109+0000 7fc697e00640 1 -- 172.21.15.45:0/1566266063 shutdown_connections 2024-04-16T17:08:36.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.109+0000 7fc697e00640 1 --2- 172.21.15.45:0/1566266063 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc6981071f0 0x7fc69810edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:36.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.109+0000 7fc697e00640 1 --2- 172.21.15.45:0/1566266063 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc698102ad0 0x7fc698106b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:36.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.109+0000 7fc697e00640 1 --2- 172.21.15.45:0/1566266063 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc698102120 0x7fc698102500 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:36.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.109+0000 7fc697e00640 1 -- 172.21.15.45:0/1566266063 >> 172.21.15.45:0/1566266063 conn(0x7fc6980fd9a0 msgr2=0x7fc6980ffdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:08:36.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.109+0000 7fc697e00640 1 -- 172.21.15.45:0/1566266063 shutdown_connections 2024-04-16T17:08:36.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.109+0000 7fc697e00640 1 -- 172.21.15.45:0/1566266063 wait complete. 2024-04-16T17:08:36.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.110+0000 7fc697e00640 1 Processor -- start 2024-04-16T17:08:36.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.110+0000 7fc697e00640 1 -- start start 2024-04-16T17:08:36.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.110+0000 7fc697e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc698102120 0x7fc698075f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:36.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.110+0000 7fc697e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc6981071f0 0x7fc698074300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:36.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.110+0000 7fc697e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc698074890 0x7fc698074d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:36.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.110+0000 7fc697e00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fc698112280 con 0x7fc698102120 2024-04-16T17:08:36.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.110+0000 7fc697e00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fc698112100 con 0x7fc698074890 2024-04-16T17:08:36.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.110+0000 7fc697e00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fc698112400 con 0x7fc6981071f0 2024-04-16T17:08:36.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.111+0000 7fc696a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc698102120 0x7fc698075f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:36.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.111+0000 7fc696a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc698102120 0x7fc698075f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:40730/0 (socket says 172.21.15.45:40730) 2024-04-16T17:08:36.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.111+0000 7fc696a00640 1 -- 172.21.15.45:0/4004203746 learned_addr learned my addr 172.21.15.45:0/4004203746 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:08:36.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.111+0000 7fc697400640 1 --2- 172.21.15.45:0/4004203746 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc698074890 0x7fc698074d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:36.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.111+0000 7fc68fe00640 1 --2- 172.21.15.45:0/4004203746 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc6981071f0 0x7fc698074300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:36.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.111+0000 7fc696a00640 1 -- 172.21.15.45:0/4004203746 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc6981071f0 msgr2=0x7fc698074300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:36.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.111+0000 7fc696a00640 1 --2- 172.21.15.45:0/4004203746 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc6981071f0 0x7fc698074300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:36.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.111+0000 7fc696a00640 1 -- 172.21.15.45:0/4004203746 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc698074890 msgr2=0x7fc698074d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:36.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.111+0000 7fc696a00640 1 --2- 172.21.15.45:0/4004203746 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc698074890 0x7fc698074d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:36.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.111+0000 7fc696a00640 1 -- 172.21.15.45:0/4004203746 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6981ab4e0 con 0x7fc698102120 2024-04-16T17:08:36.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.111+0000 7fc697400640 1 --2- 172.21.15.45:0/4004203746 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc698074890 0x7fc698074d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:08:36.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.111+0000 7fc696a00640 1 --2- 172.21.15.45:0/4004203746 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc698102120 0x7fc698075f00 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fc688037040 tx=0x7fc6880045b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:08:36.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.112+0000 7fc68f400640 1 -- 172.21.15.45:0/4004203746 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc688041020 con 0x7fc698102120 2024-04-16T17:08:36.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.112+0000 7fc697e00640 1 -- 172.21.15.45:0/4004203746 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6981ab770 con 0x7fc698102120 2024-04-16T17:08:36.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.112+0000 7fc68f400640 1 -- 172.21.15.45:0/4004203746 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc68802c9f0 con 0x7fc698102120 2024-04-16T17:08:36.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.112+0000 7fc68f400640 1 -- 172.21.15.45:0/4004203746 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc688039040 con 0x7fc698102120 2024-04-16T17:08:36.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.112+0000 7fc697e00640 1 -- 172.21.15.45:0/4004203746 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6981abc60 con 0x7fc698102120 2024-04-16T17:08:36.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.116+0000 7fc6a0836740 1 Processor -- start 2024-04-16T17:08:36.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.116+0000 7fc6a0836740 1 -- start start 2024-04-16T17:08:36.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.116+0000 7fc6a0836740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d5c003df10 0x55d5c003e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:36.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.116+0000 7fc6a0836740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d5c003e830 0x55d5c0046ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:36.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.117+0000 7fc6a0836740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d5c0047410 0x55d5c0049800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:36.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.117+0000 7fc6a0836740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55d5bffa3020 con 0x55d5c0047410 2024-04-16T17:08:36.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.117+0000 7fc68f400640 1 -- 172.21.15.45:0/4004203746 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fc688044020 con 0x7fc698102120 2024-04-16T17:08:36.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.117+0000 7fc6a0836740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55d5bff49460 con 0x55d5c003df10 2024-04-16T17:08:36.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.117+0000 7fc6a0836740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55d5bff9efe0 con 0x55d5c003e830 2024-04-16T17:08:36.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.117+0000 7fc68fe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d5c0047410 0x55d5c0049800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:36.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.117+0000 7fc697400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d5c003df10 0x55d5c003e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:36.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.117+0000 7fc697400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d5c003df10 0x55d5c003e2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:44074/0 (socket says 172.21.15.45:44074) 2024-04-16T17:08:36.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.117+0000 7fc697400640 1 -- 172.21.15.45:0/444320464 learned_addr learned my addr 172.21.15.45:0/444320464 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:08:36.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.117+0000 7fc696a00640 1 --2- 172.21.15.45:0/444320464 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d5c003e830 0x55d5c0046ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:36.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.117+0000 7fc697400640 1 -- 172.21.15.45:0/444320464 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d5c003e830 msgr2=0x55d5c0046ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:08:36.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.117+0000 7fc697400640 1 --2- 172.21.15.45:0/444320464 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d5c003e830 0x55d5c0046ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:36.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.117+0000 7fc697400640 1 -- 172.21.15.45:0/444320464 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d5c0047410 msgr2=0x55d5c0049800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:08:36.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.117+0000 7fc697400640 1 --2- 172.21.15.45:0/444320464 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d5c0047410 0x55d5c0049800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:36.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.117+0000 7fc697400640 1 -- 172.21.15.45:0/444320464 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d5bff51040 con 0x55d5c003df10 2024-04-16T17:08:36.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.117+0000 7fc696a00640 1 --2- 172.21.15.45:0/444320464 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d5c003e830 0x55d5c0046ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:08:36.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.118+0000 7fc697400640 1 --2- 172.21.15.45:0/444320464 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d5c003df10 0x55d5c003e2f0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fc690009f20 tx=0x7fc690010860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b874ce311749778f server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:08:36.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.118+0000 7fc687e00640 1 -- 172.21.15.45:0/444320464 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc690011700 con 0x55d5c003df10 2024-04-16T17:08:36.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.118+0000 7fc687e00640 1 -- 172.21.15.45:0/444320464 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc690011d80 con 0x55d5c003df10 2024-04-16T17:08:36.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.118+0000 7fc687e00640 1 -- 172.21.15.45:0/444320464 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc69000e3c0 con 0x55d5c003df10 2024-04-16T17:08:36.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.118+0000 7fc68f400640 1 --2- 172.21.15.45:0/4004203746 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fc67c0a3d20 0x7fc67c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:36.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.119+0000 7fc68f400640 1 -- 172.21.15.45:0/4004203746 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fc6880f2340 con 0x7fc698102120 2024-04-16T17:08:36.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.119+0000 7fc6a0836740 1 -- 172.21.15.45:0/444320464 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d5c003df10 msgr2=0x55d5c003e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:36.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.119+0000 7fc6a0836740 1 --2- 172.21.15.45:0/444320464 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d5c003df10 0x55d5c003e2f0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fc690009f20 tx=0x7fc690010860 comp rx=0 tx=0).stop 2024-04-16T17:08:36.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.119+0000 7fc696a00640 1 --2- 172.21.15.45:0/4004203746 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fc67c0a3d20 0x7fc67c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:36.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.119+0000 7fc6a0836740 1 -- 172.21.15.45:0/444320464 shutdown_connections 2024-04-16T17:08:36.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.119+0000 7fc6a0836740 1 --2- 172.21.15.45:0/444320464 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d5c003e830 0x55d5c0046ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:36.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.119+0000 7fc6a0836740 1 --2- 172.21.15.45:0/444320464 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d5c0047410 0x55d5c0049800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:36.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.119+0000 7fc6a0836740 1 --2- 172.21.15.45:0/444320464 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d5c003df10 0x55d5c003e2f0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:36.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.119+0000 7fc6a0836740 1 -- 172.21.15.45:0/444320464 >> 172.21.15.45:0/444320464 conn(0x55d5c0038640 msgr2=0x55d5c0038a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:08:36.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.119+0000 7fc6a0836740 1 -- 172.21.15.45:0/444320464 shutdown_connections 2024-04-16T17:08:36.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.119+0000 7fc6a0836740 1 -- 172.21.15.45:0/444320464 wait complete. 2024-04-16T17:08:36.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.119+0000 7fc6a0836740 1 -- start start 2024-04-16T17:08:36.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.119+0000 7fc696a00640 1 --2- 172.21.15.45:0/4004203746 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fc67c0a3d20 0x7fc67c0a61e0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x55d5c003d9e0 tx=0x7fc688005f90 comp rx=0 tx=0).ready entity=mgr.25411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:08:36.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.120+0000 7fc6a0836740 1 Processor -- start 2024-04-16T17:08:36.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.120+0000 7fc6a0836740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d5c003df10 0x55d5c00444b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:36.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.120+0000 7fc6a0836740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d5c003e830 0x55d5c00449f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:36.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.120+0000 7fc6a0836740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d5c0047410 0x55d5c0044f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:36.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.120+0000 7fc6a0836740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55d5c004a200 con 0x55d5c003df10 2024-04-16T17:08:36.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.120+0000 7fc6a0836740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55d5bff5c830 con 0x55d5c003e830 2024-04-16T17:08:36.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.120+0000 7fc6a0836740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55d5c004a380 con 0x55d5c0047410 2024-04-16T17:08:36.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.120+0000 7fc6a0836740 10 client.0 _ll_register_callbacks cb 0x55d5bff64150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:08:36.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.120+0000 7fc68fe00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d5c003e830 0x55d5c00449f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:36.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.120+0000 7fc697400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d5c003df10 0x55d5c00444b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:36.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.120+0000 7fc68fe00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d5c003e830 0x55d5c00449f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:44084/0 (socket says 172.21.15.45:44084) 2024-04-16T17:08:36.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.120+0000 7fc68fe00640 1 -- 172.21.15.45:0/3346034572 learned_addr learned my addr 172.21.15.45:0/3346034572 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:08:36.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.121+0000 7fc68fe00640 1 -- 172.21.15.45:0/3346034572 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d5c0047410 msgr2=0x55d5c0044f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:08:36.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.121+0000 7fc68fe00640 1 --2- 172.21.15.45:0/3346034572 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d5c0047410 0x55d5c0044f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:36.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.121+0000 7fc68fe00640 1 -- 172.21.15.45:0/3346034572 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d5c003df10 msgr2=0x55d5c00444b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:08:36.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.121+0000 7fc68fe00640 1 --2- 172.21.15.45:0/3346034572 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d5c003df10 0x55d5c00444b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:36.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.121+0000 7fc68fe00640 1 -- 172.21.15.45:0/3346034572 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d5bff60920 con 0x55d5c003e830 2024-04-16T17:08:36.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.121+0000 7fc68fe00640 1 --2- 172.21.15.45:0/3346034572 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d5c003e830 0x55d5c00449f0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fc69806b5e0 tx=0x7fc680011b30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e61c85b21cac79a0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:08:36.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.121+0000 7fc684c00640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-04-16T17:08:36.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.121+0000 7fc684c00640 1 -- 172.21.15.45:0/3346034572 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc68001e060 con 0x55d5c003e830 2024-04-16T17:08:36.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.121+0000 7fc684c00640 1 -- 172.21.15.45:0/3346034572 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fc68000fbc0 con 0x55d5c003e830 2024-04-16T17:08:36.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.121+0000 7fc6a0836740 1 -- 172.21.15.45:0/3346034572 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc690011d80 con 0x55d5c003e830 2024-04-16T17:08:36.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.122+0000 7fc684c00640 1 -- 172.21.15.45:0/3346034572 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fc680025050 con 0x55d5c003e830 2024-04-16T17:08:36.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.122+0000 7fc6a0836740 10 client.26153 fetch_fsmap learned FSMap version 26 2024-04-16T17:08:36.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.122+0000 7fc6a0836740 1 -- 172.21.15.45:0/3346034572 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55d5bff2e7f0 con 0x55d5c003e830 2024-04-16T17:08:36.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.122+0000 7fc684c00640 1 -- 172.21.15.45:0/3346034572 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc690011d80 con 0x55d5c003e830 2024-04-16T17:08:36.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.122+0000 7fc684c00640 1 -- 172.21.15.45:0/3346034572 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fc680019070 con 0x55d5c003e830 2024-04-16T17:08:36.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.122+0000 7fc6a0836740 10 client.26153 fetch_fsmap finished waiting for FSMap version 26 2024-04-16T17:08:36.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.122+0000 7fc6a0836740 10 client.26153 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-04-16T17:08:36.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.122+0000 7fc6a0836740 20 client.26153 populate_metadata read hostname 'smithi045' 2024-04-16T17:08:36.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.122+0000 7fc6a0836740 1 --2- 172.21.15.45:0/3346034572 >> [v2:172.21.15.45:6840/3755310822,v1:172.21.15.45:6841/3755310822] conn(0x55d5c0057850 0x55d5c0059cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:36.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.122+0000 7fc6a0836740 4 client.26153 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:08:36.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.122+0000 7fc6a0836740 1 -- 172.21.15.45:0/3346034572 --> [v2:172.21.15.45:6840/3755310822,v1:172.21.15.45:6841/3755310822] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d5bffb0a40 con 0x55d5c0057850 2024-04-16T17:08:36.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.123+0000 7fc697400640 1 --2- 172.21.15.45:0/3346034572 >> [v2:172.21.15.45:6840/3755310822,v1:172.21.15.45:6841/3755310822] conn(0x55d5c0057850 0x55d5c0059cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:36.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.123+0000 7fc697400640 1 --2- 172.21.15.45:0/3346034572 >> [v2:172.21.15.45:6840/3755310822,v1:172.21.15.45:6841/3755310822] conn(0x55d5c0057850 0x55d5c0059cd0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5c7ba942035d188a server_cookie=e9a9b688637a5ec6 in_seq=0 out_seq=0 2024-04-16T17:08:36.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.123+0000 7fc684c00640 10 client.26153 ms_handle_connect on v2:172.21.15.45:6840/3755310822 2024-04-16T17:08:36.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.124+0000 7fc684c00640 1 -- 172.21.15.45:0/3346034572 <== mds.1 v2:172.21.15.45:6840/3755310822 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d5bffb0a40 con 0x55d5c0057850 2024-04-16T17:08:36.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.124+0000 7fc684c00640 10 client.26153 handle_command_reply: tid=0 2024-04-16T17:08:36.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.124+0000 7fc6a0836740 1 client.26153 shutdown 2024-04-16T17:08:36.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.124+0000 7fc6a0836740 1 -- 172.21.15.45:0/3346034572 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d5c003e830 msgr2=0x55d5c00449f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:36.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.124+0000 7fc6a0836740 1 --2- 172.21.15.45:0/3346034572 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d5c003e830 0x55d5c00449f0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fc69806b5e0 tx=0x7fc680011b30 comp rx=0 tx=0).stop 2024-04-16T17:08:36.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.124+0000 7fc6a0836740 1 -- 172.21.15.45:0/3346034572 shutdown_connections 2024-04-16T17:08:36.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.124+0000 7fc6a0836740 1 --2- 172.21.15.45:0/3346034572 >> [v2:172.21.15.45:6840/3755310822,v1:172.21.15.45:6841/3755310822] conn(0x55d5c0057850 0x55d5c0059cd0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:36.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.124+0000 7fc6a0836740 1 --2- 172.21.15.45:0/3346034572 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d5c0047410 0x55d5c0044f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:36.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.124+0000 7fc6a0836740 1 --2- 172.21.15.45:0/3346034572 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d5c003e830 0x55d5c00449f0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:36.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.124+0000 7fc6a0836740 1 --2- 172.21.15.45:0/3346034572 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d5c003df10 0x55d5c00444b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:36.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.124+0000 7fc6a0836740 1 -- 172.21.15.45:0/3346034572 >> 172.21.15.45:0/3346034572 conn(0x55d5c0038640 msgr2=0x55d5c003a390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:08:36.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.124+0000 7fc684c00640 0 client.26153 ms_handle_reset on v2:172.21.15.45:6840/3755310822 2024-04-16T17:08:36.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.124+0000 7fc6a0836740 1 -- 172.21.15.45:0/3346034572 shutdown_connections 2024-04-16T17:08:36.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.124+0000 7fc6a0836740 1 -- 172.21.15.45:0/3346034572 wait complete. 2024-04-16T17:08:36.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.125+0000 7fc6a0836740 20 client.26153 trim_cache size 0 max 16384 2024-04-16T17:08:36.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.140+0000 7fc6a0836740 1 Processor -- start 2024-04-16T17:08:36.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.140+0000 7fc6a0836740 1 -- start start 2024-04-16T17:08:36.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.140+0000 7fc6a0836740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d5c003df10 0x55d5c0046be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:36.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.140+0000 7fc6a0836740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d5c003e830 0x55d5c00530c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:36.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.140+0000 7fc6a0836740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d5c0047410 0x55d5c005f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:36.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.140+0000 7fc6a0836740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fc6981b81e0 con 0x55d5c003df10 2024-04-16T17:08:36.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.140+0000 7fc6a0836740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fc6981b5ab0 con 0x55d5c0047410 2024-04-16T17:08:36.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.140+0000 7fc6a0836740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fc6981b38e0 con 0x55d5c003e830 2024-04-16T17:08:36.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.141+0000 7fc697400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d5c003df10 0x55d5c0046be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:36.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.141+0000 7fc697400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d5c003df10 0x55d5c0046be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:40750/0 (socket says 172.21.15.45:40750) 2024-04-16T17:08:36.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.141+0000 7fc697400640 1 -- 172.21.15.45:0/635068659 learned_addr learned my addr 172.21.15.45:0/635068659 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:08:36.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.141+0000 7fc68fe00640 1 --2- 172.21.15.45:0/635068659 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d5c003e830 0x55d5c00530c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:36.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.141+0000 7fc697400640 1 --2- 172.21.15.45:0/635068659 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d5c0047410 0x55d5c005f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:36.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.141+0000 7fc697400640 1 -- 172.21.15.45:0/635068659 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d5c003e830 msgr2=0x55d5c00530c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:08:36.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.141+0000 7fc697400640 1 --2- 172.21.15.45:0/635068659 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d5c003e830 0x55d5c00530c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:36.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.141+0000 7fc697400640 1 -- 172.21.15.45:0/635068659 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d5c0047410 msgr2=0x55d5c005f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:08:36.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.141+0000 7fc697400640 1 --2- 172.21.15.45:0/635068659 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d5c0047410 0x55d5c005f5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:36.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.141+0000 7fc697400640 1 -- 172.21.15.45:0/635068659 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d5bfcb53f0 con 0x55d5c003df10 2024-04-16T17:08:36.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.142+0000 7fc697400640 1 --2- 172.21.15.45:0/635068659 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d5c003df10 0x55d5c0046be0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fc69001ba40 tx=0x7fc69001e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9826056b701a9c server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:08:36.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.142+0000 7fc687400640 1 -- 172.21.15.45:0/635068659 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc69001b630 con 0x55d5c003df10 2024-04-16T17:08:36.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.142+0000 7fc687400640 1 -- 172.21.15.45:0/635068659 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc69001be20 con 0x55d5c003df10 2024-04-16T17:08:36.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.142+0000 7fc687400640 1 -- 172.21.15.45:0/635068659 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc690011500 con 0x55d5c003df10 2024-04-16T17:08:36.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.143+0000 7fc6a0836740 1 -- 172.21.15.45:0/635068659 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d5c003df10 msgr2=0x55d5c0046be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:36.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.143+0000 7fc6a0836740 1 --2- 172.21.15.45:0/635068659 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d5c003df10 0x55d5c0046be0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fc69001ba40 tx=0x7fc69001e040 comp rx=0 tx=0).stop 2024-04-16T17:08:36.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.143+0000 7fc6a0836740 1 -- 172.21.15.45:0/635068659 shutdown_connections 2024-04-16T17:08:36.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.143+0000 7fc6a0836740 1 --2- 172.21.15.45:0/635068659 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d5c003e830 0x55d5c00530c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:36.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.143+0000 7fc6a0836740 1 --2- 172.21.15.45:0/635068659 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d5c0047410 0x55d5c005f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:36.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.143+0000 7fc6a0836740 1 --2- 172.21.15.45:0/635068659 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d5c003df10 0x55d5c0046be0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:36.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.143+0000 7fc6a0836740 1 -- 172.21.15.45:0/635068659 >> 172.21.15.45:0/635068659 conn(0x55d5c0038640 msgr2=0x55d5c0044d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:08:36.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.143+0000 7fc6a0836740 1 -- 172.21.15.45:0/635068659 shutdown_connections 2024-04-16T17:08:36.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.143+0000 7fc6a0836740 1 -- 172.21.15.45:0/635068659 wait complete. 2024-04-16T17:08:36.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.143+0000 7fc6a0836740 1 -- start start 2024-04-16T17:08:36.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.144+0000 7fc6a0836740 1 Processor -- start 2024-04-16T17:08:36.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.144+0000 7fc6a0836740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d5c003df10 0x55d5c004b680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:36.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.144+0000 7fc6a0836740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d5c003e830 0x55d5c004bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:36.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.144+0000 7fc6a0836740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d5c0047410 0x55d5c00bce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:36.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.144+0000 7fc6a0836740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fc6981b3f70 con 0x55d5c0047410 2024-04-16T17:08:36.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.144+0000 7fc6a0836740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fc6981b53e0 con 0x55d5c003e830 2024-04-16T17:08:36.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.144+0000 7fc6a0836740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fc6981b4640 con 0x55d5c003df10 2024-04-16T17:08:36.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.144+0000 7fc6a0836740 10 client.0 _ll_register_callbacks cb 0x7fc680009870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:08:36.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.144+0000 7fc697400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d5c003df10 0x55d5c004b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:36.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.144+0000 7fc68fe00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d5c003e830 0x55d5c004bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:36.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.144+0000 7fc697400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d5c003df10 0x55d5c004b680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:54278/0 (socket says 172.21.15.45:54278) 2024-04-16T17:08:36.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.145+0000 7fc697400640 1 -- 172.21.15.45:0/260249635 learned_addr learned my addr 172.21.15.45:0/260249635 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:08:36.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.145+0000 7fc697400640 1 --2- 172.21.15.45:0/260249635 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d5c0047410 0x55d5c00bce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:36.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.145+0000 7fc68fe00640 1 -- 172.21.15.45:0/260249635 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d5c003df10 msgr2=0x55d5c004b680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:08:36.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.145+0000 7fc68fe00640 1 --2- 172.21.15.45:0/260249635 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d5c003df10 0x55d5c004b680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:36.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.145+0000 7fc68fe00640 1 -- 172.21.15.45:0/260249635 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d5c0047410 msgr2=0x55d5c00bce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:08:36.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.145+0000 7fc68fe00640 1 --2- 172.21.15.45:0/260249635 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d5c0047410 0x55d5c00bce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:36.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.145+0000 7fc68fe00640 1 -- 172.21.15.45:0/260249635 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d5bfd7f7d0 con 0x55d5c003e830 2024-04-16T17:08:36.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.145+0000 7fc68fe00640 1 --2- 172.21.15.45:0/260249635 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d5c003e830 0x55d5c004bbc0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fc680009530 tx=0x7fc680009f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a6e078231a5f5b7b server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:08:36.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.145+0000 7fc684c00640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-04-16T17:08:36.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.146+0000 7fc684c00640 1 -- 172.21.15.45:0/260249635 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc68002d070 con 0x55d5c003e830 2024-04-16T17:08:36.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.146+0000 7fc6a0836740 1 -- 172.21.15.45:0/260249635 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc69001be20 con 0x55d5c003e830 2024-04-16T17:08:36.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.146+0000 7fc684c00640 1 -- 172.21.15.45:0/260249635 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fc680032460 con 0x55d5c003e830 2024-04-16T17:08:36.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.146+0000 7fc684c00640 1 -- 172.21.15.45:0/260249635 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fc68003b050 con 0x55d5c003e830 2024-04-16T17:08:36.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.146+0000 7fc6a0836740 10 client.26159 fetch_fsmap learned FSMap version 26 2024-04-16T17:08:36.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.146+0000 7fc6a0836740 1 -- 172.21.15.45:0/260249635 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55d5c00be3f0 con 0x55d5c003e830 2024-04-16T17:08:36.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.147+0000 7fc684c00640 1 -- 172.21.15.45:0/260249635 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc69001be20 con 0x55d5c003e830 2024-04-16T17:08:36.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.147+0000 7fc684c00640 1 -- 172.21.15.45:0/260249635 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fc680019050 con 0x55d5c003e830 2024-04-16T17:08:36.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.147+0000 7fc6a0836740 10 client.26159 fetch_fsmap finished waiting for FSMap version 26 2024-04-16T17:08:36.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.147+0000 7fc6a0836740 10 client.26159 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-04-16T17:08:36.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.147+0000 7fc6a0836740 20 client.26159 populate_metadata read hostname 'smithi045' 2024-04-16T17:08:36.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.147+0000 7fc6a0836740 1 --2- 172.21.15.45:0/260249635 >> [v2:172.21.15.45:6840/3755310822,v1:172.21.15.45:6841/3755310822] conn(0x55d5c00beb40 0x55d5c00c0f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:36.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.147+0000 7fc6a0836740 4 client.26159 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T17:08:36.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.147+0000 7fc6a0836740 1 -- 172.21.15.45:0/260249635 --> [v2:172.21.15.45:6840/3755310822,v1:172.21.15.45:6841/3755310822] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d5bffaa8b0 con 0x55d5c00beb40 2024-04-16T17:08:36.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.147+0000 7fc697400640 1 --2- 172.21.15.45:0/260249635 >> [v2:172.21.15.45:6840/3755310822,v1:172.21.15.45:6841/3755310822] conn(0x55d5c00beb40 0x55d5c00c0f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:36.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.148+0000 7fc697400640 1 --2- 172.21.15.45:0/260249635 >> [v2:172.21.15.45:6840/3755310822,v1:172.21.15.45:6841/3755310822] conn(0x55d5c00beb40 0x55d5c00c0f60 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d7b167d8a16f7d77 server_cookie=793bb74c9a11699 in_seq=0 out_seq=0 2024-04-16T17:08:36.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.148+0000 7fc684c00640 10 client.26159 ms_handle_connect on v2:172.21.15.45:6840/3755310822 2024-04-16T17:08:36.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.148+0000 7fc684c00640 1 -- 172.21.15.45:0/260249635 <== mds.1 v2:172.21.15.45:6840/3755310822 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d5bffaa8b0 con 0x55d5c00beb40 2024-04-16T17:08:36.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.148+0000 7fc684c00640 10 client.26159 handle_command_reply: tid=0 2024-04-16T17:08:36.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.148+0000 7fc6a0836740 1 client.26159 shutdown 2024-04-16T17:08:36.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.148+0000 7fc6a0836740 1 -- 172.21.15.45:0/260249635 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d5c003e830 msgr2=0x55d5c004bbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:36.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.148+0000 7fc6a0836740 1 --2- 172.21.15.45:0/260249635 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d5c003e830 0x55d5c004bbc0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fc680009530 tx=0x7fc680009f90 comp rx=0 tx=0).stop 2024-04-16T17:08:36.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.148+0000 7fc6a0836740 1 -- 172.21.15.45:0/260249635 shutdown_connections 2024-04-16T17:08:36.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.148+0000 7fc6a0836740 1 --2- 172.21.15.45:0/260249635 >> [v2:172.21.15.45:6840/3755310822,v1:172.21.15.45:6841/3755310822] conn(0x55d5c00beb40 0x55d5c00c0f60 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:36.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.148+0000 7fc6a0836740 1 --2- 172.21.15.45:0/260249635 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d5c0047410 0x55d5c00bce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:36.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.148+0000 7fc6a0836740 1 --2- 172.21.15.45:0/260249635 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d5c003e830 0x55d5c004bbc0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:36.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.148+0000 7fc6a0836740 1 --2- 172.21.15.45:0/260249635 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d5c003df10 0x55d5c004b680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:36.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.148+0000 7fc6a0836740 1 -- 172.21.15.45:0/260249635 >> 172.21.15.45:0/260249635 conn(0x55d5c0038640 msgr2=0x55d5c0044470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:08:36.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.148+0000 7fc684c00640 0 client.26159 ms_handle_reset on v2:172.21.15.45:6840/3755310822 2024-04-16T17:08:36.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.149+0000 7fc6a0836740 1 -- 172.21.15.45:0/260249635 shutdown_connections 2024-04-16T17:08:36.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.149+0000 7fc6a0836740 1 -- 172.21.15.45:0/260249635 wait complete. 2024-04-16T17:08:36.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.149+0000 7fc6a0836740 20 client.26159 trim_cache size 0 max 16384 2024-04-16T17:08:36.153 INFO:teuthology.orchestra.run.smithi045.stdout:[] 2024-04-16T17:08:36.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.149+0000 7fc697e00640 1 -- 172.21.15.45:0/4004203746 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fc67c0a3d20 msgr2=0x7fc67c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:36.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.149+0000 7fc697e00640 1 --2- 172.21.15.45:0/4004203746 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fc67c0a3d20 0x7fc67c0a61e0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x55d5c003d9e0 tx=0x7fc688005f90 comp rx=0 tx=0).stop 2024-04-16T17:08:36.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.149+0000 7fc697e00640 1 -- 172.21.15.45:0/4004203746 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc698102120 msgr2=0x7fc698075f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:36.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.149+0000 7fc697e00640 1 --2- 172.21.15.45:0/4004203746 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc698102120 0x7fc698075f00 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fc688037040 tx=0x7fc6880045b0 comp rx=0 tx=0).stop 2024-04-16T17:08:36.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.149+0000 7fc697e00640 1 -- 172.21.15.45:0/4004203746 shutdown_connections 2024-04-16T17:08:36.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.149+0000 7fc697e00640 1 --2- 172.21.15.45:0/4004203746 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fc67c0a3d20 0x7fc67c0a61e0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:36.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.149+0000 7fc697e00640 1 --2- 172.21.15.45:0/4004203746 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc6981071f0 0x7fc698074300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:36.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.149+0000 7fc697e00640 1 --2- 172.21.15.45:0/4004203746 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc698074890 0x7fc698074d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:36.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.149+0000 7fc697e00640 1 --2- 172.21.15.45:0/4004203746 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc698102120 0x7fc698075f00 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:36.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.149+0000 7fc697e00640 1 -- 172.21.15.45:0/4004203746 >> 172.21.15.45:0/4004203746 conn(0x7fc6980fd9a0 msgr2=0x7fc6980ffd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:08:36.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.150+0000 7fc697e00640 1 -- 172.21.15.45:0/4004203746 shutdown_connections 2024-04-16T17:08:36.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.150+0000 7fc697e00640 1 -- 172.21.15.45:0/4004203746 wait complete. 2024-04-16T17:08:36.169 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-16T17:08:36.218 INFO:tasks.workunit.client.0.smithi045.stdout: 1 25199 4.07 MB/sec warmup 38 sec latency 3764.150 ms 2024-04-16T17:08:36.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.381+0000 7feaefe00640 1 -- 172.21.15.45:0/2868834305 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7feaf0074d50 msgr2=0x7feaf010e1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:36.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.381+0000 7feaefe00640 1 --2- 172.21.15.45:0/2868834305 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7feaf0074d50 0x7feaf010e1a0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7feadc009900 tx=0x7feadc02bcc0 comp rx=0 tx=0).stop 2024-04-16T17:08:36.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.381+0000 7feaefe00640 1 -- 172.21.15.45:0/2868834305 shutdown_connections 2024-04-16T17:08:36.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.381+0000 7feaefe00640 1 --2- 172.21.15.45:0/2868834305 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feaf010e6e0 0x7feaf0110b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:36.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.381+0000 7feaefe00640 1 --2- 172.21.15.45:0/2868834305 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7feaf0074d50 0x7feaf010e1a0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:36.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.381+0000 7feaefe00640 1 --2- 172.21.15.45:0/2868834305 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feaf0074430 0x7feaf0074810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:36.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.381+0000 7feaefe00640 1 -- 172.21.15.45:0/2868834305 >> 172.21.15.45:0/2868834305 conn(0x7feaf00fd9d0 msgr2=0x7feaf00ffdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:08:36.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.381+0000 7feaefe00640 1 -- 172.21.15.45:0/2868834305 shutdown_connections 2024-04-16T17:08:36.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.381+0000 7feaefe00640 1 -- 172.21.15.45:0/2868834305 wait complete. 2024-04-16T17:08:36.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.382+0000 7feaefe00640 1 Processor -- start 2024-04-16T17:08:36.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.383+0000 7feaefe00640 1 -- start start 2024-04-16T17:08:36.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.383+0000 7feaefe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feaf0074430 0x7feaf01a06a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:36.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.383+0000 7feaefe00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feaf0074d50 0x7feaf01a0be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:36.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.383+0000 7feaefe00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7feaf010e6e0 0x7feaf019a810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:36.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.383+0000 7feaefe00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7feaf0114470 con 0x7feaf0074430 2024-04-16T17:08:36.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.383+0000 7feaefe00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7feaf01142f0 con 0x7feaf010e6e0 2024-04-16T17:08:36.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.383+0000 7feaefe00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7feaf01145f0 con 0x7feaf0074d50 2024-04-16T17:08:36.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.383+0000 7feaeea00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feaf0074430 0x7feaf01a06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:36.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.383+0000 7feaeea00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feaf0074430 0x7feaf01a06a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:40788/0 (socket says 172.21.15.45:40788) 2024-04-16T17:08:36.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.383+0000 7feaef400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7feaf010e6e0 0x7feaf019a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:36.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.383+0000 7feaeea00640 1 -- 172.21.15.45:0/638106821 learned_addr learned my addr 172.21.15.45:0/638106821 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:08:36.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.383+0000 7feaee000640 1 --2- 172.21.15.45:0/638106821 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feaf0074d50 0x7feaf01a0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:36.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.384+0000 7feaef400640 1 -- 172.21.15.45:0/638106821 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feaf0074d50 msgr2=0x7feaf01a0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:36.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.384+0000 7feaef400640 1 --2- 172.21.15.45:0/638106821 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feaf0074d50 0x7feaf01a0be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:36.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.384+0000 7feaef400640 1 -- 172.21.15.45:0/638106821 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feaf0074430 msgr2=0x7feaf01a06a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:36.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.384+0000 7feaef400640 1 --2- 172.21.15.45:0/638106821 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feaf0074430 0x7feaf01a06a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:36.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.384+0000 7feaef400640 1 -- 172.21.15.45:0/638106821 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feaf019afe0 con 0x7feaf010e6e0 2024-04-16T17:08:36.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.384+0000 7feaef400640 1 --2- 172.21.15.45:0/638106821 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7feaf010e6e0 0x7feaf019a810 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7feae8009fd0 tx=0x7feae800ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:08:36.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.384+0000 7feaf4a00640 1 -- 172.21.15.45:0/638106821 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feae800ce20 con 0x7feaf010e6e0 2024-04-16T17:08:36.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.384+0000 7feaf4a00640 1 -- 172.21.15.45:0/638106821 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feae8005020 con 0x7feaf010e6e0 2024-04-16T17:08:36.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.384+0000 7feaefe00640 1 -- 172.21.15.45:0/638106821 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feaf019b2d0 con 0x7feaf010e6e0 2024-04-16T17:08:36.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.384+0000 7feaf4a00640 1 -- 172.21.15.45:0/638106821 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feae8010720 con 0x7feaf010e6e0 2024-04-16T17:08:36.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.385+0000 7feaefe00640 1 -- 172.21.15.45:0/638106821 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feaf01a74f0 con 0x7feaf010e6e0 2024-04-16T17:08:36.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.388+0000 7feaf8b90740 1 Processor -- start 2024-04-16T17:08:36.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.389+0000 7feaf4a00640 1 -- 172.21.15.45:0/638106821 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7feae8005540 con 0x7feaf010e6e0 2024-04-16T17:08:36.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.389+0000 7feaf8b90740 1 -- start start 2024-04-16T17:08:36.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.389+0000 7feaf8b90740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557538d47f10 0x557538d482f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:36.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.389+0000 7feaf8b90740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557538d48830 0x557538d50ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:36.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.389+0000 7feaf8b90740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557538d51410 0x557538d53800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:36.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.389+0000 7feaf8b90740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x557538cad020 con 0x557538d47f10 2024-04-16T17:08:36.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.389+0000 7feaf8b90740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x557538c53460 con 0x557538d48830 2024-04-16T17:08:36.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.389+0000 7feaf8b90740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x557538ca8fe0 con 0x557538d51410 2024-04-16T17:08:36.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.390+0000 7feaee000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557538d47f10 0x557538d482f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:36.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.390+0000 7feaeea00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557538d48830 0x557538d50ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:36.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.390+0000 7feaee000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557538d47f10 0x557538d482f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:40800/0 (socket says 172.21.15.45:40800) 2024-04-16T17:08:36.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.390+0000 7feaee000640 1 -- 172.21.15.45:0/443201511 learned_addr learned my addr 172.21.15.45:0/443201511 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:08:36.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.390+0000 7feaee000640 1 --2- 172.21.15.45:0/443201511 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557538d51410 0x557538d53800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:36.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.390+0000 7feaeea00640 1 -- 172.21.15.45:0/443201511 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557538d51410 msgr2=0x557538d53800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:08:36.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.390+0000 7feaeea00640 1 --2- 172.21.15.45:0/443201511 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557538d51410 0x557538d53800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:36.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.390+0000 7feaeea00640 1 -- 172.21.15.45:0/443201511 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557538d47f10 msgr2=0x557538d482f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:08:36.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.390+0000 7feaeea00640 1 --2- 172.21.15.45:0/443201511 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557538d47f10 0x557538d482f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:36.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.390+0000 7feaeea00640 1 -- 172.21.15.45:0/443201511 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557538c5b040 con 0x557538d48830 2024-04-16T17:08:36.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.390+0000 7feaee000640 1 --2- 172.21.15.45:0/443201511 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557538d47f10 0x557538d482f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:08:36.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.391+0000 7feaf4a00640 1 --2- 172.21.15.45:0/638106821 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fead40a3da0 0x7fead40a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:36.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.391+0000 7feaf4a00640 1 -- 172.21.15.45:0/638106821 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7feae80d1140 con 0x7feaf010e6e0 2024-04-16T17:08:36.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.391+0000 7feaeea00640 1 --2- 172.21.15.45:0/443201511 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557538d48830 0x557538d50ed0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7feae0009cf0 tx=0x7feae0011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=315d74f197dc183e server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:08:36.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.391+0000 7feaee000640 1 --2- 172.21.15.45:0/638106821 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fead40a3da0 0x7fead40a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:36.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.391+0000 7feadbe00640 1 -- 172.21.15.45:0/443201511 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feae0005220 con 0x557538d48830 2024-04-16T17:08:36.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.391+0000 7feadbe00640 1 -- 172.21.15.45:0/443201511 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feae0004c80 con 0x557538d48830 2024-04-16T17:08:36.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.391+0000 7feadbe00640 1 -- 172.21.15.45:0/443201511 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feae0019970 con 0x557538d48830 2024-04-16T17:08:36.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.392+0000 7feaf8b90740 1 -- 172.21.15.45:0/443201511 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557538d48830 msgr2=0x557538d50ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:36.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.392+0000 7feaee000640 1 --2- 172.21.15.45:0/638106821 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fead40a3da0 0x7fead40a6260 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7feadc005c40 tx=0x7feadc005b50 comp rx=0 tx=0).ready entity=mgr.25411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:08:36.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.392+0000 7feaf8b90740 1 --2- 172.21.15.45:0/443201511 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557538d48830 0x557538d50ed0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7feae0009cf0 tx=0x7feae0011810 comp rx=0 tx=0).stop 2024-04-16T17:08:36.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.392+0000 7feaf8b90740 1 -- 172.21.15.45:0/443201511 shutdown_connections 2024-04-16T17:08:36.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.392+0000 7feaf8b90740 1 --2- 172.21.15.45:0/443201511 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557538d51410 0x557538d53800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:36.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.392+0000 7feaf8b90740 1 --2- 172.21.15.45:0/443201511 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557538d48830 0x557538d50ed0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:36.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.392+0000 7feaf8b90740 1 --2- 172.21.15.45:0/443201511 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557538d47f10 0x557538d482f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:36.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.392+0000 7feaf8b90740 1 -- 172.21.15.45:0/443201511 >> 172.21.15.45:0/443201511 conn(0x557538d42640 msgr2=0x557538d42a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:08:36.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.392+0000 7feaf8b90740 1 -- 172.21.15.45:0/443201511 shutdown_connections 2024-04-16T17:08:36.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.392+0000 7feaf8b90740 1 -- 172.21.15.45:0/443201511 wait complete. 2024-04-16T17:08:36.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.393+0000 7feaf8b90740 1 -- start start 2024-04-16T17:08:36.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.393+0000 7feaf8b90740 1 Processor -- start 2024-04-16T17:08:36.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.393+0000 7feaf8b90740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557538d47f10 0x557538d4e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:36.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.393+0000 7feaf8b90740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557538d48830 0x557538d4e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:36.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.393+0000 7feaf8b90740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557538d51410 0x557538d4ef20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:36.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.393+0000 7feaf8b90740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x557538d54200 con 0x557538d47f10 2024-04-16T17:08:36.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.393+0000 7feaf8b90740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x557538c66830 con 0x557538d51410 2024-04-16T17:08:36.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.393+0000 7feaf8b90740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x557538d54380 con 0x557538d48830 2024-04-16T17:08:36.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.393+0000 7feaf8b90740 10 client.0 _ll_register_callbacks cb 0x557538c6e150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:08:36.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.394+0000 7feaeea00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557538d47f10 0x557538d4e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:36.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.394+0000 7feaeea00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557538d47f10 0x557538d4e4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:40804/0 (socket says 172.21.15.45:40804) 2024-04-16T17:08:36.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.394+0000 7feaef400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557538d51410 0x557538d4ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:36.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.394+0000 7feaeea00640 1 -- 172.21.15.45:0/1633967142 learned_addr learned my addr 172.21.15.45:0/1633967142 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:08:36.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.394+0000 7feaee000640 1 --2- 172.21.15.45:0/1633967142 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557538d48830 0x557538d4e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:36.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.394+0000 7feaef400640 1 -- 172.21.15.45:0/1633967142 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557538d48830 msgr2=0x557538d4e9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:08:36.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.394+0000 7feaef400640 1 --2- 172.21.15.45:0/1633967142 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557538d48830 0x557538d4e9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:36.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.394+0000 7feaef400640 1 -- 172.21.15.45:0/1633967142 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557538d47f10 msgr2=0x557538d4e4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:08:36.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.394+0000 7feaef400640 1 --2- 172.21.15.45:0/1633967142 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557538d47f10 0x557538d4e4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:36.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.394+0000 7feaef400640 1 -- 172.21.15.45:0/1633967142 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557538c6a920 con 0x557538d51410 2024-04-16T17:08:36.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.394+0000 7feaeea00640 1 --2- 172.21.15.45:0/1633967142 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557538d47f10 0x557538d4e4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:08:36.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.394+0000 7feaef400640 1 --2- 172.21.15.45:0/1633967142 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557538d51410 0x557538d4ef20 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7feae80ce520 tx=0x7feae809cd70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eed81eb9a2359540 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:08:36.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.394+0000 7fead8c00640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-04-16T17:08:36.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.395+0000 7fead8c00640 1 -- 172.21.15.45:0/1633967142 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feae809cfd0 con 0x557538d51410 2024-04-16T17:08:36.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.395+0000 7feaf8b90740 1 -- 172.21.15.45:0/1633967142 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feae0004c80 con 0x557538d51410 2024-04-16T17:08:36.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.395+0000 7fead8c00640 1 -- 172.21.15.45:0/1633967142 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7feae809d170 con 0x557538d51410 2024-04-16T17:08:36.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.395+0000 7fead8c00640 1 -- 172.21.15.45:0/1633967142 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7feae80b6260 con 0x557538d51410 2024-04-16T17:08:36.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.395+0000 7feaf8b90740 10 client.26171 fetch_fsmap learned FSMap version 26 2024-04-16T17:08:36.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.395+0000 7feaf8b90740 1 -- 172.21.15.45:0/1633967142 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x557538c387f0 con 0x557538d51410 2024-04-16T17:08:36.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.395+0000 7fead8c00640 1 -- 172.21.15.45:0/1633967142 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feae0004c80 con 0x557538d51410 2024-04-16T17:08:36.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.395+0000 7fead8c00640 1 -- 172.21.15.45:0/1633967142 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7feae807bc00 con 0x557538d51410 2024-04-16T17:08:36.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.395+0000 7feaf8b90740 10 client.26171 fetch_fsmap finished waiting for FSMap version 26 2024-04-16T17:08:36.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.395+0000 7feaf8b90740 10 client.26171 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:08:36.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.395+0000 7feaf8b90740 20 client.26171 populate_metadata read hostname 'smithi045' 2024-04-16T17:08:36.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.395+0000 7feaf8b90740 1 --2- 172.21.15.45:0/1633967142 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x557538d617f0 0x557538d63c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:36.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.395+0000 7feaf8b90740 4 client.26171 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:08:36.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.395+0000 7feaf8b90740 1 -- 172.21.15.45:0/1633967142 --> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557538cbaa40 con 0x557538d617f0 2024-04-16T17:08:36.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.396+0000 7feaeea00640 1 --2- 172.21.15.45:0/1633967142 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x557538d617f0 0x557538d63c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:36.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.396+0000 7feaeea00640 1 --2- 172.21.15.45:0/1633967142 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x557538d617f0 0x557538d63c70 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3517f4c99b309881 server_cookie=35ae2d042562c984 in_seq=0 out_seq=0 2024-04-16T17:08:36.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.396+0000 7fead8c00640 10 client.26171 ms_handle_connect on v2:172.21.15.104:6838/3660070592 2024-04-16T17:08:36.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.397+0000 7fead8c00640 1 -- 172.21.15.45:0/1633967142 <== mds.0 v2:172.21.15.104:6838/3660070592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557538cbaa40 con 0x557538d617f0 2024-04-16T17:08:36.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.397+0000 7fead8c00640 10 client.26171 handle_command_reply: tid=0 2024-04-16T17:08:36.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.397+0000 7feaf8b90740 1 client.26171 shutdown 2024-04-16T17:08:36.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.397+0000 7feaf8b90740 1 -- 172.21.15.45:0/1633967142 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557538d51410 msgr2=0x557538d4ef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:36.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.397+0000 7feaf8b90740 1 --2- 172.21.15.45:0/1633967142 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557538d51410 0x557538d4ef20 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7feae80ce520 tx=0x7feae809cd70 comp rx=0 tx=0).stop 2024-04-16T17:08:36.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.397+0000 7feaf8b90740 1 -- 172.21.15.45:0/1633967142 shutdown_connections 2024-04-16T17:08:36.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.397+0000 7feaf8b90740 1 --2- 172.21.15.45:0/1633967142 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x557538d617f0 0x557538d63c70 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:36.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.397+0000 7feaf8b90740 1 --2- 172.21.15.45:0/1633967142 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557538d48830 0x557538d4e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:36.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.397+0000 7feaf8b90740 1 --2- 172.21.15.45:0/1633967142 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557538d51410 0x557538d4ef20 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:36.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.397+0000 7fead8c00640 0 client.26171 ms_handle_reset on v2:172.21.15.104:6838/3660070592 2024-04-16T17:08:36.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.397+0000 7feaf8b90740 1 --2- 172.21.15.45:0/1633967142 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557538d47f10 0x557538d4e4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:36.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.397+0000 7feaf8b90740 1 -- 172.21.15.45:0/1633967142 >> 172.21.15.45:0/1633967142 conn(0x557538d42640 msgr2=0x557538d44390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:08:36.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.398+0000 7feaf8b90740 1 -- 172.21.15.45:0/1633967142 shutdown_connections 2024-04-16T17:08:36.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.398+0000 7feaf8b90740 1 -- 172.21.15.45:0/1633967142 wait complete. 2024-04-16T17:08:36.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.398+0000 7feaf8b90740 20 client.26171 trim_cache size 0 max 16384 2024-04-16T17:08:36.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.412+0000 7feaf8b90740 1 Processor -- start 2024-04-16T17:08:36.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.412+0000 7feaf8b90740 1 -- start start 2024-04-16T17:08:36.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.413+0000 7feaf8b90740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557538d47f10 0x557538d50bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:36.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.413+0000 7feaf8b90740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557538d48830 0x557538d5d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:36.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.413+0000 7feaf8b90740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557538d51410 0x557538d44180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:36.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.413+0000 7feaf8b90740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7feaf01b3d00 con 0x557538d47f10 2024-04-16T17:08:36.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.413+0000 7feaf8b90740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7feaf01b15d0 con 0x557538d51410 2024-04-16T17:08:36.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.413+0000 7feaf8b90740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7feaf01af400 con 0x557538d48830 2024-04-16T17:08:36.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.413+0000 7feaeea00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557538d47f10 0x557538d50bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:36.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.413+0000 7feaef400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557538d51410 0x557538d44180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:36.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.413+0000 7feaeea00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557538d47f10 0x557538d50bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:40814/0 (socket says 172.21.15.45:40814) 2024-04-16T17:08:36.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.413+0000 7feaeea00640 1 -- 172.21.15.45:0/4009883850 learned_addr learned my addr 172.21.15.45:0/4009883850 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:08:36.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.413+0000 7feaee000640 1 --2- 172.21.15.45:0/4009883850 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557538d48830 0x557538d5d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:36.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.413+0000 7feaeea00640 1 -- 172.21.15.45:0/4009883850 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557538d48830 msgr2=0x557538d5d0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:08:36.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.413+0000 7feaeea00640 1 --2- 172.21.15.45:0/4009883850 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557538d48830 0x557538d5d0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:36.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.413+0000 7feaeea00640 1 -- 172.21.15.45:0/4009883850 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557538d51410 msgr2=0x557538d44180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:08:36.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.414+0000 7feaeea00640 1 --2- 172.21.15.45:0/4009883850 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557538d51410 0x557538d44180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:36.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.414+0000 7feaeea00640 1 -- 172.21.15.45:0/4009883850 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5575389bf3f0 con 0x557538d47f10 2024-04-16T17:08:36.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.414+0000 7feaef400640 1 --2- 172.21.15.45:0/4009883850 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557538d51410 0x557538d44180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:08:36.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.414+0000 7feaeea00640 1 --2- 172.21.15.45:0/4009883850 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557538d47f10 0x557538d50bd0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7feae0009fd0 tx=0x7feae00095e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a98f98defebf0894 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:08:36.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.414+0000 7feadb400640 1 -- 172.21.15.45:0/4009883850 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feae002d530 con 0x557538d47f10 2024-04-16T17:08:36.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.414+0000 7feadb400640 1 -- 172.21.15.45:0/4009883850 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feae0033070 con 0x557538d47f10 2024-04-16T17:08:36.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.414+0000 7feadb400640 1 -- 172.21.15.45:0/4009883850 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feae0024410 con 0x557538d47f10 2024-04-16T17:08:36.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.415+0000 7feaf8b90740 1 -- 172.21.15.45:0/4009883850 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557538d47f10 msgr2=0x557538d50bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:36.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.415+0000 7feaf8b90740 1 --2- 172.21.15.45:0/4009883850 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557538d47f10 0x557538d50bd0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7feae0009fd0 tx=0x7feae00095e0 comp rx=0 tx=0).stop 2024-04-16T17:08:36.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.415+0000 7feaf8b90740 1 -- 172.21.15.45:0/4009883850 shutdown_connections 2024-04-16T17:08:36.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.415+0000 7feaf8b90740 1 --2- 172.21.15.45:0/4009883850 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557538d48830 0x557538d5d0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:36.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.415+0000 7feaf8b90740 1 --2- 172.21.15.45:0/4009883850 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557538d51410 0x557538d44180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:36.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.415+0000 7feaf8b90740 1 --2- 172.21.15.45:0/4009883850 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557538d47f10 0x557538d50bd0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:36.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.415+0000 7feaf8b90740 1 -- 172.21.15.45:0/4009883850 >> 172.21.15.45:0/4009883850 conn(0x557538d42640 msgr2=0x557538d43a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:08:36.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.415+0000 7feaf8b90740 1 -- 172.21.15.45:0/4009883850 shutdown_connections 2024-04-16T17:08:36.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.415+0000 7feaf8b90740 1 -- 172.21.15.45:0/4009883850 wait complete. 2024-04-16T17:08:36.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.415+0000 7feaf8b90740 1 -- start start 2024-04-16T17:08:36.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.416+0000 7feaf8b90740 1 Processor -- start 2024-04-16T17:08:36.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.416+0000 7feaf8b90740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557538d47f10 0x557538d595d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:36.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.416+0000 7feaf8b90740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557538d48830 0x557538d59b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:36.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.416+0000 7feaf8b90740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557538d51410 0x557538d4acd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:36.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.416+0000 7feaf8b90740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7feaf01afa90 con 0x557538d48830 2024-04-16T17:08:36.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.416+0000 7feaf8b90740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7feaf01b0f00 con 0x557538d47f10 2024-04-16T17:08:36.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.416+0000 7feaf8b90740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7feaf01b0160 con 0x557538d51410 2024-04-16T17:08:36.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.416+0000 7feaf8b90740 10 client.0 _ll_register_callbacks cb 0x7feae8099010 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:08:36.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.416+0000 7feaee000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557538d48830 0x557538d59b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:36.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.416+0000 7feaee000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557538d48830 0x557538d59b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:40816/0 (socket says 172.21.15.45:40816) 2024-04-16T17:08:36.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.416+0000 7feaee000640 1 -- 172.21.15.45:0/813472161 learned_addr learned my addr 172.21.15.45:0/813472161 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:08:36.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.416+0000 7feaeea00640 1 --2- 172.21.15.45:0/813472161 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557538d47f10 0x557538d595d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:36.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.416+0000 7feaef400640 1 --2- 172.21.15.45:0/813472161 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557538d51410 0x557538d4acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:36.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.417+0000 7feaee000640 1 -- 172.21.15.45:0/813472161 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557538d51410 msgr2=0x557538d4acd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:08:36.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.417+0000 7feaee000640 1 --2- 172.21.15.45:0/813472161 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557538d51410 0x557538d4acd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:36.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.417+0000 7feaee000640 1 -- 172.21.15.45:0/813472161 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557538d47f10 msgr2=0x557538d595d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:08:36.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.417+0000 7feaee000640 1 --2- 172.21.15.45:0/813472161 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557538d47f10 0x557538d595d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:36.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.417+0000 7feaee000640 1 -- 172.21.15.45:0/813472161 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557538a897d0 con 0x557538d48830 2024-04-16T17:08:36.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.418+0000 7feaee000640 1 --2- 172.21.15.45:0/813472161 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557538d48830 0x557538d59b10 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7feadc000c00 tx=0x7feadc02bcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19dfbaa977310db0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:08:36.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.418+0000 7fead8c00640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-16T17:08:36.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.418+0000 7fead8c00640 1 -- 172.21.15.45:0/813472161 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feadc056030 con 0x557538d48830 2024-04-16T17:08:36.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.418+0000 7fead8c00640 1 -- 172.21.15.45:0/813472161 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7feadc03a040 con 0x557538d48830 2024-04-16T17:08:36.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.418+0000 7feaf8b90740 1 -- 172.21.15.45:0/813472161 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feae0033070 con 0x557538d48830 2024-04-16T17:08:36.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.418+0000 7fead8c00640 1 -- 172.21.15.45:0/813472161 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7feadc004ef0 con 0x557538d48830 2024-04-16T17:08:36.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.418+0000 7feaf8b90740 10 client.16023 fetch_fsmap learned FSMap version 26 2024-04-16T17:08:36.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.418+0000 7feaf8b90740 1 -- 172.21.15.45:0/813472161 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x557538dc7880 con 0x557538d48830 2024-04-16T17:08:36.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.419+0000 7fead8c00640 1 -- 172.21.15.45:0/813472161 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feae0033070 con 0x557538d48830 2024-04-16T17:08:36.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.419+0000 7fead8c00640 1 -- 172.21.15.45:0/813472161 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7feadc050040 con 0x557538d48830 2024-04-16T17:08:36.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.419+0000 7feaf8b90740 10 client.16023 fetch_fsmap finished waiting for FSMap version 26 2024-04-16T17:08:36.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.419+0000 7feaf8b90740 10 client.16023 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:08:36.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.419+0000 7feaf8b90740 20 client.16023 populate_metadata read hostname 'smithi045' 2024-04-16T17:08:36.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.419+0000 7feaf8b90740 1 --2- 172.21.15.45:0/813472161 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x557538dc80e0 0x557538dca500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:36.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.419+0000 7feaf8b90740 4 client.16023 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T17:08:36.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.419+0000 7feaf8b90740 1 -- 172.21.15.45:0/813472161 --> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557538cb48b0 con 0x557538dc80e0 2024-04-16T17:08:36.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.419+0000 7feaeea00640 1 --2- 172.21.15.45:0/813472161 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x557538dc80e0 0x557538dca500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:36.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.420+0000 7feaeea00640 1 --2- 172.21.15.45:0/813472161 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x557538dc80e0 0x557538dca500 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=41048f15208a645c server_cookie=132fd11703d18afb in_seq=0 out_seq=0 2024-04-16T17:08:36.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.420+0000 7fead8c00640 10 client.16023 ms_handle_connect on v2:172.21.15.104:6838/3660070592 2024-04-16T17:08:36.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.420+0000 7fead8c00640 1 -- 172.21.15.45:0/813472161 <== mds.0 v2:172.21.15.104:6838/3660070592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557538cb48b0 con 0x557538dc80e0 2024-04-16T17:08:36.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.420+0000 7fead8c00640 10 client.16023 handle_command_reply: tid=0 2024-04-16T17:08:36.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.420+0000 7feaf8b90740 1 client.16023 shutdown 2024-04-16T17:08:36.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.420+0000 7feaf8b90740 1 -- 172.21.15.45:0/813472161 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557538d48830 msgr2=0x557538d59b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:36.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.420+0000 7feaf8b90740 1 --2- 172.21.15.45:0/813472161 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557538d48830 0x557538d59b10 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7feadc000c00 tx=0x7feadc02bcc0 comp rx=0 tx=0).stop 2024-04-16T17:08:36.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.420+0000 7feaf8b90740 1 -- 172.21.15.45:0/813472161 shutdown_connections 2024-04-16T17:08:36.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.420+0000 7feaf8b90740 1 --2- 172.21.15.45:0/813472161 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x557538dc80e0 0x557538dca500 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:36.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.420+0000 7feaf8b90740 1 --2- 172.21.15.45:0/813472161 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557538d51410 0x557538d4acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:36.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.420+0000 7feaf8b90740 1 --2- 172.21.15.45:0/813472161 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557538d48830 0x557538d59b10 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:36.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.420+0000 7feaf8b90740 1 --2- 172.21.15.45:0/813472161 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557538d47f10 0x557538d595d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:36.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.420+0000 7feaf8b90740 1 -- 172.21.15.45:0/813472161 >> 172.21.15.45:0/813472161 conn(0x557538d42640 msgr2=0x557538d44300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:08:36.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.420+0000 7fead8c00640 0 client.16023 ms_handle_reset on v2:172.21.15.104:6838/3660070592 2024-04-16T17:08:36.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.420+0000 7feaf8b90740 1 -- 172.21.15.45:0/813472161 shutdown_connections 2024-04-16T17:08:36.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.421+0000 7feaf8b90740 1 -- 172.21.15.45:0/813472161 wait complete. 2024-04-16T17:08:36.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.421+0000 7feaf8b90740 20 client.16023 trim_cache size 0 max 16384 2024-04-16T17:08:36.426 INFO:teuthology.orchestra.run.smithi045.stdout:[] 2024-04-16T17:08:36.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.421+0000 7feaefe00640 1 -- 172.21.15.45:0/638106821 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fead40a3da0 msgr2=0x7fead40a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:36.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.421+0000 7feaefe00640 1 --2- 172.21.15.45:0/638106821 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fead40a3da0 0x7fead40a6260 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7feadc005c40 tx=0x7feadc005b50 comp rx=0 tx=0).stop 2024-04-16T17:08:36.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.421+0000 7feaefe00640 1 -- 172.21.15.45:0/638106821 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7feaf010e6e0 msgr2=0x7feaf019a810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:36.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.421+0000 7feaefe00640 1 --2- 172.21.15.45:0/638106821 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7feaf010e6e0 0x7feaf019a810 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7feae8009fd0 tx=0x7feae800ef10 comp rx=0 tx=0).stop 2024-04-16T17:08:36.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.421+0000 7feaefe00640 1 -- 172.21.15.45:0/638106821 shutdown_connections 2024-04-16T17:08:36.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.421+0000 7feaefe00640 1 --2- 172.21.15.45:0/638106821 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fead40a3da0 0x7fead40a6260 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:36.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.421+0000 7feaefe00640 1 --2- 172.21.15.45:0/638106821 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feaf0074d50 0x7feaf01a0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:36.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.421+0000 7feaefe00640 1 --2- 172.21.15.45:0/638106821 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7feaf010e6e0 0x7feaf019a810 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:36.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.421+0000 7feaefe00640 1 --2- 172.21.15.45:0/638106821 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feaf0074430 0x7feaf01a06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:36.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.421+0000 7feaefe00640 1 -- 172.21.15.45:0/638106821 >> 172.21.15.45:0/638106821 conn(0x7feaf00fd9d0 msgr2=0x7feaf00fe7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:08:36.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.422+0000 7feaefe00640 1 -- 172.21.15.45:0/638106821 shutdown_connections 2024-04-16T17:08:36.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:36.422+0000 7feaefe00640 1 -- 172.21.15.45:0/638106821 wait complete. 2024-04-16T17:08:36.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:08:36 smithi045 ceph-mon[31676]: pgmap v58: 129 pgs: 129 active+clean; 26 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 464 KiB/s rd, 418 KiB/s wr, 93 op/s 2024-04-16T17:08:36.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:08:36 smithi045 ceph-mon[31676]: from='client.? 172.21.15.45:0/247275965' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:08:36.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:08:36 smithi117 ceph-mon[36730]: pgmap v58: 129 pgs: 129 active+clean; 26 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 464 KiB/s rd, 418 KiB/s wr, 93 op/s 2024-04-16T17:08:36.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:08:36 smithi117 ceph-mon[36730]: from='client.? 172.21.15.45:0/247275965' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:08:36.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:08:36 smithi104 ceph-mon[36457]: pgmap v58: 129 pgs: 129 active+clean; 26 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 464 KiB/s rd, 418 KiB/s wr, 93 op/s 2024-04-16T17:08:36.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:08:36 smithi104 ceph-mon[36457]: from='client.? 172.21.15.45:0/247275965' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:08:37.218 INFO:tasks.workunit.client.0.smithi045.stdout: 1 25199 3.96 MB/sec warmup 39 sec latency 4764.308 ms 2024-04-16T17:08:37.440 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-16T17:08:37.440 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T17:08:37.440 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-16T17:08:37.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.640+0000 7fed84c00640 1 -- 172.21.15.45:0/4240744533 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fed80109580 msgr2=0x7fed80111150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:37.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.640+0000 7fed84c00640 1 --2- 172.21.15.45:0/4240744533 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fed80109580 0x7fed80111150 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fed78009ca0 tx=0x7fed7802c100 comp rx=0 tx=0).stop 2024-04-16T17:08:37.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.641+0000 7fed84c00640 1 -- 172.21.15.45:0/4240744533 shutdown_connections 2024-04-16T17:08:37.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.641+0000 7fed84c00640 1 --2- 172.21.15.45:0/4240744533 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fed80109580 0x7fed80111150 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:37.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.641+0000 7fed84c00640 1 --2- 172.21.15.45:0/4240744533 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fed80104e60 0x7fed80108ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:37.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.641+0000 7fed84c00640 1 --2- 172.21.15.45:0/4240744533 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fed801044b0 0x7fed80104890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:37.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.641+0000 7fed84c00640 1 -- 172.21.15.45:0/4240744533 >> 172.21.15.45:0/4240744533 conn(0x7fed800ffd90 msgr2=0x7fed801021b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:08:37.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.641+0000 7fed84c00640 1 -- 172.21.15.45:0/4240744533 shutdown_connections 2024-04-16T17:08:37.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.641+0000 7fed84c00640 1 -- 172.21.15.45:0/4240744533 wait complete. 2024-04-16T17:08:37.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.642+0000 7fed84c00640 1 Processor -- start 2024-04-16T17:08:37.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.642+0000 7fed84c00640 1 -- start start 2024-04-16T17:08:37.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.642+0000 7fed84c00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fed801044b0 0x7fed801a07b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:37.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.642+0000 7fed84c00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fed80109580 0x7fed801a0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:37.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.642+0000 7fed84c00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fed8019a960 0x7fed8019ae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:37.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.642+0000 7fed84c00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fed80114640 con 0x7fed80109580 2024-04-16T17:08:37.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.642+0000 7fed84c00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fed801144c0 con 0x7fed801044b0 2024-04-16T17:08:37.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.642+0000 7fed84c00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fed801147c0 con 0x7fed8019a960 2024-04-16T17:08:37.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.643+0000 7fed7d400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fed80109580 0x7fed801a0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:37.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.643+0000 7fed7d400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fed80109580 0x7fed801a0cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:40834/0 (socket says 172.21.15.45:40834) 2024-04-16T17:08:37.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.643+0000 7fed7d400640 1 -- 172.21.15.45:0/4160695357 learned_addr learned my addr 172.21.15.45:0/4160695357 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:08:37.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.643+0000 7fed7d400640 1 -- 172.21.15.45:0/4160695357 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fed8019a960 msgr2=0x7fed8019ae10 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-16T17:08:37.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.643+0000 7fed7d400640 1 --2- 172.21.15.45:0/4160695357 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fed8019a960 0x7fed8019ae10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:37.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.643+0000 7fed7d400640 1 -- 172.21.15.45:0/4160695357 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fed801044b0 msgr2=0x7fed801a07b0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-16T17:08:37.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.643+0000 7fed7d400640 1 --2- 172.21.15.45:0/4160695357 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fed801044b0 0x7fed801a07b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:37.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.643+0000 7fed7d400640 1 -- 172.21.15.45:0/4160695357 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed8019b480 con 0x7fed80109580 2024-04-16T17:08:37.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.643+0000 7fed7d400640 1 --2- 172.21.15.45:0/4160695357 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fed80109580 0x7fed801a0cf0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fed6c00e9a0 tx=0x7fed6c00ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:08:37.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.644+0000 7fed76a00640 1 -- 172.21.15.45:0/4160695357 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed6c00cd80 con 0x7fed80109580 2024-04-16T17:08:37.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.644+0000 7fed76a00640 1 -- 172.21.15.45:0/4160695357 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed6c0035f0 con 0x7fed80109580 2024-04-16T17:08:37.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.644+0000 7fed76a00640 1 -- 172.21.15.45:0/4160695357 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed6c010670 con 0x7fed80109580 2024-04-16T17:08:37.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.644+0000 7fed84c00640 1 -- 172.21.15.45:0/4160695357 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed8019b770 con 0x7fed80109580 2024-04-16T17:08:37.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.644+0000 7fed84c00640 1 -- 172.21.15.45:0/4160695357 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed801a7970 con 0x7fed80109580 2024-04-16T17:08:37.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.648+0000 7fed76a00640 1 -- 172.21.15.45:0/4160695357 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fed6c0037a0 con 0x7fed80109580 2024-04-16T17:08:37.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.649+0000 7fed879cd740 1 Processor -- start 2024-04-16T17:08:37.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.649+0000 7fed879cd740 1 -- start start 2024-04-16T17:08:37.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.649+0000 7fed879cd740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558000fa4ca0 0x558000fa5080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:37.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.649+0000 7fed879cd740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558000fa55c0 0x558000fadc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:37.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.649+0000 7fed879cd740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558000fae1a0 0x558000fb0590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:37.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.649+0000 7fed879cd740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x558000f09cb0 con 0x558000fae1a0 2024-04-16T17:08:37.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.649+0000 7fed879cd740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x558000eb0370 con 0x558000fa55c0 2024-04-16T17:08:37.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.649+0000 7fed879cd740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x558000f05c70 con 0x558000fa4ca0 2024-04-16T17:08:37.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.650+0000 7fed7e800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558000fa4ca0 0x558000fa5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:37.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.650+0000 7fed7e800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558000fa4ca0 0x558000fa5080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:54374/0 (socket says 172.21.15.45:54374) 2024-04-16T17:08:37.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.650+0000 7fed7e800640 1 -- 172.21.15.45:0/1098984817 learned_addr learned my addr 172.21.15.45:0/1098984817 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:08:37.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.650+0000 7fed7d400640 1 --2- 172.21.15.45:0/1098984817 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558000fae1a0 0x558000fb0590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:37.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.650+0000 7fed7de00640 1 --2- 172.21.15.45:0/1098984817 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558000fa55c0 0x558000fadc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:37.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.650+0000 7fed7e800640 1 -- 172.21.15.45:0/1098984817 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558000fa55c0 msgr2=0x558000fadc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:08:37.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.650+0000 7fed7e800640 1 --2- 172.21.15.45:0/1098984817 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558000fa55c0 0x558000fadc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:37.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.650+0000 7fed7e800640 1 -- 172.21.15.45:0/1098984817 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558000fae1a0 msgr2=0x558000fb0590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:08:37.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.650+0000 7fed7e800640 1 --2- 172.21.15.45:0/1098984817 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558000fae1a0 0x558000fb0590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:37.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.650+0000 7fed7e800640 1 -- 172.21.15.45:0/1098984817 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558000ecf910 con 0x558000fa4ca0 2024-04-16T17:08:37.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.650+0000 7fed76a00640 1 --2- 172.21.15.45:0/4160695357 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fed640a3da0 0x7fed640a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:37.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.650+0000 7fed7e800640 1 --2- 172.21.15.45:0/1098984817 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558000fa4ca0 0x558000fa5080 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fed78005e30 tx=0x7fed78005fc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d2bc0c0da489cd0d server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:08:37.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.651+0000 7fed76a00640 1 -- 172.21.15.45:0/4160695357 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fed6c0d0e40 con 0x7fed80109580 2024-04-16T17:08:37.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.651+0000 7fed6b000640 1 -- 172.21.15.45:0/1098984817 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed78041020 con 0x558000fa4ca0 2024-04-16T17:08:37.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.651+0000 7fed6b000640 1 -- 172.21.15.45:0/1098984817 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed78032c90 con 0x558000fa4ca0 2024-04-16T17:08:37.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.651+0000 7fed6b000640 1 -- 172.21.15.45:0/1098984817 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed78039040 con 0x558000fa4ca0 2024-04-16T17:08:37.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.652+0000 7fed7de00640 1 --2- 172.21.15.45:0/4160695357 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fed640a3da0 0x7fed640a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:37.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.652+0000 7fed879cd740 1 -- 172.21.15.45:0/1098984817 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558000fa4ca0 msgr2=0x558000fa5080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:37.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.652+0000 7fed879cd740 1 --2- 172.21.15.45:0/1098984817 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558000fa4ca0 0x558000fa5080 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fed78005e30 tx=0x7fed78005fc0 comp rx=0 tx=0).stop 2024-04-16T17:08:37.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.652+0000 7fed879cd740 1 -- 172.21.15.45:0/1098984817 shutdown_connections 2024-04-16T17:08:37.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.652+0000 7fed879cd740 1 --2- 172.21.15.45:0/1098984817 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558000fae1a0 0x558000fb0590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:37.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.652+0000 7fed879cd740 1 --2- 172.21.15.45:0/1098984817 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558000fa55c0 0x558000fadc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:37.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.652+0000 7fed879cd740 1 --2- 172.21.15.45:0/1098984817 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558000fa4ca0 0x558000fa5080 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:37.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.652+0000 7fed879cd740 1 -- 172.21.15.45:0/1098984817 >> 172.21.15.45:0/1098984817 conn(0x558000f9f3d0 msgr2=0x558000f9f7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:08:37.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.652+0000 7fed879cd740 1 -- 172.21.15.45:0/1098984817 shutdown_connections 2024-04-16T17:08:37.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.652+0000 7fed879cd740 1 -- 172.21.15.45:0/1098984817 wait complete. 2024-04-16T17:08:37.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.653+0000 7fed7de00640 1 --2- 172.21.15.45:0/4160695357 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fed640a3da0 0x7fed640a6260 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fed70005fd0 tx=0x7fed70005850 comp rx=0 tx=0).ready entity=mgr.25411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:08:37.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.653+0000 7fed879cd740 1 -- start start 2024-04-16T17:08:37.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.653+0000 7fed879cd740 1 Processor -- start 2024-04-16T17:08:37.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.654+0000 7fed879cd740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558000fa4ca0 0x558000fab260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:37.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.654+0000 7fed879cd740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558000fa55c0 0x558000fab7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:37.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.654+0000 7fed879cd740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558000fae1a0 0x558000fabce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:37.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.654+0000 7fed879cd740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x558000fb0f30 con 0x558000fa55c0 2024-04-16T17:08:37.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.654+0000 7fed879cd740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x558000ec3730 con 0x558000fae1a0 2024-04-16T17:08:37.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.654+0000 7fed879cd740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x558000fb10b0 con 0x558000fa4ca0 2024-04-16T17:08:37.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.654+0000 7fed879cd740 10 client.0 _ll_register_callbacks cb 0x558000e921e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:08:37.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.654+0000 7fed7d400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558000fa55c0 0x558000fab7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:37.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.654+0000 7fed7d400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558000fa55c0 0x558000fab7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:40844/0 (socket says 172.21.15.45:40844) 2024-04-16T17:08:37.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.654+0000 7fed7d400640 1 -- 172.21.15.45:0/2106959145 learned_addr learned my addr 172.21.15.45:0/2106959145 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:08:37.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.654+0000 7fed7e800640 1 --2- 172.21.15.45:0/2106959145 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558000fae1a0 0x558000fabce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:37.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.655+0000 7fed7d400640 1 -- 172.21.15.45:0/2106959145 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558000fa4ca0 msgr2=0x558000fab260 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-16T17:08:37.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.655+0000 7fed7d400640 1 --2- 172.21.15.45:0/2106959145 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558000fa4ca0 0x558000fab260 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:37.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.655+0000 7fed7d400640 1 -- 172.21.15.45:0/2106959145 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558000fae1a0 msgr2=0x558000fabce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:08:37.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.655+0000 7fed7d400640 1 --2- 172.21.15.45:0/2106959145 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558000fae1a0 0x558000fabce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:37.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.655+0000 7fed7d400640 1 -- 172.21.15.45:0/2106959145 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558000ce47e0 con 0x558000fa55c0 2024-04-16T17:08:37.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.655+0000 7fed7d400640 1 --2- 172.21.15.45:0/2106959145 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558000fa55c0 0x558000fab7a0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fed6c00e970 tx=0x7fed6c098dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f430515e4f9ead6e server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:08:37.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.655+0000 7fed5f400640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-16T17:08:37.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.655+0000 7fed5f400640 1 -- 172.21.15.45:0/2106959145 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed6c09d130 con 0x558000fa55c0 2024-04-16T17:08:37.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.655+0000 7fed5f400640 1 -- 172.21.15.45:0/2106959145 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fed6c0ce460 con 0x558000fa55c0 2024-04-16T17:08:37.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.655+0000 7fed879cd740 1 -- 172.21.15.45:0/2106959145 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fed78032c90 con 0x558000fa55c0 2024-04-16T17:08:37.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.656+0000 7fed5f400640 1 -- 172.21.15.45:0/2106959145 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fed6c0ce660 con 0x558000fa55c0 2024-04-16T17:08:37.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.656+0000 7fed879cd740 10 client.16035 fetch_fsmap learned FSMap version 26 2024-04-16T17:08:37.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.656+0000 7fed879cd740 1 -- 172.21.15.45:0/2106959145 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x558000d44f70 con 0x558000fa55c0 2024-04-16T17:08:37.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.656+0000 7fed5f400640 1 -- 172.21.15.45:0/2106959145 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed78032c90 con 0x558000fa55c0 2024-04-16T17:08:37.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.656+0000 7fed5f400640 1 -- 172.21.15.45:0/2106959145 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fed6c09be80 con 0x558000fa55c0 2024-04-16T17:08:37.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.656+0000 7fed879cd740 10 client.16035 fetch_fsmap finished waiting for FSMap version 26 2024-04-16T17:08:37.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.656+0000 7fed879cd740 10 client.16035 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:08:37.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.656+0000 7fed879cd740 20 client.16035 populate_metadata read hostname 'smithi045' 2024-04-16T17:08:37.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.656+0000 7fed879cd740 1 --2- 172.21.15.45:0/2106959145 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x558000fbe690 0x558000fc0b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:37.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.656+0000 7fed879cd740 4 client.16035 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:08:37.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.656+0000 7fed879cd740 1 -- 172.21.15.45:0/2106959145 --> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558000f176d0 con 0x558000fbe690 2024-04-16T17:08:37.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.657+0000 7fed7de00640 1 --2- 172.21.15.45:0/2106959145 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x558000fbe690 0x558000fc0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:37.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.657+0000 7fed7de00640 1 --2- 172.21.15.45:0/2106959145 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x558000fbe690 0x558000fc0b10 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=80fe02294489eab5 server_cookie=fa4febfb40dfbc18 in_seq=0 out_seq=0 2024-04-16T17:08:37.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.657+0000 7fed5f400640 10 client.16035 ms_handle_connect on v2:172.21.15.104:6838/3660070592 2024-04-16T17:08:37.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.658+0000 7fed5f400640 1 -- 172.21.15.45:0/2106959145 <== mds.0 v2:172.21.15.104:6838/3660070592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558000f176d0 con 0x558000fbe690 2024-04-16T17:08:37.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.658+0000 7fed5f400640 10 client.16035 handle_command_reply: tid=0 2024-04-16T17:08:37.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.658+0000 7fed879cd740 1 client.16035 shutdown 2024-04-16T17:08:37.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.659+0000 7fed879cd740 1 -- 172.21.15.45:0/2106959145 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558000fa55c0 msgr2=0x558000fab7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:37.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.659+0000 7fed879cd740 1 --2- 172.21.15.45:0/2106959145 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558000fa55c0 0x558000fab7a0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fed6c00e970 tx=0x7fed6c098dc0 comp rx=0 tx=0).stop 2024-04-16T17:08:37.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.659+0000 7fed879cd740 1 -- 172.21.15.45:0/2106959145 shutdown_connections 2024-04-16T17:08:37.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.659+0000 7fed879cd740 1 --2- 172.21.15.45:0/2106959145 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558000fae1a0 0x558000fabce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:37.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.659+0000 7fed879cd740 1 --2- 172.21.15.45:0/2106959145 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558000fa55c0 0x558000fab7a0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:37.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.659+0000 7fed879cd740 1 --2- 172.21.15.45:0/2106959145 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x558000fbe690 0x558000fc0b10 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:37.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.659+0000 7fed879cd740 1 --2- 172.21.15.45:0/2106959145 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558000fa4ca0 0x558000fab260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:37.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.659+0000 7fed879cd740 1 -- 172.21.15.45:0/2106959145 >> 172.21.15.45:0/2106959145 conn(0x558000f9f3d0 msgr2=0x558000fa1120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:08:37.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.659+0000 7fed5f400640 0 client.16035 ms_handle_reset on v2:172.21.15.104:6838/3660070592 2024-04-16T17:08:37.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.659+0000 7fed879cd740 1 -- 172.21.15.45:0/2106959145 shutdown_connections 2024-04-16T17:08:37.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.659+0000 7fed879cd740 1 -- 172.21.15.45:0/2106959145 wait complete. 2024-04-16T17:08:37.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.659+0000 7fed879cd740 20 client.16035 trim_cache size 0 max 16384 2024-04-16T17:08:37.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.692+0000 7fed879cd740 1 Processor -- start 2024-04-16T17:08:37.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.692+0000 7fed879cd740 1 -- start start 2024-04-16T17:08:37.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.692+0000 7fed879cd740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558000fa4ca0 0x558000fb9df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:37.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.692+0000 7fed879cd740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558000fa55c0 0x558000f9e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:37.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.692+0000 7fed879cd740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558000fae1a0 0x558000fc63e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:37.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.692+0000 7fed879cd740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fed801b3f90 con 0x558000fa55c0 2024-04-16T17:08:37.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.692+0000 7fed879cd740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fed801b1860 con 0x558000fae1a0 2024-04-16T17:08:37.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.692+0000 7fed879cd740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fed801af690 con 0x558000fa4ca0 2024-04-16T17:08:37.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.692+0000 7fed7de00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558000fa4ca0 0x558000fb9df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:37.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.693+0000 7fed7de00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558000fa4ca0 0x558000fb9df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:54392/0 (socket says 172.21.15.45:54392) 2024-04-16T17:08:37.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.693+0000 7fed7de00640 1 -- 172.21.15.45:0/2358551423 learned_addr learned my addr 172.21.15.45:0/2358551423 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:08:37.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.693+0000 7fed7d400640 1 --2- 172.21.15.45:0/2358551423 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558000fa55c0 0x558000f9e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:37.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.693+0000 7fed7de00640 1 -- 172.21.15.45:0/2358551423 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558000fae1a0 msgr2=0x558000fc63e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-16T17:08:37.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.693+0000 7fed7de00640 1 --2- 172.21.15.45:0/2358551423 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558000fae1a0 0x558000fc63e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:37.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.693+0000 7fed7de00640 1 -- 172.21.15.45:0/2358551423 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558000fa55c0 msgr2=0x558000f9e2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:08:37.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.693+0000 7fed7de00640 1 --2- 172.21.15.45:0/2358551423 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558000fa55c0 0x558000f9e2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:37.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.693+0000 7fed7de00640 1 -- 172.21.15.45:0/2358551423 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558000faa210 con 0x558000fa4ca0 2024-04-16T17:08:37.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.693+0000 7fed7de00640 1 --2- 172.21.15.45:0/2358551423 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558000fa4ca0 0x558000fb9df0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fed8019baa0 tx=0x7fed70015000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5b95109d4098ca91 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:08:37.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.694+0000 7fed6a600640 1 -- 172.21.15.45:0/2358551423 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed7002b070 con 0x558000fa4ca0 2024-04-16T17:08:37.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.694+0000 7fed6a600640 1 -- 172.21.15.45:0/2358551423 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed70011da0 con 0x558000fa4ca0 2024-04-16T17:08:37.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.694+0000 7fed6a600640 1 -- 172.21.15.45:0/2358551423 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed7001e420 con 0x558000fa4ca0 2024-04-16T17:08:37.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.695+0000 7fed879cd740 1 -- 172.21.15.45:0/2358551423 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558000fa4ca0 msgr2=0x558000fb9df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:37.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.695+0000 7fed879cd740 1 --2- 172.21.15.45:0/2358551423 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558000fa4ca0 0x558000fb9df0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fed8019baa0 tx=0x7fed70015000 comp rx=0 tx=0).stop 2024-04-16T17:08:37.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.695+0000 7fed879cd740 1 -- 172.21.15.45:0/2358551423 shutdown_connections 2024-04-16T17:08:37.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.695+0000 7fed879cd740 1 --2- 172.21.15.45:0/2358551423 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558000fae1a0 0x558000fc63e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:37.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.695+0000 7fed879cd740 1 --2- 172.21.15.45:0/2358551423 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558000fa55c0 0x558000f9e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:37.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.695+0000 7fed879cd740 1 --2- 172.21.15.45:0/2358551423 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558000fa4ca0 0x558000fb9df0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:37.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.695+0000 7fed879cd740 1 -- 172.21.15.45:0/2358551423 >> 172.21.15.45:0/2358551423 conn(0x558000f9f3d0 msgr2=0x558000f9dba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:08:37.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.695+0000 7fed879cd740 1 -- 172.21.15.45:0/2358551423 shutdown_connections 2024-04-16T17:08:37.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.695+0000 7fed879cd740 1 -- 172.21.15.45:0/2358551423 wait complete. 2024-04-16T17:08:37.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.695+0000 7fed879cd740 1 -- start start 2024-04-16T17:08:37.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.696+0000 7fed879cd740 1 Processor -- start 2024-04-16T17:08:37.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.696+0000 7fed879cd740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558000fa4ca0 0x558000fb1d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:37.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.696+0000 7fed879cd740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558000fa55c0 0x558000fb2250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:37.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.696+0000 7fed879cd740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558000fae1a0 0x558000fb2790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:37.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.696+0000 7fed879cd740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fed801afd20 con 0x558000fa4ca0 2024-04-16T17:08:37.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.696+0000 7fed879cd740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fed801b1190 con 0x558000fa55c0 2024-04-16T17:08:37.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.696+0000 7fed879cd740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fed801b03f0 con 0x558000fae1a0 2024-04-16T17:08:37.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.696+0000 7fed879cd740 10 client.0 _ll_register_callbacks cb 0x7fed6c00ef90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:08:37.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.697+0000 7fed7de00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558000fa4ca0 0x558000fb1d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:37.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.697+0000 7fed7e800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558000fae1a0 0x558000fb2790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:37.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.697+0000 7fed7de00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558000fa4ca0 0x558000fb1d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:40864/0 (socket says 172.21.15.45:40864) 2024-04-16T17:08:37.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.697+0000 7fed7de00640 1 -- 172.21.15.45:0/141327929 learned_addr learned my addr 172.21.15.45:0/141327929 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:08:37.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.697+0000 7fed7de00640 1 -- 172.21.15.45:0/141327929 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558000fae1a0 msgr2=0x558000fb2790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:08:37.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.697+0000 7fed7de00640 1 --2- 172.21.15.45:0/141327929 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558000fae1a0 0x558000fb2790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:37.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.697+0000 7fed7de00640 1 -- 172.21.15.45:0/141327929 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558000fa55c0 msgr2=0x558000fb2250 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-16T17:08:37.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.697+0000 7fed7de00640 1 --2- 172.21.15.45:0/141327929 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558000fa55c0 0x558000fb2250 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:37.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.697+0000 7fed7de00640 1 -- 172.21.15.45:0/141327929 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558000bdc170 con 0x558000fa4ca0 2024-04-16T17:08:37.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.697+0000 7fed7e800640 1 --2- 172.21.15.45:0/141327929 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558000fae1a0 0x558000fb2790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:08:37.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.697+0000 7fed7de00640 1 --2- 172.21.15.45:0/141327929 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558000fa4ca0 0x558000fb1d10 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fed70005d20 tx=0x7fed7000b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=666752c17938a1b8 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:08:37.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.697+0000 7fed5f400640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-16T17:08:37.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.698+0000 7fed5f400640 1 -- 172.21.15.45:0/141327929 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed70034060 con 0x558000fa4ca0 2024-04-16T17:08:37.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.698+0000 7fed5f400640 1 -- 172.21.15.45:0/141327929 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fed700118c0 con 0x558000fa4ca0 2024-04-16T17:08:37.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.698+0000 7fed879cd740 1 -- 172.21.15.45:0/141327929 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fed70011da0 con 0x558000fa4ca0 2024-04-16T17:08:37.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.698+0000 7fed5f400640 1 -- 172.21.15.45:0/141327929 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fed7003b050 con 0x558000fa4ca0 2024-04-16T17:08:37.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.698+0000 7fed879cd740 10 client.16041 fetch_fsmap learned FSMap version 26 2024-04-16T17:08:37.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.698+0000 7fed879cd740 1 -- 172.21.15.45:0/141327929 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x558001024d00 con 0x558000fa4ca0 2024-04-16T17:08:37.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.698+0000 7fed5f400640 1 -- 172.21.15.45:0/141327929 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed70011da0 con 0x558000fa4ca0 2024-04-16T17:08:37.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.699+0000 7fed5f400640 1 -- 172.21.15.45:0/141327929 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fed700046c0 con 0x558000fa4ca0 2024-04-16T17:08:37.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.699+0000 7fed879cd740 10 client.16041 fetch_fsmap finished waiting for FSMap version 26 2024-04-16T17:08:37.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.699+0000 7fed879cd740 10 client.16041 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:08:37.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.699+0000 7fed879cd740 20 client.16041 populate_metadata read hostname 'smithi045' 2024-04-16T17:08:37.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.699+0000 7fed879cd740 1 --2- 172.21.15.45:0/141327929 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x5580010253a0 0x5580010277c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:37.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.699+0000 7fed879cd740 4 client.16041 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-16T17:08:37.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.699+0000 7fed879cd740 1 -- 172.21.15.45:0/141327929 --> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x558000f11540 con 0x5580010253a0 2024-04-16T17:08:37.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.699+0000 7fed7d400640 1 --2- 172.21.15.45:0/141327929 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x5580010253a0 0x5580010277c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:37.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.700+0000 7fed7d400640 1 --2- 172.21.15.45:0/141327929 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x5580010253a0 0x5580010277c0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=da64620d7f34b0e8 server_cookie=b3d73e53211f42a7 in_seq=0 out_seq=0 2024-04-16T17:08:37.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.700+0000 7fed5f400640 10 client.16041 ms_handle_connect on v2:172.21.15.104:6838/3660070592 2024-04-16T17:08:37.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.700+0000 7fed5f400640 1 -- 172.21.15.45:0/141327929 <== mds.0 v2:172.21.15.104:6838/3660070592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x558000f11540 con 0x5580010253a0 2024-04-16T17:08:37.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.700+0000 7fed5f400640 10 client.16041 handle_command_reply: tid=0 2024-04-16T17:08:37.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.700+0000 7fed879cd740 1 client.16041 shutdown 2024-04-16T17:08:37.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.701+0000 7fed879cd740 1 -- 172.21.15.45:0/141327929 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558000fa4ca0 msgr2=0x558000fb1d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:37.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.701+0000 7fed879cd740 1 --2- 172.21.15.45:0/141327929 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558000fa4ca0 0x558000fb1d10 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fed70005d20 tx=0x7fed7000b910 comp rx=0 tx=0).stop 2024-04-16T17:08:37.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.701+0000 7fed879cd740 1 -- 172.21.15.45:0/141327929 shutdown_connections 2024-04-16T17:08:37.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.701+0000 7fed879cd740 1 --2- 172.21.15.45:0/141327929 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x5580010253a0 0x5580010277c0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:37.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.701+0000 7fed879cd740 1 --2- 172.21.15.45:0/141327929 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558000fae1a0 0x558000fb2790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:37.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.701+0000 7fed879cd740 1 --2- 172.21.15.45:0/141327929 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558000fa55c0 0x558000fb2250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:37.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.701+0000 7fed879cd740 1 --2- 172.21.15.45:0/141327929 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558000fa4ca0 0x558000fb1d10 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:37.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.701+0000 7fed879cd740 1 -- 172.21.15.45:0/141327929 >> 172.21.15.45:0/141327929 conn(0x558000f9f3d0 msgr2=0x558000fabab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:08:37.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.701+0000 7fed5f400640 0 client.16041 ms_handle_reset on v2:172.21.15.104:6838/3660070592 2024-04-16T17:08:37.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.701+0000 7fed879cd740 1 -- 172.21.15.45:0/141327929 shutdown_connections 2024-04-16T17:08:37.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.701+0000 7fed879cd740 1 -- 172.21.15.45:0/141327929 wait complete. 2024-04-16T17:08:37.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.701+0000 7fed879cd740 20 client.16041 trim_cache size 0 max 16384 2024-04-16T17:08:37.705 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-16T17:08:37.705 INFO:teuthology.orchestra.run.smithi045.stdout: "return_code": 0, 2024-04-16T17:08:37.705 INFO:teuthology.orchestra.run.smithi045.stdout: "scrub_tag": "b6b5e0c4-560c-4aa1-a1da-d1ede1fda81d", 2024-04-16T17:08:37.705 INFO:teuthology.orchestra.run.smithi045.stdout: "mode": "asynchronous" 2024-04-16T17:08:37.705 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-16T17:08:37.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.702+0000 7fed84c00640 1 -- 172.21.15.45:0/4160695357 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fed640a3da0 msgr2=0x7fed640a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:37.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.702+0000 7fed84c00640 1 --2- 172.21.15.45:0/4160695357 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fed640a3da0 0x7fed640a6260 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fed70005fd0 tx=0x7fed70005850 comp rx=0 tx=0).stop 2024-04-16T17:08:37.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.702+0000 7fed84c00640 1 -- 172.21.15.45:0/4160695357 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fed80109580 msgr2=0x7fed801a0cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:37.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.702+0000 7fed84c00640 1 --2- 172.21.15.45:0/4160695357 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fed80109580 0x7fed801a0cf0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fed6c00e9a0 tx=0x7fed6c00ee60 comp rx=0 tx=0).stop 2024-04-16T17:08:37.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.702+0000 7fed84c00640 1 -- 172.21.15.45:0/4160695357 shutdown_connections 2024-04-16T17:08:37.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.702+0000 7fed84c00640 1 --2- 172.21.15.45:0/4160695357 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fed640a3da0 0x7fed640a6260 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:37.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.702+0000 7fed84c00640 1 --2- 172.21.15.45:0/4160695357 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fed8019a960 0x7fed8019ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:37.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.702+0000 7fed84c00640 1 --2- 172.21.15.45:0/4160695357 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fed80109580 0x7fed801a0cf0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:37.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.702+0000 7fed84c00640 1 --2- 172.21.15.45:0/4160695357 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fed801044b0 0x7fed801a07b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:37.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.702+0000 7fed84c00640 1 -- 172.21.15.45:0/4160695357 >> 172.21.15.45:0/4160695357 conn(0x7fed800ffd90 msgr2=0x7fed801020a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:08:37.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.702+0000 7fed84c00640 1 -- 172.21.15.45:0/4160695357 shutdown_connections 2024-04-16T17:08:37.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.703+0000 7fed84c00640 1 -- 172.21.15.45:0/4160695357 wait complete. 2024-04-16T17:08:37.724 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-16T17:08:37.724 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-16T17:08:37.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.905+0000 7f2135800640 1 -- 172.21.15.45:0/2702869102 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f21300743d0 msgr2=0x7f21300747b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:37.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.905+0000 7f2135800640 1 --2- 172.21.15.45:0/2702869102 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f21300743d0 0x7f21300747b0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f2120009900 tx=0x7f212002bcd0 comp rx=0 tx=0).stop 2024-04-16T17:08:37.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.905+0000 7f2135800640 1 -- 172.21.15.45:0/2702869102 shutdown_connections 2024-04-16T17:08:37.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.905+0000 7f2135800640 1 --2- 172.21.15.45:0/2702869102 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f213010e740 0x7f2130110b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:37.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.905+0000 7f2135800640 1 --2- 172.21.15.45:0/2702869102 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2130074cf0 0x7f213010e200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:37.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.905+0000 7f2135800640 1 --2- 172.21.15.45:0/2702869102 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f21300743d0 0x7f21300747b0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:37.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.905+0000 7f2135800640 1 -- 172.21.15.45:0/2702869102 >> 172.21.15.45:0/2702869102 conn(0x7f21300fd9d0 msgr2=0x7f21300ffdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:08:37.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.906+0000 7f2135800640 1 -- 172.21.15.45:0/2702869102 shutdown_connections 2024-04-16T17:08:37.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.906+0000 7f2135800640 1 -- 172.21.15.45:0/2702869102 wait complete. 2024-04-16T17:08:37.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.907+0000 7f2135800640 1 Processor -- start 2024-04-16T17:08:37.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.907+0000 7f2135800640 1 -- start start 2024-04-16T17:08:37.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.907+0000 7f2135800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f21300743d0 0x7f21301a06a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:37.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.907+0000 7f2135800640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2130074cf0 0x7f21301a0be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:37.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.907+0000 7f2135800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f213010e740 0x7f213019a810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:37.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.907+0000 7f2135800640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f2130114510 con 0x7f213010e740 2024-04-16T17:08:37.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.907+0000 7f2135800640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f2130114390 con 0x7f2130074cf0 2024-04-16T17:08:37.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.907+0000 7f2135800640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f2130114690 con 0x7f21300743d0 2024-04-16T17:08:37.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.908+0000 7f212fe00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f21300743d0 0x7f21301a06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:37.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.908+0000 7f212f400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2130074cf0 0x7f21301a0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:37.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.908+0000 7f212fe00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f21300743d0 0x7f21301a06a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:54420/0 (socket says 172.21.15.45:54420) 2024-04-16T17:08:37.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.908+0000 7f212f400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2130074cf0 0x7f21301a0be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:44240/0 (socket says 172.21.15.45:44240) 2024-04-16T17:08:37.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.908+0000 7f212fe00640 1 -- 172.21.15.45:0/1289448530 learned_addr learned my addr 172.21.15.45:0/1289448530 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:08:37.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.908+0000 7f2134e00640 1 --2- 172.21.15.45:0/1289448530 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f213010e740 0x7f213019a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:37.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.908+0000 7f212fe00640 1 -- 172.21.15.45:0/1289448530 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2130074cf0 msgr2=0x7f21301a0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:37.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.908+0000 7f212fe00640 1 --2- 172.21.15.45:0/1289448530 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2130074cf0 0x7f21301a0be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:37.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.908+0000 7f212fe00640 1 -- 172.21.15.45:0/1289448530 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f213010e740 msgr2=0x7f213019a810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:37.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.908+0000 7f212fe00640 1 --2- 172.21.15.45:0/1289448530 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f213010e740 0x7f213019a810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:37.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.908+0000 7f212fe00640 1 -- 172.21.15.45:0/1289448530 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f213019afe0 con 0x7f21300743d0 2024-04-16T17:08:37.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.908+0000 7f2134e00640 1 --2- 172.21.15.45:0/1289448530 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f213010e740 0x7f213019a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:08:37.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.908+0000 7f212f400640 1 --2- 172.21.15.45:0/1289448530 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2130074cf0 0x7f21301a0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:08:37.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.909+0000 7f212fe00640 1 --2- 172.21.15.45:0/1289448530 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f21300743d0 0x7f21301a06a0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f2120000c00 tx=0x7f2120004360 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:08:37.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.909+0000 7f212cc00640 1 -- 172.21.15.45:0/1289448530 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2120042020 con 0x7f21300743d0 2024-04-16T17:08:37.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.909+0000 7f212cc00640 1 -- 172.21.15.45:0/1289448530 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f21200054e0 con 0x7f21300743d0 2024-04-16T17:08:37.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.909+0000 7f212cc00640 1 -- 172.21.15.45:0/1289448530 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2120003b50 con 0x7f21300743d0 2024-04-16T17:08:37.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.909+0000 7f2135800640 1 -- 172.21.15.45:0/1289448530 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f213019b270 con 0x7f21300743d0 2024-04-16T17:08:37.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.909+0000 7f2135800640 1 -- 172.21.15.45:0/1289448530 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21301a74f0 con 0x7f21300743d0 2024-04-16T17:08:37.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.912+0000 7f2139b6a740 1 Processor -- start 2024-04-16T17:08:37.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.912+0000 7f2139b6a740 1 -- start start 2024-04-16T17:08:37.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.912+0000 7f2139b6a740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e78845d390 0x55e78845d770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:37.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.912+0000 7f2139b6a740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e78845dcb0 0x55e788466350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:37.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.912+0000 7f2139b6a740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e788466890 0x55e788468c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:37.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.912+0000 7f2139b6a740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55e7883c3120 con 0x55e788466890 2024-04-16T17:08:37.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.912+0000 7f2139b6a740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55e788369340 con 0x55e78845d390 2024-04-16T17:08:37.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.912+0000 7f2139b6a740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55e7883bf0e0 con 0x55e78845dcb0 2024-04-16T17:08:37.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.913+0000 7f2134e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e78845d390 0x55e78845d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:37.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.913+0000 7f2134e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e78845d390 0x55e78845d770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:44244/0 (socket says 172.21.15.45:44244) 2024-04-16T17:08:37.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.913+0000 7f212f400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e788466890 0x55e788468c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:37.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.913+0000 7f2134e00640 1 -- 172.21.15.45:0/613291777 learned_addr learned my addr 172.21.15.45:0/613291777 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:08:37.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.913+0000 7f2134e00640 1 -- 172.21.15.45:0/613291777 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e78845dcb0 msgr2=0x55e788466350 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-16T17:08:37.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.913+0000 7f2134e00640 1 --2- 172.21.15.45:0/613291777 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e78845dcb0 0x55e788466350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:37.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.913+0000 7f2134e00640 1 -- 172.21.15.45:0/613291777 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e788466890 msgr2=0x55e788468c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:08:37.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.913+0000 7f2134e00640 1 --2- 172.21.15.45:0/613291777 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e788466890 0x55e788468c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:37.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.913+0000 7f2134e00640 1 -- 172.21.15.45:0/613291777 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e78819f7d0 con 0x55e78845d390 2024-04-16T17:08:37.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.913+0000 7f2134e00640 1 --2- 172.21.15.45:0/613291777 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e78845d390 0x55e78845d770 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f2128009f80 tx=0x7f2128010860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4dde223d2dbc731a server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:08:37.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.913+0000 7f212cc00640 1 -- 172.21.15.45:0/1289448530 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f2120045020 con 0x7f21300743d0 2024-04-16T17:08:37.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.914+0000 7f2125600640 1 -- 172.21.15.45:0/613291777 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2128011700 con 0x55e78845d390 2024-04-16T17:08:37.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.914+0000 7f2125600640 1 -- 172.21.15.45:0/613291777 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2128011d80 con 0x55e78845d390 2024-04-16T17:08:37.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.914+0000 7f2125600640 1 -- 172.21.15.45:0/613291777 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f212800d340 con 0x55e78845d390 2024-04-16T17:08:37.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.914+0000 7f2139b6a740 1 -- 172.21.15.45:0/613291777 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e78845d390 msgr2=0x55e78845d770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:37.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.914+0000 7f2139b6a740 1 --2- 172.21.15.45:0/613291777 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e78845d390 0x55e78845d770 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f2128009f80 tx=0x7f2128010860 comp rx=0 tx=0).stop 2024-04-16T17:08:37.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.915+0000 7f2139b6a740 1 -- 172.21.15.45:0/613291777 shutdown_connections 2024-04-16T17:08:37.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.915+0000 7f2139b6a740 1 --2- 172.21.15.45:0/613291777 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e78845dcb0 0x55e788466350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:37.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.915+0000 7f2139b6a740 1 --2- 172.21.15.45:0/613291777 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e788466890 0x55e788468c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:37.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.915+0000 7f2139b6a740 1 --2- 172.21.15.45:0/613291777 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e78845d390 0x55e78845d770 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:37.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.915+0000 7f2139b6a740 1 -- 172.21.15.45:0/613291777 >> 172.21.15.45:0/613291777 conn(0x55e788457ac0 msgr2=0x55e788457ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:08:37.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.915+0000 7f2139b6a740 1 -- 172.21.15.45:0/613291777 shutdown_connections 2024-04-16T17:08:37.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.915+0000 7f2139b6a740 1 -- 172.21.15.45:0/613291777 wait complete. 2024-04-16T17:08:37.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.915+0000 7f212cc00640 1 --2- 172.21.15.45:0/1289448530 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f21140a3cb0 0x7f21140a6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:37.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.915+0000 7f2139b6a740 1 -- start start 2024-04-16T17:08:37.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.915+0000 7f212cc00640 1 -- 172.21.15.45:0/1289448530 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f21200f2690 con 0x7f21300743d0 2024-04-16T17:08:37.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.915+0000 7f2134e00640 1 --2- 172.21.15.45:0/1289448530 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f21140a3cb0 0x7f21140a6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:37.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.915+0000 7f2139b6a740 1 Processor -- start 2024-04-16T17:08:37.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.916+0000 7f2139b6a740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e78845d390 0x55e788463960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:37.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.916+0000 7f2139b6a740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e78845dcb0 0x55e788463ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:37.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.916+0000 7f2139b6a740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e788466890 0x55e7884643e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:37.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.916+0000 7f2139b6a740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55e788469680 con 0x55e788466890 2024-04-16T17:08:37.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.916+0000 7f2139b6a740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55e78837c710 con 0x55e78845dcb0 2024-04-16T17:08:37.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.916+0000 7f2139b6a740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55e788469800 con 0x55e78845d390 2024-04-16T17:08:37.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.916+0000 7f2139b6a740 10 client.0 _ll_register_callbacks cb 0x55e788377af0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:08:37.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.916+0000 7f2134e00640 1 --2- 172.21.15.45:0/1289448530 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f21140a3cb0 0x7f21140a6170 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f2128010d60 tx=0x7f2128010fd0 comp rx=0 tx=0).ready entity=mgr.25411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:08:37.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.916+0000 7f2134e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e788466890 0x55e7884643e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:37.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.916+0000 7f2134e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e788466890 0x55e7884643e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:40874/0 (socket says 172.21.15.45:40874) 2024-04-16T17:08:37.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.916+0000 7f2134e00640 1 -- 172.21.15.45:0/2635226387 learned_addr learned my addr 172.21.15.45:0/2635226387 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:08:37.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.916+0000 7f212f400640 1 --2- 172.21.15.45:0/2635226387 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e78845dcb0 0x55e788463ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:37.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.917+0000 7f2134e00640 1 -- 172.21.15.45:0/2635226387 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e78845d390 msgr2=0x55e788463960 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-16T17:08:37.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.917+0000 7f2134e00640 1 --2- 172.21.15.45:0/2635226387 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e78845d390 0x55e788463960 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:37.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.917+0000 7f2134e00640 1 -- 172.21.15.45:0/2635226387 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e78845dcb0 msgr2=0x55e788463ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:08:37.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.917+0000 7f2134e00640 1 --2- 172.21.15.45:0/2635226387 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e78845dcb0 0x55e788463ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:37.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.917+0000 7f2134e00640 1 -- 172.21.15.45:0/2635226387 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e7880d53f0 con 0x55e788466890 2024-04-16T17:08:37.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.917+0000 7f212f400640 1 --2- 172.21.15.45:0/2635226387 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e78845dcb0 0x55e788463ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:08:37.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.918+0000 7f2134e00640 1 --2- 172.21.15.45:0/2635226387 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e788466890 0x55e7884643e0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f2128009f80 tx=0x7f212800eb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52d1c5ff48f655e8 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:08:37.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.918+0000 7f211a000640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-16T17:08:37.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.918+0000 7f211a000640 1 -- 172.21.15.45:0/2635226387 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2128035060 con 0x55e788466890 2024-04-16T17:08:37.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.918+0000 7f211a000640 1 -- 172.21.15.45:0/2635226387 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f2128005050 con 0x55e788466890 2024-04-16T17:08:37.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.918+0000 7f2139b6a740 1 -- 172.21.15.45:0/2635226387 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2128011d80 con 0x55e788466890 2024-04-16T17:08:37.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.918+0000 7f211a000640 1 -- 172.21.15.45:0/2635226387 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f212803c050 con 0x55e788466890 2024-04-16T17:08:37.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.918+0000 7f2139b6a740 10 client.16053 fetch_fsmap learned FSMap version 26 2024-04-16T17:08:37.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.918+0000 7f2139b6a740 1 -- 172.21.15.45:0/2635226387 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55e7880950f0 con 0x55e788466890 2024-04-16T17:08:37.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.919+0000 7f211a000640 1 -- 172.21.15.45:0/2635226387 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2128011d80 con 0x55e788466890 2024-04-16T17:08:37.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.919+0000 7f211a000640 1 -- 172.21.15.45:0/2635226387 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f212802f070 con 0x55e788466890 2024-04-16T17:08:37.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.919+0000 7f2139b6a740 10 client.16053 fetch_fsmap finished waiting for FSMap version 26 2024-04-16T17:08:37.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.919+0000 7f2139b6a740 10 client.16053 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:08:37.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.919+0000 7f2139b6a740 20 client.16053 populate_metadata read hostname 'smithi045' 2024-04-16T17:08:37.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.919+0000 7f2139b6a740 1 --2- 172.21.15.45:0/2635226387 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55e788476c20 0x55e788479010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:37.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.919+0000 7f2139b6a740 4 client.16053 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:08:37.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.919+0000 7f2139b6a740 1 -- 172.21.15.45:0/2635226387 --> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e7883d0b40 con 0x55e788476c20 2024-04-16T17:08:37.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.922+0000 7f212fe00640 1 --2- 172.21.15.45:0/2635226387 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55e788476c20 0x55e788479010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:37.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.923+0000 7f212fe00640 1 --2- 172.21.15.45:0/2635226387 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55e788476c20 0x55e788479010 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=61ce1b28bd7dd05 server_cookie=ec59c167bfeda2fd in_seq=0 out_seq=0 2024-04-16T17:08:37.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.923+0000 7f211a000640 10 client.16053 ms_handle_connect on v2:172.21.15.104:6838/3660070592 2024-04-16T17:08:37.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.923+0000 7f211a000640 1 -- 172.21.15.45:0/2635226387 <== mds.0 v2:172.21.15.104:6838/3660070592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e7883d0b40 con 0x55e788476c20 2024-04-16T17:08:37.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.923+0000 7f211a000640 10 client.16053 handle_command_reply: tid=0 2024-04-16T17:08:37.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.923+0000 7f2139b6a740 1 client.16053 shutdown 2024-04-16T17:08:37.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.924+0000 7f2139b6a740 1 -- 172.21.15.45:0/2635226387 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e788466890 msgr2=0x55e7884643e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:37.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.924+0000 7f2139b6a740 1 --2- 172.21.15.45:0/2635226387 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e788466890 0x55e7884643e0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f2128009f80 tx=0x7f212800eb00 comp rx=0 tx=0).stop 2024-04-16T17:08:37.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.924+0000 7f2139b6a740 1 -- 172.21.15.45:0/2635226387 shutdown_connections 2024-04-16T17:08:37.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.924+0000 7f2139b6a740 1 --2- 172.21.15.45:0/2635226387 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e788466890 0x55e7884643e0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:37.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.924+0000 7f2139b6a740 1 --2- 172.21.15.45:0/2635226387 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e78845dcb0 0x55e788463ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:37.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.924+0000 7f2139b6a740 1 --2- 172.21.15.45:0/2635226387 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55e788476c20 0x55e788479010 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:37.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.924+0000 7f2139b6a740 1 --2- 172.21.15.45:0/2635226387 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e78845d390 0x55e788463960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:37.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.924+0000 7f2139b6a740 1 -- 172.21.15.45:0/2635226387 >> 172.21.15.45:0/2635226387 conn(0x55e788457ac0 msgr2=0x55e788459810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:08:37.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.925+0000 7f211a000640 0 client.16053 ms_handle_reset on v2:172.21.15.104:6838/3660070592 2024-04-16T17:08:37.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.925+0000 7f2139b6a740 1 -- 172.21.15.45:0/2635226387 shutdown_connections 2024-04-16T17:08:37.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.925+0000 7f2139b6a740 1 -- 172.21.15.45:0/2635226387 wait complete. 2024-04-16T17:08:37.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.925+0000 7f2139b6a740 20 client.16053 trim_cache size 0 max 16384 2024-04-16T17:08:37.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.957+0000 7f2139b6a740 1 Processor -- start 2024-04-16T17:08:37.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.958+0000 7f2139b6a740 1 -- start start 2024-04-16T17:08:37.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.958+0000 7f2139b6a740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e78845d390 0x55e788466090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:37.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.958+0000 7f2139b6a740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e78845dcb0 0x55e78845ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:37.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.958+0000 7f2139b6a740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e788466890 0x55e788472540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:37.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.958+0000 7f2139b6a740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f21301b3d80 con 0x55e78845dcb0 2024-04-16T17:08:37.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.958+0000 7f2139b6a740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f21301b1650 con 0x55e788466890 2024-04-16T17:08:37.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.958+0000 7f2139b6a740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f21301af480 con 0x55e78845d390 2024-04-16T17:08:37.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.958+0000 7f212f400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e78845dcb0 0x55e78845ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:37.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.958+0000 7f212f400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e78845dcb0 0x55e78845ecb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:40888/0 (socket says 172.21.15.45:40888) 2024-04-16T17:08:37.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.958+0000 7f212f400640 1 -- 172.21.15.45:0/4019892874 learned_addr learned my addr 172.21.15.45:0/4019892874 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:08:37.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.958+0000 7f212fe00640 1 --2- 172.21.15.45:0/4019892874 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e78845d390 0x55e788466090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:37.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.959+0000 7f2134e00640 1 --2- 172.21.15.45:0/4019892874 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e788466890 0x55e788472540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:37.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.959+0000 7f212f400640 1 -- 172.21.15.45:0/4019892874 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e78845d390 msgr2=0x55e788466090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:08:37.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.959+0000 7f212f400640 1 --2- 172.21.15.45:0/4019892874 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e78845d390 0x55e788466090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:37.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.959+0000 7f212f400640 1 -- 172.21.15.45:0/4019892874 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e788466890 msgr2=0x55e788472540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:08:37.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.959+0000 7f212f400640 1 --2- 172.21.15.45:0/4019892874 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e788466890 0x55e788472540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:37.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.959+0000 7f212f400640 1 -- 172.21.15.45:0/4019892874 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e788462930 con 0x55e78845dcb0 2024-04-16T17:08:37.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.959+0000 7f212fe00640 1 --2- 172.21.15.45:0/4019892874 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e78845d390 0x55e788466090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:08:37.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.959+0000 7f2134e00640 1 --2- 172.21.15.45:0/4019892874 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e788466890 0x55e788472540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:08:37.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.959+0000 7f212f400640 1 --2- 172.21.15.45:0/4019892874 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e78845dcb0 0x55e78845ecb0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f211c012150 tx=0x7f211c012b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e695f4c8e39f1a1d server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:08:37.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.959+0000 7f2124c00640 1 -- 172.21.15.45:0/4019892874 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f211c021070 con 0x55e78845dcb0 2024-04-16T17:08:37.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.960+0000 7f2124c00640 1 -- 172.21.15.45:0/4019892874 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f211c010910 con 0x55e78845dcb0 2024-04-16T17:08:37.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.960+0000 7f2124c00640 1 -- 172.21.15.45:0/4019892874 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f211c0039d0 con 0x55e78845dcb0 2024-04-16T17:08:37.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.960+0000 7f2139b6a740 1 -- 172.21.15.45:0/4019892874 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e78845dcb0 msgr2=0x55e78845ecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:37.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.960+0000 7f2139b6a740 1 --2- 172.21.15.45:0/4019892874 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e78845dcb0 0x55e78845ecb0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f211c012150 tx=0x7f211c012b30 comp rx=0 tx=0).stop 2024-04-16T17:08:37.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.960+0000 7f2139b6a740 1 -- 172.21.15.45:0/4019892874 shutdown_connections 2024-04-16T17:08:37.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.960+0000 7f2139b6a740 1 --2- 172.21.15.45:0/4019892874 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e788466890 0x55e788472540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:37.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.960+0000 7f2139b6a740 1 --2- 172.21.15.45:0/4019892874 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e78845dcb0 0x55e78845ecb0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:37.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.960+0000 7f2139b6a740 1 --2- 172.21.15.45:0/4019892874 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e78845d390 0x55e788466090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:37.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.960+0000 7f2139b6a740 1 -- 172.21.15.45:0/4019892874 >> 172.21.15.45:0/4019892874 conn(0x55e788457ac0 msgr2=0x55e788456290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:08:37.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.960+0000 7f2139b6a740 1 -- 172.21.15.45:0/4019892874 shutdown_connections 2024-04-16T17:08:37.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.960+0000 7f2139b6a740 1 -- 172.21.15.45:0/4019892874 wait complete. 2024-04-16T17:08:37.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.961+0000 7f2139b6a740 1 -- start start 2024-04-16T17:08:37.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.961+0000 7f2139b6a740 1 Processor -- start 2024-04-16T17:08:37.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.961+0000 7f2139b6a740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e78845d390 0x55e78846ed40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:37.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.961+0000 7f2139b6a740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e78845dcb0 0x55e7884602d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:37.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.961+0000 7f2139b6a740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e788466890 0x55e788460810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:37.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.961+0000 7f2139b6a740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f21301afb10 con 0x55e78845d390 2024-04-16T17:08:37.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.961+0000 7f2139b6a740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f21301b0f80 con 0x55e788466890 2024-04-16T17:08:37.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.961+0000 7f2139b6a740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f21301b01e0 con 0x55e78845dcb0 2024-04-16T17:08:37.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.962+0000 7f2139b6a740 10 client.0 _ll_register_callbacks cb 0x7f211c009b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:08:37.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.962+0000 7f212fe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e78845d390 0x55e78846ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:37.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.962+0000 7f212fe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e78845d390 0x55e78846ed40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:40892/0 (socket says 172.21.15.45:40892) 2024-04-16T17:08:37.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.962+0000 7f212fe00640 1 -- 172.21.15.45:0/436895959 learned_addr learned my addr 172.21.15.45:0/436895959 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:08:37.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.962+0000 7f2134e00640 1 --2- 172.21.15.45:0/436895959 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e788466890 0x55e788460810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:37.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.962+0000 7f212fe00640 1 -- 172.21.15.45:0/436895959 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e78845dcb0 msgr2=0x55e7884602d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-16T17:08:37.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.962+0000 7f212fe00640 1 --2- 172.21.15.45:0/436895959 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e78845dcb0 0x55e7884602d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:37.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.962+0000 7f212fe00640 1 -- 172.21.15.45:0/436895959 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e788466890 msgr2=0x55e788460810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:08:37.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.962+0000 7f212fe00640 1 --2- 172.21.15.45:0/436895959 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e788466890 0x55e788460810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:37.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.962+0000 7f212fe00640 1 -- 172.21.15.45:0/436895959 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e78845dae0 con 0x55e78845d390 2024-04-16T17:08:37.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.962+0000 7f2134e00640 1 --2- 172.21.15.45:0/436895959 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e788466890 0x55e788460810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:08:37.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.963+0000 7f212fe00640 1 --2- 172.21.15.45:0/436895959 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e78845d390 0x55e78846ed40 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f21200ba7f0 tx=0x7f2120003680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a0ef38f0ef6ed7e server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:08:37.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.963+0000 7f211a000640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-16T17:08:37.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.963+0000 7f211a000640 1 -- 172.21.15.45:0/436895959 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21200c2740 con 0x55e78845d390 2024-04-16T17:08:37.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.963+0000 7f211a000640 1 -- 172.21.15.45:0/436895959 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f2120003cf0 con 0x55e78845d390 2024-04-16T17:08:37.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.963+0000 7f2139b6a740 1 -- 172.21.15.45:0/436895959 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f211c010910 con 0x55e78845d390 2024-04-16T17:08:37.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.963+0000 7f211a000640 1 -- 172.21.15.45:0/436895959 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f21200c7000 con 0x55e78845d390 2024-04-16T17:08:37.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.963+0000 7f2139b6a740 10 client.16059 fetch_fsmap learned FSMap version 26 2024-04-16T17:08:37.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.963+0000 7f2139b6a740 1 -- 172.21.15.45:0/436895959 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55e7884dcdb0 con 0x55e78845d390 2024-04-16T17:08:37.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.963+0000 7f211a000640 1 -- 172.21.15.45:0/436895959 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f211c010910 con 0x55e78845d390 2024-04-16T17:08:37.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.964+0000 7f211a000640 1 -- 172.21.15.45:0/436895959 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f21200be5f0 con 0x55e78845d390 2024-04-16T17:08:37.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.964+0000 7f2139b6a740 10 client.16059 fetch_fsmap finished waiting for FSMap version 26 2024-04-16T17:08:37.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.964+0000 7f2139b6a740 10 client.16059 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:08:37.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.964+0000 7f2139b6a740 20 client.16059 populate_metadata read hostname 'smithi045' 2024-04-16T17:08:37.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.964+0000 7f2139b6a740 1 --2- 172.21.15.45:0/436895959 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55e7884dd540 0x55e7884df9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:37.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.964+0000 7f2139b6a740 4 client.16059 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-16T17:08:37.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.964+0000 7f2139b6a740 1 -- 172.21.15.45:0/436895959 --> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55e7883ca9b0 con 0x55e7884dd540 2024-04-16T17:08:37.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.964+0000 7f212f400640 1 --2- 172.21.15.45:0/436895959 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55e7884dd540 0x55e7884df9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:37.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.965+0000 7f212f400640 1 --2- 172.21.15.45:0/436895959 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55e7884dd540 0x55e7884df9a0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e31443f702a6c576 server_cookie=a6a1a04c5fc1f214 in_seq=0 out_seq=0 2024-04-16T17:08:37.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.965+0000 7f211a000640 10 client.16059 ms_handle_connect on v2:172.21.15.104:6838/3660070592 2024-04-16T17:08:37.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.965+0000 7f211a000640 1 -- 172.21.15.45:0/436895959 <== mds.0 v2:172.21.15.104:6838/3660070592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55e7883ca9b0 con 0x55e7884dd540 2024-04-16T17:08:37.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.965+0000 7f211a000640 10 client.16059 handle_command_reply: tid=0 2024-04-16T17:08:37.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.965+0000 7f2139b6a740 1 client.16059 shutdown 2024-04-16T17:08:37.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.965+0000 7f2139b6a740 1 -- 172.21.15.45:0/436895959 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e78845d390 msgr2=0x55e78846ed40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:37.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.965+0000 7f2139b6a740 1 --2- 172.21.15.45:0/436895959 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e78845d390 0x55e78846ed40 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f21200ba7f0 tx=0x7f2120003680 comp rx=0 tx=0).stop 2024-04-16T17:08:37.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.965+0000 7f2139b6a740 1 -- 172.21.15.45:0/436895959 shutdown_connections 2024-04-16T17:08:37.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.965+0000 7f2139b6a740 1 --2- 172.21.15.45:0/436895959 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55e7884dd540 0x55e7884df9a0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:37.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.965+0000 7f2139b6a740 1 --2- 172.21.15.45:0/436895959 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e78845dcb0 0x55e7884602d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:37.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.965+0000 7f2139b6a740 1 --2- 172.21.15.45:0/436895959 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e788466890 0x55e788460810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:37.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.965+0000 7f2139b6a740 1 --2- 172.21.15.45:0/436895959 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e78845d390 0x55e78846ed40 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:37.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.965+0000 7f2139b6a740 1 -- 172.21.15.45:0/436895959 >> 172.21.15.45:0/436895959 conn(0x55e788457ac0 msgr2=0x55e788456b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:08:37.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.965+0000 7f211a000640 0 client.16059 ms_handle_reset on v2:172.21.15.104:6838/3660070592 2024-04-16T17:08:37.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.965+0000 7f2139b6a740 1 -- 172.21.15.45:0/436895959 shutdown_connections 2024-04-16T17:08:37.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.965+0000 7f2139b6a740 1 -- 172.21.15.45:0/436895959 wait complete. 2024-04-16T17:08:37.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.966+0000 7f2139b6a740 20 client.16059 trim_cache size 0 max 16384 2024-04-16T17:08:37.971 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-16T17:08:37.971 INFO:teuthology.orchestra.run.smithi045.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-16T17:08:37.971 INFO:teuthology.orchestra.run.smithi045.stdout: "scrubs": { 2024-04-16T17:08:37.971 INFO:teuthology.orchestra.run.smithi045.stdout: "b6b5e0c4-560c-4aa1-a1da-d1ede1fda81d": { 2024-04-16T17:08:37.971 INFO:teuthology.orchestra.run.smithi045.stdout: "path": "/", 2024-04-16T17:08:37.971 INFO:teuthology.orchestra.run.smithi045.stdout: "tag": "b6b5e0c4-560c-4aa1-a1da-d1ede1fda81d", 2024-04-16T17:08:37.971 INFO:teuthology.orchestra.run.smithi045.stdout: "options": "recursive,force" 2024-04-16T17:08:37.971 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:08:37.972 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:08:37.972 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-16T17:08:37.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.966+0000 7f2135800640 1 -- 172.21.15.45:0/1289448530 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f21140a3cb0 msgr2=0x7f21140a6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:37.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.966+0000 7f2135800640 1 --2- 172.21.15.45:0/1289448530 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f21140a3cb0 0x7f21140a6170 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f2128010d60 tx=0x7f2128010fd0 comp rx=0 tx=0).stop 2024-04-16T17:08:37.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.966+0000 7f2135800640 1 -- 172.21.15.45:0/1289448530 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f21300743d0 msgr2=0x7f21301a06a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:37.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.966+0000 7f2135800640 1 --2- 172.21.15.45:0/1289448530 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f21300743d0 0x7f21301a06a0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f2120000c00 tx=0x7f2120004360 comp rx=0 tx=0).stop 2024-04-16T17:08:37.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.966+0000 7f2135800640 1 -- 172.21.15.45:0/1289448530 shutdown_connections 2024-04-16T17:08:37.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.966+0000 7f2135800640 1 --2- 172.21.15.45:0/1289448530 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f21140a3cb0 0x7f21140a6170 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:37.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.966+0000 7f2135800640 1 --2- 172.21.15.45:0/1289448530 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f213010e740 0x7f213019a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:37.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.966+0000 7f2135800640 1 --2- 172.21.15.45:0/1289448530 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2130074cf0 0x7f21301a0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:37.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.966+0000 7f2135800640 1 --2- 172.21.15.45:0/1289448530 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f21300743d0 0x7f21301a06a0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:37.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.966+0000 7f2135800640 1 -- 172.21.15.45:0/1289448530 >> 172.21.15.45:0/1289448530 conn(0x7f21300fd9d0 msgr2=0x7f21300fe700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:08:37.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.966+0000 7f2135800640 1 -- 172.21.15.45:0/1289448530 shutdown_connections 2024-04-16T17:08:37.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:08:37.966+0000 7f2135800640 1 -- 172.21.15.45:0/1289448530 wait complete. 2024-04-16T17:08:37.987 INFO:tasks.cephfs.filesystem:scrub status for tag:b6b5e0c4-560c-4aa1-a1da-d1ede1fda81d - {'path': '/', 'tag': 'b6b5e0c4-560c-4aa1-a1da-d1ede1fda81d', 'options': 'recursive,force'} 2024-04-16T17:08:38.219 INFO:tasks.workunit.client.0.smithi045.stdout: 1 25391 3.87 MB/sec warmup 40 sec latency 4813.312 ms 2024-04-16T17:08:38.693 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:08:38 smithi045 ceph-mon[31676]: pgmap v59: 129 pgs: 129 active+clean; 26 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 514 KiB/s rd, 468 KiB/s wr, 108 op/s 2024-04-16T17:08:38.693 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:08:38 smithi045 ceph-mon[31676]: scrub summary: active paths [/] 2024-04-16T17:08:38.694 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:08:38 smithi045 ceph-mon[31676]: scrub summary: active paths [/] 2024-04-16T17:08:38.694 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:08:38 smithi045 ceph-mon[31676]: scrub summary: idle+waiting paths [/] 2024-04-16T17:08:38.694 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:08:38 smithi045 ceph-mon[31676]: scrub summary: idle+waiting paths [/] 2024-04-16T17:08:38.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:08:38 smithi117 ceph-mon[36730]: pgmap v59: 129 pgs: 129 active+clean; 26 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 514 KiB/s rd, 468 KiB/s wr, 108 op/s 2024-04-16T17:08:38.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:08:38 smithi117 ceph-mon[36730]: scrub summary: active paths [/] 2024-04-16T17:08:38.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:08:38 smithi117 ceph-mon[36730]: scrub summary: active paths [/] 2024-04-16T17:08:38.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:08:38 smithi117 ceph-mon[36730]: scrub summary: idle+waiting paths [/] 2024-04-16T17:08:38.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:08:38 smithi117 ceph-mon[36730]: scrub summary: idle+waiting paths [/] 2024-04-16T17:08:38.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:08:38 smithi104 ceph-mon[36457]: pgmap v59: 129 pgs: 129 active+clean; 26 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 514 KiB/s rd, 468 KiB/s wr, 108 op/s 2024-04-16T17:08:38.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:08:38 smithi104 ceph-mon[36457]: scrub summary: active paths [/] 2024-04-16T17:08:38.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:08:38 smithi104 ceph-mon[36457]: scrub summary: active paths [/] 2024-04-16T17:08:38.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:08:38 smithi104 ceph-mon[36457]: scrub summary: idle+waiting paths [/] 2024-04-16T17:08:38.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:08:38 smithi104 ceph-mon[36457]: scrub summary: idle+waiting paths [/] 2024-04-16T17:08:39.219 INFO:tasks.workunit.client.0.smithi045.stdout: 1 25818 3.80 MB/sec warmup 41 sec latency 86.195 ms 2024-04-16T17:08:39.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:08:39 smithi117 ceph-mon[36730]: scrub queued for path: / 2024-04-16T17:08:39.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:08:39 smithi117 ceph-mon[36730]: scrub summary: idle+waiting paths [/] 2024-04-16T17:08:39.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:08:39 smithi117 ceph-mon[36730]: scrub summary: active paths [/] 2024-04-16T17:08:39.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:08:39 smithi117 ceph-mon[36730]: scrub summary: idle+waiting paths [/] 2024-04-16T17:08:39.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:08:39 smithi104 ceph-mon[36457]: scrub queued for path: / 2024-04-16T17:08:39.957 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:08:39 smithi104 ceph-mon[36457]: scrub summary: idle+waiting paths [/] 2024-04-16T17:08:39.957 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:08:39 smithi104 ceph-mon[36457]: scrub summary: active paths [/] 2024-04-16T17:08:39.957 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:08:39 smithi104 ceph-mon[36457]: scrub summary: idle+waiting paths [/] 2024-04-16T17:08:40.219 INFO:tasks.workunit.client.0.smithi045.stdout: 1 26125 3.74 MB/sec warmup 42 sec latency 74.668 ms 2024-04-16T17:08:40.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:08:40 smithi117 ceph-mon[36730]: pgmap v60: 129 pgs: 129 active+clean; 26 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 398 KiB/s rd, 272 KiB/s wr, 83 op/s 2024-04-16T17:08:40.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:08:40 smithi104 ceph-mon[36457]: pgmap v60: 129 pgs: 129 active+clean; 26 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 398 KiB/s rd, 272 KiB/s wr, 83 op/s 2024-04-16T17:08:41.219 INFO:tasks.workunit.client.0.smithi045.stdout: 1 27034 3.73 MB/sec warmup 43 sec latency 24.226 ms 2024-04-16T17:08:42.219 INFO:tasks.workunit.client.0.smithi045.stdout: 1 28382 3.91 MB/sec warmup 44 sec latency 14.914 ms 2024-04-16T17:08:42.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:08:42 smithi117 ceph-mon[36730]: pgmap v61: 129 pgs: 129 active+clean; 27 MiB data, 451 MiB used, 1.0 TiB / 1.0 TiB avail; 722 KiB/s rd, 397 KiB/s wr, 117 op/s 2024-04-16T17:08:42.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:08:42 smithi104 ceph-mon[36457]: pgmap v61: 129 pgs: 129 active+clean; 27 MiB data, 451 MiB used, 1.0 TiB / 1.0 TiB avail; 722 KiB/s rd, 397 KiB/s wr, 117 op/s 2024-04-16T17:08:42.963 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:42.963+0000 7f5812000640 1 -- 172.21.15.45:0/4054025360 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f580c108190 msgr2=0x7f580c108570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:42.963 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:42.963+0000 7f5812000640 1 --2- 172.21.15.45:0/4054025360 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f580c108190 0x7f580c108570 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f57fc009900 tx=0x7f57fc02bcd0 comp rx=0 tx=0).stop 2024-04-16T17:08:42.964 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:42.964+0000 7f5812000640 1 -- 172.21.15.45:0/4054025360 shutdown_connections 2024-04-16T17:08:42.964 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:42.964+0000 7f5812000640 1 --2- 172.21.15.45:0/4054025360 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f580c06b1d0 0x7f580c10eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:42.964 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:42.964+0000 7f5812000640 1 --2- 172.21.15.45:0/4054025360 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f580c06a830 0x7f580c06ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:42.964 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:42.964+0000 7f5812000640 1 --2- 172.21.15.45:0/4054025360 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f580c108190 0x7f580c108570 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:42.964 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:42.964+0000 7f5812000640 1 -- 172.21.15.45:0/4054025360 >> 172.21.15.45:0/4054025360 conn(0x7f580c076d00 msgr2=0x7f580c077110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:08:42.964 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:42.964+0000 7f5812000640 1 -- 172.21.15.45:0/4054025360 shutdown_connections 2024-04-16T17:08:42.964 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:42.964+0000 7f5812000640 1 -- 172.21.15.45:0/4054025360 wait complete. 2024-04-16T17:08:42.964 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:42.965+0000 7f5812000640 1 Processor -- start 2024-04-16T17:08:42.964 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:42.965+0000 7f5812000640 1 -- start start 2024-04-16T17:08:42.966 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:42.965+0000 7f5812000640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f580c06a830 0x7f580c101320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:42.966 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:42.965+0000 7f5812000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f580c06b1d0 0x7f580c101860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:42.966 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:42.965+0000 7f5812000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f580c108190 0x7f580c101da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:42.966 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:42.965+0000 7f5812000640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f580c1123e0 con 0x7f580c06b1d0 2024-04-16T17:08:42.966 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:42.965+0000 7f5812000640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f580c112260 con 0x7f580c06a830 2024-04-16T17:08:42.966 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:42.965+0000 7f5812000640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f580c112560 con 0x7f580c108190 2024-04-16T17:08:42.966 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:42.965+0000 7f580b400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f580c06a830 0x7f580c101320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:42.966 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:42.965+0000 7f580b400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f580c06a830 0x7f580c101320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:46998/0 (socket says 172.21.15.45:46998) 2024-04-16T17:08:42.966 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:42.965+0000 7f580b400640 1 -- 172.21.15.45:0/2282387722 learned_addr learned my addr 172.21.15.45:0/2282387722 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:08:42.966 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:42.965+0000 7f580be00640 1 --2- 172.21.15.45:0/2282387722 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f580c108190 0x7f580c101da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:42.966 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:42.965+0000 7f580aa00640 1 --2- 172.21.15.45:0/2282387722 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f580c06b1d0 0x7f580c101860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:42.966 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:42.966+0000 7f580b400640 1 -- 172.21.15.45:0/2282387722 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f580c108190 msgr2=0x7f580c101da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:42.966 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:42.966+0000 7f580b400640 1 --2- 172.21.15.45:0/2282387722 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f580c108190 0x7f580c101da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:42.966 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:42.966+0000 7f580b400640 1 -- 172.21.15.45:0/2282387722 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f580c06b1d0 msgr2=0x7f580c101860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:42.966 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:42.966+0000 7f580b400640 1 --2- 172.21.15.45:0/2282387722 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f580c06b1d0 0x7f580c101860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:42.967 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:42.966+0000 7f580b400640 1 -- 172.21.15.45:0/2282387722 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f580c1a6ca0 con 0x7f580c06a830 2024-04-16T17:08:42.967 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:42.966+0000 7f580be00640 1 --2- 172.21.15.45:0/2282387722 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f580c108190 0x7f580c101da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:08:42.967 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:42.966+0000 7f580b400640 1 --2- 172.21.15.45:0/2282387722 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f580c06a830 0x7f580c101320 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f57fc000c00 tx=0x7f57fc004360 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:08:42.967 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:42.966+0000 7f5803e00640 1 -- 172.21.15.45:0/2282387722 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57fc042020 con 0x7f580c06a830 2024-04-16T17:08:42.968 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:42.967+0000 7f5803e00640 1 -- 172.21.15.45:0/2282387722 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f57fc0054e0 con 0x7f580c06a830 2024-04-16T17:08:42.968 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:42.967+0000 7f5812000640 1 -- 172.21.15.45:0/2282387722 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f580c1a6f30 con 0x7f580c06a830 2024-04-16T17:08:42.968 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:42.967+0000 7f5803e00640 1 -- 172.21.15.45:0/2282387722 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57fc003af0 con 0x7f580c06a830 2024-04-16T17:08:42.968 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:42.967+0000 7f5812000640 1 -- 172.21.15.45:0/2282387722 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f580c1a7440 con 0x7f580c06a830 2024-04-16T17:08:42.970 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:42.971+0000 7f5803e00640 1 -- 172.21.15.45:0/2282387722 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f57fc045020 con 0x7f580c06a830 2024-04-16T17:08:42.971 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:42.971+0000 7f5814d7e740 1 Processor -- start 2024-04-16T17:08:42.971 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:42.971+0000 7f5814d7e740 1 -- start start 2024-04-16T17:08:42.972 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:42.972+0000 7f5814d7e740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f5fd174610 0x55f5fd173360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:42.972 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:42.972+0000 7f5814d7e740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f5fd1738a0 0x55f5fd173d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:42.972 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:42.972+0000 7f5814d7e740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f5fd17ce20 0x55f5fd17f210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:42.972 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:42.972+0000 7f5814d7e740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55f5fd0d8780 con 0x55f5fd1738a0 2024-04-16T17:08:42.972 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:42.972+0000 7f5814d7e740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55f5fd07ef40 con 0x55f5fd174610 2024-04-16T17:08:42.972 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:42.972+0000 7f5814d7e740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55f5fd0d4740 con 0x55f5fd17ce20 2024-04-16T17:08:42.972 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:42.972+0000 7f5803e00640 1 --2- 172.21.15.45:0/2282387722 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f57f00a3e40 0x7f57f00a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:42.972 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:42.972+0000 7f580b400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f5fd1738a0 0x55f5fd173d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:42.973 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:42.972+0000 7f580aa00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f5fd17ce20 0x55f5fd17f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:42.973 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:42.972+0000 7f580be00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f5fd174610 0x55f5fd173360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:42.973 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:42.972+0000 7f580b400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f5fd1738a0 0x55f5fd173d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54956/0 (socket says 172.21.15.45:54956) 2024-04-16T17:08:42.973 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:42.972+0000 7f580b400640 1 -- 172.21.15.45:0/2475778635 learned_addr learned my addr 172.21.15.45:0/2475778635 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:08:42.973 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:42.973+0000 7f5803e00640 1 -- 172.21.15.45:0/2282387722 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f57fc0f2600 con 0x7f580c06a830 2024-04-16T17:08:42.974 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:42.973+0000 7f580b400640 1 -- 172.21.15.45:0/2475778635 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f5fd17ce20 msgr2=0x55f5fd17f210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:08:42.974 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:42.973+0000 7f580b400640 1 --2- 172.21.15.45:0/2475778635 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f5fd17ce20 0x55f5fd17f210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:42.974 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:42.973+0000 7f580b400640 1 -- 172.21.15.45:0/2475778635 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f5fd174610 msgr2=0x55f5fd173360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:08:42.974 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:42.973+0000 7f580b400640 1 --2- 172.21.15.45:0/2475778635 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f5fd174610 0x55f5fd173360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:42.974 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:42.973+0000 7f580b400640 1 -- 172.21.15.45:0/2475778635 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f5fcdeb210 con 0x55f5fd1738a0 2024-04-16T17:08:42.974 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:42.973+0000 7f580be00640 1 --2- 172.21.15.45:0/2282387722 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f57f00a3e40 0x7f57f00a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:42.974 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:42.973+0000 7f580b400640 1 --2- 172.21.15.45:0/2475778635 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f5fd1738a0 0x55f5fd173d20 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f57fc0ba710 tx=0x7f57fc0be200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e42c7ed939c971e9 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:08:42.974 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:42.974+0000 7f580be00640 1 --2- 172.21.15.45:0/2282387722 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f57f00a3e40 0x7f57f00a6300 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f58040094d0 tx=0x7f5804004ab0 comp rx=0 tx=0).ready entity=mgr.25411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:08:42.974 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:42.974+0000 7f57f7e00640 1 -- 172.21.15.45:0/2475778635 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57fc0bea00 con 0x55f5fd1738a0 2024-04-16T17:08:42.974 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:42.974+0000 7f57f7e00640 1 -- 172.21.15.45:0/2475778635 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f57fc0be300 con 0x55f5fd1738a0 2024-04-16T17:08:42.974 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:42.974+0000 7f57f7e00640 1 -- 172.21.15.45:0/2475778635 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57fc003c90 con 0x55f5fd1738a0 2024-04-16T17:08:42.975 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:42.974+0000 7f5814d7e740 1 -- 172.21.15.45:0/2475778635 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f5fd1738a0 msgr2=0x55f5fd173d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:42.975 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:42.974+0000 7f5814d7e740 1 --2- 172.21.15.45:0/2475778635 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f5fd1738a0 0x55f5fd173d20 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f57fc0ba710 tx=0x7f57fc0be200 comp rx=0 tx=0).stop 2024-04-16T17:08:42.975 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:42.975+0000 7f5814d7e740 1 -- 172.21.15.45:0/2475778635 shutdown_connections 2024-04-16T17:08:42.975 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:42.975+0000 7f5814d7e740 1 --2- 172.21.15.45:0/2475778635 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f5fd17ce20 0x55f5fd17f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:42.975 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:42.975+0000 7f5814d7e740 1 --2- 172.21.15.45:0/2475778635 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f5fd1738a0 0x55f5fd173d20 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:42.975 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:42.975+0000 7f5814d7e740 1 --2- 172.21.15.45:0/2475778635 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f5fd174610 0x55f5fd173360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:42.975 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:42.975+0000 7f5814d7e740 1 -- 172.21.15.45:0/2475778635 >> 172.21.15.45:0/2475778635 conn(0x55f5fd10e5d0 msgr2=0x55f5fd16e590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:08:42.975 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:42.975+0000 7f5814d7e740 1 -- 172.21.15.45:0/2475778635 shutdown_connections 2024-04-16T17:08:42.976 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:42.975+0000 7f5814d7e740 1 -- 172.21.15.45:0/2475778635 wait complete. 2024-04-16T17:08:42.976 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:42.975+0000 7f5814d7e740 1 -- start start 2024-04-16T17:08:42.977 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:42.976+0000 7f5814d7e740 1 Processor -- start 2024-04-16T17:08:42.978 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:42.976+0000 7f5814d7e740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f5fd1738a0 0x55f5fd17c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:42.978 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:42.976+0000 7f5814d7e740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f5fd174610 0x55f5fd179c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:42.978 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:42.976+0000 7f5814d7e740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f5fd17ce20 0x55f5fd17a340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:42.978 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:42.976+0000 7f5814d7e740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55f5fd17f910 con 0x55f5fd174610 2024-04-16T17:08:42.978 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:42.976+0000 7f5814d7e740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55f5fd092390 con 0x55f5fd17ce20 2024-04-16T17:08:42.978 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:42.976+0000 7f5814d7e740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55f5fd17fa90 con 0x55f5fd1738a0 2024-04-16T17:08:42.978 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:42.977+0000 7f5814d7e740 10 client.0 _ll_register_callbacks cb 0x55f5fd07bf60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:08:42.978 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:42.977+0000 7f580b400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f5fd1738a0 0x55f5fd17c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:42.978 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:42.977+0000 7f580be00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f5fd17ce20 0x55f5fd17a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:42.978 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:42.977+0000 7f580b400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f5fd1738a0 0x55f5fd17c8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:33170/0 (socket says 172.21.15.45:33170) 2024-04-16T17:08:42.978 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:42.977+0000 7f580b400640 1 -- 172.21.15.45:0/3589613247 learned_addr learned my addr 172.21.15.45:0/3589613247 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:08:42.978 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:42.977+0000 7f580aa00640 1 --2- 172.21.15.45:0/3589613247 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f5fd174610 0x55f5fd179c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:42.978 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:42.977+0000 7f580b400640 1 -- 172.21.15.45:0/3589613247 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f5fd17ce20 msgr2=0x55f5fd17a340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:08:42.978 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:42.977+0000 7f580b400640 1 --2- 172.21.15.45:0/3589613247 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f5fd17ce20 0x55f5fd17a340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:42.978 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:42.977+0000 7f580b400640 1 -- 172.21.15.45:0/3589613247 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f5fd174610 msgr2=0x55f5fd179c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:08:42.979 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:42.977+0000 7f580b400640 1 --2- 172.21.15.45:0/3589613247 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f5fd174610 0x55f5fd179c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:42.979 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:42.977+0000 7f580b400640 1 -- 172.21.15.45:0/3589613247 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f5fceb7620 con 0x55f5fd1738a0 2024-04-16T17:08:42.979 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:42.977+0000 7f580aa00640 1 --2- 172.21.15.45:0/3589613247 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f5fd174610 0x55f5fd179c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:08:42.979 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:42.977+0000 7f580b400640 1 --2- 172.21.15.45:0/3589613247 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f5fd1738a0 0x55f5fd17c8c0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f57fc03c040 tx=0x7f57fc0035f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cb8a61e6261c5fc1 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:08:42.979 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:42.977+0000 7f57f4c00640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-04-16T17:08:42.979 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:42.978+0000 7f57f4c00640 1 -- 172.21.15.45:0/3589613247 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57fc0c26b0 con 0x55f5fd1738a0 2024-04-16T17:08:42.979 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:42.978+0000 7f57f4c00640 1 -- 172.21.15.45:0/3589613247 <== mon.1 v2:172.21.15.117:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f57fc0efc20 con 0x55f5fd1738a0 2024-04-16T17:08:42.979 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:42.978+0000 7f5814d7e740 1 -- 172.21.15.45:0/3589613247 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f57fc0be300 con 0x55f5fd1738a0 2024-04-16T17:08:42.979 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:42.978+0000 7f57f4c00640 1 -- 172.21.15.45:0/3589613247 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f57fc0efe20 con 0x55f5fd1738a0 2024-04-16T17:08:42.979 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:42.978+0000 7f5814d7e740 10 client.26197 fetch_fsmap learned FSMap version 27 2024-04-16T17:08:42.979 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:42.978+0000 7f5814d7e740 1 -- 172.21.15.45:0/3589613247 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55f5fd17f910 con 0x55f5fd1738a0 2024-04-16T17:08:42.980 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:42.978+0000 7f57f4c00640 1 -- 172.21.15.45:0/3589613247 <== mon.1 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f57fc0be300 con 0x55f5fd1738a0 2024-04-16T17:08:42.980 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:42.978+0000 7f57f4c00640 1 -- 172.21.15.45:0/3589613247 <== mon.1 v2:172.21.15.117:3300/0 5 ==== fsmap(e 27) ==== 8358+0+0 (secure 0 0 0) 0x7f57fc0c6ae0 con 0x55f5fd1738a0 2024-04-16T17:08:42.980 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:42.978+0000 7f5814d7e740 10 client.26197 fetch_fsmap finished waiting for FSMap version 27 2024-04-16T17:08:42.981 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:42.978+0000 7f5814d7e740 10 client.26197 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T17:08:42.981 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:42.979+0000 7f5814d7e740 20 client.26197 populate_metadata read hostname 'smithi045' 2024-04-16T17:08:42.981 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:42.979+0000 7f5814d7e740 1 --2- 172.21.15.45:0/3589613247 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55f5fd195340 0x55f5fd197750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:42.981 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:42.979+0000 7f5814d7e740 4 client.26197 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:08:42.981 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:42.979+0000 7f5814d7e740 1 -- 172.21.15.45:0/3589613247 --> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f5fd0e61a0 con 0x55f5fd195340 2024-04-16T17:08:42.981 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:42.979+0000 7f580aa00640 1 --2- 172.21.15.45:0/3589613247 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55f5fd195340 0x55f5fd197750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:42.981 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:42.980+0000 7f580aa00640 1 --2- 172.21.15.45:0/3589613247 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55f5fd195340 0x55f5fd197750 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fa7cff385ebf82e8 server_cookie=6dcef820a6ce13c6 in_seq=0 out_seq=0 2024-04-16T17:08:42.981 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:42.980+0000 7f57f4c00640 10 client.26197 ms_handle_connect on v2:172.21.15.104:6838/3660070592 2024-04-16T17:08:42.981 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:42.980+0000 7f57f4c00640 1 -- 172.21.15.45:0/3589613247 <== mds.0 v2:172.21.15.104:6838/3660070592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f5fd0e61a0 con 0x55f5fd195340 2024-04-16T17:08:42.981 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:42.980+0000 7f57f4c00640 10 client.26197 handle_command_reply: tid=0 2024-04-16T17:08:42.981 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:42.980+0000 7f5814d7e740 1 client.26197 shutdown 2024-04-16T17:08:42.981 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:42.980+0000 7f5814d7e740 1 -- 172.21.15.45:0/3589613247 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f5fd1738a0 msgr2=0x55f5fd17c8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:42.981 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:42.980+0000 7f5814d7e740 1 --2- 172.21.15.45:0/3589613247 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f5fd1738a0 0x55f5fd17c8c0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f57fc03c040 tx=0x7f57fc0035f0 comp rx=0 tx=0).stop 2024-04-16T17:08:42.982 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:42.980+0000 7f5814d7e740 1 -- 172.21.15.45:0/3589613247 shutdown_connections 2024-04-16T17:08:42.982 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:42.980+0000 7f5814d7e740 1 --2- 172.21.15.45:0/3589613247 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f5fd17ce20 0x55f5fd17a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:42.982 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:42.980+0000 7f5814d7e740 1 --2- 172.21.15.45:0/3589613247 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f5fd174610 0x55f5fd179c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:42.982 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:42.980+0000 7f5814d7e740 1 --2- 172.21.15.45:0/3589613247 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55f5fd195340 0x55f5fd197750 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:42.982 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:42.980+0000 7f5814d7e740 1 --2- 172.21.15.45:0/3589613247 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f5fd1738a0 0x55f5fd17c8c0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:42.982 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:42.980+0000 7f57f4c00640 0 client.26197 ms_handle_reset on v2:172.21.15.104:6838/3660070592 2024-04-16T17:08:42.982 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:42.980+0000 7f5814d7e740 1 -- 172.21.15.45:0/3589613247 >> 172.21.15.45:0/3589613247 conn(0x55f5fd10e5d0 msgr2=0x55f5fd1702e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:08:42.982 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:42.980+0000 7f5814d7e740 1 -- 172.21.15.45:0/3589613247 shutdown_connections 2024-04-16T17:08:42.983 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:42.980+0000 7f5814d7e740 1 -- 172.21.15.45:0/3589613247 wait complete. 2024-04-16T17:08:42.983 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:42.980+0000 7f5814d7e740 20 client.26197 trim_cache size 0 max 16384 2024-04-16T17:08:43.011 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:43.010+0000 7f5814d7e740 1 Processor -- start 2024-04-16T17:08:43.011 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:43.011+0000 7f5814d7e740 1 -- start start 2024-04-16T17:08:43.011 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:43.011+0000 7f5814d7e740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f5fd1738a0 0x55f5fd16d530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:43.011 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:43.011+0000 7f5814d7e740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f5fd174610 0x55f5fd17c380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:43.011 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:43.011+0000 7f5814d7e740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f5fd17ce20 0x55f5fd17c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:43.011 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:43.011+0000 7f5814d7e740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f580c1afb30 con 0x55f5fd17ce20 2024-04-16T17:08:43.011 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:43.011+0000 7f5814d7e740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f580c1ad3d0 con 0x55f5fd1738a0 2024-04-16T17:08:43.011 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:43.011+0000 7f5814d7e740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f580c1ab240 con 0x55f5fd174610 2024-04-16T17:08:43.011 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:43.011+0000 7f580b400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f5fd1738a0 0x55f5fd16d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:43.012 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:43.011+0000 7f580b400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f5fd1738a0 0x55f5fd16d530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:47014/0 (socket says 172.21.15.45:47014) 2024-04-16T17:08:43.012 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:43.011+0000 7f580b400640 1 -- 172.21.15.45:0/1546572057 learned_addr learned my addr 172.21.15.45:0/1546572057 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:08:43.012 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:43.011+0000 7f580aa00640 1 --2- 172.21.15.45:0/1546572057 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f5fd174610 0x55f5fd17c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:43.013 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:43.012+0000 7f580be00640 1 --2- 172.21.15.45:0/1546572057 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f5fd17ce20 0x55f5fd17c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:43.013 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:43.012+0000 7f580b400640 1 -- 172.21.15.45:0/1546572057 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f5fd174610 msgr2=0x55f5fd17c380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:08:43.013 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:43.012+0000 7f580b400640 1 --2- 172.21.15.45:0/1546572057 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f5fd174610 0x55f5fd17c380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:43.013 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:43.012+0000 7f580b400640 1 -- 172.21.15.45:0/1546572057 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f5fd17ce20 msgr2=0x55f5fd17c8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:08:43.013 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:43.012+0000 7f580b400640 1 --2- 172.21.15.45:0/1546572057 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f5fd17ce20 0x55f5fd17c8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:43.013 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:43.012+0000 7f580b400640 1 -- 172.21.15.45:0/1546572057 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f5fd197dd0 con 0x55f5fd1738a0 2024-04-16T17:08:43.013 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:43.012+0000 7f580be00640 1 --2- 172.21.15.45:0/1546572057 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f5fd17ce20 0x55f5fd17c8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:08:43.013 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:43.012+0000 7f580b400640 1 --2- 172.21.15.45:0/1546572057 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f5fd1738a0 0x55f5fd16d530 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f57fc005ec0 tx=0x7f57fc02bcd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8791c961ca337149 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:08:43.013 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:43.012+0000 7f57f7400640 1 -- 172.21.15.45:0/1546572057 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57fc0bde40 con 0x55f5fd1738a0 2024-04-16T17:08:43.014 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:43.012+0000 7f57f7400640 1 -- 172.21.15.45:0/1546572057 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f57fc0bea00 con 0x55f5fd1738a0 2024-04-16T17:08:43.014 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:43.012+0000 7f57f7400640 1 -- 172.21.15.45:0/1546572057 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57fc0f21e0 con 0x55f5fd1738a0 2024-04-16T17:08:43.014 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:43.013+0000 7f5814d7e740 1 -- 172.21.15.45:0/1546572057 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f5fd1738a0 msgr2=0x55f5fd16d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:43.014 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:43.013+0000 7f5814d7e740 1 --2- 172.21.15.45:0/1546572057 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f5fd1738a0 0x55f5fd16d530 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f57fc005ec0 tx=0x7f57fc02bcd0 comp rx=0 tx=0).stop 2024-04-16T17:08:43.014 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:43.013+0000 7f5814d7e740 1 -- 172.21.15.45:0/1546572057 shutdown_connections 2024-04-16T17:08:43.014 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:43.013+0000 7f5814d7e740 1 --2- 172.21.15.45:0/1546572057 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f5fd174610 0x55f5fd17c380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:43.014 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:43.013+0000 7f5814d7e740 1 --2- 172.21.15.45:0/1546572057 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f5fd17ce20 0x55f5fd17c8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:43.014 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:43.013+0000 7f5814d7e740 1 --2- 172.21.15.45:0/1546572057 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f5fd1738a0 0x55f5fd16d530 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:43.014 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:43.013+0000 7f5814d7e740 1 -- 172.21.15.45:0/1546572057 >> 172.21.15.45:0/1546572057 conn(0x55f5fd10e5d0 msgr2=0x55f5fd16ce20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:08:43.014 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:43.013+0000 7f5814d7e740 1 -- 172.21.15.45:0/1546572057 shutdown_connections 2024-04-16T17:08:43.014 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:43.013+0000 7f5814d7e740 1 -- 172.21.15.45:0/1546572057 wait complete. 2024-04-16T17:08:43.016 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:43.014+0000 7f5814d7e740 1 -- start start 2024-04-16T17:08:43.016 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:43.014+0000 7f5814d7e740 1 Processor -- start 2024-04-16T17:08:43.016 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:43.015+0000 7f5814d7e740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f5fd1738a0 0x55f5fd18d820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:43.016 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:43.015+0000 7f5814d7e740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f5fd174610 0x55f5fd18a500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:43.016 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:43.015+0000 7f5814d7e740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f5fd17ce20 0x55f5fd18aa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:43.016 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:43.015+0000 7f5814d7e740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f580c1ab890 con 0x55f5fd174610 2024-04-16T17:08:43.016 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:43.015+0000 7f5814d7e740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f580c1acd00 con 0x55f5fd17ce20 2024-04-16T17:08:43.016 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:43.015+0000 7f5814d7e740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f580c1abf60 con 0x55f5fd1738a0 2024-04-16T17:08:43.016 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:43.015+0000 7f5814d7e740 10 client.0 _ll_register_callbacks cb 0x7f5804009ee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:08:43.017 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:43.015+0000 7f580b400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f5fd1738a0 0x55f5fd18d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:43.017 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:43.015+0000 7f580aa00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f5fd174610 0x55f5fd18a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:43.017 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:43.015+0000 7f580b400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f5fd1738a0 0x55f5fd18d820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:33190/0 (socket says 172.21.15.45:33190) 2024-04-16T17:08:43.017 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:43.015+0000 7f580b400640 1 -- 172.21.15.45:0/1435472985 learned_addr learned my addr 172.21.15.45:0/1435472985 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:08:43.017 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:43.015+0000 7f580be00640 1 --2- 172.21.15.45:0/1435472985 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f5fd17ce20 0x55f5fd18aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:43.017 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:43.016+0000 7f580be00640 1 -- 172.21.15.45:0/1435472985 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f5fd1738a0 msgr2=0x55f5fd18d820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:08:43.017 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:43.016+0000 7f580be00640 1 --2- 172.21.15.45:0/1435472985 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f5fd1738a0 0x55f5fd18d820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:43.017 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:43.016+0000 7f580be00640 1 -- 172.21.15.45:0/1435472985 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f5fd174610 msgr2=0x55f5fd18a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:08:43.017 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:43.016+0000 7f580be00640 1 --2- 172.21.15.45:0/1435472985 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f5fd174610 0x55f5fd18a500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:43.017 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:43.016+0000 7f580be00640 1 -- 172.21.15.45:0/1435472985 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f5fd172b70 con 0x55f5fd17ce20 2024-04-16T17:08:43.017 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:43.016+0000 7f580be00640 1 --2- 172.21.15.45:0/1435472985 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f5fd17ce20 0x55f5fd18aa40 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f5804005fd0 tx=0x7f5804004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ec0067a06d29e0c server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:08:43.017 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:43.016+0000 7f57f4c00640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-04-16T17:08:43.017 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:43.016+0000 7f57f4c00640 1 -- 172.21.15.45:0/1435472985 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5804004d20 con 0x55f5fd17ce20 2024-04-16T17:08:43.017 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:43.016+0000 7f5814d7e740 1 -- 172.21.15.45:0/1435472985 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f57fc0bea00 con 0x55f5fd17ce20 2024-04-16T17:08:43.017 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:43.016+0000 7f57f4c00640 1 -- 172.21.15.45:0/1435472985 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f5804005020 con 0x55f5fd17ce20 2024-04-16T17:08:43.018 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:43.017+0000 7f57f4c00640 1 -- 172.21.15.45:0/1435472985 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f5804033050 con 0x55f5fd17ce20 2024-04-16T17:08:43.018 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:43.017+0000 7f5814d7e740 10 client.26222 fetch_fsmap learned FSMap version 27 2024-04-16T17:08:43.019 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:43.017+0000 7f5814d7e740 1 -- 172.21.15.45:0/1435472985 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55f5fd1fc660 con 0x55f5fd17ce20 2024-04-16T17:08:43.019 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:43.017+0000 7f57f4c00640 1 -- 172.21.15.45:0/1435472985 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f57fc0bea00 con 0x55f5fd17ce20 2024-04-16T17:08:43.019 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:43.017+0000 7f57f4c00640 1 -- 172.21.15.45:0/1435472985 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 27) ==== 8358+0+0 (secure 0 0 0) 0x7f580401f040 con 0x55f5fd17ce20 2024-04-16T17:08:43.019 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:43.017+0000 7f5814d7e740 10 client.26222 fetch_fsmap finished waiting for FSMap version 27 2024-04-16T17:08:43.019 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:43.017+0000 7f5814d7e740 10 client.26222 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T17:08:43.019 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:43.017+0000 7f5814d7e740 20 client.26222 populate_metadata read hostname 'smithi045' 2024-04-16T17:08:43.019 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:43.017+0000 7f5814d7e740 1 --2- 172.21.15.45:0/1435472985 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55f5fd1fce70 0x55f5fd1ff290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:43.019 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:43.017+0000 7f5814d7e740 4 client.26222 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-16T17:08:43.019 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:43.017+0000 7f5814d7e740 1 -- 172.21.15.45:0/1435472985 --> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f5fd0e0010 con 0x55f5fd1fce70 2024-04-16T17:08:43.019 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:43.018+0000 7f580b400640 1 --2- 172.21.15.45:0/1435472985 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55f5fd1fce70 0x55f5fd1ff290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:43.019 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:43.018+0000 7f580b400640 1 --2- 172.21.15.45:0/1435472985 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55f5fd1fce70 0x55f5fd1ff290 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=541be7eb97bfe972 server_cookie=f725a5ed34ce4ecd in_seq=0 out_seq=0 2024-04-16T17:08:43.019 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:43.018+0000 7f57f4c00640 10 client.26222 ms_handle_connect on v2:172.21.15.104:6838/3660070592 2024-04-16T17:08:43.082 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:43.081+0000 7f57f4c00640 1 -- 172.21.15.45:0/1435472985 <== mds.0 v2:172.21.15.104:6838/3660070592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f5fd0e0010 con 0x55f5fd1fce70 2024-04-16T17:08:43.082 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:43.081+0000 7f57f4c00640 10 client.26222 handle_command_reply: tid=0 2024-04-16T17:08:43.082 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:43.081+0000 7f5814d7e740 1 client.26222 shutdown 2024-04-16T17:08:43.082 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:43.081+0000 7f5814d7e740 1 -- 172.21.15.45:0/1435472985 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f5fd17ce20 msgr2=0x55f5fd18aa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:43.082 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:43.082+0000 7f5814d7e740 1 --2- 172.21.15.45:0/1435472985 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f5fd17ce20 0x55f5fd18aa40 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f5804005fd0 tx=0x7f5804004a40 comp rx=0 tx=0).stop 2024-04-16T17:08:43.082 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:43.082+0000 7f5814d7e740 1 -- 172.21.15.45:0/1435472985 shutdown_connections 2024-04-16T17:08:43.082 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:43.082+0000 7f5814d7e740 1 --2- 172.21.15.45:0/1435472985 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f5fd17ce20 0x55f5fd18aa40 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:43.082 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:43.082+0000 7f5814d7e740 1 --2- 172.21.15.45:0/1435472985 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f5fd174610 0x55f5fd18a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:43.082 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:43.082+0000 7f5814d7e740 1 --2- 172.21.15.45:0/1435472985 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55f5fd1fce70 0x55f5fd1ff290 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:43.082 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:43.082+0000 7f5814d7e740 1 --2- 172.21.15.45:0/1435472985 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f5fd1738a0 0x55f5fd18d820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:43.082 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:43.082+0000 7f57f4c00640 0 client.26222 ms_handle_reset on v2:172.21.15.104:6838/3660070592 2024-04-16T17:08:43.082 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:43.082+0000 7f5814d7e740 1 -- 172.21.15.45:0/1435472985 >> 172.21.15.45:0/1435472985 conn(0x55f5fd10e5d0 msgr2=0x55f5fd16d6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:08:43.082 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:43.082+0000 7f5814d7e740 1 -- 172.21.15.45:0/1435472985 shutdown_connections 2024-04-16T17:08:43.082 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:43.082+0000 7f5814d7e740 1 -- 172.21.15.45:0/1435472985 wait complete. 2024-04-16T17:08:43.083 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:43.082+0000 7f5814d7e740 20 client.26222 trim_cache size 0 max 16384 2024-04-16T17:08:43.083 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-04-16T17:08:43.083 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-04-16T17:08:43.083 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-04-16T17:08:43.083 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-04-16T17:08:43.084 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:43.083+0000 7f5812000640 1 -- 172.21.15.45:0/2282387722 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f57f00a3e40 msgr2=0x7f57f00a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:43.084 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:43.083+0000 7f5812000640 1 --2- 172.21.15.45:0/2282387722 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f57f00a3e40 0x7f57f00a6300 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f58040094d0 tx=0x7f5804004ab0 comp rx=0 tx=0).stop 2024-04-16T17:08:43.084 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:43.083+0000 7f5812000640 1 -- 172.21.15.45:0/2282387722 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f580c06a830 msgr2=0x7f580c101320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:43.084 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:43.083+0000 7f5812000640 1 --2- 172.21.15.45:0/2282387722 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f580c06a830 0x7f580c101320 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f57fc000c00 tx=0x7f57fc004360 comp rx=0 tx=0).stop 2024-04-16T17:08:43.084 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:43.083+0000 7f5812000640 1 -- 172.21.15.45:0/2282387722 shutdown_connections 2024-04-16T17:08:43.084 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:43.083+0000 7f5812000640 1 --2- 172.21.15.45:0/2282387722 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f57f00a3e40 0x7f57f00a6300 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:43.084 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:43.083+0000 7f5812000640 1 --2- 172.21.15.45:0/2282387722 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f580c108190 0x7f580c101da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:43.084 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:43.083+0000 7f5812000640 1 --2- 172.21.15.45:0/2282387722 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f580c06b1d0 0x7f580c101860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:43.084 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:43.083+0000 7f5812000640 1 --2- 172.21.15.45:0/2282387722 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f580c06a830 0x7f580c101320 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:43.084 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:43.083+0000 7f5812000640 1 -- 172.21.15.45:0/2282387722 >> 172.21.15.45:0/2282387722 conn(0x7f580c076d00 msgr2=0x7f580c0ffc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:08:43.084 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:43.083+0000 7f5812000640 1 -- 172.21.15.45:0/2282387722 shutdown_connections 2024-04-16T17:08:43.084 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:43.083+0000 7f5812000640 1 -- 172.21.15.45:0/2282387722 wait complete. 2024-04-16T17:08:43.219 INFO:tasks.workunit.client.0.smithi045.stdout: 1 28737 3.86 MB/sec warmup 45 sec latency 38.536 ms 2024-04-16T17:08:43.799 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:08:39 smithi045 ceph-mon[31676]: scrub queued for path: / 2024-04-16T17:08:43.799 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:08:39 smithi045 ceph-mon[31676]: scrub summary: idle+waiting paths [/] 2024-04-16T17:08:43.800 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:08:39 smithi045 ceph-mon[31676]: scrub summary: active paths [/] 2024-04-16T17:08:43.800 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:08:39 smithi045 ceph-mon[31676]: scrub summary: idle+waiting paths [/] 2024-04-16T17:08:43.800 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:08:40 smithi045 ceph-mon[31676]: pgmap v60: 129 pgs: 129 active+clean; 26 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 398 KiB/s rd, 272 KiB/s wr, 83 op/s 2024-04-16T17:08:43.800 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:08:42 smithi045 ceph-mon[31676]: pgmap v61: 129 pgs: 129 active+clean; 27 MiB data, 451 MiB used, 1.0 TiB / 1.0 TiB avail; 722 KiB/s rd, 397 KiB/s wr, 117 op/s 2024-04-16T17:08:43.800 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:08:43 smithi045 ceph-mon[31676]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-16T17:08:43.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:08:43 smithi117 ceph-mon[36730]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-16T17:08:43.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:08:43 smithi104 ceph-mon[36457]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-16T17:08:44.219 INFO:tasks.workunit.client.0.smithi045.stdout: 1 28908 3.78 MB/sec warmup 46 sec latency 85.007 ms 2024-04-16T17:08:44.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:08:44 smithi117 ceph-mon[36730]: pgmap v62: 129 pgs: 129 active+clean; 27 MiB data, 451 MiB used, 1.0 TiB / 1.0 TiB avail; 374 KiB/s rd, 176 KiB/s wr, 48 op/s 2024-04-16T17:08:44.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:08:44 smithi117 ceph-mon[36730]: scrub summary: idle 2024-04-16T17:08:44.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:08:44 smithi117 ceph-mon[36730]: scrub summary: idle 2024-04-16T17:08:44.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:08:44 smithi117 ceph-mon[36730]: scrub summary: idle 2024-04-16T17:08:44.828 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:08:44 smithi104 ceph-mon[36457]: pgmap v62: 129 pgs: 129 active+clean; 27 MiB data, 451 MiB used, 1.0 TiB / 1.0 TiB avail; 374 KiB/s rd, 176 KiB/s wr, 48 op/s 2024-04-16T17:08:44.828 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:08:44 smithi104 ceph-mon[36457]: scrub summary: idle 2024-04-16T17:08:44.828 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:08:44 smithi104 ceph-mon[36457]: scrub summary: idle 2024-04-16T17:08:44.828 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:08:44 smithi104 ceph-mon[36457]: scrub summary: idle 2024-04-16T17:08:45.219 INFO:tasks.workunit.client.0.smithi045.stdout: 1 29206 3.71 MB/sec warmup 47 sec latency 172.839 ms 2024-04-16T17:08:45.319 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:08:44 smithi045 ceph-mon[31676]: pgmap v62: 129 pgs: 129 active+clean; 27 MiB data, 451 MiB used, 1.0 TiB / 1.0 TiB avail; 374 KiB/s rd, 176 KiB/s wr, 48 op/s 2024-04-16T17:08:45.319 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:08:44 smithi045 ceph-mon[31676]: scrub summary: idle 2024-04-16T17:08:45.319 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:08:44 smithi045 ceph-mon[31676]: scrub summary: idle 2024-04-16T17:08:45.319 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:08:44 smithi045 ceph-mon[31676]: scrub summary: idle 2024-04-16T17:08:46.219 INFO:tasks.workunit.client.0.smithi045.stdout: 1 29656 3.67 MB/sec warmup 48 sec latency 138.496 ms 2024-04-16T17:08:46.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:08:46 smithi117 ceph-mon[36730]: pgmap v63: 129 pgs: 129 active+clean; 27 MiB data, 451 MiB used, 1.0 TiB / 1.0 TiB avail; 374 KiB/s rd, 176 KiB/s wr, 48 op/s 2024-04-16T17:08:46.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:08:46 smithi104 ceph-mon[36457]: pgmap v63: 129 pgs: 129 active+clean; 27 MiB data, 451 MiB used, 1.0 TiB / 1.0 TiB avail; 374 KiB/s rd, 176 KiB/s wr, 48 op/s 2024-04-16T17:08:47.219 INFO:tasks.workunit.client.0.smithi045.stdout: 1 30819 3.69 MB/sec warmup 49 sec latency 37.250 ms 2024-04-16T17:08:48.219 INFO:tasks.workunit.client.0.smithi045.stdout: 1 31331 3.73 MB/sec warmup 50 sec latency 82.359 ms 2024-04-16T17:08:48.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:08:48 smithi117 ceph-mon[36730]: pgmap v64: 129 pgs: 129 active+clean; 36 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 546 KiB/s rd, 1008 KiB/s wr, 112 op/s 2024-04-16T17:08:48.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:08:48 smithi117 ceph-mon[36730]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-16T17:08:48.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:08:48 smithi104 ceph-mon[36457]: pgmap v64: 129 pgs: 129 active+clean; 36 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 546 KiB/s rd, 1008 KiB/s wr, 112 op/s 2024-04-16T17:08:48.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:08:48 smithi104 ceph-mon[36457]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-16T17:08:49.220 INFO:tasks.workunit.client.0.smithi045.stdout: 1 31727 3.69 MB/sec warmup 51 sec latency 74.405 ms 2024-04-16T17:08:49.837 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:08:46 smithi045 ceph-mon[31676]: pgmap v63: 129 pgs: 129 active+clean; 27 MiB data, 451 MiB used, 1.0 TiB / 1.0 TiB avail; 374 KiB/s rd, 176 KiB/s wr, 48 op/s 2024-04-16T17:08:49.838 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:08:48 smithi045 ceph-mon[31676]: pgmap v64: 129 pgs: 129 active+clean; 36 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 546 KiB/s rd, 1008 KiB/s wr, 112 op/s 2024-04-16T17:08:49.838 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:08:48 smithi045 ceph-mon[31676]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-16T17:08:50.220 INFO:tasks.workunit.client.0.smithi045.stdout: 1 32345 3.71 MB/sec warmup 52 sec latency 16.143 ms 2024-04-16T17:08:50.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:08:50 smithi117 ceph-mon[36730]: pgmap v65: 129 pgs: 129 active+clean; 36 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 496 KiB/s rd, 958 KiB/s wr, 97 op/s 2024-04-16T17:08:50.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:08:50 smithi104 ceph-mon[36457]: pgmap v65: 129 pgs: 129 active+clean; 36 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 496 KiB/s rd, 958 KiB/s wr, 97 op/s 2024-04-16T17:08:51.221 INFO:tasks.workunit.client.0.smithi045.stdout: 1 32631 3.64 MB/sec warmup 53 sec latency 297.539 ms 2024-04-16T17:08:51.227 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:08:50 smithi045 ceph-mon[31676]: pgmap v65: 129 pgs: 129 active+clean; 36 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 496 KiB/s rd, 958 KiB/s wr, 97 op/s 2024-04-16T17:08:52.221 INFO:tasks.workunit.client.0.smithi045.stdout: 1 32631 3.58 MB/sec warmup 54 sec latency 1297.631 ms 2024-04-16T17:08:52.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:08:52 smithi117 ceph-mon[36730]: pgmap v66: 129 pgs: 129 active+clean; 37 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 910 KiB/s rd, 1.7 MiB/s wr, 167 op/s 2024-04-16T17:08:52.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:08:52 smithi104 ceph-mon[36457]: pgmap v66: 129 pgs: 129 active+clean; 37 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 910 KiB/s rd, 1.7 MiB/s wr, 167 op/s 2024-04-16T17:08:52.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:08:52 smithi045 ceph-mon[31676]: pgmap v66: 129 pgs: 129 active+clean; 37 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 910 KiB/s rd, 1.7 MiB/s wr, 167 op/s 2024-04-16T17:08:53.221 INFO:tasks.workunit.client.0.smithi045.stdout: 1 32631 3.51 MB/sec warmup 55 sec latency 2297.690 ms 2024-04-16T17:08:54.221 INFO:tasks.workunit.client.0.smithi045.stdout: 1 32631 3.45 MB/sec warmup 56 sec latency 3297.745 ms 2024-04-16T17:08:54.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:08:54 smithi117 ceph-mon[36730]: pgmap v67: 129 pgs: 129 active+clean; 37 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 586 KiB/s rd, 1.6 MiB/s wr, 133 op/s 2024-04-16T17:08:54.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:08:54 smithi104 ceph-mon[36457]: pgmap v67: 129 pgs: 129 active+clean; 37 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 586 KiB/s rd, 1.6 MiB/s wr, 133 op/s 2024-04-16T17:08:54.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:08:54 smithi045 ceph-mon[31676]: pgmap v67: 129 pgs: 129 active+clean; 37 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 586 KiB/s rd, 1.6 MiB/s wr, 133 op/s 2024-04-16T17:08:55.221 INFO:tasks.workunit.client.0.smithi045.stdout: 1 32631 3.39 MB/sec warmup 57 sec latency 4297.801 ms 2024-04-16T17:08:55.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:08:55 smithi117 ceph-mon[36730]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T17:08:55.854 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:08:55 smithi045 ceph-mon[31676]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T17:08:55.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:08:55 smithi104 ceph-mon[36457]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T17:08:56.218 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:56.218+0000 7f41f4e00640 1 -- 172.21.15.45:0/3184595043 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f41f0102080 msgr2=0x7f41f0102460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:56.219 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:56.218+0000 7f41f4e00640 1 --2- 172.21.15.45:0/3184595043 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f41f0102080 0x7f41f0102460 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f41e0009900 tx=0x7f41e002bc30 comp rx=0 tx=0).stop 2024-04-16T17:08:56.219 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:56.218+0000 7f41f4e00640 1 -- 172.21.15.45:0/3184595043 shutdown_connections 2024-04-16T17:08:56.219 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:56.218+0000 7f41f4e00640 1 --2- 172.21.15.45:0/3184595043 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f41f0102a30 0x7f41f0106a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:56.219 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:56.218+0000 7f41f4e00640 1 --2- 172.21.15.45:0/3184595043 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f41f0107150 0x7f41f010ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:56.219 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:56.218+0000 7f41f4e00640 1 --2- 172.21.15.45:0/3184595043 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f41f0102080 0x7f41f0102460 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:56.219 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:56.218+0000 7f41f4e00640 1 -- 172.21.15.45:0/3184595043 >> 172.21.15.45:0/3184595043 conn(0x7f41f00fd980 msgr2=0x7f41f00ffda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:08:56.219 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:56.218+0000 7f41f4e00640 1 -- 172.21.15.45:0/3184595043 shutdown_connections 2024-04-16T17:08:56.219 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:56.218+0000 7f41f4e00640 1 -- 172.21.15.45:0/3184595043 wait complete. 2024-04-16T17:08:56.220 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:56.220+0000 7f41f4e00640 1 Processor -- start 2024-04-16T17:08:56.220 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:56.220+0000 7f41f4e00640 1 -- start start 2024-04-16T17:08:56.221 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:56.220+0000 7f41f4e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f41f0102080 0x7f41f01a0670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:56.221 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:56.220+0000 7f41f4e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f41f0107150 0x7f41f01a0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:56.221 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:56.221+0000 7f41f4e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f41f019a730 0x7f41f019abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:56.221 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:56.221+0000 7f41f4e00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f41f0112230 con 0x7f41f019a730 2024-04-16T17:08:56.221 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:56.221+0000 7f41f4e00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f41f01120b0 con 0x7f41f0107150 2024-04-16T17:08:56.221 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:56.221+0000 7f41f4e00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f41f01123b0 con 0x7f41f0102080 2024-04-16T17:08:56.221 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:56.221+0000 7f41e6a00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f41f0107150 0x7f41f01a0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:56.221 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:56.221+0000 7f41ef400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f41f0102080 0x7f41f01a0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:56.221 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:56.221+0000 7f41efe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f41f019a730 0x7f41f019abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:56.221 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:56.221+0000 7f41e6a00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f41f0107150 0x7f41f01a0bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:56184/0 (socket says 172.21.15.45:56184) 2024-04-16T17:08:56.221 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:56.221+0000 7f41e6a00640 1 -- 172.21.15.45:0/4183826823 learned_addr learned my addr 172.21.15.45:0/4183826823 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:08:56.222 INFO:tasks.workunit.client.0.smithi045.stdout: 1 32687 3.33 MB/sec warmup 58 sec latency 4804.185 ms 2024-04-16T17:08:56.223 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:56.221+0000 7f41ef400640 1 -- 172.21.15.45:0/4183826823 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f41f0107150 msgr2=0x7f41f01a0bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:56.223 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:56.221+0000 7f41ef400640 1 --2- 172.21.15.45:0/4183826823 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f41f0107150 0x7f41f01a0bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:56.223 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:56.221+0000 7f41ef400640 1 -- 172.21.15.45:0/4183826823 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f41f019a730 msgr2=0x7f41f019abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:56.223 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:56.221+0000 7f41ef400640 1 --2- 172.21.15.45:0/4183826823 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f41f019a730 0x7f41f019abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:56.223 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:56.221+0000 7f41ef400640 1 -- 172.21.15.45:0/4183826823 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41f019b330 con 0x7f41f0102080 2024-04-16T17:08:56.223 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:56.221+0000 7f41e6a00640 1 --2- 172.21.15.45:0/4183826823 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f41f0107150 0x7f41f01a0bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:08:56.223 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:56.222+0000 7f41ef400640 1 --2- 172.21.15.45:0/4183826823 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f41f0102080 0x7f41f01a0670 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f41e002c170 tx=0x7f41e0004360 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:08:56.223 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:56.222+0000 7f41ecc00640 1 -- 172.21.15.45:0/4183826823 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41e0043020 con 0x7f41f0102080 2024-04-16T17:08:56.223 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:56.222+0000 7f41ecc00640 1 -- 172.21.15.45:0/4183826823 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f41e00054e0 con 0x7f41f0102080 2024-04-16T17:08:56.223 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:56.222+0000 7f41f4e00640 1 -- 172.21.15.45:0/4183826823 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41f019b5c0 con 0x7f41f0102080 2024-04-16T17:08:56.223 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:56.222+0000 7f41ecc00640 1 -- 172.21.15.45:0/4183826823 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41e0003b80 con 0x7f41f0102080 2024-04-16T17:08:56.223 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:56.222+0000 7f41f4e00640 1 -- 172.21.15.45:0/4183826823 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41f01a77b0 con 0x7f41f0102080 2024-04-16T17:08:56.226 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:56.226+0000 7f41f922c740 1 Processor -- start 2024-04-16T17:08:56.226 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:56.226+0000 7f41f922c740 1 -- start start 2024-04-16T17:08:56.227 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:56.227+0000 7f41f922c740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564bd46bd7f0 0x564bd46bc540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:56.227 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:56.227+0000 7f41f922c740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564bd46bca80 0x564bd46bcf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:56.227 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:56.227+0000 7f41f922c740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564bd46c6000 0x564bd46c83f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:56.227 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:56.227+0000 7f41f922c740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x564bd4621780 con 0x564bd46c6000 2024-04-16T17:08:56.227 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:56.227+0000 7f41f922c740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x564bd45c7f40 con 0x564bd46bca80 2024-04-16T17:08:56.227 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:56.227+0000 7f41f922c740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x564bd461d740 con 0x564bd46bd7f0 2024-04-16T17:08:56.227 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:56.227+0000 7f41ecc00640 1 -- 172.21.15.45:0/4183826823 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f41e0046020 con 0x7f41f0102080 2024-04-16T17:08:56.228 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:56.227+0000 7f41e6a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564bd46c6000 0x564bd46c83f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:56.228 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:56.227+0000 7f41efe00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564bd46bd7f0 0x564bd46bc540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:56.228 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:56.227+0000 7f41efe00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564bd46bd7f0 0x564bd46bc540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:34386/0 (socket says 172.21.15.45:34386) 2024-04-16T17:08:56.228 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:56.227+0000 7f41e6a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564bd46c6000 0x564bd46c83f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41928/0 (socket says 172.21.15.45:41928) 2024-04-16T17:08:56.228 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:56.227+0000 7f41efe00640 1 -- 172.21.15.45:0/1317937065 learned_addr learned my addr 172.21.15.45:0/1317937065 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:08:56.228 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:56.227+0000 7f41ef400640 1 --2- 172.21.15.45:0/1317937065 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564bd46bca80 0x564bd46bcf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:56.229 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:56.228+0000 7f41efe00640 1 -- 172.21.15.45:0/1317937065 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564bd46bca80 msgr2=0x564bd46bcf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:08:56.229 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:56.228+0000 7f41efe00640 1 --2- 172.21.15.45:0/1317937065 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564bd46bca80 0x564bd46bcf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:56.229 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:56.228+0000 7f41efe00640 1 -- 172.21.15.45:0/1317937065 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564bd46c6000 msgr2=0x564bd46c83f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:08:56.229 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:56.228+0000 7f41efe00640 1 --2- 172.21.15.45:0/1317937065 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564bd46c6000 0x564bd46c83f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:56.229 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:56.228+0000 7f41efe00640 1 -- 172.21.15.45:0/1317937065 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564bd4334210 con 0x564bd46bd7f0 2024-04-16T17:08:56.229 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:56.228+0000 7f41efe00640 1 --2- 172.21.15.45:0/1317937065 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564bd46bd7f0 0x564bd46bc540 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f41e8009f50 tx=0x7f41e80118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c66cc68d850f296d server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:08:56.229 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:56.228+0000 7f41e4c00640 1 -- 172.21.15.45:0/1317937065 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41e8004d20 con 0x564bd46bd7f0 2024-04-16T17:08:56.229 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:56.228+0000 7f41e4c00640 1 -- 172.21.15.45:0/1317937065 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f41e8004050 con 0x564bd46bd7f0 2024-04-16T17:08:56.229 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:56.228+0000 7f41e4c00640 1 -- 172.21.15.45:0/1317937065 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41e8005280 con 0x564bd46bd7f0 2024-04-16T17:08:56.229 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:56.229+0000 7f41ecc00640 1 --2- 172.21.15.45:0/4183826823 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f41d40a3e10 0x7f41d40a62d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:56.231 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:56.229+0000 7f41ecc00640 1 -- 172.21.15.45:0/4183826823 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f41e00f29c0 con 0x7f41f0102080 2024-04-16T17:08:56.231 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:56.229+0000 7f41f922c740 1 -- 172.21.15.45:0/1317937065 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564bd46bd7f0 msgr2=0x564bd46bc540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:56.231 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:56.229+0000 7f41f922c740 1 --2- 172.21.15.45:0/1317937065 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564bd46bd7f0 0x564bd46bc540 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f41e8009f50 tx=0x7f41e80118d0 comp rx=0 tx=0).stop 2024-04-16T17:08:56.232 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:56.229+0000 7f41ef400640 1 --2- 172.21.15.45:0/4183826823 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f41d40a3e10 0x7f41d40a62d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:56.232 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:56.229+0000 7f41f922c740 1 -- 172.21.15.45:0/1317937065 shutdown_connections 2024-04-16T17:08:56.232 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:56.229+0000 7f41f922c740 1 --2- 172.21.15.45:0/1317937065 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564bd46c6000 0x564bd46c83f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:56.232 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:56.229+0000 7f41f922c740 1 --2- 172.21.15.45:0/1317937065 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564bd46bca80 0x564bd46bcf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:56.232 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:56.229+0000 7f41f922c740 1 --2- 172.21.15.45:0/1317937065 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564bd46bd7f0 0x564bd46bc540 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:56.232 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:56.229+0000 7f41f922c740 1 -- 172.21.15.45:0/1317937065 >> 172.21.15.45:0/1317937065 conn(0x564bd46575d0 msgr2=0x564bd46b7770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:08:56.232 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:56.230+0000 7f41f922c740 1 -- 172.21.15.45:0/1317937065 shutdown_connections 2024-04-16T17:08:56.232 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:56.230+0000 7f41f922c740 1 -- 172.21.15.45:0/1317937065 wait complete. 2024-04-16T17:08:56.232 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:56.230+0000 7f41f922c740 1 -- start start 2024-04-16T17:08:56.232 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:56.230+0000 7f41ef400640 1 --2- 172.21.15.45:0/4183826823 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f41d40a3e10 0x7f41d40a62d0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x564bd46bdf50 tx=0x7f41e00035f0 comp rx=0 tx=0).ready entity=mgr.25411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:08:56.232 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:56.230+0000 7f41f922c740 1 Processor -- start 2024-04-16T17:08:56.232 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:56.231+0000 7f41f922c740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564bd46bca80 0x564bd46c5ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:56.232 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:56.231+0000 7f41f922c740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564bd46bd7f0 0x564bd46c2e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:56.232 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:56.231+0000 7f41f922c740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564bd46c6000 0x564bd46c3590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:56.232 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:56.231+0000 7f41f922c740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x564bd46c8d30 con 0x564bd46bca80 2024-04-16T17:08:56.233 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:56.231+0000 7f41f922c740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x564bd45db390 con 0x564bd46bd7f0 2024-04-16T17:08:56.233 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:56.231+0000 7f41f922c740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x564bd46c8eb0 con 0x564bd46c6000 2024-04-16T17:08:56.233 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:56.231+0000 7f41f922c740 10 client.0 _ll_register_callbacks cb 0x564bd45c4f60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:08:56.233 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:56.231+0000 7f41efe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564bd46bca80 0x564bd46c5ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:56.233 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:56.231+0000 7f41efe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564bd46bca80 0x564bd46c5ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41942/0 (socket says 172.21.15.45:41942) 2024-04-16T17:08:56.233 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:56.231+0000 7f41efe00640 1 -- 172.21.15.45:0/1785962404 learned_addr learned my addr 172.21.15.45:0/1785962404 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:08:56.233 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:56.231+0000 7f41e6a00640 1 --2- 172.21.15.45:0/1785962404 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564bd46bd7f0 0x564bd46c2e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:56.233 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:56.231+0000 7f41efe00640 1 --2- 172.21.15.45:0/1785962404 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564bd46c6000 0x564bd46c3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:56.233 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:56.232+0000 7f41efe00640 1 -- 172.21.15.45:0/1785962404 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564bd46c6000 msgr2=0x564bd46c3590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:08:56.233 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:56.232+0000 7f41efe00640 1 --2- 172.21.15.45:0/1785962404 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564bd46c6000 0x564bd46c3590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:56.233 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:56.232+0000 7f41efe00640 1 -- 172.21.15.45:0/1785962404 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564bd46bd7f0 msgr2=0x564bd46c2e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:08:56.233 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:56.232+0000 7f41efe00640 1 --2- 172.21.15.45:0/1785962404 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564bd46bd7f0 0x564bd46c2e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:56.233 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:56.232+0000 7f41efe00640 1 -- 172.21.15.45:0/1785962404 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564bd4400620 con 0x564bd46bca80 2024-04-16T17:08:56.235 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:56.232+0000 7f41efe00640 1 --2- 172.21.15.45:0/1785962404 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564bd46bca80 0x564bd46c5ac0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f41e801b040 tx=0x7f41e80047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bef54f1f3a68ceac server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:08:56.235 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:56.232+0000 7f41dd600640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-16T17:08:56.235 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:56.232+0000 7f41dd600640 1 -- 172.21.15.45:0/1785962404 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41e802b070 con 0x564bd46bca80 2024-04-16T17:08:56.235 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:56.232+0000 7f41dd600640 1 -- 172.21.15.45:0/1785962404 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f41e8025450 con 0x564bd46bca80 2024-04-16T17:08:56.235 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:56.232+0000 7f41f922c740 1 -- 172.21.15.45:0/1785962404 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f41e8004050 con 0x564bd46bca80 2024-04-16T17:08:56.235 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:56.233+0000 7f41dd600640 1 -- 172.21.15.45:0/1785962404 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f41e8003e40 con 0x564bd46bca80 2024-04-16T17:08:56.235 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:56.233+0000 7f41f922c740 10 client.16086 fetch_fsmap learned FSMap version 28 2024-04-16T17:08:56.235 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:56.233+0000 7f41f922c740 1 -- 172.21.15.45:0/1785962404 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x564bd46d4e20 con 0x564bd46bca80 2024-04-16T17:08:56.235 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:56.233+0000 7f41dd600640 1 -- 172.21.15.45:0/1785962404 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f41e8004050 con 0x564bd46bca80 2024-04-16T17:08:56.235 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:56.234+0000 7f41dd600640 1 -- 172.21.15.45:0/1785962404 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f41e8027440 con 0x564bd46bca80 2024-04-16T17:08:56.235 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:56.234+0000 7f41f922c740 10 client.16086 fetch_fsmap finished waiting for FSMap version 28 2024-04-16T17:08:56.235 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:56.234+0000 7f41f922c740 10 client.16086 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T17:08:56.235 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:56.234+0000 7f41f922c740 20 client.16086 populate_metadata read hostname 'smithi045' 2024-04-16T17:08:56.235 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:56.234+0000 7f41f922c740 1 --2- 172.21.15.45:0/1785962404 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x564bd46d6280 0x564bd46d8670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:56.236 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:56.234+0000 7f41f922c740 4 client.16086 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:08:56.236 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:56.234+0000 7f41f922c740 1 -- 172.21.15.45:0/1785962404 --> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564bd462f1a0 con 0x564bd46d6280 2024-04-16T17:08:56.236 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:56.234+0000 7f41e6a00640 1 --2- 172.21.15.45:0/1785962404 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x564bd46d6280 0x564bd46d8670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:56.237 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:56.235+0000 7f41e6a00640 1 --2- 172.21.15.45:0/1785962404 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x564bd46d6280 0x564bd46d8670 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ec4d66a7e9eb6916 server_cookie=bdedf946ad5c3389 in_seq=0 out_seq=0 2024-04-16T17:08:56.237 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:56.235+0000 7f41dd600640 10 client.16086 ms_handle_connect on v2:172.21.15.104:6838/3660070592 2024-04-16T17:08:56.237 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:56.236+0000 7f41dd600640 1 -- 172.21.15.45:0/1785962404 <== mds.0 v2:172.21.15.104:6838/3660070592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564bd462f1a0 con 0x564bd46d6280 2024-04-16T17:08:56.237 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:56.236+0000 7f41dd600640 10 client.16086 handle_command_reply: tid=0 2024-04-16T17:08:56.237 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:56.236+0000 7f41f922c740 1 client.16086 shutdown 2024-04-16T17:08:56.237 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:56.237+0000 7f41f922c740 1 -- 172.21.15.45:0/1785962404 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564bd46bca80 msgr2=0x564bd46c5ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:56.237 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:56.237+0000 7f41f922c740 1 --2- 172.21.15.45:0/1785962404 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564bd46bca80 0x564bd46c5ac0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f41e801b040 tx=0x7f41e80047b0 comp rx=0 tx=0).stop 2024-04-16T17:08:56.237 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:56.237+0000 7f41f922c740 1 -- 172.21.15.45:0/1785962404 shutdown_connections 2024-04-16T17:08:56.237 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:56.237+0000 7f41f922c740 1 --2- 172.21.15.45:0/1785962404 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x564bd46d6280 0x564bd46d8670 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:56.237 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:56.237+0000 7f41f922c740 1 --2- 172.21.15.45:0/1785962404 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564bd46c6000 0x564bd46c3590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:56.237 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:56.237+0000 7f41f922c740 1 --2- 172.21.15.45:0/1785962404 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564bd46bd7f0 0x564bd46c2e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:56.237 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:56.237+0000 7f41f922c740 1 --2- 172.21.15.45:0/1785962404 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564bd46bca80 0x564bd46c5ac0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:56.237 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:56.237+0000 7f41dd600640 0 client.16086 ms_handle_reset on v2:172.21.15.104:6838/3660070592 2024-04-16T17:08:56.238 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:56.237+0000 7f41f922c740 1 -- 172.21.15.45:0/1785962404 >> 172.21.15.45:0/1785962404 conn(0x564bd46575d0 msgr2=0x564bd46b94c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:08:56.238 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:56.237+0000 7f41f922c740 1 -- 172.21.15.45:0/1785962404 shutdown_connections 2024-04-16T17:08:56.238 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:56.237+0000 7f41f922c740 1 -- 172.21.15.45:0/1785962404 wait complete. 2024-04-16T17:08:56.238 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:56.237+0000 7f41f922c740 20 client.16086 trim_cache size 0 max 16384 2024-04-16T17:08:56.254 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:56.254+0000 7f41f922c740 1 Processor -- start 2024-04-16T17:08:56.255 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:56.254+0000 7f41f922c740 1 -- start start 2024-04-16T17:08:56.255 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:56.254+0000 7f41f922c740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564bd46bca80 0x564bd46c2e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:56.255 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:56.254+0000 7f41f922c740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564bd46bd7f0 0x564bd46d1c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:56.255 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:56.254+0000 7f41f922c740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564bd46c6000 0x564bd46c5580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:56.255 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:56.254+0000 7f41f922c740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f41f01b3e20 con 0x564bd46c6000 2024-04-16T17:08:56.255 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:56.254+0000 7f41f922c740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f41f01b1690 con 0x564bd46bca80 2024-04-16T17:08:56.255 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:56.254+0000 7f41f922c740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f41f01af4c0 con 0x564bd46bd7f0 2024-04-16T17:08:56.255 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:56.255+0000 7f41efe00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564bd46bca80 0x564bd46c2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:56.255 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:56.255+0000 7f41efe00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564bd46bca80 0x564bd46c2e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:56220/0 (socket says 172.21.15.45:56220) 2024-04-16T17:08:56.255 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:56.255+0000 7f41efe00640 1 -- 172.21.15.45:0/887767259 learned_addr learned my addr 172.21.15.45:0/887767259 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:08:56.255 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:56.255+0000 7f41efe00640 1 --2- 172.21.15.45:0/887767259 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564bd46c6000 0x564bd46c5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:56.255 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:56.255+0000 7f41e6a00640 1 --2- 172.21.15.45:0/887767259 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564bd46bd7f0 0x564bd46d1c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:56.256 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:56.255+0000 7f41efe00640 1 -- 172.21.15.45:0/887767259 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564bd46bd7f0 msgr2=0x564bd46d1c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:08:56.257 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:56.255+0000 7f41efe00640 1 --2- 172.21.15.45:0/887767259 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564bd46bd7f0 0x564bd46d1c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:56.257 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:56.255+0000 7f41efe00640 1 -- 172.21.15.45:0/887767259 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564bd46c6000 msgr2=0x564bd46c5580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:08:56.257 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:56.255+0000 7f41efe00640 1 --2- 172.21.15.45:0/887767259 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564bd46c6000 0x564bd46c5580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:56.257 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:56.255+0000 7f41efe00640 1 -- 172.21.15.45:0/887767259 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564bd46d89e0 con 0x564bd46bca80 2024-04-16T17:08:56.257 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:56.255+0000 7f41efe00640 1 --2- 172.21.15.45:0/887767259 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564bd46bca80 0x564bd46c2e10 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f41e8005d50 tx=0x7f41e8014040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5125582b10ead743 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:08:56.257 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:56.256+0000 7f41dfe00640 1 -- 172.21.15.45:0/887767259 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41e802b020 con 0x564bd46bca80 2024-04-16T17:08:56.257 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:56.256+0000 7f41dfe00640 1 -- 172.21.15.45:0/887767259 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f41e80046b0 con 0x564bd46bca80 2024-04-16T17:08:56.257 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:56.256+0000 7f41dfe00640 1 -- 172.21.15.45:0/887767259 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41e8027b50 con 0x564bd46bca80 2024-04-16T17:08:56.257 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:56.256+0000 7f41f922c740 1 -- 172.21.15.45:0/887767259 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564bd46bca80 msgr2=0x564bd46c2e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:56.257 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:56.256+0000 7f41f922c740 1 --2- 172.21.15.45:0/887767259 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564bd46bca80 0x564bd46c2e10 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f41e8005d50 tx=0x7f41e8014040 comp rx=0 tx=0).stop 2024-04-16T17:08:56.257 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:56.257+0000 7f41f922c740 1 -- 172.21.15.45:0/887767259 shutdown_connections 2024-04-16T17:08:56.257 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:56.257+0000 7f41f922c740 1 --2- 172.21.15.45:0/887767259 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564bd46bd7f0 0x564bd46d1c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:56.257 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:56.257+0000 7f41f922c740 1 --2- 172.21.15.45:0/887767259 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564bd46c6000 0x564bd46c5580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:56.257 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:56.257+0000 7f41f922c740 1 --2- 172.21.15.45:0/887767259 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564bd46bca80 0x564bd46c2e10 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:56.258 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:56.257+0000 7f41f922c740 1 -- 172.21.15.45:0/887767259 >> 172.21.15.45:0/887767259 conn(0x564bd46575d0 msgr2=0x564bd46c2700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:08:56.258 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:56.257+0000 7f41f922c740 1 -- 172.21.15.45:0/887767259 shutdown_connections 2024-04-16T17:08:56.258 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:56.257+0000 7f41f922c740 1 -- 172.21.15.45:0/887767259 wait complete. 2024-04-16T17:08:56.260 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:56.257+0000 7f41f922c740 1 -- start start 2024-04-16T17:08:56.260 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:56.258+0000 7f41f922c740 1 Processor -- start 2024-04-16T17:08:56.261 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:56.258+0000 7f41f922c740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564bd46bca80 0x564bd46ce400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:56.261 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:56.258+0000 7f41f922c740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564bd46bd7f0 0x564bd46bf800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:56.261 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:56.258+0000 7f41f922c740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564bd46c6000 0x564bd46bfd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:56.261 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:56.258+0000 7f41f922c740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f41f01afb50 con 0x564bd46c6000 2024-04-16T17:08:56.261 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:56.258+0000 7f41f922c740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f41f01b0fc0 con 0x564bd46bd7f0 2024-04-16T17:08:56.261 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:56.258+0000 7f41f922c740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f41f01b0220 con 0x564bd46bca80 2024-04-16T17:08:56.261 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:56.258+0000 7f41f922c740 10 client.0 _ll_register_callbacks cb 0x7f41d8009530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:08:56.261 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:56.258+0000 7f41efe00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564bd46bca80 0x564bd46ce400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:56.261 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:56.259+0000 7f41efe00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564bd46bca80 0x564bd46ce400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:34404/0 (socket says 172.21.15.45:34404) 2024-04-16T17:08:56.261 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:56.259+0000 7f41efe00640 1 -- 172.21.15.45:0/2097810740 learned_addr learned my addr 172.21.15.45:0/2097810740 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:08:56.261 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:56.259+0000 7f41efe00640 1 --2- 172.21.15.45:0/2097810740 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564bd46c6000 0x564bd46bfd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:56.261 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:56.259+0000 7f41e6a00640 1 --2- 172.21.15.45:0/2097810740 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564bd46bd7f0 0x564bd46bf800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:56.261 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:56.259+0000 7f41efe00640 1 -- 172.21.15.45:0/2097810740 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564bd46bd7f0 msgr2=0x564bd46bf800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:08:56.262 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:56.259+0000 7f41efe00640 1 --2- 172.21.15.45:0/2097810740 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564bd46bd7f0 0x564bd46bf800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:56.262 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:56.259+0000 7f41efe00640 1 -- 172.21.15.45:0/2097810740 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564bd46c6000 msgr2=0x564bd46bfd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:08:56.262 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:56.259+0000 7f41efe00640 1 --2- 172.21.15.45:0/2097810740 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564bd46c6000 0x564bd46bfd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:56.262 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:56.259+0000 7f41efe00640 1 -- 172.21.15.45:0/2097810740 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564bd46c0480 con 0x564bd46bca80 2024-04-16T17:08:56.262 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:56.259+0000 7f41efe00640 1 --2- 172.21.15.45:0/2097810740 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564bd46bca80 0x564bd46ce400 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f41e8009fd0 tx=0x7f41e8004640 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1fcc98e0635f74aa server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:08:56.262 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:56.259+0000 7f41dd600640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-04-16T17:08:56.262 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:56.260+0000 7f41dd600640 1 -- 172.21.15.45:0/2097810740 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41e802b030 con 0x564bd46bca80 2024-04-16T17:08:56.262 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:56.260+0000 7f41dd600640 1 -- 172.21.15.45:0/2097810740 <== mon.1 v2:172.21.15.117:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f41e8027af0 con 0x564bd46bca80 2024-04-16T17:08:56.262 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:56.260+0000 7f41f922c740 1 -- 172.21.15.45:0/2097810740 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f41e80046b0 con 0x564bd46bca80 2024-04-16T17:08:56.262 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:56.260+0000 7f41dd600640 1 -- 172.21.15.45:0/2097810740 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f41e8004860 con 0x564bd46bca80 2024-04-16T17:08:56.262 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:56.260+0000 7f41f922c740 10 client.26221 fetch_fsmap learned FSMap version 28 2024-04-16T17:08:56.262 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:56.260+0000 7f41f922c740 1 -- 172.21.15.45:0/2097810740 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x564bd473c680 con 0x564bd46bca80 2024-04-16T17:08:56.262 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:56.260+0000 7f41dd600640 1 -- 172.21.15.45:0/2097810740 <== mon.1 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f41e80046b0 con 0x564bd46bca80 2024-04-16T17:08:56.262 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:56.260+0000 7f41dd600640 1 -- 172.21.15.45:0/2097810740 <== mon.1 v2:172.21.15.117:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f41e803b070 con 0x564bd46bca80 2024-04-16T17:08:56.262 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:56.260+0000 7f41f922c740 10 client.26221 fetch_fsmap finished waiting for FSMap version 28 2024-04-16T17:08:56.263 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:56.260+0000 7f41f922c740 10 client.26221 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T17:08:56.263 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:56.260+0000 7f41f922c740 20 client.26221 populate_metadata read hostname 'smithi045' 2024-04-16T17:08:56.263 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:56.260+0000 7f41f922c740 1 --2- 172.21.15.45:0/2097810740 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x564bd473cc30 0x564bd473f020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:08:56.263 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:56.260+0000 7f41f922c740 4 client.26221 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-16T17:08:56.263 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:56.260+0000 7f41f922c740 1 -- 172.21.15.45:0/2097810740 --> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564bd4629010 con 0x564bd473cc30 2024-04-16T17:08:56.263 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:56.261+0000 7f41e6a00640 1 --2- 172.21.15.45:0/2097810740 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x564bd473cc30 0x564bd473f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:08:56.263 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:56.261+0000 7f41e6a00640 1 --2- 172.21.15.45:0/2097810740 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x564bd473cc30 0x564bd473f020 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ee512d0fed89fd37 server_cookie=8656ba3c3660deaf in_seq=0 out_seq=0 2024-04-16T17:08:56.263 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:56.262+0000 7f41dd600640 10 client.26221 ms_handle_connect on v2:172.21.15.104:6838/3660070592 2024-04-16T17:08:56.299 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:56.299+0000 7f41dd600640 1 -- 172.21.15.45:0/2097810740 <== mds.0 v2:172.21.15.104:6838/3660070592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x564bd4629010 con 0x564bd473cc30 2024-04-16T17:08:56.299 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:56.299+0000 7f41dd600640 10 client.26221 handle_command_reply: tid=0 2024-04-16T17:08:56.299 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:56.299+0000 7f41f922c740 1 client.26221 shutdown 2024-04-16T17:08:56.300 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:56.299+0000 7f41f922c740 1 -- 172.21.15.45:0/2097810740 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564bd46bca80 msgr2=0x564bd46ce400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:56.300 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:56.299+0000 7f41f922c740 1 --2- 172.21.15.45:0/2097810740 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564bd46bca80 0x564bd46ce400 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f41e8009fd0 tx=0x7f41e8004640 comp rx=0 tx=0).stop 2024-04-16T17:08:56.300 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:56.299+0000 7f41f922c740 1 -- 172.21.15.45:0/2097810740 shutdown_connections 2024-04-16T17:08:56.300 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:56.299+0000 7f41f922c740 1 --2- 172.21.15.45:0/2097810740 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564bd46c6000 0x564bd46bfd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:56.300 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:56.299+0000 7f41f922c740 1 --2- 172.21.15.45:0/2097810740 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564bd46bd7f0 0x564bd46bf800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:56.300 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:56.299+0000 7f41f922c740 1 --2- 172.21.15.45:0/2097810740 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x564bd473cc30 0x564bd473f020 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:56.300 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:56.299+0000 7f41f922c740 1 --2- 172.21.15.45:0/2097810740 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564bd46bca80 0x564bd46ce400 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:56.300 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:56.299+0000 7f41dd600640 0 client.26221 ms_handle_reset on v2:172.21.15.104:6838/3660070592 2024-04-16T17:08:56.300 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:56.299+0000 7f41f922c740 1 -- 172.21.15.45:0/2097810740 >> 172.21.15.45:0/2097810740 conn(0x564bd46575d0 msgr2=0x564bd46b5f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:08:56.300 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:56.299+0000 7f41f922c740 1 -- 172.21.15.45:0/2097810740 shutdown_connections 2024-04-16T17:08:56.300 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:56.299+0000 7f41f922c740 1 -- 172.21.15.45:0/2097810740 wait complete. 2024-04-16T17:08:56.300 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:56.300+0000 7f41f922c740 20 client.26221 trim_cache size 0 max 16384 2024-04-16T17:08:56.301 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-04-16T17:08:56.301 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-04-16T17:08:56.301 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-04-16T17:08:56.301 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-04-16T17:08:56.301 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:56.300+0000 7f41f4e00640 1 -- 172.21.15.45:0/4183826823 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f41d40a3e10 msgr2=0x7f41d40a62d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:56.302 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:56.300+0000 7f41f4e00640 1 --2- 172.21.15.45:0/4183826823 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f41d40a3e10 0x7f41d40a62d0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x564bd46bdf50 tx=0x7f41e00035f0 comp rx=0 tx=0).stop 2024-04-16T17:08:56.302 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:56.300+0000 7f41f4e00640 1 -- 172.21.15.45:0/4183826823 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f41f0102080 msgr2=0x7f41f01a0670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:08:56.302 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:56.300+0000 7f41f4e00640 1 --2- 172.21.15.45:0/4183826823 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f41f0102080 0x7f41f01a0670 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f41e002c170 tx=0x7f41e0004360 comp rx=0 tx=0).stop 2024-04-16T17:08:56.302 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:56.300+0000 7f41f4e00640 1 -- 172.21.15.45:0/4183826823 shutdown_connections 2024-04-16T17:08:56.302 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:56.300+0000 7f41f4e00640 1 --2- 172.21.15.45:0/4183826823 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f41d40a3e10 0x7f41d40a62d0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:56.302 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:56.300+0000 7f41f4e00640 1 --2- 172.21.15.45:0/4183826823 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f41f019a730 0x7f41f019abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:56.302 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:56.300+0000 7f41f4e00640 1 --2- 172.21.15.45:0/4183826823 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f41f0107150 0x7f41f01a0bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:56.302 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:56.300+0000 7f41f4e00640 1 --2- 172.21.15.45:0/4183826823 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f41f0102080 0x7f41f01a0670 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:08:56.302 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:56.300+0000 7f41f4e00640 1 -- 172.21.15.45:0/4183826823 >> 172.21.15.45:0/4183826823 conn(0x7f41f00fd980 msgr2=0x7f41f00ffc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:08:56.302 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:56.301+0000 7f41f4e00640 1 -- 172.21.15.45:0/4183826823 shutdown_connections 2024-04-16T17:08:56.303 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:08:56.301+0000 7f41f4e00640 1 -- 172.21.15.45:0/4183826823 wait complete. 2024-04-16T17:08:56.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:08:56 smithi117 ceph-mon[36730]: pgmap v68: 129 pgs: 129 active+clean; 37 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 586 KiB/s rd, 1.6 MiB/s wr, 133 op/s 2024-04-16T17:08:56.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:08:56 smithi104 ceph-mon[36457]: pgmap v68: 129 pgs: 129 active+clean; 37 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 586 KiB/s rd, 1.6 MiB/s wr, 133 op/s 2024-04-16T17:08:56.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:08:56 smithi045 ceph-mon[31676]: pgmap v68: 129 pgs: 129 active+clean; 37 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 586 KiB/s rd, 1.6 MiB/s wr, 133 op/s 2024-04-16T17:08:57.221 INFO:tasks.workunit.client.0.smithi045.stdout: 1 32687 3.28 MB/sec warmup 59 sec latency 1375.100 ms 2024-04-16T17:08:57.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:08:57 smithi117 ceph-mon[36730]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:08:57.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:08:57 smithi117 ceph-mon[36730]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T17:08:57.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:08:57 smithi117 ceph-mon[36730]: from='mgr.25411 ' entity='mgr.x' 2024-04-16T17:08:57.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:08:57 smithi104 ceph-mon[36457]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:08:57.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:08:57 smithi104 ceph-mon[36457]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T17:08:57.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:08:57 smithi104 ceph-mon[36457]: from='mgr.25411 ' entity='mgr.x' 2024-04-16T17:08:57.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:08:57 smithi045 ceph-mon[31676]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:08:57.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:08:57 smithi045 ceph-mon[31676]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T17:08:57.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:08:57 smithi045 ceph-mon[31676]: from='mgr.25411 ' entity='mgr.x' 2024-04-16T17:08:58.221 INFO:tasks.workunit.client.0.smithi045.stdout: 1 32687 3.22 MB/sec warmup 60 sec latency 2375.221 ms 2024-04-16T17:08:58.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:08:58 smithi117 ceph-mon[36730]: pgmap v69: 129 pgs: 129 active+clean; 29 MiB data, 457 MiB used, 1.0 TiB / 1.0 TiB avail; 633 KiB/s rd, 1.6 MiB/s wr, 155 op/s 2024-04-16T17:08:58.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:08:58 smithi104 ceph-mon[36457]: pgmap v69: 129 pgs: 129 active+clean; 29 MiB data, 457 MiB used, 1.0 TiB / 1.0 TiB avail; 633 KiB/s rd, 1.6 MiB/s wr, 155 op/s 2024-04-16T17:08:58.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:08:58 smithi045 ceph-mon[31676]: pgmap v69: 129 pgs: 129 active+clean; 29 MiB data, 457 MiB used, 1.0 TiB / 1.0 TiB avail; 633 KiB/s rd, 1.6 MiB/s wr, 155 op/s 2024-04-16T17:08:59.221 INFO:tasks.workunit.client.0.smithi045.stdout: 1 32687 3.17 MB/sec warmup 61 sec latency 3375.334 ms 2024-04-16T17:09:00.221 INFO:tasks.workunit.client.0.smithi045.stdout: 1 32745 3.12 MB/sec warmup 62 sec latency 4219.325 ms 2024-04-16T17:09:00.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:09:00 smithi104 ceph-mon[36457]: pgmap v70: 129 pgs: 129 active+clean; 29 MiB data, 457 MiB used, 1.0 TiB / 1.0 TiB avail; 461 KiB/s rd, 849 KiB/s wr, 91 op/s 2024-04-16T17:09:00.937 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:09:00 smithi045 ceph-mon[31676]: pgmap v70: 129 pgs: 129 active+clean; 29 MiB data, 457 MiB used, 1.0 TiB / 1.0 TiB avail; 461 KiB/s rd, 849 KiB/s wr, 91 op/s 2024-04-16T17:09:01.055 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:09:00 smithi117 ceph-mon[36730]: pgmap v70: 129 pgs: 129 active+clean; 29 MiB data, 457 MiB used, 1.0 TiB / 1.0 TiB avail; 461 KiB/s rd, 849 KiB/s wr, 91 op/s 2024-04-16T17:09:01.221 INFO:tasks.workunit.client.0.smithi045.stdout: 1 32981 3.08 MB/sec warmup 63 sec latency 96.817 ms 2024-04-16T17:09:02.222 INFO:tasks.workunit.client.0.smithi045.stdout: 1 33185 3.05 MB/sec warmup 64 sec latency 54.345 ms 2024-04-16T17:09:02.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:09:02 smithi104 ceph-mon[36457]: pgmap v71: 129 pgs: 129 active+clean; 26 MiB data, 453 MiB used, 1.0 TiB / 1.0 TiB avail; 532 KiB/s rd, 951 KiB/s wr, 133 op/s 2024-04-16T17:09:03.055 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:09:02 smithi117 ceph-mon[36730]: pgmap v71: 129 pgs: 129 active+clean; 26 MiB data, 453 MiB used, 1.0 TiB / 1.0 TiB avail; 532 KiB/s rd, 951 KiB/s wr, 133 op/s 2024-04-16T17:09:03.223 INFO:tasks.workunit.client.0.smithi045.stdout: 1 33489 3.05 MB/sec warmup 65 sec latency 68.289 ms 2024-04-16T17:09:04.224 INFO:tasks.workunit.client.0.smithi045.stdout: 1 33999 3.01 MB/sec warmup 66 sec latency 30.149 ms 2024-04-16T17:09:04.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:09:04 smithi104 ceph-mon[36457]: pgmap v72: 129 pgs: 129 active+clean; 26 MiB data, 453 MiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 129 KiB/s wr, 63 op/s 2024-04-16T17:09:05.055 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:09:04 smithi117 ceph-mon[36730]: pgmap v72: 129 pgs: 129 active+clean; 26 MiB data, 453 MiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 129 KiB/s wr, 63 op/s 2024-04-16T17:09:05.224 INFO:tasks.workunit.client.0.smithi045.stdout: 1 35232 3.10 MB/sec warmup 67 sec latency 27.417 ms 2024-04-16T17:09:05.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:09:05 smithi104 ceph-mon[36457]: pgmap v73: 129 pgs: 129 active+clean; 26 MiB data, 453 MiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 129 KiB/s wr, 63 op/s 2024-04-16T17:09:06.055 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:09:05 smithi117 ceph-mon[36730]: pgmap v73: 129 pgs: 129 active+clean; 26 MiB data, 453 MiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 129 KiB/s wr, 63 op/s 2024-04-16T17:09:06.224 INFO:tasks.workunit.client.0.smithi045.stdout: 1 35601 3.10 MB/sec warmup 68 sec latency 46.003 ms 2024-04-16T17:09:07.226 INFO:tasks.workunit.client.0.smithi045.stdout: 1 35836 3.07 MB/sec warmup 69 sec latency 64.305 ms 2024-04-16T17:09:07.398 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:09:02 smithi045 ceph-mon[31676]: pgmap v71: 129 pgs: 129 active+clean; 26 MiB data, 453 MiB used, 1.0 TiB / 1.0 TiB avail; 532 KiB/s rd, 951 KiB/s wr, 133 op/s 2024-04-16T17:09:07.398 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:09:04 smithi045 ceph-mon[31676]: pgmap v72: 129 pgs: 129 active+clean; 26 MiB data, 453 MiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 129 KiB/s wr, 63 op/s 2024-04-16T17:09:07.398 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:09:05 smithi045 ceph-mon[31676]: pgmap v73: 129 pgs: 129 active+clean; 26 MiB data, 453 MiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 129 KiB/s wr, 63 op/s 2024-04-16T17:09:07.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:09:07 smithi104 ceph-mon[36457]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-16T17:09:07.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:09:07 smithi104 ceph-mon[36457]: pgmap v74: 129 pgs: 129 active+clean; 27 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 483 KiB/s rd, 305 KiB/s wr, 100 op/s 2024-04-16T17:09:07.988 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-16T17:09:07.989 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-16T17:09:08.055 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:09:07 smithi117 ceph-mon[36730]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-16T17:09:08.055 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:09:07 smithi117 ceph-mon[36730]: pgmap v74: 129 pgs: 129 active+clean; 27 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 483 KiB/s rd, 305 KiB/s wr, 100 op/s 2024-04-16T17:09:08.184 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.184+0000 7f322fe00640 1 -- 172.21.15.45:0/3802293241 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3230074cc0 msgr2=0x7f3230112970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:08.184 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.184+0000 7f322fe00640 1 --2- 172.21.15.45:0/3802293241 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3230074cc0 0x7f3230112970 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f321c009900 tx=0x7f321c02bcd0 comp rx=0 tx=0).stop 2024-04-16T17:09:08.184 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.184+0000 7f322fe00640 1 -- 172.21.15.45:0/3802293241 shutdown_connections 2024-04-16T17:09:08.184 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.184+0000 7f322fe00640 1 --2- 172.21.15.45:0/3802293241 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3230074cc0 0x7f3230112970 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:08.184 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.184+0000 7f322fe00640 1 --2- 172.21.15.45:0/3802293241 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3230112eb0 0x7f32301152a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:08.184 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.184+0000 7f322fe00640 1 --2- 172.21.15.45:0/3802293241 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f32300743a0 0x7f3230074780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:08.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.184+0000 7f322fe00640 1 -- 172.21.15.45:0/3802293241 >> 172.21.15.45:0/3802293241 conn(0x7f32300fd9d0 msgr2=0x7f32300ffdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:09:08.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.184+0000 7f322fe00640 1 -- 172.21.15.45:0/3802293241 shutdown_connections 2024-04-16T17:09:08.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.184+0000 7f322fe00640 1 -- 172.21.15.45:0/3802293241 wait complete. 2024-04-16T17:09:08.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.185+0000 7f322fe00640 1 Processor -- start 2024-04-16T17:09:08.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.185+0000 7f322fe00640 1 -- start start 2024-04-16T17:09:08.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.186+0000 7f322fe00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f32300743a0 0x7f323019ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:08.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.186+0000 7f322fe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3230074cc0 0x7f323019f340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:08.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.186+0000 7f322fe00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3230112eb0 0x7f32301a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:08.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.186+0000 7f322fe00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f3230118880 con 0x7f3230074cc0 2024-04-16T17:09:08.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.186+0000 7f322fe00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f3230118700 con 0x7f32300743a0 2024-04-16T17:09:08.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.186+0000 7f322fe00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f3230118a00 con 0x7f3230112eb0 2024-04-16T17:09:08.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.186+0000 7f322ea00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f32300743a0 0x7f323019ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:08.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.186+0000 7f322ea00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f32300743a0 0x7f323019ee00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:47648/0 (socket says 172.21.15.45:47648) 2024-04-16T17:09:08.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.186+0000 7f322ea00640 1 -- 172.21.15.45:0/384369854 learned_addr learned my addr 172.21.15.45:0/384369854 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:09:08.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.186+0000 7f322f400640 1 --2- 172.21.15.45:0/384369854 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3230112eb0 0x7f32301a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:08.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.186+0000 7f322e000640 1 --2- 172.21.15.45:0/384369854 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3230074cc0 0x7f323019f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:08.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.186+0000 7f322ea00640 1 -- 172.21.15.45:0/384369854 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3230112eb0 msgr2=0x7f32301a3670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:08.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.186+0000 7f322ea00640 1 --2- 172.21.15.45:0/384369854 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3230112eb0 0x7f32301a3670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:08.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.186+0000 7f322ea00640 1 -- 172.21.15.45:0/384369854 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3230074cc0 msgr2=0x7f323019f340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:08.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.186+0000 7f322ea00640 1 --2- 172.21.15.45:0/384369854 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3230074cc0 0x7f323019f340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:08.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.186+0000 7f322ea00640 1 -- 172.21.15.45:0/384369854 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32301a3da0 con 0x7f32300743a0 2024-04-16T17:09:08.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.187+0000 7f322ea00640 1 --2- 172.21.15.45:0/384369854 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f32300743a0 0x7f323019ee00 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f322000ea60 tx=0x7f322000ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:09:08.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.187+0000 7f3227400640 1 -- 172.21.15.45:0/384369854 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f322000cde0 con 0x7f32300743a0 2024-04-16T17:09:08.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.187+0000 7f322fe00640 1 -- 172.21.15.45:0/384369854 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32301a4090 con 0x7f32300743a0 2024-04-16T17:09:08.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.187+0000 7f3227400640 1 -- 172.21.15.45:0/384369854 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f32200035f0 con 0x7f32300743a0 2024-04-16T17:09:08.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.187+0000 7f3227400640 1 -- 172.21.15.45:0/384369854 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3220010670 con 0x7f32300743a0 2024-04-16T17:09:08.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.187+0000 7f322fe00640 1 -- 172.21.15.45:0/384369854 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32301ab920 con 0x7f32300743a0 2024-04-16T17:09:08.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.191+0000 7f323877d740 1 Processor -- start 2024-04-16T17:09:08.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.191+0000 7f3227400640 1 -- 172.21.15.45:0/384369854 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f32200037a0 con 0x7f32300743a0 2024-04-16T17:09:08.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.191+0000 7f323877d740 1 -- start start 2024-04-16T17:09:08.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.191+0000 7f323877d740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fe724ae390 0x55fe724ae770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:08.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.191+0000 7f323877d740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fe724aecb0 0x55fe724b7350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:08.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.192+0000 7f323877d740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fe724b7890 0x55fe724b9c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:08.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.192+0000 7f323877d740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55fe72414120 con 0x55fe724ae390 2024-04-16T17:09:08.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.192+0000 7f323877d740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55fe723ba340 con 0x55fe724b7890 2024-04-16T17:09:08.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.192+0000 7f323877d740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55fe724100e0 con 0x55fe724aecb0 2024-04-16T17:09:08.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.192+0000 7f322f400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fe724ae390 0x55fe724ae770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:08.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.192+0000 7f322e000640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fe724b7890 0x55fe724b9c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:08.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.192+0000 7f322f400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fe724ae390 0x55fe724ae770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45066/0 (socket says 172.21.15.45:45066) 2024-04-16T17:09:08.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.192+0000 7f322f400640 1 -- 172.21.15.45:0/1413944666 learned_addr learned my addr 172.21.15.45:0/1413944666 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:09:08.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.192+0000 7f322ea00640 1 --2- 172.21.15.45:0/1413944666 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fe724aecb0 0x55fe724b7350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:08.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.193+0000 7f322e000640 1 -- 172.21.15.45:0/1413944666 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fe724aecb0 msgr2=0x55fe724b7350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:09:08.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.193+0000 7f322e000640 1 --2- 172.21.15.45:0/1413944666 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fe724aecb0 0x55fe724b7350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:08.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.193+0000 7f322e000640 1 -- 172.21.15.45:0/1413944666 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fe724ae390 msgr2=0x55fe724ae770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:09:08.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.193+0000 7f322e000640 1 --2- 172.21.15.45:0/1413944666 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fe724ae390 0x55fe724ae770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:08.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.193+0000 7f322e000640 1 -- 172.21.15.45:0/1413944666 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fe721f07d0 con 0x55fe724b7890 2024-04-16T17:09:08.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.193+0000 7f322e000640 1 --2- 172.21.15.45:0/1413944666 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fe724b7890 0x55fe724b9c80 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f321c000c00 tx=0x7f321c009560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e474fd7ae78ab162 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:09:08.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.193+0000 7f3227400640 1 --2- 172.21.15.45:0/384369854 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f32140a3d20 0x7f32140a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:08.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.193+0000 7f321be00640 1 -- 172.21.15.45:0/1413944666 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f321c042070 con 0x55fe724b7890 2024-04-16T17:09:08.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.194+0000 7f321be00640 1 -- 172.21.15.45:0/1413944666 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f321c004040 con 0x55fe724b7890 2024-04-16T17:09:08.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.194+0000 7f3227400640 1 -- 172.21.15.45:0/384369854 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f32200d0e70 con 0x7f32300743a0 2024-04-16T17:09:08.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.194+0000 7f321be00640 1 -- 172.21.15.45:0/1413944666 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f321c004c90 con 0x55fe724b7890 2024-04-16T17:09:08.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.194+0000 7f322f400640 1 --2- 172.21.15.45:0/384369854 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f32140a3d20 0x7f32140a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:08.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.194+0000 7f323877d740 1 -- 172.21.15.45:0/1413944666 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fe724b7890 msgr2=0x55fe724b9c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:08.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.194+0000 7f323877d740 1 --2- 172.21.15.45:0/1413944666 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fe724b7890 0x55fe724b9c80 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f321c000c00 tx=0x7f321c009560 comp rx=0 tx=0).stop 2024-04-16T17:09:08.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.194+0000 7f323877d740 1 -- 172.21.15.45:0/1413944666 shutdown_connections 2024-04-16T17:09:08.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.194+0000 7f323877d740 1 --2- 172.21.15.45:0/1413944666 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fe724aecb0 0x55fe724b7350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:08.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.194+0000 7f323877d740 1 --2- 172.21.15.45:0/1413944666 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fe724b7890 0x55fe724b9c80 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:08.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.194+0000 7f323877d740 1 --2- 172.21.15.45:0/1413944666 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fe724ae390 0x55fe724ae770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:08.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.194+0000 7f322f400640 1 --2- 172.21.15.45:0/384369854 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f32140a3d20 0x7f32140a61e0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f3228011630 tx=0x7f3228009500 comp rx=0 tx=0).ready entity=mgr.25411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:09:08.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.194+0000 7f323877d740 1 -- 172.21.15.45:0/1413944666 >> 172.21.15.45:0/1413944666 conn(0x55fe724a8ac0 msgr2=0x55fe724a8ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:09:08.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.195+0000 7f323877d740 1 -- 172.21.15.45:0/1413944666 shutdown_connections 2024-04-16T17:09:08.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.195+0000 7f323877d740 1 -- 172.21.15.45:0/1413944666 wait complete. 2024-04-16T17:09:08.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.195+0000 7f323877d740 1 -- start start 2024-04-16T17:09:08.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.195+0000 7f323877d740 1 Processor -- start 2024-04-16T17:09:08.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.196+0000 7f323877d740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fe724ae390 0x55fe724b4930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:08.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.196+0000 7f323877d740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fe724aecb0 0x55fe724b4e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:08.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.196+0000 7f323877d740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fe724b7890 0x55fe724b53b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:08.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.196+0000 7f323877d740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55fe724ba620 con 0x55fe724b7890 2024-04-16T17:09:08.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.196+0000 7f323877d740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55fe723cd710 con 0x55fe724aecb0 2024-04-16T17:09:08.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.196+0000 7f323877d740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55fe724ba7a0 con 0x55fe724ae390 2024-04-16T17:09:08.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.196+0000 7f323877d740 10 client.0 _ll_register_callbacks cb 0x55fe723c8af0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:09:08.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.196+0000 7f322f400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fe724b7890 0x55fe724b53b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:08.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.196+0000 7f322e000640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fe724aecb0 0x55fe724b4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:08.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.196+0000 7f322e000640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fe724aecb0 0x55fe724b4e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:47654/0 (socket says 172.21.15.45:47654) 2024-04-16T17:09:08.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.196+0000 7f322e000640 1 -- 172.21.15.45:0/779161674 learned_addr learned my addr 172.21.15.45:0/779161674 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:09:08.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.196+0000 7f322ea00640 1 --2- 172.21.15.45:0/779161674 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fe724ae390 0x55fe724b4930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:08.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.196+0000 7f322e000640 1 -- 172.21.15.45:0/779161674 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fe724ae390 msgr2=0x55fe724b4930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:09:08.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.196+0000 7f322e000640 1 --2- 172.21.15.45:0/779161674 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fe724ae390 0x55fe724b4930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:08.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.196+0000 7f322e000640 1 -- 172.21.15.45:0/779161674 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fe724b7890 msgr2=0x55fe724b53b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:09:08.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.197+0000 7f322e000640 1 --2- 172.21.15.45:0/779161674 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fe724b7890 0x55fe724b53b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:08.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.197+0000 7f322e000640 1 -- 172.21.15.45:0/779161674 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fe721263f0 con 0x55fe724aecb0 2024-04-16T17:09:08.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.197+0000 7f322e000640 1 --2- 172.21.15.45:0/779161674 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fe724aecb0 0x55fe724b4e70 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f321c009900 tx=0x7f321c0044a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=be4adf66d1f91928 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:09:08.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.197+0000 7f3218c00640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-04-16T17:09:08.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.197+0000 7f3218c00640 1 -- 172.21.15.45:0/779161674 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f321c042070 con 0x55fe724aecb0 2024-04-16T17:09:08.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.197+0000 7f3218c00640 1 -- 172.21.15.45:0/779161674 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f321c02cb30 con 0x55fe724aecb0 2024-04-16T17:09:08.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.197+0000 7f323877d740 1 -- 172.21.15.45:0/779161674 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f321c004040 con 0x55fe724aecb0 2024-04-16T17:09:08.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.197+0000 7f3218c00640 1 -- 172.21.15.45:0/779161674 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f321c0046c0 con 0x55fe724aecb0 2024-04-16T17:09:08.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.197+0000 7f323877d740 10 client.26252 fetch_fsmap learned FSMap version 29 2024-04-16T17:09:08.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.197+0000 7f323877d740 1 -- 172.21.15.45:0/779161674 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55fe720e60f0 con 0x55fe724aecb0 2024-04-16T17:09:08.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.198+0000 7f3218c00640 1 -- 172.21.15.45:0/779161674 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f321c004040 con 0x55fe724aecb0 2024-04-16T17:09:08.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.198+0000 7f3218c00640 1 -- 172.21.15.45:0/779161674 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 29) ==== 8366+0+0 (secure 0 0 0) 0x7f321c04f030 con 0x55fe724aecb0 2024-04-16T17:09:08.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.198+0000 7f323877d740 10 client.26252 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T17:09:08.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.198+0000 7f323877d740 10 client.26252 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:09:08.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.198+0000 7f323877d740 20 client.26252 populate_metadata read hostname 'smithi045' 2024-04-16T17:09:08.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.198+0000 7f323877d740 1 --2- 172.21.15.45:0/779161674 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55fe724c7aa0 0x55fe724c9e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:08.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.198+0000 7f323877d740 4 client.26252 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:09:08.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.198+0000 7f323877d740 1 -- 172.21.15.45:0/779161674 --> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fe72421b40 con 0x55fe724c7aa0 2024-04-16T17:09:08.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.198+0000 7f322ea00640 1 --2- 172.21.15.45:0/779161674 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55fe724c7aa0 0x55fe724c9e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:08.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.199+0000 7f322ea00640 1 --2- 172.21.15.45:0/779161674 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55fe724c7aa0 0x55fe724c9e90 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1c47043983b103c7 server_cookie=7d8cdae6debd44cd in_seq=0 out_seq=0 2024-04-16T17:09:08.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.199+0000 7f3218c00640 10 client.26252 ms_handle_connect on v2:172.21.15.104:6838/3660070592 2024-04-16T17:09:08.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.200+0000 7f3218c00640 1 -- 172.21.15.45:0/779161674 <== mds.0 v2:172.21.15.104:6838/3660070592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55fe72421b40 con 0x55fe724c7aa0 2024-04-16T17:09:08.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.200+0000 7f3218c00640 10 client.26252 handle_command_reply: tid=0 2024-04-16T17:09:08.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.200+0000 7f323877d740 1 client.26252 shutdown 2024-04-16T17:09:08.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.200+0000 7f323877d740 1 -- 172.21.15.45:0/779161674 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fe724aecb0 msgr2=0x55fe724b4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:08.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.200+0000 7f323877d740 1 --2- 172.21.15.45:0/779161674 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fe724aecb0 0x55fe724b4e70 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f321c009900 tx=0x7f321c0044a0 comp rx=0 tx=0).stop 2024-04-16T17:09:08.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.200+0000 7f323877d740 1 -- 172.21.15.45:0/779161674 shutdown_connections 2024-04-16T17:09:08.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.200+0000 7f323877d740 1 --2- 172.21.15.45:0/779161674 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fe724b7890 0x55fe724b53b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:08.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.200+0000 7f323877d740 1 --2- 172.21.15.45:0/779161674 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fe724aecb0 0x55fe724b4e70 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:08.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.200+0000 7f323877d740 1 --2- 172.21.15.45:0/779161674 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55fe724c7aa0 0x55fe724c9e90 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:08.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.200+0000 7f323877d740 1 --2- 172.21.15.45:0/779161674 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fe724ae390 0x55fe724b4930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:08.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.200+0000 7f3218c00640 0 client.26252 ms_handle_reset on v2:172.21.15.104:6838/3660070592 2024-04-16T17:09:08.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.200+0000 7f323877d740 1 -- 172.21.15.45:0/779161674 >> 172.21.15.45:0/779161674 conn(0x55fe724a8ac0 msgr2=0x55fe724aa810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:09:08.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.200+0000 7f323877d740 1 -- 172.21.15.45:0/779161674 shutdown_connections 2024-04-16T17:09:08.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.200+0000 7f323877d740 1 -- 172.21.15.45:0/779161674 wait complete. 2024-04-16T17:09:08.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.200+0000 7f323877d740 20 client.26252 trim_cache size 0 max 16384 2024-04-16T17:09:08.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.220+0000 7f323877d740 1 Processor -- start 2024-04-16T17:09:08.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.220+0000 7f323877d740 1 -- start start 2024-04-16T17:09:08.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.220+0000 7f323877d740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fe724ae390 0x55fe724b7060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:08.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.220+0000 7f323877d740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fe724aecb0 0x55fe724b3870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:08.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.220+0000 7f323877d740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fe724b7890 0x55fe724c34e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:08.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.220+0000 7f323877d740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f32301b81a0 con 0x55fe724aecb0 2024-04-16T17:09:08.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.220+0000 7f323877d740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f32301b59e0 con 0x55fe724b7890 2024-04-16T17:09:08.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.220+0000 7f323877d740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f32301b3810 con 0x55fe724ae390 2024-04-16T17:09:08.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.220+0000 7f322e000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fe724aecb0 0x55fe724b3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:08.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.220+0000 7f322e000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fe724aecb0 0x55fe724b3870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45084/0 (socket says 172.21.15.45:45084) 2024-04-16T17:09:08.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.220+0000 7f322e000640 1 -- 172.21.15.45:0/2986530453 learned_addr learned my addr 172.21.15.45:0/2986530453 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:09:08.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.220+0000 7f322f400640 1 --2- 172.21.15.45:0/2986530453 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fe724b7890 0x55fe724c34e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:08.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.220+0000 7f322ea00640 1 --2- 172.21.15.45:0/2986530453 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fe724ae390 0x55fe724b7060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:08.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.221+0000 7f322ea00640 1 -- 172.21.15.45:0/2986530453 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fe724b7890 msgr2=0x55fe724c34e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:09:08.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.221+0000 7f322ea00640 1 --2- 172.21.15.45:0/2986530453 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fe724b7890 0x55fe724c34e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:08.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.221+0000 7f322ea00640 1 -- 172.21.15.45:0/2986530453 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fe724aecb0 msgr2=0x55fe724b3870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:09:08.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.221+0000 7f322ea00640 1 --2- 172.21.15.45:0/2986530453 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fe724aecb0 0x55fe724b3870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:08.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.221+0000 7f322ea00640 1 -- 172.21.15.45:0/2986530453 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fe724a85f0 con 0x55fe724ae390 2024-04-16T17:09:08.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.221+0000 7f322e000640 1 --2- 172.21.15.45:0/2986530453 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fe724aecb0 0x55fe724b3870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:09:08.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.221+0000 7f322ea00640 1 --2- 172.21.15.45:0/2986530453 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fe724ae390 0x55fe724b7060 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f32200ce210 tx=0x7f32200d32c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=88a1bde1c2ed9f60 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:09:08.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.221+0000 7f321b400640 1 -- 172.21.15.45:0/2986530453 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f322009c5a0 con 0x55fe724ae390 2024-04-16T17:09:08.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.221+0000 7f321b400640 1 -- 172.21.15.45:0/2986530453 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f32200bd0b0 con 0x55fe724ae390 2024-04-16T17:09:08.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.221+0000 7f321b400640 1 -- 172.21.15.45:0/2986530453 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f322009c3c0 con 0x55fe724ae390 2024-04-16T17:09:08.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.222+0000 7f323877d740 1 -- 172.21.15.45:0/2986530453 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fe724ae390 msgr2=0x55fe724b7060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:08.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.222+0000 7f323877d740 1 --2- 172.21.15.45:0/2986530453 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fe724ae390 0x55fe724b7060 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f32200ce210 tx=0x7f32200d32c0 comp rx=0 tx=0).stop 2024-04-16T17:09:08.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.222+0000 7f323877d740 1 -- 172.21.15.45:0/2986530453 shutdown_connections 2024-04-16T17:09:08.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.222+0000 7f323877d740 1 --2- 172.21.15.45:0/2986530453 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fe724b7890 0x55fe724c34e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:08.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.222+0000 7f323877d740 1 --2- 172.21.15.45:0/2986530453 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fe724aecb0 0x55fe724b3870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:08.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.222+0000 7f323877d740 1 --2- 172.21.15.45:0/2986530453 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fe724ae390 0x55fe724b7060 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:08.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.222+0000 7f323877d740 1 -- 172.21.15.45:0/2986530453 >> 172.21.15.45:0/2986530453 conn(0x55fe724a8ac0 msgr2=0x55fe724b5180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:09:08.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.222+0000 7f323877d740 1 -- 172.21.15.45:0/2986530453 shutdown_connections 2024-04-16T17:09:08.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.223+0000 7f323877d740 1 -- 172.21.15.45:0/2986530453 wait complete. 2024-04-16T17:09:08.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.223+0000 7f323877d740 1 -- start start 2024-04-16T17:09:08.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.223+0000 7f323877d740 1 Processor -- start 2024-04-16T17:09:08.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.224+0000 7f323877d740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fe724ae390 0x55fe724bfee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:08.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.224+0000 7f323877d740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fe724aecb0 0x55fe724bb250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:08.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.224+0000 7f323877d740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fe724b7890 0x55fe724bb860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:08.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.224+0000 7f323877d740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f32301b3ea0 con 0x55fe724ae390 2024-04-16T17:09:08.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.224+0000 7f323877d740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f32301b5310 con 0x55fe724aecb0 2024-04-16T17:09:08.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.224+0000 7f323877d740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f32301b4570 con 0x55fe724b7890 2024-04-16T17:09:08.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.224+0000 7f323877d740 10 client.0 _ll_register_callbacks cb 0x7f3228009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:09:08.226 INFO:tasks.workunit.client.0.smithi045.stdout: 1 36062 3.03 MB/sec warmup 70 sec latency 104.273 ms 2024-04-16T17:09:08.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.224+0000 7f322ea00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fe724ae390 0x55fe724bfee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:08.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.224+0000 7f322f400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fe724b7890 0x55fe724bb860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:08.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.224+0000 7f322ea00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fe724ae390 0x55fe724bfee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45096/0 (socket says 172.21.15.45:45096) 2024-04-16T17:09:08.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.224+0000 7f322f400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fe724b7890 0x55fe724bb860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:53438/0 (socket says 172.21.15.45:53438) 2024-04-16T17:09:08.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.224+0000 7f322f400640 1 -- 172.21.15.45:0/4021560415 learned_addr learned my addr 172.21.15.45:0/4021560415 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:09:08.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.224+0000 7f322e000640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fe724aecb0 0x55fe724bb250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:08.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.224+0000 7f322f400640 1 -- 172.21.15.45:0/4021560415 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fe724aecb0 msgr2=0x55fe724bb250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:09:08.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.224+0000 7f322f400640 1 --2- 172.21.15.45:0/4021560415 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fe724aecb0 0x55fe724bb250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:08.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.224+0000 7f322f400640 1 -- 172.21.15.45:0/4021560415 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fe724ae390 msgr2=0x55fe724bfee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:09:08.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.224+0000 7f322f400640 1 --2- 172.21.15.45:0/4021560415 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fe724ae390 0x55fe724bfee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:08.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.224+0000 7f322f400640 1 -- 172.21.15.45:0/4021560415 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fe724bc090 con 0x55fe724b7890 2024-04-16T17:09:08.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.225+0000 7f322e000640 1 --2- 172.21.15.45:0/4021560415 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fe724aecb0 0x55fe724bb250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:09:08.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.225+0000 7f322f400640 1 --2- 172.21.15.45:0/4021560415 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fe724b7890 0x55fe724bb860 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f3228004170 tx=0x7f3228004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4dcb5b560c86bf41 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:09:08.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.225+0000 7f3218c00640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-04-16T17:09:08.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.225+0000 7f3218c00640 1 -- 172.21.15.45:0/4021560415 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f322802e060 con 0x55fe724b7890 2024-04-16T17:09:08.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.225+0000 7f3218c00640 1 -- 172.21.15.45:0/4021560415 <== mon.1 v2:172.21.15.117:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f322800fa70 con 0x55fe724b7890 2024-04-16T17:09:08.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.225+0000 7f323877d740 1 -- 172.21.15.45:0/4021560415 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f32200bd0b0 con 0x55fe724b7890 2024-04-16T17:09:08.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.225+0000 7f3218c00640 1 -- 172.21.15.45:0/4021560415 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f3228035050 con 0x55fe724b7890 2024-04-16T17:09:08.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.225+0000 7f323877d740 10 client.26239 fetch_fsmap learned FSMap version 29 2024-04-16T17:09:08.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.225+0000 7f323877d740 1 -- 172.21.15.45:0/4021560415 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55fe7252df30 con 0x55fe724b7890 2024-04-16T17:09:08.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.226+0000 7f3218c00640 1 -- 172.21.15.45:0/4021560415 <== mon.1 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f32200bd0b0 con 0x55fe724b7890 2024-04-16T17:09:08.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.226+0000 7f3218c00640 1 -- 172.21.15.45:0/4021560415 <== mon.1 v2:172.21.15.117:3300/0 5 ==== fsmap(e 29) ==== 8366+0+0 (secure 0 0 0) 0x7f322801d3f0 con 0x55fe724b7890 2024-04-16T17:09:08.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.226+0000 7f323877d740 10 client.26239 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T17:09:08.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.226+0000 7f323877d740 10 client.26239 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:09:08.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.226+0000 7f323877d740 20 client.26239 populate_metadata read hostname 'smithi045' 2024-04-16T17:09:08.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.226+0000 7f323877d740 1 --2- 172.21.15.45:0/4021560415 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55fe7252e710 0x55fe72530b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:08.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.226+0000 7f323877d740 4 client.26239 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-16T17:09:08.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.226+0000 7f323877d740 1 -- 172.21.15.45:0/4021560415 --> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55fe7241b9b0 con 0x55fe7252e710 2024-04-16T17:09:08.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.226+0000 7f322ea00640 1 --2- 172.21.15.45:0/4021560415 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55fe7252e710 0x55fe72530b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:08.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.227+0000 7f3218c00640 10 client.26239 ms_handle_connect on v2:172.21.15.104:6838/3660070592 2024-04-16T17:09:08.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.227+0000 7f322ea00640 1 --2- 172.21.15.45:0/4021560415 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55fe7252e710 0x55fe72530b30 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a8c19dcfdbbb86c5 server_cookie=3b42bf06c90c8d1c in_seq=0 out_seq=0 2024-04-16T17:09:08.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.227+0000 7f3218c00640 1 -- 172.21.15.45:0/4021560415 <== mds.0 v2:172.21.15.104:6838/3660070592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55fe7241b9b0 con 0x55fe7252e710 2024-04-16T17:09:08.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.227+0000 7f3218c00640 10 client.26239 handle_command_reply: tid=0 2024-04-16T17:09:08.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.227+0000 7f323877d740 1 client.26239 shutdown 2024-04-16T17:09:08.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.227+0000 7f323877d740 1 -- 172.21.15.45:0/4021560415 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fe724b7890 msgr2=0x55fe724bb860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:08.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.227+0000 7f323877d740 1 --2- 172.21.15.45:0/4021560415 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fe724b7890 0x55fe724bb860 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f3228004170 tx=0x7f3228004a40 comp rx=0 tx=0).stop 2024-04-16T17:09:08.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.227+0000 7f323877d740 1 -- 172.21.15.45:0/4021560415 shutdown_connections 2024-04-16T17:09:08.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.227+0000 7f323877d740 1 --2- 172.21.15.45:0/4021560415 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55fe7252e710 0x55fe72530b30 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:08.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.227+0000 7f323877d740 1 --2- 172.21.15.45:0/4021560415 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fe724b7890 0x55fe724bb860 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:08.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.227+0000 7f323877d740 1 --2- 172.21.15.45:0/4021560415 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fe724aecb0 0x55fe724bb250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:08.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.227+0000 7f323877d740 1 --2- 172.21.15.45:0/4021560415 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fe724ae390 0x55fe724bfee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:08.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.227+0000 7f323877d740 1 -- 172.21.15.45:0/4021560415 >> 172.21.15.45:0/4021560415 conn(0x55fe724a8ac0 msgr2=0x55fe724a7290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:09:08.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.227+0000 7f3218c00640 0 client.26239 ms_handle_reset on v2:172.21.15.104:6838/3660070592 2024-04-16T17:09:08.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.228+0000 7f323877d740 1 -- 172.21.15.45:0/4021560415 shutdown_connections 2024-04-16T17:09:08.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.228+0000 7f323877d740 1 -- 172.21.15.45:0/4021560415 wait complete. 2024-04-16T17:09:08.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.228+0000 7f323877d740 20 client.26239 trim_cache size 0 max 16384 2024-04-16T17:09:08.232 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-16T17:09:08.232 INFO:teuthology.orchestra.run.smithi045.stdout: "status": "no active scrubs running", 2024-04-16T17:09:08.232 INFO:teuthology.orchestra.run.smithi045.stdout: "scrubs": {} 2024-04-16T17:09:08.232 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-16T17:09:08.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.228+0000 7f322fe00640 1 -- 172.21.15.45:0/384369854 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f32140a3d20 msgr2=0x7f32140a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:08.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.228+0000 7f322fe00640 1 --2- 172.21.15.45:0/384369854 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f32140a3d20 0x7f32140a61e0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f3228011630 tx=0x7f3228009500 comp rx=0 tx=0).stop 2024-04-16T17:09:08.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.228+0000 7f322fe00640 1 -- 172.21.15.45:0/384369854 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f32300743a0 msgr2=0x7f323019ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:08.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.228+0000 7f322fe00640 1 --2- 172.21.15.45:0/384369854 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f32300743a0 0x7f323019ee00 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f322000ea60 tx=0x7f322000ef20 comp rx=0 tx=0).stop 2024-04-16T17:09:08.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.229+0000 7f322fe00640 1 -- 172.21.15.45:0/384369854 shutdown_connections 2024-04-16T17:09:08.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.229+0000 7f322fe00640 1 --2- 172.21.15.45:0/384369854 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f32140a3d20 0x7f32140a61e0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:08.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.229+0000 7f322fe00640 1 --2- 172.21.15.45:0/384369854 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3230112eb0 0x7f32301a3670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:08.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.229+0000 7f322fe00640 1 --2- 172.21.15.45:0/384369854 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3230074cc0 0x7f323019f340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:08.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.229+0000 7f322fe00640 1 --2- 172.21.15.45:0/384369854 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f32300743a0 0x7f323019ee00 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:08.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.229+0000 7f322fe00640 1 -- 172.21.15.45:0/384369854 >> 172.21.15.45:0/384369854 conn(0x7f32300fd9d0 msgr2=0x7f32300ff2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:09:08.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.229+0000 7f322fe00640 1 -- 172.21.15.45:0/384369854 shutdown_connections 2024-04-16T17:09:08.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.229+0000 7f322fe00640 1 -- 172.21.15.45:0/384369854 wait complete. 2024-04-16T17:09:08.245 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-16T17:09:08.246 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-16T17:09:08.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.451+0000 7f1c1ce00640 1 -- 172.21.15.45:0/4032471482 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1c18107200 msgr2=0x7f1c1810edf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:08.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.451+0000 7f1c1ce00640 1 --2- 172.21.15.45:0/4032471482 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1c18107200 0x7f1c1810edf0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f1c10009900 tx=0x7f1c1002bc30 comp rx=0 tx=0).stop 2024-04-16T17:09:08.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.451+0000 7f1c1ce00640 1 -- 172.21.15.45:0/4032471482 shutdown_connections 2024-04-16T17:09:08.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.451+0000 7f1c1ce00640 1 --2- 172.21.15.45:0/4032471482 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1c181013f0 0x7f1c18106b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:08.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.451+0000 7f1c1ce00640 1 --2- 172.21.15.45:0/4032471482 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1c18107200 0x7f1c1810edf0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:08.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.451+0000 7f1c1ce00640 1 --2- 172.21.15.45:0/4032471482 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1c18100a40 0x7f1c18100e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:08.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.451+0000 7f1c1ce00640 1 -- 172.21.15.45:0/4032471482 >> 172.21.15.45:0/4032471482 conn(0x7f1c18076040 msgr2=0x7f1c18076450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:09:08.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.451+0000 7f1c1ce00640 1 -- 172.21.15.45:0/4032471482 shutdown_connections 2024-04-16T17:09:08.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.451+0000 7f1c1ce00640 1 -- 172.21.15.45:0/4032471482 wait complete. 2024-04-16T17:09:08.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.452+0000 7f1c1ce00640 1 Processor -- start 2024-04-16T17:09:08.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.453+0000 7f1c1ce00640 1 -- start start 2024-04-16T17:09:08.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.453+0000 7f1c1ce00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1c18100a40 0x7f1c1819e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:08.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.453+0000 7f1c1ce00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1c18107200 0x7f1c1819e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:08.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.453+0000 7f1c1ce00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1c181985f0 0x7f1c18198aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:08.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.453+0000 7f1c1ce00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f1c18112310 con 0x7f1c18100a40 2024-04-16T17:09:08.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.453+0000 7f1c1ce00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f1c18112190 con 0x7f1c18107200 2024-04-16T17:09:08.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.453+0000 7f1c1ce00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f1c18112490 con 0x7f1c181985f0 2024-04-16T17:09:08.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.453+0000 7f1c0ea00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1c18107200 0x7f1c1819e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:08.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.453+0000 7f1c17400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1c18100a40 0x7f1c1819e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:08.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.453+0000 7f1c0ea00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1c18107200 0x7f1c1819e930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:47684/0 (socket says 172.21.15.45:47684) 2024-04-16T17:09:08.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.453+0000 7f1c0ea00640 1 -- 172.21.15.45:0/187334853 learned_addr learned my addr 172.21.15.45:0/187334853 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:09:08.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.453+0000 7f1c17e00640 1 --2- 172.21.15.45:0/187334853 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1c181985f0 0x7f1c18198aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:08.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.454+0000 7f1c0ea00640 1 -- 172.21.15.45:0/187334853 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1c181985f0 msgr2=0x7f1c18198aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:08.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.454+0000 7f1c0ea00640 1 --2- 172.21.15.45:0/187334853 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1c181985f0 0x7f1c18198aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:08.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.454+0000 7f1c0ea00640 1 -- 172.21.15.45:0/187334853 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1c18100a40 msgr2=0x7f1c1819e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:08.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.454+0000 7f1c0ea00640 1 --2- 172.21.15.45:0/187334853 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1c18100a40 0x7f1c1819e3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:08.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.454+0000 7f1c0ea00640 1 -- 172.21.15.45:0/187334853 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c18199110 con 0x7f1c18107200 2024-04-16T17:09:08.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.454+0000 7f1c0ea00640 1 --2- 172.21.15.45:0/187334853 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1c18107200 0x7f1c1819e930 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f1c0000e9a0 tx=0x7f1c0000ee60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:09:08.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.454+0000 7f1c14c00640 1 -- 172.21.15.45:0/187334853 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c0000cd80 con 0x7f1c18107200 2024-04-16T17:09:08.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.454+0000 7f1c14c00640 1 -- 172.21.15.45:0/187334853 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1c000035f0 con 0x7f1c18107200 2024-04-16T17:09:08.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.454+0000 7f1c1ce00640 1 -- 172.21.15.45:0/187334853 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c18199400 con 0x7f1c18107200 2024-04-16T17:09:08.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.454+0000 7f1c14c00640 1 -- 172.21.15.45:0/187334853 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c00010670 con 0x7f1c18107200 2024-04-16T17:09:08.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.454+0000 7f1c1ce00640 1 -- 172.21.15.45:0/187334853 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c181a5600 con 0x7f1c18107200 2024-04-16T17:09:08.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.455+0000 7f1c1ce00640 1 -- 172.21.15.45:0/187334853 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c1806a050 con 0x7f1c18107200 2024-04-16T17:09:08.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.457+0000 7f1c14c00640 1 -- 172.21.15.45:0/187334853 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f1c000037a0 con 0x7f1c18107200 2024-04-16T17:09:08.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.459+0000 7f1c14c00640 1 --2- 172.21.15.45:0/187334853 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f1bfc0a3d20 0x7f1bfc0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:08.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.459+0000 7f1c14c00640 1 -- 172.21.15.45:0/187334853 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f1c000d0d90 con 0x7f1c18107200 2024-04-16T17:09:08.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.459+0000 7f1c17400640 1 --2- 172.21.15.45:0/187334853 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f1bfc0a3d20 0x7f1bfc0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:08.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.460+0000 7f1c17400640 1 --2- 172.21.15.45:0/187334853 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f1bfc0a3d20 0x7f1bfc0a61e0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f1c080099a0 tx=0x7f1c08009500 comp rx=0 tx=0).ready entity=mgr.25411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:09:08.462 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.463+0000 7f1c14c00640 1 -- 172.21.15.45:0/187334853 <== mon.2 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1c0009d080 con 0x7f1c18107200 2024-04-16T17:09:08.477 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:09:07 smithi045 ceph-mon[31676]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-16T17:09:08.477 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:09:07 smithi045 ceph-mon[31676]: pgmap v74: 129 pgs: 129 active+clean; 27 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 483 KiB/s rd, 305 KiB/s wr, 100 op/s 2024-04-16T17:09:08.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.714+0000 7f1c1ce00640 1 -- 172.21.15.45:0/187334853 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1c1810a3b0 con 0x7f1c18107200 2024-04-16T17:09:08.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.716+0000 7f1c14c00640 1 -- 172.21.15.45:0/187334853 <== mon.2 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11894 (secure 0 0 0) 0x7f1c0009c980 con 0x7f1c18107200 2024-04-16T17:09:08.716 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-16T17:09:08.716 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/1748842443","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":1748842443},{"type":"v1","addr":"172.21.15.45:6835","nonce":1748842443}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6839/1308627051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":1308627051},{"type":"v1","addr":"172.21.15.45:6839","nonce":1308627051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/1768242489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1768242489},{"type":"v1","addr":"172.21.15.117:6833","nonce":1768242489}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6833/3558902585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":3558902585},{"type":"v1","addr":"172.21.15.104:6833","nonce":3558902585}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24478,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/1308142903","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":1308142903},{"type":"v1","addr":"172.21.15.45:6837","nonce":1308142903}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24491,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6835/3430679937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":3430679937},{"type":"v1","addr":"172.21.15.104:6835","nonce":3430679937}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-16T17:03:24.663611+0000","modified":"2024-04-16T17:09:05.713992+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24581,"mds_1":24569,"mds_2":24532},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24499":{"gid":24499,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/2224123841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2224123841},{"type":"v1","addr":"172.21.15.117:6835","nonce":2224123841}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6837/2471356815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":2471356815},{"type":"v1","addr":"172.21.15.117:6837","nonce":2471356815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24524":{"gid":24524,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6837/2647756509","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":2647756509},{"type":"v1","addr":"172.21.15.104:6837","nonce":2647756509}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24532":{"gid":24532,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":13,"addr":"172.21.15.117:6839/2378675949","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":2378675949},{"type":"v1","addr":"172.21.15.117:6839","nonce":2378675949}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24569":{"gid":24569,"name":"a","rank":1,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.45:6841/3755310822","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6840","nonce":3755310822},{"type":"v1","addr":"172.21.15.45:6841","nonce":3755310822}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24581":{"gid":24581,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.104:6839/3660070592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6838","nonce":3660070592},{"type":"v1","addr":"172.21.15.104:6839","nonce":3660070592}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24581,"qdb_cluster":[24569,24581,24532]},"id":1}]} 2024-04-16T17:09:08.716 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 29 2024-04-16T17:09:08.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.718+0000 7f1c1ce00640 1 -- 172.21.15.45:0/187334853 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f1bfc0a3d20 msgr2=0x7f1bfc0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:08.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.718+0000 7f1c1ce00640 1 --2- 172.21.15.45:0/187334853 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f1bfc0a3d20 0x7f1bfc0a61e0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f1c080099a0 tx=0x7f1c08009500 comp rx=0 tx=0).stop 2024-04-16T17:09:08.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.718+0000 7f1c1ce00640 1 -- 172.21.15.45:0/187334853 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1c18107200 msgr2=0x7f1c1819e930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:08.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.718+0000 7f1c1ce00640 1 --2- 172.21.15.45:0/187334853 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1c18107200 0x7f1c1819e930 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f1c0000e9a0 tx=0x7f1c0000ee60 comp rx=0 tx=0).stop 2024-04-16T17:09:08.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.718+0000 7f1c1ce00640 1 -- 172.21.15.45:0/187334853 shutdown_connections 2024-04-16T17:09:08.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.718+0000 7f1c1ce00640 1 --2- 172.21.15.45:0/187334853 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f1bfc0a3d20 0x7f1bfc0a61e0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:08.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.718+0000 7f1c1ce00640 1 --2- 172.21.15.45:0/187334853 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1c181985f0 0x7f1c18198aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:08.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.718+0000 7f1c1ce00640 1 --2- 172.21.15.45:0/187334853 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1c18107200 0x7f1c1819e930 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:08.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.718+0000 7f1c1ce00640 1 --2- 172.21.15.45:0/187334853 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1c18100a40 0x7f1c1819e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:08.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.718+0000 7f1c1ce00640 1 -- 172.21.15.45:0/187334853 >> 172.21.15.45:0/187334853 conn(0x7f1c18076040 msgr2=0x7f1c180fe9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:09:08.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.718+0000 7f1c1ce00640 1 -- 172.21.15.45:0/187334853 shutdown_connections 2024-04-16T17:09:08.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.719+0000 7f1c1ce00640 1 -- 172.21.15.45:0/187334853 wait complete. 2024-04-16T17:09:08.738 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-16T17:09:08.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.941+0000 7f6f6e200640 1 -- 172.21.15.45:0/3697282792 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6f681044b0 msgr2=0x7f6f68104890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:08.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.941+0000 7f6f6e200640 1 --2- 172.21.15.45:0/3697282792 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6f681044b0 0x7f6f68104890 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f6f58009900 tx=0x7f6f5802bc30 comp rx=0 tx=0).stop 2024-04-16T17:09:08.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.941+0000 7f6f6e200640 1 -- 172.21.15.45:0/3697282792 shutdown_connections 2024-04-16T17:09:08.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.941+0000 7f6f6e200640 1 --2- 172.21.15.45:0/3697282792 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6f68109580 0x7f6f68111150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:08.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.941+0000 7f6f6e200640 1 --2- 172.21.15.45:0/3697282792 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6f68104e60 0x7f6f68108ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:08.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.941+0000 7f6f6e200640 1 --2- 172.21.15.45:0/3697282792 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6f681044b0 0x7f6f68104890 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:08.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.941+0000 7f6f6e200640 1 -- 172.21.15.45:0/3697282792 >> 172.21.15.45:0/3697282792 conn(0x7f6f680ffd90 msgr2=0x7f6f681021b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:09:08.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.941+0000 7f6f6e200640 1 -- 172.21.15.45:0/3697282792 shutdown_connections 2024-04-16T17:09:08.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.942+0000 7f6f6e200640 1 -- 172.21.15.45:0/3697282792 wait complete. 2024-04-16T17:09:08.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.943+0000 7f6f6e200640 1 Processor -- start 2024-04-16T17:09:08.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.943+0000 7f6f6e200640 1 -- start start 2024-04-16T17:09:08.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.943+0000 7f6f6e200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6f681044b0 0x7f6f681a07e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:08.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.943+0000 7f6f6e200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6f68109580 0x7f6f681a0d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:08.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.943+0000 7f6f6e200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6f6819a9e0 0x7f6f6819ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:08.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.943+0000 7f6f6e200640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f6f68114640 con 0x7f6f68109580 2024-04-16T17:09:08.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.943+0000 7f6f6e200640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f6f681144c0 con 0x7f6f6819a9e0 2024-04-16T17:09:08.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.943+0000 7f6f6e200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f6f681147c0 con 0x7f6f681044b0 2024-04-16T17:09:08.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.944+0000 7f6f66a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6f68109580 0x7f6f681a0d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:08.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.944+0000 7f6f67400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6f681044b0 0x7f6f681a07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:08.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.944+0000 7f6f66a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6f68109580 0x7f6f681a0d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45136/0 (socket says 172.21.15.45:45136) 2024-04-16T17:09:08.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.944+0000 7f6f66a00640 1 -- 172.21.15.45:0/489729692 learned_addr learned my addr 172.21.15.45:0/489729692 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:09:08.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.944+0000 7f6f67e00640 1 --2- 172.21.15.45:0/489729692 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6f6819a9e0 0x7f6f6819ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:08.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.944+0000 7f6f67400640 1 -- 172.21.15.45:0/489729692 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6f6819a9e0 msgr2=0x7f6f6819ae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:08.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.944+0000 7f6f67400640 1 --2- 172.21.15.45:0/489729692 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6f6819a9e0 0x7f6f6819ae90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:08.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.944+0000 7f6f67400640 1 -- 172.21.15.45:0/489729692 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6f68109580 msgr2=0x7f6f681a0d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:08.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.944+0000 7f6f67400640 1 --2- 172.21.15.45:0/489729692 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6f68109580 0x7f6f681a0d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:08.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.944+0000 7f6f67400640 1 -- 172.21.15.45:0/489729692 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f6819b500 con 0x7f6f681044b0 2024-04-16T17:09:08.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.945+0000 7f6f67400640 1 --2- 172.21.15.45:0/489729692 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6f681044b0 0x7f6f681a07e0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f6f58036040 tx=0x7f6f5802cb20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:09:08.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.945+0000 7f6f5fe00640 1 -- 172.21.15.45:0/489729692 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f58041070 con 0x7f6f681044b0 2024-04-16T17:09:08.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.945+0000 7f6f6e200640 1 -- 172.21.15.45:0/489729692 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f6819b790 con 0x7f6f681044b0 2024-04-16T17:09:08.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.945+0000 7f6f5fe00640 1 -- 172.21.15.45:0/489729692 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6f58004450 con 0x7f6f681044b0 2024-04-16T17:09:08.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.945+0000 7f6f5fe00640 1 -- 172.21.15.45:0/489729692 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f58003d90 con 0x7f6f681044b0 2024-04-16T17:09:08.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.945+0000 7f6f6e200640 1 -- 172.21.15.45:0/489729692 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f681a7990 con 0x7f6f681044b0 2024-04-16T17:09:08.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.948+0000 7f6f5fe00640 1 -- 172.21.15.45:0/489729692 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f6f58044070 con 0x7f6f681044b0 2024-04-16T17:09:08.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.948+0000 7f6f71008740 1 Processor -- start 2024-04-16T17:09:08.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.948+0000 7f6f71008740 1 -- start start 2024-04-16T17:09:08.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.949+0000 7f6f71008740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f36a212d50 0x55f36a213130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:08.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.949+0000 7f6f71008740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f36a213670 0x55f36a21bd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:08.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.949+0000 7f6f71008740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f36a21c250 0x55f36a21e640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:08.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.949+0000 7f6f71008740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55f36a178020 con 0x55f36a212d50 2024-04-16T17:09:08.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.949+0000 7f6f71008740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55f36a11e460 con 0x55f36a213670 2024-04-16T17:09:08.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.949+0000 7f6f71008740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55f36a173fe0 con 0x55f36a21c250 2024-04-16T17:09:08.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.949+0000 7f6f67e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f36a212d50 0x55f36a213130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:08.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.949+0000 7f6f67e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f36a212d50 0x55f36a213130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45150/0 (socket says 172.21.15.45:45150) 2024-04-16T17:09:08.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.949+0000 7f6f67e00640 1 -- 172.21.15.45:0/1362236760 learned_addr learned my addr 172.21.15.45:0/1362236760 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:09:08.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.949+0000 7f6f67400640 1 --2- 172.21.15.45:0/1362236760 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f36a213670 0x55f36a21bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:08.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.949+0000 7f6f66a00640 1 --2- 172.21.15.45:0/1362236760 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f36a21c250 0x55f36a21e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:08.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.950+0000 7f6f66a00640 1 -- 172.21.15.45:0/1362236760 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f36a213670 msgr2=0x55f36a21bd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:09:08.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.950+0000 7f6f66a00640 1 --2- 172.21.15.45:0/1362236760 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f36a213670 0x55f36a21bd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:08.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.950+0000 7f6f66a00640 1 -- 172.21.15.45:0/1362236760 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f36a212d50 msgr2=0x55f36a213130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:09:08.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.950+0000 7f6f66a00640 1 --2- 172.21.15.45:0/1362236760 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f36a212d50 0x55f36a213130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:08.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.950+0000 7f6f66a00640 1 -- 172.21.15.45:0/1362236760 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f36a126040 con 0x55f36a21c250 2024-04-16T17:09:08.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.950+0000 7f6f67e00640 1 --2- 172.21.15.45:0/1362236760 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f36a212d50 0x55f36a213130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:09:08.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.950+0000 7f6f5fe00640 1 --2- 172.21.15.45:0/489729692 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f6f4c0a3d20 0x7f6f4c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:08.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.950+0000 7f6f5fe00640 1 -- 172.21.15.45:0/489729692 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f6f580f2480 con 0x7f6f681044b0 2024-04-16T17:09:08.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.950+0000 7f6f66a00640 1 --2- 172.21.15.45:0/1362236760 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f36a21c250 0x55f36a21e640 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f6f6819c180 tx=0x7f6f54011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c44f6d0b63c58008 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:09:08.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.950+0000 7f6f67e00640 1 --2- 172.21.15.45:0/489729692 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f6f4c0a3d20 0x7f6f4c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:08.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.950+0000 7f6f53e00640 1 -- 172.21.15.45:0/1362236760 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f54009d50 con 0x55f36a21c250 2024-04-16T17:09:08.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.950+0000 7f6f53e00640 1 -- 172.21.15.45:0/1362236760 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6f54005500 con 0x55f36a21c250 2024-04-16T17:09:08.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.950+0000 7f6f53e00640 1 -- 172.21.15.45:0/1362236760 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f54019670 con 0x55f36a21c250 2024-04-16T17:09:08.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.951+0000 7f6f71008740 1 -- 172.21.15.45:0/1362236760 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f36a21c250 msgr2=0x55f36a21e640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:08.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.951+0000 7f6f67e00640 1 --2- 172.21.15.45:0/489729692 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f6f4c0a3d20 0x7f6f4c0a61e0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f6f60009fd0 tx=0x7f6f60009500 comp rx=0 tx=0).ready entity=mgr.25411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:09:08.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.951+0000 7f6f71008740 1 --2- 172.21.15.45:0/1362236760 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f36a21c250 0x55f36a21e640 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f6f6819c180 tx=0x7f6f54011810 comp rx=0 tx=0).stop 2024-04-16T17:09:08.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.951+0000 7f6f71008740 1 -- 172.21.15.45:0/1362236760 shutdown_connections 2024-04-16T17:09:08.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.951+0000 7f6f71008740 1 --2- 172.21.15.45:0/1362236760 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f36a21c250 0x55f36a21e640 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:08.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.951+0000 7f6f71008740 1 --2- 172.21.15.45:0/1362236760 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f36a213670 0x55f36a21bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:08.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.951+0000 7f6f71008740 1 --2- 172.21.15.45:0/1362236760 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f36a212d50 0x55f36a213130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:08.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.951+0000 7f6f71008740 1 -- 172.21.15.45:0/1362236760 >> 172.21.15.45:0/1362236760 conn(0x55f36a20d480 msgr2=0x55f36a20d860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:09:08.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.951+0000 7f6f71008740 1 -- 172.21.15.45:0/1362236760 shutdown_connections 2024-04-16T17:09:08.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.951+0000 7f6f71008740 1 -- 172.21.15.45:0/1362236760 wait complete. 2024-04-16T17:09:08.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.952+0000 7f6f71008740 1 -- start start 2024-04-16T17:09:08.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.952+0000 7f6f71008740 1 Processor -- start 2024-04-16T17:09:08.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.952+0000 7f6f71008740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f36a212d50 0x55f36a219270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:08.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.952+0000 7f6f71008740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f36a213670 0x55f36a2197b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:08.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.952+0000 7f6f71008740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f36a21c250 0x55f36a219cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:08.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.952+0000 7f6f71008740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55f36a21f040 con 0x55f36a213670 2024-04-16T17:09:08.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.952+0000 7f6f71008740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55f36a131830 con 0x55f36a212d50 2024-04-16T17:09:08.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.952+0000 7f6f71008740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55f36a21f1c0 con 0x55f36a21c250 2024-04-16T17:09:08.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.953+0000 7f6f71008740 10 client.0 _ll_register_callbacks cb 0x55f36a139150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:09:08.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.953+0000 7f6f67400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f36a212d50 0x55f36a219270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:08.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.953+0000 7f6f67400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f36a212d50 0x55f36a219270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:47724/0 (socket says 172.21.15.45:47724) 2024-04-16T17:09:08.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.953+0000 7f6f67400640 1 -- 172.21.15.45:0/836799065 learned_addr learned my addr 172.21.15.45:0/836799065 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:09:08.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.953+0000 7f6f66a00640 1 --2- 172.21.15.45:0/836799065 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f36a213670 0x55f36a2197b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:08.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.953+0000 7f6f67e00640 1 --2- 172.21.15.45:0/836799065 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f36a21c250 0x55f36a219cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:08.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.953+0000 7f6f67400640 1 -- 172.21.15.45:0/836799065 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f36a21c250 msgr2=0x55f36a219cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:09:08.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.953+0000 7f6f67400640 1 --2- 172.21.15.45:0/836799065 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f36a21c250 0x55f36a219cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:08.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.953+0000 7f6f67400640 1 -- 172.21.15.45:0/836799065 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f36a213670 msgr2=0x55f36a2197b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:09:08.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.953+0000 7f6f67400640 1 --2- 172.21.15.45:0/836799065 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f36a213670 0x55f36a2197b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:08.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.953+0000 7f6f67400640 1 -- 172.21.15.45:0/836799065 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f36a135920 con 0x55f36a212d50 2024-04-16T17:09:08.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.953+0000 7f6f66a00640 1 --2- 172.21.15.45:0/836799065 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f36a213670 0x55f36a2197b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:09:08.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.953+0000 7f6f67400640 1 --2- 172.21.15.45:0/836799065 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f36a212d50 0x55f36a219270 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f6f5802c820 tx=0x7f6f58005be0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=13352034bcd6c3ad server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:09:08.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.953+0000 7f6f50c00640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-04-16T17:09:08.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.954+0000 7f6f50c00640 1 -- 172.21.15.45:0/836799065 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f5804b070 con 0x55f36a212d50 2024-04-16T17:09:08.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.954+0000 7f6f50c00640 1 -- 172.21.15.45:0/836799065 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f6f580efaa0 con 0x55f36a212d50 2024-04-16T17:09:08.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.954+0000 7f6f71008740 1 -- 172.21.15.45:0/836799065 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6f54005500 con 0x55f36a212d50 2024-04-16T17:09:08.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.954+0000 7f6f50c00640 1 -- 172.21.15.45:0/836799065 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f6f580be000 con 0x55f36a212d50 2024-04-16T17:09:08.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.954+0000 7f6f71008740 10 client.26276 fetch_fsmap learned FSMap version 29 2024-04-16T17:09:08.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.954+0000 7f6f71008740 1 -- 172.21.15.45:0/836799065 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55f36a1037f0 con 0x55f36a212d50 2024-04-16T17:09:08.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.954+0000 7f6f50c00640 1 -- 172.21.15.45:0/836799065 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6f54005500 con 0x55f36a212d50 2024-04-16T17:09:08.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.954+0000 7f6f50c00640 1 -- 172.21.15.45:0/836799065 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 29) ==== 8366+0+0 (secure 0 0 0) 0x7f6f580c2150 con 0x55f36a212d50 2024-04-16T17:09:08.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.954+0000 7f6f71008740 10 client.26276 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T17:09:08.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.954+0000 7f6f71008740 10 client.26276 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-16T17:09:08.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.954+0000 7f6f71008740 20 client.26276 populate_metadata read hostname 'smithi045' 2024-04-16T17:09:08.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.954+0000 7f6f71008740 1 --2- 172.21.15.45:0/836799065 >> [v2:172.21.15.117:6838/2378675949,v1:172.21.15.117:6839/2378675949] conn(0x55f36a22c490 0x55f36a22e910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:08.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.955+0000 7f6f71008740 4 client.26276 mds_command: new command op to 24532 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:09:08.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.955+0000 7f6f71008740 1 -- 172.21.15.45:0/836799065 --> [v2:172.21.15.117:6838/2378675949,v1:172.21.15.117:6839/2378675949] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f36a185a40 con 0x55f36a22c490 2024-04-16T17:09:08.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.955+0000 7f6f66a00640 1 --2- 172.21.15.45:0/836799065 >> [v2:172.21.15.117:6838/2378675949,v1:172.21.15.117:6839/2378675949] conn(0x55f36a22c490 0x55f36a22e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:08.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.955+0000 7f6f66a00640 1 --2- 172.21.15.45:0/836799065 >> [v2:172.21.15.117:6838/2378675949,v1:172.21.15.117:6839/2378675949] conn(0x55f36a22c490 0x55f36a22e910 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8f55718e27d0db9d server_cookie=4c8a2c345490f9c9 in_seq=0 out_seq=0 2024-04-16T17:09:08.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.955+0000 7f6f50c00640 10 client.26276 ms_handle_connect on v2:172.21.15.117:6838/2378675949 2024-04-16T17:09:08.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.956+0000 7f6f50c00640 1 -- 172.21.15.45:0/836799065 <== mds.2 v2:172.21.15.117:6838/2378675949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f36a185a40 con 0x55f36a22c490 2024-04-16T17:09:08.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.956+0000 7f6f50c00640 10 client.26276 handle_command_reply: tid=0 2024-04-16T17:09:08.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.956+0000 7f6f71008740 1 client.26276 shutdown 2024-04-16T17:09:08.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.956+0000 7f6f71008740 1 -- 172.21.15.45:0/836799065 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f36a212d50 msgr2=0x55f36a219270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:08.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.956+0000 7f6f71008740 1 --2- 172.21.15.45:0/836799065 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f36a212d50 0x55f36a219270 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f6f5802c820 tx=0x7f6f58005be0 comp rx=0 tx=0).stop 2024-04-16T17:09:08.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.956+0000 7f6f71008740 1 -- 172.21.15.45:0/836799065 shutdown_connections 2024-04-16T17:09:08.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.956+0000 7f6f71008740 1 --2- 172.21.15.45:0/836799065 >> [v2:172.21.15.117:6838/2378675949,v1:172.21.15.117:6839/2378675949] conn(0x55f36a22c490 0x55f36a22e910 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:08.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.956+0000 7f6f71008740 1 --2- 172.21.15.45:0/836799065 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f36a21c250 0x55f36a219cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:08.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.956+0000 7f6f71008740 1 --2- 172.21.15.45:0/836799065 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f36a213670 0x55f36a2197b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:08.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.956+0000 7f6f71008740 1 --2- 172.21.15.45:0/836799065 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f36a212d50 0x55f36a219270 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:08.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.956+0000 7f6f50c00640 0 client.26276 ms_handle_reset on v2:172.21.15.117:6838/2378675949 2024-04-16T17:09:08.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.956+0000 7f6f71008740 1 -- 172.21.15.45:0/836799065 >> 172.21.15.45:0/836799065 conn(0x55f36a20d480 msgr2=0x55f36a20f1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:09:08.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.957+0000 7f6f71008740 1 -- 172.21.15.45:0/836799065 shutdown_connections 2024-04-16T17:09:08.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.957+0000 7f6f71008740 1 -- 172.21.15.45:0/836799065 wait complete. 2024-04-16T17:09:08.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.957+0000 7f6f71008740 20 client.26276 trim_cache size 0 max 16384 2024-04-16T17:09:08.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.971+0000 7f6f71008740 1 Processor -- start 2024-04-16T17:09:08.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.971+0000 7f6f71008740 1 -- start start 2024-04-16T17:09:08.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.971+0000 7f6f71008740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f36a212d50 0x55f36a227f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:08.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.971+0000 7f6f71008740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f36a213670 0x55f36a219b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:08.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.971+0000 7f6f71008740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f36a21c250 0x55f36a20c360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:08.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.971+0000 7f6f71008740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f6f681b4000 con 0x55f36a213670 2024-04-16T17:09:08.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.971+0000 7f6f71008740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f6f681b1840 con 0x55f36a21c250 2024-04-16T17:09:08.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.971+0000 7f6f71008740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f6f681af670 con 0x55f36a212d50 2024-04-16T17:09:08.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.971+0000 7f6f67400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f36a212d50 0x55f36a227f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:08.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.971+0000 7f6f66a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f36a213670 0x55f36a219b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:08.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.971+0000 7f6f67400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f36a212d50 0x55f36a227f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:53502/0 (socket says 172.21.15.45:53502) 2024-04-16T17:09:08.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.971+0000 7f6f67400640 1 -- 172.21.15.45:0/2933939909 learned_addr learned my addr 172.21.15.45:0/2933939909 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:09:08.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.972+0000 7f6f67e00640 1 --2- 172.21.15.45:0/2933939909 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f36a21c250 0x55f36a20c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:08.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.972+0000 7f6f67400640 1 -- 172.21.15.45:0/2933939909 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f36a21c250 msgr2=0x55f36a20c360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:09:08.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.972+0000 7f6f67400640 1 --2- 172.21.15.45:0/2933939909 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f36a21c250 0x55f36a20c360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:08.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.972+0000 7f6f67400640 1 -- 172.21.15.45:0/2933939909 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f36a213670 msgr2=0x55f36a219b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:09:08.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.972+0000 7f6f67400640 1 --2- 172.21.15.45:0/2933939909 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f36a213670 0x55f36a219b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:08.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.972+0000 7f6f67400640 1 -- 172.21.15.45:0/2933939909 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f369e8a3f0 con 0x55f36a212d50 2024-04-16T17:09:08.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.972+0000 7f6f66a00640 1 --2- 172.21.15.45:0/2933939909 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f36a213670 0x55f36a219b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:09:08.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.972+0000 7f6f67400640 1 --2- 172.21.15.45:0/2933939909 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f36a212d50 0x55f36a227f00 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f6f580bac00 tx=0x7f6f58004ee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9e645f24a5b5ddb6 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:09:08.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.972+0000 7f6f53400640 1 -- 172.21.15.45:0/2933939909 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f580d6070 con 0x55f36a212d50 2024-04-16T17:09:08.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.972+0000 7f6f53400640 1 -- 172.21.15.45:0/2933939909 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6f58004c80 con 0x55f36a212d50 2024-04-16T17:09:08.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.972+0000 7f6f53400640 1 -- 172.21.15.45:0/2933939909 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f5804b070 con 0x55f36a212d50 2024-04-16T17:09:08.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.973+0000 7f6f71008740 1 -- 172.21.15.45:0/2933939909 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f36a212d50 msgr2=0x55f36a227f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:08.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.973+0000 7f6f71008740 1 --2- 172.21.15.45:0/2933939909 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f36a212d50 0x55f36a227f00 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f6f580bac00 tx=0x7f6f58004ee0 comp rx=0 tx=0).stop 2024-04-16T17:09:08.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.973+0000 7f6f71008740 1 -- 172.21.15.45:0/2933939909 shutdown_connections 2024-04-16T17:09:08.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.973+0000 7f6f71008740 1 --2- 172.21.15.45:0/2933939909 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f36a21c250 0x55f36a20c360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:08.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.973+0000 7f6f71008740 1 --2- 172.21.15.45:0/2933939909 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f36a213670 0x55f36a219b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:08.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.973+0000 7f6f71008740 1 --2- 172.21.15.45:0/2933939909 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f36a212d50 0x55f36a227f00 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:08.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.973+0000 7f6f71008740 1 -- 172.21.15.45:0/2933939909 >> 172.21.15.45:0/2933939909 conn(0x55f36a20d480 msgr2=0x55f36a20bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:09:08.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.973+0000 7f6f71008740 1 -- 172.21.15.45:0/2933939909 shutdown_connections 2024-04-16T17:09:08.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.973+0000 7f6f71008740 1 -- 172.21.15.45:0/2933939909 wait complete. 2024-04-16T17:09:08.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.973+0000 7f6f71008740 1 -- start start 2024-04-16T17:09:08.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.974+0000 7f6f71008740 1 Processor -- start 2024-04-16T17:09:08.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.974+0000 7f6f71008740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f36a212d50 0x55f36a224100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:08.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.974+0000 7f6f71008740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f36a213670 0x55f36a224640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:08.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.974+0000 7f6f71008740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f36a21c250 0x55f36a21f780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:08.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.974+0000 7f6f71008740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f6f681afd00 con 0x55f36a212d50 2024-04-16T17:09:08.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.974+0000 7f6f71008740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f6f681b1170 con 0x55f36a21c250 2024-04-16T17:09:08.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.974+0000 7f6f71008740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f6f681b03d0 con 0x55f36a213670 2024-04-16T17:09:08.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.974+0000 7f6f71008740 10 client.0 _ll_register_callbacks cb 0x7f6f58009880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:09:08.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.974+0000 7f6f67e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f36a21c250 0x55f36a21f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:08.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.974+0000 7f6f67400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f36a212d50 0x55f36a224100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:08.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.974+0000 7f6f66a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f36a213670 0x55f36a224640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:08.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.975+0000 7f6f66a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f36a213670 0x55f36a224640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:53510/0 (socket says 172.21.15.45:53510) 2024-04-16T17:09:08.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.975+0000 7f6f67400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f36a212d50 0x55f36a224100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45180/0 (socket says 172.21.15.45:45180) 2024-04-16T17:09:08.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.975+0000 7f6f67400640 1 -- 172.21.15.45:0/2520174287 learned_addr learned my addr 172.21.15.45:0/2520174287 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:09:08.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.975+0000 7f6f67e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f36a21c250 0x55f36a21f780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:47740/0 (socket says 172.21.15.45:47740) 2024-04-16T17:09:08.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.975+0000 7f6f67400640 1 -- 172.21.15.45:0/2520174287 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f36a213670 msgr2=0x55f36a224640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:09:08.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.975+0000 7f6f67400640 1 --2- 172.21.15.45:0/2520174287 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f36a213670 0x55f36a224640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:08.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.975+0000 7f6f67400640 1 -- 172.21.15.45:0/2520174287 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f36a21c250 msgr2=0x55f36a21f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:09:08.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.975+0000 7f6f67400640 1 --2- 172.21.15.45:0/2520174287 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f36a21c250 0x55f36a21f780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:08.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.975+0000 7f6f67400640 1 -- 172.21.15.45:0/2520174287 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f369f547d0 con 0x55f36a212d50 2024-04-16T17:09:08.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.975+0000 7f6f66a00640 1 --2- 172.21.15.45:0/2520174287 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f36a213670 0x55f36a224640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:09:08.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.975+0000 7f6f67e00640 1 --2- 172.21.15.45:0/2520174287 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f36a21c250 0x55f36a21f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:09:08.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.975+0000 7f6f67400640 1 --2- 172.21.15.45:0/2520174287 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f36a212d50 0x55f36a224100 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f6f580bc4e0 tx=0x7f6f58000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4b0328824cd9df7 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:09:08.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.975+0000 7f6f50c00640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-16T17:09:08.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.975+0000 7f6f50c00640 1 -- 172.21.15.45:0/2520174287 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f580d6020 con 0x55f36a212d50 2024-04-16T17:09:08.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.976+0000 7f6f50c00640 1 -- 172.21.15.45:0/2520174287 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f6f580bd680 con 0x55f36a212d50 2024-04-16T17:09:08.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.976+0000 7f6f71008740 1 -- 172.21.15.45:0/2520174287 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6f58004c80 con 0x55f36a212d50 2024-04-16T17:09:08.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.976+0000 7f6f50c00640 1 -- 172.21.15.45:0/2520174287 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f6f5804b070 con 0x55f36a212d50 2024-04-16T17:09:08.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.976+0000 7f6f71008740 10 client.16134 fetch_fsmap learned FSMap version 29 2024-04-16T17:09:08.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.976+0000 7f6f71008740 1 -- 172.21.15.45:0/2520174287 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55f369f547d0 con 0x55f36a212d50 2024-04-16T17:09:08.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.976+0000 7f6f50c00640 1 -- 172.21.15.45:0/2520174287 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6f58004c80 con 0x55f36a212d50 2024-04-16T17:09:08.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.976+0000 7f6f50c00640 1 -- 172.21.15.45:0/2520174287 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 29) ==== 8366+0+0 (secure 0 0 0) 0x7f6f580bd0f0 con 0x55f36a212d50 2024-04-16T17:09:08.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.976+0000 7f6f71008740 10 client.16134 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T17:09:08.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.976+0000 7f6f71008740 10 client.16134 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-16T17:09:08.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.976+0000 7f6f71008740 20 client.16134 populate_metadata read hostname 'smithi045' 2024-04-16T17:09:08.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.976+0000 7f6f71008740 1 --2- 172.21.15.45:0/2520174287 >> [v2:172.21.15.117:6838/2378675949,v1:172.21.15.117:6839/2378675949] conn(0x55f36a292610 0x55f36a294a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:08.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.976+0000 7f6f71008740 4 client.16134 mds_command: new command op to 24532 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T17:09:08.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.976+0000 7f6f71008740 1 -- 172.21.15.45:0/2520174287 --> [v2:172.21.15.117:6838/2378675949,v1:172.21.15.117:6839/2378675949] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f36a17f8b0 con 0x55f36a292610 2024-04-16T17:09:08.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.977+0000 7f6f66a00640 1 --2- 172.21.15.45:0/2520174287 >> [v2:172.21.15.117:6838/2378675949,v1:172.21.15.117:6839/2378675949] conn(0x55f36a292610 0x55f36a294a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:08.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.977+0000 7f6f66a00640 1 --2- 172.21.15.45:0/2520174287 >> [v2:172.21.15.117:6838/2378675949,v1:172.21.15.117:6839/2378675949] conn(0x55f36a292610 0x55f36a294a00 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=42b0a902ad8e6d3a server_cookie=3b9f0e40a046223f in_seq=0 out_seq=0 2024-04-16T17:09:08.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.977+0000 7f6f50c00640 10 client.16134 ms_handle_connect on v2:172.21.15.117:6838/2378675949 2024-04-16T17:09:08.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.977+0000 7f6f50c00640 1 -- 172.21.15.45:0/2520174287 <== mds.2 v2:172.21.15.117:6838/2378675949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f36a17f8b0 con 0x55f36a292610 2024-04-16T17:09:08.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.977+0000 7f6f50c00640 10 client.16134 handle_command_reply: tid=0 2024-04-16T17:09:08.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.977+0000 7f6f71008740 1 client.16134 shutdown 2024-04-16T17:09:08.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.977+0000 7f6f71008740 1 -- 172.21.15.45:0/2520174287 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f36a212d50 msgr2=0x55f36a224100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:08.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.977+0000 7f6f71008740 1 --2- 172.21.15.45:0/2520174287 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f36a212d50 0x55f36a224100 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f6f580bc4e0 tx=0x7f6f58000bc0 comp rx=0 tx=0).stop 2024-04-16T17:09:08.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.978+0000 7f6f71008740 1 -- 172.21.15.45:0/2520174287 shutdown_connections 2024-04-16T17:09:08.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.978+0000 7f6f71008740 1 --2- 172.21.15.45:0/2520174287 >> [v2:172.21.15.117:6838/2378675949,v1:172.21.15.117:6839/2378675949] conn(0x55f36a292610 0x55f36a294a00 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:08.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.978+0000 7f6f71008740 1 --2- 172.21.15.45:0/2520174287 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f36a213670 0x55f36a224640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:08.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.978+0000 7f6f71008740 1 --2- 172.21.15.45:0/2520174287 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f36a21c250 0x55f36a21f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:08.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.978+0000 7f6f71008740 1 --2- 172.21.15.45:0/2520174287 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f36a212d50 0x55f36a224100 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:08.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.978+0000 7f6f71008740 1 -- 172.21.15.45:0/2520174287 >> 172.21.15.45:0/2520174287 conn(0x55f36a20d480 msgr2=0x55f36a20c4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:09:08.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.978+0000 7f6f50c00640 0 client.16134 ms_handle_reset on v2:172.21.15.117:6838/2378675949 2024-04-16T17:09:08.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.978+0000 7f6f71008740 1 -- 172.21.15.45:0/2520174287 shutdown_connections 2024-04-16T17:09:08.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.978+0000 7f6f71008740 1 -- 172.21.15.45:0/2520174287 wait complete. 2024-04-16T17:09:08.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.978+0000 7f6f71008740 20 client.16134 trim_cache size 0 max 16384 2024-04-16T17:09:08.982 INFO:teuthology.orchestra.run.smithi045.stdout:[] 2024-04-16T17:09:08.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.978+0000 7f6f6e200640 1 -- 172.21.15.45:0/489729692 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f6f4c0a3d20 msgr2=0x7f6f4c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:08.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.978+0000 7f6f6e200640 1 --2- 172.21.15.45:0/489729692 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f6f4c0a3d20 0x7f6f4c0a61e0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f6f60009fd0 tx=0x7f6f60009500 comp rx=0 tx=0).stop 2024-04-16T17:09:08.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.978+0000 7f6f6e200640 1 -- 172.21.15.45:0/489729692 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6f681044b0 msgr2=0x7f6f681a07e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:08.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.978+0000 7f6f6e200640 1 --2- 172.21.15.45:0/489729692 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6f681044b0 0x7f6f681a07e0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f6f58036040 tx=0x7f6f5802cb20 comp rx=0 tx=0).stop 2024-04-16T17:09:08.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.979+0000 7f6f6e200640 1 -- 172.21.15.45:0/489729692 shutdown_connections 2024-04-16T17:09:08.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.979+0000 7f6f6e200640 1 --2- 172.21.15.45:0/489729692 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f6f4c0a3d20 0x7f6f4c0a61e0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:08.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.979+0000 7f6f6e200640 1 --2- 172.21.15.45:0/489729692 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6f6819a9e0 0x7f6f6819ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:08.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.979+0000 7f6f6e200640 1 --2- 172.21.15.45:0/489729692 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6f68109580 0x7f6f681a0d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:08.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.979+0000 7f6f6e200640 1 --2- 172.21.15.45:0/489729692 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6f681044b0 0x7f6f681a07e0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:08.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.979+0000 7f6f6e200640 1 -- 172.21.15.45:0/489729692 >> 172.21.15.45:0/489729692 conn(0x7f6f680ffd90 msgr2=0x7f6f681010a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:09:08.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.979+0000 7f6f6e200640 1 -- 172.21.15.45:0/489729692 shutdown_connections 2024-04-16T17:09:08.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:08.979+0000 7f6f6e200640 1 -- 172.21.15.45:0/489729692 wait complete. 2024-04-16T17:09:09.000 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:09:08 smithi045 ceph-mon[31676]: from='client.? 172.21.15.45:0/187334853' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:09:09.004 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-16T17:09:09.055 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:09:08 smithi117 ceph-mon[36730]: from='client.? 172.21.15.45:0/187334853' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:09:09.178 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:09:08 smithi104 ceph-mon[36457]: from='client.? 172.21.15.45:0/187334853' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:09:09.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.218+0000 7fa032400640 1 -- 172.21.15.45:0/625466627 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa02c0743d0 msgr2=0x7fa02c0747b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:09.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.218+0000 7fa032400640 1 --2- 172.21.15.45:0/625466627 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa02c0743d0 0x7fa02c0747b0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fa020009900 tx=0x7fa02002bcd0 comp rx=0 tx=0).stop 2024-04-16T17:09:09.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.218+0000 7fa032400640 1 -- 172.21.15.45:0/625466627 shutdown_connections 2024-04-16T17:09:09.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.218+0000 7fa032400640 1 --2- 172.21.15.45:0/625466627 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa02c074cf0 0x7fa02c10e200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:09.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.218+0000 7fa032400640 1 --2- 172.21.15.45:0/625466627 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa02c10e740 0x7fa02c110b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:09.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.218+0000 7fa032400640 1 --2- 172.21.15.45:0/625466627 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa02c0743d0 0x7fa02c0747b0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:09.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.218+0000 7fa032400640 1 -- 172.21.15.45:0/625466627 >> 172.21.15.45:0/625466627 conn(0x7fa02c0fd9d0 msgr2=0x7fa02c0ffdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:09:09.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.218+0000 7fa032400640 1 -- 172.21.15.45:0/625466627 shutdown_connections 2024-04-16T17:09:09.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.218+0000 7fa032400640 1 -- 172.21.15.45:0/625466627 wait complete. 2024-04-16T17:09:09.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.219+0000 7fa032400640 1 Processor -- start 2024-04-16T17:09:09.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.219+0000 7fa032400640 1 -- start start 2024-04-16T17:09:09.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.219+0000 7fa032400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa02c0743d0 0x7fa02c1a0620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:09.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.219+0000 7fa032400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa02c074cf0 0x7fa02c1a0b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:09.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.219+0000 7fa032400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa02c10e740 0x7fa02c19a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:09.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.219+0000 7fa032400640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fa02c114500 con 0x7fa02c10e740 2024-04-16T17:09:09.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.219+0000 7fa032400640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fa02c114380 con 0x7fa02c0743d0 2024-04-16T17:09:09.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.219+0000 7fa032400640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa02c114680 con 0x7fa02c074cf0 2024-04-16T17:09:09.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.220+0000 7fa031000640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa02c0743d0 0x7fa02c1a0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:09.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.220+0000 7fa031a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa02c10e740 0x7fa02c19a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:09.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.220+0000 7fa031000640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa02c0743d0 0x7fa02c1a0620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:47762/0 (socket says 172.21.15.45:47762) 2024-04-16T17:09:09.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.220+0000 7fa031000640 1 -- 172.21.15.45:0/1418461155 learned_addr learned my addr 172.21.15.45:0/1418461155 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:09:09.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.220+0000 7fa02be00640 1 --2- 172.21.15.45:0/1418461155 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa02c074cf0 0x7fa02c1a0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:09.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.220+0000 7fa031000640 1 -- 172.21.15.45:0/1418461155 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa02c074cf0 msgr2=0x7fa02c1a0b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:09.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.220+0000 7fa031000640 1 --2- 172.21.15.45:0/1418461155 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa02c074cf0 0x7fa02c1a0b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:09.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.220+0000 7fa031000640 1 -- 172.21.15.45:0/1418461155 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa02c10e740 msgr2=0x7fa02c19a7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:09.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.220+0000 7fa031000640 1 --2- 172.21.15.45:0/1418461155 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa02c10e740 0x7fa02c19a7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:09.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.220+0000 7fa031000640 1 -- 172.21.15.45:0/1418461155 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa02c19afb0 con 0x7fa02c0743d0 2024-04-16T17:09:09.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.221+0000 7fa031000640 1 --2- 172.21.15.45:0/1418461155 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa02c0743d0 0x7fa02c1a0620 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fa020000c00 tx=0x7fa0200043c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:09:09.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.221+0000 7fa029600640 1 -- 172.21.15.45:0/1418461155 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa020042020 con 0x7fa02c0743d0 2024-04-16T17:09:09.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.221+0000 7fa029600640 1 -- 172.21.15.45:0/1418461155 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa0200054e0 con 0x7fa02c0743d0 2024-04-16T17:09:09.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.221+0000 7fa032400640 1 -- 172.21.15.45:0/1418461155 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa02c19b240 con 0x7fa02c0743d0 2024-04-16T17:09:09.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.221+0000 7fa029600640 1 -- 172.21.15.45:0/1418461155 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa020003a80 con 0x7fa02c0743d0 2024-04-16T17:09:09.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.221+0000 7fa032400640 1 -- 172.21.15.45:0/1418461155 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa02c1a7470 con 0x7fa02c0743d0 2024-04-16T17:09:09.224 INFO:tasks.workunit.client.0.smithi045.stdout: 1 36062 2.99 MB/sec warmup 71 sec latency 1071.179 ms 2024-04-16T17:09:09.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.226+0000 7fa03662a740 1 Processor -- start 2024-04-16T17:09:09.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.226+0000 7fa03662a740 1 -- start start 2024-04-16T17:09:09.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.226+0000 7fa029600640 1 -- 172.21.15.45:0/1418461155 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fa020045020 con 0x7fa02c0743d0 2024-04-16T17:09:09.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.226+0000 7fa03662a740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5570a38fdf10 0x5570a38fe2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:09.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.226+0000 7fa03662a740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5570a38fe830 0x5570a3906ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:09.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.226+0000 7fa03662a740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5570a3907410 0x5570a3909800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:09.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.226+0000 7fa03662a740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5570a3863020 con 0x5570a38fe830 2024-04-16T17:09:09.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.226+0000 7fa03662a740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5570a3809460 con 0x5570a38fdf10 2024-04-16T17:09:09.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.226+0000 7fa03662a740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5570a385efe0 con 0x5570a3907410 2024-04-16T17:09:09.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.227+0000 7fa031000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5570a38fe830 0x5570a3906ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:09.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.227+0000 7fa031000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5570a38fe830 0x5570a3906ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45202/0 (socket says 172.21.15.45:45202) 2024-04-16T17:09:09.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.227+0000 7fa031000640 1 -- 172.21.15.45:0/975842612 learned_addr learned my addr 172.21.15.45:0/975842612 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:09:09.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.227+0000 7fa02be00640 1 --2- 172.21.15.45:0/975842612 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5570a3907410 0x5570a3909800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:09.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.227+0000 7fa031a00640 1 --2- 172.21.15.45:0/975842612 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5570a38fdf10 0x5570a38fe2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:09.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.227+0000 7fa031000640 1 -- 172.21.15.45:0/975842612 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5570a3907410 msgr2=0x5570a3909800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:09:09.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.227+0000 7fa031000640 1 --2- 172.21.15.45:0/975842612 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5570a3907410 0x5570a3909800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:09.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.227+0000 7fa031000640 1 -- 172.21.15.45:0/975842612 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5570a38fdf10 msgr2=0x5570a38fe2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:09:09.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.227+0000 7fa031000640 1 --2- 172.21.15.45:0/975842612 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5570a38fdf10 0x5570a38fe2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:09.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.227+0000 7fa031000640 1 -- 172.21.15.45:0/975842612 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5570a3811040 con 0x5570a38fe830 2024-04-16T17:09:09.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.228+0000 7fa031000640 1 --2- 172.21.15.45:0/975842612 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5570a38fe830 0x5570a3906ed0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fa0200efa70 tx=0x7fa0200f8b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee66338335eab38a server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:09:09.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.228+0000 7fa029600640 1 --2- 172.21.15.45:0/1418461155 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fa0140a3da0 0x7fa0140a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:09.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.228+0000 7fa029600640 1 -- 172.21.15.45:0/1418461155 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fa0200f2640 con 0x7fa02c0743d0 2024-04-16T17:09:09.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.228+0000 7fa01e000640 1 -- 172.21.15.45:0/975842612 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa020101070 con 0x5570a38fe830 2024-04-16T17:09:09.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.228+0000 7fa031a00640 1 --2- 172.21.15.45:0/1418461155 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fa0140a3da0 0x7fa0140a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:09.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.228+0000 7fa01e000640 1 -- 172.21.15.45:0/975842612 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa020005690 con 0x5570a38fe830 2024-04-16T17:09:09.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.228+0000 7fa01e000640 1 -- 172.21.15.45:0/975842612 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0200be150 con 0x5570a38fe830 2024-04-16T17:09:09.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.229+0000 7fa03662a740 1 -- 172.21.15.45:0/975842612 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5570a38fe830 msgr2=0x5570a3906ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:09.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.229+0000 7fa031a00640 1 --2- 172.21.15.45:0/1418461155 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fa0140a3da0 0x7fa0140a6260 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fa02c06b560 tx=0x7fa024009500 comp rx=0 tx=0).ready entity=mgr.25411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:09:09.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.229+0000 7fa03662a740 1 --2- 172.21.15.45:0/975842612 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5570a38fe830 0x5570a3906ed0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fa0200efa70 tx=0x7fa0200f8b20 comp rx=0 tx=0).stop 2024-04-16T17:09:09.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.229+0000 7fa03662a740 1 -- 172.21.15.45:0/975842612 shutdown_connections 2024-04-16T17:09:09.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.229+0000 7fa03662a740 1 --2- 172.21.15.45:0/975842612 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5570a3907410 0x5570a3909800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:09.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.229+0000 7fa03662a740 1 --2- 172.21.15.45:0/975842612 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5570a38fe830 0x5570a3906ed0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:09.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.229+0000 7fa03662a740 1 --2- 172.21.15.45:0/975842612 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5570a38fdf10 0x5570a38fe2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:09.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.229+0000 7fa03662a740 1 -- 172.21.15.45:0/975842612 >> 172.21.15.45:0/975842612 conn(0x5570a38f8640 msgr2=0x5570a38f8a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:09:09.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.229+0000 7fa03662a740 1 -- 172.21.15.45:0/975842612 shutdown_connections 2024-04-16T17:09:09.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.229+0000 7fa03662a740 1 -- 172.21.15.45:0/975842612 wait complete. 2024-04-16T17:09:09.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.230+0000 7fa03662a740 1 -- start start 2024-04-16T17:09:09.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.230+0000 7fa03662a740 1 Processor -- start 2024-04-16T17:09:09.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.231+0000 7fa03662a740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5570a38fdf10 0x5570a39043e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:09.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.231+0000 7fa03662a740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5570a38fe830 0x5570a3904920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:09.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.231+0000 7fa03662a740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5570a3907410 0x5570a3904e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:09.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.231+0000 7fa03662a740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5570a3909fc0 con 0x5570a38fe830 2024-04-16T17:09:09.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.231+0000 7fa03662a740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5570a381c830 con 0x5570a3907410 2024-04-16T17:09:09.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.231+0000 7fa03662a740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5570a390a140 con 0x5570a38fdf10 2024-04-16T17:09:09.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.231+0000 7fa03662a740 10 client.0 _ll_register_callbacks cb 0x5570a3824150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:09:09.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.231+0000 7fa02be00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5570a38fe830 0x5570a3904920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:09.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.231+0000 7fa02be00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5570a38fe830 0x5570a3904920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45216/0 (socket says 172.21.15.45:45216) 2024-04-16T17:09:09.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.231+0000 7fa02be00640 1 -- 172.21.15.45:0/3738107938 learned_addr learned my addr 172.21.15.45:0/3738107938 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:09:09.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.231+0000 7fa031000640 1 --2- 172.21.15.45:0/3738107938 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5570a38fdf10 0x5570a39043e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:09.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.231+0000 7fa031a00640 1 --2- 172.21.15.45:0/3738107938 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5570a3907410 0x5570a3904e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:09.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.232+0000 7fa031000640 1 -- 172.21.15.45:0/3738107938 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5570a3907410 msgr2=0x5570a3904e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:09:09.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.232+0000 7fa031000640 1 --2- 172.21.15.45:0/3738107938 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5570a3907410 0x5570a3904e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:09.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.232+0000 7fa031000640 1 -- 172.21.15.45:0/3738107938 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5570a38fe830 msgr2=0x5570a3904920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:09:09.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.232+0000 7fa031000640 1 --2- 172.21.15.45:0/3738107938 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5570a38fe830 0x5570a3904920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:09.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.232+0000 7fa031000640 1 -- 172.21.15.45:0/3738107938 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5570a3820920 con 0x5570a38fdf10 2024-04-16T17:09:09.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.232+0000 7fa031a00640 1 --2- 172.21.15.45:0/3738107938 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5570a3907410 0x5570a3904e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:09:09.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.232+0000 7fa02be00640 1 --2- 172.21.15.45:0/3738107938 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5570a38fe830 0x5570a3904920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:09:09.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.232+0000 7fa031000640 1 --2- 172.21.15.45:0/3738107938 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5570a38fdf10 0x5570a39043e0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fa0200fa670 tx=0x7fa0200f8300 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7728ae00c2ce91c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:09:09.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.232+0000 7fa012a00640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-04-16T17:09:09.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.232+0000 7fa012a00640 1 -- 172.21.15.45:0/3738107938 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0200f1070 con 0x5570a38fdf10 2024-04-16T17:09:09.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.232+0000 7fa012a00640 1 -- 172.21.15.45:0/3738107938 <== mon.1 v2:172.21.15.117:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fa0200f2270 con 0x5570a38fdf10 2024-04-16T17:09:09.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.232+0000 7fa03662a740 1 -- 172.21.15.45:0/3738107938 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5570a3870a40 con 0x5570a38fdf10 2024-04-16T17:09:09.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.232+0000 7fa012a00640 1 -- 172.21.15.45:0/3738107938 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fa020101030 con 0x5570a38fdf10 2024-04-16T17:09:09.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.232+0000 7fa03662a740 10 client.26272 fetch_fsmap learned FSMap version 29 2024-04-16T17:09:09.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.232+0000 7fa03662a740 1 -- 172.21.15.45:0/3738107938 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5570a37ee7f0 con 0x5570a38fdf10 2024-04-16T17:09:09.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.233+0000 7fa012a00640 1 -- 172.21.15.45:0/3738107938 <== mon.1 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x5570a3870a40 con 0x5570a38fdf10 2024-04-16T17:09:09.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.233+0000 7fa012a00640 1 -- 172.21.15.45:0/3738107938 <== mon.1 v2:172.21.15.117:3300/0 5 ==== fsmap(e 29) ==== 8366+0+0 (secure 0 0 0) 0x7fa0200d6070 con 0x5570a38fdf10 2024-04-16T17:09:09.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.233+0000 7fa03662a740 10 client.26272 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T17:09:09.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.233+0000 7fa03662a740 10 client.26272 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-04-16T17:09:09.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.233+0000 7fa03662a740 20 client.26272 populate_metadata read hostname 'smithi045' 2024-04-16T17:09:09.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.233+0000 7fa03662a740 1 --2- 172.21.15.45:0/3738107938 >> [v2:172.21.15.45:6840/3755310822,v1:172.21.15.45:6841/3755310822] conn(0x5570a3917340 0x5570a3919730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:09.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.233+0000 7fa03662a740 4 client.26272 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:09:09.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.233+0000 7fa03662a740 1 -- 172.21.15.45:0/3738107938 --> [v2:172.21.15.45:6840/3755310822,v1:172.21.15.45:6841/3755310822] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5570a386a8b0 con 0x5570a3917340 2024-04-16T17:09:09.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.233+0000 7fa02be00640 1 --2- 172.21.15.45:0/3738107938 >> [v2:172.21.15.45:6840/3755310822,v1:172.21.15.45:6841/3755310822] conn(0x5570a3917340 0x5570a3919730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:09.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.234+0000 7fa02be00640 1 --2- 172.21.15.45:0/3738107938 >> [v2:172.21.15.45:6840/3755310822,v1:172.21.15.45:6841/3755310822] conn(0x5570a3917340 0x5570a3919730 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1cede89e9acefa4a server_cookie=16c9e59666d8ba08 in_seq=0 out_seq=0 2024-04-16T17:09:09.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.234+0000 7fa012a00640 10 client.26272 ms_handle_connect on v2:172.21.15.45:6840/3755310822 2024-04-16T17:09:09.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.235+0000 7fa012a00640 1 -- 172.21.15.45:0/3738107938 <== mds.1 v2:172.21.15.45:6840/3755310822 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5570a386a8b0 con 0x5570a3917340 2024-04-16T17:09:09.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.235+0000 7fa012a00640 10 client.26272 handle_command_reply: tid=0 2024-04-16T17:09:09.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.235+0000 7fa03662a740 1 client.26272 shutdown 2024-04-16T17:09:09.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.235+0000 7fa03662a740 1 -- 172.21.15.45:0/3738107938 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5570a38fdf10 msgr2=0x5570a39043e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:09.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.235+0000 7fa03662a740 1 --2- 172.21.15.45:0/3738107938 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5570a38fdf10 0x5570a39043e0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fa0200fa670 tx=0x7fa0200f8300 comp rx=0 tx=0).stop 2024-04-16T17:09:09.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.235+0000 7fa03662a740 1 -- 172.21.15.45:0/3738107938 shutdown_connections 2024-04-16T17:09:09.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.235+0000 7fa03662a740 1 --2- 172.21.15.45:0/3738107938 >> [v2:172.21.15.45:6840/3755310822,v1:172.21.15.45:6841/3755310822] conn(0x5570a3917340 0x5570a3919730 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:09.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.235+0000 7fa03662a740 1 --2- 172.21.15.45:0/3738107938 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5570a3907410 0x5570a3904e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:09.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.235+0000 7fa03662a740 1 --2- 172.21.15.45:0/3738107938 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5570a38fe830 0x5570a3904920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:09.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.235+0000 7fa03662a740 1 --2- 172.21.15.45:0/3738107938 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5570a38fdf10 0x5570a39043e0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:09.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.235+0000 7fa012a00640 0 client.26272 ms_handle_reset on v2:172.21.15.45:6840/3755310822 2024-04-16T17:09:09.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.235+0000 7fa03662a740 1 -- 172.21.15.45:0/3738107938 >> 172.21.15.45:0/3738107938 conn(0x5570a38f8640 msgr2=0x5570a38fa390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:09:09.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.236+0000 7fa03662a740 1 -- 172.21.15.45:0/3738107938 shutdown_connections 2024-04-16T17:09:09.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.236+0000 7fa03662a740 1 -- 172.21.15.45:0/3738107938 wait complete. 2024-04-16T17:09:09.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.236+0000 7fa03662a740 20 client.26272 trim_cache size 0 max 16384 2024-04-16T17:09:09.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.265+0000 7fa03662a740 1 Processor -- start 2024-04-16T17:09:09.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.265+0000 7fa03662a740 1 -- start start 2024-04-16T17:09:09.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.265+0000 7fa03662a740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5570a38fdf10 0x5570a3906b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:09.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.265+0000 7fa03662a740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5570a38fe830 0x5570a3912e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:09.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.265+0000 7fa03662a740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5570a3907410 0x5570a3903540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:09.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.265+0000 7fa03662a740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fa02c1b3de0 con 0x5570a3907410 2024-04-16T17:09:09.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.265+0000 7fa03662a740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fa02c1b1620 con 0x5570a38fdf10 2024-04-16T17:09:09.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.265+0000 7fa03662a740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa02c1af450 con 0x5570a38fe830 2024-04-16T17:09:09.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.265+0000 7fa031a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5570a3907410 0x5570a3903540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:09.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.265+0000 7fa02be00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5570a38fe830 0x5570a3912e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:09.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.265+0000 7fa02be00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5570a38fe830 0x5570a3912e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:53564/0 (socket says 172.21.15.45:53564) 2024-04-16T17:09:09.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.266+0000 7fa02be00640 1 -- 172.21.15.45:0/2460353535 learned_addr learned my addr 172.21.15.45:0/2460353535 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:09:09.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.266+0000 7fa031000640 1 --2- 172.21.15.45:0/2460353535 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5570a38fdf10 0x5570a3906b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:09.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.266+0000 7fa02be00640 1 -- 172.21.15.45:0/2460353535 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5570a38fdf10 msgr2=0x5570a3906b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:09:09.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.266+0000 7fa02be00640 1 --2- 172.21.15.45:0/2460353535 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5570a38fdf10 0x5570a3906b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:09.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.266+0000 7fa02be00640 1 -- 172.21.15.45:0/2460353535 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5570a3907410 msgr2=0x5570a3903540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:09:09.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.266+0000 7fa02be00640 1 --2- 172.21.15.45:0/2460353535 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5570a3907410 0x5570a3903540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:09.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.266+0000 7fa02be00640 1 -- 172.21.15.45:0/2460353535 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5570a3919aa0 con 0x5570a38fe830 2024-04-16T17:09:09.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.266+0000 7fa02be00640 1 --2- 172.21.15.45:0/2460353535 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5570a38fe830 0x5570a3912e80 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fa018010540 tx=0x7fa018010ae0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d08b85ea2ddeda0b server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:09:09.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.266+0000 7fa01d600640 1 -- 172.21.15.45:0/2460353535 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa018005220 con 0x5570a38fe830 2024-04-16T17:09:09.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.266+0000 7fa01d600640 1 -- 172.21.15.45:0/2460353535 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa018024030 con 0x5570a38fe830 2024-04-16T17:09:09.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.266+0000 7fa01d600640 1 -- 172.21.15.45:0/2460353535 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa018020620 con 0x5570a38fe830 2024-04-16T17:09:09.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.267+0000 7fa03662a740 1 -- 172.21.15.45:0/2460353535 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5570a38fe830 msgr2=0x5570a3912e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:09.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.267+0000 7fa03662a740 1 --2- 172.21.15.45:0/2460353535 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5570a38fe830 0x5570a3912e80 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fa018010540 tx=0x7fa018010ae0 comp rx=0 tx=0).stop 2024-04-16T17:09:09.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.267+0000 7fa03662a740 1 -- 172.21.15.45:0/2460353535 shutdown_connections 2024-04-16T17:09:09.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.267+0000 7fa03662a740 1 --2- 172.21.15.45:0/2460353535 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5570a38fe830 0x5570a3912e80 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:09.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.267+0000 7fa03662a740 1 --2- 172.21.15.45:0/2460353535 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5570a3907410 0x5570a3903540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:09.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.267+0000 7fa03662a740 1 --2- 172.21.15.45:0/2460353535 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5570a38fdf10 0x5570a3906b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:09.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.267+0000 7fa03662a740 1 -- 172.21.15.45:0/2460353535 >> 172.21.15.45:0/2460353535 conn(0x5570a38f8640 msgr2=0x5570a38f6e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:09:09.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.268+0000 7fa03662a740 1 -- 172.21.15.45:0/2460353535 shutdown_connections 2024-04-16T17:09:09.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.268+0000 7fa03662a740 1 -- 172.21.15.45:0/2460353535 wait complete. 2024-04-16T17:09:09.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.268+0000 7fa03662a740 1 -- start start 2024-04-16T17:09:09.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.268+0000 7fa03662a740 1 Processor -- start 2024-04-16T17:09:09.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.269+0000 7fa03662a740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5570a38fdf10 0x5570a390a350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:09.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.269+0000 7fa03662a740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5570a38fe830 0x5570a390a890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:09.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.269+0000 7fa03662a740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5570a3907410 0x5570a397b820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:09.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.269+0000 7fa03662a740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fa02c1afae0 con 0x5570a38fdf10 2024-04-16T17:09:09.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.269+0000 7fa03662a740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fa02c1b0f50 con 0x5570a38fe830 2024-04-16T17:09:09.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.269+0000 7fa03662a740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa02c1b01b0 con 0x5570a3907410 2024-04-16T17:09:09.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.269+0000 7fa03662a740 10 client.0 _ll_register_callbacks cb 0x7fa024009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:09:09.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.269+0000 7fa031000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5570a38fdf10 0x5570a390a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:09.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.269+0000 7fa031000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5570a38fdf10 0x5570a390a350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45234/0 (socket says 172.21.15.45:45234) 2024-04-16T17:09:09.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.269+0000 7fa031000640 1 -- 172.21.15.45:0/2658856854 learned_addr learned my addr 172.21.15.45:0/2658856854 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:09:09.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.269+0000 7fa02be00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5570a38fe830 0x5570a390a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:09.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.269+0000 7fa031a00640 1 --2- 172.21.15.45:0/2658856854 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5570a3907410 0x5570a397b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:09.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.270+0000 7fa031000640 1 -- 172.21.15.45:0/2658856854 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5570a3907410 msgr2=0x5570a397b820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:09:09.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.270+0000 7fa031000640 1 --2- 172.21.15.45:0/2658856854 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5570a3907410 0x5570a397b820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:09.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.270+0000 7fa031000640 1 -- 172.21.15.45:0/2658856854 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5570a38fe830 msgr2=0x5570a390a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:09:09.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.270+0000 7fa031000640 1 --2- 172.21.15.45:0/2658856854 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5570a38fe830 0x5570a390a890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:09.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.270+0000 7fa031000640 1 -- 172.21.15.45:0/2658856854 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5570a35753f0 con 0x5570a38fdf10 2024-04-16T17:09:09.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.270+0000 7fa02be00640 1 --2- 172.21.15.45:0/2658856854 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5570a38fe830 0x5570a390a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:09:09.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.270+0000 7fa031000640 1 --2- 172.21.15.45:0/2658856854 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5570a38fdf10 0x5570a390a350 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fa02003c040 tx=0x7fa02002cb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52dc678c3cb42fb5 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:09:09.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.270+0000 7fa012a00640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-16T17:09:09.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.270+0000 7fa012a00640 1 -- 172.21.15.45:0/2658856854 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0200f1070 con 0x5570a38fdf10 2024-04-16T17:09:09.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.270+0000 7fa012a00640 1 -- 172.21.15.45:0/2658856854 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fa020003c20 con 0x5570a38fdf10 2024-04-16T17:09:09.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.270+0000 7fa03662a740 1 -- 172.21.15.45:0/2658856854 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa018024030 con 0x5570a38fdf10 2024-04-16T17:09:09.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.271+0000 7fa012a00640 1 -- 172.21.15.45:0/2658856854 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fa0200bdce0 con 0x5570a38fdf10 2024-04-16T17:09:09.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.271+0000 7fa03662a740 10 client.16152 fetch_fsmap learned FSMap version 29 2024-04-16T17:09:09.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.271+0000 7fa03662a740 1 -- 172.21.15.45:0/2658856854 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5570a363f7d0 con 0x5570a38fdf10 2024-04-16T17:09:09.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.271+0000 7fa012a00640 1 -- 172.21.15.45:0/2658856854 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa018024030 con 0x5570a38fdf10 2024-04-16T17:09:09.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.271+0000 7fa012a00640 1 -- 172.21.15.45:0/2658856854 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 29) ==== 8366+0+0 (secure 0 0 0) 0x7fa0200c6b60 con 0x5570a38fdf10 2024-04-16T17:09:09.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.271+0000 7fa03662a740 10 client.16152 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T17:09:09.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.271+0000 7fa03662a740 10 client.16152 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-04-16T17:09:09.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.271+0000 7fa03662a740 20 client.16152 populate_metadata read hostname 'smithi045' 2024-04-16T17:09:09.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.271+0000 7fa03662a740 1 --2- 172.21.15.45:0/2658856854 >> [v2:172.21.15.45:6840/3755310822,v1:172.21.15.45:6841/3755310822] conn(0x5570a397ce20 0x5570a397f210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:09.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.271+0000 7fa03662a740 4 client.16152 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T17:09:09.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.271+0000 7fa03662a740 1 -- 172.21.15.45:0/2658856854 --> [v2:172.21.15.45:6840/3755310822,v1:172.21.15.45:6841/3755310822] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5570a37e6570 con 0x5570a397ce20 2024-04-16T17:09:09.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.271+0000 7fa02be00640 1 --2- 172.21.15.45:0/2658856854 >> [v2:172.21.15.45:6840/3755310822,v1:172.21.15.45:6841/3755310822] conn(0x5570a397ce20 0x5570a397f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:09.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.272+0000 7fa02be00640 1 --2- 172.21.15.45:0/2658856854 >> [v2:172.21.15.45:6840/3755310822,v1:172.21.15.45:6841/3755310822] conn(0x5570a397ce20 0x5570a397f210 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f251d8faedb4f440 server_cookie=478e3099f1fa2a83 in_seq=0 out_seq=0 2024-04-16T17:09:09.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.272+0000 7fa012a00640 10 client.16152 ms_handle_connect on v2:172.21.15.45:6840/3755310822 2024-04-16T17:09:09.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.272+0000 7fa012a00640 1 -- 172.21.15.45:0/2658856854 <== mds.1 v2:172.21.15.45:6840/3755310822 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5570a37e6570 con 0x5570a397ce20 2024-04-16T17:09:09.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.272+0000 7fa012a00640 10 client.16152 handle_command_reply: tid=0 2024-04-16T17:09:09.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.272+0000 7fa03662a740 1 client.16152 shutdown 2024-04-16T17:09:09.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.273+0000 7fa03662a740 1 -- 172.21.15.45:0/2658856854 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5570a38fdf10 msgr2=0x5570a390a350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:09.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.273+0000 7fa03662a740 1 --2- 172.21.15.45:0/2658856854 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5570a38fdf10 0x5570a390a350 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fa02003c040 tx=0x7fa02002cb30 comp rx=0 tx=0).stop 2024-04-16T17:09:09.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.273+0000 7fa03662a740 1 -- 172.21.15.45:0/2658856854 shutdown_connections 2024-04-16T17:09:09.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.273+0000 7fa03662a740 1 --2- 172.21.15.45:0/2658856854 >> [v2:172.21.15.45:6840/3755310822,v1:172.21.15.45:6841/3755310822] conn(0x5570a397ce20 0x5570a397f210 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:09.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.273+0000 7fa03662a740 1 --2- 172.21.15.45:0/2658856854 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5570a3907410 0x5570a397b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:09.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.273+0000 7fa03662a740 1 --2- 172.21.15.45:0/2658856854 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5570a38fe830 0x5570a390a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:09.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.273+0000 7fa03662a740 1 --2- 172.21.15.45:0/2658856854 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5570a38fdf10 0x5570a390a350 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:09.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.273+0000 7fa012a00640 0 client.16152 ms_handle_reset on v2:172.21.15.45:6840/3755310822 2024-04-16T17:09:09.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.273+0000 7fa03662a740 1 -- 172.21.15.45:0/2658856854 >> 172.21.15.45:0/2658856854 conn(0x5570a38f8640 msgr2=0x5570a39033c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:09:09.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.273+0000 7fa03662a740 1 -- 172.21.15.45:0/2658856854 shutdown_connections 2024-04-16T17:09:09.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.273+0000 7fa03662a740 1 -- 172.21.15.45:0/2658856854 wait complete. 2024-04-16T17:09:09.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.273+0000 7fa03662a740 20 client.16152 trim_cache size 0 max 16384 2024-04-16T17:09:09.277 INFO:teuthology.orchestra.run.smithi045.stdout:[] 2024-04-16T17:09:09.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.274+0000 7fa032400640 1 -- 172.21.15.45:0/1418461155 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fa0140a3da0 msgr2=0x7fa0140a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:09.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.274+0000 7fa032400640 1 --2- 172.21.15.45:0/1418461155 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fa0140a3da0 0x7fa0140a6260 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fa02c06b560 tx=0x7fa024009500 comp rx=0 tx=0).stop 2024-04-16T17:09:09.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.274+0000 7fa032400640 1 -- 172.21.15.45:0/1418461155 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa02c0743d0 msgr2=0x7fa02c1a0620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:09.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.274+0000 7fa032400640 1 --2- 172.21.15.45:0/1418461155 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa02c0743d0 0x7fa02c1a0620 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fa020000c00 tx=0x7fa0200043c0 comp rx=0 tx=0).stop 2024-04-16T17:09:09.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.274+0000 7fa032400640 1 -- 172.21.15.45:0/1418461155 shutdown_connections 2024-04-16T17:09:09.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.274+0000 7fa032400640 1 --2- 172.21.15.45:0/1418461155 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fa0140a3da0 0x7fa0140a6260 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:09.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.274+0000 7fa032400640 1 --2- 172.21.15.45:0/1418461155 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa02c074cf0 0x7fa02c1a0b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:09.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.274+0000 7fa032400640 1 --2- 172.21.15.45:0/1418461155 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa02c10e740 0x7fa02c19a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:09.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.274+0000 7fa032400640 1 --2- 172.21.15.45:0/1418461155 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa02c0743d0 0x7fa02c1a0620 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:09.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.274+0000 7fa032400640 1 -- 172.21.15.45:0/1418461155 >> 172.21.15.45:0/1418461155 conn(0x7fa02c0fd9d0 msgr2=0x7fa02c0fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:09:09.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.274+0000 7fa032400640 1 -- 172.21.15.45:0/1418461155 shutdown_connections 2024-04-16T17:09:09.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.274+0000 7fa032400640 1 -- 172.21.15.45:0/1418461155 wait complete. 2024-04-16T17:09:09.293 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-16T17:09:09.434 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:09.434+0000 7f895b000640 1 -- 172.21.15.45:0/2527138930 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f895410c3e0 msgr2=0x7f895410e880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:09.434 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:09.434+0000 7f895b000640 1 --2- 172.21.15.45:0/2527138930 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f895410c3e0 0x7f895410e880 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f894c009ca0 tx=0x7f894c02c190 comp rx=0 tx=0).stop 2024-04-16T17:09:09.434 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:09.434+0000 7f895b000640 1 -- 172.21.15.45:0/2527138930 shutdown_connections 2024-04-16T17:09:09.434 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:09.434+0000 7f895b000640 1 --2- 172.21.15.45:0/2527138930 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f895410c3e0 0x7f895410e880 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:09.434 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:09.434+0000 7f895b000640 1 --2- 172.21.15.45:0/2527138930 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f89541038e0 0x7f895410bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:09.435 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:09.434+0000 7f895b000640 1 --2- 172.21.15.45:0/2527138930 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8954102fc0 0x7f89541033a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:09.435 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:09.434+0000 7f895b000640 1 -- 172.21.15.45:0/2527138930 >> 172.21.15.45:0/2527138930 conn(0x7f89540fc820 msgr2=0x7f89540fec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:09:09.435 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:09.434+0000 7f895b000640 1 -- 172.21.15.45:0/2527138930 shutdown_connections 2024-04-16T17:09:09.435 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:09.434+0000 7f895b000640 1 -- 172.21.15.45:0/2527138930 wait complete. 2024-04-16T17:09:09.436 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:09.435+0000 7f895b000640 1 Processor -- start 2024-04-16T17:09:09.436 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:09.435+0000 7f895b000640 1 -- start start 2024-04-16T17:09:09.436 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:09.436+0000 7f895b000640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8954102fc0 0x7f89541a0620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:09.436 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:09.436+0000 7f895b000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f89541038e0 0x7f89541a0b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:09.436 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:09.436+0000 7f895b000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f895410c3e0 0x7f895419a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:09.436 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:09.436+0000 7f895b000640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f89541121c0 con 0x7f895410c3e0 2024-04-16T17:09:09.436 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:09.436+0000 7f895b000640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f8954112040 con 0x7f8954102fc0 2024-04-16T17:09:09.436 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:09.436+0000 7f895b000640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f8954112340 con 0x7f89541038e0 2024-04-16T17:09:09.437 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:09.436+0000 7f8959c00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8954102fc0 0x7f89541a0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:09.437 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:09.436+0000 7f895a600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f895410c3e0 0x7f895419a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:09.437 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:09.436+0000 7f8959c00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8954102fc0 0x7f89541a0620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:47838/0 (socket says 172.21.15.45:47838) 2024-04-16T17:09:09.438 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:09.436+0000 7f8959c00640 1 -- 172.21.15.45:0/1210052558 learned_addr learned my addr 172.21.15.45:0/1210052558 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:09:09.438 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:09.436+0000 7f8959200640 1 --2- 172.21.15.45:0/1210052558 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f89541038e0 0x7f89541a0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:09.438 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:09.437+0000 7f8959c00640 1 -- 172.21.15.45:0/1210052558 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f89541038e0 msgr2=0x7f89541a0b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:09.438 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:09.437+0000 7f8959c00640 1 --2- 172.21.15.45:0/1210052558 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f89541038e0 0x7f89541a0b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:09.438 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:09.437+0000 7f8959c00640 1 -- 172.21.15.45:0/1210052558 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f895410c3e0 msgr2=0x7f895419a880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:09.438 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:09.437+0000 7f8959c00640 1 --2- 172.21.15.45:0/1210052558 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f895410c3e0 0x7f895419a880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:09.438 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:09.437+0000 7f8959c00640 1 -- 172.21.15.45:0/1210052558 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f895419afb0 con 0x7f8954102fc0 2024-04-16T17:09:09.438 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:09.437+0000 7f8959c00640 1 --2- 172.21.15.45:0/1210052558 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8954102fc0 0x7f89541a0620 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f894800e9f0 tx=0x7f894800edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:09:09.438 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:09.437+0000 7f8952000640 1 -- 172.21.15.45:0/1210052558 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f894800cd80 con 0x7f8954102fc0 2024-04-16T17:09:09.438 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:09.438+0000 7f895b000640 1 -- 172.21.15.45:0/1210052558 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f895419b2a0 con 0x7f8954102fc0 2024-04-16T17:09:09.438 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:09.438+0000 7f8952000640 1 -- 172.21.15.45:0/1210052558 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8948005500 con 0x7f8954102fc0 2024-04-16T17:09:09.438 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:09.438+0000 7f8952000640 1 -- 172.21.15.45:0/1210052558 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8948010690 con 0x7f8954102fc0 2024-04-16T17:09:09.438 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:09.438+0000 7f895b000640 1 -- 172.21.15.45:0/1210052558 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89541a7470 con 0x7f8954102fc0 2024-04-16T17:09:09.441 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:09.441+0000 7f895f3b2740 1 Processor -- start 2024-04-16T17:09:09.442 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:09.442+0000 7f895f3b2740 1 -- start start 2024-04-16T17:09:09.442 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:09.442+0000 7f895f3b2740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ce1061f7f0 0x55ce1061e540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:09.442 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:09.442+0000 7f895f3b2740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ce1061ea80 0x55ce1061ef00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:09.442 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:09.442+0000 7f895f3b2740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ce10628000 0x55ce1062a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:09.442 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:09.442+0000 7f895f3b2740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55ce10583780 con 0x55ce10628000 2024-04-16T17:09:09.442 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:09.442+0000 7f895f3b2740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55ce10529f40 con 0x55ce1061f7f0 2024-04-16T17:09:09.442 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:09.442+0000 7f895f3b2740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55ce1057f740 con 0x55ce1061ea80 2024-04-16T17:09:09.443 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:09.442+0000 7f8952000640 1 -- 172.21.15.45:0/1210052558 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f8948010830 con 0x7f8954102fc0 2024-04-16T17:09:09.444 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:09.443+0000 7f895a600640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ce1061f7f0 0x55ce1061e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:09.444 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:09.443+0000 7f895a600640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ce1061f7f0 0x55ce1061e540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:47848/0 (socket says 172.21.15.45:47848) 2024-04-16T17:09:09.444 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:09.443+0000 7f8959200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ce10628000 0x55ce1062a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:09.444 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:09.443+0000 7f895a600640 1 -- 172.21.15.45:0/1993674871 learned_addr learned my addr 172.21.15.45:0/1993674871 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:09:09.444 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:09.443+0000 7f8959c00640 1 --2- 172.21.15.45:0/1993674871 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ce1061ea80 0x55ce1061ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:09.444 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:09.443+0000 7f895a600640 1 -- 172.21.15.45:0/1993674871 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ce1061ea80 msgr2=0x55ce1061ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:09:09.444 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:09.443+0000 7f895a600640 1 --2- 172.21.15.45:0/1993674871 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ce1061ea80 0x55ce1061ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:09.444 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:09.443+0000 7f895a600640 1 -- 172.21.15.45:0/1993674871 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ce10628000 msgr2=0x55ce1062a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:09:09.444 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:09.443+0000 7f895a600640 1 --2- 172.21.15.45:0/1993674871 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ce10628000 0x55ce1062a3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:09.444 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:09.443+0000 7f895a600640 1 -- 172.21.15.45:0/1993674871 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ce10296210 con 0x55ce1061f7f0 2024-04-16T17:09:09.444 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:09.443+0000 7f895a600640 1 --2- 172.21.15.45:0/1993674871 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ce1061f7f0 0x55ce1061e540 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f894c02c7a0 tx=0x7f894c005f80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=596bae7af36ac2e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:09:09.445 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:09.444+0000 7f8946a00640 1 -- 172.21.15.45:0/1993674871 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f894c043070 con 0x55ce1061f7f0 2024-04-16T17:09:09.445 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:09.444+0000 7f8946a00640 1 -- 172.21.15.45:0/1993674871 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f894c003e50 con 0x55ce1061f7f0 2024-04-16T17:09:09.445 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:09.444+0000 7f8946a00640 1 -- 172.21.15.45:0/1993674871 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f894c039040 con 0x55ce1061f7f0 2024-04-16T17:09:09.445 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:09.444+0000 7f8952000640 1 --2- 172.21.15.45:0/1210052558 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f893c0a3d10 0x7f893c0a61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:09.446 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:09.444+0000 7f8952000640 1 -- 172.21.15.45:0/1210052558 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f89480d0fa0 con 0x7f8954102fc0 2024-04-16T17:09:09.446 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:09.445+0000 7f8959c00640 1 --2- 172.21.15.45:0/1210052558 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f893c0a3d10 0x7f893c0a61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:09.446 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:09.445+0000 7f895f3b2740 1 -- 172.21.15.45:0/1993674871 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ce1061f7f0 msgr2=0x55ce1061e540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:09.446 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:09.445+0000 7f895f3b2740 1 --2- 172.21.15.45:0/1993674871 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ce1061f7f0 0x55ce1061e540 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f894c02c7a0 tx=0x7f894c005f80 comp rx=0 tx=0).stop 2024-04-16T17:09:09.446 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:09.445+0000 7f895f3b2740 1 -- 172.21.15.45:0/1993674871 shutdown_connections 2024-04-16T17:09:09.447 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:09.445+0000 7f895f3b2740 1 --2- 172.21.15.45:0/1993674871 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ce1061ea80 0x55ce1061ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:09.447 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:09.445+0000 7f895f3b2740 1 --2- 172.21.15.45:0/1993674871 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ce10628000 0x55ce1062a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:09.447 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:09.445+0000 7f895f3b2740 1 --2- 172.21.15.45:0/1993674871 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ce1061f7f0 0x55ce1061e540 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:09.447 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:09.445+0000 7f895f3b2740 1 -- 172.21.15.45:0/1993674871 >> 172.21.15.45:0/1993674871 conn(0x55ce105b95d0 msgr2=0x55ce10619770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:09:09.447 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:09.445+0000 7f895f3b2740 1 -- 172.21.15.45:0/1993674871 shutdown_connections 2024-04-16T17:09:09.447 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:09.445+0000 7f895f3b2740 1 -- 172.21.15.45:0/1993674871 wait complete. 2024-04-16T17:09:09.447 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:09.445+0000 7f8959c00640 1 --2- 172.21.15.45:0/1210052558 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f893c0a3d10 0x7f893c0a61d0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x55ce1061ff50 tx=0x7f89480d4a50 comp rx=0 tx=0).ready entity=mgr.25411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:09:09.447 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:09.446+0000 7f895f3b2740 1 -- start start 2024-04-16T17:09:09.447 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:09.446+0000 7f895f3b2740 1 Processor -- start 2024-04-16T17:09:09.450 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:09.447+0000 7f895f3b2740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ce1061ea80 0x55ce10627a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:09.450 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:09.447+0000 7f895f3b2740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ce1061f7f0 0x55ce10624e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:09.450 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:09.447+0000 7f895f3b2740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ce10628000 0x55ce10625550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:09.450 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:09.447+0000 7f895f3b2740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55ce1062acd0 con 0x55ce10628000 2024-04-16T17:09:09.450 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:09.447+0000 7f895f3b2740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55ce1053d390 con 0x55ce1061f7f0 2024-04-16T17:09:09.450 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:09.447+0000 7f895f3b2740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55ce1062ae50 con 0x55ce1061ea80 2024-04-16T17:09:09.450 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:09.447+0000 7f895f3b2740 10 client.0 _ll_register_callbacks cb 0x55ce10526f60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:09:09.450 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:09.447+0000 7f895a600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ce10628000 0x55ce10625550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:09.450 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:09.447+0000 7f895a600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ce10628000 0x55ce10625550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45258/0 (socket says 172.21.15.45:45258) 2024-04-16T17:09:09.450 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:09.447+0000 7f895a600640 1 -- 172.21.15.45:0/2023869689 learned_addr learned my addr 172.21.15.45:0/2023869689 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:09:09.450 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:09.447+0000 7f895a600640 1 --2- 172.21.15.45:0/2023869689 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ce1061ea80 0x55ce10627a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:09.450 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:09.448+0000 7f8959200640 1 --2- 172.21.15.45:0/2023869689 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ce1061f7f0 0x55ce10624e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:09.450 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:09.448+0000 7f895a600640 1 -- 172.21.15.45:0/2023869689 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ce1061ea80 msgr2=0x55ce10627a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:09:09.451 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:09.448+0000 7f895a600640 1 --2- 172.21.15.45:0/2023869689 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ce1061ea80 0x55ce10627a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:09.451 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:09.448+0000 7f895a600640 1 -- 172.21.15.45:0/2023869689 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ce1061f7f0 msgr2=0x55ce10624e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:09:09.451 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:09.448+0000 7f895a600640 1 --2- 172.21.15.45:0/2023869689 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ce1061f7f0 0x55ce10624e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:09.451 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:09.448+0000 7f895a600640 1 -- 172.21.15.45:0/2023869689 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ce10362620 con 0x55ce10628000 2024-04-16T17:09:09.451 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:09.448+0000 7f895a600640 1 --2- 172.21.15.45:0/2023869689 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ce10628000 0x55ce10625550 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f894c004780 tx=0x7f894c02e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=479cda627718d807 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:09:09.451 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:09.448+0000 7f893b400640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-16T17:09:09.451 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:09.448+0000 7f893b400640 1 -- 172.21.15.45:0/2023869689 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f894c039030 con 0x55ce10628000 2024-04-16T17:09:09.451 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:09.449+0000 7f893b400640 1 -- 172.21.15.45:0/2023869689 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f894c003970 con 0x55ce10628000 2024-04-16T17:09:09.451 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:09.449+0000 7f895f3b2740 1 -- 172.21.15.45:0/2023869689 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f894c003e50 con 0x55ce10628000 2024-04-16T17:09:09.451 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:09.449+0000 7f893b400640 1 -- 172.21.15.45:0/2023869689 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f894c05c050 con 0x55ce10628000 2024-04-16T17:09:09.451 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:09.449+0000 7f895f3b2740 10 client.16164 fetch_fsmap learned FSMap version 29 2024-04-16T17:09:09.451 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:09.449+0000 7f895f3b2740 1 -- 172.21.15.45:0/2023869689 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55ce10636da0 con 0x55ce10628000 2024-04-16T17:09:09.451 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:09.449+0000 7f893b400640 1 -- 172.21.15.45:0/2023869689 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f894c003e50 con 0x55ce10628000 2024-04-16T17:09:09.451 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:09.449+0000 7f893b400640 1 -- 172.21.15.45:0/2023869689 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 29) ==== 8366+0+0 (secure 0 0 0) 0x7f894c048550 con 0x55ce10628000 2024-04-16T17:09:09.452 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:09.449+0000 7f895f3b2740 10 client.16164 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T17:09:09.452 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:09.449+0000 7f895f3b2740 10 client.16164 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T17:09:09.452 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:09.450+0000 7f895f3b2740 20 client.16164 populate_metadata read hostname 'smithi045' 2024-04-16T17:09:09.452 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:09.450+0000 7f895f3b2740 1 --2- 172.21.15.45:0/2023869689 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55ce106380f0 0x55ce1063a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:09.452 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:09.450+0000 7f895f3b2740 4 client.16164 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:09:09.452 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:09.450+0000 7f895f3b2740 1 -- 172.21.15.45:0/2023869689 --> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ce105911a0 con 0x55ce106380f0 2024-04-16T17:09:09.452 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:09.450+0000 7f8959200640 1 --2- 172.21.15.45:0/2023869689 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55ce106380f0 0x55ce1063a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:09.453 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:09.451+0000 7f8959200640 1 --2- 172.21.15.45:0/2023869689 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55ce106380f0 0x55ce1063a4e0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=84bb6e04477c2cd9 server_cookie=8f6c667cedc9f5cb in_seq=0 out_seq=0 2024-04-16T17:09:09.453 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:09.451+0000 7f893b400640 10 client.16164 ms_handle_connect on v2:172.21.15.104:6838/3660070592 2024-04-16T17:09:09.453 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:09.451+0000 7f893b400640 1 -- 172.21.15.45:0/2023869689 <== mds.0 v2:172.21.15.104:6838/3660070592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ce105911a0 con 0x55ce106380f0 2024-04-16T17:09:09.453 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:09.451+0000 7f893b400640 10 client.16164 handle_command_reply: tid=0 2024-04-16T17:09:09.453 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:09.451+0000 7f895f3b2740 1 client.16164 shutdown 2024-04-16T17:09:09.454 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:09.452+0000 7f895f3b2740 1 -- 172.21.15.45:0/2023869689 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ce10628000 msgr2=0x55ce10625550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:09.454 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:09.452+0000 7f895f3b2740 1 --2- 172.21.15.45:0/2023869689 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ce10628000 0x55ce10625550 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f894c004780 tx=0x7f894c02e040 comp rx=0 tx=0).stop 2024-04-16T17:09:09.454 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:09.452+0000 7f895f3b2740 1 -- 172.21.15.45:0/2023869689 shutdown_connections 2024-04-16T17:09:09.454 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:09.452+0000 7f895f3b2740 1 --2- 172.21.15.45:0/2023869689 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ce10628000 0x55ce10625550 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:09.454 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:09.452+0000 7f895f3b2740 1 --2- 172.21.15.45:0/2023869689 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ce1061f7f0 0x55ce10624e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:09.454 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:09.452+0000 7f895f3b2740 1 --2- 172.21.15.45:0/2023869689 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55ce106380f0 0x55ce1063a4e0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:09.454 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:09.452+0000 7f895f3b2740 1 --2- 172.21.15.45:0/2023869689 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ce1061ea80 0x55ce10627a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:09.454 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:09.452+0000 7f895f3b2740 1 -- 172.21.15.45:0/2023869689 >> 172.21.15.45:0/2023869689 conn(0x55ce105b95d0 msgr2=0x55ce1061b4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:09:09.454 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:09.452+0000 7f893b400640 0 client.16164 ms_handle_reset on v2:172.21.15.104:6838/3660070592 2024-04-16T17:09:09.454 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:09.452+0000 7f895f3b2740 1 -- 172.21.15.45:0/2023869689 shutdown_connections 2024-04-16T17:09:09.454 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:09.452+0000 7f895f3b2740 1 -- 172.21.15.45:0/2023869689 wait complete. 2024-04-16T17:09:09.454 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:09.452+0000 7f895f3b2740 20 client.16164 trim_cache size 0 max 16384 2024-04-16T17:09:09.482 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:09.481+0000 7f895f3b2740 1 Processor -- start 2024-04-16T17:09:09.482 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:09.481+0000 7f895f3b2740 1 -- start start 2024-04-16T17:09:09.482 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:09.481+0000 7f895f3b2740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ce1061ea80 0x55ce10633bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:09.482 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:09.481+0000 7f895f3b2740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ce1061f7f0 0x55ce10623e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:09.482 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:09.481+0000 7f895f3b2740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ce10628000 0x55ce10627540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:09.482 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:09.482+0000 7f895f3b2740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f89541b3dc0 con 0x55ce1061f7f0 2024-04-16T17:09:09.482 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:09.482+0000 7f895f3b2740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f89541b1600 con 0x55ce10628000 2024-04-16T17:09:09.482 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:09.482+0000 7f895f3b2740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f89541af430 con 0x55ce1061ea80 2024-04-16T17:09:09.482 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:09.482+0000 7f8959200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ce1061f7f0 0x55ce10623e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:09.483 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:09.482+0000 7f895a600640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ce1061ea80 0x55ce10633bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:09.483 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:09.482+0000 7f8959200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ce1061f7f0 0x55ce10623e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45260/0 (socket says 172.21.15.45:45260) 2024-04-16T17:09:09.483 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:09.482+0000 7f8959200640 1 -- 172.21.15.45:0/983169735 learned_addr learned my addr 172.21.15.45:0/983169735 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:09:09.483 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:09.482+0000 7f895a600640 1 --2- 172.21.15.45:0/983169735 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ce10628000 0x55ce10627540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:09.484 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:09.482+0000 7f8959200640 1 -- 172.21.15.45:0/983169735 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ce1061ea80 msgr2=0x55ce10633bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:09:09.484 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:09.482+0000 7f8959200640 1 --2- 172.21.15.45:0/983169735 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ce1061ea80 0x55ce10633bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:09.484 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:09.482+0000 7f8959200640 1 -- 172.21.15.45:0/983169735 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ce10628000 msgr2=0x55ce10627540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:09:09.484 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:09.482+0000 7f8959200640 1 --2- 172.21.15.45:0/983169735 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ce10628000 0x55ce10627540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:09.485 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:09.483+0000 7f8959200640 1 -- 172.21.15.45:0/983169735 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ce1061dd50 con 0x55ce1061f7f0 2024-04-16T17:09:09.485 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:09.483+0000 7f8959200640 1 --2- 172.21.15.45:0/983169735 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ce1061f7f0 0x55ce10623e30 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f8940009dc0 tx=0x7f8940005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54a65e8b9a7a1440 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:09:09.485 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:09.483+0000 7f8946000640 1 -- 172.21.15.45:0/983169735 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8940024030 con 0x55ce1061f7f0 2024-04-16T17:09:09.485 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:09.483+0000 7f8946000640 1 -- 172.21.15.45:0/983169735 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8940020070 con 0x55ce1061f7f0 2024-04-16T17:09:09.485 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:09.483+0000 7f8946000640 1 -- 172.21.15.45:0/983169735 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8940010690 con 0x55ce1061f7f0 2024-04-16T17:09:09.485 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:09.484+0000 7f895f3b2740 1 -- 172.21.15.45:0/983169735 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ce1061f7f0 msgr2=0x55ce10623e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:09.485 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:09.484+0000 7f895f3b2740 1 --2- 172.21.15.45:0/983169735 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ce1061f7f0 0x55ce10623e30 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f8940009dc0 tx=0x7f8940005f90 comp rx=0 tx=0).stop 2024-04-16T17:09:09.485 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:09.484+0000 7f895f3b2740 1 -- 172.21.15.45:0/983169735 shutdown_connections 2024-04-16T17:09:09.485 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:09.484+0000 7f895f3b2740 1 --2- 172.21.15.45:0/983169735 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ce10628000 0x55ce10627540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:09.485 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:09.484+0000 7f895f3b2740 1 --2- 172.21.15.45:0/983169735 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ce1061f7f0 0x55ce10623e30 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:09.485 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:09.484+0000 7f895f3b2740 1 --2- 172.21.15.45:0/983169735 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ce1061ea80 0x55ce10633bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:09.485 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:09.484+0000 7f895f3b2740 1 -- 172.21.15.45:0/983169735 >> 172.21.15.45:0/983169735 conn(0x55ce105b95d0 msgr2=0x55ce10623720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:09:09.485 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:09.484+0000 7f895f3b2740 1 -- 172.21.15.45:0/983169735 shutdown_connections 2024-04-16T17:09:09.485 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:09.485+0000 7f895f3b2740 1 -- 172.21.15.45:0/983169735 wait complete. 2024-04-16T17:09:09.488 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:09.485+0000 7f895f3b2740 1 -- start start 2024-04-16T17:09:09.488 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:09.485+0000 7f895f3b2740 1 Processor -- start 2024-04-16T17:09:09.488 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:09.486+0000 7f895f3b2740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ce1061ea80 0x55ce10621d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:09.489 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:09.486+0000 7f895f3b2740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ce1061f7f0 0x55ce10622250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:09.489 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:09.486+0000 7f895f3b2740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ce10628000 0x55ce10622790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:09.489 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:09.486+0000 7f895f3b2740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f89541afac0 con 0x55ce1061ea80 2024-04-16T17:09:09.489 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:09.486+0000 7f895f3b2740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f89541b0f30 con 0x55ce10628000 2024-04-16T17:09:09.489 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:09.486+0000 7f895f3b2740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f89541b0190 con 0x55ce1061f7f0 2024-04-16T17:09:09.489 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:09.486+0000 7f895f3b2740 10 client.0 _ll_register_callbacks cb 0x7f8940009790 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:09:09.489 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:09.486+0000 7f895a600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ce1061ea80 0x55ce10621d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:09.489 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:09.486+0000 7f895a600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ce1061ea80 0x55ce10621d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45270/0 (socket says 172.21.15.45:45270) 2024-04-16T17:09:09.489 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:09.486+0000 7f895a600640 1 -- 172.21.15.45:0/161967281 learned_addr learned my addr 172.21.15.45:0/161967281 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:09:09.489 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:09.486+0000 7f8959200640 1 --2- 172.21.15.45:0/161967281 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ce1061f7f0 0x55ce10622250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:09.489 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:09.486+0000 7f895a600640 1 --2- 172.21.15.45:0/161967281 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ce10628000 0x55ce10622790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:09.489 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:09.486+0000 7f895a600640 1 -- 172.21.15.45:0/161967281 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ce1061f7f0 msgr2=0x55ce10622250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:09:09.490 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:09.486+0000 7f895a600640 1 --2- 172.21.15.45:0/161967281 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ce1061f7f0 0x55ce10622250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:09.490 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:09.486+0000 7f895a600640 1 -- 172.21.15.45:0/161967281 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ce10628000 msgr2=0x55ce10622790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:09:09.490 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:09.486+0000 7f895a600640 1 --2- 172.21.15.45:0/161967281 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ce10628000 0x55ce10622790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:09.490 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:09.486+0000 7f895a600640 1 -- 172.21.15.45:0/161967281 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ce1069d5b0 con 0x55ce1061ea80 2024-04-16T17:09:09.490 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:09.487+0000 7f895a600640 1 --2- 172.21.15.45:0/161967281 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ce1061ea80 0x55ce10621d10 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f894c009430 tx=0x7f894c004680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=212473ae3bbc1f97 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:09:09.490 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:09.487+0000 7f893b400640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-16T17:09:09.490 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:09.487+0000 7f893b400640 1 -- 172.21.15.45:0/161967281 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f894c039040 con 0x55ce1061ea80 2024-04-16T17:09:09.490 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:09.487+0000 7f893b400640 1 -- 172.21.15.45:0/161967281 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f894c048be0 con 0x55ce1061ea80 2024-04-16T17:09:09.490 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:09.487+0000 7f895f3b2740 1 -- 172.21.15.45:0/161967281 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8940020070 con 0x55ce1061ea80 2024-04-16T17:09:09.490 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:09.487+0000 7f893b400640 1 -- 172.21.15.45:0/161967281 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f894c043040 con 0x55ce1061ea80 2024-04-16T17:09:09.490 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:09.487+0000 7f895f3b2740 10 client.16170 fetch_fsmap learned FSMap version 29 2024-04-16T17:09:09.491 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:09.487+0000 7f895f3b2740 1 -- 172.21.15.45:0/161967281 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55ce1069e8a0 con 0x55ce1061ea80 2024-04-16T17:09:09.491 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:09.488+0000 7f893b400640 1 -- 172.21.15.45:0/161967281 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8940020070 con 0x55ce1061ea80 2024-04-16T17:09:09.491 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:09.488+0000 7f893b400640 1 -- 172.21.15.45:0/161967281 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 29) ==== 8366+0+0 (secure 0 0 0) 0x7f894c048450 con 0x55ce1061ea80 2024-04-16T17:09:09.491 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:09.488+0000 7f895f3b2740 10 client.16170 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T17:09:09.491 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:09.488+0000 7f895f3b2740 10 client.16170 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T17:09:09.491 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:09.488+0000 7f895f3b2740 20 client.16170 populate_metadata read hostname 'smithi045' 2024-04-16T17:09:09.491 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:09.488+0000 7f895f3b2740 1 --2- 172.21.15.45:0/161967281 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55ce1069ef40 0x55ce106a1360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:09.491 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:09.488+0000 7f895f3b2740 4 client.16170 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-16T17:09:09.491 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:09.488+0000 7f895f3b2740 1 -- 172.21.15.45:0/161967281 --> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ce1058b010 con 0x55ce1069ef40 2024-04-16T17:09:09.491 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:09.488+0000 7f8959200640 1 --2- 172.21.15.45:0/161967281 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55ce1069ef40 0x55ce106a1360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:09.492 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:09.489+0000 7f893b400640 10 client.16170 ms_handle_connect on v2:172.21.15.104:6838/3660070592 2024-04-16T17:09:09.492 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:09.489+0000 7f8959200640 1 --2- 172.21.15.45:0/161967281 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55ce1069ef40 0x55ce106a1360 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=12584ab67bc150b9 server_cookie=89fbd470c0835319 in_seq=0 out_seq=0 2024-04-16T17:09:09.531 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:09.531+0000 7f893b400640 1 -- 172.21.15.45:0/161967281 <== mds.0 v2:172.21.15.104:6838/3660070592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ce1058b010 con 0x55ce1069ef40 2024-04-16T17:09:09.531 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:09.531+0000 7f893b400640 10 client.16170 handle_command_reply: tid=0 2024-04-16T17:09:09.531 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:09.531+0000 7f895f3b2740 1 client.16170 shutdown 2024-04-16T17:09:09.532 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:09.531+0000 7f895f3b2740 1 -- 172.21.15.45:0/161967281 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ce1061ea80 msgr2=0x55ce10621d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:09.532 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:09.531+0000 7f895f3b2740 1 --2- 172.21.15.45:0/161967281 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ce1061ea80 0x55ce10621d10 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f894c009430 tx=0x7f894c004680 comp rx=0 tx=0).stop 2024-04-16T17:09:09.532 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:09.531+0000 7f895f3b2740 1 -- 172.21.15.45:0/161967281 shutdown_connections 2024-04-16T17:09:09.532 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:09.531+0000 7f895f3b2740 1 --2- 172.21.15.45:0/161967281 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55ce1069ef40 0x55ce106a1360 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:09.532 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:09.531+0000 7f895f3b2740 1 --2- 172.21.15.45:0/161967281 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ce1061f7f0 0x55ce10622250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:09.532 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:09.531+0000 7f895f3b2740 1 --2- 172.21.15.45:0/161967281 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ce10628000 0x55ce10622790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:09.532 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:09.531+0000 7f895f3b2740 1 --2- 172.21.15.45:0/161967281 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ce1061ea80 0x55ce10621d10 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:09.532 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:09.531+0000 7f895f3b2740 1 -- 172.21.15.45:0/161967281 >> 172.21.15.45:0/161967281 conn(0x55ce105b95d0 msgr2=0x55ce10617f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:09:09.532 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:09.531+0000 7f893b400640 0 client.16170 ms_handle_reset on v2:172.21.15.104:6838/3660070592 2024-04-16T17:09:09.532 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:09.531+0000 7f895f3b2740 1 -- 172.21.15.45:0/161967281 shutdown_connections 2024-04-16T17:09:09.532 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:09.531+0000 7f895f3b2740 1 -- 172.21.15.45:0/161967281 wait complete. 2024-04-16T17:09:09.532 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:09.532+0000 7f895f3b2740 20 client.16170 trim_cache size 0 max 16384 2024-04-16T17:09:09.533 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-04-16T17:09:09.533 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-04-16T17:09:09.533 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-04-16T17:09:09.533 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-04-16T17:09:09.534 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:09.532+0000 7f895b000640 1 -- 172.21.15.45:0/1210052558 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f893c0a3d10 msgr2=0x7f893c0a61d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:09.534 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:09.532+0000 7f895b000640 1 --2- 172.21.15.45:0/1210052558 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f893c0a3d10 0x7f893c0a61d0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x55ce1061ff50 tx=0x7f89480d4a50 comp rx=0 tx=0).stop 2024-04-16T17:09:09.534 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:09.532+0000 7f895b000640 1 -- 172.21.15.45:0/1210052558 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8954102fc0 msgr2=0x7f89541a0620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:09.534 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:09.532+0000 7f895b000640 1 --2- 172.21.15.45:0/1210052558 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8954102fc0 0x7f89541a0620 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f894800e9f0 tx=0x7f894800edf0 comp rx=0 tx=0).stop 2024-04-16T17:09:09.534 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:09.532+0000 7f895b000640 1 -- 172.21.15.45:0/1210052558 shutdown_connections 2024-04-16T17:09:09.534 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:09.532+0000 7f895b000640 1 --2- 172.21.15.45:0/1210052558 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f893c0a3d10 0x7f893c0a61d0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:09.534 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:09.532+0000 7f895b000640 1 --2- 172.21.15.45:0/1210052558 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f89541038e0 0x7f89541a0b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:09.534 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:09.532+0000 7f895b000640 1 --2- 172.21.15.45:0/1210052558 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f895410c3e0 0x7f895419a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:09.534 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:09.532+0000 7f895b000640 1 --2- 172.21.15.45:0/1210052558 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8954102fc0 0x7f89541a0620 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:09.534 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:09.532+0000 7f895b000640 1 -- 172.21.15.45:0/1210052558 >> 172.21.15.45:0/1210052558 conn(0x7f89540fc820 msgr2=0x7f8954101db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:09:09.534 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:09.533+0000 7f895b000640 1 -- 172.21.15.45:0/1210052558 shutdown_connections 2024-04-16T17:09:09.535 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:09.533+0000 7f895b000640 1 -- 172.21.15.45:0/1210052558 wait complete. 2024-04-16T17:09:09.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.541+0000 7f14f3200640 1 -- 172.21.15.45:0/2736981087 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f14f4100e50 msgr2=0x7f14f4101230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:09.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.541+0000 7f14f3200640 1 --2- 172.21.15.45:0/2736981087 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f14f4100e50 0x7f14f4101230 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f14e4009900 tx=0x7f14e402bd10 comp rx=0 tx=0).stop 2024-04-16T17:09:09.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.541+0000 7f14f3200640 1 -- 172.21.15.45:0/2736981087 shutdown_connections 2024-04-16T17:09:09.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.541+0000 7f14f3200640 1 --2- 172.21.15.45:0/2736981087 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f14f4106250 0x7f14f41066e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:09.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.541+0000 7f14f3200640 1 --2- 172.21.15.45:0/2736981087 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f14f4101770 0x7f14f4105c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:09.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.541+0000 7f14f3200640 1 --2- 172.21.15.45:0/2736981087 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f14f4100e50 0x7f14f4101230 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:09.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.541+0000 7f14f3200640 1 -- 172.21.15.45:0/2736981087 >> 172.21.15.45:0/2736981087 conn(0x7f14f40fc880 msgr2=0x7f14f40feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:09:09.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.541+0000 7f14f3200640 1 -- 172.21.15.45:0/2736981087 shutdown_connections 2024-04-16T17:09:09.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.541+0000 7f14f3200640 1 -- 172.21.15.45:0/2736981087 wait complete. 2024-04-16T17:09:09.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.542+0000 7f14f3200640 1 Processor -- start 2024-04-16T17:09:09.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.543+0000 7f14f3200640 1 -- start start 2024-04-16T17:09:09.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.543+0000 7f14f3200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f14f4100e50 0x7f14f419eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:09.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.543+0000 7f14f3200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f14f4101770 0x7f14f419f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:09.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.543+0000 7f14f3200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f14f4106250 0x7f14f41a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:09.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.543+0000 7f14f3200640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f14f41188c0 con 0x7f14f4100e50 2024-04-16T17:09:09.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.543+0000 7f14f3200640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f14f4118740 con 0x7f14f4106250 2024-04-16T17:09:09.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.543+0000 7f14f3200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f14f4118a40 con 0x7f14f4101770 2024-04-16T17:09:09.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.543+0000 7f14f2800640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f14f4106250 0x7f14f41a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:09.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.543+0000 7f14f2800640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f14f4106250 0x7f14f41a3660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:47894/0 (socket says 172.21.15.45:47894) 2024-04-16T17:09:09.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.543+0000 7f14f2800640 1 -- 172.21.15.45:0/1230801541 learned_addr learned my addr 172.21.15.45:0/1230801541 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:09:09.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.543+0000 7f14f1400640 1 --2- 172.21.15.45:0/1230801541 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f14f4101770 0x7f14f419f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:09.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.543+0000 7f14f1e00640 1 --2- 172.21.15.45:0/1230801541 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f14f4100e50 0x7f14f419eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:09.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.544+0000 7f14f2800640 1 -- 172.21.15.45:0/1230801541 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f14f4101770 msgr2=0x7f14f419f2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:09.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.544+0000 7f14f2800640 1 --2- 172.21.15.45:0/1230801541 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f14f4101770 0x7f14f419f2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:09.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.544+0000 7f14f2800640 1 -- 172.21.15.45:0/1230801541 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f14f4100e50 msgr2=0x7f14f419eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:09.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.544+0000 7f14f2800640 1 --2- 172.21.15.45:0/1230801541 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f14f4100e50 0x7f14f419eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:09.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.544+0000 7f14f2800640 1 -- 172.21.15.45:0/1230801541 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14f41a3d90 con 0x7f14f4106250 2024-04-16T17:09:09.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.544+0000 7f14f2800640 1 --2- 172.21.15.45:0/1230801541 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f14f4106250 0x7f14f41a3660 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f14ec00eb10 tx=0x7f14ec00c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:09:09.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.544+0000 7f14eaa00640 1 -- 172.21.15.45:0/1230801541 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14ec004b20 con 0x7f14f4106250 2024-04-16T17:09:09.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.544+0000 7f14f3200640 1 -- 172.21.15.45:0/1230801541 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14f41a4080 con 0x7f14f4106250 2024-04-16T17:09:09.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.544+0000 7f14eaa00640 1 -- 172.21.15.45:0/1230801541 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f14ec004cc0 con 0x7f14f4106250 2024-04-16T17:09:09.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.544+0000 7f14eaa00640 1 -- 172.21.15.45:0/1230801541 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14ec010760 con 0x7f14f4106250 2024-04-16T17:09:09.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.545+0000 7f14f3200640 1 -- 172.21.15.45:0/1230801541 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14f41ab910 con 0x7f14f4106250 2024-04-16T17:09:09.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.549+0000 7f14fbdce740 1 Processor -- start 2024-04-16T17:09:09.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.549+0000 7f14eaa00640 1 -- 172.21.15.45:0/1230801541 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f14ec005020 con 0x7f14f4106250 2024-04-16T17:09:09.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.549+0000 7f14fbdce740 1 -- start start 2024-04-16T17:09:09.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.550+0000 7f14fbdce740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b1272acf10 0x55b1272ad2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:09.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.550+0000 7f14fbdce740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b1272ad830 0x55b1272b5ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:09.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.550+0000 7f14fbdce740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b1272b6410 0x55b1272b8800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:09.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.550+0000 7f14fbdce740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55b127212020 con 0x55b1272b6410 2024-04-16T17:09:09.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.550+0000 7f14fbdce740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55b1271b8460 con 0x55b1272ad830 2024-04-16T17:09:09.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.550+0000 7f14fbdce740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55b12720dfe0 con 0x55b1272acf10 2024-04-16T17:09:09.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.550+0000 7f14f1400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b1272b6410 0x55b1272b8800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:09.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.550+0000 7f14f1e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b1272ad830 0x55b1272b5ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:09.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.550+0000 7f14f1400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b1272b6410 0x55b1272b8800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45284/0 (socket says 172.21.15.45:45284) 2024-04-16T17:09:09.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.550+0000 7f14f1400640 1 -- 172.21.15.45:0/2757240353 learned_addr learned my addr 172.21.15.45:0/2757240353 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:09:09.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.550+0000 7f14f1400640 1 --2- 172.21.15.45:0/2757240353 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b1272acf10 0x55b1272ad2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:09.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.551+0000 7f14f1e00640 1 -- 172.21.15.45:0/2757240353 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b1272acf10 msgr2=0x55b1272ad2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:09:09.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.551+0000 7f14f1e00640 1 --2- 172.21.15.45:0/2757240353 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b1272acf10 0x55b1272ad2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:09.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.551+0000 7f14f1e00640 1 -- 172.21.15.45:0/2757240353 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b1272b6410 msgr2=0x55b1272b8800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:09:09.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.551+0000 7f14f1e00640 1 --2- 172.21.15.45:0/2757240353 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b1272b6410 0x55b1272b8800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:09.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.551+0000 7f14f1e00640 1 -- 172.21.15.45:0/2757240353 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b1271c0040 con 0x55b1272ad830 2024-04-16T17:09:09.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.551+0000 7f14f1400640 1 --2- 172.21.15.45:0/2757240353 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b1272acf10 0x55b1272ad2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:09:09.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.551+0000 7f14eaa00640 1 --2- 172.21.15.45:0/1230801541 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f14d80a3d20 0x7f14d80a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:09.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.551+0000 7f14eaa00640 1 -- 172.21.15.45:0/1230801541 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f14ec0d10b0 con 0x7f14f4106250 2024-04-16T17:09:09.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.551+0000 7f14f1400640 1 --2- 172.21.15.45:0/1230801541 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f14d80a3d20 0x7f14d80a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:09.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.551+0000 7f14f1e00640 1 --2- 172.21.15.45:0/2757240353 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b1272ad830 0x55b1272b5ed0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f14e402c250 tx=0x7f14e4009560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=32d2ff7d29bb8890 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:09:09.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.552+0000 7f14df400640 1 -- 172.21.15.45:0/2757240353 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14e4043070 con 0x55b1272ad830 2024-04-16T17:09:09.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.552+0000 7f14f1400640 1 --2- 172.21.15.45:0/1230801541 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f14d80a3d20 0x7f14d80a61e0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x55b1272a87d0 tx=0x7f14e00097c0 comp rx=0 tx=0).ready entity=mgr.25411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:09:09.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.552+0000 7f14df400640 1 -- 172.21.15.45:0/2757240353 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f14e402c6d0 con 0x55b1272ad830 2024-04-16T17:09:09.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.552+0000 7f14df400640 1 -- 172.21.15.45:0/2757240353 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14e4004a40 con 0x55b1272ad830 2024-04-16T17:09:09.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.553+0000 7f14fbdce740 1 -- 172.21.15.45:0/2757240353 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b1272ad830 msgr2=0x55b1272b5ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:09.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.553+0000 7f14fbdce740 1 --2- 172.21.15.45:0/2757240353 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b1272ad830 0x55b1272b5ed0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f14e402c250 tx=0x7f14e4009560 comp rx=0 tx=0).stop 2024-04-16T17:09:09.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.553+0000 7f14fbdce740 1 -- 172.21.15.45:0/2757240353 shutdown_connections 2024-04-16T17:09:09.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.553+0000 7f14fbdce740 1 --2- 172.21.15.45:0/2757240353 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b1272b6410 0x55b1272b8800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:09.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.553+0000 7f14fbdce740 1 --2- 172.21.15.45:0/2757240353 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b1272ad830 0x55b1272b5ed0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:09.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.553+0000 7f14fbdce740 1 --2- 172.21.15.45:0/2757240353 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b1272acf10 0x55b1272ad2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:09.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.553+0000 7f14fbdce740 1 -- 172.21.15.45:0/2757240353 >> 172.21.15.45:0/2757240353 conn(0x55b1272a7640 msgr2=0x55b1272a7a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:09:09.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.553+0000 7f14fbdce740 1 -- 172.21.15.45:0/2757240353 shutdown_connections 2024-04-16T17:09:09.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.553+0000 7f14fbdce740 1 -- 172.21.15.45:0/2757240353 wait complete. 2024-04-16T17:09:09.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.554+0000 7f14fbdce740 1 -- start start 2024-04-16T17:09:09.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.554+0000 7f14fbdce740 1 Processor -- start 2024-04-16T17:09:09.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.554+0000 7f14fbdce740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b1272acf10 0x55b1272b3450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:09.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.555+0000 7f14fbdce740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b1272ad830 0x55b1272b3990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:09.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.555+0000 7f14fbdce740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b1272b6410 0x55b1272b3ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:09.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.555+0000 7f14fbdce740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55b1272b91a0 con 0x55b1272b6410 2024-04-16T17:09:09.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.555+0000 7f14fbdce740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55b1271cb830 con 0x55b1272acf10 2024-04-16T17:09:09.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.555+0000 7f14fbdce740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55b1272b9320 con 0x55b1272ad830 2024-04-16T17:09:09.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.555+0000 7f14fbdce740 10 client.0 _ll_register_callbacks cb 0x55b1271d3150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:09:09.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.555+0000 7f14f1e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b1272acf10 0x55b1272b3450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:09.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.555+0000 7f14f2800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b1272b6410 0x55b1272b3ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:09.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.555+0000 7f14f1e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b1272acf10 0x55b1272b3450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:47924/0 (socket says 172.21.15.45:47924) 2024-04-16T17:09:09.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.555+0000 7f14f1e00640 1 -- 172.21.15.45:0/3692034730 learned_addr learned my addr 172.21.15.45:0/3692034730 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:09:09.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.555+0000 7f14f1400640 1 --2- 172.21.15.45:0/3692034730 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b1272ad830 0x55b1272b3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:09.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.555+0000 7f14f1e00640 1 -- 172.21.15.45:0/3692034730 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b1272ad830 msgr2=0x55b1272b3990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:09:09.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.555+0000 7f14f1e00640 1 --2- 172.21.15.45:0/3692034730 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b1272ad830 0x55b1272b3990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:09.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.555+0000 7f14f1e00640 1 -- 172.21.15.45:0/3692034730 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b1272b6410 msgr2=0x55b1272b3ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:09:09.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.555+0000 7f14f1e00640 1 --2- 172.21.15.45:0/3692034730 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b1272b6410 0x55b1272b3ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:09.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.555+0000 7f14f1e00640 1 -- 172.21.15.45:0/3692034730 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b1271cf920 con 0x55b1272acf10 2024-04-16T17:09:09.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.555+0000 7f14f1400640 1 --2- 172.21.15.45:0/3692034730 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b1272ad830 0x55b1272b3990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:09:09.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.556+0000 7f14f1e00640 1 --2- 172.21.15.45:0/3692034730 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b1272acf10 0x55b1272b3450 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f14e402c8a0 tx=0x7f14e40058d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b27dc710f53c0231 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:09:09.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.556+0000 7f14d3e00640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-04-16T17:09:09.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.556+0000 7f14d3e00640 1 -- 172.21.15.45:0/3692034730 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14e4043040 con 0x55b1272acf10 2024-04-16T17:09:09.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.556+0000 7f14d3e00640 1 -- 172.21.15.45:0/3692034730 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f14e4004ee0 con 0x55b1272acf10 2024-04-16T17:09:09.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.556+0000 7f14fbdce740 1 -- 172.21.15.45:0/3692034730 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f14e402c6d0 con 0x55b1272acf10 2024-04-16T17:09:09.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.556+0000 7f14d3e00640 1 -- 172.21.15.45:0/3692034730 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f14e4059050 con 0x55b1272acf10 2024-04-16T17:09:09.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.556+0000 7f14fbdce740 10 client.26330 fetch_fsmap learned FSMap version 29 2024-04-16T17:09:09.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.556+0000 7f14fbdce740 1 -- 172.21.15.45:0/3692034730 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55b12719d7f0 con 0x55b1272acf10 2024-04-16T17:09:09.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.557+0000 7f14d3e00640 1 -- 172.21.15.45:0/3692034730 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f14e402c6d0 con 0x55b1272acf10 2024-04-16T17:09:09.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.557+0000 7f14d3e00640 1 -- 172.21.15.45:0/3692034730 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 29) ==== 8366+0+0 (secure 0 0 0) 0x7f14e4048aa0 con 0x55b1272acf10 2024-04-16T17:09:09.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.557+0000 7f14fbdce740 10 client.26330 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T17:09:09.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.557+0000 7f14fbdce740 10 client.26330 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:09:09.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.557+0000 7f14fbdce740 20 client.26330 populate_metadata read hostname 'smithi045' 2024-04-16T17:09:09.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.557+0000 7f14fbdce740 1 --2- 172.21.15.45:0/3692034730 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55b1272c65e0 0x55b1272c89d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:09.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.557+0000 7f14fbdce740 4 client.26330 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:09:09.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.557+0000 7f14fbdce740 1 -- 172.21.15.45:0/3692034730 --> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b12721fa40 con 0x55b1272c65e0 2024-04-16T17:09:09.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.558+0000 7f14f1400640 1 --2- 172.21.15.45:0/3692034730 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55b1272c65e0 0x55b1272c89d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:09.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.558+0000 7f14f1400640 1 --2- 172.21.15.45:0/3692034730 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55b1272c65e0 0x55b1272c89d0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=28b6fdfa68dd46f9 server_cookie=b063e6049b978cec in_seq=0 out_seq=0 2024-04-16T17:09:09.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.558+0000 7f14d3e00640 10 client.26330 ms_handle_connect on v2:172.21.15.104:6838/3660070592 2024-04-16T17:09:09.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.559+0000 7f14d3e00640 1 -- 172.21.15.45:0/3692034730 <== mds.0 v2:172.21.15.104:6838/3660070592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b12721fa40 con 0x55b1272c65e0 2024-04-16T17:09:09.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.559+0000 7f14d3e00640 10 client.26330 handle_command_reply: tid=0 2024-04-16T17:09:09.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.559+0000 7f14fbdce740 1 client.26330 shutdown 2024-04-16T17:09:09.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.559+0000 7f14fbdce740 1 -- 172.21.15.45:0/3692034730 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b1272acf10 msgr2=0x55b1272b3450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:09.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.559+0000 7f14fbdce740 1 --2- 172.21.15.45:0/3692034730 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b1272acf10 0x55b1272b3450 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f14e402c8a0 tx=0x7f14e40058d0 comp rx=0 tx=0).stop 2024-04-16T17:09:09.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.559+0000 7f14fbdce740 1 -- 172.21.15.45:0/3692034730 shutdown_connections 2024-04-16T17:09:09.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.559+0000 7f14fbdce740 1 --2- 172.21.15.45:0/3692034730 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55b1272c65e0 0x55b1272c89d0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:09.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.559+0000 7f14fbdce740 1 --2- 172.21.15.45:0/3692034730 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b1272ad830 0x55b1272b3990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:09.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.559+0000 7f14fbdce740 1 --2- 172.21.15.45:0/3692034730 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b1272b6410 0x55b1272b3ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:09.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.559+0000 7f14d3e00640 0 client.26330 ms_handle_reset on v2:172.21.15.104:6838/3660070592 2024-04-16T17:09:09.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.559+0000 7f14fbdce740 1 --2- 172.21.15.45:0/3692034730 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b1272acf10 0x55b1272b3450 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:09.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.559+0000 7f14fbdce740 1 -- 172.21.15.45:0/3692034730 >> 172.21.15.45:0/3692034730 conn(0x55b1272a7640 msgr2=0x55b1272a9390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:09:09.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.559+0000 7f14fbdce740 1 -- 172.21.15.45:0/3692034730 shutdown_connections 2024-04-16T17:09:09.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.559+0000 7f14fbdce740 1 -- 172.21.15.45:0/3692034730 wait complete. 2024-04-16T17:09:09.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.559+0000 7f14fbdce740 20 client.26330 trim_cache size 0 max 16384 2024-04-16T17:09:09.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.574+0000 7f14fbdce740 1 Processor -- start 2024-04-16T17:09:09.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.574+0000 7f14fbdce740 1 -- start start 2024-04-16T17:09:09.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.575+0000 7f14fbdce740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b1272acf10 0x55b1272b5b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:09.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.575+0000 7f14fbdce740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b1272ad830 0x55b1272c2060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:09.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.575+0000 7f14fbdce740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b1272b6410 0x55b1272ce0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:09.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.575+0000 7f14fbdce740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f14f41b8160 con 0x55b1272acf10 2024-04-16T17:09:09.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.575+0000 7f14fbdce740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f14f41b59a0 con 0x55b1272b6410 2024-04-16T17:09:09.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.575+0000 7f14fbdce740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f14f41b37d0 con 0x55b1272ad830 2024-04-16T17:09:09.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.575+0000 7f14f1e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b1272acf10 0x55b1272b5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:09.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.575+0000 7f14f1e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b1272acf10 0x55b1272b5b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45304/0 (socket says 172.21.15.45:45304) 2024-04-16T17:09:09.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.575+0000 7f14f1e00640 1 -- 172.21.15.45:0/2444755523 learned_addr learned my addr 172.21.15.45:0/2444755523 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:09:09.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.575+0000 7f14f2800640 1 --2- 172.21.15.45:0/2444755523 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b1272b6410 0x55b1272ce0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:09.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.575+0000 7f14f1400640 1 --2- 172.21.15.45:0/2444755523 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b1272ad830 0x55b1272c2060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:09.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.575+0000 7f14f1e00640 1 -- 172.21.15.45:0/2444755523 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b1272ad830 msgr2=0x55b1272c2060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:09:09.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.575+0000 7f14f1e00640 1 --2- 172.21.15.45:0/2444755523 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b1272ad830 0x55b1272c2060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:09.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.575+0000 7f14f1e00640 1 -- 172.21.15.45:0/2444755523 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b1272b6410 msgr2=0x55b1272ce0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:09:09.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.575+0000 7f14f1e00640 1 --2- 172.21.15.45:0/2444755523 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b1272b6410 0x55b1272ce0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:09.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.575+0000 7f14f1e00640 1 -- 172.21.15.45:0/2444755523 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b126f243f0 con 0x55b1272acf10 2024-04-16T17:09:09.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.575+0000 7f14f2800640 1 --2- 172.21.15.45:0/2444755523 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b1272b6410 0x55b1272ce0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:09:09.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.576+0000 7f14f1e00640 1 --2- 172.21.15.45:0/2444755523 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b1272acf10 0x55b1272b5b80 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f14e402c250 tx=0x7f14e4005a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab5e7ce9e33e969d server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:09:09.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.576+0000 7f14dea00640 1 -- 172.21.15.45:0/2444755523 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14e4062070 con 0x55b1272acf10 2024-04-16T17:09:09.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.576+0000 7f14dea00640 1 -- 172.21.15.45:0/2444755523 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f14e4009520 con 0x55b1272acf10 2024-04-16T17:09:09.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.576+0000 7f14dea00640 1 -- 172.21.15.45:0/2444755523 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14e40053e0 con 0x55b1272acf10 2024-04-16T17:09:09.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.577+0000 7f14fbdce740 1 -- 172.21.15.45:0/2444755523 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b1272acf10 msgr2=0x55b1272b5b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:09.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.577+0000 7f14fbdce740 1 --2- 172.21.15.45:0/2444755523 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b1272acf10 0x55b1272b5b80 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f14e402c250 tx=0x7f14e4005a40 comp rx=0 tx=0).stop 2024-04-16T17:09:09.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.577+0000 7f14fbdce740 1 -- 172.21.15.45:0/2444755523 shutdown_connections 2024-04-16T17:09:09.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.577+0000 7f14fbdce740 1 --2- 172.21.15.45:0/2444755523 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b1272ad830 0x55b1272c2060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:09.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.577+0000 7f14fbdce740 1 --2- 172.21.15.45:0/2444755523 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b1272b6410 0x55b1272ce0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:09.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.577+0000 7f14fbdce740 1 --2- 172.21.15.45:0/2444755523 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b1272acf10 0x55b1272b5b80 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:09.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.577+0000 7f14fbdce740 1 -- 172.21.15.45:0/2444755523 >> 172.21.15.45:0/2444755523 conn(0x55b1272a7640 msgr2=0x55b1272a6700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:09:09.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.577+0000 7f14fbdce740 1 -- 172.21.15.45:0/2444755523 shutdown_connections 2024-04-16T17:09:09.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.577+0000 7f14fbdce740 1 -- 172.21.15.45:0/2444755523 wait complete. 2024-04-16T17:09:09.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.577+0000 7f14fbdce740 1 -- start start 2024-04-16T17:09:09.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.577+0000 7f14fbdce740 1 Processor -- start 2024-04-16T17:09:09.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.578+0000 7f14fbdce740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b1272acf10 0x55b1272b0340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:09.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.578+0000 7f14fbdce740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b1272ad830 0x55b1272b0880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:09.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.578+0000 7f14fbdce740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b1272b6410 0x55b1272b0dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:09.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.578+0000 7f14fbdce740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f14f41b3e60 con 0x55b1272ad830 2024-04-16T17:09:09.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.578+0000 7f14fbdce740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f14f41b52d0 con 0x55b1272b6410 2024-04-16T17:09:09.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.578+0000 7f14fbdce740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f14f41b4530 con 0x55b1272acf10 2024-04-16T17:09:09.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.578+0000 7f14fbdce740 10 client.0 _ll_register_callbacks cb 0x7f14e0009ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:09:09.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.578+0000 7f14f1400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b1272ad830 0x55b1272b0880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:09.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.578+0000 7f14f1e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b1272acf10 0x55b1272b0340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:09.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.578+0000 7f14f1400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b1272ad830 0x55b1272b0880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45318/0 (socket says 172.21.15.45:45318) 2024-04-16T17:09:09.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.578+0000 7f14f1400640 1 -- 172.21.15.45:0/666626459 learned_addr learned my addr 172.21.15.45:0/666626459 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:09:09.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.578+0000 7f14f2800640 1 --2- 172.21.15.45:0/666626459 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b1272b6410 0x55b1272b0dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:09.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.578+0000 7f14f1400640 1 -- 172.21.15.45:0/666626459 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b1272acf10 msgr2=0x55b1272b0340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:09:09.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.578+0000 7f14f1400640 1 --2- 172.21.15.45:0/666626459 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b1272acf10 0x55b1272b0340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:09.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.578+0000 7f14f1400640 1 -- 172.21.15.45:0/666626459 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b1272b6410 msgr2=0x55b1272b0dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:09:09.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.578+0000 7f14f1400640 1 --2- 172.21.15.45:0/666626459 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b1272b6410 0x55b1272b0dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:09.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.578+0000 7f14f1400640 1 -- 172.21.15.45:0/666626459 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b126fee7d0 con 0x55b1272ad830 2024-04-16T17:09:09.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.578+0000 7f14f1e00640 1 --2- 172.21.15.45:0/666626459 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b1272acf10 0x55b1272b0340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:09:09.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.579+0000 7f14f1400640 1 --2- 172.21.15.45:0/666626459 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b1272ad830 0x55b1272b0880 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f14e0009a80 tx=0x7f14e0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab2139b3f82486c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:09:09.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.579+0000 7f14d3e00640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-16T17:09:09.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.579+0000 7f14d3e00640 1 -- 172.21.15.45:0/666626459 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14e0033070 con 0x55b1272ad830 2024-04-16T17:09:09.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.579+0000 7f14d3e00640 1 -- 172.21.15.45:0/666626459 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f14e0004020 con 0x55b1272ad830 2024-04-16T17:09:09.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.579+0000 7f14fbdce740 1 -- 172.21.15.45:0/666626459 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f14e4009520 con 0x55b1272ad830 2024-04-16T17:09:09.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.579+0000 7f14d3e00640 1 -- 172.21.15.45:0/666626459 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f14e0010e20 con 0x55b1272ad830 2024-04-16T17:09:09.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.579+0000 7f14fbdce740 10 client.16188 fetch_fsmap learned FSMap version 29 2024-04-16T17:09:09.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.579+0000 7f14fbdce740 1 -- 172.21.15.45:0/666626459 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55b12732cca0 con 0x55b1272ad830 2024-04-16T17:09:09.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.580+0000 7f14d3e00640 1 -- 172.21.15.45:0/666626459 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f14e4009520 con 0x55b1272ad830 2024-04-16T17:09:09.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.580+0000 7f14d3e00640 1 -- 172.21.15.45:0/666626459 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 29) ==== 8366+0+0 (secure 0 0 0) 0x7f14e0005410 con 0x55b1272ad830 2024-04-16T17:09:09.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.580+0000 7f14fbdce740 10 client.16188 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T17:09:09.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.580+0000 7f14fbdce740 10 client.16188 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:09:09.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.580+0000 7f14fbdce740 20 client.16188 populate_metadata read hostname 'smithi045' 2024-04-16T17:09:09.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.580+0000 7f14fbdce740 1 --2- 172.21.15.45:0/666626459 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55b12732d500 0x55b12732f920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:09.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.580+0000 7f14fbdce740 4 client.16188 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T17:09:09.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.580+0000 7f14fbdce740 1 -- 172.21.15.45:0/666626459 --> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b1272198b0 con 0x55b12732d500 2024-04-16T17:09:09.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.580+0000 7f14f1e00640 1 --2- 172.21.15.45:0/666626459 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55b12732d500 0x55b12732f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:09.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.581+0000 7f14f1e00640 1 --2- 172.21.15.45:0/666626459 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55b12732d500 0x55b12732f920 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=734e1e974e3e1cd0 server_cookie=88973cbb5fcdd85a in_seq=0 out_seq=0 2024-04-16T17:09:09.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.581+0000 7f14d3e00640 10 client.16188 ms_handle_connect on v2:172.21.15.104:6838/3660070592 2024-04-16T17:09:09.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.581+0000 7f14d3e00640 1 -- 172.21.15.45:0/666626459 <== mds.0 v2:172.21.15.104:6838/3660070592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b1272198b0 con 0x55b12732d500 2024-04-16T17:09:09.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.581+0000 7f14d3e00640 10 client.16188 handle_command_reply: tid=0 2024-04-16T17:09:09.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.581+0000 7f14fbdce740 1 client.16188 shutdown 2024-04-16T17:09:09.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.581+0000 7f14fbdce740 1 -- 172.21.15.45:0/666626459 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b1272ad830 msgr2=0x55b1272b0880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:09.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.581+0000 7f14fbdce740 1 --2- 172.21.15.45:0/666626459 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b1272ad830 0x55b1272b0880 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f14e0009a80 tx=0x7f14e0004730 comp rx=0 tx=0).stop 2024-04-16T17:09:09.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.581+0000 7f14fbdce740 1 -- 172.21.15.45:0/666626459 shutdown_connections 2024-04-16T17:09:09.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.581+0000 7f14fbdce740 1 --2- 172.21.15.45:0/666626459 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b1272b6410 0x55b1272b0dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:09.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.581+0000 7f14fbdce740 1 --2- 172.21.15.45:0/666626459 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b1272ad830 0x55b1272b0880 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:09.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.581+0000 7f14fbdce740 1 --2- 172.21.15.45:0/666626459 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55b12732d500 0x55b12732f920 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:09.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.581+0000 7f14fbdce740 1 --2- 172.21.15.45:0/666626459 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b1272acf10 0x55b1272b0340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:09.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.581+0000 7f14fbdce740 1 -- 172.21.15.45:0/666626459 >> 172.21.15.45:0/666626459 conn(0x55b1272a7640 msgr2=0x55b1272a5e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:09:09.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.581+0000 7f14d3e00640 0 client.16188 ms_handle_reset on v2:172.21.15.104:6838/3660070592 2024-04-16T17:09:09.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.582+0000 7f14fbdce740 1 -- 172.21.15.45:0/666626459 shutdown_connections 2024-04-16T17:09:09.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.582+0000 7f14fbdce740 1 -- 172.21.15.45:0/666626459 wait complete. 2024-04-16T17:09:09.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.582+0000 7f14fbdce740 20 client.16188 trim_cache size 0 max 16384 2024-04-16T17:09:09.586 INFO:teuthology.orchestra.run.smithi045.stdout:[] 2024-04-16T17:09:09.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.582+0000 7f14f3200640 1 -- 172.21.15.45:0/1230801541 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f14d80a3d20 msgr2=0x7f14d80a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:09.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.582+0000 7f14f3200640 1 --2- 172.21.15.45:0/1230801541 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f14d80a3d20 0x7f14d80a61e0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x55b1272a87d0 tx=0x7f14e00097c0 comp rx=0 tx=0).stop 2024-04-16T17:09:09.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.582+0000 7f14f3200640 1 -- 172.21.15.45:0/1230801541 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f14f4106250 msgr2=0x7f14f41a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:09.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.582+0000 7f14f3200640 1 --2- 172.21.15.45:0/1230801541 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f14f4106250 0x7f14f41a3660 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f14ec00eb10 tx=0x7f14ec00c2d0 comp rx=0 tx=0).stop 2024-04-16T17:09:09.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.582+0000 7f14f3200640 1 -- 172.21.15.45:0/1230801541 shutdown_connections 2024-04-16T17:09:09.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.582+0000 7f14f3200640 1 --2- 172.21.15.45:0/1230801541 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f14d80a3d20 0x7f14d80a61e0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:09.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.582+0000 7f14f3200640 1 --2- 172.21.15.45:0/1230801541 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f14f4101770 0x7f14f419f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:09.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.582+0000 7f14f3200640 1 --2- 172.21.15.45:0/1230801541 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f14f4106250 0x7f14f41a3660 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:09.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.582+0000 7f14f3200640 1 --2- 172.21.15.45:0/1230801541 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f14f4100e50 0x7f14f419eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:09.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.582+0000 7f14f3200640 1 -- 172.21.15.45:0/1230801541 >> 172.21.15.45:0/1230801541 conn(0x7f14f40fc880 msgr2=0x7f14f40fd8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:09:09.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.583+0000 7f14f3200640 1 -- 172.21.15.45:0/1230801541 shutdown_connections 2024-04-16T17:09:09.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:09.583+0000 7f14f3200640 1 -- 172.21.15.45:0/1230801541 wait complete. 2024-04-16T17:09:09.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:09:09 smithi045 ceph-mon[31676]: pgmap v75: 129 pgs: 129 active+clean; 27 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 436 KiB/s rd, 278 KiB/s wr, 79 op/s 2024-04-16T17:09:10.055 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:09:09 smithi117 ceph-mon[36730]: pgmap v75: 129 pgs: 129 active+clean; 27 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 436 KiB/s rd, 278 KiB/s wr, 79 op/s 2024-04-16T17:09:10.177 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:09:09 smithi104 ceph-mon[36457]: pgmap v75: 129 pgs: 129 active+clean; 27 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 436 KiB/s rd, 278 KiB/s wr, 79 op/s 2024-04-16T17:09:10.224 INFO:tasks.workunit.client.0.smithi045.stdout: 1 36062 2.95 MB/sec warmup 72 sec latency 2071.259 ms 2024-04-16T17:09:10.599 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-16T17:09:10.779 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T17:09:10.779 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-16T17:09:10.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:10.996+0000 7f994be00640 1 -- 172.21.15.45:0/635491517 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f994c102110 msgr2=0x7f994c1024f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:10.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:10.996+0000 7f994be00640 1 --2- 172.21.15.45:0/635491517 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f994c102110 0x7f994c1024f0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f993c009900 tx=0x7f993c02bc30 comp rx=0 tx=0).stop 2024-04-16T17:09:10.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:10.996+0000 7f994be00640 1 -- 172.21.15.45:0/635491517 shutdown_connections 2024-04-16T17:09:10.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:10.996+0000 7f994be00640 1 --2- 172.21.15.45:0/635491517 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f994c102ac0 0x7f994c106b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:10.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:10.997+0000 7f994be00640 1 --2- 172.21.15.45:0/635491517 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f994c1071e0 0x7f994c10edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:10.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:10.997+0000 7f994be00640 1 --2- 172.21.15.45:0/635491517 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f994c102110 0x7f994c1024f0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:10.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:10.997+0000 7f994be00640 1 -- 172.21.15.45:0/635491517 >> 172.21.15.45:0/635491517 conn(0x7f994c0fd9d0 msgr2=0x7f994c0ffdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:09:10.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:10.997+0000 7f994be00640 1 -- 172.21.15.45:0/635491517 shutdown_connections 2024-04-16T17:09:10.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:10.997+0000 7f994be00640 1 -- 172.21.15.45:0/635491517 wait complete. 2024-04-16T17:09:10.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:10.998+0000 7f994be00640 1 Processor -- start 2024-04-16T17:09:10.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:10.998+0000 7f994be00640 1 -- start start 2024-04-16T17:09:11.000 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:10.999+0000 7f994be00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f994c102110 0x7f994c074300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:11.000 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:10.999+0000 7f994be00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f994c1071e0 0x7f994c074840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:11.000 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:10.999+0000 7f994be00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f994c074d80 0x7f994c1a69f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:11.000 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:10.999+0000 7f994be00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f994c1122e0 con 0x7f994c102110 2024-04-16T17:09:11.000 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:10.999+0000 7f994be00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f994c112160 con 0x7f994c074d80 2024-04-16T17:09:11.000 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:10.999+0000 7f994be00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f994c112460 con 0x7f994c1071e0 2024-04-16T17:09:11.000 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:10.999+0000 7f994aa00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f994c102110 0x7f994c074300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:11.000 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:10.999+0000 7f994a000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f994c1071e0 0x7f994c074840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:11.000 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:10.999+0000 7f994a000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f994c1071e0 0x7f994c074840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:58434/0 (socket says 172.21.15.45:58434) 2024-04-16T17:09:11.000 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:10.999+0000 7f994aa00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f994c102110 0x7f994c074300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57750/0 (socket says 172.21.15.45:57750) 2024-04-16T17:09:11.000 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:10.999+0000 7f994a000640 1 -- 172.21.15.45:0/3504964432 learned_addr learned my addr 172.21.15.45:0/3504964432 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:09:11.000 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:10.999+0000 7f994b400640 1 --2- 172.21.15.45:0/3504964432 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f994c074d80 0x7f994c1a69f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:11.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.000+0000 7f994aa00640 1 -- 172.21.15.45:0/3504964432 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f994c1071e0 msgr2=0x7f994c074840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:11.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.000+0000 7f994aa00640 1 --2- 172.21.15.45:0/3504964432 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f994c1071e0 0x7f994c074840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:11.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.000+0000 7f994aa00640 1 -- 172.21.15.45:0/3504964432 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f994c074d80 msgr2=0x7f994c1a69f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:11.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.000+0000 7f994aa00640 1 --2- 172.21.15.45:0/3504964432 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f994c074d80 0x7f994c1a69f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:11.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.000+0000 7f994aa00640 1 -- 172.21.15.45:0/3504964432 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f994c1a70f0 con 0x7f994c102110 2024-04-16T17:09:11.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.000+0000 7f994b400640 1 --2- 172.21.15.45:0/3504964432 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f994c074d80 0x7f994c1a69f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:09:11.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.000+0000 7f994aa00640 1 --2- 172.21.15.45:0/3504964432 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f994c102110 0x7f994c074300 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f993c02c170 tx=0x7f993c004360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:09:11.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.000+0000 7f9943400640 1 -- 172.21.15.45:0/3504964432 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f993c043020 con 0x7f994c102110 2024-04-16T17:09:11.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.000+0000 7f994be00640 1 -- 172.21.15.45:0/3504964432 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f994c1a7380 con 0x7f994c102110 2024-04-16T17:09:11.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.000+0000 7f9943400640 1 -- 172.21.15.45:0/3504964432 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f993c0054e0 con 0x7f994c102110 2024-04-16T17:09:11.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.000+0000 7f9943400640 1 -- 172.21.15.45:0/3504964432 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f993c003b20 con 0x7f994c102110 2024-04-16T17:09:11.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.001+0000 7f994be00640 1 -- 172.21.15.45:0/3504964432 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f994c1a78f0 con 0x7f994c102110 2024-04-16T17:09:11.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.004+0000 7f995497e740 1 Processor -- start 2024-04-16T17:09:11.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.005+0000 7f995497e740 1 -- start start 2024-04-16T17:09:11.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.005+0000 7f995497e740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56284c7dee90 0x56284c7df270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:11.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.005+0000 7f995497e740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56284c7df7b0 0x56284c7e7e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:11.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.005+0000 7f995497e740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56284c7e8390 0x56284c7ea780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:11.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.005+0000 7f995497e740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x56284c743cb0 con 0x56284c7dee90 2024-04-16T17:09:11.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.005+0000 7f995497e740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x56284c6ea370 con 0x56284c7e8390 2024-04-16T17:09:11.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.005+0000 7f995497e740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x56284c73fc70 con 0x56284c7df7b0 2024-04-16T17:09:11.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.005+0000 7f9943400640 1 -- 172.21.15.45:0/3504964432 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f993c046020 con 0x7f994c102110 2024-04-16T17:09:11.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.005+0000 7f994b400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56284c7dee90 0x56284c7df270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:11.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.006+0000 7f994b400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56284c7dee90 0x56284c7df270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57758/0 (socket says 172.21.15.45:57758) 2024-04-16T17:09:11.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.006+0000 7f994b400640 1 -- 172.21.15.45:0/3371531897 learned_addr learned my addr 172.21.15.45:0/3371531897 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:09:11.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.006+0000 7f994a000640 1 --2- 172.21.15.45:0/3371531897 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56284c7e8390 0x56284c7ea780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:11.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.006+0000 7f994aa00640 1 --2- 172.21.15.45:0/3371531897 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56284c7df7b0 0x56284c7e7e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:11.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.006+0000 7f994b400640 1 -- 172.21.15.45:0/3371531897 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56284c7df7b0 msgr2=0x56284c7e7e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:09:11.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.006+0000 7f994b400640 1 --2- 172.21.15.45:0/3371531897 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56284c7df7b0 0x56284c7e7e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:11.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.006+0000 7f994b400640 1 -- 172.21.15.45:0/3371531897 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56284c7e8390 msgr2=0x56284c7ea780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:09:11.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.006+0000 7f994b400640 1 --2- 172.21.15.45:0/3371531897 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56284c7e8390 0x56284c7ea780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:11.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.006+0000 7f994b400640 1 -- 172.21.15.45:0/3371531897 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56284c709910 con 0x56284c7dee90 2024-04-16T17:09:11.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.006+0000 7f994b400640 1 --2- 172.21.15.45:0/3371531897 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56284c7dee90 0x56284c7df270 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f9944011790 tx=0x7f9944011c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e241cf15141ab88 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:09:11.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.007+0000 7f9937e00640 1 -- 172.21.15.45:0/3371531897 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9944004d20 con 0x56284c7dee90 2024-04-16T17:09:11.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.007+0000 7f9937e00640 1 -- 172.21.15.45:0/3371531897 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9944003970 con 0x56284c7dee90 2024-04-16T17:09:11.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.007+0000 7f9937e00640 1 -- 172.21.15.45:0/3371531897 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9944004500 con 0x56284c7dee90 2024-04-16T17:09:11.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.007+0000 7f995497e740 1 -- 172.21.15.45:0/3371531897 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56284c7dee90 msgr2=0x56284c7df270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:11.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.007+0000 7f995497e740 1 --2- 172.21.15.45:0/3371531897 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56284c7dee90 0x56284c7df270 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f9944011790 tx=0x7f9944011c50 comp rx=0 tx=0).stop 2024-04-16T17:09:11.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.007+0000 7f995497e740 1 -- 172.21.15.45:0/3371531897 shutdown_connections 2024-04-16T17:09:11.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.007+0000 7f995497e740 1 --2- 172.21.15.45:0/3371531897 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56284c7df7b0 0x56284c7e7e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:11.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.007+0000 7f995497e740 1 --2- 172.21.15.45:0/3371531897 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56284c7e8390 0x56284c7ea780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:11.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.007+0000 7f995497e740 1 --2- 172.21.15.45:0/3371531897 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56284c7dee90 0x56284c7df270 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:11.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.007+0000 7f995497e740 1 -- 172.21.15.45:0/3371531897 >> 172.21.15.45:0/3371531897 conn(0x56284c7d95c0 msgr2=0x56284c7d99a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:09:11.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.008+0000 7f995497e740 1 -- 172.21.15.45:0/3371531897 shutdown_connections 2024-04-16T17:09:11.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.008+0000 7f995497e740 1 -- 172.21.15.45:0/3371531897 wait complete. 2024-04-16T17:09:11.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.008+0000 7f9943400640 1 --2- 172.21.15.45:0/3504964432 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f99300a3c60 0x7f99300a6120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:11.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.008+0000 7f9943400640 1 -- 172.21.15.45:0/3504964432 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f993c0f29e0 con 0x7f994c102110 2024-04-16T17:09:11.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.008+0000 7f994a000640 1 --2- 172.21.15.45:0/3504964432 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f99300a3c60 0x7f99300a6120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:11.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.008+0000 7f995497e740 1 -- start start 2024-04-16T17:09:11.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.008+0000 7f995497e740 1 Processor -- start 2024-04-16T17:09:11.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.009+0000 7f994a000640 1 --2- 172.21.15.45:0/3504964432 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f99300a3c60 0x7f99300a6120 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x56284c7da750 tx=0x7f9938005f90 comp rx=0 tx=0).ready entity=mgr.25411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:09:11.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.009+0000 7f995497e740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56284c7dee90 0x56284c7e5330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:11.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.009+0000 7f995497e740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56284c7df7b0 0x56284c7e5870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:11.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.009+0000 7f995497e740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56284c7e8390 0x56284c7e5db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:11.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.009+0000 7f995497e740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x56284c7eb180 con 0x56284c7dee90 2024-04-16T17:09:11.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.009+0000 7f995497e740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x56284c6fd730 con 0x56284c7e8390 2024-04-16T17:09:11.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.009+0000 7f995497e740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x56284c7eb300 con 0x56284c7df7b0 2024-04-16T17:09:11.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.009+0000 7f995497e740 10 client.0 _ll_register_callbacks cb 0x56284c6cc1e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:09:11.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.009+0000 7f994aa00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56284c7dee90 0x56284c7e5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:11.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.009+0000 7f994aa00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56284c7dee90 0x56284c7e5330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57766/0 (socket says 172.21.15.45:57766) 2024-04-16T17:09:11.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.009+0000 7f994aa00640 1 -- 172.21.15.45:0/646597220 learned_addr learned my addr 172.21.15.45:0/646597220 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:09:11.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.009+0000 7f994a000640 1 --2- 172.21.15.45:0/646597220 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56284c7df7b0 0x56284c7e5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:11.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.009+0000 7f994b400640 1 --2- 172.21.15.45:0/646597220 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56284c7e8390 0x56284c7e5db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:11.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.010+0000 7f994aa00640 1 -- 172.21.15.45:0/646597220 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56284c7df7b0 msgr2=0x56284c7e5870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:09:11.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.010+0000 7f994aa00640 1 --2- 172.21.15.45:0/646597220 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56284c7df7b0 0x56284c7e5870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:11.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.010+0000 7f994aa00640 1 -- 172.21.15.45:0/646597220 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56284c7e8390 msgr2=0x56284c7e5db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:09:11.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.010+0000 7f994aa00640 1 --2- 172.21.15.45:0/646597220 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56284c7e8390 0x56284c7e5db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:11.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.010+0000 7f994aa00640 1 -- 172.21.15.45:0/646597220 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56284c51e7e0 con 0x56284c7dee90 2024-04-16T17:09:11.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.010+0000 7f994aa00640 1 --2- 172.21.15.45:0/646597220 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56284c7dee90 0x56284c7e5330 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f993c0efef0 tx=0x7f993c0035f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f2fb4828d2aaf84 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:09:11.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.010+0000 7f9934c00640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-16T17:09:11.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.010+0000 7f9934c00640 1 -- 172.21.15.45:0/646597220 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f993c0ba980 con 0x56284c7dee90 2024-04-16T17:09:11.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.010+0000 7f9934c00640 1 -- 172.21.15.45:0/646597220 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f993c02cb20 con 0x56284c7dee90 2024-04-16T17:09:11.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.010+0000 7f995497e740 1 -- 172.21.15.45:0/646597220 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9944003970 con 0x56284c7dee90 2024-04-16T17:09:11.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.010+0000 7f9934c00640 1 -- 172.21.15.45:0/646597220 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f993c0be040 con 0x56284c7dee90 2024-04-16T17:09:11.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.010+0000 7f995497e740 10 client.16200 fetch_fsmap learned FSMap version 29 2024-04-16T17:09:11.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.010+0000 7f995497e740 1 -- 172.21.15.45:0/646597220 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56284c57ef70 con 0x56284c7dee90 2024-04-16T17:09:11.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.011+0000 7f9934c00640 1 -- 172.21.15.45:0/646597220 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9944003970 con 0x56284c7dee90 2024-04-16T17:09:11.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.011+0000 7f9934c00640 1 -- 172.21.15.45:0/646597220 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 29) ==== 8366+0+0 (secure 0 0 0) 0x7f993c0c6ec0 con 0x56284c7dee90 2024-04-16T17:09:11.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.011+0000 7f995497e740 10 client.16200 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T17:09:11.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.011+0000 7f995497e740 10 client.16200 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:09:11.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.011+0000 7f995497e740 20 client.16200 populate_metadata read hostname 'smithi045' 2024-04-16T17:09:11.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.011+0000 7f995497e740 1 --2- 172.21.15.45:0/646597220 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x56284c7f8530 0x56284c7fa9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:11.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.011+0000 7f995497e740 4 client.16200 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:09:11.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.011+0000 7f995497e740 1 -- 172.21.15.45:0/646597220 --> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56284c7516d0 con 0x56284c7f8530 2024-04-16T17:09:11.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.011+0000 7f994a000640 1 --2- 172.21.15.45:0/646597220 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x56284c7f8530 0x56284c7fa9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:11.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.012+0000 7f994a000640 1 --2- 172.21.15.45:0/646597220 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x56284c7f8530 0x56284c7fa9d0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a73001a664bb6650 server_cookie=98c1401e0bae4ae7 in_seq=0 out_seq=0 2024-04-16T17:09:11.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.012+0000 7f9934c00640 10 client.16200 ms_handle_connect on v2:172.21.15.104:6838/3660070592 2024-04-16T17:09:11.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.013+0000 7f9934c00640 1 -- 172.21.15.45:0/646597220 <== mds.0 v2:172.21.15.104:6838/3660070592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56284c7516d0 con 0x56284c7f8530 2024-04-16T17:09:11.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.013+0000 7f9934c00640 10 client.16200 handle_command_reply: tid=0 2024-04-16T17:09:11.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.013+0000 7f995497e740 1 client.16200 shutdown 2024-04-16T17:09:11.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.013+0000 7f995497e740 1 -- 172.21.15.45:0/646597220 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56284c7dee90 msgr2=0x56284c7e5330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:11.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.013+0000 7f995497e740 1 --2- 172.21.15.45:0/646597220 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56284c7dee90 0x56284c7e5330 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f993c0efef0 tx=0x7f993c0035f0 comp rx=0 tx=0).stop 2024-04-16T17:09:11.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.013+0000 7f995497e740 1 -- 172.21.15.45:0/646597220 shutdown_connections 2024-04-16T17:09:11.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.013+0000 7f995497e740 1 --2- 172.21.15.45:0/646597220 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x56284c7f8530 0x56284c7fa9d0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:11.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.013+0000 7f995497e740 1 --2- 172.21.15.45:0/646597220 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56284c7df7b0 0x56284c7e5870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:11.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.013+0000 7f995497e740 1 --2- 172.21.15.45:0/646597220 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56284c7e8390 0x56284c7e5db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:11.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.013+0000 7f995497e740 1 --2- 172.21.15.45:0/646597220 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56284c7dee90 0x56284c7e5330 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:11.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.013+0000 7f995497e740 1 -- 172.21.15.45:0/646597220 >> 172.21.15.45:0/646597220 conn(0x56284c7d95c0 msgr2=0x56284c7db310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:09:11.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.013+0000 7f9934c00640 0 client.16200 ms_handle_reset on v2:172.21.15.104:6838/3660070592 2024-04-16T17:09:11.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.013+0000 7f995497e740 1 -- 172.21.15.45:0/646597220 shutdown_connections 2024-04-16T17:09:11.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.013+0000 7f995497e740 1 -- 172.21.15.45:0/646597220 wait complete. 2024-04-16T17:09:11.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.014+0000 7f995497e740 20 client.16200 trim_cache size 0 max 16384 2024-04-16T17:09:11.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.038+0000 7f995497e740 1 Processor -- start 2024-04-16T17:09:11.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.038+0000 7f995497e740 1 -- start start 2024-04-16T17:09:11.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.038+0000 7f995497e740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56284c7dee90 0x56284c7e7a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:11.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.038+0000 7f995497e740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56284c7df7b0 0x56284c7f4070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:11.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.038+0000 7f995497e740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56284c7e8390 0x56284c7d83e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:11.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.038+0000 7f995497e740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f994c1b3e40 con 0x56284c7dee90 2024-04-16T17:09:11.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.038+0000 7f995497e740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f994c1b1680 con 0x56284c7e8390 2024-04-16T17:09:11.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.038+0000 7f995497e740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f994c1af4b0 con 0x56284c7df7b0 2024-04-16T17:09:11.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.038+0000 7f994aa00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56284c7dee90 0x56284c7e7a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:11.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.038+0000 7f994a000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56284c7df7b0 0x56284c7f4070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:11.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.039+0000 7f994a000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56284c7df7b0 0x56284c7f4070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:58450/0 (socket says 172.21.15.45:58450) 2024-04-16T17:09:11.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.039+0000 7f994a000640 1 -- 172.21.15.45:0/4231586841 learned_addr learned my addr 172.21.15.45:0/4231586841 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:09:11.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.039+0000 7f994b400640 1 --2- 172.21.15.45:0/4231586841 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56284c7e8390 0x56284c7d83e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:11.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.039+0000 7f994a000640 1 -- 172.21.15.45:0/4231586841 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56284c7e8390 msgr2=0x56284c7d83e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:09:11.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.039+0000 7f994a000640 1 --2- 172.21.15.45:0/4231586841 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56284c7e8390 0x56284c7d83e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:11.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.039+0000 7f994a000640 1 -- 172.21.15.45:0/4231586841 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56284c7dee90 msgr2=0x56284c7e7a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:09:11.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.039+0000 7f994a000640 1 --2- 172.21.15.45:0/4231586841 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56284c7dee90 0x56284c7e7a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:11.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.039+0000 7f994a000640 1 -- 172.21.15.45:0/4231586841 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56284c416170 con 0x56284c7df7b0 2024-04-16T17:09:11.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.039+0000 7f994aa00640 1 --2- 172.21.15.45:0/4231586841 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56284c7dee90 0x56284c7e7a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:09:11.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.039+0000 7f994a000640 1 --2- 172.21.15.45:0/4231586841 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56284c7df7b0 0x56284c7f4070 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f9938013010 tx=0x7f993801b000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d386f28db3243083 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:09:11.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.039+0000 7f9937400640 1 -- 172.21.15.45:0/4231586841 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f993801dde0 con 0x56284c7df7b0 2024-04-16T17:09:11.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.039+0000 7f9937400640 1 -- 172.21.15.45:0/4231586841 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f993802b910 con 0x56284c7df7b0 2024-04-16T17:09:11.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.039+0000 7f9937400640 1 -- 172.21.15.45:0/4231586841 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9938004ad0 con 0x56284c7df7b0 2024-04-16T17:09:11.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.040+0000 7f995497e740 1 -- 172.21.15.45:0/4231586841 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56284c7df7b0 msgr2=0x56284c7f4070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:11.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.040+0000 7f995497e740 1 --2- 172.21.15.45:0/4231586841 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56284c7df7b0 0x56284c7f4070 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f9938013010 tx=0x7f993801b000 comp rx=0 tx=0).stop 2024-04-16T17:09:11.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.040+0000 7f995497e740 1 -- 172.21.15.45:0/4231586841 shutdown_connections 2024-04-16T17:09:11.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.040+0000 7f995497e740 1 --2- 172.21.15.45:0/4231586841 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56284c7df7b0 0x56284c7f4070 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:11.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.040+0000 7f995497e740 1 --2- 172.21.15.45:0/4231586841 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56284c7e8390 0x56284c7d83e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:11.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.040+0000 7f995497e740 1 --2- 172.21.15.45:0/4231586841 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56284c7dee90 0x56284c7e7a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:11.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.040+0000 7f995497e740 1 -- 172.21.15.45:0/4231586841 >> 172.21.15.45:0/4231586841 conn(0x56284c7d95c0 msgr2=0x56284c7d7cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:09:11.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.040+0000 7f995497e740 1 -- 172.21.15.45:0/4231586841 shutdown_connections 2024-04-16T17:09:11.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.040+0000 7f995497e740 1 -- 172.21.15.45:0/4231586841 wait complete. 2024-04-16T17:09:11.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.041+0000 7f995497e740 1 -- start start 2024-04-16T17:09:11.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.041+0000 7f995497e740 1 Processor -- start 2024-04-16T17:09:11.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.041+0000 7f995497e740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56284c7dee90 0x56284c7e1fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:11.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.041+0000 7f995497e740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56284c7df7b0 0x56284c7e2520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:11.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.041+0000 7f995497e740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56284c7e8390 0x56284c7e2a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:11.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.041+0000 7f995497e740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f994c1afb40 con 0x56284c7dee90 2024-04-16T17:09:11.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.041+0000 7f995497e740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f994c1b0fb0 con 0x56284c7e8390 2024-04-16T17:09:11.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.041+0000 7f995497e740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f994c1b0210 con 0x56284c7df7b0 2024-04-16T17:09:11.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.041+0000 7f995497e740 10 client.0 _ll_register_callbacks cb 0x7f993c004550 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:09:11.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.041+0000 7f994a000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56284c7df7b0 0x56284c7e2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:11.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.041+0000 7f994aa00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56284c7dee90 0x56284c7e1fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:11.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.042+0000 7f994a000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56284c7df7b0 0x56284c7e2520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:58454/0 (socket says 172.21.15.45:58454) 2024-04-16T17:09:11.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.042+0000 7f994a000640 1 -- 172.21.15.45:0/2315866342 learned_addr learned my addr 172.21.15.45:0/2315866342 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:09:11.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.042+0000 7f994b400640 1 --2- 172.21.15.45:0/2315866342 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56284c7e8390 0x56284c7e2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:11.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.042+0000 7f994a000640 1 -- 172.21.15.45:0/2315866342 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56284c7e8390 msgr2=0x56284c7e2a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:09:11.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.042+0000 7f994a000640 1 --2- 172.21.15.45:0/2315866342 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56284c7e8390 0x56284c7e2a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:11.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.042+0000 7f994a000640 1 -- 172.21.15.45:0/2315866342 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56284c7dee90 msgr2=0x56284c7e1fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:09:11.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.042+0000 7f994a000640 1 --2- 172.21.15.45:0/2315866342 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56284c7dee90 0x56284c7e1fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:11.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.042+0000 7f994a000640 1 -- 172.21.15.45:0/2315866342 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56284c85db10 con 0x56284c7df7b0 2024-04-16T17:09:11.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.042+0000 7f994b400640 1 --2- 172.21.15.45:0/2315866342 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56284c7e8390 0x56284c7e2a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:09:11.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.042+0000 7f994a000640 1 --2- 172.21.15.45:0/2315866342 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56284c7df7b0 0x56284c7e2520 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f993802e040 tx=0x7f9938009e20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1dc189c723734a58 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:09:11.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.042+0000 7f9934c00640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-04-16T17:09:11.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.042+0000 7f9934c00640 1 -- 172.21.15.45:0/2315866342 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f993802b4f0 con 0x56284c7df7b0 2024-04-16T17:09:11.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.043+0000 7f995497e740 1 -- 172.21.15.45:0/2315866342 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f993802b910 con 0x56284c7df7b0 2024-04-16T17:09:11.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.043+0000 7f9934c00640 1 -- 172.21.15.45:0/2315866342 <== mon.1 v2:172.21.15.117:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f993802b690 con 0x56284c7df7b0 2024-04-16T17:09:11.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.043+0000 7f9934c00640 1 -- 172.21.15.45:0/2315866342 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f9938044050 con 0x56284c7df7b0 2024-04-16T17:09:11.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.043+0000 7f995497e740 10 client.26332 fetch_fsmap learned FSMap version 29 2024-04-16T17:09:11.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.043+0000 7f995497e740 1 -- 172.21.15.45:0/2315866342 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56284c85ee30 con 0x56284c7df7b0 2024-04-16T17:09:11.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.043+0000 7f9934c00640 1 -- 172.21.15.45:0/2315866342 <== mon.1 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f993802b910 con 0x56284c7df7b0 2024-04-16T17:09:11.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.043+0000 7f9934c00640 1 -- 172.21.15.45:0/2315866342 <== mon.1 v2:172.21.15.117:3300/0 5 ==== fsmap(e 29) ==== 8366+0+0 (secure 0 0 0) 0x7f9938029600 con 0x56284c7df7b0 2024-04-16T17:09:11.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.043+0000 7f995497e740 10 client.26332 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T17:09:11.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.043+0000 7f995497e740 10 client.26332 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:09:11.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.043+0000 7f995497e740 20 client.26332 populate_metadata read hostname 'smithi045' 2024-04-16T17:09:11.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.043+0000 7f995497e740 1 --2- 172.21.15.45:0/2315866342 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x56284c85f670 0x56284c861a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:11.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.043+0000 7f995497e740 4 client.26332 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-16T17:09:11.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.043+0000 7f995497e740 1 -- 172.21.15.45:0/2315866342 --> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x56284c74b540 con 0x56284c85f670 2024-04-16T17:09:11.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.044+0000 7f994aa00640 1 --2- 172.21.15.45:0/2315866342 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x56284c85f670 0x56284c861a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:11.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.044+0000 7f994aa00640 1 --2- 172.21.15.45:0/2315866342 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x56284c85f670 0x56284c861a90 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dc74ada5ee99969b server_cookie=ae44810a6dd86c7b in_seq=0 out_seq=0 2024-04-16T17:09:11.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.044+0000 7f9934c00640 10 client.26332 ms_handle_connect on v2:172.21.15.104:6838/3660070592 2024-04-16T17:09:11.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.045+0000 7f9934c00640 1 -- 172.21.15.45:0/2315866342 <== mds.0 v2:172.21.15.104:6838/3660070592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x56284c74b540 con 0x56284c85f670 2024-04-16T17:09:11.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.045+0000 7f9934c00640 10 client.26332 handle_command_reply: tid=0 2024-04-16T17:09:11.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.045+0000 7f995497e740 1 client.26332 shutdown 2024-04-16T17:09:11.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.046+0000 7f995497e740 1 -- 172.21.15.45:0/2315866342 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56284c7df7b0 msgr2=0x56284c7e2520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:11.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.046+0000 7f995497e740 1 --2- 172.21.15.45:0/2315866342 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56284c7df7b0 0x56284c7e2520 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f993802e040 tx=0x7f9938009e20 comp rx=0 tx=0).stop 2024-04-16T17:09:11.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.046+0000 7f995497e740 1 -- 172.21.15.45:0/2315866342 shutdown_connections 2024-04-16T17:09:11.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.046+0000 7f995497e740 1 --2- 172.21.15.45:0/2315866342 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x56284c85f670 0x56284c861a90 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:11.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.046+0000 7f995497e740 1 --2- 172.21.15.45:0/2315866342 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56284c7df7b0 0x56284c7e2520 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:11.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.046+0000 7f995497e740 1 --2- 172.21.15.45:0/2315866342 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56284c7e8390 0x56284c7e2a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:11.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.046+0000 7f995497e740 1 --2- 172.21.15.45:0/2315866342 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56284c7dee90 0x56284c7e1fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:11.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.046+0000 7f9934c00640 0 client.26332 ms_handle_reset on v2:172.21.15.104:6838/3660070592 2024-04-16T17:09:11.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.046+0000 7f995497e740 1 -- 172.21.15.45:0/2315866342 >> 172.21.15.45:0/2315866342 conn(0x56284c7d95c0 msgr2=0x56284c7e5b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:09:11.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.046+0000 7f995497e740 1 -- 172.21.15.45:0/2315866342 shutdown_connections 2024-04-16T17:09:11.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.046+0000 7f995497e740 1 -- 172.21.15.45:0/2315866342 wait complete. 2024-04-16T17:09:11.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.046+0000 7f995497e740 20 client.26332 trim_cache size 0 max 16384 2024-04-16T17:09:11.049 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-16T17:09:11.049 INFO:teuthology.orchestra.run.smithi045.stdout: "return_code": 0, 2024-04-16T17:09:11.049 INFO:teuthology.orchestra.run.smithi045.stdout: "scrub_tag": "5ac5279e-5783-4f64-9f5b-8f25ed2ce3f4", 2024-04-16T17:09:11.049 INFO:teuthology.orchestra.run.smithi045.stdout: "mode": "asynchronous" 2024-04-16T17:09:11.049 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-16T17:09:11.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.047+0000 7f994be00640 1 -- 172.21.15.45:0/3504964432 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f99300a3c60 msgr2=0x7f99300a6120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:11.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.047+0000 7f994be00640 1 --2- 172.21.15.45:0/3504964432 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f99300a3c60 0x7f99300a6120 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x56284c7da750 tx=0x7f9938005f90 comp rx=0 tx=0).stop 2024-04-16T17:09:11.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.047+0000 7f994be00640 1 -- 172.21.15.45:0/3504964432 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f994c102110 msgr2=0x7f994c074300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:11.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.047+0000 7f994be00640 1 --2- 172.21.15.45:0/3504964432 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f994c102110 0x7f994c074300 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f993c02c170 tx=0x7f993c004360 comp rx=0 tx=0).stop 2024-04-16T17:09:11.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.047+0000 7f994be00640 1 -- 172.21.15.45:0/3504964432 shutdown_connections 2024-04-16T17:09:11.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.047+0000 7f994be00640 1 --2- 172.21.15.45:0/3504964432 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f99300a3c60 0x7f99300a6120 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:11.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.047+0000 7f994be00640 1 --2- 172.21.15.45:0/3504964432 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f994c1071e0 0x7f994c074840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:11.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.047+0000 7f994be00640 1 --2- 172.21.15.45:0/3504964432 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f994c074d80 0x7f994c1a69f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:11.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.047+0000 7f994be00640 1 --2- 172.21.15.45:0/3504964432 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f994c102110 0x7f994c074300 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:11.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.047+0000 7f994be00640 1 -- 172.21.15.45:0/3504964432 >> 172.21.15.45:0/3504964432 conn(0x7f994c0fd9d0 msgr2=0x7f994c0ffd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:09:11.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.047+0000 7f994be00640 1 -- 172.21.15.45:0/3504964432 shutdown_connections 2024-04-16T17:09:11.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.047+0000 7f994be00640 1 -- 172.21.15.45:0/3504964432 wait complete. 2024-04-16T17:09:11.069 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-16T17:09:11.070 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-16T17:09:11.224 INFO:tasks.workunit.client.0.smithi045.stdout: 1 36062 2.91 MB/sec warmup 73 sec latency 3071.331 ms 2024-04-16T17:09:11.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.278+0000 7f2a77200640 1 -- 172.21.15.45:0/3594028208 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2a78103060 msgr2=0x7f2a78103440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:11.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.278+0000 7f2a77200640 1 --2- 172.21.15.45:0/3594028208 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2a78103060 0x7f2a78103440 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f2a68009860 tx=0x7f2a6802bc10 comp rx=0 tx=0).stop 2024-04-16T17:09:11.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.278+0000 7f2a77200640 1 -- 172.21.15.45:0/3594028208 shutdown_connections 2024-04-16T17:09:11.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.278+0000 7f2a77200640 1 --2- 172.21.15.45:0/3594028208 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2a7810c480 0x7f2a7810e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:11.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.278+0000 7f2a77200640 1 --2- 172.21.15.45:0/3594028208 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2a78103980 0x7f2a7810be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:11.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.278+0000 7f2a77200640 1 --2- 172.21.15.45:0/3594028208 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2a78103060 0x7f2a78103440 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:11.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.278+0000 7f2a77200640 1 -- 172.21.15.45:0/3594028208 >> 172.21.15.45:0/3594028208 conn(0x7f2a780fc880 msgr2=0x7f2a780feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:09:11.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.278+0000 7f2a77200640 1 -- 172.21.15.45:0/3594028208 shutdown_connections 2024-04-16T17:09:11.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.278+0000 7f2a77200640 1 -- 172.21.15.45:0/3594028208 wait complete. 2024-04-16T17:09:11.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.279+0000 7f2a77200640 1 Processor -- start 2024-04-16T17:09:11.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.280+0000 7f2a77200640 1 -- start start 2024-04-16T17:09:11.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.280+0000 7f2a77200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2a78103060 0x7f2a7819e400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:11.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.280+0000 7f2a77200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2a78103980 0x7f2a7819e940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:11.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.280+0000 7f2a77200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2a7810c480 0x7f2a78198600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:11.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.280+0000 7f2a77200640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f2a78112260 con 0x7f2a78103980 2024-04-16T17:09:11.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.280+0000 7f2a77200640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f2a781120e0 con 0x7f2a78103060 2024-04-16T17:09:11.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.280+0000 7f2a77200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f2a781123e0 con 0x7f2a7810c480 2024-04-16T17:09:11.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.280+0000 7f2a75e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2a78103060 0x7f2a7819e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:11.282 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.280+0000 7f2a75400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2a78103980 0x7f2a7819e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:11.282 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.280+0000 7f2a75e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2a78103060 0x7f2a7819e400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:53962/0 (socket says 172.21.15.45:53962) 2024-04-16T17:09:11.282 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.280+0000 7f2a75e00640 1 -- 172.21.15.45:0/1706477792 learned_addr learned my addr 172.21.15.45:0/1706477792 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:09:11.282 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.280+0000 7f2a76800640 1 --2- 172.21.15.45:0/1706477792 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2a7810c480 0x7f2a78198600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:11.282 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.281+0000 7f2a75e00640 1 -- 172.21.15.45:0/1706477792 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2a7810c480 msgr2=0x7f2a78198600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:11.282 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.281+0000 7f2a75e00640 1 --2- 172.21.15.45:0/1706477792 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2a7810c480 0x7f2a78198600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:11.282 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.281+0000 7f2a75e00640 1 -- 172.21.15.45:0/1706477792 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2a78103980 msgr2=0x7f2a7819e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:11.282 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.281+0000 7f2a75e00640 1 --2- 172.21.15.45:0/1706477792 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2a78103980 0x7f2a7819e940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:11.282 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.281+0000 7f2a75e00640 1 -- 172.21.15.45:0/1706477792 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a78198d80 con 0x7f2a78103060 2024-04-16T17:09:11.282 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.281+0000 7f2a75e00640 1 --2- 172.21.15.45:0/1706477792 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2a78103060 0x7f2a7819e400 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f2a68037040 tx=0x7f2a680344c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:09:11.282 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.281+0000 7f2a6ea00640 1 -- 172.21.15.45:0/1706477792 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a68041020 con 0x7f2a78103060 2024-04-16T17:09:11.282 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.281+0000 7f2a6ea00640 1 -- 172.21.15.45:0/1706477792 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2a6802ca90 con 0x7f2a78103060 2024-04-16T17:09:11.282 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.281+0000 7f2a6ea00640 1 -- 172.21.15.45:0/1706477792 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a68039040 con 0x7f2a78103060 2024-04-16T17:09:11.282 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.282+0000 7f2a77200640 1 -- 172.21.15.45:0/1706477792 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a78199010 con 0x7f2a78103060 2024-04-16T17:09:11.282 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.282+0000 7f2a77200640 1 -- 172.21.15.45:0/1706477792 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a781a52c0 con 0x7f2a78103060 2024-04-16T17:09:11.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.285+0000 7f2a7fc05740 1 Processor -- start 2024-04-16T17:09:11.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.285+0000 7f2a7fc05740 1 -- start start 2024-04-16T17:09:11.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.285+0000 7f2a7fc05740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55675f4c4390 0x55675f4c4770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:11.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.285+0000 7f2a7fc05740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55675f4c4cb0 0x55675f4cd350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:11.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.285+0000 7f2a7fc05740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55675f4cd890 0x55675f4cfc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:11.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.285+0000 7f2a7fc05740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55675f42a120 con 0x55675f4c4cb0 2024-04-16T17:09:11.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.285+0000 7f2a7fc05740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55675f3d0340 con 0x55675f4cd890 2024-04-16T17:09:11.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.285+0000 7f2a7fc05740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55675f4260e0 con 0x55675f4c4390 2024-04-16T17:09:11.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.286+0000 7f2a76800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55675f4c4390 0x55675f4c4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:11.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.286+0000 7f2a76800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55675f4c4390 0x55675f4c4770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:58488/0 (socket says 172.21.15.45:58488) 2024-04-16T17:09:11.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.286+0000 7f2a76800640 1 -- 172.21.15.45:0/3244792692 learned_addr learned my addr 172.21.15.45:0/3244792692 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:09:11.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.286+0000 7f2a75400640 1 --2- 172.21.15.45:0/3244792692 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55675f4cd890 0x55675f4cfc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:11.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.286+0000 7f2a76800640 1 -- 172.21.15.45:0/3244792692 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55675f4cd890 msgr2=0x55675f4cfc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:09:11.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.286+0000 7f2a76800640 1 --2- 172.21.15.45:0/3244792692 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55675f4cd890 0x55675f4cfc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:11.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.286+0000 7f2a76800640 1 -- 172.21.15.45:0/3244792692 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55675f4c4cb0 msgr2=0x55675f4cd350 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-16T17:09:11.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.286+0000 7f2a76800640 1 --2- 172.21.15.45:0/3244792692 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55675f4c4cb0 0x55675f4cd350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:11.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.286+0000 7f2a76800640 1 -- 172.21.15.45:0/3244792692 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55675f2067d0 con 0x55675f4c4390 2024-04-16T17:09:11.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.286+0000 7f2a76800640 1 --2- 172.21.15.45:0/3244792692 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55675f4c4390 0x55675f4c4770 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f2a70009f50 tx=0x7f2a70011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1a7ac4867a0dff31 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:09:11.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.287+0000 7f2a63400640 1 -- 172.21.15.45:0/3244792692 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a70004d20 con 0x55675f4c4390 2024-04-16T17:09:11.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.287+0000 7f2a6ea00640 1 -- 172.21.15.45:0/1706477792 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f2a68044020 con 0x7f2a78103060 2024-04-16T17:09:11.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.287+0000 7f2a63400640 1 -- 172.21.15.45:0/3244792692 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2a70004050 con 0x55675f4c4390 2024-04-16T17:09:11.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.287+0000 7f2a63400640 1 -- 172.21.15.45:0/3244792692 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a70005280 con 0x55675f4c4390 2024-04-16T17:09:11.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.287+0000 7f2a7fc05740 1 -- 172.21.15.45:0/3244792692 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55675f4c4390 msgr2=0x55675f4c4770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:11.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.287+0000 7f2a7fc05740 1 --2- 172.21.15.45:0/3244792692 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55675f4c4390 0x55675f4c4770 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f2a70009f50 tx=0x7f2a70011810 comp rx=0 tx=0).stop 2024-04-16T17:09:11.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.287+0000 7f2a7fc05740 1 -- 172.21.15.45:0/3244792692 shutdown_connections 2024-04-16T17:09:11.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.287+0000 7f2a7fc05740 1 --2- 172.21.15.45:0/3244792692 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55675f4cd890 0x55675f4cfc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:11.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.287+0000 7f2a7fc05740 1 --2- 172.21.15.45:0/3244792692 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55675f4c4cb0 0x55675f4cd350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:11.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.287+0000 7f2a7fc05740 1 --2- 172.21.15.45:0/3244792692 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55675f4c4390 0x55675f4c4770 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:11.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.287+0000 7f2a7fc05740 1 -- 172.21.15.45:0/3244792692 >> 172.21.15.45:0/3244792692 conn(0x55675f4beac0 msgr2=0x55675f4beea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:09:11.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.288+0000 7f2a7fc05740 1 -- 172.21.15.45:0/3244792692 shutdown_connections 2024-04-16T17:09:11.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.288+0000 7f2a7fc05740 1 -- 172.21.15.45:0/3244792692 wait complete. 2024-04-16T17:09:11.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.288+0000 7f2a7fc05740 1 -- start start 2024-04-16T17:09:11.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.288+0000 7f2a6ea00640 1 --2- 172.21.15.45:0/1706477792 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f2a5c0a3cb0 0x7f2a5c0a6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:11.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.288+0000 7f2a7fc05740 1 Processor -- start 2024-04-16T17:09:11.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.289+0000 7f2a6ea00640 1 -- 172.21.15.45:0/1706477792 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f2a680f2120 con 0x7f2a78103060 2024-04-16T17:09:11.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.289+0000 7f2a76800640 1 --2- 172.21.15.45:0/1706477792 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f2a5c0a3cb0 0x7f2a5c0a6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:11.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.289+0000 7f2a7fc05740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55675f4c4390 0x55675f4d7b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:11.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.289+0000 7f2a7fc05740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55675f4c4cb0 0x55675f4d8080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:11.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.289+0000 7f2a7fc05740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55675f4cd890 0x55675f4d1f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:11.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.289+0000 7f2a7fc05740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55675f4d0680 con 0x55675f4c4cb0 2024-04-16T17:09:11.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.289+0000 7f2a7fc05740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55675f3e3710 con 0x55675f4c4390 2024-04-16T17:09:11.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.289+0000 7f2a7fc05740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55675f4d0800 con 0x55675f4cd890 2024-04-16T17:09:11.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.289+0000 7f2a7fc05740 10 client.0 _ll_register_callbacks cb 0x55675f3deaf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:09:11.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.289+0000 7f2a75e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55675f4c4390 0x55675f4d7b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:11.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.289+0000 7f2a75e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55675f4c4390 0x55675f4d7b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:53974/0 (socket says 172.21.15.45:53974) 2024-04-16T17:09:11.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.289+0000 7f2a75e00640 1 -- 172.21.15.45:0/2015834308 learned_addr learned my addr 172.21.15.45:0/2015834308 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:09:11.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.290+0000 7f2a75400640 1 --2- 172.21.15.45:0/2015834308 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55675f4c4cb0 0x55675f4d8080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:11.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.290+0000 7f2a76800640 1 --2- 172.21.15.45:0/1706477792 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f2a5c0a3cb0 0x7f2a5c0a6170 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f2a70009d60 tx=0x7f2a70005fd0 comp rx=0 tx=0).ready entity=mgr.25411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:09:11.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.290+0000 7f2a75e00640 1 -- 172.21.15.45:0/2015834308 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55675f4cd890 msgr2=0x55675f4d1f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:09:11.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.290+0000 7f2a75e00640 1 --2- 172.21.15.45:0/2015834308 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55675f4cd890 0x55675f4d1f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:11.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.290+0000 7f2a75e00640 1 -- 172.21.15.45:0/2015834308 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55675f4c4cb0 msgr2=0x55675f4d8080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:09:11.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.290+0000 7f2a75e00640 1 --2- 172.21.15.45:0/2015834308 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55675f4c4cb0 0x55675f4d8080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:11.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.290+0000 7f2a75e00640 1 -- 172.21.15.45:0/2015834308 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55675f13c3f0 con 0x55675f4c4390 2024-04-16T17:09:11.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.290+0000 7f2a75e00640 1 --2- 172.21.15.45:0/2015834308 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55675f4c4390 0x55675f4d7b40 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f2a6802c220 tx=0x7f2a680bd840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c75927a43f113b2 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:09:11.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.290+0000 7f2a57e00640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-04-16T17:09:11.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.290+0000 7f2a57e00640 1 -- 172.21.15.45:0/2015834308 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a680c6a90 con 0x55675f4c4390 2024-04-16T17:09:11.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.291+0000 7f2a7fc05740 1 -- 172.21.15.45:0/2015834308 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2a70004050 con 0x55675f4c4390 2024-04-16T17:09:11.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.291+0000 7f2a57e00640 1 -- 172.21.15.45:0/2015834308 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f2a680be080 con 0x55675f4c4390 2024-04-16T17:09:11.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.291+0000 7f2a57e00640 1 -- 172.21.15.45:0/2015834308 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f2a680be280 con 0x55675f4c4390 2024-04-16T17:09:11.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.291+0000 7f2a7fc05740 10 client.26366 fetch_fsmap learned FSMap version 29 2024-04-16T17:09:11.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.291+0000 7f2a7fc05740 1 -- 172.21.15.45:0/2015834308 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55675f0fc0f0 con 0x55675f4c4390 2024-04-16T17:09:11.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.291+0000 7f2a57e00640 1 -- 172.21.15.45:0/2015834308 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2a70004050 con 0x55675f4c4390 2024-04-16T17:09:11.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.291+0000 7f2a57e00640 1 -- 172.21.15.45:0/2015834308 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 29) ==== 8366+0+0 (secure 0 0 0) 0x7f2a6803c030 con 0x55675f4c4390 2024-04-16T17:09:11.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.291+0000 7f2a7fc05740 10 client.26366 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T17:09:11.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.291+0000 7f2a7fc05740 10 client.26366 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:09:11.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.291+0000 7f2a7fc05740 20 client.26366 populate_metadata read hostname 'smithi045' 2024-04-16T17:09:11.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.292+0000 7f2a7fc05740 1 --2- 172.21.15.45:0/2015834308 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55675f4e2030 0x55675f4e4450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:11.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.292+0000 7f2a7fc05740 4 client.26366 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:09:11.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.292+0000 7f2a7fc05740 1 -- 172.21.15.45:0/2015834308 --> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55675f437b40 con 0x55675f4e2030 2024-04-16T17:09:11.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.292+0000 7f2a75400640 1 --2- 172.21.15.45:0/2015834308 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55675f4e2030 0x55675f4e4450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:11.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.293+0000 7f2a57e00640 10 client.26366 ms_handle_connect on v2:172.21.15.104:6838/3660070592 2024-04-16T17:09:11.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.293+0000 7f2a75400640 1 --2- 172.21.15.45:0/2015834308 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55675f4e2030 0x55675f4e4450 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5430476e8a838e65 server_cookie=a1a97b78f52f0d58 in_seq=0 out_seq=0 2024-04-16T17:09:11.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.293+0000 7f2a57e00640 1 -- 172.21.15.45:0/2015834308 <== mds.0 v2:172.21.15.104:6838/3660070592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55675f437b40 con 0x55675f4e2030 2024-04-16T17:09:11.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.293+0000 7f2a57e00640 10 client.26366 handle_command_reply: tid=0 2024-04-16T17:09:11.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.293+0000 7f2a7fc05740 1 client.26366 shutdown 2024-04-16T17:09:11.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.294+0000 7f2a7fc05740 1 -- 172.21.15.45:0/2015834308 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55675f4c4390 msgr2=0x55675f4d7b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:11.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.294+0000 7f2a7fc05740 1 --2- 172.21.15.45:0/2015834308 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55675f4c4390 0x55675f4d7b40 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f2a6802c220 tx=0x7f2a680bd840 comp rx=0 tx=0).stop 2024-04-16T17:09:11.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.294+0000 7f2a7fc05740 1 -- 172.21.15.45:0/2015834308 shutdown_connections 2024-04-16T17:09:11.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.294+0000 7f2a7fc05740 1 --2- 172.21.15.45:0/2015834308 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55675f4e2030 0x55675f4e4450 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:11.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.294+0000 7f2a7fc05740 1 --2- 172.21.15.45:0/2015834308 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55675f4cd890 0x55675f4d1f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:11.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.294+0000 7f2a7fc05740 1 --2- 172.21.15.45:0/2015834308 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55675f4c4cb0 0x55675f4d8080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:11.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.294+0000 7f2a7fc05740 1 --2- 172.21.15.45:0/2015834308 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55675f4c4390 0x55675f4d7b40 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:11.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.294+0000 7f2a7fc05740 1 -- 172.21.15.45:0/2015834308 >> 172.21.15.45:0/2015834308 conn(0x55675f4beac0 msgr2=0x55675f4c0810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:09:11.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.294+0000 7f2a57e00640 0 client.26366 ms_handle_reset on v2:172.21.15.104:6838/3660070592 2024-04-16T17:09:11.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.294+0000 7f2a7fc05740 1 -- 172.21.15.45:0/2015834308 shutdown_connections 2024-04-16T17:09:11.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.294+0000 7f2a7fc05740 1 -- 172.21.15.45:0/2015834308 wait complete. 2024-04-16T17:09:11.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.294+0000 7f2a7fc05740 20 client.26366 trim_cache size 0 max 16384 2024-04-16T17:09:11.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.314+0000 7f2a7fc05740 1 Processor -- start 2024-04-16T17:09:11.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.314+0000 7f2a7fc05740 1 -- start start 2024-04-16T17:09:11.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.314+0000 7f2a7fc05740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55675f4c4390 0x55675f4ca020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:11.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.314+0000 7f2a7fc05740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55675f4c4cb0 0x55675f4d1e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:11.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.314+0000 7f2a7fc05740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55675f4cd890 0x55675f4d7f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:11.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.314+0000 7f2a7fc05740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f2a781af910 con 0x55675f4cd890 2024-04-16T17:09:11.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.314+0000 7f2a7fc05740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f2a781ad150 con 0x55675f4c4cb0 2024-04-16T17:09:11.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.314+0000 7f2a7fc05740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f2a781aaf80 con 0x55675f4c4390 2024-04-16T17:09:11.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.314+0000 7f2a76800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55675f4cd890 0x55675f4d7f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:11.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.314+0000 7f2a76800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55675f4cd890 0x55675f4d7f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57808/0 (socket says 172.21.15.45:57808) 2024-04-16T17:09:11.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.314+0000 7f2a76800640 1 -- 172.21.15.45:0/3122574403 learned_addr learned my addr 172.21.15.45:0/3122574403 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:09:11.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.314+0000 7f2a75400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55675f4c4cb0 0x55675f4d1e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:11.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.314+0000 7f2a75e00640 1 --2- 172.21.15.45:0/3122574403 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55675f4c4390 0x55675f4ca020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:11.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.315+0000 7f2a75400640 1 -- 172.21.15.45:0/3122574403 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55675f4c4390 msgr2=0x55675f4ca020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:09:11.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.315+0000 7f2a75400640 1 --2- 172.21.15.45:0/3122574403 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55675f4c4390 0x55675f4ca020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:11.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.315+0000 7f2a75400640 1 -- 172.21.15.45:0/3122574403 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55675f4cd890 msgr2=0x55675f4d7f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:09:11.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.315+0000 7f2a75400640 1 --2- 172.21.15.45:0/3122574403 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55675f4cd890 0x55675f4d7f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:11.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.315+0000 7f2a75400640 1 -- 172.21.15.45:0/3122574403 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55675f4be5f0 con 0x55675f4c4cb0 2024-04-16T17:09:11.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.315+0000 7f2a76800640 1 --2- 172.21.15.45:0/3122574403 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55675f4cd890 0x55675f4d7f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:09:11.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.315+0000 7f2a75400640 1 --2- 172.21.15.45:0/3122574403 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55675f4c4cb0 0x55675f4d1e50 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f2a640096c0 tx=0x7f2a64009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=acbc38cbbf7b4bb2 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:09:11.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.316+0000 7f2a62a00640 1 -- 172.21.15.45:0/3122574403 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a6401d430 con 0x55675f4c4cb0 2024-04-16T17:09:11.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.316+0000 7f2a62a00640 1 -- 172.21.15.45:0/3122574403 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2a64010430 con 0x55675f4c4cb0 2024-04-16T17:09:11.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.316+0000 7f2a62a00640 1 -- 172.21.15.45:0/3122574403 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a6401c610 con 0x55675f4c4cb0 2024-04-16T17:09:11.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.316+0000 7f2a7fc05740 1 -- 172.21.15.45:0/3122574403 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55675f4c4cb0 msgr2=0x55675f4d1e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:11.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.316+0000 7f2a7fc05740 1 --2- 172.21.15.45:0/3122574403 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55675f4c4cb0 0x55675f4d1e50 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f2a640096c0 tx=0x7f2a64009260 comp rx=0 tx=0).stop 2024-04-16T17:09:11.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.316+0000 7f2a7fc05740 1 -- 172.21.15.45:0/3122574403 shutdown_connections 2024-04-16T17:09:11.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.316+0000 7f2a7fc05740 1 --2- 172.21.15.45:0/3122574403 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55675f4cd890 0x55675f4d7f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:11.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.316+0000 7f2a7fc05740 1 --2- 172.21.15.45:0/3122574403 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55675f4c4cb0 0x55675f4d1e50 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:11.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.316+0000 7f2a7fc05740 1 --2- 172.21.15.45:0/3122574403 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55675f4c4390 0x55675f4ca020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:11.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.316+0000 7f2a7fc05740 1 -- 172.21.15.45:0/3122574403 >> 172.21.15.45:0/3122574403 conn(0x55675f4beac0 msgr2=0x55675f4bd290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:09:11.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.316+0000 7f2a7fc05740 1 -- 172.21.15.45:0/3122574403 shutdown_connections 2024-04-16T17:09:11.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.316+0000 7f2a7fc05740 1 -- 172.21.15.45:0/3122574403 wait complete. 2024-04-16T17:09:11.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.317+0000 7f2a7fc05740 1 -- start start 2024-04-16T17:09:11.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.317+0000 7f2a7fc05740 1 Processor -- start 2024-04-16T17:09:11.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.317+0000 7f2a7fc05740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55675f4c4390 0x55675f4dc310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:11.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.317+0000 7f2a7fc05740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55675f4c4cb0 0x55675f4dc850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:11.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.318+0000 7f2a7fc05740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55675f4cd890 0x55675f541570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:11.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.318+0000 7f2a7fc05740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f2a781ab610 con 0x55675f4cd890 2024-04-16T17:09:11.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.318+0000 7f2a7fc05740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f2a781aca80 con 0x55675f4c4cb0 2024-04-16T17:09:11.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.318+0000 7f2a7fc05740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f2a781abce0 con 0x55675f4c4390 2024-04-16T17:09:11.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.318+0000 7f2a7fc05740 10 client.0 _ll_register_callbacks cb 0x7f2a680bd960 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:09:11.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.318+0000 7f2a76800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55675f4cd890 0x55675f541570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:11.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.318+0000 7f2a75400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55675f4c4cb0 0x55675f4dc850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:11.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.318+0000 7f2a76800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55675f4cd890 0x55675f541570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57814/0 (socket says 172.21.15.45:57814) 2024-04-16T17:09:11.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.318+0000 7f2a76800640 1 -- 172.21.15.45:0/3366269257 learned_addr learned my addr 172.21.15.45:0/3366269257 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:09:11.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.318+0000 7f2a75e00640 1 --2- 172.21.15.45:0/3366269257 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55675f4c4390 0x55675f4dc310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:11.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.318+0000 7f2a76800640 1 -- 172.21.15.45:0/3366269257 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55675f4c4390 msgr2=0x55675f4dc310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:09:11.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.318+0000 7f2a76800640 1 --2- 172.21.15.45:0/3366269257 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55675f4c4390 0x55675f4dc310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:11.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.318+0000 7f2a76800640 1 -- 172.21.15.45:0/3366269257 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55675f4c4cb0 msgr2=0x55675f4dc850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:09:11.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.318+0000 7f2a76800640 1 --2- 172.21.15.45:0/3366269257 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55675f4c4cb0 0x55675f4dc850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:11.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.318+0000 7f2a76800640 1 -- 172.21.15.45:0/3366269257 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55675f541b60 con 0x55675f4cd890 2024-04-16T17:09:11.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.319+0000 7f2a76800640 1 --2- 172.21.15.45:0/3366269257 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55675f4cd890 0x55675f541570 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f2a70011d10 tx=0x7f2a70005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2568409b295baf4 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:09:11.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.319+0000 7f2a57e00640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-16T17:09:11.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.319+0000 7f2a57e00640 1 -- 172.21.15.45:0/3366269257 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a70034030 con 0x55675f4cd890 2024-04-16T17:09:11.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.319+0000 7f2a57e00640 1 -- 172.21.15.45:0/3366269257 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f2a70005720 con 0x55675f4cd890 2024-04-16T17:09:11.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.319+0000 7f2a7fc05740 1 -- 172.21.15.45:0/3366269257 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2a64010430 con 0x55675f4cd890 2024-04-16T17:09:11.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.319+0000 7f2a57e00640 1 -- 172.21.15.45:0/3366269257 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f2a70041050 con 0x55675f4cd890 2024-04-16T17:09:11.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.319+0000 7f2a7fc05740 10 client.16221 fetch_fsmap learned FSMap version 29 2024-04-16T17:09:11.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.319+0000 7f2a7fc05740 1 -- 172.21.15.45:0/3366269257 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55675f548ef0 con 0x55675f4cd890 2024-04-16T17:09:11.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.320+0000 7f2a57e00640 1 -- 172.21.15.45:0/3366269257 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2a64010430 con 0x55675f4cd890 2024-04-16T17:09:11.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.320+0000 7f2a57e00640 1 -- 172.21.15.45:0/3366269257 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 29) ==== 8366+0+0 (secure 0 0 0) 0x7f2a70025a30 con 0x55675f4cd890 2024-04-16T17:09:11.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.320+0000 7f2a7fc05740 10 client.16221 fetch_fsmap finished waiting for FSMap version 29 2024-04-16T17:09:11.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.320+0000 7f2a7fc05740 10 client.16221 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:09:11.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.320+0000 7f2a7fc05740 20 client.16221 populate_metadata read hostname 'smithi045' 2024-04-16T17:09:11.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.320+0000 7f2a7fc05740 1 --2- 172.21.15.45:0/3366269257 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55675f5495b0 0x55675f54b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:11.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.320+0000 7f2a7fc05740 4 client.16221 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-16T17:09:11.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.320+0000 7f2a7fc05740 1 -- 172.21.15.45:0/3366269257 --> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55675f4319b0 con 0x55675f5495b0 2024-04-16T17:09:11.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.320+0000 7f2a75e00640 1 --2- 172.21.15.45:0/3366269257 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55675f5495b0 0x55675f54b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:11.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.321+0000 7f2a75e00640 1 --2- 172.21.15.45:0/3366269257 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55675f5495b0 0x55675f54b9d0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e73a4855a9e2f91 server_cookie=d7c7e9bc2c6873d1 in_seq=0 out_seq=0 2024-04-16T17:09:11.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.321+0000 7f2a57e00640 10 client.16221 ms_handle_connect on v2:172.21.15.104:6838/3660070592 2024-04-16T17:09:11.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.321+0000 7f2a57e00640 1 -- 172.21.15.45:0/3366269257 <== mds.0 v2:172.21.15.104:6838/3660070592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+265 (crc 0 0 0) 0x55675f4319b0 con 0x55675f5495b0 2024-04-16T17:09:11.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.321+0000 7f2a57e00640 10 client.16221 handle_command_reply: tid=0 2024-04-16T17:09:11.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.321+0000 7f2a7fc05740 1 client.16221 shutdown 2024-04-16T17:09:11.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.321+0000 7f2a7fc05740 1 -- 172.21.15.45:0/3366269257 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55675f4cd890 msgr2=0x55675f541570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:11.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.321+0000 7f2a7fc05740 1 --2- 172.21.15.45:0/3366269257 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55675f4cd890 0x55675f541570 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f2a70011d10 tx=0x7f2a70005bd0 comp rx=0 tx=0).stop 2024-04-16T17:09:11.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.321+0000 7f2a7fc05740 1 -- 172.21.15.45:0/3366269257 shutdown_connections 2024-04-16T17:09:11.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.321+0000 7f2a7fc05740 1 --2- 172.21.15.45:0/3366269257 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55675f4cd890 0x55675f541570 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:11.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.321+0000 7f2a7fc05740 1 --2- 172.21.15.45:0/3366269257 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55675f4c4cb0 0x55675f4dc850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:11.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.321+0000 7f2a7fc05740 1 --2- 172.21.15.45:0/3366269257 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55675f5495b0 0x55675f54b9d0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:11.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.321+0000 7f2a7fc05740 1 --2- 172.21.15.45:0/3366269257 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55675f4c4390 0x55675f4dc310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:11.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.321+0000 7f2a7fc05740 1 -- 172.21.15.45:0/3366269257 >> 172.21.15.45:0/3366269257 conn(0x55675f4beac0 msgr2=0x55675f4bdb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:09:11.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.321+0000 7f2a57e00640 0 client.16221 ms_handle_reset on v2:172.21.15.104:6838/3660070592 2024-04-16T17:09:11.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.322+0000 7f2a7fc05740 1 -- 172.21.15.45:0/3366269257 shutdown_connections 2024-04-16T17:09:11.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.322+0000 7f2a7fc05740 1 -- 172.21.15.45:0/3366269257 wait complete. 2024-04-16T17:09:11.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.322+0000 7f2a7fc05740 20 client.16221 trim_cache size 0 max 16384 2024-04-16T17:09:11.325 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-16T17:09:11.325 INFO:teuthology.orchestra.run.smithi045.stdout: "status": "scrub active (1 inodes in the stack)", 2024-04-16T17:09:11.325 INFO:teuthology.orchestra.run.smithi045.stdout: "scrubs": { 2024-04-16T17:09:11.325 INFO:teuthology.orchestra.run.smithi045.stdout: "5ac5279e-5783-4f64-9f5b-8f25ed2ce3f4": { 2024-04-16T17:09:11.325 INFO:teuthology.orchestra.run.smithi045.stdout: "path": "/", 2024-04-16T17:09:11.325 INFO:teuthology.orchestra.run.smithi045.stdout: "tag": "5ac5279e-5783-4f64-9f5b-8f25ed2ce3f4", 2024-04-16T17:09:11.325 INFO:teuthology.orchestra.run.smithi045.stdout: "options": "recursive,force" 2024-04-16T17:09:11.325 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:09:11.326 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:09:11.326 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-16T17:09:11.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.322+0000 7f2a77200640 1 -- 172.21.15.45:0/1706477792 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f2a5c0a3cb0 msgr2=0x7f2a5c0a6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:11.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.322+0000 7f2a77200640 1 --2- 172.21.15.45:0/1706477792 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f2a5c0a3cb0 0x7f2a5c0a6170 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f2a70009d60 tx=0x7f2a70005fd0 comp rx=0 tx=0).stop 2024-04-16T17:09:11.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.322+0000 7f2a77200640 1 -- 172.21.15.45:0/1706477792 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2a78103060 msgr2=0x7f2a7819e400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:11.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.322+0000 7f2a77200640 1 --2- 172.21.15.45:0/1706477792 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2a78103060 0x7f2a7819e400 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f2a68037040 tx=0x7f2a680344c0 comp rx=0 tx=0).stop 2024-04-16T17:09:11.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.323+0000 7f2a77200640 1 -- 172.21.15.45:0/1706477792 shutdown_connections 2024-04-16T17:09:11.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.323+0000 7f2a77200640 1 --2- 172.21.15.45:0/1706477792 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f2a5c0a3cb0 0x7f2a5c0a6170 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:11.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.323+0000 7f2a77200640 1 --2- 172.21.15.45:0/1706477792 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2a7810c480 0x7f2a78198600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:11.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.323+0000 7f2a77200640 1 --2- 172.21.15.45:0/1706477792 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2a78103980 0x7f2a7819e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:11.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.323+0000 7f2a77200640 1 --2- 172.21.15.45:0/1706477792 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2a78103060 0x7f2a7819e400 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:11.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.323+0000 7f2a77200640 1 -- 172.21.15.45:0/1706477792 >> 172.21.15.45:0/1706477792 conn(0x7f2a780fc880 msgr2=0x7f2a780fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:09:11.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.323+0000 7f2a77200640 1 -- 172.21.15.45:0/1706477792 shutdown_connections 2024-04-16T17:09:11.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:11.323+0000 7f2a77200640 1 -- 172.21.15.45:0/1706477792 wait complete. 2024-04-16T17:09:11.342 INFO:tasks.cephfs.filesystem:scrub status for tag:5ac5279e-5783-4f64-9f5b-8f25ed2ce3f4 - {'path': '/', 'tag': '5ac5279e-5783-4f64-9f5b-8f25ed2ce3f4', 'options': 'recursive,force'} 2024-04-16T17:09:12.224 INFO:tasks.workunit.client.0.smithi045.stdout: 1 36062 2.87 MB/sec warmup 74 sec latency 4071.407 ms 2024-04-16T17:09:12.555 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:09:12 smithi117 ceph-mon[36730]: scrub queued for path: / 2024-04-16T17:09:12.555 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:09:12 smithi117 ceph-mon[36730]: scrub summary: idle+waiting paths [/] 2024-04-16T17:09:12.555 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:09:12 smithi117 ceph-mon[36730]: scrub summary: active paths [/] 2024-04-16T17:09:12.555 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:09:12 smithi117 ceph-mon[36730]: scrub summary: active paths [/] 2024-04-16T17:09:12.555 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:09:12 smithi117 ceph-mon[36730]: scrub summary: active paths [/] 2024-04-16T17:09:12.555 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:09:12 smithi117 ceph-mon[36730]: scrub summary: idle+waiting paths [/] 2024-04-16T17:09:12.555 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:09:12 smithi117 ceph-mon[36730]: scrub summary: idle+waiting paths [/] 2024-04-16T17:09:12.555 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:09:12 smithi117 ceph-mon[36730]: pgmap v76: 129 pgs: 129 active+clean; 28 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 573 KiB/s rd, 353 KiB/s wr, 115 op/s 2024-04-16T17:09:12.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:09:12 smithi104 ceph-mon[36457]: scrub queued for path: / 2024-04-16T17:09:12.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:09:12 smithi104 ceph-mon[36457]: scrub summary: idle+waiting paths [/] 2024-04-16T17:09:12.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:09:12 smithi104 ceph-mon[36457]: scrub summary: active paths [/] 2024-04-16T17:09:12.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:09:12 smithi104 ceph-mon[36457]: scrub summary: active paths [/] 2024-04-16T17:09:12.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:09:12 smithi104 ceph-mon[36457]: scrub summary: active paths [/] 2024-04-16T17:09:12.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:09:12 smithi104 ceph-mon[36457]: scrub summary: idle+waiting paths [/] 2024-04-16T17:09:12.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:09:12 smithi104 ceph-mon[36457]: scrub summary: idle+waiting paths [/] 2024-04-16T17:09:12.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:09:12 smithi104 ceph-mon[36457]: pgmap v76: 129 pgs: 129 active+clean; 28 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 573 KiB/s rd, 353 KiB/s wr, 115 op/s 2024-04-16T17:09:12.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:09:12 smithi045 ceph-mon[31676]: scrub queued for path: / 2024-04-16T17:09:12.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:09:12 smithi045 ceph-mon[31676]: scrub summary: idle+waiting paths [/] 2024-04-16T17:09:12.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:09:12 smithi045 ceph-mon[31676]: scrub summary: active paths [/] 2024-04-16T17:09:12.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:09:12 smithi045 ceph-mon[31676]: scrub summary: active paths [/] 2024-04-16T17:09:12.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:09:12 smithi045 ceph-mon[31676]: scrub summary: active paths [/] 2024-04-16T17:09:12.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:09:12 smithi045 ceph-mon[31676]: scrub summary: idle+waiting paths [/] 2024-04-16T17:09:12.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:09:12 smithi045 ceph-mon[31676]: scrub summary: idle+waiting paths [/] 2024-04-16T17:09:12.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:09:12 smithi045 ceph-mon[31676]: pgmap v76: 129 pgs: 129 active+clean; 28 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 573 KiB/s rd, 353 KiB/s wr, 115 op/s 2024-04-16T17:09:13.224 INFO:tasks.workunit.client.0.smithi045.stdout: 1 36062 2.83 MB/sec warmup 75 sec latency 5071.496 ms 2024-04-16T17:09:14.224 INFO:tasks.workunit.client.0.smithi045.stdout: 1 36062 2.79 MB/sec warmup 76 sec latency 6071.590 ms 2024-04-16T17:09:14.555 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:09:14 smithi117 ceph-mon[36730]: pgmap v77: 129 pgs: 129 active+clean; 28 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 502 KiB/s rd, 251 KiB/s wr, 72 op/s 2024-04-16T17:09:14.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:09:14 smithi104 ceph-mon[36457]: pgmap v77: 129 pgs: 129 active+clean; 28 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 502 KiB/s rd, 251 KiB/s wr, 72 op/s 2024-04-16T17:09:14.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:09:14 smithi045 ceph-mon[31676]: pgmap v77: 129 pgs: 129 active+clean; 28 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 502 KiB/s rd, 251 KiB/s wr, 72 op/s 2024-04-16T17:09:15.225 INFO:tasks.workunit.client.0.smithi045.stdout: 1 36062 2.76 MB/sec warmup 77 sec latency 7071.720 ms 2024-04-16T17:09:16.225 INFO:tasks.workunit.client.0.smithi045.stdout: 1 36169 2.72 MB/sec warmup 78 sec latency 7584.294 ms 2024-04-16T17:09:16.516 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:09:16 smithi045 ceph-mon[31676]: pgmap v78: 129 pgs: 129 active+clean; 28 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 502 KiB/s rd, 251 KiB/s wr, 72 op/s 2024-04-16T17:09:16.517 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:09:16 smithi045 ceph-mon[31676]: scrub summary: active paths [/] 2024-04-16T17:09:16.517 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:09:16 smithi045 ceph-mon[31676]: scrub summary: idle+waiting paths [/] 2024-04-16T17:09:16.555 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:09:16 smithi117 ceph-mon[36730]: pgmap v78: 129 pgs: 129 active+clean; 28 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 502 KiB/s rd, 251 KiB/s wr, 72 op/s 2024-04-16T17:09:16.555 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:09:16 smithi117 ceph-mon[36730]: scrub summary: active paths [/] 2024-04-16T17:09:16.555 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:09:16 smithi117 ceph-mon[36730]: scrub summary: idle+waiting paths [/] 2024-04-16T17:09:16.677 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:09:16 smithi104 ceph-mon[36457]: pgmap v78: 129 pgs: 129 active+clean; 28 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 502 KiB/s rd, 251 KiB/s wr, 72 op/s 2024-04-16T17:09:16.677 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:09:16 smithi104 ceph-mon[36457]: scrub summary: active paths [/] 2024-04-16T17:09:16.677 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:09:16 smithi104 ceph-mon[36457]: scrub summary: idle+waiting paths [/] 2024-04-16T17:09:17.225 INFO:tasks.workunit.client.0.smithi045.stdout: 1 36517 2.70 MB/sec warmup 79 sec latency 255.743 ms 2024-04-16T17:09:17.555 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:09:17 smithi117 ceph-mon[36730]: scrub summary: idle+waiting paths [/] 2024-04-16T17:09:17.677 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:09:17 smithi104 ceph-mon[36457]: scrub summary: idle+waiting paths [/] 2024-04-16T17:09:18.225 INFO:tasks.workunit.client.0.smithi045.stdout: 1 36811 2.68 MB/sec warmup 80 sec latency 98.115 ms 2024-04-16T17:09:18.555 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:09:18 smithi117 ceph-mon[36730]: pgmap v79: 129 pgs: 129 active+clean; 30 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 510 KiB/s rd, 464 KiB/s wr, 81 op/s 2024-04-16T17:09:18.677 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:09:18 smithi104 ceph-mon[36457]: pgmap v79: 129 pgs: 129 active+clean; 30 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 510 KiB/s rd, 464 KiB/s wr, 81 op/s 2024-04-16T17:09:19.225 INFO:tasks.workunit.client.0.smithi045.stdout: 1 37701 2.69 MB/sec warmup 81 sec latency 29.963 ms 2024-04-16T17:09:20.225 INFO:tasks.workunit.client.0.smithi045.stdout: 1 38843 2.76 MB/sec warmup 82 sec latency 23.944 ms 2024-04-16T17:09:20.555 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:09:20 smithi117 ceph-mon[36730]: pgmap v80: 129 pgs: 129 active+clean; 30 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 288 KiB/s wr, 44 op/s 2024-04-16T17:09:20.677 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:09:20 smithi104 ceph-mon[36457]: pgmap v80: 129 pgs: 129 active+clean; 30 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 288 KiB/s wr, 44 op/s 2024-04-16T17:09:21.225 INFO:tasks.workunit.client.0.smithi045.stdout: 1 39394 2.79 MB/sec warmup 83 sec latency 44.115 ms 2024-04-16T17:09:21.649 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:09:17 smithi045 ceph-mon[31676]: scrub summary: idle+waiting paths [/] 2024-04-16T17:09:21.649 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:09:18 smithi045 ceph-mon[31676]: pgmap v79: 129 pgs: 129 active+clean; 30 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 510 KiB/s rd, 464 KiB/s wr, 81 op/s 2024-04-16T17:09:21.649 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:09:20 smithi045 ceph-mon[31676]: pgmap v80: 129 pgs: 129 active+clean; 30 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 288 KiB/s wr, 44 op/s 2024-04-16T17:09:22.225 INFO:tasks.workunit.client.0.smithi045.stdout: 1 39501 2.75 MB/sec warmup 84 sec latency 431.718 ms 2024-04-16T17:09:22.341 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:09:22 smithi045 ceph-mon[31676]: pgmap v81: 129 pgs: 129 active+clean; 30 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 528 KiB/s rd, 552 KiB/s wr, 84 op/s 2024-04-16T17:09:22.341 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:09:22 smithi045 ceph-mon[31676]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-16T17:09:22.555 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:09:22 smithi117 ceph-mon[36730]: pgmap v81: 129 pgs: 129 active+clean; 30 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 528 KiB/s rd, 552 KiB/s wr, 84 op/s 2024-04-16T17:09:22.555 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:09:22 smithi117 ceph-mon[36730]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-16T17:09:22.677 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:09:22 smithi104 ceph-mon[36457]: pgmap v81: 129 pgs: 129 active+clean; 30 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 528 KiB/s rd, 552 KiB/s wr, 84 op/s 2024-04-16T17:09:22.677 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:09:22 smithi104 ceph-mon[36457]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-16T17:09:22.680 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:22.679+0000 7f5b53200640 1 -- 172.21.15.45:0/111439447 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5b54102fc0 msgr2=0x7f5b541033a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:22.680 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:22.679+0000 7f5b53200640 1 --2- 172.21.15.45:0/111439447 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5b54102fc0 0x7f5b541033a0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f5b44009900 tx=0x7f5b4402bcd0 comp rx=0 tx=0).stop 2024-04-16T17:09:22.680 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:22.679+0000 7f5b53200640 1 -- 172.21.15.45:0/111439447 shutdown_connections 2024-04-16T17:09:22.680 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:22.679+0000 7f5b53200640 1 --2- 172.21.15.45:0/111439447 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5b541038e0 0x7f5b5410bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:22.681 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:22.679+0000 7f5b53200640 1 --2- 172.21.15.45:0/111439447 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5b5410c3e0 0x7f5b5410e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:22.681 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:22.679+0000 7f5b53200640 1 --2- 172.21.15.45:0/111439447 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5b54102fc0 0x7f5b541033a0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:22.681 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:22.679+0000 7f5b53200640 1 -- 172.21.15.45:0/111439447 >> 172.21.15.45:0/111439447 conn(0x7f5b540fc820 msgr2=0x7f5b540fec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:09:22.681 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:22.679+0000 7f5b53200640 1 -- 172.21.15.45:0/111439447 shutdown_connections 2024-04-16T17:09:22.681 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:22.679+0000 7f5b53200640 1 -- 172.21.15.45:0/111439447 wait complete. 2024-04-16T17:09:22.683 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:22.680+0000 7f5b53200640 1 Processor -- start 2024-04-16T17:09:22.683 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:22.680+0000 7f5b53200640 1 -- start start 2024-04-16T17:09:22.683 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:22.681+0000 7f5b53200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5b54102fc0 0x7f5b541a0690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:22.683 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:22.681+0000 7f5b53200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5b541038e0 0x7f5b541a0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:22.683 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:22.681+0000 7f5b53200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5b5410c3e0 0x7f5b5419a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:22.684 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:22.681+0000 7f5b53200640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f5b541121f0 con 0x7f5b54102fc0 2024-04-16T17:09:22.684 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:22.681+0000 7f5b53200640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f5b54112070 con 0x7f5b5410c3e0 2024-04-16T17:09:22.684 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:22.681+0000 7f5b53200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f5b54112370 con 0x7f5b541038e0 2024-04-16T17:09:22.684 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:22.681+0000 7f5b51e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5b54102fc0 0x7f5b541a0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:22.684 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:22.681+0000 7f5b52800640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5b5410c3e0 0x7f5b5419a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:22.684 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:22.681+0000 7f5b51e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5b54102fc0 0x7f5b541a0690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:40082/0 (socket says 172.21.15.45:40082) 2024-04-16T17:09:22.684 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:22.681+0000 7f5b51e00640 1 -- 172.21.15.45:0/3746200796 learned_addr learned my addr 172.21.15.45:0/3746200796 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:09:22.684 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:22.681+0000 7f5b4be00640 1 --2- 172.21.15.45:0/3746200796 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5b541038e0 0x7f5b541a0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:22.684 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:22.681+0000 7f5b51e00640 1 -- 172.21.15.45:0/3746200796 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5b541038e0 msgr2=0x7f5b541a0bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:22.684 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:22.681+0000 7f5b51e00640 1 --2- 172.21.15.45:0/3746200796 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5b541038e0 0x7f5b541a0bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:22.684 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:22.681+0000 7f5b51e00640 1 -- 172.21.15.45:0/3746200796 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5b5410c3e0 msgr2=0x7f5b5419a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:22.684 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:22.681+0000 7f5b51e00640 1 --2- 172.21.15.45:0/3746200796 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5b5410c3e0 0x7f5b5419a750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:22.684 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:22.681+0000 7f5b51e00640 1 -- 172.21.15.45:0/3746200796 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b5419b010 con 0x7f5b54102fc0 2024-04-16T17:09:22.684 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:22.681+0000 7f5b52800640 1 --2- 172.21.15.45:0/3746200796 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5b5410c3e0 0x7f5b5419a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:09:22.684 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:22.682+0000 7f5b51e00640 1 --2- 172.21.15.45:0/3746200796 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5b54102fc0 0x7f5b541a0690 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f5b44000c00 tx=0x7f5b44004430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:09:22.685 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:22.682+0000 7f5b4aa00640 1 -- 172.21.15.45:0/3746200796 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b44042020 con 0x7f5b54102fc0 2024-04-16T17:09:22.685 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:22.682+0000 7f5b53200640 1 -- 172.21.15.45:0/3746200796 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b5419b2a0 con 0x7f5b54102fc0 2024-04-16T17:09:22.685 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:22.682+0000 7f5b4aa00640 1 -- 172.21.15.45:0/3746200796 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5b440054e0 con 0x7f5b54102fc0 2024-04-16T17:09:22.685 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:22.682+0000 7f5b4aa00640 1 -- 172.21.15.45:0/3746200796 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b44003c90 con 0x7f5b54102fc0 2024-04-16T17:09:22.685 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:22.682+0000 7f5b53200640 1 -- 172.21.15.45:0/3746200796 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b541a7610 con 0x7f5b54102fc0 2024-04-16T17:09:22.687 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:22.686+0000 7f5b4aa00640 1 -- 172.21.15.45:0/3746200796 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f5b44045020 con 0x7f5b54102fc0 2024-04-16T17:09:22.688 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:22.687+0000 7f5b5bda4740 1 Processor -- start 2024-04-16T17:09:22.688 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:22.687+0000 7f5b5bda4740 1 -- start start 2024-04-16T17:09:22.690 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:22.687+0000 7f5b5bda4740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5575496727f0 0x557549671540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:22.690 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:22.688+0000 7f5b5bda4740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557549671a80 0x557549671f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:22.690 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:22.688+0000 7f5b5bda4740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55754967b000 0x55754967d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:22.690 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:22.688+0000 7f5b5bda4740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5575495d6780 con 0x557549671a80 2024-04-16T17:09:22.690 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:22.688+0000 7f5b5bda4740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55754957cf40 con 0x55754967b000 2024-04-16T17:09:22.690 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:22.688+0000 7f5b5bda4740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5575495d2740 con 0x5575496727f0 2024-04-16T17:09:22.690 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:22.688+0000 7f5b52800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5575496727f0 0x557549671540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:22.690 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:22.688+0000 7f5b52800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5575496727f0 0x557549671540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:47544/0 (socket says 172.21.15.45:47544) 2024-04-16T17:09:22.690 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:22.688+0000 7f5b52800640 1 -- 172.21.15.45:0/3851654923 learned_addr learned my addr 172.21.15.45:0/3851654923 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:09:22.690 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:22.688+0000 7f5b4be00640 1 --2- 172.21.15.45:0/3851654923 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55754967b000 0x55754967d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:22.690 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:22.688+0000 7f5b51e00640 1 --2- 172.21.15.45:0/3851654923 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557549671a80 0x557549671f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:22.690 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:22.688+0000 7f5b4aa00640 1 --2- 172.21.15.45:0/3746200796 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f5b380a3d10 0x7f5b380a61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:22.690 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:22.688+0000 7f5b52800640 1 -- 172.21.15.45:0/3851654923 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55754967b000 msgr2=0x55754967d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:09:22.691 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:22.688+0000 7f5b52800640 1 --2- 172.21.15.45:0/3851654923 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55754967b000 0x55754967d3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:22.691 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:22.688+0000 7f5b52800640 1 -- 172.21.15.45:0/3851654923 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557549671a80 msgr2=0x557549671f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:09:22.691 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:22.688+0000 7f5b52800640 1 --2- 172.21.15.45:0/3851654923 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557549671a80 0x557549671f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:22.691 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:22.688+0000 7f5b52800640 1 -- 172.21.15.45:0/3851654923 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5575492e9210 con 0x5575496727f0 2024-04-16T17:09:22.691 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:22.688+0000 7f5b4aa00640 1 -- 172.21.15.45:0/3746200796 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f5b440f2800 con 0x7f5b54102fc0 2024-04-16T17:09:22.691 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:22.688+0000 7f5b4be00640 1 --2- 172.21.15.45:0/3851654923 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55754967b000 0x55754967d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:09:22.691 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:22.689+0000 7f5b52800640 1 --2- 172.21.15.45:0/3851654923 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5575496727f0 0x557549671540 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f5b4c0107a0 tx=0x7f5b4c010c60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1843f5164a0fb68c server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:09:22.691 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:22.689+0000 7f5b52800640 1 --2- 172.21.15.45:0/3746200796 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f5b380a3d10 0x7f5b380a61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:22.692 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:22.689+0000 7f5b43400640 1 -- 172.21.15.45:0/3851654923 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b4c01f040 con 0x5575496727f0 2024-04-16T17:09:22.692 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:22.689+0000 7f5b43400640 1 -- 172.21.15.45:0/3851654923 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5b4c00ee30 con 0x5575496727f0 2024-04-16T17:09:22.692 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:22.689+0000 7f5b43400640 1 -- 172.21.15.45:0/3851654923 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b4c01ab10 con 0x5575496727f0 2024-04-16T17:09:22.692 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:22.690+0000 7f5b5bda4740 1 -- 172.21.15.45:0/3851654923 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5575496727f0 msgr2=0x557549671540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:22.692 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:22.690+0000 7f5b52800640 1 --2- 172.21.15.45:0/3746200796 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f5b380a3d10 0x7f5b380a61d0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f5b4c01a000 tx=0x7f5b4c0197a0 comp rx=0 tx=0).ready entity=mgr.25411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:09:22.692 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:22.690+0000 7f5b5bda4740 1 --2- 172.21.15.45:0/3851654923 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5575496727f0 0x557549671540 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f5b4c0107a0 tx=0x7f5b4c010c60 comp rx=0 tx=0).stop 2024-04-16T17:09:22.693 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:22.692+0000 7f5b5bda4740 1 -- 172.21.15.45:0/3851654923 shutdown_connections 2024-04-16T17:09:22.693 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:22.692+0000 7f5b5bda4740 1 --2- 172.21.15.45:0/3851654923 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55754967b000 0x55754967d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:22.693 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:22.692+0000 7f5b5bda4740 1 --2- 172.21.15.45:0/3851654923 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557549671a80 0x557549671f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:22.693 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:22.692+0000 7f5b5bda4740 1 --2- 172.21.15.45:0/3851654923 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5575496727f0 0x557549671540 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:22.693 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:22.692+0000 7f5b5bda4740 1 -- 172.21.15.45:0/3851654923 >> 172.21.15.45:0/3851654923 conn(0x55754960c5d0 msgr2=0x55754966c770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:09:22.693 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:22.692+0000 7f5b5bda4740 1 -- 172.21.15.45:0/3851654923 shutdown_connections 2024-04-16T17:09:22.693 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:22.692+0000 7f5b5bda4740 1 -- 172.21.15.45:0/3851654923 wait complete. 2024-04-16T17:09:22.694 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:22.692+0000 7f5b5bda4740 1 -- start start 2024-04-16T17:09:22.694 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:22.693+0000 7f5b5bda4740 1 Processor -- start 2024-04-16T17:09:22.694 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:22.694+0000 7f5b5bda4740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557549671a80 0x55754967aab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:22.695 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:22.694+0000 7f5b5bda4740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5575496727f0 0x557549677e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:22.695 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:22.694+0000 7f5b5bda4740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55754967b000 0x557549678580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:22.695 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:22.694+0000 7f5b5bda4740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55754967dd30 con 0x557549671a80 2024-04-16T17:09:22.695 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:22.694+0000 7f5b5bda4740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x557549590390 con 0x5575496727f0 2024-04-16T17:09:22.695 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:22.694+0000 7f5b5bda4740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55754967deb0 con 0x55754967b000 2024-04-16T17:09:22.696 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:22.695+0000 7f5b5bda4740 10 client.0 _ll_register_callbacks cb 0x557549579f60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:09:22.696 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:22.695+0000 7f5b52800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55754967b000 0x557549678580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:22.696 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:22.695+0000 7f5b52800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55754967b000 0x557549678580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:47552/0 (socket says 172.21.15.45:47552) 2024-04-16T17:09:22.696 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:22.695+0000 7f5b52800640 1 -- 172.21.15.45:0/3865863125 learned_addr learned my addr 172.21.15.45:0/3865863125 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:09:22.697 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:22.695+0000 7f5b51e00640 1 --2- 172.21.15.45:0/3865863125 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557549671a80 0x55754967aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:22.697 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:22.695+0000 7f5b52800640 1 -- 172.21.15.45:0/3865863125 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5575496727f0 msgr2=0x557549677e40 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-16T17:09:22.697 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:22.695+0000 7f5b52800640 1 --2- 172.21.15.45:0/3865863125 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5575496727f0 0x557549677e40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:22.697 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:22.695+0000 7f5b52800640 1 -- 172.21.15.45:0/3865863125 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557549671a80 msgr2=0x55754967aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:09:22.697 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:22.695+0000 7f5b52800640 1 --2- 172.21.15.45:0/3865863125 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557549671a80 0x55754967aab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:22.697 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:22.696+0000 7f5b52800640 1 -- 172.21.15.45:0/3865863125 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5575493b5620 con 0x55754967b000 2024-04-16T17:09:22.697 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:22.696+0000 7f5b51e00640 1 --2- 172.21.15.45:0/3865863125 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557549671a80 0x55754967aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:09:22.698 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:22.696+0000 7f5b52800640 1 --2- 172.21.15.45:0/3865863125 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55754967b000 0x557549678580 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f5b4c01a720 tx=0x7f5b4c023000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=26af59a7e9918d8b server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:09:22.698 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:22.696+0000 7f5b37e00640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-04-16T17:09:22.698 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:22.696+0000 7f5b37e00640 1 -- 172.21.15.45:0/3865863125 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b4c01f040 con 0x55754967b000 2024-04-16T17:09:22.698 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:22.697+0000 7f5b5bda4740 1 -- 172.21.15.45:0/3865863125 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5b4c00ee30 con 0x55754967b000 2024-04-16T17:09:22.698 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:22.697+0000 7f5b37e00640 1 -- 172.21.15.45:0/3865863125 <== mon.1 v2:172.21.15.117:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f5b4c01a7d0 con 0x55754967b000 2024-04-16T17:09:22.698 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:22.697+0000 7f5b37e00640 1 -- 172.21.15.45:0/3865863125 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f5b4c01a9d0 con 0x55754967b000 2024-04-16T17:09:22.698 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:22.697+0000 7f5b5bda4740 10 client.26359 fetch_fsmap learned FSMap version 30 2024-04-16T17:09:22.699 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:22.697+0000 7f5b5bda4740 1 -- 172.21.15.45:0/3865863125 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x557549689ea0 con 0x55754967b000 2024-04-16T17:09:22.699 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:22.698+0000 7f5b37e00640 1 -- 172.21.15.45:0/3865863125 <== mon.1 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5b4c00ee30 con 0x55754967b000 2024-04-16T17:09:22.699 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:22.698+0000 7f5b37e00640 1 -- 172.21.15.45:0/3865863125 <== mon.1 v2:172.21.15.117:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f5b4c030070 con 0x55754967b000 2024-04-16T17:09:22.699 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:22.698+0000 7f5b5bda4740 10 client.26359 fetch_fsmap finished waiting for FSMap version 30 2024-04-16T17:09:22.699 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:22.698+0000 7f5b5bda4740 10 client.26359 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T17:09:22.699 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:22.698+0000 7f5b5bda4740 20 client.26359 populate_metadata read hostname 'smithi045' 2024-04-16T17:09:22.700 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:22.698+0000 7f5b5bda4740 1 --2- 172.21.15.45:0/3865863125 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55754968b2b0 0x55754968d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:22.700 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:22.698+0000 7f5b5bda4740 4 client.26359 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:09:22.700 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:22.698+0000 7f5b5bda4740 1 -- 172.21.15.45:0/3865863125 --> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5575495e41a0 con 0x55754968b2b0 2024-04-16T17:09:22.700 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:22.699+0000 7f5b51e00640 1 --2- 172.21.15.45:0/3865863125 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55754968b2b0 0x55754968d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:22.700 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:22.699+0000 7f5b37e00640 10 client.26359 ms_handle_connect on v2:172.21.15.104:6838/3660070592 2024-04-16T17:09:22.700 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:22.699+0000 7f5b51e00640 1 --2- 172.21.15.45:0/3865863125 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55754968b2b0 0x55754968d6a0 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2241a19b7cabefe3 server_cookie=5e13989f38c8793 in_seq=0 out_seq=0 2024-04-16T17:09:22.701 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:22.700+0000 7f5b37e00640 1 -- 172.21.15.45:0/3865863125 <== mds.0 v2:172.21.15.104:6838/3660070592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5575495e41a0 con 0x55754968b2b0 2024-04-16T17:09:22.701 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:22.700+0000 7f5b37e00640 10 client.26359 handle_command_reply: tid=0 2024-04-16T17:09:22.701 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:22.700+0000 7f5b5bda4740 1 client.26359 shutdown 2024-04-16T17:09:22.701 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:22.700+0000 7f5b5bda4740 1 -- 172.21.15.45:0/3865863125 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55754967b000 msgr2=0x557549678580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:22.701 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:22.700+0000 7f5b5bda4740 1 --2- 172.21.15.45:0/3865863125 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55754967b000 0x557549678580 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f5b4c01a720 tx=0x7f5b4c023000 comp rx=0 tx=0).stop 2024-04-16T17:09:22.702 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:22.700+0000 7f5b5bda4740 1 -- 172.21.15.45:0/3865863125 shutdown_connections 2024-04-16T17:09:22.702 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:22.700+0000 7f5b5bda4740 1 --2- 172.21.15.45:0/3865863125 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55754968b2b0 0x55754968d6a0 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:22.702 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:22.701+0000 7f5b5bda4740 1 --2- 172.21.15.45:0/3865863125 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55754967b000 0x557549678580 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:22.702 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:22.701+0000 7f5b5bda4740 1 --2- 172.21.15.45:0/3865863125 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5575496727f0 0x557549677e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:22.702 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:22.701+0000 7f5b5bda4740 1 --2- 172.21.15.45:0/3865863125 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557549671a80 0x55754967aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:22.702 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:22.701+0000 7f5b37e00640 0 client.26359 ms_handle_reset on v2:172.21.15.104:6838/3660070592 2024-04-16T17:09:22.703 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:22.701+0000 7f5b5bda4740 1 -- 172.21.15.45:0/3865863125 >> 172.21.15.45:0/3865863125 conn(0x55754960c5d0 msgr2=0x55754966e4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:09:22.703 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:22.701+0000 7f5b5bda4740 1 -- 172.21.15.45:0/3865863125 shutdown_connections 2024-04-16T17:09:22.703 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:22.701+0000 7f5b5bda4740 1 -- 172.21.15.45:0/3865863125 wait complete. 2024-04-16T17:09:22.703 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:22.701+0000 7f5b5bda4740 20 client.26359 trim_cache size 0 max 16384 2024-04-16T17:09:22.718 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:22.717+0000 7f5b5bda4740 1 Processor -- start 2024-04-16T17:09:22.718 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:22.717+0000 7f5b5bda4740 1 -- start start 2024-04-16T17:09:22.719 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:22.718+0000 7f5b5bda4740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557549671a80 0x557549676fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:22.719 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:22.718+0000 7f5b5bda4740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5575496727f0 0x55754966e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:22.719 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:22.718+0000 7f5b5bda4740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55754967b000 0x55754967a570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:22.719 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:22.718+0000 7f5b5bda4740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f5b541b3f00 con 0x55754967b000 2024-04-16T17:09:22.719 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:22.718+0000 7f5b4be00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5575496727f0 0x55754966e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:22.720 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:22.718+0000 7f5b51e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557549671a80 0x557549676fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:22.720 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:22.718+0000 7f5b4be00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5575496727f0 0x55754966e2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:47092/0 (socket says 172.21.15.45:47092) 2024-04-16T17:09:22.720 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:22.718+0000 7f5b5bda4740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f5b541b1710 con 0x5575496727f0 2024-04-16T17:09:22.720 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:22.718+0000 7f5b4be00640 1 -- 172.21.15.45:0/3422855039 learned_addr learned my addr 172.21.15.45:0/3422855039 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:09:22.720 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:22.718+0000 7f5b5bda4740 1 -- 172.21.15.45:0/3422855039 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f5b541af540 con 0x557549671a80 2024-04-16T17:09:22.720 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:22.718+0000 7f5b52800640 1 --2- 172.21.15.45:0/3422855039 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55754967b000 0x55754967a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:22.720 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:22.719+0000 7f5b51e00640 1 -- 172.21.15.45:0/3422855039 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5575496727f0 msgr2=0x55754966e2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:09:22.720 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:22.719+0000 7f5b51e00640 1 --2- 172.21.15.45:0/3422855039 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5575496727f0 0x55754966e2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:22.720 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:22.719+0000 7f5b51e00640 1 -- 172.21.15.45:0/3422855039 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55754967b000 msgr2=0x55754967a570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:09:22.720 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:22.719+0000 7f5b51e00640 1 --2- 172.21.15.45:0/3422855039 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55754967b000 0x55754967a570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:22.720 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:22.719+0000 7f5b51e00640 1 -- 172.21.15.45:0/3422855039 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557549676390 con 0x557549671a80 2024-04-16T17:09:22.720 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:22.719+0000 7f5b4be00640 1 --2- 172.21.15.45:0/3422855039 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5575496727f0 0x55754966e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:09:22.720 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:22.719+0000 7f5b52800640 1 --2- 172.21.15.45:0/3422855039 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55754967b000 0x55754967a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:09:22.722 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:22.719+0000 7f5b51e00640 1 --2- 172.21.15.45:0/3422855039 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557549671a80 0x557549676fa0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f5b440efcb0 tx=0x7f5b4402bcd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9610b83a6f1d95e2 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:09:22.722 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:22.719+0000 7f5b42a00640 1 -- 172.21.15.45:0/3422855039 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b44005690 con 0x557549671a80 2024-04-16T17:09:22.722 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:22.719+0000 7f5b42a00640 1 -- 172.21.15.45:0/3422855039 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5b44003e30 con 0x557549671a80 2024-04-16T17:09:22.722 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:22.719+0000 7f5b42a00640 1 -- 172.21.15.45:0/3422855039 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b440bc800 con 0x557549671a80 2024-04-16T17:09:22.722 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:22.720+0000 7f5b5bda4740 1 -- 172.21.15.45:0/3422855039 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557549671a80 msgr2=0x557549676fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:22.722 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:22.720+0000 7f5b5bda4740 1 --2- 172.21.15.45:0/3422855039 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557549671a80 0x557549676fa0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f5b440efcb0 tx=0x7f5b4402bcd0 comp rx=0 tx=0).stop 2024-04-16T17:09:22.722 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:22.720+0000 7f5b5bda4740 1 -- 172.21.15.45:0/3422855039 shutdown_connections 2024-04-16T17:09:22.722 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:22.720+0000 7f5b5bda4740 1 --2- 172.21.15.45:0/3422855039 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55754967b000 0x55754967a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:22.722 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:22.721+0000 7f5b5bda4740 1 --2- 172.21.15.45:0/3422855039 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5575496727f0 0x55754966e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:22.722 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:22.721+0000 7f5b5bda4740 1 --2- 172.21.15.45:0/3422855039 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557549671a80 0x557549676fa0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:22.722 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:22.721+0000 7f5b5bda4740 1 -- 172.21.15.45:0/3422855039 >> 172.21.15.45:0/3422855039 conn(0x55754960c5d0 msgr2=0x55754966dba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:09:22.722 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:22.721+0000 7f5b5bda4740 1 -- 172.21.15.45:0/3422855039 shutdown_connections 2024-04-16T17:09:22.723 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:22.721+0000 7f5b5bda4740 1 -- 172.21.15.45:0/3422855039 wait complete. 2024-04-16T17:09:22.723 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:22.721+0000 7f5b5bda4740 1 -- start start 2024-04-16T17:09:22.725 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:22.721+0000 7f5b5bda4740 1 Processor -- start 2024-04-16T17:09:22.725 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:22.722+0000 7f5b5bda4740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557549671a80 0x5575496e9a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:22.725 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:22.722+0000 7f5b5bda4740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5575496727f0 0x5575496e9fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:22.726 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:22.722+0000 7f5b5bda4740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55754967b000 0x55754967f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:22.726 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:22.722+0000 7f5b5bda4740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f5b541afbd0 con 0x5575496727f0 2024-04-16T17:09:22.726 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:22.722+0000 7f5b5bda4740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f5b541b1040 con 0x55754967b000 2024-04-16T17:09:22.726 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:22.722+0000 7f5b5bda4740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f5b541b02a0 con 0x557549671a80 2024-04-16T17:09:22.726 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:22.722+0000 7f5b5bda4740 10 client.0 _ll_register_callbacks cb 0x7f5b4c0105f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:09:22.726 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:22.722+0000 7f5b51e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557549671a80 0x5575496e9a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:22.726 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:22.722+0000 7f5b51e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557549671a80 0x5575496e9a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:47564/0 (socket says 172.21.15.45:47564) 2024-04-16T17:09:22.726 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:22.722+0000 7f5b51e00640 1 -- 172.21.15.45:0/4277816449 learned_addr learned my addr 172.21.15.45:0/4277816449 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:09:22.726 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:22.722+0000 7f5b4be00640 1 --2- 172.21.15.45:0/4277816449 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5575496727f0 0x5575496e9fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:22.726 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:22.722+0000 7f5b52800640 1 --2- 172.21.15.45:0/4277816449 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55754967b000 0x55754967f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:22.726 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:22.722+0000 7f5b51e00640 1 -- 172.21.15.45:0/4277816449 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55754967b000 msgr2=0x55754967f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:09:22.726 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:22.722+0000 7f5b51e00640 1 --2- 172.21.15.45:0/4277816449 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55754967b000 0x55754967f4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:22.727 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:22.723+0000 7f5b51e00640 1 -- 172.21.15.45:0/4277816449 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5575496727f0 msgr2=0x5575496e9fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:09:22.727 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:22.723+0000 7f5b51e00640 1 --2- 172.21.15.45:0/4277816449 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5575496727f0 0x5575496e9fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:22.727 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:22.723+0000 7f5b51e00640 1 -- 172.21.15.45:0/4277816449 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55754967fcb0 con 0x557549671a80 2024-04-16T17:09:22.727 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:22.723+0000 7f5b51e00640 1 --2- 172.21.15.45:0/4277816449 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557549671a80 0x5575496e9a60 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f5b44006010 tx=0x7f5b44003760 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7fb7d65f7dd8a15b server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:09:22.727 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:22.723+0000 7f5b37e00640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-04-16T17:09:22.727 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:22.723+0000 7f5b37e00640 1 -- 172.21.15.45:0/4277816449 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b440be500 con 0x557549671a80 2024-04-16T17:09:22.727 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:22.723+0000 7f5b37e00640 1 -- 172.21.15.45:0/4277816449 <== mon.1 v2:172.21.15.117:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f5b440bab70 con 0x557549671a80 2024-04-16T17:09:22.727 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:22.723+0000 7f5b5bda4740 1 -- 172.21.15.45:0/4277816449 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5b44003e30 con 0x557549671a80 2024-04-16T17:09:22.727 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:22.723+0000 7f5b37e00640 1 -- 172.21.15.45:0/4277816449 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f5b440bec00 con 0x557549671a80 2024-04-16T17:09:22.727 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:22.724+0000 7f5b5bda4740 10 client.26365 fetch_fsmap learned FSMap version 30 2024-04-16T17:09:22.727 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:22.724+0000 7f5b5bda4740 1 -- 172.21.15.45:0/4277816449 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5575496f2300 con 0x557549671a80 2024-04-16T17:09:22.727 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:22.724+0000 7f5b37e00640 1 -- 172.21.15.45:0/4277816449 <== mon.1 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5b44003e30 con 0x557549671a80 2024-04-16T17:09:22.727 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:22.724+0000 7f5b37e00640 1 -- 172.21.15.45:0/4277816449 <== mon.1 v2:172.21.15.117:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f5b440f1070 con 0x557549671a80 2024-04-16T17:09:22.727 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:22.724+0000 7f5b5bda4740 10 client.26365 fetch_fsmap finished waiting for FSMap version 30 2024-04-16T17:09:22.728 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:22.724+0000 7f5b5bda4740 10 client.26365 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T17:09:22.728 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:22.724+0000 7f5b5bda4740 20 client.26365 populate_metadata read hostname 'smithi045' 2024-04-16T17:09:22.728 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:22.724+0000 7f5b5bda4740 1 --2- 172.21.15.45:0/4277816449 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x5575496f27a0 0x5575496f4bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:22.728 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:22.724+0000 7f5b5bda4740 4 client.26365 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-16T17:09:22.728 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:22.724+0000 7f5b5bda4740 1 -- 172.21.15.45:0/4277816449 --> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5575495de010 con 0x5575496f27a0 2024-04-16T17:09:22.728 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:22.724+0000 7f5b4be00640 1 --2- 172.21.15.45:0/4277816449 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x5575496f27a0 0x5575496f4bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:22.728 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:22.725+0000 7f5b4be00640 1 --2- 172.21.15.45:0/4277816449 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x5575496f27a0 0x5575496f4bc0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=533ac92d23f189fa server_cookie=6505af7c1c72ddb6 in_seq=0 out_seq=0 2024-04-16T17:09:22.728 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:22.725+0000 7f5b37e00640 10 client.26365 ms_handle_connect on v2:172.21.15.104:6838/3660070592 2024-04-16T17:09:22.780 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:22.779+0000 7f5b37e00640 1 -- 172.21.15.45:0/4277816449 <== mds.0 v2:172.21.15.104:6838/3660070592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5575495de010 con 0x5575496f27a0 2024-04-16T17:09:22.780 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:22.779+0000 7f5b37e00640 10 client.26365 handle_command_reply: tid=0 2024-04-16T17:09:22.780 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:22.779+0000 7f5b5bda4740 1 client.26365 shutdown 2024-04-16T17:09:22.780 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:22.779+0000 7f5b5bda4740 1 -- 172.21.15.45:0/4277816449 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557549671a80 msgr2=0x5575496e9a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:22.780 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:22.779+0000 7f5b5bda4740 1 --2- 172.21.15.45:0/4277816449 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557549671a80 0x5575496e9a60 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f5b44006010 tx=0x7f5b44003760 comp rx=0 tx=0).stop 2024-04-16T17:09:22.781 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:22.779+0000 7f5b5bda4740 1 -- 172.21.15.45:0/4277816449 shutdown_connections 2024-04-16T17:09:22.781 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:22.779+0000 7f5b5bda4740 1 --2- 172.21.15.45:0/4277816449 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55754967b000 0x55754967f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:22.781 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:22.779+0000 7f5b5bda4740 1 --2- 172.21.15.45:0/4277816449 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5575496727f0 0x5575496e9fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:22.781 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:22.779+0000 7f5b5bda4740 1 --2- 172.21.15.45:0/4277816449 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x5575496f27a0 0x5575496f4bc0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:22.781 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:22.779+0000 7f5b5bda4740 1 --2- 172.21.15.45:0/4277816449 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557549671a80 0x5575496e9a60 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:22.781 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:22.779+0000 7f5b37e00640 0 client.26365 ms_handle_reset on v2:172.21.15.104:6838/3660070592 2024-04-16T17:09:22.781 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:22.779+0000 7f5b5bda4740 1 -- 172.21.15.45:0/4277816449 >> 172.21.15.45:0/4277816449 conn(0x55754960c5d0 msgr2=0x55754966dae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:09:22.781 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:22.779+0000 7f5b5bda4740 1 -- 172.21.15.45:0/4277816449 shutdown_connections 2024-04-16T17:09:22.781 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:22.779+0000 7f5b5bda4740 1 -- 172.21.15.45:0/4277816449 wait complete. 2024-04-16T17:09:22.781 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:22.779+0000 7f5b5bda4740 20 client.26365 trim_cache size 0 max 16384 2024-04-16T17:09:22.781 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-04-16T17:09:22.782 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-04-16T17:09:22.782 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-04-16T17:09:22.782 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-04-16T17:09:22.782 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:22.780+0000 7f5b53200640 1 -- 172.21.15.45:0/3746200796 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f5b380a3d10 msgr2=0x7f5b380a61d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:22.782 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:22.780+0000 7f5b53200640 1 --2- 172.21.15.45:0/3746200796 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f5b380a3d10 0x7f5b380a61d0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f5b4c01a000 tx=0x7f5b4c0197a0 comp rx=0 tx=0).stop 2024-04-16T17:09:22.782 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:22.780+0000 7f5b53200640 1 -- 172.21.15.45:0/3746200796 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5b54102fc0 msgr2=0x7f5b541a0690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:22.782 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:22.780+0000 7f5b53200640 1 --2- 172.21.15.45:0/3746200796 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5b54102fc0 0x7f5b541a0690 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f5b44000c00 tx=0x7f5b44004430 comp rx=0 tx=0).stop 2024-04-16T17:09:22.783 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:22.780+0000 7f5b53200640 1 -- 172.21.15.45:0/3746200796 shutdown_connections 2024-04-16T17:09:22.783 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:22.780+0000 7f5b53200640 1 --2- 172.21.15.45:0/3746200796 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f5b380a3d10 0x7f5b380a61d0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:22.783 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:22.780+0000 7f5b53200640 1 --2- 172.21.15.45:0/3746200796 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5b541038e0 0x7f5b541a0bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:22.783 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:22.780+0000 7f5b53200640 1 --2- 172.21.15.45:0/3746200796 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5b5410c3e0 0x7f5b5419a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:22.783 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:22.780+0000 7f5b53200640 1 --2- 172.21.15.45:0/3746200796 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5b54102fc0 0x7f5b541a0690 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:22.783 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:22.780+0000 7f5b53200640 1 -- 172.21.15.45:0/3746200796 >> 172.21.15.45:0/3746200796 conn(0x7f5b540fc820 msgr2=0x7f5b54101c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:09:22.783 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:22.780+0000 7f5b53200640 1 -- 172.21.15.45:0/3746200796 shutdown_connections 2024-04-16T17:09:22.783 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:22.781+0000 7f5b53200640 1 -- 172.21.15.45:0/3746200796 wait complete. 2024-04-16T17:09:23.225 INFO:tasks.workunit.client.0.smithi045.stdout: 1 39722 2.72 MB/sec warmup 85 sec latency 845.918 ms 2024-04-16T17:09:24.225 INFO:tasks.workunit.client.0.smithi045.stdout: 1 39722 2.69 MB/sec warmup 86 sec latency 1217.408 ms 2024-04-16T17:09:24.555 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:09:24 smithi117 ceph-mon[36730]: pgmap v82: 129 pgs: 129 active+clean; 30 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 392 KiB/s rd, 477 KiB/s wr, 48 op/s 2024-04-16T17:09:24.555 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:09:24 smithi117 ceph-mon[36730]: scrub summary: idle 2024-04-16T17:09:24.555 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:09:24 smithi117 ceph-mon[36730]: scrub summary: idle 2024-04-16T17:09:24.677 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:09:24 smithi104 ceph-mon[36457]: pgmap v82: 129 pgs: 129 active+clean; 30 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 392 KiB/s rd, 477 KiB/s wr, 48 op/s 2024-04-16T17:09:24.677 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:09:24 smithi104 ceph-mon[36457]: scrub summary: idle 2024-04-16T17:09:24.677 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:09:24 smithi104 ceph-mon[36457]: scrub summary: idle 2024-04-16T17:09:24.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:09:24 smithi045 ceph-mon[31676]: pgmap v82: 129 pgs: 129 active+clean; 30 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 392 KiB/s rd, 477 KiB/s wr, 48 op/s 2024-04-16T17:09:24.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:09:24 smithi045 ceph-mon[31676]: scrub summary: idle 2024-04-16T17:09:24.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:09:24 smithi045 ceph-mon[31676]: scrub summary: idle 2024-04-16T17:09:25.225 INFO:tasks.workunit.client.0.smithi045.stdout: 1 39722 2.66 MB/sec warmup 87 sec latency 2217.465 ms 2024-04-16T17:09:25.555 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:09:25 smithi117 ceph-mon[36730]: scrub summary: idle 2024-04-16T17:09:25.677 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:09:25 smithi104 ceph-mon[36457]: scrub summary: idle 2024-04-16T17:09:25.725 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:09:25 smithi045 ceph-mon[31676]: scrub summary: idle 2024-04-16T17:09:26.225 INFO:tasks.workunit.client.0.smithi045.stdout: 1 39722 2.63 MB/sec warmup 88 sec latency 3217.533 ms 2024-04-16T17:09:26.555 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:09:26 smithi117 ceph-mon[36730]: pgmap v83: 129 pgs: 129 active+clean; 30 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 392 KiB/s rd, 477 KiB/s wr, 48 op/s 2024-04-16T17:09:26.677 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:09:26 smithi104 ceph-mon[36457]: pgmap v83: 129 pgs: 129 active+clean; 30 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 392 KiB/s rd, 477 KiB/s wr, 48 op/s 2024-04-16T17:09:26.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:09:26 smithi045 ceph-mon[31676]: pgmap v83: 129 pgs: 129 active+clean; 30 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 392 KiB/s rd, 477 KiB/s wr, 48 op/s 2024-04-16T17:09:27.225 INFO:tasks.workunit.client.0.smithi045.stdout: 1 39722 2.60 MB/sec warmup 89 sec latency 4217.679 ms 2024-04-16T17:09:28.225 INFO:tasks.workunit.client.0.smithi045.stdout: 1 39845 2.57 MB/sec warmup 90 sec latency 4264.174 ms 2024-04-16T17:09:28.677 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:09:28 smithi104 ceph-mon[36457]: pgmap v84: 129 pgs: 129 active+clean; 30 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 502 KiB/s rd, 852 KiB/s wr, 90 op/s 2024-04-16T17:09:28.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:09:28 smithi045 ceph-mon[31676]: pgmap v84: 129 pgs: 129 active+clean; 30 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 502 KiB/s rd, 852 KiB/s wr, 90 op/s 2024-04-16T17:09:28.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:09:28 smithi117 ceph-mon[36730]: pgmap v84: 129 pgs: 129 active+clean; 30 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 502 KiB/s rd, 852 KiB/s wr, 90 op/s 2024-04-16T17:09:29.225 INFO:tasks.workunit.client.0.smithi045.stdout: 1 39845 2.55 MB/sec warmup 91 sec latency 1722.328 ms 2024-04-16T17:09:30.225 INFO:tasks.workunit.client.0.smithi045.stdout: 1 39845 2.52 MB/sec warmup 92 sec latency 2722.420 ms 2024-04-16T17:09:30.677 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:09:30 smithi104 ceph-mon[36457]: pgmap v85: 129 pgs: 129 active+clean; 30 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 494 KiB/s rd, 638 KiB/s wr, 82 op/s 2024-04-16T17:09:30.722 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:09:30 smithi045 ceph-mon[31676]: pgmap v85: 129 pgs: 129 active+clean; 30 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 494 KiB/s rd, 638 KiB/s wr, 82 op/s 2024-04-16T17:09:30.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:09:30 smithi117 ceph-mon[36730]: pgmap v85: 129 pgs: 129 active+clean; 30 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 494 KiB/s rd, 638 KiB/s wr, 82 op/s 2024-04-16T17:09:31.226 INFO:tasks.workunit.client.0.smithi045.stdout: 1 39845 2.49 MB/sec warmup 93 sec latency 3722.534 ms 2024-04-16T17:09:32.226 INFO:tasks.workunit.client.0.smithi045.stdout: 1 39845 2.46 MB/sec warmup 94 sec latency 4722.652 ms 2024-04-16T17:09:32.677 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:09:32 smithi104 ceph-mon[36457]: pgmap v86: 129 pgs: 129 active+clean; 31 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 508 KiB/s rd, 670 KiB/s wr, 89 op/s 2024-04-16T17:09:32.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:09:32 smithi045 ceph-mon[31676]: pgmap v86: 129 pgs: 129 active+clean; 31 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 508 KiB/s rd, 670 KiB/s wr, 89 op/s 2024-04-16T17:09:32.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:09:32 smithi117 ceph-mon[36730]: pgmap v86: 129 pgs: 129 active+clean; 31 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 508 KiB/s rd, 670 KiB/s wr, 89 op/s 2024-04-16T17:09:33.226 INFO:tasks.workunit.client.0.smithi045.stdout: 1 39846 2.44 MB/sec warmup 95 sec latency 4780.061 ms 2024-04-16T17:09:34.226 INFO:tasks.workunit.client.0.smithi045.stdout: 1 39846 2.41 MB/sec warmup 96 sec latency 1942.771 ms 2024-04-16T17:09:34.677 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:09:34 smithi104 ceph-mon[36457]: pgmap v87: 129 pgs: 129 active+clean; 31 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 407 KiB/s wr, 49 op/s 2024-04-16T17:09:34.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:09:34 smithi045 ceph-mon[31676]: pgmap v87: 129 pgs: 129 active+clean; 31 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 407 KiB/s wr, 49 op/s 2024-04-16T17:09:34.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:09:34 smithi117 ceph-mon[36730]: pgmap v87: 129 pgs: 129 active+clean; 31 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 407 KiB/s wr, 49 op/s 2024-04-16T17:09:35.226 INFO:tasks.workunit.client.0.smithi045.stdout: 1 39846 2.39 MB/sec warmup 97 sec latency 2942.875 ms 2024-04-16T17:09:35.918 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:35.917+0000 7fab31600640 1 -- 172.21.15.45:0/2722235849 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fab2c1038e0 msgr2=0x7fab2c10bdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:35.918 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:35.917+0000 7fab31600640 1 --2- 172.21.15.45:0/2722235849 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fab2c1038e0 0x7fab2c10bdc0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fab18009900 tx=0x7fab1802bcd0 comp rx=0 tx=0).stop 2024-04-16T17:09:35.918 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:35.917+0000 7fab31600640 1 -- 172.21.15.45:0/2722235849 shutdown_connections 2024-04-16T17:09:35.919 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:35.917+0000 7fab31600640 1 --2- 172.21.15.45:0/2722235849 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fab2c10c3e0 0x7fab2c10e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:35.919 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:35.917+0000 7fab31600640 1 --2- 172.21.15.45:0/2722235849 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fab2c1038e0 0x7fab2c10bdc0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:35.919 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:35.917+0000 7fab31600640 1 --2- 172.21.15.45:0/2722235849 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fab2c102fc0 0x7fab2c1033a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:35.919 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:35.917+0000 7fab31600640 1 -- 172.21.15.45:0/2722235849 >> 172.21.15.45:0/2722235849 conn(0x7fab2c0fc820 msgr2=0x7fab2c0fec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:09:35.919 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:35.917+0000 7fab31600640 1 -- 172.21.15.45:0/2722235849 shutdown_connections 2024-04-16T17:09:35.919 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:35.917+0000 7fab31600640 1 -- 172.21.15.45:0/2722235849 wait complete. 2024-04-16T17:09:35.921 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:35.918+0000 7fab31600640 1 Processor -- start 2024-04-16T17:09:35.921 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:35.918+0000 7fab31600640 1 -- start start 2024-04-16T17:09:35.921 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:35.919+0000 7fab31600640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fab2c102fc0 0x7fab2c1a0670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:35.921 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:35.919+0000 7fab31600640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fab2c1038e0 0x7fab2c1a0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:35.921 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:35.919+0000 7fab31600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fab2c10c3e0 0x7fab2c19a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:35.921 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:35.919+0000 7fab31600640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fab2c112210 con 0x7fab2c10c3e0 2024-04-16T17:09:35.921 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:35.919+0000 7fab31600640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fab2c112090 con 0x7fab2c102fc0 2024-04-16T17:09:35.921 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:35.919+0000 7fab31600640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fab2c112390 con 0x7fab2c1038e0 2024-04-16T17:09:35.922 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:35.919+0000 7fab30c00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fab2c10c3e0 0x7fab2c19a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:35.922 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:35.919+0000 7fab2be00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fab2c102fc0 0x7fab2c1a0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:35.922 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:35.919+0000 7fab30c00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fab2c10c3e0 0x7fab2c19a730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41756/0 (socket says 172.21.15.45:41756) 2024-04-16T17:09:35.922 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:35.919+0000 7fab30c00640 1 -- 172.21.15.45:0/2664775720 learned_addr learned my addr 172.21.15.45:0/2664775720 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:09:35.922 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:35.919+0000 7fab2b400640 1 --2- 172.21.15.45:0/2664775720 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fab2c1038e0 0x7fab2c1a0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:35.922 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:35.919+0000 7fab2be00640 1 -- 172.21.15.45:0/2664775720 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fab2c1038e0 msgr2=0x7fab2c1a0bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:35.922 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:35.919+0000 7fab2be00640 1 --2- 172.21.15.45:0/2664775720 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fab2c1038e0 0x7fab2c1a0bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:35.922 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:35.919+0000 7fab2be00640 1 -- 172.21.15.45:0/2664775720 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fab2c10c3e0 msgr2=0x7fab2c19a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:35.922 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:35.919+0000 7fab2be00640 1 --2- 172.21.15.45:0/2664775720 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fab2c10c3e0 0x7fab2c19a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:35.922 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:35.919+0000 7fab2be00640 1 -- 172.21.15.45:0/2664775720 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab2c19aff0 con 0x7fab2c102fc0 2024-04-16T17:09:35.922 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:35.920+0000 7fab2be00640 1 --2- 172.21.15.45:0/2664775720 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fab2c102fc0 0x7fab2c1a0670 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fab1c00e9f0 tx=0x7fab1c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:09:35.922 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:35.920+0000 7fab28c00640 1 -- 172.21.15.45:0/2664775720 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab1c00ccc0 con 0x7fab2c102fc0 2024-04-16T17:09:35.922 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:35.920+0000 7fab31600640 1 -- 172.21.15.45:0/2664775720 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab2c19b2e0 con 0x7fab2c102fc0 2024-04-16T17:09:35.923 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:35.920+0000 7fab28c00640 1 -- 172.21.15.45:0/2664775720 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fab1c010910 con 0x7fab2c102fc0 2024-04-16T17:09:35.923 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:35.920+0000 7fab28c00640 1 -- 172.21.15.45:0/2664775720 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab1c018960 con 0x7fab2c102fc0 2024-04-16T17:09:35.923 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:35.920+0000 7fab31600640 1 -- 172.21.15.45:0/2664775720 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab2c1a75f0 con 0x7fab2c102fc0 2024-04-16T17:09:35.924 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:35.923+0000 7fab3599c740 1 Processor -- start 2024-04-16T17:09:35.924 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:35.923+0000 7fab3599c740 1 -- start start 2024-04-16T17:09:35.924 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:35.923+0000 7fab28c00640 1 -- 172.21.15.45:0/2664775720 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fab1c010ac0 con 0x7fab2c102fc0 2024-04-16T17:09:35.925 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:35.924+0000 7fab3599c740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c62daad7f0 0x55c62daac540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:35.925 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:35.924+0000 7fab3599c740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c62daaca80 0x55c62daacf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:35.925 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:35.924+0000 7fab3599c740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c62dab6000 0x55c62dab83f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:35.926 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:35.924+0000 7fab3599c740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55c62da11780 con 0x55c62daaca80 2024-04-16T17:09:35.926 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:35.924+0000 7fab3599c740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55c62d9b7f40 con 0x55c62daad7f0 2024-04-16T17:09:35.926 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:35.924+0000 7fab3599c740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55c62da0d740 con 0x55c62dab6000 2024-04-16T17:09:35.926 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:35.924+0000 7fab2be00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c62daaca80 0x55c62daacf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:35.926 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:35.924+0000 7fab2be00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c62daaca80 0x55c62daacf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41766/0 (socket says 172.21.15.45:41766) 2024-04-16T17:09:35.926 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:35.924+0000 7fab2be00640 1 -- 172.21.15.45:0/853949568 learned_addr learned my addr 172.21.15.45:0/853949568 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:09:35.926 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:35.924+0000 7fab30c00640 1 --2- 172.21.15.45:0/853949568 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c62daad7f0 0x55c62daac540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:35.926 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:35.924+0000 7fab2b400640 1 --2- 172.21.15.45:0/853949568 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c62dab6000 0x55c62dab83f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:35.926 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:35.924+0000 7fab2be00640 1 -- 172.21.15.45:0/853949568 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c62dab6000 msgr2=0x55c62dab83f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:09:35.926 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:35.924+0000 7fab2be00640 1 --2- 172.21.15.45:0/853949568 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c62dab6000 0x55c62dab83f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:35.926 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:35.924+0000 7fab2be00640 1 -- 172.21.15.45:0/853949568 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c62daad7f0 msgr2=0x55c62daac540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:09:35.926 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:35.924+0000 7fab2be00640 1 --2- 172.21.15.45:0/853949568 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c62daad7f0 0x55c62daac540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:35.927 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:35.924+0000 7fab2be00640 1 -- 172.21.15.45:0/853949568 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c62d724210 con 0x55c62daaca80 2024-04-16T17:09:35.927 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:35.924+0000 7fab30c00640 1 --2- 172.21.15.45:0/853949568 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c62daad7f0 0x55c62daac540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:09:35.927 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:35.925+0000 7fab2be00640 1 --2- 172.21.15.45:0/853949568 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c62daaca80 0x55c62daacf00 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fab1c0ce1c0 tx=0x7fab1c0d9570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=876042d76409ef33 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:09:35.931 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:35.925+0000 7fab21200640 1 -- 172.21.15.45:0/853949568 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab1c0e4070 con 0x55c62daaca80 2024-04-16T17:09:35.931 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:35.925+0000 7fab28c00640 1 --2- 172.21.15.45:0/2664775720 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fab100a3d10 0x7fab100a61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:35.931 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:35.925+0000 7fab21200640 1 -- 172.21.15.45:0/853949568 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fab1c0a5740 con 0x55c62daaca80 2024-04-16T17:09:35.931 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:35.925+0000 7fab21200640 1 -- 172.21.15.45:0/853949568 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab1c0a27f0 con 0x55c62daaca80 2024-04-16T17:09:35.931 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:35.925+0000 7fab28c00640 1 -- 172.21.15.45:0/2664775720 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fab1c0d0ea0 con 0x7fab2c102fc0 2024-04-16T17:09:35.931 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:35.925+0000 7fab30c00640 1 --2- 172.21.15.45:0/2664775720 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fab100a3d10 0x7fab100a61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:35.931 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:35.926+0000 7fab3599c740 1 -- 172.21.15.45:0/853949568 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c62daaca80 msgr2=0x55c62daacf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:35.931 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:35.926+0000 7fab3599c740 1 --2- 172.21.15.45:0/853949568 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c62daaca80 0x55c62daacf00 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fab1c0ce1c0 tx=0x7fab1c0d9570 comp rx=0 tx=0).stop 2024-04-16T17:09:35.931 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:35.926+0000 7fab3599c740 1 -- 172.21.15.45:0/853949568 shutdown_connections 2024-04-16T17:09:35.931 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:35.926+0000 7fab3599c740 1 --2- 172.21.15.45:0/853949568 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c62dab6000 0x55c62dab83f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:35.931 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:35.926+0000 7fab3599c740 1 --2- 172.21.15.45:0/853949568 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c62daaca80 0x55c62daacf00 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:35.931 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:35.926+0000 7fab3599c740 1 --2- 172.21.15.45:0/853949568 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c62daad7f0 0x55c62daac540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:35.932 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:35.926+0000 7fab3599c740 1 -- 172.21.15.45:0/853949568 >> 172.21.15.45:0/853949568 conn(0x55c62da475d0 msgr2=0x55c62daa7770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:09:35.932 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:35.926+0000 7fab3599c740 1 -- 172.21.15.45:0/853949568 shutdown_connections 2024-04-16T17:09:35.932 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:35.926+0000 7fab30c00640 1 --2- 172.21.15.45:0/2664775720 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fab100a3d10 0x7fab100a61d0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fab24009270 tx=0x7fab24009680 comp rx=0 tx=0).ready entity=mgr.25411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:09:35.932 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:35.926+0000 7fab3599c740 1 -- 172.21.15.45:0/853949568 wait complete. 2024-04-16T17:09:35.932 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:35.927+0000 7fab3599c740 1 -- start start 2024-04-16T17:09:35.932 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:35.927+0000 7fab3599c740 1 Processor -- start 2024-04-16T17:09:35.932 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:35.927+0000 7fab3599c740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c62daaca80 0x55c62dab5a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:35.932 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:35.927+0000 7fab3599c740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c62daad7f0 0x55c62dab2de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:35.932 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:35.927+0000 7fab3599c740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c62dab6000 0x55c62dab3520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:35.932 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:35.927+0000 7fab3599c740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55c62dab8d30 con 0x55c62dab6000 2024-04-16T17:09:35.932 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:35.927+0000 7fab3599c740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55c62d9cb390 con 0x55c62daaca80 2024-04-16T17:09:35.932 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:35.927+0000 7fab3599c740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55c62dab8eb0 con 0x55c62daad7f0 2024-04-16T17:09:35.932 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:35.928+0000 7fab3599c740 10 client.0 _ll_register_callbacks cb 0x55c62d9b4f60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:09:35.932 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:35.928+0000 7fab30c00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c62dab6000 0x55c62dab3520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:35.933 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:35.928+0000 7fab2be00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c62daaca80 0x55c62dab5a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:35.933 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:35.928+0000 7fab30c00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c62dab6000 0x55c62dab3520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41780/0 (socket says 172.21.15.45:41780) 2024-04-16T17:09:35.933 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:35.928+0000 7fab30c00640 1 -- 172.21.15.45:0/3530635460 learned_addr learned my addr 172.21.15.45:0/3530635460 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:09:35.933 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:35.928+0000 7fab2b400640 1 --2- 172.21.15.45:0/3530635460 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c62daad7f0 0x55c62dab2de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:35.933 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:35.928+0000 7fab2be00640 1 -- 172.21.15.45:0/3530635460 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c62daad7f0 msgr2=0x55c62dab2de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:09:35.933 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:35.928+0000 7fab2be00640 1 --2- 172.21.15.45:0/3530635460 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c62daad7f0 0x55c62dab2de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:35.933 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:35.928+0000 7fab2be00640 1 -- 172.21.15.45:0/3530635460 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c62dab6000 msgr2=0x55c62dab3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:09:35.933 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:35.928+0000 7fab2be00640 1 --2- 172.21.15.45:0/3530635460 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c62dab6000 0x55c62dab3520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:35.933 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:35.928+0000 7fab2be00640 1 -- 172.21.15.45:0/3530635460 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c62d7f0620 con 0x55c62daaca80 2024-04-16T17:09:35.933 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:35.928+0000 7fab30c00640 1 --2- 172.21.15.45:0/3530635460 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c62dab6000 0x55c62dab3520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:09:35.933 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:35.929+0000 7fab2be00640 1 --2- 172.21.15.45:0/3530635460 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c62daaca80 0x55c62dab5a50 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fab1c09c880 tx=0x7fab1c09b960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5ee9079bae426f0c server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:09:35.933 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:35.929+0000 7fab15600640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-04-16T17:09:35.933 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:35.929+0000 7fab15600640 1 -- 172.21.15.45:0/3530635460 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab1c0a1740 con 0x55c62daaca80 2024-04-16T17:09:35.933 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:35.929+0000 7fab3599c740 1 -- 172.21.15.45:0/3530635460 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fab1c0a5740 con 0x55c62daaca80 2024-04-16T17:09:35.934 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:35.929+0000 7fab15600640 1 -- 172.21.15.45:0/3530635460 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fab1c09ae80 con 0x55c62daaca80 2024-04-16T17:09:35.934 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:35.929+0000 7fab15600640 1 -- 172.21.15.45:0/3530635460 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fab1c0a19b0 con 0x55c62daaca80 2024-04-16T17:09:35.934 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:35.929+0000 7fab3599c740 10 client.26399 fetch_fsmap learned FSMap version 30 2024-04-16T17:09:35.934 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:35.929+0000 7fab3599c740 1 -- 172.21.15.45:0/3530635460 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55c62dab8d30 con 0x55c62daaca80 2024-04-16T17:09:35.934 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:35.929+0000 7fab15600640 1 -- 172.21.15.45:0/3530635460 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fab1c0a5740 con 0x55c62daaca80 2024-04-16T17:09:35.934 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:35.930+0000 7fab15600640 1 -- 172.21.15.45:0/3530635460 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fab1c0ce500 con 0x55c62daaca80 2024-04-16T17:09:35.934 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:35.930+0000 7fab3599c740 10 client.26399 fetch_fsmap finished waiting for FSMap version 30 2024-04-16T17:09:35.934 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:35.930+0000 7fab3599c740 10 client.26399 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T17:09:35.934 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:35.930+0000 7fab3599c740 20 client.26399 populate_metadata read hostname 'smithi045' 2024-04-16T17:09:35.934 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:35.930+0000 7fab3599c740 1 --2- 172.21.15.45:0/3530635460 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55c62dac5f40 0x55c62dac8330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:35.934 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:35.930+0000 7fab3599c740 4 client.26399 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:09:35.934 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:35.930+0000 7fab3599c740 1 -- 172.21.15.45:0/3530635460 --> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c62da1f1a0 con 0x55c62dac5f40 2024-04-16T17:09:35.936 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:35.930+0000 7fab2b400640 1 --2- 172.21.15.45:0/3530635460 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55c62dac5f40 0x55c62dac8330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:35.936 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:35.931+0000 7fab2b400640 1 --2- 172.21.15.45:0/3530635460 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55c62dac5f40 0x55c62dac8330 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f22ba4e3d6022073 server_cookie=637d7a6277e77645 in_seq=0 out_seq=0 2024-04-16T17:09:35.936 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:35.931+0000 7fab15600640 10 client.26399 ms_handle_connect on v2:172.21.15.104:6838/3660070592 2024-04-16T17:09:35.936 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:35.932+0000 7fab15600640 1 -- 172.21.15.45:0/3530635460 <== mds.0 v2:172.21.15.104:6838/3660070592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c62da1f1a0 con 0x55c62dac5f40 2024-04-16T17:09:35.936 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:35.932+0000 7fab15600640 10 client.26399 handle_command_reply: tid=0 2024-04-16T17:09:35.936 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:35.932+0000 7fab3599c740 1 client.26399 shutdown 2024-04-16T17:09:35.936 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:35.932+0000 7fab3599c740 1 -- 172.21.15.45:0/3530635460 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c62daaca80 msgr2=0x55c62dab5a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:35.936 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:35.932+0000 7fab3599c740 1 --2- 172.21.15.45:0/3530635460 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c62daaca80 0x55c62dab5a50 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fab1c09c880 tx=0x7fab1c09b960 comp rx=0 tx=0).stop 2024-04-16T17:09:35.936 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:35.932+0000 7fab3599c740 1 -- 172.21.15.45:0/3530635460 shutdown_connections 2024-04-16T17:09:35.936 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:35.932+0000 7fab3599c740 1 --2- 172.21.15.45:0/3530635460 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55c62dac5f40 0x55c62dac8330 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:35.936 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:35.932+0000 7fab3599c740 1 --2- 172.21.15.45:0/3530635460 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c62daad7f0 0x55c62dab2de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:35.937 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:35.932+0000 7fab3599c740 1 --2- 172.21.15.45:0/3530635460 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c62dab6000 0x55c62dab3520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:35.937 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:35.932+0000 7fab15600640 0 client.26399 ms_handle_reset on v2:172.21.15.104:6838/3660070592 2024-04-16T17:09:35.937 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:35.932+0000 7fab3599c740 1 --2- 172.21.15.45:0/3530635460 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c62daaca80 0x55c62dab5a50 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:35.937 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:35.932+0000 7fab3599c740 1 -- 172.21.15.45:0/3530635460 >> 172.21.15.45:0/3530635460 conn(0x55c62da475d0 msgr2=0x55c62daa94c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:09:35.937 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:35.932+0000 7fab3599c740 1 -- 172.21.15.45:0/3530635460 shutdown_connections 2024-04-16T17:09:35.937 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:35.932+0000 7fab3599c740 1 -- 172.21.15.45:0/3530635460 wait complete. 2024-04-16T17:09:35.937 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:35.932+0000 7fab3599c740 20 client.26399 trim_cache size 0 max 16384 2024-04-16T17:09:35.949 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:35.948+0000 7fab3599c740 1 Processor -- start 2024-04-16T17:09:35.949 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:35.949+0000 7fab3599c740 1 -- start start 2024-04-16T17:09:35.951 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:35.949+0000 7fab3599c740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c62daaca80 0x55c62dac1c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:35.951 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:35.949+0000 7fab3599c740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c62daad7f0 0x55c62dacda50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:35.951 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:35.949+0000 7fab3599c740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c62dab6000 0x55c62dacdf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:35.951 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:35.949+0000 7fab3599c740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fab2c1b3eb0 con 0x55c62daad7f0 2024-04-16T17:09:35.952 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:35.949+0000 7fab3599c740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fab2c1b16c0 con 0x55c62daaca80 2024-04-16T17:09:35.952 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:35.949+0000 7fab3599c740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fab2c1af4f0 con 0x55c62dab6000 2024-04-16T17:09:35.952 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:35.949+0000 7fab2b400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c62daad7f0 0x55c62dacda50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:35.952 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:35.949+0000 7fab2b400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c62daad7f0 0x55c62dacda50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41790/0 (socket says 172.21.15.45:41790) 2024-04-16T17:09:35.952 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:35.949+0000 7fab2b400640 1 -- 172.21.15.45:0/4026321319 learned_addr learned my addr 172.21.15.45:0/4026321319 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:09:35.952 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:35.949+0000 7fab2be00640 1 --2- 172.21.15.45:0/4026321319 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c62daaca80 0x55c62dac1c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:35.952 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:35.949+0000 7fab30c00640 1 --2- 172.21.15.45:0/4026321319 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c62dab6000 0x55c62dacdf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:35.952 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:35.950+0000 7fab2b400640 1 -- 172.21.15.45:0/4026321319 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c62dab6000 msgr2=0x55c62dacdf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:09:35.952 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:35.950+0000 7fab2b400640 1 --2- 172.21.15.45:0/4026321319 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c62dab6000 0x55c62dacdf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:35.952 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:35.950+0000 7fab2b400640 1 -- 172.21.15.45:0/4026321319 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c62daaca80 msgr2=0x55c62dac1c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:09:35.952 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:35.950+0000 7fab2b400640 1 --2- 172.21.15.45:0/4026321319 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c62daaca80 0x55c62dac1c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:35.952 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:35.950+0000 7fab2b400640 1 -- 172.21.15.45:0/4026321319 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c62daabd50 con 0x55c62daad7f0 2024-04-16T17:09:35.952 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:35.950+0000 7fab2be00640 1 --2- 172.21.15.45:0/4026321319 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c62daaca80 0x55c62dac1c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:09:35.952 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:35.950+0000 7fab2b400640 1 --2- 172.21.15.45:0/4026321319 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c62daad7f0 0x55c62dacda50 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fab1802c2b0 tx=0x7fab1802e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=390731762a64ad8d server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:09:35.953 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:35.950+0000 7fab17e00640 1 -- 172.21.15.45:0/4026321319 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab1804e070 con 0x55c62daad7f0 2024-04-16T17:09:35.953 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:35.950+0000 7fab17e00640 1 -- 172.21.15.45:0/4026321319 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fab1802ce00 con 0x55c62daad7f0 2024-04-16T17:09:35.953 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:35.950+0000 7fab17e00640 1 -- 172.21.15.45:0/4026321319 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab180046c0 con 0x55c62daad7f0 2024-04-16T17:09:35.956 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:35.951+0000 7fab3599c740 1 -- 172.21.15.45:0/4026321319 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c62daad7f0 msgr2=0x55c62dacda50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:35.957 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:35.951+0000 7fab3599c740 1 --2- 172.21.15.45:0/4026321319 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c62daad7f0 0x55c62dacda50 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fab1802c2b0 tx=0x7fab1802e040 comp rx=0 tx=0).stop 2024-04-16T17:09:35.957 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:35.951+0000 7fab3599c740 1 -- 172.21.15.45:0/4026321319 shutdown_connections 2024-04-16T17:09:35.957 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:35.951+0000 7fab3599c740 1 --2- 172.21.15.45:0/4026321319 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c62dab6000 0x55c62dacdf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:35.957 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:35.951+0000 7fab3599c740 1 --2- 172.21.15.45:0/4026321319 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c62daad7f0 0x55c62dacda50 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:35.957 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:35.951+0000 7fab3599c740 1 --2- 172.21.15.45:0/4026321319 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c62daaca80 0x55c62dac1c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:35.957 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:35.951+0000 7fab3599c740 1 -- 172.21.15.45:0/4026321319 >> 172.21.15.45:0/4026321319 conn(0x55c62da475d0 msgr2=0x55c62daa72a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:09:35.957 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:35.951+0000 7fab3599c740 1 -- 172.21.15.45:0/4026321319 shutdown_connections 2024-04-16T17:09:35.957 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:35.951+0000 7fab3599c740 1 -- 172.21.15.45:0/4026321319 wait complete. 2024-04-16T17:09:35.957 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:35.952+0000 7fab3599c740 1 -- start start 2024-04-16T17:09:35.957 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:35.952+0000 7fab3599c740 1 Processor -- start 2024-04-16T17:09:35.957 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:35.953+0000 7fab3599c740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c62daaca80 0x55c62dabe6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:35.957 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:35.953+0000 7fab3599c740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c62daad7f0 0x55c62dab01f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:35.957 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:35.953+0000 7fab3599c740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c62dab6000 0x55c62dab08e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:35.958 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:35.953+0000 7fab3599c740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fab2c1afb80 con 0x55c62dab6000 2024-04-16T17:09:35.958 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:35.953+0000 7fab3599c740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fab2c1b0ff0 con 0x55c62daaca80 2024-04-16T17:09:35.958 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:35.953+0000 7fab3599c740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fab2c1b0250 con 0x55c62daad7f0 2024-04-16T17:09:35.958 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:35.953+0000 7fab3599c740 10 client.0 _ll_register_callbacks cb 0x7fab1c0e2070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:09:35.958 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:35.953+0000 7fab30c00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c62dab6000 0x55c62dab08e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:35.958 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:35.953+0000 7fab30c00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c62dab6000 0x55c62dab08e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41806/0 (socket says 172.21.15.45:41806) 2024-04-16T17:09:35.958 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:35.953+0000 7fab30c00640 1 -- 172.21.15.45:0/3036629947 learned_addr learned my addr 172.21.15.45:0/3036629947 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:09:35.958 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:35.953+0000 7fab2be00640 1 --2- 172.21.15.45:0/3036629947 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c62daaca80 0x55c62dabe6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:35.958 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:35.953+0000 7fab2b400640 1 --2- 172.21.15.45:0/3036629947 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c62daad7f0 0x55c62dab01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:35.958 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:35.954+0000 7fab30c00640 1 -- 172.21.15.45:0/3036629947 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c62daad7f0 msgr2=0x55c62dab01f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:09:35.958 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:35.954+0000 7fab30c00640 1 --2- 172.21.15.45:0/3036629947 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c62daad7f0 0x55c62dab01f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:35.958 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:35.954+0000 7fab30c00640 1 -- 172.21.15.45:0/3036629947 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c62daaca80 msgr2=0x55c62dabe6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:09:35.958 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:35.954+0000 7fab30c00640 1 --2- 172.21.15.45:0/3036629947 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c62daaca80 0x55c62dabe6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:35.958 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:35.954+0000 7fab30c00640 1 -- 172.21.15.45:0/3036629947 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c62dab4a50 con 0x55c62dab6000 2024-04-16T17:09:35.959 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:35.954+0000 7fab30c00640 1 --2- 172.21.15.45:0/3036629947 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c62dab6000 0x55c62dab08e0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fab24013010 tx=0x7fab24004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cb074bfb6c06efe server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:09:35.959 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:35.954+0000 7fab15600640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-16T17:09:35.959 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:35.954+0000 7fab15600640 1 -- 172.21.15.45:0/3036629947 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab2402f060 con 0x55c62dab6000 2024-04-16T17:09:35.959 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:35.954+0000 7fab15600640 1 -- 172.21.15.45:0/3036629947 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fab2401f970 con 0x55c62dab6000 2024-04-16T17:09:35.959 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:35.954+0000 7fab3599c740 1 -- 172.21.15.45:0/3036629947 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fab1802ce00 con 0x55c62dab6000 2024-04-16T17:09:35.959 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:35.955+0000 7fab15600640 1 -- 172.21.15.45:0/3036629947 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fab24036050 con 0x55c62dab6000 2024-04-16T17:09:35.959 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:35.955+0000 7fab3599c740 10 client.16254 fetch_fsmap learned FSMap version 30 2024-04-16T17:09:35.959 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:35.955+0000 7fab3599c740 1 -- 172.21.15.45:0/3036629947 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55c62db2c5e0 con 0x55c62dab6000 2024-04-16T17:09:35.959 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:35.955+0000 7fab15600640 1 -- 172.21.15.45:0/3036629947 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fab1802ce00 con 0x55c62dab6000 2024-04-16T17:09:35.959 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:35.955+0000 7fab15600640 1 -- 172.21.15.45:0/3036629947 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fab2400f040 con 0x55c62dab6000 2024-04-16T17:09:35.959 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:35.955+0000 7fab3599c740 10 client.16254 fetch_fsmap finished waiting for FSMap version 30 2024-04-16T17:09:35.959 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:35.955+0000 7fab3599c740 10 client.16254 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T17:09:35.959 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:35.955+0000 7fab3599c740 20 client.16254 populate_metadata read hostname 'smithi045' 2024-04-16T17:09:35.959 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:35.955+0000 7fab3599c740 1 --2- 172.21.15.45:0/3036629947 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55c62db2ce10 0x55c62db2f250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:35.960 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:35.955+0000 7fab3599c740 4 client.16254 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-16T17:09:35.960 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:35.955+0000 7fab3599c740 1 -- 172.21.15.45:0/3036629947 --> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c62da19010 con 0x55c62db2ce10 2024-04-16T17:09:35.960 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:35.956+0000 7fab2be00640 1 --2- 172.21.15.45:0/3036629947 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55c62db2ce10 0x55c62db2f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:35.960 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:35.956+0000 7fab15600640 10 client.16254 ms_handle_connect on v2:172.21.15.104:6838/3660070592 2024-04-16T17:09:35.960 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:35.956+0000 7fab2be00640 1 --2- 172.21.15.45:0/3036629947 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55c62db2ce10 0x55c62db2f250 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2ed5e2e6fcf07f1d server_cookie=cf87b26e6d8dd3b in_seq=0 out_seq=0 2024-04-16T17:09:35.986 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:35.984+0000 7fab15600640 1 -- 172.21.15.45:0/3036629947 <== mds.0 v2:172.21.15.104:6838/3660070592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c62da19010 con 0x55c62db2ce10 2024-04-16T17:09:35.986 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:35.984+0000 7fab15600640 10 client.16254 handle_command_reply: tid=0 2024-04-16T17:09:35.986 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:35.984+0000 7fab3599c740 1 client.16254 shutdown 2024-04-16T17:09:35.986 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:35.984+0000 7fab3599c740 1 -- 172.21.15.45:0/3036629947 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c62dab6000 msgr2=0x55c62dab08e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:35.986 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:35.984+0000 7fab3599c740 1 --2- 172.21.15.45:0/3036629947 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c62dab6000 0x55c62dab08e0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fab24013010 tx=0x7fab24004a40 comp rx=0 tx=0).stop 2024-04-16T17:09:35.986 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:35.985+0000 7fab3599c740 1 -- 172.21.15.45:0/3036629947 shutdown_connections 2024-04-16T17:09:35.986 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:35.985+0000 7fab3599c740 1 --2- 172.21.15.45:0/3036629947 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55c62db2ce10 0x55c62db2f250 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:35.986 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:35.985+0000 7fab3599c740 1 --2- 172.21.15.45:0/3036629947 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c62daad7f0 0x55c62dab01f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:35.987 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:35.985+0000 7fab3599c740 1 --2- 172.21.15.45:0/3036629947 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c62dab6000 0x55c62dab08e0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:35.987 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:35.985+0000 7fab3599c740 1 --2- 172.21.15.45:0/3036629947 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c62daaca80 0x55c62dabe6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:35.987 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:35.985+0000 7fab3599c740 1 -- 172.21.15.45:0/3036629947 >> 172.21.15.45:0/3036629947 conn(0x55c62da475d0 msgr2=0x55c62dab25f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:09:35.987 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:35.985+0000 7fab15600640 0 client.16254 ms_handle_reset on v2:172.21.15.104:6838/3660070592 2024-04-16T17:09:35.987 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:35.985+0000 7fab3599c740 1 -- 172.21.15.45:0/3036629947 shutdown_connections 2024-04-16T17:09:35.987 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:35.985+0000 7fab3599c740 1 -- 172.21.15.45:0/3036629947 wait complete. 2024-04-16T17:09:35.987 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:35.985+0000 7fab3599c740 20 client.16254 trim_cache size 0 max 16384 2024-04-16T17:09:35.987 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-04-16T17:09:35.987 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-04-16T17:09:35.987 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-04-16T17:09:35.987 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-04-16T17:09:35.988 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:35.986+0000 7fab31600640 1 -- 172.21.15.45:0/2664775720 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fab100a3d10 msgr2=0x7fab100a61d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:35.988 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:35.986+0000 7fab31600640 1 --2- 172.21.15.45:0/2664775720 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fab100a3d10 0x7fab100a61d0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fab24009270 tx=0x7fab24009680 comp rx=0 tx=0).stop 2024-04-16T17:09:35.988 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:35.986+0000 7fab31600640 1 -- 172.21.15.45:0/2664775720 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fab2c102fc0 msgr2=0x7fab2c1a0670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:35.988 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:35.986+0000 7fab31600640 1 --2- 172.21.15.45:0/2664775720 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fab2c102fc0 0x7fab2c1a0670 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fab1c00e9f0 tx=0x7fab1c00eeb0 comp rx=0 tx=0).stop 2024-04-16T17:09:35.989 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:35.986+0000 7fab31600640 1 -- 172.21.15.45:0/2664775720 shutdown_connections 2024-04-16T17:09:35.989 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:35.986+0000 7fab31600640 1 --2- 172.21.15.45:0/2664775720 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fab100a3d10 0x7fab100a61d0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:35.989 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:35.986+0000 7fab31600640 1 --2- 172.21.15.45:0/2664775720 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fab2c1038e0 0x7fab2c1a0bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:35.989 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:35.986+0000 7fab31600640 1 --2- 172.21.15.45:0/2664775720 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fab2c10c3e0 0x7fab2c19a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:35.989 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:35.986+0000 7fab31600640 1 --2- 172.21.15.45:0/2664775720 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fab2c102fc0 0x7fab2c1a0670 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:35.989 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:35.986+0000 7fab31600640 1 -- 172.21.15.45:0/2664775720 >> 172.21.15.45:0/2664775720 conn(0x7fab2c0fc820 msgr2=0x7fab2c100b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:09:35.989 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:35.986+0000 7fab31600640 1 -- 172.21.15.45:0/2664775720 shutdown_connections 2024-04-16T17:09:35.989 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:35.986+0000 7fab31600640 1 -- 172.21.15.45:0/2664775720 wait complete. 2024-04-16T17:09:36.226 INFO:tasks.workunit.client.0.smithi045.stdout: 1 39846 2.36 MB/sec warmup 98 sec latency 3942.996 ms 2024-04-16T17:09:36.677 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:09:36 smithi104 ceph-mon[36457]: pgmap v88: 129 pgs: 129 active+clean; 31 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 407 KiB/s wr, 49 op/s 2024-04-16T17:09:36.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:09:36 smithi045 ceph-mon[31676]: pgmap v88: 129 pgs: 129 active+clean; 31 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 407 KiB/s wr, 49 op/s 2024-04-16T17:09:36.804 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:09:36 smithi117 ceph-mon[36730]: pgmap v88: 129 pgs: 129 active+clean; 31 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 407 KiB/s wr, 49 op/s 2024-04-16T17:09:37.226 INFO:tasks.workunit.client.0.smithi045.stdout: 1 39846 2.34 MB/sec warmup 99 sec latency 4943.086 ms 2024-04-16T17:09:38.226 INFO:tasks.workunit.client.0.smithi045.stdout: 1 40034 2.32 MB/sec warmup 100 sec latency 4993.532 ms 2024-04-16T17:09:38.607 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:09:38 smithi117 ceph-mon[36730]: pgmap v89: 129 pgs: 129 active+clean; 31 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 421 KiB/s wr, 62 op/s 2024-04-16T17:09:38.677 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:09:38 smithi104 ceph-mon[36457]: pgmap v89: 129 pgs: 129 active+clean; 31 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 421 KiB/s wr, 62 op/s 2024-04-16T17:09:38.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:09:38 smithi045 ceph-mon[31676]: pgmap v89: 129 pgs: 129 active+clean; 31 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 421 KiB/s wr, 62 op/s 2024-04-16T17:09:39.226 INFO:tasks.workunit.client.0.smithi045.stdout: 1 40034 2.30 MB/sec warmup 101 sec latency 1522.830 ms 2024-04-16T17:09:40.227 INFO:tasks.workunit.client.0.smithi045.stdout: 1 40034 2.28 MB/sec warmup 102 sec latency 2522.931 ms 2024-04-16T17:09:40.477 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:09:40 smithi045 ceph-mon[31676]: pgmap v90: 129 pgs: 129 active+clean; 31 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 46 KiB/s wr, 20 op/s 2024-04-16T17:09:40.677 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:09:40 smithi104 ceph-mon[36457]: pgmap v90: 129 pgs: 129 active+clean; 31 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 46 KiB/s wr, 20 op/s 2024-04-16T17:09:40.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:09:40 smithi117 ceph-mon[36730]: pgmap v90: 129 pgs: 129 active+clean; 31 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 46 KiB/s wr, 20 op/s 2024-04-16T17:09:41.227 INFO:tasks.workunit.client.0.smithi045.stdout: 1 40034 2.25 MB/sec warmup 103 sec latency 3523.023 ms 2024-04-16T17:09:41.343 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-16T17:09:41.343 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-16T17:09:41.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.554+0000 7f4e43e00640 1 -- 172.21.15.45:0/3484242000 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4e44104310 msgr2=0x7f4e441046f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:41.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.554+0000 7f4e43e00640 1 --2- 172.21.15.45:0/3484242000 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4e44104310 0x7f4e441046f0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f4e34009900 tx=0x7f4e3402bc40 comp rx=0 tx=0).stop 2024-04-16T17:09:41.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.554+0000 7f4e43e00640 1 -- 172.21.15.45:0/3484242000 shutdown_connections 2024-04-16T17:09:41.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.554+0000 7f4e43e00640 1 --2- 172.21.15.45:0/3484242000 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4e44104cc0 0x7f4e44108d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:41.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.554+0000 7f4e43e00640 1 --2- 172.21.15.45:0/3484242000 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4e441093e0 0x7f4e44110fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:41.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.554+0000 7f4e43e00640 1 --2- 172.21.15.45:0/3484242000 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4e44104310 0x7f4e441046f0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:41.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.554+0000 7f4e43e00640 1 -- 172.21.15.45:0/3484242000 >> 172.21.15.45:0/3484242000 conn(0x7f4e440ffb10 msgr2=0x7f4e44101f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:09:41.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.554+0000 7f4e43e00640 1 -- 172.21.15.45:0/3484242000 shutdown_connections 2024-04-16T17:09:41.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.554+0000 7f4e43e00640 1 -- 172.21.15.45:0/3484242000 wait complete. 2024-04-16T17:09:41.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.555+0000 7f4e43e00640 1 Processor -- start 2024-04-16T17:09:41.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.555+0000 7f4e43e00640 1 -- start start 2024-04-16T17:09:41.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.556+0000 7f4e43e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4e44104310 0x7f4e441a0660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:41.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.556+0000 7f4e43e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4e441093e0 0x7f4e441a0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:41.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.556+0000 7f4e43e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4e4419a820 0x7f4e4419acd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:41.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.556+0000 7f4e43e00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f4e441144e0 con 0x7f4e441093e0 2024-04-16T17:09:41.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.556+0000 7f4e43e00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f4e44114360 con 0x7f4e4419a820 2024-04-16T17:09:41.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.556+0000 7f4e43e00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f4e44114660 con 0x7f4e44104310 2024-04-16T17:09:41.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.556+0000 7f4e42000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4e441093e0 0x7f4e441a0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:41.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.556+0000 7f4e42000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4e441093e0 0x7f4e441a0ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:47778/0 (socket says 172.21.15.45:47778) 2024-04-16T17:09:41.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.556+0000 7f4e42000640 1 -- 172.21.15.45:0/1559566420 learned_addr learned my addr 172.21.15.45:0/1559566420 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:09:41.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.556+0000 7f4e42a00640 1 --2- 172.21.15.45:0/1559566420 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4e44104310 0x7f4e441a0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:41.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.556+0000 7f4e43400640 1 --2- 172.21.15.45:0/1559566420 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4e4419a820 0x7f4e4419acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:41.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.557+0000 7f4e42000640 1 -- 172.21.15.45:0/1559566420 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4e44104310 msgr2=0x7f4e441a0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:41.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.557+0000 7f4e42000640 1 --2- 172.21.15.45:0/1559566420 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4e44104310 0x7f4e441a0660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:41.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.557+0000 7f4e42000640 1 -- 172.21.15.45:0/1559566420 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4e4419a820 msgr2=0x7f4e4419acd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:41.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.557+0000 7f4e42000640 1 --2- 172.21.15.45:0/1559566420 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4e4419a820 0x7f4e4419acd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:41.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.557+0000 7f4e42000640 1 -- 172.21.15.45:0/1559566420 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e4419b3b0 con 0x7f4e441093e0 2024-04-16T17:09:41.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.557+0000 7f4e43400640 1 --2- 172.21.15.45:0/1559566420 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4e4419a820 0x7f4e4419acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:09:41.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.557+0000 7f4e42000640 1 --2- 172.21.15.45:0/1559566420 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4e441093e0 0x7f4e441a0ba0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f4e3000d740 tx=0x7f4e3000dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:09:41.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.557+0000 7f4e3b400640 1 -- 172.21.15.45:0/1559566420 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e30017070 con 0x7f4e441093e0 2024-04-16T17:09:41.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.557+0000 7f4e3b400640 1 -- 172.21.15.45:0/1559566420 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4e3000de50 con 0x7f4e441093e0 2024-04-16T17:09:41.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.557+0000 7f4e43e00640 1 -- 172.21.15.45:0/1559566420 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e4419b6a0 con 0x7f4e441093e0 2024-04-16T17:09:41.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.557+0000 7f4e3b400640 1 -- 172.21.15.45:0/1559566420 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e30004e00 con 0x7f4e441093e0 2024-04-16T17:09:41.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.558+0000 7f4e43e00640 1 -- 172.21.15.45:0/1559566420 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e441a7830 con 0x7f4e441093e0 2024-04-16T17:09:41.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.561+0000 7f4e4c3aa740 1 Processor -- start 2024-04-16T17:09:41.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.561+0000 7f4e4c3aa740 1 -- start start 2024-04-16T17:09:41.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.561+0000 7f4e3b400640 1 -- 172.21.15.45:0/1559566420 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f4e30004fa0 con 0x7f4e441093e0 2024-04-16T17:09:41.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.561+0000 7f4e4c3aa740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ea6bd91390 0x55ea6bd91770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:41.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.561+0000 7f4e4c3aa740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ea6bd91cb0 0x55ea6bd9a350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:41.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.561+0000 7f4e4c3aa740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ea6bd9a890 0x55ea6bd9cc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:41.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.561+0000 7f4e4c3aa740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55ea6bcf7120 con 0x55ea6bd91cb0 2024-04-16T17:09:41.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.561+0000 7f4e4c3aa740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55ea6bc9d340 con 0x55ea6bd91390 2024-04-16T17:09:41.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.561+0000 7f4e4c3aa740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55ea6bcf30e0 con 0x55ea6bd9a890 2024-04-16T17:09:41.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.562+0000 7f4e42a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ea6bd91cb0 0x55ea6bd9a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:41.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.562+0000 7f4e42a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ea6bd91cb0 0x55ea6bd9a350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:47792/0 (socket says 172.21.15.45:47792) 2024-04-16T17:09:41.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.562+0000 7f4e42a00640 1 -- 172.21.15.45:0/3651319193 learned_addr learned my addr 172.21.15.45:0/3651319193 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:09:41.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.562+0000 7f4e43400640 1 --2- 172.21.15.45:0/3651319193 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ea6bd91390 0x55ea6bd91770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:41.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.562+0000 7f4e42000640 1 --2- 172.21.15.45:0/3651319193 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ea6bd9a890 0x55ea6bd9cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:41.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.562+0000 7f4e42a00640 1 -- 172.21.15.45:0/3651319193 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ea6bd9a890 msgr2=0x55ea6bd9cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:09:41.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.562+0000 7f4e42a00640 1 --2- 172.21.15.45:0/3651319193 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ea6bd9a890 0x55ea6bd9cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:41.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.562+0000 7f4e42a00640 1 -- 172.21.15.45:0/3651319193 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ea6bd91390 msgr2=0x55ea6bd91770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:09:41.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.562+0000 7f4e42a00640 1 --2- 172.21.15.45:0/3651319193 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ea6bd91390 0x55ea6bd91770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:41.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.562+0000 7f4e42a00640 1 -- 172.21.15.45:0/3651319193 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ea6bad37d0 con 0x55ea6bd91cb0 2024-04-16T17:09:41.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.562+0000 7f4e42000640 1 --2- 172.21.15.45:0/3651319193 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ea6bd9a890 0x55ea6bd9cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:09:41.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.562+0000 7f4e43400640 1 --2- 172.21.15.45:0/3651319193 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ea6bd91390 0x55ea6bd91770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:09:41.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.563+0000 7f4e42a00640 1 --2- 172.21.15.45:0/3651319193 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ea6bd91cb0 0x55ea6bd9a350 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f4e34009900 tx=0x7f4e34000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70c2407b7f27921b server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:09:41.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.563+0000 7f4e3b400640 1 --2- 172.21.15.45:0/1559566420 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f4e280a3da0 0x7f4e280a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:41.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.563+0000 7f4e2f400640 1 -- 172.21.15.45:0/3651319193 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e34042070 con 0x55ea6bd91cb0 2024-04-16T17:09:41.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.563+0000 7f4e2f400640 1 -- 172.21.15.45:0/3651319193 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4e3402c6a0 con 0x55ea6bd91cb0 2024-04-16T17:09:41.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.563+0000 7f4e2f400640 1 -- 172.21.15.45:0/3651319193 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e34004f80 con 0x55ea6bd91cb0 2024-04-16T17:09:41.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.563+0000 7f4e3b400640 1 -- 172.21.15.45:0/1559566420 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f4e300d0440 con 0x7f4e441093e0 2024-04-16T17:09:41.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.563+0000 7f4e43400640 1 --2- 172.21.15.45:0/1559566420 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f4e280a3da0 0x7f4e280a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:41.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.564+0000 7f4e4c3aa740 1 -- 172.21.15.45:0/3651319193 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ea6bd91cb0 msgr2=0x55ea6bd9a350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:41.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.564+0000 7f4e4c3aa740 1 --2- 172.21.15.45:0/3651319193 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ea6bd91cb0 0x55ea6bd9a350 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f4e34009900 tx=0x7f4e34000bc0 comp rx=0 tx=0).stop 2024-04-16T17:09:41.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.564+0000 7f4e43400640 1 --2- 172.21.15.45:0/1559566420 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f4e280a3da0 0x7f4e280a6260 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f4e3c009fd0 tx=0x7f4e3c009500 comp rx=0 tx=0).ready entity=mgr.25411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:09:41.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.564+0000 7f4e4c3aa740 1 -- 172.21.15.45:0/3651319193 shutdown_connections 2024-04-16T17:09:41.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.564+0000 7f4e4c3aa740 1 --2- 172.21.15.45:0/3651319193 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ea6bd9a890 0x55ea6bd9cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:41.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.564+0000 7f4e4c3aa740 1 --2- 172.21.15.45:0/3651319193 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ea6bd91cb0 0x55ea6bd9a350 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:41.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.564+0000 7f4e4c3aa740 1 --2- 172.21.15.45:0/3651319193 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ea6bd91390 0x55ea6bd91770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:41.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.564+0000 7f4e4c3aa740 1 -- 172.21.15.45:0/3651319193 >> 172.21.15.45:0/3651319193 conn(0x55ea6bd8bac0 msgr2=0x55ea6bd8bea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:09:41.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.564+0000 7f4e4c3aa740 1 -- 172.21.15.45:0/3651319193 shutdown_connections 2024-04-16T17:09:41.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.564+0000 7f4e4c3aa740 1 -- 172.21.15.45:0/3651319193 wait complete. 2024-04-16T17:09:41.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.565+0000 7f4e4c3aa740 1 -- start start 2024-04-16T17:09:41.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.565+0000 7f4e4c3aa740 1 Processor -- start 2024-04-16T17:09:41.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.565+0000 7f4e4c3aa740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ea6bd91390 0x55ea6bd97940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:41.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.565+0000 7f4e4c3aa740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ea6bd91cb0 0x55ea6bd97e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:41.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.565+0000 7f4e4c3aa740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ea6bd9a890 0x55ea6bd983c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:41.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.565+0000 7f4e4c3aa740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55ea6bd9d620 con 0x55ea6bd91cb0 2024-04-16T17:09:41.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.565+0000 7f4e4c3aa740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55ea6bcb0710 con 0x55ea6bd91390 2024-04-16T17:09:41.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.565+0000 7f4e4c3aa740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55ea6bd9d7a0 con 0x55ea6bd9a890 2024-04-16T17:09:41.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.566+0000 7f4e4c3aa740 10 client.0 _ll_register_callbacks cb 0x55ea6bcabaf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:09:41.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.566+0000 7f4e42a00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ea6bd91390 0x55ea6bd97940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:41.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.566+0000 7f4e42000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ea6bd91cb0 0x55ea6bd97e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:41.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.566+0000 7f4e42a00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ea6bd91390 0x55ea6bd97940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:46930/0 (socket says 172.21.15.45:46930) 2024-04-16T17:09:41.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.566+0000 7f4e42a00640 1 -- 172.21.15.45:0/2569270593 learned_addr learned my addr 172.21.15.45:0/2569270593 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:09:41.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.566+0000 7f4e43400640 1 --2- 172.21.15.45:0/2569270593 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ea6bd9a890 0x55ea6bd983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:41.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.566+0000 7f4e42a00640 1 -- 172.21.15.45:0/2569270593 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ea6bd9a890 msgr2=0x55ea6bd983c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:09:41.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.566+0000 7f4e42a00640 1 --2- 172.21.15.45:0/2569270593 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ea6bd9a890 0x55ea6bd983c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:41.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.566+0000 7f4e42a00640 1 -- 172.21.15.45:0/2569270593 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ea6bd91cb0 msgr2=0x55ea6bd97e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:09:41.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.566+0000 7f4e42a00640 1 --2- 172.21.15.45:0/2569270593 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ea6bd91cb0 0x55ea6bd97e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:41.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.566+0000 7f4e42a00640 1 -- 172.21.15.45:0/2569270593 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ea6ba093f0 con 0x55ea6bd91390 2024-04-16T17:09:41.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.566+0000 7f4e42000640 1 --2- 172.21.15.45:0/2569270593 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ea6bd91cb0 0x55ea6bd97e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:09:41.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.566+0000 7f4e42a00640 1 --2- 172.21.15.45:0/2569270593 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ea6bd91390 0x55ea6bd97940 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f4e34005c40 tx=0x7f4e3402cac0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e652365aa597c40c server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:09:41.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.566+0000 7f4e23e00640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-04-16T17:09:41.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.567+0000 7f4e23e00640 1 -- 172.21.15.45:0/2569270593 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e34042070 con 0x55ea6bd91390 2024-04-16T17:09:41.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.567+0000 7f4e23e00640 1 -- 172.21.15.45:0/2569270593 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f4e34005180 con 0x55ea6bd91390 2024-04-16T17:09:41.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.567+0000 7f4e4c3aa740 1 -- 172.21.15.45:0/2569270593 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55ea6bd04b40 con 0x55ea6bd91390 2024-04-16T17:09:41.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.567+0000 7f4e23e00640 1 -- 172.21.15.45:0/2569270593 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f4e34058050 con 0x55ea6bd91390 2024-04-16T17:09:41.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.567+0000 7f4e4c3aa740 10 client.26417 fetch_fsmap learned FSMap version 30 2024-04-16T17:09:41.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.567+0000 7f4e4c3aa740 1 -- 172.21.15.45:0/2569270593 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55ea6b9c90f0 con 0x55ea6bd91390 2024-04-16T17:09:41.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.567+0000 7f4e23e00640 1 -- 172.21.15.45:0/2569270593 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55ea6bd04b40 con 0x55ea6bd91390 2024-04-16T17:09:41.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.568+0000 7f4e23e00640 1 -- 172.21.15.45:0/2569270593 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f4e34047cd0 con 0x55ea6bd91390 2024-04-16T17:09:41.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.568+0000 7f4e4c3aa740 10 client.26417 fetch_fsmap finished waiting for FSMap version 30 2024-04-16T17:09:41.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.568+0000 7f4e4c3aa740 10 client.26417 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:09:41.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.568+0000 7f4e4c3aa740 20 client.26417 populate_metadata read hostname 'smithi045' 2024-04-16T17:09:41.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.568+0000 7f4e4c3aa740 1 --2- 172.21.15.45:0/2569270593 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55ea6bdaaad0 0x55ea6bdacec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:41.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.568+0000 7f4e4c3aa740 4 client.26417 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:09:41.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.568+0000 7f4e4c3aa740 1 -- 172.21.15.45:0/2569270593 --> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ea6bcfe9b0 con 0x55ea6bdaaad0 2024-04-16T17:09:41.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.568+0000 7f4e42000640 1 --2- 172.21.15.45:0/2569270593 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55ea6bdaaad0 0x55ea6bdacec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:41.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.569+0000 7f4e42000640 1 --2- 172.21.15.45:0/2569270593 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55ea6bdaaad0 0x55ea6bdacec0 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9f389ca37e8760a0 server_cookie=d468679653335f5f in_seq=0 out_seq=0 2024-04-16T17:09:41.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.569+0000 7f4e23e00640 10 client.26417 ms_handle_connect on v2:172.21.15.104:6838/3660070592 2024-04-16T17:09:41.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.569+0000 7f4e23e00640 1 -- 172.21.15.45:0/2569270593 <== mds.0 v2:172.21.15.104:6838/3660070592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ea6bcfe9b0 con 0x55ea6bdaaad0 2024-04-16T17:09:41.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.569+0000 7f4e23e00640 10 client.26417 handle_command_reply: tid=0 2024-04-16T17:09:41.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.569+0000 7f4e4c3aa740 1 client.26417 shutdown 2024-04-16T17:09:41.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.570+0000 7f4e4c3aa740 1 -- 172.21.15.45:0/2569270593 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ea6bd91390 msgr2=0x55ea6bd97940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:41.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.570+0000 7f4e4c3aa740 1 --2- 172.21.15.45:0/2569270593 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ea6bd91390 0x55ea6bd97940 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f4e34005c40 tx=0x7f4e3402cac0 comp rx=0 tx=0).stop 2024-04-16T17:09:41.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.570+0000 7f4e4c3aa740 1 -- 172.21.15.45:0/2569270593 shutdown_connections 2024-04-16T17:09:41.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.570+0000 7f4e4c3aa740 1 --2- 172.21.15.45:0/2569270593 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55ea6bdaaad0 0x55ea6bdacec0 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:41.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.570+0000 7f4e4c3aa740 1 --2- 172.21.15.45:0/2569270593 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ea6bd9a890 0x55ea6bd983c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:41.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.570+0000 7f4e4c3aa740 1 --2- 172.21.15.45:0/2569270593 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ea6bd91cb0 0x55ea6bd97e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:41.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.570+0000 7f4e4c3aa740 1 --2- 172.21.15.45:0/2569270593 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ea6bd91390 0x55ea6bd97940 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:41.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.570+0000 7f4e23e00640 0 client.26417 ms_handle_reset on v2:172.21.15.104:6838/3660070592 2024-04-16T17:09:41.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.570+0000 7f4e4c3aa740 1 -- 172.21.15.45:0/2569270593 >> 172.21.15.45:0/2569270593 conn(0x55ea6bd8bac0 msgr2=0x55ea6bd8d810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:09:41.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.570+0000 7f4e4c3aa740 1 -- 172.21.15.45:0/2569270593 shutdown_connections 2024-04-16T17:09:41.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.570+0000 7f4e4c3aa740 1 -- 172.21.15.45:0/2569270593 wait complete. 2024-04-16T17:09:41.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.570+0000 7f4e4c3aa740 20 client.26417 trim_cache size 0 max 16384 2024-04-16T17:09:41.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.589+0000 7f4e4c3aa740 1 Processor -- start 2024-04-16T17:09:41.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.589+0000 7f4e4c3aa740 1 -- start start 2024-04-16T17:09:41.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.589+0000 7f4e4c3aa740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ea6bd91390 0x55ea6bd9a070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:41.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.589+0000 7f4e4c3aa740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ea6bd91cb0 0x55ea6bda64e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:41.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.589+0000 7f4e4c3aa740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ea6bd9a890 0x55ea6bdb24f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:41.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.589+0000 7f4e4c3aa740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f4e441b3cc0 con 0x55ea6bd91cb0 2024-04-16T17:09:41.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.589+0000 7f4e4c3aa740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f4e441b14d0 con 0x55ea6bd9a890 2024-04-16T17:09:41.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.589+0000 7f4e4c3aa740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f4e441af300 con 0x55ea6bd91390 2024-04-16T17:09:41.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.589+0000 7f4e42000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ea6bd91cb0 0x55ea6bda64e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:41.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.589+0000 7f4e42000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ea6bd91cb0 0x55ea6bda64e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:47808/0 (socket says 172.21.15.45:47808) 2024-04-16T17:09:41.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.589+0000 7f4e42000640 1 -- 172.21.15.45:0/1923950467 learned_addr learned my addr 172.21.15.45:0/1923950467 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:09:41.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.590+0000 7f4e42a00640 1 --2- 172.21.15.45:0/1923950467 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ea6bd91390 0x55ea6bd9a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:41.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.590+0000 7f4e43400640 1 --2- 172.21.15.45:0/1923950467 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ea6bd9a890 0x55ea6bdb24f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:41.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.590+0000 7f4e42000640 1 -- 172.21.15.45:0/1923950467 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ea6bd91390 msgr2=0x55ea6bd9a070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:09:41.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.590+0000 7f4e42000640 1 --2- 172.21.15.45:0/1923950467 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ea6bd91390 0x55ea6bd9a070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:41.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.590+0000 7f4e42000640 1 -- 172.21.15.45:0/1923950467 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ea6bd9a890 msgr2=0x55ea6bdb24f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:09:41.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.590+0000 7f4e42000640 1 --2- 172.21.15.45:0/1923950467 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ea6bd9a890 0x55ea6bdb24f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:41.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.590+0000 7f4e42000640 1 -- 172.21.15.45:0/1923950467 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ea6bd95e90 con 0x55ea6bd91cb0 2024-04-16T17:09:41.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.590+0000 7f4e43400640 1 --2- 172.21.15.45:0/1923950467 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ea6bd9a890 0x55ea6bdb24f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:09:41.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.590+0000 7f4e42000640 1 --2- 172.21.15.45:0/1923950467 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ea6bd91cb0 0x55ea6bda64e0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f4e30098ab0 tx=0x7f4e300d2890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=174861c0c4f25e53 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:09:41.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.590+0000 7f4e2ea00640 1 -- 172.21.15.45:0/1923950467 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e30020040 con 0x55ea6bd91cb0 2024-04-16T17:09:41.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.590+0000 7f4e2ea00640 1 -- 172.21.15.45:0/1923950467 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4e300a4ca0 con 0x55ea6bd91cb0 2024-04-16T17:09:41.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.590+0000 7f4e2ea00640 1 -- 172.21.15.45:0/1923950467 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e30009cc0 con 0x55ea6bd91cb0 2024-04-16T17:09:41.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.591+0000 7f4e4c3aa740 1 -- 172.21.15.45:0/1923950467 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ea6bd91cb0 msgr2=0x55ea6bda64e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:41.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.591+0000 7f4e4c3aa740 1 --2- 172.21.15.45:0/1923950467 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ea6bd91cb0 0x55ea6bda64e0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f4e30098ab0 tx=0x7f4e300d2890 comp rx=0 tx=0).stop 2024-04-16T17:09:41.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.591+0000 7f4e4c3aa740 1 -- 172.21.15.45:0/1923950467 shutdown_connections 2024-04-16T17:09:41.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.591+0000 7f4e4c3aa740 1 --2- 172.21.15.45:0/1923950467 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ea6bd9a890 0x55ea6bdb24f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:41.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.591+0000 7f4e4c3aa740 1 --2- 172.21.15.45:0/1923950467 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ea6bd91cb0 0x55ea6bda64e0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:41.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.591+0000 7f4e4c3aa740 1 --2- 172.21.15.45:0/1923950467 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ea6bd91390 0x55ea6bd9a070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:41.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.591+0000 7f4e4c3aa740 1 -- 172.21.15.45:0/1923950467 >> 172.21.15.45:0/1923950467 conn(0x55ea6bd8bac0 msgr2=0x55ea6bd96aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:09:41.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.591+0000 7f4e4c3aa740 1 -- 172.21.15.45:0/1923950467 shutdown_connections 2024-04-16T17:09:41.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.591+0000 7f4e4c3aa740 1 -- 172.21.15.45:0/1923950467 wait complete. 2024-04-16T17:09:41.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.592+0000 7f4e4c3aa740 1 -- start start 2024-04-16T17:09:41.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.592+0000 7f4e4c3aa740 1 Processor -- start 2024-04-16T17:09:41.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.592+0000 7f4e4c3aa740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ea6bd91390 0x55ea6bd94980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:41.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.592+0000 7f4e4c3aa740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ea6bd91cb0 0x55ea6bd94ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:41.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.592+0000 7f4e4c3aa740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ea6bd9a890 0x55ea6be0fd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:41.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.592+0000 7f4e4c3aa740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f4e441af990 con 0x55ea6bd91cb0 2024-04-16T17:09:41.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.592+0000 7f4e4c3aa740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f4e441b0e00 con 0x55ea6bd9a890 2024-04-16T17:09:41.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.592+0000 7f4e4c3aa740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f4e441b0060 con 0x55ea6bd91390 2024-04-16T17:09:41.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.592+0000 7f4e4c3aa740 10 client.0 _ll_register_callbacks cb 0x7f4e3c009ee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:09:41.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.592+0000 7f4e42000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ea6bd91cb0 0x55ea6bd94ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:41.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.592+0000 7f4e42000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ea6bd91cb0 0x55ea6bd94ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:47810/0 (socket says 172.21.15.45:47810) 2024-04-16T17:09:41.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.592+0000 7f4e42000640 1 -- 172.21.15.45:0/462959660 learned_addr learned my addr 172.21.15.45:0/462959660 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:09:41.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.593+0000 7f4e43400640 1 --2- 172.21.15.45:0/462959660 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ea6bd9a890 0x55ea6be0fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:41.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.593+0000 7f4e42a00640 1 --2- 172.21.15.45:0/462959660 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ea6bd91390 0x55ea6bd94980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:41.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.593+0000 7f4e42000640 1 -- 172.21.15.45:0/462959660 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ea6bd91390 msgr2=0x55ea6bd94980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:09:41.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.593+0000 7f4e42000640 1 --2- 172.21.15.45:0/462959660 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ea6bd91390 0x55ea6bd94980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:41.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.593+0000 7f4e42000640 1 -- 172.21.15.45:0/462959660 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ea6bd9a890 msgr2=0x55ea6be0fd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:09:41.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.593+0000 7f4e42000640 1 --2- 172.21.15.45:0/462959660 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ea6bd9a890 0x55ea6be0fd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:41.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.593+0000 7f4e42000640 1 -- 172.21.15.45:0/462959660 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ea6be102a0 con 0x55ea6bd91cb0 2024-04-16T17:09:41.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.593+0000 7f4e42a00640 1 --2- 172.21.15.45:0/462959660 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ea6bd91390 0x55ea6bd94980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:09:41.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.593+0000 7f4e42000640 1 --2- 172.21.15.45:0/462959660 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ea6bd91cb0 0x55ea6bd94ec0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f4e300a4770 tx=0x7f4e300d01d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ac3babda523f480 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:09:41.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.593+0000 7f4e23e00640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-16T17:09:41.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.593+0000 7f4e23e00640 1 -- 172.21.15.45:0/462959660 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e30020040 con 0x55ea6bd91cb0 2024-04-16T17:09:41.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.593+0000 7f4e23e00640 1 -- 172.21.15.45:0/462959660 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f4e30009cc0 con 0x55ea6bd91cb0 2024-04-16T17:09:41.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.593+0000 7f4e4c3aa740 1 -- 172.21.15.45:0/462959660 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4e300a4ca0 con 0x55ea6bd91cb0 2024-04-16T17:09:41.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.594+0000 7f4e23e00640 1 -- 172.21.15.45:0/462959660 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f4e300d0270 con 0x55ea6bd91cb0 2024-04-16T17:09:41.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.594+0000 7f4e4c3aa740 10 client.16272 fetch_fsmap learned FSMap version 30 2024-04-16T17:09:41.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.594+0000 7f4e4c3aa740 1 -- 172.21.15.45:0/462959660 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55ea6be11420 con 0x55ea6bd91cb0 2024-04-16T17:09:41.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.594+0000 7f4e23e00640 1 -- 172.21.15.45:0/462959660 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4e300a4ca0 con 0x55ea6bd91cb0 2024-04-16T17:09:41.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.594+0000 7f4e23e00640 1 -- 172.21.15.45:0/462959660 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f4e3009b5c0 con 0x55ea6bd91cb0 2024-04-16T17:09:41.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.594+0000 7f4e4c3aa740 10 client.16272 fetch_fsmap finished waiting for FSMap version 30 2024-04-16T17:09:41.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.594+0000 7f4e4c3aa740 10 client.16272 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:09:41.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.594+0000 7f4e4c3aa740 20 client.16272 populate_metadata read hostname 'smithi045' 2024-04-16T17:09:41.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.594+0000 7f4e4c3aa740 1 --2- 172.21.15.45:0/462959660 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55ea6be11be0 0x55ea6be14000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:41.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.594+0000 7f4e4c3aa740 4 client.16272 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-16T17:09:41.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.594+0000 7f4e4c3aa740 1 -- 172.21.15.45:0/462959660 --> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55ea6bc7a420 con 0x55ea6be11be0 2024-04-16T17:09:41.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.594+0000 7f4e42a00640 1 --2- 172.21.15.45:0/462959660 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55ea6be11be0 0x55ea6be14000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:41.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.595+0000 7f4e42a00640 1 --2- 172.21.15.45:0/462959660 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55ea6be11be0 0x55ea6be14000 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ab368c66679fc5f1 server_cookie=72c3e69a6ba37620 in_seq=0 out_seq=0 2024-04-16T17:09:41.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.595+0000 7f4e23e00640 10 client.16272 ms_handle_connect on v2:172.21.15.104:6838/3660070592 2024-04-16T17:09:41.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.595+0000 7f4e23e00640 1 -- 172.21.15.45:0/462959660 <== mds.0 v2:172.21.15.104:6838/3660070592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55ea6bc7a420 con 0x55ea6be11be0 2024-04-16T17:09:41.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.595+0000 7f4e23e00640 10 client.16272 handle_command_reply: tid=0 2024-04-16T17:09:41.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.595+0000 7f4e4c3aa740 1 client.16272 shutdown 2024-04-16T17:09:41.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.595+0000 7f4e4c3aa740 1 -- 172.21.15.45:0/462959660 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ea6bd91cb0 msgr2=0x55ea6bd94ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:41.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.595+0000 7f4e4c3aa740 1 --2- 172.21.15.45:0/462959660 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ea6bd91cb0 0x55ea6bd94ec0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f4e300a4770 tx=0x7f4e300d01d0 comp rx=0 tx=0).stop 2024-04-16T17:09:41.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.595+0000 7f4e4c3aa740 1 -- 172.21.15.45:0/462959660 shutdown_connections 2024-04-16T17:09:41.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.595+0000 7f4e4c3aa740 1 --2- 172.21.15.45:0/462959660 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ea6bd9a890 0x55ea6be0fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:41.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.595+0000 7f4e4c3aa740 1 --2- 172.21.15.45:0/462959660 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ea6bd91cb0 0x55ea6bd94ec0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:41.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.595+0000 7f4e4c3aa740 1 --2- 172.21.15.45:0/462959660 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55ea6be11be0 0x55ea6be14000 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:41.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.595+0000 7f4e4c3aa740 1 --2- 172.21.15.45:0/462959660 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ea6bd91390 0x55ea6bd94980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:41.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.595+0000 7f4e4c3aa740 1 -- 172.21.15.45:0/462959660 >> 172.21.15.45:0/462959660 conn(0x55ea6bd8bac0 msgr2=0x55ea6bd978e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:09:41.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.595+0000 7f4e23e00640 0 client.16272 ms_handle_reset on v2:172.21.15.104:6838/3660070592 2024-04-16T17:09:41.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.595+0000 7f4e4c3aa740 1 -- 172.21.15.45:0/462959660 shutdown_connections 2024-04-16T17:09:41.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.596+0000 7f4e4c3aa740 1 -- 172.21.15.45:0/462959660 wait complete. 2024-04-16T17:09:41.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.596+0000 7f4e4c3aa740 20 client.16272 trim_cache size 0 max 16384 2024-04-16T17:09:41.601 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-16T17:09:41.601 INFO:teuthology.orchestra.run.smithi045.stdout: "status": "no active scrubs running", 2024-04-16T17:09:41.601 INFO:teuthology.orchestra.run.smithi045.stdout: "scrubs": {} 2024-04-16T17:09:41.601 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-16T17:09:41.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.596+0000 7f4e43e00640 1 -- 172.21.15.45:0/1559566420 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f4e280a3da0 msgr2=0x7f4e280a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:41.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.596+0000 7f4e43e00640 1 --2- 172.21.15.45:0/1559566420 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f4e280a3da0 0x7f4e280a6260 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f4e3c009fd0 tx=0x7f4e3c009500 comp rx=0 tx=0).stop 2024-04-16T17:09:41.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.596+0000 7f4e43e00640 1 -- 172.21.15.45:0/1559566420 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4e441093e0 msgr2=0x7f4e441a0ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:41.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.596+0000 7f4e43e00640 1 --2- 172.21.15.45:0/1559566420 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4e441093e0 0x7f4e441a0ba0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f4e3000d740 tx=0x7f4e3000dc00 comp rx=0 tx=0).stop 2024-04-16T17:09:41.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.596+0000 7f4e43e00640 1 -- 172.21.15.45:0/1559566420 shutdown_connections 2024-04-16T17:09:41.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.596+0000 7f4e43e00640 1 --2- 172.21.15.45:0/1559566420 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f4e280a3da0 0x7f4e280a6260 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:41.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.596+0000 7f4e43e00640 1 --2- 172.21.15.45:0/1559566420 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4e4419a820 0x7f4e4419acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:41.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.596+0000 7f4e43e00640 1 --2- 172.21.15.45:0/1559566420 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4e441093e0 0x7f4e441a0ba0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:41.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.596+0000 7f4e43e00640 1 --2- 172.21.15.45:0/1559566420 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4e44104310 0x7f4e441a0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:41.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.596+0000 7f4e43e00640 1 -- 172.21.15.45:0/1559566420 >> 172.21.15.45:0/1559566420 conn(0x7f4e440ffb10 msgr2=0x7f4e44101dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:09:41.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.597+0000 7f4e43e00640 1 -- 172.21.15.45:0/1559566420 shutdown_connections 2024-04-16T17:09:41.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.597+0000 7f4e43e00640 1 -- 172.21.15.45:0/1559566420 wait complete. 2024-04-16T17:09:41.616 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-16T17:09:41.616 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-16T17:09:41.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.824+0000 7f2e67000640 1 -- 172.21.15.45:0/1300784470 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2e60069db0 msgr2=0x7f2e60106a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:41.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.824+0000 7f2e67000640 1 --2- 172.21.15.45:0/1300784470 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2e60069db0 0x7f2e60106a70 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f2e4c009860 tx=0x7f2e4c02bb70 comp rx=0 tx=0).stop 2024-04-16T17:09:41.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.824+0000 7f2e67000640 1 -- 172.21.15.45:0/1300784470 shutdown_connections 2024-04-16T17:09:41.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.824+0000 7f2e67000640 1 --2- 172.21.15.45:0/1300784470 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2e60069db0 0x7f2e60106a70 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:41.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.824+0000 7f2e67000640 1 --2- 172.21.15.45:0/1300784470 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2e601071a0 0x7f2e6010ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:41.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.824+0000 7f2e67000640 1 --2- 172.21.15.45:0/1300784470 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2e60069400 0x7f2e600697e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:41.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.824+0000 7f2e67000640 1 -- 172.21.15.45:0/1300784470 >> 172.21.15.45:0/1300784470 conn(0x7f2e60076840 msgr2=0x7f2e60076c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:09:41.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.824+0000 7f2e67000640 1 -- 172.21.15.45:0/1300784470 shutdown_connections 2024-04-16T17:09:41.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.824+0000 7f2e67000640 1 -- 172.21.15.45:0/1300784470 wait complete. 2024-04-16T17:09:41.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.825+0000 7f2e67000640 1 Processor -- start 2024-04-16T17:09:41.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.826+0000 7f2e67000640 1 -- start start 2024-04-16T17:09:41.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.826+0000 7f2e67000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2e60069400 0x7f2e601a0620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:41.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.826+0000 7f2e67000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2e601071a0 0x7f2e601a0b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:41.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.826+0000 7f2e67000640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2e6019a790 0x7f2e6019ac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:41.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.826+0000 7f2e67000640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f2e601121f0 con 0x7f2e60069400 2024-04-16T17:09:41.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.826+0000 7f2e67000640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f2e60112070 con 0x7f2e6019a790 2024-04-16T17:09:41.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.826+0000 7f2e67000640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f2e60112370 con 0x7f2e601071a0 2024-04-16T17:09:41.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.826+0000 7f2e65c00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2e60069400 0x7f2e601a0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:41.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.827+0000 7f2e65c00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2e60069400 0x7f2e601a0620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:47840/0 (socket says 172.21.15.45:47840) 2024-04-16T17:09:41.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.827+0000 7f2e65c00640 1 -- 172.21.15.45:0/582844847 learned_addr learned my addr 172.21.15.45:0/582844847 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:09:41.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.827+0000 7f2e65200640 1 --2- 172.21.15.45:0/582844847 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2e601071a0 0x7f2e601a0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:41.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.827+0000 7f2e66600640 1 --2- 172.21.15.45:0/582844847 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2e6019a790 0x7f2e6019ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:41.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.827+0000 7f2e65c00640 1 -- 172.21.15.45:0/582844847 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2e601071a0 msgr2=0x7f2e601a0b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:41.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.827+0000 7f2e65c00640 1 --2- 172.21.15.45:0/582844847 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2e601071a0 0x7f2e601a0b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:41.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.827+0000 7f2e65c00640 1 -- 172.21.15.45:0/582844847 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2e6019a790 msgr2=0x7f2e6019ac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:41.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.827+0000 7f2e65c00640 1 --2- 172.21.15.45:0/582844847 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2e6019a790 0x7f2e6019ac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:41.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.827+0000 7f2e65c00640 1 -- 172.21.15.45:0/582844847 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e6019b320 con 0x7f2e60069400 2024-04-16T17:09:41.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.827+0000 7f2e65200640 1 --2- 172.21.15.45:0/582844847 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2e601071a0 0x7f2e601a0b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:09:41.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.827+0000 7f2e66600640 1 --2- 172.21.15.45:0/582844847 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2e6019a790 0x7f2e6019ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:09:41.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.827+0000 7f2e65c00640 1 --2- 172.21.15.45:0/582844847 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2e60069400 0x7f2e601a0620 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f2e5400e9c0 tx=0x7f2e5400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:09:41.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.828+0000 7f2e5e000640 1 -- 172.21.15.45:0/582844847 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e5400cde0 con 0x7f2e60069400 2024-04-16T17:09:41.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.828+0000 7f2e5e000640 1 -- 172.21.15.45:0/582844847 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2e540035f0 con 0x7f2e60069400 2024-04-16T17:09:41.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.828+0000 7f2e67000640 1 -- 172.21.15.45:0/582844847 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e6019b610 con 0x7f2e60069400 2024-04-16T17:09:41.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.828+0000 7f2e5e000640 1 -- 172.21.15.45:0/582844847 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e54010690 con 0x7f2e60069400 2024-04-16T17:09:41.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.828+0000 7f2e67000640 1 -- 172.21.15.45:0/582844847 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e601a7810 con 0x7f2e60069400 2024-04-16T17:09:41.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.829+0000 7f2e67000640 1 -- 172.21.15.45:0/582844847 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e60103550 con 0x7f2e60069400 2024-04-16T17:09:41.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.833+0000 7f2e5e000640 1 -- 172.21.15.45:0/582844847 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f2e540037a0 con 0x7f2e60069400 2024-04-16T17:09:41.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.834+0000 7f2e5e000640 1 --2- 172.21.15.45:0/582844847 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f2e480a3da0 0x7f2e480a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:41.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.834+0000 7f2e5e000640 1 -- 172.21.15.45:0/582844847 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f2e540d0ff0 con 0x7f2e60069400 2024-04-16T17:09:41.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.834+0000 7f2e65200640 1 --2- 172.21.15.45:0/582844847 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f2e480a3da0 0x7f2e480a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:41.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.835+0000 7f2e65200640 1 --2- 172.21.15.45:0/582844847 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f2e480a3da0 0x7f2e480a6260 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f2e4c005940 tx=0x7f2e4c02e040 comp rx=0 tx=0).ready entity=mgr.25411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:09:41.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:41.837+0000 7f2e5e000640 1 -- 172.21.15.45:0/582844847 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2e6019b610 con 0x7f2e60069400 2024-04-16T17:09:42.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.083+0000 7f2e67000640 1 -- 172.21.15.45:0/582844847 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2e60068d40 con 0x7f2e60069400 2024-04-16T17:09:42.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.084+0000 7f2e5e000640 1 -- 172.21.15.45:0/582844847 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11896 (secure 0 0 0) 0x7f2e540a3020 con 0x7f2e60069400 2024-04-16T17:09:42.085 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-16T17:09:42.085 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/1748842443","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":1748842443},{"type":"v1","addr":"172.21.15.45:6835","nonce":1748842443}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6839/1308627051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":1308627051},{"type":"v1","addr":"172.21.15.45:6839","nonce":1308627051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/1768242489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1768242489},{"type":"v1","addr":"172.21.15.117:6833","nonce":1768242489}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6833/3558902585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":3558902585},{"type":"v1","addr":"172.21.15.104:6833","nonce":3558902585}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24478,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/1308142903","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":1308142903},{"type":"v1","addr":"172.21.15.45:6837","nonce":1308142903}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24491,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6835/3430679937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":3430679937},{"type":"v1","addr":"172.21.15.104:6835","nonce":3430679937}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-16T17:03:24.663611+0000","modified":"2024-04-16T17:09:20.713725+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24581,"mds_1":24569,"mds_2":24532},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24499":{"gid":24499,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/2224123841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2224123841},{"type":"v1","addr":"172.21.15.117:6835","nonce":2224123841}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6837/2471356815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":2471356815},{"type":"v1","addr":"172.21.15.117:6837","nonce":2471356815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24524":{"gid":24524,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6837/2647756509","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":2647756509},{"type":"v1","addr":"172.21.15.104:6837","nonce":2647756509}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24532":{"gid":24532,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":13,"addr":"172.21.15.117:6839/2378675949","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":2378675949},{"type":"v1","addr":"172.21.15.117:6839","nonce":2378675949}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24569":{"gid":24569,"name":"a","rank":1,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.45:6841/3755310822","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6840","nonce":3755310822},{"type":"v1","addr":"172.21.15.45:6841","nonce":3755310822}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24581":{"gid":24581,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.104:6839/3660070592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6838","nonce":3660070592},{"type":"v1","addr":"172.21.15.104:6839","nonce":3660070592}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24581,"qdb_cluster":[24569,24532,24581]},"id":1}]} 2024-04-16T17:09:42.085 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 30 2024-04-16T17:09:42.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.086+0000 7f2e67000640 1 -- 172.21.15.45:0/582844847 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f2e480a3da0 msgr2=0x7f2e480a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:42.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.086+0000 7f2e67000640 1 --2- 172.21.15.45:0/582844847 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f2e480a3da0 0x7f2e480a6260 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f2e4c005940 tx=0x7f2e4c02e040 comp rx=0 tx=0).stop 2024-04-16T17:09:42.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.086+0000 7f2e67000640 1 -- 172.21.15.45:0/582844847 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2e60069400 msgr2=0x7f2e601a0620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:42.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.086+0000 7f2e67000640 1 --2- 172.21.15.45:0/582844847 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2e60069400 0x7f2e601a0620 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f2e5400e9c0 tx=0x7f2e5400ee80 comp rx=0 tx=0).stop 2024-04-16T17:09:42.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.086+0000 7f2e67000640 1 -- 172.21.15.45:0/582844847 shutdown_connections 2024-04-16T17:09:42.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.086+0000 7f2e67000640 1 --2- 172.21.15.45:0/582844847 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f2e480a3da0 0x7f2e480a6260 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:42.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.086+0000 7f2e67000640 1 --2- 172.21.15.45:0/582844847 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2e601071a0 0x7f2e601a0b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:42.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.086+0000 7f2e67000640 1 --2- 172.21.15.45:0/582844847 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2e6019a790 0x7f2e6019ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:42.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.086+0000 7f2e67000640 1 --2- 172.21.15.45:0/582844847 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2e60069400 0x7f2e601a0620 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:42.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.086+0000 7f2e67000640 1 -- 172.21.15.45:0/582844847 >> 172.21.15.45:0/582844847 conn(0x7f2e60076840 msgr2=0x7f2e6010d1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:09:42.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.087+0000 7f2e67000640 1 -- 172.21.15.45:0/582844847 shutdown_connections 2024-04-16T17:09:42.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.087+0000 7f2e67000640 1 -- 172.21.15.45:0/582844847 wait complete. 2024-04-16T17:09:42.106 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-16T17:09:42.227 INFO:tasks.workunit.client.0.smithi045.stdout: 1 40034 2.23 MB/sec warmup 104 sec latency 4523.119 ms 2024-04-16T17:09:42.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.321+0000 7f784e800640 1 -- 172.21.15.45:0/3536203417 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f784810d500 msgr2=0x7f784810d8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:42.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.321+0000 7f784e800640 1 --2- 172.21.15.45:0/3536203417 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f784810d500 0x7f784810d8e0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f783c009890 tx=0x7f783c02bb70 comp rx=0 tx=0).stop 2024-04-16T17:09:42.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.321+0000 7f784e800640 1 -- 172.21.15.45:0/3536203417 shutdown_connections 2024-04-16T17:09:42.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.321+0000 7f784e800640 1 --2- 172.21.15.45:0/3536203417 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7848108f40 0x7f78481153b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:42.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.321+0000 7f784e800640 1 --2- 172.21.15.45:0/3536203417 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7848108580 0x7f7848108a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:42.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.321+0000 7f784e800640 1 --2- 172.21.15.45:0/3536203417 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f784810d500 0x7f784810d8e0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:42.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.321+0000 7f784e800640 1 -- 172.21.15.45:0/3536203417 >> 172.21.15.45:0/3536203417 conn(0x7f7848077e20 msgr2=0x7f7848106e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:09:42.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.321+0000 7f784e800640 1 -- 172.21.15.45:0/3536203417 shutdown_connections 2024-04-16T17:09:42.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.321+0000 7f784e800640 1 -- 172.21.15.45:0/3536203417 wait complete. 2024-04-16T17:09:42.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.322+0000 7f784e800640 1 Processor -- start 2024-04-16T17:09:42.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.322+0000 7f784e800640 1 -- start start 2024-04-16T17:09:42.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.322+0000 7f784e800640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7848108580 0x7f784819ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:42.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.322+0000 7f784e800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7848108f40 0x7f784819f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:42.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.322+0000 7f784e800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f784810d500 0x7f78481a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:42.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.322+0000 7f784e800640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f78481188b0 con 0x7f7848108f40 2024-04-16T17:09:42.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.322+0000 7f784e800640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f7848118730 con 0x7f7848108580 2024-04-16T17:09:42.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.322+0000 7f784e800640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f7848118a30 con 0x7f784810d500 2024-04-16T17:09:42.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.323+0000 7f784ca00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7848108f40 0x7f784819f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:42.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.323+0000 7f784ca00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7848108f40 0x7f784819f2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:47858/0 (socket says 172.21.15.45:47858) 2024-04-16T17:09:42.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.323+0000 7f784ca00640 1 -- 172.21.15.45:0/439417814 learned_addr learned my addr 172.21.15.45:0/439417814 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:09:42.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.323+0000 7f784de00640 1 --2- 172.21.15.45:0/439417814 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f784810d500 0x7f78481a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:42.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.323+0000 7f784d400640 1 --2- 172.21.15.45:0/439417814 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7848108580 0x7f784819ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:42.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.323+0000 7f784ca00640 1 -- 172.21.15.45:0/439417814 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f784810d500 msgr2=0x7f78481a35e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:42.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.323+0000 7f784ca00640 1 --2- 172.21.15.45:0/439417814 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f784810d500 0x7f78481a35e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:42.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.323+0000 7f784ca00640 1 -- 172.21.15.45:0/439417814 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7848108580 msgr2=0x7f784819ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:42.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.323+0000 7f784ca00640 1 --2- 172.21.15.45:0/439417814 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7848108580 0x7f784819ed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:42.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.323+0000 7f784ca00640 1 -- 172.21.15.45:0/439417814 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78481a3d10 con 0x7f7848108f40 2024-04-16T17:09:42.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.323+0000 7f784de00640 1 --2- 172.21.15.45:0/439417814 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f784810d500 0x7f78481a35e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:09:42.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.324+0000 7f784ca00640 1 --2- 172.21.15.45:0/439417814 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7848108f40 0x7f784819f2b0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f783400e9f0 tx=0x7f783400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:09:42.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.324+0000 7f7846000640 1 -- 172.21.15.45:0/439417814 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f783400cd80 con 0x7f7848108f40 2024-04-16T17:09:42.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.324+0000 7f7846000640 1 -- 172.21.15.45:0/439417814 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7834010910 con 0x7f7848108f40 2024-04-16T17:09:42.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.324+0000 7f784e800640 1 -- 172.21.15.45:0/439417814 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78481a4000 con 0x7f7848108f40 2024-04-16T17:09:42.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.324+0000 7f7846000640 1 -- 172.21.15.45:0/439417814 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7834018990 con 0x7f7848108f40 2024-04-16T17:09:42.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.324+0000 7f784e800640 1 -- 172.21.15.45:0/439417814 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78481ab890 con 0x7f7848108f40 2024-04-16T17:09:42.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.327+0000 7f7846000640 1 -- 172.21.15.45:0/439417814 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f7834010ac0 con 0x7f7848108f40 2024-04-16T17:09:42.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.327+0000 7f7852c07740 1 Processor -- start 2024-04-16T17:09:42.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.328+0000 7f7852c07740 1 -- start start 2024-04-16T17:09:42.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.328+0000 7f7852c07740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56480f570f10 0x56480f5712f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:42.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.328+0000 7f7852c07740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56480f571830 0x56480f579ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:42.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.328+0000 7f7852c07740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56480f57a410 0x56480f57c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:42.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.328+0000 7f7852c07740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x56480f4d6020 con 0x56480f57a410 2024-04-16T17:09:42.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.328+0000 7f7852c07740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x56480f47c460 con 0x56480f570f10 2024-04-16T17:09:42.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.328+0000 7f7852c07740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x56480f4d1fe0 con 0x56480f571830 2024-04-16T17:09:42.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.328+0000 7f784ca00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56480f57a410 0x56480f57c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:42.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.328+0000 7f784d400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56480f571830 0x56480f579ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:42.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.328+0000 7f784de00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56480f570f10 0x56480f5712f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:42.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.328+0000 7f784d400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56480f571830 0x56480f579ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:36950/0 (socket says 172.21.15.45:36950) 2024-04-16T17:09:42.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.328+0000 7f784d400640 1 -- 172.21.15.45:0/1564527245 learned_addr learned my addr 172.21.15.45:0/1564527245 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:09:42.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.329+0000 7f784d400640 1 -- 172.21.15.45:0/1564527245 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56480f570f10 msgr2=0x56480f5712f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:09:42.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.329+0000 7f784d400640 1 --2- 172.21.15.45:0/1564527245 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56480f570f10 0x56480f5712f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:42.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.329+0000 7f784d400640 1 -- 172.21.15.45:0/1564527245 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56480f57a410 msgr2=0x56480f57c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:09:42.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.329+0000 7f784d400640 1 --2- 172.21.15.45:0/1564527245 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56480f57a410 0x56480f57c800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:42.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.329+0000 7f784d400640 1 -- 172.21.15.45:0/1564527245 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56480f484040 con 0x56480f571830 2024-04-16T17:09:42.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.329+0000 7f7846000640 1 --2- 172.21.15.45:0/439417814 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f78300a3da0 0x7f78300a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:42.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.329+0000 7f784d400640 1 --2- 172.21.15.45:0/1564527245 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56480f571830 0x56480f579ed0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f783c009890 tx=0x7f783c005b20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=df6eb1293aaae82c server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:09:42.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.329+0000 7f7846000640 1 -- 172.21.15.45:0/439417814 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f78340d0df0 con 0x7f7848108f40 2024-04-16T17:09:42.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.329+0000 7f783a600640 1 -- 172.21.15.45:0/1564527245 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f783c039040 con 0x56480f571830 2024-04-16T17:09:42.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.330+0000 7f784de00640 1 --2- 172.21.15.45:0/439417814 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f78300a3da0 0x7f78300a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:42.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.330+0000 7f783a600640 1 -- 172.21.15.45:0/1564527245 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f783c034cb0 con 0x56480f571830 2024-04-16T17:09:42.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.330+0000 7f783a600640 1 -- 172.21.15.45:0/1564527245 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f783c02c770 con 0x56480f571830 2024-04-16T17:09:42.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.330+0000 7f7852c07740 1 -- 172.21.15.45:0/1564527245 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56480f571830 msgr2=0x56480f579ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:42.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.330+0000 7f7852c07740 1 --2- 172.21.15.45:0/1564527245 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56480f571830 0x56480f579ed0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f783c009890 tx=0x7f783c005b20 comp rx=0 tx=0).stop 2024-04-16T17:09:42.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.330+0000 7f784de00640 1 --2- 172.21.15.45:0/439417814 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f78300a3da0 0x7f78300a6260 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f7840009fd0 tx=0x7f7840010040 comp rx=0 tx=0).ready entity=mgr.25411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:09:42.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.331+0000 7f7852c07740 1 -- 172.21.15.45:0/1564527245 shutdown_connections 2024-04-16T17:09:42.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.331+0000 7f7852c07740 1 --2- 172.21.15.45:0/1564527245 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56480f571830 0x56480f579ed0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:42.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.331+0000 7f7852c07740 1 --2- 172.21.15.45:0/1564527245 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56480f57a410 0x56480f57c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:42.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.331+0000 7f7852c07740 1 --2- 172.21.15.45:0/1564527245 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56480f570f10 0x56480f5712f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:42.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.331+0000 7f7852c07740 1 -- 172.21.15.45:0/1564527245 >> 172.21.15.45:0/1564527245 conn(0x56480f56b640 msgr2=0x56480f56ba20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:09:42.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.331+0000 7f7852c07740 1 -- 172.21.15.45:0/1564527245 shutdown_connections 2024-04-16T17:09:42.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.331+0000 7f7852c07740 1 -- 172.21.15.45:0/1564527245 wait complete. 2024-04-16T17:09:42.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.331+0000 7f7852c07740 1 -- start start 2024-04-16T17:09:42.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.331+0000 7f7852c07740 1 Processor -- start 2024-04-16T17:09:42.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.332+0000 7f7852c07740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56480f570f10 0x56480f5773e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:42.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.332+0000 7f7852c07740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56480f571830 0x56480f577920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:42.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.332+0000 7f7852c07740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56480f57a410 0x56480f577e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:42.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.332+0000 7f7852c07740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x56480f57d1a0 con 0x56480f571830 2024-04-16T17:09:42.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.332+0000 7f7852c07740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x56480f48f830 con 0x56480f57a410 2024-04-16T17:09:42.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.332+0000 7f7852c07740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x56480f57d320 con 0x56480f570f10 2024-04-16T17:09:42.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.332+0000 7f7852c07740 10 client.0 _ll_register_callbacks cb 0x56480f497150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:09:42.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.332+0000 7f784ca00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56480f571830 0x56480f577920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:42.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.332+0000 7f784d400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56480f570f10 0x56480f5773e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:42.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.332+0000 7f784ca00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56480f571830 0x56480f577920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:47882/0 (socket says 172.21.15.45:47882) 2024-04-16T17:09:42.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.332+0000 7f784ca00640 1 -- 172.21.15.45:0/4247977390 learned_addr learned my addr 172.21.15.45:0/4247977390 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:09:42.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.332+0000 7f784de00640 1 --2- 172.21.15.45:0/4247977390 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56480f57a410 0x56480f577e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:42.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.332+0000 7f784ca00640 1 -- 172.21.15.45:0/4247977390 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56480f570f10 msgr2=0x56480f5773e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:09:42.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.332+0000 7f784ca00640 1 --2- 172.21.15.45:0/4247977390 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56480f570f10 0x56480f5773e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:42.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.332+0000 7f784ca00640 1 -- 172.21.15.45:0/4247977390 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56480f57a410 msgr2=0x56480f577e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:09:42.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.332+0000 7f784ca00640 1 --2- 172.21.15.45:0/4247977390 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56480f57a410 0x56480f577e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:42.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.332+0000 7f784ca00640 1 -- 172.21.15.45:0/4247977390 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56480f493920 con 0x56480f571830 2024-04-16T17:09:42.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.333+0000 7f784d400640 1 --2- 172.21.15.45:0/4247977390 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56480f570f10 0x56480f5773e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:09:42.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.333+0000 7f784ca00640 1 --2- 172.21.15.45:0/4247977390 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56480f571830 0x56480f577920 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f7834000c00 tx=0x7f7834098e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a60542f52ecb714e server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:09:42.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.333+0000 7f782ea00640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-16T17:09:42.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.333+0000 7f782ea00640 1 -- 172.21.15.45:0/4247977390 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f783409c430 con 0x56480f571830 2024-04-16T17:09:42.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.333+0000 7f782ea00640 1 -- 172.21.15.45:0/4247977390 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f78340a5200 con 0x56480f571830 2024-04-16T17:09:42.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.333+0000 7f7852c07740 1 -- 172.21.15.45:0/4247977390 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f783c034cb0 con 0x56480f571830 2024-04-16T17:09:42.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.333+0000 7f782ea00640 1 -- 172.21.15.45:0/4247977390 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f78340ce650 con 0x56480f571830 2024-04-16T17:09:42.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.333+0000 7f7852c07740 10 client.16290 fetch_fsmap learned FSMap version 30 2024-04-16T17:09:42.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.333+0000 7f7852c07740 1 -- 172.21.15.45:0/4247977390 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56480f4617f0 con 0x56480f571830 2024-04-16T17:09:42.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.334+0000 7f782ea00640 1 -- 172.21.15.45:0/4247977390 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f783c034cb0 con 0x56480f571830 2024-04-16T17:09:42.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.334+0000 7f782ea00640 1 -- 172.21.15.45:0/4247977390 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f7834018c50 con 0x56480f571830 2024-04-16T17:09:42.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.334+0000 7f7852c07740 10 client.16290 fetch_fsmap finished waiting for FSMap version 30 2024-04-16T17:09:42.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.334+0000 7f7852c07740 10 client.16290 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-16T17:09:42.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.334+0000 7f7852c07740 20 client.16290 populate_metadata read hostname 'smithi045' 2024-04-16T17:09:42.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.334+0000 7f7852c07740 1 --2- 172.21.15.45:0/4247977390 >> [v2:172.21.15.117:6838/2378675949,v1:172.21.15.117:6839/2378675949] conn(0x56480f58a680 0x56480f58cb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:42.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.334+0000 7f7852c07740 4 client.16290 mds_command: new command op to 24532 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:09:42.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.334+0000 7f7852c07740 1 -- 172.21.15.45:0/4247977390 --> [v2:172.21.15.117:6838/2378675949,v1:172.21.15.117:6839/2378675949] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56480f4e3a40 con 0x56480f58a680 2024-04-16T17:09:42.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.334+0000 7f784d400640 1 --2- 172.21.15.45:0/4247977390 >> [v2:172.21.15.117:6838/2378675949,v1:172.21.15.117:6839/2378675949] conn(0x56480f58a680 0x56480f58cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:42.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.335+0000 7f784d400640 1 --2- 172.21.15.45:0/4247977390 >> [v2:172.21.15.117:6838/2378675949,v1:172.21.15.117:6839/2378675949] conn(0x56480f58a680 0x56480f58cb00 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7f5f332888b48fc0 server_cookie=9c5c136fd0925a4e in_seq=0 out_seq=0 2024-04-16T17:09:42.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.335+0000 7f782ea00640 10 client.16290 ms_handle_connect on v2:172.21.15.117:6838/2378675949 2024-04-16T17:09:42.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.336+0000 7f782ea00640 1 -- 172.21.15.45:0/4247977390 <== mds.2 v2:172.21.15.117:6838/2378675949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56480f4e3a40 con 0x56480f58a680 2024-04-16T17:09:42.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.336+0000 7f782ea00640 10 client.16290 handle_command_reply: tid=0 2024-04-16T17:09:42.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.336+0000 7f7852c07740 1 client.16290 shutdown 2024-04-16T17:09:42.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.336+0000 7f7852c07740 1 -- 172.21.15.45:0/4247977390 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56480f571830 msgr2=0x56480f577920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:42.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.336+0000 7f7852c07740 1 --2- 172.21.15.45:0/4247977390 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56480f571830 0x56480f577920 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f7834000c00 tx=0x7f7834098e30 comp rx=0 tx=0).stop 2024-04-16T17:09:42.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.336+0000 7f7852c07740 1 -- 172.21.15.45:0/4247977390 shutdown_connections 2024-04-16T17:09:42.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.336+0000 7f7852c07740 1 --2- 172.21.15.45:0/4247977390 >> [v2:172.21.15.117:6838/2378675949,v1:172.21.15.117:6839/2378675949] conn(0x56480f58a680 0x56480f58cb00 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:42.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.336+0000 7f7852c07740 1 --2- 172.21.15.45:0/4247977390 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56480f57a410 0x56480f577e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:42.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.336+0000 7f7852c07740 1 --2- 172.21.15.45:0/4247977390 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56480f571830 0x56480f577920 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:42.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.336+0000 7f7852c07740 1 --2- 172.21.15.45:0/4247977390 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56480f570f10 0x56480f5773e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:42.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.336+0000 7f782ea00640 0 client.16290 ms_handle_reset on v2:172.21.15.117:6838/2378675949 2024-04-16T17:09:42.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.336+0000 7f7852c07740 1 -- 172.21.15.45:0/4247977390 >> 172.21.15.45:0/4247977390 conn(0x56480f56b640 msgr2=0x56480f56d390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:09:42.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.336+0000 7f7852c07740 1 -- 172.21.15.45:0/4247977390 shutdown_connections 2024-04-16T17:09:42.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.336+0000 7f7852c07740 1 -- 172.21.15.45:0/4247977390 wait complete. 2024-04-16T17:09:42.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.337+0000 7f7852c07740 20 client.16290 trim_cache size 0 max 16384 2024-04-16T17:09:42.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.352+0000 7f7852c07740 1 Processor -- start 2024-04-16T17:09:42.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.352+0000 7f7852c07740 1 -- start start 2024-04-16T17:09:42.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.352+0000 7f7852c07740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56480f570f10 0x56480f579b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:42.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.352+0000 7f7852c07740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56480f571830 0x56480f5860c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:42.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.352+0000 7f7852c07740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56480f57a410 0x56480f56a580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:42.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.352+0000 7f7852c07740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f78481b8230 con 0x56480f570f10 2024-04-16T17:09:42.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.352+0000 7f7852c07740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f78481b5a40 con 0x56480f571830 2024-04-16T17:09:42.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.352+0000 7f7852c07740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f78481b3870 con 0x56480f57a410 2024-04-16T17:09:42.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.352+0000 7f784d400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56480f570f10 0x56480f579b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:42.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.352+0000 7f784d400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56480f570f10 0x56480f579b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:47888/0 (socket says 172.21.15.45:47888) 2024-04-16T17:09:42.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.352+0000 7f784d400640 1 -- 172.21.15.45:0/293148229 learned_addr learned my addr 172.21.15.45:0/293148229 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:09:42.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.352+0000 7f784ca00640 1 --2- 172.21.15.45:0/293148229 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56480f571830 0x56480f5860c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:42.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.352+0000 7f784de00640 1 --2- 172.21.15.45:0/293148229 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56480f57a410 0x56480f56a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:42.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.353+0000 7f784d400640 1 -- 172.21.15.45:0/293148229 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56480f57a410 msgr2=0x56480f56a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:09:42.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.353+0000 7f784d400640 1 --2- 172.21.15.45:0/293148229 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56480f57a410 0x56480f56a580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:42.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.353+0000 7f784d400640 1 -- 172.21.15.45:0/293148229 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56480f571830 msgr2=0x56480f5860c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:09:42.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.353+0000 7f784d400640 1 --2- 172.21.15.45:0/293148229 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56480f571830 0x56480f5860c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:42.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.353+0000 7f784d400640 1 -- 172.21.15.45:0/293148229 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56480f1e83f0 con 0x56480f570f10 2024-04-16T17:09:42.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.353+0000 7f784ca00640 1 --2- 172.21.15.45:0/293148229 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56480f571830 0x56480f5860c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:09:42.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.353+0000 7f784d400640 1 --2- 172.21.15.45:0/293148229 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56480f570f10 0x56480f579b10 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f783c009890 tx=0x7f783c02e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64dbaa560faaee8f server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:09:42.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.353+0000 7f7839c00640 1 -- 172.21.15.45:0/293148229 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f783c055070 con 0x56480f570f10 2024-04-16T17:09:42.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.353+0000 7f7839c00640 1 -- 172.21.15.45:0/293148229 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f783c034c70 con 0x56480f570f10 2024-04-16T17:09:42.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.353+0000 7f7839c00640 1 -- 172.21.15.45:0/293148229 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f783c039040 con 0x56480f570f10 2024-04-16T17:09:42.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.354+0000 7f7852c07740 1 -- 172.21.15.45:0/293148229 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56480f570f10 msgr2=0x56480f579b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:42.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.354+0000 7f7852c07740 1 --2- 172.21.15.45:0/293148229 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56480f570f10 0x56480f579b10 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f783c009890 tx=0x7f783c02e040 comp rx=0 tx=0).stop 2024-04-16T17:09:42.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.354+0000 7f7852c07740 1 -- 172.21.15.45:0/293148229 shutdown_connections 2024-04-16T17:09:42.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.354+0000 7f7852c07740 1 --2- 172.21.15.45:0/293148229 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56480f57a410 0x56480f56a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:42.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.354+0000 7f7852c07740 1 --2- 172.21.15.45:0/293148229 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56480f571830 0x56480f5860c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:42.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.354+0000 7f7852c07740 1 --2- 172.21.15.45:0/293148229 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56480f570f10 0x56480f579b10 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:42.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.354+0000 7f7852c07740 1 -- 172.21.15.45:0/293148229 >> 172.21.15.45:0/293148229 conn(0x56480f56b640 msgr2=0x56480f569e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:09:42.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.354+0000 7f7852c07740 1 -- 172.21.15.45:0/293148229 shutdown_connections 2024-04-16T17:09:42.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.354+0000 7f7852c07740 1 -- 172.21.15.45:0/293148229 wait complete. 2024-04-16T17:09:42.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.355+0000 7f7852c07740 1 -- start start 2024-04-16T17:09:42.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.355+0000 7f7852c07740 1 Processor -- start 2024-04-16T17:09:42.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.355+0000 7f7852c07740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56480f570f10 0x56480f582ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:42.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.355+0000 7f7852c07740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56480f571830 0x56480f57de80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:42.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.356+0000 7f7852c07740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56480f57a410 0x56480f57e490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:42.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.356+0000 7f7852c07740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f78481b3f00 con 0x56480f570f10 2024-04-16T17:09:42.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.356+0000 7f7852c07740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f78481b5370 con 0x56480f571830 2024-04-16T17:09:42.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.356+0000 7f7852c07740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f78481b45d0 con 0x56480f57a410 2024-04-16T17:09:42.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.356+0000 7f7852c07740 10 client.0 _ll_register_callbacks cb 0x7f783400e940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:09:42.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.356+0000 7f784d400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56480f570f10 0x56480f582ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:42.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.356+0000 7f784d400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56480f570f10 0x56480f582ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:47898/0 (socket says 172.21.15.45:47898) 2024-04-16T17:09:42.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.356+0000 7f784d400640 1 -- 172.21.15.45:0/3105205252 learned_addr learned my addr 172.21.15.45:0/3105205252 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:09:42.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.356+0000 7f784ca00640 1 --2- 172.21.15.45:0/3105205252 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56480f571830 0x56480f57de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:42.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.356+0000 7f784de00640 1 --2- 172.21.15.45:0/3105205252 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56480f57a410 0x56480f57e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:42.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.356+0000 7f784d400640 1 -- 172.21.15.45:0/3105205252 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56480f57a410 msgr2=0x56480f57e490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:09:42.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.356+0000 7f784d400640 1 --2- 172.21.15.45:0/3105205252 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56480f57a410 0x56480f57e490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:42.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.356+0000 7f784d400640 1 -- 172.21.15.45:0/3105205252 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56480f571830 msgr2=0x56480f57de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:09:42.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.356+0000 7f784d400640 1 --2- 172.21.15.45:0/3105205252 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56480f571830 0x56480f57de80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:42.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.356+0000 7f784d400640 1 -- 172.21.15.45:0/3105205252 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56480f2b27d0 con 0x56480f570f10 2024-04-16T17:09:42.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.356+0000 7f784ca00640 1 --2- 172.21.15.45:0/3105205252 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56480f571830 0x56480f57de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:09:42.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.357+0000 7f784d400640 1 --2- 172.21.15.45:0/3105205252 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56480f570f10 0x56480f582ac0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f783c009550 tx=0x7f783c032340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cd4e6144748653f server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:09:42.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.357+0000 7f782ea00640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-16T17:09:42.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.357+0000 7f782ea00640 1 -- 172.21.15.45:0/3105205252 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f783c055070 con 0x56480f570f10 2024-04-16T17:09:42.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.357+0000 7f7852c07740 1 -- 172.21.15.45:0/3105205252 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f783c034c70 con 0x56480f570f10 2024-04-16T17:09:42.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.357+0000 7f782ea00640 1 -- 172.21.15.45:0/3105205252 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f783c034750 con 0x56480f570f10 2024-04-16T17:09:42.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.357+0000 7f782ea00640 1 -- 172.21.15.45:0/3105205252 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f783c039040 con 0x56480f570f10 2024-04-16T17:09:42.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.357+0000 7f7852c07740 10 client.16296 fetch_fsmap learned FSMap version 30 2024-04-16T17:09:42.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.357+0000 7f7852c07740 1 -- 172.21.15.45:0/3105205252 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56480f5f0bb0 con 0x56480f570f10 2024-04-16T17:09:42.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.357+0000 7f782ea00640 1 -- 172.21.15.45:0/3105205252 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f783c034c70 con 0x56480f570f10 2024-04-16T17:09:42.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.358+0000 7f782ea00640 1 -- 172.21.15.45:0/3105205252 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f783c02c8d0 con 0x56480f570f10 2024-04-16T17:09:42.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.358+0000 7f7852c07740 10 client.16296 fetch_fsmap finished waiting for FSMap version 30 2024-04-16T17:09:42.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.358+0000 7f7852c07740 10 client.16296 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-16T17:09:42.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.358+0000 7f7852c07740 20 client.16296 populate_metadata read hostname 'smithi045' 2024-04-16T17:09:42.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.358+0000 7f7852c07740 1 --2- 172.21.15.45:0/3105205252 >> [v2:172.21.15.117:6838/2378675949,v1:172.21.15.117:6839/2378675949] conn(0x56480f5f0e30 0x56480f5f3220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:42.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.358+0000 7f7852c07740 4 client.16296 mds_command: new command op to 24532 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T17:09:42.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.358+0000 7f7852c07740 1 -- 172.21.15.45:0/3105205252 --> [v2:172.21.15.117:6838/2378675949,v1:172.21.15.117:6839/2378675949] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56480f4dd8b0 con 0x56480f5f0e30 2024-04-16T17:09:42.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.358+0000 7f784ca00640 1 --2- 172.21.15.45:0/3105205252 >> [v2:172.21.15.117:6838/2378675949,v1:172.21.15.117:6839/2378675949] conn(0x56480f5f0e30 0x56480f5f3220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:42.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.359+0000 7f784ca00640 1 --2- 172.21.15.45:0/3105205252 >> [v2:172.21.15.117:6838/2378675949,v1:172.21.15.117:6839/2378675949] conn(0x56480f5f0e30 0x56480f5f3220 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a154672d53348cae server_cookie=94d404430379e892 in_seq=0 out_seq=0 2024-04-16T17:09:42.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.359+0000 7f782ea00640 10 client.16296 ms_handle_connect on v2:172.21.15.117:6838/2378675949 2024-04-16T17:09:42.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.359+0000 7f782ea00640 1 -- 172.21.15.45:0/3105205252 <== mds.2 v2:172.21.15.117:6838/2378675949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56480f4dd8b0 con 0x56480f5f0e30 2024-04-16T17:09:42.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.359+0000 7f782ea00640 10 client.16296 handle_command_reply: tid=0 2024-04-16T17:09:42.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.359+0000 7f7852c07740 1 client.16296 shutdown 2024-04-16T17:09:42.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.359+0000 7f7852c07740 1 -- 172.21.15.45:0/3105205252 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56480f570f10 msgr2=0x56480f582ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:42.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.359+0000 7f7852c07740 1 --2- 172.21.15.45:0/3105205252 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56480f570f10 0x56480f582ac0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f783c009550 tx=0x7f783c032340 comp rx=0 tx=0).stop 2024-04-16T17:09:42.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.359+0000 7f7852c07740 1 -- 172.21.15.45:0/3105205252 shutdown_connections 2024-04-16T17:09:42.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.359+0000 7f7852c07740 1 --2- 172.21.15.45:0/3105205252 >> [v2:172.21.15.117:6838/2378675949,v1:172.21.15.117:6839/2378675949] conn(0x56480f5f0e30 0x56480f5f3220 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:42.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.359+0000 7f7852c07740 1 --2- 172.21.15.45:0/3105205252 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56480f57a410 0x56480f57e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:42.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.359+0000 7f7852c07740 1 --2- 172.21.15.45:0/3105205252 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56480f571830 0x56480f57de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:42.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.359+0000 7f7852c07740 1 --2- 172.21.15.45:0/3105205252 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56480f570f10 0x56480f582ac0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:42.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.359+0000 7f782ea00640 0 client.16296 ms_handle_reset on v2:172.21.15.117:6838/2378675949 2024-04-16T17:09:42.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.359+0000 7f7852c07740 1 -- 172.21.15.45:0/3105205252 >> 172.21.15.45:0/3105205252 conn(0x56480f56b640 msgr2=0x56480f577c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:09:42.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.359+0000 7f7852c07740 1 -- 172.21.15.45:0/3105205252 shutdown_connections 2024-04-16T17:09:42.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.359+0000 7f7852c07740 1 -- 172.21.15.45:0/3105205252 wait complete. 2024-04-16T17:09:42.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.360+0000 7f7852c07740 20 client.16296 trim_cache size 0 max 16384 2024-04-16T17:09:42.364 INFO:teuthology.orchestra.run.smithi045.stdout:[] 2024-04-16T17:09:42.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.360+0000 7f784e800640 1 -- 172.21.15.45:0/439417814 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f78300a3da0 msgr2=0x7f78300a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:42.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.360+0000 7f784e800640 1 --2- 172.21.15.45:0/439417814 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f78300a3da0 0x7f78300a6260 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f7840009fd0 tx=0x7f7840010040 comp rx=0 tx=0).stop 2024-04-16T17:09:42.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.360+0000 7f784e800640 1 -- 172.21.15.45:0/439417814 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7848108f40 msgr2=0x7f784819f2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:42.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.360+0000 7f784e800640 1 --2- 172.21.15.45:0/439417814 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7848108f40 0x7f784819f2b0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f783400e9f0 tx=0x7f783400eeb0 comp rx=0 tx=0).stop 2024-04-16T17:09:42.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.360+0000 7f784e800640 1 -- 172.21.15.45:0/439417814 shutdown_connections 2024-04-16T17:09:42.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.360+0000 7f784e800640 1 --2- 172.21.15.45:0/439417814 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f78300a3da0 0x7f78300a6260 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:42.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.360+0000 7f784e800640 1 --2- 172.21.15.45:0/439417814 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f784810d500 0x7f78481a35e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:42.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.360+0000 7f784e800640 1 --2- 172.21.15.45:0/439417814 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7848108f40 0x7f784819f2b0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:42.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.360+0000 7f784e800640 1 --2- 172.21.15.45:0/439417814 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7848108580 0x7f784819ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:42.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.360+0000 7f784e800640 1 -- 172.21.15.45:0/439417814 >> 172.21.15.45:0/439417814 conn(0x7f7848077e20 msgr2=0x7f78480783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:09:42.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.360+0000 7f784e800640 1 -- 172.21.15.45:0/439417814 shutdown_connections 2024-04-16T17:09:42.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.360+0000 7f784e800640 1 -- 172.21.15.45:0/439417814 wait complete. 2024-04-16T17:09:42.375 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:09:42 smithi045 ceph-mon[31676]: pgmap v91: 129 pgs: 129 active+clean; 31 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 78 KiB/s wr, 27 op/s 2024-04-16T17:09:42.375 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:09:42 smithi045 ceph-mon[31676]: from='client.? 172.21.15.45:0/582844847' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:09:42.377 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-16T17:09:42.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.573+0000 7fc144e00640 1 -- 172.21.15.45:0/1350138261 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc140103b80 msgr2=0x7fc14010c060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:42.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.573+0000 7fc144e00640 1 --2- 172.21.15.45:0/1350138261 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc140103b80 0x7fc14010c060 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fc128009860 tx=0x7fc12802bc00 comp rx=0 tx=0).stop 2024-04-16T17:09:42.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.573+0000 7fc144e00640 1 -- 172.21.15.45:0/1350138261 shutdown_connections 2024-04-16T17:09:42.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.573+0000 7fc144e00640 1 --2- 172.21.15.45:0/1350138261 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc140103b80 0x7fc14010c060 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:42.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.573+0000 7fc144e00640 1 --2- 172.21.15.45:0/1350138261 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc14010c630 0x7fc14010ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:42.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.573+0000 7fc144e00640 1 --2- 172.21.15.45:0/1350138261 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc140103260 0x7fc140103640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:42.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.573+0000 7fc144e00640 1 -- 172.21.15.45:0/1350138261 >> 172.21.15.45:0/1350138261 conn(0x7fc140077280 msgr2=0x7fc140077690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:09:42.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.573+0000 7fc144e00640 1 -- 172.21.15.45:0/1350138261 shutdown_connections 2024-04-16T17:09:42.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.573+0000 7fc144e00640 1 -- 172.21.15.45:0/1350138261 wait complete. 2024-04-16T17:09:42.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.574+0000 7fc144e00640 1 Processor -- start 2024-04-16T17:09:42.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.575+0000 7fc144e00640 1 -- start start 2024-04-16T17:09:42.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.575+0000 7fc144e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc140103260 0x7fc140100eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:42.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.575+0000 7fc144e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc140103b80 0x7fc1401013f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:42.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.575+0000 7fc144e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc14010c630 0x7fc140101930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:42.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.575+0000 7fc144e00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fc1401123c0 con 0x7fc140103260 2024-04-16T17:09:42.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.575+0000 7fc144e00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fc140112240 con 0x7fc14010c630 2024-04-16T17:09:42.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.575+0000 7fc144e00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fc140112540 con 0x7fc140103b80 2024-04-16T17:09:42.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.575+0000 7fc13de00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc140103260 0x7fc140100eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:42.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.575+0000 7fc13de00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc140103260 0x7fc140100eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:47918/0 (socket says 172.21.15.45:47918) 2024-04-16T17:09:42.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.575+0000 7fc13de00640 1 -- 172.21.15.45:0/2763445859 learned_addr learned my addr 172.21.15.45:0/2763445859 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:09:42.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.575+0000 7fc13d400640 1 --2- 172.21.15.45:0/2763445859 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc140103b80 0x7fc1401013f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:42.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.575+0000 7fc13e800640 1 --2- 172.21.15.45:0/2763445859 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc14010c630 0x7fc140101930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:42.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.576+0000 7fc13de00640 1 -- 172.21.15.45:0/2763445859 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc140103b80 msgr2=0x7fc1401013f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:42.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.576+0000 7fc13de00640 1 --2- 172.21.15.45:0/2763445859 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc140103b80 0x7fc1401013f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:42.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.576+0000 7fc13de00640 1 -- 172.21.15.45:0/2763445859 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc14010c630 msgr2=0x7fc140101930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:42.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.576+0000 7fc13de00640 1 --2- 172.21.15.45:0/2763445859 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc14010c630 0x7fc140101930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:42.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.576+0000 7fc13de00640 1 -- 172.21.15.45:0/2763445859 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1401a6de0 con 0x7fc140103260 2024-04-16T17:09:42.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.576+0000 7fc13d400640 1 --2- 172.21.15.45:0/2763445859 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc140103b80 0x7fc1401013f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:09:42.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.576+0000 7fc13e800640 1 --2- 172.21.15.45:0/2763445859 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc14010c630 0x7fc140101930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:09:42.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.576+0000 7fc13de00640 1 --2- 172.21.15.45:0/2763445859 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc140103260 0x7fc140100eb0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fc13000e9c0 tx=0x7fc13000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:09:42.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.576+0000 7fc136a00640 1 -- 172.21.15.45:0/2763445859 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc13000cde0 con 0x7fc140103260 2024-04-16T17:09:42.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.576+0000 7fc136a00640 1 -- 172.21.15.45:0/2763445859 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc1300035f0 con 0x7fc140103260 2024-04-16T17:09:42.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.576+0000 7fc144e00640 1 -- 172.21.15.45:0/2763445859 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1401a70d0 con 0x7fc140103260 2024-04-16T17:09:42.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.576+0000 7fc136a00640 1 -- 172.21.15.45:0/2763445859 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc130010690 con 0x7fc140103260 2024-04-16T17:09:42.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.577+0000 7fc144e00640 1 -- 172.21.15.45:0/2763445859 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1401a7640 con 0x7fc140103260 2024-04-16T17:09:42.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.581+0000 7fc147be8740 1 Processor -- start 2024-04-16T17:09:42.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.581+0000 7fc147be8740 1 -- start start 2024-04-16T17:09:42.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.582+0000 7fc136a00640 1 -- 172.21.15.45:0/2763445859 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fc1300037a0 con 0x7fc140103260 2024-04-16T17:09:42.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.582+0000 7fc147be8740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555c73850d50 0x555c73851130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:42.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.582+0000 7fc147be8740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555c73851670 0x555c73859d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:42.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.582+0000 7fc147be8740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555c7385a250 0x555c7385c640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:42.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.582+0000 7fc147be8740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x555c737b6020 con 0x555c73850d50 2024-04-16T17:09:42.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.582+0000 7fc147be8740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x555c7375c460 con 0x555c7385a250 2024-04-16T17:09:42.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.582+0000 7fc147be8740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x555c737b1fe0 con 0x555c73851670 2024-04-16T17:09:42.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.582+0000 7fc13e800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555c73850d50 0x555c73851130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:42.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.582+0000 7fc13d400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555c7385a250 0x555c7385c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:42.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.582+0000 7fc13d400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555c7385a250 0x555c7385c640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:47062/0 (socket says 172.21.15.45:47062) 2024-04-16T17:09:42.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.582+0000 7fc13d400640 1 -- 172.21.15.45:0/1745388671 learned_addr learned my addr 172.21.15.45:0/1745388671 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:09:42.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.582+0000 7fc13de00640 1 --2- 172.21.15.45:0/1745388671 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555c73851670 0x555c73859d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:42.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.582+0000 7fc13d400640 1 -- 172.21.15.45:0/1745388671 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555c73851670 msgr2=0x555c73859d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:09:42.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.583+0000 7fc13d400640 1 --2- 172.21.15.45:0/1745388671 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555c73851670 0x555c73859d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:42.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.583+0000 7fc13d400640 1 -- 172.21.15.45:0/1745388671 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555c73850d50 msgr2=0x555c73851130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:09:42.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.583+0000 7fc13d400640 1 --2- 172.21.15.45:0/1745388671 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555c73850d50 0x555c73851130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:42.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.583+0000 7fc13d400640 1 -- 172.21.15.45:0/1745388671 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555c73764040 con 0x555c7385a250 2024-04-16T17:09:42.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.583+0000 7fc13de00640 1 --2- 172.21.15.45:0/1745388671 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555c73851670 0x555c73859d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:09:42.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.583+0000 7fc13e800640 1 --2- 172.21.15.45:0/1745388671 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555c73850d50 0x555c73851130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:09:42.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.583+0000 7fc13d400640 1 --2- 172.21.15.45:0/1745388671 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555c7385a250 0x555c7385c640 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fc128009860 tx=0x7fc1280056d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=24e16992022ee355 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:09:42.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.583+0000 7fc12f400640 1 -- 172.21.15.45:0/1745388671 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc128044020 con 0x555c7385a250 2024-04-16T17:09:42.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.583+0000 7fc12f400640 1 -- 172.21.15.45:0/1745388671 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc128034920 con 0x555c7385a250 2024-04-16T17:09:42.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.583+0000 7fc12f400640 1 -- 172.21.15.45:0/1745388671 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc128039040 con 0x555c7385a250 2024-04-16T17:09:42.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.583+0000 7fc136a00640 1 --2- 172.21.15.45:0/2763445859 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fc1240a3da0 0x7fc1240a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:42.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.584+0000 7fc136a00640 1 -- 172.21.15.45:0/2763445859 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fc1300d0dc0 con 0x7fc140103260 2024-04-16T17:09:42.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.584+0000 7fc13e800640 1 --2- 172.21.15.45:0/2763445859 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fc1240a3da0 0x7fc1240a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:42.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.584+0000 7fc147be8740 1 -- 172.21.15.45:0/1745388671 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555c7385a250 msgr2=0x555c7385c640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:42.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.584+0000 7fc147be8740 1 --2- 172.21.15.45:0/1745388671 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555c7385a250 0x555c7385c640 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fc128009860 tx=0x7fc1280056d0 comp rx=0 tx=0).stop 2024-04-16T17:09:42.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.584+0000 7fc147be8740 1 -- 172.21.15.45:0/1745388671 shutdown_connections 2024-04-16T17:09:42.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.584+0000 7fc147be8740 1 --2- 172.21.15.45:0/1745388671 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555c73851670 0x555c73859d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:42.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.584+0000 7fc147be8740 1 --2- 172.21.15.45:0/1745388671 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555c7385a250 0x555c7385c640 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:42.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.584+0000 7fc147be8740 1 --2- 172.21.15.45:0/1745388671 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555c73850d50 0x555c73851130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:42.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.584+0000 7fc147be8740 1 -- 172.21.15.45:0/1745388671 >> 172.21.15.45:0/1745388671 conn(0x555c7384b480 msgr2=0x555c7384b860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:09:42.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.585+0000 7fc147be8740 1 -- 172.21.15.45:0/1745388671 shutdown_connections 2024-04-16T17:09:42.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.585+0000 7fc13e800640 1 --2- 172.21.15.45:0/2763445859 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fc1240a3da0 0x7fc1240a6260 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fc138009fd0 tx=0x7fc138009500 comp rx=0 tx=0).ready entity=mgr.25411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:09:42.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.585+0000 7fc147be8740 1 -- 172.21.15.45:0/1745388671 wait complete. 2024-04-16T17:09:42.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.585+0000 7fc147be8740 1 -- start start 2024-04-16T17:09:42.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.585+0000 7fc147be8740 1 Processor -- start 2024-04-16T17:09:42.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.586+0000 7fc147be8740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555c73850d50 0x555c73857270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:42.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.586+0000 7fc147be8740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555c73851670 0x555c738577b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:42.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.586+0000 7fc147be8740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555c7385a250 0x555c73857cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:42.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.586+0000 7fc147be8740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x555c7385cfe0 con 0x555c73851670 2024-04-16T17:09:42.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.586+0000 7fc147be8740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x555c7376f830 con 0x555c7385a250 2024-04-16T17:09:42.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.586+0000 7fc147be8740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x555c7385d160 con 0x555c73850d50 2024-04-16T17:09:42.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.586+0000 7fc147be8740 10 client.0 _ll_register_callbacks cb 0x555c73777150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:09:42.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.586+0000 7fc13d400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555c73851670 0x555c738577b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:42.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.586+0000 7fc13d400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555c73851670 0x555c738577b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:47930/0 (socket says 172.21.15.45:47930) 2024-04-16T17:09:42.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.586+0000 7fc13d400640 1 -- 172.21.15.45:0/1145024171 learned_addr learned my addr 172.21.15.45:0/1145024171 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:09:42.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.586+0000 7fc13de00640 1 --2- 172.21.15.45:0/1145024171 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555c73850d50 0x555c73857270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:42.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.586+0000 7fc13e800640 1 --2- 172.21.15.45:0/1145024171 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555c7385a250 0x555c73857cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:42.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.587+0000 7fc13d400640 1 -- 172.21.15.45:0/1145024171 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555c73850d50 msgr2=0x555c73857270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:09:42.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.587+0000 7fc13d400640 1 --2- 172.21.15.45:0/1145024171 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555c73850d50 0x555c73857270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:42.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.587+0000 7fc13d400640 1 -- 172.21.15.45:0/1145024171 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555c7385a250 msgr2=0x555c73857cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:09:42.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.587+0000 7fc13d400640 1 --2- 172.21.15.45:0/1145024171 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555c7385a250 0x555c73857cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:42.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.587+0000 7fc13d400640 1 -- 172.21.15.45:0/1145024171 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555c73773920 con 0x555c73851670 2024-04-16T17:09:42.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.587+0000 7fc13e800640 1 --2- 172.21.15.45:0/1145024171 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555c7385a250 0x555c73857cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:09:42.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.587+0000 7fc13d400640 1 --2- 172.21.15.45:0/1145024171 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555c73851670 0x555c738577b0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fc12802e010 tx=0x7fc12802c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efe55be9b658d634 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:09:42.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.587+0000 7fc123e00640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-16T17:09:42.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.587+0000 7fc123e00640 1 -- 172.21.15.45:0/1145024171 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc128050070 con 0x555c73851670 2024-04-16T17:09:42.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.587+0000 7fc123e00640 1 -- 172.21.15.45:0/1145024171 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fc128004100 con 0x555c73851670 2024-04-16T17:09:42.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.587+0000 7fc147be8740 1 -- 172.21.15.45:0/1145024171 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc128034920 con 0x555c73851670 2024-04-16T17:09:42.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.587+0000 7fc123e00640 1 -- 172.21.15.45:0/1145024171 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fc12805c050 con 0x555c73851670 2024-04-16T17:09:42.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.588+0000 7fc147be8740 10 client.16308 fetch_fsmap learned FSMap version 30 2024-04-16T17:09:42.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.588+0000 7fc147be8740 1 -- 172.21.15.45:0/1145024171 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x555c737417f0 con 0x555c73851670 2024-04-16T17:09:42.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.588+0000 7fc123e00640 1 -- 172.21.15.45:0/1145024171 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc128034920 con 0x555c73851670 2024-04-16T17:09:42.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.588+0000 7fc123e00640 1 -- 172.21.15.45:0/1145024171 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fc128049450 con 0x555c73851670 2024-04-16T17:09:42.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.588+0000 7fc147be8740 10 client.16308 fetch_fsmap finished waiting for FSMap version 30 2024-04-16T17:09:42.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.588+0000 7fc147be8740 10 client.16308 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-04-16T17:09:42.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.588+0000 7fc147be8740 20 client.16308 populate_metadata read hostname 'smithi045' 2024-04-16T17:09:42.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.588+0000 7fc147be8740 1 --2- 172.21.15.45:0/1145024171 >> [v2:172.21.15.45:6840/3755310822,v1:172.21.15.45:6841/3755310822] conn(0x555c7386a410 0x555c7386c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:42.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.588+0000 7fc147be8740 4 client.16308 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:09:42.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.588+0000 7fc147be8740 1 -- 172.21.15.45:0/1145024171 --> [v2:172.21.15.45:6840/3755310822,v1:172.21.15.45:6841/3755310822] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555c737c3a40 con 0x555c7386a410 2024-04-16T17:09:42.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.589+0000 7fc13de00640 1 --2- 172.21.15.45:0/1145024171 >> [v2:172.21.15.45:6840/3755310822,v1:172.21.15.45:6841/3755310822] conn(0x555c7386a410 0x555c7386c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:42.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.589+0000 7fc13de00640 1 --2- 172.21.15.45:0/1145024171 >> [v2:172.21.15.45:6840/3755310822,v1:172.21.15.45:6841/3755310822] conn(0x555c7386a410 0x555c7386c800 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8a556feb34f9f45e server_cookie=51d2d7d68b668208 in_seq=0 out_seq=0 2024-04-16T17:09:42.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.589+0000 7fc123e00640 10 client.16308 ms_handle_connect on v2:172.21.15.45:6840/3755310822 2024-04-16T17:09:42.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.590+0000 7fc123e00640 1 -- 172.21.15.45:0/1145024171 <== mds.1 v2:172.21.15.45:6840/3755310822 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555c737c3a40 con 0x555c7386a410 2024-04-16T17:09:42.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.590+0000 7fc123e00640 10 client.16308 handle_command_reply: tid=0 2024-04-16T17:09:42.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.590+0000 7fc147be8740 1 client.16308 shutdown 2024-04-16T17:09:42.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.590+0000 7fc147be8740 1 -- 172.21.15.45:0/1145024171 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555c73851670 msgr2=0x555c738577b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:42.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.590+0000 7fc147be8740 1 --2- 172.21.15.45:0/1145024171 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555c73851670 0x555c738577b0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fc12802e010 tx=0x7fc12802c920 comp rx=0 tx=0).stop 2024-04-16T17:09:42.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.590+0000 7fc147be8740 1 -- 172.21.15.45:0/1145024171 shutdown_connections 2024-04-16T17:09:42.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.590+0000 7fc147be8740 1 --2- 172.21.15.45:0/1145024171 >> [v2:172.21.15.45:6840/3755310822,v1:172.21.15.45:6841/3755310822] conn(0x555c7386a410 0x555c7386c800 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:42.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.590+0000 7fc147be8740 1 --2- 172.21.15.45:0/1145024171 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555c7385a250 0x555c73857cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:42.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.590+0000 7fc147be8740 1 --2- 172.21.15.45:0/1145024171 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555c73851670 0x555c738577b0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:42.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.590+0000 7fc147be8740 1 --2- 172.21.15.45:0/1145024171 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555c73850d50 0x555c73857270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:42.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.590+0000 7fc123e00640 0 client.16308 ms_handle_reset on v2:172.21.15.45:6840/3755310822 2024-04-16T17:09:42.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.590+0000 7fc147be8740 1 -- 172.21.15.45:0/1145024171 >> 172.21.15.45:0/1145024171 conn(0x555c7384b480 msgr2=0x555c7384d1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:09:42.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.590+0000 7fc147be8740 1 -- 172.21.15.45:0/1145024171 shutdown_connections 2024-04-16T17:09:42.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.590+0000 7fc147be8740 1 -- 172.21.15.45:0/1145024171 wait complete. 2024-04-16T17:09:42.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.591+0000 7fc147be8740 20 client.16308 trim_cache size 0 max 16384 2024-04-16T17:09:42.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.606+0000 7fc147be8740 1 Processor -- start 2024-04-16T17:09:42.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.606+0000 7fc147be8740 1 -- start start 2024-04-16T17:09:42.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.606+0000 7fc147be8740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555c73850d50 0x555c738599a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:42.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.606+0000 7fc147be8740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555c73851670 0x555c73865ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:42.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.606+0000 7fc147be8740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555c7385a250 0x555c73871e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:42.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.606+0000 7fc147be8740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fc1401b4040 con 0x555c7385a250 2024-04-16T17:09:42.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.606+0000 7fc147be8740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fc1401b1850 con 0x555c73850d50 2024-04-16T17:09:42.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.606+0000 7fc147be8740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fc1401af680 con 0x555c73851670 2024-04-16T17:09:42.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.606+0000 7fc13e800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555c7385a250 0x555c73871e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:42.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.606+0000 7fc13e800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555c7385a250 0x555c73871e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:47946/0 (socket says 172.21.15.45:47946) 2024-04-16T17:09:42.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.606+0000 7fc13e800640 1 -- 172.21.15.45:0/297094018 learned_addr learned my addr 172.21.15.45:0/297094018 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:09:42.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.606+0000 7fc13d400640 1 --2- 172.21.15.45:0/297094018 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555c73851670 0x555c73865ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:42.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.607+0000 7fc13de00640 1 --2- 172.21.15.45:0/297094018 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555c73850d50 0x555c738599a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:42.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.607+0000 7fc13d400640 1 -- 172.21.15.45:0/297094018 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555c73850d50 msgr2=0x555c738599a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:09:42.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.607+0000 7fc13d400640 1 --2- 172.21.15.45:0/297094018 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555c73850d50 0x555c738599a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:42.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.607+0000 7fc13d400640 1 -- 172.21.15.45:0/297094018 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555c7385a250 msgr2=0x555c73871e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:09:42.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.607+0000 7fc13d400640 1 --2- 172.21.15.45:0/297094018 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555c7385a250 0x555c73871e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:42.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.607+0000 7fc13d400640 1 -- 172.21.15.45:0/297094018 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555c734c83f0 con 0x555c73851670 2024-04-16T17:09:42.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.607+0000 7fc13d400640 1 --2- 172.21.15.45:0/297094018 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555c73851670 0x555c73865ed0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fc12802c870 tx=0x7fc128000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7f72faf4ada9b4cc server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:09:42.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.607+0000 7fc12ea00640 1 -- 172.21.15.45:0/297094018 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc128049e60 con 0x555c73851670 2024-04-16T17:09:42.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.608+0000 7fc12ea00640 1 -- 172.21.15.45:0/297094018 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc128005520 con 0x555c73851670 2024-04-16T17:09:42.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.608+0000 7fc12ea00640 1 -- 172.21.15.45:0/297094018 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc128050070 con 0x555c73851670 2024-04-16T17:09:42.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.608+0000 7fc147be8740 1 -- 172.21.15.45:0/297094018 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555c73851670 msgr2=0x555c73865ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:42.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.608+0000 7fc147be8740 1 --2- 172.21.15.45:0/297094018 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555c73851670 0x555c73865ed0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fc12802c870 tx=0x7fc128000bc0 comp rx=0 tx=0).stop 2024-04-16T17:09:42.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.608+0000 7fc147be8740 1 -- 172.21.15.45:0/297094018 shutdown_connections 2024-04-16T17:09:42.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.608+0000 7fc147be8740 1 --2- 172.21.15.45:0/297094018 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555c73851670 0x555c73865ed0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:42.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.608+0000 7fc147be8740 1 --2- 172.21.15.45:0/297094018 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555c7385a250 0x555c73871e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:42.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.608+0000 7fc147be8740 1 --2- 172.21.15.45:0/297094018 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555c73850d50 0x555c738599a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:42.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.608+0000 7fc147be8740 1 -- 172.21.15.45:0/297094018 >> 172.21.15.45:0/297094018 conn(0x555c7384b480 msgr2=0x555c738524d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:09:42.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.608+0000 7fc147be8740 1 -- 172.21.15.45:0/297094018 shutdown_connections 2024-04-16T17:09:42.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.608+0000 7fc147be8740 1 -- 172.21.15.45:0/297094018 wait complete. 2024-04-16T17:09:42.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.609+0000 7fc147be8740 1 -- start start 2024-04-16T17:09:42.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.609+0000 7fc147be8740 1 Processor -- start 2024-04-16T17:09:42.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.609+0000 7fc147be8740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555c73850d50 0x555c73862850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:42.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.609+0000 7fc147be8740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555c73851670 0x555c7385dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:42.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.609+0000 7fc147be8740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555c7385a250 0x555c7385e160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:42.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.609+0000 7fc147be8740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fc1401afd10 con 0x555c73851670 2024-04-16T17:09:42.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.609+0000 7fc147be8740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fc1401b1180 con 0x555c73850d50 2024-04-16T17:09:42.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.609+0000 7fc147be8740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fc1401b03e0 con 0x555c7385a250 2024-04-16T17:09:42.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.609+0000 7fc147be8740 10 client.0 _ll_register_callbacks cb 0x7fc138009ee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:09:42.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.610+0000 7fc13de00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555c73850d50 0x555c73862850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:42.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.610+0000 7fc13d400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555c73851670 0x555c7385dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:42.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.610+0000 7fc13de00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555c73850d50 0x555c73862850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:47094/0 (socket says 172.21.15.45:47094) 2024-04-16T17:09:42.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.610+0000 7fc13de00640 1 -- 172.21.15.45:0/795079662 learned_addr learned my addr 172.21.15.45:0/795079662 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:09:42.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.610+0000 7fc13e800640 1 --2- 172.21.15.45:0/795079662 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555c7385a250 0x555c7385e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:42.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.610+0000 7fc13d400640 1 -- 172.21.15.45:0/795079662 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555c7385a250 msgr2=0x555c7385e160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:09:42.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.610+0000 7fc13d400640 1 --2- 172.21.15.45:0/795079662 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555c7385a250 0x555c7385e160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:42.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.610+0000 7fc13d400640 1 -- 172.21.15.45:0/795079662 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555c73850d50 msgr2=0x555c73862850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:09:42.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.610+0000 7fc13d400640 1 --2- 172.21.15.45:0/795079662 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555c73850d50 0x555c73862850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:42.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.610+0000 7fc13d400640 1 -- 172.21.15.45:0/795079662 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555c735927d0 con 0x555c73851670 2024-04-16T17:09:42.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.610+0000 7fc13de00640 1 --2- 172.21.15.45:0/795079662 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555c73850d50 0x555c73862850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:09:42.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.610+0000 7fc13d400640 1 --2- 172.21.15.45:0/795079662 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555c73851670 0x555c7385dbc0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fc12802c870 tx=0x7fc128005dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39d28356700ff91d server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:09:42.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.610+0000 7fc123e00640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-16T17:09:42.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.611+0000 7fc123e00640 1 -- 172.21.15.45:0/795079662 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc128066070 con 0x555c73851670 2024-04-16T17:09:42.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.611+0000 7fc123e00640 1 -- 172.21.15.45:0/795079662 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fc1280348c0 con 0x555c73851670 2024-04-16T17:09:42.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.611+0000 7fc147be8740 1 -- 172.21.15.45:0/795079662 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc128005520 con 0x555c73851670 2024-04-16T17:09:42.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.611+0000 7fc123e00640 1 -- 172.21.15.45:0/795079662 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fc128072050 con 0x555c73851670 2024-04-16T17:09:42.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.611+0000 7fc147be8740 10 client.16314 fetch_fsmap learned FSMap version 30 2024-04-16T17:09:42.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.611+0000 7fc147be8740 1 -- 172.21.15.45:0/795079662 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x555c738d05c0 con 0x555c73851670 2024-04-16T17:09:42.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.611+0000 7fc123e00640 1 -- 172.21.15.45:0/795079662 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc128005520 con 0x555c73851670 2024-04-16T17:09:42.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.611+0000 7fc123e00640 1 -- 172.21.15.45:0/795079662 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fc128055ba0 con 0x555c73851670 2024-04-16T17:09:42.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.612+0000 7fc147be8740 10 client.16314 fetch_fsmap finished waiting for FSMap version 30 2024-04-16T17:09:42.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.612+0000 7fc147be8740 10 client.16314 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-04-16T17:09:42.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.612+0000 7fc147be8740 20 client.16314 populate_metadata read hostname 'smithi045' 2024-04-16T17:09:42.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.612+0000 7fc147be8740 1 --2- 172.21.15.45:0/795079662 >> [v2:172.21.15.45:6840/3755310822,v1:172.21.15.45:6841/3755310822] conn(0x555c738d09e0 0x555c738d2e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:42.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.612+0000 7fc147be8740 4 client.16314 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T17:09:42.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.612+0000 7fc147be8740 1 -- 172.21.15.45:0/795079662 --> [v2:172.21.15.45:6840/3755310822,v1:172.21.15.45:6841/3755310822] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555c737bd8b0 con 0x555c738d09e0 2024-04-16T17:09:42.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.612+0000 7fc13de00640 1 --2- 172.21.15.45:0/795079662 >> [v2:172.21.15.45:6840/3755310822,v1:172.21.15.45:6841/3755310822] conn(0x555c738d09e0 0x555c738d2e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:42.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.612+0000 7fc13de00640 1 --2- 172.21.15.45:0/795079662 >> [v2:172.21.15.45:6840/3755310822,v1:172.21.15.45:6841/3755310822] conn(0x555c738d09e0 0x555c738d2e00 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a044cd6188fef445 server_cookie=7e5578524148c92e in_seq=0 out_seq=0 2024-04-16T17:09:42.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.612+0000 7fc123e00640 10 client.16314 ms_handle_connect on v2:172.21.15.45:6840/3755310822 2024-04-16T17:09:42.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.613+0000 7fc123e00640 1 -- 172.21.15.45:0/795079662 <== mds.1 v2:172.21.15.45:6840/3755310822 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555c737bd8b0 con 0x555c738d09e0 2024-04-16T17:09:42.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.613+0000 7fc123e00640 10 client.16314 handle_command_reply: tid=0 2024-04-16T17:09:42.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.613+0000 7fc147be8740 1 client.16314 shutdown 2024-04-16T17:09:42.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.613+0000 7fc147be8740 1 -- 172.21.15.45:0/795079662 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555c73851670 msgr2=0x555c7385dbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:42.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.613+0000 7fc147be8740 1 --2- 172.21.15.45:0/795079662 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555c73851670 0x555c7385dbc0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fc12802c870 tx=0x7fc128005dc0 comp rx=0 tx=0).stop 2024-04-16T17:09:42.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.613+0000 7fc147be8740 1 -- 172.21.15.45:0/795079662 shutdown_connections 2024-04-16T17:09:42.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.613+0000 7fc147be8740 1 --2- 172.21.15.45:0/795079662 >> [v2:172.21.15.45:6840/3755310822,v1:172.21.15.45:6841/3755310822] conn(0x555c738d09e0 0x555c738d2e00 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:42.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.613+0000 7fc147be8740 1 --2- 172.21.15.45:0/795079662 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555c7385a250 0x555c7385e160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:42.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.613+0000 7fc147be8740 1 --2- 172.21.15.45:0/795079662 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555c73851670 0x555c7385dbc0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:42.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.613+0000 7fc147be8740 1 --2- 172.21.15.45:0/795079662 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555c73850d50 0x555c73862850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:42.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.613+0000 7fc147be8740 1 -- 172.21.15.45:0/795079662 >> 172.21.15.45:0/795079662 conn(0x555c7384b480 msgr2=0x555c73857210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:09:42.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.613+0000 7fc123e00640 0 client.16314 ms_handle_reset on v2:172.21.15.45:6840/3755310822 2024-04-16T17:09:42.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.613+0000 7fc147be8740 1 -- 172.21.15.45:0/795079662 shutdown_connections 2024-04-16T17:09:42.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.613+0000 7fc147be8740 1 -- 172.21.15.45:0/795079662 wait complete. 2024-04-16T17:09:42.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.613+0000 7fc147be8740 20 client.16314 trim_cache size 0 max 16384 2024-04-16T17:09:42.618 INFO:teuthology.orchestra.run.smithi045.stdout:[] 2024-04-16T17:09:42.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.614+0000 7fc144e00640 1 -- 172.21.15.45:0/2763445859 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fc1240a3da0 msgr2=0x7fc1240a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:42.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.614+0000 7fc144e00640 1 --2- 172.21.15.45:0/2763445859 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fc1240a3da0 0x7fc1240a6260 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fc138009fd0 tx=0x7fc138009500 comp rx=0 tx=0).stop 2024-04-16T17:09:42.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.614+0000 7fc144e00640 1 -- 172.21.15.45:0/2763445859 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc140103260 msgr2=0x7fc140100eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:42.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.614+0000 7fc144e00640 1 --2- 172.21.15.45:0/2763445859 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc140103260 0x7fc140100eb0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fc13000e9c0 tx=0x7fc13000ee80 comp rx=0 tx=0).stop 2024-04-16T17:09:42.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.614+0000 7fc144e00640 1 -- 172.21.15.45:0/2763445859 shutdown_connections 2024-04-16T17:09:42.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.614+0000 7fc144e00640 1 --2- 172.21.15.45:0/2763445859 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fc1240a3da0 0x7fc1240a6260 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:42.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.614+0000 7fc144e00640 1 --2- 172.21.15.45:0/2763445859 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc140103b80 0x7fc1401013f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:42.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.614+0000 7fc144e00640 1 --2- 172.21.15.45:0/2763445859 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc14010c630 0x7fc140101930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:42.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.614+0000 7fc144e00640 1 --2- 172.21.15.45:0/2763445859 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc140103260 0x7fc140100eb0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:42.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.614+0000 7fc144e00640 1 -- 172.21.15.45:0/2763445859 >> 172.21.15.45:0/2763445859 conn(0x7fc140077280 msgr2=0x7fc1400fdde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:09:42.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.614+0000 7fc144e00640 1 -- 172.21.15.45:0/2763445859 shutdown_connections 2024-04-16T17:09:42.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.614+0000 7fc144e00640 1 -- 172.21.15.45:0/2763445859 wait complete. 2024-04-16T17:09:42.631 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-16T17:09:42.677 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:09:42 smithi104 ceph-mon[36457]: pgmap v91: 129 pgs: 129 active+clean; 31 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 78 KiB/s wr, 27 op/s 2024-04-16T17:09:42.677 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:09:42 smithi104 ceph-mon[36457]: from='client.? 172.21.15.45:0/582844847' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:09:42.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:09:42 smithi117 ceph-mon[36730]: pgmap v91: 129 pgs: 129 active+clean; 31 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 78 KiB/s wr, 27 op/s 2024-04-16T17:09:42.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:09:42 smithi117 ceph-mon[36730]: from='client.? 172.21.15.45:0/582844847' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:09:42.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.880+0000 7fc7dce00640 1 -- 172.21.15.45:0/298111890 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc7d8104e60 msgr2=0x7fc7d8108ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:42.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.880+0000 7fc7dce00640 1 --2- 172.21.15.45:0/298111890 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc7d8104e60 0x7fc7d8108ea0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fc7c4009860 tx=0x7fc7c402bb70 comp rx=0 tx=0).stop 2024-04-16T17:09:42.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.880+0000 7fc7dce00640 1 -- 172.21.15.45:0/298111890 shutdown_connections 2024-04-16T17:09:42.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.880+0000 7fc7dce00640 1 --2- 172.21.15.45:0/298111890 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc7d8109580 0x7fc7d8111150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:42.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.880+0000 7fc7dce00640 1 --2- 172.21.15.45:0/298111890 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc7d8104e60 0x7fc7d8108ea0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:42.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.880+0000 7fc7dce00640 1 --2- 172.21.15.45:0/298111890 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc7d81044b0 0x7fc7d8104890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:42.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.880+0000 7fc7dce00640 1 -- 172.21.15.45:0/298111890 >> 172.21.15.45:0/298111890 conn(0x7fc7d80ffd90 msgr2=0x7fc7d81021b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:09:42.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.880+0000 7fc7dce00640 1 -- 172.21.15.45:0/298111890 shutdown_connections 2024-04-16T17:09:42.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.880+0000 7fc7dce00640 1 -- 172.21.15.45:0/298111890 wait complete. 2024-04-16T17:09:42.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.882+0000 7fc7dce00640 1 Processor -- start 2024-04-16T17:09:42.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.882+0000 7fc7dce00640 1 -- start start 2024-04-16T17:09:42.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.882+0000 7fc7dce00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc7d81044b0 0x7fc7d81a07f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:42.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.882+0000 7fc7dce00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc7d8109580 0x7fc7d81a0d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:42.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.883+0000 7fc7dce00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc7d819a9f0 0x7fc7d819aea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:42.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.883+0000 7fc7dce00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fc7d81145f0 con 0x7fc7d81044b0 2024-04-16T17:09:42.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.883+0000 7fc7dce00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fc7d8114470 con 0x7fc7d8109580 2024-04-16T17:09:42.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.883+0000 7fc7dce00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fc7d8114770 con 0x7fc7d819a9f0 2024-04-16T17:09:42.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.883+0000 7fc7d5e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc7d81044b0 0x7fc7d81a07f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:42.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.883+0000 7fc7d5400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc7d8109580 0x7fc7d81a0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:42.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.883+0000 7fc7d5e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc7d81044b0 0x7fc7d81a07f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:47966/0 (socket says 172.21.15.45:47966) 2024-04-16T17:09:42.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.883+0000 7fc7d5e00640 1 -- 172.21.15.45:0/4161855496 learned_addr learned my addr 172.21.15.45:0/4161855496 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:09:42.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.883+0000 7fc7d6800640 1 --2- 172.21.15.45:0/4161855496 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc7d819a9f0 0x7fc7d819aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:42.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.883+0000 7fc7d5400640 1 -- 172.21.15.45:0/4161855496 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc7d819a9f0 msgr2=0x7fc7d819aea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:42.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.883+0000 7fc7d5400640 1 --2- 172.21.15.45:0/4161855496 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc7d819a9f0 0x7fc7d819aea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:42.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.883+0000 7fc7d5400640 1 -- 172.21.15.45:0/4161855496 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc7d81044b0 msgr2=0x7fc7d81a07f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:42.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.883+0000 7fc7d5400640 1 --2- 172.21.15.45:0/4161855496 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc7d81044b0 0x7fc7d81a07f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:42.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.883+0000 7fc7d5400640 1 -- 172.21.15.45:0/4161855496 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7d819b510 con 0x7fc7d8109580 2024-04-16T17:09:42.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.884+0000 7fc7d5400640 1 --2- 172.21.15.45:0/4161855496 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc7d8109580 0x7fc7d81a0d30 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fc7c4037040 tx=0x7fc7c40045b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:09:42.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.884+0000 7fc7cea00640 1 -- 172.21.15.45:0/4161855496 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7c4041020 con 0x7fc7d8109580 2024-04-16T17:09:42.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.884+0000 7fc7cea00640 1 -- 172.21.15.45:0/4161855496 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc7c402c9f0 con 0x7fc7d8109580 2024-04-16T17:09:42.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.884+0000 7fc7cea00640 1 -- 172.21.15.45:0/4161855496 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7c4039040 con 0x7fc7d8109580 2024-04-16T17:09:42.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.884+0000 7fc7dce00640 1 -- 172.21.15.45:0/4161855496 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7d819b7a0 con 0x7fc7d8109580 2024-04-16T17:09:42.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.884+0000 7fc7dce00640 1 -- 172.21.15.45:0/4161855496 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7d81a78f0 con 0x7fc7d8109580 2024-04-16T17:09:42.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.887+0000 7fc7cea00640 1 -- 172.21.15.45:0/4161855496 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fc7c4044020 con 0x7fc7d8109580 2024-04-16T17:09:42.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.888+0000 7fc7dfa6d740 1 Processor -- start 2024-04-16T17:09:42.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.888+0000 7fc7dfa6d740 1 -- start start 2024-04-16T17:09:42.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.888+0000 7fc7dfa6d740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55681e767d50 0x55681e768130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:42.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.888+0000 7fc7dfa6d740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55681e768670 0x55681e770d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:42.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.888+0000 7fc7dfa6d740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55681e771250 0x55681e773640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:42.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.888+0000 7fc7dfa6d740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55681e6cd020 con 0x55681e771250 2024-04-16T17:09:42.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.888+0000 7fc7dfa6d740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55681e673460 con 0x55681e768670 2024-04-16T17:09:42.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.888+0000 7fc7dfa6d740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55681e6c8fe0 con 0x55681e767d50 2024-04-16T17:09:42.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.888+0000 7fc7d6800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55681e767d50 0x55681e768130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:42.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.888+0000 7fc7d5e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55681e768670 0x55681e770d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:42.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.888+0000 7fc7d6800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55681e767d50 0x55681e768130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:37076/0 (socket says 172.21.15.45:37076) 2024-04-16T17:09:42.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.888+0000 7fc7d6800640 1 -- 172.21.15.45:0/1708997825 learned_addr learned my addr 172.21.15.45:0/1708997825 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:09:42.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.889+0000 7fc7d5400640 1 --2- 172.21.15.45:0/1708997825 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55681e771250 0x55681e773640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:42.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.889+0000 7fc7d5e00640 1 -- 172.21.15.45:0/1708997825 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55681e767d50 msgr2=0x55681e768130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:09:42.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.889+0000 7fc7d5e00640 1 --2- 172.21.15.45:0/1708997825 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55681e767d50 0x55681e768130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:42.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.889+0000 7fc7d5e00640 1 -- 172.21.15.45:0/1708997825 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55681e771250 msgr2=0x55681e773640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:09:42.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.889+0000 7fc7d5e00640 1 --2- 172.21.15.45:0/1708997825 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55681e771250 0x55681e773640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:42.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.889+0000 7fc7d5e00640 1 -- 172.21.15.45:0/1708997825 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55681e67b040 con 0x55681e768670 2024-04-16T17:09:42.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.889+0000 7fc7d5e00640 1 --2- 172.21.15.45:0/1708997825 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55681e768670 0x55681e770d10 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fc7c8005c80 tx=0x7fc7c8011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2071054c3662108b server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:09:42.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.889+0000 7fc7c3400640 1 -- 172.21.15.45:0/1708997825 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7c8004230 con 0x55681e768670 2024-04-16T17:09:42.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.889+0000 7fc7c3400640 1 -- 172.21.15.45:0/1708997825 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc7c8009e40 con 0x55681e768670 2024-04-16T17:09:42.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.889+0000 7fc7c3400640 1 -- 172.21.15.45:0/1708997825 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7c8004e10 con 0x55681e768670 2024-04-16T17:09:42.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.890+0000 7fc7cea00640 1 --2- 172.21.15.45:0/4161855496 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fc7bc0a3da0 0x7fc7bc0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:42.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.890+0000 7fc7cea00640 1 -- 172.21.15.45:0/4161855496 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fc7c40f2120 con 0x7fc7d8109580 2024-04-16T17:09:42.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.890+0000 7fc7d6800640 1 --2- 172.21.15.45:0/4161855496 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fc7bc0a3da0 0x7fc7bc0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:42.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.890+0000 7fc7dfa6d740 1 -- 172.21.15.45:0/1708997825 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55681e768670 msgr2=0x55681e770d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:42.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.890+0000 7fc7dfa6d740 1 --2- 172.21.15.45:0/1708997825 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55681e768670 0x55681e770d10 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fc7c8005c80 tx=0x7fc7c8011810 comp rx=0 tx=0).stop 2024-04-16T17:09:42.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.891+0000 7fc7dfa6d740 1 -- 172.21.15.45:0/1708997825 shutdown_connections 2024-04-16T17:09:42.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.891+0000 7fc7dfa6d740 1 --2- 172.21.15.45:0/1708997825 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55681e771250 0x55681e773640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:42.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.891+0000 7fc7dfa6d740 1 --2- 172.21.15.45:0/1708997825 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55681e768670 0x55681e770d10 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:42.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.891+0000 7fc7dfa6d740 1 --2- 172.21.15.45:0/1708997825 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55681e767d50 0x55681e768130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:42.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.891+0000 7fc7dfa6d740 1 -- 172.21.15.45:0/1708997825 >> 172.21.15.45:0/1708997825 conn(0x55681e762480 msgr2=0x55681e762860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:09:42.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.891+0000 7fc7dfa6d740 1 -- 172.21.15.45:0/1708997825 shutdown_connections 2024-04-16T17:09:42.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.891+0000 7fc7d6800640 1 --2- 172.21.15.45:0/4161855496 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fc7bc0a3da0 0x7fc7bc0a6260 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fc7d0011630 tx=0x7fc7d0009500 comp rx=0 tx=0).ready entity=mgr.25411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:09:42.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.891+0000 7fc7dfa6d740 1 -- 172.21.15.45:0/1708997825 wait complete. 2024-04-16T17:09:42.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.891+0000 7fc7dfa6d740 1 -- start start 2024-04-16T17:09:42.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.891+0000 7fc7dfa6d740 1 Processor -- start 2024-04-16T17:09:42.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.892+0000 7fc7dfa6d740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55681e767d50 0x55681e76e300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:42.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.892+0000 7fc7dfa6d740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55681e768670 0x55681e76e840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:42.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.892+0000 7fc7dfa6d740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55681e771250 0x55681e76ed80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:42.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.892+0000 7fc7dfa6d740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55681e774040 con 0x55681e768670 2024-04-16T17:09:42.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.892+0000 7fc7dfa6d740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55681e686830 con 0x55681e771250 2024-04-16T17:09:42.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.892+0000 7fc7dfa6d740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55681e7741c0 con 0x55681e767d50 2024-04-16T17:09:42.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.892+0000 7fc7dfa6d740 10 client.0 _ll_register_callbacks cb 0x55681e68e150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:09:42.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.892+0000 7fc7d5400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55681e768670 0x55681e76e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:42.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.892+0000 7fc7d5e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55681e767d50 0x55681e76e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:42.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.892+0000 7fc7d5400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55681e768670 0x55681e76e840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:47982/0 (socket says 172.21.15.45:47982) 2024-04-16T17:09:42.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.892+0000 7fc7d5e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55681e767d50 0x55681e76e300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:37086/0 (socket says 172.21.15.45:37086) 2024-04-16T17:09:42.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.892+0000 7fc7d5400640 1 -- 172.21.15.45:0/3031622854 learned_addr learned my addr 172.21.15.45:0/3031622854 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:09:42.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.892+0000 7fc7d6800640 1 --2- 172.21.15.45:0/3031622854 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55681e771250 0x55681e76ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:42.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.893+0000 7fc7d5e00640 1 -- 172.21.15.45:0/3031622854 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55681e771250 msgr2=0x55681e76ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:09:42.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.893+0000 7fc7d5e00640 1 --2- 172.21.15.45:0/3031622854 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55681e771250 0x55681e76ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:42.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.893+0000 7fc7d5e00640 1 -- 172.21.15.45:0/3031622854 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55681e768670 msgr2=0x55681e76e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:09:42.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.893+0000 7fc7d5e00640 1 --2- 172.21.15.45:0/3031622854 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55681e768670 0x55681e76e840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:42.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.893+0000 7fc7d5e00640 1 -- 172.21.15.45:0/3031622854 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55681e68a920 con 0x55681e767d50 2024-04-16T17:09:42.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.893+0000 7fc7d5e00640 1 --2- 172.21.15.45:0/3031622854 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55681e767d50 0x55681e76e300 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fc7c8000c00 tx=0x7fc7c8003820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=69b82c0d4dc91771 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:09:42.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.893+0000 7fc7b7e00640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-04-16T17:09:42.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.893+0000 7fc7b7e00640 1 -- 172.21.15.45:0/3031622854 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7c8027020 con 0x55681e767d50 2024-04-16T17:09:42.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.893+0000 7fc7b7e00640 1 -- 172.21.15.45:0/3031622854 <== mon.1 v2:172.21.15.117:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fc7c8003c10 con 0x55681e767d50 2024-04-16T17:09:42.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.893+0000 7fc7dfa6d740 1 -- 172.21.15.45:0/3031622854 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55681e6daa40 con 0x55681e767d50 2024-04-16T17:09:42.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.894+0000 7fc7b7e00640 1 -- 172.21.15.45:0/3031622854 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fc7c8032050 con 0x55681e767d50 2024-04-16T17:09:42.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.894+0000 7fc7dfa6d740 10 client.26455 fetch_fsmap learned FSMap version 30 2024-04-16T17:09:42.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.894+0000 7fc7dfa6d740 1 -- 172.21.15.45:0/3031622854 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55681e6587f0 con 0x55681e767d50 2024-04-16T17:09:42.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.894+0000 7fc7b7e00640 1 -- 172.21.15.45:0/3031622854 <== mon.1 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55681e6daa40 con 0x55681e767d50 2024-04-16T17:09:42.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.894+0000 7fc7b7e00640 1 -- 172.21.15.45:0/3031622854 <== mon.1 v2:172.21.15.117:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fc7c8018070 con 0x55681e767d50 2024-04-16T17:09:42.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.894+0000 7fc7dfa6d740 10 client.26455 fetch_fsmap finished waiting for FSMap version 30 2024-04-16T17:09:42.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.894+0000 7fc7dfa6d740 10 client.26455 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:09:42.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.894+0000 7fc7dfa6d740 20 client.26455 populate_metadata read hostname 'smithi045' 2024-04-16T17:09:42.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.894+0000 7fc7dfa6d740 1 --2- 172.21.15.45:0/3031622854 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55681e781600 0x55681e7839f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:42.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.894+0000 7fc7dfa6d740 4 client.26455 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:09:42.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.894+0000 7fc7dfa6d740 1 -- 172.21.15.45:0/3031622854 --> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55681e6d48b0 con 0x55681e781600 2024-04-16T17:09:42.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.895+0000 7fc7d5400640 1 --2- 172.21.15.45:0/3031622854 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55681e781600 0x55681e7839f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:42.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.895+0000 7fc7d5400640 1 --2- 172.21.15.45:0/3031622854 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55681e781600 0x55681e7839f0 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=32e6b922476ee2fc server_cookie=77f293154d6447ff in_seq=0 out_seq=0 2024-04-16T17:09:42.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.895+0000 7fc7b7e00640 10 client.26455 ms_handle_connect on v2:172.21.15.104:6838/3660070592 2024-04-16T17:09:42.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.896+0000 7fc7b7e00640 1 -- 172.21.15.45:0/3031622854 <== mds.0 v2:172.21.15.104:6838/3660070592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55681e6d48b0 con 0x55681e781600 2024-04-16T17:09:42.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.896+0000 7fc7b7e00640 10 client.26455 handle_command_reply: tid=0 2024-04-16T17:09:42.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.896+0000 7fc7dfa6d740 1 client.26455 shutdown 2024-04-16T17:09:42.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.896+0000 7fc7dfa6d740 1 -- 172.21.15.45:0/3031622854 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55681e767d50 msgr2=0x55681e76e300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:42.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.896+0000 7fc7dfa6d740 1 --2- 172.21.15.45:0/3031622854 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55681e767d50 0x55681e76e300 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fc7c8000c00 tx=0x7fc7c8003820 comp rx=0 tx=0).stop 2024-04-16T17:09:42.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.896+0000 7fc7dfa6d740 1 -- 172.21.15.45:0/3031622854 shutdown_connections 2024-04-16T17:09:42.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.896+0000 7fc7dfa6d740 1 --2- 172.21.15.45:0/3031622854 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55681e771250 0x55681e76ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:42.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.896+0000 7fc7dfa6d740 1 --2- 172.21.15.45:0/3031622854 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55681e768670 0x55681e76e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:42.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.896+0000 7fc7dfa6d740 1 --2- 172.21.15.45:0/3031622854 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55681e781600 0x55681e7839f0 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:42.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.896+0000 7fc7dfa6d740 1 --2- 172.21.15.45:0/3031622854 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55681e767d50 0x55681e76e300 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:42.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.896+0000 7fc7b7e00640 0 client.26455 ms_handle_reset on v2:172.21.15.104:6838/3660070592 2024-04-16T17:09:42.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.896+0000 7fc7dfa6d740 1 -- 172.21.15.45:0/3031622854 >> 172.21.15.45:0/3031622854 conn(0x55681e762480 msgr2=0x55681e7641d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:09:42.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.896+0000 7fc7dfa6d740 1 -- 172.21.15.45:0/3031622854 shutdown_connections 2024-04-16T17:09:42.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.896+0000 7fc7dfa6d740 1 -- 172.21.15.45:0/3031622854 wait complete. 2024-04-16T17:09:42.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.897+0000 7fc7dfa6d740 20 client.26455 trim_cache size 0 max 16384 2024-04-16T17:09:42.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.912+0000 7fc7dfa6d740 1 Processor -- start 2024-04-16T17:09:42.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.912+0000 7fc7dfa6d740 1 -- start start 2024-04-16T17:09:42.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.912+0000 7fc7dfa6d740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55681e767d50 0x55681e770a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:42.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.912+0000 7fc7dfa6d740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55681e768670 0x55681e77cf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:42.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.912+0000 7fc7dfa6d740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55681e771250 0x55681e761360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:42.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.912+0000 7fc7dfa6d740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fc7d81b4010 con 0x55681e767d50 2024-04-16T17:09:42.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.912+0000 7fc7dfa6d740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fc7d81b1820 con 0x55681e771250 2024-04-16T17:09:42.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.912+0000 7fc7dfa6d740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fc7d81af650 con 0x55681e768670 2024-04-16T17:09:42.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.912+0000 7fc7d5e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55681e767d50 0x55681e770a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:42.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.912+0000 7fc7d5e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55681e767d50 0x55681e770a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:47994/0 (socket says 172.21.15.45:47994) 2024-04-16T17:09:42.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.912+0000 7fc7d5e00640 1 -- 172.21.15.45:0/2583152931 learned_addr learned my addr 172.21.15.45:0/2583152931 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:09:42.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.913+0000 7fc7d5400640 1 --2- 172.21.15.45:0/2583152931 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55681e768670 0x55681e77cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:42.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.913+0000 7fc7d6800640 1 --2- 172.21.15.45:0/2583152931 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55681e771250 0x55681e761360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:42.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.913+0000 7fc7d5e00640 1 -- 172.21.15.45:0/2583152931 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55681e768670 msgr2=0x55681e77cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:09:42.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.913+0000 7fc7d5e00640 1 --2- 172.21.15.45:0/2583152931 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55681e768670 0x55681e77cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:42.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.913+0000 7fc7d5e00640 1 -- 172.21.15.45:0/2583152931 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55681e771250 msgr2=0x55681e761360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:09:42.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.913+0000 7fc7d5e00640 1 --2- 172.21.15.45:0/2583152931 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55681e771250 0x55681e761360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:42.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.913+0000 7fc7d5e00640 1 -- 172.21.15.45:0/2583152931 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55681e7684a0 con 0x55681e767d50 2024-04-16T17:09:42.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.913+0000 7fc7d6800640 1 --2- 172.21.15.45:0/2583152931 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55681e771250 0x55681e761360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:09:42.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.913+0000 7fc7d5400640 1 --2- 172.21.15.45:0/2583152931 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55681e768670 0x55681e77cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:09:42.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.913+0000 7fc7d5e00640 1 --2- 172.21.15.45:0/2583152931 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55681e767d50 0x55681e770a30 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fc7c80098b0 tx=0x7fc7c8009b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=beea920eb88eb65c server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:09:42.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.913+0000 7fc7c2a00640 1 -- 172.21.15.45:0/2583152931 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7c8027040 con 0x55681e767d50 2024-04-16T17:09:42.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.913+0000 7fc7c2a00640 1 -- 172.21.15.45:0/2583152931 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc7c8005620 con 0x55681e767d50 2024-04-16T17:09:42.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.913+0000 7fc7c2a00640 1 -- 172.21.15.45:0/2583152931 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7c801e040 con 0x55681e767d50 2024-04-16T17:09:42.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.914+0000 7fc7dfa6d740 1 -- 172.21.15.45:0/2583152931 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55681e767d50 msgr2=0x55681e770a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:42.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.914+0000 7fc7dfa6d740 1 --2- 172.21.15.45:0/2583152931 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55681e767d50 0x55681e770a30 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fc7c80098b0 tx=0x7fc7c8009b40 comp rx=0 tx=0).stop 2024-04-16T17:09:42.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.914+0000 7fc7dfa6d740 1 -- 172.21.15.45:0/2583152931 shutdown_connections 2024-04-16T17:09:42.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.914+0000 7fc7dfa6d740 1 --2- 172.21.15.45:0/2583152931 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55681e768670 0x55681e77cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:42.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.914+0000 7fc7dfa6d740 1 --2- 172.21.15.45:0/2583152931 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55681e771250 0x55681e761360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:42.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.914+0000 7fc7dfa6d740 1 --2- 172.21.15.45:0/2583152931 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55681e767d50 0x55681e770a30 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:42.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.914+0000 7fc7dfa6d740 1 -- 172.21.15.45:0/2583152931 >> 172.21.15.45:0/2583152931 conn(0x55681e762480 msgr2=0x55681e760c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:09:42.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.914+0000 7fc7dfa6d740 1 -- 172.21.15.45:0/2583152931 shutdown_connections 2024-04-16T17:09:42.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.914+0000 7fc7dfa6d740 1 -- 172.21.15.45:0/2583152931 wait complete. 2024-04-16T17:09:42.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.915+0000 7fc7dfa6d740 1 -- start start 2024-04-16T17:09:42.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.915+0000 7fc7dfa6d740 1 Processor -- start 2024-04-16T17:09:42.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.915+0000 7fc7dfa6d740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55681e767d50 0x55681e7796e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:42.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.915+0000 7fc7dfa6d740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55681e768670 0x55681e76ab90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:42.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.915+0000 7fc7dfa6d740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55681e771250 0x55681e76b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:42.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.915+0000 7fc7dfa6d740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fc7d81afce0 con 0x55681e771250 2024-04-16T17:09:42.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.915+0000 7fc7dfa6d740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fc7d81b1150 con 0x55681e768670 2024-04-16T17:09:42.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.915+0000 7fc7dfa6d740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fc7d81b03b0 con 0x55681e767d50 2024-04-16T17:09:42.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.915+0000 7fc7dfa6d740 10 client.0 _ll_register_callbacks cb 0x7fc7d0009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:09:42.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.916+0000 7fc7d5e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55681e767d50 0x55681e7796e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:42.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.916+0000 7fc7d5e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55681e767d50 0x55681e7796e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:37100/0 (socket says 172.21.15.45:37100) 2024-04-16T17:09:42.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.916+0000 7fc7d5e00640 1 -- 172.21.15.45:0/2997115995 learned_addr learned my addr 172.21.15.45:0/2997115995 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:09:42.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.916+0000 7fc7d6800640 1 --2- 172.21.15.45:0/2997115995 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55681e771250 0x55681e76b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:42.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.916+0000 7fc7d5400640 1 --2- 172.21.15.45:0/2997115995 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55681e768670 0x55681e76ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:42.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.916+0000 7fc7d6800640 1 -- 172.21.15.45:0/2997115995 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55681e767d50 msgr2=0x55681e7796e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:09:42.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.916+0000 7fc7d6800640 1 --2- 172.21.15.45:0/2997115995 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55681e767d50 0x55681e7796e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:42.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.916+0000 7fc7d6800640 1 -- 172.21.15.45:0/2997115995 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55681e768670 msgr2=0x55681e76ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:09:42.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.916+0000 7fc7d6800640 1 --2- 172.21.15.45:0/2997115995 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55681e768670 0x55681e76ab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:42.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.916+0000 7fc7d6800640 1 -- 172.21.15.45:0/2997115995 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55681e3df3f0 con 0x55681e771250 2024-04-16T17:09:42.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.916+0000 7fc7d5e00640 1 --2- 172.21.15.45:0/2997115995 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55681e767d50 0x55681e7796e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:09:42.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.916+0000 7fc7d6800640 1 --2- 172.21.15.45:0/2997115995 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55681e771250 0x55681e76b0d0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x55681e761d80 tx=0x7fc7d0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b949abf7424e274b server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:09:42.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.917+0000 7fc7b7e00640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-16T17:09:42.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.917+0000 7fc7b7e00640 1 -- 172.21.15.45:0/2997115995 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7d002d060 con 0x55681e771250 2024-04-16T17:09:42.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.917+0000 7fc7dfa6d740 1 -- 172.21.15.45:0/2997115995 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc7c8005620 con 0x55681e771250 2024-04-16T17:09:42.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.917+0000 7fc7b7e00640 1 -- 172.21.15.45:0/2997115995 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fc7d000f8f0 con 0x55681e771250 2024-04-16T17:09:42.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.917+0000 7fc7b7e00640 1 -- 172.21.15.45:0/2997115995 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fc7d0034050 con 0x55681e771250 2024-04-16T17:09:42.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.917+0000 7fc7dfa6d740 10 client.16332 fetch_fsmap learned FSMap version 30 2024-04-16T17:09:42.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.917+0000 7fc7dfa6d740 1 -- 172.21.15.45:0/2997115995 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55681e4a97d0 con 0x55681e771250 2024-04-16T17:09:42.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.917+0000 7fc7b7e00640 1 -- 172.21.15.45:0/2997115995 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc7c8005620 con 0x55681e771250 2024-04-16T17:09:42.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.918+0000 7fc7b7e00640 1 -- 172.21.15.45:0/2997115995 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fc7d001ecb0 con 0x55681e771250 2024-04-16T17:09:42.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.918+0000 7fc7dfa6d740 10 client.16332 fetch_fsmap finished waiting for FSMap version 30 2024-04-16T17:09:42.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.918+0000 7fc7dfa6d740 10 client.16332 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:09:42.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.918+0000 7fc7dfa6d740 20 client.16332 populate_metadata read hostname 'smithi045' 2024-04-16T17:09:42.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.918+0000 7fc7dfa6d740 1 --2- 172.21.15.45:0/2997115995 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55681e7e7e30 0x55681e7ea250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:42.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.918+0000 7fc7dfa6d740 4 client.16332 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T17:09:42.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.918+0000 7fc7dfa6d740 1 -- 172.21.15.45:0/2997115995 --> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55681e650570 con 0x55681e7e7e30 2024-04-16T17:09:42.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.918+0000 7fc7d5e00640 1 --2- 172.21.15.45:0/2997115995 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55681e7e7e30 0x55681e7ea250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:42.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.919+0000 7fc7d5e00640 1 --2- 172.21.15.45:0/2997115995 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55681e7e7e30 0x55681e7ea250 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=11e5d61c4900cdc0 server_cookie=e83cdc6b46663848 in_seq=0 out_seq=0 2024-04-16T17:09:42.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.919+0000 7fc7b7e00640 10 client.16332 ms_handle_connect on v2:172.21.15.104:6838/3660070592 2024-04-16T17:09:42.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.919+0000 7fc7b7e00640 1 -- 172.21.15.45:0/2997115995 <== mds.0 v2:172.21.15.104:6838/3660070592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55681e650570 con 0x55681e7e7e30 2024-04-16T17:09:42.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.919+0000 7fc7b7e00640 10 client.16332 handle_command_reply: tid=0 2024-04-16T17:09:42.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.919+0000 7fc7dfa6d740 1 client.16332 shutdown 2024-04-16T17:09:42.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.919+0000 7fc7dfa6d740 1 -- 172.21.15.45:0/2997115995 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55681e771250 msgr2=0x55681e76b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:42.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.919+0000 7fc7dfa6d740 1 --2- 172.21.15.45:0/2997115995 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55681e771250 0x55681e76b0d0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x55681e761d80 tx=0x7fc7d0004a40 comp rx=0 tx=0).stop 2024-04-16T17:09:42.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.919+0000 7fc7dfa6d740 1 -- 172.21.15.45:0/2997115995 shutdown_connections 2024-04-16T17:09:42.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.919+0000 7fc7dfa6d740 1 --2- 172.21.15.45:0/2997115995 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55681e771250 0x55681e76b0d0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:42.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.919+0000 7fc7dfa6d740 1 --2- 172.21.15.45:0/2997115995 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55681e768670 0x55681e76ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:42.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.919+0000 7fc7dfa6d740 1 --2- 172.21.15.45:0/2997115995 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55681e7e7e30 0x55681e7ea250 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:42.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.919+0000 7fc7dfa6d740 1 --2- 172.21.15.45:0/2997115995 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55681e767d50 0x55681e7796e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:42.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.919+0000 7fc7dfa6d740 1 -- 172.21.15.45:0/2997115995 >> 172.21.15.45:0/2997115995 conn(0x55681e762480 msgr2=0x55681e76eb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:09:42.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.919+0000 7fc7b7e00640 0 client.16332 ms_handle_reset on v2:172.21.15.104:6838/3660070592 2024-04-16T17:09:42.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.919+0000 7fc7dfa6d740 1 -- 172.21.15.45:0/2997115995 shutdown_connections 2024-04-16T17:09:42.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.919+0000 7fc7dfa6d740 1 -- 172.21.15.45:0/2997115995 wait complete. 2024-04-16T17:09:42.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.919+0000 7fc7dfa6d740 20 client.16332 trim_cache size 0 max 16384 2024-04-16T17:09:42.924 INFO:teuthology.orchestra.run.smithi045.stdout:[] 2024-04-16T17:09:42.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.920+0000 7fc7dce00640 1 -- 172.21.15.45:0/4161855496 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fc7bc0a3da0 msgr2=0x7fc7bc0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:42.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.920+0000 7fc7dce00640 1 --2- 172.21.15.45:0/4161855496 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fc7bc0a3da0 0x7fc7bc0a6260 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fc7d0011630 tx=0x7fc7d0009500 comp rx=0 tx=0).stop 2024-04-16T17:09:42.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.920+0000 7fc7dce00640 1 -- 172.21.15.45:0/4161855496 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc7d8109580 msgr2=0x7fc7d81a0d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:42.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.920+0000 7fc7dce00640 1 --2- 172.21.15.45:0/4161855496 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc7d8109580 0x7fc7d81a0d30 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fc7c4037040 tx=0x7fc7c40045b0 comp rx=0 tx=0).stop 2024-04-16T17:09:42.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.920+0000 7fc7dce00640 1 -- 172.21.15.45:0/4161855496 shutdown_connections 2024-04-16T17:09:42.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.920+0000 7fc7dce00640 1 --2- 172.21.15.45:0/4161855496 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fc7bc0a3da0 0x7fc7bc0a6260 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:42.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.920+0000 7fc7dce00640 1 --2- 172.21.15.45:0/4161855496 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc7d819a9f0 0x7fc7d819aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:42.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.920+0000 7fc7dce00640 1 --2- 172.21.15.45:0/4161855496 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc7d8109580 0x7fc7d81a0d30 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:42.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.920+0000 7fc7dce00640 1 --2- 172.21.15.45:0/4161855496 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc7d81044b0 0x7fc7d81a07f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:42.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.920+0000 7fc7dce00640 1 -- 172.21.15.45:0/4161855496 >> 172.21.15.45:0/4161855496 conn(0x7fc7d80ffd90 msgr2=0x7fc7d8101150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:09:42.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.920+0000 7fc7dce00640 1 -- 172.21.15.45:0/4161855496 shutdown_connections 2024-04-16T17:09:42.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:42.920+0000 7fc7dce00640 1 -- 172.21.15.45:0/4161855496 wait complete. 2024-04-16T17:09:43.227 INFO:tasks.workunit.client.0.smithi045.stdout: 1 40035 2.21 MB/sec warmup 105 sec latency 4577.028 ms 2024-04-16T17:09:43.939 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-16T17:09:43.939 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T17:09:43.939 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-16T17:09:44.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.129+0000 7f4fdb200640 1 -- 172.21.15.45:0/1991498657 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4fd4104320 msgr2=0x7f4fd4104700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:44.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.129+0000 7f4fdb200640 1 --2- 172.21.15.45:0/1991498657 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4fd4104320 0x7f4fd4104700 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f4fc8009900 tx=0x7f4fc802bc30 comp rx=0 tx=0).stop 2024-04-16T17:09:44.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.129+0000 7f4fdb200640 1 -- 172.21.15.45:0/1991498657 shutdown_connections 2024-04-16T17:09:44.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.129+0000 7f4fdb200640 1 --2- 172.21.15.45:0/1991498657 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4fd41093f0 0x7f4fd4110fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:44.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.129+0000 7f4fdb200640 1 --2- 172.21.15.45:0/1991498657 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4fd4104cd0 0x7f4fd4108d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:44.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.129+0000 7f4fdb200640 1 --2- 172.21.15.45:0/1991498657 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4fd4104320 0x7f4fd4104700 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:44.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.129+0000 7f4fdb200640 1 -- 172.21.15.45:0/1991498657 >> 172.21.15.45:0/1991498657 conn(0x7f4fd40ffb40 msgr2=0x7f4fd4101f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:09:44.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.129+0000 7f4fdb200640 1 -- 172.21.15.45:0/1991498657 shutdown_connections 2024-04-16T17:09:44.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.129+0000 7f4fdb200640 1 -- 172.21.15.45:0/1991498657 wait complete. 2024-04-16T17:09:44.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.130+0000 7f4fdb200640 1 Processor -- start 2024-04-16T17:09:44.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.130+0000 7f4fdb200640 1 -- start start 2024-04-16T17:09:44.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.131+0000 7f4fdb200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4fd4104320 0x7f4fd41a0660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:44.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.131+0000 7f4fdb200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4fd41093f0 0x7f4fd41a0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:44.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.131+0000 7f4fdb200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4fd419a7d0 0x7f4fd419ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:44.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.131+0000 7f4fdb200640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f4fd41144b0 con 0x7f4fd419a7d0 2024-04-16T17:09:44.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.131+0000 7f4fdb200640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f4fd4114330 con 0x7f4fd4104320 2024-04-16T17:09:44.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.131+0000 7f4fdb200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f4fd4114630 con 0x7f4fd41093f0 2024-04-16T17:09:44.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.131+0000 7f4fd9e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4fd4104320 0x7f4fd41a0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:44.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.131+0000 7f4fd9e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4fd4104320 0x7f4fd41a0660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:47176/0 (socket says 172.21.15.45:47176) 2024-04-16T17:09:44.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.131+0000 7f4fda800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4fd419a7d0 0x7f4fd419ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:44.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.131+0000 7f4fd9e00640 1 -- 172.21.15.45:0/3854704071 learned_addr learned my addr 172.21.15.45:0/3854704071 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:09:44.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.131+0000 7f4fd9e00640 1 -- 172.21.15.45:0/3854704071 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4fd41093f0 msgr2=0x7f4fd41a0ba0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-16T17:09:44.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.131+0000 7f4fd9e00640 1 --2- 172.21.15.45:0/3854704071 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4fd41093f0 0x7f4fd41a0ba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:44.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.132+0000 7f4fd9e00640 1 -- 172.21.15.45:0/3854704071 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4fd419a7d0 msgr2=0x7f4fd419ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:44.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.132+0000 7f4fd9e00640 1 --2- 172.21.15.45:0/3854704071 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4fd419a7d0 0x7f4fd419ac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:44.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.132+0000 7f4fd9e00640 1 -- 172.21.15.45:0/3854704071 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fd419b360 con 0x7f4fd4104320 2024-04-16T17:09:44.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.132+0000 7f4fda800640 1 --2- 172.21.15.45:0/3854704071 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4fd419a7d0 0x7f4fd419ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:09:44.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.132+0000 7f4fd9e00640 1 --2- 172.21.15.45:0/3854704071 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4fd4104320 0x7f4fd41a0660 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f4fc8006010 tx=0x7f4fc802cb30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:09:44.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.132+0000 7f4fcea00640 1 -- 172.21.15.45:0/3854704071 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fc8041070 con 0x7f4fd4104320 2024-04-16T17:09:44.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.132+0000 7f4fdb200640 1 -- 172.21.15.45:0/3854704071 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4fd419b5f0 con 0x7f4fd4104320 2024-04-16T17:09:44.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.132+0000 7f4fcea00640 1 -- 172.21.15.45:0/3854704071 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4fc8004450 con 0x7f4fd4104320 2024-04-16T17:09:44.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.132+0000 7f4fcea00640 1 -- 172.21.15.45:0/3854704071 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fc8003e00 con 0x7f4fd4104320 2024-04-16T17:09:44.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.132+0000 7f4fdb200640 1 -- 172.21.15.45:0/3854704071 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4fd41a77d0 con 0x7f4fd4104320 2024-04-16T17:09:44.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.135+0000 7f4fdf57f740 1 Processor -- start 2024-04-16T17:09:44.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.135+0000 7f4fdf57f740 1 -- start start 2024-04-16T17:09:44.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.136+0000 7f4fcea00640 1 -- 172.21.15.45:0/3854704071 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f4fc8044070 con 0x7f4fd4104320 2024-04-16T17:09:44.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.136+0000 7f4fdf57f740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b4f36a5e90 0x55b4f36a6270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:44.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.136+0000 7f4fdf57f740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b4f36a67b0 0x55b4f36aee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:44.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.136+0000 7f4fdf57f740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b4f36af390 0x55b4f36b1780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:44.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.136+0000 7f4fdf57f740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55b4f360acb0 con 0x55b4f36a67b0 2024-04-16T17:09:44.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.136+0000 7f4fdf57f740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55b4f35b1370 con 0x55b4f36af390 2024-04-16T17:09:44.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.136+0000 7f4fdf57f740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55b4f3606c70 con 0x55b4f36a5e90 2024-04-16T17:09:44.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.136+0000 7f4fda800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b4f36a5e90 0x55b4f36a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:44.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.136+0000 7f4fda800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b4f36a5e90 0x55b4f36a6270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:37122/0 (socket says 172.21.15.45:37122) 2024-04-16T17:09:44.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.136+0000 7f4fda800640 1 -- 172.21.15.45:0/1940249431 learned_addr learned my addr 172.21.15.45:0/1940249431 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:09:44.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.136+0000 7f4fd9400640 1 --2- 172.21.15.45:0/1940249431 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b4f36af390 0x55b4f36b1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:44.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.136+0000 7f4fd9e00640 1 --2- 172.21.15.45:0/1940249431 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b4f36a67b0 0x55b4f36aee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:44.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.137+0000 7f4fda800640 1 -- 172.21.15.45:0/1940249431 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b4f36af390 msgr2=0x55b4f36b1780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:09:44.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.137+0000 7f4fda800640 1 --2- 172.21.15.45:0/1940249431 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b4f36af390 0x55b4f36b1780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:44.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.137+0000 7f4fda800640 1 -- 172.21.15.45:0/1940249431 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b4f36a67b0 msgr2=0x55b4f36aee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:09:44.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.137+0000 7f4fda800640 1 --2- 172.21.15.45:0/1940249431 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b4f36a67b0 0x55b4f36aee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:44.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.137+0000 7f4fda800640 1 -- 172.21.15.45:0/1940249431 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b4f35d0910 con 0x55b4f36a5e90 2024-04-16T17:09:44.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.137+0000 7f4fd9e00640 1 --2- 172.21.15.45:0/1940249431 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b4f36a67b0 0x55b4f36aee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:09:44.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.137+0000 7f4fcea00640 1 --2- 172.21.15.45:0/3854704071 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f4fbc0a3da0 0x7f4fbc0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:44.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.137+0000 7f4fda800640 1 --2- 172.21.15.45:0/1940249431 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b4f36a5e90 0x55b4f36a6270 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f4fd0010710 tx=0x7f4fd0010bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7511a563debdbb3c server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:09:44.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.138+0000 7f4fc3000640 1 -- 172.21.15.45:0/1940249431 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fd000e8d0 con 0x55b4f36a5e90 2024-04-16T17:09:44.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.138+0000 7f4fcea00640 1 -- 172.21.15.45:0/3854704071 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f4fc80f2550 con 0x7f4fd4104320 2024-04-16T17:09:44.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.138+0000 7f4fc3000640 1 -- 172.21.15.45:0/1940249431 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4fd0012450 con 0x55b4f36a5e90 2024-04-16T17:09:44.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.138+0000 7f4fc3000640 1 -- 172.21.15.45:0/1940249431 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fd001c670 con 0x55b4f36a5e90 2024-04-16T17:09:44.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.138+0000 7f4fd9e00640 1 --2- 172.21.15.45:0/3854704071 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f4fbc0a3da0 0x7f4fbc0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:44.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.138+0000 7f4fdf57f740 1 -- 172.21.15.45:0/1940249431 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b4f36a5e90 msgr2=0x55b4f36a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:44.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.138+0000 7f4fdf57f740 1 --2- 172.21.15.45:0/1940249431 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b4f36a5e90 0x55b4f36a6270 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f4fd0010710 tx=0x7f4fd0010bd0 comp rx=0 tx=0).stop 2024-04-16T17:09:44.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.139+0000 7f4fdf57f740 1 -- 172.21.15.45:0/1940249431 shutdown_connections 2024-04-16T17:09:44.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.139+0000 7f4fdf57f740 1 --2- 172.21.15.45:0/1940249431 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b4f36af390 0x55b4f36b1780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:44.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.139+0000 7f4fdf57f740 1 --2- 172.21.15.45:0/1940249431 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b4f36a67b0 0x55b4f36aee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:44.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.139+0000 7f4fdf57f740 1 --2- 172.21.15.45:0/1940249431 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b4f36a5e90 0x55b4f36a6270 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:44.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.139+0000 7f4fdf57f740 1 -- 172.21.15.45:0/1940249431 >> 172.21.15.45:0/1940249431 conn(0x55b4f36a05c0 msgr2=0x55b4f36a09a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:09:44.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.139+0000 7f4fd9e00640 1 --2- 172.21.15.45:0/3854704071 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f4fbc0a3da0 0x7f4fbc0a6260 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x55b4f36a5960 tx=0x7f4fc80f80e0 comp rx=0 tx=0).ready entity=mgr.25411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:09:44.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.139+0000 7f4fdf57f740 1 -- 172.21.15.45:0/1940249431 shutdown_connections 2024-04-16T17:09:44.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.139+0000 7f4fdf57f740 1 -- 172.21.15.45:0/1940249431 wait complete. 2024-04-16T17:09:44.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.139+0000 7f4fdf57f740 1 -- start start 2024-04-16T17:09:44.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.139+0000 7f4fdf57f740 1 Processor -- start 2024-04-16T17:09:44.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.140+0000 7f4fdf57f740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b4f36a5e90 0x55b4f36ac3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:44.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.140+0000 7f4fdf57f740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b4f36a67b0 0x55b4f36ac8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:44.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.140+0000 7f4fdf57f740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b4f36af390 0x55b4f36ace30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:44.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.140+0000 7f4fdf57f740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55b4f36b2180 con 0x55b4f36a67b0 2024-04-16T17:09:44.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.140+0000 7f4fdf57f740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55b4f35c4730 con 0x55b4f36af390 2024-04-16T17:09:44.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.140+0000 7f4fdf57f740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55b4f36b2300 con 0x55b4f36a5e90 2024-04-16T17:09:44.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.140+0000 7f4fdf57f740 10 client.0 _ll_register_callbacks cb 0x55b4f35931e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:09:44.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.140+0000 7f4fd9400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b4f36a67b0 0x55b4f36ac8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:44.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.140+0000 7f4fd9400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b4f36a67b0 0x55b4f36ac8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:48032/0 (socket says 172.21.15.45:48032) 2024-04-16T17:09:44.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.140+0000 7f4fd9400640 1 -- 172.21.15.45:0/1966779259 learned_addr learned my addr 172.21.15.45:0/1966779259 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:09:44.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.141+0000 7f4fd9400640 1 -- 172.21.15.45:0/1966779259 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b4f36a5e90 msgr2=0x55b4f36ac3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:09:44.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.141+0000 7f4fd9400640 1 --2- 172.21.15.45:0/1966779259 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b4f36a5e90 0x55b4f36ac3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:44.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.141+0000 7f4fd9400640 1 -- 172.21.15.45:0/1966779259 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b4f36af390 msgr2=0x55b4f36ace30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:09:44.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.141+0000 7f4fd9400640 1 --2- 172.21.15.45:0/1966779259 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b4f36af390 0x55b4f36ace30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:44.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.141+0000 7f4fd9400640 1 -- 172.21.15.45:0/1966779259 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b4f33e57e0 con 0x55b4f36a67b0 2024-04-16T17:09:44.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.141+0000 7f4fd9400640 1 --2- 172.21.15.45:0/1966779259 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b4f36a67b0 0x55b4f36ac8f0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f4fc4011440 tx=0x7f4fc4011900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71ccf2b64573493b server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:09:44.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.141+0000 7f4fb7400640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-16T17:09:44.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.141+0000 7f4fb7400640 1 -- 172.21.15.45:0/1966779259 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fc4005020 con 0x55b4f36a67b0 2024-04-16T17:09:44.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.141+0000 7f4fb7400640 1 -- 172.21.15.45:0/1966779259 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f4fc40051c0 con 0x55b4f36a67b0 2024-04-16T17:09:44.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.141+0000 7f4fdf57f740 1 -- 172.21.15.45:0/1966779259 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4fd0012450 con 0x55b4f36a67b0 2024-04-16T17:09:44.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.142+0000 7f4fb7400640 1 -- 172.21.15.45:0/1966779259 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f4fc4024050 con 0x55b4f36a67b0 2024-04-16T17:09:44.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.142+0000 7f4fdf57f740 10 client.16344 fetch_fsmap learned FSMap version 30 2024-04-16T17:09:44.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.142+0000 7f4fdf57f740 1 -- 172.21.15.45:0/1966779259 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55b4f3445f70 con 0x55b4f36a67b0 2024-04-16T17:09:44.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.142+0000 7f4fb7400640 1 -- 172.21.15.45:0/1966779259 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4fd0012450 con 0x55b4f36a67b0 2024-04-16T17:09:44.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.143+0000 7f4fb7400640 1 -- 172.21.15.45:0/1966779259 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f4fc40047a0 con 0x55b4f36a67b0 2024-04-16T17:09:44.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.143+0000 7f4fdf57f740 10 client.16344 fetch_fsmap finished waiting for FSMap version 30 2024-04-16T17:09:44.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.143+0000 7f4fdf57f740 10 client.16344 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:09:44.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.143+0000 7f4fdf57f740 20 client.16344 populate_metadata read hostname 'smithi045' 2024-04-16T17:09:44.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.143+0000 7f4fdf57f740 1 --2- 172.21.15.45:0/1966779259 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55b4f36bf820 0x55b4f36c1cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:44.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.143+0000 7f4fdf57f740 4 client.16344 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:09:44.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.143+0000 7f4fdf57f740 1 -- 172.21.15.45:0/1966779259 --> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b4f36186d0 con 0x55b4f36bf820 2024-04-16T17:09:44.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.148+0000 7f4fda800640 1 --2- 172.21.15.45:0/1966779259 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55b4f36bf820 0x55b4f36c1cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:44.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.149+0000 7f4fda800640 1 --2- 172.21.15.45:0/1966779259 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55b4f36bf820 0x55b4f36c1cc0 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dbd0cb39e25cdebb server_cookie=7398d74da05ca3b8 in_seq=0 out_seq=0 2024-04-16T17:09:44.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.149+0000 7f4fb7400640 10 client.16344 ms_handle_connect on v2:172.21.15.104:6838/3660070592 2024-04-16T17:09:44.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.150+0000 7f4fb7400640 1 -- 172.21.15.45:0/1966779259 <== mds.0 v2:172.21.15.104:6838/3660070592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b4f36186d0 con 0x55b4f36bf820 2024-04-16T17:09:44.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.150+0000 7f4fb7400640 10 client.16344 handle_command_reply: tid=0 2024-04-16T17:09:44.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.150+0000 7f4fdf57f740 1 client.16344 shutdown 2024-04-16T17:09:44.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.150+0000 7f4fdf57f740 1 -- 172.21.15.45:0/1966779259 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b4f36a67b0 msgr2=0x55b4f36ac8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:44.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.150+0000 7f4fdf57f740 1 --2- 172.21.15.45:0/1966779259 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b4f36a67b0 0x55b4f36ac8f0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f4fc4011440 tx=0x7f4fc4011900 comp rx=0 tx=0).stop 2024-04-16T17:09:44.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.150+0000 7f4fdf57f740 1 -- 172.21.15.45:0/1966779259 shutdown_connections 2024-04-16T17:09:44.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.150+0000 7f4fdf57f740 1 --2- 172.21.15.45:0/1966779259 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b4f36af390 0x55b4f36ace30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:44.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.150+0000 7f4fdf57f740 1 --2- 172.21.15.45:0/1966779259 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b4f36a67b0 0x55b4f36ac8f0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:44.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.150+0000 7f4fdf57f740 1 --2- 172.21.15.45:0/1966779259 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55b4f36bf820 0x55b4f36c1cc0 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:44.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.150+0000 7f4fdf57f740 1 --2- 172.21.15.45:0/1966779259 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b4f36a5e90 0x55b4f36ac3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:44.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.150+0000 7f4fdf57f740 1 -- 172.21.15.45:0/1966779259 >> 172.21.15.45:0/1966779259 conn(0x55b4f36a05c0 msgr2=0x55b4f36a2310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:09:44.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.150+0000 7f4fb7400640 0 client.16344 ms_handle_reset on v2:172.21.15.104:6838/3660070592 2024-04-16T17:09:44.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.150+0000 7f4fdf57f740 1 -- 172.21.15.45:0/1966779259 shutdown_connections 2024-04-16T17:09:44.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.150+0000 7f4fdf57f740 1 -- 172.21.15.45:0/1966779259 wait complete. 2024-04-16T17:09:44.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.150+0000 7f4fdf57f740 20 client.16344 trim_cache size 0 max 16384 2024-04-16T17:09:44.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.171+0000 7f4fdf57f740 1 Processor -- start 2024-04-16T17:09:44.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.171+0000 7f4fdf57f740 1 -- start start 2024-04-16T17:09:44.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.171+0000 7f4fdf57f740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b4f36a5e90 0x55b4f36aeae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:44.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.171+0000 7f4fdf57f740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b4f36af390 0x55b4f36bb070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:44.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.171+0000 7f4fdf57f740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b4f36c7590 0x55b4f36c79f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:44.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.171+0000 7f4fdf57f740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f4fd41b3e60 con 0x55b4f36c7590 2024-04-16T17:09:44.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.171+0000 7f4fdf57f740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f4fd41b1670 con 0x55b4f36af390 2024-04-16T17:09:44.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.171+0000 7f4fdf57f740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f4fd41af4a0 con 0x55b4f36a5e90 2024-04-16T17:09:44.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.171+0000 7f4fda800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b4f36c7590 0x55b4f36c79f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:44.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.171+0000 7f4fda800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b4f36c7590 0x55b4f36c79f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:48046/0 (socket says 172.21.15.45:48046) 2024-04-16T17:09:44.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.171+0000 7f4fda800640 1 -- 172.21.15.45:0/797887164 learned_addr learned my addr 172.21.15.45:0/797887164 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:09:44.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.171+0000 7f4fd9400640 1 --2- 172.21.15.45:0/797887164 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b4f36af390 0x55b4f36bb070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:44.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.171+0000 7f4fda800640 1 -- 172.21.15.45:0/797887164 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b4f36a5e90 msgr2=0x55b4f36aeae0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-16T17:09:44.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.171+0000 7f4fda800640 1 --2- 172.21.15.45:0/797887164 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b4f36a5e90 0x55b4f36aeae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:44.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.171+0000 7f4fda800640 1 -- 172.21.15.45:0/797887164 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b4f36af390 msgr2=0x55b4f36bb070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:09:44.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.171+0000 7f4fda800640 1 --2- 172.21.15.45:0/797887164 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b4f36af390 0x55b4f36bb070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:44.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.171+0000 7f4fda800640 1 -- 172.21.15.45:0/797887164 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b4f32dd170 con 0x55b4f36c7590 2024-04-16T17:09:44.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.171+0000 7f4fd9400640 1 --2- 172.21.15.45:0/797887164 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b4f36af390 0x55b4f36bb070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:09:44.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.171+0000 7f4fda800640 1 --2- 172.21.15.45:0/797887164 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b4f36c7590 0x55b4f36c79f0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f4fd00104d0 tx=0x7f4fd0010fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ee37cc4169b621c server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:09:44.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.171+0000 7f4fc2600640 1 -- 172.21.15.45:0/797887164 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fd00049a0 con 0x55b4f36c7590 2024-04-16T17:09:44.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.171+0000 7f4fc2600640 1 -- 172.21.15.45:0/797887164 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4fd0012930 con 0x55b4f36c7590 2024-04-16T17:09:44.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.171+0000 7f4fc2600640 1 -- 172.21.15.45:0/797887164 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fd000edb0 con 0x55b4f36c7590 2024-04-16T17:09:44.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.172+0000 7f4fdf57f740 1 -- 172.21.15.45:0/797887164 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b4f36c7590 msgr2=0x55b4f36c79f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:44.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.172+0000 7f4fdf57f740 1 --2- 172.21.15.45:0/797887164 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b4f36c7590 0x55b4f36c79f0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f4fd00104d0 tx=0x7f4fd0010fa0 comp rx=0 tx=0).stop 2024-04-16T17:09:44.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.172+0000 7f4fdf57f740 1 -- 172.21.15.45:0/797887164 shutdown_connections 2024-04-16T17:09:44.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.172+0000 7f4fdf57f740 1 --2- 172.21.15.45:0/797887164 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b4f36c7590 0x55b4f36c79f0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:44.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.172+0000 7f4fdf57f740 1 --2- 172.21.15.45:0/797887164 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b4f36af390 0x55b4f36bb070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:44.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.172+0000 7f4fdf57f740 1 --2- 172.21.15.45:0/797887164 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b4f36a5e90 0x55b4f36aeae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:44.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.172+0000 7f4fdf57f740 1 -- 172.21.15.45:0/797887164 >> 172.21.15.45:0/797887164 conn(0x55b4f36a05c0 msgr2=0x55b4f36acc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:09:44.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.172+0000 7f4fdf57f740 1 -- 172.21.15.45:0/797887164 shutdown_connections 2024-04-16T17:09:44.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.172+0000 7f4fdf57f740 1 -- 172.21.15.45:0/797887164 wait complete. 2024-04-16T17:09:44.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.173+0000 7f4fdf57f740 1 -- start start 2024-04-16T17:09:44.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.173+0000 7f4fdf57f740 1 Processor -- start 2024-04-16T17:09:44.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.173+0000 7f4fdf57f740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b4f36a5e90 0x55b4f36b7580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:44.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.173+0000 7f4fdf57f740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b4f36af390 0x55b4f36b7ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:44.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.173+0000 7f4fdf57f740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b4f36c7590 0x55b4f36b4550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:44.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.173+0000 7f4fdf57f740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f4fd41afb30 con 0x55b4f36af390 2024-04-16T17:09:44.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.173+0000 7f4fdf57f740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f4fd41b0fa0 con 0x55b4f36c7590 2024-04-16T17:09:44.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.173+0000 7f4fdf57f740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f4fd41b0200 con 0x55b4f36a5e90 2024-04-16T17:09:44.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.173+0000 7f4fdf57f740 10 client.0 _ll_register_callbacks cb 0x7f4fc4009c10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:09:44.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.173+0000 7f4fd9400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b4f36af390 0x55b4f36b7ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:44.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.174+0000 7f4fd9400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b4f36af390 0x55b4f36b7ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:48050/0 (socket says 172.21.15.45:48050) 2024-04-16T17:09:44.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.174+0000 7f4fd9400640 1 -- 172.21.15.45:0/1780378666 learned_addr learned my addr 172.21.15.45:0/1780378666 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:09:44.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.174+0000 7f4fd9400640 1 -- 172.21.15.45:0/1780378666 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b4f36a5e90 msgr2=0x55b4f36b7580 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-16T17:09:44.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.174+0000 7f4fd9400640 1 --2- 172.21.15.45:0/1780378666 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b4f36a5e90 0x55b4f36b7580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:44.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.174+0000 7f4fd9400640 1 -- 172.21.15.45:0/1780378666 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b4f36c7590 msgr2=0x55b4f36b4550 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-16T17:09:44.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.174+0000 7f4fd9400640 1 --2- 172.21.15.45:0/1780378666 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b4f36c7590 0x55b4f36b4550 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:44.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.174+0000 7f4fd9400640 1 -- 172.21.15.45:0/1780378666 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b4f36b4b90 con 0x55b4f36af390 2024-04-16T17:09:44.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.174+0000 7f4fd9400640 1 --2- 172.21.15.45:0/1780378666 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b4f36af390 0x55b4f36b7ac0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f4fc4011940 tx=0x7f4fc4011f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b41017b948cd8e03 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:09:44.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.174+0000 7f4fb7400640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-16T17:09:44.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.174+0000 7f4fb7400640 1 -- 172.21.15.45:0/1780378666 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fc4003960 con 0x55b4f36af390 2024-04-16T17:09:44.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.174+0000 7f4fb7400640 1 -- 172.21.15.45:0/1780378666 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f4fc4003b00 con 0x55b4f36af390 2024-04-16T17:09:44.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.174+0000 7f4fdf57f740 1 -- 172.21.15.45:0/1780378666 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4fd0012930 con 0x55b4f36af390 2024-04-16T17:09:44.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.175+0000 7f4fb7400640 1 -- 172.21.15.45:0/1780378666 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f4fc403b050 con 0x55b4f36af390 2024-04-16T17:09:44.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.175+0000 7f4fdf57f740 10 client.16350 fetch_fsmap learned FSMap version 30 2024-04-16T17:09:44.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.175+0000 7f4fdf57f740 1 -- 172.21.15.45:0/1780378666 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55b4f3726750 con 0x55b4f36af390 2024-04-16T17:09:44.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.175+0000 7f4fb7400640 1 -- 172.21.15.45:0/1780378666 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4fd0012930 con 0x55b4f36af390 2024-04-16T17:09:44.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.175+0000 7f4fb7400640 1 -- 172.21.15.45:0/1780378666 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f4fc400f480 con 0x55b4f36af390 2024-04-16T17:09:44.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.175+0000 7f4fdf57f740 10 client.16350 fetch_fsmap finished waiting for FSMap version 30 2024-04-16T17:09:44.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.175+0000 7f4fdf57f740 10 client.16350 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:09:44.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.175+0000 7f4fdf57f740 20 client.16350 populate_metadata read hostname 'smithi045' 2024-04-16T17:09:44.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.175+0000 7f4fdf57f740 1 --2- 172.21.15.45:0/1780378666 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55b4f3726ed0 0x55b4f37292f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:44.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.175+0000 7f4fdf57f740 4 client.16350 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-16T17:09:44.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.175+0000 7f4fdf57f740 1 -- 172.21.15.45:0/1780378666 --> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55b4f3612540 con 0x55b4f3726ed0 2024-04-16T17:09:44.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.177+0000 7f4fda800640 1 --2- 172.21.15.45:0/1780378666 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55b4f3726ed0 0x55b4f37292f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:44.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.178+0000 7f4fda800640 1 --2- 172.21.15.45:0/1780378666 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55b4f3726ed0 0x55b4f37292f0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d75cd11fbfdb40a6 server_cookie=e96d9f38122d6fc3 in_seq=0 out_seq=0 2024-04-16T17:09:44.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.178+0000 7f4fb7400640 10 client.16350 ms_handle_connect on v2:172.21.15.104:6838/3660070592 2024-04-16T17:09:44.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.179+0000 7f4fb7400640 1 -- 172.21.15.45:0/1780378666 <== mds.0 v2:172.21.15.104:6838/3660070592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55b4f3612540 con 0x55b4f3726ed0 2024-04-16T17:09:44.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.179+0000 7f4fb7400640 10 client.16350 handle_command_reply: tid=0 2024-04-16T17:09:44.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.179+0000 7f4fdf57f740 1 client.16350 shutdown 2024-04-16T17:09:44.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.180+0000 7f4fdf57f740 1 -- 172.21.15.45:0/1780378666 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b4f36af390 msgr2=0x55b4f36b7ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:44.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.180+0000 7f4fdf57f740 1 --2- 172.21.15.45:0/1780378666 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b4f36af390 0x55b4f36b7ac0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f4fc4011940 tx=0x7f4fc4011f90 comp rx=0 tx=0).stop 2024-04-16T17:09:44.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.180+0000 7f4fdf57f740 1 -- 172.21.15.45:0/1780378666 shutdown_connections 2024-04-16T17:09:44.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.180+0000 7f4fdf57f740 1 --2- 172.21.15.45:0/1780378666 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b4f36c7590 0x55b4f36b4550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:44.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.180+0000 7f4fdf57f740 1 --2- 172.21.15.45:0/1780378666 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b4f36af390 0x55b4f36b7ac0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:44.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.180+0000 7f4fdf57f740 1 --2- 172.21.15.45:0/1780378666 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55b4f3726ed0 0x55b4f37292f0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:44.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.180+0000 7f4fdf57f740 1 --2- 172.21.15.45:0/1780378666 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b4f36a5e90 0x55b4f36b7580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:44.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.180+0000 7f4fdf57f740 1 -- 172.21.15.45:0/1780378666 >> 172.21.15.45:0/1780378666 conn(0x55b4f36a05c0 msgr2=0x55b4f36c6580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:09:44.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.180+0000 7f4fb7400640 0 client.16350 ms_handle_reset on v2:172.21.15.104:6838/3660070592 2024-04-16T17:09:44.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.181+0000 7f4fdf57f740 1 -- 172.21.15.45:0/1780378666 shutdown_connections 2024-04-16T17:09:44.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.181+0000 7f4fdf57f740 1 -- 172.21.15.45:0/1780378666 wait complete. 2024-04-16T17:09:44.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.181+0000 7f4fdf57f740 20 client.16350 trim_cache size 0 max 16384 2024-04-16T17:09:44.182 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-16T17:09:44.182 INFO:teuthology.orchestra.run.smithi045.stdout: "return_code": 0, 2024-04-16T17:09:44.182 INFO:teuthology.orchestra.run.smithi045.stdout: "scrub_tag": "ca978396-84e5-4663-a7e8-011fcdcba200", 2024-04-16T17:09:44.183 INFO:teuthology.orchestra.run.smithi045.stdout: "mode": "asynchronous" 2024-04-16T17:09:44.183 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-16T17:09:44.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.182+0000 7f4fdb200640 1 -- 172.21.15.45:0/3854704071 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f4fbc0a3da0 msgr2=0x7f4fbc0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:44.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.182+0000 7f4fdb200640 1 --2- 172.21.15.45:0/3854704071 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f4fbc0a3da0 0x7f4fbc0a6260 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x55b4f36a5960 tx=0x7f4fc80f80e0 comp rx=0 tx=0).stop 2024-04-16T17:09:44.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.182+0000 7f4fdb200640 1 -- 172.21.15.45:0/3854704071 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4fd4104320 msgr2=0x7f4fd41a0660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:44.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.182+0000 7f4fdb200640 1 --2- 172.21.15.45:0/3854704071 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4fd4104320 0x7f4fd41a0660 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f4fc8006010 tx=0x7f4fc802cb30 comp rx=0 tx=0).stop 2024-04-16T17:09:44.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.182+0000 7f4fdb200640 1 -- 172.21.15.45:0/3854704071 shutdown_connections 2024-04-16T17:09:44.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.182+0000 7f4fdb200640 1 --2- 172.21.15.45:0/3854704071 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f4fbc0a3da0 0x7f4fbc0a6260 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:44.184 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.182+0000 7f4fdb200640 1 --2- 172.21.15.45:0/3854704071 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4fd41093f0 0x7f4fd41a0ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:44.184 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.182+0000 7f4fdb200640 1 --2- 172.21.15.45:0/3854704071 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4fd419a7d0 0x7f4fd419ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:44.184 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.182+0000 7f4fdb200640 1 --2- 172.21.15.45:0/3854704071 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4fd4104320 0x7f4fd41a0660 secure :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f4fc8006010 tx=0x7f4fc802cb30 comp rx=0 tx=0).stop 2024-04-16T17:09:44.184 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.182+0000 7f4fdb200640 1 -- 172.21.15.45:0/3854704071 >> 172.21.15.45:0/3854704071 conn(0x7f4fd40ffb40 msgr2=0x7f4fd4101ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:09:44.184 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.182+0000 7f4fdb200640 1 -- 172.21.15.45:0/3854704071 shutdown_connections 2024-04-16T17:09:44.184 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.182+0000 7f4fdb200640 1 -- 172.21.15.45:0/3854704071 wait complete. 2024-04-16T17:09:44.208 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-16T17:09:44.208 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-16T17:09:44.227 INFO:tasks.workunit.client.0.smithi045.stdout: 1 40218 2.20 MB/sec warmup 106 sec latency 1253.727 ms 2024-04-16T17:09:44.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.404+0000 7f18a1200640 1 -- 172.21.15.45:0/1522991243 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f189c1044b0 msgr2=0x7f189c104890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:44.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.404+0000 7f18a1200640 1 --2- 172.21.15.45:0/1522991243 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f189c1044b0 0x7f189c104890 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f188c009900 tx=0x7f188c02bc60 comp rx=0 tx=0).stop 2024-04-16T17:09:44.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.407+0000 7f18a1200640 1 -- 172.21.15.45:0/1522991243 shutdown_connections 2024-04-16T17:09:44.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.407+0000 7f18a1200640 1 --2- 172.21.15.45:0/1522991243 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f189c104e60 0x7f189c108ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:44.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.407+0000 7f18a1200640 1 --2- 172.21.15.45:0/1522991243 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f189c109580 0x7f189c111150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:44.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.407+0000 7f18a1200640 1 --2- 172.21.15.45:0/1522991243 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f189c1044b0 0x7f189c104890 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:44.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.407+0000 7f18a1200640 1 -- 172.21.15.45:0/1522991243 >> 172.21.15.45:0/1522991243 conn(0x7f189c0ffd90 msgr2=0x7f189c1021b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:09:44.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.407+0000 7f18a1200640 1 -- 172.21.15.45:0/1522991243 shutdown_connections 2024-04-16T17:09:44.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.407+0000 7f18a1200640 1 -- 172.21.15.45:0/1522991243 wait complete. 2024-04-16T17:09:44.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.407+0000 7f18a1200640 1 Processor -- start 2024-04-16T17:09:44.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.408+0000 7f18a1200640 1 -- start start 2024-04-16T17:09:44.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.408+0000 7f18a1200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f189c1044b0 0x7f189c1a08a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:44.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.408+0000 7f18a1200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f189c109580 0x7f189c1a0de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:44.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.408+0000 7f18a1200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f189c19aa50 0x7f189c19af00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:44.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.408+0000 7f18a1200640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f189c114690 con 0x7f189c1044b0 2024-04-16T17:09:44.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.408+0000 7f18a1200640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f189c114510 con 0x7f189c19aa50 2024-04-16T17:09:44.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.408+0000 7f18a1200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f189c114810 con 0x7f189c109580 2024-04-16T17:09:44.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.408+0000 7f1899e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f189c1044b0 0x7f189c1a08a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:44.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.408+0000 7f1899e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f189c1044b0 0x7f189c1a08a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:48068/0 (socket says 172.21.15.45:48068) 2024-04-16T17:09:44.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.408+0000 7f1899e00640 1 -- 172.21.15.45:0/3388711785 learned_addr learned my addr 172.21.15.45:0/3388711785 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:09:44.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.408+0000 7f1899400640 1 --2- 172.21.15.45:0/3388711785 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f189c109580 0x7f189c1a0de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:44.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.409+0000 7f189a800640 1 --2- 172.21.15.45:0/3388711785 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f189c19aa50 0x7f189c19af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:44.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.409+0000 7f1899e00640 1 -- 172.21.15.45:0/3388711785 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f189c109580 msgr2=0x7f189c1a0de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:44.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.409+0000 7f1899e00640 1 --2- 172.21.15.45:0/3388711785 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f189c109580 0x7f189c1a0de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:44.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.409+0000 7f1899e00640 1 -- 172.21.15.45:0/3388711785 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f189c19aa50 msgr2=0x7f189c19af00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:44.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.409+0000 7f1899e00640 1 --2- 172.21.15.45:0/3388711785 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f189c19aa50 0x7f189c19af00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:44.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.409+0000 7f1899e00640 1 -- 172.21.15.45:0/3388711785 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f189c19b570 con 0x7f189c1044b0 2024-04-16T17:09:44.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.409+0000 7f1899e00640 1 --2- 172.21.15.45:0/3388711785 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f189c1044b0 0x7f189c1a08a0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f188c006010 tx=0x7f188c02cb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:09:44.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.410+0000 7f1892a00640 1 -- 172.21.15.45:0/3388711785 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f188c042020 con 0x7f189c1044b0 2024-04-16T17:09:44.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.410+0000 7f1892a00640 1 -- 172.21.15.45:0/3388711785 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f188c004450 con 0x7f189c1044b0 2024-04-16T17:09:44.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.410+0000 7f18a1200640 1 -- 172.21.15.45:0/3388711785 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f189c19b800 con 0x7f189c1044b0 2024-04-16T17:09:44.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.410+0000 7f1892a00640 1 -- 172.21.15.45:0/3388711785 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f188c003d30 con 0x7f189c1044b0 2024-04-16T17:09:44.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.410+0000 7f18a1200640 1 -- 172.21.15.45:0/3388711785 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f189c1a7a50 con 0x7f189c1044b0 2024-04-16T17:09:44.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.414+0000 7f1892a00640 1 -- 172.21.15.45:0/3388711785 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f188c045020 con 0x7f189c1044b0 2024-04-16T17:09:44.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.415+0000 7f18a3fb2740 1 Processor -- start 2024-04-16T17:09:44.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.415+0000 7f18a3fb2740 1 -- start start 2024-04-16T17:09:44.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.415+0000 7f18a3fb2740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ad0cfd91b0 0x55ad0cfd9590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:44.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.415+0000 7f18a3fb2740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ad0cfd9ad0 0x55ad0cfe2170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:44.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.415+0000 7f18a3fb2740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ad0cfe26b0 0x55ad0cfe4aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:44.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.415+0000 7f18a3fb2740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55ad0cf3f120 con 0x55ad0cfd9ad0 2024-04-16T17:09:44.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.415+0000 7f18a3fb2740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55ad0cee5340 con 0x55ad0cfd91b0 2024-04-16T17:09:44.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.415+0000 7f18a3fb2740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55ad0cf3b0e0 con 0x55ad0cfe26b0 2024-04-16T17:09:44.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.416+0000 7f189a800640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ad0cfd91b0 0x55ad0cfd9590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:44.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.416+0000 7f189a800640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ad0cfd91b0 0x55ad0cfd9590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:47246/0 (socket says 172.21.15.45:47246) 2024-04-16T17:09:44.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.416+0000 7f1892a00640 1 --2- 172.21.15.45:0/3388711785 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f18800a3d20 0x7f18800a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:44.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.416+0000 7f189a800640 1 -- 172.21.15.45:0/641207374 learned_addr learned my addr 172.21.15.45:0/641207374 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:09:44.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.416+0000 7f1899400640 1 --2- 172.21.15.45:0/641207374 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ad0cfe26b0 0x55ad0cfe4aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:44.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.416+0000 7f1892a00640 1 -- 172.21.15.45:0/3388711785 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f188c0f2b50 con 0x7f189c1044b0 2024-04-16T17:09:44.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.416+0000 7f1899400640 1 -- 172.21.15.45:0/641207374 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ad0cfd91b0 msgr2=0x55ad0cfd9590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:09:44.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.416+0000 7f1899400640 1 --2- 172.21.15.45:0/641207374 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ad0cfd91b0 0x55ad0cfd9590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:44.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.416+0000 7f1899400640 1 -- 172.21.15.45:0/641207374 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ad0cfd9ad0 msgr2=0x55ad0cfe2170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:09:44.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.416+0000 7f1899400640 1 --2- 172.21.15.45:0/641207374 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ad0cfd9ad0 0x55ad0cfe2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:44.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.416+0000 7f1899400640 1 -- 172.21.15.45:0/641207374 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ad0cd1b7d0 con 0x55ad0cfe26b0 2024-04-16T17:09:44.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.416+0000 7f189a800640 1 --2- 172.21.15.45:0/641207374 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ad0cfd91b0 0x55ad0cfd9590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:09:44.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.416+0000 7f189a800640 1 --2- 172.21.15.45:0/3388711785 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f18800a3d20 0x7f18800a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:44.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.417+0000 7f1899400640 1 --2- 172.21.15.45:0/641207374 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ad0cfe26b0 0x55ad0cfe4aa0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f189c19c240 tx=0x7f1888009df0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bbafb5212a7d4de5 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:09:44.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.417+0000 7f1887400640 1 -- 172.21.15.45:0/641207374 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1888017070 con 0x55ad0cfe26b0 2024-04-16T17:09:44.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.417+0000 7f1887400640 1 -- 172.21.15.45:0/641207374 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1888013470 con 0x55ad0cfe26b0 2024-04-16T17:09:44.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.417+0000 7f1887400640 1 -- 172.21.15.45:0/641207374 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1888012610 con 0x55ad0cfe26b0 2024-04-16T17:09:44.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.418+0000 7f18a3fb2740 1 -- 172.21.15.45:0/641207374 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ad0cfe26b0 msgr2=0x55ad0cfe4aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:44.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.418+0000 7f18a3fb2740 1 --2- 172.21.15.45:0/641207374 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ad0cfe26b0 0x55ad0cfe4aa0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f189c19c240 tx=0x7f1888009df0 comp rx=0 tx=0).stop 2024-04-16T17:09:44.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.418+0000 7f189a800640 1 --2- 172.21.15.45:0/3388711785 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f18800a3d20 0x7f18800a61e0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f1894009fd0 tx=0x7f1894014040 comp rx=0 tx=0).ready entity=mgr.25411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:09:44.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.418+0000 7f18a3fb2740 1 -- 172.21.15.45:0/641207374 shutdown_connections 2024-04-16T17:09:44.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.418+0000 7f18a3fb2740 1 --2- 172.21.15.45:0/641207374 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ad0cfe26b0 0x55ad0cfe4aa0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:44.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.418+0000 7f18a3fb2740 1 --2- 172.21.15.45:0/641207374 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ad0cfd9ad0 0x55ad0cfe2170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:44.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.418+0000 7f18a3fb2740 1 --2- 172.21.15.45:0/641207374 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ad0cfd91b0 0x55ad0cfd9590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:44.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.418+0000 7f18a3fb2740 1 -- 172.21.15.45:0/641207374 >> 172.21.15.45:0/641207374 conn(0x55ad0cfd38e0 msgr2=0x55ad0cfd3cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:09:44.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.418+0000 7f18a3fb2740 1 -- 172.21.15.45:0/641207374 shutdown_connections 2024-04-16T17:09:44.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.418+0000 7f18a3fb2740 1 -- 172.21.15.45:0/641207374 wait complete. 2024-04-16T17:09:44.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.419+0000 7f18a3fb2740 1 -- start start 2024-04-16T17:09:44.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.419+0000 7f18a3fb2740 1 Processor -- start 2024-04-16T17:09:44.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.419+0000 7f18a3fb2740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ad0cfd91b0 0x55ad0cfdf720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:44.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.419+0000 7f18a3fb2740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ad0cfd9ad0 0x55ad0cfdfc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:44.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.419+0000 7f18a3fb2740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ad0cfe26b0 0x55ad0cfe01a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:44.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.420+0000 7f18a3fb2740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55ad0cfe54a0 con 0x55ad0cfd9ad0 2024-04-16T17:09:44.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.420+0000 7f18a3fb2740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55ad0cef8710 con 0x55ad0cfd91b0 2024-04-16T17:09:44.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.420+0000 7f18a3fb2740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55ad0cfe5620 con 0x55ad0cfe26b0 2024-04-16T17:09:44.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.420+0000 7f18a3fb2740 10 client.0 _ll_register_callbacks cb 0x55ad0cef3af0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:09:44.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.420+0000 7f1899e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ad0cfd91b0 0x55ad0cfdf720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:44.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.420+0000 7f1899e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ad0cfd91b0 0x55ad0cfdf720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:47254/0 (socket says 172.21.15.45:47254) 2024-04-16T17:09:44.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.420+0000 7f1899e00640 1 -- 172.21.15.45:0/1293283336 learned_addr learned my addr 172.21.15.45:0/1293283336 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:09:44.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.420+0000 7f189a800640 1 --2- 172.21.15.45:0/1293283336 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ad0cfe26b0 0x55ad0cfe01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:44.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.420+0000 7f1899400640 1 --2- 172.21.15.45:0/1293283336 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ad0cfd9ad0 0x55ad0cfdfc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:44.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.420+0000 7f1899e00640 1 -- 172.21.15.45:0/1293283336 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ad0cfe26b0 msgr2=0x55ad0cfe01a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:09:44.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.420+0000 7f1899e00640 1 --2- 172.21.15.45:0/1293283336 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ad0cfe26b0 0x55ad0cfe01a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:44.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.420+0000 7f1899e00640 1 -- 172.21.15.45:0/1293283336 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ad0cfd9ad0 msgr2=0x55ad0cfdfc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:09:44.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.420+0000 7f1899e00640 1 --2- 172.21.15.45:0/1293283336 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ad0cfd9ad0 0x55ad0cfdfc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:44.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.421+0000 7f1899e00640 1 -- 172.21.15.45:0/1293283336 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ad0cc513f0 con 0x55ad0cfd91b0 2024-04-16T17:09:44.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.421+0000 7f1899e00640 1 --2- 172.21.15.45:0/1293283336 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ad0cfd91b0 0x55ad0cfdf720 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f188c0be7e0 tx=0x7f188c0be620 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e6e71aa88175ad31 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:09:44.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.421+0000 7f187be00640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-04-16T17:09:44.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.421+0000 7f187be00640 1 -- 172.21.15.45:0/1293283336 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f188c0f0370 con 0x55ad0cfd91b0 2024-04-16T17:09:44.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.421+0000 7f187be00640 1 -- 172.21.15.45:0/1293283336 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f188c0c7030 con 0x55ad0cfd91b0 2024-04-16T17:09:44.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.421+0000 7f18a3fb2740 1 -- 172.21.15.45:0/1293283336 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1888013470 con 0x55ad0cfd91b0 2024-04-16T17:09:44.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.422+0000 7f187be00640 1 -- 172.21.15.45:0/1293283336 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f188c0035f0 con 0x55ad0cfd91b0 2024-04-16T17:09:44.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.422+0000 7f18a3fb2740 10 client.26507 fetch_fsmap learned FSMap version 30 2024-04-16T17:09:44.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.422+0000 7f18a3fb2740 1 -- 172.21.15.45:0/1293283336 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55ad0cc110f0 con 0x55ad0cfd91b0 2024-04-16T17:09:44.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.422+0000 7f187be00640 1 -- 172.21.15.45:0/1293283336 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1888013470 con 0x55ad0cfd91b0 2024-04-16T17:09:44.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.422+0000 7f187be00640 1 -- 172.21.15.45:0/1293283336 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f188c0d4020 con 0x55ad0cfd91b0 2024-04-16T17:09:44.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.422+0000 7f18a3fb2740 10 client.26507 fetch_fsmap finished waiting for FSMap version 30 2024-04-16T17:09:44.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.422+0000 7f18a3fb2740 10 client.26507 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:09:44.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.422+0000 7f18a3fb2740 20 client.26507 populate_metadata read hostname 'smithi045' 2024-04-16T17:09:44.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.422+0000 7f18a3fb2740 1 --2- 172.21.15.45:0/1293283336 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55ad0cff2960 0x55ad0cff4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:44.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.422+0000 7f18a3fb2740 4 client.26507 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:09:44.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.422+0000 7f18a3fb2740 1 -- 172.21.15.45:0/1293283336 --> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ad0cf4cb40 con 0x55ad0cff2960 2024-04-16T17:09:44.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.423+0000 7f1899400640 1 --2- 172.21.15.45:0/1293283336 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55ad0cff2960 0x55ad0cff4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:44.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.423+0000 7f1899400640 1 --2- 172.21.15.45:0/1293283336 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55ad0cff2960 0x55ad0cff4dc0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c380f0eefe934b1d server_cookie=da40fa7ad6f8c8eb in_seq=0 out_seq=0 2024-04-16T17:09:44.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.423+0000 7f187be00640 10 client.26507 ms_handle_connect on v2:172.21.15.104:6838/3660070592 2024-04-16T17:09:44.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.425+0000 7f187be00640 1 -- 172.21.15.45:0/1293283336 <== mds.0 v2:172.21.15.104:6838/3660070592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ad0cf4cb40 con 0x55ad0cff2960 2024-04-16T17:09:44.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.425+0000 7f187be00640 10 client.26507 handle_command_reply: tid=0 2024-04-16T17:09:44.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.425+0000 7f18a3fb2740 1 client.26507 shutdown 2024-04-16T17:09:44.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.425+0000 7f18a3fb2740 1 -- 172.21.15.45:0/1293283336 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ad0cfd91b0 msgr2=0x55ad0cfdf720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:44.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.425+0000 7f18a3fb2740 1 --2- 172.21.15.45:0/1293283336 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ad0cfd91b0 0x55ad0cfdf720 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f188c0be7e0 tx=0x7f188c0be620 comp rx=0 tx=0).stop 2024-04-16T17:09:44.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.425+0000 7f18a3fb2740 1 -- 172.21.15.45:0/1293283336 shutdown_connections 2024-04-16T17:09:44.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.425+0000 7f18a3fb2740 1 --2- 172.21.15.45:0/1293283336 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55ad0cff2960 0x55ad0cff4dc0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:44.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.425+0000 7f18a3fb2740 1 --2- 172.21.15.45:0/1293283336 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ad0cfe26b0 0x55ad0cfe01a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:44.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.425+0000 7f18a3fb2740 1 --2- 172.21.15.45:0/1293283336 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ad0cfd9ad0 0x55ad0cfdfc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:44.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.425+0000 7f18a3fb2740 1 --2- 172.21.15.45:0/1293283336 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ad0cfd91b0 0x55ad0cfdf720 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:44.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.425+0000 7f18a3fb2740 1 -- 172.21.15.45:0/1293283336 >> 172.21.15.45:0/1293283336 conn(0x55ad0cfd38e0 msgr2=0x55ad0cfd5630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:09:44.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.425+0000 7f187be00640 0 client.26507 ms_handle_reset on v2:172.21.15.104:6838/3660070592 2024-04-16T17:09:44.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.425+0000 7f18a3fb2740 1 -- 172.21.15.45:0/1293283336 shutdown_connections 2024-04-16T17:09:44.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.425+0000 7f18a3fb2740 1 -- 172.21.15.45:0/1293283336 wait complete. 2024-04-16T17:09:44.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.426+0000 7f18a3fb2740 20 client.26507 trim_cache size 0 max 16384 2024-04-16T17:09:44.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.447+0000 7f18a3fb2740 1 Processor -- start 2024-04-16T17:09:44.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.447+0000 7f18a3fb2740 1 -- start start 2024-04-16T17:09:44.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.447+0000 7f18a3fb2740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ad0cfd91b0 0x55ad0cfe1e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:44.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.447+0000 7f18a3fb2740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ad0cfd9ad0 0x55ad0cfee360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:44.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.447+0000 7f18a3fb2740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ad0cfe26b0 0x55ad0cfd2750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:44.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.447+0000 7f18a3fb2740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f189c1b3f40 con 0x55ad0cfe26b0 2024-04-16T17:09:44.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.447+0000 7f18a3fb2740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f189c1b1750 con 0x55ad0cfd91b0 2024-04-16T17:09:44.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.447+0000 7f18a3fb2740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f189c1af580 con 0x55ad0cfd9ad0 2024-04-16T17:09:44.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.447+0000 7f1899e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ad0cfd91b0 0x55ad0cfe1e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:44.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.447+0000 7f1899e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ad0cfd91b0 0x55ad0cfe1e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:47260/0 (socket says 172.21.15.45:47260) 2024-04-16T17:09:44.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.447+0000 7f1899e00640 1 -- 172.21.15.45:0/2216508280 learned_addr learned my addr 172.21.15.45:0/2216508280 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:09:44.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.447+0000 7f1899400640 1 --2- 172.21.15.45:0/2216508280 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ad0cfd9ad0 0x55ad0cfee360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:44.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.447+0000 7f189a800640 1 --2- 172.21.15.45:0/2216508280 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ad0cfe26b0 0x55ad0cfd2750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:44.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.447+0000 7f1899e00640 1 -- 172.21.15.45:0/2216508280 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ad0cfd9ad0 msgr2=0x55ad0cfee360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:09:44.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.447+0000 7f1899e00640 1 --2- 172.21.15.45:0/2216508280 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ad0cfd9ad0 0x55ad0cfee360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:44.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.447+0000 7f1899e00640 1 -- 172.21.15.45:0/2216508280 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ad0cfe26b0 msgr2=0x55ad0cfd2750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:09:44.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.447+0000 7f1899e00640 1 --2- 172.21.15.45:0/2216508280 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ad0cfe26b0 0x55ad0cfd2750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:44.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.448+0000 7f1899e00640 1 -- 172.21.15.45:0/2216508280 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ad0cfcf1a0 con 0x55ad0cfd91b0 2024-04-16T17:09:44.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.448+0000 7f189a800640 1 --2- 172.21.15.45:0/2216508280 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ad0cfe26b0 0x55ad0cfd2750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:09:44.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.448+0000 7f1899e00640 1 --2- 172.21.15.45:0/2216508280 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ad0cfd91b0 0x55ad0cfe1e50 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f188c005940 tx=0x7f188c0057e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=be7ef45a07c0874 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:09:44.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.448+0000 7f1886a00640 1 -- 172.21.15.45:0/2216508280 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f188c005570 con 0x55ad0cfd91b0 2024-04-16T17:09:44.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.448+0000 7f1886a00640 1 -- 172.21.15.45:0/2216508280 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f188c0bef50 con 0x55ad0cfd91b0 2024-04-16T17:09:44.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.448+0000 7f1886a00640 1 -- 172.21.15.45:0/2216508280 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f188c0c7030 con 0x55ad0cfd91b0 2024-04-16T17:09:44.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.448+0000 7f18a3fb2740 1 -- 172.21.15.45:0/2216508280 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ad0cfd91b0 msgr2=0x55ad0cfe1e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:44.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.448+0000 7f18a3fb2740 1 --2- 172.21.15.45:0/2216508280 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ad0cfd91b0 0x55ad0cfe1e50 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f188c005940 tx=0x7f188c0057e0 comp rx=0 tx=0).stop 2024-04-16T17:09:44.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.448+0000 7f18a3fb2740 1 -- 172.21.15.45:0/2216508280 shutdown_connections 2024-04-16T17:09:44.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.448+0000 7f18a3fb2740 1 --2- 172.21.15.45:0/2216508280 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ad0cfd9ad0 0x55ad0cfee360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:44.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.448+0000 7f18a3fb2740 1 --2- 172.21.15.45:0/2216508280 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ad0cfe26b0 0x55ad0cfd2750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:44.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.448+0000 7f18a3fb2740 1 --2- 172.21.15.45:0/2216508280 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ad0cfd91b0 0x55ad0cfe1e50 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:44.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.448+0000 7f18a3fb2740 1 -- 172.21.15.45:0/2216508280 >> 172.21.15.45:0/2216508280 conn(0x55ad0cfd38e0 msgr2=0x55ad0cfd2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:09:44.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.448+0000 7f18a3fb2740 1 -- 172.21.15.45:0/2216508280 shutdown_connections 2024-04-16T17:09:44.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.448+0000 7f18a3fb2740 1 -- 172.21.15.45:0/2216508280 wait complete. 2024-04-16T17:09:44.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.448+0000 7f18a3fb2740 1 -- start start 2024-04-16T17:09:44.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.448+0000 7f18a3fb2740 1 Processor -- start 2024-04-16T17:09:44.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.448+0000 7f18a3fb2740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ad0cfd91b0 0x55ad0cfeac00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:44.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.448+0000 7f18a3fb2740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ad0cfd9ad0 0x55ad0cfe5f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:44.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.448+0000 7f18a3fb2740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ad0cfe26b0 0x55ad0cfe6460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:44.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.448+0000 7f18a3fb2740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f189c1afc10 con 0x55ad0cfd9ad0 2024-04-16T17:09:44.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.448+0000 7f18a3fb2740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f189c1b1080 con 0x55ad0cfe26b0 2024-04-16T17:09:44.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.448+0000 7f18a3fb2740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f189c1b02e0 con 0x55ad0cfd91b0 2024-04-16T17:09:44.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.448+0000 7f18a3fb2740 10 client.0 _ll_register_callbacks cb 0x7f188c0ba3d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:09:44.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.448+0000 7f1899400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ad0cfd9ad0 0x55ad0cfe5f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:44.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.448+0000 7f1899e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ad0cfd91b0 0x55ad0cfeac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:44.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.448+0000 7f1899400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ad0cfd9ad0 0x55ad0cfe5f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:48094/0 (socket says 172.21.15.45:48094) 2024-04-16T17:09:44.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.448+0000 7f1899400640 1 -- 172.21.15.45:0/1553230021 learned_addr learned my addr 172.21.15.45:0/1553230021 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:09:44.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.448+0000 7f189a800640 1 --2- 172.21.15.45:0/1553230021 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ad0cfe26b0 0x55ad0cfe6460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:44.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.448+0000 7f189a800640 1 -- 172.21.15.45:0/1553230021 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ad0cfd91b0 msgr2=0x55ad0cfeac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:09:44.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.448+0000 7f189a800640 1 --2- 172.21.15.45:0/1553230021 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ad0cfd91b0 0x55ad0cfeac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:44.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.448+0000 7f189a800640 1 -- 172.21.15.45:0/1553230021 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ad0cfd9ad0 msgr2=0x55ad0cfe5f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:09:44.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.448+0000 7f189a800640 1 --2- 172.21.15.45:0/1553230021 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ad0cfd9ad0 0x55ad0cfe5f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:44.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.448+0000 7f189a800640 1 -- 172.21.15.45:0/1553230021 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ad0cfe6c40 con 0x55ad0cfe26b0 2024-04-16T17:09:44.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.448+0000 7f189a800640 1 --2- 172.21.15.45:0/1553230021 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ad0cfe26b0 0x55ad0cfe6460 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f1894005fd0 tx=0x7f1894009500 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9846ca8b3e1e5691 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:09:44.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.448+0000 7f187be00640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-04-16T17:09:44.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.448+0000 7f187be00640 1 -- 172.21.15.45:0/1553230021 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1894010900 con 0x55ad0cfe26b0 2024-04-16T17:09:44.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.448+0000 7f187be00640 1 -- 172.21.15.45:0/1553230021 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f1894010aa0 con 0x55ad0cfe26b0 2024-04-16T17:09:44.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.448+0000 7f18a3fb2740 1 -- 172.21.15.45:0/1553230021 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f188c0bef50 con 0x55ad0cfe26b0 2024-04-16T17:09:44.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.448+0000 7f187be00640 1 -- 172.21.15.45:0/1553230021 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f1894010ca0 con 0x55ad0cfe26b0 2024-04-16T17:09:44.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.448+0000 7f18a3fb2740 10 client.26513 fetch_fsmap learned FSMap version 30 2024-04-16T17:09:44.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.448+0000 7f18a3fb2740 1 -- 172.21.15.45:0/1553230021 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55ad0d058ed0 con 0x55ad0cfe26b0 2024-04-16T17:09:44.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.448+0000 7f187be00640 1 -- 172.21.15.45:0/1553230021 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f188c0bef50 con 0x55ad0cfe26b0 2024-04-16T17:09:44.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.448+0000 7f187be00640 1 -- 172.21.15.45:0/1553230021 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f189401b400 con 0x55ad0cfe26b0 2024-04-16T17:09:44.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.448+0000 7f18a3fb2740 10 client.26513 fetch_fsmap finished waiting for FSMap version 30 2024-04-16T17:09:44.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.448+0000 7f18a3fb2740 10 client.26513 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:09:44.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.448+0000 7f18a3fb2740 20 client.26513 populate_metadata read hostname 'smithi045' 2024-04-16T17:09:44.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.448+0000 7f18a3fb2740 1 --2- 172.21.15.45:0/1553230021 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55ad0d059680 0x55ad0d05baa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:44.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.448+0000 7f18a3fb2740 4 client.26513 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-16T17:09:44.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.448+0000 7f18a3fb2740 1 -- 172.21.15.45:0/1553230021 --> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55ad0cf469b0 con 0x55ad0d059680 2024-04-16T17:09:44.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.453+0000 7f1899e00640 1 --2- 172.21.15.45:0/1553230021 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55ad0d059680 0x55ad0d05baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:44.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.453+0000 7f1899e00640 1 --2- 172.21.15.45:0/1553230021 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55ad0d059680 0x55ad0d05baa0 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1533ea3f31e08c42 server_cookie=aed7ffd6125d3665 in_seq=0 out_seq=0 2024-04-16T17:09:44.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.453+0000 7f187be00640 10 client.26513 ms_handle_connect on v2:172.21.15.104:6838/3660070592 2024-04-16T17:09:44.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.453+0000 7f187be00640 1 -- 172.21.15.45:0/1553230021 <== mds.0 v2:172.21.15.104:6838/3660070592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55ad0cf469b0 con 0x55ad0d059680 2024-04-16T17:09:44.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.453+0000 7f187be00640 10 client.26513 handle_command_reply: tid=0 2024-04-16T17:09:44.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.453+0000 7f18a3fb2740 1 client.26513 shutdown 2024-04-16T17:09:44.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.453+0000 7f18a3fb2740 1 -- 172.21.15.45:0/1553230021 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ad0cfe26b0 msgr2=0x55ad0cfe6460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:44.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.453+0000 7f18a3fb2740 1 --2- 172.21.15.45:0/1553230021 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ad0cfe26b0 0x55ad0cfe6460 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f1894005fd0 tx=0x7f1894009500 comp rx=0 tx=0).stop 2024-04-16T17:09:44.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.453+0000 7f18a3fb2740 1 -- 172.21.15.45:0/1553230021 shutdown_connections 2024-04-16T17:09:44.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.453+0000 7f18a3fb2740 1 --2- 172.21.15.45:0/1553230021 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ad0cfe26b0 0x55ad0cfe6460 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:44.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.453+0000 7f18a3fb2740 1 --2- 172.21.15.45:0/1553230021 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ad0cfd9ad0 0x55ad0cfe5f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:44.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.453+0000 7f18a3fb2740 1 --2- 172.21.15.45:0/1553230021 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55ad0d059680 0x55ad0d05baa0 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:44.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.453+0000 7f18a3fb2740 1 --2- 172.21.15.45:0/1553230021 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ad0cfd91b0 0x55ad0cfeac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:44.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.453+0000 7f187be00640 0 client.26513 ms_handle_reset on v2:172.21.15.104:6838/3660070592 2024-04-16T17:09:44.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.453+0000 7f18a3fb2740 1 -- 172.21.15.45:0/1553230021 >> 172.21.15.45:0/1553230021 conn(0x55ad0cfd38e0 msgr2=0x55ad0cfdff50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:09:44.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.453+0000 7f18a3fb2740 1 -- 172.21.15.45:0/1553230021 shutdown_connections 2024-04-16T17:09:44.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.453+0000 7f18a3fb2740 1 -- 172.21.15.45:0/1553230021 wait complete. 2024-04-16T17:09:44.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.453+0000 7f18a3fb2740 20 client.26513 trim_cache size 0 max 16384 2024-04-16T17:09:44.457 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-16T17:09:44.458 INFO:teuthology.orchestra.run.smithi045.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-16T17:09:44.458 INFO:teuthology.orchestra.run.smithi045.stdout: "scrubs": { 2024-04-16T17:09:44.458 INFO:teuthology.orchestra.run.smithi045.stdout: "ca978396-84e5-4663-a7e8-011fcdcba200": { 2024-04-16T17:09:44.458 INFO:teuthology.orchestra.run.smithi045.stdout: "path": "/", 2024-04-16T17:09:44.458 INFO:teuthology.orchestra.run.smithi045.stdout: "tag": "ca978396-84e5-4663-a7e8-011fcdcba200", 2024-04-16T17:09:44.458 INFO:teuthology.orchestra.run.smithi045.stdout: "options": "recursive,force" 2024-04-16T17:09:44.458 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:09:44.458 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:09:44.458 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-16T17:09:44.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.453+0000 7f18a1200640 1 -- 172.21.15.45:0/3388711785 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f18800a3d20 msgr2=0x7f18800a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:44.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.453+0000 7f18a1200640 1 --2- 172.21.15.45:0/3388711785 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f18800a3d20 0x7f18800a61e0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f1894009fd0 tx=0x7f1894014040 comp rx=0 tx=0).stop 2024-04-16T17:09:44.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.453+0000 7f18a1200640 1 -- 172.21.15.45:0/3388711785 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f189c1044b0 msgr2=0x7f189c1a08a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:44.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.453+0000 7f18a1200640 1 --2- 172.21.15.45:0/3388711785 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f189c1044b0 0x7f189c1a08a0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f188c006010 tx=0x7f188c02cb60 comp rx=0 tx=0).stop 2024-04-16T17:09:44.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.456+0000 7f18a1200640 1 -- 172.21.15.45:0/3388711785 shutdown_connections 2024-04-16T17:09:44.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.456+0000 7f18a1200640 1 --2- 172.21.15.45:0/3388711785 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f18800a3d20 0x7f18800a61e0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:44.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.456+0000 7f18a1200640 1 --2- 172.21.15.45:0/3388711785 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f189c109580 0x7f189c1a0de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:44.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.456+0000 7f18a1200640 1 --2- 172.21.15.45:0/3388711785 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f189c19aa50 0x7f189c19af00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:44.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.456+0000 7f18a1200640 1 --2- 172.21.15.45:0/3388711785 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f189c1044b0 0x7f189c1a08a0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:44.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.456+0000 7f18a1200640 1 -- 172.21.15.45:0/3388711785 >> 172.21.15.45:0/3388711785 conn(0x7f189c0ffd90 msgr2=0x7f189c101110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:09:44.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.456+0000 7f18a1200640 1 -- 172.21.15.45:0/3388711785 shutdown_connections 2024-04-16T17:09:44.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:09:44.456+0000 7f18a1200640 1 -- 172.21.15.45:0/3388711785 wait complete. 2024-04-16T17:09:44.473 INFO:tasks.cephfs.filesystem:scrub status for tag:ca978396-84e5-4663-a7e8-011fcdcba200 - {'path': '/', 'tag': 'ca978396-84e5-4663-a7e8-011fcdcba200', 'options': 'recursive,force'} 2024-04-16T17:09:44.479 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:09:44 smithi104 ceph-mon[36457]: pgmap v92: 129 pgs: 129 active+clean; 31 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 46 KiB/s wr, 19 op/s 2024-04-16T17:09:44.479 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:09:44 smithi104 ceph-mon[36457]: scrub summary: active paths [/] 2024-04-16T17:09:44.479 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:09:44 smithi104 ceph-mon[36457]: scrub summary: idle+waiting paths [/] 2024-04-16T17:09:44.480 INFO:journalctl@ceph.mds.b.smithi104.stdout:Apr 16 17:09:44 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mds-b[67798]: 2024-04-16T17:09:44.193+0000 7fde259be700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000225 [...5,head] /volumes/qa/sv_1/584ee6b0-43a5-48e1-b843-9914b8e65254/client.0/tmp/clients/client0/~dmtmp/PM/ auth{2=1} v7847 ap=2 f(v190 m2024-04-16T17:09:22.892847+0000 3=3+0) n(v7 rc2024-04-16T17:09:22.892847+0000 b780288 4=3+1) old_inodes=2 (inest sync->mix g=2) caps={25436=pAsLsXsFs/pAsLsXsFsx@1457} mcw={2=AsLsXsFs} | dirtyscattered=0 request=0 lock=0 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 0x55c7331ba100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x10000000225:[<0x10000000009/PM v66>,<0x10000000008/~dmtmp v34>,<0x10000000007/client0 v20>,<0x10000000006/clients v10>,<0x10000000005/tmp v10>,<0x100000001fb/client.0 v14>,<0x100000001fa/584ee6b0-43a5-48e1-b843-9914b8e65254 v30>,<0x10000000001/sv_1 v53>,<0x10000000000/qa v41>,<0x1/volumes v39>]//[]","memoryvalue":"(2)0x10000000225:[<0x10000000009/PM v7847>,<0x10000000008/~dmtmp v1162>,<0x10000000007/client0 v224>,<0x10000000006/clients v192>,<0x10000000005/tmp v192>,<0x100000001fb/client.0 v191>,<0x100000001fa/584ee6b0-43a5-48e1-b843-9914b8e65254 v201>,<0x10000000001/sv_1 v211>,<0x10000000000/qa v195>,<0x1/volumes v192>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-04-16T17:09:22.712850+0000 4=4+0)","ondisk_value.rstat":"n(v0 rc2024-04-16T17:09:22.892847+0000 b780288 4=3+1)","memory_value.dirstat":"f(v190 m2024-04-16T17:09:22.892847+0000 3=3+0)","memory_value.rstat":"n(v7 rc2024-04-16T17:09:22.892847+0000 b780288 4=3+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-04-16T17:09:44.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:09:44 smithi117 ceph-mon[36730]: pgmap v92: 129 pgs: 129 active+clean; 31 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 46 KiB/s wr, 19 op/s 2024-04-16T17:09:44.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:09:44 smithi117 ceph-mon[36730]: scrub summary: active paths [/] 2024-04-16T17:09:44.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:09:44 smithi117 ceph-mon[36730]: scrub summary: idle+waiting paths [/] 2024-04-16T17:09:45.227 INFO:tasks.workunit.client.0.smithi045.stdout: 1 40565 2.21 MB/sec warmup 107 sec latency 46.744 ms 2024-04-16T17:09:45.677 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:09:45 smithi104 ceph-mon[36457]: scrub queued for path: / 2024-04-16T17:09:45.677 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:09:45 smithi104 ceph-mon[36457]: scrub summary: idle+waiting paths [/] 2024-04-16T17:09:45.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:09:45 smithi104 ceph-mon[36457]: scrub summary: active paths [/] 2024-04-16T17:09:45.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:09:45 smithi104 ceph-mon[36457]: scrub summary: active paths [/] 2024-04-16T17:09:45.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:09:45 smithi104 ceph-mon[36457]: Scrub error on inode 0x10000000225 (/volumes/qa/sv_1/584ee6b0-43a5-48e1-b843-9914b8e65254/client.0/tmp/clients/client0/~dmtmp/PM) see mds.b log and `damage ls` output for details 2024-04-16T17:09:45.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:09:45 smithi104 ceph-mon[36457]: scrub summary: idle+waiting paths [/] 2024-04-16T17:09:45.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:09:45 smithi104 ceph-mon[36457]: scrub summary: idle+waiting paths [/] 2024-04-16T17:09:45.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:09:45 smithi117 ceph-mon[36730]: scrub queued for path: / 2024-04-16T17:09:45.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:09:45 smithi117 ceph-mon[36730]: scrub summary: idle+waiting paths [/] 2024-04-16T17:09:45.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:09:45 smithi117 ceph-mon[36730]: scrub summary: active paths [/] 2024-04-16T17:09:45.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:09:45 smithi117 ceph-mon[36730]: scrub summary: active paths [/] 2024-04-16T17:09:45.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:09:45 smithi117 ceph-mon[36730]: Scrub error on inode 0x10000000225 (/volumes/qa/sv_1/584ee6b0-43a5-48e1-b843-9914b8e65254/client.0/tmp/clients/client0/~dmtmp/PM) see mds.b log and `damage ls` output for details 2024-04-16T17:09:45.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:09:45 smithi117 ceph-mon[36730]: scrub summary: idle+waiting paths [/] 2024-04-16T17:09:45.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:09:45 smithi117 ceph-mon[36730]: scrub summary: idle+waiting paths [/] 2024-04-16T17:09:46.227 INFO:tasks.workunit.client.0.smithi045.stdout: 1 41425 2.21 MB/sec warmup 108 sec latency 45.950 ms 2024-04-16T17:09:46.677 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:09:46 smithi104 ceph-mon[36457]: pgmap v93: 129 pgs: 129 active+clean; 31 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 46 KiB/s wr, 19 op/s 2024-04-16T17:09:46.804 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:09:46 smithi117 ceph-mon[36730]: pgmap v93: 129 pgs: 129 active+clean; 31 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 46 KiB/s wr, 19 op/s 2024-04-16T17:09:47.227 INFO:tasks.workunit.client.0.smithi045.stdout: 1 42564 2.28 MB/sec warmup 109 sec latency 30.807 ms 2024-04-16T17:09:47.478 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:09:44 smithi045 ceph-mon[31676]: pgmap v92: 129 pgs: 129 active+clean; 31 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 46 KiB/s wr, 19 op/s 2024-04-16T17:09:47.478 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:09:44 smithi045 ceph-mon[31676]: scrub summary: active paths [/] 2024-04-16T17:09:47.478 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:09:44 smithi045 ceph-mon[31676]: scrub summary: idle+waiting paths [/] 2024-04-16T17:09:47.478 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:09:45 smithi045 ceph-mon[31676]: scrub queued for path: / 2024-04-16T17:09:47.478 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:09:45 smithi045 ceph-mon[31676]: scrub summary: idle+waiting paths [/] 2024-04-16T17:09:47.478 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:09:45 smithi045 ceph-mon[31676]: scrub summary: active paths [/] 2024-04-16T17:09:47.478 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:09:45 smithi045 ceph-mon[31676]: scrub summary: active paths [/] 2024-04-16T17:09:47.478 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:09:45 smithi045 ceph-mon[31676]: Scrub error on inode 0x10000000225 (/volumes/qa/sv_1/584ee6b0-43a5-48e1-b843-9914b8e65254/client.0/tmp/clients/client0/~dmtmp/PM) see mds.b log and `damage ls` output for details 2024-04-16T17:09:47.478 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:09:45 smithi045 ceph-mon[31676]: scrub summary: idle+waiting paths [/] 2024-04-16T17:09:47.478 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:09:45 smithi045 ceph-mon[31676]: scrub summary: idle+waiting paths [/] 2024-04-16T17:09:47.479 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:09:46 smithi045 ceph-mon[31676]: pgmap v93: 129 pgs: 129 active+clean; 31 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 46 KiB/s wr, 19 op/s 2024-04-16T17:09:48.227 INFO:tasks.workunit.client.0.smithi045.stdout: 1 42582 2.26 MB/sec warmup 110 sec latency 972.573 ms 2024-04-16T17:09:48.677 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:09:48 smithi104 ceph-mon[36457]: pgmap v94: 129 pgs: 129 active+clean; 31 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 359 KiB/s rd, 436 KiB/s wr, 56 op/s 2024-04-16T17:09:48.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:09:48 smithi045 ceph-mon[31676]: pgmap v94: 129 pgs: 129 active+clean; 31 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 359 KiB/s rd, 436 KiB/s wr, 56 op/s 2024-04-16T17:09:48.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:09:48 smithi117 ceph-mon[36730]: pgmap v94: 129 pgs: 129 active+clean; 31 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 359 KiB/s rd, 436 KiB/s wr, 56 op/s 2024-04-16T17:09:49.119 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:49.119+0000 7fd5f5800640 1 -- 172.21.15.45:0/1563618752 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd5f0107160 msgr2=0x7fd5f010ed30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:49.119 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:49.119+0000 7fd5f5800640 1 --2- 172.21.15.45:0/1563618752 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd5f0107160 0x7fd5f010ed30 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fd5e8009ca0 tx=0x7fd5e802c0f0 comp rx=0 tx=0).stop 2024-04-16T17:09:49.119 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:49.119+0000 7fd5f5800640 1 -- 172.21.15.45:0/1563618752 shutdown_connections 2024-04-16T17:09:49.119 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:49.119+0000 7fd5f5800640 1 --2- 172.21.15.45:0/1563618752 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd5f0102a40 0x7fd5f0106a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:49.119 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:49.119+0000 7fd5f5800640 1 --2- 172.21.15.45:0/1563618752 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd5f0107160 0x7fd5f010ed30 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:49.119 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:49.119+0000 7fd5f5800640 1 --2- 172.21.15.45:0/1563618752 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd5f0102090 0x7fd5f0102470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:49.120 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:49.119+0000 7fd5f5800640 1 -- 172.21.15.45:0/1563618752 >> 172.21.15.45:0/1563618752 conn(0x7fd5f00fd950 msgr2=0x7fd5f00ffd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:09:49.120 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:49.119+0000 7fd5f5800640 1 -- 172.21.15.45:0/1563618752 shutdown_connections 2024-04-16T17:09:49.120 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:49.119+0000 7fd5f5800640 1 -- 172.21.15.45:0/1563618752 wait complete. 2024-04-16T17:09:49.121 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:49.120+0000 7fd5f5800640 1 Processor -- start 2024-04-16T17:09:49.121 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:49.120+0000 7fd5f5800640 1 -- start start 2024-04-16T17:09:49.121 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:49.121+0000 7fd5f5800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd5f0102090 0x7fd5f01a0670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:49.121 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:49.121+0000 7fd5f5800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd5f0107160 0x7fd5f01a0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:49.121 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:49.121+0000 7fd5f5800640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd5f019a780 0x7fd5f019ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:49.121 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:49.121+0000 7fd5f5800640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fd5f0112220 con 0x7fd5f0102090 2024-04-16T17:09:49.121 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:49.121+0000 7fd5f5800640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fd5f01120a0 con 0x7fd5f019a780 2024-04-16T17:09:49.121 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:49.121+0000 7fd5f5800640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fd5f01123a0 con 0x7fd5f0107160 2024-04-16T17:09:49.121 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:49.121+0000 7fd5efe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd5f0102090 0x7fd5f01a0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:49.121 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:49.121+0000 7fd5efe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd5f0102090 0x7fd5f01a0670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:48118/0 (socket says 172.21.15.45:48118) 2024-04-16T17:09:49.121 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:49.121+0000 7fd5efe00640 1 -- 172.21.15.45:0/2923850149 learned_addr learned my addr 172.21.15.45:0/2923850149 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:09:49.122 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:49.121+0000 7fd5ef400640 1 --2- 172.21.15.45:0/2923850149 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd5f0107160 0x7fd5f01a0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:49.122 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:49.121+0000 7fd5f4e00640 1 --2- 172.21.15.45:0/2923850149 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd5f019a780 0x7fd5f019ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:49.122 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:49.122+0000 7fd5efe00640 1 -- 172.21.15.45:0/2923850149 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd5f0107160 msgr2=0x7fd5f01a0bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:49.123 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:49.122+0000 7fd5efe00640 1 --2- 172.21.15.45:0/2923850149 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd5f0107160 0x7fd5f01a0bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:49.123 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:49.122+0000 7fd5efe00640 1 -- 172.21.15.45:0/2923850149 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd5f019a780 msgr2=0x7fd5f019ac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:49.123 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:49.122+0000 7fd5efe00640 1 --2- 172.21.15.45:0/2923850149 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd5f019a780 0x7fd5f019ac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:49.123 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:49.122+0000 7fd5efe00640 1 -- 172.21.15.45:0/2923850149 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5f019b380 con 0x7fd5f0102090 2024-04-16T17:09:49.123 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:49.122+0000 7fd5f4e00640 1 --2- 172.21.15.45:0/2923850149 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd5f019a780 0x7fd5f019ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:09:49.123 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:49.122+0000 7fd5efe00640 1 --2- 172.21.15.45:0/2923850149 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd5f0102090 0x7fd5f01a0670 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fd5e000e9f0 tx=0x7fd5e000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:09:49.123 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:49.122+0000 7fd5ecc00640 1 -- 172.21.15.45:0/2923850149 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5e000cd80 con 0x7fd5f0102090 2024-04-16T17:09:49.123 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:49.122+0000 7fd5ecc00640 1 -- 172.21.15.45:0/2923850149 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd5e0010910 con 0x7fd5f0102090 2024-04-16T17:09:49.123 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:49.122+0000 7fd5ecc00640 1 -- 172.21.15.45:0/2923850149 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5e0018990 con 0x7fd5f0102090 2024-04-16T17:09:49.123 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:49.122+0000 7fd5f5800640 1 -- 172.21.15.45:0/2923850149 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5f01a7410 con 0x7fd5f0102090 2024-04-16T17:09:49.123 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:49.122+0000 7fd5f5800640 1 -- 172.21.15.45:0/2923850149 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5f01a77c0 con 0x7fd5f0102090 2024-04-16T17:09:49.125 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:49.126+0000 7fd5f9a47740 1 Processor -- start 2024-04-16T17:09:49.125 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:49.126+0000 7fd5f9a47740 1 -- start start 2024-04-16T17:09:49.127 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:49.126+0000 7fd5f9a47740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56303bff07f0 0x56303bfef540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:49.127 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:49.126+0000 7fd5f9a47740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56303bfefa80 0x56303bfeff00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:49.127 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:49.126+0000 7fd5f9a47740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56303bff9000 0x56303bffb3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:49.127 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:49.126+0000 7fd5f9a47740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x56303bf54780 con 0x56303bfefa80 2024-04-16T17:09:49.127 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:49.126+0000 7fd5f9a47740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x56303befaf40 con 0x56303bff9000 2024-04-16T17:09:49.127 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:49.126+0000 7fd5f9a47740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x56303bf50740 con 0x56303bff07f0 2024-04-16T17:09:49.127 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:49.126+0000 7fd5f4e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56303bff07f0 0x56303bfef540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:49.127 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:49.126+0000 7fd5f4e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56303bff07f0 0x56303bfef540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:37228/0 (socket says 172.21.15.45:37228) 2024-04-16T17:09:49.127 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:49.126+0000 7fd5f4e00640 1 -- 172.21.15.45:0/1400546736 learned_addr learned my addr 172.21.15.45:0/1400546736 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:09:49.127 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:49.126+0000 7fd5ef400640 1 --2- 172.21.15.45:0/1400546736 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56303bff9000 0x56303bffb3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:49.127 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:49.127+0000 7fd5f4e00640 1 -- 172.21.15.45:0/1400546736 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56303bff9000 msgr2=0x56303bffb3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:09:49.127 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:49.127+0000 7fd5f4e00640 1 --2- 172.21.15.45:0/1400546736 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56303bff9000 0x56303bffb3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:49.127 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:49.127+0000 7fd5f4e00640 1 -- 172.21.15.45:0/1400546736 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56303bfefa80 msgr2=0x56303bfeff00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-16T17:09:49.127 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:49.127+0000 7fd5f4e00640 1 --2- 172.21.15.45:0/1400546736 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56303bfefa80 0x56303bfeff00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:49.127 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:49.127+0000 7fd5f4e00640 1 -- 172.21.15.45:0/1400546736 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56303bc67210 con 0x56303bff07f0 2024-04-16T17:09:49.128 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:49.127+0000 7fd5f4e00640 1 --2- 172.21.15.45:0/1400546736 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56303bff07f0 0x56303bfef540 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fd5e8009ca0 tx=0x7fd5e8009900 comp rx=0 tx=0).ready entity=mon.1 client_cookie=582e1ac19506217b server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:09:49.129 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:49.127+0000 7fd5e5600640 1 -- 172.21.15.45:0/1400546736 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5e8043070 con 0x56303bff07f0 2024-04-16T17:09:49.129 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:49.127+0000 7fd5ecc00640 1 -- 172.21.15.45:0/2923850149 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fd5e0010ac0 con 0x7fd5f0102090 2024-04-16T17:09:49.129 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:49.127+0000 7fd5e5600640 1 -- 172.21.15.45:0/1400546736 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd5e8032c90 con 0x56303bff07f0 2024-04-16T17:09:49.129 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:49.127+0000 7fd5e5600640 1 -- 172.21.15.45:0/1400546736 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5e8039040 con 0x56303bff07f0 2024-04-16T17:09:49.129 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:49.128+0000 7fd5f9a47740 1 -- 172.21.15.45:0/1400546736 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56303bff07f0 msgr2=0x56303bfef540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:49.129 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:49.128+0000 7fd5f9a47740 1 --2- 172.21.15.45:0/1400546736 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56303bff07f0 0x56303bfef540 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fd5e8009ca0 tx=0x7fd5e8009900 comp rx=0 tx=0).stop 2024-04-16T17:09:49.129 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:49.128+0000 7fd5f9a47740 1 -- 172.21.15.45:0/1400546736 shutdown_connections 2024-04-16T17:09:49.129 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:49.128+0000 7fd5f9a47740 1 --2- 172.21.15.45:0/1400546736 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56303bff9000 0x56303bffb3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:49.129 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:49.128+0000 7fd5f9a47740 1 --2- 172.21.15.45:0/1400546736 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56303bfefa80 0x56303bfeff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:49.129 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:49.128+0000 7fd5f9a47740 1 --2- 172.21.15.45:0/1400546736 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56303bff07f0 0x56303bfef540 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:49.130 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:49.128+0000 7fd5f9a47740 1 -- 172.21.15.45:0/1400546736 >> 172.21.15.45:0/1400546736 conn(0x56303bf8a5d0 msgr2=0x56303bfea770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:09:49.130 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:49.128+0000 7fd5f9a47740 1 -- 172.21.15.45:0/1400546736 shutdown_connections 2024-04-16T17:09:49.130 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:49.128+0000 7fd5f9a47740 1 -- 172.21.15.45:0/1400546736 wait complete. 2024-04-16T17:09:49.130 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:49.129+0000 7fd5f9a47740 1 -- start start 2024-04-16T17:09:49.130 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:49.129+0000 7fd5ecc00640 1 --2- 172.21.15.45:0/2923850149 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fd5d40a3ca0 0x7fd5d40a6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:49.130 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:49.129+0000 7fd5f9a47740 1 Processor -- start 2024-04-16T17:09:49.130 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:49.129+0000 7fd5ecc00640 1 -- 172.21.15.45:0/2923850149 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fd5e00d0e30 con 0x7fd5f0102090 2024-04-16T17:09:49.130 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:49.129+0000 7fd5f4e00640 1 --2- 172.21.15.45:0/2923850149 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fd5d40a3ca0 0x7fd5d40a6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:49.133 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:49.130+0000 7fd5f9a47740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56303bfefa80 0x56303bff8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:49.133 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:49.130+0000 7fd5f9a47740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56303bff07f0 0x56303bff5e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:49.133 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:49.130+0000 7fd5f9a47740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56303bff9000 0x56303bff65d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:49.133 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:49.130+0000 7fd5f9a47740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x56303bffbcd0 con 0x56303bff07f0 2024-04-16T17:09:49.133 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:49.130+0000 7fd5f9a47740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x56303bf0e390 con 0x56303bfefa80 2024-04-16T17:09:49.133 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:49.130+0000 7fd5f9a47740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x56303bffbe50 con 0x56303bff9000 2024-04-16T17:09:49.133 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:49.130+0000 7fd5f9a47740 10 client.0 _ll_register_callbacks cb 0x56303bef7f60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:09:49.133 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:49.130+0000 7fd5efe00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56303bfefa80 0x56303bff8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:49.133 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:49.130+0000 7fd5f4e00640 1 --2- 172.21.15.45:0/2923850149 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fd5d40a3ca0 0x7fd5d40a6160 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fd5e802c630 tx=0x7fd5e802c0f0 comp rx=0 tx=0).ready entity=mgr.25411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:09:49.133 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:49.130+0000 7fd5efe00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56303bfefa80 0x56303bff8aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:47310/0 (socket says 172.21.15.45:47310) 2024-04-16T17:09:49.133 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:49.130+0000 7fd5efe00640 1 -- 172.21.15.45:0/1733124481 learned_addr learned my addr 172.21.15.45:0/1733124481 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:09:49.133 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:49.130+0000 7fd5f4e00640 1 --2- 172.21.15.45:0/1733124481 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56303bff9000 0x56303bff65d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:49.133 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:49.130+0000 7fd5ef400640 1 --2- 172.21.15.45:0/1733124481 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56303bff07f0 0x56303bff5e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:49.134 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:49.131+0000 7fd5efe00640 1 -- 172.21.15.45:0/1733124481 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56303bff9000 msgr2=0x56303bff65d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:09:49.134 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:49.131+0000 7fd5efe00640 1 --2- 172.21.15.45:0/1733124481 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56303bff9000 0x56303bff65d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:49.134 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:49.131+0000 7fd5efe00640 1 -- 172.21.15.45:0/1733124481 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56303bff07f0 msgr2=0x56303bff5e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:09:49.134 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:49.131+0000 7fd5efe00640 1 --2- 172.21.15.45:0/1733124481 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56303bff07f0 0x56303bff5e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:49.134 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:49.131+0000 7fd5efe00640 1 -- 172.21.15.45:0/1733124481 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56303bd33620 con 0x56303bfefa80 2024-04-16T17:09:49.134 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:49.131+0000 7fd5efe00640 1 --2- 172.21.15.45:0/1733124481 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56303bfefa80 0x56303bff8aa0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fd5e000e9c0 tx=0x7fd5e009cad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=40e07cb320b4ede3 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:09:49.134 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:49.131+0000 7fd5da000640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-04-16T17:09:49.134 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:49.131+0000 7fd5da000640 1 -- 172.21.15.45:0/1733124481 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5e00a5400 con 0x56303bfefa80 2024-04-16T17:09:49.134 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:49.131+0000 7fd5da000640 1 -- 172.21.15.45:0/1733124481 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fd5e0018b30 con 0x56303bfefa80 2024-04-16T17:09:49.134 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:49.131+0000 7fd5f9a47740 1 -- 172.21.15.45:0/1733124481 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd5e8032c90 con 0x56303bfefa80 2024-04-16T17:09:49.134 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:49.131+0000 7fd5da000640 1 -- 172.21.15.45:0/1733124481 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fd5e00a5690 con 0x56303bfefa80 2024-04-16T17:09:49.134 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:49.131+0000 7fd5f9a47740 10 client.26525 fetch_fsmap learned FSMap version 30 2024-04-16T17:09:49.134 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:49.131+0000 7fd5f9a47740 1 -- 172.21.15.45:0/1733124481 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56303c007e00 con 0x56303bfefa80 2024-04-16T17:09:49.134 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:49.132+0000 7fd5da000640 1 -- 172.21.15.45:0/1733124481 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd5e8032c90 con 0x56303bfefa80 2024-04-16T17:09:49.134 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:49.132+0000 7fd5da000640 1 -- 172.21.15.45:0/1733124481 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fd5e0098820 con 0x56303bfefa80 2024-04-16T17:09:49.134 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:49.132+0000 7fd5f9a47740 10 client.26525 fetch_fsmap finished waiting for FSMap version 30 2024-04-16T17:09:49.135 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:49.132+0000 7fd5f9a47740 10 client.26525 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T17:09:49.135 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:49.132+0000 7fd5f9a47740 20 client.26525 populate_metadata read hostname 'smithi045' 2024-04-16T17:09:49.135 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:49.132+0000 7fd5f9a47740 1 --2- 172.21.15.45:0/1733124481 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x56303c009360 0x56303c00b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:49.135 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:49.132+0000 7fd5f9a47740 4 client.26525 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:09:49.135 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:49.132+0000 7fd5f9a47740 1 -- 172.21.15.45:0/1733124481 --> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56303bf621a0 con 0x56303c009360 2024-04-16T17:09:49.135 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:49.133+0000 7fd5ef400640 1 --2- 172.21.15.45:0/1733124481 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x56303c009360 0x56303c00b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:49.135 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:49.133+0000 7fd5ef400640 1 --2- 172.21.15.45:0/1733124481 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x56303c009360 0x56303c00b7d0 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ef70914450e3d30b server_cookie=bdf13c9359bdcd29 in_seq=0 out_seq=0 2024-04-16T17:09:49.135 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:49.133+0000 7fd5da000640 10 client.26525 ms_handle_connect on v2:172.21.15.104:6838/3660070592 2024-04-16T17:09:49.136 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:49.134+0000 7fd5da000640 1 -- 172.21.15.45:0/1733124481 <== mds.0 v2:172.21.15.104:6838/3660070592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56303bf621a0 con 0x56303c009360 2024-04-16T17:09:49.136 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:49.134+0000 7fd5da000640 10 client.26525 handle_command_reply: tid=0 2024-04-16T17:09:49.136 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:49.134+0000 7fd5f9a47740 1 client.26525 shutdown 2024-04-16T17:09:49.136 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:49.134+0000 7fd5f9a47740 1 -- 172.21.15.45:0/1733124481 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56303bfefa80 msgr2=0x56303bff8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:49.136 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:49.134+0000 7fd5f9a47740 1 --2- 172.21.15.45:0/1733124481 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56303bfefa80 0x56303bff8aa0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fd5e000e9c0 tx=0x7fd5e009cad0 comp rx=0 tx=0).stop 2024-04-16T17:09:49.136 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:49.134+0000 7fd5f9a47740 1 -- 172.21.15.45:0/1733124481 shutdown_connections 2024-04-16T17:09:49.136 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:49.134+0000 7fd5f9a47740 1 --2- 172.21.15.45:0/1733124481 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x56303c009360 0x56303c00b7d0 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:49.136 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:49.134+0000 7fd5f9a47740 1 --2- 172.21.15.45:0/1733124481 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56303bff9000 0x56303bff65d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:49.137 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:49.134+0000 7fd5f9a47740 1 --2- 172.21.15.45:0/1733124481 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56303bff07f0 0x56303bff5e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:49.137 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:49.134+0000 7fd5da000640 0 client.26525 ms_handle_reset on v2:172.21.15.104:6838/3660070592 2024-04-16T17:09:49.137 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:49.134+0000 7fd5f9a47740 1 --2- 172.21.15.45:0/1733124481 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56303bfefa80 0x56303bff8aa0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:49.137 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:49.134+0000 7fd5f9a47740 1 -- 172.21.15.45:0/1733124481 >> 172.21.15.45:0/1733124481 conn(0x56303bf8a5d0 msgr2=0x56303bfec4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:09:49.137 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:49.135+0000 7fd5f9a47740 1 -- 172.21.15.45:0/1733124481 shutdown_connections 2024-04-16T17:09:49.137 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:49.135+0000 7fd5f9a47740 1 -- 172.21.15.45:0/1733124481 wait complete. 2024-04-16T17:09:49.137 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:49.135+0000 7fd5f9a47740 20 client.26525 trim_cache size 0 max 16384 2024-04-16T17:09:49.166 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:49.166+0000 7fd5f9a47740 1 Processor -- start 2024-04-16T17:09:49.166 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:49.166+0000 7fd5f9a47740 1 -- start start 2024-04-16T17:09:49.167 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:49.167+0000 7fd5f9a47740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56303bfefa80 0x56303c004c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:49.167 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:49.167+0000 7fd5f9a47740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56303bff07f0 0x56303c0110a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:49.167 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:49.167+0000 7fd5f9a47740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56303bff9000 0x56303c0115e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:49.167 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:49.167+0000 7fd5f9a47740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fd5f01b3e90 con 0x56303bfefa80 2024-04-16T17:09:49.167 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:49.167+0000 7fd5f9a47740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fd5f01b16a0 con 0x56303bff9000 2024-04-16T17:09:49.167 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:49.167+0000 7fd5f9a47740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fd5f01af4d0 con 0x56303bff07f0 2024-04-16T17:09:49.168 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:49.167+0000 7fd5efe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56303bfefa80 0x56303c004c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:49.168 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:49.167+0000 7fd5efe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56303bfefa80 0x56303c004c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:48142/0 (socket says 172.21.15.45:48142) 2024-04-16T17:09:49.168 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:49.167+0000 7fd5efe00640 1 -- 172.21.15.45:0/2918202860 learned_addr learned my addr 172.21.15.45:0/2918202860 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:09:49.168 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:49.167+0000 7fd5ef400640 1 --2- 172.21.15.45:0/2918202860 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56303bff07f0 0x56303c0110a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:49.168 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:49.167+0000 7fd5f4e00640 1 --2- 172.21.15.45:0/2918202860 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56303bff9000 0x56303c0115e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:49.168 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:49.168+0000 7fd5ef400640 1 -- 172.21.15.45:0/2918202860 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56303bff9000 msgr2=0x56303c0115e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:09:49.168 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:49.168+0000 7fd5ef400640 1 --2- 172.21.15.45:0/2918202860 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56303bff9000 0x56303c0115e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:49.168 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:49.168+0000 7fd5ef400640 1 -- 172.21.15.45:0/2918202860 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56303bfefa80 msgr2=0x56303c004c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:09:49.168 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:49.168+0000 7fd5ef400640 1 --2- 172.21.15.45:0/2918202860 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56303bfefa80 0x56303c004c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:49.168 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:49.168+0000 7fd5ef400640 1 -- 172.21.15.45:0/2918202860 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56303bff1ab0 con 0x56303bff07f0 2024-04-16T17:09:49.169 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:49.168+0000 7fd5ef400640 1 --2- 172.21.15.45:0/2918202860 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56303bff07f0 0x56303c0110a0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x56303c00c040 tx=0x7fd5dc005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1fd3f8202b3ebd32 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:09:49.169 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:49.168+0000 7fd5e4c00640 1 -- 172.21.15.45:0/2918202860 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5dc025020 con 0x56303bff07f0 2024-04-16T17:09:49.169 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:49.168+0000 7fd5e4c00640 1 -- 172.21.15.45:0/2918202860 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd5dc0047a0 con 0x56303bff07f0 2024-04-16T17:09:49.169 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:49.168+0000 7fd5e4c00640 1 -- 172.21.15.45:0/2918202860 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5dc0205c0 con 0x56303bff07f0 2024-04-16T17:09:49.170 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:49.169+0000 7fd5f9a47740 1 -- 172.21.15.45:0/2918202860 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56303bff07f0 msgr2=0x56303c0110a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:49.170 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:49.169+0000 7fd5f9a47740 1 --2- 172.21.15.45:0/2918202860 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56303bff07f0 0x56303c0110a0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x56303c00c040 tx=0x7fd5dc005f90 comp rx=0 tx=0).stop 2024-04-16T17:09:49.170 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:49.169+0000 7fd5f9a47740 1 -- 172.21.15.45:0/2918202860 shutdown_connections 2024-04-16T17:09:49.170 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:49.169+0000 7fd5f9a47740 1 --2- 172.21.15.45:0/2918202860 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56303bff07f0 0x56303c0110a0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:49.170 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:49.169+0000 7fd5f9a47740 1 --2- 172.21.15.45:0/2918202860 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56303bff9000 0x56303c0115e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:49.170 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:49.169+0000 7fd5f9a47740 1 --2- 172.21.15.45:0/2918202860 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56303bfefa80 0x56303c004c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:49.170 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:49.169+0000 7fd5f9a47740 1 -- 172.21.15.45:0/2918202860 >> 172.21.15.45:0/2918202860 conn(0x56303bf8a5d0 msgr2=0x56303bff7590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:09:49.170 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:49.169+0000 7fd5f9a47740 1 -- 172.21.15.45:0/2918202860 shutdown_connections 2024-04-16T17:09:49.170 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:49.169+0000 7fd5f9a47740 1 -- 172.21.15.45:0/2918202860 wait complete. 2024-04-16T17:09:49.170 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:49.169+0000 7fd5f9a47740 1 -- start start 2024-04-16T17:09:49.170 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:49.170+0000 7fd5f9a47740 1 Processor -- start 2024-04-16T17:09:49.170 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:49.170+0000 7fd5f9a47740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56303bfefa80 0x56303c0015a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:49.171 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:49.170+0000 7fd5f9a47740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56303bff07f0 0x56303bff28a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:49.171 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:49.170+0000 7fd5f9a47740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56303bff9000 0x56303bff2e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:49.171 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:49.170+0000 7fd5f9a47740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fd5f01afb60 con 0x56303bff9000 2024-04-16T17:09:49.171 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:49.170+0000 7fd5f9a47740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fd5f01b0fd0 con 0x56303bff07f0 2024-04-16T17:09:49.171 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:49.170+0000 7fd5f9a47740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fd5f01b0230 con 0x56303bfefa80 2024-04-16T17:09:49.171 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:49.170+0000 7fd5f9a47740 10 client.0 _ll_register_callbacks cb 0x7fd5e0009700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:09:49.172 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:49.170+0000 7fd5efe00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56303bfefa80 0x56303c0015a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:49.172 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:49.170+0000 7fd5efe00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56303bfefa80 0x56303c0015a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:37264/0 (socket says 172.21.15.45:37264) 2024-04-16T17:09:49.172 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:49.170+0000 7fd5efe00640 1 -- 172.21.15.45:0/2796985983 learned_addr learned my addr 172.21.15.45:0/2796985983 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:09:49.173 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:49.170+0000 7fd5f4e00640 1 --2- 172.21.15.45:0/2796985983 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56303bff9000 0x56303bff2e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:49.173 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:49.171+0000 7fd5ef400640 1 --2- 172.21.15.45:0/2796985983 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56303bff07f0 0x56303bff28a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:49.173 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:49.171+0000 7fd5efe00640 1 -- 172.21.15.45:0/2796985983 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56303bff07f0 msgr2=0x56303bff28a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:09:49.173 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:49.171+0000 7fd5efe00640 1 --2- 172.21.15.45:0/2796985983 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56303bff07f0 0x56303bff28a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:49.173 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:49.171+0000 7fd5efe00640 1 -- 172.21.15.45:0/2796985983 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56303bff9000 msgr2=0x56303bff2e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:09:49.173 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:49.171+0000 7fd5efe00640 1 --2- 172.21.15.45:0/2796985983 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56303bff9000 0x56303bff2e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:49.173 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:49.171+0000 7fd5efe00640 1 -- 172.21.15.45:0/2796985983 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56303bffb760 con 0x56303bfefa80 2024-04-16T17:09:49.173 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:49.171+0000 7fd5efe00640 1 --2- 172.21.15.45:0/2796985983 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56303bfefa80 0x56303c0015a0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fd5e00a0f90 tx=0x7fd5e0005ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b86f44964048739 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:09:49.173 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:49.171+0000 7fd5da000640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-04-16T17:09:49.173 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:49.171+0000 7fd5da000640 1 -- 172.21.15.45:0/2796985983 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5e00aa060 con 0x56303bfefa80 2024-04-16T17:09:49.173 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:49.171+0000 7fd5da000640 1 -- 172.21.15.45:0/2796985983 <== mon.1 v2:172.21.15.117:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fd5e00a5400 con 0x56303bfefa80 2024-04-16T17:09:49.173 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:49.171+0000 7fd5f9a47740 1 -- 172.21.15.45:0/2796985983 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd5dc0047a0 con 0x56303bfefa80 2024-04-16T17:09:49.173 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:49.172+0000 7fd5da000640 1 -- 172.21.15.45:0/2796985983 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fd5e00b1050 con 0x56303bfefa80 2024-04-16T17:09:49.173 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:49.172+0000 7fd5f9a47740 10 client.26500 fetch_fsmap learned FSMap version 30 2024-04-16T17:09:49.173 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:49.172+0000 7fd5f9a47740 1 -- 172.21.15.45:0/2796985983 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56303c06f7f0 con 0x56303bfefa80 2024-04-16T17:09:49.173 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:49.172+0000 7fd5da000640 1 -- 172.21.15.45:0/2796985983 <== mon.1 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd5dc0047a0 con 0x56303bfefa80 2024-04-16T17:09:49.174 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:49.172+0000 7fd5da000640 1 -- 172.21.15.45:0/2796985983 <== mon.1 v2:172.21.15.117:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fd5e0018bc0 con 0x56303bfefa80 2024-04-16T17:09:49.174 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:49.172+0000 7fd5f9a47740 10 client.26500 fetch_fsmap finished waiting for FSMap version 30 2024-04-16T17:09:49.174 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:49.172+0000 7fd5f9a47740 10 client.26500 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T17:09:49.174 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:49.172+0000 7fd5f9a47740 20 client.26500 populate_metadata read hostname 'smithi045' 2024-04-16T17:09:49.174 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:49.172+0000 7fd5f9a47740 1 --2- 172.21.15.45:0/2796985983 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x56303c06fde0 0x56303c072220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:09:49.174 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:49.172+0000 7fd5f9a47740 4 client.26500 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-16T17:09:49.174 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:49.172+0000 7fd5f9a47740 1 -- 172.21.15.45:0/2796985983 --> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56303bf5c010 con 0x56303c06fde0 2024-04-16T17:09:49.174 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:49.173+0000 7fd5ef400640 1 --2- 172.21.15.45:0/2796985983 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x56303c06fde0 0x56303c072220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:09:49.174 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:49.173+0000 7fd5ef400640 1 --2- 172.21.15.45:0/2796985983 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x56303c06fde0 0x56303c072220 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6929b8273f4ce463 server_cookie=c604233adb3c7a03 in_seq=0 out_seq=0 2024-04-16T17:09:49.174 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:49.173+0000 7fd5da000640 10 client.26500 ms_handle_connect on v2:172.21.15.104:6838/3660070592 2024-04-16T17:09:49.219 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:49.218+0000 7fd5da000640 1 -- 172.21.15.45:0/2796985983 <== mds.0 v2:172.21.15.104:6838/3660070592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56303bf5c010 con 0x56303c06fde0 2024-04-16T17:09:49.219 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:49.218+0000 7fd5da000640 10 client.26500 handle_command_reply: tid=0 2024-04-16T17:09:49.219 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:49.218+0000 7fd5f9a47740 1 client.26500 shutdown 2024-04-16T17:09:49.219 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:49.218+0000 7fd5f9a47740 1 -- 172.21.15.45:0/2796985983 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56303bfefa80 msgr2=0x56303c0015a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:49.219 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:49.218+0000 7fd5f9a47740 1 --2- 172.21.15.45:0/2796985983 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56303bfefa80 0x56303c0015a0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fd5e00a0f90 tx=0x7fd5e0005ab0 comp rx=0 tx=0).stop 2024-04-16T17:09:49.219 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:49.218+0000 7fd5f9a47740 1 -- 172.21.15.45:0/2796985983 shutdown_connections 2024-04-16T17:09:49.219 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:49.218+0000 7fd5f9a47740 1 --2- 172.21.15.45:0/2796985983 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56303bff9000 0x56303bff2e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:49.219 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:49.218+0000 7fd5f9a47740 1 --2- 172.21.15.45:0/2796985983 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56303bff07f0 0x56303bff28a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:49.219 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:49.218+0000 7fd5f9a47740 1 --2- 172.21.15.45:0/2796985983 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x56303c06fde0 0x56303c072220 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:49.219 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:49.218+0000 7fd5f9a47740 1 --2- 172.21.15.45:0/2796985983 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56303bfefa80 0x56303c0015a0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:49.219 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:49.219+0000 7fd5da000640 0 client.26500 ms_handle_reset on v2:172.21.15.104:6838/3660070592 2024-04-16T17:09:49.219 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:49.219+0000 7fd5f9a47740 1 -- 172.21.15.45:0/2796985983 >> 172.21.15.45:0/2796985983 conn(0x56303bf8a5d0 msgr2=0x56303bfec2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:09:49.219 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:49.219+0000 7fd5f9a47740 1 -- 172.21.15.45:0/2796985983 shutdown_connections 2024-04-16T17:09:49.219 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:49.219+0000 7fd5f9a47740 1 -- 172.21.15.45:0/2796985983 wait complete. 2024-04-16T17:09:49.219 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:49.219+0000 7fd5f9a47740 20 client.26500 trim_cache size 0 max 16384 2024-04-16T17:09:49.220 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-04-16T17:09:49.220 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-04-16T17:09:49.220 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-04-16T17:09:49.220 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-04-16T17:09:49.221 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:49.220+0000 7fd5f5800640 1 -- 172.21.15.45:0/2923850149 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fd5d40a3ca0 msgr2=0x7fd5d40a6160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:49.221 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:49.220+0000 7fd5f5800640 1 --2- 172.21.15.45:0/2923850149 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fd5d40a3ca0 0x7fd5d40a6160 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fd5e802c630 tx=0x7fd5e802c0f0 comp rx=0 tx=0).stop 2024-04-16T17:09:49.221 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:49.220+0000 7fd5f5800640 1 -- 172.21.15.45:0/2923850149 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd5f0102090 msgr2=0x7fd5f01a0670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:09:49.221 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:49.220+0000 7fd5f5800640 1 --2- 172.21.15.45:0/2923850149 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd5f0102090 0x7fd5f01a0670 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fd5e000e9f0 tx=0x7fd5e000eeb0 comp rx=0 tx=0).stop 2024-04-16T17:09:49.221 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:49.220+0000 7fd5f5800640 1 -- 172.21.15.45:0/2923850149 shutdown_connections 2024-04-16T17:09:49.221 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:49.220+0000 7fd5f5800640 1 --2- 172.21.15.45:0/2923850149 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fd5d40a3ca0 0x7fd5d40a6160 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:49.221 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:49.220+0000 7fd5f5800640 1 --2- 172.21.15.45:0/2923850149 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd5f0107160 0x7fd5f01a0bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:49.221 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:49.220+0000 7fd5f5800640 1 --2- 172.21.15.45:0/2923850149 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd5f019a780 0x7fd5f019ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:49.221 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:49.220+0000 7fd5f5800640 1 --2- 172.21.15.45:0/2923850149 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd5f0102090 0x7fd5f01a0670 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:09:49.221 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:49.220+0000 7fd5f5800640 1 -- 172.21.15.45:0/2923850149 >> 172.21.15.45:0/2923850149 conn(0x7fd5f00fd950 msgr2=0x7fd5f010d4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:09:49.221 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:49.220+0000 7fd5f5800640 1 -- 172.21.15.45:0/2923850149 shutdown_connections 2024-04-16T17:09:49.221 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:09:49.220+0000 7fd5f5800640 1 -- 172.21.15.45:0/2923850149 wait complete. 2024-04-16T17:09:49.227 INFO:tasks.workunit.client.0.smithi045.stdout: 1 42582 2.24 MB/sec warmup 111 sec latency 1972.647 ms 2024-04-16T17:09:50.227 INFO:tasks.workunit.client.0.smithi045.stdout: 1 42582 2.22 MB/sec warmup 112 sec latency 2972.709 ms 2024-04-16T17:09:50.677 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:09:50 smithi104 ceph-mon[36457]: pgmap v95: 129 pgs: 129 active+clean; 31 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 352 KiB/s rd, 422 KiB/s wr, 43 op/s 2024-04-16T17:09:50.718 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:09:50 smithi045 ceph-mon[31676]: pgmap v95: 129 pgs: 129 active+clean; 31 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 352 KiB/s rd, 422 KiB/s wr, 43 op/s 2024-04-16T17:09:50.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:09:50 smithi117 ceph-mon[36730]: pgmap v95: 129 pgs: 129 active+clean; 31 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 352 KiB/s rd, 422 KiB/s wr, 43 op/s 2024-04-16T17:09:51.228 INFO:tasks.workunit.client.0.smithi045.stdout: 1 42607 2.20 MB/sec warmup 113 sec latency 3477.687 ms 2024-04-16T17:09:52.228 INFO:tasks.workunit.client.0.smithi045.stdout: 1 42607 2.18 MB/sec warmup 114 sec latency 1430.033 ms 2024-04-16T17:09:52.677 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:09:52 smithi104 ceph-mon[36457]: pgmap v96: 129 pgs: 129 active+clean; 28 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 403 KiB/s rd, 508 KiB/s wr, 73 op/s 2024-04-16T17:09:52.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:09:52 smithi045 ceph-mon[31676]: pgmap v96: 129 pgs: 129 active+clean; 28 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 403 KiB/s rd, 508 KiB/s wr, 73 op/s 2024-04-16T17:09:52.804 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:09:52 smithi117 ceph-mon[36730]: pgmap v96: 129 pgs: 129 active+clean; 28 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 403 KiB/s rd, 508 KiB/s wr, 73 op/s 2024-04-16T17:09:53.228 INFO:tasks.workunit.client.0.smithi045.stdout: 1 42607 2.16 MB/sec warmup 115 sec latency 2430.133 ms 2024-04-16T17:09:54.228 INFO:tasks.workunit.client.0.smithi045.stdout: 1 42607 2.14 MB/sec warmup 116 sec latency 3430.220 ms 2024-04-16T17:09:54.677 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:09:54 smithi104 ceph-mon[36457]: pgmap v97: 129 pgs: 129 active+clean; 28 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 368 KiB/s rd, 476 KiB/s wr, 66 op/s 2024-04-16T17:09:54.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:09:54 smithi104 ceph-mon[36457]: scrub summary: idle 2024-04-16T17:09:54.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:09:54 smithi104 ceph-mon[36457]: scrub summary: idle 2024-04-16T17:09:54.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:09:54 smithi104 ceph-mon[36457]: scrub summary: idle 2024-04-16T17:09:54.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:09:54 smithi045 ceph-mon[31676]: pgmap v97: 129 pgs: 129 active+clean; 28 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 368 KiB/s rd, 476 KiB/s wr, 66 op/s 2024-04-16T17:09:54.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:09:54 smithi045 ceph-mon[31676]: scrub summary: idle 2024-04-16T17:09:54.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:09:54 smithi045 ceph-mon[31676]: scrub summary: idle 2024-04-16T17:09:54.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:09:54 smithi045 ceph-mon[31676]: scrub summary: idle 2024-04-16T17:09:54.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:09:54 smithi117 ceph-mon[36730]: pgmap v97: 129 pgs: 129 active+clean; 28 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 368 KiB/s rd, 476 KiB/s wr, 66 op/s 2024-04-16T17:09:54.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:09:54 smithi117 ceph-mon[36730]: scrub summary: idle 2024-04-16T17:09:54.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:09:54 smithi117 ceph-mon[36730]: scrub summary: idle 2024-04-16T17:09:54.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:09:54 smithi117 ceph-mon[36730]: scrub summary: idle 2024-04-16T17:09:55.228 INFO:tasks.workunit.client.0.smithi045.stdout: 1 42607 2.13 MB/sec warmup 117 sec latency 4430.309 ms 2024-04-16T17:09:56.228 INFO:tasks.workunit.client.0.smithi045.stdout: 1 42739 2.11 MB/sec warmup 118 sec latency 4932.390 ms 2024-04-16T17:09:56.677 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:09:56 smithi104 ceph-mon[36457]: pgmap v98: 129 pgs: 129 active+clean; 28 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 368 KiB/s rd, 476 KiB/s wr, 66 op/s 2024-04-16T17:09:56.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:09:56 smithi045 ceph-mon[31676]: pgmap v98: 129 pgs: 129 active+clean; 28 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 368 KiB/s rd, 476 KiB/s wr, 66 op/s 2024-04-16T17:09:56.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:09:56 smithi117 ceph-mon[36730]: pgmap v98: 129 pgs: 129 active+clean; 28 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 368 KiB/s rd, 476 KiB/s wr, 66 op/s 2024-04-16T17:09:57.228 INFO:tasks.workunit.client.0.smithi045.stdout: 1 42739 2.09 MB/sec warmup 119 sec latency 1015.624 ms 2024-04-16T17:09:57.677 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:09:57 smithi104 ceph-mon[36457]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T17:09:57.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:09:57 smithi045 ceph-mon[31676]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T17:09:57.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:09:57 smithi117 ceph-mon[36730]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T17:09:58.677 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:09:58 smithi104 ceph-mon[36457]: pgmap v99: 129 pgs: 129 active+clean; 28 MiB data, 469 MiB used, 1.0 TiB / 1.0 TiB avail; 374 KiB/s rd, 485 KiB/s wr, 74 op/s 2024-04-16T17:09:58.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:09:58 smithi104 ceph-mon[36457]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:09:58.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:09:58 smithi104 ceph-mon[36457]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T17:09:58.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:09:58 smithi104 ceph-mon[36457]: from='mgr.25411 ' entity='mgr.x' 2024-04-16T17:09:58.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:09:58 smithi045 ceph-mon[31676]: pgmap v99: 129 pgs: 129 active+clean; 28 MiB data, 469 MiB used, 1.0 TiB / 1.0 TiB avail; 374 KiB/s rd, 485 KiB/s wr, 74 op/s 2024-04-16T17:09:58.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:09:58 smithi045 ceph-mon[31676]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:09:58.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:09:58 smithi045 ceph-mon[31676]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T17:09:58.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:09:58 smithi045 ceph-mon[31676]: from='mgr.25411 ' entity='mgr.x' 2024-04-16T17:09:58.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:09:58 smithi117 ceph-mon[36730]: pgmap v99: 129 pgs: 129 active+clean; 28 MiB data, 469 MiB used, 1.0 TiB / 1.0 TiB avail; 374 KiB/s rd, 485 KiB/s wr, 74 op/s 2024-04-16T17:09:58.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:09:58 smithi117 ceph-mon[36730]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:09:58.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:09:58 smithi117 ceph-mon[36730]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T17:09:58.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:09:58 smithi117 ceph-mon[36730]: from='mgr.25411 ' entity='mgr.x' 2024-04-16T17:09:59.228 INFO:tasks.workunit.client.0.smithi045.stdout: 1 42748 0.03 MB/sec execute 1 sec latency 2982.425 ms 2024-04-16T17:10:00.228 INFO:tasks.workunit.client.0.smithi045.stdout: 1 42974 0.66 MB/sec execute 2 sec latency 67.658 ms 2024-04-16T17:10:00.677 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:10:00 smithi104 ceph-mon[36457]: pgmap v100: 129 pgs: 129 active+clean; 28 MiB data, 469 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 94 KiB/s wr, 37 op/s 2024-04-16T17:10:00.677 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:10:00 smithi104 ceph-mon[36457]: overall HEALTH_OK 2024-04-16T17:10:00.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:10:00 smithi045 ceph-mon[31676]: pgmap v100: 129 pgs: 129 active+clean; 28 MiB data, 469 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 94 KiB/s wr, 37 op/s 2024-04-16T17:10:00.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:10:00 smithi045 ceph-mon[31676]: overall HEALTH_OK 2024-04-16T17:10:00.804 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:10:00 smithi117 ceph-mon[36730]: pgmap v100: 129 pgs: 129 active+clean; 28 MiB data, 469 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 94 KiB/s wr, 37 op/s 2024-04-16T17:10:00.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:10:00 smithi117 ceph-mon[36730]: overall HEALTH_OK 2024-04-16T17:10:01.229 INFO:tasks.workunit.client.0.smithi045.stdout: 1 42974 0.44 MB/sec execute 3 sec latency 1028.830 ms 2024-04-16T17:10:02.229 INFO:tasks.workunit.client.0.smithi045.stdout: 1 42974 0.33 MB/sec execute 4 sec latency 2028.920 ms 2024-04-16T17:10:02.412 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:02.412+0000 7f7c61200640 1 -- 172.21.15.45:0/1544704019 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7c5c107060 msgr2=0x7f7c5c107440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:02.412 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:02.412+0000 7f7c61200640 1 --2- 172.21.15.45:0/1544704019 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7c5c107060 0x7f7c5c107440 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f7c4c009900 tx=0x7f7c4c02bc90 comp rx=0 tx=0).stop 2024-04-16T17:10:02.412 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:02.412+0000 7f7c61200640 1 -- 172.21.15.45:0/1544704019 shutdown_connections 2024-04-16T17:10:02.412 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:02.412+0000 7f7c61200640 1 --2- 172.21.15.45:0/1544704019 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7c5c101c10 0x7f7c5c10dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:02.413 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:02.412+0000 7f7c61200640 1 --2- 172.21.15.45:0/1544704019 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7c5c101250 0x7f7c5c1016d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:02.413 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:02.412+0000 7f7c61200640 1 --2- 172.21.15.45:0/1544704019 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7c5c107060 0x7f7c5c107440 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:02.413 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:02.412+0000 7f7c61200640 1 -- 172.21.15.45:0/1544704019 >> 172.21.15.45:0/1544704019 conn(0x7f7c5c076d00 msgr2=0x7f7c5c077110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:10:02.413 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:02.412+0000 7f7c61200640 1 -- 172.21.15.45:0/1544704019 shutdown_connections 2024-04-16T17:10:02.413 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:02.412+0000 7f7c61200640 1 -- 172.21.15.45:0/1544704019 wait complete. 2024-04-16T17:10:02.413 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:02.414+0000 7f7c61200640 1 Processor -- start 2024-04-16T17:10:02.413 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:02.414+0000 7f7c61200640 1 -- start start 2024-04-16T17:10:02.415 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:02.414+0000 7f7c61200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7c5c101250 0x7f7c5c199650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:02.415 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:02.414+0000 7f7c61200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7c5c101c10 0x7f7c5c199b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:02.415 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:02.414+0000 7f7c61200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7c5c107060 0x7f7c5c1a46d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:02.415 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:02.414+0000 7f7c61200640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f7c5c1112b0 con 0x7f7c5c101250 2024-04-16T17:10:02.415 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:02.414+0000 7f7c61200640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f7c5c111130 con 0x7f7c5c101c10 2024-04-16T17:10:02.415 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:02.414+0000 7f7c61200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f7c5c111430 con 0x7f7c5c107060 2024-04-16T17:10:02.415 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:02.415+0000 7f7c59e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7c5c101250 0x7f7c5c199650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:02.415 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:02.415+0000 7f7c59400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7c5c101c10 0x7f7c5c199b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:02.415 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:02.415+0000 7f7c59400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7c5c101c10 0x7f7c5c199b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:36460/0 (socket says 172.21.15.45:36460) 2024-04-16T17:10:02.415 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:02.415+0000 7f7c59400640 1 -- 172.21.15.45:0/3138004223 learned_addr learned my addr 172.21.15.45:0/3138004223 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:10:02.415 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:02.415+0000 7f7c5a800640 1 --2- 172.21.15.45:0/3138004223 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7c5c107060 0x7f7c5c1a46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:02.415 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:02.415+0000 7f7c59400640 1 -- 172.21.15.45:0/3138004223 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7c5c107060 msgr2=0x7f7c5c1a46d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:02.416 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:02.415+0000 7f7c59400640 1 --2- 172.21.15.45:0/3138004223 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7c5c107060 0x7f7c5c1a46d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:02.416 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:02.415+0000 7f7c59400640 1 -- 172.21.15.45:0/3138004223 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7c5c101250 msgr2=0x7f7c5c199650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:02.416 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:02.415+0000 7f7c59400640 1 --2- 172.21.15.45:0/3138004223 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7c5c101250 0x7f7c5c199650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:02.416 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:02.415+0000 7f7c59400640 1 -- 172.21.15.45:0/3138004223 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c5c1a4c10 con 0x7f7c5c101c10 2024-04-16T17:10:02.417 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:02.416+0000 7f7c59400640 1 --2- 172.21.15.45:0/3138004223 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7c5c101c10 0x7f7c5c199b90 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f7c4800ea60 tx=0x7f7c4800ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:10:02.417 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:02.416+0000 7f7c52a00640 1 -- 172.21.15.45:0/3138004223 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c4800cde0 con 0x7f7c5c101c10 2024-04-16T17:10:02.417 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:02.416+0000 7f7c52a00640 1 -- 172.21.15.45:0/3138004223 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c480035f0 con 0x7f7c5c101c10 2024-04-16T17:10:02.417 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:02.416+0000 7f7c61200640 1 -- 172.21.15.45:0/3138004223 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c5c1a4de0 con 0x7f7c5c101c10 2024-04-16T17:10:02.417 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:02.416+0000 7f7c52a00640 1 -- 172.21.15.45:0/3138004223 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c48010670 con 0x7f7c5c101c10 2024-04-16T17:10:02.417 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:02.416+0000 7f7c61200640 1 -- 172.21.15.45:0/3138004223 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c5c1a52f0 con 0x7f7c5c101c10 2024-04-16T17:10:02.419 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:02.420+0000 7f7c52a00640 1 -- 172.21.15.45:0/3138004223 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f7c48004020 con 0x7f7c5c101c10 2024-04-16T17:10:02.420 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:02.421+0000 7f7c63fef740 1 Processor -- start 2024-04-16T17:10:02.420 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:02.421+0000 7f7c63fef740 1 -- start start 2024-04-16T17:10:02.422 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:02.421+0000 7f7c63fef740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55921ce86610 0x55921ce85360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:02.422 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:02.421+0000 7f7c63fef740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55921ce858a0 0x55921ce85d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:02.422 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:02.421+0000 7f7c63fef740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55921ce93230 0x55921ce95620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:02.422 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:02.421+0000 7f7c63fef740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55921cdea780 con 0x55921ce858a0 2024-04-16T17:10:02.422 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:02.421+0000 7f7c63fef740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55921cd90f40 con 0x55921ce93230 2024-04-16T17:10:02.422 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:02.421+0000 7f7c63fef740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55921cde6740 con 0x55921ce86610 2024-04-16T17:10:02.422 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:02.422+0000 7f7c52a00640 1 --2- 172.21.15.45:0/3138004223 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f7c400a3e40 0x7f7c400a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:02.422 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:02.422+0000 7f7c59400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55921ce93230 0x55921ce95620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:02.422 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:02.422+0000 7f7c59e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55921ce858a0 0x55921ce85d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:02.422 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:02.422+0000 7f7c59400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55921ce93230 0x55921ce95620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:36468/0 (socket says 172.21.15.45:36468) 2024-04-16T17:10:02.423 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:02.422+0000 7f7c59400640 1 -- 172.21.15.45:0/1305858139 learned_addr learned my addr 172.21.15.45:0/1305858139 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:10:02.423 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:02.422+0000 7f7c59e00640 1 --2- 172.21.15.45:0/1305858139 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55921ce858a0 0x55921ce85d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45970/0 (socket says 172.21.15.45:45970) 2024-04-16T17:10:02.423 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:02.422+0000 7f7c52a00640 1 -- 172.21.15.45:0/3138004223 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f7c480d0db0 con 0x7f7c5c101c10 2024-04-16T17:10:02.423 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:02.422+0000 7f7c59e00640 1 -- 172.21.15.45:0/1305858139 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55921ce86610 msgr2=0x55921ce85360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:10:02.423 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:02.422+0000 7f7c59e00640 1 --2- 172.21.15.45:0/1305858139 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55921ce86610 0x55921ce85360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:02.424 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:02.422+0000 7f7c59e00640 1 -- 172.21.15.45:0/1305858139 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55921ce93230 msgr2=0x55921ce95620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:10:02.424 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:02.422+0000 7f7c59e00640 1 --2- 172.21.15.45:0/1305858139 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55921ce93230 0x55921ce95620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:02.424 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:02.422+0000 7f7c59e00640 1 -- 172.21.15.45:0/1305858139 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55921cafd210 con 0x55921ce858a0 2024-04-16T17:10:02.424 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:02.422+0000 7f7c59400640 1 --2- 172.21.15.45:0/1305858139 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55921ce93230 0x55921ce95620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:10:02.424 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:02.423+0000 7f7c5a800640 1 --2- 172.21.15.45:0/3138004223 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f7c400a3e40 0x7f7c400a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:02.424 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:02.423+0000 7f7c59e00640 1 --2- 172.21.15.45:0/1305858139 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55921ce858a0 0x55921ce85d20 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f7c4c02c1d0 tx=0x7f7c4c005be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc34396ac07d7a3c server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:10:02.424 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:02.423+0000 7f7c5a800640 1 --2- 172.21.15.45:0/3138004223 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f7c400a3e40 0x7f7c400a6300 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f7c54005fd0 tx=0x7f7c54013040 comp rx=0 tx=0).ready entity=mgr.25411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:10:02.424 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:02.423+0000 7f7c47400640 1 -- 172.21.15.45:0/1305858139 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c4c045070 con 0x55921ce858a0 2024-04-16T17:10:02.424 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:02.423+0000 7f7c47400640 1 -- 172.21.15.45:0/1305858139 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c4c02c880 con 0x55921ce858a0 2024-04-16T17:10:02.424 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:02.424+0000 7f7c47400640 1 -- 172.21.15.45:0/1305858139 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c4c004bd0 con 0x55921ce858a0 2024-04-16T17:10:02.425 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:02.424+0000 7f7c63fef740 1 -- 172.21.15.45:0/1305858139 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55921ce858a0 msgr2=0x55921ce85d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:02.425 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:02.424+0000 7f7c63fef740 1 --2- 172.21.15.45:0/1305858139 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55921ce858a0 0x55921ce85d20 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f7c4c02c1d0 tx=0x7f7c4c005be0 comp rx=0 tx=0).stop 2024-04-16T17:10:02.425 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:02.425+0000 7f7c63fef740 1 -- 172.21.15.45:0/1305858139 shutdown_connections 2024-04-16T17:10:02.425 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:02.425+0000 7f7c63fef740 1 --2- 172.21.15.45:0/1305858139 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55921ce93230 0x55921ce95620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:02.425 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:02.425+0000 7f7c63fef740 1 --2- 172.21.15.45:0/1305858139 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55921ce858a0 0x55921ce85d20 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:02.425 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:02.425+0000 7f7c63fef740 1 --2- 172.21.15.45:0/1305858139 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55921ce86610 0x55921ce85360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:02.425 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:02.425+0000 7f7c63fef740 1 -- 172.21.15.45:0/1305858139 >> 172.21.15.45:0/1305858139 conn(0x55921ce205d0 msgr2=0x55921ce80590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:10:02.425 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:02.425+0000 7f7c63fef740 1 -- 172.21.15.45:0/1305858139 shutdown_connections 2024-04-16T17:10:02.426 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:02.425+0000 7f7c63fef740 1 -- 172.21.15.45:0/1305858139 wait complete. 2024-04-16T17:10:02.426 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:02.425+0000 7f7c63fef740 1 -- start start 2024-04-16T17:10:02.427 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:02.426+0000 7f7c63fef740 1 Processor -- start 2024-04-16T17:10:02.427 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:02.426+0000 7f7c63fef740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55921ce858a0 0x55921ce8c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:02.427 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:02.426+0000 7f7c63fef740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55921ce86610 0x55921ce8c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:02.427 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:02.426+0000 7f7c63fef740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55921ce93230 0x55921ce8cd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:02.427 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:02.427+0000 7f7c63fef740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55921ce86fa0 con 0x55921ce93230 2024-04-16T17:10:02.427 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:02.427+0000 7f7c63fef740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55921cda4390 con 0x55921ce86610 2024-04-16T17:10:02.427 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:02.427+0000 7f7c63fef740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55921ce87120 con 0x55921ce858a0 2024-04-16T17:10:02.427 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:02.427+0000 7f7c63fef740 10 client.0 _ll_register_callbacks cb 0x55921cd8df60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:10:02.427 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:02.427+0000 7f7c59400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55921ce86610 0x55921ce8c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:02.427 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:02.427+0000 7f7c59e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55921ce858a0 0x55921ce8c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:02.427 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:02.427+0000 7f7c59400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55921ce86610 0x55921ce8c7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:36480/0 (socket says 172.21.15.45:36480) 2024-04-16T17:10:02.427 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:02.427+0000 7f7c59400640 1 -- 172.21.15.45:0/1274070963 learned_addr learned my addr 172.21.15.45:0/1274070963 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:10:02.428 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:02.427+0000 7f7c59e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55921ce858a0 0x55921ce8c2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:41104/0 (socket says 172.21.15.45:41104) 2024-04-16T17:10:02.428 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:02.427+0000 7f7c5a800640 1 --2- 172.21.15.45:0/1274070963 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55921ce93230 0x55921ce8cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:02.428 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:02.427+0000 7f7c59400640 1 -- 172.21.15.45:0/1274070963 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55921ce858a0 msgr2=0x55921ce8c2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:10:02.428 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:02.427+0000 7f7c59400640 1 --2- 172.21.15.45:0/1274070963 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55921ce858a0 0x55921ce8c2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:02.428 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:02.427+0000 7f7c59400640 1 -- 172.21.15.45:0/1274070963 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55921ce93230 msgr2=0x55921ce8cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:10:02.428 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:02.427+0000 7f7c59400640 1 --2- 172.21.15.45:0/1274070963 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55921ce93230 0x55921ce8cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:02.429 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:02.427+0000 7f7c59400640 1 -- 172.21.15.45:0/1274070963 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55921cbc9620 con 0x55921ce86610 2024-04-16T17:10:02.429 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:02.427+0000 7f7c59e00640 1 --2- 172.21.15.45:0/1274070963 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55921ce858a0 0x55921ce8c2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:10:02.429 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:02.427+0000 7f7c59400640 1 --2- 172.21.15.45:0/1274070963 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55921ce86610 0x55921ce8c7f0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f7c48099370 tx=0x7f7c480bd120 comp rx=0 tx=0).ready entity=mon.2 client_cookie=698bfa6c239d5bba server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:10:02.429 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:02.427+0000 7f7c3be00640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-04-16T17:10:02.429 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:02.428+0000 7f7c3be00640 1 -- 172.21.15.45:0/1274070963 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c4809c300 con 0x55921ce86610 2024-04-16T17:10:02.429 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:02.428+0000 7f7c3be00640 1 -- 172.21.15.45:0/1274070963 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f7c480037a0 con 0x55921ce86610 2024-04-16T17:10:02.429 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:02.428+0000 7f7c63fef740 1 -- 172.21.15.45:0/1274070963 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7c4c02c880 con 0x55921ce86610 2024-04-16T17:10:02.429 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:02.428+0000 7f7c3be00640 1 -- 172.21.15.45:0/1274070963 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f7c4809d0a0 con 0x55921ce86610 2024-04-16T17:10:02.430 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:02.428+0000 7f7c63fef740 10 client.26543 fetch_fsmap learned FSMap version 30 2024-04-16T17:10:02.430 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:02.428+0000 7f7c63fef740 1 -- 172.21.15.45:0/1274070963 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55921cea6470 con 0x55921ce86610 2024-04-16T17:10:02.430 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:02.428+0000 7f7c3be00640 1 -- 172.21.15.45:0/1274070963 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c4c02c880 con 0x55921ce86610 2024-04-16T17:10:02.430 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:02.428+0000 7f7c3be00640 1 -- 172.21.15.45:0/1274070963 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f7c480ce3d0 con 0x55921ce86610 2024-04-16T17:10:02.430 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:02.429+0000 7f7c63fef740 10 client.26543 fetch_fsmap finished waiting for FSMap version 30 2024-04-16T17:10:02.430 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:02.429+0000 7f7c63fef740 10 client.26543 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T17:10:02.430 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:02.429+0000 7f7c63fef740 20 client.26543 populate_metadata read hostname 'smithi045' 2024-04-16T17:10:02.430 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:02.429+0000 7f7c63fef740 1 --2- 172.21.15.45:0/1274070963 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55921cea7980 0x55921cea9d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:02.430 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:02.429+0000 7f7c63fef740 4 client.26543 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:10:02.430 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:02.429+0000 7f7c63fef740 1 -- 172.21.15.45:0/1274070963 --> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55921cdf81a0 con 0x55921cea7980 2024-04-16T17:10:02.430 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:02.429+0000 7f7c59e00640 1 --2- 172.21.15.45:0/1274070963 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55921cea7980 0x55921cea9d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:02.431 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:02.430+0000 7f7c59e00640 1 --2- 172.21.15.45:0/1274070963 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55921cea7980 0x55921cea9d70 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=79092e23af4586ce server_cookie=b9344a8e4b4743db in_seq=0 out_seq=0 2024-04-16T17:10:02.431 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:02.430+0000 7f7c3be00640 10 client.26543 ms_handle_connect on v2:172.21.15.104:6838/3660070592 2024-04-16T17:10:02.432 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:02.430+0000 7f7c3be00640 1 -- 172.21.15.45:0/1274070963 <== mds.0 v2:172.21.15.104:6838/3660070592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55921cdf81a0 con 0x55921cea7980 2024-04-16T17:10:02.432 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:02.430+0000 7f7c3be00640 10 client.26543 handle_command_reply: tid=0 2024-04-16T17:10:02.432 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:02.431+0000 7f7c63fef740 1 client.26543 shutdown 2024-04-16T17:10:02.432 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:02.431+0000 7f7c63fef740 1 -- 172.21.15.45:0/1274070963 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55921ce86610 msgr2=0x55921ce8c7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:02.432 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:02.431+0000 7f7c63fef740 1 --2- 172.21.15.45:0/1274070963 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55921ce86610 0x55921ce8c7f0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f7c48099370 tx=0x7f7c480bd120 comp rx=0 tx=0).stop 2024-04-16T17:10:02.432 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:02.431+0000 7f7c63fef740 1 -- 172.21.15.45:0/1274070963 shutdown_connections 2024-04-16T17:10:02.432 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:02.431+0000 7f7c63fef740 1 --2- 172.21.15.45:0/1274070963 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55921ce93230 0x55921ce8cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:02.432 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:02.431+0000 7f7c63fef740 1 --2- 172.21.15.45:0/1274070963 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55921ce86610 0x55921ce8c7f0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:02.432 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:02.431+0000 7f7c63fef740 1 --2- 172.21.15.45:0/1274070963 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55921cea7980 0x55921cea9d70 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:02.432 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:02.431+0000 7f7c63fef740 1 --2- 172.21.15.45:0/1274070963 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55921ce858a0 0x55921ce8c2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:02.432 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:02.431+0000 7f7c63fef740 1 -- 172.21.15.45:0/1274070963 >> 172.21.15.45:0/1274070963 conn(0x55921ce205d0 msgr2=0x55921ce822e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:10:02.432 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:02.431+0000 7f7c3be00640 0 client.26543 ms_handle_reset on v2:172.21.15.104:6838/3660070592 2024-04-16T17:10:02.432 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:02.431+0000 7f7c63fef740 1 -- 172.21.15.45:0/1274070963 shutdown_connections 2024-04-16T17:10:02.433 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:02.432+0000 7f7c63fef740 1 -- 172.21.15.45:0/1274070963 wait complete. 2024-04-16T17:10:02.433 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:02.432+0000 7f7c63fef740 20 client.26543 trim_cache size 0 max 16384 2024-04-16T17:10:02.456 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:02.456+0000 7f7c63fef740 1 Processor -- start 2024-04-16T17:10:02.456 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:02.456+0000 7f7c63fef740 1 -- start start 2024-04-16T17:10:02.456 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:02.456+0000 7f7c63fef740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55921ce858a0 0x55921ce8cb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:02.456 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:02.456+0000 7f7c63fef740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55921ce86610 0x55921ce7f530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:02.456 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:02.456+0000 7f7c63fef740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55921ce93230 0x55921ceaf640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:02.456 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:02.457+0000 7f7c63fef740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f7c5c1afb20 con 0x55921ce86610 2024-04-16T17:10:02.456 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:02.457+0000 7f7c63fef740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f7c5c1ad330 con 0x55921ce858a0 2024-04-16T17:10:02.457 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:02.457+0000 7f7c63fef740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f7c5c1ab160 con 0x55921ce93230 2024-04-16T17:10:02.458 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:02.457+0000 7f7c59e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55921ce858a0 0x55921ce8cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:02.458 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:02.457+0000 7f7c59e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55921ce858a0 0x55921ce8cb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:36486/0 (socket says 172.21.15.45:36486) 2024-04-16T17:10:02.458 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:02.457+0000 7f7c59e00640 1 -- 172.21.15.45:0/3792092651 learned_addr learned my addr 172.21.15.45:0/3792092651 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:10:02.458 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:02.457+0000 7f7c5a800640 1 --2- 172.21.15.45:0/3792092651 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55921ce93230 0x55921ceaf640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:02.458 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:02.457+0000 7f7c59e00640 1 -- 172.21.15.45:0/3792092651 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55921ce93230 msgr2=0x55921ceaf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:10:02.458 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:02.457+0000 7f7c59e00640 1 --2- 172.21.15.45:0/3792092651 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55921ce93230 0x55921ceaf640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:02.458 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:02.457+0000 7f7c59e00640 1 -- 172.21.15.45:0/3792092651 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55921ce86610 msgr2=0x55921ce7f530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:10:02.458 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:02.457+0000 7f7c59e00640 1 --2- 172.21.15.45:0/3792092651 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55921ce86610 0x55921ce7f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:02.458 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:02.457+0000 7f7c59e00640 1 -- 172.21.15.45:0/3792092651 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55921ce84b70 con 0x55921ce858a0 2024-04-16T17:10:02.458 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:02.458+0000 7f7c59e00640 1 --2- 172.21.15.45:0/3792092651 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55921ce858a0 0x55921ce8cb50 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f7c4c0047a0 tx=0x7f7c4c000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=45e05478fe13a440 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:10:02.458 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:02.458+0000 7f7c46a00640 1 -- 172.21.15.45:0/3792092651 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c4c045040 con 0x55921ce858a0 2024-04-16T17:10:02.458 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:02.458+0000 7f7c46a00640 1 -- 172.21.15.45:0/3792092651 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c4c02c880 con 0x55921ce858a0 2024-04-16T17:10:02.458 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:02.458+0000 7f7c46a00640 1 -- 172.21.15.45:0/3792092651 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c4c03f6a0 con 0x55921ce858a0 2024-04-16T17:10:02.459 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:02.459+0000 7f7c63fef740 1 -- 172.21.15.45:0/3792092651 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55921ce858a0 msgr2=0x55921ce8cb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:02.459 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:02.459+0000 7f7c63fef740 1 --2- 172.21.15.45:0/3792092651 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55921ce858a0 0x55921ce8cb50 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f7c4c0047a0 tx=0x7f7c4c000bc0 comp rx=0 tx=0).stop 2024-04-16T17:10:02.459 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:02.459+0000 7f7c63fef740 1 -- 172.21.15.45:0/3792092651 shutdown_connections 2024-04-16T17:10:02.459 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:02.459+0000 7f7c63fef740 1 --2- 172.21.15.45:0/3792092651 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55921ce93230 0x55921ceaf640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:02.459 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:02.459+0000 7f7c63fef740 1 --2- 172.21.15.45:0/3792092651 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55921ce86610 0x55921ce7f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:02.460 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:02.459+0000 7f7c63fef740 1 --2- 172.21.15.45:0/3792092651 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55921ce858a0 0x55921ce8cb50 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:02.460 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:02.459+0000 7f7c63fef740 1 -- 172.21.15.45:0/3792092651 >> 172.21.15.45:0/3792092651 conn(0x55921ce205d0 msgr2=0x55921ce7ee20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:10:02.460 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:02.459+0000 7f7c63fef740 1 -- 172.21.15.45:0/3792092651 shutdown_connections 2024-04-16T17:10:02.460 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:02.459+0000 7f7c63fef740 1 -- 172.21.15.45:0/3792092651 wait complete. 2024-04-16T17:10:02.460 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:02.459+0000 7f7c63fef740 1 -- start start 2024-04-16T17:10:02.460 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:02.460+0000 7f7c63fef740 1 Processor -- start 2024-04-16T17:10:02.462 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:02.460+0000 7f7c63fef740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55921ce858a0 0x55921cea3570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:02.462 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:02.460+0000 7f7c63fef740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55921ce86610 0x55921cea3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:02.462 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:02.460+0000 7f7c63fef740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55921ce93230 0x55921cea3ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:02.462 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:02.460+0000 7f7c63fef740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f7c5c1ab7f0 con 0x55921ce858a0 2024-04-16T17:10:02.462 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:02.460+0000 7f7c63fef740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f7c5c1acc60 con 0x55921ce86610 2024-04-16T17:10:02.462 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:02.460+0000 7f7c63fef740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f7c5c1abec0 con 0x55921ce93230 2024-04-16T17:10:02.463 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:02.460+0000 7f7c63fef740 10 client.0 _ll_register_callbacks cb 0x7f7c480d4550 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:10:02.463 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:02.460+0000 7f7c5a800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55921ce93230 0x55921cea3ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:02.463 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:02.460+0000 7f7c59400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55921ce86610 0x55921cea3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:02.463 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:02.460+0000 7f7c5a800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55921ce93230 0x55921cea3ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:41122/0 (socket says 172.21.15.45:41122) 2024-04-16T17:10:02.463 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:02.460+0000 7f7c5a800640 1 -- 172.21.15.45:0/2282851221 learned_addr learned my addr 172.21.15.45:0/2282851221 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:10:02.463 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:02.460+0000 7f7c59e00640 1 --2- 172.21.15.45:0/2282851221 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55921ce858a0 0x55921cea3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:02.463 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:02.461+0000 7f7c59400640 1 -- 172.21.15.45:0/2282851221 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55921ce93230 msgr2=0x55921cea3ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:10:02.463 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:02.461+0000 7f7c59400640 1 --2- 172.21.15.45:0/2282851221 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55921ce93230 0x55921cea3ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:02.463 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:02.461+0000 7f7c59400640 1 -- 172.21.15.45:0/2282851221 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55921ce858a0 msgr2=0x55921cea3570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:10:02.463 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:02.461+0000 7f7c59400640 1 --2- 172.21.15.45:0/2282851221 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55921ce858a0 0x55921cea3570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:02.463 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:02.461+0000 7f7c59400640 1 -- 172.21.15.45:0/2282851221 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55921ce908d0 con 0x55921ce86610 2024-04-16T17:10:02.463 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:02.461+0000 7f7c59400640 1 --2- 172.21.15.45:0/2282851221 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55921ce86610 0x55921cea3ab0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x55921cd8c2e0 tx=0x7f7c480039d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f451050c15046636 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:10:02.463 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:02.461+0000 7f7c3be00640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-04-16T17:10:02.463 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:02.461+0000 7f7c3be00640 1 -- 172.21.15.45:0/2282851221 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c480bbe60 con 0x55921ce86610 2024-04-16T17:10:02.463 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:02.461+0000 7f7c63fef740 1 -- 172.21.15.45:0/2282851221 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7c4c02c880 con 0x55921ce86610 2024-04-16T17:10:02.464 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:02.461+0000 7f7c3be00640 1 -- 172.21.15.45:0/2282851221 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f7c48087310 con 0x55921ce86610 2024-04-16T17:10:02.464 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:02.461+0000 7f7c3be00640 1 -- 172.21.15.45:0/2282851221 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f7c48080050 con 0x55921ce86610 2024-04-16T17:10:02.464 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:02.461+0000 7f7c63fef740 10 client.26549 fetch_fsmap learned FSMap version 30 2024-04-16T17:10:02.464 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:02.461+0000 7f7c63fef740 1 -- 172.21.15.45:0/2282851221 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55921ce908d0 con 0x55921ce86610 2024-04-16T17:10:02.464 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:02.462+0000 7f7c3be00640 1 -- 172.21.15.45:0/2282851221 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c4c02c880 con 0x55921ce86610 2024-04-16T17:10:02.464 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:02.462+0000 7f7c3be00640 1 -- 172.21.15.45:0/2282851221 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f7c480bb060 con 0x55921ce86610 2024-04-16T17:10:02.464 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:02.462+0000 7f7c63fef740 10 client.26549 fetch_fsmap finished waiting for FSMap version 30 2024-04-16T17:10:02.464 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:02.462+0000 7f7c63fef740 10 client.26549 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T17:10:02.464 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:02.462+0000 7f7c63fef740 20 client.26549 populate_metadata read hostname 'smithi045' 2024-04-16T17:10:02.464 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:02.462+0000 7f7c63fef740 1 --2- 172.21.15.45:0/2282851221 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55921cf0e3e0 0x55921cf10800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:02.464 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:02.462+0000 7f7c63fef740 4 client.26549 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-16T17:10:02.464 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:02.462+0000 7f7c63fef740 1 -- 172.21.15.45:0/2282851221 --> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55921cdf2010 con 0x55921cf0e3e0 2024-04-16T17:10:02.464 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:02.462+0000 7f7c59e00640 1 --2- 172.21.15.45:0/2282851221 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55921cf0e3e0 0x55921cf10800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:02.465 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:02.463+0000 7f7c59e00640 1 --2- 172.21.15.45:0/2282851221 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55921cf0e3e0 0x55921cf10800 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d2e76d4863d51948 server_cookie=2a181ef702d206e0 in_seq=0 out_seq=0 2024-04-16T17:10:02.465 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:02.463+0000 7f7c3be00640 10 client.26549 ms_handle_connect on v2:172.21.15.104:6838/3660070592 2024-04-16T17:10:02.485 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:02.485+0000 7f7c3be00640 1 -- 172.21.15.45:0/2282851221 <== mds.0 v2:172.21.15.104:6838/3660070592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55921cdf2010 con 0x55921cf0e3e0 2024-04-16T17:10:02.485 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:02.485+0000 7f7c3be00640 10 client.26549 handle_command_reply: tid=0 2024-04-16T17:10:02.485 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:02.485+0000 7f7c63fef740 1 client.26549 shutdown 2024-04-16T17:10:02.485 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:02.485+0000 7f7c63fef740 1 -- 172.21.15.45:0/2282851221 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55921ce86610 msgr2=0x55921cea3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:02.485 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:02.485+0000 7f7c63fef740 1 --2- 172.21.15.45:0/2282851221 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55921ce86610 0x55921cea3ab0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x55921cd8c2e0 tx=0x7f7c480039d0 comp rx=0 tx=0).stop 2024-04-16T17:10:02.485 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:02.485+0000 7f7c63fef740 1 -- 172.21.15.45:0/2282851221 shutdown_connections 2024-04-16T17:10:02.485 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:02.485+0000 7f7c63fef740 1 --2- 172.21.15.45:0/2282851221 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55921cf0e3e0 0x55921cf10800 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:02.485 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:02.485+0000 7f7c63fef740 1 --2- 172.21.15.45:0/2282851221 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55921ce93230 0x55921cea3ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:02.485 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:02.485+0000 7f7c3be00640 0 client.26549 ms_handle_reset on v2:172.21.15.104:6838/3660070592 2024-04-16T17:10:02.485 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:02.485+0000 7f7c63fef740 1 --2- 172.21.15.45:0/2282851221 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55921ce86610 0x55921cea3ab0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:02.486 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:02.485+0000 7f7c63fef740 1 --2- 172.21.15.45:0/2282851221 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55921ce858a0 0x55921cea3570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:02.486 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:02.485+0000 7f7c63fef740 1 -- 172.21.15.45:0/2282851221 >> 172.21.15.45:0/2282851221 conn(0x55921ce205d0 msgr2=0x55921ce90150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:10:02.486 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:02.485+0000 7f7c63fef740 1 -- 172.21.15.45:0/2282851221 shutdown_connections 2024-04-16T17:10:02.486 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:02.485+0000 7f7c63fef740 1 -- 172.21.15.45:0/2282851221 wait complete. 2024-04-16T17:10:02.486 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:02.485+0000 7f7c63fef740 20 client.26549 trim_cache size 0 max 16384 2024-04-16T17:10:02.486 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-04-16T17:10:02.486 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-04-16T17:10:02.486 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-04-16T17:10:02.486 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-04-16T17:10:02.487 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:02.486+0000 7f7c61200640 1 -- 172.21.15.45:0/3138004223 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f7c400a3e40 msgr2=0x7f7c400a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:02.487 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:02.486+0000 7f7c61200640 1 --2- 172.21.15.45:0/3138004223 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f7c400a3e40 0x7f7c400a6300 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f7c54005fd0 tx=0x7f7c54013040 comp rx=0 tx=0).stop 2024-04-16T17:10:02.487 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:02.486+0000 7f7c61200640 1 -- 172.21.15.45:0/3138004223 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7c5c101c10 msgr2=0x7f7c5c199b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:02.487 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:02.486+0000 7f7c61200640 1 --2- 172.21.15.45:0/3138004223 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7c5c101c10 0x7f7c5c199b90 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f7c4800ea60 tx=0x7f7c4800ef20 comp rx=0 tx=0).stop 2024-04-16T17:10:02.487 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:02.486+0000 7f7c61200640 1 -- 172.21.15.45:0/3138004223 shutdown_connections 2024-04-16T17:10:02.487 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:02.486+0000 7f7c61200640 1 --2- 172.21.15.45:0/3138004223 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f7c400a3e40 0x7f7c400a6300 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:02.488 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:02.486+0000 7f7c61200640 1 --2- 172.21.15.45:0/3138004223 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7c5c107060 0x7f7c5c1a46d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:02.488 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:02.486+0000 7f7c61200640 1 --2- 172.21.15.45:0/3138004223 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7c5c101c10 0x7f7c5c199b90 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:02.488 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:02.486+0000 7f7c61200640 1 --2- 172.21.15.45:0/3138004223 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7c5c101250 0x7f7c5c199650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:02.488 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:02.486+0000 7f7c61200640 1 -- 172.21.15.45:0/3138004223 >> 172.21.15.45:0/3138004223 conn(0x7f7c5c076d00 msgr2=0x7f7c5c0ffe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:10:02.488 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:02.487+0000 7f7c61200640 1 -- 172.21.15.45:0/3138004223 shutdown_connections 2024-04-16T17:10:02.488 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:02.487+0000 7f7c61200640 1 -- 172.21.15.45:0/3138004223 wait complete. 2024-04-16T17:10:02.677 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:10:02 smithi104 ceph-mon[36457]: pgmap v101: 129 pgs: 129 active+clean; 30 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 424 KiB/s wr, 58 op/s 2024-04-16T17:10:02.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:10:02 smithi117 ceph-mon[36730]: pgmap v101: 129 pgs: 129 active+clean; 30 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 424 KiB/s wr, 58 op/s 2024-04-16T17:10:03.229 INFO:tasks.workunit.client.0.smithi045.stdout: 1 43159 0.27 MB/sec execute 5 sec latency 2085.529 ms 2024-04-16T17:10:03.674 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:10:02 smithi045 ceph-mon[31676]: pgmap v101: 129 pgs: 129 active+clean; 30 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 424 KiB/s wr, 58 op/s 2024-04-16T17:10:04.229 INFO:tasks.workunit.client.0.smithi045.stdout: 1 43320 0.26 MB/sec execute 6 sec latency 545.851 ms 2024-04-16T17:10:04.677 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:10:04 smithi104 ceph-mon[36457]: pgmap v102: 129 pgs: 129 active+clean; 30 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 338 KiB/s wr, 28 op/s 2024-04-16T17:10:04.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:10:04 smithi045 ceph-mon[31676]: pgmap v102: 129 pgs: 129 active+clean; 30 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 338 KiB/s wr, 28 op/s 2024-04-16T17:10:04.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:10:04 smithi117 ceph-mon[36730]: pgmap v102: 129 pgs: 129 active+clean; 30 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 338 KiB/s wr, 28 op/s 2024-04-16T17:10:05.229 INFO:tasks.workunit.client.0.smithi045.stdout: 1 43320 0.22 MB/sec execute 7 sec latency 1545.941 ms 2024-04-16T17:10:06.229 INFO:tasks.workunit.client.0.smithi045.stdout: 1 43320 0.20 MB/sec execute 8 sec latency 2546.039 ms 2024-04-16T17:10:06.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:10:06 smithi104 ceph-mon[36457]: pgmap v103: 129 pgs: 129 active+clean; 30 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 338 KiB/s wr, 28 op/s 2024-04-16T17:10:06.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:10:06 smithi045 ceph-mon[31676]: pgmap v103: 129 pgs: 129 active+clean; 30 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 338 KiB/s wr, 28 op/s 2024-04-16T17:10:06.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:10:06 smithi117 ceph-mon[36730]: pgmap v103: 129 pgs: 129 active+clean; 30 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 338 KiB/s wr, 28 op/s 2024-04-16T17:10:07.229 INFO:tasks.workunit.client.0.smithi045.stdout: 1 43320 0.17 MB/sec execute 9 sec latency 3546.151 ms 2024-04-16T17:10:08.229 INFO:tasks.workunit.client.0.smithi045.stdout: 1 43388 0.17 MB/sec execute 10 sec latency 3585.823 ms 2024-04-16T17:10:08.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:10:08 smithi104 ceph-mon[36457]: pgmap v104: 129 pgs: 129 active+clean; 31 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 426 KiB/s wr, 57 op/s 2024-04-16T17:10:08.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:10:08 smithi045 ceph-mon[31676]: pgmap v104: 129 pgs: 129 active+clean; 31 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 426 KiB/s wr, 57 op/s 2024-04-16T17:10:08.804 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:10:08 smithi117 ceph-mon[36730]: pgmap v104: 129 pgs: 129 active+clean; 31 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 426 KiB/s wr, 57 op/s 2024-04-16T17:10:09.230 INFO:tasks.workunit.client.0.smithi045.stdout: 1 43388 0.16 MB/sec execute 11 sec latency 1804.316 ms 2024-04-16T17:10:10.230 INFO:tasks.workunit.client.0.smithi045.stdout: 1 43388 0.14 MB/sec execute 12 sec latency 2804.833 ms 2024-04-16T17:10:10.677 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:10:10 smithi104 ceph-mon[36457]: pgmap v105: 129 pgs: 129 active+clean; 31 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 418 KiB/s wr, 49 op/s 2024-04-16T17:10:10.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:10:10 smithi045 ceph-mon[31676]: pgmap v105: 129 pgs: 129 active+clean; 31 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 418 KiB/s wr, 49 op/s 2024-04-16T17:10:10.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:10:10 smithi117 ceph-mon[36730]: pgmap v105: 129 pgs: 129 active+clean; 31 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 418 KiB/s wr, 49 op/s 2024-04-16T17:10:11.230 INFO:tasks.workunit.client.0.smithi045.stdout: 1 43388 0.13 MB/sec execute 13 sec latency 3804.970 ms 2024-04-16T17:10:12.230 INFO:tasks.workunit.client.0.smithi045.stdout: 1 43388 0.12 MB/sec execute 14 sec latency 4805.069 ms 2024-04-16T17:10:12.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:10:12 smithi117 ceph-mon[36730]: pgmap v106: 129 pgs: 129 active+clean; 31 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 442 KiB/s wr, 60 op/s 2024-04-16T17:10:12.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:10:12 smithi104 ceph-mon[36457]: pgmap v106: 129 pgs: 129 active+clean; 31 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 442 KiB/s wr, 60 op/s 2024-04-16T17:10:12.995 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:10:12 smithi045 ceph-mon[31676]: pgmap v106: 129 pgs: 129 active+clean; 31 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 442 KiB/s wr, 60 op/s 2024-04-16T17:10:13.231 INFO:tasks.workunit.client.0.smithi045.stdout: 1 43625 0.18 MB/sec execute 15 sec latency 4845.602 ms 2024-04-16T17:10:14.230 INFO:tasks.workunit.client.0.smithi045.stdout: 1 44000 0.40 MB/sec execute 16 sec latency 60.543 ms 2024-04-16T17:10:14.474 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-16T17:10:14.474 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-16T17:10:14.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.693+0000 7f9f5e800640 1 -- 172.21.15.45:0/3910108171 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9f580743b0 msgr2=0x7f9f58074790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:14.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.693+0000 7f9f5e800640 1 --2- 172.21.15.45:0/3910108171 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9f580743b0 0x7f9f58074790 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f9f4c009900 tx=0x7f9f4c02bcc0 comp rx=0 tx=0).stop 2024-04-16T17:10:14.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.693+0000 7f9f5e800640 1 -- 172.21.15.45:0/3910108171 shutdown_connections 2024-04-16T17:10:14.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.693+0000 7f9f5e800640 1 --2- 172.21.15.45:0/3910108171 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9f5810e760 0x7f9f58110b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:14.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.693+0000 7f9f5e800640 1 --2- 172.21.15.45:0/3910108171 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9f58074cd0 0x7f9f5810e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:14.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.693+0000 7f9f5e800640 1 --2- 172.21.15.45:0/3910108171 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9f580743b0 0x7f9f58074790 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:14.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.693+0000 7f9f5e800640 1 -- 172.21.15.45:0/3910108171 >> 172.21.15.45:0/3910108171 conn(0x7f9f580fd9d0 msgr2=0x7f9f580ffdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:10:14.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.693+0000 7f9f5e800640 1 -- 172.21.15.45:0/3910108171 shutdown_connections 2024-04-16T17:10:14.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.693+0000 7f9f5e800640 1 -- 172.21.15.45:0/3910108171 wait complete. 2024-04-16T17:10:14.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.693+0000 7f9f5e800640 1 Processor -- start 2024-04-16T17:10:14.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.693+0000 7f9f5e800640 1 -- start start 2024-04-16T17:10:14.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.693+0000 7f9f5e800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9f580743b0 0x7f9f581a0690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:14.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.693+0000 7f9f5e800640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9f58074cd0 0x7f9f581a0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:14.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.693+0000 7f9f5e800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9f5810e760 0x7f9f5819a850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:14.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.693+0000 7f9f5e800640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f9f581144b0 con 0x7f9f5810e760 2024-04-16T17:10:14.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.693+0000 7f9f5e800640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f9f58114330 con 0x7f9f58074cd0 2024-04-16T17:10:14.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.693+0000 7f9f5e800640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f9f58114630 con 0x7f9f580743b0 2024-04-16T17:10:14.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.693+0000 7f9f5ca00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9f58074cd0 0x7f9f581a0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:14.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.693+0000 7f9f5ca00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9f58074cd0 0x7f9f581a0bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:39936/0 (socket says 172.21.15.45:39936) 2024-04-16T17:10:14.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.693+0000 7f9f5ca00640 1 -- 172.21.15.45:0/1787254327 learned_addr learned my addr 172.21.15.45:0/1787254327 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:10:14.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.693+0000 7f9f5de00640 1 --2- 172.21.15.45:0/1787254327 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9f5810e760 0x7f9f5819a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:14.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.693+0000 7f9f5ca00640 1 -- 172.21.15.45:0/1787254327 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9f580743b0 msgr2=0x7f9f581a0690 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-16T17:10:14.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.693+0000 7f9f5ca00640 1 --2- 172.21.15.45:0/1787254327 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9f580743b0 0x7f9f581a0690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:14.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.693+0000 7f9f5ca00640 1 -- 172.21.15.45:0/1787254327 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9f5810e760 msgr2=0x7f9f5819a850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:14.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.693+0000 7f9f5ca00640 1 --2- 172.21.15.45:0/1787254327 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9f5810e760 0x7f9f5819a850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:14.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.693+0000 7f9f5ca00640 1 -- 172.21.15.45:0/1787254327 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f5819b020 con 0x7f9f58074cd0 2024-04-16T17:10:14.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.693+0000 7f9f5de00640 1 --2- 172.21.15.45:0/1787254327 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9f5810e760 0x7f9f5819a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:10:14.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.693+0000 7f9f5ca00640 1 --2- 172.21.15.45:0/1787254327 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9f58074cd0 0x7f9f581a0bd0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f9f4400e9f0 tx=0x7f9f4400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:10:14.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.693+0000 7f9f56000640 1 -- 172.21.15.45:0/1787254327 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f4400cd80 con 0x7f9f58074cd0 2024-04-16T17:10:14.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.693+0000 7f9f56000640 1 -- 172.21.15.45:0/1787254327 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f44010910 con 0x7f9f58074cd0 2024-04-16T17:10:14.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.693+0000 7f9f56000640 1 -- 172.21.15.45:0/1787254327 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f44018990 con 0x7f9f58074cd0 2024-04-16T17:10:14.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.693+0000 7f9f5e800640 1 -- 172.21.15.45:0/1787254327 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f5819b310 con 0x7f9f58074cd0 2024-04-16T17:10:14.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.693+0000 7f9f5e800640 1 -- 172.21.15.45:0/1787254327 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f581a74e0 con 0x7f9f58074cd0 2024-04-16T17:10:14.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.700+0000 7f9f62b6f740 1 Processor -- start 2024-04-16T17:10:14.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.700+0000 7f9f62b6f740 1 -- start start 2024-04-16T17:10:14.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.700+0000 7f9f56000640 1 -- 172.21.15.45:0/1787254327 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f9f44010ac0 con 0x7f9f58074cd0 2024-04-16T17:10:14.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.700+0000 7f9f62b6f740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f26bfd5390 0x55f26bfd5770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:14.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.700+0000 7f9f62b6f740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f26bfd5cb0 0x55f26bfde350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:14.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.700+0000 7f9f62b6f740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f26bfde890 0x55f26bfe0c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:14.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.700+0000 7f9f62b6f740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55f26bf3b120 con 0x55f26bfde890 2024-04-16T17:10:14.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.700+0000 7f9f62b6f740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55f26bee1340 con 0x55f26bfd5390 2024-04-16T17:10:14.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.700+0000 7f9f62b6f740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55f26bf370e0 con 0x55f26bfd5cb0 2024-04-16T17:10:14.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.700+0000 7f9f5de00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f26bfd5390 0x55f26bfd5770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:14.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.700+0000 7f9f5de00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f26bfd5390 0x55f26bfd5770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:39942/0 (socket says 172.21.15.45:39942) 2024-04-16T17:10:14.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.700+0000 7f9f5de00640 1 -- 172.21.15.45:0/2520095190 learned_addr learned my addr 172.21.15.45:0/2520095190 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:10:14.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.700+0000 7f9f5de00640 1 -- 172.21.15.45:0/2520095190 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f26bfd5cb0 msgr2=0x55f26bfde350 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-16T17:10:14.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.700+0000 7f9f5de00640 1 --2- 172.21.15.45:0/2520095190 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f26bfd5cb0 0x55f26bfde350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:14.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.700+0000 7f9f5de00640 1 -- 172.21.15.45:0/2520095190 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f26bfde890 msgr2=0x55f26bfe0c80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-16T17:10:14.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.700+0000 7f9f5de00640 1 --2- 172.21.15.45:0/2520095190 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f26bfde890 0x55f26bfe0c80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:14.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.700+0000 7f9f5de00640 1 -- 172.21.15.45:0/2520095190 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f26bd177d0 con 0x55f26bfd5390 2024-04-16T17:10:14.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.700+0000 7f9f56000640 1 --2- 172.21.15.45:0/1787254327 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f9f400a3da0 0x7f9f400a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:14.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.700+0000 7f9f5de00640 1 --2- 172.21.15.45:0/2520095190 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f26bfd5390 0x55f26bfd5770 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f9f5000ae80 tx=0x7f9f5000e720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c75434f2357280e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:10:14.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.700+0000 7f9f56000640 1 -- 172.21.15.45:0/1787254327 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f9f440d11c0 con 0x7f9f58074cd0 2024-04-16T17:10:14.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.700+0000 7f9f5d400640 1 --2- 172.21.15.45:0/1787254327 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f9f400a3da0 0x7f9f400a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:14.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.700+0000 7f9f4a600640 1 -- 172.21.15.45:0/2520095190 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f50003b70 con 0x55f26bfd5390 2024-04-16T17:10:14.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.700+0000 7f9f4a600640 1 -- 172.21.15.45:0/2520095190 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f50003d10 con 0x55f26bfd5390 2024-04-16T17:10:14.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.700+0000 7f9f4a600640 1 -- 172.21.15.45:0/2520095190 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f50010be0 con 0x55f26bfd5390 2024-04-16T17:10:14.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.700+0000 7f9f62b6f740 1 -- 172.21.15.45:0/2520095190 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f26bfd5390 msgr2=0x55f26bfd5770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:14.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.700+0000 7f9f5d400640 1 --2- 172.21.15.45:0/1787254327 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f9f400a3da0 0x7f9f400a6260 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f9f4c036040 tx=0x7f9f4c005fd0 comp rx=0 tx=0).ready entity=mgr.25411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:10:14.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.700+0000 7f9f62b6f740 1 --2- 172.21.15.45:0/2520095190 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f26bfd5390 0x55f26bfd5770 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f9f5000ae80 tx=0x7f9f5000e720 comp rx=0 tx=0).stop 2024-04-16T17:10:14.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.700+0000 7f9f62b6f740 1 -- 172.21.15.45:0/2520095190 shutdown_connections 2024-04-16T17:10:14.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.700+0000 7f9f62b6f740 1 --2- 172.21.15.45:0/2520095190 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f26bfd5cb0 0x55f26bfde350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:14.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.700+0000 7f9f62b6f740 1 --2- 172.21.15.45:0/2520095190 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f26bfde890 0x55f26bfe0c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:14.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.700+0000 7f9f62b6f740 1 --2- 172.21.15.45:0/2520095190 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f26bfd5390 0x55f26bfd5770 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:14.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.700+0000 7f9f62b6f740 1 -- 172.21.15.45:0/2520095190 >> 172.21.15.45:0/2520095190 conn(0x55f26bfcfac0 msgr2=0x55f26bfcfea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:10:14.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.700+0000 7f9f62b6f740 1 -- 172.21.15.45:0/2520095190 shutdown_connections 2024-04-16T17:10:14.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.700+0000 7f9f62b6f740 1 -- 172.21.15.45:0/2520095190 wait complete. 2024-04-16T17:10:14.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.700+0000 7f9f62b6f740 1 -- start start 2024-04-16T17:10:14.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.700+0000 7f9f62b6f740 1 Processor -- start 2024-04-16T17:10:14.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.700+0000 7f9f62b6f740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f26bfd5390 0x55f26bfdba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:14.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.700+0000 7f9f62b6f740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f26bfd5cb0 0x55f26bfdbfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:14.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.700+0000 7f9f62b6f740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f26bfde890 0x55f26bfdc4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:14.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.700+0000 7f9f62b6f740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55f26bfe1680 con 0x55f26bfd5390 2024-04-16T17:10:14.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.700+0000 7f9f62b6f740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55f26bef4710 con 0x55f26bfde890 2024-04-16T17:10:14.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.700+0000 7f9f62b6f740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55f26bfe1800 con 0x55f26bfd5cb0 2024-04-16T17:10:14.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.700+0000 7f9f62b6f740 10 client.0 _ll_register_callbacks cb 0x55f26beefaf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:10:14.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.700+0000 7f9f5ca00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f26bfd5cb0 0x55f26bfdbfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:14.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.700+0000 7f9f5d400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f26bfd5390 0x55f26bfdba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:14.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.700+0000 7f9f5ca00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f26bfd5cb0 0x55f26bfdbfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:33294/0 (socket says 172.21.15.45:33294) 2024-04-16T17:10:14.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.700+0000 7f9f5ca00640 1 -- 172.21.15.45:0/3726501565 learned_addr learned my addr 172.21.15.45:0/3726501565 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:10:14.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.700+0000 7f9f5de00640 1 --2- 172.21.15.45:0/3726501565 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f26bfde890 0x55f26bfdc4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:14.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.706+0000 7f9f5ca00640 1 -- 172.21.15.45:0/3726501565 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f26bfde890 msgr2=0x55f26bfdc4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:10:14.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.706+0000 7f9f5ca00640 1 --2- 172.21.15.45:0/3726501565 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f26bfde890 0x55f26bfdc4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:14.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.706+0000 7f9f5ca00640 1 -- 172.21.15.45:0/3726501565 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f26bfd5390 msgr2=0x55f26bfdba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:10:14.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.706+0000 7f9f5ca00640 1 --2- 172.21.15.45:0/3726501565 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f26bfd5390 0x55f26bfdba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:14.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.706+0000 7f9f5ca00640 1 -- 172.21.15.45:0/3726501565 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f26bc4d3f0 con 0x55f26bfd5cb0 2024-04-16T17:10:14.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.706+0000 7f9f5ca00640 1 --2- 172.21.15.45:0/3726501565 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f26bfd5cb0 0x55f26bfdbfb0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f9f4409cab0 tx=0x7f9f4409cdb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cc5269cabbf8961b server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:10:14.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.706+0000 7f9f3ea00640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-04-16T17:10:14.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.706+0000 7f9f3ea00640 1 -- 172.21.15.45:0/3726501565 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f440ce9f0 con 0x55f26bfd5cb0 2024-04-16T17:10:14.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.706+0000 7f9f3ea00640 1 -- 172.21.15.45:0/3726501565 <== mon.1 v2:172.21.15.117:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f9f440a5790 con 0x55f26bfd5cb0 2024-04-16T17:10:14.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.706+0000 7f9f62b6f740 1 -- 172.21.15.45:0/3726501565 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9f50003d10 con 0x55f26bfd5cb0 2024-04-16T17:10:14.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.706+0000 7f9f3ea00640 1 -- 172.21.15.45:0/3726501565 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f9f440b6200 con 0x55f26bfd5cb0 2024-04-16T17:10:14.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.706+0000 7f9f62b6f740 10 client.26521 fetch_fsmap learned FSMap version 30 2024-04-16T17:10:14.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.706+0000 7f9f62b6f740 1 -- 172.21.15.45:0/3726501565 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55f26bc0d0f0 con 0x55f26bfd5cb0 2024-04-16T17:10:14.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.706+0000 7f9f3ea00640 1 -- 172.21.15.45:0/3726501565 <== mon.1 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f50003d10 con 0x55f26bfd5cb0 2024-04-16T17:10:14.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.706+0000 7f9f3ea00640 1 -- 172.21.15.45:0/3726501565 <== mon.1 v2:172.21.15.117:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f9f44087040 con 0x55f26bfd5cb0 2024-04-16T17:10:14.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.706+0000 7f9f62b6f740 10 client.26521 fetch_fsmap finished waiting for FSMap version 30 2024-04-16T17:10:14.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.706+0000 7f9f62b6f740 10 client.26521 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:10:14.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.706+0000 7f9f62b6f740 20 client.26521 populate_metadata read hostname 'smithi045' 2024-04-16T17:10:14.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.706+0000 7f9f62b6f740 1 --2- 172.21.15.45:0/3726501565 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55f26bfeed80 0x55f26bff1170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:14.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.706+0000 7f9f62b6f740 4 client.26521 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:10:14.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.706+0000 7f9f62b6f740 1 -- 172.21.15.45:0/3726501565 --> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f26bf48b40 con 0x55f26bfeed80 2024-04-16T17:10:14.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.708+0000 7f9f5d400640 1 --2- 172.21.15.45:0/3726501565 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55f26bfeed80 0x55f26bff1170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:14.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.708+0000 7f9f5d400640 1 --2- 172.21.15.45:0/3726501565 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55f26bfeed80 0x55f26bff1170 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4aee807634aed791 server_cookie=26575ec54ac0de92 in_seq=0 out_seq=0 2024-04-16T17:10:14.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.708+0000 7f9f3ea00640 10 client.26521 ms_handle_connect on v2:172.21.15.104:6838/3660070592 2024-04-16T17:10:14.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.708+0000 7f9f3ea00640 1 -- 172.21.15.45:0/3726501565 <== mds.0 v2:172.21.15.104:6838/3660070592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f26bf48b40 con 0x55f26bfeed80 2024-04-16T17:10:14.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.708+0000 7f9f3ea00640 10 client.26521 handle_command_reply: tid=0 2024-04-16T17:10:14.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.708+0000 7f9f62b6f740 1 client.26521 shutdown 2024-04-16T17:10:14.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.708+0000 7f9f62b6f740 1 -- 172.21.15.45:0/3726501565 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f26bfd5cb0 msgr2=0x55f26bfdbfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:14.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.708+0000 7f9f62b6f740 1 --2- 172.21.15.45:0/3726501565 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f26bfd5cb0 0x55f26bfdbfb0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f9f4409cab0 tx=0x7f9f4409cdb0 comp rx=0 tx=0).stop 2024-04-16T17:10:14.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.708+0000 7f9f62b6f740 1 -- 172.21.15.45:0/3726501565 shutdown_connections 2024-04-16T17:10:14.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.708+0000 7f9f62b6f740 1 --2- 172.21.15.45:0/3726501565 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55f26bfeed80 0x55f26bff1170 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:14.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.708+0000 7f9f62b6f740 1 --2- 172.21.15.45:0/3726501565 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f26bfd5cb0 0x55f26bfdbfb0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:14.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.708+0000 7f9f62b6f740 1 --2- 172.21.15.45:0/3726501565 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f26bfde890 0x55f26bfdc4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:14.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.708+0000 7f9f3ea00640 0 client.26521 ms_handle_reset on v2:172.21.15.104:6838/3660070592 2024-04-16T17:10:14.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.708+0000 7f9f62b6f740 1 --2- 172.21.15.45:0/3726501565 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f26bfd5390 0x55f26bfdba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:14.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.708+0000 7f9f62b6f740 1 -- 172.21.15.45:0/3726501565 >> 172.21.15.45:0/3726501565 conn(0x55f26bfcfac0 msgr2=0x55f26bfd1810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:10:14.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.708+0000 7f9f62b6f740 1 -- 172.21.15.45:0/3726501565 shutdown_connections 2024-04-16T17:10:14.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.708+0000 7f9f62b6f740 1 -- 172.21.15.45:0/3726501565 wait complete. 2024-04-16T17:10:14.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.708+0000 7f9f62b6f740 20 client.26521 trim_cache size 0 max 16384 2024-04-16T17:10:14.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.728+0000 7f9f62b6f740 1 Processor -- start 2024-04-16T17:10:14.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.728+0000 7f9f62b6f740 1 -- start start 2024-04-16T17:10:14.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.728+0000 7f9f62b6f740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f26bfd5390 0x55f26bfde1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:14.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.728+0000 7f9f62b6f740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f26bfd5cb0 0x55f26bfdc310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:14.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.728+0000 7f9f62b6f740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f26bfde890 0x55f26bff69f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:14.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.728+0000 7f9f62b6f740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f9f581b3f70 con 0x55f26bfd5cb0 2024-04-16T17:10:14.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.728+0000 7f9f62b6f740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f9f581b1780 con 0x55f26bfde890 2024-04-16T17:10:14.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.728+0000 7f9f62b6f740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f9f581af5b0 con 0x55f26bfd5390 2024-04-16T17:10:14.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.728+0000 7f9f5de00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f26bfde890 0x55f26bff69f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:14.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.728+0000 7f9f5ca00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f26bfd5cb0 0x55f26bfdc310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:14.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.728+0000 7f9f5d400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f26bfd5390 0x55f26bfde1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:14.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.728+0000 7f9f5de00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f26bfde890 0x55f26bff69f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:39966/0 (socket says 172.21.15.45:39966) 2024-04-16T17:10:14.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.728+0000 7f9f5de00640 1 -- 172.21.15.45:0/3467878345 learned_addr learned my addr 172.21.15.45:0/3467878345 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:10:14.731 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:10:14 smithi104 ceph-mon[36457]: pgmap v107: 129 pgs: 129 active+clean; 31 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 112 KiB/s wr, 40 op/s 2024-04-16T17:10:14.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.730+0000 7f9f5de00640 1 -- 172.21.15.45:0/3467878345 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f26bfd5390 msgr2=0x55f26bfde1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:10:14.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.730+0000 7f9f5de00640 1 --2- 172.21.15.45:0/3467878345 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f26bfd5390 0x55f26bfde1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:14.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.730+0000 7f9f5de00640 1 -- 172.21.15.45:0/3467878345 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f26bfd5cb0 msgr2=0x55f26bfdc310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:10:14.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.730+0000 7f9f5de00640 1 --2- 172.21.15.45:0/3467878345 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f26bfd5cb0 0x55f26bfdc310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:14.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.730+0000 7f9f5de00640 1 -- 172.21.15.45:0/3467878345 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f26bff14e0 con 0x55f26bfde890 2024-04-16T17:10:14.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.730+0000 7f9f5de00640 1 --2- 172.21.15.45:0/3467878345 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f26bfde890 0x55f26bff69f0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f9f5000ae80 tx=0x7f9f5000a620 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5c52cf308ff75c77 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:10:14.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.731+0000 7f9f49c00640 1 -- 172.21.15.45:0/3467878345 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f5000a990 con 0x55f26bfde890 2024-04-16T17:10:14.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.731+0000 7f9f49c00640 1 -- 172.21.15.45:0/3467878345 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f50003e50 con 0x55f26bfde890 2024-04-16T17:10:14.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.731+0000 7f9f49c00640 1 -- 172.21.15.45:0/3467878345 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f50004020 con 0x55f26bfde890 2024-04-16T17:10:14.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.732+0000 7f9f62b6f740 1 -- 172.21.15.45:0/3467878345 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f26bfde890 msgr2=0x55f26bff69f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:14.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.732+0000 7f9f62b6f740 1 --2- 172.21.15.45:0/3467878345 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f26bfde890 0x55f26bff69f0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f9f5000ae80 tx=0x7f9f5000a620 comp rx=0 tx=0).stop 2024-04-16T17:10:14.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.732+0000 7f9f62b6f740 1 -- 172.21.15.45:0/3467878345 shutdown_connections 2024-04-16T17:10:14.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.732+0000 7f9f62b6f740 1 --2- 172.21.15.45:0/3467878345 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f26bfde890 0x55f26bff69f0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:14.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.732+0000 7f9f62b6f740 1 --2- 172.21.15.45:0/3467878345 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f26bfd5cb0 0x55f26bfdc310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:14.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.732+0000 7f9f62b6f740 1 --2- 172.21.15.45:0/3467878345 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f26bfd5390 0x55f26bfde1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:14.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.732+0000 7f9f62b6f740 1 -- 172.21.15.45:0/3467878345 >> 172.21.15.45:0/3467878345 conn(0x55f26bfcfac0 msgr2=0x55f26bfd0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:10:14.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.732+0000 7f9f62b6f740 1 -- 172.21.15.45:0/3467878345 shutdown_connections 2024-04-16T17:10:14.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.732+0000 7f9f62b6f740 1 -- 172.21.15.45:0/3467878345 wait complete. 2024-04-16T17:10:14.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.732+0000 7f9f62b6f740 1 -- start start 2024-04-16T17:10:14.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.732+0000 7f9f62b6f740 1 Processor -- start 2024-04-16T17:10:14.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.733+0000 7f9f62b6f740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f26bfd5390 0x55f26bfe6f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:14.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.733+0000 7f9f62b6f740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f26bfd5cb0 0x55f26bfd8590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:14.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.733+0000 7f9f62b6f740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f26bfde890 0x55f26bfd8c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:14.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.733+0000 7f9f62b6f740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f9f581afc40 con 0x55f26bfd5cb0 2024-04-16T17:10:14.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.733+0000 7f9f62b6f740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f9f581b10b0 con 0x55f26bfd5390 2024-04-16T17:10:14.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.733+0000 7f9f62b6f740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f9f581b0310 con 0x55f26bfde890 2024-04-16T17:10:14.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.733+0000 7f9f62b6f740 10 client.0 _ll_register_callbacks cb 0x7f9f4400cf20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:10:14.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.733+0000 7f9f5ca00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f26bfd5cb0 0x55f26bfd8590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:14.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.733+0000 7f9f5ca00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f26bfd5cb0 0x55f26bfd8590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39668/0 (socket says 172.21.15.45:39668) 2024-04-16T17:10:14.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.733+0000 7f9f5ca00640 1 -- 172.21.15.45:0/4070007770 learned_addr learned my addr 172.21.15.45:0/4070007770 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:10:14.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.733+0000 7f9f5de00640 1 --2- 172.21.15.45:0/4070007770 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f26bfde890 0x55f26bfd8c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:14.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.734+0000 7f9f5ca00640 1 -- 172.21.15.45:0/4070007770 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f26bfde890 msgr2=0x55f26bfd8c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:10:14.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.734+0000 7f9f5ca00640 1 --2- 172.21.15.45:0/4070007770 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f26bfde890 0x55f26bfd8c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:14.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.734+0000 7f9f5ca00640 1 -- 172.21.15.45:0/4070007770 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f26bfd5390 msgr2=0x55f26bfe6f50 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-16T17:10:14.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.734+0000 7f9f5ca00640 1 --2- 172.21.15.45:0/4070007770 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f26bfd5390 0x55f26bfe6f50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:14.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.734+0000 7f9f5ca00640 1 -- 172.21.15.45:0/4070007770 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f26c054260 con 0x55f26bfd5cb0 2024-04-16T17:10:14.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.734+0000 7f9f5ca00640 1 --2- 172.21.15.45:0/4070007770 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f26bfd5cb0 0x55f26bfd8590 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f9f4409cd40 tx=0x7f9f44018420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4dacc5945d0d70b3 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:10:14.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.734+0000 7f9f3ea00640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-16T17:10:14.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.734+0000 7f9f3ea00640 1 -- 172.21.15.45:0/4070007770 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f440ce9f0 con 0x55f26bfd5cb0 2024-04-16T17:10:14.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.734+0000 7f9f3ea00640 1 -- 172.21.15.45:0/4070007770 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f9f44018d40 con 0x55f26bfd5cb0 2024-04-16T17:10:14.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.734+0000 7f9f62b6f740 1 -- 172.21.15.45:0/4070007770 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9f50003e50 con 0x55f26bfd5cb0 2024-04-16T17:10:14.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.735+0000 7f9f3ea00640 1 -- 172.21.15.45:0/4070007770 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f9f440aa050 con 0x55f26bfd5cb0 2024-04-16T17:10:14.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.735+0000 7f9f62b6f740 10 client.16404 fetch_fsmap learned FSMap version 30 2024-04-16T17:10:14.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.735+0000 7f9f62b6f740 1 -- 172.21.15.45:0/4070007770 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55f26c055380 con 0x55f26bfd5cb0 2024-04-16T17:10:14.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.735+0000 7f9f3ea00640 1 -- 172.21.15.45:0/4070007770 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f50003e50 con 0x55f26bfd5cb0 2024-04-16T17:10:14.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.735+0000 7f9f3ea00640 1 -- 172.21.15.45:0/4070007770 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f9f44087050 con 0x55f26bfd5cb0 2024-04-16T17:10:14.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.735+0000 7f9f62b6f740 10 client.16404 fetch_fsmap finished waiting for FSMap version 30 2024-04-16T17:10:14.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.735+0000 7f9f62b6f740 10 client.16404 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:10:14.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.735+0000 7f9f62b6f740 20 client.16404 populate_metadata read hostname 'smithi045' 2024-04-16T17:10:14.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.735+0000 7f9f62b6f740 1 --2- 172.21.15.45:0/4070007770 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55f26c055b20 0x55f26c057f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:14.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.735+0000 7f9f62b6f740 4 client.16404 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-16T17:10:14.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.735+0000 7f9f62b6f740 1 -- 172.21.15.45:0/4070007770 --> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55f26bf429b0 con 0x55f26c055b20 2024-04-16T17:10:14.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.745+0000 7f9f5d400640 1 --2- 172.21.15.45:0/4070007770 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55f26c055b20 0x55f26c057f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:14.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.753+0000 7f9f5d400640 1 --2- 172.21.15.45:0/4070007770 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55f26c055b20 0x55f26c057f40 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d6f2bd14d8efd40b server_cookie=103ee999e3b9a6d5 in_seq=0 out_seq=0 2024-04-16T17:10:14.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.753+0000 7f9f3ea00640 10 client.16404 ms_handle_connect on v2:172.21.15.104:6838/3660070592 2024-04-16T17:10:14.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.753+0000 7f9f3ea00640 1 -- 172.21.15.45:0/4070007770 <== mds.0 v2:172.21.15.104:6838/3660070592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55f26bf429b0 con 0x55f26c055b20 2024-04-16T17:10:14.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.753+0000 7f9f3ea00640 10 client.16404 handle_command_reply: tid=0 2024-04-16T17:10:14.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.753+0000 7f9f62b6f740 1 client.16404 shutdown 2024-04-16T17:10:14.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.753+0000 7f9f62b6f740 1 -- 172.21.15.45:0/4070007770 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f26bfd5cb0 msgr2=0x55f26bfd8590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:14.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.753+0000 7f9f62b6f740 1 --2- 172.21.15.45:0/4070007770 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f26bfd5cb0 0x55f26bfd8590 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f9f4409cd40 tx=0x7f9f44018420 comp rx=0 tx=0).stop 2024-04-16T17:10:14.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.753+0000 7f9f62b6f740 1 -- 172.21.15.45:0/4070007770 shutdown_connections 2024-04-16T17:10:14.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.753+0000 7f9f62b6f740 1 --2- 172.21.15.45:0/4070007770 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55f26c055b20 0x55f26c057f40 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:14.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.753+0000 7f9f62b6f740 1 --2- 172.21.15.45:0/4070007770 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f26bfde890 0x55f26bfd8c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:14.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.753+0000 7f9f62b6f740 1 --2- 172.21.15.45:0/4070007770 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f26bfd5cb0 0x55f26bfd8590 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:14.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.753+0000 7f9f62b6f740 1 --2- 172.21.15.45:0/4070007770 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f26bfd5390 0x55f26bfe6f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:14.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.753+0000 7f9f62b6f740 1 -- 172.21.15.45:0/4070007770 >> 172.21.15.45:0/4070007770 conn(0x55f26bfcfac0 msgr2=0x55f26bfd14e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:10:14.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.754+0000 7f9f3ea00640 0 client.16404 ms_handle_reset on v2:172.21.15.104:6838/3660070592 2024-04-16T17:10:14.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.754+0000 7f9f62b6f740 1 -- 172.21.15.45:0/4070007770 shutdown_connections 2024-04-16T17:10:14.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.754+0000 7f9f62b6f740 1 -- 172.21.15.45:0/4070007770 wait complete. 2024-04-16T17:10:14.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.754+0000 7f9f62b6f740 20 client.16404 trim_cache size 0 max 16384 2024-04-16T17:10:14.755 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-16T17:10:14.755 INFO:teuthology.orchestra.run.smithi045.stdout: "status": "no active scrubs running", 2024-04-16T17:10:14.756 INFO:teuthology.orchestra.run.smithi045.stdout: "scrubs": {} 2024-04-16T17:10:14.756 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-16T17:10:14.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.755+0000 7f9f5e800640 1 -- 172.21.15.45:0/1787254327 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f9f400a3da0 msgr2=0x7f9f400a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:14.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.755+0000 7f9f5e800640 1 --2- 172.21.15.45:0/1787254327 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f9f400a3da0 0x7f9f400a6260 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f9f4c036040 tx=0x7f9f4c005fd0 comp rx=0 tx=0).stop 2024-04-16T17:10:14.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.755+0000 7f9f5e800640 1 -- 172.21.15.45:0/1787254327 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9f58074cd0 msgr2=0x7f9f581a0bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:14.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.755+0000 7f9f5e800640 1 --2- 172.21.15.45:0/1787254327 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9f58074cd0 0x7f9f581a0bd0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f9f4400e9f0 tx=0x7f9f4400eeb0 comp rx=0 tx=0).stop 2024-04-16T17:10:14.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.755+0000 7f9f5e800640 1 -- 172.21.15.45:0/1787254327 shutdown_connections 2024-04-16T17:10:14.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.755+0000 7f9f5e800640 1 --2- 172.21.15.45:0/1787254327 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f9f400a3da0 0x7f9f400a6260 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:14.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.755+0000 7f9f5e800640 1 --2- 172.21.15.45:0/1787254327 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9f5810e760 0x7f9f5819a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:14.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.755+0000 7f9f5e800640 1 --2- 172.21.15.45:0/1787254327 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9f58074cd0 0x7f9f581a0bd0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:14.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.755+0000 7f9f5e800640 1 --2- 172.21.15.45:0/1787254327 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9f580743b0 0x7f9f581a0690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:14.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.755+0000 7f9f5e800640 1 -- 172.21.15.45:0/1787254327 >> 172.21.15.45:0/1787254327 conn(0x7f9f580fd9d0 msgr2=0x7f9f580fe720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:10:14.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.755+0000 7f9f5e800640 1 -- 172.21.15.45:0/1787254327 shutdown_connections 2024-04-16T17:10:14.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.755+0000 7f9f5e800640 1 -- 172.21.15.45:0/1787254327 wait complete. 2024-04-16T17:10:14.770 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-16T17:10:14.770 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-16T17:10:14.804 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:10:14 smithi117 ceph-mon[36730]: pgmap v107: 129 pgs: 129 active+clean; 31 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 112 KiB/s wr, 40 op/s 2024-04-16T17:10:14.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.958+0000 7fa4a6400640 1 -- 172.21.15.45:0/2973240268 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa4a0102a40 msgr2=0x7fa4a0106a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:14.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.958+0000 7fa4a6400640 1 --2- 172.21.15.45:0/2973240268 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa4a0102a40 0x7fa4a0106a80 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fa48c009900 tx=0x7fa48c02bc40 comp rx=0 tx=0).stop 2024-04-16T17:10:14.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.958+0000 7fa4a6400640 1 -- 172.21.15.45:0/2973240268 shutdown_connections 2024-04-16T17:10:14.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.959+0000 7fa4a6400640 1 --2- 172.21.15.45:0/2973240268 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa4a0107160 0x7fa4a010ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:14.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.959+0000 7fa4a6400640 1 --2- 172.21.15.45:0/2973240268 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa4a0102a40 0x7fa4a0106a80 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:14.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.959+0000 7fa4a6400640 1 --2- 172.21.15.45:0/2973240268 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa4a0102090 0x7fa4a0102470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:14.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.959+0000 7fa4a6400640 1 -- 172.21.15.45:0/2973240268 >> 172.21.15.45:0/2973240268 conn(0x7fa4a00fd930 msgr2=0x7fa4a00ffd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:10:14.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.959+0000 7fa4a6400640 1 -- 172.21.15.45:0/2973240268 shutdown_connections 2024-04-16T17:10:14.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.959+0000 7fa4a6400640 1 -- 172.21.15.45:0/2973240268 wait complete. 2024-04-16T17:10:14.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.960+0000 7fa4a6400640 1 Processor -- start 2024-04-16T17:10:14.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.960+0000 7fa4a6400640 1 -- start start 2024-04-16T17:10:14.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.960+0000 7fa4a6400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa4a0102090 0x7fa4a01a0630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:14.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.960+0000 7fa4a6400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa4a0107160 0x7fa4a01a0b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:14.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.961+0000 7fa4a6400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa4a019a6f0 0x7fa4a019ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:14.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.961+0000 7fa4a6400640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fa4a0112220 con 0x7fa4a0107160 2024-04-16T17:10:14.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.961+0000 7fa4a6400640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fa4a01120a0 con 0x7fa4a0102090 2024-04-16T17:10:14.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.961+0000 7fa4a6400640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa4a01123a0 con 0x7fa4a019a6f0 2024-04-16T17:10:14.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.961+0000 7fa4a5000640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa4a0102090 0x7fa4a01a0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:14.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.961+0000 7fa4a5000640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa4a0102090 0x7fa4a01a0630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:39976/0 (socket says 172.21.15.45:39976) 2024-04-16T17:10:14.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.961+0000 7fa4a5000640 1 -- 172.21.15.45:0/711476606 learned_addr learned my addr 172.21.15.45:0/711476606 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:10:14.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.961+0000 7fa49fe00640 1 --2- 172.21.15.45:0/711476606 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa4a0107160 0x7fa4a01a0b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:14.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.961+0000 7fa4a5a00640 1 --2- 172.21.15.45:0/711476606 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa4a019a6f0 0x7fa4a019ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:14.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.961+0000 7fa4a5000640 1 -- 172.21.15.45:0/711476606 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa4a019a6f0 msgr2=0x7fa4a019ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:14.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.961+0000 7fa4a5000640 1 --2- 172.21.15.45:0/711476606 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa4a019a6f0 0x7fa4a019ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:14.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.961+0000 7fa4a5000640 1 -- 172.21.15.45:0/711476606 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa4a0107160 msgr2=0x7fa4a01a0b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:14.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.961+0000 7fa4a5000640 1 --2- 172.21.15.45:0/711476606 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa4a0107160 0x7fa4a01a0b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:14.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.961+0000 7fa4a5000640 1 -- 172.21.15.45:0/711476606 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4a019b2f0 con 0x7fa4a0102090 2024-04-16T17:10:14.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.962+0000 7fa4a5000640 1 --2- 172.21.15.45:0/711476606 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa4a0102090 0x7fa4a01a0630 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fa49400e9f0 tx=0x7fa49400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:10:14.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.962+0000 7fa49d600640 1 -- 172.21.15.45:0/711476606 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa49400cd80 con 0x7fa4a0102090 2024-04-16T17:10:14.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.962+0000 7fa49d600640 1 -- 172.21.15.45:0/711476606 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa494010910 con 0x7fa4a0102090 2024-04-16T17:10:14.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.962+0000 7fa49d600640 1 -- 172.21.15.45:0/711476606 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa494018990 con 0x7fa4a0102090 2024-04-16T17:10:14.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.962+0000 7fa4a6400640 1 -- 172.21.15.45:0/711476606 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4a01a73d0 con 0x7fa4a0102090 2024-04-16T17:10:14.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.962+0000 7fa4a6400640 1 -- 172.21.15.45:0/711476606 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4a01a77d0 con 0x7fa4a0102090 2024-04-16T17:10:14.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.964+0000 7fa4a6400640 1 -- 172.21.15.45:0/711476606 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4a0103560 con 0x7fa4a0102090 2024-04-16T17:10:14.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.965+0000 7fa49d600640 1 -- 172.21.15.45:0/711476606 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fa494010ac0 con 0x7fa4a0102090 2024-04-16T17:10:14.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.968+0000 7fa49d600640 1 --2- 172.21.15.45:0/711476606 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fa4880a3da0 0x7fa4880a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:14.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.968+0000 7fa49d600640 1 -- 172.21.15.45:0/711476606 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fa4940d0ee0 con 0x7fa4a0102090 2024-04-16T17:10:14.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.969+0000 7fa49fe00640 1 --2- 172.21.15.45:0/711476606 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fa4880a3da0 0x7fa4880a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:14.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.970+0000 7fa49fe00640 1 --2- 172.21.15.45:0/711476606 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fa4880a3da0 0x7fa4880a6260 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fa48c02c250 tx=0x7fa48c005fd0 comp rx=0 tx=0).ready entity=mgr.25411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:10:14.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:14.971+0000 7fa49d600640 1 -- 172.21.15.45:0/711476606 <== mon.2 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa49409d1d0 con 0x7fa4a0102090 2024-04-16T17:10:15.230 INFO:tasks.workunit.client.0.smithi045.stdout: 1 44219 0.38 MB/sec execute 17 sec latency 102.816 ms 2024-04-16T17:10:15.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.232+0000 7fa4a6400640 1 -- 172.21.15.45:0/711476606 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa4a019b4c0 con 0x7fa4a0102090 2024-04-16T17:10:15.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.237+0000 7fa49d600640 1 -- 172.21.15.45:0/711476606 <== mon.2 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11896 (secure 0 0 0) 0x7fa49409cad0 con 0x7fa4a0102090 2024-04-16T17:10:15.238 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-16T17:10:15.238 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/1748842443","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":1748842443},{"type":"v1","addr":"172.21.15.45:6835","nonce":1748842443}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6839/1308627051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":1308627051},{"type":"v1","addr":"172.21.15.45:6839","nonce":1308627051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/1768242489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1768242489},{"type":"v1","addr":"172.21.15.117:6833","nonce":1768242489}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6833/3558902585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":3558902585},{"type":"v1","addr":"172.21.15.104:6833","nonce":3558902585}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24478,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/1308142903","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":1308142903},{"type":"v1","addr":"172.21.15.45:6837","nonce":1308142903}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24491,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6835/3430679937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":3430679937},{"type":"v1","addr":"172.21.15.104:6835","nonce":3430679937}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-16T17:03:24.663611+0000","modified":"2024-04-16T17:09:20.713725+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24581,"mds_1":24569,"mds_2":24532},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24499":{"gid":24499,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/2224123841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2224123841},{"type":"v1","addr":"172.21.15.117:6835","nonce":2224123841}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6837/2471356815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":2471356815},{"type":"v1","addr":"172.21.15.117:6837","nonce":2471356815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24524":{"gid":24524,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6837/2647756509","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":2647756509},{"type":"v1","addr":"172.21.15.104:6837","nonce":2647756509}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24532":{"gid":24532,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":13,"addr":"172.21.15.117:6839/2378675949","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":2378675949},{"type":"v1","addr":"172.21.15.117:6839","nonce":2378675949}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24569":{"gid":24569,"name":"a","rank":1,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.45:6841/3755310822","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6840","nonce":3755310822},{"type":"v1","addr":"172.21.15.45:6841","nonce":3755310822}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24581":{"gid":24581,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.104:6839/3660070592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6838","nonce":3660070592},{"type":"v1","addr":"172.21.15.104:6839","nonce":3660070592}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24581,"qdb_cluster":[24569,24532,24581]},"id":1}]} 2024-04-16T17:10:15.238 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 30 2024-04-16T17:10:15.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.240+0000 7fa4a6400640 1 -- 172.21.15.45:0/711476606 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fa4880a3da0 msgr2=0x7fa4880a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:15.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.240+0000 7fa4a6400640 1 --2- 172.21.15.45:0/711476606 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fa4880a3da0 0x7fa4880a6260 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fa48c02c250 tx=0x7fa48c005fd0 comp rx=0 tx=0).stop 2024-04-16T17:10:15.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.240+0000 7fa4a6400640 1 -- 172.21.15.45:0/711476606 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa4a0102090 msgr2=0x7fa4a01a0630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:15.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.240+0000 7fa4a6400640 1 --2- 172.21.15.45:0/711476606 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa4a0102090 0x7fa4a01a0630 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fa49400e9f0 tx=0x7fa49400eeb0 comp rx=0 tx=0).stop 2024-04-16T17:10:15.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.240+0000 7fa4a6400640 1 -- 172.21.15.45:0/711476606 shutdown_connections 2024-04-16T17:10:15.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.240+0000 7fa4a6400640 1 --2- 172.21.15.45:0/711476606 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fa4880a3da0 0x7fa4880a6260 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:15.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.240+0000 7fa4a6400640 1 --2- 172.21.15.45:0/711476606 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa4a019a6f0 0x7fa4a019ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:15.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.240+0000 7fa4a6400640 1 --2- 172.21.15.45:0/711476606 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa4a0107160 0x7fa4a01a0b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:15.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.240+0000 7fa4a6400640 1 --2- 172.21.15.45:0/711476606 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa4a0102090 0x7fa4a01a0630 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:15.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.240+0000 7fa4a6400640 1 -- 172.21.15.45:0/711476606 >> 172.21.15.45:0/711476606 conn(0x7fa4a00fd930 msgr2=0x7fa4a00ffbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:10:15.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.240+0000 7fa4a6400640 1 -- 172.21.15.45:0/711476606 shutdown_connections 2024-04-16T17:10:15.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.241+0000 7fa4a6400640 1 -- 172.21.15.45:0/711476606 wait complete. 2024-04-16T17:10:15.263 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-16T17:10:15.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.475+0000 7f9b46c00640 1 -- 172.21.15.45:0/3788501241 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9b40102c40 msgr2=0x7f9b40106c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:15.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.475+0000 7f9b46c00640 1 --2- 172.21.15.45:0/3788501241 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9b40102c40 0x7f9b40106c80 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f9b2c009900 tx=0x7f9b2c02bc60 comp rx=0 tx=0).stop 2024-04-16T17:10:15.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.475+0000 7f9b46c00640 1 -- 172.21.15.45:0/3788501241 shutdown_connections 2024-04-16T17:10:15.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.475+0000 7f9b46c00640 1 --2- 172.21.15.45:0/3788501241 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9b40107360 0x7f9b4010ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:15.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.475+0000 7f9b46c00640 1 --2- 172.21.15.45:0/3788501241 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9b40102c40 0x7f9b40106c80 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:15.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.475+0000 7f9b46c00640 1 --2- 172.21.15.45:0/3788501241 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9b40102290 0x7f9b40102670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:15.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.475+0000 7f9b46c00640 1 -- 172.21.15.45:0/3788501241 >> 172.21.15.45:0/3788501241 conn(0x7f9b400fdb10 msgr2=0x7f9b400fff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:10:15.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.475+0000 7f9b46c00640 1 -- 172.21.15.45:0/3788501241 shutdown_connections 2024-04-16T17:10:15.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.475+0000 7f9b46c00640 1 -- 172.21.15.45:0/3788501241 wait complete. 2024-04-16T17:10:15.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.475+0000 7f9b46c00640 1 Processor -- start 2024-04-16T17:10:15.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.475+0000 7f9b46c00640 1 -- start start 2024-04-16T17:10:15.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.475+0000 7f9b46c00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9b40102290 0x7f9b40074300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:15.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.481+0000 7f9b46c00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9b40107360 0x7f9b40074840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:15.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.481+0000 7f9b46c00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9b40074d80 0x7f9b401a6c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:15.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.481+0000 7f9b46c00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f9b401124c0 con 0x7f9b40107360 2024-04-16T17:10:15.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.481+0000 7f9b46c00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f9b40112340 con 0x7f9b40074d80 2024-04-16T17:10:15.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.481+0000 7f9b46c00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f9b40112640 con 0x7f9b40102290 2024-04-16T17:10:15.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.481+0000 7f9b3f400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9b40107360 0x7f9b40074840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:15.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.481+0000 7f9b3f400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9b40107360 0x7f9b40074840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39702/0 (socket says 172.21.15.45:39702) 2024-04-16T17:10:15.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.481+0000 7f9b3f400640 1 -- 172.21.15.45:0/3238423378 learned_addr learned my addr 172.21.15.45:0/3238423378 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:10:15.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.481+0000 7f9b44c00640 1 --2- 172.21.15.45:0/3238423378 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9b40074d80 0x7f9b401a6c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:15.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.481+0000 7f9b3f400640 1 -- 172.21.15.45:0/3238423378 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9b40102290 msgr2=0x7f9b40074300 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-16T17:10:15.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.481+0000 7f9b3f400640 1 --2- 172.21.15.45:0/3238423378 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9b40102290 0x7f9b40074300 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:15.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.481+0000 7f9b3f400640 1 -- 172.21.15.45:0/3238423378 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9b40074d80 msgr2=0x7f9b401a6c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:15.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.481+0000 7f9b3f400640 1 --2- 172.21.15.45:0/3238423378 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9b40074d80 0x7f9b401a6c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:15.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.481+0000 7f9b3f400640 1 -- 172.21.15.45:0/3238423378 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b401a72f0 con 0x7f9b40107360 2024-04-16T17:10:15.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.481+0000 7f9b3f400640 1 --2- 172.21.15.45:0/3238423378 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9b40107360 0x7f9b40074840 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f9b2c02c270 tx=0x7f9b2c004360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:10:15.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.481+0000 7f9b3cc00640 1 -- 172.21.15.45:0/3238423378 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b2c041070 con 0x7f9b40107360 2024-04-16T17:10:15.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.481+0000 7f9b3cc00640 1 -- 172.21.15.45:0/3238423378 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9b2c02ccc0 con 0x7f9b40107360 2024-04-16T17:10:15.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.481+0000 7f9b3cc00640 1 -- 172.21.15.45:0/3238423378 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b2c003cf0 con 0x7f9b40107360 2024-04-16T17:10:15.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.481+0000 7f9b46c00640 1 -- 172.21.15.45:0/3238423378 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b401a7580 con 0x7f9b40107360 2024-04-16T17:10:15.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.481+0000 7f9b46c00640 1 -- 172.21.15.45:0/3238423378 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b401a7a70 con 0x7f9b40107360 2024-04-16T17:10:15.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.485+0000 7f9b49842740 1 Processor -- start 2024-04-16T17:10:15.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.485+0000 7f9b49842740 1 -- start start 2024-04-16T17:10:15.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.486+0000 7f9b49842740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5582591b8d50 0x5582591b9130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:15.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.486+0000 7f9b49842740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5582591b9670 0x5582591c1d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:15.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.486+0000 7f9b49842740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5582591c2250 0x5582591c4640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:15.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.486+0000 7f9b49842740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55825911e020 con 0x5582591b9670 2024-04-16T17:10:15.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.486+0000 7f9b49842740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5582590c4460 con 0x5582591b8d50 2024-04-16T17:10:15.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.486+0000 7f9b49842740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x558259119fe0 con 0x5582591c2250 2024-04-16T17:10:15.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.486+0000 7f9b44c00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5582591b8d50 0x5582591b9130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:15.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.486+0000 7f9b3fe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5582591b9670 0x5582591c1d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:15.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.486+0000 7f9b44c00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5582591b8d50 0x5582591b9130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:40010/0 (socket says 172.21.15.45:40010) 2024-04-16T17:10:15.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.486+0000 7f9b44c00640 1 -- 172.21.15.45:0/204233136 learned_addr learned my addr 172.21.15.45:0/204233136 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:10:15.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.486+0000 7f9b3cc00640 1 -- 172.21.15.45:0/3238423378 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f9b2c044070 con 0x7f9b40107360 2024-04-16T17:10:15.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.486+0000 7f9b3fe00640 1 -- 172.21.15.45:0/204233136 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5582591c2250 msgr2=0x5582591c4640 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-16T17:10:15.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.486+0000 7f9b3fe00640 1 --2- 172.21.15.45:0/204233136 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5582591c2250 0x5582591c4640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:15.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.486+0000 7f9b3fe00640 1 -- 172.21.15.45:0/204233136 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5582591b8d50 msgr2=0x5582591b9130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:10:15.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.486+0000 7f9b3fe00640 1 --2- 172.21.15.45:0/204233136 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5582591b8d50 0x5582591b9130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:15.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.487+0000 7f9b3fe00640 1 -- 172.21.15.45:0/204233136 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5582590cc040 con 0x5582591b9670 2024-04-16T17:10:15.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.487+0000 7f9b3fe00640 1 --2- 172.21.15.45:0/204233136 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5582591b9670 0x5582591c1d10 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f9b30009690 tx=0x7f9b30009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=965e61c0c3506a93 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:10:15.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.487+0000 7f9b35200640 1 -- 172.21.15.45:0/204233136 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b30015020 con 0x5582591b9670 2024-04-16T17:10:15.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.487+0000 7f9b35200640 1 -- 172.21.15.45:0/204233136 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9b3000bda0 con 0x5582591b9670 2024-04-16T17:10:15.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.487+0000 7f9b35200640 1 -- 172.21.15.45:0/204233136 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b30010380 con 0x5582591b9670 2024-04-16T17:10:15.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.488+0000 7f9b3cc00640 1 --2- 172.21.15.45:0/3238423378 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f9b240a3d20 0x7f9b240a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:15.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.488+0000 7f9b3cc00640 1 -- 172.21.15.45:0/3238423378 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f9b2c0f28b0 con 0x7f9b40107360 2024-04-16T17:10:15.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.488+0000 7f9b49842740 1 -- 172.21.15.45:0/204233136 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5582591b9670 msgr2=0x5582591c1d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:15.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.488+0000 7f9b49842740 1 --2- 172.21.15.45:0/204233136 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5582591b9670 0x5582591c1d10 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f9b30009690 tx=0x7f9b30009b50 comp rx=0 tx=0).stop 2024-04-16T17:10:15.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.488+0000 7f9b44c00640 1 --2- 172.21.15.45:0/3238423378 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f9b240a3d20 0x7f9b240a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:15.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.488+0000 7f9b49842740 1 -- 172.21.15.45:0/204233136 shutdown_connections 2024-04-16T17:10:15.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.488+0000 7f9b49842740 1 --2- 172.21.15.45:0/204233136 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5582591c2250 0x5582591c4640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:15.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.488+0000 7f9b49842740 1 --2- 172.21.15.45:0/204233136 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5582591b9670 0x5582591c1d10 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:15.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.488+0000 7f9b49842740 1 --2- 172.21.15.45:0/204233136 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5582591b8d50 0x5582591b9130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:15.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.488+0000 7f9b49842740 1 -- 172.21.15.45:0/204233136 >> 172.21.15.45:0/204233136 conn(0x5582591b3480 msgr2=0x5582591b3860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:10:15.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.489+0000 7f9b49842740 1 -- 172.21.15.45:0/204233136 shutdown_connections 2024-04-16T17:10:15.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.489+0000 7f9b49842740 1 -- 172.21.15.45:0/204233136 wait complete. 2024-04-16T17:10:15.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.489+0000 7f9b44c00640 1 --2- 172.21.15.45:0/3238423378 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f9b240a3d20 0x7f9b240a61e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f9b38012640 tx=0x7f9b38009500 comp rx=0 tx=0).ready entity=mgr.25411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:10:15.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.489+0000 7f9b49842740 1 -- start start 2024-04-16T17:10:15.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.489+0000 7f9b49842740 1 Processor -- start 2024-04-16T17:10:15.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.490+0000 7f9b49842740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5582591b8d50 0x5582591bf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:15.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.490+0000 7f9b49842740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5582591b9670 0x5582591bf7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:15.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.490+0000 7f9b49842740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5582591c2250 0x5582591bfd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:15.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.490+0000 7f9b49842740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5582591c5040 con 0x5582591b9670 2024-04-16T17:10:15.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.490+0000 7f9b49842740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5582590d7830 con 0x5582591c2250 2024-04-16T17:10:15.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.490+0000 7f9b49842740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5582591c51c0 con 0x5582591b8d50 2024-04-16T17:10:15.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.490+0000 7f9b49842740 10 client.0 _ll_register_callbacks cb 0x5582590df150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:10:15.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.490+0000 7f9b3f400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5582591b9670 0x5582591bf7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:15.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.490+0000 7f9b3fe00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5582591b8d50 0x5582591bf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:15.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.490+0000 7f9b3fe00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5582591b8d50 0x5582591bf290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:33364/0 (socket says 172.21.15.45:33364) 2024-04-16T17:10:15.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.490+0000 7f9b3fe00640 1 -- 172.21.15.45:0/1518942783 learned_addr learned my addr 172.21.15.45:0/1518942783 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:10:15.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.490+0000 7f9b44c00640 1 --2- 172.21.15.45:0/1518942783 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5582591c2250 0x5582591bfd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:15.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.491+0000 7f9b3fe00640 1 -- 172.21.15.45:0/1518942783 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5582591c2250 msgr2=0x5582591bfd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:10:15.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.491+0000 7f9b3fe00640 1 --2- 172.21.15.45:0/1518942783 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5582591c2250 0x5582591bfd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:15.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.491+0000 7f9b3fe00640 1 -- 172.21.15.45:0/1518942783 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5582591b9670 msgr2=0x5582591bf7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:10:15.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.491+0000 7f9b3fe00640 1 --2- 172.21.15.45:0/1518942783 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5582591b9670 0x5582591bf7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:15.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.491+0000 7f9b3fe00640 1 -- 172.21.15.45:0/1518942783 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5582590db920 con 0x5582591b8d50 2024-04-16T17:10:15.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.491+0000 7f9b3f400640 1 --2- 172.21.15.45:0/1518942783 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5582591b9670 0x5582591bf7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:10:15.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.491+0000 7f9b3fe00640 1 --2- 172.21.15.45:0/1518942783 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5582591b8d50 0x5582591bf290 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f9b30006b40 tx=0x7f9b30012f60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2856956ff783c63 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:10:15.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.491+0000 7f9b29600640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-04-16T17:10:15.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.491+0000 7f9b29600640 1 -- 172.21.15.45:0/1518942783 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b30020070 con 0x5582591b8d50 2024-04-16T17:10:15.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.491+0000 7f9b29600640 1 -- 172.21.15.45:0/1518942783 <== mon.1 v2:172.21.15.117:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f9b3000b8c0 con 0x5582591b8d50 2024-04-16T17:10:15.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.492+0000 7f9b49842740 1 -- 172.21.15.45:0/1518942783 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9b3000bda0 con 0x5582591b8d50 2024-04-16T17:10:15.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.492+0000 7f9b29600640 1 -- 172.21.15.45:0/1518942783 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f9b3000bac0 con 0x5582591b8d50 2024-04-16T17:10:15.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.492+0000 7f9b49842740 10 client.26539 fetch_fsmap learned FSMap version 30 2024-04-16T17:10:15.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.492+0000 7f9b49842740 1 -- 172.21.15.45:0/1518942783 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5582590a97f0 con 0x5582591b8d50 2024-04-16T17:10:15.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.492+0000 7f9b29600640 1 -- 172.21.15.45:0/1518942783 <== mon.1 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9b3000bda0 con 0x5582591b8d50 2024-04-16T17:10:15.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.493+0000 7f9b29600640 1 -- 172.21.15.45:0/1518942783 <== mon.1 v2:172.21.15.117:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f9b30003a30 con 0x5582591b8d50 2024-04-16T17:10:15.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.493+0000 7f9b49842740 10 client.26539 fetch_fsmap finished waiting for FSMap version 30 2024-04-16T17:10:15.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.493+0000 7f9b49842740 10 client.26539 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-16T17:10:15.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.493+0000 7f9b49842740 20 client.26539 populate_metadata read hostname 'smithi045' 2024-04-16T17:10:15.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.493+0000 7f9b49842740 1 --2- 172.21.15.45:0/1518942783 >> [v2:172.21.15.117:6838/2378675949,v1:172.21.15.117:6839/2378675949] conn(0x5582591d2510 0x5582591d4900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:15.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.493+0000 7f9b49842740 4 client.26539 mds_command: new command op to 24532 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:10:15.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.493+0000 7f9b49842740 1 -- 172.21.15.45:0/1518942783 --> [v2:172.21.15.117:6838/2378675949,v1:172.21.15.117:6839/2378675949] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55825912ba40 con 0x5582591d2510 2024-04-16T17:10:15.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.493+0000 7f9b3f400640 1 --2- 172.21.15.45:0/1518942783 >> [v2:172.21.15.117:6838/2378675949,v1:172.21.15.117:6839/2378675949] conn(0x5582591d2510 0x5582591d4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:15.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.494+0000 7f9b3f400640 1 --2- 172.21.15.45:0/1518942783 >> [v2:172.21.15.117:6838/2378675949,v1:172.21.15.117:6839/2378675949] conn(0x5582591d2510 0x5582591d4900 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d4772e1d670fd774 server_cookie=c97111091669e6b8 in_seq=0 out_seq=0 2024-04-16T17:10:15.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.494+0000 7f9b29600640 10 client.26539 ms_handle_connect on v2:172.21.15.117:6838/2378675949 2024-04-16T17:10:15.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.494+0000 7f9b29600640 1 -- 172.21.15.45:0/1518942783 <== mds.2 v2:172.21.15.117:6838/2378675949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55825912ba40 con 0x5582591d2510 2024-04-16T17:10:15.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.494+0000 7f9b29600640 10 client.26539 handle_command_reply: tid=0 2024-04-16T17:10:15.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.494+0000 7f9b49842740 1 client.26539 shutdown 2024-04-16T17:10:15.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.495+0000 7f9b49842740 1 -- 172.21.15.45:0/1518942783 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5582591b8d50 msgr2=0x5582591bf290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:15.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.495+0000 7f9b49842740 1 --2- 172.21.15.45:0/1518942783 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5582591b8d50 0x5582591bf290 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f9b30006b40 tx=0x7f9b30012f60 comp rx=0 tx=0).stop 2024-04-16T17:10:15.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.495+0000 7f9b49842740 1 -- 172.21.15.45:0/1518942783 shutdown_connections 2024-04-16T17:10:15.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.495+0000 7f9b49842740 1 --2- 172.21.15.45:0/1518942783 >> [v2:172.21.15.117:6838/2378675949,v1:172.21.15.117:6839/2378675949] conn(0x5582591d2510 0x5582591d4900 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:15.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.495+0000 7f9b49842740 1 --2- 172.21.15.45:0/1518942783 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5582591c2250 0x5582591bfd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:15.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.495+0000 7f9b49842740 1 --2- 172.21.15.45:0/1518942783 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5582591b9670 0x5582591bf7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:15.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.495+0000 7f9b49842740 1 --2- 172.21.15.45:0/1518942783 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5582591b8d50 0x5582591bf290 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:15.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.495+0000 7f9b49842740 1 -- 172.21.15.45:0/1518942783 >> 172.21.15.45:0/1518942783 conn(0x5582591b3480 msgr2=0x5582591b51d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:10:15.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.495+0000 7f9b29600640 0 client.26539 ms_handle_reset on v2:172.21.15.117:6838/2378675949 2024-04-16T17:10:15.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.495+0000 7f9b49842740 1 -- 172.21.15.45:0/1518942783 shutdown_connections 2024-04-16T17:10:15.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.495+0000 7f9b49842740 1 -- 172.21.15.45:0/1518942783 wait complete. 2024-04-16T17:10:15.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.495+0000 7f9b49842740 20 client.26539 trim_cache size 0 max 16384 2024-04-16T17:10:15.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.513+0000 7f9b49842740 1 Processor -- start 2024-04-16T17:10:15.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.513+0000 7f9b49842740 1 -- start start 2024-04-16T17:10:15.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.513+0000 7f9b49842740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5582591b8d50 0x5582591c19c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:15.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.513+0000 7f9b49842740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5582591b9670 0x5582591cdf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:15.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.513+0000 7f9b49842740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5582591c2250 0x5582591b4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:15.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.513+0000 7f9b49842740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f9b401b40d0 con 0x5582591b8d50 2024-04-16T17:10:15.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.513+0000 7f9b49842740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f9b401b18e0 con 0x5582591b9670 2024-04-16T17:10:15.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.513+0000 7f9b49842740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f9b401af710 con 0x5582591c2250 2024-04-16T17:10:15.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.513+0000 7f9b3fe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5582591b8d50 0x5582591c19c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:15.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.513+0000 7f9b3fe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5582591b8d50 0x5582591c19c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39732/0 (socket says 172.21.15.45:39732) 2024-04-16T17:10:15.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.513+0000 7f9b3fe00640 1 -- 172.21.15.45:0/930557924 learned_addr learned my addr 172.21.15.45:0/930557924 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:10:15.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.514+0000 7f9b44c00640 1 --2- 172.21.15.45:0/930557924 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5582591c2250 0x5582591b4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:15.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.514+0000 7f9b3fe00640 1 -- 172.21.15.45:0/930557924 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5582591c2250 msgr2=0x5582591b4fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:10:15.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.514+0000 7f9b3fe00640 1 --2- 172.21.15.45:0/930557924 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5582591c2250 0x5582591b4fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:15.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.514+0000 7f9b3fe00640 1 -- 172.21.15.45:0/930557924 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5582591b9670 msgr2=0x5582591cdf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:10:15.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.514+0000 7f9b3fe00640 1 --2- 172.21.15.45:0/930557924 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5582591b9670 0x5582591cdf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:15.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.514+0000 7f9b3fe00640 1 -- 172.21.15.45:0/930557924 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5582591d4ea0 con 0x5582591b8d50 2024-04-16T17:10:15.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.514+0000 7f9b44c00640 1 --2- 172.21.15.45:0/930557924 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5582591c2250 0x5582591b4fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:10:15.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.514+0000 7f9b3fe00640 1 --2- 172.21.15.45:0/930557924 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5582591b8d50 0x5582591c19c0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f9b30014040 tx=0x7f9b3001a420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e8920f47f820bbe server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:10:15.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.514+0000 7f9b2be00640 1 -- 172.21.15.45:0/930557924 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b30020070 con 0x5582591b8d50 2024-04-16T17:10:15.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.514+0000 7f9b2be00640 1 -- 172.21.15.45:0/930557924 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9b3000be60 con 0x5582591b8d50 2024-04-16T17:10:15.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.514+0000 7f9b2be00640 1 -- 172.21.15.45:0/930557924 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b30015040 con 0x5582591b8d50 2024-04-16T17:10:15.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.514+0000 7f9b49842740 1 -- 172.21.15.45:0/930557924 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5582591b8d50 msgr2=0x5582591c19c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:15.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.514+0000 7f9b49842740 1 --2- 172.21.15.45:0/930557924 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5582591b8d50 0x5582591c19c0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f9b30014040 tx=0x7f9b3001a420 comp rx=0 tx=0).stop 2024-04-16T17:10:15.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.514+0000 7f9b49842740 1 -- 172.21.15.45:0/930557924 shutdown_connections 2024-04-16T17:10:15.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.514+0000 7f9b49842740 1 --2- 172.21.15.45:0/930557924 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5582591c2250 0x5582591b4fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:15.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.514+0000 7f9b49842740 1 --2- 172.21.15.45:0/930557924 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5582591b9670 0x5582591cdf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:15.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.514+0000 7f9b49842740 1 --2- 172.21.15.45:0/930557924 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5582591b8d50 0x5582591c19c0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:15.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.514+0000 7f9b49842740 1 -- 172.21.15.45:0/930557924 >> 172.21.15.45:0/930557924 conn(0x5582591b3480 msgr2=0x5582591b48d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:10:15.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.514+0000 7f9b49842740 1 -- 172.21.15.45:0/930557924 shutdown_connections 2024-04-16T17:10:15.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.514+0000 7f9b49842740 1 -- 172.21.15.45:0/930557924 wait complete. 2024-04-16T17:10:15.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.514+0000 7f9b49842740 1 -- start start 2024-04-16T17:10:15.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.514+0000 7f9b49842740 1 Processor -- start 2024-04-16T17:10:15.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.514+0000 7f9b49842740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5582591b8d50 0x5582591c6030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:15.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.514+0000 7f9b49842740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5582591b9670 0x5582591c6570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:15.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.514+0000 7f9b49842740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5582591c2250 0x5582591c6ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:15.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.514+0000 7f9b49842740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f9b401afda0 con 0x5582591b9670 2024-04-16T17:10:15.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.514+0000 7f9b49842740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f9b401b1210 con 0x5582591b8d50 2024-04-16T17:10:15.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.514+0000 7f9b49842740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f9b401b0470 con 0x5582591c2250 2024-04-16T17:10:15.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.514+0000 7f9b49842740 10 client.0 _ll_register_callbacks cb 0x7f9b30009510 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:10:15.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.514+0000 7f9b3f400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5582591b9670 0x5582591c6570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:15.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.514+0000 7f9b3f400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5582591b9670 0x5582591c6570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39738/0 (socket says 172.21.15.45:39738) 2024-04-16T17:10:15.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.514+0000 7f9b3f400640 1 -- 172.21.15.45:0/873353108 learned_addr learned my addr 172.21.15.45:0/873353108 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:10:15.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.514+0000 7f9b44c00640 1 --2- 172.21.15.45:0/873353108 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5582591c2250 0x5582591c6ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:15.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.514+0000 7f9b3f400640 1 -- 172.21.15.45:0/873353108 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5582591c2250 msgr2=0x5582591c6ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:10:15.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.514+0000 7f9b3f400640 1 --2- 172.21.15.45:0/873353108 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5582591c2250 0x5582591c6ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:15.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.514+0000 7f9b3f400640 1 -- 172.21.15.45:0/873353108 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5582591b8d50 msgr2=0x5582591c6030 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-16T17:10:15.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.514+0000 7f9b3f400640 1 --2- 172.21.15.45:0/873353108 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5582591b8d50 0x5582591c6030 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:15.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.514+0000 7f9b3f400640 1 -- 172.21.15.45:0/873353108 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5582591b94a0 con 0x5582591b9670 2024-04-16T17:10:15.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.514+0000 7f9b44c00640 1 --2- 172.21.15.45:0/873353108 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5582591c2250 0x5582591c6ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:10:15.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.514+0000 7f9b3f400640 1 --2- 172.21.15.45:0/873353108 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5582591b9670 0x5582591c6570 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f9b2c0be2b0 tx=0x7f9b2c009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b9dec8b0bb44ee server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:10:15.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.514+0000 7f9b29600640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-16T17:10:15.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.519+0000 7f9b29600640 1 -- 172.21.15.45:0/873353108 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b2c0a7070 con 0x5582591b9670 2024-04-16T17:10:15.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.519+0000 7f9b29600640 1 -- 172.21.15.45:0/873353108 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f9b2c0efed0 con 0x5582591b9670 2024-04-16T17:10:15.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.519+0000 7f9b49842740 1 -- 172.21.15.45:0/873353108 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9b3000be60 con 0x5582591b9670 2024-04-16T17:10:15.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.519+0000 7f9b29600640 1 -- 172.21.15.45:0/873353108 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f9b2c04b040 con 0x5582591b9670 2024-04-16T17:10:15.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.519+0000 7f9b49842740 10 client.16428 fetch_fsmap learned FSMap version 30 2024-04-16T17:10:15.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.519+0000 7f9b49842740 1 -- 172.21.15.45:0/873353108 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x558258e303f0 con 0x5582591b9670 2024-04-16T17:10:15.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.519+0000 7f9b29600640 1 -- 172.21.15.45:0/873353108 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9b3000be60 con 0x5582591b9670 2024-04-16T17:10:15.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.519+0000 7f9b29600640 1 -- 172.21.15.45:0/873353108 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f9b2c0be810 con 0x5582591b9670 2024-04-16T17:10:15.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.519+0000 7f9b49842740 10 client.16428 fetch_fsmap finished waiting for FSMap version 30 2024-04-16T17:10:15.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.519+0000 7f9b49842740 10 client.16428 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-16T17:10:15.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.519+0000 7f9b49842740 20 client.16428 populate_metadata read hostname 'smithi045' 2024-04-16T17:10:15.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.519+0000 7f9b49842740 1 --2- 172.21.15.45:0/873353108 >> [v2:172.21.15.117:6838/2378675949,v1:172.21.15.117:6839/2378675949] conn(0x558259239250 0x55825923b670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:15.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.519+0000 7f9b49842740 4 client.16428 mds_command: new command op to 24532 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T17:10:15.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.519+0000 7f9b49842740 1 -- 172.21.15.45:0/873353108 --> [v2:172.21.15.117:6838/2378675949,v1:172.21.15.117:6839/2378675949] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5582591258b0 con 0x558259239250 2024-04-16T17:10:15.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.529+0000 7f9b3fe00640 1 --2- 172.21.15.45:0/873353108 >> [v2:172.21.15.117:6838/2378675949,v1:172.21.15.117:6839/2378675949] conn(0x558259239250 0x55825923b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:15.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.530+0000 7f9b3fe00640 1 --2- 172.21.15.45:0/873353108 >> [v2:172.21.15.117:6838/2378675949,v1:172.21.15.117:6839/2378675949] conn(0x558259239250 0x55825923b670 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4c068f5ed544630b server_cookie=c59bfa3a70259f99 in_seq=0 out_seq=0 2024-04-16T17:10:15.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.530+0000 7f9b29600640 10 client.16428 ms_handle_connect on v2:172.21.15.117:6838/2378675949 2024-04-16T17:10:15.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.530+0000 7f9b29600640 1 -- 172.21.15.45:0/873353108 <== mds.2 v2:172.21.15.117:6838/2378675949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5582591258b0 con 0x558259239250 2024-04-16T17:10:15.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.530+0000 7f9b29600640 10 client.16428 handle_command_reply: tid=0 2024-04-16T17:10:15.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.530+0000 7f9b49842740 1 client.16428 shutdown 2024-04-16T17:10:15.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.530+0000 7f9b49842740 1 -- 172.21.15.45:0/873353108 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5582591b9670 msgr2=0x5582591c6570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:15.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.530+0000 7f9b49842740 1 --2- 172.21.15.45:0/873353108 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5582591b9670 0x5582591c6570 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f9b2c0be2b0 tx=0x7f9b2c009260 comp rx=0 tx=0).stop 2024-04-16T17:10:15.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.530+0000 7f9b49842740 1 -- 172.21.15.45:0/873353108 shutdown_connections 2024-04-16T17:10:15.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.530+0000 7f9b49842740 1 --2- 172.21.15.45:0/873353108 >> [v2:172.21.15.117:6838/2378675949,v1:172.21.15.117:6839/2378675949] conn(0x558259239250 0x55825923b670 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:15.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.530+0000 7f9b49842740 1 --2- 172.21.15.45:0/873353108 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5582591c2250 0x5582591c6ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:15.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.530+0000 7f9b49842740 1 --2- 172.21.15.45:0/873353108 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5582591b9670 0x5582591c6570 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:15.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.530+0000 7f9b49842740 1 --2- 172.21.15.45:0/873353108 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5582591b8d50 0x5582591c6030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:15.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.530+0000 7f9b49842740 1 -- 172.21.15.45:0/873353108 >> 172.21.15.45:0/873353108 conn(0x5582591b3480 msgr2=0x5582591b24e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:10:15.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.531+0000 7f9b29600640 0 client.16428 ms_handle_reset on v2:172.21.15.117:6838/2378675949 2024-04-16T17:10:15.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.531+0000 7f9b49842740 1 -- 172.21.15.45:0/873353108 shutdown_connections 2024-04-16T17:10:15.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.531+0000 7f9b49842740 1 -- 172.21.15.45:0/873353108 wait complete. 2024-04-16T17:10:15.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.531+0000 7f9b49842740 20 client.16428 trim_cache size 0 max 16384 2024-04-16T17:10:15.533 INFO:teuthology.orchestra.run.smithi045.stdout:[] 2024-04-16T17:10:15.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.532+0000 7f9b46c00640 1 -- 172.21.15.45:0/3238423378 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f9b240a3d20 msgr2=0x7f9b240a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:15.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.532+0000 7f9b46c00640 1 --2- 172.21.15.45:0/3238423378 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f9b240a3d20 0x7f9b240a61e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f9b38012640 tx=0x7f9b38009500 comp rx=0 tx=0).stop 2024-04-16T17:10:15.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.532+0000 7f9b46c00640 1 -- 172.21.15.45:0/3238423378 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9b40107360 msgr2=0x7f9b40074840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:15.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.532+0000 7f9b46c00640 1 --2- 172.21.15.45:0/3238423378 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9b40107360 0x7f9b40074840 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f9b2c02c270 tx=0x7f9b2c004360 comp rx=0 tx=0).stop 2024-04-16T17:10:15.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.532+0000 7f9b46c00640 1 -- 172.21.15.45:0/3238423378 shutdown_connections 2024-04-16T17:10:15.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.532+0000 7f9b46c00640 1 --2- 172.21.15.45:0/3238423378 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f9b240a3d20 0x7f9b240a61e0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:15.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.532+0000 7f9b46c00640 1 --2- 172.21.15.45:0/3238423378 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9b40074d80 0x7f9b401a6c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:15.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.532+0000 7f9b46c00640 1 --2- 172.21.15.45:0/3238423378 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9b40107360 0x7f9b40074840 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:15.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.532+0000 7f9b46c00640 1 --2- 172.21.15.45:0/3238423378 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9b40102290 0x7f9b40074300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:15.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.532+0000 7f9b46c00640 1 -- 172.21.15.45:0/3238423378 >> 172.21.15.45:0/3238423378 conn(0x7f9b400fdb10 msgr2=0x7f9b400ffe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:10:15.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.533+0000 7f9b46c00640 1 -- 172.21.15.45:0/3238423378 shutdown_connections 2024-04-16T17:10:15.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.533+0000 7f9b46c00640 1 -- 172.21.15.45:0/3238423378 wait complete. 2024-04-16T17:10:15.556 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-16T17:10:15.628 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:15.628+0000 7fc4dbe00640 1 -- 172.21.15.45:0/2354695483 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc4dc102a40 msgr2=0x7fc4dc106a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:15.628 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:15.628+0000 7fc4dbe00640 1 --2- 172.21.15.45:0/2354695483 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc4dc102a40 0x7fc4dc106a80 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fc4c8009900 tx=0x7fc4c802bc40 comp rx=0 tx=0).stop 2024-04-16T17:10:15.629 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:15.628+0000 7fc4dbe00640 1 -- 172.21.15.45:0/2354695483 shutdown_connections 2024-04-16T17:10:15.629 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:15.628+0000 7fc4dbe00640 1 --2- 172.21.15.45:0/2354695483 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc4dc102a40 0x7fc4dc106a80 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:15.629 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:15.628+0000 7fc4dbe00640 1 --2- 172.21.15.45:0/2354695483 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc4dc107160 0x7fc4dc10ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:15.629 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:15.628+0000 7fc4dbe00640 1 --2- 172.21.15.45:0/2354695483 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc4dc102090 0x7fc4dc102470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:15.629 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:15.628+0000 7fc4dbe00640 1 -- 172.21.15.45:0/2354695483 >> 172.21.15.45:0/2354695483 conn(0x7fc4dc0fd950 msgr2=0x7fc4dc0ffd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:10:15.629 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:15.628+0000 7fc4dbe00640 1 -- 172.21.15.45:0/2354695483 shutdown_connections 2024-04-16T17:10:15.629 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:15.628+0000 7fc4dbe00640 1 -- 172.21.15.45:0/2354695483 wait complete. 2024-04-16T17:10:15.631 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:15.630+0000 7fc4dbe00640 1 Processor -- start 2024-04-16T17:10:15.631 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:15.630+0000 7fc4dbe00640 1 -- start start 2024-04-16T17:10:15.631 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:15.630+0000 7fc4dbe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc4dc102090 0x7fc4dc19e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:15.632 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:15.630+0000 7fc4dbe00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc4dc107160 0x7fc4dc19e950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:15.632 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:15.630+0000 7fc4dbe00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc4dc1985c0 0x7fc4dc198a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:15.632 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:15.630+0000 7fc4dbe00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fc4dc112220 con 0x7fc4dc102090 2024-04-16T17:10:15.632 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:15.630+0000 7fc4dbe00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fc4dc1120a0 con 0x7fc4dc107160 2024-04-16T17:10:15.632 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:15.630+0000 7fc4dbe00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fc4dc1123a0 con 0x7fc4dc1985c0 2024-04-16T17:10:15.632 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:15.631+0000 7fc4daa00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc4dc102090 0x7fc4dc19e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:15.632 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:15.631+0000 7fc4db400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc4dc1985c0 0x7fc4dc198a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:15.632 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:15.631+0000 7fc4db400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc4dc1985c0 0x7fc4dc198a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:33392/0 (socket says 172.21.15.45:33392) 2024-04-16T17:10:15.632 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:15.631+0000 7fc4db400640 1 -- 172.21.15.45:0/1652111908 learned_addr learned my addr 172.21.15.45:0/1652111908 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:10:15.632 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:15.631+0000 7fc4da000640 1 --2- 172.21.15.45:0/1652111908 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc4dc107160 0x7fc4dc19e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:15.633 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:15.631+0000 7fc4db400640 1 -- 172.21.15.45:0/1652111908 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc4dc107160 msgr2=0x7fc4dc19e950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:15.633 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:15.631+0000 7fc4db400640 1 --2- 172.21.15.45:0/1652111908 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc4dc107160 0x7fc4dc19e950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:15.633 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:15.631+0000 7fc4db400640 1 -- 172.21.15.45:0/1652111908 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc4dc102090 msgr2=0x7fc4dc19e410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:15.633 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:15.632+0000 7fc4db400640 1 --2- 172.21.15.45:0/1652111908 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc4dc102090 0x7fc4dc19e410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:15.633 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:15.632+0000 7fc4db400640 1 -- 172.21.15.45:0/1652111908 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4dc1990e0 con 0x7fc4dc1985c0 2024-04-16T17:10:15.633 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:15.632+0000 7fc4daa00640 1 --2- 172.21.15.45:0/1652111908 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc4dc102090 0x7fc4dc19e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:10:15.634 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:15.632+0000 7fc4db400640 1 --2- 172.21.15.45:0/1652111908 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc4dc1985c0 0x7fc4dc198a70 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fc4d400eaa0 tx=0x7fc4d400ef60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:10:15.634 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:15.632+0000 7fc4d3400640 1 -- 172.21.15.45:0/1652111908 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4d400cdb0 con 0x7fc4dc1985c0 2024-04-16T17:10:15.634 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:15.632+0000 7fc4d3400640 1 -- 172.21.15.45:0/1652111908 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc4d4005020 con 0x7fc4dc1985c0 2024-04-16T17:10:15.634 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:15.632+0000 7fc4dbe00640 1 -- 172.21.15.45:0/1652111908 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4dc1993d0 con 0x7fc4dc1985c0 2024-04-16T17:10:15.634 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:15.632+0000 7fc4d3400640 1 -- 172.21.15.45:0/1652111908 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4d40116b0 con 0x7fc4dc1985c0 2024-04-16T17:10:15.635 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:15.632+0000 7fc4dbe00640 1 -- 172.21.15.45:0/1652111908 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4dc074230 con 0x7fc4dc1985c0 2024-04-16T17:10:15.637 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:15.637+0000 7fc4d3400640 1 -- 172.21.15.45:0/1652111908 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fc4d4005560 con 0x7fc4dc1985c0 2024-04-16T17:10:15.637 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:15.637+0000 7fc4e43ac740 1 Processor -- start 2024-04-16T17:10:15.638 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:15.637+0000 7fc4e43ac740 1 -- start start 2024-04-16T17:10:15.639 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:15.638+0000 7fc4e43ac740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5605c285d7f0 0x5605c285c540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:15.639 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:15.638+0000 7fc4e43ac740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5605c285ca80 0x5605c285cf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:15.639 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:15.638+0000 7fc4da000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5605c285d7f0 0x5605c285c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:15.639 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:15.638+0000 7fc4d3400640 1 --2- 172.21.15.45:0/1652111908 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fc4c00a3d20 0x7fc4c00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:15.639 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:15.638+0000 7fc4da000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5605c285d7f0 0x5605c285c540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:33404/0 (socket says 172.21.15.45:33404) 2024-04-16T17:10:15.639 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:15.638+0000 7fc4da000640 1 -- 172.21.15.45:0/2911554694 learned_addr learned my addr 172.21.15.45:0/2911554694 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:10:15.639 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:15.639+0000 7fc4e43ac740 1 --2- 172.21.15.45:0/2911554694 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5605c286e810 0x5605c2870c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:15.639 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:15.639+0000 7fc4e43ac740 1 -- 172.21.15.45:0/2911554694 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5605c27c1780 con 0x5605c286e810 2024-04-16T17:10:15.639 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:15.639+0000 7fc4d3400640 1 -- 172.21.15.45:0/1652111908 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fc4d40d0d90 con 0x7fc4dc1985c0 2024-04-16T17:10:15.640 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:15.639+0000 7fc4daa00640 1 --2- 172.21.15.45:0/2911554694 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5605c285ca80 0x5605c285cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:15.640 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:15.639+0000 7fc4db400640 1 --2- 172.21.15.45:0/2911554694 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5605c286e810 0x5605c2870c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:15.641 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:15.639+0000 7fc4e43ac740 1 -- 172.21.15.45:0/2911554694 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5605c2767f40 con 0x5605c285ca80 2024-04-16T17:10:15.641 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:15.639+0000 7fc4e43ac740 1 -- 172.21.15.45:0/2911554694 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5605c27bd740 con 0x5605c285d7f0 2024-04-16T17:10:15.641 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:15.640+0000 7fc4db400640 1 -- 172.21.15.45:0/2911554694 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5605c285d7f0 msgr2=0x5605c285c540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:10:15.641 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:15.640+0000 7fc4db400640 1 --2- 172.21.15.45:0/2911554694 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5605c285d7f0 0x5605c285c540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:15.641 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:15.640+0000 7fc4db400640 1 -- 172.21.15.45:0/2911554694 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5605c285ca80 msgr2=0x5605c285cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:10:15.641 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:15.640+0000 7fc4db400640 1 --2- 172.21.15.45:0/2911554694 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5605c285ca80 0x5605c285cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:15.642 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:15.640+0000 7fc4db400640 1 -- 172.21.15.45:0/2911554694 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5605c24d4210 con 0x5605c286e810 2024-04-16T17:10:15.642 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:15.640+0000 7fc4daa00640 1 --2- 172.21.15.45:0/2911554694 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5605c285ca80 0x5605c285cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:10:15.642 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:15.640+0000 7fc4da000640 1 --2- 172.21.15.45:0/1652111908 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fc4c00a3d20 0x7fc4c00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:15.642 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:15.640+0000 7fc4db400640 1 --2- 172.21.15.45:0/2911554694 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5605c286e810 0x5605c2870c00 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fc4d40ce130 tx=0x7fc4d4098d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1534c184a0471534 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:10:15.642 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:15.640+0000 7fc4c7400640 1 -- 172.21.15.45:0/2911554694 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4d401f070 con 0x5605c286e810 2024-04-16T17:10:15.642 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:15.640+0000 7fc4c7400640 1 -- 172.21.15.45:0/2911554694 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc4d40a55f0 con 0x5605c286e810 2024-04-16T17:10:15.642 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:15.640+0000 7fc4c7400640 1 -- 172.21.15.45:0/2911554694 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4d4087310 con 0x5605c286e810 2024-04-16T17:10:15.642 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:15.641+0000 7fc4e43ac740 1 -- 172.21.15.45:0/2911554694 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5605c286e810 msgr2=0x5605c2870c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:15.642 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:15.641+0000 7fc4e43ac740 1 --2- 172.21.15.45:0/2911554694 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5605c286e810 0x5605c2870c00 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fc4d40ce130 tx=0x7fc4d4098d50 comp rx=0 tx=0).stop 2024-04-16T17:10:15.643 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:15.641+0000 7fc4e43ac740 1 -- 172.21.15.45:0/2911554694 shutdown_connections 2024-04-16T17:10:15.643 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:15.641+0000 7fc4e43ac740 1 --2- 172.21.15.45:0/2911554694 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5605c286e810 0x5605c2870c00 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:15.643 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:15.641+0000 7fc4e43ac740 1 --2- 172.21.15.45:0/2911554694 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5605c285ca80 0x5605c285cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:15.643 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:15.641+0000 7fc4e43ac740 1 --2- 172.21.15.45:0/2911554694 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5605c285d7f0 0x5605c285c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:15.643 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:15.641+0000 7fc4e43ac740 1 -- 172.21.15.45:0/2911554694 >> 172.21.15.45:0/2911554694 conn(0x5605c27f75d0 msgr2=0x5605c2857770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:10:15.646 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:15.641+0000 7fc4da000640 1 --2- 172.21.15.45:0/1652111908 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fc4c00a3d20 0x7fc4c00a61e0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fc4c8006010 tx=0x7fc4c802bc40 comp rx=0 tx=0).ready entity=mgr.25411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:10:15.646 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:15.641+0000 7fc4e43ac740 1 -- 172.21.15.45:0/2911554694 shutdown_connections 2024-04-16T17:10:15.646 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:15.641+0000 7fc4e43ac740 1 -- 172.21.15.45:0/2911554694 wait complete. 2024-04-16T17:10:15.646 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:15.641+0000 7fc4e43ac740 1 -- start start 2024-04-16T17:10:15.646 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:15.641+0000 7fc4e43ac740 1 Processor -- start 2024-04-16T17:10:15.646 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:15.641+0000 7fc4e43ac740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5605c285ca80 0x5605c286b710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:15.646 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:15.641+0000 7fc4e43ac740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5605c285d7f0 0x5605c286bc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:15.647 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:15.641+0000 7fc4e43ac740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5605c286e810 0x5605c286c190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:15.647 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:15.641+0000 7fc4e43ac740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5605c2871540 con 0x5605c286e810 2024-04-16T17:10:15.647 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:15.641+0000 7fc4e43ac740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5605c277b390 con 0x5605c285ca80 2024-04-16T17:10:15.647 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:15.641+0000 7fc4e43ac740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5605c28716c0 con 0x5605c285d7f0 2024-04-16T17:10:15.647 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:15.641+0000 7fc4e43ac740 10 client.0 _ll_register_callbacks cb 0x5605c2764f60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:10:15.647 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:15.641+0000 7fc4daa00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5605c285ca80 0x5605c286b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:15.647 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:15.641+0000 7fc4daa00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5605c285ca80 0x5605c286b710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:40096/0 (socket says 172.21.15.45:40096) 2024-04-16T17:10:15.647 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:15.641+0000 7fc4daa00640 1 -- 172.21.15.45:0/674850060 learned_addr learned my addr 172.21.15.45:0/674850060 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:10:15.647 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:15.641+0000 7fc4db400640 1 --2- 172.21.15.45:0/674850060 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5605c286e810 0x5605c286c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:15.647 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:15.641+0000 7fc4da000640 1 --2- 172.21.15.45:0/674850060 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5605c285d7f0 0x5605c286bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:15.648 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:15.641+0000 7fc4db400640 1 -- 172.21.15.45:0/674850060 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5605c285d7f0 msgr2=0x5605c286bc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:10:15.648 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:15.641+0000 7fc4db400640 1 --2- 172.21.15.45:0/674850060 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5605c285d7f0 0x5605c286bc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:15.648 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:15.641+0000 7fc4db400640 1 -- 172.21.15.45:0/674850060 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5605c285ca80 msgr2=0x5605c286b710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:10:15.648 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:15.641+0000 7fc4db400640 1 --2- 172.21.15.45:0/674850060 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5605c285ca80 0x5605c286b710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:15.648 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:15.641+0000 7fc4db400640 1 -- 172.21.15.45:0/674850060 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5605c25a0620 con 0x5605c286e810 2024-04-16T17:10:15.648 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:15.641+0000 7fc4db400640 1 --2- 172.21.15.45:0/674850060 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5605c286e810 0x5605c286c190 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x5605c285d520 tx=0x7fc4d4005400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97c51ef03dd26a3e server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:10:15.648 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:15.641+0000 7fc4bbe00640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-16T17:10:15.648 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:15.641+0000 7fc4bbe00640 1 -- 172.21.15.45:0/674850060 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4d401f030 con 0x5605c286e810 2024-04-16T17:10:15.648 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:15.641+0000 7fc4bbe00640 1 -- 172.21.15.45:0/674850060 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fc4d40872d0 con 0x5605c286e810 2024-04-16T17:10:15.648 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:15.641+0000 7fc4e43ac740 1 -- 172.21.15.45:0/674850060 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc4d40a55f0 con 0x5605c286e810 2024-04-16T17:10:15.648 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:15.641+0000 7fc4bbe00640 1 -- 172.21.15.45:0/674850060 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fc4d40bbb40 con 0x5605c286e810 2024-04-16T17:10:15.649 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:15.641+0000 7fc4e43ac740 10 client.16440 fetch_fsmap learned FSMap version 30 2024-04-16T17:10:15.649 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:15.641+0000 7fc4e43ac740 1 -- 172.21.15.45:0/674850060 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5605c287d6f0 con 0x5605c286e810 2024-04-16T17:10:15.649 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:15.641+0000 7fc4bbe00640 1 -- 172.21.15.45:0/674850060 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc4d40a55f0 con 0x5605c286e810 2024-04-16T17:10:15.649 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:15.646+0000 7fc4bbe00640 1 -- 172.21.15.45:0/674850060 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fc4d400f040 con 0x5605c286e810 2024-04-16T17:10:15.649 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:15.646+0000 7fc4e43ac740 10 client.16440 fetch_fsmap finished waiting for FSMap version 30 2024-04-16T17:10:15.649 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:15.646+0000 7fc4e43ac740 10 client.16440 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T17:10:15.649 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:15.646+0000 7fc4e43ac740 20 client.16440 populate_metadata read hostname 'smithi045' 2024-04-16T17:10:15.649 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:15.646+0000 7fc4e43ac740 1 --2- 172.21.15.45:0/674850060 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x5605c287ea40 0x5605c2880e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:15.649 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:15.646+0000 7fc4e43ac740 4 client.16440 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:10:15.649 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:15.646+0000 7fc4e43ac740 1 -- 172.21.15.45:0/674850060 --> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5605c27cf1a0 con 0x5605c287ea40 2024-04-16T17:10:15.651 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:15.646+0000 7fc4daa00640 1 --2- 172.21.15.45:0/674850060 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x5605c287ea40 0x5605c2880e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:15.651 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:15.646+0000 7fc4daa00640 1 --2- 172.21.15.45:0/674850060 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x5605c287ea40 0x5605c2880e30 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8b75321cc8876c54 server_cookie=422a4d6cc5da3154 in_seq=0 out_seq=0 2024-04-16T17:10:15.651 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:15.646+0000 7fc4bbe00640 10 client.16440 ms_handle_connect on v2:172.21.15.104:6838/3660070592 2024-04-16T17:10:15.652 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:15.646+0000 7fc4bbe00640 1 -- 172.21.15.45:0/674850060 <== mds.0 v2:172.21.15.104:6838/3660070592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5605c27cf1a0 con 0x5605c287ea40 2024-04-16T17:10:15.652 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:15.646+0000 7fc4bbe00640 10 client.16440 handle_command_reply: tid=0 2024-04-16T17:10:15.652 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:15.646+0000 7fc4e43ac740 1 client.16440 shutdown 2024-04-16T17:10:15.652 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:15.646+0000 7fc4e43ac740 1 -- 172.21.15.45:0/674850060 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5605c286e810 msgr2=0x5605c286c190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:15.652 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:15.646+0000 7fc4e43ac740 1 --2- 172.21.15.45:0/674850060 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5605c286e810 0x5605c286c190 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x5605c285d520 tx=0x7fc4d4005400 comp rx=0 tx=0).stop 2024-04-16T17:10:15.652 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:15.646+0000 7fc4e43ac740 1 -- 172.21.15.45:0/674850060 shutdown_connections 2024-04-16T17:10:15.652 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:15.646+0000 7fc4e43ac740 1 --2- 172.21.15.45:0/674850060 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x5605c287ea40 0x5605c2880e30 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:15.652 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:15.646+0000 7fc4e43ac740 1 --2- 172.21.15.45:0/674850060 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5605c285d7f0 0x5605c286bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:15.652 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:15.646+0000 7fc4e43ac740 1 --2- 172.21.15.45:0/674850060 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5605c286e810 0x5605c286c190 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:15.652 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:15.646+0000 7fc4bbe00640 0 client.16440 ms_handle_reset on v2:172.21.15.104:6838/3660070592 2024-04-16T17:10:15.652 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:15.646+0000 7fc4e43ac740 1 --2- 172.21.15.45:0/674850060 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5605c285ca80 0x5605c286b710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:15.653 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:15.646+0000 7fc4e43ac740 1 -- 172.21.15.45:0/674850060 >> 172.21.15.45:0/674850060 conn(0x5605c27f75d0 msgr2=0x5605c28594c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:10:15.653 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:15.646+0000 7fc4e43ac740 1 -- 172.21.15.45:0/674850060 shutdown_connections 2024-04-16T17:10:15.653 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:15.646+0000 7fc4e43ac740 1 -- 172.21.15.45:0/674850060 wait complete. 2024-04-16T17:10:15.653 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:15.649+0000 7fc4e43ac740 20 client.16440 trim_cache size 0 max 16384 2024-04-16T17:10:15.676 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:15.672+0000 7fc4e43ac740 1 Processor -- start 2024-04-16T17:10:15.676 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:15.672+0000 7fc4e43ac740 1 -- start start 2024-04-16T17:10:15.676 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:15.672+0000 7fc4e43ac740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5605c285ca80 0x5605c286de40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:15.676 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:15.672+0000 7fc4e43ac740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5605c285d7f0 0x5605c2859690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:15.676 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:15.672+0000 7fc4e43ac740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5605c286e810 0x5605c286b030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:15.676 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:15.672+0000 7fc4e43ac740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fc4dc1afb20 con 0x5605c285d7f0 2024-04-16T17:10:15.676 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:15.672+0000 7fc4e43ac740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fc4dc1ad330 con 0x5605c285ca80 2024-04-16T17:10:15.676 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:15.672+0000 7fc4e43ac740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fc4dc1ab160 con 0x5605c286e810 2024-04-16T17:10:15.676 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:15.672+0000 7fc4daa00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5605c285ca80 0x5605c286de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:15.676 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:15.672+0000 7fc4daa00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5605c285ca80 0x5605c286de40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:40104/0 (socket says 172.21.15.45:40104) 2024-04-16T17:10:15.677 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:15.672+0000 7fc4daa00640 1 -- 172.21.15.45:0/1730795103 learned_addr learned my addr 172.21.15.45:0/1730795103 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:10:15.677 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:15.672+0000 7fc4da000640 1 --2- 172.21.15.45:0/1730795103 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5605c285d7f0 0x5605c2859690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:15.677 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:15.672+0000 7fc4daa00640 1 -- 172.21.15.45:0/1730795103 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5605c286e810 msgr2=0x5605c286b030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:10:15.677 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:15.672+0000 7fc4daa00640 1 --2- 172.21.15.45:0/1730795103 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5605c286e810 0x5605c286b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:15.677 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:15.672+0000 7fc4daa00640 1 -- 172.21.15.45:0/1730795103 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5605c285d7f0 msgr2=0x5605c2859690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:10:15.677 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:15.672+0000 7fc4daa00640 1 --2- 172.21.15.45:0/1730795103 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5605c285d7f0 0x5605c2859690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:15.677 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:15.672+0000 7fc4daa00640 1 -- 172.21.15.45:0/1730795103 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5605c285bd50 con 0x5605c285ca80 2024-04-16T17:10:15.677 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:15.672+0000 7fc4da000640 1 --2- 172.21.15.45:0/1730795103 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5605c285d7f0 0x5605c2859690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:10:15.677 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:15.672+0000 7fc4daa00640 1 --2- 172.21.15.45:0/1730795103 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5605c285ca80 0x5605c286de40 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fc4cc011ae0 tx=0x7fc4cc00b410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f25723f5a49ad02e server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:10:15.679 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:15.677+0000 7fc4c6a00640 1 -- 172.21.15.45:0/1730795103 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4cc004220 con 0x5605c285ca80 2024-04-16T17:10:15.679 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:15.677+0000 7fc4c6a00640 1 -- 172.21.15.45:0/1730795103 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc4cc023030 con 0x5605c285ca80 2024-04-16T17:10:15.679 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:15.677+0000 7fc4c6a00640 1 -- 172.21.15.45:0/1730795103 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4cc0047a0 con 0x5605c285ca80 2024-04-16T17:10:15.679 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:15.677+0000 7fc4e43ac740 1 -- 172.21.15.45:0/1730795103 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5605c285ca80 msgr2=0x5605c286de40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:15.679 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:15.677+0000 7fc4e43ac740 1 --2- 172.21.15.45:0/1730795103 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5605c285ca80 0x5605c286de40 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fc4cc011ae0 tx=0x7fc4cc00b410 comp rx=0 tx=0).stop 2024-04-16T17:10:15.679 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:15.677+0000 7fc4e43ac740 1 -- 172.21.15.45:0/1730795103 shutdown_connections 2024-04-16T17:10:15.679 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:15.677+0000 7fc4e43ac740 1 --2- 172.21.15.45:0/1730795103 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5605c286e810 0x5605c286b030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:15.680 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:15.677+0000 7fc4e43ac740 1 --2- 172.21.15.45:0/1730795103 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5605c285d7f0 0x5605c2859690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:15.680 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:15.677+0000 7fc4e43ac740 1 --2- 172.21.15.45:0/1730795103 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5605c285ca80 0x5605c286de40 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:15.680 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:15.677+0000 7fc4e43ac740 1 -- 172.21.15.45:0/1730795103 >> 172.21.15.45:0/1730795103 conn(0x5605c27f75d0 msgr2=0x5605c287a1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:10:15.680 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:15.677+0000 7fc4e43ac740 1 -- 172.21.15.45:0/1730795103 shutdown_connections 2024-04-16T17:10:15.680 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:15.677+0000 7fc4e43ac740 1 -- 172.21.15.45:0/1730795103 wait complete. 2024-04-16T17:10:15.680 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:15.677+0000 7fc4e43ac740 1 -- start start 2024-04-16T17:10:15.680 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:15.677+0000 7fc4e43ac740 1 Processor -- start 2024-04-16T17:10:15.682 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:15.679+0000 7fc4e43ac740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5605c285ca80 0x5605c2868790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:15.682 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:15.679+0000 7fc4e43ac740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5605c285d7f0 0x5605c2868cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:15.682 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:15.679+0000 7fc4e43ac740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5605c286e810 0x5605c28e3c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:15.683 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:15.679+0000 7fc4e43ac740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fc4dc1ab7f0 con 0x5605c285d7f0 2024-04-16T17:10:15.683 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:15.679+0000 7fc4e43ac740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fc4dc1acc60 con 0x5605c286e810 2024-04-16T17:10:15.683 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:15.679+0000 7fc4e43ac740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fc4dc1abec0 con 0x5605c285ca80 2024-04-16T17:10:15.683 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:15.679+0000 7fc4e43ac740 10 client.0 _ll_register_callbacks cb 0x7fc4cc00bae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:10:15.683 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:15.679+0000 7fc4db400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5605c286e810 0x5605c28e3c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:15.683 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:15.680+0000 7fc4daa00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5605c285ca80 0x5605c2868790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:15.683 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:15.680+0000 7fc4db400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5605c286e810 0x5605c28e3c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:40120/0 (socket says 172.21.15.45:40120) 2024-04-16T17:10:15.683 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:15.680+0000 7fc4db400640 1 -- 172.21.15.45:0/261608367 learned_addr learned my addr 172.21.15.45:0/261608367 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:10:15.683 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:15.680+0000 7fc4da000640 1 --2- 172.21.15.45:0/261608367 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5605c285d7f0 0x5605c2868cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:15.683 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:15.680+0000 7fc4db400640 1 -- 172.21.15.45:0/261608367 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5605c285ca80 msgr2=0x5605c2868790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:10:15.683 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:15.680+0000 7fc4db400640 1 --2- 172.21.15.45:0/261608367 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5605c285ca80 0x5605c2868790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:15.684 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:15.680+0000 7fc4db400640 1 -- 172.21.15.45:0/261608367 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5605c285d7f0 msgr2=0x5605c2868cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:10:15.684 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:15.680+0000 7fc4db400640 1 --2- 172.21.15.45:0/261608367 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5605c285d7f0 0x5605c2868cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:15.684 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:15.680+0000 7fc4db400640 1 -- 172.21.15.45:0/261608367 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5605c28695a0 con 0x5605c286e810 2024-04-16T17:10:15.684 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:15.680+0000 7fc4da000640 1 --2- 172.21.15.45:0/261608367 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5605c285d7f0 0x5605c2868cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:10:15.684 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:15.680+0000 7fc4daa00640 1 --2- 172.21.15.45:0/261608367 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5605c285ca80 0x5605c2868790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:10:15.684 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:15.680+0000 7fc4db400640 1 --2- 172.21.15.45:0/261608367 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5605c286e810 0x5605c28e3c10 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fc4d409c680 tx=0x7fc4d409bb50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c8f1dbced96793cc server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:10:15.684 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:15.680+0000 7fc4bbe00640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-04-16T17:10:15.684 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:15.681+0000 7fc4bbe00640 1 -- 172.21.15.45:0/261608367 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4d401f070 con 0x5605c286e810 2024-04-16T17:10:15.684 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:15.681+0000 7fc4bbe00640 1 -- 172.21.15.45:0/261608367 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fc4d409bdd0 con 0x5605c286e810 2024-04-16T17:10:15.684 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:15.681+0000 7fc4e43ac740 1 -- 172.21.15.45:0/261608367 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc4cc023030 con 0x5605c286e810 2024-04-16T17:10:15.684 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:15.681+0000 7fc4bbe00640 1 -- 172.21.15.45:0/261608367 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fc4d40a55f0 con 0x5605c286e810 2024-04-16T17:10:15.685 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:15.681+0000 7fc4e43ac740 10 client.26597 fetch_fsmap learned FSMap version 30 2024-04-16T17:10:15.685 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:15.681+0000 7fc4e43ac740 1 -- 172.21.15.45:0/261608367 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5605c28e5260 con 0x5605c286e810 2024-04-16T17:10:15.685 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:15.681+0000 7fc4bbe00640 1 -- 172.21.15.45:0/261608367 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc4cc023030 con 0x5605c286e810 2024-04-16T17:10:15.685 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:15.681+0000 7fc4bbe00640 1 -- 172.21.15.45:0/261608367 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fc4d4004a40 con 0x5605c286e810 2024-04-16T17:10:15.685 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:15.682+0000 7fc4e43ac740 10 client.26597 fetch_fsmap finished waiting for FSMap version 30 2024-04-16T17:10:15.685 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:15.682+0000 7fc4e43ac740 10 client.26597 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T17:10:15.685 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:15.682+0000 7fc4e43ac740 20 client.26597 populate_metadata read hostname 'smithi045' 2024-04-16T17:10:15.685 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:15.682+0000 7fc4e43ac740 1 --2- 172.21.15.45:0/261608367 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x5605c287ea40 0x5605c28e7860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:15.685 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:15.682+0000 7fc4e43ac740 4 client.26597 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-16T17:10:15.685 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:15.682+0000 7fc4e43ac740 1 -- 172.21.15.45:0/261608367 --> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5605c27c9010 con 0x5605c287ea40 2024-04-16T17:10:15.685 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:15.682+0000 7fc4daa00640 1 --2- 172.21.15.45:0/261608367 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x5605c287ea40 0x5605c28e7860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:15.686 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:15.687+0000 7fc4daa00640 1 --2- 172.21.15.45:0/261608367 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x5605c287ea40 0x5605c28e7860 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c9a9e43dbaead19f server_cookie=3dde0c90029e1462 in_seq=0 out_seq=0 2024-04-16T17:10:15.687 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:15.687+0000 7fc4bbe00640 10 client.26597 ms_handle_connect on v2:172.21.15.104:6838/3660070592 2024-04-16T17:10:15.708 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:15.707+0000 7fc4bbe00640 1 -- 172.21.15.45:0/261608367 <== mds.0 v2:172.21.15.104:6838/3660070592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5605c27c9010 con 0x5605c287ea40 2024-04-16T17:10:15.708 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:15.707+0000 7fc4bbe00640 10 client.26597 handle_command_reply: tid=0 2024-04-16T17:10:15.708 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:15.708+0000 7fc4e43ac740 1 client.26597 shutdown 2024-04-16T17:10:15.708 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:15.708+0000 7fc4e43ac740 1 -- 172.21.15.45:0/261608367 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5605c286e810 msgr2=0x5605c28e3c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:15.709 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:15.708+0000 7fc4e43ac740 1 --2- 172.21.15.45:0/261608367 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5605c286e810 0x5605c28e3c10 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fc4d409c680 tx=0x7fc4d409bb50 comp rx=0 tx=0).stop 2024-04-16T17:10:15.709 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:15.708+0000 7fc4e43ac740 1 -- 172.21.15.45:0/261608367 shutdown_connections 2024-04-16T17:10:15.709 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:15.708+0000 7fc4e43ac740 1 --2- 172.21.15.45:0/261608367 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5605c286e810 0x5605c28e3c10 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:15.709 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:15.708+0000 7fc4e43ac740 1 --2- 172.21.15.45:0/261608367 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5605c285d7f0 0x5605c2868cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:15.709 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:15.708+0000 7fc4e43ac740 1 --2- 172.21.15.45:0/261608367 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x5605c287ea40 0x5605c28e7860 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:15.709 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:15.708+0000 7fc4e43ac740 1 --2- 172.21.15.45:0/261608367 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5605c285ca80 0x5605c2868790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:15.709 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:15.708+0000 7fc4bbe00640 0 client.26597 ms_handle_reset on v2:172.21.15.104:6838/3660070592 2024-04-16T17:10:15.709 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:15.708+0000 7fc4e43ac740 1 -- 172.21.15.45:0/261608367 >> 172.21.15.45:0/261608367 conn(0x5605c27f75d0 msgr2=0x5605c286b6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:10:15.709 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:15.708+0000 7fc4e43ac740 1 -- 172.21.15.45:0/261608367 shutdown_connections 2024-04-16T17:10:15.709 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:15.708+0000 7fc4e43ac740 1 -- 172.21.15.45:0/261608367 wait complete. 2024-04-16T17:10:15.709 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:15.709+0000 7fc4e43ac740 20 client.26597 trim_cache size 0 max 16384 2024-04-16T17:10:15.710 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-04-16T17:10:15.710 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-04-16T17:10:15.710 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-04-16T17:10:15.711 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-04-16T17:10:15.711 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:15.709+0000 7fc4dbe00640 1 -- 172.21.15.45:0/1652111908 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fc4c00a3d20 msgr2=0x7fc4c00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:15.711 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:15.709+0000 7fc4dbe00640 1 --2- 172.21.15.45:0/1652111908 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fc4c00a3d20 0x7fc4c00a61e0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fc4c8006010 tx=0x7fc4c802bc40 comp rx=0 tx=0).stop 2024-04-16T17:10:15.712 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:15.709+0000 7fc4dbe00640 1 -- 172.21.15.45:0/1652111908 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc4dc1985c0 msgr2=0x7fc4dc198a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:15.712 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:15.709+0000 7fc4dbe00640 1 --2- 172.21.15.45:0/1652111908 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc4dc1985c0 0x7fc4dc198a70 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fc4d400eaa0 tx=0x7fc4d400ef60 comp rx=0 tx=0).stop 2024-04-16T17:10:15.712 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:15.710+0000 7fc4dbe00640 1 -- 172.21.15.45:0/1652111908 shutdown_connections 2024-04-16T17:10:15.712 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:15.710+0000 7fc4dbe00640 1 --2- 172.21.15.45:0/1652111908 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fc4c00a3d20 0x7fc4c00a61e0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:15.712 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:15.710+0000 7fc4dbe00640 1 --2- 172.21.15.45:0/1652111908 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc4dc1985c0 0x7fc4dc198a70 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:15.712 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:15.710+0000 7fc4dbe00640 1 --2- 172.21.15.45:0/1652111908 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc4dc107160 0x7fc4dc19e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:15.712 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:15.710+0000 7fc4dbe00640 1 --2- 172.21.15.45:0/1652111908 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc4dc102090 0x7fc4dc19e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:15.712 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:15.710+0000 7fc4dbe00640 1 -- 172.21.15.45:0/1652111908 >> 172.21.15.45:0/1652111908 conn(0x7fc4dc0fd950 msgr2=0x7fc4dc0fecd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:10:15.712 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:15.710+0000 7fc4dbe00640 1 -- 172.21.15.45:0/1652111908 shutdown_connections 2024-04-16T17:10:15.712 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:15.711+0000 7fc4dbe00640 1 -- 172.21.15.45:0/1652111908 wait complete. 2024-04-16T17:10:15.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.780+0000 7fc5a9a00640 1 -- 172.21.15.45:0/4106050959 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc5a410d500 msgr2=0x7fc5a410d8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:15.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.780+0000 7fc5a9a00640 1 --2- 172.21.15.45:0/4106050959 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc5a410d500 0x7fc5a410d8e0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fc594009930 tx=0x7fc59402bc30 comp rx=0 tx=0).stop 2024-04-16T17:10:15.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.780+0000 7fc5a9a00640 1 -- 172.21.15.45:0/4106050959 shutdown_connections 2024-04-16T17:10:15.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.780+0000 7fc5a9a00640 1 --2- 172.21.15.45:0/4106050959 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc5a4108f40 0x7fc5a41153b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:15.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.780+0000 7fc5a9a00640 1 --2- 172.21.15.45:0/4106050959 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc5a4108580 0x7fc5a4108a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:15.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.780+0000 7fc5a9a00640 1 --2- 172.21.15.45:0/4106050959 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc5a410d500 0x7fc5a410d8e0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:15.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.780+0000 7fc5a9a00640 1 -- 172.21.15.45:0/4106050959 >> 172.21.15.45:0/4106050959 conn(0x7fc5a4077e20 msgr2=0x7fc5a4106e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:10:15.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.780+0000 7fc5a9a00640 1 -- 172.21.15.45:0/4106050959 shutdown_connections 2024-04-16T17:10:15.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.780+0000 7fc5a9a00640 1 -- 172.21.15.45:0/4106050959 wait complete. 2024-04-16T17:10:15.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.781+0000 7fc5a9a00640 1 Processor -- start 2024-04-16T17:10:15.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.781+0000 7fc5a9a00640 1 -- start start 2024-04-16T17:10:15.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.782+0000 7fc5a9a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc5a4108580 0x7fc5a419ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:15.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.782+0000 7fc5a9a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc5a4108f40 0x7fc5a419f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:15.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.782+0000 7fc5a9a00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc5a410d500 0x7fc5a41a3620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:15.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.782+0000 7fc5a9a00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fc5a41188c0 con 0x7fc5a4108f40 2024-04-16T17:10:15.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.782+0000 7fc5a9a00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fc5a4118740 con 0x7fc5a410d500 2024-04-16T17:10:15.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.782+0000 7fc5a9a00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fc5a4118a40 con 0x7fc5a4108580 2024-04-16T17:10:15.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.782+0000 7fc5a3400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc5a4108f40 0x7fc5a419f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:15.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.782+0000 7fc5a3e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc5a4108580 0x7fc5a419ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:15.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.782+0000 7fc5a3e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc5a4108580 0x7fc5a419ed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:33452/0 (socket says 172.21.15.45:33452) 2024-04-16T17:10:15.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.782+0000 7fc5a3e00640 1 -- 172.21.15.45:0/1101147214 learned_addr learned my addr 172.21.15.45:0/1101147214 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:10:15.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.782+0000 7fc5a9000640 1 --2- 172.21.15.45:0/1101147214 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc5a410d500 0x7fc5a41a3620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:15.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.783+0000 7fc5a3e00640 1 -- 172.21.15.45:0/1101147214 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc5a410d500 msgr2=0x7fc5a41a3620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:15.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.783+0000 7fc5a3e00640 1 --2- 172.21.15.45:0/1101147214 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc5a410d500 0x7fc5a41a3620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:15.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.783+0000 7fc5a3e00640 1 -- 172.21.15.45:0/1101147214 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc5a4108f40 msgr2=0x7fc5a419f2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:15.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.783+0000 7fc5a3e00640 1 --2- 172.21.15.45:0/1101147214 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc5a4108f40 0x7fc5a419f2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:15.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.783+0000 7fc5a3e00640 1 -- 172.21.15.45:0/1101147214 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5a41a3d50 con 0x7fc5a4108580 2024-04-16T17:10:15.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.783+0000 7fc5a3e00640 1 --2- 172.21.15.45:0/1101147214 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc5a4108580 0x7fc5a419ed60 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fc594009230 tx=0x7fc59402cb20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:10:15.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.783+0000 7fc5a0c00640 1 -- 172.21.15.45:0/1101147214 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc594041070 con 0x7fc5a4108580 2024-04-16T17:10:15.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.783+0000 7fc5a9a00640 1 -- 172.21.15.45:0/1101147214 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5a41a3fe0 con 0x7fc5a4108580 2024-04-16T17:10:15.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.783+0000 7fc5a0c00640 1 -- 172.21.15.45:0/1101147214 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc594004450 con 0x7fc5a4108580 2024-04-16T17:10:15.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.783+0000 7fc5a0c00640 1 -- 172.21.15.45:0/1101147214 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc594003d90 con 0x7fc5a4108580 2024-04-16T17:10:15.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.783+0000 7fc5a9a00640 1 -- 172.21.15.45:0/1101147214 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5a41ab8d0 con 0x7fc5a4108580 2024-04-16T17:10:15.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.786+0000 7fc5adc39740 1 Processor -- start 2024-04-16T17:10:15.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.787+0000 7fc5adc39740 1 -- start start 2024-04-16T17:10:15.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.787+0000 7fc5adc39740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5566c55ecf10 0x5566c55ed2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:15.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.787+0000 7fc5adc39740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5566c55ed830 0x5566c55f5ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:15.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.787+0000 7fc5adc39740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5566c55f6410 0x5566c55f8800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:15.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.787+0000 7fc5adc39740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5566c5552020 con 0x5566c55ecf10 2024-04-16T17:10:15.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.787+0000 7fc5adc39740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5566c54f8460 con 0x5566c55ed830 2024-04-16T17:10:15.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.787+0000 7fc5adc39740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5566c554dfe0 con 0x5566c55f6410 2024-04-16T17:10:15.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.787+0000 7fc5a9000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5566c55ecf10 0x5566c55ed2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:15.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.787+0000 7fc5a9000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5566c55ecf10 0x5566c55ed2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39818/0 (socket says 172.21.15.45:39818) 2024-04-16T17:10:15.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.787+0000 7fc5a9000640 1 -- 172.21.15.45:0/2851244242 learned_addr learned my addr 172.21.15.45:0/2851244242 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:10:15.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.788+0000 7fc5a3400640 1 --2- 172.21.15.45:0/2851244242 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5566c55f6410 0x5566c55f8800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:15.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.788+0000 7fc5a9000640 1 -- 172.21.15.45:0/2851244242 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5566c55f6410 msgr2=0x5566c55f8800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:10:15.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.788+0000 7fc5a9000640 1 --2- 172.21.15.45:0/2851244242 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5566c55f6410 0x5566c55f8800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:15.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.788+0000 7fc5a9000640 1 -- 172.21.15.45:0/2851244242 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5566c55ed830 msgr2=0x5566c55f5ed0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-16T17:10:15.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.788+0000 7fc5a9000640 1 --2- 172.21.15.45:0/2851244242 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5566c55ed830 0x5566c55f5ed0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:15.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.788+0000 7fc5a9000640 1 -- 172.21.15.45:0/2851244242 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5566c5500040 con 0x5566c55ecf10 2024-04-16T17:10:15.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.788+0000 7fc5a9000640 1 --2- 172.21.15.45:0/2851244242 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5566c55ecf10 0x5566c55ed2f0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fc59c009fc0 tx=0x7fc59c0118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13eee37d01770f3b server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:10:15.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.789+0000 7fc599600640 1 -- 172.21.15.45:0/2851244242 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc59c004d20 con 0x5566c55ecf10 2024-04-16T17:10:15.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.789+0000 7fc599600640 1 -- 172.21.15.45:0/2851244242 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc59c004050 con 0x5566c55ecf10 2024-04-16T17:10:15.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.789+0000 7fc599600640 1 -- 172.21.15.45:0/2851244242 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc59c005280 con 0x5566c55ecf10 2024-04-16T17:10:15.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.789+0000 7fc5a0c00640 1 -- 172.21.15.45:0/1101147214 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fc594044070 con 0x7fc5a4108580 2024-04-16T17:10:15.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.789+0000 7fc5adc39740 1 -- 172.21.15.45:0/2851244242 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5566c55ecf10 msgr2=0x5566c55ed2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:15.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.789+0000 7fc5adc39740 1 --2- 172.21.15.45:0/2851244242 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5566c55ecf10 0x5566c55ed2f0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fc59c009fc0 tx=0x7fc59c0118d0 comp rx=0 tx=0).stop 2024-04-16T17:10:15.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.789+0000 7fc5adc39740 1 -- 172.21.15.45:0/2851244242 shutdown_connections 2024-04-16T17:10:15.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.789+0000 7fc5adc39740 1 --2- 172.21.15.45:0/2851244242 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5566c55f6410 0x5566c55f8800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:15.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.789+0000 7fc5adc39740 1 --2- 172.21.15.45:0/2851244242 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5566c55ed830 0x5566c55f5ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:15.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.789+0000 7fc5adc39740 1 --2- 172.21.15.45:0/2851244242 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5566c55ecf10 0x5566c55ed2f0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:15.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.789+0000 7fc5adc39740 1 -- 172.21.15.45:0/2851244242 >> 172.21.15.45:0/2851244242 conn(0x5566c55e7640 msgr2=0x5566c55e7a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:10:15.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.790+0000 7fc5adc39740 1 -- 172.21.15.45:0/2851244242 shutdown_connections 2024-04-16T17:10:15.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.790+0000 7fc5adc39740 1 -- 172.21.15.45:0/2851244242 wait complete. 2024-04-16T17:10:15.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.790+0000 7fc5adc39740 1 -- start start 2024-04-16T17:10:15.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.790+0000 7fc5adc39740 1 Processor -- start 2024-04-16T17:10:15.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.791+0000 7fc5adc39740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5566c55ecf10 0x5566c55f3520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:15.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.791+0000 7fc5adc39740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5566c55ed830 0x5566c55f3a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:15.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.791+0000 7fc5a0c00640 1 --2- 172.21.15.45:0/1101147214 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fc5880a3c30 0x7fc5880a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:15.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.791+0000 7fc5adc39740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5566c55f6410 0x5566c55f3fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:15.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.791+0000 7fc5adc39740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5566c55f9200 con 0x5566c55f6410 2024-04-16T17:10:15.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.791+0000 7fc5adc39740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5566c550b830 con 0x5566c55ecf10 2024-04-16T17:10:15.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.791+0000 7fc5adc39740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5566c55f9380 con 0x5566c55ed830 2024-04-16T17:10:15.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.791+0000 7fc5adc39740 10 client.0 _ll_register_callbacks cb 0x5566c5513150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:10:15.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.791+0000 7fc5a0c00640 1 -- 172.21.15.45:0/1101147214 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fc5940f24d0 con 0x7fc5a4108580 2024-04-16T17:10:15.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.791+0000 7fc5a9000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5566c55f6410 0x5566c55f3fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:15.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.791+0000 7fc5a9000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5566c55f6410 0x5566c55f3fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39828/0 (socket says 172.21.15.45:39828) 2024-04-16T17:10:15.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.791+0000 7fc5a9000640 1 -- 172.21.15.45:0/1287440840 learned_addr learned my addr 172.21.15.45:0/1287440840 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:10:15.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.791+0000 7fc5a9000640 1 --2- 172.21.15.45:0/1287440840 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5566c55ecf10 0x5566c55f3520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:15.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.791+0000 7fc5a3400640 1 --2- 172.21.15.45:0/1287440840 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5566c55ed830 0x5566c55f3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:15.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.792+0000 7fc5a9000640 1 -- 172.21.15.45:0/1287440840 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5566c55ed830 msgr2=0x5566c55f3a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:10:15.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.792+0000 7fc5a9000640 1 --2- 172.21.15.45:0/1287440840 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5566c55ed830 0x5566c55f3a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:15.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.792+0000 7fc5a9000640 1 -- 172.21.15.45:0/1287440840 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5566c55ecf10 msgr2=0x5566c55f3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:10:15.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.792+0000 7fc5a9000640 1 --2- 172.21.15.45:0/1287440840 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5566c55ecf10 0x5566c55f3520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:15.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.792+0000 7fc5a9000640 1 -- 172.21.15.45:0/1287440840 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5566c550f920 con 0x5566c55f6410 2024-04-16T17:10:15.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.792+0000 7fc5a9000640 1 --2- 172.21.15.45:0/1287440840 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5566c55f6410 0x5566c55f3fa0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fc59c009dd0 tx=0x7fc59c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9f4740b580e39c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:10:15.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.792+0000 7fc58e000640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-16T17:10:15.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.792+0000 7fc58e000640 1 -- 172.21.15.45:0/1287440840 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc59c02d060 con 0x5566c55f6410 2024-04-16T17:10:15.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.792+0000 7fc5adc39740 1 -- 172.21.15.45:0/1287440840 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc59c004050 con 0x5566c55f6410 2024-04-16T17:10:15.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.793+0000 7fc5a3e00640 1 --2- 172.21.15.45:0/1101147214 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fc5880a3c30 0x7fc5880a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:15.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.793+0000 7fc58e000640 1 -- 172.21.15.45:0/1287440840 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fc59c00f6e0 con 0x5566c55f6410 2024-04-16T17:10:15.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.793+0000 7fc58e000640 1 -- 172.21.15.45:0/1287440840 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fc59c00f8e0 con 0x5566c55f6410 2024-04-16T17:10:15.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.793+0000 7fc5adc39740 10 client.16455 fetch_fsmap learned FSMap version 30 2024-04-16T17:10:15.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.793+0000 7fc5adc39740 1 -- 172.21.15.45:0/1287440840 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5566c54dd7f0 con 0x5566c55f6410 2024-04-16T17:10:15.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.793+0000 7fc5a3e00640 1 --2- 172.21.15.45:0/1101147214 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fc5880a3c30 0x7fc5880a60f0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x5566c55ec9e0 tx=0x7fc59402cdd0 comp rx=0 tx=0).ready entity=mgr.25411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:10:15.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.794+0000 7fc58e000640 1 -- 172.21.15.45:0/1287440840 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc59c004050 con 0x5566c55f6410 2024-04-16T17:10:15.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.794+0000 7fc58e000640 1 -- 172.21.15.45:0/1287440840 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fc59c0279e0 con 0x5566c55f6410 2024-04-16T17:10:15.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.794+0000 7fc5adc39740 10 client.16455 fetch_fsmap finished waiting for FSMap version 30 2024-04-16T17:10:15.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.794+0000 7fc5adc39740 10 client.16455 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-04-16T17:10:15.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.794+0000 7fc5adc39740 20 client.16455 populate_metadata read hostname 'smithi045' 2024-04-16T17:10:15.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.794+0000 7fc5adc39740 1 --2- 172.21.15.45:0/1287440840 >> [v2:172.21.15.45:6840/3755310822,v1:172.21.15.45:6841/3755310822] conn(0x5566c5606750 0x5566c5608b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:15.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.794+0000 7fc5adc39740 4 client.16455 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:10:15.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.794+0000 7fc5adc39740 1 -- 172.21.15.45:0/1287440840 --> [v2:172.21.15.45:6840/3755310822,v1:172.21.15.45:6841/3755310822] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5566c555fa40 con 0x5566c5606750 2024-04-16T17:10:15.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.794+0000 7fc5a3400640 1 --2- 172.21.15.45:0/1287440840 >> [v2:172.21.15.45:6840/3755310822,v1:172.21.15.45:6841/3755310822] conn(0x5566c5606750 0x5566c5608b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:15.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.795+0000 7fc5a3400640 1 --2- 172.21.15.45:0/1287440840 >> [v2:172.21.15.45:6840/3755310822,v1:172.21.15.45:6841/3755310822] conn(0x5566c5606750 0x5566c5608b40 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=46bb05a6a5f524c7 server_cookie=7847f17f9564e4d in_seq=0 out_seq=0 2024-04-16T17:10:15.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.795+0000 7fc58e000640 10 client.16455 ms_handle_connect on v2:172.21.15.45:6840/3755310822 2024-04-16T17:10:15.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.795+0000 7fc58e000640 1 -- 172.21.15.45:0/1287440840 <== mds.1 v2:172.21.15.45:6840/3755310822 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5566c555fa40 con 0x5566c5606750 2024-04-16T17:10:15.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.795+0000 7fc58e000640 10 client.16455 handle_command_reply: tid=0 2024-04-16T17:10:15.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.796+0000 7fc5adc39740 1 client.16455 shutdown 2024-04-16T17:10:15.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.796+0000 7fc5adc39740 1 -- 172.21.15.45:0/1287440840 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5566c55f6410 msgr2=0x5566c55f3fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:15.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.796+0000 7fc5adc39740 1 --2- 172.21.15.45:0/1287440840 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5566c55f6410 0x5566c55f3fa0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fc59c009dd0 tx=0x7fc59c009bd0 comp rx=0 tx=0).stop 2024-04-16T17:10:15.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.796+0000 7fc5adc39740 1 -- 172.21.15.45:0/1287440840 shutdown_connections 2024-04-16T17:10:15.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.796+0000 7fc5adc39740 1 --2- 172.21.15.45:0/1287440840 >> [v2:172.21.15.45:6840/3755310822,v1:172.21.15.45:6841/3755310822] conn(0x5566c5606750 0x5566c5608b40 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:15.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.796+0000 7fc5adc39740 1 --2- 172.21.15.45:0/1287440840 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5566c55ed830 0x5566c55f3a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:15.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.796+0000 7fc5adc39740 1 --2- 172.21.15.45:0/1287440840 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5566c55f6410 0x5566c55f3fa0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:15.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.796+0000 7fc5adc39740 1 --2- 172.21.15.45:0/1287440840 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5566c55ecf10 0x5566c55f3520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:15.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.796+0000 7fc58e000640 0 client.16455 ms_handle_reset on v2:172.21.15.45:6840/3755310822 2024-04-16T17:10:15.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.796+0000 7fc5adc39740 1 -- 172.21.15.45:0/1287440840 >> 172.21.15.45:0/1287440840 conn(0x5566c55e7640 msgr2=0x5566c55e9390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:10:15.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.796+0000 7fc5adc39740 1 -- 172.21.15.45:0/1287440840 shutdown_connections 2024-04-16T17:10:15.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.796+0000 7fc5adc39740 1 -- 172.21.15.45:0/1287440840 wait complete. 2024-04-16T17:10:15.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.796+0000 7fc5adc39740 20 client.16455 trim_cache size 0 max 16384 2024-04-16T17:10:15.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:10:15 smithi117 ceph-mon[36730]: from='client.? 172.21.15.45:0/711476606' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:10:15.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.811+0000 7fc5adc39740 1 Processor -- start 2024-04-16T17:10:15.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.811+0000 7fc5adc39740 1 -- start start 2024-04-16T17:10:15.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.811+0000 7fc5adc39740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5566c55ecf10 0x5566c56020c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:15.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.811+0000 7fc5adc39740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5566c55ed830 0x5566c55e6580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:15.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.811+0000 7fc5adc39740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5566c55f6410 0x5566c560e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:15.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.811+0000 7fc5adc39740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fc5a41b8240 con 0x5566c55ecf10 2024-04-16T17:10:15.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.811+0000 7fc5adc39740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fc5a41b5a50 con 0x5566c55f6410 2024-04-16T17:10:15.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.811+0000 7fc5adc39740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fc5a41b3880 con 0x5566c55ed830 2024-04-16T17:10:15.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.811+0000 7fc5a9000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5566c55ecf10 0x5566c56020c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:15.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.811+0000 7fc5a3400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5566c55ed830 0x5566c55e6580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:15.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.811+0000 7fc5a9000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5566c55ecf10 0x5566c56020c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39834/0 (socket says 172.21.15.45:39834) 2024-04-16T17:10:15.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.811+0000 7fc5a9000640 1 -- 172.21.15.45:0/4160143147 learned_addr learned my addr 172.21.15.45:0/4160143147 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:10:15.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.812+0000 7fc5a9000640 1 --2- 172.21.15.45:0/4160143147 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5566c55f6410 0x5566c560e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:15.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.812+0000 7fc5a3400640 1 -- 172.21.15.45:0/4160143147 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5566c55f6410 msgr2=0x5566c560e2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:10:15.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.812+0000 7fc5a3400640 1 --2- 172.21.15.45:0/4160143147 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5566c55f6410 0x5566c560e2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:15.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.812+0000 7fc5a3400640 1 -- 172.21.15.45:0/4160143147 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5566c55ecf10 msgr2=0x5566c56020c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:10:15.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.812+0000 7fc5a3400640 1 --2- 172.21.15.45:0/4160143147 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5566c55ecf10 0x5566c56020c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:15.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.812+0000 7fc5a3400640 1 -- 172.21.15.45:0/4160143147 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5566c52643f0 con 0x5566c55ed830 2024-04-16T17:10:15.814 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.812+0000 7fc5a9000640 1 --2- 172.21.15.45:0/4160143147 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5566c55ecf10 0x5566c56020c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:10:15.814 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.812+0000 7fc5a3400640 1 --2- 172.21.15.45:0/4160143147 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5566c55ed830 0x5566c55e6580 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x5566c5609210 tx=0x7fc590005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=645d9b8769a6b937 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:10:15.814 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.812+0000 7fc598c00640 1 -- 172.21.15.45:0/4160143147 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc590023070 con 0x5566c55ed830 2024-04-16T17:10:15.814 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.812+0000 7fc598c00640 1 -- 172.21.15.45:0/4160143147 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc590005500 con 0x5566c55ed830 2024-04-16T17:10:15.814 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.812+0000 7fc598c00640 1 -- 172.21.15.45:0/4160143147 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc590004e40 con 0x5566c55ed830 2024-04-16T17:10:15.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.813+0000 7fc5adc39740 1 -- 172.21.15.45:0/4160143147 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5566c55ed830 msgr2=0x5566c55e6580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:15.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.813+0000 7fc5adc39740 1 --2- 172.21.15.45:0/4160143147 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5566c55ed830 0x5566c55e6580 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x5566c5609210 tx=0x7fc590005f90 comp rx=0 tx=0).stop 2024-04-16T17:10:15.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.813+0000 7fc5adc39740 1 -- 172.21.15.45:0/4160143147 shutdown_connections 2024-04-16T17:10:15.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.813+0000 7fc5adc39740 1 --2- 172.21.15.45:0/4160143147 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5566c55ed830 0x5566c55e6580 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:15.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.813+0000 7fc5adc39740 1 --2- 172.21.15.45:0/4160143147 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5566c55f6410 0x5566c560e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:15.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.813+0000 7fc5adc39740 1 --2- 172.21.15.45:0/4160143147 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5566c55ecf10 0x5566c56020c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:15.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.813+0000 7fc5adc39740 1 -- 172.21.15.45:0/4160143147 >> 172.21.15.45:0/4160143147 conn(0x5566c55e7640 msgr2=0x5566c55e5e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:10:15.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.813+0000 7fc5adc39740 1 -- 172.21.15.45:0/4160143147 shutdown_connections 2024-04-16T17:10:15.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.813+0000 7fc5adc39740 1 -- 172.21.15.45:0/4160143147 wait complete. 2024-04-16T17:10:15.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.814+0000 7fc5adc39740 1 -- start start 2024-04-16T17:10:15.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.814+0000 7fc5adc39740 1 Processor -- start 2024-04-16T17:10:15.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.814+0000 7fc5adc39740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5566c55ecf10 0x5566c55fe530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:15.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.814+0000 7fc5adc39740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5566c55ed830 0x5566c55fea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:15.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.814+0000 7fc5adc39740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5566c55f6410 0x5566c55f9a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:15.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.814+0000 7fc5adc39740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fc5a41b3f10 con 0x5566c55ed830 2024-04-16T17:10:15.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.814+0000 7fc5adc39740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fc5a41b5380 con 0x5566c55ecf10 2024-04-16T17:10:15.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.814+0000 7fc5adc39740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fc5a41b45e0 con 0x5566c55f6410 2024-04-16T17:10:15.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.814+0000 7fc5adc39740 10 client.0 _ll_register_callbacks cb 0x7fc5900097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:10:15.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.815+0000 7fc5a9000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5566c55f6410 0x5566c55f9a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:15.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.815+0000 7fc5a3400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5566c55ed830 0x5566c55fea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:15.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.815+0000 7fc5a9000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5566c55f6410 0x5566c55f9a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:33492/0 (socket says 172.21.15.45:33492) 2024-04-16T17:10:15.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.815+0000 7fc5a9000640 1 -- 172.21.15.45:0/1454184072 learned_addr learned my addr 172.21.15.45:0/1454184072 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:10:15.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.815+0000 7fc5a9000640 1 --2- 172.21.15.45:0/1454184072 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5566c55ecf10 0x5566c55fe530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:15.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.815+0000 7fc5a9000640 1 -- 172.21.15.45:0/1454184072 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5566c55ecf10 msgr2=0x5566c55fe530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:10:15.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.815+0000 7fc5a9000640 1 --2- 172.21.15.45:0/1454184072 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5566c55ecf10 0x5566c55fe530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:15.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.815+0000 7fc5a9000640 1 -- 172.21.15.45:0/1454184072 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5566c55ed830 msgr2=0x5566c55fea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:10:15.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.815+0000 7fc5a9000640 1 --2- 172.21.15.45:0/1454184072 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5566c55ed830 0x5566c55fea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:15.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.815+0000 7fc5a9000640 1 -- 172.21.15.45:0/1454184072 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5566c532e7d0 con 0x5566c55f6410 2024-04-16T17:10:15.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.815+0000 7fc5a9000640 1 --2- 172.21.15.45:0/1454184072 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5566c55f6410 0x5566c55f9a30 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fc59c005d70 tx=0x7fc59c005bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2076b373a0249e7c server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:10:15.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.815+0000 7fc58e000640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-04-16T17:10:15.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.815+0000 7fc58e000640 1 -- 172.21.15.45:0/1454184072 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc59c004920 con 0x5566c55f6410 2024-04-16T17:10:15.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.815+0000 7fc58e000640 1 -- 172.21.15.45:0/1454184072 <== mon.1 v2:172.21.15.117:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fc59c003dd0 con 0x5566c55f6410 2024-04-16T17:10:15.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.816+0000 7fc5adc39740 1 -- 172.21.15.45:0/1454184072 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc590005500 con 0x5566c55f6410 2024-04-16T17:10:15.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.816+0000 7fc58e000640 1 -- 172.21.15.45:0/1454184072 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fc59c003fd0 con 0x5566c55f6410 2024-04-16T17:10:15.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.816+0000 7fc5adc39740 10 client.26575 fetch_fsmap learned FSMap version 30 2024-04-16T17:10:15.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.816+0000 7fc5adc39740 1 -- 172.21.15.45:0/1454184072 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5566c566c500 con 0x5566c55f6410 2024-04-16T17:10:15.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.816+0000 7fc58e000640 1 -- 172.21.15.45:0/1454184072 <== mon.1 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc590005500 con 0x5566c55f6410 2024-04-16T17:10:15.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.816+0000 7fc58e000640 1 -- 172.21.15.45:0/1454184072 <== mon.1 v2:172.21.15.117:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fc59c019af0 con 0x5566c55f6410 2024-04-16T17:10:15.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.816+0000 7fc5adc39740 10 client.26575 fetch_fsmap finished waiting for FSMap version 30 2024-04-16T17:10:15.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.816+0000 7fc5adc39740 10 client.26575 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-04-16T17:10:15.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.816+0000 7fc5adc39740 20 client.26575 populate_metadata read hostname 'smithi045' 2024-04-16T17:10:15.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.816+0000 7fc5adc39740 1 --2- 172.21.15.45:0/1454184072 >> [v2:172.21.15.45:6840/3755310822,v1:172.21.15.45:6841/3755310822] conn(0x5566c566cc80 0x5566c566f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:15.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.816+0000 7fc5adc39740 4 client.26575 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T17:10:15.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.816+0000 7fc5adc39740 1 -- 172.21.15.45:0/1454184072 --> [v2:172.21.15.45:6840/3755310822,v1:172.21.15.45:6841/3755310822] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5566c55598b0 con 0x5566c566cc80 2024-04-16T17:10:15.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.817+0000 7fc5a3400640 1 --2- 172.21.15.45:0/1454184072 >> [v2:172.21.15.45:6840/3755310822,v1:172.21.15.45:6841/3755310822] conn(0x5566c566cc80 0x5566c566f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:15.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.817+0000 7fc5a3400640 1 --2- 172.21.15.45:0/1454184072 >> [v2:172.21.15.45:6840/3755310822,v1:172.21.15.45:6841/3755310822] conn(0x5566c566cc80 0x5566c566f0a0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a4c7745cb235ae73 server_cookie=b4e5daa5e956ecbb in_seq=0 out_seq=0 2024-04-16T17:10:15.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.817+0000 7fc58e000640 10 client.26575 ms_handle_connect on v2:172.21.15.45:6840/3755310822 2024-04-16T17:10:15.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.817+0000 7fc58e000640 1 -- 172.21.15.45:0/1454184072 <== mds.1 v2:172.21.15.45:6840/3755310822 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5566c55598b0 con 0x5566c566cc80 2024-04-16T17:10:15.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.817+0000 7fc58e000640 10 client.26575 handle_command_reply: tid=0 2024-04-16T17:10:15.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.818+0000 7fc5adc39740 1 client.26575 shutdown 2024-04-16T17:10:15.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.818+0000 7fc5adc39740 1 -- 172.21.15.45:0/1454184072 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5566c55f6410 msgr2=0x5566c55f9a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:15.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.818+0000 7fc5adc39740 1 --2- 172.21.15.45:0/1454184072 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5566c55f6410 0x5566c55f9a30 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fc59c005d70 tx=0x7fc59c005bd0 comp rx=0 tx=0).stop 2024-04-16T17:10:15.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.818+0000 7fc5adc39740 1 -- 172.21.15.45:0/1454184072 shutdown_connections 2024-04-16T17:10:15.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.818+0000 7fc5adc39740 1 --2- 172.21.15.45:0/1454184072 >> [v2:172.21.15.45:6840/3755310822,v1:172.21.15.45:6841/3755310822] conn(0x5566c566cc80 0x5566c566f0a0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:15.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.818+0000 7fc5adc39740 1 --2- 172.21.15.45:0/1454184072 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5566c55f6410 0x5566c55f9a30 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:15.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.818+0000 7fc5adc39740 1 --2- 172.21.15.45:0/1454184072 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5566c55ed830 0x5566c55fea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:15.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.818+0000 7fc5adc39740 1 --2- 172.21.15.45:0/1454184072 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5566c55ecf10 0x5566c55fe530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:15.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.818+0000 7fc58e000640 0 client.26575 ms_handle_reset on v2:172.21.15.45:6840/3755310822 2024-04-16T17:10:15.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.818+0000 7fc5adc39740 1 -- 172.21.15.45:0/1454184072 >> 172.21.15.45:0/1454184072 conn(0x5566c55e7640 msgr2=0x5566c55f3d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:10:15.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.818+0000 7fc5adc39740 1 -- 172.21.15.45:0/1454184072 shutdown_connections 2024-04-16T17:10:15.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.818+0000 7fc5adc39740 1 -- 172.21.15.45:0/1454184072 wait complete. 2024-04-16T17:10:15.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.818+0000 7fc5adc39740 20 client.26575 trim_cache size 0 max 16384 2024-04-16T17:10:15.824 INFO:teuthology.orchestra.run.smithi045.stdout:[] 2024-04-16T17:10:15.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.819+0000 7fc5a9a00640 1 -- 172.21.15.45:0/1101147214 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fc5880a3c30 msgr2=0x7fc5880a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:15.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.819+0000 7fc5a9a00640 1 --2- 172.21.15.45:0/1101147214 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fc5880a3c30 0x7fc5880a60f0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x5566c55ec9e0 tx=0x7fc59402cdd0 comp rx=0 tx=0).stop 2024-04-16T17:10:15.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.819+0000 7fc5a9a00640 1 -- 172.21.15.45:0/1101147214 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc5a4108580 msgr2=0x7fc5a419ed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:15.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.819+0000 7fc5a9a00640 1 --2- 172.21.15.45:0/1101147214 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc5a4108580 0x7fc5a419ed60 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fc594009230 tx=0x7fc59402cb20 comp rx=0 tx=0).stop 2024-04-16T17:10:15.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.819+0000 7fc5a9a00640 1 -- 172.21.15.45:0/1101147214 shutdown_connections 2024-04-16T17:10:15.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.819+0000 7fc5a9a00640 1 --2- 172.21.15.45:0/1101147214 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fc5880a3c30 0x7fc5880a60f0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:15.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.819+0000 7fc5a9a00640 1 --2- 172.21.15.45:0/1101147214 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc5a410d500 0x7fc5a41a3620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:15.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.819+0000 7fc5a9a00640 1 --2- 172.21.15.45:0/1101147214 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc5a4108f40 0x7fc5a419f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:15.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.819+0000 7fc5a9a00640 1 --2- 172.21.15.45:0/1101147214 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc5a4108580 0x7fc5a419ed60 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:15.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.819+0000 7fc5a9a00640 1 -- 172.21.15.45:0/1101147214 >> 172.21.15.45:0/1101147214 conn(0x7fc5a4077e20 msgr2=0x7fc5a4078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:10:15.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.819+0000 7fc5a9a00640 1 -- 172.21.15.45:0/1101147214 shutdown_connections 2024-04-16T17:10:15.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:15.819+0000 7fc5a9a00640 1 -- 172.21.15.45:0/1101147214 wait complete. 2024-04-16T17:10:15.838 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-16T17:10:15.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:10:15 smithi104 ceph-mon[36457]: from='client.? 172.21.15.45:0/711476606' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:10:16.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:16.088+0000 7f1645e00640 1 -- 172.21.15.45:0/1429616170 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f16400743b0 msgr2=0x7f1640074790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:16.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:16.088+0000 7f1645e00640 1 --2- 172.21.15.45:0/1429616170 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f16400743b0 0x7f1640074790 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f1634009900 tx=0x7f163402bcc0 comp rx=0 tx=0).stop 2024-04-16T17:10:16.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:16.089+0000 7f1645e00640 1 -- 172.21.15.45:0/1429616170 shutdown_connections 2024-04-16T17:10:16.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:16.089+0000 7f1645e00640 1 --2- 172.21.15.45:0/1429616170 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f164010e760 0x7f1640110b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:16.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:16.089+0000 7f1645e00640 1 --2- 172.21.15.45:0/1429616170 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1640074cd0 0x7f164010e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:16.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:16.089+0000 7f1645e00640 1 --2- 172.21.15.45:0/1429616170 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f16400743b0 0x7f1640074790 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:16.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:16.089+0000 7f1645e00640 1 -- 172.21.15.45:0/1429616170 >> 172.21.15.45:0/1429616170 conn(0x7f16400fd9d0 msgr2=0x7f16400ffdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:10:16.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:16.089+0000 7f1645e00640 1 -- 172.21.15.45:0/1429616170 shutdown_connections 2024-04-16T17:10:16.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:16.089+0000 7f1645e00640 1 -- 172.21.15.45:0/1429616170 wait complete. 2024-04-16T17:10:16.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:16.090+0000 7f1645e00640 1 Processor -- start 2024-04-16T17:10:16.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:16.090+0000 7f1645e00640 1 -- start start 2024-04-16T17:10:16.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:16.090+0000 7f1645e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f16400743b0 0x7f164019aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:16.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:16.090+0000 7f1645e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1640074cd0 0x7f164019af90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:16.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:16.090+0000 7f1645e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f164010e760 0x7f164019f340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:16.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:16.090+0000 7f1645e00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f16401144b0 con 0x7f16400743b0 2024-04-16T17:10:16.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:16.090+0000 7f1645e00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f1640114330 con 0x7f1640074cd0 2024-04-16T17:10:16.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:16.090+0000 7f1645e00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f1640114630 con 0x7f164010e760 2024-04-16T17:10:16.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:16.090+0000 7f1644a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f16400743b0 0x7f164019aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:16.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:16.090+0000 7f163fe00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1640074cd0 0x7f164019af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:16.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:16.090+0000 7f163fe00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1640074cd0 0x7f164019af90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:40176/0 (socket says 172.21.15.45:40176) 2024-04-16T17:10:16.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:16.090+0000 7f163fe00640 1 -- 172.21.15.45:0/3180988996 learned_addr learned my addr 172.21.15.45:0/3180988996 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:10:16.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:16.090+0000 7f1644a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f16400743b0 0x7f164019aa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39864/0 (socket says 172.21.15.45:39864) 2024-04-16T17:10:16.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:16.091+0000 7f1645400640 1 --2- 172.21.15.45:0/3180988996 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f164010e760 0x7f164019f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:16.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:16.091+0000 7f163fe00640 1 -- 172.21.15.45:0/3180988996 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f164010e760 msgr2=0x7f164019f340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:16.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:16.091+0000 7f163fe00640 1 --2- 172.21.15.45:0/3180988996 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f164010e760 0x7f164019f340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:16.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:16.091+0000 7f163fe00640 1 -- 172.21.15.45:0/3180988996 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f16400743b0 msgr2=0x7f164019aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:16.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:16.091+0000 7f163fe00640 1 --2- 172.21.15.45:0/3180988996 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f16400743b0 0x7f164019aa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:16.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:16.091+0000 7f163fe00640 1 -- 172.21.15.45:0/3180988996 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f164019fa70 con 0x7f1640074cd0 2024-04-16T17:10:16.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:16.091+0000 7f163fe00640 1 --2- 172.21.15.45:0/3180988996 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1640074cd0 0x7f164019af90 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f162c00e9f0 tx=0x7f162c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:10:16.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:16.091+0000 7f163d600640 1 -- 172.21.15.45:0/3180988996 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f162c00cd80 con 0x7f1640074cd0 2024-04-16T17:10:16.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:16.091+0000 7f1645e00640 1 -- 172.21.15.45:0/3180988996 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f164019fd60 con 0x7f1640074cd0 2024-04-16T17:10:16.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:16.092+0000 7f1645e00640 1 -- 172.21.15.45:0/3180988996 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16401a75f0 con 0x7f1640074cd0 2024-04-16T17:10:16.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:16.092+0000 7f163d600640 1 -- 172.21.15.45:0/3180988996 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f162c010910 con 0x7f1640074cd0 2024-04-16T17:10:16.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:16.092+0000 7f163d600640 1 -- 172.21.15.45:0/3180988996 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f162c0188f0 con 0x7f1640074cd0 2024-04-16T17:10:16.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:16.096+0000 7f164a20a740 1 Processor -- start 2024-04-16T17:10:16.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:16.096+0000 7f164a20a740 1 -- start start 2024-04-16T17:10:16.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:16.096+0000 7f164a20a740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563725775720 0x563725775b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:16.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:16.096+0000 7f164a20a740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563725776040 0x56372577e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:16.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:16.096+0000 7f164a20a740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56372577ec20 0x563725781010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:16.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:16.096+0000 7f164a20a740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5637256d2020 con 0x56372577ec20 2024-04-16T17:10:16.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:16.096+0000 7f164a20a740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x563725678460 con 0x563725776040 2024-04-16T17:10:16.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:16.096+0000 7f164a20a740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5637256cdfe0 con 0x563725775720 2024-04-16T17:10:16.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:16.097+0000 7f163d600640 1 -- 172.21.15.45:0/3180988996 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f162c018bb0 con 0x7f1640074cd0 2024-04-16T17:10:16.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:16.097+0000 7f1645400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563725775720 0x563725775b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:16.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:16.097+0000 7f1645400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563725775720 0x563725775b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:33528/0 (socket says 172.21.15.45:33528) 2024-04-16T17:10:16.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:16.097+0000 7f1645400640 1 -- 172.21.15.45:0/1816714506 learned_addr learned my addr 172.21.15.45:0/1816714506 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:10:16.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:16.097+0000 7f1644a00640 1 --2- 172.21.15.45:0/1816714506 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563725776040 0x56372577e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:16.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:16.097+0000 7f1645400640 1 -- 172.21.15.45:0/1816714506 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563725776040 msgr2=0x56372577e6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:10:16.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:16.097+0000 7f163fe00640 1 --2- 172.21.15.45:0/1816714506 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56372577ec20 0x563725781010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:16.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:16.097+0000 7f1645400640 1 --2- 172.21.15.45:0/1816714506 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563725776040 0x56372577e6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:16.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:16.097+0000 7f1645400640 1 -- 172.21.15.45:0/1816714506 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56372577ec20 msgr2=0x563725781010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:10:16.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:16.097+0000 7f1645400640 1 --2- 172.21.15.45:0/1816714506 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56372577ec20 0x563725781010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:16.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:16.097+0000 7f1645400640 1 -- 172.21.15.45:0/1816714506 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563725680040 con 0x563725775720 2024-04-16T17:10:16.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:16.098+0000 7f163fe00640 1 --2- 172.21.15.45:0/1816714506 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56372577ec20 0x563725781010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:10:16.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:16.098+0000 7f1645400640 1 --2- 172.21.15.45:0/1816714506 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563725775720 0x563725775b00 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f1638009e60 tx=0x7f1638010740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=df6c88891f4149ea server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:10:16.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:16.098+0000 7f1631c00640 1 -- 172.21.15.45:0/1816714506 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1638016070 con 0x563725775720 2024-04-16T17:10:16.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:16.098+0000 7f1631c00640 1 -- 172.21.15.45:0/1816714506 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1638004d20 con 0x563725775720 2024-04-16T17:10:16.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:16.098+0000 7f1631c00640 1 -- 172.21.15.45:0/1816714506 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1638012870 con 0x563725775720 2024-04-16T17:10:16.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:16.099+0000 7f163d600640 1 --2- 172.21.15.45:0/3180988996 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f16280a3da0 0x7f16280a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:16.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:16.099+0000 7f164a20a740 1 -- 172.21.15.45:0/1816714506 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563725775720 msgr2=0x563725775b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:16.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:16.099+0000 7f164a20a740 1 --2- 172.21.15.45:0/1816714506 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563725775720 0x563725775b00 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f1638009e60 tx=0x7f1638010740 comp rx=0 tx=0).stop 2024-04-16T17:10:16.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:16.099+0000 7f164a20a740 1 -- 172.21.15.45:0/1816714506 shutdown_connections 2024-04-16T17:10:16.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:16.099+0000 7f164a20a740 1 --2- 172.21.15.45:0/1816714506 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56372577ec20 0x563725781010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:16.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:16.099+0000 7f164a20a740 1 --2- 172.21.15.45:0/1816714506 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563725776040 0x56372577e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:16.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:16.099+0000 7f164a20a740 1 --2- 172.21.15.45:0/1816714506 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563725775720 0x563725775b00 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:16.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:16.099+0000 7f164a20a740 1 -- 172.21.15.45:0/1816714506 >> 172.21.15.45:0/1816714506 conn(0x56372576fe50 msgr2=0x563725770230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:10:16.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:16.099+0000 7f163d600640 1 -- 172.21.15.45:0/3180988996 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f162c0d0dd0 con 0x7f1640074cd0 2024-04-16T17:10:16.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:16.100+0000 7f164a20a740 1 -- 172.21.15.45:0/1816714506 shutdown_connections 2024-04-16T17:10:16.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:16.100+0000 7f1644a00640 1 --2- 172.21.15.45:0/3180988996 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f16280a3da0 0x7f16280a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:16.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:16.101+0000 7f164a20a740 1 -- 172.21.15.45:0/1816714506 wait complete. 2024-04-16T17:10:16.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:16.101+0000 7f164a20a740 1 -- start start 2024-04-16T17:10:16.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:16.101+0000 7f1644a00640 1 --2- 172.21.15.45:0/3180988996 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f16280a3da0 0x7f16280a6260 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f1634005ab0 tx=0x7f1634005a20 comp rx=0 tx=0).ready entity=mgr.25411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:10:16.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:16.102+0000 7f164a20a740 1 Processor -- start 2024-04-16T17:10:16.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:16.102+0000 7f164a20a740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563725775720 0x56372577bc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:16.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:16.102+0000 7f164a20a740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56372577ec20 0x56372577c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:16.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:16.102+0000 7f164a20a740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56372577c6f0 0x56372578af10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:16.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:16.102+0000 7f164a20a740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x563725781a10 con 0x56372577c6f0 2024-04-16T17:10:16.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:16.102+0000 7f164a20a740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x56372568b830 con 0x563725775720 2024-04-16T17:10:16.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:16.102+0000 7f164a20a740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x563725781b90 con 0x56372577ec20 2024-04-16T17:10:16.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:16.102+0000 7f164a20a740 10 client.0 _ll_register_callbacks cb 0x563725693150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:10:16.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:16.103+0000 7f163fe00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56372577ec20 0x56372577c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:16.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:16.103+0000 7f1644a00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563725775720 0x56372577bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:16.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:16.103+0000 7f163fe00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56372577ec20 0x56372577c1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:33530/0 (socket says 172.21.15.45:33530) 2024-04-16T17:10:16.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:16.103+0000 7f163fe00640 1 -- 172.21.15.45:0/185687343 learned_addr learned my addr 172.21.15.45:0/185687343 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:10:16.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:16.103+0000 7f163fe00640 1 -- 172.21.15.45:0/185687343 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563725775720 msgr2=0x56372577bc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:10:16.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:16.103+0000 7f163fe00640 1 --2- 172.21.15.45:0/185687343 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563725775720 0x56372577bc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:16.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:16.103+0000 7f163fe00640 1 -- 172.21.15.45:0/185687343 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56372577c6f0 msgr2=0x56372578af10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:10:16.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:16.103+0000 7f163fe00640 1 --2- 172.21.15.45:0/185687343 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56372577c6f0 0x56372578af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:16.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:16.103+0000 7f163fe00640 1 -- 172.21.15.45:0/185687343 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56372568f920 con 0x56372577ec20 2024-04-16T17:10:16.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:16.103+0000 7f163fe00640 1 --2- 172.21.15.45:0/185687343 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56372577ec20 0x56372577c1b0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f162c0ce210 tx=0x7f162c09d100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c46a629737371e9c server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:10:16.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:16.103+0000 7f1626000640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-04-16T17:10:16.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:16.104+0000 7f1626000640 1 -- 172.21.15.45:0/185687343 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f162c0ce640 con 0x56372577ec20 2024-04-16T17:10:16.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:16.104+0000 7f1626000640 1 -- 172.21.15.45:0/185687343 <== mon.1 v2:172.21.15.117:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f162c0bbdc0 con 0x56372577ec20 2024-04-16T17:10:16.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:16.104+0000 7f164a20a740 1 -- 172.21.15.45:0/185687343 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1638004d20 con 0x56372577ec20 2024-04-16T17:10:16.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:16.104+0000 7f1626000640 1 -- 172.21.15.45:0/185687343 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f162c0a0db0 con 0x56372577ec20 2024-04-16T17:10:16.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:16.104+0000 7f164a20a740 10 client.26587 fetch_fsmap learned FSMap version 30 2024-04-16T17:10:16.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:16.104+0000 7f164a20a740 1 -- 172.21.15.45:0/185687343 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56372565d7f0 con 0x56372577ec20 2024-04-16T17:10:16.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:16.104+0000 7f1626000640 1 -- 172.21.15.45:0/185687343 <== mon.1 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f162c0b7030 con 0x56372577ec20 2024-04-16T17:10:16.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:16.105+0000 7f1626000640 1 -- 172.21.15.45:0/185687343 <== mon.1 v2:172.21.15.117:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f162c098750 con 0x56372577ec20 2024-04-16T17:10:16.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:16.105+0000 7f164a20a740 10 client.26587 fetch_fsmap finished waiting for FSMap version 30 2024-04-16T17:10:16.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:16.105+0000 7f164a20a740 10 client.26587 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:10:16.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:16.105+0000 7f164a20a740 20 client.26587 populate_metadata read hostname 'smithi045' 2024-04-16T17:10:16.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:16.105+0000 7f164a20a740 1 --2- 172.21.15.45:0/185687343 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x56372578f3f0 0x563725791850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:16.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:16.105+0000 7f164a20a740 4 client.26587 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:10:16.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:16.105+0000 7f164a20a740 1 -- 172.21.15.45:0/185687343 --> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5637256dfa40 con 0x56372578f3f0 2024-04-16T17:10:16.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:16.105+0000 7f1644a00640 1 --2- 172.21.15.45:0/185687343 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x56372578f3f0 0x563725791850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:16.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:16.106+0000 7f1644a00640 1 --2- 172.21.15.45:0/185687343 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x56372578f3f0 0x563725791850 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=39ee103f9121ef93 server_cookie=ac1063521f926963 in_seq=0 out_seq=0 2024-04-16T17:10:16.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:16.106+0000 7f1626000640 10 client.26587 ms_handle_connect on v2:172.21.15.104:6838/3660070592 2024-04-16T17:10:16.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:16.107+0000 7f1626000640 1 -- 172.21.15.45:0/185687343 <== mds.0 v2:172.21.15.104:6838/3660070592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5637256dfa40 con 0x56372578f3f0 2024-04-16T17:10:16.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:16.107+0000 7f1626000640 10 client.26587 handle_command_reply: tid=0 2024-04-16T17:10:16.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:16.107+0000 7f164a20a740 1 client.26587 shutdown 2024-04-16T17:10:16.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:16.108+0000 7f164a20a740 1 -- 172.21.15.45:0/185687343 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56372577ec20 msgr2=0x56372577c1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:16.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:16.108+0000 7f164a20a740 1 --2- 172.21.15.45:0/185687343 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56372577ec20 0x56372577c1b0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f162c0ce210 tx=0x7f162c09d100 comp rx=0 tx=0).stop 2024-04-16T17:10:16.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:16.108+0000 7f164a20a740 1 -- 172.21.15.45:0/185687343 shutdown_connections 2024-04-16T17:10:16.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:16.108+0000 7f164a20a740 1 --2- 172.21.15.45:0/185687343 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x56372578f3f0 0x563725791850 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:16.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:16.108+0000 7f164a20a740 1 --2- 172.21.15.45:0/185687343 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56372577ec20 0x56372577c1b0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:16.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:16.108+0000 7f164a20a740 1 --2- 172.21.15.45:0/185687343 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56372577c6f0 0x56372578af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:16.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:16.108+0000 7f164a20a740 1 --2- 172.21.15.45:0/185687343 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563725775720 0x56372577bc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:16.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:16.108+0000 7f164a20a740 1 -- 172.21.15.45:0/185687343 >> 172.21.15.45:0/185687343 conn(0x56372576fe50 msgr2=0x563725771ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:10:16.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:16.108+0000 7f1626000640 0 client.26587 ms_handle_reset on v2:172.21.15.104:6838/3660070592 2024-04-16T17:10:16.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:16.108+0000 7f164a20a740 1 -- 172.21.15.45:0/185687343 shutdown_connections 2024-04-16T17:10:16.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:16.108+0000 7f164a20a740 1 -- 172.21.15.45:0/185687343 wait complete. 2024-04-16T17:10:16.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:16.108+0000 7f164a20a740 20 client.26587 trim_cache size 0 max 16384 2024-04-16T17:10:16.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:16.136+0000 7f164a20a740 1 Processor -- start 2024-04-16T17:10:16.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:16.137+0000 7f164a20a740 1 -- start start 2024-04-16T17:10:16.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:16.137+0000 7f164a20a740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563725775720 0x56372577add0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:16.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:16.137+0000 7f164a20a740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56372577c6f0 0x563725766580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:16.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:16.137+0000 7f164a20a740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56372577ec20 0x563725797160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:16.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:16.137+0000 7f164a20a740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f16200127c0 con 0x563725775720 2024-04-16T17:10:16.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:16.137+0000 7f164a20a740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f162000ffd0 con 0x56372577c6f0 2024-04-16T17:10:16.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:16.137+0000 7f164a20a740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f162000de00 con 0x56372577ec20 2024-04-16T17:10:16.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:16.137+0000 7f1644a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563725775720 0x56372577add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:16.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:16.137+0000 7f163fe00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56372577c6f0 0x563725766580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:16.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:16.137+0000 7f163fe00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56372577c6f0 0x563725766580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:40214/0 (socket says 172.21.15.45:40214) 2024-04-16T17:10:16.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:16.137+0000 7f163fe00640 1 -- 172.21.15.45:0/3939787993 learned_addr learned my addr 172.21.15.45:0/3939787993 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:10:16.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:16.138+0000 7f1645400640 1 --2- 172.21.15.45:0/3939787993 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56372577ec20 0x563725797160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:16.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:16.138+0000 7f163fe00640 1 -- 172.21.15.45:0/3939787993 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56372577ec20 msgr2=0x563725797160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:10:16.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:16.138+0000 7f163fe00640 1 --2- 172.21.15.45:0/3939787993 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56372577ec20 0x563725797160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:16.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:16.138+0000 7f163fe00640 1 -- 172.21.15.45:0/3939787993 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563725775720 msgr2=0x56372577add0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:10:16.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:16.138+0000 7f163fe00640 1 --2- 172.21.15.45:0/3939787993 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563725775720 0x56372577add0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:16.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:16.138+0000 7f163fe00640 1 -- 172.21.15.45:0/3939787993 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563725775e70 con 0x56372577c6f0 2024-04-16T17:10:16.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:16.138+0000 7f163fe00640 1 --2- 172.21.15.45:0/3939787993 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56372577c6f0 0x563725766580 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f162c099380 tx=0x7f162c098c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=15bfac265e7fce97 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:10:16.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:16.138+0000 7f1631200640 1 -- 172.21.15.45:0/3939787993 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f162c0a5670 con 0x56372577c6f0 2024-04-16T17:10:16.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:16.138+0000 7f1631200640 1 -- 172.21.15.45:0/3939787993 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f162c0b7070 con 0x56372577c6f0 2024-04-16T17:10:16.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:16.138+0000 7f1631200640 1 -- 172.21.15.45:0/3939787993 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f162c09be50 con 0x56372577c6f0 2024-04-16T17:10:16.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:16.139+0000 7f164a20a740 1 -- 172.21.15.45:0/3939787993 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56372577c6f0 msgr2=0x563725766580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:16.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:16.139+0000 7f164a20a740 1 --2- 172.21.15.45:0/3939787993 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56372577c6f0 0x563725766580 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f162c099380 tx=0x7f162c098c00 comp rx=0 tx=0).stop 2024-04-16T17:10:16.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:16.140+0000 7f164a20a740 1 -- 172.21.15.45:0/3939787993 shutdown_connections 2024-04-16T17:10:16.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:16.140+0000 7f164a20a740 1 --2- 172.21.15.45:0/3939787993 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56372577ec20 0x563725797160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:16.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:16.140+0000 7f164a20a740 1 --2- 172.21.15.45:0/3939787993 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56372577c6f0 0x563725766580 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:16.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:16.140+0000 7f164a20a740 1 --2- 172.21.15.45:0/3939787993 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563725775720 0x56372577add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:16.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:16.140+0000 7f164a20a740 1 -- 172.21.15.45:0/3939787993 >> 172.21.15.45:0/3939787993 conn(0x56372576fe50 msgr2=0x563725765e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:10:16.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:16.140+0000 7f164a20a740 1 -- 172.21.15.45:0/3939787993 shutdown_connections 2024-04-16T17:10:16.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:16.140+0000 7f164a20a740 1 -- 172.21.15.45:0/3939787993 wait complete. 2024-04-16T17:10:16.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:16.140+0000 7f164a20a740 1 -- start start 2024-04-16T17:10:16.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:16.141+0000 7f164a20a740 1 Processor -- start 2024-04-16T17:10:16.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:16.141+0000 7f164a20a740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563725775720 0x563725782c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:16.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:16.141+0000 7f164a20a740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56372577c6f0 0x563725783140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:16.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:16.141+0000 7f164a20a740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56372577ec20 0x5637257f49d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:16.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:16.141+0000 7f164a20a740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f162000e490 con 0x56372577c6f0 2024-04-16T17:10:16.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:16.141+0000 7f164a20a740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f162000f900 con 0x56372577ec20 2024-04-16T17:10:16.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:16.141+0000 7f164a20a740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f162000eb60 con 0x563725775720 2024-04-16T17:10:16.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:16.141+0000 7f164a20a740 10 client.0 _ll_register_callbacks cb 0x7f162c00cf20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:10:16.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:16.141+0000 7f1644a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563725775720 0x563725782c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:16.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:16.142+0000 7f1644a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563725775720 0x563725782c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:33548/0 (socket says 172.21.15.45:33548) 2024-04-16T17:10:16.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:16.142+0000 7f1644a00640 1 -- 172.21.15.45:0/1211336145 learned_addr learned my addr 172.21.15.45:0/1211336145 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:10:16.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:16.142+0000 7f163fe00640 1 --2- 172.21.15.45:0/1211336145 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56372577c6f0 0x563725783140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:16.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:16.142+0000 7f1645400640 1 --2- 172.21.15.45:0/1211336145 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56372577ec20 0x5637257f49d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:16.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:16.142+0000 7f1644a00640 1 -- 172.21.15.45:0/1211336145 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56372577ec20 msgr2=0x5637257f49d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:10:16.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:16.142+0000 7f1644a00640 1 --2- 172.21.15.45:0/1211336145 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56372577ec20 0x5637257f49d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:16.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:16.142+0000 7f1644a00640 1 -- 172.21.15.45:0/1211336145 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56372577c6f0 msgr2=0x563725783140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:10:16.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:16.142+0000 7f1644a00640 1 --2- 172.21.15.45:0/1211336145 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56372577c6f0 0x563725783140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:16.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:16.142+0000 7f1644a00640 1 -- 172.21.15.45:0/1211336145 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5637253e43f0 con 0x563725775720 2024-04-16T17:10:16.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:16.142+0000 7f1645400640 1 --2- 172.21.15.45:0/1211336145 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56372577ec20 0x5637257f49d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:10:16.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:16.143+0000 7f1644a00640 1 --2- 172.21.15.45:0/1211336145 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563725775720 0x563725782c00 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f16340041d0 tx=0x7f163402e050 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b7e73112f6f3b252 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:10:16.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:16.143+0000 7f1626000640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-04-16T17:10:16.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:16.144+0000 7f1626000640 1 -- 172.21.15.45:0/1211336145 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1634054cd0 con 0x563725775720 2024-04-16T17:10:16.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:16.144+0000 7f1626000640 1 -- 172.21.15.45:0/1211336145 <== mon.1 v2:172.21.15.117:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f163403a040 con 0x563725775720 2024-04-16T17:10:16.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:16.144+0000 7f164a20a740 1 -- 172.21.15.45:0/1211336145 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f162c0b7070 con 0x563725775720 2024-04-16T17:10:16.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:16.144+0000 7f1626000640 1 -- 172.21.15.45:0/1211336145 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f163402c910 con 0x563725775720 2024-04-16T17:10:16.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:16.144+0000 7f164a20a740 10 client.26593 fetch_fsmap learned FSMap version 30 2024-04-16T17:10:16.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:16.144+0000 7f164a20a740 1 -- 172.21.15.45:0/1211336145 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5637254ae7d0 con 0x563725775720 2024-04-16T17:10:16.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:16.145+0000 7f1626000640 1 -- 172.21.15.45:0/1211336145 <== mon.1 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f162c0b7070 con 0x563725775720 2024-04-16T17:10:16.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:16.145+0000 7f1626000640 1 -- 172.21.15.45:0/1211336145 <== mon.1 v2:172.21.15.117:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f1634054240 con 0x563725775720 2024-04-16T17:10:16.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:16.145+0000 7f164a20a740 10 client.26593 fetch_fsmap finished waiting for FSMap version 30 2024-04-16T17:10:16.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:16.145+0000 7f164a20a740 10 client.26593 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:10:16.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:16.145+0000 7f164a20a740 20 client.26593 populate_metadata read hostname 'smithi045' 2024-04-16T17:10:16.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:16.145+0000 7f164a20a740 1 --2- 172.21.15.45:0/1211336145 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x5637257f6450 0x5637257f8870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:16.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:16.145+0000 7f164a20a740 4 client.26593 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T17:10:16.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:16.145+0000 7f164a20a740 1 -- 172.21.15.45:0/1211336145 --> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5637256d98b0 con 0x5637257f6450 2024-04-16T17:10:16.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:16.145+0000 7f163fe00640 1 --2- 172.21.15.45:0/1211336145 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x5637257f6450 0x5637257f8870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:16.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:16.146+0000 7f163fe00640 1 --2- 172.21.15.45:0/1211336145 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x5637257f6450 0x5637257f8870 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c804cc42c5ec7061 server_cookie=e11f2e2807a5889e in_seq=0 out_seq=0 2024-04-16T17:10:16.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:16.146+0000 7f1626000640 10 client.26593 ms_handle_connect on v2:172.21.15.104:6838/3660070592 2024-04-16T17:10:16.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:16.146+0000 7f1626000640 1 -- 172.21.15.45:0/1211336145 <== mds.0 v2:172.21.15.104:6838/3660070592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5637256d98b0 con 0x5637257f6450 2024-04-16T17:10:16.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:16.146+0000 7f1626000640 10 client.26593 handle_command_reply: tid=0 2024-04-16T17:10:16.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:16.146+0000 7f164a20a740 1 client.26593 shutdown 2024-04-16T17:10:16.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:16.147+0000 7f164a20a740 1 -- 172.21.15.45:0/1211336145 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563725775720 msgr2=0x563725782c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:16.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:16.147+0000 7f164a20a740 1 --2- 172.21.15.45:0/1211336145 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563725775720 0x563725782c00 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f16340041d0 tx=0x7f163402e050 comp rx=0 tx=0).stop 2024-04-16T17:10:16.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:16.147+0000 7f164a20a740 1 -- 172.21.15.45:0/1211336145 shutdown_connections 2024-04-16T17:10:16.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:16.147+0000 7f164a20a740 1 --2- 172.21.15.45:0/1211336145 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56372577ec20 0x5637257f49d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:16.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:16.147+0000 7f164a20a740 1 --2- 172.21.15.45:0/1211336145 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56372577c6f0 0x563725783140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:16.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:16.147+0000 7f164a20a740 1 --2- 172.21.15.45:0/1211336145 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x5637257f6450 0x5637257f8870 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:16.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:16.147+0000 7f164a20a740 1 --2- 172.21.15.45:0/1211336145 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563725775720 0x563725782c00 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:16.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:16.147+0000 7f164a20a740 1 -- 172.21.15.45:0/1211336145 >> 172.21.15.45:0/1211336145 conn(0x56372576fe50 msgr2=0x563725765e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:10:16.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:16.147+0000 7f1626000640 0 client.26593 ms_handle_reset on v2:172.21.15.104:6838/3660070592 2024-04-16T17:10:16.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:16.147+0000 7f164a20a740 1 -- 172.21.15.45:0/1211336145 shutdown_connections 2024-04-16T17:10:16.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:16.147+0000 7f164a20a740 1 -- 172.21.15.45:0/1211336145 wait complete. 2024-04-16T17:10:16.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:16.147+0000 7f164a20a740 20 client.26593 trim_cache size 0 max 16384 2024-04-16T17:10:16.149 INFO:teuthology.orchestra.run.smithi045.stdout:[] 2024-04-16T17:10:16.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:16.148+0000 7f1645e00640 1 -- 172.21.15.45:0/3180988996 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f16280a3da0 msgr2=0x7f16280a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:16.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:16.148+0000 7f1645e00640 1 --2- 172.21.15.45:0/3180988996 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f16280a3da0 0x7f16280a6260 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f1634005ab0 tx=0x7f1634005a20 comp rx=0 tx=0).stop 2024-04-16T17:10:16.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:16.148+0000 7f1645e00640 1 -- 172.21.15.45:0/3180988996 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1640074cd0 msgr2=0x7f164019af90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:16.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:16.148+0000 7f1645e00640 1 --2- 172.21.15.45:0/3180988996 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1640074cd0 0x7f164019af90 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f162c00e9f0 tx=0x7f162c00eeb0 comp rx=0 tx=0).stop 2024-04-16T17:10:16.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:16.149+0000 7f1645e00640 1 -- 172.21.15.45:0/3180988996 shutdown_connections 2024-04-16T17:10:16.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:16.149+0000 7f1645e00640 1 --2- 172.21.15.45:0/3180988996 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f16280a3da0 0x7f16280a6260 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:16.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:16.149+0000 7f1645e00640 1 --2- 172.21.15.45:0/3180988996 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f164010e760 0x7f164019f340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:16.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:16.149+0000 7f1645e00640 1 --2- 172.21.15.45:0/3180988996 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1640074cd0 0x7f164019af90 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:16.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:16.149+0000 7f1645e00640 1 --2- 172.21.15.45:0/3180988996 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f16400743b0 0x7f164019aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:16.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:16.149+0000 7f1645e00640 1 -- 172.21.15.45:0/3180988996 >> 172.21.15.45:0/3180988996 conn(0x7f16400fd9d0 msgr2=0x7f16400fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:10:16.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:16.149+0000 7f1645e00640 1 -- 172.21.15.45:0/3180988996 shutdown_connections 2024-04-16T17:10:16.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:16.149+0000 7f1645e00640 1 -- 172.21.15.45:0/3180988996 wait complete. 2024-04-16T17:10:16.230 INFO:tasks.workunit.client.0.smithi045.stdout: 1 45015 0.47 MB/sec execute 18 sec latency 91.252 ms 2024-04-16T17:10:16.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:10:16 smithi117 ceph-mon[36730]: pgmap v108: 129 pgs: 129 active+clean; 31 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 112 KiB/s wr, 40 op/s 2024-04-16T17:10:16.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:10:16 smithi104 ceph-mon[36457]: pgmap v108: 129 pgs: 129 active+clean; 31 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 112 KiB/s wr, 40 op/s 2024-04-16T17:10:17.170 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-16T17:10:17.170 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T17:10:17.170 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-16T17:10:17.230 INFO:tasks.workunit.client.0.smithi045.stdout: 1 46157 0.99 MB/sec execute 19 sec latency 17.606 ms 2024-04-16T17:10:17.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.393+0000 7f8237400640 1 -- 172.21.15.45:0/3206262443 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8230109580 msgr2=0x7f8230111150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:17.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.393+0000 7f8237400640 1 --2- 172.21.15.45:0/3206262443 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8230109580 0x7f8230111150 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f8228009ca0 tx=0x7f822802c0f0 comp rx=0 tx=0).stop 2024-04-16T17:10:17.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.393+0000 7f8237400640 1 -- 172.21.15.45:0/3206262443 shutdown_connections 2024-04-16T17:10:17.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.393+0000 7f8237400640 1 --2- 172.21.15.45:0/3206262443 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8230109580 0x7f8230111150 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:17.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.393+0000 7f8237400640 1 --2- 172.21.15.45:0/3206262443 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8230104e60 0x7f8230108ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:17.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.393+0000 7f8237400640 1 --2- 172.21.15.45:0/3206262443 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f82301044b0 0x7f8230104890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:17.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.393+0000 7f8237400640 1 -- 172.21.15.45:0/3206262443 >> 172.21.15.45:0/3206262443 conn(0x7f82300ffd90 msgr2=0x7f82301021b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:10:17.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.393+0000 7f8237400640 1 -- 172.21.15.45:0/3206262443 shutdown_connections 2024-04-16T17:10:17.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.393+0000 7f8237400640 1 -- 172.21.15.45:0/3206262443 wait complete. 2024-04-16T17:10:17.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.394+0000 7f8237400640 1 Processor -- start 2024-04-16T17:10:17.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.395+0000 7f8237400640 1 -- start start 2024-04-16T17:10:17.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.395+0000 7f8237400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f82301044b0 0x7f82301a0830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:17.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.395+0000 7f8237400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8230109580 0x7f82301a0d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:17.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.395+0000 7f8237400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f823019a9e0 0x7f823019ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:17.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.395+0000 7f8237400640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f8230114620 con 0x7f823019a9e0 2024-04-16T17:10:17.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.395+0000 7f8237400640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f82301144a0 con 0x7f82301044b0 2024-04-16T17:10:17.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.395+0000 7f8237400640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f82301147a0 con 0x7f8230109580 2024-04-16T17:10:17.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.395+0000 7f822fe00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8230109580 0x7f82301a0d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:17.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.395+0000 7f822fe00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8230109580 0x7f82301a0d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:33580/0 (socket says 172.21.15.45:33580) 2024-04-16T17:10:17.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.395+0000 7f822fe00640 1 -- 172.21.15.45:0/38726426 learned_addr learned my addr 172.21.15.45:0/38726426 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:10:17.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.395+0000 7f8234a00640 1 --2- 172.21.15.45:0/38726426 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f82301044b0 0x7f82301a0830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:17.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.395+0000 7f8235400640 1 --2- 172.21.15.45:0/38726426 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f823019a9e0 0x7f823019ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:17.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.395+0000 7f822fe00640 1 -- 172.21.15.45:0/38726426 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f82301044b0 msgr2=0x7f82301a0830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:17.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.395+0000 7f822fe00640 1 --2- 172.21.15.45:0/38726426 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f82301044b0 0x7f82301a0830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:17.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.395+0000 7f822fe00640 1 -- 172.21.15.45:0/38726426 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f823019a9e0 msgr2=0x7f823019ae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:17.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.395+0000 7f822fe00640 1 --2- 172.21.15.45:0/38726426 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f823019a9e0 0x7f823019ae90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:17.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.395+0000 7f822fe00640 1 -- 172.21.15.45:0/38726426 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f823019b500 con 0x7f8230109580 2024-04-16T17:10:17.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.397+0000 7f822fe00640 1 --2- 172.21.15.45:0/38726426 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8230109580 0x7f82301a0d70 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f821c00e9c0 tx=0x7f821c00ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:10:17.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.397+0000 7f822d600640 1 -- 172.21.15.45:0/38726426 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f821c00cd80 con 0x7f8230109580 2024-04-16T17:10:17.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.397+0000 7f822d600640 1 -- 172.21.15.45:0/38726426 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f821c005500 con 0x7f8230109580 2024-04-16T17:10:17.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.397+0000 7f822d600640 1 -- 172.21.15.45:0/38726426 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f821c010690 con 0x7f8230109580 2024-04-16T17:10:17.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.397+0000 7f8237400640 1 -- 172.21.15.45:0/38726426 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f823019b7f0 con 0x7f8230109580 2024-04-16T17:10:17.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.397+0000 7f8237400640 1 -- 172.21.15.45:0/38726426 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8230076740 con 0x7f8230109580 2024-04-16T17:10:17.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.398+0000 7f823a1cf740 1 Processor -- start 2024-04-16T17:10:17.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.398+0000 7f823a1cf740 1 -- start start 2024-04-16T17:10:17.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.398+0000 7f823a1cf740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556563ac8ca0 0x556563ac9080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:17.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.398+0000 7f823a1cf740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556563ac95c0 0x556563ad1c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:17.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.398+0000 7f823a1cf740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556563ad21a0 0x556563ad4590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:17.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.398+0000 7f823a1cf740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x556563a2dcb0 con 0x556563ac95c0 2024-04-16T17:10:17.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.398+0000 7f823a1cf740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5565639d4370 con 0x556563ad21a0 2024-04-16T17:10:17.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.398+0000 7f823a1cf740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x556563a29c70 con 0x556563ac8ca0 2024-04-16T17:10:17.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.398+0000 7f8234a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556563ac95c0 0x556563ad1c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:17.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.398+0000 7f8234a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556563ac95c0 0x556563ad1c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39932/0 (socket says 172.21.15.45:39932) 2024-04-16T17:10:17.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.398+0000 7f8234a00640 1 -- 172.21.15.45:0/115693225 learned_addr learned my addr 172.21.15.45:0/115693225 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:10:17.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.398+0000 7f822d600640 1 -- 172.21.15.45:0/38726426 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f821c0046c0 con 0x7f8230109580 2024-04-16T17:10:17.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.398+0000 7f8234a00640 1 -- 172.21.15.45:0/115693225 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556563ac8ca0 msgr2=0x556563ac9080 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-16T17:10:17.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.398+0000 7f822fe00640 1 --2- 172.21.15.45:0/115693225 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556563ad21a0 0x556563ad4590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:17.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.398+0000 7f8234a00640 1 --2- 172.21.15.45:0/115693225 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556563ac8ca0 0x556563ac9080 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:17.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.398+0000 7f8234a00640 1 -- 172.21.15.45:0/115693225 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556563ad21a0 msgr2=0x556563ad4590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:10:17.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.398+0000 7f8234a00640 1 --2- 172.21.15.45:0/115693225 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556563ad21a0 0x556563ad4590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:17.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.398+0000 7f8234a00640 1 -- 172.21.15.45:0/115693225 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5565639f3910 con 0x556563ac95c0 2024-04-16T17:10:17.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.402+0000 7f822fe00640 1 --2- 172.21.15.45:0/115693225 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556563ad21a0 0x556563ad4590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:10:17.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.402+0000 7f8234a00640 1 --2- 172.21.15.45:0/115693225 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556563ac95c0 0x556563ad1c60 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f822400bc90 tx=0x7f822400f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61a88ec07bc77190 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:10:17.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.402+0000 7f8221c00640 1 -- 172.21.15.45:0/115693225 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8224015070 con 0x556563ac95c0 2024-04-16T17:10:17.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.402+0000 7f8221c00640 1 -- 172.21.15.45:0/115693225 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f822401a940 con 0x556563ac95c0 2024-04-16T17:10:17.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.402+0000 7f8221c00640 1 -- 172.21.15.45:0/115693225 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82240119d0 con 0x556563ac95c0 2024-04-16T17:10:17.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.402+0000 7f823a1cf740 1 -- 172.21.15.45:0/115693225 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556563ac95c0 msgr2=0x556563ad1c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:17.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.402+0000 7f823a1cf740 1 --2- 172.21.15.45:0/115693225 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556563ac95c0 0x556563ad1c60 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f822400bc90 tx=0x7f822400f730 comp rx=0 tx=0).stop 2024-04-16T17:10:17.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.402+0000 7f823a1cf740 1 -- 172.21.15.45:0/115693225 shutdown_connections 2024-04-16T17:10:17.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.402+0000 7f823a1cf740 1 --2- 172.21.15.45:0/115693225 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556563ad21a0 0x556563ad4590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:17.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.402+0000 7f823a1cf740 1 --2- 172.21.15.45:0/115693225 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556563ac95c0 0x556563ad1c60 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:17.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.402+0000 7f823a1cf740 1 --2- 172.21.15.45:0/115693225 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556563ac8ca0 0x556563ac9080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:17.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.402+0000 7f823a1cf740 1 -- 172.21.15.45:0/115693225 >> 172.21.15.45:0/115693225 conn(0x556563ac33d0 msgr2=0x556563ac37b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:10:17.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.402+0000 7f823a1cf740 1 -- 172.21.15.45:0/115693225 shutdown_connections 2024-04-16T17:10:17.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.402+0000 7f823a1cf740 1 -- 172.21.15.45:0/115693225 wait complete. 2024-04-16T17:10:17.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.402+0000 7f823a1cf740 1 -- start start 2024-04-16T17:10:17.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.402+0000 7f822d600640 1 --2- 172.21.15.45:0/38726426 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f82180a3c40 0x7f82180a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:17.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.404+0000 7f823a1cf740 1 Processor -- start 2024-04-16T17:10:17.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.404+0000 7f822d600640 1 -- 172.21.15.45:0/38726426 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f821c0d0f60 con 0x7f8230109580 2024-04-16T17:10:17.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.404+0000 7f8235400640 1 --2- 172.21.15.45:0/38726426 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f82180a3c40 0x7f82180a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:17.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.404+0000 7f823a1cf740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556563ac8ca0 0x556563acf1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:17.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.404+0000 7f823a1cf740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556563ac95c0 0x556563acf720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:17.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.404+0000 7f823a1cf740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556563ad21a0 0x556563acfc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:17.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.404+0000 7f823a1cf740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x556563ad4f90 con 0x556563ad21a0 2024-04-16T17:10:17.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.404+0000 7f823a1cf740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5565639e7730 con 0x556563ac95c0 2024-04-16T17:10:17.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.404+0000 7f823a1cf740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x556563ad5110 con 0x556563ac8ca0 2024-04-16T17:10:17.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.404+0000 7f823a1cf740 10 client.0 _ll_register_callbacks cb 0x5565639b61e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:10:17.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.404+0000 7f822fe00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556563ac95c0 0x556563acf720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:17.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.404+0000 7f822fe00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556563ac95c0 0x556563acf720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:40260/0 (socket says 172.21.15.45:40260) 2024-04-16T17:10:17.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.404+0000 7f8235400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556563ad21a0 0x556563acfc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:17.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.404+0000 7f822fe00640 1 -- 172.21.15.45:0/2591407882 learned_addr learned my addr 172.21.15.45:0/2591407882 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:10:17.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.404+0000 7f8235400640 1 --2- 172.21.15.45:0/38726426 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f82180a3c40 0x7f82180a6100 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f8228005e30 tx=0x7f8228042040 comp rx=0 tx=0).ready entity=mgr.25411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:10:17.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.404+0000 7f822fe00640 1 -- 172.21.15.45:0/2591407882 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556563ac8ca0 msgr2=0x556563acf1e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-16T17:10:17.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.404+0000 7f822fe00640 1 --2- 172.21.15.45:0/2591407882 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556563ac8ca0 0x556563acf1e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:17.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.404+0000 7f822fe00640 1 -- 172.21.15.45:0/2591407882 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556563ad21a0 msgr2=0x556563acfc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:10:17.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.404+0000 7f822fe00640 1 --2- 172.21.15.45:0/2591407882 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556563ad21a0 0x556563acfc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:17.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.404+0000 7f822fe00640 1 -- 172.21.15.45:0/2591407882 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5565638087e0 con 0x556563ac95c0 2024-04-16T17:10:17.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.404+0000 7f8235400640 1 --2- 172.21.15.45:0/2591407882 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556563ad21a0 0x556563acfc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:10:17.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.404+0000 7f822fe00640 1 --2- 172.21.15.45:0/2591407882 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556563ac95c0 0x556563acf720 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f821c00c6a0 tx=0x7f821c09c690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7f82f119dd3bca8e server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:10:17.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.404+0000 7f8216000640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-04-16T17:10:17.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.404+0000 7f8216000640 1 -- 172.21.15.45:0/2591407882 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f821c0b6060 con 0x556563ac95c0 2024-04-16T17:10:17.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.404+0000 7f8216000640 1 -- 172.21.15.45:0/2591407882 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f821c0a5330 con 0x556563ac95c0 2024-04-16T17:10:17.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.404+0000 7f823a1cf740 1 -- 172.21.15.45:0/2591407882 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f822401a940 con 0x556563ac95c0 2024-04-16T17:10:17.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.404+0000 7f8216000640 1 -- 172.21.15.45:0/2591407882 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f821c0ce780 con 0x556563ac95c0 2024-04-16T17:10:17.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.404+0000 7f823a1cf740 10 client.26639 fetch_fsmap learned FSMap version 30 2024-04-16T17:10:17.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.404+0000 7f823a1cf740 1 -- 172.21.15.45:0/2591407882 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x556563868f70 con 0x556563ac95c0 2024-04-16T17:10:17.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.404+0000 7f8216000640 1 -- 172.21.15.45:0/2591407882 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f822401a940 con 0x556563ac95c0 2024-04-16T17:10:17.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.404+0000 7f8216000640 1 -- 172.21.15.45:0/2591407882 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f821c09bfa0 con 0x556563ac95c0 2024-04-16T17:10:17.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.404+0000 7f823a1cf740 10 client.26639 fetch_fsmap finished waiting for FSMap version 30 2024-04-16T17:10:17.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.404+0000 7f823a1cf740 10 client.26639 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:10:17.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.404+0000 7f823a1cf740 20 client.26639 populate_metadata read hostname 'smithi045' 2024-04-16T17:10:17.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.404+0000 7f823a1cf740 1 --2- 172.21.15.45:0/2591407882 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x556563ae24d0 0x556563ae4900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:17.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.404+0000 7f823a1cf740 4 client.26639 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:10:17.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.404+0000 7f823a1cf740 1 -- 172.21.15.45:0/2591407882 --> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556563a3b6d0 con 0x556563ae24d0 2024-04-16T17:10:17.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.404+0000 7f8234a00640 1 --2- 172.21.15.45:0/2591407882 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x556563ae24d0 0x556563ae4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:17.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.410+0000 7f8216000640 10 client.26639 ms_handle_connect on v2:172.21.15.104:6838/3660070592 2024-04-16T17:10:17.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.410+0000 7f8234a00640 1 --2- 172.21.15.45:0/2591407882 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x556563ae24d0 0x556563ae4900 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=45cfc2c7febbf405 server_cookie=4315a3c5f710f4a5 in_seq=0 out_seq=0 2024-04-16T17:10:17.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.410+0000 7f8216000640 1 -- 172.21.15.45:0/2591407882 <== mds.0 v2:172.21.15.104:6838/3660070592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556563a3b6d0 con 0x556563ae24d0 2024-04-16T17:10:17.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.410+0000 7f8216000640 10 client.26639 handle_command_reply: tid=0 2024-04-16T17:10:17.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.410+0000 7f823a1cf740 1 client.26639 shutdown 2024-04-16T17:10:17.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.410+0000 7f823a1cf740 1 -- 172.21.15.45:0/2591407882 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556563ac95c0 msgr2=0x556563acf720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:17.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.410+0000 7f823a1cf740 1 --2- 172.21.15.45:0/2591407882 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556563ac95c0 0x556563acf720 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f821c00c6a0 tx=0x7f821c09c690 comp rx=0 tx=0).stop 2024-04-16T17:10:17.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.410+0000 7f823a1cf740 1 -- 172.21.15.45:0/2591407882 shutdown_connections 2024-04-16T17:10:17.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.410+0000 7f823a1cf740 1 --2- 172.21.15.45:0/2591407882 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556563ad21a0 0x556563acfc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:17.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.410+0000 7f823a1cf740 1 --2- 172.21.15.45:0/2591407882 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556563ac95c0 0x556563acf720 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:17.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.410+0000 7f823a1cf740 1 --2- 172.21.15.45:0/2591407882 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x556563ae24d0 0x556563ae4900 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:17.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.410+0000 7f823a1cf740 1 --2- 172.21.15.45:0/2591407882 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556563ac8ca0 0x556563acf1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:17.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.410+0000 7f823a1cf740 1 -- 172.21.15.45:0/2591407882 >> 172.21.15.45:0/2591407882 conn(0x556563ac33d0 msgr2=0x556563ac5120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:10:17.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.410+0000 7f8216000640 0 client.26639 ms_handle_reset on v2:172.21.15.104:6838/3660070592 2024-04-16T17:10:17.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.410+0000 7f823a1cf740 1 -- 172.21.15.45:0/2591407882 shutdown_connections 2024-04-16T17:10:17.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.410+0000 7f823a1cf740 1 -- 172.21.15.45:0/2591407882 wait complete. 2024-04-16T17:10:17.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.410+0000 7f823a1cf740 20 client.26639 trim_cache size 0 max 16384 2024-04-16T17:10:17.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.431+0000 7f823a1cf740 1 Processor -- start 2024-04-16T17:10:17.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.431+0000 7f823a1cf740 1 -- start start 2024-04-16T17:10:17.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.431+0000 7f823a1cf740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556563ac8ca0 0x556563adde50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:17.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.431+0000 7f823a1cf740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556563ac95c0 0x556563ad1010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:17.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.431+0000 7f823a1cf740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556563ad21a0 0x556563aea1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:17.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.431+0000 7f823a1cf740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f82301afd90 con 0x556563ac95c0 2024-04-16T17:10:17.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.431+0000 7f823a1cf740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f82301ad5a0 con 0x556563ac8ca0 2024-04-16T17:10:17.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.431+0000 7f823a1cf740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f82301ab3d0 con 0x556563ad21a0 2024-04-16T17:10:17.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.432+0000 7f8234a00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556563ac8ca0 0x556563adde50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:17.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.432+0000 7f822fe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556563ac95c0 0x556563ad1010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:17.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.432+0000 7f8234a00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556563ac8ca0 0x556563adde50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:40268/0 (socket says 172.21.15.45:40268) 2024-04-16T17:10:17.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.432+0000 7f8234a00640 1 -- 172.21.15.45:0/2230986702 learned_addr learned my addr 172.21.15.45:0/2230986702 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:10:17.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.432+0000 7f8235400640 1 --2- 172.21.15.45:0/2230986702 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556563ad21a0 0x556563aea1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:17.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.432+0000 7f8234a00640 1 -- 172.21.15.45:0/2230986702 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556563ad21a0 msgr2=0x556563aea1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:10:17.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.432+0000 7f8234a00640 1 --2- 172.21.15.45:0/2230986702 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556563ad21a0 0x556563aea1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:17.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.432+0000 7f8234a00640 1 -- 172.21.15.45:0/2230986702 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556563ac95c0 msgr2=0x556563ad1010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:10:17.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.432+0000 7f8234a00640 1 --2- 172.21.15.45:0/2230986702 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556563ac95c0 0x556563ad1010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:17.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.432+0000 7f8234a00640 1 -- 172.21.15.45:0/2230986702 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556563700170 con 0x556563ac8ca0 2024-04-16T17:10:17.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.432+0000 7f8234a00640 1 --2- 172.21.15.45:0/2230986702 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556563ac8ca0 0x556563adde50 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f82240115e0 tx=0x7f8224011610 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6e3561792c50d144 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:10:17.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.433+0000 7f8221200640 1 -- 172.21.15.45:0/2230986702 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8224015040 con 0x556563ac8ca0 2024-04-16T17:10:17.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.433+0000 7f8221200640 1 -- 172.21.15.45:0/2230986702 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8224005520 con 0x556563ac8ca0 2024-04-16T17:10:17.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.433+0000 7f8221200640 1 -- 172.21.15.45:0/2230986702 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8224011b60 con 0x556563ac8ca0 2024-04-16T17:10:17.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.434+0000 7f823a1cf740 1 -- 172.21.15.45:0/2230986702 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556563ac8ca0 msgr2=0x556563adde50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:17.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.434+0000 7f823a1cf740 1 --2- 172.21.15.45:0/2230986702 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556563ac8ca0 0x556563adde50 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f82240115e0 tx=0x7f8224011610 comp rx=0 tx=0).stop 2024-04-16T17:10:17.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.434+0000 7f823a1cf740 1 -- 172.21.15.45:0/2230986702 shutdown_connections 2024-04-16T17:10:17.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.434+0000 7f823a1cf740 1 --2- 172.21.15.45:0/2230986702 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556563ad21a0 0x556563aea1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:17.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.434+0000 7f823a1cf740 1 --2- 172.21.15.45:0/2230986702 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556563ac95c0 0x556563ad1010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:17.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.434+0000 7f823a1cf740 1 --2- 172.21.15.45:0/2230986702 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556563ac8ca0 0x556563adde50 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:17.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.434+0000 7f823a1cf740 1 -- 172.21.15.45:0/2230986702 >> 172.21.15.45:0/2230986702 conn(0x556563ac33d0 msgr2=0x556563ad0900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:10:17.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.434+0000 7f823a1cf740 1 -- 172.21.15.45:0/2230986702 shutdown_connections 2024-04-16T17:10:17.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.434+0000 7f823a1cf740 1 -- 172.21.15.45:0/2230986702 wait complete. 2024-04-16T17:10:17.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.434+0000 7f823a1cf740 1 -- start start 2024-04-16T17:10:17.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.434+0000 7f823a1cf740 1 Processor -- start 2024-04-16T17:10:17.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.435+0000 7f823a1cf740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556563ac8ca0 0x556563ada760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:17.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.435+0000 7f823a1cf740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556563ac95c0 0x556563ad5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:17.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.435+0000 7f823a1cf740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556563ad21a0 0x556563ad6010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:17.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.435+0000 7f823a1cf740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f82301aba60 con 0x556563ad21a0 2024-04-16T17:10:17.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.435+0000 7f823a1cf740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f82301aced0 con 0x556563ac95c0 2024-04-16T17:10:17.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.435+0000 7f823a1cf740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f82301ac130 con 0x556563ac8ca0 2024-04-16T17:10:17.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.435+0000 7f823a1cf740 10 client.0 _ll_register_callbacks cb 0x7f821c098ee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:10:17.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.435+0000 7f8235400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556563ad21a0 0x556563ad6010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:17.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.435+0000 7f822fe00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556563ac95c0 0x556563ad5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:17.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.435+0000 7f8234a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556563ac8ca0 0x556563ada760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:17.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.435+0000 7f822fe00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556563ac95c0 0x556563ad5ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:40284/0 (socket says 172.21.15.45:40284) 2024-04-16T17:10:17.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.435+0000 7f822fe00640 1 -- 172.21.15.45:0/2623132731 learned_addr learned my addr 172.21.15.45:0/2623132731 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:10:17.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.436+0000 7f822fe00640 1 -- 172.21.15.45:0/2623132731 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556563ac8ca0 msgr2=0x556563ada760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:10:17.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.436+0000 7f822fe00640 1 --2- 172.21.15.45:0/2623132731 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556563ac8ca0 0x556563ada760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:17.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.436+0000 7f822fe00640 1 -- 172.21.15.45:0/2623132731 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556563ad21a0 msgr2=0x556563ad6010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:10:17.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.436+0000 7f822fe00640 1 --2- 172.21.15.45:0/2623132731 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556563ad21a0 0x556563ad6010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:17.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.436+0000 7f822fe00640 1 -- 172.21.15.45:0/2623132731 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556563ad6840 con 0x556563ac95c0 2024-04-16T17:10:17.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.436+0000 7f822fe00640 1 --2- 172.21.15.45:0/2623132731 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556563ac95c0 0x556563ad5ad0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f821c009250 tx=0x7f821c0ce830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=52c1b7bb0d05288a server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:10:17.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.436+0000 7f8216000640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-04-16T17:10:17.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.436+0000 7f8216000640 1 -- 172.21.15.45:0/2623132731 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f821c09c1b0 con 0x556563ac95c0 2024-04-16T17:10:17.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.436+0000 7f8216000640 1 -- 172.21.15.45:0/2623132731 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f821c0d42b0 con 0x556563ac95c0 2024-04-16T17:10:17.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.436+0000 7f823a1cf740 1 -- 172.21.15.45:0/2623132731 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8224005520 con 0x556563ac95c0 2024-04-16T17:10:17.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.436+0000 7f8216000640 1 -- 172.21.15.45:0/2623132731 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f821c082050 con 0x556563ac95c0 2024-04-16T17:10:17.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.436+0000 7f823a1cf740 10 client.26645 fetch_fsmap learned FSMap version 30 2024-04-16T17:10:17.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.436+0000 7f823a1cf740 1 -- 172.21.15.45:0/2623132731 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x556563b48ae0 con 0x556563ac95c0 2024-04-16T17:10:17.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.437+0000 7f8216000640 1 -- 172.21.15.45:0/2623132731 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8224005520 con 0x556563ac95c0 2024-04-16T17:10:17.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.437+0000 7f8216000640 1 -- 172.21.15.45:0/2623132731 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f821c0a5330 con 0x556563ac95c0 2024-04-16T17:10:17.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.437+0000 7f823a1cf740 10 client.26645 fetch_fsmap finished waiting for FSMap version 30 2024-04-16T17:10:17.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.437+0000 7f823a1cf740 10 client.26645 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:10:17.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.437+0000 7f823a1cf740 20 client.26645 populate_metadata read hostname 'smithi045' 2024-04-16T17:10:17.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.437+0000 7f823a1cf740 1 --2- 172.21.15.45:0/2623132731 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x556563b491a0 0x556563b4b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:17.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.437+0000 7f823a1cf740 4 client.26645 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-16T17:10:17.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.437+0000 7f823a1cf740 1 -- 172.21.15.45:0/2623132731 --> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x556563a35540 con 0x556563b491a0 2024-04-16T17:10:17.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.437+0000 7f8234a00640 1 --2- 172.21.15.45:0/2623132731 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x556563b491a0 0x556563b4b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:17.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.438+0000 7f8234a00640 1 --2- 172.21.15.45:0/2623132731 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x556563b491a0 0x556563b4b5c0 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=71bde7ab8e7376fd server_cookie=2e0ab2e3c8a3146f in_seq=0 out_seq=0 2024-04-16T17:10:17.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.438+0000 7f8216000640 10 client.26645 ms_handle_connect on v2:172.21.15.104:6838/3660070592 2024-04-16T17:10:17.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.438+0000 7f8216000640 1 -- 172.21.15.45:0/2623132731 <== mds.0 v2:172.21.15.104:6838/3660070592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x556563a35540 con 0x556563b491a0 2024-04-16T17:10:17.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.438+0000 7f8216000640 10 client.26645 handle_command_reply: tid=0 2024-04-16T17:10:17.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.439+0000 7f823a1cf740 1 client.26645 shutdown 2024-04-16T17:10:17.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.439+0000 7f823a1cf740 1 -- 172.21.15.45:0/2623132731 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556563ac95c0 msgr2=0x556563ad5ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:17.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.439+0000 7f823a1cf740 1 --2- 172.21.15.45:0/2623132731 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556563ac95c0 0x556563ad5ad0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f821c009250 tx=0x7f821c0ce830 comp rx=0 tx=0).stop 2024-04-16T17:10:17.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.439+0000 7f823a1cf740 1 -- 172.21.15.45:0/2623132731 shutdown_connections 2024-04-16T17:10:17.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.439+0000 7f823a1cf740 1 --2- 172.21.15.45:0/2623132731 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556563ad21a0 0x556563ad6010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:17.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.439+0000 7f823a1cf740 1 --2- 172.21.15.45:0/2623132731 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556563ac95c0 0x556563ad5ad0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:17.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.439+0000 7f823a1cf740 1 --2- 172.21.15.45:0/2623132731 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x556563b491a0 0x556563b4b5c0 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:17.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.439+0000 7f8216000640 0 client.26645 ms_handle_reset on v2:172.21.15.104:6838/3660070592 2024-04-16T17:10:17.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.439+0000 7f823a1cf740 1 --2- 172.21.15.45:0/2623132731 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556563ac8ca0 0x556563ada760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:17.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.439+0000 7f823a1cf740 1 -- 172.21.15.45:0/2623132731 >> 172.21.15.45:0/2623132731 conn(0x556563ac33d0 msgr2=0x556563ac2430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:10:17.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.439+0000 7f823a1cf740 1 -- 172.21.15.45:0/2623132731 shutdown_connections 2024-04-16T17:10:17.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.439+0000 7f823a1cf740 1 -- 172.21.15.45:0/2623132731 wait complete. 2024-04-16T17:10:17.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.440+0000 7f823a1cf740 20 client.26645 trim_cache size 0 max 16384 2024-04-16T17:10:17.445 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-16T17:10:17.445 INFO:teuthology.orchestra.run.smithi045.stdout: "return_code": 0, 2024-04-16T17:10:17.445 INFO:teuthology.orchestra.run.smithi045.stdout: "scrub_tag": "f3ed5dc5-fd78-4e2e-a876-9aa279219074", 2024-04-16T17:10:17.445 INFO:teuthology.orchestra.run.smithi045.stdout: "mode": "asynchronous" 2024-04-16T17:10:17.445 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-16T17:10:17.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.440+0000 7f8237400640 1 -- 172.21.15.45:0/38726426 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f82180a3c40 msgr2=0x7f82180a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:17.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.440+0000 7f8237400640 1 --2- 172.21.15.45:0/38726426 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f82180a3c40 0x7f82180a6100 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f8228005e30 tx=0x7f8228042040 comp rx=0 tx=0).stop 2024-04-16T17:10:17.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.440+0000 7f8237400640 1 -- 172.21.15.45:0/38726426 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8230109580 msgr2=0x7f82301a0d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:17.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.440+0000 7f8237400640 1 --2- 172.21.15.45:0/38726426 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8230109580 0x7f82301a0d70 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f821c00e9c0 tx=0x7f821c00ee80 comp rx=0 tx=0).stop 2024-04-16T17:10:17.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.441+0000 7f8237400640 1 -- 172.21.15.45:0/38726426 shutdown_connections 2024-04-16T17:10:17.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.441+0000 7f8237400640 1 --2- 172.21.15.45:0/38726426 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f82180a3c40 0x7f82180a6100 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:17.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.441+0000 7f8237400640 1 --2- 172.21.15.45:0/38726426 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8230109580 0x7f82301a0d70 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:17.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.441+0000 7f8237400640 1 --2- 172.21.15.45:0/38726426 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f823019a9e0 0x7f823019ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:17.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.441+0000 7f8237400640 1 --2- 172.21.15.45:0/38726426 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f82301044b0 0x7f82301a0830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:17.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.441+0000 7f8237400640 1 -- 172.21.15.45:0/38726426 >> 172.21.15.45:0/38726426 conn(0x7f82300ffd90 msgr2=0x7f8230102060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:10:17.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.441+0000 7f8237400640 1 -- 172.21.15.45:0/38726426 shutdown_connections 2024-04-16T17:10:17.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.441+0000 7f8237400640 1 -- 172.21.15.45:0/38726426 wait complete. 2024-04-16T17:10:17.466 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-16T17:10:17.466 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-16T17:10:17.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.662+0000 7f4987200640 1 -- 172.21.15.45:0/2575205476 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f49881019b0 msgr2=0x7f49881059f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:17.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.662+0000 7f4987200640 1 --2- 172.21.15.45:0/2575205476 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f49881019b0 0x7f49881059f0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f4974009900 tx=0x7f497402bc40 comp rx=0 tx=0).stop 2024-04-16T17:10:17.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.662+0000 7f4987200640 1 -- 172.21.15.45:0/2575205476 shutdown_connections 2024-04-16T17:10:17.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.662+0000 7f4987200640 1 --2- 172.21.15.45:0/2575205476 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f49881019b0 0x7f49881059f0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:17.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.662+0000 7f4987200640 1 --2- 172.21.15.45:0/2575205476 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f49881060d0 0x7f498810dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:17.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.662+0000 7f4987200640 1 --2- 172.21.15.45:0/2575205476 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4988101000 0x7f49881013e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:17.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.662+0000 7f4987200640 1 -- 172.21.15.45:0/2575205476 >> 172.21.15.45:0/2575205476 conn(0x7f49880fc880 msgr2=0x7f49880feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:10:17.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.662+0000 7f4987200640 1 -- 172.21.15.45:0/2575205476 shutdown_connections 2024-04-16T17:10:17.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.662+0000 7f4987200640 1 -- 172.21.15.45:0/2575205476 wait complete. 2024-04-16T17:10:17.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.663+0000 7f4987200640 1 Processor -- start 2024-04-16T17:10:17.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.663+0000 7f4987200640 1 -- start start 2024-04-16T17:10:17.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.664+0000 7f4987200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4988101000 0x7f49881a0690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:17.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.664+0000 7f4987200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f49881060d0 0x7f49881a0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:17.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.664+0000 7f4987200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f498819a8e0 0x7f498819ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:17.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.664+0000 7f4987200640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f49881122a0 con 0x7f498819a8e0 2024-04-16T17:10:17.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.664+0000 7f4987200640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f4988112120 con 0x7f4988101000 2024-04-16T17:10:17.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.664+0000 7f4987200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f4988112420 con 0x7f49881060d0 2024-04-16T17:10:17.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.664+0000 7f4985e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4988101000 0x7f49881a0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:17.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.664+0000 7f4985e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4988101000 0x7f49881a0690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:40310/0 (socket says 172.21.15.45:40310) 2024-04-16T17:10:17.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.664+0000 7f4985e00640 1 -- 172.21.15.45:0/262329420 learned_addr learned my addr 172.21.15.45:0/262329420 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:10:17.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.664+0000 7f4985400640 1 --2- 172.21.15.45:0/262329420 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f49881060d0 0x7f49881a0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:17.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.664+0000 7f4986800640 1 --2- 172.21.15.45:0/262329420 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f498819a8e0 0x7f498819ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:17.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.664+0000 7f4985e00640 1 -- 172.21.15.45:0/262329420 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f49881060d0 msgr2=0x7f49881a0bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:17.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.664+0000 7f4985e00640 1 --2- 172.21.15.45:0/262329420 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f49881060d0 0x7f49881a0bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:17.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.664+0000 7f4985e00640 1 -- 172.21.15.45:0/262329420 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f498819a8e0 msgr2=0x7f498819ad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:17.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.664+0000 7f4985e00640 1 --2- 172.21.15.45:0/262329420 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f498819a8e0 0x7f498819ad60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:17.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.664+0000 7f4985e00640 1 -- 172.21.15.45:0/262329420 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f498819b3a0 con 0x7f4988101000 2024-04-16T17:10:17.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.664+0000 7f4986800640 1 --2- 172.21.15.45:0/262329420 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f498819a8e0 0x7f498819ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:10:17.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.665+0000 7f4985e00640 1 --2- 172.21.15.45:0/262329420 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4988101000 0x7f49881a0690 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f497800ea60 tx=0x7f497800ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:10:17.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.665+0000 7f497ea00640 1 -- 172.21.15.45:0/262329420 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f497800cde0 con 0x7f4988101000 2024-04-16T17:10:17.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.665+0000 7f497ea00640 1 -- 172.21.15.45:0/262329420 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f49780035f0 con 0x7f4988101000 2024-04-16T17:10:17.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.665+0000 7f497ea00640 1 -- 172.21.15.45:0/262329420 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4978010670 con 0x7f4988101000 2024-04-16T17:10:17.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.666+0000 7f4987200640 1 -- 172.21.15.45:0/262329420 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f498819b690 con 0x7f4988101000 2024-04-16T17:10:17.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.666+0000 7f4987200640 1 -- 172.21.15.45:0/262329420 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49881a77f0 con 0x7f4988101000 2024-04-16T17:10:17.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.669+0000 7f497ea00640 1 -- 172.21.15.45:0/262329420 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f49780037a0 con 0x7f4988101000 2024-04-16T17:10:17.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.670+0000 7f498fc27740 1 Processor -- start 2024-04-16T17:10:17.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.671+0000 7f498fc27740 1 -- start start 2024-04-16T17:10:17.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.671+0000 7f498fc27740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e20f801390 0x55e20f801770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:17.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.671+0000 7f498fc27740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e20f801cb0 0x55e20f80a350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:17.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.671+0000 7f498fc27740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e20f80a890 0x55e20f80cc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:17.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.671+0000 7f498fc27740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55e20f767120 con 0x55e20f80a890 2024-04-16T17:10:17.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.671+0000 7f498fc27740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55e20f70d340 con 0x55e20f801cb0 2024-04-16T17:10:17.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.671+0000 7f498fc27740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55e20f7630e0 con 0x55e20f801390 2024-04-16T17:10:17.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.671+0000 7f4986800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e20f801390 0x55e20f801770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:17.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.671+0000 7f4985e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e20f801cb0 0x55e20f80a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:17.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.672+0000 7f4986800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e20f801390 0x55e20f801770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:33642/0 (socket says 172.21.15.45:33642) 2024-04-16T17:10:17.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.672+0000 7f4986800640 1 -- 172.21.15.45:0/2472619564 learned_addr learned my addr 172.21.15.45:0/2472619564 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:10:17.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.672+0000 7f497ea00640 1 --2- 172.21.15.45:0/262329420 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f496c0a3da0 0x7f496c0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:17.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.672+0000 7f4985400640 1 --2- 172.21.15.45:0/2472619564 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e20f80a890 0x55e20f80cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:17.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.672+0000 7f497ea00640 1 -- 172.21.15.45:0/262329420 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f49780d1190 con 0x7f4988101000 2024-04-16T17:10:17.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.672+0000 7f4985e00640 1 -- 172.21.15.45:0/2472619564 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e20f801390 msgr2=0x55e20f801770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:10:17.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.672+0000 7f4985e00640 1 --2- 172.21.15.45:0/2472619564 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e20f801390 0x55e20f801770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:17.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.672+0000 7f4985e00640 1 -- 172.21.15.45:0/2472619564 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e20f80a890 msgr2=0x55e20f80cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:10:17.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.672+0000 7f4985e00640 1 --2- 172.21.15.45:0/2472619564 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e20f80a890 0x55e20f80cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:17.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.672+0000 7f4985e00640 1 -- 172.21.15.45:0/2472619564 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e20f5437d0 con 0x55e20f801cb0 2024-04-16T17:10:17.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.672+0000 7f4986800640 1 --2- 172.21.15.45:0/262329420 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f496c0a3da0 0x7f496c0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:17.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.673+0000 7f4985e00640 1 --2- 172.21.15.45:0/2472619564 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e20f801cb0 0x55e20f80a350 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f49780ce530 tx=0x7f4978003f30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=21b892d0327b8603 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:10:17.679 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.673+0000 7f4973400640 1 -- 172.21.15.45:0/2472619564 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f497809d480 con 0x55e20f801cb0 2024-04-16T17:10:17.679 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.673+0000 7f4973400640 1 -- 172.21.15.45:0/2472619564 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f497809cd80 con 0x55e20f801cb0 2024-04-16T17:10:17.679 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.673+0000 7f4973400640 1 -- 172.21.15.45:0/2472619564 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49780ce7b0 con 0x55e20f801cb0 2024-04-16T17:10:17.679 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.674+0000 7f498fc27740 1 -- 172.21.15.45:0/2472619564 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e20f801cb0 msgr2=0x55e20f80a350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:17.679 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.674+0000 7f4986800640 1 --2- 172.21.15.45:0/262329420 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f496c0a3da0 0x7f496c0a6260 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f4980005c40 tx=0x7f4980004a40 comp rx=0 tx=0).ready entity=mgr.25411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:10:17.679 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.674+0000 7f498fc27740 1 --2- 172.21.15.45:0/2472619564 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e20f801cb0 0x55e20f80a350 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f49780ce530 tx=0x7f4978003f30 comp rx=0 tx=0).stop 2024-04-16T17:10:17.679 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.674+0000 7f498fc27740 1 -- 172.21.15.45:0/2472619564 shutdown_connections 2024-04-16T17:10:17.679 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.674+0000 7f498fc27740 1 --2- 172.21.15.45:0/2472619564 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e20f80a890 0x55e20f80cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:17.679 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.674+0000 7f498fc27740 1 --2- 172.21.15.45:0/2472619564 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e20f801cb0 0x55e20f80a350 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:17.679 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.674+0000 7f498fc27740 1 --2- 172.21.15.45:0/2472619564 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e20f801390 0x55e20f801770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:17.679 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.674+0000 7f498fc27740 1 -- 172.21.15.45:0/2472619564 >> 172.21.15.45:0/2472619564 conn(0x55e20f7fbac0 msgr2=0x55e20f7fbea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:10:17.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.674+0000 7f498fc27740 1 -- 172.21.15.45:0/2472619564 shutdown_connections 2024-04-16T17:10:17.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.674+0000 7f498fc27740 1 -- 172.21.15.45:0/2472619564 wait complete. 2024-04-16T17:10:17.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.675+0000 7f498fc27740 1 -- start start 2024-04-16T17:10:17.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.675+0000 7f498fc27740 1 Processor -- start 2024-04-16T17:10:17.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.676+0000 7f498fc27740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e20f801390 0x55e20f807980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:17.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.676+0000 7f498fc27740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e20f801cb0 0x55e20f807ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:17.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.676+0000 7f498fc27740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e20f80a890 0x55e20f808400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:17.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.676+0000 7f498fc27740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55e20f80d5c0 con 0x55e20f801cb0 2024-04-16T17:10:17.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.676+0000 7f498fc27740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55e20f720710 con 0x55e20f801390 2024-04-16T17:10:17.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.676+0000 7f498fc27740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55e20f80d740 con 0x55e20f80a890 2024-04-16T17:10:17.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.676+0000 7f498fc27740 10 client.0 _ll_register_callbacks cb 0x55e20f71baf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:10:17.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.676+0000 7f4985e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e20f801390 0x55e20f807980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:17.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.676+0000 7f4986800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e20f80a890 0x55e20f808400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:17.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.676+0000 7f4985e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e20f801390 0x55e20f807980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:40328/0 (socket says 172.21.15.45:40328) 2024-04-16T17:10:17.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.676+0000 7f4985e00640 1 -- 172.21.15.45:0/3744112124 learned_addr learned my addr 172.21.15.45:0/3744112124 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:10:17.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.676+0000 7f4985400640 1 --2- 172.21.15.45:0/3744112124 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e20f801cb0 0x55e20f807ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:17.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.677+0000 7f4985e00640 1 -- 172.21.15.45:0/3744112124 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e20f80a890 msgr2=0x55e20f808400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:10:17.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.677+0000 7f4985e00640 1 --2- 172.21.15.45:0/3744112124 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e20f80a890 0x55e20f808400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:17.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.677+0000 7f4985e00640 1 -- 172.21.15.45:0/3744112124 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e20f801cb0 msgr2=0x55e20f807ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:10:17.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.677+0000 7f4985e00640 1 --2- 172.21.15.45:0/3744112124 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e20f801cb0 0x55e20f807ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:17.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.677+0000 7f4985e00640 1 -- 172.21.15.45:0/3744112124 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e20f4793f0 con 0x55e20f801390 2024-04-16T17:10:17.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.677+0000 7f4985e00640 1 --2- 172.21.15.45:0/3744112124 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e20f801390 0x55e20f807980 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f497809cd10 tx=0x7f49780d4710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1980dbaecfaa6966 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:10:17.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.677+0000 7f4967e00640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-04-16T17:10:17.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.677+0000 7f4967e00640 1 -- 172.21.15.45:0/3744112124 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f497809d480 con 0x55e20f801390 2024-04-16T17:10:17.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.677+0000 7f4967e00640 1 -- 172.21.15.45:0/3744112124 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f49780bd0f0 con 0x55e20f801390 2024-04-16T17:10:17.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.677+0000 7f498fc27740 1 -- 172.21.15.45:0/3744112124 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f497809cd80 con 0x55e20f801390 2024-04-16T17:10:17.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.678+0000 7f4967e00640 1 -- 172.21.15.45:0/3744112124 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f49780b6210 con 0x55e20f801390 2024-04-16T17:10:17.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.678+0000 7f498fc27740 10 client.26657 fetch_fsmap learned FSMap version 30 2024-04-16T17:10:17.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.678+0000 7f498fc27740 1 -- 172.21.15.45:0/3744112124 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55e20f4390f0 con 0x55e20f801390 2024-04-16T17:10:17.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.678+0000 7f4967e00640 1 -- 172.21.15.45:0/3744112124 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f497809cd80 con 0x55e20f801390 2024-04-16T17:10:17.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.678+0000 7f4967e00640 1 -- 172.21.15.45:0/3744112124 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f4978020070 con 0x55e20f801390 2024-04-16T17:10:17.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.678+0000 7f498fc27740 10 client.26657 fetch_fsmap finished waiting for FSMap version 30 2024-04-16T17:10:17.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.678+0000 7f498fc27740 10 client.26657 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:10:17.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.678+0000 7f498fc27740 20 client.26657 populate_metadata read hostname 'smithi045' 2024-04-16T17:10:17.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.679+0000 7f498fc27740 1 --2- 172.21.15.45:0/3744112124 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55e20f81aaa0 0x55e20f81ce90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:17.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.679+0000 7f498fc27740 4 client.26657 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:10:17.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.679+0000 7f498fc27740 1 -- 172.21.15.45:0/3744112124 --> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e20f774b40 con 0x55e20f81aaa0 2024-04-16T17:10:17.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.679+0000 7f4985400640 1 --2- 172.21.15.45:0/3744112124 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55e20f81aaa0 0x55e20f81ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:17.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.679+0000 7f4985400640 1 --2- 172.21.15.45:0/3744112124 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55e20f81aaa0 0x55e20f81ce90 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=21f4426a4e9816c4 server_cookie=9bc097bc1353bd4c in_seq=0 out_seq=0 2024-04-16T17:10:17.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.679+0000 7f4967e00640 10 client.26657 ms_handle_connect on v2:172.21.15.104:6838/3660070592 2024-04-16T17:10:17.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.681+0000 7f4967e00640 1 -- 172.21.15.45:0/3744112124 <== mds.0 v2:172.21.15.104:6838/3660070592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e20f774b40 con 0x55e20f81aaa0 2024-04-16T17:10:17.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.681+0000 7f4967e00640 10 client.26657 handle_command_reply: tid=0 2024-04-16T17:10:17.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.681+0000 7f498fc27740 1 client.26657 shutdown 2024-04-16T17:10:17.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.681+0000 7f498fc27740 1 -- 172.21.15.45:0/3744112124 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e20f801390 msgr2=0x55e20f807980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:17.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.681+0000 7f498fc27740 1 --2- 172.21.15.45:0/3744112124 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e20f801390 0x55e20f807980 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f497809cd10 tx=0x7f49780d4710 comp rx=0 tx=0).stop 2024-04-16T17:10:17.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.681+0000 7f498fc27740 1 -- 172.21.15.45:0/3744112124 shutdown_connections 2024-04-16T17:10:17.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.681+0000 7f498fc27740 1 --2- 172.21.15.45:0/3744112124 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55e20f81aaa0 0x55e20f81ce90 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:17.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.681+0000 7f498fc27740 1 --2- 172.21.15.45:0/3744112124 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e20f80a890 0x55e20f808400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:17.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.681+0000 7f498fc27740 1 --2- 172.21.15.45:0/3744112124 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e20f801cb0 0x55e20f807ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:17.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.681+0000 7f498fc27740 1 --2- 172.21.15.45:0/3744112124 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e20f801390 0x55e20f807980 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:17.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.681+0000 7f4967e00640 0 client.26657 ms_handle_reset on v2:172.21.15.104:6838/3660070592 2024-04-16T17:10:17.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.681+0000 7f498fc27740 1 -- 172.21.15.45:0/3744112124 >> 172.21.15.45:0/3744112124 conn(0x55e20f7fbac0 msgr2=0x55e20f7fd810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:10:17.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.681+0000 7f498fc27740 1 -- 172.21.15.45:0/3744112124 shutdown_connections 2024-04-16T17:10:17.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.681+0000 7f498fc27740 1 -- 172.21.15.45:0/3744112124 wait complete. 2024-04-16T17:10:17.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.681+0000 7f498fc27740 20 client.26657 trim_cache size 0 max 16384 2024-04-16T17:10:17.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.703+0000 7f498fc27740 1 Processor -- start 2024-04-16T17:10:17.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.703+0000 7f498fc27740 1 -- start start 2024-04-16T17:10:17.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.703+0000 7f498fc27740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e20f801390 0x55e20f80a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:17.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.703+0000 7f498fc27740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e20f801cb0 0x55e20f816480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:17.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.703+0000 7f498fc27740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e20f80a890 0x55e20f808050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:17.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.703+0000 7f498fc27740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f49881b3ed0 con 0x55e20f801390 2024-04-16T17:10:17.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.703+0000 7f498fc27740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f49881b16e0 con 0x55e20f801cb0 2024-04-16T17:10:17.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.703+0000 7f498fc27740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f49881af510 con 0x55e20f80a890 2024-04-16T17:10:17.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.704+0000 7f4985e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e20f801390 0x55e20f80a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:17.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.704+0000 7f4986800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e20f80a890 0x55e20f808050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:17.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.704+0000 7f4985e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e20f801390 0x55e20f80a0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:40006/0 (socket says 172.21.15.45:40006) 2024-04-16T17:10:17.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.704+0000 7f4986800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e20f80a890 0x55e20f808050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:33658/0 (socket says 172.21.15.45:33658) 2024-04-16T17:10:17.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.704+0000 7f4985e00640 1 -- 172.21.15.45:0/3794450953 learned_addr learned my addr 172.21.15.45:0/3794450953 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:10:17.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.704+0000 7f4985400640 1 --2- 172.21.15.45:0/3794450953 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e20f801cb0 0x55e20f816480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:17.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.704+0000 7f4986800640 1 -- 172.21.15.45:0/3794450953 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e20f801cb0 msgr2=0x55e20f816480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:10:17.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.704+0000 7f4986800640 1 --2- 172.21.15.45:0/3794450953 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e20f801cb0 0x55e20f816480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:17.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.704+0000 7f4986800640 1 -- 172.21.15.45:0/3794450953 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e20f801390 msgr2=0x55e20f80a0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:10:17.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.704+0000 7f4986800640 1 --2- 172.21.15.45:0/3794450953 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e20f801390 0x55e20f80a0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:17.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.704+0000 7f4986800640 1 -- 172.21.15.45:0/3794450953 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e20f7fb5f0 con 0x55e20f80a890 2024-04-16T17:10:17.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.704+0000 7f4985e00640 1 --2- 172.21.15.45:0/3794450953 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e20f801390 0x55e20f80a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:10:17.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.705+0000 7f4986800640 1 --2- 172.21.15.45:0/3794450953 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e20f80a890 0x55e20f808050 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f4980005bf0 tx=0x7f4980016000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ebec758b3283df50 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:10:17.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.705+0000 7f4972a00640 1 -- 172.21.15.45:0/3794450953 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4980011440 con 0x55e20f80a890 2024-04-16T17:10:17.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.705+0000 7f4972a00640 1 -- 172.21.15.45:0/3794450953 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4980011ac0 con 0x55e20f80a890 2024-04-16T17:10:17.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.705+0000 7f4972a00640 1 -- 172.21.15.45:0/3794450953 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f498001fbe0 con 0x55e20f80a890 2024-04-16T17:10:17.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.706+0000 7f498fc27740 1 -- 172.21.15.45:0/3794450953 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e20f80a890 msgr2=0x55e20f808050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:17.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.706+0000 7f498fc27740 1 --2- 172.21.15.45:0/3794450953 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e20f80a890 0x55e20f808050 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f4980005bf0 tx=0x7f4980016000 comp rx=0 tx=0).stop 2024-04-16T17:10:17.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.706+0000 7f498fc27740 1 -- 172.21.15.45:0/3794450953 shutdown_connections 2024-04-16T17:10:17.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.706+0000 7f498fc27740 1 --2- 172.21.15.45:0/3794450953 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e20f80a890 0x55e20f808050 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:17.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.706+0000 7f498fc27740 1 --2- 172.21.15.45:0/3794450953 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e20f801cb0 0x55e20f816480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:17.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.706+0000 7f498fc27740 1 --2- 172.21.15.45:0/3794450953 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e20f801390 0x55e20f80a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:17.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.706+0000 7f498fc27740 1 -- 172.21.15.45:0/3794450953 >> 172.21.15.45:0/3794450953 conn(0x55e20f7fbac0 msgr2=0x55e20f807940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:10:17.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.706+0000 7f498fc27740 1 -- 172.21.15.45:0/3794450953 shutdown_connections 2024-04-16T17:10:17.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.706+0000 7f498fc27740 1 -- 172.21.15.45:0/3794450953 wait complete. 2024-04-16T17:10:17.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.706+0000 7f498fc27740 1 -- start start 2024-04-16T17:10:17.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.707+0000 7f498fc27740 1 Processor -- start 2024-04-16T17:10:17.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.707+0000 7f498fc27740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e20f801390 0x55e20f812d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:17.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.707+0000 7f498fc27740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e20f801cb0 0x55e20f804380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:17.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.707+0000 7f498fc27740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e20f80a890 0x55e20f8048c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:17.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.707+0000 7f498fc27740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f49881afba0 con 0x55e20f801cb0 2024-04-16T17:10:17.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.707+0000 7f498fc27740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f49881b1010 con 0x55e20f801390 2024-04-16T17:10:17.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.707+0000 7f498fc27740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f49881b0270 con 0x55e20f80a890 2024-04-16T17:10:17.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.707+0000 7f498fc27740 10 client.0 _ll_register_callbacks cb 0x7f49800093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:10:17.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.708+0000 7f4985e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e20f801390 0x55e20f812d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:17.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.708+0000 7f4985e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e20f801390 0x55e20f812d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:40348/0 (socket says 172.21.15.45:40348) 2024-04-16T17:10:17.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.708+0000 7f4985e00640 1 -- 172.21.15.45:0/2826821577 learned_addr learned my addr 172.21.15.45:0/2826821577 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:10:17.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.708+0000 7f4985400640 1 --2- 172.21.15.45:0/2826821577 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e20f801cb0 0x55e20f804380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:17.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.708+0000 7f4986800640 1 --2- 172.21.15.45:0/2826821577 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e20f80a890 0x55e20f8048c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:17.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.708+0000 7f4985e00640 1 -- 172.21.15.45:0/2826821577 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e20f80a890 msgr2=0x55e20f8048c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:10:17.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.708+0000 7f4985e00640 1 --2- 172.21.15.45:0/2826821577 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e20f80a890 0x55e20f8048c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:17.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.708+0000 7f4985e00640 1 -- 172.21.15.45:0/2826821577 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e20f801cb0 msgr2=0x55e20f804380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:10:17.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.708+0000 7f4985e00640 1 --2- 172.21.15.45:0/2826821577 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e20f801cb0 0x55e20f804380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:17.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.708+0000 7f4985e00640 1 -- 172.21.15.45:0/2826821577 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e20f8050f0 con 0x55e20f801390 2024-04-16T17:10:17.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.708+0000 7f4985e00640 1 --2- 172.21.15.45:0/2826821577 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e20f801390 0x55e20f812d30 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f497809ca80 tx=0x7f497809c3e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a2f83f0f24099bb1 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:10:17.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.708+0000 7f4967e00640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-04-16T17:10:17.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.708+0000 7f4967e00640 1 -- 172.21.15.45:0/2826821577 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f497809c6e0 con 0x55e20f801390 2024-04-16T17:10:17.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.709+0000 7f498fc27740 1 -- 172.21.15.45:0/2826821577 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4980011ac0 con 0x55e20f801390 2024-04-16T17:10:17.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.709+0000 7f4967e00640 1 -- 172.21.15.45:0/2826821577 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f49780bd300 con 0x55e20f801390 2024-04-16T17:10:17.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.709+0000 7f4967e00640 1 -- 172.21.15.45:0/2826821577 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f497809d480 con 0x55e20f801390 2024-04-16T17:10:17.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.709+0000 7f498fc27740 10 client.26663 fetch_fsmap learned FSMap version 30 2024-04-16T17:10:17.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.709+0000 7f498fc27740 1 -- 172.21.15.45:0/2826821577 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55e20f880d10 con 0x55e20f801390 2024-04-16T17:10:17.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.709+0000 7f4967e00640 1 -- 172.21.15.45:0/2826821577 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4980011ac0 con 0x55e20f801390 2024-04-16T17:10:17.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.710+0000 7f4967e00640 1 -- 172.21.15.45:0/2826821577 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f4978020070 con 0x55e20f801390 2024-04-16T17:10:17.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.710+0000 7f498fc27740 10 client.26663 fetch_fsmap finished waiting for FSMap version 30 2024-04-16T17:10:17.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.710+0000 7f498fc27740 10 client.26663 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:10:17.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.710+0000 7f498fc27740 20 client.26663 populate_metadata read hostname 'smithi045' 2024-04-16T17:10:17.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.710+0000 7f498fc27740 1 --2- 172.21.15.45:0/2826821577 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55e20f881240 0x55e20f883630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:17.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.710+0000 7f498fc27740 4 client.26663 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-16T17:10:17.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.710+0000 7f498fc27740 1 -- 172.21.15.45:0/2826821577 --> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55e20f76e9b0 con 0x55e20f881240 2024-04-16T17:10:17.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.710+0000 7f4985400640 1 --2- 172.21.15.45:0/2826821577 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55e20f881240 0x55e20f883630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:17.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.710+0000 7f4985400640 1 --2- 172.21.15.45:0/2826821577 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55e20f881240 0x55e20f883630 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=515441991fed9b3f server_cookie=dc259335dd34a1ea in_seq=0 out_seq=0 2024-04-16T17:10:17.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.710+0000 7f4967e00640 10 client.26663 ms_handle_connect on v2:172.21.15.104:6838/3660070592 2024-04-16T17:10:17.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.711+0000 7f4967e00640 1 -- 172.21.15.45:0/2826821577 <== mds.0 v2:172.21.15.104:6838/3660070592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55e20f76e9b0 con 0x55e20f881240 2024-04-16T17:10:17.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.711+0000 7f4967e00640 10 client.26663 handle_command_reply: tid=0 2024-04-16T17:10:17.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.711+0000 7f498fc27740 1 client.26663 shutdown 2024-04-16T17:10:17.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.711+0000 7f498fc27740 1 -- 172.21.15.45:0/2826821577 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e20f801390 msgr2=0x55e20f812d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:17.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.711+0000 7f498fc27740 1 --2- 172.21.15.45:0/2826821577 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e20f801390 0x55e20f812d30 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f497809ca80 tx=0x7f497809c3e0 comp rx=0 tx=0).stop 2024-04-16T17:10:17.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.711+0000 7f498fc27740 1 -- 172.21.15.45:0/2826821577 shutdown_connections 2024-04-16T17:10:17.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.711+0000 7f498fc27740 1 --2- 172.21.15.45:0/2826821577 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55e20f881240 0x55e20f883630 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:17.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.711+0000 7f498fc27740 1 --2- 172.21.15.45:0/2826821577 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e20f80a890 0x55e20f8048c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:17.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.711+0000 7f498fc27740 1 --2- 172.21.15.45:0/2826821577 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e20f801cb0 0x55e20f804380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:17.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.711+0000 7f498fc27740 1 --2- 172.21.15.45:0/2826821577 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e20f801390 0x55e20f812d30 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:17.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.711+0000 7f498fc27740 1 -- 172.21.15.45:0/2826821577 >> 172.21.15.45:0/2826821577 conn(0x55e20f7fbac0 msgr2=0x55e20f7fab20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:10:17.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.711+0000 7f4967e00640 0 client.26663 ms_handle_reset on v2:172.21.15.104:6838/3660070592 2024-04-16T17:10:17.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.711+0000 7f498fc27740 1 -- 172.21.15.45:0/2826821577 shutdown_connections 2024-04-16T17:10:17.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.711+0000 7f498fc27740 1 -- 172.21.15.45:0/2826821577 wait complete. 2024-04-16T17:10:17.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.711+0000 7f498fc27740 20 client.26663 trim_cache size 0 max 16384 2024-04-16T17:10:17.718 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-16T17:10:17.718 INFO:teuthology.orchestra.run.smithi045.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-16T17:10:17.718 INFO:teuthology.orchestra.run.smithi045.stdout: "scrubs": { 2024-04-16T17:10:17.719 INFO:teuthology.orchestra.run.smithi045.stdout: "f3ed5dc5-fd78-4e2e-a876-9aa279219074": { 2024-04-16T17:10:17.719 INFO:teuthology.orchestra.run.smithi045.stdout: "path": "/", 2024-04-16T17:10:17.719 INFO:teuthology.orchestra.run.smithi045.stdout: "tag": "f3ed5dc5-fd78-4e2e-a876-9aa279219074", 2024-04-16T17:10:17.719 INFO:teuthology.orchestra.run.smithi045.stdout: "options": "recursive,force" 2024-04-16T17:10:17.719 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:10:17.719 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:10:17.719 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-16T17:10:17.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.712+0000 7f4987200640 1 -- 172.21.15.45:0/262329420 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f496c0a3da0 msgr2=0x7f496c0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:17.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.712+0000 7f4987200640 1 --2- 172.21.15.45:0/262329420 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f496c0a3da0 0x7f496c0a6260 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f4980005c40 tx=0x7f4980004a40 comp rx=0 tx=0).stop 2024-04-16T17:10:17.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.712+0000 7f4987200640 1 -- 172.21.15.45:0/262329420 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4988101000 msgr2=0x7f49881a0690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:17.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.712+0000 7f4987200640 1 --2- 172.21.15.45:0/262329420 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4988101000 0x7f49881a0690 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f497800ea60 tx=0x7f497800ef20 comp rx=0 tx=0).stop 2024-04-16T17:10:17.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.712+0000 7f4987200640 1 -- 172.21.15.45:0/262329420 shutdown_connections 2024-04-16T17:10:17.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.712+0000 7f4987200640 1 --2- 172.21.15.45:0/262329420 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f496c0a3da0 0x7f496c0a6260 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:17.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.712+0000 7f4987200640 1 --2- 172.21.15.45:0/262329420 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f49881060d0 0x7f49881a0bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:17.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.712+0000 7f4987200640 1 --2- 172.21.15.45:0/262329420 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f498819a8e0 0x7f498819ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:17.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.712+0000 7f4987200640 1 --2- 172.21.15.45:0/262329420 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4988101000 0x7f49881a0690 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:17.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.712+0000 7f4987200640 1 -- 172.21.15.45:0/262329420 >> 172.21.15.45:0/262329420 conn(0x7f49880fc880 msgr2=0x7f49880fdb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:10:17.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.713+0000 7f4987200640 1 -- 172.21.15.45:0/262329420 shutdown_connections 2024-04-16T17:10:17.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:17.713+0000 7f4987200640 1 -- 172.21.15.45:0/262329420 wait complete. 2024-04-16T17:10:17.735 INFO:tasks.cephfs.filesystem:scrub status for tag:f3ed5dc5-fd78-4e2e-a876-9aa279219074 - {'path': '/', 'tag': 'f3ed5dc5-fd78-4e2e-a876-9aa279219074', 'options': 'recursive,force'} 2024-04-16T17:10:18.165 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:10:14 smithi045 ceph-mon[31676]: pgmap v107: 129 pgs: 129 active+clean; 31 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 112 KiB/s wr, 40 op/s 2024-04-16T17:10:18.165 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:10:15 smithi045 ceph-mon[31676]: from='client.? 172.21.15.45:0/711476606' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:10:18.165 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:10:16 smithi045 ceph-mon[31676]: pgmap v108: 129 pgs: 129 active+clean; 31 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 112 KiB/s wr, 40 op/s 2024-04-16T17:10:18.231 INFO:tasks.workunit.client.0.smithi045.stdout: 1 46527 1.02 MB/sec execute 20 sec latency 60.020 ms 2024-04-16T17:10:18.646 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:10:18 smithi045 ceph-mon[31676]: pgmap v109: 129 pgs: 129 active+clean; 32 MiB data, 468 MiB used, 1.0 TiB / 1.0 TiB avail; 631 KiB/s rd, 294 KiB/s wr, 85 op/s 2024-04-16T17:10:18.647 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:10:18 smithi045 ceph-mon[31676]: scrub queued for path: / 2024-04-16T17:10:18.647 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:10:18 smithi045 ceph-mon[31676]: scrub summary: idle+waiting paths [/] 2024-04-16T17:10:18.647 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:10:18 smithi045 ceph-mon[31676]: scrub summary: active paths [/] 2024-04-16T17:10:18.647 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:10:18 smithi045 ceph-mon[31676]: scrub summary: active paths [/] 2024-04-16T17:10:18.647 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:10:18 smithi045 ceph-mon[31676]: scrub summary: active paths [/] 2024-04-16T17:10:18.647 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:10:18 smithi045 ceph-mon[31676]: scrub summary: idle+waiting paths [/] 2024-04-16T17:10:18.647 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:10:18 smithi045 ceph-mon[31676]: scrub summary: active paths [/] 2024-04-16T17:10:18.647 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:10:18 smithi045 ceph-mon[31676]: scrub summary: idle+waiting paths [/] 2024-04-16T17:10:18.647 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:10:18 smithi045 ceph-mon[31676]: scrub summary: idle+waiting paths [/] 2024-04-16T17:10:18.648 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:10:18 smithi045 ceph-mon[31676]: scrub summary: idle+waiting paths [/] 2024-04-16T17:10:18.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:10:18 smithi117 ceph-mon[36730]: pgmap v109: 129 pgs: 129 active+clean; 32 MiB data, 468 MiB used, 1.0 TiB / 1.0 TiB avail; 631 KiB/s rd, 294 KiB/s wr, 85 op/s 2024-04-16T17:10:18.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:10:18 smithi117 ceph-mon[36730]: scrub queued for path: / 2024-04-16T17:10:18.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:10:18 smithi117 ceph-mon[36730]: scrub summary: idle+waiting paths [/] 2024-04-16T17:10:18.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:10:18 smithi117 ceph-mon[36730]: scrub summary: active paths [/] 2024-04-16T17:10:18.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:10:18 smithi117 ceph-mon[36730]: scrub summary: active paths [/] 2024-04-16T17:10:18.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:10:18 smithi117 ceph-mon[36730]: scrub summary: active paths [/] 2024-04-16T17:10:18.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:10:18 smithi117 ceph-mon[36730]: scrub summary: idle+waiting paths [/] 2024-04-16T17:10:18.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:10:18 smithi117 ceph-mon[36730]: scrub summary: active paths [/] 2024-04-16T17:10:18.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:10:18 smithi117 ceph-mon[36730]: scrub summary: idle+waiting paths [/] 2024-04-16T17:10:18.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:10:18 smithi117 ceph-mon[36730]: scrub summary: idle+waiting paths [/] 2024-04-16T17:10:18.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:10:18 smithi117 ceph-mon[36730]: scrub summary: idle+waiting paths [/] 2024-04-16T17:10:18.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:10:18 smithi104 ceph-mon[36457]: pgmap v109: 129 pgs: 129 active+clean; 32 MiB data, 468 MiB used, 1.0 TiB / 1.0 TiB avail; 631 KiB/s rd, 294 KiB/s wr, 85 op/s 2024-04-16T17:10:18.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:10:18 smithi104 ceph-mon[36457]: scrub queued for path: / 2024-04-16T17:10:18.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:10:18 smithi104 ceph-mon[36457]: scrub summary: idle+waiting paths [/] 2024-04-16T17:10:18.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:10:18 smithi104 ceph-mon[36457]: scrub summary: active paths [/] 2024-04-16T17:10:18.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:10:18 smithi104 ceph-mon[36457]: scrub summary: active paths [/] 2024-04-16T17:10:18.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:10:18 smithi104 ceph-mon[36457]: scrub summary: active paths [/] 2024-04-16T17:10:18.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:10:18 smithi104 ceph-mon[36457]: scrub summary: idle+waiting paths [/] 2024-04-16T17:10:18.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:10:18 smithi104 ceph-mon[36457]: scrub summary: active paths [/] 2024-04-16T17:10:18.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:10:18 smithi104 ceph-mon[36457]: scrub summary: idle+waiting paths [/] 2024-04-16T17:10:18.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:10:18 smithi104 ceph-mon[36457]: scrub summary: idle+waiting paths [/] 2024-04-16T17:10:18.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:10:18 smithi104 ceph-mon[36457]: scrub summary: idle+waiting paths [/] 2024-04-16T17:10:19.230 INFO:tasks.workunit.client.0.smithi045.stdout: 1 46769 0.97 MB/sec execute 21 sec latency 48.677 ms 2024-04-16T17:10:20.231 INFO:tasks.workunit.client.0.smithi045.stdout: 1 46814 0.93 MB/sec execute 22 sec latency 923.855 ms 2024-04-16T17:10:20.720 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:10:20 smithi045 ceph-mon[31676]: pgmap v110: 129 pgs: 129 active+clean; 32 MiB data, 468 MiB used, 1.0 TiB / 1.0 TiB avail; 576 KiB/s rd, 207 KiB/s wr, 55 op/s 2024-04-16T17:10:20.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:10:20 smithi117 ceph-mon[36730]: pgmap v110: 129 pgs: 129 active+clean; 32 MiB data, 468 MiB used, 1.0 TiB / 1.0 TiB avail; 576 KiB/s rd, 207 KiB/s wr, 55 op/s 2024-04-16T17:10:20.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:10:20 smithi104 ceph-mon[36457]: pgmap v110: 129 pgs: 129 active+clean; 32 MiB data, 468 MiB used, 1.0 TiB / 1.0 TiB avail; 576 KiB/s rd, 207 KiB/s wr, 55 op/s 2024-04-16T17:10:21.231 INFO:tasks.workunit.client.0.smithi045.stdout: 1 46814 0.89 MB/sec execute 23 sec latency 1923.968 ms 2024-04-16T17:10:22.231 INFO:tasks.workunit.client.0.smithi045.stdout: 1 46814 0.86 MB/sec execute 24 sec latency 2924.069 ms 2024-04-16T17:10:22.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:10:22 smithi045 ceph-mon[31676]: pgmap v111: 129 pgs: 129 active+clean; 34 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 773 KiB/s rd, 384 KiB/s wr, 87 op/s 2024-04-16T17:10:22.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:10:22 smithi117 ceph-mon[36730]: pgmap v111: 129 pgs: 129 active+clean; 34 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 773 KiB/s rd, 384 KiB/s wr, 87 op/s 2024-04-16T17:10:22.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:10:22 smithi104 ceph-mon[36457]: pgmap v111: 129 pgs: 129 active+clean; 34 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 773 KiB/s rd, 384 KiB/s wr, 87 op/s 2024-04-16T17:10:23.231 INFO:tasks.workunit.client.0.smithi045.stdout: 1 46870 0.83 MB/sec execute 25 sec latency 2966.993 ms 2024-04-16T17:10:24.231 INFO:tasks.workunit.client.0.smithi045.stdout: 1 46870 0.80 MB/sec execute 26 sec latency 1855.520 ms 2024-04-16T17:10:24.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:10:24 smithi045 ceph-mon[31676]: pgmap v112: 129 pgs: 129 active+clean; 34 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 757 KiB/s rd, 360 KiB/s wr, 76 op/s 2024-04-16T17:10:24.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:10:24 smithi045 ceph-mon[31676]: scrub summary: idle 2024-04-16T17:10:24.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:10:24 smithi045 ceph-mon[31676]: scrub summary: idle 2024-04-16T17:10:24.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:10:24 smithi045 ceph-mon[31676]: scrub summary: idle 2024-04-16T17:10:24.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:10:24 smithi117 ceph-mon[36730]: pgmap v112: 129 pgs: 129 active+clean; 34 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 757 KiB/s rd, 360 KiB/s wr, 76 op/s 2024-04-16T17:10:24.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:10:24 smithi117 ceph-mon[36730]: scrub summary: idle 2024-04-16T17:10:24.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:10:24 smithi117 ceph-mon[36730]: scrub summary: idle 2024-04-16T17:10:24.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:10:24 smithi117 ceph-mon[36730]: scrub summary: idle 2024-04-16T17:10:24.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:10:24 smithi104 ceph-mon[36457]: pgmap v112: 129 pgs: 129 active+clean; 34 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 757 KiB/s rd, 360 KiB/s wr, 76 op/s 2024-04-16T17:10:24.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:10:24 smithi104 ceph-mon[36457]: scrub summary: idle 2024-04-16T17:10:24.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:10:24 smithi104 ceph-mon[36457]: scrub summary: idle 2024-04-16T17:10:24.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:10:24 smithi104 ceph-mon[36457]: scrub summary: idle 2024-04-16T17:10:25.238 INFO:tasks.workunit.client.0.smithi045.stdout: 1 46870 0.77 MB/sec execute 27 sec latency 2862.336 ms 2024-04-16T17:10:26.238 INFO:tasks.workunit.client.0.smithi045.stdout: 1 46938 0.74 MB/sec execute 28 sec latency 3119.614 ms 2024-04-16T17:10:26.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:10:26 smithi045 ceph-mon[31676]: pgmap v113: 129 pgs: 129 active+clean; 34 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 757 KiB/s rd, 359 KiB/s wr, 76 op/s 2024-04-16T17:10:26.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:10:26 smithi117 ceph-mon[36730]: pgmap v113: 129 pgs: 129 active+clean; 34 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 757 KiB/s rd, 359 KiB/s wr, 76 op/s 2024-04-16T17:10:26.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:10:26 smithi104 ceph-mon[36457]: pgmap v113: 129 pgs: 129 active+clean; 34 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 757 KiB/s rd, 359 KiB/s wr, 76 op/s 2024-04-16T17:10:27.238 INFO:tasks.workunit.client.0.smithi045.stdout: 1 46938 0.72 MB/sec execute 29 sec latency 1644.400 ms 2024-04-16T17:10:28.238 INFO:tasks.workunit.client.0.smithi045.stdout: 1 47443 0.76 MB/sec execute 30 sec latency 1685.760 ms 2024-04-16T17:10:28.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:10:28 smithi117 ceph-mon[36730]: pgmap v114: 129 pgs: 129 active+clean; 37 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 787 KiB/s rd, 729 KiB/s wr, 95 op/s 2024-04-16T17:10:28.875 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:28.874+0000 7f1864e00640 1 -- 172.21.15.45:0/3917711865 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f18601042a0 msgr2=0x7f1860104680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:28.875 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:28.874+0000 7f1864e00640 1 --2- 172.21.15.45:0/3917711865 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f18601042a0 0x7f1860104680 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f1850009900 tx=0x7f185002bc40 comp rx=0 tx=0).stop 2024-04-16T17:10:28.875 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:28.875+0000 7f1864e00640 1 -- 172.21.15.45:0/3917711865 shutdown_connections 2024-04-16T17:10:28.876 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:28.875+0000 7f1864e00640 1 --2- 172.21.15.45:0/3917711865 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1860109370 0x7f1860110f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:28.876 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:28.875+0000 7f1864e00640 1 --2- 172.21.15.45:0/3917711865 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1860104c50 0x7f1860108c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:28.876 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:28.875+0000 7f1864e00640 1 --2- 172.21.15.45:0/3917711865 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f18601042a0 0x7f1860104680 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:28.876 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:28.875+0000 7f1864e00640 1 -- 172.21.15.45:0/3917711865 >> 172.21.15.45:0/3917711865 conn(0x7f18600ffb00 msgr2=0x7f1860101f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:10:28.876 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:28.875+0000 7f1864e00640 1 -- 172.21.15.45:0/3917711865 shutdown_connections 2024-04-16T17:10:28.876 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:28.875+0000 7f1864e00640 1 -- 172.21.15.45:0/3917711865 wait complete. 2024-04-16T17:10:28.877 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:28.876+0000 7f1864e00640 1 Processor -- start 2024-04-16T17:10:28.877 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:28.877+0000 7f1864e00640 1 -- start start 2024-04-16T17:10:28.878 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:28.877+0000 7f1864e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f18601042a0 0x7f18601a0620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:28.878 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:28.877+0000 7f1864e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1860109370 0x7f18601a0b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:28.878 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:28.877+0000 7f1864e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f186019a7e0 0x7f186019ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:28.878 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:28.877+0000 7f1864e00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f1860114400 con 0x7f1860109370 2024-04-16T17:10:28.879 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:28.877+0000 7f1864e00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f1860114280 con 0x7f186019a7e0 2024-04-16T17:10:28.879 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:28.877+0000 7f1864e00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f1860114580 con 0x7f18601042a0 2024-04-16T17:10:28.879 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:28.877+0000 7f185f400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f18601042a0 0x7f18601a0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:28.879 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:28.878+0000 7f185f400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f18601042a0 0x7f18601a0620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:41734/0 (socket says 172.21.15.45:41734) 2024-04-16T17:10:28.879 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:28.878+0000 7f185f400640 1 -- 172.21.15.45:0/3621837875 learned_addr learned my addr 172.21.15.45:0/3621837875 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:10:28.879 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:28.878+0000 7f185ea00640 1 --2- 172.21.15.45:0/3621837875 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1860109370 0x7f18601a0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:28.879 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:28.878+0000 7f185fe00640 1 --2- 172.21.15.45:0/3621837875 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f186019a7e0 0x7f186019ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:28.880 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:28.878+0000 7f185f400640 1 -- 172.21.15.45:0/3621837875 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f186019a7e0 msgr2=0x7f186019ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:28.880 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:28.878+0000 7f185f400640 1 --2- 172.21.15.45:0/3621837875 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f186019a7e0 0x7f186019ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:28.880 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:28.878+0000 7f185f400640 1 -- 172.21.15.45:0/3621837875 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1860109370 msgr2=0x7f18601a0b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:28.880 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:28.878+0000 7f185f400640 1 --2- 172.21.15.45:0/3621837875 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1860109370 0x7f18601a0b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:28.880 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:28.878+0000 7f185f400640 1 -- 172.21.15.45:0/3621837875 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f186019b370 con 0x7f18601042a0 2024-04-16T17:10:28.880 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:28.878+0000 7f185ea00640 1 --2- 172.21.15.45:0/3621837875 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1860109370 0x7f18601a0b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:10:28.880 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:28.878+0000 7f185f400640 1 --2- 172.21.15.45:0/3621837875 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f18601042a0 0x7f18601a0620 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f1850006010 tx=0x7f185002cb40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:10:28.880 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:28.879+0000 7f1857e00640 1 -- 172.21.15.45:0/3621837875 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1850041070 con 0x7f18601042a0 2024-04-16T17:10:28.880 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:28.879+0000 7f1864e00640 1 -- 172.21.15.45:0/3621837875 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f186019b600 con 0x7f18601042a0 2024-04-16T17:10:28.881 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:28.879+0000 7f1857e00640 1 -- 172.21.15.45:0/3621837875 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1850004450 con 0x7f18601042a0 2024-04-16T17:10:28.881 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:28.879+0000 7f1857e00640 1 -- 172.21.15.45:0/3621837875 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1850003e30 con 0x7f18601042a0 2024-04-16T17:10:28.881 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:28.879+0000 7f1864e00640 1 -- 172.21.15.45:0/3621837875 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18601a7760 con 0x7f18601042a0 2024-04-16T17:10:28.883 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:28.882+0000 7f1857e00640 1 -- 172.21.15.45:0/3621837875 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f1850044070 con 0x7f18601042a0 2024-04-16T17:10:28.884 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:28.883+0000 7f1857e00640 1 --2- 172.21.15.45:0/3621837875 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f18440a3d10 0x7f18440a61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:28.884 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:28.884+0000 7f18691d4740 1 Processor -- start 2024-04-16T17:10:28.884 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:28.884+0000 7f1857e00640 1 -- 172.21.15.45:0/3621837875 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f18500f26b0 con 0x7f18601042a0 2024-04-16T17:10:28.885 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:28.884+0000 7f18691d4740 1 -- start start 2024-04-16T17:10:28.885 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:28.884+0000 7f185ea00640 1 --2- 172.21.15.45:0/3621837875 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f18440a3d10 0x7f18440a61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:28.886 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:28.884+0000 7f18691d4740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e621df17f0 0x55e621df0540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:28.886 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:28.884+0000 7f18691d4740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e621df0a80 0x55e621df0f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:28.886 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:28.884+0000 7f18691d4740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e621e02810 0x55e621e04c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:28.886 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:28.884+0000 7f18691d4740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55e621d55780 con 0x55e621df0a80 2024-04-16T17:10:28.886 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:28.884+0000 7f18691d4740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55e621cfbf40 con 0x55e621df17f0 2024-04-16T17:10:28.886 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:28.885+0000 7f18691d4740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55e621d51740 con 0x55e621e02810 2024-04-16T17:10:28.886 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:28.885+0000 7f185f400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e621df17f0 0x55e621df0540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:28.886 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:28.885+0000 7f185f400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e621df17f0 0x55e621df0540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:52004/0 (socket says 172.21.15.45:52004) 2024-04-16T17:10:28.887 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:28.885+0000 7f185f400640 1 -- 172.21.15.45:0/1539139152 learned_addr learned my addr 172.21.15.45:0/1539139152 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:10:28.887 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:28.885+0000 7f185fe00640 1 --2- 172.21.15.45:0/1539139152 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e621e02810 0x55e621e04c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:28.887 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:28.885+0000 7f185ea00640 1 --2- 172.21.15.45:0/3621837875 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f18440a3d10 0x7f18440a61d0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f184c0058c0 tx=0x7f184c005850 comp rx=0 tx=0).ready entity=mgr.25411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:10:28.887 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:28.885+0000 7f185ea00640 1 --2- 172.21.15.45:0/1539139152 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e621df0a80 0x55e621df0f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:28.887 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:28.885+0000 7f185fe00640 1 -- 172.21.15.45:0/1539139152 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e621df17f0 msgr2=0x55e621df0540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:10:28.887 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:28.885+0000 7f185fe00640 1 --2- 172.21.15.45:0/1539139152 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e621df17f0 0x55e621df0540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:28.887 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:28.885+0000 7f185fe00640 1 -- 172.21.15.45:0/1539139152 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e621df0a80 msgr2=0x55e621df0f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:10:28.887 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:28.885+0000 7f185fe00640 1 --2- 172.21.15.45:0/1539139152 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e621df0a80 0x55e621df0f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:28.887 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:28.885+0000 7f185fe00640 1 -- 172.21.15.45:0/1539139152 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e621a68210 con 0x55e621e02810 2024-04-16T17:10:28.887 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:28.885+0000 7f185f400640 1 --2- 172.21.15.45:0/1539139152 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e621df17f0 0x55e621df0540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:10:28.887 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:28.886+0000 7f185fe00640 1 --2- 172.21.15.45:0/1539139152 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e621e02810 0x55e621e04c00 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f1858009e50 tx=0x7f18580107a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4bd1bc383df2648e server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:10:28.888 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:28.886+0000 7f1854c00640 1 -- 172.21.15.45:0/1539139152 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1858003b70 con 0x55e621e02810 2024-04-16T17:10:28.888 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:28.886+0000 7f1854c00640 1 -- 172.21.15.45:0/1539139152 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1858015070 con 0x55e621e02810 2024-04-16T17:10:28.888 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:28.886+0000 7f1854c00640 1 -- 172.21.15.45:0/1539139152 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1858011820 con 0x55e621e02810 2024-04-16T17:10:28.888 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:28.887+0000 7f18691d4740 1 -- 172.21.15.45:0/1539139152 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e621e02810 msgr2=0x55e621e04c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:28.888 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:28.887+0000 7f18691d4740 1 --2- 172.21.15.45:0/1539139152 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e621e02810 0x55e621e04c00 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f1858009e50 tx=0x7f18580107a0 comp rx=0 tx=0).stop 2024-04-16T17:10:28.889 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:28.887+0000 7f18691d4740 1 -- 172.21.15.45:0/1539139152 shutdown_connections 2024-04-16T17:10:28.889 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:28.888+0000 7f18691d4740 1 --2- 172.21.15.45:0/1539139152 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e621e02810 0x55e621e04c00 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:28.889 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:28.888+0000 7f18691d4740 1 --2- 172.21.15.45:0/1539139152 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e621df0a80 0x55e621df0f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:28.889 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:28.888+0000 7f18691d4740 1 --2- 172.21.15.45:0/1539139152 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e621df17f0 0x55e621df0540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:28.890 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:28.888+0000 7f18691d4740 1 -- 172.21.15.45:0/1539139152 >> 172.21.15.45:0/1539139152 conn(0x55e621d8b5d0 msgr2=0x55e621deb770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:10:28.890 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:28.888+0000 7f18691d4740 1 -- 172.21.15.45:0/1539139152 shutdown_connections 2024-04-16T17:10:28.890 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:28.888+0000 7f18691d4740 1 -- 172.21.15.45:0/1539139152 wait complete. 2024-04-16T17:10:28.890 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:28.889+0000 7f18691d4740 1 -- start start 2024-04-16T17:10:28.890 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:28.889+0000 7f18691d4740 1 Processor -- start 2024-04-16T17:10:28.891 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:28.889+0000 7f18691d4740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e621df0a80 0x55e621e022a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:28.891 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:28.890+0000 7f18691d4740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e621df17f0 0x55e621dff630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:28.891 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:28.890+0000 7f18691d4740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e621e02810 0x55e621dffd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:28.891 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:28.890+0000 7f18691d4740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55e621e05540 con 0x55e621e02810 2024-04-16T17:10:28.891 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:28.890+0000 7f18691d4740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55e621d0f390 con 0x55e621df17f0 2024-04-16T17:10:28.891 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:28.890+0000 7f18691d4740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55e621e056c0 con 0x55e621df0a80 2024-04-16T17:10:28.891 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:28.890+0000 7f18691d4740 10 client.0 _ll_register_callbacks cb 0x55e621cf8f60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:10:28.892 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:28.890+0000 7f185fe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e621e02810 0x55e621dffd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:28.892 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:28.890+0000 7f185fe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e621e02810 0x55e621dffd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54114/0 (socket says 172.21.15.45:54114) 2024-04-16T17:10:28.892 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:28.890+0000 7f185fe00640 1 -- 172.21.15.45:0/727437709 learned_addr learned my addr 172.21.15.45:0/727437709 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:10:28.892 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:28.890+0000 7f185f400640 1 --2- 172.21.15.45:0/727437709 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e621df0a80 0x55e621e022a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:28.892 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:28.890+0000 7f185ea00640 1 --2- 172.21.15.45:0/727437709 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e621df17f0 0x55e621dff630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:28.892 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:28.890+0000 7f185f400640 1 -- 172.21.15.45:0/727437709 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e621df17f0 msgr2=0x55e621dff630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:10:28.892 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:28.890+0000 7f185f400640 1 --2- 172.21.15.45:0/727437709 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e621df17f0 0x55e621dff630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:28.892 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:28.890+0000 7f185f400640 1 -- 172.21.15.45:0/727437709 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e621e02810 msgr2=0x55e621dffd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:10:28.892 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:28.891+0000 7f185f400640 1 --2- 172.21.15.45:0/727437709 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e621e02810 0x55e621dffd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:28.892 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:28.891+0000 7f185f400640 1 -- 172.21.15.45:0/727437709 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e621b34620 con 0x55e621df0a80 2024-04-16T17:10:28.894 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:28.891+0000 7f185f400640 1 --2- 172.21.15.45:0/727437709 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e621df0a80 0x55e621e022a0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f18500bad80 tx=0x7f18500bddd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ea197750336e2f63 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:10:28.894 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:28.891+0000 7f1849600640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-04-16T17:10:28.894 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:28.891+0000 7f1849600640 1 -- 172.21.15.45:0/727437709 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f185004b040 con 0x55e621df0a80 2024-04-16T17:10:28.894 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:28.891+0000 7f1849600640 1 -- 172.21.15.45:0/727437709 <== mon.1 v2:172.21.15.117:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f18500f2280 con 0x55e621df0a80 2024-04-16T17:10:28.894 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:28.891+0000 7f18691d4740 1 -- 172.21.15.45:0/727437709 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1858015070 con 0x55e621df0a80 2024-04-16T17:10:28.894 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:28.891+0000 7f1849600640 1 -- 172.21.15.45:0/727437709 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f18500c2760 con 0x55e621df0a80 2024-04-16T17:10:28.894 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:28.891+0000 7f18691d4740 10 client.26635 fetch_fsmap learned FSMap version 30 2024-04-16T17:10:28.894 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:28.891+0000 7f18691d4740 1 -- 172.21.15.45:0/727437709 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55e621e05540 con 0x55e621df0a80 2024-04-16T17:10:28.894 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:28.892+0000 7f1849600640 1 -- 172.21.15.45:0/727437709 <== mon.1 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1858015070 con 0x55e621df0a80 2024-04-16T17:10:28.894 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:28.892+0000 7f1849600640 1 -- 172.21.15.45:0/727437709 <== mon.1 v2:172.21.15.117:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f18500dd1b0 con 0x55e621df0a80 2024-04-16T17:10:28.895 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:28.892+0000 7f18691d4740 10 client.26635 fetch_fsmap finished waiting for FSMap version 30 2024-04-16T17:10:28.895 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:28.892+0000 7f18691d4740 10 client.26635 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T17:10:28.895 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:28.892+0000 7f18691d4740 20 client.26635 populate_metadata read hostname 'smithi045' 2024-04-16T17:10:28.895 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:28.892+0000 7f18691d4740 1 --2- 172.21.15.45:0/727437709 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55e621e12a90 0x55e621e14ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:28.895 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:28.892+0000 7f18691d4740 4 client.26635 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:10:28.895 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:28.892+0000 7f18691d4740 1 -- 172.21.15.45:0/727437709 --> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e621d631a0 con 0x55e621e12a90 2024-04-16T17:10:28.895 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:28.892+0000 7f185ea00640 1 --2- 172.21.15.45:0/727437709 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55e621e12a90 0x55e621e14ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:28.895 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:28.893+0000 7f1849600640 10 client.26635 ms_handle_connect on v2:172.21.15.104:6838/3660070592 2024-04-16T17:10:28.895 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:28.893+0000 7f185ea00640 1 --2- 172.21.15.45:0/727437709 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55e621e12a90 0x55e621e14ef0 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b2ea7f71b703f85 server_cookie=a04a866bf8568f2e in_seq=0 out_seq=0 2024-04-16T17:10:28.896 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:28.894+0000 7f1849600640 1 -- 172.21.15.45:0/727437709 <== mds.0 v2:172.21.15.104:6838/3660070592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e621d631a0 con 0x55e621e12a90 2024-04-16T17:10:28.896 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:28.894+0000 7f1849600640 10 client.26635 handle_command_reply: tid=0 2024-04-16T17:10:28.896 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:28.894+0000 7f18691d4740 1 client.26635 shutdown 2024-04-16T17:10:28.896 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:28.894+0000 7f18691d4740 1 -- 172.21.15.45:0/727437709 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e621df0a80 msgr2=0x55e621e022a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:28.897 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:28.894+0000 7f18691d4740 1 --2- 172.21.15.45:0/727437709 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e621df0a80 0x55e621e022a0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f18500bad80 tx=0x7f18500bddd0 comp rx=0 tx=0).stop 2024-04-16T17:10:28.897 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:28.894+0000 7f18691d4740 1 -- 172.21.15.45:0/727437709 shutdown_connections 2024-04-16T17:10:28.897 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:28.894+0000 7f18691d4740 1 --2- 172.21.15.45:0/727437709 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e621e02810 0x55e621dffd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:28.897 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:28.894+0000 7f18691d4740 1 --2- 172.21.15.45:0/727437709 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e621df17f0 0x55e621dff630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:28.897 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:28.894+0000 7f18691d4740 1 --2- 172.21.15.45:0/727437709 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55e621e12a90 0x55e621e14ef0 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:28.897 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:28.895+0000 7f18691d4740 1 --2- 172.21.15.45:0/727437709 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e621df0a80 0x55e621e022a0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:28.897 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:28.895+0000 7f18691d4740 1 -- 172.21.15.45:0/727437709 >> 172.21.15.45:0/727437709 conn(0x55e621d8b5d0 msgr2=0x55e621ded4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:10:28.897 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:28.895+0000 7f1849600640 0 client.26635 ms_handle_reset on v2:172.21.15.104:6838/3660070592 2024-04-16T17:10:28.897 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:28.895+0000 7f18691d4740 1 -- 172.21.15.45:0/727437709 shutdown_connections 2024-04-16T17:10:28.897 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:28.895+0000 7f18691d4740 1 -- 172.21.15.45:0/727437709 wait complete. 2024-04-16T17:10:28.897 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:28.895+0000 7f18691d4740 20 client.26635 trim_cache size 0 max 16384 2024-04-16T17:10:28.927 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:28.926+0000 7f18691d4740 1 Processor -- start 2024-04-16T17:10:28.927 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:28.926+0000 7f18691d4740 1 -- start start 2024-04-16T17:10:28.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:10:28 smithi104 ceph-mon[36457]: pgmap v114: 129 pgs: 129 active+clean; 37 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 787 KiB/s rd, 729 KiB/s wr, 95 op/s 2024-04-16T17:10:28.928 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:28.927+0000 7f18691d4740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e621df0a80 0x55e621e0e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:28.928 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:28.927+0000 7f18691d4740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e621df17f0 0x55e621e01d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:28.928 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:28.927+0000 7f18691d4740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e621e02810 0x55e621e022a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:28.929 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:28.927+0000 7f18691d4740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f18601afac0 con 0x55e621e02810 2024-04-16T17:10:28.929 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:28.927+0000 7f18691d4740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f18601ad2d0 con 0x55e621df17f0 2024-04-16T17:10:28.929 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:28.927+0000 7f18691d4740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f18601ab100 con 0x55e621df0a80 2024-04-16T17:10:28.929 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:28.927+0000 7f185ea00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e621df17f0 0x55e621e01d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:28.929 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:28.927+0000 7f185ea00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e621df17f0 0x55e621e01d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:52016/0 (socket says 172.21.15.45:52016) 2024-04-16T17:10:28.929 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:28.927+0000 7f185ea00640 1 -- 172.21.15.45:0/2761009197 learned_addr learned my addr 172.21.15.45:0/2761009197 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:10:28.929 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:28.927+0000 7f185f400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e621df0a80 0x55e621e0e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:28.929 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:28.927+0000 7f185fe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e621e02810 0x55e621e022a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:28.929 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:28.927+0000 7f185ea00640 1 -- 172.21.15.45:0/2761009197 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e621df0a80 msgr2=0x55e621e0e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:10:28.929 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:28.927+0000 7f185ea00640 1 --2- 172.21.15.45:0/2761009197 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e621df0a80 0x55e621e0e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:28.929 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:28.927+0000 7f185ea00640 1 -- 172.21.15.45:0/2761009197 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e621e02810 msgr2=0x55e621e022a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:10:28.929 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:28.927+0000 7f185ea00640 1 --2- 172.21.15.45:0/2761009197 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e621e02810 0x55e621e022a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:28.929 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:28.927+0000 7f185ea00640 1 -- 172.21.15.45:0/2761009197 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e621defd50 con 0x55e621df17f0 2024-04-16T17:10:28.929 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:28.928+0000 7f185ea00640 1 --2- 172.21.15.45:0/2761009197 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e621df17f0 0x55e621e01d60 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f186019bae0 tx=0x7f184c015000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a92f363e106a22a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:10:28.931 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:28.928+0000 7f184be00640 1 -- 172.21.15.45:0/2761009197 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f184c021d80 con 0x55e621df17f0 2024-04-16T17:10:28.931 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:28.928+0000 7f184be00640 1 -- 172.21.15.45:0/2761009197 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f184c0293b0 con 0x55e621df17f0 2024-04-16T17:10:28.931 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:28.928+0000 7f184be00640 1 -- 172.21.15.45:0/2761009197 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f184c003850 con 0x55e621df17f0 2024-04-16T17:10:28.931 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:28.929+0000 7f18691d4740 1 -- 172.21.15.45:0/2761009197 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e621df17f0 msgr2=0x55e621e01d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:28.931 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:28.929+0000 7f18691d4740 1 --2- 172.21.15.45:0/2761009197 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e621df17f0 0x55e621e01d60 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f186019bae0 tx=0x7f184c015000 comp rx=0 tx=0).stop 2024-04-16T17:10:28.931 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:28.929+0000 7f18691d4740 1 -- 172.21.15.45:0/2761009197 shutdown_connections 2024-04-16T17:10:28.931 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:28.929+0000 7f18691d4740 1 --2- 172.21.15.45:0/2761009197 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e621e02810 0x55e621e022a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:28.931 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:28.929+0000 7f18691d4740 1 --2- 172.21.15.45:0/2761009197 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e621df17f0 0x55e621e01d60 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:28.931 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:28.929+0000 7f18691d4740 1 --2- 172.21.15.45:0/2761009197 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e621df0a80 0x55e621e0e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:28.931 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:28.929+0000 7f18691d4740 1 -- 172.21.15.45:0/2761009197 >> 172.21.15.45:0/2761009197 conn(0x55e621d8b5d0 msgr2=0x55e621deb2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:10:28.931 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:28.929+0000 7f18691d4740 1 -- 172.21.15.45:0/2761009197 shutdown_connections 2024-04-16T17:10:28.931 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:28.929+0000 7f18691d4740 1 -- 172.21.15.45:0/2761009197 wait complete. 2024-04-16T17:10:28.933 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:28.930+0000 7f18691d4740 1 -- start start 2024-04-16T17:10:28.933 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:28.930+0000 7f18691d4740 1 Processor -- start 2024-04-16T17:10:28.933 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:28.931+0000 7f18691d4740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e621df0a80 0x55e621dfbcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:28.933 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:28.931+0000 7f18691d4740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e621df17f0 0x55e621dfc1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:28.933 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:28.931+0000 7f18691d4740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e621e02810 0x55e621e09950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:28.933 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:28.931+0000 7f18691d4740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f18601ab790 con 0x55e621e02810 2024-04-16T17:10:28.933 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:28.931+0000 7f18691d4740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f18601acc00 con 0x55e621df17f0 2024-04-16T17:10:28.933 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:28.931+0000 7f18691d4740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f18601abe60 con 0x55e621df0a80 2024-04-16T17:10:28.933 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:28.931+0000 7f18691d4740 10 client.0 _ll_register_callbacks cb 0x7f18500ba650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:10:28.933 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:28.931+0000 7f185ea00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e621df17f0 0x55e621dfc1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:28.933 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:28.931+0000 7f185fe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e621e02810 0x55e621e09950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:28.933 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:28.931+0000 7f185ea00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e621df17f0 0x55e621dfc1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:52018/0 (socket says 172.21.15.45:52018) 2024-04-16T17:10:28.933 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:28.931+0000 7f185ea00640 1 -- 172.21.15.45:0/3373646183 learned_addr learned my addr 172.21.15.45:0/3373646183 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:10:28.933 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:28.931+0000 7f185fe00640 1 -- 172.21.15.45:0/3373646183 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e621df0a80 msgr2=0x55e621dfbcb0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-16T17:10:28.934 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:28.931+0000 7f185fe00640 1 --2- 172.21.15.45:0/3373646183 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e621df0a80 0x55e621dfbcb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:28.934 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:28.931+0000 7f185fe00640 1 -- 172.21.15.45:0/3373646183 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e621df17f0 msgr2=0x55e621dfc1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:10:28.934 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:28.931+0000 7f185fe00640 1 --2- 172.21.15.45:0/3373646183 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e621df17f0 0x55e621dfc1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:28.934 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:28.931+0000 7f185fe00640 1 -- 172.21.15.45:0/3373646183 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e621e09e90 con 0x55e621e02810 2024-04-16T17:10:28.935 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:28.932+0000 7f185fe00640 1 --2- 172.21.15.45:0/3373646183 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e621e02810 0x55e621e09950 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f1858006010 tx=0x7f1858003e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc3ad33443185d11 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:10:28.935 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:28.932+0000 7f1849600640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-16T17:10:28.936 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:28.932+0000 7f1849600640 1 -- 172.21.15.45:0/3373646183 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f185802f060 con 0x55e621e02810 2024-04-16T17:10:28.936 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:28.932+0000 7f1849600640 1 -- 172.21.15.45:0/3373646183 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f1858004be0 con 0x55e621e02810 2024-04-16T17:10:28.936 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:28.932+0000 7f18691d4740 1 -- 172.21.15.45:0/3373646183 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f184c0293b0 con 0x55e621e02810 2024-04-16T17:10:28.936 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:28.932+0000 7f1849600640 1 -- 172.21.15.45:0/3373646183 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f1858036050 con 0x55e621e02810 2024-04-16T17:10:28.936 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:28.932+0000 7f18691d4740 10 client.16524 fetch_fsmap learned FSMap version 30 2024-04-16T17:10:28.936 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:28.932+0000 7f18691d4740 1 -- 172.21.15.45:0/3373646183 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55e621e0b100 con 0x55e621e02810 2024-04-16T17:10:28.936 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:28.933+0000 7f1849600640 1 -- 172.21.15.45:0/3373646183 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f184c0293b0 con 0x55e621e02810 2024-04-16T17:10:28.936 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:28.933+0000 7f1849600640 1 -- 172.21.15.45:0/3373646183 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f185801a070 con 0x55e621e02810 2024-04-16T17:10:28.936 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:28.933+0000 7f18691d4740 10 client.16524 fetch_fsmap finished waiting for FSMap version 30 2024-04-16T17:10:28.936 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:28.933+0000 7f18691d4740 10 client.16524 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T17:10:28.936 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:28.933+0000 7f18691d4740 20 client.16524 populate_metadata read hostname 'smithi045' 2024-04-16T17:10:28.936 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:28.933+0000 7f18691d4740 1 --2- 172.21.15.45:0/3373646183 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55e621e79310 0x55e621e7b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:28.936 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:28.934+0000 7f18691d4740 4 client.16524 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-16T17:10:28.936 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:28.934+0000 7f18691d4740 1 -- 172.21.15.45:0/3373646183 --> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e621d5d010 con 0x55e621e79310 2024-04-16T17:10:28.936 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:28.934+0000 7f185f400640 1 --2- 172.21.15.45:0/3373646183 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55e621e79310 0x55e621e7b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:28.936 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:28.934+0000 7f185f400640 1 --2- 172.21.15.45:0/3373646183 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55e621e79310 0x55e621e7b730 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=acc72534599d5d18 server_cookie=9aaf52daee166e84 in_seq=0 out_seq=0 2024-04-16T17:10:28.937 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:28.934+0000 7f1849600640 10 client.16524 ms_handle_connect on v2:172.21.15.104:6838/3660070592 2024-04-16T17:10:28.955 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:28.953+0000 7f1849600640 1 -- 172.21.15.45:0/3373646183 <== mds.0 v2:172.21.15.104:6838/3660070592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e621d5d010 con 0x55e621e79310 2024-04-16T17:10:28.955 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:28.953+0000 7f1849600640 10 client.16524 handle_command_reply: tid=0 2024-04-16T17:10:28.955 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:28.953+0000 7f18691d4740 1 client.16524 shutdown 2024-04-16T17:10:28.955 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:28.954+0000 7f18691d4740 1 -- 172.21.15.45:0/3373646183 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e621e02810 msgr2=0x55e621e09950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:28.955 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:28.954+0000 7f18691d4740 1 --2- 172.21.15.45:0/3373646183 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e621e02810 0x55e621e09950 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f1858006010 tx=0x7f1858003e00 comp rx=0 tx=0).stop 2024-04-16T17:10:28.955 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:28.954+0000 7f18691d4740 1 -- 172.21.15.45:0/3373646183 shutdown_connections 2024-04-16T17:10:28.955 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:28.954+0000 7f18691d4740 1 --2- 172.21.15.45:0/3373646183 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e621e02810 0x55e621e09950 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:28.955 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:28.954+0000 7f18691d4740 1 --2- 172.21.15.45:0/3373646183 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e621df17f0 0x55e621dfc1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:28.955 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:28.954+0000 7f18691d4740 1 --2- 172.21.15.45:0/3373646183 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55e621e79310 0x55e621e7b730 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:28.955 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:28.954+0000 7f18691d4740 1 --2- 172.21.15.45:0/3373646183 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e621df0a80 0x55e621dfbcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:28.956 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:28.954+0000 7f18691d4740 1 -- 172.21.15.45:0/3373646183 >> 172.21.15.45:0/3373646183 conn(0x55e621d8b5d0 msgr2=0x55e621de9f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:10:28.956 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:28.954+0000 7f1849600640 0 client.16524 ms_handle_reset on v2:172.21.15.104:6838/3660070592 2024-04-16T17:10:28.956 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:28.954+0000 7f18691d4740 1 -- 172.21.15.45:0/3373646183 shutdown_connections 2024-04-16T17:10:28.956 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:28.954+0000 7f18691d4740 1 -- 172.21.15.45:0/3373646183 wait complete. 2024-04-16T17:10:28.956 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:28.954+0000 7f18691d4740 20 client.16524 trim_cache size 0 max 16384 2024-04-16T17:10:28.956 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-04-16T17:10:28.956 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-04-16T17:10:28.956 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-04-16T17:10:28.956 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-04-16T17:10:28.957 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:28.955+0000 7f1864e00640 1 -- 172.21.15.45:0/3621837875 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f18440a3d10 msgr2=0x7f18440a61d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:28.957 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:28.955+0000 7f1864e00640 1 --2- 172.21.15.45:0/3621837875 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f18440a3d10 0x7f18440a61d0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f184c0058c0 tx=0x7f184c005850 comp rx=0 tx=0).stop 2024-04-16T17:10:28.957 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:28.955+0000 7f1864e00640 1 -- 172.21.15.45:0/3621837875 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f18601042a0 msgr2=0x7f18601a0620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:28.957 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:28.955+0000 7f1864e00640 1 --2- 172.21.15.45:0/3621837875 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f18601042a0 0x7f18601a0620 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f1850006010 tx=0x7f185002cb40 comp rx=0 tx=0).stop 2024-04-16T17:10:28.957 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:28.955+0000 7f1864e00640 1 -- 172.21.15.45:0/3621837875 shutdown_connections 2024-04-16T17:10:28.957 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:28.955+0000 7f1864e00640 1 --2- 172.21.15.45:0/3621837875 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f18440a3d10 0x7f18440a61d0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:28.957 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:28.955+0000 7f1864e00640 1 --2- 172.21.15.45:0/3621837875 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f186019a7e0 0x7f186019ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:28.957 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:28.955+0000 7f1864e00640 1 --2- 172.21.15.45:0/3621837875 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1860109370 0x7f18601a0b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:28.958 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:28.955+0000 7f1864e00640 1 --2- 172.21.15.45:0/3621837875 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f18601042a0 0x7f18601a0620 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:28.958 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:28.955+0000 7f1864e00640 1 -- 172.21.15.45:0/3621837875 >> 172.21.15.45:0/3621837875 conn(0x7f18600ffb00 msgr2=0x7f1860100e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:10:28.958 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:28.955+0000 7f1864e00640 1 -- 172.21.15.45:0/3621837875 shutdown_connections 2024-04-16T17:10:28.958 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:28.956+0000 7f1864e00640 1 -- 172.21.15.45:0/3621837875 wait complete. 2024-04-16T17:10:29.238 INFO:tasks.workunit.client.0.smithi045.stdout: 1 47722 0.83 MB/sec execute 31 sec latency 77.612 ms 2024-04-16T17:10:30.106 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:10:28 smithi045 ceph-mon[31676]: pgmap v114: 129 pgs: 129 active+clean; 37 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 787 KiB/s rd, 729 KiB/s wr, 95 op/s 2024-04-16T17:10:30.243 INFO:tasks.workunit.client.0.smithi045.stdout: 1 48359 0.82 MB/sec execute 32 sec latency 52.520 ms 2024-04-16T17:10:30.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:10:30 smithi117 ceph-mon[36730]: pgmap v115: 129 pgs: 129 active+clean; 37 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 227 KiB/s rd, 547 KiB/s wr, 50 op/s 2024-04-16T17:10:30.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:10:30 smithi104 ceph-mon[36457]: pgmap v115: 129 pgs: 129 active+clean; 37 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 227 KiB/s rd, 547 KiB/s wr, 50 op/s 2024-04-16T17:10:31.244 INFO:tasks.workunit.client.0.smithi045.stdout: 1 48722 0.86 MB/sec execute 33 sec latency 74.776 ms 2024-04-16T17:10:32.244 INFO:tasks.workunit.client.0.smithi045.stdout: 1 49158 0.97 MB/sec execute 34 sec latency 74.330 ms 2024-04-16T17:10:32.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:10:32 smithi117 ceph-mon[36730]: pgmap v116: 129 pgs: 129 active+clean; 36 MiB data, 508 MiB used, 1.0 TiB / 1.0 TiB avail; 576 KiB/s rd, 684 KiB/s wr, 80 op/s 2024-04-16T17:10:32.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:10:32 smithi104 ceph-mon[36457]: pgmap v116: 129 pgs: 129 active+clean; 36 MiB data, 508 MiB used, 1.0 TiB / 1.0 TiB avail; 576 KiB/s rd, 684 KiB/s wr, 80 op/s 2024-04-16T17:10:33.064 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:10:30 smithi045 ceph-mon[31676]: pgmap v115: 129 pgs: 129 active+clean; 37 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 227 KiB/s rd, 547 KiB/s wr, 50 op/s 2024-04-16T17:10:33.064 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:10:32 smithi045 ceph-mon[31676]: pgmap v116: 129 pgs: 129 active+clean; 36 MiB data, 508 MiB used, 1.0 TiB / 1.0 TiB avail; 576 KiB/s rd, 684 KiB/s wr, 80 op/s 2024-04-16T17:10:33.243 INFO:tasks.workunit.client.0.smithi045.stdout: 1 49967 1.12 MB/sec execute 35 sec latency 18.500 ms 2024-04-16T17:10:34.244 INFO:tasks.workunit.client.0.smithi045.stdout: 1 50284 1.10 MB/sec execute 36 sec latency 24.074 ms 2024-04-16T17:10:34.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:10:34 smithi117 ceph-mon[36730]: pgmap v117: 129 pgs: 129 active+clean; 36 MiB data, 508 MiB used, 1.0 TiB / 1.0 TiB avail; 378 KiB/s rd, 507 KiB/s wr, 48 op/s 2024-04-16T17:10:34.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:10:34 smithi104 ceph-mon[36457]: pgmap v117: 129 pgs: 129 active+clean; 36 MiB data, 508 MiB used, 1.0 TiB / 1.0 TiB avail; 378 KiB/s rd, 507 KiB/s wr, 48 op/s 2024-04-16T17:10:34.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:10:34 smithi045 ceph-mon[31676]: pgmap v117: 129 pgs: 129 active+clean; 36 MiB data, 508 MiB used, 1.0 TiB / 1.0 TiB avail; 378 KiB/s rd, 507 KiB/s wr, 48 op/s 2024-04-16T17:10:35.244 INFO:tasks.workunit.client.0.smithi045.stdout: 1 50411 1.08 MB/sec execute 37 sec latency 837.017 ms 2024-04-16T17:10:36.244 INFO:tasks.workunit.client.0.smithi045.stdout: 1 50411 1.05 MB/sec execute 38 sec latency 1837.134 ms 2024-04-16T17:10:36.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:10:36 smithi117 ceph-mon[36730]: pgmap v118: 129 pgs: 129 active+clean; 36 MiB data, 508 MiB used, 1.0 TiB / 1.0 TiB avail; 378 KiB/s rd, 507 KiB/s wr, 48 op/s 2024-04-16T17:10:36.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:10:36 smithi104 ceph-mon[36457]: pgmap v118: 129 pgs: 129 active+clean; 36 MiB data, 508 MiB used, 1.0 TiB / 1.0 TiB avail; 378 KiB/s rd, 507 KiB/s wr, 48 op/s 2024-04-16T17:10:36.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:10:36 smithi045 ceph-mon[31676]: pgmap v118: 129 pgs: 129 active+clean; 36 MiB data, 508 MiB used, 1.0 TiB / 1.0 TiB avail; 378 KiB/s rd, 507 KiB/s wr, 48 op/s 2024-04-16T17:10:37.244 INFO:tasks.workunit.client.0.smithi045.stdout: 1 50411 1.02 MB/sec execute 39 sec latency 2837.251 ms 2024-04-16T17:10:38.244 INFO:tasks.workunit.client.0.smithi045.stdout: 1 50832 1.04 MB/sec execute 40 sec latency 2864.256 ms 2024-04-16T17:10:38.804 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:10:38 smithi117 ceph-mon[36730]: pgmap v119: 129 pgs: 129 active+clean; 37 MiB data, 494 MiB used, 1.0 TiB / 1.0 TiB avail; 506 KiB/s rd, 634 KiB/s wr, 69 op/s 2024-04-16T17:10:38.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:10:38 smithi104 ceph-mon[36457]: pgmap v119: 129 pgs: 129 active+clean; 37 MiB data, 494 MiB used, 1.0 TiB / 1.0 TiB avail; 506 KiB/s rd, 634 KiB/s wr, 69 op/s 2024-04-16T17:10:39.244 INFO:tasks.workunit.client.0.smithi045.stdout: 1 51271 1.09 MB/sec execute 41 sec latency 67.992 ms 2024-04-16T17:10:40.245 INFO:tasks.workunit.client.0.smithi045.stdout: 1 51581 1.07 MB/sec execute 42 sec latency 67.281 ms 2024-04-16T17:10:40.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:10:40 smithi117 ceph-mon[36730]: pgmap v120: 129 pgs: 129 active+clean; 37 MiB data, 494 MiB used, 1.0 TiB / 1.0 TiB avail; 476 KiB/s rd, 264 KiB/s wr, 50 op/s 2024-04-16T17:10:40.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:10:40 smithi104 ceph-mon[36457]: pgmap v120: 129 pgs: 129 active+clean; 37 MiB data, 494 MiB used, 1.0 TiB / 1.0 TiB avail; 476 KiB/s rd, 264 KiB/s wr, 50 op/s 2024-04-16T17:10:41.245 INFO:tasks.workunit.client.0.smithi045.stdout: 1 52835 1.22 MB/sec execute 43 sec latency 20.570 ms 2024-04-16T17:10:41.914 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:10:38 smithi045 ceph-mon[31676]: pgmap v119: 129 pgs: 129 active+clean; 37 MiB data, 494 MiB used, 1.0 TiB / 1.0 TiB avail; 506 KiB/s rd, 634 KiB/s wr, 69 op/s 2024-04-16T17:10:41.914 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:10:40 smithi045 ceph-mon[31676]: pgmap v120: 129 pgs: 129 active+clean; 37 MiB data, 494 MiB used, 1.0 TiB / 1.0 TiB avail; 476 KiB/s rd, 264 KiB/s wr, 50 op/s 2024-04-16T17:10:42.101 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:42.100+0000 7fe1f9000640 1 -- 172.21.15.45:0/35446089 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe1f4074430 msgr2=0x7fe1f4074810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:42.102 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:42.100+0000 7fe1f9000640 1 --2- 172.21.15.45:0/35446089 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe1f4074430 0x7fe1f4074810 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fe1dc009900 tx=0x7fe1dc02bcd0 comp rx=0 tx=0).stop 2024-04-16T17:10:42.102 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:42.100+0000 7fe1f9000640 1 -- 172.21.15.45:0/35446089 shutdown_connections 2024-04-16T17:10:42.102 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:42.100+0000 7fe1f9000640 1 --2- 172.21.15.45:0/35446089 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe1f410e640 0x7fe1f4110a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:42.102 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:42.100+0000 7fe1f9000640 1 --2- 172.21.15.45:0/35446089 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe1f4074d50 0x7fe1f410e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:42.102 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:42.100+0000 7fe1f9000640 1 --2- 172.21.15.45:0/35446089 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe1f4074430 0x7fe1f4074810 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:42.102 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:42.100+0000 7fe1f9000640 1 -- 172.21.15.45:0/35446089 >> 172.21.15.45:0/35446089 conn(0x7fe1f40fd950 msgr2=0x7fe1f40ffd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:10:42.102 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:42.100+0000 7fe1f9000640 1 -- 172.21.15.45:0/35446089 shutdown_connections 2024-04-16T17:10:42.102 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:42.100+0000 7fe1f9000640 1 -- 172.21.15.45:0/35446089 wait complete. 2024-04-16T17:10:42.104 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:42.101+0000 7fe1f9000640 1 Processor -- start 2024-04-16T17:10:42.104 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:42.102+0000 7fe1f9000640 1 -- start start 2024-04-16T17:10:42.104 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:42.102+0000 7fe1f9000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe1f4074430 0x7fe1f41a06a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:42.104 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:42.102+0000 7fe1f9000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe1f4074d50 0x7fe1f41a0be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:42.104 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:42.102+0000 7fe1f9000640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe1f410e640 0x7fe1f419a760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:42.104 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:42.102+0000 7fe1f9000640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fe1f4114420 con 0x7fe1f4074430 2024-04-16T17:10:42.104 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:42.102+0000 7fe1f9000640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fe1f41142a0 con 0x7fe1f410e640 2024-04-16T17:10:42.105 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:42.102+0000 7fe1f9000640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fe1f41145a0 con 0x7fe1f4074d50 2024-04-16T17:10:42.105 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:42.103+0000 7fe1f3400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe1f4074430 0x7fe1f41a06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:42.105 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:42.103+0000 7fe1f2a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe1f4074d50 0x7fe1f41a0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:42.105 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:42.103+0000 7fe1f2a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe1f4074d50 0x7fe1f41a0be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:35034/0 (socket says 172.21.15.45:35034) 2024-04-16T17:10:42.105 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:42.103+0000 7fe1f3e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe1f410e640 0x7fe1f419a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:42.105 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:42.103+0000 7fe1f2a00640 1 -- 172.21.15.45:0/1549135039 learned_addr learned my addr 172.21.15.45:0/1549135039 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:10:42.105 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:42.103+0000 7fe1f2a00640 1 -- 172.21.15.45:0/1549135039 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe1f410e640 msgr2=0x7fe1f419a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:42.105 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:42.103+0000 7fe1f2a00640 1 --2- 172.21.15.45:0/1549135039 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe1f410e640 0x7fe1f419a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:42.105 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:42.103+0000 7fe1f2a00640 1 -- 172.21.15.45:0/1549135039 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe1f4074430 msgr2=0x7fe1f41a06a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:42.105 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:42.103+0000 7fe1f2a00640 1 --2- 172.21.15.45:0/1549135039 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe1f4074430 0x7fe1f41a06a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:42.105 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:42.103+0000 7fe1f2a00640 1 -- 172.21.15.45:0/1549135039 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1f419b020 con 0x7fe1f4074d50 2024-04-16T17:10:42.106 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:42.103+0000 7fe1f2a00640 1 --2- 172.21.15.45:0/1549135039 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe1f4074d50 0x7fe1f41a0be0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fe1e400eab0 tx=0x7fe1e400ef70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:10:42.106 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:42.104+0000 7fe1ebe00640 1 -- 172.21.15.45:0/1549135039 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1e400ce40 con 0x7fe1f4074d50 2024-04-16T17:10:42.106 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:42.104+0000 7fe1ebe00640 1 -- 172.21.15.45:0/1549135039 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe1e4011950 con 0x7fe1f4074d50 2024-04-16T17:10:42.106 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:42.104+0000 7fe1ebe00640 1 -- 172.21.15.45:0/1549135039 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1e4010ae0 con 0x7fe1f4074d50 2024-04-16T17:10:42.107 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:42.104+0000 7fe1f9000640 1 -- 172.21.15.45:0/1549135039 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1f419b310 con 0x7fe1f4074d50 2024-04-16T17:10:42.107 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:42.104+0000 7fe1f9000640 1 -- 172.21.15.45:0/1549135039 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1f41a7620 con 0x7fe1f4074d50 2024-04-16T17:10:42.108 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:42.107+0000 7fe1ebe00640 1 -- 172.21.15.45:0/1549135039 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fe1e4011b00 con 0x7fe1f4074d50 2024-04-16T17:10:42.109 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:42.107+0000 7fe1fd3bc740 1 Processor -- start 2024-04-16T17:10:42.109 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:42.107+0000 7fe1fd3bc740 1 -- start start 2024-04-16T17:10:42.109 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:42.108+0000 7fe1fd3bc740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560fbd2727f0 0x560fbd271540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:42.109 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:42.108+0000 7fe1fd3bc740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560fbd271a80 0x560fbd271f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:42.109 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:42.108+0000 7fe1fd3bc740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560fbd27b000 0x560fbd27d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:42.109 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:42.108+0000 7fe1fd3bc740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x560fbd1d6780 con 0x560fbd27b000 2024-04-16T17:10:42.109 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:42.108+0000 7fe1fd3bc740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x560fbd17cf40 con 0x560fbd271a80 2024-04-16T17:10:42.109 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:42.108+0000 7fe1fd3bc740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x560fbd1d2740 con 0x560fbd2727f0 2024-04-16T17:10:42.109 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:42.108+0000 7fe1f3e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560fbd2727f0 0x560fbd271540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:42.110 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:42.108+0000 7fe1f3e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560fbd2727f0 0x560fbd271540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:35050/0 (socket says 172.21.15.45:35050) 2024-04-16T17:10:42.110 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:42.108+0000 7fe1f3e00640 1 -- 172.21.15.45:0/2233313415 learned_addr learned my addr 172.21.15.45:0/2233313415 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:10:42.110 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:42.108+0000 7fe1f2a00640 1 --2- 172.21.15.45:0/2233313415 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560fbd27b000 0x560fbd27d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:42.110 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:42.108+0000 7fe1f3400640 1 --2- 172.21.15.45:0/2233313415 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560fbd271a80 0x560fbd271f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:42.110 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:42.109+0000 7fe1f3e00640 1 -- 172.21.15.45:0/2233313415 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560fbd271a80 msgr2=0x560fbd271f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:10:42.110 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:42.109+0000 7fe1f3e00640 1 --2- 172.21.15.45:0/2233313415 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560fbd271a80 0x560fbd271f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:42.111 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:42.109+0000 7fe1f3e00640 1 -- 172.21.15.45:0/2233313415 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560fbd27b000 msgr2=0x560fbd27d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:10:42.111 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:42.109+0000 7fe1f3e00640 1 --2- 172.21.15.45:0/2233313415 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560fbd27b000 0x560fbd27d3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:42.111 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:42.109+0000 7fe1f3e00640 1 -- 172.21.15.45:0/2233313415 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560fbcee9210 con 0x560fbd2727f0 2024-04-16T17:10:42.111 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:42.109+0000 7fe1f3e00640 1 --2- 172.21.15.45:0/2233313415 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560fbd2727f0 0x560fbd271540 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fe1ec00b6b0 tx=0x7fe1ec004610 comp rx=0 tx=0).ready entity=mon.1 client_cookie=65409eb555e6712 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:10:42.111 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:42.109+0000 7fe1ebe00640 1 --2- 172.21.15.45:0/1549135039 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fe1d80a3d10 0x7fe1d80a61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:42.114 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:42.109+0000 7fe1e8c00640 1 -- 172.21.15.45:0/2233313415 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1ec003b20 con 0x560fbd2727f0 2024-04-16T17:10:42.114 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:42.109+0000 7fe1e8c00640 1 -- 172.21.15.45:0/2233313415 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe1ec015070 con 0x560fbd2727f0 2024-04-16T17:10:42.114 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:42.109+0000 7fe1e8c00640 1 -- 172.21.15.45:0/2233313415 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1ec0116a0 con 0x560fbd2727f0 2024-04-16T17:10:42.114 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:42.109+0000 7fe1ebe00640 1 -- 172.21.15.45:0/1549135039 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fe1e40d11b0 con 0x7fe1f4074d50 2024-04-16T17:10:42.114 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:42.110+0000 7fe1f3400640 1 --2- 172.21.15.45:0/1549135039 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fe1d80a3d10 0x7fe1d80a61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:42.115 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:42.110+0000 7fe1fd3bc740 1 -- 172.21.15.45:0/2233313415 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560fbd2727f0 msgr2=0x560fbd271540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:42.115 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:42.110+0000 7fe1fd3bc740 1 --2- 172.21.15.45:0/2233313415 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560fbd2727f0 0x560fbd271540 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fe1ec00b6b0 tx=0x7fe1ec004610 comp rx=0 tx=0).stop 2024-04-16T17:10:42.115 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:42.110+0000 7fe1fd3bc740 1 -- 172.21.15.45:0/2233313415 shutdown_connections 2024-04-16T17:10:42.115 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:42.110+0000 7fe1fd3bc740 1 --2- 172.21.15.45:0/2233313415 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560fbd27b000 0x560fbd27d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:42.115 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:42.110+0000 7fe1fd3bc740 1 --2- 172.21.15.45:0/2233313415 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560fbd271a80 0x560fbd271f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:42.115 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:42.110+0000 7fe1fd3bc740 1 --2- 172.21.15.45:0/2233313415 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560fbd2727f0 0x560fbd271540 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:42.115 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:42.110+0000 7fe1fd3bc740 1 -- 172.21.15.45:0/2233313415 >> 172.21.15.45:0/2233313415 conn(0x560fbd20c5d0 msgr2=0x560fbd26c770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:10:42.115 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:42.110+0000 7fe1f3400640 1 --2- 172.21.15.45:0/1549135039 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fe1d80a3d10 0x7fe1d80a61d0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fe1dc005ab0 tx=0x7fe1dc02e040 comp rx=0 tx=0).ready entity=mgr.25411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:10:42.115 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:42.110+0000 7fe1fd3bc740 1 -- 172.21.15.45:0/2233313415 shutdown_connections 2024-04-16T17:10:42.115 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:42.111+0000 7fe1fd3bc740 1 -- 172.21.15.45:0/2233313415 wait complete. 2024-04-16T17:10:42.115 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:42.111+0000 7fe1fd3bc740 1 -- start start 2024-04-16T17:10:42.115 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:42.111+0000 7fe1fd3bc740 1 Processor -- start 2024-04-16T17:10:42.116 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:42.112+0000 7fe1fd3bc740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560fbd271a80 0x560fbd27aac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:42.116 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:42.112+0000 7fe1fd3bc740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560fbd2727f0 0x560fbd277e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:42.116 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:42.112+0000 7fe1fd3bc740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560fbd27b000 0x560fbd278590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:42.116 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:42.112+0000 7fe1fd3bc740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x560fbd27dd30 con 0x560fbd2727f0 2024-04-16T17:10:42.116 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:42.112+0000 7fe1fd3bc740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x560fbd190390 con 0x560fbd271a80 2024-04-16T17:10:42.116 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:42.112+0000 7fe1fd3bc740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x560fbd27deb0 con 0x560fbd27b000 2024-04-16T17:10:42.116 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:42.112+0000 7fe1fd3bc740 10 client.0 _ll_register_callbacks cb 0x560fbd179f60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:10:42.116 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:42.112+0000 7fe1f3400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560fbd271a80 0x560fbd27aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:42.116 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:42.112+0000 7fe1f3400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560fbd271a80 0x560fbd27aac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:44644/0 (socket says 172.21.15.45:44644) 2024-04-16T17:10:42.116 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:42.112+0000 7fe1f3400640 1 -- 172.21.15.45:0/1989313069 learned_addr learned my addr 172.21.15.45:0/1989313069 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:10:42.116 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:42.112+0000 7fe1f3e00640 1 --2- 172.21.15.45:0/1989313069 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560fbd27b000 0x560fbd278590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:42.116 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:42.112+0000 7fe1f2a00640 1 --2- 172.21.15.45:0/1989313069 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560fbd2727f0 0x560fbd277e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:42.116 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:42.112+0000 7fe1f3400640 1 -- 172.21.15.45:0/1989313069 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560fbd27b000 msgr2=0x560fbd278590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:10:42.116 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:42.112+0000 7fe1f3400640 1 --2- 172.21.15.45:0/1989313069 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560fbd27b000 0x560fbd278590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:42.117 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:42.112+0000 7fe1f3400640 1 -- 172.21.15.45:0/1989313069 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560fbd2727f0 msgr2=0x560fbd277e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:10:42.117 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:42.112+0000 7fe1f3400640 1 --2- 172.21.15.45:0/1989313069 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560fbd2727f0 0x560fbd277e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:42.117 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:42.112+0000 7fe1f3400640 1 -- 172.21.15.45:0/1989313069 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560fbcfb5620 con 0x560fbd271a80 2024-04-16T17:10:42.117 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:42.113+0000 7fe1f3400640 1 --2- 172.21.15.45:0/1989313069 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560fbd271a80 0x560fbd27aac0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fe1dc006010 tx=0x7fe1dc02bcd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d77ff331c4724ffe server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:10:42.117 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:42.113+0000 7fe1e1600640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-04-16T17:10:42.117 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:42.113+0000 7fe1e1600640 1 -- 172.21.15.45:0/1989313069 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1dc051020 con 0x560fbd271a80 2024-04-16T17:10:42.117 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:42.113+0000 7fe1e1600640 1 -- 172.21.15.45:0/1989313069 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fe1dc0048d0 con 0x560fbd271a80 2024-04-16T17:10:42.117 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:42.113+0000 7fe1fd3bc740 1 -- 172.21.15.45:0/1989313069 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe1ec015070 con 0x560fbd271a80 2024-04-16T17:10:42.117 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:42.113+0000 7fe1e1600640 1 -- 172.21.15.45:0/1989313069 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fe1dc05c050 con 0x560fbd271a80 2024-04-16T17:10:42.117 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:42.113+0000 7fe1fd3bc740 10 client.26693 fetch_fsmap learned FSMap version 30 2024-04-16T17:10:42.117 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:42.113+0000 7fe1fd3bc740 1 -- 172.21.15.45:0/1989313069 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x560fbd289e20 con 0x560fbd271a80 2024-04-16T17:10:42.119 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:42.114+0000 7fe1e1600640 1 -- 172.21.15.45:0/1989313069 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe1ec015070 con 0x560fbd271a80 2024-04-16T17:10:42.119 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:42.114+0000 7fe1e1600640 1 -- 172.21.15.45:0/1989313069 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fe1dc003720 con 0x560fbd271a80 2024-04-16T17:10:42.119 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:42.114+0000 7fe1fd3bc740 10 client.26693 fetch_fsmap finished waiting for FSMap version 30 2024-04-16T17:10:42.119 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:42.114+0000 7fe1fd3bc740 10 client.26693 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T17:10:42.119 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:42.114+0000 7fe1fd3bc740 20 client.26693 populate_metadata read hostname 'smithi045' 2024-04-16T17:10:42.119 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:42.114+0000 7fe1fd3bc740 1 --2- 172.21.15.45:0/1989313069 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x560fbd28b380 0x560fbd28d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:42.119 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:42.114+0000 7fe1fd3bc740 4 client.26693 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:10:42.120 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:42.114+0000 7fe1fd3bc740 1 -- 172.21.15.45:0/1989313069 --> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560fbd1e41a0 con 0x560fbd28b380 2024-04-16T17:10:42.120 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:42.115+0000 7fe1f2a00640 1 --2- 172.21.15.45:0/1989313069 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x560fbd28b380 0x560fbd28d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:42.120 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:42.115+0000 7fe1f2a00640 1 --2- 172.21.15.45:0/1989313069 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x560fbd28b380 0x560fbd28d7b0 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a8a616153d51db40 server_cookie=8c39b2480eeabc31 in_seq=0 out_seq=0 2024-04-16T17:10:42.120 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:42.115+0000 7fe1e1600640 10 client.26693 ms_handle_connect on v2:172.21.15.104:6838/3660070592 2024-04-16T17:10:42.120 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:42.116+0000 7fe1e1600640 1 -- 172.21.15.45:0/1989313069 <== mds.0 v2:172.21.15.104:6838/3660070592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560fbd1e41a0 con 0x560fbd28b380 2024-04-16T17:10:42.120 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:42.116+0000 7fe1e1600640 10 client.26693 handle_command_reply: tid=0 2024-04-16T17:10:42.120 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:42.116+0000 7fe1fd3bc740 1 client.26693 shutdown 2024-04-16T17:10:42.120 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:42.116+0000 7fe1fd3bc740 1 -- 172.21.15.45:0/1989313069 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560fbd271a80 msgr2=0x560fbd27aac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:42.120 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:42.116+0000 7fe1fd3bc740 1 --2- 172.21.15.45:0/1989313069 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560fbd271a80 0x560fbd27aac0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fe1dc006010 tx=0x7fe1dc02bcd0 comp rx=0 tx=0).stop 2024-04-16T17:10:42.120 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:42.116+0000 7fe1fd3bc740 1 -- 172.21.15.45:0/1989313069 shutdown_connections 2024-04-16T17:10:42.120 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:42.116+0000 7fe1fd3bc740 1 --2- 172.21.15.45:0/1989313069 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x560fbd28b380 0x560fbd28d7b0 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:42.120 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:42.117+0000 7fe1fd3bc740 1 --2- 172.21.15.45:0/1989313069 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560fbd27b000 0x560fbd278590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:42.120 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:42.117+0000 7fe1fd3bc740 1 --2- 172.21.15.45:0/1989313069 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560fbd2727f0 0x560fbd277e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:42.120 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:42.117+0000 7fe1fd3bc740 1 --2- 172.21.15.45:0/1989313069 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560fbd271a80 0x560fbd27aac0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:42.121 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:42.117+0000 7fe1e1600640 0 client.26693 ms_handle_reset on v2:172.21.15.104:6838/3660070592 2024-04-16T17:10:42.121 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:42.117+0000 7fe1fd3bc740 1 -- 172.21.15.45:0/1989313069 >> 172.21.15.45:0/1989313069 conn(0x560fbd20c5d0 msgr2=0x560fbd26e4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:10:42.121 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:42.117+0000 7fe1fd3bc740 1 -- 172.21.15.45:0/1989313069 shutdown_connections 2024-04-16T17:10:42.121 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:42.117+0000 7fe1fd3bc740 1 -- 172.21.15.45:0/1989313069 wait complete. 2024-04-16T17:10:42.121 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:42.117+0000 7fe1fd3bc740 20 client.26693 trim_cache size 0 max 16384 2024-04-16T17:10:42.136 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:42.135+0000 7fe1fd3bc740 1 Processor -- start 2024-04-16T17:10:42.136 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:42.135+0000 7fe1fd3bc740 1 -- start start 2024-04-16T17:10:42.136 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:42.135+0000 7fe1fd3bc740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560fbd271a80 0x560fbd277df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:42.136 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:42.135+0000 7fe1fd3bc740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560fbd2727f0 0x560fbd286c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:42.136 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:42.135+0000 7fe1fd3bc740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560fbd27b000 0x560fbd27a580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:42.136 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:42.135+0000 7fe1fd3bc740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fe1f41b3dc0 con 0x560fbd2727f0 2024-04-16T17:10:42.137 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:42.135+0000 7fe1fd3bc740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fe1f41b15d0 con 0x560fbd27b000 2024-04-16T17:10:42.137 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:42.135+0000 7fe1fd3bc740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fe1f41af400 con 0x560fbd271a80 2024-04-16T17:10:42.137 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:42.135+0000 7fe1f3400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560fbd271a80 0x560fbd277df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:42.137 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:42.135+0000 7fe1f3e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560fbd27b000 0x560fbd27a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:42.137 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:42.135+0000 7fe1f3400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560fbd271a80 0x560fbd277df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:35066/0 (socket says 172.21.15.45:35066) 2024-04-16T17:10:42.137 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:42.135+0000 7fe1f3e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560fbd27b000 0x560fbd27a580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:44650/0 (socket says 172.21.15.45:44650) 2024-04-16T17:10:42.137 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:42.135+0000 7fe1f3400640 1 -- 172.21.15.45:0/1391757238 learned_addr learned my addr 172.21.15.45:0/1391757238 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:10:42.137 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:42.136+0000 7fe1f2a00640 1 --2- 172.21.15.45:0/1391757238 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560fbd2727f0 0x560fbd286c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:42.138 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:42.136+0000 7fe1f3400640 1 -- 172.21.15.45:0/1391757238 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560fbd27b000 msgr2=0x560fbd27a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:10:42.139 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:42.136+0000 7fe1f3400640 1 --2- 172.21.15.45:0/1391757238 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560fbd27b000 0x560fbd27a580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:42.139 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:42.136+0000 7fe1f3400640 1 -- 172.21.15.45:0/1391757238 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560fbd2727f0 msgr2=0x560fbd286c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:10:42.139 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:42.136+0000 7fe1f3400640 1 --2- 172.21.15.45:0/1391757238 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560fbd2727f0 0x560fbd286c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:42.139 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:42.136+0000 7fe1f3400640 1 -- 172.21.15.45:0/1391757238 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560fbd27d760 con 0x560fbd271a80 2024-04-16T17:10:42.139 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:42.136+0000 7fe1f3e00640 1 --2- 172.21.15.45:0/1391757238 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560fbd27b000 0x560fbd27a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:10:42.139 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:42.136+0000 7fe1f2a00640 1 --2- 172.21.15.45:0/1391757238 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560fbd2727f0 0x560fbd286c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:10:42.139 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:42.136+0000 7fe1f3400640 1 --2- 172.21.15.45:0/1391757238 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560fbd271a80 0x560fbd277df0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fe1dc003bc0 tx=0x7fe1dc009290 comp rx=0 tx=0).ready entity=mon.1 client_cookie=acfed2b2677cfdbd server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:10:42.139 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:42.136+0000 7fe1e3e00640 1 -- 172.21.15.45:0/1391757238 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1dc051070 con 0x560fbd271a80 2024-04-16T17:10:42.139 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:42.136+0000 7fe1e3e00640 1 -- 172.21.15.45:0/1391757238 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe1dc0048d0 con 0x560fbd271a80 2024-04-16T17:10:42.139 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:42.136+0000 7fe1e3e00640 1 -- 172.21.15.45:0/1391757238 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1dc0052d0 con 0x560fbd271a80 2024-04-16T17:10:42.139 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:42.137+0000 7fe1fd3bc740 1 -- 172.21.15.45:0/1391757238 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560fbd271a80 msgr2=0x560fbd277df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:42.139 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:42.137+0000 7fe1fd3bc740 1 --2- 172.21.15.45:0/1391757238 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560fbd271a80 0x560fbd277df0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fe1dc003bc0 tx=0x7fe1dc009290 comp rx=0 tx=0).stop 2024-04-16T17:10:42.139 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:42.137+0000 7fe1fd3bc740 1 -- 172.21.15.45:0/1391757238 shutdown_connections 2024-04-16T17:10:42.139 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:42.137+0000 7fe1fd3bc740 1 --2- 172.21.15.45:0/1391757238 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560fbd27b000 0x560fbd27a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:42.140 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:42.137+0000 7fe1fd3bc740 1 --2- 172.21.15.45:0/1391757238 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560fbd2727f0 0x560fbd286c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:42.140 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:42.137+0000 7fe1fd3bc740 1 --2- 172.21.15.45:0/1391757238 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560fbd271a80 0x560fbd277df0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:42.140 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:42.137+0000 7fe1fd3bc740 1 -- 172.21.15.45:0/1391757238 >> 172.21.15.45:0/1391757238 conn(0x560fbd20c5d0 msgr2=0x560fbd2776e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:10:42.140 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:42.137+0000 7fe1fd3bc740 1 -- 172.21.15.45:0/1391757238 shutdown_connections 2024-04-16T17:10:42.140 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:42.137+0000 7fe1fd3bc740 1 -- 172.21.15.45:0/1391757238 wait complete. 2024-04-16T17:10:42.140 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:42.138+0000 7fe1fd3bc740 1 -- start start 2024-04-16T17:10:42.140 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:42.138+0000 7fe1fd3bc740 1 Processor -- start 2024-04-16T17:10:42.142 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:42.138+0000 7fe1fd3bc740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560fbd271a80 0x560fbd274e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:42.142 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:42.138+0000 7fe1fd3bc740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560fbd2727f0 0x560fbd275350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:42.143 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:42.138+0000 7fe1fd3bc740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560fbd27b000 0x560fbd2f9140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:42.143 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:42.138+0000 7fe1fd3bc740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fe1f41afa90 con 0x560fbd271a80 2024-04-16T17:10:42.143 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:42.138+0000 7fe1fd3bc740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fe1f41b0f00 con 0x560fbd27b000 2024-04-16T17:10:42.143 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:42.138+0000 7fe1fd3bc740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fe1f41b0160 con 0x560fbd2727f0 2024-04-16T17:10:42.143 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:42.139+0000 7fe1fd3bc740 10 client.0 _ll_register_callbacks cb 0x7fe1dc044070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:10:42.143 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:42.139+0000 7fe1f3400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560fbd271a80 0x560fbd274e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:42.143 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:42.139+0000 7fe1f2a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560fbd2727f0 0x560fbd275350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:42.143 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:42.139+0000 7fe1f3400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560fbd271a80 0x560fbd274e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:46350/0 (socket says 172.21.15.45:46350) 2024-04-16T17:10:42.143 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:42.139+0000 7fe1f3400640 1 -- 172.21.15.45:0/2081864230 learned_addr learned my addr 172.21.15.45:0/2081864230 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:10:42.143 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:42.139+0000 7fe1f2a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560fbd2727f0 0x560fbd275350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:35072/0 (socket says 172.21.15.45:35072) 2024-04-16T17:10:42.143 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:42.139+0000 7fe1f3e00640 1 --2- 172.21.15.45:0/2081864230 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560fbd27b000 0x560fbd2f9140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:42.143 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:42.139+0000 7fe1f2a00640 1 -- 172.21.15.45:0/2081864230 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560fbd27b000 msgr2=0x560fbd2f9140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:10:42.143 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:42.139+0000 7fe1f2a00640 1 --2- 172.21.15.45:0/2081864230 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560fbd27b000 0x560fbd2f9140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:42.144 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:42.139+0000 7fe1f2a00640 1 -- 172.21.15.45:0/2081864230 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560fbd271a80 msgr2=0x560fbd274e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:10:42.144 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:42.139+0000 7fe1f2a00640 1 --2- 172.21.15.45:0/2081864230 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560fbd271a80 0x560fbd274e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:42.144 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:42.139+0000 7fe1f2a00640 1 -- 172.21.15.45:0/2081864230 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560fbd275be0 con 0x560fbd2727f0 2024-04-16T17:10:42.144 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:42.139+0000 7fe1f3400640 1 --2- 172.21.15.45:0/2081864230 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560fbd271a80 0x560fbd274e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:10:42.144 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:42.140+0000 7fe1f2a00640 1 --2- 172.21.15.45:0/2081864230 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560fbd2727f0 0x560fbd275350 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fe1e40ce550 tx=0x7fe1e4010540 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5615ea6699499407 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:10:42.144 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:42.140+0000 7fe1e1600640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-04-16T17:10:42.144 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:42.140+0000 7fe1e1600640 1 -- 172.21.15.45:0/2081864230 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1e409d4a0 con 0x560fbd2727f0 2024-04-16T17:10:42.144 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:42.140+0000 7fe1e1600640 1 -- 172.21.15.45:0/2081864230 <== mon.1 v2:172.21.15.117:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fe1e4099410 con 0x560fbd2727f0 2024-04-16T17:10:42.144 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:42.140+0000 7fe1fd3bc740 1 -- 172.21.15.45:0/2081864230 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe1dc0048d0 con 0x560fbd2727f0 2024-04-16T17:10:42.144 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:42.140+0000 7fe1e1600640 1 -- 172.21.15.45:0/2081864230 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fe1e409cda0 con 0x560fbd2727f0 2024-04-16T17:10:42.144 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:42.140+0000 7fe1fd3bc740 10 client.26656 fetch_fsmap learned FSMap version 30 2024-04-16T17:10:42.144 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:42.140+0000 7fe1fd3bc740 1 -- 172.21.15.45:0/2081864230 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x560fbd2fa7c0 con 0x560fbd2727f0 2024-04-16T17:10:42.144 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:42.140+0000 7fe1e1600640 1 -- 172.21.15.45:0/2081864230 <== mon.1 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe1dc0048d0 con 0x560fbd2727f0 2024-04-16T17:10:42.144 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:42.141+0000 7fe1e1600640 1 -- 172.21.15.45:0/2081864230 <== mon.1 v2:172.21.15.117:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fe1e400f040 con 0x560fbd2727f0 2024-04-16T17:10:42.145 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:42.141+0000 7fe1fd3bc740 10 client.26656 fetch_fsmap finished waiting for FSMap version 30 2024-04-16T17:10:42.145 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:42.141+0000 7fe1fd3bc740 10 client.26656 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T17:10:42.145 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:42.141+0000 7fe1fd3bc740 20 client.26656 populate_metadata read hostname 'smithi045' 2024-04-16T17:10:42.145 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:42.141+0000 7fe1fd3bc740 1 --2- 172.21.15.45:0/2081864230 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x560fbd2faf50 0x560fbd2fd370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:42.145 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:42.141+0000 7fe1fd3bc740 4 client.26656 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-16T17:10:42.145 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:42.141+0000 7fe1fd3bc740 1 -- 172.21.15.45:0/2081864230 --> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560fbd1de010 con 0x560fbd2faf50 2024-04-16T17:10:42.145 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:42.141+0000 7fe1f3400640 1 --2- 172.21.15.45:0/2081864230 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x560fbd2faf50 0x560fbd2fd370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:42.145 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:42.142+0000 7fe1f3400640 1 --2- 172.21.15.45:0/2081864230 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x560fbd2faf50 0x560fbd2fd370 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=780ccf15c1ade1bf server_cookie=ce9fd98a868a25f8 in_seq=0 out_seq=0 2024-04-16T17:10:42.145 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:42.142+0000 7fe1e1600640 10 client.26656 ms_handle_connect on v2:172.21.15.104:6838/3660070592 2024-04-16T17:10:42.204 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:42.201+0000 7fe1e1600640 1 -- 172.21.15.45:0/2081864230 <== mds.0 v2:172.21.15.104:6838/3660070592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x560fbd1de010 con 0x560fbd2faf50 2024-04-16T17:10:42.204 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:42.201+0000 7fe1e1600640 10 client.26656 handle_command_reply: tid=0 2024-04-16T17:10:42.204 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:42.201+0000 7fe1fd3bc740 1 client.26656 shutdown 2024-04-16T17:10:42.204 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:42.201+0000 7fe1fd3bc740 1 -- 172.21.15.45:0/2081864230 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560fbd2727f0 msgr2=0x560fbd275350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:42.204 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:42.201+0000 7fe1fd3bc740 1 --2- 172.21.15.45:0/2081864230 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560fbd2727f0 0x560fbd275350 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fe1e40ce550 tx=0x7fe1e4010540 comp rx=0 tx=0).stop 2024-04-16T17:10:42.204 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:42.201+0000 7fe1fd3bc740 1 -- 172.21.15.45:0/2081864230 shutdown_connections 2024-04-16T17:10:42.204 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:42.201+0000 7fe1fd3bc740 1 --2- 172.21.15.45:0/2081864230 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x560fbd2faf50 0x560fbd2fd370 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:42.204 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:42.201+0000 7fe1fd3bc740 1 --2- 172.21.15.45:0/2081864230 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560fbd2727f0 0x560fbd275350 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:42.204 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:42.201+0000 7fe1fd3bc740 1 --2- 172.21.15.45:0/2081864230 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560fbd27b000 0x560fbd2f9140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:42.204 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:42.201+0000 7fe1fd3bc740 1 --2- 172.21.15.45:0/2081864230 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560fbd271a80 0x560fbd274e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:42.204 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:42.201+0000 7fe1fd3bc740 1 -- 172.21.15.45:0/2081864230 >> 172.21.15.45:0/2081864230 conn(0x560fbd20c5d0 msgr2=0x560fbd26af80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:10:42.205 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:42.201+0000 7fe1e1600640 0 client.26656 ms_handle_reset on v2:172.21.15.104:6838/3660070592 2024-04-16T17:10:42.205 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:42.202+0000 7fe1fd3bc740 1 -- 172.21.15.45:0/2081864230 shutdown_connections 2024-04-16T17:10:42.205 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:42.202+0000 7fe1fd3bc740 1 -- 172.21.15.45:0/2081864230 wait complete. 2024-04-16T17:10:42.205 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:42.202+0000 7fe1fd3bc740 20 client.26656 trim_cache size 0 max 16384 2024-04-16T17:10:42.205 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-04-16T17:10:42.205 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-04-16T17:10:42.205 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-04-16T17:10:42.205 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-04-16T17:10:42.206 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:42.203+0000 7fe1f9000640 1 -- 172.21.15.45:0/1549135039 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fe1d80a3d10 msgr2=0x7fe1d80a61d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:42.206 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:42.203+0000 7fe1f9000640 1 --2- 172.21.15.45:0/1549135039 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fe1d80a3d10 0x7fe1d80a61d0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fe1dc005ab0 tx=0x7fe1dc02e040 comp rx=0 tx=0).stop 2024-04-16T17:10:42.207 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:42.203+0000 7fe1f9000640 1 -- 172.21.15.45:0/1549135039 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe1f4074d50 msgr2=0x7fe1f41a0be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:42.207 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:42.203+0000 7fe1f9000640 1 --2- 172.21.15.45:0/1549135039 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe1f4074d50 0x7fe1f41a0be0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fe1e400eab0 tx=0x7fe1e400ef70 comp rx=0 tx=0).stop 2024-04-16T17:10:42.207 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:42.203+0000 7fe1f9000640 1 -- 172.21.15.45:0/1549135039 shutdown_connections 2024-04-16T17:10:42.207 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:42.203+0000 7fe1f9000640 1 --2- 172.21.15.45:0/1549135039 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fe1d80a3d10 0x7fe1d80a61d0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:42.207 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:42.203+0000 7fe1f9000640 1 --2- 172.21.15.45:0/1549135039 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe1f4074d50 0x7fe1f41a0be0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:42.207 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:42.203+0000 7fe1f9000640 1 --2- 172.21.15.45:0/1549135039 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe1f410e640 0x7fe1f419a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:42.207 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:42.203+0000 7fe1f9000640 1 --2- 172.21.15.45:0/1549135039 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe1f4074430 0x7fe1f41a06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:42.207 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:42.203+0000 7fe1f9000640 1 -- 172.21.15.45:0/1549135039 >> 172.21.15.45:0/1549135039 conn(0x7fe1f40fd950 msgr2=0x7fe1f40fe6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:10:42.207 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:42.204+0000 7fe1f9000640 1 -- 172.21.15.45:0/1549135039 shutdown_connections 2024-04-16T17:10:42.207 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:42.204+0000 7fe1f9000640 1 -- 172.21.15.45:0/1549135039 wait complete. 2024-04-16T17:10:42.244 INFO:tasks.workunit.client.0.smithi045.stdout: 1 53513 1.33 MB/sec execute 44 sec latency 22.738 ms 2024-04-16T17:10:42.804 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:10:42 smithi117 ceph-mon[36730]: pgmap v121: 129 pgs: 129 active+clean; 39 MiB data, 512 MiB used, 1.0 TiB / 1.0 TiB avail; 822 KiB/s rd, 420 KiB/s wr, 77 op/s 2024-04-16T17:10:42.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:10:42 smithi104 ceph-mon[36457]: pgmap v121: 129 pgs: 129 active+clean; 39 MiB data, 512 MiB used, 1.0 TiB / 1.0 TiB avail; 822 KiB/s rd, 420 KiB/s wr, 77 op/s 2024-04-16T17:10:43.085 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:10:42 smithi045 ceph-mon[31676]: pgmap v121: 129 pgs: 129 active+clean; 39 MiB data, 512 MiB used, 1.0 TiB / 1.0 TiB avail; 822 KiB/s rd, 420 KiB/s wr, 77 op/s 2024-04-16T17:10:43.244 INFO:tasks.workunit.client.0.smithi045.stdout: 1 53807 1.30 MB/sec execute 45 sec latency 33.611 ms 2024-04-16T17:10:44.245 INFO:tasks.workunit.client.0.smithi045.stdout: 1 54027 1.28 MB/sec execute 46 sec latency 613.638 ms 2024-04-16T17:10:44.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:10:44 smithi117 ceph-mon[36730]: pgmap v122: 129 pgs: 129 active+clean; 39 MiB data, 512 MiB used, 1.0 TiB / 1.0 TiB avail; 473 KiB/s rd, 282 KiB/s wr, 47 op/s 2024-04-16T17:10:44.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:10:44 smithi104 ceph-mon[36457]: pgmap v122: 129 pgs: 129 active+clean; 39 MiB data, 512 MiB used, 1.0 TiB / 1.0 TiB avail; 473 KiB/s rd, 282 KiB/s wr, 47 op/s 2024-04-16T17:10:44.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:10:44 smithi045 ceph-mon[31676]: pgmap v122: 129 pgs: 129 active+clean; 39 MiB data, 512 MiB used, 1.0 TiB / 1.0 TiB avail; 473 KiB/s rd, 282 KiB/s wr, 47 op/s 2024-04-16T17:10:45.245 INFO:tasks.workunit.client.0.smithi045.stdout: 1 54027 1.26 MB/sec execute 47 sec latency 1613.721 ms 2024-04-16T17:10:45.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:10:45 smithi117 ceph-mon[36730]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-16T17:10:45.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:10:45 smithi104 ceph-mon[36457]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-16T17:10:45.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:10:45 smithi045 ceph-mon[31676]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-16T17:10:46.246 INFO:tasks.workunit.client.0.smithi045.stdout: 1 54027 1.23 MB/sec execute 48 sec latency 2614.871 ms 2024-04-16T17:10:46.700 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:10:46 smithi045 ceph-mon[31676]: pgmap v123: 129 pgs: 129 active+clean; 39 MiB data, 512 MiB used, 1.0 TiB / 1.0 TiB avail; 473 KiB/s rd, 282 KiB/s wr, 47 op/s 2024-04-16T17:10:46.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:10:46 smithi117 ceph-mon[36730]: pgmap v123: 129 pgs: 129 active+clean; 39 MiB data, 512 MiB used, 1.0 TiB / 1.0 TiB avail; 473 KiB/s rd, 282 KiB/s wr, 47 op/s 2024-04-16T17:10:46.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:10:46 smithi104 ceph-mon[36457]: pgmap v123: 129 pgs: 129 active+clean; 39 MiB data, 512 MiB used, 1.0 TiB / 1.0 TiB avail; 473 KiB/s rd, 282 KiB/s wr, 47 op/s 2024-04-16T17:10:47.246 INFO:tasks.workunit.client.0.smithi045.stdout: 1 54027 1.21 MB/sec execute 49 sec latency 3614.986 ms 2024-04-16T17:10:47.736 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-16T17:10:47.737 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-16T17:10:47.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:47.939+0000 7f1c47e00640 1 -- 172.21.15.45:0/3935562100 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1c48102ac0 msgr2=0x7f1c48106b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:47.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:47.939+0000 7f1c47e00640 1 --2- 172.21.15.45:0/3935562100 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1c48102ac0 0x7f1c48106b00 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f1c34009900 tx=0x7f1c3402bc50 comp rx=0 tx=0).stop 2024-04-16T17:10:47.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:47.939+0000 7f1c47e00640 1 -- 172.21.15.45:0/3935562100 shutdown_connections 2024-04-16T17:10:47.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:47.939+0000 7f1c47e00640 1 --2- 172.21.15.45:0/3935562100 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1c481071e0 0x7f1c4810edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:47.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:47.939+0000 7f1c47e00640 1 --2- 172.21.15.45:0/3935562100 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1c48102ac0 0x7f1c48106b00 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:47.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:47.939+0000 7f1c47e00640 1 --2- 172.21.15.45:0/3935562100 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1c48102110 0x7f1c481024f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:47.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:47.939+0000 7f1c47e00640 1 -- 172.21.15.45:0/3935562100 >> 172.21.15.45:0/3935562100 conn(0x7f1c480fd9d0 msgr2=0x7f1c480ffdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:10:47.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:47.940+0000 7f1c47e00640 1 -- 172.21.15.45:0/3935562100 shutdown_connections 2024-04-16T17:10:47.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:47.940+0000 7f1c47e00640 1 -- 172.21.15.45:0/3935562100 wait complete. 2024-04-16T17:10:47.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:47.941+0000 7f1c47e00640 1 Processor -- start 2024-04-16T17:10:47.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:47.941+0000 7f1c47e00640 1 -- start start 2024-04-16T17:10:47.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:47.941+0000 7f1c47e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1c48102110 0x7f1c4819e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:47.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:47.941+0000 7f1c47e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1c481071e0 0x7f1c4819eae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:47.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:47.941+0000 7f1c47e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1c48198660 0x7f1c48198ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:47.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:47.941+0000 7f1c47e00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f1c48112340 con 0x7f1c48102110 2024-04-16T17:10:47.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:47.941+0000 7f1c47e00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f1c481121c0 con 0x7f1c48198660 2024-04-16T17:10:47.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:47.941+0000 7f1c47e00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f1c481124c0 con 0x7f1c481071e0 2024-04-16T17:10:47.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:47.941+0000 7f1c46a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1c48102110 0x7f1c4819e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:47.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:47.942+0000 7f1c46a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1c48102110 0x7f1c4819e5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:46372/0 (socket says 172.21.15.45:46372) 2024-04-16T17:10:47.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:47.942+0000 7f1c46a00640 1 -- 172.21.15.45:0/1216850763 learned_addr learned my addr 172.21.15.45:0/1216850763 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:10:47.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:47.942+0000 7f1c47400640 1 --2- 172.21.15.45:0/1216850763 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1c48198660 0x7f1c48198ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:47.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:47.942+0000 7f1c46a00640 1 -- 172.21.15.45:0/1216850763 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1c481071e0 msgr2=0x7f1c4819eae0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-16T17:10:47.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:47.942+0000 7f1c46a00640 1 --2- 172.21.15.45:0/1216850763 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1c481071e0 0x7f1c4819eae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:47.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:47.942+0000 7f1c46a00640 1 -- 172.21.15.45:0/1216850763 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1c48198660 msgr2=0x7f1c48198ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:47.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:47.942+0000 7f1c46a00640 1 --2- 172.21.15.45:0/1216850763 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1c48198660 0x7f1c48198ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:47.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:47.942+0000 7f1c46a00640 1 -- 172.21.15.45:0/1216850763 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c48199260 con 0x7f1c48102110 2024-04-16T17:10:47.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:47.942+0000 7f1c47400640 1 --2- 172.21.15.45:0/1216850763 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1c48198660 0x7f1c48198ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:10:47.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:47.942+0000 7f1c46a00640 1 --2- 172.21.15.45:0/1216850763 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1c48102110 0x7f1c4819e5a0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f1c3800e9f0 tx=0x7f1c3800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:10:47.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:47.943+0000 7f1c3f400640 1 -- 172.21.15.45:0/1216850763 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c3800cd80 con 0x7f1c48102110 2024-04-16T17:10:47.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:47.943+0000 7f1c3f400640 1 -- 172.21.15.45:0/1216850763 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1c38010910 con 0x7f1c48102110 2024-04-16T17:10:47.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:47.943+0000 7f1c47e00640 1 -- 172.21.15.45:0/1216850763 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c48074230 con 0x7f1c48102110 2024-04-16T17:10:47.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:47.943+0000 7f1c47e00640 1 -- 172.21.15.45:0/1216850763 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c48074780 con 0x7f1c48102110 2024-04-16T17:10:47.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:47.943+0000 7f1c3f400640 1 -- 172.21.15.45:0/1216850763 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c38018990 con 0x7f1c48102110 2024-04-16T17:10:47.947 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:47.947+0000 7f1c50830740 1 Processor -- start 2024-04-16T17:10:47.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:47.947+0000 7f1c50830740 1 -- start start 2024-04-16T17:10:47.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:47.948+0000 7f1c50830740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a0b213d390 0x55a0b213d770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:47.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:47.948+0000 7f1c50830740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a0b213dcb0 0x55a0b2146350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:47.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:47.948+0000 7f1c50830740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a0b2146890 0x55a0b2148c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:47.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:47.948+0000 7f1c50830740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55a0b20a3120 con 0x55a0b213dcb0 2024-04-16T17:10:47.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:47.948+0000 7f1c50830740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55a0b2049340 con 0x55a0b2146890 2024-04-16T17:10:47.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:47.948+0000 7f1c50830740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55a0b209f0e0 con 0x55a0b213d390 2024-04-16T17:10:47.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:47.948+0000 7f1c3f400640 1 -- 172.21.15.45:0/1216850763 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f1c38010ac0 con 0x7f1c48102110 2024-04-16T17:10:47.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:47.948+0000 7f1c47400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a0b213d390 0x55a0b213d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:47.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:47.948+0000 7f1c47400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a0b213d390 0x55a0b213d770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:35092/0 (socket says 172.21.15.45:35092) 2024-04-16T17:10:47.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:47.948+0000 7f1c47400640 1 -- 172.21.15.45:0/3936838493 learned_addr learned my addr 172.21.15.45:0/3936838493 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:10:47.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:47.948+0000 7f1c46000640 1 --2- 172.21.15.45:0/3936838493 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a0b2146890 0x55a0b2148c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:47.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:47.948+0000 7f1c47400640 1 -- 172.21.15.45:0/3936838493 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a0b2146890 msgr2=0x55a0b2148c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:10:47.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:47.948+0000 7f1c47400640 1 --2- 172.21.15.45:0/3936838493 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a0b2146890 0x55a0b2148c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:47.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:47.948+0000 7f1c47400640 1 -- 172.21.15.45:0/3936838493 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a0b213dcb0 msgr2=0x55a0b2146350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:10:47.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:47.948+0000 7f1c47400640 1 --2- 172.21.15.45:0/3936838493 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a0b213dcb0 0x55a0b2146350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:47.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:47.948+0000 7f1c47400640 1 -- 172.21.15.45:0/3936838493 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a0b1e7f7d0 con 0x55a0b213d390 2024-04-16T17:10:47.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:47.950+0000 7f1c3f400640 1 --2- 172.21.15.45:0/1216850763 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f1c2c0a3d20 0x7f1c2c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:47.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:47.950+0000 7f1c3f400640 1 -- 172.21.15.45:0/1216850763 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f1c380d0f50 con 0x7f1c48102110 2024-04-16T17:10:47.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:47.950+0000 7f1c46a00640 1 --2- 172.21.15.45:0/1216850763 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f1c2c0a3d20 0x7f1c2c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:47.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:47.950+0000 7f1c46a00640 1 --2- 172.21.15.45:0/1216850763 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f1c2c0a3d20 0x7f1c2c0a61e0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x55a0b213ce60 tx=0x7f1c3800cf20 comp rx=0 tx=0).ready entity=mgr.25411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:10:47.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:47.953+0000 7f1c47400640 1 --2- 172.21.15.45:0/3936838493 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a0b213d390 0x55a0b213d770 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f1c4000a870 tx=0x7f1c40009650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c37ebb73dc91a2e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:10:47.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:47.954+0000 7f1c33e00640 1 -- 172.21.15.45:0/3936838493 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c40018020 con 0x55a0b213d390 2024-04-16T17:10:47.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:47.954+0000 7f1c33e00640 1 -- 172.21.15.45:0/3936838493 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1c40014400 con 0x55a0b213d390 2024-04-16T17:10:47.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:47.954+0000 7f1c33e00640 1 -- 172.21.15.45:0/3936838493 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c400135c0 con 0x55a0b213d390 2024-04-16T17:10:47.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:47.955+0000 7f1c50830740 1 -- 172.21.15.45:0/3936838493 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a0b213d390 msgr2=0x55a0b213d770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:47.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:47.955+0000 7f1c50830740 1 --2- 172.21.15.45:0/3936838493 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a0b213d390 0x55a0b213d770 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f1c4000a870 tx=0x7f1c40009650 comp rx=0 tx=0).stop 2024-04-16T17:10:47.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:47.955+0000 7f1c50830740 1 -- 172.21.15.45:0/3936838493 shutdown_connections 2024-04-16T17:10:47.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:47.955+0000 7f1c50830740 1 --2- 172.21.15.45:0/3936838493 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a0b2146890 0x55a0b2148c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:47.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:47.955+0000 7f1c50830740 1 --2- 172.21.15.45:0/3936838493 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a0b213dcb0 0x55a0b2146350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:47.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:47.955+0000 7f1c50830740 1 --2- 172.21.15.45:0/3936838493 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a0b213d390 0x55a0b213d770 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:47.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:47.955+0000 7f1c50830740 1 -- 172.21.15.45:0/3936838493 >> 172.21.15.45:0/3936838493 conn(0x55a0b2137ac0 msgr2=0x55a0b2137ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:10:47.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:47.955+0000 7f1c50830740 1 -- 172.21.15.45:0/3936838493 shutdown_connections 2024-04-16T17:10:47.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:47.956+0000 7f1c50830740 1 -- 172.21.15.45:0/3936838493 wait complete. 2024-04-16T17:10:47.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:47.956+0000 7f1c50830740 1 -- start start 2024-04-16T17:10:47.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:47.956+0000 7f1c50830740 1 Processor -- start 2024-04-16T17:10:47.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:47.958+0000 7f1c50830740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a0b213d390 0x55a0b2143980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:47.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:47.958+0000 7f1c50830740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a0b213dcb0 0x55a0b2143ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:47.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:47.958+0000 7f1c50830740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a0b2146890 0x55a0b2144400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:47.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:47.958+0000 7f1c50830740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55a0b2149680 con 0x55a0b2146890 2024-04-16T17:10:47.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:47.958+0000 7f1c50830740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55a0b205c710 con 0x55a0b213dcb0 2024-04-16T17:10:47.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:47.958+0000 7f1c50830740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55a0b2149800 con 0x55a0b213d390 2024-04-16T17:10:47.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:47.958+0000 7f1c50830740 10 client.0 _ll_register_callbacks cb 0x55a0b2057af0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:10:47.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:47.958+0000 7f1c47400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a0b2146890 0x55a0b2144400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:47.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:47.958+0000 7f1c47400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a0b2146890 0x55a0b2144400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:46396/0 (socket says 172.21.15.45:46396) 2024-04-16T17:10:47.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:47.958+0000 7f1c47400640 1 -- 172.21.15.45:0/1859313251 learned_addr learned my addr 172.21.15.45:0/1859313251 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:10:47.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:47.958+0000 7f1c46000640 1 --2- 172.21.15.45:0/1859313251 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a0b213dcb0 0x55a0b2143ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:47.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:47.958+0000 7f1c47400640 1 -- 172.21.15.45:0/1859313251 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a0b213d390 msgr2=0x55a0b2143980 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-16T17:10:47.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:47.958+0000 7f1c47400640 1 --2- 172.21.15.45:0/1859313251 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a0b213d390 0x55a0b2143980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:47.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:47.958+0000 7f1c47400640 1 -- 172.21.15.45:0/1859313251 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a0b213dcb0 msgr2=0x55a0b2143ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:10:47.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:47.958+0000 7f1c47400640 1 --2- 172.21.15.45:0/1859313251 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a0b213dcb0 0x55a0b2143ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:47.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:47.958+0000 7f1c47400640 1 -- 172.21.15.45:0/1859313251 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a0b1db53f0 con 0x55a0b2146890 2024-04-16T17:10:47.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:47.959+0000 7f1c46000640 1 --2- 172.21.15.45:0/1859313251 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a0b213dcb0 0x55a0b2143ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:10:47.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:47.959+0000 7f1c47400640 1 --2- 172.21.15.45:0/1859313251 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a0b2146890 0x55a0b2144400 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f1c4000ef30 tx=0x7f1c4000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e448443577ced443 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:10:47.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:47.959+0000 7f1c30c00640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-16T17:10:47.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:47.959+0000 7f1c30c00640 1 -- 172.21.15.45:0/1859313251 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c40018040 con 0x55a0b2146890 2024-04-16T17:10:47.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:47.959+0000 7f1c50830740 1 -- 172.21.15.45:0/1859313251 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1c40014400 con 0x55a0b2146890 2024-04-16T17:10:47.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:47.959+0000 7f1c30c00640 1 -- 172.21.15.45:0/1859313251 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f1c400149c0 con 0x55a0b2146890 2024-04-16T17:10:47.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:47.960+0000 7f1c30c00640 1 -- 172.21.15.45:0/1859313251 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f1c40014bc0 con 0x55a0b2146890 2024-04-16T17:10:47.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:47.960+0000 7f1c50830740 10 client.16551 fetch_fsmap learned FSMap version 31 2024-04-16T17:10:47.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:47.960+0000 7f1c50830740 1 -- 172.21.15.45:0/1859313251 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55a0b1d750f0 con 0x55a0b2146890 2024-04-16T17:10:47.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:47.960+0000 7f1c30c00640 1 -- 172.21.15.45:0/1859313251 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1c40014400 con 0x55a0b2146890 2024-04-16T17:10:47.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:47.960+0000 7f1c30c00640 1 -- 172.21.15.45:0/1859313251 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 31) ==== 8366+0+0 (secure 0 0 0) 0x7f1c4001e670 con 0x55a0b2146890 2024-04-16T17:10:47.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:47.960+0000 7f1c50830740 10 client.16551 fetch_fsmap finished waiting for FSMap version 31 2024-04-16T17:10:47.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:47.960+0000 7f1c50830740 10 client.16551 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:10:47.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:47.960+0000 7f1c50830740 20 client.16551 populate_metadata read hostname 'smithi045' 2024-04-16T17:10:47.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:47.960+0000 7f1c50830740 1 --2- 172.21.15.45:0/1859313251 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55a0b2156c10 0x55a0b2159000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:47.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:47.960+0000 7f1c50830740 4 client.16551 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:10:47.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:47.960+0000 7f1c50830740 1 -- 172.21.15.45:0/1859313251 --> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a0b20b0b40 con 0x55a0b2156c10 2024-04-16T17:10:47.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:47.961+0000 7f1c46000640 1 --2- 172.21.15.45:0/1859313251 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55a0b2156c10 0x55a0b2159000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:47.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:47.961+0000 7f1c46000640 1 --2- 172.21.15.45:0/1859313251 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55a0b2156c10 0x55a0b2159000 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=56d72d67b670b7bd server_cookie=fd60b756bbca5a30 in_seq=0 out_seq=0 2024-04-16T17:10:47.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:47.961+0000 7f1c30c00640 10 client.16551 ms_handle_connect on v2:172.21.15.104:6838/3660070592 2024-04-16T17:10:47.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:47.962+0000 7f1c30c00640 1 -- 172.21.15.45:0/1859313251 <== mds.0 v2:172.21.15.104:6838/3660070592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a0b20b0b40 con 0x55a0b2156c10 2024-04-16T17:10:47.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:47.962+0000 7f1c30c00640 10 client.16551 handle_command_reply: tid=0 2024-04-16T17:10:47.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:47.962+0000 7f1c50830740 1 client.16551 shutdown 2024-04-16T17:10:47.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:47.962+0000 7f1c50830740 1 -- 172.21.15.45:0/1859313251 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a0b2146890 msgr2=0x55a0b2144400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:47.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:47.962+0000 7f1c50830740 1 --2- 172.21.15.45:0/1859313251 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a0b2146890 0x55a0b2144400 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f1c4000ef30 tx=0x7f1c4000ef60 comp rx=0 tx=0).stop 2024-04-16T17:10:47.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:47.962+0000 7f1c50830740 1 -- 172.21.15.45:0/1859313251 shutdown_connections 2024-04-16T17:10:47.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:47.962+0000 7f1c50830740 1 --2- 172.21.15.45:0/1859313251 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a0b2146890 0x55a0b2144400 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:47.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:47.962+0000 7f1c50830740 1 --2- 172.21.15.45:0/1859313251 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a0b213dcb0 0x55a0b2143ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:47.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:47.962+0000 7f1c50830740 1 --2- 172.21.15.45:0/1859313251 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55a0b2156c10 0x55a0b2159000 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:47.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:47.962+0000 7f1c30c00640 0 client.16551 ms_handle_reset on v2:172.21.15.104:6838/3660070592 2024-04-16T17:10:47.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:47.962+0000 7f1c50830740 1 --2- 172.21.15.45:0/1859313251 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a0b213d390 0x55a0b2143980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:47.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:47.962+0000 7f1c50830740 1 -- 172.21.15.45:0/1859313251 >> 172.21.15.45:0/1859313251 conn(0x55a0b2137ac0 msgr2=0x55a0b2139810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:10:47.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:47.963+0000 7f1c50830740 1 -- 172.21.15.45:0/1859313251 shutdown_connections 2024-04-16T17:10:47.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:47.963+0000 7f1c50830740 1 -- 172.21.15.45:0/1859313251 wait complete. 2024-04-16T17:10:47.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:47.963+0000 7f1c50830740 20 client.16551 trim_cache size 0 max 16384 2024-04-16T17:10:47.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:47.981+0000 7f1c50830740 1 Processor -- start 2024-04-16T17:10:47.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:47.981+0000 7f1c50830740 1 -- start start 2024-04-16T17:10:47.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:47.982+0000 7f1c50830740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a0b213d390 0x55a0b21460b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:47.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:47.982+0000 7f1c50830740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a0b213dcb0 0x55a0b2152540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:47.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:47.982+0000 7f1c50830740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a0b2146890 0x55a0b2142ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:47.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:47.982+0000 7f1c50830740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f1c481b3f50 con 0x55a0b2146890 2024-04-16T17:10:47.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:47.982+0000 7f1c50830740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f1c481b1760 con 0x55a0b213d390 2024-04-16T17:10:47.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:47.982+0000 7f1c50830740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f1c481af590 con 0x55a0b213dcb0 2024-04-16T17:10:47.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:47.982+0000 7f1c47400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a0b213d390 0x55a0b21460b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:47.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:47.982+0000 7f1c47400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a0b213d390 0x55a0b21460b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:44694/0 (socket says 172.21.15.45:44694) 2024-04-16T17:10:47.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:47.982+0000 7f1c47400640 1 -- 172.21.15.45:0/3231694835 learned_addr learned my addr 172.21.15.45:0/3231694835 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:10:47.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:47.982+0000 7f1c46000640 1 --2- 172.21.15.45:0/3231694835 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a0b213dcb0 0x55a0b2152540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:47.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:47.982+0000 7f1c47400640 1 --2- 172.21.15.45:0/3231694835 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a0b2146890 0x55a0b2142ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:47.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:47.983+0000 7f1c47400640 1 -- 172.21.15.45:0/3231694835 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a0b213dcb0 msgr2=0x55a0b2152540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:10:47.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:47.983+0000 7f1c47400640 1 --2- 172.21.15.45:0/3231694835 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a0b213dcb0 0x55a0b2152540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:47.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:47.983+0000 7f1c47400640 1 -- 172.21.15.45:0/3231694835 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a0b2146890 msgr2=0x55a0b2142ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:10:47.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:47.983+0000 7f1c47400640 1 --2- 172.21.15.45:0/3231694835 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a0b2146890 0x55a0b2142ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:47.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:47.983+0000 7f1c47400640 1 -- 172.21.15.45:0/3231694835 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a0b21497a0 con 0x55a0b213d390 2024-04-16T17:10:47.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:47.983+0000 7f1c47400640 1 --2- 172.21.15.45:0/3231694835 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a0b213d390 0x55a0b21460b0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f1c4000ecd0 tx=0x7f1c4000a5c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=330981b69e2099e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:10:47.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:47.983+0000 7f1c33400640 1 -- 172.21.15.45:0/3231694835 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c4003c020 con 0x55a0b213d390 2024-04-16T17:10:47.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:47.983+0000 7f1c33400640 1 -- 172.21.15.45:0/3231694835 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1c40014400 con 0x55a0b213d390 2024-04-16T17:10:47.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:47.983+0000 7f1c33400640 1 -- 172.21.15.45:0/3231694835 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c40018040 con 0x55a0b213d390 2024-04-16T17:10:47.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:47.984+0000 7f1c50830740 1 -- 172.21.15.45:0/3231694835 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a0b213d390 msgr2=0x55a0b21460b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:47.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:47.984+0000 7f1c50830740 1 --2- 172.21.15.45:0/3231694835 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a0b213d390 0x55a0b21460b0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f1c4000ecd0 tx=0x7f1c4000a5c0 comp rx=0 tx=0).stop 2024-04-16T17:10:47.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:47.985+0000 7f1c50830740 1 -- 172.21.15.45:0/3231694835 shutdown_connections 2024-04-16T17:10:47.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:47.985+0000 7f1c50830740 1 --2- 172.21.15.45:0/3231694835 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a0b213dcb0 0x55a0b2152540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:47.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:47.985+0000 7f1c50830740 1 --2- 172.21.15.45:0/3231694835 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a0b2146890 0x55a0b2142ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:47.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:47.985+0000 7f1c50830740 1 --2- 172.21.15.45:0/3231694835 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a0b213d390 0x55a0b21460b0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:47.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:47.985+0000 7f1c50830740 1 -- 172.21.15.45:0/3231694835 >> 172.21.15.45:0/3231694835 conn(0x55a0b2137ac0 msgr2=0x55a0b2136290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:10:47.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:47.985+0000 7f1c50830740 1 -- 172.21.15.45:0/3231694835 shutdown_connections 2024-04-16T17:10:47.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:47.985+0000 7f1c50830740 1 -- 172.21.15.45:0/3231694835 wait complete. 2024-04-16T17:10:47.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:47.985+0000 7f1c50830740 1 -- start start 2024-04-16T17:10:47.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:47.986+0000 7f1c50830740 1 Processor -- start 2024-04-16T17:10:47.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:47.986+0000 7f1c50830740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a0b213d390 0x55a0b214ed20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:47.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:47.986+0000 7f1c50830740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a0b213dcb0 0x55a0b2140230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:47.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:47.986+0000 7f1c50830740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a0b2146890 0x55a0b2140770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:47.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:47.986+0000 7f1c50830740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f1c481afc20 con 0x55a0b213d390 2024-04-16T17:10:47.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:47.986+0000 7f1c50830740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f1c481b1090 con 0x55a0b2146890 2024-04-16T17:10:47.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:47.986+0000 7f1c50830740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f1c481b02f0 con 0x55a0b213dcb0 2024-04-16T17:10:47.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:47.987+0000 7f1c50830740 10 client.0 _ll_register_callbacks cb 0x7f1c38005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:10:47.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:47.987+0000 7f1c46000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a0b213dcb0 0x55a0b2140230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:47.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:47.987+0000 7f1c46000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a0b213dcb0 0x55a0b2140230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:35120/0 (socket says 172.21.15.45:35120) 2024-04-16T17:10:47.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:47.987+0000 7f1c46000640 1 -- 172.21.15.45:0/3226739807 learned_addr learned my addr 172.21.15.45:0/3226739807 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:10:47.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:47.987+0000 7f1c47400640 1 --2- 172.21.15.45:0/3226739807 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a0b213d390 0x55a0b214ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:47.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:47.987+0000 7f1c47400640 1 --2- 172.21.15.45:0/3226739807 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a0b2146890 0x55a0b2140770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:47.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:47.987+0000 7f1c46000640 1 -- 172.21.15.45:0/3226739807 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a0b2146890 msgr2=0x55a0b2140770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:10:47.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:47.987+0000 7f1c46000640 1 --2- 172.21.15.45:0/3226739807 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a0b2146890 0x55a0b2140770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:47.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:47.987+0000 7f1c46000640 1 -- 172.21.15.45:0/3226739807 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a0b213d390 msgr2=0x55a0b214ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:10:47.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:47.987+0000 7f1c46000640 1 --2- 172.21.15.45:0/3226739807 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a0b213d390 0x55a0b214ed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:47.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:47.987+0000 7f1c46000640 1 -- 172.21.15.45:0/3226739807 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a0b2140e30 con 0x55a0b213dcb0 2024-04-16T17:10:47.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:47.988+0000 7f1c47400640 1 --2- 172.21.15.45:0/3226739807 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a0b2146890 0x55a0b2140770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:10:47.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:47.988+0000 7f1c46000640 1 --2- 172.21.15.45:0/3226739807 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a0b213dcb0 0x55a0b2140230 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f1c34009230 tx=0x7f1c34004370 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ee42e7cb1bf7a836 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:10:47.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:47.988+0000 7f1c30c00640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-04-16T17:10:47.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:47.988+0000 7f1c30c00640 1 -- 172.21.15.45:0/3226739807 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c3402ccb0 con 0x55a0b213dcb0 2024-04-16T17:10:47.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:47.988+0000 7f1c30c00640 1 -- 172.21.15.45:0/3226739807 <== mon.1 v2:172.21.15.117:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f1c34005040 con 0x55a0b213dcb0 2024-04-16T17:10:47.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:47.988+0000 7f1c50830740 1 -- 172.21.15.45:0/3226739807 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1c40014400 con 0x55a0b213dcb0 2024-04-16T17:10:47.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:47.989+0000 7f1c30c00640 1 -- 172.21.15.45:0/3226739807 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f1c340044a0 con 0x55a0b213dcb0 2024-04-16T17:10:47.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:47.989+0000 7f1c50830740 10 client.26668 fetch_fsmap learned FSMap version 31 2024-04-16T17:10:47.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:47.989+0000 7f1c50830740 1 -- 172.21.15.45:0/3226739807 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55a0b21bcd80 con 0x55a0b213dcb0 2024-04-16T17:10:47.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:47.989+0000 7f1c30c00640 1 -- 172.21.15.45:0/3226739807 <== mon.1 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1c40014400 con 0x55a0b213dcb0 2024-04-16T17:10:47.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:47.989+0000 7f1c30c00640 1 -- 172.21.15.45:0/3226739807 <== mon.1 v2:172.21.15.117:3300/0 5 ==== fsmap(e 31) ==== 8366+0+0 (secure 0 0 0) 0x7f1c340035f0 con 0x55a0b213dcb0 2024-04-16T17:10:47.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:47.989+0000 7f1c50830740 10 client.26668 fetch_fsmap finished waiting for FSMap version 31 2024-04-16T17:10:47.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:47.989+0000 7f1c50830740 10 client.26668 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:10:47.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:47.990+0000 7f1c50830740 20 client.26668 populate_metadata read hostname 'smithi045' 2024-04-16T17:10:47.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:47.990+0000 7f1c50830740 1 --2- 172.21.15.45:0/3226739807 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55a0b21bd530 0x55a0b21bf950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:47.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:47.990+0000 7f1c50830740 4 client.26668 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-16T17:10:47.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:47.990+0000 7f1c50830740 1 -- 172.21.15.45:0/3226739807 --> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55a0b20aa9b0 con 0x55a0b21bd530 2024-04-16T17:10:47.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:47.990+0000 7f1c47400640 1 --2- 172.21.15.45:0/3226739807 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55a0b21bd530 0x55a0b21bf950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:47.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:47.990+0000 7f1c47400640 1 --2- 172.21.15.45:0/3226739807 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55a0b21bd530 0x55a0b21bf950 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=96f31c5389c75104 server_cookie=84c13358bd2174cc in_seq=0 out_seq=0 2024-04-16T17:10:47.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:47.990+0000 7f1c30c00640 10 client.26668 ms_handle_connect on v2:172.21.15.104:6838/3660070592 2024-04-16T17:10:47.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:47.991+0000 7f1c30c00640 1 -- 172.21.15.45:0/3226739807 <== mds.0 v2:172.21.15.104:6838/3660070592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55a0b20aa9b0 con 0x55a0b21bd530 2024-04-16T17:10:47.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:47.991+0000 7f1c30c00640 10 client.26668 handle_command_reply: tid=0 2024-04-16T17:10:47.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:47.991+0000 7f1c50830740 1 client.26668 shutdown 2024-04-16T17:10:47.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:47.991+0000 7f1c50830740 1 -- 172.21.15.45:0/3226739807 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a0b213dcb0 msgr2=0x55a0b2140230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:47.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:47.991+0000 7f1c50830740 1 --2- 172.21.15.45:0/3226739807 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a0b213dcb0 0x55a0b2140230 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f1c34009230 tx=0x7f1c34004370 comp rx=0 tx=0).stop 2024-04-16T17:10:47.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:47.991+0000 7f1c50830740 1 -- 172.21.15.45:0/3226739807 shutdown_connections 2024-04-16T17:10:47.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:47.991+0000 7f1c50830740 1 --2- 172.21.15.45:0/3226739807 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55a0b21bd530 0x55a0b21bf950 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:47.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:47.991+0000 7f1c50830740 1 --2- 172.21.15.45:0/3226739807 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a0b213dcb0 0x55a0b2140230 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:47.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:47.991+0000 7f1c50830740 1 --2- 172.21.15.45:0/3226739807 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a0b2146890 0x55a0b2140770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:47.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:47.991+0000 7f1c50830740 1 --2- 172.21.15.45:0/3226739807 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a0b213d390 0x55a0b214ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:47.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:47.991+0000 7f1c50830740 1 -- 172.21.15.45:0/3226739807 >> 172.21.15.45:0/3226739807 conn(0x55a0b2137ac0 msgr2=0x55a0b2136290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:10:47.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:47.991+0000 7f1c30c00640 0 client.26668 ms_handle_reset on v2:172.21.15.104:6838/3660070592 2024-04-16T17:10:47.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:47.991+0000 7f1c50830740 1 -- 172.21.15.45:0/3226739807 shutdown_connections 2024-04-16T17:10:47.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:47.991+0000 7f1c50830740 1 -- 172.21.15.45:0/3226739807 wait complete. 2024-04-16T17:10:47.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:47.991+0000 7f1c50830740 20 client.26668 trim_cache size 0 max 16384 2024-04-16T17:10:47.995 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-16T17:10:47.995 INFO:teuthology.orchestra.run.smithi045.stdout: "status": "no active scrubs running", 2024-04-16T17:10:47.995 INFO:teuthology.orchestra.run.smithi045.stdout: "scrubs": {} 2024-04-16T17:10:47.995 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-16T17:10:47.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:47.992+0000 7f1c47e00640 1 -- 172.21.15.45:0/1216850763 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f1c2c0a3d20 msgr2=0x7f1c2c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:47.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:47.992+0000 7f1c47e00640 1 --2- 172.21.15.45:0/1216850763 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f1c2c0a3d20 0x7f1c2c0a61e0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x55a0b213ce60 tx=0x7f1c3800cf20 comp rx=0 tx=0).stop 2024-04-16T17:10:47.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:47.992+0000 7f1c47e00640 1 -- 172.21.15.45:0/1216850763 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1c48102110 msgr2=0x7f1c4819e5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:47.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:47.992+0000 7f1c47e00640 1 --2- 172.21.15.45:0/1216850763 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1c48102110 0x7f1c4819e5a0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f1c3800e9f0 tx=0x7f1c3800eeb0 comp rx=0 tx=0).stop 2024-04-16T17:10:47.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:47.992+0000 7f1c47e00640 1 -- 172.21.15.45:0/1216850763 shutdown_connections 2024-04-16T17:10:47.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:47.992+0000 7f1c47e00640 1 --2- 172.21.15.45:0/1216850763 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f1c2c0a3d20 0x7f1c2c0a61e0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:47.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:47.992+0000 7f1c47e00640 1 --2- 172.21.15.45:0/1216850763 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1c481071e0 0x7f1c4819eae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:47.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:47.992+0000 7f1c47e00640 1 --2- 172.21.15.45:0/1216850763 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1c48198660 0x7f1c48198ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:47.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:47.992+0000 7f1c47e00640 1 --2- 172.21.15.45:0/1216850763 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1c48102110 0x7f1c4819e5a0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:47.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:47.992+0000 7f1c47e00640 1 -- 172.21.15.45:0/1216850763 >> 172.21.15.45:0/1216850763 conn(0x7f1c480fd9d0 msgr2=0x7f1c480ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:10:47.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:47.992+0000 7f1c47e00640 1 -- 172.21.15.45:0/1216850763 shutdown_connections 2024-04-16T17:10:47.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:47.992+0000 7f1c47e00640 1 -- 172.21.15.45:0/1216850763 wait complete. 2024-04-16T17:10:48.009 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-16T17:10:48.010 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-16T17:10:48.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.220+0000 7f0048e00640 1 -- 172.21.15.45:0/3135231161 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0044076530 msgr2=0x7f0044076910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:48.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.220+0000 7f0048e00640 1 --2- 172.21.15.45:0/3135231161 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0044076530 0x7f0044076910 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f0034009860 tx=0x7f003402bb70 comp rx=0 tx=0).stop 2024-04-16T17:10:48.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.222+0000 7f0048e00640 1 -- 172.21.15.45:0/3135231161 shutdown_connections 2024-04-16T17:10:48.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.222+0000 7f0048e00640 1 --2- 172.21.15.45:0/3135231161 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0044112ca0 0x7f0044115090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:48.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.222+0000 7f0048e00640 1 --2- 172.21.15.45:0/3135231161 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0044076e50 0x7f0044112760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:48.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.222+0000 7f0048e00640 1 --2- 172.21.15.45:0/3135231161 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0044076530 0x7f0044076910 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:48.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.222+0000 7f0048e00640 1 -- 172.21.15.45:0/3135231161 >> 172.21.15.45:0/3135231161 conn(0x7f00440ffad0 msgr2=0x7f0044101ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:10:48.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.222+0000 7f0048e00640 1 -- 172.21.15.45:0/3135231161 shutdown_connections 2024-04-16T17:10:48.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.222+0000 7f0048e00640 1 -- 172.21.15.45:0/3135231161 wait complete. 2024-04-16T17:10:48.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.222+0000 7f0048e00640 1 Processor -- start 2024-04-16T17:10:48.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.222+0000 7f0048e00640 1 -- start start 2024-04-16T17:10:48.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.222+0000 7f0048e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0044076530 0x7f004419edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:48.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.222+0000 7f0048e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0044076e50 0x7f004419f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:48.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.222+0000 7f0048e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0044112ca0 0x7f00441a3620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:48.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.222+0000 7f0048e00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f00441187f0 con 0x7f0044112ca0 2024-04-16T17:10:48.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.222+0000 7f0048e00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f0044118670 con 0x7f0044076530 2024-04-16T17:10:48.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.222+0000 7f0048e00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f0044118970 con 0x7f0044076e50 2024-04-16T17:10:48.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.222+0000 7f0043400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0044076530 0x7f004419edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:48.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.222+0000 7f0043400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0044076530 0x7f004419edb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:44736/0 (socket says 172.21.15.45:44736) 2024-04-16T17:10:48.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.222+0000 7f0043400640 1 -- 172.21.15.45:0/636722744 learned_addr learned my addr 172.21.15.45:0/636722744 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:10:48.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.222+0000 7f0043e00640 1 --2- 172.21.15.45:0/636722744 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0044112ca0 0x7f00441a3620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:48.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.222+0000 7f0043400640 1 -- 172.21.15.45:0/636722744 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0044076e50 msgr2=0x7f004419f2f0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-16T17:10:48.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.222+0000 7f0043400640 1 --2- 172.21.15.45:0/636722744 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0044076e50 0x7f004419f2f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:48.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.222+0000 7f0043400640 1 -- 172.21.15.45:0/636722744 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0044112ca0 msgr2=0x7f00441a3620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:48.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.222+0000 7f0043400640 1 --2- 172.21.15.45:0/636722744 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0044112ca0 0x7f00441a3620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:48.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.222+0000 7f0043400640 1 -- 172.21.15.45:0/636722744 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00441a3d50 con 0x7f0044076530 2024-04-16T17:10:48.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.222+0000 7f0043400640 1 --2- 172.21.15.45:0/636722744 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0044076530 0x7f004419edb0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f0034037040 tx=0x7f00340323f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:10:48.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.222+0000 7f003be00640 1 -- 172.21.15.45:0/636722744 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0034041020 con 0x7f0044076530 2024-04-16T17:10:48.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.222+0000 7f003be00640 1 -- 172.21.15.45:0/636722744 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f003402c9f0 con 0x7f0044076530 2024-04-16T17:10:48.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.222+0000 7f003be00640 1 -- 172.21.15.45:0/636722744 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0034039040 con 0x7f0044076530 2024-04-16T17:10:48.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.222+0000 7f0048e00640 1 -- 172.21.15.45:0/636722744 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00441a3fe0 con 0x7f0044076530 2024-04-16T17:10:48.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.222+0000 7f0048e00640 1 -- 172.21.15.45:0/636722744 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00441ab8d0 con 0x7f0044076530 2024-04-16T17:10:48.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.222+0000 7f0048e00640 1 -- 172.21.15.45:0/636722744 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0044069190 con 0x7f0044076530 2024-04-16T17:10:48.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.227+0000 7f003be00640 1 -- 172.21.15.45:0/636722744 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f0034044020 con 0x7f0044076530 2024-04-16T17:10:48.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.232+0000 7f003be00640 1 --2- 172.21.15.45:0/636722744 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f00280a3da0 0x7f00280a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:48.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.232+0000 7f003be00640 1 -- 172.21.15.45:0/636722744 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f00340f2340 con 0x7f0044076530 2024-04-16T17:10:48.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.232+0000 7f0042a00640 1 --2- 172.21.15.45:0/636722744 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f00280a3da0 0x7f00280a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:48.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.232+0000 7f0042a00640 1 --2- 172.21.15.45:0/636722744 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f00280a3da0 0x7f00280a6260 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f00300058c0 tx=0x7f0030005850 comp rx=0 tx=0).ready entity=mgr.25411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:10:48.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.236+0000 7f003be00640 1 -- 172.21.15.45:0/636722744 <== mon.2 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f003404b020 con 0x7f0044076530 2024-04-16T17:10:48.246 INFO:tasks.workunit.client.0.smithi045.stdout: 1 54417 1.21 MB/sec execute 50 sec latency 3647.850 ms 2024-04-16T17:10:48.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.489+0000 7f0048e00640 1 -- 172.21.15.45:0/636722744 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f004410de30 con 0x7f0044076530 2024-04-16T17:10:48.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.491+0000 7f003be00640 1 -- 172.21.15.45:0/636722744 <== mon.2 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) ==== 94+0+11894 (secure 0 0 0) 0x7f00340be740 con 0x7f0044076530 2024-04-16T17:10:48.491 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-16T17:10:48.491 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":31,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/1748842443","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":1748842443},{"type":"v1","addr":"172.21.15.45:6835","nonce":1748842443}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6839/1308627051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":1308627051},{"type":"v1","addr":"172.21.15.45:6839","nonce":1308627051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/1768242489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1768242489},{"type":"v1","addr":"172.21.15.117:6833","nonce":1768242489}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6833/3558902585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":3558902585},{"type":"v1","addr":"172.21.15.104:6833","nonce":3558902585}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24478,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/1308142903","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":1308142903},{"type":"v1","addr":"172.21.15.45:6837","nonce":1308142903}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24491,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6835/3430679937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":3430679937},{"type":"v1","addr":"172.21.15.104:6835","nonce":3430679937}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":31,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-16T17:03:24.663611+0000","modified":"2024-04-16T17:10:43.520749+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24581,"mds_1":24569,"mds_2":24532},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24499":{"gid":24499,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/2224123841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2224123841},{"type":"v1","addr":"172.21.15.117:6835","nonce":2224123841}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6837/2471356815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":2471356815},{"type":"v1","addr":"172.21.15.117:6837","nonce":2471356815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24524":{"gid":24524,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6837/2647756509","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":2647756509},{"type":"v1","addr":"172.21.15.104:6837","nonce":2647756509}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24532":{"gid":24532,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":13,"addr":"172.21.15.117:6839/2378675949","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":2378675949},{"type":"v1","addr":"172.21.15.117:6839","nonce":2378675949}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24569":{"gid":24569,"name":"a","rank":1,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.45:6841/3755310822","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6840","nonce":3755310822},{"type":"v1","addr":"172.21.15.45:6841","nonce":3755310822}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24581":{"gid":24581,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.104:6839/3660070592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6838","nonce":3660070592},{"type":"v1","addr":"172.21.15.104:6839","nonce":3660070592}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24581,"qdb_cluster":[24569,24581,24532]},"id":1}]} 2024-04-16T17:10:48.499 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 31 2024-04-16T17:10:48.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.493+0000 7f0048e00640 1 -- 172.21.15.45:0/636722744 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f00280a3da0 msgr2=0x7f00280a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:48.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.493+0000 7f0048e00640 1 --2- 172.21.15.45:0/636722744 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f00280a3da0 0x7f00280a6260 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f00300058c0 tx=0x7f0030005850 comp rx=0 tx=0).stop 2024-04-16T17:10:48.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.493+0000 7f0048e00640 1 -- 172.21.15.45:0/636722744 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0044076530 msgr2=0x7f004419edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:48.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.493+0000 7f0048e00640 1 --2- 172.21.15.45:0/636722744 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0044076530 0x7f004419edb0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f0034037040 tx=0x7f00340323f0 comp rx=0 tx=0).stop 2024-04-16T17:10:48.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.493+0000 7f0048e00640 1 -- 172.21.15.45:0/636722744 shutdown_connections 2024-04-16T17:10:48.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.493+0000 7f0048e00640 1 --2- 172.21.15.45:0/636722744 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f00280a3da0 0x7f00280a6260 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:48.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.493+0000 7f0048e00640 1 --2- 172.21.15.45:0/636722744 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0044076e50 0x7f004419f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:48.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.493+0000 7f0048e00640 1 --2- 172.21.15.45:0/636722744 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0044112ca0 0x7f00441a3620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:48.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.493+0000 7f0048e00640 1 --2- 172.21.15.45:0/636722744 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0044076530 0x7f004419edb0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:48.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.493+0000 7f0048e00640 1 -- 172.21.15.45:0/636722744 >> 172.21.15.45:0/636722744 conn(0x7f00440ffad0 msgr2=0x7f0044114390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:10:48.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.493+0000 7f0048e00640 1 -- 172.21.15.45:0/636722744 shutdown_connections 2024-04-16T17:10:48.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.493+0000 7f0048e00640 1 -- 172.21.15.45:0/636722744 wait complete. 2024-04-16T17:10:48.523 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-16T17:10:48.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.706+0000 7f4eea400640 1 -- 172.21.15.45:0/2990257747 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4ee4103980 msgr2=0x7f4ee410be60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:48.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.706+0000 7f4eea400640 1 --2- 172.21.15.45:0/2990257747 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4ee4103980 0x7f4ee410be60 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f4ed0009900 tx=0x7f4ed002bce0 comp rx=0 tx=0).stop 2024-04-16T17:10:48.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.706+0000 7f4eea400640 1 -- 172.21.15.45:0/2990257747 shutdown_connections 2024-04-16T17:10:48.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.706+0000 7f4eea400640 1 --2- 172.21.15.45:0/2990257747 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4ee410c480 0x7f4ee410e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:48.708 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.706+0000 7f4eea400640 1 --2- 172.21.15.45:0/2990257747 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4ee4103980 0x7f4ee410be60 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:48.708 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.706+0000 7f4eea400640 1 --2- 172.21.15.45:0/2990257747 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4ee4103060 0x7f4ee4103440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:48.708 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.706+0000 7f4eea400640 1 -- 172.21.15.45:0/2990257747 >> 172.21.15.45:0/2990257747 conn(0x7f4ee40fc880 msgr2=0x7f4ee40feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:10:48.708 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.706+0000 7f4eea400640 1 -- 172.21.15.45:0/2990257747 shutdown_connections 2024-04-16T17:10:48.708 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.706+0000 7f4eea400640 1 -- 172.21.15.45:0/2990257747 wait complete. 2024-04-16T17:10:48.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.707+0000 7f4eea400640 1 Processor -- start 2024-04-16T17:10:48.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.707+0000 7f4eea400640 1 -- start start 2024-04-16T17:10:48.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.708+0000 7f4eea400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4ee4103060 0x7f4ee419ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:48.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.708+0000 7f4eea400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4ee4103980 0x7f4ee419b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:48.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.708+0000 7f4eea400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4ee410c480 0x7f4ee419f420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:48.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.708+0000 7f4eea400640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f4ee4112330 con 0x7f4ee4103980 2024-04-16T17:10:48.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.708+0000 7f4eea400640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f4ee41121b0 con 0x7f4ee410c480 2024-04-16T17:10:48.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.708+0000 7f4eea400640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f4ee41124b0 con 0x7f4ee4103060 2024-04-16T17:10:48.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.708+0000 7f4ee9a00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4ee410c480 0x7f4ee419f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:48.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.708+0000 7f4ee3e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4ee4103980 0x7f4ee419b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:48.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.708+0000 7f4ee9a00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4ee410c480 0x7f4ee419f420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:44746/0 (socket says 172.21.15.45:44746) 2024-04-16T17:10:48.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.708+0000 7f4ee9a00640 1 -- 172.21.15.45:0/3052058602 learned_addr learned my addr 172.21.15.45:0/3052058602 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:10:48.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.709+0000 7f4ee9000640 1 --2- 172.21.15.45:0/3052058602 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4ee4103060 0x7f4ee419ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:48.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.709+0000 7f4ee9a00640 1 -- 172.21.15.45:0/3052058602 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4ee4103060 msgr2=0x7f4ee419ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:48.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.709+0000 7f4ee9a00640 1 --2- 172.21.15.45:0/3052058602 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4ee4103060 0x7f4ee419ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:48.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.709+0000 7f4ee9a00640 1 -- 172.21.15.45:0/3052058602 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4ee4103980 msgr2=0x7f4ee419b070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:48.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.709+0000 7f4ee9a00640 1 --2- 172.21.15.45:0/3052058602 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4ee4103980 0x7f4ee419b070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:48.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.709+0000 7f4ee9a00640 1 -- 172.21.15.45:0/3052058602 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ee419fb50 con 0x7f4ee410c480 2024-04-16T17:10:48.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.709+0000 7f4ee9000640 1 --2- 172.21.15.45:0/3052058602 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4ee4103060 0x7f4ee419ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:10:48.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.709+0000 7f4ee9a00640 1 --2- 172.21.15.45:0/3052058602 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4ee410c480 0x7f4ee419f420 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f4edc00eb10 tx=0x7f4edc00c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:10:48.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.709+0000 7f4ee1600640 1 -- 172.21.15.45:0/3052058602 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4edc004b20 con 0x7f4ee410c480 2024-04-16T17:10:48.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.710+0000 7f4ee1600640 1 -- 172.21.15.45:0/3052058602 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4edc004cc0 con 0x7f4ee410c480 2024-04-16T17:10:48.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.710+0000 7f4ee1600640 1 -- 172.21.15.45:0/3052058602 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4edc010760 con 0x7f4ee410c480 2024-04-16T17:10:48.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.710+0000 7f4eea400640 1 -- 172.21.15.45:0/3052058602 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ee419fe40 con 0x7f4ee410c480 2024-04-16T17:10:48.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.710+0000 7f4eea400640 1 -- 172.21.15.45:0/3052058602 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ee41a76d0 con 0x7f4ee410c480 2024-04-16T17:10:48.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.710+0000 7f4eee77a740 1 Processor -- start 2024-04-16T17:10:48.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.710+0000 7f4eee77a740 1 -- start start 2024-04-16T17:10:48.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.710+0000 7f4eee77a740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561a3ad2df10 0x561a3ad2e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:48.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.710+0000 7f4eee77a740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561a3ad2e830 0x561a3ad36ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:48.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.710+0000 7f4eee77a740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561a3ad37410 0x561a3ad39800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:48.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.710+0000 7f4eee77a740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x561a3ac93020 con 0x561a3ad2e830 2024-04-16T17:10:48.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.710+0000 7f4eee77a740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x561a3ac39460 con 0x561a3ad37410 2024-04-16T17:10:48.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.710+0000 7f4eee77a740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x561a3ac8efe0 con 0x561a3ad2df10 2024-04-16T17:10:48.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.715+0000 7f4ee9000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561a3ad2e830 0x561a3ad36ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:48.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.715+0000 7f4ee9000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561a3ad2e830 0x561a3ad36ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:46462/0 (socket says 172.21.15.45:46462) 2024-04-16T17:10:48.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.715+0000 7f4ee9000640 1 -- 172.21.15.45:0/562102654 learned_addr learned my addr 172.21.15.45:0/562102654 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:10:48.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.715+0000 7f4ee3e00640 1 --2- 172.21.15.45:0/562102654 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561a3ad2df10 0x561a3ad2e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:48.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.715+0000 7f4ee3e00640 1 --2- 172.21.15.45:0/562102654 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561a3ad37410 0x561a3ad39800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:48.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.715+0000 7f4ee9000640 1 -- 172.21.15.45:0/562102654 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561a3ad2df10 msgr2=0x561a3ad2e2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:10:48.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.715+0000 7f4ee9000640 1 --2- 172.21.15.45:0/562102654 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561a3ad2df10 0x561a3ad2e2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:48.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.715+0000 7f4ee9000640 1 -- 172.21.15.45:0/562102654 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561a3ad37410 msgr2=0x561a3ad39800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:10:48.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.715+0000 7f4ee9000640 1 --2- 172.21.15.45:0/562102654 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561a3ad37410 0x561a3ad39800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:48.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.715+0000 7f4ee9000640 1 -- 172.21.15.45:0/562102654 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561a3ac41040 con 0x561a3ad2e830 2024-04-16T17:10:48.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.715+0000 7f4ee3e00640 1 --2- 172.21.15.45:0/562102654 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561a3ad2df10 0x561a3ad2e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:10:48.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.715+0000 7f4ee9000640 1 --2- 172.21.15.45:0/562102654 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561a3ad2e830 0x561a3ad36ed0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f4ed8007820 tx=0x7f4ed8007ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55ad6f0783f44d26 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:10:48.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.715+0000 7f4ed6000640 1 -- 172.21.15.45:0/562102654 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ed800d950 con 0x561a3ad2e830 2024-04-16T17:10:48.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.715+0000 7f4ed6000640 1 -- 172.21.15.45:0/562102654 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ed8018070 con 0x561a3ad2e830 2024-04-16T17:10:48.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.715+0000 7f4ed6000640 1 -- 172.21.15.45:0/562102654 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ed80123e0 con 0x561a3ad2e830 2024-04-16T17:10:48.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.715+0000 7f4eee77a740 1 -- 172.21.15.45:0/562102654 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561a3ad2e830 msgr2=0x561a3ad36ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:48.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.715+0000 7f4eee77a740 1 --2- 172.21.15.45:0/562102654 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561a3ad2e830 0x561a3ad36ed0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f4ed8007820 tx=0x7f4ed8007ce0 comp rx=0 tx=0).stop 2024-04-16T17:10:48.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.715+0000 7f4eee77a740 1 -- 172.21.15.45:0/562102654 shutdown_connections 2024-04-16T17:10:48.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.715+0000 7f4eee77a740 1 --2- 172.21.15.45:0/562102654 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561a3ad37410 0x561a3ad39800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:48.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.715+0000 7f4eee77a740 1 --2- 172.21.15.45:0/562102654 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561a3ad2e830 0x561a3ad36ed0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:48.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.715+0000 7f4eee77a740 1 --2- 172.21.15.45:0/562102654 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561a3ad2df10 0x561a3ad2e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:48.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.715+0000 7f4eee77a740 1 -- 172.21.15.45:0/562102654 >> 172.21.15.45:0/562102654 conn(0x561a3ad28640 msgr2=0x561a3ad28a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:10:48.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.715+0000 7f4eee77a740 1 -- 172.21.15.45:0/562102654 shutdown_connections 2024-04-16T17:10:48.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.715+0000 7f4eee77a740 1 -- 172.21.15.45:0/562102654 wait complete. 2024-04-16T17:10:48.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.715+0000 7f4eee77a740 1 -- start start 2024-04-16T17:10:48.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.719+0000 7f4eee77a740 1 Processor -- start 2024-04-16T17:10:48.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.719+0000 7f4eee77a740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561a3ad2df10 0x561a3ad34450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:48.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.719+0000 7f4eee77a740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561a3ad37410 0x561a3ad34990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:48.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.719+0000 7f4eee77a740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561a3ad34ed0 0x561a3ad436d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:48.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.719+0000 7f4eee77a740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x561a3ad3a200 con 0x561a3ad37410 2024-04-16T17:10:48.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.719+0000 7f4eee77a740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x561a3ac4c830 con 0x561a3ad2df10 2024-04-16T17:10:48.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.719+0000 7f4eee77a740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x561a3ad3a380 con 0x561a3ad34ed0 2024-04-16T17:10:48.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.719+0000 7f4eee77a740 10 client.0 _ll_register_callbacks cb 0x561a3ac54150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:10:48.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.719+0000 7f4ee9000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561a3ad37410 0x561a3ad34990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:48.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.719+0000 7f4ee3e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561a3ad2df10 0x561a3ad34450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:48.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.719+0000 7f4ee3e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561a3ad2df10 0x561a3ad34450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:44762/0 (socket says 172.21.15.45:44762) 2024-04-16T17:10:48.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.719+0000 7f4ee3e00640 1 -- 172.21.15.45:0/3014806717 learned_addr learned my addr 172.21.15.45:0/3014806717 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:10:48.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.719+0000 7f4ee9000640 1 -- 172.21.15.45:0/3014806717 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561a3ad34ed0 msgr2=0x561a3ad436d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-16T17:10:48.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.719+0000 7f4ee9000640 1 --2- 172.21.15.45:0/3014806717 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561a3ad34ed0 0x561a3ad436d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:48.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.719+0000 7f4ee9000640 1 -- 172.21.15.45:0/3014806717 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561a3ad2df10 msgr2=0x561a3ad34450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:10:48.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.719+0000 7f4ee9000640 1 --2- 172.21.15.45:0/3014806717 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561a3ad2df10 0x561a3ad34450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:48.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.719+0000 7f4ee9000640 1 -- 172.21.15.45:0/3014806717 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561a3ac50920 con 0x561a3ad37410 2024-04-16T17:10:48.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.719+0000 7f4ee3e00640 1 --2- 172.21.15.45:0/3014806717 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561a3ad2df10 0x561a3ad34450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:10:48.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.719+0000 7f4ee1600640 1 -- 172.21.15.45:0/3052058602 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f4edc005020 con 0x7f4ee410c480 2024-04-16T17:10:48.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.719+0000 7f4ee9000640 1 --2- 172.21.15.45:0/3014806717 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561a3ad37410 0x561a3ad34990 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x561a3ad29540 tx=0x7f4ed800dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7085d41f0272823 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:10:48.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.719+0000 7f4ecaa00640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-16T17:10:48.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.719+0000 7f4ecaa00640 1 -- 172.21.15.45:0/3014806717 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ed8024620 con 0x561a3ad37410 2024-04-16T17:10:48.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.719+0000 7f4ecaa00640 1 -- 172.21.15.45:0/3014806717 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f4ed801c3f0 con 0x561a3ad37410 2024-04-16T17:10:48.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.719+0000 7f4eee77a740 1 -- 172.21.15.45:0/3014806717 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4ed8018070 con 0x561a3ad37410 2024-04-16T17:10:48.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.719+0000 7f4ecaa00640 1 -- 172.21.15.45:0/3014806717 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f4ed80137f0 con 0x561a3ad37410 2024-04-16T17:10:48.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.719+0000 7f4eee77a740 10 client.16575 fetch_fsmap learned FSMap version 31 2024-04-16T17:10:48.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.719+0000 7f4eee77a740 1 -- 172.21.15.45:0/3014806717 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x561a3ac1e7f0 con 0x561a3ad37410 2024-04-16T17:10:48.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.719+0000 7f4ecaa00640 1 -- 172.21.15.45:0/3014806717 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ed8018070 con 0x561a3ad37410 2024-04-16T17:10:48.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.719+0000 7f4ecaa00640 1 -- 172.21.15.45:0/3014806717 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 31) ==== 8366+0+0 (secure 0 0 0) 0x7f4ed802e770 con 0x561a3ad37410 2024-04-16T17:10:48.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.719+0000 7f4eee77a740 10 client.16575 fetch_fsmap finished waiting for FSMap version 31 2024-04-16T17:10:48.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.719+0000 7f4eee77a740 10 client.16575 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-16T17:10:48.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.719+0000 7f4eee77a740 20 client.16575 populate_metadata read hostname 'smithi045' 2024-04-16T17:10:48.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.719+0000 7f4eee77a740 1 --2- 172.21.15.45:0/3014806717 >> [v2:172.21.15.117:6838/2378675949,v1:172.21.15.117:6839/2378675949] conn(0x561a3ad47cc0 0x561a3ad4a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:48.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.719+0000 7f4eee77a740 4 client.16575 mds_command: new command op to 24532 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:10:48.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.719+0000 7f4eee77a740 1 -- 172.21.15.45:0/3014806717 --> [v2:172.21.15.117:6838/2378675949,v1:172.21.15.117:6839/2378675949] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561a3aca0a40 con 0x561a3ad47cc0 2024-04-16T17:10:48.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.719+0000 7f4ee1600640 1 --2- 172.21.15.45:0/3052058602 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f4ecc0a3c40 0x7f4ecc0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:48.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.719+0000 7f4ee1600640 1 -- 172.21.15.45:0/3052058602 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f4edc0d0ff0 con 0x7f4ee410c480 2024-04-16T17:10:48.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.719+0000 7f4ee3e00640 1 --2- 172.21.15.45:0/3014806717 >> [v2:172.21.15.117:6838/2378675949,v1:172.21.15.117:6839/2378675949] conn(0x561a3ad47cc0 0x561a3ad4a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:48.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.719+0000 7f4ee3e00640 1 --2- 172.21.15.45:0/3052058602 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f4ecc0a3c40 0x7f4ecc0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:48.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.724+0000 7f4ee3e00640 1 --2- 172.21.15.45:0/3014806717 >> [v2:172.21.15.117:6838/2378675949,v1:172.21.15.117:6839/2378675949] conn(0x561a3ad47cc0 0x561a3ad4a0b0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4d532c12e999cc68 server_cookie=960ed557f77818e7 in_seq=0 out_seq=0 2024-04-16T17:10:48.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.724+0000 7f4ecaa00640 10 client.16575 ms_handle_connect on v2:172.21.15.117:6838/2378675949 2024-04-16T17:10:48.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.724+0000 7f4ee3e00640 1 --2- 172.21.15.45:0/3052058602 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f4ecc0a3c40 0x7f4ecc0a6100 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f4ed00096c0 tx=0x7f4ed0005a20 comp rx=0 tx=0).ready entity=mgr.25411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:10:48.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.724+0000 7f4ecaa00640 1 -- 172.21.15.45:0/3014806717 <== mds.2 v2:172.21.15.117:6838/2378675949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561a3aca0a40 con 0x561a3ad47cc0 2024-04-16T17:10:48.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.724+0000 7f4ecaa00640 10 client.16575 handle_command_reply: tid=0 2024-04-16T17:10:48.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.724+0000 7f4eee77a740 1 client.16575 shutdown 2024-04-16T17:10:48.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.724+0000 7f4eee77a740 1 -- 172.21.15.45:0/3014806717 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561a3ad37410 msgr2=0x561a3ad34990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:48.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.724+0000 7f4eee77a740 1 --2- 172.21.15.45:0/3014806717 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561a3ad37410 0x561a3ad34990 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x561a3ad29540 tx=0x7f4ed800dc00 comp rx=0 tx=0).stop 2024-04-16T17:10:48.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.724+0000 7f4eee77a740 1 -- 172.21.15.45:0/3014806717 shutdown_connections 2024-04-16T17:10:48.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.724+0000 7f4eee77a740 1 --2- 172.21.15.45:0/3014806717 >> [v2:172.21.15.117:6838/2378675949,v1:172.21.15.117:6839/2378675949] conn(0x561a3ad47cc0 0x561a3ad4a0b0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:48.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.724+0000 7f4eee77a740 1 --2- 172.21.15.45:0/3014806717 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561a3ad34ed0 0x561a3ad436d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:48.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.724+0000 7f4eee77a740 1 --2- 172.21.15.45:0/3014806717 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561a3ad37410 0x561a3ad34990 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:48.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.724+0000 7f4eee77a740 1 --2- 172.21.15.45:0/3014806717 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561a3ad2df10 0x561a3ad34450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:48.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.724+0000 7f4eee77a740 1 -- 172.21.15.45:0/3014806717 >> 172.21.15.45:0/3014806717 conn(0x561a3ad28640 msgr2=0x561a3ad2a390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:10:48.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.724+0000 7f4ecaa00640 0 client.16575 ms_handle_reset on v2:172.21.15.117:6838/2378675949 2024-04-16T17:10:48.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.724+0000 7f4eee77a740 1 -- 172.21.15.45:0/3014806717 shutdown_connections 2024-04-16T17:10:48.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.724+0000 7f4eee77a740 1 -- 172.21.15.45:0/3014806717 wait complete. 2024-04-16T17:10:48.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.727+0000 7f4eee77a740 20 client.16575 trim_cache size 0 max 16384 2024-04-16T17:10:48.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.745+0000 7f4eee77a740 1 Processor -- start 2024-04-16T17:10:48.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.745+0000 7f4eee77a740 1 -- start start 2024-04-16T17:10:48.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.745+0000 7f4eee77a740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561a3ad2df10 0x561a3ad4f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:48.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.745+0000 7f4eee77a740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561a3ad34ed0 0x561a3ad4fce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:48.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.746+0000 7f4eee77a740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561a3ad37410 0x561a3ad43120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:48.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.746+0000 7f4eee77a740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f4ec4012810 con 0x561a3ad34ed0 2024-04-16T17:10:48.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.746+0000 7f4eee77a740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f4ec4010020 con 0x561a3ad37410 2024-04-16T17:10:48.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.746+0000 7f4eee77a740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f4ec400de50 con 0x561a3ad2df10 2024-04-16T17:10:48.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.746+0000 7f4ee3e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561a3ad34ed0 0x561a3ad4fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:48.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.746+0000 7f4ee9000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561a3ad2df10 0x561a3ad4f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:48.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.746+0000 7f4ee9000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561a3ad2df10 0x561a3ad4f7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:35162/0 (socket says 172.21.15.45:35162) 2024-04-16T17:10:48.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.746+0000 7f4ee9000640 1 -- 172.21.15.45:0/3623387942 learned_addr learned my addr 172.21.15.45:0/3623387942 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:10:48.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.746+0000 7f4ee9a00640 1 --2- 172.21.15.45:0/3623387942 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561a3ad37410 0x561a3ad43120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:48.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.746+0000 7f4ee9000640 1 -- 172.21.15.45:0/3623387942 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561a3ad37410 msgr2=0x561a3ad43120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:10:48.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.746+0000 7f4ee9000640 1 --2- 172.21.15.45:0/3623387942 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561a3ad37410 0x561a3ad43120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:48.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.746+0000 7f4ee9000640 1 -- 172.21.15.45:0/3623387942 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561a3ad34ed0 msgr2=0x561a3ad4fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:10:48.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.746+0000 7f4ee9000640 1 --2- 172.21.15.45:0/3623387942 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561a3ad34ed0 0x561a3ad4fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:48.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.746+0000 7f4ee9000640 1 -- 172.21.15.45:0/3623387942 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561a3a9a53f0 con 0x561a3ad2df10 2024-04-16T17:10:48.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.746+0000 7f4ee9000640 1 --2- 172.21.15.45:0/3623387942 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561a3ad2df10 0x561a3ad4f7a0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f4ed802ecb0 tx=0x7f4ed8013b70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=40b6adcc2fbee9f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:10:48.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.746+0000 7f4ed5600640 1 -- 172.21.15.45:0/3623387942 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ed802e480 con 0x561a3ad2df10 2024-04-16T17:10:48.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.746+0000 7f4ed5600640 1 -- 172.21.15.45:0/3623387942 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ed8018040 con 0x561a3ad2df10 2024-04-16T17:10:48.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.746+0000 7f4ed5600640 1 -- 172.21.15.45:0/3623387942 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ed8023d70 con 0x561a3ad2df10 2024-04-16T17:10:48.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.748+0000 7f4eee77a740 1 -- 172.21.15.45:0/3623387942 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561a3ad2df10 msgr2=0x561a3ad4f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:48.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.748+0000 7f4eee77a740 1 --2- 172.21.15.45:0/3623387942 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561a3ad2df10 0x561a3ad4f7a0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f4ed802ecb0 tx=0x7f4ed8013b70 comp rx=0 tx=0).stop 2024-04-16T17:10:48.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.748+0000 7f4eee77a740 1 -- 172.21.15.45:0/3623387942 shutdown_connections 2024-04-16T17:10:48.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.748+0000 7f4eee77a740 1 --2- 172.21.15.45:0/3623387942 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561a3ad37410 0x561a3ad43120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:48.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.748+0000 7f4eee77a740 1 --2- 172.21.15.45:0/3623387942 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561a3ad34ed0 0x561a3ad4fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:48.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.748+0000 7f4eee77a740 1 --2- 172.21.15.45:0/3623387942 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561a3ad2df10 0x561a3ad4f7a0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:48.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.748+0000 7f4eee77a740 1 -- 172.21.15.45:0/3623387942 >> 172.21.15.45:0/3623387942 conn(0x561a3ad28640 msgr2=0x561a3ad32d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:10:48.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.748+0000 7f4eee77a740 1 -- 172.21.15.45:0/3623387942 shutdown_connections 2024-04-16T17:10:48.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.748+0000 7f4eee77a740 1 -- 172.21.15.45:0/3623387942 wait complete. 2024-04-16T17:10:48.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.748+0000 7f4eee77a740 1 -- start start 2024-04-16T17:10:48.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.748+0000 7f4eee77a740 1 Processor -- start 2024-04-16T17:10:48.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.748+0000 7f4eee77a740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561a3ad2df10 0x561a3ad3a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:48.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.748+0000 7f4eee77a740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561a3ad34ed0 0x561a3ad3ab30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:48.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.748+0000 7f4eee77a740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561a3ad37410 0x561a3adac000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:48.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.748+0000 7f4eee77a740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f4ec400e4e0 con 0x561a3ad2df10 2024-04-16T17:10:48.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.748+0000 7f4eee77a740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f4ec400f950 con 0x561a3ad34ed0 2024-04-16T17:10:48.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.748+0000 7f4eee77a740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f4ec400ebb0 con 0x561a3ad37410 2024-04-16T17:10:48.753 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.748+0000 7f4eee77a740 10 client.0 _ll_register_callbacks cb 0x7f4ed8007670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:10:48.753 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.749+0000 7f4ee3e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561a3ad34ed0 0x561a3ad3ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:48.753 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.749+0000 7f4ee9000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561a3ad2df10 0x561a3ad3a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:48.753 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.749+0000 7f4ee3e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561a3ad34ed0 0x561a3ad3ab30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:44782/0 (socket says 172.21.15.45:44782) 2024-04-16T17:10:48.753 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.749+0000 7f4ee3e00640 1 -- 172.21.15.45:0/2283357332 learned_addr learned my addr 172.21.15.45:0/2283357332 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:10:48.753 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.749+0000 7f4ee9a00640 1 --2- 172.21.15.45:0/2283357332 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561a3ad37410 0x561a3adac000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:48.753 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.749+0000 7f4ee9000640 1 -- 172.21.15.45:0/2283357332 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561a3ad37410 msgr2=0x561a3adac000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:10:48.753 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.749+0000 7f4ee9000640 1 --2- 172.21.15.45:0/2283357332 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561a3ad37410 0x561a3adac000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:48.753 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.749+0000 7f4ee9000640 1 -- 172.21.15.45:0/2283357332 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561a3ad34ed0 msgr2=0x561a3ad3ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:10:48.753 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.749+0000 7f4ee9000640 1 --2- 172.21.15.45:0/2283357332 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561a3ad34ed0 0x561a3ad3ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:48.753 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.749+0000 7f4ee9000640 1 -- 172.21.15.45:0/2283357332 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561a3aa6f7d0 con 0x561a3ad2df10 2024-04-16T17:10:48.753 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.749+0000 7f4ee9a00640 1 --2- 172.21.15.45:0/2283357332 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561a3ad37410 0x561a3adac000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:10:48.753 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.750+0000 7f4ee9000640 1 --2- 172.21.15.45:0/2283357332 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561a3ad2df10 0x561a3ad3a5f0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f4ed802ec30 tx=0x7f4ed8013690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47b8a21f36560a20 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:10:48.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.750+0000 7f4ecaa00640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-16T17:10:48.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.750+0000 7f4ecaa00640 1 -- 172.21.15.45:0/2283357332 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ed80233d0 con 0x561a3ad2df10 2024-04-16T17:10:48.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.750+0000 7f4ecaa00640 1 -- 172.21.15.45:0/2283357332 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f4ed8023570 con 0x561a3ad2df10 2024-04-16T17:10:48.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.750+0000 7f4eee77a740 1 -- 172.21.15.45:0/2283357332 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4ed8018040 con 0x561a3ad2df10 2024-04-16T17:10:48.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.750+0000 7f4ecaa00640 1 -- 172.21.15.45:0/2283357332 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f4ed8023770 con 0x561a3ad2df10 2024-04-16T17:10:48.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.750+0000 7f4eee77a740 10 client.16581 fetch_fsmap learned FSMap version 31 2024-04-16T17:10:48.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.750+0000 7f4eee77a740 1 -- 172.21.15.45:0/2283357332 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x561a3adad5a0 con 0x561a3ad2df10 2024-04-16T17:10:48.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.750+0000 7f4ecaa00640 1 -- 172.21.15.45:0/2283357332 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ed8018040 con 0x561a3ad2df10 2024-04-16T17:10:48.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.751+0000 7f4ecaa00640 1 -- 172.21.15.45:0/2283357332 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 31) ==== 8366+0+0 (secure 0 0 0) 0x7f4ed80249d0 con 0x561a3ad2df10 2024-04-16T17:10:48.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.751+0000 7f4eee77a740 10 client.16581 fetch_fsmap finished waiting for FSMap version 31 2024-04-16T17:10:48.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.751+0000 7f4eee77a740 10 client.16581 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-16T17:10:48.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.751+0000 7f4eee77a740 20 client.16581 populate_metadata read hostname 'smithi045' 2024-04-16T17:10:48.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.751+0000 7f4eee77a740 1 --2- 172.21.15.45:0/2283357332 >> [v2:172.21.15.117:6838/2378675949,v1:172.21.15.117:6839/2378675949] conn(0x561a3adadc50 0x561a3adb0070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:48.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.751+0000 7f4eee77a740 4 client.16581 mds_command: new command op to 24532 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T17:10:48.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.751+0000 7f4eee77a740 1 -- 172.21.15.45:0/2283357332 --> [v2:172.21.15.117:6838/2378675949,v1:172.21.15.117:6839/2378675949] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561a3ac9a8b0 con 0x561a3adadc50 2024-04-16T17:10:48.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.755+0000 7f4ee3e00640 1 --2- 172.21.15.45:0/2283357332 >> [v2:172.21.15.117:6838/2378675949,v1:172.21.15.117:6839/2378675949] conn(0x561a3adadc50 0x561a3adb0070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:48.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.755+0000 7f4ee3e00640 1 --2- 172.21.15.45:0/2283357332 >> [v2:172.21.15.117:6838/2378675949,v1:172.21.15.117:6839/2378675949] conn(0x561a3adadc50 0x561a3adb0070 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a1f0a8187597923c server_cookie=2fb7486b150a642c in_seq=0 out_seq=0 2024-04-16T17:10:48.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.755+0000 7f4ecaa00640 10 client.16581 ms_handle_connect on v2:172.21.15.117:6838/2378675949 2024-04-16T17:10:48.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.756+0000 7f4ecaa00640 1 -- 172.21.15.45:0/2283357332 <== mds.2 v2:172.21.15.117:6838/2378675949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561a3ac9a8b0 con 0x561a3adadc50 2024-04-16T17:10:48.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.756+0000 7f4ecaa00640 10 client.16581 handle_command_reply: tid=0 2024-04-16T17:10:48.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.756+0000 7f4eee77a740 1 client.16581 shutdown 2024-04-16T17:10:48.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.756+0000 7f4eee77a740 1 -- 172.21.15.45:0/2283357332 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561a3ad2df10 msgr2=0x561a3ad3a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:48.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.756+0000 7f4eee77a740 1 --2- 172.21.15.45:0/2283357332 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561a3ad2df10 0x561a3ad3a5f0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f4ed802ec30 tx=0x7f4ed8013690 comp rx=0 tx=0).stop 2024-04-16T17:10:48.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.756+0000 7f4eee77a740 1 -- 172.21.15.45:0/2283357332 shutdown_connections 2024-04-16T17:10:48.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.756+0000 7f4eee77a740 1 --2- 172.21.15.45:0/2283357332 >> [v2:172.21.15.117:6838/2378675949,v1:172.21.15.117:6839/2378675949] conn(0x561a3adadc50 0x561a3adb0070 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:48.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.756+0000 7f4eee77a740 1 --2- 172.21.15.45:0/2283357332 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561a3ad37410 0x561a3adac000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:48.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.756+0000 7f4eee77a740 1 --2- 172.21.15.45:0/2283357332 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561a3ad34ed0 0x561a3ad3ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:48.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.756+0000 7f4eee77a740 1 --2- 172.21.15.45:0/2283357332 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561a3ad2df10 0x561a3ad3a5f0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:48.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.756+0000 7f4eee77a740 1 -- 172.21.15.45:0/2283357332 >> 172.21.15.45:0/2283357332 conn(0x561a3ad28640 msgr2=0x561a3ad26e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:10:48.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.756+0000 7f4ecaa00640 0 client.16581 ms_handle_reset on v2:172.21.15.117:6838/2378675949 2024-04-16T17:10:48.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.756+0000 7f4eee77a740 1 -- 172.21.15.45:0/2283357332 shutdown_connections 2024-04-16T17:10:48.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.756+0000 7f4eee77a740 1 -- 172.21.15.45:0/2283357332 wait complete. 2024-04-16T17:10:48.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.756+0000 7f4eee77a740 20 client.16581 trim_cache size 0 max 16384 2024-04-16T17:10:48.759 INFO:teuthology.orchestra.run.smithi045.stdout:[] 2024-04-16T17:10:48.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.757+0000 7f4eea400640 1 -- 172.21.15.45:0/3052058602 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f4ecc0a3c40 msgr2=0x7f4ecc0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:48.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.757+0000 7f4eea400640 1 --2- 172.21.15.45:0/3052058602 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f4ecc0a3c40 0x7f4ecc0a6100 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f4ed00096c0 tx=0x7f4ed0005a20 comp rx=0 tx=0).stop 2024-04-16T17:10:48.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.757+0000 7f4eea400640 1 -- 172.21.15.45:0/3052058602 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4ee410c480 msgr2=0x7f4ee419f420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:48.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.757+0000 7f4eea400640 1 --2- 172.21.15.45:0/3052058602 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4ee410c480 0x7f4ee419f420 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f4edc00eb10 tx=0x7f4edc00c2d0 comp rx=0 tx=0).stop 2024-04-16T17:10:48.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.757+0000 7f4eea400640 1 -- 172.21.15.45:0/3052058602 shutdown_connections 2024-04-16T17:10:48.761 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.757+0000 7f4eea400640 1 --2- 172.21.15.45:0/3052058602 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f4ecc0a3c40 0x7f4ecc0a6100 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:48.761 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.757+0000 7f4eea400640 1 --2- 172.21.15.45:0/3052058602 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4ee410c480 0x7f4ee419f420 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:48.761 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.757+0000 7f4eea400640 1 --2- 172.21.15.45:0/3052058602 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4ee4103980 0x7f4ee419b070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:48.761 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.757+0000 7f4eea400640 1 --2- 172.21.15.45:0/3052058602 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4ee4103060 0x7f4ee419ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:48.761 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.757+0000 7f4eea400640 1 -- 172.21.15.45:0/3052058602 >> 172.21.15.45:0/3052058602 conn(0x7f4ee40fc880 msgr2=0x7f4ee4100be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:10:48.761 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.757+0000 7f4eea400640 1 -- 172.21.15.45:0/3052058602 shutdown_connections 2024-04-16T17:10:48.761 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.758+0000 7f4eea400640 1 -- 172.21.15.45:0/3052058602 wait complete. 2024-04-16T17:10:48.777 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-16T17:10:48.804 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:10:48 smithi117 ceph-mon[36730]: pgmap v124: 129 pgs: 129 active+clean; 36 MiB data, 485 MiB used, 1.0 TiB / 1.0 TiB avail; 597 KiB/s rd, 414 KiB/s wr, 85 op/s 2024-04-16T17:10:48.804 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:10:48 smithi117 ceph-mon[36730]: from='client.? 172.21.15.45:0/636722744' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:10:48.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:10:48 smithi104 ceph-mon[36457]: pgmap v124: 129 pgs: 129 active+clean; 36 MiB data, 485 MiB used, 1.0 TiB / 1.0 TiB avail; 597 KiB/s rd, 414 KiB/s wr, 85 op/s 2024-04-16T17:10:48.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:10:48 smithi104 ceph-mon[36457]: from='client.? 172.21.15.45:0/636722744' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:10:48.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.972+0000 7f37d4e00640 1 -- 172.21.15.45:0/3238707987 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f37d00743a0 msgr2=0x7f37d0074780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:48.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.972+0000 7f37d4e00640 1 --2- 172.21.15.45:0/3238707987 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f37d00743a0 0x7f37d0074780 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f37c0009900 tx=0x7f37c002bcd0 comp rx=0 tx=0).stop 2024-04-16T17:10:48.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.972+0000 7f37d4e00640 1 -- 172.21.15.45:0/3238707987 shutdown_connections 2024-04-16T17:10:48.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.972+0000 7f37d4e00640 1 --2- 172.21.15.45:0/3238707987 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f37d0112eb0 0x7f37d01152a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:48.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.972+0000 7f37d4e00640 1 --2- 172.21.15.45:0/3238707987 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f37d0074cc0 0x7f37d0112970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:48.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.972+0000 7f37d4e00640 1 --2- 172.21.15.45:0/3238707987 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f37d00743a0 0x7f37d0074780 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:48.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.972+0000 7f37d4e00640 1 -- 172.21.15.45:0/3238707987 >> 172.21.15.45:0/3238707987 conn(0x7f37d00fd9d0 msgr2=0x7f37d00ffdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:10:48.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.972+0000 7f37d4e00640 1 -- 172.21.15.45:0/3238707987 shutdown_connections 2024-04-16T17:10:48.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.973+0000 7f37d4e00640 1 -- 172.21.15.45:0/3238707987 wait complete. 2024-04-16T17:10:48.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.973+0000 7f37d4e00640 1 Processor -- start 2024-04-16T17:10:48.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.974+0000 7f37d4e00640 1 -- start start 2024-04-16T17:10:48.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.974+0000 7f37d4e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f37d00743a0 0x7f37d019edf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:48.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.974+0000 7f37d4e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f37d0074cc0 0x7f37d019f330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:48.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.974+0000 7f37d4e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f37d0112eb0 0x7f37d01a36b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:48.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.974+0000 7f37d4e00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f37d01188d0 con 0x7f37d00743a0 2024-04-16T17:10:48.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.974+0000 7f37d4e00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f37d0118750 con 0x7f37d0074cc0 2024-04-16T17:10:48.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.974+0000 7f37d4e00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f37d0118a50 con 0x7f37d0112eb0 2024-04-16T17:10:48.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.974+0000 7f37cea00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f37d0074cc0 0x7f37d019f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:48.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.974+0000 7f37cf400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f37d00743a0 0x7f37d019edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:48.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.974+0000 7f37cea00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f37d0074cc0 0x7f37d019f330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:44802/0 (socket says 172.21.15.45:44802) 2024-04-16T17:10:48.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.974+0000 7f37cea00640 1 -- 172.21.15.45:0/1553758600 learned_addr learned my addr 172.21.15.45:0/1553758600 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:10:48.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.974+0000 7f37cfe00640 1 --2- 172.21.15.45:0/1553758600 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f37d0112eb0 0x7f37d01a36b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:48.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.975+0000 7f37cea00640 1 -- 172.21.15.45:0/1553758600 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f37d0112eb0 msgr2=0x7f37d01a36b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:48.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.975+0000 7f37cea00640 1 --2- 172.21.15.45:0/1553758600 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f37d0112eb0 0x7f37d01a36b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:48.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.975+0000 7f37cea00640 1 -- 172.21.15.45:0/1553758600 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f37d00743a0 msgr2=0x7f37d019edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:48.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.975+0000 7f37cea00640 1 --2- 172.21.15.45:0/1553758600 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f37d00743a0 0x7f37d019edf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:48.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.975+0000 7f37cea00640 1 -- 172.21.15.45:0/1553758600 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37d01a3de0 con 0x7f37d0074cc0 2024-04-16T17:10:48.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.975+0000 7f37cfe00640 1 --2- 172.21.15.45:0/1553758600 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f37d0112eb0 0x7f37d01a36b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:10:48.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.975+0000 7f37cf400640 1 --2- 172.21.15.45:0/1553758600 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f37d00743a0 0x7f37d019edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:10:48.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.975+0000 7f37cea00640 1 --2- 172.21.15.45:0/1553758600 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f37d0074cc0 0x7f37d019f330 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f37bc00e9f0 tx=0x7f37bc00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:10:48.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.976+0000 7f37c7e00640 1 -- 172.21.15.45:0/1553758600 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37bc00cd80 con 0x7f37d0074cc0 2024-04-16T17:10:48.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.976+0000 7f37d4e00640 1 -- 172.21.15.45:0/1553758600 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37d01a40d0 con 0x7f37d0074cc0 2024-04-16T17:10:48.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.976+0000 7f37d4e00640 1 -- 172.21.15.45:0/1553758600 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37d01ab960 con 0x7f37d0074cc0 2024-04-16T17:10:48.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.976+0000 7f37c7e00640 1 -- 172.21.15.45:0/1553758600 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f37bc010910 con 0x7f37d0074cc0 2024-04-16T17:10:48.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.976+0000 7f37c7e00640 1 -- 172.21.15.45:0/1553758600 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37bc0188f0 con 0x7f37d0074cc0 2024-04-16T17:10:48.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.979+0000 7f37d9049740 1 Processor -- start 2024-04-16T17:10:48.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.979+0000 7f37d9049740 1 -- start start 2024-04-16T17:10:48.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.979+0000 7f37d9049740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561739800f10 0x5617398012f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:48.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.979+0000 7f37d9049740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561739801830 0x561739809ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:48.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.979+0000 7f37d9049740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56173980a410 0x56173980c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:48.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.979+0000 7f37d9049740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x561739766020 con 0x56173980a410 2024-04-16T17:10:48.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.979+0000 7f37d9049740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x56173970c460 con 0x561739801830 2024-04-16T17:10:48.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.979+0000 7f37d9049740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x561739761fe0 con 0x561739800f10 2024-04-16T17:10:48.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.980+0000 7f37cfe00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561739800f10 0x5617398012f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:48.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.980+0000 7f37cf400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561739801830 0x561739809ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:48.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.980+0000 7f37cfe00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561739800f10 0x5617398012f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:35190/0 (socket says 172.21.15.45:35190) 2024-04-16T17:10:48.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.980+0000 7f37cfe00640 1 -- 172.21.15.45:0/57054571 learned_addr learned my addr 172.21.15.45:0/57054571 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:10:48.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.980+0000 7f37c7e00640 1 -- 172.21.15.45:0/1553758600 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f37bc018b70 con 0x7f37d0074cc0 2024-04-16T17:10:48.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.980+0000 7f37cf400640 1 -- 172.21.15.45:0/57054571 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561739800f10 msgr2=0x5617398012f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:10:48.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.980+0000 7f37cf400640 1 --2- 172.21.15.45:0/57054571 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561739800f10 0x5617398012f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:48.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.980+0000 7f37cf400640 1 -- 172.21.15.45:0/57054571 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56173980a410 msgr2=0x56173980c800 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-16T17:10:48.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.980+0000 7f37cf400640 1 --2- 172.21.15.45:0/57054571 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56173980a410 0x56173980c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:48.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.980+0000 7f37cf400640 1 -- 172.21.15.45:0/57054571 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561739714040 con 0x561739801830 2024-04-16T17:10:48.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.981+0000 7f37cf400640 1 --2- 172.21.15.45:0/57054571 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561739801830 0x561739809ed0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f37c002c210 tx=0x7f37c0009560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ddc7157b7b8ddc2d server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:10:48.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.981+0000 7f37c4c00640 1 -- 172.21.15.45:0/57054571 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37c0043070 con 0x561739801830 2024-04-16T17:10:48.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.981+0000 7f37c4c00640 1 -- 172.21.15.45:0/57054571 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f37c002c6f0 con 0x561739801830 2024-04-16T17:10:48.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.981+0000 7f37c4c00640 1 -- 172.21.15.45:0/57054571 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37c0004ee0 con 0x561739801830 2024-04-16T17:10:48.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.982+0000 7f37d9049740 1 -- 172.21.15.45:0/57054571 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561739801830 msgr2=0x561739809ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:48.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.982+0000 7f37d9049740 1 --2- 172.21.15.45:0/57054571 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561739801830 0x561739809ed0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f37c002c210 tx=0x7f37c0009560 comp rx=0 tx=0).stop 2024-04-16T17:10:48.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.982+0000 7f37c7e00640 1 --2- 172.21.15.45:0/1553758600 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f37b40a3da0 0x7f37b40a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:48.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.982+0000 7f37c7e00640 1 -- 172.21.15.45:0/1553758600 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f37bc0d0e40 con 0x7f37d0074cc0 2024-04-16T17:10:48.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.982+0000 7f37d9049740 1 -- 172.21.15.45:0/57054571 shutdown_connections 2024-04-16T17:10:48.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.982+0000 7f37d9049740 1 --2- 172.21.15.45:0/57054571 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56173980a410 0x56173980c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:48.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.982+0000 7f37d9049740 1 --2- 172.21.15.45:0/57054571 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561739801830 0x561739809ed0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:48.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.982+0000 7f37d9049740 1 --2- 172.21.15.45:0/57054571 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561739800f10 0x5617398012f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:48.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.982+0000 7f37d9049740 1 -- 172.21.15.45:0/57054571 >> 172.21.15.45:0/57054571 conn(0x5617397fb640 msgr2=0x5617397fba20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:10:48.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.982+0000 7f37cfe00640 1 --2- 172.21.15.45:0/1553758600 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f37b40a3da0 0x7f37b40a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:48.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.982+0000 7f37d9049740 1 -- 172.21.15.45:0/57054571 shutdown_connections 2024-04-16T17:10:48.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.983+0000 7f37d9049740 1 -- 172.21.15.45:0/57054571 wait complete. 2024-04-16T17:10:48.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.983+0000 7f37d9049740 1 -- start start 2024-04-16T17:10:48.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.983+0000 7f37cfe00640 1 --2- 172.21.15.45:0/1553758600 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f37b40a3da0 0x7f37b40a6260 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f37c8005fd0 tx=0x7f37c8009500 comp rx=0 tx=0).ready entity=mgr.25411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:10:48.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.983+0000 7f37d9049740 1 Processor -- start 2024-04-16T17:10:48.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.984+0000 7f37d9049740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561739800f10 0x561739807430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:48.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.984+0000 7f37d9049740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561739801830 0x561739807970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:48.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.984+0000 7f37d9049740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56173980a410 0x561739807eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:48.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.984+0000 7f37d9049740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x56173980d1a0 con 0x56173980a410 2024-04-16T17:10:48.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.984+0000 7f37d9049740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x56173971f830 con 0x561739801830 2024-04-16T17:10:48.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.984+0000 7f37d9049740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x56173980d320 con 0x561739800f10 2024-04-16T17:10:48.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.984+0000 7f37d9049740 10 client.0 _ll_register_callbacks cb 0x561739727150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:10:48.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.984+0000 7f37cfe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56173980a410 0x561739807eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:48.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.984+0000 7f37cfe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56173980a410 0x561739807eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:46524/0 (socket says 172.21.15.45:46524) 2024-04-16T17:10:48.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.984+0000 7f37cfe00640 1 -- 172.21.15.45:0/3656534373 learned_addr learned my addr 172.21.15.45:0/3656534373 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:10:48.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.984+0000 7f37cea00640 1 --2- 172.21.15.45:0/3656534373 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561739801830 0x561739807970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:48.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.984+0000 7f37cf400640 1 --2- 172.21.15.45:0/3656534373 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561739800f10 0x561739807430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:48.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.985+0000 7f37cfe00640 1 -- 172.21.15.45:0/3656534373 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561739800f10 msgr2=0x561739807430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:10:48.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.985+0000 7f37cfe00640 1 --2- 172.21.15.45:0/3656534373 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561739800f10 0x561739807430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:48.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.985+0000 7f37cfe00640 1 -- 172.21.15.45:0/3656534373 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561739801830 msgr2=0x561739807970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:10:48.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.985+0000 7f37cfe00640 1 --2- 172.21.15.45:0/3656534373 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561739801830 0x561739807970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:48.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.985+0000 7f37cfe00640 1 -- 172.21.15.45:0/3656534373 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561739723920 con 0x56173980a410 2024-04-16T17:10:48.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.985+0000 7f37cea00640 1 --2- 172.21.15.45:0/3656534373 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561739801830 0x561739807970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:10:48.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.985+0000 7f37cf400640 1 --2- 172.21.15.45:0/3656534373 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561739800f10 0x561739807430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:10:48.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.985+0000 7f37cfe00640 1 --2- 172.21.15.45:0/3656534373 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56173980a410 0x561739807eb0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f37c8010b30 tx=0x7f37c8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d7c337c54416073 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:10:48.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.985+0000 7f37b9600640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-16T17:10:48.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.985+0000 7f37b9600640 1 -- 172.21.15.45:0/3656534373 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37c801de00 con 0x56173980a410 2024-04-16T17:10:48.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.985+0000 7f37b9600640 1 -- 172.21.15.45:0/3656534373 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f37c8004b20 con 0x56173980a410 2024-04-16T17:10:48.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.985+0000 7f37d9049740 1 -- 172.21.15.45:0/3656534373 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f37c002c6f0 con 0x56173980a410 2024-04-16T17:10:48.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.986+0000 7f37b9600640 1 -- 172.21.15.45:0/3656534373 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f37c8031050 con 0x56173980a410 2024-04-16T17:10:48.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.986+0000 7f37d9049740 10 client.16590 fetch_fsmap learned FSMap version 31 2024-04-16T17:10:48.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.986+0000 7f37d9049740 1 -- 172.21.15.45:0/3656534373 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5617396f17f0 con 0x56173980a410 2024-04-16T17:10:48.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.986+0000 7f37b9600640 1 -- 172.21.15.45:0/3656534373 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f37c002c6f0 con 0x56173980a410 2024-04-16T17:10:48.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.986+0000 7f37b9600640 1 -- 172.21.15.45:0/3656534373 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 31) ==== 8366+0+0 (secure 0 0 0) 0x7f37c800e6e0 con 0x56173980a410 2024-04-16T17:10:48.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.986+0000 7f37d9049740 10 client.16590 fetch_fsmap finished waiting for FSMap version 31 2024-04-16T17:10:48.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.986+0000 7f37d9049740 10 client.16590 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-04-16T17:10:48.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.986+0000 7f37d9049740 20 client.16590 populate_metadata read hostname 'smithi045' 2024-04-16T17:10:48.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.986+0000 7f37d9049740 1 --2- 172.21.15.45:0/3656534373 >> [v2:172.21.15.45:6840/3755310822,v1:172.21.15.45:6841/3755310822] conn(0x56173981a820 0x56173981cc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:48.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.986+0000 7f37d9049740 4 client.16590 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:10:48.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.986+0000 7f37d9049740 1 -- 172.21.15.45:0/3656534373 --> [v2:172.21.15.45:6840/3755310822,v1:172.21.15.45:6841/3755310822] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561739773a40 con 0x56173981a820 2024-04-16T17:10:48.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.987+0000 7f37cf400640 1 --2- 172.21.15.45:0/3656534373 >> [v2:172.21.15.45:6840/3755310822,v1:172.21.15.45:6841/3755310822] conn(0x56173981a820 0x56173981cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:48.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.987+0000 7f37cf400640 1 --2- 172.21.15.45:0/3656534373 >> [v2:172.21.15.45:6840/3755310822,v1:172.21.15.45:6841/3755310822] conn(0x56173981a820 0x56173981cc30 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=36a29e2901f16e31 server_cookie=9df68afbd2658e6b in_seq=0 out_seq=0 2024-04-16T17:10:48.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.987+0000 7f37b9600640 10 client.16590 ms_handle_connect on v2:172.21.15.45:6840/3755310822 2024-04-16T17:10:48.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.989+0000 7f37b9600640 1 -- 172.21.15.45:0/3656534373 <== mds.1 v2:172.21.15.45:6840/3755310822 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561739773a40 con 0x56173981a820 2024-04-16T17:10:48.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.989+0000 7f37b9600640 10 client.16590 handle_command_reply: tid=0 2024-04-16T17:10:48.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.989+0000 7f37d9049740 1 client.16590 shutdown 2024-04-16T17:10:48.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.989+0000 7f37d9049740 1 -- 172.21.15.45:0/3656534373 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56173980a410 msgr2=0x561739807eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:48.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.989+0000 7f37d9049740 1 --2- 172.21.15.45:0/3656534373 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56173980a410 0x561739807eb0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f37c8010b30 tx=0x7f37c8004a40 comp rx=0 tx=0).stop 2024-04-16T17:10:48.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.990+0000 7f37d9049740 1 -- 172.21.15.45:0/3656534373 shutdown_connections 2024-04-16T17:10:48.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.990+0000 7f37d9049740 1 --2- 172.21.15.45:0/3656534373 >> [v2:172.21.15.45:6840/3755310822,v1:172.21.15.45:6841/3755310822] conn(0x56173981a820 0x56173981cc30 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:48.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.990+0000 7f37d9049740 1 --2- 172.21.15.45:0/3656534373 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56173980a410 0x561739807eb0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:48.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.990+0000 7f37d9049740 1 --2- 172.21.15.45:0/3656534373 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561739801830 0x561739807970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:48.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.990+0000 7f37d9049740 1 --2- 172.21.15.45:0/3656534373 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561739800f10 0x561739807430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:48.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.990+0000 7f37b9600640 0 client.16590 ms_handle_reset on v2:172.21.15.45:6840/3755310822 2024-04-16T17:10:48.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.990+0000 7f37d9049740 1 -- 172.21.15.45:0/3656534373 >> 172.21.15.45:0/3656534373 conn(0x5617397fb640 msgr2=0x5617397fd390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:10:48.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.990+0000 7f37d9049740 1 -- 172.21.15.45:0/3656534373 shutdown_connections 2024-04-16T17:10:48.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.990+0000 7f37d9049740 1 -- 172.21.15.45:0/3656534373 wait complete. 2024-04-16T17:10:48.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:48.990+0000 7f37d9049740 20 client.16590 trim_cache size 0 max 16384 2024-04-16T17:10:49.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.013+0000 7f37d9049740 1 Processor -- start 2024-04-16T17:10:49.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.013+0000 7f37d9049740 1 -- start start 2024-04-16T17:10:49.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.013+0000 7f37d9049740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561739800f10 0x561739809b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:49.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.013+0000 7f37d9049740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561739801830 0x5617398160c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:49.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.013+0000 7f37d9049740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56173980a410 0x5617397fa580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:49.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.013+0000 7f37d9049740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f37b0012810 con 0x561739800f10 2024-04-16T17:10:49.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.013+0000 7f37d9049740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f37b0010020 con 0x56173980a410 2024-04-16T17:10:49.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.013+0000 7f37d9049740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f37b000de50 con 0x561739801830 2024-04-16T17:10:49.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.013+0000 7f37cea00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561739801830 0x5617398160c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:49.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.013+0000 7f37cfe00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56173980a410 0x5617397fa580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:49.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.014+0000 7f37cf400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561739800f10 0x561739809b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:49.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.014+0000 7f37cfe00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56173980a410 0x5617397fa580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:44826/0 (socket says 172.21.15.45:44826) 2024-04-16T17:10:49.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.014+0000 7f37cea00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561739801830 0x5617398160c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:35194/0 (socket says 172.21.15.45:35194) 2024-04-16T17:10:49.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.014+0000 7f37cfe00640 1 -- 172.21.15.45:0/3160721374 learned_addr learned my addr 172.21.15.45:0/3160721374 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:10:49.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.014+0000 7f37cfe00640 1 -- 172.21.15.45:0/3160721374 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561739801830 msgr2=0x5617398160c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:10:49.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.014+0000 7f37cfe00640 1 --2- 172.21.15.45:0/3160721374 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561739801830 0x5617398160c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:49.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.014+0000 7f37cfe00640 1 -- 172.21.15.45:0/3160721374 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561739800f10 msgr2=0x561739809b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:10:49.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.014+0000 7f37cfe00640 1 --2- 172.21.15.45:0/3160721374 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561739800f10 0x561739809b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:49.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.014+0000 7f37cfe00640 1 -- 172.21.15.45:0/3160721374 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5617394783f0 con 0x56173980a410 2024-04-16T17:10:49.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.014+0000 7f37cea00640 1 --2- 172.21.15.45:0/3160721374 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561739801830 0x5617398160c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:10:49.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.014+0000 7f37cf400640 1 --2- 172.21.15.45:0/3160721374 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561739800f10 0x561739809b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:10:49.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.014+0000 7f37cfe00640 1 --2- 172.21.15.45:0/3160721374 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56173980a410 0x5617397fa580 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f37c8011010 tx=0x7f37c8009e30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=815644c5882341cd server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:10:49.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.014+0000 7f37bbe00640 1 -- 172.21.15.45:0/3160721374 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37c803a070 con 0x56173980a410 2024-04-16T17:10:49.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.015+0000 7f37bbe00640 1 -- 172.21.15.45:0/3160721374 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f37c80108f0 con 0x56173980a410 2024-04-16T17:10:49.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.015+0000 7f37bbe00640 1 -- 172.21.15.45:0/3160721374 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37c8005030 con 0x56173980a410 2024-04-16T17:10:49.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.015+0000 7f37d9049740 1 -- 172.21.15.45:0/3160721374 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56173980a410 msgr2=0x5617397fa580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:49.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.015+0000 7f37d9049740 1 --2- 172.21.15.45:0/3160721374 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56173980a410 0x5617397fa580 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f37c8011010 tx=0x7f37c8009e30 comp rx=0 tx=0).stop 2024-04-16T17:10:49.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.016+0000 7f37d9049740 1 -- 172.21.15.45:0/3160721374 shutdown_connections 2024-04-16T17:10:49.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.016+0000 7f37d9049740 1 --2- 172.21.15.45:0/3160721374 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561739801830 0x5617398160c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:49.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.016+0000 7f37d9049740 1 --2- 172.21.15.45:0/3160721374 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56173980a410 0x5617397fa580 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:49.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.016+0000 7f37d9049740 1 --2- 172.21.15.45:0/3160721374 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561739800f10 0x561739809b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:49.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.016+0000 7f37d9049740 1 -- 172.21.15.45:0/3160721374 >> 172.21.15.45:0/3160721374 conn(0x5617397fb640 msgr2=0x5617397f9e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:10:49.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.016+0000 7f37d9049740 1 -- 172.21.15.45:0/3160721374 shutdown_connections 2024-04-16T17:10:49.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.016+0000 7f37d9049740 1 -- 172.21.15.45:0/3160721374 wait complete. 2024-04-16T17:10:49.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.016+0000 7f37d9049740 1 -- start start 2024-04-16T17:10:49.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.016+0000 7f37d9049740 1 Processor -- start 2024-04-16T17:10:49.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.017+0000 7f37d9049740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561739800f10 0x5617398040a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:49.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.017+0000 7f37d9049740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561739801830 0x5617398045e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:49.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.017+0000 7f37d9049740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56173980a410 0x561739804b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:49.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.017+0000 7f37d9049740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f37b000e4e0 con 0x561739801830 2024-04-16T17:10:49.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.017+0000 7f37d9049740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f37b000f950 con 0x56173980a410 2024-04-16T17:10:49.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.017+0000 7f37d9049740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f37b000ebb0 con 0x561739800f10 2024-04-16T17:10:49.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.017+0000 7f37d9049740 10 client.0 _ll_register_callbacks cb 0x7f37c8009cc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:10:49.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.017+0000 7f37cf400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561739800f10 0x5617398040a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:49.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.017+0000 7f37cf400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561739800f10 0x5617398040a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:35206/0 (socket says 172.21.15.45:35206) 2024-04-16T17:10:49.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.017+0000 7f37cf400640 1 -- 172.21.15.45:0/1939568910 learned_addr learned my addr 172.21.15.45:0/1939568910 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:10:49.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.017+0000 7f37cea00640 1 --2- 172.21.15.45:0/1939568910 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561739801830 0x5617398045e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:49.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.017+0000 7f37cfe00640 1 --2- 172.21.15.45:0/1939568910 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56173980a410 0x561739804b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:49.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.018+0000 7f37cf400640 1 -- 172.21.15.45:0/1939568910 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56173980a410 msgr2=0x561739804b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:10:49.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.018+0000 7f37cf400640 1 --2- 172.21.15.45:0/1939568910 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56173980a410 0x561739804b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:49.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.018+0000 7f37cf400640 1 -- 172.21.15.45:0/1939568910 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561739801830 msgr2=0x5617398045e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:10:49.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.018+0000 7f37cf400640 1 --2- 172.21.15.45:0/1939568910 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561739801830 0x5617398045e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:49.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.018+0000 7f37cf400640 1 -- 172.21.15.45:0/1939568910 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5617395427d0 con 0x561739800f10 2024-04-16T17:10:49.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.018+0000 7f37cf400640 1 --2- 172.21.15.45:0/1939568910 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561739800f10 0x5617398040a0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f37c0009900 tx=0x7f37c002c350 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ca942a2060662f2e server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:10:49.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.018+0000 7f37b9600640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-04-16T17:10:49.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.018+0000 7f37b9600640 1 -- 172.21.15.45:0/1939568910 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37c0043040 con 0x561739800f10 2024-04-16T17:10:49.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.018+0000 7f37b9600640 1 -- 172.21.15.45:0/1939568910 <== mon.1 v2:172.21.15.117:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f37c0004230 con 0x561739800f10 2024-04-16T17:10:49.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.018+0000 7f37d9049740 1 -- 172.21.15.45:0/1939568910 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56173976d8b0 con 0x561739800f10 2024-04-16T17:10:49.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.018+0000 7f37b9600640 1 -- 172.21.15.45:0/1939568910 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f37c0004430 con 0x561739800f10 2024-04-16T17:10:49.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.018+0000 7f37d9049740 10 client.26701 fetch_fsmap learned FSMap version 31 2024-04-16T17:10:49.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.019+0000 7f37d9049740 1 -- 172.21.15.45:0/1939568910 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x561739880ef0 con 0x561739800f10 2024-04-16T17:10:49.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.019+0000 7f37b9600640 1 -- 172.21.15.45:0/1939568910 <== mon.1 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x56173976d8b0 con 0x561739800f10 2024-04-16T17:10:49.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.019+0000 7f37b9600640 1 -- 172.21.15.45:0/1939568910 <== mon.1 v2:172.21.15.117:3300/0 5 ==== fsmap(e 31) ==== 8366+0+0 (secure 0 0 0) 0x7f37c0051020 con 0x561739800f10 2024-04-16T17:10:49.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.019+0000 7f37d9049740 10 client.26701 fetch_fsmap finished waiting for FSMap version 31 2024-04-16T17:10:49.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.019+0000 7f37d9049740 10 client.26701 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-04-16T17:10:49.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.019+0000 7f37d9049740 20 client.26701 populate_metadata read hostname 'smithi045' 2024-04-16T17:10:49.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.019+0000 7f37d9049740 1 --2- 172.21.15.45:0/1939568910 >> [v2:172.21.15.45:6840/3755310822,v1:172.21.15.45:6841/3755310822] conn(0x5617398812f0 0x5617398836e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:49.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.019+0000 7f37d9049740 4 client.26701 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T17:10:49.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.019+0000 7f37d9049740 1 -- 172.21.15.45:0/1939568910 --> [v2:172.21.15.45:6840/3755310822,v1:172.21.15.45:6841/3755310822] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5617396e9570 con 0x5617398812f0 2024-04-16T17:10:49.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.019+0000 7f37cea00640 1 --2- 172.21.15.45:0/1939568910 >> [v2:172.21.15.45:6840/3755310822,v1:172.21.15.45:6841/3755310822] conn(0x5617398812f0 0x5617398836e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:49.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.020+0000 7f37cea00640 1 --2- 172.21.15.45:0/1939568910 >> [v2:172.21.15.45:6840/3755310822,v1:172.21.15.45:6841/3755310822] conn(0x5617398812f0 0x5617398836e0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=aabf92f663975b17 server_cookie=6f5f909a98a096a5 in_seq=0 out_seq=0 2024-04-16T17:10:49.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.020+0000 7f37b9600640 10 client.26701 ms_handle_connect on v2:172.21.15.45:6840/3755310822 2024-04-16T17:10:49.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.020+0000 7f37b9600640 1 -- 172.21.15.45:0/1939568910 <== mds.1 v2:172.21.15.45:6840/3755310822 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5617396e9570 con 0x5617398812f0 2024-04-16T17:10:49.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.020+0000 7f37b9600640 10 client.26701 handle_command_reply: tid=0 2024-04-16T17:10:49.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.020+0000 7f37d9049740 1 client.26701 shutdown 2024-04-16T17:10:49.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.020+0000 7f37d9049740 1 -- 172.21.15.45:0/1939568910 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561739800f10 msgr2=0x5617398040a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:49.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.020+0000 7f37d9049740 1 --2- 172.21.15.45:0/1939568910 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561739800f10 0x5617398040a0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f37c0009900 tx=0x7f37c002c350 comp rx=0 tx=0).stop 2024-04-16T17:10:49.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.021+0000 7f37d9049740 1 -- 172.21.15.45:0/1939568910 shutdown_connections 2024-04-16T17:10:49.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.021+0000 7f37d9049740 1 --2- 172.21.15.45:0/1939568910 >> [v2:172.21.15.45:6840/3755310822,v1:172.21.15.45:6841/3755310822] conn(0x5617398812f0 0x5617398836e0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:49.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.021+0000 7f37d9049740 1 --2- 172.21.15.45:0/1939568910 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56173980a410 0x561739804b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:49.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.021+0000 7f37d9049740 1 --2- 172.21.15.45:0/1939568910 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561739801830 0x5617398045e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:49.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.021+0000 7f37d9049740 1 --2- 172.21.15.45:0/1939568910 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561739800f10 0x5617398040a0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:49.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.021+0000 7f37d9049740 1 -- 172.21.15.45:0/1939568910 >> 172.21.15.45:0/1939568910 conn(0x5617397fb640 msgr2=0x561739807c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:10:49.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.021+0000 7f37b9600640 0 client.26701 ms_handle_reset on v2:172.21.15.45:6840/3755310822 2024-04-16T17:10:49.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.021+0000 7f37d9049740 1 -- 172.21.15.45:0/1939568910 shutdown_connections 2024-04-16T17:10:49.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.021+0000 7f37d9049740 1 -- 172.21.15.45:0/1939568910 wait complete. 2024-04-16T17:10:49.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.021+0000 7f37d9049740 20 client.26701 trim_cache size 0 max 16384 2024-04-16T17:10:49.026 INFO:teuthology.orchestra.run.smithi045.stdout:[] 2024-04-16T17:10:49.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.022+0000 7f37d4e00640 1 -- 172.21.15.45:0/1553758600 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f37b40a3da0 msgr2=0x7f37b40a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:49.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.022+0000 7f37d4e00640 1 --2- 172.21.15.45:0/1553758600 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f37b40a3da0 0x7f37b40a6260 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f37c8005fd0 tx=0x7f37c8009500 comp rx=0 tx=0).stop 2024-04-16T17:10:49.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.022+0000 7f37d4e00640 1 -- 172.21.15.45:0/1553758600 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f37d0074cc0 msgr2=0x7f37d019f330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:49.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.022+0000 7f37d4e00640 1 --2- 172.21.15.45:0/1553758600 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f37d0074cc0 0x7f37d019f330 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f37bc00e9f0 tx=0x7f37bc00eeb0 comp rx=0 tx=0).stop 2024-04-16T17:10:49.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.022+0000 7f37d4e00640 1 -- 172.21.15.45:0/1553758600 shutdown_connections 2024-04-16T17:10:49.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.022+0000 7f37d4e00640 1 --2- 172.21.15.45:0/1553758600 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f37b40a3da0 0x7f37b40a6260 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:49.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.022+0000 7f37d4e00640 1 --2- 172.21.15.45:0/1553758600 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f37d0112eb0 0x7f37d01a36b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:49.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.022+0000 7f37d4e00640 1 --2- 172.21.15.45:0/1553758600 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f37d0074cc0 0x7f37d019f330 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:49.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.022+0000 7f37d4e00640 1 --2- 172.21.15.45:0/1553758600 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f37d00743a0 0x7f37d019edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:49.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.022+0000 7f37d4e00640 1 -- 172.21.15.45:0/1553758600 >> 172.21.15.45:0/1553758600 conn(0x7f37d00fd9d0 msgr2=0x7f37d00fe740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:10:49.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.022+0000 7f37d4e00640 1 -- 172.21.15.45:0/1553758600 shutdown_connections 2024-04-16T17:10:49.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.022+0000 7f37d4e00640 1 -- 172.21.15.45:0/1553758600 wait complete. 2024-04-16T17:10:49.038 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-16T17:10:49.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.219+0000 7fb810200640 1 -- 172.21.15.45:0/4150996771 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb808112ee0 msgr2=0x7fb8081152d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:49.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.219+0000 7fb810200640 1 --2- 172.21.15.45:0/4150996771 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb808112ee0 0x7fb8081152d0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fb800009ca0 tx=0x7fb80002c100 comp rx=0 tx=0).stop 2024-04-16T17:10:49.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.219+0000 7fb810200640 1 -- 172.21.15.45:0/4150996771 shutdown_connections 2024-04-16T17:10:49.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.219+0000 7fb810200640 1 --2- 172.21.15.45:0/4150996771 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb808112ee0 0x7fb8081152d0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:49.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.219+0000 7fb810200640 1 --2- 172.21.15.45:0/4150996771 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb808076e50 0x7fb8081129a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:49.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.219+0000 7fb810200640 1 --2- 172.21.15.45:0/4150996771 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb808076530 0x7fb808076910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:49.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.219+0000 7fb810200640 1 -- 172.21.15.45:0/4150996771 >> 172.21.15.45:0/4150996771 conn(0x7fb8080ffd90 msgr2=0x7fb8081021b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:10:49.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.219+0000 7fb810200640 1 -- 172.21.15.45:0/4150996771 shutdown_connections 2024-04-16T17:10:49.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.220+0000 7fb810200640 1 -- 172.21.15.45:0/4150996771 wait complete. 2024-04-16T17:10:49.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.221+0000 7fb810200640 1 Processor -- start 2024-04-16T17:10:49.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.221+0000 7fb810200640 1 -- start start 2024-04-16T17:10:49.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.222+0000 7fb810200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb808076530 0x7fb80819efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:49.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.222+0000 7fb810200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb808076e50 0x7fb80819f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:49.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.222+0000 7fb810200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb808112ee0 0x7fb8081a3860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:49.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.222+0000 7fb810200640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fb808118a40 con 0x7fb808112ee0 2024-04-16T17:10:49.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.222+0000 7fb810200640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fb8081188c0 con 0x7fb808076530 2024-04-16T17:10:49.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.222+0000 7fb810200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb808118bc0 con 0x7fb808076e50 2024-04-16T17:10:49.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.222+0000 7fb80e200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb808112ee0 0x7fb8081a3860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:49.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.222+0000 7fb80e200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb808112ee0 0x7fb8081a3860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:46548/0 (socket says 172.21.15.45:46548) 2024-04-16T17:10:49.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.222+0000 7fb80e200640 1 -- 172.21.15.45:0/3204716486 learned_addr learned my addr 172.21.15.45:0/3204716486 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:10:49.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.222+0000 7fb80ce00640 1 --2- 172.21.15.45:0/3204716486 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb808076e50 0x7fb80819f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:49.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.223+0000 7fb80e200640 1 -- 172.21.15.45:0/3204716486 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb808076e50 msgr2=0x7fb80819f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:49.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.223+0000 7fb80e200640 1 --2- 172.21.15.45:0/3204716486 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb808076e50 0x7fb80819f4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:49.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.223+0000 7fb80e200640 1 -- 172.21.15.45:0/3204716486 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb808076530 msgr2=0x7fb80819efa0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-16T17:10:49.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.223+0000 7fb80e200640 1 --2- 172.21.15.45:0/3204716486 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb808076530 0x7fb80819efa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:49.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.223+0000 7fb80e200640 1 -- 172.21.15.45:0/3204716486 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8081a3f90 con 0x7fb808112ee0 2024-04-16T17:10:49.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.223+0000 7fb80e200640 1 --2- 172.21.15.45:0/3204716486 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb808112ee0 0x7fb8081a3860 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fb800037040 tx=0x7fb800034670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:10:49.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.223+0000 7fb806000640 1 -- 172.21.15.45:0/3204716486 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb800039050 con 0x7fb808112ee0 2024-04-16T17:10:49.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.224+0000 7fb806000640 1 -- 172.21.15.45:0/3204716486 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb800045070 con 0x7fb808112ee0 2024-04-16T17:10:49.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.224+0000 7fb806000640 1 -- 172.21.15.45:0/3204716486 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb800004500 con 0x7fb808112ee0 2024-04-16T17:10:49.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.224+0000 7fb810200640 1 -- 172.21.15.45:0/3204716486 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8081a4220 con 0x7fb808112ee0 2024-04-16T17:10:49.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.224+0000 7fb810200640 1 -- 172.21.15.45:0/3204716486 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8081abb10 con 0x7fb808112ee0 2024-04-16T17:10:49.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.227+0000 7fb806000640 1 -- 172.21.15.45:0/3204716486 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fb8000046a0 con 0x7fb808112ee0 2024-04-16T17:10:49.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.228+0000 7fb812fa0740 1 Processor -- start 2024-04-16T17:10:49.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.229+0000 7fb806000640 1 --2- 172.21.15.45:0/3204716486 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fb7f00a3d60 0x7fb7f00a6220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:49.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.229+0000 7fb806000640 1 -- 172.21.15.45:0/3204716486 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fb8000f31a0 con 0x7fb808112ee0 2024-04-16T17:10:49.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.229+0000 7fb812fa0740 1 -- start start 2024-04-16T17:10:49.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.229+0000 7fb80ce00640 1 --2- 172.21.15.45:0/3204716486 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fb7f00a3d60 0x7fb7f00a6220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:49.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.229+0000 7fb812fa0740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d6ea215d50 0x55d6ea216130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:49.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.229+0000 7fb812fa0740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d6ea216670 0x55d6ea21ed10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:49.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.229+0000 7fb812fa0740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d6ea21f250 0x55d6ea221640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:49.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.229+0000 7fb812fa0740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55d6ea17b020 con 0x55d6ea216670 2024-04-16T17:10:49.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.229+0000 7fb812fa0740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55d6ea121460 con 0x55d6ea215d50 2024-04-16T17:10:49.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.229+0000 7fb812fa0740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55d6ea176fe0 con 0x55d6ea21f250 2024-04-16T17:10:49.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.230+0000 7fb80ce00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d6ea216670 0x55d6ea21ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:49.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.230+0000 7fb80d800640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d6ea215d50 0x55d6ea216130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:49.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.230+0000 7fb80d800640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d6ea215d50 0x55d6ea216130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:44860/0 (socket says 172.21.15.45:44860) 2024-04-16T17:10:49.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.230+0000 7fb80d800640 1 -- 172.21.15.45:0/3591094088 learned_addr learned my addr 172.21.15.45:0/3591094088 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:10:49.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.230+0000 7fb80e200640 1 --2- 172.21.15.45:0/3591094088 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d6ea21f250 0x55d6ea221640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:49.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.230+0000 7fb80d800640 1 -- 172.21.15.45:0/3591094088 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d6ea21f250 msgr2=0x55d6ea221640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:10:49.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.230+0000 7fb80d800640 1 --2- 172.21.15.45:0/3591094088 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d6ea21f250 0x55d6ea221640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:49.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.230+0000 7fb80d800640 1 -- 172.21.15.45:0/3591094088 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d6ea216670 msgr2=0x55d6ea21ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:10:49.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.230+0000 7fb80d800640 1 --2- 172.21.15.45:0/3591094088 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d6ea216670 0x55d6ea21ed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:49.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.230+0000 7fb80d800640 1 -- 172.21.15.45:0/3591094088 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d6ea129040 con 0x55d6ea215d50 2024-04-16T17:10:49.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.231+0000 7fb80d800640 1 --2- 172.21.15.45:0/3591094088 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d6ea215d50 0x55d6ea216130 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fb7fc009bd0 tx=0x7fb7fc00e720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=50384692fc62f860 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:10:49.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.231+0000 7fb80ce00640 1 --2- 172.21.15.45:0/3204716486 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fb7f00a3d60 0x7fb7f00a6220 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fb8081021d0 tx=0x7fb7f400bf90 comp rx=0 tx=0).ready entity=mgr.25411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:10:49.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.231+0000 7fb7faa00640 1 -- 172.21.15.45:0/3591094088 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7fc005020 con 0x55d6ea215d50 2024-04-16T17:10:49.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.231+0000 7fb7faa00640 1 -- 172.21.15.45:0/3591094088 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb7fc0047a0 con 0x55d6ea215d50 2024-04-16T17:10:49.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.231+0000 7fb7faa00640 1 -- 172.21.15.45:0/3591094088 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7fc0186d0 con 0x55d6ea215d50 2024-04-16T17:10:49.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.232+0000 7fb812fa0740 1 -- 172.21.15.45:0/3591094088 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d6ea215d50 msgr2=0x55d6ea216130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:49.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.232+0000 7fb812fa0740 1 --2- 172.21.15.45:0/3591094088 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d6ea215d50 0x55d6ea216130 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fb7fc009bd0 tx=0x7fb7fc00e720 comp rx=0 tx=0).stop 2024-04-16T17:10:49.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.232+0000 7fb812fa0740 1 -- 172.21.15.45:0/3591094088 shutdown_connections 2024-04-16T17:10:49.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.232+0000 7fb812fa0740 1 --2- 172.21.15.45:0/3591094088 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d6ea21f250 0x55d6ea221640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:49.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.232+0000 7fb812fa0740 1 --2- 172.21.15.45:0/3591094088 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d6ea216670 0x55d6ea21ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:49.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.232+0000 7fb812fa0740 1 --2- 172.21.15.45:0/3591094088 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d6ea215d50 0x55d6ea216130 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:49.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.232+0000 7fb812fa0740 1 -- 172.21.15.45:0/3591094088 >> 172.21.15.45:0/3591094088 conn(0x55d6ea210480 msgr2=0x55d6ea210860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:10:49.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.232+0000 7fb812fa0740 1 -- 172.21.15.45:0/3591094088 shutdown_connections 2024-04-16T17:10:49.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.232+0000 7fb812fa0740 1 -- 172.21.15.45:0/3591094088 wait complete. 2024-04-16T17:10:49.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.233+0000 7fb812fa0740 1 -- start start 2024-04-16T17:10:49.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.233+0000 7fb812fa0740 1 Processor -- start 2024-04-16T17:10:49.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.234+0000 7fb812fa0740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d6ea215d50 0x55d6ea21c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:49.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.234+0000 7fb812fa0740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d6ea216670 0x55d6ea21c820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:49.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.234+0000 7fb812fa0740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d6ea21f250 0x55d6ea21cd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:49.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.234+0000 7fb812fa0740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55d6ea222040 con 0x55d6ea21f250 2024-04-16T17:10:49.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.234+0000 7fb812fa0740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55d6ea134830 con 0x55d6ea215d50 2024-04-16T17:10:49.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.234+0000 7fb812fa0740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55d6ea2221c0 con 0x55d6ea216670 2024-04-16T17:10:49.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.234+0000 7fb812fa0740 10 client.0 _ll_register_callbacks cb 0x55d6ea13c150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:10:49.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.234+0000 7fb80ce00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d6ea216670 0x55d6ea21c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:49.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.234+0000 7fb80ce00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d6ea216670 0x55d6ea21c820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:35238/0 (socket says 172.21.15.45:35238) 2024-04-16T17:10:49.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.234+0000 7fb80ce00640 1 -- 172.21.15.45:0/2280282273 learned_addr learned my addr 172.21.15.45:0/2280282273 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:10:49.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.234+0000 7fb80e200640 1 --2- 172.21.15.45:0/2280282273 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d6ea21f250 0x55d6ea21cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:49.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.235+0000 7fb80d800640 1 --2- 172.21.15.45:0/2280282273 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d6ea215d50 0x55d6ea21c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:49.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.235+0000 7fb80ce00640 1 -- 172.21.15.45:0/2280282273 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d6ea215d50 msgr2=0x55d6ea21c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:10:49.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.235+0000 7fb80ce00640 1 --2- 172.21.15.45:0/2280282273 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d6ea215d50 0x55d6ea21c2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:49.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.235+0000 7fb80ce00640 1 -- 172.21.15.45:0/2280282273 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d6ea21f250 msgr2=0x55d6ea21cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:10:49.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.235+0000 7fb80ce00640 1 --2- 172.21.15.45:0/2280282273 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d6ea21f250 0x55d6ea21cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:49.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.235+0000 7fb80ce00640 1 -- 172.21.15.45:0/2280282273 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d6ea138920 con 0x55d6ea216670 2024-04-16T17:10:49.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.235+0000 7fb80d800640 1 --2- 172.21.15.45:0/2280282273 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d6ea215d50 0x55d6ea21c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:10:49.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.235+0000 7fb80ce00640 1 --2- 172.21.15.45:0/2280282273 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d6ea216670 0x55d6ea21c820 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fb7f400e800 tx=0x7fb7f4004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8a8b4f2cd5cb0204 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:10:49.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.235+0000 7fb7ef400640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-04-16T17:10:49.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.235+0000 7fb7ef400640 1 -- 172.21.15.45:0/2280282273 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7f400fb40 con 0x55d6ea216670 2024-04-16T17:10:49.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.235+0000 7fb7ef400640 1 -- 172.21.15.45:0/2280282273 <== mon.1 v2:172.21.15.117:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fb7f400fce0 con 0x55d6ea216670 2024-04-16T17:10:49.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.235+0000 7fb812fa0740 1 -- 172.21.15.45:0/2280282273 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb7fc0047a0 con 0x55d6ea216670 2024-04-16T17:10:49.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.236+0000 7fb7ef400640 1 -- 172.21.15.45:0/2280282273 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fb7f4030050 con 0x55d6ea216670 2024-04-16T17:10:49.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.236+0000 7fb812fa0740 10 client.26713 fetch_fsmap learned FSMap version 31 2024-04-16T17:10:49.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.236+0000 7fb812fa0740 1 -- 172.21.15.45:0/2280282273 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55d6ea1067f0 con 0x55d6ea216670 2024-04-16T17:10:49.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.236+0000 7fb7ef400640 1 -- 172.21.15.45:0/2280282273 <== mon.1 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb7fc0047a0 con 0x55d6ea216670 2024-04-16T17:10:49.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.236+0000 7fb7ef400640 1 -- 172.21.15.45:0/2280282273 <== mon.1 v2:172.21.15.117:3300/0 5 ==== fsmap(e 31) ==== 8366+0+0 (secure 0 0 0) 0x7fb7f401f480 con 0x55d6ea216670 2024-04-16T17:10:49.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.237+0000 7fb812fa0740 10 client.26713 fetch_fsmap finished waiting for FSMap version 31 2024-04-16T17:10:49.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.237+0000 7fb812fa0740 10 client.26713 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:10:49.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.237+0000 7fb812fa0740 20 client.26713 populate_metadata read hostname 'smithi045' 2024-04-16T17:10:49.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.237+0000 7fb812fa0740 1 --2- 172.21.15.45:0/2280282273 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55d6ea22f680 0x55d6ea231ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:49.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.237+0000 7fb812fa0740 4 client.26713 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:10:49.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.237+0000 7fb812fa0740 1 -- 172.21.15.45:0/2280282273 --> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d6ea188a40 con 0x55d6ea22f680 2024-04-16T17:10:49.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.237+0000 7fb80d800640 1 --2- 172.21.15.45:0/2280282273 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55d6ea22f680 0x55d6ea231ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:49.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.238+0000 7fb80d800640 1 --2- 172.21.15.45:0/2280282273 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55d6ea22f680 0x55d6ea231ac0 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a0404200b45c0736 server_cookie=9c8b2fb5f9a858ba in_seq=0 out_seq=0 2024-04-16T17:10:49.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.238+0000 7fb7ef400640 10 client.26713 ms_handle_connect on v2:172.21.15.104:6838/3660070592 2024-04-16T17:10:49.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.239+0000 7fb7ef400640 1 -- 172.21.15.45:0/2280282273 <== mds.0 v2:172.21.15.104:6838/3660070592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d6ea188a40 con 0x55d6ea22f680 2024-04-16T17:10:49.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.239+0000 7fb7ef400640 10 client.26713 handle_command_reply: tid=0 2024-04-16T17:10:49.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.239+0000 7fb812fa0740 1 client.26713 shutdown 2024-04-16T17:10:49.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.239+0000 7fb812fa0740 1 -- 172.21.15.45:0/2280282273 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d6ea216670 msgr2=0x55d6ea21c820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:49.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.239+0000 7fb812fa0740 1 --2- 172.21.15.45:0/2280282273 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d6ea216670 0x55d6ea21c820 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fb7f400e800 tx=0x7fb7f4004730 comp rx=0 tx=0).stop 2024-04-16T17:10:49.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.239+0000 7fb812fa0740 1 -- 172.21.15.45:0/2280282273 shutdown_connections 2024-04-16T17:10:49.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.239+0000 7fb812fa0740 1 --2- 172.21.15.45:0/2280282273 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55d6ea22f680 0x55d6ea231ac0 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:49.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.239+0000 7fb812fa0740 1 --2- 172.21.15.45:0/2280282273 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d6ea216670 0x55d6ea21c820 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:49.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.239+0000 7fb812fa0740 1 --2- 172.21.15.45:0/2280282273 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d6ea21f250 0x55d6ea21cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:49.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.239+0000 7fb812fa0740 1 --2- 172.21.15.45:0/2280282273 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d6ea215d50 0x55d6ea21c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:49.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.239+0000 7fb7ef400640 0 client.26713 ms_handle_reset on v2:172.21.15.104:6838/3660070592 2024-04-16T17:10:49.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.239+0000 7fb812fa0740 1 -- 172.21.15.45:0/2280282273 >> 172.21.15.45:0/2280282273 conn(0x55d6ea210480 msgr2=0x55d6ea2121d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:10:49.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.239+0000 7fb812fa0740 1 -- 172.21.15.45:0/2280282273 shutdown_connections 2024-04-16T17:10:49.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.239+0000 7fb812fa0740 1 -- 172.21.15.45:0/2280282273 wait complete. 2024-04-16T17:10:49.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.240+0000 7fb812fa0740 20 client.26713 trim_cache size 0 max 16384 2024-04-16T17:10:49.246 INFO:tasks.workunit.client.0.smithi045.stdout: 1 54745 1.25 MB/sec execute 51 sec latency 59.916 ms 2024-04-16T17:10:49.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.261+0000 7fb812fa0740 1 Processor -- start 2024-04-16T17:10:49.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.261+0000 7fb812fa0740 1 -- start start 2024-04-16T17:10:49.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.261+0000 7fb812fa0740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d6ea215d50 0x55d6ea21ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:49.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.261+0000 7fb812fa0740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d6ea216670 0x55d6ea22af00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:49.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.261+0000 7fb812fa0740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d6ea21f250 0x55d6ea21c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:49.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.261+0000 7fb812fa0740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fb8081b84e0 con 0x55d6ea21f250 2024-04-16T17:10:49.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.261+0000 7fb812fa0740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fb8081b5cf0 con 0x55d6ea216670 2024-04-16T17:10:49.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.261+0000 7fb812fa0740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb8081b3b20 con 0x55d6ea215d50 2024-04-16T17:10:49.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.262+0000 7fb80d800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d6ea215d50 0x55d6ea21ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:49.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.262+0000 7fb80d800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d6ea215d50 0x55d6ea21ea10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:35246/0 (socket says 172.21.15.45:35246) 2024-04-16T17:10:49.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.262+0000 7fb80d800640 1 -- 172.21.15.45:0/1478138414 learned_addr learned my addr 172.21.15.45:0/1478138414 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:10:49.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.262+0000 7fb80ce00640 1 --2- 172.21.15.45:0/1478138414 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d6ea216670 0x55d6ea22af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:49.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.262+0000 7fb80e200640 1 --2- 172.21.15.45:0/1478138414 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d6ea21f250 0x55d6ea21c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:49.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.262+0000 7fb80d800640 1 -- 172.21.15.45:0/1478138414 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d6ea216670 msgr2=0x55d6ea22af00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:10:49.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.262+0000 7fb80d800640 1 --2- 172.21.15.45:0/1478138414 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d6ea216670 0x55d6ea22af00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:49.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.262+0000 7fb80d800640 1 -- 172.21.15.45:0/1478138414 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d6ea21f250 msgr2=0x55d6ea21c9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:10:49.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.262+0000 7fb80d800640 1 --2- 172.21.15.45:0/1478138414 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d6ea21f250 0x55d6ea21c9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:49.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.262+0000 7fb80d800640 1 -- 172.21.15.45:0/1478138414 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d6e9e8d3f0 con 0x55d6ea215d50 2024-04-16T17:10:49.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.262+0000 7fb80ce00640 1 --2- 172.21.15.45:0/1478138414 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d6ea216670 0x55d6ea22af00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:10:49.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.262+0000 7fb80d800640 1 --2- 172.21.15.45:0/1478138414 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d6ea215d50 0x55d6ea21ea10 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fb7fc0057c0 tx=0x7fb7fc009ec0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3fe414c9ae92253a server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:10:49.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.263+0000 7fb7fa000640 1 -- 172.21.15.45:0/1478138414 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7fc02b070 con 0x55d6ea215d50 2024-04-16T17:10:49.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.263+0000 7fb7fa000640 1 -- 172.21.15.45:0/1478138414 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb7fc009810 con 0x55d6ea215d50 2024-04-16T17:10:49.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.263+0000 7fb7fa000640 1 -- 172.21.15.45:0/1478138414 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7fc0267e0 con 0x55d6ea215d50 2024-04-16T17:10:49.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.264+0000 7fb812fa0740 1 -- 172.21.15.45:0/1478138414 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d6ea215d50 msgr2=0x55d6ea21ea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:49.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.264+0000 7fb812fa0740 1 --2- 172.21.15.45:0/1478138414 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d6ea215d50 0x55d6ea21ea10 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fb7fc0057c0 tx=0x7fb7fc009ec0 comp rx=0 tx=0).stop 2024-04-16T17:10:49.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.264+0000 7fb812fa0740 1 -- 172.21.15.45:0/1478138414 shutdown_connections 2024-04-16T17:10:49.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.264+0000 7fb812fa0740 1 --2- 172.21.15.45:0/1478138414 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d6ea21f250 0x55d6ea21c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:49.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.264+0000 7fb812fa0740 1 --2- 172.21.15.45:0/1478138414 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d6ea216670 0x55d6ea22af00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:49.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.264+0000 7fb812fa0740 1 --2- 172.21.15.45:0/1478138414 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d6ea215d50 0x55d6ea21ea10 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:49.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.264+0000 7fb812fa0740 1 -- 172.21.15.45:0/1478138414 >> 172.21.15.45:0/1478138414 conn(0x55d6ea210480 msgr2=0x55d6ea21c2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:10:49.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.264+0000 7fb812fa0740 1 -- 172.21.15.45:0/1478138414 shutdown_connections 2024-04-16T17:10:49.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.264+0000 7fb812fa0740 1 -- 172.21.15.45:0/1478138414 wait complete. 2024-04-16T17:10:49.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.264+0000 7fb812fa0740 1 -- start start 2024-04-16T17:10:49.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.264+0000 7fb812fa0740 1 Processor -- start 2024-04-16T17:10:49.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.265+0000 7fb812fa0740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d6ea215d50 0x55d6ea227870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:49.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.265+0000 7fb812fa0740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d6ea216670 0x55d6ea222be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:49.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.265+0000 7fb812fa0740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d6ea21f250 0x55d6ea223160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:49.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.265+0000 7fb812fa0740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fb8081b41b0 con 0x55d6ea21f250 2024-04-16T17:10:49.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.265+0000 7fb812fa0740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fb8081b5620 con 0x55d6ea215d50 2024-04-16T17:10:49.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.265+0000 7fb812fa0740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb8081b4880 con 0x55d6ea216670 2024-04-16T17:10:49.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.265+0000 7fb812fa0740 10 client.0 _ll_register_callbacks cb 0x7fb7f400b950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:10:49.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.265+0000 7fb80d800640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d6ea215d50 0x55d6ea227870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:49.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.265+0000 7fb80d800640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d6ea215d50 0x55d6ea227870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:44892/0 (socket says 172.21.15.45:44892) 2024-04-16T17:10:49.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.265+0000 7fb80d800640 1 -- 172.21.15.45:0/628159255 learned_addr learned my addr 172.21.15.45:0/628159255 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:10:49.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.265+0000 7fb80e200640 1 --2- 172.21.15.45:0/628159255 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d6ea21f250 0x55d6ea223160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:49.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.265+0000 7fb80ce00640 1 --2- 172.21.15.45:0/628159255 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d6ea216670 0x55d6ea222be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:49.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.266+0000 7fb80d800640 1 -- 172.21.15.45:0/628159255 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d6ea216670 msgr2=0x55d6ea222be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:10:49.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.266+0000 7fb80d800640 1 --2- 172.21.15.45:0/628159255 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d6ea216670 0x55d6ea222be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:49.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.266+0000 7fb80d800640 1 -- 172.21.15.45:0/628159255 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d6ea21f250 msgr2=0x55d6ea223160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:10:49.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.266+0000 7fb80d800640 1 --2- 172.21.15.45:0/628159255 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d6ea21f250 0x55d6ea223160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:49.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.266+0000 7fb80d800640 1 -- 172.21.15.45:0/628159255 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d6e9f577d0 con 0x55d6ea215d50 2024-04-16T17:10:49.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.266+0000 7fb80e200640 1 --2- 172.21.15.45:0/628159255 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d6ea21f250 0x55d6ea223160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:10:49.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.266+0000 7fb80d800640 1 --2- 172.21.15.45:0/628159255 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d6ea215d50 0x55d6ea227870 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fb7fc00e760 tx=0x7fb7fc009530 comp rx=0 tx=0).ready entity=mon.2 client_cookie=22a46d08a75384ca server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:10:49.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.266+0000 7fb7ef400640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-04-16T17:10:49.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.266+0000 7fb7ef400640 1 -- 172.21.15.45:0/628159255 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7fc02b030 con 0x55d6ea215d50 2024-04-16T17:10:49.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.266+0000 7fb7ef400640 1 -- 172.21.15.45:0/628159255 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fb7fc026230 con 0x55d6ea215d50 2024-04-16T17:10:49.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.266+0000 7fb812fa0740 1 -- 172.21.15.45:0/628159255 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb7fc009810 con 0x55d6ea215d50 2024-04-16T17:10:49.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.267+0000 7fb7ef400640 1 -- 172.21.15.45:0/628159255 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fb7fc034050 con 0x55d6ea215d50 2024-04-16T17:10:49.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.267+0000 7fb812fa0740 10 client.26762 fetch_fsmap learned FSMap version 31 2024-04-16T17:10:49.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.267+0000 7fb812fa0740 1 -- 172.21.15.45:0/628159255 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55d6ea2a6ae0 con 0x55d6ea215d50 2024-04-16T17:10:49.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.267+0000 7fb7ef400640 1 -- 172.21.15.45:0/628159255 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb7fc009810 con 0x55d6ea215d50 2024-04-16T17:10:49.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.267+0000 7fb7ef400640 1 -- 172.21.15.45:0/628159255 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 31) ==== 8366+0+0 (secure 0 0 0) 0x7fb7fc003960 con 0x55d6ea215d50 2024-04-16T17:10:49.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.267+0000 7fb812fa0740 10 client.26762 fetch_fsmap finished waiting for FSMap version 31 2024-04-16T17:10:49.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.267+0000 7fb812fa0740 10 client.26762 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:10:49.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.267+0000 7fb812fa0740 20 client.26762 populate_metadata read hostname 'smithi045' 2024-04-16T17:10:49.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.268+0000 7fb812fa0740 1 --2- 172.21.15.45:0/628159255 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55d6ea2a6f60 0x55d6ea2a9350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:49.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.268+0000 7fb812fa0740 4 client.26762 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T17:10:49.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.268+0000 7fb812fa0740 1 -- 172.21.15.45:0/628159255 --> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d6ea1828b0 con 0x55d6ea2a6f60 2024-04-16T17:10:49.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.268+0000 7fb80ce00640 1 --2- 172.21.15.45:0/628159255 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55d6ea2a6f60 0x55d6ea2a9350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:49.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.271+0000 7fb80ce00640 1 --2- 172.21.15.45:0/628159255 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55d6ea2a6f60 0x55d6ea2a9350 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=feac5a5d5013cee5 server_cookie=39b415c2d531d1db in_seq=0 out_seq=0 2024-04-16T17:10:49.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.272+0000 7fb7ef400640 10 client.26762 ms_handle_connect on v2:172.21.15.104:6838/3660070592 2024-04-16T17:10:49.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.272+0000 7fb7ef400640 1 -- 172.21.15.45:0/628159255 <== mds.0 v2:172.21.15.104:6838/3660070592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d6ea1828b0 con 0x55d6ea2a6f60 2024-04-16T17:10:49.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.272+0000 7fb7ef400640 10 client.26762 handle_command_reply: tid=0 2024-04-16T17:10:49.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.272+0000 7fb812fa0740 1 client.26762 shutdown 2024-04-16T17:10:49.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.272+0000 7fb812fa0740 1 -- 172.21.15.45:0/628159255 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d6ea215d50 msgr2=0x55d6ea227870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:49.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.272+0000 7fb812fa0740 1 --2- 172.21.15.45:0/628159255 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d6ea215d50 0x55d6ea227870 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fb7fc00e760 tx=0x7fb7fc009530 comp rx=0 tx=0).stop 2024-04-16T17:10:49.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.272+0000 7fb812fa0740 1 -- 172.21.15.45:0/628159255 shutdown_connections 2024-04-16T17:10:49.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.272+0000 7fb812fa0740 1 --2- 172.21.15.45:0/628159255 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55d6ea2a6f60 0x55d6ea2a9350 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:49.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.272+0000 7fb812fa0740 1 --2- 172.21.15.45:0/628159255 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d6ea216670 0x55d6ea222be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:49.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.272+0000 7fb812fa0740 1 --2- 172.21.15.45:0/628159255 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d6ea21f250 0x55d6ea223160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:49.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.272+0000 7fb812fa0740 1 --2- 172.21.15.45:0/628159255 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d6ea215d50 0x55d6ea227870 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:49.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.272+0000 7fb812fa0740 1 -- 172.21.15.45:0/628159255 >> 172.21.15.45:0/628159255 conn(0x55d6ea210480 msgr2=0x55d6ea21cb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:10:49.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.272+0000 7fb7ef400640 0 client.26762 ms_handle_reset on v2:172.21.15.104:6838/3660070592 2024-04-16T17:10:49.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.272+0000 7fb812fa0740 1 -- 172.21.15.45:0/628159255 shutdown_connections 2024-04-16T17:10:49.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.273+0000 7fb812fa0740 1 -- 172.21.15.45:0/628159255 wait complete. 2024-04-16T17:10:49.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.273+0000 7fb812fa0740 20 client.26762 trim_cache size 0 max 16384 2024-04-16T17:10:49.274 INFO:teuthology.orchestra.run.smithi045.stdout:[] 2024-04-16T17:10:49.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.274+0000 7fb810200640 1 -- 172.21.15.45:0/3204716486 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fb7f00a3d60 msgr2=0x7fb7f00a6220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:49.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.274+0000 7fb810200640 1 --2- 172.21.15.45:0/3204716486 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fb7f00a3d60 0x7fb7f00a6220 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fb8081021d0 tx=0x7fb7f400bf90 comp rx=0 tx=0).stop 2024-04-16T17:10:49.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.274+0000 7fb810200640 1 -- 172.21.15.45:0/3204716486 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb808112ee0 msgr2=0x7fb8081a3860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:49.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.274+0000 7fb810200640 1 --2- 172.21.15.45:0/3204716486 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb808112ee0 0x7fb8081a3860 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fb800037040 tx=0x7fb800034670 comp rx=0 tx=0).stop 2024-04-16T17:10:49.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.274+0000 7fb810200640 1 -- 172.21.15.45:0/3204716486 shutdown_connections 2024-04-16T17:10:49.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.274+0000 7fb810200640 1 --2- 172.21.15.45:0/3204716486 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fb7f00a3d60 0x7fb7f00a6220 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:49.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.274+0000 7fb810200640 1 --2- 172.21.15.45:0/3204716486 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb808076e50 0x7fb80819f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:49.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.274+0000 7fb810200640 1 --2- 172.21.15.45:0/3204716486 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb808112ee0 0x7fb8081a3860 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:49.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.274+0000 7fb810200640 1 --2- 172.21.15.45:0/3204716486 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb808076530 0x7fb80819efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:49.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.274+0000 7fb810200640 1 -- 172.21.15.45:0/3204716486 >> 172.21.15.45:0/3204716486 conn(0x7fb8080ffd90 msgr2=0x7fb808114630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:10:49.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.274+0000 7fb810200640 1 -- 172.21.15.45:0/3204716486 shutdown_connections 2024-04-16T17:10:49.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:49.275+0000 7fb810200640 1 -- 172.21.15.45:0/3204716486 wait complete. 2024-04-16T17:10:50.247 INFO:tasks.workunit.client.0.smithi045.stdout: 1 55423 1.24 MB/sec execute 52 sec latency 52.268 ms 2024-04-16T17:10:50.295 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-16T17:10:50.295 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T17:10:50.295 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-16T17:10:50.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.511+0000 7ff51f200640 1 -- 172.21.15.45:0/4085203646 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff520100e50 msgr2=0x7ff520101230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:50.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.511+0000 7ff51f200640 1 --2- 172.21.15.45:0/4085203646 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff520100e50 0x7ff520101230 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7ff510009900 tx=0x7ff51002bcd0 comp rx=0 tx=0).stop 2024-04-16T17:10:50.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.512+0000 7ff51f200640 1 -- 172.21.15.45:0/4085203646 shutdown_connections 2024-04-16T17:10:50.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.512+0000 7ff51f200640 1 --2- 172.21.15.45:0/4085203646 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff520101770 0x7ff520105c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:50.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.512+0000 7ff51f200640 1 --2- 172.21.15.45:0/4085203646 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff520106250 0x7ff5201066e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:50.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.512+0000 7ff51f200640 1 --2- 172.21.15.45:0/4085203646 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff520100e50 0x7ff520101230 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:50.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.512+0000 7ff51f200640 1 -- 172.21.15.45:0/4085203646 >> 172.21.15.45:0/4085203646 conn(0x7ff5200fc880 msgr2=0x7ff5200feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:10:50.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.512+0000 7ff51f200640 1 -- 172.21.15.45:0/4085203646 shutdown_connections 2024-04-16T17:10:50.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.512+0000 7ff51f200640 1 -- 172.21.15.45:0/4085203646 wait complete. 2024-04-16T17:10:50.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.513+0000 7ff51f200640 1 Processor -- start 2024-04-16T17:10:50.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.513+0000 7ff51f200640 1 -- start start 2024-04-16T17:10:50.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.513+0000 7ff51f200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff520100e50 0x7ff52019ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:50.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.513+0000 7ff51f200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff520101770 0x7ff52019f280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:50.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.513+0000 7ff51f200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff520106250 0x7ff5201a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:50.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.513+0000 7ff51f200640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7ff520118800 con 0x7ff520106250 2024-04-16T17:10:50.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.513+0000 7ff51f200640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7ff520118680 con 0x7ff520100e50 2024-04-16T17:10:50.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.513+0000 7ff51f200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7ff520118980 con 0x7ff520101770 2024-04-16T17:10:50.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.514+0000 7ff51de00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff520100e50 0x7ff52019ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:50.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.514+0000 7ff51de00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff520100e50 0x7ff52019ed40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:47746/0 (socket says 172.21.15.45:47746) 2024-04-16T17:10:50.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.514+0000 7ff51de00640 1 -- 172.21.15.45:0/1319569344 learned_addr learned my addr 172.21.15.45:0/1319569344 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:10:50.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.514+0000 7ff51d400640 1 --2- 172.21.15.45:0/1319569344 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff520101770 0x7ff52019f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:50.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.514+0000 7ff51e800640 1 --2- 172.21.15.45:0/1319569344 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff520106250 0x7ff5201a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:50.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.514+0000 7ff51de00640 1 -- 172.21.15.45:0/1319569344 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff520101770 msgr2=0x7ff52019f280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:50.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.514+0000 7ff51de00640 1 --2- 172.21.15.45:0/1319569344 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff520101770 0x7ff52019f280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:50.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.514+0000 7ff51de00640 1 -- 172.21.15.45:0/1319569344 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff520106250 msgr2=0x7ff5201a3600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:50.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.514+0000 7ff51de00640 1 --2- 172.21.15.45:0/1319569344 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff520106250 0x7ff5201a3600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:50.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.514+0000 7ff51de00640 1 -- 172.21.15.45:0/1319569344 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5201a3d30 con 0x7ff520100e50 2024-04-16T17:10:50.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.514+0000 7ff51de00640 1 --2- 172.21.15.45:0/1319569344 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff520100e50 0x7ff52019ed40 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7ff510005ab0 tx=0x7ff510004970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:10:50.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.515+0000 7ff516a00640 1 -- 172.21.15.45:0/1319569344 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff510043020 con 0x7ff520100e50 2024-04-16T17:10:50.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.515+0000 7ff51f200640 1 -- 172.21.15.45:0/1319569344 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5201a3fc0 con 0x7ff520100e50 2024-04-16T17:10:50.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.515+0000 7ff516a00640 1 -- 172.21.15.45:0/1319569344 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff5100054e0 con 0x7ff520100e50 2024-04-16T17:10:50.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.515+0000 7ff516a00640 1 -- 172.21.15.45:0/1319569344 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff510003bd0 con 0x7ff520100e50 2024-04-16T17:10:50.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.515+0000 7ff51f200640 1 -- 172.21.15.45:0/1319569344 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5201ab8b0 con 0x7ff520100e50 2024-04-16T17:10:50.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.518+0000 7ff516a00640 1 -- 172.21.15.45:0/1319569344 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7ff510046020 con 0x7ff520100e50 2024-04-16T17:10:50.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.520+0000 7ff5279fb740 1 Processor -- start 2024-04-16T17:10:50.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.520+0000 7ff5279fb740 1 -- start start 2024-04-16T17:10:50.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.520+0000 7ff516a00640 1 --2- 172.21.15.45:0/1319569344 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7ff5040a3d20 0x7ff5040a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:50.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.520+0000 7ff516a00640 1 -- 172.21.15.45:0/1319569344 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7ff5100f2a90 con 0x7ff520100e50 2024-04-16T17:10:50.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.520+0000 7ff5279fb740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55be3d59be90 0x55be3d59c270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:50.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.520+0000 7ff5279fb740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55be3d59c7b0 0x55be3d5a4e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:50.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.520+0000 7ff5279fb740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55be3d5a5390 0x55be3d5a7780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:50.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.520+0000 7ff5279fb740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55be3d500cb0 con 0x55be3d59c7b0 2024-04-16T17:10:50.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.520+0000 7ff5279fb740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55be3d4a7370 con 0x55be3d59be90 2024-04-16T17:10:50.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.520+0000 7ff5279fb740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55be3d4fcc70 con 0x55be3d5a5390 2024-04-16T17:10:50.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.520+0000 7ff51e800640 1 --2- 172.21.15.45:0/1319569344 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7ff5040a3d20 0x7ff5040a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:50.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.521+0000 7ff51de00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55be3d59be90 0x55be3d59c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:50.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.521+0000 7ff51de00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55be3d59be90 0x55be3d59c270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:47758/0 (socket says 172.21.15.45:47758) 2024-04-16T17:10:50.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.521+0000 7ff51de00640 1 -- 172.21.15.45:0/3278101050 learned_addr learned my addr 172.21.15.45:0/3278101050 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:10:50.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.521+0000 7ff51de00640 1 -- 172.21.15.45:0/3278101050 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55be3d5a5390 msgr2=0x55be3d5a7780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:10:50.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.521+0000 7ff51de00640 1 --2- 172.21.15.45:0/3278101050 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55be3d5a5390 0x55be3d5a7780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:50.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.521+0000 7ff51de00640 1 -- 172.21.15.45:0/3278101050 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55be3d59c7b0 msgr2=0x55be3d5a4e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:10:50.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.521+0000 7ff51e800640 1 --2- 172.21.15.45:0/1319569344 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7ff5040a3d20 0x7ff5040a61e0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7ff518005fd0 tx=0x7ff5180107a0 comp rx=0 tx=0).ready entity=mgr.25411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:10:50.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.522+0000 7ff51d400640 1 --2- 172.21.15.45:0/3278101050 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55be3d59c7b0 0x55be3d5a4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:50.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.522+0000 7ff51de00640 1 --2- 172.21.15.45:0/3278101050 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55be3d59c7b0 0x55be3d5a4e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:50.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.522+0000 7ff51de00640 1 -- 172.21.15.45:0/3278101050 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55be3d4c6910 con 0x55be3d59be90 2024-04-16T17:10:50.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.522+0000 7ff51d400640 1 --2- 172.21.15.45:0/3278101050 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55be3d59c7b0 0x55be3d5a4e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:10:50.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.522+0000 7ff51de00640 1 --2- 172.21.15.45:0/3278101050 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55be3d59be90 0x55be3d59c270 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7ff5100c6ed0 tx=0x7ff5100baaf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=97952edc281cc845 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:10:50.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.522+0000 7ff50b400640 1 -- 172.21.15.45:0/3278101050 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff510004c80 con 0x55be3d59be90 2024-04-16T17:10:50.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.522+0000 7ff50b400640 1 -- 172.21.15.45:0/3278101050 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff510004c80 con 0x55be3d59be90 2024-04-16T17:10:50.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.522+0000 7ff50b400640 1 -- 172.21.15.45:0/3278101050 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5100c7400 con 0x55be3d59be90 2024-04-16T17:10:50.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.523+0000 7ff5279fb740 1 -- 172.21.15.45:0/3278101050 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55be3d59be90 msgr2=0x55be3d59c270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:50.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.523+0000 7ff5279fb740 1 --2- 172.21.15.45:0/3278101050 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55be3d59be90 0x55be3d59c270 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7ff5100c6ed0 tx=0x7ff5100baaf0 comp rx=0 tx=0).stop 2024-04-16T17:10:50.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.524+0000 7ff5279fb740 1 -- 172.21.15.45:0/3278101050 shutdown_connections 2024-04-16T17:10:50.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.524+0000 7ff5279fb740 1 --2- 172.21.15.45:0/3278101050 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55be3d5a5390 0x55be3d5a7780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:50.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.524+0000 7ff5279fb740 1 --2- 172.21.15.45:0/3278101050 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55be3d59c7b0 0x55be3d5a4e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:50.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.524+0000 7ff5279fb740 1 --2- 172.21.15.45:0/3278101050 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55be3d59be90 0x55be3d59c270 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:50.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.524+0000 7ff5279fb740 1 -- 172.21.15.45:0/3278101050 >> 172.21.15.45:0/3278101050 conn(0x55be3d5965c0 msgr2=0x55be3d5969a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:10:50.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.524+0000 7ff5279fb740 1 -- 172.21.15.45:0/3278101050 shutdown_connections 2024-04-16T17:10:50.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.524+0000 7ff5279fb740 1 -- 172.21.15.45:0/3278101050 wait complete. 2024-04-16T17:10:50.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.525+0000 7ff5279fb740 1 -- start start 2024-04-16T17:10:50.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.525+0000 7ff5279fb740 1 Processor -- start 2024-04-16T17:10:50.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.526+0000 7ff5279fb740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55be3d59be90 0x55be3d5a2440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:50.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.526+0000 7ff5279fb740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55be3d59c7b0 0x55be3d5a2980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:50.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.526+0000 7ff5279fb740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55be3d5a5390 0x55be3d5a2ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:50.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.526+0000 7ff5279fb740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55be3d5a7f40 con 0x55be3d5a5390 2024-04-16T17:10:50.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.526+0000 7ff5279fb740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55be3d4ba730 con 0x55be3d59be90 2024-04-16T17:10:50.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.526+0000 7ff5279fb740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55be3d5a80c0 con 0x55be3d59c7b0 2024-04-16T17:10:50.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.526+0000 7ff5279fb740 10 client.0 _ll_register_callbacks cb 0x55be3d4891e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:10:50.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.526+0000 7ff51de00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55be3d59be90 0x55be3d5a2440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:50.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.526+0000 7ff51de00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55be3d59be90 0x55be3d5a2440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:47770/0 (socket says 172.21.15.45:47770) 2024-04-16T17:10:50.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.526+0000 7ff51de00640 1 -- 172.21.15.45:0/1158556252 learned_addr learned my addr 172.21.15.45:0/1158556252 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:10:50.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.527+0000 7ff51d400640 1 --2- 172.21.15.45:0/1158556252 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55be3d59c7b0 0x55be3d5a2980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:50.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.527+0000 7ff51de00640 1 -- 172.21.15.45:0/1158556252 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55be3d59c7b0 msgr2=0x55be3d5a2980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:10:50.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.527+0000 7ff51de00640 1 --2- 172.21.15.45:0/1158556252 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55be3d59c7b0 0x55be3d5a2980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:50.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.527+0000 7ff51de00640 1 -- 172.21.15.45:0/1158556252 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55be3d5a5390 msgr2=0x55be3d5a2ec0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-16T17:10:50.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.527+0000 7ff51e800640 1 --2- 172.21.15.45:0/1158556252 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55be3d5a5390 0x55be3d5a2ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:50.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.527+0000 7ff51de00640 1 --2- 172.21.15.45:0/1158556252 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55be3d5a5390 0x55be3d5a2ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:50.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.527+0000 7ff51de00640 1 -- 172.21.15.45:0/1158556252 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55be3d2db7e0 con 0x55be3d59be90 2024-04-16T17:10:50.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.527+0000 7ff51d400640 1 --2- 172.21.15.45:0/1158556252 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55be3d59c7b0 0x55be3d5a2980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:10:50.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.527+0000 7ff51de00640 1 --2- 172.21.15.45:0/1158556252 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55be3d59be90 0x55be3d5a2440 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7ff510000c00 tx=0x7ff5100a92c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f0e81f4eeb224315 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:10:50.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.527+0000 7ff4ffe00640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-04-16T17:10:50.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.528+0000 7ff51e800640 1 --2- 172.21.15.45:0/1158556252 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55be3d5a5390 0x55be3d5a2ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:10:50.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.528+0000 7ff4ffe00640 1 -- 172.21.15.45:0/1158556252 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5100db020 con 0x55be3d59be90 2024-04-16T17:10:50.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.528+0000 7ff4ffe00640 1 -- 172.21.15.45:0/1158556252 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7ff5100043c0 con 0x55be3d59be90 2024-04-16T17:10:50.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.528+0000 7ff5279fb740 1 -- 172.21.15.45:0/1158556252 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff510004c80 con 0x55be3d59be90 2024-04-16T17:10:50.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.528+0000 7ff4ffe00640 1 -- 172.21.15.45:0/1158556252 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7ff5100c7400 con 0x55be3d59be90 2024-04-16T17:10:50.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.528+0000 7ff5279fb740 10 client.26774 fetch_fsmap learned FSMap version 31 2024-04-16T17:10:50.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.528+0000 7ff5279fb740 1 -- 172.21.15.45:0/1158556252 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55be3d33bf70 con 0x55be3d59be90 2024-04-16T17:10:50.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.529+0000 7ff4ffe00640 1 -- 172.21.15.45:0/1158556252 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff510004c80 con 0x55be3d59be90 2024-04-16T17:10:50.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.529+0000 7ff4ffe00640 1 -- 172.21.15.45:0/1158556252 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 31) ==== 8366+0+0 (secure 0 0 0) 0x7ff5100c6f70 con 0x55be3d59be90 2024-04-16T17:10:50.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.529+0000 7ff5279fb740 10 client.26774 fetch_fsmap finished waiting for FSMap version 31 2024-04-16T17:10:50.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.529+0000 7ff5279fb740 10 client.26774 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:10:50.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.529+0000 7ff5279fb740 20 client.26774 populate_metadata read hostname 'smithi045' 2024-04-16T17:10:50.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.529+0000 7ff5279fb740 1 --2- 172.21.15.45:0/1158556252 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55be3d5b5250 0x55be3d5b7640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:50.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.529+0000 7ff5279fb740 4 client.26774 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:10:50.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.529+0000 7ff5279fb740 1 -- 172.21.15.45:0/1158556252 --> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55be3d50e6d0 con 0x55be3d5b5250 2024-04-16T17:10:50.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.529+0000 7ff51d400640 1 --2- 172.21.15.45:0/1158556252 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55be3d5b5250 0x55be3d5b7640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:50.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.530+0000 7ff4ffe00640 10 client.26774 ms_handle_connect on v2:172.21.15.104:6838/3660070592 2024-04-16T17:10:50.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.530+0000 7ff51d400640 1 --2- 172.21.15.45:0/1158556252 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55be3d5b5250 0x55be3d5b7640 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=88184b7545a505f6 server_cookie=62fc00d71147f0ed in_seq=0 out_seq=0 2024-04-16T17:10:50.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.531+0000 7ff4ffe00640 1 -- 172.21.15.45:0/1158556252 <== mds.0 v2:172.21.15.104:6838/3660070592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55be3d50e6d0 con 0x55be3d5b5250 2024-04-16T17:10:50.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.531+0000 7ff4ffe00640 10 client.26774 handle_command_reply: tid=0 2024-04-16T17:10:50.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.531+0000 7ff5279fb740 1 client.26774 shutdown 2024-04-16T17:10:50.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.531+0000 7ff5279fb740 1 -- 172.21.15.45:0/1158556252 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55be3d59be90 msgr2=0x55be3d5a2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:50.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.531+0000 7ff5279fb740 1 --2- 172.21.15.45:0/1158556252 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55be3d59be90 0x55be3d5a2440 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7ff510000c00 tx=0x7ff5100a92c0 comp rx=0 tx=0).stop 2024-04-16T17:10:50.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.531+0000 7ff5279fb740 1 -- 172.21.15.45:0/1158556252 shutdown_connections 2024-04-16T17:10:50.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.531+0000 7ff5279fb740 1 --2- 172.21.15.45:0/1158556252 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55be3d5b5250 0x55be3d5b7640 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:50.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.531+0000 7ff5279fb740 1 --2- 172.21.15.45:0/1158556252 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55be3d59c7b0 0x55be3d5a2980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:50.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.531+0000 7ff5279fb740 1 --2- 172.21.15.45:0/1158556252 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55be3d5a5390 0x55be3d5a2ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:50.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.531+0000 7ff5279fb740 1 --2- 172.21.15.45:0/1158556252 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55be3d59be90 0x55be3d5a2440 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:50.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.531+0000 7ff4ffe00640 0 client.26774 ms_handle_reset on v2:172.21.15.104:6838/3660070592 2024-04-16T17:10:50.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.531+0000 7ff5279fb740 1 -- 172.21.15.45:0/1158556252 >> 172.21.15.45:0/1158556252 conn(0x55be3d5965c0 msgr2=0x55be3d598310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:10:50.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.531+0000 7ff5279fb740 1 -- 172.21.15.45:0/1158556252 shutdown_connections 2024-04-16T17:10:50.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.531+0000 7ff5279fb740 1 -- 172.21.15.45:0/1158556252 wait complete. 2024-04-16T17:10:50.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.532+0000 7ff5279fb740 20 client.26774 trim_cache size 0 max 16384 2024-04-16T17:10:50.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.556+0000 7ff5279fb740 1 Processor -- start 2024-04-16T17:10:50.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.556+0000 7ff5279fb740 1 -- start start 2024-04-16T17:10:50.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.557+0000 7ff5279fb740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55be3d59be90 0x55be3d5a4b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:50.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.557+0000 7ff5279fb740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55be3d59c7b0 0x55be3d5b0cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:50.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.557+0000 7ff5279fb740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55be3d5a5390 0x55be3d5953e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:50.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.557+0000 7ff5279fb740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7ff5201b8220 con 0x55be3d5a5390 2024-04-16T17:10:50.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.557+0000 7ff5279fb740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7ff5201b5a30 con 0x55be3d59be90 2024-04-16T17:10:50.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.557+0000 7ff5279fb740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7ff5201b3860 con 0x55be3d59c7b0 2024-04-16T17:10:50.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.557+0000 7ff51de00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55be3d59be90 0x55be3d5a4b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:50.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.557+0000 7ff51de00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55be3d59be90 0x55be3d5a4b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:47778/0 (socket says 172.21.15.45:47778) 2024-04-16T17:10:50.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.557+0000 7ff51de00640 1 -- 172.21.15.45:0/634588930 learned_addr learned my addr 172.21.15.45:0/634588930 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:10:50.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.557+0000 7ff51d400640 1 --2- 172.21.15.45:0/634588930 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55be3d59c7b0 0x55be3d5b0cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:50.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.557+0000 7ff51e800640 1 --2- 172.21.15.45:0/634588930 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55be3d5a5390 0x55be3d5953e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:50.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.557+0000 7ff51de00640 1 -- 172.21.15.45:0/634588930 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55be3d59c7b0 msgr2=0x55be3d5b0cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:10:50.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.558+0000 7ff51de00640 1 --2- 172.21.15.45:0/634588930 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55be3d59c7b0 0x55be3d5b0cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:50.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.558+0000 7ff51de00640 1 -- 172.21.15.45:0/634588930 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55be3d5a5390 msgr2=0x55be3d5953e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:10:50.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.558+0000 7ff51de00640 1 --2- 172.21.15.45:0/634588930 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55be3d5a5390 0x55be3d5953e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:50.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.558+0000 7ff51de00640 1 -- 172.21.15.45:0/634588930 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55be3d1d3170 con 0x55be3d59be90 2024-04-16T17:10:50.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.558+0000 7ff51de00640 1 --2- 172.21.15.45:0/634588930 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55be3d59be90 0x55be3d5a4b70 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7ff5100bb160 tx=0x7ff5100be400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fc8a9573ea8d64b6 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:10:50.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.558+0000 7ff50aa00640 1 -- 172.21.15.45:0/634588930 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5100a2070 con 0x55be3d59be90 2024-04-16T17:10:50.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.558+0000 7ff50aa00640 1 -- 172.21.15.45:0/634588930 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff510004c80 con 0x55be3d59be90 2024-04-16T17:10:50.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.558+0000 7ff50aa00640 1 -- 172.21.15.45:0/634588930 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5100db070 con 0x55be3d59be90 2024-04-16T17:10:50.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.559+0000 7ff5279fb740 1 -- 172.21.15.45:0/634588930 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55be3d59be90 msgr2=0x55be3d5a4b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:50.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.559+0000 7ff5279fb740 1 --2- 172.21.15.45:0/634588930 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55be3d59be90 0x55be3d5a4b70 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7ff5100bb160 tx=0x7ff5100be400 comp rx=0 tx=0).stop 2024-04-16T17:10:50.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.559+0000 7ff5279fb740 1 -- 172.21.15.45:0/634588930 shutdown_connections 2024-04-16T17:10:50.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.559+0000 7ff5279fb740 1 --2- 172.21.15.45:0/634588930 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55be3d59c7b0 0x55be3d5b0cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:50.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.559+0000 7ff5279fb740 1 --2- 172.21.15.45:0/634588930 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55be3d5a5390 0x55be3d5953e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:50.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.559+0000 7ff5279fb740 1 --2- 172.21.15.45:0/634588930 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55be3d59be90 0x55be3d5a4b70 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:50.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.559+0000 7ff5279fb740 1 -- 172.21.15.45:0/634588930 >> 172.21.15.45:0/634588930 conn(0x55be3d5965c0 msgr2=0x55be3d594cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:10:50.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.560+0000 7ff5279fb740 1 -- 172.21.15.45:0/634588930 shutdown_connections 2024-04-16T17:10:50.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.560+0000 7ff5279fb740 1 -- 172.21.15.45:0/634588930 wait complete. 2024-04-16T17:10:50.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.560+0000 7ff5279fb740 1 -- start start 2024-04-16T17:10:50.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.560+0000 7ff5279fb740 1 Processor -- start 2024-04-16T17:10:50.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.561+0000 7ff5279fb740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55be3d59be90 0x55be3d5ad6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:50.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.561+0000 7ff5279fb740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55be3d59c7b0 0x55be3d59efd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:50.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.561+0000 7ff5279fb740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55be3d5a5390 0x55be3d59f710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:50.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.561+0000 7ff5279fb740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7ff5201b3ef0 con 0x55be3d59c7b0 2024-04-16T17:10:50.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.561+0000 7ff5279fb740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7ff5201b5360 con 0x55be3d59be90 2024-04-16T17:10:50.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.561+0000 7ff5279fb740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7ff5201b45c0 con 0x55be3d5a5390 2024-04-16T17:10:50.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.561+0000 7ff5279fb740 10 client.0 _ll_register_callbacks cb 0x7ff5100be350 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:10:50.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.561+0000 7ff51de00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55be3d59be90 0x55be3d5ad6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:50.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.561+0000 7ff51de00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55be3d59be90 0x55be3d5ad6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:47780/0 (socket says 172.21.15.45:47780) 2024-04-16T17:10:50.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.561+0000 7ff51de00640 1 -- 172.21.15.45:0/1279207046 learned_addr learned my addr 172.21.15.45:0/1279207046 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:10:50.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.561+0000 7ff51e800640 1 --2- 172.21.15.45:0/1279207046 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55be3d5a5390 0x55be3d59f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:50.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.561+0000 7ff51d400640 1 --2- 172.21.15.45:0/1279207046 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55be3d59c7b0 0x55be3d59efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:50.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.562+0000 7ff51de00640 1 -- 172.21.15.45:0/1279207046 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55be3d5a5390 msgr2=0x55be3d59f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:10:50.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.562+0000 7ff51de00640 1 --2- 172.21.15.45:0/1279207046 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55be3d5a5390 0x55be3d59f710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:50.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.562+0000 7ff51de00640 1 -- 172.21.15.45:0/1279207046 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55be3d59c7b0 msgr2=0x55be3d59efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:10:50.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.562+0000 7ff51de00640 1 --2- 172.21.15.45:0/1279207046 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55be3d59c7b0 0x55be3d59efd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:50.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.562+0000 7ff51de00640 1 -- 172.21.15.45:0/1279207046 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55be3d61a710 con 0x55be3d59be90 2024-04-16T17:10:50.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.562+0000 7ff51de00640 1 --2- 172.21.15.45:0/1279207046 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55be3d59be90 0x55be3d5ad6c0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7ff5100be720 tx=0x7ff5100045f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=22e2142145a75b9e server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:10:50.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.562+0000 7ff4ffe00640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-04-16T17:10:50.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.562+0000 7ff4ffe00640 1 -- 172.21.15.45:0/1279207046 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5100db070 con 0x55be3d59be90 2024-04-16T17:10:50.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.562+0000 7ff4ffe00640 1 -- 172.21.15.45:0/1279207046 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7ff5100bca90 con 0x55be3d59be90 2024-04-16T17:10:50.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.562+0000 7ff5279fb740 1 -- 172.21.15.45:0/1279207046 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff510004c80 con 0x55be3d59be90 2024-04-16T17:10:50.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.563+0000 7ff4ffe00640 1 -- 172.21.15.45:0/1279207046 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7ff510005690 con 0x55be3d59be90 2024-04-16T17:10:50.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.563+0000 7ff5279fb740 10 client.26780 fetch_fsmap learned FSMap version 31 2024-04-16T17:10:50.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.563+0000 7ff5279fb740 1 -- 172.21.15.45:0/1279207046 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55be3d61a710 con 0x55be3d59be90 2024-04-16T17:10:50.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.563+0000 7ff4ffe00640 1 -- 172.21.15.45:0/1279207046 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff510004c80 con 0x55be3d59be90 2024-04-16T17:10:50.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.563+0000 7ff4ffe00640 1 -- 172.21.15.45:0/1279207046 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 31) ==== 8366+0+0 (secure 0 0 0) 0x7ff5100bd650 con 0x55be3d59be90 2024-04-16T17:10:50.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.563+0000 7ff5279fb740 10 client.26780 fetch_fsmap finished waiting for FSMap version 31 2024-04-16T17:10:50.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.563+0000 7ff5279fb740 10 client.26780 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:10:50.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.563+0000 7ff5279fb740 20 client.26780 populate_metadata read hostname 'smithi045' 2024-04-16T17:10:50.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.563+0000 7ff5279fb740 1 --2- 172.21.15.45:0/1279207046 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55be3d61bc10 0x55be3d61e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:50.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.563+0000 7ff5279fb740 4 client.26780 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-16T17:10:50.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.563+0000 7ff5279fb740 1 -- 172.21.15.45:0/1279207046 --> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55be3d508540 con 0x55be3d61bc10 2024-04-16T17:10:50.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.564+0000 7ff51d400640 1 --2- 172.21.15.45:0/1279207046 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55be3d61bc10 0x55be3d61e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:50.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.565+0000 7ff51d400640 1 --2- 172.21.15.45:0/1279207046 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55be3d61bc10 0x55be3d61e000 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fc48aed41e7d1bd8 server_cookie=ae7464ef7d4e7bde in_seq=0 out_seq=0 2024-04-16T17:10:50.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.565+0000 7ff4ffe00640 10 client.26780 ms_handle_connect on v2:172.21.15.104:6838/3660070592 2024-04-16T17:10:50.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.566+0000 7ff4ffe00640 1 -- 172.21.15.45:0/1279207046 <== mds.0 v2:172.21.15.104:6838/3660070592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55be3d508540 con 0x55be3d61bc10 2024-04-16T17:10:50.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.566+0000 7ff4ffe00640 10 client.26780 handle_command_reply: tid=0 2024-04-16T17:10:50.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.566+0000 7ff5279fb740 1 client.26780 shutdown 2024-04-16T17:10:50.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.566+0000 7ff5279fb740 1 -- 172.21.15.45:0/1279207046 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55be3d59be90 msgr2=0x55be3d5ad6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:50.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.566+0000 7ff5279fb740 1 --2- 172.21.15.45:0/1279207046 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55be3d59be90 0x55be3d5ad6c0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7ff5100be720 tx=0x7ff5100045f0 comp rx=0 tx=0).stop 2024-04-16T17:10:50.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.566+0000 7ff5279fb740 1 -- 172.21.15.45:0/1279207046 shutdown_connections 2024-04-16T17:10:50.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.566+0000 7ff5279fb740 1 --2- 172.21.15.45:0/1279207046 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55be3d61bc10 0x55be3d61e000 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:50.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.566+0000 7ff5279fb740 1 --2- 172.21.15.45:0/1279207046 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55be3d5a5390 0x55be3d59f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:50.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.566+0000 7ff5279fb740 1 --2- 172.21.15.45:0/1279207046 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55be3d59c7b0 0x55be3d59efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:50.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.566+0000 7ff5279fb740 1 --2- 172.21.15.45:0/1279207046 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55be3d59be90 0x55be3d5ad6c0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:50.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.566+0000 7ff4ffe00640 0 client.26780 ms_handle_reset on v2:172.21.15.104:6838/3660070592 2024-04-16T17:10:50.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.566+0000 7ff5279fb740 1 -- 172.21.15.45:0/1279207046 >> 172.21.15.45:0/1279207046 conn(0x55be3d5965c0 msgr2=0x55be3d5a2c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:10:50.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.566+0000 7ff5279fb740 1 -- 172.21.15.45:0/1279207046 shutdown_connections 2024-04-16T17:10:50.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.566+0000 7ff5279fb740 1 -- 172.21.15.45:0/1279207046 wait complete. 2024-04-16T17:10:50.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.567+0000 7ff5279fb740 20 client.26780 trim_cache size 0 max 16384 2024-04-16T17:10:50.568 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-16T17:10:50.568 INFO:teuthology.orchestra.run.smithi045.stdout: "return_code": 0, 2024-04-16T17:10:50.568 INFO:teuthology.orchestra.run.smithi045.stdout: "scrub_tag": "67516e11-3f36-4d52-bf46-ecb24b234808", 2024-04-16T17:10:50.568 INFO:teuthology.orchestra.run.smithi045.stdout: "mode": "asynchronous" 2024-04-16T17:10:50.568 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-16T17:10:50.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.567+0000 7ff51f200640 1 -- 172.21.15.45:0/1319569344 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7ff5040a3d20 msgr2=0x7ff5040a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:50.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.567+0000 7ff51f200640 1 --2- 172.21.15.45:0/1319569344 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7ff5040a3d20 0x7ff5040a61e0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7ff518005fd0 tx=0x7ff5180107a0 comp rx=0 tx=0).stop 2024-04-16T17:10:50.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.567+0000 7ff51f200640 1 -- 172.21.15.45:0/1319569344 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff520100e50 msgr2=0x7ff52019ed40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:50.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.567+0000 7ff51f200640 1 --2- 172.21.15.45:0/1319569344 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff520100e50 0x7ff52019ed40 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7ff510005ab0 tx=0x7ff510004970 comp rx=0 tx=0).stop 2024-04-16T17:10:50.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.568+0000 7ff51f200640 1 -- 172.21.15.45:0/1319569344 shutdown_connections 2024-04-16T17:10:50.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.568+0000 7ff51f200640 1 --2- 172.21.15.45:0/1319569344 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7ff5040a3d20 0x7ff5040a61e0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:50.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.568+0000 7ff51f200640 1 --2- 172.21.15.45:0/1319569344 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff520101770 0x7ff52019f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:50.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.568+0000 7ff51f200640 1 --2- 172.21.15.45:0/1319569344 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff520106250 0x7ff5201a3600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:50.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.568+0000 7ff51f200640 1 --2- 172.21.15.45:0/1319569344 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff520100e50 0x7ff52019ed40 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:50.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.568+0000 7ff51f200640 1 -- 172.21.15.45:0/1319569344 >> 172.21.15.45:0/1319569344 conn(0x7ff5200fc880 msgr2=0x7ff5200fd940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:10:50.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.568+0000 7ff51f200640 1 -- 172.21.15.45:0/1319569344 shutdown_connections 2024-04-16T17:10:50.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.568+0000 7ff51f200640 1 -- 172.21.15.45:0/1319569344 wait complete. 2024-04-16T17:10:50.595 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-16T17:10:50.595 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-16T17:10:50.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.800+0000 7f985a200640 1 -- 172.21.15.45:0/2946599291 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f98541044b0 msgr2=0x7f9854104890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:50.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.800+0000 7f985a200640 1 --2- 172.21.15.45:0/2946599291 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f98541044b0 0x7f9854104890 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f9844009860 tx=0x7f984402bb70 comp rx=0 tx=0).stop 2024-04-16T17:10:50.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.800+0000 7f985a200640 1 -- 172.21.15.45:0/2946599291 shutdown_connections 2024-04-16T17:10:50.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.800+0000 7f985a200640 1 --2- 172.21.15.45:0/2946599291 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9854109580 0x7f9854111150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:50.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.800+0000 7f985a200640 1 --2- 172.21.15.45:0/2946599291 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9854104e60 0x7f9854108ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:50.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.800+0000 7f985a200640 1 --2- 172.21.15.45:0/2946599291 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f98541044b0 0x7f9854104890 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:50.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.800+0000 7f985a200640 1 -- 172.21.15.45:0/2946599291 >> 172.21.15.45:0/2946599291 conn(0x7f98540ffd90 msgr2=0x7f98541021b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:10:50.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.800+0000 7f985a200640 1 -- 172.21.15.45:0/2946599291 shutdown_connections 2024-04-16T17:10:50.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.800+0000 7f985a200640 1 -- 172.21.15.45:0/2946599291 wait complete. 2024-04-16T17:10:50.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.801+0000 7f985a200640 1 Processor -- start 2024-04-16T17:10:50.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.802+0000 7f985a200640 1 -- start start 2024-04-16T17:10:50.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.802+0000 7f985a200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f98541044b0 0x7f98541a0890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:50.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.802+0000 7f985a200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9854109580 0x7f98541a0dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:50.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.802+0000 7f985a200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f985419aa90 0x7f985419af40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:50.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.802+0000 7f985a200640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f9854114660 con 0x7f9854109580 2024-04-16T17:10:50.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.802+0000 7f985a200640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f98541144e0 con 0x7f98541044b0 2024-04-16T17:10:50.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.802+0000 7f985a200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f98541147e0 con 0x7f985419aa90 2024-04-16T17:10:50.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.802+0000 7f9853400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f98541044b0 0x7f98541a0890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:50.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.802+0000 7f9853400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f98541044b0 0x7f98541a0890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:47784/0 (socket says 172.21.15.45:47784) 2024-04-16T17:10:50.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.802+0000 7f9853400640 1 -- 172.21.15.45:0/3067518888 learned_addr learned my addr 172.21.15.45:0/3067518888 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:10:50.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.803+0000 7f9853e00640 1 --2- 172.21.15.45:0/3067518888 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f985419aa90 0x7f985419af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:50.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.803+0000 7f9852a00640 1 --2- 172.21.15.45:0/3067518888 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9854109580 0x7f98541a0dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:50.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.803+0000 7f9853e00640 1 -- 172.21.15.45:0/3067518888 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f98541044b0 msgr2=0x7f98541a0890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:50.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.803+0000 7f9853e00640 1 --2- 172.21.15.45:0/3067518888 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f98541044b0 0x7f98541a0890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:50.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.803+0000 7f9853e00640 1 -- 172.21.15.45:0/3067518888 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9854109580 msgr2=0x7f98541a0dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:50.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.803+0000 7f9853e00640 1 --2- 172.21.15.45:0/3067518888 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9854109580 0x7f98541a0dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:50.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.803+0000 7f9853e00640 1 -- 172.21.15.45:0/3067518888 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f985419b5b0 con 0x7f985419aa90 2024-04-16T17:10:50.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:10:50 smithi117 ceph-mon[36730]: pgmap v125: 129 pgs: 129 active+clean; 36 MiB data, 485 MiB used, 1.0 TiB / 1.0 TiB avail; 470 KiB/s rd, 288 KiB/s wr, 65 op/s 2024-04-16T17:10:50.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.804+0000 7f9853e00640 1 --2- 172.21.15.45:0/3067518888 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f985419aa90 0x7f985419af40 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f984c00ef00 tx=0x7f984c00c740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:10:50.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.804+0000 7f984be00640 1 -- 172.21.15.45:0/3067518888 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f984c004b20 con 0x7f985419aa90 2024-04-16T17:10:50.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.804+0000 7f984be00640 1 -- 172.21.15.45:0/3067518888 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f984c003e50 con 0x7f985419aa90 2024-04-16T17:10:50.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.804+0000 7f984be00640 1 -- 172.21.15.45:0/3067518888 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f984c00b570 con 0x7f985419aa90 2024-04-16T17:10:50.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.804+0000 7f985a200640 1 -- 172.21.15.45:0/3067518888 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f985419b8a0 con 0x7f985419aa90 2024-04-16T17:10:50.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.804+0000 7f985a200640 1 -- 172.21.15.45:0/3067518888 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98541a7a50 con 0x7f985419aa90 2024-04-16T17:10:50.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.809+0000 7f985cfef740 1 Processor -- start 2024-04-16T17:10:50.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.809+0000 7f984be00640 1 -- 172.21.15.45:0/3067518888 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f984c01d020 con 0x7f985419aa90 2024-04-16T17:10:50.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.809+0000 7f985cfef740 1 -- start start 2024-04-16T17:10:50.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.810+0000 7f985cfef740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558f5372f9c0 0x558f5372fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:50.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.810+0000 7f985cfef740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558f537302e0 0x558f53738980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:50.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.810+0000 7f985cfef740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558f53738ec0 0x558f5373b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:50.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.810+0000 7f985cfef740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x558f5368d120 con 0x558f537302e0 2024-04-16T17:10:50.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.810+0000 7f985cfef740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x558f53633340 con 0x558f53738ec0 2024-04-16T17:10:50.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.810+0000 7f985cfef740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x558f536890e0 con 0x558f5372f9c0 2024-04-16T17:10:50.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.810+0000 7f9852a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558f5372f9c0 0x558f5372fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:50.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.810+0000 7f9852a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558f5372f9c0 0x558f5372fda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:36034/0 (socket says 172.21.15.45:36034) 2024-04-16T17:10:50.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.810+0000 7f9852a00640 1 -- 172.21.15.45:0/2948767356 learned_addr learned my addr 172.21.15.45:0/2948767356 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:10:50.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.810+0000 7f9853400640 1 --2- 172.21.15.45:0/2948767356 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558f537302e0 0x558f53738980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:50.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.810+0000 7f9852a00640 1 --2- 172.21.15.45:0/2948767356 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558f53738ec0 0x558f5373b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:50.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.811+0000 7f9853400640 1 -- 172.21.15.45:0/2948767356 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558f5372f9c0 msgr2=0x558f5372fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:10:50.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.811+0000 7f9853400640 1 --2- 172.21.15.45:0/2948767356 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558f5372f9c0 0x558f5372fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:50.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.811+0000 7f9853400640 1 -- 172.21.15.45:0/2948767356 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558f53738ec0 msgr2=0x558f5373b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:10:50.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.811+0000 7f9853400640 1 --2- 172.21.15.45:0/2948767356 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558f53738ec0 0x558f5373b2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:50.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.811+0000 7f9853400640 1 -- 172.21.15.45:0/2948767356 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558f534697d0 con 0x558f537302e0 2024-04-16T17:10:50.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.812+0000 7f9853400640 1 --2- 172.21.15.45:0/2948767356 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558f537302e0 0x558f53738980 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f9854100190 tx=0x7f9844009520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32fb729768535de3 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:10:50.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.813+0000 7f983fe00640 1 -- 172.21.15.45:0/2948767356 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9844043020 con 0x558f537302e0 2024-04-16T17:10:50.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.813+0000 7f983fe00640 1 -- 172.21.15.45:0/2948767356 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9844034cb0 con 0x558f537302e0 2024-04-16T17:10:50.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.813+0000 7f983fe00640 1 -- 172.21.15.45:0/2948767356 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9844039040 con 0x558f537302e0 2024-04-16T17:10:50.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.813+0000 7f984be00640 1 --2- 172.21.15.45:0/3067518888 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f98380a3d60 0x7f98380a6220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:50.814 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.814+0000 7f985cfef740 1 -- 172.21.15.45:0/2948767356 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558f537302e0 msgr2=0x558f53738980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:50.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.814+0000 7f985cfef740 1 --2- 172.21.15.45:0/2948767356 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558f537302e0 0x558f53738980 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f9854100190 tx=0x7f9844009520 comp rx=0 tx=0).stop 2024-04-16T17:10:50.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.814+0000 7f985cfef740 1 -- 172.21.15.45:0/2948767356 shutdown_connections 2024-04-16T17:10:50.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.814+0000 7f985cfef740 1 --2- 172.21.15.45:0/2948767356 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558f53738ec0 0x558f5373b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:50.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.814+0000 7f985cfef740 1 --2- 172.21.15.45:0/2948767356 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558f537302e0 0x558f53738980 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:50.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.814+0000 7f985cfef740 1 --2- 172.21.15.45:0/2948767356 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558f5372f9c0 0x558f5372fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:50.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.814+0000 7f985cfef740 1 -- 172.21.15.45:0/2948767356 >> 172.21.15.45:0/2948767356 conn(0x558f5372a0f0 msgr2=0x558f5372a4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:10:50.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.814+0000 7f984be00640 1 -- 172.21.15.45:0/3067518888 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f984c0d09e0 con 0x7f985419aa90 2024-04-16T17:10:50.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.814+0000 7f985cfef740 1 -- 172.21.15.45:0/2948767356 shutdown_connections 2024-04-16T17:10:50.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.814+0000 7f9852a00640 1 --2- 172.21.15.45:0/3067518888 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f98380a3d60 0x7f98380a6220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:50.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.814+0000 7f985cfef740 1 -- 172.21.15.45:0/2948767356 wait complete. 2024-04-16T17:10:50.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.815+0000 7f985cfef740 1 -- start start 2024-04-16T17:10:50.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.815+0000 7f985cfef740 1 Processor -- start 2024-04-16T17:10:50.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.815+0000 7f9852a00640 1 --2- 172.21.15.45:0/3067518888 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f98380a3d60 0x7f98380a6220 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x558f5372b280 tx=0x7f9840005f90 comp rx=0 tx=0).ready entity=mgr.25411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:10:50.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.816+0000 7f985cfef740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558f5372f9c0 0x558f53735fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:50.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.816+0000 7f985cfef740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558f537302e0 0x558f53736500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:50.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.816+0000 7f985cfef740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558f53738ec0 0x558f53736a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:50.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.816+0000 7f985cfef740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x558f5373bc50 con 0x558f5372f9c0 2024-04-16T17:10:50.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.816+0000 7f985cfef740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x558f53646710 con 0x558f53738ec0 2024-04-16T17:10:50.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.816+0000 7f985cfef740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x558f5373bdd0 con 0x558f537302e0 2024-04-16T17:10:50.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.816+0000 7f985cfef740 10 client.0 _ll_register_callbacks cb 0x558f53641af0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:10:50.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.816+0000 7f9853400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558f5372f9c0 0x558f53735fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:50.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.816+0000 7f9853400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558f5372f9c0 0x558f53735fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33152/0 (socket says 172.21.15.45:33152) 2024-04-16T17:10:50.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.816+0000 7f9853400640 1 -- 172.21.15.45:0/88717665 learned_addr learned my addr 172.21.15.45:0/88717665 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:10:50.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.816+0000 7f9853e00640 1 --2- 172.21.15.45:0/88717665 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558f53738ec0 0x558f53736a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:50.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.816+0000 7f9852a00640 1 --2- 172.21.15.45:0/88717665 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558f537302e0 0x558f53736500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:50.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.817+0000 7f9853400640 1 -- 172.21.15.45:0/88717665 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558f537302e0 msgr2=0x558f53736500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:10:50.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.817+0000 7f9853400640 1 --2- 172.21.15.45:0/88717665 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558f537302e0 0x558f53736500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:50.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.817+0000 7f9853400640 1 -- 172.21.15.45:0/88717665 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558f53738ec0 msgr2=0x558f53736a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:10:50.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.817+0000 7f9853400640 1 --2- 172.21.15.45:0/88717665 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558f53738ec0 0x558f53736a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:50.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.817+0000 7f9853400640 1 -- 172.21.15.45:0/88717665 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558f5339f3f0 con 0x558f5372f9c0 2024-04-16T17:10:50.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.817+0000 7f9853e00640 1 --2- 172.21.15.45:0/88717665 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558f53738ec0 0x558f53736a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:10:50.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.817+0000 7f9853400640 1 --2- 172.21.15.45:0/88717665 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558f5372f9c0 0x558f53735fc0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f9844006010 tx=0x7f9844034940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcc4201bad230f09 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:10:50.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.817+0000 7f983cc00640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-16T17:10:50.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.818+0000 7f983cc00640 1 -- 172.21.15.45:0/88717665 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9844043030 con 0x558f5372f9c0 2024-04-16T17:10:50.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.818+0000 7f983cc00640 1 -- 172.21.15.45:0/88717665 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f984402c510 con 0x558f5372f9c0 2024-04-16T17:10:50.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.818+0000 7f985cfef740 1 -- 172.21.15.45:0/88717665 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9844034cb0 con 0x558f5372f9c0 2024-04-16T17:10:50.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.818+0000 7f983cc00640 1 -- 172.21.15.45:0/88717665 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f9844039040 con 0x558f5372f9c0 2024-04-16T17:10:50.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.818+0000 7f985cfef740 10 client.16638 fetch_fsmap learned FSMap version 31 2024-04-16T17:10:50.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.818+0000 7f985cfef740 1 -- 172.21.15.45:0/88717665 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x558f5335f0f0 con 0x558f5372f9c0 2024-04-16T17:10:50.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.819+0000 7f983cc00640 1 -- 172.21.15.45:0/88717665 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9844034cb0 con 0x558f5372f9c0 2024-04-16T17:10:50.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.819+0000 7f983cc00640 1 -- 172.21.15.45:0/88717665 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 31) ==== 8366+0+0 (secure 0 0 0) 0x7f98440329c0 con 0x558f5372f9c0 2024-04-16T17:10:50.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.819+0000 7f985cfef740 10 client.16638 fetch_fsmap finished waiting for FSMap version 31 2024-04-16T17:10:50.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.819+0000 7f985cfef740 10 client.16638 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:10:50.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.819+0000 7f985cfef740 20 client.16638 populate_metadata read hostname 'smithi045' 2024-04-16T17:10:50.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.819+0000 7f985cfef740 1 --2- 172.21.15.45:0/88717665 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x558f53749150 0x558f5374b540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:50.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.820+0000 7f985cfef740 4 client.16638 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:10:50.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.820+0000 7f985cfef740 1 -- 172.21.15.45:0/88717665 --> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558f5369ab40 con 0x558f53749150 2024-04-16T17:10:50.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.820+0000 7f9852a00640 1 --2- 172.21.15.45:0/88717665 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x558f53749150 0x558f5374b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:50.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.820+0000 7f9852a00640 1 --2- 172.21.15.45:0/88717665 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x558f53749150 0x558f5374b540 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3d62af843e584c11 server_cookie=525f4e191484b951 in_seq=0 out_seq=0 2024-04-16T17:10:50.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.820+0000 7f983cc00640 10 client.16638 ms_handle_connect on v2:172.21.15.104:6838/3660070592 2024-04-16T17:10:50.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.821+0000 7f983cc00640 1 -- 172.21.15.45:0/88717665 <== mds.0 v2:172.21.15.104:6838/3660070592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558f5369ab40 con 0x558f53749150 2024-04-16T17:10:50.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.821+0000 7f983cc00640 10 client.16638 handle_command_reply: tid=0 2024-04-16T17:10:50.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.821+0000 7f985cfef740 1 client.16638 shutdown 2024-04-16T17:10:50.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.821+0000 7f985cfef740 1 -- 172.21.15.45:0/88717665 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558f5372f9c0 msgr2=0x558f53735fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:50.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.822+0000 7f985cfef740 1 --2- 172.21.15.45:0/88717665 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558f5372f9c0 0x558f53735fc0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f9844006010 tx=0x7f9844034940 comp rx=0 tx=0).stop 2024-04-16T17:10:50.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.822+0000 7f985cfef740 1 -- 172.21.15.45:0/88717665 shutdown_connections 2024-04-16T17:10:50.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.822+0000 7f985cfef740 1 --2- 172.21.15.45:0/88717665 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x558f53749150 0x558f5374b540 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:50.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.822+0000 7f985cfef740 1 --2- 172.21.15.45:0/88717665 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558f537302e0 0x558f53736500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:50.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.822+0000 7f985cfef740 1 --2- 172.21.15.45:0/88717665 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558f53738ec0 0x558f53736a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:50.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.822+0000 7f985cfef740 1 --2- 172.21.15.45:0/88717665 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558f5372f9c0 0x558f53735fc0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:50.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.822+0000 7f985cfef740 1 -- 172.21.15.45:0/88717665 >> 172.21.15.45:0/88717665 conn(0x558f5372a0f0 msgr2=0x558f5372be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:10:50.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.822+0000 7f983cc00640 0 client.16638 ms_handle_reset on v2:172.21.15.104:6838/3660070592 2024-04-16T17:10:50.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.822+0000 7f985cfef740 1 -- 172.21.15.45:0/88717665 shutdown_connections 2024-04-16T17:10:50.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.822+0000 7f985cfef740 1 -- 172.21.15.45:0/88717665 wait complete. 2024-04-16T17:10:50.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.822+0000 7f985cfef740 20 client.16638 trim_cache size 0 max 16384 2024-04-16T17:10:50.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.845+0000 7f985cfef740 1 Processor -- start 2024-04-16T17:10:50.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.845+0000 7f985cfef740 1 -- start start 2024-04-16T17:10:50.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.846+0000 7f985cfef740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558f5372f9c0 0x558f53744b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:50.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.846+0000 7f985cfef740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558f537302e0 0x558f537208d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:50.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.846+0000 7f985cfef740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558f53738ec0 0x558f53736690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:50.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.846+0000 7f985cfef740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f98541afb00 con 0x558f5372f9c0 2024-04-16T17:10:50.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.846+0000 7f985cfef740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f98541ad310 con 0x558f53738ec0 2024-04-16T17:10:50.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.846+0000 7f985cfef740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f98541ab140 con 0x558f537302e0 2024-04-16T17:10:50.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.846+0000 7f9853e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558f53738ec0 0x558f53736690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:50.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.846+0000 7f9853400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558f5372f9c0 0x558f53744b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:50.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.846+0000 7f9852a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558f537302e0 0x558f537208d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:50.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.846+0000 7f9852a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558f537302e0 0x558f537208d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:36054/0 (socket says 172.21.15.45:36054) 2024-04-16T17:10:50.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.846+0000 7f9853e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558f53738ec0 0x558f53736690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:47816/0 (socket says 172.21.15.45:47816) 2024-04-16T17:10:50.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.846+0000 7f9852a00640 1 -- 172.21.15.45:0/672342972 learned_addr learned my addr 172.21.15.45:0/672342972 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:10:50.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.847+0000 7f9853e00640 1 -- 172.21.15.45:0/672342972 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558f537302e0 msgr2=0x558f537208d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:10:50.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.847+0000 7f9853e00640 1 --2- 172.21.15.45:0/672342972 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558f537302e0 0x558f537208d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:50.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.847+0000 7f9853e00640 1 -- 172.21.15.45:0/672342972 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558f5372f9c0 msgr2=0x558f53744b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:10:50.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.847+0000 7f9853e00640 1 --2- 172.21.15.45:0/672342972 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558f5372f9c0 0x558f53744b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:50.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.847+0000 7f9853e00640 1 -- 172.21.15.45:0/672342972 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558f53730110 con 0x558f53738ec0 2024-04-16T17:10:50.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.847+0000 7f9852a00640 1 --2- 172.21.15.45:0/672342972 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558f537302e0 0x558f537208d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:10:50.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.847+0000 7f9853400640 1 --2- 172.21.15.45:0/672342972 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558f5372f9c0 0x558f53744b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:10:50.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.847+0000 7f9853e00640 1 --2- 172.21.15.45:0/672342972 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558f53738ec0 0x558f53736690 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f984c0cdd80 tx=0x7f984c09c1d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=91e0dd9c3c6985eb server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:10:50.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.847+0000 7f983f400640 1 -- 172.21.15.45:0/672342972 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f984c01a040 con 0x558f53738ec0 2024-04-16T17:10:50.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.847+0000 7f983f400640 1 -- 172.21.15.45:0/672342972 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f984c09c5d0 con 0x558f53738ec0 2024-04-16T17:10:50.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.847+0000 7f983f400640 1 -- 172.21.15.45:0/672342972 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f984c098c40 con 0x558f53738ec0 2024-04-16T17:10:50.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.848+0000 7f985cfef740 1 -- 172.21.15.45:0/672342972 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558f53738ec0 msgr2=0x558f53736690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:50.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.848+0000 7f985cfef740 1 --2- 172.21.15.45:0/672342972 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558f53738ec0 0x558f53736690 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f984c0cdd80 tx=0x7f984c09c1d0 comp rx=0 tx=0).stop 2024-04-16T17:10:50.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.849+0000 7f985cfef740 1 -- 172.21.15.45:0/672342972 shutdown_connections 2024-04-16T17:10:50.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.849+0000 7f985cfef740 1 --2- 172.21.15.45:0/672342972 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558f537302e0 0x558f537208d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:50.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.849+0000 7f985cfef740 1 --2- 172.21.15.45:0/672342972 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558f53738ec0 0x558f53736690 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:50.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.849+0000 7f985cfef740 1 --2- 172.21.15.45:0/672342972 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558f5372f9c0 0x558f53744b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:50.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.849+0000 7f985cfef740 1 -- 172.21.15.45:0/672342972 >> 172.21.15.45:0/672342972 conn(0x558f5372a0f0 msgr2=0x558f53735f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:10:50.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.849+0000 7f985cfef740 1 -- 172.21.15.45:0/672342972 shutdown_connections 2024-04-16T17:10:50.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.849+0000 7f985cfef740 1 -- 172.21.15.45:0/672342972 wait complete. 2024-04-16T17:10:50.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.849+0000 7f985cfef740 1 -- start start 2024-04-16T17:10:50.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.850+0000 7f985cfef740 1 Processor -- start 2024-04-16T17:10:50.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.850+0000 7f985cfef740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558f5372f9c0 0x558f53741020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:50.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.850+0000 7f985cfef740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558f537302e0 0x558f53741560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:50.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.851+0000 7f985cfef740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558f53738ec0 0x558f5373dff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:50.852 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.851+0000 7f985cfef740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f98541ab7d0 con 0x558f5372f9c0 2024-04-16T17:10:50.852 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.851+0000 7f985cfef740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f98541acc40 con 0x558f537302e0 2024-04-16T17:10:50.852 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.851+0000 7f985cfef740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f98541abea0 con 0x558f53738ec0 2024-04-16T17:10:50.852 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.851+0000 7f985cfef740 10 client.0 _ll_register_callbacks cb 0x7f9840009e40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:10:50.852 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.851+0000 7f9852a00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558f537302e0 0x558f53741560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:50.852 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.851+0000 7f9852a00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558f537302e0 0x558f53741560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:47832/0 (socket says 172.21.15.45:47832) 2024-04-16T17:10:50.852 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.851+0000 7f9852a00640 1 -- 172.21.15.45:0/3065689648 learned_addr learned my addr 172.21.15.45:0/3065689648 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:10:50.852 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.851+0000 7f9853e00640 1 --2- 172.21.15.45:0/3065689648 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558f53738ec0 0x558f5373dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:50.852 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.851+0000 7f9853400640 1 --2- 172.21.15.45:0/3065689648 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558f5372f9c0 0x558f53741020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:50.852 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.851+0000 7f9852a00640 1 -- 172.21.15.45:0/3065689648 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558f53738ec0 msgr2=0x558f5373dff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:10:50.852 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.851+0000 7f9852a00640 1 --2- 172.21.15.45:0/3065689648 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558f53738ec0 0x558f5373dff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:50.852 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.851+0000 7f9852a00640 1 -- 172.21.15.45:0/3065689648 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558f5372f9c0 msgr2=0x558f53741020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:10:50.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.851+0000 7f9852a00640 1 --2- 172.21.15.45:0/3065689648 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558f5372f9c0 0x558f53741020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:50.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.851+0000 7f9852a00640 1 -- 172.21.15.45:0/3065689648 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558f5373e7c0 con 0x558f537302e0 2024-04-16T17:10:50.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.852+0000 7f9852a00640 1 --2- 172.21.15.45:0/3065689648 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558f537302e0 0x558f53741560 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x558f53735820 tx=0x7f98400046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7fd13bf18429f54f server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:10:50.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.852+0000 7f983cc00640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-04-16T17:10:50.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.853+0000 7f983cc00640 1 -- 172.21.15.45:0/3065689648 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9840011a40 con 0x558f537302e0 2024-04-16T17:10:50.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.853+0000 7f983cc00640 1 -- 172.21.15.45:0/3065689648 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f9840011be0 con 0x558f537302e0 2024-04-16T17:10:50.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.853+0000 7f985cfef740 1 -- 172.21.15.45:0/3065689648 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f984c09c5d0 con 0x558f537302e0 2024-04-16T17:10:50.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.853+0000 7f983cc00640 1 -- 172.21.15.45:0/3065689648 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f9840011de0 con 0x558f537302e0 2024-04-16T17:10:50.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.853+0000 7f985cfef740 10 client.26798 fetch_fsmap learned FSMap version 31 2024-04-16T17:10:50.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.853+0000 7f985cfef740 1 -- 172.21.15.45:0/3065689648 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x558f537b0010 con 0x558f537302e0 2024-04-16T17:10:50.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.854+0000 7f983cc00640 1 -- 172.21.15.45:0/3065689648 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f984c09c5d0 con 0x558f537302e0 2024-04-16T17:10:50.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.854+0000 7f983cc00640 1 -- 172.21.15.45:0/3065689648 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 31) ==== 8366+0+0 (secure 0 0 0) 0x7f98400377c0 con 0x558f537302e0 2024-04-16T17:10:50.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.854+0000 7f985cfef740 10 client.26798 fetch_fsmap finished waiting for FSMap version 31 2024-04-16T17:10:50.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.855+0000 7f985cfef740 10 client.26798 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:10:50.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.855+0000 7f985cfef740 20 client.26798 populate_metadata read hostname 'smithi045' 2024-04-16T17:10:50.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.855+0000 7f985cfef740 1 --2- 172.21.15.45:0/3065689648 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x558f537b0790 0x558f537b2bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:50.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.855+0000 7f985cfef740 4 client.26798 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-16T17:10:50.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.855+0000 7f985cfef740 1 -- 172.21.15.45:0/3065689648 --> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] -- command(tid 0: {"prefix": "scrub status"}) -- 0x558f536949b0 con 0x558f537b0790 2024-04-16T17:10:50.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.855+0000 7f9853400640 1 --2- 172.21.15.45:0/3065689648 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x558f537b0790 0x558f537b2bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:50.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.856+0000 7f9853400640 1 --2- 172.21.15.45:0/3065689648 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x558f537b0790 0x558f537b2bb0 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c9b1ed5159e2ad70 server_cookie=cece06eeb51d710f in_seq=0 out_seq=0 2024-04-16T17:10:50.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.856+0000 7f983cc00640 10 client.26798 ms_handle_connect on v2:172.21.15.104:6838/3660070592 2024-04-16T17:10:50.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.856+0000 7f983cc00640 1 -- 172.21.15.45:0/3065689648 <== mds.0 v2:172.21.15.104:6838/3660070592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x558f536949b0 con 0x558f537b0790 2024-04-16T17:10:50.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.856+0000 7f983cc00640 10 client.26798 handle_command_reply: tid=0 2024-04-16T17:10:50.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.857+0000 7f985cfef740 1 client.26798 shutdown 2024-04-16T17:10:50.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.857+0000 7f985cfef740 1 -- 172.21.15.45:0/3065689648 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558f537302e0 msgr2=0x558f53741560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:50.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.857+0000 7f985cfef740 1 --2- 172.21.15.45:0/3065689648 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558f537302e0 0x558f53741560 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x558f53735820 tx=0x7f98400046c0 comp rx=0 tx=0).stop 2024-04-16T17:10:50.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.858+0000 7f985cfef740 1 -- 172.21.15.45:0/3065689648 shutdown_connections 2024-04-16T17:10:50.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.858+0000 7f985cfef740 1 --2- 172.21.15.45:0/3065689648 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x558f537b0790 0x558f537b2bb0 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:50.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.858+0000 7f985cfef740 1 --2- 172.21.15.45:0/3065689648 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558f53738ec0 0x558f5373dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:50.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.858+0000 7f985cfef740 1 --2- 172.21.15.45:0/3065689648 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558f537302e0 0x558f53741560 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:50.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.858+0000 7f985cfef740 1 --2- 172.21.15.45:0/3065689648 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558f5372f9c0 0x558f53741020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:50.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.858+0000 7f983cc00640 0 client.26798 ms_handle_reset on v2:172.21.15.104:6838/3660070592 2024-04-16T17:10:50.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.858+0000 7f985cfef740 1 -- 172.21.15.45:0/3065689648 >> 172.21.15.45:0/3065689648 conn(0x558f5372a0f0 msgr2=0x558f53720750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:10:50.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.858+0000 7f985cfef740 1 -- 172.21.15.45:0/3065689648 shutdown_connections 2024-04-16T17:10:50.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.858+0000 7f985cfef740 1 -- 172.21.15.45:0/3065689648 wait complete. 2024-04-16T17:10:50.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.858+0000 7f985cfef740 20 client.26798 trim_cache size 0 max 16384 2024-04-16T17:10:50.859 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-16T17:10:50.860 INFO:teuthology.orchestra.run.smithi045.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-16T17:10:50.860 INFO:teuthology.orchestra.run.smithi045.stdout: "scrubs": { 2024-04-16T17:10:50.860 INFO:teuthology.orchestra.run.smithi045.stdout: "67516e11-3f36-4d52-bf46-ecb24b234808": { 2024-04-16T17:10:50.860 INFO:teuthology.orchestra.run.smithi045.stdout: "path": "/", 2024-04-16T17:10:50.860 INFO:teuthology.orchestra.run.smithi045.stdout: "tag": "67516e11-3f36-4d52-bf46-ecb24b234808", 2024-04-16T17:10:50.860 INFO:teuthology.orchestra.run.smithi045.stdout: "options": "recursive,force" 2024-04-16T17:10:50.860 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:10:50.860 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:10:50.860 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-16T17:10:50.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.859+0000 7f985a200640 1 -- 172.21.15.45:0/3067518888 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f98380a3d60 msgr2=0x7f98380a6220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:50.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.859+0000 7f985a200640 1 --2- 172.21.15.45:0/3067518888 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f98380a3d60 0x7f98380a6220 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x558f5372b280 tx=0x7f9840005f90 comp rx=0 tx=0).stop 2024-04-16T17:10:50.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.859+0000 7f985a200640 1 -- 172.21.15.45:0/3067518888 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f985419aa90 msgr2=0x7f985419af40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:50.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.859+0000 7f985a200640 1 --2- 172.21.15.45:0/3067518888 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f985419aa90 0x7f985419af40 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f984c00ef00 tx=0x7f984c00c740 comp rx=0 tx=0).stop 2024-04-16T17:10:50.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.859+0000 7f985a200640 1 -- 172.21.15.45:0/3067518888 shutdown_connections 2024-04-16T17:10:50.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.859+0000 7f985a200640 1 --2- 172.21.15.45:0/3067518888 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f98380a3d60 0x7f98380a6220 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:50.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.859+0000 7f985a200640 1 --2- 172.21.15.45:0/3067518888 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f985419aa90 0x7f985419af40 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:50.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.859+0000 7f985a200640 1 --2- 172.21.15.45:0/3067518888 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9854109580 0x7f98541a0dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:50.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.859+0000 7f985a200640 1 --2- 172.21.15.45:0/3067518888 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f98541044b0 0x7f98541a0890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:50.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.859+0000 7f985a200640 1 -- 172.21.15.45:0/3067518888 >> 172.21.15.45:0/3067518888 conn(0x7f98540ffd90 msgr2=0x7f98541010e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:10:50.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.859+0000 7f985a200640 1 -- 172.21.15.45:0/3067518888 shutdown_connections 2024-04-16T17:10:50.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:10:50.859+0000 7f985a200640 1 -- 172.21.15.45:0/3067518888 wait complete. 2024-04-16T17:10:50.876 INFO:tasks.cephfs.filesystem:scrub status for tag:67516e11-3f36-4d52-bf46-ecb24b234808 - {'path': '/', 'tag': '67516e11-3f36-4d52-bf46-ecb24b234808', 'options': 'recursive,force'} 2024-04-16T17:10:50.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:10:50 smithi104 ceph-mon[36457]: pgmap v125: 129 pgs: 129 active+clean; 36 MiB data, 485 MiB used, 1.0 TiB / 1.0 TiB avail; 470 KiB/s rd, 288 KiB/s wr, 65 op/s 2024-04-16T17:10:51.246 INFO:tasks.workunit.client.0.smithi045.stdout: 1 56803 1.44 MB/sec execute 53 sec latency 16.746 ms 2024-04-16T17:10:51.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:10:51 smithi117 ceph-mon[36730]: scrub summary: active paths [/] 2024-04-16T17:10:51.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:10:51 smithi117 ceph-mon[36730]: scrub summary: active paths [/] 2024-04-16T17:10:51.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:10:51 smithi117 ceph-mon[36730]: scrub summary: idle+waiting paths [/] 2024-04-16T17:10:51.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:10:51 smithi117 ceph-mon[36730]: scrub summary: active paths [/] 2024-04-16T17:10:51.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:10:51 smithi117 ceph-mon[36730]: scrub summary: idle+waiting paths [/] 2024-04-16T17:10:51.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:10:51 smithi117 ceph-mon[36730]: scrub summary: idle+waiting paths [/] 2024-04-16T17:10:51.860 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:10:48 smithi045 ceph-mon[31676]: pgmap v124: 129 pgs: 129 active+clean; 36 MiB data, 485 MiB used, 1.0 TiB / 1.0 TiB avail; 597 KiB/s rd, 414 KiB/s wr, 85 op/s 2024-04-16T17:10:51.860 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:10:48 smithi045 ceph-mon[31676]: from='client.? 172.21.15.45:0/636722744' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:10:51.861 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:10:50 smithi045 ceph-mon[31676]: pgmap v125: 129 pgs: 129 active+clean; 36 MiB data, 485 MiB used, 1.0 TiB / 1.0 TiB avail; 470 KiB/s rd, 288 KiB/s wr, 65 op/s 2024-04-16T17:10:51.861 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:10:51 smithi045 ceph-mon[31676]: scrub summary: active paths [/] 2024-04-16T17:10:51.861 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:10:51 smithi045 ceph-mon[31676]: scrub summary: active paths [/] 2024-04-16T17:10:51.861 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:10:51 smithi045 ceph-mon[31676]: scrub summary: idle+waiting paths [/] 2024-04-16T17:10:51.861 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:10:51 smithi045 ceph-mon[31676]: scrub summary: active paths [/] 2024-04-16T17:10:51.861 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:10:51 smithi045 ceph-mon[31676]: scrub summary: idle+waiting paths [/] 2024-04-16T17:10:51.861 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:10:51 smithi045 ceph-mon[31676]: scrub summary: idle+waiting paths [/] 2024-04-16T17:10:51.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:10:51 smithi104 ceph-mon[36457]: scrub summary: active paths [/] 2024-04-16T17:10:51.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:10:51 smithi104 ceph-mon[36457]: scrub summary: active paths [/] 2024-04-16T17:10:51.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:10:51 smithi104 ceph-mon[36457]: scrub summary: idle+waiting paths [/] 2024-04-16T17:10:51.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:10:51 smithi104 ceph-mon[36457]: scrub summary: active paths [/] 2024-04-16T17:10:51.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:10:51 smithi104 ceph-mon[36457]: scrub summary: idle+waiting paths [/] 2024-04-16T17:10:51.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:10:51 smithi104 ceph-mon[36457]: scrub summary: idle+waiting paths [/] 2024-04-16T17:10:52.246 INFO:tasks.workunit.client.0.smithi045.stdout: 1 57207 1.44 MB/sec execute 54 sec latency 18.806 ms 2024-04-16T17:10:52.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:10:52 smithi117 ceph-mon[36730]: scrub queued for path: / 2024-04-16T17:10:52.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:10:52 smithi117 ceph-mon[36730]: scrub summary: idle+waiting paths [/] 2024-04-16T17:10:52.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:10:52 smithi117 ceph-mon[36730]: scrub summary: active paths [/] 2024-04-16T17:10:52.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:10:52 smithi117 ceph-mon[36730]: scrub summary: idle+waiting paths [/] 2024-04-16T17:10:52.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:10:52 smithi117 ceph-mon[36730]: pgmap v126: 129 pgs: 129 active+clean; 37 MiB data, 468 MiB used, 1.0 TiB / 1.0 TiB avail; 869 KiB/s rd, 999 KiB/s wr, 124 op/s 2024-04-16T17:10:52.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:10:52 smithi104 ceph-mon[36457]: scrub queued for path: / 2024-04-16T17:10:52.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:10:52 smithi104 ceph-mon[36457]: scrub summary: idle+waiting paths [/] 2024-04-16T17:10:52.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:10:52 smithi104 ceph-mon[36457]: scrub summary: active paths [/] 2024-04-16T17:10:52.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:10:52 smithi104 ceph-mon[36457]: scrub summary: idle+waiting paths [/] 2024-04-16T17:10:52.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:10:52 smithi104 ceph-mon[36457]: pgmap v126: 129 pgs: 129 active+clean; 37 MiB data, 468 MiB used, 1.0 TiB / 1.0 TiB avail; 869 KiB/s rd, 999 KiB/s wr, 124 op/s 2024-04-16T17:10:53.154 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:10:52 smithi045 ceph-mon[31676]: scrub queued for path: / 2024-04-16T17:10:53.154 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:10:52 smithi045 ceph-mon[31676]: scrub summary: idle+waiting paths [/] 2024-04-16T17:10:53.155 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:10:52 smithi045 ceph-mon[31676]: scrub summary: active paths [/] 2024-04-16T17:10:53.155 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:10:52 smithi045 ceph-mon[31676]: scrub summary: idle+waiting paths [/] 2024-04-16T17:10:53.155 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:10:52 smithi045 ceph-mon[31676]: pgmap v126: 129 pgs: 129 active+clean; 37 MiB data, 468 MiB used, 1.0 TiB / 1.0 TiB avail; 869 KiB/s rd, 999 KiB/s wr, 124 op/s 2024-04-16T17:10:53.247 INFO:tasks.workunit.client.0.smithi045.stdout: 1 57448 1.42 MB/sec execute 55 sec latency 52.990 ms 2024-04-16T17:10:54.246 INFO:tasks.workunit.client.0.smithi045.stdout: 1 57906 1.42 MB/sec execute 56 sec latency 52.164 ms 2024-04-16T17:10:54.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:10:54 smithi104 ceph-mon[36457]: pgmap v127: 129 pgs: 129 active+clean; 37 MiB data, 468 MiB used, 1.0 TiB / 1.0 TiB avail; 524 KiB/s rd, 843 KiB/s wr, 97 op/s 2024-04-16T17:10:55.054 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:10:54 smithi117 ceph-mon[36730]: pgmap v127: 129 pgs: 129 active+clean; 37 MiB data, 468 MiB used, 1.0 TiB / 1.0 TiB avail; 524 KiB/s rd, 843 KiB/s wr, 97 op/s 2024-04-16T17:10:55.246 INFO:tasks.workunit.client.0.smithi045.stdout: 1 58215 1.45 MB/sec execute 57 sec latency 96.885 ms 2024-04-16T17:10:55.360 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:55.360+0000 7f0371e00640 1 -- 172.21.15.45:0/3161902945 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f036c107140 msgr2=0x7f036c10ed10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:55.360 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:55.360+0000 7f0371e00640 1 --2- 172.21.15.45:0/3161902945 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f036c107140 0x7f036c10ed10 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f0364009ca0 tx=0x7f036402c100 comp rx=0 tx=0).stop 2024-04-16T17:10:55.360 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:55.360+0000 7f0371e00640 1 -- 172.21.15.45:0/3161902945 shutdown_connections 2024-04-16T17:10:55.360 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:55.360+0000 7f0371e00640 1 --2- 172.21.15.45:0/3161902945 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f036c102a20 0x7f036c106a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:55.361 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:55.360+0000 7f0371e00640 1 --2- 172.21.15.45:0/3161902945 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f036c107140 0x7f036c10ed10 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:55.361 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:55.360+0000 7f0371e00640 1 --2- 172.21.15.45:0/3161902945 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f036c102070 0x7f036c102450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:55.361 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:55.360+0000 7f0371e00640 1 -- 172.21.15.45:0/3161902945 >> 172.21.15.45:0/3161902945 conn(0x7f036c0fd950 msgr2=0x7f036c0ffd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:10:55.361 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:55.361+0000 7f0371e00640 1 -- 172.21.15.45:0/3161902945 shutdown_connections 2024-04-16T17:10:55.361 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:55.361+0000 7f0371e00640 1 -- 172.21.15.45:0/3161902945 wait complete. 2024-04-16T17:10:55.362 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:55.362+0000 7f0371e00640 1 Processor -- start 2024-04-16T17:10:55.362 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:55.362+0000 7f0371e00640 1 -- start start 2024-04-16T17:10:55.362 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:55.362+0000 7f0371e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f036c102070 0x7f036c108d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:55.362 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:55.362+0000 7f0371e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f036c107140 0x7f036c109270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:55.363 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:55.363+0000 7f0371e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f036c1097b0 0x7f036c1aaee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:55.363 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:55.363+0000 7f0371e00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f036c1121e0 con 0x7f036c102070 2024-04-16T17:10:55.363 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:55.363+0000 7f0371e00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f036c112060 con 0x7f036c1097b0 2024-04-16T17:10:55.363 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:55.363+0000 7f0371e00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f036c112360 con 0x7f036c107140 2024-04-16T17:10:55.363 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:55.363+0000 7f0370a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f036c102070 0x7f036c108d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:55.363 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:55.363+0000 7f036be00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f036c107140 0x7f036c109270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:55.363 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:55.363+0000 7f036be00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f036c107140 0x7f036c109270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:36082/0 (socket says 172.21.15.45:36082) 2024-04-16T17:10:55.363 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:55.363+0000 7f036be00640 1 -- 172.21.15.45:0/3041833871 learned_addr learned my addr 172.21.15.45:0/3041833871 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:10:55.363 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:55.363+0000 7f0371400640 1 --2- 172.21.15.45:0/3041833871 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f036c1097b0 0x7f036c1aaee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:55.363 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:55.363+0000 7f0370a00640 1 -- 172.21.15.45:0/3041833871 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f036c107140 msgr2=0x7f036c109270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:55.364 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:55.363+0000 7f0370a00640 1 --2- 172.21.15.45:0/3041833871 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f036c107140 0x7f036c109270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:55.364 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:55.363+0000 7f0370a00640 1 -- 172.21.15.45:0/3041833871 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f036c1097b0 msgr2=0x7f036c1aaee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:55.364 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:55.363+0000 7f0370a00640 1 --2- 172.21.15.45:0/3041833871 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f036c1097b0 0x7f036c1aaee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:55.364 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:55.363+0000 7f0370a00640 1 -- 172.21.15.45:0/3041833871 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f036c1ab420 con 0x7f036c102070 2024-04-16T17:10:55.364 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:55.363+0000 7f036be00640 1 --2- 172.21.15.45:0/3041833871 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f036c107140 0x7f036c109270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:10:55.364 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:55.364+0000 7f0370a00640 1 --2- 172.21.15.45:0/3041833871 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f036c102070 0x7f036c108d30 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f036000e9f0 tx=0x7f036000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:10:55.364 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:55.364+0000 7f0369600640 1 -- 172.21.15.45:0/3041833871 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f036000ccc0 con 0x7f036c102070 2024-04-16T17:10:55.364 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:55.364+0000 7f0369600640 1 -- 172.21.15.45:0/3041833871 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0360010910 con 0x7f036c102070 2024-04-16T17:10:55.364 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:55.364+0000 7f0369600640 1 -- 172.21.15.45:0/3041833871 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0360018960 con 0x7f036c102070 2024-04-16T17:10:55.364 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:55.364+0000 7f0371e00640 1 -- 172.21.15.45:0/3041833871 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f036c1ab710 con 0x7f036c102070 2024-04-16T17:10:55.365 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:55.365+0000 7f0371e00640 1 -- 172.21.15.45:0/3041833871 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f036c1abc80 con 0x7f036c102070 2024-04-16T17:10:55.368 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:55.369+0000 7f03761e3740 1 Processor -- start 2024-04-16T17:10:55.368 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:55.369+0000 7f03761e3740 1 -- start start 2024-04-16T17:10:55.369 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:55.369+0000 7f0369600640 1 -- 172.21.15.45:0/3041833871 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f0360010ac0 con 0x7f036c102070 2024-04-16T17:10:55.369 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:55.369+0000 7f03761e3740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c5278cf7f0 0x55c5278ce540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:55.369 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:55.369+0000 7f03761e3740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c5278cea80 0x55c5278cef00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:55.369 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:55.369+0000 7f03761e3740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c5278d8000 0x55c5278da3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:55.369 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:55.369+0000 7f03761e3740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55c527833780 con 0x55c5278cf7f0 2024-04-16T17:10:55.369 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:55.369+0000 7f03761e3740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55c5277d9f40 con 0x55c5278cea80 2024-04-16T17:10:55.369 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:55.369+0000 7f03761e3740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55c52782f740 con 0x55c5278d8000 2024-04-16T17:10:55.370 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:55.370+0000 7f0370a00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c5278cea80 0x55c5278cef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:55.370 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:55.370+0000 7f0370a00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c5278cea80 0x55c5278cef00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:47868/0 (socket says 172.21.15.45:47868) 2024-04-16T17:10:55.370 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:55.370+0000 7f0370a00640 1 -- 172.21.15.45:0/1728902713 learned_addr learned my addr 172.21.15.45:0/1728902713 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:10:55.370 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:55.370+0000 7f036be00640 1 --2- 172.21.15.45:0/1728902713 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c5278d8000 0x55c5278da3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:55.370 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:55.370+0000 7f0371400640 1 --2- 172.21.15.45:0/1728902713 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c5278cf7f0 0x55c5278ce540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:55.370 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:55.371+0000 7f036be00640 1 -- 172.21.15.45:0/1728902713 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c5278cea80 msgr2=0x55c5278cef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:10:55.371 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:55.371+0000 7f036be00640 1 --2- 172.21.15.45:0/1728902713 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c5278cea80 0x55c5278cef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:55.371 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:55.371+0000 7f036be00640 1 -- 172.21.15.45:0/1728902713 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c5278cf7f0 msgr2=0x55c5278ce540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:10:55.371 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:55.371+0000 7f036be00640 1 --2- 172.21.15.45:0/1728902713 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c5278cf7f0 0x55c5278ce540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:55.371 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:55.371+0000 7f036be00640 1 -- 172.21.15.45:0/1728902713 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c527546210 con 0x55c5278d8000 2024-04-16T17:10:55.371 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:55.371+0000 7f0370a00640 1 --2- 172.21.15.45:0/1728902713 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c5278cea80 0x55c5278cef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:10:55.372 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:55.371+0000 7f0371400640 1 --2- 172.21.15.45:0/1728902713 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c5278cf7f0 0x55c5278ce540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:10:55.372 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:55.372+0000 7f036be00640 1 --2- 172.21.15.45:0/1728902713 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c5278d8000 0x55c5278da3f0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f036c10a4f0 tx=0x7f03580118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fbca2e9bbe4f2181 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:10:55.372 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:55.372+0000 7f0369600640 1 --2- 172.21.15.45:0/3041833871 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f03540a3d90 0x7f03540a6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:55.372 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:55.372+0000 7f035dc00640 1 -- 172.21.15.45:0/1728902713 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0358004030 con 0x55c5278d8000 2024-04-16T17:10:55.372 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:55.372+0000 7f0369600640 1 -- 172.21.15.45:0/3041833871 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f03600d0e90 con 0x7f036c102070 2024-04-16T17:10:55.372 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:55.372+0000 7f035dc00640 1 -- 172.21.15.45:0/1728902713 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f03580041d0 con 0x55c5278d8000 2024-04-16T17:10:55.372 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:55.372+0000 7f035dc00640 1 -- 172.21.15.45:0/1728902713 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f035800f890 con 0x55c5278d8000 2024-04-16T17:10:55.372 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:55.372+0000 7f0371400640 1 --2- 172.21.15.45:0/3041833871 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f03540a3d90 0x7f03540a6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:55.373 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:55.373+0000 7f03761e3740 1 -- 172.21.15.45:0/1728902713 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c5278d8000 msgr2=0x55c5278da3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:55.373 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:55.373+0000 7f03761e3740 1 --2- 172.21.15.45:0/1728902713 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c5278d8000 0x55c5278da3f0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f036c10a4f0 tx=0x7f03580118d0 comp rx=0 tx=0).stop 2024-04-16T17:10:55.373 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:55.373+0000 7f0371400640 1 --2- 172.21.15.45:0/3041833871 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f03540a3d90 0x7f03540a6250 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f0364004af0 tx=0x7f036402e040 comp rx=0 tx=0).ready entity=mgr.25411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:10:55.373 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:55.373+0000 7f03761e3740 1 -- 172.21.15.45:0/1728902713 shutdown_connections 2024-04-16T17:10:55.373 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:55.373+0000 7f03761e3740 1 --2- 172.21.15.45:0/1728902713 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c5278d8000 0x55c5278da3f0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:55.373 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:55.373+0000 7f03761e3740 1 --2- 172.21.15.45:0/1728902713 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c5278cea80 0x55c5278cef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:55.373 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:55.373+0000 7f03761e3740 1 --2- 172.21.15.45:0/1728902713 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c5278cf7f0 0x55c5278ce540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:55.373 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:55.373+0000 7f03761e3740 1 -- 172.21.15.45:0/1728902713 >> 172.21.15.45:0/1728902713 conn(0x55c5278695d0 msgr2=0x55c5278c9770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:10:55.374 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:55.374+0000 7f03761e3740 1 -- 172.21.15.45:0/1728902713 shutdown_connections 2024-04-16T17:10:55.374 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:55.374+0000 7f03761e3740 1 -- 172.21.15.45:0/1728902713 wait complete. 2024-04-16T17:10:55.374 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:55.375+0000 7f03761e3740 1 -- start start 2024-04-16T17:10:55.375 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:55.375+0000 7f03761e3740 1 Processor -- start 2024-04-16T17:10:55.376 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:55.376+0000 7f03761e3740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c5278cea80 0x55c5278d7a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:55.376 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:55.376+0000 7f03761e3740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c5278cf7f0 0x55c5278d4e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:55.377 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:55.376+0000 7f03761e3740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c5278d8000 0x55c5278d5550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:55.377 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:55.376+0000 7f03761e3740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55c5278dad30 con 0x55c5278d8000 2024-04-16T17:10:55.377 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:55.376+0000 7f03761e3740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55c5277ed390 con 0x55c5278cea80 2024-04-16T17:10:55.377 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:55.376+0000 7f03761e3740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55c5278daeb0 con 0x55c5278cf7f0 2024-04-16T17:10:55.377 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:55.376+0000 7f03761e3740 10 client.0 _ll_register_callbacks cb 0x55c5277d6f60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:10:55.377 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:55.377+0000 7f0370a00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c5278cea80 0x55c5278d7a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:55.377 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:55.377+0000 7f0370a00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c5278cea80 0x55c5278d7a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:47876/0 (socket says 172.21.15.45:47876) 2024-04-16T17:10:55.377 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:55.377+0000 7f0370a00640 1 -- 172.21.15.45:0/2648740909 learned_addr learned my addr 172.21.15.45:0/2648740909 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:10:55.377 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:55.377+0000 7f0370a00640 1 -- 172.21.15.45:0/2648740909 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c5278cf7f0 msgr2=0x55c5278d4e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:10:55.377 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:55.377+0000 7f0370a00640 1 --2- 172.21.15.45:0/2648740909 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c5278cf7f0 0x55c5278d4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:55.377 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:55.377+0000 7f0370a00640 1 -- 172.21.15.45:0/2648740909 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c5278d8000 msgr2=0x55c5278d5550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:10:55.377 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:55.377+0000 7f0370a00640 1 --2- 172.21.15.45:0/2648740909 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c5278d8000 0x55c5278d5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:55.377 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:55.377+0000 7f0370a00640 1 -- 172.21.15.45:0/2648740909 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c527612620 con 0x55c5278cea80 2024-04-16T17:10:55.379 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:55.378+0000 7f0370a00640 1 --2- 172.21.15.45:0/2648740909 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c5278cea80 0x55c5278d7a80 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f03600a51c0 tx=0x7f0360010ef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=28301d463ce3f4c1 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:10:55.379 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:55.378+0000 7f0352000640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-04-16T17:10:55.379 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:55.378+0000 7f0352000640 1 -- 172.21.15.45:0/2648740909 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f036009c490 con 0x55c5278cea80 2024-04-16T17:10:55.379 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:55.378+0000 7f0352000640 1 -- 172.21.15.45:0/2648740909 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f036009c180 con 0x55c5278cea80 2024-04-16T17:10:55.379 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:55.378+0000 7f03761e3740 1 -- 172.21.15.45:0/2648740909 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f03580041d0 con 0x55c5278cea80 2024-04-16T17:10:55.379 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:55.378+0000 7f0352000640 1 -- 172.21.15.45:0/2648740909 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f03600ce6c0 con 0x55c5278cea80 2024-04-16T17:10:55.379 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:55.378+0000 7f03761e3740 10 client.26810 fetch_fsmap learned FSMap version 31 2024-04-16T17:10:55.379 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:55.378+0000 7f03761e3740 1 -- 172.21.15.45:0/2648740909 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55c5278e6e30 con 0x55c5278cea80 2024-04-16T17:10:55.379 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:55.379+0000 7f0352000640 1 -- 172.21.15.45:0/2648740909 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f03580041d0 con 0x55c5278cea80 2024-04-16T17:10:55.379 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:55.379+0000 7f0352000640 1 -- 172.21.15.45:0/2648740909 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 31) ==== 8366+0+0 (secure 0 0 0) 0x7f03600989a0 con 0x55c5278cea80 2024-04-16T17:10:55.379 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:55.379+0000 7f03761e3740 10 client.26810 fetch_fsmap finished waiting for FSMap version 31 2024-04-16T17:10:55.380 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:55.379+0000 7f03761e3740 10 client.26810 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T17:10:55.380 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:55.379+0000 7f03761e3740 20 client.26810 populate_metadata read hostname 'smithi045' 2024-04-16T17:10:55.380 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:55.379+0000 7f03761e3740 1 --2- 172.21.15.45:0/2648740909 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55c5278e8340 0x55c5278ea7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:55.380 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:55.379+0000 7f03761e3740 4 client.26810 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:10:55.380 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:55.379+0000 7f03761e3740 1 -- 172.21.15.45:0/2648740909 --> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c5278411a0 con 0x55c5278e8340 2024-04-16T17:10:55.381 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:55.380+0000 7f036be00640 1 --2- 172.21.15.45:0/2648740909 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55c5278e8340 0x55c5278ea7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:55.381 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:55.380+0000 7f036be00640 1 --2- 172.21.15.45:0/2648740909 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55c5278e8340 0x55c5278ea7d0 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=29697aeff2670727 server_cookie=1da3456f9dbac26e in_seq=0 out_seq=0 2024-04-16T17:10:55.381 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:55.380+0000 7f0352000640 10 client.26810 ms_handle_connect on v2:172.21.15.104:6838/3660070592 2024-04-16T17:10:55.381 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:55.381+0000 7f0352000640 1 -- 172.21.15.45:0/2648740909 <== mds.0 v2:172.21.15.104:6838/3660070592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c5278411a0 con 0x55c5278e8340 2024-04-16T17:10:55.381 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:55.381+0000 7f0352000640 10 client.26810 handle_command_reply: tid=0 2024-04-16T17:10:55.381 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:55.381+0000 7f03761e3740 1 client.26810 shutdown 2024-04-16T17:10:55.382 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:55.381+0000 7f03761e3740 1 -- 172.21.15.45:0/2648740909 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c5278cea80 msgr2=0x55c5278d7a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:55.382 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:55.381+0000 7f03761e3740 1 --2- 172.21.15.45:0/2648740909 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c5278cea80 0x55c5278d7a80 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f03600a51c0 tx=0x7f0360010ef0 comp rx=0 tx=0).stop 2024-04-16T17:10:55.382 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:55.382+0000 7f03761e3740 1 -- 172.21.15.45:0/2648740909 shutdown_connections 2024-04-16T17:10:55.382 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:55.382+0000 7f03761e3740 1 --2- 172.21.15.45:0/2648740909 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55c5278e8340 0x55c5278ea7d0 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:55.382 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:55.382+0000 7f03761e3740 1 --2- 172.21.15.45:0/2648740909 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c5278cf7f0 0x55c5278d4e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:55.382 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:55.382+0000 7f03761e3740 1 --2- 172.21.15.45:0/2648740909 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c5278d8000 0x55c5278d5550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:55.382 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:55.382+0000 7f0352000640 0 client.26810 ms_handle_reset on v2:172.21.15.104:6838/3660070592 2024-04-16T17:10:55.382 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:55.382+0000 7f03761e3740 1 --2- 172.21.15.45:0/2648740909 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c5278cea80 0x55c5278d7a80 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:55.382 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:55.382+0000 7f03761e3740 1 -- 172.21.15.45:0/2648740909 >> 172.21.15.45:0/2648740909 conn(0x55c5278695d0 msgr2=0x55c5278cb4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:10:55.382 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:55.382+0000 7f03761e3740 1 -- 172.21.15.45:0/2648740909 shutdown_connections 2024-04-16T17:10:55.383 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:55.382+0000 7f03761e3740 1 -- 172.21.15.45:0/2648740909 wait complete. 2024-04-16T17:10:55.383 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:55.382+0000 7f03761e3740 20 client.26810 trim_cache size 0 max 16384 2024-04-16T17:10:55.408 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:55.409+0000 7f03761e3740 1 Processor -- start 2024-04-16T17:10:55.408 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:55.409+0000 7f03761e3740 1 -- start start 2024-04-16T17:10:55.409 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:55.409+0000 7f03761e3740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c5278cea80 0x55c5278e3c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:55.409 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:55.409+0000 7f03761e3740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c5278cf7f0 0x55c5278d3e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:55.409 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:55.409+0000 7f03761e3740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c5278d8000 0x55c5278f00e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:55.409 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:55.409+0000 7f03761e3740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f036c1b83a0 con 0x55c5278d8000 2024-04-16T17:10:55.409 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:55.409+0000 7f03761e3740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f036c1b5bb0 con 0x55c5278cf7f0 2024-04-16T17:10:55.409 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:55.409+0000 7f03761e3740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f036c1b39e0 con 0x55c5278cea80 2024-04-16T17:10:55.410 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:55.409+0000 7f036be00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c5278cf7f0 0x55c5278d3e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:55.410 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:55.410+0000 7f036be00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c5278cf7f0 0x55c5278d3e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:47880/0 (socket says 172.21.15.45:47880) 2024-04-16T17:10:55.410 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:55.410+0000 7f036be00640 1 -- 172.21.15.45:0/3483283664 learned_addr learned my addr 172.21.15.45:0/3483283664 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:10:55.410 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:55.410+0000 7f0370a00640 1 --2- 172.21.15.45:0/3483283664 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c5278cea80 0x55c5278e3c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:55.411 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:55.410+0000 7f0371400640 1 --2- 172.21.15.45:0/3483283664 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c5278d8000 0x55c5278f00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:55.411 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:55.410+0000 7f036be00640 1 -- 172.21.15.45:0/3483283664 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c5278cea80 msgr2=0x55c5278e3c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:10:55.411 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:55.410+0000 7f036be00640 1 --2- 172.21.15.45:0/3483283664 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c5278cea80 0x55c5278e3c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:55.411 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:55.410+0000 7f036be00640 1 -- 172.21.15.45:0/3483283664 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c5278d8000 msgr2=0x55c5278f00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:10:55.411 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:55.410+0000 7f036be00640 1 --2- 172.21.15.45:0/3483283664 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c5278d8000 0x55c5278f00e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:55.411 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:55.410+0000 7f036be00640 1 -- 172.21.15.45:0/3483283664 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c5278ce8b0 con 0x55c5278cf7f0 2024-04-16T17:10:55.411 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:55.411+0000 7f036be00640 1 --2- 172.21.15.45:0/3483283664 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c5278cf7f0 0x55c5278d3e30 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f0358009600 tx=0x7f0358009310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=17ac6ca913723dc9 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:10:55.411 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:55.411+0000 7f035d200640 1 -- 172.21.15.45:0/3483283664 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f035802d030 con 0x55c5278cf7f0 2024-04-16T17:10:55.411 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:55.411+0000 7f035d200640 1 -- 172.21.15.45:0/3483283664 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0358005520 con 0x55c5278cf7f0 2024-04-16T17:10:55.411 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:55.411+0000 7f035d200640 1 -- 172.21.15.45:0/3483283664 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0358003b80 con 0x55c5278cf7f0 2024-04-16T17:10:55.412 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:55.412+0000 7f03761e3740 1 -- 172.21.15.45:0/3483283664 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c5278cf7f0 msgr2=0x55c5278d3e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:55.412 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:55.412+0000 7f03761e3740 1 --2- 172.21.15.45:0/3483283664 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c5278cf7f0 0x55c5278d3e30 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f0358009600 tx=0x7f0358009310 comp rx=0 tx=0).stop 2024-04-16T17:10:55.412 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:55.412+0000 7f03761e3740 1 -- 172.21.15.45:0/3483283664 shutdown_connections 2024-04-16T17:10:55.412 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:55.412+0000 7f03761e3740 1 --2- 172.21.15.45:0/3483283664 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c5278d8000 0x55c5278f00e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:55.412 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:55.412+0000 7f03761e3740 1 --2- 172.21.15.45:0/3483283664 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c5278cf7f0 0x55c5278d3e30 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:55.412 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:55.412+0000 7f03761e3740 1 --2- 172.21.15.45:0/3483283664 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c5278cea80 0x55c5278e3c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:55.412 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:55.412+0000 7f03761e3740 1 -- 172.21.15.45:0/3483283664 >> 172.21.15.45:0/3483283664 conn(0x55c5278695d0 msgr2=0x55c5278d3720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:10:55.412 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:55.412+0000 7f03761e3740 1 -- 172.21.15.45:0/3483283664 shutdown_connections 2024-04-16T17:10:55.413 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:55.412+0000 7f03761e3740 1 -- 172.21.15.45:0/3483283664 wait complete. 2024-04-16T17:10:55.413 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:55.413+0000 7f03761e3740 1 -- start start 2024-04-16T17:10:55.413 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:55.413+0000 7f03761e3740 1 Processor -- start 2024-04-16T17:10:55.414 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:55.414+0000 7f03761e3740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c5278cea80 0x55c5278d2b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:55.414 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:55.414+0000 7f03761e3740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c5278cf7f0 0x55c5278e0170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:55.414 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:55.414+0000 7f03761e3740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c5278d8000 0x55c5278d41e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:55.414 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:55.414+0000 7f03761e3740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f036c1b4070 con 0x55c5278cea80 2024-04-16T17:10:55.414 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:55.414+0000 7f03761e3740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f036c1b54e0 con 0x55c5278cf7f0 2024-04-16T17:10:55.414 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:55.414+0000 7f03761e3740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f036c1b4740 con 0x55c5278d8000 2024-04-16T17:10:55.414 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:55.414+0000 7f03761e3740 10 client.0 _ll_register_callbacks cb 0x7f03600d4630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:10:55.414 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:55.414+0000 7f036be00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c5278cf7f0 0x55c5278e0170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:55.414 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:55.414+0000 7f036be00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c5278cf7f0 0x55c5278e0170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:47890/0 (socket says 172.21.15.45:47890) 2024-04-16T17:10:55.414 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:55.414+0000 7f0371400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c5278d8000 0x55c5278d41e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:55.414 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:55.414+0000 7f036be00640 1 -- 172.21.15.45:0/2027608129 learned_addr learned my addr 172.21.15.45:0/2027608129 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:10:55.416 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:55.414+0000 7f036be00640 1 -- 172.21.15.45:0/2027608129 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c5278d8000 msgr2=0x55c5278d41e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:10:55.416 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:55.414+0000 7f036be00640 1 --2- 172.21.15.45:0/2027608129 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c5278d8000 0x55c5278d41e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:55.416 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:55.414+0000 7f036be00640 1 -- 172.21.15.45:0/2027608129 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c5278cea80 msgr2=0x55c5278d2b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:10:55.416 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:55.414+0000 7f036be00640 1 --2- 172.21.15.45:0/2027608129 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c5278cea80 0x55c5278d2b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:55.416 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:55.414+0000 7f036be00640 1 -- 172.21.15.45:0/2027608129 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c5278d58c0 con 0x55c5278cf7f0 2024-04-16T17:10:55.416 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:55.414+0000 7f0371400640 1 --2- 172.21.15.45:0/2027608129 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c5278d8000 0x55c5278d41e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:10:55.416 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:55.415+0000 7f036be00640 1 --2- 172.21.15.45:0/2027608129 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c5278cf7f0 0x55c5278e0170 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f0358009460 tx=0x7f0358009530 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ae6537b7b4c7c19e server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:10:55.416 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:55.415+0000 7f0352000640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-04-16T17:10:55.416 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:55.415+0000 7f0352000640 1 -- 172.21.15.45:0/2027608129 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f035802d040 con 0x55c5278cf7f0 2024-04-16T17:10:55.416 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:55.415+0000 7f0352000640 1 -- 172.21.15.45:0/2027608129 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f03580035f0 con 0x55c5278cf7f0 2024-04-16T17:10:55.416 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:55.415+0000 7f03761e3740 1 -- 172.21.15.45:0/2027608129 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0358005520 con 0x55c5278cf7f0 2024-04-16T17:10:55.417 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:55.416+0000 7f0352000640 1 -- 172.21.15.45:0/2027608129 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f0358009c10 con 0x55c5278cf7f0 2024-04-16T17:10:55.417 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:55.416+0000 7f03761e3740 10 client.26816 fetch_fsmap learned FSMap version 31 2024-04-16T17:10:55.417 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:55.416+0000 7f03761e3740 1 -- 172.21.15.45:0/2027608129 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55c52794ee60 con 0x55c5278cf7f0 2024-04-16T17:10:55.417 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:55.416+0000 7f0352000640 1 -- 172.21.15.45:0/2027608129 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0358005520 con 0x55c5278cf7f0 2024-04-16T17:10:55.417 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:55.417+0000 7f0352000640 1 -- 172.21.15.45:0/2027608129 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 31) ==== 8366+0+0 (secure 0 0 0) 0x7f0358003a00 con 0x55c5278cf7f0 2024-04-16T17:10:55.417 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:55.417+0000 7f03761e3740 10 client.26816 fetch_fsmap finished waiting for FSMap version 31 2024-04-16T17:10:55.417 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:55.417+0000 7f03761e3740 10 client.26816 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T17:10:55.417 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:55.417+0000 7f03761e3740 20 client.26816 populate_metadata read hostname 'smithi045' 2024-04-16T17:10:55.417 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:55.417+0000 7f03761e3740 1 --2- 172.21.15.45:0/2027608129 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55c52794f330 0x55c527951750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:10:55.418 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:55.417+0000 7f03761e3740 4 client.26816 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-16T17:10:55.418 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:55.417+0000 7f03761e3740 1 -- 172.21.15.45:0/2027608129 --> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c52783b010 con 0x55c52794f330 2024-04-16T17:10:55.418 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:55.418+0000 7f0370a00640 1 --2- 172.21.15.45:0/2027608129 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55c52794f330 0x55c527951750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:10:55.418 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:55.418+0000 7f0370a00640 1 --2- 172.21.15.45:0/2027608129 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55c52794f330 0x55c527951750 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5afc02cec24a20c3 server_cookie=ffdf6e7b4c22facc in_seq=0 out_seq=0 2024-04-16T17:10:55.418 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:55.418+0000 7f0352000640 10 client.26816 ms_handle_connect on v2:172.21.15.104:6838/3660070592 2024-04-16T17:10:55.458 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:55.457+0000 7f0352000640 1 -- 172.21.15.45:0/2027608129 <== mds.0 v2:172.21.15.104:6838/3660070592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c52783b010 con 0x55c52794f330 2024-04-16T17:10:55.458 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:55.457+0000 7f0352000640 10 client.26816 handle_command_reply: tid=0 2024-04-16T17:10:55.458 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:55.457+0000 7f03761e3740 1 client.26816 shutdown 2024-04-16T17:10:55.458 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:55.458+0000 7f03761e3740 1 -- 172.21.15.45:0/2027608129 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c5278cf7f0 msgr2=0x55c5278e0170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:55.458 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:55.458+0000 7f03761e3740 1 --2- 172.21.15.45:0/2027608129 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c5278cf7f0 0x55c5278e0170 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f0358009460 tx=0x7f0358009530 comp rx=0 tx=0).stop 2024-04-16T17:10:55.458 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:55.458+0000 7f03761e3740 1 -- 172.21.15.45:0/2027608129 shutdown_connections 2024-04-16T17:10:55.458 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:55.458+0000 7f03761e3740 1 --2- 172.21.15.45:0/2027608129 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55c52794f330 0x55c527951750 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:55.458 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:55.458+0000 7f03761e3740 1 --2- 172.21.15.45:0/2027608129 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c5278d8000 0x55c5278d41e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:55.458 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:55.458+0000 7f03761e3740 1 --2- 172.21.15.45:0/2027608129 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c5278cf7f0 0x55c5278e0170 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:55.458 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:55.458+0000 7f0352000640 0 client.26816 ms_handle_reset on v2:172.21.15.104:6838/3660070592 2024-04-16T17:10:55.458 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:55.458+0000 7f03761e3740 1 --2- 172.21.15.45:0/2027608129 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c5278cea80 0x55c5278d2b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:55.458 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:55.458+0000 7f03761e3740 1 -- 172.21.15.45:0/2027608129 >> 172.21.15.45:0/2027608129 conn(0x55c5278695d0 msgr2=0x55c5278ca830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:10:55.458 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:55.458+0000 7f03761e3740 1 -- 172.21.15.45:0/2027608129 shutdown_connections 2024-04-16T17:10:55.459 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:55.458+0000 7f03761e3740 1 -- 172.21.15.45:0/2027608129 wait complete. 2024-04-16T17:10:55.459 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:55.458+0000 7f03761e3740 20 client.26816 trim_cache size 0 max 16384 2024-04-16T17:10:55.459 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-04-16T17:10:55.459 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-04-16T17:10:55.459 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-04-16T17:10:55.459 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-04-16T17:10:55.460 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:55.459+0000 7f0371e00640 1 -- 172.21.15.45:0/3041833871 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f03540a3d90 msgr2=0x7f03540a6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:55.460 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:55.459+0000 7f0371e00640 1 --2- 172.21.15.45:0/3041833871 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f03540a3d90 0x7f03540a6250 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f0364004af0 tx=0x7f036402e040 comp rx=0 tx=0).stop 2024-04-16T17:10:55.460 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:55.459+0000 7f0371e00640 1 -- 172.21.15.45:0/3041833871 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f036c102070 msgr2=0x7f036c108d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:10:55.460 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:55.459+0000 7f0371e00640 1 --2- 172.21.15.45:0/3041833871 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f036c102070 0x7f036c108d30 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f036000e9f0 tx=0x7f036000eeb0 comp rx=0 tx=0).stop 2024-04-16T17:10:55.460 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:55.459+0000 7f0371e00640 1 -- 172.21.15.45:0/3041833871 shutdown_connections 2024-04-16T17:10:55.460 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:55.459+0000 7f0371e00640 1 --2- 172.21.15.45:0/3041833871 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f03540a3d90 0x7f03540a6250 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:55.460 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:55.459+0000 7f0371e00640 1 --2- 172.21.15.45:0/3041833871 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f036c107140 0x7f036c109270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:55.460 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:55.459+0000 7f0371e00640 1 --2- 172.21.15.45:0/3041833871 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f036c1097b0 0x7f036c1aaee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:55.460 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:55.459+0000 7f0371e00640 1 --2- 172.21.15.45:0/3041833871 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f036c102070 0x7f036c108d30 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:10:55.460 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:55.459+0000 7f0371e00640 1 -- 172.21.15.45:0/3041833871 >> 172.21.15.45:0/3041833871 conn(0x7f036c0fd950 msgr2=0x7f036c0fec80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:10:55.460 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:55.459+0000 7f0371e00640 1 -- 172.21.15.45:0/3041833871 shutdown_connections 2024-04-16T17:10:55.460 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:10:55.459+0000 7f0371e00640 1 -- 172.21.15.45:0/3041833871 wait complete. 2024-04-16T17:10:56.246 INFO:tasks.workunit.client.0.smithi045.stdout: 1 59243 1.47 MB/sec execute 58 sec latency 21.105 ms 2024-04-16T17:10:56.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:10:56 smithi104 ceph-mon[36457]: pgmap v128: 129 pgs: 129 active+clean; 37 MiB data, 468 MiB used, 1.0 TiB / 1.0 TiB avail; 524 KiB/s rd, 843 KiB/s wr, 97 op/s 2024-04-16T17:10:57.054 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:10:56 smithi117 ceph-mon[36730]: pgmap v128: 129 pgs: 129 active+clean; 37 MiB data, 468 MiB used, 1.0 TiB / 1.0 TiB avail; 524 KiB/s rd, 843 KiB/s wr, 97 op/s 2024-04-16T17:10:57.246 INFO:tasks.workunit.client.0.smithi045.stdout: 1 60334 1.61 MB/sec execute 59 sec latency 30.081 ms 2024-04-16T17:10:57.630 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:10:54 smithi045 ceph-mon[31676]: pgmap v127: 129 pgs: 129 active+clean; 37 MiB data, 468 MiB used, 1.0 TiB / 1.0 TiB avail; 524 KiB/s rd, 843 KiB/s wr, 97 op/s 2024-04-16T17:10:57.630 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:10:56 smithi045 ceph-mon[31676]: pgmap v128: 129 pgs: 129 active+clean; 37 MiB data, 468 MiB used, 1.0 TiB / 1.0 TiB avail; 524 KiB/s rd, 843 KiB/s wr, 97 op/s 2024-04-16T17:10:58.247 INFO:tasks.workunit.client.0.smithi045.stdout: 1 60689 1.61 MB/sec execute 60 sec latency 62.550 ms 2024-04-16T17:10:58.930 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:10:58 smithi104 ceph-mon[36457]: pgmap v129: 129 pgs: 129 active+clean; 39 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 947 KiB/s rd, 1.1 MiB/s wr, 144 op/s 2024-04-16T17:10:58.930 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:10:58 smithi104 ceph-mon[36457]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T17:10:58.930 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:10:58 smithi104 ceph-mon[36457]: scrub summary: idle 2024-04-16T17:10:59.054 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:10:58 smithi117 ceph-mon[36730]: pgmap v129: 129 pgs: 129 active+clean; 39 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 947 KiB/s rd, 1.1 MiB/s wr, 144 op/s 2024-04-16T17:10:59.054 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:10:58 smithi117 ceph-mon[36730]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T17:10:59.055 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:10:58 smithi117 ceph-mon[36730]: scrub summary: idle 2024-04-16T17:10:59.247 INFO:tasks.workunit.client.0.smithi045.stdout: 1 60865 1.59 MB/sec execute 61 sec latency 85.246 ms 2024-04-16T17:10:59.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:10:59 smithi104 ceph-mon[36457]: scrub summary: idle 2024-04-16T17:10:59.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:10:59 smithi104 ceph-mon[36457]: scrub summary: idle 2024-04-16T17:10:59.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:10:58 smithi045 ceph-mon[31676]: pgmap v129: 129 pgs: 129 active+clean; 39 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 947 KiB/s rd, 1.1 MiB/s wr, 144 op/s 2024-04-16T17:10:59.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:10:58 smithi045 ceph-mon[31676]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T17:10:59.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:10:58 smithi045 ceph-mon[31676]: scrub summary: idle 2024-04-16T17:10:59.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:10:59 smithi045 ceph-mon[31676]: scrub summary: idle 2024-04-16T17:10:59.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:10:59 smithi045 ceph-mon[31676]: scrub summary: idle 2024-04-16T17:11:00.054 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:10:59 smithi117 ceph-mon[36730]: scrub summary: idle 2024-04-16T17:11:00.054 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:10:59 smithi117 ceph-mon[36730]: scrub summary: idle 2024-04-16T17:11:00.247 INFO:tasks.workunit.client.0.smithi045.stdout: 1 60995 1.56 MB/sec execute 62 sec latency 635.848 ms 2024-04-16T17:11:00.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:11:00 smithi104 ceph-mon[36457]: pgmap v130: 129 pgs: 129 active+clean; 39 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 823 KiB/s rd, 953 KiB/s wr, 106 op/s 2024-04-16T17:11:00.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:11:00 smithi104 ceph-mon[36457]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:11:00.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:11:00 smithi104 ceph-mon[36457]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T17:11:00.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:11:00 smithi104 ceph-mon[36457]: from='mgr.25411 ' entity='mgr.x' 2024-04-16T17:11:00.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:11:00 smithi045 ceph-mon[31676]: pgmap v130: 129 pgs: 129 active+clean; 39 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 823 KiB/s rd, 953 KiB/s wr, 106 op/s 2024-04-16T17:11:00.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:11:00 smithi045 ceph-mon[31676]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:11:00.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:11:00 smithi045 ceph-mon[31676]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T17:11:00.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:11:00 smithi045 ceph-mon[31676]: from='mgr.25411 ' entity='mgr.x' 2024-04-16T17:11:01.054 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:11:00 smithi117 ceph-mon[36730]: pgmap v130: 129 pgs: 129 active+clean; 39 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 823 KiB/s rd, 953 KiB/s wr, 106 op/s 2024-04-16T17:11:01.054 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:11:00 smithi117 ceph-mon[36730]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:11:01.054 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:11:00 smithi117 ceph-mon[36730]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T17:11:01.055 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:11:00 smithi117 ceph-mon[36730]: from='mgr.25411 ' entity='mgr.x' 2024-04-16T17:11:01.247 INFO:tasks.workunit.client.0.smithi045.stdout: 1 61119 1.54 MB/sec execute 63 sec latency 838.867 ms 2024-04-16T17:11:02.247 INFO:tasks.workunit.client.0.smithi045.stdout: 1 61119 1.52 MB/sec execute 64 sec latency 1544.662 ms 2024-04-16T17:11:02.770 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:11:02 smithi045 ceph-mon[31676]: pgmap v131: 129 pgs: 129 active+clean; 40 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 979 KiB/s rd, 1.1 MiB/s wr, 147 op/s 2024-04-16T17:11:02.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:11:02 smithi104 ceph-mon[36457]: pgmap v131: 129 pgs: 129 active+clean; 40 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 979 KiB/s rd, 1.1 MiB/s wr, 147 op/s 2024-04-16T17:11:03.054 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:11:02 smithi117 ceph-mon[36730]: pgmap v131: 129 pgs: 129 active+clean; 40 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 979 KiB/s rd, 1.1 MiB/s wr, 147 op/s 2024-04-16T17:11:03.247 INFO:tasks.workunit.client.0.smithi045.stdout: 1 61361 1.52 MB/sec execute 65 sec latency 1569.907 ms 2024-04-16T17:11:04.247 INFO:tasks.workunit.client.0.smithi045.stdout: 1 61421 1.50 MB/sec execute 66 sec latency 707.859 ms 2024-04-16T17:11:04.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:11:04 smithi104 ceph-mon[36457]: pgmap v132: 129 pgs: 129 active+clean; 40 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 580 KiB/s rd, 426 KiB/s wr, 88 op/s 2024-04-16T17:11:04.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:11:04 smithi045 ceph-mon[31676]: pgmap v132: 129 pgs: 129 active+clean; 40 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 580 KiB/s rd, 426 KiB/s wr, 88 op/s 2024-04-16T17:11:05.054 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:11:04 smithi117 ceph-mon[36730]: pgmap v132: 129 pgs: 129 active+clean; 40 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 580 KiB/s rd, 426 KiB/s wr, 88 op/s 2024-04-16T17:11:05.247 INFO:tasks.workunit.client.0.smithi045.stdout: 1 61421 1.48 MB/sec execute 67 sec latency 1707.954 ms 2024-04-16T17:11:06.249 INFO:tasks.workunit.client.0.smithi045.stdout: 1 61421 1.45 MB/sec execute 68 sec latency 2708.050 ms 2024-04-16T17:11:06.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:11:06 smithi104 ceph-mon[36457]: pgmap v133: 129 pgs: 129 active+clean; 40 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 580 KiB/s rd, 426 KiB/s wr, 88 op/s 2024-04-16T17:11:06.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:11:06 smithi045 ceph-mon[31676]: pgmap v133: 129 pgs: 129 active+clean; 40 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 580 KiB/s rd, 426 KiB/s wr, 88 op/s 2024-04-16T17:11:07.055 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:11:06 smithi117 ceph-mon[36730]: pgmap v133: 129 pgs: 129 active+clean; 40 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 580 KiB/s rd, 426 KiB/s wr, 88 op/s 2024-04-16T17:11:07.248 INFO:tasks.workunit.client.0.smithi045.stdout: 1 61421 1.43 MB/sec execute 69 sec latency 3708.154 ms 2024-04-16T17:11:08.248 INFO:tasks.workunit.client.0.smithi045.stdout: 1 61835 1.46 MB/sec execute 70 sec latency 3727.739 ms 2024-04-16T17:11:08.592 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:08.592+0000 7f5c99e00640 1 -- 172.21.15.45:0/3046465009 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5c94102a40 msgr2=0x7f5c94106a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:08.592 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:08.592+0000 7f5c99e00640 1 --2- 172.21.15.45:0/3046465009 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5c94102a40 0x7f5c94106a80 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f5c80009860 tx=0x7f5c8002bb70 comp rx=0 tx=0).stop 2024-04-16T17:11:08.592 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:08.592+0000 7f5c99e00640 1 -- 172.21.15.45:0/3046465009 shutdown_connections 2024-04-16T17:11:08.593 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:08.592+0000 7f5c99e00640 1 --2- 172.21.15.45:0/3046465009 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5c94107160 0x7f5c9410ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:08.593 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:08.592+0000 7f5c99e00640 1 --2- 172.21.15.45:0/3046465009 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5c94102a40 0x7f5c94106a80 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:08.593 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:08.592+0000 7f5c99e00640 1 --2- 172.21.15.45:0/3046465009 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5c94102090 0x7f5c94102470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:08.593 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:08.592+0000 7f5c99e00640 1 -- 172.21.15.45:0/3046465009 >> 172.21.15.45:0/3046465009 conn(0x7f5c940fd950 msgr2=0x7f5c940ffd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:11:08.593 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:08.592+0000 7f5c99e00640 1 -- 172.21.15.45:0/3046465009 shutdown_connections 2024-04-16T17:11:08.593 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:08.592+0000 7f5c99e00640 1 -- 172.21.15.45:0/3046465009 wait complete. 2024-04-16T17:11:08.593 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:08.594+0000 7f5c99e00640 1 Processor -- start 2024-04-16T17:11:08.596 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:08.594+0000 7f5c99e00640 1 -- start start 2024-04-16T17:11:08.596 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:08.595+0000 7f5c99e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5c94102090 0x7f5c941a39f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:08.597 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:08.595+0000 7f5c99e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5c94107160 0x7f5c941a3f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:08.597 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:08.595+0000 7f5c99e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5c9410df30 0x7f5c9410e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:08.597 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:08.595+0000 7f5c99e00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f5c94112250 con 0x7f5c94102090 2024-04-16T17:11:08.597 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:08.595+0000 7f5c99e00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f5c941120d0 con 0x7f5c9410df30 2024-04-16T17:11:08.597 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:08.595+0000 7f5c99e00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f5c941123d0 con 0x7f5c94107160 2024-04-16T17:11:08.597 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:08.595+0000 7f5c99400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5c9410df30 0x7f5c9410e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:08.597 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:08.595+0000 7f5c99400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5c9410df30 0x7f5c9410e3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:58012/0 (socket says 172.21.15.45:58012) 2024-04-16T17:11:08.598 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:08.595+0000 7f5c99400640 1 -- 172.21.15.45:0/793884466 learned_addr learned my addr 172.21.15.45:0/793884466 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:11:08.598 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:08.595+0000 7f5c98a00640 1 --2- 172.21.15.45:0/793884466 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5c94102090 0x7f5c941a39f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:08.598 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:08.595+0000 7f5c93e00640 1 --2- 172.21.15.45:0/793884466 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5c94107160 0x7f5c941a3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:08.598 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:08.596+0000 7f5c99400640 1 -- 172.21.15.45:0/793884466 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5c94107160 msgr2=0x7f5c941a3f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:08.599 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:08.596+0000 7f5c99400640 1 --2- 172.21.15.45:0/793884466 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5c94107160 0x7f5c941a3f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:08.599 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:08.596+0000 7f5c99400640 1 -- 172.21.15.45:0/793884466 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5c94102090 msgr2=0x7f5c941a39f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:08.599 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:08.596+0000 7f5c99400640 1 --2- 172.21.15.45:0/793884466 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5c94102090 0x7f5c941a39f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:08.599 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:08.596+0000 7f5c99400640 1 -- 172.21.15.45:0/793884466 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c9410ea90 con 0x7f5c9410df30 2024-04-16T17:11:08.599 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:08.596+0000 7f5c93e00640 1 --2- 172.21.15.45:0/793884466 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5c94107160 0x7f5c941a3f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:11:08.599 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:08.596+0000 7f5c99400640 1 --2- 172.21.15.45:0/793884466 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5c9410df30 0x7f5c9410e3e0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f5c8c00b9a0 tx=0x7f5c8c00bda0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:11:08.600 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:08.599+0000 7f5c91600640 1 -- 172.21.15.45:0/793884466 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c8c019070 con 0x7f5c9410df30 2024-04-16T17:11:08.600 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:08.599+0000 7f5c91600640 1 -- 172.21.15.45:0/793884466 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5c8c014440 con 0x7f5c9410df30 2024-04-16T17:11:08.600 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:08.599+0000 7f5c91600640 1 -- 172.21.15.45:0/793884466 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c8c0075b0 con 0x7f5c9410df30 2024-04-16T17:11:08.600 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:08.599+0000 7f5c99e00640 1 -- 172.21.15.45:0/793884466 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c9406ba20 con 0x7f5c9410df30 2024-04-16T17:11:08.600 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:08.599+0000 7f5c99e00640 1 -- 172.21.15.45:0/793884466 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c9406bf40 con 0x7f5c9410df30 2024-04-16T17:11:08.602 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:08.602+0000 7f5c9e1cf740 1 Processor -- start 2024-04-16T17:11:08.602 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:08.603+0000 7f5c91600640 1 -- 172.21.15.45:0/793884466 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f5c8c01c070 con 0x7f5c9410df30 2024-04-16T17:11:08.603 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:08.603+0000 7f5c9e1cf740 1 -- start start 2024-04-16T17:11:08.604 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:08.603+0000 7f5c9e1cf740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562a0abad7f0 0x562a0abac540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:08.604 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:08.603+0000 7f5c9e1cf740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562a0abaca80 0x562a0abacf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:08.604 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:08.603+0000 7f5c9e1cf740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562a0abb6000 0x562a0abb83f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:08.604 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:08.603+0000 7f5c9e1cf740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x562a0ab11780 con 0x562a0abb6000 2024-04-16T17:11:08.604 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:08.603+0000 7f5c9e1cf740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x562a0aab7f40 con 0x562a0abaca80 2024-04-16T17:11:08.604 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:08.603+0000 7f5c9e1cf740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x562a0ab0d740 con 0x562a0abad7f0 2024-04-16T17:11:08.605 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:08.604+0000 7f5c93e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562a0abb6000 0x562a0abb83f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:08.605 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:08.604+0000 7f5c93e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562a0abb6000 0x562a0abb83f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43800/0 (socket says 172.21.15.45:43800) 2024-04-16T17:11:08.606 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:08.604+0000 7f5c93e00640 1 -- 172.21.15.45:0/2090509637 learned_addr learned my addr 172.21.15.45:0/2090509637 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:11:08.606 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:08.604+0000 7f5c98a00640 1 --2- 172.21.15.45:0/2090509637 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562a0abaca80 0x562a0abacf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:08.606 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:08.604+0000 7f5c93e00640 1 -- 172.21.15.45:0/2090509637 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562a0abad7f0 msgr2=0x562a0abac540 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-16T17:11:08.606 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:08.604+0000 7f5c93e00640 1 --2- 172.21.15.45:0/2090509637 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562a0abad7f0 0x562a0abac540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:08.606 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:08.604+0000 7f5c93e00640 1 -- 172.21.15.45:0/2090509637 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562a0abaca80 msgr2=0x562a0abacf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:11:08.606 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:08.604+0000 7f5c93e00640 1 --2- 172.21.15.45:0/2090509637 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562a0abaca80 0x562a0abacf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:08.606 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:08.604+0000 7f5c93e00640 1 -- 172.21.15.45:0/2090509637 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562a0a824210 con 0x562a0abb6000 2024-04-16T17:11:08.606 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:08.604+0000 7f5c98a00640 1 --2- 172.21.15.45:0/2090509637 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562a0abaca80 0x562a0abacf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:11:08.606 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:08.605+0000 7f5c93e00640 1 --2- 172.21.15.45:0/2090509637 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562a0abb6000 0x562a0abb83f0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f5c9406b460 tx=0x7f5c800323f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6451391a8fb64a4 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:11:08.606 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:08.605+0000 7f5c91600640 1 --2- 172.21.15.45:0/793884466 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f5c7c0a3d90 0x7f5c7c0a6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:08.612 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:08.605+0000 7f5c91600640 1 -- 172.21.15.45:0/793884466 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f5c8c0cfff0 con 0x7f5c9410df30 2024-04-16T17:11:08.612 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:08.605+0000 7f5c85c00640 1 -- 172.21.15.45:0/2090509637 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c80042020 con 0x562a0abb6000 2024-04-16T17:11:08.613 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:08.605+0000 7f5c85c00640 1 -- 172.21.15.45:0/2090509637 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5c80035d40 con 0x562a0abb6000 2024-04-16T17:11:08.613 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:08.605+0000 7f5c98a00640 1 --2- 172.21.15.45:0/793884466 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f5c7c0a3d90 0x7f5c7c0a6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:08.613 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:08.605+0000 7f5c85c00640 1 -- 172.21.15.45:0/2090509637 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c80039040 con 0x562a0abb6000 2024-04-16T17:11:08.613 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:08.606+0000 7f5c9e1cf740 1 -- 172.21.15.45:0/2090509637 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562a0abb6000 msgr2=0x562a0abb83f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:08.613 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:08.606+0000 7f5c98a00640 1 --2- 172.21.15.45:0/793884466 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f5c7c0a3d90 0x7f5c7c0a6250 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x562a0abadf50 tx=0x7f5c88009260 comp rx=0 tx=0).ready entity=mgr.25411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:11:08.613 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:08.606+0000 7f5c9e1cf740 1 --2- 172.21.15.45:0/2090509637 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562a0abb6000 0x562a0abb83f0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f5c9406b460 tx=0x7f5c800323f0 comp rx=0 tx=0).stop 2024-04-16T17:11:08.613 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:08.606+0000 7f5c9e1cf740 1 -- 172.21.15.45:0/2090509637 shutdown_connections 2024-04-16T17:11:08.613 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:08.606+0000 7f5c9e1cf740 1 --2- 172.21.15.45:0/2090509637 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562a0abb6000 0x562a0abb83f0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:08.613 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:08.606+0000 7f5c9e1cf740 1 --2- 172.21.15.45:0/2090509637 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562a0abaca80 0x562a0abacf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:08.613 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:08.606+0000 7f5c9e1cf740 1 --2- 172.21.15.45:0/2090509637 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562a0abad7f0 0x562a0abac540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:08.614 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:08.606+0000 7f5c9e1cf740 1 -- 172.21.15.45:0/2090509637 >> 172.21.15.45:0/2090509637 conn(0x562a0ab475d0 msgr2=0x562a0aba7770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:11:08.614 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:08.606+0000 7f5c9e1cf740 1 -- 172.21.15.45:0/2090509637 shutdown_connections 2024-04-16T17:11:08.614 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:08.606+0000 7f5c9e1cf740 1 -- 172.21.15.45:0/2090509637 wait complete. 2024-04-16T17:11:08.614 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:08.607+0000 7f5c9e1cf740 1 -- start start 2024-04-16T17:11:08.614 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:08.607+0000 7f5c9e1cf740 1 Processor -- start 2024-04-16T17:11:08.614 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:08.608+0000 7f5c9e1cf740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562a0abaca80 0x562a0abb2f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:08.614 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:08.608+0000 7f5c9e1cf740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562a0abad7f0 0x562a0abb34a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:08.614 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:08.608+0000 7f5c9e1cf740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562a0abb6000 0x562a0abb39e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:08.614 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:08.608+0000 7f5c9e1cf740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x562a0abb8cd0 con 0x562a0abaca80 2024-04-16T17:11:08.615 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:08.608+0000 7f5c9e1cf740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x562a0aacb390 con 0x562a0abad7f0 2024-04-16T17:11:08.615 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:08.608+0000 7f5c9e1cf740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x562a0abb8e50 con 0x562a0abb6000 2024-04-16T17:11:08.615 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:08.608+0000 7f5c9e1cf740 10 client.0 _ll_register_callbacks cb 0x562a0aab4f60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:11:08.615 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:08.608+0000 7f5c98a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562a0abaca80 0x562a0abb2f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:08.615 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:08.608+0000 7f5c98a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562a0abaca80 0x562a0abb2f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43808/0 (socket says 172.21.15.45:43808) 2024-04-16T17:11:08.615 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:08.608+0000 7f5c98a00640 1 -- 172.21.15.45:0/2804605735 learned_addr learned my addr 172.21.15.45:0/2804605735 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:11:08.615 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:08.608+0000 7f5c93e00640 1 --2- 172.21.15.45:0/2804605735 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562a0abad7f0 0x562a0abb34a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:08.615 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:08.608+0000 7f5c99400640 1 --2- 172.21.15.45:0/2804605735 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562a0abb6000 0x562a0abb39e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:08.615 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:08.609+0000 7f5c98a00640 1 -- 172.21.15.45:0/2804605735 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562a0abb6000 msgr2=0x562a0abb39e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:11:08.615 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:08.609+0000 7f5c98a00640 1 --2- 172.21.15.45:0/2804605735 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562a0abb6000 0x562a0abb39e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:08.615 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:08.609+0000 7f5c98a00640 1 -- 172.21.15.45:0/2804605735 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562a0abad7f0 msgr2=0x562a0abb34a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:11:08.616 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:08.609+0000 7f5c98a00640 1 --2- 172.21.15.45:0/2804605735 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562a0abad7f0 0x562a0abb34a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:08.616 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:08.609+0000 7f5c98a00640 1 -- 172.21.15.45:0/2804605735 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562a0a8f0620 con 0x562a0abaca80 2024-04-16T17:11:08.616 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:08.609+0000 7f5c98a00640 1 --2- 172.21.15.45:0/2804605735 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562a0abaca80 0x562a0abb2f60 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f5c8801ea00 tx=0x7f5c88004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2d3e203fdea155e server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:11:08.616 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:08.609+0000 7f5c7a000640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-16T17:11:08.616 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:08.609+0000 7f5c7a000640 1 -- 172.21.15.45:0/2804605735 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c8802e060 con 0x562a0abaca80 2024-04-16T17:11:08.616 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:08.609+0000 7f5c7a000640 1 -- 172.21.15.45:0/2804605735 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f5c88018030 con 0x562a0abaca80 2024-04-16T17:11:08.616 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:08.609+0000 7f5c9e1cf740 1 -- 172.21.15.45:0/2804605735 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5c80035d40 con 0x562a0abaca80 2024-04-16T17:11:08.616 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:08.610+0000 7f5c7a000640 1 -- 172.21.15.45:0/2804605735 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f5c88029420 con 0x562a0abaca80 2024-04-16T17:11:08.616 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:08.610+0000 7f5c9e1cf740 10 client.16665 fetch_fsmap learned FSMap version 31 2024-04-16T17:11:08.616 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:08.610+0000 7f5c9e1cf740 1 -- 172.21.15.45:0/2804605735 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x562a0abc4f40 con 0x562a0abaca80 2024-04-16T17:11:08.616 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:08.610+0000 7f5c7a000640 1 -- 172.21.15.45:0/2804605735 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5c80035d40 con 0x562a0abaca80 2024-04-16T17:11:08.619 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:08.611+0000 7f5c7a000640 1 -- 172.21.15.45:0/2804605735 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 31) ==== 8366+0+0 (secure 0 0 0) 0x7f5c8800f6c0 con 0x562a0abaca80 2024-04-16T17:11:08.619 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:08.611+0000 7f5c9e1cf740 10 client.16665 fetch_fsmap finished waiting for FSMap version 31 2024-04-16T17:11:08.619 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:08.611+0000 7f5c9e1cf740 10 client.16665 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T17:11:08.619 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:08.611+0000 7f5c9e1cf740 20 client.16665 populate_metadata read hostname 'smithi045' 2024-04-16T17:11:08.619 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:08.611+0000 7f5c9e1cf740 1 --2- 172.21.15.45:0/2804605735 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x562a0abc6400 0x562a0abc8870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:08.619 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:08.611+0000 7f5c9e1cf740 4 client.16665 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:11:08.619 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:08.611+0000 7f5c9e1cf740 1 -- 172.21.15.45:0/2804605735 --> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562a0ab1f1a0 con 0x562a0abc6400 2024-04-16T17:11:08.619 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:08.611+0000 7f5c93e00640 1 --2- 172.21.15.45:0/2804605735 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x562a0abc6400 0x562a0abc8870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:08.619 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:08.612+0000 7f5c93e00640 1 --2- 172.21.15.45:0/2804605735 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x562a0abc6400 0x562a0abc8870 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a76d104734ddc273 server_cookie=6231152e00085643 in_seq=0 out_seq=0 2024-04-16T17:11:08.619 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:08.612+0000 7f5c7a000640 10 client.16665 ms_handle_connect on v2:172.21.15.104:6838/3660070592 2024-04-16T17:11:08.619 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:08.613+0000 7f5c7a000640 1 -- 172.21.15.45:0/2804605735 <== mds.0 v2:172.21.15.104:6838/3660070592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562a0ab1f1a0 con 0x562a0abc6400 2024-04-16T17:11:08.620 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:08.613+0000 7f5c7a000640 10 client.16665 handle_command_reply: tid=0 2024-04-16T17:11:08.620 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:08.613+0000 7f5c9e1cf740 1 client.16665 shutdown 2024-04-16T17:11:08.620 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:08.613+0000 7f5c9e1cf740 1 -- 172.21.15.45:0/2804605735 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562a0abaca80 msgr2=0x562a0abb2f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:08.620 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:08.613+0000 7f5c9e1cf740 1 --2- 172.21.15.45:0/2804605735 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562a0abaca80 0x562a0abb2f60 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f5c8801ea00 tx=0x7f5c88004730 comp rx=0 tx=0).stop 2024-04-16T17:11:08.620 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:08.613+0000 7f5c9e1cf740 1 -- 172.21.15.45:0/2804605735 shutdown_connections 2024-04-16T17:11:08.620 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:08.613+0000 7f5c9e1cf740 1 --2- 172.21.15.45:0/2804605735 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x562a0abc6400 0x562a0abc8870 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:08.620 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:08.613+0000 7f5c9e1cf740 1 --2- 172.21.15.45:0/2804605735 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562a0abb6000 0x562a0abb39e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:08.620 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:08.613+0000 7f5c9e1cf740 1 --2- 172.21.15.45:0/2804605735 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562a0abad7f0 0x562a0abb34a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:08.620 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:08.613+0000 7f5c7a000640 0 client.16665 ms_handle_reset on v2:172.21.15.104:6838/3660070592 2024-04-16T17:11:08.620 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:08.613+0000 7f5c9e1cf740 1 --2- 172.21.15.45:0/2804605735 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562a0abaca80 0x562a0abb2f60 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:08.620 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:08.613+0000 7f5c9e1cf740 1 -- 172.21.15.45:0/2804605735 >> 172.21.15.45:0/2804605735 conn(0x562a0ab475d0 msgr2=0x562a0aba94c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:11:08.620 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:08.613+0000 7f5c9e1cf740 1 -- 172.21.15.45:0/2804605735 shutdown_connections 2024-04-16T17:11:08.621 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:08.613+0000 7f5c9e1cf740 1 -- 172.21.15.45:0/2804605735 wait complete. 2024-04-16T17:11:08.621 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:08.614+0000 7f5c9e1cf740 20 client.16665 trim_cache size 0 max 16384 2024-04-16T17:11:08.640 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:08.638+0000 7f5c9e1cf740 1 Processor -- start 2024-04-16T17:11:08.640 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:08.638+0000 7f5c9e1cf740 1 -- start start 2024-04-16T17:11:08.640 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:08.639+0000 7f5c9e1cf740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562a0abaca80 0x562a0abc1bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:08.640 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:08.639+0000 7f5c9e1cf740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562a0abad7f0 0x562a0abb3800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:08.640 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:08.639+0000 7f5c9e1cf740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562a0abb6000 0x562a0abce180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:08.640 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:08.639+0000 7f5c9e1cf740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f5c941c0a80 con 0x562a0abb6000 2024-04-16T17:11:08.641 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:08.639+0000 7f5c9e1cf740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f5c941be290 con 0x562a0abaca80 2024-04-16T17:11:08.641 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:08.639+0000 7f5c9e1cf740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f5c940739c0 con 0x562a0abad7f0 2024-04-16T17:11:08.641 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:08.639+0000 7f5c93e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562a0abad7f0 0x562a0abb3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:08.641 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:08.639+0000 7f5c98a00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562a0abaca80 0x562a0abc1bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:08.641 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:08.639+0000 7f5c93e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562a0abad7f0 0x562a0abb3800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:48628/0 (socket says 172.21.15.45:48628) 2024-04-16T17:11:08.641 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:08.639+0000 7f5c98a00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562a0abaca80 0x562a0abc1bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:58028/0 (socket says 172.21.15.45:58028) 2024-04-16T17:11:08.641 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:08.639+0000 7f5c93e00640 1 -- 172.21.15.45:0/1483123394 learned_addr learned my addr 172.21.15.45:0/1483123394 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:11:08.641 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:08.639+0000 7f5c99400640 1 --2- 172.21.15.45:0/1483123394 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562a0abb6000 0x562a0abce180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:08.641 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:08.640+0000 7f5c93e00640 1 -- 172.21.15.45:0/1483123394 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562a0abaca80 msgr2=0x562a0abc1bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:11:08.641 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:08.640+0000 7f5c93e00640 1 --2- 172.21.15.45:0/1483123394 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562a0abaca80 0x562a0abc1bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:08.641 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:08.640+0000 7f5c93e00640 1 -- 172.21.15.45:0/1483123394 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562a0abb6000 msgr2=0x562a0abce180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:11:08.641 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:08.640+0000 7f5c93e00640 1 --2- 172.21.15.45:0/1483123394 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562a0abb6000 0x562a0abce180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:08.642 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:08.640+0000 7f5c93e00640 1 -- 172.21.15.45:0/1483123394 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562a0abb8c10 con 0x562a0abad7f0 2024-04-16T17:11:08.642 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:08.640+0000 7f5c98a00640 1 --2- 172.21.15.45:0/1483123394 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562a0abaca80 0x562a0abc1bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:11:08.642 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:08.640+0000 7f5c99400640 1 --2- 172.21.15.45:0/1483123394 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562a0abb6000 0x562a0abce180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:11:08.642 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:08.640+0000 7f5c93e00640 1 --2- 172.21.15.45:0/1483123394 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562a0abad7f0 0x562a0abb3800 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f5c8002c260 tx=0x7f5c8002c1f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aa88aa893ac822b7 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:11:08.642 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:08.640+0000 7f5c85200640 1 -- 172.21.15.45:0/1483123394 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c80042050 con 0x562a0abad7f0 2024-04-16T17:11:08.642 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:08.640+0000 7f5c85200640 1 -- 172.21.15.45:0/1483123394 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5c8004f070 con 0x562a0abad7f0 2024-04-16T17:11:08.642 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:08.640+0000 7f5c85200640 1 -- 172.21.15.45:0/1483123394 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c80039040 con 0x562a0abad7f0 2024-04-16T17:11:08.646 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:08.641+0000 7f5c9e1cf740 1 -- 172.21.15.45:0/1483123394 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562a0abad7f0 msgr2=0x562a0abb3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:08.646 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:08.641+0000 7f5c9e1cf740 1 --2- 172.21.15.45:0/1483123394 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562a0abad7f0 0x562a0abb3800 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f5c8002c260 tx=0x7f5c8002c1f0 comp rx=0 tx=0).stop 2024-04-16T17:11:08.646 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:08.642+0000 7f5c9e1cf740 1 -- 172.21.15.45:0/1483123394 shutdown_connections 2024-04-16T17:11:08.646 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:08.642+0000 7f5c9e1cf740 1 --2- 172.21.15.45:0/1483123394 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562a0abad7f0 0x562a0abb3800 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:08.646 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:08.642+0000 7f5c9e1cf740 1 --2- 172.21.15.45:0/1483123394 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562a0abb6000 0x562a0abce180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:08.646 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:08.642+0000 7f5c9e1cf740 1 --2- 172.21.15.45:0/1483123394 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562a0abaca80 0x562a0abc1bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:08.646 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:08.642+0000 7f5c9e1cf740 1 -- 172.21.15.45:0/1483123394 >> 172.21.15.45:0/1483123394 conn(0x562a0ab475d0 msgr2=0x562a0aba9450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:11:08.646 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:08.642+0000 7f5c9e1cf740 1 -- 172.21.15.45:0/1483123394 shutdown_connections 2024-04-16T17:11:08.646 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:08.642+0000 7f5c9e1cf740 1 -- 172.21.15.45:0/1483123394 wait complete. 2024-04-16T17:11:08.646 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:08.642+0000 7f5c9e1cf740 1 -- start start 2024-04-16T17:11:08.646 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:08.642+0000 7f5c9e1cf740 1 Processor -- start 2024-04-16T17:11:08.647 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:08.643+0000 7f5c9e1cf740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562a0abaca80 0x562a0abb6d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:08.647 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:08.643+0000 7f5c9e1cf740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562a0abad7f0 0x562a0abb7250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:08.647 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:08.643+0000 7f5c9e1cf740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562a0abb6000 0x562a0abb7790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:08.647 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:08.643+0000 7f5c9e1cf740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f5c94074050 con 0x562a0abb6000 2024-04-16T17:11:08.647 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:08.643+0000 7f5c9e1cf740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f5c941bdbc0 con 0x562a0abad7f0 2024-04-16T17:11:08.647 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:08.643+0000 7f5c9e1cf740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f5c941bce20 con 0x562a0abaca80 2024-04-16T17:11:08.647 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:08.643+0000 7f5c9e1cf740 10 client.0 _ll_register_callbacks cb 0x7f5c88009460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:11:08.647 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:08.643+0000 7f5c99400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562a0abb6000 0x562a0abb7790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:08.647 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:08.643+0000 7f5c99400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562a0abb6000 0x562a0abb7790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43838/0 (socket says 172.21.15.45:43838) 2024-04-16T17:11:08.647 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:08.643+0000 7f5c99400640 1 -- 172.21.15.45:0/3054383161 learned_addr learned my addr 172.21.15.45:0/3054383161 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:11:08.647 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:08.643+0000 7f5c93e00640 1 --2- 172.21.15.45:0/3054383161 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562a0abad7f0 0x562a0abb7250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:08.647 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:08.643+0000 7f5c93e00640 1 -- 172.21.15.45:0/3054383161 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562a0abaca80 msgr2=0x562a0abb6d10 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-16T17:11:08.648 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:08.643+0000 7f5c93e00640 1 --2- 172.21.15.45:0/3054383161 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562a0abaca80 0x562a0abb6d10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:08.648 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:08.643+0000 7f5c93e00640 1 -- 172.21.15.45:0/3054383161 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562a0abb6000 msgr2=0x562a0abb7790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:11:08.648 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:08.643+0000 7f5c93e00640 1 --2- 172.21.15.45:0/3054383161 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562a0abb6000 0x562a0abb7790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:08.648 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:08.643+0000 7f5c93e00640 1 -- 172.21.15.45:0/3054383161 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562a0abbc6d0 con 0x562a0abad7f0 2024-04-16T17:11:08.648 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:08.643+0000 7f5c99400640 1 --2- 172.21.15.45:0/3054383161 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562a0abb6000 0x562a0abb7790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:11:08.648 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:08.643+0000 7f5c93e00640 1 --2- 172.21.15.45:0/3054383161 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562a0abad7f0 0x562a0abb7250 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x562a0abb5f50 tx=0x7f5c800348b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=da7f13c7afde806d server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:11:08.648 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:08.643+0000 7f5c7a000640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-04-16T17:11:08.648 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:08.643+0000 7f5c7a000640 1 -- 172.21.15.45:0/3054383161 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c80054060 con 0x562a0abad7f0 2024-04-16T17:11:08.648 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:08.643+0000 7f5c9e1cf740 1 -- 172.21.15.45:0/3054383161 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5c8004f070 con 0x562a0abad7f0 2024-04-16T17:11:08.648 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:08.643+0000 7f5c7a000640 1 -- 172.21.15.45:0/3054383161 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f5c800042c0 con 0x562a0abad7f0 2024-04-16T17:11:08.648 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:08.643+0000 7f5c7a000640 1 -- 172.21.15.45:0/3054383161 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f5c80042020 con 0x562a0abad7f0 2024-04-16T17:11:08.648 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:08.643+0000 7f5c9e1cf740 10 client.26834 fetch_fsmap learned FSMap version 31 2024-04-16T17:11:08.648 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:08.643+0000 7f5c9e1cf740 1 -- 172.21.15.45:0/3054383161 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x562a0ac2ceb0 con 0x562a0abad7f0 2024-04-16T17:11:08.649 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:08.643+0000 7f5c7a000640 1 -- 172.21.15.45:0/3054383161 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5c8004f070 con 0x562a0abad7f0 2024-04-16T17:11:08.649 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:08.643+0000 7f5c7a000640 1 -- 172.21.15.45:0/3054383161 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 31) ==== 8366+0+0 (secure 0 0 0) 0x7f5c80032a20 con 0x562a0abad7f0 2024-04-16T17:11:08.649 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:08.643+0000 7f5c9e1cf740 10 client.26834 fetch_fsmap finished waiting for FSMap version 31 2024-04-16T17:11:08.649 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:08.643+0000 7f5c9e1cf740 10 client.26834 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T17:11:08.649 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:08.643+0000 7f5c9e1cf740 20 client.26834 populate_metadata read hostname 'smithi045' 2024-04-16T17:11:08.649 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:08.643+0000 7f5c9e1cf740 1 --2- 172.21.15.45:0/3054383161 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x562a0abc6400 0x562a0ac2f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:08.649 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:08.643+0000 7f5c9e1cf740 4 client.26834 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-16T17:11:08.649 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:08.643+0000 7f5c9e1cf740 1 -- 172.21.15.45:0/3054383161 --> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562a0ab19010 con 0x562a0abc6400 2024-04-16T17:11:08.649 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:08.643+0000 7f5c98a00640 1 --2- 172.21.15.45:0/3054383161 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x562a0abc6400 0x562a0ac2f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:08.649 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:08.643+0000 7f5c98a00640 1 --2- 172.21.15.45:0/3054383161 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x562a0abc6400 0x562a0ac2f1a0 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b3da77ea48866ef6 server_cookie=95b4c2c145ebb58a in_seq=0 out_seq=0 2024-04-16T17:11:08.649 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:08.643+0000 7f5c7a000640 10 client.26834 ms_handle_connect on v2:172.21.15.104:6838/3660070592 2024-04-16T17:11:08.676 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:08.676+0000 7f5c7a000640 1 -- 172.21.15.45:0/3054383161 <== mds.0 v2:172.21.15.104:6838/3660070592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x562a0ab19010 con 0x562a0abc6400 2024-04-16T17:11:08.676 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:08.676+0000 7f5c7a000640 10 client.26834 handle_command_reply: tid=0 2024-04-16T17:11:08.676 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:08.676+0000 7f5c9e1cf740 1 client.26834 shutdown 2024-04-16T17:11:08.677 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:08.677+0000 7f5c9e1cf740 1 -- 172.21.15.45:0/3054383161 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562a0abad7f0 msgr2=0x562a0abb7250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:08.677 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:08.677+0000 7f5c9e1cf740 1 --2- 172.21.15.45:0/3054383161 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562a0abad7f0 0x562a0abb7250 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x562a0abb5f50 tx=0x7f5c800348b0 comp rx=0 tx=0).stop 2024-04-16T17:11:08.677 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:08.677+0000 7f5c9e1cf740 1 -- 172.21.15.45:0/3054383161 shutdown_connections 2024-04-16T17:11:08.677 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:08.677+0000 7f5c9e1cf740 1 --2- 172.21.15.45:0/3054383161 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562a0abb6000 0x562a0abb7790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:08.677 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:08.677+0000 7f5c9e1cf740 1 --2- 172.21.15.45:0/3054383161 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562a0abad7f0 0x562a0abb7250 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:08.678 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:08.677+0000 7f5c9e1cf740 1 --2- 172.21.15.45:0/3054383161 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x562a0abc6400 0x562a0ac2f1a0 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:08.678 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:08.677+0000 7f5c9e1cf740 1 --2- 172.21.15.45:0/3054383161 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562a0abaca80 0x562a0abb6d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:08.678 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:08.677+0000 7f5c7a000640 0 client.26834 ms_handle_reset on v2:172.21.15.104:6838/3660070592 2024-04-16T17:11:08.678 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:08.677+0000 7f5c9e1cf740 1 -- 172.21.15.45:0/3054383161 >> 172.21.15.45:0/3054383161 conn(0x562a0ab475d0 msgr2=0x562a0aba6810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:11:08.678 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:08.678+0000 7f5c9e1cf740 1 -- 172.21.15.45:0/3054383161 shutdown_connections 2024-04-16T17:11:08.679 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:08.678+0000 7f5c9e1cf740 1 -- 172.21.15.45:0/3054383161 wait complete. 2024-04-16T17:11:08.679 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:08.678+0000 7f5c9e1cf740 20 client.26834 trim_cache size 0 max 16384 2024-04-16T17:11:08.679 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-04-16T17:11:08.679 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-04-16T17:11:08.679 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-04-16T17:11:08.679 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-04-16T17:11:08.680 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:08.680+0000 7f5c99e00640 1 -- 172.21.15.45:0/793884466 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f5c7c0a3d90 msgr2=0x7f5c7c0a6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:08.680 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:08.680+0000 7f5c99e00640 1 --2- 172.21.15.45:0/793884466 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f5c7c0a3d90 0x7f5c7c0a6250 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x562a0abadf50 tx=0x7f5c88009260 comp rx=0 tx=0).stop 2024-04-16T17:11:08.680 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:08.680+0000 7f5c99e00640 1 -- 172.21.15.45:0/793884466 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5c9410df30 msgr2=0x7f5c9410e3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:08.680 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:08.680+0000 7f5c99e00640 1 --2- 172.21.15.45:0/793884466 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5c9410df30 0x7f5c9410e3e0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f5c8c00b9a0 tx=0x7f5c8c00bda0 comp rx=0 tx=0).stop 2024-04-16T17:11:08.680 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:08.680+0000 7f5c99e00640 1 -- 172.21.15.45:0/793884466 shutdown_connections 2024-04-16T17:11:08.680 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:08.680+0000 7f5c99e00640 1 --2- 172.21.15.45:0/793884466 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f5c7c0a3d90 0x7f5c7c0a6250 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:08.680 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:08.680+0000 7f5c99e00640 1 --2- 172.21.15.45:0/793884466 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5c94107160 0x7f5c941a3f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:08.680 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:08.680+0000 7f5c99e00640 1 --2- 172.21.15.45:0/793884466 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5c9410df30 0x7f5c9410e3e0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:08.680 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:08.680+0000 7f5c99e00640 1 --2- 172.21.15.45:0/793884466 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5c94102090 0x7f5c941a39f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:08.680 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:08.680+0000 7f5c99e00640 1 -- 172.21.15.45:0/793884466 >> 172.21.15.45:0/793884466 conn(0x7f5c940fd950 msgr2=0x7f5c940ffd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:11:08.681 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:08.681+0000 7f5c99e00640 1 -- 172.21.15.45:0/793884466 shutdown_connections 2024-04-16T17:11:08.681 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:08.681+0000 7f5c99e00640 1 -- 172.21.15.45:0/793884466 wait complete. 2024-04-16T17:11:08.857 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:11:08 smithi117 ceph-mon[36730]: pgmap v134: 129 pgs: 129 active+clean; 41 MiB data, 522 MiB used, 1.0 TiB / 1.0 TiB avail; 634 KiB/s rd, 560 KiB/s wr, 118 op/s 2024-04-16T17:11:08.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:11:08 smithi104 ceph-mon[36457]: pgmap v134: 129 pgs: 129 active+clean; 41 MiB data, 522 MiB used, 1.0 TiB / 1.0 TiB avail; 634 KiB/s rd, 560 KiB/s wr, 118 op/s 2024-04-16T17:11:09.248 INFO:tasks.workunit.client.0.smithi045.stdout: 1 62175 1.44 MB/sec execute 71 sec latency 72.605 ms 2024-04-16T17:11:10.248 INFO:tasks.workunit.client.0.smithi045.stdout: 1 62538 1.43 MB/sec execute 72 sec latency 60.001 ms 2024-04-16T17:11:10.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:11:10 smithi104 ceph-mon[36457]: pgmap v135: 129 pgs: 129 active+clean; 41 MiB data, 522 MiB used, 1.0 TiB / 1.0 TiB avail; 211 KiB/s rd, 318 KiB/s wr, 71 op/s 2024-04-16T17:11:11.055 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:11:10 smithi117 ceph-mon[36730]: pgmap v135: 129 pgs: 129 active+clean; 41 MiB data, 522 MiB used, 1.0 TiB / 1.0 TiB avail; 211 KiB/s rd, 318 KiB/s wr, 71 op/s 2024-04-16T17:11:11.248 INFO:tasks.workunit.client.0.smithi045.stdout: 1 63712 1.53 MB/sec execute 73 sec latency 59.818 ms 2024-04-16T17:11:12.043 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:11:08 smithi045 ceph-mon[31676]: pgmap v134: 129 pgs: 129 active+clean; 41 MiB data, 522 MiB used, 1.0 TiB / 1.0 TiB avail; 634 KiB/s rd, 560 KiB/s wr, 118 op/s 2024-04-16T17:11:12.044 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:11:10 smithi045 ceph-mon[31676]: pgmap v135: 129 pgs: 129 active+clean; 41 MiB data, 522 MiB used, 1.0 TiB / 1.0 TiB avail; 211 KiB/s rd, 318 KiB/s wr, 71 op/s 2024-04-16T17:11:12.248 INFO:tasks.workunit.client.0.smithi045.stdout: 1 64240 1.57 MB/sec execute 74 sec latency 20.076 ms 2024-04-16T17:11:12.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:11:12 smithi104 ceph-mon[36457]: pgmap v136: 129 pgs: 129 active+clean; 38 MiB data, 518 MiB used, 1.0 TiB / 1.0 TiB avail; 562 KiB/s rd, 450 KiB/s wr, 102 op/s 2024-04-16T17:11:13.055 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:11:12 smithi117 ceph-mon[36730]: pgmap v136: 129 pgs: 129 active+clean; 38 MiB data, 518 MiB used, 1.0 TiB / 1.0 TiB avail; 562 KiB/s rd, 450 KiB/s wr, 102 op/s 2024-04-16T17:11:13.248 INFO:tasks.workunit.client.0.smithi045.stdout: 1 64536 1.55 MB/sec execute 75 sec latency 189.158 ms 2024-04-16T17:11:13.477 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:11:12 smithi045 ceph-mon[31676]: pgmap v136: 129 pgs: 129 active+clean; 38 MiB data, 518 MiB used, 1.0 TiB / 1.0 TiB avail; 562 KiB/s rd, 450 KiB/s wr, 102 op/s 2024-04-16T17:11:14.248 INFO:tasks.workunit.client.0.smithi045.stdout: 1 64536 1.53 MB/sec execute 76 sec latency 1189.274 ms 2024-04-16T17:11:14.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:11:14 smithi104 ceph-mon[36457]: pgmap v137: 129 pgs: 129 active+clean; 38 MiB data, 518 MiB used, 1.0 TiB / 1.0 TiB avail; 406 KiB/s rd, 266 KiB/s wr, 61 op/s 2024-04-16T17:11:14.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:11:14 smithi045 ceph-mon[31676]: pgmap v137: 129 pgs: 129 active+clean; 38 MiB data, 518 MiB used, 1.0 TiB / 1.0 TiB avail; 406 KiB/s rd, 266 KiB/s wr, 61 op/s 2024-04-16T17:11:15.055 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:11:14 smithi117 ceph-mon[36730]: pgmap v137: 129 pgs: 129 active+clean; 38 MiB data, 518 MiB used, 1.0 TiB / 1.0 TiB avail; 406 KiB/s rd, 266 KiB/s wr, 61 op/s 2024-04-16T17:11:15.248 INFO:tasks.workunit.client.0.smithi045.stdout: 1 64536 1.51 MB/sec execute 77 sec latency 2189.360 ms 2024-04-16T17:11:16.249 INFO:tasks.workunit.client.0.smithi045.stdout: 1 64536 1.49 MB/sec execute 78 sec latency 3189.480 ms 2024-04-16T17:11:16.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:11:16 smithi104 ceph-mon[36457]: pgmap v138: 129 pgs: 129 active+clean; 38 MiB data, 518 MiB used, 1.0 TiB / 1.0 TiB avail; 406 KiB/s rd, 266 KiB/s wr, 61 op/s 2024-04-16T17:11:17.055 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:11:16 smithi117 ceph-mon[36730]: pgmap v138: 129 pgs: 129 active+clean; 38 MiB data, 518 MiB used, 1.0 TiB / 1.0 TiB avail; 406 KiB/s rd, 266 KiB/s wr, 61 op/s 2024-04-16T17:11:17.249 INFO:tasks.workunit.client.0.smithi045.stdout: 1 64591 1.47 MB/sec execute 79 sec latency 4037.167 ms 2024-04-16T17:11:17.296 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:11:16 smithi045 ceph-mon[31676]: pgmap v138: 129 pgs: 129 active+clean; 38 MiB data, 518 MiB used, 1.0 TiB / 1.0 TiB avail; 406 KiB/s rd, 266 KiB/s wr, 61 op/s 2024-04-16T17:11:17.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:11:17 smithi104 ceph-mon[36457]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-16T17:11:17.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:11:17 smithi104 ceph-mon[36457]: pgmap v139: 129 pgs: 129 active+clean; 39 MiB data, 521 MiB used, 1.0 TiB / 1.0 TiB avail; 521 KiB/s rd, 369 KiB/s wr, 80 op/s 2024-04-16T17:11:18.054 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:11:17 smithi117 ceph-mon[36730]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-16T17:11:18.055 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:11:17 smithi117 ceph-mon[36730]: pgmap v139: 129 pgs: 129 active+clean; 39 MiB data, 521 MiB used, 1.0 TiB / 1.0 TiB avail; 521 KiB/s rd, 369 KiB/s wr, 80 op/s 2024-04-16T17:11:18.249 INFO:tasks.workunit.client.0.smithi045.stdout: 1 64993 1.48 MB/sec execute 80 sec latency 46.196 ms 2024-04-16T17:11:18.267 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:11:17 smithi045 ceph-mon[31676]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-16T17:11:18.267 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:11:17 smithi045 ceph-mon[31676]: pgmap v139: 129 pgs: 129 active+clean; 39 MiB data, 521 MiB used, 1.0 TiB / 1.0 TiB avail; 521 KiB/s rd, 369 KiB/s wr, 80 op/s 2024-04-16T17:11:19.249 INFO:tasks.workunit.client.0.smithi045.stdout: 1 65056 1.46 MB/sec execute 81 sec latency 722.777 ms 2024-04-16T17:11:20.249 INFO:tasks.workunit.client.0.smithi045.stdout: 1 65056 1.44 MB/sec execute 82 sec latency 1722.895 ms 2024-04-16T17:11:20.677 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:11:20 smithi104 ceph-mon[36457]: pgmap v140: 129 pgs: 129 active+clean; 39 MiB data, 521 MiB used, 1.0 TiB / 1.0 TiB avail; 466 KiB/s rd, 236 KiB/s wr, 50 op/s 2024-04-16T17:11:20.726 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:11:20 smithi045 ceph-mon[31676]: pgmap v140: 129 pgs: 129 active+clean; 39 MiB data, 521 MiB used, 1.0 TiB / 1.0 TiB avail; 466 KiB/s rd, 236 KiB/s wr, 50 op/s 2024-04-16T17:11:20.804 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:11:20 smithi117 ceph-mon[36730]: pgmap v140: 129 pgs: 129 active+clean; 39 MiB data, 521 MiB used, 1.0 TiB / 1.0 TiB avail; 466 KiB/s rd, 236 KiB/s wr, 50 op/s 2024-04-16T17:11:20.877 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-16T17:11:20.878 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-16T17:11:21.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.103+0000 7fe85be00640 1 -- 172.21.15.45:0/3455683755 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe85c102ac0 msgr2=0x7fe85c106b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:21.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.103+0000 7fe85be00640 1 --2- 172.21.15.45:0/3455683755 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe85c102ac0 0x7fe85c106b00 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fe848009900 tx=0x7fe84802bc50 comp rx=0 tx=0).stop 2024-04-16T17:11:21.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.103+0000 7fe85be00640 1 -- 172.21.15.45:0/3455683755 shutdown_connections 2024-04-16T17:11:21.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.103+0000 7fe85be00640 1 --2- 172.21.15.45:0/3455683755 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe85c1071e0 0x7fe85c10edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:21.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.103+0000 7fe85be00640 1 --2- 172.21.15.45:0/3455683755 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe85c102ac0 0x7fe85c106b00 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:21.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.103+0000 7fe85be00640 1 --2- 172.21.15.45:0/3455683755 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe85c102110 0x7fe85c1024f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:21.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.103+0000 7fe85be00640 1 -- 172.21.15.45:0/3455683755 >> 172.21.15.45:0/3455683755 conn(0x7fe85c0fd9d0 msgr2=0x7fe85c0ffdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:11:21.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.103+0000 7fe85be00640 1 -- 172.21.15.45:0/3455683755 shutdown_connections 2024-04-16T17:11:21.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.104+0000 7fe85be00640 1 -- 172.21.15.45:0/3455683755 wait complete. 2024-04-16T17:11:21.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.104+0000 7fe85be00640 1 Processor -- start 2024-04-16T17:11:21.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.105+0000 7fe85be00640 1 -- start start 2024-04-16T17:11:21.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.105+0000 7fe85be00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe85c102110 0x7fe85c1a06e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:21.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.105+0000 7fe85be00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe85c1071e0 0x7fe85c1a0c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:21.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.105+0000 7fe85be00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe85c19a890 0x7fe85c19ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:21.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.105+0000 7fe85be00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fe85c112270 con 0x7fe85c1071e0 2024-04-16T17:11:21.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.105+0000 7fe85be00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fe85c1120f0 con 0x7fe85c19a890 2024-04-16T17:11:21.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.105+0000 7fe85be00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fe85c1123f0 con 0x7fe85c102110 2024-04-16T17:11:21.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.105+0000 7fe85a000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe85c1071e0 0x7fe85c1a0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:21.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.105+0000 7fe85aa00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe85c102110 0x7fe85c1a06e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:21.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.105+0000 7fe85b400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe85c19a890 0x7fe85c19ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:21.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.105+0000 7fe85a000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe85c1071e0 0x7fe85c1a0c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54900/0 (socket says 172.21.15.45:54900) 2024-04-16T17:11:21.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.105+0000 7fe85aa00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe85c102110 0x7fe85c1a06e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:59320/0 (socket says 172.21.15.45:59320) 2024-04-16T17:11:21.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.105+0000 7fe85a000640 1 -- 172.21.15.45:0/3842403851 learned_addr learned my addr 172.21.15.45:0/3842403851 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:11:21.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.106+0000 7fe85b400640 1 -- 172.21.15.45:0/3842403851 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe85c102110 msgr2=0x7fe85c1a06e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:21.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.106+0000 7fe85b400640 1 --2- 172.21.15.45:0/3842403851 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe85c102110 0x7fe85c1a06e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:21.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.106+0000 7fe85b400640 1 -- 172.21.15.45:0/3842403851 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe85c1071e0 msgr2=0x7fe85c1a0c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:21.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.106+0000 7fe85b400640 1 --2- 172.21.15.45:0/3842403851 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe85c1071e0 0x7fe85c1a0c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:21.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.106+0000 7fe85b400640 1 -- 172.21.15.45:0/3842403851 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe85c19b3b0 con 0x7fe85c19a890 2024-04-16T17:11:21.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.106+0000 7fe85b400640 1 --2- 172.21.15.45:0/3842403851 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe85c19a890 0x7fe85c19ad40 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fe854009fd0 tx=0x7fe85400ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:11:21.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.106+0000 7fe853400640 1 -- 172.21.15.45:0/3842403851 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe85400ce20 con 0x7fe85c19a890 2024-04-16T17:11:21.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.106+0000 7fe85be00640 1 -- 172.21.15.45:0/3842403851 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe85c19b6a0 con 0x7fe85c19a890 2024-04-16T17:11:21.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.106+0000 7fe853400640 1 -- 172.21.15.45:0/3842403851 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe854005020 con 0x7fe85c19a890 2024-04-16T17:11:21.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.106+0000 7fe853400640 1 -- 172.21.15.45:0/3842403851 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe854010720 con 0x7fe85c19a890 2024-04-16T17:11:21.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.107+0000 7fe85be00640 1 -- 172.21.15.45:0/3842403851 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe85c1a78f0 con 0x7fe85c19a890 2024-04-16T17:11:21.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.110+0000 7fe864a13740 1 Processor -- start 2024-04-16T17:11:21.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.110+0000 7fe853400640 1 -- 172.21.15.45:0/3842403851 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fe854005540 con 0x7fe85c19a890 2024-04-16T17:11:21.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.110+0000 7fe864a13740 1 -- start start 2024-04-16T17:11:21.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.111+0000 7fe864a13740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564e46ba1390 0x564e46ba1770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:21.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.111+0000 7fe864a13740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564e46ba1cb0 0x564e46baa350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:21.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.111+0000 7fe864a13740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564e46baa890 0x564e46bacc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:21.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.111+0000 7fe864a13740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x564e46b07120 con 0x564e46ba1390 2024-04-16T17:11:21.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.111+0000 7fe864a13740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x564e46aad340 con 0x564e46ba1cb0 2024-04-16T17:11:21.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.111+0000 7fe864a13740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x564e46b030e0 con 0x564e46baa890 2024-04-16T17:11:21.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.111+0000 7fe85a000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564e46ba1390 0x564e46ba1770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:21.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.111+0000 7fe85a000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564e46ba1390 0x564e46ba1770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54904/0 (socket says 172.21.15.45:54904) 2024-04-16T17:11:21.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.111+0000 7fe85a000640 1 -- 172.21.15.45:0/2687436087 learned_addr learned my addr 172.21.15.45:0/2687436087 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:11:21.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.111+0000 7fe85aa00640 1 --2- 172.21.15.45:0/2687436087 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564e46ba1cb0 0x564e46baa350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:21.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.111+0000 7fe85a000640 1 --2- 172.21.15.45:0/2687436087 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564e46baa890 0x564e46bacc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:21.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.112+0000 7fe85aa00640 1 -- 172.21.15.45:0/2687436087 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564e46baa890 msgr2=0x564e46bacc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:11:21.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.112+0000 7fe85aa00640 1 --2- 172.21.15.45:0/2687436087 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564e46baa890 0x564e46bacc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:21.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.112+0000 7fe85aa00640 1 -- 172.21.15.45:0/2687436087 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564e46ba1390 msgr2=0x564e46ba1770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:11:21.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.112+0000 7fe85aa00640 1 --2- 172.21.15.45:0/2687436087 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564e46ba1390 0x564e46ba1770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:21.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.112+0000 7fe85aa00640 1 -- 172.21.15.45:0/2687436087 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564e468e37d0 con 0x564e46ba1cb0 2024-04-16T17:11:21.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.112+0000 7fe85a000640 1 --2- 172.21.15.45:0/2687436087 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564e46ba1390 0x564e46ba1770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:11:21.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.112+0000 7fe85aa00640 1 --2- 172.21.15.45:0/2687436087 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564e46ba1cb0 0x564e46baa350 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fe84c009bd0 tx=0x7fe84c011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2db5bb25b54cfb86 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:11:21.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.112+0000 7fe853400640 1 --2- 172.21.15.45:0/3842403851 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fe8400a3d20 0x7fe8400a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:21.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.112+0000 7fe847e00640 1 -- 172.21.15.45:0/2687436087 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe84c004230 con 0x564e46ba1cb0 2024-04-16T17:11:21.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.112+0000 7fe853400640 1 -- 172.21.15.45:0/3842403851 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fe8540d1220 con 0x7fe85c19a890 2024-04-16T17:11:21.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.112+0000 7fe847e00640 1 -- 172.21.15.45:0/2687436087 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe84c009e40 con 0x564e46ba1cb0 2024-04-16T17:11:21.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.112+0000 7fe847e00640 1 -- 172.21.15.45:0/2687436087 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe84c004d70 con 0x564e46ba1cb0 2024-04-16T17:11:21.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.112+0000 7fe85a000640 1 --2- 172.21.15.45:0/3842403851 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fe8400a3d20 0x7fe8400a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:21.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.113+0000 7fe864a13740 1 -- 172.21.15.45:0/2687436087 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564e46ba1cb0 msgr2=0x564e46baa350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:21.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.113+0000 7fe864a13740 1 --2- 172.21.15.45:0/2687436087 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564e46ba1cb0 0x564e46baa350 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fe84c009bd0 tx=0x7fe84c011810 comp rx=0 tx=0).stop 2024-04-16T17:11:21.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.113+0000 7fe85a000640 1 --2- 172.21.15.45:0/3842403851 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fe8400a3d20 0x7fe8400a61e0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fe848005f70 tx=0x7fe848005bc0 comp rx=0 tx=0).ready entity=mgr.25411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:11:21.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.113+0000 7fe864a13740 1 -- 172.21.15.45:0/2687436087 shutdown_connections 2024-04-16T17:11:21.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.113+0000 7fe864a13740 1 --2- 172.21.15.45:0/2687436087 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564e46baa890 0x564e46bacc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:21.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.113+0000 7fe864a13740 1 --2- 172.21.15.45:0/2687436087 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564e46ba1cb0 0x564e46baa350 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:21.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.113+0000 7fe864a13740 1 --2- 172.21.15.45:0/2687436087 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564e46ba1390 0x564e46ba1770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:21.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.113+0000 7fe864a13740 1 -- 172.21.15.45:0/2687436087 >> 172.21.15.45:0/2687436087 conn(0x564e46b9bac0 msgr2=0x564e46b9bea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:11:21.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.113+0000 7fe864a13740 1 -- 172.21.15.45:0/2687436087 shutdown_connections 2024-04-16T17:11:21.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.113+0000 7fe864a13740 1 -- 172.21.15.45:0/2687436087 wait complete. 2024-04-16T17:11:21.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.114+0000 7fe864a13740 1 -- start start 2024-04-16T17:11:21.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.114+0000 7fe864a13740 1 Processor -- start 2024-04-16T17:11:21.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.114+0000 7fe864a13740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564e46ba1390 0x564e46ba78b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:21.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.114+0000 7fe864a13740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564e46ba1cb0 0x564e46ba7df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:21.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.114+0000 7fe864a13740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564e46baa890 0x564e46ba8330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:21.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.115+0000 7fe864a13740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x564e46bad680 con 0x564e46ba1cb0 2024-04-16T17:11:21.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.115+0000 7fe864a13740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x564e46ac0710 con 0x564e46baa890 2024-04-16T17:11:21.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.115+0000 7fe864a13740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x564e46bad800 con 0x564e46ba1390 2024-04-16T17:11:21.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.115+0000 7fe864a13740 10 client.0 _ll_register_callbacks cb 0x564e46abbaf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:11:21.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.115+0000 7fe85a000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564e46ba1cb0 0x564e46ba7df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:21.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.115+0000 7fe85aa00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564e46ba1390 0x564e46ba78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:21.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.115+0000 7fe85a000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564e46ba1cb0 0x564e46ba7df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54912/0 (socket says 172.21.15.45:54912) 2024-04-16T17:11:21.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.115+0000 7fe85aa00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564e46ba1390 0x564e46ba78b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:59342/0 (socket says 172.21.15.45:59342) 2024-04-16T17:11:21.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.115+0000 7fe85a000640 1 -- 172.21.15.45:0/3951672985 learned_addr learned my addr 172.21.15.45:0/3951672985 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:11:21.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.115+0000 7fe85b400640 1 --2- 172.21.15.45:0/3951672985 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564e46baa890 0x564e46ba8330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:21.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.115+0000 7fe85aa00640 1 -- 172.21.15.45:0/3951672985 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564e46baa890 msgr2=0x564e46ba8330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:11:21.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.115+0000 7fe85aa00640 1 --2- 172.21.15.45:0/3951672985 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564e46baa890 0x564e46ba8330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:21.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.115+0000 7fe85aa00640 1 -- 172.21.15.45:0/3951672985 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564e46ba1cb0 msgr2=0x564e46ba7df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:11:21.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.115+0000 7fe85aa00640 1 --2- 172.21.15.45:0/3951672985 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564e46ba1cb0 0x564e46ba7df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:21.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.115+0000 7fe85aa00640 1 -- 172.21.15.45:0/3951672985 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564e468193f0 con 0x564e46ba1390 2024-04-16T17:11:21.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.115+0000 7fe85a000640 1 --2- 172.21.15.45:0/3951672985 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564e46ba1cb0 0x564e46ba7df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:11:21.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.116+0000 7fe85aa00640 1 --2- 172.21.15.45:0/3951672985 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564e46ba1390 0x564e46ba78b0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fe84c011d10 tx=0x7fe84c003820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1a4232a9770fbe15 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:11:21.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.116+0000 7fe844c00640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-04-16T17:11:21.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.116+0000 7fe844c00640 1 -- 172.21.15.45:0/3951672985 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe84c027020 con 0x564e46ba1390 2024-04-16T17:11:21.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.116+0000 7fe864a13740 1 -- 172.21.15.45:0/3951672985 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x564e46b14b40 con 0x564e46ba1390 2024-04-16T17:11:21.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.116+0000 7fe844c00640 1 -- 172.21.15.45:0/3951672985 <== mon.1 v2:172.21.15.117:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fe84c003c10 con 0x564e46ba1390 2024-04-16T17:11:21.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.116+0000 7fe844c00640 1 -- 172.21.15.45:0/3951672985 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fe84c032050 con 0x564e46ba1390 2024-04-16T17:11:21.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.116+0000 7fe864a13740 10 client.26785 fetch_fsmap learned FSMap version 32 2024-04-16T17:11:21.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.116+0000 7fe864a13740 1 -- 172.21.15.45:0/3951672985 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x564e467d90f0 con 0x564e46ba1390 2024-04-16T17:11:21.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.117+0000 7fe844c00640 1 -- 172.21.15.45:0/3951672985 <== mon.1 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x564e46b14b40 con 0x564e46ba1390 2024-04-16T17:11:21.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.117+0000 7fe844c00640 1 -- 172.21.15.45:0/3951672985 <== mon.1 v2:172.21.15.117:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fe84c018070 con 0x564e46ba1390 2024-04-16T17:11:21.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.117+0000 7fe864a13740 10 client.26785 fetch_fsmap finished waiting for FSMap version 32 2024-04-16T17:11:21.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.117+0000 7fe864a13740 10 client.26785 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:11:21.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.117+0000 7fe864a13740 20 client.26785 populate_metadata read hostname 'smithi045' 2024-04-16T17:11:21.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.117+0000 7fe864a13740 1 --2- 172.21.15.45:0/3951672985 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x564e46bbab30 0x564e46bbcf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:21.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.117+0000 7fe864a13740 4 client.26785 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:11:21.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.117+0000 7fe864a13740 1 -- 172.21.15.45:0/3951672985 --> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564e46b0e9b0 con 0x564e46bbab30 2024-04-16T17:11:21.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.117+0000 7fe85a000640 1 --2- 172.21.15.45:0/3951672985 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x564e46bbab30 0x564e46bbcf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:21.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.118+0000 7fe85a000640 1 --2- 172.21.15.45:0/3951672985 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x564e46bbab30 0x564e46bbcf20 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=31ccf4ad46743909 server_cookie=6506d9a5b55c4a36 in_seq=0 out_seq=0 2024-04-16T17:11:21.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.118+0000 7fe844c00640 10 client.26785 ms_handle_connect on v2:172.21.15.104:6838/3660070592 2024-04-16T17:11:21.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.119+0000 7fe844c00640 1 -- 172.21.15.45:0/3951672985 <== mds.0 v2:172.21.15.104:6838/3660070592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564e46b0e9b0 con 0x564e46bbab30 2024-04-16T17:11:21.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.119+0000 7fe844c00640 10 client.26785 handle_command_reply: tid=0 2024-04-16T17:11:21.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.119+0000 7fe864a13740 1 client.26785 shutdown 2024-04-16T17:11:21.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.119+0000 7fe864a13740 1 -- 172.21.15.45:0/3951672985 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564e46ba1390 msgr2=0x564e46ba78b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:21.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.119+0000 7fe864a13740 1 --2- 172.21.15.45:0/3951672985 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564e46ba1390 0x564e46ba78b0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fe84c011d10 tx=0x7fe84c003820 comp rx=0 tx=0).stop 2024-04-16T17:11:21.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.119+0000 7fe864a13740 1 -- 172.21.15.45:0/3951672985 shutdown_connections 2024-04-16T17:11:21.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.119+0000 7fe864a13740 1 --2- 172.21.15.45:0/3951672985 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564e46baa890 0x564e46ba8330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:21.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.119+0000 7fe864a13740 1 --2- 172.21.15.45:0/3951672985 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564e46ba1cb0 0x564e46ba7df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:21.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.119+0000 7fe864a13740 1 --2- 172.21.15.45:0/3951672985 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x564e46bbab30 0x564e46bbcf20 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:21.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.119+0000 7fe864a13740 1 --2- 172.21.15.45:0/3951672985 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564e46ba1390 0x564e46ba78b0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:21.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.119+0000 7fe864a13740 1 -- 172.21.15.45:0/3951672985 >> 172.21.15.45:0/3951672985 conn(0x564e46b9bac0 msgr2=0x564e46b9d810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:11:21.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.119+0000 7fe844c00640 0 client.26785 ms_handle_reset on v2:172.21.15.104:6838/3660070592 2024-04-16T17:11:21.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.119+0000 7fe864a13740 1 -- 172.21.15.45:0/3951672985 shutdown_connections 2024-04-16T17:11:21.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.119+0000 7fe864a13740 1 -- 172.21.15.45:0/3951672985 wait complete. 2024-04-16T17:11:21.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.119+0000 7fe864a13740 20 client.26785 trim_cache size 0 max 16384 2024-04-16T17:11:21.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.139+0000 7fe864a13740 1 Processor -- start 2024-04-16T17:11:21.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.139+0000 7fe864a13740 1 -- start start 2024-04-16T17:11:21.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.139+0000 7fe864a13740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564e46ba1390 0x564e46ba9fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:21.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.139+0000 7fe864a13740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564e46ba1cb0 0x564e46bb6540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:21.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.139+0000 7fe864a13740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564e46baa890 0x564e46b9a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:21.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.139+0000 7fe864a13740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fe85c1b3da0 con 0x564e46baa890 2024-04-16T17:11:21.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.139+0000 7fe864a13740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fe85c1b15e0 con 0x564e46ba1cb0 2024-04-16T17:11:21.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.139+0000 7fe864a13740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fe85c1af410 con 0x564e46ba1390 2024-04-16T17:11:21.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.139+0000 7fe85b400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564e46baa890 0x564e46b9a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:21.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.139+0000 7fe85aa00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564e46ba1390 0x564e46ba9fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:21.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.139+0000 7fe85b400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564e46baa890 0x564e46b9a9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54922/0 (socket says 172.21.15.45:54922) 2024-04-16T17:11:21.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.139+0000 7fe85b400640 1 -- 172.21.15.45:0/1104474050 learned_addr learned my addr 172.21.15.45:0/1104474050 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:11:21.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.140+0000 7fe85a000640 1 --2- 172.21.15.45:0/1104474050 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564e46ba1cb0 0x564e46bb6540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:21.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.140+0000 7fe85aa00640 1 -- 172.21.15.45:0/1104474050 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564e46ba1cb0 msgr2=0x564e46bb6540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:11:21.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.140+0000 7fe85aa00640 1 --2- 172.21.15.45:0/1104474050 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564e46ba1cb0 0x564e46bb6540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:21.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.140+0000 7fe85aa00640 1 -- 172.21.15.45:0/1104474050 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564e46baa890 msgr2=0x564e46b9a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:11:21.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.140+0000 7fe85aa00640 1 --2- 172.21.15.45:0/1104474050 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564e46baa890 0x564e46b9a9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:21.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.140+0000 7fe85aa00640 1 -- 172.21.15.45:0/1104474050 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564e46bbd290 con 0x564e46ba1390 2024-04-16T17:11:21.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.140+0000 7fe85b400640 1 --2- 172.21.15.45:0/1104474050 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564e46baa890 0x564e46b9a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:11:21.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.140+0000 7fe85aa00640 1 --2- 172.21.15.45:0/1104474050 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564e46ba1390 0x564e46ba9fe0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fe84c009b20 tx=0x7fe84c0048d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a15bb52c923ad711 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:11:21.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.140+0000 7fe847400640 1 -- 172.21.15.45:0/1104474050 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe84c027040 con 0x564e46ba1390 2024-04-16T17:11:21.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.140+0000 7fe847400640 1 -- 172.21.15.45:0/1104474050 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe84c003890 con 0x564e46ba1390 2024-04-16T17:11:21.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.140+0000 7fe847400640 1 -- 172.21.15.45:0/1104474050 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe84c01e040 con 0x564e46ba1390 2024-04-16T17:11:21.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.141+0000 7fe864a13740 1 -- 172.21.15.45:0/1104474050 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564e46ba1390 msgr2=0x564e46ba9fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:21.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.141+0000 7fe864a13740 1 --2- 172.21.15.45:0/1104474050 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564e46ba1390 0x564e46ba9fe0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fe84c009b20 tx=0x7fe84c0048d0 comp rx=0 tx=0).stop 2024-04-16T17:11:21.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.141+0000 7fe864a13740 1 -- 172.21.15.45:0/1104474050 shutdown_connections 2024-04-16T17:11:21.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.141+0000 7fe864a13740 1 --2- 172.21.15.45:0/1104474050 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564e46baa890 0x564e46b9a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:21.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.141+0000 7fe864a13740 1 --2- 172.21.15.45:0/1104474050 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564e46ba1cb0 0x564e46bb6540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:21.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.141+0000 7fe864a13740 1 --2- 172.21.15.45:0/1104474050 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564e46ba1390 0x564e46ba9fe0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:21.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.141+0000 7fe864a13740 1 -- 172.21.15.45:0/1104474050 >> 172.21.15.45:0/1104474050 conn(0x564e46b9bac0 msgr2=0x564e46b9a290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:11:21.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.141+0000 7fe864a13740 1 -- 172.21.15.45:0/1104474050 shutdown_connections 2024-04-16T17:11:21.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.141+0000 7fe864a13740 1 -- 172.21.15.45:0/1104474050 wait complete. 2024-04-16T17:11:21.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.142+0000 7fe864a13740 1 -- start start 2024-04-16T17:11:21.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.142+0000 7fe864a13740 1 Processor -- start 2024-04-16T17:11:21.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.142+0000 7fe864a13740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564e46ba1390 0x564e46bb2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:21.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.142+0000 7fe864a13740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564e46ba1cb0 0x564e46ba4120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:21.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.142+0000 7fe864a13740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564e46baa890 0x564e46ba4660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:21.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.142+0000 7fe864a13740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fe85c1afaa0 con 0x564e46ba1cb0 2024-04-16T17:11:21.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.142+0000 7fe864a13740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fe85c1b0f10 con 0x564e46baa890 2024-04-16T17:11:21.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.142+0000 7fe864a13740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fe85c1b0170 con 0x564e46ba1390 2024-04-16T17:11:21.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.142+0000 7fe864a13740 10 client.0 _ll_register_callbacks cb 0x7fe84c009a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:11:21.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.143+0000 7fe85aa00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564e46ba1390 0x564e46bb2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:21.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.143+0000 7fe85a000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564e46ba1cb0 0x564e46ba4120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:21.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.143+0000 7fe85b400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564e46baa890 0x564e46ba4660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:21.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.143+0000 7fe85aa00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564e46ba1390 0x564e46bb2cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:59358/0 (socket says 172.21.15.45:59358) 2024-04-16T17:11:21.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.143+0000 7fe85aa00640 1 -- 172.21.15.45:0/3544846792 learned_addr learned my addr 172.21.15.45:0/3544846792 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:11:21.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.143+0000 7fe85aa00640 1 -- 172.21.15.45:0/3544846792 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564e46baa890 msgr2=0x564e46ba4660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:11:21.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.143+0000 7fe85aa00640 1 --2- 172.21.15.45:0/3544846792 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564e46baa890 0x564e46ba4660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:21.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.143+0000 7fe85aa00640 1 -- 172.21.15.45:0/3544846792 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564e46ba1cb0 msgr2=0x564e46ba4120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:11:21.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.143+0000 7fe85aa00640 1 --2- 172.21.15.45:0/3544846792 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564e46ba1cb0 0x564e46ba4120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:21.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.143+0000 7fe85aa00640 1 -- 172.21.15.45:0/3544846792 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564e46ba4cd0 con 0x564e46ba1390 2024-04-16T17:11:21.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.143+0000 7fe85aa00640 1 --2- 172.21.15.45:0/3544846792 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564e46ba1390 0x564e46bb2cb0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fe84c009380 tx=0x7fe84c0043d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=98688e9b0f34f1ae server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:11:21.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.143+0000 7fe844c00640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-04-16T17:11:21.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.144+0000 7fe844c00640 1 -- 172.21.15.45:0/3544846792 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe84c044060 con 0x564e46ba1390 2024-04-16T17:11:21.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.144+0000 7fe864a13740 1 -- 172.21.15.45:0/3544846792 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe84c003890 con 0x564e46ba1390 2024-04-16T17:11:21.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.144+0000 7fe844c00640 1 -- 172.21.15.45:0/3544846792 <== mon.1 v2:172.21.15.117:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fe84c003a40 con 0x564e46ba1390 2024-04-16T17:11:21.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.144+0000 7fe844c00640 1 -- 172.21.15.45:0/3544846792 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fe84c04b050 con 0x564e46ba1390 2024-04-16T17:11:21.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.144+0000 7fe864a13740 10 client.26791 fetch_fsmap learned FSMap version 32 2024-04-16T17:11:21.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.144+0000 7fe864a13740 1 -- 172.21.15.45:0/3544846792 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x564e46c20e10 con 0x564e46ba1390 2024-04-16T17:11:21.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.144+0000 7fe844c00640 1 -- 172.21.15.45:0/3544846792 <== mon.1 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe84c003890 con 0x564e46ba1390 2024-04-16T17:11:21.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.144+0000 7fe844c00640 1 -- 172.21.15.45:0/3544846792 <== mon.1 v2:172.21.15.117:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fe84c018070 con 0x564e46ba1390 2024-04-16T17:11:21.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.145+0000 7fe864a13740 10 client.26791 fetch_fsmap finished waiting for FSMap version 32 2024-04-16T17:11:21.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.145+0000 7fe864a13740 10 client.26791 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:11:21.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.145+0000 7fe864a13740 20 client.26791 populate_metadata read hostname 'smithi045' 2024-04-16T17:11:21.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.145+0000 7fe864a13740 1 --2- 172.21.15.45:0/3544846792 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x564e46c21420 0x564e46c23840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:21.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.145+0000 7fe864a13740 4 client.26791 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-16T17:11:21.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.145+0000 7fe864a13740 1 -- 172.21.15.45:0/3544846792 --> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] -- command(tid 0: {"prefix": "scrub status"}) -- 0x564e46a8a420 con 0x564e46c21420 2024-04-16T17:11:21.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.145+0000 7fe85a000640 1 --2- 172.21.15.45:0/3544846792 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x564e46c21420 0x564e46c23840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:21.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.146+0000 7fe844c00640 10 client.26791 ms_handle_connect on v2:172.21.15.104:6838/3660070592 2024-04-16T17:11:21.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.146+0000 7fe85a000640 1 --2- 172.21.15.45:0/3544846792 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x564e46c21420 0x564e46c23840 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f310852836ee02ab server_cookie=e0797ad70cfd3f8a in_seq=0 out_seq=0 2024-04-16T17:11:21.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.146+0000 7fe844c00640 1 -- 172.21.15.45:0/3544846792 <== mds.0 v2:172.21.15.104:6838/3660070592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x564e46a8a420 con 0x564e46c21420 2024-04-16T17:11:21.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.146+0000 7fe844c00640 10 client.26791 handle_command_reply: tid=0 2024-04-16T17:11:21.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.146+0000 7fe864a13740 1 client.26791 shutdown 2024-04-16T17:11:21.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.146+0000 7fe864a13740 1 -- 172.21.15.45:0/3544846792 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564e46ba1390 msgr2=0x564e46bb2cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:21.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.146+0000 7fe864a13740 1 --2- 172.21.15.45:0/3544846792 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564e46ba1390 0x564e46bb2cb0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fe84c009380 tx=0x7fe84c0043d0 comp rx=0 tx=0).stop 2024-04-16T17:11:21.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.146+0000 7fe864a13740 1 -- 172.21.15.45:0/3544846792 shutdown_connections 2024-04-16T17:11:21.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.146+0000 7fe864a13740 1 --2- 172.21.15.45:0/3544846792 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564e46baa890 0x564e46ba4660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:21.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.146+0000 7fe864a13740 1 --2- 172.21.15.45:0/3544846792 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564e46ba1cb0 0x564e46ba4120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:21.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.146+0000 7fe864a13740 1 --2- 172.21.15.45:0/3544846792 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x564e46c21420 0x564e46c23840 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:21.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.146+0000 7fe864a13740 1 --2- 172.21.15.45:0/3544846792 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564e46ba1390 0x564e46bb2cb0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:21.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.146+0000 7fe864a13740 1 -- 172.21.15.45:0/3544846792 >> 172.21.15.45:0/3544846792 conn(0x564e46b9bac0 msgr2=0x564e46b9ab20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:11:21.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.146+0000 7fe844c00640 0 client.26791 ms_handle_reset on v2:172.21.15.104:6838/3660070592 2024-04-16T17:11:21.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.146+0000 7fe864a13740 1 -- 172.21.15.45:0/3544846792 shutdown_connections 2024-04-16T17:11:21.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.147+0000 7fe864a13740 1 -- 172.21.15.45:0/3544846792 wait complete. 2024-04-16T17:11:21.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.147+0000 7fe864a13740 20 client.26791 trim_cache size 0 max 16384 2024-04-16T17:11:21.152 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-16T17:11:21.152 INFO:teuthology.orchestra.run.smithi045.stdout: "status": "no active scrubs running", 2024-04-16T17:11:21.152 INFO:teuthology.orchestra.run.smithi045.stdout: "scrubs": {} 2024-04-16T17:11:21.152 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-16T17:11:21.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.147+0000 7fe85be00640 1 -- 172.21.15.45:0/3842403851 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fe8400a3d20 msgr2=0x7fe8400a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:21.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.147+0000 7fe85be00640 1 --2- 172.21.15.45:0/3842403851 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fe8400a3d20 0x7fe8400a61e0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fe848005f70 tx=0x7fe848005bc0 comp rx=0 tx=0).stop 2024-04-16T17:11:21.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.147+0000 7fe85be00640 1 -- 172.21.15.45:0/3842403851 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe85c19a890 msgr2=0x7fe85c19ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:21.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.147+0000 7fe85be00640 1 --2- 172.21.15.45:0/3842403851 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe85c19a890 0x7fe85c19ad40 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fe854009fd0 tx=0x7fe85400ef10 comp rx=0 tx=0).stop 2024-04-16T17:11:21.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.147+0000 7fe85be00640 1 -- 172.21.15.45:0/3842403851 shutdown_connections 2024-04-16T17:11:21.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.147+0000 7fe85be00640 1 --2- 172.21.15.45:0/3842403851 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fe8400a3d20 0x7fe8400a61e0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:21.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.147+0000 7fe85be00640 1 --2- 172.21.15.45:0/3842403851 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe85c19a890 0x7fe85c19ad40 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:21.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.147+0000 7fe85be00640 1 --2- 172.21.15.45:0/3842403851 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe85c1071e0 0x7fe85c1a0c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:21.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.147+0000 7fe85be00640 1 --2- 172.21.15.45:0/3842403851 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe85c102110 0x7fe85c1a06e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:21.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.147+0000 7fe85be00640 1 -- 172.21.15.45:0/3842403851 >> 172.21.15.45:0/3842403851 conn(0x7fe85c0fd9d0 msgr2=0x7fe85c0fecd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:11:21.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.148+0000 7fe85be00640 1 -- 172.21.15.45:0/3842403851 shutdown_connections 2024-04-16T17:11:21.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.148+0000 7fe85be00640 1 -- 172.21.15.45:0/3842403851 wait complete. 2024-04-16T17:11:21.165 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-16T17:11:21.165 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-16T17:11:21.249 INFO:tasks.workunit.client.0.smithi045.stdout: 1 65056 1.42 MB/sec execute 83 sec latency 2722.969 ms 2024-04-16T17:11:21.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.378+0000 7f653fe00640 1 -- 172.21.15.45:0/3923854348 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6540069db0 msgr2=0x7f6540106a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:21.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.378+0000 7f653fe00640 1 --2- 172.21.15.45:0/3923854348 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6540069db0 0x7f6540106a70 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f652c009900 tx=0x7f652c02bc40 comp rx=0 tx=0).stop 2024-04-16T17:11:21.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.379+0000 7f653fe00640 1 -- 172.21.15.45:0/3923854348 shutdown_connections 2024-04-16T17:11:21.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.379+0000 7f653fe00640 1 --2- 172.21.15.45:0/3923854348 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f65401071a0 0x7f654010ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:21.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.379+0000 7f653fe00640 1 --2- 172.21.15.45:0/3923854348 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6540069db0 0x7f6540106a70 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:21.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.379+0000 7f653fe00640 1 --2- 172.21.15.45:0/3923854348 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6540069400 0x7f65400697e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:21.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.379+0000 7f653fe00640 1 -- 172.21.15.45:0/3923854348 >> 172.21.15.45:0/3923854348 conn(0x7f6540076840 msgr2=0x7f6540076c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:11:21.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.379+0000 7f653fe00640 1 -- 172.21.15.45:0/3923854348 shutdown_connections 2024-04-16T17:11:21.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.379+0000 7f653fe00640 1 -- 172.21.15.45:0/3923854348 wait complete. 2024-04-16T17:11:21.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.380+0000 7f653fe00640 1 Processor -- start 2024-04-16T17:11:21.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.380+0000 7f653fe00640 1 -- start start 2024-04-16T17:11:21.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.380+0000 7f653fe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6540069400 0x7f65401a0610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:21.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.380+0000 7f653fe00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f65401071a0 0x7f65401a0b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:21.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.380+0000 7f653fe00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f654019a780 0x7f654019ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:21.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.380+0000 7f653fe00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f65401121c0 con 0x7f6540069400 2024-04-16T17:11:21.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.380+0000 7f653fe00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f6540112040 con 0x7f654019a780 2024-04-16T17:11:21.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.380+0000 7f653fe00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f6540112340 con 0x7f65401071a0 2024-04-16T17:11:21.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.381+0000 7f653ea00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6540069400 0x7f65401a0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:21.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.381+0000 7f653ea00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6540069400 0x7f65401a0610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54956/0 (socket says 172.21.15.45:54956) 2024-04-16T17:11:21.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.381+0000 7f653e000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f65401071a0 0x7f65401a0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:21.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.381+0000 7f653ea00640 1 -- 172.21.15.45:0/3816587143 learned_addr learned my addr 172.21.15.45:0/3816587143 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:11:21.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.381+0000 7f653f400640 1 --2- 172.21.15.45:0/3816587143 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f654019a780 0x7f654019ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:21.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.381+0000 7f653ea00640 1 -- 172.21.15.45:0/3816587143 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f65401071a0 msgr2=0x7f65401a0b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:21.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.381+0000 7f653ea00640 1 --2- 172.21.15.45:0/3816587143 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f65401071a0 0x7f65401a0b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:21.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.381+0000 7f653ea00640 1 -- 172.21.15.45:0/3816587143 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f654019a780 msgr2=0x7f654019ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:21.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.381+0000 7f653ea00640 1 --2- 172.21.15.45:0/3816587143 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f654019a780 0x7f654019ac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:21.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.381+0000 7f653ea00640 1 -- 172.21.15.45:0/3816587143 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f654019b310 con 0x7f6540069400 2024-04-16T17:11:21.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.381+0000 7f653ea00640 1 --2- 172.21.15.45:0/3816587143 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6540069400 0x7f65401a0610 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f653000e9f0 tx=0x7f653000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:11:21.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.382+0000 7f6544a00640 1 -- 172.21.15.45:0/3816587143 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f653000cd80 con 0x7f6540069400 2024-04-16T17:11:21.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.382+0000 7f6544a00640 1 -- 172.21.15.45:0/3816587143 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6530010910 con 0x7f6540069400 2024-04-16T17:11:21.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.382+0000 7f6544a00640 1 -- 172.21.15.45:0/3816587143 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6530018990 con 0x7f6540069400 2024-04-16T17:11:21.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.382+0000 7f653fe00640 1 -- 172.21.15.45:0/3816587143 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f654019b600 con 0x7f6540069400 2024-04-16T17:11:21.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.382+0000 7f653fe00640 1 -- 172.21.15.45:0/3816587143 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65401a7760 con 0x7f6540069400 2024-04-16T17:11:21.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.383+0000 7f653fe00640 1 -- 172.21.15.45:0/3816587143 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6540103550 con 0x7f6540069400 2024-04-16T17:11:21.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.385+0000 7f6544a00640 1 -- 172.21.15.45:0/3816587143 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f6530010ac0 con 0x7f6540069400 2024-04-16T17:11:21.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.386+0000 7f6544a00640 1 --2- 172.21.15.45:0/3816587143 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f65240a3da0 0x7f65240a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:21.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.387+0000 7f6544a00640 1 -- 172.21.15.45:0/3816587143 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f65300d0e30 con 0x7f6540069400 2024-04-16T17:11:21.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.387+0000 7f653e000640 1 --2- 172.21.15.45:0/3816587143 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f65240a3da0 0x7f65240a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:21.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.388+0000 7f653e000640 1 --2- 172.21.15.45:0/3816587143 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f65240a3da0 0x7f65240a6260 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f652c000c00 tx=0x7f652c005fd0 comp rx=0 tx=0).ready entity=mgr.25411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:11:21.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.391+0000 7f6544a00640 1 -- 172.21.15.45:0/3816587143 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f654019b600 con 0x7f6540069400 2024-04-16T17:11:21.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.640+0000 7f653fe00640 1 -- 172.21.15.45:0/3816587143 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6540068d40 con 0x7f6540069400 2024-04-16T17:11:21.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.642+0000 7f6544a00640 1 -- 172.21.15.45:0/3816587143 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11892 (secure 0 0 0) 0x7f653009cad0 con 0x7f6540069400 2024-04-16T17:11:21.642 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-16T17:11:21.642 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/1748842443","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":1748842443},{"type":"v1","addr":"172.21.15.45:6835","nonce":1748842443}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6839/1308627051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":1308627051},{"type":"v1","addr":"172.21.15.45:6839","nonce":1308627051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/1768242489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1768242489},{"type":"v1","addr":"172.21.15.117:6833","nonce":1768242489}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6833/3558902585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":3558902585},{"type":"v1","addr":"172.21.15.104:6833","nonce":3558902585}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24478,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/1308142903","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":1308142903},{"type":"v1","addr":"172.21.15.45:6837","nonce":1308142903}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24491,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6835/3430679937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":3430679937},{"type":"v1","addr":"172.21.15.104:6835","nonce":3430679937}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":32,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-16T17:03:24.663611+0000","modified":"2024-04-16T17:11:15.716160+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24581,"mds_1":24569,"mds_2":24532},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24499":{"gid":24499,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/2224123841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2224123841},{"type":"v1","addr":"172.21.15.117:6835","nonce":2224123841}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6837/2471356815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":2471356815},{"type":"v1","addr":"172.21.15.117:6837","nonce":2471356815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24524":{"gid":24524,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6837/2647756509","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":2647756509},{"type":"v1","addr":"172.21.15.104:6837","nonce":2647756509}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24532":{"gid":24532,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":13,"addr":"172.21.15.117:6839/2378675949","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":2378675949},{"type":"v1","addr":"172.21.15.117:6839","nonce":2378675949}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24569":{"gid":24569,"name":"a","rank":1,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.45:6841/3755310822","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6840","nonce":3755310822},{"type":"v1","addr":"172.21.15.45:6841","nonce":3755310822}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24581":{"gid":24581,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.104:6839/3660070592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6838","nonce":3660070592},{"type":"v1","addr":"172.21.15.104:6839","nonce":3660070592}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24581,"qdb_cluster":[24569,24532,24581]},"id":1}]} 2024-04-16T17:11:21.642 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 32 2024-04-16T17:11:21.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.645+0000 7f653fe00640 1 -- 172.21.15.45:0/3816587143 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f65240a3da0 msgr2=0x7f65240a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:21.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.645+0000 7f653fe00640 1 --2- 172.21.15.45:0/3816587143 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f65240a3da0 0x7f65240a6260 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f652c000c00 tx=0x7f652c005fd0 comp rx=0 tx=0).stop 2024-04-16T17:11:21.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.645+0000 7f653fe00640 1 -- 172.21.15.45:0/3816587143 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6540069400 msgr2=0x7f65401a0610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:21.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.645+0000 7f653fe00640 1 --2- 172.21.15.45:0/3816587143 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6540069400 0x7f65401a0610 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f653000e9f0 tx=0x7f653000eeb0 comp rx=0 tx=0).stop 2024-04-16T17:11:21.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.645+0000 7f653fe00640 1 -- 172.21.15.45:0/3816587143 shutdown_connections 2024-04-16T17:11:21.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.645+0000 7f653fe00640 1 --2- 172.21.15.45:0/3816587143 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f65240a3da0 0x7f65240a6260 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:21.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.645+0000 7f653fe00640 1 --2- 172.21.15.45:0/3816587143 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f65401071a0 0x7f65401a0b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:21.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.645+0000 7f653fe00640 1 --2- 172.21.15.45:0/3816587143 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f654019a780 0x7f654019ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:21.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.645+0000 7f653fe00640 1 --2- 172.21.15.45:0/3816587143 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6540069400 0x7f65401a0610 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:21.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.645+0000 7f653fe00640 1 -- 172.21.15.45:0/3816587143 >> 172.21.15.45:0/3816587143 conn(0x7f6540076840 msgr2=0x7f654010d230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:11:21.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.645+0000 7f653fe00640 1 -- 172.21.15.45:0/3816587143 shutdown_connections 2024-04-16T17:11:21.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.645+0000 7f653fe00640 1 -- 172.21.15.45:0/3816587143 wait complete. 2024-04-16T17:11:21.664 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-16T17:11:21.824 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:21.823+0000 7fa357800640 1 -- 172.21.15.45:0/1011620745 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa350104450 msgr2=0x7fa350104830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:21.825 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:21.823+0000 7fa357800640 1 --2- 172.21.15.45:0/1011620745 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa350104450 0x7fa350104830 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fa348009ca0 tx=0x7fa34802c100 comp rx=0 tx=0).stop 2024-04-16T17:11:21.825 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:21.823+0000 7fa357800640 1 -- 172.21.15.45:0/1011620745 shutdown_connections 2024-04-16T17:11:21.825 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:21.823+0000 7fa357800640 1 --2- 172.21.15.45:0/1011620745 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa350104e00 0x7fa350108e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:21.825 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:21.823+0000 7fa357800640 1 --2- 172.21.15.45:0/1011620745 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa350109520 0x7fa3501110f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:21.825 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:21.823+0000 7fa357800640 1 --2- 172.21.15.45:0/1011620745 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa350104450 0x7fa350104830 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:21.825 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:21.823+0000 7fa357800640 1 -- 172.21.15.45:0/1011620745 >> 172.21.15.45:0/1011620745 conn(0x7fa3500ffcf0 msgr2=0x7fa350102110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:11:21.825 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:21.824+0000 7fa357800640 1 -- 172.21.15.45:0/1011620745 shutdown_connections 2024-04-16T17:11:21.825 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:21.824+0000 7fa357800640 1 -- 172.21.15.45:0/1011620745 wait complete. 2024-04-16T17:11:21.825 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:21.825+0000 7fa357800640 1 Processor -- start 2024-04-16T17:11:21.826 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:21.825+0000 7fa357800640 1 -- start start 2024-04-16T17:11:21.827 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:21.826+0000 7fa357800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa350104450 0x7fa3501a06d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:21.827 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:21.826+0000 7fa357800640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa350109520 0x7fa3501a0c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:21.827 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:21.826+0000 7fa357800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa35019a890 0x7fa35019ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:21.827 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:21.826+0000 7fa357800640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fa3501145d0 con 0x7fa350104450 2024-04-16T17:11:21.827 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:21.826+0000 7fa357800640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fa350114450 con 0x7fa350109520 2024-04-16T17:11:21.827 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:21.826+0000 7fa357800640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa350114750 con 0x7fa35019a890 2024-04-16T17:11:21.827 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:21.826+0000 7fa354e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa350104450 0x7fa3501a06d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:21.827 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:21.826+0000 7fa354e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa350104450 0x7fa3501a06d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54986/0 (socket says 172.21.15.45:54986) 2024-04-16T17:11:21.827 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:21.826+0000 7fa354e00640 1 -- 172.21.15.45:0/3502169411 learned_addr learned my addr 172.21.15.45:0/3502169411 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:11:21.827 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:21.826+0000 7fa34fe00640 1 --2- 172.21.15.45:0/3502169411 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa350109520 0x7fa3501a0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:21.828 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:21.826+0000 7fa355800640 1 --2- 172.21.15.45:0/3502169411 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa35019a890 0x7fa35019ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:21.828 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:21.826+0000 7fa354e00640 1 -- 172.21.15.45:0/3502169411 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa35019a890 msgr2=0x7fa35019ad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:21.828 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:21.826+0000 7fa354e00640 1 --2- 172.21.15.45:0/3502169411 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa35019a890 0x7fa35019ad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:21.828 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:21.826+0000 7fa354e00640 1 -- 172.21.15.45:0/3502169411 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa350109520 msgr2=0x7fa3501a0c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:21.828 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:21.826+0000 7fa354e00640 1 --2- 172.21.15.45:0/3502169411 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa350109520 0x7fa3501a0c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:21.828 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:21.826+0000 7fa354e00640 1 -- 172.21.15.45:0/3502169411 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa35019b420 con 0x7fa350104450 2024-04-16T17:11:21.829 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:21.827+0000 7fa354e00640 1 --2- 172.21.15.45:0/3502169411 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa350104450 0x7fa3501a06d0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fa348005e30 tx=0x7fa348035d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:11:21.829 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:21.827+0000 7fa34d600640 1 -- 172.21.15.45:0/3502169411 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa348039040 con 0x7fa350104450 2024-04-16T17:11:21.829 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:21.827+0000 7fa34d600640 1 -- 172.21.15.45:0/3502169411 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa348044070 con 0x7fa350104450 2024-04-16T17:11:21.829 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:21.827+0000 7fa357800640 1 -- 172.21.15.45:0/3502169411 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa35019b6b0 con 0x7fa350104450 2024-04-16T17:11:21.829 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:21.827+0000 7fa34d600640 1 -- 172.21.15.45:0/3502169411 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3480043c0 con 0x7fa350104450 2024-04-16T17:11:21.829 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:21.827+0000 7fa357800640 1 -- 172.21.15.45:0/3502169411 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3501a7810 con 0x7fa350104450 2024-04-16T17:11:21.832 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:21.831+0000 7fa35a5dd740 1 Processor -- start 2024-04-16T17:11:21.832 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:21.831+0000 7fa35a5dd740 1 -- start start 2024-04-16T17:11:21.832 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:21.831+0000 7fa34d600640 1 -- 172.21.15.45:0/3502169411 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fa348004560 con 0x7fa350104450 2024-04-16T17:11:21.833 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:21.832+0000 7fa35a5dd740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55699e997610 0x55699e996360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:21.833 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:21.832+0000 7fa35a5dd740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55699e9968a0 0x55699e996d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:21.833 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:21.832+0000 7fa35a5dd740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55699e99fe20 0x55699e9a2210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:21.833 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:21.832+0000 7fa35a5dd740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55699e8fb780 con 0x55699e9968a0 2024-04-16T17:11:21.833 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:21.832+0000 7fa35a5dd740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55699e8a1f40 con 0x55699e997610 2024-04-16T17:11:21.833 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:21.832+0000 7fa35a5dd740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55699e8f7740 con 0x55699e99fe20 2024-04-16T17:11:21.833 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:21.832+0000 7fa354e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55699e9968a0 0x55699e996d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:21.833 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:21.832+0000 7fa355800640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55699e997610 0x55699e996360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:21.833 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:21.832+0000 7fa354e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55699e9968a0 0x55699e996d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54996/0 (socket says 172.21.15.45:54996) 2024-04-16T17:11:21.833 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:21.832+0000 7fa354e00640 1 -- 172.21.15.45:0/2152147337 learned_addr learned my addr 172.21.15.45:0/2152147337 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:11:21.833 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:21.832+0000 7fa34fe00640 1 --2- 172.21.15.45:0/2152147337 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55699e99fe20 0x55699e9a2210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:21.835 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:21.833+0000 7fa355800640 1 -- 172.21.15.45:0/2152147337 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55699e99fe20 msgr2=0x55699e9a2210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:11:21.835 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:21.833+0000 7fa355800640 1 --2- 172.21.15.45:0/2152147337 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55699e99fe20 0x55699e9a2210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:21.835 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:21.833+0000 7fa355800640 1 -- 172.21.15.45:0/2152147337 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55699e9968a0 msgr2=0x55699e996d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:11:21.835 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:21.833+0000 7fa355800640 1 --2- 172.21.15.45:0/2152147337 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55699e9968a0 0x55699e996d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:21.835 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:21.833+0000 7fa355800640 1 -- 172.21.15.45:0/2152147337 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55699e60e210 con 0x55699e997610 2024-04-16T17:11:21.835 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:21.833+0000 7fa355800640 1 --2- 172.21.15.45:0/2152147337 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55699e997610 0x55699e996360 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fa340009b80 tx=0x7fa340011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a2502760ef547713 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:11:21.835 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:21.834+0000 7fa34d600640 1 --2- 172.21.15.45:0/3502169411 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fa3340a3dc0 0x7fa3340a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:21.835 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:21.834+0000 7fa346000640 1 -- 172.21.15.45:0/2152147337 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa340004050 con 0x55699e997610 2024-04-16T17:11:21.835 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:21.834+0000 7fa346000640 1 -- 172.21.15.45:0/2152147337 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa340005500 con 0x55699e997610 2024-04-16T17:11:21.835 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:21.834+0000 7fa346000640 1 -- 172.21.15.45:0/2152147337 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa340004d90 con 0x55699e997610 2024-04-16T17:11:21.835 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:21.834+0000 7fa34d600640 1 -- 172.21.15.45:0/3502169411 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fa3480f2f80 con 0x7fa350104450 2024-04-16T17:11:21.835 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:21.834+0000 7fa354e00640 1 --2- 172.21.15.45:0/3502169411 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fa3340a3dc0 0x7fa3340a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:21.836 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:21.835+0000 7fa35a5dd740 1 -- 172.21.15.45:0/2152147337 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55699e997610 msgr2=0x55699e996360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:21.836 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:21.835+0000 7fa35a5dd740 1 --2- 172.21.15.45:0/2152147337 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55699e997610 0x55699e996360 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fa340009b80 tx=0x7fa340011810 comp rx=0 tx=0).stop 2024-04-16T17:11:21.836 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:21.835+0000 7fa35a5dd740 1 -- 172.21.15.45:0/2152147337 shutdown_connections 2024-04-16T17:11:21.836 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:21.835+0000 7fa35a5dd740 1 --2- 172.21.15.45:0/2152147337 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55699e99fe20 0x55699e9a2210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:21.836 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:21.835+0000 7fa35a5dd740 1 --2- 172.21.15.45:0/2152147337 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55699e9968a0 0x55699e996d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:21.836 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:21.835+0000 7fa35a5dd740 1 --2- 172.21.15.45:0/2152147337 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55699e997610 0x55699e996360 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:21.836 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:21.835+0000 7fa35a5dd740 1 -- 172.21.15.45:0/2152147337 >> 172.21.15.45:0/2152147337 conn(0x55699e9315d0 msgr2=0x55699e991590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:11:21.836 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:21.835+0000 7fa35a5dd740 1 -- 172.21.15.45:0/2152147337 shutdown_connections 2024-04-16T17:11:21.837 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:21.835+0000 7fa354e00640 1 --2- 172.21.15.45:0/3502169411 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fa3340a3dc0 0x7fa3340a6280 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fa3480bb0e0 tx=0x7fa3480f6a30 comp rx=0 tx=0).ready entity=mgr.25411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:11:21.837 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:21.835+0000 7fa35a5dd740 1 -- 172.21.15.45:0/2152147337 wait complete. 2024-04-16T17:11:21.838 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:21.836+0000 7fa35a5dd740 1 -- start start 2024-04-16T17:11:21.838 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:21.836+0000 7fa35a5dd740 1 Processor -- start 2024-04-16T17:11:21.838 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:21.837+0000 7fa35a5dd740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55699e9968a0 0x55699e99cc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:21.839 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:21.837+0000 7fa35a5dd740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55699e997610 0x55699e99d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:21.839 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:21.837+0000 7fa35a5dd740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55699e99fe20 0x55699e99d700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:21.839 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:21.837+0000 7fa35a5dd740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55699e9a2b50 con 0x55699e99fe20 2024-04-16T17:11:21.839 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:21.837+0000 7fa35a5dd740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55699e8b5390 con 0x55699e997610 2024-04-16T17:11:21.839 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:21.837+0000 7fa35a5dd740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55699e9a2cd0 con 0x55699e9968a0 2024-04-16T17:11:21.839 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:21.837+0000 7fa35a5dd740 10 client.0 _ll_register_callbacks cb 0x55699e89ef60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:11:21.839 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:21.837+0000 7fa355800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55699e9968a0 0x55699e99cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:21.839 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:21.837+0000 7fa355800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55699e9968a0 0x55699e99cc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:59420/0 (socket says 172.21.15.45:59420) 2024-04-16T17:11:21.839 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:21.837+0000 7fa355800640 1 -- 172.21.15.45:0/1672257192 learned_addr learned my addr 172.21.15.45:0/1672257192 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:11:21.839 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:21.837+0000 7fa34fe00640 1 --2- 172.21.15.45:0/1672257192 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55699e997610 0x55699e99d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:21.839 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:21.837+0000 7fa355800640 1 --2- 172.21.15.45:0/1672257192 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55699e99fe20 0x55699e99d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:21.839 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:21.837+0000 7fa34fe00640 1 -- 172.21.15.45:0/1672257192 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55699e9968a0 msgr2=0x55699e99cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:11:21.839 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:21.837+0000 7fa34fe00640 1 --2- 172.21.15.45:0/1672257192 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55699e9968a0 0x55699e99cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:21.839 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:21.838+0000 7fa34fe00640 1 -- 172.21.15.45:0/1672257192 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55699e99fe20 msgr2=0x55699e99d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:11:21.839 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:21.838+0000 7fa34fe00640 1 --2- 172.21.15.45:0/1672257192 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55699e99fe20 0x55699e99d700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:21.840 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:21.838+0000 7fa34fe00640 1 -- 172.21.15.45:0/1672257192 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55699e6da620 con 0x55699e997610 2024-04-16T17:11:21.840 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:21.838+0000 7fa34fe00640 1 --2- 172.21.15.45:0/1672257192 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55699e997610 0x55699e99d1c0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x55699e997340 tx=0x7fa338012b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b679474fce8f557d server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:11:21.840 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:21.838+0000 7fa33ea00640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-04-16T17:11:21.841 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:21.838+0000 7fa33ea00640 1 -- 172.21.15.45:0/1672257192 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa338004020 con 0x55699e997610 2024-04-16T17:11:21.841 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:21.838+0000 7fa35a5dd740 1 -- 172.21.15.45:0/1672257192 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa340005500 con 0x55699e997610 2024-04-16T17:11:21.841 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:21.838+0000 7fa33ea00640 1 -- 172.21.15.45:0/1672257192 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fa3380041c0 con 0x55699e997610 2024-04-16T17:11:21.841 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:21.839+0000 7fa33ea00640 1 -- 172.21.15.45:0/1672257192 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fa338025050 con 0x55699e997610 2024-04-16T17:11:21.841 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:21.839+0000 7fa35a5dd740 10 client.26870 fetch_fsmap learned FSMap version 32 2024-04-16T17:11:21.841 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:21.839+0000 7fa35a5dd740 1 -- 172.21.15.45:0/1672257192 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55699e9aed60 con 0x55699e997610 2024-04-16T17:11:21.841 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:21.839+0000 7fa33ea00640 1 -- 172.21.15.45:0/1672257192 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa340005500 con 0x55699e997610 2024-04-16T17:11:21.841 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:21.839+0000 7fa33ea00640 1 -- 172.21.15.45:0/1672257192 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fa338003a60 con 0x55699e997610 2024-04-16T17:11:21.841 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:21.839+0000 7fa35a5dd740 10 client.26870 fetch_fsmap finished waiting for FSMap version 32 2024-04-16T17:11:21.842 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:21.839+0000 7fa35a5dd740 10 client.26870 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T17:11:21.842 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:21.840+0000 7fa35a5dd740 20 client.26870 populate_metadata read hostname 'smithi045' 2024-04-16T17:11:21.842 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:21.840+0000 7fa35a5dd740 1 --2- 172.21.15.45:0/1672257192 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55699e9b0220 0x55699e9b2610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:21.842 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:21.840+0000 7fa35a5dd740 4 client.26870 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:11:21.842 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:21.840+0000 7fa35a5dd740 1 -- 172.21.15.45:0/1672257192 --> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55699e9091a0 con 0x55699e9b0220 2024-04-16T17:11:21.842 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:21.840+0000 7fa355800640 1 --2- 172.21.15.45:0/1672257192 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55699e9b0220 0x55699e9b2610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:21.842 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:21.840+0000 7fa355800640 1 --2- 172.21.15.45:0/1672257192 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55699e9b0220 0x55699e9b2610 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7dd7d5e9d2de289 server_cookie=d2c203d6340f0688 in_seq=0 out_seq=0 2024-04-16T17:11:21.842 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:21.840+0000 7fa33ea00640 10 client.26870 ms_handle_connect on v2:172.21.15.104:6838/3660070592 2024-04-16T17:11:21.843 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:21.841+0000 7fa33ea00640 1 -- 172.21.15.45:0/1672257192 <== mds.0 v2:172.21.15.104:6838/3660070592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55699e9091a0 con 0x55699e9b0220 2024-04-16T17:11:21.843 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:21.841+0000 7fa33ea00640 10 client.26870 handle_command_reply: tid=0 2024-04-16T17:11:21.843 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:21.841+0000 7fa35a5dd740 1 client.26870 shutdown 2024-04-16T17:11:21.843 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:21.841+0000 7fa35a5dd740 1 -- 172.21.15.45:0/1672257192 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55699e997610 msgr2=0x55699e99d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:21.843 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:21.841+0000 7fa35a5dd740 1 --2- 172.21.15.45:0/1672257192 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55699e997610 0x55699e99d1c0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x55699e997340 tx=0x7fa338012b60 comp rx=0 tx=0).stop 2024-04-16T17:11:21.843 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:21.841+0000 7fa35a5dd740 1 -- 172.21.15.45:0/1672257192 shutdown_connections 2024-04-16T17:11:21.843 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:21.841+0000 7fa35a5dd740 1 --2- 172.21.15.45:0/1672257192 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55699e99fe20 0x55699e99d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:21.843 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:21.841+0000 7fa35a5dd740 1 --2- 172.21.15.45:0/1672257192 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55699e997610 0x55699e99d1c0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:21.843 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:21.841+0000 7fa35a5dd740 1 --2- 172.21.15.45:0/1672257192 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55699e9b0220 0x55699e9b2610 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:21.844 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:21.841+0000 7fa35a5dd740 1 --2- 172.21.15.45:0/1672257192 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55699e9968a0 0x55699e99cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:21.844 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:21.842+0000 7fa35a5dd740 1 -- 172.21.15.45:0/1672257192 >> 172.21.15.45:0/1672257192 conn(0x55699e9315d0 msgr2=0x55699e9932e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:11:21.844 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:21.842+0000 7fa33ea00640 0 client.26870 ms_handle_reset on v2:172.21.15.104:6838/3660070592 2024-04-16T17:11:21.844 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:21.842+0000 7fa35a5dd740 1 -- 172.21.15.45:0/1672257192 shutdown_connections 2024-04-16T17:11:21.844 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:21.842+0000 7fa35a5dd740 1 -- 172.21.15.45:0/1672257192 wait complete. 2024-04-16T17:11:21.844 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:21.842+0000 7fa35a5dd740 20 client.26870 trim_cache size 0 max 16384 2024-04-16T17:11:21.875 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:21.875+0000 7fa35a5dd740 1 Processor -- start 2024-04-16T17:11:21.876 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:21.875+0000 7fa35a5dd740 1 -- start start 2024-04-16T17:11:21.877 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:21.875+0000 7fa35a5dd740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55699e9968a0 0x55699e9abaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:21.877 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:21.875+0000 7fa35a5dd740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55699e997610 0x55699e99bce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:21.877 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:21.875+0000 7fa35a5dd740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55699e99fe20 0x55699e9b7ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:21.877 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:21.875+0000 7fa35a5dd740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fa3501b3ea0 con 0x55699e997610 2024-04-16T17:11:21.877 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:21.875+0000 7fa35a5dd740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fa3501b16e0 con 0x55699e99fe20 2024-04-16T17:11:21.877 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:21.875+0000 7fa35a5dd740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa3501af510 con 0x55699e9968a0 2024-04-16T17:11:21.877 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:21.876+0000 7fa34fe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55699e997610 0x55699e99bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:21.877 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:21.876+0000 7fa34fe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55699e997610 0x55699e99bce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55018/0 (socket says 172.21.15.45:55018) 2024-04-16T17:11:21.877 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:21.876+0000 7fa34fe00640 1 -- 172.21.15.45:0/2334970947 learned_addr learned my addr 172.21.15.45:0/2334970947 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:11:21.877 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:21.876+0000 7fa355800640 1 --2- 172.21.15.45:0/2334970947 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55699e9968a0 0x55699e9abaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:21.877 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:21.876+0000 7fa355800640 1 --2- 172.21.15.45:0/2334970947 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55699e99fe20 0x55699e9b7ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:21.877 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:21.876+0000 7fa34fe00640 1 -- 172.21.15.45:0/2334970947 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55699e9968a0 msgr2=0x55699e9abaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:11:21.877 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:21.876+0000 7fa34fe00640 1 --2- 172.21.15.45:0/2334970947 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55699e9968a0 0x55699e9abaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:21.878 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:21.876+0000 7fa34fe00640 1 -- 172.21.15.45:0/2334970947 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55699e99fe20 msgr2=0x55699e9b7ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:11:21.878 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:21.876+0000 7fa34fe00640 1 --2- 172.21.15.45:0/2334970947 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55699e99fe20 0x55699e9b7ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:21.878 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:21.876+0000 7fa34fe00640 1 -- 172.21.15.45:0/2334970947 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55699e995b70 con 0x55699e997610 2024-04-16T17:11:21.879 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:21.877+0000 7fa34fe00640 1 --2- 172.21.15.45:0/2334970947 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55699e997610 0x55699e99bce0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fa338009de0 tx=0x7fa338009620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91dab87915a19f21 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:11:21.879 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:21.877+0000 7fa345600640 1 -- 172.21.15.45:0/2334970947 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa33801c470 con 0x55699e997610 2024-04-16T17:11:21.879 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:21.877+0000 7fa345600640 1 -- 172.21.15.45:0/2334970947 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa33801c610 con 0x55699e997610 2024-04-16T17:11:21.879 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:21.877+0000 7fa345600640 1 -- 172.21.15.45:0/2334970947 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa33800fb50 con 0x55699e997610 2024-04-16T17:11:21.879 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:21.878+0000 7fa35a5dd740 1 -- 172.21.15.45:0/2334970947 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55699e997610 msgr2=0x55699e99bce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:21.879 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:21.878+0000 7fa35a5dd740 1 --2- 172.21.15.45:0/2334970947 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55699e997610 0x55699e99bce0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fa338009de0 tx=0x7fa338009620 comp rx=0 tx=0).stop 2024-04-16T17:11:21.879 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:21.878+0000 7fa35a5dd740 1 -- 172.21.15.45:0/2334970947 shutdown_connections 2024-04-16T17:11:21.879 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:21.878+0000 7fa35a5dd740 1 --2- 172.21.15.45:0/2334970947 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55699e99fe20 0x55699e9b7ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:21.879 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:21.878+0000 7fa35a5dd740 1 --2- 172.21.15.45:0/2334970947 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55699e997610 0x55699e99bce0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:21.879 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:21.878+0000 7fa35a5dd740 1 --2- 172.21.15.45:0/2334970947 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55699e9968a0 0x55699e9abaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:21.879 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:21.878+0000 7fa35a5dd740 1 -- 172.21.15.45:0/2334970947 >> 172.21.15.45:0/2334970947 conn(0x55699e9315d0 msgr2=0x55699e99b5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:11:21.880 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:21.878+0000 7fa35a5dd740 1 -- 172.21.15.45:0/2334970947 shutdown_connections 2024-04-16T17:11:21.880 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:21.878+0000 7fa35a5dd740 1 -- 172.21.15.45:0/2334970947 wait complete. 2024-04-16T17:11:21.881 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:21.879+0000 7fa35a5dd740 1 -- start start 2024-04-16T17:11:21.881 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:21.879+0000 7fa35a5dd740 1 Processor -- start 2024-04-16T17:11:21.881 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:21.880+0000 7fa35a5dd740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55699e9968a0 0x55699e9a81a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:21.881 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:21.880+0000 7fa35a5dd740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55699e997610 0x55699e9994f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:21.881 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:21.880+0000 7fa35a5dd740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55699e99fe20 0x55699e999a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:21.881 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:21.880+0000 7fa35a5dd740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fa3501afba0 con 0x55699e99fe20 2024-04-16T17:11:21.881 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:21.880+0000 7fa35a5dd740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fa3501b1010 con 0x55699e997610 2024-04-16T17:11:21.881 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:21.880+0000 7fa35a5dd740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa3501b0270 con 0x55699e9968a0 2024-04-16T17:11:21.881 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:21.880+0000 7fa35a5dd740 10 client.0 _ll_register_callbacks cb 0x7fa338009910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:11:21.881 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:21.880+0000 7fa355800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55699e9968a0 0x55699e9a81a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:21.881 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:21.880+0000 7fa34fe00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55699e997610 0x55699e9994f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:21.882 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:21.880+0000 7fa355800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55699e9968a0 0x55699e9a81a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:59436/0 (socket says 172.21.15.45:59436) 2024-04-16T17:11:21.882 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:21.880+0000 7fa355800640 1 -- 172.21.15.45:0/3745808293 learned_addr learned my addr 172.21.15.45:0/3745808293 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:11:21.882 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:21.880+0000 7fa355800640 1 --2- 172.21.15.45:0/3745808293 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55699e99fe20 0x55699e999a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:21.883 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:21.881+0000 7fa34fe00640 1 -- 172.21.15.45:0/3745808293 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55699e9968a0 msgr2=0x55699e9a81a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:11:21.883 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:21.881+0000 7fa34fe00640 1 --2- 172.21.15.45:0/3745808293 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55699e9968a0 0x55699e9a81a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:21.883 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:21.881+0000 7fa34fe00640 1 -- 172.21.15.45:0/3745808293 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55699e99fe20 msgr2=0x55699e999a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:11:21.884 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:21.881+0000 7fa34fe00640 1 --2- 172.21.15.45:0/3745808293 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55699e99fe20 0x55699e999a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:21.884 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:21.881+0000 7fa34fe00640 1 -- 172.21.15.45:0/3745808293 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55699e999ff0 con 0x55699e997610 2024-04-16T17:11:21.884 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:21.881+0000 7fa355800640 1 --2- 172.21.15.45:0/3745808293 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55699e9968a0 0x55699e9a81a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:11:21.884 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:21.881+0000 7fa34fe00640 1 --2- 172.21.15.45:0/3745808293 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55699e997610 0x55699e9994f0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x55699e9b2750 tx=0x7fa33801df90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=84d4ce284b04635b server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:11:21.884 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:21.881+0000 7fa33ea00640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-04-16T17:11:21.884 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:21.881+0000 7fa33ea00640 1 -- 172.21.15.45:0/3745808293 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa33801c470 con 0x55699e997610 2024-04-16T17:11:21.884 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:21.882+0000 7fa35a5dd740 1 -- 172.21.15.45:0/3745808293 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa33801c610 con 0x55699e997610 2024-04-16T17:11:21.884 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:21.882+0000 7fa33ea00640 1 -- 172.21.15.45:0/3745808293 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fa3380041b0 con 0x55699e997610 2024-04-16T17:11:21.884 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:21.882+0000 7fa33ea00640 1 -- 172.21.15.45:0/3745808293 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fa33803b050 con 0x55699e997610 2024-04-16T17:11:21.884 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:21.882+0000 7fa35a5dd740 10 client.26876 fetch_fsmap learned FSMap version 32 2024-04-16T17:11:21.884 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:21.882+0000 7fa35a5dd740 1 -- 172.21.15.45:0/3745808293 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55699ea16500 con 0x55699e997610 2024-04-16T17:11:21.884 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:21.882+0000 7fa33ea00640 1 -- 172.21.15.45:0/3745808293 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa33801c610 con 0x55699e997610 2024-04-16T17:11:21.884 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:21.882+0000 7fa33ea00640 1 -- 172.21.15.45:0/3745808293 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fa338028020 con 0x55699e997610 2024-04-16T17:11:21.884 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:21.882+0000 7fa35a5dd740 10 client.26876 fetch_fsmap finished waiting for FSMap version 32 2024-04-16T17:11:21.884 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:21.882+0000 7fa35a5dd740 10 client.26876 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T17:11:21.885 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:21.882+0000 7fa35a5dd740 20 client.26876 populate_metadata read hostname 'smithi045' 2024-04-16T17:11:21.885 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:21.883+0000 7fa35a5dd740 1 --2- 172.21.15.45:0/3745808293 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55699ea16b30 0x55699ea18f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:21.885 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:21.883+0000 7fa35a5dd740 4 client.26876 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-16T17:11:21.885 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:21.883+0000 7fa35a5dd740 1 -- 172.21.15.45:0/3745808293 --> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55699e903010 con 0x55699ea16b30 2024-04-16T17:11:21.885 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:21.883+0000 7fa355800640 1 --2- 172.21.15.45:0/3745808293 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55699ea16b30 0x55699ea18f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:21.885 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:21.883+0000 7fa355800640 1 --2- 172.21.15.45:0/3745808293 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55699ea16b30 0x55699ea18f50 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=30e5fa66519d8a96 server_cookie=81d9597fc1a26e17 in_seq=0 out_seq=0 2024-04-16T17:11:21.885 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:21.883+0000 7fa33ea00640 10 client.26876 ms_handle_connect on v2:172.21.15.104:6838/3660070592 2024-04-16T17:11:21.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.898+0000 7f8f9b200640 1 -- 172.21.15.45:0/3676966211 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8f9c102110 msgr2=0x7f8f9c1024f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:21.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.898+0000 7f8f9b200640 1 --2- 172.21.15.45:0/3676966211 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8f9c102110 0x7f8f9c1024f0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f8f8c009900 tx=0x7f8f8c02bc40 comp rx=0 tx=0).stop 2024-04-16T17:11:21.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.899+0000 7f8f9b200640 1 -- 172.21.15.45:0/3676966211 shutdown_connections 2024-04-16T17:11:21.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.899+0000 7f8f9b200640 1 --2- 172.21.15.45:0/3676966211 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8f9c102ac0 0x7f8f9c106b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:21.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.899+0000 7f8f9b200640 1 --2- 172.21.15.45:0/3676966211 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8f9c1071e0 0x7f8f9c10edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:21.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.899+0000 7f8f9b200640 1 --2- 172.21.15.45:0/3676966211 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8f9c102110 0x7f8f9c1024f0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:21.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.899+0000 7f8f9b200640 1 -- 172.21.15.45:0/3676966211 >> 172.21.15.45:0/3676966211 conn(0x7f8f9c0fd9d0 msgr2=0x7f8f9c0ffdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:11:21.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.899+0000 7f8f9b200640 1 -- 172.21.15.45:0/3676966211 shutdown_connections 2024-04-16T17:11:21.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.900+0000 7f8f9b200640 1 -- 172.21.15.45:0/3676966211 wait complete. 2024-04-16T17:11:21.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.901+0000 7f8f9b200640 1 Processor -- start 2024-04-16T17:11:21.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.902+0000 7f8f9b200640 1 -- start start 2024-04-16T17:11:21.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.902+0000 7f8f9b200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8f9c102110 0x7f8f9c19ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:21.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.902+0000 7f8f9b200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8f9c1071e0 0x7f8f9c19f320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:21.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.902+0000 7f8f9b200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8f9c1a36a0 0x7f8f9c1a3b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:21.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.902+0000 7f8f9b200640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f8f9c118900 con 0x7f8f9c1a36a0 2024-04-16T17:11:21.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.902+0000 7f8f9b200640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f8f9c118780 con 0x7f8f9c1071e0 2024-04-16T17:11:21.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.902+0000 7f8f9b200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f8f9c118a80 con 0x7f8f9c102110 2024-04-16T17:11:21.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.903+0000 7f8f99e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8f9c102110 0x7f8f9c19ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:21.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.903+0000 7f8f99400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8f9c1071e0 0x7f8f9c19f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:21.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.903+0000 7f8f9a800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8f9c1a36a0 0x7f8f9c1a3b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:21.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.903+0000 7f8f99400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8f9c1071e0 0x7f8f9c19f320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:41444/0 (socket says 172.21.15.45:41444) 2024-04-16T17:11:21.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.903+0000 7f8f99400640 1 -- 172.21.15.45:0/3260015377 learned_addr learned my addr 172.21.15.45:0/3260015377 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:11:21.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.903+0000 7f8f9a800640 1 -- 172.21.15.45:0/3260015377 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8f9c102110 msgr2=0x7f8f9c19ede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:21.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.903+0000 7f8f9a800640 1 --2- 172.21.15.45:0/3260015377 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8f9c102110 0x7f8f9c19ede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:21.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.904+0000 7f8f9a800640 1 -- 172.21.15.45:0/3260015377 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8f9c1071e0 msgr2=0x7f8f9c19f320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:21.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.904+0000 7f8f9a800640 1 --2- 172.21.15.45:0/3260015377 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8f9c1071e0 0x7f8f9c19f320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:21.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.904+0000 7f8f9a800640 1 -- 172.21.15.45:0/3260015377 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f9c1a4190 con 0x7f8f9c1a36a0 2024-04-16T17:11:21.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.904+0000 7f8f9a800640 1 --2- 172.21.15.45:0/3260015377 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8f9c1a36a0 0x7f8f9c1a3b50 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f8f94009fd0 tx=0x7f8f9400ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:11:21.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.904+0000 7f8f92a00640 1 -- 172.21.15.45:0/3260015377 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f9400ce20 con 0x7f8f9c1a36a0 2024-04-16T17:11:21.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.904+0000 7f8f92a00640 1 -- 172.21.15.45:0/3260015377 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8f94005020 con 0x7f8f9c1a36a0 2024-04-16T17:11:21.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.904+0000 7f8f92a00640 1 -- 172.21.15.45:0/3260015377 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f94010720 con 0x7f8f9c1a36a0 2024-04-16T17:11:21.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.905+0000 7f8f9b200640 1 -- 172.21.15.45:0/3260015377 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f9c1a4480 con 0x7f8f9c1a36a0 2024-04-16T17:11:21.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.905+0000 7f8f9b200640 1 -- 172.21.15.45:0/3260015377 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f9c1abc20 con 0x7f8f9c1a36a0 2024-04-16T17:11:21.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.910+0000 7f8fa3c39740 1 Processor -- start 2024-04-16T17:11:21.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.912+0000 7f8f92a00640 1 -- 172.21.15.45:0/3260015377 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f8f94005540 con 0x7f8f9c1a36a0 2024-04-16T17:11:21.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.912+0000 7f8fa3c39740 1 -- start start 2024-04-16T17:11:21.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.912+0000 7f8fa3c39740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55784de0ef10 0x55784de0f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:21.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.912+0000 7f8fa3c39740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55784de0f830 0x55784de17ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:21.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.912+0000 7f8fa3c39740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55784de18410 0x55784de1a800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:21.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.912+0000 7f8fa3c39740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55784dd74020 con 0x55784de18410 2024-04-16T17:11:21.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.912+0000 7f8fa3c39740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55784dd1a460 con 0x55784de0ef10 2024-04-16T17:11:21.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.912+0000 7f8fa3c39740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55784dd6ffe0 con 0x55784de0f830 2024-04-16T17:11:21.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.913+0000 7f8f99400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55784de18410 0x55784de1a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:21.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.913+0000 7f8f99e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55784de0f830 0x55784de17ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:21.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.913+0000 7f8f99e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55784de0f830 0x55784de17ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:59474/0 (socket says 172.21.15.45:59474) 2024-04-16T17:11:21.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.913+0000 7f8f99e00640 1 -- 172.21.15.45:0/572928463 learned_addr learned my addr 172.21.15.45:0/572928463 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:11:21.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.914+0000 7f8f99400640 1 -- 172.21.15.45:0/572928463 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55784de0f830 msgr2=0x55784de17ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:11:21.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.914+0000 7f8f99400640 1 --2- 172.21.15.45:0/572928463 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55784de0f830 0x55784de17ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:21.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.914+0000 7f8f99400640 1 -- 172.21.15.45:0/572928463 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55784de0ef10 msgr2=0x55784de0f2f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-16T17:11:21.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.914+0000 7f8f99400640 1 --2- 172.21.15.45:0/572928463 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55784de0ef10 0x55784de0f2f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:21.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.914+0000 7f8f99400640 1 -- 172.21.15.45:0/572928463 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55784dd22040 con 0x55784de18410 2024-04-16T17:11:21.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.914+0000 7f8f99400640 1 --2- 172.21.15.45:0/572928463 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55784de18410 0x55784de1a800 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f8f9c108790 tx=0x7f8f88011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e40cb2e66b3b248a server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:11:21.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.915+0000 7f8f92a00640 1 --2- 172.21.15.45:0/3260015377 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f8f800a3df0 0x7f8f800a62b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:21.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.915+0000 7f8f92a00640 1 -- 172.21.15.45:0/3260015377 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f8f940d1540 con 0x7f8f9c1a36a0 2024-04-16T17:11:21.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.915+0000 7f8f87400640 1 -- 172.21.15.45:0/572928463 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f88004250 con 0x55784de18410 2024-04-16T17:11:21.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.915+0000 7f8f87400640 1 -- 172.21.15.45:0/572928463 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8f880043f0 con 0x55784de18410 2024-04-16T17:11:21.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.915+0000 7f8f87400640 1 -- 172.21.15.45:0/572928463 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f88004d90 con 0x55784de18410 2024-04-16T17:11:21.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.915+0000 7f8f99e00640 1 --2- 172.21.15.45:0/3260015377 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f8f800a3df0 0x7f8f800a62b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:21.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.916+0000 7f8fa3c39740 1 -- 172.21.15.45:0/572928463 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55784de18410 msgr2=0x55784de1a800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:21.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.916+0000 7f8fa3c39740 1 --2- 172.21.15.45:0/572928463 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55784de18410 0x55784de1a800 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f8f9c108790 tx=0x7f8f88011810 comp rx=0 tx=0).stop 2024-04-16T17:11:21.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.916+0000 7f8fa3c39740 1 -- 172.21.15.45:0/572928463 shutdown_connections 2024-04-16T17:11:21.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.916+0000 7f8fa3c39740 1 --2- 172.21.15.45:0/572928463 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55784de0f830 0x55784de17ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:21.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.916+0000 7f8fa3c39740 1 --2- 172.21.15.45:0/572928463 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55784de18410 0x55784de1a800 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:21.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.916+0000 7f8fa3c39740 1 --2- 172.21.15.45:0/572928463 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55784de0ef10 0x55784de0f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:21.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.916+0000 7f8fa3c39740 1 -- 172.21.15.45:0/572928463 >> 172.21.15.45:0/572928463 conn(0x55784de09640 msgr2=0x55784de09a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:11:21.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.917+0000 7f8fa3c39740 1 -- 172.21.15.45:0/572928463 shutdown_connections 2024-04-16T17:11:21.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.917+0000 7f8fa3c39740 1 -- 172.21.15.45:0/572928463 wait complete. 2024-04-16T17:11:21.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.917+0000 7f8f99e00640 1 --2- 172.21.15.45:0/3260015377 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f8f800a3df0 0x7f8f800a62b0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f8f8c005c40 tx=0x7f8f8c02cf90 comp rx=0 tx=0).ready entity=mgr.25411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:11:21.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.917+0000 7f8fa3c39740 1 -- start start 2024-04-16T17:11:21.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.918+0000 7f8fa3c39740 1 Processor -- start 2024-04-16T17:11:21.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.918+0000 7f8fa3c39740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55784de0ef10 0x55784de154c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:21.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.918+0000 7f8fa3c39740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55784de0f830 0x55784de15a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:21.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.918+0000 7f8fa3c39740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55784de18410 0x55784de15f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:21.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.918+0000 7f8fa3c39740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55784de1b200 con 0x55784de0f830 2024-04-16T17:11:21.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.918+0000 7f8fa3c39740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55784dd2d830 con 0x55784de18410 2024-04-16T17:11:21.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.918+0000 7f8fa3c39740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55784de1b380 con 0x55784de0ef10 2024-04-16T17:11:21.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.918+0000 7f8fa3c39740 10 client.0 _ll_register_callbacks cb 0x55784dd35150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:11:21.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.918+0000 7f8f99e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55784de0ef10 0x55784de154c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:21.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.918+0000 7f8f99e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55784de0ef10 0x55784de154c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:59478/0 (socket says 172.21.15.45:59478) 2024-04-16T17:11:21.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.918+0000 7f8f99e00640 1 -- 172.21.15.45:0/2414233022 learned_addr learned my addr 172.21.15.45:0/2414233022 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:11:21.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.919+0000 7f8f99400640 1 --2- 172.21.15.45:0/2414233022 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55784de0f830 0x55784de15a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:21.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.919+0000 7f8f99e00640 1 -- 172.21.15.45:0/2414233022 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55784de18410 msgr2=0x55784de15f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:11:21.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.919+0000 7f8f99e00640 1 --2- 172.21.15.45:0/2414233022 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55784de18410 0x55784de15f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:21.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.919+0000 7f8f99e00640 1 -- 172.21.15.45:0/2414233022 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55784de0f830 msgr2=0x55784de15a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:11:21.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.919+0000 7f8f99e00640 1 --2- 172.21.15.45:0/2414233022 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55784de0f830 0x55784de15a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:21.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.919+0000 7f8f99e00640 1 -- 172.21.15.45:0/2414233022 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55784dd31920 con 0x55784de0ef10 2024-04-16T17:11:21.921 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:21.919+0000 7fa33ea00640 1 -- 172.21.15.45:0/3745808293 <== mds.0 v2:172.21.15.104:6838/3660070592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55699e903010 con 0x55699ea16b30 2024-04-16T17:11:21.921 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:21.919+0000 7fa33ea00640 10 client.26876 handle_command_reply: tid=0 2024-04-16T17:11:21.921 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:21.919+0000 7fa35a5dd740 1 client.26876 shutdown 2024-04-16T17:11:21.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.919+0000 7f8f99e00640 1 --2- 172.21.15.45:0/2414233022 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55784de0ef10 0x55784de154c0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f8f8c036040 tx=0x7f8f8c02bc40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=119f287c09b97b92 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:11:21.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.919+0000 7f8f7be00640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-04-16T17:11:21.922 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:21.919+0000 7fa35a5dd740 1 -- 172.21.15.45:0/3745808293 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55699e997610 msgr2=0x55699e9994f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:21.922 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:21.919+0000 7fa35a5dd740 1 --2- 172.21.15.45:0/3745808293 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55699e997610 0x55699e9994f0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x55699e9b2750 tx=0x7fa33801df90 comp rx=0 tx=0).stop 2024-04-16T17:11:21.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.919+0000 7f8f7be00640 1 -- 172.21.15.45:0/2414233022 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f8c005070 con 0x55784de0ef10 2024-04-16T17:11:21.922 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:21.919+0000 7fa35a5dd740 1 -- 172.21.15.45:0/3745808293 shutdown_connections 2024-04-16T17:11:21.923 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:21.919+0000 7fa35a5dd740 1 --2- 172.21.15.45:0/3745808293 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55699e99fe20 0x55699e999a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:21.923 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:21.919+0000 7fa35a5dd740 1 --2- 172.21.15.45:0/3745808293 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55699e997610 0x55699e9994f0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:21.923 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:21.919+0000 7fa35a5dd740 1 --2- 172.21.15.45:0/3745808293 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55699ea16b30 0x55699ea18f50 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:21.923 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:21.919+0000 7fa35a5dd740 1 --2- 172.21.15.45:0/3745808293 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55699e9968a0 0x55699e9a81a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:21.923 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:21.919+0000 7fa35a5dd740 1 -- 172.21.15.45:0/3745808293 >> 172.21.15.45:0/3745808293 conn(0x55699e9315d0 msgr2=0x55699e99cc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:11:21.923 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:21.919+0000 7fa33ea00640 0 client.26876 ms_handle_reset on v2:172.21.15.104:6838/3660070592 2024-04-16T17:11:21.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.919+0000 7f8fa3c39740 1 -- 172.21.15.45:0/2414233022 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8f880043f0 con 0x55784de0ef10 2024-04-16T17:11:21.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.919+0000 7f8f7be00640 1 -- 172.21.15.45:0/2414233022 <== mon.1 v2:172.21.15.117:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f8f8c005210 con 0x55784de0ef10 2024-04-16T17:11:21.924 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:21.919+0000 7fa35a5dd740 1 -- 172.21.15.45:0/3745808293 shutdown_connections 2024-04-16T17:11:21.924 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:21.920+0000 7fa35a5dd740 1 -- 172.21.15.45:0/3745808293 wait complete. 2024-04-16T17:11:21.924 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:21.920+0000 7fa35a5dd740 20 client.26876 trim_cache size 0 max 16384 2024-04-16T17:11:21.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.920+0000 7f8f7be00640 1 -- 172.21.15.45:0/2414233022 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f8f8c005410 con 0x55784de0ef10 2024-04-16T17:11:21.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.920+0000 7f8fa3c39740 10 client.26827 fetch_fsmap learned FSMap version 32 2024-04-16T17:11:21.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.920+0000 7f8fa3c39740 1 -- 172.21.15.45:0/2414233022 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55784dcff7f0 con 0x55784de0ef10 2024-04-16T17:11:21.925 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-04-16T17:11:21.925 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-04-16T17:11:21.925 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-04-16T17:11:21.925 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-04-16T17:11:21.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.920+0000 7f8f7be00640 1 -- 172.21.15.45:0/2414233022 <== mon.1 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8f880043f0 con 0x55784de0ef10 2024-04-16T17:11:21.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.920+0000 7f8f7be00640 1 -- 172.21.15.45:0/2414233022 <== mon.1 v2:172.21.15.117:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f8f8c051070 con 0x55784de0ef10 2024-04-16T17:11:21.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.920+0000 7f8fa3c39740 10 client.26827 fetch_fsmap finished waiting for FSMap version 32 2024-04-16T17:11:21.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.920+0000 7f8fa3c39740 10 client.26827 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-16T17:11:21.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.920+0000 7f8fa3c39740 20 client.26827 populate_metadata read hostname 'smithi045' 2024-04-16T17:11:21.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.920+0000 7f8fa3c39740 1 --2- 172.21.15.45:0/2414233022 >> [v2:172.21.15.117:6838/2378675949,v1:172.21.15.117:6839/2378675949] conn(0x55784de28840 0x55784de2ad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:21.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.920+0000 7f8fa3c39740 4 client.26827 mds_command: new command op to 24532 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:11:21.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.920+0000 7f8fa3c39740 1 -- 172.21.15.45:0/2414233022 --> [v2:172.21.15.117:6838/2378675949,v1:172.21.15.117:6839/2378675949] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55784dd81a40 con 0x55784de28840 2024-04-16T17:11:21.926 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:21.920+0000 7fa357800640 1 -- 172.21.15.45:0/3502169411 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fa3340a3dc0 msgr2=0x7fa3340a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:21.927 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:21.920+0000 7fa357800640 1 --2- 172.21.15.45:0/3502169411 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fa3340a3dc0 0x7fa3340a6280 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fa3480bb0e0 tx=0x7fa3480f6a30 comp rx=0 tx=0).stop 2024-04-16T17:11:21.927 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:21.920+0000 7fa357800640 1 -- 172.21.15.45:0/3502169411 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa350104450 msgr2=0x7fa3501a06d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:21.927 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:21.920+0000 7fa357800640 1 --2- 172.21.15.45:0/3502169411 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa350104450 0x7fa3501a06d0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fa348005e30 tx=0x7fa348035d90 comp rx=0 tx=0).stop 2024-04-16T17:11:21.927 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:21.921+0000 7fa357800640 1 -- 172.21.15.45:0/3502169411 shutdown_connections 2024-04-16T17:11:21.927 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:21.921+0000 7fa357800640 1 --2- 172.21.15.45:0/3502169411 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fa3340a3dc0 0x7fa3340a6280 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:21.927 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:21.921+0000 7fa357800640 1 --2- 172.21.15.45:0/3502169411 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa35019a890 0x7fa35019ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:21.927 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:21.921+0000 7fa357800640 1 --2- 172.21.15.45:0/3502169411 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa350109520 0x7fa3501a0c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:21.927 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:21.921+0000 7fa357800640 1 --2- 172.21.15.45:0/3502169411 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa350104450 0x7fa3501a06d0 secure :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fa348005e30 tx=0x7fa348035d90 comp rx=0 tx=0).stop 2024-04-16T17:11:21.927 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:21.921+0000 7fa357800640 1 -- 172.21.15.45:0/3502169411 >> 172.21.15.45:0/3502169411 conn(0x7fa3500ffcf0 msgr2=0x7fa350101ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:11:21.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.921+0000 7f8f99400640 1 --2- 172.21.15.45:0/2414233022 >> [v2:172.21.15.117:6838/2378675949,v1:172.21.15.117:6839/2378675949] conn(0x55784de28840 0x55784de2ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:21.928 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:21.921+0000 7fa357800640 1 -- 172.21.15.45:0/3502169411 shutdown_connections 2024-04-16T17:11:21.928 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:21.921+0000 7fa357800640 1 -- 172.21.15.45:0/3502169411 wait complete. 2024-04-16T17:11:21.929 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.922+0000 7f8f7be00640 10 client.26827 ms_handle_connect on v2:172.21.15.117:6838/2378675949 2024-04-16T17:11:21.929 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.921+0000 7f8f99400640 1 --2- 172.21.15.45:0/2414233022 >> [v2:172.21.15.117:6838/2378675949,v1:172.21.15.117:6839/2378675949] conn(0x55784de28840 0x55784de2ad00 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=28ec06f21bc8853a server_cookie=46e255da4f6f26 in_seq=0 out_seq=0 2024-04-16T17:11:21.929 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.922+0000 7f8f7be00640 1 -- 172.21.15.45:0/2414233022 <== mds.2 v2:172.21.15.117:6838/2378675949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55784dd81a40 con 0x55784de28840 2024-04-16T17:11:21.929 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.922+0000 7f8f7be00640 10 client.26827 handle_command_reply: tid=0 2024-04-16T17:11:21.929 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.922+0000 7f8fa3c39740 1 client.26827 shutdown 2024-04-16T17:11:21.929 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.923+0000 7f8fa3c39740 1 -- 172.21.15.45:0/2414233022 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55784de0ef10 msgr2=0x55784de154c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:21.929 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.923+0000 7f8fa3c39740 1 --2- 172.21.15.45:0/2414233022 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55784de0ef10 0x55784de154c0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f8f8c036040 tx=0x7f8f8c02bc40 comp rx=0 tx=0).stop 2024-04-16T17:11:21.929 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.923+0000 7f8fa3c39740 1 -- 172.21.15.45:0/2414233022 shutdown_connections 2024-04-16T17:11:21.929 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.923+0000 7f8fa3c39740 1 --2- 172.21.15.45:0/2414233022 >> [v2:172.21.15.117:6838/2378675949,v1:172.21.15.117:6839/2378675949] conn(0x55784de28840 0x55784de2ad00 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:21.929 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.923+0000 7f8fa3c39740 1 --2- 172.21.15.45:0/2414233022 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55784de18410 0x55784de15f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:21.930 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.923+0000 7f8fa3c39740 1 --2- 172.21.15.45:0/2414233022 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55784de0f830 0x55784de15a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:21.930 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.923+0000 7f8f7be00640 0 client.26827 ms_handle_reset on v2:172.21.15.117:6838/2378675949 2024-04-16T17:11:21.930 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.923+0000 7f8fa3c39740 1 --2- 172.21.15.45:0/2414233022 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55784de0ef10 0x55784de154c0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:21.930 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.923+0000 7f8fa3c39740 1 -- 172.21.15.45:0/2414233022 >> 172.21.15.45:0/2414233022 conn(0x55784de09640 msgr2=0x55784de0b390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:11:21.930 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.923+0000 7f8fa3c39740 1 -- 172.21.15.45:0/2414233022 shutdown_connections 2024-04-16T17:11:21.930 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.923+0000 7f8fa3c39740 1 -- 172.21.15.45:0/2414233022 wait complete. 2024-04-16T17:11:21.930 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.923+0000 7f8fa3c39740 20 client.26827 trim_cache size 0 max 16384 2024-04-16T17:11:21.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.939+0000 7f8fa3c39740 1 Processor -- start 2024-04-16T17:11:21.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.939+0000 7f8fa3c39740 1 -- start start 2024-04-16T17:11:21.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.939+0000 7f8fa3c39740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55784de0ef10 0x55784de17bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:21.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.939+0000 7f8fa3c39740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55784de0f830 0x55784de24110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:21.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.939+0000 7f8fa3c39740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55784de18410 0x55784de08580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:21.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.939+0000 7f8fa3c39740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f8f9c1b8140 con 0x55784de18410 2024-04-16T17:11:21.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.939+0000 7f8fa3c39740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f8f9c1b5980 con 0x55784de0ef10 2024-04-16T17:11:21.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.939+0000 7f8fa3c39740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f8f9c1b37b0 con 0x55784de0f830 2024-04-16T17:11:21.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.939+0000 7f8f99400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55784de0f830 0x55784de24110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:21.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.939+0000 7f8f99e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55784de0ef10 0x55784de17bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:21.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.939+0000 7f8f99400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55784de0f830 0x55784de24110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:59488/0 (socket says 172.21.15.45:59488) 2024-04-16T17:11:21.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.939+0000 7f8f99400640 1 -- 172.21.15.45:0/2451205340 learned_addr learned my addr 172.21.15.45:0/2451205340 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:11:21.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.939+0000 7f8f99e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55784de0ef10 0x55784de17bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:41468/0 (socket says 172.21.15.45:41468) 2024-04-16T17:11:21.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.939+0000 7f8f9a800640 1 --2- 172.21.15.45:0/2451205340 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55784de18410 0x55784de08580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:21.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.940+0000 7f8f99400640 1 -- 172.21.15.45:0/2451205340 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55784de0ef10 msgr2=0x55784de17bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:11:21.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.940+0000 7f8f99400640 1 --2- 172.21.15.45:0/2451205340 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55784de0ef10 0x55784de17bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:21.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.940+0000 7f8f99400640 1 -- 172.21.15.45:0/2451205340 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55784de18410 msgr2=0x55784de08580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:11:21.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.940+0000 7f8f99400640 1 --2- 172.21.15.45:0/2451205340 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55784de18410 0x55784de08580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:21.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.940+0000 7f8f99400640 1 -- 172.21.15.45:0/2451205340 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55784de1ab70 con 0x55784de0f830 2024-04-16T17:11:21.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.940+0000 7f8f99400640 1 --2- 172.21.15.45:0/2451205340 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55784de0f830 0x55784de24110 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f8f88009c90 tx=0x7f8f88009fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1b943e3ad4b28bff server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:11:21.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.940+0000 7f8f86a00640 1 -- 172.21.15.45:0/2451205340 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f88004050 con 0x55784de0f830 2024-04-16T17:11:21.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.940+0000 7f8f86a00640 1 -- 172.21.15.45:0/2451205340 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8f88005520 con 0x55784de0f830 2024-04-16T17:11:21.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.940+0000 7f8f86a00640 1 -- 172.21.15.45:0/2451205340 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f88004c30 con 0x55784de0f830 2024-04-16T17:11:21.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.941+0000 7f8fa3c39740 1 -- 172.21.15.45:0/2451205340 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55784de0f830 msgr2=0x55784de24110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:21.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.941+0000 7f8fa3c39740 1 --2- 172.21.15.45:0/2451205340 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55784de0f830 0x55784de24110 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f8f88009c90 tx=0x7f8f88009fd0 comp rx=0 tx=0).stop 2024-04-16T17:11:21.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.941+0000 7f8fa3c39740 1 -- 172.21.15.45:0/2451205340 shutdown_connections 2024-04-16T17:11:21.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.941+0000 7f8fa3c39740 1 --2- 172.21.15.45:0/2451205340 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55784de0f830 0x55784de24110 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:21.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.941+0000 7f8fa3c39740 1 --2- 172.21.15.45:0/2451205340 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55784de18410 0x55784de08580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:21.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.941+0000 7f8fa3c39740 1 --2- 172.21.15.45:0/2451205340 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55784de0ef10 0x55784de17bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:21.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.941+0000 7f8fa3c39740 1 -- 172.21.15.45:0/2451205340 >> 172.21.15.45:0/2451205340 conn(0x55784de09640 msgr2=0x55784de07e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:11:21.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.941+0000 7f8fa3c39740 1 -- 172.21.15.45:0/2451205340 shutdown_connections 2024-04-16T17:11:21.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.941+0000 7f8fa3c39740 1 -- 172.21.15.45:0/2451205340 wait complete. 2024-04-16T17:11:21.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.942+0000 7f8fa3c39740 1 -- start start 2024-04-16T17:11:21.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.942+0000 7f8fa3c39740 1 Processor -- start 2024-04-16T17:11:21.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.942+0000 7f8fa3c39740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55784de0ef10 0x55784de12250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:21.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.942+0000 7f8fa3c39740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55784de0f830 0x55784de12790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:21.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.942+0000 7f8fa3c39740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55784de18410 0x55784de12cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:21.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.942+0000 7f8fa3c39740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f8f9c1b3e40 con 0x55784de18410 2024-04-16T17:11:21.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.942+0000 7f8fa3c39740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f8f9c1b52b0 con 0x55784de0ef10 2024-04-16T17:11:21.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.942+0000 7f8fa3c39740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f8f9c1b4510 con 0x55784de0f830 2024-04-16T17:11:21.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.942+0000 7f8fa3c39740 10 client.0 _ll_register_callbacks cb 0x7f8f8c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:11:21.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.943+0000 7f8f99e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55784de0ef10 0x55784de12250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:21.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.943+0000 7f8f99e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55784de0ef10 0x55784de12250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:41470/0 (socket says 172.21.15.45:41470) 2024-04-16T17:11:21.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.943+0000 7f8f99e00640 1 -- 172.21.15.45:0/923568302 learned_addr learned my addr 172.21.15.45:0/923568302 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:11:21.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.943+0000 7f8f99400640 1 --2- 172.21.15.45:0/923568302 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55784de0f830 0x55784de12790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:21.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.943+0000 7f8f9a800640 1 --2- 172.21.15.45:0/923568302 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55784de18410 0x55784de12cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:21.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.943+0000 7f8f99e00640 1 -- 172.21.15.45:0/923568302 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55784de0f830 msgr2=0x55784de12790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:11:21.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.943+0000 7f8f99e00640 1 --2- 172.21.15.45:0/923568302 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55784de0f830 0x55784de12790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:21.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.943+0000 7f8f99e00640 1 -- 172.21.15.45:0/923568302 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55784de18410 msgr2=0x55784de12cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:11:21.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.943+0000 7f8f99e00640 1 --2- 172.21.15.45:0/923568302 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55784de18410 0x55784de12cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:21.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.943+0000 7f8f99e00640 1 -- 172.21.15.45:0/923568302 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55784da863f0 con 0x55784de0ef10 2024-04-16T17:11:21.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.943+0000 7f8f9a800640 1 --2- 172.21.15.45:0/923568302 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55784de18410 0x55784de12cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:11:21.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.943+0000 7f8f99e00640 1 --2- 172.21.15.45:0/923568302 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55784de0ef10 0x55784de12250 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f8f8c0487e0 tx=0x7f8f8c0057e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e23c5deac985df7b server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:11:21.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.943+0000 7f8f7be00640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-04-16T17:11:21.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.944+0000 7f8f7be00640 1 -- 172.21.15.45:0/923568302 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f8c048a50 con 0x55784de0ef10 2024-04-16T17:11:21.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.944+0000 7f8fa3c39740 1 -- 172.21.15.45:0/923568302 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8f88005520 con 0x55784de0ef10 2024-04-16T17:11:21.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.944+0000 7f8f7be00640 1 -- 172.21.15.45:0/923568302 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f8f8c048bf0 con 0x55784de0ef10 2024-04-16T17:11:21.947 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.944+0000 7f8f7be00640 1 -- 172.21.15.45:0/923568302 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f8f8c048df0 con 0x55784de0ef10 2024-04-16T17:11:21.947 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.944+0000 7f8fa3c39740 10 client.26888 fetch_fsmap learned FSMap version 32 2024-04-16T17:11:21.947 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.944+0000 7f8fa3c39740 1 -- 172.21.15.45:0/923568302 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55784db507d0 con 0x55784de0ef10 2024-04-16T17:11:21.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.944+0000 7f8f7be00640 1 -- 172.21.15.45:0/923568302 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8f88005520 con 0x55784de0ef10 2024-04-16T17:11:21.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.945+0000 7f8f7be00640 1 -- 172.21.15.45:0/923568302 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f8f8c051070 con 0x55784de0ef10 2024-04-16T17:11:21.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.945+0000 7f8fa3c39740 10 client.26888 fetch_fsmap finished waiting for FSMap version 32 2024-04-16T17:11:21.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.945+0000 7f8fa3c39740 10 client.26888 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-16T17:11:21.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.945+0000 7f8fa3c39740 20 client.26888 populate_metadata read hostname 'smithi045' 2024-04-16T17:11:21.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.945+0000 7f8fa3c39740 1 --2- 172.21.15.45:0/923568302 >> [v2:172.21.15.117:6838/2378675949,v1:172.21.15.117:6839/2378675949] conn(0x55784de8f4d0 0x55784de918f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:21.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.945+0000 7f8fa3c39740 4 client.26888 mds_command: new command op to 24532 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T17:11:21.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.945+0000 7f8fa3c39740 1 -- 172.21.15.45:0/923568302 --> [v2:172.21.15.117:6838/2378675949,v1:172.21.15.117:6839/2378675949] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55784dd7b8b0 con 0x55784de8f4d0 2024-04-16T17:11:21.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.945+0000 7f8f99400640 1 --2- 172.21.15.45:0/923568302 >> [v2:172.21.15.117:6838/2378675949,v1:172.21.15.117:6839/2378675949] conn(0x55784de8f4d0 0x55784de918f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:21.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.946+0000 7f8f99400640 1 --2- 172.21.15.45:0/923568302 >> [v2:172.21.15.117:6838/2378675949,v1:172.21.15.117:6839/2378675949] conn(0x55784de8f4d0 0x55784de918f0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8e3849b2e9fe442b server_cookie=a7d73258a2fa906f in_seq=0 out_seq=0 2024-04-16T17:11:21.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.946+0000 7f8f7be00640 10 client.26888 ms_handle_connect on v2:172.21.15.117:6838/2378675949 2024-04-16T17:11:21.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.946+0000 7f8f7be00640 1 -- 172.21.15.45:0/923568302 <== mds.2 v2:172.21.15.117:6838/2378675949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55784dd7b8b0 con 0x55784de8f4d0 2024-04-16T17:11:21.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.946+0000 7f8f7be00640 10 client.26888 handle_command_reply: tid=0 2024-04-16T17:11:21.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.946+0000 7f8fa3c39740 1 client.26888 shutdown 2024-04-16T17:11:21.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.946+0000 7f8fa3c39740 1 -- 172.21.15.45:0/923568302 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55784de0ef10 msgr2=0x55784de12250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:21.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.946+0000 7f8fa3c39740 1 --2- 172.21.15.45:0/923568302 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55784de0ef10 0x55784de12250 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f8f8c0487e0 tx=0x7f8f8c0057e0 comp rx=0 tx=0).stop 2024-04-16T17:11:21.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.946+0000 7f8fa3c39740 1 -- 172.21.15.45:0/923568302 shutdown_connections 2024-04-16T17:11:21.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.946+0000 7f8fa3c39740 1 --2- 172.21.15.45:0/923568302 >> [v2:172.21.15.117:6838/2378675949,v1:172.21.15.117:6839/2378675949] conn(0x55784de8f4d0 0x55784de918f0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:21.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.946+0000 7f8fa3c39740 1 --2- 172.21.15.45:0/923568302 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55784de0f830 0x55784de12790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:21.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.946+0000 7f8fa3c39740 1 --2- 172.21.15.45:0/923568302 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55784de18410 0x55784de12cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:21.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.946+0000 7f8fa3c39740 1 --2- 172.21.15.45:0/923568302 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55784de0ef10 0x55784de12250 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:21.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.946+0000 7f8f7be00640 0 client.26888 ms_handle_reset on v2:172.21.15.117:6838/2378675949 2024-04-16T17:11:21.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.946+0000 7f8fa3c39740 1 -- 172.21.15.45:0/923568302 >> 172.21.15.45:0/923568302 conn(0x55784de09640 msgr2=0x55784de15480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:11:21.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.946+0000 7f8fa3c39740 1 -- 172.21.15.45:0/923568302 shutdown_connections 2024-04-16T17:11:21.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.946+0000 7f8fa3c39740 1 -- 172.21.15.45:0/923568302 wait complete. 2024-04-16T17:11:21.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.946+0000 7f8fa3c39740 20 client.26888 trim_cache size 0 max 16384 2024-04-16T17:11:21.950 INFO:teuthology.orchestra.run.smithi045.stdout:[] 2024-04-16T17:11:21.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.947+0000 7f8f9b200640 1 -- 172.21.15.45:0/3260015377 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f8f800a3df0 msgr2=0x7f8f800a62b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:21.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.947+0000 7f8f9b200640 1 --2- 172.21.15.45:0/3260015377 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f8f800a3df0 0x7f8f800a62b0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f8f8c005c40 tx=0x7f8f8c02cf90 comp rx=0 tx=0).stop 2024-04-16T17:11:21.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.947+0000 7f8f9b200640 1 -- 172.21.15.45:0/3260015377 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8f9c1a36a0 msgr2=0x7f8f9c1a3b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:21.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.947+0000 7f8f9b200640 1 --2- 172.21.15.45:0/3260015377 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8f9c1a36a0 0x7f8f9c1a3b50 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f8f94009fd0 tx=0x7f8f9400ef10 comp rx=0 tx=0).stop 2024-04-16T17:11:21.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.947+0000 7f8f9b200640 1 -- 172.21.15.45:0/3260015377 shutdown_connections 2024-04-16T17:11:21.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.947+0000 7f8f9b200640 1 --2- 172.21.15.45:0/3260015377 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f8f800a3df0 0x7f8f800a62b0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:21.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.947+0000 7f8f9b200640 1 --2- 172.21.15.45:0/3260015377 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8f9c1a36a0 0x7f8f9c1a3b50 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:21.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.947+0000 7f8f9b200640 1 --2- 172.21.15.45:0/3260015377 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8f9c1071e0 0x7f8f9c19f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:21.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.947+0000 7f8f9b200640 1 --2- 172.21.15.45:0/3260015377 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8f9c102110 0x7f8f9c19ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:21.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.947+0000 7f8f9b200640 1 -- 172.21.15.45:0/3260015377 >> 172.21.15.45:0/3260015377 conn(0x7f8f9c0fd9d0 msgr2=0x7f8f9c0ffd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:11:21.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.947+0000 7f8f9b200640 1 -- 172.21.15.45:0/3260015377 shutdown_connections 2024-04-16T17:11:21.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:21.947+0000 7f8f9b200640 1 -- 172.21.15.45:0/3260015377 wait complete. 2024-04-16T17:11:21.964 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-16T17:11:22.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.235+0000 7fa599800640 1 -- 172.21.15.45:0/3017254317 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa594103980 msgr2=0x7fa59410be60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:22.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.235+0000 7fa599800640 1 --2- 172.21.15.45:0/3017254317 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa594103980 0x7fa59410be60 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fa584009900 tx=0x7fa58402bcc0 comp rx=0 tx=0).stop 2024-04-16T17:11:22.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.236+0000 7fa599800640 1 -- 172.21.15.45:0/3017254317 shutdown_connections 2024-04-16T17:11:22.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.236+0000 7fa599800640 1 --2- 172.21.15.45:0/3017254317 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa594103980 0x7fa59410be60 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:22.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.236+0000 7fa599800640 1 --2- 172.21.15.45:0/3017254317 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa59410c480 0x7fa59410e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:22.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.236+0000 7fa599800640 1 --2- 172.21.15.45:0/3017254317 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa594103060 0x7fa594103440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:22.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.236+0000 7fa599800640 1 -- 172.21.15.45:0/3017254317 >> 172.21.15.45:0/3017254317 conn(0x7fa5940fc880 msgr2=0x7fa5940feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:11:22.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.236+0000 7fa599800640 1 -- 172.21.15.45:0/3017254317 shutdown_connections 2024-04-16T17:11:22.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.236+0000 7fa599800640 1 -- 172.21.15.45:0/3017254317 wait complete. 2024-04-16T17:11:22.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.237+0000 7fa599800640 1 Processor -- start 2024-04-16T17:11:22.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.237+0000 7fa599800640 1 -- start start 2024-04-16T17:11:22.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.237+0000 7fa599800640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa594103060 0x7fa5941a0680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:22.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.237+0000 7fa599800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa594103980 0x7fa5941a0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:22.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.237+0000 7fa599800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa59410c480 0x7fa59419a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:22.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.237+0000 7fa599800640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fa5941122b0 con 0x7fa59410c480 2024-04-16T17:11:22.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.237+0000 7fa599800640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fa594112130 con 0x7fa594103060 2024-04-16T17:11:22.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.237+0000 7fa599800640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa594112430 con 0x7fa594103980 2024-04-16T17:11:22.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.237+0000 7fa593e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa594103060 0x7fa5941a0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:22.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.237+0000 7fa593e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa594103060 0x7fa5941a0680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:41488/0 (socket says 172.21.15.45:41488) 2024-04-16T17:11:22.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.237+0000 7fa598e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa59410c480 0x7fa59419a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:22.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.237+0000 7fa593e00640 1 -- 172.21.15.45:0/2307093134 learned_addr learned my addr 172.21.15.45:0/2307093134 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:11:22.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.237+0000 7fa593400640 1 --2- 172.21.15.45:0/2307093134 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa594103980 0x7fa5941a0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:22.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.238+0000 7fa593e00640 1 -- 172.21.15.45:0/2307093134 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa594103980 msgr2=0x7fa5941a0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:22.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.238+0000 7fa593e00640 1 --2- 172.21.15.45:0/2307093134 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa594103980 0x7fa5941a0bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:22.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.238+0000 7fa593e00640 1 -- 172.21.15.45:0/2307093134 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa59410c480 msgr2=0x7fa59419a7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:22.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.238+0000 7fa593e00640 1 --2- 172.21.15.45:0/2307093134 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa59410c480 0x7fa59419a7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:22.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.238+0000 7fa593e00640 1 -- 172.21.15.45:0/2307093134 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa59419afc0 con 0x7fa594103060 2024-04-16T17:11:22.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.238+0000 7fa593e00640 1 --2- 172.21.15.45:0/2307093134 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa594103060 0x7fa5941a0680 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fa57c00e9f0 tx=0x7fa57c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:11:22.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.238+0000 7fa590c00640 1 -- 172.21.15.45:0/2307093134 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa57c00cd80 con 0x7fa594103060 2024-04-16T17:11:22.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.238+0000 7fa599800640 1 -- 172.21.15.45:0/2307093134 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa59419b2b0 con 0x7fa594103060 2024-04-16T17:11:22.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.238+0000 7fa590c00640 1 -- 172.21.15.45:0/2307093134 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa57c010910 con 0x7fa594103060 2024-04-16T17:11:22.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.238+0000 7fa590c00640 1 -- 172.21.15.45:0/2307093134 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa57c0188f0 con 0x7fa594103060 2024-04-16T17:11:22.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.239+0000 7fa599800640 1 -- 172.21.15.45:0/2307093134 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5941a74d0 con 0x7fa594103060 2024-04-16T17:11:22.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.243+0000 7fa59dbd2740 1 Processor -- start 2024-04-16T17:11:22.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.243+0000 7fa59dbd2740 1 -- start start 2024-04-16T17:11:22.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.244+0000 7fa59dbd2740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fe980dcf10 0x55fe980dd2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:22.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.244+0000 7fa59dbd2740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fe980dd830 0x55fe980e5ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:22.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.244+0000 7fa59dbd2740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fe980e6410 0x55fe980e8800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:22.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.244+0000 7fa59dbd2740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55fe98042020 con 0x55fe980dcf10 2024-04-16T17:11:22.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.244+0000 7fa59dbd2740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55fe97fe8460 con 0x55fe980e6410 2024-04-16T17:11:22.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.244+0000 7fa59dbd2740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55fe9803dfe0 con 0x55fe980dd830 2024-04-16T17:11:22.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.244+0000 7fa590c00640 1 -- 172.21.15.45:0/2307093134 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fa57c010ac0 con 0x7fa594103060 2024-04-16T17:11:22.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.244+0000 7fa593400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fe980e6410 0x55fe980e8800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:22.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.244+0000 7fa598e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fe980dcf10 0x55fe980dd2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:22.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.244+0000 7fa593400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fe980e6410 0x55fe980e8800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:41502/0 (socket says 172.21.15.45:41502) 2024-04-16T17:11:22.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.244+0000 7fa593400640 1 -- 172.21.15.45:0/3330398021 learned_addr learned my addr 172.21.15.45:0/3330398021 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:11:22.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.244+0000 7fa593e00640 1 --2- 172.21.15.45:0/3330398021 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fe980dd830 0x55fe980e5ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:22.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.245+0000 7fa593e00640 1 -- 172.21.15.45:0/3330398021 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fe980e6410 msgr2=0x55fe980e8800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:11:22.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.245+0000 7fa593e00640 1 --2- 172.21.15.45:0/3330398021 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fe980e6410 0x55fe980e8800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:22.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.245+0000 7fa593e00640 1 -- 172.21.15.45:0/3330398021 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fe980dcf10 msgr2=0x55fe980dd2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:11:22.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.245+0000 7fa593e00640 1 --2- 172.21.15.45:0/3330398021 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fe980dcf10 0x55fe980dd2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:22.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.245+0000 7fa593e00640 1 -- 172.21.15.45:0/3330398021 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fe97ff0040 con 0x55fe980dd830 2024-04-16T17:11:22.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.245+0000 7fa598e00640 1 --2- 172.21.15.45:0/3330398021 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fe980dcf10 0x55fe980dd2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:11:22.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.245+0000 7fa593400640 1 --2- 172.21.15.45:0/3330398021 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fe980e6410 0x55fe980e8800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:11:22.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.245+0000 7fa593e00640 1 --2- 172.21.15.45:0/3330398021 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fe980dd830 0x55fe980e5ed0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fa57c0ce2f0 tx=0x7fa57c0d95f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1ae5aa77cc9f0bee server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:11:22.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.245+0000 7fa589600640 1 -- 172.21.15.45:0/3330398021 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa57c0e4070 con 0x55fe980dd830 2024-04-16T17:11:22.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.246+0000 7fa589600640 1 -- 172.21.15.45:0/3330398021 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa57c0dfae0 con 0x55fe980dd830 2024-04-16T17:11:22.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.246+0000 7fa589600640 1 -- 172.21.15.45:0/3330398021 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa57c0debb0 con 0x55fe980dd830 2024-04-16T17:11:22.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.246+0000 7fa590c00640 1 --2- 172.21.15.45:0/2307093134 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fa5780a3d20 0x7fa5780a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:22.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.246+0000 7fa590c00640 1 -- 172.21.15.45:0/2307093134 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fa57c0d0f50 con 0x7fa594103060 2024-04-16T17:11:22.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.246+0000 7fa59dbd2740 1 -- 172.21.15.45:0/3330398021 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fe980dd830 msgr2=0x55fe980e5ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:22.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.246+0000 7fa59dbd2740 1 --2- 172.21.15.45:0/3330398021 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fe980dd830 0x55fe980e5ed0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fa57c0ce2f0 tx=0x7fa57c0d95f0 comp rx=0 tx=0).stop 2024-04-16T17:11:22.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.246+0000 7fa59dbd2740 1 -- 172.21.15.45:0/3330398021 shutdown_connections 2024-04-16T17:11:22.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.246+0000 7fa59dbd2740 1 --2- 172.21.15.45:0/3330398021 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fe980dd830 0x55fe980e5ed0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:22.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.246+0000 7fa59dbd2740 1 --2- 172.21.15.45:0/3330398021 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fe980e6410 0x55fe980e8800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:22.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.246+0000 7fa59dbd2740 1 --2- 172.21.15.45:0/3330398021 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fe980dcf10 0x55fe980dd2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:22.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.246+0000 7fa59dbd2740 1 -- 172.21.15.45:0/3330398021 >> 172.21.15.45:0/3330398021 conn(0x55fe980d7640 msgr2=0x55fe980d7a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:11:22.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.247+0000 7fa59dbd2740 1 -- 172.21.15.45:0/3330398021 shutdown_connections 2024-04-16T17:11:22.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.247+0000 7fa598e00640 1 --2- 172.21.15.45:0/2307093134 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fa5780a3d20 0x7fa5780a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:22.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.247+0000 7fa59dbd2740 1 -- 172.21.15.45:0/3330398021 wait complete. 2024-04-16T17:11:22.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.247+0000 7fa59dbd2740 1 -- start start 2024-04-16T17:11:22.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.247+0000 7fa59dbd2740 1 Processor -- start 2024-04-16T17:11:22.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.247+0000 7fa598e00640 1 --2- 172.21.15.45:0/2307093134 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fa5780a3d20 0x7fa5780a61e0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fa58c005fd0 tx=0x7fa58c009500 comp rx=0 tx=0).ready entity=mgr.25411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:11:22.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.248+0000 7fa59dbd2740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fe980dcf10 0x55fe980e3530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:22.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.248+0000 7fa59dbd2740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fe980dd830 0x55fe980e3a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:22.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.248+0000 7fa59dbd2740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fe980e6410 0x55fe980e3fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:22.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.248+0000 7fa59dbd2740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55fe980e9200 con 0x55fe980dcf10 2024-04-16T17:11:22.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.248+0000 7fa59dbd2740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55fe97ffb830 con 0x55fe980e6410 2024-04-16T17:11:22.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.248+0000 7fa59dbd2740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55fe980e9380 con 0x55fe980dd830 2024-04-16T17:11:22.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.248+0000 7fa59dbd2740 10 client.0 _ll_register_callbacks cb 0x55fe98003150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:11:22.250 INFO:tasks.workunit.client.0.smithi045.stdout: 1 65056 1.41 MB/sec execute 84 sec latency 3723.028 ms 2024-04-16T17:11:22.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.248+0000 7fa593400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fe980dd830 0x55fe980e3a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:22.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.248+0000 7fa593400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fe980dd830 0x55fe980e3a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:59544/0 (socket says 172.21.15.45:59544) 2024-04-16T17:11:22.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.248+0000 7fa593400640 1 -- 172.21.15.45:0/347407250 learned_addr learned my addr 172.21.15.45:0/347407250 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:11:22.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.248+0000 7fa593e00640 1 --2- 172.21.15.45:0/347407250 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fe980dcf10 0x55fe980e3530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:22.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.248+0000 7fa598e00640 1 --2- 172.21.15.45:0/347407250 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fe980e6410 0x55fe980e3fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:22.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.249+0000 7fa593400640 1 -- 172.21.15.45:0/347407250 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fe980e6410 msgr2=0x55fe980e3fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:11:22.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.249+0000 7fa593400640 1 --2- 172.21.15.45:0/347407250 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fe980e6410 0x55fe980e3fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:22.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.249+0000 7fa593400640 1 -- 172.21.15.45:0/347407250 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fe980dcf10 msgr2=0x55fe980e3530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:11:22.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.249+0000 7fa593400640 1 --2- 172.21.15.45:0/347407250 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fe980dcf10 0x55fe980e3530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:22.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.249+0000 7fa593400640 1 -- 172.21.15.45:0/347407250 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fe97fff920 con 0x55fe980dd830 2024-04-16T17:11:22.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.249+0000 7fa593400640 1 --2- 172.21.15.45:0/347407250 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fe980dd830 0x55fe980e3a70 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fa584005ab0 tx=0x7fa5840045b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a81e4eff9b324dfe server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:11:22.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.249+0000 7fa582000640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-04-16T17:11:22.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.249+0000 7fa582000640 1 -- 172.21.15.45:0/347407250 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa584043070 con 0x55fe980dd830 2024-04-16T17:11:22.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.249+0000 7fa582000640 1 -- 172.21.15.45:0/347407250 <== mon.1 v2:172.21.15.117:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fa5840046c0 con 0x55fe980dd830 2024-04-16T17:11:22.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.249+0000 7fa59dbd2740 1 -- 172.21.15.45:0/347407250 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa57c0dfae0 con 0x55fe980dd830 2024-04-16T17:11:22.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.249+0000 7fa582000640 1 -- 172.21.15.45:0/347407250 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fa584050050 con 0x55fe980dd830 2024-04-16T17:11:22.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.249+0000 7fa59dbd2740 10 client.26845 fetch_fsmap learned FSMap version 32 2024-04-16T17:11:22.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.249+0000 7fa59dbd2740 1 -- 172.21.15.45:0/347407250 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55fe97fcd7f0 con 0x55fe980dd830 2024-04-16T17:11:22.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.250+0000 7fa582000640 1 -- 172.21.15.45:0/347407250 <== mon.1 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa57c0dfae0 con 0x55fe980dd830 2024-04-16T17:11:22.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.250+0000 7fa582000640 1 -- 172.21.15.45:0/347407250 <== mon.1 v2:172.21.15.117:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fa5840035f0 con 0x55fe980dd830 2024-04-16T17:11:22.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.250+0000 7fa59dbd2740 10 client.26845 fetch_fsmap finished waiting for FSMap version 32 2024-04-16T17:11:22.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.250+0000 7fa59dbd2740 10 client.26845 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-04-16T17:11:22.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.250+0000 7fa59dbd2740 20 client.26845 populate_metadata read hostname 'smithi045' 2024-04-16T17:11:22.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.250+0000 7fa59dbd2740 1 --2- 172.21.15.45:0/347407250 >> [v2:172.21.15.45:6840/3755310822,v1:172.21.15.45:6841/3755310822] conn(0x55fe980f6900 0x55fe980f8d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:22.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.250+0000 7fa59dbd2740 4 client.26845 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:11:22.254 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.250+0000 7fa59dbd2740 1 -- 172.21.15.45:0/347407250 --> [v2:172.21.15.45:6840/3755310822,v1:172.21.15.45:6841/3755310822] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fe9804fa40 con 0x55fe980f6900 2024-04-16T17:11:22.254 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.250+0000 7fa593e00640 1 --2- 172.21.15.45:0/347407250 >> [v2:172.21.15.45:6840/3755310822,v1:172.21.15.45:6841/3755310822] conn(0x55fe980f6900 0x55fe980f8d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:22.254 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.251+0000 7fa593e00640 1 --2- 172.21.15.45:0/347407250 >> [v2:172.21.15.45:6840/3755310822,v1:172.21.15.45:6841/3755310822] conn(0x55fe980f6900 0x55fe980f8d60 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=94e792b58b052a0 server_cookie=318d8e8fc2df9bc1 in_seq=0 out_seq=0 2024-04-16T17:11:22.254 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.251+0000 7fa582000640 10 client.26845 ms_handle_connect on v2:172.21.15.45:6840/3755310822 2024-04-16T17:11:22.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.252+0000 7fa582000640 1 -- 172.21.15.45:0/347407250 <== mds.1 v2:172.21.15.45:6840/3755310822 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55fe9804fa40 con 0x55fe980f6900 2024-04-16T17:11:22.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.252+0000 7fa582000640 10 client.26845 handle_command_reply: tid=0 2024-04-16T17:11:22.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.252+0000 7fa59dbd2740 1 client.26845 shutdown 2024-04-16T17:11:22.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.252+0000 7fa59dbd2740 1 -- 172.21.15.45:0/347407250 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fe980dd830 msgr2=0x55fe980e3a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:22.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.252+0000 7fa59dbd2740 1 --2- 172.21.15.45:0/347407250 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fe980dd830 0x55fe980e3a70 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fa584005ab0 tx=0x7fa5840045b0 comp rx=0 tx=0).stop 2024-04-16T17:11:22.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.252+0000 7fa59dbd2740 1 -- 172.21.15.45:0/347407250 shutdown_connections 2024-04-16T17:11:22.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.252+0000 7fa59dbd2740 1 --2- 172.21.15.45:0/347407250 >> [v2:172.21.15.45:6840/3755310822,v1:172.21.15.45:6841/3755310822] conn(0x55fe980f6900 0x55fe980f8d60 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:22.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.252+0000 7fa59dbd2740 1 --2- 172.21.15.45:0/347407250 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fe980dd830 0x55fe980e3a70 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:22.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.252+0000 7fa59dbd2740 1 --2- 172.21.15.45:0/347407250 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fe980e6410 0x55fe980e3fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:22.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.252+0000 7fa59dbd2740 1 --2- 172.21.15.45:0/347407250 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fe980dcf10 0x55fe980e3530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:22.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.252+0000 7fa582000640 0 client.26845 ms_handle_reset on v2:172.21.15.45:6840/3755310822 2024-04-16T17:11:22.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.252+0000 7fa59dbd2740 1 -- 172.21.15.45:0/347407250 >> 172.21.15.45:0/347407250 conn(0x55fe980d7640 msgr2=0x55fe980d9390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:11:22.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.252+0000 7fa59dbd2740 1 -- 172.21.15.45:0/347407250 shutdown_connections 2024-04-16T17:11:22.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.252+0000 7fa59dbd2740 1 -- 172.21.15.45:0/347407250 wait complete. 2024-04-16T17:11:22.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.253+0000 7fa59dbd2740 20 client.26845 trim_cache size 0 max 16384 2024-04-16T17:11:22.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.269+0000 7fa59dbd2740 1 Processor -- start 2024-04-16T17:11:22.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.269+0000 7fa59dbd2740 1 -- start start 2024-04-16T17:11:22.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.269+0000 7fa59dbd2740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fe980dcf10 0x55fe980e5c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:22.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.269+0000 7fa59dbd2740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fe980dd830 0x55fe980f20f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:22.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.269+0000 7fa59dbd2740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fe980e6410 0x55fe980e2690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:22.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.269+0000 7fa59dbd2740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fa5941b3ea0 con 0x55fe980dd830 2024-04-16T17:11:22.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.269+0000 7fa59dbd2740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fa5941b16e0 con 0x55fe980e6410 2024-04-16T17:11:22.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.269+0000 7fa59dbd2740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa5941af510 con 0x55fe980dcf10 2024-04-16T17:11:22.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.269+0000 7fa593e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fe980dcf10 0x55fe980e5c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:22.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.269+0000 7fa598e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fe980e6410 0x55fe980e2690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:22.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.270+0000 7fa598e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fe980e6410 0x55fe980e2690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:41528/0 (socket says 172.21.15.45:41528) 2024-04-16T17:11:22.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.270+0000 7fa598e00640 1 -- 172.21.15.45:0/3249330106 learned_addr learned my addr 172.21.15.45:0/3249330106 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:11:22.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.270+0000 7fa593e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fe980dcf10 0x55fe980e5c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:59550/0 (socket says 172.21.15.45:59550) 2024-04-16T17:11:22.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.270+0000 7fa593400640 1 --2- 172.21.15.45:0/3249330106 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fe980dd830 0x55fe980f20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:22.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.270+0000 7fa593e00640 1 -- 172.21.15.45:0/3249330106 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fe980e6410 msgr2=0x55fe980e2690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:11:22.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.270+0000 7fa593e00640 1 --2- 172.21.15.45:0/3249330106 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fe980e6410 0x55fe980e2690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:22.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.270+0000 7fa593e00640 1 -- 172.21.15.45:0/3249330106 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fe980dd830 msgr2=0x55fe980f20f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:11:22.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.270+0000 7fa593e00640 1 --2- 172.21.15.45:0/3249330106 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fe980dd830 0x55fe980f20f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:22.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.270+0000 7fa593e00640 1 -- 172.21.15.45:0/3249330106 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fe980f3070 con 0x55fe980dcf10 2024-04-16T17:11:22.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.270+0000 7fa593400640 1 --2- 172.21.15.45:0/3249330106 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fe980dd830 0x55fe980f20f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:11:22.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.271+0000 7fa593e00640 1 --2- 172.21.15.45:0/3249330106 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fe980dcf10 0x55fe980e5c60 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fa57c00e9c0 tx=0x7fa57c0bd040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=738c5f0fbb3a6965 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:11:22.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.271+0000 7fa588c00640 1 -- 172.21.15.45:0/3249330106 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa57c09c9c0 con 0x55fe980dcf10 2024-04-16T17:11:22.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.271+0000 7fa588c00640 1 -- 172.21.15.45:0/3249330106 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa57c09cbf0 con 0x55fe980dcf10 2024-04-16T17:11:22.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.271+0000 7fa588c00640 1 -- 172.21.15.45:0/3249330106 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa57c09c550 con 0x55fe980dcf10 2024-04-16T17:11:22.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.272+0000 7fa59dbd2740 1 -- 172.21.15.45:0/3249330106 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fe980dcf10 msgr2=0x55fe980e5c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:22.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.272+0000 7fa59dbd2740 1 --2- 172.21.15.45:0/3249330106 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fe980dcf10 0x55fe980e5c60 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fa57c00e9c0 tx=0x7fa57c0bd040 comp rx=0 tx=0).stop 2024-04-16T17:11:22.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.272+0000 7fa59dbd2740 1 -- 172.21.15.45:0/3249330106 shutdown_connections 2024-04-16T17:11:22.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.272+0000 7fa59dbd2740 1 --2- 172.21.15.45:0/3249330106 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fe980e6410 0x55fe980e2690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:22.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.272+0000 7fa59dbd2740 1 --2- 172.21.15.45:0/3249330106 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fe980dd830 0x55fe980f20f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:22.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.272+0000 7fa59dbd2740 1 --2- 172.21.15.45:0/3249330106 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fe980dcf10 0x55fe980e5c60 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:22.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.272+0000 7fa59dbd2740 1 -- 172.21.15.45:0/3249330106 >> 172.21.15.45:0/3249330106 conn(0x55fe980d7640 msgr2=0x55fe980e34f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:11:22.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.272+0000 7fa59dbd2740 1 -- 172.21.15.45:0/3249330106 shutdown_connections 2024-04-16T17:11:22.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.272+0000 7fa59dbd2740 1 -- 172.21.15.45:0/3249330106 wait complete. 2024-04-16T17:11:22.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.272+0000 7fa59dbd2740 1 -- start start 2024-04-16T17:11:22.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.273+0000 7fa59dbd2740 1 Processor -- start 2024-04-16T17:11:22.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.273+0000 7fa59dbd2740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fe980dcf10 0x55fe980ea210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:22.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.273+0000 7fa59dbd2740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fe980dd830 0x55fe980ea750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:22.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.273+0000 7fa59dbd2740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fe980e6410 0x55fe980eac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:22.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.273+0000 7fa59dbd2740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fa5941afba0 con 0x55fe980e6410 2024-04-16T17:11:22.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.273+0000 7fa59dbd2740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fa5941b1010 con 0x55fe980dcf10 2024-04-16T17:11:22.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.273+0000 7fa59dbd2740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa5941b0270 con 0x55fe980dd830 2024-04-16T17:11:22.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.273+0000 7fa59dbd2740 10 client.0 _ll_register_callbacks cb 0x7fa58c0093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:11:22.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.273+0000 7fa593e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fe980dcf10 0x55fe980ea210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:22.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.273+0000 7fa593e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fe980dcf10 0x55fe980ea210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:41530/0 (socket says 172.21.15.45:41530) 2024-04-16T17:11:22.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.273+0000 7fa593e00640 1 -- 172.21.15.45:0/1277511408 learned_addr learned my addr 172.21.15.45:0/1277511408 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:11:22.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.274+0000 7fa593400640 1 --2- 172.21.15.45:0/1277511408 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fe980dd830 0x55fe980ea750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:22.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.274+0000 7fa598e00640 1 --2- 172.21.15.45:0/1277511408 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fe980e6410 0x55fe980eac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:22.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.274+0000 7fa593e00640 1 -- 172.21.15.45:0/1277511408 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fe980dd830 msgr2=0x55fe980ea750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:11:22.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.274+0000 7fa593e00640 1 --2- 172.21.15.45:0/1277511408 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fe980dd830 0x55fe980ea750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:22.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.274+0000 7fa593e00640 1 -- 172.21.15.45:0/1277511408 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fe980e6410 msgr2=0x55fe980eac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:11:22.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.274+0000 7fa593e00640 1 --2- 172.21.15.45:0/1277511408 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fe980e6410 0x55fe980eac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:22.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.274+0000 7fa593e00640 1 -- 172.21.15.45:0/1277511408 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fe97d543f0 con 0x55fe980dcf10 2024-04-16T17:11:22.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.274+0000 7fa593e00640 1 --2- 172.21.15.45:0/1277511408 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fe980dcf10 0x55fe980ea210 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fa57c09c8f0 tx=0x7fa57c01f000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a85bec4f348175b8 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:11:22.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.274+0000 7fa582000640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-04-16T17:11:22.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.274+0000 7fa582000640 1 -- 172.21.15.45:0/1277511408 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa57c09c730 con 0x55fe980dcf10 2024-04-16T17:11:22.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.274+0000 7fa59dbd2740 1 -- 172.21.15.45:0/1277511408 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa57c09cbf0 con 0x55fe980dcf10 2024-04-16T17:11:22.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.274+0000 7fa582000640 1 -- 172.21.15.45:0/1277511408 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fa57c099260 con 0x55fe980dcf10 2024-04-16T17:11:22.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.275+0000 7fa582000640 1 -- 172.21.15.45:0/1277511408 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fa57c09c9c0 con 0x55fe980dcf10 2024-04-16T17:11:22.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.275+0000 7fa59dbd2740 10 client.26906 fetch_fsmap learned FSMap version 32 2024-04-16T17:11:22.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.275+0000 7fa59dbd2740 1 -- 172.21.15.45:0/1277511408 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55fe97e1e7d0 con 0x55fe980dcf10 2024-04-16T17:11:22.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.275+0000 7fa582000640 1 -- 172.21.15.45:0/1277511408 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa57c09cbf0 con 0x55fe980dcf10 2024-04-16T17:11:22.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.275+0000 7fa582000640 1 -- 172.21.15.45:0/1277511408 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fa57c09cee0 con 0x55fe980dcf10 2024-04-16T17:11:22.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.275+0000 7fa59dbd2740 10 client.26906 fetch_fsmap finished waiting for FSMap version 32 2024-04-16T17:11:22.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.275+0000 7fa59dbd2740 10 client.26906 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-04-16T17:11:22.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.275+0000 7fa59dbd2740 20 client.26906 populate_metadata read hostname 'smithi045' 2024-04-16T17:11:22.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.275+0000 7fa59dbd2740 1 --2- 172.21.15.45:0/1277511408 >> [v2:172.21.15.45:6840/3755310822,v1:172.21.15.45:6841/3755310822] conn(0x55fe9815d1b0 0x55fe9815f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:22.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.275+0000 7fa59dbd2740 4 client.26906 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T17:11:22.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.275+0000 7fa59dbd2740 1 -- 172.21.15.45:0/1277511408 --> [v2:172.21.15.45:6840/3755310822,v1:172.21.15.45:6841/3755310822] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55fe980498b0 con 0x55fe9815d1b0 2024-04-16T17:11:22.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.276+0000 7fa593400640 1 --2- 172.21.15.45:0/1277511408 >> [v2:172.21.15.45:6840/3755310822,v1:172.21.15.45:6841/3755310822] conn(0x55fe9815d1b0 0x55fe9815f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:22.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.276+0000 7fa593400640 1 --2- 172.21.15.45:0/1277511408 >> [v2:172.21.15.45:6840/3755310822,v1:172.21.15.45:6841/3755310822] conn(0x55fe9815d1b0 0x55fe9815f5a0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e63b6bf7f570dbfc server_cookie=2f37f99f4aef532 in_seq=0 out_seq=0 2024-04-16T17:11:22.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.276+0000 7fa582000640 10 client.26906 ms_handle_connect on v2:172.21.15.45:6840/3755310822 2024-04-16T17:11:22.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.276+0000 7fa582000640 1 -- 172.21.15.45:0/1277511408 <== mds.1 v2:172.21.15.45:6840/3755310822 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55fe980498b0 con 0x55fe9815d1b0 2024-04-16T17:11:22.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.276+0000 7fa582000640 10 client.26906 handle_command_reply: tid=0 2024-04-16T17:11:22.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.276+0000 7fa59dbd2740 1 client.26906 shutdown 2024-04-16T17:11:22.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.277+0000 7fa59dbd2740 1 -- 172.21.15.45:0/1277511408 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fe980dcf10 msgr2=0x55fe980ea210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:22.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.277+0000 7fa59dbd2740 1 --2- 172.21.15.45:0/1277511408 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fe980dcf10 0x55fe980ea210 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fa57c09c8f0 tx=0x7fa57c01f000 comp rx=0 tx=0).stop 2024-04-16T17:11:22.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.277+0000 7fa59dbd2740 1 -- 172.21.15.45:0/1277511408 shutdown_connections 2024-04-16T17:11:22.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.277+0000 7fa59dbd2740 1 --2- 172.21.15.45:0/1277511408 >> [v2:172.21.15.45:6840/3755310822,v1:172.21.15.45:6841/3755310822] conn(0x55fe9815d1b0 0x55fe9815f5a0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:22.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.277+0000 7fa59dbd2740 1 --2- 172.21.15.45:0/1277511408 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fe980dd830 0x55fe980ea750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:22.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.277+0000 7fa59dbd2740 1 --2- 172.21.15.45:0/1277511408 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fe980e6410 0x55fe980eac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:22.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.277+0000 7fa59dbd2740 1 --2- 172.21.15.45:0/1277511408 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fe980dcf10 0x55fe980ea210 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:22.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.277+0000 7fa59dbd2740 1 -- 172.21.15.45:0/1277511408 >> 172.21.15.45:0/1277511408 conn(0x55fe980d7640 msgr2=0x55fe980d9300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:11:22.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.277+0000 7fa582000640 0 client.26906 ms_handle_reset on v2:172.21.15.45:6840/3755310822 2024-04-16T17:11:22.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.277+0000 7fa59dbd2740 1 -- 172.21.15.45:0/1277511408 shutdown_connections 2024-04-16T17:11:22.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.277+0000 7fa59dbd2740 1 -- 172.21.15.45:0/1277511408 wait complete. 2024-04-16T17:11:22.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.277+0000 7fa59dbd2740 20 client.26906 trim_cache size 0 max 16384 2024-04-16T17:11:22.281 INFO:teuthology.orchestra.run.smithi045.stdout:[] 2024-04-16T17:11:22.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.278+0000 7fa599800640 1 -- 172.21.15.45:0/2307093134 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fa5780a3d20 msgr2=0x7fa5780a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:22.282 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.278+0000 7fa599800640 1 --2- 172.21.15.45:0/2307093134 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fa5780a3d20 0x7fa5780a61e0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fa58c005fd0 tx=0x7fa58c009500 comp rx=0 tx=0).stop 2024-04-16T17:11:22.282 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.278+0000 7fa599800640 1 -- 172.21.15.45:0/2307093134 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa594103060 msgr2=0x7fa5941a0680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:22.282 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.278+0000 7fa599800640 1 --2- 172.21.15.45:0/2307093134 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa594103060 0x7fa5941a0680 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fa57c00e9f0 tx=0x7fa57c00eeb0 comp rx=0 tx=0).stop 2024-04-16T17:11:22.282 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.278+0000 7fa599800640 1 -- 172.21.15.45:0/2307093134 shutdown_connections 2024-04-16T17:11:22.282 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.278+0000 7fa599800640 1 --2- 172.21.15.45:0/2307093134 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fa5780a3d20 0x7fa5780a61e0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:22.282 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.278+0000 7fa599800640 1 --2- 172.21.15.45:0/2307093134 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa594103980 0x7fa5941a0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:22.282 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.278+0000 7fa599800640 1 --2- 172.21.15.45:0/2307093134 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa59410c480 0x7fa59419a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:22.282 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.278+0000 7fa599800640 1 --2- 172.21.15.45:0/2307093134 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa594103060 0x7fa5941a0680 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:22.282 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.278+0000 7fa599800640 1 -- 172.21.15.45:0/2307093134 >> 172.21.15.45:0/2307093134 conn(0x7fa5940fc880 msgr2=0x7fa594100be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:11:22.282 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.279+0000 7fa599800640 1 -- 172.21.15.45:0/2307093134 shutdown_connections 2024-04-16T17:11:22.282 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.279+0000 7fa599800640 1 -- 172.21.15.45:0/2307093134 wait complete. 2024-04-16T17:11:22.297 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-16T17:11:22.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.506+0000 7f961b200640 1 -- 172.21.15.45:0/2251109705 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9614102ac0 msgr2=0x7f9614106b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:22.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.506+0000 7f961b200640 1 --2- 172.21.15.45:0/2251109705 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9614102ac0 0x7f9614106b00 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f9604009900 tx=0x7f960402bc40 comp rx=0 tx=0).stop 2024-04-16T17:11:22.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.506+0000 7f961b200640 1 -- 172.21.15.45:0/2251109705 shutdown_connections 2024-04-16T17:11:22.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.506+0000 7f961b200640 1 --2- 172.21.15.45:0/2251109705 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9614102ac0 0x7f9614106b00 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:22.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.506+0000 7f961b200640 1 --2- 172.21.15.45:0/2251109705 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f96141071e0 0x7f961410edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:22.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.506+0000 7f961b200640 1 --2- 172.21.15.45:0/2251109705 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9614102110 0x7f96141024f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:22.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.506+0000 7f961b200640 1 -- 172.21.15.45:0/2251109705 >> 172.21.15.45:0/2251109705 conn(0x7f96140fd9d0 msgr2=0x7f96140ffdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:11:22.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.506+0000 7f961b200640 1 -- 172.21.15.45:0/2251109705 shutdown_connections 2024-04-16T17:11:22.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.506+0000 7f961b200640 1 -- 172.21.15.45:0/2251109705 wait complete. 2024-04-16T17:11:22.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.507+0000 7f961b200640 1 Processor -- start 2024-04-16T17:11:22.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.507+0000 7f961b200640 1 -- start start 2024-04-16T17:11:22.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.508+0000 7f961b200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9614102110 0x7f961410c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:22.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.508+0000 7f961b200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f96141071e0 0x7f961410c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:22.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.508+0000 7f961b200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f961410cb30 0x7f9614072a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:22.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.508+0000 7f961b200640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f9614075e30 con 0x7f961410cb30 2024-04-16T17:11:22.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.508+0000 7f961b200640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f9614075cb0 con 0x7f9614102110 2024-04-16T17:11:22.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.508+0000 7f961b200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f9614075fb0 con 0x7f96141071e0 2024-04-16T17:11:22.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.508+0000 7f9619e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9614102110 0x7f961410c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:22.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.508+0000 7f9619400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f96141071e0 0x7f961410c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:22.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.508+0000 7f9619e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9614102110 0x7f961410c0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:41538/0 (socket says 172.21.15.45:41538) 2024-04-16T17:11:22.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.508+0000 7f9619e00640 1 -- 172.21.15.45:0/2978211883 learned_addr learned my addr 172.21.15.45:0/2978211883 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:11:22.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.508+0000 7f9619400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f96141071e0 0x7f961410c5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:59564/0 (socket says 172.21.15.45:59564) 2024-04-16T17:11:22.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.508+0000 7f961a800640 1 --2- 172.21.15.45:0/2978211883 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f961410cb30 0x7f9614072a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:22.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.509+0000 7f9619400640 1 -- 172.21.15.45:0/2978211883 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9614102110 msgr2=0x7f961410c0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:22.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.509+0000 7f9619400640 1 --2- 172.21.15.45:0/2978211883 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9614102110 0x7f961410c0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:22.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.509+0000 7f9619400640 1 -- 172.21.15.45:0/2978211883 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f961410cb30 msgr2=0x7f9614072a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:22.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.509+0000 7f9619400640 1 --2- 172.21.15.45:0/2978211883 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f961410cb30 0x7f9614072a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:22.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.509+0000 7f9619400640 1 -- 172.21.15.45:0/2978211883 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9614072fe0 con 0x7f96141071e0 2024-04-16T17:11:22.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.509+0000 7f9619e00640 1 --2- 172.21.15.45:0/2978211883 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9614102110 0x7f961410c0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:11:22.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.509+0000 7f9619400640 1 --2- 172.21.15.45:0/2978211883 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f96141071e0 0x7f961410c5f0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f9604036040 tx=0x7f960402cb30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:11:22.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.509+0000 7f960ea00640 1 -- 172.21.15.45:0/2978211883 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9604041070 con 0x7f96141071e0 2024-04-16T17:11:22.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.510+0000 7f960ea00640 1 -- 172.21.15.45:0/2978211883 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9604004450 con 0x7f96141071e0 2024-04-16T17:11:22.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.510+0000 7f960ea00640 1 -- 172.21.15.45:0/2978211883 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9604003d40 con 0x7f96141071e0 2024-04-16T17:11:22.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.510+0000 7f961b200640 1 -- 172.21.15.45:0/2978211883 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9614073270 con 0x7f96141071e0 2024-04-16T17:11:22.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.510+0000 7f961b200640 1 -- 172.21.15.45:0/2978211883 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96140736b0 con 0x7f96141071e0 2024-04-16T17:11:22.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.514+0000 7f961f569740 1 Processor -- start 2024-04-16T17:11:22.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.514+0000 7f961f569740 1 -- start start 2024-04-16T17:11:22.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.515+0000 7f960ea00640 1 -- 172.21.15.45:0/2978211883 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f9604044070 con 0x7f96141071e0 2024-04-16T17:11:22.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.516+0000 7f961f569740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555973885f10 0x5559738862f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:22.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.516+0000 7f961f569740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555973886830 0x55597388eed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:22.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.516+0000 7f961f569740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55597388f410 0x555973891800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:22.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.516+0000 7f961f569740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5559737eb020 con 0x555973886830 2024-04-16T17:11:22.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.516+0000 7f961f569740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x555973791460 con 0x555973885f10 2024-04-16T17:11:22.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.516+0000 7f961f569740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5559737e6fe0 con 0x55597388f410 2024-04-16T17:11:22.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.516+0000 7f960ea00640 1 --2- 172.21.15.45:0/2978211883 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f95fc0a3da0 0x7f95fc0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:22.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.516+0000 7f9619e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555973886830 0x55597388eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:22.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.516+0000 7f9619400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55597388f410 0x555973891800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:22.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.516+0000 7f9619e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555973886830 0x55597388eed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55152/0 (socket says 172.21.15.45:55152) 2024-04-16T17:11:22.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.516+0000 7f9619e00640 1 -- 172.21.15.45:0/842621525 learned_addr learned my addr 172.21.15.45:0/842621525 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:11:22.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.516+0000 7f960ea00640 1 -- 172.21.15.45:0/2978211883 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f96040f2450 con 0x7f96141071e0 2024-04-16T17:11:22.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.517+0000 7f9619e00640 1 -- 172.21.15.45:0/842621525 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55597388f410 msgr2=0x555973891800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:11:22.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.517+0000 7f9619e00640 1 --2- 172.21.15.45:0/842621525 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55597388f410 0x555973891800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:22.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.517+0000 7f9619e00640 1 -- 172.21.15.45:0/842621525 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555973885f10 msgr2=0x5559738862f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:11:22.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.517+0000 7f9619e00640 1 --2- 172.21.15.45:0/842621525 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555973885f10 0x5559738862f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:22.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.517+0000 7f9619e00640 1 -- 172.21.15.45:0/842621525 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555973799040 con 0x555973886830 2024-04-16T17:11:22.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.517+0000 7f961a800640 1 --2- 172.21.15.45:0/2978211883 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f95fc0a3da0 0x7f95fc0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:22.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.517+0000 7f9619e00640 1 --2- 172.21.15.45:0/842621525 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555973886830 0x55597388eed0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f9608011650 tx=0x7f9608009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2983e111fc4bf316 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:11:22.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.518+0000 7f9603000640 1 -- 172.21.15.45:0/842621525 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f960801e070 con 0x555973886830 2024-04-16T17:11:22.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.518+0000 7f9603000640 1 -- 172.21.15.45:0/842621525 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f960800fba0 con 0x555973886830 2024-04-16T17:11:22.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.518+0000 7f9603000640 1 -- 172.21.15.45:0/842621525 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9608019d20 con 0x555973886830 2024-04-16T17:11:22.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.519+0000 7f961f569740 1 -- 172.21.15.45:0/842621525 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555973886830 msgr2=0x55597388eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:22.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.519+0000 7f961a800640 1 --2- 172.21.15.45:0/2978211883 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f95fc0a3da0 0x7f95fc0a6260 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f9610009270 tx=0x7f96100091e0 comp rx=0 tx=0).ready entity=mgr.25411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:11:22.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.519+0000 7f961f569740 1 --2- 172.21.15.45:0/842621525 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555973886830 0x55597388eed0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f9608011650 tx=0x7f9608009f90 comp rx=0 tx=0).stop 2024-04-16T17:11:22.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.519+0000 7f961f569740 1 -- 172.21.15.45:0/842621525 shutdown_connections 2024-04-16T17:11:22.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.519+0000 7f961f569740 1 --2- 172.21.15.45:0/842621525 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55597388f410 0x555973891800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:22.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.519+0000 7f961f569740 1 --2- 172.21.15.45:0/842621525 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555973886830 0x55597388eed0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:22.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.519+0000 7f961f569740 1 --2- 172.21.15.45:0/842621525 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555973885f10 0x5559738862f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:22.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.519+0000 7f961f569740 1 -- 172.21.15.45:0/842621525 >> 172.21.15.45:0/842621525 conn(0x555973880640 msgr2=0x555973880a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:11:22.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.519+0000 7f961f569740 1 -- 172.21.15.45:0/842621525 shutdown_connections 2024-04-16T17:11:22.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.519+0000 7f961f569740 1 -- 172.21.15.45:0/842621525 wait complete. 2024-04-16T17:11:22.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.520+0000 7f961f569740 1 -- start start 2024-04-16T17:11:22.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.520+0000 7f961f569740 1 Processor -- start 2024-04-16T17:11:22.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.521+0000 7f961f569740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555973885f10 0x55597388c500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:22.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.521+0000 7f961f569740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555973886830 0x55597388ca40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:22.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.521+0000 7f961f569740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55597388f410 0x55597388cf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:22.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.521+0000 7f961f569740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x555973892200 con 0x555973885f10 2024-04-16T17:11:22.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.521+0000 7f961f569740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5559737a4830 con 0x55597388f410 2024-04-16T17:11:22.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.521+0000 7f961f569740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x555973892380 con 0x555973886830 2024-04-16T17:11:22.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.521+0000 7f961f569740 10 client.0 _ll_register_callbacks cb 0x5559737ac150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:11:22.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.521+0000 7f9619e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555973885f10 0x55597388c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:22.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.521+0000 7f9619e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555973885f10 0x55597388c500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55154/0 (socket says 172.21.15.45:55154) 2024-04-16T17:11:22.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.521+0000 7f9619e00640 1 -- 172.21.15.45:0/756909208 learned_addr learned my addr 172.21.15.45:0/756909208 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:11:22.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.521+0000 7f9619400640 1 --2- 172.21.15.45:0/756909208 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555973886830 0x55597388ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:22.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.521+0000 7f961a800640 1 --2- 172.21.15.45:0/756909208 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55597388f410 0x55597388cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:22.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.521+0000 7f9619e00640 1 -- 172.21.15.45:0/756909208 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555973886830 msgr2=0x55597388ca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:11:22.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.521+0000 7f9619e00640 1 --2- 172.21.15.45:0/756909208 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555973886830 0x55597388ca40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:22.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.521+0000 7f9619e00640 1 -- 172.21.15.45:0/756909208 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55597388f410 msgr2=0x55597388cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:11:22.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.521+0000 7f9619e00640 1 --2- 172.21.15.45:0/756909208 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55597388f410 0x55597388cf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:22.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.521+0000 7f9619e00640 1 -- 172.21.15.45:0/756909208 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5559737a8920 con 0x555973885f10 2024-04-16T17:11:22.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.522+0000 7f9619400640 1 --2- 172.21.15.45:0/756909208 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555973886830 0x55597388ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:11:22.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.522+0000 7f961a800640 1 --2- 172.21.15.45:0/756909208 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55597388f410 0x55597388cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:11:22.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.522+0000 7f9619e00640 1 --2- 172.21.15.45:0/756909208 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555973885f10 0x55597388c500 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f9608019860 tx=0x7f960800faf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=603b17350df67f46 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:11:22.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.522+0000 7f95f7400640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-16T17:11:22.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.522+0000 7f95f7400640 1 -- 172.21.15.45:0/756909208 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9608027060 con 0x555973885f10 2024-04-16T17:11:22.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.522+0000 7f95f7400640 1 -- 172.21.15.45:0/756909208 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f960800fd50 con 0x555973885f10 2024-04-16T17:11:22.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.522+0000 7f961f569740 1 -- 172.21.15.45:0/756909208 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f960800fba0 con 0x555973885f10 2024-04-16T17:11:22.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.523+0000 7f95f7400640 1 -- 172.21.15.45:0/756909208 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f960802e050 con 0x555973885f10 2024-04-16T17:11:22.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.523+0000 7f961f569740 10 client.16758 fetch_fsmap learned FSMap version 32 2024-04-16T17:11:22.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.523+0000 7f961f569740 1 -- 172.21.15.45:0/756909208 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5559737767f0 con 0x555973885f10 2024-04-16T17:11:22.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.523+0000 7f95f7400640 1 -- 172.21.15.45:0/756909208 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f960800fba0 con 0x555973885f10 2024-04-16T17:11:22.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.523+0000 7f95f7400640 1 -- 172.21.15.45:0/756909208 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f9608021030 con 0x555973885f10 2024-04-16T17:11:22.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.523+0000 7f961f569740 10 client.16758 fetch_fsmap finished waiting for FSMap version 32 2024-04-16T17:11:22.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.523+0000 7f961f569740 10 client.16758 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:11:22.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.523+0000 7f961f569740 20 client.16758 populate_metadata read hostname 'smithi045' 2024-04-16T17:11:22.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.523+0000 7f961f569740 1 --2- 172.21.15.45:0/756909208 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55597389f740 0x5559738a1b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:22.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.523+0000 7f961f569740 4 client.16758 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:11:22.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.523+0000 7f961f569740 1 -- 172.21.15.45:0/756909208 --> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5559737f8a40 con 0x55597389f740 2024-04-16T17:11:22.527 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.524+0000 7f9619400640 1 --2- 172.21.15.45:0/756909208 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55597389f740 0x5559738a1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:22.527 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.524+0000 7f9619400640 1 --2- 172.21.15.45:0/756909208 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55597389f740 0x5559738a1b30 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b6c52d483b2df920 server_cookie=58233b4181830670 in_seq=0 out_seq=0 2024-04-16T17:11:22.527 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.524+0000 7f95f7400640 10 client.16758 ms_handle_connect on v2:172.21.15.104:6838/3660070592 2024-04-16T17:11:22.527 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.525+0000 7f95f7400640 1 -- 172.21.15.45:0/756909208 <== mds.0 v2:172.21.15.104:6838/3660070592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5559737f8a40 con 0x55597389f740 2024-04-16T17:11:22.527 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.525+0000 7f95f7400640 10 client.16758 handle_command_reply: tid=0 2024-04-16T17:11:22.527 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.525+0000 7f961f569740 1 client.16758 shutdown 2024-04-16T17:11:22.527 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.525+0000 7f961f569740 1 -- 172.21.15.45:0/756909208 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555973885f10 msgr2=0x55597388c500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:22.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.525+0000 7f961f569740 1 --2- 172.21.15.45:0/756909208 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555973885f10 0x55597388c500 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f9608019860 tx=0x7f960800faf0 comp rx=0 tx=0).stop 2024-04-16T17:11:22.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.526+0000 7f961f569740 1 -- 172.21.15.45:0/756909208 shutdown_connections 2024-04-16T17:11:22.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.526+0000 7f961f569740 1 --2- 172.21.15.45:0/756909208 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55597389f740 0x5559738a1b30 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:22.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.526+0000 7f961f569740 1 --2- 172.21.15.45:0/756909208 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555973886830 0x55597388ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:22.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.526+0000 7f961f569740 1 --2- 172.21.15.45:0/756909208 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55597388f410 0x55597388cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:22.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.526+0000 7f961f569740 1 --2- 172.21.15.45:0/756909208 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555973885f10 0x55597388c500 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:22.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.526+0000 7f961f569740 1 -- 172.21.15.45:0/756909208 >> 172.21.15.45:0/756909208 conn(0x555973880640 msgr2=0x555973882390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:11:22.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.526+0000 7f95f7400640 0 client.16758 ms_handle_reset on v2:172.21.15.104:6838/3660070592 2024-04-16T17:11:22.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.526+0000 7f961f569740 1 -- 172.21.15.45:0/756909208 shutdown_connections 2024-04-16T17:11:22.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.526+0000 7f961f569740 1 -- 172.21.15.45:0/756909208 wait complete. 2024-04-16T17:11:22.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.526+0000 7f961f569740 20 client.16758 trim_cache size 0 max 16384 2024-04-16T17:11:22.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.541+0000 7f961f569740 1 Processor -- start 2024-04-16T17:11:22.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.541+0000 7f961f569740 1 -- start start 2024-04-16T17:11:22.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.542+0000 7f961f569740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555973885f10 0x55597389b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:22.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.542+0000 7f961f569740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555973886830 0x55597388cda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:22.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.542+0000 7f961f569740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55597388f410 0x55597387f580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:22.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.542+0000 7f961f569740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f96141c09b0 con 0x555973886830 2024-04-16T17:11:22.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.542+0000 7f961f569740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f96141be1f0 con 0x55597388f410 2024-04-16T17:11:22.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.542+0000 7f961f569740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f96141bc020 con 0x555973885f10 2024-04-16T17:11:22.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.542+0000 7f9619400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555973886830 0x55597388cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:22.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.542+0000 7f9619e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555973885f10 0x55597389b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:22.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.542+0000 7f9619400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555973886830 0x55597388cda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55162/0 (socket says 172.21.15.45:55162) 2024-04-16T17:11:22.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.542+0000 7f9619400640 1 -- 172.21.15.45:0/2539999691 learned_addr learned my addr 172.21.15.45:0/2539999691 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:11:22.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.542+0000 7f961a800640 1 --2- 172.21.15.45:0/2539999691 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55597388f410 0x55597387f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:22.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.542+0000 7f9619400640 1 -- 172.21.15.45:0/2539999691 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555973885f10 msgr2=0x55597389b0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:11:22.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.542+0000 7f9619400640 1 --2- 172.21.15.45:0/2539999691 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555973885f10 0x55597389b0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:22.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.542+0000 7f9619400640 1 -- 172.21.15.45:0/2539999691 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55597388f410 msgr2=0x55597387f580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:11:22.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.542+0000 7f9619400640 1 --2- 172.21.15.45:0/2539999691 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55597388f410 0x55597387f580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:22.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.542+0000 7f9619400640 1 -- 172.21.15.45:0/2539999691 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555973891b70 con 0x555973886830 2024-04-16T17:11:22.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.543+0000 7f9619400640 1 --2- 172.21.15.45:0/2539999691 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555973886830 0x55597388cda0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f960404c000 tx=0x7f96040c2040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3abca7dd846f9e13 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:11:22.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.543+0000 7f9602600640 1 -- 172.21.15.45:0/2539999691 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f960404b040 con 0x555973886830 2024-04-16T17:11:22.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.543+0000 7f9602600640 1 -- 172.21.15.45:0/2539999691 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9604004c80 con 0x555973886830 2024-04-16T17:11:22.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.543+0000 7f9602600640 1 -- 172.21.15.45:0/2539999691 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96040c6e00 con 0x555973886830 2024-04-16T17:11:22.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.544+0000 7f961f569740 1 -- 172.21.15.45:0/2539999691 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555973886830 msgr2=0x55597388cda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:22.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.544+0000 7f961f569740 1 --2- 172.21.15.45:0/2539999691 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555973886830 0x55597388cda0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f960404c000 tx=0x7f96040c2040 comp rx=0 tx=0).stop 2024-04-16T17:11:22.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.544+0000 7f961f569740 1 -- 172.21.15.45:0/2539999691 shutdown_connections 2024-04-16T17:11:22.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.544+0000 7f961f569740 1 --2- 172.21.15.45:0/2539999691 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55597388f410 0x55597387f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:22.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.544+0000 7f961f569740 1 --2- 172.21.15.45:0/2539999691 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555973886830 0x55597388cda0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:22.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.544+0000 7f961f569740 1 --2- 172.21.15.45:0/2539999691 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555973885f10 0x55597389b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:22.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.544+0000 7f961f569740 1 -- 172.21.15.45:0/2539999691 >> 172.21.15.45:0/2539999691 conn(0x555973880640 msgr2=0x55597387ee70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:11:22.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.544+0000 7f961f569740 1 -- 172.21.15.45:0/2539999691 shutdown_connections 2024-04-16T17:11:22.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.544+0000 7f961f569740 1 -- 172.21.15.45:0/2539999691 wait complete. 2024-04-16T17:11:22.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.544+0000 7f961f569740 1 -- start start 2024-04-16T17:11:22.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.545+0000 7f961f569740 1 Processor -- start 2024-04-16T17:11:22.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.545+0000 7f961f569740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555973885f10 0x5559738975d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:22.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.545+0000 7f961f569740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555973886830 0x555973897b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:22.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.546+0000 7f961f569740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55597388f410 0x555973888b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:22.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.546+0000 7f961f569740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f96141bc6b0 con 0x55597388f410 2024-04-16T17:11:22.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.546+0000 7f961f569740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f96141bdb20 con 0x555973885f10 2024-04-16T17:11:22.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.546+0000 7f961f569740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f96141bcd80 con 0x555973886830 2024-04-16T17:11:22.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.546+0000 7f9619e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555973885f10 0x5559738975d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:22.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.546+0000 7f961f569740 10 client.0 _ll_register_callbacks cb 0x7f960801c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:11:22.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.546+0000 7f9619e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555973885f10 0x5559738975d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:41566/0 (socket says 172.21.15.45:41566) 2024-04-16T17:11:22.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.546+0000 7f9619e00640 1 -- 172.21.15.45:0/724317505 learned_addr learned my addr 172.21.15.45:0/724317505 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:11:22.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.546+0000 7f9619400640 1 --2- 172.21.15.45:0/724317505 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555973886830 0x555973897b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:22.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.546+0000 7f961a800640 1 --2- 172.21.15.45:0/724317505 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55597388f410 0x555973888b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:22.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.546+0000 7f9619400640 1 -- 172.21.15.45:0/724317505 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555973885f10 msgr2=0x5559738975d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:11:22.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.546+0000 7f9619400640 1 --2- 172.21.15.45:0/724317505 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555973885f10 0x5559738975d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:22.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.546+0000 7f9619400640 1 -- 172.21.15.45:0/724317505 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55597388f410 msgr2=0x555973888b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:11:22.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.546+0000 7f9619400640 1 --2- 172.21.15.45:0/724317505 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55597388f410 0x555973888b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:22.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.546+0000 7f9619400640 1 -- 172.21.15.45:0/724317505 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5559734fd3f0 con 0x555973886830 2024-04-16T17:11:22.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.547+0000 7f961a800640 1 --2- 172.21.15.45:0/724317505 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55597388f410 0x555973888b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:11:22.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.547+0000 7f9619400640 1 --2- 172.21.15.45:0/724317505 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555973886830 0x555973897b10 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x5559737e7130 tx=0x7f96040056a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1c6a08c8f925c3e6 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:11:22.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.547+0000 7f95f7400640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-04-16T17:11:22.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.547+0000 7f95f7400640 1 -- 172.21.15.45:0/724317505 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f960404b070 con 0x555973886830 2024-04-16T17:11:22.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.547+0000 7f95f7400640 1 -- 172.21.15.45:0/724317505 <== mon.1 v2:172.21.15.117:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f96040ba800 con 0x555973886830 2024-04-16T17:11:22.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.547+0000 7f961f569740 1 -- 172.21.15.45:0/724317505 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9604004c80 con 0x555973886830 2024-04-16T17:11:22.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.547+0000 7f95f7400640 1 -- 172.21.15.45:0/724317505 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f96040bdcd0 con 0x555973886830 2024-04-16T17:11:22.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.547+0000 7f961f569740 10 client.26869 fetch_fsmap learned FSMap version 32 2024-04-16T17:11:22.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.548+0000 7f961f569740 1 -- 172.21.15.45:0/724317505 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5559735c77d0 con 0x555973886830 2024-04-16T17:11:22.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.548+0000 7f95f7400640 1 -- 172.21.15.45:0/724317505 <== mon.1 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9604004c80 con 0x555973886830 2024-04-16T17:11:22.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.548+0000 7f95f7400640 1 -- 172.21.15.45:0/724317505 <== mon.1 v2:172.21.15.117:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f96040f20b0 con 0x555973886830 2024-04-16T17:11:22.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.548+0000 7f961f569740 10 client.26869 fetch_fsmap finished waiting for FSMap version 32 2024-04-16T17:11:22.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.548+0000 7f961f569740 10 client.26869 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:11:22.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.548+0000 7f961f569740 20 client.26869 populate_metadata read hostname 'smithi045' 2024-04-16T17:11:22.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.548+0000 7f961f569740 1 --2- 172.21.15.45:0/724317505 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x5559739058d0 0x555973907cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:22.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.548+0000 7f961f569740 4 client.26869 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T17:11:22.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.548+0000 7f961f569740 1 -- 172.21.15.45:0/724317505 --> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5559737f28b0 con 0x5559739058d0 2024-04-16T17:11:22.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.549+0000 7f9619e00640 1 --2- 172.21.15.45:0/724317505 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x5559739058d0 0x555973907cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:22.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.549+0000 7f9619e00640 1 --2- 172.21.15.45:0/724317505 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x5559739058d0 0x555973907cf0 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=23490a2813500f0e server_cookie=1187fc0da74a1809 in_seq=0 out_seq=0 2024-04-16T17:11:22.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.549+0000 7f95f7400640 10 client.26869 ms_handle_connect on v2:172.21.15.104:6838/3660070592 2024-04-16T17:11:22.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.549+0000 7f95f7400640 1 -- 172.21.15.45:0/724317505 <== mds.0 v2:172.21.15.104:6838/3660070592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5559737f28b0 con 0x5559739058d0 2024-04-16T17:11:22.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.549+0000 7f95f7400640 10 client.26869 handle_command_reply: tid=0 2024-04-16T17:11:22.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.549+0000 7f961f569740 1 client.26869 shutdown 2024-04-16T17:11:22.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.549+0000 7f961f569740 1 -- 172.21.15.45:0/724317505 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555973886830 msgr2=0x555973897b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:22.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.549+0000 7f961f569740 1 --2- 172.21.15.45:0/724317505 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555973886830 0x555973897b10 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x5559737e7130 tx=0x7f96040056a0 comp rx=0 tx=0).stop 2024-04-16T17:11:22.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.550+0000 7f961f569740 1 -- 172.21.15.45:0/724317505 shutdown_connections 2024-04-16T17:11:22.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.550+0000 7f961f569740 1 --2- 172.21.15.45:0/724317505 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x5559739058d0 0x555973907cf0 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:22.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.550+0000 7f961f569740 1 --2- 172.21.15.45:0/724317505 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555973886830 0x555973897b10 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:22.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.550+0000 7f961f569740 1 --2- 172.21.15.45:0/724317505 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55597388f410 0x555973888b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:22.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.550+0000 7f961f569740 1 --2- 172.21.15.45:0/724317505 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555973885f10 0x5559738975d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:22.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.550+0000 7f961f569740 1 -- 172.21.15.45:0/724317505 >> 172.21.15.45:0/724317505 conn(0x555973880640 msgr2=0x55597387f700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:11:22.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.550+0000 7f95f7400640 0 client.26869 ms_handle_reset on v2:172.21.15.104:6838/3660070592 2024-04-16T17:11:22.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.550+0000 7f961f569740 1 -- 172.21.15.45:0/724317505 shutdown_connections 2024-04-16T17:11:22.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.550+0000 7f961f569740 1 -- 172.21.15.45:0/724317505 wait complete. 2024-04-16T17:11:22.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.550+0000 7f961f569740 20 client.26869 trim_cache size 0 max 16384 2024-04-16T17:11:22.553 INFO:teuthology.orchestra.run.smithi045.stdout:[] 2024-04-16T17:11:22.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.551+0000 7f961b200640 1 -- 172.21.15.45:0/2978211883 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f95fc0a3da0 msgr2=0x7f95fc0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:22.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.551+0000 7f961b200640 1 --2- 172.21.15.45:0/2978211883 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f95fc0a3da0 0x7f95fc0a6260 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f9610009270 tx=0x7f96100091e0 comp rx=0 tx=0).stop 2024-04-16T17:11:22.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.551+0000 7f961b200640 1 -- 172.21.15.45:0/2978211883 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f96141071e0 msgr2=0x7f961410c5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:22.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.551+0000 7f961b200640 1 --2- 172.21.15.45:0/2978211883 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f96141071e0 0x7f961410c5f0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f9604036040 tx=0x7f960402cb30 comp rx=0 tx=0).stop 2024-04-16T17:11:22.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.551+0000 7f961b200640 1 -- 172.21.15.45:0/2978211883 shutdown_connections 2024-04-16T17:11:22.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.551+0000 7f961b200640 1 --2- 172.21.15.45:0/2978211883 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f95fc0a3da0 0x7f95fc0a6260 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:22.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.551+0000 7f961b200640 1 --2- 172.21.15.45:0/2978211883 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f96141071e0 0x7f961410c5f0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:22.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.551+0000 7f961b200640 1 --2- 172.21.15.45:0/2978211883 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f961410cb30 0x7f9614072a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:22.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.551+0000 7f961b200640 1 --2- 172.21.15.45:0/2978211883 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9614102110 0x7f961410c0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:22.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.551+0000 7f961b200640 1 -- 172.21.15.45:0/2978211883 >> 172.21.15.45:0/2978211883 conn(0x7f96140fd9d0 msgr2=0x7f96140fed80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:11:22.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.551+0000 7f961b200640 1 -- 172.21.15.45:0/2978211883 shutdown_connections 2024-04-16T17:11:22.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:22.551+0000 7f961b200640 1 -- 172.21.15.45:0/2978211883 wait complete. 2024-04-16T17:11:22.677 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:11:22 smithi104 ceph-mon[36457]: pgmap v141: 129 pgs: 129 active+clean; 40 MiB data, 527 MiB used, 1.0 TiB / 1.0 TiB avail; 602 KiB/s rd, 731 KiB/s wr, 84 op/s 2024-04-16T17:11:22.677 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:11:22 smithi104 ceph-mon[36457]: from='client.? 172.21.15.45:0/3816587143' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:11:22.804 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:11:22 smithi117 ceph-mon[36730]: pgmap v141: 129 pgs: 129 active+clean; 40 MiB data, 527 MiB used, 1.0 TiB / 1.0 TiB avail; 602 KiB/s rd, 731 KiB/s wr, 84 op/s 2024-04-16T17:11:22.804 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:11:22 smithi117 ceph-mon[36730]: from='client.? 172.21.15.45:0/3816587143' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:11:23.249 INFO:tasks.workunit.client.0.smithi045.stdout: 1 65325 1.42 MB/sec execute 85 sec latency 3742.708 ms 2024-04-16T17:11:23.569 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-16T17:11:23.569 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T17:11:23.569 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-16T17:11:23.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:23.778+0000 7f4db6c00640 1 -- 172.21.15.45:0/735956669 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4db0102110 msgr2=0x7f4db01024f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:23.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:23.778+0000 7f4db6c00640 1 --2- 172.21.15.45:0/735956669 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4db0102110 0x7f4db01024f0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f4da4009be0 tx=0x7f4da402bef0 comp rx=0 tx=0).stop 2024-04-16T17:11:23.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:23.778+0000 7f4db6c00640 1 -- 172.21.15.45:0/735956669 shutdown_connections 2024-04-16T17:11:23.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:23.778+0000 7f4db6c00640 1 --2- 172.21.15.45:0/735956669 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4db01071e0 0x7f4db010edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:23.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:23.778+0000 7f4db6c00640 1 --2- 172.21.15.45:0/735956669 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4db0102ac0 0x7f4db0106b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:23.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:23.778+0000 7f4db6c00640 1 --2- 172.21.15.45:0/735956669 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4db0102110 0x7f4db01024f0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:23.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:23.778+0000 7f4db6c00640 1 -- 172.21.15.45:0/735956669 >> 172.21.15.45:0/735956669 conn(0x7f4db00fd9d0 msgr2=0x7f4db00ffdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:11:23.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:23.778+0000 7f4db6c00640 1 -- 172.21.15.45:0/735956669 shutdown_connections 2024-04-16T17:11:23.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:23.779+0000 7f4db6c00640 1 -- 172.21.15.45:0/735956669 wait complete. 2024-04-16T17:11:23.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:23.780+0000 7f4db6c00640 1 Processor -- start 2024-04-16T17:11:23.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:23.780+0000 7f4db6c00640 1 -- start start 2024-04-16T17:11:23.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:23.780+0000 7f4db6c00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4db0102110 0x7f4db019e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:23.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:23.780+0000 7f4db6c00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4db01071e0 0x7f4db019e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:23.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:23.780+0000 7f4db6c00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4db01985f0 0x7f4db0198aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:23.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:23.780+0000 7f4db6c00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f4db01122f0 con 0x7f4db01071e0 2024-04-16T17:11:23.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:23.780+0000 7f4db6c00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f4db0112170 con 0x7f4db01985f0 2024-04-16T17:11:23.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:23.780+0000 7f4db6c00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f4db0112470 con 0x7f4db0102110 2024-04-16T17:11:23.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:23.780+0000 7f4db5800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4db0102110 0x7f4db019e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:23.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:23.780+0000 7f4db6200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4db01985f0 0x7f4db0198aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:23.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:23.780+0000 7f4db5800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4db0102110 0x7f4db019e3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:59614/0 (socket says 172.21.15.45:59614) 2024-04-16T17:11:23.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:23.780+0000 7f4db5800640 1 -- 172.21.15.45:0/1004370564 learned_addr learned my addr 172.21.15.45:0/1004370564 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:11:23.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:23.780+0000 7f4db4e00640 1 --2- 172.21.15.45:0/1004370564 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4db01071e0 0x7f4db019e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:23.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:23.781+0000 7f4db5800640 1 -- 172.21.15.45:0/1004370564 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4db01985f0 msgr2=0x7f4db0198aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:23.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:23.781+0000 7f4db5800640 1 --2- 172.21.15.45:0/1004370564 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4db01985f0 0x7f4db0198aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:23.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:23.781+0000 7f4db5800640 1 -- 172.21.15.45:0/1004370564 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4db01071e0 msgr2=0x7f4db019e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:23.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:23.781+0000 7f4db5800640 1 --2- 172.21.15.45:0/1004370564 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4db01071e0 0x7f4db019e930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:23.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:23.781+0000 7f4db5800640 1 -- 172.21.15.45:0/1004370564 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4db0199110 con 0x7f4db0102110 2024-04-16T17:11:23.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:23.781+0000 7f4db6200640 1 --2- 172.21.15.45:0/1004370564 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4db01985f0 0x7f4db0198aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:11:23.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:23.781+0000 7f4db4e00640 1 --2- 172.21.15.45:0/1004370564 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4db01071e0 0x7f4db019e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:11:23.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:23.781+0000 7f4db5800640 1 --2- 172.21.15.45:0/1004370564 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4db0102110 0x7f4db019e3f0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f4da4004af0 tx=0x7f4da40336b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:11:23.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:23.782+0000 7f4dae000640 1 -- 172.21.15.45:0/1004370564 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4da4043020 con 0x7f4db0102110 2024-04-16T17:11:23.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:23.782+0000 7f4dae000640 1 -- 172.21.15.45:0/1004370564 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4da4003e50 con 0x7f4db0102110 2024-04-16T17:11:23.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:23.782+0000 7f4dae000640 1 -- 172.21.15.45:0/1004370564 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4da403e6b0 con 0x7f4db0102110 2024-04-16T17:11:23.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:23.782+0000 7f4db6c00640 1 -- 172.21.15.45:0/1004370564 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4db01993a0 con 0x7f4db0102110 2024-04-16T17:11:23.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:23.782+0000 7f4db6c00640 1 -- 172.21.15.45:0/1004370564 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4db01a55f0 con 0x7f4db0102110 2024-04-16T17:11:23.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:23.786+0000 7f4dbafd7740 1 Processor -- start 2024-04-16T17:11:23.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:23.787+0000 7f4dbafd7740 1 -- start start 2024-04-16T17:11:23.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:23.787+0000 7f4dae000640 1 -- 172.21.15.45:0/1004370564 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f4da4046020 con 0x7f4db0102110 2024-04-16T17:11:23.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:23.787+0000 7f4dbafd7740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55575a906e90 0x55575a907270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:23.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:23.787+0000 7f4dbafd7740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55575a9077b0 0x55575a90fe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:23.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:23.787+0000 7f4dbafd7740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55575a910390 0x55575a912780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:23.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:23.787+0000 7f4dbafd7740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55575a86bcb0 con 0x55575a9077b0 2024-04-16T17:11:23.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:23.787+0000 7f4dbafd7740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55575a812370 con 0x55575a906e90 2024-04-16T17:11:23.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:23.787+0000 7f4dbafd7740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55575a867c70 con 0x55575a910390 2024-04-16T17:11:23.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:23.787+0000 7f4db6200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55575a906e90 0x55575a907270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:23.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:23.787+0000 7f4db6200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55575a906e90 0x55575a907270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:41590/0 (socket says 172.21.15.45:41590) 2024-04-16T17:11:23.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:23.787+0000 7f4db6200640 1 -- 172.21.15.45:0/3729929079 learned_addr learned my addr 172.21.15.45:0/3729929079 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:11:23.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:23.788+0000 7f4db5800640 1 --2- 172.21.15.45:0/3729929079 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55575a9077b0 0x55575a90fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:23.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:23.788+0000 7f4db4e00640 1 --2- 172.21.15.45:0/3729929079 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55575a910390 0x55575a912780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:23.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:23.788+0000 7f4db6200640 1 -- 172.21.15.45:0/3729929079 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55575a910390 msgr2=0x55575a912780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:11:23.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:23.788+0000 7f4db6200640 1 --2- 172.21.15.45:0/3729929079 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55575a910390 0x55575a912780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:23.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:23.788+0000 7f4db6200640 1 -- 172.21.15.45:0/3729929079 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55575a9077b0 msgr2=0x55575a90fe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:11:23.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:23.788+0000 7f4db6200640 1 --2- 172.21.15.45:0/3729929079 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55575a9077b0 0x55575a90fe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:23.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:23.788+0000 7f4db6200640 1 -- 172.21.15.45:0/3729929079 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55575a831910 con 0x55575a906e90 2024-04-16T17:11:23.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:23.789+0000 7f4db4e00640 1 --2- 172.21.15.45:0/3729929079 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55575a910390 0x55575a912780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:11:23.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:23.789+0000 7f4db6200640 1 --2- 172.21.15.45:0/3729929079 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55575a906e90 0x55575a907270 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f4da8011440 tx=0x7f4da8011900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a2729b5f9aa57bed server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:11:23.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:23.789+0000 7f4da2a00640 1 -- 172.21.15.45:0/3729929079 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4da8004020 con 0x55575a906e90 2024-04-16T17:11:23.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:23.789+0000 7f4dae000640 1 --2- 172.21.15.45:0/1004370564 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f4d980a3de0 0x7f4d980a62a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:23.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:23.789+0000 7f4da2a00640 1 -- 172.21.15.45:0/3729929079 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4da80041c0 con 0x55575a906e90 2024-04-16T17:11:23.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:23.789+0000 7f4dae000640 1 -- 172.21.15.45:0/1004370564 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f4da40f30b0 con 0x7f4db0102110 2024-04-16T17:11:23.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:23.790+0000 7f4da2a00640 1 -- 172.21.15.45:0/3729929079 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4da800f8d0 con 0x55575a906e90 2024-04-16T17:11:23.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:23.790+0000 7f4dbafd7740 1 -- 172.21.15.45:0/3729929079 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55575a906e90 msgr2=0x55575a907270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:23.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:23.790+0000 7f4dbafd7740 1 --2- 172.21.15.45:0/3729929079 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55575a906e90 0x55575a907270 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f4da8011440 tx=0x7f4da8011900 comp rx=0 tx=0).stop 2024-04-16T17:11:23.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:23.790+0000 7f4db5800640 1 --2- 172.21.15.45:0/1004370564 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f4d980a3de0 0x7f4d980a62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:23.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:23.790+0000 7f4dbafd7740 1 -- 172.21.15.45:0/3729929079 shutdown_connections 2024-04-16T17:11:23.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:23.790+0000 7f4dbafd7740 1 --2- 172.21.15.45:0/3729929079 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55575a910390 0x55575a912780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:23.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:23.790+0000 7f4dbafd7740 1 --2- 172.21.15.45:0/3729929079 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55575a9077b0 0x55575a90fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:23.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:23.790+0000 7f4dbafd7740 1 --2- 172.21.15.45:0/3729929079 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55575a906e90 0x55575a907270 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:23.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:23.790+0000 7f4dbafd7740 1 -- 172.21.15.45:0/3729929079 >> 172.21.15.45:0/3729929079 conn(0x55575a9015c0 msgr2=0x55575a9019a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:11:23.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:23.790+0000 7f4dbafd7740 1 -- 172.21.15.45:0/3729929079 shutdown_connections 2024-04-16T17:11:23.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:23.790+0000 7f4dbafd7740 1 -- 172.21.15.45:0/3729929079 wait complete. 2024-04-16T17:11:23.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:23.791+0000 7f4dbafd7740 1 -- start start 2024-04-16T17:11:23.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:23.791+0000 7f4db5800640 1 --2- 172.21.15.45:0/1004370564 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f4d980a3de0 0x7f4d980a62a0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x55575a906960 tx=0x7f4da40f6b60 comp rx=0 tx=0).ready entity=mgr.25411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:11:23.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:23.791+0000 7f4dbafd7740 1 Processor -- start 2024-04-16T17:11:23.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:23.792+0000 7f4dbafd7740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55575a906e90 0x55575a90d330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:23.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:23.792+0000 7f4dbafd7740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55575a9077b0 0x55575a90d870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:23.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:23.792+0000 7f4dbafd7740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55575a910390 0x55575a90ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:23.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:23.792+0000 7f4dbafd7740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55575a917630 con 0x55575a910390 2024-04-16T17:11:23.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:23.792+0000 7f4dbafd7740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55575a825730 con 0x55575a906e90 2024-04-16T17:11:23.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:23.792+0000 7f4dbafd7740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55575a9177b0 con 0x55575a9077b0 2024-04-16T17:11:23.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:23.792+0000 7f4dbafd7740 10 client.0 _ll_register_callbacks cb 0x55575a7f41e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:11:23.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:23.792+0000 7f4db4e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55575a9077b0 0x55575a90d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:23.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:23.792+0000 7f4db6200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55575a910390 0x55575a90ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:23.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:23.792+0000 7f4db4e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55575a9077b0 0x55575a90d870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:59634/0 (socket says 172.21.15.45:59634) 2024-04-16T17:11:23.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:23.792+0000 7f4db4e00640 1 -- 172.21.15.45:0/934487555 learned_addr learned my addr 172.21.15.45:0/934487555 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:11:23.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:23.792+0000 7f4db6200640 1 --2- 172.21.15.45:0/934487555 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55575a906e90 0x55575a90d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:23.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:23.793+0000 7f4db4e00640 1 -- 172.21.15.45:0/934487555 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55575a906e90 msgr2=0x55575a90d330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:11:23.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:23.793+0000 7f4db4e00640 1 --2- 172.21.15.45:0/934487555 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55575a906e90 0x55575a90d330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:23.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:23.793+0000 7f4db4e00640 1 -- 172.21.15.45:0/934487555 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55575a910390 msgr2=0x55575a90ddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:11:23.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:23.793+0000 7f4db4e00640 1 --2- 172.21.15.45:0/934487555 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55575a910390 0x55575a90ddb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:23.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:23.793+0000 7f4db4e00640 1 -- 172.21.15.45:0/934487555 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55575a6467e0 con 0x55575a9077b0 2024-04-16T17:11:23.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:23.793+0000 7f4db6200640 1 --2- 172.21.15.45:0/934487555 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55575a910390 0x55575a90ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:11:23.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:23.793+0000 7f4db4e00640 1 --2- 172.21.15.45:0/934487555 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55575a9077b0 0x55575a90d870 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f4d9c012cf0 tx=0x7f4d9c0107a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dfdb72928113cd9b server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:11:23.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:23.793+0000 7f4d97400640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-04-16T17:11:23.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:23.793+0000 7f4d97400640 1 -- 172.21.15.45:0/934487555 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d9c005220 con 0x55575a9077b0 2024-04-16T17:11:23.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:23.794+0000 7f4dbafd7740 1 -- 172.21.15.45:0/934487555 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4da80041c0 con 0x55575a9077b0 2024-04-16T17:11:23.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:23.794+0000 7f4d97400640 1 -- 172.21.15.45:0/934487555 <== mon.1 v2:172.21.15.117:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f4d9c0053c0 con 0x55575a9077b0 2024-04-16T17:11:23.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:23.794+0000 7f4d97400640 1 -- 172.21.15.45:0/934487555 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f4d9c0055c0 con 0x55575a9077b0 2024-04-16T17:11:23.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:23.794+0000 7f4dbafd7740 10 client.26878 fetch_fsmap learned FSMap version 32 2024-04-16T17:11:23.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:23.794+0000 7f4dbafd7740 1 -- 172.21.15.45:0/934487555 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55575a6a6f70 con 0x55575a9077b0 2024-04-16T17:11:23.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:23.794+0000 7f4d97400640 1 -- 172.21.15.45:0/934487555 <== mon.1 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4da80041c0 con 0x55575a9077b0 2024-04-16T17:11:23.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:23.794+0000 7f4d97400640 1 -- 172.21.15.45:0/934487555 <== mon.1 v2:172.21.15.117:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f4d9c01c990 con 0x55575a9077b0 2024-04-16T17:11:23.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:23.795+0000 7f4dbafd7740 10 client.26878 fetch_fsmap finished waiting for FSMap version 32 2024-04-16T17:11:23.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:23.795+0000 7f4dbafd7740 10 client.26878 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:11:23.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:23.795+0000 7f4dbafd7740 20 client.26878 populate_metadata read hostname 'smithi045' 2024-04-16T17:11:23.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:23.795+0000 7f4dbafd7740 1 --2- 172.21.15.45:0/934487555 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55575a924ce0 0x55575a927100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:23.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:23.795+0000 7f4dbafd7740 4 client.26878 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:11:23.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:23.795+0000 7f4dbafd7740 1 -- 172.21.15.45:0/934487555 --> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55575a8796d0 con 0x55575a924ce0 2024-04-16T17:11:23.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:23.795+0000 7f4db6200640 1 --2- 172.21.15.45:0/934487555 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55575a924ce0 0x55575a927100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:23.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:23.796+0000 7f4db6200640 1 --2- 172.21.15.45:0/934487555 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55575a924ce0 0x55575a927100 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ebc2e5bae1e8dd06 server_cookie=b4df34fa00a404fc in_seq=0 out_seq=0 2024-04-16T17:11:23.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:23.796+0000 7f4d97400640 10 client.26878 ms_handle_connect on v2:172.21.15.104:6838/3660070592 2024-04-16T17:11:23.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:23.796+0000 7f4d97400640 1 -- 172.21.15.45:0/934487555 <== mds.0 v2:172.21.15.104:6838/3660070592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55575a8796d0 con 0x55575a924ce0 2024-04-16T17:11:23.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:23.796+0000 7f4d97400640 10 client.26878 handle_command_reply: tid=0 2024-04-16T17:11:23.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:23.796+0000 7f4dbafd7740 1 client.26878 shutdown 2024-04-16T17:11:23.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:23.797+0000 7f4dbafd7740 1 -- 172.21.15.45:0/934487555 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55575a9077b0 msgr2=0x55575a90d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:23.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:23.797+0000 7f4dbafd7740 1 --2- 172.21.15.45:0/934487555 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55575a9077b0 0x55575a90d870 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f4d9c012cf0 tx=0x7f4d9c0107a0 comp rx=0 tx=0).stop 2024-04-16T17:11:23.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:23.797+0000 7f4dbafd7740 1 -- 172.21.15.45:0/934487555 shutdown_connections 2024-04-16T17:11:23.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:23.797+0000 7f4dbafd7740 1 --2- 172.21.15.45:0/934487555 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55575a924ce0 0x55575a927100 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:23.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:23.797+0000 7f4dbafd7740 1 --2- 172.21.15.45:0/934487555 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55575a9077b0 0x55575a90d870 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:23.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:23.797+0000 7f4d97400640 0 client.26878 ms_handle_reset on v2:172.21.15.104:6838/3660070592 2024-04-16T17:11:23.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:23.797+0000 7f4dbafd7740 1 --2- 172.21.15.45:0/934487555 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55575a910390 0x55575a90ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:23.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:23.797+0000 7f4dbafd7740 1 --2- 172.21.15.45:0/934487555 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55575a906e90 0x55575a90d330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:23.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:23.797+0000 7f4dbafd7740 1 -- 172.21.15.45:0/934487555 >> 172.21.15.45:0/934487555 conn(0x55575a9015c0 msgr2=0x55575a903310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:11:23.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:23.797+0000 7f4dbafd7740 1 -- 172.21.15.45:0/934487555 shutdown_connections 2024-04-16T17:11:23.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:23.797+0000 7f4dbafd7740 1 -- 172.21.15.45:0/934487555 wait complete. 2024-04-16T17:11:23.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:23.798+0000 7f4dbafd7740 20 client.26878 trim_cache size 0 max 16384 2024-04-16T17:11:23.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:23.827+0000 7f4dbafd7740 1 Processor -- start 2024-04-16T17:11:23.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:23.827+0000 7f4dbafd7740 1 -- start start 2024-04-16T17:11:23.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:23.828+0000 7f4dbafd7740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55575a906e90 0x55575a90c270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:23.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:23.828+0000 7f4dbafd7740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55575a9077b0 0x55575a9205a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:23.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:23.828+0000 7f4dbafd7740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55575a910390 0x55575a92c990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:23.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:23.828+0000 7f4dbafd7740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f4db01af920 con 0x55575a906e90 2024-04-16T17:11:23.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:23.828+0000 7f4dbafd7740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f4db01ad160 con 0x55575a910390 2024-04-16T17:11:23.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:23.828+0000 7f4dbafd7740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f4db01aaf90 con 0x55575a9077b0 2024-04-16T17:11:23.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:23.828+0000 7f4db4e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55575a9077b0 0x55575a9205a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:23.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:23.828+0000 7f4db6200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55575a906e90 0x55575a90c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:23.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:23.828+0000 7f4db4e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55575a9077b0 0x55575a9205a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:59648/0 (socket says 172.21.15.45:59648) 2024-04-16T17:11:23.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:23.828+0000 7f4db4e00640 1 -- 172.21.15.45:0/1227959487 learned_addr learned my addr 172.21.15.45:0/1227959487 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:11:23.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:23.828+0000 7f4db6200640 1 --2- 172.21.15.45:0/1227959487 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55575a910390 0x55575a92c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:23.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:23.829+0000 7f4db4e00640 1 -- 172.21.15.45:0/1227959487 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55575a910390 msgr2=0x55575a92c990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:11:23.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:23.829+0000 7f4db4e00640 1 --2- 172.21.15.45:0/1227959487 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55575a910390 0x55575a92c990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:23.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:23.829+0000 7f4db4e00640 1 -- 172.21.15.45:0/1227959487 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55575a906e90 msgr2=0x55575a90c270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:11:23.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:23.829+0000 7f4db4e00640 1 --2- 172.21.15.45:0/1227959487 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55575a906e90 0x55575a90c270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:23.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:23.829+0000 7f4db4e00640 1 -- 172.21.15.45:0/1227959487 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55575a9075e0 con 0x55575a9077b0 2024-04-16T17:11:23.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:23.829+0000 7f4db6200640 1 --2- 172.21.15.45:0/1227959487 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55575a910390 0x55575a92c990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:11:23.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:23.829+0000 7f4db4e00640 1 --2- 172.21.15.45:0/1227959487 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55575a9077b0 0x55575a9205a0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f4d9c011010 tx=0x7f4d9c010d20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=439af4b856063121 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:11:23.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:23.829+0000 7f4da2000640 1 -- 172.21.15.45:0/1227959487 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d9c0052d0 con 0x55575a9077b0 2024-04-16T17:11:23.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:23.829+0000 7f4da2000640 1 -- 172.21.15.45:0/1227959487 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d9c01c8e0 con 0x55575a9077b0 2024-04-16T17:11:23.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:23.829+0000 7f4da2000640 1 -- 172.21.15.45:0/1227959487 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d9c003ca0 con 0x55575a9077b0 2024-04-16T17:11:23.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:23.830+0000 7f4dbafd7740 1 -- 172.21.15.45:0/1227959487 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55575a9077b0 msgr2=0x55575a9205a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:23.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:23.830+0000 7f4dbafd7740 1 --2- 172.21.15.45:0/1227959487 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55575a9077b0 0x55575a9205a0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f4d9c011010 tx=0x7f4d9c010d20 comp rx=0 tx=0).stop 2024-04-16T17:11:23.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:23.830+0000 7f4dbafd7740 1 -- 172.21.15.45:0/1227959487 shutdown_connections 2024-04-16T17:11:23.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:23.830+0000 7f4dbafd7740 1 --2- 172.21.15.45:0/1227959487 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55575a9077b0 0x55575a9205a0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:23.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:23.830+0000 7f4dbafd7740 1 --2- 172.21.15.45:0/1227959487 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55575a910390 0x55575a92c990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:23.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:23.830+0000 7f4dbafd7740 1 --2- 172.21.15.45:0/1227959487 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55575a906e90 0x55575a90c270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:23.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:23.830+0000 7f4dbafd7740 1 -- 172.21.15.45:0/1227959487 >> 172.21.15.45:0/1227959487 conn(0x55575a9015c0 msgr2=0x55575a900560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:11:23.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:23.830+0000 7f4dbafd7740 1 -- 172.21.15.45:0/1227959487 shutdown_connections 2024-04-16T17:11:23.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:23.830+0000 7f4dbafd7740 1 -- 172.21.15.45:0/1227959487 wait complete. 2024-04-16T17:11:23.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:23.831+0000 7f4dbafd7740 1 -- start start 2024-04-16T17:11:23.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:23.831+0000 7f4dbafd7740 1 Processor -- start 2024-04-16T17:11:23.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:23.832+0000 7f4dbafd7740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55575a906e90 0x55575a918590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:23.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:23.832+0000 7f4dbafd7740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55575a9077b0 0x55575a918ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:23.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:23.832+0000 7f4dbafd7740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55575a910390 0x55575a919010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:23.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:23.832+0000 7f4dbafd7740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f4db01ab620 con 0x55575a906e90 2024-04-16T17:11:23.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:23.832+0000 7f4dbafd7740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f4db01aca90 con 0x55575a9077b0 2024-04-16T17:11:23.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:23.832+0000 7f4dbafd7740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f4db01abcf0 con 0x55575a910390 2024-04-16T17:11:23.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:23.832+0000 7f4dbafd7740 10 client.0 _ll_register_callbacks cb 0x7f4d9c012930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:11:23.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:23.832+0000 7f4db6200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55575a906e90 0x55575a918590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:23.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:23.832+0000 7f4db6200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55575a906e90 0x55575a918590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55208/0 (socket says 172.21.15.45:55208) 2024-04-16T17:11:23.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:23.832+0000 7f4db6200640 1 -- 172.21.15.45:0/2659890867 learned_addr learned my addr 172.21.15.45:0/2659890867 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:11:23.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:23.832+0000 7f4db6200640 1 --2- 172.21.15.45:0/2659890867 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55575a910390 0x55575a919010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:23.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:23.833+0000 7f4db6200640 1 -- 172.21.15.45:0/2659890867 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55575a910390 msgr2=0x55575a919010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:11:23.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:23.833+0000 7f4db6200640 1 --2- 172.21.15.45:0/2659890867 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55575a910390 0x55575a919010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:23.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:23.833+0000 7f4db6200640 1 -- 172.21.15.45:0/2659890867 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55575a9077b0 msgr2=0x55575a918ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:11:23.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:23.833+0000 7f4db6200640 1 --2- 172.21.15.45:0/2659890867 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55575a9077b0 0x55575a918ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:23.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:23.833+0000 7f4db6200640 1 -- 172.21.15.45:0/2659890867 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55575a53e170 con 0x55575a906e90 2024-04-16T17:11:23.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:23.833+0000 7f4db6200640 1 --2- 172.21.15.45:0/2659890867 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55575a906e90 0x55575a918590 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f4da8009710 tx=0x7f4da8011900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a42b94c3c1e1f64 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:11:23.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:23.833+0000 7f4d97400640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-16T17:11:23.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:23.833+0000 7f4d97400640 1 -- 172.21.15.45:0/2659890867 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4da801d040 con 0x55575a906e90 2024-04-16T17:11:23.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:23.834+0000 7f4d97400640 1 -- 172.21.15.45:0/2659890867 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f4da8005040 con 0x55575a906e90 2024-04-16T17:11:23.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:23.834+0000 7f4dbafd7740 1 -- 172.21.15.45:0/2659890867 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4d9c01c8e0 con 0x55575a906e90 2024-04-16T17:11:23.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:23.834+0000 7f4d97400640 1 -- 172.21.15.45:0/2659890867 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f4da803e050 con 0x55575a906e90 2024-04-16T17:11:23.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:23.834+0000 7f4dbafd7740 10 client.16782 fetch_fsmap learned FSMap version 32 2024-04-16T17:11:23.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:23.834+0000 7f4dbafd7740 1 -- 172.21.15.45:0/2659890867 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55575a98b380 con 0x55575a906e90 2024-04-16T17:11:23.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:23.834+0000 7f4d97400640 1 -- 172.21.15.45:0/2659890867 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d9c01c8e0 con 0x55575a906e90 2024-04-16T17:11:23.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:23.834+0000 7f4d97400640 1 -- 172.21.15.45:0/2659890867 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f4da802c6d0 con 0x55575a906e90 2024-04-16T17:11:23.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:23.835+0000 7f4dbafd7740 10 client.16782 fetch_fsmap finished waiting for FSMap version 32 2024-04-16T17:11:23.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:23.835+0000 7f4dbafd7740 10 client.16782 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:11:23.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:23.835+0000 7f4dbafd7740 20 client.16782 populate_metadata read hostname 'smithi045' 2024-04-16T17:11:23.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:23.835+0000 7f4dbafd7740 1 --2- 172.21.15.45:0/2659890867 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55575a98b8f0 0x55575a98dd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:23.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:23.835+0000 7f4dbafd7740 4 client.16782 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-16T17:11:23.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:23.835+0000 7f4dbafd7740 1 -- 172.21.15.45:0/2659890867 --> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55575a873540 con 0x55575a98b8f0 2024-04-16T17:11:23.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:23.835+0000 7f4db4e00640 1 --2- 172.21.15.45:0/2659890867 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55575a98b8f0 0x55575a98dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:23.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:23.835+0000 7f4db4e00640 1 --2- 172.21.15.45:0/2659890867 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55575a98b8f0 0x55575a98dd10 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bd3fd0af5c3ef08b server_cookie=b30b8664a71d0b65 in_seq=0 out_seq=0 2024-04-16T17:11:23.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:23.835+0000 7f4d97400640 10 client.16782 ms_handle_connect on v2:172.21.15.104:6838/3660070592 2024-04-16T17:11:23.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:23.836+0000 7f4d97400640 1 -- 172.21.15.45:0/2659890867 <== mds.0 v2:172.21.15.104:6838/3660070592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55575a873540 con 0x55575a98b8f0 2024-04-16T17:11:23.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:23.836+0000 7f4d97400640 10 client.16782 handle_command_reply: tid=0 2024-04-16T17:11:23.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:23.836+0000 7f4dbafd7740 1 client.16782 shutdown 2024-04-16T17:11:23.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:23.837+0000 7f4dbafd7740 1 -- 172.21.15.45:0/2659890867 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55575a906e90 msgr2=0x55575a918590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:23.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:23.837+0000 7f4dbafd7740 1 --2- 172.21.15.45:0/2659890867 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55575a906e90 0x55575a918590 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f4da8009710 tx=0x7f4da8011900 comp rx=0 tx=0).stop 2024-04-16T17:11:23.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:23.839+0000 7f4dbafd7740 1 -- 172.21.15.45:0/2659890867 shutdown_connections 2024-04-16T17:11:23.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:23.839+0000 7f4dbafd7740 1 --2- 172.21.15.45:0/2659890867 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55575a98b8f0 0x55575a98dd10 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:23.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:23.839+0000 7f4dbafd7740 1 --2- 172.21.15.45:0/2659890867 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55575a910390 0x55575a919010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:23.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:23.839+0000 7f4dbafd7740 1 --2- 172.21.15.45:0/2659890867 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55575a9077b0 0x55575a918ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:23.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:23.839+0000 7f4dbafd7740 1 --2- 172.21.15.45:0/2659890867 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55575a906e90 0x55575a918590 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:23.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:23.839+0000 7f4dbafd7740 1 -- 172.21.15.45:0/2659890867 >> 172.21.15.45:0/2659890867 conn(0x55575a9015c0 msgr2=0x55575a90d2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:11:23.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:23.839+0000 7f4d97400640 0 client.16782 ms_handle_reset on v2:172.21.15.104:6838/3660070592 2024-04-16T17:11:23.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:23.839+0000 7f4dbafd7740 1 -- 172.21.15.45:0/2659890867 shutdown_connections 2024-04-16T17:11:23.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:23.839+0000 7f4dbafd7740 1 -- 172.21.15.45:0/2659890867 wait complete. 2024-04-16T17:11:23.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:23.839+0000 7f4dbafd7740 20 client.16782 trim_cache size 0 max 16384 2024-04-16T17:11:23.841 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-16T17:11:23.841 INFO:teuthology.orchestra.run.smithi045.stdout: "return_code": 0, 2024-04-16T17:11:23.841 INFO:teuthology.orchestra.run.smithi045.stdout: "scrub_tag": "e99870ef-51e6-4d3c-a90a-ccff17939e5a", 2024-04-16T17:11:23.841 INFO:teuthology.orchestra.run.smithi045.stdout: "mode": "asynchronous" 2024-04-16T17:11:23.841 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-16T17:11:23.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:23.840+0000 7f4db6c00640 1 -- 172.21.15.45:0/1004370564 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f4d980a3de0 msgr2=0x7f4d980a62a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:23.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:23.840+0000 7f4db6c00640 1 --2- 172.21.15.45:0/1004370564 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f4d980a3de0 0x7f4d980a62a0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x55575a906960 tx=0x7f4da40f6b60 comp rx=0 tx=0).stop 2024-04-16T17:11:23.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:23.840+0000 7f4db6c00640 1 -- 172.21.15.45:0/1004370564 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4db0102110 msgr2=0x7f4db019e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:23.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:23.840+0000 7f4db6c00640 1 --2- 172.21.15.45:0/1004370564 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4db0102110 0x7f4db019e3f0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f4da4004af0 tx=0x7f4da40336b0 comp rx=0 tx=0).stop 2024-04-16T17:11:23.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:23.840+0000 7f4db6c00640 1 -- 172.21.15.45:0/1004370564 shutdown_connections 2024-04-16T17:11:23.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:23.840+0000 7f4db6c00640 1 --2- 172.21.15.45:0/1004370564 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f4d980a3de0 0x7f4d980a62a0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:23.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:23.840+0000 7f4db6c00640 1 --2- 172.21.15.45:0/1004370564 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4db01985f0 0x7f4db0198aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:23.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:23.840+0000 7f4db6c00640 1 --2- 172.21.15.45:0/1004370564 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4db01071e0 0x7f4db019e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:23.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:23.840+0000 7f4db6c00640 1 --2- 172.21.15.45:0/1004370564 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4db0102110 0x7f4db019e3f0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:23.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:23.840+0000 7f4db6c00640 1 -- 172.21.15.45:0/1004370564 >> 172.21.15.45:0/1004370564 conn(0x7f4db00fd9d0 msgr2=0x7f4db00ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:11:23.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:23.841+0000 7f4db6c00640 1 -- 172.21.15.45:0/1004370564 shutdown_connections 2024-04-16T17:11:23.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:23.841+0000 7f4db6c00640 1 -- 172.21.15.45:0/1004370564 wait complete. 2024-04-16T17:11:23.864 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-16T17:11:23.864 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-16T17:11:24.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:24.058+0000 7f35e4600640 1 -- 172.21.15.45:0/2061283651 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f35dc1044b0 msgr2=0x7f35dc104890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:24.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:24.058+0000 7f35e4600640 1 --2- 172.21.15.45:0/2061283651 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f35dc1044b0 0x7f35dc104890 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f35d0009900 tx=0x7f35d002bc50 comp rx=0 tx=0).stop 2024-04-16T17:11:24.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:24.059+0000 7f35e4600640 1 -- 172.21.15.45:0/2061283651 shutdown_connections 2024-04-16T17:11:24.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:24.059+0000 7f35e4600640 1 --2- 172.21.15.45:0/2061283651 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f35dc104e60 0x7f35dc108ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:24.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:24.059+0000 7f35e4600640 1 --2- 172.21.15.45:0/2061283651 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f35dc109580 0x7f35dc111150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:24.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:24.059+0000 7f35e4600640 1 --2- 172.21.15.45:0/2061283651 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f35dc1044b0 0x7f35dc104890 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:24.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:24.059+0000 7f35e4600640 1 -- 172.21.15.45:0/2061283651 >> 172.21.15.45:0/2061283651 conn(0x7f35dc0ffd90 msgr2=0x7f35dc1021b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:11:24.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:24.059+0000 7f35e4600640 1 -- 172.21.15.45:0/2061283651 shutdown_connections 2024-04-16T17:11:24.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:24.059+0000 7f35e4600640 1 -- 172.21.15.45:0/2061283651 wait complete. 2024-04-16T17:11:24.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:24.060+0000 7f35e4600640 1 Processor -- start 2024-04-16T17:11:24.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:24.060+0000 7f35e4600640 1 -- start start 2024-04-16T17:11:24.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:24.060+0000 7f35e4600640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f35dc1044b0 0x7f35dc0738b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:24.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:24.060+0000 7f35e4600640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f35dc109580 0x7f35dc06c980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:24.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:24.060+0000 7f35e4600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f35dc06cec0 0x7f35dc06d370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:24.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:24.060+0000 7f35e4600640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f35dc1146e0 con 0x7f35dc06cec0 2024-04-16T17:11:24.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:24.060+0000 7f35e4600640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f35dc114560 con 0x7f35dc109580 2024-04-16T17:11:24.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:24.060+0000 7f35e4600640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f35dc114860 con 0x7f35dc1044b0 2024-04-16T17:11:24.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:24.061+0000 7f35e1c00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f35dc1044b0 0x7f35dc0738b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:24.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:24.061+0000 7f35e2600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f35dc06cec0 0x7f35dc06d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:24.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:24.061+0000 7f35e1200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f35dc109580 0x7f35dc06c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:24.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:24.061+0000 7f35e1c00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f35dc1044b0 0x7f35dc0738b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:59660/0 (socket says 172.21.15.45:59660) 2024-04-16T17:11:24.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:24.061+0000 7f35e1c00640 1 -- 172.21.15.45:0/1536690963 learned_addr learned my addr 172.21.15.45:0/1536690963 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:11:24.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:24.061+0000 7f35e2600640 1 -- 172.21.15.45:0/1536690963 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f35dc1044b0 msgr2=0x7f35dc0738b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:24.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:24.061+0000 7f35e2600640 1 --2- 172.21.15.45:0/1536690963 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f35dc1044b0 0x7f35dc0738b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:24.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:24.061+0000 7f35e2600640 1 -- 172.21.15.45:0/1536690963 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f35dc109580 msgr2=0x7f35dc06c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:24.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:24.061+0000 7f35e2600640 1 --2- 172.21.15.45:0/1536690963 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f35dc109580 0x7f35dc06c980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:24.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:24.061+0000 7f35e2600640 1 -- 172.21.15.45:0/1536690963 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35dc0713d0 con 0x7f35dc06cec0 2024-04-16T17:11:24.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:24.061+0000 7f35e1c00640 1 --2- 172.21.15.45:0/1536690963 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f35dc1044b0 0x7f35dc0738b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:11:24.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:24.062+0000 7f35e2600640 1 --2- 172.21.15.45:0/1536690963 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f35dc06cec0 0x7f35dc06d370 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f35d400ec50 tx=0x7f35d400c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:11:24.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:24.062+0000 7f35da000640 1 -- 172.21.15.45:0/1536690963 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35d4003b70 con 0x7f35dc06cec0 2024-04-16T17:11:24.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:24.062+0000 7f35e4600640 1 -- 172.21.15.45:0/1536690963 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35dc0716c0 con 0x7f35dc06cec0 2024-04-16T17:11:24.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:24.062+0000 7f35e4600640 1 -- 172.21.15.45:0/1536690963 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35dc071c10 con 0x7f35dc06cec0 2024-04-16T17:11:24.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:24.062+0000 7f35da000640 1 -- 172.21.15.45:0/1536690963 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f35d400ee90 con 0x7f35dc06cec0 2024-04-16T17:11:24.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:24.062+0000 7f35da000640 1 -- 172.21.15.45:0/1536690963 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35d40107a0 con 0x7f35dc06cec0 2024-04-16T17:11:24.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:24.067+0000 7f35e738b740 1 Processor -- start 2024-04-16T17:11:24.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:24.068+0000 7f35da000640 1 -- 172.21.15.45:0/1536690963 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f35d4004020 con 0x7f35dc06cec0 2024-04-16T17:11:24.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:24.068+0000 7f35e738b740 1 -- start start 2024-04-16T17:11:24.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:24.069+0000 7f35e738b740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55798e1c19c0 0x55798e1c1da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:24.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:24.069+0000 7f35e738b740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55798e1c22e0 0x55798e1ca980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:24.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:24.069+0000 7f35e738b740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55798e1caec0 0x55798e1cd2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:24.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:24.069+0000 7f35e738b740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55798e11f120 con 0x55798e1c22e0 2024-04-16T17:11:24.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:24.069+0000 7f35e738b740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55798e0c5340 con 0x55798e1c19c0 2024-04-16T17:11:24.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:24.069+0000 7f35e738b740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55798e11b0e0 con 0x55798e1caec0 2024-04-16T17:11:24.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:24.069+0000 7f35e1200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55798e1c19c0 0x55798e1c1da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:24.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:24.069+0000 7f35e1200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55798e1c19c0 0x55798e1c1da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:41634/0 (socket says 172.21.15.45:41634) 2024-04-16T17:11:24.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:24.069+0000 7f35e1200640 1 -- 172.21.15.45:0/2254125781 learned_addr learned my addr 172.21.15.45:0/2254125781 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:11:24.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:24.069+0000 7f35e1c00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55798e1c22e0 0x55798e1ca980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:24.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:24.069+0000 7f35e1200640 1 --2- 172.21.15.45:0/2254125781 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55798e1caec0 0x55798e1cd2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:24.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:24.069+0000 7f35e1200640 1 -- 172.21.15.45:0/2254125781 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55798e1caec0 msgr2=0x55798e1cd2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:11:24.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:24.069+0000 7f35e1200640 1 --2- 172.21.15.45:0/2254125781 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55798e1caec0 0x55798e1cd2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:24.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:24.069+0000 7f35da000640 1 --2- 172.21.15.45:0/1536690963 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f35c40a3da0 0x7f35c40a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:24.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:24.069+0000 7f35e1200640 1 -- 172.21.15.45:0/2254125781 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55798e1c22e0 msgr2=0x55798e1ca980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:11:24.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:24.069+0000 7f35e1200640 1 --2- 172.21.15.45:0/2254125781 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55798e1c22e0 0x55798e1ca980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:24.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:24.069+0000 7f35e1200640 1 -- 172.21.15.45:0/2254125781 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55798defb7d0 con 0x55798e1c19c0 2024-04-16T17:11:24.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:24.069+0000 7f35da000640 1 -- 172.21.15.45:0/1536690963 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f35d40d1070 con 0x7f35dc06cec0 2024-04-16T17:11:24.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:24.069+0000 7f35e1200640 1 --2- 172.21.15.45:0/2254125781 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55798e1c19c0 0x55798e1c1da0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f35c8009330 tx=0x7f35c80094d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=58f6e43a0e29089e server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:11:24.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:24.069+0000 7f35cea00640 1 -- 172.21.15.45:0/2254125781 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35c8017e60 con 0x55798e1c19c0 2024-04-16T17:11:24.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:24.069+0000 7f35e2600640 1 --2- 172.21.15.45:0/1536690963 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f35c40a3da0 0x7f35c40a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:24.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:24.069+0000 7f35cea00640 1 -- 172.21.15.45:0/2254125781 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f35c801d070 con 0x55798e1c19c0 2024-04-16T17:11:24.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:24.069+0000 7f35cea00640 1 -- 172.21.15.45:0/2254125781 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35c8018b20 con 0x55798e1c19c0 2024-04-16T17:11:24.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:24.072+0000 7f35e738b740 1 -- 172.21.15.45:0/2254125781 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55798e1c19c0 msgr2=0x55798e1c1da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:24.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:24.072+0000 7f35e738b740 1 --2- 172.21.15.45:0/2254125781 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55798e1c19c0 0x55798e1c1da0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f35c8009330 tx=0x7f35c80094d0 comp rx=0 tx=0).stop 2024-04-16T17:11:24.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:24.072+0000 7f35e2600640 1 --2- 172.21.15.45:0/1536690963 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f35c40a3da0 0x7f35c40a6260 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f35d4009bc0 tx=0x7f35d4009b50 comp rx=0 tx=0).ready entity=mgr.25411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:11:24.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:24.072+0000 7f35e738b740 1 -- 172.21.15.45:0/2254125781 shutdown_connections 2024-04-16T17:11:24.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:24.072+0000 7f35e738b740 1 --2- 172.21.15.45:0/2254125781 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55798e1caec0 0x55798e1cd2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:24.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:24.072+0000 7f35e738b740 1 --2- 172.21.15.45:0/2254125781 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55798e1c22e0 0x55798e1ca980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:24.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:24.072+0000 7f35e738b740 1 --2- 172.21.15.45:0/2254125781 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55798e1c19c0 0x55798e1c1da0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:24.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:24.072+0000 7f35e738b740 1 -- 172.21.15.45:0/2254125781 >> 172.21.15.45:0/2254125781 conn(0x55798e1bc0f0 msgr2=0x55798e1bc4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:11:24.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:24.072+0000 7f35e738b740 1 -- 172.21.15.45:0/2254125781 shutdown_connections 2024-04-16T17:11:24.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:24.072+0000 7f35e738b740 1 -- 172.21.15.45:0/2254125781 wait complete. 2024-04-16T17:11:24.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:24.072+0000 7f35e738b740 1 -- start start 2024-04-16T17:11:24.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:24.073+0000 7f35e738b740 1 Processor -- start 2024-04-16T17:11:24.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:24.073+0000 7f35e738b740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55798e1c19c0 0x55798e1c7f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:24.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:24.073+0000 7f35e738b740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55798e1c22e0 0x55798e1c8460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:24.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:24.073+0000 7f35e738b740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55798e1caec0 0x55798e1c89a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:24.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:24.073+0000 7f35e738b740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55798e1cdc50 con 0x55798e1c19c0 2024-04-16T17:11:24.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:24.073+0000 7f35e738b740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55798e0d8710 con 0x55798e1caec0 2024-04-16T17:11:24.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:24.073+0000 7f35e738b740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55798e1cddd0 con 0x55798e1c22e0 2024-04-16T17:11:24.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:24.073+0000 7f35e738b740 10 client.0 _ll_register_callbacks cb 0x55798e0d3af0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:11:24.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:24.073+0000 7f35e1200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55798e1caec0 0x55798e1c89a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:24.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:24.073+0000 7f35e1c00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55798e1c22e0 0x55798e1c8460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:24.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:24.073+0000 7f35e1200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55798e1caec0 0x55798e1c89a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:41650/0 (socket says 172.21.15.45:41650) 2024-04-16T17:11:24.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:24.073+0000 7f35e1200640 1 -- 172.21.15.45:0/3670000956 learned_addr learned my addr 172.21.15.45:0/3670000956 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:11:24.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:24.073+0000 7f35e1200640 1 --2- 172.21.15.45:0/3670000956 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55798e1c19c0 0x55798e1c7f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:24.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:24.073+0000 7f35e1200640 1 -- 172.21.15.45:0/3670000956 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55798e1c22e0 msgr2=0x55798e1c8460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:11:24.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:24.073+0000 7f35e1200640 1 --2- 172.21.15.45:0/3670000956 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55798e1c22e0 0x55798e1c8460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:24.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:24.073+0000 7f35e1200640 1 -- 172.21.15.45:0/3670000956 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55798e1c19c0 msgr2=0x55798e1c7f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:11:24.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:24.073+0000 7f35e1200640 1 --2- 172.21.15.45:0/3670000956 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55798e1c19c0 0x55798e1c7f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:24.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:24.073+0000 7f35e1200640 1 -- 172.21.15.45:0/3670000956 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55798de313f0 con 0x55798e1caec0 2024-04-16T17:11:24.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:24.073+0000 7f35e1200640 1 --2- 172.21.15.45:0/3670000956 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55798e1caec0 0x55798e1c89a0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f35c8017540 tx=0x7f35c8018700 comp rx=0 tx=0).ready entity=mon.2 client_cookie=810da182c798c724 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:11:24.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:24.073+0000 7f35c3400640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-04-16T17:11:24.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:24.073+0000 7f35c3400640 1 -- 172.21.15.45:0/3670000956 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35c8029060 con 0x55798e1caec0 2024-04-16T17:11:24.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:24.073+0000 7f35c3400640 1 -- 172.21.15.45:0/3670000956 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f35c8015760 con 0x55798e1caec0 2024-04-16T17:11:24.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:24.073+0000 7f35e738b740 1 -- 172.21.15.45:0/3670000956 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f35c801d070 con 0x55798e1caec0 2024-04-16T17:11:24.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:24.073+0000 7f35c3400640 1 -- 172.21.15.45:0/3670000956 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f35c8030050 con 0x55798e1caec0 2024-04-16T17:11:24.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:24.073+0000 7f35e738b740 10 client.26945 fetch_fsmap learned FSMap version 32 2024-04-16T17:11:24.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:24.073+0000 7f35e738b740 1 -- 172.21.15.45:0/3670000956 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55798ddf10f0 con 0x55798e1caec0 2024-04-16T17:11:24.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:24.073+0000 7f35c3400640 1 -- 172.21.15.45:0/3670000956 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f35c801d070 con 0x55798e1caec0 2024-04-16T17:11:24.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:24.073+0000 7f35c3400640 1 -- 172.21.15.45:0/3670000956 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f35c8021870 con 0x55798e1caec0 2024-04-16T17:11:24.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:24.073+0000 7f35e738b740 10 client.26945 fetch_fsmap finished waiting for FSMap version 32 2024-04-16T17:11:24.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:24.073+0000 7f35e738b740 10 client.26945 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:11:24.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:24.073+0000 7f35e738b740 20 client.26945 populate_metadata read hostname 'smithi045' 2024-04-16T17:11:24.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:24.073+0000 7f35e738b740 1 --2- 172.21.15.45:0/3670000956 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55798e1db130 0x55798e1dd520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:24.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:24.073+0000 7f35e738b740 4 client.26945 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:11:24.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:24.073+0000 7f35e738b740 1 -- 172.21.15.45:0/3670000956 --> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55798e12cb40 con 0x55798e1db130 2024-04-16T17:11:24.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:24.073+0000 7f35e1c00640 1 --2- 172.21.15.45:0/3670000956 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55798e1db130 0x55798e1dd520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:24.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:24.077+0000 7f35e1c00640 1 --2- 172.21.15.45:0/3670000956 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55798e1db130 0x55798e1dd520 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=37e5158d62406009 server_cookie=8f1035a0a623002f in_seq=0 out_seq=0 2024-04-16T17:11:24.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:24.077+0000 7f35c3400640 10 client.26945 ms_handle_connect on v2:172.21.15.104:6838/3660070592 2024-04-16T17:11:24.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:24.077+0000 7f35c3400640 1 -- 172.21.15.45:0/3670000956 <== mds.0 v2:172.21.15.104:6838/3660070592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55798e12cb40 con 0x55798e1db130 2024-04-16T17:11:24.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:24.077+0000 7f35c3400640 10 client.26945 handle_command_reply: tid=0 2024-04-16T17:11:24.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:24.078+0000 7f35e738b740 1 client.26945 shutdown 2024-04-16T17:11:24.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:24.078+0000 7f35e738b740 1 -- 172.21.15.45:0/3670000956 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55798e1caec0 msgr2=0x55798e1c89a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:24.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:24.078+0000 7f35e738b740 1 --2- 172.21.15.45:0/3670000956 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55798e1caec0 0x55798e1c89a0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f35c8017540 tx=0x7f35c8018700 comp rx=0 tx=0).stop 2024-04-16T17:11:24.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:24.078+0000 7f35e738b740 1 -- 172.21.15.45:0/3670000956 shutdown_connections 2024-04-16T17:11:24.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:24.078+0000 7f35e738b740 1 --2- 172.21.15.45:0/3670000956 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55798e1db130 0x55798e1dd520 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:24.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:24.078+0000 7f35c3400640 0 client.26945 ms_handle_reset on v2:172.21.15.104:6838/3660070592 2024-04-16T17:11:24.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:24.078+0000 7f35e738b740 1 --2- 172.21.15.45:0/3670000956 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55798e1c22e0 0x55798e1c8460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:24.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:24.078+0000 7f35e738b740 1 --2- 172.21.15.45:0/3670000956 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55798e1caec0 0x55798e1c89a0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:24.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:24.078+0000 7f35e738b740 1 --2- 172.21.15.45:0/3670000956 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55798e1c19c0 0x55798e1c7f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:24.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:24.078+0000 7f35e738b740 1 -- 172.21.15.45:0/3670000956 >> 172.21.15.45:0/3670000956 conn(0x55798e1bc0f0 msgr2=0x55798e1bde40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:11:24.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:24.078+0000 7f35e738b740 1 -- 172.21.15.45:0/3670000956 shutdown_connections 2024-04-16T17:11:24.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:24.078+0000 7f35e738b740 1 -- 172.21.15.45:0/3670000956 wait complete. 2024-04-16T17:11:24.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:24.079+0000 7f35e738b740 20 client.26945 trim_cache size 0 max 16384 2024-04-16T17:11:24.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:24.113+0000 7f35e738b740 1 Processor -- start 2024-04-16T17:11:24.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:24.113+0000 7f35e738b740 1 -- start start 2024-04-16T17:11:24.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:24.113+0000 7f35e738b740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55798e1c19c0 0x55798e1ca650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:24.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:24.113+0000 7f35e738b740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55798e1c22e0 0x55798e1d6b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:24.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:24.113+0000 7f35e738b740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55798e1caec0 0x55798e1c85f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:24.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:24.113+0000 7f35e738b740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f35dc1b83d0 con 0x55798e1c22e0 2024-04-16T17:11:24.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:24.113+0000 7f35e738b740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f35dc1b5c10 con 0x55798e1c19c0 2024-04-16T17:11:24.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:24.113+0000 7f35e738b740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f35dc1b3a40 con 0x55798e1caec0 2024-04-16T17:11:24.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:24.114+0000 7f35e1c00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55798e1c22e0 0x55798e1d6b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:24.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:24.114+0000 7f35e1c00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55798e1c22e0 0x55798e1d6b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55240/0 (socket says 172.21.15.45:55240) 2024-04-16T17:11:24.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:24.114+0000 7f35e1c00640 1 -- 172.21.15.45:0/3179092122 learned_addr learned my addr 172.21.15.45:0/3179092122 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:11:24.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:24.114+0000 7f35e1c00640 1 -- 172.21.15.45:0/3179092122 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55798e1caec0 msgr2=0x55798e1c85f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-16T17:11:24.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:24.114+0000 7f35e1c00640 1 --2- 172.21.15.45:0/3179092122 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55798e1caec0 0x55798e1c85f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:24.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:24.114+0000 7f35e1c00640 1 -- 172.21.15.45:0/3179092122 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55798e1c19c0 msgr2=0x55798e1ca650 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-16T17:11:24.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:24.114+0000 7f35e1c00640 1 --2- 172.21.15.45:0/3179092122 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55798e1c19c0 0x55798e1ca650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:24.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:24.114+0000 7f35e1c00640 1 -- 172.21.15.45:0/3179092122 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55798e1af1a0 con 0x55798e1c22e0 2024-04-16T17:11:24.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:24.115+0000 7f35e1c00640 1 --2- 172.21.15.45:0/3179092122 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55798e1c22e0 0x55798e1d6b40 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f35d002c230 tx=0x7f35d0005a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1077d494146b8153 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:11:24.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:24.115+0000 7f35ce000640 1 -- 172.21.15.45:0/3179092122 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35d0050070 con 0x55798e1c22e0 2024-04-16T17:11:24.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:24.115+0000 7f35ce000640 1 -- 172.21.15.45:0/3179092122 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f35d0004240 con 0x55798e1c22e0 2024-04-16T17:11:24.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:24.115+0000 7f35ce000640 1 -- 172.21.15.45:0/3179092122 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35d0004ee0 con 0x55798e1c22e0 2024-04-16T17:11:24.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:24.115+0000 7f35e738b740 1 -- 172.21.15.45:0/3179092122 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55798e1c22e0 msgr2=0x55798e1d6b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:24.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:24.115+0000 7f35e738b740 1 --2- 172.21.15.45:0/3179092122 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55798e1c22e0 0x55798e1d6b40 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f35d002c230 tx=0x7f35d0005a20 comp rx=0 tx=0).stop 2024-04-16T17:11:24.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:24.115+0000 7f35e738b740 1 -- 172.21.15.45:0/3179092122 shutdown_connections 2024-04-16T17:11:24.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:24.115+0000 7f35e738b740 1 --2- 172.21.15.45:0/3179092122 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55798e1caec0 0x55798e1c85f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:24.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:24.115+0000 7f35e738b740 1 --2- 172.21.15.45:0/3179092122 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55798e1c22e0 0x55798e1d6b40 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:24.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:24.115+0000 7f35e738b740 1 --2- 172.21.15.45:0/3179092122 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55798e1c19c0 0x55798e1ca650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:24.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:24.115+0000 7f35e738b740 1 -- 172.21.15.45:0/3179092122 >> 172.21.15.45:0/3179092122 conn(0x55798e1bc0f0 msgr2=0x55798e1c7ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:11:24.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:24.115+0000 7f35e738b740 1 -- 172.21.15.45:0/3179092122 shutdown_connections 2024-04-16T17:11:24.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:24.115+0000 7f35e738b740 1 -- 172.21.15.45:0/3179092122 wait complete. 2024-04-16T17:11:24.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:24.117+0000 7f35e738b740 1 -- start start 2024-04-16T17:11:24.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:24.117+0000 7f35e738b740 1 Processor -- start 2024-04-16T17:11:24.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:24.117+0000 7f35e738b740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55798e1c19c0 0x55798e1d3550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:24.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:24.117+0000 7f35e738b740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55798e1c22e0 0x55798e1c4a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:24.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:24.117+0000 7f35e738b740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55798e1caec0 0x55798e1c5110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:24.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:24.117+0000 7f35e738b740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f35dc1b40d0 con 0x55798e1c19c0 2024-04-16T17:11:24.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:24.117+0000 7f35e738b740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f35dc1b5540 con 0x55798e1c22e0 2024-04-16T17:11:24.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:24.117+0000 7f35e738b740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f35dc1b47a0 con 0x55798e1caec0 2024-04-16T17:11:24.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:24.117+0000 7f35e738b740 10 client.0 _ll_register_callbacks cb 0x7f35d0009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:11:24.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:24.117+0000 7f35e1200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55798e1c19c0 0x55798e1d3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:24.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:24.117+0000 7f35e1200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55798e1c19c0 0x55798e1d3550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55248/0 (socket says 172.21.15.45:55248) 2024-04-16T17:11:24.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:24.117+0000 7f35e1200640 1 -- 172.21.15.45:0/4004382513 learned_addr learned my addr 172.21.15.45:0/4004382513 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:11:24.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:24.117+0000 7f35e1c00640 1 --2- 172.21.15.45:0/4004382513 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55798e1c22e0 0x55798e1c4a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:24.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:24.117+0000 7f35e1200640 1 --2- 172.21.15.45:0/4004382513 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55798e1caec0 0x55798e1c5110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:24.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:24.117+0000 7f35e1200640 1 -- 172.21.15.45:0/4004382513 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55798e1caec0 msgr2=0x55798e1c5110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:11:24.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:24.117+0000 7f35e1200640 1 --2- 172.21.15.45:0/4004382513 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55798e1caec0 0x55798e1c5110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:24.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:24.117+0000 7f35e1200640 1 -- 172.21.15.45:0/4004382513 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55798e1c22e0 msgr2=0x55798e1c4a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:11:24.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:24.117+0000 7f35e1200640 1 --2- 172.21.15.45:0/4004382513 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55798e1c22e0 0x55798e1c4a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:24.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:24.117+0000 7f35e1200640 1 -- 172.21.15.45:0/4004382513 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55798e240360 con 0x55798e1c19c0 2024-04-16T17:11:24.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:24.117+0000 7f35e1200640 1 --2- 172.21.15.45:0/4004382513 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55798e1c19c0 0x55798e1d3550 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f35c80091f0 tx=0x7f35c8009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b98313a1b30f16f server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:11:24.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:24.117+0000 7f35c3400640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-16T17:11:24.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:24.117+0000 7f35c3400640 1 -- 172.21.15.45:0/4004382513 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35c8029050 con 0x55798e1c19c0 2024-04-16T17:11:24.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:24.117+0000 7f35e738b740 1 -- 172.21.15.45:0/4004382513 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f35d0004240 con 0x55798e1c19c0 2024-04-16T17:11:24.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:24.117+0000 7f35c3400640 1 -- 172.21.15.45:0/4004382513 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f35c8009a00 con 0x55798e1c19c0 2024-04-16T17:11:24.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:24.117+0000 7f35c3400640 1 -- 172.21.15.45:0/4004382513 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f35c8017e60 con 0x55798e1c19c0 2024-04-16T17:11:24.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:24.117+0000 7f35e738b740 10 client.16800 fetch_fsmap learned FSMap version 32 2024-04-16T17:11:24.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:24.117+0000 7f35e738b740 1 -- 172.21.15.45:0/4004382513 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55798e241480 con 0x55798e1c19c0 2024-04-16T17:11:24.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:24.117+0000 7f35c3400640 1 -- 172.21.15.45:0/4004382513 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f35d0004240 con 0x55798e1c19c0 2024-04-16T17:11:24.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:24.117+0000 7f35c3400640 1 -- 172.21.15.45:0/4004382513 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f35c8015830 con 0x55798e1c19c0 2024-04-16T17:11:24.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:24.117+0000 7f35e738b740 10 client.16800 fetch_fsmap finished waiting for FSMap version 32 2024-04-16T17:11:24.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:24.117+0000 7f35e738b740 10 client.16800 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:11:24.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:24.117+0000 7f35e738b740 20 client.16800 populate_metadata read hostname 'smithi045' 2024-04-16T17:11:24.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:24.117+0000 7f35e738b740 1 --2- 172.21.15.45:0/4004382513 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55798e1db130 0x55798e243b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:24.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:24.117+0000 7f35e738b740 4 client.16800 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-16T17:11:24.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:24.117+0000 7f35e738b740 1 -- 172.21.15.45:0/4004382513 --> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55798e1269b0 con 0x55798e1db130 2024-04-16T17:11:24.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:24.117+0000 7f35e1c00640 1 --2- 172.21.15.45:0/4004382513 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55798e1db130 0x55798e243b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:24.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:24.117+0000 7f35e1c00640 1 --2- 172.21.15.45:0/4004382513 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55798e1db130 0x55798e243b00 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e5baa141687a9d91 server_cookie=2d1d1a9b6f07f506 in_seq=0 out_seq=0 2024-04-16T17:11:24.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:24.117+0000 7f35c3400640 10 client.16800 ms_handle_connect on v2:172.21.15.104:6838/3660070592 2024-04-16T17:11:24.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:24.117+0000 7f35c3400640 1 -- 172.21.15.45:0/4004382513 <== mds.0 v2:172.21.15.104:6838/3660070592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55798e1269b0 con 0x55798e1db130 2024-04-16T17:11:24.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:24.117+0000 7f35c3400640 10 client.16800 handle_command_reply: tid=0 2024-04-16T17:11:24.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:24.117+0000 7f35e738b740 1 client.16800 shutdown 2024-04-16T17:11:24.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:24.117+0000 7f35e738b740 1 -- 172.21.15.45:0/4004382513 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55798e1c19c0 msgr2=0x55798e1d3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:24.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:24.117+0000 7f35e738b740 1 --2- 172.21.15.45:0/4004382513 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55798e1c19c0 0x55798e1d3550 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f35c80091f0 tx=0x7f35c8009da0 comp rx=0 tx=0).stop 2024-04-16T17:11:24.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:24.117+0000 7f35e738b740 1 -- 172.21.15.45:0/4004382513 shutdown_connections 2024-04-16T17:11:24.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:24.117+0000 7f35e738b740 1 --2- 172.21.15.45:0/4004382513 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55798e1db130 0x55798e243b00 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:24.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:24.117+0000 7f35e738b740 1 --2- 172.21.15.45:0/4004382513 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55798e1caec0 0x55798e1c5110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:24.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:24.117+0000 7f35e738b740 1 --2- 172.21.15.45:0/4004382513 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55798e1c22e0 0x55798e1c4a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:24.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:24.117+0000 7f35e738b740 1 --2- 172.21.15.45:0/4004382513 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55798e1c19c0 0x55798e1d3550 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:24.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:24.117+0000 7f35e738b740 1 -- 172.21.15.45:0/4004382513 >> 172.21.15.45:0/4004382513 conn(0x55798e1bc0f0 msgr2=0x55798e1b28d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:11:24.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:24.117+0000 7f35c3400640 0 client.16800 ms_handle_reset on v2:172.21.15.104:6838/3660070592 2024-04-16T17:11:24.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:24.117+0000 7f35e738b740 1 -- 172.21.15.45:0/4004382513 shutdown_connections 2024-04-16T17:11:24.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:24.117+0000 7f35e738b740 1 -- 172.21.15.45:0/4004382513 wait complete. 2024-04-16T17:11:24.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:24.117+0000 7f35e738b740 20 client.16800 trim_cache size 0 max 16384 2024-04-16T17:11:24.126 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-16T17:11:24.126 INFO:teuthology.orchestra.run.smithi045.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-16T17:11:24.126 INFO:teuthology.orchestra.run.smithi045.stdout: "scrubs": { 2024-04-16T17:11:24.127 INFO:teuthology.orchestra.run.smithi045.stdout: "e99870ef-51e6-4d3c-a90a-ccff17939e5a": { 2024-04-16T17:11:24.127 INFO:teuthology.orchestra.run.smithi045.stdout: "path": "/", 2024-04-16T17:11:24.127 INFO:teuthology.orchestra.run.smithi045.stdout: "tag": "e99870ef-51e6-4d3c-a90a-ccff17939e5a", 2024-04-16T17:11:24.127 INFO:teuthology.orchestra.run.smithi045.stdout: "options": "recursive,force" 2024-04-16T17:11:24.127 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:11:24.127 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:11:24.127 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-16T17:11:24.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:24.122+0000 7f35e4600640 1 -- 172.21.15.45:0/1536690963 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f35c40a3da0 msgr2=0x7f35c40a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:24.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:24.122+0000 7f35e4600640 1 --2- 172.21.15.45:0/1536690963 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f35c40a3da0 0x7f35c40a6260 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f35d4009bc0 tx=0x7f35d4009b50 comp rx=0 tx=0).stop 2024-04-16T17:11:24.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:24.122+0000 7f35e4600640 1 -- 172.21.15.45:0/1536690963 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f35dc06cec0 msgr2=0x7f35dc06d370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:24.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:24.122+0000 7f35e4600640 1 --2- 172.21.15.45:0/1536690963 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f35dc06cec0 0x7f35dc06d370 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f35d400ec50 tx=0x7f35d400c6a0 comp rx=0 tx=0).stop 2024-04-16T17:11:24.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:24.122+0000 7f35e4600640 1 -- 172.21.15.45:0/1536690963 shutdown_connections 2024-04-16T17:11:24.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:24.122+0000 7f35e4600640 1 --2- 172.21.15.45:0/1536690963 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f35c40a3da0 0x7f35c40a6260 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:24.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:24.122+0000 7f35e4600640 1 --2- 172.21.15.45:0/1536690963 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f35dc06cec0 0x7f35dc06d370 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:24.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:24.122+0000 7f35e4600640 1 --2- 172.21.15.45:0/1536690963 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f35dc109580 0x7f35dc06c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:24.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:24.122+0000 7f35e4600640 1 --2- 172.21.15.45:0/1536690963 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f35dc1044b0 0x7f35dc0738b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:24.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:24.122+0000 7f35e4600640 1 -- 172.21.15.45:0/1536690963 >> 172.21.15.45:0/1536690963 conn(0x7f35dc0ffd90 msgr2=0x7f35dc1010b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:11:24.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:24.122+0000 7f35e4600640 1 -- 172.21.15.45:0/1536690963 shutdown_connections 2024-04-16T17:11:24.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:24.122+0000 7f35e4600640 1 -- 172.21.15.45:0/1536690963 wait complete. 2024-04-16T17:11:24.139 INFO:tasks.cephfs.filesystem:scrub status for tag:e99870ef-51e6-4d3c-a90a-ccff17939e5a - {'path': '/', 'tag': 'e99870ef-51e6-4d3c-a90a-ccff17939e5a', 'options': 'recursive,force'} 2024-04-16T17:11:24.177 INFO:journalctl@ceph.mds.b.smithi104.stdout:Apr 16 17:11:23 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mds-b[67798]: 2024-04-16T17:11:23.850+0000 7fde259be700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000215 [...7,head] /volumes/qa/sv_1/584ee6b0-43a5-48e1-b843-9914b8e65254/client.0/tmp/clients/client0/~dmtmp/COREL/ auth{1=1,2=1} v11522 ap=2 f(v192 m2024-04-16T17:11:12.890137+0000 11=11+0) n(v14 rc2024-04-16T17:11:12.890137+0000 b2861056 12=11+1) old_inodes=4 (inest sync->mix g=1,2) caps={25436=pAsLsXsFs/pAsLsXsFsx@795} mcw={1=AsLsXsFs,2=AsLsXsFs} | dirtyscattered=0 request=0 lock=0 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 0x55c73319c580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x10000000215:[<0x10000000009/COREL v634>,<0x10000000008/~dmtmp v34>,<0x10000000007/client0 v20>,<0x10000000006/clients v10>,<0x10000000005/tmp v10>,<0x100000001fb/client.0 v14>,<0x100000001fa/584ee6b0-43a5-48e1-b843-9914b8e65254 v30>,<0x10000000001/sv_1 v53>,<0x10000000000/qa v41>,<0x1/volumes v39>]//[]","memoryvalue":"(2)0x10000000215:[<0x10000000009/COREL v11522>,<0x10000000008/~dmtmp v2911>,<0x10000000007/client0 v398>,<0x10000000006/clients v365>,<0x10000000005/tmp v365>,<0x100000001fb/client.0 v365>,<0x100000001fa/584ee6b0-43a5-48e1-b843-9914b8e65254 v374>,<0x10000000001/sv_1 v384>,<0x10000000000/qa v368>,<0x1/volumes v363>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-04-16T17:11:11.707155+0000 15=15+0)","ondisk_value.rstat":"n(v0 rc2024-04-16T17:11:12.890137+0000 b2861056 12=11+1)","memory_value.dirstat":"f(v192 m2024-04-16T17:11:12.890137+0000 11=11+0)","memory_value.rstat":"n(v14 rc2024-04-16T17:11:12.890137+0000 b2861056 12=11+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-04-16T17:11:24.249 INFO:tasks.workunit.client.0.smithi045.stdout: 1 65479 1.41 MB/sec execute 86 sec latency 134.019 ms 2024-04-16T17:11:24.625 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:11:22 smithi045 ceph-mon[31676]: pgmap v141: 129 pgs: 129 active+clean; 40 MiB data, 527 MiB used, 1.0 TiB / 1.0 TiB avail; 602 KiB/s rd, 731 KiB/s wr, 84 op/s 2024-04-16T17:11:24.625 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:11:22 smithi045 ceph-mon[31676]: from='client.? 172.21.15.45:0/3816587143' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:11:24.677 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:11:24 smithi104 ceph-mon[36457]: pgmap v142: 129 pgs: 129 active+clean; 40 MiB data, 527 MiB used, 1.0 TiB / 1.0 TiB avail; 251 KiB/s rd, 599 KiB/s wr, 54 op/s 2024-04-16T17:11:24.804 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:11:24 smithi117 ceph-mon[36730]: pgmap v142: 129 pgs: 129 active+clean; 40 MiB data, 527 MiB used, 1.0 TiB / 1.0 TiB avail; 251 KiB/s rd, 599 KiB/s wr, 54 op/s 2024-04-16T17:11:25.046 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:11:24 smithi045 ceph-mon[31676]: pgmap v142: 129 pgs: 129 active+clean; 40 MiB data, 527 MiB used, 1.0 TiB / 1.0 TiB avail; 251 KiB/s rd, 599 KiB/s wr, 54 op/s 2024-04-16T17:11:25.249 INFO:tasks.workunit.client.0.smithi045.stdout: 1 65716 1.40 MB/sec execute 87 sec latency 75.586 ms 2024-04-16T17:11:25.677 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:11:25 smithi104 ceph-mon[36457]: scrub queued for path: / 2024-04-16T17:11:25.677 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:11:25 smithi104 ceph-mon[36457]: scrub summary: idle+waiting paths [/] 2024-04-16T17:11:25.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:11:25 smithi104 ceph-mon[36457]: scrub summary: active paths [/] 2024-04-16T17:11:25.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:11:25 smithi104 ceph-mon[36457]: scrub summary: active paths [/] 2024-04-16T17:11:25.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:11:25 smithi104 ceph-mon[36457]: scrub summary: active paths [/] 2024-04-16T17:11:25.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:11:25 smithi104 ceph-mon[36457]: scrub summary: idle+waiting paths [/] 2024-04-16T17:11:25.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:11:25 smithi104 ceph-mon[36457]: Scrub error on inode 0x10000000215 (/volumes/qa/sv_1/584ee6b0-43a5-48e1-b843-9914b8e65254/client.0/tmp/clients/client0/~dmtmp/COREL) see mds.b log and `damage ls` output for details 2024-04-16T17:11:25.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:11:25 smithi104 ceph-mon[36457]: scrub summary: idle+waiting paths [/] 2024-04-16T17:11:25.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:11:25 smithi104 ceph-mon[36457]: scrub summary: idle+waiting paths [/] 2024-04-16T17:11:25.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:11:25 smithi117 ceph-mon[36730]: scrub queued for path: / 2024-04-16T17:11:25.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:11:25 smithi117 ceph-mon[36730]: scrub summary: idle+waiting paths [/] 2024-04-16T17:11:25.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:11:25 smithi117 ceph-mon[36730]: scrub summary: active paths [/] 2024-04-16T17:11:25.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:11:25 smithi117 ceph-mon[36730]: scrub summary: active paths [/] 2024-04-16T17:11:25.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:11:25 smithi117 ceph-mon[36730]: scrub summary: active paths [/] 2024-04-16T17:11:25.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:11:25 smithi117 ceph-mon[36730]: scrub summary: idle+waiting paths [/] 2024-04-16T17:11:25.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:11:25 smithi117 ceph-mon[36730]: Scrub error on inode 0x10000000215 (/volumes/qa/sv_1/584ee6b0-43a5-48e1-b843-9914b8e65254/client.0/tmp/clients/client0/~dmtmp/COREL) see mds.b log and `damage ls` output for details 2024-04-16T17:11:25.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:11:25 smithi117 ceph-mon[36730]: scrub summary: idle+waiting paths [/] 2024-04-16T17:11:25.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:11:25 smithi117 ceph-mon[36730]: scrub summary: idle+waiting paths [/] 2024-04-16T17:11:26.249 INFO:tasks.workunit.client.0.smithi045.stdout: 1 66423 1.41 MB/sec execute 88 sec latency 32.777 ms 2024-04-16T17:11:26.677 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:11:26 smithi104 ceph-mon[36457]: pgmap v143: 129 pgs: 129 active+clean; 40 MiB data, 527 MiB used, 1.0 TiB / 1.0 TiB avail; 251 KiB/s rd, 599 KiB/s wr, 54 op/s 2024-04-16T17:11:26.804 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:11:26 smithi117 ceph-mon[36730]: pgmap v143: 129 pgs: 129 active+clean; 40 MiB data, 527 MiB used, 1.0 TiB / 1.0 TiB avail; 251 KiB/s rd, 599 KiB/s wr, 54 op/s 2024-04-16T17:11:27.227 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:11:25 smithi045 ceph-mon[31676]: scrub queued for path: / 2024-04-16T17:11:27.228 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:11:25 smithi045 ceph-mon[31676]: scrub summary: idle+waiting paths [/] 2024-04-16T17:11:27.228 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:11:25 smithi045 ceph-mon[31676]: scrub summary: active paths [/] 2024-04-16T17:11:27.228 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:11:25 smithi045 ceph-mon[31676]: scrub summary: active paths [/] 2024-04-16T17:11:27.228 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:11:25 smithi045 ceph-mon[31676]: scrub summary: active paths [/] 2024-04-16T17:11:27.228 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:11:25 smithi045 ceph-mon[31676]: scrub summary: idle+waiting paths [/] 2024-04-16T17:11:27.228 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:11:25 smithi045 ceph-mon[31676]: Scrub error on inode 0x10000000215 (/volumes/qa/sv_1/584ee6b0-43a5-48e1-b843-9914b8e65254/client.0/tmp/clients/client0/~dmtmp/COREL) see mds.b log and `damage ls` output for details 2024-04-16T17:11:27.228 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:11:25 smithi045 ceph-mon[31676]: scrub summary: idle+waiting paths [/] 2024-04-16T17:11:27.228 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:11:25 smithi045 ceph-mon[31676]: scrub summary: idle+waiting paths [/] 2024-04-16T17:11:27.228 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:11:26 smithi045 ceph-mon[31676]: pgmap v143: 129 pgs: 129 active+clean; 40 MiB data, 527 MiB used, 1.0 TiB / 1.0 TiB avail; 251 KiB/s rd, 599 KiB/s wr, 54 op/s 2024-04-16T17:11:27.249 INFO:tasks.workunit.client.0.smithi045.stdout: 1 67525 1.50 MB/sec execute 89 sec latency 42.793 ms 2024-04-16T17:11:28.249 INFO:tasks.workunit.client.0.smithi045.stdout: 1 67828 1.50 MB/sec execute 90 sec latency 36.695 ms 2024-04-16T17:11:28.677 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:11:28 smithi104 ceph-mon[36457]: pgmap v144: 129 pgs: 129 active+clean; 41 MiB data, 558 MiB used, 1.0 TiB / 1.0 TiB avail; 722 KiB/s rd, 1.1 MiB/s wr, 103 op/s 2024-04-16T17:11:28.803 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:11:28 smithi045 ceph-mon[31676]: pgmap v144: 129 pgs: 129 active+clean; 41 MiB data, 558 MiB used, 1.0 TiB / 1.0 TiB avail; 722 KiB/s rd, 1.1 MiB/s wr, 103 op/s 2024-04-16T17:11:28.804 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:11:28 smithi117 ceph-mon[36730]: pgmap v144: 129 pgs: 129 active+clean; 41 MiB data, 558 MiB used, 1.0 TiB / 1.0 TiB avail; 722 KiB/s rd, 1.1 MiB/s wr, 103 op/s 2024-04-16T17:11:29.249 INFO:tasks.workunit.client.0.smithi045.stdout: 1 68210 1.49 MB/sec execute 91 sec latency 53.639 ms 2024-04-16T17:11:30.250 INFO:tasks.workunit.client.0.smithi045.stdout: 1 68679 1.49 MB/sec execute 92 sec latency 52.933 ms 2024-04-16T17:11:30.677 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:11:30 smithi104 ceph-mon[36457]: pgmap v145: 129 pgs: 129 active+clean; 41 MiB data, 558 MiB used, 1.0 TiB / 1.0 TiB avail; 608 KiB/s rd, 1023 KiB/s wr, 84 op/s 2024-04-16T17:11:30.743 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:11:30 smithi045 ceph-mon[31676]: pgmap v145: 129 pgs: 129 active+clean; 41 MiB data, 558 MiB used, 1.0 TiB / 1.0 TiB avail; 608 KiB/s rd, 1023 KiB/s wr, 84 op/s 2024-04-16T17:11:30.804 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:11:30 smithi117 ceph-mon[36730]: pgmap v145: 129 pgs: 129 active+clean; 41 MiB data, 558 MiB used, 1.0 TiB / 1.0 TiB avail; 608 KiB/s rd, 1023 KiB/s wr, 84 op/s 2024-04-16T17:11:31.250 INFO:tasks.workunit.client.0.smithi045.stdout: 1 68988 1.51 MB/sec execute 93 sec latency 44.362 ms 2024-04-16T17:11:32.250 INFO:tasks.workunit.client.0.smithi045.stdout: 1 69247 1.49 MB/sec execute 94 sec latency 99.776 ms 2024-04-16T17:11:32.677 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:11:32 smithi104 ceph-mon[36457]: pgmap v146: 129 pgs: 129 active+clean; 43 MiB data, 572 MiB used, 1.0 TiB / 1.0 TiB avail; 865 KiB/s rd, 1.2 MiB/s wr, 116 op/s 2024-04-16T17:11:32.677 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:11:32 smithi104 ceph-mon[36457]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-16T17:11:32.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:11:32 smithi117 ceph-mon[36730]: pgmap v146: 129 pgs: 129 active+clean; 43 MiB data, 572 MiB used, 1.0 TiB / 1.0 TiB avail; 865 KiB/s rd, 1.2 MiB/s wr, 116 op/s 2024-04-16T17:11:32.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:11:32 smithi117 ceph-mon[36730]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-16T17:11:33.121 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:11:32 smithi045 ceph-mon[31676]: pgmap v146: 129 pgs: 129 active+clean; 43 MiB data, 572 MiB used, 1.0 TiB / 1.0 TiB avail; 865 KiB/s rd, 1.2 MiB/s wr, 116 op/s 2024-04-16T17:11:33.121 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:11:32 smithi045 ceph-mon[31676]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-16T17:11:33.250 INFO:tasks.workunit.client.0.smithi045.stdout: 1 69577 1.48 MB/sec execute 95 sec latency 78.829 ms 2024-04-16T17:11:34.250 INFO:tasks.workunit.client.0.smithi045.stdout: 1 69970 1.49 MB/sec execute 96 sec latency 81.764 ms 2024-04-16T17:11:34.677 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:11:34 smithi104 ceph-mon[36457]: pgmap v147: 129 pgs: 129 active+clean; 43 MiB data, 572 MiB used, 1.0 TiB / 1.0 TiB avail; 729 KiB/s rd, 756 KiB/s wr, 81 op/s 2024-04-16T17:11:34.677 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:11:34 smithi104 ceph-mon[36457]: scrub summary: idle 2024-04-16T17:11:34.677 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:11:34 smithi104 ceph-mon[36457]: scrub summary: idle 2024-04-16T17:11:34.677 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:11:34 smithi104 ceph-mon[36457]: scrub summary: idle 2024-04-16T17:11:34.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:11:34 smithi117 ceph-mon[36730]: pgmap v147: 129 pgs: 129 active+clean; 43 MiB data, 572 MiB used, 1.0 TiB / 1.0 TiB avail; 729 KiB/s rd, 756 KiB/s wr, 81 op/s 2024-04-16T17:11:34.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:11:34 smithi117 ceph-mon[36730]: scrub summary: idle 2024-04-16T17:11:34.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:11:34 smithi117 ceph-mon[36730]: scrub summary: idle 2024-04-16T17:11:34.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:11:34 smithi117 ceph-mon[36730]: scrub summary: idle 2024-04-16T17:11:35.085 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:35.084+0000 7fcadea00640 1 -- 172.21.15.45:0/3099245176 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcad8102290 msgr2=0x7fcad8102670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:35.085 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:35.084+0000 7fcadea00640 1 --2- 172.21.15.45:0/3099245176 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcad8102290 0x7fcad8102670 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fcac8009900 tx=0x7fcac802bc30 comp rx=0 tx=0).stop 2024-04-16T17:11:35.085 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:35.084+0000 7fcadea00640 1 -- 172.21.15.45:0/3099245176 shutdown_connections 2024-04-16T17:11:35.085 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:35.084+0000 7fcadea00640 1 --2- 172.21.15.45:0/3099245176 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcad8107360 0x7fcad810ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:35.085 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:35.084+0000 7fcadea00640 1 --2- 172.21.15.45:0/3099245176 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcad8102c40 0x7fcad8106c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:35.085 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:35.084+0000 7fcadea00640 1 --2- 172.21.15.45:0/3099245176 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcad8102290 0x7fcad8102670 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:35.085 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:35.084+0000 7fcadea00640 1 -- 172.21.15.45:0/3099245176 >> 172.21.15.45:0/3099245176 conn(0x7fcad80fdab0 msgr2=0x7fcad80ffed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:11:35.085 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:35.084+0000 7fcadea00640 1 -- 172.21.15.45:0/3099245176 shutdown_connections 2024-04-16T17:11:35.085 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:35.084+0000 7fcadea00640 1 -- 172.21.15.45:0/3099245176 wait complete. 2024-04-16T17:11:35.086 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:35.085+0000 7fcadea00640 1 Processor -- start 2024-04-16T17:11:35.086 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:35.086+0000 7fcadea00640 1 -- start start 2024-04-16T17:11:35.087 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:35.086+0000 7fcadea00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcad8102290 0x7fcad819e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:35.087 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:35.086+0000 7fcadea00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcad8107360 0x7fcad819eb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:35.087 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:35.086+0000 7fcadea00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcad81986a0 0x7fcad8198b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:35.087 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:35.086+0000 7fcadea00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fcad8112440 con 0x7fcad8102290 2024-04-16T17:11:35.087 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:35.086+0000 7fcadea00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fcad81122c0 con 0x7fcad81986a0 2024-04-16T17:11:35.087 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:35.086+0000 7fcadea00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fcad81125c0 con 0x7fcad8107360 2024-04-16T17:11:35.087 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:35.086+0000 7fcad7e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcad8102290 0x7fcad819e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:35.087 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:35.086+0000 7fcadca00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcad81986a0 0x7fcad8198b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:35.087 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:35.086+0000 7fcad7e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcad8102290 0x7fcad819e5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37036/0 (socket says 172.21.15.45:37036) 2024-04-16T17:11:35.088 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:35.086+0000 7fcad7e00640 1 -- 172.21.15.45:0/3759086755 learned_addr learned my addr 172.21.15.45:0/3759086755 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:11:35.088 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:35.087+0000 7fcacfe00640 1 --2- 172.21.15.45:0/3759086755 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcad8107360 0x7fcad819eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:35.088 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:35.087+0000 7fcad7e00640 1 -- 172.21.15.45:0/3759086755 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcad8107360 msgr2=0x7fcad819eb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:35.088 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:35.087+0000 7fcad7e00640 1 --2- 172.21.15.45:0/3759086755 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcad8107360 0x7fcad819eb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:35.089 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:35.087+0000 7fcad7e00640 1 -- 172.21.15.45:0/3759086755 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcad81986a0 msgr2=0x7fcad8198b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:35.089 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:35.087+0000 7fcad7e00640 1 --2- 172.21.15.45:0/3759086755 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcad81986a0 0x7fcad8198b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:35.089 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:35.087+0000 7fcad7e00640 1 -- 172.21.15.45:0/3759086755 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcad81992a0 con 0x7fcad8102290 2024-04-16T17:11:35.089 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:35.087+0000 7fcadca00640 1 --2- 172.21.15.45:0/3759086755 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcad81986a0 0x7fcad8198b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:11:35.089 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:35.087+0000 7fcad7e00640 1 --2- 172.21.15.45:0/3759086755 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcad8102290 0x7fcad819e5e0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fcac802c240 tx=0x7fcac80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:11:35.089 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:35.088+0000 7fcad5600640 1 -- 172.21.15.45:0/3759086755 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcac8043020 con 0x7fcad8102290 2024-04-16T17:11:35.089 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:35.088+0000 7fcad5600640 1 -- 172.21.15.45:0/3759086755 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcac80054e0 con 0x7fcad8102290 2024-04-16T17:11:35.089 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:35.088+0000 7fcad5600640 1 -- 172.21.15.45:0/3759086755 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcac8003b20 con 0x7fcad8102290 2024-04-16T17:11:35.089 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:35.089+0000 7fcadea00640 1 -- 172.21.15.45:0/3759086755 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcad8199530 con 0x7fcad8102290 2024-04-16T17:11:35.089 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:35.089+0000 7fcadea00640 1 -- 172.21.15.45:0/3759086755 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcad81a5680 con 0x7fcad8102290 2024-04-16T17:11:35.093 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:35.093+0000 7fcad5600640 1 -- 172.21.15.45:0/3759086755 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fcac8046020 con 0x7fcad8102290 2024-04-16T17:11:35.094 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:35.093+0000 7fcae15fc740 1 Processor -- start 2024-04-16T17:11:35.094 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:35.094+0000 7fcae15fc740 1 -- start start 2024-04-16T17:11:35.095 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:35.094+0000 7fcae15fc740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d0eb318610 0x55d0eb317360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:35.095 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:35.094+0000 7fcae15fc740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d0eb3178a0 0x55d0eb317d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:35.095 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:35.094+0000 7fcae15fc740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d0eb325230 0x55d0eb327620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:35.095 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:35.094+0000 7fcae15fc740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55d0eb27c780 con 0x55d0eb3178a0 2024-04-16T17:11:35.095 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:35.094+0000 7fcae15fc740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55d0eb222f40 con 0x55d0eb325230 2024-04-16T17:11:35.095 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:35.094+0000 7fcae15fc740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55d0eb278740 con 0x55d0eb318610 2024-04-16T17:11:35.096 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:35.094+0000 7fcadca00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d0eb318610 0x55d0eb317360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:35.096 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:35.094+0000 7fcadca00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d0eb318610 0x55d0eb317360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:50712/0 (socket says 172.21.15.45:50712) 2024-04-16T17:11:35.096 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:35.094+0000 7fcadca00640 1 -- 172.21.15.45:0/573964148 learned_addr learned my addr 172.21.15.45:0/573964148 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:11:35.096 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:35.095+0000 7fcad5600640 1 --2- 172.21.15.45:0/3759086755 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fcabc0a3dc0 0x7fcabc0a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:35.096 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:35.095+0000 7fcad5600640 1 -- 172.21.15.45:0/3759086755 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fcac80f2750 con 0x7fcad8102290 2024-04-16T17:11:35.096 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:35.095+0000 7fcacfe00640 1 --2- 172.21.15.45:0/573964148 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d0eb325230 0x55d0eb327620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:35.096 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:35.095+0000 7fcad7e00640 1 --2- 172.21.15.45:0/573964148 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d0eb3178a0 0x55d0eb317d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:35.096 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:35.095+0000 7fcadca00640 1 -- 172.21.15.45:0/573964148 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d0eb325230 msgr2=0x55d0eb327620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:11:35.096 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:35.095+0000 7fcadca00640 1 --2- 172.21.15.45:0/573964148 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d0eb325230 0x55d0eb327620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:35.096 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:35.095+0000 7fcadca00640 1 -- 172.21.15.45:0/573964148 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d0eb3178a0 msgr2=0x55d0eb317d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:11:35.097 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:35.095+0000 7fcadca00640 1 --2- 172.21.15.45:0/573964148 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d0eb3178a0 0x55d0eb317d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:35.097 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:35.095+0000 7fcadca00640 1 -- 172.21.15.45:0/573964148 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d0eaf8f210 con 0x55d0eb318610 2024-04-16T17:11:35.097 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:35.095+0000 7fcad7e00640 1 --2- 172.21.15.45:0/573964148 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d0eb3178a0 0x55d0eb317d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:11:35.097 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:35.095+0000 7fcadca00640 1 --2- 172.21.15.45:0/3759086755 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fcabc0a3dc0 0x7fcabc0a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:35.097 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:35.096+0000 7fcadca00640 1 --2- 172.21.15.45:0/573964148 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d0eb318610 0x55d0eb317360 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fcad0011900 tx=0x7fcad0011dc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=feb87912874b8fe9 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:11:35.098 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:35.096+0000 7fcacd200640 1 -- 172.21.15.45:0/573964148 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcad0017050 con 0x55d0eb318610 2024-04-16T17:11:35.098 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:35.096+0000 7fcacd200640 1 -- 172.21.15.45:0/573964148 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcad001bb90 con 0x55d0eb318610 2024-04-16T17:11:35.098 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:35.096+0000 7fcacd200640 1 -- 172.21.15.45:0/573964148 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcad000e3b0 con 0x55d0eb318610 2024-04-16T17:11:35.098 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:35.097+0000 7fcae15fc740 1 -- 172.21.15.45:0/573964148 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d0eb318610 msgr2=0x55d0eb317360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:35.098 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:35.097+0000 7fcadca00640 1 --2- 172.21.15.45:0/3759086755 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fcabc0a3dc0 0x7fcabc0a6280 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fcad0023000 tx=0x7fcad0024000 comp rx=0 tx=0).ready entity=mgr.25411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:11:35.098 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:35.097+0000 7fcae15fc740 1 --2- 172.21.15.45:0/573964148 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d0eb318610 0x55d0eb317360 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fcad0011900 tx=0x7fcad0011dc0 comp rx=0 tx=0).stop 2024-04-16T17:11:35.098 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:35.097+0000 7fcae15fc740 1 -- 172.21.15.45:0/573964148 shutdown_connections 2024-04-16T17:11:35.098 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:35.097+0000 7fcae15fc740 1 --2- 172.21.15.45:0/573964148 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d0eb325230 0x55d0eb327620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:35.098 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:35.097+0000 7fcae15fc740 1 --2- 172.21.15.45:0/573964148 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d0eb3178a0 0x55d0eb317d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:35.098 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:35.097+0000 7fcae15fc740 1 --2- 172.21.15.45:0/573964148 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d0eb318610 0x55d0eb317360 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:35.098 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:35.097+0000 7fcae15fc740 1 -- 172.21.15.45:0/573964148 >> 172.21.15.45:0/573964148 conn(0x55d0eb2b25d0 msgr2=0x55d0eb312590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:11:35.099 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:35.097+0000 7fcae15fc740 1 -- 172.21.15.45:0/573964148 shutdown_connections 2024-04-16T17:11:35.099 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:35.098+0000 7fcae15fc740 1 -- 172.21.15.45:0/573964148 wait complete. 2024-04-16T17:11:35.099 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:35.098+0000 7fcae15fc740 1 -- start start 2024-04-16T17:11:35.099 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:35.098+0000 7fcae15fc740 1 Processor -- start 2024-04-16T17:11:35.101 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:35.099+0000 7fcae15fc740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d0eb3178a0 0x55d0eb319fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:35.101 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:35.099+0000 7fcae15fc740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d0eb318610 0x55d0eb31a500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:35.101 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:35.099+0000 7fcae15fc740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d0eb325230 0x55d0eb31aa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:35.101 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:35.099+0000 7fcae15fc740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55d0eb3211a0 con 0x55d0eb325230 2024-04-16T17:11:35.101 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:35.099+0000 7fcae15fc740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55d0eb236390 con 0x55d0eb318610 2024-04-16T17:11:35.101 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:35.099+0000 7fcae15fc740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55d0eb321320 con 0x55d0eb3178a0 2024-04-16T17:11:35.101 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:35.099+0000 7fcae15fc740 10 client.0 _ll_register_callbacks cb 0x55d0eb21ff60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:11:35.102 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:35.100+0000 7fcadca00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d0eb325230 0x55d0eb31aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:35.102 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:35.100+0000 7fcadca00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d0eb325230 0x55d0eb31aa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37064/0 (socket says 172.21.15.45:37064) 2024-04-16T17:11:35.102 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:35.100+0000 7fcadca00640 1 -- 172.21.15.45:0/3149772459 learned_addr learned my addr 172.21.15.45:0/3149772459 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:11:35.102 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:35.100+0000 7fcad7e00640 1 --2- 172.21.15.45:0/3149772459 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d0eb3178a0 0x55d0eb319fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:35.102 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:35.100+0000 7fcacfe00640 1 --2- 172.21.15.45:0/3149772459 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d0eb318610 0x55d0eb31a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:35.102 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:35.100+0000 7fcadca00640 1 -- 172.21.15.45:0/3149772459 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d0eb3178a0 msgr2=0x55d0eb319fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:11:35.102 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:35.100+0000 7fcadca00640 1 --2- 172.21.15.45:0/3149772459 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d0eb3178a0 0x55d0eb319fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:35.102 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:35.100+0000 7fcadca00640 1 -- 172.21.15.45:0/3149772459 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d0eb318610 msgr2=0x55d0eb31a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:11:35.102 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:35.100+0000 7fcadca00640 1 --2- 172.21.15.45:0/3149772459 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d0eb318610 0x55d0eb31a500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:35.102 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:35.100+0000 7fcadca00640 1 -- 172.21.15.45:0/3149772459 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d0eb05b620 con 0x55d0eb325230 2024-04-16T17:11:35.102 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:35.100+0000 7fcacfe00640 1 --2- 172.21.15.45:0/3149772459 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d0eb318610 0x55d0eb31a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:11:35.102 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:35.100+0000 7fcad7e00640 1 --2- 172.21.15.45:0/3149772459 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d0eb3178a0 0x55d0eb319fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:11:35.102 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:35.101+0000 7fcadca00640 1 --2- 172.21.15.45:0/3149772459 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d0eb325230 0x55d0eb31aa40 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fcad000ffa0 tx=0x7fcad000f840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4a8b3da6ccf411a server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:11:35.102 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:35.101+0000 7fcac5600640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-16T17:11:35.103 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:35.101+0000 7fcac5600640 1 -- 172.21.15.45:0/3149772459 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcad002b070 con 0x55d0eb325230 2024-04-16T17:11:35.104 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:35.101+0000 7fcae15fc740 1 -- 172.21.15.45:0/3149772459 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcad001bb90 con 0x55d0eb325230 2024-04-16T17:11:35.104 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:35.101+0000 7fcac5600640 1 -- 172.21.15.45:0/3149772459 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fcad0026d30 con 0x55d0eb325230 2024-04-16T17:11:35.104 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:35.101+0000 7fcac5600640 1 -- 172.21.15.45:0/3149772459 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fcad003d050 con 0x55d0eb325230 2024-04-16T17:11:35.104 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:35.101+0000 7fcae15fc740 10 client.16809 fetch_fsmap learned FSMap version 33 2024-04-16T17:11:35.104 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:35.101+0000 7fcae15fc740 1 -- 172.21.15.45:0/3149772459 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55d0eb338380 con 0x55d0eb325230 2024-04-16T17:11:35.104 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:35.102+0000 7fcac5600640 1 -- 172.21.15.45:0/3149772459 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcad001bb90 con 0x55d0eb325230 2024-04-16T17:11:35.104 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:35.102+0000 7fcac5600640 1 -- 172.21.15.45:0/3149772459 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 33) ==== 8358+0+0 (secure 0 0 0) 0x7fcad00057f0 con 0x55d0eb325230 2024-04-16T17:11:35.104 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:35.102+0000 7fcae15fc740 10 client.16809 fetch_fsmap finished waiting for FSMap version 33 2024-04-16T17:11:35.104 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:35.102+0000 7fcae15fc740 10 client.16809 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T17:11:35.104 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:35.102+0000 7fcae15fc740 20 client.16809 populate_metadata read hostname 'smithi045' 2024-04-16T17:11:35.104 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:35.102+0000 7fcae15fc740 1 --2- 172.21.15.45:0/3149772459 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55d0eb3397e0 0x55d0eb33bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:35.104 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:35.102+0000 7fcae15fc740 4 client.16809 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:11:35.104 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:35.102+0000 7fcae15fc740 1 -- 172.21.15.45:0/3149772459 --> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d0eb28a1a0 con 0x55d0eb3397e0 2024-04-16T17:11:35.104 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:35.103+0000 7fcad7e00640 1 --2- 172.21.15.45:0/3149772459 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55d0eb3397e0 0x55d0eb33bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:35.105 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:35.104+0000 7fcad7e00640 1 --2- 172.21.15.45:0/3149772459 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55d0eb3397e0 0x55d0eb33bbd0 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5d6c5a9cf3b1eecf server_cookie=a382c520aeaf51ce in_seq=0 out_seq=0 2024-04-16T17:11:35.105 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:35.104+0000 7fcac5600640 10 client.16809 ms_handle_connect on v2:172.21.15.104:6838/3660070592 2024-04-16T17:11:35.106 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:35.105+0000 7fcac5600640 1 -- 172.21.15.45:0/3149772459 <== mds.0 v2:172.21.15.104:6838/3660070592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d0eb28a1a0 con 0x55d0eb3397e0 2024-04-16T17:11:35.106 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:35.105+0000 7fcac5600640 10 client.16809 handle_command_reply: tid=0 2024-04-16T17:11:35.106 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:35.105+0000 7fcae15fc740 1 client.16809 shutdown 2024-04-16T17:11:35.106 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:35.105+0000 7fcae15fc740 1 -- 172.21.15.45:0/3149772459 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d0eb325230 msgr2=0x55d0eb31aa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:35.106 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:35.105+0000 7fcae15fc740 1 --2- 172.21.15.45:0/3149772459 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d0eb325230 0x55d0eb31aa40 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fcad000ffa0 tx=0x7fcad000f840 comp rx=0 tx=0).stop 2024-04-16T17:11:35.106 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:35.105+0000 7fcae15fc740 1 -- 172.21.15.45:0/3149772459 shutdown_connections 2024-04-16T17:11:35.106 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:35.105+0000 7fcae15fc740 1 --2- 172.21.15.45:0/3149772459 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d0eb325230 0x55d0eb31aa40 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:35.106 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:35.105+0000 7fcae15fc740 1 --2- 172.21.15.45:0/3149772459 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d0eb318610 0x55d0eb31a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:35.106 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:35.105+0000 7fcae15fc740 1 --2- 172.21.15.45:0/3149772459 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55d0eb3397e0 0x55d0eb33bbd0 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:35.106 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:35.105+0000 7fcae15fc740 1 --2- 172.21.15.45:0/3149772459 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d0eb3178a0 0x55d0eb319fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:35.107 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:35.105+0000 7fcae15fc740 1 -- 172.21.15.45:0/3149772459 >> 172.21.15.45:0/3149772459 conn(0x55d0eb2b25d0 msgr2=0x55d0eb3142e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:11:35.107 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:35.105+0000 7fcac5600640 0 client.16809 ms_handle_reset on v2:172.21.15.104:6838/3660070592 2024-04-16T17:11:35.107 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:35.105+0000 7fcae15fc740 1 -- 172.21.15.45:0/3149772459 shutdown_connections 2024-04-16T17:11:35.107 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:35.105+0000 7fcae15fc740 1 -- 172.21.15.45:0/3149772459 wait complete. 2024-04-16T17:11:35.107 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:35.106+0000 7fcae15fc740 20 client.16809 trim_cache size 0 max 16384 2024-04-16T17:11:35.126 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:35.126+0000 7fcae15fc740 1 Processor -- start 2024-04-16T17:11:35.126 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:35.126+0000 7fcae15fc740 1 -- start start 2024-04-16T17:11:35.127 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:35.126+0000 7fcae15fc740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d0eb3178a0 0x55d0eb311530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:35.127 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:35.126+0000 7fcae15fc740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d0eb318610 0x55d0eb341420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:35.128 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:35.126+0000 7fcae15fc740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d0eb325230 0x55d0eb341960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:35.128 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:35.126+0000 7fcae15fc740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fcad81afae0 con 0x55d0eb318610 2024-04-16T17:11:35.128 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:35.126+0000 7fcae15fc740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fcad81ad350 con 0x55d0eb325230 2024-04-16T17:11:35.128 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:35.126+0000 7fcae15fc740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fcad81ab180 con 0x55d0eb3178a0 2024-04-16T17:11:35.128 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:35.127+0000 7fcacfe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d0eb318610 0x55d0eb341420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:35.128 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:35.127+0000 7fcad7e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d0eb3178a0 0x55d0eb311530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:35.128 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:35.127+0000 7fcadca00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d0eb325230 0x55d0eb341960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:35.128 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:35.127+0000 7fcacfe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d0eb318610 0x55d0eb341420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37070/0 (socket says 172.21.15.45:37070) 2024-04-16T17:11:35.128 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:35.127+0000 7fcad7e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d0eb3178a0 0x55d0eb311530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:50736/0 (socket says 172.21.15.45:50736) 2024-04-16T17:11:35.129 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:35.127+0000 7fcacfe00640 1 -- 172.21.15.45:0/836782583 learned_addr learned my addr 172.21.15.45:0/836782583 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:11:35.129 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:35.127+0000 7fcadca00640 1 -- 172.21.15.45:0/836782583 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d0eb3178a0 msgr2=0x55d0eb311530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:11:35.129 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:35.127+0000 7fcadca00640 1 --2- 172.21.15.45:0/836782583 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d0eb3178a0 0x55d0eb311530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:35.129 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:35.127+0000 7fcadca00640 1 -- 172.21.15.45:0/836782583 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d0eb318610 msgr2=0x55d0eb341420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:11:35.129 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:35.127+0000 7fcadca00640 1 --2- 172.21.15.45:0/836782583 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d0eb318610 0x55d0eb341420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:35.129 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:35.127+0000 7fcadca00640 1 -- 172.21.15.45:0/836782583 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d0eb316b70 con 0x55d0eb325230 2024-04-16T17:11:35.130 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:35.128+0000 7fcadca00640 1 --2- 172.21.15.45:0/836782583 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d0eb325230 0x55d0eb341960 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fcad0004fd0 tx=0x7fcad0005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=37133fc8c53506e5 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:11:35.130 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:35.128+0000 7fcac7e00640 1 -- 172.21.15.45:0/836782583 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcad0046070 con 0x55d0eb325230 2024-04-16T17:11:35.130 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:35.128+0000 7fcac7e00640 1 -- 172.21.15.45:0/836782583 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcad002c040 con 0x55d0eb325230 2024-04-16T17:11:35.130 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:35.128+0000 7fcac7e00640 1 -- 172.21.15.45:0/836782583 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcad002b070 con 0x55d0eb325230 2024-04-16T17:11:35.130 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:35.129+0000 7fcae15fc740 1 -- 172.21.15.45:0/836782583 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d0eb325230 msgr2=0x55d0eb341960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:35.131 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:35.129+0000 7fcae15fc740 1 --2- 172.21.15.45:0/836782583 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d0eb325230 0x55d0eb341960 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fcad0004fd0 tx=0x7fcad0005fd0 comp rx=0 tx=0).stop 2024-04-16T17:11:35.131 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:35.129+0000 7fcae15fc740 1 -- 172.21.15.45:0/836782583 shutdown_connections 2024-04-16T17:11:35.131 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:35.129+0000 7fcae15fc740 1 --2- 172.21.15.45:0/836782583 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d0eb325230 0x55d0eb341960 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:35.131 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:35.129+0000 7fcae15fc740 1 --2- 172.21.15.45:0/836782583 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d0eb318610 0x55d0eb341420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:35.131 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:35.129+0000 7fcae15fc740 1 --2- 172.21.15.45:0/836782583 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d0eb3178a0 0x55d0eb311530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:35.131 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:35.129+0000 7fcae15fc740 1 -- 172.21.15.45:0/836782583 >> 172.21.15.45:0/836782583 conn(0x55d0eb2b25d0 msgr2=0x55d0eb310e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:11:35.131 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:35.129+0000 7fcae15fc740 1 -- 172.21.15.45:0/836782583 shutdown_connections 2024-04-16T17:11:35.131 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:35.129+0000 7fcae15fc740 1 -- 172.21.15.45:0/836782583 wait complete. 2024-04-16T17:11:35.131 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:35.130+0000 7fcae15fc740 1 -- start start 2024-04-16T17:11:35.131 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:35.130+0000 7fcae15fc740 1 Processor -- start 2024-04-16T17:11:35.134 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:35.130+0000 7fcae15fc740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d0eb3178a0 0x55d0eb331530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:35.134 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:35.130+0000 7fcae15fc740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d0eb318610 0x55d0eb331a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:35.134 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:35.131+0000 7fcae15fc740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d0eb325230 0x55d0eb331fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:35.134 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:35.131+0000 7fcae15fc740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fcad81ab810 con 0x55d0eb318610 2024-04-16T17:11:35.134 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:35.131+0000 7fcae15fc740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fcad81acc80 con 0x55d0eb3178a0 2024-04-16T17:11:35.134 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:35.131+0000 7fcae15fc740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fcad81abee0 con 0x55d0eb325230 2024-04-16T17:11:35.134 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:35.131+0000 7fcae15fc740 10 client.0 _ll_register_callbacks cb 0x7fcad0011780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:11:35.134 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:35.131+0000 7fcad7e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d0eb3178a0 0x55d0eb331530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:35.134 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:35.131+0000 7fcad7e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d0eb3178a0 0x55d0eb331530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:44148/0 (socket says 172.21.15.45:44148) 2024-04-16T17:11:35.134 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:35.131+0000 7fcad7e00640 1 -- 172.21.15.45:0/3128627101 learned_addr learned my addr 172.21.15.45:0/3128627101 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:11:35.135 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:35.131+0000 7fcacfe00640 1 --2- 172.21.15.45:0/3128627101 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d0eb318610 0x55d0eb331a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:35.135 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:35.131+0000 7fcadca00640 1 --2- 172.21.15.45:0/3128627101 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d0eb325230 0x55d0eb331fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:35.135 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:35.131+0000 7fcad7e00640 1 -- 172.21.15.45:0/3128627101 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d0eb325230 msgr2=0x55d0eb331fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:11:35.135 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:35.131+0000 7fcad7e00640 1 --2- 172.21.15.45:0/3128627101 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d0eb325230 0x55d0eb331fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:35.135 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:35.131+0000 7fcad7e00640 1 -- 172.21.15.45:0/3128627101 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d0eb318610 msgr2=0x55d0eb331a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:11:35.135 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:35.131+0000 7fcad7e00640 1 --2- 172.21.15.45:0/3128627101 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d0eb318610 0x55d0eb331a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:35.135 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:35.131+0000 7fcad7e00640 1 -- 172.21.15.45:0/3128627101 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d0eb39ec90 con 0x55d0eb3178a0 2024-04-16T17:11:35.135 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:35.131+0000 7fcadca00640 1 --2- 172.21.15.45:0/3128627101 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d0eb325230 0x55d0eb331fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:11:35.135 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:35.132+0000 7fcad7e00640 1 --2- 172.21.15.45:0/3128627101 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d0eb3178a0 0x55d0eb331530 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fcac8000c00 tx=0x7fcac802ca90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6aa22f1550fb10b1 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:11:35.135 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:35.132+0000 7fcac5600640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-04-16T17:11:35.135 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:35.132+0000 7fcac5600640 1 -- 172.21.15.45:0/3128627101 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcac80c2800 con 0x55d0eb3178a0 2024-04-16T17:11:35.135 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:35.132+0000 7fcac5600640 1 -- 172.21.15.45:0/3128627101 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fcac802cb20 con 0x55d0eb3178a0 2024-04-16T17:11:35.135 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:35.132+0000 7fcae15fc740 1 -- 172.21.15.45:0/3128627101 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcad002c040 con 0x55d0eb3178a0 2024-04-16T17:11:35.135 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:35.132+0000 7fcac5600640 1 -- 172.21.15.45:0/3128627101 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fcac80c70c0 con 0x55d0eb3178a0 2024-04-16T17:11:35.135 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:35.132+0000 7fcae15fc740 10 client.26966 fetch_fsmap learned FSMap version 33 2024-04-16T17:11:35.135 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:35.132+0000 7fcae15fc740 1 -- 172.21.15.45:0/3128627101 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55d0eb39fee0 con 0x55d0eb3178a0 2024-04-16T17:11:35.136 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:35.132+0000 7fcac5600640 1 -- 172.21.15.45:0/3128627101 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcad002c040 con 0x55d0eb3178a0 2024-04-16T17:11:35.136 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:35.132+0000 7fcac5600640 1 -- 172.21.15.45:0/3128627101 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 33) ==== 8358+0+0 (secure 0 0 0) 0x7fcac80f23e0 con 0x55d0eb3178a0 2024-04-16T17:11:35.136 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:35.133+0000 7fcae15fc740 10 client.26966 fetch_fsmap finished waiting for FSMap version 33 2024-04-16T17:11:35.136 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:35.133+0000 7fcae15fc740 10 client.26966 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T17:11:35.136 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:35.133+0000 7fcae15fc740 20 client.26966 populate_metadata read hostname 'smithi045' 2024-04-16T17:11:35.136 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:35.133+0000 7fcae15fc740 1 --2- 172.21.15.45:0/3128627101 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55d0eb3a0600 0x55d0eb3a2a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:35.136 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:35.133+0000 7fcae15fc740 4 client.26966 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-16T17:11:35.136 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:35.133+0000 7fcae15fc740 1 -- 172.21.15.45:0/3128627101 --> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d0eb284010 con 0x55d0eb3a0600 2024-04-16T17:11:35.136 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:35.133+0000 7fcacfe00640 1 --2- 172.21.15.45:0/3128627101 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55d0eb3a0600 0x55d0eb3a2a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:35.136 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:35.134+0000 7fcacfe00640 1 --2- 172.21.15.45:0/3128627101 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55d0eb3a0600 0x55d0eb3a2a20 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4a8079f7c78e3b0f server_cookie=c86eaf3913ac2f8f in_seq=0 out_seq=0 2024-04-16T17:11:35.136 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:35.134+0000 7fcac5600640 10 client.26966 ms_handle_connect on v2:172.21.15.104:6838/3660070592 2024-04-16T17:11:35.193 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:35.192+0000 7fcac5600640 1 -- 172.21.15.45:0/3128627101 <== mds.0 v2:172.21.15.104:6838/3660070592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d0eb284010 con 0x55d0eb3a0600 2024-04-16T17:11:35.193 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:35.192+0000 7fcac5600640 10 client.26966 handle_command_reply: tid=0 2024-04-16T17:11:35.193 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:35.192+0000 7fcae15fc740 1 client.26966 shutdown 2024-04-16T17:11:35.193 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:35.192+0000 7fcae15fc740 1 -- 172.21.15.45:0/3128627101 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d0eb3178a0 msgr2=0x55d0eb331530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:35.194 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:35.192+0000 7fcae15fc740 1 --2- 172.21.15.45:0/3128627101 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d0eb3178a0 0x55d0eb331530 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fcac8000c00 tx=0x7fcac802ca90 comp rx=0 tx=0).stop 2024-04-16T17:11:35.194 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:35.192+0000 7fcae15fc740 1 -- 172.21.15.45:0/3128627101 shutdown_connections 2024-04-16T17:11:35.194 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:35.192+0000 7fcae15fc740 1 --2- 172.21.15.45:0/3128627101 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55d0eb3a0600 0x55d0eb3a2a20 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:35.194 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:35.192+0000 7fcae15fc740 1 --2- 172.21.15.45:0/3128627101 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d0eb325230 0x55d0eb331fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:35.194 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:35.192+0000 7fcae15fc740 1 --2- 172.21.15.45:0/3128627101 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d0eb318610 0x55d0eb331a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:35.194 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:35.192+0000 7fcae15fc740 1 --2- 172.21.15.45:0/3128627101 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d0eb3178a0 0x55d0eb331530 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:35.194 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:35.192+0000 7fcac5600640 0 client.26966 ms_handle_reset on v2:172.21.15.104:6838/3660070592 2024-04-16T17:11:35.194 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:35.192+0000 7fcae15fc740 1 -- 172.21.15.45:0/3128627101 >> 172.21.15.45:0/3128627101 conn(0x55d0eb2b25d0 msgr2=0x55d0eb31de60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:11:35.194 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:35.193+0000 7fcae15fc740 1 -- 172.21.15.45:0/3128627101 shutdown_connections 2024-04-16T17:11:35.194 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:35.193+0000 7fcae15fc740 1 -- 172.21.15.45:0/3128627101 wait complete. 2024-04-16T17:11:35.194 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:35.193+0000 7fcae15fc740 20 client.26966 trim_cache size 0 max 16384 2024-04-16T17:11:35.194 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-04-16T17:11:35.195 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-04-16T17:11:35.195 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-04-16T17:11:35.195 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-04-16T17:11:35.195 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:35.193+0000 7fcadea00640 1 -- 172.21.15.45:0/3759086755 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fcabc0a3dc0 msgr2=0x7fcabc0a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:35.195 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:35.193+0000 7fcadea00640 1 --2- 172.21.15.45:0/3759086755 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fcabc0a3dc0 0x7fcabc0a6280 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fcad0023000 tx=0x7fcad0024000 comp rx=0 tx=0).stop 2024-04-16T17:11:35.195 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:35.194+0000 7fcadea00640 1 -- 172.21.15.45:0/3759086755 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcad8102290 msgr2=0x7fcad819e5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:35.195 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:35.194+0000 7fcadea00640 1 --2- 172.21.15.45:0/3759086755 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcad8102290 0x7fcad819e5e0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fcac802c240 tx=0x7fcac80042f0 comp rx=0 tx=0).stop 2024-04-16T17:11:35.196 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:35.194+0000 7fcadea00640 1 -- 172.21.15.45:0/3759086755 shutdown_connections 2024-04-16T17:11:35.196 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:35.194+0000 7fcadea00640 1 --2- 172.21.15.45:0/3759086755 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fcabc0a3dc0 0x7fcabc0a6280 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:35.196 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:35.194+0000 7fcadea00640 1 --2- 172.21.15.45:0/3759086755 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcad8107360 0x7fcad819eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:35.196 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:35.194+0000 7fcadea00640 1 --2- 172.21.15.45:0/3759086755 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcad81986a0 0x7fcad8198b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:35.196 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:35.194+0000 7fcadea00640 1 --2- 172.21.15.45:0/3759086755 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcad8102290 0x7fcad819e5e0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:35.196 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:35.194+0000 7fcadea00640 1 -- 172.21.15.45:0/3759086755 >> 172.21.15.45:0/3759086755 conn(0x7fcad80fdab0 msgr2=0x7fcad80ffd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:11:35.196 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:35.194+0000 7fcadea00640 1 -- 172.21.15.45:0/3759086755 shutdown_connections 2024-04-16T17:11:35.196 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:35.194+0000 7fcadea00640 1 -- 172.21.15.45:0/3759086755 wait complete. 2024-04-16T17:11:35.250 INFO:tasks.workunit.client.0.smithi045.stdout: 1 71084 1.58 MB/sec execute 97 sec latency 38.397 ms 2024-04-16T17:11:35.598 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:11:34 smithi045 ceph-mon[31676]: pgmap v147: 129 pgs: 129 active+clean; 43 MiB data, 572 MiB used, 1.0 TiB / 1.0 TiB avail; 729 KiB/s rd, 756 KiB/s wr, 81 op/s 2024-04-16T17:11:35.598 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:11:34 smithi045 ceph-mon[31676]: scrub summary: idle 2024-04-16T17:11:35.598 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:11:34 smithi045 ceph-mon[31676]: scrub summary: idle 2024-04-16T17:11:35.598 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:11:34 smithi045 ceph-mon[31676]: scrub summary: idle 2024-04-16T17:11:36.250 INFO:tasks.workunit.client.0.smithi045.stdout: 1 71412 1.57 MB/sec execute 98 sec latency 19.428 ms 2024-04-16T17:11:36.677 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:11:36 smithi104 ceph-mon[36457]: pgmap v148: 129 pgs: 129 active+clean; 43 MiB data, 572 MiB used, 1.0 TiB / 1.0 TiB avail; 729 KiB/s rd, 756 KiB/s wr, 81 op/s 2024-04-16T17:11:36.804 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:11:36 smithi117 ceph-mon[36730]: pgmap v148: 129 pgs: 129 active+clean; 43 MiB data, 572 MiB used, 1.0 TiB / 1.0 TiB avail; 729 KiB/s rd, 756 KiB/s wr, 81 op/s 2024-04-16T17:11:37.250 INFO:tasks.workunit.client.0.smithi045.stdout: 1 71924 1.57 MB/sec execute 99 sec latency 32.124 ms 2024-04-16T17:11:38.250 INFO:tasks.workunit.client.0.smithi045.stdout: 1 72194 1.56 MB/sec execute 100 sec latency 67.966 ms 2024-04-16T17:11:38.677 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:11:38 smithi104 ceph-mon[36457]: pgmap v149: 129 pgs: 129 active+clean; 41 MiB data, 574 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 933 KiB/s wr, 129 op/s 2024-04-16T17:11:38.804 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:11:38 smithi117 ceph-mon[36730]: pgmap v149: 129 pgs: 129 active+clean; 41 MiB data, 574 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 933 KiB/s wr, 129 op/s 2024-04-16T17:11:38.852 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:11:36 smithi045 ceph-mon[31676]: pgmap v148: 129 pgs: 129 active+clean; 43 MiB data, 572 MiB used, 1.0 TiB / 1.0 TiB avail; 729 KiB/s rd, 756 KiB/s wr, 81 op/s 2024-04-16T17:11:38.853 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:11:38 smithi045 ceph-mon[31676]: pgmap v149: 129 pgs: 129 active+clean; 41 MiB data, 574 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 933 KiB/s wr, 129 op/s 2024-04-16T17:11:39.250 INFO:tasks.workunit.client.0.smithi045.stdout: 1 72622 1.58 MB/sec execute 101 sec latency 70.033 ms 2024-04-16T17:11:40.250 INFO:tasks.workunit.client.0.smithi045.stdout: 1 72967 1.57 MB/sec execute 102 sec latency 67.218 ms 2024-04-16T17:11:40.677 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:11:40 smithi104 ceph-mon[36457]: pgmap v150: 129 pgs: 129 active+clean; 41 MiB data, 574 MiB used, 1.0 TiB / 1.0 TiB avail; 662 KiB/s rd, 406 KiB/s wr, 79 op/s 2024-04-16T17:11:40.804 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:11:40 smithi117 ceph-mon[36730]: pgmap v150: 129 pgs: 129 active+clean; 41 MiB data, 574 MiB used, 1.0 TiB / 1.0 TiB avail; 662 KiB/s rd, 406 KiB/s wr, 79 op/s 2024-04-16T17:11:41.181 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:11:40 smithi045 ceph-mon[31676]: pgmap v150: 129 pgs: 129 active+clean; 41 MiB data, 574 MiB used, 1.0 TiB / 1.0 TiB avail; 662 KiB/s rd, 406 KiB/s wr, 79 op/s 2024-04-16T17:11:41.251 INFO:tasks.workunit.client.0.smithi045.stdout: 1 74005 1.62 MB/sec execute 103 sec latency 37.942 ms 2024-04-16T17:11:42.251 INFO:tasks.workunit.client.0.smithi045.stdout: 1 74804 1.66 MB/sec execute 104 sec latency 22.958 ms 2024-04-16T17:11:42.606 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:11:42 smithi045 ceph-mon[31676]: pgmap v151: 129 pgs: 129 active+clean; 43 MiB data, 587 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 613 KiB/s wr, 111 op/s 2024-04-16T17:11:42.677 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:11:42 smithi104 ceph-mon[36457]: pgmap v151: 129 pgs: 129 active+clean; 43 MiB data, 587 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 613 KiB/s wr, 111 op/s 2024-04-16T17:11:42.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:11:42 smithi117 ceph-mon[36730]: pgmap v151: 129 pgs: 129 active+clean; 43 MiB data, 587 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 613 KiB/s wr, 111 op/s 2024-04-16T17:11:43.251 INFO:tasks.workunit.client.0.smithi045.stdout: 1 75055 1.65 MB/sec execute 105 sec latency 67.021 ms 2024-04-16T17:11:44.251 INFO:tasks.workunit.client.0.smithi045.stdout: 1 75457 1.64 MB/sec execute 106 sec latency 59.957 ms 2024-04-16T17:11:44.677 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:11:44 smithi104 ceph-mon[36457]: pgmap v152: 129 pgs: 129 active+clean; 43 MiB data, 587 MiB used, 1.0 TiB / 1.0 TiB avail; 831 KiB/s rd, 385 KiB/s wr, 78 op/s 2024-04-16T17:11:44.804 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:11:44 smithi117 ceph-mon[36730]: pgmap v152: 129 pgs: 129 active+clean; 43 MiB data, 587 MiB used, 1.0 TiB / 1.0 TiB avail; 831 KiB/s rd, 385 KiB/s wr, 78 op/s 2024-04-16T17:11:44.876 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:11:44 smithi045 ceph-mon[31676]: pgmap v152: 129 pgs: 129 active+clean; 43 MiB data, 587 MiB used, 1.0 TiB / 1.0 TiB avail; 831 KiB/s rd, 385 KiB/s wr, 78 op/s 2024-04-16T17:11:45.256 INFO:tasks.workunit.client.0.smithi045.stdout: 1 76018 1.67 MB/sec execute 107 sec latency 77.030 ms 2024-04-16T17:11:46.262 INFO:tasks.workunit.client.0.smithi045.stdout: 1 76328 1.66 MB/sec execute 108 sec latency 59.557 ms 2024-04-16T17:11:46.669 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:11:46 smithi045 ceph-mon[31676]: pgmap v153: 129 pgs: 129 active+clean; 43 MiB data, 587 MiB used, 1.0 TiB / 1.0 TiB avail; 831 KiB/s rd, 385 KiB/s wr, 78 op/s 2024-04-16T17:11:46.677 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:11:46 smithi104 ceph-mon[36457]: pgmap v153: 129 pgs: 129 active+clean; 43 MiB data, 587 MiB used, 1.0 TiB / 1.0 TiB avail; 831 KiB/s rd, 385 KiB/s wr, 78 op/s 2024-04-16T17:11:46.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:11:46 smithi117 ceph-mon[36730]: pgmap v153: 129 pgs: 129 active+clean; 43 MiB data, 587 MiB used, 1.0 TiB / 1.0 TiB avail; 831 KiB/s rd, 385 KiB/s wr, 78 op/s 2024-04-16T17:11:47.263 INFO:tasks.workunit.client.0.smithi045.stdout: 1 76389 1.64 MB/sec execute 109 sec latency 681.600 ms 2024-04-16T17:11:47.677 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:11:47 smithi104 ceph-mon[36457]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-16T17:11:47.804 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:11:47 smithi117 ceph-mon[36730]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-16T17:11:47.838 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:11:47 smithi045 ceph-mon[31676]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-16T17:11:48.262 INFO:tasks.workunit.client.0.smithi045.stdout: 1 76389 1.63 MB/sec execute 110 sec latency 1681.701 ms 2024-04-16T17:11:48.329 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:48.328+0000 7f6ef4e00640 1 -- 172.21.15.45:0/877311551 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6ef0104450 msgr2=0x7f6ef0104830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:48.329 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:48.328+0000 7f6ef4e00640 1 --2- 172.21.15.45:0/877311551 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6ef0104450 0x7f6ef0104830 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f6ee0009be0 tx=0x7f6ee002bef0 comp rx=0 tx=0).stop 2024-04-16T17:11:48.329 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:48.328+0000 7f6ef4e00640 1 -- 172.21.15.45:0/877311551 shutdown_connections 2024-04-16T17:11:48.329 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:48.328+0000 7f6ef4e00640 1 --2- 172.21.15.45:0/877311551 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6ef0109520 0x7f6ef01110f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:48.329 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:48.328+0000 7f6ef4e00640 1 --2- 172.21.15.45:0/877311551 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6ef0104e00 0x7f6ef0108e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:48.329 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:48.328+0000 7f6ef4e00640 1 --2- 172.21.15.45:0/877311551 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6ef0104450 0x7f6ef0104830 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:48.329 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:48.328+0000 7f6ef4e00640 1 -- 172.21.15.45:0/877311551 >> 172.21.15.45:0/877311551 conn(0x7f6ef00ffcf0 msgr2=0x7f6ef0102110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:11:48.330 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:48.328+0000 7f6ef4e00640 1 -- 172.21.15.45:0/877311551 shutdown_connections 2024-04-16T17:11:48.330 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:48.328+0000 7f6ef4e00640 1 -- 172.21.15.45:0/877311551 wait complete. 2024-04-16T17:11:48.332 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:48.329+0000 7f6ef4e00640 1 Processor -- start 2024-04-16T17:11:48.332 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:48.329+0000 7f6ef4e00640 1 -- start start 2024-04-16T17:11:48.332 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:48.330+0000 7f6ef4e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6ef0104450 0x7f6ef01a06d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:48.332 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:48.330+0000 7f6ef4e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6ef0109520 0x7f6ef01a0c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:48.332 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:48.330+0000 7f6ef4e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6ef019a890 0x7f6ef019ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:48.333 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:48.330+0000 7f6ef4e00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f6ef0114630 con 0x7f6ef0104450 2024-04-16T17:11:48.333 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:48.330+0000 7f6ef4e00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f6ef01144b0 con 0x7f6ef019a890 2024-04-16T17:11:48.333 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:48.330+0000 7f6ef4e00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f6ef01147b0 con 0x7f6ef0109520 2024-04-16T17:11:48.333 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:48.330+0000 7f6eede00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6ef0104450 0x7f6ef01a06d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:48.333 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:48.330+0000 7f6ee5400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6ef0109520 0x7f6ef01a0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:48.333 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:48.330+0000 7f6eede00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6ef0104450 0x7f6ef01a06d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53766/0 (socket says 172.21.15.45:53766) 2024-04-16T17:11:48.333 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:48.330+0000 7f6eede00640 1 -- 172.21.15.45:0/385370203 learned_addr learned my addr 172.21.15.45:0/385370203 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:11:48.333 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:48.330+0000 7f6ee5400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6ef0109520 0x7f6ef01a0c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:39568/0 (socket says 172.21.15.45:39568) 2024-04-16T17:11:48.333 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:48.330+0000 7f6eee800640 1 --2- 172.21.15.45:0/385370203 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6ef019a890 0x7f6ef019ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:48.333 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:48.331+0000 7f6eede00640 1 -- 172.21.15.45:0/385370203 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6ef0109520 msgr2=0x7f6ef01a0c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:48.333 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:48.331+0000 7f6eede00640 1 --2- 172.21.15.45:0/385370203 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6ef0109520 0x7f6ef01a0c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:48.333 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:48.331+0000 7f6eede00640 1 -- 172.21.15.45:0/385370203 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6ef019a890 msgr2=0x7f6ef019ad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:48.333 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:48.331+0000 7f6eede00640 1 --2- 172.21.15.45:0/385370203 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6ef019a890 0x7f6ef019ad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:48.333 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:48.331+0000 7f6eede00640 1 -- 172.21.15.45:0/385370203 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ef019b420 con 0x7f6ef0104450 2024-04-16T17:11:48.333 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:48.331+0000 7f6ee5400640 1 --2- 172.21.15.45:0/385370203 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6ef0109520 0x7f6ef01a0c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:11:48.333 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:48.331+0000 7f6eee800640 1 --2- 172.21.15.45:0/385370203 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6ef019a890 0x7f6ef019ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:11:48.333 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:48.331+0000 7f6eede00640 1 --2- 172.21.15.45:0/385370203 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6ef0104450 0x7f6ef01a06d0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f6ee0004af0 tx=0x7f6ee00336b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:11:48.334 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:48.331+0000 7f6ee7400640 1 -- 172.21.15.45:0/385370203 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ee0043020 con 0x7f6ef0104450 2024-04-16T17:11:48.334 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:48.331+0000 7f6ee7400640 1 -- 172.21.15.45:0/385370203 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6ee00338a0 con 0x7f6ef0104450 2024-04-16T17:11:48.334 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:48.331+0000 7f6ee7400640 1 -- 172.21.15.45:0/385370203 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ee003e6b0 con 0x7f6ef0104450 2024-04-16T17:11:48.334 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:48.331+0000 7f6ef4e00640 1 -- 172.21.15.45:0/385370203 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ef019b6b0 con 0x7f6ef0104450 2024-04-16T17:11:48.334 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:48.332+0000 7f6ef4e00640 1 -- 172.21.15.45:0/385370203 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ef01a77f0 con 0x7f6ef0104450 2024-04-16T17:11:48.335 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:48.335+0000 7f6ee7400640 1 -- 172.21.15.45:0/385370203 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f6ee0046020 con 0x7f6ef0104450 2024-04-16T17:11:48.338 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:48.336+0000 7f6ee7400640 1 --2- 172.21.15.45:0/385370203 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f6ed40a3dc0 0x7f6ed40a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:48.338 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:48.336+0000 7f6ef7bd0740 1 Processor -- start 2024-04-16T17:11:48.338 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:48.336+0000 7f6ee7400640 1 -- 172.21.15.45:0/385370203 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f6ee00f30c0 con 0x7f6ef0104450 2024-04-16T17:11:48.338 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:48.336+0000 7f6ef7bd0740 1 -- start start 2024-04-16T17:11:48.338 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:48.336+0000 7f6ee5400640 1 --2- 172.21.15.45:0/385370203 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f6ed40a3dc0 0x7f6ed40a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:48.338 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:48.337+0000 7f6ef7bd0740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5625e321c610 0x5625e321b360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:48.338 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:48.337+0000 7f6ef7bd0740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5625e321b8a0 0x5625e321bd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:48.338 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:48.337+0000 7f6ef7bd0740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5625e3224e20 0x5625e3227210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:48.338 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:48.337+0000 7f6ef7bd0740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5625e3180780 con 0x5625e321b8a0 2024-04-16T17:11:48.338 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:48.337+0000 7f6ef7bd0740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5625e3126f40 con 0x5625e321c610 2024-04-16T17:11:48.339 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:48.337+0000 7f6ef7bd0740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5625e317c740 con 0x5625e3224e20 2024-04-16T17:11:48.339 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:48.337+0000 7f6eede00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5625e321c610 0x5625e321b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:48.339 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:48.337+0000 7f6ee5400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5625e321b8a0 0x5625e321bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:48.339 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:48.337+0000 7f6eede00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5625e321c610 0x5625e321b360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:46894/0 (socket says 172.21.15.45:46894) 2024-04-16T17:11:48.339 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:48.337+0000 7f6eede00640 1 -- 172.21.15.45:0/3277713096 learned_addr learned my addr 172.21.15.45:0/3277713096 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:11:48.339 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:48.337+0000 7f6ee5400640 1 --2- 172.21.15.45:0/385370203 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f6ed40a3dc0 0x7f6ed40a6280 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f6ed80096c0 tx=0x7f6ed8009260 comp rx=0 tx=0).ready entity=mgr.25411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:11:48.339 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:48.337+0000 7f6eee800640 1 --2- 172.21.15.45:0/3277713096 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5625e3224e20 0x5625e3227210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:48.342 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:48.338+0000 7f6eede00640 1 -- 172.21.15.45:0/3277713096 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5625e3224e20 msgr2=0x5625e3227210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:11:48.342 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:48.338+0000 7f6eede00640 1 --2- 172.21.15.45:0/3277713096 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5625e3224e20 0x5625e3227210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:48.342 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:48.338+0000 7f6eede00640 1 -- 172.21.15.45:0/3277713096 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5625e321b8a0 msgr2=0x5625e321bd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:11:48.342 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:48.338+0000 7f6eede00640 1 --2- 172.21.15.45:0/3277713096 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5625e321b8a0 0x5625e321bd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:48.342 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:48.338+0000 7f6eede00640 1 -- 172.21.15.45:0/3277713096 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5625e2e93210 con 0x5625e321c610 2024-04-16T17:11:48.342 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:48.338+0000 7f6eede00640 1 --2- 172.21.15.45:0/3277713096 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5625e321c610 0x5625e321b360 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f6ee00bb220 tx=0x7f6ee00be9c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f31b66b8cb8027fb server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:11:48.342 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:48.338+0000 7f6edf400640 1 -- 172.21.15.45:0/3277713096 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ee00bedc0 con 0x5625e321c610 2024-04-16T17:11:48.342 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:48.338+0000 7f6edf400640 1 -- 172.21.15.45:0/3277713096 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6ee00c3170 con 0x5625e321c610 2024-04-16T17:11:48.343 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:48.338+0000 7f6edf400640 1 -- 172.21.15.45:0/3277713096 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ee00bb430 con 0x5625e321c610 2024-04-16T17:11:48.343 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:48.339+0000 7f6ef7bd0740 1 -- 172.21.15.45:0/3277713096 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5625e321c610 msgr2=0x5625e321b360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:48.343 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:48.339+0000 7f6ef7bd0740 1 --2- 172.21.15.45:0/3277713096 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5625e321c610 0x5625e321b360 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f6ee00bb220 tx=0x7f6ee00be9c0 comp rx=0 tx=0).stop 2024-04-16T17:11:48.343 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:48.339+0000 7f6ef7bd0740 1 -- 172.21.15.45:0/3277713096 shutdown_connections 2024-04-16T17:11:48.343 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:48.339+0000 7f6ef7bd0740 1 --2- 172.21.15.45:0/3277713096 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5625e3224e20 0x5625e3227210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:48.343 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:48.339+0000 7f6ef7bd0740 1 --2- 172.21.15.45:0/3277713096 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5625e321b8a0 0x5625e321bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:48.343 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:48.339+0000 7f6ef7bd0740 1 --2- 172.21.15.45:0/3277713096 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5625e321c610 0x5625e321b360 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:48.343 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:48.339+0000 7f6ef7bd0740 1 -- 172.21.15.45:0/3277713096 >> 172.21.15.45:0/3277713096 conn(0x5625e31b65d0 msgr2=0x5625e3216590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:11:48.343 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:48.339+0000 7f6ef7bd0740 1 -- 172.21.15.45:0/3277713096 shutdown_connections 2024-04-16T17:11:48.343 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:48.339+0000 7f6ef7bd0740 1 -- 172.21.15.45:0/3277713096 wait complete. 2024-04-16T17:11:48.343 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:48.340+0000 7f6ef7bd0740 1 -- start start 2024-04-16T17:11:48.343 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:48.340+0000 7f6ef7bd0740 1 Processor -- start 2024-04-16T17:11:48.343 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:48.340+0000 7f6ef7bd0740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5625e321b8a0 0x5625e3221ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:48.343 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:48.340+0000 7f6ef7bd0740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5625e321c610 0x5625e3222220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:48.343 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:48.340+0000 7f6ef7bd0740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5625e3224e20 0x5625e3222760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:48.344 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:48.340+0000 7f6ef7bd0740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5625e3227910 con 0x5625e3224e20 2024-04-16T17:11:48.344 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:48.340+0000 7f6ef7bd0740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5625e313a390 con 0x5625e321b8a0 2024-04-16T17:11:48.344 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:48.340+0000 7f6ef7bd0740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5625e3227a90 con 0x5625e321c610 2024-04-16T17:11:48.344 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:48.340+0000 7f6ef7bd0740 10 client.0 _ll_register_callbacks cb 0x5625e3123f60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:11:48.344 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:48.341+0000 7f6eede00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5625e321b8a0 0x5625e3221ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:48.344 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:48.341+0000 7f6eee800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5625e3224e20 0x5625e3222760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:48.344 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:48.341+0000 7f6eede00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5625e321b8a0 0x5625e3221ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:46900/0 (socket says 172.21.15.45:46900) 2024-04-16T17:11:48.344 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:48.341+0000 7f6eede00640 1 -- 172.21.15.45:0/976566541 learned_addr learned my addr 172.21.15.45:0/976566541 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:11:48.344 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:48.341+0000 7f6ee5400640 1 --2- 172.21.15.45:0/976566541 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5625e321c610 0x5625e3222220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:48.344 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:48.341+0000 7f6eede00640 1 -- 172.21.15.45:0/976566541 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5625e321c610 msgr2=0x5625e3222220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:11:48.344 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:48.341+0000 7f6eede00640 1 --2- 172.21.15.45:0/976566541 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5625e321c610 0x5625e3222220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:48.344 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:48.341+0000 7f6eede00640 1 -- 172.21.15.45:0/976566541 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5625e3224e20 msgr2=0x5625e3222760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:11:48.344 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:48.341+0000 7f6eede00640 1 --2- 172.21.15.45:0/976566541 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5625e3224e20 0x5625e3222760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:48.344 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:48.341+0000 7f6eede00640 1 -- 172.21.15.45:0/976566541 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5625e2f5f620 con 0x5625e321b8a0 2024-04-16T17:11:48.344 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:48.341+0000 7f6eede00640 1 --2- 172.21.15.45:0/976566541 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5625e321b8a0 0x5625e3221ce0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f6ee0005f00 tx=0x7f6ee0009520 comp rx=0 tx=0).ready entity=mon.2 client_cookie=783e8e88c5071230 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:11:48.345 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:48.341+0000 7f6ed3e00640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-04-16T17:11:48.345 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:48.341+0000 7f6ed3e00640 1 -- 172.21.15.45:0/976566541 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ee00d8260 con 0x5625e321b8a0 2024-04-16T17:11:48.345 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:48.342+0000 7f6ed3e00640 1 -- 172.21.15.45:0/976566541 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f6ee00f0bb0 con 0x5625e321b8a0 2024-04-16T17:11:48.345 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:48.342+0000 7f6ef7bd0740 1 -- 172.21.15.45:0/976566541 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6ee00c3170 con 0x5625e321b8a0 2024-04-16T17:11:48.349 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:48.342+0000 7f6ed3e00640 1 -- 172.21.15.45:0/976566541 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f6ee00f0db0 con 0x5625e321b8a0 2024-04-16T17:11:48.349 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:48.342+0000 7f6ef7bd0740 10 client.26978 fetch_fsmap learned FSMap version 34 2024-04-16T17:11:48.349 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:48.342+0000 7f6ef7bd0740 1 -- 172.21.15.45:0/976566541 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5625e3227910 con 0x5625e321b8a0 2024-04-16T17:11:48.349 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:48.342+0000 7f6ed3e00640 1 -- 172.21.15.45:0/976566541 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6ee00c3170 con 0x5625e321b8a0 2024-04-16T17:11:48.349 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:48.342+0000 7f6ed3e00640 1 -- 172.21.15.45:0/976566541 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 34) ==== 8362+0+0 (secure 0 0 0) 0x7f6ee00be330 con 0x5625e321b8a0 2024-04-16T17:11:48.349 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:48.342+0000 7f6ef7bd0740 10 client.26978 fetch_fsmap finished waiting for FSMap version 34 2024-04-16T17:11:48.349 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:48.342+0000 7f6ef7bd0740 10 client.26978 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T17:11:48.349 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:48.342+0000 7f6ef7bd0740 20 client.26978 populate_metadata read hostname 'smithi045' 2024-04-16T17:11:48.350 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:48.342+0000 7f6ef7bd0740 1 --2- 172.21.15.45:0/976566541 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x5625e3234b90 0x5625e3236f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:48.350 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:48.342+0000 7f6ef7bd0740 4 client.26978 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:11:48.350 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:48.342+0000 7f6ef7bd0740 1 -- 172.21.15.45:0/976566541 --> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5625e318e1a0 con 0x5625e3234b90 2024-04-16T17:11:48.350 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:48.343+0000 7f6ee5400640 1 --2- 172.21.15.45:0/976566541 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x5625e3234b90 0x5625e3236f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:48.350 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:48.343+0000 7f6ee5400640 1 --2- 172.21.15.45:0/976566541 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x5625e3234b90 0x5625e3236f80 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e0d1fcb471061be server_cookie=9520f649f99173e7 in_seq=0 out_seq=0 2024-04-16T17:11:48.350 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:48.343+0000 7f6ed3e00640 10 client.26978 ms_handle_connect on v2:172.21.15.104:6838/3660070592 2024-04-16T17:11:48.350 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:48.344+0000 7f6ed3e00640 1 -- 172.21.15.45:0/976566541 <== mds.0 v2:172.21.15.104:6838/3660070592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5625e318e1a0 con 0x5625e3234b90 2024-04-16T17:11:48.350 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:48.344+0000 7f6ed3e00640 10 client.26978 handle_command_reply: tid=0 2024-04-16T17:11:48.350 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:48.344+0000 7f6ef7bd0740 1 client.26978 shutdown 2024-04-16T17:11:48.350 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:48.344+0000 7f6ef7bd0740 1 -- 172.21.15.45:0/976566541 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5625e321b8a0 msgr2=0x5625e3221ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:48.350 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:48.344+0000 7f6ef7bd0740 1 --2- 172.21.15.45:0/976566541 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5625e321b8a0 0x5625e3221ce0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f6ee0005f00 tx=0x7f6ee0009520 comp rx=0 tx=0).stop 2024-04-16T17:11:48.350 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:48.344+0000 7f6ef7bd0740 1 -- 172.21.15.45:0/976566541 shutdown_connections 2024-04-16T17:11:48.350 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:48.344+0000 7f6ef7bd0740 1 --2- 172.21.15.45:0/976566541 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x5625e3234b90 0x5625e3236f80 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:48.350 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:48.344+0000 7f6ef7bd0740 1 --2- 172.21.15.45:0/976566541 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5625e321c610 0x5625e3222220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:48.351 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:48.345+0000 7f6ef7bd0740 1 --2- 172.21.15.45:0/976566541 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5625e3224e20 0x5625e3222760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:48.351 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:48.345+0000 7f6ed3e00640 0 client.26978 ms_handle_reset on v2:172.21.15.104:6838/3660070592 2024-04-16T17:11:48.351 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:48.345+0000 7f6ef7bd0740 1 --2- 172.21.15.45:0/976566541 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5625e321b8a0 0x5625e3221ce0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:48.351 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:48.345+0000 7f6ef7bd0740 1 -- 172.21.15.45:0/976566541 >> 172.21.15.45:0/976566541 conn(0x5625e31b65d0 msgr2=0x5625e32182e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:11:48.351 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:48.345+0000 7f6ef7bd0740 1 -- 172.21.15.45:0/976566541 shutdown_connections 2024-04-16T17:11:48.351 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:48.345+0000 7f6ef7bd0740 1 -- 172.21.15.45:0/976566541 wait complete. 2024-04-16T17:11:48.351 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:48.345+0000 7f6ef7bd0740 20 client.26978 trim_cache size 0 max 16384 2024-04-16T17:11:48.363 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:48.361+0000 7f6ef7bd0740 1 Processor -- start 2024-04-16T17:11:48.363 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:48.361+0000 7f6ef7bd0740 1 -- start start 2024-04-16T17:11:48.363 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:48.362+0000 7f6ef7bd0740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5625e321b8a0 0x5625e3224410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:48.363 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:48.362+0000 7f6ef7bd0740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5625e321c610 0x5625e3222580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:48.363 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:48.362+0000 7f6ef7bd0740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5625e3224e20 0x5625e323c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:48.363 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:48.362+0000 7f6ef7bd0740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f6ef01b3f90 con 0x5625e3224e20 2024-04-16T17:11:48.363 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:48.362+0000 7f6ef7bd0740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f6ef01b17d0 con 0x5625e321c610 2024-04-16T17:11:48.363 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:48.362+0000 7f6ef7bd0740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f6ef01af600 con 0x5625e321b8a0 2024-04-16T17:11:48.364 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:48.362+0000 7f6eee800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5625e3224e20 0x5625e323c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:48.364 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:48.362+0000 7f6eee800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5625e3224e20 0x5625e323c800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53792/0 (socket says 172.21.15.45:53792) 2024-04-16T17:11:48.364 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:48.362+0000 7f6eede00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5625e321b8a0 0x5625e3224410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:48.364 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:48.362+0000 7f6eee800640 1 -- 172.21.15.45:0/2317494603 learned_addr learned my addr 172.21.15.45:0/2317494603 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:11:48.364 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:48.362+0000 7f6ee5400640 1 --2- 172.21.15.45:0/2317494603 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5625e321c610 0x5625e3222580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:48.364 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:48.363+0000 7f6eede00640 1 -- 172.21.15.45:0/2317494603 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5625e321c610 msgr2=0x5625e3222580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:11:48.364 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:48.363+0000 7f6eede00640 1 --2- 172.21.15.45:0/2317494603 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5625e321c610 0x5625e3222580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:48.364 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:48.363+0000 7f6eede00640 1 -- 172.21.15.45:0/2317494603 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5625e3224e20 msgr2=0x5625e323c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:11:48.364 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:48.363+0000 7f6eede00640 1 --2- 172.21.15.45:0/2317494603 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5625e3224e20 0x5625e323c800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:48.364 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:48.363+0000 7f6eede00640 1 -- 172.21.15.45:0/2317494603 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5625e321ab70 con 0x5625e321b8a0 2024-04-16T17:11:48.368 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:48.363+0000 7f6eede00640 1 --2- 172.21.15.45:0/2317494603 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5625e321b8a0 0x5625e3224410 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f6ee00bb740 tx=0x7f6ee002c750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a9db479acf097d94 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:11:48.368 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:48.363+0000 7f6edea00640 1 -- 172.21.15.45:0/2317494603 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ee00d3860 con 0x5625e321b8a0 2024-04-16T17:11:48.368 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:48.363+0000 7f6edea00640 1 -- 172.21.15.45:0/2317494603 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6ee00bce60 con 0x5625e321b8a0 2024-04-16T17:11:48.368 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:48.363+0000 7f6edea00640 1 -- 172.21.15.45:0/2317494603 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ee00c3170 con 0x5625e321b8a0 2024-04-16T17:11:48.368 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:48.364+0000 7f6ef7bd0740 1 -- 172.21.15.45:0/2317494603 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5625e321b8a0 msgr2=0x5625e3224410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:48.368 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:48.364+0000 7f6ef7bd0740 1 --2- 172.21.15.45:0/2317494603 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5625e321b8a0 0x5625e3224410 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f6ee00bb740 tx=0x7f6ee002c750 comp rx=0 tx=0).stop 2024-04-16T17:11:48.368 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:48.364+0000 7f6ef7bd0740 1 -- 172.21.15.45:0/2317494603 shutdown_connections 2024-04-16T17:11:48.368 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:48.364+0000 7f6ef7bd0740 1 --2- 172.21.15.45:0/2317494603 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5625e3224e20 0x5625e323c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:48.368 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:48.364+0000 7f6ef7bd0740 1 --2- 172.21.15.45:0/2317494603 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5625e321c610 0x5625e3222580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:48.368 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:48.364+0000 7f6ef7bd0740 1 --2- 172.21.15.45:0/2317494603 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5625e321b8a0 0x5625e3224410 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:48.368 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:48.364+0000 7f6ef7bd0740 1 -- 172.21.15.45:0/2317494603 >> 172.21.15.45:0/2317494603 conn(0x5625e31b65d0 msgr2=0x5625e32156b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:11:48.368 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:48.364+0000 7f6ef7bd0740 1 -- 172.21.15.45:0/2317494603 shutdown_connections 2024-04-16T17:11:48.369 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:48.364+0000 7f6ef7bd0740 1 -- 172.21.15.45:0/2317494603 wait complete. 2024-04-16T17:11:48.369 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:48.364+0000 7f6ef7bd0740 1 -- start start 2024-04-16T17:11:48.369 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:48.365+0000 7f6ef7bd0740 1 Processor -- start 2024-04-16T17:11:48.369 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:48.365+0000 7f6ef7bd0740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5625e321b8a0 0x5625e321edd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:48.369 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:48.365+0000 7f6ef7bd0740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5625e321c610 0x5625e321f310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:48.369 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:48.365+0000 7f6ef7bd0740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5625e3224e20 0x5625e329a070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:48.369 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:48.365+0000 7f6ef7bd0740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f6ef01afc90 con 0x5625e3224e20 2024-04-16T17:11:48.369 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:48.365+0000 7f6ef7bd0740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f6ef01b1100 con 0x5625e321c610 2024-04-16T17:11:48.369 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:48.365+0000 7f6ef7bd0740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f6ef01b0360 con 0x5625e321b8a0 2024-04-16T17:11:48.369 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:48.365+0000 7f6ef7bd0740 10 client.0 _ll_register_callbacks cb 0x7f6ed8009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:11:48.369 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:48.365+0000 7f6eede00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5625e321b8a0 0x5625e321edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:48.369 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:48.365+0000 7f6eede00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5625e321b8a0 0x5625e321edd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:39594/0 (socket says 172.21.15.45:39594) 2024-04-16T17:11:48.369 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:48.365+0000 7f6eede00640 1 -- 172.21.15.45:0/252977348 learned_addr learned my addr 172.21.15.45:0/252977348 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:11:48.369 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:48.365+0000 7f6eee800640 1 --2- 172.21.15.45:0/252977348 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5625e3224e20 0x5625e329a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:48.369 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:48.365+0000 7f6ee5400640 1 --2- 172.21.15.45:0/252977348 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5625e321c610 0x5625e321f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:48.369 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:48.366+0000 7f6eede00640 1 -- 172.21.15.45:0/252977348 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5625e321c610 msgr2=0x5625e321f310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:11:48.370 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:48.366+0000 7f6eede00640 1 --2- 172.21.15.45:0/252977348 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5625e321c610 0x5625e321f310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:48.370 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:48.366+0000 7f6eede00640 1 -- 172.21.15.45:0/252977348 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5625e3224e20 msgr2=0x5625e329a070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:11:48.370 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:48.366+0000 7f6eede00640 1 --2- 172.21.15.45:0/252977348 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5625e3224e20 0x5625e329a070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:48.370 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:48.366+0000 7f6eede00640 1 -- 172.21.15.45:0/252977348 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5625e329a5b0 con 0x5625e321b8a0 2024-04-16T17:11:48.370 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:48.366+0000 7f6eede00640 1 --2- 172.21.15.45:0/252977348 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5625e321b8a0 0x5625e321edd0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f6ee0033a70 tx=0x7f6ee0004930 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d734fd1252d50c16 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:11:48.370 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:48.366+0000 7f6ed3e00640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-04-16T17:11:48.370 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:48.366+0000 7f6ed3e00640 1 -- 172.21.15.45:0/252977348 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ee00bb060 con 0x5625e321b8a0 2024-04-16T17:11:48.370 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:48.366+0000 7f6ed3e00640 1 -- 172.21.15.45:0/252977348 <== mon.1 v2:172.21.15.117:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f6ee00f0bb0 con 0x5625e321b8a0 2024-04-16T17:11:48.370 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:48.366+0000 7f6ef7bd0740 1 -- 172.21.15.45:0/252977348 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5625e3188010 con 0x5625e321b8a0 2024-04-16T17:11:48.370 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:48.366+0000 7f6ed3e00640 1 -- 172.21.15.45:0/252977348 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f6ee00c3170 con 0x5625e321b8a0 2024-04-16T17:11:48.370 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:48.366+0000 7f6ef7bd0740 10 client.26929 fetch_fsmap learned FSMap version 34 2024-04-16T17:11:48.370 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:48.366+0000 7f6ef7bd0740 1 -- 172.21.15.45:0/252977348 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5625e329a5b0 con 0x5625e321b8a0 2024-04-16T17:11:48.370 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:48.367+0000 7f6ed3e00640 1 -- 172.21.15.45:0/252977348 <== mon.1 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x5625e3188010 con 0x5625e321b8a0 2024-04-16T17:11:48.370 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:48.367+0000 7f6ed3e00640 1 -- 172.21.15.45:0/252977348 <== mon.1 v2:172.21.15.117:3300/0 5 ==== fsmap(e 34) ==== 8362+0+0 (secure 0 0 0) 0x7f6ee00bdc80 con 0x5625e321b8a0 2024-04-16T17:11:48.371 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:48.367+0000 7f6ef7bd0740 10 client.26929 fetch_fsmap finished waiting for FSMap version 34 2024-04-16T17:11:48.371 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:48.367+0000 7f6ef7bd0740 10 client.26929 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T17:11:48.371 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:48.367+0000 7f6ef7bd0740 20 client.26929 populate_metadata read hostname 'smithi045' 2024-04-16T17:11:48.371 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:48.367+0000 7f6ef7bd0740 1 --2- 172.21.15.45:0/252977348 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x5625e329b9d0 0x5625e329ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:48.371 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:48.367+0000 7f6ef7bd0740 4 client.26929 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-16T17:11:48.371 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:48.367+0000 7f6ef7bd0740 1 -- 172.21.15.45:0/252977348 --> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5625e3104530 con 0x5625e329b9d0 2024-04-16T17:11:48.371 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:48.367+0000 7f6ee5400640 1 --2- 172.21.15.45:0/252977348 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x5625e329b9d0 0x5625e329ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:48.371 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:48.368+0000 7f6ee5400640 1 --2- 172.21.15.45:0/252977348 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x5625e329b9d0 0x5625e329ddc0 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eb2cc18ae1fdd68c server_cookie=e08210e6f4428e8d in_seq=0 out_seq=0 2024-04-16T17:11:48.371 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:48.368+0000 7f6ed3e00640 10 client.26929 ms_handle_connect on v2:172.21.15.104:6838/3660070592 2024-04-16T17:11:48.410 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:48.409+0000 7f6ed3e00640 1 -- 172.21.15.45:0/252977348 <== mds.0 v2:172.21.15.104:6838/3660070592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5625e3104530 con 0x5625e329b9d0 2024-04-16T17:11:48.410 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:48.409+0000 7f6ed3e00640 10 client.26929 handle_command_reply: tid=0 2024-04-16T17:11:48.410 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:48.409+0000 7f6ef7bd0740 1 client.26929 shutdown 2024-04-16T17:11:48.410 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:48.409+0000 7f6ef7bd0740 1 -- 172.21.15.45:0/252977348 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5625e321b8a0 msgr2=0x5625e321edd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:48.410 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:48.409+0000 7f6ef7bd0740 1 --2- 172.21.15.45:0/252977348 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5625e321b8a0 0x5625e321edd0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f6ee0033a70 tx=0x7f6ee0004930 comp rx=0 tx=0).stop 2024-04-16T17:11:48.411 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:48.409+0000 7f6ef7bd0740 1 -- 172.21.15.45:0/252977348 shutdown_connections 2024-04-16T17:11:48.411 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:48.409+0000 7f6ef7bd0740 1 --2- 172.21.15.45:0/252977348 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5625e3224e20 0x5625e329a070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:48.411 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:48.409+0000 7f6ef7bd0740 1 --2- 172.21.15.45:0/252977348 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5625e321c610 0x5625e321f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:48.411 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:48.409+0000 7f6ef7bd0740 1 --2- 172.21.15.45:0/252977348 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x5625e329b9d0 0x5625e329ddc0 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:48.411 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:48.409+0000 7f6ef7bd0740 1 --2- 172.21.15.45:0/252977348 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5625e321b8a0 0x5625e321edd0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:48.411 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:48.409+0000 7f6ed3e00640 0 client.26929 ms_handle_reset on v2:172.21.15.104:6838/3660070592 2024-04-16T17:11:48.411 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:48.409+0000 7f6ef7bd0740 1 -- 172.21.15.45:0/252977348 >> 172.21.15.45:0/252977348 conn(0x5625e31b65d0 msgr2=0x5625e32177c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:11:48.411 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:48.409+0000 7f6ef7bd0740 1 -- 172.21.15.45:0/252977348 shutdown_connections 2024-04-16T17:11:48.411 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:48.409+0000 7f6ef7bd0740 1 -- 172.21.15.45:0/252977348 wait complete. 2024-04-16T17:11:48.411 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:48.410+0000 7f6ef7bd0740 20 client.26929 trim_cache size 0 max 16384 2024-04-16T17:11:48.411 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-04-16T17:11:48.412 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-04-16T17:11:48.412 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-04-16T17:11:48.412 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-04-16T17:11:48.412 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:48.410+0000 7f6ef4e00640 1 -- 172.21.15.45:0/385370203 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f6ed40a3dc0 msgr2=0x7f6ed40a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:48.412 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:48.410+0000 7f6ef4e00640 1 --2- 172.21.15.45:0/385370203 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f6ed40a3dc0 0x7f6ed40a6280 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f6ed80096c0 tx=0x7f6ed8009260 comp rx=0 tx=0).stop 2024-04-16T17:11:48.412 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:48.410+0000 7f6ef4e00640 1 -- 172.21.15.45:0/385370203 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6ef0104450 msgr2=0x7f6ef01a06d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:48.413 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:48.410+0000 7f6ef4e00640 1 --2- 172.21.15.45:0/385370203 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6ef0104450 0x7f6ef01a06d0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f6ee0004af0 tx=0x7f6ee00336b0 comp rx=0 tx=0).stop 2024-04-16T17:11:48.413 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:48.411+0000 7f6ef4e00640 1 -- 172.21.15.45:0/385370203 shutdown_connections 2024-04-16T17:11:48.413 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:48.411+0000 7f6ef4e00640 1 --2- 172.21.15.45:0/385370203 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f6ed40a3dc0 0x7f6ed40a6280 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:48.413 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:48.411+0000 7f6ef4e00640 1 --2- 172.21.15.45:0/385370203 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6ef0109520 0x7f6ef01a0c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:48.413 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:48.411+0000 7f6ef4e00640 1 --2- 172.21.15.45:0/385370203 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6ef019a890 0x7f6ef019ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:48.413 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:48.411+0000 7f6ef4e00640 1 --2- 172.21.15.45:0/385370203 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6ef0104450 0x7f6ef01a06d0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:48.413 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:48.411+0000 7f6ef4e00640 1 -- 172.21.15.45:0/385370203 >> 172.21.15.45:0/385370203 conn(0x7f6ef00ffcf0 msgr2=0x7f6ef01007a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:11:48.413 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:48.411+0000 7f6ef4e00640 1 -- 172.21.15.45:0/385370203 shutdown_connections 2024-04-16T17:11:48.413 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:11:48.411+0000 7f6ef4e00640 1 -- 172.21.15.45:0/385370203 wait complete. 2024-04-16T17:11:48.677 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:11:48 smithi104 ceph-mon[36457]: pgmap v154: 129 pgs: 129 active+clean; 45 MiB data, 613 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 600 KiB/s wr, 113 op/s 2024-04-16T17:11:48.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:11:48 smithi045 ceph-mon[31676]: pgmap v154: 129 pgs: 129 active+clean; 45 MiB data, 613 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 600 KiB/s wr, 113 op/s 2024-04-16T17:11:48.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:11:48 smithi117 ceph-mon[36730]: pgmap v154: 129 pgs: 129 active+clean; 45 MiB data, 613 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 600 KiB/s wr, 113 op/s 2024-04-16T17:11:49.263 INFO:tasks.workunit.client.0.smithi045.stdout: 1 76389 1.61 MB/sec execute 111 sec latency 2681.829 ms 2024-04-16T17:11:50.263 INFO:tasks.workunit.client.0.smithi045.stdout: 1 76389 1.60 MB/sec execute 112 sec latency 3681.918 ms 2024-04-16T17:11:50.677 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:11:50 smithi104 ceph-mon[36457]: pgmap v155: 129 pgs: 129 active+clean; 45 MiB data, 613 MiB used, 1.0 TiB / 1.0 TiB avail; 834 KiB/s rd, 422 KiB/s wr, 65 op/s 2024-04-16T17:11:50.726 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:11:50 smithi045 ceph-mon[31676]: pgmap v155: 129 pgs: 129 active+clean; 45 MiB data, 613 MiB used, 1.0 TiB / 1.0 TiB avail; 834 KiB/s rd, 422 KiB/s wr, 65 op/s 2024-04-16T17:11:50.804 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:11:50 smithi117 ceph-mon[36730]: pgmap v155: 129 pgs: 129 active+clean; 45 MiB data, 613 MiB used, 1.0 TiB / 1.0 TiB avail; 834 KiB/s rd, 422 KiB/s wr, 65 op/s 2024-04-16T17:11:51.263 INFO:tasks.workunit.client.0.smithi045.stdout: 1 76923 1.60 MB/sec execute 113 sec latency 4171.848 ms 2024-04-16T17:11:52.263 INFO:tasks.workunit.client.0.smithi045.stdout: 1 78171 1.68 MB/sec execute 114 sec latency 21.996 ms 2024-04-16T17:11:52.524 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:11:52 smithi045 ceph-mon[31676]: pgmap v156: 129 pgs: 129 active+clean; 45 MiB data, 640 MiB used, 1.0 TiB / 1.0 TiB avail; 908 KiB/s rd, 1.1 MiB/s wr, 112 op/s 2024-04-16T17:11:52.677 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:11:52 smithi104 ceph-mon[36457]: pgmap v156: 129 pgs: 129 active+clean; 45 MiB data, 640 MiB used, 1.0 TiB / 1.0 TiB avail; 908 KiB/s rd, 1.1 MiB/s wr, 112 op/s 2024-04-16T17:11:52.804 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:11:52 smithi117 ceph-mon[36730]: pgmap v156: 129 pgs: 129 active+clean; 45 MiB data, 640 MiB used, 1.0 TiB / 1.0 TiB avail; 908 KiB/s rd, 1.1 MiB/s wr, 112 op/s 2024-04-16T17:11:53.263 INFO:tasks.workunit.client.0.smithi045.stdout: 1 78427 1.67 MB/sec execute 115 sec latency 187.643 ms 2024-04-16T17:11:53.677 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:11:53 smithi104 ceph-mon[36457]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-16T17:11:53.804 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:11:53 smithi117 ceph-mon[36730]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-16T17:11:54.140 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-16T17:11:54.140 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-16T17:11:54.263 INFO:tasks.workunit.client.0.smithi045.stdout: 1 78615 1.66 MB/sec execute 116 sec latency 76.326 ms 2024-04-16T17:11:54.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.334+0000 7f64a7e00640 1 -- 172.21.15.45:0/2627131836 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f64a8102120 msgr2=0x7f64a8102500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:54.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.334+0000 7f64a7e00640 1 --2- 172.21.15.45:0/2627131836 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f64a8102120 0x7f64a8102500 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f6498009900 tx=0x7f649802bc30 comp rx=0 tx=0).stop 2024-04-16T17:11:54.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.334+0000 7f64a7e00640 1 -- 172.21.15.45:0/2627131836 shutdown_connections 2024-04-16T17:11:54.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.334+0000 7f64a7e00640 1 --2- 172.21.15.45:0/2627131836 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f64a81071f0 0x7f64a810edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:54.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.334+0000 7f64a7e00640 1 --2- 172.21.15.45:0/2627131836 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f64a8102ad0 0x7f64a8106b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:54.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.334+0000 7f64a7e00640 1 --2- 172.21.15.45:0/2627131836 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f64a8102120 0x7f64a8102500 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:54.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.334+0000 7f64a7e00640 1 -- 172.21.15.45:0/2627131836 >> 172.21.15.45:0/2627131836 conn(0x7f64a80fd9a0 msgr2=0x7f64a80ffdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:11:54.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.334+0000 7f64a7e00640 1 -- 172.21.15.45:0/2627131836 shutdown_connections 2024-04-16T17:11:54.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.334+0000 7f64a7e00640 1 -- 172.21.15.45:0/2627131836 wait complete. 2024-04-16T17:11:54.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.335+0000 7f64a7e00640 1 Processor -- start 2024-04-16T17:11:54.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.335+0000 7f64a7e00640 1 -- start start 2024-04-16T17:11:54.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.336+0000 7f64a7e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f64a8102120 0x7f64a81a0650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:54.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.336+0000 7f64a7e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f64a81071f0 0x7f64a81a0b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:54.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.336+0000 7f64a7e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f64a819a7c0 0x7f64a819ac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:54.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.336+0000 7f64a7e00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f64a8112280 con 0x7f64a8102120 2024-04-16T17:11:54.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.336+0000 7f64a7e00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f64a8112100 con 0x7f64a81071f0 2024-04-16T17:11:54.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.336+0000 7f64a7e00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f64a8112400 con 0x7f64a819a7c0 2024-04-16T17:11:54.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.336+0000 7f64a6a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f64a8102120 0x7f64a81a0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:54.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.336+0000 7f64a6000640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f64a81071f0 0x7f64a81a0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:54.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.336+0000 7f64a6000640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f64a81071f0 0x7f64a81a0b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:60766/0 (socket says 172.21.15.45:60766) 2024-04-16T17:11:54.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.336+0000 7f64a7400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f64a819a7c0 0x7f64a819ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:54.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.336+0000 7f64a6000640 1 -- 172.21.15.45:0/2266732040 learned_addr learned my addr 172.21.15.45:0/2266732040 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:11:54.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.336+0000 7f64a6000640 1 -- 172.21.15.45:0/2266732040 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f64a819a7c0 msgr2=0x7f64a819ac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:54.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.336+0000 7f64a6000640 1 --2- 172.21.15.45:0/2266732040 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f64a819a7c0 0x7f64a819ac70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:54.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.336+0000 7f64a6000640 1 -- 172.21.15.45:0/2266732040 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f64a8102120 msgr2=0x7f64a81a0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:54.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.336+0000 7f64a6000640 1 --2- 172.21.15.45:0/2266732040 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f64a8102120 0x7f64a81a0650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:54.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.336+0000 7f64a6000640 1 -- 172.21.15.45:0/2266732040 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64a819b350 con 0x7f64a81071f0 2024-04-16T17:11:54.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.337+0000 7f64a6000640 1 --2- 172.21.15.45:0/2266732040 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f64a81071f0 0x7f64a81a0b90 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f649400e9c0 tx=0x7f649400edc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:11:54.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.337+0000 7f649f400640 1 -- 172.21.15.45:0/2266732040 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f649400ccc0 con 0x7f64a81071f0 2024-04-16T17:11:54.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.337+0000 7f64a7e00640 1 -- 172.21.15.45:0/2266732040 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64a819b640 con 0x7f64a81071f0 2024-04-16T17:11:54.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.337+0000 7f649f400640 1 -- 172.21.15.45:0/2266732040 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6494005500 con 0x7f64a81071f0 2024-04-16T17:11:54.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.337+0000 7f649f400640 1 -- 172.21.15.45:0/2266732040 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6494010430 con 0x7f64a81071f0 2024-04-16T17:11:54.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.337+0000 7f64a7e00640 1 -- 172.21.15.45:0/2266732040 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64a81a7870 con 0x7f64a81071f0 2024-04-16T17:11:54.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.341+0000 7f64b080c740 1 Processor -- start 2024-04-16T17:11:54.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.341+0000 7f64b080c740 1 -- start start 2024-04-16T17:11:54.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.341+0000 7f64b080c740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55daf66cb390 0x55daf66cb770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:54.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.341+0000 7f64b080c740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55daf66cbcb0 0x55daf66d4350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:54.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.341+0000 7f64b080c740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55daf66d4890 0x55daf66d6c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:54.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.341+0000 7f64b080c740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55daf6631120 con 0x55daf66cb390 2024-04-16T17:11:54.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.341+0000 7f64b080c740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55daf65d7340 con 0x55daf66cbcb0 2024-04-16T17:11:54.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.341+0000 7f64b080c740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55daf662d0e0 con 0x55daf66d4890 2024-04-16T17:11:54.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.342+0000 7f64a7400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55daf66cb390 0x55daf66cb770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:54.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.342+0000 7f64a6a00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55daf66cbcb0 0x55daf66d4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:54.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.342+0000 7f64a6a00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55daf66cbcb0 0x55daf66d4350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:60780/0 (socket says 172.21.15.45:60780) 2024-04-16T17:11:54.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.342+0000 7f64a6a00640 1 -- 172.21.15.45:0/982371801 learned_addr learned my addr 172.21.15.45:0/982371801 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:11:54.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.342+0000 7f64a6a00640 1 -- 172.21.15.45:0/982371801 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55daf66d4890 msgr2=0x55daf66d6c80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-16T17:11:54.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.342+0000 7f64a6a00640 1 --2- 172.21.15.45:0/982371801 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55daf66d4890 0x55daf66d6c80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:54.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.342+0000 7f64a6a00640 1 -- 172.21.15.45:0/982371801 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55daf66cb390 msgr2=0x55daf66cb770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:11:54.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.342+0000 7f64a6a00640 1 --2- 172.21.15.45:0/982371801 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55daf66cb390 0x55daf66cb770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:54.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.342+0000 7f64a6a00640 1 -- 172.21.15.45:0/982371801 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55daf640d7d0 con 0x55daf66cbcb0 2024-04-16T17:11:54.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.342+0000 7f64a6a00640 1 --2- 172.21.15.45:0/982371801 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55daf66cbcb0 0x55daf66d4350 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f649802cfd0 tx=0x7f6498005c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=98cc72ce49db4378 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:11:54.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.343+0000 7f649f400640 1 -- 172.21.15.45:0/2266732040 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f6494005000 con 0x7f64a81071f0 2024-04-16T17:11:54.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.343+0000 7f6493e00640 1 -- 172.21.15.45:0/982371801 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6498042070 con 0x55daf66cbcb0 2024-04-16T17:11:54.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.343+0000 7f6493e00640 1 -- 172.21.15.45:0/982371801 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f649802ca80 con 0x55daf66cbcb0 2024-04-16T17:11:54.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.343+0000 7f6493e00640 1 -- 172.21.15.45:0/982371801 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6498004f00 con 0x55daf66cbcb0 2024-04-16T17:11:54.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.343+0000 7f64b080c740 1 -- 172.21.15.45:0/982371801 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55daf66cbcb0 msgr2=0x55daf66d4350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:54.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.343+0000 7f64b080c740 1 --2- 172.21.15.45:0/982371801 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55daf66cbcb0 0x55daf66d4350 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f649802cfd0 tx=0x7f6498005c00 comp rx=0 tx=0).stop 2024-04-16T17:11:54.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.344+0000 7f64b080c740 1 -- 172.21.15.45:0/982371801 shutdown_connections 2024-04-16T17:11:54.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.344+0000 7f64b080c740 1 --2- 172.21.15.45:0/982371801 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55daf66d4890 0x55daf66d6c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:54.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.344+0000 7f64b080c740 1 --2- 172.21.15.45:0/982371801 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55daf66cbcb0 0x55daf66d4350 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:54.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.344+0000 7f64b080c740 1 --2- 172.21.15.45:0/982371801 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55daf66cb390 0x55daf66cb770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:54.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.344+0000 7f64b080c740 1 -- 172.21.15.45:0/982371801 >> 172.21.15.45:0/982371801 conn(0x55daf66c5ac0 msgr2=0x55daf66c5ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:11:54.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.344+0000 7f64b080c740 1 -- 172.21.15.45:0/982371801 shutdown_connections 2024-04-16T17:11:54.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.344+0000 7f64b080c740 1 -- 172.21.15.45:0/982371801 wait complete. 2024-04-16T17:11:54.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.345+0000 7f64b080c740 1 -- start start 2024-04-16T17:11:54.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.345+0000 7f649f400640 1 --2- 172.21.15.45:0/2266732040 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f648c0a3c30 0x7f648c0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:54.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.345+0000 7f649f400640 1 -- 172.21.15.45:0/2266732040 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f64940d0f70 con 0x7f64a81071f0 2024-04-16T17:11:54.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.345+0000 7f64a7400640 1 --2- 172.21.15.45:0/2266732040 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f648c0a3c30 0x7f648c0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:54.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.345+0000 7f64b080c740 1 Processor -- start 2024-04-16T17:11:54.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.346+0000 7f64b080c740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55daf66cb390 0x55daf66d1940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:54.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.346+0000 7f64b080c740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55daf66cbcb0 0x55daf66d1e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:54.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.346+0000 7f64b080c740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55daf66d4890 0x55daf66d23c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:54.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.346+0000 7f64b080c740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55daf66d7620 con 0x55daf66cbcb0 2024-04-16T17:11:54.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.346+0000 7f64b080c740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55daf65ea710 con 0x55daf66d4890 2024-04-16T17:11:54.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.346+0000 7f64b080c740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55daf66d77a0 con 0x55daf66cb390 2024-04-16T17:11:54.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.346+0000 7f64b080c740 10 client.0 _ll_register_callbacks cb 0x55daf65e5af0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:11:54.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.346+0000 7f64a6000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55daf66cbcb0 0x55daf66d1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:54.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.346+0000 7f64a7400640 1 --2- 172.21.15.45:0/2266732040 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f648c0a3c30 0x7f648c0a60f0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f64a0005fd0 tx=0x7f64a0009500 comp rx=0 tx=0).ready entity=mgr.25411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:11:54.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.346+0000 7f64a6000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55daf66cbcb0 0x55daf66d1e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39298/0 (socket says 172.21.15.45:39298) 2024-04-16T17:11:54.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.346+0000 7f64a6000640 1 -- 172.21.15.45:0/1452317705 learned_addr learned my addr 172.21.15.45:0/1452317705 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:11:54.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.346+0000 7f64a6a00640 1 --2- 172.21.15.45:0/1452317705 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55daf66cb390 0x55daf66d1940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:54.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.347+0000 7f64a7400640 1 --2- 172.21.15.45:0/1452317705 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55daf66d4890 0x55daf66d23c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:54.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.347+0000 7f64a6a00640 1 -- 172.21.15.45:0/1452317705 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55daf66d4890 msgr2=0x55daf66d23c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:11:54.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.347+0000 7f64a6a00640 1 --2- 172.21.15.45:0/1452317705 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55daf66d4890 0x55daf66d23c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:54.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.347+0000 7f64a6a00640 1 -- 172.21.15.45:0/1452317705 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55daf66cbcb0 msgr2=0x55daf66d1e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:11:54.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.347+0000 7f64a6a00640 1 --2- 172.21.15.45:0/1452317705 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55daf66cbcb0 0x55daf66d1e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:54.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.347+0000 7f64a6a00640 1 -- 172.21.15.45:0/1452317705 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55daf63433f0 con 0x55daf66cb390 2024-04-16T17:11:54.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.347+0000 7f64a6000640 1 --2- 172.21.15.45:0/1452317705 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55daf66cbcb0 0x55daf66d1e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:11:54.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.347+0000 7f64a7400640 1 --2- 172.21.15.45:0/1452317705 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55daf66d4890 0x55daf66d23c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:11:54.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.347+0000 7f64a6a00640 1 --2- 172.21.15.45:0/1452317705 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55daf66cb390 0x55daf66d1940 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f649802c170 tx=0x7f649802c330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=43e16febfb716a72 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:11:54.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.347+0000 7f6490c00640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-04-16T17:11:54.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.347+0000 7f6490c00640 1 -- 172.21.15.45:0/1452317705 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6498042030 con 0x55daf66cb390 2024-04-16T17:11:54.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.347+0000 7f64b080c740 1 -- 172.21.15.45:0/1452317705 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f649802ca80 con 0x55daf66cb390 2024-04-16T17:11:54.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.347+0000 7f6490c00640 1 -- 172.21.15.45:0/1452317705 <== mon.1 v2:172.21.15.117:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f64980046c0 con 0x55daf66cb390 2024-04-16T17:11:54.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.348+0000 7f6490c00640 1 -- 172.21.15.45:0/1452317705 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f6498057050 con 0x55daf66cb390 2024-04-16T17:11:54.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.348+0000 7f64b080c740 10 client.26938 fetch_fsmap learned FSMap version 35 2024-04-16T17:11:54.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.348+0000 7f64b080c740 1 -- 172.21.15.45:0/1452317705 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55daf63030f0 con 0x55daf66cb390 2024-04-16T17:11:54.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.348+0000 7f6490c00640 1 -- 172.21.15.45:0/1452317705 <== mon.1 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f649802ca80 con 0x55daf66cb390 2024-04-16T17:11:54.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.348+0000 7f6490c00640 1 -- 172.21.15.45:0/1452317705 <== mon.1 v2:172.21.15.117:3300/0 5 ==== fsmap(e 35) ==== 8358+0+0 (secure 0 0 0) 0x7f64980037f0 con 0x55daf66cb390 2024-04-16T17:11:54.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.348+0000 7f64b080c740 10 client.26938 fetch_fsmap finished waiting for FSMap version 35 2024-04-16T17:11:54.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.348+0000 7f64b080c740 10 client.26938 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:11:54.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.348+0000 7f64b080c740 20 client.26938 populate_metadata read hostname 'smithi045' 2024-04-16T17:11:54.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.348+0000 7f64b080c740 1 --2- 172.21.15.45:0/1452317705 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55daf66e4ad0 0x55daf66e6ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:54.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.348+0000 7f64b080c740 4 client.26938 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:11:54.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.348+0000 7f64b080c740 1 -- 172.21.15.45:0/1452317705 --> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55daf663eb40 con 0x55daf66e4ad0 2024-04-16T17:11:54.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.349+0000 7f64a6000640 1 --2- 172.21.15.45:0/1452317705 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55daf66e4ad0 0x55daf66e6ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:54.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.349+0000 7f64a6000640 1 --2- 172.21.15.45:0/1452317705 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55daf66e4ad0 0x55daf66e6ec0 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ccfd828a7b17fe4b server_cookie=73f69dda38122fdb in_seq=0 out_seq=0 2024-04-16T17:11:54.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.349+0000 7f6490c00640 10 client.26938 ms_handle_connect on v2:172.21.15.104:6838/3660070592 2024-04-16T17:11:54.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.350+0000 7f6490c00640 1 -- 172.21.15.45:0/1452317705 <== mds.0 v2:172.21.15.104:6838/3660070592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55daf663eb40 con 0x55daf66e4ad0 2024-04-16T17:11:54.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.350+0000 7f6490c00640 10 client.26938 handle_command_reply: tid=0 2024-04-16T17:11:54.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.350+0000 7f64b080c740 1 client.26938 shutdown 2024-04-16T17:11:54.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.350+0000 7f64b080c740 1 -- 172.21.15.45:0/1452317705 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55daf66cb390 msgr2=0x55daf66d1940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:54.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.350+0000 7f64b080c740 1 --2- 172.21.15.45:0/1452317705 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55daf66cb390 0x55daf66d1940 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f649802c170 tx=0x7f649802c330 comp rx=0 tx=0).stop 2024-04-16T17:11:54.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.350+0000 7f64b080c740 1 -- 172.21.15.45:0/1452317705 shutdown_connections 2024-04-16T17:11:54.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.350+0000 7f64b080c740 1 --2- 172.21.15.45:0/1452317705 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55daf66d4890 0x55daf66d23c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:54.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.350+0000 7f64b080c740 1 --2- 172.21.15.45:0/1452317705 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55daf66cbcb0 0x55daf66d1e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:54.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.350+0000 7f64b080c740 1 --2- 172.21.15.45:0/1452317705 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55daf66e4ad0 0x55daf66e6ec0 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:54.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.350+0000 7f64b080c740 1 --2- 172.21.15.45:0/1452317705 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55daf66cb390 0x55daf66d1940 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:54.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.350+0000 7f6490c00640 0 client.26938 ms_handle_reset on v2:172.21.15.104:6838/3660070592 2024-04-16T17:11:54.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.350+0000 7f64b080c740 1 -- 172.21.15.45:0/1452317705 >> 172.21.15.45:0/1452317705 conn(0x55daf66c5ac0 msgr2=0x55daf66c7810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:11:54.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.350+0000 7f64b080c740 1 -- 172.21.15.45:0/1452317705 shutdown_connections 2024-04-16T17:11:54.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.351+0000 7f64b080c740 1 -- 172.21.15.45:0/1452317705 wait complete. 2024-04-16T17:11:54.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.351+0000 7f64b080c740 20 client.26938 trim_cache size 0 max 16384 2024-04-16T17:11:54.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.370+0000 7f64b080c740 1 Processor -- start 2024-04-16T17:11:54.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.370+0000 7f64b080c740 1 -- start start 2024-04-16T17:11:54.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.370+0000 7f64b080c740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55daf66c5ac0 0x55daf66d4070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:54.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.370+0000 7f64b080c740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55daf66cb390 0x55daf66d08a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:54.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.370+0000 7f64b080c740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55daf66cbcb0 0x55daf66e9ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:54.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.370+0000 7f64b080c740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f64a81b3d90 con 0x55daf66cbcb0 2024-04-16T17:11:54.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.370+0000 7f64b080c740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f64a81b1600 con 0x55daf66cb390 2024-04-16T17:11:54.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.370+0000 7f64b080c740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f64a81af430 con 0x55daf66c5ac0 2024-04-16T17:11:54.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.370+0000 7f64a7400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55daf66cbcb0 0x55daf66e9ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:54.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.370+0000 7f64a6000640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55daf66cb390 0x55daf66d08a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:54.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.370+0000 7f64a6000640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55daf66cb390 0x55daf66d08a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:60808/0 (socket says 172.21.15.45:60808) 2024-04-16T17:11:54.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.370+0000 7f64a7400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55daf66cbcb0 0x55daf66e9ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39300/0 (socket says 172.21.15.45:39300) 2024-04-16T17:11:54.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.370+0000 7f64a6000640 1 -- 172.21.15.45:0/1943184034 learned_addr learned my addr 172.21.15.45:0/1943184034 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:11:54.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.370+0000 7f64a6a00640 1 --2- 172.21.15.45:0/1943184034 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55daf66c5ac0 0x55daf66d4070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:54.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.371+0000 7f64a6000640 1 -- 172.21.15.45:0/1943184034 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55daf66c5ac0 msgr2=0x55daf66d4070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:11:54.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.371+0000 7f64a6000640 1 --2- 172.21.15.45:0/1943184034 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55daf66c5ac0 0x55daf66d4070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:54.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.371+0000 7f64a6000640 1 -- 172.21.15.45:0/1943184034 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55daf66cbcb0 msgr2=0x55daf66e9ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:11:54.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.371+0000 7f64a6000640 1 --2- 172.21.15.45:0/1943184034 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55daf66cbcb0 0x55daf66e9ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:54.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.371+0000 7f64a6000640 1 -- 172.21.15.45:0/1943184034 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55daf66cbae0 con 0x55daf66cb390 2024-04-16T17:11:54.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.371+0000 7f64a6a00640 1 --2- 172.21.15.45:0/1943184034 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55daf66c5ac0 0x55daf66d4070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:11:54.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.371+0000 7f64a6000640 1 --2- 172.21.15.45:0/1943184034 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55daf66cb390 0x55daf66d08a0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f64940097b0 tx=0x7f6494005880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b57ba1e839998e0e server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:11:54.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.371+0000 7f6493400640 1 -- 172.21.15.45:0/1943184034 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64940b5030 con 0x55daf66cb390 2024-04-16T17:11:54.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.371+0000 7f6493400640 1 -- 172.21.15.45:0/1943184034 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f649409cba0 con 0x55daf66cb390 2024-04-16T17:11:54.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.371+0000 7f6493400640 1 -- 172.21.15.45:0/1943184034 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f649409aea0 con 0x55daf66cb390 2024-04-16T17:11:54.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.372+0000 7f64b080c740 1 -- 172.21.15.45:0/1943184034 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55daf66cb390 msgr2=0x55daf66d08a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:54.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.372+0000 7f64b080c740 1 --2- 172.21.15.45:0/1943184034 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55daf66cb390 0x55daf66d08a0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f64940097b0 tx=0x7f6494005880 comp rx=0 tx=0).stop 2024-04-16T17:11:54.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.372+0000 7f64b080c740 1 -- 172.21.15.45:0/1943184034 shutdown_connections 2024-04-16T17:11:54.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.372+0000 7f64b080c740 1 --2- 172.21.15.45:0/1943184034 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55daf66cbcb0 0x55daf66e9ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:54.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.372+0000 7f64b080c740 1 --2- 172.21.15.45:0/1943184034 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55daf66cb390 0x55daf66d08a0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:54.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.372+0000 7f64b080c740 1 --2- 172.21.15.45:0/1943184034 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55daf66c5ac0 0x55daf66d4070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:54.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.372+0000 7f64b080c740 1 -- 172.21.15.45:0/1943184034 >> 172.21.15.45:0/1943184034 conn(0x55daf66cd330 msgr2=0x55daf66e0280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:11:54.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.372+0000 7f64b080c740 1 -- 172.21.15.45:0/1943184034 shutdown_connections 2024-04-16T17:11:54.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.372+0000 7f64b080c740 1 -- 172.21.15.45:0/1943184034 wait complete. 2024-04-16T17:11:54.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.373+0000 7f64b080c740 1 -- start start 2024-04-16T17:11:54.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.373+0000 7f64b080c740 1 Processor -- start 2024-04-16T17:11:54.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.373+0000 7f64b080c740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55daf66c5ac0 0x55daf66e7c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:54.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.373+0000 7f64b080c740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55daf66cb390 0x55daf66e8150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:54.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.373+0000 7f64b080c740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55daf66cbcb0 0x55daf66e8690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:54.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.373+0000 7f64b080c740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f64a81afac0 con 0x55daf66c5ac0 2024-04-16T17:11:54.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.373+0000 7f64b080c740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f64a81b0f30 con 0x55daf66cbcb0 2024-04-16T17:11:54.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.373+0000 7f64b080c740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f64a81b0190 con 0x55daf66cb390 2024-04-16T17:11:54.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.373+0000 7f64b080c740 10 client.0 _ll_register_callbacks cb 0x7f64a0009900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:11:54.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.374+0000 7f64a6000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55daf66cb390 0x55daf66e8150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:54.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.374+0000 7f64a6a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55daf66c5ac0 0x55daf66e7c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:54.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.374+0000 7f64a6000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55daf66cb390 0x55daf66e8150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:44254/0 (socket says 172.21.15.45:44254) 2024-04-16T17:11:54.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.374+0000 7f64a6000640 1 -- 172.21.15.45:0/2684230730 learned_addr learned my addr 172.21.15.45:0/2684230730 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:11:54.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.374+0000 7f64a7400640 1 --2- 172.21.15.45:0/2684230730 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55daf66cbcb0 0x55daf66e8690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:54.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.374+0000 7f64a6000640 1 -- 172.21.15.45:0/2684230730 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55daf66cbcb0 msgr2=0x55daf66e8690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:11:54.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.374+0000 7f64a6000640 1 --2- 172.21.15.45:0/2684230730 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55daf66cbcb0 0x55daf66e8690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:54.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.374+0000 7f64a6000640 1 -- 172.21.15.45:0/2684230730 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55daf66c5ac0 msgr2=0x55daf66e7c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:11:54.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.374+0000 7f64a6000640 1 --2- 172.21.15.45:0/2684230730 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55daf66c5ac0 0x55daf66e7c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:54.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.374+0000 7f64a6000640 1 -- 172.21.15.45:0/2684230730 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55daf6749f50 con 0x55daf66cb390 2024-04-16T17:11:54.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.374+0000 7f64a7400640 1 --2- 172.21.15.45:0/2684230730 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55daf66cbcb0 0x55daf66e8690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:11:54.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.374+0000 7f64a6000640 1 --2- 172.21.15.45:0/2684230730 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55daf66cb390 0x55daf66e8150 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f64940a52e0 tx=0x7f6494004990 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f01e4745e73d34ac server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:11:54.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.374+0000 7f6490c00640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-04-16T17:11:54.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.374+0000 7f6490c00640 1 -- 172.21.15.45:0/2684230730 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64940b5070 con 0x55daf66cb390 2024-04-16T17:11:54.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.374+0000 7f6490c00640 1 -- 172.21.15.45:0/2684230730 <== mon.1 v2:172.21.15.117:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f6494067680 con 0x55daf66cb390 2024-04-16T17:11:54.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.374+0000 7f64b080c740 1 -- 172.21.15.45:0/2684230730 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f649409cba0 con 0x55daf66cb390 2024-04-16T17:11:54.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.375+0000 7f6490c00640 1 -- 172.21.15.45:0/2684230730 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f64940a5810 con 0x55daf66cb390 2024-04-16T17:11:54.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.375+0000 7f64b080c740 10 client.26944 fetch_fsmap learned FSMap version 35 2024-04-16T17:11:54.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.375+0000 7f64b080c740 1 -- 172.21.15.45:0/2684230730 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55daf674b260 con 0x55daf66cb390 2024-04-16T17:11:54.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.375+0000 7f6490c00640 1 -- 172.21.15.45:0/2684230730 <== mon.1 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f649409cba0 con 0x55daf66cb390 2024-04-16T17:11:54.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.375+0000 7f6490c00640 1 -- 172.21.15.45:0/2684230730 <== mon.1 v2:172.21.15.117:3300/0 5 ==== fsmap(e 35) ==== 8358+0+0 (secure 0 0 0) 0x7f64940a5380 con 0x55daf66cb390 2024-04-16T17:11:54.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.375+0000 7f64b080c740 10 client.26944 fetch_fsmap finished waiting for FSMap version 35 2024-04-16T17:11:54.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.375+0000 7f64b080c740 10 client.26944 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:11:54.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.375+0000 7f64b080c740 20 client.26944 populate_metadata read hostname 'smithi045' 2024-04-16T17:11:54.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.375+0000 7f64b080c740 1 --2- 172.21.15.45:0/2684230730 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55daf66d4890 0x55daf674d970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:54.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.375+0000 7f64b080c740 4 client.26944 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-16T17:11:54.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.375+0000 7f64b080c740 1 -- 172.21.15.45:0/2684230730 --> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55daf66389b0 con 0x55daf66d4890 2024-04-16T17:11:54.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.375+0000 7f64a6a00640 1 --2- 172.21.15.45:0/2684230730 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55daf66d4890 0x55daf674d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:54.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.376+0000 7f64a6a00640 1 --2- 172.21.15.45:0/2684230730 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55daf66d4890 0x55daf674d970 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2ecc0dd3720fecf0 server_cookie=8eb6a9e4b9bf677b in_seq=0 out_seq=0 2024-04-16T17:11:54.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.376+0000 7f6490c00640 10 client.26944 ms_handle_connect on v2:172.21.15.104:6838/3660070592 2024-04-16T17:11:54.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.376+0000 7f6490c00640 1 -- 172.21.15.45:0/2684230730 <== mds.0 v2:172.21.15.104:6838/3660070592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55daf66389b0 con 0x55daf66d4890 2024-04-16T17:11:54.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.376+0000 7f6490c00640 10 client.26944 handle_command_reply: tid=0 2024-04-16T17:11:54.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.376+0000 7f64b080c740 1 client.26944 shutdown 2024-04-16T17:11:54.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.376+0000 7f64b080c740 1 -- 172.21.15.45:0/2684230730 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55daf66cb390 msgr2=0x55daf66e8150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:54.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.376+0000 7f64b080c740 1 --2- 172.21.15.45:0/2684230730 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55daf66cb390 0x55daf66e8150 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f64940a52e0 tx=0x7f6494004990 comp rx=0 tx=0).stop 2024-04-16T17:11:54.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.376+0000 7f64b080c740 1 -- 172.21.15.45:0/2684230730 shutdown_connections 2024-04-16T17:11:54.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.376+0000 7f64b080c740 1 --2- 172.21.15.45:0/2684230730 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55daf66d4890 0x55daf674d970 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:54.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.376+0000 7f64b080c740 1 --2- 172.21.15.45:0/2684230730 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55daf66cb390 0x55daf66e8150 secure :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f64940a52e0 tx=0x7f6494004990 comp rx=0 tx=0).stop 2024-04-16T17:11:54.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.376+0000 7f64b080c740 1 --2- 172.21.15.45:0/2684230730 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55daf66cbcb0 0x55daf66e8690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:54.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.376+0000 7f64b080c740 1 --2- 172.21.15.45:0/2684230730 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55daf66c5ac0 0x55daf66e7c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:54.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.377+0000 7f64b080c740 1 -- 172.21.15.45:0/2684230730 >> 172.21.15.45:0/2684230730 conn(0x55daf66cd330 msgr2=0x55daf66c4290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:11:54.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.377+0000 7f6490c00640 0 client.26944 ms_handle_reset on v2:172.21.15.104:6838/3660070592 2024-04-16T17:11:54.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.377+0000 7f64b080c740 1 -- 172.21.15.45:0/2684230730 shutdown_connections 2024-04-16T17:11:54.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.377+0000 7f64b080c740 1 -- 172.21.15.45:0/2684230730 wait complete. 2024-04-16T17:11:54.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.377+0000 7f64b080c740 20 client.26944 trim_cache size 0 max 16384 2024-04-16T17:11:54.381 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-16T17:11:54.382 INFO:teuthology.orchestra.run.smithi045.stdout: "status": "no active scrubs running", 2024-04-16T17:11:54.382 INFO:teuthology.orchestra.run.smithi045.stdout: "scrubs": {} 2024-04-16T17:11:54.382 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-16T17:11:54.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.377+0000 7f64a7e00640 1 -- 172.21.15.45:0/2266732040 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f648c0a3c30 msgr2=0x7f648c0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:54.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.377+0000 7f64a7e00640 1 --2- 172.21.15.45:0/2266732040 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f648c0a3c30 0x7f648c0a60f0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f64a0005fd0 tx=0x7f64a0009500 comp rx=0 tx=0).stop 2024-04-16T17:11:54.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.377+0000 7f64a7e00640 1 -- 172.21.15.45:0/2266732040 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f64a81071f0 msgr2=0x7f64a81a0b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:54.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.377+0000 7f64a7e00640 1 --2- 172.21.15.45:0/2266732040 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f64a81071f0 0x7f64a81a0b90 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f649400e9c0 tx=0x7f649400edc0 comp rx=0 tx=0).stop 2024-04-16T17:11:54.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.378+0000 7f64a7e00640 1 -- 172.21.15.45:0/2266732040 shutdown_connections 2024-04-16T17:11:54.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.378+0000 7f64a7e00640 1 --2- 172.21.15.45:0/2266732040 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f648c0a3c30 0x7f648c0a60f0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:54.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.378+0000 7f64a7e00640 1 --2- 172.21.15.45:0/2266732040 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f64a819a7c0 0x7f64a819ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:54.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.378+0000 7f64a7e00640 1 --2- 172.21.15.45:0/2266732040 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f64a81071f0 0x7f64a81a0b90 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:54.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.378+0000 7f64a7e00640 1 --2- 172.21.15.45:0/2266732040 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f64a8102120 0x7f64a81a0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:54.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.378+0000 7f64a7e00640 1 -- 172.21.15.45:0/2266732040 >> 172.21.15.45:0/2266732040 conn(0x7f64a80fd9a0 msgr2=0x7f64a80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:11:54.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.378+0000 7f64a7e00640 1 -- 172.21.15.45:0/2266732040 shutdown_connections 2024-04-16T17:11:54.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.378+0000 7f64a7e00640 1 -- 172.21.15.45:0/2266732040 wait complete. 2024-04-16T17:11:54.394 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-16T17:11:54.394 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-16T17:11:54.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.592+0000 7faf18a00640 1 -- 172.21.15.45:0/3221576096 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faf14074cd0 msgr2=0x7faf1410e1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:54.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.592+0000 7faf18a00640 1 --2- 172.21.15.45:0/3221576096 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faf14074cd0 0x7faf1410e1a0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7faf00009900 tx=0x7faf0002bcd0 comp rx=0 tx=0).stop 2024-04-16T17:11:54.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.592+0000 7faf18a00640 1 -- 172.21.15.45:0/3221576096 shutdown_connections 2024-04-16T17:11:54.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.592+0000 7faf18a00640 1 --2- 172.21.15.45:0/3221576096 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faf1410e6e0 0x7faf14110ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:54.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.592+0000 7faf18a00640 1 --2- 172.21.15.45:0/3221576096 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faf14074cd0 0x7faf1410e1a0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:54.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.592+0000 7faf18a00640 1 --2- 172.21.15.45:0/3221576096 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7faf140743b0 0x7faf14074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:54.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.592+0000 7faf18a00640 1 -- 172.21.15.45:0/3221576096 >> 172.21.15.45:0/3221576096 conn(0x7faf140fd930 msgr2=0x7faf140ffd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:11:54.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.592+0000 7faf18a00640 1 -- 172.21.15.45:0/3221576096 shutdown_connections 2024-04-16T17:11:54.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.592+0000 7faf18a00640 1 -- 172.21.15.45:0/3221576096 wait complete. 2024-04-16T17:11:54.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.593+0000 7faf18a00640 1 Processor -- start 2024-04-16T17:11:54.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.594+0000 7faf18a00640 1 -- start start 2024-04-16T17:11:54.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.594+0000 7faf18a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faf140743b0 0x7faf141a06b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:54.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.594+0000 7faf18a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faf14074cd0 0x7faf141a0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:54.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.594+0000 7faf18a00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7faf1410e6e0 0x7faf1419a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:54.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.594+0000 7faf18a00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7faf14114420 con 0x7faf14074cd0 2024-04-16T17:11:54.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.594+0000 7faf18a00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7faf141142a0 con 0x7faf1410e6e0 2024-04-16T17:11:54.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.594+0000 7faf18a00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7faf141145a0 con 0x7faf140743b0 2024-04-16T17:11:54.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.594+0000 7faf11e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faf14074cd0 0x7faf141a0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:54.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.594+0000 7faf12800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faf140743b0 0x7faf141a06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:54.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.594+0000 7faf11e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faf14074cd0 0x7faf141a0bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39334/0 (socket says 172.21.15.45:39334) 2024-04-16T17:11:54.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.594+0000 7faf11e00640 1 -- 172.21.15.45:0/1561395277 learned_addr learned my addr 172.21.15.45:0/1561395277 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:11:54.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.595+0000 7faf13200640 1 --2- 172.21.15.45:0/1561395277 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7faf1410e6e0 0x7faf1419a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:54.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.595+0000 7faf12800640 1 -- 172.21.15.45:0/1561395277 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7faf1410e6e0 msgr2=0x7faf1419a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:54.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.595+0000 7faf12800640 1 --2- 172.21.15.45:0/1561395277 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7faf1410e6e0 0x7faf1419a7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:54.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.595+0000 7faf12800640 1 -- 172.21.15.45:0/1561395277 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faf14074cd0 msgr2=0x7faf141a0bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:54.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.595+0000 7faf12800640 1 --2- 172.21.15.45:0/1561395277 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faf14074cd0 0x7faf141a0bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:54.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.595+0000 7faf12800640 1 -- 172.21.15.45:0/1561395277 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf1419b080 con 0x7faf140743b0 2024-04-16T17:11:54.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.595+0000 7faf11e00640 1 --2- 172.21.15.45:0/1561395277 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faf14074cd0 0x7faf141a0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:11:54.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.595+0000 7faf13200640 1 --2- 172.21.15.45:0/1561395277 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7faf1410e6e0 0x7faf1419a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:11:54.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.595+0000 7faf12800640 1 --2- 172.21.15.45:0/1561395277 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faf140743b0 0x7faf141a06b0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7faf0400e9f0 tx=0x7faf0400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:11:54.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.595+0000 7faf0b400640 1 -- 172.21.15.45:0/1561395277 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf0400ccc0 con 0x7faf140743b0 2024-04-16T17:11:54.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.595+0000 7faf18a00640 1 -- 172.21.15.45:0/1561395277 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf1419b370 con 0x7faf140743b0 2024-04-16T17:11:54.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.595+0000 7faf0b400640 1 -- 172.21.15.45:0/1561395277 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faf04010910 con 0x7faf140743b0 2024-04-16T17:11:54.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.595+0000 7faf0b400640 1 -- 172.21.15.45:0/1561395277 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf04018960 con 0x7faf140743b0 2024-04-16T17:11:54.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.596+0000 7faf18a00640 1 -- 172.21.15.45:0/1561395277 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf141a7630 con 0x7faf140743b0 2024-04-16T17:11:54.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.597+0000 7faf18a00640 1 -- 172.21.15.45:0/1561395277 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf14075a60 con 0x7faf140743b0 2024-04-16T17:11:54.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.600+0000 7faf0b400640 1 -- 172.21.15.45:0/1561395277 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7faf04010ac0 con 0x7faf140743b0 2024-04-16T17:11:54.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.601+0000 7faf0b400640 1 --2- 172.21.15.45:0/1561395277 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7faef80a3d20 0x7faef80a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:54.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.601+0000 7faf0b400640 1 -- 172.21.15.45:0/1561395277 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7faf040d1000 con 0x7faf140743b0 2024-04-16T17:11:54.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.601+0000 7faf11e00640 1 --2- 172.21.15.45:0/1561395277 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7faef80a3d20 0x7faef80a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:54.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.602+0000 7faf11e00640 1 --2- 172.21.15.45:0/1561395277 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7faef80a3d20 0x7faef80a61e0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7faf00005ab0 tx=0x7faf00005a20 comp rx=0 tx=0).ready entity=mgr.25411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:11:54.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.606+0000 7faf0b400640 1 -- 172.21.15.45:0/1561395277 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7faf0409d330 con 0x7faf140743b0 2024-04-16T17:11:54.677 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:11:54 smithi104 ceph-mon[36457]: pgmap v157: 129 pgs: 129 active+clean; 45 MiB data, 640 MiB used, 1.0 TiB / 1.0 TiB avail; 481 KiB/s rd, 898 KiB/s wr, 81 op/s 2024-04-16T17:11:54.804 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:11:54 smithi117 ceph-mon[36730]: pgmap v157: 129 pgs: 129 active+clean; 45 MiB data, 640 MiB used, 1.0 TiB / 1.0 TiB avail; 481 KiB/s rd, 898 KiB/s wr, 81 op/s 2024-04-16T17:11:54.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.903+0000 7faf18a00640 1 -- 172.21.15.45:0/1561395277 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7faf1419bca0 con 0x7faf140743b0 2024-04-16T17:11:54.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.904+0000 7faf0b400640 1 -- 172.21.15.45:0/1561395277 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+11890 (secure 0 0 0) 0x7faf0409cc30 con 0x7faf140743b0 2024-04-16T17:11:54.904 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-16T17:11:54.905 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":35,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/1748842443","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":1748842443},{"type":"v1","addr":"172.21.15.45:6835","nonce":1748842443}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6839/1308627051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":1308627051},{"type":"v1","addr":"172.21.15.45:6839","nonce":1308627051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/1768242489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1768242489},{"type":"v1","addr":"172.21.15.117:6833","nonce":1768242489}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6833/3558902585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":3558902585},{"type":"v1","addr":"172.21.15.104:6833","nonce":3558902585}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24478,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/1308142903","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":1308142903},{"type":"v1","addr":"172.21.15.45:6837","nonce":1308142903}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24491,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6835/3430679937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":3430679937},{"type":"v1","addr":"172.21.15.104:6835","nonce":3430679937}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":35,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-16T17:03:24.663611+0000","modified":"2024-04-16T17:11:52.259935+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24581,"mds_1":24569,"mds_2":24532},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24499":{"gid":24499,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/2224123841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2224123841},{"type":"v1","addr":"172.21.15.117:6835","nonce":2224123841}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6837/2471356815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":2471356815},{"type":"v1","addr":"172.21.15.117:6837","nonce":2471356815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24524":{"gid":24524,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6837/2647756509","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":2647756509},{"type":"v1","addr":"172.21.15.104:6837","nonce":2647756509}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24532":{"gid":24532,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":13,"addr":"172.21.15.117:6839/2378675949","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":2378675949},{"type":"v1","addr":"172.21.15.117:6839","nonce":2378675949}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24569":{"gid":24569,"name":"a","rank":1,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.45:6841/3755310822","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6840","nonce":3755310822},{"type":"v1","addr":"172.21.15.45:6841","nonce":3755310822}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24581":{"gid":24581,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.104:6839/3660070592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6838","nonce":3660070592},{"type":"v1","addr":"172.21.15.104:6839","nonce":3660070592}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24581,"qdb_cluster":[24569,24581,24532]},"id":1}]} 2024-04-16T17:11:54.905 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 35 2024-04-16T17:11:54.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.907+0000 7faf18a00640 1 -- 172.21.15.45:0/1561395277 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7faef80a3d20 msgr2=0x7faef80a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:54.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.907+0000 7faf18a00640 1 --2- 172.21.15.45:0/1561395277 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7faef80a3d20 0x7faef80a61e0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7faf00005ab0 tx=0x7faf00005a20 comp rx=0 tx=0).stop 2024-04-16T17:11:54.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.907+0000 7faf18a00640 1 -- 172.21.15.45:0/1561395277 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faf140743b0 msgr2=0x7faf141a06b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:54.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.907+0000 7faf18a00640 1 --2- 172.21.15.45:0/1561395277 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faf140743b0 0x7faf141a06b0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7faf0400e9f0 tx=0x7faf0400eeb0 comp rx=0 tx=0).stop 2024-04-16T17:11:54.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.907+0000 7faf18a00640 1 -- 172.21.15.45:0/1561395277 shutdown_connections 2024-04-16T17:11:54.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.907+0000 7faf18a00640 1 --2- 172.21.15.45:0/1561395277 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7faef80a3d20 0x7faef80a61e0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:54.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.907+0000 7faf18a00640 1 --2- 172.21.15.45:0/1561395277 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7faf1410e6e0 0x7faf1419a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:54.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.907+0000 7faf18a00640 1 --2- 172.21.15.45:0/1561395277 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faf14074cd0 0x7faf141a0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:54.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.907+0000 7faf18a00640 1 --2- 172.21.15.45:0/1561395277 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faf140743b0 0x7faf141a06b0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:54.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.907+0000 7faf18a00640 1 -- 172.21.15.45:0/1561395277 >> 172.21.15.45:0/1561395277 conn(0x7faf140fd930 msgr2=0x7faf140fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:11:54.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.907+0000 7faf18a00640 1 -- 172.21.15.45:0/1561395277 shutdown_connections 2024-04-16T17:11:54.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:54.907+0000 7faf18a00640 1 -- 172.21.15.45:0/1561395277 wait complete. 2024-04-16T17:11:54.926 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-16T17:11:55.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.121+0000 7efe51a00640 1 -- 172.21.15.45:0/2710999214 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efe4c10c480 msgr2=0x7efe4c10e920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:55.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.121+0000 7efe51a00640 1 --2- 172.21.15.45:0/2710999214 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efe4c10c480 0x7efe4c10e920 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7efe44009ca0 tx=0x7efe4402c190 comp rx=0 tx=0).stop 2024-04-16T17:11:55.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.122+0000 7efe51a00640 1 -- 172.21.15.45:0/2710999214 shutdown_connections 2024-04-16T17:11:55.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.122+0000 7efe51a00640 1 --2- 172.21.15.45:0/2710999214 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efe4c10c480 0x7efe4c10e920 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:55.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.122+0000 7efe51a00640 1 --2- 172.21.15.45:0/2710999214 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efe4c103980 0x7efe4c10be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:55.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.122+0000 7efe51a00640 1 --2- 172.21.15.45:0/2710999214 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efe4c103060 0x7efe4c103440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:55.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.122+0000 7efe51a00640 1 -- 172.21.15.45:0/2710999214 >> 172.21.15.45:0/2710999214 conn(0x7efe4c0fc880 msgr2=0x7efe4c0feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:11:55.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.122+0000 7efe51a00640 1 -- 172.21.15.45:0/2710999214 shutdown_connections 2024-04-16T17:11:55.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.122+0000 7efe51a00640 1 -- 172.21.15.45:0/2710999214 wait complete. 2024-04-16T17:11:55.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.123+0000 7efe51a00640 1 Processor -- start 2024-04-16T17:11:55.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.123+0000 7efe51a00640 1 -- start start 2024-04-16T17:11:55.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.123+0000 7efe51a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efe4c103060 0x7efe4c100cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:55.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.123+0000 7efe51a00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efe4c103980 0x7efe4c1011f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:55.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.123+0000 7efe51a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efe4c10c480 0x7efe4c101730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:55.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.123+0000 7efe51a00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7efe4c112260 con 0x7efe4c10c480 2024-04-16T17:11:55.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.123+0000 7efe51a00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7efe4c1120e0 con 0x7efe4c103980 2024-04-16T17:11:55.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.123+0000 7efe51a00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7efe4c1123e0 con 0x7efe4c103060 2024-04-16T17:11:55.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.124+0000 7efe4be00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efe4c103060 0x7efe4c100cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:55.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.124+0000 7efe4be00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efe4c103060 0x7efe4c100cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:44286/0 (socket says 172.21.15.45:44286) 2024-04-16T17:11:55.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.124+0000 7efe4be00640 1 -- 172.21.15.45:0/2691426856 learned_addr learned my addr 172.21.15.45:0/2691426856 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:11:55.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.124+0000 7efe43e00640 1 --2- 172.21.15.45:0/2691426856 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efe4c103980 0x7efe4c1011f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:55.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.124+0000 7efe51000640 1 --2- 172.21.15.45:0/2691426856 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efe4c10c480 0x7efe4c101730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:55.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.124+0000 7efe4be00640 1 -- 172.21.15.45:0/2691426856 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efe4c103980 msgr2=0x7efe4c1011f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:55.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.124+0000 7efe4be00640 1 --2- 172.21.15.45:0/2691426856 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efe4c103980 0x7efe4c1011f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:55.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.124+0000 7efe4be00640 1 -- 172.21.15.45:0/2691426856 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efe4c10c480 msgr2=0x7efe4c101730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:55.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.124+0000 7efe4be00640 1 --2- 172.21.15.45:0/2691426856 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efe4c10c480 0x7efe4c101730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:55.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.124+0000 7efe4be00640 1 -- 172.21.15.45:0/2691426856 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe4c1a6c00 con 0x7efe4c103060 2024-04-16T17:11:55.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.125+0000 7efe4be00640 1 --2- 172.21.15.45:0/2691426856 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efe4c103060 0x7efe4c100cb0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7efe3c00e9f0 tx=0x7efe3c00edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:11:55.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.125+0000 7efe49600640 1 -- 172.21.15.45:0/2691426856 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe3c00cd80 con 0x7efe4c103060 2024-04-16T17:11:55.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.125+0000 7efe49600640 1 -- 172.21.15.45:0/2691426856 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efe3c005500 con 0x7efe4c103060 2024-04-16T17:11:55.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.125+0000 7efe51a00640 1 -- 172.21.15.45:0/2691426856 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe4c1a6ef0 con 0x7efe4c103060 2024-04-16T17:11:55.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.125+0000 7efe49600640 1 -- 172.21.15.45:0/2691426856 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe3c010690 con 0x7efe4c103060 2024-04-16T17:11:55.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.125+0000 7efe51a00640 1 -- 172.21.15.45:0/2691426856 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe4c1a7360 con 0x7efe4c103060 2024-04-16T17:11:55.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.130+0000 7efe49600640 1 -- 172.21.15.45:0/2691426856 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7efe3c0046c0 con 0x7efe4c103060 2024-04-16T17:11:55.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.130+0000 7efe55e2a740 1 Processor -- start 2024-04-16T17:11:55.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.131+0000 7efe55e2a740 1 -- start start 2024-04-16T17:11:55.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.131+0000 7efe55e2a740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b65774cf10 0x55b65774d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:55.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.131+0000 7efe55e2a740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b65774d830 0x55b657755ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:55.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.131+0000 7efe55e2a740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b657756410 0x55b657758800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:55.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.131+0000 7efe55e2a740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55b6576b2020 con 0x55b65774cf10 2024-04-16T17:11:55.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.131+0000 7efe55e2a740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55b657658460 con 0x55b657756410 2024-04-16T17:11:55.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.131+0000 7efe55e2a740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55b6576adfe0 con 0x55b65774d830 2024-04-16T17:11:55.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.132+0000 7efe51000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b65774cf10 0x55b65774d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:55.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.132+0000 7efe4be00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b65774d830 0x55b657755ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:55.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.132+0000 7efe4be00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b65774d830 0x55b657755ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:44302/0 (socket says 172.21.15.45:44302) 2024-04-16T17:11:55.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.132+0000 7efe4be00640 1 -- 172.21.15.45:0/2961990646 learned_addr learned my addr 172.21.15.45:0/2961990646 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:11:55.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.132+0000 7efe4be00640 1 -- 172.21.15.45:0/2961990646 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b657756410 msgr2=0x55b657758800 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-16T17:11:55.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.132+0000 7efe4be00640 1 --2- 172.21.15.45:0/2961990646 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b657756410 0x55b657758800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:55.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.132+0000 7efe4be00640 1 -- 172.21.15.45:0/2961990646 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b65774cf10 msgr2=0x55b65774d2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:11:55.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.132+0000 7efe4be00640 1 --2- 172.21.15.45:0/2961990646 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b65774cf10 0x55b65774d2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:55.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.132+0000 7efe4be00640 1 -- 172.21.15.45:0/2961990646 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b657660040 con 0x55b65774d830 2024-04-16T17:11:55.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.132+0000 7efe51000640 1 --2- 172.21.15.45:0/2961990646 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b65774cf10 0x55b65774d2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:11:55.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.133+0000 7efe49600640 1 --2- 172.21.15.45:0/2691426856 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7efe300a3d50 0x7efe300a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:55.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.133+0000 7efe4be00640 1 --2- 172.21.15.45:0/2961990646 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b65774d830 0x55b657755ed0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7efe3c0ce1b0 tx=0x7efe3c0d4ee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=57e19412d5fbe73c server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:11:55.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.133+0000 7efe41600640 1 -- 172.21.15.45:0/2961990646 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe3c098d60 con 0x55b65774d830 2024-04-16T17:11:55.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.133+0000 7efe41600640 1 -- 172.21.15.45:0/2961990646 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efe3c09d1c0 con 0x55b65774d830 2024-04-16T17:11:55.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.133+0000 7efe41600640 1 -- 172.21.15.45:0/2961990646 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe3c09c420 con 0x55b65774d830 2024-04-16T17:11:55.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.133+0000 7efe49600640 1 -- 172.21.15.45:0/2691426856 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7efe3c0d0e90 con 0x7efe4c103060 2024-04-16T17:11:55.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.134+0000 7efe51000640 1 --2- 172.21.15.45:0/2691426856 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7efe300a3d50 0x7efe300a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:55.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.135+0000 7efe55e2a740 1 -- 172.21.15.45:0/2961990646 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b65774d830 msgr2=0x55b657755ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:55.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.135+0000 7efe55e2a740 1 --2- 172.21.15.45:0/2961990646 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b65774d830 0x55b657755ed0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7efe3c0ce1b0 tx=0x7efe3c0d4ee0 comp rx=0 tx=0).stop 2024-04-16T17:11:55.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.135+0000 7efe51000640 1 --2- 172.21.15.45:0/2691426856 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7efe300a3d50 0x7efe300a6210 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7efe44004af0 tx=0x7efe4402e040 comp rx=0 tx=0).ready entity=mgr.25411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:11:55.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.135+0000 7efe55e2a740 1 -- 172.21.15.45:0/2961990646 shutdown_connections 2024-04-16T17:11:55.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.135+0000 7efe55e2a740 1 --2- 172.21.15.45:0/2961990646 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b65774d830 0x55b657755ed0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:55.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.135+0000 7efe55e2a740 1 --2- 172.21.15.45:0/2961990646 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b657756410 0x55b657758800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:55.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.135+0000 7efe55e2a740 1 --2- 172.21.15.45:0/2961990646 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b65774cf10 0x55b65774d2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:55.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.135+0000 7efe55e2a740 1 -- 172.21.15.45:0/2961990646 >> 172.21.15.45:0/2961990646 conn(0x55b657747640 msgr2=0x55b657747a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:11:55.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.136+0000 7efe55e2a740 1 -- 172.21.15.45:0/2961990646 shutdown_connections 2024-04-16T17:11:55.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.136+0000 7efe55e2a740 1 -- 172.21.15.45:0/2961990646 wait complete. 2024-04-16T17:11:55.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.137+0000 7efe55e2a740 1 -- start start 2024-04-16T17:11:55.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.137+0000 7efe55e2a740 1 Processor -- start 2024-04-16T17:11:55.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.138+0000 7efe55e2a740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b65774cf10 0x55b6577534b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:55.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.138+0000 7efe55e2a740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b65774d830 0x55b6577539f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:55.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.138+0000 7efe55e2a740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b657756410 0x55b657753f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:55.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.138+0000 7efe55e2a740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55b657759200 con 0x55b65774d830 2024-04-16T17:11:55.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.138+0000 7efe55e2a740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55b65766b830 con 0x55b65774cf10 2024-04-16T17:11:55.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.138+0000 7efe55e2a740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55b657759380 con 0x55b657756410 2024-04-16T17:11:55.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.138+0000 7efe55e2a740 10 client.0 _ll_register_callbacks cb 0x55b657673150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:11:55.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.138+0000 7efe51000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b657756410 0x55b657753f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:55.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.139+0000 7efe51000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b657756410 0x55b657753f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:44318/0 (socket says 172.21.15.45:44318) 2024-04-16T17:11:55.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.139+0000 7efe51000640 1 -- 172.21.15.45:0/4076555256 learned_addr learned my addr 172.21.15.45:0/4076555256 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:11:55.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.139+0000 7efe51000640 1 -- 172.21.15.45:0/4076555256 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b65774cf10 msgr2=0x55b6577534b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-16T17:11:55.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.139+0000 7efe51000640 1 --2- 172.21.15.45:0/4076555256 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b65774cf10 0x55b6577534b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:55.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.139+0000 7efe51000640 1 -- 172.21.15.45:0/4076555256 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b65774d830 msgr2=0x55b6577539f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-16T17:11:55.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.139+0000 7efe51000640 1 --2- 172.21.15.45:0/4076555256 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b65774d830 0x55b6577539f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:55.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.139+0000 7efe51000640 1 -- 172.21.15.45:0/4076555256 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b65766f920 con 0x55b657756410 2024-04-16T17:11:55.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.139+0000 7efe51000640 1 --2- 172.21.15.45:0/4076555256 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b657756410 0x55b657753f30 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7efe44009ca0 tx=0x7efe4402c190 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1e213194b1f7f4a1 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:11:55.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.139+0000 7efe3a000640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-04-16T17:11:55.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.140+0000 7efe3a000640 1 -- 172.21.15.45:0/4076555256 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe44046070 con 0x55b657756410 2024-04-16T17:11:55.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.140+0000 7efe55e2a740 1 -- 172.21.15.45:0/4076555256 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efe3c09d1c0 con 0x55b657756410 2024-04-16T17:11:55.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.140+0000 7efe3a000640 1 -- 172.21.15.45:0/4076555256 <== mon.1 v2:172.21.15.117:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7efe44004610 con 0x55b657756410 2024-04-16T17:11:55.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.140+0000 7efe3a000640 1 -- 172.21.15.45:0/4076555256 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7efe4405b050 con 0x55b657756410 2024-04-16T17:11:55.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.140+0000 7efe55e2a740 10 client.26962 fetch_fsmap learned FSMap version 35 2024-04-16T17:11:55.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.140+0000 7efe55e2a740 1 -- 172.21.15.45:0/4076555256 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55b65763d7f0 con 0x55b657756410 2024-04-16T17:11:55.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.140+0000 7efe3a000640 1 -- 172.21.15.45:0/4076555256 <== mon.1 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efe3c09d1c0 con 0x55b657756410 2024-04-16T17:11:55.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.140+0000 7efe3a000640 1 -- 172.21.15.45:0/4076555256 <== mon.1 v2:172.21.15.117:3300/0 5 ==== fsmap(e 35) ==== 8358+0+0 (secure 0 0 0) 0x7efe4404d6b0 con 0x55b657756410 2024-04-16T17:11:55.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.140+0000 7efe55e2a740 10 client.26962 fetch_fsmap finished waiting for FSMap version 35 2024-04-16T17:11:55.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.140+0000 7efe55e2a740 10 client.26962 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-16T17:11:55.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.140+0000 7efe55e2a740 20 client.26962 populate_metadata read hostname 'smithi045' 2024-04-16T17:11:55.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.140+0000 7efe55e2a740 1 --2- 172.21.15.45:0/4076555256 >> [v2:172.21.15.117:6838/2378675949,v1:172.21.15.117:6839/2378675949] conn(0x55b65776f0b0 0x55b657771570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:55.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.140+0000 7efe55e2a740 4 client.26962 mds_command: new command op to 24532 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:11:55.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.140+0000 7efe55e2a740 1 -- 172.21.15.45:0/4076555256 --> [v2:172.21.15.117:6838/2378675949,v1:172.21.15.117:6839/2378675949] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b6576bfa40 con 0x55b65776f0b0 2024-04-16T17:11:55.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.155+0000 7efe4be00640 1 --2- 172.21.15.45:0/4076555256 >> [v2:172.21.15.117:6838/2378675949,v1:172.21.15.117:6839/2378675949] conn(0x55b65776f0b0 0x55b657771570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:55.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.156+0000 7efe4be00640 1 --2- 172.21.15.45:0/4076555256 >> [v2:172.21.15.117:6838/2378675949,v1:172.21.15.117:6839/2378675949] conn(0x55b65776f0b0 0x55b657771570 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=bc3f2ea83f26847e server_cookie=6a35ad1b2b1a8ffe in_seq=0 out_seq=0 2024-04-16T17:11:55.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.156+0000 7efe3a000640 10 client.26962 ms_handle_connect on v2:172.21.15.117:6838/2378675949 2024-04-16T17:11:55.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.157+0000 7efe3a000640 1 -- 172.21.15.45:0/4076555256 <== mds.2 v2:172.21.15.117:6838/2378675949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b6576bfa40 con 0x55b65776f0b0 2024-04-16T17:11:55.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.157+0000 7efe3a000640 10 client.26962 handle_command_reply: tid=0 2024-04-16T17:11:55.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.157+0000 7efe55e2a740 1 client.26962 shutdown 2024-04-16T17:11:55.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.157+0000 7efe55e2a740 1 -- 172.21.15.45:0/4076555256 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b657756410 msgr2=0x55b657753f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:55.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.157+0000 7efe55e2a740 1 --2- 172.21.15.45:0/4076555256 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b657756410 0x55b657753f30 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7efe44009ca0 tx=0x7efe4402c190 comp rx=0 tx=0).stop 2024-04-16T17:11:55.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.158+0000 7efe55e2a740 1 -- 172.21.15.45:0/4076555256 shutdown_connections 2024-04-16T17:11:55.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.158+0000 7efe55e2a740 1 --2- 172.21.15.45:0/4076555256 >> [v2:172.21.15.117:6838/2378675949,v1:172.21.15.117:6839/2378675949] conn(0x55b65776f0b0 0x55b657771570 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:55.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.158+0000 7efe55e2a740 1 --2- 172.21.15.45:0/4076555256 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b657756410 0x55b657753f30 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:55.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.158+0000 7efe55e2a740 1 --2- 172.21.15.45:0/4076555256 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b65774d830 0x55b6577539f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:55.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.158+0000 7efe55e2a740 1 --2- 172.21.15.45:0/4076555256 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b65774cf10 0x55b6577534b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:55.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.158+0000 7efe55e2a740 1 -- 172.21.15.45:0/4076555256 >> 172.21.15.45:0/4076555256 conn(0x55b657747640 msgr2=0x55b657749390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:11:55.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.158+0000 7efe3a000640 0 client.26962 ms_handle_reset on v2:172.21.15.117:6838/2378675949 2024-04-16T17:11:55.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.158+0000 7efe55e2a740 1 -- 172.21.15.45:0/4076555256 shutdown_connections 2024-04-16T17:11:55.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.158+0000 7efe55e2a740 1 -- 172.21.15.45:0/4076555256 wait complete. 2024-04-16T17:11:55.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.158+0000 7efe55e2a740 20 client.26962 trim_cache size 0 max 16384 2024-04-16T17:11:55.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.173+0000 7efe55e2a740 1 Processor -- start 2024-04-16T17:11:55.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.173+0000 7efe55e2a740 1 -- start start 2024-04-16T17:11:55.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.173+0000 7efe55e2a740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b65774cf10 0x55b657755be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:55.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.173+0000 7efe55e2a740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b65774d830 0x55b657762110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:55.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.173+0000 7efe55e2a740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b657756410 0x55b657752610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:55.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.173+0000 7efe55e2a740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7efe4c1af770 con 0x55b657756410 2024-04-16T17:11:55.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.173+0000 7efe55e2a740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7efe4c1acfe0 con 0x55b65774d830 2024-04-16T17:11:55.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.173+0000 7efe55e2a740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7efe4c1aaf00 con 0x55b65774cf10 2024-04-16T17:11:55.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.174+0000 7efe43e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b65774d830 0x55b657762110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:55.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.174+0000 7efe43e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b65774d830 0x55b657762110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:60848/0 (socket says 172.21.15.45:60848) 2024-04-16T17:11:55.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.174+0000 7efe43e00640 1 -- 172.21.15.45:0/4092394549 learned_addr learned my addr 172.21.15.45:0/4092394549 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:11:55.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.174+0000 7efe4be00640 1 --2- 172.21.15.45:0/4092394549 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b65774cf10 0x55b657755be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:55.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.174+0000 7efe51000640 1 --2- 172.21.15.45:0/4092394549 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b657756410 0x55b657752610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:55.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.174+0000 7efe43e00640 1 -- 172.21.15.45:0/4092394549 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b65774cf10 msgr2=0x55b657755be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:11:55.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.174+0000 7efe43e00640 1 --2- 172.21.15.45:0/4092394549 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b65774cf10 0x55b657755be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:55.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.174+0000 7efe43e00640 1 -- 172.21.15.45:0/4092394549 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b657756410 msgr2=0x55b657752610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:11:55.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.174+0000 7efe43e00640 1 --2- 172.21.15.45:0/4092394549 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b657756410 0x55b657752610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:55.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.174+0000 7efe43e00640 1 -- 172.21.15.45:0/4092394549 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b65774d660 con 0x55b65774d830 2024-04-16T17:11:55.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.174+0000 7efe51000640 1 --2- 172.21.15.45:0/4092394549 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b657756410 0x55b657752610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:11:55.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.186+0000 7efe43e00640 1 --2- 172.21.15.45:0/4092394549 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b65774d830 0x55b657762110 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7efe34009b90 tx=0x7efe34011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=166eca192d74e46 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:11:55.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.187+0000 7efe40c00640 1 -- 172.21.15.45:0/4092394549 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe34004050 con 0x55b65774d830 2024-04-16T17:11:55.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.187+0000 7efe40c00640 1 -- 172.21.15.45:0/4092394549 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efe340041f0 con 0x55b65774d830 2024-04-16T17:11:55.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.187+0000 7efe40c00640 1 -- 172.21.15.45:0/4092394549 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe340196f0 con 0x55b65774d830 2024-04-16T17:11:55.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.187+0000 7efe55e2a740 1 -- 172.21.15.45:0/4092394549 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b65774d830 msgr2=0x55b657762110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:55.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.187+0000 7efe55e2a740 1 --2- 172.21.15.45:0/4092394549 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b65774d830 0x55b657762110 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7efe34009b90 tx=0x7efe34011810 comp rx=0 tx=0).stop 2024-04-16T17:11:55.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.187+0000 7efe55e2a740 1 -- 172.21.15.45:0/4092394549 shutdown_connections 2024-04-16T17:11:55.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.187+0000 7efe55e2a740 1 --2- 172.21.15.45:0/4092394549 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b657756410 0x55b657752610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:55.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.187+0000 7efe55e2a740 1 --2- 172.21.15.45:0/4092394549 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b65774d830 0x55b657762110 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:55.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.187+0000 7efe55e2a740 1 --2- 172.21.15.45:0/4092394549 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b65774cf10 0x55b657755be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:55.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.187+0000 7efe55e2a740 1 -- 172.21.15.45:0/4092394549 >> 172.21.15.45:0/4092394549 conn(0x55b657747640 msgr2=0x55b657745e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:11:55.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.188+0000 7efe55e2a740 1 -- 172.21.15.45:0/4092394549 shutdown_connections 2024-04-16T17:11:55.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.188+0000 7efe55e2a740 1 -- 172.21.15.45:0/4092394549 wait complete. 2024-04-16T17:11:55.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.188+0000 7efe55e2a740 1 -- start start 2024-04-16T17:11:55.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.188+0000 7efe55e2a740 1 Processor -- start 2024-04-16T17:11:55.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.189+0000 7efe55e2a740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b65774cf10 0x55b65775a070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:55.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.189+0000 7efe55e2a740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b65774d830 0x55b65775a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:55.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.189+0000 7efe55e2a740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b657756410 0x55b65775aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:55.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.189+0000 7efe55e2a740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7efe4c1ab4c0 con 0x55b657756410 2024-04-16T17:11:55.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.189+0000 7efe55e2a740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7efe4c1ac910 con 0x55b65774cf10 2024-04-16T17:11:55.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.189+0000 7efe55e2a740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7efe4c1abb70 con 0x55b65774d830 2024-04-16T17:11:55.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.189+0000 7efe55e2a740 10 client.0 _ll_register_callbacks cb 0x7efe34005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:11:55.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.189+0000 7efe4be00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b65774cf10 0x55b65775a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:55.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.189+0000 7efe51000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b657756410 0x55b65775aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:55.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.189+0000 7efe43e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b65774d830 0x55b65775a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:55.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.189+0000 7efe4be00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b65774cf10 0x55b65775a070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:60858/0 (socket says 172.21.15.45:60858) 2024-04-16T17:11:55.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.189+0000 7efe4be00640 1 -- 172.21.15.45:0/3177259459 learned_addr learned my addr 172.21.15.45:0/3177259459 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:11:55.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.189+0000 7efe4be00640 1 -- 172.21.15.45:0/3177259459 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b65774d830 msgr2=0x55b65775a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:11:55.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.189+0000 7efe4be00640 1 --2- 172.21.15.45:0/3177259459 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b65774d830 0x55b65775a5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:55.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.189+0000 7efe4be00640 1 -- 172.21.15.45:0/3177259459 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b657756410 msgr2=0x55b65775aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:11:55.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.189+0000 7efe4be00640 1 --2- 172.21.15.45:0/3177259459 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b657756410 0x55b65775aaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:55.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.189+0000 7efe4be00640 1 -- 172.21.15.45:0/3177259459 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b6573c43f0 con 0x55b65774cf10 2024-04-16T17:11:55.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.190+0000 7efe4be00640 1 --2- 172.21.15.45:0/3177259459 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b65774cf10 0x55b65775a070 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7efe3c0d9570 tx=0x7efe3c009330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=deb87b6fc8817352 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:11:55.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.190+0000 7efe3a000640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-04-16T17:11:55.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.190+0000 7efe3a000640 1 -- 172.21.15.45:0/3177259459 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe3c09c600 con 0x55b65774cf10 2024-04-16T17:11:55.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.190+0000 7efe3a000640 1 -- 172.21.15.45:0/3177259459 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7efe3c0d49f0 con 0x55b65774cf10 2024-04-16T17:11:55.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.190+0000 7efe55e2a740 1 -- 172.21.15.45:0/3177259459 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efe340041f0 con 0x55b65774cf10 2024-04-16T17:11:55.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.191+0000 7efe3a000640 1 -- 172.21.15.45:0/3177259459 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7efe3c09c890 con 0x55b65774cf10 2024-04-16T17:11:55.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.191+0000 7efe55e2a740 10 client.27020 fetch_fsmap learned FSMap version 35 2024-04-16T17:11:55.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.191+0000 7efe55e2a740 1 -- 172.21.15.45:0/3177259459 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55b65748e7d0 con 0x55b65774cf10 2024-04-16T17:11:55.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.191+0000 7efe3a000640 1 -- 172.21.15.45:0/3177259459 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efe340041f0 con 0x55b65774cf10 2024-04-16T17:11:55.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.191+0000 7efe3a000640 1 -- 172.21.15.45:0/3177259459 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 35) ==== 8358+0+0 (secure 0 0 0) 0x7efe3c0b9070 con 0x55b65774cf10 2024-04-16T17:11:55.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.192+0000 7efe55e2a740 10 client.27020 fetch_fsmap finished waiting for FSMap version 35 2024-04-16T17:11:55.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.192+0000 7efe55e2a740 10 client.27020 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-16T17:11:55.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.192+0000 7efe55e2a740 20 client.27020 populate_metadata read hostname 'smithi045' 2024-04-16T17:11:55.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.192+0000 7efe55e2a740 1 --2- 172.21.15.45:0/3177259459 >> [v2:172.21.15.117:6838/2378675949,v1:172.21.15.117:6839/2378675949] conn(0x55b6577d58b0 0x55b6577d7ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:55.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.192+0000 7efe55e2a740 4 client.27020 mds_command: new command op to 24532 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T17:11:55.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.192+0000 7efe55e2a740 1 -- 172.21.15.45:0/3177259459 --> [v2:172.21.15.117:6838/2378675949,v1:172.21.15.117:6839/2378675949] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b6576b98b0 con 0x55b6577d58b0 2024-04-16T17:11:55.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.192+0000 7efe43e00640 1 --2- 172.21.15.45:0/3177259459 >> [v2:172.21.15.117:6838/2378675949,v1:172.21.15.117:6839/2378675949] conn(0x55b6577d58b0 0x55b6577d7ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:55.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.193+0000 7efe43e00640 1 --2- 172.21.15.45:0/3177259459 >> [v2:172.21.15.117:6838/2378675949,v1:172.21.15.117:6839/2378675949] conn(0x55b6577d58b0 0x55b6577d7ca0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=82056987da932894 server_cookie=3826ba008c60c28a in_seq=0 out_seq=0 2024-04-16T17:11:55.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.193+0000 7efe3a000640 10 client.27020 ms_handle_connect on v2:172.21.15.117:6838/2378675949 2024-04-16T17:11:55.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.193+0000 7efe3a000640 1 -- 172.21.15.45:0/3177259459 <== mds.2 v2:172.21.15.117:6838/2378675949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b6576b98b0 con 0x55b6577d58b0 2024-04-16T17:11:55.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.193+0000 7efe3a000640 10 client.27020 handle_command_reply: tid=0 2024-04-16T17:11:55.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.193+0000 7efe55e2a740 1 client.27020 shutdown 2024-04-16T17:11:55.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.194+0000 7efe55e2a740 1 -- 172.21.15.45:0/3177259459 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b65774cf10 msgr2=0x55b65775a070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:55.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.194+0000 7efe55e2a740 1 --2- 172.21.15.45:0/3177259459 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b65774cf10 0x55b65775a070 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7efe3c0d9570 tx=0x7efe3c009330 comp rx=0 tx=0).stop 2024-04-16T17:11:55.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.194+0000 7efe55e2a740 1 -- 172.21.15.45:0/3177259459 shutdown_connections 2024-04-16T17:11:55.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.194+0000 7efe55e2a740 1 --2- 172.21.15.45:0/3177259459 >> [v2:172.21.15.117:6838/2378675949,v1:172.21.15.117:6839/2378675949] conn(0x55b6577d58b0 0x55b6577d7ca0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:55.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.194+0000 7efe55e2a740 1 --2- 172.21.15.45:0/3177259459 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b65774d830 0x55b65775a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:55.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.194+0000 7efe55e2a740 1 --2- 172.21.15.45:0/3177259459 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b657756410 0x55b65775aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:55.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.194+0000 7efe55e2a740 1 --2- 172.21.15.45:0/3177259459 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b65774cf10 0x55b65775a070 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:55.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.194+0000 7efe3a000640 0 client.27020 ms_handle_reset on v2:172.21.15.117:6838/2378675949 2024-04-16T17:11:55.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.194+0000 7efe55e2a740 1 -- 172.21.15.45:0/3177259459 >> 172.21.15.45:0/3177259459 conn(0x55b657747640 msgr2=0x55b6577533d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:11:55.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.194+0000 7efe55e2a740 1 -- 172.21.15.45:0/3177259459 shutdown_connections 2024-04-16T17:11:55.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.194+0000 7efe55e2a740 1 -- 172.21.15.45:0/3177259459 wait complete. 2024-04-16T17:11:55.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.194+0000 7efe55e2a740 20 client.27020 trim_cache size 0 max 16384 2024-04-16T17:11:55.197 INFO:teuthology.orchestra.run.smithi045.stdout:[] 2024-04-16T17:11:55.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.195+0000 7efe51a00640 1 -- 172.21.15.45:0/2691426856 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7efe300a3d50 msgr2=0x7efe300a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:55.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.195+0000 7efe51a00640 1 --2- 172.21.15.45:0/2691426856 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7efe300a3d50 0x7efe300a6210 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7efe44004af0 tx=0x7efe4402e040 comp rx=0 tx=0).stop 2024-04-16T17:11:55.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.195+0000 7efe51a00640 1 -- 172.21.15.45:0/2691426856 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efe4c103060 msgr2=0x7efe4c100cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:55.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.195+0000 7efe51a00640 1 --2- 172.21.15.45:0/2691426856 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efe4c103060 0x7efe4c100cb0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7efe3c00e9f0 tx=0x7efe3c00edf0 comp rx=0 tx=0).stop 2024-04-16T17:11:55.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.195+0000 7efe51a00640 1 -- 172.21.15.45:0/2691426856 shutdown_connections 2024-04-16T17:11:55.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.195+0000 7efe51a00640 1 --2- 172.21.15.45:0/2691426856 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7efe300a3d50 0x7efe300a6210 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:55.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.195+0000 7efe51a00640 1 --2- 172.21.15.45:0/2691426856 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efe4c10c480 0x7efe4c101730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:55.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.195+0000 7efe51a00640 1 --2- 172.21.15.45:0/2691426856 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efe4c103980 0x7efe4c1011f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:55.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.195+0000 7efe51a00640 1 --2- 172.21.15.45:0/2691426856 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efe4c103060 0x7efe4c100cb0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:55.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.195+0000 7efe51a00640 1 -- 172.21.15.45:0/2691426856 >> 172.21.15.45:0/2691426856 conn(0x7efe4c0fc880 msgr2=0x7efe4c0fdfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:11:55.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.195+0000 7efe51a00640 1 -- 172.21.15.45:0/2691426856 shutdown_connections 2024-04-16T17:11:55.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.195+0000 7efe51a00640 1 -- 172.21.15.45:0/2691426856 wait complete. 2024-04-16T17:11:55.211 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-16T17:11:55.263 INFO:tasks.workunit.client.0.smithi045.stdout: 1 79023 1.65 MB/sec execute 117 sec latency 60.026 ms 2024-04-16T17:11:55.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.417+0000 7f2b73200640 1 -- 172.21.15.45:0/1326723902 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2b74102ac0 msgr2=0x7f2b74106b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:55.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.417+0000 7f2b73200640 1 --2- 172.21.15.45:0/1326723902 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2b74102ac0 0x7f2b74106b00 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f2b60009900 tx=0x7f2b6002bc30 comp rx=0 tx=0).stop 2024-04-16T17:11:55.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.417+0000 7f2b73200640 1 -- 172.21.15.45:0/1326723902 shutdown_connections 2024-04-16T17:11:55.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.417+0000 7f2b73200640 1 --2- 172.21.15.45:0/1326723902 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2b741071e0 0x7f2b7410edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:55.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.417+0000 7f2b73200640 1 --2- 172.21.15.45:0/1326723902 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2b74102ac0 0x7f2b74106b00 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:55.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.417+0000 7f2b73200640 1 --2- 172.21.15.45:0/1326723902 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2b74102110 0x7f2b741024f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:55.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.417+0000 7f2b73200640 1 -- 172.21.15.45:0/1326723902 >> 172.21.15.45:0/1326723902 conn(0x7f2b740fd9d0 msgr2=0x7f2b740ffdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:11:55.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.417+0000 7f2b73200640 1 -- 172.21.15.45:0/1326723902 shutdown_connections 2024-04-16T17:11:55.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.417+0000 7f2b73200640 1 -- 172.21.15.45:0/1326723902 wait complete. 2024-04-16T17:11:55.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.419+0000 7f2b73200640 1 Processor -- start 2024-04-16T17:11:55.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.419+0000 7f2b73200640 1 -- start start 2024-04-16T17:11:55.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.419+0000 7f2b73200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2b74102110 0x7f2b7419e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:55.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.419+0000 7f2b73200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2b741071e0 0x7f2b7419e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:55.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.420+0000 7f2b71e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2b74102110 0x7f2b7419e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:55.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.420+0000 7f2b71e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2b74102110 0x7f2b7419e460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39422/0 (socket says 172.21.15.45:39422) 2024-04-16T17:11:55.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.420+0000 7f2b71e00640 1 -- 172.21.15.45:0/4130364930 learned_addr learned my addr 172.21.15.45:0/4130364930 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:11:55.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.420+0000 7f2b73200640 1 --2- 172.21.15.45:0/4130364930 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2b74198610 0x7f2b74198ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:55.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.420+0000 7f2b73200640 1 -- 172.21.15.45:0/4130364930 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f2b74112270 con 0x7f2b74102110 2024-04-16T17:11:55.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.420+0000 7f2b73200640 1 -- 172.21.15.45:0/4130364930 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f2b741120f0 con 0x7f2b741071e0 2024-04-16T17:11:55.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.420+0000 7f2b73200640 1 -- 172.21.15.45:0/4130364930 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f2b741123f0 con 0x7f2b74198610 2024-04-16T17:11:55.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.420+0000 7f2b71400640 1 --2- 172.21.15.45:0/4130364930 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2b741071e0 0x7f2b7419e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:55.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.421+0000 7f2b72800640 1 --2- 172.21.15.45:0/4130364930 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2b74198610 0x7f2b74198ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:55.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.421+0000 7f2b71e00640 1 -- 172.21.15.45:0/4130364930 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2b74198610 msgr2=0x7f2b74198ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:55.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.421+0000 7f2b71e00640 1 --2- 172.21.15.45:0/4130364930 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2b74198610 0x7f2b74198ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:55.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.421+0000 7f2b71e00640 1 -- 172.21.15.45:0/4130364930 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2b741071e0 msgr2=0x7f2b7419e9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:55.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.421+0000 7f2b71e00640 1 --2- 172.21.15.45:0/4130364930 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2b741071e0 0x7f2b7419e9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:55.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.421+0000 7f2b71e00640 1 -- 172.21.15.45:0/4130364930 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b74199130 con 0x7f2b74102110 2024-04-16T17:11:55.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.422+0000 7f2b71e00640 1 --2- 172.21.15.45:0/4130364930 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2b74102110 0x7f2b7419e460 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f2b6400d950 tx=0x7f2b6400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:11:55.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.422+0000 7f2b6aa00640 1 -- 172.21.15.45:0/4130364930 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b64017030 con 0x7f2b74102110 2024-04-16T17:11:55.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.423+0000 7f2b6aa00640 1 -- 172.21.15.45:0/4130364930 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2b64004020 con 0x7f2b74102110 2024-04-16T17:11:55.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.423+0000 7f2b6aa00640 1 -- 172.21.15.45:0/4130364930 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b640047a0 con 0x7f2b74102110 2024-04-16T17:11:55.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.423+0000 7f2b73200640 1 -- 172.21.15.45:0/4130364930 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b74199420 con 0x7f2b74102110 2024-04-16T17:11:55.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.423+0000 7f2b73200640 1 -- 172.21.15.45:0/4130364930 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b741a5620 con 0x7f2b74102110 2024-04-16T17:11:55.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.426+0000 7f2b7bbb8740 1 Processor -- start 2024-04-16T17:11:55.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.427+0000 7f2b7bbb8740 1 -- start start 2024-04-16T17:11:55.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.427+0000 7f2b7bbb8740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55efe654ff10 0x55efe65502f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:55.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.427+0000 7f2b7bbb8740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55efe6550830 0x55efe6558ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:55.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.427+0000 7f2b7bbb8740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55efe6559410 0x55efe655b800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:55.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.427+0000 7f2b7bbb8740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55efe64b5020 con 0x55efe6550830 2024-04-16T17:11:55.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.427+0000 7f2b7bbb8740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55efe645b460 con 0x55efe6559410 2024-04-16T17:11:55.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.427+0000 7f2b7bbb8740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55efe64b0fe0 con 0x55efe654ff10 2024-04-16T17:11:55.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.427+0000 7f2b72800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55efe654ff10 0x55efe65502f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:55.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.427+0000 7f2b72800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55efe654ff10 0x55efe65502f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:44360/0 (socket says 172.21.15.45:44360) 2024-04-16T17:11:55.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.427+0000 7f2b72800640 1 -- 172.21.15.45:0/2303533354 learned_addr learned my addr 172.21.15.45:0/2303533354 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:11:55.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.428+0000 7f2b6aa00640 1 -- 172.21.15.45:0/4130364930 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f2b640041d0 con 0x7f2b74102110 2024-04-16T17:11:55.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.428+0000 7f2b72800640 1 -- 172.21.15.45:0/2303533354 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55efe6559410 msgr2=0x55efe655b800 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-16T17:11:55.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.428+0000 7f2b72800640 1 --2- 172.21.15.45:0/2303533354 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55efe6559410 0x55efe655b800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:55.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.428+0000 7f2b72800640 1 -- 172.21.15.45:0/2303533354 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55efe6550830 msgr2=0x55efe6558ed0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-16T17:11:55.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.428+0000 7f2b72800640 1 --2- 172.21.15.45:0/2303533354 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55efe6550830 0x55efe6558ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:55.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.428+0000 7f2b72800640 1 -- 172.21.15.45:0/2303533354 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55efe6463040 con 0x55efe654ff10 2024-04-16T17:11:55.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.430+0000 7f2b6aa00640 1 --2- 172.21.15.45:0/4130364930 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f2b580a3da0 0x7f2b580a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:55.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.430+0000 7f2b6aa00640 1 -- 172.21.15.45:0/4130364930 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f2b640d0480 con 0x7f2b74102110 2024-04-16T17:11:55.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.431+0000 7f2b71e00640 1 --2- 172.21.15.45:0/4130364930 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f2b580a3da0 0x7f2b580a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:55.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.431+0000 7f2b72800640 1 --2- 172.21.15.45:0/2303533354 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55efe654ff10 0x55efe65502f0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f2b6c009f50 tx=0x7f2b6c0118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1a2f79335cbebfe6 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:11:55.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.431+0000 7f2b5f400640 1 -- 172.21.15.45:0/2303533354 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b6c004d20 con 0x55efe654ff10 2024-04-16T17:11:55.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.432+0000 7f2b71e00640 1 --2- 172.21.15.45:0/4130364930 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f2b580a3da0 0x7f2b580a6260 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x55efe654f9e0 tx=0x7f2b640051b0 comp rx=0 tx=0).ready entity=mgr.25411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:11:55.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.434+0000 7f2b5f400640 1 -- 172.21.15.45:0/2303533354 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2b6c004050 con 0x55efe654ff10 2024-04-16T17:11:55.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.434+0000 7f2b5f400640 1 -- 172.21.15.45:0/2303533354 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b6c005280 con 0x55efe654ff10 2024-04-16T17:11:55.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.434+0000 7f2b7bbb8740 1 -- 172.21.15.45:0/2303533354 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55efe654ff10 msgr2=0x55efe65502f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:55.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.434+0000 7f2b7bbb8740 1 --2- 172.21.15.45:0/2303533354 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55efe654ff10 0x55efe65502f0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f2b6c009f50 tx=0x7f2b6c0118d0 comp rx=0 tx=0).stop 2024-04-16T17:11:55.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.435+0000 7f2b7bbb8740 1 -- 172.21.15.45:0/2303533354 shutdown_connections 2024-04-16T17:11:55.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.435+0000 7f2b7bbb8740 1 --2- 172.21.15.45:0/2303533354 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55efe6559410 0x55efe655b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:55.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.435+0000 7f2b7bbb8740 1 --2- 172.21.15.45:0/2303533354 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55efe6550830 0x55efe6558ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:55.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.435+0000 7f2b7bbb8740 1 --2- 172.21.15.45:0/2303533354 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55efe654ff10 0x55efe65502f0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:55.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.435+0000 7f2b7bbb8740 1 -- 172.21.15.45:0/2303533354 >> 172.21.15.45:0/2303533354 conn(0x55efe654a640 msgr2=0x55efe654aa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:11:55.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.435+0000 7f2b7bbb8740 1 -- 172.21.15.45:0/2303533354 shutdown_connections 2024-04-16T17:11:55.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.435+0000 7f2b7bbb8740 1 -- 172.21.15.45:0/2303533354 wait complete. 2024-04-16T17:11:55.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.435+0000 7f2b7bbb8740 1 -- start start 2024-04-16T17:11:55.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.435+0000 7f2b7bbb8740 1 Processor -- start 2024-04-16T17:11:55.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.436+0000 7f2b7bbb8740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55efe654ff10 0x55efe6558ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:55.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.436+0000 7f2b7bbb8740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55efe6550830 0x55efe6552240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:55.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.436+0000 7f2b7bbb8740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55efe6559410 0x55efe6552930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:55.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.436+0000 7f2b7bbb8740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55efe6560610 con 0x55efe6550830 2024-04-16T17:11:55.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.436+0000 7f2b7bbb8740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55efe646e830 con 0x55efe654ff10 2024-04-16T17:11:55.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.436+0000 7f2b7bbb8740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55efe6560790 con 0x55efe6559410 2024-04-16T17:11:55.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.436+0000 7f2b7bbb8740 10 client.0 _ll_register_callbacks cb 0x55efe6476150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:11:55.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.437+0000 7f2b72800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55efe6559410 0x55efe6552930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:55.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.437+0000 7f2b72800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55efe6559410 0x55efe6552930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:44366/0 (socket says 172.21.15.45:44366) 2024-04-16T17:11:55.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.437+0000 7f2b72800640 1 -- 172.21.15.45:0/1245803188 learned_addr learned my addr 172.21.15.45:0/1245803188 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:11:55.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.437+0000 7f2b72800640 1 --2- 172.21.15.45:0/1245803188 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55efe654ff10 0x55efe6558ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:55.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.437+0000 7f2b71400640 1 --2- 172.21.15.45:0/1245803188 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55efe6550830 0x55efe6552240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:55.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.437+0000 7f2b72800640 1 -- 172.21.15.45:0/1245803188 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55efe654ff10 msgr2=0x55efe6558ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:11:55.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.437+0000 7f2b72800640 1 --2- 172.21.15.45:0/1245803188 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55efe654ff10 0x55efe6558ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:55.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.437+0000 7f2b72800640 1 -- 172.21.15.45:0/1245803188 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55efe6550830 msgr2=0x55efe6552240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:11:55.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.437+0000 7f2b72800640 1 --2- 172.21.15.45:0/1245803188 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55efe6550830 0x55efe6552240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:55.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.437+0000 7f2b72800640 1 -- 172.21.15.45:0/1245803188 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55efe6472920 con 0x55efe6559410 2024-04-16T17:11:55.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.444+0000 7f2b72800640 1 --2- 172.21.15.45:0/1245803188 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55efe6559410 0x55efe6552930 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f2b6c009250 tx=0x7f2b6c005bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=24a9a2a567441b99 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:11:55.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.444+0000 7f2b53e00640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-04-16T17:11:55.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.444+0000 7f2b53e00640 1 -- 172.21.15.45:0/1245803188 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b6c02b020 con 0x55efe6559410 2024-04-16T17:11:55.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.444+0000 7f2b53e00640 1 -- 172.21.15.45:0/1245803188 <== mon.1 v2:172.21.15.117:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f2b6c027640 con 0x55efe6559410 2024-04-16T17:11:55.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.444+0000 7f2b7bbb8740 1 -- 172.21.15.45:0/1245803188 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2b6c004050 con 0x55efe6559410 2024-04-16T17:11:55.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.444+0000 7f2b53e00640 1 -- 172.21.15.45:0/1245803188 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f2b6c036050 con 0x55efe6559410 2024-04-16T17:11:55.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.444+0000 7f2b7bbb8740 10 client.26980 fetch_fsmap learned FSMap version 35 2024-04-16T17:11:55.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.444+0000 7f2b7bbb8740 1 -- 172.21.15.45:0/1245803188 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55efe64407f0 con 0x55efe6559410 2024-04-16T17:11:55.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.444+0000 7f2b53e00640 1 -- 172.21.15.45:0/1245803188 <== mon.1 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2b6c004050 con 0x55efe6559410 2024-04-16T17:11:55.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.445+0000 7f2b53e00640 1 -- 172.21.15.45:0/1245803188 <== mon.1 v2:172.21.15.117:3300/0 5 ==== fsmap(e 35) ==== 8358+0+0 (secure 0 0 0) 0x7f2b6c00f8f0 con 0x55efe6559410 2024-04-16T17:11:55.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.445+0000 7f2b7bbb8740 10 client.26980 fetch_fsmap finished waiting for FSMap version 35 2024-04-16T17:11:55.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.445+0000 7f2b7bbb8740 10 client.26980 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-04-16T17:11:55.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.445+0000 7f2b7bbb8740 20 client.26980 populate_metadata read hostname 'smithi045' 2024-04-16T17:11:55.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.445+0000 7f2b7bbb8740 1 --2- 172.21.15.45:0/1245803188 >> [v2:172.21.15.45:6840/3755310822,v1:172.21.15.45:6841/3755310822] conn(0x55efe65831e0 0x55efe65855d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:55.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.445+0000 7f2b7bbb8740 4 client.26980 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:11:55.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.445+0000 7f2b7bbb8740 1 -- 172.21.15.45:0/1245803188 --> [v2:172.21.15.45:6840/3755310822,v1:172.21.15.45:6841/3755310822] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55efe64c2a40 con 0x55efe65831e0 2024-04-16T17:11:55.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.447+0000 7f2b71400640 1 --2- 172.21.15.45:0/1245803188 >> [v2:172.21.15.45:6840/3755310822,v1:172.21.15.45:6841/3755310822] conn(0x55efe65831e0 0x55efe65855d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:55.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.447+0000 7f2b71400640 1 --2- 172.21.15.45:0/1245803188 >> [v2:172.21.15.45:6840/3755310822,v1:172.21.15.45:6841/3755310822] conn(0x55efe65831e0 0x55efe65855d0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5a2604c98371e1d0 server_cookie=3cf53c6b1cd1070 in_seq=0 out_seq=0 2024-04-16T17:11:55.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.447+0000 7f2b53e00640 10 client.26980 ms_handle_connect on v2:172.21.15.45:6840/3755310822 2024-04-16T17:11:55.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.448+0000 7f2b53e00640 1 -- 172.21.15.45:0/1245803188 <== mds.1 v2:172.21.15.45:6840/3755310822 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55efe64c2a40 con 0x55efe65831e0 2024-04-16T17:11:55.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.448+0000 7f2b53e00640 10 client.26980 handle_command_reply: tid=0 2024-04-16T17:11:55.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.448+0000 7f2b7bbb8740 1 client.26980 shutdown 2024-04-16T17:11:55.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.449+0000 7f2b7bbb8740 1 -- 172.21.15.45:0/1245803188 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55efe6559410 msgr2=0x55efe6552930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:55.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.449+0000 7f2b7bbb8740 1 --2- 172.21.15.45:0/1245803188 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55efe6559410 0x55efe6552930 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f2b6c009250 tx=0x7f2b6c005bd0 comp rx=0 tx=0).stop 2024-04-16T17:11:55.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.449+0000 7f2b7bbb8740 1 -- 172.21.15.45:0/1245803188 shutdown_connections 2024-04-16T17:11:55.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.449+0000 7f2b7bbb8740 1 --2- 172.21.15.45:0/1245803188 >> [v2:172.21.15.45:6840/3755310822,v1:172.21.15.45:6841/3755310822] conn(0x55efe65831e0 0x55efe65855d0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:55.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.449+0000 7f2b7bbb8740 1 --2- 172.21.15.45:0/1245803188 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55efe6559410 0x55efe6552930 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:55.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.449+0000 7f2b7bbb8740 1 --2- 172.21.15.45:0/1245803188 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55efe6550830 0x55efe6552240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:55.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.449+0000 7f2b53e00640 0 client.26980 ms_handle_reset on v2:172.21.15.45:6840/3755310822 2024-04-16T17:11:55.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.449+0000 7f2b7bbb8740 1 --2- 172.21.15.45:0/1245803188 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55efe654ff10 0x55efe6558ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:55.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.449+0000 7f2b7bbb8740 1 -- 172.21.15.45:0/1245803188 >> 172.21.15.45:0/1245803188 conn(0x55efe654a640 msgr2=0x55efe654c390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:11:55.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.449+0000 7f2b7bbb8740 1 -- 172.21.15.45:0/1245803188 shutdown_connections 2024-04-16T17:11:55.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.449+0000 7f2b7bbb8740 1 -- 172.21.15.45:0/1245803188 wait complete. 2024-04-16T17:11:55.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.449+0000 7f2b7bbb8740 20 client.26980 trim_cache size 0 max 16384 2024-04-16T17:11:55.466 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.466+0000 7f2b7bbb8740 1 Processor -- start 2024-04-16T17:11:55.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.466+0000 7f2b7bbb8740 1 -- start start 2024-04-16T17:11:55.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.466+0000 7f2b7bbb8740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55efe654ff10 0x55efe658ad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:55.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.466+0000 7f2b7bbb8740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55efe6550830 0x55efe658b280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:55.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.466+0000 7f2b7bbb8740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55efe6559410 0x55efe65587d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:55.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.466+0000 7f2b7bbb8740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f2b741a7210 con 0x55efe6559410 2024-04-16T17:11:55.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.466+0000 7f2b7bbb8740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f2b740755a0 con 0x55efe6550830 2024-04-16T17:11:55.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.466+0000 7f2b7bbb8740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f2b740733d0 con 0x55efe654ff10 2024-04-16T17:11:55.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.467+0000 7f2b71400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55efe6550830 0x55efe658b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:55.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.467+0000 7f2b72800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55efe654ff10 0x55efe658ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:55.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.467+0000 7f2b71400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55efe6550830 0x55efe658b280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:60890/0 (socket says 172.21.15.45:60890) 2024-04-16T17:11:55.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.467+0000 7f2b71400640 1 -- 172.21.15.45:0/1464294028 learned_addr learned my addr 172.21.15.45:0/1464294028 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:11:55.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.467+0000 7f2b72800640 1 --2- 172.21.15.45:0/1464294028 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55efe6559410 0x55efe65587d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:55.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.467+0000 7f2b71400640 1 -- 172.21.15.45:0/1464294028 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55efe654ff10 msgr2=0x55efe658ad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:11:55.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.467+0000 7f2b71400640 1 --2- 172.21.15.45:0/1464294028 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55efe654ff10 0x55efe658ad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:55.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.467+0000 7f2b71400640 1 -- 172.21.15.45:0/1464294028 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55efe6559410 msgr2=0x55efe65587d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:11:55.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.467+0000 7f2b71400640 1 --2- 172.21.15.45:0/1464294028 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55efe6559410 0x55efe65587d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:55.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.467+0000 7f2b71400640 1 -- 172.21.15.45:0/1464294028 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55efe61c73f0 con 0x55efe6550830 2024-04-16T17:11:55.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.467+0000 7f2b71400640 1 --2- 172.21.15.45:0/1464294028 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55efe6550830 0x55efe658b280 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f2b60009900 tx=0x7f2b6002e040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=804b2ee20ebb8278 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:11:55.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.468+0000 7f2b5ea00640 1 -- 172.21.15.45:0/1464294028 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b60046b50 con 0x55efe6550830 2024-04-16T17:11:55.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.468+0000 7f2b5ea00640 1 -- 172.21.15.45:0/1464294028 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2b6004b070 con 0x55efe6550830 2024-04-16T17:11:55.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.468+0000 7f2b5ea00640 1 -- 172.21.15.45:0/1464294028 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b60004920 con 0x55efe6550830 2024-04-16T17:11:55.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.469+0000 7f2b7bbb8740 1 -- 172.21.15.45:0/1464294028 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55efe6550830 msgr2=0x55efe658b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:55.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.469+0000 7f2b7bbb8740 1 --2- 172.21.15.45:0/1464294028 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55efe6550830 0x55efe658b280 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f2b60009900 tx=0x7f2b6002e040 comp rx=0 tx=0).stop 2024-04-16T17:11:55.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.469+0000 7f2b7bbb8740 1 -- 172.21.15.45:0/1464294028 shutdown_connections 2024-04-16T17:11:55.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.469+0000 7f2b7bbb8740 1 --2- 172.21.15.45:0/1464294028 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55efe6559410 0x55efe65587d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:55.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.469+0000 7f2b7bbb8740 1 --2- 172.21.15.45:0/1464294028 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55efe6550830 0x55efe658b280 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:55.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.469+0000 7f2b7bbb8740 1 --2- 172.21.15.45:0/1464294028 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55efe654ff10 0x55efe658ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:55.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.469+0000 7f2b7bbb8740 1 -- 172.21.15.45:0/1464294028 >> 172.21.15.45:0/1464294028 conn(0x55efe654a640 msgr2=0x55efe655bdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:11:55.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.469+0000 7f2b7bbb8740 1 -- 172.21.15.45:0/1464294028 shutdown_connections 2024-04-16T17:11:55.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.469+0000 7f2b7bbb8740 1 -- 172.21.15.45:0/1464294028 wait complete. 2024-04-16T17:11:55.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.470+0000 7f2b7bbb8740 1 -- start start 2024-04-16T17:11:55.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.470+0000 7f2b7bbb8740 1 Processor -- start 2024-04-16T17:11:55.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.470+0000 7f2b7bbb8740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55efe654ff10 0x55efe656e840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:55.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.470+0000 7f2b7bbb8740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55efe6550830 0x55efe6569bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:55.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.470+0000 7f2b7bbb8740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55efe6559410 0x55efe656a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:55.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.470+0000 7f2b7bbb8740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f2b74073a60 con 0x55efe654ff10 2024-04-16T17:11:55.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.470+0000 7f2b7bbb8740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f2b74074ed0 con 0x55efe6550830 2024-04-16T17:11:55.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.470+0000 7f2b7bbb8740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f2b74074130 con 0x55efe6559410 2024-04-16T17:11:55.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.471+0000 7f2b7bbb8740 10 client.0 _ll_register_callbacks cb 0x7f2b6c009e20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:11:55.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.471+0000 7f2b71400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55efe6550830 0x55efe6569bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:55.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.471+0000 7f2b71400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55efe6550830 0x55efe6569bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:60898/0 (socket says 172.21.15.45:60898) 2024-04-16T17:11:55.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.471+0000 7f2b71400640 1 -- 172.21.15.45:0/652818635 learned_addr learned my addr 172.21.15.45:0/652818635 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:11:55.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.471+0000 7f2b72800640 1 --2- 172.21.15.45:0/652818635 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55efe654ff10 0x55efe656e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:55.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.471+0000 7f2b72800640 1 --2- 172.21.15.45:0/652818635 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55efe6559410 0x55efe656a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:55.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.471+0000 7f2b71400640 1 -- 172.21.15.45:0/652818635 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55efe6559410 msgr2=0x55efe656a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:11:55.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.471+0000 7f2b71400640 1 --2- 172.21.15.45:0/652818635 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55efe6559410 0x55efe656a0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:55.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.471+0000 7f2b71400640 1 -- 172.21.15.45:0/652818635 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55efe654ff10 msgr2=0x55efe656e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:11:55.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.471+0000 7f2b71400640 1 --2- 172.21.15.45:0/652818635 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55efe654ff10 0x55efe656e840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:55.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.471+0000 7f2b71400640 1 -- 172.21.15.45:0/652818635 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55efe62917d0 con 0x55efe6550830 2024-04-16T17:11:55.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.471+0000 7f2b72800640 1 --2- 172.21.15.45:0/652818635 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55efe6559410 0x55efe656a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:11:55.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.472+0000 7f2b71400640 1 --2- 172.21.15.45:0/652818635 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55efe6550830 0x55efe6569bb0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f2b6002c170 tx=0x7f2b60005c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5024635a495d6077 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:11:55.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.472+0000 7f2b53e00640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-04-16T17:11:55.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.472+0000 7f2b53e00640 1 -- 172.21.15.45:0/652818635 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b60004100 con 0x55efe6550830 2024-04-16T17:11:55.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.472+0000 7f2b53e00640 1 -- 172.21.15.45:0/652818635 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f2b60045070 con 0x55efe6550830 2024-04-16T17:11:55.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.472+0000 7f2b7bbb8740 1 -- 172.21.15.45:0/652818635 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2b6004b070 con 0x55efe6550830 2024-04-16T17:11:55.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.472+0000 7f2b53e00640 1 -- 172.21.15.45:0/652818635 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f2b6005c050 con 0x55efe6550830 2024-04-16T17:11:55.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.472+0000 7f2b7bbb8740 10 client.27035 fetch_fsmap learned FSMap version 35 2024-04-16T17:11:55.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.472+0000 7f2b7bbb8740 1 -- 172.21.15.45:0/652818635 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55efe65e90d0 con 0x55efe6550830 2024-04-16T17:11:55.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.473+0000 7f2b53e00640 1 -- 172.21.15.45:0/652818635 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2b6004b070 con 0x55efe6550830 2024-04-16T17:11:55.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.473+0000 7f2b53e00640 1 -- 172.21.15.45:0/652818635 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 35) ==== 8358+0+0 (secure 0 0 0) 0x7f2b60005160 con 0x55efe6550830 2024-04-16T17:11:55.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.473+0000 7f2b7bbb8740 10 client.27035 fetch_fsmap finished waiting for FSMap version 35 2024-04-16T17:11:55.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.473+0000 7f2b7bbb8740 10 client.27035 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-04-16T17:11:55.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.473+0000 7f2b7bbb8740 20 client.27035 populate_metadata read hostname 'smithi045' 2024-04-16T17:11:55.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.473+0000 7f2b7bbb8740 1 --2- 172.21.15.45:0/652818635 >> [v2:172.21.15.45:6840/3755310822,v1:172.21.15.45:6841/3755310822] conn(0x55efe65e97f0 0x55efe65ebc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:55.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.473+0000 7f2b7bbb8740 4 client.27035 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T17:11:55.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.473+0000 7f2b7bbb8740 1 -- 172.21.15.45:0/652818635 --> [v2:172.21.15.45:6840/3755310822,v1:172.21.15.45:6841/3755310822] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55efe64bc8b0 con 0x55efe65e97f0 2024-04-16T17:11:55.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.474+0000 7f2b72800640 1 --2- 172.21.15.45:0/652818635 >> [v2:172.21.15.45:6840/3755310822,v1:172.21.15.45:6841/3755310822] conn(0x55efe65e97f0 0x55efe65ebc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:55.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.474+0000 7f2b72800640 1 --2- 172.21.15.45:0/652818635 >> [v2:172.21.15.45:6840/3755310822,v1:172.21.15.45:6841/3755310822] conn(0x55efe65e97f0 0x55efe65ebc10 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d8f37fdb50d3b6fa server_cookie=d29ae12a2201800 in_seq=0 out_seq=0 2024-04-16T17:11:55.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.474+0000 7f2b53e00640 10 client.27035 ms_handle_connect on v2:172.21.15.45:6840/3755310822 2024-04-16T17:11:55.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.474+0000 7f2b53e00640 1 -- 172.21.15.45:0/652818635 <== mds.1 v2:172.21.15.45:6840/3755310822 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55efe64bc8b0 con 0x55efe65e97f0 2024-04-16T17:11:55.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.474+0000 7f2b53e00640 10 client.27035 handle_command_reply: tid=0 2024-04-16T17:11:55.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.474+0000 7f2b7bbb8740 1 client.27035 shutdown 2024-04-16T17:11:55.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.475+0000 7f2b7bbb8740 1 -- 172.21.15.45:0/652818635 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55efe6550830 msgr2=0x55efe6569bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:55.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.475+0000 7f2b7bbb8740 1 --2- 172.21.15.45:0/652818635 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55efe6550830 0x55efe6569bb0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f2b6002c170 tx=0x7f2b60005c00 comp rx=0 tx=0).stop 2024-04-16T17:11:55.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.475+0000 7f2b7bbb8740 1 -- 172.21.15.45:0/652818635 shutdown_connections 2024-04-16T17:11:55.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.475+0000 7f2b7bbb8740 1 --2- 172.21.15.45:0/652818635 >> [v2:172.21.15.45:6840/3755310822,v1:172.21.15.45:6841/3755310822] conn(0x55efe65e97f0 0x55efe65ebc10 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:55.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.475+0000 7f2b7bbb8740 1 --2- 172.21.15.45:0/652818635 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55efe6559410 0x55efe656a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:55.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.475+0000 7f2b7bbb8740 1 --2- 172.21.15.45:0/652818635 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55efe6550830 0x55efe6569bb0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:55.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.475+0000 7f2b7bbb8740 1 --2- 172.21.15.45:0/652818635 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55efe654ff10 0x55efe656e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:55.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.475+0000 7f2b53e00640 0 client.27035 ms_handle_reset on v2:172.21.15.45:6840/3755310822 2024-04-16T17:11:55.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.475+0000 7f2b7bbb8740 1 -- 172.21.15.45:0/652818635 >> 172.21.15.45:0/652818635 conn(0x55efe654a640 msgr2=0x55efe6555800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:11:55.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.475+0000 7f2b7bbb8740 1 -- 172.21.15.45:0/652818635 shutdown_connections 2024-04-16T17:11:55.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.475+0000 7f2b7bbb8740 1 -- 172.21.15.45:0/652818635 wait complete. 2024-04-16T17:11:55.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.475+0000 7f2b7bbb8740 20 client.27035 trim_cache size 0 max 16384 2024-04-16T17:11:55.479 INFO:teuthology.orchestra.run.smithi045.stdout:[] 2024-04-16T17:11:55.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.476+0000 7f2b73200640 1 -- 172.21.15.45:0/4130364930 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f2b580a3da0 msgr2=0x7f2b580a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:55.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.476+0000 7f2b73200640 1 --2- 172.21.15.45:0/4130364930 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f2b580a3da0 0x7f2b580a6260 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x55efe654f9e0 tx=0x7f2b640051b0 comp rx=0 tx=0).stop 2024-04-16T17:11:55.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.476+0000 7f2b73200640 1 -- 172.21.15.45:0/4130364930 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2b74102110 msgr2=0x7f2b7419e460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:55.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.476+0000 7f2b73200640 1 --2- 172.21.15.45:0/4130364930 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2b74102110 0x7f2b7419e460 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f2b6400d950 tx=0x7f2b6400de10 comp rx=0 tx=0).stop 2024-04-16T17:11:55.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.476+0000 7f2b73200640 1 -- 172.21.15.45:0/4130364930 shutdown_connections 2024-04-16T17:11:55.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.476+0000 7f2b73200640 1 --2- 172.21.15.45:0/4130364930 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f2b580a3da0 0x7f2b580a6260 secure :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0x55efe654f9e0 tx=0x7f2b640051b0 comp rx=0 tx=0).stop 2024-04-16T17:11:55.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.476+0000 7f2b73200640 1 --2- 172.21.15.45:0/4130364930 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2b74198610 0x7f2b74198ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:55.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.476+0000 7f2b73200640 1 --2- 172.21.15.45:0/4130364930 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2b741071e0 0x7f2b7419e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:55.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.476+0000 7f2b73200640 1 --2- 172.21.15.45:0/4130364930 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2b74102110 0x7f2b7419e460 secure :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f2b6400d950 tx=0x7f2b6400de10 comp rx=0 tx=0).stop 2024-04-16T17:11:55.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.476+0000 7f2b73200640 1 -- 172.21.15.45:0/4130364930 >> 172.21.15.45:0/4130364930 conn(0x7f2b740fd9d0 msgr2=0x7f2b740fedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:11:55.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.476+0000 7f2b73200640 1 -- 172.21.15.45:0/4130364930 shutdown_connections 2024-04-16T17:11:55.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.476+0000 7f2b73200640 1 -- 172.21.15.45:0/4130364930 wait complete. 2024-04-16T17:11:55.492 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-16T17:11:55.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.684+0000 7f5fe1400640 1 -- 172.21.15.45:0/3667072358 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5fdc0743d0 msgr2=0x7f5fdc0747b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:55.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.684+0000 7f5fe1400640 1 --2- 172.21.15.45:0/3667072358 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5fdc0743d0 0x7f5fdc0747b0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f5fcc009900 tx=0x7f5fcc02bcc0 comp rx=0 tx=0).stop 2024-04-16T17:11:55.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.684+0000 7f5fe1400640 1 -- 172.21.15.45:0/3667072358 shutdown_connections 2024-04-16T17:11:55.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.684+0000 7f5fe1400640 1 --2- 172.21.15.45:0/3667072358 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5fdc10e740 0x7f5fdc110b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:55.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.684+0000 7f5fe1400640 1 --2- 172.21.15.45:0/3667072358 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5fdc074cf0 0x7f5fdc10e200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:55.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.684+0000 7f5fe1400640 1 --2- 172.21.15.45:0/3667072358 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5fdc0743d0 0x7f5fdc0747b0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:55.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.684+0000 7f5fe1400640 1 -- 172.21.15.45:0/3667072358 >> 172.21.15.45:0/3667072358 conn(0x7f5fdc0fd9d0 msgr2=0x7f5fdc0ffdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:11:55.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.684+0000 7f5fe1400640 1 -- 172.21.15.45:0/3667072358 shutdown_connections 2024-04-16T17:11:55.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.685+0000 7f5fe1400640 1 -- 172.21.15.45:0/3667072358 wait complete. 2024-04-16T17:11:55.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.686+0000 7f5fe1400640 1 Processor -- start 2024-04-16T17:11:55.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.687+0000 7f5fe1400640 1 -- start start 2024-04-16T17:11:55.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.687+0000 7f5fe1400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5fdc0743d0 0x7f5fdc10e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:55.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.687+0000 7f5fe1400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5fdc074cf0 0x7f5fdc10b150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:55.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.687+0000 7f5fe1400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5fdc10e740 0x7f5fdc10b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:55.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.687+0000 7f5fe1400640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f5fdc114510 con 0x7f5fdc10e740 2024-04-16T17:11:55.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.687+0000 7f5fe1400640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f5fdc114390 con 0x7f5fdc074cf0 2024-04-16T17:11:55.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.687+0000 7f5fe1400640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f5fdc114690 con 0x7f5fdc0743d0 2024-04-16T17:11:55.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.687+0000 7f5fdbe00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5fdc0743d0 0x7f5fdc10e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:55.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.687+0000 7f5fdbe00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5fdc0743d0 0x7f5fdc10e0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:44406/0 (socket says 172.21.15.45:44406) 2024-04-16T17:11:55.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.687+0000 7f5fdbe00640 1 -- 172.21.15.45:0/2673639475 learned_addr learned my addr 172.21.15.45:0/2673639475 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:11:55.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.687+0000 7f5fdb400640 1 --2- 172.21.15.45:0/2673639475 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5fdc074cf0 0x7f5fdc10b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:55.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.688+0000 7f5fe0a00640 1 --2- 172.21.15.45:0/2673639475 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5fdc10e740 0x7f5fdc10b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:55.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.688+0000 7f5fdbe00640 1 -- 172.21.15.45:0/2673639475 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5fdc074cf0 msgr2=0x7f5fdc10b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:55.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.688+0000 7f5fdbe00640 1 --2- 172.21.15.45:0/2673639475 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5fdc074cf0 0x7f5fdc10b150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:55.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.688+0000 7f5fdbe00640 1 -- 172.21.15.45:0/2673639475 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5fdc10e740 msgr2=0x7f5fdc10b6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:55.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.688+0000 7f5fdbe00640 1 --2- 172.21.15.45:0/2673639475 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5fdc10e740 0x7f5fdc10b6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:55.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.688+0000 7f5fdbe00640 1 -- 172.21.15.45:0/2673639475 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fdc10bf70 con 0x7f5fdc0743d0 2024-04-16T17:11:55.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.688+0000 7f5fe0a00640 1 --2- 172.21.15.45:0/2673639475 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5fdc10e740 0x7f5fdc10b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:11:55.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.688+0000 7f5fdbe00640 1 --2- 172.21.15.45:0/2673639475 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5fdc0743d0 0x7f5fdc10e0b0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f5fcc000c00 tx=0x7f5fcc004360 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:11:55.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.688+0000 7f5fd8c00640 1 -- 172.21.15.45:0/2673639475 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fcc042020 con 0x7f5fdc0743d0 2024-04-16T17:11:55.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.688+0000 7f5fd8c00640 1 -- 172.21.15.45:0/2673639475 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5fcc005000 con 0x7f5fdc0743d0 2024-04-16T17:11:55.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.688+0000 7f5fe1400640 1 -- 172.21.15.45:0/2673639475 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5fdc1ab310 con 0x7f5fdc0743d0 2024-04-16T17:11:55.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.688+0000 7f5fd8c00640 1 -- 172.21.15.45:0/2673639475 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fcc003b80 con 0x7f5fdc0743d0 2024-04-16T17:11:55.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.689+0000 7f5fe1400640 1 -- 172.21.15.45:0/2673639475 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5fdc1ab860 con 0x7f5fdc0743d0 2024-04-16T17:11:55.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.692+0000 7f5fd8c00640 1 -- 172.21.15.45:0/2673639475 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f5fcc045020 con 0x7f5fdc0743d0 2024-04-16T17:11:55.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.693+0000 7f5fe57a0740 1 Processor -- start 2024-04-16T17:11:55.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.694+0000 7f5fe57a0740 1 -- start start 2024-04-16T17:11:55.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.694+0000 7f5fe57a0740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564e934fdf10 0x564e934fe2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:55.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.694+0000 7f5fd8c00640 1 --2- 172.21.15.45:0/2673639475 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f5fc00a3d20 0x7f5fc00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:55.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.694+0000 7f5fe57a0740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564e934fe830 0x564e93506ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:55.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.694+0000 7f5fe57a0740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564e93507410 0x564e93509800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:55.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.694+0000 7f5fe57a0740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x564e93463020 con 0x564e934fe830 2024-04-16T17:11:55.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.694+0000 7f5fe57a0740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x564e93409460 con 0x564e934fdf10 2024-04-16T17:11:55.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.694+0000 7f5fe57a0740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x564e9345efe0 con 0x564e93507410 2024-04-16T17:11:55.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.694+0000 7f5fd8c00640 1 -- 172.21.15.45:0/2673639475 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f5fcc0f26e0 con 0x7f5fdc0743d0 2024-04-16T17:11:55.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.694+0000 7f5fe0a00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564e934fdf10 0x564e934fe2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:55.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.695+0000 7f5fe0a00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564e934fdf10 0x564e934fe2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:60926/0 (socket says 172.21.15.45:60926) 2024-04-16T17:11:55.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.695+0000 7f5fe0a00640 1 -- 172.21.15.45:0/664272515 learned_addr learned my addr 172.21.15.45:0/664272515 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:11:55.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.695+0000 7f5fe0a00640 1 --2- 172.21.15.45:0/664272515 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564e93507410 0x564e93509800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:55.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.695+0000 7f5fdb400640 1 --2- 172.21.15.45:0/664272515 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564e934fe830 0x564e93506ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:55.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.695+0000 7f5fe0a00640 1 -- 172.21.15.45:0/664272515 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564e93507410 msgr2=0x564e93509800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:11:55.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.695+0000 7f5fe0a00640 1 --2- 172.21.15.45:0/664272515 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564e93507410 0x564e93509800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:55.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.695+0000 7f5fe0a00640 1 -- 172.21.15.45:0/664272515 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564e934fe830 msgr2=0x564e93506ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:11:55.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.695+0000 7f5fe0a00640 1 --2- 172.21.15.45:0/664272515 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564e934fe830 0x564e93506ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:55.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.695+0000 7f5fe0a00640 1 -- 172.21.15.45:0/664272515 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564e93411040 con 0x564e934fdf10 2024-04-16T17:11:55.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.695+0000 7f5fdbe00640 1 --2- 172.21.15.45:0/2673639475 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f5fc00a3d20 0x7f5fc00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:55.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.695+0000 7f5fe0a00640 1 --2- 172.21.15.45:0/664272515 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564e934fdf10 0x564e934fe2f0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f5fd4009270 tx=0x7f5fd40099c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=22b49d3bf4ab8c98 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:11:55.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.696+0000 7f5fd1200640 1 -- 172.21.15.45:0/664272515 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fd4012110 con 0x564e934fdf10 2024-04-16T17:11:55.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.696+0000 7f5fd1200640 1 -- 172.21.15.45:0/664272515 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5fd40122b0 con 0x564e934fdf10 2024-04-16T17:11:55.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.696+0000 7f5fd1200640 1 -- 172.21.15.45:0/664272515 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fd4005280 con 0x564e934fdf10 2024-04-16T17:11:55.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.697+0000 7f5fe57a0740 1 -- 172.21.15.45:0/664272515 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564e934fdf10 msgr2=0x564e934fe2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:55.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.697+0000 7f5fdbe00640 1 --2- 172.21.15.45:0/2673639475 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f5fc00a3d20 0x7f5fc00a61e0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f5fcc02c8b0 tx=0x7f5fcc0ba040 comp rx=0 tx=0).ready entity=mgr.25411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:11:55.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.697+0000 7f5fe57a0740 1 --2- 172.21.15.45:0/664272515 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564e934fdf10 0x564e934fe2f0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f5fd4009270 tx=0x7f5fd40099c0 comp rx=0 tx=0).stop 2024-04-16T17:11:55.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.697+0000 7f5fe57a0740 1 -- 172.21.15.45:0/664272515 shutdown_connections 2024-04-16T17:11:55.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.697+0000 7f5fe57a0740 1 --2- 172.21.15.45:0/664272515 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564e93507410 0x564e93509800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:55.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.697+0000 7f5fe57a0740 1 --2- 172.21.15.45:0/664272515 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564e934fe830 0x564e93506ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:55.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.697+0000 7f5fe57a0740 1 --2- 172.21.15.45:0/664272515 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564e934fdf10 0x564e934fe2f0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:55.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.697+0000 7f5fe57a0740 1 -- 172.21.15.45:0/664272515 >> 172.21.15.45:0/664272515 conn(0x564e934f8640 msgr2=0x564e934f8a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:11:55.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.697+0000 7f5fe57a0740 1 -- 172.21.15.45:0/664272515 shutdown_connections 2024-04-16T17:11:55.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.697+0000 7f5fe57a0740 1 -- 172.21.15.45:0/664272515 wait complete. 2024-04-16T17:11:55.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.698+0000 7f5fe57a0740 1 -- start start 2024-04-16T17:11:55.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.698+0000 7f5fe57a0740 1 Processor -- start 2024-04-16T17:11:55.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.698+0000 7f5fe57a0740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564e934fdf10 0x564e93504470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:55.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.699+0000 7f5fe57a0740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564e934fe830 0x564e935049b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:55.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.699+0000 7f5fe57a0740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564e93507410 0x564e93504ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:55.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.699+0000 7f5fe57a0740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x564e9350a200 con 0x564e934fe830 2024-04-16T17:11:55.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.699+0000 7f5fe57a0740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x564e9341c830 con 0x564e93507410 2024-04-16T17:11:55.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.699+0000 7f5fe57a0740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x564e9350a380 con 0x564e934fdf10 2024-04-16T17:11:55.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.699+0000 7f5fdb400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564e934fe830 0x564e935049b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:55.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.699+0000 7f5fe0a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564e934fdf10 0x564e93504470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:55.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.699+0000 7f5fdb400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564e934fe830 0x564e935049b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39494/0 (socket says 172.21.15.45:39494) 2024-04-16T17:11:55.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.699+0000 7f5fdb400640 1 -- 172.21.15.45:0/307961039 learned_addr learned my addr 172.21.15.45:0/307961039 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:11:55.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.699+0000 7f5fe57a0740 10 client.0 _ll_register_callbacks cb 0x564e93424150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:11:55.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.699+0000 7f5fe0a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564e934fdf10 0x564e93504470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:44432/0 (socket says 172.21.15.45:44432) 2024-04-16T17:11:55.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.699+0000 7f5fe0a00640 1 --2- 172.21.15.45:0/307961039 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564e93507410 0x564e93504ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:55.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.699+0000 7f5fdb400640 1 -- 172.21.15.45:0/307961039 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564e934fdf10 msgr2=0x564e93504470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:11:55.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.699+0000 7f5fdb400640 1 --2- 172.21.15.45:0/307961039 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564e934fdf10 0x564e93504470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:55.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.699+0000 7f5fdb400640 1 -- 172.21.15.45:0/307961039 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564e93507410 msgr2=0x564e93504ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:11:55.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.699+0000 7f5fdb400640 1 --2- 172.21.15.45:0/307961039 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564e93507410 0x564e93504ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:55.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.699+0000 7f5fdb400640 1 -- 172.21.15.45:0/307961039 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564e93420920 con 0x564e934fe830 2024-04-16T17:11:55.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.700+0000 7f5fdb400640 1 --2- 172.21.15.45:0/307961039 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564e934fe830 0x564e935049b0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f5fc8013710 tx=0x7f5fc8013bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d413322aa03ba9cf server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:11:55.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.700+0000 7f5fc5600640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-16T17:11:55.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.700+0000 7f5fc5600640 1 -- 172.21.15.45:0/307961039 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fc8004460 con 0x564e934fe830 2024-04-16T17:11:55.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.700+0000 7f5fc5600640 1 -- 172.21.15.45:0/307961039 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f5fc80119c0 con 0x564e934fe830 2024-04-16T17:11:55.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.700+0000 7f5fe57a0740 1 -- 172.21.15.45:0/307961039 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5fd40122b0 con 0x564e934fe830 2024-04-16T17:11:55.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.700+0000 7f5fc5600640 1 -- 172.21.15.45:0/307961039 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f5fc8011bc0 con 0x564e934fe830 2024-04-16T17:11:55.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.700+0000 7f5fe57a0740 10 client.16896 fetch_fsmap learned FSMap version 35 2024-04-16T17:11:55.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.700+0000 7f5fe57a0740 1 -- 172.21.15.45:0/307961039 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x564e933ee7f0 con 0x564e934fe830 2024-04-16T17:11:55.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.701+0000 7f5fc5600640 1 -- 172.21.15.45:0/307961039 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5fd40122b0 con 0x564e934fe830 2024-04-16T17:11:55.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.701+0000 7f5fc5600640 1 -- 172.21.15.45:0/307961039 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 35) ==== 8358+0+0 (secure 0 0 0) 0x7f5fc801a070 con 0x564e934fe830 2024-04-16T17:11:55.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.702+0000 7f5fe57a0740 10 client.16896 fetch_fsmap finished waiting for FSMap version 35 2024-04-16T17:11:55.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.702+0000 7f5fe57a0740 10 client.16896 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:11:55.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.702+0000 7f5fe57a0740 20 client.16896 populate_metadata read hostname 'smithi045' 2024-04-16T17:11:55.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.702+0000 7f5fe57a0740 1 --2- 172.21.15.45:0/307961039 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x564e935178f0 0x564e93519d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:55.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.702+0000 7f5fe57a0740 4 client.16896 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:11:55.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.702+0000 7f5fe57a0740 1 -- 172.21.15.45:0/307961039 --> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564e93470a40 con 0x564e935178f0 2024-04-16T17:11:55.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.702+0000 7f5fe0a00640 1 --2- 172.21.15.45:0/307961039 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x564e935178f0 0x564e93519d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:55.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.703+0000 7f5fe0a00640 1 --2- 172.21.15.45:0/307961039 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x564e935178f0 0x564e93519d30 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f6a58c8682c74183 server_cookie=53abaef28a280605 in_seq=0 out_seq=0 2024-04-16T17:11:55.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.703+0000 7f5fc5600640 10 client.16896 ms_handle_connect on v2:172.21.15.104:6838/3660070592 2024-04-16T17:11:55.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.704+0000 7f5fc5600640 1 -- 172.21.15.45:0/307961039 <== mds.0 v2:172.21.15.104:6838/3660070592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564e93470a40 con 0x564e935178f0 2024-04-16T17:11:55.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.704+0000 7f5fc5600640 10 client.16896 handle_command_reply: tid=0 2024-04-16T17:11:55.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.704+0000 7f5fe57a0740 1 client.16896 shutdown 2024-04-16T17:11:55.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.704+0000 7f5fe57a0740 1 -- 172.21.15.45:0/307961039 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564e934fe830 msgr2=0x564e935049b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:55.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.704+0000 7f5fe57a0740 1 --2- 172.21.15.45:0/307961039 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564e934fe830 0x564e935049b0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f5fc8013710 tx=0x7f5fc8013bd0 comp rx=0 tx=0).stop 2024-04-16T17:11:55.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.704+0000 7f5fe57a0740 1 -- 172.21.15.45:0/307961039 shutdown_connections 2024-04-16T17:11:55.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.704+0000 7f5fe57a0740 1 --2- 172.21.15.45:0/307961039 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564e93507410 0x564e93504ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:55.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.704+0000 7f5fe57a0740 1 --2- 172.21.15.45:0/307961039 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564e934fe830 0x564e935049b0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:55.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.704+0000 7f5fe57a0740 1 --2- 172.21.15.45:0/307961039 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x564e935178f0 0x564e93519d30 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:55.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.704+0000 7f5fe57a0740 1 --2- 172.21.15.45:0/307961039 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564e934fdf10 0x564e93504470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:55.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.704+0000 7f5fe57a0740 1 -- 172.21.15.45:0/307961039 >> 172.21.15.45:0/307961039 conn(0x564e934f8640 msgr2=0x564e934fa390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:11:55.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.704+0000 7f5fc5600640 0 client.16896 ms_handle_reset on v2:172.21.15.104:6838/3660070592 2024-04-16T17:11:55.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.704+0000 7f5fe57a0740 1 -- 172.21.15.45:0/307961039 shutdown_connections 2024-04-16T17:11:55.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.704+0000 7f5fe57a0740 1 -- 172.21.15.45:0/307961039 wait complete. 2024-04-16T17:11:55.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.705+0000 7f5fe57a0740 20 client.16896 trim_cache size 0 max 16384 2024-04-16T17:11:55.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.733+0000 7f5fe57a0740 1 Processor -- start 2024-04-16T17:11:55.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.733+0000 7f5fe57a0740 1 -- start start 2024-04-16T17:11:55.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.733+0000 7f5fe57a0740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564e934fdf10 0x564e935130f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:55.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.733+0000 7f5fe57a0740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564e934fe830 0x564e93504d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:55.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.733+0000 7f5fe57a0740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564e93507410 0x564e934fa0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:55.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.733+0000 7f5fe57a0740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f5fdc1b8190 con 0x564e934fdf10 2024-04-16T17:11:55.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.733+0000 7f5fe57a0740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f5fdc1b5a00 con 0x564e93507410 2024-04-16T17:11:55.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.733+0000 7f5fe57a0740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f5fdc1b3830 con 0x564e934fe830 2024-04-16T17:11:55.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.733+0000 7f5fdb400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564e934fe830 0x564e93504d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:55.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.734+0000 7f5fdb400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564e934fe830 0x564e93504d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:44438/0 (socket says 172.21.15.45:44438) 2024-04-16T17:11:55.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.734+0000 7f5fdb400640 1 -- 172.21.15.45:0/2823458074 learned_addr learned my addr 172.21.15.45:0/2823458074 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:11:55.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.734+0000 7f5fe0a00640 1 --2- 172.21.15.45:0/2823458074 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564e934fdf10 0x564e935130f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:55.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.734+0000 7f5fe0a00640 1 --2- 172.21.15.45:0/2823458074 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564e93507410 0x564e934fa0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:55.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.734+0000 7f5fdb400640 1 -- 172.21.15.45:0/2823458074 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564e93507410 msgr2=0x564e934fa0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:11:55.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.734+0000 7f5fdb400640 1 --2- 172.21.15.45:0/2823458074 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564e93507410 0x564e934fa0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:55.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.734+0000 7f5fdb400640 1 -- 172.21.15.45:0/2823458074 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564e934fdf10 msgr2=0x564e935130f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:11:55.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.734+0000 7f5fdb400640 1 --2- 172.21.15.45:0/2823458074 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564e934fdf10 0x564e935130f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:55.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.734+0000 7f5fdb400640 1 -- 172.21.15.45:0/2823458074 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564e931753f0 con 0x564e934fe830 2024-04-16T17:11:55.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.734+0000 7f5fe0a00640 1 --2- 172.21.15.45:0/2823458074 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564e93507410 0x564e934fa0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:11:55.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.734+0000 7f5fdb400640 1 --2- 172.21.15.45:0/2823458074 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564e934fe830 0x564e93504d10 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f5fc8000c00 tx=0x7f5fc8009df0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=554836769abfbfd4 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:11:55.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.735+0000 7f5fc7e00640 1 -- 172.21.15.45:0/2823458074 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fc802d020 con 0x564e934fe830 2024-04-16T17:11:55.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.735+0000 7f5fc7e00640 1 -- 172.21.15.45:0/2823458074 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5fc8011d60 con 0x564e934fe830 2024-04-16T17:11:55.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.735+0000 7f5fc7e00640 1 -- 172.21.15.45:0/2823458074 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fc8003cd0 con 0x564e934fe830 2024-04-16T17:11:55.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.735+0000 7f5fe57a0740 1 -- 172.21.15.45:0/2823458074 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564e934fe830 msgr2=0x564e93504d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:55.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.735+0000 7f5fe57a0740 1 --2- 172.21.15.45:0/2823458074 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564e934fe830 0x564e93504d10 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f5fc8000c00 tx=0x7f5fc8009df0 comp rx=0 tx=0).stop 2024-04-16T17:11:55.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.735+0000 7f5fe57a0740 1 -- 172.21.15.45:0/2823458074 shutdown_connections 2024-04-16T17:11:55.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.735+0000 7f5fe57a0740 1 --2- 172.21.15.45:0/2823458074 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564e934fe830 0x564e93504d10 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:55.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.735+0000 7f5fe57a0740 1 --2- 172.21.15.45:0/2823458074 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564e93507410 0x564e934fa0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:55.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.735+0000 7f5fe57a0740 1 --2- 172.21.15.45:0/2823458074 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564e934fdf10 0x564e935130f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:55.738 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.735+0000 7f5fe57a0740 1 -- 172.21.15.45:0/2823458074 >> 172.21.15.45:0/2823458074 conn(0x564e934f8640 msgr2=0x564e934f99e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:11:55.738 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.736+0000 7f5fe57a0740 1 -- 172.21.15.45:0/2823458074 shutdown_connections 2024-04-16T17:11:55.738 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.736+0000 7f5fe57a0740 1 -- 172.21.15.45:0/2823458074 wait complete. 2024-04-16T17:11:55.738 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.736+0000 7f5fe57a0740 1 -- start start 2024-04-16T17:11:55.738 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.736+0000 7f5fe57a0740 1 Processor -- start 2024-04-16T17:11:55.738 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.737+0000 7f5fe57a0740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564e934fdf10 0x564e9350f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:55.738 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.737+0000 7f5fe57a0740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564e934fe830 0x564e9350ad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:55.738 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.737+0000 7f5fe57a0740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564e93507410 0x564e9350b270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:55.738 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.737+0000 7f5fe57a0740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f5fdc1b3ec0 con 0x564e93507410 2024-04-16T17:11:55.738 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.737+0000 7f5fe57a0740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f5fdc1b5330 con 0x564e934fe830 2024-04-16T17:11:55.738 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.737+0000 7f5fe57a0740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f5fdc1b4590 con 0x564e934fdf10 2024-04-16T17:11:55.738 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.737+0000 7f5fe57a0740 10 client.0 _ll_register_callbacks cb 0x7f5fc8009490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:11:55.738 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.737+0000 7f5fdb400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564e934fe830 0x564e9350ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:55.739 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.737+0000 7f5fe0a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564e934fdf10 0x564e9350f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:55.739 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.737+0000 7f5fdb400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564e934fe830 0x564e9350ad30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:60966/0 (socket says 172.21.15.45:60966) 2024-04-16T17:11:55.739 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.737+0000 7f5fdb400640 1 -- 172.21.15.45:0/328317497 learned_addr learned my addr 172.21.15.45:0/328317497 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:11:55.739 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.737+0000 7f5fe0a00640 1 --2- 172.21.15.45:0/328317497 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564e93507410 0x564e9350b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:55.739 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.737+0000 7f5fdb400640 1 -- 172.21.15.45:0/328317497 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564e934fdf10 msgr2=0x564e9350f9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:11:55.739 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.737+0000 7f5fdb400640 1 --2- 172.21.15.45:0/328317497 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564e934fdf10 0x564e9350f9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:55.739 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.737+0000 7f5fdb400640 1 -- 172.21.15.45:0/328317497 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564e93507410 msgr2=0x564e9350b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:11:55.739 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.737+0000 7f5fdb400640 1 --2- 172.21.15.45:0/328317497 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564e93507410 0x564e9350b270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:55.739 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.737+0000 7f5fdb400640 1 -- 172.21.15.45:0/328317497 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564e9323f7d0 con 0x564e934fe830 2024-04-16T17:11:55.739 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.737+0000 7f5fe0a00640 1 --2- 172.21.15.45:0/328317497 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564e93507410 0x564e9350b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:11:55.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.738+0000 7f5fdb400640 1 --2- 172.21.15.45:0/328317497 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564e934fe830 0x564e9350ad30 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f5fc801b5c0 tx=0x7f5fc8009790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=264c88343bd2a59d server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:11:55.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.738+0000 7f5fc5600640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-04-16T17:11:55.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.738+0000 7f5fc5600640 1 -- 172.21.15.45:0/328317497 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fc802d030 con 0x564e934fe830 2024-04-16T17:11:55.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.738+0000 7f5fe57a0740 1 -- 172.21.15.45:0/328317497 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5fc8011d60 con 0x564e934fe830 2024-04-16T17:11:55.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.738+0000 7f5fc5600640 1 -- 172.21.15.45:0/328317497 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f5fc80054f0 con 0x564e934fe830 2024-04-16T17:11:55.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.738+0000 7f5fc5600640 1 -- 172.21.15.45:0/328317497 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f5fc80117b0 con 0x564e934fe830 2024-04-16T17:11:55.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.738+0000 7f5fe57a0740 10 client.27050 fetch_fsmap learned FSMap version 35 2024-04-16T17:11:55.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.738+0000 7f5fe57a0740 1 -- 172.21.15.45:0/328317497 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x564e9357dc70 con 0x564e934fe830 2024-04-16T17:11:55.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.739+0000 7f5fc5600640 1 -- 172.21.15.45:0/328317497 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5fc8011d60 con 0x564e934fe830 2024-04-16T17:11:55.743 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.739+0000 7f5fc5600640 1 -- 172.21.15.45:0/328317497 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 35) ==== 8358+0+0 (secure 0 0 0) 0x7f5fc801a070 con 0x564e934fe830 2024-04-16T17:11:55.743 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.739+0000 7f5fe57a0740 10 client.27050 fetch_fsmap finished waiting for FSMap version 35 2024-04-16T17:11:55.743 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.739+0000 7f5fe57a0740 10 client.27050 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:11:55.743 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.739+0000 7f5fe57a0740 20 client.27050 populate_metadata read hostname 'smithi045' 2024-04-16T17:11:55.743 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.739+0000 7f5fe57a0740 1 --2- 172.21.15.45:0/328317497 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x564e9357e2f0 0x564e93580710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:55.743 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.739+0000 7f5fe57a0740 4 client.27050 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T17:11:55.743 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.739+0000 7f5fe57a0740 1 -- 172.21.15.45:0/328317497 --> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564e9346a8b0 con 0x564e9357e2f0 2024-04-16T17:11:55.743 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.740+0000 7f5fe0a00640 1 --2- 172.21.15.45:0/328317497 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x564e9357e2f0 0x564e93580710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:55.743 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.740+0000 7f5fe0a00640 1 --2- 172.21.15.45:0/328317497 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x564e9357e2f0 0x564e93580710 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fc4e716d12bba718 server_cookie=129fd4482bd48991 in_seq=0 out_seq=0 2024-04-16T17:11:55.743 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.740+0000 7f5fc5600640 10 client.27050 ms_handle_connect on v2:172.21.15.104:6838/3660070592 2024-04-16T17:11:55.743 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.741+0000 7f5fc5600640 1 -- 172.21.15.45:0/328317497 <== mds.0 v2:172.21.15.104:6838/3660070592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564e9346a8b0 con 0x564e9357e2f0 2024-04-16T17:11:55.743 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.741+0000 7f5fc5600640 10 client.27050 handle_command_reply: tid=0 2024-04-16T17:11:55.743 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.741+0000 7f5fe57a0740 1 client.27050 shutdown 2024-04-16T17:11:55.743 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.741+0000 7f5fe57a0740 1 -- 172.21.15.45:0/328317497 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564e934fe830 msgr2=0x564e9350ad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:55.744 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.741+0000 7f5fe57a0740 1 --2- 172.21.15.45:0/328317497 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564e934fe830 0x564e9350ad30 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f5fc801b5c0 tx=0x7f5fc8009790 comp rx=0 tx=0).stop 2024-04-16T17:11:55.744 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.741+0000 7f5fe57a0740 1 -- 172.21.15.45:0/328317497 shutdown_connections 2024-04-16T17:11:55.744 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.741+0000 7f5fe57a0740 1 --2- 172.21.15.45:0/328317497 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564e93507410 0x564e9350b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:55.744 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.741+0000 7f5fe57a0740 1 --2- 172.21.15.45:0/328317497 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564e934fe830 0x564e9350ad30 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:55.744 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.741+0000 7f5fe57a0740 1 --2- 172.21.15.45:0/328317497 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x564e9357e2f0 0x564e93580710 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:55.744 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.741+0000 7f5fe57a0740 1 --2- 172.21.15.45:0/328317497 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564e934fdf10 0x564e9350f9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:55.744 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.741+0000 7f5fe57a0740 1 -- 172.21.15.45:0/328317497 >> 172.21.15.45:0/328317497 conn(0x564e934f8640 msgr2=0x564e934f7700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:11:55.744 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.741+0000 7f5fc5600640 0 client.27050 ms_handle_reset on v2:172.21.15.104:6838/3660070592 2024-04-16T17:11:55.744 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.741+0000 7f5fe57a0740 1 -- 172.21.15.45:0/328317497 shutdown_connections 2024-04-16T17:11:55.744 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.741+0000 7f5fe57a0740 1 -- 172.21.15.45:0/328317497 wait complete. 2024-04-16T17:11:55.744 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.741+0000 7f5fe57a0740 20 client.27050 trim_cache size 0 max 16384 2024-04-16T17:11:55.745 INFO:teuthology.orchestra.run.smithi045.stdout:[] 2024-04-16T17:11:55.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.742+0000 7f5fe1400640 1 -- 172.21.15.45:0/2673639475 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f5fc00a3d20 msgr2=0x7f5fc00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:55.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.742+0000 7f5fe1400640 1 --2- 172.21.15.45:0/2673639475 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f5fc00a3d20 0x7f5fc00a61e0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f5fcc02c8b0 tx=0x7f5fcc0ba040 comp rx=0 tx=0).stop 2024-04-16T17:11:55.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.742+0000 7f5fe1400640 1 -- 172.21.15.45:0/2673639475 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5fdc0743d0 msgr2=0x7f5fdc10e0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:55.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.742+0000 7f5fe1400640 1 --2- 172.21.15.45:0/2673639475 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5fdc0743d0 0x7f5fdc10e0b0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f5fcc000c00 tx=0x7f5fcc004360 comp rx=0 tx=0).stop 2024-04-16T17:11:55.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.742+0000 7f5fe1400640 1 -- 172.21.15.45:0/2673639475 shutdown_connections 2024-04-16T17:11:55.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.742+0000 7f5fe1400640 1 --2- 172.21.15.45:0/2673639475 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f5fc00a3d20 0x7f5fc00a61e0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:55.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.742+0000 7f5fe1400640 1 --2- 172.21.15.45:0/2673639475 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5fdc10e740 0x7f5fdc10b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:55.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.742+0000 7f5fe1400640 1 --2- 172.21.15.45:0/2673639475 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5fdc074cf0 0x7f5fdc10b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:55.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.742+0000 7f5fe1400640 1 --2- 172.21.15.45:0/2673639475 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5fdc0743d0 0x7f5fdc10e0b0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:55.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.742+0000 7f5fe1400640 1 -- 172.21.15.45:0/2673639475 >> 172.21.15.45:0/2673639475 conn(0x7f5fdc0fd9d0 msgr2=0x7f5fdc0fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:11:55.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.743+0000 7f5fe1400640 1 -- 172.21.15.45:0/2673639475 shutdown_connections 2024-04-16T17:11:55.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:55.743+0000 7f5fe1400640 1 -- 172.21.15.45:0/2673639475 wait complete. 2024-04-16T17:11:55.804 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:11:55 smithi117 ceph-mon[36730]: from='client.? 172.21.15.45:0/1561395277' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:11:55.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:11:55 smithi104 ceph-mon[36457]: from='client.? 172.21.15.45:0/1561395277' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:11:56.263 INFO:tasks.workunit.client.0.smithi045.stdout: 1 79605 1.68 MB/sec execute 118 sec latency 49.725 ms 2024-04-16T17:11:56.759 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-16T17:11:56.759 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T17:11:56.759 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-16T17:11:56.804 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:11:56 smithi117 ceph-mon[36730]: pgmap v158: 129 pgs: 129 active+clean; 45 MiB data, 640 MiB used, 1.0 TiB / 1.0 TiB avail; 481 KiB/s rd, 898 KiB/s wr, 81 op/s 2024-04-16T17:11:56.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:11:56 smithi104 ceph-mon[36457]: pgmap v158: 129 pgs: 129 active+clean; 45 MiB data, 640 MiB used, 1.0 TiB / 1.0 TiB avail; 481 KiB/s rd, 898 KiB/s wr, 81 op/s 2024-04-16T17:11:56.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:56.978+0000 7f4fa1a00640 1 -- 172.21.15.45:0/595159223 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4f9c0743d0 msgr2=0x7f4f9c0747b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:56.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:56.978+0000 7f4fa1a00640 1 --2- 172.21.15.45:0/595159223 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4f9c0743d0 0x7f4f9c0747b0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f4f8c009900 tx=0x7f4f8c02bcd0 comp rx=0 tx=0).stop 2024-04-16T17:11:56.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:56.980+0000 7f4fa1a00640 1 -- 172.21.15.45:0/595159223 shutdown_connections 2024-04-16T17:11:56.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:56.980+0000 7f4fa1a00640 1 --2- 172.21.15.45:0/595159223 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4f9c10e740 0x7f4f9c110b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:56.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:56.980+0000 7f4fa1a00640 1 --2- 172.21.15.45:0/595159223 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4f9c074cf0 0x7f4f9c10e200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:56.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:56.980+0000 7f4fa1a00640 1 --2- 172.21.15.45:0/595159223 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4f9c0743d0 0x7f4f9c0747b0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:56.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:56.980+0000 7f4fa1a00640 1 -- 172.21.15.45:0/595159223 >> 172.21.15.45:0/595159223 conn(0x7f4f9c0fd9d0 msgr2=0x7f4f9c0ffdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:11:56.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:56.980+0000 7f4fa1a00640 1 -- 172.21.15.45:0/595159223 shutdown_connections 2024-04-16T17:11:56.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:56.980+0000 7f4fa1a00640 1 -- 172.21.15.45:0/595159223 wait complete. 2024-04-16T17:11:56.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:56.981+0000 7f4fa1a00640 1 Processor -- start 2024-04-16T17:11:56.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:56.982+0000 7f4fa1a00640 1 -- start start 2024-04-16T17:11:56.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:56.982+0000 7f4fa1a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4f9c0743d0 0x7f4f9c10d890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:56.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:56.982+0000 7f4fa1a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4f9c074cf0 0x7f4f9c10ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:56.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:56.982+0000 7f4fa1a00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4f9c10e740 0x7f4f9c1a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:56.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:56.982+0000 7f4fa1a00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f4f9c103b50 con 0x7f4f9c0743d0 2024-04-16T17:11:56.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:56.982+0000 7f4fa1a00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f4f9c1039d0 con 0x7f4f9c10e740 2024-04-16T17:11:56.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:56.982+0000 7f4fa1a00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f4f9c103cd0 con 0x7f4f9c074cf0 2024-04-16T17:11:56.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:56.982+0000 7f4f9be00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4f9c0743d0 0x7f4f9c10d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:56.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:56.982+0000 7f4fa1000640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4f9c10e740 0x7f4f9c1a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:56.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:56.982+0000 7f4fa1000640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4f9c10e740 0x7f4f9c1a3680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:60978/0 (socket says 172.21.15.45:60978) 2024-04-16T17:11:56.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:56.982+0000 7f4f9be00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4f9c0743d0 0x7f4f9c10d890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39534/0 (socket says 172.21.15.45:39534) 2024-04-16T17:11:56.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:56.982+0000 7f4fa1000640 1 -- 172.21.15.45:0/2545723039 learned_addr learned my addr 172.21.15.45:0/2545723039 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:11:56.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:56.982+0000 7f4f9b400640 1 --2- 172.21.15.45:0/2545723039 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4f9c074cf0 0x7f4f9c10ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:56.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:56.983+0000 7f4fa1000640 1 -- 172.21.15.45:0/2545723039 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4f9c074cf0 msgr2=0x7f4f9c10ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:56.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:56.983+0000 7f4fa1000640 1 --2- 172.21.15.45:0/2545723039 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4f9c074cf0 0x7f4f9c10ddd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:56.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:56.983+0000 7f4fa1000640 1 -- 172.21.15.45:0/2545723039 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4f9c0743d0 msgr2=0x7f4f9c10d890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:56.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:56.983+0000 7f4fa1000640 1 --2- 172.21.15.45:0/2545723039 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4f9c0743d0 0x7f4f9c10d890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:56.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:56.983+0000 7f4fa1000640 1 -- 172.21.15.45:0/2545723039 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f9c1a3db0 con 0x7f4f9c10e740 2024-04-16T17:11:56.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:56.983+0000 7f4f9b400640 1 --2- 172.21.15.45:0/2545723039 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4f9c074cf0 0x7f4f9c10ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:11:56.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:56.983+0000 7f4fa1000640 1 --2- 172.21.15.45:0/2545723039 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4f9c10e740 0x7f4f9c1a3680 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f4f94009fd0 tx=0x7f4f9400ee60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:11:56.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:56.984+0000 7f4f98c00640 1 -- 172.21.15.45:0/2545723039 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f94004d20 con 0x7f4f9c10e740 2024-04-16T17:11:56.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:56.984+0000 7f4f98c00640 1 -- 172.21.15.45:0/2545723039 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4f94005020 con 0x7f4f9c10e740 2024-04-16T17:11:56.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:56.984+0000 7f4fa1a00640 1 -- 172.21.15.45:0/2545723039 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f9c1a40a0 con 0x7f4f9c10e740 2024-04-16T17:11:56.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:56.984+0000 7f4f98c00640 1 -- 172.21.15.45:0/2545723039 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f940116d0 con 0x7f4f9c10e740 2024-04-16T17:11:56.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:56.984+0000 7f4fa1a00640 1 -- 172.21.15.45:0/2545723039 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f9c1ab930 con 0x7f4f9c10e740 2024-04-16T17:11:56.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:56.987+0000 7f4fa5e19740 1 Processor -- start 2024-04-16T17:11:56.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:56.989+0000 7f4f98c00640 1 -- 172.21.15.45:0/2545723039 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f4f940051d0 con 0x7f4f9c10e740 2024-04-16T17:11:56.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:56.989+0000 7f4fa5e19740 1 -- start start 2024-04-16T17:11:56.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:56.989+0000 7f4fa5e19740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559b582c8e90 0x559b582c9270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:56.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:56.989+0000 7f4fa5e19740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559b582c97b0 0x559b582d1e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:56.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:56.989+0000 7f4fa5e19740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559b582d2390 0x559b582d4780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:56.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:56.989+0000 7f4fa5e19740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x559b5822dcb0 con 0x559b582c8e90 2024-04-16T17:11:56.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:56.989+0000 7f4fa5e19740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x559b581d4370 con 0x559b582d2390 2024-04-16T17:11:56.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:56.989+0000 7f4fa5e19740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x559b58229c70 con 0x559b582c97b0 2024-04-16T17:11:56.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:56.990+0000 7f4f9b400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559b582c8e90 0x559b582c9270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:56.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:56.990+0000 7f4f9b400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559b582c8e90 0x559b582c9270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39542/0 (socket says 172.21.15.45:39542) 2024-04-16T17:11:56.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:56.990+0000 7f4f9b400640 1 -- 172.21.15.45:0/3563493478 learned_addr learned my addr 172.21.15.45:0/3563493478 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:11:56.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:56.990+0000 7f4f9be00640 1 --2- 172.21.15.45:0/3563493478 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559b582c97b0 0x559b582d1e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:56.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:56.990+0000 7f4f9b400640 1 --2- 172.21.15.45:0/3563493478 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559b582d2390 0x559b582d4780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:56.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:56.990+0000 7f4f9b400640 1 -- 172.21.15.45:0/3563493478 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559b582c97b0 msgr2=0x559b582d1e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:11:56.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:56.990+0000 7f4f9b400640 1 --2- 172.21.15.45:0/3563493478 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559b582c97b0 0x559b582d1e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:56.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:56.990+0000 7f4f9b400640 1 -- 172.21.15.45:0/3563493478 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559b582d2390 msgr2=0x559b582d4780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:11:56.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:56.990+0000 7f4f9b400640 1 --2- 172.21.15.45:0/3563493478 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559b582d2390 0x559b582d4780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:56.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:56.990+0000 7f4f98c00640 1 --2- 172.21.15.45:0/2545723039 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f4f800a3da0 0x7f4f800a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:56.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:56.990+0000 7f4f9b400640 1 -- 172.21.15.45:0/3563493478 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559b581f3910 con 0x559b582c8e90 2024-04-16T17:11:56.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:56.991+0000 7f4f98c00640 1 -- 172.21.15.45:0/2545723039 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f4f940d0f80 con 0x7f4f9c10e740 2024-04-16T17:11:56.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:56.991+0000 7f4f9b400640 1 --2- 172.21.15.45:0/3563493478 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559b582c8e90 0x559b582c9270 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f4f880139e0 tx=0x7f4f88009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45125da75d3cd4b8 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:11:56.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:56.991+0000 7f4fa1000640 1 --2- 172.21.15.45:0/2545723039 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f4f800a3da0 0x7f4f800a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:56.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:56.992+0000 7f4f91600640 1 -- 172.21.15.45:0/3563493478 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f88004030 con 0x559b582c8e90 2024-04-16T17:11:56.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:56.992+0000 7f4f91600640 1 -- 172.21.15.45:0/3563493478 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4f8800fe60 con 0x559b582c8e90 2024-04-16T17:11:56.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:56.992+0000 7f4f91600640 1 -- 172.21.15.45:0/3563493478 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f88003af0 con 0x559b582c8e90 2024-04-16T17:11:56.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:56.993+0000 7f4fa5e19740 1 -- 172.21.15.45:0/3563493478 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559b582c8e90 msgr2=0x559b582c9270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:56.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:56.993+0000 7f4fa5e19740 1 --2- 172.21.15.45:0/3563493478 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559b582c8e90 0x559b582c9270 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f4f880139e0 tx=0x7f4f88009c00 comp rx=0 tx=0).stop 2024-04-16T17:11:56.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:56.993+0000 7f4fa1000640 1 --2- 172.21.15.45:0/2545723039 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f4f800a3da0 0x7f4f800a6260 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f4f94099080 tx=0x7f4f94005840 comp rx=0 tx=0).ready entity=mgr.25411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:11:56.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:56.993+0000 7f4fa5e19740 1 -- 172.21.15.45:0/3563493478 shutdown_connections 2024-04-16T17:11:56.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:56.993+0000 7f4fa5e19740 1 --2- 172.21.15.45:0/3563493478 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559b582c97b0 0x559b582d1e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:56.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:56.993+0000 7f4fa5e19740 1 --2- 172.21.15.45:0/3563493478 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559b582d2390 0x559b582d4780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:56.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:56.993+0000 7f4fa5e19740 1 --2- 172.21.15.45:0/3563493478 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559b582c8e90 0x559b582c9270 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:56.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:56.993+0000 7f4fa5e19740 1 -- 172.21.15.45:0/3563493478 >> 172.21.15.45:0/3563493478 conn(0x559b582c35c0 msgr2=0x559b582c39a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:11:56.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:56.994+0000 7f4fa5e19740 1 -- 172.21.15.45:0/3563493478 shutdown_connections 2024-04-16T17:11:56.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:56.994+0000 7f4fa5e19740 1 -- 172.21.15.45:0/3563493478 wait complete. 2024-04-16T17:11:56.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:56.995+0000 7f4fa5e19740 1 -- start start 2024-04-16T17:11:56.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:56.995+0000 7f4fa5e19740 1 Processor -- start 2024-04-16T17:11:56.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:56.996+0000 7f4fa5e19740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559b582c8e90 0x559b582cf420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:56.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:56.996+0000 7f4fa5e19740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559b582c97b0 0x559b582cf960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:56.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:56.996+0000 7f4fa5e19740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559b582d2390 0x559b582cfea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:56.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:56.996+0000 7f4fa5e19740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x559b582d5180 con 0x559b582c8e90 2024-04-16T17:11:56.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:56.996+0000 7f4fa5e19740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x559b581e7730 con 0x559b582c97b0 2024-04-16T17:11:56.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:56.996+0000 7f4fa5e19740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x559b582d5300 con 0x559b582d2390 2024-04-16T17:11:56.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:56.996+0000 7f4fa5e19740 10 client.0 _ll_register_callbacks cb 0x559b581b61e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:11:56.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:56.996+0000 7f4f9b400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559b582c8e90 0x559b582cf420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:56.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:56.996+0000 7f4f9be00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559b582c97b0 0x559b582cf960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:56.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:56.996+0000 7f4f9be00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559b582c97b0 0x559b582cf960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:60988/0 (socket says 172.21.15.45:60988) 2024-04-16T17:11:56.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:56.996+0000 7f4f9be00640 1 -- 172.21.15.45:0/2828823488 learned_addr learned my addr 172.21.15.45:0/2828823488 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:11:56.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:56.996+0000 7f4f9b400640 1 --2- 172.21.15.45:0/2828823488 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559b582c8e90 0x559b582cf420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39546/0 (socket says 172.21.15.45:39546) 2024-04-16T17:11:56.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:56.997+0000 7f4f9b400640 1 --2- 172.21.15.45:0/2828823488 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559b582d2390 0x559b582cfea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:56.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:56.997+0000 7f4f9be00640 1 -- 172.21.15.45:0/2828823488 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559b582d2390 msgr2=0x559b582cfea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:11:56.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:56.997+0000 7f4f9be00640 1 --2- 172.21.15.45:0/2828823488 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559b582d2390 0x559b582cfea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:56.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:56.997+0000 7f4f9be00640 1 -- 172.21.15.45:0/2828823488 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559b582c8e90 msgr2=0x559b582cf420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:11:56.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:56.997+0000 7f4f9be00640 1 --2- 172.21.15.45:0/2828823488 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559b582c8e90 0x559b582cf420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:56.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:56.997+0000 7f4f9be00640 1 -- 172.21.15.45:0/2828823488 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559b580087e0 con 0x559b582c97b0 2024-04-16T17:11:56.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:56.997+0000 7f4f9b400640 1 --2- 172.21.15.45:0/2828823488 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559b582c8e90 0x559b582cf420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:11:56.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.000+0000 7f4f9be00640 1 --2- 172.21.15.45:0/2828823488 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559b582c97b0 0x559b582cf960 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f4f8c006010 tx=0x7f4f8c004610 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9234ac13b74a5b4c server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:11:57.000 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.000+0000 7f4f86000640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-04-16T17:11:57.000 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.000+0000 7f4f86000640 1 -- 172.21.15.45:0/2828823488 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f8c044070 con 0x559b582c97b0 2024-04-16T17:11:57.000 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.000+0000 7f4f86000640 1 -- 172.21.15.45:0/2828823488 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f4f8c0046c0 con 0x559b582c97b0 2024-04-16T17:11:57.000 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.000+0000 7f4fa5e19740 1 -- 172.21.15.45:0/2828823488 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4f8800fe60 con 0x559b582c97b0 2024-04-16T17:11:57.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.000+0000 7f4f86000640 1 -- 172.21.15.45:0/2828823488 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f4f8c0048c0 con 0x559b582c97b0 2024-04-16T17:11:57.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.000+0000 7f4fa5e19740 10 client.27062 fetch_fsmap learned FSMap version 35 2024-04-16T17:11:57.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.000+0000 7f4fa5e19740 1 -- 172.21.15.45:0/2828823488 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x559b58068f70 con 0x559b582c97b0 2024-04-16T17:11:57.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.001+0000 7f4f86000640 1 -- 172.21.15.45:0/2828823488 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4f8800fe60 con 0x559b582c97b0 2024-04-16T17:11:57.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.001+0000 7f4f86000640 1 -- 172.21.15.45:0/2828823488 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 35) ==== 8358+0+0 (secure 0 0 0) 0x7f4f8c003790 con 0x559b582c97b0 2024-04-16T17:11:57.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.001+0000 7f4fa5e19740 10 client.27062 fetch_fsmap finished waiting for FSMap version 35 2024-04-16T17:11:57.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.001+0000 7f4fa5e19740 10 client.27062 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:11:57.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.001+0000 7f4fa5e19740 20 client.27062 populate_metadata read hostname 'smithi045' 2024-04-16T17:11:57.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.001+0000 7f4fa5e19740 1 --2- 172.21.15.45:0/2828823488 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x559b582e2770 0x559b582e4b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:57.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.001+0000 7f4fa5e19740 4 client.27062 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:11:57.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.001+0000 7f4fa5e19740 1 -- 172.21.15.45:0/2828823488 --> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559b5823b6d0 con 0x559b582e2770 2024-04-16T17:11:57.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.002+0000 7f4f9b400640 1 --2- 172.21.15.45:0/2828823488 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x559b582e2770 0x559b582e4b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:57.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.002+0000 7f4f9b400640 1 --2- 172.21.15.45:0/2828823488 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x559b582e2770 0x559b582e4b80 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6b7ed40479294d4f server_cookie=63b644f717fc02ec in_seq=0 out_seq=0 2024-04-16T17:11:57.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.002+0000 7f4f86000640 10 client.27062 ms_handle_connect on v2:172.21.15.104:6838/3660070592 2024-04-16T17:11:57.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.003+0000 7f4f86000640 1 -- 172.21.15.45:0/2828823488 <== mds.0 v2:172.21.15.104:6838/3660070592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559b5823b6d0 con 0x559b582e2770 2024-04-16T17:11:57.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.003+0000 7f4f86000640 10 client.27062 handle_command_reply: tid=0 2024-04-16T17:11:57.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.003+0000 7f4fa5e19740 1 client.27062 shutdown 2024-04-16T17:11:57.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.003+0000 7f4fa5e19740 1 -- 172.21.15.45:0/2828823488 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559b582c97b0 msgr2=0x559b582cf960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:57.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.003+0000 7f4fa5e19740 1 --2- 172.21.15.45:0/2828823488 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559b582c97b0 0x559b582cf960 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f4f8c006010 tx=0x7f4f8c004610 comp rx=0 tx=0).stop 2024-04-16T17:11:57.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.004+0000 7f4fa5e19740 1 -- 172.21.15.45:0/2828823488 shutdown_connections 2024-04-16T17:11:57.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.004+0000 7f4fa5e19740 1 --2- 172.21.15.45:0/2828823488 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x559b582e2770 0x559b582e4b80 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:57.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.004+0000 7f4fa5e19740 1 --2- 172.21.15.45:0/2828823488 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559b582d2390 0x559b582cfea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:57.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.004+0000 7f4fa5e19740 1 --2- 172.21.15.45:0/2828823488 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559b582c97b0 0x559b582cf960 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:57.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.004+0000 7f4fa5e19740 1 --2- 172.21.15.45:0/2828823488 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559b582c8e90 0x559b582cf420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:57.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.004+0000 7f4f86000640 0 client.27062 ms_handle_reset on v2:172.21.15.104:6838/3660070592 2024-04-16T17:11:57.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.004+0000 7f4fa5e19740 1 -- 172.21.15.45:0/2828823488 >> 172.21.15.45:0/2828823488 conn(0x559b582c35c0 msgr2=0x559b582c5310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:11:57.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.004+0000 7f4fa5e19740 1 -- 172.21.15.45:0/2828823488 shutdown_connections 2024-04-16T17:11:57.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.004+0000 7f4fa5e19740 1 -- 172.21.15.45:0/2828823488 wait complete. 2024-04-16T17:11:57.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.004+0000 7f4fa5e19740 20 client.27062 trim_cache size 0 max 16384 2024-04-16T17:11:57.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.030+0000 7f4fa5e19740 1 Processor -- start 2024-04-16T17:11:57.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.030+0000 7f4fa5e19740 1 -- start start 2024-04-16T17:11:57.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.030+0000 7f4fa5e19740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559b582c8e90 0x559b582d1b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:57.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.030+0000 7f4fa5e19740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559b582c97b0 0x559b582de040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:57.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.030+0000 7f4fa5e19740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559b582d2390 0x559b582c5080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:57.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.030+0000 7f4fa5e19740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f4f9c1b82e0 con 0x559b582c8e90 2024-04-16T17:11:57.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.030+0000 7f4fa5e19740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f4f9c1b5b50 con 0x559b582c97b0 2024-04-16T17:11:57.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.030+0000 7f4fa5e19740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f4f9c1b3980 con 0x559b582d2390 2024-04-16T17:11:57.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.030+0000 7f4f9b400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559b582c8e90 0x559b582d1b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:57.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.030+0000 7f4f9be00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559b582c97b0 0x559b582de040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:57.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.030+0000 7f4f9b400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559b582c8e90 0x559b582d1b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39558/0 (socket says 172.21.15.45:39558) 2024-04-16T17:11:57.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.030+0000 7f4f9b400640 1 -- 172.21.15.45:0/3804179598 learned_addr learned my addr 172.21.15.45:0/3804179598 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:11:57.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.031+0000 7f4f9b400640 1 --2- 172.21.15.45:0/3804179598 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559b582d2390 0x559b582c5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:57.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.031+0000 7f4f9be00640 1 -- 172.21.15.45:0/3804179598 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559b582d2390 msgr2=0x559b582c5080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:11:57.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.031+0000 7f4f9be00640 1 --2- 172.21.15.45:0/3804179598 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559b582d2390 0x559b582c5080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:57.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.031+0000 7f4f9be00640 1 -- 172.21.15.45:0/3804179598 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559b582c8e90 msgr2=0x559b582d1b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:11:57.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.031+0000 7f4f9be00640 1 --2- 172.21.15.45:0/3804179598 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559b582c8e90 0x559b582d1b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:57.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.031+0000 7f4f9be00640 1 -- 172.21.15.45:0/3804179598 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559b57f00170 con 0x559b582c97b0 2024-04-16T17:11:57.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.031+0000 7f4f9be00640 1 --2- 172.21.15.45:0/3804179598 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559b582c97b0 0x559b582de040 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f4f8c003d20 tx=0x7f4f8c02c350 comp rx=0 tx=0).ready entity=mon.2 client_cookie=40be71742b1c708a server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:11:57.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.031+0000 7f4f90c00640 1 -- 172.21.15.45:0/3804179598 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f8c05a070 con 0x559b582c97b0 2024-04-16T17:11:57.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.032+0000 7f4f90c00640 1 -- 172.21.15.45:0/3804179598 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4f8c003da0 con 0x559b582c97b0 2024-04-16T17:11:57.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.032+0000 7f4f90c00640 1 -- 172.21.15.45:0/3804179598 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f8c044070 con 0x559b582c97b0 2024-04-16T17:11:57.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.032+0000 7f4fa5e19740 1 -- 172.21.15.45:0/3804179598 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559b582c97b0 msgr2=0x559b582de040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:57.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.032+0000 7f4fa5e19740 1 --2- 172.21.15.45:0/3804179598 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559b582c97b0 0x559b582de040 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f4f8c003d20 tx=0x7f4f8c02c350 comp rx=0 tx=0).stop 2024-04-16T17:11:57.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.032+0000 7f4fa5e19740 1 -- 172.21.15.45:0/3804179598 shutdown_connections 2024-04-16T17:11:57.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.032+0000 7f4fa5e19740 1 --2- 172.21.15.45:0/3804179598 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559b582d2390 0x559b582c5080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:57.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.032+0000 7f4fa5e19740 1 --2- 172.21.15.45:0/3804179598 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559b582c97b0 0x559b582de040 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:57.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.032+0000 7f4fa5e19740 1 --2- 172.21.15.45:0/3804179598 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559b582c8e90 0x559b582d1b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:57.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.032+0000 7f4fa5e19740 1 -- 172.21.15.45:0/3804179598 >> 172.21.15.45:0/3804179598 conn(0x559b582c35c0 msgr2=0x559b582c4970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:11:57.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.032+0000 7f4fa5e19740 1 -- 172.21.15.45:0/3804179598 shutdown_connections 2024-04-16T17:11:57.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.033+0000 7f4fa5e19740 1 -- 172.21.15.45:0/3804179598 wait complete. 2024-04-16T17:11:57.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.033+0000 7f4fa5e19740 1 -- start start 2024-04-16T17:11:57.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.033+0000 7f4fa5e19740 1 Processor -- start 2024-04-16T17:11:57.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.034+0000 7f4fa5e19740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559b582c8e90 0x559b582d66d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:57.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.034+0000 7f4fa5e19740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559b582c97b0 0x559b582d6c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:57.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.034+0000 7f4fa5e19740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559b582d2390 0x559b58347d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:57.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.034+0000 7f4fa5e19740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f4f9c1b4010 con 0x559b582c97b0 2024-04-16T17:11:57.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.034+0000 7f4fa5e19740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f4f9c1b5480 con 0x559b582c8e90 2024-04-16T17:11:57.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.034+0000 7f4fa5e19740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f4f9c1b46e0 con 0x559b582d2390 2024-04-16T17:11:57.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.034+0000 7f4fa5e19740 10 client.0 _ll_register_callbacks cb 0x7f4f8c0044d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:11:57.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.034+0000 7f4f9be00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559b582c97b0 0x559b582d6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:57.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.034+0000 7f4f9be00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559b582c97b0 0x559b582d6c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39566/0 (socket says 172.21.15.45:39566) 2024-04-16T17:11:57.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.034+0000 7f4f9be00640 1 -- 172.21.15.45:0/1695909513 learned_addr learned my addr 172.21.15.45:0/1695909513 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:11:57.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.034+0000 7f4f9b400640 1 --2- 172.21.15.45:0/1695909513 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559b582c8e90 0x559b582d66d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:57.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.034+0000 7f4f9b400640 1 --2- 172.21.15.45:0/1695909513 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559b582d2390 0x559b58347d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:57.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.034+0000 7f4f9be00640 1 -- 172.21.15.45:0/1695909513 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559b582d2390 msgr2=0x559b58347d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:11:57.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.034+0000 7f4f9be00640 1 --2- 172.21.15.45:0/1695909513 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559b582d2390 0x559b58347d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:57.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.034+0000 7f4f9be00640 1 -- 172.21.15.45:0/1695909513 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559b582c8e90 msgr2=0x559b582d66d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:11:57.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.034+0000 7f4f9be00640 1 --2- 172.21.15.45:0/1695909513 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559b582c8e90 0x559b582d66d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:57.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.034+0000 7f4f9be00640 1 -- 172.21.15.45:0/1695909513 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559b58348270 con 0x559b582c97b0 2024-04-16T17:11:57.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.034+0000 7f4f9b400640 1 --2- 172.21.15.45:0/1695909513 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559b582c8e90 0x559b582d66d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:11:57.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.035+0000 7f4f9be00640 1 --2- 172.21.15.45:0/1695909513 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559b582c97b0 0x559b582d6c10 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f4f8c02f020 tx=0x7f4f8c004c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d738712e18486d3 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:11:57.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.035+0000 7f4f86000640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-16T17:11:57.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.035+0000 7f4f86000640 1 -- 172.21.15.45:0/1695909513 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f8c044070 con 0x559b582c97b0 2024-04-16T17:11:57.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.035+0000 7f4f86000640 1 -- 172.21.15.45:0/1695909513 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f4f8c0046c0 con 0x559b582c97b0 2024-04-16T17:11:57.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.035+0000 7f4fa5e19740 1 -- 172.21.15.45:0/1695909513 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4f8c003da0 con 0x559b582c97b0 2024-04-16T17:11:57.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.035+0000 7f4f86000640 1 -- 172.21.15.45:0/1695909513 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f4f8c005160 con 0x559b582c97b0 2024-04-16T17:11:57.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.035+0000 7f4fa5e19740 10 client.16920 fetch_fsmap learned FSMap version 35 2024-04-16T17:11:57.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.035+0000 7f4fa5e19740 1 -- 172.21.15.45:0/1695909513 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x559b583495b0 con 0x559b582c97b0 2024-04-16T17:11:57.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.036+0000 7f4f86000640 1 -- 172.21.15.45:0/1695909513 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4f8c003da0 con 0x559b582c97b0 2024-04-16T17:11:57.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.036+0000 7f4f86000640 1 -- 172.21.15.45:0/1695909513 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 35) ==== 8358+0+0 (secure 0 0 0) 0x7f4f8c055b00 con 0x559b582c97b0 2024-04-16T17:11:57.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.036+0000 7f4fa5e19740 10 client.16920 fetch_fsmap finished waiting for FSMap version 35 2024-04-16T17:11:57.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.036+0000 7f4fa5e19740 10 client.16920 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:11:57.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.036+0000 7f4fa5e19740 20 client.16920 populate_metadata read hostname 'smithi045' 2024-04-16T17:11:57.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.036+0000 7f4fa5e19740 1 --2- 172.21.15.45:0/1695909513 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x559b58349cc0 0x559b5834c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:57.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.036+0000 7f4fa5e19740 4 client.16920 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-16T17:11:57.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.036+0000 7f4fa5e19740 1 -- 172.21.15.45:0/1695909513 --> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x559b58235540 con 0x559b58349cc0 2024-04-16T17:11:57.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.036+0000 7f4f9b400640 1 --2- 172.21.15.45:0/1695909513 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x559b58349cc0 0x559b5834c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:57.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.037+0000 7f4f9b400640 1 --2- 172.21.15.45:0/1695909513 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x559b58349cc0 0x559b5834c0e0 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d0c937bad1b233b5 server_cookie=9cbe708d42a75dba in_seq=0 out_seq=0 2024-04-16T17:11:57.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.037+0000 7f4f86000640 10 client.16920 ms_handle_connect on v2:172.21.15.104:6838/3660070592 2024-04-16T17:11:57.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.038+0000 7f4f86000640 1 -- 172.21.15.45:0/1695909513 <== mds.0 v2:172.21.15.104:6838/3660070592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x559b58235540 con 0x559b58349cc0 2024-04-16T17:11:57.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.038+0000 7f4f86000640 10 client.16920 handle_command_reply: tid=0 2024-04-16T17:11:57.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.038+0000 7f4fa5e19740 1 client.16920 shutdown 2024-04-16T17:11:57.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.038+0000 7f4fa5e19740 1 -- 172.21.15.45:0/1695909513 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559b582c97b0 msgr2=0x559b582d6c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:57.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.038+0000 7f4fa5e19740 1 --2- 172.21.15.45:0/1695909513 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559b582c97b0 0x559b582d6c10 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f4f8c02f020 tx=0x7f4f8c004c90 comp rx=0 tx=0).stop 2024-04-16T17:11:57.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.038+0000 7f4fa5e19740 1 -- 172.21.15.45:0/1695909513 shutdown_connections 2024-04-16T17:11:57.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.038+0000 7f4fa5e19740 1 --2- 172.21.15.45:0/1695909513 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x559b58349cc0 0x559b5834c0e0 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:57.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.038+0000 7f4fa5e19740 1 --2- 172.21.15.45:0/1695909513 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559b582d2390 0x559b58347d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:57.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.038+0000 7f4fa5e19740 1 --2- 172.21.15.45:0/1695909513 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559b582c97b0 0x559b582d6c10 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:57.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.038+0000 7f4fa5e19740 1 --2- 172.21.15.45:0/1695909513 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559b582c8e90 0x559b582d66d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:57.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.038+0000 7f4fa5e19740 1 -- 172.21.15.45:0/1695909513 >> 172.21.15.45:0/1695909513 conn(0x559b582c35c0 msgr2=0x559b582cfc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:11:57.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.038+0000 7f4f86000640 0 client.16920 ms_handle_reset on v2:172.21.15.104:6838/3660070592 2024-04-16T17:11:57.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.039+0000 7f4fa5e19740 1 -- 172.21.15.45:0/1695909513 shutdown_connections 2024-04-16T17:11:57.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.039+0000 7f4fa5e19740 1 -- 172.21.15.45:0/1695909513 wait complete. 2024-04-16T17:11:57.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.039+0000 7f4fa5e19740 20 client.16920 trim_cache size 0 max 16384 2024-04-16T17:11:57.041 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-16T17:11:57.041 INFO:teuthology.orchestra.run.smithi045.stdout: "return_code": 0, 2024-04-16T17:11:57.041 INFO:teuthology.orchestra.run.smithi045.stdout: "scrub_tag": "eb2684a1-7e8a-43ed-b085-a779e2454205", 2024-04-16T17:11:57.041 INFO:teuthology.orchestra.run.smithi045.stdout: "mode": "asynchronous" 2024-04-16T17:11:57.041 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-16T17:11:57.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.040+0000 7f4fa1a00640 1 -- 172.21.15.45:0/2545723039 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f4f800a3da0 msgr2=0x7f4f800a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:57.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.040+0000 7f4fa1a00640 1 --2- 172.21.15.45:0/2545723039 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f4f800a3da0 0x7f4f800a6260 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f4f94099080 tx=0x7f4f94005840 comp rx=0 tx=0).stop 2024-04-16T17:11:57.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.040+0000 7f4fa1a00640 1 -- 172.21.15.45:0/2545723039 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4f9c10e740 msgr2=0x7f4f9c1a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:57.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.040+0000 7f4fa1a00640 1 --2- 172.21.15.45:0/2545723039 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4f9c10e740 0x7f4f9c1a3680 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f4f94009fd0 tx=0x7f4f9400ee60 comp rx=0 tx=0).stop 2024-04-16T17:11:57.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.040+0000 7f4fa1a00640 1 -- 172.21.15.45:0/2545723039 shutdown_connections 2024-04-16T17:11:57.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.040+0000 7f4fa1a00640 1 --2- 172.21.15.45:0/2545723039 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f4f800a3da0 0x7f4f800a6260 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:57.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.040+0000 7f4fa1a00640 1 --2- 172.21.15.45:0/2545723039 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4f9c074cf0 0x7f4f9c10ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:57.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.040+0000 7f4fa1a00640 1 --2- 172.21.15.45:0/2545723039 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4f9c10e740 0x7f4f9c1a3680 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:57.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.040+0000 7f4fa1a00640 1 --2- 172.21.15.45:0/2545723039 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4f9c0743d0 0x7f4f9c10d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:57.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.040+0000 7f4fa1a00640 1 -- 172.21.15.45:0/2545723039 >> 172.21.15.45:0/2545723039 conn(0x7f4f9c0fd9d0 msgr2=0x7f4f9c0fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:11:57.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.040+0000 7f4fa1a00640 1 -- 172.21.15.45:0/2545723039 shutdown_connections 2024-04-16T17:11:57.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.040+0000 7f4fa1a00640 1 -- 172.21.15.45:0/2545723039 wait complete. 2024-04-16T17:11:57.064 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-16T17:11:57.064 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-16T17:11:57.263 INFO:tasks.workunit.client.0.smithi045.stdout: 1 80511 1.68 MB/sec execute 119 sec latency 31.888 ms 2024-04-16T17:11:57.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.268+0000 7fef06400640 1 -- 172.21.15.45:0/3743911103 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fef00103980 msgr2=0x7fef0010be60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:57.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.268+0000 7fef06400640 1 --2- 172.21.15.45:0/3743911103 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fef00103980 0x7fef0010be60 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7feeec009900 tx=0x7feeec02bcc0 comp rx=0 tx=0).stop 2024-04-16T17:11:57.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.268+0000 7fef06400640 1 -- 172.21.15.45:0/3743911103 shutdown_connections 2024-04-16T17:11:57.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.268+0000 7fef06400640 1 --2- 172.21.15.45:0/3743911103 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fef0010c480 0x7fef0010e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:57.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.268+0000 7fef06400640 1 --2- 172.21.15.45:0/3743911103 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fef00103980 0x7fef0010be60 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:57.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.268+0000 7fef06400640 1 --2- 172.21.15.45:0/3743911103 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef00103060 0x7fef00103440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:57.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.268+0000 7fef06400640 1 -- 172.21.15.45:0/3743911103 >> 172.21.15.45:0/3743911103 conn(0x7fef000fc880 msgr2=0x7fef000feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:11:57.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.268+0000 7fef06400640 1 -- 172.21.15.45:0/3743911103 shutdown_connections 2024-04-16T17:11:57.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.268+0000 7fef06400640 1 -- 172.21.15.45:0/3743911103 wait complete. 2024-04-16T17:11:57.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.269+0000 7fef06400640 1 Processor -- start 2024-04-16T17:11:57.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.269+0000 7fef06400640 1 -- start start 2024-04-16T17:11:57.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.269+0000 7fef06400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef00103060 0x7fef001a0680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:57.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.269+0000 7fef06400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fef00103980 0x7fef001a0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:57.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.269+0000 7fef06400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fef0010c480 0x7fef0019a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:57.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.269+0000 7fef06400640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fef001122b0 con 0x7fef00103060 2024-04-16T17:11:57.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.269+0000 7fef06400640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fef00112130 con 0x7fef00103980 2024-04-16T17:11:57.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.269+0000 7fef06400640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fef00112430 con 0x7fef0010c480 2024-04-16T17:11:57.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.270+0000 7feeffe00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fef00103980 0x7fef001a0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:57.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.270+0000 7fef05a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fef0010c480 0x7fef0019a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:57.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.270+0000 7feeffe00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fef00103980 0x7fef001a0bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:32802/0 (socket says 172.21.15.45:32802) 2024-04-16T17:11:57.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.270+0000 7feeffe00640 1 -- 172.21.15.45:0/2019104256 learned_addr learned my addr 172.21.15.45:0/2019104256 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:11:57.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.270+0000 7fef05000640 1 --2- 172.21.15.45:0/2019104256 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef00103060 0x7fef001a0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:57.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.270+0000 7feeffe00640 1 -- 172.21.15.45:0/2019104256 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fef0010c480 msgr2=0x7fef0019a7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:57.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.270+0000 7feeffe00640 1 --2- 172.21.15.45:0/2019104256 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fef0010c480 0x7fef0019a7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:57.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.270+0000 7feeffe00640 1 -- 172.21.15.45:0/2019104256 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef00103060 msgr2=0x7fef001a0680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:57.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.270+0000 7feeffe00640 1 --2- 172.21.15.45:0/2019104256 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef00103060 0x7fef001a0680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:57.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.270+0000 7feeffe00640 1 -- 172.21.15.45:0/2019104256 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef0019afc0 con 0x7fef00103980 2024-04-16T17:11:57.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.270+0000 7fef05a00640 1 --2- 172.21.15.45:0/2019104256 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fef0010c480 0x7fef0019a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:11:57.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.270+0000 7feeffe00640 1 --2- 172.21.15.45:0/2019104256 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fef00103980 0x7fef001a0bc0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7feeec000c00 tx=0x7feeec0042f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:11:57.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.271+0000 7feefd600640 1 -- 172.21.15.45:0/2019104256 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeec042020 con 0x7fef00103980 2024-04-16T17:11:57.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.271+0000 7feefd600640 1 -- 172.21.15.45:0/2019104256 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feeec005000 con 0x7fef00103980 2024-04-16T17:11:57.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.271+0000 7feefd600640 1 -- 172.21.15.45:0/2019104256 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeec003a80 con 0x7fef00103980 2024-04-16T17:11:57.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.271+0000 7fef06400640 1 -- 172.21.15.45:0/2019104256 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef0019b250 con 0x7fef00103980 2024-04-16T17:11:57.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.271+0000 7fef06400640 1 -- 172.21.15.45:0/2019104256 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef001a74d0 con 0x7fef00103980 2024-04-16T17:11:57.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.274+0000 7fef0a7db740 1 Processor -- start 2024-04-16T17:11:57.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.274+0000 7fef0a7db740 1 -- start start 2024-04-16T17:11:57.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.274+0000 7fef0a7db740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559b43fac390 0x559b43fac770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:57.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.274+0000 7fef0a7db740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559b43faccb0 0x559b43fb5350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:57.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.274+0000 7fef0a7db740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559b43fb5890 0x559b43fb7c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:57.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.275+0000 7fef0a7db740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x559b43f12120 con 0x559b43fac390 2024-04-16T17:11:57.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.275+0000 7fef0a7db740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x559b43eb8340 con 0x559b43fb5890 2024-04-16T17:11:57.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.275+0000 7fef0a7db740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x559b43f0e0e0 con 0x559b43faccb0 2024-04-16T17:11:57.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.275+0000 7fef05a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559b43fac390 0x559b43fac770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:57.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.275+0000 7fef05a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559b43fac390 0x559b43fac770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39604/0 (socket says 172.21.15.45:39604) 2024-04-16T17:11:57.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.275+0000 7fef05a00640 1 -- 172.21.15.45:0/1913696943 learned_addr learned my addr 172.21.15.45:0/1913696943 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:11:57.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.275+0000 7fef05000640 1 --2- 172.21.15.45:0/1913696943 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559b43faccb0 0x559b43fb5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:57.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.275+0000 7fef05a00640 1 -- 172.21.15.45:0/1913696943 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559b43faccb0 msgr2=0x559b43fb5350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:11:57.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.275+0000 7fef05a00640 1 --2- 172.21.15.45:0/1913696943 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559b43faccb0 0x559b43fb5350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:57.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.275+0000 7fef05a00640 1 -- 172.21.15.45:0/1913696943 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559b43fb5890 msgr2=0x559b43fb7c80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-16T17:11:57.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.275+0000 7fef05a00640 1 --2- 172.21.15.45:0/1913696943 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559b43fb5890 0x559b43fb7c80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:57.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.275+0000 7fef05a00640 1 -- 172.21.15.45:0/1913696943 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559b43cee7d0 con 0x559b43fac390 2024-04-16T17:11:57.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.276+0000 7feefd600640 1 -- 172.21.15.45:0/2019104256 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7feeec045020 con 0x7fef00103980 2024-04-16T17:11:57.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.276+0000 7fef05a00640 1 --2- 172.21.15.45:0/1913696943 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559b43fac390 0x559b43fac770 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7feef8011790 tx=0x7feef8011c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df77e7eaf459c0a4 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:11:57.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.277+0000 7feef2000640 1 -- 172.21.15.45:0/1913696943 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feef8003c30 con 0x559b43fac390 2024-04-16T17:11:57.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.277+0000 7feef2000640 1 -- 172.21.15.45:0/1913696943 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feef800fe80 con 0x559b43fac390 2024-04-16T17:11:57.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.277+0000 7feef2000640 1 -- 172.21.15.45:0/1913696943 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feef80193e0 con 0x559b43fac390 2024-04-16T17:11:57.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.277+0000 7feefd600640 1 --2- 172.21.15.45:0/2019104256 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7feee80a3da0 0x7feee80a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:57.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.277+0000 7feefd600640 1 -- 172.21.15.45:0/2019104256 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7feeec0f2500 con 0x7fef00103980 2024-04-16T17:11:57.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.277+0000 7fef0a7db740 1 -- 172.21.15.45:0/1913696943 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559b43fac390 msgr2=0x559b43fac770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:57.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.277+0000 7fef0a7db740 1 --2- 172.21.15.45:0/1913696943 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559b43fac390 0x559b43fac770 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7feef8011790 tx=0x7feef8011c50 comp rx=0 tx=0).stop 2024-04-16T17:11:57.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.278+0000 7fef05000640 1 --2- 172.21.15.45:0/2019104256 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7feee80a3da0 0x7feee80a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:57.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.278+0000 7fef0a7db740 1 -- 172.21.15.45:0/1913696943 shutdown_connections 2024-04-16T17:11:57.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.278+0000 7fef0a7db740 1 --2- 172.21.15.45:0/1913696943 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559b43faccb0 0x559b43fb5350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:57.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.278+0000 7fef0a7db740 1 --2- 172.21.15.45:0/1913696943 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559b43fb5890 0x559b43fb7c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:57.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.278+0000 7fef0a7db740 1 --2- 172.21.15.45:0/1913696943 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559b43fac390 0x559b43fac770 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:57.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.278+0000 7fef0a7db740 1 -- 172.21.15.45:0/1913696943 >> 172.21.15.45:0/1913696943 conn(0x559b43fa6ac0 msgr2=0x559b43fa6ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:11:57.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.278+0000 7fef0a7db740 1 -- 172.21.15.45:0/1913696943 shutdown_connections 2024-04-16T17:11:57.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.278+0000 7fef0a7db740 1 -- 172.21.15.45:0/1913696943 wait complete. 2024-04-16T17:11:57.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.278+0000 7fef0a7db740 1 -- start start 2024-04-16T17:11:57.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.278+0000 7fef05000640 1 --2- 172.21.15.45:0/2019104256 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7feee80a3da0 0x7feee80a6260 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x559b43fabe60 tx=0x7feef4005f90 comp rx=0 tx=0).ready entity=mgr.25411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:11:57.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.278+0000 7fef0a7db740 1 Processor -- start 2024-04-16T17:11:57.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.279+0000 7fef0a7db740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559b43fac390 0x559b43fb29d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:57.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.279+0000 7fef0a7db740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559b43faccb0 0x559b43fb2f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:57.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.279+0000 7fef0a7db740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559b43fb5890 0x559b43fb3450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:57.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.279+0000 7fef0a7db740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x559b43fb8680 con 0x559b43fb5890 2024-04-16T17:11:57.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.279+0000 7fef0a7db740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x559b43ecb710 con 0x559b43fac390 2024-04-16T17:11:57.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.279+0000 7fef0a7db740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x559b43fb8800 con 0x559b43faccb0 2024-04-16T17:11:57.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.279+0000 7fef0a7db740 10 client.0 _ll_register_callbacks cb 0x559b43ec6af0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:11:57.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.280+0000 7fef05000640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559b43fac390 0x559b43fb29d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:57.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.280+0000 7fef05000640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559b43fac390 0x559b43fb29d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:32816/0 (socket says 172.21.15.45:32816) 2024-04-16T17:11:57.282 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.280+0000 7fef05000640 1 -- 172.21.15.45:0/1797127719 learned_addr learned my addr 172.21.15.45:0/1797127719 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:11:57.282 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.280+0000 7fef05a00640 1 --2- 172.21.15.45:0/1797127719 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559b43fb5890 0x559b43fb3450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:57.282 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.280+0000 7fef05000640 1 -- 172.21.15.45:0/1797127719 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559b43faccb0 msgr2=0x559b43fb2f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:11:57.282 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.280+0000 7fef05000640 1 --2- 172.21.15.45:0/1797127719 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559b43faccb0 0x559b43fb2f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:57.282 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.280+0000 7fef05000640 1 -- 172.21.15.45:0/1797127719 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559b43fb5890 msgr2=0x559b43fb3450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:11:57.282 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.280+0000 7fef05000640 1 --2- 172.21.15.45:0/1797127719 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559b43fb5890 0x559b43fb3450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:57.282 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.280+0000 7fef05000640 1 -- 172.21.15.45:0/1797127719 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559b43c243f0 con 0x559b43fac390 2024-04-16T17:11:57.282 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.280+0000 7fef05000640 1 --2- 172.21.15.45:0/1797127719 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559b43fac390 0x559b43fb29d0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7feef401ea00 tx=0x7feef4004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ddef86b9b93193e8 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:11:57.282 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.280+0000 7feee6a00640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-04-16T17:11:57.282 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.281+0000 7feee6a00640 1 -- 172.21.15.45:0/1797127719 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feef402e060 con 0x559b43fac390 2024-04-16T17:11:57.282 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.281+0000 7feee6a00640 1 -- 172.21.15.45:0/1797127719 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7feef4018030 con 0x559b43fac390 2024-04-16T17:11:57.282 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.281+0000 7fef0a7db740 1 -- 172.21.15.45:0/1797127719 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x559b43f1fb40 con 0x559b43fac390 2024-04-16T17:11:57.282 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.281+0000 7feee6a00640 1 -- 172.21.15.45:0/1797127719 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7feef4029420 con 0x559b43fac390 2024-04-16T17:11:57.282 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.281+0000 7fef0a7db740 10 client.27077 fetch_fsmap learned FSMap version 35 2024-04-16T17:11:57.282 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.281+0000 7fef0a7db740 1 -- 172.21.15.45:0/1797127719 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x559b43be40f0 con 0x559b43fac390 2024-04-16T17:11:57.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.282+0000 7feee6a00640 1 -- 172.21.15.45:0/1797127719 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x559b43f1fb40 con 0x559b43fac390 2024-04-16T17:11:57.284 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.282+0000 7feee6a00640 1 -- 172.21.15.45:0/1797127719 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 35) ==== 8358+0+0 (secure 0 0 0) 0x7feef400f740 con 0x559b43fac390 2024-04-16T17:11:57.284 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.282+0000 7fef0a7db740 10 client.27077 fetch_fsmap finished waiting for FSMap version 35 2024-04-16T17:11:57.284 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.282+0000 7fef0a7db740 10 client.27077 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:11:57.284 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.282+0000 7fef0a7db740 20 client.27077 populate_metadata read hostname 'smithi045' 2024-04-16T17:11:57.284 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.282+0000 7fef0a7db740 1 --2- 172.21.15.45:0/1797127719 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x559b43fc5d30 0x559b43fc8160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:57.284 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.282+0000 7fef0a7db740 4 client.27077 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:11:57.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.282+0000 7fef0a7db740 1 -- 172.21.15.45:0/1797127719 --> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559b43f199b0 con 0x559b43fc5d30 2024-04-16T17:11:57.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.282+0000 7feeffe00640 1 --2- 172.21.15.45:0/1797127719 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x559b43fc5d30 0x559b43fc8160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:57.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.283+0000 7feee6a00640 10 client.27077 ms_handle_connect on v2:172.21.15.104:6838/3660070592 2024-04-16T17:11:57.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.283+0000 7feeffe00640 1 --2- 172.21.15.45:0/1797127719 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x559b43fc5d30 0x559b43fc8160 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a59526cdaffb16f0 server_cookie=df25973e3d4628f3 in_seq=0 out_seq=0 2024-04-16T17:11:57.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.284+0000 7feee6a00640 1 -- 172.21.15.45:0/1797127719 <== mds.0 v2:172.21.15.104:6838/3660070592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559b43f199b0 con 0x559b43fc5d30 2024-04-16T17:11:57.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.284+0000 7feee6a00640 10 client.27077 handle_command_reply: tid=0 2024-04-16T17:11:57.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.284+0000 7fef0a7db740 1 client.27077 shutdown 2024-04-16T17:11:57.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.284+0000 7fef0a7db740 1 -- 172.21.15.45:0/1797127719 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559b43fac390 msgr2=0x559b43fb29d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:57.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.284+0000 7fef0a7db740 1 --2- 172.21.15.45:0/1797127719 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559b43fac390 0x559b43fb29d0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7feef401ea00 tx=0x7feef4004730 comp rx=0 tx=0).stop 2024-04-16T17:11:57.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.284+0000 7fef0a7db740 1 -- 172.21.15.45:0/1797127719 shutdown_connections 2024-04-16T17:11:57.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.284+0000 7fef0a7db740 1 --2- 172.21.15.45:0/1797127719 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x559b43fc5d30 0x559b43fc8160 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:57.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.284+0000 7fef0a7db740 1 --2- 172.21.15.45:0/1797127719 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559b43faccb0 0x559b43fb2f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:57.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.284+0000 7fef0a7db740 1 --2- 172.21.15.45:0/1797127719 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559b43fb5890 0x559b43fb3450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:57.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.284+0000 7fef0a7db740 1 --2- 172.21.15.45:0/1797127719 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559b43fac390 0x559b43fb29d0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:57.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.284+0000 7fef0a7db740 1 -- 172.21.15.45:0/1797127719 >> 172.21.15.45:0/1797127719 conn(0x559b43fa6ac0 msgr2=0x559b43fa8810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:11:57.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.284+0000 7feee6a00640 0 client.27077 ms_handle_reset on v2:172.21.15.104:6838/3660070592 2024-04-16T17:11:57.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.284+0000 7fef0a7db740 1 -- 172.21.15.45:0/1797127719 shutdown_connections 2024-04-16T17:11:57.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.284+0000 7fef0a7db740 1 -- 172.21.15.45:0/1797127719 wait complete. 2024-04-16T17:11:57.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.284+0000 7fef0a7db740 20 client.27077 trim_cache size 0 max 16384 2024-04-16T17:11:57.304 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.304+0000 7fef0a7db740 1 Processor -- start 2024-04-16T17:11:57.304 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.304+0000 7fef0a7db740 1 -- start start 2024-04-16T17:11:57.304 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.305+0000 7fef0a7db740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559b43fac390 0x559b43fc15a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:57.304 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.305+0000 7fef0a7db740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559b43faccb0 0x559b43fb1910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:57.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.305+0000 7fef0a7db740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559b43fb5890 0x559b43fb3270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:57.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.305+0000 7fef0a7db740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fef001b3e70 con 0x559b43fac390 2024-04-16T17:11:57.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.305+0000 7fef0a7db740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fef001b16e0 con 0x559b43fb5890 2024-04-16T17:11:57.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.305+0000 7fef0a7db740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fef001af510 con 0x559b43faccb0 2024-04-16T17:11:57.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.305+0000 7fef05000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559b43fac390 0x559b43fc15a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:57.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.305+0000 7fef05000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559b43fac390 0x559b43fc15a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39616/0 (socket says 172.21.15.45:39616) 2024-04-16T17:11:57.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.305+0000 7fef05000640 1 -- 172.21.15.45:0/3499130254 learned_addr learned my addr 172.21.15.45:0/3499130254 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:11:57.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.305+0000 7feeffe00640 1 --2- 172.21.15.45:0/3499130254 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559b43faccb0 0x559b43fb1910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:57.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.305+0000 7fef05a00640 1 --2- 172.21.15.45:0/3499130254 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559b43fb5890 0x559b43fb3270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:57.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.305+0000 7fef05000640 1 -- 172.21.15.45:0/3499130254 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559b43faccb0 msgr2=0x559b43fb1910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:11:57.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.305+0000 7fef05000640 1 --2- 172.21.15.45:0/3499130254 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559b43faccb0 0x559b43fb1910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:57.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.305+0000 7fef05000640 1 -- 172.21.15.45:0/3499130254 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559b43fb5890 msgr2=0x559b43fb3270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:11:57.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.305+0000 7fef05000640 1 --2- 172.21.15.45:0/3499130254 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559b43fb5890 0x559b43fb3270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:57.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.305+0000 7fef05000640 1 -- 172.21.15.45:0/3499130254 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559b43fb8560 con 0x559b43fac390 2024-04-16T17:11:57.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.305+0000 7feeffe00640 1 --2- 172.21.15.45:0/3499130254 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559b43faccb0 0x559b43fb1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:11:57.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.306+0000 7fef05000640 1 --2- 172.21.15.45:0/3499130254 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559b43fac390 0x559b43fc15a0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7feef4012d50 tx=0x7feef4009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8f2333c263e3403 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:11:57.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.306+0000 7feef1600640 1 -- 172.21.15.45:0/3499130254 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feef4041070 con 0x559b43fac390 2024-04-16T17:11:57.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.306+0000 7feef1600640 1 -- 172.21.15.45:0/3499130254 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feef400f8a0 con 0x559b43fac390 2024-04-16T17:11:57.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.306+0000 7feef1600640 1 -- 172.21.15.45:0/3499130254 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feef4038080 con 0x559b43fac390 2024-04-16T17:11:57.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.307+0000 7fef0a7db740 1 -- 172.21.15.45:0/3499130254 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559b43fac390 msgr2=0x559b43fc15a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:57.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.307+0000 7fef0a7db740 1 --2- 172.21.15.45:0/3499130254 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559b43fac390 0x559b43fc15a0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7feef4012d50 tx=0x7feef4009260 comp rx=0 tx=0).stop 2024-04-16T17:11:57.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.307+0000 7fef0a7db740 1 -- 172.21.15.45:0/3499130254 shutdown_connections 2024-04-16T17:11:57.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.307+0000 7fef0a7db740 1 --2- 172.21.15.45:0/3499130254 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559b43faccb0 0x559b43fb1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:57.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.307+0000 7fef0a7db740 1 --2- 172.21.15.45:0/3499130254 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559b43fb5890 0x559b43fb3270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:57.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.307+0000 7fef0a7db740 1 --2- 172.21.15.45:0/3499130254 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559b43fac390 0x559b43fc15a0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:57.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.307+0000 7fef0a7db740 1 -- 172.21.15.45:0/3499130254 >> 172.21.15.45:0/3499130254 conn(0x559b43fa6ac0 msgr2=0x559b43fa5b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:11:57.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.307+0000 7fef0a7db740 1 -- 172.21.15.45:0/3499130254 shutdown_connections 2024-04-16T17:11:57.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.307+0000 7fef0a7db740 1 -- 172.21.15.45:0/3499130254 wait complete. 2024-04-16T17:11:57.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.308+0000 7fef0a7db740 1 -- start start 2024-04-16T17:11:57.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.308+0000 7fef0a7db740 1 Processor -- start 2024-04-16T17:11:57.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.308+0000 7fef0a7db740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559b43fac390 0x559b43fb9480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:57.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.308+0000 7fef0a7db740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559b43faccb0 0x559b43fb99c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:57.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.308+0000 7fef0a7db740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559b43fb5890 0x559b43fb9f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:57.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.308+0000 7fef0a7db740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fef001afba0 con 0x559b43faccb0 2024-04-16T17:11:57.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.308+0000 7fef0a7db740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fef001b1010 con 0x559b43fac390 2024-04-16T17:11:57.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.308+0000 7fef0a7db740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fef001b0270 con 0x559b43fb5890 2024-04-16T17:11:57.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.308+0000 7fef0a7db740 10 client.0 _ll_register_callbacks cb 0x7feef4009ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:11:57.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.309+0000 7fef05a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559b43fb5890 0x559b43fb9f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:57.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.309+0000 7fef05000640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559b43fac390 0x559b43fb9480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:57.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.309+0000 7feeffe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559b43faccb0 0x559b43fb99c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:57.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.309+0000 7fef05a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559b43fb5890 0x559b43fb9f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:44548/0 (socket says 172.21.15.45:44548) 2024-04-16T17:11:57.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.309+0000 7fef05a00640 1 -- 172.21.15.45:0/1378638604 learned_addr learned my addr 172.21.15.45:0/1378638604 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:11:57.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.309+0000 7fef05a00640 1 -- 172.21.15.45:0/1378638604 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559b43fac390 msgr2=0x559b43fb9480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:11:57.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.309+0000 7fef05a00640 1 --2- 172.21.15.45:0/1378638604 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559b43fac390 0x559b43fb9480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:57.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.309+0000 7fef05a00640 1 -- 172.21.15.45:0/1378638604 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559b43faccb0 msgr2=0x559b43fb99c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:11:57.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.309+0000 7fef05a00640 1 --2- 172.21.15.45:0/1378638604 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559b43faccb0 0x559b43fb99c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:57.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.309+0000 7fef05a00640 1 -- 172.21.15.45:0/1378638604 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559b4402b2a0 con 0x559b43fb5890 2024-04-16T17:11:57.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.309+0000 7fef05a00640 1 --2- 172.21.15.45:0/1378638604 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559b43fb5890 0x559b43fb9f00 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7feef800f790 tx=0x7feef800fca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=797efbc2998b1297 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:11:57.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.309+0000 7feee6a00640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-04-16T17:11:57.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.310+0000 7feee6a00640 1 -- 172.21.15.45:0/1378638604 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feef801f040 con 0x559b43fb5890 2024-04-16T17:11:57.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.310+0000 7fef0a7db740 1 -- 172.21.15.45:0/1378638604 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feef400f8a0 con 0x559b43fb5890 2024-04-16T17:11:57.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.310+0000 7feee6a00640 1 -- 172.21.15.45:0/1378638604 <== mon.1 v2:172.21.15.117:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7feef8011040 con 0x559b43fb5890 2024-04-16T17:11:57.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.310+0000 7feee6a00640 1 -- 172.21.15.45:0/1378638604 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7feef8036050 con 0x559b43fb5890 2024-04-16T17:11:57.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.310+0000 7fef0a7db740 10 client.27034 fetch_fsmap learned FSMap version 35 2024-04-16T17:11:57.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.310+0000 7fef0a7db740 1 -- 172.21.15.45:0/1378638604 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x559b4402c5a0 con 0x559b43fb5890 2024-04-16T17:11:57.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.310+0000 7feee6a00640 1 -- 172.21.15.45:0/1378638604 <== mon.1 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feef400f8a0 con 0x559b43fb5890 2024-04-16T17:11:57.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.310+0000 7feee6a00640 1 -- 172.21.15.45:0/1378638604 <== mon.1 v2:172.21.15.117:3300/0 5 ==== fsmap(e 35) ==== 8358+0+0 (secure 0 0 0) 0x7feef8018070 con 0x559b43fb5890 2024-04-16T17:11:57.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.310+0000 7fef0a7db740 10 client.27034 fetch_fsmap finished waiting for FSMap version 35 2024-04-16T17:11:57.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.310+0000 7fef0a7db740 10 client.27034 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:11:57.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.310+0000 7fef0a7db740 20 client.27034 populate_metadata read hostname 'smithi045' 2024-04-16T17:11:57.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.310+0000 7fef0a7db740 1 --2- 172.21.15.45:0/1378638604 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x559b4402cbb0 0x559b4402efd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:11:57.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.310+0000 7fef0a7db740 4 client.27034 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-16T17:11:57.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.310+0000 7fef0a7db740 1 -- 172.21.15.45:0/1378638604 --> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] -- command(tid 0: {"prefix": "scrub status"}) -- 0x559b43e95420 con 0x559b4402cbb0 2024-04-16T17:11:57.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.311+0000 7fef05000640 1 --2- 172.21.15.45:0/1378638604 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x559b4402cbb0 0x559b4402efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:11:57.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.311+0000 7fef05000640 1 --2- 172.21.15.45:0/1378638604 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x559b4402cbb0 0x559b4402efd0 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b3d7d20bece1a122 server_cookie=825b78b9fd3a6506 in_seq=0 out_seq=0 2024-04-16T17:11:57.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.311+0000 7feee6a00640 10 client.27034 ms_handle_connect on v2:172.21.15.104:6838/3660070592 2024-04-16T17:11:57.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.311+0000 7feee6a00640 1 -- 172.21.15.45:0/1378638604 <== mds.0 v2:172.21.15.104:6838/3660070592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x559b43e95420 con 0x559b4402cbb0 2024-04-16T17:11:57.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.311+0000 7feee6a00640 10 client.27034 handle_command_reply: tid=0 2024-04-16T17:11:57.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.311+0000 7fef0a7db740 1 client.27034 shutdown 2024-04-16T17:11:57.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.312+0000 7fef0a7db740 1 -- 172.21.15.45:0/1378638604 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559b43fb5890 msgr2=0x559b43fb9f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:57.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.312+0000 7fef0a7db740 1 --2- 172.21.15.45:0/1378638604 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559b43fb5890 0x559b43fb9f00 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7feef800f790 tx=0x7feef800fca0 comp rx=0 tx=0).stop 2024-04-16T17:11:57.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.312+0000 7fef0a7db740 1 -- 172.21.15.45:0/1378638604 shutdown_connections 2024-04-16T17:11:57.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.312+0000 7fef0a7db740 1 --2- 172.21.15.45:0/1378638604 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x559b4402cbb0 0x559b4402efd0 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:57.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.312+0000 7fef0a7db740 1 --2- 172.21.15.45:0/1378638604 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559b43fb5890 0x559b43fb9f00 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:57.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.312+0000 7fef0a7db740 1 --2- 172.21.15.45:0/1378638604 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559b43faccb0 0x559b43fb99c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:57.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.312+0000 7fef0a7db740 1 --2- 172.21.15.45:0/1378638604 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559b43fac390 0x559b43fb9480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:57.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.312+0000 7feee6a00640 0 client.27034 ms_handle_reset on v2:172.21.15.104:6838/3660070592 2024-04-16T17:11:57.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.312+0000 7fef0a7db740 1 -- 172.21.15.45:0/1378638604 >> 172.21.15.45:0/1378638604 conn(0x559b43fa6ac0 msgr2=0x559b43fa7e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:11:57.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.312+0000 7fef0a7db740 1 -- 172.21.15.45:0/1378638604 shutdown_connections 2024-04-16T17:11:57.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.312+0000 7fef0a7db740 1 -- 172.21.15.45:0/1378638604 wait complete. 2024-04-16T17:11:57.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.312+0000 7fef0a7db740 20 client.27034 trim_cache size 0 max 16384 2024-04-16T17:11:57.314 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-16T17:11:57.315 INFO:teuthology.orchestra.run.smithi045.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-16T17:11:57.315 INFO:teuthology.orchestra.run.smithi045.stdout: "scrubs": { 2024-04-16T17:11:57.315 INFO:teuthology.orchestra.run.smithi045.stdout: "eb2684a1-7e8a-43ed-b085-a779e2454205": { 2024-04-16T17:11:57.315 INFO:teuthology.orchestra.run.smithi045.stdout: "path": "/", 2024-04-16T17:11:57.315 INFO:teuthology.orchestra.run.smithi045.stdout: "tag": "eb2684a1-7e8a-43ed-b085-a779e2454205", 2024-04-16T17:11:57.315 INFO:teuthology.orchestra.run.smithi045.stdout: "options": "recursive,force" 2024-04-16T17:11:57.315 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:11:57.315 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:11:57.315 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-16T17:11:57.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.313+0000 7fef06400640 1 -- 172.21.15.45:0/2019104256 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7feee80a3da0 msgr2=0x7feee80a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:57.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.313+0000 7fef06400640 1 --2- 172.21.15.45:0/2019104256 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7feee80a3da0 0x7feee80a6260 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x559b43fabe60 tx=0x7feef4005f90 comp rx=0 tx=0).stop 2024-04-16T17:11:57.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.313+0000 7fef06400640 1 -- 172.21.15.45:0/2019104256 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fef00103980 msgr2=0x7fef001a0bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:11:57.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.313+0000 7fef06400640 1 --2- 172.21.15.45:0/2019104256 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fef00103980 0x7fef001a0bc0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7feeec000c00 tx=0x7feeec0042f0 comp rx=0 tx=0).stop 2024-04-16T17:11:57.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.313+0000 7fef06400640 1 -- 172.21.15.45:0/2019104256 shutdown_connections 2024-04-16T17:11:57.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.313+0000 7fef06400640 1 --2- 172.21.15.45:0/2019104256 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7feee80a3da0 0x7feee80a6260 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:57.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.313+0000 7fef06400640 1 --2- 172.21.15.45:0/2019104256 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fef0010c480 0x7fef0019a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:57.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.313+0000 7fef06400640 1 --2- 172.21.15.45:0/2019104256 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fef00103980 0x7fef001a0bc0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:57.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.313+0000 7fef06400640 1 --2- 172.21.15.45:0/2019104256 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef00103060 0x7fef001a0680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:11:57.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.313+0000 7fef06400640 1 -- 172.21.15.45:0/2019104256 >> 172.21.15.45:0/2019104256 conn(0x7fef000fc880 msgr2=0x7fef00100be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:11:57.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.315+0000 7fef06400640 1 -- 172.21.15.45:0/2019104256 shutdown_connections 2024-04-16T17:11:57.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:11:57.316+0000 7fef06400640 1 -- 172.21.15.45:0/2019104256 wait complete. 2024-04-16T17:11:57.331 INFO:tasks.cephfs.filesystem:scrub status for tag:eb2684a1-7e8a-43ed-b085-a779e2454205 - {'path': '/', 'tag': 'eb2684a1-7e8a-43ed-b085-a779e2454205', 'options': 'recursive,force'} 2024-04-16T17:11:57.804 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:11:57 smithi117 ceph-mon[36730]: scrub summary: active paths [/] 2024-04-16T17:11:57.804 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:11:57 smithi117 ceph-mon[36730]: scrub summary: idle+waiting paths [/] 2024-04-16T17:11:57.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:11:57 smithi104 ceph-mon[36457]: scrub summary: active paths [/] 2024-04-16T17:11:57.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:11:57 smithi104 ceph-mon[36457]: scrub summary: idle+waiting paths [/] 2024-04-16T17:11:58.265 INFO:tasks.workunit.client.0.smithi045.stdout: 1 81403 1.72 MB/sec execute 120 sec latency 52.495 ms 2024-04-16T17:11:58.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:11:58 smithi117 ceph-mon[36730]: scrub queued for path: / 2024-04-16T17:11:58.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:11:58 smithi117 ceph-mon[36730]: scrub summary: idle+waiting paths [/] 2024-04-16T17:11:58.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:11:58 smithi117 ceph-mon[36730]: scrub summary: active paths [/] 2024-04-16T17:11:58.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:11:58 smithi117 ceph-mon[36730]: scrub summary: active paths [/] 2024-04-16T17:11:58.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:11:58 smithi117 ceph-mon[36730]: scrub summary: idle+waiting paths [/] 2024-04-16T17:11:58.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:11:58 smithi117 ceph-mon[36730]: scrub summary: idle+waiting paths [/] 2024-04-16T17:11:58.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:11:58 smithi117 ceph-mon[36730]: pgmap v159: 129 pgs: 129 active+clean; 46 MiB data, 615 MiB used, 1.0 TiB / 1.0 TiB avail; 706 KiB/s rd, 1.3 MiB/s wr, 125 op/s 2024-04-16T17:11:58.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:11:58 smithi104 ceph-mon[36457]: scrub queued for path: / 2024-04-16T17:11:58.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:11:58 smithi104 ceph-mon[36457]: scrub summary: idle+waiting paths [/] 2024-04-16T17:11:58.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:11:58 smithi104 ceph-mon[36457]: scrub summary: active paths [/] 2024-04-16T17:11:58.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:11:58 smithi104 ceph-mon[36457]: scrub summary: active paths [/] 2024-04-16T17:11:58.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:11:58 smithi104 ceph-mon[36457]: scrub summary: idle+waiting paths [/] 2024-04-16T17:11:58.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:11:58 smithi104 ceph-mon[36457]: scrub summary: idle+waiting paths [/] 2024-04-16T17:11:58.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:11:58 smithi104 ceph-mon[36457]: pgmap v159: 129 pgs: 129 active+clean; 46 MiB data, 615 MiB used, 1.0 TiB / 1.0 TiB avail; 706 KiB/s rd, 1.3 MiB/s wr, 125 op/s 2024-04-16T17:11:58.959 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:11:53 smithi045 ceph-mon[31676]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-16T17:11:58.959 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:11:54 smithi045 ceph-mon[31676]: pgmap v157: 129 pgs: 129 active+clean; 45 MiB data, 640 MiB used, 1.0 TiB / 1.0 TiB avail; 481 KiB/s rd, 898 KiB/s wr, 81 op/s 2024-04-16T17:11:58.959 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:11:55 smithi045 ceph-mon[31676]: from='client.? 172.21.15.45:0/1561395277' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:11:58.959 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:11:56 smithi045 ceph-mon[31676]: pgmap v158: 129 pgs: 129 active+clean; 45 MiB data, 640 MiB used, 1.0 TiB / 1.0 TiB avail; 481 KiB/s rd, 898 KiB/s wr, 81 op/s 2024-04-16T17:11:58.959 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:11:57 smithi045 ceph-mon[31676]: scrub summary: active paths [/] 2024-04-16T17:11:58.959 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:11:57 smithi045 ceph-mon[31676]: scrub summary: idle+waiting paths [/] 2024-04-16T17:11:58.959 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:11:58 smithi045 ceph-mon[31676]: scrub queued for path: / 2024-04-16T17:11:58.959 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:11:58 smithi045 ceph-mon[31676]: scrub summary: idle+waiting paths [/] 2024-04-16T17:11:58.959 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:11:58 smithi045 ceph-mon[31676]: scrub summary: active paths [/] 2024-04-16T17:11:58.959 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:11:58 smithi045 ceph-mon[31676]: scrub summary: active paths [/] 2024-04-16T17:11:58.960 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:11:58 smithi045 ceph-mon[31676]: scrub summary: idle+waiting paths [/] 2024-04-16T17:11:59.265 INFO:tasks.workunit.client.0.smithi045.stdout: 1 81822 1.74 MB/sec execute 121 sec latency 63.719 ms 2024-04-16T17:11:59.701 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:11:58 smithi045 ceph-mon[31676]: scrub summary: idle+waiting paths [/] 2024-04-16T17:11:59.701 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:11:58 smithi045 ceph-mon[31676]: pgmap v159: 129 pgs: 129 active+clean; 46 MiB data, 615 MiB used, 1.0 TiB / 1.0 TiB avail; 706 KiB/s rd, 1.3 MiB/s wr, 125 op/s 2024-04-16T17:12:00.265 INFO:tasks.workunit.client.0.smithi045.stdout: 1 82049 1.73 MB/sec execute 122 sec latency 65.842 ms 2024-04-16T17:12:00.804 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:12:00 smithi117 ceph-mon[36730]: pgmap v160: 129 pgs: 129 active+clean; 46 MiB data, 615 MiB used, 1.0 TiB / 1.0 TiB avail; 298 KiB/s rd, 1.1 MiB/s wr, 91 op/s 2024-04-16T17:12:00.804 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:12:00 smithi117 ceph-mon[36730]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T17:12:00.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:12:00 smithi104 ceph-mon[36457]: pgmap v160: 129 pgs: 129 active+clean; 46 MiB data, 615 MiB used, 1.0 TiB / 1.0 TiB avail; 298 KiB/s rd, 1.1 MiB/s wr, 91 op/s 2024-04-16T17:12:00.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:12:00 smithi104 ceph-mon[36457]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T17:12:01.265 INFO:tasks.workunit.client.0.smithi045.stdout: 1 82409 1.72 MB/sec execute 123 sec latency 55.114 ms 2024-04-16T17:12:01.576 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:01.576+0000 7f7a4ee00640 1 -- 172.21.15.45:0/3765005005 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7a4810e6e0 msgr2=0x7f7a48110ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:01.576 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:01.576+0000 7f7a4ee00640 1 --2- 172.21.15.45:0/3765005005 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7a4810e6e0 0x7f7a48110ad0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f7a40009900 tx=0x7f7a4002bcc0 comp rx=0 tx=0).stop 2024-04-16T17:12:01.577 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:01.577+0000 7f7a4ee00640 1 -- 172.21.15.45:0/3765005005 shutdown_connections 2024-04-16T17:12:01.577 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:01.577+0000 7f7a4ee00640 1 --2- 172.21.15.45:0/3765005005 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7a4810e6e0 0x7f7a48110ad0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:01.577 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:01.577+0000 7f7a4ee00640 1 --2- 172.21.15.45:0/3765005005 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7a48074cd0 0x7f7a4810e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:01.577 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:01.577+0000 7f7a4ee00640 1 --2- 172.21.15.45:0/3765005005 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7a480743b0 0x7f7a48074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:01.577 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:01.577+0000 7f7a4ee00640 1 -- 172.21.15.45:0/3765005005 >> 172.21.15.45:0/3765005005 conn(0x7f7a480fd950 msgr2=0x7f7a480ffd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:12:01.577 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:01.577+0000 7f7a4ee00640 1 -- 172.21.15.45:0/3765005005 shutdown_connections 2024-04-16T17:12:01.577 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:01.577+0000 7f7a4ee00640 1 -- 172.21.15.45:0/3765005005 wait complete. 2024-04-16T17:12:01.578 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:01.578+0000 7f7a4ee00640 1 Processor -- start 2024-04-16T17:12:01.578 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:01.578+0000 7f7a4ee00640 1 -- start start 2024-04-16T17:12:01.579 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:01.579+0000 7f7a4ee00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7a480743b0 0x7f7a481a0640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:01.579 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:01.579+0000 7f7a4ee00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7a48074cd0 0x7f7a481a0b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:01.579 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:01.579+0000 7f7a4ee00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7a4810e6e0 0x7f7a4819a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:01.579 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:01.579+0000 7f7a4ee00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f7a48114430 con 0x7f7a480743b0 2024-04-16T17:12:01.579 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:01.579+0000 7f7a4ee00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f7a481142b0 con 0x7f7a48074cd0 2024-04-16T17:12:01.579 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:01.579+0000 7f7a4ee00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f7a481145b0 con 0x7f7a4810e6e0 2024-04-16T17:12:01.579 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:01.579+0000 7f7a4e400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7a4810e6e0 0x7f7a4819a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:01.579 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:01.579+0000 7f7a4d000640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7a48074cd0 0x7f7a481a0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:01.580 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:01.579+0000 7f7a4da00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7a480743b0 0x7f7a481a0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:01.580 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:01.579+0000 7f7a4e400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7a4810e6e0 0x7f7a4819a750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:56582/0 (socket says 172.21.15.45:56582) 2024-04-16T17:12:01.580 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:01.579+0000 7f7a4e400640 1 -- 172.21.15.45:0/884237536 learned_addr learned my addr 172.21.15.45:0/884237536 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:12:01.581 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:01.580+0000 7f7a4e400640 1 -- 172.21.15.45:0/884237536 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7a48074cd0 msgr2=0x7f7a481a0b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:01.581 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:01.580+0000 7f7a4e400640 1 --2- 172.21.15.45:0/884237536 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7a48074cd0 0x7f7a481a0b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:01.581 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:01.580+0000 7f7a4e400640 1 -- 172.21.15.45:0/884237536 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7a480743b0 msgr2=0x7f7a481a0640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:01.581 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:01.580+0000 7f7a4e400640 1 --2- 172.21.15.45:0/884237536 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7a480743b0 0x7f7a481a0640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:01.581 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:01.580+0000 7f7a4e400640 1 -- 172.21.15.45:0/884237536 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a4819b010 con 0x7f7a4810e6e0 2024-04-16T17:12:01.581 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:01.580+0000 7f7a4d000640 1 --2- 172.21.15.45:0/884237536 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7a48074cd0 0x7f7a481a0b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:12:01.581 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:01.580+0000 7f7a4e400640 1 --2- 172.21.15.45:0/884237536 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7a4810e6e0 0x7f7a4819a750 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f7a40005ab0 tx=0x7f7a400042e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:12:01.581 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:01.580+0000 7f7a46000640 1 -- 172.21.15.45:0/884237536 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a40042020 con 0x7f7a4810e6e0 2024-04-16T17:12:01.581 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:01.580+0000 7f7a46000640 1 -- 172.21.15.45:0/884237536 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7a400044d0 con 0x7f7a4810e6e0 2024-04-16T17:12:01.581 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:01.580+0000 7f7a46000640 1 -- 172.21.15.45:0/884237536 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a40003b50 con 0x7f7a4810e6e0 2024-04-16T17:12:01.581 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:01.580+0000 7f7a4ee00640 1 -- 172.21.15.45:0/884237536 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a4819b2a0 con 0x7f7a4810e6e0 2024-04-16T17:12:01.582 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:01.581+0000 7f7a4ee00640 1 -- 172.21.15.45:0/884237536 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a481a73e0 con 0x7f7a4810e6e0 2024-04-16T17:12:01.584 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:01.585+0000 7f7a531f1740 1 Processor -- start 2024-04-16T17:12:01.585 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:01.586+0000 7f7a46000640 1 -- 172.21.15.45:0/884237536 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f7a40045020 con 0x7f7a4810e6e0 2024-04-16T17:12:01.586 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:01.586+0000 7f7a531f1740 1 -- start start 2024-04-16T17:12:01.587 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:01.587+0000 7f7a531f1740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b68e20b7f0 0x55b68e20a540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:01.587 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:01.587+0000 7f7a531f1740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b68e20aa80 0x55b68e20af00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:01.587 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:01.587+0000 7f7a531f1740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b68e21c810 0x55b68e21ec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:01.587 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:01.587+0000 7f7a531f1740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55b68e16f780 con 0x55b68e20b7f0 2024-04-16T17:12:01.587 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:01.587+0000 7f7a531f1740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55b68e115f40 con 0x55b68e21c810 2024-04-16T17:12:01.587 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:01.587+0000 7f7a531f1740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55b68e16b740 con 0x55b68e20aa80 2024-04-16T17:12:01.587 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:01.587+0000 7f7a4da00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b68e20aa80 0x55b68e20af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:01.587 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:01.587+0000 7f7a4da00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b68e20aa80 0x55b68e20af00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:56588/0 (socket says 172.21.15.45:56588) 2024-04-16T17:12:01.587 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:01.587+0000 7f7a4da00640 1 -- 172.21.15.45:0/4056284610 learned_addr learned my addr 172.21.15.45:0/4056284610 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:12:01.587 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:01.587+0000 7f7a4d000640 1 --2- 172.21.15.45:0/4056284610 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b68e21c810 0x55b68e21ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:01.588 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:01.588+0000 7f7a4d000640 1 --2- 172.21.15.45:0/4056284610 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b68e20b7f0 0x55b68e20a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:01.589 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:01.588+0000 7f7a4da00640 1 -- 172.21.15.45:0/4056284610 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b68e21c810 msgr2=0x55b68e21ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:12:01.589 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:01.588+0000 7f7a4da00640 1 --2- 172.21.15.45:0/4056284610 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b68e21c810 0x55b68e21ec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:01.589 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:01.588+0000 7f7a4da00640 1 -- 172.21.15.45:0/4056284610 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b68e20b7f0 msgr2=0x55b68e20a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:12:01.589 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:01.588+0000 7f7a4da00640 1 --2- 172.21.15.45:0/4056284610 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b68e20b7f0 0x55b68e20a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:01.589 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:01.588+0000 7f7a4da00640 1 -- 172.21.15.45:0/4056284610 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b68de82210 con 0x55b68e20aa80 2024-04-16T17:12:01.589 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:01.588+0000 7f7a46000640 1 --2- 172.21.15.45:0/884237536 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f7a300a3d90 0x7f7a300a6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:01.589 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:01.588+0000 7f7a4da00640 1 --2- 172.21.15.45:0/4056284610 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b68e20aa80 0x55b68e20af00 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f7a3c0114f0 tx=0x7f7a3c0119b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=350221c2e9e5450a server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:12:01.589 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:01.588+0000 7f7a46000640 1 -- 172.21.15.45:0/884237536 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f7a400f2460 con 0x7f7a4810e6e0 2024-04-16T17:12:01.589 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:01.588+0000 7f7a4d000640 1 --2- 172.21.15.45:0/884237536 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f7a300a3d90 0x7f7a300a6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:01.589 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:01.588+0000 7f7a3aa00640 1 -- 172.21.15.45:0/4056284610 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a3c01e070 con 0x55b68e20aa80 2024-04-16T17:12:01.589 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:01.589+0000 7f7a3aa00640 1 -- 172.21.15.45:0/4056284610 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7a3c00fb00 con 0x55b68e20aa80 2024-04-16T17:12:01.589 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:01.589+0000 7f7a3aa00640 1 -- 172.21.15.45:0/4056284610 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a3c019bf0 con 0x55b68e20aa80 2024-04-16T17:12:01.590 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:01.590+0000 7f7a531f1740 1 -- 172.21.15.45:0/4056284610 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b68e20aa80 msgr2=0x55b68e20af00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:01.591 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:01.590+0000 7f7a531f1740 1 --2- 172.21.15.45:0/4056284610 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b68e20aa80 0x55b68e20af00 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f7a3c0114f0 tx=0x7f7a3c0119b0 comp rx=0 tx=0).stop 2024-04-16T17:12:01.591 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:01.590+0000 7f7a4d000640 1 --2- 172.21.15.45:0/884237536 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f7a300a3d90 0x7f7a300a6250 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f7a34009fd0 tx=0x7f7a34005f90 comp rx=0 tx=0).ready entity=mgr.25411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:12:01.591 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:01.590+0000 7f7a531f1740 1 -- 172.21.15.45:0/4056284610 shutdown_connections 2024-04-16T17:12:01.591 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:01.590+0000 7f7a531f1740 1 --2- 172.21.15.45:0/4056284610 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b68e20aa80 0x55b68e20af00 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:01.591 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:01.590+0000 7f7a531f1740 1 --2- 172.21.15.45:0/4056284610 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b68e21c810 0x55b68e21ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:01.591 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:01.590+0000 7f7a531f1740 1 --2- 172.21.15.45:0/4056284610 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b68e20b7f0 0x55b68e20a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:01.591 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:01.590+0000 7f7a531f1740 1 -- 172.21.15.45:0/4056284610 >> 172.21.15.45:0/4056284610 conn(0x55b68e1a55d0 msgr2=0x55b68e205770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:12:01.591 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:01.590+0000 7f7a531f1740 1 -- 172.21.15.45:0/4056284610 shutdown_connections 2024-04-16T17:12:01.591 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:01.590+0000 7f7a531f1740 1 -- 172.21.15.45:0/4056284610 wait complete. 2024-04-16T17:12:01.591 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:01.591+0000 7f7a531f1740 1 -- start start 2024-04-16T17:12:01.591 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:01.591+0000 7f7a531f1740 1 Processor -- start 2024-04-16T17:12:01.592 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:01.592+0000 7f7a531f1740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b68e20aa80 0x55b68e219680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:01.592 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:01.592+0000 7f7a531f1740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b68e20b7f0 0x55b68e219bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:01.592 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:01.592+0000 7f7a531f1740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b68e21c810 0x55b68e21a100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:01.592 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:01.592+0000 7f7a531f1740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55b68e21f540 con 0x55b68e21c810 2024-04-16T17:12:01.592 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:01.592+0000 7f7a531f1740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55b68e129390 con 0x55b68e20b7f0 2024-04-16T17:12:01.592 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:01.592+0000 7f7a531f1740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55b68e21f6c0 con 0x55b68e20aa80 2024-04-16T17:12:01.593 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:01.592+0000 7f7a531f1740 10 client.0 _ll_register_callbacks cb 0x55b68e112f60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:12:01.593 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:01.592+0000 7f7a4da00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b68e20aa80 0x55b68e219680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:01.593 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:01.592+0000 7f7a4da00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b68e20aa80 0x55b68e219680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:56604/0 (socket says 172.21.15.45:56604) 2024-04-16T17:12:01.593 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:01.592+0000 7f7a4da00640 1 -- 172.21.15.45:0/2978535630 learned_addr learned my addr 172.21.15.45:0/2978535630 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:12:01.593 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:01.592+0000 7f7a4d000640 1 --2- 172.21.15.45:0/2978535630 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b68e20b7f0 0x55b68e219bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:01.594 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:01.592+0000 7f7a4e400640 1 --2- 172.21.15.45:0/2978535630 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b68e21c810 0x55b68e21a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:01.594 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:01.592+0000 7f7a4da00640 1 -- 172.21.15.45:0/2978535630 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b68e20b7f0 msgr2=0x55b68e219bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:12:01.595 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:01.592+0000 7f7a4da00640 1 --2- 172.21.15.45:0/2978535630 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b68e20b7f0 0x55b68e219bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:01.595 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:01.592+0000 7f7a4da00640 1 -- 172.21.15.45:0/2978535630 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b68e21c810 msgr2=0x55b68e21a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:12:01.595 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:01.592+0000 7f7a4da00640 1 --2- 172.21.15.45:0/2978535630 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b68e21c810 0x55b68e21a100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:01.595 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:01.592+0000 7f7a4da00640 1 -- 172.21.15.45:0/2978535630 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b68df4e620 con 0x55b68e20aa80 2024-04-16T17:12:01.595 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:01.593+0000 7f7a4e400640 1 --2- 172.21.15.45:0/2978535630 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b68e21c810 0x55b68e21a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:12:01.595 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:01.593+0000 7f7a4da00640 1 --2- 172.21.15.45:0/2978535630 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b68e20aa80 0x55b68e219680 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f7a3c005c10 tx=0x7f7a3c019410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=409e9734d538db8e server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:12:01.595 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:01.593+0000 7f7a2f400640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-04-16T17:12:01.595 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:01.593+0000 7f7a2f400640 1 -- 172.21.15.45:0/2978535630 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a3c01e040 con 0x55b68e20aa80 2024-04-16T17:12:01.595 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:01.593+0000 7f7a2f400640 1 -- 172.21.15.45:0/2978535630 <== mon.1 v2:172.21.15.117:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f7a3c004a40 con 0x55b68e20aa80 2024-04-16T17:12:01.595 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:01.593+0000 7f7a531f1740 1 -- 172.21.15.45:0/2978535630 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7a3c00fb00 con 0x55b68e20aa80 2024-04-16T17:12:01.595 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:01.594+0000 7f7a2f400640 1 -- 172.21.15.45:0/2978535630 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f7a3c0194c0 con 0x55b68e20aa80 2024-04-16T17:12:01.595 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:01.594+0000 7f7a531f1740 10 client.27046 fetch_fsmap learned FSMap version 36 2024-04-16T17:12:01.595 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:01.594+0000 7f7a531f1740 1 -- 172.21.15.45:0/2978535630 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55b68e22b670 con 0x55b68e20aa80 2024-04-16T17:12:01.596 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:01.594+0000 7f7a2f400640 1 -- 172.21.15.45:0/2978535630 <== mon.1 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7a3c00fb00 con 0x55b68e20aa80 2024-04-16T17:12:01.596 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:01.594+0000 7f7a2f400640 1 -- 172.21.15.45:0/2978535630 <== mon.1 v2:172.21.15.117:3300/0 5 ==== fsmap(e 36) ==== 8362+0+0 (secure 0 0 0) 0x7f7a3c02c030 con 0x55b68e20aa80 2024-04-16T17:12:01.596 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:01.595+0000 7f7a531f1740 10 client.27046 fetch_fsmap finished waiting for FSMap version 36 2024-04-16T17:12:01.596 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:01.595+0000 7f7a531f1740 10 client.27046 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T17:12:01.596 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:01.595+0000 7f7a531f1740 20 client.27046 populate_metadata read hostname 'smithi045' 2024-04-16T17:12:01.596 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:01.595+0000 7f7a531f1740 1 --2- 172.21.15.45:0/2978535630 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55b68e22ca70 0x55b68e22ee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:01.596 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:01.595+0000 7f7a531f1740 4 client.27046 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:12:01.596 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:01.595+0000 7f7a531f1740 1 -- 172.21.15.45:0/2978535630 --> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b68e17d1a0 con 0x55b68e22ca70 2024-04-16T17:12:01.597 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:01.595+0000 7f7a4d000640 1 --2- 172.21.15.45:0/2978535630 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55b68e22ca70 0x55b68e22ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:01.598 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:01.596+0000 7f7a4d000640 1 --2- 172.21.15.45:0/2978535630 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55b68e22ca70 0x55b68e22ee60 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fcadb6d36ec8e8f5 server_cookie=369ed3fd9064bb1a in_seq=0 out_seq=0 2024-04-16T17:12:01.598 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:01.596+0000 7f7a2f400640 10 client.27046 ms_handle_connect on v2:172.21.15.104:6838/3660070592 2024-04-16T17:12:01.598 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:01.596+0000 7f7a2f400640 1 -- 172.21.15.45:0/2978535630 <== mds.0 v2:172.21.15.104:6838/3660070592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b68e17d1a0 con 0x55b68e22ca70 2024-04-16T17:12:01.598 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:01.596+0000 7f7a2f400640 10 client.27046 handle_command_reply: tid=0 2024-04-16T17:12:01.598 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:01.597+0000 7f7a531f1740 1 client.27046 shutdown 2024-04-16T17:12:01.598 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:01.597+0000 7f7a531f1740 1 -- 172.21.15.45:0/2978535630 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b68e20aa80 msgr2=0x55b68e219680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:01.598 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:01.597+0000 7f7a531f1740 1 --2- 172.21.15.45:0/2978535630 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b68e20aa80 0x55b68e219680 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f7a3c005c10 tx=0x7f7a3c019410 comp rx=0 tx=0).stop 2024-04-16T17:12:01.598 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:01.597+0000 7f7a531f1740 1 -- 172.21.15.45:0/2978535630 shutdown_connections 2024-04-16T17:12:01.598 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:01.597+0000 7f7a531f1740 1 --2- 172.21.15.45:0/2978535630 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b68e21c810 0x55b68e21a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:01.598 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:01.597+0000 7f7a531f1740 1 --2- 172.21.15.45:0/2978535630 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b68e20b7f0 0x55b68e219bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:01.598 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:01.597+0000 7f7a531f1740 1 --2- 172.21.15.45:0/2978535630 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55b68e22ca70 0x55b68e22ee60 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:01.598 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:01.597+0000 7f7a531f1740 1 --2- 172.21.15.45:0/2978535630 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b68e20aa80 0x55b68e219680 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:01.598 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:01.597+0000 7f7a531f1740 1 -- 172.21.15.45:0/2978535630 >> 172.21.15.45:0/2978535630 conn(0x55b68e1a55d0 msgr2=0x55b68e2074c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:12:01.598 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:01.597+0000 7f7a2f400640 0 client.27046 ms_handle_reset on v2:172.21.15.104:6838/3660070592 2024-04-16T17:12:01.598 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:01.597+0000 7f7a531f1740 1 -- 172.21.15.45:0/2978535630 shutdown_connections 2024-04-16T17:12:01.599 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:01.597+0000 7f7a531f1740 1 -- 172.21.15.45:0/2978535630 wait complete. 2024-04-16T17:12:01.599 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:01.597+0000 7f7a531f1740 20 client.27046 trim_cache size 0 max 16384 2024-04-16T17:12:01.616 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:01.616+0000 7f7a531f1740 1 Processor -- start 2024-04-16T17:12:01.616 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:01.616+0000 7f7a531f1740 1 -- start start 2024-04-16T17:12:01.617 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:01.617+0000 7f7a531f1740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b68e20aa80 0x55b68e21bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:01.617 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:01.617+0000 7f7a531f1740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b68e20b7f0 0x55b68e228440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:01.617 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:01.617+0000 7f7a531f1740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b68e21c810 0x55b68e218660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:01.618 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:01.617+0000 7f7a531f1740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f7a481af9d0 con 0x55b68e20b7f0 2024-04-16T17:12:01.618 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:01.617+0000 7f7a531f1740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f7a481ad210 con 0x55b68e20aa80 2024-04-16T17:12:01.618 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:01.617+0000 7f7a531f1740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f7a481ab040 con 0x55b68e21c810 2024-04-16T17:12:01.618 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:01.617+0000 7f7a4da00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b68e20aa80 0x55b68e21bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:01.618 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:01.617+0000 7f7a4d000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b68e20b7f0 0x55b68e228440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:01.618 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:01.617+0000 7f7a4da00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b68e20aa80 0x55b68e21bdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:34222/0 (socket says 172.21.15.45:34222) 2024-04-16T17:12:01.618 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:01.617+0000 7f7a4da00640 1 -- 172.21.15.45:0/2284660047 learned_addr learned my addr 172.21.15.45:0/2284660047 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:12:01.618 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:01.617+0000 7f7a4e400640 1 --2- 172.21.15.45:0/2284660047 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b68e21c810 0x55b68e218660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:01.618 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:01.618+0000 7f7a4d000640 1 -- 172.21.15.45:0/2284660047 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b68e21c810 msgr2=0x55b68e218660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:12:01.618 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:01.618+0000 7f7a4d000640 1 --2- 172.21.15.45:0/2284660047 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b68e21c810 0x55b68e218660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:01.618 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:01.618+0000 7f7a4d000640 1 -- 172.21.15.45:0/2284660047 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b68e20aa80 msgr2=0x55b68e21bdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:12:01.618 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:01.618+0000 7f7a4d000640 1 --2- 172.21.15.45:0/2284660047 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b68e20aa80 0x55b68e21bdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:01.618 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:01.618+0000 7f7a4d000640 1 -- 172.21.15.45:0/2284660047 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b68e20a8b0 con 0x55b68e20b7f0 2024-04-16T17:12:01.619 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:01.618+0000 7f7a4da00640 1 --2- 172.21.15.45:0/2284660047 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b68e20aa80 0x55b68e21bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:12:01.619 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:01.618+0000 7f7a4d000640 1 --2- 172.21.15.45:0/2284660047 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b68e20b7f0 0x55b68e228440 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x55b68e206900 tx=0x7f7a34019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1d52b883ce8d6af server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:12:01.621 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:01.618+0000 7f7a3a000640 1 -- 172.21.15.45:0/2284660047 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a3402c3e0 con 0x55b68e20b7f0 2024-04-16T17:12:01.621 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:01.618+0000 7f7a3a000640 1 -- 172.21.15.45:0/2284660047 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7a340225e0 con 0x55b68e20b7f0 2024-04-16T17:12:01.621 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:01.618+0000 7f7a3a000640 1 -- 172.21.15.45:0/2284660047 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a340206a0 con 0x55b68e20b7f0 2024-04-16T17:12:01.624 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:01.619+0000 7f7a531f1740 1 -- 172.21.15.45:0/2284660047 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b68e20b7f0 msgr2=0x55b68e228440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:01.624 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:01.619+0000 7f7a531f1740 1 --2- 172.21.15.45:0/2284660047 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b68e20b7f0 0x55b68e228440 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x55b68e206900 tx=0x7f7a34019000 comp rx=0 tx=0).stop 2024-04-16T17:12:01.624 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:01.619+0000 7f7a531f1740 1 -- 172.21.15.45:0/2284660047 shutdown_connections 2024-04-16T17:12:01.624 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:01.619+0000 7f7a531f1740 1 --2- 172.21.15.45:0/2284660047 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b68e21c810 0x55b68e218660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:01.624 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:01.619+0000 7f7a531f1740 1 --2- 172.21.15.45:0/2284660047 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b68e20b7f0 0x55b68e228440 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:01.624 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:01.619+0000 7f7a531f1740 1 --2- 172.21.15.45:0/2284660047 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b68e20aa80 0x55b68e21bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:01.624 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:01.619+0000 7f7a531f1740 1 -- 172.21.15.45:0/2284660047 >> 172.21.15.45:0/2284660047 conn(0x55b68e1a55d0 msgr2=0x55b68e217f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:12:01.625 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:01.619+0000 7f7a531f1740 1 -- 172.21.15.45:0/2284660047 shutdown_connections 2024-04-16T17:12:01.625 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:01.619+0000 7f7a531f1740 1 -- 172.21.15.45:0/2284660047 wait complete. 2024-04-16T17:12:01.625 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:01.620+0000 7f7a531f1740 1 -- start start 2024-04-16T17:12:01.625 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:01.620+0000 7f7a531f1740 1 Processor -- start 2024-04-16T17:12:01.625 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:01.620+0000 7f7a531f1740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b68e20aa80 0x55b68e21cde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:01.625 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:01.620+0000 7f7a531f1740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b68e20b7f0 0x55b68e21d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:01.625 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:01.620+0000 7f7a531f1740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b68e21c810 0x55b68e223950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:01.625 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:01.620+0000 7f7a531f1740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f7a481ab6d0 con 0x55b68e21c810 2024-04-16T17:12:01.625 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:01.620+0000 7f7a531f1740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f7a481acb40 con 0x55b68e20b7f0 2024-04-16T17:12:01.625 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:01.620+0000 7f7a531f1740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f7a481abda0 con 0x55b68e20aa80 2024-04-16T17:12:01.625 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:01.620+0000 7f7a531f1740 10 client.0 _ll_register_callbacks cb 0x55b68e21c760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:12:01.625 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:01.621+0000 7f7a4d000640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b68e20b7f0 0x55b68e21d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:01.625 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:01.621+0000 7f7a4da00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b68e20aa80 0x55b68e21cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:01.625 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:01.621+0000 7f7a4d000640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b68e20b7f0 0x55b68e21d320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:34230/0 (socket says 172.21.15.45:34230) 2024-04-16T17:12:01.625 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:01.621+0000 7f7a4da00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b68e20aa80 0x55b68e21cde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:56634/0 (socket says 172.21.15.45:56634) 2024-04-16T17:12:01.625 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:01.621+0000 7f7a4d000640 1 -- 172.21.15.45:0/1257297837 learned_addr learned my addr 172.21.15.45:0/1257297837 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:12:01.625 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:01.621+0000 7f7a4e400640 1 --2- 172.21.15.45:0/1257297837 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b68e21c810 0x55b68e223950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:01.626 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:01.621+0000 7f7a4d000640 1 -- 172.21.15.45:0/1257297837 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b68e20aa80 msgr2=0x55b68e21cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:12:01.626 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:01.621+0000 7f7a4d000640 1 --2- 172.21.15.45:0/1257297837 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b68e20aa80 0x55b68e21cde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:01.626 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:01.621+0000 7f7a4d000640 1 -- 172.21.15.45:0/1257297837 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b68e21c810 msgr2=0x55b68e223950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:12:01.626 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:01.621+0000 7f7a4d000640 1 --2- 172.21.15.45:0/1257297837 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b68e21c810 0x55b68e223950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:01.626 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:01.621+0000 7f7a4d000640 1 -- 172.21.15.45:0/1257297837 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b68e21ef70 con 0x55b68e20b7f0 2024-04-16T17:12:01.626 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:01.621+0000 7f7a4da00640 1 --2- 172.21.15.45:0/1257297837 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b68e20aa80 0x55b68e21cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:12:01.626 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:01.621+0000 7f7a4d000640 1 --2- 172.21.15.45:0/1257297837 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b68e20b7f0 0x55b68e21d320 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f7a340058c0 tx=0x7f7a34022150 comp rx=0 tx=0).ready entity=mon.2 client_cookie=22e363b771c21efb server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:12:01.626 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:01.621+0000 7f7a2f400640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-04-16T17:12:01.626 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:01.622+0000 7f7a2f400640 1 -- 172.21.15.45:0/1257297837 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a3402c5f0 con 0x55b68e20b7f0 2024-04-16T17:12:01.626 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:01.622+0000 7f7a2f400640 1 -- 172.21.15.45:0/1257297837 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f7a3402c790 con 0x55b68e20b7f0 2024-04-16T17:12:01.626 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:01.622+0000 7f7a531f1740 1 -- 172.21.15.45:0/1257297837 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7a340225e0 con 0x55b68e20b7f0 2024-04-16T17:12:01.626 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:01.622+0000 7f7a2f400640 1 -- 172.21.15.45:0/1257297837 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f7a34042050 con 0x55b68e20b7f0 2024-04-16T17:12:01.626 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:01.622+0000 7f7a531f1740 10 client.27101 fetch_fsmap learned FSMap version 36 2024-04-16T17:12:01.626 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:01.622+0000 7f7a531f1740 1 -- 172.21.15.45:0/1257297837 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55b68e224e50 con 0x55b68e20b7f0 2024-04-16T17:12:01.626 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:01.622+0000 7f7a2f400640 1 -- 172.21.15.45:0/1257297837 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7a340225e0 con 0x55b68e20b7f0 2024-04-16T17:12:01.627 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:01.623+0000 7f7a2f400640 1 -- 172.21.15.45:0/1257297837 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 36) ==== 8362+0+0 (secure 0 0 0) 0x7f7a340046c0 con 0x55b68e20b7f0 2024-04-16T17:12:01.627 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:01.623+0000 7f7a531f1740 10 client.27101 fetch_fsmap finished waiting for FSMap version 36 2024-04-16T17:12:01.627 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:01.623+0000 7f7a531f1740 10 client.27101 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T17:12:01.627 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:01.623+0000 7f7a531f1740 20 client.27101 populate_metadata read hostname 'smithi045' 2024-04-16T17:12:01.627 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:01.623+0000 7f7a531f1740 1 --2- 172.21.15.45:0/1257297837 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55b68e293030 0x55b68e295450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:01.627 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:01.623+0000 7f7a531f1740 4 client.27101 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-16T17:12:01.627 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:01.623+0000 7f7a531f1740 1 -- 172.21.15.45:0/1257297837 --> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b68e177010 con 0x55b68e293030 2024-04-16T17:12:01.627 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:01.623+0000 7f7a4da00640 1 --2- 172.21.15.45:0/1257297837 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55b68e293030 0x55b68e295450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:01.627 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:01.624+0000 7f7a4da00640 1 --2- 172.21.15.45:0/1257297837 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55b68e293030 0x55b68e295450 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=72db589af1c0abdd server_cookie=6743b08b7cea1cfe in_seq=0 out_seq=0 2024-04-16T17:12:01.627 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:01.624+0000 7f7a2f400640 10 client.27101 ms_handle_connect on v2:172.21.15.104:6838/3660070592 2024-04-16T17:12:01.678 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:01.678+0000 7f7a2f400640 1 -- 172.21.15.45:0/1257297837 <== mds.0 v2:172.21.15.104:6838/3660070592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55b68e177010 con 0x55b68e293030 2024-04-16T17:12:01.678 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:01.678+0000 7f7a2f400640 10 client.27101 handle_command_reply: tid=0 2024-04-16T17:12:01.678 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:01.678+0000 7f7a531f1740 1 client.27101 shutdown 2024-04-16T17:12:01.679 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:01.679+0000 7f7a531f1740 1 -- 172.21.15.45:0/1257297837 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b68e20b7f0 msgr2=0x55b68e21d320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:01.679 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:01.679+0000 7f7a531f1740 1 --2- 172.21.15.45:0/1257297837 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b68e20b7f0 0x55b68e21d320 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f7a340058c0 tx=0x7f7a34022150 comp rx=0 tx=0).stop 2024-04-16T17:12:01.679 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:01.679+0000 7f7a531f1740 1 -- 172.21.15.45:0/1257297837 shutdown_connections 2024-04-16T17:12:01.679 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:01.679+0000 7f7a531f1740 1 --2- 172.21.15.45:0/1257297837 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b68e21c810 0x55b68e223950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:01.679 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:01.679+0000 7f7a531f1740 1 --2- 172.21.15.45:0/1257297837 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b68e20b7f0 0x55b68e21d320 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:01.679 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:01.679+0000 7f7a531f1740 1 --2- 172.21.15.45:0/1257297837 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55b68e293030 0x55b68e295450 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:01.679 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:01.679+0000 7f7a531f1740 1 --2- 172.21.15.45:0/1257297837 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b68e20aa80 0x55b68e21cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:01.679 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:01.679+0000 7f7a531f1740 1 -- 172.21.15.45:0/1257297837 >> 172.21.15.45:0/1257297837 conn(0x55b68e1a55d0 msgr2=0x55b68e2195a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:12:01.679 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:01.679+0000 7f7a2f400640 0 client.27101 ms_handle_reset on v2:172.21.15.104:6838/3660070592 2024-04-16T17:12:01.680 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:01.679+0000 7f7a531f1740 1 -- 172.21.15.45:0/1257297837 shutdown_connections 2024-04-16T17:12:01.680 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:01.679+0000 7f7a531f1740 1 -- 172.21.15.45:0/1257297837 wait complete. 2024-04-16T17:12:01.680 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:01.680+0000 7f7a531f1740 20 client.27101 trim_cache size 0 max 16384 2024-04-16T17:12:01.680 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-04-16T17:12:01.680 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-04-16T17:12:01.680 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-04-16T17:12:01.680 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-04-16T17:12:01.681 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:01.680+0000 7f7a4ee00640 1 -- 172.21.15.45:0/884237536 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f7a300a3d90 msgr2=0x7f7a300a6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:01.681 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:01.680+0000 7f7a4ee00640 1 --2- 172.21.15.45:0/884237536 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f7a300a3d90 0x7f7a300a6250 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f7a34009fd0 tx=0x7f7a34005f90 comp rx=0 tx=0).stop 2024-04-16T17:12:01.681 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:01.680+0000 7f7a4ee00640 1 -- 172.21.15.45:0/884237536 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7a4810e6e0 msgr2=0x7f7a4819a750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:01.681 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:01.680+0000 7f7a4ee00640 1 --2- 172.21.15.45:0/884237536 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7a4810e6e0 0x7f7a4819a750 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f7a40005ab0 tx=0x7f7a400042e0 comp rx=0 tx=0).stop 2024-04-16T17:12:01.681 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:01.681+0000 7f7a4ee00640 1 -- 172.21.15.45:0/884237536 shutdown_connections 2024-04-16T17:12:01.682 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:01.681+0000 7f7a4ee00640 1 --2- 172.21.15.45:0/884237536 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f7a300a3d90 0x7f7a300a6250 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:01.682 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:01.681+0000 7f7a4ee00640 1 --2- 172.21.15.45:0/884237536 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7a4810e6e0 0x7f7a4819a750 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:01.682 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:01.681+0000 7f7a4ee00640 1 --2- 172.21.15.45:0/884237536 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7a48074cd0 0x7f7a481a0b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:01.682 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:01.681+0000 7f7a4ee00640 1 --2- 172.21.15.45:0/884237536 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7a480743b0 0x7f7a481a0640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:01.682 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:01.681+0000 7f7a4ee00640 1 -- 172.21.15.45:0/884237536 >> 172.21.15.45:0/884237536 conn(0x7f7a480fd950 msgr2=0x7f7a480fe5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:12:01.682 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:01.681+0000 7f7a4ee00640 1 -- 172.21.15.45:0/884237536 shutdown_connections 2024-04-16T17:12:01.682 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:01.681+0000 7f7a4ee00640 1 -- 172.21.15.45:0/884237536 wait complete. 2024-04-16T17:12:01.804 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:12:01 smithi117 ceph-mon[36730]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:12:01.804 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:12:01 smithi117 ceph-mon[36730]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T17:12:01.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:12:01 smithi117 ceph-mon[36730]: from='mgr.25411 ' entity='mgr.x' 2024-04-16T17:12:01.846 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:12:00 smithi045 ceph-mon[31676]: pgmap v160: 129 pgs: 129 active+clean; 46 MiB data, 615 MiB used, 1.0 TiB / 1.0 TiB avail; 298 KiB/s rd, 1.1 MiB/s wr, 91 op/s 2024-04-16T17:12:01.847 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:12:00 smithi045 ceph-mon[31676]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T17:12:01.847 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:12:01 smithi045 ceph-mon[31676]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:12:01.847 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:12:01 smithi045 ceph-mon[31676]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T17:12:01.847 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:12:01 smithi045 ceph-mon[31676]: from='mgr.25411 ' entity='mgr.x' 2024-04-16T17:12:01.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:12:01 smithi104 ceph-mon[36457]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:12:01.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:12:01 smithi104 ceph-mon[36457]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T17:12:01.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:12:01 smithi104 ceph-mon[36457]: from='mgr.25411 ' entity='mgr.x' 2024-04-16T17:12:02.266 INFO:tasks.workunit.client.0.smithi045.stdout: 1 82669 1.72 MB/sec execute 124 sec latency 76.290 ms 2024-04-16T17:12:02.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:12:02 smithi117 ceph-mon[36730]: pgmap v161: 129 pgs: 129 active+clean; 48 MiB data, 623 MiB used, 1.0 TiB / 1.0 TiB avail; 738 KiB/s rd, 1.3 MiB/s wr, 145 op/s 2024-04-16T17:12:02.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:12:02 smithi117 ceph-mon[36730]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-16T17:12:02.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:12:02 smithi104 ceph-mon[36457]: pgmap v161: 129 pgs: 129 active+clean; 48 MiB data, 623 MiB used, 1.0 TiB / 1.0 TiB avail; 738 KiB/s rd, 1.3 MiB/s wr, 145 op/s 2024-04-16T17:12:02.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:12:02 smithi104 ceph-mon[36457]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-16T17:12:03.265 INFO:tasks.workunit.client.0.smithi045.stdout: 1 83045 1.73 MB/sec execute 125 sec latency 246.359 ms 2024-04-16T17:12:03.477 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:12:02 smithi045 ceph-mon[31676]: pgmap v161: 129 pgs: 129 active+clean; 48 MiB data, 623 MiB used, 1.0 TiB / 1.0 TiB avail; 738 KiB/s rd, 1.3 MiB/s wr, 145 op/s 2024-04-16T17:12:03.477 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:12:02 smithi045 ceph-mon[31676]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-16T17:12:04.265 INFO:tasks.workunit.client.0.smithi045.stdout: 1 83045 1.72 MB/sec execute 126 sec latency 1246.442 ms 2024-04-16T17:12:04.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:12:04 smithi117 ceph-mon[36730]: pgmap v162: 129 pgs: 129 active+clean; 48 MiB data, 623 MiB used, 1.0 TiB / 1.0 TiB avail; 665 KiB/s rd, 677 KiB/s wr, 98 op/s 2024-04-16T17:12:04.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:12:04 smithi117 ceph-mon[36730]: scrub summary: idle 2024-04-16T17:12:04.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:12:04 smithi117 ceph-mon[36730]: scrub summary: idle 2024-04-16T17:12:04.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:12:04 smithi117 ceph-mon[36730]: scrub summary: idle 2024-04-16T17:12:04.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:12:04 smithi104 ceph-mon[36457]: pgmap v162: 129 pgs: 129 active+clean; 48 MiB data, 623 MiB used, 1.0 TiB / 1.0 TiB avail; 665 KiB/s rd, 677 KiB/s wr, 98 op/s 2024-04-16T17:12:04.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:12:04 smithi104 ceph-mon[36457]: scrub summary: idle 2024-04-16T17:12:04.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:12:04 smithi104 ceph-mon[36457]: scrub summary: idle 2024-04-16T17:12:04.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:12:04 smithi104 ceph-mon[36457]: scrub summary: idle 2024-04-16T17:12:04.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:12:04 smithi045 ceph-mon[31676]: pgmap v162: 129 pgs: 129 active+clean; 48 MiB data, 623 MiB used, 1.0 TiB / 1.0 TiB avail; 665 KiB/s rd, 677 KiB/s wr, 98 op/s 2024-04-16T17:12:04.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:12:04 smithi045 ceph-mon[31676]: scrub summary: idle 2024-04-16T17:12:04.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:12:04 smithi045 ceph-mon[31676]: scrub summary: idle 2024-04-16T17:12:04.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:12:04 smithi045 ceph-mon[31676]: scrub summary: idle 2024-04-16T17:12:05.265 INFO:tasks.workunit.client.0.smithi045.stdout: 1 83045 1.71 MB/sec execute 127 sec latency 2246.531 ms 2024-04-16T17:12:06.266 INFO:tasks.workunit.client.0.smithi045.stdout: 1 83045 1.69 MB/sec execute 128 sec latency 3246.632 ms 2024-04-16T17:12:06.804 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:12:06 smithi117 ceph-mon[36730]: pgmap v163: 129 pgs: 129 active+clean; 48 MiB data, 623 MiB used, 1.0 TiB / 1.0 TiB avail; 665 KiB/s rd, 677 KiB/s wr, 98 op/s 2024-04-16T17:12:06.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:12:06 smithi104 ceph-mon[36457]: pgmap v163: 129 pgs: 129 active+clean; 48 MiB data, 623 MiB used, 1.0 TiB / 1.0 TiB avail; 665 KiB/s rd, 677 KiB/s wr, 98 op/s 2024-04-16T17:12:06.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:12:06 smithi045 ceph-mon[31676]: pgmap v163: 129 pgs: 129 active+clean; 48 MiB data, 623 MiB used, 1.0 TiB / 1.0 TiB avail; 665 KiB/s rd, 677 KiB/s wr, 98 op/s 2024-04-16T17:12:07.266 INFO:tasks.workunit.client.0.smithi045.stdout: 1 83045 1.68 MB/sec execute 129 sec latency 4246.749 ms 2024-04-16T17:12:08.266 INFO:tasks.workunit.client.0.smithi045.stdout: 1 83045 1.67 MB/sec execute 130 sec latency 5246.859 ms 2024-04-16T17:12:08.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:12:08 smithi117 ceph-mon[36730]: pgmap v164: 129 pgs: 129 active+clean; 45 MiB data, 643 MiB used, 1.0 TiB / 1.0 TiB avail; 769 KiB/s rd, 866 KiB/s wr, 149 op/s 2024-04-16T17:12:08.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:12:08 smithi104 ceph-mon[36457]: pgmap v164: 129 pgs: 129 active+clean; 45 MiB data, 643 MiB used, 1.0 TiB / 1.0 TiB avail; 769 KiB/s rd, 866 KiB/s wr, 149 op/s 2024-04-16T17:12:08.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:12:08 smithi045 ceph-mon[31676]: pgmap v164: 129 pgs: 129 active+clean; 45 MiB data, 643 MiB used, 1.0 TiB / 1.0 TiB avail; 769 KiB/s rd, 866 KiB/s wr, 149 op/s 2024-04-16T17:12:09.266 INFO:tasks.workunit.client.0.smithi045.stdout: 1 83045 1.65 MB/sec execute 131 sec latency 6246.950 ms 2024-04-16T17:12:10.266 INFO:tasks.workunit.client.0.smithi045.stdout: 1 83045 1.64 MB/sec execute 132 sec latency 7247.034 ms 2024-04-16T17:12:10.804 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:12:10 smithi117 ceph-mon[36730]: pgmap v165: 129 pgs: 129 active+clean; 45 MiB data, 643 MiB used, 1.0 TiB / 1.0 TiB avail; 544 KiB/s rd, 418 KiB/s wr, 104 op/s 2024-04-16T17:12:10.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:12:10 smithi117 ceph-mon[36730]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-16T17:12:10.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:12:10 smithi104 ceph-mon[36457]: pgmap v165: 129 pgs: 129 active+clean; 45 MiB data, 643 MiB used, 1.0 TiB / 1.0 TiB avail; 544 KiB/s rd, 418 KiB/s wr, 104 op/s 2024-04-16T17:12:10.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:12:10 smithi104 ceph-mon[36457]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-16T17:12:10.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:12:10 smithi045 ceph-mon[31676]: pgmap v165: 129 pgs: 129 active+clean; 45 MiB data, 643 MiB used, 1.0 TiB / 1.0 TiB avail; 544 KiB/s rd, 418 KiB/s wr, 104 op/s 2024-04-16T17:12:10.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:12:10 smithi045 ceph-mon[31676]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-16T17:12:11.266 INFO:tasks.workunit.client.0.smithi045.stdout: 1 83045 1.63 MB/sec execute 133 sec latency 8247.132 ms 2024-04-16T17:12:12.266 INFO:tasks.workunit.client.0.smithi045.stdout: 1 83045 1.62 MB/sec execute 134 sec latency 9247.268 ms 2024-04-16T17:12:12.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:12:12 smithi045 ceph-mon[31676]: pgmap v166: 129 pgs: 129 active+clean; 45 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 568 KiB/s rd, 438 KiB/s wr, 110 op/s 2024-04-16T17:12:12.804 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:12:12 smithi117 ceph-mon[36730]: pgmap v166: 129 pgs: 129 active+clean; 45 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 568 KiB/s rd, 438 KiB/s wr, 110 op/s 2024-04-16T17:12:12.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:12:12 smithi104 ceph-mon[36457]: pgmap v166: 129 pgs: 129 active+clean; 45 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 568 KiB/s rd, 438 KiB/s wr, 110 op/s 2024-04-16T17:12:13.266 INFO:tasks.workunit.client.0.smithi045.stdout: 1 83045 1.61 MB/sec execute 135 sec latency 10247.406 ms 2024-04-16T17:12:14.266 INFO:tasks.workunit.client.0.smithi045.stdout: 1 83045 1.59 MB/sec execute 136 sec latency 11247.496 ms 2024-04-16T17:12:14.804 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:12:14 smithi117 ceph-mon[36730]: pgmap v167: 129 pgs: 129 active+clean; 45 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 208 KiB/s wr, 55 op/s 2024-04-16T17:12:14.823 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:14.823+0000 7f805c400640 1 -- 172.21.15.45:0/1096612787 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8054104470 msgr2=0x7f8054104850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:14.823 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:14.823+0000 7f805c400640 1 --2- 172.21.15.45:0/1096612787 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8054104470 0x7f8054104850 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f8048009be0 tx=0x7f804802bef0 comp rx=0 tx=0).stop 2024-04-16T17:12:14.823 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:14.823+0000 7f805c400640 1 -- 172.21.15.45:0/1096612787 shutdown_connections 2024-04-16T17:12:14.823 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:14.823+0000 7f805c400640 1 --2- 172.21.15.45:0/1096612787 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8054109540 0x7f8054111110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:14.823 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:14.823+0000 7f805c400640 1 --2- 172.21.15.45:0/1096612787 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8054104e20 0x7f8054108e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:14.823 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:14.823+0000 7f805c400640 1 --2- 172.21.15.45:0/1096612787 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8054104470 0x7f8054104850 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:14.823 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:14.823+0000 7f805c400640 1 -- 172.21.15.45:0/1096612787 >> 172.21.15.45:0/1096612787 conn(0x7f80540ffcd0 msgr2=0x7f80541020f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:12:14.823 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:14.823+0000 7f805c400640 1 -- 172.21.15.45:0/1096612787 shutdown_connections 2024-04-16T17:12:14.823 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:14.823+0000 7f805c400640 1 -- 172.21.15.45:0/1096612787 wait complete. 2024-04-16T17:12:14.826 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:14.824+0000 7f805c400640 1 Processor -- start 2024-04-16T17:12:14.826 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:14.824+0000 7f805c400640 1 -- start start 2024-04-16T17:12:14.826 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:14.825+0000 7f805c400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8054104470 0x7f80541a06c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:14.826 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:14.825+0000 7f805c400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8054109540 0x7f80541a0c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:14.826 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:14.825+0000 7f805c400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f805419a870 0x7f805419ad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:14.826 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:14.825+0000 7f805c400640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f8054114650 con 0x7f805419a870 2024-04-16T17:12:14.826 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:14.825+0000 7f805c400640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f80541144d0 con 0x7f8054109540 2024-04-16T17:12:14.826 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:14.825+0000 7f805c400640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f80541147d0 con 0x7f8054104470 2024-04-16T17:12:14.826 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:14.825+0000 7f8059000640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8054109540 0x7f80541a0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:14.826 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:14.825+0000 7f805a400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f805419a870 0x7f805419ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:14.826 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:14.825+0000 7f8059000640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8054109540 0x7f80541a0c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:35312/0 (socket says 172.21.15.45:35312) 2024-04-16T17:12:14.826 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:14.825+0000 7f8059000640 1 -- 172.21.15.45:0/3625543233 learned_addr learned my addr 172.21.15.45:0/3625543233 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:12:14.827 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:14.825+0000 7f805a400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f805419a870 0x7f805419ad20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56472/0 (socket says 172.21.15.45:56472) 2024-04-16T17:12:14.827 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:14.825+0000 7f8059a00640 1 --2- 172.21.15.45:0/3625543233 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8054104470 0x7f80541a06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:14.827 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:14.825+0000 7f8059000640 1 -- 172.21.15.45:0/3625543233 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8054104470 msgr2=0x7f80541a06c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:14.827 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:14.825+0000 7f8059000640 1 --2- 172.21.15.45:0/3625543233 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8054104470 0x7f80541a06c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:14.827 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:14.825+0000 7f8059000640 1 -- 172.21.15.45:0/3625543233 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f805419a870 msgr2=0x7f805419ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:14.827 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:14.825+0000 7f8059000640 1 --2- 172.21.15.45:0/3625543233 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f805419a870 0x7f805419ad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:14.827 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:14.825+0000 7f8059000640 1 -- 172.21.15.45:0/3625543233 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f805419b390 con 0x7f8054109540 2024-04-16T17:12:14.827 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:14.825+0000 7f805a400640 1 --2- 172.21.15.45:0/3625543233 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f805419a870 0x7f805419ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:12:14.827 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:14.826+0000 7f8059000640 1 --2- 172.21.15.45:0/3625543233 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8054109540 0x7f80541a0c00 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f804000e9f0 tx=0x7f804000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:12:14.827 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:14.826+0000 7f8052000640 1 -- 172.21.15.45:0/3625543233 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f804000ccc0 con 0x7f8054109540 2024-04-16T17:12:14.827 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:14.826+0000 7f8052000640 1 -- 172.21.15.45:0/3625543233 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8040010910 con 0x7f8054109540 2024-04-16T17:12:14.827 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:14.826+0000 7f8052000640 1 -- 172.21.15.45:0/3625543233 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8040018960 con 0x7f8054109540 2024-04-16T17:12:14.827 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:14.826+0000 7f805c400640 1 -- 172.21.15.45:0/3625543233 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f805419b680 con 0x7f8054109540 2024-04-16T17:12:14.827 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:14.826+0000 7f805c400640 1 -- 172.21.15.45:0/3625543233 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80541a7920 con 0x7f8054109540 2024-04-16T17:12:14.829 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:14.829+0000 7f805f185740 1 Processor -- start 2024-04-16T17:12:14.829 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:14.829+0000 7f805f185740 1 -- start start 2024-04-16T17:12:14.830 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:14.830+0000 7f805f185740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5611ccda2610 0x5611ccda1360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:14.830 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:14.830+0000 7f805f185740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5611ccda18a0 0x5611ccda1d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:14.830 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:14.830+0000 7f805f185740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5611ccdaae20 0x5611ccdad210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:14.830 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:14.830+0000 7f8052000640 1 -- 172.21.15.45:0/3625543233 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f8040010ac0 con 0x7f8054109540 2024-04-16T17:12:14.830 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:14.830+0000 7f805f185740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5611ccd06780 con 0x5611ccda2610 2024-04-16T17:12:14.830 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:14.830+0000 7f805f185740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5611cccacf40 con 0x5611ccdaae20 2024-04-16T17:12:14.830 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:14.830+0000 7f805f185740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5611ccd02740 con 0x5611ccda18a0 2024-04-16T17:12:14.830 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:14.830+0000 7f805a400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5611ccda2610 0x5611ccda1360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:14.831 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:14.830+0000 7f8059a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5611ccda18a0 0x5611ccda1d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:14.831 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:14.830+0000 7f8059000640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5611ccdaae20 0x5611ccdad210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:14.831 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:14.830+0000 7f805a400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5611ccda2610 0x5611ccda1360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56476/0 (socket says 172.21.15.45:56476) 2024-04-16T17:12:14.831 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:14.830+0000 7f805a400640 1 -- 172.21.15.45:0/3534925182 learned_addr learned my addr 172.21.15.45:0/3534925182 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:12:14.831 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:14.831+0000 7f8059a00640 1 -- 172.21.15.45:0/3534925182 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5611ccdaae20 msgr2=0x5611ccdad210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:12:14.831 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:14.831+0000 7f8059a00640 1 --2- 172.21.15.45:0/3534925182 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5611ccdaae20 0x5611ccdad210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:14.831 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:14.831+0000 7f8059a00640 1 -- 172.21.15.45:0/3534925182 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5611ccda2610 msgr2=0x5611ccda1360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:12:14.831 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:14.831+0000 7f8059a00640 1 --2- 172.21.15.45:0/3534925182 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5611ccda2610 0x5611ccda1360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:14.831 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:14.831+0000 7f8059a00640 1 -- 172.21.15.45:0/3534925182 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5611cca19210 con 0x5611ccda18a0 2024-04-16T17:12:14.833 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:14.831+0000 7f8059a00640 1 --2- 172.21.15.45:0/3534925182 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5611ccda18a0 0x5611ccda1d20 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f804802c500 tx=0x7f8048005ec0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7f297462788bd6a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:12:14.833 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:14.831+0000 7f8046a00640 1 -- 172.21.15.45:0/3534925182 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8048043030 con 0x5611ccda18a0 2024-04-16T17:12:14.833 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:14.831+0000 7f8046a00640 1 -- 172.21.15.45:0/3534925182 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8048033bc0 con 0x5611ccda18a0 2024-04-16T17:12:14.833 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:14.831+0000 7f8046a00640 1 -- 172.21.15.45:0/3534925182 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f804802c890 con 0x5611ccda18a0 2024-04-16T17:12:14.833 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:14.832+0000 7f8052000640 1 --2- 172.21.15.45:0/3625543233 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f803c0a3e40 0x7f803c0a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:14.833 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:14.832+0000 7f805f185740 1 -- 172.21.15.45:0/3534925182 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5611ccda18a0 msgr2=0x5611ccda1d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:14.833 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:14.832+0000 7f805f185740 1 --2- 172.21.15.45:0/3534925182 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5611ccda18a0 0x5611ccda1d20 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f804802c500 tx=0x7f8048005ec0 comp rx=0 tx=0).stop 2024-04-16T17:12:14.833 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:14.832+0000 7f8052000640 1 -- 172.21.15.45:0/3625543233 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f80400d0df0 con 0x7f8054109540 2024-04-16T17:12:14.833 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:14.832+0000 7f805a400640 1 --2- 172.21.15.45:0/3625543233 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f803c0a3e40 0x7f803c0a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:14.833 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:14.832+0000 7f805f185740 1 -- 172.21.15.45:0/3534925182 shutdown_connections 2024-04-16T17:12:14.833 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:14.832+0000 7f805f185740 1 --2- 172.21.15.45:0/3534925182 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5611ccda18a0 0x5611ccda1d20 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:14.834 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:14.832+0000 7f805f185740 1 --2- 172.21.15.45:0/3534925182 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5611ccdaae20 0x5611ccdad210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:14.834 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:14.832+0000 7f805f185740 1 --2- 172.21.15.45:0/3534925182 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5611ccda2610 0x5611ccda1360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:14.834 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:14.832+0000 7f805f185740 1 -- 172.21.15.45:0/3534925182 >> 172.21.15.45:0/3534925182 conn(0x5611ccd3c5d0 msgr2=0x5611ccd9c590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:12:14.834 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:14.833+0000 7f805f185740 1 -- 172.21.15.45:0/3534925182 shutdown_connections 2024-04-16T17:12:14.834 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:14.833+0000 7f805f185740 1 -- 172.21.15.45:0/3534925182 wait complete. 2024-04-16T17:12:14.834 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:14.833+0000 7f805a400640 1 --2- 172.21.15.45:0/3625543233 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f803c0a3e40 0x7f803c0a6300 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f804c005f60 tx=0x7f804c005f90 comp rx=0 tx=0).ready entity=mgr.25411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:12:14.834 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:14.833+0000 7f805f185740 1 -- start start 2024-04-16T17:12:14.834 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:14.833+0000 7f805f185740 1 Processor -- start 2024-04-16T17:12:14.838 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:14.834+0000 7f805f185740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5611ccda18a0 0x5611ccdaa890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:14.838 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:14.834+0000 7f805f185740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5611ccda2610 0x5611ccda7c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:14.838 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:14.834+0000 7f805f185740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5611ccdaae20 0x5611ccda8360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:14.838 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:14.834+0000 7f805f185740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5611ccdadaf0 con 0x5611ccdaae20 2024-04-16T17:12:14.838 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:14.834+0000 7f805f185740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5611cccc0390 con 0x5611ccda18a0 2024-04-16T17:12:14.838 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:14.834+0000 7f805f185740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5611ccdadc70 con 0x5611ccda2610 2024-04-16T17:12:14.839 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:14.834+0000 7f805f185740 10 client.0 _ll_register_callbacks cb 0x5611ccca9f60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:12:14.839 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:14.834+0000 7f805a400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5611ccdaae20 0x5611ccda8360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:14.839 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:14.834+0000 7f805a400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5611ccdaae20 0x5611ccda8360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56486/0 (socket says 172.21.15.45:56486) 2024-04-16T17:12:14.839 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:14.834+0000 7f805a400640 1 -- 172.21.15.45:0/28309432 learned_addr learned my addr 172.21.15.45:0/28309432 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:12:14.839 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:14.834+0000 7f8059000640 1 --2- 172.21.15.45:0/28309432 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5611ccda2610 0x5611ccda7c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:14.839 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:14.834+0000 7f8059a00640 1 --2- 172.21.15.45:0/28309432 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5611ccda18a0 0x5611ccdaa890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:14.839 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:14.835+0000 7f805a400640 1 -- 172.21.15.45:0/28309432 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5611ccda2610 msgr2=0x5611ccda7c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:12:14.839 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:14.835+0000 7f805a400640 1 --2- 172.21.15.45:0/28309432 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5611ccda2610 0x5611ccda7c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:14.839 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:14.835+0000 7f805a400640 1 -- 172.21.15.45:0/28309432 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5611ccda18a0 msgr2=0x5611ccdaa890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:12:14.839 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:14.835+0000 7f805a400640 1 --2- 172.21.15.45:0/28309432 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5611ccda18a0 0x5611ccdaa890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:14.839 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:14.835+0000 7f805a400640 1 -- 172.21.15.45:0/28309432 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5611ccae5620 con 0x5611ccdaae20 2024-04-16T17:12:14.839 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:14.835+0000 7f8059a00640 1 --2- 172.21.15.45:0/28309432 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5611ccda18a0 0x5611ccdaa890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:12:14.839 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:14.835+0000 7f805a400640 1 --2- 172.21.15.45:0/28309432 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5611ccdaae20 0x5611ccda8360 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f804c00fb20 tx=0x7f804c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5db29618f86f94ef server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:12:14.840 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:14.835+0000 7f803b400640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-16T17:12:14.840 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:14.835+0000 7f803b400640 1 -- 172.21.15.45:0/28309432 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f804c012020 con 0x5611ccdaae20 2024-04-16T17:12:14.840 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:14.835+0000 7f803b400640 1 -- 172.21.15.45:0/28309432 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f804c018030 con 0x5611ccdaae20 2024-04-16T17:12:14.840 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:14.835+0000 7f805f185740 1 -- 172.21.15.45:0/28309432 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8048033bc0 con 0x5611ccdaae20 2024-04-16T17:12:14.840 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:14.836+0000 7f803b400640 1 -- 172.21.15.45:0/28309432 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f804c034050 con 0x5611ccdaae20 2024-04-16T17:12:14.840 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:14.836+0000 7f805f185740 10 client.16965 fetch_fsmap learned FSMap version 37 2024-04-16T17:12:14.840 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:14.836+0000 7f805f185740 1 -- 172.21.15.45:0/28309432 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x5611ccdb9c20 con 0x5611ccdaae20 2024-04-16T17:12:14.840 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:14.836+0000 7f803b400640 1 -- 172.21.15.45:0/28309432 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8048033bc0 con 0x5611ccdaae20 2024-04-16T17:12:14.840 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:14.836+0000 7f803b400640 1 -- 172.21.15.45:0/28309432 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 37) ==== 8358+0+0 (secure 0 0 0) 0x7f804c011460 con 0x5611ccdaae20 2024-04-16T17:12:14.840 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:14.836+0000 7f805f185740 10 client.16965 fetch_fsmap finished waiting for FSMap version 37 2024-04-16T17:12:14.840 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:14.836+0000 7f805f185740 10 client.16965 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T17:12:14.840 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:14.836+0000 7f805f185740 20 client.16965 populate_metadata read hostname 'smithi045' 2024-04-16T17:12:14.840 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:14.836+0000 7f805f185740 1 --2- 172.21.15.45:0/28309432 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x5611ccdbb0e0 0x5611ccdbd550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:14.840 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:14.836+0000 7f805f185740 4 client.16965 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:12:14.841 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:14.836+0000 7f805f185740 1 -- 172.21.15.45:0/28309432 --> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5611ccd141a0 con 0x5611ccdbb0e0 2024-04-16T17:12:14.841 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:14.837+0000 7f8059a00640 1 --2- 172.21.15.45:0/28309432 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x5611ccdbb0e0 0x5611ccdbd550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:14.841 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:14.837+0000 7f8059a00640 1 --2- 172.21.15.45:0/28309432 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x5611ccdbb0e0 0x5611ccdbd550 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=47b6d1a6c8221dc9 server_cookie=468902ff0024a9a9 in_seq=0 out_seq=0 2024-04-16T17:12:14.841 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:14.837+0000 7f803b400640 10 client.16965 ms_handle_connect on v2:172.21.15.104:6838/3660070592 2024-04-16T17:12:14.841 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:14.838+0000 7f803b400640 1 -- 172.21.15.45:0/28309432 <== mds.0 v2:172.21.15.104:6838/3660070592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5611ccd141a0 con 0x5611ccdbb0e0 2024-04-16T17:12:14.841 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:14.838+0000 7f803b400640 10 client.16965 handle_command_reply: tid=0 2024-04-16T17:12:14.841 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:14.838+0000 7f805f185740 1 client.16965 shutdown 2024-04-16T17:12:14.841 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:14.838+0000 7f805f185740 1 -- 172.21.15.45:0/28309432 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5611ccdaae20 msgr2=0x5611ccda8360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:14.841 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:14.838+0000 7f805f185740 1 --2- 172.21.15.45:0/28309432 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5611ccdaae20 0x5611ccda8360 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f804c00fb20 tx=0x7f804c004730 comp rx=0 tx=0).stop 2024-04-16T17:12:14.841 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:14.838+0000 7f805f185740 1 -- 172.21.15.45:0/28309432 shutdown_connections 2024-04-16T17:12:14.841 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:14.838+0000 7f805f185740 1 --2- 172.21.15.45:0/28309432 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x5611ccdbb0e0 0x5611ccdbd550 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:14.841 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:14.838+0000 7f805f185740 1 --2- 172.21.15.45:0/28309432 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5611ccda2610 0x5611ccda7c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:14.841 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:14.838+0000 7f805f185740 1 --2- 172.21.15.45:0/28309432 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5611ccdaae20 0x5611ccda8360 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:14.842 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:14.838+0000 7f805f185740 1 --2- 172.21.15.45:0/28309432 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5611ccda18a0 0x5611ccdaa890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:14.842 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:14.838+0000 7f803b400640 0 client.16965 ms_handle_reset on v2:172.21.15.104:6838/3660070592 2024-04-16T17:12:14.842 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:14.838+0000 7f805f185740 1 -- 172.21.15.45:0/28309432 >> 172.21.15.45:0/28309432 conn(0x5611ccd3c5d0 msgr2=0x5611ccd9e2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:12:14.842 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:14.839+0000 7f805f185740 1 -- 172.21.15.45:0/28309432 shutdown_connections 2024-04-16T17:12:14.842 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:14.839+0000 7f805f185740 1 -- 172.21.15.45:0/28309432 wait complete. 2024-04-16T17:12:14.842 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:14.839+0000 7f805f185740 20 client.16965 trim_cache size 0 max 16384 2024-04-16T17:12:14.856 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:14.855+0000 7f805f185740 1 Processor -- start 2024-04-16T17:12:14.856 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:14.855+0000 7f805f185740 1 -- start start 2024-04-16T17:12:14.856 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:14.855+0000 7f805f185740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5611ccda18a0 0x5611ccdb69f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:14.856 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:14.855+0000 7f805f185740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5611ccda2610 0x5611ccdaa350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:14.856 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:14.855+0000 7f805f185740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5611ccdaae20 0x5611ccdaa890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:14.856 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:14.855+0000 7f805f185740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f80541b3f20 con 0x5611ccda2610 2024-04-16T17:12:14.856 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:14.856+0000 7f805f185740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f80541b1760 con 0x5611ccda18a0 2024-04-16T17:12:14.857 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:14.856+0000 7f805f185740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f80541af590 con 0x5611ccdaae20 2024-04-16T17:12:14.857 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:14.856+0000 7f8059000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5611ccda2610 0x5611ccdaa350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:14.857 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:14.856+0000 7f8059000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5611ccda2610 0x5611ccdaa350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56494/0 (socket says 172.21.15.45:56494) 2024-04-16T17:12:14.857 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:14.856+0000 7f8059000640 1 -- 172.21.15.45:0/174853263 learned_addr learned my addr 172.21.15.45:0/174853263 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:12:14.857 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:14.856+0000 7f805a400640 1 --2- 172.21.15.45:0/174853263 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5611ccdaae20 0x5611ccdaa890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:14.857 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:14.856+0000 7f8059a00640 1 --2- 172.21.15.45:0/174853263 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5611ccda18a0 0x5611ccdb69f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:14.857 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:14.856+0000 7f8059000640 1 -- 172.21.15.45:0/174853263 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5611ccdaae20 msgr2=0x5611ccdaa890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:12:14.857 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:14.856+0000 7f8059000640 1 --2- 172.21.15.45:0/174853263 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5611ccdaae20 0x5611ccdaa890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:14.857 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:14.856+0000 7f8059000640 1 -- 172.21.15.45:0/174853263 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5611ccda18a0 msgr2=0x5611ccdb69f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:12:14.857 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:14.856+0000 7f8059000640 1 --2- 172.21.15.45:0/174853263 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5611ccda18a0 0x5611ccdb69f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:14.857 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:14.856+0000 7f8059000640 1 -- 172.21.15.45:0/174853263 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5611ccda0b70 con 0x5611ccda2610 2024-04-16T17:12:14.857 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:14.856+0000 7f8059000640 1 --2- 172.21.15.45:0/174853263 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5611ccda2610 0x5611ccdaa350 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f80400d4640 tx=0x7f80400bd040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a01dccc9886b002d server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:12:14.857 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:14.857+0000 7f8046000640 1 -- 172.21.15.45:0/174853263 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f804009c3f0 con 0x5611ccda2610 2024-04-16T17:12:14.857 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:14.857+0000 7f8046000640 1 -- 172.21.15.45:0/174853263 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f804009ca90 con 0x5611ccda2610 2024-04-16T17:12:14.858 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:14.857+0000 7f8046000640 1 -- 172.21.15.45:0/174853263 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80400ce410 con 0x5611ccda2610 2024-04-16T17:12:14.861 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:14.857+0000 7f805f185740 1 -- 172.21.15.45:0/174853263 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5611ccda2610 msgr2=0x5611ccdaa350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:14.861 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:14.857+0000 7f805f185740 1 --2- 172.21.15.45:0/174853263 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5611ccda2610 0x5611ccdaa350 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f80400d4640 tx=0x7f80400bd040 comp rx=0 tx=0).stop 2024-04-16T17:12:14.861 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:14.857+0000 7f805f185740 1 -- 172.21.15.45:0/174853263 shutdown_connections 2024-04-16T17:12:14.861 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:14.857+0000 7f805f185740 1 --2- 172.21.15.45:0/174853263 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5611ccdaae20 0x5611ccdaa890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:14.861 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:14.857+0000 7f805f185740 1 --2- 172.21.15.45:0/174853263 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5611ccda2610 0x5611ccdaa350 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:14.862 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:14.857+0000 7f805f185740 1 --2- 172.21.15.45:0/174853263 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5611ccda18a0 0x5611ccdb69f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:14.862 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:14.857+0000 7f805f185740 1 -- 172.21.15.45:0/174853263 >> 172.21.15.45:0/174853263 conn(0x5611ccd3c5d0 msgr2=0x5611ccd9c0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:12:14.862 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:14.857+0000 7f805f185740 1 -- 172.21.15.45:0/174853263 shutdown_connections 2024-04-16T17:12:14.862 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:14.858+0000 7f805f185740 1 -- 172.21.15.45:0/174853263 wait complete. 2024-04-16T17:12:14.862 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:14.858+0000 7f805f185740 1 -- start start 2024-04-16T17:12:14.862 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:14.858+0000 7f805f185740 1 Processor -- start 2024-04-16T17:12:14.862 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:14.859+0000 7f805f185740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5611ccda18a0 0x5611ccdac1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:14.862 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:14.859+0000 7f805f185740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5611ccdaae20 0x5611ccdac700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:14.862 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:14.859+0000 7f805f185740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5611ccdacc40 0x5611ccdb1260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:14.862 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:14.859+0000 7f805f185740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f80541afc20 con 0x5611ccdacc40 2024-04-16T17:12:14.862 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:14.859+0000 7f805f185740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f80541b1090 con 0x5611ccdaae20 2024-04-16T17:12:14.862 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:14.859+0000 7f805f185740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f80541b02f0 con 0x5611ccda18a0 2024-04-16T17:12:14.863 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:14.859+0000 7f805f185740 10 client.0 _ll_register_callbacks cb 0x7f804c009c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:12:14.863 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:14.859+0000 7f805a400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5611ccdacc40 0x5611ccdb1260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:14.863 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:14.859+0000 7f805a400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5611ccdacc40 0x5611ccdb1260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56498/0 (socket says 172.21.15.45:56498) 2024-04-16T17:12:14.863 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:14.859+0000 7f805a400640 1 -- 172.21.15.45:0/747853087 learned_addr learned my addr 172.21.15.45:0/747853087 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:12:14.863 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:14.859+0000 7f8059a00640 1 --2- 172.21.15.45:0/747853087 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5611ccda18a0 0x5611ccdac1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:14.863 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:14.859+0000 7f8059000640 1 --2- 172.21.15.45:0/747853087 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5611ccdaae20 0x5611ccdac700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:14.863 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:14.859+0000 7f805a400640 1 -- 172.21.15.45:0/747853087 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5611ccda18a0 msgr2=0x5611ccdac1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:12:14.863 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:14.859+0000 7f805a400640 1 --2- 172.21.15.45:0/747853087 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5611ccda18a0 0x5611ccdac1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:14.863 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:14.859+0000 7f805a400640 1 -- 172.21.15.45:0/747853087 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5611ccdaae20 msgr2=0x5611ccdac700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:12:14.863 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:14.859+0000 7f805a400640 1 --2- 172.21.15.45:0/747853087 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5611ccdaae20 0x5611ccdac700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:14.863 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:14.859+0000 7f805a400640 1 -- 172.21.15.45:0/747853087 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5611ccdad680 con 0x5611ccdacc40 2024-04-16T17:12:14.863 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:14.859+0000 7f8059a00640 1 --2- 172.21.15.45:0/747853087 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5611ccda18a0 0x5611ccdac1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:12:14.863 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:14.860+0000 7f805a400640 1 --2- 172.21.15.45:0/747853087 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5611ccdacc40 0x5611ccdb1260 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f804c000c00 tx=0x7f804c011670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c933e8a1b925918 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:12:14.864 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:14.860+0000 7f803b400640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-16T17:12:14.864 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:14.860+0000 7f803b400640 1 -- 172.21.15.45:0/747853087 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f804c03e020 con 0x5611ccdacc40 2024-04-16T17:12:14.864 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:14.860+0000 7f803b400640 1 -- 172.21.15.45:0/747853087 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f804c01e250 con 0x5611ccdacc40 2024-04-16T17:12:14.864 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:14.860+0000 7f805f185740 1 -- 172.21.15.45:0/747853087 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f804009ca90 con 0x5611ccdacc40 2024-04-16T17:12:14.864 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:14.860+0000 7f803b400640 1 -- 172.21.15.45:0/747853087 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f804c049050 con 0x5611ccdacc40 2024-04-16T17:12:14.864 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:14.860+0000 7f805f185740 10 client.16971 fetch_fsmap learned FSMap version 37 2024-04-16T17:12:14.864 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:14.860+0000 7f805f185740 1 -- 172.21.15.45:0/747853087 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x5611cce21d80 con 0x5611ccdacc40 2024-04-16T17:12:14.864 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:14.861+0000 7f803b400640 1 -- 172.21.15.45:0/747853087 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f804009ca90 con 0x5611ccdacc40 2024-04-16T17:12:14.864 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:14.861+0000 7f803b400640 1 -- 172.21.15.45:0/747853087 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 37) ==== 8358+0+0 (secure 0 0 0) 0x7f804c011d00 con 0x5611ccdacc40 2024-04-16T17:12:14.864 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:14.861+0000 7f805f185740 10 client.16971 fetch_fsmap finished waiting for FSMap version 37 2024-04-16T17:12:14.864 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:14.861+0000 7f805f185740 10 client.16971 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T17:12:14.864 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:14.861+0000 7f805f185740 20 client.16971 populate_metadata read hostname 'smithi045' 2024-04-16T17:12:14.865 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:14.861+0000 7f805f185740 1 --2- 172.21.15.45:0/747853087 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x5611cce22550 0x5611cce24970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:14.865 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:14.861+0000 7f805f185740 4 client.16971 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-16T17:12:14.865 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:14.861+0000 7f805f185740 1 -- 172.21.15.45:0/747853087 --> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5611ccd0e010 con 0x5611cce22550 2024-04-16T17:12:14.865 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:14.861+0000 7f8059a00640 1 --2- 172.21.15.45:0/747853087 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x5611cce22550 0x5611cce24970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:14.865 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:14.862+0000 7f8059a00640 1 --2- 172.21.15.45:0/747853087 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x5611cce22550 0x5611cce24970 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e3ff62fc977d8e7e server_cookie=532b3139a4ae0307 in_seq=0 out_seq=0 2024-04-16T17:12:14.865 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:14.862+0000 7f803b400640 10 client.16971 ms_handle_connect on v2:172.21.15.104:6838/3660070592 2024-04-16T17:12:14.866 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:12:14 smithi045 ceph-mon[31676]: pgmap v167: 129 pgs: 129 active+clean; 45 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 208 KiB/s wr, 55 op/s 2024-04-16T17:12:14.894 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:14.893+0000 7f803b400640 1 -- 172.21.15.45:0/747853087 <== mds.0 v2:172.21.15.104:6838/3660070592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5611ccd0e010 con 0x5611cce22550 2024-04-16T17:12:14.915 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:14.893+0000 7f803b400640 10 client.16971 handle_command_reply: tid=0 2024-04-16T17:12:14.915 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:14.893+0000 7f805f185740 1 client.16971 shutdown 2024-04-16T17:12:14.915 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:14.894+0000 7f805f185740 1 -- 172.21.15.45:0/747853087 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5611ccdacc40 msgr2=0x5611ccdb1260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:14.915 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:14.894+0000 7f805f185740 1 --2- 172.21.15.45:0/747853087 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5611ccdacc40 0x5611ccdb1260 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f804c000c00 tx=0x7f804c011670 comp rx=0 tx=0).stop 2024-04-16T17:12:14.915 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:14.894+0000 7f805f185740 1 -- 172.21.15.45:0/747853087 shutdown_connections 2024-04-16T17:12:14.915 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:14.894+0000 7f805f185740 1 --2- 172.21.15.45:0/747853087 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5611ccdacc40 0x5611ccdb1260 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:14.915 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:14.894+0000 7f805f185740 1 --2- 172.21.15.45:0/747853087 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5611ccdaae20 0x5611ccdac700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:14.915 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:14.894+0000 7f805f185740 1 --2- 172.21.15.45:0/747853087 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x5611cce22550 0x5611cce24970 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:14.915 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:14.894+0000 7f805f185740 1 --2- 172.21.15.45:0/747853087 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5611ccda18a0 0x5611ccdac1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:14.916 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:14.894+0000 7f803b400640 0 client.16971 ms_handle_reset on v2:172.21.15.104:6838/3660070592 2024-04-16T17:12:14.916 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:14.894+0000 7f805f185740 1 -- 172.21.15.45:0/747853087 >> 172.21.15.45:0/747853087 conn(0x5611ccd3c5d0 msgr2=0x5611ccd9ae20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:12:14.916 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:14.894+0000 7f805f185740 1 -- 172.21.15.45:0/747853087 shutdown_connections 2024-04-16T17:12:14.916 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:14.894+0000 7f805f185740 1 -- 172.21.15.45:0/747853087 wait complete. 2024-04-16T17:12:14.916 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:14.894+0000 7f805f185740 20 client.16971 trim_cache size 0 max 16384 2024-04-16T17:12:14.916 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-04-16T17:12:14.916 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-04-16T17:12:14.916 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-04-16T17:12:14.916 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-04-16T17:12:14.917 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:14.895+0000 7f805c400640 1 -- 172.21.15.45:0/3625543233 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f803c0a3e40 msgr2=0x7f803c0a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:14.917 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:14.895+0000 7f805c400640 1 --2- 172.21.15.45:0/3625543233 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f803c0a3e40 0x7f803c0a6300 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f804c005f60 tx=0x7f804c005f90 comp rx=0 tx=0).stop 2024-04-16T17:12:14.917 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:14.895+0000 7f805c400640 1 -- 172.21.15.45:0/3625543233 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8054109540 msgr2=0x7f80541a0c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:14.917 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:14.895+0000 7f805c400640 1 --2- 172.21.15.45:0/3625543233 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8054109540 0x7f80541a0c00 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f804000e9f0 tx=0x7f804000eeb0 comp rx=0 tx=0).stop 2024-04-16T17:12:14.917 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:14.895+0000 7f805c400640 1 -- 172.21.15.45:0/3625543233 shutdown_connections 2024-04-16T17:12:14.917 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:14.895+0000 7f805c400640 1 --2- 172.21.15.45:0/3625543233 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f803c0a3e40 0x7f803c0a6300 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:14.917 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:14.895+0000 7f805c400640 1 --2- 172.21.15.45:0/3625543233 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f805419a870 0x7f805419ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:14.917 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:14.895+0000 7f805c400640 1 --2- 172.21.15.45:0/3625543233 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8054109540 0x7f80541a0c00 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:14.918 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:14.895+0000 7f805c400640 1 --2- 172.21.15.45:0/3625543233 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8054104470 0x7f80541a06c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:14.918 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:14.895+0000 7f805c400640 1 -- 172.21.15.45:0/3625543233 >> 172.21.15.45:0/3625543233 conn(0x7f80540ffcd0 msgr2=0x7f80541007e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:12:14.918 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:14.896+0000 7f805c400640 1 -- 172.21.15.45:0/3625543233 shutdown_connections 2024-04-16T17:12:14.918 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:14.896+0000 7f805c400640 1 -- 172.21.15.45:0/3625543233 wait complete. 2024-04-16T17:12:14.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:12:14 smithi104 ceph-mon[36457]: pgmap v167: 129 pgs: 129 active+clean; 45 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 208 KiB/s wr, 55 op/s 2024-04-16T17:12:15.266 INFO:tasks.workunit.client.0.smithi045.stdout: 1 83045 1.58 MB/sec execute 137 sec latency 12247.575 ms 2024-04-16T17:12:16.267 INFO:tasks.workunit.client.0.smithi045.stdout: 1 83045 1.57 MB/sec execute 138 sec latency 13247.700 ms 2024-04-16T17:12:16.804 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:12:16 smithi117 ceph-mon[36730]: pgmap v168: 129 pgs: 129 active+clean; 45 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 208 KiB/s wr, 55 op/s 2024-04-16T17:12:16.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:12:16 smithi104 ceph-mon[36457]: pgmap v168: 129 pgs: 129 active+clean; 45 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 208 KiB/s wr, 55 op/s 2024-04-16T17:12:16.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:12:16 smithi045 ceph-mon[31676]: pgmap v168: 129 pgs: 129 active+clean; 45 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 208 KiB/s wr, 55 op/s 2024-04-16T17:12:17.267 INFO:tasks.workunit.client.0.smithi045.stdout: 1 83045 1.56 MB/sec execute 139 sec latency 14247.761 ms 2024-04-16T17:12:18.267 INFO:tasks.workunit.client.0.smithi045.stdout: 1 83045 1.55 MB/sec execute 140 sec latency 15247.857 ms 2024-04-16T17:12:18.804 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:12:18 smithi117 ceph-mon[36730]: pgmap v169: 129 pgs: 129 active+clean; 48 MiB data, 649 MiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 464 KiB/s wr, 69 op/s 2024-04-16T17:12:18.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:12:18 smithi104 ceph-mon[36457]: pgmap v169: 129 pgs: 129 active+clean; 48 MiB data, 649 MiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 464 KiB/s wr, 69 op/s 2024-04-16T17:12:18.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:12:18 smithi045 ceph-mon[31676]: pgmap v169: 129 pgs: 129 active+clean; 48 MiB data, 649 MiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 464 KiB/s wr, 69 op/s 2024-04-16T17:12:19.267 INFO:tasks.workunit.client.0.smithi045.stdout: 1 83045 1.54 MB/sec execute 141 sec latency 16247.957 ms 2024-04-16T17:12:20.267 INFO:tasks.workunit.client.0.smithi045.stdout: 1 83045 1.53 MB/sec execute 142 sec latency 17248.079 ms 2024-04-16T17:12:20.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:12:20 smithi117 ceph-mon[36730]: pgmap v170: 129 pgs: 129 active+clean; 48 MiB data, 649 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 276 KiB/s wr, 19 op/s 2024-04-16T17:12:20.883 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:12:20 smithi045 ceph-mon[31676]: pgmap v170: 129 pgs: 129 active+clean; 48 MiB data, 649 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 276 KiB/s wr, 19 op/s 2024-04-16T17:12:20.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:12:20 smithi104 ceph-mon[36457]: pgmap v170: 129 pgs: 129 active+clean; 48 MiB data, 649 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 276 KiB/s wr, 19 op/s 2024-04-16T17:12:21.267 INFO:tasks.workunit.client.0.smithi045.stdout: 1 83193 1.52 MB/sec execute 143 sec latency 17737.466 ms 2024-04-16T17:12:22.267 INFO:tasks.workunit.client.0.smithi045.stdout: 1 83503 1.51 MB/sec execute 144 sec latency 52.779 ms 2024-04-16T17:12:22.804 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:12:22 smithi117 ceph-mon[36730]: pgmap v171: 129 pgs: 129 active+clean; 49 MiB data, 649 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 295 KiB/s wr, 25 op/s 2024-04-16T17:12:22.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:12:22 smithi104 ceph-mon[36457]: pgmap v171: 129 pgs: 129 active+clean; 49 MiB data, 649 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 295 KiB/s wr, 25 op/s 2024-04-16T17:12:23.267 INFO:tasks.workunit.client.0.smithi045.stdout: 1 83793 1.50 MB/sec execute 145 sec latency 66.937 ms 2024-04-16T17:12:24.268 INFO:tasks.workunit.client.0.smithi045.stdout: 1 85139 1.57 MB/sec execute 146 sec latency 15.542 ms 2024-04-16T17:12:24.804 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:12:24 smithi117 ceph-mon[36730]: pgmap v172: 129 pgs: 129 active+clean; 49 MiB data, 649 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 275 KiB/s wr, 20 op/s 2024-04-16T17:12:24.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:12:24 smithi104 ceph-mon[36457]: pgmap v172: 129 pgs: 129 active+clean; 49 MiB data, 649 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 275 KiB/s wr, 20 op/s 2024-04-16T17:12:24.966 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:12:22 smithi045 ceph-mon[31676]: pgmap v171: 129 pgs: 129 active+clean; 49 MiB data, 649 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 295 KiB/s wr, 25 op/s 2024-04-16T17:12:24.967 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:12:24 smithi045 ceph-mon[31676]: pgmap v172: 129 pgs: 129 active+clean; 49 MiB data, 649 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 275 KiB/s wr, 20 op/s 2024-04-16T17:12:25.267 INFO:tasks.workunit.client.0.smithi045.stdout: 1 85575 1.57 MB/sec execute 147 sec latency 20.946 ms 2024-04-16T17:12:26.268 INFO:tasks.workunit.client.0.smithi045.stdout: 1 85898 1.56 MB/sec execute 148 sec latency 62.755 ms 2024-04-16T17:12:26.804 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:12:26 smithi117 ceph-mon[36730]: pgmap v173: 129 pgs: 129 active+clean; 49 MiB data, 649 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 275 KiB/s wr, 20 op/s 2024-04-16T17:12:26.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:12:26 smithi104 ceph-mon[36457]: pgmap v173: 129 pgs: 129 active+clean; 49 MiB data, 649 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 275 KiB/s wr, 20 op/s 2024-04-16T17:12:27.268 INFO:tasks.workunit.client.0.smithi045.stdout: 1 86275 1.56 MB/sec execute 149 sec latency 57.553 ms 2024-04-16T17:12:27.332 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-16T17:12:27.332 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-16T17:12:27.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.533+0000 7f5660e00640 1 -- 172.21.15.45:0/1837677228 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f565c074d50 msgr2=0x7f565c10e1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:27.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.533+0000 7f5660e00640 1 --2- 172.21.15.45:0/1837677228 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f565c074d50 0x7f565c10e1a0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f5648009900 tx=0x7f564802bce0 comp rx=0 tx=0).stop 2024-04-16T17:12:27.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.533+0000 7f5660e00640 1 -- 172.21.15.45:0/1837677228 shutdown_connections 2024-04-16T17:12:27.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.533+0000 7f5660e00640 1 --2- 172.21.15.45:0/1837677228 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f565c10e6e0 0x7f565c110b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:27.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.533+0000 7f5660e00640 1 --2- 172.21.15.45:0/1837677228 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f565c074d50 0x7f565c10e1a0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:27.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.533+0000 7f5660e00640 1 --2- 172.21.15.45:0/1837677228 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f565c074430 0x7f565c074810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:27.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.533+0000 7f5660e00640 1 -- 172.21.15.45:0/1837677228 >> 172.21.15.45:0/1837677228 conn(0x7f565c0fd9d0 msgr2=0x7f565c0ffdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:12:27.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.534+0000 7f5660e00640 1 -- 172.21.15.45:0/1837677228 shutdown_connections 2024-04-16T17:12:27.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.534+0000 7f5660e00640 1 -- 172.21.15.45:0/1837677228 wait complete. 2024-04-16T17:12:27.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.535+0000 7f5660e00640 1 Processor -- start 2024-04-16T17:12:27.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.535+0000 7f5660e00640 1 -- start start 2024-04-16T17:12:27.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.535+0000 7f5660e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f565c074430 0x7f565c1a06e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:27.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.536+0000 7f5660e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f565c074d50 0x7f565c1a0c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:27.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.536+0000 7f5660e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f565c10e6e0 0x7f565c19a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:27.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.536+0000 7f5660e00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f565c114540 con 0x7f565c074d50 2024-04-16T17:12:27.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.536+0000 7f5660e00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f565c1143c0 con 0x7f565c074430 2024-04-16T17:12:27.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.536+0000 7f5660e00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f565c1146c0 con 0x7f565c10e6e0 2024-04-16T17:12:27.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.536+0000 7f565b400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f565c074430 0x7f565c1a06e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:27.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.536+0000 7f565b400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f565c074430 0x7f565c1a06e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:37652/0 (socket says 172.21.15.45:37652) 2024-04-16T17:12:27.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.536+0000 7f565b400640 1 -- 172.21.15.45:0/1330013309 learned_addr learned my addr 172.21.15.45:0/1330013309 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:12:27.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.536+0000 7f565be00640 1 --2- 172.21.15.45:0/1330013309 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f565c10e6e0 0x7f565c19a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:27.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.536+0000 7f565aa00640 1 --2- 172.21.15.45:0/1330013309 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f565c074d50 0x7f565c1a0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:27.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.536+0000 7f565b400640 1 -- 172.21.15.45:0/1330013309 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f565c10e6e0 msgr2=0x7f565c19a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:27.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.536+0000 7f565b400640 1 --2- 172.21.15.45:0/1330013309 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f565c10e6e0 0x7f565c19a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:27.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.536+0000 7f565b400640 1 -- 172.21.15.45:0/1330013309 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f565c074d50 msgr2=0x7f565c1a0c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:27.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.536+0000 7f565b400640 1 --2- 172.21.15.45:0/1330013309 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f565c074d50 0x7f565c1a0c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:27.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.536+0000 7f565b400640 1 -- 172.21.15.45:0/1330013309 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f565c19b070 con 0x7f565c074430 2024-04-16T17:12:27.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.537+0000 7f565b400640 1 --2- 172.21.15.45:0/1330013309 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f565c074430 0x7f565c1a06e0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f564c00b450 tx=0x7f564c00b910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:12:27.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.537+0000 7f5653e00640 1 -- 172.21.15.45:0/1330013309 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f564c015020 con 0x7f565c074430 2024-04-16T17:12:27.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.537+0000 7f5660e00640 1 -- 172.21.15.45:0/1330013309 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f565c19b360 con 0x7f565c074430 2024-04-16T17:12:27.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.537+0000 7f5653e00640 1 -- 172.21.15.45:0/1330013309 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f564c011d70 con 0x7f565c074430 2024-04-16T17:12:27.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.537+0000 7f5653e00640 1 -- 172.21.15.45:0/1330013309 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f564c00f340 con 0x7f565c074430 2024-04-16T17:12:27.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.537+0000 7f5660e00640 1 -- 172.21.15.45:0/1330013309 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f565c1a7530 con 0x7f565c074430 2024-04-16T17:12:27.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.542+0000 7f56651ee740 1 Processor -- start 2024-04-16T17:12:27.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.542+0000 7f56651ee740 1 -- start start 2024-04-16T17:12:27.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.542+0000 7f5653e00640 1 -- 172.21.15.45:0/1330013309 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f564c018020 con 0x7f565c074430 2024-04-16T17:12:27.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.543+0000 7f56651ee740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e080890390 0x55e080890770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:27.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.543+0000 7f56651ee740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e080890cb0 0x55e080899350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:27.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.543+0000 7f56651ee740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e080899890 0x55e08089bc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:27.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.543+0000 7f56651ee740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55e0807f6120 con 0x55e080890390 2024-04-16T17:12:27.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.543+0000 7f56651ee740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55e08079c340 con 0x55e080890cb0 2024-04-16T17:12:27.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.543+0000 7f56651ee740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55e0807f20e0 con 0x55e080899890 2024-04-16T17:12:27.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.543+0000 7f565be00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e080890390 0x55e080890770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:27.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.543+0000 7f565be00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e080890390 0x55e080890770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35288/0 (socket says 172.21.15.45:35288) 2024-04-16T17:12:27.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.543+0000 7f565be00640 1 -- 172.21.15.45:0/2461274300 learned_addr learned my addr 172.21.15.45:0/2461274300 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:12:27.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.543+0000 7f565b400640 1 --2- 172.21.15.45:0/2461274300 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e080890cb0 0x55e080899350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:27.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.543+0000 7f565aa00640 1 --2- 172.21.15.45:0/2461274300 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e080899890 0x55e08089bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:27.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.544+0000 7f565b400640 1 -- 172.21.15.45:0/2461274300 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e080899890 msgr2=0x55e08089bc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:12:27.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.544+0000 7f565b400640 1 --2- 172.21.15.45:0/2461274300 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e080899890 0x55e08089bc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:27.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.544+0000 7f565b400640 1 -- 172.21.15.45:0/2461274300 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e080890390 msgr2=0x55e080890770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:12:27.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.544+0000 7f565b400640 1 --2- 172.21.15.45:0/2461274300 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e080890390 0x55e080890770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:27.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.544+0000 7f565b400640 1 -- 172.21.15.45:0/2461274300 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e0805d27d0 con 0x55e080890cb0 2024-04-16T17:12:27.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.544+0000 7f565be00640 1 --2- 172.21.15.45:0/2461274300 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e080890390 0x55e080890770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:12:27.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.544+0000 7f565b400640 1 --2- 172.21.15.45:0/2461274300 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e080890cb0 0x55e080899350 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f564c0cd990 tx=0x7f564c0d6ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9c07bf5b912bd030 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:12:27.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.544+0000 7f5653e00640 1 --2- 172.21.15.45:0/1330013309 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f56400a3d20 0x7f56400a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:27.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.545+0000 7f5650c00640 1 -- 172.21.15.45:0/2461274300 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f564c0e3070 con 0x55e080890cb0 2024-04-16T17:12:27.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.545+0000 7f5650c00640 1 -- 172.21.15.45:0/2461274300 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f564c09c8e0 con 0x55e080890cb0 2024-04-16T17:12:27.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.545+0000 7f5650c00640 1 -- 172.21.15.45:0/2461274300 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f564c09bd20 con 0x55e080890cb0 2024-04-16T17:12:27.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.545+0000 7f5653e00640 1 -- 172.21.15.45:0/1330013309 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f564c0d04a0 con 0x7f565c074430 2024-04-16T17:12:27.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.545+0000 7f565be00640 1 --2- 172.21.15.45:0/1330013309 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f56400a3d20 0x7f56400a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:27.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.546+0000 7f56651ee740 1 -- 172.21.15.45:0/2461274300 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e080890cb0 msgr2=0x55e080899350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:27.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.546+0000 7f56651ee740 1 --2- 172.21.15.45:0/2461274300 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e080890cb0 0x55e080899350 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f564c0cd990 tx=0x7f564c0d6ab0 comp rx=0 tx=0).stop 2024-04-16T17:12:27.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.546+0000 7f565be00640 1 --2- 172.21.15.45:0/1330013309 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f56400a3d20 0x7f56400a61e0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f5654011890 tx=0x7f5654009500 comp rx=0 tx=0).ready entity=mgr.25411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:12:27.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.546+0000 7f56651ee740 1 -- 172.21.15.45:0/2461274300 shutdown_connections 2024-04-16T17:12:27.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.546+0000 7f56651ee740 1 --2- 172.21.15.45:0/2461274300 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e080899890 0x55e08089bc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:27.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.546+0000 7f56651ee740 1 --2- 172.21.15.45:0/2461274300 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e080890cb0 0x55e080899350 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:27.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.546+0000 7f56651ee740 1 --2- 172.21.15.45:0/2461274300 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e080890390 0x55e080890770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:27.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.546+0000 7f56651ee740 1 -- 172.21.15.45:0/2461274300 >> 172.21.15.45:0/2461274300 conn(0x55e08088aac0 msgr2=0x55e08088aea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:12:27.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.547+0000 7f56651ee740 1 -- 172.21.15.45:0/2461274300 shutdown_connections 2024-04-16T17:12:27.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.547+0000 7f56651ee740 1 -- 172.21.15.45:0/2461274300 wait complete. 2024-04-16T17:12:27.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.547+0000 7f56651ee740 1 -- start start 2024-04-16T17:12:27.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.548+0000 7f56651ee740 1 Processor -- start 2024-04-16T17:12:27.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.548+0000 7f56651ee740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e080890390 0x55e080896910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:27.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.548+0000 7f56651ee740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e080890cb0 0x55e080896e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:27.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.549+0000 7f565b400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e080890390 0x55e080896910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:27.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.549+0000 7f56651ee740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e080899890 0x55e080897390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:27.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.549+0000 7f56651ee740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55e08089c680 con 0x55e080890390 2024-04-16T17:12:27.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.549+0000 7f565b400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e080890390 0x55e080896910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35304/0 (socket says 172.21.15.45:35304) 2024-04-16T17:12:27.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.549+0000 7f565b400640 1 -- 172.21.15.45:0/418842861 learned_addr learned my addr 172.21.15.45:0/418842861 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:12:27.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.549+0000 7f56651ee740 1 -- 172.21.15.45:0/418842861 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55e0807af710 con 0x55e080890cb0 2024-04-16T17:12:27.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.549+0000 7f56651ee740 1 -- 172.21.15.45:0/418842861 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55e08089c800 con 0x55e080899890 2024-04-16T17:12:27.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.549+0000 7f56651ee740 10 client.0 _ll_register_callbacks cb 0x55e0807aaaf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:12:27.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.549+0000 7f565aa00640 1 --2- 172.21.15.45:0/418842861 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e080890cb0 0x55e080896e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:27.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.549+0000 7f565be00640 1 --2- 172.21.15.45:0/418842861 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e080899890 0x55e080897390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:27.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.550+0000 7f565b400640 1 -- 172.21.15.45:0/418842861 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e080899890 msgr2=0x55e080897390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:12:27.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.550+0000 7f565b400640 1 --2- 172.21.15.45:0/418842861 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e080899890 0x55e080897390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:27.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.550+0000 7f565b400640 1 -- 172.21.15.45:0/418842861 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e080890cb0 msgr2=0x55e080896e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:12:27.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.550+0000 7f565b400640 1 --2- 172.21.15.45:0/418842861 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e080890cb0 0x55e080896e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:27.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.550+0000 7f565b400640 1 -- 172.21.15.45:0/418842861 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e0805083f0 con 0x55e080890390 2024-04-16T17:12:27.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.550+0000 7f565b400640 1 --2- 172.21.15.45:0/418842861 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e080890390 0x55e080896910 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f564c009fd0 tx=0x7f564c0d0720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f6be284aa1a9fdd server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:12:27.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.550+0000 7f5645600640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-16T17:12:27.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.550+0000 7f5645600640 1 -- 172.21.15.45:0/418842861 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f564c05c9c0 con 0x55e080890390 2024-04-16T17:12:27.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.550+0000 7f5645600640 1 -- 172.21.15.45:0/418842861 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f564c010550 con 0x55e080890390 2024-04-16T17:12:27.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.550+0000 7f56651ee740 1 -- 172.21.15.45:0/418842861 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f564c09c8e0 con 0x55e080890390 2024-04-16T17:12:27.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.551+0000 7f5645600640 1 -- 172.21.15.45:0/418842861 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f564c05cf40 con 0x55e080890390 2024-04-16T17:12:27.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.551+0000 7f56651ee740 10 client.16983 fetch_fsmap learned FSMap version 37 2024-04-16T17:12:27.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.551+0000 7f56651ee740 1 -- 172.21.15.45:0/418842861 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55e0804c80f0 con 0x55e080890390 2024-04-16T17:12:27.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.551+0000 7f5645600640 1 -- 172.21.15.45:0/418842861 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f564c09c8e0 con 0x55e080890390 2024-04-16T17:12:27.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.551+0000 7f5645600640 1 -- 172.21.15.45:0/418842861 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 37) ==== 8358+0+0 (secure 0 0 0) 0x7f564c0d0120 con 0x55e080890390 2024-04-16T17:12:27.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.551+0000 7f56651ee740 10 client.16983 fetch_fsmap finished waiting for FSMap version 37 2024-04-16T17:12:27.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.551+0000 7f56651ee740 10 client.16983 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:12:27.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.551+0000 7f56651ee740 20 client.16983 populate_metadata read hostname 'smithi045' 2024-04-16T17:12:27.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.551+0000 7f56651ee740 1 --2- 172.21.15.45:0/418842861 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55e0808a9af0 0x55e0808abee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:27.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.551+0000 7f56651ee740 4 client.16983 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:12:27.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.551+0000 7f56651ee740 1 -- 172.21.15.45:0/418842861 --> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e080803b40 con 0x55e0808a9af0 2024-04-16T17:12:27.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.552+0000 7f565aa00640 1 --2- 172.21.15.45:0/418842861 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55e0808a9af0 0x55e0808abee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:27.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.552+0000 7f565aa00640 1 --2- 172.21.15.45:0/418842861 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55e0808a9af0 0x55e0808abee0 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fdf61835578a369a server_cookie=82d3489fef3eee57 in_seq=0 out_seq=0 2024-04-16T17:12:27.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.552+0000 7f5645600640 10 client.16983 ms_handle_connect on v2:172.21.15.104:6838/3660070592 2024-04-16T17:12:27.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.553+0000 7f5645600640 1 -- 172.21.15.45:0/418842861 <== mds.0 v2:172.21.15.104:6838/3660070592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e080803b40 con 0x55e0808a9af0 2024-04-16T17:12:27.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.553+0000 7f5645600640 10 client.16983 handle_command_reply: tid=0 2024-04-16T17:12:27.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.553+0000 7f56651ee740 1 client.16983 shutdown 2024-04-16T17:12:27.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.553+0000 7f56651ee740 1 -- 172.21.15.45:0/418842861 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e080890390 msgr2=0x55e080896910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:27.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.553+0000 7f56651ee740 1 --2- 172.21.15.45:0/418842861 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e080890390 0x55e080896910 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f564c009fd0 tx=0x7f564c0d0720 comp rx=0 tx=0).stop 2024-04-16T17:12:27.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.554+0000 7f56651ee740 1 -- 172.21.15.45:0/418842861 shutdown_connections 2024-04-16T17:12:27.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.554+0000 7f56651ee740 1 --2- 172.21.15.45:0/418842861 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55e0808a9af0 0x55e0808abee0 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:27.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.554+0000 7f56651ee740 1 --2- 172.21.15.45:0/418842861 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e080899890 0x55e080897390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:27.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.554+0000 7f56651ee740 1 --2- 172.21.15.45:0/418842861 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e080890cb0 0x55e080896e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:27.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.554+0000 7f56651ee740 1 --2- 172.21.15.45:0/418842861 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e080890390 0x55e080896910 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:27.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.554+0000 7f56651ee740 1 -- 172.21.15.45:0/418842861 >> 172.21.15.45:0/418842861 conn(0x55e08088aac0 msgr2=0x55e08088c810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:12:27.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.554+0000 7f5645600640 0 client.16983 ms_handle_reset on v2:172.21.15.104:6838/3660070592 2024-04-16T17:12:27.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.554+0000 7f56651ee740 1 -- 172.21.15.45:0/418842861 shutdown_connections 2024-04-16T17:12:27.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.554+0000 7f56651ee740 1 -- 172.21.15.45:0/418842861 wait complete. 2024-04-16T17:12:27.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.554+0000 7f56651ee740 20 client.16983 trim_cache size 0 max 16384 2024-04-16T17:12:27.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.581+0000 7f56651ee740 1 Processor -- start 2024-04-16T17:12:27.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.581+0000 7f56651ee740 1 -- start start 2024-04-16T17:12:27.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.581+0000 7f56651ee740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e080890390 0x55e080899040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:27.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.582+0000 7f56651ee740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e080890cb0 0x55e0808a5540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:27.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.582+0000 7f56651ee740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e080899890 0x55e0808abb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:27.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.582+0000 7f56651ee740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f565c1b3e40 con 0x55e080890cb0 2024-04-16T17:12:27.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.582+0000 7f56651ee740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f565c1b1680 con 0x55e080890390 2024-04-16T17:12:27.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.582+0000 7f56651ee740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f565c1af4b0 con 0x55e080899890 2024-04-16T17:12:27.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.582+0000 7f565b400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e080890390 0x55e080899040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:27.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.582+0000 7f565b400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e080890390 0x55e080899040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:37668/0 (socket says 172.21.15.45:37668) 2024-04-16T17:12:27.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.582+0000 7f565b400640 1 -- 172.21.15.45:0/861919598 learned_addr learned my addr 172.21.15.45:0/861919598 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:12:27.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.582+0000 7f565be00640 1 --2- 172.21.15.45:0/861919598 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e080899890 0x55e0808abb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:27.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.582+0000 7f565aa00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e080890cb0 0x55e0808a5540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:27.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.582+0000 7f565b400640 1 -- 172.21.15.45:0/861919598 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e080899890 msgr2=0x55e0808abb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:12:27.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.582+0000 7f565b400640 1 --2- 172.21.15.45:0/861919598 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e080899890 0x55e0808abb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:27.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.582+0000 7f565b400640 1 -- 172.21.15.45:0/861919598 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e080890cb0 msgr2=0x55e0808a5540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:12:27.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.582+0000 7f565b400640 1 --2- 172.21.15.45:0/861919598 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e080890cb0 0x55e0808a5540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:27.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.582+0000 7f565b400640 1 -- 172.21.15.45:0/861919598 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e08088a5f0 con 0x55e080890390 2024-04-16T17:12:27.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.582+0000 7f565be00640 1 --2- 172.21.15.45:0/861919598 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e080899890 0x55e0808abb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:12:27.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.582+0000 7f565b400640 1 --2- 172.21.15.45:0/861919598 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e080890390 0x55e080899040 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f564c098b60 tx=0x7f564c0d6fb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=df3c955507f160b1 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:12:27.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.582+0000 7f5647e00640 1 -- 172.21.15.45:0/861919598 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f564c05b080 con 0x55e080890390 2024-04-16T17:12:27.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.582+0000 7f5647e00640 1 -- 172.21.15.45:0/861919598 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f564c05f030 con 0x55e080890390 2024-04-16T17:12:27.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.582+0000 7f5647e00640 1 -- 172.21.15.45:0/861919598 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f564c05b250 con 0x55e080890390 2024-04-16T17:12:27.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.584+0000 7f56651ee740 1 -- 172.21.15.45:0/861919598 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e080890390 msgr2=0x55e080899040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:27.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.584+0000 7f56651ee740 1 --2- 172.21.15.45:0/861919598 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e080890390 0x55e080899040 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f564c098b60 tx=0x7f564c0d6fb0 comp rx=0 tx=0).stop 2024-04-16T17:12:27.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.584+0000 7f56651ee740 1 -- 172.21.15.45:0/861919598 shutdown_connections 2024-04-16T17:12:27.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.584+0000 7f56651ee740 1 --2- 172.21.15.45:0/861919598 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e080899890 0x55e0808abb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:27.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.584+0000 7f56651ee740 1 --2- 172.21.15.45:0/861919598 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e080890cb0 0x55e0808a5540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:27.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.584+0000 7f56651ee740 1 --2- 172.21.15.45:0/861919598 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e080890390 0x55e080899040 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:27.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.584+0000 7f56651ee740 1 -- 172.21.15.45:0/861919598 >> 172.21.15.45:0/861919598 conn(0x55e08088aac0 msgr2=0x55e08088bea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:12:27.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.584+0000 7f56651ee740 1 -- 172.21.15.45:0/861919598 shutdown_connections 2024-04-16T17:12:27.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.584+0000 7f56651ee740 1 -- 172.21.15.45:0/861919598 wait complete. 2024-04-16T17:12:27.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.585+0000 7f56651ee740 1 -- start start 2024-04-16T17:12:27.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.585+0000 7f56651ee740 1 Processor -- start 2024-04-16T17:12:27.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.586+0000 7f56651ee740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e080890390 0x55e0808a1fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:27.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.586+0000 7f56651ee740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e080890cb0 0x55e080893400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:27.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.586+0000 7f56651ee740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e080899890 0x55e080893b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:27.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.586+0000 7f56651ee740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f565c1afb40 con 0x55e080890cb0 2024-04-16T17:12:27.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.586+0000 7f56651ee740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f565c1b0fb0 con 0x55e080890390 2024-04-16T17:12:27.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.586+0000 7f56651ee740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f565c1b0210 con 0x55e080899890 2024-04-16T17:12:27.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.586+0000 7f56651ee740 10 client.0 _ll_register_callbacks cb 0x7f564c0e1070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:12:27.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.586+0000 7f565b400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e080890390 0x55e0808a1fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:27.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.586+0000 7f565b400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e080890390 0x55e0808a1fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:37674/0 (socket says 172.21.15.45:37674) 2024-04-16T17:12:27.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.586+0000 7f565b400640 1 -- 172.21.15.45:0/1567951544 learned_addr learned my addr 172.21.15.45:0/1567951544 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:12:27.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.586+0000 7f565be00640 1 --2- 172.21.15.45:0/1567951544 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e080899890 0x55e080893b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:27.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.587+0000 7f565aa00640 1 --2- 172.21.15.45:0/1567951544 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e080890cb0 0x55e080893400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:27.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.587+0000 7f565b400640 1 -- 172.21.15.45:0/1567951544 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e080899890 msgr2=0x55e080893b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:12:27.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.587+0000 7f565b400640 1 --2- 172.21.15.45:0/1567951544 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e080899890 0x55e080893b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:27.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.587+0000 7f565b400640 1 -- 172.21.15.45:0/1567951544 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e080890cb0 msgr2=0x55e080893400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:12:27.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.587+0000 7f565b400640 1 --2- 172.21.15.45:0/1567951544 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e080890cb0 0x55e080893400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:27.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.587+0000 7f565b400640 1 -- 172.21.15.45:0/1567951544 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e080917500 con 0x55e080890390 2024-04-16T17:12:27.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.587+0000 7f565aa00640 1 --2- 172.21.15.45:0/1567951544 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e080890cb0 0x55e080893400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:12:27.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.587+0000 7f565b400640 1 --2- 172.21.15.45:0/1567951544 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e080890390 0x55e0808a1fa0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f564c098640 tx=0x7f564c0a4e50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1583f4c63e66ac9a server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:12:27.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.587+0000 7f5645600640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-04-16T17:12:27.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.587+0000 7f5645600640 1 -- 172.21.15.45:0/1567951544 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f564c05b070 con 0x55e080890390 2024-04-16T17:12:27.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.588+0000 7f56651ee740 1 -- 172.21.15.45:0/1567951544 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f564c05f030 con 0x55e080890390 2024-04-16T17:12:27.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.588+0000 7f5645600640 1 -- 172.21.15.45:0/1567951544 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f564c0d3c80 con 0x55e080890390 2024-04-16T17:12:27.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.588+0000 7f5645600640 1 -- 172.21.15.45:0/1567951544 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f564c05b070 con 0x55e080890390 2024-04-16T17:12:27.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.588+0000 7f56651ee740 10 client.27137 fetch_fsmap learned FSMap version 37 2024-04-16T17:12:27.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.588+0000 7f56651ee740 1 -- 172.21.15.45:0/1567951544 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55e080917500 con 0x55e080890390 2024-04-16T17:12:27.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.589+0000 7f5645600640 1 -- 172.21.15.45:0/1567951544 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f564c05f030 con 0x55e080890390 2024-04-16T17:12:27.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.589+0000 7f5645600640 1 -- 172.21.15.45:0/1567951544 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 37) ==== 8358+0+0 (secure 0 0 0) 0x7f564c0a0180 con 0x55e080890390 2024-04-16T17:12:27.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.589+0000 7f56651ee740 10 client.27137 fetch_fsmap finished waiting for FSMap version 37 2024-04-16T17:12:27.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.589+0000 7f56651ee740 10 client.27137 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:12:27.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.589+0000 7f56651ee740 20 client.27137 populate_metadata read hostname 'smithi045' 2024-04-16T17:12:27.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.589+0000 7f56651ee740 1 --2- 172.21.15.45:0/1567951544 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55e0809188e0 0x55e08091acd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:27.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.589+0000 7f56651ee740 4 client.27137 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-16T17:12:27.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.589+0000 7f56651ee740 1 -- 172.21.15.45:0/1567951544 --> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55e0807fd9b0 con 0x55e0809188e0 2024-04-16T17:12:27.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.589+0000 7f565aa00640 1 --2- 172.21.15.45:0/1567951544 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55e0809188e0 0x55e08091acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:27.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.590+0000 7f565aa00640 1 --2- 172.21.15.45:0/1567951544 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55e0809188e0 0x55e08091acd0 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cff42cf20f9839c0 server_cookie=9454761218d9033 in_seq=0 out_seq=0 2024-04-16T17:12:27.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.590+0000 7f5645600640 10 client.27137 ms_handle_connect on v2:172.21.15.104:6838/3660070592 2024-04-16T17:12:27.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.590+0000 7f5645600640 1 -- 172.21.15.45:0/1567951544 <== mds.0 v2:172.21.15.104:6838/3660070592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55e0807fd9b0 con 0x55e0809188e0 2024-04-16T17:12:27.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.590+0000 7f5645600640 10 client.27137 handle_command_reply: tid=0 2024-04-16T17:12:27.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.590+0000 7f56651ee740 1 client.27137 shutdown 2024-04-16T17:12:27.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.590+0000 7f56651ee740 1 -- 172.21.15.45:0/1567951544 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e080890390 msgr2=0x55e0808a1fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:27.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.590+0000 7f56651ee740 1 --2- 172.21.15.45:0/1567951544 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e080890390 0x55e0808a1fa0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f564c098640 tx=0x7f564c0a4e50 comp rx=0 tx=0).stop 2024-04-16T17:12:27.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.591+0000 7f56651ee740 1 -- 172.21.15.45:0/1567951544 shutdown_connections 2024-04-16T17:12:27.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.591+0000 7f56651ee740 1 --2- 172.21.15.45:0/1567951544 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55e0809188e0 0x55e08091acd0 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:27.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.591+0000 7f56651ee740 1 --2- 172.21.15.45:0/1567951544 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e080899890 0x55e080893b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:27.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.591+0000 7f56651ee740 1 --2- 172.21.15.45:0/1567951544 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e080890cb0 0x55e080893400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:27.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.591+0000 7f56651ee740 1 --2- 172.21.15.45:0/1567951544 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e080890390 0x55e0808a1fa0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:27.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.591+0000 7f56651ee740 1 -- 172.21.15.45:0/1567951544 >> 172.21.15.45:0/1567951544 conn(0x55e08088aac0 msgr2=0x55e08088bea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:12:27.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.591+0000 7f5645600640 0 client.27137 ms_handle_reset on v2:172.21.15.104:6838/3660070592 2024-04-16T17:12:27.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.591+0000 7f56651ee740 1 -- 172.21.15.45:0/1567951544 shutdown_connections 2024-04-16T17:12:27.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.591+0000 7f56651ee740 1 -- 172.21.15.45:0/1567951544 wait complete. 2024-04-16T17:12:27.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.591+0000 7f56651ee740 20 client.27137 trim_cache size 0 max 16384 2024-04-16T17:12:27.594 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-16T17:12:27.594 INFO:teuthology.orchestra.run.smithi045.stdout: "status": "no active scrubs running", 2024-04-16T17:12:27.594 INFO:teuthology.orchestra.run.smithi045.stdout: "scrubs": {} 2024-04-16T17:12:27.594 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-16T17:12:27.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.592+0000 7f5660e00640 1 -- 172.21.15.45:0/1330013309 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f56400a3d20 msgr2=0x7f56400a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:27.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.592+0000 7f5660e00640 1 --2- 172.21.15.45:0/1330013309 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f56400a3d20 0x7f56400a61e0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f5654011890 tx=0x7f5654009500 comp rx=0 tx=0).stop 2024-04-16T17:12:27.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.592+0000 7f5660e00640 1 -- 172.21.15.45:0/1330013309 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f565c074430 msgr2=0x7f565c1a06e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:27.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.592+0000 7f5660e00640 1 --2- 172.21.15.45:0/1330013309 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f565c074430 0x7f565c1a06e0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f564c00b450 tx=0x7f564c00b910 comp rx=0 tx=0).stop 2024-04-16T17:12:27.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.592+0000 7f5660e00640 1 -- 172.21.15.45:0/1330013309 shutdown_connections 2024-04-16T17:12:27.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.592+0000 7f5660e00640 1 --2- 172.21.15.45:0/1330013309 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f56400a3d20 0x7f56400a61e0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:27.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.592+0000 7f5660e00640 1 --2- 172.21.15.45:0/1330013309 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f565c10e6e0 0x7f565c19a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:27.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.592+0000 7f5660e00640 1 --2- 172.21.15.45:0/1330013309 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f565c074d50 0x7f565c1a0c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:27.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.592+0000 7f5660e00640 1 --2- 172.21.15.45:0/1330013309 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f565c074430 0x7f565c1a06e0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:27.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.592+0000 7f5660e00640 1 -- 172.21.15.45:0/1330013309 >> 172.21.15.45:0/1330013309 conn(0x7f565c0fd9d0 msgr2=0x7f565c0fe780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:12:27.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.592+0000 7f5660e00640 1 -- 172.21.15.45:0/1330013309 shutdown_connections 2024-04-16T17:12:27.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.592+0000 7f5660e00640 1 -- 172.21.15.45:0/1330013309 wait complete. 2024-04-16T17:12:27.610 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-16T17:12:27.611 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-16T17:12:27.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.809+0000 7fe35f200640 1 -- 172.21.15.45:0/1405791091 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe3580743d0 msgr2=0x7fe3580747b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:27.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.809+0000 7fe35f200640 1 --2- 172.21.15.45:0/1405791091 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe3580743d0 0x7fe3580747b0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fe34c009900 tx=0x7fe34c02bcc0 comp rx=0 tx=0).stop 2024-04-16T17:12:27.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.810+0000 7fe35f200640 1 -- 172.21.15.45:0/1405791091 shutdown_connections 2024-04-16T17:12:27.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.810+0000 7fe35f200640 1 --2- 172.21.15.45:0/1405791091 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe35810e6c0 0x7fe358110ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:27.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.810+0000 7fe35f200640 1 --2- 172.21.15.45:0/1405791091 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe358074cf0 0x7fe35810e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:27.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.810+0000 7fe35f200640 1 --2- 172.21.15.45:0/1405791091 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe3580743d0 0x7fe3580747b0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:27.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.810+0000 7fe35f200640 1 -- 172.21.15.45:0/1405791091 >> 172.21.15.45:0/1405791091 conn(0x7fe3580fd930 msgr2=0x7fe3580ffd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:12:27.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.810+0000 7fe35f200640 1 -- 172.21.15.45:0/1405791091 shutdown_connections 2024-04-16T17:12:27.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.810+0000 7fe35f200640 1 -- 172.21.15.45:0/1405791091 wait complete. 2024-04-16T17:12:27.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.811+0000 7fe35f200640 1 Processor -- start 2024-04-16T17:12:27.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.811+0000 7fe35f200640 1 -- start start 2024-04-16T17:12:27.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.811+0000 7fe35f200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe3580743d0 0x7fe3581a06b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:27.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.812+0000 7fe35f200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe358074cf0 0x7fe3581a0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:27.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.812+0000 7fe35f200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe35810e6c0 0x7fe35819a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:27.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.812+0000 7fe35f200640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fe358114480 con 0x7fe35810e6c0 2024-04-16T17:12:27.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.812+0000 7fe35f200640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fe358114300 con 0x7fe358074cf0 2024-04-16T17:12:27.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.812+0000 7fe35f200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fe358114600 con 0x7fe3580743d0 2024-04-16T17:12:27.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.812+0000 7fe35de00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe3580743d0 0x7fe3581a06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:27.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.812+0000 7fe35d400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe358074cf0 0x7fe3581a0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:27.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.812+0000 7fe35de00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe3580743d0 0x7fe3581a06b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:43630/0 (socket says 172.21.15.45:43630) 2024-04-16T17:12:27.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.812+0000 7fe35de00640 1 -- 172.21.15.45:0/2023581863 learned_addr learned my addr 172.21.15.45:0/2023581863 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:12:27.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.812+0000 7fe35d400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe358074cf0 0x7fe3581a0bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:37692/0 (socket says 172.21.15.45:37692) 2024-04-16T17:12:27.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.812+0000 7fe35e800640 1 --2- 172.21.15.45:0/2023581863 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe35810e6c0 0x7fe35819a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:27.814 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.812+0000 7fe35de00640 1 -- 172.21.15.45:0/2023581863 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe358074cf0 msgr2=0x7fe3581a0bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:27.814 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.812+0000 7fe35de00640 1 --2- 172.21.15.45:0/2023581863 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe358074cf0 0x7fe3581a0bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:27.814 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.812+0000 7fe35de00640 1 -- 172.21.15.45:0/2023581863 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe35810e6c0 msgr2=0x7fe35819a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:27.814 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.812+0000 7fe35de00640 1 --2- 172.21.15.45:0/2023581863 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe35810e6c0 0x7fe35819a7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:27.814 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.812+0000 7fe35de00640 1 -- 172.21.15.45:0/2023581863 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe35819b080 con 0x7fe3580743d0 2024-04-16T17:12:27.814 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.813+0000 7fe35de00640 1 --2- 172.21.15.45:0/2023581863 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe3580743d0 0x7fe3581a06b0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fe34c000c00 tx=0x7fe34c0043c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:12:27.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.813+0000 7fe352a00640 1 -- 172.21.15.45:0/2023581863 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe34c042020 con 0x7fe3580743d0 2024-04-16T17:12:27.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.813+0000 7fe35f200640 1 -- 172.21.15.45:0/2023581863 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe35819b310 con 0x7fe3580743d0 2024-04-16T17:12:27.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.813+0000 7fe352a00640 1 -- 172.21.15.45:0/2023581863 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe34c005000 con 0x7fe3580743d0 2024-04-16T17:12:27.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.813+0000 7fe352a00640 1 -- 172.21.15.45:0/2023581863 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe34c003b20 con 0x7fe3580743d0 2024-04-16T17:12:27.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.813+0000 7fe35f200640 1 -- 172.21.15.45:0/2023581863 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3581a7630 con 0x7fe3580743d0 2024-04-16T17:12:27.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.815+0000 7fe35f200640 1 -- 172.21.15.45:0/2023581863 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe358075a80 con 0x7fe3580743d0 2024-04-16T17:12:27.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.816+0000 7fe352a00640 1 -- 172.21.15.45:0/2023581863 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fe34c045020 con 0x7fe3580743d0 2024-04-16T17:12:27.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.817+0000 7fe352a00640 1 --2- 172.21.15.45:0/2023581863 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fe3400a3d20 0x7fe3400a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:27.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.818+0000 7fe352a00640 1 -- 172.21.15.45:0/2023581863 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fe34c0f28a0 con 0x7fe3580743d0 2024-04-16T17:12:27.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.818+0000 7fe35d400640 1 --2- 172.21.15.45:0/2023581863 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fe3400a3d20 0x7fe3400a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:27.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.819+0000 7fe35d400640 1 --2- 172.21.15.45:0/2023581863 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fe3400a3d20 0x7fe3400a61e0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fe35819bfc0 tx=0x7fe34800b590 comp rx=0 tx=0).ready entity=mgr.25411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:12:27.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:27.822+0000 7fe352a00640 1 -- 172.21.15.45:0/2023581863 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe34c04c020 con 0x7fe3580743d0 2024-04-16T17:12:28.029 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:28.028+0000 7f4942e00640 1 -- 172.21.15.45:0/2792591605 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f493c1042a0 msgr2=0x7f493c104680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:28.029 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:28.028+0000 7f4942e00640 1 --2- 172.21.15.45:0/2792591605 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f493c1042a0 0x7f493c104680 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f4930009900 tx=0x7f493002bc30 comp rx=0 tx=0).stop 2024-04-16T17:12:28.029 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:28.028+0000 7f4942e00640 1 -- 172.21.15.45:0/2792591605 shutdown_connections 2024-04-16T17:12:28.029 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:28.028+0000 7f4942e00640 1 --2- 172.21.15.45:0/2792591605 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f493c109370 0x7f493c110f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:28.029 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:28.028+0000 7f4942e00640 1 --2- 172.21.15.45:0/2792591605 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f493c104c50 0x7f493c108c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:28.029 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:28.028+0000 7f4942e00640 1 --2- 172.21.15.45:0/2792591605 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f493c1042a0 0x7f493c104680 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:28.029 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:28.028+0000 7f4942e00640 1 -- 172.21.15.45:0/2792591605 >> 172.21.15.45:0/2792591605 conn(0x7f493c0ffae0 msgr2=0x7f493c101f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:12:28.029 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:28.028+0000 7f4942e00640 1 -- 172.21.15.45:0/2792591605 shutdown_connections 2024-04-16T17:12:28.029 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:28.028+0000 7f4942e00640 1 -- 172.21.15.45:0/2792591605 wait complete. 2024-04-16T17:12:28.030 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:28.029+0000 7f4942e00640 1 Processor -- start 2024-04-16T17:12:28.030 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:28.029+0000 7f4942e00640 1 -- start start 2024-04-16T17:12:28.030 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:28.030+0000 7f4942e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f493c1042a0 0x7f493c110ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:28.030 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:28.030+0000 7f4942e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f493c109370 0x7f493c10e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:28.030 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:28.030+0000 7f4942e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f493c10e750 0x7f493c1ab190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:28.030 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:28.030+0000 7f4942e00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f493c114450 con 0x7f493c109370 2024-04-16T17:12:28.031 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:28.030+0000 7f4942e00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f493c1142d0 con 0x7f493c1042a0 2024-04-16T17:12:28.031 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:28.030+0000 7f4942e00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f493c1145d0 con 0x7f493c10e750 2024-04-16T17:12:28.031 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:28.030+0000 7f4941a00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f493c1042a0 0x7f493c110ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:28.031 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:28.030+0000 7f4942400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f493c10e750 0x7f493c1ab190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:28.031 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:28.030+0000 7f4941a00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f493c1042a0 0x7f493c110ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:37708/0 (socket says 172.21.15.45:37708) 2024-04-16T17:12:28.032 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:28.030+0000 7f4941a00640 1 -- 172.21.15.45:0/4078322149 learned_addr learned my addr 172.21.15.45:0/4078322149 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:12:28.032 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:28.031+0000 7f4941000640 1 --2- 172.21.15.45:0/4078322149 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f493c109370 0x7f493c10e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:28.032 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:28.031+0000 7f4941a00640 1 -- 172.21.15.45:0/4078322149 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f493c10e750 msgr2=0x7f493c1ab190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:28.032 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:28.031+0000 7f4941a00640 1 --2- 172.21.15.45:0/4078322149 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f493c10e750 0x7f493c1ab190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:28.032 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:28.031+0000 7f4941a00640 1 -- 172.21.15.45:0/4078322149 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f493c109370 msgr2=0x7f493c10e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:28.032 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:28.031+0000 7f4941a00640 1 --2- 172.21.15.45:0/4078322149 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f493c109370 0x7f493c10e0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:28.032 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:28.031+0000 7f4941a00640 1 -- 172.21.15.45:0/4078322149 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f493c1ab6d0 con 0x7f493c1042a0 2024-04-16T17:12:28.032 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:28.031+0000 7f4941a00640 1 --2- 172.21.15.45:0/4078322149 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f493c1042a0 0x7f493c110ef0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f493002c240 tx=0x7f49300042f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:12:28.033 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:28.031+0000 7f493a000640 1 -- 172.21.15.45:0/4078322149 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4930043020 con 0x7f493c1042a0 2024-04-16T17:12:28.033 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:28.031+0000 7f493a000640 1 -- 172.21.15.45:0/4078322149 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f49300054e0 con 0x7f493c1042a0 2024-04-16T17:12:28.033 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:28.031+0000 7f493a000640 1 -- 172.21.15.45:0/4078322149 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4930003b50 con 0x7f493c1042a0 2024-04-16T17:12:28.033 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:28.031+0000 7f4942e00640 1 -- 172.21.15.45:0/4078322149 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f493c1ab8a0 con 0x7f493c1042a0 2024-04-16T17:12:28.033 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:28.032+0000 7f4942e00640 1 -- 172.21.15.45:0/4078322149 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f493c1abd10 con 0x7f493c1042a0 2024-04-16T17:12:28.035 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:28.035+0000 7f493a000640 1 -- 172.21.15.45:0/4078322149 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f4930046020 con 0x7f493c1042a0 2024-04-16T17:12:28.037 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:28.036+0000 7f493a000640 1 --2- 172.21.15.45:0/4078322149 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f49240a3d10 0x7f49240a61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:28.037 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:28.037+0000 7f493a000640 1 -- 172.21.15.45:0/4078322149 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f49300f2ba0 con 0x7f493c1042a0 2024-04-16T17:12:28.038 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:28.037+0000 7f4941000640 1 --2- 172.21.15.45:0/4078322149 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f49240a3d10 0x7f49240a61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:28.038 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:28.038+0000 7f4947233740 1 Processor -- start 2024-04-16T17:12:28.040 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:28.038+0000 7f4947233740 1 -- start start 2024-04-16T17:12:28.040 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:28.038+0000 7f4941000640 1 --2- 172.21.15.45:0/4078322149 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f49240a3d10 0x7f49240a61d0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f49280095f0 tx=0x7f4928005f90 comp rx=0 tx=0).ready entity=mgr.25411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:12:28.040 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:28.038+0000 7f4947233740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56477fe207f0 0x56477fe1f540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:28.040 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:28.038+0000 7f4947233740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56477fe1fa80 0x56477fe1ff00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:28.040 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:28.038+0000 7f4947233740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56477fe29000 0x56477fe2b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:28.040 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:28.038+0000 7f4947233740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x56477fd84780 con 0x56477fe1fa80 2024-04-16T17:12:28.040 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:28.038+0000 7f4947233740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x56477fd2af40 con 0x56477fe207f0 2024-04-16T17:12:28.040 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:28.038+0000 7f4947233740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x56477fd80740 con 0x56477fe29000 2024-04-16T17:12:28.040 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:28.039+0000 7f4941a00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56477fe207f0 0x56477fe1f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:28.040 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:28.039+0000 7f4941a00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56477fe207f0 0x56477fe1f540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:37720/0 (socket says 172.21.15.45:37720) 2024-04-16T17:12:28.040 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:28.039+0000 7f4941a00640 1 -- 172.21.15.45:0/1403976747 learned_addr learned my addr 172.21.15.45:0/1403976747 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:12:28.040 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:28.039+0000 7f4941000640 1 --2- 172.21.15.45:0/1403976747 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56477fe1fa80 0x56477fe1ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:28.040 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:28.039+0000 7f4941a00640 1 -- 172.21.15.45:0/1403976747 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56477fe29000 msgr2=0x56477fe2b3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:12:28.041 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:28.039+0000 7f4941a00640 1 --2- 172.21.15.45:0/1403976747 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56477fe29000 0x56477fe2b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:28.041 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:28.039+0000 7f4941a00640 1 -- 172.21.15.45:0/1403976747 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56477fe1fa80 msgr2=0x56477fe1ff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:12:28.041 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:28.039+0000 7f4941a00640 1 --2- 172.21.15.45:0/1403976747 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56477fe1fa80 0x56477fe1ff00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:28.041 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:28.039+0000 7f4941a00640 1 -- 172.21.15.45:0/1403976747 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56477fa97210 con 0x56477fe207f0 2024-04-16T17:12:28.041 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:28.039+0000 7f4941a00640 1 --2- 172.21.15.45:0/1403976747 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56477fe207f0 0x56477fe1f540 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f49300bad00 tx=0x7f49300035f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3a47eb0da612db21 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:12:28.041 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:28.040+0000 7f492ea00640 1 -- 172.21.15.45:0/1403976747 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4930005690 con 0x56477fe207f0 2024-04-16T17:12:28.041 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:28.040+0000 7f492ea00640 1 -- 172.21.15.45:0/1403976747 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f49300befa0 con 0x56477fe207f0 2024-04-16T17:12:28.041 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:28.040+0000 7f492ea00640 1 -- 172.21.15.45:0/1403976747 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49300baf10 con 0x56477fe207f0 2024-04-16T17:12:28.042 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:28.041+0000 7f4947233740 1 -- 172.21.15.45:0/1403976747 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56477fe207f0 msgr2=0x56477fe1f540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:28.042 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:28.041+0000 7f4947233740 1 --2- 172.21.15.45:0/1403976747 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56477fe207f0 0x56477fe1f540 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f49300bad00 tx=0x7f49300035f0 comp rx=0 tx=0).stop 2024-04-16T17:12:28.042 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:28.041+0000 7f4947233740 1 -- 172.21.15.45:0/1403976747 shutdown_connections 2024-04-16T17:12:28.043 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:28.041+0000 7f4947233740 1 --2- 172.21.15.45:0/1403976747 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56477fe29000 0x56477fe2b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:28.043 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:28.041+0000 7f4947233740 1 --2- 172.21.15.45:0/1403976747 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56477fe1fa80 0x56477fe1ff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:28.043 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:28.041+0000 7f4947233740 1 --2- 172.21.15.45:0/1403976747 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56477fe207f0 0x56477fe1f540 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:28.043 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:28.041+0000 7f4947233740 1 -- 172.21.15.45:0/1403976747 >> 172.21.15.45:0/1403976747 conn(0x56477fdba5d0 msgr2=0x56477fe1a770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:12:28.043 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:28.042+0000 7f4947233740 1 -- 172.21.15.45:0/1403976747 shutdown_connections 2024-04-16T17:12:28.043 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:28.042+0000 7f4947233740 1 -- 172.21.15.45:0/1403976747 wait complete. 2024-04-16T17:12:28.043 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:28.042+0000 7f4947233740 1 -- start start 2024-04-16T17:12:28.043 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:28.043+0000 7f4947233740 1 Processor -- start 2024-04-16T17:12:28.044 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:28.043+0000 7f4947233740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56477fe1fa80 0x56477fe25e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:28.045 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:28.043+0000 7f4947233740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56477fe207f0 0x56477fe263d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:28.045 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:28.044+0000 7f4947233740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56477fe29000 0x56477fe26910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:28.045 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:28.044+0000 7f4947233740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x56477fd2af40 con 0x56477fe1fa80 2024-04-16T17:12:28.045 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:28.044+0000 7f4947233740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x56477fd3e390 con 0x56477fe207f0 2024-04-16T17:12:28.045 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:28.044+0000 7f4947233740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x56477fe2baf0 con 0x56477fe29000 2024-04-16T17:12:28.045 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:28.044+0000 7f4947233740 10 client.0 _ll_register_callbacks cb 0x56477fd27f60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:12:28.045 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:28.044+0000 7f4942400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56477fe29000 0x56477fe26910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:28.046 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:28.044+0000 7f4942400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56477fe29000 0x56477fe26910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:43674/0 (socket says 172.21.15.45:43674) 2024-04-16T17:12:28.046 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:28.044+0000 7f4942400640 1 -- 172.21.15.45:0/926365861 learned_addr learned my addr 172.21.15.45:0/926365861 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:12:28.046 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:28.044+0000 7f4941a00640 1 --2- 172.21.15.45:0/926365861 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56477fe1fa80 0x56477fe25e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:28.046 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:28.044+0000 7f4942400640 1 -- 172.21.15.45:0/926365861 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56477fe207f0 msgr2=0x56477fe263d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-16T17:12:28.046 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:28.044+0000 7f4942400640 1 --2- 172.21.15.45:0/926365861 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56477fe207f0 0x56477fe263d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:28.046 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:28.044+0000 7f4942400640 1 -- 172.21.15.45:0/926365861 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56477fe1fa80 msgr2=0x56477fe25e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:12:28.046 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:28.044+0000 7f4942400640 1 --2- 172.21.15.45:0/926365861 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56477fe1fa80 0x56477fe25e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:28.046 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:28.044+0000 7f4942400640 1 -- 172.21.15.45:0/926365861 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56477fb63620 con 0x56477fe29000 2024-04-16T17:12:28.046 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:28.044+0000 7f4941a00640 1 --2- 172.21.15.45:0/926365861 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56477fe1fa80 0x56477fe25e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:12:28.048 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:28.045+0000 7f4942400640 1 --2- 172.21.15.45:0/926365861 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56477fe29000 0x56477fe26910 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f49340107f0 tx=0x7f4934010cb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8eb3e60dcebc07f8 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:12:28.048 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:28.045+0000 7f4923400640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-04-16T17:12:28.048 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:28.045+0000 7f4923400640 1 -- 172.21.15.45:0/926365861 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f493400ebe0 con 0x56477fe29000 2024-04-16T17:12:28.048 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:28.045+0000 7f4923400640 1 -- 172.21.15.45:0/926365861 <== mon.1 v2:172.21.15.117:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f493400ed80 con 0x56477fe29000 2024-04-16T17:12:28.048 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:28.045+0000 7f4947233740 1 -- 172.21.15.45:0/926365861 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f49300befa0 con 0x56477fe29000 2024-04-16T17:12:28.048 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:28.045+0000 7f4923400640 1 -- 172.21.15.45:0/926365861 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f4934012450 con 0x56477fe29000 2024-04-16T17:12:28.048 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:28.046+0000 7f4947233740 10 client.27100 fetch_fsmap learned FSMap version 37 2024-04-16T17:12:28.049 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:28.046+0000 7f4947233740 1 -- 172.21.15.45:0/926365861 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x56477fe37b10 con 0x56477fe29000 2024-04-16T17:12:28.049 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:28.046+0000 7f4923400640 1 -- 172.21.15.45:0/926365861 <== mon.1 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f49300befa0 con 0x56477fe29000 2024-04-16T17:12:28.049 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:28.046+0000 7f4923400640 1 -- 172.21.15.45:0/926365861 <== mon.1 v2:172.21.15.117:3300/0 5 ==== fsmap(e 37) ==== 8358+0+0 (secure 0 0 0) 0x7f493400f040 con 0x56477fe29000 2024-04-16T17:12:28.049 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:28.046+0000 7f4947233740 10 client.27100 fetch_fsmap finished waiting for FSMap version 37 2024-04-16T17:12:28.049 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:28.046+0000 7f4947233740 10 client.27100 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T17:12:28.049 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:28.046+0000 7f4947233740 20 client.27100 populate_metadata read hostname 'smithi045' 2024-04-16T17:12:28.049 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:28.046+0000 7f4947233740 1 --2- 172.21.15.45:0/926365861 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x56477fe39020 0x56477fe3b410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:28.049 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:28.046+0000 7f4947233740 4 client.27100 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:12:28.049 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:28.046+0000 7f4947233740 1 -- 172.21.15.45:0/926365861 --> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56477fd921a0 con 0x56477fe39020 2024-04-16T17:12:28.049 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:28.047+0000 7f4941a00640 1 --2- 172.21.15.45:0/926365861 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x56477fe39020 0x56477fe3b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:28.049 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:28.048+0000 7f4941a00640 1 --2- 172.21.15.45:0/926365861 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x56477fe39020 0x56477fe3b410 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8ca6e5838fb2fe54 server_cookie=c562c6b91a41776f in_seq=0 out_seq=0 2024-04-16T17:12:28.049 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:28.048+0000 7f4923400640 10 client.27100 ms_handle_connect on v2:172.21.15.104:6838/3660070592 2024-04-16T17:12:28.051 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:28.048+0000 7f4923400640 1 -- 172.21.15.45:0/926365861 <== mds.0 v2:172.21.15.104:6838/3660070592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56477fd921a0 con 0x56477fe39020 2024-04-16T17:12:28.051 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:28.048+0000 7f4923400640 10 client.27100 handle_command_reply: tid=0 2024-04-16T17:12:28.051 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:28.048+0000 7f4947233740 1 client.27100 shutdown 2024-04-16T17:12:28.051 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:28.049+0000 7f4947233740 1 -- 172.21.15.45:0/926365861 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56477fe29000 msgr2=0x56477fe26910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:28.051 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:28.049+0000 7f4947233740 1 --2- 172.21.15.45:0/926365861 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56477fe29000 0x56477fe26910 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f49340107f0 tx=0x7f4934010cb0 comp rx=0 tx=0).stop 2024-04-16T17:12:28.051 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:28.049+0000 7f4947233740 1 -- 172.21.15.45:0/926365861 shutdown_connections 2024-04-16T17:12:28.051 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:28.049+0000 7f4947233740 1 --2- 172.21.15.45:0/926365861 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x56477fe39020 0x56477fe3b410 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:28.051 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:28.049+0000 7f4947233740 1 --2- 172.21.15.45:0/926365861 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56477fe29000 0x56477fe26910 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:28.051 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:28.049+0000 7f4947233740 1 --2- 172.21.15.45:0/926365861 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56477fe207f0 0x56477fe263d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:28.051 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:28.049+0000 7f4947233740 1 --2- 172.21.15.45:0/926365861 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56477fe1fa80 0x56477fe25e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:28.051 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:28.049+0000 7f4923400640 0 client.27100 ms_handle_reset on v2:172.21.15.104:6838/3660070592 2024-04-16T17:12:28.051 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:28.049+0000 7f4947233740 1 -- 172.21.15.45:0/926365861 >> 172.21.15.45:0/926365861 conn(0x56477fdba5d0 msgr2=0x56477fe1c4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:12:28.051 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:28.050+0000 7f4947233740 1 -- 172.21.15.45:0/926365861 shutdown_connections 2024-04-16T17:12:28.051 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:28.050+0000 7f4947233740 1 -- 172.21.15.45:0/926365861 wait complete. 2024-04-16T17:12:28.052 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:28.050+0000 7f4947233740 20 client.27100 trim_cache size 0 max 16384 2024-04-16T17:12:28.080 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:28.079+0000 7f4947233740 1 Processor -- start 2024-04-16T17:12:28.080 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:28.080+0000 7f4947233740 1 -- start start 2024-04-16T17:12:28.081 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:28.080+0000 7f4947233740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56477fe1fa80 0x56477fe285c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:28.082 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:28.080+0000 7f4947233740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56477fe207f0 0x56477fe34870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:28.082 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:28.080+0000 7f4947233740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56477fe29000 0x56477fe25010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:28.082 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:28.080+0000 7f4947233740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f493c1b8260 con 0x56477fe29000 2024-04-16T17:12:28.082 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:28.080+0000 7f4947233740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f493c1b5aa0 con 0x56477fe1fa80 2024-04-16T17:12:28.082 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:28.080+0000 7f4947233740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f493c1b38d0 con 0x56477fe207f0 2024-04-16T17:12:28.082 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:28.080+0000 7f4941a00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56477fe1fa80 0x56477fe285c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:28.082 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:28.080+0000 7f4941a00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56477fe1fa80 0x56477fe285c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:37744/0 (socket says 172.21.15.45:37744) 2024-04-16T17:12:28.082 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:28.080+0000 7f4941a00640 1 -- 172.21.15.45:0/2872801483 learned_addr learned my addr 172.21.15.45:0/2872801483 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:12:28.082 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:28.080+0000 7f4942400640 1 --2- 172.21.15.45:0/2872801483 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56477fe29000 0x56477fe25010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:28.082 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:28.080+0000 7f4941000640 1 --2- 172.21.15.45:0/2872801483 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56477fe207f0 0x56477fe34870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:28.082 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:28.081+0000 7f4941a00640 1 -- 172.21.15.45:0/2872801483 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56477fe207f0 msgr2=0x56477fe34870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:12:28.082 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:28.081+0000 7f4941a00640 1 --2- 172.21.15.45:0/2872801483 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56477fe207f0 0x56477fe34870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:28.082 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:28.081+0000 7f4941a00640 1 -- 172.21.15.45:0/2872801483 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56477fe29000 msgr2=0x56477fe25010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:12:28.083 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:28.081+0000 7f4941a00640 1 --2- 172.21.15.45:0/2872801483 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56477fe29000 0x56477fe25010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:28.083 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:28.081+0000 7f4941a00640 1 -- 172.21.15.45:0/2872801483 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56477fe1ed50 con 0x56477fe1fa80 2024-04-16T17:12:28.083 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:28.081+0000 7f4942400640 1 --2- 172.21.15.45:0/2872801483 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56477fe29000 0x56477fe25010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:12:28.083 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:28.081+0000 7f4941a00640 1 --2- 172.21.15.45:0/2872801483 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56477fe1fa80 0x56477fe285c0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f49300bcb50 tx=0x7f493002ca90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=45ce51e6418dacaf server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:12:28.083 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:28.081+0000 7f492e000640 1 -- 172.21.15.45:0/2872801483 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49300dc020 con 0x56477fe1fa80 2024-04-16T17:12:28.083 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:28.081+0000 7f492e000640 1 -- 172.21.15.45:0/2872801483 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f49300befa0 con 0x56477fe1fa80 2024-04-16T17:12:28.083 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:28.081+0000 7f492e000640 1 -- 172.21.15.45:0/2872801483 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49300baf10 con 0x56477fe1fa80 2024-04-16T17:12:28.084 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:28.082+0000 7f4947233740 1 -- 172.21.15.45:0/2872801483 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56477fe1fa80 msgr2=0x56477fe285c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:28.084 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:28.082+0000 7f4947233740 1 --2- 172.21.15.45:0/2872801483 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56477fe1fa80 0x56477fe285c0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f49300bcb50 tx=0x7f493002ca90 comp rx=0 tx=0).stop 2024-04-16T17:12:28.084 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:28.082+0000 7f4947233740 1 -- 172.21.15.45:0/2872801483 shutdown_connections 2024-04-16T17:12:28.084 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:28.082+0000 7f4947233740 1 --2- 172.21.15.45:0/2872801483 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56477fe207f0 0x56477fe34870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:28.084 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:28.082+0000 7f4947233740 1 --2- 172.21.15.45:0/2872801483 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56477fe29000 0x56477fe25010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:28.084 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:28.082+0000 7f4947233740 1 --2- 172.21.15.45:0/2872801483 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56477fe1fa80 0x56477fe285c0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:28.084 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:28.082+0000 7f4947233740 1 -- 172.21.15.45:0/2872801483 >> 172.21.15.45:0/2872801483 conn(0x56477fdba5d0 msgr2=0x56477fe25e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:12:28.084 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:28.082+0000 7f4947233740 1 -- 172.21.15.45:0/2872801483 shutdown_connections 2024-04-16T17:12:28.084 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:28.082+0000 7f4947233740 1 -- 172.21.15.45:0/2872801483 wait complete. 2024-04-16T17:12:28.084 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:28.083+0000 7f4947233740 1 -- start start 2024-04-16T17:12:28.084 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:28.083+0000 7f4947233740 1 Processor -- start 2024-04-16T17:12:28.086 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:28.083+0000 7f4947233740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56477fe1fa80 0x56477fe22d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:28.086 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:28.083+0000 7f4947233740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56477fe207f0 0x56477fe23270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:28.086 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:28.083+0000 7f4947233740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56477fe29000 0x56477fe237b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:28.086 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:28.083+0000 7f4947233740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f493c1b3f60 con 0x56477fe29000 2024-04-16T17:12:28.087 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:28.083+0000 7f4947233740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f493c1b53d0 con 0x56477fe207f0 2024-04-16T17:12:28.087 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:28.083+0000 7f4947233740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f493c1b4630 con 0x56477fe1fa80 2024-04-16T17:12:28.087 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:28.084+0000 7f4947233740 10 client.0 _ll_register_callbacks cb 0x7f4934010400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:12:28.087 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:28.084+0000 7f4941000640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56477fe207f0 0x56477fe23270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:28.087 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:28.084+0000 7f4941000640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56477fe207f0 0x56477fe23270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:37756/0 (socket says 172.21.15.45:37756) 2024-04-16T17:12:28.087 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:28.084+0000 7f4941000640 1 -- 172.21.15.45:0/1831644055 learned_addr learned my addr 172.21.15.45:0/1831644055 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:12:28.087 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:28.084+0000 7f4941a00640 1 --2- 172.21.15.45:0/1831644055 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56477fe1fa80 0x56477fe22d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:28.087 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:28.084+0000 7f4942400640 1 --2- 172.21.15.45:0/1831644055 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56477fe29000 0x56477fe237b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:28.087 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:28.084+0000 7f4941000640 1 -- 172.21.15.45:0/1831644055 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56477fe1fa80 msgr2=0x56477fe22d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:12:28.087 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:28.084+0000 7f4941000640 1 --2- 172.21.15.45:0/1831644055 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56477fe1fa80 0x56477fe22d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:28.087 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:28.084+0000 7f4941000640 1 -- 172.21.15.45:0/1831644055 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56477fe29000 msgr2=0x56477fe237b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:12:28.087 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:28.084+0000 7f4941000640 1 --2- 172.21.15.45:0/1831644055 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56477fe29000 0x56477fe237b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:28.087 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:28.084+0000 7f4941000640 1 -- 172.21.15.45:0/1831644055 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56477fe21ab0 con 0x56477fe207f0 2024-04-16T17:12:28.088 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:28.084+0000 7f4942400640 1 --2- 172.21.15.45:0/1831644055 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56477fe29000 0x56477fe237b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:12:28.088 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:28.085+0000 7f4941000640 1 --2- 172.21.15.45:0/1831644055 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56477fe207f0 0x56477fe23270 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f493c0ffee0 tx=0x7f49280046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9f95f749dee6c8a1 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:12:28.088 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:28.085+0000 7f4923400640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-04-16T17:12:28.088 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:28.085+0000 7f4923400640 1 -- 172.21.15.45:0/1831644055 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4928005220 con 0x56477fe207f0 2024-04-16T17:12:28.088 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:28.085+0000 7f4923400640 1 -- 172.21.15.45:0/1831644055 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f49280053c0 con 0x56477fe207f0 2024-04-16T17:12:28.088 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:28.085+0000 7f4947233740 1 -- 172.21.15.45:0/1831644055 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f49300befa0 con 0x56477fe207f0 2024-04-16T17:12:28.088 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:28.085+0000 7f4923400640 1 -- 172.21.15.45:0/1831644055 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f4928032050 con 0x56477fe207f0 2024-04-16T17:12:28.088 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:28.085+0000 7f4947233740 10 client.27158 fetch_fsmap learned FSMap version 37 2024-04-16T17:12:28.088 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:28.085+0000 7f4947233740 1 -- 172.21.15.45:0/1831644055 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x56477fe9f8f0 con 0x56477fe207f0 2024-04-16T17:12:28.088 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:28.086+0000 7f4923400640 1 -- 172.21.15.45:0/1831644055 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f49300befa0 con 0x56477fe207f0 2024-04-16T17:12:28.089 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:28.087+0000 7f4923400640 1 -- 172.21.15.45:0/1831644055 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 37) ==== 8358+0+0 (secure 0 0 0) 0x7f492800f040 con 0x56477fe207f0 2024-04-16T17:12:28.089 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:28.087+0000 7f4947233740 10 client.27158 fetch_fsmap finished waiting for FSMap version 37 2024-04-16T17:12:28.089 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:28.087+0000 7f4947233740 10 client.27158 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T17:12:28.089 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:28.087+0000 7f4947233740 20 client.27158 populate_metadata read hostname 'smithi045' 2024-04-16T17:12:28.089 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:28.087+0000 7f4947233740 1 --2- 172.21.15.45:0/1831644055 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x56477fea0070 0x56477fea2490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:28.090 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:28.087+0000 7f4947233740 4 client.27158 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-16T17:12:28.090 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:28.087+0000 7f4947233740 1 -- 172.21.15.45:0/1831644055 --> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56477fd8c010 con 0x56477fea0070 2024-04-16T17:12:28.090 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:28.087+0000 7f4941a00640 1 --2- 172.21.15.45:0/1831644055 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x56477fea0070 0x56477fea2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:28.090 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:28.088+0000 7f4923400640 10 client.27158 ms_handle_connect on v2:172.21.15.104:6838/3660070592 2024-04-16T17:12:28.090 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:28.088+0000 7f4941a00640 1 --2- 172.21.15.45:0/1831644055 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x56477fea0070 0x56477fea2490 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a35859f56d94915 server_cookie=553971de87e123d5 in_seq=0 out_seq=0 2024-04-16T17:12:28.126 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:28.125+0000 7f4923400640 1 -- 172.21.15.45:0/1831644055 <== mds.0 v2:172.21.15.104:6838/3660070592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56477fd8c010 con 0x56477fea0070 2024-04-16T17:12:28.126 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:28.125+0000 7f4923400640 10 client.27158 handle_command_reply: tid=0 2024-04-16T17:12:28.126 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:28.125+0000 7f4947233740 1 client.27158 shutdown 2024-04-16T17:12:28.127 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:28.126+0000 7f4947233740 1 -- 172.21.15.45:0/1831644055 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56477fe207f0 msgr2=0x56477fe23270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:28.127 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:28.126+0000 7f4947233740 1 --2- 172.21.15.45:0/1831644055 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56477fe207f0 0x56477fe23270 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f493c0ffee0 tx=0x7f49280046c0 comp rx=0 tx=0).stop 2024-04-16T17:12:28.127 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:28.126+0000 7f4947233740 1 -- 172.21.15.45:0/1831644055 shutdown_connections 2024-04-16T17:12:28.127 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:28.126+0000 7f4947233740 1 --2- 172.21.15.45:0/1831644055 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56477fe29000 0x56477fe237b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:28.127 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:28.126+0000 7f4947233740 1 --2- 172.21.15.45:0/1831644055 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56477fe207f0 0x56477fe23270 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:28.127 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:28.126+0000 7f4947233740 1 --2- 172.21.15.45:0/1831644055 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x56477fea0070 0x56477fea2490 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:28.127 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:28.126+0000 7f4947233740 1 --2- 172.21.15.45:0/1831644055 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56477fe1fa80 0x56477fe22d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:28.128 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:28.126+0000 7f4923400640 0 client.27158 ms_handle_reset on v2:172.21.15.104:6838/3660070592 2024-04-16T17:12:28.128 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:28.126+0000 7f4947233740 1 -- 172.21.15.45:0/1831644055 >> 172.21.15.45:0/1831644055 conn(0x56477fdba5d0 msgr2=0x56477fe266e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:12:28.128 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:28.126+0000 7f4947233740 1 -- 172.21.15.45:0/1831644055 shutdown_connections 2024-04-16T17:12:28.128 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:28.126+0000 7f4947233740 1 -- 172.21.15.45:0/1831644055 wait complete. 2024-04-16T17:12:28.128 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-04-16T17:12:28.128 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-04-16T17:12:28.128 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-04-16T17:12:28.128 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-04-16T17:12:28.129 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:28.126+0000 7f4947233740 20 client.27158 trim_cache size 0 max 16384 2024-04-16T17:12:28.129 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:28.127+0000 7f4942e00640 1 -- 172.21.15.45:0/4078322149 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f49240a3d10 msgr2=0x7f49240a61d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:28.129 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:28.127+0000 7f4942e00640 1 --2- 172.21.15.45:0/4078322149 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f49240a3d10 0x7f49240a61d0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f49280095f0 tx=0x7f4928005f90 comp rx=0 tx=0).stop 2024-04-16T17:12:28.129 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:28.127+0000 7f4942e00640 1 -- 172.21.15.45:0/4078322149 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f493c1042a0 msgr2=0x7f493c110ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:28.129 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:28.127+0000 7f4942e00640 1 --2- 172.21.15.45:0/4078322149 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f493c1042a0 0x7f493c110ef0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f493002c240 tx=0x7f49300042f0 comp rx=0 tx=0).stop 2024-04-16T17:12:28.129 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:28.127+0000 7f4942e00640 1 -- 172.21.15.45:0/4078322149 shutdown_connections 2024-04-16T17:12:28.130 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:28.127+0000 7f4942e00640 1 --2- 172.21.15.45:0/4078322149 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f49240a3d10 0x7f49240a61d0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:28.130 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:28.127+0000 7f4942e00640 1 --2- 172.21.15.45:0/4078322149 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f493c10e750 0x7f493c1ab190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:28.130 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:28.127+0000 7f4942e00640 1 --2- 172.21.15.45:0/4078322149 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f493c109370 0x7f493c10e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:28.130 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:28.127+0000 7f4942e00640 1 --2- 172.21.15.45:0/4078322149 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f493c1042a0 0x7f493c110ef0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:28.130 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:28.127+0000 7f4942e00640 1 -- 172.21.15.45:0/4078322149 >> 172.21.15.45:0/4078322149 conn(0x7f493c0ffae0 msgr2=0x7f493c101e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:12:28.130 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:28.127+0000 7f4942e00640 1 -- 172.21.15.45:0/4078322149 shutdown_connections 2024-04-16T17:12:28.130 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:28.128+0000 7f4942e00640 1 -- 172.21.15.45:0/4078322149 wait complete. 2024-04-16T17:12:28.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.160+0000 7fe35f200640 1 -- 172.21.15.45:0/2023581863 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe3581098a0 con 0x7fe3580743d0 2024-04-16T17:12:28.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.170+0000 7fe352a00640 1 -- 172.21.15.45:0/2023581863 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) ==== 94+0+11891 (secure 0 0 0) 0x7fe34c0beca0 con 0x7fe3580743d0 2024-04-16T17:12:28.174 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-16T17:12:28.174 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":37,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/1748842443","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":1748842443},{"type":"v1","addr":"172.21.15.45:6835","nonce":1748842443}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6839/1308627051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":1308627051},{"type":"v1","addr":"172.21.15.45:6839","nonce":1308627051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/1768242489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1768242489},{"type":"v1","addr":"172.21.15.117:6833","nonce":1768242489}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6833/3558902585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":3558902585},{"type":"v1","addr":"172.21.15.104:6833","nonce":3558902585}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24478,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/1308142903","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":1308142903},{"type":"v1","addr":"172.21.15.45:6837","nonce":1308142903}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24491,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6835/3430679937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":3430679937},{"type":"v1","addr":"172.21.15.104:6835","nonce":3430679937}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":37,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-16T17:03:24.663611+0000","modified":"2024-04-16T17:12:08.516204+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24581,"mds_1":24569,"mds_2":24532},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24499":{"gid":24499,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/2224123841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2224123841},{"type":"v1","addr":"172.21.15.117:6835","nonce":2224123841}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6837/2471356815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":2471356815},{"type":"v1","addr":"172.21.15.117:6837","nonce":2471356815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24524":{"gid":24524,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6837/2647756509","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":2647756509},{"type":"v1","addr":"172.21.15.104:6837","nonce":2647756509}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24532":{"gid":24532,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":13,"addr":"172.21.15.117:6839/2378675949","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":2378675949},{"type":"v1","addr":"172.21.15.117:6839","nonce":2378675949}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24569":{"gid":24569,"name":"a","rank":1,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.45:6841/3755310822","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6840","nonce":3755310822},{"type":"v1","addr":"172.21.15.45:6841","nonce":3755310822}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24581":{"gid":24581,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.104:6839/3660070592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6838","nonce":3660070592},{"type":"v1","addr":"172.21.15.104:6839","nonce":3660070592}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24581,"qdb_cluster":[24569,24581,24532]},"id":1}]} 2024-04-16T17:12:28.175 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 37 2024-04-16T17:12:28.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.174+0000 7fe35f200640 1 -- 172.21.15.45:0/2023581863 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fe3400a3d20 msgr2=0x7fe3400a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:28.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.174+0000 7fe35f200640 1 --2- 172.21.15.45:0/2023581863 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fe3400a3d20 0x7fe3400a61e0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fe35819bfc0 tx=0x7fe34800b590 comp rx=0 tx=0).stop 2024-04-16T17:12:28.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.174+0000 7fe35f200640 1 -- 172.21.15.45:0/2023581863 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe3580743d0 msgr2=0x7fe3581a06b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:28.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.174+0000 7fe35f200640 1 --2- 172.21.15.45:0/2023581863 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe3580743d0 0x7fe3581a06b0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fe34c000c00 tx=0x7fe34c0043c0 comp rx=0 tx=0).stop 2024-04-16T17:12:28.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.174+0000 7fe35f200640 1 -- 172.21.15.45:0/2023581863 shutdown_connections 2024-04-16T17:12:28.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.174+0000 7fe35f200640 1 --2- 172.21.15.45:0/2023581863 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fe3400a3d20 0x7fe3400a61e0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:28.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.174+0000 7fe35f200640 1 --2- 172.21.15.45:0/2023581863 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe35810e6c0 0x7fe35819a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:28.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.174+0000 7fe35f200640 1 --2- 172.21.15.45:0/2023581863 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe358074cf0 0x7fe3581a0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:28.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.174+0000 7fe35f200640 1 --2- 172.21.15.45:0/2023581863 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe3580743d0 0x7fe3581a06b0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:28.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.174+0000 7fe35f200640 1 -- 172.21.15.45:0/2023581863 >> 172.21.15.45:0/2023581863 conn(0x7fe3580fd930 msgr2=0x7fe3580fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:12:28.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.174+0000 7fe35f200640 1 -- 172.21.15.45:0/2023581863 shutdown_connections 2024-04-16T17:12:28.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.174+0000 7fe35f200640 1 -- 172.21.15.45:0/2023581863 wait complete. 2024-04-16T17:12:28.195 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-16T17:12:28.268 INFO:tasks.workunit.client.0.smithi045.stdout: 1 86648 1.57 MB/sec execute 150 sec latency 51.998 ms 2024-04-16T17:12:28.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.381+0000 7fb9b6200640 1 -- 172.21.15.45:0/1460726823 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb9b010e760 msgr2=0x7fb9b0110b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:28.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.381+0000 7fb9b6200640 1 --2- 172.21.15.45:0/1460726823 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb9b010e760 0x7fb9b0110b50 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fb9a8009be0 tx=0x7fb9a802bf80 comp rx=0 tx=0).stop 2024-04-16T17:12:28.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.381+0000 7fb9b6200640 1 -- 172.21.15.45:0/1460726823 shutdown_connections 2024-04-16T17:12:28.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.381+0000 7fb9b6200640 1 --2- 172.21.15.45:0/1460726823 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb9b010e760 0x7fb9b0110b50 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:28.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.381+0000 7fb9b6200640 1 --2- 172.21.15.45:0/1460726823 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb9b0074cd0 0x7fb9b010e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:28.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.381+0000 7fb9b6200640 1 --2- 172.21.15.45:0/1460726823 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb9b00743b0 0x7fb9b0074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:28.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.381+0000 7fb9b6200640 1 -- 172.21.15.45:0/1460726823 >> 172.21.15.45:0/1460726823 conn(0x7fb9b00fd9d0 msgr2=0x7fb9b00ffdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:12:28.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.381+0000 7fb9b6200640 1 -- 172.21.15.45:0/1460726823 shutdown_connections 2024-04-16T17:12:28.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.381+0000 7fb9b6200640 1 -- 172.21.15.45:0/1460726823 wait complete. 2024-04-16T17:12:28.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.383+0000 7fb9b6200640 1 Processor -- start 2024-04-16T17:12:28.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.383+0000 7fb9b6200640 1 -- start start 2024-04-16T17:12:28.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.384+0000 7fb9b6200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb9b00743b0 0x7fb9b01a06b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:28.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.384+0000 7fb9b6200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb9b0074cd0 0x7fb9b01a0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:28.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.384+0000 7fb9b6200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb9b010e760 0x7fb9b019a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:28.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.384+0000 7fb9b6200640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fb9b0114440 con 0x7fb9b0074cd0 2024-04-16T17:12:28.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.384+0000 7fb9b6200640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fb9b01142c0 con 0x7fb9b010e760 2024-04-16T17:12:28.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.384+0000 7fb9b6200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb9b01145c0 con 0x7fb9b00743b0 2024-04-16T17:12:28.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.384+0000 7fb9b4e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb9b00743b0 0x7fb9b01a06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:28.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.384+0000 7fb9b4e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb9b00743b0 0x7fb9b01a06b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:43720/0 (socket says 172.21.15.45:43720) 2024-04-16T17:12:28.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.384+0000 7fb9b4e00640 1 -- 172.21.15.45:0/2573137115 learned_addr learned my addr 172.21.15.45:0/2573137115 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:12:28.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.384+0000 7fb9b5800640 1 --2- 172.21.15.45:0/2573137115 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb9b010e760 0x7fb9b019a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:28.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.384+0000 7fb9afe00640 1 --2- 172.21.15.45:0/2573137115 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb9b0074cd0 0x7fb9b01a0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:28.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.384+0000 7fb9b4e00640 1 -- 172.21.15.45:0/2573137115 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb9b010e760 msgr2=0x7fb9b019a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:28.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.384+0000 7fb9b4e00640 1 --2- 172.21.15.45:0/2573137115 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb9b010e760 0x7fb9b019a7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:28.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.384+0000 7fb9b4e00640 1 -- 172.21.15.45:0/2573137115 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb9b0074cd0 msgr2=0x7fb9b01a0bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:28.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.384+0000 7fb9b4e00640 1 --2- 172.21.15.45:0/2573137115 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb9b0074cd0 0x7fb9b01a0bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:28.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.384+0000 7fb9b4e00640 1 -- 172.21.15.45:0/2573137115 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9b019b080 con 0x7fb9b00743b0 2024-04-16T17:12:28.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.385+0000 7fb9b4e00640 1 --2- 172.21.15.45:0/2573137115 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb9b00743b0 0x7fb9b01a06b0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fb9a400e9f0 tx=0x7fb9a400edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:12:28.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.385+0000 7fb9ad600640 1 -- 172.21.15.45:0/2573137115 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9a400ccc0 con 0x7fb9b00743b0 2024-04-16T17:12:28.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.385+0000 7fb9b6200640 1 -- 172.21.15.45:0/2573137115 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9b019b370 con 0x7fb9b00743b0 2024-04-16T17:12:28.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.385+0000 7fb9ad600640 1 -- 172.21.15.45:0/2573137115 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb9a4010910 con 0x7fb9b00743b0 2024-04-16T17:12:28.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.385+0000 7fb9ad600640 1 -- 172.21.15.45:0/2573137115 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9a40188a0 con 0x7fb9b00743b0 2024-04-16T17:12:28.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.385+0000 7fb9b6200640 1 -- 172.21.15.45:0/2573137115 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9b01a7630 con 0x7fb9b00743b0 2024-04-16T17:12:28.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.389+0000 7fb9ba613740 1 Processor -- start 2024-04-16T17:12:28.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.389+0000 7fb9ba613740 1 -- start start 2024-04-16T17:12:28.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.390+0000 7fb9ba613740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bc6c9f2f10 0x55bc6c9f32f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:28.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.390+0000 7fb9ba613740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bc6c9f3830 0x55bc6c9fbed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:28.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.390+0000 7fb9ba613740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bc6c9fc410 0x55bc6c9fe800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:28.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.390+0000 7fb9ba613740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55bc6c958020 con 0x55bc6c9f2f10 2024-04-16T17:12:28.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.390+0000 7fb9ba613740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55bc6c8fe460 con 0x55bc6c9fc410 2024-04-16T17:12:28.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.390+0000 7fb9ba613740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55bc6c953fe0 con 0x55bc6c9f3830 2024-04-16T17:12:28.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.390+0000 7fb9b5800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bc6c9f2f10 0x55bc6c9f32f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:28.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.390+0000 7fb9b5800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bc6c9f2f10 0x55bc6c9f32f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35406/0 (socket says 172.21.15.45:35406) 2024-04-16T17:12:28.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.390+0000 7fb9b5800640 1 -- 172.21.15.45:0/1252199773 learned_addr learned my addr 172.21.15.45:0/1252199773 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:12:28.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.390+0000 7fb9afe00640 1 --2- 172.21.15.45:0/1252199773 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bc6c9fc410 0x55bc6c9fe800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:28.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.391+0000 7fb9ad600640 1 -- 172.21.15.45:0/2573137115 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fb9a4010ac0 con 0x7fb9b00743b0 2024-04-16T17:12:28.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.391+0000 7fb9b5800640 1 -- 172.21.15.45:0/1252199773 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bc6c9f3830 msgr2=0x55bc6c9fbed0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-16T17:12:28.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.391+0000 7fb9b5800640 1 --2- 172.21.15.45:0/1252199773 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bc6c9f3830 0x55bc6c9fbed0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:28.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.391+0000 7fb9b5800640 1 -- 172.21.15.45:0/1252199773 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bc6c9fc410 msgr2=0x55bc6c9fe800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:12:28.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.391+0000 7fb9b5800640 1 --2- 172.21.15.45:0/1252199773 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bc6c9fc410 0x55bc6c9fe800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:28.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.391+0000 7fb9b5800640 1 -- 172.21.15.45:0/1252199773 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bc6c906040 con 0x55bc6c9f2f10 2024-04-16T17:12:28.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.391+0000 7fb9b5800640 1 --2- 172.21.15.45:0/1252199773 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bc6c9f2f10 0x55bc6c9f32f0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fb9a8009be0 tx=0x7fb9a8033470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac6346002186fa04 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:12:28.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.392+0000 7fb9a2000640 1 -- 172.21.15.45:0/1252199773 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9a8043030 con 0x55bc6c9f2f10 2024-04-16T17:12:28.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.392+0000 7fb9a2000640 1 -- 172.21.15.45:0/1252199773 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb9a8003e50 con 0x55bc6c9f2f10 2024-04-16T17:12:28.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.392+0000 7fb9a2000640 1 -- 172.21.15.45:0/1252199773 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9a8004320 con 0x55bc6c9f2f10 2024-04-16T17:12:28.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.392+0000 7fb9ba613740 1 -- 172.21.15.45:0/1252199773 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bc6c9f2f10 msgr2=0x55bc6c9f32f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:28.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.392+0000 7fb9ba613740 1 --2- 172.21.15.45:0/1252199773 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bc6c9f2f10 0x55bc6c9f32f0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fb9a8009be0 tx=0x7fb9a8033470 comp rx=0 tx=0).stop 2024-04-16T17:12:28.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.393+0000 7fb9ba613740 1 -- 172.21.15.45:0/1252199773 shutdown_connections 2024-04-16T17:12:28.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.393+0000 7fb9ba613740 1 --2- 172.21.15.45:0/1252199773 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bc6c9f3830 0x55bc6c9fbed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:28.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.393+0000 7fb9ba613740 1 --2- 172.21.15.45:0/1252199773 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bc6c9fc410 0x55bc6c9fe800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:28.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.393+0000 7fb9ba613740 1 --2- 172.21.15.45:0/1252199773 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bc6c9f2f10 0x55bc6c9f32f0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:28.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.393+0000 7fb9ba613740 1 -- 172.21.15.45:0/1252199773 >> 172.21.15.45:0/1252199773 conn(0x55bc6c9ed640 msgr2=0x55bc6c9eda20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:12:28.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.393+0000 7fb9ba613740 1 -- 172.21.15.45:0/1252199773 shutdown_connections 2024-04-16T17:12:28.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.393+0000 7fb9ba613740 1 -- 172.21.15.45:0/1252199773 wait complete. 2024-04-16T17:12:28.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.393+0000 7fb9ad600640 1 --2- 172.21.15.45:0/2573137115 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fb9980a3c30 0x7fb9980a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:28.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.393+0000 7fb9ad600640 1 -- 172.21.15.45:0/2573137115 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fb9a40d0ff0 con 0x7fb9b00743b0 2024-04-16T17:12:28.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.393+0000 7fb9ba613740 1 -- start start 2024-04-16T17:12:28.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.393+0000 7fb9afe00640 1 --2- 172.21.15.45:0/2573137115 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fb9980a3c30 0x7fb9980a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:28.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.394+0000 7fb9ba613740 1 Processor -- start 2024-04-16T17:12:28.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.394+0000 7fb9ba613740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bc6c9f2f10 0x55bc6c9f9580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:28.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.394+0000 7fb9ba613740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bc6c9f3830 0x55bc6c9f9ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:28.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.394+0000 7fb9ba613740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bc6c9fc410 0x55bc6c9fa000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:28.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.394+0000 7fb9ba613740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55bc6c9ff1a0 con 0x55bc6c9fc410 2024-04-16T17:12:28.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.394+0000 7fb9ba613740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55bc6c911830 con 0x55bc6c9f2f10 2024-04-16T17:12:28.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.394+0000 7fb9ba613740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55bc6c9ff320 con 0x55bc6c9f3830 2024-04-16T17:12:28.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.395+0000 7fb9ba613740 10 client.0 _ll_register_callbacks cb 0x55bc6c919150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:12:28.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.395+0000 7fb9b4e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bc6c9f2f10 0x55bc6c9f9580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:28.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.395+0000 7fb9b4e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bc6c9f2f10 0x55bc6c9f9580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:37784/0 (socket says 172.21.15.45:37784) 2024-04-16T17:12:28.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.395+0000 7fb9b4e00640 1 -- 172.21.15.45:0/3475818095 learned_addr learned my addr 172.21.15.45:0/3475818095 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:12:28.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.395+0000 7fb9afe00640 1 --2- 172.21.15.45:0/3475818095 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bc6c9f3830 0x55bc6c9f9ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:28.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.395+0000 7fb9b5800640 1 --2- 172.21.15.45:0/3475818095 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bc6c9fc410 0x55bc6c9fa000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:28.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.395+0000 7fb9afe00640 1 --2- 172.21.15.45:0/2573137115 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fb9980a3c30 0x7fb9980a60f0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x55bc6c9ee7d0 tx=0x7fb99c009260 comp rx=0 tx=0).ready entity=mgr.25411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:12:28.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.395+0000 7fb9b4e00640 1 -- 172.21.15.45:0/3475818095 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bc6c9f3830 msgr2=0x55bc6c9f9ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:12:28.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.395+0000 7fb9b4e00640 1 --2- 172.21.15.45:0/3475818095 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bc6c9f3830 0x55bc6c9f9ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:28.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.395+0000 7fb9b4e00640 1 -- 172.21.15.45:0/3475818095 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bc6c9fc410 msgr2=0x55bc6c9fa000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:12:28.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.395+0000 7fb9b4e00640 1 --2- 172.21.15.45:0/3475818095 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bc6c9fc410 0x55bc6c9fa000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:28.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.395+0000 7fb9b4e00640 1 -- 172.21.15.45:0/3475818095 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bc6c915920 con 0x55bc6c9f2f10 2024-04-16T17:12:28.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.395+0000 7fb9afe00640 1 --2- 172.21.15.45:0/3475818095 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bc6c9f3830 0x55bc6c9f9ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:12:28.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.395+0000 7fb9b4e00640 1 --2- 172.21.15.45:0/3475818095 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bc6c9f2f10 0x55bc6c9f9580 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fb9a400e9c0 tx=0x7fb9a409c810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e3ab586a7dc9ca0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:12:28.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.395+0000 7fb996a00640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-04-16T17:12:28.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.396+0000 7fb996a00640 1 -- 172.21.15.45:0/3475818095 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9a40b6060 con 0x55bc6c9f2f10 2024-04-16T17:12:28.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.396+0000 7fb996a00640 1 -- 172.21.15.45:0/3475818095 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fb9a409c030 con 0x55bc6c9f2f10 2024-04-16T17:12:28.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.396+0000 7fb9ba613740 1 -- 172.21.15.45:0/3475818095 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb9a8003e50 con 0x55bc6c9f2f10 2024-04-16T17:12:28.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.396+0000 7fb996a00640 1 -- 172.21.15.45:0/3475818095 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fb9a40ce810 con 0x55bc6c9f2f10 2024-04-16T17:12:28.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.396+0000 7fb9ba613740 10 client.27170 fetch_fsmap learned FSMap version 37 2024-04-16T17:12:28.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.396+0000 7fb9ba613740 1 -- 172.21.15.45:0/3475818095 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55bc6c8e37f0 con 0x55bc6c9f2f10 2024-04-16T17:12:28.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.396+0000 7fb996a00640 1 -- 172.21.15.45:0/3475818095 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb9a8003e50 con 0x55bc6c9f2f10 2024-04-16T17:12:28.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.397+0000 7fb996a00640 1 -- 172.21.15.45:0/3475818095 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 37) ==== 8358+0+0 (secure 0 0 0) 0x7fb9a40ceae0 con 0x55bc6c9f2f10 2024-04-16T17:12:28.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.397+0000 7fb9ba613740 10 client.27170 fetch_fsmap finished waiting for FSMap version 37 2024-04-16T17:12:28.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.397+0000 7fb9ba613740 10 client.27170 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-16T17:12:28.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.397+0000 7fb9ba613740 20 client.27170 populate_metadata read hostname 'smithi045' 2024-04-16T17:12:28.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.397+0000 7fb9ba613740 1 --2- 172.21.15.45:0/3475818095 >> [v2:172.21.15.117:6838/2378675949,v1:172.21.15.117:6839/2378675949] conn(0x55bc6ca0c870 0x55bc6ca0ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:28.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.397+0000 7fb9ba613740 4 client.27170 mds_command: new command op to 24532 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:12:28.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.397+0000 7fb9ba613740 1 -- 172.21.15.45:0/3475818095 --> [v2:172.21.15.117:6838/2378675949,v1:172.21.15.117:6839/2378675949] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bc6c965a40 con 0x55bc6ca0c870 2024-04-16T17:12:28.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.397+0000 7fb9afe00640 1 --2- 172.21.15.45:0/3475818095 >> [v2:172.21.15.117:6838/2378675949,v1:172.21.15.117:6839/2378675949] conn(0x55bc6ca0c870 0x55bc6ca0ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:28.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.398+0000 7fb9afe00640 1 --2- 172.21.15.45:0/3475818095 >> [v2:172.21.15.117:6838/2378675949,v1:172.21.15.117:6839/2378675949] conn(0x55bc6ca0c870 0x55bc6ca0ecb0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=66ffb4ba1f671da9 server_cookie=c985872294c316ac in_seq=0 out_seq=0 2024-04-16T17:12:28.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.398+0000 7fb996a00640 10 client.27170 ms_handle_connect on v2:172.21.15.117:6838/2378675949 2024-04-16T17:12:28.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.398+0000 7fb996a00640 1 -- 172.21.15.45:0/3475818095 <== mds.2 v2:172.21.15.117:6838/2378675949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55bc6c965a40 con 0x55bc6ca0c870 2024-04-16T17:12:28.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.398+0000 7fb996a00640 10 client.27170 handle_command_reply: tid=0 2024-04-16T17:12:28.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.399+0000 7fb9ba613740 1 client.27170 shutdown 2024-04-16T17:12:28.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.399+0000 7fb9ba613740 1 -- 172.21.15.45:0/3475818095 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bc6c9f2f10 msgr2=0x55bc6c9f9580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:28.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.399+0000 7fb9ba613740 1 --2- 172.21.15.45:0/3475818095 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bc6c9f2f10 0x55bc6c9f9580 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fb9a400e9c0 tx=0x7fb9a409c810 comp rx=0 tx=0).stop 2024-04-16T17:12:28.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.399+0000 7fb9ba613740 1 -- 172.21.15.45:0/3475818095 shutdown_connections 2024-04-16T17:12:28.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.399+0000 7fb9ba613740 1 --2- 172.21.15.45:0/3475818095 >> [v2:172.21.15.117:6838/2378675949,v1:172.21.15.117:6839/2378675949] conn(0x55bc6ca0c870 0x55bc6ca0ecb0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:28.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.399+0000 7fb9ba613740 1 --2- 172.21.15.45:0/3475818095 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bc6c9f3830 0x55bc6c9f9ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:28.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.399+0000 7fb9ba613740 1 --2- 172.21.15.45:0/3475818095 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bc6c9fc410 0x55bc6c9fa000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:28.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.399+0000 7fb9ba613740 1 --2- 172.21.15.45:0/3475818095 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bc6c9f2f10 0x55bc6c9f9580 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:28.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.399+0000 7fb996a00640 0 client.27170 ms_handle_reset on v2:172.21.15.117:6838/2378675949 2024-04-16T17:12:28.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.399+0000 7fb9ba613740 1 -- 172.21.15.45:0/3475818095 >> 172.21.15.45:0/3475818095 conn(0x55bc6c9ed640 msgr2=0x55bc6c9ef390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:12:28.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.399+0000 7fb9ba613740 1 -- 172.21.15.45:0/3475818095 shutdown_connections 2024-04-16T17:12:28.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.399+0000 7fb9ba613740 1 -- 172.21.15.45:0/3475818095 wait complete. 2024-04-16T17:12:28.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.399+0000 7fb9ba613740 20 client.27170 trim_cache size 0 max 16384 2024-04-16T17:12:28.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.418+0000 7fb9ba613740 1 Processor -- start 2024-04-16T17:12:28.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.418+0000 7fb9ba613740 1 -- start start 2024-04-16T17:12:28.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.418+0000 7fb9ba613740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bc6c9f2f10 0x55bc6c9fbcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:28.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.418+0000 7fb9ba613740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bc6c9f3830 0x55bc6ca08060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:28.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.418+0000 7fb9ba613740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bc6c9fc410 0x55bc6ca14580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:28.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.418+0000 7fb9ba613740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fb9b01b3d80 con 0x55bc6c9f3830 2024-04-16T17:12:28.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.418+0000 7fb9ba613740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fb9b01b15c0 con 0x55bc6c9fc410 2024-04-16T17:12:28.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.418+0000 7fb9ba613740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb9b01af3f0 con 0x55bc6c9f2f10 2024-04-16T17:12:28.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.418+0000 7fb9afe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bc6c9f3830 0x55bc6ca08060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:28.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.418+0000 7fb9b4e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bc6c9f2f10 0x55bc6c9fbcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:28.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.419+0000 7fb9b4e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bc6c9f2f10 0x55bc6c9fbcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:43758/0 (socket says 172.21.15.45:43758) 2024-04-16T17:12:28.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.419+0000 7fb9b4e00640 1 -- 172.21.15.45:0/16639550 learned_addr learned my addr 172.21.15.45:0/16639550 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:12:28.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.419+0000 7fb9afe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bc6c9f3830 0x55bc6ca08060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35412/0 (socket says 172.21.15.45:35412) 2024-04-16T17:12:28.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.419+0000 7fb9b5800640 1 --2- 172.21.15.45:0/16639550 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bc6c9fc410 0x55bc6ca14580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:28.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.419+0000 7fb9b4e00640 1 -- 172.21.15.45:0/16639550 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bc6c9fc410 msgr2=0x55bc6ca14580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:12:28.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.419+0000 7fb9b4e00640 1 --2- 172.21.15.45:0/16639550 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bc6c9fc410 0x55bc6ca14580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:28.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.419+0000 7fb9b4e00640 1 -- 172.21.15.45:0/16639550 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bc6c9f3830 msgr2=0x55bc6ca08060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:12:28.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.419+0000 7fb9b4e00640 1 --2- 172.21.15.45:0/16639550 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bc6c9f3830 0x55bc6ca08060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:28.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.419+0000 7fb9b4e00640 1 -- 172.21.15.45:0/16639550 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bc6c66a3f0 con 0x55bc6c9f2f10 2024-04-16T17:12:28.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.419+0000 7fb9b5800640 1 --2- 172.21.15.45:0/16639550 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bc6c9fc410 0x55bc6ca14580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:12:28.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.419+0000 7fb9b4e00640 1 --2- 172.21.15.45:0/16639550 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bc6c9f2f10 0x55bc6c9fbcb0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fb9a409cb70 tx=0x7fb9a4009700 comp rx=0 tx=0).ready entity=mon.1 client_cookie=64ba623cfe274c33 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:12:28.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.420+0000 7fb9a1600640 1 -- 172.21.15.45:0/16639550 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9a409d2e0 con 0x55bc6c9f2f10 2024-04-16T17:12:28.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.420+0000 7fb9a1600640 1 -- 172.21.15.45:0/16639550 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb9a40a0f90 con 0x55bc6c9f2f10 2024-04-16T17:12:28.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.420+0000 7fb9a1600640 1 -- 172.21.15.45:0/16639550 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9a409c030 con 0x55bc6c9f2f10 2024-04-16T17:12:28.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.420+0000 7fb9ba613740 1 -- 172.21.15.45:0/16639550 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bc6c9f2f10 msgr2=0x55bc6c9fbcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:28.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.420+0000 7fb9ba613740 1 --2- 172.21.15.45:0/16639550 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bc6c9f2f10 0x55bc6c9fbcb0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fb9a409cb70 tx=0x7fb9a4009700 comp rx=0 tx=0).stop 2024-04-16T17:12:28.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.421+0000 7fb9ba613740 1 -- 172.21.15.45:0/16639550 shutdown_connections 2024-04-16T17:12:28.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.421+0000 7fb9ba613740 1 --2- 172.21.15.45:0/16639550 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bc6c9fc410 0x55bc6ca14580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:28.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.421+0000 7fb9ba613740 1 --2- 172.21.15.45:0/16639550 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bc6c9f3830 0x55bc6ca08060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:28.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.421+0000 7fb9ba613740 1 --2- 172.21.15.45:0/16639550 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bc6c9f2f10 0x55bc6c9fbcb0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:28.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.421+0000 7fb9ba613740 1 -- 172.21.15.45:0/16639550 >> 172.21.15.45:0/16639550 conn(0x55bc6c9ed640 msgr2=0x55bc6c9ec700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:12:28.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.421+0000 7fb9ba613740 1 -- 172.21.15.45:0/16639550 shutdown_connections 2024-04-16T17:12:28.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.421+0000 7fb9ba613740 1 -- 172.21.15.45:0/16639550 wait complete. 2024-04-16T17:12:28.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.422+0000 7fb9ba613740 1 -- start start 2024-04-16T17:12:28.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.422+0000 7fb9ba613740 1 Processor -- start 2024-04-16T17:12:28.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.422+0000 7fb9ba613740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bc6c9f2f10 0x55bc6c9f6350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:28.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.422+0000 7fb9ba613740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bc6c9f3830 0x55bc6c9f6890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:28.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.422+0000 7fb9ba613740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bc6c9fc410 0x55bc6c9f6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:28.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.422+0000 7fb9ba613740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fb9b01afa80 con 0x55bc6c9f2f10 2024-04-16T17:12:28.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.422+0000 7fb9ba613740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fb9b01b0ef0 con 0x55bc6c9fc410 2024-04-16T17:12:28.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.422+0000 7fb9ba613740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb9b01b0150 con 0x55bc6c9f3830 2024-04-16T17:12:28.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.422+0000 7fb9ba613740 10 client.0 _ll_register_callbacks cb 0x7fb9a4010e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:12:28.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.423+0000 7fb9afe00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bc6c9f3830 0x55bc6c9f6890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:28.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.423+0000 7fb9afe00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bc6c9f3830 0x55bc6c9f6890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:43770/0 (socket says 172.21.15.45:43770) 2024-04-16T17:12:28.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.423+0000 7fb9afe00640 1 -- 172.21.15.45:0/1537729668 learned_addr learned my addr 172.21.15.45:0/1537729668 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:12:28.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.423+0000 7fb9b4e00640 1 --2- 172.21.15.45:0/1537729668 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bc6c9f2f10 0x55bc6c9f6350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:28.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.423+0000 7fb9b5800640 1 --2- 172.21.15.45:0/1537729668 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bc6c9fc410 0x55bc6c9f6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:28.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.423+0000 7fb9afe00640 1 -- 172.21.15.45:0/1537729668 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bc6c9fc410 msgr2=0x55bc6c9f6dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:12:28.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.423+0000 7fb9afe00640 1 --2- 172.21.15.45:0/1537729668 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bc6c9fc410 0x55bc6c9f6dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:28.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.423+0000 7fb9afe00640 1 -- 172.21.15.45:0/1537729668 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bc6c9f2f10 msgr2=0x55bc6c9f6350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:12:28.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.423+0000 7fb9afe00640 1 --2- 172.21.15.45:0/1537729668 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bc6c9f2f10 0x55bc6c9f6350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:28.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.423+0000 7fb9afe00640 1 -- 172.21.15.45:0/1537729668 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bc6c7347d0 con 0x55bc6c9f3830 2024-04-16T17:12:28.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.423+0000 7fb9afe00640 1 --2- 172.21.15.45:0/1537729668 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bc6c9f3830 0x55bc6c9f6890 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fb9b019bfc0 tx=0x7fb99c0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8148a2eb0a5fa916 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:12:28.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.423+0000 7fb996a00640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-04-16T17:12:28.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.424+0000 7fb996a00640 1 -- 172.21.15.45:0/1537729668 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb99c02d070 con 0x55bc6c9f3830 2024-04-16T17:12:28.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.424+0000 7fb996a00640 1 -- 172.21.15.45:0/1537729668 <== mon.1 v2:172.21.15.117:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fb99c01cbc0 con 0x55bc6c9f3830 2024-04-16T17:12:28.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.424+0000 7fb9ba613740 1 -- 172.21.15.45:0/1537729668 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb9a40a0f90 con 0x55bc6c9f3830 2024-04-16T17:12:28.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.425+0000 7fb996a00640 1 -- 172.21.15.45:0/1537729668 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fb99c01cdc0 con 0x55bc6c9f3830 2024-04-16T17:12:28.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.425+0000 7fb9ba613740 10 client.27121 fetch_fsmap learned FSMap version 37 2024-04-16T17:12:28.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.425+0000 7fb9ba613740 1 -- 172.21.15.45:0/1537729668 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55bc6ca7b890 con 0x55bc6c9f3830 2024-04-16T17:12:28.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.425+0000 7fb996a00640 1 -- 172.21.15.45:0/1537729668 <== mon.1 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb9a40a0f90 con 0x55bc6c9f3830 2024-04-16T17:12:28.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.426+0000 7fb996a00640 1 -- 172.21.15.45:0/1537729668 <== mon.1 v2:172.21.15.117:3300/0 5 ==== fsmap(e 37) ==== 8358+0+0 (secure 0 0 0) 0x7fb99c032aa0 con 0x55bc6c9f3830 2024-04-16T17:12:28.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.426+0000 7fb9ba613740 10 client.27121 fetch_fsmap finished waiting for FSMap version 37 2024-04-16T17:12:28.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.426+0000 7fb9ba613740 10 client.27121 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-16T17:12:28.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.426+0000 7fb9ba613740 20 client.27121 populate_metadata read hostname 'smithi045' 2024-04-16T17:12:28.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.426+0000 7fb9ba613740 1 --2- 172.21.15.45:0/1537729668 >> [v2:172.21.15.117:6838/2378675949,v1:172.21.15.117:6839/2378675949] conn(0x55bc6ca7c050 0x55bc6ca7e470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:28.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.426+0000 7fb9ba613740 4 client.27121 mds_command: new command op to 24532 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T17:12:28.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.426+0000 7fb9ba613740 1 -- 172.21.15.45:0/1537729668 --> [v2:172.21.15.117:6838/2378675949,v1:172.21.15.117:6839/2378675949] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55bc6c95f8b0 con 0x55bc6ca7c050 2024-04-16T17:12:28.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.426+0000 7fb9b4e00640 1 --2- 172.21.15.45:0/1537729668 >> [v2:172.21.15.117:6838/2378675949,v1:172.21.15.117:6839/2378675949] conn(0x55bc6ca7c050 0x55bc6ca7e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:28.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.426+0000 7fb996a00640 10 client.27121 ms_handle_connect on v2:172.21.15.117:6838/2378675949 2024-04-16T17:12:28.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.426+0000 7fb9b4e00640 1 --2- 172.21.15.45:0/1537729668 >> [v2:172.21.15.117:6838/2378675949,v1:172.21.15.117:6839/2378675949] conn(0x55bc6ca7c050 0x55bc6ca7e470 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4b2ecb36ad84b36 server_cookie=e7fece227326fda in_seq=0 out_seq=0 2024-04-16T17:12:28.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.427+0000 7fb996a00640 1 -- 172.21.15.45:0/1537729668 <== mds.2 v2:172.21.15.117:6838/2378675949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55bc6c95f8b0 con 0x55bc6ca7c050 2024-04-16T17:12:28.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.427+0000 7fb996a00640 10 client.27121 handle_command_reply: tid=0 2024-04-16T17:12:28.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.427+0000 7fb9ba613740 1 client.27121 shutdown 2024-04-16T17:12:28.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.429+0000 7fb9ba613740 1 -- 172.21.15.45:0/1537729668 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bc6c9f3830 msgr2=0x55bc6c9f6890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:28.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.429+0000 7fb9ba613740 1 --2- 172.21.15.45:0/1537729668 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bc6c9f3830 0x55bc6c9f6890 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fb9b019bfc0 tx=0x7fb99c0046c0 comp rx=0 tx=0).stop 2024-04-16T17:12:28.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.429+0000 7fb9ba613740 1 -- 172.21.15.45:0/1537729668 shutdown_connections 2024-04-16T17:12:28.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.429+0000 7fb9ba613740 1 --2- 172.21.15.45:0/1537729668 >> [v2:172.21.15.117:6838/2378675949,v1:172.21.15.117:6839/2378675949] conn(0x55bc6ca7c050 0x55bc6ca7e470 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:28.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.429+0000 7fb996a00640 0 client.27121 ms_handle_reset on v2:172.21.15.117:6838/2378675949 2024-04-16T17:12:28.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.429+0000 7fb9ba613740 1 --2- 172.21.15.45:0/1537729668 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bc6c9f3830 0x55bc6c9f6890 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:28.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.429+0000 7fb9ba613740 1 --2- 172.21.15.45:0/1537729668 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bc6c9fc410 0x55bc6c9f6dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:28.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.429+0000 7fb9ba613740 1 --2- 172.21.15.45:0/1537729668 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bc6c9f2f10 0x55bc6c9f6350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:28.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.429+0000 7fb9ba613740 1 -- 172.21.15.45:0/1537729668 >> 172.21.15.45:0/1537729668 conn(0x55bc6c9ed640 msgr2=0x55bc6c9f9520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:12:28.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.429+0000 7fb9ba613740 1 -- 172.21.15.45:0/1537729668 shutdown_connections 2024-04-16T17:12:28.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.429+0000 7fb9ba613740 1 -- 172.21.15.45:0/1537729668 wait complete. 2024-04-16T17:12:28.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.429+0000 7fb9ba613740 20 client.27121 trim_cache size 0 max 16384 2024-04-16T17:12:28.431 INFO:teuthology.orchestra.run.smithi045.stdout:[] 2024-04-16T17:12:28.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.430+0000 7fb9b6200640 1 -- 172.21.15.45:0/2573137115 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fb9980a3c30 msgr2=0x7fb9980a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:28.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.430+0000 7fb9b6200640 1 --2- 172.21.15.45:0/2573137115 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fb9980a3c30 0x7fb9980a60f0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x55bc6c9ee7d0 tx=0x7fb99c009260 comp rx=0 tx=0).stop 2024-04-16T17:12:28.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.430+0000 7fb9b6200640 1 -- 172.21.15.45:0/2573137115 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb9b00743b0 msgr2=0x7fb9b01a06b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:28.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.430+0000 7fb9b6200640 1 --2- 172.21.15.45:0/2573137115 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb9b00743b0 0x7fb9b01a06b0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fb9a400e9f0 tx=0x7fb9a400edf0 comp rx=0 tx=0).stop 2024-04-16T17:12:28.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.430+0000 7fb9b6200640 1 -- 172.21.15.45:0/2573137115 shutdown_connections 2024-04-16T17:12:28.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.430+0000 7fb9b6200640 1 --2- 172.21.15.45:0/2573137115 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fb9980a3c30 0x7fb9980a60f0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:28.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.430+0000 7fb9b6200640 1 --2- 172.21.15.45:0/2573137115 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb9b010e760 0x7fb9b019a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:28.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.430+0000 7fb9b6200640 1 --2- 172.21.15.45:0/2573137115 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb9b0074cd0 0x7fb9b01a0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:28.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.430+0000 7fb9b6200640 1 --2- 172.21.15.45:0/2573137115 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb9b00743b0 0x7fb9b01a06b0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:28.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.430+0000 7fb9b6200640 1 -- 172.21.15.45:0/2573137115 >> 172.21.15.45:0/2573137115 conn(0x7fb9b00fd9d0 msgr2=0x7fb9b00fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:12:28.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.430+0000 7fb9b6200640 1 -- 172.21.15.45:0/2573137115 shutdown_connections 2024-04-16T17:12:28.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.431+0000 7fb9b6200640 1 -- 172.21.15.45:0/2573137115 wait complete. 2024-04-16T17:12:28.447 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-16T17:12:28.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.685+0000 7fe423c00640 1 -- 172.21.15.45:0/354888240 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe41c106210 msgr2=0x7fe41c10dde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:28.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.685+0000 7fe423c00640 1 --2- 172.21.15.45:0/354888240 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe41c106210 0x7fe41c10dde0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fe414009ca0 tx=0x7fe41402c100 comp rx=0 tx=0).stop 2024-04-16T17:12:28.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.685+0000 7fe423c00640 1 -- 172.21.15.45:0/354888240 shutdown_connections 2024-04-16T17:12:28.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.685+0000 7fe423c00640 1 --2- 172.21.15.45:0/354888240 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe41c106210 0x7fe41c10dde0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:28.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.685+0000 7fe423c00640 1 --2- 172.21.15.45:0/354888240 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe41c101af0 0x7fe41c105b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:28.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.685+0000 7fe423c00640 1 --2- 172.21.15.45:0/354888240 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe41c101140 0x7fe41c101520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:28.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.685+0000 7fe423c00640 1 -- 172.21.15.45:0/354888240 >> 172.21.15.45:0/354888240 conn(0x7fe41c077280 msgr2=0x7fe41c077690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:12:28.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.685+0000 7fe423c00640 1 -- 172.21.15.45:0/354888240 shutdown_connections 2024-04-16T17:12:28.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.685+0000 7fe423c00640 1 -- 172.21.15.45:0/354888240 wait complete. 2024-04-16T17:12:28.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.686+0000 7fe423c00640 1 Processor -- start 2024-04-16T17:12:28.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.686+0000 7fe423c00640 1 -- start start 2024-04-16T17:12:28.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.687+0000 7fe423c00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe41c101140 0x7fe41c1a0730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:28.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.687+0000 7fe423c00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe41c106210 0x7fe41c1a0c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:28.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.687+0000 7fe423c00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe41c19abb0 0x7fe41c19b060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:28.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.687+0000 7fe423c00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fe41c1112d0 con 0x7fe41c19abb0 2024-04-16T17:12:28.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.687+0000 7fe423c00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fe41c111150 con 0x7fe41c101140 2024-04-16T17:12:28.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.687+0000 7fe423c00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fe41c111450 con 0x7fe41c106210 2024-04-16T17:12:28.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.687+0000 7fe421200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe41c101140 0x7fe41c1a0730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:28.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.687+0000 7fe421200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe41c101140 0x7fe41c1a0730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:37824/0 (socket says 172.21.15.45:37824) 2024-04-16T17:12:28.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.687+0000 7fe41be00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe41c106210 0x7fe41c1a0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:28.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.687+0000 7fe421200640 1 -- 172.21.15.45:0/274835780 learned_addr learned my addr 172.21.15.45:0/274835780 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:12:28.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.687+0000 7fe421c00640 1 --2- 172.21.15.45:0/274835780 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe41c19abb0 0x7fe41c19b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:28.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.688+0000 7fe41be00640 1 -- 172.21.15.45:0/274835780 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe41c101140 msgr2=0x7fe41c1a0730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:28.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.688+0000 7fe41be00640 1 --2- 172.21.15.45:0/274835780 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe41c101140 0x7fe41c1a0730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:28.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.688+0000 7fe41be00640 1 -- 172.21.15.45:0/274835780 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe41c19abb0 msgr2=0x7fe41c19b060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:28.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.688+0000 7fe41be00640 1 --2- 172.21.15.45:0/274835780 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe41c19abb0 0x7fe41c19b060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:28.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.688+0000 7fe41be00640 1 -- 172.21.15.45:0/274835780 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe41c19b6d0 con 0x7fe41c106210 2024-04-16T17:12:28.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.688+0000 7fe41be00640 1 --2- 172.21.15.45:0/274835780 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe41c106210 0x7fe41c1a0c70 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fe408005fd0 tx=0x7fe40800dde0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:12:28.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.695+0000 7fe419600640 1 -- 172.21.15.45:0/274835780 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe408009c00 con 0x7fe41c106210 2024-04-16T17:12:28.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.695+0000 7fe419600640 1 -- 172.21.15.45:0/274835780 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe408009da0 con 0x7fe41c106210 2024-04-16T17:12:28.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.695+0000 7fe419600640 1 -- 172.21.15.45:0/274835780 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe408010460 con 0x7fe41c106210 2024-04-16T17:12:28.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.695+0000 7fe423c00640 1 -- 172.21.15.45:0/274835780 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe41c19b9c0 con 0x7fe41c106210 2024-04-16T17:12:28.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.696+0000 7fe423c00640 1 -- 172.21.15.45:0/274835780 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe41c1a7990 con 0x7fe41c106210 2024-04-16T17:12:28.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.699+0000 7fe4269a8740 1 Processor -- start 2024-04-16T17:12:28.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.699+0000 7fe4269a8740 1 -- start start 2024-04-16T17:12:28.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.699+0000 7fe4269a8740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55cec7f5ed50 0x55cec7f5f130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:28.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.699+0000 7fe4269a8740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cec7f5f670 0x55cec7f67d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:28.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.699+0000 7fe4269a8740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cec7f68250 0x55cec7f6a640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:28.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.699+0000 7fe4269a8740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55cec7ec4020 con 0x55cec7f5f670 2024-04-16T17:12:28.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.699+0000 7fe4269a8740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55cec7e6a460 con 0x55cec7f5ed50 2024-04-16T17:12:28.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.699+0000 7fe4269a8740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55cec7ebffe0 con 0x55cec7f68250 2024-04-16T17:12:28.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.700+0000 7fe421c00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55cec7f5ed50 0x55cec7f5f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:28.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.700+0000 7fe421c00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55cec7f5ed50 0x55cec7f5f130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:37830/0 (socket says 172.21.15.45:37830) 2024-04-16T17:12:28.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.700+0000 7fe421c00640 1 -- 172.21.15.45:0/960873606 learned_addr learned my addr 172.21.15.45:0/960873606 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:12:28.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.700+0000 7fe421200640 1 --2- 172.21.15.45:0/960873606 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cec7f5f670 0x55cec7f67d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:28.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.700+0000 7fe41be00640 1 --2- 172.21.15.45:0/960873606 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cec7f68250 0x55cec7f6a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:28.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.700+0000 7fe421c00640 1 -- 172.21.15.45:0/960873606 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cec7f68250 msgr2=0x55cec7f6a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:12:28.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.700+0000 7fe421c00640 1 --2- 172.21.15.45:0/960873606 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cec7f68250 0x55cec7f6a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:28.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.700+0000 7fe421c00640 1 -- 172.21.15.45:0/960873606 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cec7f5f670 msgr2=0x55cec7f67d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:12:28.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.700+0000 7fe421c00640 1 --2- 172.21.15.45:0/960873606 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cec7f5f670 0x55cec7f67d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:28.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.700+0000 7fe421c00640 1 -- 172.21.15.45:0/960873606 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cec7e72040 con 0x55cec7f5ed50 2024-04-16T17:12:28.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.701+0000 7fe421c00640 1 --2- 172.21.15.45:0/960873606 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55cec7f5ed50 0x55cec7f5f130 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fe414009ca0 tx=0x7fe414009900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=70fbaa6cf0f6c6a5 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:12:28.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.701+0000 7fe40e000640 1 -- 172.21.15.45:0/960873606 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe414043070 con 0x55cec7f5ed50 2024-04-16T17:12:28.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.701+0000 7fe40e000640 1 -- 172.21.15.45:0/960873606 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe414003970 con 0x55cec7f5ed50 2024-04-16T17:12:28.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.701+0000 7fe40e000640 1 -- 172.21.15.45:0/960873606 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe414039040 con 0x55cec7f5ed50 2024-04-16T17:12:28.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.701+0000 7fe4269a8740 1 -- 172.21.15.45:0/960873606 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55cec7f5ed50 msgr2=0x55cec7f5f130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:28.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.701+0000 7fe4269a8740 1 --2- 172.21.15.45:0/960873606 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55cec7f5ed50 0x55cec7f5f130 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fe414009ca0 tx=0x7fe414009900 comp rx=0 tx=0).stop 2024-04-16T17:12:28.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.702+0000 7fe4269a8740 1 -- 172.21.15.45:0/960873606 shutdown_connections 2024-04-16T17:12:28.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.702+0000 7fe4269a8740 1 --2- 172.21.15.45:0/960873606 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cec7f68250 0x55cec7f6a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:28.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.702+0000 7fe4269a8740 1 --2- 172.21.15.45:0/960873606 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cec7f5f670 0x55cec7f67d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:28.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.702+0000 7fe4269a8740 1 --2- 172.21.15.45:0/960873606 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55cec7f5ed50 0x55cec7f5f130 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:28.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.702+0000 7fe4269a8740 1 -- 172.21.15.45:0/960873606 >> 172.21.15.45:0/960873606 conn(0x55cec7f59480 msgr2=0x55cec7f59860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:12:28.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.702+0000 7fe4269a8740 1 -- 172.21.15.45:0/960873606 shutdown_connections 2024-04-16T17:12:28.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.702+0000 7fe4269a8740 1 -- 172.21.15.45:0/960873606 wait complete. 2024-04-16T17:12:28.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.702+0000 7fe4269a8740 1 -- start start 2024-04-16T17:12:28.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.702+0000 7fe4269a8740 1 Processor -- start 2024-04-16T17:12:28.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.703+0000 7fe4269a8740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55cec7f5ed50 0x55cec7f65320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:28.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.703+0000 7fe4269a8740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cec7f5f670 0x55cec7f65860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:28.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.703+0000 7fe4269a8740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cec7f68250 0x55cec7f65da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:28.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.703+0000 7fe4269a8740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55cec7f6afe0 con 0x55cec7f68250 2024-04-16T17:12:28.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.703+0000 7fe4269a8740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55cec7e7d830 con 0x55cec7f5ed50 2024-04-16T17:12:28.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.703+0000 7fe4269a8740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55cec7f6b160 con 0x55cec7f5f670 2024-04-16T17:12:28.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.703+0000 7fe4269a8740 10 client.0 _ll_register_callbacks cb 0x55cec7e85150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:12:28.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.703+0000 7fe421200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cec7f5f670 0x55cec7f65860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:28.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.703+0000 7fe421200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cec7f5f670 0x55cec7f65860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:43810/0 (socket says 172.21.15.45:43810) 2024-04-16T17:12:28.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.703+0000 7fe421200640 1 -- 172.21.15.45:0/3924449544 learned_addr learned my addr 172.21.15.45:0/3924449544 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:12:28.708 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.703+0000 7fe421c00640 1 --2- 172.21.15.45:0/3924449544 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55cec7f5ed50 0x55cec7f65320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:28.708 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.704+0000 7fe421200640 1 -- 172.21.15.45:0/3924449544 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55cec7f5ed50 msgr2=0x55cec7f65320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:12:28.708 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.704+0000 7fe421200640 1 --2- 172.21.15.45:0/3924449544 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55cec7f5ed50 0x55cec7f65320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:28.708 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.704+0000 7fe421200640 1 -- 172.21.15.45:0/3924449544 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cec7f68250 msgr2=0x55cec7f65da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:12:28.708 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.704+0000 7fe421200640 1 --2- 172.21.15.45:0/3924449544 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cec7f68250 0x55cec7f65da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:28.708 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.704+0000 7fe421200640 1 -- 172.21.15.45:0/3924449544 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cec7e81920 con 0x55cec7f5f670 2024-04-16T17:12:28.708 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.704+0000 7fe421200640 1 --2- 172.21.15.45:0/3924449544 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cec7f5f670 0x55cec7f65860 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fe410012790 tx=0x7fe410012c50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=80bb9e76b854c158 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:12:28.708 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.704+0000 7fe402a00640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-04-16T17:12:28.708 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.704+0000 7fe402a00640 1 -- 172.21.15.45:0/3924449544 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe41000f920 con 0x55cec7f5f670 2024-04-16T17:12:28.708 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.704+0000 7fe402a00640 1 -- 172.21.15.45:0/3924449544 <== mon.1 v2:172.21.15.117:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fe41000fac0 con 0x55cec7f5f670 2024-04-16T17:12:28.708 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.705+0000 7fe4269a8740 1 -- 172.21.15.45:0/3924449544 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe414003970 con 0x55cec7f5f670 2024-04-16T17:12:28.708 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.705+0000 7fe402a00640 1 -- 172.21.15.45:0/3924449544 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fe41000fcc0 con 0x55cec7f5f670 2024-04-16T17:12:28.708 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.705+0000 7fe4269a8740 10 client.27133 fetch_fsmap learned FSMap version 37 2024-04-16T17:12:28.708 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.705+0000 7fe4269a8740 1 -- 172.21.15.45:0/3924449544 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55cec7e4f7f0 con 0x55cec7f5f670 2024-04-16T17:12:28.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.706+0000 7fe402a00640 1 -- 172.21.15.45:0/3924449544 <== mon.1 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe414003970 con 0x55cec7f5f670 2024-04-16T17:12:28.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.706+0000 7fe402a00640 1 -- 172.21.15.45:0/3924449544 <== mon.1 v2:172.21.15.117:3300/0 5 ==== fsmap(e 37) ==== 8358+0+0 (secure 0 0 0) 0x7fe410004c90 con 0x55cec7f5f670 2024-04-16T17:12:28.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.706+0000 7fe419600640 1 -- 172.21.15.45:0/274835780 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fe408003f50 con 0x7fe41c106210 2024-04-16T17:12:28.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.706+0000 7fe4269a8740 10 client.27133 fetch_fsmap finished waiting for FSMap version 37 2024-04-16T17:12:28.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.706+0000 7fe4269a8740 10 client.27133 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-04-16T17:12:28.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.706+0000 7fe4269a8740 20 client.27133 populate_metadata read hostname 'smithi045' 2024-04-16T17:12:28.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.706+0000 7fe4269a8740 1 --2- 172.21.15.45:0/3924449544 >> [v2:172.21.15.45:6840/3755310822,v1:172.21.15.45:6841/3755310822] conn(0x55cec7f80e10 0x55cec7f83240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:28.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.706+0000 7fe4269a8740 4 client.27133 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:12:28.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.706+0000 7fe4269a8740 1 -- 172.21.15.45:0/3924449544 --> [v2:172.21.15.45:6840/3755310822,v1:172.21.15.45:6841/3755310822] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cec7ed1a40 con 0x55cec7f80e10 2024-04-16T17:12:28.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.707+0000 7fe421c00640 1 --2- 172.21.15.45:0/3924449544 >> [v2:172.21.15.45:6840/3755310822,v1:172.21.15.45:6841/3755310822] conn(0x55cec7f80e10 0x55cec7f83240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:28.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.708+0000 7fe419600640 1 --2- 172.21.15.45:0/274835780 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fe4040a3cc0 0x7fe4040a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:28.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.709+0000 7fe41be00640 1 --2- 172.21.15.45:0/274835780 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fe4040a3cc0 0x7fe4040a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:28.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.710+0000 7fe41be00640 1 --2- 172.21.15.45:0/274835780 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fe4040a3cc0 0x7fe4040a6180 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fe4080a6eb0 tx=0x7fe4080038f0 comp rx=0 tx=0).ready entity=mgr.25411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:12:28.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.709+0000 7fe419600640 1 -- 172.21.15.45:0/274835780 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fe4080d2b80 con 0x7fe41c106210 2024-04-16T17:12:28.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.712+0000 7fe421c00640 1 --2- 172.21.15.45:0/3924449544 >> [v2:172.21.15.45:6840/3755310822,v1:172.21.15.45:6841/3755310822] conn(0x55cec7f80e10 0x55cec7f83240 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1d1a344cd256e4c3 server_cookie=2d07805fdae0846 in_seq=0 out_seq=0 2024-04-16T17:12:28.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.713+0000 7fe402a00640 10 client.27133 ms_handle_connect on v2:172.21.15.45:6840/3755310822 2024-04-16T17:12:28.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.713+0000 7fe402a00640 1 -- 172.21.15.45:0/3924449544 <== mds.1 v2:172.21.15.45:6840/3755310822 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55cec7ed1a40 con 0x55cec7f80e10 2024-04-16T17:12:28.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.713+0000 7fe402a00640 10 client.27133 handle_command_reply: tid=0 2024-04-16T17:12:28.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.714+0000 7fe4269a8740 1 client.27133 shutdown 2024-04-16T17:12:28.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.714+0000 7fe4269a8740 1 -- 172.21.15.45:0/3924449544 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cec7f5f670 msgr2=0x55cec7f65860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:28.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.714+0000 7fe4269a8740 1 --2- 172.21.15.45:0/3924449544 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cec7f5f670 0x55cec7f65860 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fe410012790 tx=0x7fe410012c50 comp rx=0 tx=0).stop 2024-04-16T17:12:28.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.714+0000 7fe4269a8740 1 -- 172.21.15.45:0/3924449544 shutdown_connections 2024-04-16T17:12:28.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.714+0000 7fe4269a8740 1 --2- 172.21.15.45:0/3924449544 >> [v2:172.21.15.45:6840/3755310822,v1:172.21.15.45:6841/3755310822] conn(0x55cec7f80e10 0x55cec7f83240 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:28.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.714+0000 7fe4269a8740 1 --2- 172.21.15.45:0/3924449544 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cec7f5f670 0x55cec7f65860 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:28.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.714+0000 7fe402a00640 0 client.27133 ms_handle_reset on v2:172.21.15.45:6840/3755310822 2024-04-16T17:12:28.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.714+0000 7fe4269a8740 1 --2- 172.21.15.45:0/3924449544 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cec7f68250 0x55cec7f65da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:28.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.714+0000 7fe4269a8740 1 --2- 172.21.15.45:0/3924449544 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55cec7f5ed50 0x55cec7f65320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:28.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.714+0000 7fe4269a8740 1 -- 172.21.15.45:0/3924449544 >> 172.21.15.45:0/3924449544 conn(0x55cec7f59480 msgr2=0x55cec7f5b1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:12:28.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.714+0000 7fe4269a8740 1 -- 172.21.15.45:0/3924449544 shutdown_connections 2024-04-16T17:12:28.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.714+0000 7fe4269a8740 1 -- 172.21.15.45:0/3924449544 wait complete. 2024-04-16T17:12:28.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.715+0000 7fe4269a8740 20 client.27133 trim_cache size 0 max 16384 2024-04-16T17:12:28.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.729+0000 7fe4269a8740 1 Processor -- start 2024-04-16T17:12:28.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.729+0000 7fe4269a8740 1 -- start start 2024-04-16T17:12:28.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.730+0000 7fe4269a8740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cec7f5ed50 0x55cec7f67a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:28.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.730+0000 7fe4269a8740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55cec7f5f670 0x55cec7f7c730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:28.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.730+0000 7fe4269a8740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cec7f68250 0x55cec7f5afc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:28.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.730+0000 7fe4269a8740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fe41c1afd70 con 0x55cec7f5ed50 2024-04-16T17:12:28.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.730+0000 7fe4269a8740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fe41c1ad5b0 con 0x55cec7f5f670 2024-04-16T17:12:28.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.730+0000 7fe4269a8740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fe41c1ab400 con 0x55cec7f68250 2024-04-16T17:12:28.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.730+0000 7fe421200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55cec7f5f670 0x55cec7f7c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:28.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.730+0000 7fe421200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55cec7f5f670 0x55cec7f7c730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:37854/0 (socket says 172.21.15.45:37854) 2024-04-16T17:12:28.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.730+0000 7fe421200640 1 -- 172.21.15.45:0/2335235249 learned_addr learned my addr 172.21.15.45:0/2335235249 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:12:28.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.730+0000 7fe421c00640 1 --2- 172.21.15.45:0/2335235249 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cec7f68250 0x55cec7f5afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:28.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.731+0000 7fe421200640 1 -- 172.21.15.45:0/2335235249 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cec7f68250 msgr2=0x55cec7f5afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:12:28.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.731+0000 7fe421200640 1 --2- 172.21.15.45:0/2335235249 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cec7f68250 0x55cec7f5afc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:28.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.731+0000 7fe421200640 1 -- 172.21.15.45:0/2335235249 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cec7f5ed50 msgr2=0x55cec7f67a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:12:28.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.731+0000 7fe421200640 1 --2- 172.21.15.45:0/2335235249 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cec7f5ed50 0x55cec7f67a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:28.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.731+0000 7fe421200640 1 -- 172.21.15.45:0/2335235249 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cec7f5f4a0 con 0x55cec7f5f670 2024-04-16T17:12:28.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.731+0000 7fe421c00640 1 --2- 172.21.15.45:0/2335235249 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cec7f68250 0x55cec7f5afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:12:28.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.731+0000 7fe421200640 1 --2- 172.21.15.45:0/2335235249 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55cec7f5f670 0x55cec7f7c730 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fe410012760 tx=0x7fe4100097d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e93fef35a3e7475b server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:12:28.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.731+0000 7fe40d600640 1 -- 172.21.15.45:0/2335235249 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe410000e60 con 0x55cec7f5f670 2024-04-16T17:12:28.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.731+0000 7fe40d600640 1 -- 172.21.15.45:0/2335235249 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe41002b030 con 0x55cec7f5f670 2024-04-16T17:12:28.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.731+0000 7fe40d600640 1 -- 172.21.15.45:0/2335235249 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe41001c470 con 0x55cec7f5f670 2024-04-16T17:12:28.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.732+0000 7fe4269a8740 1 -- 172.21.15.45:0/2335235249 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55cec7f5f670 msgr2=0x55cec7f7c730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:28.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.732+0000 7fe4269a8740 1 --2- 172.21.15.45:0/2335235249 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55cec7f5f670 0x55cec7f7c730 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fe410012760 tx=0x7fe4100097d0 comp rx=0 tx=0).stop 2024-04-16T17:12:28.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.732+0000 7fe4269a8740 1 -- 172.21.15.45:0/2335235249 shutdown_connections 2024-04-16T17:12:28.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.732+0000 7fe4269a8740 1 --2- 172.21.15.45:0/2335235249 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cec7f68250 0x55cec7f5afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:28.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.732+0000 7fe4269a8740 1 --2- 172.21.15.45:0/2335235249 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55cec7f5f670 0x55cec7f7c730 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:28.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.732+0000 7fe4269a8740 1 --2- 172.21.15.45:0/2335235249 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cec7f5ed50 0x55cec7f67a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:28.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.732+0000 7fe4269a8740 1 -- 172.21.15.45:0/2335235249 >> 172.21.15.45:0/2335235249 conn(0x55cec7f59480 msgr2=0x55cec7f5a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:12:28.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.732+0000 7fe4269a8740 1 -- 172.21.15.45:0/2335235249 shutdown_connections 2024-04-16T17:12:28.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.732+0000 7fe4269a8740 1 -- 172.21.15.45:0/2335235249 wait complete. 2024-04-16T17:12:28.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.733+0000 7fe4269a8740 1 -- start start 2024-04-16T17:12:28.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.733+0000 7fe4269a8740 1 Processor -- start 2024-04-16T17:12:28.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.733+0000 7fe4269a8740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55cec7f5ed50 0x55cec7f78eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:28.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.733+0000 7fe4269a8740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cec7f5f670 0x55cec7f74220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:28.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.733+0000 7fe4269a8740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cec7f68250 0x55cec7f74760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:28.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.733+0000 7fe4269a8740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fe41c1aba70 con 0x55cec7f5f670 2024-04-16T17:12:28.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.733+0000 7fe4269a8740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fe41c1acee0 con 0x55cec7f5ed50 2024-04-16T17:12:28.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.733+0000 7fe4269a8740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fe41c1ac140 con 0x55cec7f68250 2024-04-16T17:12:28.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.733+0000 7fe4269a8740 10 client.0 _ll_register_callbacks cb 0x7fe410009880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:12:28.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.734+0000 7fe421200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cec7f5f670 0x55cec7f74220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:28.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.734+0000 7fe421200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cec7f5f670 0x55cec7f74220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35464/0 (socket says 172.21.15.45:35464) 2024-04-16T17:12:28.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.734+0000 7fe421200640 1 -- 172.21.15.45:0/2406568276 learned_addr learned my addr 172.21.15.45:0/2406568276 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:12:28.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.734+0000 7fe421c00640 1 --2- 172.21.15.45:0/2406568276 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55cec7f5ed50 0x55cec7f78eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:28.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.734+0000 7fe421c00640 1 --2- 172.21.15.45:0/2406568276 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cec7f68250 0x55cec7f74760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:28.738 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.734+0000 7fe421c00640 1 -- 172.21.15.45:0/2406568276 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cec7f68250 msgr2=0x55cec7f74760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:12:28.738 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.734+0000 7fe421c00640 1 --2- 172.21.15.45:0/2406568276 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cec7f68250 0x55cec7f74760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:28.738 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.734+0000 7fe421c00640 1 -- 172.21.15.45:0/2406568276 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cec7f5f670 msgr2=0x55cec7f74220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:12:28.738 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.734+0000 7fe421c00640 1 --2- 172.21.15.45:0/2406568276 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cec7f5f670 0x55cec7f74220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:28.738 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.734+0000 7fe421c00640 1 -- 172.21.15.45:0/2406568276 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cec7bd63f0 con 0x55cec7f5ed50 2024-04-16T17:12:28.738 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.735+0000 7fe421c00640 1 --2- 172.21.15.45:0/2406568276 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55cec7f5ed50 0x55cec7f78eb0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fe41402c710 tx=0x7fe41404a040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c73325ef4135a0e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:12:28.738 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.735+0000 7fe402a00640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-04-16T17:12:28.738 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.735+0000 7fe402a00640 1 -- 172.21.15.45:0/2406568276 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe414043040 con 0x55cec7f5ed50 2024-04-16T17:12:28.738 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.735+0000 7fe402a00640 1 -- 172.21.15.45:0/2406568276 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fe41404fd50 con 0x55cec7f5ed50 2024-04-16T17:12:28.738 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.735+0000 7fe4269a8740 1 -- 172.21.15.45:0/2406568276 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe41002b030 con 0x55cec7f5ed50 2024-04-16T17:12:28.738 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.735+0000 7fe402a00640 1 -- 172.21.15.45:0/2406568276 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fe414039040 con 0x55cec7f5ed50 2024-04-16T17:12:28.738 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.735+0000 7fe4269a8740 10 client.27194 fetch_fsmap learned FSMap version 37 2024-04-16T17:12:28.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.735+0000 7fe4269a8740 1 -- 172.21.15.45:0/2406568276 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55cec7ca07d0 con 0x55cec7f5ed50 2024-04-16T17:12:28.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.736+0000 7fe402a00640 1 -- 172.21.15.45:0/2406568276 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe41002b030 con 0x55cec7f5ed50 2024-04-16T17:12:28.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.736+0000 7fe402a00640 1 -- 172.21.15.45:0/2406568276 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 37) ==== 8358+0+0 (secure 0 0 0) 0x7fe41404f2b0 con 0x55cec7f5ed50 2024-04-16T17:12:28.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.737+0000 7fe4269a8740 10 client.27194 fetch_fsmap finished waiting for FSMap version 37 2024-04-16T17:12:28.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.737+0000 7fe4269a8740 10 client.27194 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-04-16T17:12:28.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.737+0000 7fe4269a8740 20 client.27194 populate_metadata read hostname 'smithi045' 2024-04-16T17:12:28.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.737+0000 7fe4269a8740 1 --2- 172.21.15.45:0/2406568276 >> [v2:172.21.15.45:6840/3755310822,v1:172.21.15.45:6841/3755310822] conn(0x55cec7fe7270 0x55cec7fe9660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:28.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.737+0000 7fe4269a8740 4 client.27194 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T17:12:28.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.737+0000 7fe4269a8740 1 -- 172.21.15.45:0/2406568276 --> [v2:172.21.15.45:6840/3755310822,v1:172.21.15.45:6841/3755310822] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55cec7ecb8b0 con 0x55cec7fe7270 2024-04-16T17:12:28.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.737+0000 7fe421200640 1 --2- 172.21.15.45:0/2406568276 >> [v2:172.21.15.45:6840/3755310822,v1:172.21.15.45:6841/3755310822] conn(0x55cec7fe7270 0x55cec7fe9660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:28.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.738+0000 7fe421200640 1 --2- 172.21.15.45:0/2406568276 >> [v2:172.21.15.45:6840/3755310822,v1:172.21.15.45:6841/3755310822] conn(0x55cec7fe7270 0x55cec7fe9660 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6e1f9e16830aef35 server_cookie=b21fff5a5cede390 in_seq=0 out_seq=0 2024-04-16T17:12:28.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.738+0000 7fe402a00640 10 client.27194 ms_handle_connect on v2:172.21.15.45:6840/3755310822 2024-04-16T17:12:28.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.738+0000 7fe402a00640 1 -- 172.21.15.45:0/2406568276 <== mds.1 v2:172.21.15.45:6840/3755310822 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55cec7ecb8b0 con 0x55cec7fe7270 2024-04-16T17:12:28.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.738+0000 7fe402a00640 10 client.27194 handle_command_reply: tid=0 2024-04-16T17:12:28.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.738+0000 7fe4269a8740 1 client.27194 shutdown 2024-04-16T17:12:28.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.738+0000 7fe4269a8740 1 -- 172.21.15.45:0/2406568276 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55cec7f5ed50 msgr2=0x55cec7f78eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:28.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.738+0000 7fe4269a8740 1 --2- 172.21.15.45:0/2406568276 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55cec7f5ed50 0x55cec7f78eb0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fe41402c710 tx=0x7fe41404a040 comp rx=0 tx=0).stop 2024-04-16T17:12:28.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.739+0000 7fe4269a8740 1 -- 172.21.15.45:0/2406568276 shutdown_connections 2024-04-16T17:12:28.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.739+0000 7fe4269a8740 1 --2- 172.21.15.45:0/2406568276 >> [v2:172.21.15.45:6840/3755310822,v1:172.21.15.45:6841/3755310822] conn(0x55cec7fe7270 0x55cec7fe9660 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:28.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.739+0000 7fe4269a8740 1 --2- 172.21.15.45:0/2406568276 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cec7f68250 0x55cec7f74760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:28.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.739+0000 7fe4269a8740 1 --2- 172.21.15.45:0/2406568276 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cec7f5f670 0x55cec7f74220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:28.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.739+0000 7fe4269a8740 1 --2- 172.21.15.45:0/2406568276 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55cec7f5ed50 0x55cec7f78eb0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:28.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.739+0000 7fe4269a8740 1 -- 172.21.15.45:0/2406568276 >> 172.21.15.45:0/2406568276 conn(0x55cec7f59480 msgr2=0x55cec7f5b140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:12:28.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.739+0000 7fe402a00640 0 client.27194 ms_handle_reset on v2:172.21.15.45:6840/3755310822 2024-04-16T17:12:28.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.739+0000 7fe4269a8740 1 -- 172.21.15.45:0/2406568276 shutdown_connections 2024-04-16T17:12:28.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.739+0000 7fe4269a8740 1 -- 172.21.15.45:0/2406568276 wait complete. 2024-04-16T17:12:28.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.739+0000 7fe4269a8740 20 client.27194 trim_cache size 0 max 16384 2024-04-16T17:12:28.742 INFO:teuthology.orchestra.run.smithi045.stdout:[] 2024-04-16T17:12:28.743 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.740+0000 7fe423c00640 1 -- 172.21.15.45:0/274835780 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fe4040a3cc0 msgr2=0x7fe4040a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:28.743 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.740+0000 7fe423c00640 1 --2- 172.21.15.45:0/274835780 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fe4040a3cc0 0x7fe4040a6180 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fe4080a6eb0 tx=0x7fe4080038f0 comp rx=0 tx=0).stop 2024-04-16T17:12:28.743 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.740+0000 7fe423c00640 1 -- 172.21.15.45:0/274835780 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe41c106210 msgr2=0x7fe41c1a0c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:28.743 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.740+0000 7fe423c00640 1 --2- 172.21.15.45:0/274835780 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe41c106210 0x7fe41c1a0c70 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fe408005fd0 tx=0x7fe40800dde0 comp rx=0 tx=0).stop 2024-04-16T17:12:28.743 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.740+0000 7fe423c00640 1 -- 172.21.15.45:0/274835780 shutdown_connections 2024-04-16T17:12:28.744 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.740+0000 7fe423c00640 1 --2- 172.21.15.45:0/274835780 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fe4040a3cc0 0x7fe4040a6180 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:28.744 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.740+0000 7fe423c00640 1 --2- 172.21.15.45:0/274835780 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe41c106210 0x7fe41c1a0c70 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:28.744 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.740+0000 7fe423c00640 1 --2- 172.21.15.45:0/274835780 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe41c19abb0 0x7fe41c19b060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:28.744 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.740+0000 7fe423c00640 1 --2- 172.21.15.45:0/274835780 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe41c101140 0x7fe41c1a0730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:28.744 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.740+0000 7fe423c00640 1 -- 172.21.15.45:0/274835780 >> 172.21.15.45:0/274835780 conn(0x7fe41c077280 msgr2=0x7fe41c0ff620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:12:28.744 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.740+0000 7fe423c00640 1 -- 172.21.15.45:0/274835780 shutdown_connections 2024-04-16T17:12:28.744 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.740+0000 7fe423c00640 1 -- 172.21.15.45:0/274835780 wait complete. 2024-04-16T17:12:28.757 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-16T17:12:28.804 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:12:28 smithi117 ceph-mon[36730]: pgmap v174: 129 pgs: 129 active+clean; 51 MiB data, 687 MiB used, 1.0 TiB / 1.0 TiB avail; 510 KiB/s rd, 791 KiB/s wr, 57 op/s 2024-04-16T17:12:28.804 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:12:28 smithi117 ceph-mon[36730]: from='client.? 172.21.15.45:0/2023581863' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:12:28.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:12:28 smithi104 ceph-mon[36457]: pgmap v174: 129 pgs: 129 active+clean; 51 MiB data, 687 MiB used, 1.0 TiB / 1.0 TiB avail; 510 KiB/s rd, 791 KiB/s wr, 57 op/s 2024-04-16T17:12:28.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:12:28 smithi104 ceph-mon[36457]: from='client.? 172.21.15.45:0/2023581863' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:12:28.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.961+0000 7f97b9e00640 1 -- 172.21.15.45:0/179790437 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f97b4102110 msgr2=0x7f97b41024f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:28.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.961+0000 7f97b9e00640 1 --2- 172.21.15.45:0/179790437 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f97b4102110 0x7f97b41024f0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f97a8009900 tx=0x7f97a802bc40 comp rx=0 tx=0).stop 2024-04-16T17:12:28.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.961+0000 7f97b9e00640 1 -- 172.21.15.45:0/179790437 shutdown_connections 2024-04-16T17:12:28.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.961+0000 7f97b9e00640 1 --2- 172.21.15.45:0/179790437 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f97b41071e0 0x7f97b410edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:28.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.961+0000 7f97b9e00640 1 --2- 172.21.15.45:0/179790437 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f97b4102ac0 0x7f97b4106b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:28.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.961+0000 7f97b9e00640 1 --2- 172.21.15.45:0/179790437 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f97b4102110 0x7f97b41024f0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:28.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.961+0000 7f97b9e00640 1 -- 172.21.15.45:0/179790437 >> 172.21.15.45:0/179790437 conn(0x7f97b40fd9d0 msgr2=0x7f97b40ffdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:12:28.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.962+0000 7f97b9e00640 1 -- 172.21.15.45:0/179790437 shutdown_connections 2024-04-16T17:12:28.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.962+0000 7f97b9e00640 1 -- 172.21.15.45:0/179790437 wait complete. 2024-04-16T17:12:28.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.963+0000 7f97b9e00640 1 Processor -- start 2024-04-16T17:12:28.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.963+0000 7f97b9e00640 1 -- start start 2024-04-16T17:12:28.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.963+0000 7f97b9e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f97b4102110 0x7f97b419e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:28.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.963+0000 7f97b9e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f97b41071e0 0x7f97b419e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:28.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.963+0000 7f97b9e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f97b41985a0 0x7f97b4198a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:28.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.963+0000 7f97b9e00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f97b41122c0 con 0x7f97b41071e0 2024-04-16T17:12:28.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.963+0000 7f97b9e00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f97b4112140 con 0x7f97b41985a0 2024-04-16T17:12:28.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.963+0000 7f97b9e00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f97b4112440 con 0x7f97b4102110 2024-04-16T17:12:28.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.964+0000 7f97b8a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f97b4102110 0x7f97b419e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:28.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.964+0000 7f97b9400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f97b41985a0 0x7f97b4198a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:28.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.964+0000 7f97b8a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f97b4102110 0x7f97b419e3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:43846/0 (socket says 172.21.15.45:43846) 2024-04-16T17:12:28.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.964+0000 7f97b8a00640 1 -- 172.21.15.45:0/545679107 learned_addr learned my addr 172.21.15.45:0/545679107 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:12:28.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.964+0000 7f97b3e00640 1 --2- 172.21.15.45:0/545679107 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f97b41071e0 0x7f97b419e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:28.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.964+0000 7f97b9400640 1 -- 172.21.15.45:0/545679107 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f97b4102110 msgr2=0x7f97b419e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:28.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.964+0000 7f97b9400640 1 --2- 172.21.15.45:0/545679107 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f97b4102110 0x7f97b419e3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:28.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.964+0000 7f97b9400640 1 -- 172.21.15.45:0/545679107 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f97b41071e0 msgr2=0x7f97b419e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:28.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.964+0000 7f97b9400640 1 --2- 172.21.15.45:0/545679107 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f97b41071e0 0x7f97b419e930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:28.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.964+0000 7f97b9400640 1 -- 172.21.15.45:0/545679107 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97b41990c0 con 0x7f97b41985a0 2024-04-16T17:12:28.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.964+0000 7f97b8a00640 1 --2- 172.21.15.45:0/545679107 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f97b4102110 0x7f97b419e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:12:28.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.965+0000 7f97b9400640 1 --2- 172.21.15.45:0/545679107 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f97b41985a0 0x7f97b4198a50 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f97ac009fd0 tx=0x7f97ac00ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:12:28.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.965+0000 7f97b1600640 1 -- 172.21.15.45:0/545679107 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97ac00ce20 con 0x7f97b41985a0 2024-04-16T17:12:28.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.965+0000 7f97b1600640 1 -- 172.21.15.45:0/545679107 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f97ac005020 con 0x7f97b41985a0 2024-04-16T17:12:28.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.965+0000 7f97b1600640 1 -- 172.21.15.45:0/545679107 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97ac010720 con 0x7f97b41985a0 2024-04-16T17:12:28.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.965+0000 7f97b9e00640 1 -- 172.21.15.45:0/545679107 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97b41993b0 con 0x7f97b41985a0 2024-04-16T17:12:28.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.965+0000 7f97b9e00640 1 -- 172.21.15.45:0/545679107 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97b41a55b0 con 0x7f97b41985a0 2024-04-16T17:12:28.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.968+0000 7f97be030740 1 Processor -- start 2024-04-16T17:12:28.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.968+0000 7f97b1600640 1 -- 172.21.15.45:0/545679107 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f97ac005540 con 0x7f97b41985a0 2024-04-16T17:12:28.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.969+0000 7f97be030740 1 -- start start 2024-04-16T17:12:28.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.969+0000 7f97be030740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559a52db7320 0x559a52db7700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:28.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.969+0000 7f97be030740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559a52db7c40 0x559a52dc02e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:28.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.969+0000 7f97be030740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559a52dc0820 0x559a52dc2c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:28.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.969+0000 7f97be030740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x559a52d18020 con 0x559a52dc0820 2024-04-16T17:12:28.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.969+0000 7f97be030740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x559a52cbe460 con 0x559a52db7c40 2024-04-16T17:12:28.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.969+0000 7f97be030740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x559a52d13fe0 con 0x559a52db7320 2024-04-16T17:12:28.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.970+0000 7f97b8a00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559a52db7c40 0x559a52dc02e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:28.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.970+0000 7f97b3e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559a52dc0820 0x559a52dc2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:28.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.970+0000 7f97b8a00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559a52db7c40 0x559a52dc02e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:37882/0 (socket says 172.21.15.45:37882) 2024-04-16T17:12:28.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.970+0000 7f97b8a00640 1 -- 172.21.15.45:0/1511324830 learned_addr learned my addr 172.21.15.45:0/1511324830 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:12:28.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.970+0000 7f97b1600640 1 --2- 172.21.15.45:0/545679107 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f979c0a3da0 0x7f979c0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:28.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.971+0000 7f97b1600640 1 -- 172.21.15.45:0/545679107 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f97ac0d1140 con 0x7f97b41985a0 2024-04-16T17:12:28.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.971+0000 7f97b8a00640 1 -- 172.21.15.45:0/1511324830 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559a52db7320 msgr2=0x559a52db7700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:12:28.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.971+0000 7f97b9400640 1 --2- 172.21.15.45:0/545679107 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f979c0a3da0 0x7f979c0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:28.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.971+0000 7f97b3e00640 1 --2- 172.21.15.45:0/1511324830 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559a52db7320 0x559a52db7700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:28.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.971+0000 7f97b8a00640 1 --2- 172.21.15.45:0/1511324830 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559a52db7320 0x559a52db7700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:28.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.971+0000 7f97b8a00640 1 -- 172.21.15.45:0/1511324830 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559a52dc0820 msgr2=0x559a52dc2c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:12:28.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.971+0000 7f97b8a00640 1 --2- 172.21.15.45:0/1511324830 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559a52dc0820 0x559a52dc2c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:28.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.971+0000 7f97b8a00640 1 -- 172.21.15.45:0/1511324830 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559a52cc6040 con 0x559a52db7c40 2024-04-16T17:12:28.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.971+0000 7f97b3e00640 1 --2- 172.21.15.45:0/1511324830 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559a52db7320 0x559a52db7700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:12:28.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.972+0000 7f97b9400640 1 --2- 172.21.15.45:0/545679107 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f979c0a3da0 0x7f979c0a6260 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f97ac005ba0 tx=0x7f97ac005f70 comp rx=0 tx=0).ready entity=mgr.25411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:12:28.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.972+0000 7f97b8a00640 1 --2- 172.21.15.45:0/1511324830 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559a52db7c40 0x559a52dc02e0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f97a8036040 tx=0x7f97a8005a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7f8012bde49e6ff3 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:12:28.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.972+0000 7f97a5c00640 1 -- 172.21.15.45:0/1511324830 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97a8042070 con 0x559a52db7c40 2024-04-16T17:12:28.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.972+0000 7f97a5c00640 1 -- 172.21.15.45:0/1511324830 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f97a8004a70 con 0x559a52db7c40 2024-04-16T17:12:28.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.972+0000 7f97a5c00640 1 -- 172.21.15.45:0/1511324830 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97a8005260 con 0x559a52db7c40 2024-04-16T17:12:28.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.973+0000 7f97be030740 1 -- 172.21.15.45:0/1511324830 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559a52db7c40 msgr2=0x559a52dc02e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:28.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.973+0000 7f97be030740 1 --2- 172.21.15.45:0/1511324830 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559a52db7c40 0x559a52dc02e0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f97a8036040 tx=0x7f97a8005a20 comp rx=0 tx=0).stop 2024-04-16T17:12:28.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.973+0000 7f97be030740 1 -- 172.21.15.45:0/1511324830 shutdown_connections 2024-04-16T17:12:28.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.973+0000 7f97be030740 1 --2- 172.21.15.45:0/1511324830 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559a52dc0820 0x559a52dc2c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:28.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.973+0000 7f97be030740 1 --2- 172.21.15.45:0/1511324830 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559a52db7c40 0x559a52dc02e0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:28.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.973+0000 7f97be030740 1 --2- 172.21.15.45:0/1511324830 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559a52db7320 0x559a52db7700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:28.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.973+0000 7f97be030740 1 -- 172.21.15.45:0/1511324830 >> 172.21.15.45:0/1511324830 conn(0x559a52db1a50 msgr2=0x559a52db1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:12:28.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.973+0000 7f97be030740 1 -- 172.21.15.45:0/1511324830 shutdown_connections 2024-04-16T17:12:28.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.973+0000 7f97be030740 1 -- 172.21.15.45:0/1511324830 wait complete. 2024-04-16T17:12:28.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.974+0000 7f97be030740 1 -- start start 2024-04-16T17:12:28.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.974+0000 7f97be030740 1 Processor -- start 2024-04-16T17:12:28.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.974+0000 7f97be030740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559a52db7320 0x559a52dbd9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:28.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.974+0000 7f97be030740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559a52db7c40 0x559a52dbdf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:28.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.974+0000 7f97be030740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559a52dc0820 0x559a52dbe450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:28.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.974+0000 7f97be030740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x559a52dc35b0 con 0x559a52db7c40 2024-04-16T17:12:28.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.974+0000 7f97be030740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x559a52cd1830 con 0x559a52db7320 2024-04-16T17:12:28.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.974+0000 7f97be030740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x559a52dc3730 con 0x559a52dc0820 2024-04-16T17:12:28.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.975+0000 7f97be030740 10 client.0 _ll_register_callbacks cb 0x559a52cd9150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:12:28.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.975+0000 7f97b3e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559a52db7320 0x559a52dbd9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:28.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.975+0000 7f97b3e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559a52db7320 0x559a52dbd9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:37894/0 (socket says 172.21.15.45:37894) 2024-04-16T17:12:28.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.975+0000 7f97b3e00640 1 -- 172.21.15.45:0/1721949626 learned_addr learned my addr 172.21.15.45:0/1721949626 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:12:28.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.975+0000 7f97b8a00640 1 --2- 172.21.15.45:0/1721949626 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559a52db7c40 0x559a52dbdf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:28.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.975+0000 7f97b3e00640 1 --2- 172.21.15.45:0/1721949626 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559a52dc0820 0x559a52dbe450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:28.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.975+0000 7f97b3e00640 1 -- 172.21.15.45:0/1721949626 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559a52dc0820 msgr2=0x559a52dbe450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:12:28.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.975+0000 7f97b3e00640 1 --2- 172.21.15.45:0/1721949626 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559a52dc0820 0x559a52dbe450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:28.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.975+0000 7f97b3e00640 1 -- 172.21.15.45:0/1721949626 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559a52db7c40 msgr2=0x559a52dbdf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:12:28.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.975+0000 7f97b3e00640 1 --2- 172.21.15.45:0/1721949626 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559a52db7c40 0x559a52dbdf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:28.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.975+0000 7f97b3e00640 1 -- 172.21.15.45:0/1721949626 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559a52cd5920 con 0x559a52db7320 2024-04-16T17:12:28.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.975+0000 7f97b8a00640 1 --2- 172.21.15.45:0/1721949626 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559a52db7c40 0x559a52dbdf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:12:28.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.976+0000 7f97b3e00640 1 --2- 172.21.15.45:0/1721949626 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559a52db7320 0x559a52dbd9d0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f97a0009fd0 tx=0x7f97a0005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4d0580569444b21b server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:12:28.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.976+0000 7f979a000640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-04-16T17:12:28.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.976+0000 7f979a000640 1 -- 172.21.15.45:0/1721949626 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97a0013120 con 0x559a52db7320 2024-04-16T17:12:28.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.976+0000 7f97be030740 1 -- 172.21.15.45:0/1721949626 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f97a8004a70 con 0x559a52db7320 2024-04-16T17:12:28.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.976+0000 7f979a000640 1 -- 172.21.15.45:0/1721949626 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f97a0025c70 con 0x559a52db7320 2024-04-16T17:12:28.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.976+0000 7f979a000640 1 -- 172.21.15.45:0/1721949626 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f97a00132c0 con 0x559a52db7320 2024-04-16T17:12:28.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.976+0000 7f97be030740 10 client.27206 fetch_fsmap learned FSMap version 37 2024-04-16T17:12:28.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.976+0000 7f97be030740 1 -- 172.21.15.45:0/1721949626 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x559a52ca37f0 con 0x559a52db7320 2024-04-16T17:12:28.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.977+0000 7f979a000640 1 -- 172.21.15.45:0/1721949626 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f97a8004a70 con 0x559a52db7320 2024-04-16T17:12:28.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.977+0000 7f979a000640 1 -- 172.21.15.45:0/1721949626 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 37) ==== 8358+0+0 (secure 0 0 0) 0x7f97a000ebd0 con 0x559a52db7320 2024-04-16T17:12:28.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.977+0000 7f97be030740 10 client.27206 fetch_fsmap finished waiting for FSMap version 37 2024-04-16T17:12:28.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.977+0000 7f97be030740 10 client.27206 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:12:28.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.977+0000 7f97be030740 20 client.27206 populate_metadata read hostname 'smithi045' 2024-04-16T17:12:28.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.977+0000 7f97be030740 1 --2- 172.21.15.45:0/1721949626 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x559a52dae160 0x559a52db05a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:28.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.977+0000 7f97be030740 4 client.27206 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:12:28.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.977+0000 7f97be030740 1 -- 172.21.15.45:0/1721949626 --> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559a52d25a40 con 0x559a52dae160 2024-04-16T17:12:28.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.977+0000 7f97b8a00640 1 --2- 172.21.15.45:0/1721949626 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x559a52dae160 0x559a52db05a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:28.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.978+0000 7f97b8a00640 1 --2- 172.21.15.45:0/1721949626 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x559a52dae160 0x559a52db05a0 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6746b6bf09f997e server_cookie=2e153deab0d397da in_seq=0 out_seq=0 2024-04-16T17:12:28.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.978+0000 7f979a000640 10 client.27206 ms_handle_connect on v2:172.21.15.104:6838/3660070592 2024-04-16T17:12:28.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.979+0000 7f979a000640 1 -- 172.21.15.45:0/1721949626 <== mds.0 v2:172.21.15.104:6838/3660070592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559a52d25a40 con 0x559a52dae160 2024-04-16T17:12:28.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.979+0000 7f979a000640 10 client.27206 handle_command_reply: tid=0 2024-04-16T17:12:28.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.979+0000 7f97be030740 1 client.27206 shutdown 2024-04-16T17:12:28.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.979+0000 7f97be030740 1 -- 172.21.15.45:0/1721949626 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559a52db7320 msgr2=0x559a52dbd9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:28.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.979+0000 7f97be030740 1 --2- 172.21.15.45:0/1721949626 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559a52db7320 0x559a52dbd9d0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f97a0009fd0 tx=0x7f97a0005f90 comp rx=0 tx=0).stop 2024-04-16T17:12:28.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.979+0000 7f97be030740 1 -- 172.21.15.45:0/1721949626 shutdown_connections 2024-04-16T17:12:28.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.979+0000 7f97be030740 1 --2- 172.21.15.45:0/1721949626 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x559a52dae160 0x559a52db05a0 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:28.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.979+0000 7f97be030740 1 --2- 172.21.15.45:0/1721949626 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559a52dc0820 0x559a52dbe450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:28.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.979+0000 7f97be030740 1 --2- 172.21.15.45:0/1721949626 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559a52db7c40 0x559a52dbdf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:28.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.979+0000 7f97be030740 1 --2- 172.21.15.45:0/1721949626 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559a52db7320 0x559a52dbd9d0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:28.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.979+0000 7f979a000640 0 client.27206 ms_handle_reset on v2:172.21.15.104:6838/3660070592 2024-04-16T17:12:28.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.979+0000 7f97be030740 1 -- 172.21.15.45:0/1721949626 >> 172.21.15.45:0/1721949626 conn(0x559a52db1a50 msgr2=0x559a52db37a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:12:28.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.980+0000 7f97be030740 1 -- 172.21.15.45:0/1721949626 shutdown_connections 2024-04-16T17:12:28.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.980+0000 7f97be030740 1 -- 172.21.15.45:0/1721949626 wait complete. 2024-04-16T17:12:28.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.980+0000 7f97be030740 20 client.27206 trim_cache size 0 max 16384 2024-04-16T17:12:28.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.996+0000 7f97be030740 1 Processor -- start 2024-04-16T17:12:28.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.996+0000 7f97be030740 1 -- start start 2024-04-16T17:12:28.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.996+0000 7f97be030740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559a52db7320 0x559a52db3590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:28.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.996+0000 7f97be030740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559a52db7c40 0x559a52dd4880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:28.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.996+0000 7f97be030740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559a52dc0820 0x559a52dd4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:28.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.996+0000 7f97be030740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f97b41af960 con 0x559a52db7c40 2024-04-16T17:12:28.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.996+0000 7f97be030740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f97b41ad1a0 con 0x559a52db7320 2024-04-16T17:12:28.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.996+0000 7f97be030740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f97b41aafd0 con 0x559a52dc0820 2024-04-16T17:12:28.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.996+0000 7f97b3e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559a52db7320 0x559a52db3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:28.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.996+0000 7f97b3e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559a52db7320 0x559a52db3590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:37910/0 (socket says 172.21.15.45:37910) 2024-04-16T17:12:28.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.996+0000 7f97b3e00640 1 -- 172.21.15.45:0/3534088978 learned_addr learned my addr 172.21.15.45:0/3534088978 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:12:28.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.997+0000 7f97b3e00640 1 --2- 172.21.15.45:0/3534088978 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559a52dc0820 0x559a52dd4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:28.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.997+0000 7f97b8a00640 1 --2- 172.21.15.45:0/3534088978 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559a52db7c40 0x559a52dd4880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:28.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.997+0000 7f97b3e00640 1 -- 172.21.15.45:0/3534088978 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559a52dc0820 msgr2=0x559a52dd4dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:12:28.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.997+0000 7f97b3e00640 1 --2- 172.21.15.45:0/3534088978 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559a52dc0820 0x559a52dd4dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:28.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.997+0000 7f97b3e00640 1 -- 172.21.15.45:0/3534088978 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559a52db7c40 msgr2=0x559a52dd4880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:12:28.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.997+0000 7f97b3e00640 1 --2- 172.21.15.45:0/3534088978 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559a52db7c40 0x559a52dd4880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:28.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.997+0000 7f97b3e00640 1 -- 172.21.15.45:0/3534088978 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559a52a2a3f0 con 0x559a52db7320 2024-04-16T17:12:28.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.997+0000 7f97b8a00640 1 --2- 172.21.15.45:0/3534088978 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559a52db7c40 0x559a52dd4880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:12:28.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.997+0000 7f97b3e00640 1 --2- 172.21.15.45:0/3534088978 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559a52db7320 0x559a52db3590 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f97a0000c00 tx=0x7f97a0013140 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1c87f61e523f1f3c server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:12:28.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.997+0000 7f97a5200640 1 -- 172.21.15.45:0/3534088978 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97a0009c70 con 0x559a52db7320 2024-04-16T17:12:28.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.998+0000 7f97a5200640 1 -- 172.21.15.45:0/3534088978 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f97a000ec60 con 0x559a52db7320 2024-04-16T17:12:28.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.998+0000 7f97a5200640 1 -- 172.21.15.45:0/3534088978 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97a00055d0 con 0x559a52db7320 2024-04-16T17:12:29.000 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.999+0000 7f97be030740 1 -- 172.21.15.45:0/3534088978 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559a52db7320 msgr2=0x559a52db3590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:29.000 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.999+0000 7f97be030740 1 --2- 172.21.15.45:0/3534088978 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559a52db7320 0x559a52db3590 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f97a0000c00 tx=0x7f97a0013140 comp rx=0 tx=0).stop 2024-04-16T17:12:29.000 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.999+0000 7f97be030740 1 -- 172.21.15.45:0/3534088978 shutdown_connections 2024-04-16T17:12:29.000 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.999+0000 7f97be030740 1 --2- 172.21.15.45:0/3534088978 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559a52dc0820 0x559a52dd4dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:29.000 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.999+0000 7f97be030740 1 --2- 172.21.15.45:0/3534088978 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559a52db7c40 0x559a52dd4880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:29.000 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.999+0000 7f97be030740 1 --2- 172.21.15.45:0/3534088978 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559a52db7320 0x559a52db3590 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:29.000 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.999+0000 7f97be030740 1 -- 172.21.15.45:0/3534088978 >> 172.21.15.45:0/3534088978 conn(0x559a52db1a50 msgr2=0x559a52db2e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:12:29.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.999+0000 7f97be030740 1 -- 172.21.15.45:0/3534088978 shutdown_connections 2024-04-16T17:12:29.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:28.999+0000 7f97be030740 1 -- 172.21.15.45:0/3534088978 wait complete. 2024-04-16T17:12:29.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:29.000+0000 7f97be030740 1 -- start start 2024-04-16T17:12:29.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:29.000+0000 7f97be030740 1 Processor -- start 2024-04-16T17:12:29.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:29.001+0000 7f97be030740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559a52db7320 0x559a52dd45a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:29.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:29.001+0000 7f97be030740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559a52db7c40 0x559a52dd4ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:29.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:29.001+0000 7f97be030740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559a52dc0820 0x559a52dd1c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:29.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:29.001+0000 7f97be030740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f97b41ab660 con 0x559a52db7c40 2024-04-16T17:12:29.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:29.001+0000 7f97be030740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f97b41acad0 con 0x559a52db7320 2024-04-16T17:12:29.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:29.001+0000 7f97be030740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f97b41abd30 con 0x559a52dc0820 2024-04-16T17:12:29.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:29.001+0000 7f97be030740 10 client.0 _ll_register_callbacks cb 0x7f97a0009ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:12:29.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:29.001+0000 7f97b3e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559a52db7320 0x559a52dd45a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:29.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:29.001+0000 7f97b3e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559a52db7320 0x559a52dd45a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:37926/0 (socket says 172.21.15.45:37926) 2024-04-16T17:12:29.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:29.001+0000 7f97b3e00640 1 -- 172.21.15.45:0/1284986452 learned_addr learned my addr 172.21.15.45:0/1284986452 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:12:29.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:29.001+0000 7f97b8a00640 1 --2- 172.21.15.45:0/1284986452 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559a52db7c40 0x559a52dd4ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:29.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:29.001+0000 7f97b3e00640 1 --2- 172.21.15.45:0/1284986452 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559a52dc0820 0x559a52dd1c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:29.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:29.002+0000 7f97b3e00640 1 -- 172.21.15.45:0/1284986452 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559a52dc0820 msgr2=0x559a52dd1c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:12:29.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:29.002+0000 7f97b3e00640 1 --2- 172.21.15.45:0/1284986452 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559a52dc0820 0x559a52dd1c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:29.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:29.002+0000 7f97b3e00640 1 -- 172.21.15.45:0/1284986452 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559a52db7c40 msgr2=0x559a52dd4ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:12:29.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:29.002+0000 7f97b3e00640 1 --2- 172.21.15.45:0/1284986452 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559a52db7c40 0x559a52dd4ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:29.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:29.002+0000 7f97b3e00640 1 -- 172.21.15.45:0/1284986452 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559a52af47d0 con 0x559a52db7320 2024-04-16T17:12:29.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:29.002+0000 7f97b3e00640 1 --2- 172.21.15.45:0/1284986452 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559a52db7320 0x559a52dd45a0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f97a0013860 tx=0x7f97a0028040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8b779ffd077510bc server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:12:29.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:29.002+0000 7f979a000640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-04-16T17:12:29.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:29.002+0000 7f979a000640 1 -- 172.21.15.45:0/1284986452 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97a0005350 con 0x559a52db7320 2024-04-16T17:12:29.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:29.002+0000 7f97be030740 1 -- 172.21.15.45:0/1284986452 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f97a000ec60 con 0x559a52db7320 2024-04-16T17:12:29.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:29.002+0000 7f979a000640 1 -- 172.21.15.45:0/1284986452 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f97a00054f0 con 0x559a52db7320 2024-04-16T17:12:29.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:29.003+0000 7f979a000640 1 -- 172.21.15.45:0/1284986452 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f97a0047050 con 0x559a52db7320 2024-04-16T17:12:29.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:29.003+0000 7f97be030740 10 client.27212 fetch_fsmap learned FSMap version 37 2024-04-16T17:12:29.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:29.003+0000 7f97be030740 1 -- 172.21.15.45:0/1284986452 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x559a52e32af0 con 0x559a52db7320 2024-04-16T17:12:29.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:29.003+0000 7f979a000640 1 -- 172.21.15.45:0/1284986452 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f97a000ec60 con 0x559a52db7320 2024-04-16T17:12:29.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:29.003+0000 7f979a000640 1 -- 172.21.15.45:0/1284986452 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 37) ==== 8358+0+0 (secure 0 0 0) 0x7f97a0026200 con 0x559a52db7320 2024-04-16T17:12:29.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:29.003+0000 7f97be030740 10 client.27212 fetch_fsmap finished waiting for FSMap version 37 2024-04-16T17:12:29.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:29.003+0000 7f97be030740 10 client.27212 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:12:29.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:29.003+0000 7f97be030740 20 client.27212 populate_metadata read hostname 'smithi045' 2024-04-16T17:12:29.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:29.003+0000 7f97be030740 1 --2- 172.21.15.45:0/1284986452 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x559a52e33130 0x559a52e35550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:29.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:29.003+0000 7f97be030740 4 client.27212 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T17:12:29.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:29.003+0000 7f97be030740 1 -- 172.21.15.45:0/1284986452 --> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559a52d1f8b0 con 0x559a52e33130 2024-04-16T17:12:29.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:29.004+0000 7f97b8a00640 1 --2- 172.21.15.45:0/1284986452 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x559a52e33130 0x559a52e35550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:29.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:29.004+0000 7f97b8a00640 1 --2- 172.21.15.45:0/1284986452 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x559a52e33130 0x559a52e35550 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=926321bc8aed97b0 server_cookie=cbc645a2dcf5eba1 in_seq=0 out_seq=0 2024-04-16T17:12:29.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:29.004+0000 7f979a000640 10 client.27212 ms_handle_connect on v2:172.21.15.104:6838/3660070592 2024-04-16T17:12:29.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:29.005+0000 7f979a000640 1 -- 172.21.15.45:0/1284986452 <== mds.0 v2:172.21.15.104:6838/3660070592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559a52d1f8b0 con 0x559a52e33130 2024-04-16T17:12:29.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:29.005+0000 7f979a000640 10 client.27212 handle_command_reply: tid=0 2024-04-16T17:12:29.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:29.005+0000 7f97be030740 1 client.27212 shutdown 2024-04-16T17:12:29.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:29.005+0000 7f97be030740 1 -- 172.21.15.45:0/1284986452 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559a52db7320 msgr2=0x559a52dd45a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:29.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:29.005+0000 7f97be030740 1 --2- 172.21.15.45:0/1284986452 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559a52db7320 0x559a52dd45a0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f97a0013860 tx=0x7f97a0028040 comp rx=0 tx=0).stop 2024-04-16T17:12:29.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:29.005+0000 7f97be030740 1 -- 172.21.15.45:0/1284986452 shutdown_connections 2024-04-16T17:12:29.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:29.005+0000 7f97be030740 1 --2- 172.21.15.45:0/1284986452 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x559a52e33130 0x559a52e35550 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:29.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:29.005+0000 7f97be030740 1 --2- 172.21.15.45:0/1284986452 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559a52dc0820 0x559a52dd1c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:29.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:29.005+0000 7f97be030740 1 --2- 172.21.15.45:0/1284986452 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559a52db7c40 0x559a52dd4ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:29.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:29.005+0000 7f97be030740 1 --2- 172.21.15.45:0/1284986452 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559a52db7320 0x559a52dd45a0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:29.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:29.005+0000 7f97be030740 1 -- 172.21.15.45:0/1284986452 >> 172.21.15.45:0/1284986452 conn(0x559a52db1a50 msgr2=0x559a52dac700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:12:29.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:29.005+0000 7f979a000640 0 client.27212 ms_handle_reset on v2:172.21.15.104:6838/3660070592 2024-04-16T17:12:29.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:29.006+0000 7f97be030740 1 -- 172.21.15.45:0/1284986452 shutdown_connections 2024-04-16T17:12:29.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:29.006+0000 7f97be030740 1 -- 172.21.15.45:0/1284986452 wait complete. 2024-04-16T17:12:29.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:29.006+0000 7f97be030740 20 client.27212 trim_cache size 0 max 16384 2024-04-16T17:12:29.008 INFO:teuthology.orchestra.run.smithi045.stdout:[] 2024-04-16T17:12:29.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:29.007+0000 7f97b9e00640 1 -- 172.21.15.45:0/545679107 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f979c0a3da0 msgr2=0x7f979c0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:29.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:29.007+0000 7f97b9e00640 1 --2- 172.21.15.45:0/545679107 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f979c0a3da0 0x7f979c0a6260 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f97ac005ba0 tx=0x7f97ac005f70 comp rx=0 tx=0).stop 2024-04-16T17:12:29.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:29.007+0000 7f97b9e00640 1 -- 172.21.15.45:0/545679107 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f97b41985a0 msgr2=0x7f97b4198a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:29.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:29.007+0000 7f97b9e00640 1 --2- 172.21.15.45:0/545679107 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f97b41985a0 0x7f97b4198a50 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f97ac009fd0 tx=0x7f97ac00ef10 comp rx=0 tx=0).stop 2024-04-16T17:12:29.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:29.007+0000 7f97b9e00640 1 -- 172.21.15.45:0/545679107 shutdown_connections 2024-04-16T17:12:29.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:29.007+0000 7f97b9e00640 1 --2- 172.21.15.45:0/545679107 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f979c0a3da0 0x7f979c0a6260 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:29.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:29.007+0000 7f97b9e00640 1 --2- 172.21.15.45:0/545679107 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f97b41985a0 0x7f97b4198a50 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:29.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:29.007+0000 7f97b9e00640 1 --2- 172.21.15.45:0/545679107 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f97b41071e0 0x7f97b419e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:29.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:29.007+0000 7f97b9e00640 1 --2- 172.21.15.45:0/545679107 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f97b4102110 0x7f97b419e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:29.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:29.007+0000 7f97b9e00640 1 -- 172.21.15.45:0/545679107 >> 172.21.15.45:0/545679107 conn(0x7f97b40fd9d0 msgr2=0x7f97b40ffc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:12:29.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:29.007+0000 7f97b9e00640 1 -- 172.21.15.45:0/545679107 shutdown_connections 2024-04-16T17:12:29.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:29.008+0000 7f97b9e00640 1 -- 172.21.15.45:0/545679107 wait complete. 2024-04-16T17:12:29.268 INFO:tasks.workunit.client.0.smithi045.stdout: 1 87375 1.57 MB/sec execute 151 sec latency 46.536 ms 2024-04-16T17:12:30.028 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-16T17:12:30.029 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T17:12:30.029 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-16T17:12:30.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.254+0000 7faa17200640 1 -- 172.21.15.45:0/1246916318 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7faa18104cd0 msgr2=0x7faa18108d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:30.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.254+0000 7faa17200640 1 --2- 172.21.15.45:0/1246916318 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7faa18104cd0 0x7faa18108d10 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7faa04009900 tx=0x7faa0402bc40 comp rx=0 tx=0).stop 2024-04-16T17:12:30.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.254+0000 7faa17200640 1 -- 172.21.15.45:0/1246916318 shutdown_connections 2024-04-16T17:12:30.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.254+0000 7faa17200640 1 --2- 172.21.15.45:0/1246916318 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faa181093f0 0x7faa18110fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:30.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.254+0000 7faa17200640 1 --2- 172.21.15.45:0/1246916318 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7faa18104cd0 0x7faa18108d10 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:30.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.254+0000 7faa17200640 1 --2- 172.21.15.45:0/1246916318 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faa18104320 0x7faa18104700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:30.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.254+0000 7faa17200640 1 -- 172.21.15.45:0/1246916318 >> 172.21.15.45:0/1246916318 conn(0x7faa180ffb40 msgr2=0x7faa18101f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:12:30.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.254+0000 7faa17200640 1 -- 172.21.15.45:0/1246916318 shutdown_connections 2024-04-16T17:12:30.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.254+0000 7faa17200640 1 -- 172.21.15.45:0/1246916318 wait complete. 2024-04-16T17:12:30.284 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.256+0000 7faa17200640 1 Processor -- start 2024-04-16T17:12:30.284 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.256+0000 7faa17200640 1 -- start start 2024-04-16T17:12:30.284 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.256+0000 7faa17200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faa18104320 0x7faa181a0680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:30.284 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.256+0000 7faa17200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7faa181093f0 0x7faa181a0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:30.284 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.256+0000 7faa17200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faa1819a7f0 0x7faa1819aca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:30.284 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.256+0000 7faa17200640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7faa181144d0 con 0x7faa18104320 2024-04-16T17:12:30.284 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.256+0000 7faa17200640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7faa18114350 con 0x7faa181093f0 2024-04-16T17:12:30.284 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.256+0000 7faa17200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7faa18114650 con 0x7faa1819a7f0 2024-04-16T17:12:30.284 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.257+0000 7faa15400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7faa181093f0 0x7faa181a0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:30.284 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.257+0000 7faa15400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7faa181093f0 0x7faa181a0bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:38738/0 (socket says 172.21.15.45:38738) 2024-04-16T17:12:30.284 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.257+0000 7faa15400640 1 -- 172.21.15.45:0/3080797013 learned_addr learned my addr 172.21.15.45:0/3080797013 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:12:30.284 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.257+0000 7faa16800640 1 --2- 172.21.15.45:0/3080797013 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faa1819a7f0 0x7faa1819aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:30.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.257+0000 7faa15e00640 1 --2- 172.21.15.45:0/3080797013 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faa18104320 0x7faa181a0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:30.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.257+0000 7faa15400640 1 -- 172.21.15.45:0/3080797013 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faa1819a7f0 msgr2=0x7faa1819aca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:30.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.257+0000 7faa15400640 1 --2- 172.21.15.45:0/3080797013 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faa1819a7f0 0x7faa1819aca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:30.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.257+0000 7faa15400640 1 -- 172.21.15.45:0/3080797013 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faa18104320 msgr2=0x7faa181a0680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:30.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.257+0000 7faa15400640 1 --2- 172.21.15.45:0/3080797013 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faa18104320 0x7faa181a0680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:30.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.257+0000 7faa15400640 1 -- 172.21.15.45:0/3080797013 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa1819b380 con 0x7faa181093f0 2024-04-16T17:12:30.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.258+0000 7faa15400640 1 --2- 172.21.15.45:0/3080797013 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7faa181093f0 0x7faa181a0bc0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7faa0402c250 tx=0x7faa04004360 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:12:30.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.258+0000 7faa0ea00640 1 -- 172.21.15.45:0/3080797013 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa04041070 con 0x7faa181093f0 2024-04-16T17:12:30.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.258+0000 7faa0ea00640 1 -- 172.21.15.45:0/3080797013 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faa0402cca0 con 0x7faa181093f0 2024-04-16T17:12:30.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.258+0000 7faa0ea00640 1 -- 172.21.15.45:0/3080797013 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa04003da0 con 0x7faa181093f0 2024-04-16T17:12:30.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.258+0000 7faa17200640 1 -- 172.21.15.45:0/3080797013 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa1819b610 con 0x7faa181093f0 2024-04-16T17:12:30.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.258+0000 7faa17200640 1 -- 172.21.15.45:0/3080797013 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa181a7770 con 0x7faa181093f0 2024-04-16T17:12:30.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.262+0000 7faa1f771740 1 Processor -- start 2024-04-16T17:12:30.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.262+0000 7faa1f771740 1 -- start start 2024-04-16T17:12:30.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.262+0000 7faa1f771740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557356c6ce90 0x557356c6d270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:30.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.262+0000 7faa1f771740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557356c6d7b0 0x557356c75e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:30.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.262+0000 7faa1f771740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557356c76390 0x557356c78780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:30.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.262+0000 7faa1f771740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x557356bd1cb0 con 0x557356c6d7b0 2024-04-16T17:12:30.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.263+0000 7faa1f771740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x557356b78370 con 0x557356c76390 2024-04-16T17:12:30.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.263+0000 7faa1f771740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x557356bcdc70 con 0x557356c6ce90 2024-04-16T17:12:30.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.263+0000 7faa16800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557356c6ce90 0x557356c6d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:30.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.263+0000 7faa16800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557356c6ce90 0x557356c6d270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:49856/0 (socket says 172.21.15.45:49856) 2024-04-16T17:12:30.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.263+0000 7faa16800640 1 -- 172.21.15.45:0/3643332202 learned_addr learned my addr 172.21.15.45:0/3643332202 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:12:30.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.263+0000 7faa0ea00640 1 -- 172.21.15.45:0/3080797013 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7faa04044070 con 0x7faa181093f0 2024-04-16T17:12:30.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.263+0000 7faa16800640 1 -- 172.21.15.45:0/3643332202 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557356c76390 msgr2=0x557356c78780 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-16T17:12:30.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.263+0000 7faa16800640 1 --2- 172.21.15.45:0/3643332202 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557356c76390 0x557356c78780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:30.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.263+0000 7faa16800640 1 -- 172.21.15.45:0/3643332202 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557356c6d7b0 msgr2=0x557356c75e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:12:30.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.263+0000 7faa16800640 1 --2- 172.21.15.45:0/3643332202 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557356c6d7b0 0x557356c75e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:30.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.263+0000 7faa16800640 1 -- 172.21.15.45:0/3643332202 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557356b97910 con 0x557356c6ce90 2024-04-16T17:12:30.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.264+0000 7faa16800640 1 --2- 172.21.15.45:0/3643332202 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557356c6ce90 0x557356c6d270 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7faa10009f70 tx=0x7faa10010800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6fe6911354e5abd server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:12:30.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.264+0000 7faa03000640 1 -- 172.21.15.45:0/3643332202 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa10016070 con 0x557356c6ce90 2024-04-16T17:12:30.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.264+0000 7faa03000640 1 -- 172.21.15.45:0/3643332202 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faa10004d20 con 0x557356c6ce90 2024-04-16T17:12:30.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.264+0000 7faa03000640 1 -- 172.21.15.45:0/3643332202 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa10012950 con 0x557356c6ce90 2024-04-16T17:12:30.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.265+0000 7faa0ea00640 1 --2- 172.21.15.45:0/3080797013 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fa9fc0a3da0 0x7fa9fc0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:30.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.265+0000 7faa0ea00640 1 -- 172.21.15.45:0/3080797013 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7faa040f2700 con 0x7faa181093f0 2024-04-16T17:12:30.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.265+0000 7faa1f771740 1 -- 172.21.15.45:0/3643332202 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557356c6ce90 msgr2=0x557356c6d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:30.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.265+0000 7faa1f771740 1 --2- 172.21.15.45:0/3643332202 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557356c6ce90 0x557356c6d270 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7faa10009f70 tx=0x7faa10010800 comp rx=0 tx=0).stop 2024-04-16T17:12:30.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.265+0000 7faa15e00640 1 --2- 172.21.15.45:0/3080797013 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fa9fc0a3da0 0x7fa9fc0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:30.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.266+0000 7faa1f771740 1 -- 172.21.15.45:0/3643332202 shutdown_connections 2024-04-16T17:12:30.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.266+0000 7faa1f771740 1 --2- 172.21.15.45:0/3643332202 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557356c76390 0x557356c78780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:30.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.266+0000 7faa1f771740 1 --2- 172.21.15.45:0/3643332202 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557356c6d7b0 0x557356c75e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:30.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.266+0000 7faa1f771740 1 --2- 172.21.15.45:0/3643332202 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557356c6ce90 0x557356c6d270 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:30.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.266+0000 7faa1f771740 1 -- 172.21.15.45:0/3643332202 >> 172.21.15.45:0/3643332202 conn(0x557356c675c0 msgr2=0x557356c679a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:12:30.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.266+0000 7faa1f771740 1 -- 172.21.15.45:0/3643332202 shutdown_connections 2024-04-16T17:12:30.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.266+0000 7faa1f771740 1 -- 172.21.15.45:0/3643332202 wait complete. 2024-04-16T17:12:30.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.266+0000 7faa15e00640 1 --2- 172.21.15.45:0/3080797013 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fa9fc0a3da0 0x7fa9fc0a6260 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x557356c6c960 tx=0x7faa08009840 comp rx=0 tx=0).ready entity=mgr.25411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:12:30.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.267+0000 7faa1f771740 1 -- start start 2024-04-16T17:12:30.288 INFO:tasks.workunit.client.0.smithi045.stdout: 1 88705 1.63 MB/sec execute 152 sec latency 17.311 ms 2024-04-16T17:12:30.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.267+0000 7faa1f771740 1 Processor -- start 2024-04-16T17:12:30.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.269+0000 7faa1f771740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557356c6ce90 0x557356c73450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:30.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.269+0000 7faa1f771740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557356c6d7b0 0x557356c73990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:30.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.269+0000 7faa1f771740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557356c76390 0x557356c73ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:30.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.269+0000 7faa1f771740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x557356c79180 con 0x557356c76390 2024-04-16T17:12:30.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.269+0000 7faa1f771740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x557356b8b730 con 0x557356c6d7b0 2024-04-16T17:12:30.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.269+0000 7faa1f771740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x557356c79300 con 0x557356c6ce90 2024-04-16T17:12:30.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.269+0000 7faa1f771740 10 client.0 _ll_register_callbacks cb 0x557356b5a1e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:12:30.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.269+0000 7faa15e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557356c6ce90 0x557356c73450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:30.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.269+0000 7faa15e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557356c6ce90 0x557356c73450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:49868/0 (socket says 172.21.15.45:49868) 2024-04-16T17:12:30.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.269+0000 7faa15e00640 1 -- 172.21.15.45:0/715817712 learned_addr learned my addr 172.21.15.45:0/715817712 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:12:30.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.269+0000 7faa15400640 1 --2- 172.21.15.45:0/715817712 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557356c6d7b0 0x557356c73990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:30.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.269+0000 7faa16800640 1 --2- 172.21.15.45:0/715817712 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557356c76390 0x557356c73ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:30.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.270+0000 7faa15e00640 1 -- 172.21.15.45:0/715817712 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557356c6d7b0 msgr2=0x557356c73990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:12:30.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.270+0000 7faa15e00640 1 --2- 172.21.15.45:0/715817712 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557356c6d7b0 0x557356c73990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:30.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.270+0000 7faa15e00640 1 -- 172.21.15.45:0/715817712 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557356c76390 msgr2=0x557356c73ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:12:30.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.270+0000 7faa15e00640 1 --2- 172.21.15.45:0/715817712 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557356c76390 0x557356c73ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:30.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.270+0000 7faa15e00640 1 -- 172.21.15.45:0/715817712 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5573569ac7e0 con 0x557356c6ce90 2024-04-16T17:12:30.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.270+0000 7faa15e00640 1 --2- 172.21.15.45:0/715817712 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557356c6ce90 0x557356c73450 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7faa08005f60 tx=0x7faa08005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f1cd330f66757b25 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:12:30.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.270+0000 7fa9f7400640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-04-16T17:12:30.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.271+0000 7fa9f7400640 1 -- 172.21.15.45:0/715817712 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa0801d8e0 con 0x557356c6ce90 2024-04-16T17:12:30.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.271+0000 7faa1f771740 1 -- 172.21.15.45:0/715817712 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faa10004d20 con 0x557356c6ce90 2024-04-16T17:12:30.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.271+0000 7fa9f7400640 1 -- 172.21.15.45:0/715817712 <== mon.1 v2:172.21.15.117:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7faa0801da80 con 0x557356c6ce90 2024-04-16T17:12:30.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.271+0000 7fa9f7400640 1 -- 172.21.15.45:0/715817712 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7faa0801dc80 con 0x557356c6ce90 2024-04-16T17:12:30.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.271+0000 7faa1f771740 10 client.27166 fetch_fsmap learned FSMap version 37 2024-04-16T17:12:30.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.271+0000 7faa1f771740 1 -- 172.21.15.45:0/715817712 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x557356a0cf70 con 0x557356c6ce90 2024-04-16T17:12:30.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.271+0000 7fa9f7400640 1 -- 172.21.15.45:0/715817712 <== mon.1 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faa10004d20 con 0x557356c6ce90 2024-04-16T17:12:30.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.272+0000 7fa9f7400640 1 -- 172.21.15.45:0/715817712 <== mon.1 v2:172.21.15.117:3300/0 5 ==== fsmap(e 37) ==== 8358+0+0 (secure 0 0 0) 0x7faa0801a420 con 0x557356c6ce90 2024-04-16T17:12:30.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.272+0000 7faa1f771740 10 client.27166 fetch_fsmap finished waiting for FSMap version 37 2024-04-16T17:12:30.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.272+0000 7faa1f771740 10 client.27166 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:12:30.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.272+0000 7faa1f771740 20 client.27166 populate_metadata read hostname 'smithi045' 2024-04-16T17:12:30.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.272+0000 7faa1f771740 1 --2- 172.21.15.45:0/715817712 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x557356c97890 0x557356c99c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:30.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.272+0000 7faa1f771740 4 client.27166 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:12:30.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.272+0000 7faa1f771740 1 -- 172.21.15.45:0/715817712 --> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557356bdf6d0 con 0x557356c97890 2024-04-16T17:12:30.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.273+0000 7faa15400640 1 --2- 172.21.15.45:0/715817712 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x557356c97890 0x557356c99c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:30.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.273+0000 7faa15400640 1 --2- 172.21.15.45:0/715817712 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x557356c97890 0x557356c99c80 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d20557b4f170613d server_cookie=56ef0847f2569c95 in_seq=0 out_seq=0 2024-04-16T17:12:30.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.273+0000 7fa9f7400640 10 client.27166 ms_handle_connect on v2:172.21.15.104:6838/3660070592 2024-04-16T17:12:30.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.274+0000 7fa9f7400640 1 -- 172.21.15.45:0/715817712 <== mds.0 v2:172.21.15.104:6838/3660070592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557356bdf6d0 con 0x557356c97890 2024-04-16T17:12:30.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.274+0000 7fa9f7400640 10 client.27166 handle_command_reply: tid=0 2024-04-16T17:12:30.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.274+0000 7faa1f771740 1 client.27166 shutdown 2024-04-16T17:12:30.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.275+0000 7faa1f771740 1 -- 172.21.15.45:0/715817712 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557356c6ce90 msgr2=0x557356c73450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:30.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.275+0000 7faa1f771740 1 --2- 172.21.15.45:0/715817712 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557356c6ce90 0x557356c73450 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7faa08005f60 tx=0x7faa08005f90 comp rx=0 tx=0).stop 2024-04-16T17:12:30.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.275+0000 7faa1f771740 1 -- 172.21.15.45:0/715817712 shutdown_connections 2024-04-16T17:12:30.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.275+0000 7faa1f771740 1 --2- 172.21.15.45:0/715817712 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557356c76390 0x557356c73ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:30.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.275+0000 7faa1f771740 1 --2- 172.21.15.45:0/715817712 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557356c6d7b0 0x557356c73990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:30.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.275+0000 7faa1f771740 1 --2- 172.21.15.45:0/715817712 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x557356c97890 0x557356c99c80 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:30.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.275+0000 7faa1f771740 1 --2- 172.21.15.45:0/715817712 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557356c6ce90 0x557356c73450 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:30.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.275+0000 7faa1f771740 1 -- 172.21.15.45:0/715817712 >> 172.21.15.45:0/715817712 conn(0x557356c675c0 msgr2=0x557356c69310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:12:30.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.275+0000 7fa9f7400640 0 client.27166 ms_handle_reset on v2:172.21.15.104:6838/3660070592 2024-04-16T17:12:30.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.275+0000 7faa1f771740 1 -- 172.21.15.45:0/715817712 shutdown_connections 2024-04-16T17:12:30.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.275+0000 7faa1f771740 1 -- 172.21.15.45:0/715817712 wait complete. 2024-04-16T17:12:30.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.275+0000 7faa1f771740 20 client.27166 trim_cache size 0 max 16384 2024-04-16T17:12:30.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.313+0000 7faa1f771740 1 Processor -- start 2024-04-16T17:12:30.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.313+0000 7faa1f771740 1 -- start start 2024-04-16T17:12:30.315 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.314+0000 7faa1f771740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557356c6ce90 0x557356c75b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:30.315 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.314+0000 7faa1f771740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557356c6d7b0 0x557356c820c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:30.315 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.314+0000 7faa1f771740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557356c76390 0x557356c9f550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:30.315 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.314+0000 7faa1f771740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7faa181ab580 con 0x557356c6d7b0 2024-04-16T17:12:30.315 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.314+0000 7faa1f771740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7faa181a8dc0 con 0x557356c76390 2024-04-16T17:12:30.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.314+0000 7faa1f771740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7faa18073470 con 0x557356c6ce90 2024-04-16T17:12:30.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.314+0000 7faa16800640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557356c76390 0x557356c9f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:30.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.314+0000 7faa15400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557356c6d7b0 0x557356c820c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:30.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.314+0000 7faa16800640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557356c76390 0x557356c9f550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:38758/0 (socket says 172.21.15.45:38758) 2024-04-16T17:12:30.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.314+0000 7faa16800640 1 -- 172.21.15.45:0/112743238 learned_addr learned my addr 172.21.15.45:0/112743238 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:12:30.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.314+0000 7faa15e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557356c6ce90 0x557356c75b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:30.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.315+0000 7faa16800640 1 -- 172.21.15.45:0/112743238 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557356c6ce90 msgr2=0x557356c75b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:12:30.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.315+0000 7faa16800640 1 --2- 172.21.15.45:0/112743238 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557356c6ce90 0x557356c75b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:30.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.315+0000 7faa16800640 1 -- 172.21.15.45:0/112743238 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557356c6d7b0 msgr2=0x557356c820c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:12:30.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.315+0000 7faa16800640 1 --2- 172.21.15.45:0/112743238 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557356c6d7b0 0x557356c820c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:30.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.315+0000 7faa16800640 1 -- 172.21.15.45:0/112743238 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5573568a4170 con 0x557356c76390 2024-04-16T17:12:30.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.315+0000 7faa15400640 1 --2- 172.21.15.45:0/112743238 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557356c6d7b0 0x557356c820c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:12:30.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.315+0000 7faa16800640 1 --2- 172.21.15.45:0/112743238 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557356c76390 0x557356c9f550 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7faa10009f70 tx=0x7faa100124f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2b3c6e3b6be0ff59 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:12:30.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.315+0000 7faa02600640 1 -- 172.21.15.45:0/112743238 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa10016040 con 0x557356c76390 2024-04-16T17:12:30.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.315+0000 7faa02600640 1 -- 172.21.15.45:0/112743238 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faa100055b0 con 0x557356c76390 2024-04-16T17:12:30.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.315+0000 7faa02600640 1 -- 172.21.15.45:0/112743238 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa10012c70 con 0x557356c76390 2024-04-16T17:12:30.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.316+0000 7faa1f771740 1 -- 172.21.15.45:0/112743238 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557356c76390 msgr2=0x557356c9f550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:30.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.316+0000 7faa1f771740 1 --2- 172.21.15.45:0/112743238 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557356c76390 0x557356c9f550 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7faa10009f70 tx=0x7faa100124f0 comp rx=0 tx=0).stop 2024-04-16T17:12:30.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.316+0000 7faa1f771740 1 -- 172.21.15.45:0/112743238 shutdown_connections 2024-04-16T17:12:30.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.316+0000 7faa1f771740 1 --2- 172.21.15.45:0/112743238 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557356c76390 0x557356c9f550 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:30.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.316+0000 7faa1f771740 1 --2- 172.21.15.45:0/112743238 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557356c6d7b0 0x557356c820c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:30.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.316+0000 7faa1f771740 1 --2- 172.21.15.45:0/112743238 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557356c6ce90 0x557356c75b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:30.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.316+0000 7faa1f771740 1 -- 172.21.15.45:0/112743238 >> 172.21.15.45:0/112743238 conn(0x557356c675c0 msgr2=0x557356c66560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:12:30.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.316+0000 7faa1f771740 1 -- 172.21.15.45:0/112743238 shutdown_connections 2024-04-16T17:12:30.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.316+0000 7faa1f771740 1 -- 172.21.15.45:0/112743238 wait complete. 2024-04-16T17:12:30.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.317+0000 7faa1f771740 1 -- start start 2024-04-16T17:12:30.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.317+0000 7faa1f771740 1 Processor -- start 2024-04-16T17:12:30.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.317+0000 7faa1f771740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557356c6ce90 0x557356c7a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:30.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.317+0000 7faa1f771740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557356c6d7b0 0x557356c7ae00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:30.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.317+0000 7faa1f771740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557356c76390 0x557356cf6e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:30.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.318+0000 7faa1f771740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7faa18073b00 con 0x557356c6ce90 2024-04-16T17:12:30.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.318+0000 7faa1f771740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7faa181a86f0 con 0x557356c76390 2024-04-16T17:12:30.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.318+0000 7faa1f771740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7faa181a79a0 con 0x557356c6d7b0 2024-04-16T17:12:30.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.318+0000 7faa1f771740 10 client.0 _ll_register_callbacks cb 0x7faa0801c830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:12:30.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.318+0000 7faa15400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557356c6d7b0 0x557356c7ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:30.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.318+0000 7faa15400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557356c6d7b0 0x557356c7ae00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:49882/0 (socket says 172.21.15.45:49882) 2024-04-16T17:12:30.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.318+0000 7faa15e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557356c6ce90 0x557356c7a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:30.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.318+0000 7faa15400640 1 -- 172.21.15.45:0/1972884632 learned_addr learned my addr 172.21.15.45:0/1972884632 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:12:30.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.318+0000 7faa16800640 1 --2- 172.21.15.45:0/1972884632 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557356c76390 0x557356cf6e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:30.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.318+0000 7faa16800640 1 -- 172.21.15.45:0/1972884632 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557356c6d7b0 msgr2=0x557356c7ae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:12:30.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.318+0000 7faa16800640 1 --2- 172.21.15.45:0/1972884632 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557356c6d7b0 0x557356c7ae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:30.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.318+0000 7faa16800640 1 -- 172.21.15.45:0/1972884632 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557356c6ce90 msgr2=0x557356c7a8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:12:30.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.318+0000 7faa16800640 1 --2- 172.21.15.45:0/1972884632 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557356c6ce90 0x557356c7a8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:30.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.318+0000 7faa16800640 1 -- 172.21.15.45:0/1972884632 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557356cf74b0 con 0x557356c76390 2024-04-16T17:12:30.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.319+0000 7faa16800640 1 --2- 172.21.15.45:0/1972884632 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557356c76390 0x557356cf6e70 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7faa10011010 tx=0x7faa10005300 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f9f82d1540d2db42 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:12:30.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.319+0000 7fa9f7400640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-04-16T17:12:30.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.319+0000 7fa9f7400640 1 -- 172.21.15.45:0/1972884632 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa10016070 con 0x557356c76390 2024-04-16T17:12:30.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.319+0000 7fa9f7400640 1 -- 172.21.15.45:0/1972884632 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7faa10003b80 con 0x557356c76390 2024-04-16T17:12:30.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.319+0000 7faa1f771740 1 -- 172.21.15.45:0/1972884632 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faa100055b0 con 0x557356c76390 2024-04-16T17:12:30.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.319+0000 7fa9f7400640 1 -- 172.21.15.45:0/1972884632 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7faa10035050 con 0x557356c76390 2024-04-16T17:12:30.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.319+0000 7faa1f771740 10 client.27227 fetch_fsmap learned FSMap version 37 2024-04-16T17:12:30.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.319+0000 7faa1f771740 1 -- 172.21.15.45:0/1972884632 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x557356cfe860 con 0x557356c76390 2024-04-16T17:12:30.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.319+0000 7fa9f7400640 1 -- 172.21.15.45:0/1972884632 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faa100055b0 con 0x557356c76390 2024-04-16T17:12:30.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.320+0000 7fa9f7400640 1 -- 172.21.15.45:0/1972884632 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 37) ==== 8358+0+0 (secure 0 0 0) 0x7faa1000f040 con 0x557356c76390 2024-04-16T17:12:30.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.320+0000 7faa1f771740 10 client.27227 fetch_fsmap finished waiting for FSMap version 37 2024-04-16T17:12:30.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.320+0000 7faa1f771740 10 client.27227 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:12:30.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.320+0000 7faa1f771740 20 client.27227 populate_metadata read hostname 'smithi045' 2024-04-16T17:12:30.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.320+0000 7faa1f771740 1 --2- 172.21.15.45:0/1972884632 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x557356cfec30 0x557356d01020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:30.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.320+0000 7faa1f771740 4 client.27227 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-16T17:12:30.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.320+0000 7faa1f771740 1 -- 172.21.15.45:0/1972884632 --> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x557356bd9540 con 0x557356cfec30 2024-04-16T17:12:30.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.320+0000 7faa15e00640 1 --2- 172.21.15.45:0/1972884632 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x557356cfec30 0x557356d01020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:30.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.321+0000 7faa15e00640 1 --2- 172.21.15.45:0/1972884632 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x557356cfec30 0x557356d01020 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=24457ca80ae0a911 server_cookie=78c5c4111c81240a in_seq=0 out_seq=0 2024-04-16T17:12:30.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.321+0000 7fa9f7400640 10 client.27227 ms_handle_connect on v2:172.21.15.104:6838/3660070592 2024-04-16T17:12:30.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.322+0000 7fa9f7400640 1 -- 172.21.15.45:0/1972884632 <== mds.0 v2:172.21.15.104:6838/3660070592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x557356bd9540 con 0x557356cfec30 2024-04-16T17:12:30.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.322+0000 7fa9f7400640 10 client.27227 handle_command_reply: tid=0 2024-04-16T17:12:30.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.322+0000 7faa1f771740 1 client.27227 shutdown 2024-04-16T17:12:30.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.322+0000 7faa1f771740 1 -- 172.21.15.45:0/1972884632 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557356c76390 msgr2=0x557356cf6e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:30.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.322+0000 7faa1f771740 1 --2- 172.21.15.45:0/1972884632 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557356c76390 0x557356cf6e70 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7faa10011010 tx=0x7faa10005300 comp rx=0 tx=0).stop 2024-04-16T17:12:30.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.322+0000 7faa1f771740 1 -- 172.21.15.45:0/1972884632 shutdown_connections 2024-04-16T17:12:30.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.322+0000 7faa1f771740 1 --2- 172.21.15.45:0/1972884632 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x557356cfec30 0x557356d01020 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:30.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.322+0000 7faa1f771740 1 --2- 172.21.15.45:0/1972884632 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557356c6d7b0 0x557356c7ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:30.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.322+0000 7faa1f771740 1 --2- 172.21.15.45:0/1972884632 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557356c76390 0x557356cf6e70 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:30.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.322+0000 7faa1f771740 1 --2- 172.21.15.45:0/1972884632 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557356c6ce90 0x557356c7a8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:30.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.322+0000 7fa9f7400640 0 client.27227 ms_handle_reset on v2:172.21.15.104:6838/3660070592 2024-04-16T17:12:30.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.322+0000 7faa1f771740 1 -- 172.21.15.45:0/1972884632 >> 172.21.15.45:0/1972884632 conn(0x557356c675c0 msgr2=0x557356c689f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:12:30.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.323+0000 7faa1f771740 1 -- 172.21.15.45:0/1972884632 shutdown_connections 2024-04-16T17:12:30.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.323+0000 7faa1f771740 1 -- 172.21.15.45:0/1972884632 wait complete. 2024-04-16T17:12:30.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.323+0000 7faa1f771740 20 client.27227 trim_cache size 0 max 16384 2024-04-16T17:12:30.327 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-16T17:12:30.327 INFO:teuthology.orchestra.run.smithi045.stdout: "return_code": 0, 2024-04-16T17:12:30.327 INFO:teuthology.orchestra.run.smithi045.stdout: "scrub_tag": "7f881116-34de-4309-902c-84469a7bacd3", 2024-04-16T17:12:30.327 INFO:teuthology.orchestra.run.smithi045.stdout: "mode": "asynchronous" 2024-04-16T17:12:30.327 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-16T17:12:30.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.324+0000 7faa17200640 1 -- 172.21.15.45:0/3080797013 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fa9fc0a3da0 msgr2=0x7fa9fc0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:30.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.324+0000 7faa17200640 1 --2- 172.21.15.45:0/3080797013 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fa9fc0a3da0 0x7fa9fc0a6260 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x557356c6c960 tx=0x7faa08009840 comp rx=0 tx=0).stop 2024-04-16T17:12:30.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.324+0000 7faa17200640 1 -- 172.21.15.45:0/3080797013 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7faa181093f0 msgr2=0x7faa181a0bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:30.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.324+0000 7faa17200640 1 --2- 172.21.15.45:0/3080797013 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7faa181093f0 0x7faa181a0bc0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7faa0402c250 tx=0x7faa04004360 comp rx=0 tx=0).stop 2024-04-16T17:12:30.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.324+0000 7faa17200640 1 -- 172.21.15.45:0/3080797013 shutdown_connections 2024-04-16T17:12:30.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.324+0000 7faa17200640 1 --2- 172.21.15.45:0/3080797013 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fa9fc0a3da0 0x7fa9fc0a6260 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:30.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.324+0000 7faa17200640 1 --2- 172.21.15.45:0/3080797013 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faa1819a7f0 0x7faa1819aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:30.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.324+0000 7faa17200640 1 --2- 172.21.15.45:0/3080797013 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7faa181093f0 0x7faa181a0bc0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:30.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.324+0000 7faa17200640 1 --2- 172.21.15.45:0/3080797013 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faa18104320 0x7faa181a0680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:30.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.324+0000 7faa17200640 1 -- 172.21.15.45:0/3080797013 >> 172.21.15.45:0/3080797013 conn(0x7faa180ffb40 msgr2=0x7faa18101ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:12:30.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.325+0000 7faa17200640 1 -- 172.21.15.45:0/3080797013 shutdown_connections 2024-04-16T17:12:30.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.325+0000 7faa17200640 1 -- 172.21.15.45:0/3080797013 wait complete. 2024-04-16T17:12:30.354 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-16T17:12:30.354 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-16T17:12:30.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.559+0000 7fc317200640 1 -- 172.21.15.45:0/1468149058 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc3180743a0 msgr2=0x7fc318074780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:30.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.559+0000 7fc317200640 1 --2- 172.21.15.45:0/1468149058 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc3180743a0 0x7fc318074780 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fc308009860 tx=0x7fc30802bc00 comp rx=0 tx=0).stop 2024-04-16T17:12:30.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.560+0000 7fc317200640 1 -- 172.21.15.45:0/1468149058 shutdown_connections 2024-04-16T17:12:30.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.560+0000 7fc317200640 1 --2- 172.21.15.45:0/1468149058 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc318112eb0 0x7fc3181152a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:30.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.560+0000 7fc317200640 1 --2- 172.21.15.45:0/1468149058 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc318074cc0 0x7fc318112970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:30.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.560+0000 7fc317200640 1 --2- 172.21.15.45:0/1468149058 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc3180743a0 0x7fc318074780 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:30.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.560+0000 7fc317200640 1 -- 172.21.15.45:0/1468149058 >> 172.21.15.45:0/1468149058 conn(0x7fc3180fd9d0 msgr2=0x7fc3180ffdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:12:30.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.560+0000 7fc317200640 1 -- 172.21.15.45:0/1468149058 shutdown_connections 2024-04-16T17:12:30.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.560+0000 7fc317200640 1 -- 172.21.15.45:0/1468149058 wait complete. 2024-04-16T17:12:30.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.561+0000 7fc317200640 1 Processor -- start 2024-04-16T17:12:30.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.561+0000 7fc317200640 1 -- start start 2024-04-16T17:12:30.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.562+0000 7fc317200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc3180743a0 0x7fc31819ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:30.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.562+0000 7fc317200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc318074cc0 0x7fc31819f340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:30.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.562+0000 7fc317200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc318112eb0 0x7fc3181a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:30.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.562+0000 7fc317200640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fc318118880 con 0x7fc318074cc0 2024-04-16T17:12:30.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.562+0000 7fc317200640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fc318118700 con 0x7fc3180743a0 2024-04-16T17:12:30.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.562+0000 7fc317200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fc318118a00 con 0x7fc318112eb0 2024-04-16T17:12:30.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.562+0000 7fc316800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc318112eb0 0x7fc3181a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:30.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.562+0000 7fc315e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc3180743a0 0x7fc31819ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:30.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.562+0000 7fc316800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc318112eb0 0x7fc3181a3670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:49892/0 (socket says 172.21.15.45:49892) 2024-04-16T17:12:30.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.562+0000 7fc315e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc3180743a0 0x7fc31819ee00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:38790/0 (socket says 172.21.15.45:38790) 2024-04-16T17:12:30.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.562+0000 7fc316800640 1 -- 172.21.15.45:0/381297182 learned_addr learned my addr 172.21.15.45:0/381297182 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:12:30.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.562+0000 7fc315400640 1 --2- 172.21.15.45:0/381297182 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc318074cc0 0x7fc31819f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:30.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.563+0000 7fc316800640 1 -- 172.21.15.45:0/381297182 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc3180743a0 msgr2=0x7fc31819ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:30.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.563+0000 7fc316800640 1 --2- 172.21.15.45:0/381297182 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc3180743a0 0x7fc31819ee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:30.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.563+0000 7fc316800640 1 -- 172.21.15.45:0/381297182 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc318074cc0 msgr2=0x7fc31819f340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:30.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.563+0000 7fc316800640 1 --2- 172.21.15.45:0/381297182 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc318074cc0 0x7fc31819f340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:30.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.563+0000 7fc316800640 1 -- 172.21.15.45:0/381297182 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3181a3da0 con 0x7fc318112eb0 2024-04-16T17:12:30.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.563+0000 7fc315e00640 1 --2- 172.21.15.45:0/381297182 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc3180743a0 0x7fc31819ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:12:30.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.563+0000 7fc316800640 1 --2- 172.21.15.45:0/381297182 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc318112eb0 0x7fc3181a3670 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fc310009fa0 tx=0x7fc31000c800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:12:30.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.564+0000 7fc30ea00640 1 -- 172.21.15.45:0/381297182 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc310004b20 con 0x7fc318112eb0 2024-04-16T17:12:30.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.564+0000 7fc30ea00640 1 -- 172.21.15.45:0/381297182 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc310003e50 con 0x7fc318112eb0 2024-04-16T17:12:30.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.564+0000 7fc30ea00640 1 -- 172.21.15.45:0/381297182 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc31000b570 con 0x7fc318112eb0 2024-04-16T17:12:30.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.564+0000 7fc317200640 1 -- 172.21.15.45:0/381297182 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3181a4090 con 0x7fc318112eb0 2024-04-16T17:12:30.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.564+0000 7fc317200640 1 -- 172.21.15.45:0/381297182 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3181ab920 con 0x7fc318112eb0 2024-04-16T17:12:30.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.568+0000 7fc30ea00640 1 -- 172.21.15.45:0/381297182 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fc31001d020 con 0x7fc318112eb0 2024-04-16T17:12:30.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.569+0000 7fc31ffdb740 1 Processor -- start 2024-04-16T17:12:30.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.569+0000 7fc31ffdb740 1 -- start start 2024-04-16T17:12:30.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.569+0000 7fc30ea00640 1 --2- 172.21.15.45:0/381297182 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fc2fc0a3da0 0x7fc2fc0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:30.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.569+0000 7fc30ea00640 1 -- 172.21.15.45:0/381297182 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fc3100d0a10 con 0x7fc318112eb0 2024-04-16T17:12:30.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.570+0000 7fc31ffdb740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a534889390 0x55a534889770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:30.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.570+0000 7fc31ffdb740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a534889cb0 0x55a534892350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:30.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.570+0000 7fc31ffdb740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a534892890 0x55a534894c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:30.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.570+0000 7fc31ffdb740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55a5347ef120 con 0x55a534892890 2024-04-16T17:12:30.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.570+0000 7fc31ffdb740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55a534795340 con 0x55a534889cb0 2024-04-16T17:12:30.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.570+0000 7fc31ffdb740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55a5347eb0e0 con 0x55a534889390 2024-04-16T17:12:30.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.570+0000 7fc315400640 1 --2- 172.21.15.45:0/381297182 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fc2fc0a3da0 0x7fc2fc0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:30.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.570+0000 7fc316800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a534892890 0x55a534894c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:30.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.570+0000 7fc316800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a534892890 0x55a534894c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:48226/0 (socket says 172.21.15.45:48226) 2024-04-16T17:12:30.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.570+0000 7fc316800640 1 -- 172.21.15.45:0/3027986382 learned_addr learned my addr 172.21.15.45:0/3027986382 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:12:30.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.570+0000 7fc315e00640 1 --2- 172.21.15.45:0/3027986382 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a534889390 0x55a534889770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:30.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.570+0000 7fc315400640 1 --2- 172.21.15.45:0/3027986382 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a534889cb0 0x55a534892350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:30.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.571+0000 7fc316800640 1 -- 172.21.15.45:0/3027986382 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a534889390 msgr2=0x55a534889770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:12:30.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.571+0000 7fc316800640 1 --2- 172.21.15.45:0/3027986382 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a534889390 0x55a534889770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:30.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.571+0000 7fc316800640 1 -- 172.21.15.45:0/3027986382 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a534889cb0 msgr2=0x55a534892350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:12:30.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.571+0000 7fc316800640 1 --2- 172.21.15.45:0/3027986382 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a534889cb0 0x55a534892350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:30.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.571+0000 7fc316800640 1 -- 172.21.15.45:0/3027986382 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a5345cb7d0 con 0x55a534892890 2024-04-16T17:12:30.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.571+0000 7fc315400640 1 --2- 172.21.15.45:0/3027986382 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a534889cb0 0x55a534892350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:12:30.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.571+0000 7fc316800640 1 --2- 172.21.15.45:0/3027986382 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a534892890 0x55a534894c80 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fc31000ffd0 tx=0x7fc31009c200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb263f870b94355d server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:12:30.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.571+0000 7fc315400640 1 --2- 172.21.15.45:0/381297182 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fc2fc0a3da0 0x7fc2fc0a6260 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x55a534888e60 tx=0x7fc304009ae0 comp rx=0 tx=0).ready entity=mgr.25411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:12:30.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.571+0000 7fc303400640 1 -- 172.21.15.45:0/3027986382 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc31001a040 con 0x55a534892890 2024-04-16T17:12:30.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.572+0000 7fc303400640 1 -- 172.21.15.45:0/3027986382 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc31009c600 con 0x55a534892890 2024-04-16T17:12:30.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.572+0000 7fc303400640 1 -- 172.21.15.45:0/3027986382 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc310098040 con 0x55a534892890 2024-04-16T17:12:30.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.572+0000 7fc31ffdb740 1 -- 172.21.15.45:0/3027986382 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a534892890 msgr2=0x55a534894c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:30.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.572+0000 7fc31ffdb740 1 --2- 172.21.15.45:0/3027986382 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a534892890 0x55a534894c80 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fc31000ffd0 tx=0x7fc31009c200 comp rx=0 tx=0).stop 2024-04-16T17:12:30.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.572+0000 7fc31ffdb740 1 -- 172.21.15.45:0/3027986382 shutdown_connections 2024-04-16T17:12:30.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.573+0000 7fc31ffdb740 1 --2- 172.21.15.45:0/3027986382 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a534892890 0x55a534894c80 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:30.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.573+0000 7fc31ffdb740 1 --2- 172.21.15.45:0/3027986382 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a534889cb0 0x55a534892350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:30.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.573+0000 7fc31ffdb740 1 --2- 172.21.15.45:0/3027986382 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a534889390 0x55a534889770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:30.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.573+0000 7fc31ffdb740 1 -- 172.21.15.45:0/3027986382 >> 172.21.15.45:0/3027986382 conn(0x55a534883ac0 msgr2=0x55a534883ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:12:30.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.573+0000 7fc31ffdb740 1 -- 172.21.15.45:0/3027986382 shutdown_connections 2024-04-16T17:12:30.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.573+0000 7fc31ffdb740 1 -- 172.21.15.45:0/3027986382 wait complete. 2024-04-16T17:12:30.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.573+0000 7fc31ffdb740 1 -- start start 2024-04-16T17:12:30.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.574+0000 7fc31ffdb740 1 Processor -- start 2024-04-16T17:12:30.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.574+0000 7fc31ffdb740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a534889390 0x55a53488f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:30.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.574+0000 7fc31ffdb740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a534889cb0 0x55a53488fee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:30.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.574+0000 7fc31ffdb740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a534892890 0x55a534890420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:30.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.574+0000 7fc31ffdb740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55a5348955c0 con 0x55a534892890 2024-04-16T17:12:30.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.574+0000 7fc31ffdb740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55a5347a8710 con 0x55a534889cb0 2024-04-16T17:12:30.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.574+0000 7fc31ffdb740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55a534895740 con 0x55a534889390 2024-04-16T17:12:30.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.574+0000 7fc31ffdb740 10 client.0 _ll_register_callbacks cb 0x55a5347a3af0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:12:30.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.575+0000 7fc315400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a534889cb0 0x55a53488fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:30.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.575+0000 7fc315e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a534889390 0x55a53488f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:30.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.575+0000 7fc315400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a534889cb0 0x55a53488fee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:38822/0 (socket says 172.21.15.45:38822) 2024-04-16T17:12:30.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.575+0000 7fc315400640 1 -- 172.21.15.45:0/1612463174 learned_addr learned my addr 172.21.15.45:0/1612463174 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:12:30.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.575+0000 7fc316800640 1 --2- 172.21.15.45:0/1612463174 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a534892890 0x55a534890420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:30.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.575+0000 7fc315400640 1 -- 172.21.15.45:0/1612463174 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a534889390 msgr2=0x55a53488f9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:12:30.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.575+0000 7fc315400640 1 --2- 172.21.15.45:0/1612463174 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a534889390 0x55a53488f9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:30.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.575+0000 7fc315400640 1 -- 172.21.15.45:0/1612463174 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a534892890 msgr2=0x55a534890420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:12:30.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.575+0000 7fc315400640 1 --2- 172.21.15.45:0/1612463174 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a534892890 0x55a534890420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:30.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.575+0000 7fc315400640 1 -- 172.21.15.45:0/1612463174 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a5345013f0 con 0x55a534889cb0 2024-04-16T17:12:30.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.575+0000 7fc316800640 1 --2- 172.21.15.45:0/1612463174 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a534892890 0x55a534890420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:12:30.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.575+0000 7fc315400640 1 --2- 172.21.15.45:0/1612463174 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a534889cb0 0x55a53488fee0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fc3040038a0 tx=0x7fc3040046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=31d66c43a1628c6e server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:12:30.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.575+0000 7fc2f7e00640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-04-16T17:12:30.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.576+0000 7fc2f7e00640 1 -- 172.21.15.45:0/1612463174 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc30400eb30 con 0x55a534889cb0 2024-04-16T17:12:30.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.576+0000 7fc2f7e00640 1 -- 172.21.15.45:0/1612463174 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fc30400ecd0 con 0x55a534889cb0 2024-04-16T17:12:30.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.576+0000 7fc31ffdb740 1 -- 172.21.15.45:0/1612463174 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc31009c600 con 0x55a534889cb0 2024-04-16T17:12:30.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.576+0000 7fc2f7e00640 1 -- 172.21.15.45:0/1612463174 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fc30401f410 con 0x55a534889cb0 2024-04-16T17:12:30.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.576+0000 7fc31ffdb740 10 client.27236 fetch_fsmap learned FSMap version 37 2024-04-16T17:12:30.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.576+0000 7fc31ffdb740 1 -- 172.21.15.45:0/1612463174 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55a5344c10f0 con 0x55a534889cb0 2024-04-16T17:12:30.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.577+0000 7fc2f7e00640 1 -- 172.21.15.45:0/1612463174 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc31009c600 con 0x55a534889cb0 2024-04-16T17:12:30.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.577+0000 7fc2f7e00640 1 -- 172.21.15.45:0/1612463174 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 37) ==== 8358+0+0 (secure 0 0 0) 0x7fc30401b070 con 0x55a534889cb0 2024-04-16T17:12:30.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.577+0000 7fc31ffdb740 10 client.27236 fetch_fsmap finished waiting for FSMap version 37 2024-04-16T17:12:30.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.577+0000 7fc31ffdb740 10 client.27236 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:12:30.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.577+0000 7fc31ffdb740 20 client.27236 populate_metadata read hostname 'smithi045' 2024-04-16T17:12:30.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.577+0000 7fc31ffdb740 1 --2- 172.21.15.45:0/1612463174 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55a5348a2c00 0x55a5348a5080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:30.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.577+0000 7fc31ffdb740 4 client.27236 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:12:30.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.577+0000 7fc31ffdb740 1 -- 172.21.15.45:0/1612463174 --> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a5347fcb40 con 0x55a5348a2c00 2024-04-16T17:12:30.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.577+0000 7fc315e00640 1 --2- 172.21.15.45:0/1612463174 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55a5348a2c00 0x55a5348a5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:30.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.578+0000 7fc315e00640 1 --2- 172.21.15.45:0/1612463174 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55a5348a2c00 0x55a5348a5080 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3466508980ca5d4b server_cookie=d4926f64e1584cbb in_seq=0 out_seq=0 2024-04-16T17:12:30.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.578+0000 7fc2f7e00640 10 client.27236 ms_handle_connect on v2:172.21.15.104:6838/3660070592 2024-04-16T17:12:30.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.579+0000 7fc2f7e00640 1 -- 172.21.15.45:0/1612463174 <== mds.0 v2:172.21.15.104:6838/3660070592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a5347fcb40 con 0x55a5348a2c00 2024-04-16T17:12:30.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.579+0000 7fc2f7e00640 10 client.27236 handle_command_reply: tid=0 2024-04-16T17:12:30.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.579+0000 7fc31ffdb740 1 client.27236 shutdown 2024-04-16T17:12:30.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.579+0000 7fc31ffdb740 1 -- 172.21.15.45:0/1612463174 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a534889cb0 msgr2=0x55a53488fee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:30.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.579+0000 7fc31ffdb740 1 --2- 172.21.15.45:0/1612463174 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a534889cb0 0x55a53488fee0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fc3040038a0 tx=0x7fc3040046c0 comp rx=0 tx=0).stop 2024-04-16T17:12:30.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.579+0000 7fc31ffdb740 1 -- 172.21.15.45:0/1612463174 shutdown_connections 2024-04-16T17:12:30.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.579+0000 7fc31ffdb740 1 --2- 172.21.15.45:0/1612463174 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a534892890 0x55a534890420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:30.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.579+0000 7fc31ffdb740 1 --2- 172.21.15.45:0/1612463174 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a534889cb0 0x55a53488fee0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:30.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.579+0000 7fc31ffdb740 1 --2- 172.21.15.45:0/1612463174 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55a5348a2c00 0x55a5348a5080 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:30.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.579+0000 7fc31ffdb740 1 --2- 172.21.15.45:0/1612463174 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a534889390 0x55a53488f9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:30.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.579+0000 7fc31ffdb740 1 -- 172.21.15.45:0/1612463174 >> 172.21.15.45:0/1612463174 conn(0x55a534883ac0 msgr2=0x55a534885810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:12:30.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.579+0000 7fc2f7e00640 0 client.27236 ms_handle_reset on v2:172.21.15.104:6838/3660070592 2024-04-16T17:12:30.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.579+0000 7fc31ffdb740 1 -- 172.21.15.45:0/1612463174 shutdown_connections 2024-04-16T17:12:30.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.579+0000 7fc31ffdb740 1 -- 172.21.15.45:0/1612463174 wait complete. 2024-04-16T17:12:30.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.580+0000 7fc31ffdb740 20 client.27236 trim_cache size 0 max 16384 2024-04-16T17:12:30.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.602+0000 7fc31ffdb740 1 Processor -- start 2024-04-16T17:12:30.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.603+0000 7fc31ffdb740 1 -- start start 2024-04-16T17:12:30.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.603+0000 7fc31ffdb740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a534889390 0x55a5348920d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:30.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.603+0000 7fc31ffdb740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a534889cb0 0x55a53489e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:30.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.603+0000 7fc31ffdb740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a534892890 0x55a53488e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:30.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.603+0000 7fc31ffdb740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fc3181b8230 con 0x55a534892890 2024-04-16T17:12:30.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.603+0000 7fc31ffdb740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fc3181b5a70 con 0x55a534889390 2024-04-16T17:12:30.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.603+0000 7fc31ffdb740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fc3181b38a0 con 0x55a534889cb0 2024-04-16T17:12:30.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.603+0000 7fc315e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a534889390 0x55a5348920d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:30.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.603+0000 7fc315e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a534889390 0x55a5348920d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:38826/0 (socket says 172.21.15.45:38826) 2024-04-16T17:12:30.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.603+0000 7fc315e00640 1 -- 172.21.15.45:0/2538533133 learned_addr learned my addr 172.21.15.45:0/2538533133 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:12:30.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.603+0000 7fc316800640 1 --2- 172.21.15.45:0/2538533133 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a534892890 0x55a53488e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:30.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.604+0000 7fc315e00640 1 -- 172.21.15.45:0/2538533133 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a534889cb0 msgr2=0x55a53489e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:12:30.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.604+0000 7fc315400640 1 --2- 172.21.15.45:0/2538533133 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a534889cb0 0x55a53489e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:30.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.604+0000 7fc315e00640 1 --2- 172.21.15.45:0/2538533133 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a534889cb0 0x55a53489e480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:30.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.604+0000 7fc315e00640 1 -- 172.21.15.45:0/2538533133 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a534892890 msgr2=0x55a53488e8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:12:30.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.604+0000 7fc315e00640 1 --2- 172.21.15.45:0/2538533133 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a534892890 0x55a53488e8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:30.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.604+0000 7fc315e00640 1 -- 172.21.15.45:0/2538533133 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a534889ae0 con 0x55a534889390 2024-04-16T17:12:30.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.604+0000 7fc316800640 1 --2- 172.21.15.45:0/2538533133 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a534892890 0x55a53488e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:12:30.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.604+0000 7fc315400640 1 --2- 172.21.15.45:0/2538533133 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a534889cb0 0x55a53489e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:12:30.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.604+0000 7fc315e00640 1 --2- 172.21.15.45:0/2538533133 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a534889390 0x55a5348920d0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fc30802c210 tx=0x7fc30802bc00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=653d0d2b01647f85 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:12:30.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.605+0000 7fc302a00640 1 -- 172.21.15.45:0/2538533133 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc308039040 con 0x55a534889390 2024-04-16T17:12:30.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.605+0000 7fc302a00640 1 -- 172.21.15.45:0/2538533133 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc308034920 con 0x55a534889390 2024-04-16T17:12:30.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.605+0000 7fc302a00640 1 -- 172.21.15.45:0/2538533133 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc30802c5a0 con 0x55a534889390 2024-04-16T17:12:30.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.605+0000 7fc31ffdb740 1 -- 172.21.15.45:0/2538533133 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a534889390 msgr2=0x55a5348920d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:30.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.605+0000 7fc31ffdb740 1 --2- 172.21.15.45:0/2538533133 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a534889390 0x55a5348920d0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fc30802c210 tx=0x7fc30802bc00 comp rx=0 tx=0).stop 2024-04-16T17:12:30.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.605+0000 7fc31ffdb740 1 -- 172.21.15.45:0/2538533133 shutdown_connections 2024-04-16T17:12:30.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.605+0000 7fc31ffdb740 1 --2- 172.21.15.45:0/2538533133 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a534889cb0 0x55a53489e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:30.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.605+0000 7fc31ffdb740 1 --2- 172.21.15.45:0/2538533133 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a534892890 0x55a53488e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:30.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.605+0000 7fc31ffdb740 1 --2- 172.21.15.45:0/2538533133 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a534889390 0x55a5348920d0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:30.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.605+0000 7fc31ffdb740 1 -- 172.21.15.45:0/2538533133 >> 172.21.15.45:0/2538533133 conn(0x55a534883ac0 msgr2=0x55a534885780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:12:30.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.606+0000 7fc31ffdb740 1 -- 172.21.15.45:0/2538533133 shutdown_connections 2024-04-16T17:12:30.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.606+0000 7fc31ffdb740 1 -- 172.21.15.45:0/2538533133 wait complete. 2024-04-16T17:12:30.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.606+0000 7fc31ffdb740 1 -- start start 2024-04-16T17:12:30.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.606+0000 7fc31ffdb740 1 Processor -- start 2024-04-16T17:12:30.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.607+0000 7fc31ffdb740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a534889390 0x55a53489aef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:30.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.607+0000 7fc31ffdb740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a534889cb0 0x55a5348962b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:30.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.607+0000 7fc31ffdb740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a534892890 0x55a5348969a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:30.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.607+0000 7fc31ffdb740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fc3181b3f30 con 0x55a534892890 2024-04-16T17:12:30.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.607+0000 7fc31ffdb740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fc3181b53a0 con 0x55a534889cb0 2024-04-16T17:12:30.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.607+0000 7fc31ffdb740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fc3181b4600 con 0x55a534889390 2024-04-16T17:12:30.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.607+0000 7fc31ffdb740 10 client.0 _ll_register_callbacks cb 0x7fc304009dd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:12:30.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.607+0000 7fc315e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a534889390 0x55a53489aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:30.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.607+0000 7fc315400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a534889cb0 0x55a5348962b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:30.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.607+0000 7fc315e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a534889390 0x55a53489aef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:49934/0 (socket says 172.21.15.45:49934) 2024-04-16T17:12:30.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.607+0000 7fc315e00640 1 -- 172.21.15.45:0/3380128279 learned_addr learned my addr 172.21.15.45:0/3380128279 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:12:30.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.607+0000 7fc316800640 1 --2- 172.21.15.45:0/3380128279 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a534892890 0x55a5348969a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:30.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.608+0000 7fc315e00640 1 -- 172.21.15.45:0/3380128279 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a534889cb0 msgr2=0x55a5348962b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:12:30.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.608+0000 7fc315e00640 1 --2- 172.21.15.45:0/3380128279 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a534889cb0 0x55a5348962b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:30.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.608+0000 7fc315e00640 1 -- 172.21.15.45:0/3380128279 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a534892890 msgr2=0x55a5348969a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:12:30.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.608+0000 7fc315e00640 1 --2- 172.21.15.45:0/3380128279 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a534892890 0x55a5348969a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:30.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.608+0000 7fc315e00640 1 -- 172.21.15.45:0/3380128279 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a5349081c0 con 0x55a534889390 2024-04-16T17:12:30.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.608+0000 7fc316800640 1 --2- 172.21.15.45:0/3380128279 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a534892890 0x55a5348969a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:12:30.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.608+0000 7fc315400640 1 --2- 172.21.15.45:0/3380128279 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a534889cb0 0x55a5348962b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:12:30.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.608+0000 7fc315e00640 1 --2- 172.21.15.45:0/3380128279 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a534889390 0x55a53489aef0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fc308000c00 tx=0x7fc308005870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b426d14f7056f576 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:12:30.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.608+0000 7fc2f7e00640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-04-16T17:12:30.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.608+0000 7fc2f7e00640 1 -- 172.21.15.45:0/3380128279 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc308054070 con 0x55a534889390 2024-04-16T17:12:30.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.608+0000 7fc31ffdb740 1 -- 172.21.15.45:0/3380128279 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc308034920 con 0x55a534889390 2024-04-16T17:12:30.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.608+0000 7fc2f7e00640 1 -- 172.21.15.45:0/3380128279 <== mon.1 v2:172.21.15.117:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fc308034440 con 0x55a534889390 2024-04-16T17:12:30.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.609+0000 7fc2f7e00640 1 -- 172.21.15.45:0/3380128279 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fc308039040 con 0x55a534889390 2024-04-16T17:12:30.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.609+0000 7fc31ffdb740 10 client.27187 fetch_fsmap learned FSMap version 37 2024-04-16T17:12:30.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.609+0000 7fc31ffdb740 1 -- 172.21.15.45:0/3380128279 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55a5349092e0 con 0x55a534889390 2024-04-16T17:12:30.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.609+0000 7fc2f7e00640 1 -- 172.21.15.45:0/3380128279 <== mon.1 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc308034920 con 0x55a534889390 2024-04-16T17:12:30.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.609+0000 7fc2f7e00640 1 -- 172.21.15.45:0/3380128279 <== mon.1 v2:172.21.15.117:3300/0 5 ==== fsmap(e 37) ==== 8358+0+0 (secure 0 0 0) 0x7fc30802c5a0 con 0x55a534889390 2024-04-16T17:12:30.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.609+0000 7fc31ffdb740 10 client.27187 fetch_fsmap finished waiting for FSMap version 37 2024-04-16T17:12:30.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.609+0000 7fc31ffdb740 10 client.27187 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:12:30.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.609+0000 7fc31ffdb740 20 client.27187 populate_metadata read hostname 'smithi045' 2024-04-16T17:12:30.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.609+0000 7fc31ffdb740 1 --2- 172.21.15.45:0/3380128279 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55a534909960 0x55a53490bd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:30.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.609+0000 7fc31ffdb740 4 client.27187 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-16T17:12:30.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.609+0000 7fc31ffdb740 1 -- 172.21.15.45:0/3380128279 --> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55a5347f69b0 con 0x55a534909960 2024-04-16T17:12:30.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.610+0000 7fc315400640 1 --2- 172.21.15.45:0/3380128279 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55a534909960 0x55a53490bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:30.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.610+0000 7fc2f7e00640 10 client.27187 ms_handle_connect on v2:172.21.15.104:6838/3660070592 2024-04-16T17:12:30.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.610+0000 7fc315400640 1 --2- 172.21.15.45:0/3380128279 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55a534909960 0x55a53490bd50 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9a3b7d6a86622a08 server_cookie=d86084cc6fde770d in_seq=0 out_seq=0 2024-04-16T17:12:30.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.611+0000 7fc2f7e00640 1 -- 172.21.15.45:0/3380128279 <== mds.0 v2:172.21.15.104:6838/3660070592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55a5347f69b0 con 0x55a534909960 2024-04-16T17:12:30.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.611+0000 7fc2f7e00640 10 client.27187 handle_command_reply: tid=0 2024-04-16T17:12:30.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.611+0000 7fc31ffdb740 1 client.27187 shutdown 2024-04-16T17:12:30.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.611+0000 7fc31ffdb740 1 -- 172.21.15.45:0/3380128279 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a534889390 msgr2=0x55a53489aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:30.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.611+0000 7fc31ffdb740 1 --2- 172.21.15.45:0/3380128279 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a534889390 0x55a53489aef0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fc308000c00 tx=0x7fc308005870 comp rx=0 tx=0).stop 2024-04-16T17:12:30.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.611+0000 7fc31ffdb740 1 -- 172.21.15.45:0/3380128279 shutdown_connections 2024-04-16T17:12:30.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.611+0000 7fc31ffdb740 1 --2- 172.21.15.45:0/3380128279 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a534892890 0x55a5348969a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:30.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.611+0000 7fc31ffdb740 1 --2- 172.21.15.45:0/3380128279 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a534889cb0 0x55a5348962b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:30.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.611+0000 7fc31ffdb740 1 --2- 172.21.15.45:0/3380128279 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55a534909960 0x55a53490bd50 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:30.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.611+0000 7fc31ffdb740 1 --2- 172.21.15.45:0/3380128279 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a534889390 0x55a53489aef0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:30.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.611+0000 7fc31ffdb740 1 -- 172.21.15.45:0/3380128279 >> 172.21.15.45:0/3380128279 conn(0x55a534883ac0 msgr2=0x55a534882290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:12:30.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.611+0000 7fc2f7e00640 0 client.27187 ms_handle_reset on v2:172.21.15.104:6838/3660070592 2024-04-16T17:12:30.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.611+0000 7fc31ffdb740 1 -- 172.21.15.45:0/3380128279 shutdown_connections 2024-04-16T17:12:30.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.611+0000 7fc31ffdb740 1 -- 172.21.15.45:0/3380128279 wait complete. 2024-04-16T17:12:30.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.612+0000 7fc31ffdb740 20 client.27187 trim_cache size 0 max 16384 2024-04-16T17:12:30.616 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-16T17:12:30.616 INFO:teuthology.orchestra.run.smithi045.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-16T17:12:30.616 INFO:teuthology.orchestra.run.smithi045.stdout: "scrubs": { 2024-04-16T17:12:30.616 INFO:teuthology.orchestra.run.smithi045.stdout: "7f881116-34de-4309-902c-84469a7bacd3": { 2024-04-16T17:12:30.616 INFO:teuthology.orchestra.run.smithi045.stdout: "path": "/", 2024-04-16T17:12:30.616 INFO:teuthology.orchestra.run.smithi045.stdout: "tag": "7f881116-34de-4309-902c-84469a7bacd3", 2024-04-16T17:12:30.616 INFO:teuthology.orchestra.run.smithi045.stdout: "options": "recursive,force" 2024-04-16T17:12:30.616 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:12:30.616 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:12:30.616 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-16T17:12:30.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.612+0000 7fc317200640 1 -- 172.21.15.45:0/381297182 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fc2fc0a3da0 msgr2=0x7fc2fc0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:30.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.612+0000 7fc317200640 1 --2- 172.21.15.45:0/381297182 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fc2fc0a3da0 0x7fc2fc0a6260 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x55a534888e60 tx=0x7fc304009ae0 comp rx=0 tx=0).stop 2024-04-16T17:12:30.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.612+0000 7fc317200640 1 -- 172.21.15.45:0/381297182 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc318112eb0 msgr2=0x7fc3181a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:30.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.612+0000 7fc317200640 1 --2- 172.21.15.45:0/381297182 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc318112eb0 0x7fc3181a3670 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fc310009fa0 tx=0x7fc31000c800 comp rx=0 tx=0).stop 2024-04-16T17:12:30.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.612+0000 7fc317200640 1 -- 172.21.15.45:0/381297182 shutdown_connections 2024-04-16T17:12:30.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.612+0000 7fc317200640 1 --2- 172.21.15.45:0/381297182 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fc2fc0a3da0 0x7fc2fc0a6260 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:30.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.612+0000 7fc317200640 1 --2- 172.21.15.45:0/381297182 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc318112eb0 0x7fc3181a3670 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:30.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.612+0000 7fc317200640 1 --2- 172.21.15.45:0/381297182 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc318074cc0 0x7fc31819f340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:30.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.613+0000 7fc317200640 1 --2- 172.21.15.45:0/381297182 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc3180743a0 0x7fc31819ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:30.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.613+0000 7fc317200640 1 -- 172.21.15.45:0/381297182 >> 172.21.15.45:0/381297182 conn(0x7fc3180fd9d0 msgr2=0x7fc3180ff2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:12:30.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.613+0000 7fc317200640 1 -- 172.21.15.45:0/381297182 shutdown_connections 2024-04-16T17:12:30.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:12:30.613+0000 7fc317200640 1 -- 172.21.15.45:0/381297182 wait complete. 2024-04-16T17:12:30.633 INFO:tasks.cephfs.filesystem:scrub status for tag:7f881116-34de-4309-902c-84469a7bacd3 - {'path': '/', 'tag': '7f881116-34de-4309-902c-84469a7bacd3', 'options': 'recursive,force'} 2024-04-16T17:12:30.804 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:12:30 smithi117 ceph-mon[36730]: pgmap v175: 129 pgs: 129 active+clean; 51 MiB data, 687 MiB used, 1.0 TiB / 1.0 TiB avail; 479 KiB/s rd, 535 KiB/s wr, 44 op/s 2024-04-16T17:12:30.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:12:30 smithi104 ceph-mon[36457]: pgmap v175: 129 pgs: 129 active+clean; 51 MiB data, 687 MiB used, 1.0 TiB / 1.0 TiB avail; 479 KiB/s rd, 535 KiB/s wr, 44 op/s 2024-04-16T17:12:31.012 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:12:26 smithi045 ceph-mon[31676]: pgmap v173: 129 pgs: 129 active+clean; 49 MiB data, 649 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 275 KiB/s wr, 20 op/s 2024-04-16T17:12:31.012 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:12:28 smithi045 ceph-mon[31676]: pgmap v174: 129 pgs: 129 active+clean; 51 MiB data, 687 MiB used, 1.0 TiB / 1.0 TiB avail; 510 KiB/s rd, 791 KiB/s wr, 57 op/s 2024-04-16T17:12:31.012 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:12:28 smithi045 ceph-mon[31676]: from='client.? 172.21.15.45:0/2023581863' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:12:31.012 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:12:30 smithi045 ceph-mon[31676]: pgmap v175: 129 pgs: 129 active+clean; 51 MiB data, 687 MiB used, 1.0 TiB / 1.0 TiB avail; 479 KiB/s rd, 535 KiB/s wr, 44 op/s 2024-04-16T17:12:31.268 INFO:tasks.workunit.client.0.smithi045.stdout: 1 89149 1.63 MB/sec execute 153 sec latency 41.184 ms 2024-04-16T17:12:31.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:12:31 smithi104 ceph-mon[36457]: scrub queued for path: / 2024-04-16T17:12:31.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:12:31 smithi104 ceph-mon[36457]: scrub summary: idle+waiting paths [/] 2024-04-16T17:12:31.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:12:31 smithi104 ceph-mon[36457]: scrub summary: active paths [/] 2024-04-16T17:12:31.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:12:31 smithi104 ceph-mon[36457]: scrub summary: active paths [/] 2024-04-16T17:12:31.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:12:31 smithi104 ceph-mon[36457]: scrub summary: active paths [/] 2024-04-16T17:12:31.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:12:31 smithi104 ceph-mon[36457]: scrub summary: idle+waiting paths [/] 2024-04-16T17:12:31.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:12:31 smithi104 ceph-mon[36457]: scrub summary: active paths [/] 2024-04-16T17:12:31.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:12:31 smithi104 ceph-mon[36457]: scrub summary: idle+waiting paths [/] 2024-04-16T17:12:31.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:12:31 smithi104 ceph-mon[36457]: scrub summary: idle+waiting paths [/] 2024-04-16T17:12:31.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:12:31 smithi104 ceph-mon[36457]: scrub summary: idle+waiting paths [/] 2024-04-16T17:12:32.054 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:12:31 smithi117 ceph-mon[36730]: scrub queued for path: / 2024-04-16T17:12:32.054 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:12:31 smithi117 ceph-mon[36730]: scrub summary: idle+waiting paths [/] 2024-04-16T17:12:32.054 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:12:31 smithi117 ceph-mon[36730]: scrub summary: active paths [/] 2024-04-16T17:12:32.055 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:12:31 smithi117 ceph-mon[36730]: scrub summary: active paths [/] 2024-04-16T17:12:32.055 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:12:31 smithi117 ceph-mon[36730]: scrub summary: active paths [/] 2024-04-16T17:12:32.055 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:12:31 smithi117 ceph-mon[36730]: scrub summary: idle+waiting paths [/] 2024-04-16T17:12:32.055 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:12:31 smithi117 ceph-mon[36730]: scrub summary: active paths [/] 2024-04-16T17:12:32.055 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:12:31 smithi117 ceph-mon[36730]: scrub summary: idle+waiting paths [/] 2024-04-16T17:12:32.055 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:12:31 smithi117 ceph-mon[36730]: scrub summary: idle+waiting paths [/] 2024-04-16T17:12:32.055 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:12:31 smithi117 ceph-mon[36730]: scrub summary: idle+waiting paths [/] 2024-04-16T17:12:32.169 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:12:31 smithi045 ceph-mon[31676]: scrub queued for path: / 2024-04-16T17:12:32.169 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:12:31 smithi045 ceph-mon[31676]: scrub summary: idle+waiting paths [/] 2024-04-16T17:12:32.169 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:12:31 smithi045 ceph-mon[31676]: scrub summary: active paths [/] 2024-04-16T17:12:32.169 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:12:31 smithi045 ceph-mon[31676]: scrub summary: active paths [/] 2024-04-16T17:12:32.169 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:12:31 smithi045 ceph-mon[31676]: scrub summary: active paths [/] 2024-04-16T17:12:32.169 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:12:31 smithi045 ceph-mon[31676]: scrub summary: idle+waiting paths [/] 2024-04-16T17:12:32.170 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:12:31 smithi045 ceph-mon[31676]: scrub summary: active paths [/] 2024-04-16T17:12:32.170 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:12:31 smithi045 ceph-mon[31676]: scrub summary: idle+waiting paths [/] 2024-04-16T17:12:32.170 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:12:31 smithi045 ceph-mon[31676]: scrub summary: idle+waiting paths [/] 2024-04-16T17:12:32.170 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:12:31 smithi045 ceph-mon[31676]: scrub summary: idle+waiting paths [/] 2024-04-16T17:12:32.268 INFO:tasks.workunit.client.0.smithi045.stdout: 1 89526 1.63 MB/sec execute 154 sec latency 76.530 ms 2024-04-16T17:12:32.925 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:12:32 smithi045 ceph-mon[31676]: pgmap v176: 129 pgs: 129 active+clean; 50 MiB data, 676 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 838 KiB/s wr, 107 op/s 2024-04-16T17:12:32.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:12:32 smithi104 ceph-mon[36457]: pgmap v176: 129 pgs: 129 active+clean; 50 MiB data, 676 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 838 KiB/s wr, 107 op/s 2024-04-16T17:12:33.054 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:12:32 smithi117 ceph-mon[36730]: pgmap v176: 129 pgs: 129 active+clean; 50 MiB data, 676 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 838 KiB/s wr, 107 op/s 2024-04-16T17:12:33.268 INFO:tasks.workunit.client.0.smithi045.stdout: 1 90136 1.64 MB/sec execute 155 sec latency 64.528 ms 2024-04-16T17:12:34.268 INFO:tasks.workunit.client.0.smithi045.stdout: 1 90375 1.64 MB/sec execute 156 sec latency 126.644 ms 2024-04-16T17:12:34.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:12:34 smithi104 ceph-mon[36457]: pgmap v177: 129 pgs: 129 active+clean; 50 MiB data, 676 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 820 KiB/s wr, 101 op/s 2024-04-16T17:12:35.054 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:12:34 smithi117 ceph-mon[36730]: pgmap v177: 129 pgs: 129 active+clean; 50 MiB data, 676 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 820 KiB/s wr, 101 op/s 2024-04-16T17:12:35.268 INFO:tasks.workunit.client.0.smithi045.stdout: 1 90730 1.63 MB/sec execute 157 sec latency 107.940 ms 2024-04-16T17:12:36.269 INFO:tasks.workunit.client.0.smithi045.stdout: 1 91712 1.66 MB/sec execute 158 sec latency 52.132 ms 2024-04-16T17:12:36.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:12:36 smithi104 ceph-mon[36457]: pgmap v178: 129 pgs: 129 active+clean; 50 MiB data, 676 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 820 KiB/s wr, 101 op/s 2024-04-16T17:12:37.054 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:12:36 smithi117 ceph-mon[36730]: pgmap v178: 129 pgs: 129 active+clean; 50 MiB data, 676 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 820 KiB/s wr, 101 op/s 2024-04-16T17:12:37.268 INFO:tasks.workunit.client.0.smithi045.stdout: 1 92115 1.66 MB/sec execute 159 sec latency 89.464 ms 2024-04-16T17:12:38.269 INFO:tasks.workunit.client.0.smithi045.stdout: 1 92508 1.68 MB/sec execute 160 sec latency 97.003 ms 2024-04-16T17:12:38.468 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:12:34 smithi045 ceph-mon[31676]: pgmap v177: 129 pgs: 129 active+clean; 50 MiB data, 676 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 820 KiB/s wr, 101 op/s 2024-04-16T17:12:38.468 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:12:36 smithi045 ceph-mon[31676]: pgmap v178: 129 pgs: 129 active+clean; 50 MiB data, 676 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 820 KiB/s wr, 101 op/s 2024-04-16T17:12:38.725 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:12:38 smithi045 ceph-mon[31676]: pgmap v179: 129 pgs: 129 active+clean; 52 MiB data, 702 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.0 MiB/s wr, 132 op/s 2024-04-16T17:12:38.725 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:12:38 smithi045 ceph-mon[31676]: scrub summary: idle 2024-04-16T17:12:38.856 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:12:38 smithi117 ceph-mon[36730]: pgmap v179: 129 pgs: 129 active+clean; 52 MiB data, 702 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.0 MiB/s wr, 132 op/s 2024-04-16T17:12:38.856 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:12:38 smithi117 ceph-mon[36730]: scrub summary: idle 2024-04-16T17:12:38.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:12:38 smithi104 ceph-mon[36457]: pgmap v179: 129 pgs: 129 active+clean; 52 MiB data, 702 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.0 MiB/s wr, 132 op/s 2024-04-16T17:12:38.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:12:38 smithi104 ceph-mon[36457]: scrub summary: idle 2024-04-16T17:12:39.269 INFO:tasks.workunit.client.0.smithi045.stdout: 1 92757 1.67 MB/sec execute 161 sec latency 44.446 ms 2024-04-16T17:12:39.880 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:12:39 smithi045 ceph-mon[31676]: scrub summary: idle 2024-04-16T17:12:39.880 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:12:39 smithi045 ceph-mon[31676]: scrub summary: idle 2024-04-16T17:12:39.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:12:39 smithi104 ceph-mon[36457]: scrub summary: idle 2024-04-16T17:12:39.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:12:39 smithi104 ceph-mon[36457]: scrub summary: idle 2024-04-16T17:12:40.054 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:12:39 smithi117 ceph-mon[36730]: scrub summary: idle 2024-04-16T17:12:40.054 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:12:39 smithi117 ceph-mon[36730]: scrub summary: idle 2024-04-16T17:12:40.269 INFO:tasks.workunit.client.0.smithi045.stdout: 1 93248 1.67 MB/sec execute 162 sec latency 39.342 ms 2024-04-16T17:12:40.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:12:40 smithi104 ceph-mon[36457]: pgmap v180: 129 pgs: 129 active+clean; 52 MiB data, 702 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 534 KiB/s wr, 95 op/s 2024-04-16T17:12:41.054 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:12:40 smithi117 ceph-mon[36730]: pgmap v180: 129 pgs: 129 active+clean; 52 MiB data, 702 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 534 KiB/s wr, 95 op/s 2024-04-16T17:12:41.269 INFO:tasks.workunit.client.0.smithi045.stdout: 1 93850 1.68 MB/sec execute 163 sec latency 36.826 ms 2024-04-16T17:12:41.297 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:41.297+0000 7fc9efe00640 1 -- 172.21.15.45:0/4117954318 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc9f0104c50 msgr2=0x7fc9f0108c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:41.298 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:41.297+0000 7fc9efe00640 1 --2- 172.21.15.45:0/4117954318 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc9f0104c50 0x7fc9f0108c90 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fc9dc009860 tx=0x7fc9dc02bb70 comp rx=0 tx=0).stop 2024-04-16T17:12:41.298 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:41.297+0000 7fc9efe00640 1 -- 172.21.15.45:0/4117954318 shutdown_connections 2024-04-16T17:12:41.298 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:41.297+0000 7fc9efe00640 1 --2- 172.21.15.45:0/4117954318 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc9f0109370 0x7fc9f0110f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:41.298 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:41.297+0000 7fc9efe00640 1 --2- 172.21.15.45:0/4117954318 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc9f0104c50 0x7fc9f0108c90 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:41.298 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:41.297+0000 7fc9efe00640 1 --2- 172.21.15.45:0/4117954318 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc9f01042a0 0x7fc9f0104680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:41.298 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:41.297+0000 7fc9efe00640 1 -- 172.21.15.45:0/4117954318 >> 172.21.15.45:0/4117954318 conn(0x7fc9f00ffae0 msgr2=0x7fc9f0101f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:12:41.298 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:41.297+0000 7fc9efe00640 1 -- 172.21.15.45:0/4117954318 shutdown_connections 2024-04-16T17:12:41.298 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:41.297+0000 7fc9efe00640 1 -- 172.21.15.45:0/4117954318 wait complete. 2024-04-16T17:12:41.299 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:41.298+0000 7fc9efe00640 1 Processor -- start 2024-04-16T17:12:41.299 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:41.298+0000 7fc9efe00640 1 -- start start 2024-04-16T17:12:41.299 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:41.298+0000 7fc9efe00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc9f01042a0 0x7fc9f01a06a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:41.299 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:41.298+0000 7fc9efe00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc9f0109370 0x7fc9f01a0be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:41.300 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:41.298+0000 7fc9efe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc9f019a760 0x7fc9f019abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:41.300 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:41.298+0000 7fc9efe00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fc9f0114400 con 0x7fc9f019a760 2024-04-16T17:12:41.300 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:41.298+0000 7fc9efe00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fc9f0114280 con 0x7fc9f01042a0 2024-04-16T17:12:41.300 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:41.298+0000 7fc9efe00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fc9f0114580 con 0x7fc9f0109370 2024-04-16T17:12:41.300 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:41.299+0000 7fc9eea00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc9f01042a0 0x7fc9f01a06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:41.300 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:41.299+0000 7fc9eea00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc9f01042a0 0x7fc9f01a06a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:60480/0 (socket says 172.21.15.45:60480) 2024-04-16T17:12:41.300 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:41.299+0000 7fc9eea00640 1 -- 172.21.15.45:0/3705714267 learned_addr learned my addr 172.21.15.45:0/3705714267 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:12:41.300 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:41.299+0000 7fc9ef400640 1 --2- 172.21.15.45:0/3705714267 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc9f019a760 0x7fc9f019abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:41.301 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:41.299+0000 7fc9eea00640 1 -- 172.21.15.45:0/3705714267 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc9f0109370 msgr2=0x7fc9f01a0be0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-16T17:12:41.301 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:41.299+0000 7fc9eea00640 1 --2- 172.21.15.45:0/3705714267 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc9f0109370 0x7fc9f01a0be0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:41.301 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:41.299+0000 7fc9eea00640 1 -- 172.21.15.45:0/3705714267 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc9f019a760 msgr2=0x7fc9f019abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:41.301 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:41.299+0000 7fc9eea00640 1 --2- 172.21.15.45:0/3705714267 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc9f019a760 0x7fc9f019abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:41.301 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:41.299+0000 7fc9eea00640 1 -- 172.21.15.45:0/3705714267 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9f019b360 con 0x7fc9f01042a0 2024-04-16T17:12:41.301 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:41.300+0000 7fc9eea00640 1 --2- 172.21.15.45:0/3705714267 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc9f01042a0 0x7fc9f01a06a0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fc9e000d950 tx=0x7fc9e000de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:12:41.301 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:41.300+0000 7fc9e7400640 1 -- 172.21.15.45:0/3705714267 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9e0017070 con 0x7fc9f01042a0 2024-04-16T17:12:41.301 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:41.300+0000 7fc9efe00640 1 -- 172.21.15.45:0/3705714267 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9f01a7440 con 0x7fc9f01042a0 2024-04-16T17:12:41.302 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:41.300+0000 7fc9e7400640 1 -- 172.21.15.45:0/3705714267 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc9e0009c30 con 0x7fc9f01042a0 2024-04-16T17:12:41.302 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:41.300+0000 7fc9e7400640 1 -- 172.21.15.45:0/3705714267 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9e000f6b0 con 0x7fc9f01042a0 2024-04-16T17:12:41.302 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:41.300+0000 7fc9efe00640 1 -- 172.21.15.45:0/3705714267 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9f01a7840 con 0x7fc9f01042a0 2024-04-16T17:12:41.305 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:41.305+0000 7fc9e7400640 1 -- 172.21.15.45:0/3705714267 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fc9e00109d0 con 0x7fc9f01042a0 2024-04-16T17:12:41.306 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:41.305+0000 7fc9f85f4740 1 Processor -- start 2024-04-16T17:12:41.306 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:41.305+0000 7fc9f85f4740 1 -- start start 2024-04-16T17:12:41.307 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:41.306+0000 7fc9f85f4740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5583e3a197f0 0x5583e3a18540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:41.307 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:41.306+0000 7fc9f85f4740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5583e3a18a80 0x5583e3a18f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:41.307 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:41.306+0000 7fc9f85f4740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5583e3a22000 0x5583e3a243f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:41.307 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:41.306+0000 7fc9f85f4740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5583e397d780 con 0x5583e3a18a80 2024-04-16T17:12:41.307 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:41.306+0000 7fc9f85f4740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5583e3923f40 con 0x5583e3a197f0 2024-04-16T17:12:41.307 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:41.306+0000 7fc9f85f4740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5583e3979740 con 0x5583e3a22000 2024-04-16T17:12:41.308 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:41.306+0000 7fc9ef400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5583e3a197f0 0x5583e3a18540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:41.308 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:41.306+0000 7fc9ee000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5583e3a22000 0x5583e3a243f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:41.308 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:41.306+0000 7fc9ee000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5583e3a22000 0x5583e3a243f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:47130/0 (socket says 172.21.15.45:47130) 2024-04-16T17:12:41.308 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:41.306+0000 7fc9ee000640 1 -- 172.21.15.45:0/4252294126 learned_addr learned my addr 172.21.15.45:0/4252294126 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:12:41.308 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:41.306+0000 7fc9eea00640 1 --2- 172.21.15.45:0/4252294126 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5583e3a18a80 0x5583e3a18f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:41.308 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:41.306+0000 7fc9ee000640 1 -- 172.21.15.45:0/4252294126 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5583e3a197f0 msgr2=0x5583e3a18540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:12:41.308 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:41.306+0000 7fc9ee000640 1 --2- 172.21.15.45:0/4252294126 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5583e3a197f0 0x5583e3a18540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:41.308 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:41.306+0000 7fc9ee000640 1 -- 172.21.15.45:0/4252294126 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5583e3a18a80 msgr2=0x5583e3a18f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:12:41.308 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:41.306+0000 7fc9ee000640 1 --2- 172.21.15.45:0/4252294126 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5583e3a18a80 0x5583e3a18f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:41.308 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:41.306+0000 7fc9ee000640 1 -- 172.21.15.45:0/4252294126 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5583e3690210 con 0x5583e3a22000 2024-04-16T17:12:41.308 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:41.307+0000 7fc9e7400640 1 --2- 172.21.15.45:0/3705714267 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fc9d40a3d10 0x7fc9d40a61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:41.309 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:41.307+0000 7fc9ee000640 1 --2- 172.21.15.45:0/4252294126 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5583e3a22000 0x5583e3a243f0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fc9dc009780 tx=0x7fc9dc005da0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=da55be5ffd379832 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:12:41.309 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:41.307+0000 7fc9e7400640 1 -- 172.21.15.45:0/3705714267 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fc9e00d07f0 con 0x7fc9f01042a0 2024-04-16T17:12:41.309 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:41.307+0000 7fc9dbe00640 1 -- 172.21.15.45:0/4252294126 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9dc039040 con 0x5583e3a22000 2024-04-16T17:12:41.309 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:41.308+0000 7fc9dbe00640 1 -- 172.21.15.45:0/4252294126 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc9dc004040 con 0x5583e3a22000 2024-04-16T17:12:41.309 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:41.308+0000 7fc9ef400640 1 --2- 172.21.15.45:0/3705714267 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fc9d40a3d10 0x7fc9d40a61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:41.310 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:41.308+0000 7fc9dbe00640 1 -- 172.21.15.45:0/4252294126 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9dc032640 con 0x5583e3a22000 2024-04-16T17:12:41.310 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:41.308+0000 7fc9f85f4740 1 -- 172.21.15.45:0/4252294126 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5583e3a22000 msgr2=0x5583e3a243f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:41.310 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:41.308+0000 7fc9f85f4740 1 --2- 172.21.15.45:0/4252294126 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5583e3a22000 0x5583e3a243f0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fc9dc009780 tx=0x7fc9dc005da0 comp rx=0 tx=0).stop 2024-04-16T17:12:41.310 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:41.309+0000 7fc9ef400640 1 --2- 172.21.15.45:0/3705714267 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fc9d40a3d10 0x7fc9d40a61d0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fc9e8005fd0 tx=0x7fc9e8009600 comp rx=0 tx=0).ready entity=mgr.25411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:12:41.310 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:41.309+0000 7fc9f85f4740 1 -- 172.21.15.45:0/4252294126 shutdown_connections 2024-04-16T17:12:41.310 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:41.309+0000 7fc9f85f4740 1 --2- 172.21.15.45:0/4252294126 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5583e3a22000 0x5583e3a243f0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:41.310 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:41.309+0000 7fc9f85f4740 1 --2- 172.21.15.45:0/4252294126 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5583e3a18a80 0x5583e3a18f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:41.310 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:41.309+0000 7fc9f85f4740 1 --2- 172.21.15.45:0/4252294126 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5583e3a197f0 0x5583e3a18540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:41.310 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:41.309+0000 7fc9f85f4740 1 -- 172.21.15.45:0/4252294126 >> 172.21.15.45:0/4252294126 conn(0x5583e39b35d0 msgr2=0x5583e3a13770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:12:41.316 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:41.309+0000 7fc9f85f4740 1 -- 172.21.15.45:0/4252294126 shutdown_connections 2024-04-16T17:12:41.316 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:41.309+0000 7fc9f85f4740 1 -- 172.21.15.45:0/4252294126 wait complete. 2024-04-16T17:12:41.316 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:41.310+0000 7fc9f85f4740 1 -- start start 2024-04-16T17:12:41.316 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:41.310+0000 7fc9f85f4740 1 Processor -- start 2024-04-16T17:12:41.316 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:41.310+0000 7fc9f85f4740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5583e3a18a80 0x5583e3a21ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:41.316 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:41.310+0000 7fc9f85f4740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5583e3a197f0 0x5583e3a1ee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:41.317 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:41.310+0000 7fc9f85f4740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5583e3a22000 0x5583e3a1f580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:41.317 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:41.310+0000 7fc9f85f4740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5583e3a24cd0 con 0x5583e3a197f0 2024-04-16T17:12:41.317 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:41.310+0000 7fc9f85f4740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5583e3937390 con 0x5583e3a22000 2024-04-16T17:12:41.317 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:41.310+0000 7fc9f85f4740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5583e3a24e50 con 0x5583e3a18a80 2024-04-16T17:12:41.317 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:41.310+0000 7fc9f85f4740 10 client.0 _ll_register_callbacks cb 0x5583e3920f60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:12:41.317 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:41.311+0000 7fc9eea00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5583e3a18a80 0x5583e3a21ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:41.317 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:41.311+0000 7fc9ee000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5583e3a197f0 0x5583e3a1ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:41.317 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:41.311+0000 7fc9ee000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5583e3a197f0 0x5583e3a1ee40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38484/0 (socket says 172.21.15.45:38484) 2024-04-16T17:12:41.317 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:41.311+0000 7fc9ee000640 1 -- 172.21.15.45:0/2409167266 learned_addr learned my addr 172.21.15.45:0/2409167266 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:12:41.317 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:41.311+0000 7fc9ef400640 1 --2- 172.21.15.45:0/2409167266 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5583e3a22000 0x5583e3a1f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:41.317 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:41.311+0000 7fc9ee000640 1 -- 172.21.15.45:0/2409167266 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5583e3a18a80 msgr2=0x5583e3a21ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:12:41.318 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:41.311+0000 7fc9ee000640 1 --2- 172.21.15.45:0/2409167266 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5583e3a18a80 0x5583e3a21ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:41.318 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:41.311+0000 7fc9ee000640 1 -- 172.21.15.45:0/2409167266 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5583e3a22000 msgr2=0x5583e3a1f580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:12:41.318 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:41.311+0000 7fc9ee000640 1 --2- 172.21.15.45:0/2409167266 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5583e3a22000 0x5583e3a1f580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:41.318 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:41.311+0000 7fc9ee000640 1 -- 172.21.15.45:0/2409167266 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5583e375c620 con 0x5583e3a197f0 2024-04-16T17:12:41.318 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:41.311+0000 7fc9eea00640 1 --2- 172.21.15.45:0/2409167266 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5583e3a18a80 0x5583e3a21ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:12:41.318 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:41.312+0000 7fc9ee000640 1 --2- 172.21.15.45:0/2409167266 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5583e3a197f0 0x5583e3a1ee40 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fc9dc034480 tx=0x7fc9dc005f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=894b33be2b39874b server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:12:41.318 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:41.312+0000 7fc9d8c00640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-16T17:12:41.319 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:41.312+0000 7fc9d8c00640 1 -- 172.21.15.45:0/2409167266 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9dc039040 con 0x5583e3a197f0 2024-04-16T17:12:41.319 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:41.312+0000 7fc9d8c00640 1 -- 172.21.15.45:0/2409167266 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fc9dc034870 con 0x5583e3a197f0 2024-04-16T17:12:41.319 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:41.312+0000 7fc9f85f4740 1 -- 172.21.15.45:0/2409167266 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc9dc004040 con 0x5583e3a197f0 2024-04-16T17:12:41.319 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:41.312+0000 7fc9d8c00640 1 -- 172.21.15.45:0/2409167266 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fc9dc034ab0 con 0x5583e3a197f0 2024-04-16T17:12:41.319 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:41.312+0000 7fc9f85f4740 10 client.17085 fetch_fsmap learned FSMap version 37 2024-04-16T17:12:41.319 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:41.312+0000 7fc9f85f4740 1 -- 172.21.15.45:0/2409167266 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x5583e3a30df0 con 0x5583e3a197f0 2024-04-16T17:12:41.319 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:41.313+0000 7fc9d8c00640 1 -- 172.21.15.45:0/2409167266 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc9dc004040 con 0x5583e3a197f0 2024-04-16T17:12:41.319 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:41.313+0000 7fc9d8c00640 1 -- 172.21.15.45:0/2409167266 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 37) ==== 8358+0+0 (secure 0 0 0) 0x7fc9dc03c070 con 0x5583e3a197f0 2024-04-16T17:12:41.319 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:41.313+0000 7fc9f85f4740 10 client.17085 fetch_fsmap finished waiting for FSMap version 37 2024-04-16T17:12:41.319 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:41.313+0000 7fc9f85f4740 10 client.17085 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T17:12:41.319 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:41.313+0000 7fc9f85f4740 20 client.17085 populate_metadata read hostname 'smithi045' 2024-04-16T17:12:41.319 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:41.313+0000 7fc9f85f4740 1 --2- 172.21.15.45:0/2409167266 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x5583e3a32140 0x5583e3a34530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:41.319 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:41.313+0000 7fc9f85f4740 4 client.17085 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:12:41.319 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:41.313+0000 7fc9f85f4740 1 -- 172.21.15.45:0/2409167266 --> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5583e398b1a0 con 0x5583e3a32140 2024-04-16T17:12:41.319 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:41.313+0000 7fc9eea00640 1 --2- 172.21.15.45:0/2409167266 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x5583e3a32140 0x5583e3a34530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:41.320 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:41.314+0000 7fc9eea00640 1 --2- 172.21.15.45:0/2409167266 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x5583e3a32140 0x5583e3a34530 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5a62f6b2c4dbd60 server_cookie=284eb960ef9ee5af in_seq=0 out_seq=0 2024-04-16T17:12:41.320 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:41.314+0000 7fc9d8c00640 10 client.17085 ms_handle_connect on v2:172.21.15.104:6838/3660070592 2024-04-16T17:12:41.320 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:41.314+0000 7fc9d8c00640 1 -- 172.21.15.45:0/2409167266 <== mds.0 v2:172.21.15.104:6838/3660070592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5583e398b1a0 con 0x5583e3a32140 2024-04-16T17:12:41.320 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:41.314+0000 7fc9d8c00640 10 client.17085 handle_command_reply: tid=0 2024-04-16T17:12:41.320 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:41.315+0000 7fc9f85f4740 1 client.17085 shutdown 2024-04-16T17:12:41.320 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:41.315+0000 7fc9f85f4740 1 -- 172.21.15.45:0/2409167266 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5583e3a197f0 msgr2=0x5583e3a1ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:41.320 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:41.315+0000 7fc9f85f4740 1 --2- 172.21.15.45:0/2409167266 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5583e3a197f0 0x5583e3a1ee40 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fc9dc034480 tx=0x7fc9dc005f10 comp rx=0 tx=0).stop 2024-04-16T17:12:41.320 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:41.315+0000 7fc9f85f4740 1 -- 172.21.15.45:0/2409167266 shutdown_connections 2024-04-16T17:12:41.320 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:41.315+0000 7fc9f85f4740 1 --2- 172.21.15.45:0/2409167266 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5583e3a22000 0x5583e3a1f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:41.320 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:41.315+0000 7fc9f85f4740 1 --2- 172.21.15.45:0/2409167266 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5583e3a197f0 0x5583e3a1ee40 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:41.320 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:41.315+0000 7fc9f85f4740 1 --2- 172.21.15.45:0/2409167266 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x5583e3a32140 0x5583e3a34530 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:41.320 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:41.315+0000 7fc9f85f4740 1 --2- 172.21.15.45:0/2409167266 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5583e3a18a80 0x5583e3a21ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:41.320 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:41.315+0000 7fc9f85f4740 1 -- 172.21.15.45:0/2409167266 >> 172.21.15.45:0/2409167266 conn(0x5583e39b35d0 msgr2=0x5583e3a154c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:12:41.320 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:41.315+0000 7fc9d8c00640 0 client.17085 ms_handle_reset on v2:172.21.15.104:6838/3660070592 2024-04-16T17:12:41.321 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:41.315+0000 7fc9f85f4740 1 -- 172.21.15.45:0/2409167266 shutdown_connections 2024-04-16T17:12:41.321 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:41.315+0000 7fc9f85f4740 1 -- 172.21.15.45:0/2409167266 wait complete. 2024-04-16T17:12:41.321 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:41.315+0000 7fc9f85f4740 20 client.17085 trim_cache size 0 max 16384 2024-04-16T17:12:41.333 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:41.332+0000 7fc9f85f4740 1 Processor -- start 2024-04-16T17:12:41.333 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:41.332+0000 7fc9f85f4740 1 -- start start 2024-04-16T17:12:41.333 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:41.333+0000 7fc9f85f4740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5583e3a18a80 0x5583e3a21570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:41.333 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:41.333+0000 7fc9f85f4740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5583e3a197f0 0x5583e3a21ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:41.334 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:41.333+0000 7fc9f85f4740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5583e3a22000 0x5583e3a37550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:41.334 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:41.333+0000 7fc9f85f4740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fc9f01b3d90 con 0x5583e3a197f0 2024-04-16T17:12:41.334 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:41.333+0000 7fc9f85f4740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fc9f01b15d0 con 0x5583e3a22000 2024-04-16T17:12:41.334 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:41.333+0000 7fc9f85f4740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fc9f01af400 con 0x5583e3a18a80 2024-04-16T17:12:41.335 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:41.333+0000 7fc9ef400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5583e3a22000 0x5583e3a37550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:41.335 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:41.333+0000 7fc9eea00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5583e3a18a80 0x5583e3a21570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:41.335 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:41.333+0000 7fc9ef400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5583e3a22000 0x5583e3a37550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:60500/0 (socket says 172.21.15.45:60500) 2024-04-16T17:12:41.335 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:41.333+0000 7fc9ef400640 1 -- 172.21.15.45:0/2517595862 learned_addr learned my addr 172.21.15.45:0/2517595862 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:12:41.335 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:41.333+0000 7fc9ee000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5583e3a197f0 0x5583e3a21ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:41.335 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:41.334+0000 7fc9eea00640 1 -- 172.21.15.45:0/2517595862 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5583e3a22000 msgr2=0x5583e3a37550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:12:41.335 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:41.334+0000 7fc9eea00640 1 --2- 172.21.15.45:0/2517595862 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5583e3a22000 0x5583e3a37550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:41.335 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:41.334+0000 7fc9eea00640 1 -- 172.21.15.45:0/2517595862 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5583e3a197f0 msgr2=0x5583e3a21ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:12:41.335 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:41.334+0000 7fc9eea00640 1 --2- 172.21.15.45:0/2517595862 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5583e3a197f0 0x5583e3a21ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:41.335 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:41.334+0000 7fc9eea00640 1 -- 172.21.15.45:0/2517595862 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5583e3a17d50 con 0x5583e3a18a80 2024-04-16T17:12:41.335 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:41.334+0000 7fc9eea00640 1 --2- 172.21.15.45:0/2517595862 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5583e3a18a80 0x5583e3a21570 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fc9e00cdb90 tx=0x7fc9e000bf60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4df83ecb11e55e41 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:12:41.335 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:41.334+0000 7fc9db400640 1 -- 172.21.15.45:0/2517595862 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9e0020040 con 0x5583e3a18a80 2024-04-16T17:12:41.336 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:41.334+0000 7fc9db400640 1 -- 172.21.15.45:0/2517595862 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc9e00a0790 con 0x5583e3a18a80 2024-04-16T17:12:41.336 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:41.334+0000 7fc9db400640 1 -- 172.21.15.45:0/2517595862 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9e00d03e0 con 0x5583e3a18a80 2024-04-16T17:12:41.336 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:41.335+0000 7fc9f85f4740 1 -- 172.21.15.45:0/2517595862 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5583e3a18a80 msgr2=0x5583e3a21570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:41.337 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:41.335+0000 7fc9f85f4740 1 --2- 172.21.15.45:0/2517595862 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5583e3a18a80 0x5583e3a21570 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fc9e00cdb90 tx=0x7fc9e000bf60 comp rx=0 tx=0).stop 2024-04-16T17:12:41.337 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:41.335+0000 7fc9f85f4740 1 -- 172.21.15.45:0/2517595862 shutdown_connections 2024-04-16T17:12:41.337 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:41.335+0000 7fc9f85f4740 1 --2- 172.21.15.45:0/2517595862 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5583e3a22000 0x5583e3a37550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:41.337 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:41.335+0000 7fc9f85f4740 1 --2- 172.21.15.45:0/2517595862 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5583e3a197f0 0x5583e3a21ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:41.337 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:41.335+0000 7fc9f85f4740 1 --2- 172.21.15.45:0/2517595862 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5583e3a18a80 0x5583e3a21570 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:41.337 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:41.335+0000 7fc9f85f4740 1 -- 172.21.15.45:0/2517595862 >> 172.21.15.45:0/2517595862 conn(0x5583e39b35d0 msgr2=0x5583e3a1d710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:12:41.337 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:41.335+0000 7fc9f85f4740 1 -- 172.21.15.45:0/2517595862 shutdown_connections 2024-04-16T17:12:41.337 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:41.335+0000 7fc9f85f4740 1 -- 172.21.15.45:0/2517595862 wait complete. 2024-04-16T17:12:41.337 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:41.336+0000 7fc9f85f4740 1 -- start start 2024-04-16T17:12:41.337 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:41.336+0000 7fc9f85f4740 1 Processor -- start 2024-04-16T17:12:41.338 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:41.337+0000 7fc9f85f4740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5583e3a18a80 0x5583e3a37a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:41.338 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:41.337+0000 7fc9f85f4740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5583e3a197f0 0x5583e3a34e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:41.338 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:41.337+0000 7fc9f85f4740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5583e3a22000 0x5583e3a35540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:41.338 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:41.337+0000 7fc9f85f4740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fc9f01afa90 con 0x5583e3a18a80 2024-04-16T17:12:41.338 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:41.337+0000 7fc9f85f4740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fc9f01b0f00 con 0x5583e3a22000 2024-04-16T17:12:41.338 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:41.337+0000 7fc9f85f4740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fc9f01b0160 con 0x5583e3a197f0 2024-04-16T17:12:41.338 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:41.337+0000 7fc9f85f4740 10 client.0 _ll_register_callbacks cb 0x7fc9e8009a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:12:41.338 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:41.337+0000 7fc9ee000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5583e3a197f0 0x5583e3a34e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:41.339 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:41.337+0000 7fc9ef400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5583e3a22000 0x5583e3a35540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:41.339 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:41.337+0000 7fc9ee000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5583e3a197f0 0x5583e3a34e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:47148/0 (socket says 172.21.15.45:47148) 2024-04-16T17:12:41.339 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:41.337+0000 7fc9ee000640 1 -- 172.21.15.45:0/4047830367 learned_addr learned my addr 172.21.15.45:0/4047830367 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:12:41.339 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:41.338+0000 7fc9ee000640 1 -- 172.21.15.45:0/4047830367 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5583e3a22000 msgr2=0x5583e3a35540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:12:41.339 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:41.338+0000 7fc9ee000640 1 --2- 172.21.15.45:0/4047830367 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5583e3a22000 0x5583e3a35540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:41.339 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:41.338+0000 7fc9eea00640 1 --2- 172.21.15.45:0/4047830367 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5583e3a18a80 0x5583e3a37a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:41.339 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:41.338+0000 7fc9ee000640 1 -- 172.21.15.45:0/4047830367 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5583e3a18a80 msgr2=0x5583e3a37a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:12:41.340 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:41.338+0000 7fc9ee000640 1 --2- 172.21.15.45:0/4047830367 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5583e3a18a80 0x5583e3a37a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:41.340 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:41.338+0000 7fc9ee000640 1 -- 172.21.15.45:0/4047830367 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5583e3a97320 con 0x5583e3a197f0 2024-04-16T17:12:41.341 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:41.338+0000 7fc9eea00640 1 --2- 172.21.15.45:0/4047830367 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5583e3a18a80 0x5583e3a37a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:12:41.341 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:41.338+0000 7fc9ee000640 1 --2- 172.21.15.45:0/4047830367 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5583e3a197f0 0x5583e3a34e20 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fc9dc0057c0 tx=0x7fc9dc02c1f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ce4af5287d21fb4d server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:12:41.341 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:41.338+0000 7fc9d8c00640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-04-16T17:12:41.341 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:41.338+0000 7fc9d8c00640 1 -- 172.21.15.45:0/4047830367 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9dc039070 con 0x5583e3a197f0 2024-04-16T17:12:41.341 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:41.338+0000 7fc9d8c00640 1 -- 172.21.15.45:0/4047830367 <== mon.1 v2:172.21.15.117:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fc9dc032640 con 0x5583e3a197f0 2024-04-16T17:12:41.341 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:41.338+0000 7fc9f85f4740 1 -- 172.21.15.45:0/4047830367 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc9e00a0790 con 0x5583e3a197f0 2024-04-16T17:12:41.341 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:41.339+0000 7fc9d8c00640 1 -- 172.21.15.45:0/4047830367 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fc9dc060050 con 0x5583e3a197f0 2024-04-16T17:12:41.341 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:41.339+0000 7fc9f85f4740 10 client.27202 fetch_fsmap learned FSMap version 37 2024-04-16T17:12:41.341 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:41.339+0000 7fc9f85f4740 1 -- 172.21.15.45:0/4047830367 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x5583e3a984b0 con 0x5583e3a197f0 2024-04-16T17:12:41.341 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:41.339+0000 7fc9d8c00640 1 -- 172.21.15.45:0/4047830367 <== mon.1 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc9e00a0790 con 0x5583e3a197f0 2024-04-16T17:12:41.341 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:41.339+0000 7fc9d8c00640 1 -- 172.21.15.45:0/4047830367 <== mon.1 v2:172.21.15.117:3300/0 5 ==== fsmap(e 37) ==== 8358+0+0 (secure 0 0 0) 0x7fc9dc03c040 con 0x5583e3a197f0 2024-04-16T17:12:41.341 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:41.339+0000 7fc9f85f4740 10 client.27202 fetch_fsmap finished waiting for FSMap version 37 2024-04-16T17:12:41.341 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:41.339+0000 7fc9f85f4740 10 client.27202 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T17:12:41.341 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:41.339+0000 7fc9f85f4740 20 client.27202 populate_metadata read hostname 'smithi045' 2024-04-16T17:12:41.341 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:41.339+0000 7fc9f85f4740 1 --2- 172.21.15.45:0/4047830367 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x5583e3a98b70 0x5583e3a9af90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:41.341 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:41.339+0000 7fc9f85f4740 4 client.27202 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-16T17:12:41.342 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:41.339+0000 7fc9f85f4740 1 -- 172.21.15.45:0/4047830367 --> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5583e3985010 con 0x5583e3a98b70 2024-04-16T17:12:41.342 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:41.340+0000 7fc9eea00640 1 --2- 172.21.15.45:0/4047830367 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x5583e3a98b70 0x5583e3a9af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:41.342 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:41.340+0000 7fc9d8c00640 10 client.27202 ms_handle_connect on v2:172.21.15.104:6838/3660070592 2024-04-16T17:12:41.342 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:41.340+0000 7fc9eea00640 1 --2- 172.21.15.45:0/4047830367 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x5583e3a98b70 0x5583e3a9af90 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=efbb993fcba9d48b server_cookie=b2cbde118193ea26 in_seq=0 out_seq=0 2024-04-16T17:12:41.396 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:41.395+0000 7fc9d8c00640 1 -- 172.21.15.45:0/4047830367 <== mds.0 v2:172.21.15.104:6838/3660070592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5583e3985010 con 0x5583e3a98b70 2024-04-16T17:12:41.396 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:41.395+0000 7fc9d8c00640 10 client.27202 handle_command_reply: tid=0 2024-04-16T17:12:41.396 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:41.395+0000 7fc9f85f4740 1 client.27202 shutdown 2024-04-16T17:12:41.396 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:41.395+0000 7fc9f85f4740 1 -- 172.21.15.45:0/4047830367 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5583e3a197f0 msgr2=0x5583e3a34e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:41.396 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:41.395+0000 7fc9f85f4740 1 --2- 172.21.15.45:0/4047830367 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5583e3a197f0 0x5583e3a34e20 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fc9dc0057c0 tx=0x7fc9dc02c1f0 comp rx=0 tx=0).stop 2024-04-16T17:12:41.396 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:41.395+0000 7fc9f85f4740 1 -- 172.21.15.45:0/4047830367 shutdown_connections 2024-04-16T17:12:41.396 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:41.395+0000 7fc9f85f4740 1 --2- 172.21.15.45:0/4047830367 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x5583e3a98b70 0x5583e3a9af90 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:41.396 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:41.395+0000 7fc9f85f4740 1 --2- 172.21.15.45:0/4047830367 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5583e3a197f0 0x5583e3a34e20 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:41.396 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:41.395+0000 7fc9f85f4740 1 --2- 172.21.15.45:0/4047830367 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5583e3a22000 0x5583e3a35540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:41.396 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:41.395+0000 7fc9f85f4740 1 --2- 172.21.15.45:0/4047830367 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5583e3a18a80 0x5583e3a37a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:41.397 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:41.395+0000 7fc9f85f4740 1 -- 172.21.15.45:0/4047830367 >> 172.21.15.45:0/4047830367 conn(0x5583e39b35d0 msgr2=0x5583e3a2d110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:12:41.397 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:41.395+0000 7fc9d8c00640 0 client.27202 ms_handle_reset on v2:172.21.15.104:6838/3660070592 2024-04-16T17:12:41.397 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:41.395+0000 7fc9f85f4740 1 -- 172.21.15.45:0/4047830367 shutdown_connections 2024-04-16T17:12:41.397 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:41.395+0000 7fc9f85f4740 1 -- 172.21.15.45:0/4047830367 wait complete. 2024-04-16T17:12:41.397 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:41.396+0000 7fc9f85f4740 20 client.27202 trim_cache size 0 max 16384 2024-04-16T17:12:41.397 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-04-16T17:12:41.397 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-04-16T17:12:41.397 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-04-16T17:12:41.398 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-04-16T17:12:41.398 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:41.396+0000 7fc9efe00640 1 -- 172.21.15.45:0/3705714267 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fc9d40a3d10 msgr2=0x7fc9d40a61d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:41.398 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:41.396+0000 7fc9efe00640 1 --2- 172.21.15.45:0/3705714267 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fc9d40a3d10 0x7fc9d40a61d0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fc9e8005fd0 tx=0x7fc9e8009600 comp rx=0 tx=0).stop 2024-04-16T17:12:41.398 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:41.396+0000 7fc9efe00640 1 -- 172.21.15.45:0/3705714267 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc9f01042a0 msgr2=0x7fc9f01a06a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:41.398 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:41.396+0000 7fc9efe00640 1 --2- 172.21.15.45:0/3705714267 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc9f01042a0 0x7fc9f01a06a0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fc9e000d950 tx=0x7fc9e000de10 comp rx=0 tx=0).stop 2024-04-16T17:12:41.398 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:41.397+0000 7fc9efe00640 1 -- 172.21.15.45:0/3705714267 shutdown_connections 2024-04-16T17:12:41.399 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:41.397+0000 7fc9efe00640 1 --2- 172.21.15.45:0/3705714267 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fc9d40a3d10 0x7fc9d40a61d0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:41.399 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:41.397+0000 7fc9efe00640 1 --2- 172.21.15.45:0/3705714267 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc9f0109370 0x7fc9f01a0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:41.399 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:41.397+0000 7fc9efe00640 1 --2- 172.21.15.45:0/3705714267 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc9f019a760 0x7fc9f019abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:41.399 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:41.397+0000 7fc9efe00640 1 --2- 172.21.15.45:0/3705714267 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc9f01042a0 0x7fc9f01a06a0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:41.399 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:41.397+0000 7fc9efe00640 1 -- 172.21.15.45:0/3705714267 >> 172.21.15.45:0/3705714267 conn(0x7fc9f00ffae0 msgr2=0x7fc9f0100ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:12:41.399 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:41.397+0000 7fc9efe00640 1 -- 172.21.15.45:0/3705714267 shutdown_connections 2024-04-16T17:12:41.399 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:41.397+0000 7fc9efe00640 1 -- 172.21.15.45:0/3705714267 wait complete. 2024-04-16T17:12:42.227 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:12:40 smithi045 ceph-mon[31676]: pgmap v180: 129 pgs: 129 active+clean; 52 MiB data, 702 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 534 KiB/s wr, 95 op/s 2024-04-16T17:12:42.269 INFO:tasks.workunit.client.0.smithi045.stdout: 1 93961 1.67 MB/sec execute 164 sec latency 522.674 ms 2024-04-16T17:12:42.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:12:42 smithi104 ceph-mon[36457]: pgmap v181: 129 pgs: 129 active+clean; 54 MiB data, 710 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 759 KiB/s wr, 128 op/s 2024-04-16T17:12:43.054 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:12:42 smithi117 ceph-mon[36730]: pgmap v181: 129 pgs: 129 active+clean; 54 MiB data, 710 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 759 KiB/s wr, 128 op/s 2024-04-16T17:12:43.269 INFO:tasks.workunit.client.0.smithi045.stdout: 1 94284 1.66 MB/sec execute 165 sec latency 524.268 ms 2024-04-16T17:12:44.184 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:12:42 smithi045 ceph-mon[31676]: pgmap v181: 129 pgs: 129 active+clean; 54 MiB data, 710 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 759 KiB/s wr, 128 op/s 2024-04-16T17:12:44.269 INFO:tasks.workunit.client.0.smithi045.stdout: 1 94771 1.67 MB/sec execute 166 sec latency 50.293 ms 2024-04-16T17:12:44.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:12:44 smithi104 ceph-mon[36457]: pgmap v182: 129 pgs: 129 active+clean; 54 MiB data, 710 MiB used, 1.0 TiB / 1.0 TiB avail; 667 KiB/s rd, 455 KiB/s wr, 65 op/s 2024-04-16T17:12:45.054 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:12:44 smithi117 ceph-mon[36730]: pgmap v182: 129 pgs: 129 active+clean; 54 MiB data, 710 MiB used, 1.0 TiB / 1.0 TiB avail; 667 KiB/s rd, 455 KiB/s wr, 65 op/s 2024-04-16T17:12:45.269 INFO:tasks.workunit.client.0.smithi045.stdout: 1 96038 1.72 MB/sec execute 167 sec latency 23.841 ms 2024-04-16T17:12:45.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:12:44 smithi045 ceph-mon[31676]: pgmap v182: 129 pgs: 129 active+clean; 54 MiB data, 710 MiB used, 1.0 TiB / 1.0 TiB avail; 667 KiB/s rd, 455 KiB/s wr, 65 op/s 2024-04-16T17:12:46.270 INFO:tasks.workunit.client.0.smithi045.stdout: 1 96329 1.72 MB/sec execute 168 sec latency 33.243 ms 2024-04-16T17:12:46.835 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:12:46 smithi045 ceph-mon[31676]: pgmap v183: 129 pgs: 129 active+clean; 54 MiB data, 710 MiB used, 1.0 TiB / 1.0 TiB avail; 667 KiB/s rd, 455 KiB/s wr, 65 op/s 2024-04-16T17:12:46.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:12:46 smithi104 ceph-mon[36457]: pgmap v183: 129 pgs: 129 active+clean; 54 MiB data, 710 MiB used, 1.0 TiB / 1.0 TiB avail; 667 KiB/s rd, 455 KiB/s wr, 65 op/s 2024-04-16T17:12:47.054 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:12:46 smithi117 ceph-mon[36730]: pgmap v183: 129 pgs: 129 active+clean; 54 MiB data, 710 MiB used, 1.0 TiB / 1.0 TiB avail; 667 KiB/s rd, 455 KiB/s wr, 65 op/s 2024-04-16T17:12:47.269 INFO:tasks.workunit.client.0.smithi045.stdout: 1 96684 1.71 MB/sec execute 169 sec latency 72.293 ms 2024-04-16T17:12:48.270 INFO:tasks.workunit.client.0.smithi045.stdout: 1 97119 1.71 MB/sec execute 170 sec latency 56.526 ms 2024-04-16T17:12:48.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:12:48 smithi104 ceph-mon[36457]: pgmap v184: 129 pgs: 129 active+clean; 51 MiB data, 717 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.4 MiB/s wr, 147 op/s 2024-04-16T17:12:49.054 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:12:48 smithi117 ceph-mon[36730]: pgmap v184: 129 pgs: 129 active+clean; 51 MiB data, 717 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.4 MiB/s wr, 147 op/s 2024-04-16T17:12:49.270 INFO:tasks.workunit.client.0.smithi045.stdout: 1 97456 1.72 MB/sec execute 171 sec latency 74.990 ms 2024-04-16T17:12:49.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:12:48 smithi045 ceph-mon[31676]: pgmap v184: 129 pgs: 129 active+clean; 51 MiB data, 717 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.4 MiB/s wr, 147 op/s 2024-04-16T17:12:50.270 INFO:tasks.workunit.client.0.smithi045.stdout: 1 97508 1.71 MB/sec execute 172 sec latency 837.484 ms 2024-04-16T17:12:50.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:12:50 smithi104 ceph-mon[36457]: pgmap v185: 129 pgs: 129 active+clean; 51 MiB data, 717 MiB used, 1.0 TiB / 1.0 TiB avail; 677 KiB/s rd, 1.2 MiB/s wr, 116 op/s 2024-04-16T17:12:50.976 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:12:50 smithi045 ceph-mon[31676]: pgmap v185: 129 pgs: 129 active+clean; 51 MiB data, 717 MiB used, 1.0 TiB / 1.0 TiB avail; 677 KiB/s rd, 1.2 MiB/s wr, 116 op/s 2024-04-16T17:12:51.054 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:12:50 smithi117 ceph-mon[36730]: pgmap v185: 129 pgs: 129 active+clean; 51 MiB data, 717 MiB used, 1.0 TiB / 1.0 TiB avail; 677 KiB/s rd, 1.2 MiB/s wr, 116 op/s 2024-04-16T17:12:51.270 INFO:tasks.workunit.client.0.smithi045.stdout: 1 97508 1.70 MB/sec execute 173 sec latency 1837.573 ms 2024-04-16T17:12:52.270 INFO:tasks.workunit.client.0.smithi045.stdout: 1 97508 1.69 MB/sec execute 174 sec latency 2837.680 ms 2024-04-16T17:12:52.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:12:52 smithi104 ceph-mon[36457]: pgmap v186: 129 pgs: 129 active+clean; 52 MiB data, 738 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.4 MiB/s wr, 140 op/s 2024-04-16T17:12:53.054 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:12:52 smithi117 ceph-mon[36730]: pgmap v186: 129 pgs: 129 active+clean; 52 MiB data, 738 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.4 MiB/s wr, 140 op/s 2024-04-16T17:12:53.270 INFO:tasks.workunit.client.0.smithi045.stdout: 1 97772 1.68 MB/sec execute 175 sec latency 2842.539 ms 2024-04-16T17:12:54.089 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:12:52 smithi045 ceph-mon[31676]: pgmap v186: 129 pgs: 129 active+clean; 52 MiB data, 738 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.4 MiB/s wr, 140 op/s 2024-04-16T17:12:54.270 INFO:tasks.workunit.client.0.smithi045.stdout: 1 98155 1.68 MB/sec execute 176 sec latency 88.867 ms 2024-04-16T17:12:54.546 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:54.545+0000 7fcbede00640 1 -- 172.21.15.45:0/3483165974 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcbe8102fe0 msgr2=0x7fcbe81033c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:54.546 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:54.545+0000 7fcbede00640 1 --2- 172.21.15.45:0/3483165974 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcbe8102fe0 0x7fcbe81033c0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fcbdc009860 tx=0x7fcbdc02bb70 comp rx=0 tx=0).stop 2024-04-16T17:12:54.546 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:54.545+0000 7fcbede00640 1 -- 172.21.15.45:0/3483165974 shutdown_connections 2024-04-16T17:12:54.546 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:54.545+0000 7fcbede00640 1 --2- 172.21.15.45:0/3483165974 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcbe810c450 0x7fcbe810e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:54.546 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:54.545+0000 7fcbede00640 1 --2- 172.21.15.45:0/3483165974 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcbe8103900 0x7fcbe810bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:54.547 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:54.545+0000 7fcbede00640 1 --2- 172.21.15.45:0/3483165974 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcbe8102fe0 0x7fcbe81033c0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:54.547 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:54.545+0000 7fcbede00640 1 -- 172.21.15.45:0/3483165974 >> 172.21.15.45:0/3483165974 conn(0x7fcbe80fc820 msgr2=0x7fcbe80fec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:12:54.547 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:54.545+0000 7fcbede00640 1 -- 172.21.15.45:0/3483165974 shutdown_connections 2024-04-16T17:12:54.547 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:54.545+0000 7fcbede00640 1 -- 172.21.15.45:0/3483165974 wait complete. 2024-04-16T17:12:54.548 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:54.546+0000 7fcbede00640 1 Processor -- start 2024-04-16T17:12:54.548 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:54.546+0000 7fcbede00640 1 -- start start 2024-04-16T17:12:54.548 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:54.547+0000 7fcbede00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcbe8102fe0 0x7fcbe8100c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:54.548 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:54.547+0000 7fcbede00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcbe8103900 0x7fcbe8101170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:54.548 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:54.547+0000 7fcbede00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcbe810c450 0x7fcbe81016b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:54.548 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:54.547+0000 7fcbede00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fcbe8112200 con 0x7fcbe810c450 2024-04-16T17:12:54.548 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:54.547+0000 7fcbede00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fcbe8112080 con 0x7fcbe8102fe0 2024-04-16T17:12:54.548 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:54.547+0000 7fcbede00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fcbe8112380 con 0x7fcbe8103900 2024-04-16T17:12:54.548 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:54.547+0000 7fcbeca00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcbe8102fe0 0x7fcbe8100c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:54.548 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:54.547+0000 7fcbeca00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcbe8102fe0 0x7fcbe8100c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:42244/0 (socket says 172.21.15.45:42244) 2024-04-16T17:12:54.548 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:54.547+0000 7fcbeca00640 1 -- 172.21.15.45:0/4291110742 learned_addr learned my addr 172.21.15.45:0/4291110742 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:12:54.549 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:54.547+0000 7fcbed400640 1 --2- 172.21.15.45:0/4291110742 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcbe810c450 0x7fcbe81016b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:54.549 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:54.547+0000 7fcbe7e00640 1 --2- 172.21.15.45:0/4291110742 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcbe8103900 0x7fcbe8101170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:54.549 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:54.548+0000 7fcbeca00640 1 -- 172.21.15.45:0/4291110742 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcbe8103900 msgr2=0x7fcbe8101170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:54.549 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:54.548+0000 7fcbeca00640 1 --2- 172.21.15.45:0/4291110742 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcbe8103900 0x7fcbe8101170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:54.549 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:54.548+0000 7fcbeca00640 1 -- 172.21.15.45:0/4291110742 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcbe810c450 msgr2=0x7fcbe81016b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:54.549 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:54.548+0000 7fcbeca00640 1 --2- 172.21.15.45:0/4291110742 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcbe810c450 0x7fcbe81016b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:54.549 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:54.548+0000 7fcbeca00640 1 -- 172.21.15.45:0/4291110742 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbe81a6ba0 con 0x7fcbe8102fe0 2024-04-16T17:12:54.550 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:54.548+0000 7fcbeca00640 1 --2- 172.21.15.45:0/4291110742 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcbe8102fe0 0x7fcbe8100c30 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fcbdc037040 tx=0x7fcbdc0045b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:12:54.550 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:54.548+0000 7fcbe5600640 1 -- 172.21.15.45:0/4291110742 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbdc041020 con 0x7fcbe8102fe0 2024-04-16T17:12:54.550 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:54.548+0000 7fcbede00640 1 -- 172.21.15.45:0/4291110742 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcbe81a6e30 con 0x7fcbe8102fe0 2024-04-16T17:12:54.550 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:54.548+0000 7fcbe5600640 1 -- 172.21.15.45:0/4291110742 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcbdc02c9f0 con 0x7fcbe8102fe0 2024-04-16T17:12:54.550 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:54.548+0000 7fcbe5600640 1 -- 172.21.15.45:0/4291110742 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbdc039040 con 0x7fcbe8102fe0 2024-04-16T17:12:54.550 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:54.549+0000 7fcbede00640 1 -- 172.21.15.45:0/4291110742 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcbe81a72a0 con 0x7fcbe8102fe0 2024-04-16T17:12:54.553 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:54.553+0000 7fcbf2190740 1 Processor -- start 2024-04-16T17:12:54.554 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:54.553+0000 7fcbf2190740 1 -- start start 2024-04-16T17:12:54.554 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:54.553+0000 7fcbe5600640 1 -- 172.21.15.45:0/4291110742 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fcbdc044020 con 0x7fcbe8102fe0 2024-04-16T17:12:54.555 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:54.553+0000 7fcbf2190740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5600d4f857f0 0x5600d4f84540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:54.555 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:54.553+0000 7fcbf2190740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5600d4f84a80 0x5600d4f84f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:54.555 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:54.553+0000 7fcbf2190740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5600d4f8e000 0x5600d4f903f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:54.555 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:54.553+0000 7fcbf2190740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5600d4ee9780 con 0x5600d4f84a80 2024-04-16T17:12:54.555 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:54.553+0000 7fcbf2190740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5600d4e8ff40 con 0x5600d4f8e000 2024-04-16T17:12:54.555 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:54.553+0000 7fcbf2190740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5600d4ee5740 con 0x5600d4f857f0 2024-04-16T17:12:54.555 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:54.554+0000 7fcbe7e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5600d4f8e000 0x5600d4f903f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:54.555 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:54.554+0000 7fcbe7e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5600d4f8e000 0x5600d4f903f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:42250/0 (socket says 172.21.15.45:42250) 2024-04-16T17:12:54.555 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:54.554+0000 7fcbe7e00640 1 -- 172.21.15.45:0/1480467455 learned_addr learned my addr 172.21.15.45:0/1480467455 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:12:54.555 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:54.554+0000 7fcbed400640 1 --2- 172.21.15.45:0/1480467455 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5600d4f857f0 0x5600d4f84540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:54.555 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:54.554+0000 7fcbeca00640 1 --2- 172.21.15.45:0/1480467455 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5600d4f84a80 0x5600d4f84f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:54.556 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:54.554+0000 7fcbed400640 1 -- 172.21.15.45:0/1480467455 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5600d4f8e000 msgr2=0x5600d4f903f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:12:54.556 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:54.554+0000 7fcbed400640 1 --2- 172.21.15.45:0/1480467455 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5600d4f8e000 0x5600d4f903f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:54.556 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:54.554+0000 7fcbed400640 1 -- 172.21.15.45:0/1480467455 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5600d4f84a80 msgr2=0x5600d4f84f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:12:54.556 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:54.554+0000 7fcbed400640 1 --2- 172.21.15.45:0/1480467455 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5600d4f84a80 0x5600d4f84f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:54.556 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:54.554+0000 7fcbed400640 1 -- 172.21.15.45:0/1480467455 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5600d4bfc210 con 0x5600d4f857f0 2024-04-16T17:12:54.557 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:54.554+0000 7fcbeca00640 1 --2- 172.21.15.45:0/1480467455 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5600d4f84a80 0x5600d4f84f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:12:54.557 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:54.555+0000 7fcbed400640 1 --2- 172.21.15.45:0/1480467455 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5600d4f857f0 0x5600d4f84540 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fcbe0009f00 tx=0x7fcbe0011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d869fadeca060ff0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:12:54.557 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:54.555+0000 7fcbe5600640 1 --2- 172.21.15.45:0/4291110742 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fcbd00a3d40 0x7fcbd00a6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:54.557 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:54.555+0000 7fcbe5600640 1 -- 172.21.15.45:0/4291110742 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fcbdc0f2220 con 0x7fcbe8102fe0 2024-04-16T17:12:54.557 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:54.555+0000 7fcbd9c00640 1 -- 172.21.15.45:0/1480467455 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbe0004d20 con 0x5600d4f857f0 2024-04-16T17:12:54.557 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:54.555+0000 7fcbd9c00640 1 -- 172.21.15.45:0/1480467455 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcbe0004050 con 0x5600d4f857f0 2024-04-16T17:12:54.557 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:54.555+0000 7fcbd9c00640 1 -- 172.21.15.45:0/1480467455 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbe0005280 con 0x5600d4f857f0 2024-04-16T17:12:54.557 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:54.555+0000 7fcbeca00640 1 --2- 172.21.15.45:0/4291110742 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fcbd00a3d40 0x7fcbd00a6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:54.559 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:54.556+0000 7fcbf2190740 1 -- 172.21.15.45:0/1480467455 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5600d4f857f0 msgr2=0x5600d4f84540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:54.559 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:54.556+0000 7fcbeca00640 1 --2- 172.21.15.45:0/4291110742 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fcbd00a3d40 0x7fcbd00a6200 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x5600d4f85f50 tx=0x7fcbdc0327b0 comp rx=0 tx=0).ready entity=mgr.25411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:12:54.559 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:54.556+0000 7fcbf2190740 1 --2- 172.21.15.45:0/1480467455 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5600d4f857f0 0x5600d4f84540 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fcbe0009f00 tx=0x7fcbe0011810 comp rx=0 tx=0).stop 2024-04-16T17:12:54.559 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:54.556+0000 7fcbf2190740 1 -- 172.21.15.45:0/1480467455 shutdown_connections 2024-04-16T17:12:54.559 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:54.556+0000 7fcbf2190740 1 --2- 172.21.15.45:0/1480467455 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5600d4f8e000 0x5600d4f903f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:54.559 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:54.556+0000 7fcbf2190740 1 --2- 172.21.15.45:0/1480467455 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5600d4f84a80 0x5600d4f84f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:54.559 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:54.556+0000 7fcbf2190740 1 --2- 172.21.15.45:0/1480467455 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5600d4f857f0 0x5600d4f84540 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:54.559 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:54.556+0000 7fcbf2190740 1 -- 172.21.15.45:0/1480467455 >> 172.21.15.45:0/1480467455 conn(0x5600d4f1f5d0 msgr2=0x5600d4f7f770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:12:54.559 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:54.557+0000 7fcbf2190740 1 -- 172.21.15.45:0/1480467455 shutdown_connections 2024-04-16T17:12:54.559 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:54.557+0000 7fcbf2190740 1 -- 172.21.15.45:0/1480467455 wait complete. 2024-04-16T17:12:54.559 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:54.557+0000 7fcbf2190740 1 -- start start 2024-04-16T17:12:54.559 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:54.557+0000 7fcbf2190740 1 Processor -- start 2024-04-16T17:12:54.559 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:54.558+0000 7fcbf2190740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5600d4f84a80 0x5600d4f8ae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:54.560 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:54.558+0000 7fcbf2190740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5600d4f857f0 0x5600d4f8b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:54.560 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:54.558+0000 7fcbf2190740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5600d4f8e000 0x5600d4f8b900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:54.560 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:54.558+0000 7fcbf2190740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5600d4f99540 con 0x5600d4f8e000 2024-04-16T17:12:54.560 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:54.558+0000 7fcbf2190740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5600d4ea3390 con 0x5600d4f84a80 2024-04-16T17:12:54.560 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:54.558+0000 7fcbf2190740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5600d4f996c0 con 0x5600d4f857f0 2024-04-16T17:12:54.560 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:54.558+0000 7fcbf2190740 10 client.0 _ll_register_callbacks cb 0x5600d4e8cf60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:12:54.560 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:54.558+0000 7fcbed400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5600d4f84a80 0x5600d4f8ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:54.560 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:54.558+0000 7fcbe7e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5600d4f857f0 0x5600d4f8b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:54.560 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:54.558+0000 7fcbe7e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5600d4f857f0 0x5600d4f8b3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:36370/0 (socket says 172.21.15.45:36370) 2024-04-16T17:12:54.560 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:54.558+0000 7fcbe7e00640 1 -- 172.21.15.45:0/3867411730 learned_addr learned my addr 172.21.15.45:0/3867411730 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:12:54.560 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:54.558+0000 7fcbed400640 1 --2- 172.21.15.45:0/3867411730 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5600d4f8e000 0x5600d4f8b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:54.561 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:54.559+0000 7fcbe7e00640 1 -- 172.21.15.45:0/3867411730 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5600d4f84a80 msgr2=0x5600d4f8ae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:12:54.562 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:54.559+0000 7fcbe7e00640 1 --2- 172.21.15.45:0/3867411730 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5600d4f84a80 0x5600d4f8ae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:54.562 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:54.559+0000 7fcbe7e00640 1 -- 172.21.15.45:0/3867411730 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5600d4f8e000 msgr2=0x5600d4f8b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:12:54.562 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:54.559+0000 7fcbe7e00640 1 --2- 172.21.15.45:0/3867411730 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5600d4f8e000 0x5600d4f8b900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:54.562 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:54.559+0000 7fcbe7e00640 1 -- 172.21.15.45:0/3867411730 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5600d4cc8620 con 0x5600d4f857f0 2024-04-16T17:12:54.562 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:54.559+0000 7fcbed400640 1 --2- 172.21.15.45:0/3867411730 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5600d4f84a80 0x5600d4f8ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:12:54.562 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:54.559+0000 7fcbe7e00640 1 --2- 172.21.15.45:0/3867411730 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5600d4f857f0 0x5600d4f8b3c0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fcbd40116f0 tx=0x7fcbd4011bb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e0af65c7b3cc3ac6 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:12:54.562 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:54.559+0000 7fcbce000640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-04-16T17:12:54.562 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:54.559+0000 7fcbce000640 1 -- 172.21.15.45:0/3867411730 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbd400fb30 con 0x5600d4f857f0 2024-04-16T17:12:54.562 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:54.560+0000 7fcbf2190740 1 -- 172.21.15.45:0/3867411730 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcbe0004050 con 0x5600d4f857f0 2024-04-16T17:12:54.562 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:54.560+0000 7fcbce000640 1 -- 172.21.15.45:0/3867411730 <== mon.1 v2:172.21.15.117:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fcbd4017050 con 0x5600d4f857f0 2024-04-16T17:12:54.562 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:54.560+0000 7fcbce000640 1 -- 172.21.15.45:0/3867411730 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fcbd4027050 con 0x5600d4f857f0 2024-04-16T17:12:54.562 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:54.560+0000 7fcbf2190740 10 client.27214 fetch_fsmap learned FSMap version 37 2024-04-16T17:12:54.563 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:54.560+0000 7fcbf2190740 1 -- 172.21.15.45:0/3867411730 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x5600d4fa56d0 con 0x5600d4f857f0 2024-04-16T17:12:54.563 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:54.560+0000 7fcbce000640 1 -- 172.21.15.45:0/3867411730 <== mon.1 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcbe0004050 con 0x5600d4f857f0 2024-04-16T17:12:54.563 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:54.561+0000 7fcbce000640 1 -- 172.21.15.45:0/3867411730 <== mon.1 v2:172.21.15.117:3300/0 5 ==== fsmap(e 37) ==== 8358+0+0 (secure 0 0 0) 0x7fcbd4022030 con 0x5600d4f857f0 2024-04-16T17:12:54.563 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:54.561+0000 7fcbf2190740 10 client.27214 fetch_fsmap finished waiting for FSMap version 37 2024-04-16T17:12:54.563 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:54.561+0000 7fcbf2190740 10 client.27214 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T17:12:54.563 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:54.561+0000 7fcbf2190740 20 client.27214 populate_metadata read hostname 'smithi045' 2024-04-16T17:12:54.563 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:54.561+0000 7fcbf2190740 1 --2- 172.21.15.45:0/3867411730 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x5600d4fa6be0 0x5600d4fa9010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:54.563 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:54.561+0000 7fcbf2190740 4 client.27214 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:12:54.563 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:54.561+0000 7fcbf2190740 1 -- 172.21.15.45:0/3867411730 --> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5600d4ef71a0 con 0x5600d4fa6be0 2024-04-16T17:12:54.563 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:54.561+0000 7fcbed400640 1 --2- 172.21.15.45:0/3867411730 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x5600d4fa6be0 0x5600d4fa9010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:54.564 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:54.562+0000 7fcbed400640 1 --2- 172.21.15.45:0/3867411730 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x5600d4fa6be0 0x5600d4fa9010 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d32cefda09bd4c1e server_cookie=607da9b03229c818 in_seq=0 out_seq=0 2024-04-16T17:12:54.564 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:54.562+0000 7fcbce000640 10 client.27214 ms_handle_connect on v2:172.21.15.104:6838/3660070592 2024-04-16T17:12:54.565 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:54.563+0000 7fcbce000640 1 -- 172.21.15.45:0/3867411730 <== mds.0 v2:172.21.15.104:6838/3660070592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5600d4ef71a0 con 0x5600d4fa6be0 2024-04-16T17:12:54.565 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:54.563+0000 7fcbce000640 10 client.27214 handle_command_reply: tid=0 2024-04-16T17:12:54.565 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:54.563+0000 7fcbf2190740 1 client.27214 shutdown 2024-04-16T17:12:54.565 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:54.563+0000 7fcbf2190740 1 -- 172.21.15.45:0/3867411730 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5600d4f857f0 msgr2=0x5600d4f8b3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:54.565 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:54.563+0000 7fcbf2190740 1 --2- 172.21.15.45:0/3867411730 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5600d4f857f0 0x5600d4f8b3c0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fcbd40116f0 tx=0x7fcbd4011bb0 comp rx=0 tx=0).stop 2024-04-16T17:12:54.565 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:54.563+0000 7fcbf2190740 1 -- 172.21.15.45:0/3867411730 shutdown_connections 2024-04-16T17:12:54.565 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:54.563+0000 7fcbf2190740 1 --2- 172.21.15.45:0/3867411730 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x5600d4fa6be0 0x5600d4fa9010 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:54.565 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:54.563+0000 7fcbf2190740 1 --2- 172.21.15.45:0/3867411730 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5600d4f857f0 0x5600d4f8b3c0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:54.565 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:54.563+0000 7fcbf2190740 1 --2- 172.21.15.45:0/3867411730 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5600d4f8e000 0x5600d4f8b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:54.565 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:54.563+0000 7fcbf2190740 1 --2- 172.21.15.45:0/3867411730 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5600d4f84a80 0x5600d4f8ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:54.565 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:54.563+0000 7fcbce000640 0 client.27214 ms_handle_reset on v2:172.21.15.104:6838/3660070592 2024-04-16T17:12:54.565 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:54.563+0000 7fcbf2190740 1 -- 172.21.15.45:0/3867411730 >> 172.21.15.45:0/3867411730 conn(0x5600d4f1f5d0 msgr2=0x5600d4f814c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:12:54.565 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:54.564+0000 7fcbf2190740 1 -- 172.21.15.45:0/3867411730 shutdown_connections 2024-04-16T17:12:54.565 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:54.564+0000 7fcbf2190740 1 -- 172.21.15.45:0/3867411730 wait complete. 2024-04-16T17:12:54.565 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:54.564+0000 7fcbf2190740 20 client.27214 trim_cache size 0 max 16384 2024-04-16T17:12:54.581 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:54.580+0000 7fcbf2190740 1 Processor -- start 2024-04-16T17:12:54.581 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:54.580+0000 7fcbf2190740 1 -- start start 2024-04-16T17:12:54.581 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:54.580+0000 7fcbf2190740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5600d4f84a80 0x5600d4f8d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:54.581 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:54.580+0000 7fcbf2190740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5600d4f857f0 0x5600d4fa2440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:54.581 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:54.580+0000 7fcbf2190740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5600d4f8e000 0x5600d4f89fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:54.581 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:54.580+0000 7fcbf2190740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fcbe81af930 con 0x5600d4f857f0 2024-04-16T17:12:54.581 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:54.580+0000 7fcbf2190740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fcbe81ad170 con 0x5600d4f84a80 2024-04-16T17:12:54.581 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:54.580+0000 7fcbf2190740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fcbe81aafa0 con 0x5600d4f8e000 2024-04-16T17:12:54.582 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:54.581+0000 7fcbe7e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5600d4f857f0 0x5600d4fa2440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:54.582 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:54.581+0000 7fcbed400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5600d4f84a80 0x5600d4f8d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:54.582 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:54.581+0000 7fcbe7e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5600d4f857f0 0x5600d4fa2440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42530/0 (socket says 172.21.15.45:42530) 2024-04-16T17:12:54.583 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:54.581+0000 7fcbed400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5600d4f84a80 0x5600d4f8d5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:42262/0 (socket says 172.21.15.45:42262) 2024-04-16T17:12:54.583 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:54.581+0000 7fcbe7e00640 1 -- 172.21.15.45:0/39220773 learned_addr learned my addr 172.21.15.45:0/39220773 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:12:54.583 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:54.581+0000 7fcbed400640 1 --2- 172.21.15.45:0/39220773 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5600d4f8e000 0x5600d4f89fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:54.583 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:54.581+0000 7fcbed400640 1 -- 172.21.15.45:0/39220773 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5600d4f8e000 msgr2=0x5600d4f89fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:12:54.583 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:54.581+0000 7fcbed400640 1 --2- 172.21.15.45:0/39220773 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5600d4f8e000 0x5600d4f89fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:54.583 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:54.581+0000 7fcbed400640 1 -- 172.21.15.45:0/39220773 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5600d4f857f0 msgr2=0x5600d4fa2440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:12:54.583 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:54.581+0000 7fcbed400640 1 --2- 172.21.15.45:0/39220773 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5600d4f857f0 0x5600d4fa2440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:54.583 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:54.581+0000 7fcbed400640 1 -- 172.21.15.45:0/39220773 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5600d4f83d50 con 0x5600d4f84a80 2024-04-16T17:12:54.583 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:54.581+0000 7fcbe7e00640 1 --2- 172.21.15.45:0/39220773 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5600d4f857f0 0x5600d4fa2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:12:54.583 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:54.581+0000 7fcbed400640 1 --2- 172.21.15.45:0/39220773 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5600d4f84a80 0x5600d4f8d5b0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fcbe001b040 tx=0x7fcbe0009500 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f0d66fb7592afab8 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:12:54.583 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:54.582+0000 7fcbd9200640 1 -- 172.21.15.45:0/39220773 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbe00046c0 con 0x5600d4f84a80 2024-04-16T17:12:54.583 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:54.582+0000 7fcbd9200640 1 -- 172.21.15.45:0/39220773 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcbe0004860 con 0x5600d4f84a80 2024-04-16T17:12:54.583 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:54.582+0000 7fcbd9200640 1 -- 172.21.15.45:0/39220773 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbe0028de0 con 0x5600d4f84a80 2024-04-16T17:12:54.585 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:54.582+0000 7fcbf2190740 1 -- 172.21.15.45:0/39220773 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5600d4f84a80 msgr2=0x5600d4f8d5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:54.585 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:54.582+0000 7fcbf2190740 1 --2- 172.21.15.45:0/39220773 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5600d4f84a80 0x5600d4f8d5b0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fcbe001b040 tx=0x7fcbe0009500 comp rx=0 tx=0).stop 2024-04-16T17:12:54.585 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:54.582+0000 7fcbf2190740 1 -- 172.21.15.45:0/39220773 shutdown_connections 2024-04-16T17:12:54.585 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:54.583+0000 7fcbf2190740 1 --2- 172.21.15.45:0/39220773 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5600d4f8e000 0x5600d4f89fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:54.585 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:54.583+0000 7fcbf2190740 1 --2- 172.21.15.45:0/39220773 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5600d4f857f0 0x5600d4fa2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:54.585 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:54.583+0000 7fcbf2190740 1 --2- 172.21.15.45:0/39220773 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5600d4f84a80 0x5600d4f8d5b0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:54.586 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:54.583+0000 7fcbf2190740 1 -- 172.21.15.45:0/39220773 >> 172.21.15.45:0/39220773 conn(0x5600d4f1f5d0 msgr2=0x5600d4f7df80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:12:54.586 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:54.583+0000 7fcbf2190740 1 -- 172.21.15.45:0/39220773 shutdown_connections 2024-04-16T17:12:54.586 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:54.583+0000 7fcbf2190740 1 -- 172.21.15.45:0/39220773 wait complete. 2024-04-16T17:12:54.586 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:54.583+0000 7fcbf2190740 1 -- start start 2024-04-16T17:12:54.586 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:54.583+0000 7fcbf2190740 1 Processor -- start 2024-04-16T17:12:54.586 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:54.584+0000 7fcbf2190740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5600d4f84a80 0x5600d4f9ee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:54.586 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:54.584+0000 7fcbf2190740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5600d4f857f0 0x5600d4f87950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:54.586 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:54.584+0000 7fcbf2190740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5600d4f8e000 0x5600d4f87ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:54.586 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:54.584+0000 7fcbf2190740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fcbe81ab630 con 0x5600d4f857f0 2024-04-16T17:12:54.586 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:54.584+0000 7fcbf2190740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fcbe81acaa0 con 0x5600d4f8e000 2024-04-16T17:12:54.586 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:54.584+0000 7fcbf2190740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fcbe81abd00 con 0x5600d4f84a80 2024-04-16T17:12:54.586 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:54.584+0000 7fcbf2190740 10 client.0 _ll_register_callbacks cb 0x7fcbd4009870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:12:54.586 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:54.584+0000 7fcbe7e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5600d4f857f0 0x5600d4f87950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:54.586 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:54.584+0000 7fcbed400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5600d4f84a80 0x5600d4f9ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:54.586 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:54.584+0000 7fcbe7e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5600d4f857f0 0x5600d4f87950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42538/0 (socket says 172.21.15.45:42538) 2024-04-16T17:12:54.586 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:54.584+0000 7fcbe7e00640 1 -- 172.21.15.45:0/2284488681 learned_addr learned my addr 172.21.15.45:0/2284488681 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:12:54.587 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:54.584+0000 7fcbed400640 1 --2- 172.21.15.45:0/2284488681 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5600d4f8e000 0x5600d4f87ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:54.587 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:54.585+0000 7fcbe7e00640 1 -- 172.21.15.45:0/2284488681 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5600d4f84a80 msgr2=0x5600d4f9ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:12:54.587 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:54.585+0000 7fcbe7e00640 1 --2- 172.21.15.45:0/2284488681 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5600d4f84a80 0x5600d4f9ee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:54.587 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:54.585+0000 7fcbe7e00640 1 -- 172.21.15.45:0/2284488681 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5600d4f8e000 msgr2=0x5600d4f87ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:12:54.587 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:54.585+0000 7fcbe7e00640 1 --2- 172.21.15.45:0/2284488681 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5600d4f8e000 0x5600d4f87ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:54.587 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:54.585+0000 7fcbe7e00640 1 -- 172.21.15.45:0/2284488681 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5600d4f88700 con 0x5600d4f857f0 2024-04-16T17:12:54.587 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:54.585+0000 7fcbed400640 1 --2- 172.21.15.45:0/2284488681 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5600d4f8e000 0x5600d4f87ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:12:54.588 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:54.585+0000 7fcbe7e00640 1 --2- 172.21.15.45:0/2284488681 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5600d4f857f0 0x5600d4f87950 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fcbd4009530 tx=0x7fcbd4019890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99cffffa661f973a server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:12:54.588 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:54.585+0000 7fcbce000640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-16T17:12:54.588 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:54.585+0000 7fcbce000640 1 -- 172.21.15.45:0/2284488681 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbd4031070 con 0x5600d4f857f0 2024-04-16T17:12:54.589 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:54.585+0000 7fcbce000640 1 -- 172.21.15.45:0/2284488681 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fcbd4011e00 con 0x5600d4f857f0 2024-04-16T17:12:54.589 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:54.585+0000 7fcbf2190740 1 -- 172.21.15.45:0/2284488681 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcbe0004860 con 0x5600d4f857f0 2024-04-16T17:12:54.589 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:54.586+0000 7fcbce000640 1 -- 172.21.15.45:0/2284488681 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fcbd403c050 con 0x5600d4f857f0 2024-04-16T17:12:54.589 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:54.586+0000 7fcbf2190740 10 client.17106 fetch_fsmap learned FSMap version 37 2024-04-16T17:12:54.589 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:54.586+0000 7fcbf2190740 1 -- 172.21.15.45:0/2284488681 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x5600d500d240 con 0x5600d4f857f0 2024-04-16T17:12:54.589 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:54.586+0000 7fcbce000640 1 -- 172.21.15.45:0/2284488681 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcbe0004860 con 0x5600d4f857f0 2024-04-16T17:12:54.589 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:54.586+0000 7fcbce000640 1 -- 172.21.15.45:0/2284488681 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 37) ==== 8358+0+0 (secure 0 0 0) 0x7fcbd4022050 con 0x5600d4f857f0 2024-04-16T17:12:54.589 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:54.586+0000 7fcbf2190740 10 client.17106 fetch_fsmap finished waiting for FSMap version 37 2024-04-16T17:12:54.589 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:54.586+0000 7fcbf2190740 10 client.17106 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T17:12:54.589 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:54.586+0000 7fcbf2190740 20 client.17106 populate_metadata read hostname 'smithi045' 2024-04-16T17:12:54.589 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:54.586+0000 7fcbf2190740 1 --2- 172.21.15.45:0/2284488681 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x5600d500d930 0x5600d500fd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:12:54.589 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:54.586+0000 7fcbf2190740 4 client.17106 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-16T17:12:54.589 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:54.586+0000 7fcbf2190740 1 -- 172.21.15.45:0/2284488681 --> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5600d4ef1010 con 0x5600d500d930 2024-04-16T17:12:54.589 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:54.587+0000 7fcbed400640 1 --2- 172.21.15.45:0/2284488681 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x5600d500d930 0x5600d500fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:12:54.589 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:54.587+0000 7fcbce000640 10 client.17106 ms_handle_connect on v2:172.21.15.104:6838/3660070592 2024-04-16T17:12:54.589 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:54.587+0000 7fcbed400640 1 --2- 172.21.15.45:0/2284488681 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x5600d500d930 0x5600d500fd50 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c1d80bc6167d108f server_cookie=800afb85a1e453f9 in_seq=0 out_seq=0 2024-04-16T17:12:54.642 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:54.641+0000 7fcbce000640 1 -- 172.21.15.45:0/2284488681 <== mds.0 v2:172.21.15.104:6838/3660070592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5600d4ef1010 con 0x5600d500d930 2024-04-16T17:12:54.642 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:54.641+0000 7fcbce000640 10 client.17106 handle_command_reply: tid=0 2024-04-16T17:12:54.642 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:54.641+0000 7fcbf2190740 1 client.17106 shutdown 2024-04-16T17:12:54.642 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:54.641+0000 7fcbf2190740 1 -- 172.21.15.45:0/2284488681 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5600d4f857f0 msgr2=0x5600d4f87950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:54.642 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:54.641+0000 7fcbf2190740 1 --2- 172.21.15.45:0/2284488681 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5600d4f857f0 0x5600d4f87950 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fcbd4009530 tx=0x7fcbd4019890 comp rx=0 tx=0).stop 2024-04-16T17:12:54.642 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:54.641+0000 7fcbf2190740 1 -- 172.21.15.45:0/2284488681 shutdown_connections 2024-04-16T17:12:54.642 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:54.641+0000 7fcbf2190740 1 --2- 172.21.15.45:0/2284488681 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5600d4f8e000 0x5600d4f87ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:54.642 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:54.641+0000 7fcbf2190740 1 --2- 172.21.15.45:0/2284488681 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5600d4f857f0 0x5600d4f87950 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:54.642 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:54.641+0000 7fcbf2190740 1 --2- 172.21.15.45:0/2284488681 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x5600d500d930 0x5600d500fd50 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:54.642 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:54.641+0000 7fcbf2190740 1 --2- 172.21.15.45:0/2284488681 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5600d4f84a80 0x5600d4f9ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:54.642 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:54.641+0000 7fcbf2190740 1 -- 172.21.15.45:0/2284488681 >> 172.21.15.45:0/2284488681 conn(0x5600d4f1f5d0 msgr2=0x5600d4f7df80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:12:54.642 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:54.641+0000 7fcbce000640 0 client.17106 ms_handle_reset on v2:172.21.15.104:6838/3660070592 2024-04-16T17:12:54.643 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:54.642+0000 7fcbf2190740 1 -- 172.21.15.45:0/2284488681 shutdown_connections 2024-04-16T17:12:54.643 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:54.642+0000 7fcbf2190740 1 -- 172.21.15.45:0/2284488681 wait complete. 2024-04-16T17:12:54.643 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:54.642+0000 7fcbf2190740 20 client.17106 trim_cache size 0 max 16384 2024-04-16T17:12:54.644 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-04-16T17:12:54.644 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-04-16T17:12:54.644 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-04-16T17:12:54.644 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-04-16T17:12:54.644 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:54.643+0000 7fcbede00640 1 -- 172.21.15.45:0/4291110742 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fcbd00a3d40 msgr2=0x7fcbd00a6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:54.645 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:54.643+0000 7fcbede00640 1 --2- 172.21.15.45:0/4291110742 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fcbd00a3d40 0x7fcbd00a6200 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x5600d4f85f50 tx=0x7fcbdc0327b0 comp rx=0 tx=0).stop 2024-04-16T17:12:54.645 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:54.643+0000 7fcbede00640 1 -- 172.21.15.45:0/4291110742 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcbe8102fe0 msgr2=0x7fcbe8100c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:12:54.645 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:54.643+0000 7fcbede00640 1 --2- 172.21.15.45:0/4291110742 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcbe8102fe0 0x7fcbe8100c30 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fcbdc037040 tx=0x7fcbdc0045b0 comp rx=0 tx=0).stop 2024-04-16T17:12:54.645 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:54.643+0000 7fcbede00640 1 -- 172.21.15.45:0/4291110742 shutdown_connections 2024-04-16T17:12:54.645 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:54.643+0000 7fcbede00640 1 --2- 172.21.15.45:0/4291110742 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fcbd00a3d40 0x7fcbd00a6200 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:54.645 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:54.643+0000 7fcbede00640 1 --2- 172.21.15.45:0/4291110742 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcbe8103900 0x7fcbe8101170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:54.645 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:54.643+0000 7fcbede00640 1 --2- 172.21.15.45:0/4291110742 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcbe810c450 0x7fcbe81016b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:54.645 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:54.643+0000 7fcbede00640 1 --2- 172.21.15.45:0/4291110742 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcbe8102fe0 0x7fcbe8100c30 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:12:54.645 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:54.643+0000 7fcbede00640 1 -- 172.21.15.45:0/4291110742 >> 172.21.15.45:0/4291110742 conn(0x7fcbe80fc820 msgr2=0x7fcbe80fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:12:54.645 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:54.643+0000 7fcbede00640 1 -- 172.21.15.45:0/4291110742 shutdown_connections 2024-04-16T17:12:54.645 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:12:54.643+0000 7fcbede00640 1 -- 172.21.15.45:0/4291110742 wait complete. 2024-04-16T17:12:54.929 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:12:54 smithi104 ceph-mon[36457]: pgmap v187: 129 pgs: 129 active+clean; 52 MiB data, 738 MiB used, 1.0 TiB / 1.0 TiB avail; 871 KiB/s rd, 1.1 MiB/s wr, 106 op/s 2024-04-16T17:12:55.054 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:12:54 smithi117 ceph-mon[36730]: pgmap v187: 129 pgs: 129 active+clean; 52 MiB data, 738 MiB used, 1.0 TiB / 1.0 TiB avail; 871 KiB/s rd, 1.1 MiB/s wr, 106 op/s 2024-04-16T17:12:55.270 INFO:tasks.workunit.client.0.smithi045.stdout: 1 99361 1.73 MB/sec execute 177 sec latency 31.102 ms 2024-04-16T17:12:55.305 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:12:54 smithi045 ceph-mon[31676]: pgmap v187: 129 pgs: 129 active+clean; 52 MiB data, 738 MiB used, 1.0 TiB / 1.0 TiB avail; 871 KiB/s rd, 1.1 MiB/s wr, 106 op/s 2024-04-16T17:12:56.271 INFO:tasks.workunit.client.0.smithi045.stdout: 1 99748 1.73 MB/sec execute 178 sec latency 29.057 ms 2024-04-16T17:12:56.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:12:56 smithi104 ceph-mon[36457]: pgmap v188: 129 pgs: 129 active+clean; 52 MiB data, 738 MiB used, 1.0 TiB / 1.0 TiB avail; 871 KiB/s rd, 1.1 MiB/s wr, 106 op/s 2024-04-16T17:12:57.054 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:12:56 smithi117 ceph-mon[36730]: pgmap v188: 129 pgs: 129 active+clean; 52 MiB data, 738 MiB used, 1.0 TiB / 1.0 TiB avail; 871 KiB/s rd, 1.1 MiB/s wr, 106 op/s 2024-04-16T17:12:57.271 INFO:tasks.workunit.client.0.smithi045.stdout: 1 100120 1.72 MB/sec execute 179 sec latency 33.636 ms 2024-04-16T17:12:58.152 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:12:56 smithi045 ceph-mon[31676]: pgmap v188: 129 pgs: 129 active+clean; 52 MiB data, 738 MiB used, 1.0 TiB / 1.0 TiB avail; 871 KiB/s rd, 1.1 MiB/s wr, 106 op/s 2024-04-16T17:12:58.271 INFO:tasks.workunit.client.0.smithi045.stdout: 1 100746 1.74 MB/sec execute 180 sec latency 38.216 ms 2024-04-16T17:12:58.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:12:58 smithi104 ceph-mon[36457]: pgmap v189: 129 pgs: 129 active+clean; 53 MiB data, 741 MiB used, 1.0 TiB / 1.0 TiB avail; 1002 KiB/s rd, 1.3 MiB/s wr, 142 op/s 2024-04-16T17:12:59.002 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:12:58 smithi045 ceph-mon[31676]: pgmap v189: 129 pgs: 129 active+clean; 53 MiB data, 741 MiB used, 1.0 TiB / 1.0 TiB avail; 1002 KiB/s rd, 1.3 MiB/s wr, 142 op/s 2024-04-16T17:12:59.054 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:12:58 smithi117 ceph-mon[36730]: pgmap v189: 129 pgs: 129 active+clean; 53 MiB data, 741 MiB used, 1.0 TiB / 1.0 TiB avail; 1002 KiB/s rd, 1.3 MiB/s wr, 142 op/s 2024-04-16T17:12:59.274 INFO:tasks.workunit.client.0.smithi045.stdout: 1 100945 1.73 MB/sec execute 181 sec latency 53.610 ms 2024-04-16T17:13:00.274 INFO:tasks.workunit.client.0.smithi045.stdout: 1 101053 1.72 MB/sec execute 182 sec latency 577.605 ms 2024-04-16T17:13:00.634 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-16T17:13:00.634 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-16T17:13:00.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:00.846+0000 7f682e000640 1 -- 172.21.15.45:0/3770183188 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f68281071e0 msgr2=0x7f682810edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:00.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:00.846+0000 7f682e000640 1 --2- 172.21.15.45:0/3770183188 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f68281071e0 0x7f682810edb0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f6820009ca0 tx=0x7f682002c0f0 comp rx=0 tx=0).stop 2024-04-16T17:13:00.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:00.847+0000 7f682e000640 1 -- 172.21.15.45:0/3770183188 shutdown_connections 2024-04-16T17:13:00.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:00.847+0000 7f682e000640 1 --2- 172.21.15.45:0/3770183188 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6828102ac0 0x7f6828106b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:00.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:00.847+0000 7f682e000640 1 --2- 172.21.15.45:0/3770183188 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f68281071e0 0x7f682810edb0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:00.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:00.847+0000 7f682e000640 1 --2- 172.21.15.45:0/3770183188 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6828102110 0x7f68281024f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:00.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:00.847+0000 7f682e000640 1 -- 172.21.15.45:0/3770183188 >> 172.21.15.45:0/3770183188 conn(0x7f68280fd9d0 msgr2=0x7f68280ffdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:13:00.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:00.847+0000 7f682e000640 1 -- 172.21.15.45:0/3770183188 shutdown_connections 2024-04-16T17:13:00.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:00.847+0000 7f682e000640 1 -- 172.21.15.45:0/3770183188 wait complete. 2024-04-16T17:13:00.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:00.848+0000 7f682e000640 1 Processor -- start 2024-04-16T17:13:00.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:00.849+0000 7f682e000640 1 -- start start 2024-04-16T17:13:00.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:00.849+0000 7f682e000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6828102110 0x7f68281a0690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:00.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:00.849+0000 7f682e000640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f68281071e0 0x7f68281a0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:00.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:00.849+0000 7f682e000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f682819a800 0x7f682819acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:00.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:00.849+0000 7f682e000640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f68281122a0 con 0x7f682819a800 2024-04-16T17:13:00.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:00.849+0000 7f682e000640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f6828112120 con 0x7f68281071e0 2024-04-16T17:13:00.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:00.849+0000 7f682e000640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f6828112420 con 0x7f6828102110 2024-04-16T17:13:00.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:00.849+0000 7f682cc00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6828102110 0x7f68281a0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:00.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:00.849+0000 7f682cc00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6828102110 0x7f68281a0690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:53936/0 (socket says 172.21.15.45:53936) 2024-04-16T17:13:00.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:00.849+0000 7f682cc00640 1 -- 172.21.15.45:0/2081782678 learned_addr learned my addr 172.21.15.45:0/2081782678 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:13:00.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:00.849+0000 7f682d600640 1 --2- 172.21.15.45:0/2081782678 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f682819a800 0x7f682819acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:00.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:00.850+0000 7f6827e00640 1 --2- 172.21.15.45:0/2081782678 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f68281071e0 0x7f68281a0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:00.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:00.850+0000 7f682d600640 1 -- 172.21.15.45:0/2081782678 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6828102110 msgr2=0x7f68281a0690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:00.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:00.850+0000 7f682d600640 1 --2- 172.21.15.45:0/2081782678 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6828102110 0x7f68281a0690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:00.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:00.850+0000 7f682d600640 1 -- 172.21.15.45:0/2081782678 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f68281071e0 msgr2=0x7f68281a0bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:00.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:00.850+0000 7f682d600640 1 --2- 172.21.15.45:0/2081782678 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f68281071e0 0x7f68281a0bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:00.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:00.850+0000 7f682d600640 1 -- 172.21.15.45:0/2081782678 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f682819b390 con 0x7f682819a800 2024-04-16T17:13:00.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:00.850+0000 7f6827e00640 1 --2- 172.21.15.45:0/2081782678 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f68281071e0 0x7f68281a0bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:13:00.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:00.850+0000 7f682cc00640 1 --2- 172.21.15.45:0/2081782678 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6828102110 0x7f68281a0690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:13:00.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:00.850+0000 7f682d600640 1 --2- 172.21.15.45:0/2081782678 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f682819a800 0x7f682819acb0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f6820009bf0 tx=0x7f6820032ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:13:00.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:00.851+0000 7f6825600640 1 -- 172.21.15.45:0/2081782678 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6820041020 con 0x7f682819a800 2024-04-16T17:13:00.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:00.851+0000 7f682e000640 1 -- 172.21.15.45:0/2081782678 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f682819b620 con 0x7f682819a800 2024-04-16T17:13:00.852 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:00.851+0000 7f6825600640 1 -- 172.21.15.45:0/2081782678 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6820034cd0 con 0x7f682819a800 2024-04-16T17:13:00.852 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:00.851+0000 7f6825600640 1 -- 172.21.15.45:0/2081782678 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6820039040 con 0x7f682819a800 2024-04-16T17:13:00.852 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:00.851+0000 7f682e000640 1 -- 172.21.15.45:0/2081782678 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68281a77d0 con 0x7f682819a800 2024-04-16T17:13:00.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:00.855+0000 7f68323b4740 1 Processor -- start 2024-04-16T17:13:00.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:00.855+0000 7f6825600640 1 -- 172.21.15.45:0/2081782678 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f6820044020 con 0x7f682819a800 2024-04-16T17:13:00.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:00.855+0000 7f68323b4740 1 -- start start 2024-04-16T17:13:00.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:00.856+0000 7f68323b4740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dc6d2f3390 0x55dc6d2f3770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:00.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:00.856+0000 7f68323b4740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55dc6d2f3cb0 0x55dc6d2fc350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:00.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:00.856+0000 7f68323b4740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dc6d2fc890 0x55dc6d2fec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:00.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:00.856+0000 7f68323b4740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55dc6d259120 con 0x55dc6d2f3390 2024-04-16T17:13:00.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:00.856+0000 7f68323b4740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55dc6d1ff340 con 0x55dc6d2f3cb0 2024-04-16T17:13:00.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:00.856+0000 7f68323b4740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55dc6d2550e0 con 0x55dc6d2fc890 2024-04-16T17:13:00.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:00.856+0000 7f6827e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dc6d2f3390 0x55dc6d2f3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:00.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:00.856+0000 7f6827e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dc6d2f3390 0x55dc6d2f3770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:46620/0 (socket says 172.21.15.45:46620) 2024-04-16T17:13:00.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:00.856+0000 7f6827e00640 1 -- 172.21.15.45:0/1058232025 learned_addr learned my addr 172.21.15.45:0/1058232025 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:13:00.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:00.856+0000 7f682cc00640 1 --2- 172.21.15.45:0/1058232025 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55dc6d2f3cb0 0x55dc6d2fc350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:00.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:00.856+0000 7f6827e00640 1 --2- 172.21.15.45:0/1058232025 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dc6d2fc890 0x55dc6d2fec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:00.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:00.857+0000 7f682cc00640 1 -- 172.21.15.45:0/1058232025 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dc6d2fc890 msgr2=0x55dc6d2fec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:13:00.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:00.857+0000 7f682cc00640 1 --2- 172.21.15.45:0/1058232025 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dc6d2fc890 0x55dc6d2fec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:00.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:00.857+0000 7f682cc00640 1 -- 172.21.15.45:0/1058232025 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dc6d2f3390 msgr2=0x55dc6d2f3770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:13:00.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:00.857+0000 7f682cc00640 1 --2- 172.21.15.45:0/1058232025 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dc6d2f3390 0x55dc6d2f3770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:00.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:00.857+0000 7f682cc00640 1 -- 172.21.15.45:0/1058232025 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dc6d0357d0 con 0x55dc6d2f3cb0 2024-04-16T17:13:00.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:00.857+0000 7f6827e00640 1 --2- 172.21.15.45:0/1058232025 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dc6d2fc890 0x55dc6d2fec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:13:00.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:00.857+0000 7f682cc00640 1 --2- 172.21.15.45:0/1058232025 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55dc6d2f3cb0 0x55dc6d2fc350 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f681c0114f0 tx=0x7f681c0119b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=32d016b5abc62862 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:13:00.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:00.857+0000 7f6825600640 1 --2- 172.21.15.45:0/2081782678 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f68100a3da0 0x7f68100a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:00.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:00.857+0000 7f6819c00640 1 -- 172.21.15.45:0/1058232025 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f681c01e070 con 0x55dc6d2f3cb0 2024-04-16T17:13:00.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:00.857+0000 7f6825600640 1 -- 172.21.15.45:0/2081782678 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f68200f26b0 con 0x7f682819a800 2024-04-16T17:13:00.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:00.857+0000 7f6819c00640 1 -- 172.21.15.45:0/1058232025 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f681c0035f0 con 0x55dc6d2f3cb0 2024-04-16T17:13:00.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:00.857+0000 7f6819c00640 1 -- 172.21.15.45:0/1058232025 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f681c00f820 con 0x55dc6d2f3cb0 2024-04-16T17:13:00.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:00.858+0000 7f6827e00640 1 --2- 172.21.15.45:0/2081782678 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f68100a3da0 0x7f68100a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:00.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:00.858+0000 7f6827e00640 1 --2- 172.21.15.45:0/2081782678 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f68100a3da0 0x7f68100a6260 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x55dc6d2eec50 tx=0x7f6814015040 comp rx=0 tx=0).ready entity=mgr.25411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:13:00.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:00.858+0000 7f68323b4740 1 -- 172.21.15.45:0/1058232025 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55dc6d2f3cb0 msgr2=0x55dc6d2fc350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:00.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:00.858+0000 7f68323b4740 1 --2- 172.21.15.45:0/1058232025 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55dc6d2f3cb0 0x55dc6d2fc350 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f681c0114f0 tx=0x7f681c0119b0 comp rx=0 tx=0).stop 2024-04-16T17:13:00.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:00.859+0000 7f68323b4740 1 -- 172.21.15.45:0/1058232025 shutdown_connections 2024-04-16T17:13:00.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:00.859+0000 7f68323b4740 1 --2- 172.21.15.45:0/1058232025 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dc6d2fc890 0x55dc6d2fec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:00.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:00.859+0000 7f68323b4740 1 --2- 172.21.15.45:0/1058232025 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55dc6d2f3cb0 0x55dc6d2fc350 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:00.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:00.859+0000 7f68323b4740 1 --2- 172.21.15.45:0/1058232025 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dc6d2f3390 0x55dc6d2f3770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:00.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:00.859+0000 7f68323b4740 1 -- 172.21.15.45:0/1058232025 >> 172.21.15.45:0/1058232025 conn(0x55dc6d2edac0 msgr2=0x55dc6d2edea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:13:00.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:00.859+0000 7f68323b4740 1 -- 172.21.15.45:0/1058232025 shutdown_connections 2024-04-16T17:13:00.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:00.859+0000 7f68323b4740 1 -- 172.21.15.45:0/1058232025 wait complete. 2024-04-16T17:13:00.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:00.859+0000 7f68323b4740 1 -- start start 2024-04-16T17:13:00.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:00.860+0000 7f68323b4740 1 Processor -- start 2024-04-16T17:13:00.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:00.860+0000 7f68323b4740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55dc6d2f3390 0x55dc6d2f98b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:00.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:00.860+0000 7f68323b4740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dc6d2f3cb0 0x55dc6d2f9df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:00.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:00.860+0000 7f68323b4740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dc6d2fc890 0x55dc6d2fa330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:00.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:00.860+0000 7f68323b4740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55dc6d2ff680 con 0x55dc6d2fc890 2024-04-16T17:13:00.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:00.860+0000 7f68323b4740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55dc6d212710 con 0x55dc6d2f3390 2024-04-16T17:13:00.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:00.860+0000 7f68323b4740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55dc6d2ff800 con 0x55dc6d2f3cb0 2024-04-16T17:13:00.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:00.860+0000 7f68323b4740 10 client.0 _ll_register_callbacks cb 0x55dc6d20daf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:13:00.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:00.861+0000 7f6827e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dc6d2f3cb0 0x55dc6d2f9df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:00.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:00.861+0000 7f682d600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dc6d2fc890 0x55dc6d2fa330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:00.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:00.861+0000 7f682cc00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55dc6d2f3390 0x55dc6d2f98b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:00.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:00.861+0000 7f682cc00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55dc6d2f3390 0x55dc6d2f98b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:35766/0 (socket says 172.21.15.45:35766) 2024-04-16T17:13:00.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:00.861+0000 7f682d600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dc6d2fc890 0x55dc6d2fa330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:46626/0 (socket says 172.21.15.45:46626) 2024-04-16T17:13:00.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:00.861+0000 7f682cc00640 1 -- 172.21.15.45:0/1350236677 learned_addr learned my addr 172.21.15.45:0/1350236677 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:13:00.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:00.861+0000 7f6827e00640 1 --2- 172.21.15.45:0/1350236677 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dc6d2f3cb0 0x55dc6d2f9df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:53958/0 (socket says 172.21.15.45:53958) 2024-04-16T17:13:00.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:00.861+0000 7f682cc00640 1 -- 172.21.15.45:0/1350236677 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dc6d2f3cb0 msgr2=0x55dc6d2f9df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:13:00.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:00.861+0000 7f682cc00640 1 --2- 172.21.15.45:0/1350236677 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dc6d2f3cb0 0x55dc6d2f9df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:00.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:00.861+0000 7f682cc00640 1 -- 172.21.15.45:0/1350236677 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dc6d2fc890 msgr2=0x55dc6d2fa330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:13:00.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:00.861+0000 7f682cc00640 1 --2- 172.21.15.45:0/1350236677 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dc6d2fc890 0x55dc6d2fa330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:00.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:00.861+0000 7f682cc00640 1 -- 172.21.15.45:0/1350236677 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dc6cf6b3f0 con 0x55dc6d2f3390 2024-04-16T17:13:00.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:00.861+0000 7f682d600640 1 --2- 172.21.15.45:0/1350236677 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dc6d2fc890 0x55dc6d2fa330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:13:00.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:00.862+0000 7f682cc00640 1 --2- 172.21.15.45:0/1350236677 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55dc6d2f3390 0x55dc6d2f98b0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f681c009620 tx=0x7f681c0039d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3e022f140ab9dc55 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:13:00.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:00.862+0000 7f680e000640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-04-16T17:13:00.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:00.862+0000 7f680e000640 1 -- 172.21.15.45:0/1350236677 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f681c01e070 con 0x55dc6d2f3390 2024-04-16T17:13:00.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:00.862+0000 7f68323b4740 1 -- 172.21.15.45:0/1350236677 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f681c0035f0 con 0x55dc6d2f3390 2024-04-16T17:13:00.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:00.862+0000 7f680e000640 1 -- 172.21.15.45:0/1350236677 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f681c005240 con 0x55dc6d2f3390 2024-04-16T17:13:00.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:00.862+0000 7f680e000640 1 -- 172.21.15.45:0/1350236677 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f681c030050 con 0x55dc6d2f3390 2024-04-16T17:13:00.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:00.862+0000 7f68323b4740 10 client.27290 fetch_fsmap learned FSMap version 37 2024-04-16T17:13:00.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:00.862+0000 7f68323b4740 1 -- 172.21.15.45:0/1350236677 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55dc6cf2b0f0 con 0x55dc6d2f3390 2024-04-16T17:13:00.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:00.863+0000 7f680e000640 1 -- 172.21.15.45:0/1350236677 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f681c0035f0 con 0x55dc6d2f3390 2024-04-16T17:13:00.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:00.863+0000 7f680e000640 1 -- 172.21.15.45:0/1350236677 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 37) ==== 8358+0+0 (secure 0 0 0) 0x7f681c02b030 con 0x55dc6d2f3390 2024-04-16T17:13:00.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:00.863+0000 7f68323b4740 10 client.27290 fetch_fsmap finished waiting for FSMap version 37 2024-04-16T17:13:00.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:00.863+0000 7f68323b4740 10 client.27290 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:13:00.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:00.863+0000 7f68323b4740 20 client.27290 populate_metadata read hostname 'smithi045' 2024-04-16T17:13:00.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:00.863+0000 7f68323b4740 1 --2- 172.21.15.45:0/1350236677 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55dc6d30cbd0 0x55dc6d30efc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:00.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:00.863+0000 7f68323b4740 4 client.27290 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:13:00.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:00.863+0000 7f68323b4740 1 -- 172.21.15.45:0/1350236677 --> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dc6d266b40 con 0x55dc6d30cbd0 2024-04-16T17:13:00.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:00.863+0000 7f6827e00640 1 --2- 172.21.15.45:0/1350236677 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55dc6d30cbd0 0x55dc6d30efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:00.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:00.864+0000 7f6827e00640 1 --2- 172.21.15.45:0/1350236677 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55dc6d30cbd0 0x55dc6d30efc0 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2a5c5b290159039a server_cookie=f9fe3c5c5c649ace in_seq=0 out_seq=0 2024-04-16T17:13:00.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:00.864+0000 7f680e000640 10 client.27290 ms_handle_connect on v2:172.21.15.104:6838/3660070592 2024-04-16T17:13:00.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:00.864+0000 7f680e000640 1 -- 172.21.15.45:0/1350236677 <== mds.0 v2:172.21.15.104:6838/3660070592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55dc6d266b40 con 0x55dc6d30cbd0 2024-04-16T17:13:00.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:00.864+0000 7f680e000640 10 client.27290 handle_command_reply: tid=0 2024-04-16T17:13:00.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:00.865+0000 7f68323b4740 1 client.27290 shutdown 2024-04-16T17:13:00.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:00.865+0000 7f68323b4740 1 -- 172.21.15.45:0/1350236677 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55dc6d2f3390 msgr2=0x55dc6d2f98b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:00.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:00.865+0000 7f68323b4740 1 --2- 172.21.15.45:0/1350236677 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55dc6d2f3390 0x55dc6d2f98b0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f681c009620 tx=0x7f681c0039d0 comp rx=0 tx=0).stop 2024-04-16T17:13:00.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:00.865+0000 7f68323b4740 1 -- 172.21.15.45:0/1350236677 shutdown_connections 2024-04-16T17:13:00.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:00.865+0000 7f68323b4740 1 --2- 172.21.15.45:0/1350236677 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55dc6d30cbd0 0x55dc6d30efc0 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:00.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:00.865+0000 7f68323b4740 1 --2- 172.21.15.45:0/1350236677 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dc6d2f3cb0 0x55dc6d2f9df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:00.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:00.865+0000 7f68323b4740 1 --2- 172.21.15.45:0/1350236677 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dc6d2fc890 0x55dc6d2fa330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:00.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:00.865+0000 7f68323b4740 1 --2- 172.21.15.45:0/1350236677 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55dc6d2f3390 0x55dc6d2f98b0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:00.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:00.865+0000 7f68323b4740 1 -- 172.21.15.45:0/1350236677 >> 172.21.15.45:0/1350236677 conn(0x55dc6d2edac0 msgr2=0x55dc6d2ef810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:13:00.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:00.865+0000 7f680e000640 0 client.27290 ms_handle_reset on v2:172.21.15.104:6838/3660070592 2024-04-16T17:13:00.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:00.865+0000 7f68323b4740 1 -- 172.21.15.45:0/1350236677 shutdown_connections 2024-04-16T17:13:00.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:00.865+0000 7f68323b4740 1 -- 172.21.15.45:0/1350236677 wait complete. 2024-04-16T17:13:00.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:00.865+0000 7f68323b4740 20 client.27290 trim_cache size 0 max 16384 2024-04-16T17:13:00.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:00.884+0000 7f68323b4740 1 Processor -- start 2024-04-16T17:13:00.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:00.884+0000 7f68323b4740 1 -- start start 2024-04-16T17:13:00.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:00.884+0000 7f68323b4740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55dc6d2f3390 0x55dc6d2fbfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:00.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:00.884+0000 7f68323b4740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dc6d2f3cb0 0x55dc6d308540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:00.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:00.884+0000 7f68323b4740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dc6d2fc890 0x55dc6d3146e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:00.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:00.884+0000 7f68323b4740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f68281b3d40 con 0x55dc6d2fc890 2024-04-16T17:13:00.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:00.884+0000 7f68323b4740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f68281b1580 con 0x55dc6d2f3390 2024-04-16T17:13:00.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:00.884+0000 7f68323b4740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f68281af3b0 con 0x55dc6d2f3cb0 2024-04-16T17:13:00.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:00.885+0000 7f682d600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dc6d2fc890 0x55dc6d3146e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:00.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:00.885+0000 7f682d600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dc6d2fc890 0x55dc6d3146e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:46634/0 (socket says 172.21.15.45:46634) 2024-04-16T17:13:00.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:00.885+0000 7f682d600640 1 -- 172.21.15.45:0/2308327513 learned_addr learned my addr 172.21.15.45:0/2308327513 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:13:00.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:00.885+0000 7f6827e00640 1 --2- 172.21.15.45:0/2308327513 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dc6d2f3cb0 0x55dc6d308540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:00.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:00.885+0000 7f682cc00640 1 --2- 172.21.15.45:0/2308327513 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55dc6d2f3390 0x55dc6d2fbfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:00.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:00.885+0000 7f682d600640 1 -- 172.21.15.45:0/2308327513 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dc6d2f3cb0 msgr2=0x55dc6d308540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:13:00.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:00.885+0000 7f682d600640 1 --2- 172.21.15.45:0/2308327513 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dc6d2f3cb0 0x55dc6d308540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:00.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:00.885+0000 7f682d600640 1 -- 172.21.15.45:0/2308327513 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55dc6d2f3390 msgr2=0x55dc6d2fbfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:13:00.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:00.885+0000 7f682d600640 1 --2- 172.21.15.45:0/2308327513 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55dc6d2f3390 0x55dc6d2fbfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:00.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:00.885+0000 7f682d600640 1 -- 172.21.15.45:0/2308327513 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dc6d2feff0 con 0x55dc6d2fc890 2024-04-16T17:13:00.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:00.885+0000 7f682d600640 1 --2- 172.21.15.45:0/2308327513 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dc6d2fc890 0x55dc6d3146e0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f6820009ca0 tx=0x7f68200ba050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23a1030f311c372e server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:13:00.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:00.886+0000 7f6819200640 1 -- 172.21.15.45:0/2308327513 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68200bdd10 con 0x55dc6d2fc890 2024-04-16T17:13:00.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:00.886+0000 7f6819200640 1 -- 172.21.15.45:0/2308327513 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f68200d2e50 con 0x55dc6d2fc890 2024-04-16T17:13:00.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:00.886+0000 7f6819200640 1 -- 172.21.15.45:0/2308327513 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68200bc6b0 con 0x55dc6d2fc890 2024-04-16T17:13:00.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:00.886+0000 7f68323b4740 1 -- 172.21.15.45:0/2308327513 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dc6d2fc890 msgr2=0x55dc6d3146e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:00.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:00.886+0000 7f68323b4740 1 --2- 172.21.15.45:0/2308327513 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dc6d2fc890 0x55dc6d3146e0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f6820009ca0 tx=0x7f68200ba050 comp rx=0 tx=0).stop 2024-04-16T17:13:00.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:00.886+0000 7f68323b4740 1 -- 172.21.15.45:0/2308327513 shutdown_connections 2024-04-16T17:13:00.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:00.886+0000 7f68323b4740 1 --2- 172.21.15.45:0/2308327513 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dc6d2f3cb0 0x55dc6d308540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:00.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:00.886+0000 7f68323b4740 1 --2- 172.21.15.45:0/2308327513 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dc6d2fc890 0x55dc6d3146e0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:00.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:00.886+0000 7f68323b4740 1 --2- 172.21.15.45:0/2308327513 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55dc6d2f3390 0x55dc6d2fbfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:00.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:00.886+0000 7f68323b4740 1 -- 172.21.15.45:0/2308327513 >> 172.21.15.45:0/2308327513 conn(0x55dc6d2edac0 msgr2=0x55dc6d2f4cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:13:00.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:00.886+0000 7f68323b4740 1 -- 172.21.15.45:0/2308327513 shutdown_connections 2024-04-16T17:13:00.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:00.887+0000 7f68323b4740 1 -- 172.21.15.45:0/2308327513 wait complete. 2024-04-16T17:13:00.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:00.887+0000 7f68323b4740 1 -- start start 2024-04-16T17:13:00.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:00.887+0000 7f68323b4740 1 Processor -- start 2024-04-16T17:13:00.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:00.888+0000 7f68323b4740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dc6d2f3390 0x55dc6d304fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:00.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:00.888+0000 7f68323b4740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55dc6d2f3cb0 0x55dc6d2f6ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:00.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:00.888+0000 7f68323b4740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dc6d2fc890 0x55dc6d2f73d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:00.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:00.888+0000 7f68323b4740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f68281afa40 con 0x55dc6d2fc890 2024-04-16T17:13:00.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:00.888+0000 7f68323b4740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f68281b0eb0 con 0x55dc6d2f3cb0 2024-04-16T17:13:00.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:00.888+0000 7f68323b4740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f68281b0110 con 0x55dc6d2f3390 2024-04-16T17:13:00.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:00.888+0000 7f68323b4740 10 client.0 _ll_register_callbacks cb 0x7f681c01c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:13:00.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:00.888+0000 7f682d600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dc6d2fc890 0x55dc6d2f73d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:00.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:00.888+0000 7f6827e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55dc6d2f3cb0 0x55dc6d2f6ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:00.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:00.888+0000 7f6827e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55dc6d2f3cb0 0x55dc6d2f6ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:35778/0 (socket says 172.21.15.45:35778) 2024-04-16T17:13:00.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:00.888+0000 7f6827e00640 1 -- 172.21.15.45:0/3465072764 learned_addr learned my addr 172.21.15.45:0/3465072764 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:13:00.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:00.888+0000 7f682cc00640 1 --2- 172.21.15.45:0/3465072764 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dc6d2f3390 0x55dc6d304fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:00.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:00.888+0000 7f6827e00640 1 -- 172.21.15.45:0/3465072764 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dc6d2f3390 msgr2=0x55dc6d304fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:13:00.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:00.888+0000 7f6827e00640 1 --2- 172.21.15.45:0/3465072764 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dc6d2f3390 0x55dc6d304fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:00.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:00.888+0000 7f6827e00640 1 -- 172.21.15.45:0/3465072764 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dc6d2fc890 msgr2=0x55dc6d2f73d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:13:00.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:00.888+0000 7f6827e00640 1 --2- 172.21.15.45:0/3465072764 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dc6d2fc890 0x55dc6d2f73d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:00.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:00.888+0000 7f6827e00640 1 -- 172.21.15.45:0/3465072764 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dc6d2f5b80 con 0x55dc6d2f3cb0 2024-04-16T17:13:00.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:00.888+0000 7f682d600640 1 --2- 172.21.15.45:0/3465072764 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dc6d2fc890 0x55dc6d2f73d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:13:00.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:00.889+0000 7f6827e00640 1 --2- 172.21.15.45:0/3465072764 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55dc6d2f3cb0 0x55dc6d2f6ce0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f68140116d0 tx=0x7f6814005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e6017b3d3e8466e server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:13:00.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:00.889+0000 7f680e000640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-04-16T17:13:00.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:00.889+0000 7f680e000640 1 -- 172.21.15.45:0/3465072764 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f681402c690 con 0x55dc6d2f3cb0 2024-04-16T17:13:00.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:00.889+0000 7f68323b4740 1 -- 172.21.15.45:0/3465072764 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f68200d2e50 con 0x55dc6d2f3cb0 2024-04-16T17:13:00.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:00.889+0000 7f680e000640 1 -- 172.21.15.45:0/3465072764 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f681402c830 con 0x55dc6d2f3cb0 2024-04-16T17:13:00.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:00.889+0000 7f680e000640 1 -- 172.21.15.45:0/3465072764 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f681403e050 con 0x55dc6d2f3cb0 2024-04-16T17:13:00.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:00.889+0000 7f68323b4740 10 client.27296 fetch_fsmap learned FSMap version 37 2024-04-16T17:13:00.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:00.889+0000 7f68323b4740 1 -- 172.21.15.45:0/3465072764 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55dc6d3722a0 con 0x55dc6d2f3cb0 2024-04-16T17:13:00.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:00.890+0000 7f680e000640 1 -- 172.21.15.45:0/3465072764 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f68200d2e50 con 0x55dc6d2f3cb0 2024-04-16T17:13:00.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:00.890+0000 7f680e000640 1 -- 172.21.15.45:0/3465072764 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 37) ==== 8358+0+0 (secure 0 0 0) 0x7f68140215d0 con 0x55dc6d2f3cb0 2024-04-16T17:13:00.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:00.890+0000 7f68323b4740 10 client.27296 fetch_fsmap finished waiting for FSMap version 37 2024-04-16T17:13:00.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:00.890+0000 7f68323b4740 10 client.27296 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:13:00.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:00.890+0000 7f68323b4740 20 client.27296 populate_metadata read hostname 'smithi045' 2024-04-16T17:13:00.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:00.890+0000 7f68323b4740 1 --2- 172.21.15.45:0/3465072764 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55dc6d372a50 0x55dc6d374e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:00.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:00.890+0000 7f68323b4740 4 client.27296 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-16T17:13:00.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:00.890+0000 7f68323b4740 1 -- 172.21.15.45:0/3465072764 --> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55dc6d2609b0 con 0x55dc6d372a50 2024-04-16T17:13:00.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:00.890+0000 7f682cc00640 1 --2- 172.21.15.45:0/3465072764 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55dc6d372a50 0x55dc6d374e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:00.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:00.891+0000 7f682cc00640 1 --2- 172.21.15.45:0/3465072764 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55dc6d372a50 0x55dc6d374e70 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b5bf0bc84c43c913 server_cookie=e0066f658ce0dbc5 in_seq=0 out_seq=0 2024-04-16T17:13:00.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:00.891+0000 7f680e000640 10 client.27296 ms_handle_connect on v2:172.21.15.104:6838/3660070592 2024-04-16T17:13:00.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:00.891+0000 7f680e000640 1 -- 172.21.15.45:0/3465072764 <== mds.0 v2:172.21.15.104:6838/3660070592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55dc6d2609b0 con 0x55dc6d372a50 2024-04-16T17:13:00.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:00.891+0000 7f680e000640 10 client.27296 handle_command_reply: tid=0 2024-04-16T17:13:00.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:00.891+0000 7f68323b4740 1 client.27296 shutdown 2024-04-16T17:13:00.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:00.891+0000 7f68323b4740 1 -- 172.21.15.45:0/3465072764 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55dc6d2f3cb0 msgr2=0x55dc6d2f6ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:00.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:00.891+0000 7f68323b4740 1 --2- 172.21.15.45:0/3465072764 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55dc6d2f3cb0 0x55dc6d2f6ce0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f68140116d0 tx=0x7f6814005f90 comp rx=0 tx=0).stop 2024-04-16T17:13:00.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:00.891+0000 7f68323b4740 1 -- 172.21.15.45:0/3465072764 shutdown_connections 2024-04-16T17:13:00.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:00.891+0000 7f68323b4740 1 --2- 172.21.15.45:0/3465072764 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dc6d2fc890 0x55dc6d2f73d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:00.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:00.891+0000 7f68323b4740 1 --2- 172.21.15.45:0/3465072764 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55dc6d2f3cb0 0x55dc6d2f6ce0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:00.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:00.891+0000 7f68323b4740 1 --2- 172.21.15.45:0/3465072764 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55dc6d372a50 0x55dc6d374e70 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:00.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:00.891+0000 7f68323b4740 1 --2- 172.21.15.45:0/3465072764 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dc6d2f3390 0x55dc6d304fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:00.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:00.891+0000 7f680e000640 0 client.27296 ms_handle_reset on v2:172.21.15.104:6838/3660070592 2024-04-16T17:13:00.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:00.891+0000 7f68323b4740 1 -- 172.21.15.45:0/3465072764 >> 172.21.15.45:0/3465072764 conn(0x55dc6d2edac0 msgr2=0x55dc6d2f8100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:13:00.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:00.891+0000 7f68323b4740 1 -- 172.21.15.45:0/3465072764 shutdown_connections 2024-04-16T17:13:00.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:00.892+0000 7f68323b4740 1 -- 172.21.15.45:0/3465072764 wait complete. 2024-04-16T17:13:00.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:00.892+0000 7f68323b4740 20 client.27296 trim_cache size 0 max 16384 2024-04-16T17:13:00.895 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-16T17:13:00.895 INFO:teuthology.orchestra.run.smithi045.stdout: "status": "no active scrubs running", 2024-04-16T17:13:00.895 INFO:teuthology.orchestra.run.smithi045.stdout: "scrubs": {} 2024-04-16T17:13:00.895 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-16T17:13:00.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:00.892+0000 7f682e000640 1 -- 172.21.15.45:0/2081782678 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f68100a3da0 msgr2=0x7f68100a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:00.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:00.892+0000 7f682e000640 1 --2- 172.21.15.45:0/2081782678 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f68100a3da0 0x7f68100a6260 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x55dc6d2eec50 tx=0x7f6814015040 comp rx=0 tx=0).stop 2024-04-16T17:13:00.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:00.892+0000 7f682e000640 1 -- 172.21.15.45:0/2081782678 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f682819a800 msgr2=0x7f682819acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:00.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:00.892+0000 7f682e000640 1 --2- 172.21.15.45:0/2081782678 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f682819a800 0x7f682819acb0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f6820009bf0 tx=0x7f6820032ea0 comp rx=0 tx=0).stop 2024-04-16T17:13:00.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:00.892+0000 7f682e000640 1 -- 172.21.15.45:0/2081782678 shutdown_connections 2024-04-16T17:13:00.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:00.892+0000 7f682e000640 1 --2- 172.21.15.45:0/2081782678 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f68100a3da0 0x7f68100a6260 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:00.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:00.892+0000 7f682e000640 1 --2- 172.21.15.45:0/2081782678 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f682819a800 0x7f682819acb0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:00.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:00.892+0000 7f682e000640 1 --2- 172.21.15.45:0/2081782678 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f68281071e0 0x7f68281a0bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:00.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:00.892+0000 7f682e000640 1 --2- 172.21.15.45:0/2081782678 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6828102110 0x7f68281a0690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:00.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:00.892+0000 7f682e000640 1 -- 172.21.15.45:0/2081782678 >> 172.21.15.45:0/2081782678 conn(0x7f68280fd9d0 msgr2=0x7f68280fec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:13:00.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:00.892+0000 7f682e000640 1 -- 172.21.15.45:0/2081782678 shutdown_connections 2024-04-16T17:13:00.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:00.892+0000 7f682e000640 1 -- 172.21.15.45:0/2081782678 wait complete. 2024-04-16T17:13:00.909 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-16T17:13:00.909 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-16T17:13:00.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:13:00 smithi104 ceph-mon[36457]: pgmap v190: 129 pgs: 129 active+clean; 53 MiB data, 741 MiB used, 1.0 TiB / 1.0 TiB avail; 539 KiB/s rd, 306 KiB/s wr, 59 op/s 2024-04-16T17:13:00.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:13:00 smithi045 ceph-mon[31676]: pgmap v190: 129 pgs: 129 active+clean; 53 MiB data, 741 MiB used, 1.0 TiB / 1.0 TiB avail; 539 KiB/s rd, 306 KiB/s wr, 59 op/s 2024-04-16T17:13:01.055 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:13:00 smithi117 ceph-mon[36730]: pgmap v190: 129 pgs: 129 active+clean; 53 MiB data, 741 MiB used, 1.0 TiB / 1.0 TiB avail; 539 KiB/s rd, 306 KiB/s wr, 59 op/s 2024-04-16T17:13:01.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.119+0000 7f9162000640 1 -- 172.21.15.45:0/710435741 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f915c076530 msgr2=0x7f915c076910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:01.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.119+0000 7f9162000640 1 --2- 172.21.15.45:0/710435741 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f915c076530 0x7f915c076910 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f9150009930 tx=0x7f915002bc30 comp rx=0 tx=0).stop 2024-04-16T17:13:01.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.119+0000 7f9162000640 1 -- 172.21.15.45:0/710435741 shutdown_connections 2024-04-16T17:13:01.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.119+0000 7f9162000640 1 --2- 172.21.15.45:0/710435741 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f915c112ca0 0x7f915c115090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:01.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.119+0000 7f9162000640 1 --2- 172.21.15.45:0/710435741 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f915c076e50 0x7f915c112760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:01.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.119+0000 7f9162000640 1 --2- 172.21.15.45:0/710435741 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f915c076530 0x7f915c076910 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:01.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.119+0000 7f9162000640 1 -- 172.21.15.45:0/710435741 >> 172.21.15.45:0/710435741 conn(0x7f915c0ffa90 msgr2=0x7f915c101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:13:01.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.119+0000 7f9162000640 1 -- 172.21.15.45:0/710435741 shutdown_connections 2024-04-16T17:13:01.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.119+0000 7f9162000640 1 -- 172.21.15.45:0/710435741 wait complete. 2024-04-16T17:13:01.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.120+0000 7f9162000640 1 Processor -- start 2024-04-16T17:13:01.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.120+0000 7f9162000640 1 -- start start 2024-04-16T17:13:01.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.121+0000 7f9162000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f915c076530 0x7f915c19ee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:01.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.121+0000 7f9162000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f915c076e50 0x7f915c19f360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:01.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.121+0000 7f9162000640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f915c112ca0 0x7f915c1a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:01.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.121+0000 7f9162000640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f915c118860 con 0x7f915c076530 2024-04-16T17:13:01.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.121+0000 7f9162000640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f915c1186e0 con 0x7f915c112ca0 2024-04-16T17:13:01.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.121+0000 7f9162000640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f915c1189e0 con 0x7f915c076e50 2024-04-16T17:13:01.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.121+0000 7f915be00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f915c076e50 0x7f915c19f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:01.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.121+0000 7f9160c00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f915c076530 0x7f915c19ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:01.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.121+0000 7f915be00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f915c076e50 0x7f915c19f360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:53994/0 (socket says 172.21.15.45:53994) 2024-04-16T17:13:01.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.121+0000 7f915be00640 1 -- 172.21.15.45:0/3576663305 learned_addr learned my addr 172.21.15.45:0/3576663305 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:13:01.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.121+0000 7f9161600640 1 --2- 172.21.15.45:0/3576663305 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f915c112ca0 0x7f915c1a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:01.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.121+0000 7f915be00640 1 -- 172.21.15.45:0/3576663305 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f915c112ca0 msgr2=0x7f915c1a3690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:01.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.121+0000 7f915be00640 1 --2- 172.21.15.45:0/3576663305 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f915c112ca0 0x7f915c1a3690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:01.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.121+0000 7f915be00640 1 -- 172.21.15.45:0/3576663305 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f915c076530 msgr2=0x7f915c19ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:01.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.121+0000 7f915be00640 1 --2- 172.21.15.45:0/3576663305 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f915c076530 0x7f915c19ee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:01.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.121+0000 7f915be00640 1 -- 172.21.15.45:0/3576663305 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f915c1a3dc0 con 0x7f915c076e50 2024-04-16T17:13:01.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.121+0000 7f9160c00640 1 --2- 172.21.15.45:0/3576663305 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f915c076530 0x7f915c19ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:13:01.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.121+0000 7f9161600640 1 --2- 172.21.15.45:0/3576663305 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f915c112ca0 0x7f915c1a3690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:13:01.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.122+0000 7f915be00640 1 --2- 172.21.15.45:0/3576663305 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f915c076e50 0x7f915c19f360 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f914800e9f0 tx=0x7f914800edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:13:01.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.122+0000 7f9159600640 1 -- 172.21.15.45:0/3576663305 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f914800cd80 con 0x7f915c076e50 2024-04-16T17:13:01.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.122+0000 7f9162000640 1 -- 172.21.15.45:0/3576663305 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f915c1a40b0 con 0x7f915c076e50 2024-04-16T17:13:01.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.122+0000 7f9162000640 1 -- 172.21.15.45:0/3576663305 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f915c1ab940 con 0x7f915c076e50 2024-04-16T17:13:01.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.122+0000 7f9159600640 1 -- 172.21.15.45:0/3576663305 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9148005500 con 0x7f915c076e50 2024-04-16T17:13:01.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.122+0000 7f9159600640 1 -- 172.21.15.45:0/3576663305 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9148010690 con 0x7f915c076e50 2024-04-16T17:13:01.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.123+0000 7f9162000640 1 -- 172.21.15.45:0/3576663305 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f915c069190 con 0x7f915c076e50 2024-04-16T17:13:01.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.126+0000 7f9159600640 1 -- 172.21.15.45:0/3576663305 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f91480046c0 con 0x7f915c076e50 2024-04-16T17:13:01.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.127+0000 7f9159600640 1 --2- 172.21.15.45:0/3576663305 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f91440a3d20 0x7f91440a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:01.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.128+0000 7f9159600640 1 -- 172.21.15.45:0/3576663305 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f91480d11a0 con 0x7f915c076e50 2024-04-16T17:13:01.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.128+0000 7f9160c00640 1 --2- 172.21.15.45:0/3576663305 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f91440a3d20 0x7f91440a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:01.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.128+0000 7f9160c00640 1 --2- 172.21.15.45:0/3576663305 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f91440a3d20 0x7f91440a61e0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f9150009900 tx=0x7f915002df90 comp rx=0 tx=0).ready entity=mgr.25411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:13:01.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.129+0000 7f9159600640 1 -- 172.21.15.45:0/3576663305 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9148053ff0 con 0x7f915c076e50 2024-04-16T17:13:01.274 INFO:tasks.workunit.client.0.smithi045.stdout: 1 101053 1.71 MB/sec execute 183 sec latency 1577.748 ms 2024-04-16T17:13:01.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.380+0000 7f9162000640 1 -- 172.21.15.45:0/3576663305 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f915c10de30 con 0x7f915c076e50 2024-04-16T17:13:01.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.381+0000 7f9159600640 1 -- 172.21.15.45:0/3576663305 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) ==== 94+0+11891 (secure 0 0 0) 0x7f914809d490 con 0x7f915c076e50 2024-04-16T17:13:01.381 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-16T17:13:01.381 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":37,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/1748842443","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":1748842443},{"type":"v1","addr":"172.21.15.45:6835","nonce":1748842443}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6839/1308627051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":1308627051},{"type":"v1","addr":"172.21.15.45:6839","nonce":1308627051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/1768242489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1768242489},{"type":"v1","addr":"172.21.15.117:6833","nonce":1768242489}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6833/3558902585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":3558902585},{"type":"v1","addr":"172.21.15.104:6833","nonce":3558902585}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24478,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/1308142903","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":1308142903},{"type":"v1","addr":"172.21.15.45:6837","nonce":1308142903}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24491,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6835/3430679937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":3430679937},{"type":"v1","addr":"172.21.15.104:6835","nonce":3430679937}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":37,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-16T17:03:24.663611+0000","modified":"2024-04-16T17:12:08.516204+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24581,"mds_1":24569,"mds_2":24532},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24499":{"gid":24499,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/2224123841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2224123841},{"type":"v1","addr":"172.21.15.117:6835","nonce":2224123841}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6837/2471356815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":2471356815},{"type":"v1","addr":"172.21.15.117:6837","nonce":2471356815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24524":{"gid":24524,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6837/2647756509","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":2647756509},{"type":"v1","addr":"172.21.15.104:6837","nonce":2647756509}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24532":{"gid":24532,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":13,"addr":"172.21.15.117:6839/2378675949","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":2378675949},{"type":"v1","addr":"172.21.15.117:6839","nonce":2378675949}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24569":{"gid":24569,"name":"a","rank":1,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.45:6841/3755310822","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6840","nonce":3755310822},{"type":"v1","addr":"172.21.15.45:6841","nonce":3755310822}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24581":{"gid":24581,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.104:6839/3660070592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6838","nonce":3660070592},{"type":"v1","addr":"172.21.15.104:6839","nonce":3660070592}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24581,"qdb_cluster":[24569,24581,24532]},"id":1}]} 2024-04-16T17:13:01.381 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 37 2024-04-16T17:13:01.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.383+0000 7f9162000640 1 -- 172.21.15.45:0/3576663305 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f91440a3d20 msgr2=0x7f91440a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:01.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.383+0000 7f9162000640 1 --2- 172.21.15.45:0/3576663305 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f91440a3d20 0x7f91440a61e0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f9150009900 tx=0x7f915002df90 comp rx=0 tx=0).stop 2024-04-16T17:13:01.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.383+0000 7f9162000640 1 -- 172.21.15.45:0/3576663305 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f915c076e50 msgr2=0x7f915c19f360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:01.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.383+0000 7f9162000640 1 --2- 172.21.15.45:0/3576663305 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f915c076e50 0x7f915c19f360 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f914800e9f0 tx=0x7f914800edf0 comp rx=0 tx=0).stop 2024-04-16T17:13:01.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.384+0000 7f9162000640 1 -- 172.21.15.45:0/3576663305 shutdown_connections 2024-04-16T17:13:01.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.384+0000 7f9162000640 1 --2- 172.21.15.45:0/3576663305 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f91440a3d20 0x7f91440a61e0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:01.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.384+0000 7f9162000640 1 --2- 172.21.15.45:0/3576663305 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f915c076e50 0x7f915c19f360 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:01.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.384+0000 7f9162000640 1 --2- 172.21.15.45:0/3576663305 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f915c112ca0 0x7f915c1a3690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:01.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.384+0000 7f9162000640 1 --2- 172.21.15.45:0/3576663305 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f915c076530 0x7f915c19ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:01.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.384+0000 7f9162000640 1 -- 172.21.15.45:0/3576663305 >> 172.21.15.45:0/3576663305 conn(0x7f915c0ffa90 msgr2=0x7f915c114390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:13:01.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.384+0000 7f9162000640 1 -- 172.21.15.45:0/3576663305 shutdown_connections 2024-04-16T17:13:01.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.384+0000 7f9162000640 1 -- 172.21.15.45:0/3576663305 wait complete. 2024-04-16T17:13:01.401 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-16T17:13:01.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.615+0000 7f655fe00640 1 -- 172.21.15.45:0/821895178 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6560074cf0 msgr2=0x7f656010e200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:01.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.615+0000 7f655fe00640 1 --2- 172.21.15.45:0/821895178 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6560074cf0 0x7f656010e200 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f6548009900 tx=0x7f654802bcc0 comp rx=0 tx=0).stop 2024-04-16T17:13:01.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.615+0000 7f655fe00640 1 -- 172.21.15.45:0/821895178 shutdown_connections 2024-04-16T17:13:01.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.615+0000 7f655fe00640 1 --2- 172.21.15.45:0/821895178 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6560074cf0 0x7f656010e200 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:01.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.615+0000 7f655fe00640 1 --2- 172.21.15.45:0/821895178 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f656010e740 0x7f6560110b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:01.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.615+0000 7f655fe00640 1 --2- 172.21.15.45:0/821895178 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f65600743d0 0x7f65600747b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:01.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.615+0000 7f655fe00640 1 -- 172.21.15.45:0/821895178 >> 172.21.15.45:0/821895178 conn(0x7f65600fd9d0 msgr2=0x7f65600ffdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:13:01.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.615+0000 7f655fe00640 1 -- 172.21.15.45:0/821895178 shutdown_connections 2024-04-16T17:13:01.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.615+0000 7f655fe00640 1 -- 172.21.15.45:0/821895178 wait complete. 2024-04-16T17:13:01.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.616+0000 7f655fe00640 1 Processor -- start 2024-04-16T17:13:01.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.616+0000 7f655fe00640 1 -- start start 2024-04-16T17:13:01.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.616+0000 7f655fe00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f65600743d0 0x7f65601a06a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:01.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.616+0000 7f655fe00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6560074cf0 0x7f65601a0be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:01.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.616+0000 7f655fe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f656010e740 0x7f656019a860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:01.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.616+0000 7f655fe00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f65601144a0 con 0x7f656010e740 2024-04-16T17:13:01.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.616+0000 7f655fe00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f6560114320 con 0x7f65600743d0 2024-04-16T17:13:01.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.616+0000 7f655fe00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f6560114620 con 0x7f6560074cf0 2024-04-16T17:13:01.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.617+0000 7f655ea00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f65600743d0 0x7f65601a06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:01.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.617+0000 7f655ea00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f65600743d0 0x7f65601a06a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:35824/0 (socket says 172.21.15.45:35824) 2024-04-16T17:13:01.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.617+0000 7f655ea00640 1 -- 172.21.15.45:0/1108999681 learned_addr learned my addr 172.21.15.45:0/1108999681 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:13:01.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.617+0000 7f655f400640 1 --2- 172.21.15.45:0/1108999681 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f656010e740 0x7f656019a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:01.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.617+0000 7f655e000640 1 --2- 172.21.15.45:0/1108999681 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6560074cf0 0x7f65601a0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:01.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.617+0000 7f655ea00640 1 -- 172.21.15.45:0/1108999681 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6560074cf0 msgr2=0x7f65601a0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:01.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.617+0000 7f655ea00640 1 --2- 172.21.15.45:0/1108999681 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6560074cf0 0x7f65601a0be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:01.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.617+0000 7f655ea00640 1 -- 172.21.15.45:0/1108999681 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f656010e740 msgr2=0x7f656019a860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:01.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.617+0000 7f655ea00640 1 --2- 172.21.15.45:0/1108999681 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f656010e740 0x7f656019a860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:01.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.617+0000 7f655ea00640 1 -- 172.21.15.45:0/1108999681 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f656019b030 con 0x7f65600743d0 2024-04-16T17:13:01.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.617+0000 7f655ea00640 1 --2- 172.21.15.45:0/1108999681 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f65600743d0 0x7f65601a06a0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f655000d950 tx=0x7f655000de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:13:01.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.618+0000 7f6557400640 1 -- 172.21.15.45:0/1108999681 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6550017030 con 0x7f65600743d0 2024-04-16T17:13:01.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.618+0000 7f655fe00640 1 -- 172.21.15.45:0/1108999681 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f656019b320 con 0x7f65600743d0 2024-04-16T17:13:01.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.618+0000 7f6557400640 1 -- 172.21.15.45:0/1108999681 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6550004020 con 0x7f65600743d0 2024-04-16T17:13:01.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.618+0000 7f6557400640 1 -- 172.21.15.45:0/1108999681 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65500047a0 con 0x7f65600743d0 2024-04-16T17:13:01.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.618+0000 7f655fe00640 1 -- 172.21.15.45:0/1108999681 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65601a74f0 con 0x7f65600743d0 2024-04-16T17:13:01.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.622+0000 7f6568637740 1 Processor -- start 2024-04-16T17:13:01.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.622+0000 7f6568637740 1 -- start start 2024-04-16T17:13:01.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.622+0000 7f6568637740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ccabc7ef10 0x55ccabc7f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:01.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.622+0000 7f6568637740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ccabc7f830 0x55ccabc87ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:01.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.622+0000 7f6568637740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ccabc88410 0x55ccabc8a800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:01.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.622+0000 7f6568637740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55ccabbe4020 con 0x55ccabc88410 2024-04-16T17:13:01.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.622+0000 7f6568637740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55ccabb8a460 con 0x55ccabc7ef10 2024-04-16T17:13:01.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.622+0000 7f6568637740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55ccabbdffe0 con 0x55ccabc7f830 2024-04-16T17:13:01.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.623+0000 7f655e000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ccabc88410 0x55ccabc8a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:01.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.623+0000 7f655f400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ccabc7ef10 0x55ccabc7f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:01.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.623+0000 7f655f400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ccabc7ef10 0x55ccabc7f2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:35838/0 (socket says 172.21.15.45:35838) 2024-04-16T17:13:01.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.623+0000 7f655f400640 1 -- 172.21.15.45:0/3096174492 learned_addr learned my addr 172.21.15.45:0/3096174492 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:13:01.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.623+0000 7f6557400640 1 -- 172.21.15.45:0/1108999681 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f65500041d0 con 0x7f65600743d0 2024-04-16T17:13:01.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.623+0000 7f655f400640 1 -- 172.21.15.45:0/3096174492 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ccabc7f830 msgr2=0x55ccabc87ed0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-16T17:13:01.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.623+0000 7f655f400640 1 --2- 172.21.15.45:0/3096174492 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ccabc7f830 0x55ccabc87ed0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:01.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.623+0000 7f655f400640 1 -- 172.21.15.45:0/3096174492 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ccabc88410 msgr2=0x55ccabc8a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:13:01.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.623+0000 7f655f400640 1 --2- 172.21.15.45:0/3096174492 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ccabc88410 0x55ccabc8a800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:01.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.623+0000 7f655f400640 1 -- 172.21.15.45:0/3096174492 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ccabb92040 con 0x55ccabc7ef10 2024-04-16T17:13:01.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.623+0000 7f655e000640 1 --2- 172.21.15.45:0/3096174492 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ccabc88410 0x55ccabc8a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:13:01.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.624+0000 7f655f400640 1 --2- 172.21.15.45:0/3096174492 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ccabc7ef10 0x55ccabc7f2f0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f6558009f20 tx=0x7f6558010860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=394bfb9d71b91219 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:13:01.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.624+0000 7f654fe00640 1 -- 172.21.15.45:0/3096174492 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6558003c30 con 0x55ccabc7ef10 2024-04-16T17:13:01.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.624+0000 7f654fe00640 1 -- 172.21.15.45:0/3096174492 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6558015070 con 0x55ccabc7ef10 2024-04-16T17:13:01.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.624+0000 7f654fe00640 1 -- 172.21.15.45:0/3096174492 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65580118c0 con 0x55ccabc7ef10 2024-04-16T17:13:01.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.625+0000 7f6568637740 1 -- 172.21.15.45:0/3096174492 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ccabc7ef10 msgr2=0x55ccabc7f2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:01.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.625+0000 7f6568637740 1 --2- 172.21.15.45:0/3096174492 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ccabc7ef10 0x55ccabc7f2f0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f6558009f20 tx=0x7f6558010860 comp rx=0 tx=0).stop 2024-04-16T17:13:01.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.625+0000 7f6557400640 1 --2- 172.21.15.45:0/1108999681 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f65440a3da0 0x7f65440a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:01.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.625+0000 7f6568637740 1 -- 172.21.15.45:0/3096174492 shutdown_connections 2024-04-16T17:13:01.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.625+0000 7f6568637740 1 --2- 172.21.15.45:0/3096174492 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ccabc7f830 0x55ccabc87ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:01.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.625+0000 7f6568637740 1 --2- 172.21.15.45:0/3096174492 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ccabc88410 0x55ccabc8a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:01.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.625+0000 7f6568637740 1 --2- 172.21.15.45:0/3096174492 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ccabc7ef10 0x55ccabc7f2f0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:01.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.625+0000 7f6568637740 1 -- 172.21.15.45:0/3096174492 >> 172.21.15.45:0/3096174492 conn(0x55ccabc79640 msgr2=0x55ccabc79a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:13:01.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.625+0000 7f6557400640 1 -- 172.21.15.45:0/1108999681 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f65500d04b0 con 0x7f65600743d0 2024-04-16T17:13:01.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.625+0000 7f655f400640 1 --2- 172.21.15.45:0/1108999681 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f65440a3da0 0x7f65440a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:01.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.625+0000 7f6568637740 1 -- 172.21.15.45:0/3096174492 shutdown_connections 2024-04-16T17:13:01.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.625+0000 7f6568637740 1 -- 172.21.15.45:0/3096174492 wait complete. 2024-04-16T17:13:01.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.626+0000 7f6568637740 1 -- start start 2024-04-16T17:13:01.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.626+0000 7f655f400640 1 --2- 172.21.15.45:0/1108999681 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f65440a3da0 0x7f65440a6260 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f655800a010 tx=0x7f6558000f40 comp rx=0 tx=0).ready entity=mgr.25411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:13:01.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.626+0000 7f6568637740 1 Processor -- start 2024-04-16T17:13:01.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.626+0000 7f6568637740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ccabc7ef10 0x55ccabc854b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:01.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.626+0000 7f6568637740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ccabc7f830 0x55ccabc859f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:01.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.626+0000 7f6568637740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ccabc88410 0x55ccabc85f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:01.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.626+0000 7f6568637740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55ccabc8b200 con 0x55ccabc7f830 2024-04-16T17:13:01.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.626+0000 7f6568637740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55ccabb9d830 con 0x55ccabc88410 2024-04-16T17:13:01.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.626+0000 7f6568637740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55ccabc8b380 con 0x55ccabc7ef10 2024-04-16T17:13:01.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.626+0000 7f6568637740 10 client.0 _ll_register_callbacks cb 0x55ccabba5150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:13:01.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.627+0000 7f655ea00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ccabc7ef10 0x55ccabc854b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:01.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.627+0000 7f655ea00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ccabc7ef10 0x55ccabc854b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:54010/0 (socket says 172.21.15.45:54010) 2024-04-16T17:13:01.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.627+0000 7f655ea00640 1 -- 172.21.15.45:0/45588545 learned_addr learned my addr 172.21.15.45:0/45588545 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:13:01.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.627+0000 7f655e000640 1 --2- 172.21.15.45:0/45588545 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ccabc7f830 0x55ccabc859f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:01.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.627+0000 7f655f400640 1 --2- 172.21.15.45:0/45588545 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ccabc88410 0x55ccabc85f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:01.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.627+0000 7f655ea00640 1 -- 172.21.15.45:0/45588545 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ccabc88410 msgr2=0x55ccabc85f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:13:01.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.627+0000 7f655ea00640 1 --2- 172.21.15.45:0/45588545 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ccabc88410 0x55ccabc85f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:01.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.627+0000 7f655ea00640 1 -- 172.21.15.45:0/45588545 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ccabc7f830 msgr2=0x55ccabc859f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:13:01.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.627+0000 7f655ea00640 1 --2- 172.21.15.45:0/45588545 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ccabc7f830 0x55ccabc859f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:01.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.627+0000 7f655ea00640 1 -- 172.21.15.45:0/45588545 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ccabba1920 con 0x55ccabc7ef10 2024-04-16T17:13:01.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.627+0000 7f655ea00640 1 --2- 172.21.15.45:0/45588545 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ccabc7ef10 0x55ccabc854b0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f6550000c00 tx=0x7f6550098450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3f5ad7854c8c6160 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:13:01.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.627+0000 7f654cc00640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-04-16T17:13:01.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.627+0000 7f654cc00640 1 -- 172.21.15.45:0/45588545 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65500a0450 con 0x55ccabc7ef10 2024-04-16T17:13:01.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.628+0000 7f654cc00640 1 -- 172.21.15.45:0/45588545 <== mon.1 v2:172.21.15.117:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f6550098710 con 0x55ccabc7ef10 2024-04-16T17:13:01.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.628+0000 7f6568637740 1 -- 172.21.15.45:0/45588545 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6558015070 con 0x55ccabc7ef10 2024-04-16T17:13:01.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.628+0000 7f654cc00640 1 -- 172.21.15.45:0/45588545 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f65500a4d10 con 0x55ccabc7ef10 2024-04-16T17:13:01.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.628+0000 7f6568637740 10 client.27253 fetch_fsmap learned FSMap version 37 2024-04-16T17:13:01.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.628+0000 7f6568637740 1 -- 172.21.15.45:0/45588545 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55ccabb6f7f0 con 0x55ccabc7ef10 2024-04-16T17:13:01.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.628+0000 7f654cc00640 1 -- 172.21.15.45:0/45588545 <== mon.1 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6558015070 con 0x55ccabc7ef10 2024-04-16T17:13:01.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.628+0000 7f654cc00640 1 -- 172.21.15.45:0/45588545 <== mon.1 v2:172.21.15.117:3300/0 5 ==== fsmap(e 37) ==== 8358+0+0 (secure 0 0 0) 0x7f65500b0bb0 con 0x55ccabc7ef10 2024-04-16T17:13:01.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.628+0000 7f6568637740 10 client.27253 fetch_fsmap finished waiting for FSMap version 37 2024-04-16T17:13:01.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.628+0000 7f6568637740 10 client.27253 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-16T17:13:01.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.628+0000 7f6568637740 20 client.27253 populate_metadata read hostname 'smithi045' 2024-04-16T17:13:01.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.628+0000 7f6568637740 1 --2- 172.21.15.45:0/45588545 >> [v2:172.21.15.117:6838/2378675949,v1:172.21.15.117:6839/2378675949] conn(0x55ccabc987f0 0x55ccabc9abe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:01.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.628+0000 7f6568637740 4 client.27253 mds_command: new command op to 24532 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:13:01.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.628+0000 7f6568637740 1 -- 172.21.15.45:0/45588545 --> [v2:172.21.15.117:6838/2378675949,v1:172.21.15.117:6839/2378675949] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ccabbf1a40 con 0x55ccabc987f0 2024-04-16T17:13:01.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.629+0000 7f655e000640 1 --2- 172.21.15.45:0/45588545 >> [v2:172.21.15.117:6838/2378675949,v1:172.21.15.117:6839/2378675949] conn(0x55ccabc987f0 0x55ccabc9abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:01.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.629+0000 7f655e000640 1 --2- 172.21.15.45:0/45588545 >> [v2:172.21.15.117:6838/2378675949,v1:172.21.15.117:6839/2378675949] conn(0x55ccabc987f0 0x55ccabc9abe0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=57091f1ed3913bd6 server_cookie=4e763a1f2dec010a in_seq=0 out_seq=0 2024-04-16T17:13:01.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.629+0000 7f654cc00640 10 client.27253 ms_handle_connect on v2:172.21.15.117:6838/2378675949 2024-04-16T17:13:01.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.630+0000 7f654cc00640 1 -- 172.21.15.45:0/45588545 <== mds.2 v2:172.21.15.117:6838/2378675949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ccabbf1a40 con 0x55ccabc987f0 2024-04-16T17:13:01.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.630+0000 7f654cc00640 10 client.27253 handle_command_reply: tid=0 2024-04-16T17:13:01.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.630+0000 7f6568637740 1 client.27253 shutdown 2024-04-16T17:13:01.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.630+0000 7f6568637740 1 -- 172.21.15.45:0/45588545 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ccabc7ef10 msgr2=0x55ccabc854b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:01.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.630+0000 7f6568637740 1 --2- 172.21.15.45:0/45588545 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ccabc7ef10 0x55ccabc854b0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f6550000c00 tx=0x7f6550098450 comp rx=0 tx=0).stop 2024-04-16T17:13:01.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.630+0000 7f6568637740 1 -- 172.21.15.45:0/45588545 shutdown_connections 2024-04-16T17:13:01.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.630+0000 7f6568637740 1 --2- 172.21.15.45:0/45588545 >> [v2:172.21.15.117:6838/2378675949,v1:172.21.15.117:6839/2378675949] conn(0x55ccabc987f0 0x55ccabc9abe0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:01.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.630+0000 7f6568637740 1 --2- 172.21.15.45:0/45588545 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ccabc88410 0x55ccabc85f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:01.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.630+0000 7f6568637740 1 --2- 172.21.15.45:0/45588545 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ccabc7f830 0x55ccabc859f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:01.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.630+0000 7f6568637740 1 --2- 172.21.15.45:0/45588545 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ccabc7ef10 0x55ccabc854b0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:01.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.630+0000 7f6568637740 1 -- 172.21.15.45:0/45588545 >> 172.21.15.45:0/45588545 conn(0x55ccabc79640 msgr2=0x55ccabc7b390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:13:01.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.630+0000 7f654cc00640 0 client.27253 ms_handle_reset on v2:172.21.15.117:6838/2378675949 2024-04-16T17:13:01.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.630+0000 7f6568637740 1 -- 172.21.15.45:0/45588545 shutdown_connections 2024-04-16T17:13:01.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.630+0000 7f6568637740 1 -- 172.21.15.45:0/45588545 wait complete. 2024-04-16T17:13:01.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.631+0000 7f6568637740 20 client.27253 trim_cache size 0 max 16384 2024-04-16T17:13:01.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.646+0000 7f6568637740 1 Processor -- start 2024-04-16T17:13:01.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.646+0000 7f6568637740 1 -- start start 2024-04-16T17:13:01.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.647+0000 7f6568637740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ccabc7ef10 0x55ccabc87be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:01.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.647+0000 7f6568637740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ccabc7f830 0x55ccabc940c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:01.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.647+0000 7f6568637740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ccabc88410 0x55ccabca0440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:01.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.647+0000 7f6568637740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f65601b3df0 con 0x55ccabc7ef10 2024-04-16T17:13:01.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.647+0000 7f6568637740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f65601b1630 con 0x55ccabc7f830 2024-04-16T17:13:01.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.647+0000 7f6568637740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f65601af460 con 0x55ccabc88410 2024-04-16T17:13:01.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.647+0000 7f655e000640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ccabc7f830 0x55ccabc940c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:01.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.647+0000 7f655ea00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ccabc7ef10 0x55ccabc87be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:01.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.647+0000 7f655f400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ccabc88410 0x55ccabca0440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:01.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.647+0000 7f655e000640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ccabc7f830 0x55ccabc940c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:35848/0 (socket says 172.21.15.45:35848) 2024-04-16T17:13:01.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.647+0000 7f655e000640 1 -- 172.21.15.45:0/929172962 learned_addr learned my addr 172.21.15.45:0/929172962 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:13:01.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.648+0000 7f655f400640 1 -- 172.21.15.45:0/929172962 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ccabc7f830 msgr2=0x55ccabc940c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:13:01.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.648+0000 7f655f400640 1 --2- 172.21.15.45:0/929172962 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ccabc7f830 0x55ccabc940c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:01.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.648+0000 7f655f400640 1 -- 172.21.15.45:0/929172962 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ccabc7ef10 msgr2=0x55ccabc87be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:13:01.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.648+0000 7f655f400640 1 --2- 172.21.15.45:0/929172962 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ccabc7ef10 0x55ccabc87be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:01.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.648+0000 7f655f400640 1 -- 172.21.15.45:0/929172962 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ccab8f63f0 con 0x55ccabc88410 2024-04-16T17:13:01.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.648+0000 7f655f400640 1 --2- 172.21.15.45:0/929172962 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ccabc88410 0x55ccabca0440 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f6558009ef0 tx=0x7f65580096a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2e9bb5cb270a20c9 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:13:01.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.648+0000 7f654f400640 1 -- 172.21.15.45:0/929172962 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65580049e0 con 0x55ccabc88410 2024-04-16T17:13:01.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.648+0000 7f654f400640 1 -- 172.21.15.45:0/929172962 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6558015040 con 0x55ccabc88410 2024-04-16T17:13:01.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.648+0000 7f654f400640 1 -- 172.21.15.45:0/929172962 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6558005280 con 0x55ccabc88410 2024-04-16T17:13:01.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.649+0000 7f6568637740 1 -- 172.21.15.45:0/929172962 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ccabc88410 msgr2=0x55ccabca0440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:01.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.649+0000 7f6568637740 1 --2- 172.21.15.45:0/929172962 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ccabc88410 0x55ccabca0440 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f6558009ef0 tx=0x7f65580096a0 comp rx=0 tx=0).stop 2024-04-16T17:13:01.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.650+0000 7f6568637740 1 -- 172.21.15.45:0/929172962 shutdown_connections 2024-04-16T17:13:01.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.650+0000 7f6568637740 1 --2- 172.21.15.45:0/929172962 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ccabc88410 0x55ccabca0440 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:01.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.650+0000 7f6568637740 1 --2- 172.21.15.45:0/929172962 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ccabc7f830 0x55ccabc940c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:01.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.650+0000 7f6568637740 1 --2- 172.21.15.45:0/929172962 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ccabc7ef10 0x55ccabc87be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:01.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.650+0000 7f6568637740 1 -- 172.21.15.45:0/929172962 >> 172.21.15.45:0/929172962 conn(0x55ccabc79640 msgr2=0x55ccabc78700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:13:01.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.650+0000 7f6568637740 1 -- 172.21.15.45:0/929172962 shutdown_connections 2024-04-16T17:13:01.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.650+0000 7f6568637740 1 -- 172.21.15.45:0/929172962 wait complete. 2024-04-16T17:13:01.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.650+0000 7f6568637740 1 -- start start 2024-04-16T17:13:01.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.650+0000 7f6568637740 1 Processor -- start 2024-04-16T17:13:01.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.651+0000 7f6568637740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ccabc7ef10 0x55ccabc82160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:01.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.651+0000 7f6568637740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ccabc7f830 0x55ccabc826a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:01.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.651+0000 7f6568637740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ccabc88410 0x55ccabc82be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:01.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.651+0000 7f6568637740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f65601afaf0 con 0x55ccabc88410 2024-04-16T17:13:01.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.651+0000 7f6568637740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f65601b0f60 con 0x55ccabc7f830 2024-04-16T17:13:01.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.651+0000 7f6568637740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f65601b01c0 con 0x55ccabc7ef10 2024-04-16T17:13:01.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.651+0000 7f6568637740 10 client.0 _ll_register_callbacks cb 0x7f6550004540 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:13:01.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.651+0000 7f655ea00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ccabc7ef10 0x55ccabc82160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:01.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.651+0000 7f655e000640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ccabc7f830 0x55ccabc826a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:01.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.651+0000 7f655ea00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ccabc7ef10 0x55ccabc82160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:54026/0 (socket says 172.21.15.45:54026) 2024-04-16T17:13:01.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.651+0000 7f655ea00640 1 -- 172.21.15.45:0/3401788402 learned_addr learned my addr 172.21.15.45:0/3401788402 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:13:01.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.651+0000 7f655e000640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ccabc7f830 0x55ccabc826a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:35852/0 (socket says 172.21.15.45:35852) 2024-04-16T17:13:01.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.652+0000 7f655f400640 1 --2- 172.21.15.45:0/3401788402 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ccabc88410 0x55ccabc82be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:01.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.652+0000 7f655e000640 1 -- 172.21.15.45:0/3401788402 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ccabc7ef10 msgr2=0x55ccabc82160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:13:01.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.652+0000 7f655e000640 1 --2- 172.21.15.45:0/3401788402 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ccabc7ef10 0x55ccabc82160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:01.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.652+0000 7f655e000640 1 -- 172.21.15.45:0/3401788402 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ccabc88410 msgr2=0x55ccabc82be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:13:01.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.652+0000 7f655e000640 1 --2- 172.21.15.45:0/3401788402 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ccabc88410 0x55ccabc82be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:01.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.652+0000 7f655e000640 1 -- 172.21.15.45:0/3401788402 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ccab9c07d0 con 0x55ccabc7f830 2024-04-16T17:13:01.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.652+0000 7f655ea00640 1 --2- 172.21.15.45:0/3401788402 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ccabc7ef10 0x55ccabc82160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:13:01.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.652+0000 7f655e000640 1 --2- 172.21.15.45:0/3401788402 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ccabc7f830 0x55ccabc826a0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f6548005c40 tx=0x7f654802cca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=518cb0ff113f00f5 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:13:01.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.652+0000 7f654cc00640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-04-16T17:13:01.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.653+0000 7f654cc00640 1 -- 172.21.15.45:0/3401788402 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f654802ce50 con 0x55ccabc7f830 2024-04-16T17:13:01.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.653+0000 7f6568637740 1 -- 172.21.15.45:0/3401788402 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6558015040 con 0x55ccabc7f830 2024-04-16T17:13:01.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.653+0000 7f654cc00640 1 -- 172.21.15.45:0/3401788402 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f6548004260 con 0x55ccabc7f830 2024-04-16T17:13:01.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.653+0000 7f654cc00640 1 -- 172.21.15.45:0/3401788402 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f6548004460 con 0x55ccabc7f830 2024-04-16T17:13:01.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.653+0000 7f6568637740 10 client.27320 fetch_fsmap learned FSMap version 37 2024-04-16T17:13:01.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.653+0000 7f6568637740 1 -- 172.21.15.45:0/3401788402 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55ccabcfed50 con 0x55ccabc7f830 2024-04-16T17:13:01.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.653+0000 7f654cc00640 1 -- 172.21.15.45:0/3401788402 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6558015040 con 0x55ccabc7f830 2024-04-16T17:13:01.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.653+0000 7f654cc00640 1 -- 172.21.15.45:0/3401788402 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 37) ==== 8358+0+0 (secure 0 0 0) 0x7f6548004ef0 con 0x55ccabc7f830 2024-04-16T17:13:01.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.654+0000 7f6568637740 10 client.27320 fetch_fsmap finished waiting for FSMap version 37 2024-04-16T17:13:01.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.654+0000 7f6568637740 10 client.27320 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-16T17:13:01.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.654+0000 7f6568637740 20 client.27320 populate_metadata read hostname 'smithi045' 2024-04-16T17:13:01.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.654+0000 7f6568637740 1 --2- 172.21.15.45:0/3401788402 >> [v2:172.21.15.117:6838/2378675949,v1:172.21.15.117:6839/2378675949] conn(0x55ccabcff550 0x55ccabd01970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:01.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.654+0000 7f6568637740 4 client.27320 mds_command: new command op to 24532 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T17:13:01.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.654+0000 7f6568637740 1 -- 172.21.15.45:0/3401788402 --> [v2:172.21.15.117:6838/2378675949,v1:172.21.15.117:6839/2378675949] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ccabbeb8b0 con 0x55ccabcff550 2024-04-16T17:13:01.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.654+0000 7f655ea00640 1 --2- 172.21.15.45:0/3401788402 >> [v2:172.21.15.117:6838/2378675949,v1:172.21.15.117:6839/2378675949] conn(0x55ccabcff550 0x55ccabd01970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:01.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.654+0000 7f655ea00640 1 --2- 172.21.15.45:0/3401788402 >> [v2:172.21.15.117:6838/2378675949,v1:172.21.15.117:6839/2378675949] conn(0x55ccabcff550 0x55ccabd01970 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3a571e36f1a5a00a server_cookie=5a25b48a5a4a729a in_seq=0 out_seq=0 2024-04-16T17:13:01.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.654+0000 7f654cc00640 10 client.27320 ms_handle_connect on v2:172.21.15.117:6838/2378675949 2024-04-16T17:13:01.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.655+0000 7f654cc00640 1 -- 172.21.15.45:0/3401788402 <== mds.2 v2:172.21.15.117:6838/2378675949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ccabbeb8b0 con 0x55ccabcff550 2024-04-16T17:13:01.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.655+0000 7f654cc00640 10 client.27320 handle_command_reply: tid=0 2024-04-16T17:13:01.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.655+0000 7f6568637740 1 client.27320 shutdown 2024-04-16T17:13:01.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.655+0000 7f6568637740 1 -- 172.21.15.45:0/3401788402 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ccabc7f830 msgr2=0x55ccabc826a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:01.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.655+0000 7f6568637740 1 --2- 172.21.15.45:0/3401788402 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ccabc7f830 0x55ccabc826a0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f6548005c40 tx=0x7f654802cca0 comp rx=0 tx=0).stop 2024-04-16T17:13:01.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.655+0000 7f6568637740 1 -- 172.21.15.45:0/3401788402 shutdown_connections 2024-04-16T17:13:01.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.655+0000 7f6568637740 1 --2- 172.21.15.45:0/3401788402 >> [v2:172.21.15.117:6838/2378675949,v1:172.21.15.117:6839/2378675949] conn(0x55ccabcff550 0x55ccabd01970 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:01.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.655+0000 7f6568637740 1 --2- 172.21.15.45:0/3401788402 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ccabc88410 0x55ccabc82be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:01.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.655+0000 7f6568637740 1 --2- 172.21.15.45:0/3401788402 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ccabc7f830 0x55ccabc826a0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:01.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.655+0000 7f654cc00640 0 client.27320 ms_handle_reset on v2:172.21.15.117:6838/2378675949 2024-04-16T17:13:01.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.655+0000 7f6568637740 1 --2- 172.21.15.45:0/3401788402 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ccabc7ef10 0x55ccabc82160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:01.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.655+0000 7f6568637740 1 -- 172.21.15.45:0/3401788402 >> 172.21.15.45:0/3401788402 conn(0x55ccabc79640 msgr2=0x55ccabc78700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:13:01.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.655+0000 7f6568637740 1 -- 172.21.15.45:0/3401788402 shutdown_connections 2024-04-16T17:13:01.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.655+0000 7f6568637740 1 -- 172.21.15.45:0/3401788402 wait complete. 2024-04-16T17:13:01.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.655+0000 7f6568637740 20 client.27320 trim_cache size 0 max 16384 2024-04-16T17:13:01.658 INFO:teuthology.orchestra.run.smithi045.stdout:[] 2024-04-16T17:13:01.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.656+0000 7f655fe00640 1 -- 172.21.15.45:0/1108999681 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f65440a3da0 msgr2=0x7f65440a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:01.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.656+0000 7f655fe00640 1 --2- 172.21.15.45:0/1108999681 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f65440a3da0 0x7f65440a6260 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f655800a010 tx=0x7f6558000f40 comp rx=0 tx=0).stop 2024-04-16T17:13:01.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.656+0000 7f655fe00640 1 -- 172.21.15.45:0/1108999681 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f65600743d0 msgr2=0x7f65601a06a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:01.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.656+0000 7f655fe00640 1 --2- 172.21.15.45:0/1108999681 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f65600743d0 0x7f65601a06a0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f655000d950 tx=0x7f655000de10 comp rx=0 tx=0).stop 2024-04-16T17:13:01.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.656+0000 7f655fe00640 1 -- 172.21.15.45:0/1108999681 shutdown_connections 2024-04-16T17:13:01.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.656+0000 7f655fe00640 1 --2- 172.21.15.45:0/1108999681 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f65440a3da0 0x7f65440a6260 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:01.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.656+0000 7f655fe00640 1 --2- 172.21.15.45:0/1108999681 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6560074cf0 0x7f65601a0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:01.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.656+0000 7f655fe00640 1 --2- 172.21.15.45:0/1108999681 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f656010e740 0x7f656019a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:01.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.656+0000 7f655fe00640 1 --2- 172.21.15.45:0/1108999681 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f65600743d0 0x7f65601a06a0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:01.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.656+0000 7f655fe00640 1 -- 172.21.15.45:0/1108999681 >> 172.21.15.45:0/1108999681 conn(0x7f65600fd9d0 msgr2=0x7f65600fe780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:13:01.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.657+0000 7f655fe00640 1 -- 172.21.15.45:0/1108999681 shutdown_connections 2024-04-16T17:13:01.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.657+0000 7f655fe00640 1 -- 172.21.15.45:0/1108999681 wait complete. 2024-04-16T17:13:01.667 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:13:01 smithi045 ceph-mon[31676]: pgmap v191: 129 pgs: 129 active+clean; 55 MiB data, 736 MiB used, 1.0 TiB / 1.0 TiB avail; 923 KiB/s rd, 464 KiB/s wr, 87 op/s 2024-04-16T17:13:01.667 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:13:01 smithi045 ceph-mon[31676]: from='client.? 172.21.15.45:0/3576663305' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:13:01.667 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:13:01 smithi045 ceph-mon[31676]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T17:13:01.681 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-16T17:13:01.873 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.873+0000 7f14b7200640 1 -- 172.21.15.45:0/2608823430 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f14b8104cd0 msgr2=0x7f14b8108d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:01.873 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.873+0000 7f14b7200640 1 --2- 172.21.15.45:0/2608823430 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f14b8104cd0 0x7f14b8108d10 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f14a0009900 tx=0x7f14a002bc30 comp rx=0 tx=0).stop 2024-04-16T17:13:01.873 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.873+0000 7f14b7200640 1 -- 172.21.15.45:0/2608823430 shutdown_connections 2024-04-16T17:13:01.873 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.873+0000 7f14b7200640 1 --2- 172.21.15.45:0/2608823430 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f14b8104cd0 0x7f14b8108d10 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:01.873 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.873+0000 7f14b7200640 1 --2- 172.21.15.45:0/2608823430 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f14b81093f0 0x7f14b8110fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:01.873 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.873+0000 7f14b7200640 1 --2- 172.21.15.45:0/2608823430 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f14b8104320 0x7f14b8104700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:01.873 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.873+0000 7f14b7200640 1 -- 172.21.15.45:0/2608823430 >> 172.21.15.45:0/2608823430 conn(0x7f14b80ffb40 msgr2=0x7f14b8101f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:13:01.873 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.873+0000 7f14b7200640 1 -- 172.21.15.45:0/2608823430 shutdown_connections 2024-04-16T17:13:01.873 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.873+0000 7f14b7200640 1 -- 172.21.15.45:0/2608823430 wait complete. 2024-04-16T17:13:01.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.875+0000 7f14b7200640 1 Processor -- start 2024-04-16T17:13:01.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.875+0000 7f14b7200640 1 -- start start 2024-04-16T17:13:01.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.875+0000 7f14b7200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f14b8104320 0x7f14b81a0650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:01.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.875+0000 7f14b7200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f14b81093f0 0x7f14b81a0b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:01.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.875+0000 7f14b7200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f14b819a810 0x7f14b819acc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:01.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.875+0000 7f14b7200640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f14b81144b0 con 0x7f14b81093f0 2024-04-16T17:13:01.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.875+0000 7f14b7200640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f14b8114330 con 0x7f14b8104320 2024-04-16T17:13:01.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.875+0000 7f14b7200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f14b8114630 con 0x7f14b819a810 2024-04-16T17:13:01.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.876+0000 7f14b5e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f14b8104320 0x7f14b81a0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:01.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.876+0000 7f14b5e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f14b8104320 0x7f14b81a0650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:35874/0 (socket says 172.21.15.45:35874) 2024-04-16T17:13:01.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.876+0000 7f14b5e00640 1 -- 172.21.15.45:0/1807042602 learned_addr learned my addr 172.21.15.45:0/1807042602 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:13:01.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.876+0000 7f14b6800640 1 --2- 172.21.15.45:0/1807042602 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f14b819a810 0x7f14b819acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:01.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.876+0000 7f14afe00640 1 --2- 172.21.15.45:0/1807042602 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f14b81093f0 0x7f14b81a0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:01.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.876+0000 7f14b5e00640 1 -- 172.21.15.45:0/1807042602 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f14b819a810 msgr2=0x7f14b819acc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:01.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.876+0000 7f14b5e00640 1 --2- 172.21.15.45:0/1807042602 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f14b819a810 0x7f14b819acc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:01.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.876+0000 7f14b5e00640 1 -- 172.21.15.45:0/1807042602 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f14b81093f0 msgr2=0x7f14b81a0b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:01.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.876+0000 7f14b5e00640 1 --2- 172.21.15.45:0/1807042602 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f14b81093f0 0x7f14b81a0b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:01.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.876+0000 7f14b5e00640 1 -- 172.21.15.45:0/1807042602 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14b819b3a0 con 0x7f14b8104320 2024-04-16T17:13:01.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.876+0000 7f14afe00640 1 --2- 172.21.15.45:0/1807042602 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f14b81093f0 0x7f14b81a0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:13:01.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.876+0000 7f14b6800640 1 --2- 172.21.15.45:0/1807042602 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f14b819a810 0x7f14b819acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:13:01.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.877+0000 7f14b5e00640 1 --2- 172.21.15.45:0/1807042602 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f14b8104320 0x7f14b81a0650 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f14a800ea60 tx=0x7f14a800ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:13:01.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.877+0000 7f14aea00640 1 -- 172.21.15.45:0/1807042602 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14a800cde0 con 0x7f14b8104320 2024-04-16T17:13:01.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.877+0000 7f14aea00640 1 -- 172.21.15.45:0/1807042602 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f14a80035f0 con 0x7f14b8104320 2024-04-16T17:13:01.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.877+0000 7f14aea00640 1 -- 172.21.15.45:0/1807042602 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14a8010670 con 0x7f14b8104320 2024-04-16T17:13:01.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.877+0000 7f14b7200640 1 -- 172.21.15.45:0/1807042602 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14b819b690 con 0x7f14b8104320 2024-04-16T17:13:01.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.877+0000 7f14b7200640 1 -- 172.21.15.45:0/1807042602 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14b81a77a0 con 0x7f14b8104320 2024-04-16T17:13:01.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.880+0000 7f14aea00640 1 -- 172.21.15.45:0/1807042602 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f14a80037a0 con 0x7f14b8104320 2024-04-16T17:13:01.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.880+0000 7f14bfa1b740 1 Processor -- start 2024-04-16T17:13:01.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.880+0000 7f14bfa1b740 1 -- start start 2024-04-16T17:13:01.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.881+0000 7f14bfa1b740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ab342a3f10 0x55ab342a42f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:01.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.881+0000 7f14bfa1b740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ab342a4830 0x55ab342aced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:01.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.881+0000 7f14bfa1b740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ab342ad410 0x55ab342af800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:01.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.881+0000 7f14bfa1b740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55ab34209020 con 0x55ab342a4830 2024-04-16T17:13:01.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.881+0000 7f14bfa1b740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55ab341af460 con 0x55ab342ad410 2024-04-16T17:13:01.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.881+0000 7f14bfa1b740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55ab34204fe0 con 0x55ab342a3f10 2024-04-16T17:13:01.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.881+0000 7f14b5e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ab342a4830 0x55ab342aced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:01.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.881+0000 7f14afe00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ab342ad410 0x55ab342af800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:01.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.881+0000 7f14afe00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ab342ad410 0x55ab342af800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:35876/0 (socket says 172.21.15.45:35876) 2024-04-16T17:13:01.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.881+0000 7f14afe00640 1 -- 172.21.15.45:0/1076081098 learned_addr learned my addr 172.21.15.45:0/1076081098 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:13:01.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.881+0000 7f14b6800640 1 --2- 172.21.15.45:0/1076081098 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ab342a3f10 0x55ab342a42f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:01.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.882+0000 7f14afe00640 1 -- 172.21.15.45:0/1076081098 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ab342a3f10 msgr2=0x55ab342a42f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:13:01.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.882+0000 7f14afe00640 1 --2- 172.21.15.45:0/1076081098 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ab342a3f10 0x55ab342a42f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:01.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.882+0000 7f14afe00640 1 -- 172.21.15.45:0/1076081098 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ab342a4830 msgr2=0x55ab342aced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:13:01.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.882+0000 7f14afe00640 1 --2- 172.21.15.45:0/1076081098 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ab342a4830 0x55ab342aced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:01.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.882+0000 7f14afe00640 1 -- 172.21.15.45:0/1076081098 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ab341b7040 con 0x55ab342ad410 2024-04-16T17:13:01.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.882+0000 7f14b6800640 1 --2- 172.21.15.45:0/1076081098 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ab342a3f10 0x55ab342a42f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:13:01.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.882+0000 7f14aea00640 1 --2- 172.21.15.45:0/1807042602 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f149c0a3da0 0x7f149c0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:01.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.882+0000 7f14afe00640 1 --2- 172.21.15.45:0/1076081098 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ab342ad410 0x55ab342af800 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f14b806b560 tx=0x7f14a0005e80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5e548e090921529b server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:13:01.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.882+0000 7f14aea00640 1 -- 172.21.15.45:0/1807042602 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f14a80d0ee0 con 0x7f14b8104320 2024-04-16T17:13:01.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.882+0000 7f14b6800640 1 --2- 172.21.15.45:0/1807042602 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f149c0a3da0 0x7f149c0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:01.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.882+0000 7f14a7000640 1 -- 172.21.15.45:0/1076081098 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14a0044020 con 0x55ab342ad410 2024-04-16T17:13:01.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.882+0000 7f14a7000640 1 -- 172.21.15.45:0/1076081098 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f14a0004240 con 0x55ab342ad410 2024-04-16T17:13:01.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.882+0000 7f14a7000640 1 -- 172.21.15.45:0/1076081098 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14a00046c0 con 0x55ab342ad410 2024-04-16T17:13:01.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.883+0000 7f14bfa1b740 1 -- 172.21.15.45:0/1076081098 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ab342ad410 msgr2=0x55ab342af800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:01.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.883+0000 7f14b6800640 1 --2- 172.21.15.45:0/1807042602 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f149c0a3da0 0x7f149c0a6260 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f14b0005fd0 tx=0x7f14b0009500 comp rx=0 tx=0).ready entity=mgr.25411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:13:01.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.883+0000 7f14bfa1b740 1 --2- 172.21.15.45:0/1076081098 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ab342ad410 0x55ab342af800 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f14b806b560 tx=0x7f14a0005e80 comp rx=0 tx=0).stop 2024-04-16T17:13:01.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.883+0000 7f14bfa1b740 1 -- 172.21.15.45:0/1076081098 shutdown_connections 2024-04-16T17:13:01.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.883+0000 7f14bfa1b740 1 --2- 172.21.15.45:0/1076081098 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ab342ad410 0x55ab342af800 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:01.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.883+0000 7f14bfa1b740 1 --2- 172.21.15.45:0/1076081098 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ab342a4830 0x55ab342aced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:01.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.883+0000 7f14bfa1b740 1 --2- 172.21.15.45:0/1076081098 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ab342a3f10 0x55ab342a42f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:01.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.883+0000 7f14bfa1b740 1 -- 172.21.15.45:0/1076081098 >> 172.21.15.45:0/1076081098 conn(0x55ab3429e640 msgr2=0x55ab3429ea20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:13:01.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.883+0000 7f14bfa1b740 1 -- 172.21.15.45:0/1076081098 shutdown_connections 2024-04-16T17:13:01.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.884+0000 7f14bfa1b740 1 -- 172.21.15.45:0/1076081098 wait complete. 2024-04-16T17:13:01.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.884+0000 7f14bfa1b740 1 -- start start 2024-04-16T17:13:01.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.884+0000 7f14bfa1b740 1 Processor -- start 2024-04-16T17:13:01.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.885+0000 7f14bfa1b740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ab342a3f10 0x55ab342aa500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:01.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.885+0000 7f14bfa1b740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ab342a4830 0x55ab342aaa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:01.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.885+0000 7f14bfa1b740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ab342ad410 0x55ab342aaf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:01.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.885+0000 7f14bfa1b740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55ab342b01a0 con 0x55ab342ad410 2024-04-16T17:13:01.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.885+0000 7f14bfa1b740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55ab341c2830 con 0x55ab342a4830 2024-04-16T17:13:01.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.885+0000 7f14bfa1b740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55ab342b0320 con 0x55ab342a3f10 2024-04-16T17:13:01.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.885+0000 7f14bfa1b740 10 client.0 _ll_register_callbacks cb 0x55ab341ca150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:13:01.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.885+0000 7f14b5e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ab342a3f10 0x55ab342aa500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:01.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.885+0000 7f14b5e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ab342a3f10 0x55ab342aa500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:54042/0 (socket says 172.21.15.45:54042) 2024-04-16T17:13:01.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.885+0000 7f14b5e00640 1 -- 172.21.15.45:0/3163324743 learned_addr learned my addr 172.21.15.45:0/3163324743 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:13:01.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.885+0000 7f14afe00640 1 --2- 172.21.15.45:0/3163324743 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ab342a4830 0x55ab342aaa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:01.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.885+0000 7f14b6800640 1 --2- 172.21.15.45:0/3163324743 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ab342ad410 0x55ab342aaf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:01.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.885+0000 7f14b5e00640 1 -- 172.21.15.45:0/3163324743 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ab342a4830 msgr2=0x55ab342aaa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:13:01.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.885+0000 7f14b5e00640 1 --2- 172.21.15.45:0/3163324743 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ab342a4830 0x55ab342aaa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:01.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.885+0000 7f14b5e00640 1 -- 172.21.15.45:0/3163324743 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ab342ad410 msgr2=0x55ab342aaf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:13:01.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.885+0000 7f14b5e00640 1 --2- 172.21.15.45:0/3163324743 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ab342ad410 0x55ab342aaf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:01.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.885+0000 7f14b5e00640 1 -- 172.21.15.45:0/3163324743 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ab341c6920 con 0x55ab342a3f10 2024-04-16T17:13:01.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.886+0000 7f14b5e00640 1 --2- 172.21.15.45:0/3163324743 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ab342a3f10 0x55ab342aa500 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f14a800c760 tx=0x7f14a8098d70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5925f44530f98ea2 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:13:01.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.886+0000 7f149b400640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-04-16T17:13:01.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.886+0000 7f149b400640 1 -- 172.21.15.45:0/3163324743 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14a809c430 con 0x55ab342a3f10 2024-04-16T17:13:01.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.886+0000 7f149b400640 1 -- 172.21.15.45:0/3163324743 <== mon.1 v2:172.21.15.117:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f14a8003b30 con 0x55ab342a3f10 2024-04-16T17:13:01.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.886+0000 7f14bfa1b740 1 -- 172.21.15.45:0/3163324743 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f14a0004240 con 0x55ab342a3f10 2024-04-16T17:13:01.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.886+0000 7f149b400640 1 -- 172.21.15.45:0/3163324743 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f14a809d1d0 con 0x55ab342a3f10 2024-04-16T17:13:01.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.886+0000 7f14bfa1b740 10 client.27271 fetch_fsmap learned FSMap version 37 2024-04-16T17:13:01.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.886+0000 7f14bfa1b740 1 -- 172.21.15.45:0/3163324743 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55ab341947f0 con 0x55ab342a3f10 2024-04-16T17:13:01.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.886+0000 7f149b400640 1 -- 172.21.15.45:0/3163324743 <== mon.1 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f14a0004240 con 0x55ab342a3f10 2024-04-16T17:13:01.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.887+0000 7f149b400640 1 -- 172.21.15.45:0/3163324743 <== mon.1 v2:172.21.15.117:3300/0 5 ==== fsmap(e 37) ==== 8358+0+0 (secure 0 0 0) 0x7f14a80ce500 con 0x55ab342a3f10 2024-04-16T17:13:01.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.887+0000 7f14bfa1b740 10 client.27271 fetch_fsmap finished waiting for FSMap version 37 2024-04-16T17:13:01.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.887+0000 7f14bfa1b740 10 client.27271 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-04-16T17:13:01.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.887+0000 7f14bfa1b740 20 client.27271 populate_metadata read hostname 'smithi045' 2024-04-16T17:13:01.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.887+0000 7f14bfa1b740 1 --2- 172.21.15.45:0/3163324743 >> [v2:172.21.15.45:6840/3755310822,v1:172.21.15.45:6841/3755310822] conn(0x55ab342bd8a0 0x55ab342bfcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:01.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.887+0000 7f14bfa1b740 4 client.27271 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:13:01.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.887+0000 7f14bfa1b740 1 -- 172.21.15.45:0/3163324743 --> [v2:172.21.15.45:6840/3755310822,v1:172.21.15.45:6841/3755310822] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ab34216a40 con 0x55ab342bd8a0 2024-04-16T17:13:01.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.887+0000 7f14afe00640 1 --2- 172.21.15.45:0/3163324743 >> [v2:172.21.15.45:6840/3755310822,v1:172.21.15.45:6841/3755310822] conn(0x55ab342bd8a0 0x55ab342bfcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:01.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.888+0000 7f14afe00640 1 --2- 172.21.15.45:0/3163324743 >> [v2:172.21.15.45:6840/3755310822,v1:172.21.15.45:6841/3755310822] conn(0x55ab342bd8a0 0x55ab342bfcd0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7e3670f8d78ddcaf server_cookie=ffd80ee284164f30 in_seq=0 out_seq=0 2024-04-16T17:13:01.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.888+0000 7f149b400640 10 client.27271 ms_handle_connect on v2:172.21.15.45:6840/3755310822 2024-04-16T17:13:01.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.888+0000 7f149b400640 1 -- 172.21.15.45:0/3163324743 <== mds.1 v2:172.21.15.45:6840/3755310822 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ab34216a40 con 0x55ab342bd8a0 2024-04-16T17:13:01.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.888+0000 7f149b400640 10 client.27271 handle_command_reply: tid=0 2024-04-16T17:13:01.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.888+0000 7f14bfa1b740 1 client.27271 shutdown 2024-04-16T17:13:01.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.889+0000 7f14bfa1b740 1 -- 172.21.15.45:0/3163324743 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ab342a3f10 msgr2=0x55ab342aa500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:01.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.889+0000 7f14bfa1b740 1 --2- 172.21.15.45:0/3163324743 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ab342a3f10 0x55ab342aa500 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f14a800c760 tx=0x7f14a8098d70 comp rx=0 tx=0).stop 2024-04-16T17:13:01.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.889+0000 7f14bfa1b740 1 -- 172.21.15.45:0/3163324743 shutdown_connections 2024-04-16T17:13:01.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.889+0000 7f14bfa1b740 1 --2- 172.21.15.45:0/3163324743 >> [v2:172.21.15.45:6840/3755310822,v1:172.21.15.45:6841/3755310822] conn(0x55ab342bd8a0 0x55ab342bfcd0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:01.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.889+0000 7f14bfa1b740 1 --2- 172.21.15.45:0/3163324743 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ab342ad410 0x55ab342aaf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:01.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.889+0000 7f14bfa1b740 1 --2- 172.21.15.45:0/3163324743 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ab342a4830 0x55ab342aaa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:01.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.889+0000 7f14bfa1b740 1 --2- 172.21.15.45:0/3163324743 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ab342a3f10 0x55ab342aa500 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:01.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.889+0000 7f14bfa1b740 1 -- 172.21.15.45:0/3163324743 >> 172.21.15.45:0/3163324743 conn(0x55ab3429e640 msgr2=0x55ab342a0390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:13:01.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.889+0000 7f149b400640 0 client.27271 ms_handle_reset on v2:172.21.15.45:6840/3755310822 2024-04-16T17:13:01.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.889+0000 7f14bfa1b740 1 -- 172.21.15.45:0/3163324743 shutdown_connections 2024-04-16T17:13:01.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.889+0000 7f14bfa1b740 1 -- 172.21.15.45:0/3163324743 wait complete. 2024-04-16T17:13:01.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.889+0000 7f14bfa1b740 20 client.27271 trim_cache size 0 max 16384 2024-04-16T17:13:01.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.903+0000 7f14bfa1b740 1 Processor -- start 2024-04-16T17:13:01.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.903+0000 7f14bfa1b740 1 -- start start 2024-04-16T17:13:01.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.904+0000 7f14bfa1b740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ab342a3f10 0x55ab342acc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:01.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.904+0000 7f14bfa1b740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ab342a4830 0x55ab342b9060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:01.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.904+0000 7f14bfa1b740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ab342ad410 0x55ab3429d580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:01.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.904+0000 7f14bfa1b740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f14b81b3ed0 con 0x55ab342a3f10 2024-04-16T17:13:01.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.904+0000 7f14bfa1b740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f14b81b1710 con 0x55ab342ad410 2024-04-16T17:13:01.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.904+0000 7f14bfa1b740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f14b81af540 con 0x55ab342a4830 2024-04-16T17:13:01.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.904+0000 7f14b5e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ab342a3f10 0x55ab342acc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:01.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.904+0000 7f14b6800640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ab342ad410 0x55ab3429d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:01.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.904+0000 7f14b5e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ab342a3f10 0x55ab342acc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:46784/0 (socket says 172.21.15.45:46784) 2024-04-16T17:13:01.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.904+0000 7f14b5e00640 1 -- 172.21.15.45:0/232795278 learned_addr learned my addr 172.21.15.45:0/232795278 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:13:01.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.904+0000 7f14afe00640 1 --2- 172.21.15.45:0/232795278 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ab342a4830 0x55ab342b9060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:01.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.905+0000 7f14b5e00640 1 -- 172.21.15.45:0/232795278 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ab342a4830 msgr2=0x55ab342b9060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:13:01.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.905+0000 7f14b5e00640 1 --2- 172.21.15.45:0/232795278 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ab342a4830 0x55ab342b9060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:01.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.905+0000 7f14b5e00640 1 -- 172.21.15.45:0/232795278 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ab342ad410 msgr2=0x55ab3429d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:13:01.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.905+0000 7f14b5e00640 1 --2- 172.21.15.45:0/232795278 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ab342ad410 0x55ab3429d580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:01.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.905+0000 7f14b5e00640 1 -- 172.21.15.45:0/232795278 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ab33f1b3f0 con 0x55ab342a3f10 2024-04-16T17:13:01.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.905+0000 7f14afe00640 1 --2- 172.21.15.45:0/232795278 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ab342a4830 0x55ab342b9060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:13:01.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.905+0000 7f14b6800640 1 --2- 172.21.15.45:0/232795278 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ab342ad410 0x55ab3429d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:13:01.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.905+0000 7f14b5e00640 1 --2- 172.21.15.45:0/232795278 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ab342a3f10 0x55ab342acc30 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f14a80987b0 tx=0x7f14a800ef90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5dd1c34b75d14a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:13:01.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.905+0000 7f14a6600640 1 -- 172.21.15.45:0/232795278 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14a809c430 con 0x55ab342a3f10 2024-04-16T17:13:01.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.905+0000 7f14a6600640 1 -- 172.21.15.45:0/232795278 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f14a809cad0 con 0x55ab342a3f10 2024-04-16T17:13:01.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.905+0000 7f14a6600640 1 -- 172.21.15.45:0/232795278 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14a809c8a0 con 0x55ab342a3f10 2024-04-16T17:13:01.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.906+0000 7f14bfa1b740 1 -- 172.21.15.45:0/232795278 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ab342a3f10 msgr2=0x55ab342acc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:01.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.906+0000 7f14bfa1b740 1 --2- 172.21.15.45:0/232795278 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ab342a3f10 0x55ab342acc30 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f14a80987b0 tx=0x7f14a800ef90 comp rx=0 tx=0).stop 2024-04-16T17:13:01.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.906+0000 7f14bfa1b740 1 -- 172.21.15.45:0/232795278 shutdown_connections 2024-04-16T17:13:01.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.906+0000 7f14bfa1b740 1 --2- 172.21.15.45:0/232795278 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ab342a4830 0x55ab342b9060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:01.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.906+0000 7f14bfa1b740 1 --2- 172.21.15.45:0/232795278 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ab342ad410 0x55ab3429d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:01.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.906+0000 7f14bfa1b740 1 --2- 172.21.15.45:0/232795278 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ab342a3f10 0x55ab342acc30 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:01.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.906+0000 7f14bfa1b740 1 -- 172.21.15.45:0/232795278 >> 172.21.15.45:0/232795278 conn(0x55ab3429e640 msgr2=0x55ab3429ce70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:13:01.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.906+0000 7f14bfa1b740 1 -- 172.21.15.45:0/232795278 shutdown_connections 2024-04-16T17:13:01.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.907+0000 7f14bfa1b740 1 -- 172.21.15.45:0/232795278 wait complete. 2024-04-16T17:13:01.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.907+0000 7f14bfa1b740 1 -- start start 2024-04-16T17:13:01.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.907+0000 7f14bfa1b740 1 Processor -- start 2024-04-16T17:13:01.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.907+0000 7f14bfa1b740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ab342a3f10 0x55ab342a73e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:01.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.907+0000 7f14bfa1b740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ab342a4830 0x55ab342a7920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:01.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.907+0000 7f14bfa1b740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ab342ad410 0x55ab342a7e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:01.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.907+0000 7f14bfa1b740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f14b81afbd0 con 0x55ab342a3f10 2024-04-16T17:13:01.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.907+0000 7f14bfa1b740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f14b81b1040 con 0x55ab342a4830 2024-04-16T17:13:01.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.908+0000 7f14bfa1b740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f14b81b02a0 con 0x55ab342ad410 2024-04-16T17:13:01.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.908+0000 7f14bfa1b740 10 client.0 _ll_register_callbacks cb 0x7f14a80d4680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:13:01.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.908+0000 7f14b5e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ab342a3f10 0x55ab342a73e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:01.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.908+0000 7f14afe00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ab342a4830 0x55ab342a7920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:01.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.908+0000 7f14b5e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ab342a3f10 0x55ab342a73e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:46786/0 (socket says 172.21.15.45:46786) 2024-04-16T17:13:01.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.908+0000 7f14afe00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ab342a4830 0x55ab342a7920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:35902/0 (socket says 172.21.15.45:35902) 2024-04-16T17:13:01.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.908+0000 7f14b5e00640 1 -- 172.21.15.45:0/4179120247 learned_addr learned my addr 172.21.15.45:0/4179120247 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:13:01.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.908+0000 7f14b6800640 1 --2- 172.21.15.45:0/4179120247 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ab342ad410 0x55ab342a7e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:01.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.908+0000 7f14afe00640 1 -- 172.21.15.45:0/4179120247 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ab342ad410 msgr2=0x55ab342a7e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:13:01.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.908+0000 7f14afe00640 1 --2- 172.21.15.45:0/4179120247 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ab342ad410 0x55ab342a7e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:01.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.908+0000 7f14afe00640 1 -- 172.21.15.45:0/4179120247 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ab342a3f10 msgr2=0x55ab342a73e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:13:01.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.908+0000 7f14afe00640 1 --2- 172.21.15.45:0/4179120247 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ab342a3f10 0x55ab342a73e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:01.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.908+0000 7f14afe00640 1 -- 172.21.15.45:0/4179120247 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ab33fe57d0 con 0x55ab342a4830 2024-04-16T17:13:01.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.908+0000 7f14b5e00640 1 --2- 172.21.15.45:0/4179120247 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ab342a3f10 0x55ab342a73e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:13:01.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.909+0000 7f14afe00640 1 --2- 172.21.15.45:0/4179120247 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ab342a4830 0x55ab342a7920 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f14a002c240 tx=0x7f14a00052a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=47a4dc309255c79b server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:13:01.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.909+0000 7f149b400640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-04-16T17:13:01.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.909+0000 7f149b400640 1 -- 172.21.15.45:0/4179120247 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14a0044040 con 0x55ab342a4830 2024-04-16T17:13:01.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.909+0000 7f14bfa1b740 1 -- 172.21.15.45:0/4179120247 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f14a809cad0 con 0x55ab342a4830 2024-04-16T17:13:01.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.909+0000 7f149b400640 1 -- 172.21.15.45:0/4179120247 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f14a0053bd0 con 0x55ab342a4830 2024-04-16T17:13:01.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.909+0000 7f149b400640 1 -- 172.21.15.45:0/4179120247 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f14a005b050 con 0x55ab342a4830 2024-04-16T17:13:01.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.909+0000 7f14bfa1b740 10 client.27338 fetch_fsmap learned FSMap version 37 2024-04-16T17:13:01.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.909+0000 7f14bfa1b740 1 -- 172.21.15.45:0/4179120247 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55ab343241b0 con 0x55ab342a4830 2024-04-16T17:13:01.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.909+0000 7f149b400640 1 -- 172.21.15.45:0/4179120247 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f14a809cad0 con 0x55ab342a4830 2024-04-16T17:13:01.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.910+0000 7f149b400640 1 -- 172.21.15.45:0/4179120247 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 37) ==== 8358+0+0 (secure 0 0 0) 0x7f14a0040470 con 0x55ab342a4830 2024-04-16T17:13:01.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.910+0000 7f14bfa1b740 10 client.27338 fetch_fsmap finished waiting for FSMap version 37 2024-04-16T17:13:01.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.910+0000 7f14bfa1b740 10 client.27338 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-04-16T17:13:01.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.910+0000 7f14bfa1b740 20 client.27338 populate_metadata read hostname 'smithi045' 2024-04-16T17:13:01.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.910+0000 7f14bfa1b740 1 --2- 172.21.15.45:0/4179120247 >> [v2:172.21.15.45:6840/3755310822,v1:172.21.15.45:6841/3755310822] conn(0x55ab34324540 0x55ab34326960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:01.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.910+0000 7f14bfa1b740 4 client.27338 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T17:13:01.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.910+0000 7f14bfa1b740 1 -- 172.21.15.45:0/4179120247 --> [v2:172.21.15.45:6840/3755310822,v1:172.21.15.45:6841/3755310822] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ab342108b0 con 0x55ab34324540 2024-04-16T17:13:01.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.910+0000 7f14b5e00640 1 --2- 172.21.15.45:0/4179120247 >> [v2:172.21.15.45:6840/3755310822,v1:172.21.15.45:6841/3755310822] conn(0x55ab34324540 0x55ab34326960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:01.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.911+0000 7f14b5e00640 1 --2- 172.21.15.45:0/4179120247 >> [v2:172.21.15.45:6840/3755310822,v1:172.21.15.45:6841/3755310822] conn(0x55ab34324540 0x55ab34326960 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a3fc67760fe9ff82 server_cookie=e1ad849e87092af1 in_seq=0 out_seq=0 2024-04-16T17:13:01.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.911+0000 7f149b400640 10 client.27338 ms_handle_connect on v2:172.21.15.45:6840/3755310822 2024-04-16T17:13:01.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.911+0000 7f149b400640 1 -- 172.21.15.45:0/4179120247 <== mds.1 v2:172.21.15.45:6840/3755310822 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ab342108b0 con 0x55ab34324540 2024-04-16T17:13:01.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.911+0000 7f149b400640 10 client.27338 handle_command_reply: tid=0 2024-04-16T17:13:01.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.911+0000 7f14bfa1b740 1 client.27338 shutdown 2024-04-16T17:13:01.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.911+0000 7f14bfa1b740 1 -- 172.21.15.45:0/4179120247 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ab342a4830 msgr2=0x55ab342a7920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:01.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.911+0000 7f14bfa1b740 1 --2- 172.21.15.45:0/4179120247 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ab342a4830 0x55ab342a7920 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f14a002c240 tx=0x7f14a00052a0 comp rx=0 tx=0).stop 2024-04-16T17:13:01.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.911+0000 7f14bfa1b740 1 -- 172.21.15.45:0/4179120247 shutdown_connections 2024-04-16T17:13:01.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.911+0000 7f14bfa1b740 1 --2- 172.21.15.45:0/4179120247 >> [v2:172.21.15.45:6840/3755310822,v1:172.21.15.45:6841/3755310822] conn(0x55ab34324540 0x55ab34326960 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:01.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.911+0000 7f14bfa1b740 1 --2- 172.21.15.45:0/4179120247 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ab342ad410 0x55ab342a7e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:01.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.911+0000 7f14bfa1b740 1 --2- 172.21.15.45:0/4179120247 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ab342a4830 0x55ab342a7920 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:01.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.911+0000 7f14bfa1b740 1 --2- 172.21.15.45:0/4179120247 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ab342a3f10 0x55ab342a73e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:01.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.911+0000 7f149b400640 0 client.27338 ms_handle_reset on v2:172.21.15.45:6840/3755310822 2024-04-16T17:13:01.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.911+0000 7f14bfa1b740 1 -- 172.21.15.45:0/4179120247 >> 172.21.15.45:0/4179120247 conn(0x55ab3429e640 msgr2=0x55ab3429d700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:13:01.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.912+0000 7f14bfa1b740 1 -- 172.21.15.45:0/4179120247 shutdown_connections 2024-04-16T17:13:01.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.912+0000 7f14bfa1b740 1 -- 172.21.15.45:0/4179120247 wait complete. 2024-04-16T17:13:01.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.912+0000 7f14bfa1b740 20 client.27338 trim_cache size 0 max 16384 2024-04-16T17:13:01.915 INFO:teuthology.orchestra.run.smithi045.stdout:[] 2024-04-16T17:13:01.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.912+0000 7f14b7200640 1 -- 172.21.15.45:0/1807042602 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f149c0a3da0 msgr2=0x7f149c0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:01.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.912+0000 7f14b7200640 1 --2- 172.21.15.45:0/1807042602 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f149c0a3da0 0x7f149c0a6260 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f14b0005fd0 tx=0x7f14b0009500 comp rx=0 tx=0).stop 2024-04-16T17:13:01.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.912+0000 7f14b7200640 1 -- 172.21.15.45:0/1807042602 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f14b8104320 msgr2=0x7f14b81a0650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:01.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.912+0000 7f14b7200640 1 --2- 172.21.15.45:0/1807042602 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f14b8104320 0x7f14b81a0650 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f14a800ea60 tx=0x7f14a800ef20 comp rx=0 tx=0).stop 2024-04-16T17:13:01.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.913+0000 7f14b7200640 1 -- 172.21.15.45:0/1807042602 shutdown_connections 2024-04-16T17:13:01.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.913+0000 7f14b7200640 1 --2- 172.21.15.45:0/1807042602 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f149c0a3da0 0x7f149c0a6260 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:01.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.913+0000 7f14b7200640 1 --2- 172.21.15.45:0/1807042602 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f14b819a810 0x7f14b819acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:01.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.913+0000 7f14b7200640 1 --2- 172.21.15.45:0/1807042602 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f14b81093f0 0x7f14b81a0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:01.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.913+0000 7f14b7200640 1 --2- 172.21.15.45:0/1807042602 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f14b8104320 0x7f14b81a0650 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:01.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.913+0000 7f14b7200640 1 -- 172.21.15.45:0/1807042602 >> 172.21.15.45:0/1807042602 conn(0x7f14b80ffb40 msgr2=0x7f14b8100f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:13:01.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.913+0000 7f14b7200640 1 -- 172.21.15.45:0/1807042602 shutdown_connections 2024-04-16T17:13:01.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:01.913+0000 7f14b7200640 1 -- 172.21.15.45:0/1807042602 wait complete. 2024-04-16T17:13:01.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:13:01 smithi104 ceph-mon[36457]: pgmap v191: 129 pgs: 129 active+clean; 55 MiB data, 736 MiB used, 1.0 TiB / 1.0 TiB avail; 923 KiB/s rd, 464 KiB/s wr, 87 op/s 2024-04-16T17:13:01.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:13:01 smithi104 ceph-mon[36457]: from='client.? 172.21.15.45:0/3576663305' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:13:01.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:13:01 smithi104 ceph-mon[36457]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T17:13:01.929 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-16T17:13:02.054 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:13:01 smithi117 ceph-mon[36730]: pgmap v191: 129 pgs: 129 active+clean; 55 MiB data, 736 MiB used, 1.0 TiB / 1.0 TiB avail; 923 KiB/s rd, 464 KiB/s wr, 87 op/s 2024-04-16T17:13:02.054 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:13:01 smithi117 ceph-mon[36730]: from='client.? 172.21.15.45:0/3576663305' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:13:02.054 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:13:01 smithi117 ceph-mon[36730]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T17:13:02.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:02.130+0000 7faf29000640 1 -- 172.21.15.45:0/505953766 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7faf24102110 msgr2=0x7faf241024f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:02.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:02.130+0000 7faf29000640 1 --2- 172.21.15.45:0/505953766 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7faf24102110 0x7faf241024f0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7faf14009900 tx=0x7faf1402bc30 comp rx=0 tx=0).stop 2024-04-16T17:13:02.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:02.130+0000 7faf29000640 1 -- 172.21.15.45:0/505953766 shutdown_connections 2024-04-16T17:13:02.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:02.130+0000 7faf29000640 1 --2- 172.21.15.45:0/505953766 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faf241071e0 0x7faf2410edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:02.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:02.130+0000 7faf29000640 1 --2- 172.21.15.45:0/505953766 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faf24102ac0 0x7faf24106b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:02.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:02.130+0000 7faf29000640 1 --2- 172.21.15.45:0/505953766 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7faf24102110 0x7faf241024f0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:02.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:02.130+0000 7faf29000640 1 -- 172.21.15.45:0/505953766 >> 172.21.15.45:0/505953766 conn(0x7faf240fd9d0 msgr2=0x7faf240ffdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:13:02.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:02.130+0000 7faf29000640 1 -- 172.21.15.45:0/505953766 shutdown_connections 2024-04-16T17:13:02.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:02.130+0000 7faf29000640 1 -- 172.21.15.45:0/505953766 wait complete. 2024-04-16T17:13:02.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:02.131+0000 7faf29000640 1 Processor -- start 2024-04-16T17:13:02.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:02.132+0000 7faf29000640 1 -- start start 2024-04-16T17:13:02.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:02.132+0000 7faf29000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faf24102110 0x7faf241a0640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:02.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:02.132+0000 7faf29000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faf241071e0 0x7faf241a0b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:02.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:02.132+0000 7faf29000640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7faf2419a7b0 0x7faf2419ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:02.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:02.132+0000 7faf29000640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7faf241122c0 con 0x7faf24102110 2024-04-16T17:13:02.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:02.132+0000 7faf29000640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7faf24112140 con 0x7faf2419a7b0 2024-04-16T17:13:02.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:02.132+0000 7faf29000640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7faf24112440 con 0x7faf241071e0 2024-04-16T17:13:02.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:02.132+0000 7faf23400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faf24102110 0x7faf241a0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:02.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:02.132+0000 7faf23400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faf24102110 0x7faf241a0640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:46806/0 (socket says 172.21.15.45:46806) 2024-04-16T17:13:02.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:02.132+0000 7faf23e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7faf2419a7b0 0x7faf2419ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:02.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:02.132+0000 7faf1be00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faf241071e0 0x7faf241a0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:02.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:02.132+0000 7faf23400640 1 -- 172.21.15.45:0/1328465868 learned_addr learned my addr 172.21.15.45:0/1328465868 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:13:02.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:02.133+0000 7faf23400640 1 -- 172.21.15.45:0/1328465868 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faf241071e0 msgr2=0x7faf241a0b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:02.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:02.133+0000 7faf23400640 1 --2- 172.21.15.45:0/1328465868 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faf241071e0 0x7faf241a0b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:02.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:02.133+0000 7faf23400640 1 -- 172.21.15.45:0/1328465868 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7faf2419a7b0 msgr2=0x7faf2419ac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:02.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:02.133+0000 7faf23400640 1 --2- 172.21.15.45:0/1328465868 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7faf2419a7b0 0x7faf2419ac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:02.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:02.133+0000 7faf23400640 1 -- 172.21.15.45:0/1328465868 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf2419b340 con 0x7faf24102110 2024-04-16T17:13:02.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:02.133+0000 7faf23400640 1 --2- 172.21.15.45:0/1328465868 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faf24102110 0x7faf241a0640 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7faf1402c240 tx=0x7faf140042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:13:02.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:02.133+0000 7faf20c00640 1 -- 172.21.15.45:0/1328465868 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf14043020 con 0x7faf24102110 2024-04-16T17:13:02.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:02.133+0000 7faf20c00640 1 -- 172.21.15.45:0/1328465868 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faf140054e0 con 0x7faf24102110 2024-04-16T17:13:02.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:02.133+0000 7faf20c00640 1 -- 172.21.15.45:0/1328465868 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf14003ab0 con 0x7faf24102110 2024-04-16T17:13:02.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:02.133+0000 7faf29000640 1 -- 172.21.15.45:0/1328465868 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf2419b5d0 con 0x7faf24102110 2024-04-16T17:13:02.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:02.134+0000 7faf29000640 1 -- 172.21.15.45:0/1328465868 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf241a7760 con 0x7faf24102110 2024-04-16T17:13:02.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:02.137+0000 7faf20c00640 1 -- 172.21.15.45:0/1328465868 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7faf14046020 con 0x7faf24102110 2024-04-16T17:13:02.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:02.137+0000 7faf2d26f740 1 Processor -- start 2024-04-16T17:13:02.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:02.137+0000 7faf2d26f740 1 -- start start 2024-04-16T17:13:02.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:02.137+0000 7faf2d26f740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5566965a0f10 0x5566965a12f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:02.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:02.137+0000 7faf2d26f740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5566965a1830 0x5566965a9ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:02.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:02.137+0000 7faf2d26f740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5566965aa410 0x5566965ac800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:02.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:02.137+0000 7faf2d26f740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x556696506020 con 0x5566965a0f10 2024-04-16T17:13:02.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:02.137+0000 7faf2d26f740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5566964ac460 con 0x5566965aa410 2024-04-16T17:13:02.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:02.137+0000 7faf2d26f740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x556696501fe0 con 0x5566965a1830 2024-04-16T17:13:02.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:02.138+0000 7faf1be00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5566965aa410 0x5566965ac800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:02.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:02.138+0000 7faf23e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5566965a0f10 0x5566965a12f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:02.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:02.138+0000 7faf23400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5566965a1830 0x5566965a9ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:02.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:02.138+0000 7faf1be00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5566965aa410 0x5566965ac800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:35934/0 (socket says 172.21.15.45:35934) 2024-04-16T17:13:02.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:02.138+0000 7faf1be00640 1 -- 172.21.15.45:0/3792568995 learned_addr learned my addr 172.21.15.45:0/3792568995 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:13:02.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:02.138+0000 7faf1be00640 1 -- 172.21.15.45:0/3792568995 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5566965a1830 msgr2=0x5566965a9ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:13:02.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:02.138+0000 7faf1be00640 1 --2- 172.21.15.45:0/3792568995 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5566965a1830 0x5566965a9ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:02.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:02.138+0000 7faf1be00640 1 -- 172.21.15.45:0/3792568995 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5566965a0f10 msgr2=0x5566965a12f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:13:02.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:02.138+0000 7faf1be00640 1 --2- 172.21.15.45:0/3792568995 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5566965a0f10 0x5566965a12f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:02.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:02.138+0000 7faf1be00640 1 -- 172.21.15.45:0/3792568995 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5566964b4040 con 0x5566965aa410 2024-04-16T17:13:02.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:02.138+0000 7faf20c00640 1 --2- 172.21.15.45:0/1328465868 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7faf080a3d20 0x7faf080a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:02.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:02.138+0000 7faf1be00640 1 --2- 172.21.15.45:0/3792568995 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5566965aa410 0x5566965ac800 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7faf0c009c20 tx=0x7faf0c011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=abd1410abec7e9f7 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:13:02.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:02.139+0000 7faf20c00640 1 -- 172.21.15.45:0/1328465868 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7faf140f2900 con 0x7faf24102110 2024-04-16T17:13:02.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:02.139+0000 7faf18c00640 1 -- 172.21.15.45:0/3792568995 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf0c004230 con 0x5566965aa410 2024-04-16T17:13:02.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:02.139+0000 7faf18c00640 1 -- 172.21.15.45:0/3792568995 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faf0c0043d0 con 0x5566965aa410 2024-04-16T17:13:02.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:02.139+0000 7faf18c00640 1 -- 172.21.15.45:0/3792568995 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf0c004dd0 con 0x5566965aa410 2024-04-16T17:13:02.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:02.139+0000 7faf23e00640 1 --2- 172.21.15.45:0/1328465868 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7faf080a3d20 0x7faf080a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:02.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:02.140+0000 7faf2d26f740 1 -- 172.21.15.45:0/3792568995 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5566965aa410 msgr2=0x5566965ac800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:02.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:02.140+0000 7faf23e00640 1 --2- 172.21.15.45:0/1328465868 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7faf080a3d20 0x7faf080a61e0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7faf1c005fd0 tx=0x7faf1c009500 comp rx=0 tx=0).ready entity=mgr.25411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:13:02.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:02.140+0000 7faf2d26f740 1 --2- 172.21.15.45:0/3792568995 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5566965aa410 0x5566965ac800 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7faf0c009c20 tx=0x7faf0c011810 comp rx=0 tx=0).stop 2024-04-16T17:13:02.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:02.140+0000 7faf2d26f740 1 -- 172.21.15.45:0/3792568995 shutdown_connections 2024-04-16T17:13:02.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:02.140+0000 7faf2d26f740 1 --2- 172.21.15.45:0/3792568995 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5566965a1830 0x5566965a9ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:02.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:02.140+0000 7faf2d26f740 1 --2- 172.21.15.45:0/3792568995 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5566965aa410 0x5566965ac800 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:02.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:02.140+0000 7faf2d26f740 1 --2- 172.21.15.45:0/3792568995 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5566965a0f10 0x5566965a12f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:02.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:02.140+0000 7faf2d26f740 1 -- 172.21.15.45:0/3792568995 >> 172.21.15.45:0/3792568995 conn(0x55669659b640 msgr2=0x55669659ba20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:13:02.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:02.140+0000 7faf2d26f740 1 -- 172.21.15.45:0/3792568995 shutdown_connections 2024-04-16T17:13:02.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:02.140+0000 7faf2d26f740 1 -- 172.21.15.45:0/3792568995 wait complete. 2024-04-16T17:13:02.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:02.140+0000 7faf2d26f740 1 -- start start 2024-04-16T17:13:02.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:02.141+0000 7faf2d26f740 1 Processor -- start 2024-04-16T17:13:02.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:02.141+0000 7faf2d26f740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5566965a0f10 0x5566965a7410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:02.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:02.141+0000 7faf2d26f740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5566965a1830 0x5566965a7950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:02.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:02.141+0000 7faf2d26f740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5566965aa410 0x5566965a7e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:02.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:02.141+0000 7faf2d26f740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5566965ad200 con 0x5566965a0f10 2024-04-16T17:13:02.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:02.141+0000 7faf2d26f740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5566964bf830 con 0x5566965a1830 2024-04-16T17:13:02.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:02.141+0000 7faf2d26f740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5566965ad380 con 0x5566965aa410 2024-04-16T17:13:02.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:02.141+0000 7faf2d26f740 10 client.0 _ll_register_callbacks cb 0x5566964c7150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:13:02.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:02.141+0000 7faf23e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5566965aa410 0x5566965a7e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:02.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:02.141+0000 7faf1be00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5566965a1830 0x5566965a7950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:02.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:02.141+0000 7faf23e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5566965aa410 0x5566965a7e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:54084/0 (socket says 172.21.15.45:54084) 2024-04-16T17:13:02.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:02.141+0000 7faf1be00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5566965a1830 0x5566965a7950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:35940/0 (socket says 172.21.15.45:35940) 2024-04-16T17:13:02.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:02.141+0000 7faf23e00640 1 -- 172.21.15.45:0/1594818259 learned_addr learned my addr 172.21.15.45:0/1594818259 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:13:02.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:02.141+0000 7faf23400640 1 --2- 172.21.15.45:0/1594818259 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5566965a0f10 0x5566965a7410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:02.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:02.142+0000 7faf23e00640 1 -- 172.21.15.45:0/1594818259 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5566965a1830 msgr2=0x5566965a7950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:13:02.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:02.142+0000 7faf23e00640 1 --2- 172.21.15.45:0/1594818259 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5566965a1830 0x5566965a7950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:02.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:02.142+0000 7faf23e00640 1 -- 172.21.15.45:0/1594818259 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5566965a0f10 msgr2=0x5566965a7410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:13:02.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:02.142+0000 7faf23e00640 1 --2- 172.21.15.45:0/1594818259 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5566965a0f10 0x5566965a7410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:02.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:02.142+0000 7faf23e00640 1 -- 172.21.15.45:0/1594818259 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5566964c3920 con 0x5566965aa410 2024-04-16T17:13:02.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:02.142+0000 7faf23400640 1 --2- 172.21.15.45:0/1594818259 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5566965a0f10 0x5566965a7410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:13:02.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:02.142+0000 7faf23e00640 1 --2- 172.21.15.45:0/1594818259 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5566965aa410 0x5566965a7e90 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7faf1c009ac0 tx=0x7faf1c004ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=260f587c5ab7b22e server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:13:02.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:02.142+0000 7faf11600640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-04-16T17:13:02.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:02.142+0000 7faf11600640 1 -- 172.21.15.45:0/1594818259 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf1c02e060 con 0x5566965aa410 2024-04-16T17:13:02.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:02.142+0000 7faf2d26f740 1 -- 172.21.15.45:0/1594818259 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faf0c0043d0 con 0x5566965aa410 2024-04-16T17:13:02.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:02.143+0000 7faf11600640 1 -- 172.21.15.45:0/1594818259 <== mon.1 v2:172.21.15.117:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7faf1c018030 con 0x5566965aa410 2024-04-16T17:13:02.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:02.143+0000 7faf11600640 1 -- 172.21.15.45:0/1594818259 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7faf1c035050 con 0x5566965aa410 2024-04-16T17:13:02.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:02.143+0000 7faf2d26f740 10 client.27289 fetch_fsmap learned FSMap version 37 2024-04-16T17:13:02.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:02.143+0000 7faf2d26f740 1 -- 172.21.15.45:0/1594818259 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x5566964917f0 con 0x5566965aa410 2024-04-16T17:13:02.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:02.143+0000 7faf11600640 1 -- 172.21.15.45:0/1594818259 <== mon.1 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faf0c0043d0 con 0x5566965aa410 2024-04-16T17:13:02.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:02.143+0000 7faf11600640 1 -- 172.21.15.45:0/1594818259 <== mon.1 v2:172.21.15.117:3300/0 5 ==== fsmap(e 37) ==== 8358+0+0 (secure 0 0 0) 0x7faf1c01c740 con 0x5566965aa410 2024-04-16T17:13:02.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:02.143+0000 7faf2d26f740 10 client.27289 fetch_fsmap finished waiting for FSMap version 37 2024-04-16T17:13:02.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:02.143+0000 7faf2d26f740 10 client.27289 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:13:02.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:02.143+0000 7faf2d26f740 20 client.27289 populate_metadata read hostname 'smithi045' 2024-04-16T17:13:02.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:02.143+0000 7faf2d26f740 1 --2- 172.21.15.45:0/1594818259 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x5566965ba750 0x5566965bcba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:02.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:02.143+0000 7faf2d26f740 4 client.27289 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:13:02.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:02.143+0000 7faf2d26f740 1 -- 172.21.15.45:0/1594818259 --> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556696513a40 con 0x5566965ba750 2024-04-16T17:13:02.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:02.144+0000 7faf23400640 1 --2- 172.21.15.45:0/1594818259 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x5566965ba750 0x5566965bcba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:02.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:02.144+0000 7faf23400640 1 --2- 172.21.15.45:0/1594818259 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x5566965ba750 0x5566965bcba0 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6f27434a38ca0ba6 server_cookie=2713a0fb570da4ef in_seq=0 out_seq=0 2024-04-16T17:13:02.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:02.144+0000 7faf11600640 10 client.27289 ms_handle_connect on v2:172.21.15.104:6838/3660070592 2024-04-16T17:13:02.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:02.145+0000 7faf11600640 1 -- 172.21.15.45:0/1594818259 <== mds.0 v2:172.21.15.104:6838/3660070592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556696513a40 con 0x5566965ba750 2024-04-16T17:13:02.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:02.145+0000 7faf11600640 10 client.27289 handle_command_reply: tid=0 2024-04-16T17:13:02.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:02.145+0000 7faf2d26f740 1 client.27289 shutdown 2024-04-16T17:13:02.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:02.145+0000 7faf2d26f740 1 -- 172.21.15.45:0/1594818259 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5566965aa410 msgr2=0x5566965a7e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:02.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:02.145+0000 7faf2d26f740 1 --2- 172.21.15.45:0/1594818259 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5566965aa410 0x5566965a7e90 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7faf1c009ac0 tx=0x7faf1c004ab0 comp rx=0 tx=0).stop 2024-04-16T17:13:02.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:02.145+0000 7faf2d26f740 1 -- 172.21.15.45:0/1594818259 shutdown_connections 2024-04-16T17:13:02.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:02.145+0000 7faf2d26f740 1 --2- 172.21.15.45:0/1594818259 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x5566965ba750 0x5566965bcba0 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:02.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:02.145+0000 7faf2d26f740 1 --2- 172.21.15.45:0/1594818259 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5566965aa410 0x5566965a7e90 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:02.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:02.145+0000 7faf2d26f740 1 --2- 172.21.15.45:0/1594818259 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5566965a1830 0x5566965a7950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:02.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:02.145+0000 7faf11600640 0 client.27289 ms_handle_reset on v2:172.21.15.104:6838/3660070592 2024-04-16T17:13:02.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:02.145+0000 7faf2d26f740 1 --2- 172.21.15.45:0/1594818259 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5566965a0f10 0x5566965a7410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:02.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:02.145+0000 7faf2d26f740 1 -- 172.21.15.45:0/1594818259 >> 172.21.15.45:0/1594818259 conn(0x55669659b640 msgr2=0x55669659d390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:13:02.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:02.145+0000 7faf2d26f740 1 -- 172.21.15.45:0/1594818259 shutdown_connections 2024-04-16T17:13:02.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:02.145+0000 7faf2d26f740 1 -- 172.21.15.45:0/1594818259 wait complete. 2024-04-16T17:13:02.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:02.146+0000 7faf2d26f740 20 client.27289 trim_cache size 0 max 16384 2024-04-16T17:13:02.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:02.160+0000 7faf2d26f740 1 Processor -- start 2024-04-16T17:13:02.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:02.160+0000 7faf2d26f740 1 -- start start 2024-04-16T17:13:02.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:02.161+0000 7faf2d26f740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5566965a0f10 0x5566965a9b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:02.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:02.161+0000 7faf2d26f740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5566965a1830 0x5566965b60c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:02.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:02.161+0000 7faf2d26f740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5566965aa410 0x5566965c2470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:02.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:02.161+0000 7faf2d26f740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7faf241b3d30 con 0x5566965a0f10 2024-04-16T17:13:02.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:02.161+0000 7faf2d26f740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7faf241b1570 con 0x5566965a1830 2024-04-16T17:13:02.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:02.161+0000 7faf2d26f740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7faf241af3a0 con 0x5566965aa410 2024-04-16T17:13:02.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:02.161+0000 7faf23e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5566965aa410 0x5566965c2470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:02.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:02.161+0000 7faf23400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5566965a0f10 0x5566965a9b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:02.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:02.161+0000 7faf1be00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5566965a1830 0x5566965b60c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:02.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:02.161+0000 7faf23e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5566965aa410 0x5566965c2470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:54094/0 (socket says 172.21.15.45:54094) 2024-04-16T17:13:02.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:02.161+0000 7faf23e00640 1 -- 172.21.15.45:0/263192177 learned_addr learned my addr 172.21.15.45:0/263192177 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:13:02.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:02.162+0000 7faf23e00640 1 -- 172.21.15.45:0/263192177 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5566965a1830 msgr2=0x5566965b60c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:13:02.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:02.162+0000 7faf23e00640 1 --2- 172.21.15.45:0/263192177 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5566965a1830 0x5566965b60c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:02.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:02.162+0000 7faf23e00640 1 -- 172.21.15.45:0/263192177 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5566965a0f10 msgr2=0x5566965a9b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:13:02.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:02.162+0000 7faf23e00640 1 --2- 172.21.15.45:0/263192177 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5566965a0f10 0x5566965a9b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:02.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:02.162+0000 7faf23e00640 1 -- 172.21.15.45:0/263192177 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5566962183f0 con 0x5566965aa410 2024-04-16T17:13:02.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:02.162+0000 7faf23e00640 1 --2- 172.21.15.45:0/263192177 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5566965aa410 0x5566965c2470 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7faf1c019040 tx=0x7faf1c00ff80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=149af8c8421ada3f server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:13:02.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:02.162+0000 7faf13e00640 1 -- 172.21.15.45:0/263192177 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf1c00f840 con 0x5566965aa410 2024-04-16T17:13:02.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:02.162+0000 7faf13e00640 1 -- 172.21.15.45:0/263192177 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faf1c03e070 con 0x5566965aa410 2024-04-16T17:13:02.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:02.162+0000 7faf13e00640 1 -- 172.21.15.45:0/263192177 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf1c01e910 con 0x5566965aa410 2024-04-16T17:13:02.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:02.163+0000 7faf2d26f740 1 -- 172.21.15.45:0/263192177 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5566965aa410 msgr2=0x5566965c2470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:02.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:02.163+0000 7faf2d26f740 1 --2- 172.21.15.45:0/263192177 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5566965aa410 0x5566965c2470 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7faf1c019040 tx=0x7faf1c00ff80 comp rx=0 tx=0).stop 2024-04-16T17:13:02.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:02.163+0000 7faf2d26f740 1 -- 172.21.15.45:0/263192177 shutdown_connections 2024-04-16T17:13:02.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:02.163+0000 7faf2d26f740 1 --2- 172.21.15.45:0/263192177 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5566965aa410 0x5566965c2470 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:02.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:02.163+0000 7faf2d26f740 1 --2- 172.21.15.45:0/263192177 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5566965a1830 0x5566965b60c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:02.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:02.163+0000 7faf2d26f740 1 --2- 172.21.15.45:0/263192177 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5566965a0f10 0x5566965a9b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:02.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:02.163+0000 7faf2d26f740 1 -- 172.21.15.45:0/263192177 >> 172.21.15.45:0/263192177 conn(0x55669659b640 msgr2=0x55669659a700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:13:02.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:02.164+0000 7faf2d26f740 1 -- 172.21.15.45:0/263192177 shutdown_connections 2024-04-16T17:13:02.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:02.164+0000 7faf2d26f740 1 -- 172.21.15.45:0/263192177 wait complete. 2024-04-16T17:13:02.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:02.164+0000 7faf2d26f740 1 -- start start 2024-04-16T17:13:02.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:02.164+0000 7faf2d26f740 1 Processor -- start 2024-04-16T17:13:02.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:02.165+0000 7faf2d26f740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5566965a0f10 0x5566965ae3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:02.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:02.165+0000 7faf2d26f740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5566965a1830 0x5566965ae8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:02.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:02.165+0000 7faf2d26f740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5566965aa410 0x55669661fce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:02.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:02.165+0000 7faf2d26f740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7faf241afa30 con 0x5566965a0f10 2024-04-16T17:13:02.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:02.165+0000 7faf2d26f740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7faf241b0ea0 con 0x5566965aa410 2024-04-16T17:13:02.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:02.165+0000 7faf2d26f740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7faf241b0100 con 0x5566965a1830 2024-04-16T17:13:02.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:02.165+0000 7faf2d26f740 10 client.0 _ll_register_callbacks cb 0x7faf1c009df0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:13:02.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:02.165+0000 7faf1be00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5566965a1830 0x5566965ae8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:02.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:02.165+0000 7faf23400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5566965a0f10 0x5566965ae3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:02.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:02.165+0000 7faf1be00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5566965a1830 0x5566965ae8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:54096/0 (socket says 172.21.15.45:54096) 2024-04-16T17:13:02.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:02.165+0000 7faf1be00640 1 -- 172.21.15.45:0/1982708523 learned_addr learned my addr 172.21.15.45:0/1982708523 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:13:02.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:02.166+0000 7faf23e00640 1 --2- 172.21.15.45:0/1982708523 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5566965aa410 0x55669661fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:02.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:02.166+0000 7faf1be00640 1 -- 172.21.15.45:0/1982708523 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5566965aa410 msgr2=0x55669661fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:13:02.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:02.166+0000 7faf1be00640 1 --2- 172.21.15.45:0/1982708523 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5566965aa410 0x55669661fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:02.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:02.166+0000 7faf1be00640 1 -- 172.21.15.45:0/1982708523 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5566965a0f10 msgr2=0x5566965ae3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:13:02.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:02.166+0000 7faf1be00640 1 --2- 172.21.15.45:0/1982708523 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5566965a0f10 0x5566965ae3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:02.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:02.166+0000 7faf1be00640 1 -- 172.21.15.45:0/1982708523 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5566962e27d0 con 0x5566965a1830 2024-04-16T17:13:02.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:02.166+0000 7faf1be00640 1 --2- 172.21.15.45:0/1982708523 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5566965a1830 0x5566965ae8e0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x55669659c7d0 tx=0x7faf0c005db0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=db22a7e213f0e03e server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:13:02.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:02.166+0000 7faf11600640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-04-16T17:13:02.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:02.166+0000 7faf11600640 1 -- 172.21.15.45:0/1982708523 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf0c029070 con 0x5566965a1830 2024-04-16T17:13:02.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:02.167+0000 7faf11600640 1 -- 172.21.15.45:0/1982708523 <== mon.1 v2:172.21.15.117:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7faf0c004510 con 0x5566965a1830 2024-04-16T17:13:02.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:02.167+0000 7faf2d26f740 1 -- 172.21.15.45:0/1982708523 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faf1c03e070 con 0x5566965a1830 2024-04-16T17:13:02.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:02.167+0000 7faf11600640 1 -- 172.21.15.45:0/1982708523 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7faf0c035050 con 0x5566965a1830 2024-04-16T17:13:02.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:02.167+0000 7faf2d26f740 10 client.27295 fetch_fsmap learned FSMap version 37 2024-04-16T17:13:02.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:02.167+0000 7faf2d26f740 1 -- 172.21.15.45:0/1982708523 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x556696621180 con 0x5566965a1830 2024-04-16T17:13:02.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:02.167+0000 7faf11600640 1 -- 172.21.15.45:0/1982708523 <== mon.1 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faf1c03e070 con 0x5566965a1830 2024-04-16T17:13:02.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:02.167+0000 7faf11600640 1 -- 172.21.15.45:0/1982708523 <== mon.1 v2:172.21.15.117:3300/0 5 ==== fsmap(e 37) ==== 8358+0+0 (secure 0 0 0) 0x7faf0c018070 con 0x5566965a1830 2024-04-16T17:13:02.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:02.167+0000 7faf2d26f740 10 client.27295 fetch_fsmap finished waiting for FSMap version 37 2024-04-16T17:13:02.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:02.167+0000 7faf2d26f740 10 client.27295 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:13:02.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:02.167+0000 7faf2d26f740 20 client.27295 populate_metadata read hostname 'smithi045' 2024-04-16T17:13:02.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:02.167+0000 7faf2d26f740 1 --2- 172.21.15.45:0/1982708523 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x556696621740 0x556696623b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:02.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:02.167+0000 7faf2d26f740 4 client.27295 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T17:13:02.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:02.167+0000 7faf2d26f740 1 -- 172.21.15.45:0/1982708523 --> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55669650d8b0 con 0x556696621740 2024-04-16T17:13:02.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:02.168+0000 7faf23400640 1 --2- 172.21.15.45:0/1982708523 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x556696621740 0x556696623b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:02.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:02.168+0000 7faf11600640 10 client.27295 ms_handle_connect on v2:172.21.15.104:6838/3660070592 2024-04-16T17:13:02.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:02.168+0000 7faf23400640 1 --2- 172.21.15.45:0/1982708523 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x556696621740 0x556696623b60 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=acc0b74c8d83e001 server_cookie=4177912b9bda82d3 in_seq=0 out_seq=0 2024-04-16T17:13:02.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:02.169+0000 7faf11600640 1 -- 172.21.15.45:0/1982708523 <== mds.0 v2:172.21.15.104:6838/3660070592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55669650d8b0 con 0x556696621740 2024-04-16T17:13:02.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:02.169+0000 7faf11600640 10 client.27295 handle_command_reply: tid=0 2024-04-16T17:13:02.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:02.169+0000 7faf2d26f740 1 client.27295 shutdown 2024-04-16T17:13:02.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:02.169+0000 7faf2d26f740 1 -- 172.21.15.45:0/1982708523 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5566965a1830 msgr2=0x5566965ae8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:02.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:02.169+0000 7faf2d26f740 1 --2- 172.21.15.45:0/1982708523 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5566965a1830 0x5566965ae8e0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x55669659c7d0 tx=0x7faf0c005db0 comp rx=0 tx=0).stop 2024-04-16T17:13:02.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:02.169+0000 7faf2d26f740 1 -- 172.21.15.45:0/1982708523 shutdown_connections 2024-04-16T17:13:02.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:02.169+0000 7faf2d26f740 1 --2- 172.21.15.45:0/1982708523 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x556696621740 0x556696623b60 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:02.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:02.169+0000 7faf2d26f740 1 --2- 172.21.15.45:0/1982708523 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5566965a1830 0x5566965ae8e0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:02.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:02.169+0000 7faf2d26f740 1 --2- 172.21.15.45:0/1982708523 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5566965aa410 0x55669661fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:02.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:02.169+0000 7faf2d26f740 1 --2- 172.21.15.45:0/1982708523 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5566965a0f10 0x5566965ae3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:02.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:02.169+0000 7faf11600640 0 client.27295 ms_handle_reset on v2:172.21.15.104:6838/3660070592 2024-04-16T17:13:02.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:02.169+0000 7faf2d26f740 1 -- 172.21.15.45:0/1982708523 >> 172.21.15.45:0/1982708523 conn(0x55669659b640 msgr2=0x5566965a73d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:13:02.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:02.169+0000 7faf2d26f740 1 -- 172.21.15.45:0/1982708523 shutdown_connections 2024-04-16T17:13:02.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:02.169+0000 7faf2d26f740 1 -- 172.21.15.45:0/1982708523 wait complete. 2024-04-16T17:13:02.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:02.169+0000 7faf2d26f740 20 client.27295 trim_cache size 0 max 16384 2024-04-16T17:13:02.173 INFO:teuthology.orchestra.run.smithi045.stdout:[] 2024-04-16T17:13:02.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:02.170+0000 7faf29000640 1 -- 172.21.15.45:0/1328465868 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7faf080a3d20 msgr2=0x7faf080a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:02.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:02.170+0000 7faf29000640 1 --2- 172.21.15.45:0/1328465868 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7faf080a3d20 0x7faf080a61e0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7faf1c005fd0 tx=0x7faf1c009500 comp rx=0 tx=0).stop 2024-04-16T17:13:02.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:02.170+0000 7faf29000640 1 -- 172.21.15.45:0/1328465868 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faf24102110 msgr2=0x7faf241a0640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:02.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:02.170+0000 7faf29000640 1 --2- 172.21.15.45:0/1328465868 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faf24102110 0x7faf241a0640 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7faf1402c240 tx=0x7faf140042f0 comp rx=0 tx=0).stop 2024-04-16T17:13:02.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:02.170+0000 7faf29000640 1 -- 172.21.15.45:0/1328465868 shutdown_connections 2024-04-16T17:13:02.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:02.170+0000 7faf29000640 1 --2- 172.21.15.45:0/1328465868 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7faf080a3d20 0x7faf080a61e0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:02.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:02.170+0000 7faf29000640 1 --2- 172.21.15.45:0/1328465868 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faf241071e0 0x7faf241a0b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:02.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:02.170+0000 7faf29000640 1 --2- 172.21.15.45:0/1328465868 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7faf2419a7b0 0x7faf2419ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:02.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:02.170+0000 7faf29000640 1 --2- 172.21.15.45:0/1328465868 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faf24102110 0x7faf241a0640 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:02.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:02.170+0000 7faf29000640 1 -- 172.21.15.45:0/1328465868 >> 172.21.15.45:0/1328465868 conn(0x7faf240fd9d0 msgr2=0x7faf240fec80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:13:02.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:02.170+0000 7faf29000640 1 -- 172.21.15.45:0/1328465868 shutdown_connections 2024-04-16T17:13:02.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:02.171+0000 7faf29000640 1 -- 172.21.15.45:0/1328465868 wait complete. 2024-04-16T17:13:02.274 INFO:tasks.workunit.client.0.smithi045.stdout: 1 101056 1.70 MB/sec execute 184 sec latency 2575.700 ms 2024-04-16T17:13:03.188 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-16T17:13:03.188 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T17:13:03.188 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-16T17:13:03.274 INFO:tasks.workunit.client.0.smithi045.stdout: 1 101340 1.69 MB/sec execute 185 sec latency 121.961 ms 2024-04-16T17:13:03.304 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:13:02 smithi117 ceph-mon[36730]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:13:03.304 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:13:02 smithi117 ceph-mon[36730]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T17:13:03.304 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:13:02 smithi117 ceph-mon[36730]: from='mgr.25411 ' entity='mgr.x' 2024-04-16T17:13:03.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.394+0000 7f5e8d000640 1 -- 172.21.15.45:0/2800732273 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5e881044b0 msgr2=0x7f5e88104890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:03.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.394+0000 7f5e8d000640 1 --2- 172.21.15.45:0/2800732273 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5e881044b0 0x7f5e88104890 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f5e78009900 tx=0x7f5e7802bc30 comp rx=0 tx=0).stop 2024-04-16T17:13:03.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.394+0000 7f5e8d000640 1 -- 172.21.15.45:0/2800732273 shutdown_connections 2024-04-16T17:13:03.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.394+0000 7f5e8d000640 1 --2- 172.21.15.45:0/2800732273 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5e88109580 0x7f5e88111150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:03.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.394+0000 7f5e8d000640 1 --2- 172.21.15.45:0/2800732273 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5e88104e60 0x7f5e88108ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:03.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.394+0000 7f5e8d000640 1 --2- 172.21.15.45:0/2800732273 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5e881044b0 0x7f5e88104890 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:03.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.394+0000 7f5e8d000640 1 -- 172.21.15.45:0/2800732273 >> 172.21.15.45:0/2800732273 conn(0x7f5e880ffd90 msgr2=0x7f5e881021b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:13:03.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.395+0000 7f5e8d000640 1 -- 172.21.15.45:0/2800732273 shutdown_connections 2024-04-16T17:13:03.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.395+0000 7f5e8d000640 1 -- 172.21.15.45:0/2800732273 wait complete. 2024-04-16T17:13:03.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.396+0000 7f5e8d000640 1 Processor -- start 2024-04-16T17:13:03.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.396+0000 7f5e8d000640 1 -- start start 2024-04-16T17:13:03.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.396+0000 7f5e8d000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5e881044b0 0x7f5e881a07f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:03.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.396+0000 7f5e8d000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5e88109580 0x7f5e881a0d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:03.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.396+0000 7f5e8d000640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5e8819a9a0 0x7f5e8819ae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:03.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.396+0000 7f5e8d000640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f5e88114660 con 0x7f5e881044b0 2024-04-16T17:13:03.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.396+0000 7f5e8d000640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f5e881144e0 con 0x7f5e8819a9a0 2024-04-16T17:13:03.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.396+0000 7f5e8d000640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f5e881147e0 con 0x7f5e88109580 2024-04-16T17:13:03.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.397+0000 7f5e85e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5e881044b0 0x7f5e881a07f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:03.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.397+0000 7f5e86800640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5e8819a9a0 0x7f5e8819ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:03.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.397+0000 7f5e85e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5e881044b0 0x7f5e881a07f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:46866/0 (socket says 172.21.15.45:46866) 2024-04-16T17:13:03.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.397+0000 7f5e85e00640 1 -- 172.21.15.45:0/2122918220 learned_addr learned my addr 172.21.15.45:0/2122918220 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:13:03.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.397+0000 7f5e85400640 1 --2- 172.21.15.45:0/2122918220 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5e88109580 0x7f5e881a0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:03.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.397+0000 7f5e86800640 1 -- 172.21.15.45:0/2122918220 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5e88109580 msgr2=0x7f5e881a0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:03.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.397+0000 7f5e86800640 1 --2- 172.21.15.45:0/2122918220 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5e88109580 0x7f5e881a0d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:03.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.397+0000 7f5e86800640 1 -- 172.21.15.45:0/2122918220 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5e881044b0 msgr2=0x7f5e881a07f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:03.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.397+0000 7f5e86800640 1 --2- 172.21.15.45:0/2122918220 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5e881044b0 0x7f5e881a07f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:03.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.397+0000 7f5e86800640 1 -- 172.21.15.45:0/2122918220 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e8819b4c0 con 0x7f5e8819a9a0 2024-04-16T17:13:03.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.397+0000 7f5e86800640 1 --2- 172.21.15.45:0/2122918220 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5e8819a9a0 0x7f5e8819ae50 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f5e80009fd0 tx=0x7f5e8000ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:13:03.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.398+0000 7f5e7ea00640 1 -- 172.21.15.45:0/2122918220 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e8000ce20 con 0x7f5e8819a9a0 2024-04-16T17:13:03.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.398+0000 7f5e7ea00640 1 -- 172.21.15.45:0/2122918220 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5e80005020 con 0x7f5e8819a9a0 2024-04-16T17:13:03.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.398+0000 7f5e8d000640 1 -- 172.21.15.45:0/2122918220 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e8819b7b0 con 0x7f5e8819a9a0 2024-04-16T17:13:03.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.398+0000 7f5e7ea00640 1 -- 172.21.15.45:0/2122918220 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e80010720 con 0x7f5e8819a9a0 2024-04-16T17:13:03.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.398+0000 7f5e8d000640 1 -- 172.21.15.45:0/2122918220 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e881a7a00 con 0x7f5e8819a9a0 2024-04-16T17:13:03.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.401+0000 7f5e7ea00640 1 -- 172.21.15.45:0/2122918220 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f5e80005540 con 0x7f5e8819a9a0 2024-04-16T17:13:03.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.401+0000 7f5e8fc26740 1 Processor -- start 2024-04-16T17:13:03.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.401+0000 7f5e8fc26740 1 -- start start 2024-04-16T17:13:03.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.402+0000 7f5e8fc26740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55afdedd0ca0 0x55afdedd1080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:03.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.402+0000 7f5e8fc26740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55afdedd15c0 0x55afdedd9c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:03.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.402+0000 7f5e8fc26740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55afdedda1a0 0x55afdeddc590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:03.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.402+0000 7f5e8fc26740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55afded35cb0 con 0x55afdedda1a0 2024-04-16T17:13:03.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.402+0000 7f5e8fc26740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55afdecdc370 con 0x55afdedd0ca0 2024-04-16T17:13:03.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.402+0000 7f5e8fc26740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55afded31c70 con 0x55afdedd15c0 2024-04-16T17:13:03.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.402+0000 7f5e85400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55afdedd0ca0 0x55afdedd1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:03.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.402+0000 7f5e85400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55afdedd0ca0 0x55afdedd1080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:35984/0 (socket says 172.21.15.45:35984) 2024-04-16T17:13:03.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.402+0000 7f5e85400640 1 -- 172.21.15.45:0/34295805 learned_addr learned my addr 172.21.15.45:0/34295805 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:13:03.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.402+0000 7f5e85400640 1 --2- 172.21.15.45:0/34295805 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55afdedda1a0 0x55afdeddc590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:03.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.402+0000 7f5e85e00640 1 --2- 172.21.15.45:0/34295805 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55afdedd15c0 0x55afdedd9c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:03.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.403+0000 7f5e7ea00640 1 --2- 172.21.15.45:0/2122918220 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f5e6c0a3da0 0x7f5e6c0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:03.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.403+0000 7f5e85400640 1 -- 172.21.15.45:0/34295805 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55afdedd15c0 msgr2=0x55afdedd9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:13:03.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.403+0000 7f5e85400640 1 --2- 172.21.15.45:0/34295805 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55afdedd15c0 0x55afdedd9c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:03.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.403+0000 7f5e85400640 1 -- 172.21.15.45:0/34295805 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55afdedd0ca0 msgr2=0x55afdedd1080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:13:03.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.403+0000 7f5e85400640 1 --2- 172.21.15.45:0/34295805 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55afdedd0ca0 0x55afdedd1080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:03.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.403+0000 7f5e85400640 1 -- 172.21.15.45:0/34295805 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55afdecfb910 con 0x55afdedda1a0 2024-04-16T17:13:03.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.403+0000 7f5e7ea00640 1 -- 172.21.15.45:0/2122918220 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f5e800d1180 con 0x7f5e8819a9a0 2024-04-16T17:13:03.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.403+0000 7f5e86800640 1 --2- 172.21.15.45:0/2122918220 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f5e6c0a3da0 0x7f5e6c0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:03.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.403+0000 7f5e85400640 1 --2- 172.21.15.45:0/34295805 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55afdedda1a0 0x55afdeddc590 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f5e740117a0 tx=0x7f5e74005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9e0cad142c6a0ea server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:13:03.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.404+0000 7f5e73400640 1 -- 172.21.15.45:0/34295805 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e74011de0 con 0x55afdedda1a0 2024-04-16T17:13:03.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.404+0000 7f5e73400640 1 -- 172.21.15.45:0/34295805 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5e7400f7c0 con 0x55afdedda1a0 2024-04-16T17:13:03.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.404+0000 7f5e73400640 1 -- 172.21.15.45:0/34295805 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e74003850 con 0x55afdedda1a0 2024-04-16T17:13:03.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.405+0000 7f5e8fc26740 1 -- 172.21.15.45:0/34295805 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55afdedda1a0 msgr2=0x55afdeddc590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:03.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.405+0000 7f5e86800640 1 --2- 172.21.15.45:0/2122918220 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f5e6c0a3da0 0x7f5e6c0a6260 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f5e80005ba0 tx=0x7f5e80005f70 comp rx=0 tx=0).ready entity=mgr.25411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:13:03.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.405+0000 7f5e8fc26740 1 --2- 172.21.15.45:0/34295805 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55afdedda1a0 0x55afdeddc590 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f5e740117a0 tx=0x7f5e74005f90 comp rx=0 tx=0).stop 2024-04-16T17:13:03.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.405+0000 7f5e8fc26740 1 -- 172.21.15.45:0/34295805 shutdown_connections 2024-04-16T17:13:03.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.405+0000 7f5e8fc26740 1 --2- 172.21.15.45:0/34295805 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55afdedd15c0 0x55afdedd9c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:03.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.405+0000 7f5e8fc26740 1 --2- 172.21.15.45:0/34295805 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55afdedda1a0 0x55afdeddc590 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:03.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.405+0000 7f5e8fc26740 1 --2- 172.21.15.45:0/34295805 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55afdedd0ca0 0x55afdedd1080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:03.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.405+0000 7f5e8fc26740 1 -- 172.21.15.45:0/34295805 >> 172.21.15.45:0/34295805 conn(0x55afdedcb3d0 msgr2=0x55afdedcb7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:13:03.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.406+0000 7f5e8fc26740 1 -- 172.21.15.45:0/34295805 shutdown_connections 2024-04-16T17:13:03.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.406+0000 7f5e8fc26740 1 -- 172.21.15.45:0/34295805 wait complete. 2024-04-16T17:13:03.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.406+0000 7f5e8fc26740 1 -- start start 2024-04-16T17:13:03.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.406+0000 7f5e8fc26740 1 Processor -- start 2024-04-16T17:13:03.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.407+0000 7f5e8fc26740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55afdedd0ca0 0x55afdedd7300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:03.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.407+0000 7f5e8fc26740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55afdedd15c0 0x55afdedd7840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:03.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.407+0000 7f5e8fc26740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55afdedda1a0 0x55afdedd7d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:03.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.407+0000 7f5e8fc26740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55afdeddcf90 con 0x55afdedd15c0 2024-04-16T17:13:03.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.407+0000 7f5e8fc26740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55afdecef730 con 0x55afdedd0ca0 2024-04-16T17:13:03.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.407+0000 7f5e8fc26740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55afdeddd110 con 0x55afdedda1a0 2024-04-16T17:13:03.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.407+0000 7f5e8fc26740 10 client.0 _ll_register_callbacks cb 0x55afdecbe1e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:13:03.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.407+0000 7f5e85e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55afdedd15c0 0x55afdedd7840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:03.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.407+0000 7f5e85e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55afdedd15c0 0x55afdedd7840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:46886/0 (socket says 172.21.15.45:46886) 2024-04-16T17:13:03.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.407+0000 7f5e85e00640 1 -- 172.21.15.45:0/1607968711 learned_addr learned my addr 172.21.15.45:0/1607968711 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:13:03.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.407+0000 7f5e85400640 1 --2- 172.21.15.45:0/1607968711 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55afdedd0ca0 0x55afdedd7300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:03.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.408+0000 7f5e85400640 1 --2- 172.21.15.45:0/1607968711 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55afdedda1a0 0x55afdedd7d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:03.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.408+0000 7f5e85e00640 1 -- 172.21.15.45:0/1607968711 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55afdedda1a0 msgr2=0x55afdedd7d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:13:03.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.408+0000 7f5e85e00640 1 --2- 172.21.15.45:0/1607968711 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55afdedda1a0 0x55afdedd7d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:03.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.408+0000 7f5e85e00640 1 -- 172.21.15.45:0/1607968711 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55afdedd0ca0 msgr2=0x55afdedd7300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:13:03.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.408+0000 7f5e85e00640 1 --2- 172.21.15.45:0/1607968711 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55afdedd0ca0 0x55afdedd7300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:03.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.408+0000 7f5e85e00640 1 -- 172.21.15.45:0/1607968711 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55afdeb107e0 con 0x55afdedd15c0 2024-04-16T17:13:03.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.408+0000 7f5e85400640 1 --2- 172.21.15.45:0/1607968711 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55afdedd0ca0 0x55afdedd7300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:13:03.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.408+0000 7f5e85e00640 1 --2- 172.21.15.45:0/1607968711 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55afdedd15c0 0x55afdedd7840 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f5e78036040 tx=0x7f5e7802c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d536e1b4b7604884 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:13:03.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.408+0000 7f5e67e00640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-16T17:13:03.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.408+0000 7f5e67e00640 1 -- 172.21.15.45:0/1607968711 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e7802ca90 con 0x55afdedd15c0 2024-04-16T17:13:03.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.409+0000 7f5e67e00640 1 -- 172.21.15.45:0/1607968711 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f5e7802cc30 con 0x55afdedd15c0 2024-04-16T17:13:03.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.409+0000 7f5e8fc26740 1 -- 172.21.15.45:0/1607968711 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5e7400f7c0 con 0x55afdedd15c0 2024-04-16T17:13:03.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.409+0000 7f5e67e00640 1 -- 172.21.15.45:0/1607968711 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f5e780046c0 con 0x55afdedd15c0 2024-04-16T17:13:03.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.409+0000 7f5e8fc26740 10 client.17196 fetch_fsmap learned FSMap version 37 2024-04-16T17:13:03.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.409+0000 7f5e8fc26740 1 -- 172.21.15.45:0/1607968711 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55afdeb70f70 con 0x55afdedd15c0 2024-04-16T17:13:03.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.409+0000 7f5e67e00640 1 -- 172.21.15.45:0/1607968711 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5e7400f7c0 con 0x55afdedd15c0 2024-04-16T17:13:03.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.409+0000 7f5e67e00640 1 -- 172.21.15.45:0/1607968711 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 37) ==== 8358+0+0 (secure 0 0 0) 0x7f5e780035f0 con 0x55afdedd15c0 2024-04-16T17:13:03.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.410+0000 7f5e8fc26740 10 client.17196 fetch_fsmap finished waiting for FSMap version 37 2024-04-16T17:13:03.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.410+0000 7f5e8fc26740 10 client.17196 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:13:03.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.410+0000 7f5e8fc26740 20 client.17196 populate_metadata read hostname 'smithi045' 2024-04-16T17:13:03.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.410+0000 7f5e8fc26740 1 --2- 172.21.15.45:0/1607968711 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55afdedea650 0x55afdedeca60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:03.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.410+0000 7f5e8fc26740 4 client.17196 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:13:03.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.410+0000 7f5e8fc26740 1 -- 172.21.15.45:0/1607968711 --> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55afded436d0 con 0x55afdedea650 2024-04-16T17:13:03.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.410+0000 7f5e85400640 1 --2- 172.21.15.45:0/1607968711 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55afdedea650 0x55afdedeca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:03.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.411+0000 7f5e85400640 1 --2- 172.21.15.45:0/1607968711 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55afdedea650 0x55afdedeca60 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f2e533767f9a4f0a server_cookie=fa15b254e48ade3b in_seq=0 out_seq=0 2024-04-16T17:13:03.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.411+0000 7f5e67e00640 10 client.17196 ms_handle_connect on v2:172.21.15.104:6838/3660070592 2024-04-16T17:13:03.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.411+0000 7f5e67e00640 1 -- 172.21.15.45:0/1607968711 <== mds.0 v2:172.21.15.104:6838/3660070592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55afded436d0 con 0x55afdedea650 2024-04-16T17:13:03.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.411+0000 7f5e67e00640 10 client.17196 handle_command_reply: tid=0 2024-04-16T17:13:03.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.411+0000 7f5e8fc26740 1 client.17196 shutdown 2024-04-16T17:13:03.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.412+0000 7f5e8fc26740 1 -- 172.21.15.45:0/1607968711 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55afdedd15c0 msgr2=0x55afdedd7840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:03.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.412+0000 7f5e8fc26740 1 --2- 172.21.15.45:0/1607968711 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55afdedd15c0 0x55afdedd7840 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f5e78036040 tx=0x7f5e7802c7e0 comp rx=0 tx=0).stop 2024-04-16T17:13:03.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.412+0000 7f5e8fc26740 1 -- 172.21.15.45:0/1607968711 shutdown_connections 2024-04-16T17:13:03.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.412+0000 7f5e8fc26740 1 --2- 172.21.15.45:0/1607968711 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55afdedea650 0x55afdedeca60 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:03.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.412+0000 7f5e8fc26740 1 --2- 172.21.15.45:0/1607968711 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55afdedda1a0 0x55afdedd7d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:03.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.412+0000 7f5e8fc26740 1 --2- 172.21.15.45:0/1607968711 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55afdedd15c0 0x55afdedd7840 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:03.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.412+0000 7f5e8fc26740 1 --2- 172.21.15.45:0/1607968711 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55afdedd0ca0 0x55afdedd7300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:03.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.412+0000 7f5e8fc26740 1 -- 172.21.15.45:0/1607968711 >> 172.21.15.45:0/1607968711 conn(0x55afdedcb3d0 msgr2=0x55afdedcd120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:13:03.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.412+0000 7f5e67e00640 0 client.17196 ms_handle_reset on v2:172.21.15.104:6838/3660070592 2024-04-16T17:13:03.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.412+0000 7f5e8fc26740 1 -- 172.21.15.45:0/1607968711 shutdown_connections 2024-04-16T17:13:03.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.412+0000 7f5e8fc26740 1 -- 172.21.15.45:0/1607968711 wait complete. 2024-04-16T17:13:03.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.412+0000 7f5e8fc26740 20 client.17196 trim_cache size 0 max 16384 2024-04-16T17:13:03.427 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:13:02 smithi104 ceph-mon[36457]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:13:03.427 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:13:02 smithi104 ceph-mon[36457]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T17:13:03.427 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:13:02 smithi104 ceph-mon[36457]: from='mgr.25411 ' entity='mgr.x' 2024-04-16T17:13:03.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.432+0000 7f5e8fc26740 1 Processor -- start 2024-04-16T17:13:03.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.432+0000 7f5e8fc26740 1 -- start start 2024-04-16T17:13:03.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.432+0000 7f5e8fc26740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55afdedd0ca0 0x55afdedd9a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:03.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.432+0000 7f5e8fc26740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55afdedd15c0 0x55afdede5e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:03.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.432+0000 7f5e8fc26740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55afdedda1a0 0x55afdedd79d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:03.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.432+0000 7f5e8fc26740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f5e881b4000 con 0x55afdedd0ca0 2024-04-16T17:13:03.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.432+0000 7f5e8fc26740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f5e881b1840 con 0x55afdedda1a0 2024-04-16T17:13:03.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.432+0000 7f5e8fc26740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f5e881af670 con 0x55afdedd15c0 2024-04-16T17:13:03.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.433+0000 7f5e85400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55afdedd0ca0 0x55afdedd9a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:03.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.433+0000 7f5e85400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55afdedd0ca0 0x55afdedd9a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:46890/0 (socket says 172.21.15.45:46890) 2024-04-16T17:13:03.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.433+0000 7f5e85400640 1 -- 172.21.15.45:0/1332744333 learned_addr learned my addr 172.21.15.45:0/1332744333 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:13:03.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.433+0000 7f5e85e00640 1 --2- 172.21.15.45:0/1332744333 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55afdedd15c0 0x55afdede5e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:03.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.433+0000 7f5e85400640 1 --2- 172.21.15.45:0/1332744333 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55afdedda1a0 0x55afdedd79d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:03.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.433+0000 7f5e85400640 1 -- 172.21.15.45:0/1332744333 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55afdedd15c0 msgr2=0x55afdede5e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:13:03.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.433+0000 7f5e85400640 1 --2- 172.21.15.45:0/1332744333 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55afdedd15c0 0x55afdede5e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:03.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.433+0000 7f5e85400640 1 -- 172.21.15.45:0/1332744333 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55afdedda1a0 msgr2=0x55afdedd79d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:13:03.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.433+0000 7f5e85400640 1 --2- 172.21.15.45:0/1332744333 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55afdedda1a0 0x55afdedd79d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:03.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.433+0000 7f5e85400640 1 -- 172.21.15.45:0/1332744333 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55afdea08170 con 0x55afdedd0ca0 2024-04-16T17:13:03.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.433+0000 7f5e85400640 1 --2- 172.21.15.45:0/1332744333 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55afdedd0ca0 0x55afdedd9a30 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f5e740117a0 tx=0x7f5e74009ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=290910a380232727 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:13:03.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.434+0000 7f5e72a00640 1 -- 172.21.15.45:0/1332744333 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e74011250 con 0x55afdedd0ca0 2024-04-16T17:13:03.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.434+0000 7f5e72a00640 1 -- 172.21.15.45:0/1332744333 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5e7402e8b0 con 0x55afdedd0ca0 2024-04-16T17:13:03.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.434+0000 7f5e72a00640 1 -- 172.21.15.45:0/1332744333 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e7400fbb0 con 0x55afdedd0ca0 2024-04-16T17:13:03.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.434+0000 7f5e8fc26740 1 -- 172.21.15.45:0/1332744333 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55afdedd0ca0 msgr2=0x55afdedd9a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:03.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.434+0000 7f5e8fc26740 1 --2- 172.21.15.45:0/1332744333 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55afdedd0ca0 0x55afdedd9a30 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f5e740117a0 tx=0x7f5e74009ae0 comp rx=0 tx=0).stop 2024-04-16T17:13:03.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.434+0000 7f5e8fc26740 1 -- 172.21.15.45:0/1332744333 shutdown_connections 2024-04-16T17:13:03.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.435+0000 7f5e8fc26740 1 --2- 172.21.15.45:0/1332744333 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55afdedd15c0 0x55afdede5e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:03.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.435+0000 7f5e8fc26740 1 --2- 172.21.15.45:0/1332744333 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55afdedda1a0 0x55afdedd79d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:03.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.435+0000 7f5e8fc26740 1 --2- 172.21.15.45:0/1332744333 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55afdedd0ca0 0x55afdedd9a30 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:03.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.435+0000 7f5e8fc26740 1 -- 172.21.15.45:0/1332744333 >> 172.21.15.45:0/1332744333 conn(0x55afdedcb3d0 msgr2=0x55afdedd72c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:13:03.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.435+0000 7f5e8fc26740 1 -- 172.21.15.45:0/1332744333 shutdown_connections 2024-04-16T17:13:03.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.435+0000 7f5e8fc26740 1 -- 172.21.15.45:0/1332744333 wait complete. 2024-04-16T17:13:03.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.435+0000 7f5e8fc26740 1 -- start start 2024-04-16T17:13:03.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.435+0000 7f5e8fc26740 1 Processor -- start 2024-04-16T17:13:03.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.436+0000 7f5e8fc26740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55afdedd0ca0 0x55afdede28b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:03.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.436+0000 7f5e8fc26740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55afdedd15c0 0x55afdedddc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:03.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.436+0000 7f5e8fc26740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55afdedda1a0 0x55afdedde360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:03.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.436+0000 7f5e8fc26740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f5e881afd00 con 0x55afdedd0ca0 2024-04-16T17:13:03.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.436+0000 7f5e8fc26740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f5e881b1170 con 0x55afdedd15c0 2024-04-16T17:13:03.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.436+0000 7f5e8fc26740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f5e881b03d0 con 0x55afdedda1a0 2024-04-16T17:13:03.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.436+0000 7f5e8fc26740 10 client.0 _ll_register_callbacks cb 0x7f5e78004520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:13:03.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.436+0000 7f5e85400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55afdedd0ca0 0x55afdede28b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:03.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.436+0000 7f5e85400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55afdedd0ca0 0x55afdede28b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:46902/0 (socket says 172.21.15.45:46902) 2024-04-16T17:13:03.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.436+0000 7f5e85400640 1 -- 172.21.15.45:0/681182819 learned_addr learned my addr 172.21.15.45:0/681182819 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:13:03.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.436+0000 7f5e85e00640 1 --2- 172.21.15.45:0/681182819 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55afdedd15c0 0x55afdedddc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:03.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.436+0000 7f5e85400640 1 --2- 172.21.15.45:0/681182819 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55afdedda1a0 0x55afdedde360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:03.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.436+0000 7f5e85e00640 1 -- 172.21.15.45:0/681182819 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55afdedda1a0 msgr2=0x55afdedde360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:13:03.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.436+0000 7f5e85e00640 1 --2- 172.21.15.45:0/681182819 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55afdedda1a0 0x55afdedde360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:03.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.437+0000 7f5e85e00640 1 -- 172.21.15.45:0/681182819 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55afdedd0ca0 msgr2=0x55afdede28b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:13:03.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.437+0000 7f5e85e00640 1 --2- 172.21.15.45:0/681182819 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55afdedd0ca0 0x55afdede28b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:03.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.437+0000 7f5e85e00640 1 -- 172.21.15.45:0/681182819 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55afdee4fba0 con 0x55afdedd15c0 2024-04-16T17:13:03.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.437+0000 7f5e85400640 1 --2- 172.21.15.45:0/681182819 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55afdedda1a0 0x55afdedde360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:13:03.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.442+0000 7f5e85e00640 1 --2- 172.21.15.45:0/681182819 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55afdedd15c0 0x55afdedddc20 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f5e78006010 tx=0x7f5e7802bc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f1322e3ae62d5799 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:13:03.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.442+0000 7f5e67e00640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-04-16T17:13:03.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.442+0000 7f5e67e00640 1 -- 172.21.15.45:0/681182819 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e78005d50 con 0x55afdedd15c0 2024-04-16T17:13:03.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.442+0000 7f5e67e00640 1 -- 172.21.15.45:0/681182819 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f5e78004860 con 0x55afdedd15c0 2024-04-16T17:13:03.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.442+0000 7f5e8fc26740 1 -- 172.21.15.45:0/681182819 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5e7402e8b0 con 0x55afdedd15c0 2024-04-16T17:13:03.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.443+0000 7f5e67e00640 1 -- 172.21.15.45:0/681182819 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f5e78068050 con 0x55afdedd15c0 2024-04-16T17:13:03.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.443+0000 7f5e8fc26740 10 client.27374 fetch_fsmap learned FSMap version 37 2024-04-16T17:13:03.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.443+0000 7f5e8fc26740 1 -- 172.21.15.45:0/681182819 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55afdee50d80 con 0x55afdedd15c0 2024-04-16T17:13:03.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.443+0000 7f5e67e00640 1 -- 172.21.15.45:0/681182819 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5e7402e8b0 con 0x55afdedd15c0 2024-04-16T17:13:03.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.443+0000 7f5e67e00640 1 -- 172.21.15.45:0/681182819 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 37) ==== 8358+0+0 (secure 0 0 0) 0x7f5e7804b480 con 0x55afdedd15c0 2024-04-16T17:13:03.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.443+0000 7f5e8fc26740 10 client.27374 fetch_fsmap finished waiting for FSMap version 37 2024-04-16T17:13:03.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.443+0000 7f5e8fc26740 10 client.27374 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:13:03.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.443+0000 7f5e8fc26740 20 client.27374 populate_metadata read hostname 'smithi045' 2024-04-16T17:13:03.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.443+0000 7f5e8fc26740 1 --2- 172.21.15.45:0/681182819 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55afdee51580 0x55afdee539a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:03.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.443+0000 7f5e8fc26740 4 client.27374 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-16T17:13:03.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.443+0000 7f5e8fc26740 1 -- 172.21.15.45:0/681182819 --> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55afded3d540 con 0x55afdee51580 2024-04-16T17:13:03.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.444+0000 7f5e85400640 1 --2- 172.21.15.45:0/681182819 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55afdee51580 0x55afdee539a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:03.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.444+0000 7f5e85400640 1 --2- 172.21.15.45:0/681182819 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55afdee51580 0x55afdee539a0 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=321843b6e5aaee95 server_cookie=b4126d07fb216236 in_seq=0 out_seq=0 2024-04-16T17:13:03.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.444+0000 7f5e67e00640 10 client.27374 ms_handle_connect on v2:172.21.15.104:6838/3660070592 2024-04-16T17:13:03.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.445+0000 7f5e67e00640 1 -- 172.21.15.45:0/681182819 <== mds.0 v2:172.21.15.104:6838/3660070592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55afded3d540 con 0x55afdee51580 2024-04-16T17:13:03.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.445+0000 7f5e67e00640 10 client.27374 handle_command_reply: tid=0 2024-04-16T17:13:03.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.445+0000 7f5e8fc26740 1 client.27374 shutdown 2024-04-16T17:13:03.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.445+0000 7f5e8fc26740 1 -- 172.21.15.45:0/681182819 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55afdedd15c0 msgr2=0x55afdedddc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:03.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.445+0000 7f5e8fc26740 1 --2- 172.21.15.45:0/681182819 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55afdedd15c0 0x55afdedddc20 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f5e78006010 tx=0x7f5e7802bc30 comp rx=0 tx=0).stop 2024-04-16T17:13:03.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.446+0000 7f5e8fc26740 1 -- 172.21.15.45:0/681182819 shutdown_connections 2024-04-16T17:13:03.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.446+0000 7f5e8fc26740 1 --2- 172.21.15.45:0/681182819 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55afdee51580 0x55afdee539a0 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:03.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.446+0000 7f5e8fc26740 1 --2- 172.21.15.45:0/681182819 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55afdedda1a0 0x55afdedde360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:03.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.446+0000 7f5e8fc26740 1 --2- 172.21.15.45:0/681182819 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55afdedd15c0 0x55afdedddc20 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:03.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.446+0000 7f5e8fc26740 1 --2- 172.21.15.45:0/681182819 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55afdedd0ca0 0x55afdede28b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:03.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.446+0000 7f5e8fc26740 1 -- 172.21.15.45:0/681182819 >> 172.21.15.45:0/681182819 conn(0x55afdedcb3d0 msgr2=0x55afdedd7b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:13:03.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.446+0000 7f5e67e00640 0 client.27374 ms_handle_reset on v2:172.21.15.104:6838/3660070592 2024-04-16T17:13:03.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.446+0000 7f5e8fc26740 1 -- 172.21.15.45:0/681182819 shutdown_connections 2024-04-16T17:13:03.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.446+0000 7f5e8fc26740 1 -- 172.21.15.45:0/681182819 wait complete. 2024-04-16T17:13:03.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.446+0000 7f5e8fc26740 20 client.27374 trim_cache size 0 max 16384 2024-04-16T17:13:03.450 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-16T17:13:03.450 INFO:teuthology.orchestra.run.smithi045.stdout: "return_code": 0, 2024-04-16T17:13:03.450 INFO:teuthology.orchestra.run.smithi045.stdout: "scrub_tag": "68131f8b-72dd-4619-a321-eebff406a4de", 2024-04-16T17:13:03.450 INFO:teuthology.orchestra.run.smithi045.stdout: "mode": "asynchronous" 2024-04-16T17:13:03.450 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-16T17:13:03.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.447+0000 7f5e8d000640 1 -- 172.21.15.45:0/2122918220 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f5e6c0a3da0 msgr2=0x7f5e6c0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:03.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.447+0000 7f5e8d000640 1 --2- 172.21.15.45:0/2122918220 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f5e6c0a3da0 0x7f5e6c0a6260 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f5e80005ba0 tx=0x7f5e80005f70 comp rx=0 tx=0).stop 2024-04-16T17:13:03.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.447+0000 7f5e8d000640 1 -- 172.21.15.45:0/2122918220 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5e8819a9a0 msgr2=0x7f5e8819ae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:03.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.447+0000 7f5e8d000640 1 --2- 172.21.15.45:0/2122918220 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5e8819a9a0 0x7f5e8819ae50 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f5e80009fd0 tx=0x7f5e8000ef10 comp rx=0 tx=0).stop 2024-04-16T17:13:03.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.447+0000 7f5e8d000640 1 -- 172.21.15.45:0/2122918220 shutdown_connections 2024-04-16T17:13:03.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.447+0000 7f5e8d000640 1 --2- 172.21.15.45:0/2122918220 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f5e6c0a3da0 0x7f5e6c0a6260 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:03.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.447+0000 7f5e8d000640 1 --2- 172.21.15.45:0/2122918220 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5e88109580 0x7f5e881a0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:03.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.447+0000 7f5e8d000640 1 --2- 172.21.15.45:0/2122918220 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5e8819a9a0 0x7f5e8819ae50 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:03.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.447+0000 7f5e8d000640 1 --2- 172.21.15.45:0/2122918220 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5e881044b0 0x7f5e881a07f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:03.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.447+0000 7f5e8d000640 1 -- 172.21.15.45:0/2122918220 >> 172.21.15.45:0/2122918220 conn(0x7f5e880ffd90 msgr2=0x7f5e881020f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:13:03.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.447+0000 7f5e8d000640 1 -- 172.21.15.45:0/2122918220 shutdown_connections 2024-04-16T17:13:03.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.447+0000 7f5e8d000640 1 -- 172.21.15.45:0/2122918220 wait complete. 2024-04-16T17:13:03.470 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-16T17:13:03.470 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-16T17:13:03.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.656+0000 7f7a1b200640 1 -- 172.21.15.45:0/2074841944 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7a1c102ac0 msgr2=0x7f7a1c106b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:03.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.656+0000 7f7a1b200640 1 --2- 172.21.15.45:0/2074841944 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7a1c102ac0 0x7f7a1c106b00 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f7a08009860 tx=0x7f7a0802bb70 comp rx=0 tx=0).stop 2024-04-16T17:13:03.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.656+0000 7f7a1b200640 1 -- 172.21.15.45:0/2074841944 shutdown_connections 2024-04-16T17:13:03.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.656+0000 7f7a1b200640 1 --2- 172.21.15.45:0/2074841944 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7a1c1071e0 0x7f7a1c10edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:03.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.656+0000 7f7a1b200640 1 --2- 172.21.15.45:0/2074841944 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7a1c102ac0 0x7f7a1c106b00 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:03.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.656+0000 7f7a1b200640 1 --2- 172.21.15.45:0/2074841944 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7a1c102110 0x7f7a1c1024f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:03.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.656+0000 7f7a1b200640 1 -- 172.21.15.45:0/2074841944 >> 172.21.15.45:0/2074841944 conn(0x7f7a1c0fd9d0 msgr2=0x7f7a1c0ffdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:13:03.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.656+0000 7f7a1b200640 1 -- 172.21.15.45:0/2074841944 shutdown_connections 2024-04-16T17:13:03.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.656+0000 7f7a1b200640 1 -- 172.21.15.45:0/2074841944 wait complete. 2024-04-16T17:13:03.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.657+0000 7f7a1b200640 1 Processor -- start 2024-04-16T17:13:03.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.657+0000 7f7a1b200640 1 -- start start 2024-04-16T17:13:03.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.658+0000 7f7a1b200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7a1c102110 0x7f7a1c1a06e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:03.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.658+0000 7f7a1b200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7a1c1071e0 0x7f7a1c1a0c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:03.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.658+0000 7f7a1b200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7a1c19a890 0x7f7a1c19ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:03.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.658+0000 7f7a1b200640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f7a1c112270 con 0x7f7a1c19a890 2024-04-16T17:13:03.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.658+0000 7f7a1b200640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f7a1c1120f0 con 0x7f7a1c102110 2024-04-16T17:13:03.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.658+0000 7f7a1b200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f7a1c1123f0 con 0x7f7a1c1071e0 2024-04-16T17:13:03.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.658+0000 7f7a19e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7a1c102110 0x7f7a1c1a06e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:03.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.658+0000 7f7a19400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7a1c1071e0 0x7f7a1c1a0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:03.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.658+0000 7f7a19e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7a1c102110 0x7f7a1c1a06e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:36018/0 (socket says 172.21.15.45:36018) 2024-04-16T17:13:03.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.658+0000 7f7a19e00640 1 -- 172.21.15.45:0/3675064157 learned_addr learned my addr 172.21.15.45:0/3675064157 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:13:03.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.658+0000 7f7a1a800640 1 --2- 172.21.15.45:0/3675064157 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7a1c19a890 0x7f7a1c19ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:03.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.659+0000 7f7a19e00640 1 -- 172.21.15.45:0/3675064157 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7a1c1071e0 msgr2=0x7f7a1c1a0c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:03.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.659+0000 7f7a19e00640 1 --2- 172.21.15.45:0/3675064157 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7a1c1071e0 0x7f7a1c1a0c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:03.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.659+0000 7f7a19e00640 1 -- 172.21.15.45:0/3675064157 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7a1c19a890 msgr2=0x7f7a1c19ad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:03.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.659+0000 7f7a19e00640 1 --2- 172.21.15.45:0/3675064157 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7a1c19a890 0x7f7a1c19ad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:03.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.659+0000 7f7a19e00640 1 -- 172.21.15.45:0/3675064157 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a1c19b3b0 con 0x7f7a1c102110 2024-04-16T17:13:03.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.659+0000 7f7a19e00640 1 --2- 172.21.15.45:0/3675064157 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7a1c102110 0x7f7a1c1a06e0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f7a0c00e9c0 tx=0x7f7a0c00ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:13:03.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.659+0000 7f7a12a00640 1 -- 172.21.15.45:0/3675064157 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a0c00cde0 con 0x7f7a1c102110 2024-04-16T17:13:03.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.659+0000 7f7a12a00640 1 -- 172.21.15.45:0/3675064157 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7a0c0035f0 con 0x7f7a1c102110 2024-04-16T17:13:03.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.659+0000 7f7a12a00640 1 -- 172.21.15.45:0/3675064157 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a0c010690 con 0x7f7a1c102110 2024-04-16T17:13:03.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.659+0000 7f7a1b200640 1 -- 172.21.15.45:0/3675064157 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a1c19b6a0 con 0x7f7a1c102110 2024-04-16T17:13:03.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.659+0000 7f7a1b200640 1 -- 172.21.15.45:0/3675064157 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a1c1a78a0 con 0x7f7a1c102110 2024-04-16T17:13:03.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.663+0000 7f7a12a00640 1 -- 172.21.15.45:0/3675064157 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f7a0c0037a0 con 0x7f7a1c102110 2024-04-16T17:13:03.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.663+0000 7f7a23e74740 1 Processor -- start 2024-04-16T17:13:03.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.663+0000 7f7a23e74740 1 -- start start 2024-04-16T17:13:03.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.664+0000 7f7a23e74740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561a13305390 0x561a13305770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:03.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.664+0000 7f7a23e74740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561a13305cb0 0x561a1330e350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:03.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.664+0000 7f7a23e74740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561a1330e890 0x561a13310c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:03.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.664+0000 7f7a23e74740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x561a1326b120 con 0x561a13305390 2024-04-16T17:13:03.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.664+0000 7f7a23e74740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x561a13211340 con 0x561a1330e890 2024-04-16T17:13:03.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.664+0000 7f7a23e74740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x561a132670e0 con 0x561a13305cb0 2024-04-16T17:13:03.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.664+0000 7f7a1a800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561a13305390 0x561a13305770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:03.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.664+0000 7f7a1a800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561a13305390 0x561a13305770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:46926/0 (socket says 172.21.15.45:46926) 2024-04-16T17:13:03.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.664+0000 7f7a1a800640 1 -- 172.21.15.45:0/3592636450 learned_addr learned my addr 172.21.15.45:0/3592636450 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:13:03.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.664+0000 7f7a19400640 1 --2- 172.21.15.45:0/3592636450 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561a1330e890 0x561a13310c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:03.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.664+0000 7f7a19e00640 1 --2- 172.21.15.45:0/3592636450 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561a13305cb0 0x561a1330e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:03.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.665+0000 7f7a12a00640 1 --2- 172.21.15.45:0/3675064157 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f7a000a3d20 0x7f7a000a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:03.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.665+0000 7f7a19400640 1 -- 172.21.15.45:0/3592636450 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561a13305cb0 msgr2=0x561a1330e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:13:03.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.665+0000 7f7a19400640 1 --2- 172.21.15.45:0/3592636450 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561a13305cb0 0x561a1330e350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:03.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.665+0000 7f7a19400640 1 -- 172.21.15.45:0/3592636450 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561a13305390 msgr2=0x561a13305770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:13:03.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.665+0000 7f7a19400640 1 --2- 172.21.15.45:0/3592636450 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561a13305390 0x561a13305770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:03.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.665+0000 7f7a19400640 1 -- 172.21.15.45:0/3592636450 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561a130477d0 con 0x561a1330e890 2024-04-16T17:13:03.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.665+0000 7f7a12a00640 1 -- 172.21.15.45:0/3675064157 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f7a0c0d0ed0 con 0x7f7a1c102110 2024-04-16T17:13:03.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.665+0000 7f7a19400640 1 --2- 172.21.15.45:0/3592636450 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561a1330e890 0x561a13310c80 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f7a0802c0b0 tx=0x7f7a08009510 comp rx=0 tx=0).ready entity=mon.2 client_cookie=40cdba3981c48a31 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:13:03.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.665+0000 7f7a1a800640 1 --2- 172.21.15.45:0/3675064157 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f7a000a3d20 0x7f7a000a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:03.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.665+0000 7f7a07400640 1 -- 172.21.15.45:0/3592636450 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a08043020 con 0x561a1330e890 2024-04-16T17:13:03.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.666+0000 7f7a07400640 1 -- 172.21.15.45:0/3592636450 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7a08034cb0 con 0x561a1330e890 2024-04-16T17:13:03.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.666+0000 7f7a07400640 1 -- 172.21.15.45:0/3592636450 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a08039040 con 0x561a1330e890 2024-04-16T17:13:03.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.666+0000 7f7a23e74740 1 -- 172.21.15.45:0/3592636450 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561a1330e890 msgr2=0x561a13310c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:03.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.666+0000 7f7a1a800640 1 --2- 172.21.15.45:0/3675064157 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f7a000a3d20 0x7f7a000a61e0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f7a14017fd0 tx=0x7f7a14019040 comp rx=0 tx=0).ready entity=mgr.25411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:13:03.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.666+0000 7f7a23e74740 1 --2- 172.21.15.45:0/3592636450 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561a1330e890 0x561a13310c80 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f7a0802c0b0 tx=0x7f7a08009510 comp rx=0 tx=0).stop 2024-04-16T17:13:03.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.667+0000 7f7a23e74740 1 -- 172.21.15.45:0/3592636450 shutdown_connections 2024-04-16T17:13:03.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.667+0000 7f7a23e74740 1 --2- 172.21.15.45:0/3592636450 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561a13305cb0 0x561a1330e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:03.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.667+0000 7f7a23e74740 1 --2- 172.21.15.45:0/3592636450 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561a1330e890 0x561a13310c80 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:03.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.667+0000 7f7a23e74740 1 --2- 172.21.15.45:0/3592636450 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561a13305390 0x561a13305770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:03.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.667+0000 7f7a23e74740 1 -- 172.21.15.45:0/3592636450 >> 172.21.15.45:0/3592636450 conn(0x561a132ffac0 msgr2=0x561a132ffea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:13:03.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.667+0000 7f7a23e74740 1 -- 172.21.15.45:0/3592636450 shutdown_connections 2024-04-16T17:13:03.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.667+0000 7f7a23e74740 1 -- 172.21.15.45:0/3592636450 wait complete. 2024-04-16T17:13:03.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.667+0000 7f7a23e74740 1 -- start start 2024-04-16T17:13:03.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.667+0000 7f7a23e74740 1 Processor -- start 2024-04-16T17:13:03.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.668+0000 7f7a23e74740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561a13305390 0x561a1330b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:03.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.668+0000 7f7a23e74740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561a13305cb0 0x561a1330be20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:03.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.668+0000 7f7a23e74740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561a1330e890 0x561a1330c360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:03.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.668+0000 7f7a23e74740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x561a13311620 con 0x561a13305390 2024-04-16T17:13:03.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.668+0000 7f7a23e74740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x561a13224710 con 0x561a13305cb0 2024-04-16T17:13:03.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.668+0000 7f7a23e74740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x561a133117a0 con 0x561a1330e890 2024-04-16T17:13:03.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.668+0000 7f7a23e74740 10 client.0 _ll_register_callbacks cb 0x561a1321faf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:13:03.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.668+0000 7f7a19e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561a13305390 0x561a1330b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:03.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.668+0000 7f7a19e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561a13305390 0x561a1330b8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:46932/0 (socket says 172.21.15.45:46932) 2024-04-16T17:13:03.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.668+0000 7f7a19e00640 1 -- 172.21.15.45:0/1762353370 learned_addr learned my addr 172.21.15.45:0/1762353370 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:13:03.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.668+0000 7f7a19400640 1 --2- 172.21.15.45:0/1762353370 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561a13305cb0 0x561a1330be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:03.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.669+0000 7f7a1a800640 1 --2- 172.21.15.45:0/1762353370 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561a1330e890 0x561a1330c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:03.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.669+0000 7f7a19400640 1 -- 172.21.15.45:0/1762353370 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561a1330e890 msgr2=0x561a1330c360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:13:03.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.669+0000 7f7a19400640 1 --2- 172.21.15.45:0/1762353370 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561a1330e890 0x561a1330c360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:03.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.669+0000 7f7a19400640 1 -- 172.21.15.45:0/1762353370 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561a13305390 msgr2=0x561a1330b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:13:03.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.669+0000 7f7a19400640 1 --2- 172.21.15.45:0/1762353370 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561a13305390 0x561a1330b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:03.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.669+0000 7f7a19400640 1 -- 172.21.15.45:0/1762353370 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561a12f7d3f0 con 0x561a13305cb0 2024-04-16T17:13:03.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.669+0000 7f7a19e00640 1 --2- 172.21.15.45:0/1762353370 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561a13305390 0x561a1330b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:13:03.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.669+0000 7f7a19400640 1 --2- 172.21.15.45:0/1762353370 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561a13305cb0 0x561a1330be20 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f7a08000c00 tx=0x7f7a08005b20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7ee5a29db099a615 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:13:03.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.669+0000 7f79fbe00640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-04-16T17:13:03.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.670+0000 7f79fbe00640 1 -- 172.21.15.45:0/1762353370 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a08043030 con 0x561a13305cb0 2024-04-16T17:13:03.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.670+0000 7f79fbe00640 1 -- 172.21.15.45:0/1762353370 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f7a0802c510 con 0x561a13305cb0 2024-04-16T17:13:03.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.670+0000 7f7a23e74740 1 -- 172.21.15.45:0/1762353370 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7a08034cb0 con 0x561a13305cb0 2024-04-16T17:13:03.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.670+0000 7f79fbe00640 1 -- 172.21.15.45:0/1762353370 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f7a08034e60 con 0x561a13305cb0 2024-04-16T17:13:03.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.670+0000 7f7a23e74740 10 client.27386 fetch_fsmap learned FSMap version 37 2024-04-16T17:13:03.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.670+0000 7f7a23e74740 1 -- 172.21.15.45:0/1762353370 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x561a12f3d0f0 con 0x561a13305cb0 2024-04-16T17:13:03.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.671+0000 7f79fbe00640 1 -- 172.21.15.45:0/1762353370 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7a08034cb0 con 0x561a13305cb0 2024-04-16T17:13:03.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.672+0000 7f79fbe00640 1 -- 172.21.15.45:0/1762353370 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 37) ==== 8358+0+0 (secure 0 0 0) 0x7f7a08032910 con 0x561a13305cb0 2024-04-16T17:13:03.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.672+0000 7f7a23e74740 10 client.27386 fetch_fsmap finished waiting for FSMap version 37 2024-04-16T17:13:03.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.672+0000 7f7a23e74740 10 client.27386 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:13:03.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.672+0000 7f7a23e74740 20 client.27386 populate_metadata read hostname 'smithi045' 2024-04-16T17:13:03.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.672+0000 7f7a23e74740 1 --2- 172.21.15.45:0/1762353370 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x561a1331ea60 0x561a13320e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:03.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.672+0000 7f7a23e74740 4 client.27386 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:13:03.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.672+0000 7f7a23e74740 1 -- 172.21.15.45:0/1762353370 --> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561a13278b40 con 0x561a1331ea60 2024-04-16T17:13:03.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.677+0000 7f7a19e00640 1 --2- 172.21.15.45:0/1762353370 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x561a1331ea60 0x561a13320e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:03.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.699+0000 7f7a19e00640 1 --2- 172.21.15.45:0/1762353370 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x561a1331ea60 0x561a13320e50 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d7cc605a76659781 server_cookie=c086b67196502748 in_seq=0 out_seq=0 2024-04-16T17:13:03.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.699+0000 7f79fbe00640 10 client.27386 ms_handle_connect on v2:172.21.15.104:6838/3660070592 2024-04-16T17:13:03.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.714+0000 7f79fbe00640 1 -- 172.21.15.45:0/1762353370 <== mds.0 v2:172.21.15.104:6838/3660070592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561a13278b40 con 0x561a1331ea60 2024-04-16T17:13:03.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.714+0000 7f79fbe00640 10 client.27386 handle_command_reply: tid=0 2024-04-16T17:13:03.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.714+0000 7f7a23e74740 1 client.27386 shutdown 2024-04-16T17:13:03.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.715+0000 7f7a23e74740 1 -- 172.21.15.45:0/1762353370 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561a13305cb0 msgr2=0x561a1330be20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:03.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.715+0000 7f7a23e74740 1 --2- 172.21.15.45:0/1762353370 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561a13305cb0 0x561a1330be20 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f7a08000c00 tx=0x7f7a08005b20 comp rx=0 tx=0).stop 2024-04-16T17:13:03.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.715+0000 7f7a23e74740 1 -- 172.21.15.45:0/1762353370 shutdown_connections 2024-04-16T17:13:03.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.715+0000 7f7a23e74740 1 --2- 172.21.15.45:0/1762353370 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x561a1331ea60 0x561a13320e50 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:03.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.715+0000 7f7a23e74740 1 --2- 172.21.15.45:0/1762353370 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561a1330e890 0x561a1330c360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:03.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.715+0000 7f7a23e74740 1 --2- 172.21.15.45:0/1762353370 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561a13305cb0 0x561a1330be20 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:03.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.715+0000 7f7a23e74740 1 --2- 172.21.15.45:0/1762353370 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561a13305390 0x561a1330b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:03.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.715+0000 7f79fbe00640 0 client.27386 ms_handle_reset on v2:172.21.15.104:6838/3660070592 2024-04-16T17:13:03.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.715+0000 7f7a23e74740 1 -- 172.21.15.45:0/1762353370 >> 172.21.15.45:0/1762353370 conn(0x561a132ffac0 msgr2=0x561a13301810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:13:03.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.715+0000 7f7a23e74740 1 -- 172.21.15.45:0/1762353370 shutdown_connections 2024-04-16T17:13:03.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.715+0000 7f7a23e74740 1 -- 172.21.15.45:0/1762353370 wait complete. 2024-04-16T17:13:03.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.715+0000 7f7a23e74740 20 client.27386 trim_cache size 0 max 16384 2024-04-16T17:13:03.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.742+0000 7f7a23e74740 1 Processor -- start 2024-04-16T17:13:03.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.742+0000 7f7a23e74740 1 -- start start 2024-04-16T17:13:03.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.742+0000 7f7a23e74740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561a13305390 0x561a1330e010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:03.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.742+0000 7f7a23e74740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561a13305cb0 0x561a132ffea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:03.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.742+0000 7f7a23e74740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561a1330e890 0x561a1331a100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:03.743 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.742+0000 7f7a23e74740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f7a1c1afa90 con 0x561a1330e890 2024-04-16T17:13:03.743 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.742+0000 7f7a23e74740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f7a1c1ad2d0 con 0x561a13305390 2024-04-16T17:13:03.743 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.742+0000 7f7a23e74740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f7a1c1ab140 con 0x561a13305cb0 2024-04-16T17:13:03.743 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.743+0000 7f7a1a800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561a1330e890 0x561a1331a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:03.743 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.743+0000 7f7a19400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561a13305cb0 0x561a132ffea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:03.744 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.743+0000 7f7a1a800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561a1330e890 0x561a1331a100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:46944/0 (socket says 172.21.15.45:46944) 2024-04-16T17:13:03.744 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.743+0000 7f7a19400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561a13305cb0 0x561a132ffea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:54198/0 (socket says 172.21.15.45:54198) 2024-04-16T17:13:03.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.743+0000 7f7a1a800640 1 -- 172.21.15.45:0/3063859315 learned_addr learned my addr 172.21.15.45:0/3063859315 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:13:03.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.743+0000 7f7a19400640 1 -- 172.21.15.45:0/3063859315 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561a13305390 msgr2=0x561a1330e010 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-16T17:13:03.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.743+0000 7f7a19400640 1 --2- 172.21.15.45:0/3063859315 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561a13305390 0x561a1330e010 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:03.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.743+0000 7f7a19400640 1 -- 172.21.15.45:0/3063859315 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561a1330e890 msgr2=0x561a1331a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:13:03.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.743+0000 7f7a19400640 1 --2- 172.21.15.45:0/3063859315 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561a1330e890 0x561a1331a100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:03.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.743+0000 7f7a19400640 1 -- 172.21.15.45:0/3063859315 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561a13310ff0 con 0x561a13305cb0 2024-04-16T17:13:03.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.743+0000 7f7a19400640 1 --2- 172.21.15.45:0/3063859315 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561a13305cb0 0x561a132ffea0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f7a08032d60 tx=0x7f7a080056d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a96fc19b2a6302cb server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:13:03.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.744+0000 7f7a06a00640 1 -- 172.21.15.45:0/3063859315 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a08043070 con 0x561a13305cb0 2024-04-16T17:13:03.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.744+0000 7f7a06a00640 1 -- 172.21.15.45:0/3063859315 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7a08004350 con 0x561a13305cb0 2024-04-16T17:13:03.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.744+0000 7f7a06a00640 1 -- 172.21.15.45:0/3063859315 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a08039040 con 0x561a13305cb0 2024-04-16T17:13:03.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.744+0000 7f7a23e74740 1 -- 172.21.15.45:0/3063859315 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561a13305cb0 msgr2=0x561a132ffea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:03.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.744+0000 7f7a23e74740 1 --2- 172.21.15.45:0/3063859315 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561a13305cb0 0x561a132ffea0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f7a08032d60 tx=0x7f7a080056d0 comp rx=0 tx=0).stop 2024-04-16T17:13:03.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.744+0000 7f7a23e74740 1 -- 172.21.15.45:0/3063859315 shutdown_connections 2024-04-16T17:13:03.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.744+0000 7f7a23e74740 1 --2- 172.21.15.45:0/3063859315 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561a13305cb0 0x561a132ffea0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:03.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.744+0000 7f7a23e74740 1 --2- 172.21.15.45:0/3063859315 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561a1330e890 0x561a1331a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:03.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.744+0000 7f7a23e74740 1 --2- 172.21.15.45:0/3063859315 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561a13305390 0x561a1330e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:03.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.744+0000 7f7a23e74740 1 -- 172.21.15.45:0/3063859315 >> 172.21.15.45:0/3063859315 conn(0x561a132ffac0 msgr2=0x561a133199f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:13:03.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.744+0000 7f7a23e74740 1 -- 172.21.15.45:0/3063859315 shutdown_connections 2024-04-16T17:13:03.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.745+0000 7f7a23e74740 1 -- 172.21.15.45:0/3063859315 wait complete. 2024-04-16T17:13:03.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.745+0000 7f7a23e74740 1 -- start start 2024-04-16T17:13:03.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.745+0000 7f7a23e74740 1 Processor -- start 2024-04-16T17:13:03.748 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.745+0000 7f7a23e74740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561a13305390 0x561a13321830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:03.748 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.746+0000 7f7a23e74740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561a13305cb0 0x561a13321d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:03.748 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.746+0000 7f7a23e74740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561a1330e890 0x561a133222b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:03.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.746+0000 7f7a23e74740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f7a1c1ab790 con 0x561a13305390 2024-04-16T17:13:03.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.746+0000 7f7a23e74740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f7a1c1acc00 con 0x561a1330e890 2024-04-16T17:13:03.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.746+0000 7f7a23e74740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f7a1c1abe60 con 0x561a13305cb0 2024-04-16T17:13:03.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.746+0000 7f7a23e74740 10 client.0 _ll_register_callbacks cb 0x7f7a14017bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:13:03.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.746+0000 7f7a19e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561a13305390 0x561a13321830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:03.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.746+0000 7f7a19400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561a13305cb0 0x561a13321d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:03.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.746+0000 7f7a19400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561a13305cb0 0x561a13321d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:54208/0 (socket says 172.21.15.45:54208) 2024-04-16T17:13:03.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.746+0000 7f7a19400640 1 -- 172.21.15.45:0/3444591339 learned_addr learned my addr 172.21.15.45:0/3444591339 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:13:03.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.746+0000 7f7a1a800640 1 --2- 172.21.15.45:0/3444591339 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561a1330e890 0x561a133222b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:03.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.746+0000 7f7a19e00640 1 -- 172.21.15.45:0/3444591339 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561a13305cb0 msgr2=0x561a13321d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:13:03.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.746+0000 7f7a19e00640 1 --2- 172.21.15.45:0/3444591339 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561a13305cb0 0x561a13321d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:03.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.746+0000 7f7a19e00640 1 -- 172.21.15.45:0/3444591339 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561a1330e890 msgr2=0x561a133222b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:13:03.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.746+0000 7f7a19e00640 1 --2- 172.21.15.45:0/3444591339 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561a1330e890 0x561a133222b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:03.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.746+0000 7f7a19e00640 1 -- 172.21.15.45:0/3444591339 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561a13383c40 con 0x561a13305390 2024-04-16T17:13:03.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.747+0000 7f7a19e00640 1 --2- 172.21.15.45:0/3444591339 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561a13305390 0x561a13321830 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f7a0c098f20 tx=0x7f7a0c0bd040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c539e118a0edf560 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:13:03.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.747+0000 7f79fbe00640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-16T17:13:03.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.747+0000 7f79fbe00640 1 -- 172.21.15.45:0/3444591339 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a0c09c600 con 0x561a13305390 2024-04-16T17:13:03.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.747+0000 7f79fbe00640 1 -- 172.21.15.45:0/3444591339 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f7a0c09adc0 con 0x561a13305390 2024-04-16T17:13:03.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.747+0000 7f7a23e74740 1 -- 172.21.15.45:0/3444591339 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7a08004350 con 0x561a13305390 2024-04-16T17:13:03.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.747+0000 7f79fbe00640 1 -- 172.21.15.45:0/3444591339 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f7a0c09c420 con 0x561a13305390 2024-04-16T17:13:03.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.747+0000 7f7a23e74740 10 client.17220 fetch_fsmap learned FSMap version 37 2024-04-16T17:13:03.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.747+0000 7f7a23e74740 1 -- 172.21.15.45:0/3444591339 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x561a13384e60 con 0x561a13305390 2024-04-16T17:13:03.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.747+0000 7f79fbe00640 1 -- 172.21.15.45:0/3444591339 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7a08004350 con 0x561a13305390 2024-04-16T17:13:03.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.748+0000 7f79fbe00640 1 -- 172.21.15.45:0/3444591339 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 37) ==== 8358+0+0 (secure 0 0 0) 0x7f7a0c0ce4f0 con 0x561a13305390 2024-04-16T17:13:03.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.748+0000 7f7a23e74740 10 client.17220 fetch_fsmap finished waiting for FSMap version 37 2024-04-16T17:13:03.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.748+0000 7f7a23e74740 10 client.17220 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:13:03.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.748+0000 7f7a23e74740 20 client.17220 populate_metadata read hostname 'smithi045' 2024-04-16T17:13:03.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.748+0000 7f7a23e74740 1 --2- 172.21.15.45:0/3444591339 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x561a133854e0 0x561a13387900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:03.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.748+0000 7f7a23e74740 4 client.17220 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-16T17:13:03.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.748+0000 7f7a23e74740 1 -- 172.21.15.45:0/3444591339 --> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] -- command(tid 0: {"prefix": "scrub status"}) -- 0x561a132729b0 con 0x561a133854e0 2024-04-16T17:13:03.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.748+0000 7f7a19400640 1 --2- 172.21.15.45:0/3444591339 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x561a133854e0 0x561a13387900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:03.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.752+0000 7f7a19400640 1 --2- 172.21.15.45:0/3444591339 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x561a133854e0 0x561a13387900 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=13dd2a4c6a93b5a4 server_cookie=24f715df9f9e5a52 in_seq=0 out_seq=0 2024-04-16T17:13:03.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.752+0000 7f79fbe00640 10 client.17220 ms_handle_connect on v2:172.21.15.104:6838/3660070592 2024-04-16T17:13:03.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.752+0000 7f79fbe00640 1 -- 172.21.15.45:0/3444591339 <== mds.0 v2:172.21.15.104:6838/3660070592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x561a132729b0 con 0x561a133854e0 2024-04-16T17:13:03.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.752+0000 7f79fbe00640 10 client.17220 handle_command_reply: tid=0 2024-04-16T17:13:03.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.752+0000 7f7a23e74740 1 client.17220 shutdown 2024-04-16T17:13:03.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.752+0000 7f7a23e74740 1 -- 172.21.15.45:0/3444591339 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561a13305390 msgr2=0x561a13321830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:03.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.752+0000 7f7a23e74740 1 --2- 172.21.15.45:0/3444591339 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561a13305390 0x561a13321830 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f7a0c098f20 tx=0x7f7a0c0bd040 comp rx=0 tx=0).stop 2024-04-16T17:13:03.753 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.752+0000 7f7a23e74740 1 -- 172.21.15.45:0/3444591339 shutdown_connections 2024-04-16T17:13:03.753 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.752+0000 7f7a23e74740 1 --2- 172.21.15.45:0/3444591339 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x561a133854e0 0x561a13387900 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:03.753 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.752+0000 7f7a23e74740 1 --2- 172.21.15.45:0/3444591339 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561a13305cb0 0x561a13321d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:03.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.752+0000 7f7a23e74740 1 --2- 172.21.15.45:0/3444591339 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561a1330e890 0x561a133222b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:03.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.752+0000 7f7a23e74740 1 --2- 172.21.15.45:0/3444591339 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561a13305390 0x561a13321830 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:03.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.752+0000 7f79fbe00640 0 client.17220 ms_handle_reset on v2:172.21.15.104:6838/3660070592 2024-04-16T17:13:03.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.752+0000 7f7a23e74740 1 -- 172.21.15.45:0/3444591339 >> 172.21.15.45:0/3444591339 conn(0x561a132ffac0 msgr2=0x561a1330b880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:13:03.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.752+0000 7f7a23e74740 1 -- 172.21.15.45:0/3444591339 shutdown_connections 2024-04-16T17:13:03.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.752+0000 7f7a23e74740 1 -- 172.21.15.45:0/3444591339 wait complete. 2024-04-16T17:13:03.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.753+0000 7f7a23e74740 20 client.17220 trim_cache size 0 max 16384 2024-04-16T17:13:03.754 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-16T17:13:03.754 INFO:teuthology.orchestra.run.smithi045.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-16T17:13:03.754 INFO:teuthology.orchestra.run.smithi045.stdout: "scrubs": { 2024-04-16T17:13:03.754 INFO:teuthology.orchestra.run.smithi045.stdout: "68131f8b-72dd-4619-a321-eebff406a4de": { 2024-04-16T17:13:03.755 INFO:teuthology.orchestra.run.smithi045.stdout: "path": "/", 2024-04-16T17:13:03.755 INFO:teuthology.orchestra.run.smithi045.stdout: "tag": "68131f8b-72dd-4619-a321-eebff406a4de", 2024-04-16T17:13:03.755 INFO:teuthology.orchestra.run.smithi045.stdout: "options": "recursive,force" 2024-04-16T17:13:03.755 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:13:03.755 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:13:03.755 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-16T17:13:03.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.753+0000 7f7a1b200640 1 -- 172.21.15.45:0/3675064157 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f7a000a3d20 msgr2=0x7f7a000a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:03.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.753+0000 7f7a1b200640 1 --2- 172.21.15.45:0/3675064157 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f7a000a3d20 0x7f7a000a61e0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f7a14017fd0 tx=0x7f7a14019040 comp rx=0 tx=0).stop 2024-04-16T17:13:03.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.753+0000 7f7a1b200640 1 -- 172.21.15.45:0/3675064157 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7a1c102110 msgr2=0x7f7a1c1a06e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:03.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.753+0000 7f7a1b200640 1 --2- 172.21.15.45:0/3675064157 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7a1c102110 0x7f7a1c1a06e0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f7a0c00e9c0 tx=0x7f7a0c00ee80 comp rx=0 tx=0).stop 2024-04-16T17:13:03.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.753+0000 7f7a1b200640 1 -- 172.21.15.45:0/3675064157 shutdown_connections 2024-04-16T17:13:03.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.753+0000 7f7a1b200640 1 --2- 172.21.15.45:0/3675064157 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f7a000a3d20 0x7f7a000a61e0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:03.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.753+0000 7f7a1b200640 1 --2- 172.21.15.45:0/3675064157 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7a1c1071e0 0x7f7a1c1a0c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:03.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.753+0000 7f7a1b200640 1 --2- 172.21.15.45:0/3675064157 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7a1c19a890 0x7f7a1c19ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:03.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.753+0000 7f7a1b200640 1 --2- 172.21.15.45:0/3675064157 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7a1c102110 0x7f7a1c1a06e0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:03.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.753+0000 7f7a1b200640 1 -- 172.21.15.45:0/3675064157 >> 172.21.15.45:0/3675064157 conn(0x7f7a1c0fd9d0 msgr2=0x7f7a1c0ffdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:13:03.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.753+0000 7f7a1b200640 1 -- 172.21.15.45:0/3675064157 shutdown_connections 2024-04-16T17:13:03.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:03.754+0000 7f7a1b200640 1 -- 172.21.15.45:0/3675064157 wait complete. 2024-04-16T17:13:03.768 INFO:tasks.cephfs.filesystem:scrub status for tag:68131f8b-72dd-4619-a321-eebff406a4de - {'path': '/', 'tag': '68131f8b-72dd-4619-a321-eebff406a4de', 'options': 'recursive,force'} 2024-04-16T17:13:04.275 INFO:tasks.workunit.client.0.smithi045.stdout: 1 101567 1.68 MB/sec execute 186 sec latency 121.372 ms 2024-04-16T17:13:04.304 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:13:03 smithi117 ceph-mon[36730]: pgmap v192: 129 pgs: 129 active+clean; 55 MiB data, 736 MiB used, 1.0 TiB / 1.0 TiB avail; 516 KiB/s rd, 319 KiB/s wr, 63 op/s 2024-04-16T17:13:04.304 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:13:03 smithi117 ceph-mon[36730]: scrub queued for path: / 2024-04-16T17:13:04.304 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:13:03 smithi117 ceph-mon[36730]: scrub summary: idle+waiting paths [/] 2024-04-16T17:13:04.305 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:13:03 smithi117 ceph-mon[36730]: scrub summary: active paths [/] 2024-04-16T17:13:04.305 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:13:03 smithi117 ceph-mon[36730]: scrub summary: active paths [/] 2024-04-16T17:13:04.305 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:13:03 smithi117 ceph-mon[36730]: scrub summary: idle+waiting paths [/] 2024-04-16T17:13:04.305 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:13:03 smithi117 ceph-mon[36730]: scrub summary: idle+waiting paths [/] 2024-04-16T17:13:04.427 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:13:03 smithi104 ceph-mon[36457]: pgmap v192: 129 pgs: 129 active+clean; 55 MiB data, 736 MiB used, 1.0 TiB / 1.0 TiB avail; 516 KiB/s rd, 319 KiB/s wr, 63 op/s 2024-04-16T17:13:04.427 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:13:03 smithi104 ceph-mon[36457]: scrub queued for path: / 2024-04-16T17:13:04.427 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:13:03 smithi104 ceph-mon[36457]: scrub summary: idle+waiting paths [/] 2024-04-16T17:13:04.427 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:13:03 smithi104 ceph-mon[36457]: scrub summary: active paths [/] 2024-04-16T17:13:04.427 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:13:03 smithi104 ceph-mon[36457]: scrub summary: active paths [/] 2024-04-16T17:13:04.427 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:13:03 smithi104 ceph-mon[36457]: scrub summary: idle+waiting paths [/] 2024-04-16T17:13:04.428 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:13:03 smithi104 ceph-mon[36457]: scrub summary: idle+waiting paths [/] 2024-04-16T17:13:04.600 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:13:02 smithi045 ceph-mon[31676]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:13:04.600 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:13:02 smithi045 ceph-mon[31676]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T17:13:04.600 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:13:02 smithi045 ceph-mon[31676]: from='mgr.25411 ' entity='mgr.x' 2024-04-16T17:13:04.600 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:13:03 smithi045 ceph-mon[31676]: pgmap v192: 129 pgs: 129 active+clean; 55 MiB data, 736 MiB used, 1.0 TiB / 1.0 TiB avail; 516 KiB/s rd, 319 KiB/s wr, 63 op/s 2024-04-16T17:13:04.600 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:13:03 smithi045 ceph-mon[31676]: scrub queued for path: / 2024-04-16T17:13:04.601 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:13:03 smithi045 ceph-mon[31676]: scrub summary: idle+waiting paths [/] 2024-04-16T17:13:04.601 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:13:03 smithi045 ceph-mon[31676]: scrub summary: active paths [/] 2024-04-16T17:13:04.601 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:13:03 smithi045 ceph-mon[31676]: scrub summary: active paths [/] 2024-04-16T17:13:04.601 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:13:03 smithi045 ceph-mon[31676]: scrub summary: idle+waiting paths [/] 2024-04-16T17:13:04.601 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:13:03 smithi045 ceph-mon[31676]: scrub summary: idle+waiting paths [/] 2024-04-16T17:13:05.114 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:13:04 smithi045 ceph-mon[31676]: scrub summary: active paths [/] 2024-04-16T17:13:05.114 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:13:04 smithi045 ceph-mon[31676]: scrub summary: idle+waiting paths [/] 2024-04-16T17:13:05.114 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:13:04 smithi045 ceph-mon[31676]: osdmap e96: 12 total, 12 up, 12 in 2024-04-16T17:13:05.275 INFO:tasks.workunit.client.0.smithi045.stdout: 1 101750 1.68 MB/sec execute 187 sec latency 122.197 ms 2024-04-16T17:13:05.304 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:13:04 smithi117 ceph-mon[36730]: scrub summary: active paths [/] 2024-04-16T17:13:05.304 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:13:04 smithi117 ceph-mon[36730]: scrub summary: idle+waiting paths [/] 2024-04-16T17:13:05.304 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:13:04 smithi117 ceph-mon[36730]: osdmap e96: 12 total, 12 up, 12 in 2024-04-16T17:13:05.427 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:13:04 smithi104 ceph-mon[36457]: scrub summary: active paths [/] 2024-04-16T17:13:05.427 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:13:04 smithi104 ceph-mon[36457]: scrub summary: idle+waiting paths [/] 2024-04-16T17:13:05.427 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:13:04 smithi104 ceph-mon[36457]: osdmap e96: 12 total, 12 up, 12 in 2024-04-16T17:13:06.275 INFO:tasks.workunit.client.0.smithi045.stdout: 1 102917 1.73 MB/sec execute 188 sec latency 25.364 ms 2024-04-16T17:13:06.304 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:13:05 smithi117 ceph-mon[36730]: pgmap v194: 129 pgs: 129 active+clean; 55 MiB data, 736 MiB used, 1.0 TiB / 1.0 TiB avail; 619 KiB/s rd, 383 KiB/s wr, 75 op/s 2024-04-16T17:13:06.427 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:13:05 smithi104 ceph-mon[36457]: pgmap v194: 129 pgs: 129 active+clean; 55 MiB data, 736 MiB used, 1.0 TiB / 1.0 TiB avail; 619 KiB/s rd, 383 KiB/s wr, 75 op/s 2024-04-16T17:13:06.685 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:13:05 smithi045 ceph-mon[31676]: pgmap v194: 129 pgs: 129 active+clean; 55 MiB data, 736 MiB used, 1.0 TiB / 1.0 TiB avail; 619 KiB/s rd, 383 KiB/s wr, 75 op/s 2024-04-16T17:13:07.275 INFO:tasks.workunit.client.0.smithi045.stdout: 1 103307 1.73 MB/sec execute 189 sec latency 17.318 ms 2024-04-16T17:13:07.777 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:07.775+0000 7f380dc00640 1 -- 172.21.15.45:0/4036435130 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3808104460 msgr2=0x7f3808104840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:07.778 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:07.775+0000 7f380dc00640 1 --2- 172.21.15.45:0/4036435130 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3808104460 0x7f3808104840 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f37f8009900 tx=0x7f37f802bc40 comp rx=0 tx=0).stop 2024-04-16T17:13:07.778 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:07.775+0000 7f380dc00640 1 -- 172.21.15.45:0/4036435130 shutdown_connections 2024-04-16T17:13:07.778 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:07.775+0000 7f380dc00640 1 --2- 172.21.15.45:0/4036435130 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3808104e10 0x7f3808108e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:07.778 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:07.775+0000 7f380dc00640 1 --2- 172.21.15.45:0/4036435130 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3808109530 0x7f3808111100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:07.782 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:07.775+0000 7f380dc00640 1 --2- 172.21.15.45:0/4036435130 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3808104460 0x7f3808104840 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:07.782 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:07.775+0000 7f380dc00640 1 -- 172.21.15.45:0/4036435130 >> 172.21.15.45:0/4036435130 conn(0x7f38080ffd00 msgr2=0x7f3808102120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:13:07.782 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:07.775+0000 7f380dc00640 1 -- 172.21.15.45:0/4036435130 shutdown_connections 2024-04-16T17:13:07.782 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:07.775+0000 7f380dc00640 1 -- 172.21.15.45:0/4036435130 wait complete. 2024-04-16T17:13:07.782 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:07.775+0000 7f380dc00640 1 Processor -- start 2024-04-16T17:13:07.782 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:07.775+0000 7f380dc00640 1 -- start start 2024-04-16T17:13:07.782 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:07.775+0000 7f380dc00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3808104460 0x7f38081a0700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:07.782 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:07.775+0000 7f380dc00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3808109530 0x7f38081a0c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:07.783 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:07.775+0000 7f380dc00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f380819a8b0 0x7f380819ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:07.783 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:07.775+0000 7f380dc00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f3808114630 con 0x7f3808104460 2024-04-16T17:13:07.783 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:07.775+0000 7f380dc00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f38081144b0 con 0x7f3808109530 2024-04-16T17:13:07.783 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:07.775+0000 7f380dc00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f38081147b0 con 0x7f380819a8b0 2024-04-16T17:13:07.783 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:07.775+0000 7f3806000640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3808109530 0x7f38081a0c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:07.783 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:07.775+0000 7f3806000640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3808109530 0x7f38081a0c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:36068/0 (socket says 172.21.15.45:36068) 2024-04-16T17:13:07.783 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:07.775+0000 7f3806000640 1 -- 172.21.15.45:0/1885830492 learned_addr learned my addr 172.21.15.45:0/1885830492 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:13:07.783 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:07.775+0000 7f3806a00640 1 --2- 172.21.15.45:0/1885830492 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3808104460 0x7f38081a0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:07.783 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:07.775+0000 7f3807400640 1 --2- 172.21.15.45:0/1885830492 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f380819a8b0 0x7f380819ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:07.783 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:07.780+0000 7f3806000640 1 -- 172.21.15.45:0/1885830492 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f380819a8b0 msgr2=0x7f380819ad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:07.784 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:07.780+0000 7f3806000640 1 --2- 172.21.15.45:0/1885830492 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f380819a8b0 0x7f380819ad60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:07.784 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:07.780+0000 7f3806000640 1 -- 172.21.15.45:0/1885830492 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3808104460 msgr2=0x7f38081a0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:07.784 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:07.780+0000 7f3806000640 1 --2- 172.21.15.45:0/1885830492 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3808104460 0x7f38081a0700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:07.784 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:07.780+0000 7f3806000640 1 -- 172.21.15.45:0/1885830492 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f380819b3d0 con 0x7f3808109530 2024-04-16T17:13:07.784 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:07.780+0000 7f3807400640 1 --2- 172.21.15.45:0/1885830492 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f380819a8b0 0x7f380819ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:13:07.784 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:07.780+0000 7f3806000640 1 --2- 172.21.15.45:0/1885830492 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3808109530 0x7f38081a0c40 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f37f400e9f0 tx=0x7f37f400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:13:07.784 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:07.780+0000 7f37ff400640 1 -- 172.21.15.45:0/1885830492 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37f400ccc0 con 0x7f3808109530 2024-04-16T17:13:07.784 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:07.780+0000 7f380dc00640 1 -- 172.21.15.45:0/1885830492 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f380819b6c0 con 0x7f3808109530 2024-04-16T17:13:07.784 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:07.780+0000 7f37ff400640 1 -- 172.21.15.45:0/1885830492 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f37f4005500 con 0x7f3808109530 2024-04-16T17:13:07.784 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:07.780+0000 7f37ff400640 1 -- 172.21.15.45:0/1885830492 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37f4010690 con 0x7f3808109530 2024-04-16T17:13:07.784 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:07.780+0000 7f380dc00640 1 -- 172.21.15.45:0/1885830492 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38081a78c0 con 0x7f3808109530 2024-04-16T17:13:07.795 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:07.780+0000 7f381085d740 1 Processor -- start 2024-04-16T17:13:07.796 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:07.780+0000 7f381085d740 1 -- start start 2024-04-16T17:13:07.796 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:07.780+0000 7f381085d740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5613451e6610 0x5613451e5360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:07.796 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:07.780+0000 7f381085d740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5613451e58a0 0x5613451e5d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:07.796 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:07.780+0000 7f381085d740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5613451eee20 0x5613451f1210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:07.796 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:07.780+0000 7f381085d740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x56134514a780 con 0x5613451e6610 2024-04-16T17:13:07.796 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:07.780+0000 7f381085d740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5613450f0f40 con 0x5613451eee20 2024-04-16T17:13:07.796 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:07.780+0000 7f381085d740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x561345146740 con 0x5613451e58a0 2024-04-16T17:13:07.796 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:07.780+0000 7f3807400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5613451e6610 0x5613451e5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:07.796 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:07.780+0000 7f3806a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5613451e58a0 0x5613451e5d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:07.797 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:07.780+0000 7f3807400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5613451e6610 0x5613451e5360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:46992/0 (socket says 172.21.15.45:46992) 2024-04-16T17:13:07.797 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:07.780+0000 7f3807400640 1 -- 172.21.15.45:0/665849321 learned_addr learned my addr 172.21.15.45:0/665849321 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:13:07.797 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:07.780+0000 7f3806a00640 1 -- 172.21.15.45:0/665849321 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5613451eee20 msgr2=0x5613451f1210 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-16T17:13:07.797 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:07.780+0000 7f3806a00640 1 --2- 172.21.15.45:0/665849321 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5613451eee20 0x5613451f1210 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:07.797 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:07.780+0000 7f3806a00640 1 -- 172.21.15.45:0/665849321 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5613451e6610 msgr2=0x5613451e5360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:13:07.797 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:07.780+0000 7f3806a00640 1 --2- 172.21.15.45:0/665849321 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5613451e6610 0x5613451e5360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:07.797 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:07.780+0000 7f3806a00640 1 -- 172.21.15.45:0/665849321 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561344e5d210 con 0x5613451e58a0 2024-04-16T17:13:07.797 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:07.786+0000 7f37ff400640 1 -- 172.21.15.45:0/1885830492 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f37f40046c0 con 0x7f3808109530 2024-04-16T17:13:07.797 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:07.786+0000 7f3806a00640 1 --2- 172.21.15.45:0/665849321 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5613451e58a0 0x5613451e5d20 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f37f802c180 tx=0x7f37f8000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6f6b7983b441250b server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:13:07.797 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:07.786+0000 7f37f3e00640 1 -- 172.21.15.45:0/665849321 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37f8042070 con 0x5613451e58a0 2024-04-16T17:13:07.798 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:07.786+0000 7f37f3e00640 1 -- 172.21.15.45:0/665849321 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f37f802c890 con 0x5613451e58a0 2024-04-16T17:13:07.798 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:07.786+0000 7f37f3e00640 1 -- 172.21.15.45:0/665849321 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37f8004bd0 con 0x5613451e58a0 2024-04-16T17:13:07.798 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:07.786+0000 7f381085d740 1 -- 172.21.15.45:0/665849321 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5613451e58a0 msgr2=0x5613451e5d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:07.798 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:07.786+0000 7f381085d740 1 --2- 172.21.15.45:0/665849321 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5613451e58a0 0x5613451e5d20 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f37f802c180 tx=0x7f37f8000bc0 comp rx=0 tx=0).stop 2024-04-16T17:13:07.798 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:07.786+0000 7f381085d740 1 -- 172.21.15.45:0/665849321 shutdown_connections 2024-04-16T17:13:07.798 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:07.786+0000 7f381085d740 1 --2- 172.21.15.45:0/665849321 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5613451e58a0 0x5613451e5d20 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:07.798 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:07.786+0000 7f381085d740 1 --2- 172.21.15.45:0/665849321 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5613451eee20 0x5613451f1210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:07.798 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:07.786+0000 7f381085d740 1 --2- 172.21.15.45:0/665849321 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5613451e6610 0x5613451e5360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:07.798 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:07.786+0000 7f381085d740 1 -- 172.21.15.45:0/665849321 >> 172.21.15.45:0/665849321 conn(0x5613451805d0 msgr2=0x5613451e0590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:13:07.798 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:07.786+0000 7f381085d740 1 -- 172.21.15.45:0/665849321 shutdown_connections 2024-04-16T17:13:07.799 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:07.786+0000 7f37ff400640 1 --2- 172.21.15.45:0/1885830492 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f37ec0a3cd0 0x7f37ec0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:07.799 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:07.786+0000 7f381085d740 1 -- 172.21.15.45:0/665849321 wait complete. 2024-04-16T17:13:07.799 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:07.788+0000 7f37ff400640 1 -- 172.21.15.45:0/1885830492 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9584+0+0 (secure 0 0 0) 0x7f37f40d1060 con 0x7f3808109530 2024-04-16T17:13:07.799 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:07.788+0000 7f3806a00640 1 --2- 172.21.15.45:0/1885830492 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f37ec0a3cd0 0x7f37ec0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:07.799 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:07.788+0000 7f381085d740 1 -- start start 2024-04-16T17:13:07.799 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:07.788+0000 7f381085d740 1 Processor -- start 2024-04-16T17:13:07.799 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:07.789+0000 7f3806a00640 1 --2- 172.21.15.45:0/1885830492 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f37ec0a3cd0 0x7f37ec0a6190 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f37f8009900 tx=0x7f37f802bc40 comp rx=0 tx=0).ready entity=mgr.25411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:13:07.799 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:07.789+0000 7f381085d740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5613451e58a0 0x5613451ebd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:07.799 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:07.789+0000 7f381085d740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5613451e6610 0x5613451ec240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:07.800 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:07.789+0000 7f381085d740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5613451eee20 0x5613451ec780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:07.800 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:07.789+0000 7f381085d740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5613451f1af0 con 0x5613451eee20 2024-04-16T17:13:07.800 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:07.789+0000 7f381085d740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x561345104390 con 0x5613451e6610 2024-04-16T17:13:07.800 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:07.789+0000 7f381085d740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5613451f1c70 con 0x5613451e58a0 2024-04-16T17:13:07.800 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:07.789+0000 7f381085d740 10 client.0 _ll_register_callbacks cb 0x5613450edf60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:13:07.800 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:07.789+0000 7f3806a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5613451e58a0 0x5613451ebd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:07.800 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:07.789+0000 7f3806000640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5613451e6610 0x5613451ec240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:07.800 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:07.789+0000 7f3806000640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5613451e6610 0x5613451ec240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:36078/0 (socket says 172.21.15.45:36078) 2024-04-16T17:13:07.800 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:07.789+0000 7f3806000640 1 -- 172.21.15.45:0/464583307 learned_addr learned my addr 172.21.15.45:0/464583307 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:13:07.800 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:07.789+0000 7f3806a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5613451e58a0 0x5613451ebd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:54236/0 (socket says 172.21.15.45:54236) 2024-04-16T17:13:07.801 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:07.789+0000 7f3807400640 1 --2- 172.21.15.45:0/464583307 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5613451eee20 0x5613451ec780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:07.801 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:07.790+0000 7f3806000640 1 -- 172.21.15.45:0/464583307 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5613451e58a0 msgr2=0x5613451ebd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:13:07.801 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:07.790+0000 7f3806000640 1 --2- 172.21.15.45:0/464583307 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5613451e58a0 0x5613451ebd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:07.801 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:07.790+0000 7f3806000640 1 -- 172.21.15.45:0/464583307 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5613451eee20 msgr2=0x5613451ec780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:13:07.801 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:07.790+0000 7f3806000640 1 --2- 172.21.15.45:0/464583307 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5613451eee20 0x5613451ec780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:07.801 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:07.790+0000 7f3806000640 1 -- 172.21.15.45:0/464583307 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561344f29620 con 0x5613451e6610 2024-04-16T17:13:07.801 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:07.790+0000 7f3806000640 1 --2- 172.21.15.45:0/464583307 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5613451e6610 0x5613451ec240 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f37f400e9c0 tx=0x7f37f40056b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d5b4d568cf7dd70d server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:13:07.801 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:07.790+0000 7f37f0c00640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-04-16T17:13:07.801 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:07.790+0000 7f37f0c00640 1 -- 172.21.15.45:0/464583307 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37f40a0f50 con 0x5613451e6610 2024-04-16T17:13:07.802 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:07.790+0000 7f37f0c00640 1 -- 172.21.15.45:0/464583307 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9584+0+0 (secure 0 0 0) 0x7f37f40a5380 con 0x5613451e6610 2024-04-16T17:13:07.802 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:07.790+0000 7f381085d740 1 -- 172.21.15.45:0/464583307 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f37f802c890 con 0x5613451e6610 2024-04-16T17:13:07.802 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:07.790+0000 7f37f0c00640 1 -- 172.21.15.45:0/464583307 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f37f40a5810 con 0x5613451e6610 2024-04-16T17:13:07.802 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:07.790+0000 7f381085d740 10 client.27398 fetch_fsmap learned FSMap version 37 2024-04-16T17:13:07.802 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:07.790+0000 7f381085d740 1 -- 172.21.15.45:0/464583307 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x5613451fdc80 con 0x5613451e6610 2024-04-16T17:13:07.802 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:07.791+0000 7f37f0c00640 1 -- 172.21.15.45:0/464583307 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f37f802c890 con 0x5613451e6610 2024-04-16T17:13:07.802 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:07.791+0000 7f37f0c00640 1 -- 172.21.15.45:0/464583307 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 37) ==== 8358+0+0 (secure 0 0 0) 0x7f37f409ba60 con 0x5613451e6610 2024-04-16T17:13:07.802 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:07.791+0000 7f381085d740 10 client.27398 fetch_fsmap finished waiting for FSMap version 37 2024-04-16T17:13:07.802 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:07.791+0000 7f381085d740 10 client.27398 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T17:13:07.802 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:07.791+0000 7f381085d740 20 client.27398 populate_metadata read hostname 'smithi045' 2024-04-16T17:13:07.802 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:07.791+0000 7f381085d740 1 --2- 172.21.15.45:0/464583307 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x5613451ff140 0x561345201530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:07.803 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:07.791+0000 7f381085d740 4 client.27398 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:13:07.803 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:07.791+0000 7f381085d740 1 -- 172.21.15.45:0/464583307 --> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5613451581a0 con 0x5613451ff140 2024-04-16T17:13:07.803 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:07.791+0000 7f3806a00640 1 --2- 172.21.15.45:0/464583307 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x5613451ff140 0x561345201530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:07.803 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:07.792+0000 7f3806a00640 1 --2- 172.21.15.45:0/464583307 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x5613451ff140 0x561345201530 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d869dd8dbafd2386 server_cookie=878e3cdc7844c3d4 in_seq=0 out_seq=0 2024-04-16T17:13:07.803 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:07.792+0000 7f37f0c00640 10 client.27398 ms_handle_connect on v2:172.21.15.104:6838/3660070592 2024-04-16T17:13:07.803 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:07.792+0000 7f37f0c00640 1 -- 172.21.15.45:0/464583307 <== mds.0 v2:172.21.15.104:6838/3660070592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5613451581a0 con 0x5613451ff140 2024-04-16T17:13:07.803 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:07.792+0000 7f37f0c00640 10 client.27398 handle_command_reply: tid=0 2024-04-16T17:13:07.803 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:07.792+0000 7f381085d740 1 client.27398 shutdown 2024-04-16T17:13:07.803 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:07.792+0000 7f381085d740 1 -- 172.21.15.45:0/464583307 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5613451e6610 msgr2=0x5613451ec240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:07.803 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:07.792+0000 7f381085d740 1 --2- 172.21.15.45:0/464583307 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5613451e6610 0x5613451ec240 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f37f400e9c0 tx=0x7f37f40056b0 comp rx=0 tx=0).stop 2024-04-16T17:13:07.803 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:07.792+0000 7f381085d740 1 -- 172.21.15.45:0/464583307 shutdown_connections 2024-04-16T17:13:07.804 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:07.792+0000 7f381085d740 1 --2- 172.21.15.45:0/464583307 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5613451eee20 0x5613451ec780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:07.804 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:07.792+0000 7f381085d740 1 --2- 172.21.15.45:0/464583307 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5613451e6610 0x5613451ec240 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:07.804 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:07.792+0000 7f381085d740 1 --2- 172.21.15.45:0/464583307 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x5613451ff140 0x561345201530 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:07.804 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:07.792+0000 7f381085d740 1 --2- 172.21.15.45:0/464583307 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5613451e58a0 0x5613451ebd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:07.804 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:07.792+0000 7f381085d740 1 -- 172.21.15.45:0/464583307 >> 172.21.15.45:0/464583307 conn(0x5613451805d0 msgr2=0x5613451e22e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:13:07.804 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:07.792+0000 7f37f0c00640 0 client.27398 ms_handle_reset on v2:172.21.15.104:6838/3660070592 2024-04-16T17:13:07.804 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:07.792+0000 7f381085d740 1 -- 172.21.15.45:0/464583307 shutdown_connections 2024-04-16T17:13:07.804 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:07.792+0000 7f381085d740 1 -- 172.21.15.45:0/464583307 wait complete. 2024-04-16T17:13:07.804 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:07.792+0000 7f381085d740 20 client.27398 trim_cache size 0 max 16384 2024-04-16T17:13:07.823 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:07.823+0000 7f381085d740 1 Processor -- start 2024-04-16T17:13:07.823 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:07.823+0000 7f381085d740 1 -- start start 2024-04-16T17:13:07.823 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:07.823+0000 7f381085d740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5613451e58a0 0x5613451fa9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:07.823 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:07.824+0000 7f381085d740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5613451e6610 0x5613451eaec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:07.823 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:07.824+0000 7f381085d740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5613451eee20 0x5613451ec5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:07.823 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:07.824+0000 7f381085d740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f38081afc80 con 0x5613451e6610 2024-04-16T17:13:07.824 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:07.824+0000 7f381085d740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f38081ad4c0 con 0x5613451eee20 2024-04-16T17:13:07.824 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:07.824+0000 7f381085d740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f38081ab310 con 0x5613451e58a0 2024-04-16T17:13:07.825 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:07.824+0000 7f3806000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5613451e6610 0x5613451eaec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:07.825 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:07.824+0000 7f3807400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5613451eee20 0x5613451ec5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:07.825 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:07.824+0000 7f3806000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5613451e6610 0x5613451eaec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:47004/0 (socket says 172.21.15.45:47004) 2024-04-16T17:13:07.825 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:07.824+0000 7f3807400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5613451eee20 0x5613451ec5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:36090/0 (socket says 172.21.15.45:36090) 2024-04-16T17:13:07.825 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:07.824+0000 7f3806000640 1 -- 172.21.15.45:0/226244002 learned_addr learned my addr 172.21.15.45:0/226244002 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:13:07.825 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:07.824+0000 7f3806a00640 1 --2- 172.21.15.45:0/226244002 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5613451e58a0 0x5613451fa9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:07.825 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:07.824+0000 7f3807400640 1 -- 172.21.15.45:0/226244002 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5613451e58a0 msgr2=0x5613451fa9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:13:07.826 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:07.824+0000 7f3807400640 1 --2- 172.21.15.45:0/226244002 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5613451e58a0 0x5613451fa9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:07.826 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:07.824+0000 7f3807400640 1 -- 172.21.15.45:0/226244002 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5613451e6610 msgr2=0x5613451eaec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:13:07.826 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:07.824+0000 7f3807400640 1 --2- 172.21.15.45:0/226244002 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5613451e6610 0x5613451eaec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:07.826 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:07.824+0000 7f3807400640 1 -- 172.21.15.45:0/226244002 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5613452018a0 con 0x5613451eee20 2024-04-16T17:13:07.826 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:07.825+0000 7f3807400640 1 --2- 172.21.15.45:0/226244002 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5613451eee20 0x5613451ec5c0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f3800005c40 tx=0x7f38000118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2e494da463ec51d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:13:07.826 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:07.825+0000 7f37f3400640 1 -- 172.21.15.45:0/226244002 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3800020070 con 0x5613451eee20 2024-04-16T17:13:07.826 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:07.825+0000 7f37f3400640 1 -- 172.21.15.45:0/226244002 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3800010c90 con 0x5613451eee20 2024-04-16T17:13:07.826 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:07.825+0000 7f37f3400640 1 -- 172.21.15.45:0/226244002 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f380001bd20 con 0x5613451eee20 2024-04-16T17:13:07.830 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:07.826+0000 7f381085d740 1 -- 172.21.15.45:0/226244002 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5613451eee20 msgr2=0x5613451ec5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:07.830 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:07.826+0000 7f381085d740 1 --2- 172.21.15.45:0/226244002 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5613451eee20 0x5613451ec5c0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f3800005c40 tx=0x7f38000118d0 comp rx=0 tx=0).stop 2024-04-16T17:13:07.830 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:07.826+0000 7f381085d740 1 -- 172.21.15.45:0/226244002 shutdown_connections 2024-04-16T17:13:07.831 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:07.826+0000 7f381085d740 1 --2- 172.21.15.45:0/226244002 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5613451eee20 0x5613451ec5c0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:07.831 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:07.826+0000 7f381085d740 1 --2- 172.21.15.45:0/226244002 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5613451e6610 0x5613451eaec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:07.831 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:07.826+0000 7f381085d740 1 --2- 172.21.15.45:0/226244002 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5613451e58a0 0x5613451fa9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:07.831 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:07.826+0000 7f381085d740 1 -- 172.21.15.45:0/226244002 >> 172.21.15.45:0/226244002 conn(0x5613451805d0 msgr2=0x5613451dee20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:13:07.831 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:07.826+0000 7f381085d740 1 -- 172.21.15.45:0/226244002 shutdown_connections 2024-04-16T17:13:07.831 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:07.826+0000 7f381085d740 1 -- 172.21.15.45:0/226244002 wait complete. 2024-04-16T17:13:07.831 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:07.827+0000 7f381085d740 1 -- start start 2024-04-16T17:13:07.831 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:07.827+0000 7f381085d740 1 Processor -- start 2024-04-16T17:13:07.831 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:07.827+0000 7f381085d740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5613451e58a0 0x5613451f73e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:07.831 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:07.827+0000 7f381085d740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5613451e6610 0x5613451e91a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:07.831 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:07.827+0000 7f381085d740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5613451eee20 0x5613451e9720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:07.832 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:07.827+0000 7f381085d740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f38081ab980 con 0x5613451e6610 2024-04-16T17:13:07.832 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:07.827+0000 7f381085d740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f38081acdf0 con 0x5613451eee20 2024-04-16T17:13:07.832 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:07.827+0000 7f381085d740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f38081ac050 con 0x5613451e58a0 2024-04-16T17:13:07.832 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:07.827+0000 7f381085d740 10 client.0 _ll_register_callbacks cb 0x7f37f400ce60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:13:07.832 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:07.828+0000 7f3806a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5613451e58a0 0x5613451f73e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:07.832 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:07.828+0000 7f3807400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5613451eee20 0x5613451e9720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:07.832 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:07.828+0000 7f3806000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5613451e6610 0x5613451e91a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:07.832 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:07.828+0000 7f3806a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5613451e58a0 0x5613451f73e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:54252/0 (socket says 172.21.15.45:54252) 2024-04-16T17:13:07.832 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:07.828+0000 7f3806a00640 1 -- 172.21.15.45:0/1764196230 learned_addr learned my addr 172.21.15.45:0/1764196230 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:13:07.832 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:07.828+0000 7f3807400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5613451eee20 0x5613451e9720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:36096/0 (socket says 172.21.15.45:36096) 2024-04-16T17:13:07.832 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:07.828+0000 7f3807400640 1 -- 172.21.15.45:0/1764196230 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5613451e58a0 msgr2=0x5613451f73e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:13:07.833 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:07.828+0000 7f3807400640 1 --2- 172.21.15.45:0/1764196230 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5613451e58a0 0x5613451f73e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:07.833 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:07.828+0000 7f3807400640 1 -- 172.21.15.45:0/1764196230 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5613451e6610 msgr2=0x5613451e91a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:13:07.833 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:07.828+0000 7f3807400640 1 --2- 172.21.15.45:0/1764196230 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5613451e6610 0x5613451e91a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:07.833 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:07.828+0000 7f3807400640 1 -- 172.21.15.45:0/1764196230 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5613451ef820 con 0x5613451eee20 2024-04-16T17:13:07.833 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:07.828+0000 7f3807400640 1 --2- 172.21.15.45:0/1764196230 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5613451eee20 0x5613451e9720 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f3800005c10 tx=0x7f3800004f10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aded6e9d463f0ffa server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:13:07.833 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:07.828+0000 7f37f0c00640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-04-16T17:13:07.833 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:07.829+0000 7f37f0c00640 1 -- 172.21.15.45:0/1764196230 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3800029070 con 0x5613451eee20 2024-04-16T17:13:07.833 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:07.829+0000 7f381085d740 1 -- 172.21.15.45:0/1764196230 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3800010c90 con 0x5613451eee20 2024-04-16T17:13:07.833 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:07.829+0000 7f37f0c00640 1 -- 172.21.15.45:0/1764196230 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9584+0+0 (secure 0 0 0) 0x7f380001b9a0 con 0x5613451eee20 2024-04-16T17:13:07.833 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:07.829+0000 7f37f0c00640 1 -- 172.21.15.45:0/1764196230 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f3800032050 con 0x5613451eee20 2024-04-16T17:13:07.833 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:07.829+0000 7f381085d740 10 client.27404 fetch_fsmap learned FSMap version 37 2024-04-16T17:13:07.833 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:07.829+0000 7f381085d740 1 -- 172.21.15.45:0/1764196230 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x561345265920 con 0x5613451eee20 2024-04-16T17:13:07.834 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:07.829+0000 7f37f0c00640 1 -- 172.21.15.45:0/1764196230 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3800010c90 con 0x5613451eee20 2024-04-16T17:13:07.834 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:07.829+0000 7f37f0c00640 1 -- 172.21.15.45:0/1764196230 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 37) ==== 8358+0+0 (secure 0 0 0) 0x7f380002c020 con 0x5613451eee20 2024-04-16T17:13:07.834 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:07.830+0000 7f381085d740 10 client.27404 fetch_fsmap finished waiting for FSMap version 37 2024-04-16T17:13:07.834 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:07.830+0000 7f381085d740 10 client.27404 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T17:13:07.834 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:07.830+0000 7f381085d740 20 client.27404 populate_metadata read hostname 'smithi045' 2024-04-16T17:13:07.834 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:07.830+0000 7f381085d740 1 --2- 172.21.15.45:0/1764196230 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x561345265ff0 0x561345268410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:07.834 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:07.830+0000 7f381085d740 4 client.27404 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-16T17:13:07.834 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:07.830+0000 7f381085d740 1 -- 172.21.15.45:0/1764196230 --> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561345152010 con 0x561345265ff0 2024-04-16T17:13:07.834 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:07.830+0000 7f3806a00640 1 --2- 172.21.15.45:0/1764196230 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x561345265ff0 0x561345268410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:07.834 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:07.830+0000 7f3806a00640 1 --2- 172.21.15.45:0/1764196230 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x561345265ff0 0x561345268410 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5b0b9248bd350817 server_cookie=23054befd56428f3 in_seq=0 out_seq=0 2024-04-16T17:13:07.835 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:07.831+0000 7f37f0c00640 10 client.27404 ms_handle_connect on v2:172.21.15.104:6838/3660070592 2024-04-16T17:13:07.902 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:07.899+0000 7f37f0c00640 1 -- 172.21.15.45:0/1764196230 <== mds.0 v2:172.21.15.104:6838/3660070592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x561345152010 con 0x561345265ff0 2024-04-16T17:13:07.902 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:07.899+0000 7f37f0c00640 10 client.27404 handle_command_reply: tid=0 2024-04-16T17:13:07.902 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:07.902+0000 7f381085d740 1 client.27404 shutdown 2024-04-16T17:13:07.902 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:07.902+0000 7f381085d740 1 -- 172.21.15.45:0/1764196230 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5613451eee20 msgr2=0x5613451e9720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:07.902 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:07.902+0000 7f381085d740 1 --2- 172.21.15.45:0/1764196230 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5613451eee20 0x5613451e9720 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f3800005c10 tx=0x7f3800004f10 comp rx=0 tx=0).stop 2024-04-16T17:13:07.904 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:07.903+0000 7f381085d740 1 -- 172.21.15.45:0/1764196230 shutdown_connections 2024-04-16T17:13:07.904 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:07.903+0000 7f381085d740 1 --2- 172.21.15.45:0/1764196230 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5613451eee20 0x5613451e9720 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:07.904 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:07.903+0000 7f381085d740 1 --2- 172.21.15.45:0/1764196230 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5613451e6610 0x5613451e91a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:07.904 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:07.903+0000 7f381085d740 1 --2- 172.21.15.45:0/1764196230 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x561345265ff0 0x561345268410 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:07.904 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:07.903+0000 7f381085d740 1 --2- 172.21.15.45:0/1764196230 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5613451e58a0 0x5613451f73e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:07.904 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:07.903+0000 7f381085d740 1 -- 172.21.15.45:0/1764196230 >> 172.21.15.45:0/1764196230 conn(0x5613451805d0 msgr2=0x5613451ead40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:13:07.904 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:07.903+0000 7f37f0c00640 0 client.27404 ms_handle_reset on v2:172.21.15.104:6838/3660070592 2024-04-16T17:13:07.904 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:07.903+0000 7f381085d740 1 -- 172.21.15.45:0/1764196230 shutdown_connections 2024-04-16T17:13:07.905 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:07.903+0000 7f381085d740 1 -- 172.21.15.45:0/1764196230 wait complete. 2024-04-16T17:13:07.905 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:07.903+0000 7f381085d740 20 client.27404 trim_cache size 0 max 16384 2024-04-16T17:13:07.905 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-04-16T17:13:07.905 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-04-16T17:13:07.905 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-04-16T17:13:07.905 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-04-16T17:13:07.906 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:07.904+0000 7f380dc00640 1 -- 172.21.15.45:0/1885830492 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f37ec0a3cd0 msgr2=0x7f37ec0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:07.907 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:07.904+0000 7f380dc00640 1 --2- 172.21.15.45:0/1885830492 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f37ec0a3cd0 0x7f37ec0a6190 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f37f8009900 tx=0x7f37f802bc40 comp rx=0 tx=0).stop 2024-04-16T17:13:07.907 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:07.904+0000 7f380dc00640 1 -- 172.21.15.45:0/1885830492 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3808109530 msgr2=0x7f38081a0c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:07.907 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:07.904+0000 7f380dc00640 1 --2- 172.21.15.45:0/1885830492 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3808109530 0x7f38081a0c40 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f37f400e9f0 tx=0x7f37f400eeb0 comp rx=0 tx=0).stop 2024-04-16T17:13:07.907 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:07.905+0000 7f380dc00640 1 -- 172.21.15.45:0/1885830492 shutdown_connections 2024-04-16T17:13:07.907 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:07.905+0000 7f380dc00640 1 --2- 172.21.15.45:0/1885830492 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f37ec0a3cd0 0x7f37ec0a6190 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:07.907 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:07.905+0000 7f380dc00640 1 --2- 172.21.15.45:0/1885830492 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f380819a8b0 0x7f380819ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:07.907 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:07.905+0000 7f380dc00640 1 --2- 172.21.15.45:0/1885830492 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3808109530 0x7f38081a0c40 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:07.907 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:07.905+0000 7f380dc00640 1 --2- 172.21.15.45:0/1885830492 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3808104460 0x7f38081a0700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:07.907 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:07.905+0000 7f380dc00640 1 -- 172.21.15.45:0/1885830492 >> 172.21.15.45:0/1885830492 conn(0x7f38080ffd00 msgr2=0x7f3808101ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:13:07.907 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:07.905+0000 7f380dc00640 1 -- 172.21.15.45:0/1885830492 shutdown_connections 2024-04-16T17:13:07.908 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:07.905+0000 7f380dc00640 1 -- 172.21.15.45:0/1885830492 wait complete. 2024-04-16T17:13:08.275 INFO:tasks.workunit.client.0.smithi045.stdout: 1 103514 1.72 MB/sec execute 190 sec latency 88.365 ms 2024-04-16T17:13:08.442 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:13:08 smithi045 ceph-mon[31676]: pgmap v195: 129 pgs: 129 active+clean; 56 MiB data, 746 MiB used, 1.0 TiB / 1.0 TiB avail; 587 KiB/s rd, 497 KiB/s wr, 102 op/s 2024-04-16T17:13:08.677 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:13:08 smithi104 ceph-mon[36457]: pgmap v195: 129 pgs: 129 active+clean; 56 MiB data, 746 MiB used, 1.0 TiB / 1.0 TiB avail; 587 KiB/s rd, 497 KiB/s wr, 102 op/s 2024-04-16T17:13:08.804 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:13:08 smithi117 ceph-mon[36730]: pgmap v195: 129 pgs: 129 active+clean; 56 MiB data, 746 MiB used, 1.0 TiB / 1.0 TiB avail; 587 KiB/s rd, 497 KiB/s wr, 102 op/s 2024-04-16T17:13:09.275 INFO:tasks.workunit.client.0.smithi045.stdout: 1 103981 1.72 MB/sec execute 191 sec latency 122.657 ms 2024-04-16T17:13:09.677 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:13:09 smithi104 ceph-mon[36457]: scrub summary: idle 2024-04-16T17:13:09.677 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:13:09 smithi104 ceph-mon[36457]: scrub summary: idle 2024-04-16T17:13:09.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:13:09 smithi104 ceph-mon[36457]: scrub summary: idle 2024-04-16T17:13:09.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:13:09 smithi104 ceph-mon[36457]: osdmap e97: 12 total, 12 up, 12 in 2024-04-16T17:13:09.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:13:09 smithi117 ceph-mon[36730]: scrub summary: idle 2024-04-16T17:13:09.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:13:09 smithi117 ceph-mon[36730]: scrub summary: idle 2024-04-16T17:13:09.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:13:09 smithi117 ceph-mon[36730]: scrub summary: idle 2024-04-16T17:13:09.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:13:09 smithi117 ceph-mon[36730]: osdmap e97: 12 total, 12 up, 12 in 2024-04-16T17:13:10.275 INFO:tasks.workunit.client.0.smithi045.stdout: 1 104440 1.73 MB/sec execute 192 sec latency 74.817 ms 2024-04-16T17:13:10.677 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:13:10 smithi104 ceph-mon[36457]: pgmap v196: 129 pgs: 129 active+clean; 56 MiB data, 746 MiB used, 1.0 TiB / 1.0 TiB avail; 587 KiB/s rd, 497 KiB/s wr, 102 op/s 2024-04-16T17:13:10.804 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:13:10 smithi117 ceph-mon[36730]: pgmap v196: 129 pgs: 129 active+clean; 56 MiB data, 746 MiB used, 1.0 TiB / 1.0 TiB avail; 587 KiB/s rd, 497 KiB/s wr, 102 op/s 2024-04-16T17:13:10.894 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:13:09 smithi045 ceph-mon[31676]: scrub summary: idle 2024-04-16T17:13:10.895 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:13:09 smithi045 ceph-mon[31676]: scrub summary: idle 2024-04-16T17:13:10.895 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:13:09 smithi045 ceph-mon[31676]: scrub summary: idle 2024-04-16T17:13:10.895 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:13:09 smithi045 ceph-mon[31676]: osdmap e97: 12 total, 12 up, 12 in 2024-04-16T17:13:10.895 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:13:10 smithi045 ceph-mon[31676]: pgmap v196: 129 pgs: 129 active+clean; 56 MiB data, 746 MiB used, 1.0 TiB / 1.0 TiB avail; 587 KiB/s rd, 497 KiB/s wr, 102 op/s 2024-04-16T17:13:11.275 INFO:tasks.workunit.client.0.smithi045.stdout: 1 104602 1.72 MB/sec execute 193 sec latency 463.306 ms 2024-04-16T17:13:12.275 INFO:tasks.workunit.client.0.smithi045.stdout: 1 104605 1.71 MB/sec execute 194 sec latency 1458.799 ms 2024-04-16T17:13:12.677 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:13:12 smithi104 ceph-mon[36457]: pgmap v198: 129 pgs: 129 active+clean; 55 MiB data, 713 MiB used, 1.0 TiB / 1.0 TiB avail; 857 KiB/s rd, 735 KiB/s wr, 170 op/s 2024-04-16T17:13:12.804 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:13:12 smithi117 ceph-mon[36730]: pgmap v198: 129 pgs: 129 active+clean; 55 MiB data, 713 MiB used, 1.0 TiB / 1.0 TiB avail; 857 KiB/s rd, 735 KiB/s wr, 170 op/s 2024-04-16T17:13:13.279 INFO:tasks.workunit.client.0.smithi045.stdout: 1 104972 1.70 MB/sec execute 195 sec latency 45.022 ms 2024-04-16T17:13:14.279 INFO:tasks.workunit.client.0.smithi045.stdout: 1 105325 1.70 MB/sec execute 196 sec latency 67.274 ms 2024-04-16T17:13:14.677 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:13:14 smithi104 ceph-mon[36457]: pgmap v199: 129 pgs: 129 active+clean; 55 MiB data, 713 MiB used, 1.0 TiB / 1.0 TiB avail; 731 KiB/s rd, 627 KiB/s wr, 145 op/s 2024-04-16T17:13:14.804 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:13:14 smithi117 ceph-mon[36730]: pgmap v199: 129 pgs: 129 active+clean; 55 MiB data, 713 MiB used, 1.0 TiB / 1.0 TiB avail; 731 KiB/s rd, 627 KiB/s wr, 145 op/s 2024-04-16T17:13:15.279 INFO:tasks.workunit.client.0.smithi045.stdout: 1 106283 1.73 MB/sec execute 197 sec latency 52.585 ms 2024-04-16T17:13:16.019 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:13:12 smithi045 ceph-mon[31676]: pgmap v198: 129 pgs: 129 active+clean; 55 MiB data, 713 MiB used, 1.0 TiB / 1.0 TiB avail; 857 KiB/s rd, 735 KiB/s wr, 170 op/s 2024-04-16T17:13:16.019 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:13:14 smithi045 ceph-mon[31676]: pgmap v199: 129 pgs: 129 active+clean; 55 MiB data, 713 MiB used, 1.0 TiB / 1.0 TiB avail; 731 KiB/s rd, 627 KiB/s wr, 145 op/s 2024-04-16T17:13:16.279 INFO:tasks.workunit.client.0.smithi045.stdout: 1 106827 1.74 MB/sec execute 198 sec latency 24.276 ms 2024-04-16T17:13:16.677 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:13:16 smithi104 ceph-mon[36457]: pgmap v200: 129 pgs: 129 active+clean; 55 MiB data, 713 MiB used, 1.0 TiB / 1.0 TiB avail; 685 KiB/s rd, 588 KiB/s wr, 136 op/s 2024-04-16T17:13:16.804 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:13:16 smithi117 ceph-mon[36730]: pgmap v200: 129 pgs: 129 active+clean; 55 MiB data, 713 MiB used, 1.0 TiB / 1.0 TiB avail; 685 KiB/s rd, 588 KiB/s wr, 136 op/s 2024-04-16T17:13:17.279 INFO:tasks.workunit.client.0.smithi045.stdout: 1 107240 1.74 MB/sec execute 199 sec latency 36.756 ms 2024-04-16T17:13:17.281 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:13:16 smithi045 ceph-mon[31676]: pgmap v200: 129 pgs: 129 active+clean; 55 MiB data, 713 MiB used, 1.0 TiB / 1.0 TiB avail; 685 KiB/s rd, 588 KiB/s wr, 136 op/s 2024-04-16T17:13:18.279 INFO:tasks.workunit.client.0.smithi045.stdout: 1 107903 1.75 MB/sec execute 200 sec latency 39.680 ms 2024-04-16T17:13:18.677 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:13:18 smithi104 ceph-mon[36457]: pgmap v201: 129 pgs: 129 active+clean; 56 MiB data, 725 MiB used, 1.0 TiB / 1.0 TiB avail; 715 KiB/s rd, 682 KiB/s wr, 98 op/s 2024-04-16T17:13:18.677 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:13:18 smithi104 ceph-mon[36457]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-16T17:13:18.804 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:13:18 smithi117 ceph-mon[36730]: pgmap v201: 129 pgs: 129 active+clean; 56 MiB data, 725 MiB used, 1.0 TiB / 1.0 TiB avail; 715 KiB/s rd, 682 KiB/s wr, 98 op/s 2024-04-16T17:13:18.804 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:13:18 smithi117 ceph-mon[36730]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-16T17:13:19.227 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:13:18 smithi045 ceph-mon[31676]: pgmap v201: 129 pgs: 129 active+clean; 56 MiB data, 725 MiB used, 1.0 TiB / 1.0 TiB avail; 715 KiB/s rd, 682 KiB/s wr, 98 op/s 2024-04-16T17:13:19.227 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:13:18 smithi045 ceph-mon[31676]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-16T17:13:19.279 INFO:tasks.workunit.client.0.smithi045.stdout: 1 108142 1.75 MB/sec execute 201 sec latency 398.169 ms 2024-04-16T17:13:20.280 INFO:tasks.workunit.client.0.smithi045.stdout: 1 108142 1.74 MB/sec execute 202 sec latency 1398.287 ms 2024-04-16T17:13:20.677 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:13:20 smithi104 ceph-mon[36457]: pgmap v202: 129 pgs: 129 active+clean; 56 MiB data, 725 MiB used, 1.0 TiB / 1.0 TiB avail; 715 KiB/s rd, 682 KiB/s wr, 98 op/s 2024-04-16T17:13:20.720 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:13:20 smithi045 ceph-mon[31676]: pgmap v202: 129 pgs: 129 active+clean; 56 MiB data, 725 MiB used, 1.0 TiB / 1.0 TiB avail; 715 KiB/s rd, 682 KiB/s wr, 98 op/s 2024-04-16T17:13:20.804 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:13:20 smithi117 ceph-mon[36730]: pgmap v202: 129 pgs: 129 active+clean; 56 MiB data, 725 MiB used, 1.0 TiB / 1.0 TiB avail; 715 KiB/s rd, 682 KiB/s wr, 98 op/s 2024-04-16T17:13:21.048 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:21.047+0000 7fee8dc00640 1 -- 172.21.15.45:0/1918399437 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fee881016d0 msgr2=0x7fee88105b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:21.048 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:21.047+0000 7fee8dc00640 1 --2- 172.21.15.45:0/1918399437 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fee881016d0 0x7fee88105b90 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fee70009900 tx=0x7fee7002bcc0 comp rx=0 tx=0).stop 2024-04-16T17:13:21.048 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:21.048+0000 7fee8dc00640 1 -- 172.21.15.45:0/1918399437 shutdown_connections 2024-04-16T17:13:21.048 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:21.048+0000 7fee8dc00640 1 --2- 172.21.15.45:0/1918399437 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fee881061b0 0x7fee88106640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:21.049 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:21.048+0000 7fee8dc00640 1 --2- 172.21.15.45:0/1918399437 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fee881016d0 0x7fee88105b90 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:21.049 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:21.048+0000 7fee8dc00640 1 --2- 172.21.15.45:0/1918399437 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fee88100db0 0x7fee88101190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:21.049 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:21.048+0000 7fee8dc00640 1 -- 172.21.15.45:0/1918399437 >> 172.21.15.45:0/1918399437 conn(0x7fee880fc820 msgr2=0x7fee880fec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:13:21.049 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:21.048+0000 7fee8dc00640 1 -- 172.21.15.45:0/1918399437 shutdown_connections 2024-04-16T17:13:21.049 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:21.048+0000 7fee8dc00640 1 -- 172.21.15.45:0/1918399437 wait complete. 2024-04-16T17:13:21.052 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:21.049+0000 7fee8dc00640 1 Processor -- start 2024-04-16T17:13:21.052 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:21.049+0000 7fee8dc00640 1 -- start start 2024-04-16T17:13:21.052 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:21.049+0000 7fee8dc00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fee88100db0 0x7fee881a4940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:21.052 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:21.049+0000 7fee8dc00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fee881016d0 0x7fee881a4e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:21.052 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:21.049+0000 7fee8dc00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fee881061b0 0x7fee8819eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:21.052 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:21.049+0000 7fee8dc00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fee88118710 con 0x7fee881061b0 2024-04-16T17:13:21.052 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:21.049+0000 7fee8dc00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fee88118590 con 0x7fee881016d0 2024-04-16T17:13:21.053 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:21.049+0000 7fee8dc00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fee88118890 con 0x7fee88100db0 2024-04-16T17:13:21.053 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:21.049+0000 7fee87e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fee88100db0 0x7fee881a4940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:21.053 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:21.049+0000 7fee87e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fee88100db0 0x7fee881a4940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:59954/0 (socket says 172.21.15.45:59954) 2024-04-16T17:13:21.053 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:21.049+0000 7fee87e00640 1 -- 172.21.15.45:0/915556850 learned_addr learned my addr 172.21.15.45:0/915556850 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:13:21.053 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:21.049+0000 7fee8d200640 1 --2- 172.21.15.45:0/915556850 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fee881061b0 0x7fee8819eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:21.053 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:21.049+0000 7fee7f400640 1 --2- 172.21.15.45:0/915556850 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fee881016d0 0x7fee881a4e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:21.053 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:21.050+0000 7fee87e00640 1 -- 172.21.15.45:0/915556850 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fee881016d0 msgr2=0x7fee881a4e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:21.053 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:21.050+0000 7fee87e00640 1 --2- 172.21.15.45:0/915556850 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fee881016d0 0x7fee881a4e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:21.053 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:21.050+0000 7fee87e00640 1 -- 172.21.15.45:0/915556850 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fee881061b0 msgr2=0x7fee8819eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:21.053 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:21.050+0000 7fee87e00640 1 --2- 172.21.15.45:0/915556850 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fee881061b0 0x7fee8819eaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:21.053 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:21.050+0000 7fee87e00640 1 -- 172.21.15.45:0/915556850 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee8819f270 con 0x7fee88100db0 2024-04-16T17:13:21.054 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:21.050+0000 7fee87e00640 1 --2- 172.21.15.45:0/915556850 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fee88100db0 0x7fee881a4940 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fee7800e9f0 tx=0x7fee7800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:13:21.054 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:21.050+0000 7fee85600640 1 -- 172.21.15.45:0/915556850 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee7800cd80 con 0x7fee88100db0 2024-04-16T17:13:21.054 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:21.050+0000 7fee85600640 1 -- 172.21.15.45:0/915556850 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fee78010910 con 0x7fee88100db0 2024-04-16T17:13:21.054 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:21.050+0000 7fee85600640 1 -- 172.21.15.45:0/915556850 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee78018990 con 0x7fee88100db0 2024-04-16T17:13:21.054 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:21.050+0000 7fee8dc00640 1 -- 172.21.15.45:0/915556850 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee8819f560 con 0x7fee88100db0 2024-04-16T17:13:21.054 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:21.051+0000 7fee8dc00640 1 -- 172.21.15.45:0/915556850 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee881ab800 con 0x7fee88100db0 2024-04-16T17:13:21.061 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:21.054+0000 7fee85600640 1 -- 172.21.15.45:0/915556850 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fee78010ac0 con 0x7fee88100db0 2024-04-16T17:13:21.062 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:21.054+0000 7fee9201e740 1 Processor -- start 2024-04-16T17:13:21.062 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:21.054+0000 7fee9201e740 1 -- start start 2024-04-16T17:13:21.062 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:21.055+0000 7fee9201e740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55761884a7f0 0x557618849540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:21.062 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:21.055+0000 7fee9201e740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557618849a80 0x557618849f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:21.062 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:21.055+0000 7fee9201e740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557618853000 0x5576188553f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:21.062 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:21.055+0000 7fee9201e740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5576187ae780 con 0x557618853000 2024-04-16T17:13:21.062 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:21.055+0000 7fee9201e740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x557618754f40 con 0x55761884a7f0 2024-04-16T17:13:21.062 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:21.055+0000 7fee9201e740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5576187aa740 con 0x557618849a80 2024-04-16T17:13:21.062 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:21.055+0000 7fee7f400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557618853000 0x5576188553f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:21.063 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:21.055+0000 7fee87e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557618849a80 0x557618849f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:21.063 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:21.055+0000 7fee8d200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55761884a7f0 0x557618849540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:21.063 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:21.055+0000 7fee7f400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557618853000 0x5576188553f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36374/0 (socket says 172.21.15.45:36374) 2024-04-16T17:13:21.063 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:21.055+0000 7fee7f400640 1 -- 172.21.15.45:0/2054882072 learned_addr learned my addr 172.21.15.45:0/2054882072 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:13:21.063 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:21.055+0000 7fee8d200640 1 -- 172.21.15.45:0/2054882072 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557618849a80 msgr2=0x557618849f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:13:21.063 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:21.055+0000 7fee8d200640 1 --2- 172.21.15.45:0/2054882072 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557618849a80 0x557618849f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:21.063 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:21.055+0000 7fee8d200640 1 -- 172.21.15.45:0/2054882072 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557618853000 msgr2=0x5576188553f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:13:21.063 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:21.055+0000 7fee8d200640 1 --2- 172.21.15.45:0/2054882072 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557618853000 0x5576188553f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:21.063 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:21.056+0000 7fee8d200640 1 -- 172.21.15.45:0/2054882072 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5576184c1210 con 0x55761884a7f0 2024-04-16T17:13:21.063 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:21.056+0000 7fee7f400640 1 --2- 172.21.15.45:0/2054882072 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557618853000 0x5576188553f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:13:21.064 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:21.056+0000 7fee8d200640 1 --2- 172.21.15.45:0/2054882072 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55761884a7f0 0x557618849540 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fee80009f00 tx=0x7fee80011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=50751ec0b098be45 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:13:21.064 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:21.056+0000 7fee85600640 1 --2- 172.21.15.45:0/915556850 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fee6c0a3d10 0x7fee6c0a61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:21.064 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:21.056+0000 7fee85600640 1 -- 172.21.15.45:0/915556850 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9496+0+0 (secure 0 0 0) 0x7fee780d0f60 con 0x7fee88100db0 2024-04-16T17:13:21.064 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:21.056+0000 7fee7d600640 1 -- 172.21.15.45:0/2054882072 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee80004d20 con 0x55761884a7f0 2024-04-16T17:13:21.064 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:21.056+0000 7fee87e00640 1 --2- 172.21.15.45:0/915556850 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fee6c0a3d10 0x7fee6c0a61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:21.064 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:21.056+0000 7fee7d600640 1 -- 172.21.15.45:0/2054882072 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fee80004050 con 0x55761884a7f0 2024-04-16T17:13:21.064 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:21.056+0000 7fee7d600640 1 -- 172.21.15.45:0/2054882072 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee80005280 con 0x55761884a7f0 2024-04-16T17:13:21.064 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:21.057+0000 7fee9201e740 1 -- 172.21.15.45:0/2054882072 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55761884a7f0 msgr2=0x557618849540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:21.064 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:21.057+0000 7fee87e00640 1 --2- 172.21.15.45:0/915556850 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fee6c0a3d10 0x7fee6c0a61d0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x55761884af50 tx=0x7fee780d4a10 comp rx=0 tx=0).ready entity=mgr.25411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:13:21.064 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:21.057+0000 7fee9201e740 1 --2- 172.21.15.45:0/2054882072 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55761884a7f0 0x557618849540 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fee80009f00 tx=0x7fee80011810 comp rx=0 tx=0).stop 2024-04-16T17:13:21.064 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:21.057+0000 7fee9201e740 1 -- 172.21.15.45:0/2054882072 shutdown_connections 2024-04-16T17:13:21.065 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:21.057+0000 7fee9201e740 1 --2- 172.21.15.45:0/2054882072 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557618849a80 0x557618849f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:21.065 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:21.057+0000 7fee9201e740 1 --2- 172.21.15.45:0/2054882072 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557618853000 0x5576188553f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:21.065 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:21.057+0000 7fee9201e740 1 --2- 172.21.15.45:0/2054882072 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55761884a7f0 0x557618849540 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:21.065 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:21.057+0000 7fee9201e740 1 -- 172.21.15.45:0/2054882072 >> 172.21.15.45:0/2054882072 conn(0x5576187e45d0 msgr2=0x557618844770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:13:21.065 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:21.057+0000 7fee9201e740 1 -- 172.21.15.45:0/2054882072 shutdown_connections 2024-04-16T17:13:21.065 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:21.057+0000 7fee9201e740 1 -- 172.21.15.45:0/2054882072 wait complete. 2024-04-16T17:13:21.065 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:21.058+0000 7fee9201e740 1 -- start start 2024-04-16T17:13:21.065 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:21.058+0000 7fee9201e740 1 Processor -- start 2024-04-16T17:13:21.065 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:21.058+0000 7fee9201e740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557618849a80 0x55761884fe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:21.065 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:21.058+0000 7fee9201e740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55761884a7f0 0x5576188503c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:21.065 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:21.058+0000 7fee9201e740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557618853000 0x557618850900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:21.066 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:21.058+0000 7fee9201e740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x557618855d30 con 0x557618853000 2024-04-16T17:13:21.066 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:21.058+0000 7fee9201e740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x557618768390 con 0x55761884a7f0 2024-04-16T17:13:21.066 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:21.058+0000 7fee9201e740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x557618855eb0 con 0x557618849a80 2024-04-16T17:13:21.066 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:21.058+0000 7fee9201e740 10 client.0 _ll_register_callbacks cb 0x557618751f60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:13:21.066 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:21.059+0000 7fee8d200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557618853000 0x557618850900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:21.066 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:21.059+0000 7fee7f400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55761884a7f0 0x5576188503c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:21.066 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:21.059+0000 7fee8d200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557618849a80 0x55761884fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:21.066 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:21.059+0000 7fee7f400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55761884a7f0 0x5576188503c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:34038/0 (socket says 172.21.15.45:34038) 2024-04-16T17:13:21.066 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:21.059+0000 7fee7f400640 1 -- 172.21.15.45:0/1615503921 learned_addr learned my addr 172.21.15.45:0/1615503921 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:13:21.066 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:21.059+0000 7fee7f400640 1 -- 172.21.15.45:0/1615503921 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557618849a80 msgr2=0x55761884fe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:13:21.067 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:21.059+0000 7fee7f400640 1 --2- 172.21.15.45:0/1615503921 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557618849a80 0x55761884fe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:21.067 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:21.059+0000 7fee7f400640 1 -- 172.21.15.45:0/1615503921 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557618853000 msgr2=0x557618850900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:13:21.067 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:21.059+0000 7fee7f400640 1 --2- 172.21.15.45:0/1615503921 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557618853000 0x557618850900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:21.067 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:21.059+0000 7fee7f400640 1 -- 172.21.15.45:0/1615503921 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55761858d620 con 0x55761884a7f0 2024-04-16T17:13:21.067 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:21.059+0000 7fee7f400640 1 --2- 172.21.15.45:0/1615503921 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55761884a7f0 0x5576188503c0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fee70009230 tx=0x7fee700092d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e3e542a9b165bd65 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:13:21.067 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:21.059+0000 7fee76000640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-04-16T17:13:21.067 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:21.060+0000 7fee76000640 1 -- 172.21.15.45:0/1615503921 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee7002cbd0 con 0x55761884a7f0 2024-04-16T17:13:21.067 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:21.060+0000 7fee9201e740 1 -- 172.21.15.45:0/1615503921 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fee80004050 con 0x55761884a7f0 2024-04-16T17:13:21.067 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:21.060+0000 7fee76000640 1 -- 172.21.15.45:0/1615503921 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9496+0+0 (secure 0 0 0) 0x7fee7002cd70 con 0x55761884a7f0 2024-04-16T17:13:21.068 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:21.060+0000 7fee76000640 1 -- 172.21.15.45:0/1615503921 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fee700046c0 con 0x55761884a7f0 2024-04-16T17:13:21.068 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:21.060+0000 7fee9201e740 10 client.27416 fetch_fsmap learned FSMap version 38 2024-04-16T17:13:21.068 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:21.060+0000 7fee9201e740 1 -- 172.21.15.45:0/1615503921 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x557618861eb0 con 0x55761884a7f0 2024-04-16T17:13:21.068 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:21.060+0000 7fee76000640 1 -- 172.21.15.45:0/1615503921 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fee80004050 con 0x55761884a7f0 2024-04-16T17:13:21.068 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:21.061+0000 7fee76000640 1 -- 172.21.15.45:0/1615503921 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 38) ==== 8354+0+0 (secure 0 0 0) 0x7fee700035f0 con 0x55761884a7f0 2024-04-16T17:13:21.068 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:21.061+0000 7fee9201e740 10 client.27416 fetch_fsmap finished waiting for FSMap version 38 2024-04-16T17:13:21.068 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:21.061+0000 7fee9201e740 10 client.27416 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T17:13:21.068 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:21.061+0000 7fee9201e740 20 client.27416 populate_metadata read hostname 'smithi045' 2024-04-16T17:13:21.068 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:21.061+0000 7fee9201e740 1 --2- 172.21.15.45:0/1615503921 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x5576188633c0 0x5576188657b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:21.068 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:21.061+0000 7fee9201e740 4 client.27416 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:13:21.068 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:21.061+0000 7fee9201e740 1 -- 172.21.15.45:0/1615503921 --> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5576187bc1a0 con 0x5576188633c0 2024-04-16T17:13:21.069 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:21.061+0000 7fee8d200640 1 --2- 172.21.15.45:0/1615503921 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x5576188633c0 0x5576188657b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:21.069 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:21.062+0000 7fee8d200640 1 --2- 172.21.15.45:0/1615503921 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x5576188633c0 0x5576188657b0 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f93cf1fa3b74f3c server_cookie=d1e4aea7b17d9ab4 in_seq=0 out_seq=0 2024-04-16T17:13:21.069 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:21.062+0000 7fee76000640 10 client.27416 ms_handle_connect on v2:172.21.15.104:6838/3660070592 2024-04-16T17:13:21.070 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:21.062+0000 7fee76000640 1 -- 172.21.15.45:0/1615503921 <== mds.0 v2:172.21.15.104:6838/3660070592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5576187bc1a0 con 0x5576188633c0 2024-04-16T17:13:21.070 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:21.062+0000 7fee76000640 10 client.27416 handle_command_reply: tid=0 2024-04-16T17:13:21.070 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:21.062+0000 7fee9201e740 1 client.27416 shutdown 2024-04-16T17:13:21.070 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:21.063+0000 7fee9201e740 1 -- 172.21.15.45:0/1615503921 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55761884a7f0 msgr2=0x5576188503c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:21.070 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:21.063+0000 7fee9201e740 1 --2- 172.21.15.45:0/1615503921 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55761884a7f0 0x5576188503c0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fee70009230 tx=0x7fee700092d0 comp rx=0 tx=0).stop 2024-04-16T17:13:21.070 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:21.063+0000 7fee9201e740 1 -- 172.21.15.45:0/1615503921 shutdown_connections 2024-04-16T17:13:21.070 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:21.063+0000 7fee9201e740 1 --2- 172.21.15.45:0/1615503921 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557618853000 0x557618850900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:21.071 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:21.063+0000 7fee9201e740 1 --2- 172.21.15.45:0/1615503921 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55761884a7f0 0x5576188503c0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:21.071 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:21.063+0000 7fee9201e740 1 --2- 172.21.15.45:0/1615503921 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x5576188633c0 0x5576188657b0 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:21.071 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:21.063+0000 7fee9201e740 1 --2- 172.21.15.45:0/1615503921 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557618849a80 0x55761884fe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:21.071 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:21.063+0000 7fee9201e740 1 -- 172.21.15.45:0/1615503921 >> 172.21.15.45:0/1615503921 conn(0x5576187e45d0 msgr2=0x5576188464c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:13:21.071 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:21.063+0000 7fee76000640 0 client.27416 ms_handle_reset on v2:172.21.15.104:6838/3660070592 2024-04-16T17:13:21.071 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:21.063+0000 7fee9201e740 1 -- 172.21.15.45:0/1615503921 shutdown_connections 2024-04-16T17:13:21.071 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:21.063+0000 7fee9201e740 1 -- 172.21.15.45:0/1615503921 wait complete. 2024-04-16T17:13:21.071 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:21.063+0000 7fee9201e740 20 client.27416 trim_cache size 0 max 16384 2024-04-16T17:13:21.081 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:21.080+0000 7fee9201e740 1 Processor -- start 2024-04-16T17:13:21.081 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:21.080+0000 7fee9201e740 1 -- start start 2024-04-16T17:13:21.082 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:21.080+0000 7fee9201e740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557618849a80 0x5576188525b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:21.082 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:21.080+0000 7fee9201e740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557618853000 0x55761885ec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:21.082 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:21.080+0000 7fee9201e740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55761884f040 0x557618850550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:21.082 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:21.080+0000 7fee9201e740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fee881b8200 con 0x557618849a80 2024-04-16T17:13:21.082 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:21.080+0000 7fee9201e740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fee881b5a70 con 0x557618853000 2024-04-16T17:13:21.082 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:21.080+0000 7fee9201e740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fee881b38e0 con 0x55761884f040 2024-04-16T17:13:21.082 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:21.080+0000 7fee8d200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557618849a80 0x5576188525b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:21.082 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:21.080+0000 7fee8d200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557618849a80 0x5576188525b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36384/0 (socket says 172.21.15.45:36384) 2024-04-16T17:13:21.082 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:21.080+0000 7fee8d200640 1 -- 172.21.15.45:0/431456465 learned_addr learned my addr 172.21.15.45:0/431456465 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:13:21.082 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:21.081+0000 7fee7f400640 1 --2- 172.21.15.45:0/431456465 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557618853000 0x55761885ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:21.083 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:21.081+0000 7fee8d200640 1 --2- 172.21.15.45:0/431456465 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55761884f040 0x557618850550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:21.083 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:21.081+0000 7fee8d200640 1 -- 172.21.15.45:0/431456465 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55761884f040 msgr2=0x557618850550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:13:21.083 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:21.081+0000 7fee8d200640 1 --2- 172.21.15.45:0/431456465 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55761884f040 0x557618850550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:21.083 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:21.081+0000 7fee8d200640 1 -- 172.21.15.45:0/431456465 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557618853000 msgr2=0x55761885ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:13:21.083 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:21.081+0000 7fee8d200640 1 --2- 172.21.15.45:0/431456465 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557618853000 0x55761885ec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:21.083 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:21.081+0000 7fee8d200640 1 -- 172.21.15.45:0/431456465 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557618848d50 con 0x557618849a80 2024-04-16T17:13:21.083 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:21.081+0000 7fee8d200640 1 --2- 172.21.15.45:0/431456465 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557618849a80 0x5576188525b0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fee80011850 tx=0x7fee80004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f922d089d4cab794 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:13:21.083 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:21.081+0000 7fee7cc00640 1 -- 172.21.15.45:0/431456465 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee80034070 con 0x557618849a80 2024-04-16T17:13:21.083 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:21.081+0000 7fee7cc00640 1 -- 172.21.15.45:0/431456465 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fee80028450 con 0x557618849a80 2024-04-16T17:13:21.084 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:21.082+0000 7fee7cc00640 1 -- 172.21.15.45:0/431456465 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee80019690 con 0x557618849a80 2024-04-16T17:13:21.087 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:21.082+0000 7fee9201e740 1 -- 172.21.15.45:0/431456465 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557618849a80 msgr2=0x5576188525b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:21.088 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:21.082+0000 7fee9201e740 1 --2- 172.21.15.45:0/431456465 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557618849a80 0x5576188525b0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fee80011850 tx=0x7fee80004a40 comp rx=0 tx=0).stop 2024-04-16T17:13:21.088 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:21.082+0000 7fee9201e740 1 -- 172.21.15.45:0/431456465 shutdown_connections 2024-04-16T17:13:21.088 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:21.082+0000 7fee9201e740 1 --2- 172.21.15.45:0/431456465 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55761884f040 0x557618850550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:21.088 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:21.082+0000 7fee9201e740 1 --2- 172.21.15.45:0/431456465 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557618853000 0x55761885ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:21.088 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:21.082+0000 7fee9201e740 1 --2- 172.21.15.45:0/431456465 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557618849a80 0x5576188525b0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:21.088 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:21.082+0000 7fee9201e740 1 -- 172.21.15.45:0/431456465 >> 172.21.15.45:0/431456465 conn(0x5576187e45d0 msgr2=0x55761884fe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:13:21.088 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:21.082+0000 7fee9201e740 1 -- 172.21.15.45:0/431456465 shutdown_connections 2024-04-16T17:13:21.088 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:21.082+0000 7fee9201e740 1 -- 172.21.15.45:0/431456465 wait complete. 2024-04-16T17:13:21.088 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:21.083+0000 7fee9201e740 1 -- start start 2024-04-16T17:13:21.089 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:21.083+0000 7fee9201e740 1 Processor -- start 2024-04-16T17:13:21.089 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:21.083+0000 7fee9201e740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557618849a80 0x55761884cf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:21.089 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:21.083+0000 7fee9201e740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55761884f040 0x55761884d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:21.089 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:21.083+0000 7fee9201e740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557618853000 0x5576188c88c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:21.089 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:21.083+0000 7fee9201e740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fee881b3f30 con 0x55761884f040 2024-04-16T17:13:21.089 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:21.083+0000 7fee9201e740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fee881b53a0 con 0x557618849a80 2024-04-16T17:13:21.089 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:21.083+0000 7fee9201e740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fee881b4600 con 0x557618853000 2024-04-16T17:13:21.089 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:21.083+0000 7fee9201e740 10 client.0 _ll_register_callbacks cb 0x7fee800093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:13:21.089 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:21.084+0000 7fee8d200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557618849a80 0x55761884cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:21.089 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:21.084+0000 7fee7f400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55761884f040 0x55761884d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:21.089 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:21.084+0000 7fee8d200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557618849a80 0x55761884cf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:34066/0 (socket says 172.21.15.45:34066) 2024-04-16T17:13:21.089 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:21.084+0000 7fee8d200640 1 -- 172.21.15.45:0/1585367852 learned_addr learned my addr 172.21.15.45:0/1585367852 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:13:21.090 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:21.084+0000 7fee8d200640 1 --2- 172.21.15.45:0/1585367852 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557618853000 0x5576188c88c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:21.090 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:21.084+0000 7fee8d200640 1 -- 172.21.15.45:0/1585367852 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557618853000 msgr2=0x5576188c88c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:13:21.090 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:21.084+0000 7fee8d200640 1 --2- 172.21.15.45:0/1585367852 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557618853000 0x5576188c88c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:21.090 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:21.084+0000 7fee8d200640 1 -- 172.21.15.45:0/1585367852 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55761884f040 msgr2=0x55761884d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:13:21.090 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:21.084+0000 7fee8d200640 1 --2- 172.21.15.45:0/1585367852 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55761884f040 0x55761884d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:21.090 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:21.084+0000 7fee8d200640 1 -- 172.21.15.45:0/1585367852 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5576188c8e00 con 0x557618849a80 2024-04-16T17:13:21.090 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:21.084+0000 7fee8d200640 1 --2- 172.21.15.45:0/1585367852 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557618849a80 0x55761884cf80 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fee80009540 tx=0x7fee80028e40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=185ea9bdfc39a592 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:13:21.090 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:21.084+0000 7fee76000640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-04-16T17:13:21.090 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:21.085+0000 7fee76000640 1 -- 172.21.15.45:0/1585367852 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee80042060 con 0x557618849a80 2024-04-16T17:13:21.090 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:21.085+0000 7fee9201e740 1 -- 172.21.15.45:0/1585367852 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fee80028450 con 0x557618849a80 2024-04-16T17:13:21.091 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:21.085+0000 7fee76000640 1 -- 172.21.15.45:0/1585367852 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9496+0+0 (secure 0 0 0) 0x7fee80003b70 con 0x557618849a80 2024-04-16T17:13:21.091 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:21.085+0000 7fee76000640 1 -- 172.21.15.45:0/1585367852 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fee80034070 con 0x557618849a80 2024-04-16T17:13:21.091 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:21.085+0000 7fee9201e740 10 client.27422 fetch_fsmap learned FSMap version 38 2024-04-16T17:13:21.091 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:21.085+0000 7fee9201e740 1 -- 172.21.15.45:0/1585367852 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x5576188c9f10 con 0x557618849a80 2024-04-16T17:13:21.091 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:21.085+0000 7fee76000640 1 -- 172.21.15.45:0/1585367852 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fee80028450 con 0x557618849a80 2024-04-16T17:13:21.091 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:21.085+0000 7fee76000640 1 -- 172.21.15.45:0/1585367852 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 38) ==== 8354+0+0 (secure 0 0 0) 0x7fee8003a070 con 0x557618849a80 2024-04-16T17:13:21.091 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:21.086+0000 7fee9201e740 10 client.27422 fetch_fsmap finished waiting for FSMap version 38 2024-04-16T17:13:21.091 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:21.086+0000 7fee9201e740 10 client.27422 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T17:13:21.091 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:21.086+0000 7fee9201e740 20 client.27422 populate_metadata read hostname 'smithi045' 2024-04-16T17:13:21.091 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:21.086+0000 7fee9201e740 1 --2- 172.21.15.45:0/1585367852 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x5576188ca370 0x5576188cc790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:21.091 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:21.086+0000 7fee9201e740 4 client.27422 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-16T17:13:21.092 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:21.086+0000 7fee9201e740 1 -- 172.21.15.45:0/1585367852 --> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5576187b6010 con 0x5576188ca370 2024-04-16T17:13:21.092 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:21.086+0000 7fee7f400640 1 --2- 172.21.15.45:0/1585367852 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x5576188ca370 0x5576188cc790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:21.092 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:21.086+0000 7fee7f400640 1 --2- 172.21.15.45:0/1585367852 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x5576188ca370 0x5576188cc790 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1277e858b2925ed server_cookie=7c1c2998496ed722 in_seq=0 out_seq=0 2024-04-16T17:13:21.092 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:21.086+0000 7fee76000640 10 client.27422 ms_handle_connect on v2:172.21.15.104:6838/3660070592 2024-04-16T17:13:21.134 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:21.133+0000 7fee76000640 1 -- 172.21.15.45:0/1585367852 <== mds.0 v2:172.21.15.104:6838/3660070592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5576187b6010 con 0x5576188ca370 2024-04-16T17:13:21.134 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:21.133+0000 7fee76000640 10 client.27422 handle_command_reply: tid=0 2024-04-16T17:13:21.134 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:21.133+0000 7fee9201e740 1 client.27422 shutdown 2024-04-16T17:13:21.134 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:21.133+0000 7fee9201e740 1 -- 172.21.15.45:0/1585367852 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557618849a80 msgr2=0x55761884cf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:21.134 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:21.133+0000 7fee9201e740 1 --2- 172.21.15.45:0/1585367852 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557618849a80 0x55761884cf80 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fee80009540 tx=0x7fee80028e40 comp rx=0 tx=0).stop 2024-04-16T17:13:21.134 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:21.133+0000 7fee9201e740 1 -- 172.21.15.45:0/1585367852 shutdown_connections 2024-04-16T17:13:21.134 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:21.133+0000 7fee9201e740 1 --2- 172.21.15.45:0/1585367852 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x5576188ca370 0x5576188cc790 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:21.134 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:21.133+0000 7fee9201e740 1 --2- 172.21.15.45:0/1585367852 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557618853000 0x5576188c88c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:21.135 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:21.133+0000 7fee9201e740 1 --2- 172.21.15.45:0/1585367852 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55761884f040 0x55761884d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:21.135 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:21.133+0000 7fee9201e740 1 --2- 172.21.15.45:0/1585367852 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557618849a80 0x55761884cf80 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:21.135 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:21.133+0000 7fee76000640 0 client.27422 ms_handle_reset on v2:172.21.15.104:6838/3660070592 2024-04-16T17:13:21.135 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:21.133+0000 7fee9201e740 1 -- 172.21.15.45:0/1585367852 >> 172.21.15.45:0/1585367852 conn(0x5576187e45d0 msgr2=0x557618850620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:13:21.135 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:21.134+0000 7fee9201e740 1 -- 172.21.15.45:0/1585367852 shutdown_connections 2024-04-16T17:13:21.135 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:21.134+0000 7fee9201e740 1 -- 172.21.15.45:0/1585367852 wait complete. 2024-04-16T17:13:21.135 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:21.134+0000 7fee9201e740 20 client.27422 trim_cache size 0 max 16384 2024-04-16T17:13:21.136 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-04-16T17:13:21.136 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-04-16T17:13:21.136 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-04-16T17:13:21.136 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-04-16T17:13:21.137 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:21.134+0000 7fee8dc00640 1 -- 172.21.15.45:0/915556850 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fee6c0a3d10 msgr2=0x7fee6c0a61d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:21.137 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:21.134+0000 7fee8dc00640 1 --2- 172.21.15.45:0/915556850 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fee6c0a3d10 0x7fee6c0a61d0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x55761884af50 tx=0x7fee780d4a10 comp rx=0 tx=0).stop 2024-04-16T17:13:21.137 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:21.134+0000 7fee8dc00640 1 -- 172.21.15.45:0/915556850 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fee88100db0 msgr2=0x7fee881a4940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:21.137 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:21.134+0000 7fee8dc00640 1 --2- 172.21.15.45:0/915556850 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fee88100db0 0x7fee881a4940 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fee7800e9f0 tx=0x7fee7800eeb0 comp rx=0 tx=0).stop 2024-04-16T17:13:21.137 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:21.135+0000 7fee8dc00640 1 -- 172.21.15.45:0/915556850 shutdown_connections 2024-04-16T17:13:21.137 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:21.135+0000 7fee8dc00640 1 --2- 172.21.15.45:0/915556850 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fee6c0a3d10 0x7fee6c0a61d0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:21.137 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:21.135+0000 7fee8dc00640 1 --2- 172.21.15.45:0/915556850 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fee881061b0 0x7fee8819eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:21.138 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:21.135+0000 7fee8dc00640 1 --2- 172.21.15.45:0/915556850 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fee881016d0 0x7fee881a4e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:21.138 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:21.135+0000 7fee8dc00640 1 --2- 172.21.15.45:0/915556850 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fee88100db0 0x7fee881a4940 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:21.138 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:21.135+0000 7fee8dc00640 1 -- 172.21.15.45:0/915556850 >> 172.21.15.45:0/915556850 conn(0x7fee880fc820 msgr2=0x7fee880fd870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:13:21.138 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:21.135+0000 7fee8dc00640 1 -- 172.21.15.45:0/915556850 shutdown_connections 2024-04-16T17:13:21.138 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:21.135+0000 7fee8dc00640 1 -- 172.21.15.45:0/915556850 wait complete. 2024-04-16T17:13:21.280 INFO:tasks.workunit.client.0.smithi045.stdout: 1 108142 1.73 MB/sec execute 203 sec latency 2398.372 ms 2024-04-16T17:13:22.280 INFO:tasks.workunit.client.0.smithi045.stdout: 1 108142 1.72 MB/sec execute 204 sec latency 3398.457 ms 2024-04-16T17:13:22.514 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:13:22 smithi045 ceph-mon[31676]: pgmap v203: 129 pgs: 129 active+clean; 56 MiB data, 729 MiB used, 1.0 TiB / 1.0 TiB avail; 1007 KiB/s rd, 903 KiB/s wr, 128 op/s 2024-04-16T17:13:22.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:13:22 smithi117 ceph-mon[36730]: pgmap v203: 129 pgs: 129 active+clean; 56 MiB data, 729 MiB used, 1.0 TiB / 1.0 TiB avail; 1007 KiB/s rd, 903 KiB/s wr, 128 op/s 2024-04-16T17:13:22.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:13:22 smithi104 ceph-mon[36457]: pgmap v203: 129 pgs: 129 active+clean; 56 MiB data, 729 MiB used, 1.0 TiB / 1.0 TiB avail; 1007 KiB/s rd, 903 KiB/s wr, 128 op/s 2024-04-16T17:13:23.280 INFO:tasks.workunit.client.0.smithi045.stdout: 1 108447 1.71 MB/sec execute 205 sec latency 3402.995 ms 2024-04-16T17:13:24.282 INFO:tasks.workunit.client.0.smithi045.stdout: 1 108740 1.71 MB/sec execute 206 sec latency 146.079 ms 2024-04-16T17:13:24.804 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:13:24 smithi117 ceph-mon[36730]: pgmap v204: 129 pgs: 129 active+clean; 56 MiB data, 729 MiB used, 1.0 TiB / 1.0 TiB avail; 541 KiB/s rd, 669 KiB/s wr, 72 op/s 2024-04-16T17:13:24.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:13:24 smithi104 ceph-mon[36457]: pgmap v204: 129 pgs: 129 active+clean; 56 MiB data, 729 MiB used, 1.0 TiB / 1.0 TiB avail; 541 KiB/s rd, 669 KiB/s wr, 72 op/s 2024-04-16T17:13:25.282 INFO:tasks.workunit.client.0.smithi045.stdout: 1 109716 1.74 MB/sec execute 207 sec latency 67.274 ms 2024-04-16T17:13:26.255 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:13:24 smithi045 ceph-mon[31676]: pgmap v204: 129 pgs: 129 active+clean; 56 MiB data, 729 MiB used, 1.0 TiB / 1.0 TiB avail; 541 KiB/s rd, 669 KiB/s wr, 72 op/s 2024-04-16T17:13:26.282 INFO:tasks.workunit.client.0.smithi045.stdout: 1 110326 1.75 MB/sec execute 208 sec latency 34.678 ms 2024-04-16T17:13:26.468 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:13:26 smithi045 ceph-mon[31676]: pgmap v205: 129 pgs: 129 active+clean; 56 MiB data, 729 MiB used, 1.0 TiB / 1.0 TiB avail; 541 KiB/s rd, 669 KiB/s wr, 72 op/s 2024-04-16T17:13:26.804 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:13:26 smithi117 ceph-mon[36730]: pgmap v205: 129 pgs: 129 active+clean; 56 MiB data, 729 MiB used, 1.0 TiB / 1.0 TiB avail; 541 KiB/s rd, 669 KiB/s wr, 72 op/s 2024-04-16T17:13:26.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:13:26 smithi104 ceph-mon[36457]: pgmap v205: 129 pgs: 129 active+clean; 56 MiB data, 729 MiB used, 1.0 TiB / 1.0 TiB avail; 541 KiB/s rd, 669 KiB/s wr, 72 op/s 2024-04-16T17:13:27.282 INFO:tasks.workunit.client.0.smithi045.stdout: 1 110682 1.75 MB/sec execute 209 sec latency 21.841 ms 2024-04-16T17:13:28.282 INFO:tasks.workunit.client.0.smithi045.stdout: 1 111125 1.75 MB/sec execute 210 sec latency 99.560 ms 2024-04-16T17:13:28.757 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:13:28 smithi045 ceph-mon[31676]: pgmap v206: 129 pgs: 129 active+clean; 56 MiB data, 722 MiB used, 1.0 TiB / 1.0 TiB avail; 794 KiB/s rd, 1013 KiB/s wr, 102 op/s 2024-04-16T17:13:28.804 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:13:28 smithi117 ceph-mon[36730]: pgmap v206: 129 pgs: 129 active+clean; 56 MiB data, 722 MiB used, 1.0 TiB / 1.0 TiB avail; 794 KiB/s rd, 1013 KiB/s wr, 102 op/s 2024-04-16T17:13:28.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:13:28 smithi104 ceph-mon[36457]: pgmap v206: 129 pgs: 129 active+clean; 56 MiB data, 722 MiB used, 1.0 TiB / 1.0 TiB avail; 794 KiB/s rd, 1013 KiB/s wr, 102 op/s 2024-04-16T17:13:29.288 INFO:tasks.workunit.client.0.smithi045.stdout: 1 111509 1.75 MB/sec execute 211 sec latency 75.627 ms 2024-04-16T17:13:30.288 INFO:tasks.workunit.client.0.smithi045.stdout: 1 111692 1.74 MB/sec execute 212 sec latency 354.120 ms 2024-04-16T17:13:30.723 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:13:30 smithi045 ceph-mon[31676]: pgmap v207: 129 pgs: 129 active+clean; 56 MiB data, 722 MiB used, 1.0 TiB / 1.0 TiB avail; 663 KiB/s rd, 678 KiB/s wr, 76 op/s 2024-04-16T17:13:30.804 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:13:30 smithi117 ceph-mon[36730]: pgmap v207: 129 pgs: 129 active+clean; 56 MiB data, 722 MiB used, 1.0 TiB / 1.0 TiB avail; 663 KiB/s rd, 678 KiB/s wr, 76 op/s 2024-04-16T17:13:30.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:13:30 smithi104 ceph-mon[36457]: pgmap v207: 129 pgs: 129 active+clean; 56 MiB data, 722 MiB used, 1.0 TiB / 1.0 TiB avail; 663 KiB/s rd, 678 KiB/s wr, 76 op/s 2024-04-16T17:13:31.288 INFO:tasks.workunit.client.0.smithi045.stdout: 1 111692 1.74 MB/sec execute 213 sec latency 1354.183 ms 2024-04-16T17:13:32.288 INFO:tasks.workunit.client.0.smithi045.stdout: 1 111695 1.73 MB/sec execute 214 sec latency 2347.484 ms 2024-04-16T17:13:32.804 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:13:32 smithi117 ceph-mon[36730]: pgmap v208: 129 pgs: 129 active+clean; 59 MiB data, 735 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 919 KiB/s wr, 110 op/s 2024-04-16T17:13:32.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:13:32 smithi104 ceph-mon[36457]: pgmap v208: 129 pgs: 129 active+clean; 59 MiB data, 735 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 919 KiB/s wr, 110 op/s 2024-04-16T17:13:33.288 INFO:tasks.workunit.client.0.smithi045.stdout: 1 112001 1.72 MB/sec execute 215 sec latency 79.561 ms 2024-04-16T17:13:33.769 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-16T17:13:33.770 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-16T17:13:33.830 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:13:32 smithi045 ceph-mon[31676]: pgmap v208: 129 pgs: 129 active+clean; 59 MiB data, 735 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 919 KiB/s wr, 110 op/s 2024-04-16T17:13:33.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:33.954+0000 7ff2fa400640 1 -- 172.21.15.45:0/197321005 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff2f4102ac0 msgr2=0x7ff2f4106b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:33.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:33.954+0000 7ff2fa400640 1 --2- 172.21.15.45:0/197321005 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff2f4102ac0 0x7ff2f4106b00 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7ff2e0009900 tx=0x7ff2e002bc30 comp rx=0 tx=0).stop 2024-04-16T17:13:33.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:33.954+0000 7ff2fa400640 1 -- 172.21.15.45:0/197321005 shutdown_connections 2024-04-16T17:13:33.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:33.954+0000 7ff2fa400640 1 --2- 172.21.15.45:0/197321005 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff2f41071e0 0x7ff2f410edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:33.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:33.954+0000 7ff2fa400640 1 --2- 172.21.15.45:0/197321005 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff2f4102ac0 0x7ff2f4106b00 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:33.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:33.954+0000 7ff2fa400640 1 --2- 172.21.15.45:0/197321005 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff2f4102110 0x7ff2f41024f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:33.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:33.954+0000 7ff2fa400640 1 -- 172.21.15.45:0/197321005 >> 172.21.15.45:0/197321005 conn(0x7ff2f40fd9d0 msgr2=0x7ff2f40ffdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:13:33.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:33.954+0000 7ff2fa400640 1 -- 172.21.15.45:0/197321005 shutdown_connections 2024-04-16T17:13:33.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:33.954+0000 7ff2fa400640 1 -- 172.21.15.45:0/197321005 wait complete. 2024-04-16T17:13:33.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:33.955+0000 7ff2fa400640 1 Processor -- start 2024-04-16T17:13:33.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:33.955+0000 7ff2fa400640 1 -- start start 2024-04-16T17:13:33.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:33.956+0000 7ff2fa400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff2f4102110 0x7ff2f41a06b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:33.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:33.956+0000 7ff2fa400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff2f41071e0 0x7ff2f41a0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:33.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:33.956+0000 7ff2fa400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff2f419a820 0x7ff2f419acd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:33.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:33.956+0000 7ff2fa400640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7ff2f41122c0 con 0x7ff2f4102110 2024-04-16T17:13:33.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:33.956+0000 7ff2fa400640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7ff2f4112140 con 0x7ff2f41071e0 2024-04-16T17:13:33.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:33.956+0000 7ff2fa400640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7ff2f4112440 con 0x7ff2f419a820 2024-04-16T17:13:33.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:33.956+0000 7ff2f9000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff2f4102110 0x7ff2f41a06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:33.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:33.956+0000 7ff2f9000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff2f4102110 0x7ff2f41a06b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56568/0 (socket says 172.21.15.45:56568) 2024-04-16T17:13:33.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:33.956+0000 7ff2f9000640 1 -- 172.21.15.45:0/1676854347 learned_addr learned my addr 172.21.15.45:0/1676854347 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:13:33.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:33.956+0000 7ff2f3e00640 1 --2- 172.21.15.45:0/1676854347 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff2f41071e0 0x7ff2f41a0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:33.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:33.956+0000 7ff2f9a00640 1 --2- 172.21.15.45:0/1676854347 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff2f419a820 0x7ff2f419acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:33.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:33.957+0000 7ff2f9000640 1 -- 172.21.15.45:0/1676854347 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff2f419a820 msgr2=0x7ff2f419acd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:33.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:33.957+0000 7ff2f9000640 1 --2- 172.21.15.45:0/1676854347 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff2f419a820 0x7ff2f419acd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:33.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:33.957+0000 7ff2f9000640 1 -- 172.21.15.45:0/1676854347 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff2f41071e0 msgr2=0x7ff2f41a0bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:33.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:33.957+0000 7ff2f9000640 1 --2- 172.21.15.45:0/1676854347 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff2f41071e0 0x7ff2f41a0bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:33.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:33.957+0000 7ff2f9000640 1 -- 172.21.15.45:0/1676854347 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2f419b3b0 con 0x7ff2f4102110 2024-04-16T17:13:33.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:33.957+0000 7ff2f3e00640 1 --2- 172.21.15.45:0/1676854347 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff2f41071e0 0x7ff2f41a0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:13:33.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:33.957+0000 7ff2f9000640 1 --2- 172.21.15.45:0/1676854347 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff2f4102110 0x7ff2f41a06b0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7ff2e800ea60 tx=0x7ff2e800ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:13:33.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:33.957+0000 7ff2f1600640 1 -- 172.21.15.45:0/1676854347 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2e800cde0 con 0x7ff2f4102110 2024-04-16T17:13:33.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:33.957+0000 7ff2f1600640 1 -- 172.21.15.45:0/1676854347 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff2e80035f0 con 0x7ff2f4102110 2024-04-16T17:13:33.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:33.957+0000 7ff2fa400640 1 -- 172.21.15.45:0/1676854347 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2f419b6a0 con 0x7ff2f4102110 2024-04-16T17:13:33.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:33.958+0000 7ff2f1600640 1 -- 172.21.15.45:0/1676854347 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2e8010670 con 0x7ff2f4102110 2024-04-16T17:13:33.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:33.958+0000 7ff2fa400640 1 -- 172.21.15.45:0/1676854347 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2f41a7850 con 0x7ff2f4102110 2024-04-16T17:13:33.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:33.961+0000 7ff2fe611740 1 Processor -- start 2024-04-16T17:13:33.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:33.961+0000 7ff2fe611740 1 -- start start 2024-04-16T17:13:33.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:33.961+0000 7ff2fe611740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c96bd9a390 0x55c96bd9a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:33.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:33.961+0000 7ff2fe611740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c96bd9acb0 0x55c96bda3350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:33.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:33.961+0000 7ff2fe611740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c96bda3890 0x55c96bda5c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:33.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:33.961+0000 7ff2fe611740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55c96bd00120 con 0x55c96bda3890 2024-04-16T17:13:33.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:33.961+0000 7ff2fe611740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55c96bca6340 con 0x55c96bd9acb0 2024-04-16T17:13:33.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:33.961+0000 7ff2fe611740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55c96bcfc0e0 con 0x55c96bd9a390 2024-04-16T17:13:33.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:33.962+0000 7ff2f9a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c96bd9a390 0x55c96bd9a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:33.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:33.962+0000 7ff2f3e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c96bda3890 0x55c96bda5c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:33.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:33.962+0000 7ff2f9a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c96bd9a390 0x55c96bd9a770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:57084/0 (socket says 172.21.15.45:57084) 2024-04-16T17:13:33.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:33.962+0000 7ff2f9a00640 1 -- 172.21.15.45:0/671602832 learned_addr learned my addr 172.21.15.45:0/671602832 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:13:33.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:33.962+0000 7ff2f3e00640 1 -- 172.21.15.45:0/671602832 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c96bd9a390 msgr2=0x55c96bd9a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:13:33.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:33.962+0000 7ff2f3e00640 1 --2- 172.21.15.45:0/671602832 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c96bd9a390 0x55c96bd9a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:33.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:33.962+0000 7ff2f3e00640 1 -- 172.21.15.45:0/671602832 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c96bd9acb0 msgr2=0x55c96bda3350 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-16T17:13:33.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:33.962+0000 7ff2f3e00640 1 --2- 172.21.15.45:0/671602832 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c96bd9acb0 0x55c96bda3350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:33.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:33.962+0000 7ff2f3e00640 1 -- 172.21.15.45:0/671602832 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c96badc7d0 con 0x55c96bda3890 2024-04-16T17:13:33.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:33.962+0000 7ff2f9a00640 1 --2- 172.21.15.45:0/671602832 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c96bd9a390 0x55c96bd9a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:13:33.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:33.963+0000 7ff2f3e00640 1 --2- 172.21.15.45:0/671602832 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c96bda3890 0x55c96bda5c80 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7ff2e0036040 tx=0x7ff2e0009560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cc0b57f4c96cfa1 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:13:33.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:33.963+0000 7ff2f1600640 1 -- 172.21.15.45:0/1676854347 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7ff2e80037a0 con 0x7ff2f4102110 2024-04-16T17:13:33.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:33.963+0000 7ff2e6000640 1 -- 172.21.15.45:0/671602832 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2e0045070 con 0x55c96bda3890 2024-04-16T17:13:33.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:33.963+0000 7ff2e6000640 1 -- 172.21.15.45:0/671602832 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff2e002c650 con 0x55c96bda3890 2024-04-16T17:13:33.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:33.963+0000 7ff2e6000640 1 -- 172.21.15.45:0/671602832 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2e0004ee0 con 0x55c96bda3890 2024-04-16T17:13:33.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:33.964+0000 7ff2fe611740 1 -- 172.21.15.45:0/671602832 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c96bda3890 msgr2=0x55c96bda5c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:33.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:33.964+0000 7ff2fe611740 1 --2- 172.21.15.45:0/671602832 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c96bda3890 0x55c96bda5c80 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7ff2e0036040 tx=0x7ff2e0009560 comp rx=0 tx=0).stop 2024-04-16T17:13:33.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:33.964+0000 7ff2fe611740 1 -- 172.21.15.45:0/671602832 shutdown_connections 2024-04-16T17:13:33.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:33.964+0000 7ff2fe611740 1 --2- 172.21.15.45:0/671602832 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c96bda3890 0x55c96bda5c80 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:33.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:33.964+0000 7ff2fe611740 1 --2- 172.21.15.45:0/671602832 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c96bd9acb0 0x55c96bda3350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:33.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:33.964+0000 7ff2fe611740 1 --2- 172.21.15.45:0/671602832 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c96bd9a390 0x55c96bd9a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:33.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:33.964+0000 7ff2fe611740 1 -- 172.21.15.45:0/671602832 >> 172.21.15.45:0/671602832 conn(0x55c96bd94ac0 msgr2=0x55c96bd94ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:13:33.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:33.964+0000 7ff2fe611740 1 -- 172.21.15.45:0/671602832 shutdown_connections 2024-04-16T17:13:33.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:33.964+0000 7ff2fe611740 1 -- 172.21.15.45:0/671602832 wait complete. 2024-04-16T17:13:33.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:33.965+0000 7ff2fe611740 1 -- start start 2024-04-16T17:13:33.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:33.965+0000 7ff2fe611740 1 Processor -- start 2024-04-16T17:13:33.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:33.965+0000 7ff2f1600640 1 --2- 172.21.15.45:0/1676854347 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7ff2dc0a3cb0 0x7ff2dc0a6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:33.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:33.965+0000 7ff2f1600640 1 -- 172.21.15.45:0/1676854347 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9496+0+0 (secure 0 0 0) 0x7ff2e80d0ef0 con 0x7ff2f4102110 2024-04-16T17:13:33.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:33.965+0000 7ff2fe611740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c96bd9a390 0x55c96bda0940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:33.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:33.966+0000 7ff2fe611740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c96bd9acb0 0x55c96bda0e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:33.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:33.966+0000 7ff2fe611740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c96bda3890 0x55c96bda13c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:33.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:33.966+0000 7ff2fe611740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55c96bda6620 con 0x55c96bd9a390 2024-04-16T17:13:33.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:33.966+0000 7ff2fe611740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55c96bcb9710 con 0x55c96bda3890 2024-04-16T17:13:33.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:33.966+0000 7ff2fe611740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55c96bda67a0 con 0x55c96bd9acb0 2024-04-16T17:13:33.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:33.966+0000 7ff2f9a00640 1 --2- 172.21.15.45:0/1676854347 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7ff2dc0a3cb0 0x7ff2dc0a6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:33.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:33.966+0000 7ff2fe611740 10 client.0 _ll_register_callbacks cb 0x55c96bcb4af0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:13:33.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:33.966+0000 7ff2f3e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c96bd9acb0 0x55c96bda0e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:33.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:33.966+0000 7ff2f3e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c96bd9acb0 0x55c96bda0e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:57100/0 (socket says 172.21.15.45:57100) 2024-04-16T17:13:33.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:33.966+0000 7ff2f3e00640 1 -- 172.21.15.45:0/3135345191 learned_addr learned my addr 172.21.15.45:0/3135345191 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:13:33.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:33.966+0000 7ff2f9000640 1 --2- 172.21.15.45:0/3135345191 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c96bd9a390 0x55c96bda0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:33.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:33.966+0000 7ff2f9a00640 1 --2- 172.21.15.45:0/3135345191 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c96bda3890 0x55c96bda13c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:33.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:33.967+0000 7ff2f9a00640 1 --2- 172.21.15.45:0/1676854347 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7ff2dc0a3cb0 0x7ff2dc0a6170 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7ff2ec005fd0 tx=0x7ff2ec011920 comp rx=0 tx=0).ready entity=mgr.25411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:13:33.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:33.967+0000 7ff2f9000640 1 -- 172.21.15.45:0/3135345191 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c96bd9acb0 msgr2=0x55c96bda0e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:13:33.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:33.967+0000 7ff2f9000640 1 --2- 172.21.15.45:0/3135345191 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c96bd9acb0 0x55c96bda0e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:33.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:33.967+0000 7ff2f9000640 1 -- 172.21.15.45:0/3135345191 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c96bda3890 msgr2=0x55c96bda13c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:13:33.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:33.967+0000 7ff2f9000640 1 --2- 172.21.15.45:0/3135345191 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c96bda3890 0x55c96bda13c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:33.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:33.967+0000 7ff2f9000640 1 -- 172.21.15.45:0/3135345191 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c96ba123f0 con 0x55c96bd9a390 2024-04-16T17:13:33.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:33.967+0000 7ff2f9000640 1 --2- 172.21.15.45:0/3135345191 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c96bd9a390 0x55c96bda0940 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7ff2e80ce210 tx=0x7ff2e809c660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=561ec28aa274461 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:13:33.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:33.967+0000 7ff2daa00640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-16T17:13:33.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:33.967+0000 7ff2daa00640 1 -- 172.21.15.45:0/3135345191 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2e809ca60 con 0x55c96bd9a390 2024-04-16T17:13:33.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:33.967+0000 7ff2daa00640 1 -- 172.21.15.45:0/3135345191 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9496+0+0 (secure 0 0 0) 0x7ff2e80ce490 con 0x55c96bd9a390 2024-04-16T17:13:33.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:33.967+0000 7ff2fe611740 1 -- 172.21.15.45:0/3135345191 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff2e002c650 con 0x55c96bd9a390 2024-04-16T17:13:33.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:33.968+0000 7ff2daa00640 1 -- 172.21.15.45:0/3135345191 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7ff2e80ce690 con 0x55c96bd9a390 2024-04-16T17:13:33.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:33.968+0000 7ff2fe611740 10 client.17268 fetch_fsmap learned FSMap version 38 2024-04-16T17:13:33.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:33.968+0000 7ff2fe611740 1 -- 172.21.15.45:0/3135345191 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55c96b9d20f0 con 0x55c96bd9a390 2024-04-16T17:13:33.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:33.968+0000 7ff2daa00640 1 -- 172.21.15.45:0/3135345191 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff2e002c650 con 0x55c96bd9a390 2024-04-16T17:13:33.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:33.968+0000 7ff2daa00640 1 -- 172.21.15.45:0/3135345191 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 38) ==== 8354+0+0 (secure 0 0 0) 0x7ff2e80987b0 con 0x55c96bd9a390 2024-04-16T17:13:33.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:33.968+0000 7ff2fe611740 10 client.17268 fetch_fsmap finished waiting for FSMap version 38 2024-04-16T17:13:33.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:33.968+0000 7ff2fe611740 10 client.17268 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:13:33.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:33.968+0000 7ff2fe611740 20 client.17268 populate_metadata read hostname 'smithi045' 2024-04-16T17:13:33.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:33.968+0000 7ff2fe611740 1 --2- 172.21.15.45:0/3135345191 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55c96bdb3c90 0x55c96bdb60c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:33.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:33.968+0000 7ff2fe611740 4 client.17268 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:13:33.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:33.968+0000 7ff2fe611740 1 -- 172.21.15.45:0/3135345191 --> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c96bd0db40 con 0x55c96bdb3c90 2024-04-16T17:13:33.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:33.969+0000 7ff2f3e00640 1 --2- 172.21.15.45:0/3135345191 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55c96bdb3c90 0x55c96bdb60c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:33.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:33.969+0000 7ff2f3e00640 1 --2- 172.21.15.45:0/3135345191 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55c96bdb3c90 0x55c96bdb60c0 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a3f18b50fe1f6dc5 server_cookie=ccd51880c126393d in_seq=0 out_seq=0 2024-04-16T17:13:33.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:33.969+0000 7ff2daa00640 10 client.17268 ms_handle_connect on v2:172.21.15.104:6838/3660070592 2024-04-16T17:13:33.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:33.970+0000 7ff2daa00640 1 -- 172.21.15.45:0/3135345191 <== mds.0 v2:172.21.15.104:6838/3660070592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c96bd0db40 con 0x55c96bdb3c90 2024-04-16T17:13:33.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:33.970+0000 7ff2daa00640 10 client.17268 handle_command_reply: tid=0 2024-04-16T17:13:33.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:33.970+0000 7ff2fe611740 1 client.17268 shutdown 2024-04-16T17:13:33.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:33.970+0000 7ff2fe611740 1 -- 172.21.15.45:0/3135345191 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c96bd9a390 msgr2=0x55c96bda0940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:33.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:33.970+0000 7ff2fe611740 1 --2- 172.21.15.45:0/3135345191 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c96bd9a390 0x55c96bda0940 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7ff2e80ce210 tx=0x7ff2e809c660 comp rx=0 tx=0).stop 2024-04-16T17:13:33.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:33.970+0000 7ff2fe611740 1 -- 172.21.15.45:0/3135345191 shutdown_connections 2024-04-16T17:13:33.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:33.970+0000 7ff2fe611740 1 --2- 172.21.15.45:0/3135345191 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55c96bdb3c90 0x55c96bdb60c0 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:33.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:33.971+0000 7ff2fe611740 1 --2- 172.21.15.45:0/3135345191 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c96bd9acb0 0x55c96bda0e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:33.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:33.971+0000 7ff2fe611740 1 --2- 172.21.15.45:0/3135345191 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c96bda3890 0x55c96bda13c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:33.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:33.971+0000 7ff2daa00640 0 client.17268 ms_handle_reset on v2:172.21.15.104:6838/3660070592 2024-04-16T17:13:33.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:33.971+0000 7ff2fe611740 1 --2- 172.21.15.45:0/3135345191 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c96bd9a390 0x55c96bda0940 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:33.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:33.971+0000 7ff2fe611740 1 -- 172.21.15.45:0/3135345191 >> 172.21.15.45:0/3135345191 conn(0x55c96bd94ac0 msgr2=0x55c96bd96810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:13:33.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:33.971+0000 7ff2fe611740 1 -- 172.21.15.45:0/3135345191 shutdown_connections 2024-04-16T17:13:33.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:33.971+0000 7ff2fe611740 1 -- 172.21.15.45:0/3135345191 wait complete. 2024-04-16T17:13:33.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:33.971+0000 7ff2fe611740 20 client.17268 trim_cache size 0 max 16384 2024-04-16T17:13:34.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.004+0000 7ff2fe611740 1 Processor -- start 2024-04-16T17:13:34.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.004+0000 7ff2fe611740 1 -- start start 2024-04-16T17:13:34.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.004+0000 7ff2fe611740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c96bd9a390 0x55c96bda3070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:34.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.004+0000 7ff2fe611740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c96bd9acb0 0x55c96bdaf4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:34.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.004+0000 7ff2fe611740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c96bda3890 0x55c96bd9bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:34.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.004+0000 7ff2fe611740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7ff2f41b3de0 con 0x55c96bda3890 2024-04-16T17:13:34.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.004+0000 7ff2fe611740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7ff2f41b1650 con 0x55c96bd9a390 2024-04-16T17:13:34.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.004+0000 7ff2fe611740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7ff2f41af4c0 con 0x55c96bd9acb0 2024-04-16T17:13:34.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.005+0000 7ff2f9000640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c96bd9a390 0x55c96bda3070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:34.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.005+0000 7ff2f9000640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c96bd9a390 0x55c96bda3070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:39352/0 (socket says 172.21.15.45:39352) 2024-04-16T17:13:34.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.005+0000 7ff2f9000640 1 -- 172.21.15.45:0/2757065628 learned_addr learned my addr 172.21.15.45:0/2757065628 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:13:34.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.005+0000 7ff2f3e00640 1 --2- 172.21.15.45:0/2757065628 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c96bd9acb0 0x55c96bdaf4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:34.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.005+0000 7ff2f9a00640 1 --2- 172.21.15.45:0/2757065628 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c96bda3890 0x55c96bd9bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:34.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.005+0000 7ff2f9000640 1 -- 172.21.15.45:0/2757065628 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c96bd9acb0 msgr2=0x55c96bdaf4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:13:34.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.005+0000 7ff2f9000640 1 --2- 172.21.15.45:0/2757065628 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c96bd9acb0 0x55c96bdaf4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:34.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.005+0000 7ff2f9000640 1 -- 172.21.15.45:0/2757065628 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c96bda3890 msgr2=0x55c96bd9bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:13:34.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.005+0000 7ff2f9000640 1 --2- 172.21.15.45:0/2757065628 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c96bda3890 0x55c96bd9bcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:34.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.005+0000 7ff2f9000640 1 -- 172.21.15.45:0/2757065628 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c96bdb6740 con 0x55c96bd9a390 2024-04-16T17:13:34.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.005+0000 7ff2f9a00640 1 --2- 172.21.15.45:0/2757065628 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c96bda3890 0x55c96bd9bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:13:34.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.006+0000 7ff2f9000640 1 --2- 172.21.15.45:0/2757065628 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c96bd9a390 0x55c96bda3070 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7ff2e8099430 tx=0x7ff2e809c6d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=44709b85e82391e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:13:34.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.006+0000 7ff2e5600640 1 -- 172.21.15.45:0/2757065628 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2e80a56d0 con 0x55c96bd9a390 2024-04-16T17:13:34.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.006+0000 7ff2e5600640 1 -- 172.21.15.45:0/2757065628 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff2e80a0e10 con 0x55c96bd9a390 2024-04-16T17:13:34.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.006+0000 7ff2e5600640 1 -- 172.21.15.45:0/2757065628 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2e809beb0 con 0x55c96bd9a390 2024-04-16T17:13:34.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.006+0000 7ff2fe611740 1 -- 172.21.15.45:0/2757065628 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c96bd9a390 msgr2=0x55c96bda3070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:34.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.006+0000 7ff2fe611740 1 --2- 172.21.15.45:0/2757065628 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c96bd9a390 0x55c96bda3070 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7ff2e8099430 tx=0x7ff2e809c6d0 comp rx=0 tx=0).stop 2024-04-16T17:13:34.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.007+0000 7ff2fe611740 1 -- 172.21.15.45:0/2757065628 shutdown_connections 2024-04-16T17:13:34.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.007+0000 7ff2fe611740 1 --2- 172.21.15.45:0/2757065628 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c96bd9acb0 0x55c96bdaf4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:34.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.007+0000 7ff2fe611740 1 --2- 172.21.15.45:0/2757065628 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c96bda3890 0x55c96bd9bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:34.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.007+0000 7ff2fe611740 1 --2- 172.21.15.45:0/2757065628 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c96bd9a390 0x55c96bda3070 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:34.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.007+0000 7ff2fe611740 1 -- 172.21.15.45:0/2757065628 >> 172.21.15.45:0/2757065628 conn(0x55c96bd94ac0 msgr2=0x55c96bd93290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:13:34.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.007+0000 7ff2fe611740 1 -- 172.21.15.45:0/2757065628 shutdown_connections 2024-04-16T17:13:34.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.007+0000 7ff2fe611740 1 -- 172.21.15.45:0/2757065628 wait complete. 2024-04-16T17:13:34.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.007+0000 7ff2fe611740 1 -- start start 2024-04-16T17:13:34.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.007+0000 7ff2fe611740 1 Processor -- start 2024-04-16T17:13:34.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.008+0000 7ff2fe611740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c96bd9a390 0x55c96bdabca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:34.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.008+0000 7ff2fe611740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c96bd9acb0 0x55c96bda7010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:34.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.008+0000 7ff2fe611740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c96bda3890 0x55c96bda7550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:34.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.008+0000 7ff2fe611740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7ff2f41afb10 con 0x55c96bd9acb0 2024-04-16T17:13:34.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.008+0000 7ff2fe611740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7ff2f41b0f80 con 0x55c96bd9a390 2024-04-16T17:13:34.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.008+0000 7ff2fe611740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7ff2f41b01e0 con 0x55c96bda3890 2024-04-16T17:13:34.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.008+0000 7ff2fe611740 10 client.0 _ll_register_callbacks cb 0x7ff2e809c830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:13:34.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.008+0000 7ff2f3e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c96bd9acb0 0x55c96bda7010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:34.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.009+0000 7ff2f9a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c96bda3890 0x55c96bda7550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:34.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.009+0000 7ff2f3e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c96bd9acb0 0x55c96bda7010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56614/0 (socket says 172.21.15.45:56614) 2024-04-16T17:13:34.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.009+0000 7ff2f3e00640 1 -- 172.21.15.45:0/4012689955 learned_addr learned my addr 172.21.15.45:0/4012689955 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:13:34.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.009+0000 7ff2f9000640 1 --2- 172.21.15.45:0/4012689955 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c96bd9a390 0x55c96bdabca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:34.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.009+0000 7ff2f9a00640 1 -- 172.21.15.45:0/4012689955 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c96bd9a390 msgr2=0x55c96bdabca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:13:34.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.009+0000 7ff2f9a00640 1 --2- 172.21.15.45:0/4012689955 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c96bd9a390 0x55c96bdabca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:34.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.009+0000 7ff2f9a00640 1 -- 172.21.15.45:0/4012689955 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c96bd9acb0 msgr2=0x55c96bda7010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:13:34.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.009+0000 7ff2f9a00640 1 --2- 172.21.15.45:0/4012689955 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c96bd9acb0 0x55c96bda7010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:34.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.009+0000 7ff2f9a00640 1 -- 172.21.15.45:0/4012689955 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c96bda7c40 con 0x55c96bda3890 2024-04-16T17:13:34.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.009+0000 7ff2f9000640 1 --2- 172.21.15.45:0/4012689955 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c96bd9a390 0x55c96bdabca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:13:34.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.009+0000 7ff2f3e00640 1 --2- 172.21.15.45:0/4012689955 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c96bd9acb0 0x55c96bda7010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:13:34.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.009+0000 7ff2f9a00640 1 --2- 172.21.15.45:0/4012689955 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c96bda3890 0x55c96bda7550 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x55c96bd943c0 tx=0x7ff2ec004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=db2ab42bb48e0a05 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:13:34.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.009+0000 7ff2daa00640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-04-16T17:13:34.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.010+0000 7ff2daa00640 1 -- 172.21.15.45:0/4012689955 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2ec02c020 con 0x55c96bda3890 2024-04-16T17:13:34.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.010+0000 7ff2daa00640 1 -- 172.21.15.45:0/4012689955 <== mon.1 v2:172.21.15.117:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9496+0+0 (secure 0 0 0) 0x7ff2ec01ed00 con 0x55c96bda3890 2024-04-16T17:13:34.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.010+0000 7ff2fe611740 1 -- 172.21.15.45:0/4012689955 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff2e80a0e10 con 0x55c96bda3890 2024-04-16T17:13:34.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.010+0000 7ff2daa00640 1 -- 172.21.15.45:0/4012689955 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7ff2ec037050 con 0x55c96bda3890 2024-04-16T17:13:34.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.010+0000 7ff2fe611740 10 client.27385 fetch_fsmap learned FSMap version 38 2024-04-16T17:13:34.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.010+0000 7ff2fe611740 1 -- 172.21.15.45:0/4012689955 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55c96be1a1a0 con 0x55c96bda3890 2024-04-16T17:13:34.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.010+0000 7ff2daa00640 1 -- 172.21.15.45:0/4012689955 <== mon.1 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff2e80a0e10 con 0x55c96bda3890 2024-04-16T17:13:34.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.011+0000 7ff2daa00640 1 -- 172.21.15.45:0/4012689955 <== mon.1 v2:172.21.15.117:3300/0 5 ==== fsmap(e 38) ==== 8354+0+0 (secure 0 0 0) 0x7ff2ec0303a0 con 0x55c96bda3890 2024-04-16T17:13:34.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.011+0000 7ff2fe611740 10 client.27385 fetch_fsmap finished waiting for FSMap version 38 2024-04-16T17:13:34.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.011+0000 7ff2fe611740 10 client.27385 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:13:34.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.011+0000 7ff2fe611740 20 client.27385 populate_metadata read hostname 'smithi045' 2024-04-16T17:13:34.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.011+0000 7ff2fe611740 1 --2- 172.21.15.45:0/4012689955 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55c96be1a9a0 0x55c96be1cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:34.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.011+0000 7ff2fe611740 4 client.27385 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-16T17:13:34.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.011+0000 7ff2fe611740 1 -- 172.21.15.45:0/4012689955 --> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55c96bd079b0 con 0x55c96be1a9a0 2024-04-16T17:13:34.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.011+0000 7ff2f9000640 1 --2- 172.21.15.45:0/4012689955 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55c96be1a9a0 0x55c96be1cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:34.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.011+0000 7ff2f9000640 1 --2- 172.21.15.45:0/4012689955 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55c96be1a9a0 0x55c96be1cdc0 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=27c2bf28261d65a5 server_cookie=7f4fbacd99a53462 in_seq=0 out_seq=0 2024-04-16T17:13:34.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.011+0000 7ff2daa00640 10 client.27385 ms_handle_connect on v2:172.21.15.104:6838/3660070592 2024-04-16T17:13:34.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.012+0000 7ff2daa00640 1 -- 172.21.15.45:0/4012689955 <== mds.0 v2:172.21.15.104:6838/3660070592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55c96bd079b0 con 0x55c96be1a9a0 2024-04-16T17:13:34.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.012+0000 7ff2daa00640 10 client.27385 handle_command_reply: tid=0 2024-04-16T17:13:34.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.012+0000 7ff2fe611740 1 client.27385 shutdown 2024-04-16T17:13:34.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.012+0000 7ff2fe611740 1 -- 172.21.15.45:0/4012689955 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c96bda3890 msgr2=0x55c96bda7550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:34.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.012+0000 7ff2fe611740 1 --2- 172.21.15.45:0/4012689955 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c96bda3890 0x55c96bda7550 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x55c96bd943c0 tx=0x7ff2ec004a40 comp rx=0 tx=0).stop 2024-04-16T17:13:34.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.012+0000 7ff2fe611740 1 -- 172.21.15.45:0/4012689955 shutdown_connections 2024-04-16T17:13:34.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.012+0000 7ff2fe611740 1 --2- 172.21.15.45:0/4012689955 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55c96be1a9a0 0x55c96be1cdc0 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:34.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.012+0000 7ff2fe611740 1 --2- 172.21.15.45:0/4012689955 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c96bda3890 0x55c96bda7550 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:34.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.012+0000 7ff2fe611740 1 --2- 172.21.15.45:0/4012689955 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c96bd9acb0 0x55c96bda7010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:34.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.012+0000 7ff2daa00640 0 client.27385 ms_handle_reset on v2:172.21.15.104:6838/3660070592 2024-04-16T17:13:34.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.012+0000 7ff2fe611740 1 --2- 172.21.15.45:0/4012689955 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c96bd9a390 0x55c96bdabca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:34.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.012+0000 7ff2fe611740 1 -- 172.21.15.45:0/4012689955 >> 172.21.15.45:0/4012689955 conn(0x55c96bd94ac0 msgr2=0x55c96bd93b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:13:34.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.012+0000 7ff2fe611740 1 -- 172.21.15.45:0/4012689955 shutdown_connections 2024-04-16T17:13:34.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.012+0000 7ff2fe611740 1 -- 172.21.15.45:0/4012689955 wait complete. 2024-04-16T17:13:34.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.012+0000 7ff2fe611740 20 client.27385 trim_cache size 0 max 16384 2024-04-16T17:13:34.017 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-16T17:13:34.017 INFO:teuthology.orchestra.run.smithi045.stdout: "status": "no active scrubs running", 2024-04-16T17:13:34.017 INFO:teuthology.orchestra.run.smithi045.stdout: "scrubs": {} 2024-04-16T17:13:34.017 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-16T17:13:34.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.013+0000 7ff2fa400640 1 -- 172.21.15.45:0/1676854347 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7ff2dc0a3cb0 msgr2=0x7ff2dc0a6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:34.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.013+0000 7ff2fa400640 1 --2- 172.21.15.45:0/1676854347 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7ff2dc0a3cb0 0x7ff2dc0a6170 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7ff2ec005fd0 tx=0x7ff2ec011920 comp rx=0 tx=0).stop 2024-04-16T17:13:34.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.013+0000 7ff2fa400640 1 -- 172.21.15.45:0/1676854347 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff2f4102110 msgr2=0x7ff2f41a06b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:34.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.013+0000 7ff2fa400640 1 --2- 172.21.15.45:0/1676854347 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff2f4102110 0x7ff2f41a06b0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7ff2e800ea60 tx=0x7ff2e800ef20 comp rx=0 tx=0).stop 2024-04-16T17:13:34.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.013+0000 7ff2fa400640 1 -- 172.21.15.45:0/1676854347 shutdown_connections 2024-04-16T17:13:34.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.013+0000 7ff2fa400640 1 --2- 172.21.15.45:0/1676854347 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7ff2dc0a3cb0 0x7ff2dc0a6170 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:34.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.013+0000 7ff2fa400640 1 --2- 172.21.15.45:0/1676854347 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff2f419a820 0x7ff2f419acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:34.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.013+0000 7ff2fa400640 1 --2- 172.21.15.45:0/1676854347 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff2f41071e0 0x7ff2f41a0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:34.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.013+0000 7ff2fa400640 1 --2- 172.21.15.45:0/1676854347 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff2f4102110 0x7ff2f41a06b0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:34.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.013+0000 7ff2fa400640 1 -- 172.21.15.45:0/1676854347 >> 172.21.15.45:0/1676854347 conn(0x7ff2f40fd9d0 msgr2=0x7ff2f40fe8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:13:34.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.013+0000 7ff2fa400640 1 -- 172.21.15.45:0/1676854347 shutdown_connections 2024-04-16T17:13:34.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.013+0000 7ff2fa400640 1 -- 172.21.15.45:0/1676854347 wait complete. 2024-04-16T17:13:34.030 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-16T17:13:34.030 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-16T17:13:34.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.204+0000 7fa2b7200640 1 -- 172.21.15.45:0/1075682880 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa2b8074cf0 msgr2=0x7fa2b810e180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:34.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.204+0000 7fa2b7200640 1 --2- 172.21.15.45:0/1075682880 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa2b8074cf0 0x7fa2b810e180 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fa2a4009900 tx=0x7fa2a402bcc0 comp rx=0 tx=0).stop 2024-04-16T17:13:34.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.205+0000 7fa2b7200640 1 -- 172.21.15.45:0/1075682880 shutdown_connections 2024-04-16T17:13:34.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.205+0000 7fa2b7200640 1 --2- 172.21.15.45:0/1075682880 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa2b8074cf0 0x7fa2b810e180 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:34.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.205+0000 7fa2b7200640 1 --2- 172.21.15.45:0/1075682880 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa2b810e6c0 0x7fa2b8110ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:34.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.205+0000 7fa2b7200640 1 --2- 172.21.15.45:0/1075682880 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa2b80743d0 0x7fa2b80747b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:34.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.205+0000 7fa2b7200640 1 -- 172.21.15.45:0/1075682880 >> 172.21.15.45:0/1075682880 conn(0x7fa2b80fd930 msgr2=0x7fa2b80ffd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:13:34.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.205+0000 7fa2b7200640 1 -- 172.21.15.45:0/1075682880 shutdown_connections 2024-04-16T17:13:34.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.205+0000 7fa2b7200640 1 -- 172.21.15.45:0/1075682880 wait complete. 2024-04-16T17:13:34.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.206+0000 7fa2b7200640 1 Processor -- start 2024-04-16T17:13:34.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.206+0000 7fa2b7200640 1 -- start start 2024-04-16T17:13:34.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.207+0000 7fa2b7200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa2b80743d0 0x7fa2b81a0640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:34.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.207+0000 7fa2b7200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa2b8074cf0 0x7fa2b81a0b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:34.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.207+0000 7fa2b7200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa2b810e6c0 0x7fa2b819a800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:34.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.207+0000 7fa2b7200640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fa2b8114420 con 0x7fa2b8074cf0 2024-04-16T17:13:34.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.207+0000 7fa2b7200640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fa2b81142a0 con 0x7fa2b80743d0 2024-04-16T17:13:34.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.207+0000 7fa2b7200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa2b81145a0 con 0x7fa2b810e6c0 2024-04-16T17:13:34.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.207+0000 7fa2b6800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa2b810e6c0 0x7fa2b819a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:34.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.207+0000 7fa2b5e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa2b80743d0 0x7fa2b81a0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:34.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.207+0000 7fa2b5e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa2b80743d0 0x7fa2b81a0640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:39392/0 (socket says 172.21.15.45:39392) 2024-04-16T17:13:34.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.207+0000 7fa2b5e00640 1 -- 172.21.15.45:0/3459318323 learned_addr learned my addr 172.21.15.45:0/3459318323 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:13:34.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.207+0000 7fa2b5400640 1 --2- 172.21.15.45:0/3459318323 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa2b8074cf0 0x7fa2b81a0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:34.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.208+0000 7fa2b5e00640 1 -- 172.21.15.45:0/3459318323 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa2b810e6c0 msgr2=0x7fa2b819a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:34.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.208+0000 7fa2b5e00640 1 --2- 172.21.15.45:0/3459318323 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa2b810e6c0 0x7fa2b819a800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:34.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.208+0000 7fa2b5e00640 1 -- 172.21.15.45:0/3459318323 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa2b8074cf0 msgr2=0x7fa2b81a0b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:34.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.208+0000 7fa2b5e00640 1 --2- 172.21.15.45:0/3459318323 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa2b8074cf0 0x7fa2b81a0b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:34.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.208+0000 7fa2b5e00640 1 -- 172.21.15.45:0/3459318323 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2b819afd0 con 0x7fa2b80743d0 2024-04-16T17:13:34.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.208+0000 7fa2b5400640 1 --2- 172.21.15.45:0/3459318323 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa2b8074cf0 0x7fa2b81a0b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:13:34.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.208+0000 7fa2b5e00640 1 --2- 172.21.15.45:0/3459318323 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa2b80743d0 0x7fa2b81a0640 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fa2a800e9f0 tx=0x7fa2a800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:13:34.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.208+0000 7fa2aea00640 1 -- 172.21.15.45:0/3459318323 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2a800cd80 con 0x7fa2b80743d0 2024-04-16T17:13:34.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.209+0000 7fa2b7200640 1 -- 172.21.15.45:0/3459318323 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2b819b2c0 con 0x7fa2b80743d0 2024-04-16T17:13:34.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.209+0000 7fa2aea00640 1 -- 172.21.15.45:0/3459318323 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa2a8010910 con 0x7fa2b80743d0 2024-04-16T17:13:34.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.209+0000 7fa2aea00640 1 -- 172.21.15.45:0/3459318323 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2a80188f0 con 0x7fa2b80743d0 2024-04-16T17:13:34.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.209+0000 7fa2b7200640 1 -- 172.21.15.45:0/3459318323 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2b81a7490 con 0x7fa2b80743d0 2024-04-16T17:13:34.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.210+0000 7fa2b7200640 1 -- 172.21.15.45:0/3459318323 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2b8075a80 con 0x7fa2b80743d0 2024-04-16T17:13:34.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.213+0000 7fa2aea00640 1 -- 172.21.15.45:0/3459318323 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fa2a8010ac0 con 0x7fa2b80743d0 2024-04-16T17:13:34.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.214+0000 7fa2aea00640 1 --2- 172.21.15.45:0/3459318323 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fa29c0a3da0 0x7fa29c0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:34.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.215+0000 7fa2aea00640 1 -- 172.21.15.45:0/3459318323 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9496+0+0 (secure 0 0 0) 0x7fa2a80d1270 con 0x7fa2b80743d0 2024-04-16T17:13:34.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.215+0000 7fa2b5400640 1 --2- 172.21.15.45:0/3459318323 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fa29c0a3da0 0x7fa29c0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:34.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.216+0000 7fa2b5400640 1 --2- 172.21.15.45:0/3459318323 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fa29c0a3da0 0x7fa29c0a6260 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fa2b819bb00 tx=0x7fa2a402df90 comp rx=0 tx=0).ready entity=mgr.25411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:13:34.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.216+0000 7fa2aea00640 1 -- 172.21.15.45:0/3459318323 <== mon.2 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa2a809d4e0 con 0x7fa2b80743d0 2024-04-16T17:13:34.288 INFO:tasks.workunit.client.0.smithi045.stdout: 1 112184 1.71 MB/sec execute 216 sec latency 295.903 ms 2024-04-16T17:13:34.292 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:34.290+0000 7fcccde00640 1 -- 172.21.15.45:0/490836778 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fccc8104450 msgr2=0x7fccc8104830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:34.292 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:34.290+0000 7fcccde00640 1 --2- 172.21.15.45:0/490836778 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fccc8104450 0x7fccc8104830 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fccb8009860 tx=0x7fccb802bb70 comp rx=0 tx=0).stop 2024-04-16T17:13:34.293 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:34.290+0000 7fcccde00640 1 -- 172.21.15.45:0/490836778 shutdown_connections 2024-04-16T17:13:34.293 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:34.290+0000 7fcccde00640 1 --2- 172.21.15.45:0/490836778 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fccc8109520 0x7fccc81110f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:34.293 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:34.290+0000 7fcccde00640 1 --2- 172.21.15.45:0/490836778 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fccc8104e00 0x7fccc8108e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:34.293 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:34.290+0000 7fcccde00640 1 --2- 172.21.15.45:0/490836778 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fccc8104450 0x7fccc8104830 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:34.293 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:34.290+0000 7fcccde00640 1 -- 172.21.15.45:0/490836778 >> 172.21.15.45:0/490836778 conn(0x7fccc80ffcf0 msgr2=0x7fccc8102110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:13:34.293 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:34.291+0000 7fcccde00640 1 -- 172.21.15.45:0/490836778 shutdown_connections 2024-04-16T17:13:34.294 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:34.291+0000 7fcccde00640 1 -- 172.21.15.45:0/490836778 wait complete. 2024-04-16T17:13:34.294 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:34.292+0000 7fcccde00640 1 Processor -- start 2024-04-16T17:13:34.294 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:34.292+0000 7fcccde00640 1 -- start start 2024-04-16T17:13:34.297 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:34.292+0000 7fcccde00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fccc8104450 0x7fccc81a0740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:34.297 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:34.292+0000 7fcccde00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fccc8109520 0x7fccc81a0c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:34.297 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:34.292+0000 7fcccde00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fccc819a8f0 0x7fccc819ada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:34.297 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:34.292+0000 7fcccde00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fccc81145b0 con 0x7fccc8104450 2024-04-16T17:13:34.297 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:34.292+0000 7fcccde00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fccc8114430 con 0x7fccc819a8f0 2024-04-16T17:13:34.297 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:34.292+0000 7fcccde00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fccc8114730 con 0x7fccc8109520 2024-04-16T17:13:34.298 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:34.293+0000 7fccc6000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fccc8109520 0x7fccc81a0c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:34.298 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:34.293+0000 7fccc6000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fccc8109520 0x7fccc81a0c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:57136/0 (socket says 172.21.15.45:57136) 2024-04-16T17:13:34.298 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:34.293+0000 7fccc6000640 1 -- 172.21.15.45:0/589809298 learned_addr learned my addr 172.21.15.45:0/589809298 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:13:34.298 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:34.293+0000 7fccc6a00640 1 --2- 172.21.15.45:0/589809298 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fccc8104450 0x7fccc81a0740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:34.298 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:34.293+0000 7fccc7400640 1 --2- 172.21.15.45:0/589809298 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fccc819a8f0 0x7fccc819ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:34.298 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:34.293+0000 7fccc6000640 1 -- 172.21.15.45:0/589809298 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fccc819a8f0 msgr2=0x7fccc819ada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:34.298 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:34.293+0000 7fccc6000640 1 --2- 172.21.15.45:0/589809298 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fccc819a8f0 0x7fccc819ada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:34.299 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:34.293+0000 7fccc6000640 1 -- 172.21.15.45:0/589809298 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fccc8104450 msgr2=0x7fccc81a0740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:34.299 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:34.293+0000 7fccc6000640 1 --2- 172.21.15.45:0/589809298 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fccc8104450 0x7fccc81a0740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:34.299 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:34.293+0000 7fccc6000640 1 -- 172.21.15.45:0/589809298 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fccc819b410 con 0x7fccc8109520 2024-04-16T17:13:34.299 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:34.293+0000 7fccc6000640 1 --2- 172.21.15.45:0/589809298 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fccc8109520 0x7fccc81a0c80 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fccb400e9a0 tx=0x7fccb400ee60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:13:34.299 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:34.294+0000 7fcccca00640 1 -- 172.21.15.45:0/589809298 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccb400cd80 con 0x7fccc8109520 2024-04-16T17:13:34.299 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:34.294+0000 7fcccca00640 1 -- 172.21.15.45:0/589809298 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fccb40035f0 con 0x7fccc8109520 2024-04-16T17:13:34.299 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:34.294+0000 7fcccde00640 1 -- 172.21.15.45:0/589809298 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fccc819b700 con 0x7fccc8109520 2024-04-16T17:13:34.300 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:34.294+0000 7fcccde00640 1 -- 172.21.15.45:0/589809298 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fccc81a79a0 con 0x7fccc8109520 2024-04-16T17:13:34.300 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:34.294+0000 7fcccca00640 1 -- 172.21.15.45:0/589809298 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccb4010670 con 0x7fccc8109520 2024-04-16T17:13:34.301 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:34.298+0000 7fcccca00640 1 -- 172.21.15.45:0/589809298 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fccb40037a0 con 0x7fccc8109520 2024-04-16T17:13:34.301 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:34.300+0000 7fccd0bcc740 1 Processor -- start 2024-04-16T17:13:34.301 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:34.300+0000 7fccd0bcc740 1 -- start start 2024-04-16T17:13:34.302 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:34.300+0000 7fccd0bcc740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ece0021610 0x55ece0020360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:34.302 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:34.300+0000 7fcccca00640 1 --2- 172.21.15.45:0/589809298 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fccac0a3e40 0x7fccac0a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:34.304 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:34.301+0000 7fccd0bcc740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ece00208a0 0x55ece0020d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:34.304 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:34.301+0000 7fccd0bcc740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ece0029e20 0x55ece002c210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:34.304 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:34.301+0000 7fccd0bcc740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55ecdff85780 con 0x55ece00208a0 2024-04-16T17:13:34.304 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:34.301+0000 7fccd0bcc740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55ecdff2bf40 con 0x55ece0021610 2024-04-16T17:13:34.304 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:34.301+0000 7fccd0bcc740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55ecdff81740 con 0x55ece0029e20 2024-04-16T17:13:34.305 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:34.301+0000 7fcccca00640 1 -- 172.21.15.45:0/589809298 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9496+0+0 (secure 0 0 0) 0x7fccb40d0e30 con 0x7fccc8109520 2024-04-16T17:13:34.305 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:34.301+0000 7fccc6000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ece00208a0 0x55ece0020d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:34.305 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:34.301+0000 7fccc7400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ece0021610 0x55ece0020360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:34.305 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:34.301+0000 7fccc6000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ece00208a0 0x55ece0020d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56672/0 (socket says 172.21.15.45:56672) 2024-04-16T17:13:34.305 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:34.301+0000 7fccc6000640 1 -- 172.21.15.45:0/2621510041 learned_addr learned my addr 172.21.15.45:0/2621510041 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:13:34.305 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:34.301+0000 7fccc7400640 1 --2- 172.21.15.45:0/2621510041 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ece0029e20 0x55ece002c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:34.306 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:34.302+0000 7fccc6a00640 1 --2- 172.21.15.45:0/589809298 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fccac0a3e40 0x7fccac0a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:34.306 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:34.302+0000 7fccc7400640 1 -- 172.21.15.45:0/2621510041 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ece0029e20 msgr2=0x55ece002c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:13:34.306 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:34.302+0000 7fccc7400640 1 --2- 172.21.15.45:0/2621510041 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ece0029e20 0x55ece002c210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:34.306 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:34.302+0000 7fccc7400640 1 -- 172.21.15.45:0/2621510041 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ece00208a0 msgr2=0x55ece0020d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:13:34.306 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:34.302+0000 7fccc7400640 1 --2- 172.21.15.45:0/2621510041 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ece00208a0 0x55ece0020d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:34.306 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:34.302+0000 7fccc7400640 1 -- 172.21.15.45:0/2621510041 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ecdfc98210 con 0x55ece0021610 2024-04-16T17:13:34.306 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:34.302+0000 7fccc6000640 1 --2- 172.21.15.45:0/2621510041 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ece00208a0 0x55ece0020d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:13:34.307 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:34.302+0000 7fccc7400640 1 --2- 172.21.15.45:0/2621510041 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ece0021610 0x55ece0020360 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fccc000eb30 tx=0x7fccc0010040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bc891ddea9ee8e0b server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:13:34.307 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:34.303+0000 7fccb3e00640 1 -- 172.21.15.45:0/2621510041 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccc000cb00 con 0x55ece0021610 2024-04-16T17:13:34.307 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:34.303+0000 7fccb3e00640 1 -- 172.21.15.45:0/2621510041 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fccc0003e50 con 0x55ece0021610 2024-04-16T17:13:34.307 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:34.303+0000 7fccb3e00640 1 -- 172.21.15.45:0/2621510041 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccc0004710 con 0x55ece0021610 2024-04-16T17:13:34.312 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:34.304+0000 7fccd0bcc740 1 -- 172.21.15.45:0/2621510041 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ece0021610 msgr2=0x55ece0020360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:34.313 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:34.304+0000 7fccd0bcc740 1 --2- 172.21.15.45:0/2621510041 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ece0021610 0x55ece0020360 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fccc000eb30 tx=0x7fccc0010040 comp rx=0 tx=0).stop 2024-04-16T17:13:34.313 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:34.304+0000 7fccc6a00640 1 --2- 172.21.15.45:0/589809298 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fccac0a3e40 0x7fccac0a6300 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fccb8009830 tx=0x7fccb80056d0 comp rx=0 tx=0).ready entity=mgr.25411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:13:34.313 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:34.304+0000 7fccd0bcc740 1 -- 172.21.15.45:0/2621510041 shutdown_connections 2024-04-16T17:13:34.313 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:34.304+0000 7fccd0bcc740 1 --2- 172.21.15.45:0/2621510041 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ece0029e20 0x55ece002c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:34.313 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:34.304+0000 7fccd0bcc740 1 --2- 172.21.15.45:0/2621510041 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ece00208a0 0x55ece0020d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:34.313 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:34.304+0000 7fccd0bcc740 1 --2- 172.21.15.45:0/2621510041 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ece0021610 0x55ece0020360 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:34.314 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:34.304+0000 7fccd0bcc740 1 -- 172.21.15.45:0/2621510041 >> 172.21.15.45:0/2621510041 conn(0x55ecdffbb5d0 msgr2=0x55ece001b590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:13:34.314 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:34.304+0000 7fccd0bcc740 1 -- 172.21.15.45:0/2621510041 shutdown_connections 2024-04-16T17:13:34.314 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:34.304+0000 7fccd0bcc740 1 -- 172.21.15.45:0/2621510041 wait complete. 2024-04-16T17:13:34.314 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:34.305+0000 7fccd0bcc740 1 -- start start 2024-04-16T17:13:34.314 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:34.305+0000 7fccd0bcc740 1 Processor -- start 2024-04-16T17:13:34.314 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:34.306+0000 7fccd0bcc740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ece00208a0 0x55ece0026c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:34.315 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:34.306+0000 7fccd0bcc740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ece0021610 0x55ece00271d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:34.315 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:34.306+0000 7fccd0bcc740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ece0029e20 0x55ece0027710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:34.315 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:34.306+0000 7fccd0bcc740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55ece002cb50 con 0x55ece0029e20 2024-04-16T17:13:34.315 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:34.306+0000 7fccd0bcc740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55ecdff3f390 con 0x55ece0021610 2024-04-16T17:13:34.315 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:34.306+0000 7fccd0bcc740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55ece002ccd0 con 0x55ece00208a0 2024-04-16T17:13:34.315 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:34.306+0000 7fccd0bcc740 10 client.0 _ll_register_callbacks cb 0x55ecdff28f60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:13:34.316 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:34.306+0000 7fccc6a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ece00208a0 0x55ece0026c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:34.316 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:34.306+0000 7fccc6a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ece00208a0 0x55ece0026c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:57160/0 (socket says 172.21.15.45:57160) 2024-04-16T17:13:34.316 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:34.306+0000 7fccc6a00640 1 -- 172.21.15.45:0/419251191 learned_addr learned my addr 172.21.15.45:0/419251191 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:13:34.316 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:34.306+0000 7fccc7400640 1 --2- 172.21.15.45:0/419251191 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ece0029e20 0x55ece0027710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:34.316 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:34.306+0000 7fccc6000640 1 --2- 172.21.15.45:0/419251191 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ece0021610 0x55ece00271d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:34.316 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:34.307+0000 7fccc6a00640 1 -- 172.21.15.45:0/419251191 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ece0021610 msgr2=0x55ece00271d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:13:34.316 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:34.307+0000 7fccc6a00640 1 --2- 172.21.15.45:0/419251191 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ece0021610 0x55ece00271d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:34.317 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:34.307+0000 7fccc6a00640 1 -- 172.21.15.45:0/419251191 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ece0029e20 msgr2=0x55ece0027710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:13:34.317 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:34.307+0000 7fccc6a00640 1 --2- 172.21.15.45:0/419251191 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ece0029e20 0x55ece0027710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:34.317 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:34.307+0000 7fccc6a00640 1 -- 172.21.15.45:0/419251191 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ecdfd64620 con 0x55ece00208a0 2024-04-16T17:13:34.317 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:34.307+0000 7fccc7400640 1 --2- 172.21.15.45:0/419251191 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ece0029e20 0x55ece0027710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:13:34.317 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:34.308+0000 7fccc6a00640 1 --2- 172.21.15.45:0/419251191 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ece00208a0 0x55ece0026c90 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fccb80095f0 tx=0x7fccb8005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b1cae295369a81a1 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:13:34.317 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:34.308+0000 7fccb0c00640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-04-16T17:13:34.318 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:34.309+0000 7fccb0c00640 1 -- 172.21.15.45:0/419251191 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccb8043070 con 0x55ece00208a0 2024-04-16T17:13:34.318 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:34.309+0000 7fccd0bcc740 1 -- 172.21.15.45:0/419251191 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fccc0003e50 con 0x55ece00208a0 2024-04-16T17:13:34.318 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:34.309+0000 7fccb0c00640 1 -- 172.21.15.45:0/419251191 <== mon.1 v2:172.21.15.117:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9496+0+0 (secure 0 0 0) 0x7fccb802c510 con 0x55ece00208a0 2024-04-16T17:13:34.318 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:34.309+0000 7fccb0c00640 1 -- 172.21.15.45:0/419251191 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fccb8058050 con 0x55ece00208a0 2024-04-16T17:13:34.318 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:34.309+0000 7fccd0bcc740 10 client.27400 fetch_fsmap learned FSMap version 38 2024-04-16T17:13:34.318 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:34.309+0000 7fccd0bcc740 1 -- 172.21.15.45:0/419251191 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55ece0038cd0 con 0x55ece00208a0 2024-04-16T17:13:34.318 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:34.309+0000 7fccb0c00640 1 -- 172.21.15.45:0/419251191 <== mon.1 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fccc0003e50 con 0x55ece00208a0 2024-04-16T17:13:34.319 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:34.310+0000 7fccb0c00640 1 -- 172.21.15.45:0/419251191 <== mon.1 v2:172.21.15.117:3300/0 5 ==== fsmap(e 38) ==== 8354+0+0 (secure 0 0 0) 0x7fccb8032c70 con 0x55ece00208a0 2024-04-16T17:13:34.319 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:34.310+0000 7fccd0bcc740 10 client.27400 fetch_fsmap finished waiting for FSMap version 38 2024-04-16T17:13:34.319 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:34.310+0000 7fccd0bcc740 10 client.27400 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T17:13:34.319 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:34.310+0000 7fccd0bcc740 20 client.27400 populate_metadata read hostname 'smithi045' 2024-04-16T17:13:34.319 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:34.310+0000 7fccd0bcc740 1 --2- 172.21.15.45:0/419251191 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55ece003a190 0x55ece003c580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:34.319 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:34.310+0000 7fccd0bcc740 4 client.27400 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:13:34.321 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:34.310+0000 7fccd0bcc740 1 -- 172.21.15.45:0/419251191 --> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ecdff931a0 con 0x55ece003a190 2024-04-16T17:13:34.321 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:34.310+0000 7fccc6000640 1 --2- 172.21.15.45:0/419251191 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55ece003a190 0x55ece003c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:34.321 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:34.311+0000 7fccc6000640 1 --2- 172.21.15.45:0/419251191 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55ece003a190 0x55ece003c580 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=576a398aa027e1cc server_cookie=d7576cbafbec8c6e in_seq=0 out_seq=0 2024-04-16T17:13:34.322 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:34.311+0000 7fccb0c00640 10 client.27400 ms_handle_connect on v2:172.21.15.104:6838/3660070592 2024-04-16T17:13:34.324 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:34.319+0000 7fccb0c00640 1 -- 172.21.15.45:0/419251191 <== mds.0 v2:172.21.15.104:6838/3660070592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ecdff931a0 con 0x55ece003a190 2024-04-16T17:13:34.324 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:34.319+0000 7fccb0c00640 10 client.27400 handle_command_reply: tid=0 2024-04-16T17:13:34.324 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:34.319+0000 7fccd0bcc740 1 client.27400 shutdown 2024-04-16T17:13:34.324 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:34.319+0000 7fccd0bcc740 1 -- 172.21.15.45:0/419251191 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ece00208a0 msgr2=0x55ece0026c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:34.324 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:34.319+0000 7fccd0bcc740 1 --2- 172.21.15.45:0/419251191 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ece00208a0 0x55ece0026c90 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fccb80095f0 tx=0x7fccb8005f90 comp rx=0 tx=0).stop 2024-04-16T17:13:34.324 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:34.319+0000 7fccd0bcc740 1 -- 172.21.15.45:0/419251191 shutdown_connections 2024-04-16T17:13:34.325 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:34.319+0000 7fccd0bcc740 1 --2- 172.21.15.45:0/419251191 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ece0029e20 0x55ece0027710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:34.325 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:34.319+0000 7fccd0bcc740 1 --2- 172.21.15.45:0/419251191 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ece0021610 0x55ece00271d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:34.325 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:34.319+0000 7fccd0bcc740 1 --2- 172.21.15.45:0/419251191 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55ece003a190 0x55ece003c580 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:34.325 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:34.319+0000 7fccd0bcc740 1 --2- 172.21.15.45:0/419251191 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ece00208a0 0x55ece0026c90 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:34.325 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:34.319+0000 7fccb0c00640 0 client.27400 ms_handle_reset on v2:172.21.15.104:6838/3660070592 2024-04-16T17:13:34.325 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:34.319+0000 7fccd0bcc740 1 -- 172.21.15.45:0/419251191 >> 172.21.15.45:0/419251191 conn(0x55ecdffbb5d0 msgr2=0x55ece001d2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:13:34.326 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:34.320+0000 7fccd0bcc740 1 -- 172.21.15.45:0/419251191 shutdown_connections 2024-04-16T17:13:34.326 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:34.320+0000 7fccd0bcc740 1 -- 172.21.15.45:0/419251191 wait complete. 2024-04-16T17:13:34.326 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:34.320+0000 7fccd0bcc740 20 client.27400 trim_cache size 0 max 16384 2024-04-16T17:13:34.340 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:34.340+0000 7fccd0bcc740 1 Processor -- start 2024-04-16T17:13:34.340 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:34.340+0000 7fccd0bcc740 1 -- start start 2024-04-16T17:13:34.342 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:34.340+0000 7fccd0bcc740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ece00208a0 0x55ece0035a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:34.342 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:34.340+0000 7fccd0bcc740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ece0021610 0x55ece0027530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:34.342 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:34.340+0000 7fccd0bcc740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ece0029e20 0x55ece0028ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:34.342 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:34.340+0000 7fccd0bcc740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fccc81b40a0 con 0x55ece0029e20 2024-04-16T17:13:34.342 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:34.340+0000 7fccd0bcc740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fccc81b1910 con 0x55ece00208a0 2024-04-16T17:13:34.342 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:34.340+0000 7fccd0bcc740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fccc81af780 con 0x55ece0021610 2024-04-16T17:13:34.342 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:34.341+0000 7fccc6a00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ece00208a0 0x55ece0035a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:34.342 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:34.341+0000 7fccc6000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ece0021610 0x55ece0027530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:34.342 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:34.341+0000 7fccc6000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ece0021610 0x55ece0027530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:57170/0 (socket says 172.21.15.45:57170) 2024-04-16T17:13:34.342 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:34.341+0000 7fccc6a00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ece00208a0 0x55ece0035a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:39438/0 (socket says 172.21.15.45:39438) 2024-04-16T17:13:34.343 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:34.341+0000 7fccc6000640 1 -- 172.21.15.45:0/3602901844 learned_addr learned my addr 172.21.15.45:0/3602901844 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:13:34.343 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:34.341+0000 7fccc7400640 1 --2- 172.21.15.45:0/3602901844 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ece0029e20 0x55ece0028ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:34.343 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:34.341+0000 7fccc6a00640 1 -- 172.21.15.45:0/3602901844 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ece0021610 msgr2=0x55ece0027530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:13:34.343 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:34.341+0000 7fccc6a00640 1 --2- 172.21.15.45:0/3602901844 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ece0021610 0x55ece0027530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:34.343 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:34.341+0000 7fccc6a00640 1 -- 172.21.15.45:0/3602901844 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ece0029e20 msgr2=0x55ece0028ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:13:34.343 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:34.341+0000 7fccc6a00640 1 --2- 172.21.15.45:0/3602901844 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ece0029e20 0x55ece0028ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:34.343 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:34.341+0000 7fccc6a00640 1 -- 172.21.15.45:0/3602901844 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ece001fb70 con 0x55ece00208a0 2024-04-16T17:13:34.343 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:34.341+0000 7fccc6000640 1 --2- 172.21.15.45:0/3602901844 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ece0021610 0x55ece0027530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:13:34.343 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:34.341+0000 7fccc7400640 1 --2- 172.21.15.45:0/3602901844 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ece0029e20 0x55ece0028ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:13:34.345 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:34.342+0000 7fccc6a00640 1 --2- 172.21.15.45:0/3602901844 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ece00208a0 0x55ece0035a50 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fccb802c0b0 tx=0x7fccb802bb70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ddfb3878b887b22a server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:13:34.345 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:34.342+0000 7fccb3400640 1 -- 172.21.15.45:0/3602901844 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccb8061070 con 0x55ece00208a0 2024-04-16T17:13:34.345 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:34.342+0000 7fccb3400640 1 -- 172.21.15.45:0/3602901844 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fccb804f070 con 0x55ece00208a0 2024-04-16T17:13:34.345 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:34.342+0000 7fccb3400640 1 -- 172.21.15.45:0/3602901844 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccb8043070 con 0x55ece00208a0 2024-04-16T17:13:34.345 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:34.343+0000 7fccd0bcc740 1 -- 172.21.15.45:0/3602901844 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ece00208a0 msgr2=0x55ece0035a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:34.345 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:34.343+0000 7fccd0bcc740 1 --2- 172.21.15.45:0/3602901844 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ece00208a0 0x55ece0035a50 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fccb802c0b0 tx=0x7fccb802bb70 comp rx=0 tx=0).stop 2024-04-16T17:13:34.345 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:34.343+0000 7fccd0bcc740 1 -- 172.21.15.45:0/3602901844 shutdown_connections 2024-04-16T17:13:34.345 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:34.343+0000 7fccd0bcc740 1 --2- 172.21.15.45:0/3602901844 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ece0021610 0x55ece0027530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:34.345 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:34.343+0000 7fccd0bcc740 1 --2- 172.21.15.45:0/3602901844 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ece0029e20 0x55ece0028ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:34.345 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:34.343+0000 7fccd0bcc740 1 --2- 172.21.15.45:0/3602901844 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ece00208a0 0x55ece0035a50 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:34.346 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:34.343+0000 7fccd0bcc740 1 -- 172.21.15.45:0/3602901844 >> 172.21.15.45:0/3602901844 conn(0x55ecdffbb5d0 msgr2=0x55ece00283b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:13:34.346 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:34.343+0000 7fccd0bcc740 1 -- 172.21.15.45:0/3602901844 shutdown_connections 2024-04-16T17:13:34.346 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:34.343+0000 7fccd0bcc740 1 -- 172.21.15.45:0/3602901844 wait complete. 2024-04-16T17:13:34.346 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:34.344+0000 7fccd0bcc740 1 -- start start 2024-04-16T17:13:34.346 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:34.344+0000 7fccd0bcc740 1 Processor -- start 2024-04-16T17:13:34.346 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:34.344+0000 7fccd0bcc740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ece00208a0 0x55ece002d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:34.346 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:34.344+0000 7fccd0bcc740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ece0021610 0x55ece002ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:34.346 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:34.344+0000 7fccd0bcc740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ece0029e20 0x55ece002e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:34.346 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:34.344+0000 7fccd0bcc740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fccc81afdd0 con 0x55ece0029e20 2024-04-16T17:13:34.346 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:34.344+0000 7fccd0bcc740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fccc81b1240 con 0x55ece0021610 2024-04-16T17:13:34.346 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:34.344+0000 7fccd0bcc740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fccc81b04a0 con 0x55ece00208a0 2024-04-16T17:13:34.346 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:34.344+0000 7fccd0bcc740 10 client.0 _ll_register_callbacks cb 0x55ece0029d70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:13:34.348 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:34.345+0000 7fccc7400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ece0029e20 0x55ece002e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:34.348 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:34.345+0000 7fccc6a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ece00208a0 0x55ece002d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:34.348 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:34.345+0000 7fccc7400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ece0029e20 0x55ece002e330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56680/0 (socket says 172.21.15.45:56680) 2024-04-16T17:13:34.348 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:34.345+0000 7fccc7400640 1 -- 172.21.15.45:0/3399497082 learned_addr learned my addr 172.21.15.45:0/3399497082 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:13:34.348 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:34.345+0000 7fccc6000640 1 --2- 172.21.15.45:0/3399497082 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ece0021610 0x55ece002ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:34.348 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:34.345+0000 7fccc6a00640 1 -- 172.21.15.45:0/3399497082 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ece0021610 msgr2=0x55ece002ddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:13:34.348 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:34.345+0000 7fccc6a00640 1 --2- 172.21.15.45:0/3399497082 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ece0021610 0x55ece002ddf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:34.348 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:34.345+0000 7fccc6a00640 1 -- 172.21.15.45:0/3399497082 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ece0029e20 msgr2=0x55ece002e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:13:34.349 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:34.345+0000 7fccc6a00640 1 --2- 172.21.15.45:0/3399497082 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ece0029e20 0x55ece002e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:34.349 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:34.345+0000 7fccc6a00640 1 -- 172.21.15.45:0/3399497082 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ece009f6c0 con 0x55ece00208a0 2024-04-16T17:13:34.349 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:34.346+0000 7fccc6a00640 1 --2- 172.21.15.45:0/3399497082 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ece00208a0 0x55ece002d8b0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fccb8005e00 tx=0x7fccb80349c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=65ee45fa64b0d910 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:13:34.349 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:34.346+0000 7fccb0c00640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-04-16T17:13:34.349 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:34.346+0000 7fccb0c00640 1 -- 172.21.15.45:0/3399497082 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccb8061070 con 0x55ece00208a0 2024-04-16T17:13:34.349 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:34.346+0000 7fccb0c00640 1 -- 172.21.15.45:0/3399497082 <== mon.1 v2:172.21.15.117:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9496+0+0 (secure 0 0 0) 0x7fccb802c510 con 0x55ece00208a0 2024-04-16T17:13:34.349 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:34.346+0000 7fccd0bcc740 1 -- 172.21.15.45:0/3399497082 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fccb804f070 con 0x55ece00208a0 2024-04-16T17:13:34.349 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:34.346+0000 7fccb0c00640 1 -- 172.21.15.45:0/3399497082 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fccb8043040 con 0x55ece00208a0 2024-04-16T17:13:34.349 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:34.346+0000 7fccd0bcc740 10 client.27406 fetch_fsmap learned FSMap version 38 2024-04-16T17:13:34.349 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:34.346+0000 7fccd0bcc740 1 -- 172.21.15.45:0/3399497082 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55ece00a08f0 con 0x55ece00208a0 2024-04-16T17:13:34.349 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:34.347+0000 7fccb0c00640 1 -- 172.21.15.45:0/3399497082 <== mon.1 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fccb804f070 con 0x55ece00208a0 2024-04-16T17:13:34.349 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:34.347+0000 7fccb0c00640 1 -- 172.21.15.45:0/3399497082 <== mon.1 v2:172.21.15.117:3300/0 5 ==== fsmap(e 38) ==== 8354+0+0 (secure 0 0 0) 0x7fccb8032d00 con 0x55ece00208a0 2024-04-16T17:13:34.349 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:34.347+0000 7fccd0bcc740 10 client.27406 fetch_fsmap finished waiting for FSMap version 38 2024-04-16T17:13:34.349 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:34.347+0000 7fccd0bcc740 10 client.27406 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T17:13:34.349 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:34.347+0000 7fccd0bcc740 20 client.27406 populate_metadata read hostname 'smithi045' 2024-04-16T17:13:34.350 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:34.347+0000 7fccd0bcc740 1 --2- 172.21.15.45:0/3399497082 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55ece00a0cb0 0x55ece00a30d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:34.350 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:34.347+0000 7fccd0bcc740 4 client.27406 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-16T17:13:34.350 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:34.347+0000 7fccd0bcc740 1 -- 172.21.15.45:0/3399497082 --> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ecdff8d010 con 0x55ece00a0cb0 2024-04-16T17:13:34.350 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:34.348+0000 7fccc6000640 1 --2- 172.21.15.45:0/3399497082 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55ece00a0cb0 0x55ece00a30d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:34.350 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:34.348+0000 7fccc6000640 1 --2- 172.21.15.45:0/3399497082 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55ece00a0cb0 0x55ece00a30d0 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4a6d85bc8f1970ee server_cookie=2622c2184f5f0d5a in_seq=0 out_seq=0 2024-04-16T17:13:34.350 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:34.348+0000 7fccb0c00640 10 client.27406 ms_handle_connect on v2:172.21.15.104:6838/3660070592 2024-04-16T17:13:34.407 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:34.406+0000 7fccb0c00640 1 -- 172.21.15.45:0/3399497082 <== mds.0 v2:172.21.15.104:6838/3660070592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ecdff8d010 con 0x55ece00a0cb0 2024-04-16T17:13:34.407 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:34.406+0000 7fccb0c00640 10 client.27406 handle_command_reply: tid=0 2024-04-16T17:13:34.407 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:34.406+0000 7fccd0bcc740 1 client.27406 shutdown 2024-04-16T17:13:34.407 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:34.407+0000 7fccd0bcc740 1 -- 172.21.15.45:0/3399497082 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ece00208a0 msgr2=0x55ece002d8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:34.408 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:34.407+0000 7fccd0bcc740 1 --2- 172.21.15.45:0/3399497082 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ece00208a0 0x55ece002d8b0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fccb8005e00 tx=0x7fccb80349c0 comp rx=0 tx=0).stop 2024-04-16T17:13:34.408 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:34.407+0000 7fccd0bcc740 1 -- 172.21.15.45:0/3399497082 shutdown_connections 2024-04-16T17:13:34.408 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:34.407+0000 7fccd0bcc740 1 --2- 172.21.15.45:0/3399497082 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ece0029e20 0x55ece002e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:34.408 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:34.407+0000 7fccd0bcc740 1 --2- 172.21.15.45:0/3399497082 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ece0021610 0x55ece002ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:34.408 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:34.407+0000 7fccd0bcc740 1 --2- 172.21.15.45:0/3399497082 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55ece00a0cb0 0x55ece00a30d0 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:34.408 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:34.407+0000 7fccd0bcc740 1 --2- 172.21.15.45:0/3399497082 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ece00208a0 0x55ece002d8b0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:34.408 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:34.407+0000 7fccb0c00640 0 client.27406 ms_handle_reset on v2:172.21.15.104:6838/3660070592 2024-04-16T17:13:34.408 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:34.407+0000 7fccd0bcc740 1 -- 172.21.15.45:0/3399497082 >> 172.21.15.45:0/3399497082 conn(0x55ecdffbb5d0 msgr2=0x55ece0028340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:13:34.408 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:34.407+0000 7fccd0bcc740 1 -- 172.21.15.45:0/3399497082 shutdown_connections 2024-04-16T17:13:34.409 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:34.407+0000 7fccd0bcc740 1 -- 172.21.15.45:0/3399497082 wait complete. 2024-04-16T17:13:34.409 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:34.407+0000 7fccd0bcc740 20 client.27406 trim_cache size 0 max 16384 2024-04-16T17:13:34.409 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-04-16T17:13:34.409 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-04-16T17:13:34.409 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-04-16T17:13:34.409 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-04-16T17:13:34.410 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:34.408+0000 7fcccde00640 1 -- 172.21.15.45:0/589809298 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fccac0a3e40 msgr2=0x7fccac0a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:34.410 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:34.408+0000 7fcccde00640 1 --2- 172.21.15.45:0/589809298 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fccac0a3e40 0x7fccac0a6300 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fccb8009830 tx=0x7fccb80056d0 comp rx=0 tx=0).stop 2024-04-16T17:13:34.410 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:34.408+0000 7fcccde00640 1 -- 172.21.15.45:0/589809298 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fccc8109520 msgr2=0x7fccc81a0c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:34.410 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:34.408+0000 7fcccde00640 1 --2- 172.21.15.45:0/589809298 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fccc8109520 0x7fccc81a0c80 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fccb400e9a0 tx=0x7fccb400ee60 comp rx=0 tx=0).stop 2024-04-16T17:13:34.410 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:34.408+0000 7fcccde00640 1 -- 172.21.15.45:0/589809298 shutdown_connections 2024-04-16T17:13:34.410 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:34.408+0000 7fcccde00640 1 --2- 172.21.15.45:0/589809298 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fccac0a3e40 0x7fccac0a6300 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:34.410 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:34.408+0000 7fcccde00640 1 --2- 172.21.15.45:0/589809298 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fccc8109520 0x7fccc81a0c80 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:34.410 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:34.408+0000 7fcccde00640 1 --2- 172.21.15.45:0/589809298 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fccc819a8f0 0x7fccc819ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:34.410 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:34.408+0000 7fcccde00640 1 --2- 172.21.15.45:0/589809298 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fccc8104450 0x7fccc81a0740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:34.411 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:34.408+0000 7fcccde00640 1 -- 172.21.15.45:0/589809298 >> 172.21.15.45:0/589809298 conn(0x7fccc80ffcf0 msgr2=0x7fccc8100920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:13:34.411 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:34.408+0000 7fcccde00640 1 -- 172.21.15.45:0/589809298 shutdown_connections 2024-04-16T17:13:34.411 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:34.408+0000 7fcccde00640 1 -- 172.21.15.45:0/589809298 wait complete. 2024-04-16T17:13:34.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.535+0000 7fa2b7200640 1 -- 172.21.15.45:0/3459318323 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa2b8068d40 con 0x7fa2b80743d0 2024-04-16T17:13:34.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.536+0000 7fa2aea00640 1 -- 172.21.15.45:0/3459318323 <== mon.2 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) ==== 94+0+11890 (secure 0 0 0) 0x7fa2a809cde0 con 0x7fa2b80743d0 2024-04-16T17:13:34.537 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-16T17:13:34.537 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":38,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/1748842443","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":1748842443},{"type":"v1","addr":"172.21.15.45:6835","nonce":1748842443}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6839/1308627051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":1308627051},{"type":"v1","addr":"172.21.15.45:6839","nonce":1308627051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/1768242489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1768242489},{"type":"v1","addr":"172.21.15.117:6833","nonce":1768242489}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6833/3558902585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":3558902585},{"type":"v1","addr":"172.21.15.104:6833","nonce":3558902585}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24478,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/1308142903","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":1308142903},{"type":"v1","addr":"172.21.15.45:6837","nonce":1308142903}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24491,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6835/3430679937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":3430679937},{"type":"v1","addr":"172.21.15.104:6835","nonce":3430679937}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":38,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-16T17:03:24.663611+0000","modified":"2024-04-16T17:13:17.260913+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24581,"mds_1":24569,"mds_2":24532},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24499":{"gid":24499,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/2224123841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2224123841},{"type":"v1","addr":"172.21.15.117:6835","nonce":2224123841}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6837/2471356815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":2471356815},{"type":"v1","addr":"172.21.15.117:6837","nonce":2471356815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24524":{"gid":24524,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6837/2647756509","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":2647756509},{"type":"v1","addr":"172.21.15.104:6837","nonce":2647756509}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24532":{"gid":24532,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":13,"addr":"172.21.15.117:6839/2378675949","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":2378675949},{"type":"v1","addr":"172.21.15.117:6839","nonce":2378675949}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24569":{"gid":24569,"name":"a","rank":1,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.45:6841/3755310822","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6840","nonce":3755310822},{"type":"v1","addr":"172.21.15.45:6841","nonce":3755310822}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24581":{"gid":24581,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.104:6839/3660070592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6838","nonce":3660070592},{"type":"v1","addr":"172.21.15.104:6839","nonce":3660070592}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24581,"qdb_cluster":[24569,24532,24581]},"id":1}]} 2024-04-16T17:13:34.537 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 38 2024-04-16T17:13:34.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.539+0000 7fa2b7200640 1 -- 172.21.15.45:0/3459318323 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fa29c0a3da0 msgr2=0x7fa29c0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:34.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.539+0000 7fa2b7200640 1 --2- 172.21.15.45:0/3459318323 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fa29c0a3da0 0x7fa29c0a6260 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fa2b819bb00 tx=0x7fa2a402df90 comp rx=0 tx=0).stop 2024-04-16T17:13:34.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.539+0000 7fa2b7200640 1 -- 172.21.15.45:0/3459318323 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa2b80743d0 msgr2=0x7fa2b81a0640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:34.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.539+0000 7fa2b7200640 1 --2- 172.21.15.45:0/3459318323 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa2b80743d0 0x7fa2b81a0640 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fa2a800e9f0 tx=0x7fa2a800eeb0 comp rx=0 tx=0).stop 2024-04-16T17:13:34.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.539+0000 7fa2b7200640 1 -- 172.21.15.45:0/3459318323 shutdown_connections 2024-04-16T17:13:34.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.539+0000 7fa2b7200640 1 --2- 172.21.15.45:0/3459318323 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fa29c0a3da0 0x7fa29c0a6260 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:34.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.539+0000 7fa2b7200640 1 --2- 172.21.15.45:0/3459318323 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa2b810e6c0 0x7fa2b819a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:34.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.539+0000 7fa2b7200640 1 --2- 172.21.15.45:0/3459318323 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa2b8074cf0 0x7fa2b81a0b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:34.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.539+0000 7fa2b7200640 1 --2- 172.21.15.45:0/3459318323 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa2b80743d0 0x7fa2b81a0640 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:34.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.539+0000 7fa2b7200640 1 -- 172.21.15.45:0/3459318323 >> 172.21.15.45:0/3459318323 conn(0x7fa2b80fd930 msgr2=0x7fa2b80fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:13:34.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.539+0000 7fa2b7200640 1 -- 172.21.15.45:0/3459318323 shutdown_connections 2024-04-16T17:13:34.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.539+0000 7fa2b7200640 1 -- 172.21.15.45:0/3459318323 wait complete. 2024-04-16T17:13:34.556 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-16T17:13:34.768 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.767+0000 7f8e4fe00640 1 -- 172.21.15.45:0/1554026115 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8e501071e0 msgr2=0x7f8e5010edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:34.768 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.767+0000 7f8e4fe00640 1 --2- 172.21.15.45:0/1554026115 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8e501071e0 0x7f8e5010edb0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f8e48009ca0 tx=0x7f8e4802c100 comp rx=0 tx=0).stop 2024-04-16T17:13:34.768 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.767+0000 7f8e4fe00640 1 -- 172.21.15.45:0/1554026115 shutdown_connections 2024-04-16T17:13:34.768 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.767+0000 7f8e4fe00640 1 --2- 172.21.15.45:0/1554026115 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8e501071e0 0x7f8e5010edb0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:34.768 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.767+0000 7f8e4fe00640 1 --2- 172.21.15.45:0/1554026115 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8e50102ac0 0x7f8e50106b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:34.769 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.767+0000 7f8e4fe00640 1 --2- 172.21.15.45:0/1554026115 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8e50102110 0x7f8e501024f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:34.769 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.767+0000 7f8e4fe00640 1 -- 172.21.15.45:0/1554026115 >> 172.21.15.45:0/1554026115 conn(0x7f8e500fd9d0 msgr2=0x7f8e500ffdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:13:34.769 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.767+0000 7f8e4fe00640 1 -- 172.21.15.45:0/1554026115 shutdown_connections 2024-04-16T17:13:34.769 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.767+0000 7f8e4fe00640 1 -- 172.21.15.45:0/1554026115 wait complete. 2024-04-16T17:13:34.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.768+0000 7f8e4fe00640 1 Processor -- start 2024-04-16T17:13:34.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.768+0000 7f8e4fe00640 1 -- start start 2024-04-16T17:13:34.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.769+0000 7f8e4fe00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8e50102110 0x7f8e501a0610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:34.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.769+0000 7f8e4fe00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8e501071e0 0x7f8e501a0b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:34.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.769+0000 7f8e4fe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8e5019a780 0x7f8e5019ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:34.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.769+0000 7f8e4fe00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f8e501122a0 con 0x7f8e5019a780 2024-04-16T17:13:34.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.769+0000 7f8e4fe00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f8e50112120 con 0x7f8e50102110 2024-04-16T17:13:34.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.769+0000 7f8e4fe00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f8e50112420 con 0x7f8e501071e0 2024-04-16T17:13:34.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.769+0000 7f8e4ea00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8e50102110 0x7f8e501a0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:34.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.769+0000 7f8e4ea00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8e50102110 0x7f8e501a0610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:39464/0 (socket says 172.21.15.45:39464) 2024-04-16T17:13:34.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.769+0000 7f8e4ea00640 1 -- 172.21.15.45:0/74804379 learned_addr learned my addr 172.21.15.45:0/74804379 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:13:34.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.769+0000 7f8e4e000640 1 --2- 172.21.15.45:0/74804379 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8e501071e0 0x7f8e501a0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:34.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.769+0000 7f8e4f400640 1 --2- 172.21.15.45:0/74804379 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8e5019a780 0x7f8e5019ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:34.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.770+0000 7f8e4ea00640 1 -- 172.21.15.45:0/74804379 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8e501071e0 msgr2=0x7f8e501a0b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:34.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.770+0000 7f8e4ea00640 1 --2- 172.21.15.45:0/74804379 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8e501071e0 0x7f8e501a0b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:34.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.770+0000 7f8e4ea00640 1 -- 172.21.15.45:0/74804379 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8e5019a780 msgr2=0x7f8e5019ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:34.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.770+0000 7f8e4ea00640 1 --2- 172.21.15.45:0/74804379 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8e5019a780 0x7f8e5019ac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:34.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.770+0000 7f8e4ea00640 1 -- 172.21.15.45:0/74804379 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e5019b310 con 0x7f8e50102110 2024-04-16T17:13:34.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.770+0000 7f8e4f400640 1 --2- 172.21.15.45:0/74804379 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8e5019a780 0x7f8e5019ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:13:34.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.770+0000 7f8e4e000640 1 --2- 172.21.15.45:0/74804379 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8e501071e0 0x7f8e501a0b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:13:34.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.770+0000 7f8e4ea00640 1 --2- 172.21.15.45:0/74804379 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8e50102110 0x7f8e501a0610 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f8e4000e9f0 tx=0x7f8e4000edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:13:34.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.770+0000 7f8e47400640 1 -- 172.21.15.45:0/74804379 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e4000cd80 con 0x7f8e50102110 2024-04-16T17:13:34.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.770+0000 7f8e4fe00640 1 -- 172.21.15.45:0/74804379 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e5019b600 con 0x7f8e50102110 2024-04-16T17:13:34.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.770+0000 7f8e47400640 1 -- 172.21.15.45:0/74804379 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8e40005500 con 0x7f8e50102110 2024-04-16T17:13:34.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.770+0000 7f8e47400640 1 -- 172.21.15.45:0/74804379 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e40010690 con 0x7f8e50102110 2024-04-16T17:13:34.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.771+0000 7f8e4fe00640 1 -- 172.21.15.45:0/74804379 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e501a77e0 con 0x7f8e50102110 2024-04-16T17:13:34.775 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.774+0000 7f8e5856e740 1 Processor -- start 2024-04-16T17:13:34.775 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.775+0000 7f8e5856e740 1 -- start start 2024-04-16T17:13:34.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.775+0000 7f8e5856e740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b2ae3fdf10 0x55b2ae3fe2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:34.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.775+0000 7f8e5856e740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b2ae3fe830 0x55b2ae406ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:34.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.775+0000 7f8e5856e740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b2ae407410 0x55b2ae409800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:34.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.775+0000 7f8e5856e740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55b2ae363020 con 0x55b2ae3fdf10 2024-04-16T17:13:34.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.775+0000 7f8e5856e740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55b2ae309460 con 0x55b2ae3fe830 2024-04-16T17:13:34.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.775+0000 7f8e5856e740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55b2ae35efe0 con 0x55b2ae407410 2024-04-16T17:13:34.777 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.776+0000 7f8e4e000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b2ae407410 0x55b2ae409800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:34.777 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.776+0000 7f8e4e000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b2ae407410 0x55b2ae409800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:57206/0 (socket says 172.21.15.45:57206) 2024-04-16T17:13:34.777 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.776+0000 7f8e4e000640 1 -- 172.21.15.45:0/2321537921 learned_addr learned my addr 172.21.15.45:0/2321537921 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:13:34.777 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.776+0000 7f8e47400640 1 -- 172.21.15.45:0/74804379 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f8e400046c0 con 0x7f8e50102110 2024-04-16T17:13:34.777 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.776+0000 7f8e4f400640 1 --2- 172.21.15.45:0/2321537921 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b2ae3fdf10 0x55b2ae3fe2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:34.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.776+0000 7f8e4e000640 1 -- 172.21.15.45:0/2321537921 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b2ae3fe830 msgr2=0x55b2ae406ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:13:34.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.776+0000 7f8e4ea00640 1 --2- 172.21.15.45:0/2321537921 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b2ae3fe830 0x55b2ae406ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:34.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.776+0000 7f8e4e000640 1 --2- 172.21.15.45:0/2321537921 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b2ae3fe830 0x55b2ae406ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:34.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.776+0000 7f8e4e000640 1 -- 172.21.15.45:0/2321537921 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b2ae3fdf10 msgr2=0x55b2ae3fe2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:13:34.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.776+0000 7f8e4e000640 1 --2- 172.21.15.45:0/2321537921 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b2ae3fdf10 0x55b2ae3fe2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:34.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.776+0000 7f8e4e000640 1 -- 172.21.15.45:0/2321537921 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b2ae311040 con 0x55b2ae407410 2024-04-16T17:13:34.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.776+0000 7f8e4f400640 1 --2- 172.21.15.45:0/2321537921 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b2ae3fdf10 0x55b2ae3fe2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:13:34.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.776+0000 7f8e4ea00640 1 --2- 172.21.15.45:0/2321537921 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b2ae3fe830 0x55b2ae406ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:13:34.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.777+0000 7f8e4e000640 1 --2- 172.21.15.45:0/2321537921 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b2ae407410 0x55b2ae409800 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f8e3c009fd0 tx=0x7f8e3c0118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=daeefc8494507e82 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:13:34.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.777+0000 7f8e3be00640 1 -- 172.21.15.45:0/2321537921 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e3c01e070 con 0x55b2ae407410 2024-04-16T17:13:34.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.777+0000 7f8e3be00640 1 -- 172.21.15.45:0/2321537921 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8e3c0035f0 con 0x55b2ae407410 2024-04-16T17:13:34.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.777+0000 7f8e3be00640 1 -- 172.21.15.45:0/2321537921 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e3c00f740 con 0x55b2ae407410 2024-04-16T17:13:34.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.778+0000 7f8e5856e740 1 -- 172.21.15.45:0/2321537921 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b2ae407410 msgr2=0x55b2ae409800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:34.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.778+0000 7f8e5856e740 1 --2- 172.21.15.45:0/2321537921 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b2ae407410 0x55b2ae409800 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f8e3c009fd0 tx=0x7f8e3c0118d0 comp rx=0 tx=0).stop 2024-04-16T17:13:34.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.778+0000 7f8e5856e740 1 -- 172.21.15.45:0/2321537921 shutdown_connections 2024-04-16T17:13:34.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.778+0000 7f8e5856e740 1 --2- 172.21.15.45:0/2321537921 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b2ae407410 0x55b2ae409800 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:34.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.778+0000 7f8e5856e740 1 --2- 172.21.15.45:0/2321537921 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b2ae3fe830 0x55b2ae406ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:34.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.778+0000 7f8e5856e740 1 --2- 172.21.15.45:0/2321537921 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b2ae3fdf10 0x55b2ae3fe2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:34.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.778+0000 7f8e5856e740 1 -- 172.21.15.45:0/2321537921 >> 172.21.15.45:0/2321537921 conn(0x55b2ae3f8640 msgr2=0x55b2ae3f8a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:13:34.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.778+0000 7f8e47400640 1 --2- 172.21.15.45:0/74804379 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f8e340a3da0 0x7f8e340a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:34.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.778+0000 7f8e5856e740 1 -- 172.21.15.45:0/2321537921 shutdown_connections 2024-04-16T17:13:34.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.778+0000 7f8e47400640 1 -- 172.21.15.45:0/74804379 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9496+0+0 (secure 0 0 0) 0x7f8e400d1070 con 0x7f8e50102110 2024-04-16T17:13:34.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.778+0000 7f8e4e000640 1 --2- 172.21.15.45:0/74804379 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f8e340a3da0 0x7f8e340a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:34.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.778+0000 7f8e5856e740 1 -- 172.21.15.45:0/2321537921 wait complete. 2024-04-16T17:13:34.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.779+0000 7f8e5856e740 1 -- start start 2024-04-16T17:13:34.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.779+0000 7f8e4e000640 1 --2- 172.21.15.45:0/74804379 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f8e340a3da0 0x7f8e340a6260 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f8e3c01c040 tx=0x7f8e3c0096c0 comp rx=0 tx=0).ready entity=mgr.25411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:13:34.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.779+0000 7f8e5856e740 1 Processor -- start 2024-04-16T17:13:34.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.780+0000 7f8e5856e740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b2ae3fdf10 0x55b2ae4043f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:34.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.780+0000 7f8e5856e740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b2ae3fe830 0x55b2ae404930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:34.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.780+0000 7f8e5856e740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b2ae407410 0x55b2ae404e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:34.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.780+0000 7f8e5856e740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55b2ae40a200 con 0x55b2ae3fe830 2024-04-16T17:13:34.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.780+0000 7f8e5856e740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55b2ae31c830 con 0x55b2ae3fdf10 2024-04-16T17:13:34.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.780+0000 7f8e5856e740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55b2ae40a380 con 0x55b2ae407410 2024-04-16T17:13:34.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.780+0000 7f8e5856e740 10 client.0 _ll_register_callbacks cb 0x55b2ae324150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:13:34.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.780+0000 7f8e4ea00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b2ae3fdf10 0x55b2ae4043f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:34.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.780+0000 7f8e4ea00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b2ae3fdf10 0x55b2ae4043f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:39494/0 (socket says 172.21.15.45:39494) 2024-04-16T17:13:34.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.780+0000 7f8e4ea00640 1 -- 172.21.15.45:0/2406167477 learned_addr learned my addr 172.21.15.45:0/2406167477 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:13:34.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.780+0000 7f8e4f400640 1 --2- 172.21.15.45:0/2406167477 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b2ae407410 0x55b2ae404e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:34.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.780+0000 7f8e4e000640 1 --2- 172.21.15.45:0/2406167477 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b2ae3fe830 0x55b2ae404930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:34.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.781+0000 7f8e4ea00640 1 -- 172.21.15.45:0/2406167477 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b2ae407410 msgr2=0x55b2ae404e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:13:34.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.781+0000 7f8e4ea00640 1 --2- 172.21.15.45:0/2406167477 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b2ae407410 0x55b2ae404e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:34.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.781+0000 7f8e4ea00640 1 -- 172.21.15.45:0/2406167477 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b2ae3fe830 msgr2=0x55b2ae404930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:13:34.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.781+0000 7f8e4ea00640 1 --2- 172.21.15.45:0/2406167477 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b2ae3fe830 0x55b2ae404930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:34.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.781+0000 7f8e4ea00640 1 -- 172.21.15.45:0/2406167477 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b2ae320920 con 0x55b2ae3fdf10 2024-04-16T17:13:34.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.781+0000 7f8e4f400640 1 --2- 172.21.15.45:0/2406167477 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b2ae407410 0x55b2ae404e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:13:34.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.781+0000 7f8e4e000640 1 --2- 172.21.15.45:0/2406167477 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b2ae3fe830 0x55b2ae404930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:13:34.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.781+0000 7f8e4ea00640 1 --2- 172.21.15.45:0/2406167477 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b2ae3fdf10 0x55b2ae4043f0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f8e4000e9c0 tx=0x7f8e4009b0f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e3f35705e9e2b40b server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:13:34.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.781+0000 7f8e38c00640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-04-16T17:13:34.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.781+0000 7f8e38c00640 1 -- 172.21.15.45:0/2406167477 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e400a0f90 con 0x55b2ae3fdf10 2024-04-16T17:13:34.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.781+0000 7f8e5856e740 1 -- 172.21.15.45:0/2406167477 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8e3c0035f0 con 0x55b2ae3fdf10 2024-04-16T17:13:34.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.781+0000 7f8e38c00640 1 -- 172.21.15.45:0/2406167477 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9496+0+0 (secure 0 0 0) 0x7f8e400ce610 con 0x55b2ae3fdf10 2024-04-16T17:13:34.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.781+0000 7f8e38c00640 1 -- 172.21.15.45:0/2406167477 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f8e400ce810 con 0x55b2ae3fdf10 2024-04-16T17:13:34.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.782+0000 7f8e5856e740 10 client.27470 fetch_fsmap learned FSMap version 38 2024-04-16T17:13:34.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.782+0000 7f8e5856e740 1 -- 172.21.15.45:0/2406167477 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55b2ae2ee7f0 con 0x55b2ae3fdf10 2024-04-16T17:13:34.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.782+0000 7f8e38c00640 1 -- 172.21.15.45:0/2406167477 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8e3c0035f0 con 0x55b2ae3fdf10 2024-04-16T17:13:34.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.782+0000 7f8e38c00640 1 -- 172.21.15.45:0/2406167477 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 38) ==== 8354+0+0 (secure 0 0 0) 0x7f8e4009c030 con 0x55b2ae3fdf10 2024-04-16T17:13:34.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.782+0000 7f8e5856e740 10 client.27470 fetch_fsmap finished waiting for FSMap version 38 2024-04-16T17:13:34.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.782+0000 7f8e5856e740 10 client.27470 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-16T17:13:34.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.782+0000 7f8e5856e740 20 client.27470 populate_metadata read hostname 'smithi045' 2024-04-16T17:13:34.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.782+0000 7f8e5856e740 1 --2- 172.21.15.45:0/2406167477 >> [v2:172.21.15.117:6838/2378675949,v1:172.21.15.117:6839/2378675949] conn(0x55b2ae41fe90 0x55b2ae422320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:34.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.782+0000 7f8e5856e740 4 client.27470 mds_command: new command op to 24532 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:13:34.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.782+0000 7f8e5856e740 1 -- 172.21.15.45:0/2406167477 --> [v2:172.21.15.117:6838/2378675949,v1:172.21.15.117:6839/2378675949] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b2ae370a40 con 0x55b2ae41fe90 2024-04-16T17:13:34.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.783+0000 7f8e4e000640 1 --2- 172.21.15.45:0/2406167477 >> [v2:172.21.15.117:6838/2378675949,v1:172.21.15.117:6839/2378675949] conn(0x55b2ae41fe90 0x55b2ae422320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:34.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.783+0000 7f8e4e000640 1 --2- 172.21.15.45:0/2406167477 >> [v2:172.21.15.117:6838/2378675949,v1:172.21.15.117:6839/2378675949] conn(0x55b2ae41fe90 0x55b2ae422320 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=52542fb160bef8db server_cookie=c3d1941fd96448f4 in_seq=0 out_seq=0 2024-04-16T17:13:34.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.783+0000 7f8e38c00640 10 client.27470 ms_handle_connect on v2:172.21.15.117:6838/2378675949 2024-04-16T17:13:34.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.784+0000 7f8e38c00640 1 -- 172.21.15.45:0/2406167477 <== mds.2 v2:172.21.15.117:6838/2378675949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b2ae370a40 con 0x55b2ae41fe90 2024-04-16T17:13:34.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.784+0000 7f8e38c00640 10 client.27470 handle_command_reply: tid=0 2024-04-16T17:13:34.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.784+0000 7f8e5856e740 1 client.27470 shutdown 2024-04-16T17:13:34.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.784+0000 7f8e5856e740 1 -- 172.21.15.45:0/2406167477 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b2ae3fdf10 msgr2=0x55b2ae4043f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:34.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.784+0000 7f8e5856e740 1 --2- 172.21.15.45:0/2406167477 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b2ae3fdf10 0x55b2ae4043f0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f8e4000e9c0 tx=0x7f8e4009b0f0 comp rx=0 tx=0).stop 2024-04-16T17:13:34.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.784+0000 7f8e5856e740 1 -- 172.21.15.45:0/2406167477 shutdown_connections 2024-04-16T17:13:34.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.784+0000 7f8e5856e740 1 --2- 172.21.15.45:0/2406167477 >> [v2:172.21.15.117:6838/2378675949,v1:172.21.15.117:6839/2378675949] conn(0x55b2ae41fe90 0x55b2ae422320 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:34.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.784+0000 7f8e5856e740 1 --2- 172.21.15.45:0/2406167477 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b2ae407410 0x55b2ae404e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:34.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.784+0000 7f8e5856e740 1 --2- 172.21.15.45:0/2406167477 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b2ae3fe830 0x55b2ae404930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:34.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.784+0000 7f8e5856e740 1 --2- 172.21.15.45:0/2406167477 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b2ae3fdf10 0x55b2ae4043f0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:34.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.784+0000 7f8e38c00640 0 client.27470 ms_handle_reset on v2:172.21.15.117:6838/2378675949 2024-04-16T17:13:34.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.784+0000 7f8e5856e740 1 -- 172.21.15.45:0/2406167477 >> 172.21.15.45:0/2406167477 conn(0x55b2ae3f8640 msgr2=0x55b2ae3fa390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:13:34.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.785+0000 7f8e5856e740 1 -- 172.21.15.45:0/2406167477 shutdown_connections 2024-04-16T17:13:34.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.785+0000 7f8e5856e740 1 -- 172.21.15.45:0/2406167477 wait complete. 2024-04-16T17:13:34.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.785+0000 7f8e5856e740 20 client.27470 trim_cache size 0 max 16384 2024-04-16T17:13:34.804 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:13:34 smithi117 ceph-mon[36730]: pgmap v209: 129 pgs: 129 active+clean; 59 MiB data, 735 MiB used, 1.0 TiB / 1.0 TiB avail; 782 KiB/s rd, 584 KiB/s wr, 63 op/s 2024-04-16T17:13:34.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.816+0000 7f8e5856e740 1 Processor -- start 2024-04-16T17:13:34.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.816+0000 7f8e5856e740 1 -- start start 2024-04-16T17:13:34.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.816+0000 7f8e5856e740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b2ae3fdf10 0x55b2ae406b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:34.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.816+0000 7f8e5856e740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b2ae3fe830 0x55b2ae41b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:34.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.816+0000 7f8e5856e740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b2ae407410 0x55b2ae427c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:34.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.816+0000 7f8e5856e740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f8e501afaf0 con 0x55b2ae3fdf10 2024-04-16T17:13:34.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.816+0000 7f8e5856e740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f8e501ad360 con 0x55b2ae3fe830 2024-04-16T17:13:34.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.816+0000 7f8e5856e740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f8e501ab210 con 0x55b2ae407410 2024-04-16T17:13:34.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.817+0000 7f8e4e000640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b2ae3fe830 0x55b2ae41b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:34.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.817+0000 7f8e4ea00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b2ae3fdf10 0x55b2ae406b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:34.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.817+0000 7f8e4e000640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b2ae3fe830 0x55b2ae41b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:39510/0 (socket says 172.21.15.45:39510) 2024-04-16T17:13:34.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.817+0000 7f8e4f400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b2ae407410 0x55b2ae427c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:34.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.817+0000 7f8e4ea00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b2ae3fdf10 0x55b2ae406b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56716/0 (socket says 172.21.15.45:56716) 2024-04-16T17:13:34.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.817+0000 7f8e4e000640 1 -- 172.21.15.45:0/2116887587 learned_addr learned my addr 172.21.15.45:0/2116887587 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:13:34.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.817+0000 7f8e4ea00640 1 -- 172.21.15.45:0/2116887587 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b2ae407410 msgr2=0x55b2ae427c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:13:34.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.817+0000 7f8e4ea00640 1 --2- 172.21.15.45:0/2116887587 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b2ae407410 0x55b2ae427c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:34.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.817+0000 7f8e4ea00640 1 -- 172.21.15.45:0/2116887587 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b2ae3fe830 msgr2=0x55b2ae41b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:13:34.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.817+0000 7f8e4ea00640 1 --2- 172.21.15.45:0/2116887587 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b2ae3fe830 0x55b2ae41b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:34.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.817+0000 7f8e4ea00640 1 -- 172.21.15.45:0/2116887587 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b2ae3fe660 con 0x55b2ae3fdf10 2024-04-16T17:13:34.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.817+0000 7f8e4e000640 1 --2- 172.21.15.45:0/2116887587 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b2ae3fe830 0x55b2ae41b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:13:34.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.817+0000 7f8e4ea00640 1 --2- 172.21.15.45:0/2116887587 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b2ae3fdf10 0x55b2ae406b20 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f8e4000c6a0 tx=0x7f8e4009c820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e66cea7a57877df server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:13:34.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.818+0000 7f8e3b400640 1 -- 172.21.15.45:0/2116887587 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e4009c9b0 con 0x55b2ae3fdf10 2024-04-16T17:13:34.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.818+0000 7f8e3b400640 1 -- 172.21.15.45:0/2116887587 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8e40098c60 con 0x55b2ae3fdf10 2024-04-16T17:13:34.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.818+0000 7f8e3b400640 1 -- 172.21.15.45:0/2116887587 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e400a5620 con 0x55b2ae3fdf10 2024-04-16T17:13:34.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.819+0000 7f8e5856e740 1 -- 172.21.15.45:0/2116887587 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b2ae3fdf10 msgr2=0x55b2ae406b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:34.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.819+0000 7f8e5856e740 1 --2- 172.21.15.45:0/2116887587 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b2ae3fdf10 0x55b2ae406b20 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f8e4000c6a0 tx=0x7f8e4009c820 comp rx=0 tx=0).stop 2024-04-16T17:13:34.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.819+0000 7f8e5856e740 1 -- 172.21.15.45:0/2116887587 shutdown_connections 2024-04-16T17:13:34.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.819+0000 7f8e5856e740 1 --2- 172.21.15.45:0/2116887587 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b2ae407410 0x55b2ae427c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:34.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.819+0000 7f8e5856e740 1 --2- 172.21.15.45:0/2116887587 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b2ae3fe830 0x55b2ae41b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:34.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.819+0000 7f8e5856e740 1 --2- 172.21.15.45:0/2116887587 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b2ae3fdf10 0x55b2ae406b20 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:34.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.819+0000 7f8e5856e740 1 -- 172.21.15.45:0/2116887587 >> 172.21.15.45:0/2116887587 conn(0x55b2ae3f8640 msgr2=0x55b2ae3fa300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:13:34.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.819+0000 7f8e5856e740 1 -- 172.21.15.45:0/2116887587 shutdown_connections 2024-04-16T17:13:34.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.819+0000 7f8e5856e740 1 -- 172.21.15.45:0/2116887587 wait complete. 2024-04-16T17:13:34.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.819+0000 7f8e5856e740 1 -- start start 2024-04-16T17:13:34.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.820+0000 7f8e5856e740 1 Processor -- start 2024-04-16T17:13:34.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.820+0000 7f8e5856e740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b2ae3fdf10 0x55b2ae4012f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:34.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.820+0000 7f8e5856e740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b2ae3fe830 0x55b2ae401830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:34.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.820+0000 7f8e5856e740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b2ae407410 0x55b2ae401d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:34.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.820+0000 7f8e5856e740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f8e501ab860 con 0x55b2ae3fe830 2024-04-16T17:13:34.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.820+0000 7f8e5856e740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f8e501acc90 con 0x55b2ae3fdf10 2024-04-16T17:13:34.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.821+0000 7f8e5856e740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f8e501abef0 con 0x55b2ae407410 2024-04-16T17:13:34.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.821+0000 7f8e5856e740 10 client.0 _ll_register_callbacks cb 0x7f8e400056b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:13:34.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.821+0000 7f8e4ea00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b2ae3fdf10 0x55b2ae4012f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:34.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.821+0000 7f8e4ea00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b2ae3fdf10 0x55b2ae4012f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:39518/0 (socket says 172.21.15.45:39518) 2024-04-16T17:13:34.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.821+0000 7f8e4ea00640 1 -- 172.21.15.45:0/1963106905 learned_addr learned my addr 172.21.15.45:0/1963106905 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:13:34.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.821+0000 7f8e4e000640 1 --2- 172.21.15.45:0/1963106905 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b2ae3fe830 0x55b2ae401830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:34.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.821+0000 7f8e4ea00640 1 -- 172.21.15.45:0/1963106905 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b2ae407410 msgr2=0x55b2ae401d70 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-16T17:13:34.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.821+0000 7f8e4ea00640 1 --2- 172.21.15.45:0/1963106905 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b2ae407410 0x55b2ae401d70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:34.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.821+0000 7f8e4ea00640 1 -- 172.21.15.45:0/1963106905 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b2ae3fe830 msgr2=0x55b2ae401830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:13:34.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.821+0000 7f8e4ea00640 1 --2- 172.21.15.45:0/1963106905 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b2ae3fe830 0x55b2ae401830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:34.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.821+0000 7f8e4ea00640 1 -- 172.21.15.45:0/1963106905 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b2ae0753f0 con 0x55b2ae3fdf10 2024-04-16T17:13:34.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.821+0000 7f8e4ea00640 1 --2- 172.21.15.45:0/1963106905 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b2ae3fdf10 0x55b2ae4012f0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f8e4000e9c0 tx=0x7f8e4000cf90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2ded32b627847d04 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:13:34.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.821+0000 7f8e38c00640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-04-16T17:13:34.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.822+0000 7f8e38c00640 1 -- 172.21.15.45:0/1963106905 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e400a9060 con 0x55b2ae3fdf10 2024-04-16T17:13:34.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.822+0000 7f8e38c00640 1 -- 172.21.15.45:0/1963106905 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9496+0+0 (secure 0 0 0) 0x7f8e400a55c0 con 0x55b2ae3fdf10 2024-04-16T17:13:34.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.822+0000 7f8e5856e740 1 -- 172.21.15.45:0/1963106905 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8e40098c60 con 0x55b2ae3fdf10 2024-04-16T17:13:34.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.822+0000 7f8e38c00640 1 -- 172.21.15.45:0/1963106905 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f8e4009c9b0 con 0x55b2ae3fdf10 2024-04-16T17:13:34.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.822+0000 7f8e5856e740 10 client.27476 fetch_fsmap learned FSMap version 38 2024-04-16T17:13:34.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.822+0000 7f8e5856e740 1 -- 172.21.15.45:0/1963106905 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55b2ae13f7d0 con 0x55b2ae3fdf10 2024-04-16T17:13:34.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.822+0000 7f8e38c00640 1 -- 172.21.15.45:0/1963106905 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8e40098c60 con 0x55b2ae3fdf10 2024-04-16T17:13:34.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.823+0000 7f8e38c00640 1 -- 172.21.15.45:0/1963106905 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 38) ==== 8354+0+0 (secure 0 0 0) 0x7f8e400b1190 con 0x55b2ae3fdf10 2024-04-16T17:13:34.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.823+0000 7f8e5856e740 10 client.27476 fetch_fsmap finished waiting for FSMap version 38 2024-04-16T17:13:34.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.823+0000 7f8e5856e740 10 client.27476 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-16T17:13:34.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.823+0000 7f8e5856e740 20 client.27476 populate_metadata read hostname 'smithi045' 2024-04-16T17:13:34.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.823+0000 7f8e5856e740 1 --2- 172.21.15.45:0/1963106905 >> [v2:172.21.15.117:6838/2378675949,v1:172.21.15.117:6839/2378675949] conn(0x55b2ae486ce0 0x55b2ae4890d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:34.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.823+0000 7f8e5856e740 4 client.27476 mds_command: new command op to 24532 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T17:13:34.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.823+0000 7f8e5856e740 1 -- 172.21.15.45:0/1963106905 --> [v2:172.21.15.117:6838/2378675949,v1:172.21.15.117:6839/2378675949] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b2ae36a8b0 con 0x55b2ae486ce0 2024-04-16T17:13:34.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.823+0000 7f8e4e000640 1 --2- 172.21.15.45:0/1963106905 >> [v2:172.21.15.117:6838/2378675949,v1:172.21.15.117:6839/2378675949] conn(0x55b2ae486ce0 0x55b2ae4890d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:34.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.824+0000 7f8e4e000640 1 --2- 172.21.15.45:0/1963106905 >> [v2:172.21.15.117:6838/2378675949,v1:172.21.15.117:6839/2378675949] conn(0x55b2ae486ce0 0x55b2ae4890d0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9d03c9bed3577ad0 server_cookie=363cc61ac6f1cfbd in_seq=0 out_seq=0 2024-04-16T17:13:34.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.824+0000 7f8e38c00640 10 client.27476 ms_handle_connect on v2:172.21.15.117:6838/2378675949 2024-04-16T17:13:34.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.824+0000 7f8e38c00640 1 -- 172.21.15.45:0/1963106905 <== mds.2 v2:172.21.15.117:6838/2378675949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b2ae36a8b0 con 0x55b2ae486ce0 2024-04-16T17:13:34.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.824+0000 7f8e38c00640 10 client.27476 handle_command_reply: tid=0 2024-04-16T17:13:34.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.824+0000 7f8e5856e740 1 client.27476 shutdown 2024-04-16T17:13:34.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.824+0000 7f8e5856e740 1 -- 172.21.15.45:0/1963106905 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b2ae3fdf10 msgr2=0x55b2ae4012f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:34.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.824+0000 7f8e5856e740 1 --2- 172.21.15.45:0/1963106905 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b2ae3fdf10 0x55b2ae4012f0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f8e4000e9c0 tx=0x7f8e4000cf90 comp rx=0 tx=0).stop 2024-04-16T17:13:34.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.824+0000 7f8e5856e740 1 -- 172.21.15.45:0/1963106905 shutdown_connections 2024-04-16T17:13:34.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.824+0000 7f8e5856e740 1 --2- 172.21.15.45:0/1963106905 >> [v2:172.21.15.117:6838/2378675949,v1:172.21.15.117:6839/2378675949] conn(0x55b2ae486ce0 0x55b2ae4890d0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:34.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.824+0000 7f8e5856e740 1 --2- 172.21.15.45:0/1963106905 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b2ae407410 0x55b2ae401d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:34.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.824+0000 7f8e5856e740 1 --2- 172.21.15.45:0/1963106905 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b2ae3fe830 0x55b2ae401830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:34.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.824+0000 7f8e5856e740 1 --2- 172.21.15.45:0/1963106905 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b2ae3fdf10 0x55b2ae4012f0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:34.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.824+0000 7f8e5856e740 1 -- 172.21.15.45:0/1963106905 >> 172.21.15.45:0/1963106905 conn(0x55b2ae3f8640 msgr2=0x55b2ae4043b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:13:34.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.825+0000 7f8e38c00640 0 client.27476 ms_handle_reset on v2:172.21.15.117:6838/2378675949 2024-04-16T17:13:34.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.825+0000 7f8e5856e740 1 -- 172.21.15.45:0/1963106905 shutdown_connections 2024-04-16T17:13:34.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.825+0000 7f8e5856e740 1 -- 172.21.15.45:0/1963106905 wait complete. 2024-04-16T17:13:34.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.825+0000 7f8e5856e740 20 client.27476 trim_cache size 0 max 16384 2024-04-16T17:13:34.830 INFO:teuthology.orchestra.run.smithi045.stdout:[] 2024-04-16T17:13:34.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.826+0000 7f8e4fe00640 1 -- 172.21.15.45:0/74804379 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f8e340a3da0 msgr2=0x7f8e340a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:34.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.826+0000 7f8e4fe00640 1 --2- 172.21.15.45:0/74804379 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f8e340a3da0 0x7f8e340a6260 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f8e3c01c040 tx=0x7f8e3c0096c0 comp rx=0 tx=0).stop 2024-04-16T17:13:34.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.826+0000 7f8e4fe00640 1 -- 172.21.15.45:0/74804379 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8e50102110 msgr2=0x7f8e501a0610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:34.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.826+0000 7f8e4fe00640 1 --2- 172.21.15.45:0/74804379 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8e50102110 0x7f8e501a0610 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f8e4000e9f0 tx=0x7f8e4000edf0 comp rx=0 tx=0).stop 2024-04-16T17:13:34.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.826+0000 7f8e4fe00640 1 -- 172.21.15.45:0/74804379 shutdown_connections 2024-04-16T17:13:34.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.826+0000 7f8e4fe00640 1 --2- 172.21.15.45:0/74804379 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f8e340a3da0 0x7f8e340a6260 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:34.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.826+0000 7f8e4fe00640 1 --2- 172.21.15.45:0/74804379 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8e501071e0 0x7f8e501a0b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:34.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.826+0000 7f8e4fe00640 1 --2- 172.21.15.45:0/74804379 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8e5019a780 0x7f8e5019ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:34.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.826+0000 7f8e4fe00640 1 --2- 172.21.15.45:0/74804379 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8e50102110 0x7f8e501a0610 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:34.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.826+0000 7f8e4fe00640 1 -- 172.21.15.45:0/74804379 >> 172.21.15.45:0/74804379 conn(0x7f8e500fd9d0 msgr2=0x7f8e500fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:13:34.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.827+0000 7f8e4fe00640 1 -- 172.21.15.45:0/74804379 shutdown_connections 2024-04-16T17:13:34.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:34.827+0000 7f8e4fe00640 1 -- 172.21.15.45:0/74804379 wait complete. 2024-04-16T17:13:34.845 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-16T17:13:34.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:13:34 smithi104 ceph-mon[36457]: pgmap v209: 129 pgs: 129 active+clean; 59 MiB data, 735 MiB used, 1.0 TiB / 1.0 TiB avail; 782 KiB/s rd, 584 KiB/s wr, 63 op/s 2024-04-16T17:13:35.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.041+0000 7f757f200640 1 -- 172.21.15.45:0/3220444241 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7580074cd0 msgr2=0x7f758010e220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:35.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.041+0000 7f757f200640 1 --2- 172.21.15.45:0/3220444241 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7580074cd0 0x7f758010e220 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f7568009900 tx=0x7f756802bcc0 comp rx=0 tx=0).stop 2024-04-16T17:13:35.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.042+0000 7f757f200640 1 -- 172.21.15.45:0/3220444241 shutdown_connections 2024-04-16T17:13:35.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.042+0000 7f757f200640 1 --2- 172.21.15.45:0/3220444241 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f758010e760 0x7f7580110b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:35.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.042+0000 7f757f200640 1 --2- 172.21.15.45:0/3220444241 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7580074cd0 0x7f758010e220 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:35.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.042+0000 7f757f200640 1 --2- 172.21.15.45:0/3220444241 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f75800743b0 0x7f7580074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:35.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.042+0000 7f757f200640 1 -- 172.21.15.45:0/3220444241 >> 172.21.15.45:0/3220444241 conn(0x7f75800fd9d0 msgr2=0x7f75800ffdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:13:35.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.042+0000 7f757f200640 1 -- 172.21.15.45:0/3220444241 shutdown_connections 2024-04-16T17:13:35.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.042+0000 7f757f200640 1 -- 172.21.15.45:0/3220444241 wait complete. 2024-04-16T17:13:35.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.043+0000 7f757f200640 1 Processor -- start 2024-04-16T17:13:35.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.043+0000 7f757f200640 1 -- start start 2024-04-16T17:13:35.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.044+0000 7f757f200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f75800743b0 0x7f75801a0680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:35.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.044+0000 7f757f200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7580074cd0 0x7f75801a0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:35.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.044+0000 7f757f200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f758010e760 0x7f758019a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:35.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.044+0000 7f757f200640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f75801144c0 con 0x7f7580074cd0 2024-04-16T17:13:35.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.044+0000 7f757f200640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f7580114340 con 0x7f758010e760 2024-04-16T17:13:35.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.044+0000 7f757f200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f7580114640 con 0x7f75800743b0 2024-04-16T17:13:35.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.044+0000 7f757de00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f75800743b0 0x7f75801a0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:35.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.044+0000 7f757e800640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f758010e760 0x7f758019a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:35.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.044+0000 7f757e800640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f758010e760 0x7f758019a840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:39534/0 (socket says 172.21.15.45:39534) 2024-04-16T17:13:35.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.044+0000 7f757e800640 1 -- 172.21.15.45:0/2695869829 learned_addr learned my addr 172.21.15.45:0/2695869829 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:13:35.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.044+0000 7f757de00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f75800743b0 0x7f75801a0680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:57256/0 (socket says 172.21.15.45:57256) 2024-04-16T17:13:35.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.044+0000 7f7577e00640 1 --2- 172.21.15.45:0/2695869829 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7580074cd0 0x7f75801a0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:35.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.045+0000 7f757de00640 1 -- 172.21.15.45:0/2695869829 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f758010e760 msgr2=0x7f758019a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:35.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.045+0000 7f757de00640 1 --2- 172.21.15.45:0/2695869829 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f758010e760 0x7f758019a840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:35.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.045+0000 7f757de00640 1 -- 172.21.15.45:0/2695869829 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7580074cd0 msgr2=0x7f75801a0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:35.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.045+0000 7f757de00640 1 --2- 172.21.15.45:0/2695869829 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7580074cd0 0x7f75801a0bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:35.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.045+0000 7f757de00640 1 -- 172.21.15.45:0/2695869829 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f758019b010 con 0x7f75800743b0 2024-04-16T17:13:35.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.045+0000 7f757e800640 1 --2- 172.21.15.45:0/2695869829 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f758010e760 0x7f758019a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:13:35.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.045+0000 7f757de00640 1 --2- 172.21.15.45:0/2695869829 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f75800743b0 0x7f75801a0680 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f757000e9f0 tx=0x7f757000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:13:35.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.045+0000 7f7576a00640 1 -- 172.21.15.45:0/2695869829 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f757000ccc0 con 0x7f75800743b0 2024-04-16T17:13:35.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.045+0000 7f757f200640 1 -- 172.21.15.45:0/2695869829 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f758019b300 con 0x7f75800743b0 2024-04-16T17:13:35.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.045+0000 7f7576a00640 1 -- 172.21.15.45:0/2695869829 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7570005020 con 0x7f75800743b0 2024-04-16T17:13:35.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.045+0000 7f7576a00640 1 -- 172.21.15.45:0/2695869829 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75700049e0 con 0x7f75800743b0 2024-04-16T17:13:35.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.046+0000 7f757f200640 1 -- 172.21.15.45:0/2695869829 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75801a74d0 con 0x7f75800743b0 2024-04-16T17:13:35.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.049+0000 7f75879c1740 1 Processor -- start 2024-04-16T17:13:35.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.049+0000 7f7576a00640 1 -- 172.21.15.45:0/2695869829 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f7570005560 con 0x7f75800743b0 2024-04-16T17:13:35.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.049+0000 7f75879c1740 1 -- start start 2024-04-16T17:13:35.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.049+0000 7f75879c1740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5596a190bf10 0x5596a190c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:35.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.049+0000 7f75879c1740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5596a190c830 0x5596a1914ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:35.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.049+0000 7f75879c1740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5596a1915410 0x5596a1917800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:35.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.049+0000 7f75879c1740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5596a1871020 con 0x5596a1915410 2024-04-16T17:13:35.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.049+0000 7f75879c1740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5596a1817460 con 0x5596a190bf10 2024-04-16T17:13:35.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.049+0000 7f75879c1740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5596a186cfe0 con 0x5596a190c830 2024-04-16T17:13:35.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.050+0000 7f757e800640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5596a190bf10 0x5596a190c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:35.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.050+0000 7f757de00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5596a190c830 0x5596a1914ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:35.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.050+0000 7f757e800640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5596a190bf10 0x5596a190c2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:39546/0 (socket says 172.21.15.45:39546) 2024-04-16T17:13:35.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.050+0000 7f757e800640 1 -- 172.21.15.45:0/3412759506 learned_addr learned my addr 172.21.15.45:0/3412759506 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:13:35.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.050+0000 7f7577e00640 1 --2- 172.21.15.45:0/3412759506 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5596a1915410 0x5596a1917800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:35.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.050+0000 7f757e800640 1 -- 172.21.15.45:0/3412759506 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5596a190c830 msgr2=0x5596a1914ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:13:35.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.050+0000 7f757e800640 1 --2- 172.21.15.45:0/3412759506 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5596a190c830 0x5596a1914ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:35.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.050+0000 7f757e800640 1 -- 172.21.15.45:0/3412759506 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5596a1915410 msgr2=0x5596a1917800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:13:35.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.050+0000 7f757e800640 1 --2- 172.21.15.45:0/3412759506 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5596a1915410 0x5596a1917800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:35.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.050+0000 7f757e800640 1 -- 172.21.15.45:0/3412759506 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5596a181f040 con 0x5596a190bf10 2024-04-16T17:13:35.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.050+0000 7f7577e00640 1 --2- 172.21.15.45:0/3412759506 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5596a1915410 0x5596a1917800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:13:35.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.051+0000 7f7576a00640 1 --2- 172.21.15.45:0/2695869829 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f75640a3d20 0x7f75640a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:35.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.051+0000 7f7576a00640 1 -- 172.21.15.45:0/2695869829 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9496+0+0 (secure 0 0 0) 0x7f75700d0bc0 con 0x7f75800743b0 2024-04-16T17:13:35.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.051+0000 7f757e800640 1 --2- 172.21.15.45:0/3412759506 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5596a190bf10 0x5596a190c2f0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f7578011850 tx=0x7f7578011d10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e2de34a0c92d12bd server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:13:35.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.051+0000 7f757de00640 1 --2- 172.21.15.45:0/2695869829 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f75640a3d20 0x7f75640a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:35.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.051+0000 7f756f000640 1 -- 172.21.15.45:0/3412759506 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7578011330 con 0x5596a190bf10 2024-04-16T17:13:35.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.051+0000 7f756f000640 1 -- 172.21.15.45:0/3412759506 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f757800fe80 con 0x5596a190bf10 2024-04-16T17:13:35.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.051+0000 7f756f000640 1 -- 172.21.15.45:0/3412759506 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75780193e0 con 0x5596a190bf10 2024-04-16T17:13:35.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.052+0000 7f75879c1740 1 -- 172.21.15.45:0/3412759506 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5596a190bf10 msgr2=0x5596a190c2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:35.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.052+0000 7f75879c1740 1 --2- 172.21.15.45:0/3412759506 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5596a190bf10 0x5596a190c2f0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f7578011850 tx=0x7f7578011d10 comp rx=0 tx=0).stop 2024-04-16T17:13:35.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.052+0000 7f757de00640 1 --2- 172.21.15.45:0/2695869829 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f75640a3d20 0x7f75640a61e0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x5596a190b9e0 tx=0x7f75700d4670 comp rx=0 tx=0).ready entity=mgr.25411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:13:35.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.052+0000 7f75879c1740 1 -- 172.21.15.45:0/3412759506 shutdown_connections 2024-04-16T17:13:35.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.052+0000 7f75879c1740 1 --2- 172.21.15.45:0/3412759506 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5596a190c830 0x5596a1914ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:35.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.052+0000 7f75879c1740 1 --2- 172.21.15.45:0/3412759506 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5596a1915410 0x5596a1917800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:35.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.052+0000 7f75879c1740 1 --2- 172.21.15.45:0/3412759506 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5596a190bf10 0x5596a190c2f0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:35.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.052+0000 7f75879c1740 1 -- 172.21.15.45:0/3412759506 >> 172.21.15.45:0/3412759506 conn(0x5596a1906640 msgr2=0x5596a1906a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:13:35.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.053+0000 7f75879c1740 1 -- 172.21.15.45:0/3412759506 shutdown_connections 2024-04-16T17:13:35.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.053+0000 7f75879c1740 1 -- 172.21.15.45:0/3412759506 wait complete. 2024-04-16T17:13:35.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.053+0000 7f75879c1740 1 -- start start 2024-04-16T17:13:35.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.054+0000 7f75879c1740 1 Processor -- start 2024-04-16T17:13:35.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.054+0000 7f75879c1740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5596a190bf10 0x5596a1912430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:35.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.054+0000 7f75879c1740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5596a190c830 0x5596a1912970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:35.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.054+0000 7f75879c1740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5596a1915410 0x5596a1912eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:35.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.054+0000 7f75879c1740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5596a1918200 con 0x5596a1915410 2024-04-16T17:13:35.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.054+0000 7f75879c1740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5596a182a830 con 0x5596a190c830 2024-04-16T17:13:35.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.054+0000 7f75879c1740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5596a1918380 con 0x5596a190bf10 2024-04-16T17:13:35.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.054+0000 7f75879c1740 10 client.0 _ll_register_callbacks cb 0x5596a1832150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:13:35.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.055+0000 7f7577e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5596a190c830 0x5596a1912970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:35.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.055+0000 7f7577e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5596a190c830 0x5596a1912970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:39552/0 (socket says 172.21.15.45:39552) 2024-04-16T17:13:35.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.055+0000 7f7577e00640 1 -- 172.21.15.45:0/1227631575 learned_addr learned my addr 172.21.15.45:0/1227631575 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:13:35.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.055+0000 7f757e800640 1 --2- 172.21.15.45:0/1227631575 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5596a1915410 0x5596a1912eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:35.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.055+0000 7f757e800640 1 --2- 172.21.15.45:0/1227631575 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5596a190bf10 0x5596a1912430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:35.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.055+0000 7f7577e00640 1 -- 172.21.15.45:0/1227631575 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5596a190bf10 msgr2=0x5596a1912430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:13:35.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.055+0000 7f7577e00640 1 --2- 172.21.15.45:0/1227631575 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5596a190bf10 0x5596a1912430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:35.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.055+0000 7f7577e00640 1 -- 172.21.15.45:0/1227631575 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5596a1915410 msgr2=0x5596a1912eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:13:35.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.055+0000 7f7577e00640 1 --2- 172.21.15.45:0/1227631575 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5596a1915410 0x5596a1912eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:35.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.055+0000 7f7577e00640 1 -- 172.21.15.45:0/1227631575 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5596a182e920 con 0x5596a190c830 2024-04-16T17:13:35.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.055+0000 7f757e800640 1 --2- 172.21.15.45:0/1227631575 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5596a1915410 0x5596a1912eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:13:35.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.055+0000 7f7577e00640 1 --2- 172.21.15.45:0/1227631575 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5596a190c830 0x5596a1912970 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f7568009230 tx=0x7f7568005dd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b69bba2279463df9 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:13:35.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.055+0000 7f7563400640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-04-16T17:13:35.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.056+0000 7f7563400640 1 -- 172.21.15.45:0/1227631575 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7568043070 con 0x5596a190c830 2024-04-16T17:13:35.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.056+0000 7f75879c1740 1 -- 172.21.15.45:0/1227631575 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5596a187ea40 con 0x5596a190c830 2024-04-16T17:13:35.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.056+0000 7f7563400640 1 -- 172.21.15.45:0/1227631575 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9496+0+0 (secure 0 0 0) 0x7f756802cbe0 con 0x5596a190c830 2024-04-16T17:13:35.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.056+0000 7f7563400640 1 -- 172.21.15.45:0/1227631575 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f7568050050 con 0x5596a190c830 2024-04-16T17:13:35.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.056+0000 7f75879c1740 10 client.27488 fetch_fsmap learned FSMap version 38 2024-04-16T17:13:35.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.056+0000 7f75879c1740 1 -- 172.21.15.45:0/1227631575 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x5596a17fc7f0 con 0x5596a190c830 2024-04-16T17:13:35.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.056+0000 7f7563400640 1 -- 172.21.15.45:0/1227631575 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x5596a187ea40 con 0x5596a190c830 2024-04-16T17:13:35.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.057+0000 7f7563400640 1 -- 172.21.15.45:0/1227631575 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 38) ==== 8354+0+0 (secure 0 0 0) 0x7f75680053d0 con 0x5596a190c830 2024-04-16T17:13:35.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.057+0000 7f75879c1740 10 client.27488 fetch_fsmap finished waiting for FSMap version 38 2024-04-16T17:13:35.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.057+0000 7f75879c1740 10 client.27488 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-04-16T17:13:35.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.057+0000 7f75879c1740 20 client.27488 populate_metadata read hostname 'smithi045' 2024-04-16T17:13:35.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.057+0000 7f75879c1740 1 --2- 172.21.15.45:0/1227631575 >> [v2:172.21.15.45:6840/3755310822,v1:172.21.15.45:6841/3755310822] conn(0x5596a1925760 0x5596a1927b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:35.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.057+0000 7f75879c1740 4 client.27488 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:13:35.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.057+0000 7f75879c1740 1 -- 172.21.15.45:0/1227631575 --> [v2:172.21.15.45:6840/3755310822,v1:172.21.15.45:6841/3755310822] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5596a18788b0 con 0x5596a1925760 2024-04-16T17:13:35.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.058+0000 7f757e800640 1 --2- 172.21.15.45:0/1227631575 >> [v2:172.21.15.45:6840/3755310822,v1:172.21.15.45:6841/3755310822] conn(0x5596a1925760 0x5596a1927b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:35.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.058+0000 7f757e800640 1 --2- 172.21.15.45:0/1227631575 >> [v2:172.21.15.45:6840/3755310822,v1:172.21.15.45:6841/3755310822] conn(0x5596a1925760 0x5596a1927b80 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e7d05c37216e8cd1 server_cookie=1cf599528c742a5a in_seq=0 out_seq=0 2024-04-16T17:13:35.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.058+0000 7f7563400640 10 client.27488 ms_handle_connect on v2:172.21.15.45:6840/3755310822 2024-04-16T17:13:35.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.059+0000 7f7563400640 1 -- 172.21.15.45:0/1227631575 <== mds.1 v2:172.21.15.45:6840/3755310822 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5596a18788b0 con 0x5596a1925760 2024-04-16T17:13:35.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.059+0000 7f7563400640 10 client.27488 handle_command_reply: tid=0 2024-04-16T17:13:35.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.059+0000 7f75879c1740 1 client.27488 shutdown 2024-04-16T17:13:35.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.059+0000 7f75879c1740 1 -- 172.21.15.45:0/1227631575 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5596a190c830 msgr2=0x5596a1912970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:35.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.059+0000 7f75879c1740 1 --2- 172.21.15.45:0/1227631575 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5596a190c830 0x5596a1912970 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f7568009230 tx=0x7f7568005dd0 comp rx=0 tx=0).stop 2024-04-16T17:13:35.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.060+0000 7f75879c1740 1 -- 172.21.15.45:0/1227631575 shutdown_connections 2024-04-16T17:13:35.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.060+0000 7f75879c1740 1 --2- 172.21.15.45:0/1227631575 >> [v2:172.21.15.45:6840/3755310822,v1:172.21.15.45:6841/3755310822] conn(0x5596a1925760 0x5596a1927b80 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:35.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.060+0000 7f75879c1740 1 --2- 172.21.15.45:0/1227631575 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5596a1915410 0x5596a1912eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:35.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.060+0000 7f75879c1740 1 --2- 172.21.15.45:0/1227631575 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5596a190c830 0x5596a1912970 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:35.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.060+0000 7f75879c1740 1 --2- 172.21.15.45:0/1227631575 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5596a190bf10 0x5596a1912430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:35.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.060+0000 7f75879c1740 1 -- 172.21.15.45:0/1227631575 >> 172.21.15.45:0/1227631575 conn(0x5596a1906640 msgr2=0x5596a1908390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:13:35.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.060+0000 7f7563400640 0 client.27488 ms_handle_reset on v2:172.21.15.45:6840/3755310822 2024-04-16T17:13:35.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.060+0000 7f75879c1740 1 -- 172.21.15.45:0/1227631575 shutdown_connections 2024-04-16T17:13:35.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.060+0000 7f75879c1740 1 -- 172.21.15.45:0/1227631575 wait complete. 2024-04-16T17:13:35.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.060+0000 7f75879c1740 20 client.27488 trim_cache size 0 max 16384 2024-04-16T17:13:35.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.089+0000 7f75879c1740 1 Processor -- start 2024-04-16T17:13:35.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.089+0000 7f75879c1740 1 -- start start 2024-04-16T17:13:35.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.090+0000 7f75879c1740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5596a190bf10 0x5596a19210c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:35.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.090+0000 7f75879c1740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5596a1915410 0x5596a1905580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:35.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.090+0000 7f75879c1740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5596a1911160 0x5596a192d430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:35.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.090+0000 7f75879c1740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f75801b3f30 con 0x5596a1911160 2024-04-16T17:13:35.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.090+0000 7f75879c1740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f75801b17a0 con 0x5596a190bf10 2024-04-16T17:13:35.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.090+0000 7f75879c1740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f75801af610 con 0x5596a1915410 2024-04-16T17:13:35.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.090+0000 7f757e800640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5596a190bf10 0x5596a19210c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:35.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.090+0000 7f7577e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5596a1915410 0x5596a1905580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:35.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.090+0000 7f757e800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5596a1911160 0x5596a192d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:35.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.090+0000 7f7577e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5596a1915410 0x5596a1905580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:57294/0 (socket says 172.21.15.45:57294) 2024-04-16T17:13:35.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.090+0000 7f7577e00640 1 -- 172.21.15.45:0/2691785830 learned_addr learned my addr 172.21.15.45:0/2691785830 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:13:35.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.090+0000 7f7577e00640 1 -- 172.21.15.45:0/2691785830 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5596a190bf10 msgr2=0x5596a19210c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:13:35.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.090+0000 7f7577e00640 1 --2- 172.21.15.45:0/2691785830 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5596a190bf10 0x5596a19210c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:35.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.091+0000 7f7577e00640 1 -- 172.21.15.45:0/2691785830 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5596a1911160 msgr2=0x5596a192d430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:13:35.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.091+0000 7f7577e00640 1 --2- 172.21.15.45:0/2691785830 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5596a1911160 0x5596a192d430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:35.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.091+0000 7f7577e00640 1 -- 172.21.15.45:0/2691785830 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5596a15833f0 con 0x5596a1915410 2024-04-16T17:13:35.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.091+0000 7f757e800640 1 --2- 172.21.15.45:0/2691785830 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5596a190bf10 0x5596a19210c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:13:35.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.091+0000 7f7577e00640 1 --2- 172.21.15.45:0/2691785830 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5596a1915410 0x5596a1905580 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f756802c200 tx=0x7f756802bcc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1110f3789b80f4a server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:13:35.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.091+0000 7f756e600640 1 -- 172.21.15.45:0/2691785830 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7568059070 con 0x5596a1915410 2024-04-16T17:13:35.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.091+0000 7f756e600640 1 -- 172.21.15.45:0/2691785830 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f756802cde0 con 0x5596a1915410 2024-04-16T17:13:35.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.091+0000 7f756e600640 1 -- 172.21.15.45:0/2691785830 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7568043070 con 0x5596a1915410 2024-04-16T17:13:35.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.092+0000 7f75879c1740 1 -- 172.21.15.45:0/2691785830 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5596a1915410 msgr2=0x5596a1905580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:35.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.092+0000 7f75879c1740 1 --2- 172.21.15.45:0/2691785830 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5596a1915410 0x5596a1905580 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f756802c200 tx=0x7f756802bcc0 comp rx=0 tx=0).stop 2024-04-16T17:13:35.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.092+0000 7f75879c1740 1 -- 172.21.15.45:0/2691785830 shutdown_connections 2024-04-16T17:13:35.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.092+0000 7f75879c1740 1 --2- 172.21.15.45:0/2691785830 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5596a1915410 0x5596a1905580 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:35.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.092+0000 7f75879c1740 1 --2- 172.21.15.45:0/2691785830 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5596a1911160 0x5596a192d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:35.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.092+0000 7f75879c1740 1 --2- 172.21.15.45:0/2691785830 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5596a190bf10 0x5596a19210c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:35.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.092+0000 7f75879c1740 1 -- 172.21.15.45:0/2691785830 >> 172.21.15.45:0/2691785830 conn(0x5596a1906640 msgr2=0x5596a1904e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:13:35.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.092+0000 7f75879c1740 1 -- 172.21.15.45:0/2691785830 shutdown_connections 2024-04-16T17:13:35.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.092+0000 7f75879c1740 1 -- 172.21.15.45:0/2691785830 wait complete. 2024-04-16T17:13:35.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.093+0000 7f75879c1740 1 -- start start 2024-04-16T17:13:35.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.093+0000 7f75879c1740 1 Processor -- start 2024-04-16T17:13:35.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.093+0000 7f75879c1740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5596a190bf10 0x5596a19197c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:35.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.094+0000 7f75879c1740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5596a1911160 0x5596a1919d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:35.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.094+0000 7f75879c1740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5596a1915410 0x5596a198aca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:35.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.094+0000 7f75879c1740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f75801afc60 con 0x5596a190bf10 2024-04-16T17:13:35.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.094+0000 7f75879c1740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f75801b10d0 con 0x5596a1911160 2024-04-16T17:13:35.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.094+0000 7f7577e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5596a1911160 0x5596a1919d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:35.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.094+0000 7f75879c1740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f75801b0330 con 0x5596a1915410 2024-04-16T17:13:35.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.094+0000 7f7577e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5596a1911160 0x5596a1919d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:39578/0 (socket says 172.21.15.45:39578) 2024-04-16T17:13:35.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.094+0000 7f7577e00640 1 -- 172.21.15.45:0/2626918036 learned_addr learned my addr 172.21.15.45:0/2626918036 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:13:35.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.094+0000 7f75879c1740 10 client.0 _ll_register_callbacks cb 0x7f7568004540 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:13:35.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.094+0000 7f757e800640 1 --2- 172.21.15.45:0/2626918036 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5596a190bf10 0x5596a19197c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:35.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.094+0000 7f757e800640 1 --2- 172.21.15.45:0/2626918036 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5596a1915410 0x5596a198aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:35.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.094+0000 7f757e800640 1 -- 172.21.15.45:0/2626918036 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5596a1915410 msgr2=0x5596a198aca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:13:35.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.094+0000 7f757e800640 1 --2- 172.21.15.45:0/2626918036 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5596a1915410 0x5596a198aca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:35.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.094+0000 7f757e800640 1 -- 172.21.15.45:0/2626918036 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5596a1911160 msgr2=0x5596a1919d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:13:35.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.094+0000 7f757e800640 1 --2- 172.21.15.45:0/2626918036 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5596a1911160 0x5596a1919d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:35.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.094+0000 7f757e800640 1 -- 172.21.15.45:0/2626918036 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5596a164d7d0 con 0x5596a190bf10 2024-04-16T17:13:35.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.095+0000 7f757e800640 1 --2- 172.21.15.45:0/2626918036 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5596a190bf10 0x5596a19197c0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f7578000f80 tx=0x7f7578010d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a5d2874eaa23f93 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:13:35.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.095+0000 7f7563400640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-16T17:13:35.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.095+0000 7f7563400640 1 -- 172.21.15.45:0/2626918036 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f757801f040 con 0x5596a190bf10 2024-04-16T17:13:35.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.095+0000 7f7563400640 1 -- 172.21.15.45:0/2626918036 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9496+0+0 (secure 0 0 0) 0x7f7578011330 con 0x5596a190bf10 2024-04-16T17:13:35.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.095+0000 7f75879c1740 1 -- 172.21.15.45:0/2626918036 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5596a17f4570 con 0x5596a190bf10 2024-04-16T17:13:35.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.095+0000 7f7563400640 1 -- 172.21.15.45:0/2626918036 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f7578004a00 con 0x5596a190bf10 2024-04-16T17:13:35.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.095+0000 7f75879c1740 10 client.17331 fetch_fsmap learned FSMap version 38 2024-04-16T17:13:35.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.095+0000 7f75879c1740 1 -- 172.21.15.45:0/2626918036 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x5596a198c390 con 0x5596a190bf10 2024-04-16T17:13:35.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.096+0000 7f7563400640 1 -- 172.21.15.45:0/2626918036 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x5596a17f4570 con 0x5596a190bf10 2024-04-16T17:13:35.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.096+0000 7f7563400640 1 -- 172.21.15.45:0/2626918036 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 38) ==== 8354+0+0 (secure 0 0 0) 0x7f7578025070 con 0x5596a190bf10 2024-04-16T17:13:35.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.096+0000 7f75879c1740 10 client.17331 fetch_fsmap finished waiting for FSMap version 38 2024-04-16T17:13:35.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.096+0000 7f75879c1740 10 client.17331 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-04-16T17:13:35.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.096+0000 7f75879c1740 20 client.17331 populate_metadata read hostname 'smithi045' 2024-04-16T17:13:35.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.096+0000 7f75879c1740 1 --2- 172.21.15.45:0/2626918036 >> [v2:172.21.15.45:6840/3755310822,v1:172.21.15.45:6841/3755310822] conn(0x5596a198c970 0x5596a198edb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:35.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.096+0000 7f75879c1740 4 client.17331 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T17:13:35.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.096+0000 7f75879c1740 1 -- 172.21.15.45:0/2626918036 --> [v2:172.21.15.45:6840/3755310822,v1:172.21.15.45:6841/3755310822] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5596a1841940 con 0x5596a198c970 2024-04-16T17:13:35.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.096+0000 7f7577e00640 1 --2- 172.21.15.45:0/2626918036 >> [v2:172.21.15.45:6840/3755310822,v1:172.21.15.45:6841/3755310822] conn(0x5596a198c970 0x5596a198edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:35.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.097+0000 7f7577e00640 1 --2- 172.21.15.45:0/2626918036 >> [v2:172.21.15.45:6840/3755310822,v1:172.21.15.45:6841/3755310822] conn(0x5596a198c970 0x5596a198edb0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7f85d9b2db1951cb server_cookie=4fc553000884566c in_seq=0 out_seq=0 2024-04-16T17:13:35.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.097+0000 7f7563400640 10 client.17331 ms_handle_connect on v2:172.21.15.45:6840/3755310822 2024-04-16T17:13:35.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.097+0000 7f7563400640 1 -- 172.21.15.45:0/2626918036 <== mds.1 v2:172.21.15.45:6840/3755310822 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5596a1841940 con 0x5596a198c970 2024-04-16T17:13:35.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.097+0000 7f7563400640 10 client.17331 handle_command_reply: tid=0 2024-04-16T17:13:35.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.097+0000 7f75879c1740 1 client.17331 shutdown 2024-04-16T17:13:35.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.097+0000 7f75879c1740 1 -- 172.21.15.45:0/2626918036 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5596a190bf10 msgr2=0x5596a19197c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:35.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.097+0000 7f75879c1740 1 --2- 172.21.15.45:0/2626918036 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5596a190bf10 0x5596a19197c0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f7578000f80 tx=0x7f7578010d10 comp rx=0 tx=0).stop 2024-04-16T17:13:35.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.097+0000 7f75879c1740 1 -- 172.21.15.45:0/2626918036 shutdown_connections 2024-04-16T17:13:35.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.097+0000 7f75879c1740 1 --2- 172.21.15.45:0/2626918036 >> [v2:172.21.15.45:6840/3755310822,v1:172.21.15.45:6841/3755310822] conn(0x5596a198c970 0x5596a198edb0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:35.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.097+0000 7f75879c1740 1 --2- 172.21.15.45:0/2626918036 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5596a1915410 0x5596a198aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:35.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.097+0000 7f75879c1740 1 --2- 172.21.15.45:0/2626918036 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5596a1911160 0x5596a1919d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:35.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.097+0000 7f75879c1740 1 --2- 172.21.15.45:0/2626918036 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5596a190bf10 0x5596a19197c0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:35.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.098+0000 7f75879c1740 1 -- 172.21.15.45:0/2626918036 >> 172.21.15.45:0/2626918036 conn(0x5596a1906640 msgr2=0x5596a1904e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:13:35.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.098+0000 7f7563400640 0 client.17331 ms_handle_reset on v2:172.21.15.45:6840/3755310822 2024-04-16T17:13:35.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.098+0000 7f75879c1740 1 -- 172.21.15.45:0/2626918036 shutdown_connections 2024-04-16T17:13:35.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.098+0000 7f75879c1740 1 -- 172.21.15.45:0/2626918036 wait complete. 2024-04-16T17:13:35.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.098+0000 7f75879c1740 20 client.17331 trim_cache size 0 max 16384 2024-04-16T17:13:35.102 INFO:teuthology.orchestra.run.smithi045.stdout:[] 2024-04-16T17:13:35.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.099+0000 7f757f200640 1 -- 172.21.15.45:0/2695869829 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f75640a3d20 msgr2=0x7f75640a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:35.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.099+0000 7f757f200640 1 --2- 172.21.15.45:0/2695869829 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f75640a3d20 0x7f75640a61e0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x5596a190b9e0 tx=0x7f75700d4670 comp rx=0 tx=0).stop 2024-04-16T17:13:35.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.099+0000 7f757f200640 1 -- 172.21.15.45:0/2695869829 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f75800743b0 msgr2=0x7f75801a0680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:35.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.099+0000 7f757f200640 1 --2- 172.21.15.45:0/2695869829 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f75800743b0 0x7f75801a0680 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f757000e9f0 tx=0x7f757000eeb0 comp rx=0 tx=0).stop 2024-04-16T17:13:35.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.099+0000 7f757f200640 1 -- 172.21.15.45:0/2695869829 shutdown_connections 2024-04-16T17:13:35.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.099+0000 7f757f200640 1 --2- 172.21.15.45:0/2695869829 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f75640a3d20 0x7f75640a61e0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:35.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.099+0000 7f757f200640 1 --2- 172.21.15.45:0/2695869829 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f758010e760 0x7f758019a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:35.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.099+0000 7f757f200640 1 --2- 172.21.15.45:0/2695869829 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7580074cd0 0x7f75801a0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:35.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.099+0000 7f757f200640 1 --2- 172.21.15.45:0/2695869829 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f75800743b0 0x7f75801a0680 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:35.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.099+0000 7f757f200640 1 -- 172.21.15.45:0/2695869829 >> 172.21.15.45:0/2695869829 conn(0x7f75800fd9d0 msgr2=0x7f75800fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:13:35.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.099+0000 7f757f200640 1 -- 172.21.15.45:0/2695869829 shutdown_connections 2024-04-16T17:13:35.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.099+0000 7f757f200640 1 -- 172.21.15.45:0/2695869829 wait complete. 2024-04-16T17:13:35.123 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-16T17:13:35.288 INFO:tasks.workunit.client.0.smithi045.stdout: 1 113292 1.75 MB/sec execute 217 sec latency 44.793 ms 2024-04-16T17:13:35.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.328+0000 7f1e51400640 1 -- 172.21.15.45:0/2530109990 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1e4c103060 msgr2=0x7f1e4c103440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:35.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.328+0000 7f1e51400640 1 --2- 172.21.15.45:0/2530109990 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1e4c103060 0x7f1e4c103440 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f1e3c009860 tx=0x7f1e3c02bc20 comp rx=0 tx=0).stop 2024-04-16T17:13:35.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.328+0000 7f1e51400640 1 -- 172.21.15.45:0/2530109990 shutdown_connections 2024-04-16T17:13:35.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.328+0000 7f1e51400640 1 --2- 172.21.15.45:0/2530109990 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1e4c10c480 0x7f1e4c10e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:35.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.328+0000 7f1e51400640 1 --2- 172.21.15.45:0/2530109990 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1e4c103980 0x7f1e4c10be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:35.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.328+0000 7f1e51400640 1 --2- 172.21.15.45:0/2530109990 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1e4c103060 0x7f1e4c103440 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:35.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.328+0000 7f1e51400640 1 -- 172.21.15.45:0/2530109990 >> 172.21.15.45:0/2530109990 conn(0x7f1e4c0fc880 msgr2=0x7f1e4c0feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:13:35.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.328+0000 7f1e51400640 1 -- 172.21.15.45:0/2530109990 shutdown_connections 2024-04-16T17:13:35.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.329+0000 7f1e51400640 1 -- 172.21.15.45:0/2530109990 wait complete. 2024-04-16T17:13:35.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.329+0000 7f1e51400640 1 Processor -- start 2024-04-16T17:13:35.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.330+0000 7f1e51400640 1 -- start start 2024-04-16T17:13:35.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.330+0000 7f1e51400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1e4c103060 0x7f1e4c1965d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:35.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.330+0000 7f1e51400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1e4c103980 0x7f1e4c196b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:35.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.330+0000 7f1e51400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1e4c10c480 0x7f1e4c19ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:35.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.330+0000 7f1e51400640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f1e4c110070 con 0x7f1e4c103060 2024-04-16T17:13:35.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.330+0000 7f1e51400640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f1e4c10fef0 con 0x7f1e4c103980 2024-04-16T17:13:35.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.330+0000 7f1e51400640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f1e4c1101f0 con 0x7f1e4c10c480 2024-04-16T17:13:35.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.330+0000 7f1e4b400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1e4c103980 0x7f1e4c196b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:35.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.330+0000 7f1e4b400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1e4c103980 0x7f1e4c196b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:39584/0 (socket says 172.21.15.45:39584) 2024-04-16T17:13:35.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.330+0000 7f1e4be00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1e4c103060 0x7f1e4c1965d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:35.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.330+0000 7f1e4b400640 1 -- 172.21.15.45:0/263988841 learned_addr learned my addr 172.21.15.45:0/263988841 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:13:35.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.330+0000 7f1e50a00640 1 --2- 172.21.15.45:0/263988841 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1e4c10c480 0x7f1e4c19ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:35.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.331+0000 7f1e4b400640 1 -- 172.21.15.45:0/263988841 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1e4c10c480 msgr2=0x7f1e4c19ae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:35.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.331+0000 7f1e4b400640 1 --2- 172.21.15.45:0/263988841 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1e4c10c480 0x7f1e4c19ae40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:35.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.331+0000 7f1e4b400640 1 -- 172.21.15.45:0/263988841 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1e4c103060 msgr2=0x7f1e4c1965d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:35.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.331+0000 7f1e4b400640 1 --2- 172.21.15.45:0/263988841 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1e4c103060 0x7f1e4c1965d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:35.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.331+0000 7f1e4b400640 1 -- 172.21.15.45:0/263988841 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e4c19b570 con 0x7f1e4c103980 2024-04-16T17:13:35.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.331+0000 7f1e4b400640 1 --2- 172.21.15.45:0/263988841 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1e4c103980 0x7f1e4c196b10 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f1e3800e9f0 tx=0x7f1e3800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:13:35.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.331+0000 7f1e48c00640 1 -- 172.21.15.45:0/263988841 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e3800cd80 con 0x7f1e4c103980 2024-04-16T17:13:35.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.331+0000 7f1e48c00640 1 -- 172.21.15.45:0/263988841 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1e380035f0 con 0x7f1e4c103980 2024-04-16T17:13:35.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.331+0000 7f1e48c00640 1 -- 172.21.15.45:0/263988841 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e38010670 con 0x7f1e4c103980 2024-04-16T17:13:35.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.331+0000 7f1e51400640 1 -- 172.21.15.45:0/263988841 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e4c19b860 con 0x7f1e4c103980 2024-04-16T17:13:35.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.332+0000 7f1e51400640 1 -- 172.21.15.45:0/263988841 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e4c1a30f0 con 0x7f1e4c103980 2024-04-16T17:13:35.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.335+0000 7f1e55609740 1 Processor -- start 2024-04-16T17:13:35.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.335+0000 7f1e55609740 1 -- start start 2024-04-16T17:13:35.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.336+0000 7f1e55609740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561c679f1f10 0x561c679f22f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:35.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.336+0000 7f1e55609740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561c679f2830 0x561c679faed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:35.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.336+0000 7f1e55609740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561c679fb410 0x561c679fd800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:35.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.336+0000 7f1e55609740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x561c67957020 con 0x561c679f2830 2024-04-16T17:13:35.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.336+0000 7f1e55609740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x561c678fd460 con 0x561c679fb410 2024-04-16T17:13:35.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.336+0000 7f1e55609740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x561c67952fe0 con 0x561c679f1f10 2024-04-16T17:13:35.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.336+0000 7f1e4be00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561c679f2830 0x561c679faed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:35.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.336+0000 7f1e50a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561c679f1f10 0x561c679f22f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:35.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.336+0000 7f1e50a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561c679f1f10 0x561c679f22f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:57324/0 (socket says 172.21.15.45:57324) 2024-04-16T17:13:35.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.336+0000 7f1e50a00640 1 -- 172.21.15.45:0/2778057161 learned_addr learned my addr 172.21.15.45:0/2778057161 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:13:35.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.336+0000 7f1e50a00640 1 -- 172.21.15.45:0/2778057161 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561c679fb410 msgr2=0x561c679fd800 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-16T17:13:35.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.336+0000 7f1e50a00640 1 --2- 172.21.15.45:0/2778057161 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561c679fb410 0x561c679fd800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:35.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.336+0000 7f1e50a00640 1 -- 172.21.15.45:0/2778057161 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561c679f2830 msgr2=0x561c679faed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:13:35.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.336+0000 7f1e50a00640 1 --2- 172.21.15.45:0/2778057161 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561c679f2830 0x561c679faed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:35.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.337+0000 7f1e50a00640 1 -- 172.21.15.45:0/2778057161 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561c67905040 con 0x561c679f1f10 2024-04-16T17:13:35.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.337+0000 7f1e48c00640 1 -- 172.21.15.45:0/263988841 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f1e380037a0 con 0x7f1e4c103980 2024-04-16T17:13:35.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.337+0000 7f1e50a00640 1 --2- 172.21.15.45:0/2778057161 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561c679f1f10 0x561c679f22f0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f1e4400d530 tx=0x7f1e4400d9d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=238974b89c76ce95 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:13:35.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.337+0000 7f1e41200640 1 -- 172.21.15.45:0/2778057161 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e4401f030 con 0x561c679f1f10 2024-04-16T17:13:35.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.337+0000 7f1e41200640 1 -- 172.21.15.45:0/2778057161 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1e44003e50 con 0x561c679f1f10 2024-04-16T17:13:35.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.337+0000 7f1e41200640 1 -- 172.21.15.45:0/2778057161 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e44004240 con 0x561c679f1f10 2024-04-16T17:13:35.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.338+0000 7f1e55609740 1 -- 172.21.15.45:0/2778057161 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561c679f1f10 msgr2=0x561c679f22f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:35.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.338+0000 7f1e55609740 1 --2- 172.21.15.45:0/2778057161 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561c679f1f10 0x561c679f22f0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f1e4400d530 tx=0x7f1e4400d9d0 comp rx=0 tx=0).stop 2024-04-16T17:13:35.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.338+0000 7f1e55609740 1 -- 172.21.15.45:0/2778057161 shutdown_connections 2024-04-16T17:13:35.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.338+0000 7f1e55609740 1 --2- 172.21.15.45:0/2778057161 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561c679fb410 0x561c679fd800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:35.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.338+0000 7f1e55609740 1 --2- 172.21.15.45:0/2778057161 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561c679f2830 0x561c679faed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:35.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.338+0000 7f1e55609740 1 --2- 172.21.15.45:0/2778057161 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561c679f1f10 0x561c679f22f0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:35.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.338+0000 7f1e55609740 1 -- 172.21.15.45:0/2778057161 >> 172.21.15.45:0/2778057161 conn(0x561c679ec640 msgr2=0x561c679eca20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:13:35.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.338+0000 7f1e55609740 1 -- 172.21.15.45:0/2778057161 shutdown_connections 2024-04-16T17:13:35.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.338+0000 7f1e55609740 1 -- 172.21.15.45:0/2778057161 wait complete. 2024-04-16T17:13:35.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.338+0000 7f1e48c00640 1 --2- 172.21.15.45:0/263988841 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f1e300a3c30 0x7f1e300a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:35.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.339+0000 7f1e48c00640 1 -- 172.21.15.45:0/263988841 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9496+0+0 (secure 0 0 0) 0x7f1e380d0ec0 con 0x7f1e4c103980 2024-04-16T17:13:35.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.339+0000 7f1e4be00640 1 --2- 172.21.15.45:0/263988841 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f1e300a3c30 0x7f1e300a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:35.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.339+0000 7f1e55609740 1 -- start start 2024-04-16T17:13:35.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.339+0000 7f1e55609740 1 Processor -- start 2024-04-16T17:13:35.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.340+0000 7f1e55609740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561c679f1f10 0x561c679f84d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:35.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.340+0000 7f1e4be00640 1 --2- 172.21.15.45:0/263988841 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f1e300a3c30 0x7f1e300a60f0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f1e3c02c230 tx=0x7f1e3c0058d0 comp rx=0 tx=0).ready entity=mgr.25411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:13:35.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.340+0000 7f1e55609740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561c679f2830 0x561c679f8a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:35.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.340+0000 7f1e55609740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561c679fb410 0x561c679f8f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:35.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.340+0000 7f1e55609740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x561c679fe200 con 0x561c679fb410 2024-04-16T17:13:35.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.340+0000 7f1e55609740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x561c67910830 con 0x561c679f2830 2024-04-16T17:13:35.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.340+0000 7f1e55609740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x561c679fe380 con 0x561c679f1f10 2024-04-16T17:13:35.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.340+0000 7f1e55609740 10 client.0 _ll_register_callbacks cb 0x561c67918150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:13:35.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.340+0000 7f1e4b400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561c679f2830 0x561c679f8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:35.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.340+0000 7f1e4b400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561c679f2830 0x561c679f8a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:39592/0 (socket says 172.21.15.45:39592) 2024-04-16T17:13:35.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.340+0000 7f1e4b400640 1 -- 172.21.15.45:0/2413640938 learned_addr learned my addr 172.21.15.45:0/2413640938 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:13:35.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.340+0000 7f1e4be00640 1 --2- 172.21.15.45:0/2413640938 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561c679f1f10 0x561c679f84d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:35.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.340+0000 7f1e50a00640 1 --2- 172.21.15.45:0/2413640938 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561c679fb410 0x561c679f8f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:35.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.340+0000 7f1e4b400640 1 -- 172.21.15.45:0/2413640938 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561c679f1f10 msgr2=0x561c679f84d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:13:35.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.340+0000 7f1e4b400640 1 --2- 172.21.15.45:0/2413640938 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561c679f1f10 0x561c679f84d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:35.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.340+0000 7f1e4b400640 1 -- 172.21.15.45:0/2413640938 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561c679fb410 msgr2=0x561c679f8f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:13:35.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.340+0000 7f1e4b400640 1 --2- 172.21.15.45:0/2413640938 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561c679fb410 0x561c679f8f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:35.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.341+0000 7f1e4b400640 1 -- 172.21.15.45:0/2413640938 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561c67914920 con 0x561c679f2830 2024-04-16T17:13:35.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.341+0000 7f1e50a00640 1 --2- 172.21.15.45:0/2413640938 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561c679fb410 0x561c679f8f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:13:35.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.341+0000 7f1e4b400640 1 --2- 172.21.15.45:0/2413640938 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561c679f2830 0x561c679f8a10 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f1e3800e9c0 tx=0x7f1e3809c450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5f529136ce2e8746 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:13:35.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.341+0000 7f1e35600640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-04-16T17:13:35.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.341+0000 7f1e35600640 1 -- 172.21.15.45:0/2413640938 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e380a0de0 con 0x561c679f2830 2024-04-16T17:13:35.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.341+0000 7f1e35600640 1 -- 172.21.15.45:0/2413640938 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9496+0+0 (secure 0 0 0) 0x7f1e380990a0 con 0x561c679f2830 2024-04-16T17:13:35.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.341+0000 7f1e55609740 1 -- 172.21.15.45:0/2413640938 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1e44003e50 con 0x561c679f2830 2024-04-16T17:13:35.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.341+0000 7f1e35600640 1 -- 172.21.15.45:0/2413640938 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f1e380a56a0 con 0x561c679f2830 2024-04-16T17:13:35.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.341+0000 7f1e55609740 10 client.27503 fetch_fsmap learned FSMap version 38 2024-04-16T17:13:35.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.341+0000 7f1e55609740 1 -- 172.21.15.45:0/2413640938 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x561c678e27f0 con 0x561c679f2830 2024-04-16T17:13:35.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.342+0000 7f1e35600640 1 -- 172.21.15.45:0/2413640938 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1e44003e50 con 0x561c679f2830 2024-04-16T17:13:35.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.342+0000 7f1e35600640 1 -- 172.21.15.45:0/2413640938 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 38) ==== 8354+0+0 (secure 0 0 0) 0x7f1e38098780 con 0x561c679f2830 2024-04-16T17:13:35.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.342+0000 7f1e55609740 10 client.27503 fetch_fsmap finished waiting for FSMap version 38 2024-04-16T17:13:35.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.342+0000 7f1e55609740 10 client.27503 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:13:35.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.342+0000 7f1e55609740 20 client.27503 populate_metadata read hostname 'smithi045' 2024-04-16T17:13:35.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.342+0000 7f1e55609740 1 --2- 172.21.15.45:0/2413640938 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x561c67a0b7c0 0x561c67a0dc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:35.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.342+0000 7f1e55609740 4 client.27503 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:13:35.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.342+0000 7f1e55609740 1 -- 172.21.15.45:0/2413640938 --> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561c67964a40 con 0x561c67a0b7c0 2024-04-16T17:13:35.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.342+0000 7f1e4be00640 1 --2- 172.21.15.45:0/2413640938 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x561c67a0b7c0 0x561c67a0dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:35.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.343+0000 7f1e4be00640 1 --2- 172.21.15.45:0/2413640938 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x561c67a0b7c0 0x561c67a0dc20 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f0ec0195690bbefa server_cookie=aa8f1def89b2f1df in_seq=0 out_seq=0 2024-04-16T17:13:35.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.343+0000 7f1e35600640 10 client.27503 ms_handle_connect on v2:172.21.15.104:6838/3660070592 2024-04-16T17:13:35.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.344+0000 7f1e35600640 1 -- 172.21.15.45:0/2413640938 <== mds.0 v2:172.21.15.104:6838/3660070592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561c67964a40 con 0x561c67a0b7c0 2024-04-16T17:13:35.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.344+0000 7f1e35600640 10 client.27503 handle_command_reply: tid=0 2024-04-16T17:13:35.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.344+0000 7f1e55609740 1 client.27503 shutdown 2024-04-16T17:13:35.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.344+0000 7f1e55609740 1 -- 172.21.15.45:0/2413640938 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561c679f2830 msgr2=0x561c679f8a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:35.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.344+0000 7f1e55609740 1 --2- 172.21.15.45:0/2413640938 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561c679f2830 0x561c679f8a10 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f1e3800e9c0 tx=0x7f1e3809c450 comp rx=0 tx=0).stop 2024-04-16T17:13:35.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.344+0000 7f1e55609740 1 -- 172.21.15.45:0/2413640938 shutdown_connections 2024-04-16T17:13:35.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.344+0000 7f1e55609740 1 --2- 172.21.15.45:0/2413640938 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561c679fb410 0x561c679f8f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:35.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.344+0000 7f1e55609740 1 --2- 172.21.15.45:0/2413640938 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561c679f2830 0x561c679f8a10 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:35.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.344+0000 7f1e55609740 1 --2- 172.21.15.45:0/2413640938 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x561c67a0b7c0 0x561c67a0dc20 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:35.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.344+0000 7f1e55609740 1 --2- 172.21.15.45:0/2413640938 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561c679f1f10 0x561c679f84d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:35.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.344+0000 7f1e35600640 0 client.27503 ms_handle_reset on v2:172.21.15.104:6838/3660070592 2024-04-16T17:13:35.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.344+0000 7f1e55609740 1 -- 172.21.15.45:0/2413640938 >> 172.21.15.45:0/2413640938 conn(0x561c679ec640 msgr2=0x561c679ee390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:13:35.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.344+0000 7f1e55609740 1 -- 172.21.15.45:0/2413640938 shutdown_connections 2024-04-16T17:13:35.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.344+0000 7f1e55609740 1 -- 172.21.15.45:0/2413640938 wait complete. 2024-04-16T17:13:35.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.344+0000 7f1e55609740 20 client.27503 trim_cache size 0 max 16384 2024-04-16T17:13:35.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.359+0000 7f1e55609740 1 Processor -- start 2024-04-16T17:13:35.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.359+0000 7f1e55609740 1 -- start start 2024-04-16T17:13:35.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.359+0000 7f1e55609740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561c679f1f10 0x561c679fac00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:35.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.359+0000 7f1e55609740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561c679f2830 0x561c67a070c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:35.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.359+0000 7f1e55609740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561c679fb410 0x561c679f7630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:35.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.359+0000 7f1e55609740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f1e4c1afac0 con 0x561c679fb410 2024-04-16T17:13:35.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.359+0000 7f1e55609740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f1e4c1ad330 con 0x561c679f2830 2024-04-16T17:13:35.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.359+0000 7f1e55609740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f1e4c1ab1a0 con 0x561c679f1f10 2024-04-16T17:13:35.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.359+0000 7f1e4be00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561c679f1f10 0x561c679fac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:35.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.359+0000 7f1e50a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561c679fb410 0x561c679f7630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:35.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.359+0000 7f1e4be00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561c679f1f10 0x561c679fac00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:57346/0 (socket says 172.21.15.45:57346) 2024-04-16T17:13:35.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.359+0000 7f1e50a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561c679fb410 0x561c679f7630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56820/0 (socket says 172.21.15.45:56820) 2024-04-16T17:13:35.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.359+0000 7f1e4be00640 1 -- 172.21.15.45:0/4185784240 learned_addr learned my addr 172.21.15.45:0/4185784240 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:13:35.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.360+0000 7f1e4b400640 1 --2- 172.21.15.45:0/4185784240 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561c679f2830 0x561c67a070c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:35.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.360+0000 7f1e4be00640 1 -- 172.21.15.45:0/4185784240 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561c679f2830 msgr2=0x561c67a070c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:13:35.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.360+0000 7f1e4be00640 1 --2- 172.21.15.45:0/4185784240 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561c679f2830 0x561c67a070c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:35.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.360+0000 7f1e4be00640 1 -- 172.21.15.45:0/4185784240 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561c679fb410 msgr2=0x561c679f7630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:13:35.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.360+0000 7f1e4be00640 1 --2- 172.21.15.45:0/4185784240 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561c679fb410 0x561c679f7630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:35.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.360+0000 7f1e4be00640 1 -- 172.21.15.45:0/4185784240 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561c676693f0 con 0x561c679f1f10 2024-04-16T17:13:35.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.360+0000 7f1e4b400640 1 --2- 172.21.15.45:0/4185784240 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561c679f2830 0x561c67a070c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:13:35.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.360+0000 7f1e4be00640 1 --2- 172.21.15.45:0/4185784240 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561c679f1f10 0x561c679fac00 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f1e3c02c160 tx=0x7f1e3c03f000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2421c64bba3c9caf server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:13:35.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.360+0000 7f1e37e00640 1 -- 172.21.15.45:0/4185784240 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e3c05b070 con 0x561c679f1f10 2024-04-16T17:13:35.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.361+0000 7f1e37e00640 1 -- 172.21.15.45:0/4185784240 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1e3c046040 con 0x561c679f1f10 2024-04-16T17:13:35.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.361+0000 7f1e37e00640 1 -- 172.21.15.45:0/4185784240 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e3c034e60 con 0x561c679f1f10 2024-04-16T17:13:35.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.361+0000 7f1e55609740 1 -- 172.21.15.45:0/4185784240 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561c679f1f10 msgr2=0x561c679fac00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:35.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.361+0000 7f1e55609740 1 --2- 172.21.15.45:0/4185784240 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561c679f1f10 0x561c679fac00 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f1e3c02c160 tx=0x7f1e3c03f000 comp rx=0 tx=0).stop 2024-04-16T17:13:35.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.361+0000 7f1e55609740 1 -- 172.21.15.45:0/4185784240 shutdown_connections 2024-04-16T17:13:35.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.361+0000 7f1e55609740 1 --2- 172.21.15.45:0/4185784240 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561c679fb410 0x561c679f7630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:35.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.361+0000 7f1e55609740 1 --2- 172.21.15.45:0/4185784240 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561c679f2830 0x561c67a070c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:35.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.361+0000 7f1e55609740 1 --2- 172.21.15.45:0/4185784240 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561c679f1f10 0x561c679fac00 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:35.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.361+0000 7f1e55609740 1 -- 172.21.15.45:0/4185784240 >> 172.21.15.45:0/4185784240 conn(0x561c679ec640 msgr2=0x561c679eda90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:13:35.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.361+0000 7f1e55609740 1 -- 172.21.15.45:0/4185784240 shutdown_connections 2024-04-16T17:13:35.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.361+0000 7f1e55609740 1 -- 172.21.15.45:0/4185784240 wait complete. 2024-04-16T17:13:35.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.362+0000 7f1e55609740 1 -- start start 2024-04-16T17:13:35.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.362+0000 7f1e55609740 1 Processor -- start 2024-04-16T17:13:35.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.363+0000 7f1e55609740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561c679f1f10 0x561c679f5410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:35.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.363+0000 7f1e55609740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561c679f2830 0x561c679f5950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:35.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.363+0000 7f1e55609740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561c679fb410 0x561c679f5e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:35.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.363+0000 7f1e55609740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f1e4c1ab7f0 con 0x561c679f2830 2024-04-16T17:13:35.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.363+0000 7f1e55609740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f1e4c1acc60 con 0x561c679fb410 2024-04-16T17:13:35.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.363+0000 7f1e55609740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f1e4c1abec0 con 0x561c679f1f10 2024-04-16T17:13:35.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.363+0000 7f1e55609740 10 client.0 _ll_register_callbacks cb 0x7f1e380094b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:13:35.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.363+0000 7f1e4be00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561c679f1f10 0x561c679f5410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:35.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.363+0000 7f1e4be00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561c679f1f10 0x561c679f5410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:57352/0 (socket says 172.21.15.45:57352) 2024-04-16T17:13:35.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.363+0000 7f1e4be00640 1 -- 172.21.15.45:0/31900946 learned_addr learned my addr 172.21.15.45:0/31900946 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:13:35.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.363+0000 7f1e50a00640 1 --2- 172.21.15.45:0/31900946 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561c679fb410 0x561c679f5e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:35.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.363+0000 7f1e4b400640 1 --2- 172.21.15.45:0/31900946 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561c679f2830 0x561c679f5950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:35.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.363+0000 7f1e4be00640 1 -- 172.21.15.45:0/31900946 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561c679fb410 msgr2=0x561c679f5e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:13:35.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.363+0000 7f1e4be00640 1 --2- 172.21.15.45:0/31900946 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561c679fb410 0x561c679f5e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:35.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.363+0000 7f1e4be00640 1 -- 172.21.15.45:0/31900946 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561c679f2830 msgr2=0x561c679f5950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:13:35.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.363+0000 7f1e4be00640 1 --2- 172.21.15.45:0/31900946 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561c679f2830 0x561c679f5950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:35.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.363+0000 7f1e4be00640 1 -- 172.21.15.45:0/31900946 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561c677337d0 con 0x561c679f1f10 2024-04-16T17:13:35.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.363+0000 7f1e4be00640 1 --2- 172.21.15.45:0/31900946 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561c679f1f10 0x561c679f5410 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f1e3c004330 tx=0x7f1e3c02bc20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b77148dbff0a6cf3 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:13:35.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.363+0000 7f1e35600640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-04-16T17:13:35.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.364+0000 7f1e35600640 1 -- 172.21.15.45:0/31900946 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e3c05b070 con 0x561c679f1f10 2024-04-16T17:13:35.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.364+0000 7f1e55609740 1 -- 172.21.15.45:0/31900946 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1e3c046040 con 0x561c679f1f10 2024-04-16T17:13:35.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.364+0000 7f1e35600640 1 -- 172.21.15.45:0/31900946 <== mon.1 v2:172.21.15.117:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9496+0+0 (secure 0 0 0) 0x7f1e3c02c640 con 0x561c679f1f10 2024-04-16T17:13:35.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.364+0000 7f1e35600640 1 -- 172.21.15.45:0/31900946 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f1e3c039040 con 0x561c679f1f10 2024-04-16T17:13:35.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.364+0000 7f1e55609740 10 client.27454 fetch_fsmap learned FSMap version 38 2024-04-16T17:13:35.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.364+0000 7f1e55609740 1 -- 172.21.15.45:0/31900946 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x561c67a72110 con 0x561c679f1f10 2024-04-16T17:13:35.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.364+0000 7f1e35600640 1 -- 172.21.15.45:0/31900946 <== mon.1 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1e3c046040 con 0x561c679f1f10 2024-04-16T17:13:35.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.364+0000 7f1e35600640 1 -- 172.21.15.45:0/31900946 <== mon.1 v2:172.21.15.117:3300/0 5 ==== fsmap(e 38) ==== 8354+0+0 (secure 0 0 0) 0x7f1e3c04b040 con 0x561c679f1f10 2024-04-16T17:13:35.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.365+0000 7f1e55609740 10 client.27454 fetch_fsmap finished waiting for FSMap version 38 2024-04-16T17:13:35.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.365+0000 7f1e55609740 10 client.27454 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:13:35.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.365+0000 7f1e55609740 20 client.27454 populate_metadata read hostname 'smithi045' 2024-04-16T17:13:35.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.365+0000 7f1e55609740 1 --2- 172.21.15.45:0/31900946 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x561c67a727d0 0x561c67a74bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:35.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.365+0000 7f1e55609740 4 client.27454 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T17:13:35.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.365+0000 7f1e55609740 1 -- 172.21.15.45:0/31900946 --> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561c6795e8b0 con 0x561c67a727d0 2024-04-16T17:13:35.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.365+0000 7f1e4b400640 1 --2- 172.21.15.45:0/31900946 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x561c67a727d0 0x561c67a74bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:35.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.365+0000 7f1e4b400640 1 --2- 172.21.15.45:0/31900946 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x561c67a727d0 0x561c67a74bc0 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=106f615c3f04ccc0 server_cookie=756092c9d9098d09 in_seq=0 out_seq=0 2024-04-16T17:13:35.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.365+0000 7f1e35600640 10 client.27454 ms_handle_connect on v2:172.21.15.104:6838/3660070592 2024-04-16T17:13:35.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.366+0000 7f1e35600640 1 -- 172.21.15.45:0/31900946 <== mds.0 v2:172.21.15.104:6838/3660070592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561c6795e8b0 con 0x561c67a727d0 2024-04-16T17:13:35.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.366+0000 7f1e35600640 10 client.27454 handle_command_reply: tid=0 2024-04-16T17:13:35.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.366+0000 7f1e55609740 1 client.27454 shutdown 2024-04-16T17:13:35.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.366+0000 7f1e55609740 1 -- 172.21.15.45:0/31900946 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561c679f1f10 msgr2=0x561c679f5410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:35.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.366+0000 7f1e55609740 1 --2- 172.21.15.45:0/31900946 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561c679f1f10 0x561c679f5410 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f1e3c004330 tx=0x7f1e3c02bc20 comp rx=0 tx=0).stop 2024-04-16T17:13:35.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.366+0000 7f1e55609740 1 -- 172.21.15.45:0/31900946 shutdown_connections 2024-04-16T17:13:35.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.366+0000 7f1e55609740 1 --2- 172.21.15.45:0/31900946 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561c679fb410 0x561c679f5e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:35.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.366+0000 7f1e55609740 1 --2- 172.21.15.45:0/31900946 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561c679f2830 0x561c679f5950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:35.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.366+0000 7f1e55609740 1 --2- 172.21.15.45:0/31900946 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x561c67a727d0 0x561c67a74bc0 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:35.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.366+0000 7f1e55609740 1 --2- 172.21.15.45:0/31900946 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561c679f1f10 0x561c679f5410 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:35.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.366+0000 7f1e55609740 1 -- 172.21.15.45:0/31900946 >> 172.21.15.45:0/31900946 conn(0x561c679ec640 msgr2=0x561c679eb700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:13:35.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.366+0000 7f1e35600640 0 client.27454 ms_handle_reset on v2:172.21.15.104:6838/3660070592 2024-04-16T17:13:35.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.366+0000 7f1e55609740 1 -- 172.21.15.45:0/31900946 shutdown_connections 2024-04-16T17:13:35.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.366+0000 7f1e55609740 1 -- 172.21.15.45:0/31900946 wait complete. 2024-04-16T17:13:35.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.366+0000 7f1e55609740 20 client.27454 trim_cache size 0 max 16384 2024-04-16T17:13:35.371 INFO:teuthology.orchestra.run.smithi045.stdout:[] 2024-04-16T17:13:35.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.367+0000 7f1e51400640 1 -- 172.21.15.45:0/263988841 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f1e300a3c30 msgr2=0x7f1e300a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:35.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.367+0000 7f1e51400640 1 --2- 172.21.15.45:0/263988841 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f1e300a3c30 0x7f1e300a60f0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f1e3c02c230 tx=0x7f1e3c0058d0 comp rx=0 tx=0).stop 2024-04-16T17:13:35.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.367+0000 7f1e51400640 1 -- 172.21.15.45:0/263988841 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1e4c103980 msgr2=0x7f1e4c196b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:35.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.367+0000 7f1e51400640 1 --2- 172.21.15.45:0/263988841 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1e4c103980 0x7f1e4c196b10 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f1e3800e9f0 tx=0x7f1e3800eeb0 comp rx=0 tx=0).stop 2024-04-16T17:13:35.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.367+0000 7f1e51400640 1 -- 172.21.15.45:0/263988841 shutdown_connections 2024-04-16T17:13:35.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.367+0000 7f1e51400640 1 --2- 172.21.15.45:0/263988841 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f1e300a3c30 0x7f1e300a60f0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:35.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.367+0000 7f1e51400640 1 --2- 172.21.15.45:0/263988841 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1e4c10c480 0x7f1e4c19ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:35.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.367+0000 7f1e51400640 1 --2- 172.21.15.45:0/263988841 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1e4c103980 0x7f1e4c196b10 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:35.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.367+0000 7f1e51400640 1 --2- 172.21.15.45:0/263988841 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1e4c103060 0x7f1e4c1965d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:35.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.367+0000 7f1e51400640 1 -- 172.21.15.45:0/263988841 >> 172.21.15.45:0/263988841 conn(0x7f1e4c0fc880 msgr2=0x7f1e4c0fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:13:35.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.367+0000 7f1e51400640 1 -- 172.21.15.45:0/263988841 shutdown_connections 2024-04-16T17:13:35.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:35.367+0000 7f1e51400640 1 -- 172.21.15.45:0/263988841 wait complete. 2024-04-16T17:13:35.804 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:13:35 smithi117 ceph-mon[36730]: from='client.? 172.21.15.45:0/3459318323' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:13:35.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:13:35 smithi104 ceph-mon[36457]: from='client.? 172.21.15.45:0/3459318323' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:13:36.289 INFO:tasks.workunit.client.0.smithi045.stdout: 1 113910 1.76 MB/sec execute 218 sec latency 19.442 ms 2024-04-16T17:13:36.384 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-16T17:13:36.385 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T17:13:36.385 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-16T17:13:36.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.609+0000 7f9a29000640 1 -- 172.21.15.45:0/1160010355 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9a24074cd0 msgr2=0x7f9a2410e220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:36.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.609+0000 7f9a29000640 1 --2- 172.21.15.45:0/1160010355 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9a24074cd0 0x7f9a2410e220 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f9a10009860 tx=0x7f9a1002bc10 comp rx=0 tx=0).stop 2024-04-16T17:13:36.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.609+0000 7f9a29000640 1 -- 172.21.15.45:0/1160010355 shutdown_connections 2024-04-16T17:13:36.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.609+0000 7f9a29000640 1 --2- 172.21.15.45:0/1160010355 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9a2410e760 0x7f9a24110b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:36.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.609+0000 7f9a29000640 1 --2- 172.21.15.45:0/1160010355 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9a24074cd0 0x7f9a2410e220 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:36.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.609+0000 7f9a29000640 1 --2- 172.21.15.45:0/1160010355 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9a240743b0 0x7f9a24074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:36.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.609+0000 7f9a29000640 1 -- 172.21.15.45:0/1160010355 >> 172.21.15.45:0/1160010355 conn(0x7f9a240fd9d0 msgr2=0x7f9a240ffdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:13:36.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.609+0000 7f9a29000640 1 -- 172.21.15.45:0/1160010355 shutdown_connections 2024-04-16T17:13:36.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.610+0000 7f9a29000640 1 -- 172.21.15.45:0/1160010355 wait complete. 2024-04-16T17:13:36.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.610+0000 7f9a29000640 1 Processor -- start 2024-04-16T17:13:36.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.611+0000 7f9a29000640 1 -- start start 2024-04-16T17:13:36.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.611+0000 7f9a29000640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9a240743b0 0x7f9a241a0680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:36.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.611+0000 7f9a29000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9a24074cd0 0x7f9a241a0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:36.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.611+0000 7f9a29000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9a2410e760 0x7f9a2419a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:36.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.611+0000 7f9a29000640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f9a24114470 con 0x7f9a24074cd0 2024-04-16T17:13:36.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.611+0000 7f9a29000640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f9a241142f0 con 0x7f9a240743b0 2024-04-16T17:13:36.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.611+0000 7f9a29000640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f9a241145f0 con 0x7f9a2410e760 2024-04-16T17:13:36.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.611+0000 7f9a23400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9a240743b0 0x7f9a241a0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:36.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.611+0000 7f9a22a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9a24074cd0 0x7f9a241a0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:36.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.611+0000 7f9a23400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9a240743b0 0x7f9a241a0680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:39614/0 (socket says 172.21.15.45:39614) 2024-04-16T17:13:36.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.611+0000 7f9a23400640 1 -- 172.21.15.45:0/2442387323 learned_addr learned my addr 172.21.15.45:0/2442387323 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:13:36.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.611+0000 7f9a23e00640 1 --2- 172.21.15.45:0/2442387323 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9a2410e760 0x7f9a2419a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:36.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.612+0000 7f9a22a00640 1 -- 172.21.15.45:0/2442387323 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9a2410e760 msgr2=0x7f9a2419a7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:36.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.612+0000 7f9a22a00640 1 --2- 172.21.15.45:0/2442387323 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9a2410e760 0x7f9a2419a7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:36.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.612+0000 7f9a22a00640 1 -- 172.21.15.45:0/2442387323 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9a240743b0 msgr2=0x7f9a241a0680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:36.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.612+0000 7f9a22a00640 1 --2- 172.21.15.45:0/2442387323 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9a240743b0 0x7f9a241a0680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:36.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.612+0000 7f9a22a00640 1 -- 172.21.15.45:0/2442387323 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a2419afc0 con 0x7f9a24074cd0 2024-04-16T17:13:36.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.612+0000 7f9a22a00640 1 --2- 172.21.15.45:0/2442387323 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9a24074cd0 0x7f9a241a0bc0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f9a1002c150 tx=0x7f9a100344c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:13:36.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.612+0000 7f9a1be00640 1 -- 172.21.15.45:0/2442387323 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a10041020 con 0x7f9a24074cd0 2024-04-16T17:13:36.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.612+0000 7f9a1be00640 1 -- 172.21.15.45:0/2442387323 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9a1002ca90 con 0x7f9a24074cd0 2024-04-16T17:13:36.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.612+0000 7f9a1be00640 1 -- 172.21.15.45:0/2442387323 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a10039040 con 0x7f9a24074cd0 2024-04-16T17:13:36.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.613+0000 7f9a29000640 1 -- 172.21.15.45:0/2442387323 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a2419b250 con 0x7f9a24074cd0 2024-04-16T17:13:36.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.613+0000 7f9a29000640 1 -- 172.21.15.45:0/2442387323 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a241a74d0 con 0x7f9a24074cd0 2024-04-16T17:13:36.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.616+0000 7f9a1be00640 1 -- 172.21.15.45:0/2442387323 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f9a10044020 con 0x7f9a24074cd0 2024-04-16T17:13:36.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.616+0000 7f9a2d3b3740 1 Processor -- start 2024-04-16T17:13:36.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.616+0000 7f9a2d3b3740 1 -- start start 2024-04-16T17:13:36.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.617+0000 7f9a2d3b3740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56320775ce90 0x56320775d270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:36.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.617+0000 7f9a2d3b3740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56320775d7b0 0x563207765e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:36.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.617+0000 7f9a2d3b3740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563207766390 0x563207768780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:36.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.617+0000 7f9a2d3b3740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5632076c1cb0 con 0x563207766390 2024-04-16T17:13:36.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.617+0000 7f9a2d3b3740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x563207668370 con 0x56320775d7b0 2024-04-16T17:13:36.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.617+0000 7f9a2d3b3740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5632076bdc70 con 0x56320775ce90 2024-04-16T17:13:36.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.617+0000 7f9a23e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56320775ce90 0x56320775d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:36.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.617+0000 7f9a23e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56320775ce90 0x56320775d270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:57370/0 (socket says 172.21.15.45:57370) 2024-04-16T17:13:36.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.617+0000 7f9a23e00640 1 -- 172.21.15.45:0/2474348441 learned_addr learned my addr 172.21.15.45:0/2474348441 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:13:36.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.617+0000 7f9a23400640 1 --2- 172.21.15.45:0/2474348441 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56320775d7b0 0x563207765e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:36.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.617+0000 7f9a22a00640 1 --2- 172.21.15.45:0/2474348441 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563207766390 0x563207768780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:36.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.618+0000 7f9a1be00640 1 --2- 172.21.15.45:0/2442387323 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f9a080a3d20 0x7f9a080a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:36.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.618+0000 7f9a23e00640 1 -- 172.21.15.45:0/2474348441 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56320775d7b0 msgr2=0x563207765e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:13:36.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.618+0000 7f9a23e00640 1 --2- 172.21.15.45:0/2474348441 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56320775d7b0 0x563207765e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:36.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.618+0000 7f9a23e00640 1 -- 172.21.15.45:0/2474348441 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563207766390 msgr2=0x563207768780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:13:36.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.618+0000 7f9a1be00640 1 -- 172.21.15.45:0/2442387323 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9496+0+0 (secure 0 0 0) 0x7f9a100f2180 con 0x7f9a24074cd0 2024-04-16T17:13:36.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.618+0000 7f9a23e00640 1 --2- 172.21.15.45:0/2474348441 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563207766390 0x563207768780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:36.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.618+0000 7f9a23e00640 1 -- 172.21.15.45:0/2474348441 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563207687910 con 0x56320775ce90 2024-04-16T17:13:36.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.618+0000 7f9a22a00640 1 --2- 172.21.15.45:0/2474348441 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563207766390 0x563207768780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:13:36.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.618+0000 7f9a23e00640 1 --2- 172.21.15.45:0/2474348441 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56320775ce90 0x56320775d270 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f9a1c009f50 tx=0x7f9a1c0118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=909a5d60ca6a00e9 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:13:36.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.619+0000 7f9a23e00640 1 --2- 172.21.15.45:0/2442387323 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f9a080a3d20 0x7f9a080a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:36.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.619+0000 7f9a18c00640 1 -- 172.21.15.45:0/2474348441 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a1c01e070 con 0x56320775ce90 2024-04-16T17:13:36.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.619+0000 7f9a18c00640 1 -- 172.21.15.45:0/2474348441 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9a1c003d70 con 0x56320775ce90 2024-04-16T17:13:36.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.619+0000 7f9a18c00640 1 -- 172.21.15.45:0/2474348441 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a1c00fca0 con 0x56320775ce90 2024-04-16T17:13:36.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.620+0000 7f9a2d3b3740 1 -- 172.21.15.45:0/2474348441 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56320775ce90 msgr2=0x56320775d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:36.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.620+0000 7f9a23e00640 1 --2- 172.21.15.45:0/2442387323 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f9a080a3d20 0x7f9a080a61e0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f9a1c004020 tx=0x7f9a1c00ff20 comp rx=0 tx=0).ready entity=mgr.25411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:13:36.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.620+0000 7f9a2d3b3740 1 --2- 172.21.15.45:0/2474348441 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56320775ce90 0x56320775d270 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f9a1c009f50 tx=0x7f9a1c0118d0 comp rx=0 tx=0).stop 2024-04-16T17:13:36.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.620+0000 7f9a2d3b3740 1 -- 172.21.15.45:0/2474348441 shutdown_connections 2024-04-16T17:13:36.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.620+0000 7f9a2d3b3740 1 --2- 172.21.15.45:0/2474348441 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563207766390 0x563207768780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:36.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.620+0000 7f9a2d3b3740 1 --2- 172.21.15.45:0/2474348441 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56320775d7b0 0x563207765e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:36.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.620+0000 7f9a2d3b3740 1 --2- 172.21.15.45:0/2474348441 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56320775ce90 0x56320775d270 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:36.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.620+0000 7f9a2d3b3740 1 -- 172.21.15.45:0/2474348441 >> 172.21.15.45:0/2474348441 conn(0x5632077575c0 msgr2=0x5632077579a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:13:36.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.620+0000 7f9a2d3b3740 1 -- 172.21.15.45:0/2474348441 shutdown_connections 2024-04-16T17:13:36.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.620+0000 7f9a2d3b3740 1 -- 172.21.15.45:0/2474348441 wait complete. 2024-04-16T17:13:36.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.621+0000 7f9a2d3b3740 1 -- start start 2024-04-16T17:13:36.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.621+0000 7f9a2d3b3740 1 Processor -- start 2024-04-16T17:13:36.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.622+0000 7f9a2d3b3740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56320775ce90 0x563207763410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:36.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.622+0000 7f9a2d3b3740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56320775d7b0 0x563207763950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:36.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.622+0000 7f9a2d3b3740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563207766390 0x563207763e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:36.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.622+0000 7f9a2d3b3740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x563207769180 con 0x563207766390 2024-04-16T17:13:36.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.622+0000 7f9a2d3b3740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x56320767b730 con 0x56320775d7b0 2024-04-16T17:13:36.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.622+0000 7f9a2d3b3740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x563207769300 con 0x56320775ce90 2024-04-16T17:13:36.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.622+0000 7f9a23400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56320775ce90 0x563207763410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:36.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.622+0000 7f9a2d3b3740 10 client.0 _ll_register_callbacks cb 0x56320764a1e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:13:36.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.622+0000 7f9a23400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56320775ce90 0x563207763410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:57378/0 (socket says 172.21.15.45:57378) 2024-04-16T17:13:36.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.622+0000 7f9a23400640 1 -- 172.21.15.45:0/1445094626 learned_addr learned my addr 172.21.15.45:0/1445094626 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:13:36.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.622+0000 7f9a22a00640 1 --2- 172.21.15.45:0/1445094626 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56320775d7b0 0x563207763950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:36.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.622+0000 7f9a23e00640 1 --2- 172.21.15.45:0/1445094626 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563207766390 0x563207763e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:36.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.622+0000 7f9a23400640 1 -- 172.21.15.45:0/1445094626 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56320775d7b0 msgr2=0x563207763950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:13:36.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.622+0000 7f9a23400640 1 --2- 172.21.15.45:0/1445094626 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56320775d7b0 0x563207763950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:36.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.622+0000 7f9a23400640 1 -- 172.21.15.45:0/1445094626 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563207766390 msgr2=0x563207763e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:13:36.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.622+0000 7f9a23400640 1 --2- 172.21.15.45:0/1445094626 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563207766390 0x563207763e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:36.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.622+0000 7f9a23400640 1 -- 172.21.15.45:0/1445094626 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56320749c7e0 con 0x56320775ce90 2024-04-16T17:13:36.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.623+0000 7f9a23400640 1 --2- 172.21.15.45:0/1445094626 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56320775ce90 0x563207763410 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f9a14012840 tx=0x7f9a14012c40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ed63a1739a6e8b86 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:13:36.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.623+0000 7f9a0d600640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-04-16T17:13:36.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.623+0000 7f9a0d600640 1 -- 172.21.15.45:0/1445094626 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a1400f920 con 0x56320775ce90 2024-04-16T17:13:36.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.623+0000 7f9a2d3b3740 1 -- 172.21.15.45:0/1445094626 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9a1c003d70 con 0x56320775ce90 2024-04-16T17:13:36.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.623+0000 7f9a0d600640 1 -- 172.21.15.45:0/1445094626 <== mon.1 v2:172.21.15.117:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9496+0+0 (secure 0 0 0) 0x7f9a1400fac0 con 0x56320775ce90 2024-04-16T17:13:36.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.623+0000 7f9a0d600640 1 -- 172.21.15.45:0/1445094626 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f9a1400fcc0 con 0x56320775ce90 2024-04-16T17:13:36.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.623+0000 7f9a2d3b3740 10 client.27466 fetch_fsmap learned FSMap version 38 2024-04-16T17:13:36.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.623+0000 7f9a2d3b3740 1 -- 172.21.15.45:0/1445094626 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x5632074fcf70 con 0x56320775ce90 2024-04-16T17:13:36.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.624+0000 7f9a0d600640 1 -- 172.21.15.45:0/1445094626 <== mon.1 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9a1c003d70 con 0x56320775ce90 2024-04-16T17:13:36.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.624+0000 7f9a0d600640 1 -- 172.21.15.45:0/1445094626 <== mon.1 v2:172.21.15.117:3300/0 5 ==== fsmap(e 38) ==== 8354+0+0 (secure 0 0 0) 0x7f9a1401c420 con 0x56320775ce90 2024-04-16T17:13:36.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.624+0000 7f9a2d3b3740 10 client.27466 fetch_fsmap finished waiting for FSMap version 38 2024-04-16T17:13:36.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.624+0000 7f9a2d3b3740 10 client.27466 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:13:36.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.624+0000 7f9a2d3b3740 20 client.27466 populate_metadata read hostname 'smithi045' 2024-04-16T17:13:36.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.624+0000 7f9a2d3b3740 1 --2- 172.21.15.45:0/1445094626 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x5632077877a0 0x563207789c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:36.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.624+0000 7f9a2d3b3740 4 client.27466 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:13:36.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.624+0000 7f9a2d3b3740 1 -- 172.21.15.45:0/1445094626 --> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5632076cf6d0 con 0x5632077877a0 2024-04-16T17:13:36.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.624+0000 7f9a22a00640 1 --2- 172.21.15.45:0/1445094626 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x5632077877a0 0x563207789c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:36.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.625+0000 7f9a22a00640 1 --2- 172.21.15.45:0/1445094626 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x5632077877a0 0x563207789c10 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4fb21c34adec5d05 server_cookie=da5e31fe52d31f8c in_seq=0 out_seq=0 2024-04-16T17:13:36.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.625+0000 7f9a0d600640 10 client.27466 ms_handle_connect on v2:172.21.15.104:6838/3660070592 2024-04-16T17:13:36.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.626+0000 7f9a0d600640 1 -- 172.21.15.45:0/1445094626 <== mds.0 v2:172.21.15.104:6838/3660070592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5632076cf6d0 con 0x5632077877a0 2024-04-16T17:13:36.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.626+0000 7f9a0d600640 10 client.27466 handle_command_reply: tid=0 2024-04-16T17:13:36.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.626+0000 7f9a2d3b3740 1 client.27466 shutdown 2024-04-16T17:13:36.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.626+0000 7f9a2d3b3740 1 -- 172.21.15.45:0/1445094626 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56320775ce90 msgr2=0x563207763410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:36.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.626+0000 7f9a2d3b3740 1 --2- 172.21.15.45:0/1445094626 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56320775ce90 0x563207763410 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f9a14012840 tx=0x7f9a14012c40 comp rx=0 tx=0).stop 2024-04-16T17:13:36.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.626+0000 7f9a2d3b3740 1 -- 172.21.15.45:0/1445094626 shutdown_connections 2024-04-16T17:13:36.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.626+0000 7f9a2d3b3740 1 --2- 172.21.15.45:0/1445094626 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563207766390 0x563207763e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:36.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.626+0000 7f9a2d3b3740 1 --2- 172.21.15.45:0/1445094626 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56320775d7b0 0x563207763950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:36.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.626+0000 7f9a2d3b3740 1 --2- 172.21.15.45:0/1445094626 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x5632077877a0 0x563207789c10 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:36.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.626+0000 7f9a2d3b3740 1 --2- 172.21.15.45:0/1445094626 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56320775ce90 0x563207763410 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:36.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.626+0000 7f9a0d600640 0 client.27466 ms_handle_reset on v2:172.21.15.104:6838/3660070592 2024-04-16T17:13:36.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.626+0000 7f9a2d3b3740 1 -- 172.21.15.45:0/1445094626 >> 172.21.15.45:0/1445094626 conn(0x5632077575c0 msgr2=0x563207759310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:13:36.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.626+0000 7f9a2d3b3740 1 -- 172.21.15.45:0/1445094626 shutdown_connections 2024-04-16T17:13:36.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.626+0000 7f9a2d3b3740 1 -- 172.21.15.45:0/1445094626 wait complete. 2024-04-16T17:13:36.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.626+0000 7f9a2d3b3740 20 client.27466 trim_cache size 0 max 16384 2024-04-16T17:13:36.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.648+0000 7f9a2d3b3740 1 Processor -- start 2024-04-16T17:13:36.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.648+0000 7f9a2d3b3740 1 -- start start 2024-04-16T17:13:36.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.648+0000 7f9a2d3b3740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56320775ce90 0x563207765b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:36.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.648+0000 7f9a2d3b3740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56320775d7b0 0x563207772040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:36.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.648+0000 7f9a2d3b3740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563207766390 0x56320778f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:36.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.648+0000 7f9a2d3b3740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f9a241ab750 con 0x56320775d7b0 2024-04-16T17:13:36.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.648+0000 7f9a2d3b3740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f9a241a8fc0 con 0x563207766390 2024-04-16T17:13:36.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.648+0000 7f9a2d3b3740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f9a240739b0 con 0x56320775ce90 2024-04-16T17:13:36.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.649+0000 7f9a23e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563207766390 0x56320778f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:36.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.649+0000 7f9a23400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56320775ce90 0x563207765b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:36.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.649+0000 7f9a23e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563207766390 0x56320778f4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:39636/0 (socket says 172.21.15.45:39636) 2024-04-16T17:13:36.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.649+0000 7f9a23400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56320775ce90 0x563207765b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:57388/0 (socket says 172.21.15.45:57388) 2024-04-16T17:13:36.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.649+0000 7f9a23e00640 1 -- 172.21.15.45:0/784947193 learned_addr learned my addr 172.21.15.45:0/784947193 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:13:36.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.649+0000 7f9a22a00640 1 --2- 172.21.15.45:0/784947193 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56320775d7b0 0x563207772040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:36.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.649+0000 7f9a23e00640 1 -- 172.21.15.45:0/784947193 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56320775ce90 msgr2=0x563207765b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:13:36.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.649+0000 7f9a23e00640 1 --2- 172.21.15.45:0/784947193 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56320775ce90 0x563207765b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:36.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.649+0000 7f9a23e00640 1 -- 172.21.15.45:0/784947193 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56320775d7b0 msgr2=0x563207772040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:13:36.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.649+0000 7f9a23e00640 1 --2- 172.21.15.45:0/784947193 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56320775d7b0 0x563207772040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:36.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.649+0000 7f9a23e00640 1 -- 172.21.15.45:0/784947193 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56320775d5e0 con 0x563207766390 2024-04-16T17:13:36.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.649+0000 7f9a23400640 1 --2- 172.21.15.45:0/784947193 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56320775ce90 0x563207765b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:13:36.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.649+0000 7f9a23e00640 1 --2- 172.21.15.45:0/784947193 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563207766390 0x56320778f4e0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f9a1c01c040 tx=0x7f9a1c027000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fa250e4fc50ce5d9 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:13:36.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.650+0000 7f9a0fe00640 1 -- 172.21.15.45:0/784947193 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a1c036070 con 0x563207766390 2024-04-16T17:13:36.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.650+0000 7f9a0fe00640 1 -- 172.21.15.45:0/784947193 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9a1c02a050 con 0x563207766390 2024-04-16T17:13:36.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.650+0000 7f9a0fe00640 1 -- 172.21.15.45:0/784947193 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a1c01e040 con 0x563207766390 2024-04-16T17:13:36.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.651+0000 7f9a2d3b3740 1 -- 172.21.15.45:0/784947193 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563207766390 msgr2=0x56320778f4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:36.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.651+0000 7f9a2d3b3740 1 --2- 172.21.15.45:0/784947193 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563207766390 0x56320778f4e0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f9a1c01c040 tx=0x7f9a1c027000 comp rx=0 tx=0).stop 2024-04-16T17:13:36.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.651+0000 7f9a2d3b3740 1 -- 172.21.15.45:0/784947193 shutdown_connections 2024-04-16T17:13:36.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.651+0000 7f9a2d3b3740 1 --2- 172.21.15.45:0/784947193 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563207766390 0x56320778f4e0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:36.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.651+0000 7f9a2d3b3740 1 --2- 172.21.15.45:0/784947193 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56320775d7b0 0x563207772040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:36.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.651+0000 7f9a2d3b3740 1 --2- 172.21.15.45:0/784947193 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56320775ce90 0x563207765b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:36.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.651+0000 7f9a2d3b3740 1 -- 172.21.15.45:0/784947193 >> 172.21.15.45:0/784947193 conn(0x5632077575c0 msgr2=0x563207759280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:13:36.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.651+0000 7f9a2d3b3740 1 -- 172.21.15.45:0/784947193 shutdown_connections 2024-04-16T17:13:36.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.651+0000 7f9a2d3b3740 1 -- 172.21.15.45:0/784947193 wait complete. 2024-04-16T17:13:36.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.652+0000 7f9a2d3b3740 1 -- start start 2024-04-16T17:13:36.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.652+0000 7f9a2d3b3740 1 Processor -- start 2024-04-16T17:13:36.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.652+0000 7f9a2d3b3740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56320775ce90 0x56320776e550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:36.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.652+0000 7f9a2d3b3740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56320775d7b0 0x56320776ea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:36.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.652+0000 7f9a2d3b3740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563207766390 0x56320776b520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:36.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.652+0000 7f9a2d3b3740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f9a24074000 con 0x56320775ce90 2024-04-16T17:13:36.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.652+0000 7f9a2d3b3740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f9a241a88f0 con 0x563207766390 2024-04-16T17:13:36.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.652+0000 7f9a2d3b3740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f9a241a7b50 con 0x56320775d7b0 2024-04-16T17:13:36.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.653+0000 7f9a2d3b3740 10 client.0 _ll_register_callbacks cb 0x7f9a14012450 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:13:36.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.653+0000 7f9a23400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56320775ce90 0x56320776e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:36.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.653+0000 7f9a22a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56320775d7b0 0x56320776ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:36.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.653+0000 7f9a23400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56320775ce90 0x56320776e550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56872/0 (socket says 172.21.15.45:56872) 2024-04-16T17:13:36.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.653+0000 7f9a23400640 1 -- 172.21.15.45:0/2874991192 learned_addr learned my addr 172.21.15.45:0/2874991192 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:13:36.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.653+0000 7f9a23e00640 1 --2- 172.21.15.45:0/2874991192 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563207766390 0x56320776b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:36.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.653+0000 7f9a23400640 1 -- 172.21.15.45:0/2874991192 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56320775d7b0 msgr2=0x56320776ea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:13:36.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.653+0000 7f9a23400640 1 --2- 172.21.15.45:0/2874991192 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56320775d7b0 0x56320776ea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:36.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.653+0000 7f9a23400640 1 -- 172.21.15.45:0/2874991192 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563207766390 msgr2=0x56320776b520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:13:36.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.653+0000 7f9a23400640 1 --2- 172.21.15.45:0/2874991192 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563207766390 0x56320776b520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:36.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.653+0000 7f9a23400640 1 -- 172.21.15.45:0/2874991192 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563207394170 con 0x56320775ce90 2024-04-16T17:13:36.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.653+0000 7f9a23e00640 1 --2- 172.21.15.45:0/2874991192 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563207766390 0x56320776b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:13:36.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.654+0000 7f9a23400640 1 --2- 172.21.15.45:0/2874991192 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56320775ce90 0x56320776e550 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f9a14000c00 tx=0x7f9a1401cdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f77c3c721ce7292 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:13:36.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.654+0000 7f9a0d600640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-16T17:13:36.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.654+0000 7f9a0d600640 1 -- 172.21.15.45:0/2874991192 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a14030060 con 0x56320775ce90 2024-04-16T17:13:36.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.654+0000 7f9a0d600640 1 -- 172.21.15.45:0/2874991192 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9496+0+0 (secure 0 0 0) 0x7f9a1401c630 con 0x56320775ce90 2024-04-16T17:13:36.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.654+0000 7f9a2d3b3740 1 -- 172.21.15.45:0/2874991192 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9a1c02a050 con 0x56320775ce90 2024-04-16T17:13:36.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.654+0000 7f9a0d600640 1 -- 172.21.15.45:0/2874991192 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f9a14005d10 con 0x56320775ce90 2024-04-16T17:13:36.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.654+0000 7f9a2d3b3740 10 client.17367 fetch_fsmap learned FSMap version 38 2024-04-16T17:13:36.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.654+0000 7f9a2d3b3740 1 -- 172.21.15.45:0/2874991192 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x5632077ee700 con 0x56320775ce90 2024-04-16T17:13:36.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.654+0000 7f9a0d600640 1 -- 172.21.15.45:0/2874991192 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9a1c02a050 con 0x56320775ce90 2024-04-16T17:13:36.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.655+0000 7f9a0d600640 1 -- 172.21.15.45:0/2874991192 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 38) ==== 8354+0+0 (secure 0 0 0) 0x7f9a14003c40 con 0x56320775ce90 2024-04-16T17:13:36.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.655+0000 7f9a2d3b3740 10 client.17367 fetch_fsmap finished waiting for FSMap version 38 2024-04-16T17:13:36.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.655+0000 7f9a2d3b3740 10 client.17367 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:13:36.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.655+0000 7f9a2d3b3740 20 client.17367 populate_metadata read hostname 'smithi045' 2024-04-16T17:13:36.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.655+0000 7f9a2d3b3740 1 --2- 172.21.15.45:0/2874991192 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x5632077eee20 0x5632077f1240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:36.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.655+0000 7f9a2d3b3740 4 client.17367 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-16T17:13:36.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.655+0000 7f9a2d3b3740 1 -- 172.21.15.45:0/2874991192 --> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5632076c9540 con 0x5632077eee20 2024-04-16T17:13:36.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.655+0000 7f9a22a00640 1 --2- 172.21.15.45:0/2874991192 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x5632077eee20 0x5632077f1240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:36.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.655+0000 7f9a22a00640 1 --2- 172.21.15.45:0/2874991192 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x5632077eee20 0x5632077f1240 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5fe6ae4e43e1f729 server_cookie=58f33e04af0dde49 in_seq=0 out_seq=0 2024-04-16T17:13:36.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.655+0000 7f9a0d600640 10 client.17367 ms_handle_connect on v2:172.21.15.104:6838/3660070592 2024-04-16T17:13:36.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.656+0000 7f9a0d600640 1 -- 172.21.15.45:0/2874991192 <== mds.0 v2:172.21.15.104:6838/3660070592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5632076c9540 con 0x5632077eee20 2024-04-16T17:13:36.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.656+0000 7f9a0d600640 10 client.17367 handle_command_reply: tid=0 2024-04-16T17:13:36.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.656+0000 7f9a2d3b3740 1 client.17367 shutdown 2024-04-16T17:13:36.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.657+0000 7f9a2d3b3740 1 -- 172.21.15.45:0/2874991192 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56320775ce90 msgr2=0x56320776e550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:36.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.657+0000 7f9a2d3b3740 1 --2- 172.21.15.45:0/2874991192 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56320775ce90 0x56320776e550 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f9a14000c00 tx=0x7f9a1401cdb0 comp rx=0 tx=0).stop 2024-04-16T17:13:36.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.657+0000 7f9a2d3b3740 1 -- 172.21.15.45:0/2874991192 shutdown_connections 2024-04-16T17:13:36.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.657+0000 7f9a2d3b3740 1 --2- 172.21.15.45:0/2874991192 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x5632077eee20 0x5632077f1240 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:36.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.657+0000 7f9a2d3b3740 1 --2- 172.21.15.45:0/2874991192 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56320775d7b0 0x56320776ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:36.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.657+0000 7f9a2d3b3740 1 --2- 172.21.15.45:0/2874991192 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563207766390 0x56320776b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:36.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.657+0000 7f9a2d3b3740 1 --2- 172.21.15.45:0/2874991192 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56320775ce90 0x56320776e550 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:36.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.657+0000 7f9a0d600640 0 client.17367 ms_handle_reset on v2:172.21.15.104:6838/3660070592 2024-04-16T17:13:36.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.657+0000 7f9a2d3b3740 1 -- 172.21.15.45:0/2874991192 >> 172.21.15.45:0/2874991192 conn(0x5632077575c0 msgr2=0x563207755cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:13:36.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.657+0000 7f9a2d3b3740 1 -- 172.21.15.45:0/2874991192 shutdown_connections 2024-04-16T17:13:36.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.657+0000 7f9a2d3b3740 1 -- 172.21.15.45:0/2874991192 wait complete. 2024-04-16T17:13:36.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.657+0000 7f9a2d3b3740 20 client.17367 trim_cache size 0 max 16384 2024-04-16T17:13:36.660 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-16T17:13:36.660 INFO:teuthology.orchestra.run.smithi045.stdout: "return_code": 0, 2024-04-16T17:13:36.660 INFO:teuthology.orchestra.run.smithi045.stdout: "scrub_tag": "93f224db-eff9-48f1-8362-810ce7ba60ef", 2024-04-16T17:13:36.660 INFO:teuthology.orchestra.run.smithi045.stdout: "mode": "asynchronous" 2024-04-16T17:13:36.660 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-16T17:13:36.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.658+0000 7f9a29000640 1 -- 172.21.15.45:0/2442387323 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f9a080a3d20 msgr2=0x7f9a080a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:36.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.658+0000 7f9a29000640 1 --2- 172.21.15.45:0/2442387323 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f9a080a3d20 0x7f9a080a61e0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f9a1c004020 tx=0x7f9a1c00ff20 comp rx=0 tx=0).stop 2024-04-16T17:13:36.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.658+0000 7f9a29000640 1 -- 172.21.15.45:0/2442387323 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9a24074cd0 msgr2=0x7f9a241a0bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:36.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.658+0000 7f9a29000640 1 --2- 172.21.15.45:0/2442387323 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9a24074cd0 0x7f9a241a0bc0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f9a1002c150 tx=0x7f9a100344c0 comp rx=0 tx=0).stop 2024-04-16T17:13:36.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.658+0000 7f9a29000640 1 -- 172.21.15.45:0/2442387323 shutdown_connections 2024-04-16T17:13:36.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.658+0000 7f9a29000640 1 --2- 172.21.15.45:0/2442387323 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f9a080a3d20 0x7f9a080a61e0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:36.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.658+0000 7f9a29000640 1 --2- 172.21.15.45:0/2442387323 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9a2410e760 0x7f9a2419a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:36.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.658+0000 7f9a29000640 1 --2- 172.21.15.45:0/2442387323 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9a24074cd0 0x7f9a241a0bc0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:36.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.658+0000 7f9a29000640 1 --2- 172.21.15.45:0/2442387323 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9a240743b0 0x7f9a241a0680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:36.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.658+0000 7f9a29000640 1 -- 172.21.15.45:0/2442387323 >> 172.21.15.45:0/2442387323 conn(0x7f9a240fd9d0 msgr2=0x7f9a240fe740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:13:36.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.658+0000 7f9a29000640 1 -- 172.21.15.45:0/2442387323 shutdown_connections 2024-04-16T17:13:36.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.658+0000 7f9a29000640 1 -- 172.21.15.45:0/2442387323 wait complete. 2024-04-16T17:13:36.683 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-16T17:13:36.683 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-16T17:13:36.736 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:13:34 smithi045 ceph-mon[31676]: pgmap v209: 129 pgs: 129 active+clean; 59 MiB data, 735 MiB used, 1.0 TiB / 1.0 TiB avail; 782 KiB/s rd, 584 KiB/s wr, 63 op/s 2024-04-16T17:13:36.736 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:13:35 smithi045 ceph-mon[31676]: from='client.? 172.21.15.45:0/3459318323' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:13:36.804 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:13:36 smithi117 ceph-mon[36730]: pgmap v210: 129 pgs: 129 active+clean; 59 MiB data, 735 MiB used, 1.0 TiB / 1.0 TiB avail; 782 KiB/s rd, 584 KiB/s wr, 63 op/s 2024-04-16T17:13:36.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.878+0000 7f50da400640 1 -- 172.21.15.45:0/4288961288 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f50d41060d0 msgr2=0x7f50d410dca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:36.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.878+0000 7f50da400640 1 --2- 172.21.15.45:0/4288961288 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f50d41060d0 0x7f50d410dca0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f50cc009be0 tx=0x7f50cc02bef0 comp rx=0 tx=0).stop 2024-04-16T17:13:36.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.878+0000 7f50da400640 1 -- 172.21.15.45:0/4288961288 shutdown_connections 2024-04-16T17:13:36.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.878+0000 7f50da400640 1 --2- 172.21.15.45:0/4288961288 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f50d41060d0 0x7f50d410dca0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:36.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.878+0000 7f50da400640 1 --2- 172.21.15.45:0/4288961288 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f50d41019b0 0x7f50d41059f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:36.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.878+0000 7f50da400640 1 --2- 172.21.15.45:0/4288961288 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f50d4101000 0x7f50d41013e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:36.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.878+0000 7f50da400640 1 -- 172.21.15.45:0/4288961288 >> 172.21.15.45:0/4288961288 conn(0x7f50d40fc880 msgr2=0x7f50d40feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:13:36.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.878+0000 7f50da400640 1 -- 172.21.15.45:0/4288961288 shutdown_connections 2024-04-16T17:13:36.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.878+0000 7f50da400640 1 -- 172.21.15.45:0/4288961288 wait complete. 2024-04-16T17:13:36.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.879+0000 7f50da400640 1 Processor -- start 2024-04-16T17:13:36.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.879+0000 7f50da400640 1 -- start start 2024-04-16T17:13:36.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.879+0000 7f50da400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f50d4101000 0x7f50d4198720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:36.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.879+0000 7f50da400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f50d41060d0 0x7f50d4198c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:36.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.879+0000 7f50da400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f50d419cfe0 0x7f50d419d490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:36.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.879+0000 7f50da400640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f50d4111190 con 0x7f50d4101000 2024-04-16T17:13:36.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.879+0000 7f50da400640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f50d4111010 con 0x7f50d419cfe0 2024-04-16T17:13:36.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.879+0000 7f50da400640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f50d4111310 con 0x7f50d41060d0 2024-04-16T17:13:36.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.880+0000 7f50d9a00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f50d419cfe0 0x7f50d419d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:36.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.880+0000 7f50d9000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f50d4101000 0x7f50d4198720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:36.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.880+0000 7f50d9a00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f50d419cfe0 0x7f50d419d490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:39670/0 (socket says 172.21.15.45:39670) 2024-04-16T17:13:36.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.880+0000 7f50d9a00640 1 -- 172.21.15.45:0/3509951936 learned_addr learned my addr 172.21.15.45:0/3509951936 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:13:36.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.880+0000 7f50d3e00640 1 --2- 172.21.15.45:0/3509951936 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f50d41060d0 0x7f50d4198c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:36.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.880+0000 7f50d9a00640 1 -- 172.21.15.45:0/3509951936 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f50d41060d0 msgr2=0x7f50d4198c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:36.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.880+0000 7f50d9a00640 1 --2- 172.21.15.45:0/3509951936 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f50d41060d0 0x7f50d4198c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:36.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.880+0000 7f50d9a00640 1 -- 172.21.15.45:0/3509951936 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f50d4101000 msgr2=0x7f50d4198720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:36.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.880+0000 7f50d9a00640 1 --2- 172.21.15.45:0/3509951936 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f50d4101000 0x7f50d4198720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:36.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.880+0000 7f50d9a00640 1 -- 172.21.15.45:0/3509951936 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50d419dad0 con 0x7f50d419cfe0 2024-04-16T17:13:36.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.880+0000 7f50d9000640 1 --2- 172.21.15.45:0/3509951936 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f50d4101000 0x7f50d4198720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:13:36.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.881+0000 7f50d9a00640 1 --2- 172.21.15.45:0/3509951936 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f50d419cfe0 0x7f50d419d490 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f50cc0096f0 tx=0x7f50cc0336b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:13:36.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.881+0000 7f50d1600640 1 -- 172.21.15.45:0/3509951936 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50cc043020 con 0x7f50d419cfe0 2024-04-16T17:13:36.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.881+0000 7f50d1600640 1 -- 172.21.15.45:0/3509951936 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f50cc0338a0 con 0x7f50d419cfe0 2024-04-16T17:13:36.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.881+0000 7f50d1600640 1 -- 172.21.15.45:0/3509951936 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50cc03e6b0 con 0x7f50d419cfe0 2024-04-16T17:13:36.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.881+0000 7f50da400640 1 -- 172.21.15.45:0/3509951936 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50d419dd60 con 0x7f50d419cfe0 2024-04-16T17:13:36.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.881+0000 7f50da400640 1 -- 172.21.15.45:0/3509951936 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50d41a5580 con 0x7f50d419cfe0 2024-04-16T17:13:36.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.884+0000 7f50de79e740 1 Processor -- start 2024-04-16T17:13:36.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.885+0000 7f50d1600640 1 -- 172.21.15.45:0/3509951936 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f50cc046020 con 0x7f50d419cfe0 2024-04-16T17:13:36.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.885+0000 7f50de79e740 1 -- start start 2024-04-16T17:13:36.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.885+0000 7f50de79e740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5651854f67a0 0x5651854f6b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:36.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.885+0000 7f50de79e740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5651854f70c0 0x5651854ff760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:36.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.886+0000 7f50de79e740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5651854ffca0 0x565185502090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:36.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.886+0000 7f50de79e740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x565185458120 con 0x5651854ffca0 2024-04-16T17:13:36.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.886+0000 7f50de79e740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5651853fe340 con 0x5651854f70c0 2024-04-16T17:13:36.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.886+0000 7f50de79e740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5651854540e0 con 0x5651854f67a0 2024-04-16T17:13:36.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.886+0000 7f50d9000640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5651854f70c0 0x5651854ff760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:36.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.886+0000 7f50d3e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5651854f67a0 0x5651854f6b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:36.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.886+0000 7f50d9000640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5651854f70c0 0x5651854ff760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:39672/0 (socket says 172.21.15.45:39672) 2024-04-16T17:13:36.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.886+0000 7f50d9000640 1 -- 172.21.15.45:0/2167397484 learned_addr learned my addr 172.21.15.45:0/2167397484 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:13:36.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.886+0000 7f50d3e00640 1 --2- 172.21.15.45:0/2167397484 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5651854ffca0 0x565185502090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:36.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.886+0000 7f50d9000640 1 -- 172.21.15.45:0/2167397484 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5651854f67a0 msgr2=0x5651854f6b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:13:36.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.886+0000 7f50d9000640 1 --2- 172.21.15.45:0/2167397484 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5651854f67a0 0x5651854f6b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:36.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.886+0000 7f50d9000640 1 -- 172.21.15.45:0/2167397484 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5651854ffca0 msgr2=0x565185502090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:13:36.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.886+0000 7f50d9000640 1 --2- 172.21.15.45:0/2167397484 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5651854ffca0 0x565185502090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:36.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.886+0000 7f50d9000640 1 -- 172.21.15.45:0/2167397484 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5651852347d0 con 0x5651854f70c0 2024-04-16T17:13:36.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.886+0000 7f50d3e00640 1 --2- 172.21.15.45:0/2167397484 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5651854ffca0 0x565185502090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:13:36.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.887+0000 7f50d1600640 1 --2- 172.21.15.45:0/3509951936 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f50bc0a3d20 0x7f50bc0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:36.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.887+0000 7f50d9000640 1 --2- 172.21.15.45:0/2167397484 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5651854f70c0 0x5651854ff760 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f50c80115c0 tx=0x7f50c8011a80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=848085f311b464b3 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:13:36.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.887+0000 7f50d1600640 1 -- 172.21.15.45:0/3509951936 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9496+0+0 (secure 0 0 0) 0x7f50cc0f2fe0 con 0x7f50d419cfe0 2024-04-16T17:13:36.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.887+0000 7f50d3e00640 1 --2- 172.21.15.45:0/3509951936 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f50bc0a3d20 0x7f50bc0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:36.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.887+0000 7f50c6000640 1 -- 172.21.15.45:0/2167397484 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50c800f640 con 0x5651854f70c0 2024-04-16T17:13:36.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.887+0000 7f50c6000640 1 -- 172.21.15.45:0/2167397484 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f50c800fcc0 con 0x5651854f70c0 2024-04-16T17:13:36.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.887+0000 7f50c6000640 1 -- 172.21.15.45:0/2167397484 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50c8019dc0 con 0x5651854f70c0 2024-04-16T17:13:36.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.888+0000 7f50de79e740 1 -- 172.21.15.45:0/2167397484 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5651854f70c0 msgr2=0x5651854ff760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:36.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.888+0000 7f50de79e740 1 --2- 172.21.15.45:0/2167397484 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5651854f70c0 0x5651854ff760 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f50c80115c0 tx=0x7f50c8011a80 comp rx=0 tx=0).stop 2024-04-16T17:13:36.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.888+0000 7f50d3e00640 1 --2- 172.21.15.45:0/3509951936 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f50bc0a3d20 0x7f50bc0a61e0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f50c0009fd0 tx=0x7f50c0009260 comp rx=0 tx=0).ready entity=mgr.25411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:13:36.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.888+0000 7f50de79e740 1 -- 172.21.15.45:0/2167397484 shutdown_connections 2024-04-16T17:13:36.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.888+0000 7f50de79e740 1 --2- 172.21.15.45:0/2167397484 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5651854ffca0 0x565185502090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:36.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.888+0000 7f50de79e740 1 --2- 172.21.15.45:0/2167397484 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5651854f70c0 0x5651854ff760 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:36.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.888+0000 7f50de79e740 1 --2- 172.21.15.45:0/2167397484 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5651854f67a0 0x5651854f6b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:36.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.888+0000 7f50de79e740 1 -- 172.21.15.45:0/2167397484 >> 172.21.15.45:0/2167397484 conn(0x5651854f0ed0 msgr2=0x5651854f12b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:13:36.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.888+0000 7f50de79e740 1 -- 172.21.15.45:0/2167397484 shutdown_connections 2024-04-16T17:13:36.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.888+0000 7f50de79e740 1 -- 172.21.15.45:0/2167397484 wait complete. 2024-04-16T17:13:36.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.888+0000 7f50de79e740 1 -- start start 2024-04-16T17:13:36.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.889+0000 7f50de79e740 1 Processor -- start 2024-04-16T17:13:36.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.889+0000 7f50de79e740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5651854f67a0 0x5651854fcd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:36.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.889+0000 7f50de79e740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5651854f70c0 0x5651854fd260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:36.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.889+0000 7f50de79e740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5651854ffca0 0x5651854fd7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:36.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.889+0000 7f50de79e740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x565185502a90 con 0x5651854f70c0 2024-04-16T17:13:36.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.889+0000 7f50de79e740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x565185411710 con 0x5651854ffca0 2024-04-16T17:13:36.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.889+0000 7f50de79e740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x565185502c10 con 0x5651854f67a0 2024-04-16T17:13:36.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.889+0000 7f50d9000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5651854f67a0 0x5651854fcd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:36.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.889+0000 7f50de79e740 10 client.0 _ll_register_callbacks cb 0x56518540caf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:13:36.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.889+0000 7f50d9000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5651854f67a0 0x5651854fcd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:57448/0 (socket says 172.21.15.45:57448) 2024-04-16T17:13:36.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.889+0000 7f50d9000640 1 -- 172.21.15.45:0/4199851838 learned_addr learned my addr 172.21.15.45:0/4199851838 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:13:36.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.889+0000 7f50d3e00640 1 --2- 172.21.15.45:0/4199851838 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5651854f70c0 0x5651854fd260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:36.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.889+0000 7f50d9a00640 1 --2- 172.21.15.45:0/4199851838 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5651854ffca0 0x5651854fd7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:36.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.890+0000 7f50d9000640 1 -- 172.21.15.45:0/4199851838 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5651854ffca0 msgr2=0x5651854fd7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:13:36.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.890+0000 7f50d9000640 1 --2- 172.21.15.45:0/4199851838 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5651854ffca0 0x5651854fd7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:36.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.890+0000 7f50d9000640 1 -- 172.21.15.45:0/4199851838 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5651854f70c0 msgr2=0x5651854fd260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:13:36.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.890+0000 7f50d9000640 1 --2- 172.21.15.45:0/4199851838 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5651854f70c0 0x5651854fd260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:36.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.890+0000 7f50d9000640 1 -- 172.21.15.45:0/4199851838 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56518516a3f0 con 0x5651854f67a0 2024-04-16T17:13:36.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.890+0000 7f50d9a00640 1 --2- 172.21.15.45:0/4199851838 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5651854ffca0 0x5651854fd7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:13:36.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.890+0000 7f50d9000640 1 --2- 172.21.15.45:0/4199851838 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5651854f67a0 0x5651854fcd20 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f50c8019fc0 tx=0x7f50c8005d70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6d46c75ece7cf6cc server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:13:36.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.890+0000 7f50baa00640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-04-16T17:13:36.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.890+0000 7f50baa00640 1 -- 172.21.15.45:0/4199851838 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50c8026020 con 0x5651854f67a0 2024-04-16T17:13:36.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.890+0000 7f50de79e740 1 -- 172.21.15.45:0/4199851838 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f50c800fcc0 con 0x5651854f67a0 2024-04-16T17:13:36.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.890+0000 7f50baa00640 1 -- 172.21.15.45:0/4199851838 <== mon.1 v2:172.21.15.117:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9496+0+0 (secure 0 0 0) 0x7f50c8004040 con 0x5651854f67a0 2024-04-16T17:13:36.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.891+0000 7f50baa00640 1 -- 172.21.15.45:0/4199851838 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f50c8031050 con 0x5651854f67a0 2024-04-16T17:13:36.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.891+0000 7f50de79e740 10 client.27484 fetch_fsmap learned FSMap version 38 2024-04-16T17:13:36.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.891+0000 7f50de79e740 1 -- 172.21.15.45:0/4199851838 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x56518512a0f0 con 0x5651854f67a0 2024-04-16T17:13:36.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.891+0000 7f50baa00640 1 -- 172.21.15.45:0/4199851838 <== mon.1 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f50c800fcc0 con 0x5651854f67a0 2024-04-16T17:13:36.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.891+0000 7f50baa00640 1 -- 172.21.15.45:0/4199851838 <== mon.1 v2:172.21.15.117:3300/0 5 ==== fsmap(e 38) ==== 8354+0+0 (secure 0 0 0) 0x7f50c8003780 con 0x5651854f67a0 2024-04-16T17:13:36.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.891+0000 7f50de79e740 10 client.27484 fetch_fsmap finished waiting for FSMap version 38 2024-04-16T17:13:36.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.891+0000 7f50de79e740 10 client.27484 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:13:36.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.891+0000 7f50de79e740 20 client.27484 populate_metadata read hostname 'smithi045' 2024-04-16T17:13:36.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.891+0000 7f50de79e740 1 --2- 172.21.15.45:0/4199851838 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x565185510040 0x565185512430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:36.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.891+0000 7f50de79e740 4 client.27484 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:13:36.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.891+0000 7f50de79e740 1 -- 172.21.15.45:0/4199851838 --> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x565185465b40 con 0x565185510040 2024-04-16T17:13:36.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.892+0000 7f50d3e00640 1 --2- 172.21.15.45:0/4199851838 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x565185510040 0x565185512430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:36.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.893+0000 7f50baa00640 10 client.27484 ms_handle_connect on v2:172.21.15.104:6838/3660070592 2024-04-16T17:13:36.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.893+0000 7f50d3e00640 1 --2- 172.21.15.45:0/4199851838 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x565185510040 0x565185512430 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5e44295f20e13b66 server_cookie=ce135a4e78d1aa10 in_seq=0 out_seq=0 2024-04-16T17:13:36.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.894+0000 7f50baa00640 1 -- 172.21.15.45:0/4199851838 <== mds.0 v2:172.21.15.104:6838/3660070592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x565185465b40 con 0x565185510040 2024-04-16T17:13:36.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.894+0000 7f50baa00640 10 client.27484 handle_command_reply: tid=0 2024-04-16T17:13:36.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.894+0000 7f50de79e740 1 client.27484 shutdown 2024-04-16T17:13:36.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.894+0000 7f50de79e740 1 -- 172.21.15.45:0/4199851838 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5651854f67a0 msgr2=0x5651854fcd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:36.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.894+0000 7f50de79e740 1 --2- 172.21.15.45:0/4199851838 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5651854f67a0 0x5651854fcd20 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f50c8019fc0 tx=0x7f50c8005d70 comp rx=0 tx=0).stop 2024-04-16T17:13:36.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.894+0000 7f50de79e740 1 -- 172.21.15.45:0/4199851838 shutdown_connections 2024-04-16T17:13:36.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.894+0000 7f50de79e740 1 --2- 172.21.15.45:0/4199851838 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5651854ffca0 0x5651854fd7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:36.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.894+0000 7f50de79e740 1 --2- 172.21.15.45:0/4199851838 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5651854f70c0 0x5651854fd260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:36.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.894+0000 7f50de79e740 1 --2- 172.21.15.45:0/4199851838 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x565185510040 0x565185512430 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:36.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.894+0000 7f50de79e740 1 --2- 172.21.15.45:0/4199851838 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5651854f67a0 0x5651854fcd20 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:36.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.894+0000 7f50de79e740 1 -- 172.21.15.45:0/4199851838 >> 172.21.15.45:0/4199851838 conn(0x5651854f0ed0 msgr2=0x5651854f2c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:13:36.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.894+0000 7f50baa00640 0 client.27484 ms_handle_reset on v2:172.21.15.104:6838/3660070592 2024-04-16T17:13:36.897 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.894+0000 7f50de79e740 1 -- 172.21.15.45:0/4199851838 shutdown_connections 2024-04-16T17:13:36.897 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.894+0000 7f50de79e740 1 -- 172.21.15.45:0/4199851838 wait complete. 2024-04-16T17:13:36.897 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.894+0000 7f50de79e740 20 client.27484 trim_cache size 0 max 16384 2024-04-16T17:13:36.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.915+0000 7f50de79e740 1 Processor -- start 2024-04-16T17:13:36.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.916+0000 7f50de79e740 1 -- start start 2024-04-16T17:13:36.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.916+0000 7f50de79e740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5651854f67a0 0x56518550b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:36.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.916+0000 7f50de79e740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5651854f70c0 0x5651854fc600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:36.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.916+0000 7f50de79e740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5651854ffca0 0x565185517bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:36.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.916+0000 7f50de79e740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f50d41afa50 con 0x5651854ffca0 2024-04-16T17:13:36.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.916+0000 7f50de79e740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f50d41ad2c0 con 0x5651854f70c0 2024-04-16T17:13:36.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.916+0000 7f50de79e740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f50d41ab130 con 0x5651854f67a0 2024-04-16T17:13:36.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.916+0000 7f50d9a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5651854ffca0 0x565185517bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:36.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.916+0000 7f50d9000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5651854f67a0 0x56518550b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:36.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.916+0000 7f50d9a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5651854ffca0 0x565185517bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56914/0 (socket says 172.21.15.45:56914) 2024-04-16T17:13:36.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.916+0000 7f50d9a00640 1 -- 172.21.15.45:0/1909784411 learned_addr learned my addr 172.21.15.45:0/1909784411 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:13:36.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.916+0000 7f50d9000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5651854f67a0 0x56518550b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:57460/0 (socket says 172.21.15.45:57460) 2024-04-16T17:13:36.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.916+0000 7f50d3e00640 1 --2- 172.21.15.45:0/1909784411 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5651854f70c0 0x5651854fc600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:36.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.917+0000 7f50d9000640 1 -- 172.21.15.45:0/1909784411 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5651854f70c0 msgr2=0x5651854fc600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:13:36.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.917+0000 7f50d9000640 1 --2- 172.21.15.45:0/1909784411 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5651854f70c0 0x5651854fc600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:36.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.917+0000 7f50d9000640 1 -- 172.21.15.45:0/1909784411 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5651854ffca0 msgr2=0x565185517bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:13:36.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.917+0000 7f50d9000640 1 --2- 172.21.15.45:0/1909784411 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5651854ffca0 0x565185517bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:36.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.917+0000 7f50d9000640 1 -- 172.21.15.45:0/1909784411 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5651855129c0 con 0x5651854f67a0 2024-04-16T17:13:36.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.917+0000 7f50d3e00640 1 --2- 172.21.15.45:0/1909784411 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5651854f70c0 0x5651854fc600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:13:36.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.917+0000 7f50d9a00640 1 --2- 172.21.15.45:0/1909784411 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5651854ffca0 0x565185517bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:13:36.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.917+0000 7f50d9000640 1 --2- 172.21.15.45:0/1909784411 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5651854f67a0 0x56518550b980 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f50c8003d40 tx=0x7f50c8003f50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cea2d7550f462204 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:13:36.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.917+0000 7f50c5600640 1 -- 172.21.15.45:0/1909784411 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50c8026070 con 0x5651854f67a0 2024-04-16T17:13:36.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.917+0000 7f50c5600640 1 -- 172.21.15.45:0/1909784411 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f50c8015020 con 0x5651854f67a0 2024-04-16T17:13:36.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.917+0000 7f50c5600640 1 -- 172.21.15.45:0/1909784411 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50c80050b0 con 0x5651854f67a0 2024-04-16T17:13:36.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.918+0000 7f50de79e740 1 -- 172.21.15.45:0/1909784411 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5651854f67a0 msgr2=0x56518550b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:36.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.918+0000 7f50de79e740 1 --2- 172.21.15.45:0/1909784411 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5651854f67a0 0x56518550b980 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f50c8003d40 tx=0x7f50c8003f50 comp rx=0 tx=0).stop 2024-04-16T17:13:36.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.918+0000 7f50de79e740 1 -- 172.21.15.45:0/1909784411 shutdown_connections 2024-04-16T17:13:36.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.918+0000 7f50de79e740 1 --2- 172.21.15.45:0/1909784411 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5651854ffca0 0x565185517bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:36.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.918+0000 7f50de79e740 1 --2- 172.21.15.45:0/1909784411 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5651854f70c0 0x5651854fc600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:36.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.918+0000 7f50de79e740 1 --2- 172.21.15.45:0/1909784411 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5651854f67a0 0x56518550b980 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:36.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.918+0000 7f50de79e740 1 -- 172.21.15.45:0/1909784411 >> 172.21.15.45:0/1909784411 conn(0x5651854f0ed0 msgr2=0x5651854f21a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:13:36.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.918+0000 7f50de79e740 1 -- 172.21.15.45:0/1909784411 shutdown_connections 2024-04-16T17:13:36.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.918+0000 7f50de79e740 1 -- 172.21.15.45:0/1909784411 wait complete. 2024-04-16T17:13:36.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.919+0000 7f50de79e740 1 -- start start 2024-04-16T17:13:36.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.919+0000 7f50de79e740 1 Processor -- start 2024-04-16T17:13:36.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.919+0000 7f50de79e740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5651854f67a0 0x565185507e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:36.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.919+0000 7f50de79e740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5651854f70c0 0x5651855083d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:36.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.919+0000 7f50de79e740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5651854ffca0 0x5651854f8f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:36.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.919+0000 7f50de79e740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f50d41ab780 con 0x5651854ffca0 2024-04-16T17:13:36.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.919+0000 7f50de79e740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f50d41acbf0 con 0x5651854f67a0 2024-04-16T17:13:36.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.919+0000 7f50de79e740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f50d41abe50 con 0x5651854f70c0 2024-04-16T17:13:36.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.919+0000 7f50de79e740 10 client.0 _ll_register_callbacks cb 0x7f50c0009a10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:13:36.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.920+0000 7f50d9a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5651854ffca0 0x5651854f8f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:36.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.920+0000 7f50d9000640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5651854f67a0 0x565185507e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:36.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.920+0000 7f50d9a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5651854ffca0 0x5651854f8f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56930/0 (socket says 172.21.15.45:56930) 2024-04-16T17:13:36.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.920+0000 7f50d9a00640 1 -- 172.21.15.45:0/3999960867 learned_addr learned my addr 172.21.15.45:0/3999960867 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:13:36.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.920+0000 7f50d3e00640 1 --2- 172.21.15.45:0/3999960867 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5651854f70c0 0x5651855083d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:36.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.920+0000 7f50d9000640 1 -- 172.21.15.45:0/3999960867 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5651854f70c0 msgr2=0x5651855083d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:13:36.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.920+0000 7f50d9000640 1 --2- 172.21.15.45:0/3999960867 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5651854f70c0 0x5651855083d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:36.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.920+0000 7f50d9000640 1 -- 172.21.15.45:0/3999960867 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5651854ffca0 msgr2=0x5651854f8f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:13:36.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.920+0000 7f50d9000640 1 --2- 172.21.15.45:0/3999960867 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5651854ffca0 0x5651854f8f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:36.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.920+0000 7f50d9000640 1 -- 172.21.15.45:0/3999960867 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5651854f94d0 con 0x5651854f67a0 2024-04-16T17:13:36.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.920+0000 7f50d9a00640 1 --2- 172.21.15.45:0/3999960867 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5651854ffca0 0x5651854f8f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:13:36.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.920+0000 7f50d3e00640 1 --2- 172.21.15.45:0/3999960867 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5651854f70c0 0x5651855083d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:13:36.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.920+0000 7f50d9000640 1 --2- 172.21.15.45:0/3999960867 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5651854f67a0 0x565185507e90 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f50c8003d40 tx=0x7f50c8011400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=abf59238ded22d0e server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:13:36.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.920+0000 7f50baa00640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-04-16T17:13:36.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.921+0000 7f50baa00640 1 -- 172.21.15.45:0/3999960867 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50c8026040 con 0x5651854f67a0 2024-04-16T17:13:36.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.921+0000 7f50baa00640 1 -- 172.21.15.45:0/3999960867 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9496+0+0 (secure 0 0 0) 0x7f50c80035f0 con 0x5651854f67a0 2024-04-16T17:13:36.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.921+0000 7f50de79e740 1 -- 172.21.15.45:0/3999960867 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f50c8015020 con 0x5651854f67a0 2024-04-16T17:13:36.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.921+0000 7f50baa00640 1 -- 172.21.15.45:0/3999960867 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f50c8045050 con 0x5651854f67a0 2024-04-16T17:13:36.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.921+0000 7f50de79e740 10 client.27545 fetch_fsmap learned FSMap version 38 2024-04-16T17:13:36.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.921+0000 7f50de79e740 1 -- 172.21.15.45:0/3999960867 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x565185575c70 con 0x5651854f67a0 2024-04-16T17:13:36.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.921+0000 7f50baa00640 1 -- 172.21.15.45:0/3999960867 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f50c8015020 con 0x5651854f67a0 2024-04-16T17:13:36.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.921+0000 7f50baa00640 1 -- 172.21.15.45:0/3999960867 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 38) ==== 8354+0+0 (secure 0 0 0) 0x7f50c800f640 con 0x5651854f67a0 2024-04-16T17:13:36.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.922+0000 7f50de79e740 10 client.27545 fetch_fsmap finished waiting for FSMap version 38 2024-04-16T17:13:36.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.922+0000 7f50de79e740 10 client.27545 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:13:36.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.922+0000 7f50de79e740 20 client.27545 populate_metadata read hostname 'smithi045' 2024-04-16T17:13:36.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.922+0000 7f50de79e740 1 --2- 172.21.15.45:0/3999960867 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x565185576270 0x565185578690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:36.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.922+0000 7f50de79e740 4 client.27545 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-16T17:13:36.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.922+0000 7f50de79e740 1 -- 172.21.15.45:0/3999960867 --> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56518545f9b0 con 0x565185576270 2024-04-16T17:13:36.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.922+0000 7f50d3e00640 1 --2- 172.21.15.45:0/3999960867 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x565185576270 0x565185578690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:36.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.922+0000 7f50d3e00640 1 --2- 172.21.15.45:0/3999960867 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x565185576270 0x565185578690 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f8051effd05bc68b server_cookie=a12b61b8c98e5b57 in_seq=0 out_seq=0 2024-04-16T17:13:36.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.922+0000 7f50baa00640 10 client.27545 ms_handle_connect on v2:172.21.15.104:6838/3660070592 2024-04-16T17:13:36.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.923+0000 7f50baa00640 1 -- 172.21.15.45:0/3999960867 <== mds.0 v2:172.21.15.104:6838/3660070592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x56518545f9b0 con 0x565185576270 2024-04-16T17:13:36.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.923+0000 7f50baa00640 10 client.27545 handle_command_reply: tid=0 2024-04-16T17:13:36.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.923+0000 7f50de79e740 1 client.27545 shutdown 2024-04-16T17:13:36.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.923+0000 7f50de79e740 1 -- 172.21.15.45:0/3999960867 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5651854f67a0 msgr2=0x565185507e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:36.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.923+0000 7f50de79e740 1 --2- 172.21.15.45:0/3999960867 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5651854f67a0 0x565185507e90 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f50c8003d40 tx=0x7f50c8011400 comp rx=0 tx=0).stop 2024-04-16T17:13:36.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.923+0000 7f50de79e740 1 -- 172.21.15.45:0/3999960867 shutdown_connections 2024-04-16T17:13:36.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.923+0000 7f50de79e740 1 --2- 172.21.15.45:0/3999960867 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x565185576270 0x565185578690 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:36.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.923+0000 7f50de79e740 1 --2- 172.21.15.45:0/3999960867 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5651854f70c0 0x5651855083d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:36.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.923+0000 7f50de79e740 1 --2- 172.21.15.45:0/3999960867 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5651854ffca0 0x5651854f8f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:36.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.923+0000 7f50de79e740 1 --2- 172.21.15.45:0/3999960867 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5651854f67a0 0x565185507e90 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:36.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.923+0000 7f50de79e740 1 -- 172.21.15.45:0/3999960867 >> 172.21.15.45:0/3999960867 conn(0x5651854f0ed0 msgr2=0x5651854f2a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:13:36.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.923+0000 7f50baa00640 0 client.27545 ms_handle_reset on v2:172.21.15.104:6838/3660070592 2024-04-16T17:13:36.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.923+0000 7f50de79e740 1 -- 172.21.15.45:0/3999960867 shutdown_connections 2024-04-16T17:13:36.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.923+0000 7f50de79e740 1 -- 172.21.15.45:0/3999960867 wait complete. 2024-04-16T17:13:36.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.923+0000 7f50de79e740 20 client.27545 trim_cache size 0 max 16384 2024-04-16T17:13:36.927 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-16T17:13:36.927 INFO:teuthology.orchestra.run.smithi045.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-16T17:13:36.927 INFO:teuthology.orchestra.run.smithi045.stdout: "scrubs": { 2024-04-16T17:13:36.927 INFO:teuthology.orchestra.run.smithi045.stdout: "93f224db-eff9-48f1-8362-810ce7ba60ef": { 2024-04-16T17:13:36.927 INFO:teuthology.orchestra.run.smithi045.stdout: "path": "/", 2024-04-16T17:13:36.928 INFO:teuthology.orchestra.run.smithi045.stdout: "tag": "93f224db-eff9-48f1-8362-810ce7ba60ef", 2024-04-16T17:13:36.928 INFO:teuthology.orchestra.run.smithi045.stdout: "options": "recursive,force" 2024-04-16T17:13:36.928 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:13:36.928 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:13:36.928 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-16T17:13:36.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.924+0000 7f50da400640 1 -- 172.21.15.45:0/3509951936 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f50bc0a3d20 msgr2=0x7f50bc0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:36.929 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.924+0000 7f50da400640 1 --2- 172.21.15.45:0/3509951936 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f50bc0a3d20 0x7f50bc0a61e0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f50c0009fd0 tx=0x7f50c0009260 comp rx=0 tx=0).stop 2024-04-16T17:13:36.929 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.924+0000 7f50da400640 1 -- 172.21.15.45:0/3509951936 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f50d419cfe0 msgr2=0x7f50d419d490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:36.929 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.924+0000 7f50da400640 1 --2- 172.21.15.45:0/3509951936 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f50d419cfe0 0x7f50d419d490 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f50cc0096f0 tx=0x7f50cc0336b0 comp rx=0 tx=0).stop 2024-04-16T17:13:36.929 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.924+0000 7f50da400640 1 -- 172.21.15.45:0/3509951936 shutdown_connections 2024-04-16T17:13:36.929 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.924+0000 7f50da400640 1 --2- 172.21.15.45:0/3509951936 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f50bc0a3d20 0x7f50bc0a61e0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:36.929 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.924+0000 7f50da400640 1 --2- 172.21.15.45:0/3509951936 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f50d41060d0 0x7f50d4198c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:36.929 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.924+0000 7f50da400640 1 --2- 172.21.15.45:0/3509951936 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f50d419cfe0 0x7f50d419d490 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:36.929 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.924+0000 7f50da400640 1 --2- 172.21.15.45:0/3509951936 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f50d4101000 0x7f50d4198720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:36.929 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.924+0000 7f50da400640 1 -- 172.21.15.45:0/3509951936 >> 172.21.15.45:0/3509951936 conn(0x7f50d40fc880 msgr2=0x7f50d4069470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:13:36.929 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.924+0000 7f50da400640 1 -- 172.21.15.45:0/3509951936 shutdown_connections 2024-04-16T17:13:36.929 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:13:36.924+0000 7f50da400640 1 -- 172.21.15.45:0/3509951936 wait complete. 2024-04-16T17:13:36.929 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:13:36 smithi104 ceph-mon[36457]: pgmap v210: 129 pgs: 129 active+clean; 59 MiB data, 735 MiB used, 1.0 TiB / 1.0 TiB avail; 782 KiB/s rd, 584 KiB/s wr, 63 op/s 2024-04-16T17:13:36.951 INFO:tasks.cephfs.filesystem:scrub status for tag:93f224db-eff9-48f1-8362-810ce7ba60ef - {'path': '/', 'tag': '93f224db-eff9-48f1-8362-810ce7ba60ef', 'options': 'recursive,force'} 2024-04-16T17:13:37.151 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:13:36 smithi045 ceph-mon[31676]: pgmap v210: 129 pgs: 129 active+clean; 59 MiB data, 735 MiB used, 1.0 TiB / 1.0 TiB avail; 782 KiB/s rd, 584 KiB/s wr, 63 op/s 2024-04-16T17:13:37.288 INFO:tasks.workunit.client.0.smithi045.stdout: 1 114370 1.75 MB/sec execute 219 sec latency 33.345 ms 2024-04-16T17:13:37.804 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:13:37 smithi117 ceph-mon[36730]: scrub summary: active paths [/] 2024-04-16T17:13:37.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:13:37 smithi117 ceph-mon[36730]: scrub summary: active paths [/] 2024-04-16T17:13:37.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:13:37 smithi117 ceph-mon[36730]: scrub summary: idle+waiting paths [/] 2024-04-16T17:13:37.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:13:37 smithi117 ceph-mon[36730]: scrub summary: idle+waiting paths [/] 2024-04-16T17:13:37.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:13:37 smithi104 ceph-mon[36457]: scrub summary: active paths [/] 2024-04-16T17:13:37.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:13:37 smithi104 ceph-mon[36457]: scrub summary: active paths [/] 2024-04-16T17:13:37.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:13:37 smithi104 ceph-mon[36457]: scrub summary: idle+waiting paths [/] 2024-04-16T17:13:37.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:13:37 smithi104 ceph-mon[36457]: scrub summary: idle+waiting paths [/] 2024-04-16T17:13:38.044 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:13:37 smithi045 ceph-mon[31676]: scrub summary: active paths [/] 2024-04-16T17:13:38.044 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:13:37 smithi045 ceph-mon[31676]: scrub summary: active paths [/] 2024-04-16T17:13:38.044 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:13:37 smithi045 ceph-mon[31676]: scrub summary: idle+waiting paths [/] 2024-04-16T17:13:38.044 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:13:37 smithi045 ceph-mon[31676]: scrub summary: idle+waiting paths [/] 2024-04-16T17:13:38.289 INFO:tasks.workunit.client.0.smithi045.stdout: 1 115036 1.77 MB/sec execute 220 sec latency 59.000 ms 2024-04-16T17:13:38.804 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:13:38 smithi117 ceph-mon[36730]: scrub queued for path: / 2024-04-16T17:13:38.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:13:38 smithi117 ceph-mon[36730]: scrub summary: idle+waiting paths [/] 2024-04-16T17:13:38.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:13:38 smithi117 ceph-mon[36730]: scrub summary: active paths [/] 2024-04-16T17:13:38.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:13:38 smithi117 ceph-mon[36730]: scrub summary: idle+waiting paths [/] 2024-04-16T17:13:38.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:13:38 smithi117 ceph-mon[36730]: pgmap v211: 129 pgs: 129 active+clean; 60 MiB data, 759 MiB used, 1.0 TiB / 1.0 TiB avail; 920 KiB/s rd, 851 KiB/s wr, 103 op/s 2024-04-16T17:13:38.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:13:38 smithi117 ceph-mon[36730]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-16T17:13:38.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:13:38 smithi104 ceph-mon[36457]: scrub queued for path: / 2024-04-16T17:13:38.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:13:38 smithi104 ceph-mon[36457]: scrub summary: idle+waiting paths [/] 2024-04-16T17:13:38.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:13:38 smithi104 ceph-mon[36457]: scrub summary: active paths [/] 2024-04-16T17:13:38.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:13:38 smithi104 ceph-mon[36457]: scrub summary: idle+waiting paths [/] 2024-04-16T17:13:38.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:13:38 smithi104 ceph-mon[36457]: pgmap v211: 129 pgs: 129 active+clean; 60 MiB data, 759 MiB used, 1.0 TiB / 1.0 TiB avail; 920 KiB/s rd, 851 KiB/s wr, 103 op/s 2024-04-16T17:13:38.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:13:38 smithi104 ceph-mon[36457]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-16T17:13:39.294 INFO:tasks.workunit.client.0.smithi045.stdout: 1 115391 1.76 MB/sec execute 221 sec latency 54.222 ms 2024-04-16T17:13:40.299 INFO:tasks.workunit.client.0.smithi045.stdout: 1 115728 1.75 MB/sec execute 222 sec latency 52.322 ms 2024-04-16T17:13:40.394 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:13:38 smithi045 ceph-mon[31676]: scrub queued for path: / 2024-04-16T17:13:40.394 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:13:38 smithi045 ceph-mon[31676]: scrub summary: idle+waiting paths [/] 2024-04-16T17:13:40.394 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:13:38 smithi045 ceph-mon[31676]: scrub summary: active paths [/] 2024-04-16T17:13:40.394 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:13:38 smithi045 ceph-mon[31676]: scrub summary: idle+waiting paths [/] 2024-04-16T17:13:40.394 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:13:38 smithi045 ceph-mon[31676]: pgmap v211: 129 pgs: 129 active+clean; 60 MiB data, 759 MiB used, 1.0 TiB / 1.0 TiB avail; 920 KiB/s rd, 851 KiB/s wr, 103 op/s 2024-04-16T17:13:40.394 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:13:38 smithi045 ceph-mon[31676]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-16T17:13:40.394 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:13:40 smithi045 ceph-mon[31676]: pgmap v212: 129 pgs: 129 active+clean; 60 MiB data, 759 MiB used, 1.0 TiB / 1.0 TiB avail; 667 KiB/s rd, 507 KiB/s wr, 73 op/s 2024-04-16T17:13:40.677 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:13:40 smithi104 ceph-mon[36457]: pgmap v212: 129 pgs: 129 active+clean; 60 MiB data, 759 MiB used, 1.0 TiB / 1.0 TiB avail; 667 KiB/s rd, 507 KiB/s wr, 73 op/s 2024-04-16T17:13:40.804 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:13:40 smithi117 ceph-mon[36730]: pgmap v212: 129 pgs: 129 active+clean; 60 MiB data, 759 MiB used, 1.0 TiB / 1.0 TiB avail; 667 KiB/s rd, 507 KiB/s wr, 73 op/s 2024-04-16T17:13:41.297 INFO:tasks.workunit.client.0.smithi045.stdout: 1 116917 1.79 MB/sec execute 223 sec latency 149.199 ms 2024-04-16T17:13:42.298 INFO:tasks.workunit.client.0.smithi045.stdout: 1 117454 1.80 MB/sec execute 224 sec latency 22.869 ms 2024-04-16T17:13:42.677 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:13:42 smithi104 ceph-mon[36457]: pgmap v213: 129 pgs: 129 active+clean; 63 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1004 KiB/s wr, 122 op/s 2024-04-16T17:13:42.804 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:13:42 smithi117 ceph-mon[36730]: pgmap v213: 129 pgs: 129 active+clean; 63 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1004 KiB/s wr, 122 op/s 2024-04-16T17:13:43.123 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:13:42 smithi045 ceph-mon[31676]: pgmap v213: 129 pgs: 129 active+clean; 63 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1004 KiB/s wr, 122 op/s 2024-04-16T17:13:43.298 INFO:tasks.workunit.client.0.smithi045.stdout: 1 117709 1.79 MB/sec execute 225 sec latency 54.983 ms 2024-04-16T17:13:43.676 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:13:43 smithi045 ceph-mon[31676]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-16T17:13:43.677 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:13:43 smithi104 ceph-mon[36457]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-16T17:13:43.804 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:13:43 smithi117 ceph-mon[36730]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-16T17:13:44.298 INFO:tasks.workunit.client.0.smithi045.stdout: 1 118197 1.79 MB/sec execute 226 sec latency 70.989 ms 2024-04-16T17:13:44.473 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:13:44 smithi045 ceph-mon[31676]: pgmap v214: 129 pgs: 129 active+clean; 63 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 671 KiB/s rd, 763 KiB/s wr, 89 op/s 2024-04-16T17:13:44.473 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:13:44 smithi045 ceph-mon[31676]: scrub summary: idle 2024-04-16T17:13:44.473 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:13:44 smithi045 ceph-mon[31676]: scrub summary: idle 2024-04-16T17:13:44.473 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:13:44 smithi045 ceph-mon[31676]: scrub summary: idle 2024-04-16T17:13:44.677 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:13:44 smithi104 ceph-mon[36457]: pgmap v214: 129 pgs: 129 active+clean; 63 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 671 KiB/s rd, 763 KiB/s wr, 89 op/s 2024-04-16T17:13:44.677 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:13:44 smithi104 ceph-mon[36457]: scrub summary: idle 2024-04-16T17:13:44.677 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:13:44 smithi104 ceph-mon[36457]: scrub summary: idle 2024-04-16T17:13:44.677 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:13:44 smithi104 ceph-mon[36457]: scrub summary: idle 2024-04-16T17:13:44.804 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:13:44 smithi117 ceph-mon[36730]: pgmap v214: 129 pgs: 129 active+clean; 63 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 671 KiB/s rd, 763 KiB/s wr, 89 op/s 2024-04-16T17:13:44.804 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:13:44 smithi117 ceph-mon[36730]: scrub summary: idle 2024-04-16T17:13:44.804 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:13:44 smithi117 ceph-mon[36730]: scrub summary: idle 2024-04-16T17:13:44.804 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:13:44 smithi117 ceph-mon[36730]: scrub summary: idle 2024-04-16T17:13:45.298 INFO:tasks.workunit.client.0.smithi045.stdout: 1 118609 1.80 MB/sec execute 227 sec latency 66.992 ms 2024-04-16T17:13:46.298 INFO:tasks.workunit.client.0.smithi045.stdout: 1 118885 1.79 MB/sec execute 228 sec latency 82.411 ms 2024-04-16T17:13:46.677 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:13:46 smithi104 ceph-mon[36457]: pgmap v215: 129 pgs: 129 active+clean; 63 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 671 KiB/s rd, 763 KiB/s wr, 89 op/s 2024-04-16T17:13:46.804 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:13:46 smithi117 ceph-mon[36730]: pgmap v215: 129 pgs: 129 active+clean; 63 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 671 KiB/s rd, 763 KiB/s wr, 89 op/s 2024-04-16T17:13:47.043 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:13:46 smithi045 ceph-mon[31676]: pgmap v215: 129 pgs: 129 active+clean; 63 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 671 KiB/s rd, 763 KiB/s wr, 89 op/s 2024-04-16T17:13:47.298 INFO:tasks.workunit.client.0.smithi045.stdout: 1 119979 1.81 MB/sec execute 229 sec latency 29.588 ms 2024-04-16T17:13:47.549 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:47.549+0000 7f0073e00640 1 -- 172.21.15.45:0/310673872 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f00741038e0 msgr2=0x7f007410bdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:47.549 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:47.549+0000 7f0073e00640 1 --2- 172.21.15.45:0/310673872 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f00741038e0 0x7f007410bdc0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f0060009900 tx=0x7f006002bcd0 comp rx=0 tx=0).stop 2024-04-16T17:13:47.550 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:47.549+0000 7f0073e00640 1 -- 172.21.15.45:0/310673872 shutdown_connections 2024-04-16T17:13:47.550 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:47.549+0000 7f0073e00640 1 --2- 172.21.15.45:0/310673872 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f007410c3e0 0x7f007410e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:47.550 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:47.549+0000 7f0073e00640 1 --2- 172.21.15.45:0/310673872 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f00741038e0 0x7f007410bdc0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:47.550 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:47.549+0000 7f0073e00640 1 --2- 172.21.15.45:0/310673872 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0074102fc0 0x7f00741033a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:47.550 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:47.549+0000 7f0073e00640 1 -- 172.21.15.45:0/310673872 >> 172.21.15.45:0/310673872 conn(0x7f00740fc820 msgr2=0x7f00740fec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:13:47.550 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:47.549+0000 7f0073e00640 1 -- 172.21.15.45:0/310673872 shutdown_connections 2024-04-16T17:13:47.550 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:47.549+0000 7f0073e00640 1 -- 172.21.15.45:0/310673872 wait complete. 2024-04-16T17:13:47.551 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:47.550+0000 7f0073e00640 1 Processor -- start 2024-04-16T17:13:47.551 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:47.551+0000 7f0073e00640 1 -- start start 2024-04-16T17:13:47.552 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:47.551+0000 7f0073e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0074102fc0 0x7f00741a0620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:47.552 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:47.551+0000 7f0073e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f00741038e0 0x7f00741a0b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:47.552 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:47.551+0000 7f0073e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f007410c3e0 0x7f007419a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:47.552 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:47.551+0000 7f0073e00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f00741121c0 con 0x7f007410c3e0 2024-04-16T17:13:47.552 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:47.551+0000 7f0073e00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f0074112040 con 0x7f00741038e0 2024-04-16T17:13:47.552 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:47.551+0000 7f0073e00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f0074112340 con 0x7f0074102fc0 2024-04-16T17:13:47.553 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:47.552+0000 7f0073400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f007410c3e0 0x7f007419a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:47.553 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:47.552+0000 7f0072a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0074102fc0 0x7f00741a0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:47.553 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:47.552+0000 7f0072a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0074102fc0 0x7f00741a0620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:50286/0 (socket says 172.21.15.45:50286) 2024-04-16T17:13:47.553 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:47.552+0000 7f0072a00640 1 -- 172.21.15.45:0/3519570806 learned_addr learned my addr 172.21.15.45:0/3519570806 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:13:47.553 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:47.552+0000 7f0072000640 1 --2- 172.21.15.45:0/3519570806 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f00741038e0 0x7f00741a0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:47.553 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:47.552+0000 7f0072a00640 1 -- 172.21.15.45:0/3519570806 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f00741038e0 msgr2=0x7f00741a0b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:47.553 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:47.552+0000 7f0072a00640 1 --2- 172.21.15.45:0/3519570806 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f00741038e0 0x7f00741a0b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:47.553 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:47.552+0000 7f0072a00640 1 -- 172.21.15.45:0/3519570806 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f007410c3e0 msgr2=0x7f007419a7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:47.553 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:47.552+0000 7f0072a00640 1 --2- 172.21.15.45:0/3519570806 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f007410c3e0 0x7f007419a7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:47.553 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:47.552+0000 7f0072a00640 1 -- 172.21.15.45:0/3519570806 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f007419afb0 con 0x7f0074102fc0 2024-04-16T17:13:47.554 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:47.553+0000 7f0072a00640 1 --2- 172.21.15.45:0/3519570806 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0074102fc0 0x7f00741a0620 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f006400e9f0 tx=0x7f006400edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:13:47.554 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:47.553+0000 7f006b400640 1 -- 172.21.15.45:0/3519570806 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f006400ccc0 con 0x7f0074102fc0 2024-04-16T17:13:47.554 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:47.553+0000 7f0073e00640 1 -- 172.21.15.45:0/3519570806 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f007419b2a0 con 0x7f0074102fc0 2024-04-16T17:13:47.554 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:47.553+0000 7f006b400640 1 -- 172.21.15.45:0/3519570806 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f006400ce60 con 0x7f0074102fc0 2024-04-16T17:13:47.554 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:47.553+0000 7f006b400640 1 -- 172.21.15.45:0/3519570806 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0064010450 con 0x7f0074102fc0 2024-04-16T17:13:47.554 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:47.553+0000 7f0073e00640 1 -- 172.21.15.45:0/3519570806 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00741a7470 con 0x7f0074102fc0 2024-04-16T17:13:47.558 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:47.558+0000 7f006b400640 1 -- 172.21.15.45:0/3519570806 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f0064005020 con 0x7f0074102fc0 2024-04-16T17:13:47.558 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:47.558+0000 7f007c5c7740 1 Processor -- start 2024-04-16T17:13:47.559 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:47.558+0000 7f007c5c7740 1 -- start start 2024-04-16T17:13:47.559 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:47.559+0000 7f007c5c7740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5633f90c87f0 0x5633f90c7540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:47.559 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:47.559+0000 7f007c5c7740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5633f90c7a80 0x5633f90c7f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:47.560 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:47.559+0000 7f007c5c7740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5633f90d9810 0x5633f90dbc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:47.560 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:47.559+0000 7f007c5c7740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5633f902c780 con 0x5633f90d9810 2024-04-16T17:13:47.560 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:47.559+0000 7f007c5c7740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5633f8fd2f40 con 0x5633f90c7a80 2024-04-16T17:13:47.560 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:47.559+0000 7f007c5c7740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5633f9028740 con 0x5633f90c87f0 2024-04-16T17:13:47.560 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:47.559+0000 7f0072a00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5633f90c7a80 0x5633f90c7f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:47.560 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:47.559+0000 7f0072a00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5633f90c7a80 0x5633f90c7f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:51530/0 (socket says 172.21.15.45:51530) 2024-04-16T17:13:47.561 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:47.559+0000 7f0072a00640 1 -- 172.21.15.45:0/4038408775 learned_addr learned my addr 172.21.15.45:0/4038408775 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:13:47.561 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:47.560+0000 7f0073400640 1 --2- 172.21.15.45:0/4038408775 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5633f90c87f0 0x5633f90c7540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:47.561 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:47.560+0000 7f0072000640 1 --2- 172.21.15.45:0/4038408775 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5633f90d9810 0x5633f90dbc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:47.562 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:47.560+0000 7f0073400640 1 -- 172.21.15.45:0/4038408775 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5633f90c7a80 msgr2=0x5633f90c7f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:13:47.562 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:47.560+0000 7f0073400640 1 --2- 172.21.15.45:0/4038408775 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5633f90c7a80 0x5633f90c7f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:47.562 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:47.560+0000 7f0073400640 1 -- 172.21.15.45:0/4038408775 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5633f90d9810 msgr2=0x5633f90dbc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:13:47.562 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:47.560+0000 7f0073400640 1 --2- 172.21.15.45:0/4038408775 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5633f90d9810 0x5633f90dbc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:47.562 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:47.560+0000 7f0073400640 1 -- 172.21.15.45:0/4038408775 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5633f8d3f210 con 0x5633f90c87f0 2024-04-16T17:13:47.562 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:47.560+0000 7f006b400640 1 --2- 172.21.15.45:0/3519570806 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f00580a3d10 0x7f00580a61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:47.562 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:47.560+0000 7f006b400640 1 -- 172.21.15.45:0/3519570806 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9496+0+0 (secure 0 0 0) 0x7f00640d1030 con 0x7f0074102fc0 2024-04-16T17:13:47.562 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:47.560+0000 7f0073400640 1 --2- 172.21.15.45:0/4038408775 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5633f90c87f0 0x5633f90c7540 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f006c009f10 tx=0x7f006c011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=75cbadb246c44e2c server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:13:47.562 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:47.561+0000 7f0072a00640 1 --2- 172.21.15.45:0/3519570806 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f00580a3d10 0x7f00580a61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:47.562 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:47.561+0000 7f005fe00640 1 -- 172.21.15.45:0/4038408775 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f006c004d20 con 0x5633f90c87f0 2024-04-16T17:13:47.562 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:47.561+0000 7f005fe00640 1 -- 172.21.15.45:0/4038408775 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f006c004050 con 0x5633f90c87f0 2024-04-16T17:13:47.562 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:47.561+0000 7f005fe00640 1 -- 172.21.15.45:0/4038408775 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f006c005260 con 0x5633f90c87f0 2024-04-16T17:13:47.563 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:47.562+0000 7f007c5c7740 1 -- 172.21.15.45:0/4038408775 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5633f90c87f0 msgr2=0x5633f90c7540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:47.563 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:47.562+0000 7f007c5c7740 1 --2- 172.21.15.45:0/4038408775 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5633f90c87f0 0x5633f90c7540 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f006c009f10 tx=0x7f006c011810 comp rx=0 tx=0).stop 2024-04-16T17:13:47.563 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:47.562+0000 7f007c5c7740 1 -- 172.21.15.45:0/4038408775 shutdown_connections 2024-04-16T17:13:47.563 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:47.562+0000 7f007c5c7740 1 --2- 172.21.15.45:0/4038408775 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5633f90d9810 0x5633f90dbc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:47.564 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:47.562+0000 7f007c5c7740 1 --2- 172.21.15.45:0/4038408775 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5633f90c7a80 0x5633f90c7f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:47.564 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:47.562+0000 7f007c5c7740 1 --2- 172.21.15.45:0/4038408775 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5633f90c87f0 0x5633f90c7540 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:47.564 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:47.562+0000 7f007c5c7740 1 -- 172.21.15.45:0/4038408775 >> 172.21.15.45:0/4038408775 conn(0x5633f90625d0 msgr2=0x5633f90c2770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:13:47.564 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:47.562+0000 7f0072a00640 1 --2- 172.21.15.45:0/3519570806 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f00580a3d10 0x7f00580a61d0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f006409c900 tx=0x7f00640d4b70 comp rx=0 tx=0).ready entity=mgr.25411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:13:47.564 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:47.562+0000 7f007c5c7740 1 -- 172.21.15.45:0/4038408775 shutdown_connections 2024-04-16T17:13:47.564 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:47.563+0000 7f007c5c7740 1 -- 172.21.15.45:0/4038408775 wait complete. 2024-04-16T17:13:47.564 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:47.563+0000 7f007c5c7740 1 -- start start 2024-04-16T17:13:47.565 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:47.563+0000 7f007c5c7740 1 Processor -- start 2024-04-16T17:13:47.565 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:47.564+0000 7f007c5c7740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5633f90c7a80 0x5633f90d66e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:47.565 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:47.564+0000 7f007c5c7740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5633f90c87f0 0x5633f90d6c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:47.566 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:47.564+0000 7f007c5c7740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5633f90d9810 0x5633f90d7160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:47.566 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:47.564+0000 7f007c5c7740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5633f90dc540 con 0x5633f90d9810 2024-04-16T17:13:47.566 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:47.564+0000 7f007c5c7740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5633f8fe6390 con 0x5633f90c7a80 2024-04-16T17:13:47.566 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:47.564+0000 7f007c5c7740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5633f90dc6c0 con 0x5633f90c87f0 2024-04-16T17:13:47.566 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:47.564+0000 7f007c5c7740 10 client.0 _ll_register_callbacks cb 0x5633f8fcff60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:13:47.566 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:47.565+0000 7f0073400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5633f90c7a80 0x5633f90d66e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:47.566 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:47.565+0000 7f0073400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5633f90c7a80 0x5633f90d66e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:51536/0 (socket says 172.21.15.45:51536) 2024-04-16T17:13:47.566 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:47.565+0000 7f0073400640 1 -- 172.21.15.45:0/1204160921 learned_addr learned my addr 172.21.15.45:0/1204160921 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:13:47.566 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:47.565+0000 7f0073400640 1 --2- 172.21.15.45:0/1204160921 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5633f90d9810 0x5633f90d7160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:47.566 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:47.565+0000 7f0072000640 1 --2- 172.21.15.45:0/1204160921 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5633f90c87f0 0x5633f90d6c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:47.566 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:47.565+0000 7f0072000640 1 -- 172.21.15.45:0/1204160921 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5633f90c7a80 msgr2=0x5633f90d66e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:13:47.566 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:47.565+0000 7f0072000640 1 --2- 172.21.15.45:0/1204160921 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5633f90c7a80 0x5633f90d66e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:47.567 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:47.565+0000 7f0072000640 1 -- 172.21.15.45:0/1204160921 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5633f90d9810 msgr2=0x5633f90d7160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:13:47.567 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:47.565+0000 7f0072000640 1 --2- 172.21.15.45:0/1204160921 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5633f90d9810 0x5633f90d7160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:47.567 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:47.565+0000 7f0072000640 1 -- 172.21.15.45:0/1204160921 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5633f8e0b620 con 0x5633f90c87f0 2024-04-16T17:13:47.567 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:47.565+0000 7f0073400640 1 --2- 172.21.15.45:0/1204160921 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5633f90c7a80 0x5633f90d66e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:13:47.568 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:47.566+0000 7f0072000640 1 --2- 172.21.15.45:0/1204160921 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5633f90c87f0 0x5633f90d6c20 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f0060000c00 tx=0x7f0060005b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=720913bd1bbb28d9 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:13:47.568 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:47.566+0000 7f005cc00640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-04-16T17:13:47.568 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:47.566+0000 7f005cc00640 1 -- 172.21.15.45:0/1204160921 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0060004240 con 0x5633f90c87f0 2024-04-16T17:13:47.568 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:47.566+0000 7f007c5c7740 1 -- 172.21.15.45:0/1204160921 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f006c004050 con 0x5633f90c87f0 2024-04-16T17:13:47.568 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:47.566+0000 7f005cc00640 1 -- 172.21.15.45:0/1204160921 <== mon.1 v2:172.21.15.117:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9496+0+0 (secure 0 0 0) 0x7f00600043e0 con 0x5633f90c87f0 2024-04-16T17:13:47.568 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:47.567+0000 7f005cc00640 1 -- 172.21.15.45:0/1204160921 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f00600046c0 con 0x5633f90c87f0 2024-04-16T17:13:47.568 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:47.567+0000 7f007c5c7740 10 client.27502 fetch_fsmap learned FSMap version 40 2024-04-16T17:13:47.568 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:47.567+0000 7f007c5c7740 1 -- 172.21.15.45:0/1204160921 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x5633f90e8700 con 0x5633f90c87f0 2024-04-16T17:13:47.569 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:47.567+0000 7f005cc00640 1 -- 172.21.15.45:0/1204160921 <== mon.1 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f006c004050 con 0x5633f90c87f0 2024-04-16T17:13:47.569 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:47.567+0000 7f005cc00640 1 -- 172.21.15.45:0/1204160921 <== mon.1 v2:172.21.15.117:3300/0 5 ==== fsmap(e 40) ==== 8362+0+0 (secure 0 0 0) 0x7f00600053f0 con 0x5633f90c87f0 2024-04-16T17:13:47.569 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:47.567+0000 7f007c5c7740 10 client.27502 fetch_fsmap finished waiting for FSMap version 40 2024-04-16T17:13:47.569 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:47.567+0000 7f007c5c7740 10 client.27502 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T17:13:47.569 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:47.568+0000 7f007c5c7740 20 client.27502 populate_metadata read hostname 'smithi045' 2024-04-16T17:13:47.569 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:47.568+0000 7f007c5c7740 1 --2- 172.21.15.45:0/1204160921 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x5633f90e9b70 0x5633f90ebf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:47.569 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:47.568+0000 7f007c5c7740 4 client.27502 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:13:47.569 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:47.568+0000 7f007c5c7740 1 -- 172.21.15.45:0/1204160921 --> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5633f903a1a0 con 0x5633f90e9b70 2024-04-16T17:13:47.569 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:47.568+0000 7f0073400640 1 --2- 172.21.15.45:0/1204160921 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x5633f90e9b70 0x5633f90ebf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:47.570 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:47.569+0000 7f0073400640 1 --2- 172.21.15.45:0/1204160921 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x5633f90e9b70 0x5633f90ebf60 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ac3d0bdf06bb4953 server_cookie=8af7cdbb79eac4ca in_seq=0 out_seq=0 2024-04-16T17:13:47.570 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:47.569+0000 7f005cc00640 10 client.27502 ms_handle_connect on v2:172.21.15.104:6838/3660070592 2024-04-16T17:13:47.570 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:47.570+0000 7f005cc00640 1 -- 172.21.15.45:0/1204160921 <== mds.0 v2:172.21.15.104:6838/3660070592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5633f903a1a0 con 0x5633f90e9b70 2024-04-16T17:13:47.570 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:47.570+0000 7f005cc00640 10 client.27502 handle_command_reply: tid=0 2024-04-16T17:13:47.570 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:47.570+0000 7f007c5c7740 1 client.27502 shutdown 2024-04-16T17:13:47.571 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:47.570+0000 7f007c5c7740 1 -- 172.21.15.45:0/1204160921 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5633f90c87f0 msgr2=0x5633f90d6c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:47.571 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:47.570+0000 7f007c5c7740 1 --2- 172.21.15.45:0/1204160921 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5633f90c87f0 0x5633f90d6c20 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f0060000c00 tx=0x7f0060005b90 comp rx=0 tx=0).stop 2024-04-16T17:13:47.572 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:47.570+0000 7f007c5c7740 1 -- 172.21.15.45:0/1204160921 shutdown_connections 2024-04-16T17:13:47.572 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:47.570+0000 7f007c5c7740 1 --2- 172.21.15.45:0/1204160921 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x5633f90e9b70 0x5633f90ebf60 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:47.572 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:47.571+0000 7f007c5c7740 1 --2- 172.21.15.45:0/1204160921 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5633f90c87f0 0x5633f90d6c20 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:47.572 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:47.571+0000 7f007c5c7740 1 --2- 172.21.15.45:0/1204160921 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5633f90d9810 0x5633f90d7160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:47.572 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:47.571+0000 7f005cc00640 0 client.27502 ms_handle_reset on v2:172.21.15.104:6838/3660070592 2024-04-16T17:13:47.572 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:47.571+0000 7f007c5c7740 1 --2- 172.21.15.45:0/1204160921 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5633f90c7a80 0x5633f90d66e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:47.572 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:47.571+0000 7f007c5c7740 1 -- 172.21.15.45:0/1204160921 >> 172.21.15.45:0/1204160921 conn(0x5633f90625d0 msgr2=0x5633f90c44c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:13:47.572 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:47.571+0000 7f007c5c7740 1 -- 172.21.15.45:0/1204160921 shutdown_connections 2024-04-16T17:13:47.572 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:47.571+0000 7f007c5c7740 1 -- 172.21.15.45:0/1204160921 wait complete. 2024-04-16T17:13:47.572 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:47.571+0000 7f007c5c7740 20 client.27502 trim_cache size 0 max 16384 2024-04-16T17:13:47.591 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:47.591+0000 7f007c5c7740 1 Processor -- start 2024-04-16T17:13:47.592 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:47.591+0000 7f007c5c7740 1 -- start start 2024-04-16T17:13:47.592 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:47.592+0000 7f007c5c7740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5633f90c7a80 0x5633f90d8e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:47.592 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:47.592+0000 7f007c5c7740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5633f90c87f0 0x5633f90e5440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:47.592 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:47.592+0000 7f007c5c7740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5633f90d9810 0x5633f90c4240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:47.592 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:47.592+0000 7f007c5c7740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f00741afb70 con 0x5633f90c7a80 2024-04-16T17:13:47.592 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:47.592+0000 7f007c5c7740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f00741ad380 con 0x5633f90d9810 2024-04-16T17:13:47.592 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:47.592+0000 7f007c5c7740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f00741ab210 con 0x5633f90c87f0 2024-04-16T17:13:47.593 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:47.592+0000 7f0073400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5633f90c7a80 0x5633f90d8e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:47.593 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:47.592+0000 7f0073400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5633f90c7a80 0x5633f90d8e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34910/0 (socket says 172.21.15.45:34910) 2024-04-16T17:13:47.593 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:47.592+0000 7f0073400640 1 -- 172.21.15.45:0/902065518 learned_addr learned my addr 172.21.15.45:0/902065518 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:13:47.593 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:47.592+0000 7f0072000640 1 --2- 172.21.15.45:0/902065518 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5633f90c87f0 0x5633f90e5440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:47.593 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:47.592+0000 7f0073400640 1 --2- 172.21.15.45:0/902065518 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5633f90d9810 0x5633f90c4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:47.593 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:47.593+0000 7f0073400640 1 -- 172.21.15.45:0/902065518 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5633f90c87f0 msgr2=0x5633f90e5440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:13:47.593 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:47.593+0000 7f0073400640 1 --2- 172.21.15.45:0/902065518 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5633f90c87f0 0x5633f90e5440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:47.593 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:47.593+0000 7f0073400640 1 -- 172.21.15.45:0/902065518 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5633f90d9810 msgr2=0x5633f90c4240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:13:47.593 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:47.593+0000 7f0073400640 1 --2- 172.21.15.45:0/902065518 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5633f90d9810 0x5633f90c4240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:47.594 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:47.593+0000 7f0073400640 1 -- 172.21.15.45:0/902065518 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5633f90ec5c0 con 0x5633f90c7a80 2024-04-16T17:13:47.594 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:47.593+0000 7f0073400640 1 --2- 172.21.15.45:0/902065518 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5633f90c7a80 0x5633f90d8e10 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f006c005fd0 tx=0x7f006c009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88a208dee9630528 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:13:47.594 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:47.593+0000 7f005f400640 1 -- 172.21.15.45:0/902065518 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f006c0052c0 con 0x5633f90c7a80 2024-04-16T17:13:47.594 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:47.593+0000 7f005f400640 1 -- 172.21.15.45:0/902065518 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f006c003970 con 0x5633f90c7a80 2024-04-16T17:13:47.594 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:47.593+0000 7f005f400640 1 -- 172.21.15.45:0/902065518 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f006c019690 con 0x5633f90c7a80 2024-04-16T17:13:47.595 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:47.594+0000 7f007c5c7740 1 -- 172.21.15.45:0/902065518 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5633f90c7a80 msgr2=0x5633f90d8e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:47.595 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:47.594+0000 7f007c5c7740 1 --2- 172.21.15.45:0/902065518 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5633f90c7a80 0x5633f90d8e10 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f006c005fd0 tx=0x7f006c009500 comp rx=0 tx=0).stop 2024-04-16T17:13:47.595 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:47.594+0000 7f007c5c7740 1 -- 172.21.15.45:0/902065518 shutdown_connections 2024-04-16T17:13:47.595 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:47.594+0000 7f007c5c7740 1 --2- 172.21.15.45:0/902065518 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5633f90c87f0 0x5633f90e5440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:47.595 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:47.594+0000 7f007c5c7740 1 --2- 172.21.15.45:0/902065518 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5633f90d9810 0x5633f90c4240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:47.596 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:47.594+0000 7f007c5c7740 1 --2- 172.21.15.45:0/902065518 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5633f90c7a80 0x5633f90d8e10 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:47.596 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:47.594+0000 7f007c5c7740 1 -- 172.21.15.45:0/902065518 >> 172.21.15.45:0/902065518 conn(0x5633f90625d0 msgr2=0x5633f90c3b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:13:47.596 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:47.595+0000 7f007c5c7740 1 -- 172.21.15.45:0/902065518 shutdown_connections 2024-04-16T17:13:47.596 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:47.595+0000 7f007c5c7740 1 -- 172.21.15.45:0/902065518 wait complete. 2024-04-16T17:13:47.596 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:47.596+0000 7f007c5c7740 1 -- start start 2024-04-16T17:13:47.596 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:47.596+0000 7f007c5c7740 1 Processor -- start 2024-04-16T17:13:47.597 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:47.597+0000 7f007c5c7740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5633f90c7a80 0x5633f90dda40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:47.597 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:47.597+0000 7f007c5c7740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5633f90c87f0 0x5633f90ddf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:47.598 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:47.597+0000 7f0073400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5633f90c7a80 0x5633f90dda40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:47.598 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:47.597+0000 7f0073400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5633f90c7a80 0x5633f90dda40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:50326/0 (socket says 172.21.15.45:50326) 2024-04-16T17:13:47.598 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:47.598+0000 7f0073400640 1 -- 172.21.15.45:0/1421559086 learned_addr learned my addr 172.21.15.45:0/1421559086 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:13:47.598 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:47.598+0000 7f007c5c7740 1 --2- 172.21.15.45:0/1421559086 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5633f90d9810 0x5633f914f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:47.598 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:47.598+0000 7f007c5c7740 1 -- 172.21.15.45:0/1421559086 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f00741ab860 con 0x5633f90c87f0 2024-04-16T17:13:47.599 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:47.598+0000 7f007c5c7740 1 -- 172.21.15.45:0/1421559086 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f00741accb0 con 0x5633f90d9810 2024-04-16T17:13:47.600 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:47.598+0000 7f007c5c7740 1 -- 172.21.15.45:0/1421559086 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f00741abf10 con 0x5633f90c7a80 2024-04-16T17:13:47.600 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:47.598+0000 7f007c5c7740 10 client.0 _ll_register_callbacks cb 0x7f006c0093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:13:47.600 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:47.598+0000 7f0073400640 1 --2- 172.21.15.45:0/1421559086 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5633f90d9810 0x5633f914f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:47.600 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:47.598+0000 7f0072000640 1 --2- 172.21.15.45:0/1421559086 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5633f90c87f0 0x5633f90ddf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:47.600 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:47.598+0000 7f0073400640 1 -- 172.21.15.45:0/1421559086 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5633f90d9810 msgr2=0x5633f914f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:13:47.600 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:47.598+0000 7f0073400640 1 --2- 172.21.15.45:0/1421559086 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5633f90d9810 0x5633f914f0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:47.600 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:47.598+0000 7f0073400640 1 -- 172.21.15.45:0/1421559086 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5633f90c87f0 msgr2=0x5633f90ddf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:13:47.600 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:47.599+0000 7f0073400640 1 --2- 172.21.15.45:0/1421559086 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5633f90c87f0 0x5633f90ddf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:47.600 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:47.599+0000 7f0073400640 1 -- 172.21.15.45:0/1421559086 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5633f90d22c0 con 0x5633f90c7a80 2024-04-16T17:13:47.600 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:47.599+0000 7f0072000640 1 --2- 172.21.15.45:0/1421559086 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5633f90c87f0 0x5633f90ddf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:13:47.600 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:47.599+0000 7f0073400640 1 --2- 172.21.15.45:0/1421559086 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5633f90c7a80 0x5633f90dda40 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f006c0058a0 tx=0x7f006c005810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e972a1bb23e932a6 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:13:47.600 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:47.599+0000 7f005cc00640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-04-16T17:13:47.601 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:47.600+0000 7f005cc00640 1 -- 172.21.15.45:0/1421559086 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f006c028650 con 0x5633f90c7a80 2024-04-16T17:13:47.601 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:47.600+0000 7f005cc00640 1 -- 172.21.15.45:0/1421559086 <== mon.1 v2:172.21.15.117:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9496+0+0 (secure 0 0 0) 0x7f006c0287f0 con 0x5633f90c7a80 2024-04-16T17:13:47.601 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:47.600+0000 7f007c5c7740 1 -- 172.21.15.45:0/1421559086 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f006c003970 con 0x5633f90c7a80 2024-04-16T17:13:47.601 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:47.600+0000 7f005cc00640 1 -- 172.21.15.45:0/1421559086 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f006c028c00 con 0x5633f90c7a80 2024-04-16T17:13:47.602 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:47.601+0000 7f007c5c7740 10 client.27508 fetch_fsmap learned FSMap version 40 2024-04-16T17:13:47.602 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:47.601+0000 7f007c5c7740 1 -- 172.21.15.45:0/1421559086 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x5633f90c6d50 con 0x5633f90c7a80 2024-04-16T17:13:47.602 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:47.601+0000 7f005cc00640 1 -- 172.21.15.45:0/1421559086 <== mon.1 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f006c003970 con 0x5633f90c7a80 2024-04-16T17:13:47.602 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:47.601+0000 7f005cc00640 1 -- 172.21.15.45:0/1421559086 <== mon.1 v2:172.21.15.117:3300/0 5 ==== fsmap(e 40) ==== 8362+0+0 (secure 0 0 0) 0x7f006c047550 con 0x5633f90c7a80 2024-04-16T17:13:47.602 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:47.601+0000 7f007c5c7740 10 client.27508 fetch_fsmap finished waiting for FSMap version 40 2024-04-16T17:13:47.602 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:47.601+0000 7f007c5c7740 10 client.27508 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T17:13:47.602 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:47.601+0000 7f007c5c7740 20 client.27508 populate_metadata read hostname 'smithi045' 2024-04-16T17:13:47.603 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:47.601+0000 7f007c5c7740 1 --2- 172.21.15.45:0/1421559086 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x5633f9150960 0x5633f9152d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:13:47.603 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:47.601+0000 7f007c5c7740 4 client.27508 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-16T17:13:47.603 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:47.601+0000 7f007c5c7740 1 -- 172.21.15.45:0/1421559086 --> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5633f9034010 con 0x5633f9150960 2024-04-16T17:13:47.603 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:47.602+0000 7f0072000640 1 --2- 172.21.15.45:0/1421559086 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x5633f9150960 0x5633f9152d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:13:47.603 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:47.603+0000 7f0072000640 1 --2- 172.21.15.45:0/1421559086 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x5633f9150960 0x5633f9152d50 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c70559154424dbb4 server_cookie=dd82a35d615f853b in_seq=0 out_seq=0 2024-04-16T17:13:47.603 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:47.603+0000 7f005cc00640 10 client.27508 ms_handle_connect on v2:172.21.15.104:6838/3660070592 2024-04-16T17:13:47.695 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:47.694+0000 7f005cc00640 1 -- 172.21.15.45:0/1421559086 <== mds.0 v2:172.21.15.104:6838/3660070592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5633f9034010 con 0x5633f9150960 2024-04-16T17:13:47.695 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:47.694+0000 7f005cc00640 10 client.27508 handle_command_reply: tid=0 2024-04-16T17:13:47.696 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:47.694+0000 7f007c5c7740 1 client.27508 shutdown 2024-04-16T17:13:47.696 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:47.695+0000 7f007c5c7740 1 -- 172.21.15.45:0/1421559086 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5633f90c7a80 msgr2=0x5633f90dda40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:47.696 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:47.695+0000 7f007c5c7740 1 --2- 172.21.15.45:0/1421559086 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5633f90c7a80 0x5633f90dda40 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f006c0058a0 tx=0x7f006c005810 comp rx=0 tx=0).stop 2024-04-16T17:13:47.696 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:47.695+0000 7f007c5c7740 1 -- 172.21.15.45:0/1421559086 shutdown_connections 2024-04-16T17:13:47.696 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:47.695+0000 7f007c5c7740 1 --2- 172.21.15.45:0/1421559086 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5633f90d9810 0x5633f914f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:47.696 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:47.695+0000 7f007c5c7740 1 --2- 172.21.15.45:0/1421559086 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5633f90c87f0 0x5633f90ddf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:47.696 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:47.695+0000 7f007c5c7740 1 --2- 172.21.15.45:0/1421559086 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x5633f9150960 0x5633f9152d50 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:47.696 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:47.695+0000 7f007c5c7740 1 --2- 172.21.15.45:0/1421559086 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5633f90c7a80 0x5633f90dda40 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:47.696 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:47.695+0000 7f007c5c7740 1 -- 172.21.15.45:0/1421559086 >> 172.21.15.45:0/1421559086 conn(0x5633f90625d0 msgr2=0x5633f90d6f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:13:47.697 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:47.695+0000 7f005cc00640 0 client.27508 ms_handle_reset on v2:172.21.15.104:6838/3660070592 2024-04-16T17:13:47.697 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:47.695+0000 7f007c5c7740 1 -- 172.21.15.45:0/1421559086 shutdown_connections 2024-04-16T17:13:47.697 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:47.695+0000 7f007c5c7740 1 -- 172.21.15.45:0/1421559086 wait complete. 2024-04-16T17:13:47.697 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:47.695+0000 7f007c5c7740 20 client.27508 trim_cache size 0 max 16384 2024-04-16T17:13:47.697 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-04-16T17:13:47.697 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-04-16T17:13:47.697 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-04-16T17:13:47.697 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-04-16T17:13:47.698 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:47.696+0000 7f0073e00640 1 -- 172.21.15.45:0/3519570806 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f00580a3d10 msgr2=0x7f00580a61d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:47.698 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:47.696+0000 7f0073e00640 1 --2- 172.21.15.45:0/3519570806 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f00580a3d10 0x7f00580a61d0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f006409c900 tx=0x7f00640d4b70 comp rx=0 tx=0).stop 2024-04-16T17:13:47.698 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:47.696+0000 7f0073e00640 1 -- 172.21.15.45:0/3519570806 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0074102fc0 msgr2=0x7f00741a0620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:13:47.698 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:47.696+0000 7f0073e00640 1 --2- 172.21.15.45:0/3519570806 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0074102fc0 0x7f00741a0620 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f006400e9f0 tx=0x7f006400edf0 comp rx=0 tx=0).stop 2024-04-16T17:13:47.698 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:47.697+0000 7f0073e00640 1 -- 172.21.15.45:0/3519570806 shutdown_connections 2024-04-16T17:13:47.698 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:47.697+0000 7f0073e00640 1 --2- 172.21.15.45:0/3519570806 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f00580a3d10 0x7f00580a61d0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:47.698 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:47.697+0000 7f0073e00640 1 --2- 172.21.15.45:0/3519570806 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f007410c3e0 0x7f007419a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:47.698 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:47.697+0000 7f0073e00640 1 --2- 172.21.15.45:0/3519570806 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f00741038e0 0x7f00741a0b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:47.698 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:47.697+0000 7f0073e00640 1 --2- 172.21.15.45:0/3519570806 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0074102fc0 0x7f00741a0620 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:13:47.698 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:47.697+0000 7f0073e00640 1 -- 172.21.15.45:0/3519570806 >> 172.21.15.45:0/3519570806 conn(0x7f00740fc820 msgr2=0x7f0074101c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:13:47.699 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:47.697+0000 7f0073e00640 1 -- 172.21.15.45:0/3519570806 shutdown_connections 2024-04-16T17:13:47.699 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:13:47.697+0000 7f0073e00640 1 -- 172.21.15.45:0/3519570806 wait complete. 2024-04-16T17:13:48.298 INFO:tasks.workunit.client.0.smithi045.stdout: 1 120670 1.83 MB/sec execute 230 sec latency 90.826 ms 2024-04-16T17:13:48.407 INFO:journalctl@ceph.mds.h.smithi104.stdout:Apr 16 17:13:47 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mds-h[67477]: ignoring --setuser ceph since I am not root 2024-04-16T17:13:48.407 INFO:journalctl@ceph.mds.h.smithi104.stdout:Apr 16 17:13:47 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mds-h[67477]: ignoring --setgroup ceph since I am not root 2024-04-16T17:13:48.407 INFO:journalctl@ceph.mds.h.smithi104.stdout:Apr 16 17:13:47 smithi104 ceph-mds[67481]: -- 172.21.15.104:0/2848870246 <== mon.2 v2:172.21.15.104:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55687762fa40 con 0x556877602c00 2024-04-16T17:13:48.407 INFO:journalctl@ceph.mds.h.smithi104.stdout:Apr 16 17:13:47 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mds-h[67477]: starting mds.h at 2024-04-16T17:13:48.677 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:13:48 smithi104 ceph-mon[36457]: pgmap v216: 129 pgs: 129 active+clean; 64 MiB data, 756 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 133 op/s 2024-04-16T17:13:48.677 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:13:48 smithi104 ceph-mon[36457]: daemon mds.h restarted 2024-04-16T17:13:48.804 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:13:48 smithi117 ceph-mon[36730]: pgmap v216: 129 pgs: 129 active+clean; 64 MiB data, 756 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 133 op/s 2024-04-16T17:13:48.804 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:13:48 smithi117 ceph-mon[36730]: daemon mds.h restarted 2024-04-16T17:13:48.814 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:13:48 smithi045 ceph-mon[31676]: pgmap v216: 129 pgs: 129 active+clean; 64 MiB data, 756 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 133 op/s 2024-04-16T17:13:48.814 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:13:48 smithi045 ceph-mon[31676]: daemon mds.h restarted 2024-04-16T17:13:49.298 INFO:tasks.workunit.client.0.smithi045.stdout: 1 121009 1.83 MB/sec execute 231 sec latency 82.287 ms 2024-04-16T17:13:49.541 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:13:49 smithi045 ceph-mon[31676]: osdmap e98: 12 total, 12 up, 12 in 2024-04-16T17:13:49.541 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:13:49 smithi045 ceph-mon[31676]: mds.? [v2:172.21.15.104:6836/1833645246,v1:172.21.15.104:6837/1833645246] up:boot 2024-04-16T17:13:49.541 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:13:49 smithi045 ceph-mon[31676]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 2 up:standby-replay 7 up:standby 2024-04-16T17:13:49.541 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:13:49 smithi045 ceph-mon[31676]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-16T17:13:49.541 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:13:49 smithi045 ceph-mon[31676]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-16T17:13:49.677 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:13:49 smithi104 ceph-mon[36457]: osdmap e98: 12 total, 12 up, 12 in 2024-04-16T17:13:49.677 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:13:49 smithi104 ceph-mon[36457]: mds.? [v2:172.21.15.104:6836/1833645246,v1:172.21.15.104:6837/1833645246] up:boot 2024-04-16T17:13:49.677 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:13:49 smithi104 ceph-mon[36457]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 2 up:standby-replay 7 up:standby 2024-04-16T17:13:49.677 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:13:49 smithi104 ceph-mon[36457]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-16T17:13:49.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:13:49 smithi104 ceph-mon[36457]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-16T17:13:49.804 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:13:49 smithi117 ceph-mon[36730]: osdmap e98: 12 total, 12 up, 12 in 2024-04-16T17:13:49.804 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:13:49 smithi117 ceph-mon[36730]: mds.? [v2:172.21.15.104:6836/1833645246,v1:172.21.15.104:6837/1833645246] up:boot 2024-04-16T17:13:49.804 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:13:49 smithi117 ceph-mon[36730]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 2 up:standby-replay 7 up:standby 2024-04-16T17:13:49.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:13:49 smithi117 ceph-mon[36730]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-16T17:13:49.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:13:49 smithi117 ceph-mon[36730]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-16T17:13:50.298 INFO:tasks.workunit.client.0.smithi045.stdout: 1 121142 1.82 MB/sec execute 232 sec latency 99.861 ms 2024-04-16T17:13:50.804 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:13:50 smithi117 ceph-mon[36730]: pgmap v218: 129 pgs: 129 active+clean; 64 MiB data, 756 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 113 op/s 2024-04-16T17:13:50.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:13:50 smithi104 ceph-mon[36457]: pgmap v218: 129 pgs: 129 active+clean; 64 MiB data, 756 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 113 op/s 2024-04-16T17:13:51.298 INFO:tasks.workunit.client.0.smithi045.stdout: 1 121613 1.81 MB/sec execute 233 sec latency 76.141 ms 2024-04-16T17:13:51.470 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:13:50 smithi045 ceph-mon[31676]: pgmap v218: 129 pgs: 129 active+clean; 64 MiB data, 756 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 113 op/s 2024-04-16T17:13:52.299 INFO:tasks.workunit.client.0.smithi045.stdout: 1 122251 1.83 MB/sec execute 234 sec latency 31.550 ms 2024-04-16T17:13:52.804 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:13:52 smithi117 ceph-mon[36730]: pgmap v219: 129 pgs: 129 active+clean; 59 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 789 KiB/s rd, 907 KiB/s wr, 131 op/s 2024-04-16T17:13:52.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:13:52 smithi104 ceph-mon[36457]: pgmap v219: 129 pgs: 129 active+clean; 59 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 789 KiB/s rd, 907 KiB/s wr, 131 op/s 2024-04-16T17:13:53.105 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:13:52 smithi045 ceph-mon[31676]: pgmap v219: 129 pgs: 129 active+clean; 59 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 789 KiB/s rd, 907 KiB/s wr, 131 op/s 2024-04-16T17:13:53.298 INFO:tasks.workunit.client.0.smithi045.stdout: 1 122695 1.82 MB/sec execute 235 sec latency 60.228 ms 2024-04-16T17:13:54.299 INFO:tasks.workunit.client.0.smithi045.stdout: 1 123022 1.82 MB/sec execute 236 sec latency 60.089 ms 2024-04-16T17:13:54.804 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:13:54 smithi117 ceph-mon[36730]: pgmap v220: 129 pgs: 129 active+clean; 59 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 789 KiB/s rd, 907 KiB/s wr, 131 op/s 2024-04-16T17:13:54.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:13:54 smithi104 ceph-mon[36457]: pgmap v220: 129 pgs: 129 active+clean; 59 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 789 KiB/s rd, 907 KiB/s wr, 131 op/s 2024-04-16T17:13:55.299 INFO:tasks.workunit.client.0.smithi045.stdout: 1 123919 1.84 MB/sec execute 237 sec latency 62.409 ms 2024-04-16T17:13:56.299 INFO:tasks.workunit.client.0.smithi045.stdout: 1 124505 1.85 MB/sec execute 238 sec latency 43.242 ms 2024-04-16T17:13:56.484 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:13:54 smithi045 ceph-mon[31676]: pgmap v220: 129 pgs: 129 active+clean; 59 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 789 KiB/s rd, 907 KiB/s wr, 131 op/s 2024-04-16T17:13:56.485 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:13:56 smithi045 ceph-mon[31676]: pgmap v221: 129 pgs: 129 active+clean; 59 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 789 KiB/s rd, 907 KiB/s wr, 131 op/s 2024-04-16T17:13:56.804 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:13:56 smithi117 ceph-mon[36730]: pgmap v221: 129 pgs: 129 active+clean; 59 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 789 KiB/s rd, 907 KiB/s wr, 131 op/s 2024-04-16T17:13:56.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:13:56 smithi104 ceph-mon[36457]: pgmap v221: 129 pgs: 129 active+clean; 59 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 789 KiB/s rd, 907 KiB/s wr, 131 op/s 2024-04-16T17:13:57.299 INFO:tasks.workunit.client.0.smithi045.stdout: 1 124865 1.85 MB/sec execute 239 sec latency 36.264 ms 2024-04-16T17:13:58.299 INFO:tasks.workunit.client.0.smithi045.stdout: 1 125466 1.85 MB/sec execute 240 sec latency 37.475 ms 2024-04-16T17:13:58.804 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:13:58 smithi117 ceph-mon[36730]: pgmap v222: 129 pgs: 129 active+clean; 63 MiB data, 778 MiB used, 1.0 TiB / 1.0 TiB avail; 761 KiB/s rd, 973 KiB/s wr, 122 op/s 2024-04-16T17:13:58.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:13:58 smithi104 ceph-mon[36457]: pgmap v222: 129 pgs: 129 active+clean; 63 MiB data, 778 MiB used, 1.0 TiB / 1.0 TiB avail; 761 KiB/s rd, 973 KiB/s wr, 122 op/s 2024-04-16T17:13:59.299 INFO:tasks.workunit.client.0.smithi045.stdout: 1 125843 1.85 MB/sec execute 241 sec latency 82.380 ms 2024-04-16T17:14:00.299 INFO:tasks.workunit.client.0.smithi045.stdout: 1 126154 1.85 MB/sec execute 242 sec latency 75.818 ms 2024-04-16T17:14:00.469 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:13:58 smithi045 ceph-mon[31676]: pgmap v222: 129 pgs: 129 active+clean; 63 MiB data, 778 MiB used, 1.0 TiB / 1.0 TiB avail; 761 KiB/s rd, 973 KiB/s wr, 122 op/s 2024-04-16T17:14:00.470 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:14:00 smithi045 ceph-mon[31676]: pgmap v223: 129 pgs: 129 active+clean; 63 MiB data, 778 MiB used, 1.0 TiB / 1.0 TiB avail; 696 KiB/s rd, 890 KiB/s wr, 111 op/s 2024-04-16T17:14:00.666 INFO:tasks.workunit.client.0.smithi045.stdout:[126237] write failed on handle 11227 (Resource temporarily unavailable) 2024-04-16T17:14:00.666 INFO:tasks.workunit.client.0.smithi045.stdout:Child failed with status 1 2024-04-16T17:14:00.668 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-16T17:14:00.668 INFO:tasks.workunit:Stopping ['suites/dbench.sh'] on client.0... 2024-04-16T17:14:00.668 DEBUG:teuthology.orchestra.run.smithi045:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-04-16T17:14:00.804 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:14:00 smithi117 ceph-mon[36730]: pgmap v223: 129 pgs: 129 active+clean; 63 MiB data, 778 MiB used, 1.0 TiB / 1.0 TiB avail; 696 KiB/s rd, 890 KiB/s wr, 111 op/s 2024-04-16T17:14:00.870 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:14:00.869+0000 7f4c0b200640 1 -- 172.21.15.45:0/3044939039 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4c0c1042a0 msgr2=0x7f4c0c104680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:00.870 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:14:00.869+0000 7f4c0b200640 1 --2- 172.21.15.45:0/3044939039 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4c0c1042a0 0x7f4c0c104680 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f4bfc009900 tx=0x7f4bfc02bc30 comp rx=0 tx=0).stop 2024-04-16T17:14:00.870 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:14:00.869+0000 7f4c0b200640 1 -- 172.21.15.45:0/3044939039 shutdown_connections 2024-04-16T17:14:00.870 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:14:00.869+0000 7f4c0b200640 1 --2- 172.21.15.45:0/3044939039 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4c0c104c50 0x7f4c0c108c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:00.870 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:14:00.869+0000 7f4c0b200640 1 --2- 172.21.15.45:0/3044939039 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4c0c109370 0x7f4c0c110f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:00.870 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:14:00.869+0000 7f4c0b200640 1 --2- 172.21.15.45:0/3044939039 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4c0c1042a0 0x7f4c0c104680 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:00.870 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:14:00.869+0000 7f4c0b200640 1 -- 172.21.15.45:0/3044939039 >> 172.21.15.45:0/3044939039 conn(0x7f4c0c0ffae0 msgr2=0x7f4c0c101f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:14:00.871 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:14:00.869+0000 7f4c0b200640 1 -- 172.21.15.45:0/3044939039 shutdown_connections 2024-04-16T17:14:00.871 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:14:00.869+0000 7f4c0b200640 1 -- 172.21.15.45:0/3044939039 wait complete. 2024-04-16T17:14:00.873 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:14:00.870+0000 7f4c0b200640 1 Processor -- start 2024-04-16T17:14:00.873 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:14:00.870+0000 7f4c0b200640 1 -- start start 2024-04-16T17:14:00.873 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:14:00.870+0000 7f4c0b200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4c0c1042a0 0x7f4c0c1a0690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:00.873 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:14:00.870+0000 7f4c0b200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4c0c109370 0x7f4c0c1a0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:00.873 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:14:00.870+0000 7f4c0b200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4c0c19a850 0x7f4c0c19ad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:00.873 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:14:00.870+0000 7f4c0b200640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f4c0c114470 con 0x7f4c0c109370 2024-04-16T17:14:00.873 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:14:00.870+0000 7f4c0b200640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f4c0c1142f0 con 0x7f4c0c1042a0 2024-04-16T17:14:00.873 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:14:00.870+0000 7f4c0b200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f4c0c1145f0 con 0x7f4c0c19a850 2024-04-16T17:14:00.874 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:14:00.871+0000 7f4c09400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4c0c109370 0x7f4c0c1a0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:00.874 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:14:00.871+0000 7f4c09e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4c0c1042a0 0x7f4c0c1a0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:00.874 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:14:00.871+0000 7f4c09e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4c0c1042a0 0x7f4c0c1a0690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:48930/0 (socket says 172.21.15.45:48930) 2024-04-16T17:14:00.874 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:14:00.871+0000 7f4c09e00640 1 -- 172.21.15.45:0/775271414 learned_addr learned my addr 172.21.15.45:0/775271414 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:14:00.874 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:14:00.871+0000 7f4c09400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4c0c109370 0x7f4c0c1a0bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:46754/0 (socket says 172.21.15.45:46754) 2024-04-16T17:14:00.874 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:14:00.871+0000 7f4c0a800640 1 --2- 172.21.15.45:0/775271414 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4c0c19a850 0x7f4c0c19ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:00.874 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:14:00.871+0000 7f4c09400640 1 -- 172.21.15.45:0/775271414 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4c0c19a850 msgr2=0x7f4c0c19ad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:00.874 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:14:00.871+0000 7f4c09400640 1 --2- 172.21.15.45:0/775271414 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4c0c19a850 0x7f4c0c19ad00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:00.874 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:14:00.871+0000 7f4c09400640 1 -- 172.21.15.45:0/775271414 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4c0c1042a0 msgr2=0x7f4c0c1a0690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:00.874 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:14:00.871+0000 7f4c09400640 1 --2- 172.21.15.45:0/775271414 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4c0c1042a0 0x7f4c0c1a0690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:00.874 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:14:00.871+0000 7f4c09400640 1 -- 172.21.15.45:0/775271414 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c0c19b3e0 con 0x7f4c0c109370 2024-04-16T17:14:00.874 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:14:00.871+0000 7f4c09e00640 1 --2- 172.21.15.45:0/775271414 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4c0c1042a0 0x7f4c0c1a0690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:14:00.874 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:14:00.872+0000 7f4c09400640 1 --2- 172.21.15.45:0/775271414 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4c0c109370 0x7f4c0c1a0bd0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f4bf800e9c0 tx=0x7f4bf800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:14:00.875 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:14:00.872+0000 7f4c02a00640 1 -- 172.21.15.45:0/775271414 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bf800ccc0 con 0x7f4c0c109370 2024-04-16T17:14:00.875 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:14:00.872+0000 7f4c02a00640 1 -- 172.21.15.45:0/775271414 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4bf8005020 con 0x7f4c0c109370 2024-04-16T17:14:00.875 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:14:00.872+0000 7f4c0b200640 1 -- 172.21.15.45:0/775271414 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c0c19b6d0 con 0x7f4c0c109370 2024-04-16T17:14:00.875 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:14:00.872+0000 7f4c02a00640 1 -- 172.21.15.45:0/775271414 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bf8004a00 con 0x7f4c0c109370 2024-04-16T17:14:00.875 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:14:00.872+0000 7f4c0b200640 1 -- 172.21.15.45:0/775271414 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c0c1a7810 con 0x7f4c0c109370 2024-04-16T17:14:00.876 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:14:00.877+0000 7f4c02a00640 1 -- 172.21.15.45:0/775271414 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f4bf80051d0 con 0x7f4c0c109370 2024-04-16T17:14:00.877 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:14:00.877+0000 7f4c13b70740 1 Processor -- start 2024-04-16T17:14:00.877 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:14:00.877+0000 7f4c13b70740 1 -- start start 2024-04-16T17:14:00.878 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:14:00.877+0000 7f4c02a00640 1 --2- 172.21.15.45:0/775271414 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f4bf00a3d10 0x7f4bf00a61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:00.878 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:14:00.877+0000 7f4c13b70740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5650e89247f0 0x5650e8923540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:00.878 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:14:00.877+0000 7f4c02a00640 1 -- 172.21.15.45:0/775271414 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9408+0+0 (secure 0 0 0) 0x7f4bf80d0b90 con 0x7f4c0c109370 2024-04-16T17:14:00.878 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:14:00.877+0000 7f4c13b70740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5650e8923a80 0x5650e8923f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:00.878 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:14:00.877+0000 7f4c13b70740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5650e892d000 0x5650e892f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:00.878 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:14:00.877+0000 7f4c13b70740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5650e8888780 con 0x5650e892d000 2024-04-16T17:14:00.879 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:14:00.877+0000 7f4c13b70740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5650e882ef40 con 0x5650e89247f0 2024-04-16T17:14:00.879 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:14:00.877+0000 7f4c13b70740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5650e8884740 con 0x5650e8923a80 2024-04-16T17:14:00.879 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:14:00.878+0000 7f4c0a800640 1 --2- 172.21.15.45:0/775271414 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f4bf00a3d10 0x7f4bf00a61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:00.880 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:14:00.878+0000 7f4c09e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5650e89247f0 0x5650e8923540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:00.880 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:14:00.878+0000 7f4c09e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5650e89247f0 0x5650e8923540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:48938/0 (socket says 172.21.15.45:48938) 2024-04-16T17:14:00.880 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:14:00.878+0000 7f4c0a800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5650e892d000 0x5650e892f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:00.880 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:14:00.878+0000 7f4c09e00640 1 -- 172.21.15.45:0/1595842114 learned_addr learned my addr 172.21.15.45:0/1595842114 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:14:00.880 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:14:00.878+0000 7f4c09400640 1 --2- 172.21.15.45:0/1595842114 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5650e8923a80 0x5650e8923f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:00.880 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:14:00.878+0000 7f4c09e00640 1 -- 172.21.15.45:0/1595842114 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5650e8923a80 msgr2=0x5650e8923f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:14:00.880 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:14:00.878+0000 7f4c09e00640 1 --2- 172.21.15.45:0/1595842114 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5650e8923a80 0x5650e8923f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:00.880 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:14:00.878+0000 7f4c09e00640 1 -- 172.21.15.45:0/1595842114 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5650e892d000 msgr2=0x5650e892f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:14:00.880 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:14:00.878+0000 7f4c09e00640 1 --2- 172.21.15.45:0/1595842114 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5650e892d000 0x5650e892f3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:00.880 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:14:00.878+0000 7f4c09e00640 1 -- 172.21.15.45:0/1595842114 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5650e859b210 con 0x5650e89247f0 2024-04-16T17:14:00.880 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:14:00.879+0000 7f4c0a800640 1 --2- 172.21.15.45:0/775271414 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f4bf00a3d10 0x7f4bf00a61d0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f4c04005fd0 tx=0x7f4c04009e90 comp rx=0 tx=0).ready entity=mgr.25411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:14:00.880 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:14:00.879+0000 7f4c09e00640 1 --2- 172.21.15.45:0/1595842114 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5650e89247f0 0x5650e8923540 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f4bfc009620 tx=0x7f4bfc009650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ff15993549a881c1 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:14:00.881 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:14:00.879+0000 7f4bf7400640 1 -- 172.21.15.45:0/1595842114 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bfc045070 con 0x5650e89247f0 2024-04-16T17:14:00.881 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:14:00.879+0000 7f4bf7400640 1 -- 172.21.15.45:0/1595842114 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4bfc02c5d0 con 0x5650e89247f0 2024-04-16T17:14:00.881 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:14:00.879+0000 7f4bf7400640 1 -- 172.21.15.45:0/1595842114 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bfc0051c0 con 0x5650e89247f0 2024-04-16T17:14:00.882 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:14:00.880+0000 7f4c13b70740 1 -- 172.21.15.45:0/1595842114 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5650e89247f0 msgr2=0x5650e8923540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:00.882 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:14:00.880+0000 7f4c13b70740 1 --2- 172.21.15.45:0/1595842114 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5650e89247f0 0x5650e8923540 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f4bfc009620 tx=0x7f4bfc009650 comp rx=0 tx=0).stop 2024-04-16T17:14:00.882 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:14:00.880+0000 7f4c13b70740 1 -- 172.21.15.45:0/1595842114 shutdown_connections 2024-04-16T17:14:00.882 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:14:00.880+0000 7f4c13b70740 1 --2- 172.21.15.45:0/1595842114 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5650e8923a80 0x5650e8923f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:00.882 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:14:00.880+0000 7f4c13b70740 1 --2- 172.21.15.45:0/1595842114 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5650e892d000 0x5650e892f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:00.882 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:14:00.880+0000 7f4c13b70740 1 --2- 172.21.15.45:0/1595842114 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5650e89247f0 0x5650e8923540 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:00.882 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:14:00.880+0000 7f4c13b70740 1 -- 172.21.15.45:0/1595842114 >> 172.21.15.45:0/1595842114 conn(0x5650e88be5d0 msgr2=0x5650e891e770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:14:00.882 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:14:00.880+0000 7f4c13b70740 1 -- 172.21.15.45:0/1595842114 shutdown_connections 2024-04-16T17:14:00.882 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:14:00.880+0000 7f4c13b70740 1 -- 172.21.15.45:0/1595842114 wait complete. 2024-04-16T17:14:00.883 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:14:00.881+0000 7f4c13b70740 1 -- start start 2024-04-16T17:14:00.883 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:14:00.881+0000 7f4c13b70740 1 Processor -- start 2024-04-16T17:14:00.885 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:14:00.882+0000 7f4c13b70740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5650e8923a80 0x5650e8929e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:00.885 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:14:00.882+0000 7f4c13b70740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5650e89247f0 0x5650e892a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:00.885 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:14:00.882+0000 7f4c13b70740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5650e892d000 0x5650e892a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:00.885 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:14:00.882+0000 7f4c13b70740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5650e892fcd0 con 0x5650e89247f0 2024-04-16T17:14:00.885 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:14:00.882+0000 7f4c13b70740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5650e8842390 con 0x5650e892d000 2024-04-16T17:14:00.885 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:14:00.882+0000 7f4c13b70740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5650e892fe50 con 0x5650e8923a80 2024-04-16T17:14:00.885 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:14:00.882+0000 7f4c13b70740 10 client.0 _ll_register_callbacks cb 0x5650e882bf60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:14:00.885 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:14:00.882+0000 7f4c09e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5650e8923a80 0x5650e8929e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:00.885 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:14:00.882+0000 7f4c09e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5650e8923a80 0x5650e8929e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:45500/0 (socket says 172.21.15.45:45500) 2024-04-16T17:14:00.886 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:14:00.882+0000 7f4c09e00640 1 -- 172.21.15.45:0/2502505065 learned_addr learned my addr 172.21.15.45:0/2502505065 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:14:00.886 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:14:00.882+0000 7f4c09400640 1 --2- 172.21.15.45:0/2502505065 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5650e89247f0 0x5650e892a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:00.886 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:14:00.883+0000 7f4c0a800640 1 --2- 172.21.15.45:0/2502505065 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5650e892d000 0x5650e892a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:00.886 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:14:00.883+0000 7f4c09400640 1 -- 172.21.15.45:0/2502505065 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5650e8923a80 msgr2=0x5650e8929e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:14:00.886 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:14:00.883+0000 7f4c09400640 1 --2- 172.21.15.45:0/2502505065 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5650e8923a80 0x5650e8929e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:00.886 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:14:00.883+0000 7f4c09400640 1 -- 172.21.15.45:0/2502505065 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5650e892d000 msgr2=0x5650e892a8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:14:00.886 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:14:00.883+0000 7f4c09400640 1 --2- 172.21.15.45:0/2502505065 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5650e892d000 0x5650e892a8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:00.886 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:14:00.883+0000 7f4c09400640 1 -- 172.21.15.45:0/2502505065 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5650e8667620 con 0x5650e89247f0 2024-04-16T17:14:00.886 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:14:00.883+0000 7f4c09e00640 1 --2- 172.21.15.45:0/2502505065 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5650e8923a80 0x5650e8929e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:14:00.886 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:14:00.883+0000 7f4c0a800640 1 --2- 172.21.15.45:0/2502505065 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5650e892d000 0x5650e892a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:14:00.886 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:14:00.883+0000 7f4c09400640 1 --2- 172.21.15.45:0/2502505065 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5650e89247f0 0x5650e892a3b0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f4bf80cdf30 tx=0x7f4bf809cfc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4824340b6142814 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:14:00.887 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:14:00.883+0000 7f4bebe00640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-16T17:14:00.887 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:14:00.883+0000 7f4bebe00640 1 -- 172.21.15.45:0/2502505065 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bf801c040 con 0x5650e89247f0 2024-04-16T17:14:00.887 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:14:00.883+0000 7f4bebe00640 1 -- 172.21.15.45:0/2502505065 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9408+0+0 (secure 0 0 0) 0x7f4bf8098df0 con 0x5650e89247f0 2024-04-16T17:14:00.887 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:14:00.884+0000 7f4c13b70740 1 -- 172.21.15.45:0/2502505065 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4bfc02c5d0 con 0x5650e89247f0 2024-04-16T17:14:00.887 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:14:00.884+0000 7f4bebe00640 1 -- 172.21.15.45:0/2502505065 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f4bf8004ba0 con 0x5650e89247f0 2024-04-16T17:14:00.887 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:14:00.884+0000 7f4c13b70740 10 client.17415 fetch_fsmap learned FSMap version 42 2024-04-16T17:14:00.887 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:14:00.884+0000 7f4c13b70740 1 -- 172.21.15.45:0/2502505065 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x5650e893be00 con 0x5650e89247f0 2024-04-16T17:14:00.887 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:14:00.884+0000 7f4bebe00640 1 -- 172.21.15.45:0/2502505065 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4bfc02c5d0 con 0x5650e89247f0 2024-04-16T17:14:00.887 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:14:00.885+0000 7f4bebe00640 1 -- 172.21.15.45:0/2502505065 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 42) ==== 8362+0+0 (secure 0 0 0) 0x7f4bf8098040 con 0x5650e89247f0 2024-04-16T17:14:00.889 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:14:00.885+0000 7f4c13b70740 10 client.17415 fetch_fsmap finished waiting for FSMap version 42 2024-04-16T17:14:00.889 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:14:00.885+0000 7f4c13b70740 10 client.17415 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T17:14:00.889 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:14:00.885+0000 7f4c13b70740 20 client.17415 populate_metadata read hostname 'smithi045' 2024-04-16T17:14:00.889 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:14:00.885+0000 7f4c13b70740 1 --2- 172.21.15.45:0/2502505065 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x5650e893d310 0x5650e893f700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:00.889 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:14:00.885+0000 7f4c13b70740 4 client.17415 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:14:00.889 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:14:00.885+0000 7f4c13b70740 1 -- 172.21.15.45:0/2502505065 --> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5650e88961a0 con 0x5650e893d310 2024-04-16T17:14:00.889 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:14:00.885+0000 7f4c09e00640 1 --2- 172.21.15.45:0/2502505065 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x5650e893d310 0x5650e893f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:00.889 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:14:00.886+0000 7f4c09e00640 1 --2- 172.21.15.45:0/2502505065 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x5650e893d310 0x5650e893f700 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3098569e805cda76 server_cookie=c50051e01c00a550 in_seq=0 out_seq=0 2024-04-16T17:14:00.890 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:14:00.886+0000 7f4bebe00640 10 client.17415 ms_handle_connect on v2:172.21.15.104:6838/3660070592 2024-04-16T17:14:00.890 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:14:00.888+0000 7f4bebe00640 1 -- 172.21.15.45:0/2502505065 <== mds.0 v2:172.21.15.104:6838/3660070592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5650e88961a0 con 0x5650e893d310 2024-04-16T17:14:00.890 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:14:00.888+0000 7f4bebe00640 10 client.17415 handle_command_reply: tid=0 2024-04-16T17:14:00.890 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:14:00.888+0000 7f4c13b70740 1 client.17415 shutdown 2024-04-16T17:14:00.890 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:14:00.888+0000 7f4c13b70740 1 -- 172.21.15.45:0/2502505065 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5650e89247f0 msgr2=0x5650e892a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:00.890 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:14:00.888+0000 7f4c13b70740 1 --2- 172.21.15.45:0/2502505065 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5650e89247f0 0x5650e892a3b0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f4bf80cdf30 tx=0x7f4bf809cfc0 comp rx=0 tx=0).stop 2024-04-16T17:14:00.890 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:14:00.888+0000 7f4c13b70740 1 -- 172.21.15.45:0/2502505065 shutdown_connections 2024-04-16T17:14:00.890 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:14:00.888+0000 7f4c13b70740 1 --2- 172.21.15.45:0/2502505065 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5650e892d000 0x5650e892a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:00.890 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:14:00.888+0000 7f4c13b70740 1 --2- 172.21.15.45:0/2502505065 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5650e89247f0 0x5650e892a3b0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:00.890 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:14:00.888+0000 7f4c13b70740 1 --2- 172.21.15.45:0/2502505065 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x5650e893d310 0x5650e893f700 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:00.890 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:14:00.888+0000 7f4c13b70740 1 --2- 172.21.15.45:0/2502505065 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5650e8923a80 0x5650e8929e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:00.890 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:14:00.888+0000 7f4bebe00640 0 client.17415 ms_handle_reset on v2:172.21.15.104:6838/3660070592 2024-04-16T17:14:00.891 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:14:00.888+0000 7f4c13b70740 1 -- 172.21.15.45:0/2502505065 >> 172.21.15.45:0/2502505065 conn(0x5650e88be5d0 msgr2=0x5650e89204c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:14:00.891 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:14:00.888+0000 7f4c13b70740 1 -- 172.21.15.45:0/2502505065 shutdown_connections 2024-04-16T17:14:00.891 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:14:00.888+0000 7f4c13b70740 1 -- 172.21.15.45:0/2502505065 wait complete. 2024-04-16T17:14:00.891 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:14:00.888+0000 7f4c13b70740 20 client.17415 trim_cache size 0 max 16384 2024-04-16T17:14:00.905 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:14:00.904+0000 7f4c13b70740 1 Processor -- start 2024-04-16T17:14:00.905 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:14:00.904+0000 7f4c13b70740 1 -- start start 2024-04-16T17:14:00.905 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:14:00.904+0000 7f4c13b70740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5650e8923a80 0x5650e8938bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:00.905 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:14:00.904+0000 7f4c13b70740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5650e89247f0 0x5650e892a710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:00.905 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:14:00.904+0000 7f4c13b70740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5650e892d000 0x5650e8944fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:00.906 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:14:00.904+0000 7f4c13b70740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f4c0c1b3d60 con 0x5650e89247f0 2024-04-16T17:14:00.906 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:14:00.904+0000 7f4c13b70740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f4c0c1b15d0 con 0x5650e8923a80 2024-04-16T17:14:00.906 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:14:00.904+0000 7f4c13b70740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f4c0c1af400 con 0x5650e892d000 2024-04-16T17:14:00.906 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:14:00.904+0000 7f4c09e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5650e8923a80 0x5650e8938bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:00.906 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:14:00.904+0000 7f4c09e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5650e8923a80 0x5650e8938bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:48968/0 (socket says 172.21.15.45:48968) 2024-04-16T17:14:00.906 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:14:00.904+0000 7f4c09e00640 1 -- 172.21.15.45:0/1288377963 learned_addr learned my addr 172.21.15.45:0/1288377963 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:14:00.906 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:14:00.904+0000 7f4c0a800640 1 --2- 172.21.15.45:0/1288377963 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5650e892d000 0x5650e8944fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:00.906 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:14:00.905+0000 7f4c09400640 1 --2- 172.21.15.45:0/1288377963 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5650e89247f0 0x5650e892a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:00.906 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:14:00.905+0000 7f4c09e00640 1 -- 172.21.15.45:0/1288377963 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5650e892d000 msgr2=0x5650e8944fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:14:00.906 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:14:00.905+0000 7f4c09e00640 1 --2- 172.21.15.45:0/1288377963 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5650e892d000 0x5650e8944fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:00.906 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:14:00.905+0000 7f4c09e00640 1 -- 172.21.15.45:0/1288377963 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5650e89247f0 msgr2=0x5650e892a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:14:00.907 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:14:00.905+0000 7f4c09e00640 1 --2- 172.21.15.45:0/1288377963 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5650e89247f0 0x5650e892a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:00.907 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:14:00.905+0000 7f4c09e00640 1 -- 172.21.15.45:0/1288377963 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5650e8922d50 con 0x5650e8923a80 2024-04-16T17:14:00.911 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:14:00.905+0000 7f4c09e00640 1 --2- 172.21.15.45:0/1288377963 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5650e8923a80 0x5650e8938bd0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f4bfc006010 tx=0x7f4bfc02cb60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b5286cbd0d56eca5 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:14:00.911 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:14:00.905+0000 7f4bf6a00640 1 -- 172.21.15.45:0/1288377963 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bfc045040 con 0x5650e8923a80 2024-04-16T17:14:00.911 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:14:00.905+0000 7f4bf6a00640 1 -- 172.21.15.45:0/1288377963 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4bfc02c8f0 con 0x5650e8923a80 2024-04-16T17:14:00.911 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:14:00.905+0000 7f4bf6a00640 1 -- 172.21.15.45:0/1288377963 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bfc005420 con 0x5650e8923a80 2024-04-16T17:14:00.911 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:14:00.906+0000 7f4c13b70740 1 -- 172.21.15.45:0/1288377963 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5650e8923a80 msgr2=0x5650e8938bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:00.911 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:14:00.906+0000 7f4c13b70740 1 --2- 172.21.15.45:0/1288377963 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5650e8923a80 0x5650e8938bd0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f4bfc006010 tx=0x7f4bfc02cb60 comp rx=0 tx=0).stop 2024-04-16T17:14:00.911 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:14:00.906+0000 7f4c13b70740 1 -- 172.21.15.45:0/1288377963 shutdown_connections 2024-04-16T17:14:00.911 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:14:00.906+0000 7f4c13b70740 1 --2- 172.21.15.45:0/1288377963 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5650e892d000 0x5650e8944fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:00.911 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:14:00.906+0000 7f4c13b70740 1 --2- 172.21.15.45:0/1288377963 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5650e89247f0 0x5650e892a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:00.912 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:14:00.906+0000 7f4c13b70740 1 --2- 172.21.15.45:0/1288377963 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5650e8923a80 0x5650e8938bd0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:00.912 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:14:00.906+0000 7f4c13b70740 1 -- 172.21.15.45:0/1288377963 >> 172.21.15.45:0/1288377963 conn(0x5650e88be5d0 msgr2=0x5650e891d810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:14:00.912 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:14:00.906+0000 7f4c13b70740 1 -- 172.21.15.45:0/1288377963 shutdown_connections 2024-04-16T17:14:00.912 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:14:00.906+0000 7f4c13b70740 1 -- 172.21.15.45:0/1288377963 wait complete. 2024-04-16T17:14:00.912 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:14:00.907+0000 7f4c13b70740 1 -- start start 2024-04-16T17:14:00.912 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:14:00.907+0000 7f4c13b70740 1 Processor -- start 2024-04-16T17:14:00.912 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:14:00.907+0000 7f4c13b70740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5650e8923a80 0x5650e8926be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:00.912 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:14:00.907+0000 7f4c13b70740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5650e89247f0 0x5650e8927120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:00.912 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:14:00.907+0000 7f4c13b70740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5650e892d000 0x5650e8927660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:00.912 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:14:00.907+0000 7f4c13b70740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f4c0c1afa90 con 0x5650e8923a80 2024-04-16T17:14:00.912 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:14:00.907+0000 7f4c13b70740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f4c0c1b0f00 con 0x5650e89247f0 2024-04-16T17:14:00.912 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:14:00.907+0000 7f4c13b70740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f4c0c1b0160 con 0x5650e892d000 2024-04-16T17:14:00.913 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:14:00.907+0000 7f4c13b70740 10 client.0 _ll_register_callbacks cb 0x5650e892cf50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:14:00.913 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:14:00.908+0000 7f4c09400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5650e89247f0 0x5650e8927120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:00.913 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:14:00.908+0000 7f4c0a800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5650e892d000 0x5650e8927660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:00.913 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:14:00.908+0000 7f4c09e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5650e8923a80 0x5650e8926be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:00.913 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:14:00.908+0000 7f4c09400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5650e89247f0 0x5650e8927120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:48978/0 (socket says 172.21.15.45:48978) 2024-04-16T17:14:00.913 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:14:00.908+0000 7f4c09400640 1 -- 172.21.15.45:0/1138031163 learned_addr learned my addr 172.21.15.45:0/1138031163 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:14:00.913 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:14:00.908+0000 7f4c09e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5650e8923a80 0x5650e8926be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:46800/0 (socket says 172.21.15.45:46800) 2024-04-16T17:14:00.913 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:14:00.908+0000 7f4c0a800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5650e892d000 0x5650e8927660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:45514/0 (socket says 172.21.15.45:45514) 2024-04-16T17:14:00.913 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:14:00.908+0000 7f4c09400640 1 -- 172.21.15.45:0/1138031163 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5650e892d000 msgr2=0x5650e8927660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:14:00.913 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:14:00.908+0000 7f4c09400640 1 --2- 172.21.15.45:0/1138031163 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5650e892d000 0x5650e8927660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:00.913 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:14:00.908+0000 7f4c09400640 1 -- 172.21.15.45:0/1138031163 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5650e8923a80 msgr2=0x5650e8926be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:14:00.913 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:14:00.908+0000 7f4c09400640 1 --2- 172.21.15.45:0/1138031163 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5650e8923a80 0x5650e8926be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:00.913 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:14:00.908+0000 7f4c09400640 1 -- 172.21.15.45:0/1138031163 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5650e89a2940 con 0x5650e89247f0 2024-04-16T17:14:00.914 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:14:00.908+0000 7f4c09e00640 1 --2- 172.21.15.45:0/1138031163 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5650e8923a80 0x5650e8926be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:14:00.914 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:14:00.908+0000 7f4c09400640 1 --2- 172.21.15.45:0/1138031163 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5650e89247f0 0x5650e8927120 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f4bf8099150 tx=0x7f4bf809dc20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=891cc23359b293ed server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:14:00.914 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:14:00.908+0000 7f4bebe00640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-04-16T17:14:00.914 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:14:00.909+0000 7f4bebe00640 1 -- 172.21.15.45:0/1138031163 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bf801c040 con 0x5650e89247f0 2024-04-16T17:14:00.914 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:14:00.909+0000 7f4bebe00640 1 -- 172.21.15.45:0/1138031163 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9408+0+0 (secure 0 0 0) 0x7f4bf80984d0 con 0x5650e89247f0 2024-04-16T17:14:00.914 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:14:00.909+0000 7f4c13b70740 1 -- 172.21.15.45:0/1138031163 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4bfc02c8f0 con 0x5650e89247f0 2024-04-16T17:14:00.914 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:14:00.909+0000 7f4bebe00640 1 -- 172.21.15.45:0/1138031163 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f4bf80ac050 con 0x5650e89247f0 2024-04-16T17:14:00.914 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:14:00.909+0000 7f4c13b70740 10 client.27587 fetch_fsmap learned FSMap version 42 2024-04-16T17:14:00.914 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:14:00.909+0000 7f4c13b70740 1 -- 172.21.15.45:0/1138031163 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x5650e89a3c50 con 0x5650e89247f0 2024-04-16T17:14:00.914 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:14:00.909+0000 7f4bebe00640 1 -- 172.21.15.45:0/1138031163 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4bfc02c8f0 con 0x5650e89247f0 2024-04-16T17:14:00.914 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:14:00.909+0000 7f4bebe00640 1 -- 172.21.15.45:0/1138031163 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 42) ==== 8362+0+0 (secure 0 0 0) 0x7f4bf8098040 con 0x5650e89247f0 2024-04-16T17:14:00.914 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:14:00.910+0000 7f4c13b70740 10 client.27587 fetch_fsmap finished waiting for FSMap version 42 2024-04-16T17:14:00.914 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:14:00.910+0000 7f4c13b70740 10 client.27587 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-16T17:14:00.915 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:14:00.910+0000 7f4c13b70740 20 client.27587 populate_metadata read hostname 'smithi045' 2024-04-16T17:14:00.915 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:14:00.910+0000 7f4c13b70740 1 --2- 172.21.15.45:0/1138031163 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x5650e89a4270 0x5650e89a6690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:00.915 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:14:00.910+0000 7f4c13b70740 4 client.27587 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-16T17:14:00.915 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:14:00.910+0000 7f4c13b70740 1 -- 172.21.15.45:0/1138031163 --> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5650e8890010 con 0x5650e89a4270 2024-04-16T17:14:00.915 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:14:00.910+0000 7f4c09e00640 1 --2- 172.21.15.45:0/1138031163 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x5650e89a4270 0x5650e89a6690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:00.915 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:14:00.910+0000 7f4c09e00640 1 --2- 172.21.15.45:0/1138031163 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x5650e89a4270 0x5650e89a6690 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8461dc1757ebb6fb server_cookie=f3d934fe22eb17af in_seq=0 out_seq=0 2024-04-16T17:14:00.915 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:14:00.910+0000 7f4bebe00640 10 client.27587 ms_handle_connect on v2:172.21.15.104:6838/3660070592 2024-04-16T17:14:00.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:14:00 smithi104 ceph-mon[36457]: pgmap v223: 129 pgs: 129 active+clean; 63 MiB data, 778 MiB used, 1.0 TiB / 1.0 TiB avail; 696 KiB/s rd, 890 KiB/s wr, 111 op/s 2024-04-16T17:14:00.962 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:14:00.961+0000 7f4bebe00640 1 -- 172.21.15.45:0/1138031163 <== mds.0 v2:172.21.15.104:6838/3660070592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5650e8890010 con 0x5650e89a4270 2024-04-16T17:14:00.963 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:14:00.961+0000 7f4bebe00640 10 client.27587 handle_command_reply: tid=0 2024-04-16T17:14:00.963 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:14:00.961+0000 7f4c13b70740 1 client.27587 shutdown 2024-04-16T17:14:00.963 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:14:00.961+0000 7f4c13b70740 1 -- 172.21.15.45:0/1138031163 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5650e89247f0 msgr2=0x5650e8927120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:00.963 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:14:00.961+0000 7f4c13b70740 1 --2- 172.21.15.45:0/1138031163 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5650e89247f0 0x5650e8927120 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f4bf8099150 tx=0x7f4bf809dc20 comp rx=0 tx=0).stop 2024-04-16T17:14:00.963 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:14:00.961+0000 7f4c13b70740 1 -- 172.21.15.45:0/1138031163 shutdown_connections 2024-04-16T17:14:00.963 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:14:00.961+0000 7f4c13b70740 1 --2- 172.21.15.45:0/1138031163 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x5650e89a4270 0x5650e89a6690 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:00.963 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:14:00.961+0000 7f4c13b70740 1 --2- 172.21.15.45:0/1138031163 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5650e892d000 0x5650e8927660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:00.963 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:14:00.961+0000 7f4c13b70740 1 --2- 172.21.15.45:0/1138031163 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5650e89247f0 0x5650e8927120 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:00.963 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:14:00.961+0000 7f4bebe00640 0 client.27587 ms_handle_reset on v2:172.21.15.104:6838/3660070592 2024-04-16T17:14:00.963 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:14:00.961+0000 7f4c13b70740 1 --2- 172.21.15.45:0/1138031163 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5650e8923a80 0x5650e8926be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:00.963 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:14:00.961+0000 7f4c13b70740 1 -- 172.21.15.45:0/1138031163 >> 172.21.15.45:0/1138031163 conn(0x5650e88be5d0 msgr2=0x5650e891f930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:14:00.964 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:14:00.961+0000 7f4c13b70740 1 -- 172.21.15.45:0/1138031163 shutdown_connections 2024-04-16T17:14:00.964 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:14:00.962+0000 7f4c13b70740 1 -- 172.21.15.45:0/1138031163 wait complete. 2024-04-16T17:14:00.964 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:14:00.962+0000 7f4c13b70740 20 client.27587 trim_cache size 0 max 16384 2024-04-16T17:14:00.964 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-04-16T17:14:00.964 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-04-16T17:14:00.964 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-04-16T17:14:00.964 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-04-16T17:14:00.965 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:14:00.962+0000 7f4c0b200640 1 -- 172.21.15.45:0/775271414 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f4bf00a3d10 msgr2=0x7f4bf00a61d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:00.965 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:14:00.962+0000 7f4c0b200640 1 --2- 172.21.15.45:0/775271414 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f4bf00a3d10 0x7f4bf00a61d0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f4c04005fd0 tx=0x7f4c04009e90 comp rx=0 tx=0).stop 2024-04-16T17:14:00.965 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:14:00.962+0000 7f4c0b200640 1 -- 172.21.15.45:0/775271414 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4c0c109370 msgr2=0x7f4c0c1a0bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:00.965 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:14:00.962+0000 7f4c0b200640 1 --2- 172.21.15.45:0/775271414 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4c0c109370 0x7f4c0c1a0bd0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f4bf800e9c0 tx=0x7f4bf800ee80 comp rx=0 tx=0).stop 2024-04-16T17:14:00.965 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:14:00.963+0000 7f4c0b200640 1 -- 172.21.15.45:0/775271414 shutdown_connections 2024-04-16T17:14:00.965 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:14:00.963+0000 7f4c0b200640 1 --2- 172.21.15.45:0/775271414 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f4bf00a3d10 0x7f4bf00a61d0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:00.966 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:14:00.963+0000 7f4c0b200640 1 --2- 172.21.15.45:0/775271414 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4c0c19a850 0x7f4c0c19ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:00.966 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:14:00.963+0000 7f4c0b200640 1 --2- 172.21.15.45:0/775271414 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4c0c109370 0x7f4c0c1a0bd0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:00.966 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:14:00.963+0000 7f4c0b200640 1 --2- 172.21.15.45:0/775271414 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4c0c1042a0 0x7f4c0c1a0690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:00.966 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:14:00.963+0000 7f4c0b200640 1 -- 172.21.15.45:0/775271414 >> 172.21.15.45:0/775271414 conn(0x7f4c0c0ffae0 msgr2=0x7f4c0c101e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:14:00.966 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:14:00.963+0000 7f4c0b200640 1 -- 172.21.15.45:0/775271414 shutdown_connections 2024-04-16T17:14:00.966 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-16T17:14:00.963+0000 7f4c0b200640 1 -- 172.21.15.45:0/775271414 wait complete. 2024-04-16T17:14:01.906 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/github.com_mchangir_ceph_b137e4a675398e295567f9652210865a4d573fac/qa/tasks/workunit.py", line 145, in task _spawn_on_all_clients(ctx, refspec, all_tasks, config.get('env'), File "/home/teuthworker/src/github.com_mchangir_ceph_b137e4a675398e295567f9652210865a4d573fac/qa/tasks/workunit.py", line 295, in _spawn_on_all_clients p.spawn(_run_tests, ctx, refspec, role, [unit], env, File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/github.com_mchangir_ceph_b137e4a675398e295567f9652210865a4d573fac/qa/tasks/workunit.py", line 434, in _run_tests remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (workunit test suites/dbench.sh) on smithi045 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=b137e4a675398e295567f9652210865a4d573fac TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh' 2024-04-16T17:14:02.118 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=6df16c87d16b4e09ad8cc292d074c972 Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/github.com_mchangir_ceph_b137e4a675398e295567f9652210865a4d573fac/qa/tasks/workunit.py", line 145, in task _spawn_on_all_clients(ctx, refspec, all_tasks, config.get('env'), File "/home/teuthworker/src/github.com_mchangir_ceph_b137e4a675398e295567f9652210865a4d573fac/qa/tasks/workunit.py", line 295, in _spawn_on_all_clients p.spawn(_run_tests, ctx, refspec, role, [unit], env, File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/github.com_mchangir_ceph_b137e4a675398e295567f9652210865a4d573fac/qa/tasks/workunit.py", line 434, in _run_tests remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (workunit test suites/dbench.sh) on smithi045 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=b137e4a675398e295567f9652210865a4d573fac TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh' 2024-04-16T17:14:02.121 DEBUG:teuthology.run_tasks:Unwinding manager background_exec 2024-04-16T17:14:02.130 INFO:teuthology.task.background_exec:Stopping background command on ubuntu@smithi045.front.sepia.ceph.com 2024-04-16T17:14:02.156 DEBUG:teuthology.run_tasks:Unwinding manager full_sequential_finally 2024-04-16T17:14:02.165 INFO:teuthology.task.full_sequential_finally:In full_sequential_finally, running task exec... 2024-04-16T17:14:02.165 INFO:teuthology.task.exec:Executing custom commands... 2024-04-16T17:14:02.165 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi045.front.sepia.ceph.com 2024-04-16T17:14:02.165 DEBUG:teuthology.orchestra.run.smithi045:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > END_TIME' 2024-04-16T17:14:02.300 DEBUG:teuthology.orchestra.run.smithi045:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi' 2024-04-16T17:14:02.726 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:14:02 smithi045 ceph-mon[31676]: pgmap v224: 129 pgs: 129 active+clean; 63 MiB data, 796 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 175 op/s 2024-04-16T17:14:02.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:14:02 smithi117 ceph-mon[36730]: pgmap v224: 129 pgs: 129 active+clean; 63 MiB data, 796 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 175 op/s 2024-04-16T17:14:02.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:14:02 smithi104 ceph-mon[36457]: pgmap v224: 129 pgs: 129 active+clean; 63 MiB data, 796 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 175 op/s 2024-04-16T17:14:03.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:14:03 smithi045 ceph-mon[31676]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T17:14:03.804 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:14:03 smithi117 ceph-mon[36730]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T17:14:03.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:14:03 smithi104 ceph-mon[36457]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T17:14:04.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:14:04 smithi045 ceph-mon[31676]: pgmap v225: 129 pgs: 129 active+clean; 63 MiB data, 796 MiB used, 1.0 TiB / 1.0 TiB avail; 925 KiB/s rd, 800 KiB/s wr, 110 op/s 2024-04-16T17:14:04.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:14:04 smithi045 ceph-mon[31676]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:14:04.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:14:04 smithi045 ceph-mon[31676]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T17:14:04.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:14:04 smithi045 ceph-mon[31676]: from='mgr.25411 ' entity='mgr.x' 2024-04-16T17:14:04.804 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:14:04 smithi117 ceph-mon[36730]: pgmap v225: 129 pgs: 129 active+clean; 63 MiB data, 796 MiB used, 1.0 TiB / 1.0 TiB avail; 925 KiB/s rd, 800 KiB/s wr, 110 op/s 2024-04-16T17:14:04.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:14:04 smithi117 ceph-mon[36730]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:14:04.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:14:04 smithi117 ceph-mon[36730]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T17:14:04.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:14:04 smithi117 ceph-mon[36730]: from='mgr.25411 ' entity='mgr.x' 2024-04-16T17:14:04.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:14:04 smithi104 ceph-mon[36457]: pgmap v225: 129 pgs: 129 active+clean; 63 MiB data, 796 MiB used, 1.0 TiB / 1.0 TiB avail; 925 KiB/s rd, 800 KiB/s wr, 110 op/s 2024-04-16T17:14:04.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:14:04 smithi104 ceph-mon[36457]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:14:04.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:14:04 smithi104 ceph-mon[36457]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T17:14:04.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:14:04 smithi104 ceph-mon[36457]: from='mgr.25411 ' entity='mgr.x' 2024-04-16T17:14:05.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:14:05 smithi045 ceph-mon[31676]: osdmap e99: 12 total, 12 up, 12 in 2024-04-16T17:14:05.804 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:14:05 smithi117 ceph-mon[36730]: osdmap e99: 12 total, 12 up, 12 in 2024-04-16T17:14:05.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:14:05 smithi104 ceph-mon[36457]: osdmap e99: 12 total, 12 up, 12 in 2024-04-16T17:14:06.804 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:14:06 smithi117 ceph-mon[36730]: pgmap v227: 129 pgs: 129 active+clean; 63 MiB data, 796 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 960 KiB/s wr, 132 op/s 2024-04-16T17:14:06.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:14:06 smithi104 ceph-mon[36457]: pgmap v227: 129 pgs: 129 active+clean; 63 MiB data, 796 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 960 KiB/s wr, 132 op/s 2024-04-16T17:14:06.951 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-16T17:14:06.952 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-16T17:14:06.976 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:14:06 smithi045 ceph-mon[31676]: pgmap v227: 129 pgs: 129 active+clean; 63 MiB data, 796 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 960 KiB/s wr, 132 op/s 2024-04-16T17:14:07.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.175+0000 7f52b4c00640 1 -- 172.21.15.45:0/3694583203 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f52b0103980 msgr2=0x7f52b010be60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:07.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.175+0000 7f52b4c00640 1 --2- 172.21.15.45:0/3694583203 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f52b0103980 0x7f52b010be60 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f5298009900 tx=0x7f529802bcd0 comp rx=0 tx=0).stop 2024-04-16T17:14:07.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.175+0000 7f52b4c00640 1 -- 172.21.15.45:0/3694583203 shutdown_connections 2024-04-16T17:14:07.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.175+0000 7f52b4c00640 1 --2- 172.21.15.45:0/3694583203 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f52b010c480 0x7f52b010e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:07.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.175+0000 7f52b4c00640 1 --2- 172.21.15.45:0/3694583203 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f52b0103980 0x7f52b010be60 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:07.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.175+0000 7f52b4c00640 1 --2- 172.21.15.45:0/3694583203 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f52b0103060 0x7f52b0103440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:07.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.175+0000 7f52b4c00640 1 -- 172.21.15.45:0/3694583203 >> 172.21.15.45:0/3694583203 conn(0x7f52b00fc880 msgr2=0x7f52b00feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:14:07.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.175+0000 7f52b4c00640 1 -- 172.21.15.45:0/3694583203 shutdown_connections 2024-04-16T17:14:07.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.175+0000 7f52b4c00640 1 -- 172.21.15.45:0/3694583203 wait complete. 2024-04-16T17:14:07.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.176+0000 7f52b4c00640 1 Processor -- start 2024-04-16T17:14:07.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.176+0000 7f52b4c00640 1 -- start start 2024-04-16T17:14:07.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.177+0000 7f52b4c00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f52b0103060 0x7f52b01a0690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:07.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.177+0000 7f52b4c00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f52b0103980 0x7f52b01a0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:07.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.177+0000 7f52b4c00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f52b010c480 0x7f52b019a850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:07.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.177+0000 7f52b4c00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f52b01122b0 con 0x7f52b0103980 2024-04-16T17:14:07.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.177+0000 7f52b4c00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f52b0112130 con 0x7f52b0103060 2024-04-16T17:14:07.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.177+0000 7f52b4c00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f52b0112430 con 0x7f52b010c480 2024-04-16T17:14:07.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.177+0000 7f52af400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f52b0103060 0x7f52b01a0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:07.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.177+0000 7f52a7e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f52b0103980 0x7f52b01a0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:07.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.177+0000 7f52af400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f52b0103060 0x7f52b01a0690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:48992/0 (socket says 172.21.15.45:48992) 2024-04-16T17:14:07.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.177+0000 7f52af400640 1 -- 172.21.15.45:0/3032051471 learned_addr learned my addr 172.21.15.45:0/3032051471 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:14:07.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.177+0000 7f52afe00640 1 --2- 172.21.15.45:0/3032051471 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f52b010c480 0x7f52b019a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:07.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.178+0000 7f52a7e00640 1 -- 172.21.15.45:0/3032051471 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f52b010c480 msgr2=0x7f52b019a850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:07.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.178+0000 7f52a7e00640 1 --2- 172.21.15.45:0/3032051471 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f52b010c480 0x7f52b019a850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:07.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.178+0000 7f52a7e00640 1 -- 172.21.15.45:0/3032051471 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f52b0103060 msgr2=0x7f52b01a0690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:07.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.178+0000 7f52a7e00640 1 --2- 172.21.15.45:0/3032051471 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f52b0103060 0x7f52b01a0690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:07.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.178+0000 7f52a7e00640 1 -- 172.21.15.45:0/3032051471 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52b019b020 con 0x7f52b0103980 2024-04-16T17:14:07.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.178+0000 7f52a7e00640 1 --2- 172.21.15.45:0/3032051471 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f52b0103980 0x7f52b01a0bd0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f529802c210 tx=0x7f52980042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:14:07.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.178+0000 7f52acc00640 1 -- 172.21.15.45:0/3032051471 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5298042020 con 0x7f52b0103980 2024-04-16T17:14:07.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.178+0000 7f52b4c00640 1 -- 172.21.15.45:0/3032051471 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52b019b2b0 con 0x7f52b0103980 2024-04-16T17:14:07.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.178+0000 7f52acc00640 1 -- 172.21.15.45:0/3032051471 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f52980054e0 con 0x7f52b0103980 2024-04-16T17:14:07.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.178+0000 7f52acc00640 1 -- 172.21.15.45:0/3032051471 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5298003990 con 0x7f52b0103980 2024-04-16T17:14:07.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.179+0000 7f52b4c00640 1 -- 172.21.15.45:0/3032051471 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52b01a74e0 con 0x7f52b0103980 2024-04-16T17:14:07.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.182+0000 7f52b8f89740 1 Processor -- start 2024-04-16T17:14:07.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.182+0000 7f52acc00640 1 -- 172.21.15.45:0/3032051471 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f5298045020 con 0x7f52b0103980 2024-04-16T17:14:07.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.182+0000 7f52b8f89740 1 -- start start 2024-04-16T17:14:07.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.182+0000 7f52b8f89740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5630826a7390 0x5630826a7770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:07.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.182+0000 7f52b8f89740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5630826a7cb0 0x5630826b0350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:07.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.182+0000 7f52b8f89740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5630826b0890 0x5630826b2c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:07.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.182+0000 7f52b8f89740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x56308260d120 con 0x5630826b0890 2024-04-16T17:14:07.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.182+0000 7f52b8f89740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5630825b3340 con 0x5630826a7390 2024-04-16T17:14:07.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.182+0000 7f52b8f89740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5630826090e0 con 0x5630826a7cb0 2024-04-16T17:14:07.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.183+0000 7f52a7e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5630826b0890 0x5630826b2c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:07.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.183+0000 7f52afe00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5630826a7390 0x5630826a7770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:07.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.183+0000 7f52af400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5630826a7cb0 0x5630826b0350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:07.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.183+0000 7f52afe00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5630826a7390 0x5630826a7770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:49006/0 (socket says 172.21.15.45:49006) 2024-04-16T17:14:07.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.183+0000 7f52af400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5630826a7cb0 0x5630826b0350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:45550/0 (socket says 172.21.15.45:45550) 2024-04-16T17:14:07.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.183+0000 7f52afe00640 1 -- 172.21.15.45:0/4229800634 learned_addr learned my addr 172.21.15.45:0/4229800634 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:14:07.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.183+0000 7f52afe00640 1 -- 172.21.15.45:0/4229800634 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5630826a7cb0 msgr2=0x5630826b0350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:14:07.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.183+0000 7f52afe00640 1 --2- 172.21.15.45:0/4229800634 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5630826a7cb0 0x5630826b0350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:07.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.183+0000 7f52afe00640 1 -- 172.21.15.45:0/4229800634 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5630826b0890 msgr2=0x5630826b2c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:14:07.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.183+0000 7f52afe00640 1 --2- 172.21.15.45:0/4229800634 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5630826b0890 0x5630826b2c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:07.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.183+0000 7f52afe00640 1 -- 172.21.15.45:0/4229800634 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5630823e97d0 con 0x5630826a7390 2024-04-16T17:14:07.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.183+0000 7f52acc00640 1 --2- 172.21.15.45:0/3032051471 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f52940a3da0 0x7f52940a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:07.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.183+0000 7f52afe00640 1 --2- 172.21.15.45:0/4229800634 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5630826a7390 0x5630826a7770 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f52a8009f60 tx=0x7f52a8009f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dbf498453ea306b3 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:14:07.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.183+0000 7f52acc00640 1 -- 172.21.15.45:0/3032051471 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9584+0+0 (secure 0 0 0) 0x7f52980f24e0 con 0x7f52b0103980 2024-04-16T17:14:07.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.184+0000 7f52af400640 1 --2- 172.21.15.45:0/3032051471 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f52940a3da0 0x7f52940a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:07.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.184+0000 7f529fe00640 1 -- 172.21.15.45:0/4229800634 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52a801e070 con 0x5630826a7390 2024-04-16T17:14:07.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.184+0000 7f529fe00640 1 -- 172.21.15.45:0/4229800634 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f52a8003c30 con 0x5630826a7390 2024-04-16T17:14:07.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.184+0000 7f529fe00640 1 -- 172.21.15.45:0/4229800634 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52a800f980 con 0x5630826a7390 2024-04-16T17:14:07.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.185+0000 7f52b8f89740 1 -- 172.21.15.45:0/4229800634 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5630826a7390 msgr2=0x5630826a7770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:07.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.185+0000 7f52af400640 1 --2- 172.21.15.45:0/3032051471 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f52940a3da0 0x7f52940a6260 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f52a0009640 tx=0x7f52a0005f90 comp rx=0 tx=0).ready entity=mgr.25411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:14:07.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.185+0000 7f52b8f89740 1 --2- 172.21.15.45:0/4229800634 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5630826a7390 0x5630826a7770 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f52a8009f60 tx=0x7f52a8009f90 comp rx=0 tx=0).stop 2024-04-16T17:14:07.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.185+0000 7f52b8f89740 1 -- 172.21.15.45:0/4229800634 shutdown_connections 2024-04-16T17:14:07.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.185+0000 7f52b8f89740 1 --2- 172.21.15.45:0/4229800634 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5630826a7cb0 0x5630826b0350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:07.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.185+0000 7f52b8f89740 1 --2- 172.21.15.45:0/4229800634 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5630826b0890 0x5630826b2c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:07.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.185+0000 7f52b8f89740 1 --2- 172.21.15.45:0/4229800634 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5630826a7390 0x5630826a7770 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:07.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.185+0000 7f52b8f89740 1 -- 172.21.15.45:0/4229800634 >> 172.21.15.45:0/4229800634 conn(0x5630826a1ac0 msgr2=0x5630826a1ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:14:07.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.185+0000 7f52b8f89740 1 -- 172.21.15.45:0/4229800634 shutdown_connections 2024-04-16T17:14:07.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.185+0000 7f52b8f89740 1 -- 172.21.15.45:0/4229800634 wait complete. 2024-04-16T17:14:07.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.185+0000 7f52b8f89740 1 -- start start 2024-04-16T17:14:07.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.185+0000 7f52b8f89740 1 Processor -- start 2024-04-16T17:14:07.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.186+0000 7f52b8f89740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5630826a7390 0x5630826ad970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:07.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.186+0000 7f52b8f89740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5630826a7cb0 0x5630826adeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:07.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.186+0000 7f52b8f89740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5630826b0890 0x5630826ae3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:07.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.186+0000 7f52b8f89740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5630826b3680 con 0x5630826a7390 2024-04-16T17:14:07.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.186+0000 7f52b8f89740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5630825c6710 con 0x5630826a7cb0 2024-04-16T17:14:07.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.186+0000 7f52b8f89740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5630826b3800 con 0x5630826b0890 2024-04-16T17:14:07.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.186+0000 7f52b8f89740 10 client.0 _ll_register_callbacks cb 0x5630825c1af0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:14:07.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.186+0000 7f52af400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5630826a7390 0x5630826ad970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:07.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.186+0000 7f52a7e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5630826a7cb0 0x5630826adeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:07.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.186+0000 7f52af400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5630826a7390 0x5630826ad970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:46828/0 (socket says 172.21.15.45:46828) 2024-04-16T17:14:07.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.186+0000 7f52afe00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5630826b0890 0x5630826ae3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:07.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.186+0000 7f52af400640 1 -- 172.21.15.45:0/1685002994 learned_addr learned my addr 172.21.15.45:0/1685002994 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:14:07.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.187+0000 7f52a7e00640 1 -- 172.21.15.45:0/1685002994 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5630826b0890 msgr2=0x5630826ae3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:14:07.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.187+0000 7f52a7e00640 1 --2- 172.21.15.45:0/1685002994 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5630826b0890 0x5630826ae3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:07.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.187+0000 7f52a7e00640 1 -- 172.21.15.45:0/1685002994 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5630826a7390 msgr2=0x5630826ad970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:14:07.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.187+0000 7f52a7e00640 1 --2- 172.21.15.45:0/1685002994 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5630826a7390 0x5630826ad970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:07.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.187+0000 7f52a7e00640 1 -- 172.21.15.45:0/1685002994 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56308231f3f0 con 0x5630826a7cb0 2024-04-16T17:14:07.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.187+0000 7f52a7e00640 1 --2- 172.21.15.45:0/1685002994 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5630826a7cb0 0x5630826adeb0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f52980ef8e0 tx=0x7f52980045e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b275064078099463 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:14:07.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.187+0000 7f529cc00640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-04-16T17:14:07.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.187+0000 7f529cc00640 1 -- 172.21.15.45:0/1685002994 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52980bdf00 con 0x5630826a7cb0 2024-04-16T17:14:07.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.187+0000 7f52b8f89740 1 -- 172.21.15.45:0/1685002994 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f52a8003c30 con 0x5630826a7cb0 2024-04-16T17:14:07.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.187+0000 7f529cc00640 1 -- 172.21.15.45:0/1685002994 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9584+0+0 (secure 0 0 0) 0x7f52980c6910 con 0x5630826a7cb0 2024-04-16T17:14:07.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.187+0000 7f529cc00640 1 -- 172.21.15.45:0/1685002994 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f52980bda90 con 0x5630826a7cb0 2024-04-16T17:14:07.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.188+0000 7f52b8f89740 10 client.27599 fetch_fsmap learned FSMap version 42 2024-04-16T17:14:07.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.188+0000 7f52b8f89740 1 -- 172.21.15.45:0/1685002994 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x5630822df0f0 con 0x5630826a7cb0 2024-04-16T17:14:07.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.188+0000 7f529cc00640 1 -- 172.21.15.45:0/1685002994 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f52a8003c30 con 0x5630826a7cb0 2024-04-16T17:14:07.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.188+0000 7f529cc00640 1 -- 172.21.15.45:0/1685002994 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 42) ==== 8362+0+0 (secure 0 0 0) 0x7f52980de040 con 0x5630826a7cb0 2024-04-16T17:14:07.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.188+0000 7f52b8f89740 10 client.27599 fetch_fsmap finished waiting for FSMap version 42 2024-04-16T17:14:07.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.188+0000 7f52b8f89740 10 client.27599 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:14:07.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.188+0000 7f52b8f89740 20 client.27599 populate_metadata read hostname 'smithi045' 2024-04-16T17:14:07.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.188+0000 7f52b8f89740 1 --2- 172.21.15.45:0/1685002994 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x5630826c0b60 0x5630826c2f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:07.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.188+0000 7f52b8f89740 4 client.27599 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:14:07.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.188+0000 7f52b8f89740 1 -- 172.21.15.45:0/1685002994 --> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56308261ab40 con 0x5630826c0b60 2024-04-16T17:14:07.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.188+0000 7f52af400640 1 --2- 172.21.15.45:0/1685002994 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x5630826c0b60 0x5630826c2f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:07.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.189+0000 7f52af400640 1 --2- 172.21.15.45:0/1685002994 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x5630826c0b60 0x5630826c2f90 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a4833b37b53f3a7b server_cookie=d07d9b880ae8f3a9 in_seq=0 out_seq=0 2024-04-16T17:14:07.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.189+0000 7f529cc00640 10 client.27599 ms_handle_connect on v2:172.21.15.104:6838/3660070592 2024-04-16T17:14:07.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.190+0000 7f529cc00640 1 -- 172.21.15.45:0/1685002994 <== mds.0 v2:172.21.15.104:6838/3660070592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56308261ab40 con 0x5630826c0b60 2024-04-16T17:14:07.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.190+0000 7f529cc00640 10 client.27599 handle_command_reply: tid=0 2024-04-16T17:14:07.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.190+0000 7f52b8f89740 1 client.27599 shutdown 2024-04-16T17:14:07.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.190+0000 7f52b8f89740 1 -- 172.21.15.45:0/1685002994 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5630826a7cb0 msgr2=0x5630826adeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:07.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.190+0000 7f52b8f89740 1 --2- 172.21.15.45:0/1685002994 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5630826a7cb0 0x5630826adeb0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f52980ef8e0 tx=0x7f52980045e0 comp rx=0 tx=0).stop 2024-04-16T17:14:07.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.190+0000 7f52b8f89740 1 -- 172.21.15.45:0/1685002994 shutdown_connections 2024-04-16T17:14:07.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.190+0000 7f52b8f89740 1 --2- 172.21.15.45:0/1685002994 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x5630826c0b60 0x5630826c2f90 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:07.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.190+0000 7f52b8f89740 1 --2- 172.21.15.45:0/1685002994 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5630826b0890 0x5630826ae3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:07.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.190+0000 7f52b8f89740 1 --2- 172.21.15.45:0/1685002994 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5630826a7cb0 0x5630826adeb0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:07.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.190+0000 7f529cc00640 0 client.27599 ms_handle_reset on v2:172.21.15.104:6838/3660070592 2024-04-16T17:14:07.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.190+0000 7f52b8f89740 1 --2- 172.21.15.45:0/1685002994 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5630826a7390 0x5630826ad970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:07.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.190+0000 7f52b8f89740 1 -- 172.21.15.45:0/1685002994 >> 172.21.15.45:0/1685002994 conn(0x5630826a1ac0 msgr2=0x5630826a3810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:14:07.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.190+0000 7f52b8f89740 1 -- 172.21.15.45:0/1685002994 shutdown_connections 2024-04-16T17:14:07.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.190+0000 7f52b8f89740 1 -- 172.21.15.45:0/1685002994 wait complete. 2024-04-16T17:14:07.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.190+0000 7f52b8f89740 20 client.27599 trim_cache size 0 max 16384 2024-04-16T17:14:07.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.209+0000 7f52b8f89740 1 Processor -- start 2024-04-16T17:14:07.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.209+0000 7f52b8f89740 1 -- start start 2024-04-16T17:14:07.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.209+0000 7f52b8f89740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5630826a7390 0x5630826b00a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:07.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.209+0000 7f52b8f89740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5630826a7cb0 0x5630826bc540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:07.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.209+0000 7f52b8f89740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5630826b0890 0x5630826ae040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:07.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.210+0000 7f52b8f89740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f52b01b4030 con 0x5630826a7cb0 2024-04-16T17:14:07.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.210+0000 7f52b8f89740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f52b01b18a0 con 0x5630826b0890 2024-04-16T17:14:07.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.210+0000 7f52b8f89740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f52b01af6f0 con 0x5630826a7390 2024-04-16T17:14:07.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.210+0000 7f52a7e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5630826a7cb0 0x5630826bc540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:07.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.210+0000 7f52a7e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5630826a7cb0 0x5630826bc540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:46832/0 (socket says 172.21.15.45:46832) 2024-04-16T17:14:07.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.210+0000 7f52a7e00640 1 -- 172.21.15.45:0/3537622211 learned_addr learned my addr 172.21.15.45:0/3537622211 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:14:07.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.210+0000 7f52afe00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5630826b0890 0x5630826ae040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:07.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.210+0000 7f52af400640 1 --2- 172.21.15.45:0/3537622211 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5630826a7390 0x5630826b00a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:07.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.210+0000 7f52a7e00640 1 -- 172.21.15.45:0/3537622211 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5630826a7390 msgr2=0x5630826b00a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:14:07.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.210+0000 7f52a7e00640 1 --2- 172.21.15.45:0/3537622211 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5630826a7390 0x5630826b00a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:07.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.210+0000 7f52a7e00640 1 -- 172.21.15.45:0/3537622211 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5630826b0890 msgr2=0x5630826ae040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:14:07.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.210+0000 7f52a7e00640 1 --2- 172.21.15.45:0/3537622211 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5630826b0890 0x5630826ae040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:07.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.210+0000 7f52a7e00640 1 -- 172.21.15.45:0/3537622211 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5630826b3560 con 0x5630826a7cb0 2024-04-16T17:14:07.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.210+0000 7f52a7e00640 1 --2- 172.21.15.45:0/3537622211 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5630826a7cb0 0x5630826bc540 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f52980be0c0 tx=0x7f529802bcd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d7162650820800c server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:14:07.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.211+0000 7f529f400640 1 -- 172.21.15.45:0/3537622211 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52980bdc70 con 0x5630826a7cb0 2024-04-16T17:14:07.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.211+0000 7f529f400640 1 -- 172.21.15.45:0/3537622211 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5298004c80 con 0x5630826a7cb0 2024-04-16T17:14:07.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.211+0000 7f529f400640 1 -- 172.21.15.45:0/3537622211 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52980c6da0 con 0x5630826a7cb0 2024-04-16T17:14:07.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.211+0000 7f52b8f89740 1 -- 172.21.15.45:0/3537622211 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5630826a7cb0 msgr2=0x5630826bc540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:07.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.211+0000 7f52b8f89740 1 --2- 172.21.15.45:0/3537622211 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5630826a7cb0 0x5630826bc540 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f52980be0c0 tx=0x7f529802bcd0 comp rx=0 tx=0).stop 2024-04-16T17:14:07.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.211+0000 7f52b8f89740 1 -- 172.21.15.45:0/3537622211 shutdown_connections 2024-04-16T17:14:07.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.211+0000 7f52b8f89740 1 --2- 172.21.15.45:0/3537622211 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5630826b0890 0x5630826ae040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:07.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.211+0000 7f52b8f89740 1 --2- 172.21.15.45:0/3537622211 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5630826a7cb0 0x5630826bc540 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:07.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.211+0000 7f52b8f89740 1 --2- 172.21.15.45:0/3537622211 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5630826a7390 0x5630826b00a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:07.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.211+0000 7f52b8f89740 1 -- 172.21.15.45:0/3537622211 >> 172.21.15.45:0/3537622211 conn(0x5630826a1ac0 msgr2=0x5630826ad930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:14:07.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.211+0000 7f52b8f89740 1 -- 172.21.15.45:0/3537622211 shutdown_connections 2024-04-16T17:14:07.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.212+0000 7f52b8f89740 1 -- 172.21.15.45:0/3537622211 wait complete. 2024-04-16T17:14:07.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.212+0000 7f52b8f89740 1 -- start start 2024-04-16T17:14:07.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.212+0000 7f52b8f89740 1 Processor -- start 2024-04-16T17:14:07.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.213+0000 7f52b8f89740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5630826a7390 0x5630826b8f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:07.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.213+0000 7f52b8f89740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5630826a7cb0 0x5630826b4300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:07.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.213+0000 7f52b8f89740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5630826b0890 0x5630826b4a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:07.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.213+0000 7f52b8f89740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f52b01afd60 con 0x5630826a7cb0 2024-04-16T17:14:07.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.213+0000 7f52b8f89740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f52b01b11d0 con 0x5630826b0890 2024-04-16T17:14:07.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.213+0000 7f52b8f89740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f52b01b0430 con 0x5630826a7390 2024-04-16T17:14:07.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.213+0000 7f52b8f89740 10 client.0 _ll_register_callbacks cb 0x7f5298009880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:14:07.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.213+0000 7f52a7e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5630826a7cb0 0x5630826b4300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:07.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.213+0000 7f52a7e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5630826a7cb0 0x5630826b4300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:46848/0 (socket says 172.21.15.45:46848) 2024-04-16T17:14:07.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.213+0000 7f52a7e00640 1 -- 172.21.15.45:0/2604642093 learned_addr learned my addr 172.21.15.45:0/2604642093 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:14:07.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.213+0000 7f52afe00640 1 --2- 172.21.15.45:0/2604642093 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5630826b0890 0x5630826b4a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:07.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.213+0000 7f52af400640 1 --2- 172.21.15.45:0/2604642093 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5630826a7390 0x5630826b8f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:07.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.213+0000 7f52a7e00640 1 -- 172.21.15.45:0/2604642093 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5630826a7390 msgr2=0x5630826b8f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:14:07.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.213+0000 7f52a7e00640 1 --2- 172.21.15.45:0/2604642093 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5630826a7390 0x5630826b8f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:07.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.213+0000 7f52a7e00640 1 -- 172.21.15.45:0/2604642093 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5630826b0890 msgr2=0x5630826b4a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:14:07.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.213+0000 7f52a7e00640 1 --2- 172.21.15.45:0/2604642093 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5630826b0890 0x5630826b4a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:07.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.213+0000 7f52a7e00640 1 -- 172.21.15.45:0/2604642093 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5630827260d0 con 0x5630826a7cb0 2024-04-16T17:14:07.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.213+0000 7f52afe00640 1 --2- 172.21.15.45:0/2604642093 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5630826b0890 0x5630826b4a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:14:07.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.214+0000 7f52a7e00640 1 --2- 172.21.15.45:0/2604642093 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5630826a7cb0 0x5630826b4300 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f5298006010 tx=0x7f52980bdf00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd32af7020701748 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:14:07.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.214+0000 7f529cc00640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-16T17:14:07.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.214+0000 7f529cc00640 1 -- 172.21.15.45:0/2604642093 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52980d6030 con 0x5630826a7cb0 2024-04-16T17:14:07.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.214+0000 7f529cc00640 1 -- 172.21.15.45:0/2604642093 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9584+0+0 (secure 0 0 0) 0x7f52980f5c20 con 0x5630826a7cb0 2024-04-16T17:14:07.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.214+0000 7f52b8f89740 1 -- 172.21.15.45:0/2604642093 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5298004c80 con 0x5630826a7cb0 2024-04-16T17:14:07.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.214+0000 7f529cc00640 1 -- 172.21.15.45:0/2604642093 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f52980bdc70 con 0x5630826a7cb0 2024-04-16T17:14:07.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.214+0000 7f52b8f89740 10 client.17439 fetch_fsmap learned FSMap version 42 2024-04-16T17:14:07.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.214+0000 7f52b8f89740 1 -- 172.21.15.45:0/2604642093 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x5630827260d0 con 0x5630826a7cb0 2024-04-16T17:14:07.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.214+0000 7f529cc00640 1 -- 172.21.15.45:0/2604642093 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5298004c80 con 0x5630826a7cb0 2024-04-16T17:14:07.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.214+0000 7f529cc00640 1 -- 172.21.15.45:0/2604642093 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 42) ==== 8362+0+0 (secure 0 0 0) 0x7f52980de040 con 0x5630826a7cb0 2024-04-16T17:14:07.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.215+0000 7f52b8f89740 10 client.17439 fetch_fsmap finished waiting for FSMap version 42 2024-04-16T17:14:07.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.215+0000 7f52b8f89740 10 client.17439 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:14:07.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.215+0000 7f52b8f89740 20 client.17439 populate_metadata read hostname 'smithi045' 2024-04-16T17:14:07.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.215+0000 7f52b8f89740 1 --2- 172.21.15.45:0/2604642093 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x5630827274e0 0x5630827298d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:07.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.215+0000 7f52b8f89740 4 client.17439 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-16T17:14:07.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.215+0000 7f52b8f89740 1 -- 172.21.15.45:0/2604642093 --> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56308269d180 con 0x5630827274e0 2024-04-16T17:14:07.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.215+0000 7f52af400640 1 --2- 172.21.15.45:0/2604642093 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x5630827274e0 0x5630827298d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:07.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.216+0000 7f52af400640 1 --2- 172.21.15.45:0/2604642093 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x5630827274e0 0x5630827298d0 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c73a6bfc7c28047 server_cookie=ce4ff8480f2e00ed in_seq=0 out_seq=0 2024-04-16T17:14:07.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.216+0000 7f529cc00640 10 client.17439 ms_handle_connect on v2:172.21.15.104:6838/3660070592 2024-04-16T17:14:07.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.216+0000 7f529cc00640 1 -- 172.21.15.45:0/2604642093 <== mds.0 v2:172.21.15.104:6838/3660070592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x56308269d180 con 0x5630827274e0 2024-04-16T17:14:07.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.216+0000 7f529cc00640 10 client.17439 handle_command_reply: tid=0 2024-04-16T17:14:07.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.216+0000 7f52b8f89740 1 client.17439 shutdown 2024-04-16T17:14:07.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.216+0000 7f52b8f89740 1 -- 172.21.15.45:0/2604642093 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5630826a7cb0 msgr2=0x5630826b4300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:07.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.216+0000 7f52b8f89740 1 --2- 172.21.15.45:0/2604642093 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5630826a7cb0 0x5630826b4300 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f5298006010 tx=0x7f52980bdf00 comp rx=0 tx=0).stop 2024-04-16T17:14:07.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.216+0000 7f52b8f89740 1 -- 172.21.15.45:0/2604642093 shutdown_connections 2024-04-16T17:14:07.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.216+0000 7f52b8f89740 1 --2- 172.21.15.45:0/2604642093 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5630826b0890 0x5630826b4a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:07.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.216+0000 7f52b8f89740 1 --2- 172.21.15.45:0/2604642093 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5630826a7cb0 0x5630826b4300 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:07.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.216+0000 7f52b8f89740 1 --2- 172.21.15.45:0/2604642093 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x5630827274e0 0x5630827298d0 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:07.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.216+0000 7f52b8f89740 1 --2- 172.21.15.45:0/2604642093 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5630826a7390 0x5630826b8f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:07.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.216+0000 7f52b8f89740 1 -- 172.21.15.45:0/2604642093 >> 172.21.15.45:0/2604642093 conn(0x5630826a1ac0 msgr2=0x5630826ae1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:14:07.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.216+0000 7f529cc00640 0 client.17439 ms_handle_reset on v2:172.21.15.104:6838/3660070592 2024-04-16T17:14:07.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.216+0000 7f52b8f89740 1 -- 172.21.15.45:0/2604642093 shutdown_connections 2024-04-16T17:14:07.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.216+0000 7f52b8f89740 1 -- 172.21.15.45:0/2604642093 wait complete. 2024-04-16T17:14:07.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.216+0000 7f52b8f89740 20 client.17439 trim_cache size 0 max 16384 2024-04-16T17:14:07.227 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-16T17:14:07.227 INFO:teuthology.orchestra.run.smithi045.stdout: "status": "no active scrubs running", 2024-04-16T17:14:07.227 INFO:teuthology.orchestra.run.smithi045.stdout: "scrubs": {} 2024-04-16T17:14:07.227 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-16T17:14:07.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.217+0000 7f52b4c00640 1 -- 172.21.15.45:0/3032051471 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f52940a3da0 msgr2=0x7f52940a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:07.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.217+0000 7f52b4c00640 1 --2- 172.21.15.45:0/3032051471 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f52940a3da0 0x7f52940a6260 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f52a0009640 tx=0x7f52a0005f90 comp rx=0 tx=0).stop 2024-04-16T17:14:07.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.217+0000 7f52b4c00640 1 -- 172.21.15.45:0/3032051471 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f52b0103980 msgr2=0x7f52b01a0bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:07.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.217+0000 7f52b4c00640 1 --2- 172.21.15.45:0/3032051471 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f52b0103980 0x7f52b01a0bd0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f529802c210 tx=0x7f52980042f0 comp rx=0 tx=0).stop 2024-04-16T17:14:07.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.217+0000 7f52b4c00640 1 -- 172.21.15.45:0/3032051471 shutdown_connections 2024-04-16T17:14:07.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.217+0000 7f52b4c00640 1 --2- 172.21.15.45:0/3032051471 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f52940a3da0 0x7f52940a6260 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:07.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.217+0000 7f52b4c00640 1 --2- 172.21.15.45:0/3032051471 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f52b010c480 0x7f52b019a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:07.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.217+0000 7f52b4c00640 1 --2- 172.21.15.45:0/3032051471 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f52b0103980 0x7f52b01a0bd0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:07.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.217+0000 7f52b4c00640 1 --2- 172.21.15.45:0/3032051471 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f52b0103060 0x7f52b01a0690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:07.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.217+0000 7f52b4c00640 1 -- 172.21.15.45:0/3032051471 >> 172.21.15.45:0/3032051471 conn(0x7f52b00fc880 msgr2=0x7f52b0101d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:14:07.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.217+0000 7f52b4c00640 1 -- 172.21.15.45:0/3032051471 shutdown_connections 2024-04-16T17:14:07.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.217+0000 7f52b4c00640 1 -- 172.21.15.45:0/3032051471 wait complete. 2024-04-16T17:14:07.233 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-16T17:14:07.233 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-16T17:14:07.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.493+0000 7f6211200640 1 -- 172.21.15.45:0/1548229359 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f620c10e6e0 msgr2=0x7f620c110ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:07.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.493+0000 7f6211200640 1 --2- 172.21.15.45:0/1548229359 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f620c10e6e0 0x7f620c110ad0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f61fc009860 tx=0x7f61fc02bc00 comp rx=0 tx=0).stop 2024-04-16T17:14:07.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.494+0000 7f6211200640 1 -- 172.21.15.45:0/1548229359 shutdown_connections 2024-04-16T17:14:07.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.494+0000 7f6211200640 1 --2- 172.21.15.45:0/1548229359 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f620c074cd0 0x7f620c10e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:07.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.494+0000 7f6211200640 1 --2- 172.21.15.45:0/1548229359 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f620c10e6e0 0x7f620c110ad0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:07.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.494+0000 7f6211200640 1 --2- 172.21.15.45:0/1548229359 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f620c0743b0 0x7f620c074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:07.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.494+0000 7f6211200640 1 -- 172.21.15.45:0/1548229359 >> 172.21.15.45:0/1548229359 conn(0x7f620c0fd930 msgr2=0x7f620c0ffd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:14:07.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.494+0000 7f6211200640 1 -- 172.21.15.45:0/1548229359 shutdown_connections 2024-04-16T17:14:07.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.494+0000 7f6211200640 1 -- 172.21.15.45:0/1548229359 wait complete. 2024-04-16T17:14:07.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.495+0000 7f6211200640 1 Processor -- start 2024-04-16T17:14:07.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.495+0000 7f6211200640 1 -- start start 2024-04-16T17:14:07.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.495+0000 7f6211200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f620c0743b0 0x7f620c1a0650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:07.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.495+0000 7f6211200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f620c074cd0 0x7f620c1a0b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:07.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.495+0000 7f6211200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f620c10e6e0 0x7f620c19a760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:07.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.495+0000 7f6211200640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f620c1143e0 con 0x7f620c0743b0 2024-04-16T17:14:07.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.495+0000 7f6211200640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f620c114260 con 0x7f620c074cd0 2024-04-16T17:14:07.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.495+0000 7f6211200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f620c114560 con 0x7f620c10e6e0 2024-04-16T17:14:07.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.496+0000 7f620b400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f620c0743b0 0x7f620c1a0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:07.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.496+0000 7f620b400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f620c0743b0 0x7f620c1a0650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:46864/0 (socket says 172.21.15.45:46864) 2024-04-16T17:14:07.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.496+0000 7f620b400640 1 -- 172.21.15.45:0/129987739 learned_addr learned my addr 172.21.15.45:0/129987739 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:14:07.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.496+0000 7f620aa00640 1 --2- 172.21.15.45:0/129987739 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f620c074cd0 0x7f620c1a0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:07.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.496+0000 7f620be00640 1 --2- 172.21.15.45:0/129987739 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f620c10e6e0 0x7f620c19a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:07.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.496+0000 7f620b400640 1 -- 172.21.15.45:0/129987739 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f620c10e6e0 msgr2=0x7f620c19a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:07.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.496+0000 7f620b400640 1 --2- 172.21.15.45:0/129987739 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f620c10e6e0 0x7f620c19a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:07.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.496+0000 7f620b400640 1 -- 172.21.15.45:0/129987739 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f620c074cd0 msgr2=0x7f620c1a0b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:07.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.496+0000 7f620b400640 1 --2- 172.21.15.45:0/129987739 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f620c074cd0 0x7f620c1a0b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:07.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.496+0000 7f620b400640 1 -- 172.21.15.45:0/129987739 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f620c19b020 con 0x7f620c0743b0 2024-04-16T17:14:07.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.496+0000 7f620be00640 1 --2- 172.21.15.45:0/129987739 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f620c10e6e0 0x7f620c19a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:14:07.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.496+0000 7f620b400640 1 --2- 172.21.15.45:0/129987739 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f620c0743b0 0x7f620c1a0650 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f6204009d50 tx=0x7f620400c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:14:07.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.497+0000 7f6203e00640 1 -- 172.21.15.45:0/129987739 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6204003b70 con 0x7f620c0743b0 2024-04-16T17:14:07.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.497+0000 7f6211200640 1 -- 172.21.15.45:0/129987739 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f620c19b310 con 0x7f620c0743b0 2024-04-16T17:14:07.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.497+0000 7f6203e00640 1 -- 172.21.15.45:0/129987739 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6204003d10 con 0x7f620c0743b0 2024-04-16T17:14:07.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.497+0000 7f6203e00640 1 -- 172.21.15.45:0/129987739 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6204004d20 con 0x7f620c0743b0 2024-04-16T17:14:07.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.497+0000 7f6211200640 1 -- 172.21.15.45:0/129987739 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f620c1a75d0 con 0x7f620c0743b0 2024-04-16T17:14:07.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.498+0000 7f6211200640 1 -- 172.21.15.45:0/129987739 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f620c075a60 con 0x7f620c0743b0 2024-04-16T17:14:07.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.500+0000 7f6203e00640 1 -- 172.21.15.45:0/129987739 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f6204004000 con 0x7f620c0743b0 2024-04-16T17:14:07.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.502+0000 7f6203e00640 1 --2- 172.21.15.45:0/129987739 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f61f00a3d20 0x7f61f00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:07.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.502+0000 7f6203e00640 1 -- 172.21.15.45:0/129987739 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9584+0+0 (secure 0 0 0) 0x7f62040d12d0 con 0x7f620c0743b0 2024-04-16T17:14:07.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.502+0000 7f620aa00640 1 --2- 172.21.15.45:0/129987739 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f61f00a3d20 0x7f61f00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:07.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.503+0000 7f620aa00640 1 --2- 172.21.15.45:0/129987739 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f61f00a3d20 0x7f61f00a61e0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f620c19bf60 tx=0x7f61f4009260 comp rx=0 tx=0).ready entity=mgr.25411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:14:07.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.506+0000 7f6203e00640 1 -- 172.21.15.45:0/129987739 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f620409d510 con 0x7f620c0743b0 2024-04-16T17:14:07.774 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.774+0000 7f6211200640 1 -- 172.21.15.45:0/129987739 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f620c109860 con 0x7f620c0743b0 2024-04-16T17:14:07.775 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.775+0000 7f6203e00640 1 -- 172.21.15.45:0/129987739 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) ==== 94+0+11893 (secure 0 0 0) 0x7f620409ce10 con 0x7f620c0743b0 2024-04-16T17:14:07.775 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-16T17:14:07.775 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":42,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/1748842443","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":1748842443},{"type":"v1","addr":"172.21.15.45:6835","nonce":1748842443}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6839/1308627051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":1308627051},{"type":"v1","addr":"172.21.15.45:6839","nonce":1308627051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/1768242489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1768242489},{"type":"v1","addr":"172.21.15.117:6833","nonce":1768242489}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6833/3558902585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":3558902585},{"type":"v1","addr":"172.21.15.104:6833","nonce":3558902585}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24478,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/1308142903","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":1308142903},{"type":"v1","addr":"172.21.15.45:6837","nonce":1308142903}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24491,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6835/3430679937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":3430679937},{"type":"v1","addr":"172.21.15.104:6835","nonce":3430679937}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":42,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-16T17:03:24.663611+0000","modified":"2024-04-16T17:13:48.424453+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24581,"mds_1":24569,"mds_2":24532},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24499":{"gid":24499,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/2224123841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2224123841},{"type":"v1","addr":"172.21.15.117:6835","nonce":2224123841}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6837/2471356815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":2471356815},{"type":"v1","addr":"172.21.15.117:6837","nonce":2471356815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24532":{"gid":24532,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":13,"addr":"172.21.15.117:6839/2378675949","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":2378675949},{"type":"v1","addr":"172.21.15.117:6839","nonce":2378675949}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24569":{"gid":24569,"name":"a","rank":1,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.45:6841/3755310822","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6840","nonce":3755310822},{"type":"v1","addr":"172.21.15.45:6841","nonce":3755310822}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24581":{"gid":24581,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.104:6839/3660070592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6838","nonce":3660070592},{"type":"v1","addr":"172.21.15.104:6839","nonce":3660070592}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_27569":{"gid":27569,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6837/1833645246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":1833645246},{"type":"v1","addr":"172.21.15.104:6837","nonce":1833645246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24581,"qdb_cluster":[24569,24532,24581]},"id":1}]} 2024-04-16T17:14:07.776 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 42 2024-04-16T17:14:07.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.778+0000 7f6211200640 1 -- 172.21.15.45:0/129987739 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f61f00a3d20 msgr2=0x7f61f00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:07.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.778+0000 7f6211200640 1 --2- 172.21.15.45:0/129987739 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f61f00a3d20 0x7f61f00a61e0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f620c19bf60 tx=0x7f61f4009260 comp rx=0 tx=0).stop 2024-04-16T17:14:07.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.778+0000 7f6211200640 1 -- 172.21.15.45:0/129987739 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f620c0743b0 msgr2=0x7f620c1a0650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:07.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.778+0000 7f6211200640 1 --2- 172.21.15.45:0/129987739 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f620c0743b0 0x7f620c1a0650 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f6204009d50 tx=0x7f620400c6a0 comp rx=0 tx=0).stop 2024-04-16T17:14:07.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.778+0000 7f6211200640 1 -- 172.21.15.45:0/129987739 shutdown_connections 2024-04-16T17:14:07.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.778+0000 7f6211200640 1 --2- 172.21.15.45:0/129987739 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f61f00a3d20 0x7f61f00a61e0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:07.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.778+0000 7f6211200640 1 --2- 172.21.15.45:0/129987739 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f620c10e6e0 0x7f620c19a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:07.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.778+0000 7f6211200640 1 --2- 172.21.15.45:0/129987739 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f620c074cd0 0x7f620c1a0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:07.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.778+0000 7f6211200640 1 --2- 172.21.15.45:0/129987739 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f620c0743b0 0x7f620c1a0650 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:07.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.778+0000 7f6211200640 1 -- 172.21.15.45:0/129987739 >> 172.21.15.45:0/129987739 conn(0x7f620c0fd930 msgr2=0x7f620c0fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:14:07.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.778+0000 7f6211200640 1 -- 172.21.15.45:0/129987739 shutdown_connections 2024-04-16T17:14:07.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:07.779+0000 7f6211200640 1 -- 172.21.15.45:0/129987739 wait complete. 2024-04-16T17:14:07.797 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-16T17:14:08.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.011+0000 7fbf7fe00640 1 -- 172.21.15.45:0/2613623089 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbf80102110 msgr2=0x7fbf801024f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:08.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.011+0000 7fbf7fe00640 1 --2- 172.21.15.45:0/2613623089 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbf80102110 0x7fbf801024f0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fbf70009900 tx=0x7fbf7002bc30 comp rx=0 tx=0).stop 2024-04-16T17:14:08.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.011+0000 7fbf7fe00640 1 -- 172.21.15.45:0/2613623089 shutdown_connections 2024-04-16T17:14:08.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.011+0000 7fbf7fe00640 1 --2- 172.21.15.45:0/2613623089 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbf801071e0 0x7fbf8010edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:08.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.011+0000 7fbf7fe00640 1 --2- 172.21.15.45:0/2613623089 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbf80102ac0 0x7fbf80106b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:08.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.011+0000 7fbf7fe00640 1 --2- 172.21.15.45:0/2613623089 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbf80102110 0x7fbf801024f0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:08.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.011+0000 7fbf7fe00640 1 -- 172.21.15.45:0/2613623089 >> 172.21.15.45:0/2613623089 conn(0x7fbf800fd9d0 msgr2=0x7fbf800ffdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:14:08.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.012+0000 7fbf7fe00640 1 -- 172.21.15.45:0/2613623089 shutdown_connections 2024-04-16T17:14:08.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.012+0000 7fbf7fe00640 1 -- 172.21.15.45:0/2613623089 wait complete. 2024-04-16T17:14:08.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.012+0000 7fbf7fe00640 1 Processor -- start 2024-04-16T17:14:08.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.013+0000 7fbf7fe00640 1 -- start start 2024-04-16T17:14:08.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.013+0000 7fbf7fe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbf80102110 0x7fbf801a06c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:08.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.013+0000 7fbf7fe00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbf801071e0 0x7fbf801a0c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:08.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.013+0000 7fbf7fe00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbf8019a830 0x7fbf8019ace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:08.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.013+0000 7fbf7fe00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fbf801122f0 con 0x7fbf80102110 2024-04-16T17:14:08.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.013+0000 7fbf7fe00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fbf80112170 con 0x7fbf801071e0 2024-04-16T17:14:08.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.013+0000 7fbf7fe00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fbf80112470 con 0x7fbf8019a830 2024-04-16T17:14:08.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.013+0000 7fbf7ea00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbf80102110 0x7fbf801a06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:08.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.013+0000 7fbf7ea00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbf80102110 0x7fbf801a06c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:46876/0 (socket says 172.21.15.45:46876) 2024-04-16T17:14:08.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.013+0000 7fbf7ea00640 1 -- 172.21.15.45:0/1560342457 learned_addr learned my addr 172.21.15.45:0/1560342457 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:14:08.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.013+0000 7fbf7e000640 1 --2- 172.21.15.45:0/1560342457 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbf801071e0 0x7fbf801a0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:08.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.013+0000 7fbf7f400640 1 --2- 172.21.15.45:0/1560342457 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbf8019a830 0x7fbf8019ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:08.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.014+0000 7fbf7ea00640 1 -- 172.21.15.45:0/1560342457 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbf8019a830 msgr2=0x7fbf8019ace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:08.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.014+0000 7fbf7ea00640 1 --2- 172.21.15.45:0/1560342457 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbf8019a830 0x7fbf8019ace0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:08.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.014+0000 7fbf7ea00640 1 -- 172.21.15.45:0/1560342457 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbf801071e0 msgr2=0x7fbf801a0c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:08.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.014+0000 7fbf7ea00640 1 --2- 172.21.15.45:0/1560342457 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbf801071e0 0x7fbf801a0c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:08.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.014+0000 7fbf7ea00640 1 -- 172.21.15.45:0/1560342457 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf8019b3c0 con 0x7fbf80102110 2024-04-16T17:14:08.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.014+0000 7fbf7e000640 1 --2- 172.21.15.45:0/1560342457 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbf801071e0 0x7fbf801a0c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:14:08.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.014+0000 7fbf7ea00640 1 --2- 172.21.15.45:0/1560342457 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbf80102110 0x7fbf801a06c0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fbf7002c240 tx=0x7fbf700042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:14:08.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.014+0000 7fbf77400640 1 -- 172.21.15.45:0/1560342457 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf70043020 con 0x7fbf80102110 2024-04-16T17:14:08.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.014+0000 7fbf7fe00640 1 -- 172.21.15.45:0/1560342457 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf8019b650 con 0x7fbf80102110 2024-04-16T17:14:08.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.014+0000 7fbf77400640 1 -- 172.21.15.45:0/1560342457 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbf700054e0 con 0x7fbf80102110 2024-04-16T17:14:08.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.015+0000 7fbf77400640 1 -- 172.21.15.45:0/1560342457 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf70003b20 con 0x7fbf80102110 2024-04-16T17:14:08.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.015+0000 7fbf7fe00640 1 -- 172.21.15.45:0/1560342457 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf801a7830 con 0x7fbf80102110 2024-04-16T17:14:08.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.018+0000 7fbf77400640 1 -- 172.21.15.45:0/1560342457 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fbf70046020 con 0x7fbf80102110 2024-04-16T17:14:08.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.019+0000 7fbf889bd740 1 Processor -- start 2024-04-16T17:14:08.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.019+0000 7fbf889bd740 1 -- start start 2024-04-16T17:14:08.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.020+0000 7fbf77400640 1 --2- 172.21.15.45:0/1560342457 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fbf640a3d20 0x7fbf640a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:08.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.020+0000 7fbf889bd740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55722ede0f10 0x55722ede12f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:08.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.020+0000 7fbf77400640 1 -- 172.21.15.45:0/1560342457 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9584+0+0 (secure 0 0 0) 0x7fbf700f2a50 con 0x7fbf80102110 2024-04-16T17:14:08.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.020+0000 7fbf889bd740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55722ede1830 0x55722ede9ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:08.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.020+0000 7fbf889bd740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55722edea410 0x55722edec800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:08.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.020+0000 7fbf889bd740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55722ed46020 con 0x55722ede0f10 2024-04-16T17:14:08.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.020+0000 7fbf889bd740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55722ecec460 con 0x55722ede1830 2024-04-16T17:14:08.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.020+0000 7fbf889bd740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55722ed41fe0 con 0x55722edea410 2024-04-16T17:14:08.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.020+0000 7fbf7f400640 1 --2- 172.21.15.45:0/1560342457 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fbf640a3d20 0x7fbf640a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:08.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.020+0000 7fbf7ea00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55722ede0f10 0x55722ede12f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:08.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.020+0000 7fbf7ea00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55722ede0f10 0x55722ede12f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:46880/0 (socket says 172.21.15.45:46880) 2024-04-16T17:14:08.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.020+0000 7fbf7ea00640 1 -- 172.21.15.45:0/2882847508 learned_addr learned my addr 172.21.15.45:0/2882847508 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:14:08.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.020+0000 7fbf7e000640 1 --2- 172.21.15.45:0/2882847508 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55722ede1830 0x55722ede9ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:08.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.020+0000 7fbf7f400640 1 --2- 172.21.15.45:0/2882847508 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55722edea410 0x55722edec800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:08.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.021+0000 7fbf7ea00640 1 -- 172.21.15.45:0/2882847508 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55722edea410 msgr2=0x55722edec800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:14:08.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.021+0000 7fbf7ea00640 1 --2- 172.21.15.45:0/2882847508 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55722edea410 0x55722edec800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:08.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.021+0000 7fbf7ea00640 1 -- 172.21.15.45:0/2882847508 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55722ede1830 msgr2=0x55722ede9ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:14:08.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.021+0000 7fbf7ea00640 1 --2- 172.21.15.45:0/2882847508 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55722ede1830 0x55722ede9ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:08.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.021+0000 7fbf7ea00640 1 -- 172.21.15.45:0/2882847508 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55722ecf4040 con 0x55722ede0f10 2024-04-16T17:14:08.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.021+0000 7fbf7ea00640 1 --2- 172.21.15.45:0/2882847508 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55722ede0f10 0x55722ede12f0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fbf700c6de0 tx=0x7fbf700035f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d69a3cf1f5c8e092 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:14:08.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.021+0000 7fbf7f400640 1 --2- 172.21.15.45:0/1560342457 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fbf640a3d20 0x7fbf640a61e0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fbf78009fd0 tx=0x7fbf78009d80 comp rx=0 tx=0).ready entity=mgr.25411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:14:08.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.021+0000 7fbf6be00640 1 -- 172.21.15.45:0/2882847508 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf700a9070 con 0x55722ede0f10 2024-04-16T17:14:08.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.021+0000 7fbf6be00640 1 -- 172.21.15.45:0/2882847508 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbf700beda0 con 0x55722ede0f10 2024-04-16T17:14:08.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.021+0000 7fbf6be00640 1 -- 172.21.15.45:0/2882847508 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf700effc0 con 0x55722ede0f10 2024-04-16T17:14:08.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.022+0000 7fbf889bd740 1 -- 172.21.15.45:0/2882847508 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55722ede0f10 msgr2=0x55722ede12f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:08.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.022+0000 7fbf889bd740 1 --2- 172.21.15.45:0/2882847508 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55722ede0f10 0x55722ede12f0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fbf700c6de0 tx=0x7fbf700035f0 comp rx=0 tx=0).stop 2024-04-16T17:14:08.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.022+0000 7fbf889bd740 1 -- 172.21.15.45:0/2882847508 shutdown_connections 2024-04-16T17:14:08.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.022+0000 7fbf889bd740 1 --2- 172.21.15.45:0/2882847508 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55722edea410 0x55722edec800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:08.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.022+0000 7fbf889bd740 1 --2- 172.21.15.45:0/2882847508 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55722ede1830 0x55722ede9ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:08.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.022+0000 7fbf889bd740 1 --2- 172.21.15.45:0/2882847508 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55722ede0f10 0x55722ede12f0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:08.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.022+0000 7fbf889bd740 1 -- 172.21.15.45:0/2882847508 >> 172.21.15.45:0/2882847508 conn(0x55722eddb640 msgr2=0x55722eddba20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:14:08.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.022+0000 7fbf889bd740 1 -- 172.21.15.45:0/2882847508 shutdown_connections 2024-04-16T17:14:08.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.022+0000 7fbf889bd740 1 -- 172.21.15.45:0/2882847508 wait complete. 2024-04-16T17:14:08.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.023+0000 7fbf889bd740 1 -- start start 2024-04-16T17:14:08.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.023+0000 7fbf889bd740 1 Processor -- start 2024-04-16T17:14:08.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.024+0000 7fbf889bd740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55722ede0f10 0x55722ede7490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:08.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.024+0000 7fbf889bd740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55722ede1830 0x55722ede79d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:08.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.024+0000 7fbf889bd740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55722edea410 0x55722ede7f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:08.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.024+0000 7fbf889bd740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55722edecfc0 con 0x55722ede0f10 2024-04-16T17:14:08.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.024+0000 7fbf889bd740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55722ecff830 con 0x55722edea410 2024-04-16T17:14:08.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.024+0000 7fbf889bd740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55722eded140 con 0x55722ede1830 2024-04-16T17:14:08.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.024+0000 7fbf889bd740 10 client.0 _ll_register_callbacks cb 0x55722ed07150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:14:08.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.024+0000 7fbf7ea00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55722ede0f10 0x55722ede7490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:08.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.024+0000 7fbf7ea00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55722ede0f10 0x55722ede7490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:46894/0 (socket says 172.21.15.45:46894) 2024-04-16T17:14:08.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.024+0000 7fbf7ea00640 1 -- 172.21.15.45:0/3838431172 learned_addr learned my addr 172.21.15.45:0/3838431172 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:14:08.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.024+0000 7fbf7e000640 1 --2- 172.21.15.45:0/3838431172 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55722ede1830 0x55722ede79d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:08.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.024+0000 7fbf7f400640 1 --2- 172.21.15.45:0/3838431172 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55722edea410 0x55722ede7f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:08.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.025+0000 7fbf7ea00640 1 -- 172.21.15.45:0/3838431172 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55722ede1830 msgr2=0x55722ede79d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:14:08.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.025+0000 7fbf7ea00640 1 --2- 172.21.15.45:0/3838431172 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55722ede1830 0x55722ede79d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:08.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.025+0000 7fbf7ea00640 1 -- 172.21.15.45:0/3838431172 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55722edea410 msgr2=0x55722ede7f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:14:08.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.025+0000 7fbf7ea00640 1 --2- 172.21.15.45:0/3838431172 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55722edea410 0x55722ede7f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:08.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.025+0000 7fbf7ea00640 1 -- 172.21.15.45:0/3838431172 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55722ed03920 con 0x55722ede0f10 2024-04-16T17:14:08.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.025+0000 7fbf7f400640 1 --2- 172.21.15.45:0/3838431172 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55722edea410 0x55722ede7f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:14:08.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.025+0000 7fbf7ea00640 1 --2- 172.21.15.45:0/3838431172 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55722ede0f10 0x55722ede7490 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fbf7002ce30 tx=0x7fbf700ba9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adcfcdb594581671 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:14:08.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.025+0000 7fbf68c00640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-16T17:14:08.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.025+0000 7fbf68c00640 1 -- 172.21.15.45:0/3838431172 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf700be6a0 con 0x55722ede0f10 2024-04-16T17:14:08.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.025+0000 7fbf68c00640 1 -- 172.21.15.45:0/3838431172 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9584+0+0 (secure 0 0 0) 0x7fbf7002cb20 con 0x55722ede0f10 2024-04-16T17:14:08.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.025+0000 7fbf889bd740 1 -- 172.21.15.45:0/3838431172 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbf700beda0 con 0x55722ede0f10 2024-04-16T17:14:08.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.025+0000 7fbf68c00640 1 -- 172.21.15.45:0/3838431172 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7fbf700c2a50 con 0x55722ede0f10 2024-04-16T17:14:08.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.025+0000 7fbf889bd740 10 client.17457 fetch_fsmap learned FSMap version 42 2024-04-16T17:14:08.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.025+0000 7fbf889bd740 1 -- 172.21.15.45:0/3838431172 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55722ecd17f0 con 0x55722ede0f10 2024-04-16T17:14:08.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.026+0000 7fbf68c00640 1 -- 172.21.15.45:0/3838431172 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbf700beda0 con 0x55722ede0f10 2024-04-16T17:14:08.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.026+0000 7fbf68c00640 1 -- 172.21.15.45:0/3838431172 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 42) ==== 8362+0+0 (secure 0 0 0) 0x7fbf700a7be0 con 0x55722ede0f10 2024-04-16T17:14:08.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.026+0000 7fbf889bd740 10 client.17457 fetch_fsmap finished waiting for FSMap version 42 2024-04-16T17:14:08.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.026+0000 7fbf889bd740 10 client.17457 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-16T17:14:08.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.026+0000 7fbf889bd740 20 client.17457 populate_metadata read hostname 'smithi045' 2024-04-16T17:14:08.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.026+0000 7fbf889bd740 1 --2- 172.21.15.45:0/3838431172 >> [v2:172.21.15.117:6838/2378675949,v1:172.21.15.117:6839/2378675949] conn(0x55722edfa3a0 0x55722edfc790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:08.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.026+0000 7fbf889bd740 4 client.17457 mds_command: new command op to 24532 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:14:08.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.026+0000 7fbf889bd740 1 -- 172.21.15.45:0/3838431172 --> [v2:172.21.15.117:6838/2378675949,v1:172.21.15.117:6839/2378675949] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55722ed53a40 con 0x55722edfa3a0 2024-04-16T17:14:08.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.026+0000 7fbf7e000640 1 --2- 172.21.15.45:0/3838431172 >> [v2:172.21.15.117:6838/2378675949,v1:172.21.15.117:6839/2378675949] conn(0x55722edfa3a0 0x55722edfc790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:08.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.027+0000 7fbf7e000640 1 --2- 172.21.15.45:0/3838431172 >> [v2:172.21.15.117:6838/2378675949,v1:172.21.15.117:6839/2378675949] conn(0x55722edfa3a0 0x55722edfc790 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5dc647e6921d6490 server_cookie=34bbf87b6e7dbf97 in_seq=0 out_seq=0 2024-04-16T17:14:08.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.027+0000 7fbf68c00640 10 client.17457 ms_handle_connect on v2:172.21.15.117:6838/2378675949 2024-04-16T17:14:08.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.028+0000 7fbf68c00640 1 -- 172.21.15.45:0/3838431172 <== mds.2 v2:172.21.15.117:6838/2378675949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55722ed53a40 con 0x55722edfa3a0 2024-04-16T17:14:08.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.028+0000 7fbf68c00640 10 client.17457 handle_command_reply: tid=0 2024-04-16T17:14:08.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.028+0000 7fbf889bd740 1 client.17457 shutdown 2024-04-16T17:14:08.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.028+0000 7fbf889bd740 1 -- 172.21.15.45:0/3838431172 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55722ede0f10 msgr2=0x55722ede7490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:08.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.028+0000 7fbf889bd740 1 --2- 172.21.15.45:0/3838431172 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55722ede0f10 0x55722ede7490 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fbf7002ce30 tx=0x7fbf700ba9d0 comp rx=0 tx=0).stop 2024-04-16T17:14:08.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.028+0000 7fbf889bd740 1 -- 172.21.15.45:0/3838431172 shutdown_connections 2024-04-16T17:14:08.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.028+0000 7fbf889bd740 1 --2- 172.21.15.45:0/3838431172 >> [v2:172.21.15.117:6838/2378675949,v1:172.21.15.117:6839/2378675949] conn(0x55722edfa3a0 0x55722edfc790 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:08.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.028+0000 7fbf889bd740 1 --2- 172.21.15.45:0/3838431172 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55722ede1830 0x55722ede79d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:08.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.028+0000 7fbf889bd740 1 --2- 172.21.15.45:0/3838431172 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55722edea410 0x55722ede7f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:08.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.028+0000 7fbf889bd740 1 --2- 172.21.15.45:0/3838431172 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55722ede0f10 0x55722ede7490 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:08.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.028+0000 7fbf68c00640 0 client.17457 ms_handle_reset on v2:172.21.15.117:6838/2378675949 2024-04-16T17:14:08.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.028+0000 7fbf889bd740 1 -- 172.21.15.45:0/3838431172 >> 172.21.15.45:0/3838431172 conn(0x55722eddb640 msgr2=0x55722eddd390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:14:08.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.028+0000 7fbf889bd740 1 -- 172.21.15.45:0/3838431172 shutdown_connections 2024-04-16T17:14:08.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.028+0000 7fbf889bd740 1 -- 172.21.15.45:0/3838431172 wait complete. 2024-04-16T17:14:08.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.028+0000 7fbf889bd740 20 client.17457 trim_cache size 0 max 16384 2024-04-16T17:14:08.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.043+0000 7fbf889bd740 1 Processor -- start 2024-04-16T17:14:08.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.043+0000 7fbf889bd740 1 -- start start 2024-04-16T17:14:08.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.043+0000 7fbf889bd740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55722ede0f10 0x55722ede9bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:08.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.043+0000 7fbf889bd740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55722ede1830 0x55722edf5ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:08.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.043+0000 7fbf889bd740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55722edea410 0x55722ee01dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:08.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.043+0000 7fbf889bd740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fbf801b4040 con 0x55722edea410 2024-04-16T17:14:08.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.043+0000 7fbf889bd740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fbf801b18b0 con 0x55722ede1830 2024-04-16T17:14:08.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.043+0000 7fbf889bd740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fbf801af700 con 0x55722ede0f10 2024-04-16T17:14:08.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.044+0000 7fbf7f400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55722edea410 0x55722ee01dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:08.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.044+0000 7fbf7f400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55722edea410 0x55722ee01dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:46904/0 (socket says 172.21.15.45:46904) 2024-04-16T17:14:08.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.044+0000 7fbf7f400640 1 -- 172.21.15.45:0/805854274 learned_addr learned my addr 172.21.15.45:0/805854274 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:14:08.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.044+0000 7fbf7ea00640 1 --2- 172.21.15.45:0/805854274 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55722ede0f10 0x55722ede9bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:08.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.044+0000 7fbf7e000640 1 --2- 172.21.15.45:0/805854274 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55722ede1830 0x55722edf5ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:08.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.044+0000 7fbf7f400640 1 -- 172.21.15.45:0/805854274 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55722ede0f10 msgr2=0x55722ede9bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:14:08.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.044+0000 7fbf7f400640 1 --2- 172.21.15.45:0/805854274 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55722ede0f10 0x55722ede9bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:08.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.044+0000 7fbf7f400640 1 -- 172.21.15.45:0/805854274 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55722ede1830 msgr2=0x55722edf5ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:14:08.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.044+0000 7fbf7f400640 1 --2- 172.21.15.45:0/805854274 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55722ede1830 0x55722edf5ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:08.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.044+0000 7fbf7f400640 1 -- 172.21.15.45:0/805854274 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55722ea583f0 con 0x55722edea410 2024-04-16T17:14:08.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.044+0000 7fbf7ea00640 1 --2- 172.21.15.45:0/805854274 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55722ede0f10 0x55722ede9bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:14:08.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.045+0000 7fbf7f400640 1 --2- 172.21.15.45:0/805854274 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55722edea410 0x55722ee01dc0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fbf780137b0 tx=0x7fbf78009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e95a691cb813a01 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:14:08.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.045+0000 7fbf6b400640 1 -- 172.21.15.45:0/805854274 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf78003bb0 con 0x55722edea410 2024-04-16T17:14:08.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.045+0000 7fbf6b400640 1 -- 172.21.15.45:0/805854274 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbf78003d50 con 0x55722edea410 2024-04-16T17:14:08.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.045+0000 7fbf6b400640 1 -- 172.21.15.45:0/805854274 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf7802aa40 con 0x55722edea410 2024-04-16T17:14:08.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.046+0000 7fbf889bd740 1 -- 172.21.15.45:0/805854274 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55722edea410 msgr2=0x55722ee01dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:08.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.046+0000 7fbf889bd740 1 --2- 172.21.15.45:0/805854274 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55722edea410 0x55722ee01dc0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fbf780137b0 tx=0x7fbf78009500 comp rx=0 tx=0).stop 2024-04-16T17:14:08.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.046+0000 7fbf889bd740 1 -- 172.21.15.45:0/805854274 shutdown_connections 2024-04-16T17:14:08.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.046+0000 7fbf889bd740 1 --2- 172.21.15.45:0/805854274 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55722edea410 0x55722ee01dc0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:08.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.046+0000 7fbf889bd740 1 --2- 172.21.15.45:0/805854274 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55722ede1830 0x55722edf5ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:08.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.046+0000 7fbf889bd740 1 --2- 172.21.15.45:0/805854274 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55722ede0f10 0x55722ede9bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:08.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.046+0000 7fbf889bd740 1 -- 172.21.15.45:0/805854274 >> 172.21.15.45:0/805854274 conn(0x55722eddb640 msgr2=0x55722ede7ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:14:08.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.046+0000 7fbf889bd740 1 -- 172.21.15.45:0/805854274 shutdown_connections 2024-04-16T17:14:08.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.046+0000 7fbf889bd740 1 -- 172.21.15.45:0/805854274 wait complete. 2024-04-16T17:14:08.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.046+0000 7fbf889bd740 1 -- start start 2024-04-16T17:14:08.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.047+0000 7fbf889bd740 1 Processor -- start 2024-04-16T17:14:08.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.047+0000 7fbf889bd740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55722ede0f10 0x55722ede4350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:08.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.047+0000 7fbf889bd740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55722ede1830 0x55722ede4890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:08.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.047+0000 7fbf889bd740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55722edea410 0x55722ede4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:08.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.047+0000 7fbf889bd740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fbf801afd70 con 0x55722edea410 2024-04-16T17:14:08.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.047+0000 7fbf889bd740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fbf801b11e0 con 0x55722ede1830 2024-04-16T17:14:08.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.047+0000 7fbf889bd740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fbf801b0440 con 0x55722ede0f10 2024-04-16T17:14:08.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.047+0000 7fbf889bd740 10 client.0 _ll_register_callbacks cb 0x7fbf6c009b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:14:08.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.047+0000 7fbf7e000640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55722ede1830 0x55722ede4890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:08.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.047+0000 7fbf7e000640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55722ede1830 0x55722ede4890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:49098/0 (socket says 172.21.15.45:49098) 2024-04-16T17:14:08.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.047+0000 7fbf7e000640 1 -- 172.21.15.45:0/3371523516 learned_addr learned my addr 172.21.15.45:0/3371523516 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:14:08.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.047+0000 7fbf7ea00640 1 --2- 172.21.15.45:0/3371523516 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55722ede0f10 0x55722ede4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:08.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.047+0000 7fbf7f400640 1 --2- 172.21.15.45:0/3371523516 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55722edea410 0x55722ede4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:08.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.048+0000 7fbf7e000640 1 -- 172.21.15.45:0/3371523516 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55722ede0f10 msgr2=0x55722ede4350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:14:08.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.048+0000 7fbf7e000640 1 --2- 172.21.15.45:0/3371523516 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55722ede0f10 0x55722ede4350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:08.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.048+0000 7fbf7e000640 1 -- 172.21.15.45:0/3371523516 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55722edea410 msgr2=0x55722ede4dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:14:08.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.048+0000 7fbf7e000640 1 --2- 172.21.15.45:0/3371523516 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55722edea410 0x55722ede4dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:08.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.048+0000 7fbf7e000640 1 -- 172.21.15.45:0/3371523516 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55722eb227d0 con 0x55722ede1830 2024-04-16T17:14:08.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.048+0000 7fbf7e000640 1 --2- 172.21.15.45:0/3371523516 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55722ede1830 0x55722ede4890 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fbf6c004700 tx=0x7fbf6c012bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a4e3a62527d5ac05 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:14:08.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.048+0000 7fbf68c00640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-04-16T17:14:08.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.048+0000 7fbf68c00640 1 -- 172.21.15.45:0/3371523516 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf6c011d20 con 0x55722ede1830 2024-04-16T17:14:08.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.049+0000 7fbf889bd740 1 -- 172.21.15.45:0/3371523516 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbf78003d50 con 0x55722ede1830 2024-04-16T17:14:08.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.049+0000 7fbf68c00640 1 -- 172.21.15.45:0/3371523516 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9584+0+0 (secure 0 0 0) 0x7fbf6c005020 con 0x55722ede1830 2024-04-16T17:14:08.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.049+0000 7fbf68c00640 1 -- 172.21.15.45:0/3371523516 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7fbf6c031050 con 0x55722ede1830 2024-04-16T17:14:08.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.049+0000 7fbf889bd740 10 client.27629 fetch_fsmap learned FSMap version 42 2024-04-16T17:14:08.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.049+0000 7fbf889bd740 1 -- 172.21.15.45:0/3371523516 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55722ee691b0 con 0x55722ede1830 2024-04-16T17:14:08.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.049+0000 7fbf68c00640 1 -- 172.21.15.45:0/3371523516 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbf78003d50 con 0x55722ede1830 2024-04-16T17:14:08.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.049+0000 7fbf68c00640 1 -- 172.21.15.45:0/3371523516 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 42) ==== 8362+0+0 (secure 0 0 0) 0x7fbf6c004c90 con 0x55722ede1830 2024-04-16T17:14:08.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.049+0000 7fbf889bd740 10 client.27629 fetch_fsmap finished waiting for FSMap version 42 2024-04-16T17:14:08.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.049+0000 7fbf889bd740 10 client.27629 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-16T17:14:08.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.049+0000 7fbf889bd740 20 client.27629 populate_metadata read hostname 'smithi045' 2024-04-16T17:14:08.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.049+0000 7fbf889bd740 1 --2- 172.21.15.45:0/3371523516 >> [v2:172.21.15.117:6838/2378675949,v1:172.21.15.117:6839/2378675949] conn(0x55722ee69990 0x55722ee6bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:08.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.050+0000 7fbf889bd740 4 client.27629 mds_command: new command op to 24532 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T17:14:08.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.050+0000 7fbf889bd740 1 -- 172.21.15.45:0/3371523516 --> [v2:172.21.15.117:6838/2378675949,v1:172.21.15.117:6839/2378675949] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55722ed4d8b0 con 0x55722ee69990 2024-04-16T17:14:08.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.050+0000 7fbf7ea00640 1 --2- 172.21.15.45:0/3371523516 >> [v2:172.21.15.117:6838/2378675949,v1:172.21.15.117:6839/2378675949] conn(0x55722ee69990 0x55722ee6bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:08.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.050+0000 7fbf7ea00640 1 --2- 172.21.15.45:0/3371523516 >> [v2:172.21.15.117:6838/2378675949,v1:172.21.15.117:6839/2378675949] conn(0x55722ee69990 0x55722ee6bdb0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=724e3cf093ce6c83 server_cookie=37216dd80ae00449 in_seq=0 out_seq=0 2024-04-16T17:14:08.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.050+0000 7fbf68c00640 10 client.27629 ms_handle_connect on v2:172.21.15.117:6838/2378675949 2024-04-16T17:14:08.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.051+0000 7fbf68c00640 1 -- 172.21.15.45:0/3371523516 <== mds.2 v2:172.21.15.117:6838/2378675949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55722ed4d8b0 con 0x55722ee69990 2024-04-16T17:14:08.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.051+0000 7fbf68c00640 10 client.27629 handle_command_reply: tid=0 2024-04-16T17:14:08.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.051+0000 7fbf889bd740 1 client.27629 shutdown 2024-04-16T17:14:08.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.051+0000 7fbf889bd740 1 -- 172.21.15.45:0/3371523516 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55722ede1830 msgr2=0x55722ede4890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:08.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.051+0000 7fbf889bd740 1 --2- 172.21.15.45:0/3371523516 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55722ede1830 0x55722ede4890 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fbf6c004700 tx=0x7fbf6c012bc0 comp rx=0 tx=0).stop 2024-04-16T17:14:08.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.051+0000 7fbf889bd740 1 -- 172.21.15.45:0/3371523516 shutdown_connections 2024-04-16T17:14:08.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.051+0000 7fbf889bd740 1 --2- 172.21.15.45:0/3371523516 >> [v2:172.21.15.117:6838/2378675949,v1:172.21.15.117:6839/2378675949] conn(0x55722ee69990 0x55722ee6bdb0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:08.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.051+0000 7fbf889bd740 1 --2- 172.21.15.45:0/3371523516 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55722edea410 0x55722ede4dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:08.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.051+0000 7fbf889bd740 1 --2- 172.21.15.45:0/3371523516 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55722ede1830 0x55722ede4890 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:08.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.051+0000 7fbf889bd740 1 --2- 172.21.15.45:0/3371523516 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55722ede0f10 0x55722ede4350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:08.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.051+0000 7fbf68c00640 0 client.27629 ms_handle_reset on v2:172.21.15.117:6838/2378675949 2024-04-16T17:14:08.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.051+0000 7fbf889bd740 1 -- 172.21.15.45:0/3371523516 >> 172.21.15.45:0/3371523516 conn(0x55722eddb640 msgr2=0x55722eddc940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:14:08.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.051+0000 7fbf889bd740 1 -- 172.21.15.45:0/3371523516 shutdown_connections 2024-04-16T17:14:08.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.051+0000 7fbf889bd740 1 -- 172.21.15.45:0/3371523516 wait complete. 2024-04-16T17:14:08.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.051+0000 7fbf889bd740 20 client.27629 trim_cache size 0 max 16384 2024-04-16T17:14:08.075 INFO:teuthology.orchestra.run.smithi045.stdout:[] 2024-04-16T17:14:08.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.052+0000 7fbf7fe00640 1 -- 172.21.15.45:0/1560342457 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fbf640a3d20 msgr2=0x7fbf640a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:08.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.052+0000 7fbf7fe00640 1 --2- 172.21.15.45:0/1560342457 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fbf640a3d20 0x7fbf640a61e0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fbf78009fd0 tx=0x7fbf78009d80 comp rx=0 tx=0).stop 2024-04-16T17:14:08.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.052+0000 7fbf7fe00640 1 -- 172.21.15.45:0/1560342457 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbf80102110 msgr2=0x7fbf801a06c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:08.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.052+0000 7fbf7fe00640 1 --2- 172.21.15.45:0/1560342457 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbf80102110 0x7fbf801a06c0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fbf7002c240 tx=0x7fbf700042f0 comp rx=0 tx=0).stop 2024-04-16T17:14:08.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.052+0000 7fbf7fe00640 1 -- 172.21.15.45:0/1560342457 shutdown_connections 2024-04-16T17:14:08.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.052+0000 7fbf7fe00640 1 --2- 172.21.15.45:0/1560342457 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fbf640a3d20 0x7fbf640a61e0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:08.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.052+0000 7fbf7fe00640 1 --2- 172.21.15.45:0/1560342457 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbf8019a830 0x7fbf8019ace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:08.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.052+0000 7fbf7fe00640 1 --2- 172.21.15.45:0/1560342457 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbf801071e0 0x7fbf801a0c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:08.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.052+0000 7fbf7fe00640 1 --2- 172.21.15.45:0/1560342457 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbf80102110 0x7fbf801a06c0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:08.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.052+0000 7fbf7fe00640 1 -- 172.21.15.45:0/1560342457 >> 172.21.15.45:0/1560342457 conn(0x7fbf800fd9d0 msgr2=0x7fbf800fec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:14:08.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.052+0000 7fbf7fe00640 1 -- 172.21.15.45:0/1560342457 shutdown_connections 2024-04-16T17:14:08.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.052+0000 7fbf7fe00640 1 -- 172.21.15.45:0/1560342457 wait complete. 2024-04-16T17:14:08.077 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-16T17:14:08.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.287+0000 7fad84a00640 1 -- 172.21.15.45:0/4011738747 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fad80074cd0 msgr2=0x7fad8010e220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:08.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.287+0000 7fad84a00640 1 --2- 172.21.15.45:0/4011738747 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fad80074cd0 0x7fad8010e220 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fad6c009900 tx=0x7fad6c02bcd0 comp rx=0 tx=0).stop 2024-04-16T17:14:08.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.288+0000 7fad84a00640 1 -- 172.21.15.45:0/4011738747 shutdown_connections 2024-04-16T17:14:08.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.288+0000 7fad84a00640 1 --2- 172.21.15.45:0/4011738747 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fad8010e760 0x7fad80110b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:08.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.288+0000 7fad84a00640 1 --2- 172.21.15.45:0/4011738747 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fad80074cd0 0x7fad8010e220 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:08.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.288+0000 7fad84a00640 1 --2- 172.21.15.45:0/4011738747 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fad800743b0 0x7fad80074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:08.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.288+0000 7fad84a00640 1 -- 172.21.15.45:0/4011738747 >> 172.21.15.45:0/4011738747 conn(0x7fad800fd9d0 msgr2=0x7fad800ffdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:14:08.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.288+0000 7fad84a00640 1 -- 172.21.15.45:0/4011738747 shutdown_connections 2024-04-16T17:14:08.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.288+0000 7fad84a00640 1 -- 172.21.15.45:0/4011738747 wait complete. 2024-04-16T17:14:08.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.289+0000 7fad84a00640 1 Processor -- start 2024-04-16T17:14:08.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.289+0000 7fad84a00640 1 -- start start 2024-04-16T17:14:08.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.289+0000 7fad84a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fad800743b0 0x7fad801a0650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:08.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.289+0000 7fad84a00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fad80074cd0 0x7fad801a0b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:08.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.289+0000 7fad84a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fad8010e760 0x7fad8019a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:08.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.289+0000 7fad84a00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fad801144a0 con 0x7fad8010e760 2024-04-16T17:14:08.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.289+0000 7fad84a00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fad80114320 con 0x7fad80074cd0 2024-04-16T17:14:08.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.289+0000 7fad84a00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fad80114620 con 0x7fad800743b0 2024-04-16T17:14:08.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.289+0000 7fad7de00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fad80074cd0 0x7fad801a0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:08.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.289+0000 7fad7de00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fad80074cd0 0x7fad801a0b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:49110/0 (socket says 172.21.15.45:49110) 2024-04-16T17:14:08.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.289+0000 7fad7f200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fad8010e760 0x7fad8019a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:08.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.289+0000 7fad7de00640 1 -- 172.21.15.45:0/2528823797 learned_addr learned my addr 172.21.15.45:0/2528823797 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:14:08.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.289+0000 7fad7e800640 1 --2- 172.21.15.45:0/2528823797 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fad800743b0 0x7fad801a0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:08.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.290+0000 7fad7de00640 1 -- 172.21.15.45:0/2528823797 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fad800743b0 msgr2=0x7fad801a0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:08.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.290+0000 7fad7de00640 1 --2- 172.21.15.45:0/2528823797 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fad800743b0 0x7fad801a0650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:08.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.290+0000 7fad7de00640 1 -- 172.21.15.45:0/2528823797 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fad8010e760 msgr2=0x7fad8019a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:08.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.290+0000 7fad7de00640 1 --2- 172.21.15.45:0/2528823797 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fad8010e760 0x7fad8019a7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:08.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.290+0000 7fad7de00640 1 -- 172.21.15.45:0/2528823797 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad8019af90 con 0x7fad80074cd0 2024-04-16T17:14:08.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.290+0000 7fad7de00640 1 --2- 172.21.15.45:0/2528823797 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fad80074cd0 0x7fad801a0b90 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fad6c000c00 tx=0x7fad6c0043c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:14:08.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.290+0000 7fad77400640 1 -- 172.21.15.45:0/2528823797 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad6c042020 con 0x7fad80074cd0 2024-04-16T17:14:08.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.291+0000 7fad77400640 1 -- 172.21.15.45:0/2528823797 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fad6c0054e0 con 0x7fad80074cd0 2024-04-16T17:14:08.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.291+0000 7fad77400640 1 -- 172.21.15.45:0/2528823797 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad6c003b20 con 0x7fad80074cd0 2024-04-16T17:14:08.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.291+0000 7fad84a00640 1 -- 172.21.15.45:0/2528823797 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad8019b220 con 0x7fad80074cd0 2024-04-16T17:14:08.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.291+0000 7fad84a00640 1 -- 172.21.15.45:0/2528823797 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad801a74a0 con 0x7fad80074cd0 2024-04-16T17:14:08.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.294+0000 7fad88006740 1 Processor -- start 2024-04-16T17:14:08.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.294+0000 7fad88006740 1 -- start start 2024-04-16T17:14:08.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.294+0000 7fad88006740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5618c4c8df10 0x5618c4c8e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:08.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.294+0000 7fad88006740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5618c4c8e830 0x5618c4c96ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:08.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.294+0000 7fad88006740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5618c4c97410 0x5618c4c99800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:08.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.294+0000 7fad88006740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5618c4bf3020 con 0x5618c4c8df10 2024-04-16T17:14:08.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.294+0000 7fad88006740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5618c4b99460 con 0x5618c4c8e830 2024-04-16T17:14:08.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.294+0000 7fad88006740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5618c4beefe0 con 0x5618c4c97410 2024-04-16T17:14:08.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.295+0000 7fad7f200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5618c4c8df10 0x5618c4c8e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:08.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.295+0000 7fad7f200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5618c4c8df10 0x5618c4c8e2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:46930/0 (socket says 172.21.15.45:46930) 2024-04-16T17:14:08.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.295+0000 7fad7f200640 1 -- 172.21.15.45:0/789031576 learned_addr learned my addr 172.21.15.45:0/789031576 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:14:08.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.295+0000 7fad7e800640 1 --2- 172.21.15.45:0/789031576 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5618c4c8e830 0x5618c4c96ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:08.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.295+0000 7fad7f200640 1 -- 172.21.15.45:0/789031576 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5618c4c97410 msgr2=0x5618c4c99800 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-16T17:14:08.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.295+0000 7fad7f200640 1 --2- 172.21.15.45:0/789031576 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5618c4c97410 0x5618c4c99800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:08.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.295+0000 7fad7f200640 1 -- 172.21.15.45:0/789031576 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5618c4c8e830 msgr2=0x5618c4c96ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:14:08.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.295+0000 7fad7f200640 1 --2- 172.21.15.45:0/789031576 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5618c4c8e830 0x5618c4c96ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:08.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.295+0000 7fad7f200640 1 -- 172.21.15.45:0/789031576 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5618c4ba1040 con 0x5618c4c8df10 2024-04-16T17:14:08.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.295+0000 7fad7f200640 1 --2- 172.21.15.45:0/789031576 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5618c4c8df10 0x5618c4c8e2f0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fad78009f90 tx=0x7fad78010860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14e60b87c26a1d3 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:14:08.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.296+0000 7fad6b400640 1 -- 172.21.15.45:0/789031576 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad78011700 con 0x5618c4c8df10 2024-04-16T17:14:08.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.296+0000 7fad6b400640 1 -- 172.21.15.45:0/789031576 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fad78011d80 con 0x5618c4c8df10 2024-04-16T17:14:08.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.296+0000 7fad6b400640 1 -- 172.21.15.45:0/789031576 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad78018380 con 0x5618c4c8df10 2024-04-16T17:14:08.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.296+0000 7fad77400640 1 -- 172.21.15.45:0/2528823797 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fad6c045020 con 0x7fad80074cd0 2024-04-16T17:14:08.302 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.296+0000 7fad88006740 1 -- 172.21.15.45:0/789031576 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5618c4c8df10 msgr2=0x5618c4c8e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:08.302 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.296+0000 7fad88006740 1 --2- 172.21.15.45:0/789031576 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5618c4c8df10 0x5618c4c8e2f0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fad78009f90 tx=0x7fad78010860 comp rx=0 tx=0).stop 2024-04-16T17:14:08.302 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.296+0000 7fad88006740 1 -- 172.21.15.45:0/789031576 shutdown_connections 2024-04-16T17:14:08.302 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.296+0000 7fad88006740 1 --2- 172.21.15.45:0/789031576 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5618c4c97410 0x5618c4c99800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:08.302 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.296+0000 7fad88006740 1 --2- 172.21.15.45:0/789031576 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5618c4c8e830 0x5618c4c96ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:08.302 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.296+0000 7fad88006740 1 --2- 172.21.15.45:0/789031576 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5618c4c8df10 0x5618c4c8e2f0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:08.302 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.296+0000 7fad88006740 1 -- 172.21.15.45:0/789031576 >> 172.21.15.45:0/789031576 conn(0x5618c4c88640 msgr2=0x5618c4c88a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:14:08.303 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.297+0000 7fad88006740 1 -- 172.21.15.45:0/789031576 shutdown_connections 2024-04-16T17:14:08.303 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.297+0000 7fad88006740 1 -- 172.21.15.45:0/789031576 wait complete. 2024-04-16T17:14:08.303 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.297+0000 7fad88006740 1 -- start start 2024-04-16T17:14:08.303 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.297+0000 7fad88006740 1 Processor -- start 2024-04-16T17:14:08.303 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.297+0000 7fad77400640 1 --2- 172.21.15.45:0/2528823797 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fad640a3cc0 0x7fad640a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:08.303 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.297+0000 7fad77400640 1 -- 172.21.15.45:0/2528823797 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9584+0+0 (secure 0 0 0) 0x7fad6c0f27c0 con 0x7fad80074cd0 2024-04-16T17:14:08.303 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.298+0000 7fad7f200640 1 --2- 172.21.15.45:0/2528823797 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fad640a3cc0 0x7fad640a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:08.303 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.298+0000 7fad88006740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5618c4c8df10 0x5618c4c94450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:08.303 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.298+0000 7fad88006740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5618c4c8e830 0x5618c4c94990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:08.304 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.298+0000 7fad88006740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5618c4c97410 0x5618c4c94ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:08.304 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.298+0000 7fad88006740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5618c4c9a200 con 0x5618c4c97410 2024-04-16T17:14:08.304 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.298+0000 7fad88006740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5618c4bac830 con 0x5618c4c8e830 2024-04-16T17:14:08.304 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.298+0000 7fad88006740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5618c4c9a380 con 0x5618c4c8df10 2024-04-16T17:14:08.304 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.298+0000 7fad88006740 10 client.0 _ll_register_callbacks cb 0x5618c4bb4150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:14:08.304 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.298+0000 7fad7f200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5618c4c97410 0x5618c4c94ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:08.304 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.298+0000 7fad7de00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5618c4c8e830 0x5618c4c94990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:08.304 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.298+0000 7fad7e800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5618c4c8df10 0x5618c4c94450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:08.304 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.298+0000 7fad7f200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5618c4c97410 0x5618c4c94ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:46940/0 (socket says 172.21.15.45:46940) 2024-04-16T17:14:08.304 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.298+0000 7fad7f200640 1 -- 172.21.15.45:0/918066947 learned_addr learned my addr 172.21.15.45:0/918066947 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:14:08.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.299+0000 7fad7e800640 1 -- 172.21.15.45:0/918066947 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5618c4c8e830 msgr2=0x5618c4c94990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:14:08.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.299+0000 7fad7e800640 1 --2- 172.21.15.45:0/918066947 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5618c4c8e830 0x5618c4c94990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:08.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.299+0000 7fad7e800640 1 -- 172.21.15.45:0/918066947 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5618c4c97410 msgr2=0x5618c4c94ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:14:08.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.299+0000 7fad7e800640 1 --2- 172.21.15.45:0/918066947 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5618c4c97410 0x5618c4c94ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:08.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.299+0000 7fad7e800640 1 -- 172.21.15.45:0/918066947 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5618c4bb0920 con 0x5618c4c8df10 2024-04-16T17:14:08.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.299+0000 7fad7de00640 1 --2- 172.21.15.45:0/918066947 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5618c4c8e830 0x5618c4c94990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:14:08.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.299+0000 7fad7f200640 1 --2- 172.21.15.45:0/918066947 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5618c4c97410 0x5618c4c94ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:14:08.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.299+0000 7fad7f200640 1 --2- 172.21.15.45:0/2528823797 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fad640a3cc0 0x7fad640a6180 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fad78006010 tx=0x7fad78019a60 comp rx=0 tx=0).ready entity=mgr.25411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:14:08.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.299+0000 7fad7e800640 1 --2- 172.21.15.45:0/918066947 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5618c4c8df10 0x5618c4c94450 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fad700126d0 tx=0x7fad70012ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=346729406feea732 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:14:08.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.299+0000 7fad5fe00640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-04-16T17:14:08.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.299+0000 7fad5fe00640 1 -- 172.21.15.45:0/918066947 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad7000f560 con 0x5618c4c8df10 2024-04-16T17:14:08.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.299+0000 7fad88006740 1 -- 172.21.15.45:0/918066947 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fad78011d80 con 0x5618c4c8df10 2024-04-16T17:14:08.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.299+0000 7fad5fe00640 1 -- 172.21.15.45:0/918066947 <== mon.1 v2:172.21.15.117:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9584+0+0 (secure 0 0 0) 0x7fad7000f700 con 0x5618c4c8df10 2024-04-16T17:14:08.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.300+0000 7fad5fe00640 1 -- 172.21.15.45:0/918066947 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7fad7000f900 con 0x5618c4c8df10 2024-04-16T17:14:08.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.300+0000 7fad88006740 10 client.27580 fetch_fsmap learned FSMap version 42 2024-04-16T17:14:08.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.300+0000 7fad88006740 1 -- 172.21.15.45:0/918066947 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x5618c4b7e7f0 con 0x5618c4c8df10 2024-04-16T17:14:08.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.300+0000 7fad5fe00640 1 -- 172.21.15.45:0/918066947 <== mon.1 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fad78011d80 con 0x5618c4c8df10 2024-04-16T17:14:08.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.300+0000 7fad5fe00640 1 -- 172.21.15.45:0/918066947 <== mon.1 v2:172.21.15.117:3300/0 5 ==== fsmap(e 42) ==== 8362+0+0 (secure 0 0 0) 0x7fad70010bc0 con 0x5618c4c8df10 2024-04-16T17:14:08.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.300+0000 7fad88006740 10 client.27580 fetch_fsmap finished waiting for FSMap version 42 2024-04-16T17:14:08.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.300+0000 7fad88006740 10 client.27580 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-04-16T17:14:08.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.300+0000 7fad88006740 20 client.27580 populate_metadata read hostname 'smithi045' 2024-04-16T17:14:08.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.300+0000 7fad88006740 1 --2- 172.21.15.45:0/918066947 >> [v2:172.21.15.45:6840/3755310822,v1:172.21.15.45:6841/3755310822] conn(0x5618c4ca7840 0x5618c4ca9c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:08.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.300+0000 7fad88006740 4 client.27580 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:14:08.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.300+0000 7fad88006740 1 -- 172.21.15.45:0/918066947 --> [v2:172.21.15.45:6840/3755310822,v1:172.21.15.45:6841/3755310822] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5618c4c00a40 con 0x5618c4ca7840 2024-04-16T17:14:08.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.301+0000 7fad7de00640 1 --2- 172.21.15.45:0/918066947 >> [v2:172.21.15.45:6840/3755310822,v1:172.21.15.45:6841/3755310822] conn(0x5618c4ca7840 0x5618c4ca9c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:08.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.301+0000 7fad7de00640 1 --2- 172.21.15.45:0/918066947 >> [v2:172.21.15.45:6840/3755310822,v1:172.21.15.45:6841/3755310822] conn(0x5618c4ca7840 0x5618c4ca9c60 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=605aecaee1676105 server_cookie=84bd2ac4ab933e00 in_seq=0 out_seq=0 2024-04-16T17:14:08.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.301+0000 7fad5fe00640 10 client.27580 ms_handle_connect on v2:172.21.15.45:6840/3755310822 2024-04-16T17:14:08.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.302+0000 7fad5fe00640 1 -- 172.21.15.45:0/918066947 <== mds.1 v2:172.21.15.45:6840/3755310822 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5618c4c00a40 con 0x5618c4ca7840 2024-04-16T17:14:08.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.302+0000 7fad5fe00640 10 client.27580 handle_command_reply: tid=0 2024-04-16T17:14:08.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.302+0000 7fad88006740 1 client.27580 shutdown 2024-04-16T17:14:08.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.302+0000 7fad88006740 1 -- 172.21.15.45:0/918066947 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5618c4c8df10 msgr2=0x5618c4c94450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:08.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.302+0000 7fad88006740 1 --2- 172.21.15.45:0/918066947 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5618c4c8df10 0x5618c4c94450 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fad700126d0 tx=0x7fad70012ad0 comp rx=0 tx=0).stop 2024-04-16T17:14:08.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.302+0000 7fad88006740 1 -- 172.21.15.45:0/918066947 shutdown_connections 2024-04-16T17:14:08.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.302+0000 7fad88006740 1 --2- 172.21.15.45:0/918066947 >> [v2:172.21.15.45:6840/3755310822,v1:172.21.15.45:6841/3755310822] conn(0x5618c4ca7840 0x5618c4ca9c60 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:08.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.302+0000 7fad88006740 1 --2- 172.21.15.45:0/918066947 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5618c4c97410 0x5618c4c94ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:08.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.302+0000 7fad88006740 1 --2- 172.21.15.45:0/918066947 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5618c4c8e830 0x5618c4c94990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:08.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.302+0000 7fad88006740 1 --2- 172.21.15.45:0/918066947 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5618c4c8df10 0x5618c4c94450 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:08.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.302+0000 7fad5fe00640 0 client.27580 ms_handle_reset on v2:172.21.15.45:6840/3755310822 2024-04-16T17:14:08.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.302+0000 7fad88006740 1 -- 172.21.15.45:0/918066947 >> 172.21.15.45:0/918066947 conn(0x5618c4c88640 msgr2=0x5618c4c8a390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:14:08.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.302+0000 7fad88006740 1 -- 172.21.15.45:0/918066947 shutdown_connections 2024-04-16T17:14:08.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.302+0000 7fad88006740 1 -- 172.21.15.45:0/918066947 wait complete. 2024-04-16T17:14:08.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.302+0000 7fad88006740 20 client.27580 trim_cache size 0 max 16384 2024-04-16T17:14:08.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.317+0000 7fad88006740 1 Processor -- start 2024-04-16T17:14:08.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.317+0000 7fad88006740 1 -- start start 2024-04-16T17:14:08.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.317+0000 7fad88006740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5618c4c8df10 0x5618c4c96b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:08.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.317+0000 7fad88006740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5618c4c8e830 0x5618c4ca30f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:08.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.318+0000 7fad88006740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5618c4c97410 0x5618c4c935b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:08.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.318+0000 7fad88006740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fad801b4070 con 0x5618c4c8e830 2024-04-16T17:14:08.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.318+0000 7fad88006740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fad801b18e0 con 0x5618c4c97410 2024-04-16T17:14:08.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.318+0000 7fad88006740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fad801af730 con 0x5618c4c8df10 2024-04-16T17:14:08.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.318+0000 7fad7de00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5618c4c8e830 0x5618c4ca30f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:08.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.318+0000 7fad7e800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5618c4c8df10 0x5618c4c96b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:08.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.318+0000 7fad7de00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5618c4c8e830 0x5618c4ca30f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:46944/0 (socket says 172.21.15.45:46944) 2024-04-16T17:14:08.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.318+0000 7fad7de00640 1 -- 172.21.15.45:0/3808207600 learned_addr learned my addr 172.21.15.45:0/3808207600 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:14:08.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.318+0000 7fad7f200640 1 --2- 172.21.15.45:0/3808207600 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5618c4c97410 0x5618c4c935b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:08.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.318+0000 7fad7de00640 1 -- 172.21.15.45:0/3808207600 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5618c4c8df10 msgr2=0x5618c4c96b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:14:08.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.318+0000 7fad7de00640 1 --2- 172.21.15.45:0/3808207600 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5618c4c8df10 0x5618c4c96b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:08.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.318+0000 7fad7de00640 1 -- 172.21.15.45:0/3808207600 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5618c4c97410 msgr2=0x5618c4c935b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:14:08.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.318+0000 7fad7de00640 1 --2- 172.21.15.45:0/3808207600 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5618c4c97410 0x5618c4c935b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:08.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.318+0000 7fad7de00640 1 -- 172.21.15.45:0/3808207600 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5618c49053f0 con 0x5618c4c8e830 2024-04-16T17:14:08.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.318+0000 7fad7f200640 1 --2- 172.21.15.45:0/3808207600 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5618c4c97410 0x5618c4c935b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:14:08.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.318+0000 7fad7e800640 1 --2- 172.21.15.45:0/3808207600 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5618c4c8df10 0x5618c4c96b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:14:08.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.319+0000 7fad7de00640 1 --2- 172.21.15.45:0/3808207600 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5618c4c8e830 0x5618c4ca30f0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fad6c0bae20 tx=0x7fad6c005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1d6b0fbc4a3e70c server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:14:08.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.319+0000 7fad6aa00640 1 -- 172.21.15.45:0/3808207600 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad6c0beb50 con 0x5618c4c8e830 2024-04-16T17:14:08.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.319+0000 7fad6aa00640 1 -- 172.21.15.45:0/3808207600 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fad6c0be450 con 0x5618c4c8e830 2024-04-16T17:14:08.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.319+0000 7fad6aa00640 1 -- 172.21.15.45:0/3808207600 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad6c0bac50 con 0x5618c4c8e830 2024-04-16T17:14:08.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.320+0000 7fad88006740 1 -- 172.21.15.45:0/3808207600 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5618c4c8e830 msgr2=0x5618c4ca30f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:08.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.320+0000 7fad88006740 1 --2- 172.21.15.45:0/3808207600 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5618c4c8e830 0x5618c4ca30f0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fad6c0bae20 tx=0x7fad6c005d50 comp rx=0 tx=0).stop 2024-04-16T17:14:08.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.320+0000 7fad88006740 1 -- 172.21.15.45:0/3808207600 shutdown_connections 2024-04-16T17:14:08.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.320+0000 7fad88006740 1 --2- 172.21.15.45:0/3808207600 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5618c4c97410 0x5618c4c935b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:08.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.320+0000 7fad88006740 1 --2- 172.21.15.45:0/3808207600 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5618c4c8e830 0x5618c4ca30f0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:08.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.320+0000 7fad88006740 1 --2- 172.21.15.45:0/3808207600 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5618c4c8df10 0x5618c4c96b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:08.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.320+0000 7fad88006740 1 -- 172.21.15.45:0/3808207600 >> 172.21.15.45:0/3808207600 conn(0x5618c4c88640 msgr2=0x5618c4c89a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:14:08.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.320+0000 7fad88006740 1 -- 172.21.15.45:0/3808207600 shutdown_connections 2024-04-16T17:14:08.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.320+0000 7fad88006740 1 -- 172.21.15.45:0/3808207600 wait complete. 2024-04-16T17:14:08.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.320+0000 7fad88006740 1 -- start start 2024-04-16T17:14:08.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.321+0000 7fad88006740 1 Processor -- start 2024-04-16T17:14:08.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.321+0000 7fad88006740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5618c4c8df10 0x5618c4d06180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:08.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.321+0000 7fad88006740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5618c4c8e830 0x5618c4d066c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:08.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.321+0000 7fad88006740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5618c4c97410 0x5618c4c9a970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:08.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.321+0000 7fad88006740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fad801afda0 con 0x5618c4c8e830 2024-04-16T17:14:08.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.321+0000 7fad88006740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fad801b1210 con 0x5618c4c8df10 2024-04-16T17:14:08.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.321+0000 7fad88006740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fad801b0470 con 0x5618c4c97410 2024-04-16T17:14:08.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.321+0000 7fad88006740 10 client.0 _ll_register_callbacks cb 0x7fad70009850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:14:08.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.321+0000 7fad7de00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5618c4c8e830 0x5618c4d066c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:08.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.321+0000 7fad7e800640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5618c4c8df10 0x5618c4d06180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:08.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.321+0000 7fad7de00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5618c4c8e830 0x5618c4d066c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:46946/0 (socket says 172.21.15.45:46946) 2024-04-16T17:14:08.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.321+0000 7fad7e800640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5618c4c8df10 0x5618c4d06180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:49136/0 (socket says 172.21.15.45:49136) 2024-04-16T17:14:08.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.321+0000 7fad7de00640 1 -- 172.21.15.45:0/2731418125 learned_addr learned my addr 172.21.15.45:0/2731418125 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:14:08.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.321+0000 7fad7f200640 1 --2- 172.21.15.45:0/2731418125 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5618c4c97410 0x5618c4c9a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:08.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.322+0000 7fad7e800640 1 -- 172.21.15.45:0/2731418125 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5618c4c97410 msgr2=0x5618c4c9a970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:14:08.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.322+0000 7fad7e800640 1 --2- 172.21.15.45:0/2731418125 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5618c4c97410 0x5618c4c9a970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:08.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.322+0000 7fad7e800640 1 -- 172.21.15.45:0/2731418125 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5618c4c8e830 msgr2=0x5618c4d066c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:14:08.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.322+0000 7fad7e800640 1 --2- 172.21.15.45:0/2731418125 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5618c4c8e830 0x5618c4d066c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:08.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.322+0000 7fad7e800640 1 -- 172.21.15.45:0/2731418125 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5618c49cf7d0 con 0x5618c4c8df10 2024-04-16T17:14:08.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.322+0000 7fad7de00640 1 --2- 172.21.15.45:0/2731418125 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5618c4c8e830 0x5618c4d066c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:14:08.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.322+0000 7fad7e800640 1 --2- 172.21.15.45:0/2731418125 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5618c4c8df10 0x5618c4d06180 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fad700097e0 tx=0x7fad70005cd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f55831870cda33d2 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:14:08.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.322+0000 7fad5fe00640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-04-16T17:14:08.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.322+0000 7fad5fe00640 1 -- 172.21.15.45:0/2731418125 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad7000faa0 con 0x5618c4c8df10 2024-04-16T17:14:08.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.323+0000 7fad88006740 1 -- 172.21.15.45:0/2731418125 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fad6c0be450 con 0x5618c4c8df10 2024-04-16T17:14:08.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.323+0000 7fad5fe00640 1 -- 172.21.15.45:0/2731418125 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9584+0+0 (secure 0 0 0) 0x7fad7000fc40 con 0x5618c4c8df10 2024-04-16T17:14:08.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.323+0000 7fad5fe00640 1 -- 172.21.15.45:0/2731418125 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7fad7000fe40 con 0x5618c4c8df10 2024-04-16T17:14:08.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.323+0000 7fad88006740 10 client.27647 fetch_fsmap learned FSMap version 42 2024-04-16T17:14:08.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.323+0000 7fad88006740 1 -- 172.21.15.45:0/2731418125 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x5618c4d0d710 con 0x5618c4c8df10 2024-04-16T17:14:08.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.323+0000 7fad5fe00640 1 -- 172.21.15.45:0/2731418125 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fad6c0be450 con 0x5618c4c8df10 2024-04-16T17:14:08.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.323+0000 7fad5fe00640 1 -- 172.21.15.45:0/2731418125 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 42) ==== 8362+0+0 (secure 0 0 0) 0x7fad700035f0 con 0x5618c4c8df10 2024-04-16T17:14:08.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.323+0000 7fad88006740 10 client.27647 fetch_fsmap finished waiting for FSMap version 42 2024-04-16T17:14:08.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.323+0000 7fad88006740 10 client.27647 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-04-16T17:14:08.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.323+0000 7fad88006740 20 client.27647 populate_metadata read hostname 'smithi045' 2024-04-16T17:14:08.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.323+0000 7fad88006740 1 --2- 172.21.15.45:0/2731418125 >> [v2:172.21.15.45:6840/3755310822,v1:172.21.15.45:6841/3755310822] conn(0x5618c4d0de80 0x5618c4d102a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:08.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.324+0000 7fad88006740 4 client.27647 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T17:14:08.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.324+0000 7fad88006740 1 -- 172.21.15.45:0/2731418125 --> [v2:172.21.15.45:6840/3755310822,v1:172.21.15.45:6841/3755310822] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5618c4bfa8b0 con 0x5618c4d0de80 2024-04-16T17:14:08.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.324+0000 7fad7de00640 1 --2- 172.21.15.45:0/2731418125 >> [v2:172.21.15.45:6840/3755310822,v1:172.21.15.45:6841/3755310822] conn(0x5618c4d0de80 0x5618c4d102a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:08.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.324+0000 7fad7de00640 1 --2- 172.21.15.45:0/2731418125 >> [v2:172.21.15.45:6840/3755310822,v1:172.21.15.45:6841/3755310822] conn(0x5618c4d0de80 0x5618c4d102a0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=fac128dd53c2fbe0 server_cookie=d65bfd4a8937c367 in_seq=0 out_seq=0 2024-04-16T17:14:08.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.324+0000 7fad5fe00640 10 client.27647 ms_handle_connect on v2:172.21.15.45:6840/3755310822 2024-04-16T17:14:08.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.325+0000 7fad5fe00640 1 -- 172.21.15.45:0/2731418125 <== mds.1 v2:172.21.15.45:6840/3755310822 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5618c4bfa8b0 con 0x5618c4d0de80 2024-04-16T17:14:08.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.325+0000 7fad5fe00640 10 client.27647 handle_command_reply: tid=0 2024-04-16T17:14:08.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.325+0000 7fad88006740 1 client.27647 shutdown 2024-04-16T17:14:08.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.325+0000 7fad88006740 1 -- 172.21.15.45:0/2731418125 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5618c4c8df10 msgr2=0x5618c4d06180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:08.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.325+0000 7fad88006740 1 --2- 172.21.15.45:0/2731418125 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5618c4c8df10 0x5618c4d06180 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fad700097e0 tx=0x7fad70005cd0 comp rx=0 tx=0).stop 2024-04-16T17:14:08.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.325+0000 7fad88006740 1 -- 172.21.15.45:0/2731418125 shutdown_connections 2024-04-16T17:14:08.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.325+0000 7fad88006740 1 --2- 172.21.15.45:0/2731418125 >> [v2:172.21.15.45:6840/3755310822,v1:172.21.15.45:6841/3755310822] conn(0x5618c4d0de80 0x5618c4d102a0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:08.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.325+0000 7fad88006740 1 --2- 172.21.15.45:0/2731418125 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5618c4c97410 0x5618c4c9a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:08.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.325+0000 7fad88006740 1 --2- 172.21.15.45:0/2731418125 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5618c4c8e830 0x5618c4d066c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:08.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.325+0000 7fad5fe00640 0 client.27647 ms_handle_reset on v2:172.21.15.45:6840/3755310822 2024-04-16T17:14:08.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.325+0000 7fad88006740 1 --2- 172.21.15.45:0/2731418125 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5618c4c8df10 0x5618c4d06180 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:08.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.325+0000 7fad88006740 1 -- 172.21.15.45:0/2731418125 >> 172.21.15.45:0/2731418125 conn(0x5618c4c88640 msgr2=0x5618c4c86e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:14:08.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.325+0000 7fad88006740 1 -- 172.21.15.45:0/2731418125 shutdown_connections 2024-04-16T17:14:08.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.325+0000 7fad88006740 1 -- 172.21.15.45:0/2731418125 wait complete. 2024-04-16T17:14:08.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.325+0000 7fad88006740 20 client.27647 trim_cache size 0 max 16384 2024-04-16T17:14:08.332 INFO:teuthology.orchestra.run.smithi045.stdout:[] 2024-04-16T17:14:08.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.326+0000 7fad84a00640 1 -- 172.21.15.45:0/2528823797 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fad640a3cc0 msgr2=0x7fad640a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:08.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.326+0000 7fad84a00640 1 --2- 172.21.15.45:0/2528823797 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fad640a3cc0 0x7fad640a6180 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fad78006010 tx=0x7fad78019a60 comp rx=0 tx=0).stop 2024-04-16T17:14:08.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.326+0000 7fad84a00640 1 -- 172.21.15.45:0/2528823797 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fad80074cd0 msgr2=0x7fad801a0b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:08.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.326+0000 7fad84a00640 1 --2- 172.21.15.45:0/2528823797 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fad80074cd0 0x7fad801a0b90 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fad6c000c00 tx=0x7fad6c0043c0 comp rx=0 tx=0).stop 2024-04-16T17:14:08.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.326+0000 7fad84a00640 1 -- 172.21.15.45:0/2528823797 shutdown_connections 2024-04-16T17:14:08.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.326+0000 7fad84a00640 1 --2- 172.21.15.45:0/2528823797 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fad640a3cc0 0x7fad640a6180 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:08.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.326+0000 7fad84a00640 1 --2- 172.21.15.45:0/2528823797 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fad8010e760 0x7fad8019a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:08.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.326+0000 7fad84a00640 1 --2- 172.21.15.45:0/2528823797 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fad80074cd0 0x7fad801a0b90 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:08.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.326+0000 7fad84a00640 1 --2- 172.21.15.45:0/2528823797 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fad800743b0 0x7fad801a0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:08.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.326+0000 7fad84a00640 1 -- 172.21.15.45:0/2528823797 >> 172.21.15.45:0/2528823797 conn(0x7fad800fd9d0 msgr2=0x7fad800fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:14:08.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.326+0000 7fad84a00640 1 -- 172.21.15.45:0/2528823797 shutdown_connections 2024-04-16T17:14:08.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.326+0000 7fad84a00640 1 -- 172.21.15.45:0/2528823797 wait complete. 2024-04-16T17:14:08.341 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-16T17:14:08.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.536+0000 7ff21f200640 1 -- 172.21.15.45:0/3488317795 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff218103080 msgr2=0x7ff218103460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:08.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.536+0000 7ff21f200640 1 --2- 172.21.15.45:0/3488317795 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff218103080 0x7ff218103460 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7ff210009ca0 tx=0x7ff21002c190 comp rx=0 tx=0).stop 2024-04-16T17:14:08.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.536+0000 7ff21f200640 1 -- 172.21.15.45:0/3488317795 shutdown_connections 2024-04-16T17:14:08.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.536+0000 7ff21f200640 1 --2- 172.21.15.45:0/3488317795 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff2181039a0 0x7ff21810be80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:08.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.536+0000 7ff21f200640 1 --2- 172.21.15.45:0/3488317795 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff21810c4a0 0x7ff21810e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:08.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.536+0000 7ff21f200640 1 --2- 172.21.15.45:0/3488317795 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff218103080 0x7ff218103460 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:08.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.536+0000 7ff21f200640 1 -- 172.21.15.45:0/3488317795 >> 172.21.15.45:0/3488317795 conn(0x7ff2180fc880 msgr2=0x7ff2180feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:14:08.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.536+0000 7ff21f200640 1 -- 172.21.15.45:0/3488317795 shutdown_connections 2024-04-16T17:14:08.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.536+0000 7ff21f200640 1 -- 172.21.15.45:0/3488317795 wait complete. 2024-04-16T17:14:08.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.537+0000 7ff21f200640 1 Processor -- start 2024-04-16T17:14:08.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.537+0000 7ff21f200640 1 -- start start 2024-04-16T17:14:08.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.538+0000 7ff21f200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff218103080 0x7ff218100cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:08.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.538+0000 7ff21f200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff2181039a0 0x7ff218101210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:08.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.538+0000 7ff21f200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff21810c4a0 0x7ff218101750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:08.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.538+0000 7ff21f200640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7ff218112260 con 0x7ff21810c4a0 2024-04-16T17:14:08.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.538+0000 7ff21f200640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7ff2181120e0 con 0x7ff2181039a0 2024-04-16T17:14:08.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.538+0000 7ff21f200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7ff2181123e0 con 0x7ff218103080 2024-04-16T17:14:08.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.538+0000 7ff21e800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff21810c4a0 0x7ff218101750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:08.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.538+0000 7ff21e800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff21810c4a0 0x7ff218101750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:46964/0 (socket says 172.21.15.45:46964) 2024-04-16T17:14:08.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.538+0000 7ff21e800640 1 -- 172.21.15.45:0/783402670 learned_addr learned my addr 172.21.15.45:0/783402670 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:14:08.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.538+0000 7ff21de00640 1 --2- 172.21.15.45:0/783402670 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff218103080 0x7ff218100cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:08.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.538+0000 7ff21d400640 1 --2- 172.21.15.45:0/783402670 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff2181039a0 0x7ff218101210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:08.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.538+0000 7ff21e800640 1 -- 172.21.15.45:0/783402670 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff218103080 msgr2=0x7ff218100cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:08.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.538+0000 7ff21e800640 1 --2- 172.21.15.45:0/783402670 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff218103080 0x7ff218100cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:08.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.538+0000 7ff21e800640 1 -- 172.21.15.45:0/783402670 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff2181039a0 msgr2=0x7ff218101210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:08.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.538+0000 7ff21e800640 1 --2- 172.21.15.45:0/783402670 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff2181039a0 0x7ff218101210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:08.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.539+0000 7ff21e800640 1 -- 172.21.15.45:0/783402670 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2181a6c40 con 0x7ff21810c4a0 2024-04-16T17:14:08.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.539+0000 7ff21de00640 1 --2- 172.21.15.45:0/783402670 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff218103080 0x7ff218100cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:14:08.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.539+0000 7ff21e800640 1 --2- 172.21.15.45:0/783402670 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff21810c4a0 0x7ff218101750 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7ff21400e970 tx=0x7ff21400ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:14:08.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.539+0000 7ff20ea00640 1 -- 172.21.15.45:0/783402670 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff21400ccc0 con 0x7ff21810c4a0 2024-04-16T17:14:08.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.539+0000 7ff20ea00640 1 -- 172.21.15.45:0/783402670 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff21400ce60 con 0x7ff21810c4a0 2024-04-16T17:14:08.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.539+0000 7ff21f200640 1 -- 172.21.15.45:0/783402670 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2181a6f30 con 0x7ff21810c4a0 2024-04-16T17:14:08.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.539+0000 7ff20ea00640 1 -- 172.21.15.45:0/783402670 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff214010490 con 0x7ff21810c4a0 2024-04-16T17:14:08.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.539+0000 7ff21f200640 1 -- 172.21.15.45:0/783402670 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2181a7400 con 0x7ff21810c4a0 2024-04-16T17:14:08.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.543+0000 7ff20ea00640 1 -- 172.21.15.45:0/783402670 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7ff214005020 con 0x7ff21810c4a0 2024-04-16T17:14:08.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.543+0000 7ff223623740 1 Processor -- start 2024-04-16T17:14:08.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.543+0000 7ff223623740 1 -- start start 2024-04-16T17:14:08.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.543+0000 7ff223623740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559e4110ef10 0x559e4110f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:08.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.543+0000 7ff223623740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559e4110f830 0x559e41117ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:08.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.543+0000 7ff223623740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559e41118410 0x559e4111a800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:08.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.543+0000 7ff223623740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x559e41074020 con 0x559e4110ef10 2024-04-16T17:14:08.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.543+0000 7ff223623740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x559e4101a460 con 0x559e41118410 2024-04-16T17:14:08.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.543+0000 7ff223623740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x559e4106ffe0 con 0x559e4110f830 2024-04-16T17:14:08.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.544+0000 7ff21d400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559e4110ef10 0x559e4110f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:08.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.544+0000 7ff21d400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559e4110ef10 0x559e4110f2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:46972/0 (socket says 172.21.15.45:46972) 2024-04-16T17:14:08.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.544+0000 7ff21de00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559e4110f830 0x559e41117ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:08.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.544+0000 7ff21d400640 1 -- 172.21.15.45:0/3161404516 learned_addr learned my addr 172.21.15.45:0/3161404516 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:14:08.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.544+0000 7ff21d400640 1 --2- 172.21.15.45:0/3161404516 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559e41118410 0x559e4111a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:08.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.544+0000 7ff20ea00640 1 --2- 172.21.15.45:0/783402670 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7ff2040a3d50 0x7ff2040a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:08.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.544+0000 7ff21de00640 1 -- 172.21.15.45:0/3161404516 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559e41118410 msgr2=0x559e4111a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:14:08.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.544+0000 7ff21de00640 1 --2- 172.21.15.45:0/3161404516 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559e41118410 0x559e4111a800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:08.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.544+0000 7ff21de00640 1 -- 172.21.15.45:0/3161404516 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559e4110ef10 msgr2=0x559e4110f2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:14:08.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.544+0000 7ff21de00640 1 --2- 172.21.15.45:0/3161404516 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559e4110ef10 0x559e4110f2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:08.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.544+0000 7ff21de00640 1 -- 172.21.15.45:0/3161404516 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559e41022040 con 0x559e4110f830 2024-04-16T17:14:08.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.544+0000 7ff21d400640 1 --2- 172.21.15.45:0/3161404516 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559e4110ef10 0x559e4110f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:14:08.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.544+0000 7ff20ea00640 1 -- 172.21.15.45:0/783402670 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9584+0+0 (secure 0 0 0) 0x7ff2140d0fc0 con 0x7ff21810c4a0 2024-04-16T17:14:08.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.545+0000 7ff21e800640 1 --2- 172.21.15.45:0/783402670 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7ff2040a3d50 0x7ff2040a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:08.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.545+0000 7ff21de00640 1 --2- 172.21.15.45:0/3161404516 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559e4110f830 0x559e41117ed0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7ff210006010 tx=0x7ff2100098c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f16058781322c46a server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:14:08.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.545+0000 7ff203000640 1 -- 172.21.15.45:0/3161404516 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff210043070 con 0x559e4110f830 2024-04-16T17:14:08.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.545+0000 7ff203000640 1 -- 172.21.15.45:0/3161404516 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff210003e50 con 0x559e4110f830 2024-04-16T17:14:08.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.545+0000 7ff203000640 1 -- 172.21.15.45:0/3161404516 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff210039040 con 0x559e4110f830 2024-04-16T17:14:08.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.546+0000 7ff223623740 1 -- 172.21.15.45:0/3161404516 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559e4110f830 msgr2=0x559e41117ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:08.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.546+0000 7ff21e800640 1 --2- 172.21.15.45:0/783402670 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7ff2040a3d50 0x7ff2040a6210 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7ff21409cad0 tx=0x7ff214009720 comp rx=0 tx=0).ready entity=mgr.25411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:14:08.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.546+0000 7ff223623740 1 --2- 172.21.15.45:0/3161404516 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559e4110f830 0x559e41117ed0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7ff210006010 tx=0x7ff2100098c0 comp rx=0 tx=0).stop 2024-04-16T17:14:08.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.546+0000 7ff223623740 1 -- 172.21.15.45:0/3161404516 shutdown_connections 2024-04-16T17:14:08.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.546+0000 7ff223623740 1 --2- 172.21.15.45:0/3161404516 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559e4110f830 0x559e41117ed0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:08.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.546+0000 7ff223623740 1 --2- 172.21.15.45:0/3161404516 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559e41118410 0x559e4111a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:08.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.546+0000 7ff223623740 1 --2- 172.21.15.45:0/3161404516 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559e4110ef10 0x559e4110f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:08.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.546+0000 7ff223623740 1 -- 172.21.15.45:0/3161404516 >> 172.21.15.45:0/3161404516 conn(0x559e41109640 msgr2=0x559e41109a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:14:08.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.546+0000 7ff223623740 1 -- 172.21.15.45:0/3161404516 shutdown_connections 2024-04-16T17:14:08.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.546+0000 7ff223623740 1 -- 172.21.15.45:0/3161404516 wait complete. 2024-04-16T17:14:08.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.547+0000 7ff223623740 1 -- start start 2024-04-16T17:14:08.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.547+0000 7ff223623740 1 Processor -- start 2024-04-16T17:14:08.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.547+0000 7ff223623740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559e4110ef10 0x559e41115470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:08.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.547+0000 7ff223623740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559e4110f830 0x559e411159b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:08.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.547+0000 7ff223623740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559e41118410 0x559e41115ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:08.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.547+0000 7ff223623740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x559e4111b1a0 con 0x559e4110ef10 2024-04-16T17:14:08.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.547+0000 7ff223623740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x559e4102d830 con 0x559e41118410 2024-04-16T17:14:08.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.547+0000 7ff223623740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x559e4111b320 con 0x559e4110f830 2024-04-16T17:14:08.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.547+0000 7ff223623740 10 client.0 _ll_register_callbacks cb 0x559e41035150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:14:08.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.548+0000 7ff21de00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559e4110f830 0x559e411159b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:08.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.548+0000 7ff21de00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559e4110f830 0x559e411159b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:45726/0 (socket says 172.21.15.45:45726) 2024-04-16T17:14:08.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.548+0000 7ff21de00640 1 -- 172.21.15.45:0/430800209 learned_addr learned my addr 172.21.15.45:0/430800209 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:14:08.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.548+0000 7ff21d400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559e4110ef10 0x559e41115470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:08.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.548+0000 7ff21d400640 1 --2- 172.21.15.45:0/430800209 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559e41118410 0x559e41115ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:08.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.548+0000 7ff21de00640 1 -- 172.21.15.45:0/430800209 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559e41118410 msgr2=0x559e41115ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:14:08.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.548+0000 7ff21de00640 1 --2- 172.21.15.45:0/430800209 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559e41118410 0x559e41115ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:08.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.548+0000 7ff21de00640 1 -- 172.21.15.45:0/430800209 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559e4110ef10 msgr2=0x559e41115470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:14:08.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.548+0000 7ff21de00640 1 --2- 172.21.15.45:0/430800209 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559e4110ef10 0x559e41115470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:08.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.548+0000 7ff21de00640 1 -- 172.21.15.45:0/430800209 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559e41031920 con 0x559e4110f830 2024-04-16T17:14:08.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.548+0000 7ff21d400640 1 --2- 172.21.15.45:0/430800209 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559e41118410 0x559e41115ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:14:08.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.548+0000 7ff21de00640 1 --2- 172.21.15.45:0/430800209 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559e4110f830 0x559e411159b0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7ff210000f80 tx=0x7ff2100323c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=55d45aadf9ca9eef server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:14:08.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.548+0000 7ff1f7400640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-04-16T17:14:08.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.549+0000 7ff1f7400640 1 -- 172.21.15.45:0/430800209 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff21004e070 con 0x559e4110f830 2024-04-16T17:14:08.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.549+0000 7ff1f7400640 1 -- 172.21.15.45:0/430800209 <== mon.1 v2:172.21.15.117:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9584+0+0 (secure 0 0 0) 0x7ff210034c80 con 0x559e4110f830 2024-04-16T17:14:08.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.549+0000 7ff223623740 1 -- 172.21.15.45:0/430800209 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff210003e50 con 0x559e4110f830 2024-04-16T17:14:08.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.549+0000 7ff1f7400640 1 -- 172.21.15.45:0/430800209 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7ff21005a050 con 0x559e4110f830 2024-04-16T17:14:08.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.549+0000 7ff223623740 10 client.27598 fetch_fsmap learned FSMap version 42 2024-04-16T17:14:08.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.549+0000 7ff223623740 1 -- 172.21.15.45:0/430800209 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x559e40fff7f0 con 0x559e4110f830 2024-04-16T17:14:08.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.549+0000 7ff1f7400640 1 -- 172.21.15.45:0/430800209 <== mon.1 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff210003e50 con 0x559e4110f830 2024-04-16T17:14:08.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.550+0000 7ff1f7400640 1 -- 172.21.15.45:0/430800209 <== mon.1 v2:172.21.15.117:3300/0 5 ==== fsmap(e 42) ==== 8362+0+0 (secure 0 0 0) 0x7ff210004390 con 0x559e4110f830 2024-04-16T17:14:08.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.550+0000 7ff223623740 10 client.27598 fetch_fsmap finished waiting for FSMap version 42 2024-04-16T17:14:08.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.550+0000 7ff223623740 10 client.27598 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:14:08.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.550+0000 7ff223623740 20 client.27598 populate_metadata read hostname 'smithi045' 2024-04-16T17:14:08.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.550+0000 7ff223623740 1 --2- 172.21.15.45:0/430800209 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x559e411285f0 0x559e4112a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:08.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.550+0000 7ff223623740 4 client.27598 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:14:08.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.550+0000 7ff223623740 1 -- 172.21.15.45:0/430800209 --> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559e41081a40 con 0x559e411285f0 2024-04-16T17:14:08.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.550+0000 7ff21d400640 1 --2- 172.21.15.45:0/430800209 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x559e411285f0 0x559e4112a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:08.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.551+0000 7ff1f7400640 10 client.27598 ms_handle_connect on v2:172.21.15.104:6838/3660070592 2024-04-16T17:14:08.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.551+0000 7ff21d400640 1 --2- 172.21.15.45:0/430800209 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x559e411285f0 0x559e4112a9e0 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3a37540123fbcbf3 server_cookie=578c3c6b9175d9f9 in_seq=0 out_seq=0 2024-04-16T17:14:08.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.552+0000 7ff1f7400640 1 -- 172.21.15.45:0/430800209 <== mds.0 v2:172.21.15.104:6838/3660070592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559e41081a40 con 0x559e411285f0 2024-04-16T17:14:08.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.552+0000 7ff1f7400640 10 client.27598 handle_command_reply: tid=0 2024-04-16T17:14:08.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.552+0000 7ff223623740 1 client.27598 shutdown 2024-04-16T17:14:08.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.552+0000 7ff223623740 1 -- 172.21.15.45:0/430800209 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559e4110f830 msgr2=0x559e411159b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:08.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.552+0000 7ff223623740 1 --2- 172.21.15.45:0/430800209 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559e4110f830 0x559e411159b0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7ff210000f80 tx=0x7ff2100323c0 comp rx=0 tx=0).stop 2024-04-16T17:14:08.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.552+0000 7ff223623740 1 -- 172.21.15.45:0/430800209 shutdown_connections 2024-04-16T17:14:08.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.552+0000 7ff223623740 1 --2- 172.21.15.45:0/430800209 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x559e411285f0 0x559e4112a9e0 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:08.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.552+0000 7ff223623740 1 --2- 172.21.15.45:0/430800209 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559e4110f830 0x559e411159b0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:08.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.552+0000 7ff223623740 1 --2- 172.21.15.45:0/430800209 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559e41118410 0x559e41115ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:08.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.552+0000 7ff223623740 1 --2- 172.21.15.45:0/430800209 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559e4110ef10 0x559e41115470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:08.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.552+0000 7ff223623740 1 -- 172.21.15.45:0/430800209 >> 172.21.15.45:0/430800209 conn(0x559e41109640 msgr2=0x559e4110b390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:14:08.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.552+0000 7ff1f7400640 0 client.27598 ms_handle_reset on v2:172.21.15.104:6838/3660070592 2024-04-16T17:14:08.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.552+0000 7ff223623740 1 -- 172.21.15.45:0/430800209 shutdown_connections 2024-04-16T17:14:08.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.552+0000 7ff223623740 1 -- 172.21.15.45:0/430800209 wait complete. 2024-04-16T17:14:08.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.552+0000 7ff223623740 20 client.27598 trim_cache size 0 max 16384 2024-04-16T17:14:08.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.567+0000 7ff223623740 1 Processor -- start 2024-04-16T17:14:08.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.567+0000 7ff223623740 1 -- start start 2024-04-16T17:14:08.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.567+0000 7ff223623740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559e4110ef10 0x559e41117ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:08.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.567+0000 7ff223623740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559e4110f830 0x559e41124060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:08.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.567+0000 7ff223623740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559e41118410 0x559e4112ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:08.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.567+0000 7ff223623740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7ff2181b3fd0 con 0x559e4110ef10 2024-04-16T17:14:08.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.567+0000 7ff223623740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7ff2181b1840 con 0x559e4110f830 2024-04-16T17:14:08.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.567+0000 7ff223623740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7ff2181af690 con 0x559e41118410 2024-04-16T17:14:08.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.568+0000 7ff21d400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559e4110ef10 0x559e41117ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:08.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.568+0000 7ff21de00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559e4110f830 0x559e41124060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:08.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.568+0000 7ff21d400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559e4110ef10 0x559e41117ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:46992/0 (socket says 172.21.15.45:46992) 2024-04-16T17:14:08.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.568+0000 7ff21d400640 1 -- 172.21.15.45:0/4177083429 learned_addr learned my addr 172.21.15.45:0/4177083429 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:14:08.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.568+0000 7ff21de00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559e4110f830 0x559e41124060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:49174/0 (socket says 172.21.15.45:49174) 2024-04-16T17:14:08.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.568+0000 7ff21d400640 1 --2- 172.21.15.45:0/4177083429 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559e41118410 0x559e4112ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:08.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.568+0000 7ff21de00640 1 -- 172.21.15.45:0/4177083429 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559e41118410 msgr2=0x559e4112ffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:14:08.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.568+0000 7ff21de00640 1 --2- 172.21.15.45:0/4177083429 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559e41118410 0x559e4112ffa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:08.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.568+0000 7ff21de00640 1 -- 172.21.15.45:0/4177083429 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559e4110ef10 msgr2=0x559e41117ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:14:08.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.568+0000 7ff21de00640 1 --2- 172.21.15.45:0/4177083429 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559e4110ef10 0x559e41117ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:08.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.568+0000 7ff21de00640 1 -- 172.21.15.45:0/4177083429 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559e40d863f0 con 0x559e4110f830 2024-04-16T17:14:08.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.568+0000 7ff21d400640 1 --2- 172.21.15.45:0/4177083429 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559e4110ef10 0x559e41117ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:14:08.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.568+0000 7ff21de00640 1 --2- 172.21.15.45:0/4177083429 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559e4110f830 0x559e41124060 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7ff21002c7a0 tx=0x7ff210009b20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d60bc5f7e5b88dbc server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:14:08.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.569+0000 7ff202600640 1 -- 172.21.15.45:0/4177083429 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff21004e040 con 0x559e4110f830 2024-04-16T17:14:08.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.569+0000 7ff202600640 1 -- 172.21.15.45:0/4177083429 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff210032430 con 0x559e4110f830 2024-04-16T17:14:08.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.569+0000 7ff202600640 1 -- 172.21.15.45:0/4177083429 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff210039050 con 0x559e4110f830 2024-04-16T17:14:08.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.569+0000 7ff223623740 1 -- 172.21.15.45:0/4177083429 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559e4110f830 msgr2=0x559e41124060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:08.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.569+0000 7ff223623740 1 --2- 172.21.15.45:0/4177083429 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559e4110f830 0x559e41124060 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7ff21002c7a0 tx=0x7ff210009b20 comp rx=0 tx=0).stop 2024-04-16T17:14:08.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.569+0000 7ff223623740 1 -- 172.21.15.45:0/4177083429 shutdown_connections 2024-04-16T17:14:08.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.569+0000 7ff223623740 1 --2- 172.21.15.45:0/4177083429 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559e41118410 0x559e4112ffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:08.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.569+0000 7ff223623740 1 --2- 172.21.15.45:0/4177083429 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559e4110f830 0x559e41124060 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:08.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.569+0000 7ff223623740 1 --2- 172.21.15.45:0/4177083429 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559e4110ef10 0x559e41117ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:08.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.569+0000 7ff223623740 1 -- 172.21.15.45:0/4177083429 >> 172.21.15.45:0/4177083429 conn(0x559e41109640 msgr2=0x559e41110790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:14:08.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.569+0000 7ff223623740 1 -- 172.21.15.45:0/4177083429 shutdown_connections 2024-04-16T17:14:08.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.570+0000 7ff223623740 1 -- 172.21.15.45:0/4177083429 wait complete. 2024-04-16T17:14:08.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.570+0000 7ff223623740 1 -- start start 2024-04-16T17:14:08.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.570+0000 7ff223623740 1 Processor -- start 2024-04-16T17:14:08.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.570+0000 7ff223623740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559e4110ef10 0x559e41120ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:08.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.571+0000 7ff223623740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559e4110f830 0x559e41111f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:08.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.571+0000 7ff223623740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559e41118410 0x559e41112620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:08.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.571+0000 7ff223623740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7ff2181afd00 con 0x559e4110ef10 2024-04-16T17:14:08.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.571+0000 7ff223623740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7ff2181b1170 con 0x559e41118410 2024-04-16T17:14:08.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.571+0000 7ff223623740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7ff2181b03d0 con 0x559e4110f830 2024-04-16T17:14:08.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.571+0000 7ff223623740 10 client.0 _ll_register_callbacks cb 0x7ff208005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:14:08.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.571+0000 7ff21d400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559e4110ef10 0x559e41120ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:08.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.571+0000 7ff21d400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559e4110ef10 0x559e41120ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:47002/0 (socket says 172.21.15.45:47002) 2024-04-16T17:14:08.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.571+0000 7ff21d400640 1 -- 172.21.15.45:0/295956176 learned_addr learned my addr 172.21.15.45:0/295956176 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:14:08.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.571+0000 7ff21de00640 1 --2- 172.21.15.45:0/295956176 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559e4110f830 0x559e41111f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:08.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.571+0000 7ff21d400640 1 --2- 172.21.15.45:0/295956176 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559e41118410 0x559e41112620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:08.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.571+0000 7ff21d400640 1 -- 172.21.15.45:0/295956176 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559e4110f830 msgr2=0x559e41111f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:14:08.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.571+0000 7ff21d400640 1 --2- 172.21.15.45:0/295956176 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559e4110f830 0x559e41111f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:08.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.571+0000 7ff21d400640 1 -- 172.21.15.45:0/295956176 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559e41118410 msgr2=0x559e41112620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:14:08.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.571+0000 7ff21d400640 1 --2- 172.21.15.45:0/295956176 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559e41118410 0x559e41112620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:08.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.571+0000 7ff21d400640 1 -- 172.21.15.45:0/295956176 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559e40e507d0 con 0x559e4110ef10 2024-04-16T17:14:08.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.571+0000 7ff21d400640 1 --2- 172.21.15.45:0/295956176 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559e4110ef10 0x559e41120ae0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7ff208005bc0 tx=0x7ff208004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73c5d28ab1edbc2e server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:14:08.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.571+0000 7ff1f7400640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-16T17:14:08.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.572+0000 7ff1f7400640 1 -- 172.21.15.45:0/295956176 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff208027030 con 0x559e4110ef10 2024-04-16T17:14:08.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.572+0000 7ff1f7400640 1 -- 172.21.15.45:0/295956176 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9584+0+0 (secure 0 0 0) 0x7ff20801c040 con 0x559e4110ef10 2024-04-16T17:14:08.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.572+0000 7ff223623740 1 -- 172.21.15.45:0/295956176 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff210032430 con 0x559e4110ef10 2024-04-16T17:14:08.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.572+0000 7ff1f7400640 1 -- 172.21.15.45:0/295956176 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7ff2080044d0 con 0x559e4110ef10 2024-04-16T17:14:08.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.572+0000 7ff223623740 10 client.17499 fetch_fsmap learned FSMap version 42 2024-04-16T17:14:08.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.572+0000 7ff223623740 1 -- 172.21.15.45:0/295956176 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x559e4118e770 con 0x559e4110ef10 2024-04-16T17:14:08.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.572+0000 7ff1f7400640 1 -- 172.21.15.45:0/295956176 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff210032430 con 0x559e4110ef10 2024-04-16T17:14:08.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.572+0000 7ff1f7400640 1 -- 172.21.15.45:0/295956176 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 42) ==== 8362+0+0 (secure 0 0 0) 0x7ff208021ce0 con 0x559e4110ef10 2024-04-16T17:14:08.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.573+0000 7ff223623740 10 client.17499 fetch_fsmap finished waiting for FSMap version 42 2024-04-16T17:14:08.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.573+0000 7ff223623740 10 client.17499 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:14:08.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.573+0000 7ff223623740 20 client.17499 populate_metadata read hostname 'smithi045' 2024-04-16T17:14:08.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.573+0000 7ff223623740 1 --2- 172.21.15.45:0/295956176 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x559e4118eec0 0x559e411912e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:08.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.573+0000 7ff223623740 4 client.17499 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T17:14:08.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.573+0000 7ff223623740 1 -- 172.21.15.45:0/295956176 --> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559e4107b8b0 con 0x559e4118eec0 2024-04-16T17:14:08.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.573+0000 7ff21de00640 1 --2- 172.21.15.45:0/295956176 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x559e4118eec0 0x559e411912e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:08.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.573+0000 7ff21de00640 1 --2- 172.21.15.45:0/295956176 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x559e4118eec0 0x559e411912e0 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fe982957f527d2b9 server_cookie=72e618d7f53b748a in_seq=0 out_seq=0 2024-04-16T17:14:08.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.573+0000 7ff1f7400640 10 client.17499 ms_handle_connect on v2:172.21.15.104:6838/3660070592 2024-04-16T17:14:08.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.574+0000 7ff1f7400640 1 -- 172.21.15.45:0/295956176 <== mds.0 v2:172.21.15.104:6838/3660070592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559e4107b8b0 con 0x559e4118eec0 2024-04-16T17:14:08.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.574+0000 7ff1f7400640 10 client.17499 handle_command_reply: tid=0 2024-04-16T17:14:08.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.574+0000 7ff223623740 1 client.17499 shutdown 2024-04-16T17:14:08.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.574+0000 7ff223623740 1 -- 172.21.15.45:0/295956176 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559e4110ef10 msgr2=0x559e41120ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:08.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.574+0000 7ff223623740 1 --2- 172.21.15.45:0/295956176 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559e4110ef10 0x559e41120ae0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7ff208005bc0 tx=0x7ff208004730 comp rx=0 tx=0).stop 2024-04-16T17:14:08.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.574+0000 7ff223623740 1 -- 172.21.15.45:0/295956176 shutdown_connections 2024-04-16T17:14:08.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.574+0000 7ff223623740 1 --2- 172.21.15.45:0/295956176 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x559e4118eec0 0x559e411912e0 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:08.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.574+0000 7ff223623740 1 --2- 172.21.15.45:0/295956176 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559e4110f830 0x559e41111f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:08.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.574+0000 7ff223623740 1 --2- 172.21.15.45:0/295956176 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559e41118410 0x559e41112620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:08.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.574+0000 7ff223623740 1 --2- 172.21.15.45:0/295956176 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559e4110ef10 0x559e41120ae0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:08.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.574+0000 7ff223623740 1 -- 172.21.15.45:0/295956176 >> 172.21.15.45:0/295956176 conn(0x559e41109640 msgr2=0x559e4110a9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:14:08.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.574+0000 7ff1f7400640 0 client.17499 ms_handle_reset on v2:172.21.15.104:6838/3660070592 2024-04-16T17:14:08.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.574+0000 7ff223623740 1 -- 172.21.15.45:0/295956176 shutdown_connections 2024-04-16T17:14:08.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.574+0000 7ff223623740 1 -- 172.21.15.45:0/295956176 wait complete. 2024-04-16T17:14:08.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.574+0000 7ff223623740 20 client.17499 trim_cache size 0 max 16384 2024-04-16T17:14:08.581 INFO:teuthology.orchestra.run.smithi045.stdout:[] 2024-04-16T17:14:08.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.575+0000 7ff21f200640 1 -- 172.21.15.45:0/783402670 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7ff2040a3d50 msgr2=0x7ff2040a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:08.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.575+0000 7ff21f200640 1 --2- 172.21.15.45:0/783402670 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7ff2040a3d50 0x7ff2040a6210 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7ff21409cad0 tx=0x7ff214009720 comp rx=0 tx=0).stop 2024-04-16T17:14:08.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.575+0000 7ff21f200640 1 -- 172.21.15.45:0/783402670 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff21810c4a0 msgr2=0x7ff218101750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:08.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.575+0000 7ff21f200640 1 --2- 172.21.15.45:0/783402670 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff21810c4a0 0x7ff218101750 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7ff21400e970 tx=0x7ff21400ed70 comp rx=0 tx=0).stop 2024-04-16T17:14:08.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.575+0000 7ff21f200640 1 -- 172.21.15.45:0/783402670 shutdown_connections 2024-04-16T17:14:08.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.575+0000 7ff21f200640 1 --2- 172.21.15.45:0/783402670 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7ff2040a3d50 0x7ff2040a6210 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:08.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.575+0000 7ff21f200640 1 --2- 172.21.15.45:0/783402670 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff21810c4a0 0x7ff218101750 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:08.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.575+0000 7ff21f200640 1 --2- 172.21.15.45:0/783402670 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff2181039a0 0x7ff218101210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:08.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.575+0000 7ff21f200640 1 --2- 172.21.15.45:0/783402670 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff218103080 0x7ff218100cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:08.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.575+0000 7ff21f200640 1 -- 172.21.15.45:0/783402670 >> 172.21.15.45:0/783402670 conn(0x7ff2180fc880 msgr2=0x7ff2180fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:14:08.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.575+0000 7ff21f200640 1 -- 172.21.15.45:0/783402670 shutdown_connections 2024-04-16T17:14:08.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:08.575+0000 7ff21f200640 1 -- 172.21.15.45:0/783402670 wait complete. 2024-04-16T17:14:08.590 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:14:08 smithi045 ceph-mon[31676]: pgmap v228: 129 pgs: 129 active+clean; 63 MiB data, 807 MiB used, 1.0 TiB / 1.0 TiB avail; 612 KiB/s rd, 634 KiB/s wr, 100 op/s 2024-04-16T17:14:08.590 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:14:08 smithi045 ceph-mon[31676]: from='client.? 172.21.15.45:0/129987739' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:14:08.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:14:08 smithi117 ceph-mon[36730]: pgmap v228: 129 pgs: 129 active+clean; 63 MiB data, 807 MiB used, 1.0 TiB / 1.0 TiB avail; 612 KiB/s rd, 634 KiB/s wr, 100 op/s 2024-04-16T17:14:08.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:14:08 smithi117 ceph-mon[36730]: from='client.? 172.21.15.45:0/129987739' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:14:08.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:14:08 smithi104 ceph-mon[36457]: pgmap v228: 129 pgs: 129 active+clean; 63 MiB data, 807 MiB used, 1.0 TiB / 1.0 TiB avail; 612 KiB/s rd, 634 KiB/s wr, 100 op/s 2024-04-16T17:14:08.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:14:08 smithi104 ceph-mon[36457]: from='client.? 172.21.15.45:0/129987739' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:14:09.594 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-16T17:14:09.594 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T17:14:09.594 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-16T17:14:09.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:09.800+0000 7fa0cce00640 1 -- 172.21.15.45:0/1724841452 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa0c8104320 msgr2=0x7fa0c8104700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:09.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:09.800+0000 7fa0cce00640 1 --2- 172.21.15.45:0/1724841452 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa0c8104320 0x7fa0c8104700 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fa0b8009900 tx=0x7fa0b802bc30 comp rx=0 tx=0).stop 2024-04-16T17:14:09.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:09.800+0000 7fa0cce00640 1 -- 172.21.15.45:0/1724841452 shutdown_connections 2024-04-16T17:14:09.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:09.800+0000 7fa0cce00640 1 --2- 172.21.15.45:0/1724841452 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa0c8104cd0 0x7fa0c8108d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:09.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:09.800+0000 7fa0cce00640 1 --2- 172.21.15.45:0/1724841452 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa0c81093f0 0x7fa0c8110fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:09.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:09.800+0000 7fa0cce00640 1 --2- 172.21.15.45:0/1724841452 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa0c8104320 0x7fa0c8104700 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:09.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:09.801+0000 7fa0cce00640 1 -- 172.21.15.45:0/1724841452 >> 172.21.15.45:0/1724841452 conn(0x7fa0c80ffb40 msgr2=0x7fa0c8101f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:14:09.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:09.801+0000 7fa0cce00640 1 -- 172.21.15.45:0/1724841452 shutdown_connections 2024-04-16T17:14:09.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:09.801+0000 7fa0cce00640 1 -- 172.21.15.45:0/1724841452 wait complete. 2024-04-16T17:14:09.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:09.802+0000 7fa0cce00640 1 Processor -- start 2024-04-16T17:14:09.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:09.802+0000 7fa0cce00640 1 -- start start 2024-04-16T17:14:09.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:09.802+0000 7fa0cce00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa0c8104320 0x7fa0c81a06c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:09.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:09.802+0000 7fa0cce00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa0c81093f0 0x7fa0c81a0c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:09.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:09.802+0000 7fa0cce00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa0c819a830 0x7fa0c819ace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:09.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:09.802+0000 7fa0cce00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fa0c81144f0 con 0x7fa0c819a830 2024-04-16T17:14:09.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:09.802+0000 7fa0cce00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fa0c8114370 con 0x7fa0c8104320 2024-04-16T17:14:09.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:09.802+0000 7fa0cce00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa0c8114670 con 0x7fa0c81093f0 2024-04-16T17:14:09.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:09.802+0000 7fa0c7e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa0c819a830 0x7fa0c819ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:09.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:09.802+0000 7fa0c6a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa0c81093f0 0x7fa0c81a0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:09.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:09.802+0000 7fa0c7400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa0c8104320 0x7fa0c81a06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:09.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:09.802+0000 7fa0c7e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa0c819a830 0x7fa0c819ace0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:47024/0 (socket says 172.21.15.45:47024) 2024-04-16T17:14:09.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:09.802+0000 7fa0c7e00640 1 -- 172.21.15.45:0/1063722590 learned_addr learned my addr 172.21.15.45:0/1063722590 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:14:09.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:09.803+0000 7fa0c7400640 1 -- 172.21.15.45:0/1063722590 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa0c81093f0 msgr2=0x7fa0c81a0c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:09.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:09.803+0000 7fa0c7400640 1 --2- 172.21.15.45:0/1063722590 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa0c81093f0 0x7fa0c81a0c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:09.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:09.803+0000 7fa0c7400640 1 -- 172.21.15.45:0/1063722590 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa0c819a830 msgr2=0x7fa0c819ace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:09.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:09.803+0000 7fa0c7400640 1 --2- 172.21.15.45:0/1063722590 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa0c819a830 0x7fa0c819ace0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:09.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:09.803+0000 7fa0c7400640 1 -- 172.21.15.45:0/1063722590 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0c819b3c0 con 0x7fa0c8104320 2024-04-16T17:14:09.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:09.803+0000 7fa0c7e00640 1 --2- 172.21.15.45:0/1063722590 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa0c819a830 0x7fa0c819ace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:14:09.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:09.803+0000 7fa0c7400640 1 --2- 172.21.15.45:0/1063722590 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa0c8104320 0x7fa0c81a06c0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fa0b802c170 tx=0x7fa0b8004360 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:14:09.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:09.803+0000 7fa0bfe00640 1 -- 172.21.15.45:0/1063722590 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0b8043020 con 0x7fa0c8104320 2024-04-16T17:14:09.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:09.803+0000 7fa0bfe00640 1 -- 172.21.15.45:0/1063722590 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa0b80054e0 con 0x7fa0c8104320 2024-04-16T17:14:09.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:09.803+0000 7fa0cce00640 1 -- 172.21.15.45:0/1063722590 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0c819b650 con 0x7fa0c8104320 2024-04-16T17:14:09.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:09.803+0000 7fa0bfe00640 1 -- 172.21.15.45:0/1063722590 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0b8003b90 con 0x7fa0c8104320 2024-04-16T17:14:09.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:09.804+0000 7fa0cce00640 1 -- 172.21.15.45:0/1063722590 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0c81a7800 con 0x7fa0c8104320 2024-04-16T17:14:09.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:09.806+0000 7fa0d11ae740 1 Processor -- start 2024-04-16T17:14:09.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:09.807+0000 7fa0d11ae740 1 -- start start 2024-04-16T17:14:09.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:09.807+0000 7fa0d11ae740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564fb4707e90 0x564fb4708270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:09.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:09.807+0000 7fa0d11ae740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564fb47087b0 0x564fb4710e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:09.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:09.807+0000 7fa0d11ae740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564fb4711390 0x564fb4713780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:09.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:09.807+0000 7fa0d11ae740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x564fb466ccb0 con 0x564fb47087b0 2024-04-16T17:14:09.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:09.807+0000 7fa0d11ae740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x564fb4613370 con 0x564fb4711390 2024-04-16T17:14:09.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:09.807+0000 7fa0d11ae740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x564fb4668c70 con 0x564fb4707e90 2024-04-16T17:14:09.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:09.807+0000 7fa0bfe00640 1 -- 172.21.15.45:0/1063722590 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fa0b8046020 con 0x7fa0c8104320 2024-04-16T17:14:09.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:09.807+0000 7fa0c7e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564fb4707e90 0x564fb4708270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:09.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:09.807+0000 7fa0c7e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564fb4707e90 0x564fb4708270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:45760/0 (socket says 172.21.15.45:45760) 2024-04-16T17:14:09.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:09.807+0000 7fa0c7e00640 1 -- 172.21.15.45:0/313359896 learned_addr learned my addr 172.21.15.45:0/313359896 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:14:09.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:09.807+0000 7fa0c6a00640 1 --2- 172.21.15.45:0/313359896 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564fb4711390 0x564fb4713780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:09.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:09.807+0000 7fa0c7400640 1 --2- 172.21.15.45:0/313359896 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564fb47087b0 0x564fb4710e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:09.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:09.808+0000 7fa0c7e00640 1 -- 172.21.15.45:0/313359896 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564fb4711390 msgr2=0x564fb4713780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:14:09.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:09.808+0000 7fa0c7e00640 1 --2- 172.21.15.45:0/313359896 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564fb4711390 0x564fb4713780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:09.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:09.808+0000 7fa0c7e00640 1 -- 172.21.15.45:0/313359896 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564fb47087b0 msgr2=0x564fb4710e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:14:09.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:09.808+0000 7fa0c7e00640 1 --2- 172.21.15.45:0/313359896 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564fb47087b0 0x564fb4710e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:09.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:09.808+0000 7fa0c7e00640 1 -- 172.21.15.45:0/313359896 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564fb4632910 con 0x564fb4707e90 2024-04-16T17:14:09.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:09.808+0000 7fa0c7400640 1 --2- 172.21.15.45:0/313359896 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564fb47087b0 0x564fb4710e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:14:09.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:09.808+0000 7fa0c7e00640 1 --2- 172.21.15.45:0/313359896 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564fb4707e90 0x564fb4708270 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fa0c0011850 tx=0x7fa0c0011d10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4230a60a8708a907 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:14:09.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:09.808+0000 7fa0bcc00640 1 -- 172.21.15.45:0/313359896 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0c0011330 con 0x564fb4707e90 2024-04-16T17:14:09.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:09.809+0000 7fa0bcc00640 1 -- 172.21.15.45:0/313359896 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa0c000fe80 con 0x564fb4707e90 2024-04-16T17:14:09.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:09.809+0000 7fa0bcc00640 1 -- 172.21.15.45:0/313359896 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0c00193e0 con 0x564fb4707e90 2024-04-16T17:14:09.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:09.809+0000 7fa0bfe00640 1 --2- 172.21.15.45:0/1063722590 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fa0ac0a3d20 0x7fa0ac0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:09.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:09.809+0000 7fa0bfe00640 1 -- 172.21.15.45:0/1063722590 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9496+0+0 (secure 0 0 0) 0x7fa0b80f2a40 con 0x7fa0c8104320 2024-04-16T17:14:09.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:09.809+0000 7fa0c7400640 1 --2- 172.21.15.45:0/1063722590 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fa0ac0a3d20 0x7fa0ac0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:09.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:09.809+0000 7fa0d11ae740 1 -- 172.21.15.45:0/313359896 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564fb4707e90 msgr2=0x564fb4708270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:09.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:09.809+0000 7fa0d11ae740 1 --2- 172.21.15.45:0/313359896 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564fb4707e90 0x564fb4708270 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fa0c0011850 tx=0x7fa0c0011d10 comp rx=0 tx=0).stop 2024-04-16T17:14:09.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:09.809+0000 7fa0d11ae740 1 -- 172.21.15.45:0/313359896 shutdown_connections 2024-04-16T17:14:09.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:09.809+0000 7fa0d11ae740 1 --2- 172.21.15.45:0/313359896 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564fb4711390 0x564fb4713780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:09.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:09.809+0000 7fa0d11ae740 1 --2- 172.21.15.45:0/313359896 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564fb47087b0 0x564fb4710e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:09.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:09.809+0000 7fa0d11ae740 1 --2- 172.21.15.45:0/313359896 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564fb4707e90 0x564fb4708270 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:09.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:09.809+0000 7fa0d11ae740 1 -- 172.21.15.45:0/313359896 >> 172.21.15.45:0/313359896 conn(0x564fb47025c0 msgr2=0x564fb47029a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:14:09.814 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:09.810+0000 7fa0d11ae740 1 -- 172.21.15.45:0/313359896 shutdown_connections 2024-04-16T17:14:09.814 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:09.810+0000 7fa0d11ae740 1 -- 172.21.15.45:0/313359896 wait complete. 2024-04-16T17:14:09.814 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:09.810+0000 7fa0c7400640 1 --2- 172.21.15.45:0/1063722590 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fa0ac0a3d20 0x7fa0ac0a61e0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x564fb4707960 tx=0x7fa0b802bc30 comp rx=0 tx=0).ready entity=mgr.25411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:14:09.814 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:09.810+0000 7fa0d11ae740 1 -- start start 2024-04-16T17:14:09.814 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:09.810+0000 7fa0d11ae740 1 Processor -- start 2024-04-16T17:14:09.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:09.811+0000 7fa0d11ae740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564fb4707e90 0x564fb470e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:09.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:09.811+0000 7fa0d11ae740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564fb47087b0 0x564fb470e870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:09.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:09.811+0000 7fa0d11ae740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564fb4711390 0x564fb470edb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:09.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:09.811+0000 7fa0d11ae740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x564fb4714180 con 0x564fb47087b0 2024-04-16T17:14:09.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:09.811+0000 7fa0d11ae740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x564fb4626730 con 0x564fb4707e90 2024-04-16T17:14:09.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:09.811+0000 7fa0d11ae740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x564fb4714300 con 0x564fb4711390 2024-04-16T17:14:09.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:09.811+0000 7fa0d11ae740 10 client.0 _ll_register_callbacks cb 0x564fb45f51e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:14:09.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:09.811+0000 7fa0c6a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564fb47087b0 0x564fb470e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:09.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:09.811+0000 7fa0c6a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564fb47087b0 0x564fb470e870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:47034/0 (socket says 172.21.15.45:47034) 2024-04-16T17:14:09.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:09.811+0000 7fa0c6a00640 1 -- 172.21.15.45:0/3487575481 learned_addr learned my addr 172.21.15.45:0/3487575481 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:14:09.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:09.811+0000 7fa0c7e00640 1 --2- 172.21.15.45:0/3487575481 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564fb4707e90 0x564fb470e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:09.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:09.811+0000 7fa0c7e00640 1 --2- 172.21.15.45:0/3487575481 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564fb4711390 0x564fb470edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:09.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:09.812+0000 7fa0c6a00640 1 -- 172.21.15.45:0/3487575481 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564fb4711390 msgr2=0x564fb470edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:14:09.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:09.812+0000 7fa0c6a00640 1 --2- 172.21.15.45:0/3487575481 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564fb4711390 0x564fb470edb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:09.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:09.812+0000 7fa0c6a00640 1 -- 172.21.15.45:0/3487575481 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564fb4707e90 msgr2=0x564fb470e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:14:09.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:09.812+0000 7fa0c6a00640 1 --2- 172.21.15.45:0/3487575481 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564fb4707e90 0x564fb470e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:09.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:09.812+0000 7fa0c6a00640 1 -- 172.21.15.45:0/3487575481 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564fb44477e0 con 0x564fb47087b0 2024-04-16T17:14:09.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:09.812+0000 7fa0c6a00640 1 --2- 172.21.15.45:0/3487575481 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564fb47087b0 0x564fb470e870 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fa0b0012700 tx=0x7fa0b0012bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dce3c3e3e717bef9 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:14:09.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:09.812+0000 7fa0b5600640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-16T17:14:09.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:09.812+0000 7fa0b5600640 1 -- 172.21.15.45:0/3487575481 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0b0004020 con 0x564fb47087b0 2024-04-16T17:14:09.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:09.812+0000 7fa0b5600640 1 -- 172.21.15.45:0/3487575481 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9496+0+0 (secure 0 0 0) 0x7fa0b00041c0 con 0x564fb47087b0 2024-04-16T17:14:09.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:09.812+0000 7fa0d11ae740 1 -- 172.21.15.45:0/3487575481 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x564fb467a6d0 con 0x564fb47087b0 2024-04-16T17:14:09.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:09.813+0000 7fa0b5600640 1 -- 172.21.15.45:0/3487575481 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7fa0b0025050 con 0x564fb47087b0 2024-04-16T17:14:09.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:09.813+0000 7fa0d11ae740 10 client.17511 fetch_fsmap learned FSMap version 42 2024-04-16T17:14:09.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:09.813+0000 7fa0d11ae740 1 -- 172.21.15.45:0/3487575481 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x564fb44a7f70 con 0x564fb47087b0 2024-04-16T17:14:09.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:09.813+0000 7fa0b5600640 1 -- 172.21.15.45:0/3487575481 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x564fb467a6d0 con 0x564fb47087b0 2024-04-16T17:14:09.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:09.813+0000 7fa0b5600640 1 -- 172.21.15.45:0/3487575481 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 42) ==== 8362+0+0 (secure 0 0 0) 0x7fa0b0003a20 con 0x564fb47087b0 2024-04-16T17:14:09.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:09.813+0000 7fa0d11ae740 10 client.17511 fetch_fsmap finished waiting for FSMap version 42 2024-04-16T17:14:09.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:09.813+0000 7fa0d11ae740 10 client.17511 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:14:09.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:09.813+0000 7fa0d11ae740 20 client.17511 populate_metadata read hostname 'smithi045' 2024-04-16T17:14:09.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:09.813+0000 7fa0d11ae740 1 --2- 172.21.15.45:0/3487575481 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x564fb47216d0 0x564fb4723ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:09.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:09.813+0000 7fa0d11ae740 4 client.17511 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:14:09.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:09.813+0000 7fa0d11ae740 1 -- 172.21.15.45:0/3487575481 --> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564fb4674540 con 0x564fb47216d0 2024-04-16T17:14:09.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:09.814+0000 7fa0c7e00640 1 --2- 172.21.15.45:0/3487575481 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x564fb47216d0 0x564fb4723ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:09.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:09.814+0000 7fa0c7e00640 1 --2- 172.21.15.45:0/3487575481 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x564fb47216d0 0x564fb4723ac0 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=548fd8378d3a43e1 server_cookie=8c0d1fa332feae5 in_seq=0 out_seq=0 2024-04-16T17:14:09.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:09.814+0000 7fa0b5600640 10 client.17511 ms_handle_connect on v2:172.21.15.104:6838/3660070592 2024-04-16T17:14:09.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:09.815+0000 7fa0b5600640 1 -- 172.21.15.45:0/3487575481 <== mds.0 v2:172.21.15.104:6838/3660070592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564fb4674540 con 0x564fb47216d0 2024-04-16T17:14:09.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:09.815+0000 7fa0b5600640 10 client.17511 handle_command_reply: tid=0 2024-04-16T17:14:09.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:09.815+0000 7fa0d11ae740 1 client.17511 shutdown 2024-04-16T17:14:09.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:09.815+0000 7fa0d11ae740 1 -- 172.21.15.45:0/3487575481 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564fb47087b0 msgr2=0x564fb470e870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:09.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:09.815+0000 7fa0d11ae740 1 --2- 172.21.15.45:0/3487575481 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564fb47087b0 0x564fb470e870 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fa0b0012700 tx=0x7fa0b0012bc0 comp rx=0 tx=0).stop 2024-04-16T17:14:09.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:09.815+0000 7fa0d11ae740 1 -- 172.21.15.45:0/3487575481 shutdown_connections 2024-04-16T17:14:09.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:09.815+0000 7fa0d11ae740 1 --2- 172.21.15.45:0/3487575481 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x564fb47216d0 0x564fb4723ac0 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:09.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:09.815+0000 7fa0d11ae740 1 --2- 172.21.15.45:0/3487575481 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564fb4711390 0x564fb470edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:09.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:09.815+0000 7fa0d11ae740 1 --2- 172.21.15.45:0/3487575481 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564fb47087b0 0x564fb470e870 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:09.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:09.815+0000 7fa0d11ae740 1 --2- 172.21.15.45:0/3487575481 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564fb4707e90 0x564fb470e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:09.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:09.815+0000 7fa0d11ae740 1 -- 172.21.15.45:0/3487575481 >> 172.21.15.45:0/3487575481 conn(0x564fb47025c0 msgr2=0x564fb4704310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:14:09.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:09.815+0000 7fa0b5600640 0 client.17511 ms_handle_reset on v2:172.21.15.104:6838/3660070592 2024-04-16T17:14:09.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:09.815+0000 7fa0d11ae740 1 -- 172.21.15.45:0/3487575481 shutdown_connections 2024-04-16T17:14:09.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:09.815+0000 7fa0d11ae740 1 -- 172.21.15.45:0/3487575481 wait complete. 2024-04-16T17:14:09.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:09.816+0000 7fa0d11ae740 20 client.17511 trim_cache size 0 max 16384 2024-04-16T17:14:09.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:09.835+0000 7fa0d11ae740 1 Processor -- start 2024-04-16T17:14:09.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:09.835+0000 7fa0d11ae740 1 -- start start 2024-04-16T17:14:09.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:09.835+0000 7fa0d11ae740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564fb4707e90 0x564fb471d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:09.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:09.835+0000 7fa0d11ae740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564fb47087b0 0x564fb47293d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:09.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:09.835+0000 7fa0d11ae740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564fb4711390 0x564fb4729910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:09.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:09.835+0000 7fa0d11ae740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fa0c81b3f70 con 0x564fb4707e90 2024-04-16T17:14:09.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:09.835+0000 7fa0d11ae740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fa0c81b17e0 con 0x564fb4711390 2024-04-16T17:14:09.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:09.835+0000 7fa0d11ae740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa0c81af650 con 0x564fb47087b0 2024-04-16T17:14:09.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:09.835+0000 7fa0c7e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564fb4707e90 0x564fb471d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:09.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:09.835+0000 7fa0c7e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564fb4707e90 0x564fb471d040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41290/0 (socket says 172.21.15.45:41290) 2024-04-16T17:14:09.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:09.835+0000 7fa0c7e00640 1 -- 172.21.15.45:0/1431132828 learned_addr learned my addr 172.21.15.45:0/1431132828 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:14:09.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:09.835+0000 7fa0c6a00640 1 --2- 172.21.15.45:0/1431132828 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564fb47087b0 0x564fb47293d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:09.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:09.836+0000 7fa0c7e00640 1 --2- 172.21.15.45:0/1431132828 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564fb4711390 0x564fb4729910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:09.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:09.836+0000 7fa0c7e00640 1 -- 172.21.15.45:0/1431132828 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564fb47087b0 msgr2=0x564fb47293d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:14:09.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:09.836+0000 7fa0c7e00640 1 --2- 172.21.15.45:0/1431132828 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564fb47087b0 0x564fb47293d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:09.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:09.836+0000 7fa0c7e00640 1 -- 172.21.15.45:0/1431132828 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564fb4711390 msgr2=0x564fb4729910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:14:09.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:09.836+0000 7fa0c7e00640 1 --2- 172.21.15.45:0/1431132828 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564fb4711390 0x564fb4729910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:09.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:09.836+0000 7fa0c7e00640 1 -- 172.21.15.45:0/1431132828 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564fb433f170 con 0x564fb4707e90 2024-04-16T17:14:09.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:09.836+0000 7fa0c6a00640 1 --2- 172.21.15.45:0/1431132828 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564fb47087b0 0x564fb47293d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:14:09.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:09.836+0000 7fa0c7e00640 1 --2- 172.21.15.45:0/1431132828 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564fb4707e90 0x564fb471d040 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fa0c0009270 tx=0x7fa0c0009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc5ee27a8154f552 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:14:09.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:09.836+0000 7fa0b7e00640 1 -- 172.21.15.45:0/1431132828 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0c0033070 con 0x564fb4707e90 2024-04-16T17:14:09.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:09.836+0000 7fa0b7e00640 1 -- 172.21.15.45:0/1431132828 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa0c00113a0 con 0x564fb4707e90 2024-04-16T17:14:09.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:09.836+0000 7fa0b7e00640 1 -- 172.21.15.45:0/1431132828 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0c0010700 con 0x564fb4707e90 2024-04-16T17:14:09.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:09.837+0000 7fa0d11ae740 1 -- 172.21.15.45:0/1431132828 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564fb4707e90 msgr2=0x564fb471d040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:09.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:09.837+0000 7fa0d11ae740 1 --2- 172.21.15.45:0/1431132828 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564fb4707e90 0x564fb471d040 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fa0c0009270 tx=0x7fa0c0009500 comp rx=0 tx=0).stop 2024-04-16T17:14:09.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:09.837+0000 7fa0d11ae740 1 -- 172.21.15.45:0/1431132828 shutdown_connections 2024-04-16T17:14:09.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:09.837+0000 7fa0d11ae740 1 --2- 172.21.15.45:0/1431132828 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564fb47087b0 0x564fb47293d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:09.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:09.837+0000 7fa0d11ae740 1 --2- 172.21.15.45:0/1431132828 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564fb4711390 0x564fb4729910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:09.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:09.837+0000 7fa0d11ae740 1 --2- 172.21.15.45:0/1431132828 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564fb4707e90 0x564fb471d040 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:09.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:09.837+0000 7fa0d11ae740 1 -- 172.21.15.45:0/1431132828 >> 172.21.15.45:0/1431132828 conn(0x564fb47025c0 msgr2=0x564fb4704150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:14:09.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:09.837+0000 7fa0d11ae740 1 -- 172.21.15.45:0/1431132828 shutdown_connections 2024-04-16T17:14:09.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:09.837+0000 7fa0d11ae740 1 -- 172.21.15.45:0/1431132828 wait complete. 2024-04-16T17:14:09.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:09.838+0000 7fa0d11ae740 1 -- start start 2024-04-16T17:14:09.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:09.838+0000 7fa0d11ae740 1 Processor -- start 2024-04-16T17:14:09.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:09.838+0000 7fa0d11ae740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564fb4707e90 0x564fb4716520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:09.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:09.838+0000 7fa0d11ae740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564fb47087b0 0x564fb4716a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:09.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:09.838+0000 7fa0d11ae740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564fb4711390 0x564fb4716fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:09.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:09.838+0000 7fa0d11ae740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fa0c81afca0 con 0x564fb4711390 2024-04-16T17:14:09.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:09.838+0000 7fa0d11ae740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fa0c81b1110 con 0x564fb4707e90 2024-04-16T17:14:09.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:09.838+0000 7fa0d11ae740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa0c81b0370 con 0x564fb47087b0 2024-04-16T17:14:09.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:09.838+0000 7fa0d11ae740 10 client.0 _ll_register_callbacks cb 0x7fa0b0009d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:14:09.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:09.838+0000 7fa0c7e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564fb4711390 0x564fb4716fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:09.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:09.838+0000 7fa0c7e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564fb4711390 0x564fb4716fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41292/0 (socket says 172.21.15.45:41292) 2024-04-16T17:14:09.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:09.838+0000 7fa0c7e00640 1 -- 172.21.15.45:0/2777062565 learned_addr learned my addr 172.21.15.45:0/2777062565 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:14:09.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:09.839+0000 7fa0c7e00640 1 --2- 172.21.15.45:0/2777062565 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564fb4707e90 0x564fb4716520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:09.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:09.839+0000 7fa0c6a00640 1 --2- 172.21.15.45:0/2777062565 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564fb47087b0 0x564fb4716a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:09.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:09.839+0000 7fa0c7e00640 1 -- 172.21.15.45:0/2777062565 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564fb47087b0 msgr2=0x564fb4716a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:14:09.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:09.839+0000 7fa0c7e00640 1 --2- 172.21.15.45:0/2777062565 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564fb47087b0 0x564fb4716a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:09.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:09.839+0000 7fa0c7e00640 1 -- 172.21.15.45:0/2777062565 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564fb4707e90 msgr2=0x564fb4716520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:14:09.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:09.839+0000 7fa0c7e00640 1 --2- 172.21.15.45:0/2777062565 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564fb4707e90 0x564fb4716520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:09.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:09.839+0000 7fa0c7e00640 1 -- 172.21.15.45:0/2777062565 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564fb4787e50 con 0x564fb4711390 2024-04-16T17:14:09.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:09.839+0000 7fa0c7e00640 1 --2- 172.21.15.45:0/2777062565 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564fb4711390 0x564fb4716fa0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fa0c0010ca0 tx=0x7fa0c0009fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7573a2759f90d72f server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:14:09.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:09.839+0000 7fa0b5600640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-16T17:14:09.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:09.839+0000 7fa0b5600640 1 -- 172.21.15.45:0/2777062565 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0c0033040 con 0x564fb4711390 2024-04-16T17:14:09.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:09.839+0000 7fa0d11ae740 1 -- 172.21.15.45:0/2777062565 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x564fb45f0aa0 con 0x564fb4711390 2024-04-16T17:14:09.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:09.840+0000 7fa0b5600640 1 -- 172.21.15.45:0/2777062565 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9496+0+0 (secure 0 0 0) 0x7fa0c000f9a0 con 0x564fb4711390 2024-04-16T17:14:09.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:09.840+0000 7fa0b5600640 1 -- 172.21.15.45:0/2777062565 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7fa0c000fba0 con 0x564fb4711390 2024-04-16T17:14:09.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:09.840+0000 7fa0d11ae740 10 client.17517 fetch_fsmap learned FSMap version 42 2024-04-16T17:14:09.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:09.840+0000 7fa0d11ae740 1 -- 172.21.15.45:0/2777062565 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x564fb4789130 con 0x564fb4711390 2024-04-16T17:14:09.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:09.840+0000 7fa0b5600640 1 -- 172.21.15.45:0/2777062565 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x564fb45f0aa0 con 0x564fb4711390 2024-04-16T17:14:09.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:09.840+0000 7fa0b5600640 1 -- 172.21.15.45:0/2777062565 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 42) ==== 8362+0+0 (secure 0 0 0) 0x7fa0c0039070 con 0x564fb4711390 2024-04-16T17:14:09.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:09.840+0000 7fa0d11ae740 10 client.17517 fetch_fsmap finished waiting for FSMap version 42 2024-04-16T17:14:09.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:09.840+0000 7fa0d11ae740 10 client.17517 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:14:09.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:09.840+0000 7fa0d11ae740 20 client.17517 populate_metadata read hostname 'smithi045' 2024-04-16T17:14:09.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:09.840+0000 7fa0d11ae740 1 --2- 172.21.15.45:0/2777062565 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x564fb4789500 0x564fb478b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:09.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:09.840+0000 7fa0d11ae740 4 client.17517 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-16T17:14:09.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:09.840+0000 7fa0d11ae740 1 -- 172.21.15.45:0/2777062565 --> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x564fb463d5d0 con 0x564fb4789500 2024-04-16T17:14:09.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:09.841+0000 7fa0c6a00640 1 --2- 172.21.15.45:0/2777062565 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x564fb4789500 0x564fb478b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:09.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:09.841+0000 7fa0c6a00640 1 --2- 172.21.15.45:0/2777062565 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x564fb4789500 0x564fb478b8f0 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a74ad9cd2919950c server_cookie=8934f84c44610240 in_seq=0 out_seq=0 2024-04-16T17:14:09.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:09.841+0000 7fa0b5600640 10 client.17517 ms_handle_connect on v2:172.21.15.104:6838/3660070592 2024-04-16T17:14:09.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:09.842+0000 7fa0b5600640 1 -- 172.21.15.45:0/2777062565 <== mds.0 v2:172.21.15.104:6838/3660070592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x564fb463d5d0 con 0x564fb4789500 2024-04-16T17:14:09.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:09.842+0000 7fa0b5600640 10 client.17517 handle_command_reply: tid=0 2024-04-16T17:14:09.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:09.842+0000 7fa0d11ae740 1 client.17517 shutdown 2024-04-16T17:14:09.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:09.843+0000 7fa0d11ae740 1 -- 172.21.15.45:0/2777062565 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564fb4711390 msgr2=0x564fb4716fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:09.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:09.843+0000 7fa0d11ae740 1 --2- 172.21.15.45:0/2777062565 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564fb4711390 0x564fb4716fa0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fa0c0010ca0 tx=0x7fa0c0009fa0 comp rx=0 tx=0).stop 2024-04-16T17:14:09.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:09.843+0000 7fa0d11ae740 1 -- 172.21.15.45:0/2777062565 shutdown_connections 2024-04-16T17:14:09.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:09.843+0000 7fa0d11ae740 1 --2- 172.21.15.45:0/2777062565 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x564fb4789500 0x564fb478b8f0 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:09.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:09.843+0000 7fa0d11ae740 1 --2- 172.21.15.45:0/2777062565 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564fb47087b0 0x564fb4716a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:09.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:09.843+0000 7fa0b5600640 0 client.17517 ms_handle_reset on v2:172.21.15.104:6838/3660070592 2024-04-16T17:14:09.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:09.843+0000 7fa0d11ae740 1 --2- 172.21.15.45:0/2777062565 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564fb4711390 0x564fb4716fa0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:09.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:09.843+0000 7fa0d11ae740 1 --2- 172.21.15.45:0/2777062565 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564fb4707e90 0x564fb4716520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:09.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:09.843+0000 7fa0d11ae740 1 -- 172.21.15.45:0/2777062565 >> 172.21.15.45:0/2777062565 conn(0x564fb47025c0 msgr2=0x564fb4700cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:14:09.852 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:09.843+0000 7fa0d11ae740 1 -- 172.21.15.45:0/2777062565 shutdown_connections 2024-04-16T17:14:09.852 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:09.843+0000 7fa0d11ae740 1 -- 172.21.15.45:0/2777062565 wait complete. 2024-04-16T17:14:09.852 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:09.843+0000 7fa0d11ae740 20 client.17517 trim_cache size 0 max 16384 2024-04-16T17:14:09.852 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-16T17:14:09.852 INFO:teuthology.orchestra.run.smithi045.stdout: "return_code": 0, 2024-04-16T17:14:09.852 INFO:teuthology.orchestra.run.smithi045.stdout: "scrub_tag": "4aa26d17-9a5c-4eba-9efb-8ecb9e61e393", 2024-04-16T17:14:09.853 INFO:teuthology.orchestra.run.smithi045.stdout: "mode": "asynchronous" 2024-04-16T17:14:09.853 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-16T17:14:09.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:09.844+0000 7fa0cce00640 1 -- 172.21.15.45:0/1063722590 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fa0ac0a3d20 msgr2=0x7fa0ac0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:09.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:09.844+0000 7fa0cce00640 1 --2- 172.21.15.45:0/1063722590 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fa0ac0a3d20 0x7fa0ac0a61e0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x564fb4707960 tx=0x7fa0b802bc30 comp rx=0 tx=0).stop 2024-04-16T17:14:09.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:09.844+0000 7fa0cce00640 1 -- 172.21.15.45:0/1063722590 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa0c8104320 msgr2=0x7fa0c81a06c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:09.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:09.844+0000 7fa0cce00640 1 --2- 172.21.15.45:0/1063722590 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa0c8104320 0x7fa0c81a06c0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fa0b802c170 tx=0x7fa0b8004360 comp rx=0 tx=0).stop 2024-04-16T17:14:09.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:09.844+0000 7fa0cce00640 1 -- 172.21.15.45:0/1063722590 shutdown_connections 2024-04-16T17:14:09.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:09.844+0000 7fa0cce00640 1 --2- 172.21.15.45:0/1063722590 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fa0ac0a3d20 0x7fa0ac0a61e0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:09.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:09.844+0000 7fa0cce00640 1 --2- 172.21.15.45:0/1063722590 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa0c81093f0 0x7fa0c81a0c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:09.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:09.844+0000 7fa0cce00640 1 --2- 172.21.15.45:0/1063722590 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa0c819a830 0x7fa0c819ace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:09.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:09.844+0000 7fa0cce00640 1 --2- 172.21.15.45:0/1063722590 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa0c8104320 0x7fa0c81a06c0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:09.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:09.844+0000 7fa0cce00640 1 -- 172.21.15.45:0/1063722590 >> 172.21.15.45:0/1063722590 conn(0x7fa0c80ffb40 msgr2=0x7fa0c8101ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:14:09.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:09.844+0000 7fa0cce00640 1 -- 172.21.15.45:0/1063722590 shutdown_connections 2024-04-16T17:14:09.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:09.844+0000 7fa0cce00640 1 -- 172.21.15.45:0/1063722590 wait complete. 2024-04-16T17:14:09.863 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-16T17:14:09.863 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-16T17:14:10.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:10.056+0000 7fef64e00640 1 -- 172.21.15.45:0/649853690 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fef601093f0 msgr2=0x7fef60110fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:10.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:10.056+0000 7fef64e00640 1 --2- 172.21.15.45:0/649853690 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fef601093f0 0x7fef60110fc0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fef58009ca0 tx=0x7fef5802c100 comp rx=0 tx=0).stop 2024-04-16T17:14:10.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:10.057+0000 7fef64e00640 1 -- 172.21.15.45:0/649853690 shutdown_connections 2024-04-16T17:14:10.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:10.057+0000 7fef64e00640 1 --2- 172.21.15.45:0/649853690 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fef60104cd0 0x7fef60108d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:10.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:10.057+0000 7fef64e00640 1 --2- 172.21.15.45:0/649853690 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fef601093f0 0x7fef60110fc0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:10.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:10.057+0000 7fef64e00640 1 --2- 172.21.15.45:0/649853690 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef60104320 0x7fef60104700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:10.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:10.057+0000 7fef64e00640 1 -- 172.21.15.45:0/649853690 >> 172.21.15.45:0/649853690 conn(0x7fef600ffb40 msgr2=0x7fef60101f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:14:10.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:10.057+0000 7fef64e00640 1 -- 172.21.15.45:0/649853690 shutdown_connections 2024-04-16T17:14:10.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:10.057+0000 7fef64e00640 1 -- 172.21.15.45:0/649853690 wait complete. 2024-04-16T17:14:10.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:10.058+0000 7fef64e00640 1 Processor -- start 2024-04-16T17:14:10.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:10.058+0000 7fef64e00640 1 -- start start 2024-04-16T17:14:10.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:10.059+0000 7fef64e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fef60104320 0x7fef601a0620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:10.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:10.059+0000 7fef64e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fef601093f0 0x7fef601a0b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:10.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:10.059+0000 7fef64e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef6019a790 0x7fef6019ac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:10.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:10.059+0000 7fef64e00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fef60114490 con 0x7fef6019a790 2024-04-16T17:14:10.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:10.059+0000 7fef64e00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fef60114310 con 0x7fef601093f0 2024-04-16T17:14:10.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:10.059+0000 7fef64e00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fef60114610 con 0x7fef60104320 2024-04-16T17:14:10.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:10.059+0000 7fef5fe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef6019a790 0x7fef6019ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:10.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:10.059+0000 7fef5ea00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fef601093f0 0x7fef601a0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:10.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:10.059+0000 7fef5f400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fef60104320 0x7fef601a0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:10.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:10.059+0000 7fef5fe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef6019a790 0x7fef6019ac40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41310/0 (socket says 172.21.15.45:41310) 2024-04-16T17:14:10.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:10.059+0000 7fef5ea00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fef601093f0 0x7fef601a0b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:33356/0 (socket says 172.21.15.45:33356) 2024-04-16T17:14:10.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:10.059+0000 7fef5fe00640 1 -- 172.21.15.45:0/2649150293 learned_addr learned my addr 172.21.15.45:0/2649150293 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:14:10.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:10.060+0000 7fef5fe00640 1 -- 172.21.15.45:0/2649150293 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fef60104320 msgr2=0x7fef601a0620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:10.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:10.060+0000 7fef5fe00640 1 --2- 172.21.15.45:0/2649150293 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fef60104320 0x7fef601a0620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:10.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:10.060+0000 7fef5fe00640 1 -- 172.21.15.45:0/2649150293 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fef601093f0 msgr2=0x7fef601a0b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:10.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:10.060+0000 7fef5fe00640 1 --2- 172.21.15.45:0/2649150293 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fef601093f0 0x7fef601a0b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:10.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:10.060+0000 7fef5fe00640 1 -- 172.21.15.45:0/2649150293 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef6019b320 con 0x7fef6019a790 2024-04-16T17:14:10.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:10.060+0000 7fef5ea00640 1 --2- 172.21.15.45:0/2649150293 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fef601093f0 0x7fef601a0b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:14:10.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:10.060+0000 7fef5fe00640 1 --2- 172.21.15.45:0/2649150293 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef6019a790 0x7fef6019ac40 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fef58037040 tx=0x7fef58032f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:14:10.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:10.060+0000 7fef57e00640 1 -- 172.21.15.45:0/2649150293 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef58039040 con 0x7fef6019a790 2024-04-16T17:14:10.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:10.060+0000 7fef57e00640 1 -- 172.21.15.45:0/2649150293 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fef58044070 con 0x7fef6019a790 2024-04-16T17:14:10.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:10.060+0000 7fef57e00640 1 -- 172.21.15.45:0/2649150293 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef580042a0 con 0x7fef6019a790 2024-04-16T17:14:10.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:10.060+0000 7fef64e00640 1 -- 172.21.15.45:0/2649150293 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef6019b5b0 con 0x7fef6019a790 2024-04-16T17:14:10.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:10.061+0000 7fef64e00640 1 -- 172.21.15.45:0/2649150293 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef601a7790 con 0x7fef6019a790 2024-04-16T17:14:10.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:10.064+0000 7fef691f9740 1 Processor -- start 2024-04-16T17:14:10.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:10.064+0000 7fef57e00640 1 -- 172.21.15.45:0/2649150293 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fef58003970 con 0x7fef6019a790 2024-04-16T17:14:10.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:10.064+0000 7fef691f9740 1 -- start start 2024-04-16T17:14:10.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:10.065+0000 7fef691f9740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560c8a0a7390 0x560c8a0a7770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:10.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:10.065+0000 7fef691f9740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560c8a0a7cb0 0x560c8a0b0350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:10.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:10.065+0000 7fef691f9740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560c8a0b0890 0x560c8a0b2c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:10.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:10.065+0000 7fef691f9740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x560c8a00d120 con 0x560c8a0a7390 2024-04-16T17:14:10.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:10.065+0000 7fef691f9740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x560c89fb3340 con 0x560c8a0a7cb0 2024-04-16T17:14:10.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:10.065+0000 7fef691f9740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x560c8a0090e0 con 0x560c8a0b0890 2024-04-16T17:14:10.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:10.065+0000 7fef5ea00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560c8a0a7390 0x560c8a0a7770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:10.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:10.065+0000 7fef5f400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560c8a0a7cb0 0x560c8a0b0350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:10.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:10.065+0000 7fef5ea00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560c8a0a7390 0x560c8a0a7770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41320/0 (socket says 172.21.15.45:41320) 2024-04-16T17:14:10.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:10.065+0000 7fef5ea00640 1 -- 172.21.15.45:0/2830801997 learned_addr learned my addr 172.21.15.45:0/2830801997 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:14:10.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:10.066+0000 7fef5ea00640 1 --2- 172.21.15.45:0/2830801997 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560c8a0b0890 0x560c8a0b2c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:10.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:10.066+0000 7fef5ea00640 1 -- 172.21.15.45:0/2830801997 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560c8a0b0890 msgr2=0x560c8a0b2c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:14:10.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:10.066+0000 7fef5ea00640 1 --2- 172.21.15.45:0/2830801997 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560c8a0b0890 0x560c8a0b2c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:10.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:10.066+0000 7fef5ea00640 1 -- 172.21.15.45:0/2830801997 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560c8a0a7cb0 msgr2=0x560c8a0b0350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:14:10.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:10.066+0000 7fef5ea00640 1 --2- 172.21.15.45:0/2830801997 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560c8a0a7cb0 0x560c8a0b0350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:10.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:10.066+0000 7fef5ea00640 1 -- 172.21.15.45:0/2830801997 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560c89de97d0 con 0x560c8a0a7390 2024-04-16T17:14:10.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:10.066+0000 7fef5f400640 1 --2- 172.21.15.45:0/2830801997 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560c8a0a7cb0 0x560c8a0b0350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:14:10.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:10.066+0000 7fef57e00640 1 --2- 172.21.15.45:0/2649150293 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fef440a3da0 0x7fef440a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:10.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:10.066+0000 7fef5ea00640 1 --2- 172.21.15.45:0/2830801997 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560c8a0a7390 0x560c8a0a7770 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fef480097e0 tx=0x7fef48009770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9844f089cb068da0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:14:10.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:10.066+0000 7fef57e00640 1 -- 172.21.15.45:0/2649150293 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9496+0+0 (secure 0 0 0) 0x7fef580f2e90 con 0x7fef6019a790 2024-04-16T17:14:10.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:10.066+0000 7fef5f400640 1 --2- 172.21.15.45:0/2649150293 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fef440a3da0 0x7fef440a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:10.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:10.066+0000 7fef54c00640 1 -- 172.21.15.45:0/2830801997 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef4800f940 con 0x560c8a0a7390 2024-04-16T17:14:10.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:10.066+0000 7fef54c00640 1 -- 172.21.15.45:0/2830801997 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fef48005020 con 0x560c8a0a7390 2024-04-16T17:14:10.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:10.066+0000 7fef54c00640 1 -- 172.21.15.45:0/2830801997 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef48004ac0 con 0x560c8a0a7390 2024-04-16T17:14:10.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:10.067+0000 7fef691f9740 1 -- 172.21.15.45:0/2830801997 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560c8a0a7390 msgr2=0x560c8a0a7770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:10.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:10.067+0000 7fef5f400640 1 --2- 172.21.15.45:0/2649150293 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fef440a3da0 0x7fef440a6260 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x560c8a0a6e60 tx=0x7fef50005f90 comp rx=0 tx=0).ready entity=mgr.25411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:14:10.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:10.067+0000 7fef691f9740 1 --2- 172.21.15.45:0/2830801997 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560c8a0a7390 0x560c8a0a7770 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fef480097e0 tx=0x7fef48009770 comp rx=0 tx=0).stop 2024-04-16T17:14:10.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:10.067+0000 7fef691f9740 1 -- 172.21.15.45:0/2830801997 shutdown_connections 2024-04-16T17:14:10.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:10.067+0000 7fef691f9740 1 --2- 172.21.15.45:0/2830801997 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560c8a0b0890 0x560c8a0b2c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:10.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:10.067+0000 7fef691f9740 1 --2- 172.21.15.45:0/2830801997 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560c8a0a7cb0 0x560c8a0b0350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:10.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:10.067+0000 7fef691f9740 1 --2- 172.21.15.45:0/2830801997 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560c8a0a7390 0x560c8a0a7770 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:10.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:10.067+0000 7fef691f9740 1 -- 172.21.15.45:0/2830801997 >> 172.21.15.45:0/2830801997 conn(0x560c8a0a1ac0 msgr2=0x560c8a0a1ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:14:10.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:10.067+0000 7fef691f9740 1 -- 172.21.15.45:0/2830801997 shutdown_connections 2024-04-16T17:14:10.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:10.068+0000 7fef691f9740 1 -- 172.21.15.45:0/2830801997 wait complete. 2024-04-16T17:14:10.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:10.068+0000 7fef691f9740 1 -- start start 2024-04-16T17:14:10.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:10.068+0000 7fef691f9740 1 Processor -- start 2024-04-16T17:14:10.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:10.069+0000 7fef691f9740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560c8a0a7390 0x560c8a0ad8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:10.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:10.069+0000 7fef691f9740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560c8a0a7cb0 0x560c8a0ade00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:10.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:10.069+0000 7fef691f9740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560c8a0b0890 0x560c8a0ae340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:10.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:10.069+0000 7fef691f9740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x560c8a0b3680 con 0x560c8a0a7390 2024-04-16T17:14:10.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:10.069+0000 7fef691f9740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x560c89fc6710 con 0x560c8a0a7cb0 2024-04-16T17:14:10.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:10.069+0000 7fef691f9740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x560c8a0b3800 con 0x560c8a0b0890 2024-04-16T17:14:10.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:10.069+0000 7fef691f9740 10 client.0 _ll_register_callbacks cb 0x560c89fc1af0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:14:10.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:10.069+0000 7fef5f400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560c8a0a7390 0x560c8a0ad8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:10.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:10.069+0000 7fef5fe00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560c8a0b0890 0x560c8a0ae340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:10.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:10.069+0000 7fef5f400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560c8a0a7390 0x560c8a0ad8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41326/0 (socket says 172.21.15.45:41326) 2024-04-16T17:14:10.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:10.069+0000 7fef5f400640 1 -- 172.21.15.45:0/3081754949 learned_addr learned my addr 172.21.15.45:0/3081754949 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:14:10.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:10.069+0000 7fef5ea00640 1 --2- 172.21.15.45:0/3081754949 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560c8a0a7cb0 0x560c8a0ade00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:10.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:10.069+0000 7fef5fe00640 1 -- 172.21.15.45:0/3081754949 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560c8a0a7cb0 msgr2=0x560c8a0ade00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:14:10.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:10.069+0000 7fef5fe00640 1 --2- 172.21.15.45:0/3081754949 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560c8a0a7cb0 0x560c8a0ade00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:10.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:10.069+0000 7fef5fe00640 1 -- 172.21.15.45:0/3081754949 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560c8a0a7390 msgr2=0x560c8a0ad8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:14:10.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:10.069+0000 7fef5fe00640 1 --2- 172.21.15.45:0/3081754949 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560c8a0a7390 0x560c8a0ad8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:10.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:10.069+0000 7fef5fe00640 1 -- 172.21.15.45:0/3081754949 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560c89d1f3f0 con 0x560c8a0b0890 2024-04-16T17:14:10.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:10.069+0000 7fef5f400640 1 --2- 172.21.15.45:0/3081754949 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560c8a0a7390 0x560c8a0ad8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:14:10.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:10.070+0000 7fef5fe00640 1 --2- 172.21.15.45:0/3081754949 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560c8a0b0890 0x560c8a0ae340 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fef580f02c0 tx=0x7fef580be6e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=da176c9107580bc server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:14:10.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:10.070+0000 7fef4d600640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-04-16T17:14:10.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:10.070+0000 7fef4d600640 1 -- 172.21.15.45:0/3081754949 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef580bf2a0 con 0x560c8a0b0890 2024-04-16T17:14:10.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:10.070+0000 7fef691f9740 1 -- 172.21.15.45:0/3081754949 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fef48005020 con 0x560c8a0b0890 2024-04-16T17:14:10.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:10.070+0000 7fef4d600640 1 -- 172.21.15.45:0/3081754949 <== mon.1 v2:172.21.15.117:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9496+0+0 (secure 0 0 0) 0x7fef580bee00 con 0x560c8a0b0890 2024-04-16T17:14:10.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:10.070+0000 7fef4d600640 1 -- 172.21.15.45:0/3081754949 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7fef580beba0 con 0x560c8a0b0890 2024-04-16T17:14:10.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:10.070+0000 7fef691f9740 10 client.27631 fetch_fsmap learned FSMap version 42 2024-04-16T17:14:10.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:10.070+0000 7fef691f9740 1 -- 172.21.15.45:0/3081754949 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x560c89cdf0f0 con 0x560c8a0b0890 2024-04-16T17:14:10.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:10.071+0000 7fef4d600640 1 -- 172.21.15.45:0/3081754949 <== mon.1 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fef48005020 con 0x560c8a0b0890 2024-04-16T17:14:10.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:10.071+0000 7fef4d600640 1 -- 172.21.15.45:0/3081754949 <== mon.1 v2:172.21.15.117:3300/0 5 ==== fsmap(e 42) ==== 8362+0+0 (secure 0 0 0) 0x7fef5803c030 con 0x560c8a0b0890 2024-04-16T17:14:10.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:10.071+0000 7fef691f9740 10 client.27631 fetch_fsmap finished waiting for FSMap version 42 2024-04-16T17:14:10.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:10.071+0000 7fef691f9740 10 client.27631 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:14:10.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:10.071+0000 7fef691f9740 20 client.27631 populate_metadata read hostname 'smithi045' 2024-04-16T17:14:10.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:10.071+0000 7fef691f9740 1 --2- 172.21.15.45:0/3081754949 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x560c8a0c0c50 0x560c8a0c30f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:10.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:10.071+0000 7fef691f9740 4 client.27631 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:14:10.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:10.071+0000 7fef691f9740 1 -- 172.21.15.45:0/3081754949 --> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560c8a01ab40 con 0x560c8a0c0c50 2024-04-16T17:14:10.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:10.071+0000 7fef5f400640 1 --2- 172.21.15.45:0/3081754949 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x560c8a0c0c50 0x560c8a0c30f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:10.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:10.072+0000 7fef5f400640 1 --2- 172.21.15.45:0/3081754949 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x560c8a0c0c50 0x560c8a0c30f0 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5e01b5e58c94ad83 server_cookie=1d3d02a1abfc0879 in_seq=0 out_seq=0 2024-04-16T17:14:10.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:10.072+0000 7fef4d600640 10 client.27631 ms_handle_connect on v2:172.21.15.104:6838/3660070592 2024-04-16T17:14:10.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:10.073+0000 7fef4d600640 1 -- 172.21.15.45:0/3081754949 <== mds.0 v2:172.21.15.104:6838/3660070592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560c8a01ab40 con 0x560c8a0c0c50 2024-04-16T17:14:10.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:10.073+0000 7fef4d600640 10 client.27631 handle_command_reply: tid=0 2024-04-16T17:14:10.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:10.073+0000 7fef691f9740 1 client.27631 shutdown 2024-04-16T17:14:10.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:10.073+0000 7fef691f9740 1 -- 172.21.15.45:0/3081754949 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560c8a0b0890 msgr2=0x560c8a0ae340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:10.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:10.073+0000 7fef691f9740 1 --2- 172.21.15.45:0/3081754949 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560c8a0b0890 0x560c8a0ae340 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fef580f02c0 tx=0x7fef580be6e0 comp rx=0 tx=0).stop 2024-04-16T17:14:10.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:10.073+0000 7fef691f9740 1 -- 172.21.15.45:0/3081754949 shutdown_connections 2024-04-16T17:14:10.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:10.073+0000 7fef691f9740 1 --2- 172.21.15.45:0/3081754949 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x560c8a0c0c50 0x560c8a0c30f0 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:10.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:10.073+0000 7fef691f9740 1 --2- 172.21.15.45:0/3081754949 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560c8a0b0890 0x560c8a0ae340 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:10.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:10.073+0000 7fef691f9740 1 --2- 172.21.15.45:0/3081754949 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560c8a0a7cb0 0x560c8a0ade00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:10.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:10.073+0000 7fef691f9740 1 --2- 172.21.15.45:0/3081754949 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560c8a0a7390 0x560c8a0ad8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:10.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:10.073+0000 7fef4d600640 0 client.27631 ms_handle_reset on v2:172.21.15.104:6838/3660070592 2024-04-16T17:14:10.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:10.073+0000 7fef691f9740 1 -- 172.21.15.45:0/3081754949 >> 172.21.15.45:0/3081754949 conn(0x560c8a0a1ac0 msgr2=0x560c8a0a3810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:14:10.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:10.073+0000 7fef691f9740 1 -- 172.21.15.45:0/3081754949 shutdown_connections 2024-04-16T17:14:10.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:10.073+0000 7fef691f9740 1 -- 172.21.15.45:0/3081754949 wait complete. 2024-04-16T17:14:10.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:10.073+0000 7fef691f9740 20 client.27631 trim_cache size 0 max 16384 2024-04-16T17:14:10.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:10.095+0000 7fef691f9740 1 Processor -- start 2024-04-16T17:14:10.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:10.095+0000 7fef691f9740 1 -- start start 2024-04-16T17:14:10.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:10.095+0000 7fef691f9740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560c8a0a7390 0x560c8a0afff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:10.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:10.095+0000 7fef691f9740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560c8a0a7cb0 0x560c8a0bc540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:10.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:10.096+0000 7fef691f9740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560c8a0b0890 0x560c8a0a09a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:10.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:10.096+0000 7fef691f9740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fef601b3da0 con 0x560c8a0b0890 2024-04-16T17:14:10.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:10.096+0000 7fef691f9740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fef601b1610 con 0x560c8a0a7390 2024-04-16T17:14:10.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:10.096+0000 7fef691f9740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fef601af480 con 0x560c8a0a7cb0 2024-04-16T17:14:10.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:10.096+0000 7fef5fe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560c8a0b0890 0x560c8a0a09a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:10.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:10.096+0000 7fef5f400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560c8a0a7390 0x560c8a0afff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:10.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:10.096+0000 7fef5fe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560c8a0b0890 0x560c8a0a09a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41332/0 (socket says 172.21.15.45:41332) 2024-04-16T17:14:10.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:10.096+0000 7fef5fe00640 1 -- 172.21.15.45:0/3980628625 learned_addr learned my addr 172.21.15.45:0/3980628625 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:14:10.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:10.096+0000 7fef5f400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560c8a0a7390 0x560c8a0afff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:33368/0 (socket says 172.21.15.45:33368) 2024-04-16T17:14:10.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:10.096+0000 7fef5ea00640 1 --2- 172.21.15.45:0/3980628625 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560c8a0a7cb0 0x560c8a0bc540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:10.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:10.096+0000 7fef5fe00640 1 -- 172.21.15.45:0/3980628625 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560c8a0a7cb0 msgr2=0x560c8a0bc540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:14:10.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:10.096+0000 7fef5fe00640 1 --2- 172.21.15.45:0/3980628625 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560c8a0a7cb0 0x560c8a0bc540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:10.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:10.096+0000 7fef5fe00640 1 -- 172.21.15.45:0/3980628625 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560c8a0a7390 msgr2=0x560c8a0afff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:14:10.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:10.096+0000 7fef5fe00640 1 --2- 172.21.15.45:0/3980628625 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560c8a0a7390 0x560c8a0afff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:10.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:10.096+0000 7fef5fe00640 1 -- 172.21.15.45:0/3980628625 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560c8a0b2c80 con 0x560c8a0b0890 2024-04-16T17:14:10.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:10.097+0000 7fef5fe00640 1 --2- 172.21.15.45:0/3980628625 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560c8a0b0890 0x560c8a0a09a0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fef580bb000 tx=0x7fef580d81c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb0171404b618b28 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:14:10.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:10.097+0000 7fef4fe00640 1 -- 172.21.15.45:0/3980628625 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef580bf2a0 con 0x560c8a0b0890 2024-04-16T17:14:10.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:10.097+0000 7fef4fe00640 1 -- 172.21.15.45:0/3980628625 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fef580c2f50 con 0x560c8a0b0890 2024-04-16T17:14:10.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:10.097+0000 7fef4fe00640 1 -- 172.21.15.45:0/3980628625 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef580be040 con 0x560c8a0b0890 2024-04-16T17:14:10.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:10.098+0000 7fef691f9740 1 -- 172.21.15.45:0/3980628625 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560c8a0b0890 msgr2=0x560c8a0a09a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:10.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:10.098+0000 7fef691f9740 1 --2- 172.21.15.45:0/3980628625 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560c8a0b0890 0x560c8a0a09a0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fef580bb000 tx=0x7fef580d81c0 comp rx=0 tx=0).stop 2024-04-16T17:14:10.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:10.098+0000 7fef691f9740 1 -- 172.21.15.45:0/3980628625 shutdown_connections 2024-04-16T17:14:10.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:10.098+0000 7fef691f9740 1 --2- 172.21.15.45:0/3980628625 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560c8a0a7cb0 0x560c8a0bc540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:10.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:10.098+0000 7fef691f9740 1 --2- 172.21.15.45:0/3980628625 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560c8a0b0890 0x560c8a0a09a0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:10.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:10.098+0000 7fef691f9740 1 --2- 172.21.15.45:0/3980628625 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560c8a0a7390 0x560c8a0afff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:10.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:10.098+0000 7fef691f9740 1 -- 172.21.15.45:0/3980628625 >> 172.21.15.45:0/3980628625 conn(0x560c8a0a1ac0 msgr2=0x560c8a0a0290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:14:10.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:10.098+0000 7fef691f9740 1 -- 172.21.15.45:0/3980628625 shutdown_connections 2024-04-16T17:14:10.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:10.098+0000 7fef691f9740 1 -- 172.21.15.45:0/3980628625 wait complete. 2024-04-16T17:14:10.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:10.099+0000 7fef691f9740 1 -- start start 2024-04-16T17:14:10.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:10.099+0000 7fef691f9740 1 Processor -- start 2024-04-16T17:14:10.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:10.100+0000 7fef691f9740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560c8a0a7390 0x560c8a0aa650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:10.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:10.100+0000 7fef691f9740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560c8a0a7cb0 0x560c8a0aab90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:10.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:10.100+0000 7fef691f9740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560c8a0b0890 0x560c8a0ab0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:10.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:10.100+0000 7fef691f9740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fef601afad0 con 0x560c8a0a7cb0 2024-04-16T17:14:10.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:10.100+0000 7fef691f9740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fef601b0f40 con 0x560c8a0b0890 2024-04-16T17:14:10.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:10.100+0000 7fef691f9740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fef601b01a0 con 0x560c8a0a7390 2024-04-16T17:14:10.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:10.100+0000 7fef691f9740 10 client.0 _ll_register_callbacks cb 0x7fef580baf30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:14:10.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:10.100+0000 7fef5ea00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560c8a0a7cb0 0x560c8a0aab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:10.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:10.100+0000 7fef5f400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560c8a0a7390 0x560c8a0aa650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:10.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:10.100+0000 7fef5fe00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560c8a0b0890 0x560c8a0ab0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:10.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:10.100+0000 7fef5f400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560c8a0a7390 0x560c8a0aa650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:55192/0 (socket says 172.21.15.45:55192) 2024-04-16T17:14:10.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:10.100+0000 7fef5f400640 1 -- 172.21.15.45:0/3622922006 learned_addr learned my addr 172.21.15.45:0/3622922006 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:14:10.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:10.100+0000 7fef5f400640 1 -- 172.21.15.45:0/3622922006 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560c8a0b0890 msgr2=0x560c8a0ab0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:14:10.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:10.100+0000 7fef5f400640 1 --2- 172.21.15.45:0/3622922006 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560c8a0b0890 0x560c8a0ab0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:10.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:10.100+0000 7fef5f400640 1 -- 172.21.15.45:0/3622922006 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560c8a0a7cb0 msgr2=0x560c8a0aab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:14:10.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:10.100+0000 7fef5f400640 1 --2- 172.21.15.45:0/3622922006 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560c8a0a7cb0 0x560c8a0aab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:10.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:10.100+0000 7fef5f400640 1 -- 172.21.15.45:0/3622922006 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560c8a1262e0 con 0x560c8a0a7390 2024-04-16T17:14:10.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:10.100+0000 7fef5fe00640 1 --2- 172.21.15.45:0/3622922006 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560c8a0b0890 0x560c8a0ab0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:14:10.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:10.101+0000 7fef5f400640 1 --2- 172.21.15.45:0/3622922006 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560c8a0a7390 0x560c8a0aa650 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fef50012cb0 tx=0x7fef500046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=30301bfe8c3c498 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:14:10.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:10.101+0000 7fef4d600640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-04-16T17:14:10.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:10.101+0000 7fef4d600640 1 -- 172.21.15.45:0/3622922006 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef5002e040 con 0x560c8a0a7390 2024-04-16T17:14:10.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:10.101+0000 7fef691f9740 1 -- 172.21.15.45:0/3622922006 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fef580c2f50 con 0x560c8a0a7390 2024-04-16T17:14:10.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:10.101+0000 7fef4d600640 1 -- 172.21.15.45:0/3622922006 <== mon.1 v2:172.21.15.117:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9496+0+0 (secure 0 0 0) 0x7fef5002e1e0 con 0x560c8a0a7390 2024-04-16T17:14:10.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:10.101+0000 7fef4d600640 1 -- 172.21.15.45:0/3622922006 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7fef50042050 con 0x560c8a0a7390 2024-04-16T17:14:10.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:10.101+0000 7fef691f9740 10 client.27637 fetch_fsmap learned FSMap version 42 2024-04-16T17:14:10.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:10.101+0000 7fef691f9740 1 -- 172.21.15.45:0/3622922006 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x560c8a127630 con 0x560c8a0a7390 2024-04-16T17:14:10.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:10.102+0000 7fef4d600640 1 -- 172.21.15.45:0/3622922006 <== mon.1 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fef580c2f50 con 0x560c8a0a7390 2024-04-16T17:14:10.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:10.102+0000 7fef4d600640 1 -- 172.21.15.45:0/3622922006 <== mon.1 v2:172.21.15.117:3300/0 5 ==== fsmap(e 42) ==== 8362+0+0 (secure 0 0 0) 0x7fef50039b10 con 0x560c8a0a7390 2024-04-16T17:14:10.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:10.102+0000 7fef691f9740 10 client.27637 fetch_fsmap finished waiting for FSMap version 42 2024-04-16T17:14:10.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:10.102+0000 7fef691f9740 10 client.27637 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:14:10.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:10.102+0000 7fef691f9740 20 client.27637 populate_metadata read hostname 'smithi045' 2024-04-16T17:14:10.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:10.102+0000 7fef691f9740 1 --2- 172.21.15.45:0/3622922006 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x560c8a127d30 0x560c8a12a150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:10.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:10.102+0000 7fef691f9740 4 client.27637 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-16T17:14:10.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:10.102+0000 7fef691f9740 1 -- 172.21.15.45:0/3622922006 --> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] -- command(tid 0: {"prefix": "scrub status"}) -- 0x560c8a0149b0 con 0x560c8a127d30 2024-04-16T17:14:10.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:10.102+0000 7fef5ea00640 1 --2- 172.21.15.45:0/3622922006 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x560c8a127d30 0x560c8a12a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:10.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:10.103+0000 7fef5ea00640 1 --2- 172.21.15.45:0/3622922006 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x560c8a127d30 0x560c8a12a150 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5d9b10aa685b7766 server_cookie=b08bc4ac0e0da000 in_seq=0 out_seq=0 2024-04-16T17:14:10.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:10.103+0000 7fef4d600640 10 client.27637 ms_handle_connect on v2:172.21.15.104:6838/3660070592 2024-04-16T17:14:10.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:10.103+0000 7fef4d600640 1 -- 172.21.15.45:0/3622922006 <== mds.0 v2:172.21.15.104:6838/3660070592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x560c8a0149b0 con 0x560c8a127d30 2024-04-16T17:14:10.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:10.103+0000 7fef4d600640 10 client.27637 handle_command_reply: tid=0 2024-04-16T17:14:10.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:10.103+0000 7fef691f9740 1 client.27637 shutdown 2024-04-16T17:14:10.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:10.103+0000 7fef691f9740 1 -- 172.21.15.45:0/3622922006 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560c8a0a7390 msgr2=0x560c8a0aa650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:10.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:10.103+0000 7fef691f9740 1 --2- 172.21.15.45:0/3622922006 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560c8a0a7390 0x560c8a0aa650 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fef50012cb0 tx=0x7fef500046c0 comp rx=0 tx=0).stop 2024-04-16T17:14:10.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:10.103+0000 7fef691f9740 1 -- 172.21.15.45:0/3622922006 shutdown_connections 2024-04-16T17:14:10.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:10.103+0000 7fef691f9740 1 --2- 172.21.15.45:0/3622922006 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560c8a0b0890 0x560c8a0ab0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:10.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:10.103+0000 7fef691f9740 1 --2- 172.21.15.45:0/3622922006 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560c8a0a7cb0 0x560c8a0aab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:10.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:10.103+0000 7fef691f9740 1 --2- 172.21.15.45:0/3622922006 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x560c8a127d30 0x560c8a12a150 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:10.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:10.103+0000 7fef691f9740 1 --2- 172.21.15.45:0/3622922006 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560c8a0a7390 0x560c8a0aa650 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:10.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:10.103+0000 7fef4d600640 0 client.27637 ms_handle_reset on v2:172.21.15.104:6838/3660070592 2024-04-16T17:14:10.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:10.103+0000 7fef691f9740 1 -- 172.21.15.45:0/3622922006 >> 172.21.15.45:0/3622922006 conn(0x560c8a0a1ac0 msgr2=0x560c8a0a0b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:14:10.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:10.104+0000 7fef691f9740 1 -- 172.21.15.45:0/3622922006 shutdown_connections 2024-04-16T17:14:10.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:10.104+0000 7fef691f9740 1 -- 172.21.15.45:0/3622922006 wait complete. 2024-04-16T17:14:10.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:10.104+0000 7fef691f9740 20 client.27637 trim_cache size 0 max 16384 2024-04-16T17:14:10.109 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-16T17:14:10.110 INFO:teuthology.orchestra.run.smithi045.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-16T17:14:10.110 INFO:teuthology.orchestra.run.smithi045.stdout: "scrubs": { 2024-04-16T17:14:10.110 INFO:teuthology.orchestra.run.smithi045.stdout: "4aa26d17-9a5c-4eba-9efb-8ecb9e61e393": { 2024-04-16T17:14:10.110 INFO:teuthology.orchestra.run.smithi045.stdout: "path": "/", 2024-04-16T17:14:10.110 INFO:teuthology.orchestra.run.smithi045.stdout: "tag": "4aa26d17-9a5c-4eba-9efb-8ecb9e61e393", 2024-04-16T17:14:10.110 INFO:teuthology.orchestra.run.smithi045.stdout: "options": "recursive,force" 2024-04-16T17:14:10.110 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:14:10.110 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:14:10.110 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-16T17:14:10.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:10.104+0000 7fef64e00640 1 -- 172.21.15.45:0/2649150293 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fef440a3da0 msgr2=0x7fef440a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:10.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:10.104+0000 7fef64e00640 1 --2- 172.21.15.45:0/2649150293 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fef440a3da0 0x7fef440a6260 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x560c8a0a6e60 tx=0x7fef50005f90 comp rx=0 tx=0).stop 2024-04-16T17:14:10.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:10.104+0000 7fef64e00640 1 -- 172.21.15.45:0/2649150293 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef6019a790 msgr2=0x7fef6019ac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:10.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:10.104+0000 7fef64e00640 1 --2- 172.21.15.45:0/2649150293 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef6019a790 0x7fef6019ac40 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fef58037040 tx=0x7fef58032f30 comp rx=0 tx=0).stop 2024-04-16T17:14:10.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:10.105+0000 7fef64e00640 1 -- 172.21.15.45:0/2649150293 shutdown_connections 2024-04-16T17:14:10.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:10.105+0000 7fef64e00640 1 --2- 172.21.15.45:0/2649150293 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fef440a3da0 0x7fef440a6260 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:10.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:10.105+0000 7fef64e00640 1 --2- 172.21.15.45:0/2649150293 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef6019a790 0x7fef6019ac40 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:10.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:10.105+0000 7fef64e00640 1 --2- 172.21.15.45:0/2649150293 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fef601093f0 0x7fef601a0b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:10.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:10.105+0000 7fef64e00640 1 --2- 172.21.15.45:0/2649150293 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fef60104320 0x7fef601a0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:10.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:10.105+0000 7fef64e00640 1 -- 172.21.15.45:0/2649150293 >> 172.21.15.45:0/2649150293 conn(0x7fef600ffb40 msgr2=0x7fef60100660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:14:10.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:10.105+0000 7fef64e00640 1 -- 172.21.15.45:0/2649150293 shutdown_connections 2024-04-16T17:14:10.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:10.105+0000 7fef64e00640 1 -- 172.21.15.45:0/2649150293 wait complete. 2024-04-16T17:14:10.122 INFO:tasks.cephfs.filesystem:scrub status for tag:4aa26d17-9a5c-4eba-9efb-8ecb9e61e393 - {'path': '/', 'tag': '4aa26d17-9a5c-4eba-9efb-8ecb9e61e393', 'options': 'recursive,force'} 2024-04-16T17:14:10.677 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:14:10 smithi104 ceph-mon[36457]: osdmap e100: 12 total, 12 up, 12 in 2024-04-16T17:14:10.677 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:14:10 smithi104 ceph-mon[36457]: pgmap v230: 129 pgs: 129 active+clean; 63 MiB data, 807 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 336 KiB/s wr, 15 op/s 2024-04-16T17:14:10.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:14:10 smithi104 ceph-mon[36457]: scrub summary: active paths [/] 2024-04-16T17:14:10.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:14:10 smithi104 ceph-mon[36457]: scrub summary: active paths [/] 2024-04-16T17:14:10.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:14:10 smithi104 ceph-mon[36457]: scrub summary: idle+waiting paths [/] 2024-04-16T17:14:10.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:14:10 smithi104 ceph-mon[36457]: scrub summary: active paths [/] 2024-04-16T17:14:10.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:14:10 smithi104 ceph-mon[36457]: scrub summary: idle+waiting paths [/] 2024-04-16T17:14:10.722 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:14:10 smithi045 ceph-mon[31676]: osdmap e100: 12 total, 12 up, 12 in 2024-04-16T17:14:10.722 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:14:10 smithi045 ceph-mon[31676]: pgmap v230: 129 pgs: 129 active+clean; 63 MiB data, 807 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 336 KiB/s wr, 15 op/s 2024-04-16T17:14:10.722 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:14:10 smithi045 ceph-mon[31676]: scrub summary: active paths [/] 2024-04-16T17:14:10.722 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:14:10 smithi045 ceph-mon[31676]: scrub summary: active paths [/] 2024-04-16T17:14:10.722 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:14:10 smithi045 ceph-mon[31676]: scrub summary: idle+waiting paths [/] 2024-04-16T17:14:10.723 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:14:10 smithi045 ceph-mon[31676]: scrub summary: active paths [/] 2024-04-16T17:14:10.723 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:14:10 smithi045 ceph-mon[31676]: scrub summary: idle+waiting paths [/] 2024-04-16T17:14:10.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:14:10 smithi117 ceph-mon[36730]: osdmap e100: 12 total, 12 up, 12 in 2024-04-16T17:14:10.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:14:10 smithi117 ceph-mon[36730]: pgmap v230: 129 pgs: 129 active+clean; 63 MiB data, 807 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 336 KiB/s wr, 15 op/s 2024-04-16T17:14:10.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:14:10 smithi117 ceph-mon[36730]: scrub summary: active paths [/] 2024-04-16T17:14:10.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:14:10 smithi117 ceph-mon[36730]: scrub summary: active paths [/] 2024-04-16T17:14:10.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:14:10 smithi117 ceph-mon[36730]: scrub summary: idle+waiting paths [/] 2024-04-16T17:14:10.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:14:10 smithi117 ceph-mon[36730]: scrub summary: active paths [/] 2024-04-16T17:14:10.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:14:10 smithi117 ceph-mon[36730]: scrub summary: idle+waiting paths [/] 2024-04-16T17:14:11.677 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:14:11 smithi104 ceph-mon[36457]: scrub queued for path: / 2024-04-16T17:14:11.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:14:11 smithi104 ceph-mon[36457]: scrub summary: idle+waiting paths [/] 2024-04-16T17:14:11.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:14:11 smithi104 ceph-mon[36457]: scrub summary: active paths [/] 2024-04-16T17:14:11.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:14:11 smithi104 ceph-mon[36457]: scrub summary: idle+waiting paths [/] 2024-04-16T17:14:11.678 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:14:11 smithi104 ceph-mon[36457]: scrub summary: idle+waiting paths [/] 2024-04-16T17:14:11.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:14:11 smithi045 ceph-mon[31676]: scrub queued for path: / 2024-04-16T17:14:11.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:14:11 smithi045 ceph-mon[31676]: scrub summary: idle+waiting paths [/] 2024-04-16T17:14:11.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:14:11 smithi045 ceph-mon[31676]: scrub summary: active paths [/] 2024-04-16T17:14:11.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:14:11 smithi045 ceph-mon[31676]: scrub summary: idle+waiting paths [/] 2024-04-16T17:14:11.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:14:11 smithi045 ceph-mon[31676]: scrub summary: idle+waiting paths [/] 2024-04-16T17:14:11.804 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:14:11 smithi117 ceph-mon[36730]: scrub queued for path: / 2024-04-16T17:14:11.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:14:11 smithi117 ceph-mon[36730]: scrub summary: idle+waiting paths [/] 2024-04-16T17:14:11.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:14:11 smithi117 ceph-mon[36730]: scrub summary: active paths [/] 2024-04-16T17:14:11.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:14:11 smithi117 ceph-mon[36730]: scrub summary: idle+waiting paths [/] 2024-04-16T17:14:11.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:14:11 smithi117 ceph-mon[36730]: scrub summary: idle+waiting paths [/] 2024-04-16T17:14:12.677 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:14:12 smithi104 ceph-mon[36457]: pgmap v231: 129 pgs: 129 active+clean; 63 MiB data, 807 MiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 671 KiB/s wr, 49 op/s 2024-04-16T17:14:12.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:14:12 smithi045 ceph-mon[31676]: pgmap v231: 129 pgs: 129 active+clean; 63 MiB data, 807 MiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 671 KiB/s wr, 49 op/s 2024-04-16T17:14:12.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:14:12 smithi117 ceph-mon[36730]: pgmap v231: 129 pgs: 129 active+clean; 63 MiB data, 807 MiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 671 KiB/s wr, 49 op/s 2024-04-16T17:14:14.677 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:14:14 smithi104 ceph-mon[36457]: pgmap v232: 129 pgs: 129 active+clean; 63 MiB data, 807 MiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 605 KiB/s wr, 44 op/s 2024-04-16T17:14:14.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:14:14 smithi045 ceph-mon[31676]: pgmap v232: 129 pgs: 129 active+clean; 63 MiB data, 807 MiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 605 KiB/s wr, 44 op/s 2024-04-16T17:14:14.804 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:14:14 smithi117 ceph-mon[36730]: pgmap v232: 129 pgs: 129 active+clean; 63 MiB data, 807 MiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 605 KiB/s wr, 44 op/s 2024-04-16T17:14:16.677 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:14:16 smithi104 ceph-mon[36457]: pgmap v233: 129 pgs: 129 active+clean; 63 MiB data, 807 MiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 537 KiB/s wr, 39 op/s 2024-04-16T17:14:16.726 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:14:16 smithi045 ceph-mon[31676]: pgmap v233: 129 pgs: 129 active+clean; 63 MiB data, 807 MiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 537 KiB/s wr, 39 op/s 2024-04-16T17:14:16.804 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:14:16 smithi117 ceph-mon[36730]: pgmap v233: 129 pgs: 129 active+clean; 63 MiB data, 807 MiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 537 KiB/s wr, 39 op/s 2024-04-16T17:14:18.677 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:14:18 smithi104 ceph-mon[36457]: pgmap v234: 129 pgs: 129 active+clean; 61 MiB data, 807 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 362 KiB/s wr, 42 op/s 2024-04-16T17:14:18.726 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:14:18 smithi045 ceph-mon[31676]: pgmap v234: 129 pgs: 129 active+clean; 61 MiB data, 807 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 362 KiB/s wr, 42 op/s 2024-04-16T17:14:18.804 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:14:18 smithi117 ceph-mon[36730]: pgmap v234: 129 pgs: 129 active+clean; 61 MiB data, 807 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 362 KiB/s wr, 42 op/s 2024-04-16T17:14:19.677 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:14:19 smithi104 ceph-mon[36457]: scrub summary: idle 2024-04-16T17:14:19.677 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:14:19 smithi104 ceph-mon[36457]: scrub summary: idle 2024-04-16T17:14:19.677 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:14:19 smithi104 ceph-mon[36457]: scrub summary: idle 2024-04-16T17:14:19.726 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:14:19 smithi045 ceph-mon[31676]: scrub summary: idle 2024-04-16T17:14:19.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:14:19 smithi045 ceph-mon[31676]: scrub summary: idle 2024-04-16T17:14:19.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:14:19 smithi045 ceph-mon[31676]: scrub summary: idle 2024-04-16T17:14:19.804 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:14:19 smithi117 ceph-mon[36730]: scrub summary: idle 2024-04-16T17:14:19.804 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:14:19 smithi117 ceph-mon[36730]: scrub summary: idle 2024-04-16T17:14:19.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:14:19 smithi117 ceph-mon[36730]: scrub summary: idle 2024-04-16T17:14:20.677 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:14:20 smithi104 ceph-mon[36457]: pgmap v235: 129 pgs: 129 active+clean; 61 MiB data, 807 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 362 KiB/s wr, 42 op/s 2024-04-16T17:14:20.726 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:14:20 smithi045 ceph-mon[31676]: pgmap v235: 129 pgs: 129 active+clean; 61 MiB data, 807 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 362 KiB/s wr, 42 op/s 2024-04-16T17:14:20.804 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:14:20 smithi117 ceph-mon[36730]: pgmap v235: 129 pgs: 129 active+clean; 61 MiB data, 807 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 362 KiB/s wr, 42 op/s 2024-04-16T17:14:22.677 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:14:22 smithi104 ceph-mon[36457]: pgmap v236: 129 pgs: 129 active+clean; 61 MiB data, 824 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 320 KiB/s wr, 43 op/s 2024-04-16T17:14:22.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:14:22 smithi045 ceph-mon[31676]: pgmap v236: 129 pgs: 129 active+clean; 61 MiB data, 824 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 320 KiB/s wr, 43 op/s 2024-04-16T17:14:22.804 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:14:22 smithi117 ceph-mon[36730]: pgmap v236: 129 pgs: 129 active+clean; 61 MiB data, 824 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 320 KiB/s wr, 43 op/s 2024-04-16T17:14:24.677 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:14:24 smithi104 ceph-mon[36457]: pgmap v237: 129 pgs: 129 active+clean; 61 MiB data, 824 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 97 KiB/s wr, 21 op/s 2024-04-16T17:14:24.726 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:14:24 smithi045 ceph-mon[31676]: pgmap v237: 129 pgs: 129 active+clean; 61 MiB data, 824 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 97 KiB/s wr, 21 op/s 2024-04-16T17:14:24.804 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:14:24 smithi117 ceph-mon[36730]: pgmap v237: 129 pgs: 129 active+clean; 61 MiB data, 824 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 97 KiB/s wr, 21 op/s 2024-04-16T17:14:26.677 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:14:26 smithi104 ceph-mon[36457]: pgmap v238: 129 pgs: 129 active+clean; 61 MiB data, 824 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 97 KiB/s wr, 21 op/s 2024-04-16T17:14:26.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:14:26 smithi045 ceph-mon[31676]: pgmap v238: 129 pgs: 129 active+clean; 61 MiB data, 824 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 97 KiB/s wr, 21 op/s 2024-04-16T17:14:26.804 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:14:26 smithi117 ceph-mon[36730]: pgmap v238: 129 pgs: 129 active+clean; 61 MiB data, 824 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 97 KiB/s wr, 21 op/s 2024-04-16T17:14:28.677 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:14:28 smithi104 ceph-mon[36457]: pgmap v239: 129 pgs: 129 active+clean; 60 MiB data, 824 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 343 KiB/s wr, 34 op/s 2024-04-16T17:14:28.726 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:14:28 smithi045 ceph-mon[31676]: pgmap v239: 129 pgs: 129 active+clean; 60 MiB data, 824 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 343 KiB/s wr, 34 op/s 2024-04-16T17:14:28.804 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:14:28 smithi117 ceph-mon[36730]: pgmap v239: 129 pgs: 129 active+clean; 60 MiB data, 824 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 343 KiB/s wr, 34 op/s 2024-04-16T17:14:30.679 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:14:30 smithi104 ceph-mon[36457]: pgmap v240: 129 pgs: 129 active+clean; 60 MiB data, 824 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 265 KiB/s wr, 22 op/s 2024-04-16T17:14:30.726 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:14:30 smithi045 ceph-mon[31676]: pgmap v240: 129 pgs: 129 active+clean; 60 MiB data, 824 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 265 KiB/s wr, 22 op/s 2024-04-16T17:14:30.804 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:14:30 smithi117 ceph-mon[36730]: pgmap v240: 129 pgs: 129 active+clean; 60 MiB data, 824 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 265 KiB/s wr, 22 op/s 2024-04-16T17:14:32.677 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:14:32 smithi104 ceph-mon[36457]: pgmap v241: 129 pgs: 129 active+clean; 60 MiB data, 833 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 590 KiB/s wr, 36 op/s 2024-04-16T17:14:32.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:14:32 smithi045 ceph-mon[31676]: pgmap v241: 129 pgs: 129 active+clean; 60 MiB data, 833 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 590 KiB/s wr, 36 op/s 2024-04-16T17:14:32.804 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:14:32 smithi117 ceph-mon[36730]: pgmap v241: 129 pgs: 129 active+clean; 60 MiB data, 833 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 590 KiB/s wr, 36 op/s 2024-04-16T17:14:34.677 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:14:34 smithi104 ceph-mon[36457]: pgmap v242: 129 pgs: 129 active+clean; 60 MiB data, 833 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 572 KiB/s wr, 28 op/s 2024-04-16T17:14:34.726 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:14:34 smithi045 ceph-mon[31676]: pgmap v242: 129 pgs: 129 active+clean; 60 MiB data, 833 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 572 KiB/s wr, 28 op/s 2024-04-16T17:14:34.804 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:14:34 smithi117 ceph-mon[36730]: pgmap v242: 129 pgs: 129 active+clean; 60 MiB data, 833 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 572 KiB/s wr, 28 op/s 2024-04-16T17:14:36.677 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:14:36 smithi104 ceph-mon[36457]: pgmap v243: 129 pgs: 129 active+clean; 60 MiB data, 833 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 572 KiB/s wr, 28 op/s 2024-04-16T17:14:36.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:14:36 smithi045 ceph-mon[31676]: pgmap v243: 129 pgs: 129 active+clean; 60 MiB data, 833 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 572 KiB/s wr, 28 op/s 2024-04-16T17:14:36.804 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:14:36 smithi117 ceph-mon[36730]: pgmap v243: 129 pgs: 129 active+clean; 60 MiB data, 833 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 572 KiB/s wr, 28 op/s 2024-04-16T17:14:38.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:14:38 smithi045 ceph-mon[31676]: pgmap v244: 129 pgs: 129 active+clean; 60 MiB data, 837 MiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 605 KiB/s wr, 38 op/s 2024-04-16T17:14:38.804 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:14:38 smithi117 ceph-mon[36730]: pgmap v244: 129 pgs: 129 active+clean; 60 MiB data, 837 MiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 605 KiB/s wr, 38 op/s 2024-04-16T17:14:38.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:14:38 smithi104 ceph-mon[36457]: pgmap v244: 129 pgs: 129 active+clean; 60 MiB data, 837 MiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 605 KiB/s wr, 38 op/s 2024-04-16T17:14:40.123 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-16T17:14:40.123 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-16T17:14:40.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.344+0000 7f65ec200640 1 -- 172.21.15.45:0/2335975151 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f65e41044b0 msgr2=0x7f65e4104890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:40.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.344+0000 7f65ec200640 1 --2- 172.21.15.45:0/2335975151 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f65e41044b0 0x7f65e4104890 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f65d8009860 tx=0x7f65d802bb70 comp rx=0 tx=0).stop 2024-04-16T17:14:40.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.344+0000 7f65ec200640 1 -- 172.21.15.45:0/2335975151 shutdown_connections 2024-04-16T17:14:40.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.344+0000 7f65ec200640 1 --2- 172.21.15.45:0/2335975151 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f65e4109580 0x7f65e4111150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:40.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.344+0000 7f65ec200640 1 --2- 172.21.15.45:0/2335975151 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f65e4104e60 0x7f65e4108ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:40.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.344+0000 7f65ec200640 1 --2- 172.21.15.45:0/2335975151 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f65e41044b0 0x7f65e4104890 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:40.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.344+0000 7f65ec200640 1 -- 172.21.15.45:0/2335975151 >> 172.21.15.45:0/2335975151 conn(0x7f65e40ffd90 msgr2=0x7f65e41021b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:14:40.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.345+0000 7f65ec200640 1 -- 172.21.15.45:0/2335975151 shutdown_connections 2024-04-16T17:14:40.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.345+0000 7f65ec200640 1 -- 172.21.15.45:0/2335975151 wait complete. 2024-04-16T17:14:40.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.345+0000 7f65ec200640 1 Processor -- start 2024-04-16T17:14:40.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.346+0000 7f65ec200640 1 -- start start 2024-04-16T17:14:40.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.346+0000 7f65ec200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f65e41044b0 0x7f65e41a07b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:40.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.346+0000 7f65ec200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f65e4109580 0x7f65e41a0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:40.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.346+0000 7f65ec200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f65e419a9b0 0x7f65e419ae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:40.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.346+0000 7f65ec200640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f65e4114610 con 0x7f65e41044b0 2024-04-16T17:14:40.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.346+0000 7f65ec200640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f65e4114490 con 0x7f65e419a9b0 2024-04-16T17:14:40.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.346+0000 7f65ec200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f65e4114790 con 0x7f65e4109580 2024-04-16T17:14:40.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.346+0000 7f65ea200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f65e419a9b0 0x7f65e419ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:40.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.346+0000 7f65e8e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f65e4109580 0x7f65e41a0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:40.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.346+0000 7f65e9800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f65e41044b0 0x7f65e41a07b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:40.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.346+0000 7f65e8e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f65e4109580 0x7f65e41a0cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:34154/0 (socket says 172.21.15.45:34154) 2024-04-16T17:14:40.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.346+0000 7f65e8e00640 1 -- 172.21.15.45:0/2802975 learned_addr learned my addr 172.21.15.45:0/2802975 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:14:40.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.346+0000 7f65ea200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f65e419a9b0 0x7f65e419ae60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:46458/0 (socket says 172.21.15.45:46458) 2024-04-16T17:14:40.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.347+0000 7f65e8e00640 1 -- 172.21.15.45:0/2802975 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f65e419a9b0 msgr2=0x7f65e419ae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:40.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.347+0000 7f65e8e00640 1 --2- 172.21.15.45:0/2802975 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f65e419a9b0 0x7f65e419ae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:40.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.347+0000 7f65e8e00640 1 -- 172.21.15.45:0/2802975 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f65e41044b0 msgr2=0x7f65e41a07b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:40.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.347+0000 7f65e8e00640 1 --2- 172.21.15.45:0/2802975 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f65e41044b0 0x7f65e41a07b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:40.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.347+0000 7f65e8e00640 1 -- 172.21.15.45:0/2802975 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65e419b4d0 con 0x7f65e4109580 2024-04-16T17:14:40.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.347+0000 7f65e9800640 1 --2- 172.21.15.45:0/2802975 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f65e41044b0 0x7f65e41a07b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:14:40.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.347+0000 7f65e8e00640 1 --2- 172.21.15.45:0/2802975 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f65e4109580 0x7f65e41a0cf0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f65d000e9f0 tx=0x7f65d000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:14:40.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.347+0000 7f65e2000640 1 -- 172.21.15.45:0/2802975 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65d000ccc0 con 0x7f65e4109580 2024-04-16T17:14:40.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.347+0000 7f65ec200640 1 -- 172.21.15.45:0/2802975 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65e419b7c0 con 0x7f65e4109580 2024-04-16T17:14:40.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.347+0000 7f65e2000640 1 -- 172.21.15.45:0/2802975 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f65d0010910 con 0x7f65e4109580 2024-04-16T17:14:40.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.348+0000 7f65e2000640 1 -- 172.21.15.45:0/2802975 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65d0018960 con 0x7f65e4109580 2024-04-16T17:14:40.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.348+0000 7f65ec200640 1 -- 172.21.15.45:0/2802975 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65e41a79c0 con 0x7f65e4109580 2024-04-16T17:14:40.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.351+0000 7f65eee45740 1 Processor -- start 2024-04-16T17:14:40.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.351+0000 7f65e2000640 1 -- 172.21.15.45:0/2802975 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f65d0010ac0 con 0x7f65e4109580 2024-04-16T17:14:40.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.351+0000 7f65eee45740 1 -- start start 2024-04-16T17:14:40.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.351+0000 7f65eee45740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562cf5e2a1b0 0x562cf5e2a590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:40.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.351+0000 7f65eee45740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562cf5e2aad0 0x562cf5e33170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:40.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.351+0000 7f65eee45740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562cf5e336b0 0x562cf5e35aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:40.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.351+0000 7f65eee45740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x562cf5d90120 con 0x562cf5e2a1b0 2024-04-16T17:14:40.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.351+0000 7f65eee45740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x562cf5d36340 con 0x562cf5e336b0 2024-04-16T17:14:40.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.351+0000 7f65eee45740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x562cf5d8c0e0 con 0x562cf5e2aad0 2024-04-16T17:14:40.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.352+0000 7f65ea200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562cf5e2a1b0 0x562cf5e2a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:40.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.352+0000 7f65e9800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562cf5e2aad0 0x562cf5e33170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:40.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.352+0000 7f65e8e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562cf5e336b0 0x562cf5e35aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:40.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.352+0000 7f65e9800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562cf5e2aad0 0x562cf5e33170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:34156/0 (socket says 172.21.15.45:34156) 2024-04-16T17:14:40.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.352+0000 7f65e9800640 1 -- 172.21.15.45:0/2464170697 learned_addr learned my addr 172.21.15.45:0/2464170697 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:14:40.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.352+0000 7f65e9800640 1 -- 172.21.15.45:0/2464170697 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562cf5e336b0 msgr2=0x562cf5e35aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:14:40.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.352+0000 7f65e9800640 1 --2- 172.21.15.45:0/2464170697 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562cf5e336b0 0x562cf5e35aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:40.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.352+0000 7f65e9800640 1 -- 172.21.15.45:0/2464170697 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562cf5e2a1b0 msgr2=0x562cf5e2a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:14:40.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.352+0000 7f65e9800640 1 --2- 172.21.15.45:0/2464170697 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562cf5e2a1b0 0x562cf5e2a590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:40.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.352+0000 7f65e9800640 1 -- 172.21.15.45:0/2464170697 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562cf5b6c7d0 con 0x562cf5e2aad0 2024-04-16T17:14:40.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.353+0000 7f65e9800640 1 --2- 172.21.15.45:0/2464170697 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562cf5e2aad0 0x562cf5e33170 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f65d8009860 tx=0x7f65d80058d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2f26a9c8ea0fa2c4 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:14:40.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.353+0000 7f65e2000640 1 --2- 172.21.15.45:0/2802975 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f65cc0a3d20 0x7f65cc0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:40.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.353+0000 7f65e2000640 1 -- 172.21.15.45:0/2802975 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9496+0+0 (secure 0 0 0) 0x7f65d00d0e70 con 0x7f65e4109580 2024-04-16T17:14:40.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.353+0000 7f65ea200640 1 --2- 172.21.15.45:0/2802975 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f65cc0a3d20 0x7f65cc0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:40.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.353+0000 7f65d6a00640 1 -- 172.21.15.45:0/2464170697 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65d8039040 con 0x562cf5e2aad0 2024-04-16T17:14:40.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.353+0000 7f65d6a00640 1 -- 172.21.15.45:0/2464170697 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f65d8004030 con 0x562cf5e2aad0 2024-04-16T17:14:40.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.353+0000 7f65d6a00640 1 -- 172.21.15.45:0/2464170697 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65d802ca20 con 0x562cf5e2aad0 2024-04-16T17:14:40.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.354+0000 7f65eee45740 1 -- 172.21.15.45:0/2464170697 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562cf5e2aad0 msgr2=0x562cf5e33170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:40.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.354+0000 7f65eee45740 1 --2- 172.21.15.45:0/2464170697 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562cf5e2aad0 0x562cf5e33170 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f65d8009860 tx=0x7f65d80058d0 comp rx=0 tx=0).stop 2024-04-16T17:14:40.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.354+0000 7f65ea200640 1 --2- 172.21.15.45:0/2802975 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f65cc0a3d20 0x7f65cc0a61e0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f65dc005fd0 tx=0x7f65dc009500 comp rx=0 tx=0).ready entity=mgr.25411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:14:40.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.354+0000 7f65eee45740 1 -- 172.21.15.45:0/2464170697 shutdown_connections 2024-04-16T17:14:40.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.354+0000 7f65eee45740 1 --2- 172.21.15.45:0/2464170697 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562cf5e2aad0 0x562cf5e33170 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:40.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.354+0000 7f65eee45740 1 --2- 172.21.15.45:0/2464170697 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562cf5e336b0 0x562cf5e35aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:40.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.354+0000 7f65eee45740 1 --2- 172.21.15.45:0/2464170697 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562cf5e2a1b0 0x562cf5e2a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:40.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.354+0000 7f65eee45740 1 -- 172.21.15.45:0/2464170697 >> 172.21.15.45:0/2464170697 conn(0x562cf5e248e0 msgr2=0x562cf5e24cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:14:40.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.354+0000 7f65eee45740 1 -- 172.21.15.45:0/2464170697 shutdown_connections 2024-04-16T17:14:40.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.354+0000 7f65eee45740 1 -- 172.21.15.45:0/2464170697 wait complete. 2024-04-16T17:14:40.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.355+0000 7f65eee45740 1 -- start start 2024-04-16T17:14:40.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.355+0000 7f65eee45740 1 Processor -- start 2024-04-16T17:14:40.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.355+0000 7f65eee45740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562cf5e2a1b0 0x562cf5e30690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:40.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.355+0000 7f65eee45740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562cf5e2aad0 0x562cf5e30bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:40.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.355+0000 7f65eee45740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562cf5e336b0 0x562cf5e31110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:40.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.355+0000 7f65eee45740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x562cf5e36440 con 0x562cf5e2aad0 2024-04-16T17:14:40.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.355+0000 7f65eee45740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x562cf5d49710 con 0x562cf5e336b0 2024-04-16T17:14:40.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.355+0000 7f65eee45740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x562cf5e365c0 con 0x562cf5e2a1b0 2024-04-16T17:14:40.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.355+0000 7f65eee45740 10 client.0 _ll_register_callbacks cb 0x562cf5d44af0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:14:40.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.356+0000 7f65e8e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562cf5e2aad0 0x562cf5e30bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:40.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.356+0000 7f65e9800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562cf5e2a1b0 0x562cf5e30690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:40.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.356+0000 7f65e9800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562cf5e2a1b0 0x562cf5e30690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:34168/0 (socket says 172.21.15.45:34168) 2024-04-16T17:14:40.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.356+0000 7f65e8e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562cf5e2aad0 0x562cf5e30bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59202/0 (socket says 172.21.15.45:59202) 2024-04-16T17:14:40.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.356+0000 7f65e9800640 1 -- 172.21.15.45:0/1920261357 learned_addr learned my addr 172.21.15.45:0/1920261357 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:14:40.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.356+0000 7f65ea200640 1 --2- 172.21.15.45:0/1920261357 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562cf5e336b0 0x562cf5e31110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:40.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.356+0000 7f65e9800640 1 -- 172.21.15.45:0/1920261357 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562cf5e336b0 msgr2=0x562cf5e31110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:14:40.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.356+0000 7f65e9800640 1 --2- 172.21.15.45:0/1920261357 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562cf5e336b0 0x562cf5e31110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:40.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.356+0000 7f65e9800640 1 -- 172.21.15.45:0/1920261357 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562cf5e2aad0 msgr2=0x562cf5e30bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:14:40.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.356+0000 7f65e9800640 1 --2- 172.21.15.45:0/1920261357 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562cf5e2aad0 0x562cf5e30bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:40.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.356+0000 7f65e9800640 1 -- 172.21.15.45:0/1920261357 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562cf5aa23f0 con 0x562cf5e2a1b0 2024-04-16T17:14:40.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.356+0000 7f65ea200640 1 --2- 172.21.15.45:0/1920261357 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562cf5e336b0 0x562cf5e31110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:14:40.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.356+0000 7f65e9800640 1 --2- 172.21.15.45:0/1920261357 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562cf5e2a1b0 0x562cf5e30690 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f65d80095f0 tx=0x7f65d8005d20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=208e959a5fc841c server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:14:40.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.356+0000 7f65cb400640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-04-16T17:14:40.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.357+0000 7f65cb400640 1 -- 172.21.15.45:0/1920261357 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65d8039040 con 0x562cf5e2a1b0 2024-04-16T17:14:40.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.357+0000 7f65cb400640 1 -- 172.21.15.45:0/1920261357 <== mon.1 v2:172.21.15.117:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9496+0+0 (secure 0 0 0) 0x7f65d802c7a0 con 0x562cf5e2a1b0 2024-04-16T17:14:40.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.357+0000 7f65eee45740 1 -- 172.21.15.45:0/1920261357 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f65d8004030 con 0x562cf5e2a1b0 2024-04-16T17:14:40.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.357+0000 7f65cb400640 1 -- 172.21.15.45:0/1920261357 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f65d8034850 con 0x562cf5e2a1b0 2024-04-16T17:14:40.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.357+0000 7f65eee45740 10 client.27649 fetch_fsmap learned FSMap version 42 2024-04-16T17:14:40.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.357+0000 7f65eee45740 1 -- 172.21.15.45:0/1920261357 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x562cf5a620f0 con 0x562cf5e2a1b0 2024-04-16T17:14:40.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.357+0000 7f65cb400640 1 -- 172.21.15.45:0/1920261357 <== mon.1 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f65d8004030 con 0x562cf5e2a1b0 2024-04-16T17:14:40.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.358+0000 7f65cb400640 1 -- 172.21.15.45:0/1920261357 <== mon.1 v2:172.21.15.117:3300/0 5 ==== fsmap(e 42) ==== 8362+0+0 (secure 0 0 0) 0x7f65d80035f0 con 0x562cf5e2a1b0 2024-04-16T17:14:40.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.358+0000 7f65eee45740 10 client.27649 fetch_fsmap finished waiting for FSMap version 42 2024-04-16T17:14:40.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.358+0000 7f65eee45740 10 client.27649 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:14:40.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.358+0000 7f65eee45740 20 client.27649 populate_metadata read hostname 'smithi045' 2024-04-16T17:14:40.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.358+0000 7f65eee45740 1 --2- 172.21.15.45:0/1920261357 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x562cf5e43810 0x562cf5e45c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:40.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.358+0000 7f65eee45740 4 client.27649 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:14:40.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.358+0000 7f65eee45740 1 -- 172.21.15.45:0/1920261357 --> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562cf5d9db40 con 0x562cf5e43810 2024-04-16T17:14:40.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.358+0000 7f65e8e00640 1 --2- 172.21.15.45:0/1920261357 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x562cf5e43810 0x562cf5e45c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:40.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.359+0000 7f65cb400640 10 client.27649 ms_handle_connect on v2:172.21.15.104:6838/3660070592 2024-04-16T17:14:40.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.359+0000 7f65e8e00640 1 --2- 172.21.15.45:0/1920261357 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x562cf5e43810 0x562cf5e45c00 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=644995212496ada7 server_cookie=add622e9d3c69525 in_seq=0 out_seq=0 2024-04-16T17:14:40.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.360+0000 7f65cb400640 1 -- 172.21.15.45:0/1920261357 <== mds.0 v2:172.21.15.104:6838/3660070592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562cf5d9db40 con 0x562cf5e43810 2024-04-16T17:14:40.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.360+0000 7f65cb400640 10 client.27649 handle_command_reply: tid=0 2024-04-16T17:14:40.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.360+0000 7f65eee45740 1 client.27649 shutdown 2024-04-16T17:14:40.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.360+0000 7f65eee45740 1 -- 172.21.15.45:0/1920261357 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562cf5e2a1b0 msgr2=0x562cf5e30690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:40.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.360+0000 7f65eee45740 1 --2- 172.21.15.45:0/1920261357 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562cf5e2a1b0 0x562cf5e30690 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f65d80095f0 tx=0x7f65d8005d20 comp rx=0 tx=0).stop 2024-04-16T17:14:40.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.360+0000 7f65eee45740 1 -- 172.21.15.45:0/1920261357 shutdown_connections 2024-04-16T17:14:40.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.360+0000 7f65eee45740 1 --2- 172.21.15.45:0/1920261357 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562cf5e336b0 0x562cf5e31110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:40.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.360+0000 7f65eee45740 1 --2- 172.21.15.45:0/1920261357 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562cf5e2aad0 0x562cf5e30bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:40.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.360+0000 7f65eee45740 1 --2- 172.21.15.45:0/1920261357 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x562cf5e43810 0x562cf5e45c00 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:40.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.360+0000 7f65eee45740 1 --2- 172.21.15.45:0/1920261357 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562cf5e2a1b0 0x562cf5e30690 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:40.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.360+0000 7f65eee45740 1 -- 172.21.15.45:0/1920261357 >> 172.21.15.45:0/1920261357 conn(0x562cf5e248e0 msgr2=0x562cf5e26630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:14:40.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.360+0000 7f65cb400640 0 client.27649 ms_handle_reset on v2:172.21.15.104:6838/3660070592 2024-04-16T17:14:40.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.360+0000 7f65eee45740 1 -- 172.21.15.45:0/1920261357 shutdown_connections 2024-04-16T17:14:40.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.360+0000 7f65eee45740 1 -- 172.21.15.45:0/1920261357 wait complete. 2024-04-16T17:14:40.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.361+0000 7f65eee45740 20 client.27649 trim_cache size 0 max 16384 2024-04-16T17:14:40.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.380+0000 7f65eee45740 1 Processor -- start 2024-04-16T17:14:40.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.380+0000 7f65eee45740 1 -- start start 2024-04-16T17:14:40.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.380+0000 7f65eee45740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562cf5e2a1b0 0x562cf5e32dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:40.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.380+0000 7f65eee45740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562cf5e2aad0 0x562cf5e3f300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:40.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.380+0000 7f65eee45740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562cf5e336b0 0x562cf5e4b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:40.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.380+0000 7f65eee45740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f65e41b4130 con 0x562cf5e2aad0 2024-04-16T17:14:40.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.380+0000 7f65eee45740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f65e41b19a0 con 0x562cf5e2a1b0 2024-04-16T17:14:40.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.380+0000 7f65eee45740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f65e41af810 con 0x562cf5e336b0 2024-04-16T17:14:40.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.380+0000 7f65e8e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562cf5e2aad0 0x562cf5e3f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:40.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.380+0000 7f65e9800640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562cf5e2a1b0 0x562cf5e32dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:40.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.380+0000 7f65e8e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562cf5e2aad0 0x562cf5e3f300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59218/0 (socket says 172.21.15.45:59218) 2024-04-16T17:14:40.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.380+0000 7f65e8e00640 1 -- 172.21.15.45:0/1286453098 learned_addr learned my addr 172.21.15.45:0/1286453098 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:14:40.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.381+0000 7f65ea200640 1 --2- 172.21.15.45:0/1286453098 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562cf5e336b0 0x562cf5e4b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:40.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.381+0000 7f65e9800640 1 -- 172.21.15.45:0/1286453098 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562cf5e336b0 msgr2=0x562cf5e4b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:14:40.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.381+0000 7f65e9800640 1 --2- 172.21.15.45:0/1286453098 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562cf5e336b0 0x562cf5e4b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:40.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.381+0000 7f65e9800640 1 -- 172.21.15.45:0/1286453098 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562cf5e2aad0 msgr2=0x562cf5e3f300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:14:40.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.381+0000 7f65e9800640 1 --2- 172.21.15.45:0/1286453098 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562cf5e2aad0 0x562cf5e3f300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:40.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.381+0000 7f65e9800640 1 -- 172.21.15.45:0/1286453098 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562cf5e35aa0 con 0x562cf5e2a1b0 2024-04-16T17:14:40.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.381+0000 7f65e9800640 1 --2- 172.21.15.45:0/1286453098 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562cf5e2a1b0 0x562cf5e32dc0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f65d80039f0 tx=0x7f65d80058d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4fef522c123b9d4f server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:14:40.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.381+0000 7f65d6000640 1 -- 172.21.15.45:0/1286453098 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65d8009450 con 0x562cf5e2a1b0 2024-04-16T17:14:40.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.382+0000 7f65d6000640 1 -- 172.21.15.45:0/1286453098 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f65d80035f0 con 0x562cf5e2a1b0 2024-04-16T17:14:40.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.382+0000 7f65d6000640 1 -- 172.21.15.45:0/1286453098 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65d802ca90 con 0x562cf5e2a1b0 2024-04-16T17:14:40.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.382+0000 7f65eee45740 1 -- 172.21.15.45:0/1286453098 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562cf5e2a1b0 msgr2=0x562cf5e32dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:40.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.382+0000 7f65eee45740 1 --2- 172.21.15.45:0/1286453098 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562cf5e2a1b0 0x562cf5e32dc0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f65d80039f0 tx=0x7f65d80058d0 comp rx=0 tx=0).stop 2024-04-16T17:14:40.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.382+0000 7f65eee45740 1 -- 172.21.15.45:0/1286453098 shutdown_connections 2024-04-16T17:14:40.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.382+0000 7f65eee45740 1 --2- 172.21.15.45:0/1286453098 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562cf5e336b0 0x562cf5e4b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:40.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.382+0000 7f65eee45740 1 --2- 172.21.15.45:0/1286453098 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562cf5e2aad0 0x562cf5e3f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:40.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.382+0000 7f65eee45740 1 --2- 172.21.15.45:0/1286453098 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562cf5e2a1b0 0x562cf5e32dc0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:40.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.382+0000 7f65eee45740 1 -- 172.21.15.45:0/1286453098 >> 172.21.15.45:0/1286453098 conn(0x562cf5e248e0 msgr2=0x562cf5e26380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:14:40.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.382+0000 7f65eee45740 1 -- 172.21.15.45:0/1286453098 shutdown_connections 2024-04-16T17:14:40.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.382+0000 7f65eee45740 1 -- 172.21.15.45:0/1286453098 wait complete. 2024-04-16T17:14:40.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.383+0000 7f65eee45740 1 -- start start 2024-04-16T17:14:40.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.383+0000 7f65eee45740 1 Processor -- start 2024-04-16T17:14:40.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.383+0000 7f65eee45740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562cf5e2a1b0 0x562cf5e2d600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:40.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.383+0000 7f65eee45740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562cf5e2aad0 0x562cf5e2db40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:40.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.383+0000 7f65eee45740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562cf5e336b0 0x562cf5e2e080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:40.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.384+0000 7f65eee45740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f65e41afe60 con 0x562cf5e2a1b0 2024-04-16T17:14:40.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.384+0000 7f65eee45740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f65e41b12d0 con 0x562cf5e336b0 2024-04-16T17:14:40.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.384+0000 7f65eee45740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f65e41b0530 con 0x562cf5e2aad0 2024-04-16T17:14:40.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.384+0000 7f65eee45740 10 client.0 _ll_register_callbacks cb 0x7f65dc009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:14:40.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.384+0000 7f65e9800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562cf5e2a1b0 0x562cf5e2d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:40.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.384+0000 7f65e9800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562cf5e2a1b0 0x562cf5e2d600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59230/0 (socket says 172.21.15.45:59230) 2024-04-16T17:14:40.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.384+0000 7f65e9800640 1 -- 172.21.15.45:0/101007834 learned_addr learned my addr 172.21.15.45:0/101007834 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:14:40.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.384+0000 7f65ea200640 1 --2- 172.21.15.45:0/101007834 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562cf5e336b0 0x562cf5e2e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:40.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.384+0000 7f65e8e00640 1 --2- 172.21.15.45:0/101007834 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562cf5e2aad0 0x562cf5e2db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:40.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.384+0000 7f65e9800640 1 -- 172.21.15.45:0/101007834 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562cf5e2aad0 msgr2=0x562cf5e2db40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:14:40.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.384+0000 7f65e9800640 1 --2- 172.21.15.45:0/101007834 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562cf5e2aad0 0x562cf5e2db40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:40.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.384+0000 7f65e9800640 1 -- 172.21.15.45:0/101007834 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562cf5e336b0 msgr2=0x562cf5e2e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:14:40.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.384+0000 7f65e9800640 1 --2- 172.21.15.45:0/101007834 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562cf5e336b0 0x562cf5e2e080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:40.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.384+0000 7f65e9800640 1 -- 172.21.15.45:0/101007834 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562cf5ea8d10 con 0x562cf5e2a1b0 2024-04-16T17:14:40.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.384+0000 7f65e8e00640 1 --2- 172.21.15.45:0/101007834 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562cf5e2aad0 0x562cf5e2db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:14:40.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.385+0000 7f65e9800640 1 --2- 172.21.15.45:0/101007834 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562cf5e2a1b0 0x562cf5e2d600 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f65d802c0b0 tx=0x7f65d8034a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58801c232370f67 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:14:40.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.385+0000 7f65cb400640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-16T17:14:40.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.385+0000 7f65cb400640 1 -- 172.21.15.45:0/101007834 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65d8003fa0 con 0x562cf5e2a1b0 2024-04-16T17:14:40.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.385+0000 7f65cb400640 1 -- 172.21.15.45:0/101007834 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9496+0+0 (secure 0 0 0) 0x7f65d802c730 con 0x562cf5e2a1b0 2024-04-16T17:14:40.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.385+0000 7f65eee45740 1 -- 172.21.15.45:0/101007834 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f65d80035f0 con 0x562cf5e2a1b0 2024-04-16T17:14:40.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.385+0000 7f65cb400640 1 -- 172.21.15.45:0/101007834 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f65d802c930 con 0x562cf5e2a1b0 2024-04-16T17:14:40.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.385+0000 7f65eee45740 10 client.17553 fetch_fsmap learned FSMap version 42 2024-04-16T17:14:40.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.385+0000 7f65eee45740 1 -- 172.21.15.45:0/101007834 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x562cf5eaa000 con 0x562cf5e2a1b0 2024-04-16T17:14:40.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.386+0000 7f65cb400640 1 -- 172.21.15.45:0/101007834 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f65d80035f0 con 0x562cf5e2a1b0 2024-04-16T17:14:40.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.386+0000 7f65cb400640 1 -- 172.21.15.45:0/101007834 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 42) ==== 8362+0+0 (secure 0 0 0) 0x7f65d803f440 con 0x562cf5e2a1b0 2024-04-16T17:14:40.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.386+0000 7f65eee45740 10 client.17553 fetch_fsmap finished waiting for FSMap version 42 2024-04-16T17:14:40.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.386+0000 7f65eee45740 10 client.17553 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:14:40.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.386+0000 7f65eee45740 20 client.17553 populate_metadata read hostname 'smithi045' 2024-04-16T17:14:40.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.386+0000 7f65eee45740 1 --2- 172.21.15.45:0/101007834 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x562cf5eaa5e0 0x562cf5eaca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:40.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.386+0000 7f65eee45740 4 client.17553 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-16T17:14:40.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.386+0000 7f65eee45740 1 -- 172.21.15.45:0/101007834 --> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] -- command(tid 0: {"prefix": "scrub status"}) -- 0x562cf5d979b0 con 0x562cf5eaa5e0 2024-04-16T17:14:40.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.386+0000 7f65e8e00640 1 --2- 172.21.15.45:0/101007834 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x562cf5eaa5e0 0x562cf5eaca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:40.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.387+0000 7f65e8e00640 1 --2- 172.21.15.45:0/101007834 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x562cf5eaa5e0 0x562cf5eaca00 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f1169c5ec29199e5 server_cookie=757790d2a3ad5515 in_seq=0 out_seq=0 2024-04-16T17:14:40.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.387+0000 7f65cb400640 10 client.17553 ms_handle_connect on v2:172.21.15.104:6838/3660070592 2024-04-16T17:14:40.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.387+0000 7f65cb400640 1 -- 172.21.15.45:0/101007834 <== mds.0 v2:172.21.15.104:6838/3660070592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x562cf5d979b0 con 0x562cf5eaa5e0 2024-04-16T17:14:40.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.387+0000 7f65cb400640 10 client.17553 handle_command_reply: tid=0 2024-04-16T17:14:40.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.387+0000 7f65eee45740 1 client.17553 shutdown 2024-04-16T17:14:40.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.387+0000 7f65eee45740 1 -- 172.21.15.45:0/101007834 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562cf5e2a1b0 msgr2=0x562cf5e2d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:40.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.387+0000 7f65eee45740 1 --2- 172.21.15.45:0/101007834 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562cf5e2a1b0 0x562cf5e2d600 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f65d802c0b0 tx=0x7f65d8034a10 comp rx=0 tx=0).stop 2024-04-16T17:14:40.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.387+0000 7f65eee45740 1 -- 172.21.15.45:0/101007834 shutdown_connections 2024-04-16T17:14:40.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.387+0000 7f65eee45740 1 --2- 172.21.15.45:0/101007834 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x562cf5eaa5e0 0x562cf5eaca00 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:40.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.387+0000 7f65eee45740 1 --2- 172.21.15.45:0/101007834 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562cf5e2aad0 0x562cf5e2db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:40.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.387+0000 7f65eee45740 1 --2- 172.21.15.45:0/101007834 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562cf5e336b0 0x562cf5e2e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:40.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.387+0000 7f65cb400640 0 client.17553 ms_handle_reset on v2:172.21.15.104:6838/3660070592 2024-04-16T17:14:40.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.387+0000 7f65eee45740 1 --2- 172.21.15.45:0/101007834 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562cf5e2a1b0 0x562cf5e2d600 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:40.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.387+0000 7f65eee45740 1 -- 172.21.15.45:0/101007834 >> 172.21.15.45:0/101007834 conn(0x562cf5e248e0 msgr2=0x562cf5e26380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:14:40.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.388+0000 7f65eee45740 1 -- 172.21.15.45:0/101007834 shutdown_connections 2024-04-16T17:14:40.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.388+0000 7f65eee45740 1 -- 172.21.15.45:0/101007834 wait complete. 2024-04-16T17:14:40.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.388+0000 7f65eee45740 20 client.17553 trim_cache size 0 max 16384 2024-04-16T17:14:40.396 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-16T17:14:40.396 INFO:teuthology.orchestra.run.smithi045.stdout: "status": "no active scrubs running", 2024-04-16T17:14:40.396 INFO:teuthology.orchestra.run.smithi045.stdout: "scrubs": {} 2024-04-16T17:14:40.396 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-16T17:14:40.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.388+0000 7f65ec200640 1 -- 172.21.15.45:0/2802975 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f65cc0a3d20 msgr2=0x7f65cc0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:40.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.388+0000 7f65ec200640 1 --2- 172.21.15.45:0/2802975 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f65cc0a3d20 0x7f65cc0a61e0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f65dc005fd0 tx=0x7f65dc009500 comp rx=0 tx=0).stop 2024-04-16T17:14:40.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.388+0000 7f65ec200640 1 -- 172.21.15.45:0/2802975 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f65e4109580 msgr2=0x7f65e41a0cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:40.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.388+0000 7f65ec200640 1 --2- 172.21.15.45:0/2802975 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f65e4109580 0x7f65e41a0cf0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f65d000e9f0 tx=0x7f65d000eeb0 comp rx=0 tx=0).stop 2024-04-16T17:14:40.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.388+0000 7f65ec200640 1 -- 172.21.15.45:0/2802975 shutdown_connections 2024-04-16T17:14:40.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.388+0000 7f65ec200640 1 --2- 172.21.15.45:0/2802975 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f65cc0a3d20 0x7f65cc0a61e0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:40.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.388+0000 7f65ec200640 1 --2- 172.21.15.45:0/2802975 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f65e4109580 0x7f65e41a0cf0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:40.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.388+0000 7f65ec200640 1 --2- 172.21.15.45:0/2802975 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f65e419a9b0 0x7f65e419ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:40.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.388+0000 7f65ec200640 1 --2- 172.21.15.45:0/2802975 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f65e41044b0 0x7f65e41a07b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:40.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.388+0000 7f65ec200640 1 -- 172.21.15.45:0/2802975 >> 172.21.15.45:0/2802975 conn(0x7f65e40ffd90 msgr2=0x7f65e41010b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:14:40.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.389+0000 7f65ec200640 1 -- 172.21.15.45:0/2802975 shutdown_connections 2024-04-16T17:14:40.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.389+0000 7f65ec200640 1 -- 172.21.15.45:0/2802975 wait complete. 2024-04-16T17:14:40.408 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-16T17:14:40.408 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-16T17:14:40.478 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:14:40 smithi045 ceph-mon[31676]: pgmap v245: 129 pgs: 129 active+clean; 60 MiB data, 837 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 359 KiB/s wr, 24 op/s 2024-04-16T17:14:40.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.610+0000 7f0c37e00640 1 -- 172.21.15.45:0/229731876 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0c381071a0 msgr2=0x7f0c3810ed40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:40.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.610+0000 7f0c37e00640 1 --2- 172.21.15.45:0/229731876 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0c381071a0 0x7f0c3810ed40 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f0c30009be0 tx=0x7f0c3002bef0 comp rx=0 tx=0).stop 2024-04-16T17:14:40.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.610+0000 7f0c37e00640 1 -- 172.21.15.45:0/229731876 shutdown_connections 2024-04-16T17:14:40.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.610+0000 7f0c37e00640 1 --2- 172.21.15.45:0/229731876 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0c381071a0 0x7f0c3810ed40 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:40.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.610+0000 7f0c37e00640 1 --2- 172.21.15.45:0/229731876 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0c38069db0 0x7f0c38106a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:40.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.610+0000 7f0c37e00640 1 --2- 172.21.15.45:0/229731876 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0c38069400 0x7f0c380697e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:40.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.610+0000 7f0c37e00640 1 -- 172.21.15.45:0/229731876 >> 172.21.15.45:0/229731876 conn(0x7f0c38076840 msgr2=0x7f0c38076c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:14:40.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.610+0000 7f0c37e00640 1 -- 172.21.15.45:0/229731876 shutdown_connections 2024-04-16T17:14:40.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.610+0000 7f0c37e00640 1 -- 172.21.15.45:0/229731876 wait complete. 2024-04-16T17:14:40.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.611+0000 7f0c37e00640 1 Processor -- start 2024-04-16T17:14:40.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.611+0000 7f0c37e00640 1 -- start start 2024-04-16T17:14:40.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.612+0000 7f0c37e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0c38069400 0x7f0c381010c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:40.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.612+0000 7f0c37e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0c381071a0 0x7f0c38101600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:40.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.612+0000 7f0c37e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0c38101b40 0x7f0c381a6880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:40.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.612+0000 7f0c37e00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f0c38112210 con 0x7f0c38101b40 2024-04-16T17:14:40.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.612+0000 7f0c37e00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f0c38112090 con 0x7f0c38069400 2024-04-16T17:14:40.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.612+0000 7f0c37e00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f0c38112390 con 0x7f0c381071a0 2024-04-16T17:14:40.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.612+0000 7f0c36000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0c381071a0 0x7f0c38101600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:40.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.612+0000 7f0c36a00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0c38069400 0x7f0c381010c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:40.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.612+0000 7f0c36000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0c381071a0 0x7f0c38101600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:34218/0 (socket says 172.21.15.45:34218) 2024-04-16T17:14:40.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.612+0000 7f0c36000640 1 -- 172.21.15.45:0/1557852504 learned_addr learned my addr 172.21.15.45:0/1557852504 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:14:40.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.612+0000 7f0c36a00640 1 --2- 172.21.15.45:0/1557852504 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0c38069400 0x7f0c381010c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:46512/0 (socket says 172.21.15.45:46512) 2024-04-16T17:14:40.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.612+0000 7f0c37400640 1 --2- 172.21.15.45:0/1557852504 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0c38101b40 0x7f0c381a6880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:40.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.612+0000 7f0c36000640 1 -- 172.21.15.45:0/1557852504 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0c38069400 msgr2=0x7f0c381010c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:40.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.612+0000 7f0c36000640 1 --2- 172.21.15.45:0/1557852504 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0c38069400 0x7f0c381010c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:40.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.612+0000 7f0c36000640 1 -- 172.21.15.45:0/1557852504 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0c38101b40 msgr2=0x7f0c381a6880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:40.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.612+0000 7f0c36000640 1 --2- 172.21.15.45:0/1557852504 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0c38101b40 0x7f0c381a6880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:40.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.612+0000 7f0c36000640 1 -- 172.21.15.45:0/1557852504 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c381a6f80 con 0x7f0c381071a0 2024-04-16T17:14:40.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.612+0000 7f0c36a00640 1 --2- 172.21.15.45:0/1557852504 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0c38069400 0x7f0c381010c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:14:40.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.613+0000 7f0c36000640 1 --2- 172.21.15.45:0/1557852504 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0c381071a0 0x7f0c38101600 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f0c2400e9c0 tx=0x7f0c2400ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:14:40.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.613+0000 7f0c2f400640 1 -- 172.21.15.45:0/1557852504 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c2400ccc0 con 0x7f0c381071a0 2024-04-16T17:14:40.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.613+0000 7f0c37e00640 1 -- 172.21.15.45:0/1557852504 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c381a7270 con 0x7f0c381071a0 2024-04-16T17:14:40.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.613+0000 7f0c37e00640 1 -- 172.21.15.45:0/1557852504 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c381a7810 con 0x7f0c381071a0 2024-04-16T17:14:40.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.613+0000 7f0c2f400640 1 -- 172.21.15.45:0/1557852504 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0c24010910 con 0x7f0c381071a0 2024-04-16T17:14:40.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.613+0000 7f0c2f400640 1 -- 172.21.15.45:0/1557852504 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c24026060 con 0x7f0c381071a0 2024-04-16T17:14:40.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.614+0000 7f0c37e00640 1 -- 172.21.15.45:0/1557852504 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c18005290 con 0x7f0c381071a0 2024-04-16T17:14:40.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.617+0000 7f0c2f400640 1 -- 172.21.15.45:0/1557852504 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f0c24010ac0 con 0x7f0c381071a0 2024-04-16T17:14:40.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.618+0000 7f0c2f400640 1 --2- 172.21.15.45:0/1557852504 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f0c1c0a3dd0 0x7f0c1c0a6290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:40.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.618+0000 7f0c2f400640 1 -- 172.21.15.45:0/1557852504 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9496+0+0 (secure 0 0 0) 0x7f0c240d0ca0 con 0x7f0c381071a0 2024-04-16T17:14:40.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.618+0000 7f0c36a00640 1 --2- 172.21.15.45:0/1557852504 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f0c1c0a3dd0 0x7f0c1c0a6290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:40.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.619+0000 7f0c36a00640 1 --2- 172.21.15.45:0/1557852504 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f0c1c0a3dd0 0x7f0c1c0a6290 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f0c28009fc0 tx=0x7f0c28005f90 comp rx=0 tx=0).ready entity=mgr.25411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:14:40.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.622+0000 7f0c2f400640 1 -- 172.21.15.45:0/1557852504 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0c2409c8c0 con 0x7f0c381071a0 2024-04-16T17:14:40.804 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:14:40 smithi117 ceph-mon[36730]: pgmap v245: 129 pgs: 129 active+clean; 60 MiB data, 837 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 359 KiB/s wr, 24 op/s 2024-04-16T17:14:40.873 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.872+0000 7f0c37e00640 1 -- 172.21.15.45:0/1557852504 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0c18005850 con 0x7f0c381071a0 2024-04-16T17:14:40.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.873+0000 7f0c2f400640 1 -- 172.21.15.45:0/1557852504 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) ==== 94+0+11893 (secure 0 0 0) 0x7f0c240a0bc0 con 0x7f0c381071a0 2024-04-16T17:14:40.875 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-16T17:14:40.876 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":42,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/1748842443","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":1748842443},{"type":"v1","addr":"172.21.15.45:6835","nonce":1748842443}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6839/1308627051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":1308627051},{"type":"v1","addr":"172.21.15.45:6839","nonce":1308627051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/1768242489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1768242489},{"type":"v1","addr":"172.21.15.117:6833","nonce":1768242489}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6833/3558902585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":3558902585},{"type":"v1","addr":"172.21.15.104:6833","nonce":3558902585}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24478,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/1308142903","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":1308142903},{"type":"v1","addr":"172.21.15.45:6837","nonce":1308142903}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24491,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6835/3430679937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":3430679937},{"type":"v1","addr":"172.21.15.104:6835","nonce":3430679937}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":42,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-16T17:03:24.663611+0000","modified":"2024-04-16T17:13:48.424453+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24581,"mds_1":24569,"mds_2":24532},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24499":{"gid":24499,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/2224123841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2224123841},{"type":"v1","addr":"172.21.15.117:6835","nonce":2224123841}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6837/2471356815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":2471356815},{"type":"v1","addr":"172.21.15.117:6837","nonce":2471356815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24532":{"gid":24532,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":13,"addr":"172.21.15.117:6839/2378675949","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":2378675949},{"type":"v1","addr":"172.21.15.117:6839","nonce":2378675949}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24569":{"gid":24569,"name":"a","rank":1,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.45:6841/3755310822","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6840","nonce":3755310822},{"type":"v1","addr":"172.21.15.45:6841","nonce":3755310822}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24581":{"gid":24581,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.104:6839/3660070592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6838","nonce":3660070592},{"type":"v1","addr":"172.21.15.104:6839","nonce":3660070592}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_27569":{"gid":27569,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6837/1833645246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":1833645246},{"type":"v1","addr":"172.21.15.104:6837","nonce":1833645246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24581,"qdb_cluster":[24569,24532,24581]},"id":1}]} 2024-04-16T17:14:40.877 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 42 2024-04-16T17:14:40.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.875+0000 7f0c37e00640 1 -- 172.21.15.45:0/1557852504 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f0c1c0a3dd0 msgr2=0x7f0c1c0a6290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:40.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.875+0000 7f0c37e00640 1 --2- 172.21.15.45:0/1557852504 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f0c1c0a3dd0 0x7f0c1c0a6290 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f0c28009fc0 tx=0x7f0c28005f90 comp rx=0 tx=0).stop 2024-04-16T17:14:40.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.875+0000 7f0c37e00640 1 -- 172.21.15.45:0/1557852504 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0c381071a0 msgr2=0x7f0c38101600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:40.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.875+0000 7f0c37e00640 1 --2- 172.21.15.45:0/1557852504 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0c381071a0 0x7f0c38101600 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f0c2400e9c0 tx=0x7f0c2400ee80 comp rx=0 tx=0).stop 2024-04-16T17:14:40.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.875+0000 7f0c37e00640 1 -- 172.21.15.45:0/1557852504 shutdown_connections 2024-04-16T17:14:40.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.876+0000 7f0c37e00640 1 --2- 172.21.15.45:0/1557852504 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f0c1c0a3dd0 0x7f0c1c0a6290 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:40.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.876+0000 7f0c37e00640 1 --2- 172.21.15.45:0/1557852504 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0c381071a0 0x7f0c38101600 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:40.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.876+0000 7f0c37e00640 1 --2- 172.21.15.45:0/1557852504 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0c38101b40 0x7f0c381a6880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:40.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.876+0000 7f0c37e00640 1 --2- 172.21.15.45:0/1557852504 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0c38069400 0x7f0c381010c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:40.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.876+0000 7f0c37e00640 1 -- 172.21.15.45:0/1557852504 >> 172.21.15.45:0/1557852504 conn(0x7f0c38076840 msgr2=0x7f0c38100770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:14:40.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.876+0000 7f0c37e00640 1 -- 172.21.15.45:0/1557852504 shutdown_connections 2024-04-16T17:14:40.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:40.876+0000 7f0c37e00640 1 -- 172.21.15.45:0/1557852504 wait complete. 2024-04-16T17:14:40.895 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-16T17:14:40.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:14:40 smithi104 ceph-mon[36457]: pgmap v245: 129 pgs: 129 active+clean; 60 MiB data, 837 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 359 KiB/s wr, 24 op/s 2024-04-16T17:14:41.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.116+0000 7f00c6200640 1 -- 172.21.15.45:0/2739622686 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f00c0076530 msgr2=0x7f00c0076910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:41.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.116+0000 7f00c6200640 1 --2- 172.21.15.45:0/2739622686 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f00c0076530 0x7f00c0076910 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f00b4009930 tx=0x7f00b402bc20 comp rx=0 tx=0).stop 2024-04-16T17:14:41.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.116+0000 7f00c6200640 1 -- 172.21.15.45:0/2739622686 shutdown_connections 2024-04-16T17:14:41.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.116+0000 7f00c6200640 1 --2- 172.21.15.45:0/2739622686 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f00c0076e50 0x7f00c0112800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:41.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.116+0000 7f00c6200640 1 --2- 172.21.15.45:0/2739622686 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f00c0112d40 0x7f00c0115130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:41.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.116+0000 7f00c6200640 1 --2- 172.21.15.45:0/2739622686 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f00c0076530 0x7f00c0076910 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:41.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.116+0000 7f00c6200640 1 -- 172.21.15.45:0/2739622686 >> 172.21.15.45:0/2739622686 conn(0x7f00c00ffb10 msgr2=0x7f00c0101f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:14:41.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.117+0000 7f00c6200640 1 -- 172.21.15.45:0/2739622686 shutdown_connections 2024-04-16T17:14:41.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.117+0000 7f00c6200640 1 -- 172.21.15.45:0/2739622686 wait complete. 2024-04-16T17:14:41.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.118+0000 7f00c6200640 1 Processor -- start 2024-04-16T17:14:41.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.118+0000 7f00c6200640 1 -- start start 2024-04-16T17:14:41.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.118+0000 7f00c6200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f00c0076530 0x7f00c019edf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:41.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.118+0000 7f00c6200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f00c0076e50 0x7f00c019f330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:41.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.118+0000 7f00c6200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f00c0112d40 0x7f00c01a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:41.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.118+0000 7f00c6200640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f00c0118890 con 0x7f00c0076530 2024-04-16T17:14:41.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.118+0000 7f00c6200640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f00c0118710 con 0x7f00c0112d40 2024-04-16T17:14:41.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.118+0000 7f00c6200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f00c0118a10 con 0x7f00c0076e50 2024-04-16T17:14:41.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.118+0000 7f00c4e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f00c0076530 0x7f00c019edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:41.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.118+0000 7f00c4e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f00c0076530 0x7f00c019edf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59272/0 (socket says 172.21.15.45:59272) 2024-04-16T17:14:41.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.118+0000 7f00c4e00640 1 -- 172.21.15.45:0/1796674374 learned_addr learned my addr 172.21.15.45:0/1796674374 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:14:41.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.118+0000 7f00bfe00640 1 --2- 172.21.15.45:0/1796674374 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f00c0076e50 0x7f00c019f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:41.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.118+0000 7f00c5800640 1 --2- 172.21.15.45:0/1796674374 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f00c0112d40 0x7f00c01a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:41.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.119+0000 7f00bfe00640 1 -- 172.21.15.45:0/1796674374 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f00c0112d40 msgr2=0x7f00c01a3660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:41.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.119+0000 7f00bfe00640 1 --2- 172.21.15.45:0/1796674374 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f00c0112d40 0x7f00c01a3660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:41.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.119+0000 7f00bfe00640 1 -- 172.21.15.45:0/1796674374 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f00c0076530 msgr2=0x7f00c019edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:41.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.119+0000 7f00bfe00640 1 --2- 172.21.15.45:0/1796674374 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f00c0076530 0x7f00c019edf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:41.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.119+0000 7f00bfe00640 1 -- 172.21.15.45:0/1796674374 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00c01a3d90 con 0x7f00c0076e50 2024-04-16T17:14:41.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.119+0000 7f00c4e00640 1 --2- 172.21.15.45:0/1796674374 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f00c0076530 0x7f00c019edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:14:41.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.119+0000 7f00bfe00640 1 --2- 172.21.15.45:0/1796674374 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f00c0076e50 0x7f00c019f330 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f00ac00e9f0 tx=0x7f00ac00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:14:41.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.119+0000 7f00bd600640 1 -- 172.21.15.45:0/1796674374 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00ac00ccc0 con 0x7f00c0076e50 2024-04-16T17:14:41.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.119+0000 7f00c6200640 1 -- 172.21.15.45:0/1796674374 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00c01a4080 con 0x7f00c0076e50 2024-04-16T17:14:41.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.120+0000 7f00bd600640 1 -- 172.21.15.45:0/1796674374 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f00ac010910 con 0x7f00c0076e50 2024-04-16T17:14:41.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.120+0000 7f00bd600640 1 -- 172.21.15.45:0/1796674374 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00ac018960 con 0x7f00c0076e50 2024-04-16T17:14:41.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.120+0000 7f00c6200640 1 -- 172.21.15.45:0/1796674374 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00c01ab910 con 0x7f00c0076e50 2024-04-16T17:14:41.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.123+0000 7f00ca570740 1 Processor -- start 2024-04-16T17:14:41.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.123+0000 7f00ca570740 1 -- start start 2024-04-16T17:14:41.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.123+0000 7f00ca570740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ab65a1af10 0x55ab65a1b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:41.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.123+0000 7f00ca570740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ab65a1b830 0x55ab65a23ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:41.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.123+0000 7f00ca570740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ab65a24410 0x55ab65a26800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:41.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.123+0000 7f00ca570740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55ab65980020 con 0x55ab65a1b830 2024-04-16T17:14:41.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.123+0000 7f00ca570740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55ab65926460 con 0x55ab65a24410 2024-04-16T17:14:41.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.123+0000 7f00ca570740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55ab6597bfe0 con 0x55ab65a1af10 2024-04-16T17:14:41.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.123+0000 7f00c4e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ab65a1b830 0x55ab65a23ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:41.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.123+0000 7f00c4e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ab65a1b830 0x55ab65a23ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59286/0 (socket says 172.21.15.45:59286) 2024-04-16T17:14:41.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.123+0000 7f00c4e00640 1 -- 172.21.15.45:0/4175618112 learned_addr learned my addr 172.21.15.45:0/4175618112 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:14:41.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.123+0000 7f00c5800640 1 --2- 172.21.15.45:0/4175618112 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ab65a1af10 0x55ab65a1b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:41.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.124+0000 7f00bd600640 1 -- 172.21.15.45:0/1796674374 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f00ac010ac0 con 0x7f00c0076e50 2024-04-16T17:14:41.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.124+0000 7f00c4e00640 1 -- 172.21.15.45:0/4175618112 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ab65a1af10 msgr2=0x55ab65a1b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:14:41.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.124+0000 7f00c4e00640 1 --2- 172.21.15.45:0/4175618112 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ab65a1af10 0x55ab65a1b2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:41.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.124+0000 7f00c4e00640 1 -- 172.21.15.45:0/4175618112 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ab65a24410 msgr2=0x55ab65a26800 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-16T17:14:41.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.124+0000 7f00c4e00640 1 --2- 172.21.15.45:0/4175618112 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ab65a24410 0x55ab65a26800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:41.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.124+0000 7f00c4e00640 1 -- 172.21.15.45:0/4175618112 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ab6592e040 con 0x55ab65a1b830 2024-04-16T17:14:41.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.124+0000 7f00c4e00640 1 --2- 172.21.15.45:0/4175618112 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ab65a1b830 0x55ab65a23ed0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f00b4036040 tx=0x7f00b4009670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4f22d1adb18f3aa server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:14:41.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.124+0000 7f00b2000640 1 -- 172.21.15.45:0/4175618112 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00b4004c70 con 0x55ab65a1b830 2024-04-16T17:14:41.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.125+0000 7f00b2000640 1 -- 172.21.15.45:0/4175618112 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f00b4004030 con 0x55ab65a1b830 2024-04-16T17:14:41.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.125+0000 7f00b2000640 1 -- 172.21.15.45:0/4175618112 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00b4005370 con 0x55ab65a1b830 2024-04-16T17:14:41.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.125+0000 7f00ca570740 1 -- 172.21.15.45:0/4175618112 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ab65a1b830 msgr2=0x55ab65a23ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:41.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.125+0000 7f00ca570740 1 --2- 172.21.15.45:0/4175618112 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ab65a1b830 0x55ab65a23ed0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f00b4036040 tx=0x7f00b4009670 comp rx=0 tx=0).stop 2024-04-16T17:14:41.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.125+0000 7f00bd600640 1 --2- 172.21.15.45:0/1796674374 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f00a80a3d20 0x7f00a80a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:41.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.125+0000 7f00bd600640 1 -- 172.21.15.45:0/1796674374 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9496+0+0 (secure 0 0 0) 0x7f00ac0d0f60 con 0x7f00c0076e50 2024-04-16T17:14:41.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.125+0000 7f00ca570740 1 -- 172.21.15.45:0/4175618112 shutdown_connections 2024-04-16T17:14:41.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.125+0000 7f00ca570740 1 --2- 172.21.15.45:0/4175618112 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ab65a24410 0x55ab65a26800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:41.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.125+0000 7f00ca570740 1 --2- 172.21.15.45:0/4175618112 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ab65a1b830 0x55ab65a23ed0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:41.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.125+0000 7f00ca570740 1 --2- 172.21.15.45:0/4175618112 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ab65a1af10 0x55ab65a1b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:41.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.125+0000 7f00ca570740 1 -- 172.21.15.45:0/4175618112 >> 172.21.15.45:0/4175618112 conn(0x55ab65a15640 msgr2=0x55ab65a15a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:14:41.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.126+0000 7f00c5800640 1 --2- 172.21.15.45:0/1796674374 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f00a80a3d20 0x7f00a80a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:41.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.126+0000 7f00ca570740 1 -- 172.21.15.45:0/4175618112 shutdown_connections 2024-04-16T17:14:41.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.126+0000 7f00ca570740 1 -- 172.21.15.45:0/4175618112 wait complete. 2024-04-16T17:14:41.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.126+0000 7f00ca570740 1 -- start start 2024-04-16T17:14:41.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.126+0000 7f00c5800640 1 --2- 172.21.15.45:0/1796674374 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f00a80a3d20 0x7f00a80a61e0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f00c006b440 tx=0x7f00b8009500 comp rx=0 tx=0).ready entity=mgr.25411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:14:41.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.126+0000 7f00ca570740 1 Processor -- start 2024-04-16T17:14:41.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.127+0000 7f00ca570740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ab65a1af10 0x55ab65a21500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:41.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.127+0000 7f00ca570740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ab65a1b830 0x55ab65a21a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:41.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.127+0000 7f00ca570740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ab65a24410 0x55ab65a21f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:41.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.127+0000 7f00ca570740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55ab65a271a0 con 0x55ab65a1af10 2024-04-16T17:14:41.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.127+0000 7f00ca570740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55ab65939830 con 0x55ab65a1b830 2024-04-16T17:14:41.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.127+0000 7f00ca570740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55ab65a27320 con 0x55ab65a24410 2024-04-16T17:14:41.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.127+0000 7f00ca570740 10 client.0 _ll_register_callbacks cb 0x55ab65941150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:14:41.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.127+0000 7f00c4e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ab65a1af10 0x55ab65a21500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:41.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.127+0000 7f00c5800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ab65a24410 0x55ab65a21f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:41.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.127+0000 7f00c4e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ab65a1af10 0x55ab65a21500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59294/0 (socket says 172.21.15.45:59294) 2024-04-16T17:14:41.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.127+0000 7f00c4e00640 1 -- 172.21.15.45:0/3847198030 learned_addr learned my addr 172.21.15.45:0/3847198030 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:14:41.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.127+0000 7f00bfe00640 1 --2- 172.21.15.45:0/3847198030 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ab65a1b830 0x55ab65a21a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:41.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.128+0000 7f00c5800640 1 -- 172.21.15.45:0/3847198030 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ab65a1b830 msgr2=0x55ab65a21a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:14:41.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.128+0000 7f00c5800640 1 --2- 172.21.15.45:0/3847198030 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ab65a1b830 0x55ab65a21a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:41.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.128+0000 7f00c5800640 1 -- 172.21.15.45:0/3847198030 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ab65a1af10 msgr2=0x55ab65a21500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:14:41.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.128+0000 7f00c5800640 1 --2- 172.21.15.45:0/3847198030 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ab65a1af10 0x55ab65a21500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:41.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.128+0000 7f00c5800640 1 -- 172.21.15.45:0/3847198030 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ab6593d920 con 0x55ab65a24410 2024-04-16T17:14:41.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.128+0000 7f00c5800640 1 --2- 172.21.15.45:0/3847198030 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ab65a24410 0x55ab65a21f80 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f00b8003f20 tx=0x7f00b8004ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5673aac52dd3ccde server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:14:41.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.128+0000 7f00a6a00640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-04-16T17:14:41.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.128+0000 7f00a6a00640 1 -- 172.21.15.45:0/3847198030 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00b800f750 con 0x55ab65a24410 2024-04-16T17:14:41.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.128+0000 7f00ca570740 1 -- 172.21.15.45:0/3847198030 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f00b4004030 con 0x55ab65a24410 2024-04-16T17:14:41.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.128+0000 7f00a6a00640 1 -- 172.21.15.45:0/3847198030 <== mon.1 v2:172.21.15.117:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9496+0+0 (secure 0 0 0) 0x7f00b800f8f0 con 0x55ab65a24410 2024-04-16T17:14:41.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.129+0000 7f00a6a00640 1 -- 172.21.15.45:0/3847198030 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f00b800faf0 con 0x55ab65a24410 2024-04-16T17:14:41.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.129+0000 7f00ca570740 10 client.27673 fetch_fsmap learned FSMap version 42 2024-04-16T17:14:41.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.129+0000 7f00ca570740 1 -- 172.21.15.45:0/3847198030 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55ab6590b7f0 con 0x55ab65a24410 2024-04-16T17:14:41.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.129+0000 7f00a6a00640 1 -- 172.21.15.45:0/3847198030 <== mon.1 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f00b4004030 con 0x55ab65a24410 2024-04-16T17:14:41.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.129+0000 7f00a6a00640 1 -- 172.21.15.45:0/3847198030 <== mon.1 v2:172.21.15.117:3300/0 5 ==== fsmap(e 42) ==== 8362+0+0 (secure 0 0 0) 0x7f00b8026070 con 0x55ab65a24410 2024-04-16T17:14:41.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.129+0000 7f00ca570740 10 client.27673 fetch_fsmap finished waiting for FSMap version 42 2024-04-16T17:14:41.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.129+0000 7f00ca570740 10 client.27673 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-16T17:14:41.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.129+0000 7f00ca570740 20 client.27673 populate_metadata read hostname 'smithi045' 2024-04-16T17:14:41.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.129+0000 7f00ca570740 1 --2- 172.21.15.45:0/3847198030 >> [v2:172.21.15.117:6838/2378675949,v1:172.21.15.117:6839/2378675949] conn(0x55ab65a348c0 0x55ab65a36d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:41.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.129+0000 7f00ca570740 4 client.27673 mds_command: new command op to 24532 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:14:41.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.129+0000 7f00ca570740 1 -- 172.21.15.45:0/3847198030 --> [v2:172.21.15.117:6838/2378675949,v1:172.21.15.117:6839/2378675949] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ab6598da40 con 0x55ab65a348c0 2024-04-16T17:14:41.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.130+0000 7f00c4e00640 1 --2- 172.21.15.45:0/3847198030 >> [v2:172.21.15.117:6838/2378675949,v1:172.21.15.117:6839/2378675949] conn(0x55ab65a348c0 0x55ab65a36d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:41.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.130+0000 7f00c4e00640 1 --2- 172.21.15.45:0/3847198030 >> [v2:172.21.15.117:6838/2378675949,v1:172.21.15.117:6839/2378675949] conn(0x55ab65a348c0 0x55ab65a36d20 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=37c9975448b800c server_cookie=80eb178d1104458a in_seq=0 out_seq=0 2024-04-16T17:14:41.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.130+0000 7f00a6a00640 10 client.27673 ms_handle_connect on v2:172.21.15.117:6838/2378675949 2024-04-16T17:14:41.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.131+0000 7f00a6a00640 1 -- 172.21.15.45:0/3847198030 <== mds.2 v2:172.21.15.117:6838/2378675949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ab6598da40 con 0x55ab65a348c0 2024-04-16T17:14:41.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.131+0000 7f00a6a00640 10 client.27673 handle_command_reply: tid=0 2024-04-16T17:14:41.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.131+0000 7f00ca570740 1 client.27673 shutdown 2024-04-16T17:14:41.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.131+0000 7f00ca570740 1 -- 172.21.15.45:0/3847198030 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ab65a24410 msgr2=0x55ab65a21f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:41.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.131+0000 7f00ca570740 1 --2- 172.21.15.45:0/3847198030 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ab65a24410 0x55ab65a21f80 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f00b8003f20 tx=0x7f00b8004ab0 comp rx=0 tx=0).stop 2024-04-16T17:14:41.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.131+0000 7f00ca570740 1 -- 172.21.15.45:0/3847198030 shutdown_connections 2024-04-16T17:14:41.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.131+0000 7f00ca570740 1 --2- 172.21.15.45:0/3847198030 >> [v2:172.21.15.117:6838/2378675949,v1:172.21.15.117:6839/2378675949] conn(0x55ab65a348c0 0x55ab65a36d20 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:41.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.131+0000 7f00ca570740 1 --2- 172.21.15.45:0/3847198030 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ab65a24410 0x55ab65a21f80 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:41.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.131+0000 7f00ca570740 1 --2- 172.21.15.45:0/3847198030 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ab65a1b830 0x55ab65a21a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:41.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.131+0000 7f00a6a00640 0 client.27673 ms_handle_reset on v2:172.21.15.117:6838/2378675949 2024-04-16T17:14:41.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.131+0000 7f00ca570740 1 --2- 172.21.15.45:0/3847198030 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ab65a1af10 0x55ab65a21500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:41.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.131+0000 7f00ca570740 1 -- 172.21.15.45:0/3847198030 >> 172.21.15.45:0/3847198030 conn(0x55ab65a15640 msgr2=0x55ab65a17390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:14:41.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.131+0000 7f00ca570740 1 -- 172.21.15.45:0/3847198030 shutdown_connections 2024-04-16T17:14:41.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.131+0000 7f00ca570740 1 -- 172.21.15.45:0/3847198030 wait complete. 2024-04-16T17:14:41.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.131+0000 7f00ca570740 20 client.27673 trim_cache size 0 max 16384 2024-04-16T17:14:41.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.146+0000 7f00ca570740 1 Processor -- start 2024-04-16T17:14:41.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.146+0000 7f00ca570740 1 -- start start 2024-04-16T17:14:41.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.146+0000 7f00ca570740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ab65a1af10 0x55ab65a23c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:41.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.147+0000 7f00ca570740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ab65a1b830 0x55ab65a30090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:41.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.147+0000 7f00ca570740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ab65a24410 0x55ab65a20660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:41.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.147+0000 7f00ca570740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f00c01b82e0 con 0x55ab65a1af10 2024-04-16T17:14:41.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.147+0000 7f00ca570740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f00c01b5b50 con 0x55ab65a24410 2024-04-16T17:14:41.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.147+0000 7f00ca570740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f00c01b39c0 con 0x55ab65a1b830 2024-04-16T17:14:41.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.147+0000 7f00c4e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ab65a1af10 0x55ab65a23c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:41.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.147+0000 7f00bfe00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ab65a1b830 0x55ab65a30090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:41.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.147+0000 7f00bfe00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ab65a1b830 0x55ab65a30090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:34272/0 (socket says 172.21.15.45:34272) 2024-04-16T17:14:41.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.147+0000 7f00bfe00640 1 -- 172.21.15.45:0/1054992287 learned_addr learned my addr 172.21.15.45:0/1054992287 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:14:41.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.147+0000 7f00c4e00640 1 --2- 172.21.15.45:0/1054992287 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ab65a1af10 0x55ab65a23c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59304/0 (socket says 172.21.15.45:59304) 2024-04-16T17:14:41.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.147+0000 7f00c5800640 1 --2- 172.21.15.45:0/1054992287 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ab65a24410 0x55ab65a20660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:41.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.147+0000 7f00bfe00640 1 -- 172.21.15.45:0/1054992287 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ab65a24410 msgr2=0x55ab65a20660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:14:41.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.147+0000 7f00bfe00640 1 --2- 172.21.15.45:0/1054992287 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ab65a24410 0x55ab65a20660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:41.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.147+0000 7f00bfe00640 1 -- 172.21.15.45:0/1054992287 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ab65a1af10 msgr2=0x55ab65a23c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:14:41.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.147+0000 7f00bfe00640 1 --2- 172.21.15.45:0/1054992287 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ab65a1af10 0x55ab65a23c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:41.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.147+0000 7f00bfe00640 1 -- 172.21.15.45:0/1054992287 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ab656923f0 con 0x55ab65a1b830 2024-04-16T17:14:41.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.147+0000 7f00c4e00640 1 --2- 172.21.15.45:0/1054992287 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ab65a1af10 0x55ab65a23c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:14:41.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.148+0000 7f00bfe00640 1 --2- 172.21.15.45:0/1054992287 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ab65a1b830 0x55ab65a30090 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f00ac00c6a0 tx=0x7f00ac09ccc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=966affc63bd2e8b7 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:14:41.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.148+0000 7f00b1600640 1 -- 172.21.15.45:0/1054992287 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00ac09ac20 con 0x55ab65a1b830 2024-04-16T17:14:41.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.148+0000 7f00b1600640 1 -- 172.21.15.45:0/1054992287 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f00ac09ae80 con 0x55ab65a1b830 2024-04-16T17:14:41.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.148+0000 7f00b1600640 1 -- 172.21.15.45:0/1054992287 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00ac018b00 con 0x55ab65a1b830 2024-04-16T17:14:41.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.149+0000 7f00ca570740 1 -- 172.21.15.45:0/1054992287 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ab65a1b830 msgr2=0x55ab65a30090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:41.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.149+0000 7f00ca570740 1 --2- 172.21.15.45:0/1054992287 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ab65a1b830 0x55ab65a30090 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f00ac00c6a0 tx=0x7f00ac09ccc0 comp rx=0 tx=0).stop 2024-04-16T17:14:41.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.149+0000 7f00ca570740 1 -- 172.21.15.45:0/1054992287 shutdown_connections 2024-04-16T17:14:41.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.149+0000 7f00ca570740 1 --2- 172.21.15.45:0/1054992287 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ab65a1b830 0x55ab65a30090 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:41.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.149+0000 7f00ca570740 1 --2- 172.21.15.45:0/1054992287 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ab65a24410 0x55ab65a20660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:41.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.149+0000 7f00ca570740 1 --2- 172.21.15.45:0/1054992287 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ab65a1af10 0x55ab65a23c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:41.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.149+0000 7f00ca570740 1 -- 172.21.15.45:0/1054992287 >> 172.21.15.45:0/1054992287 conn(0x55ab65a15640 msgr2=0x55ab65a13e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:14:41.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.149+0000 7f00ca570740 1 -- 172.21.15.45:0/1054992287 shutdown_connections 2024-04-16T17:14:41.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.149+0000 7f00ca570740 1 -- 172.21.15.45:0/1054992287 wait complete. 2024-04-16T17:14:41.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.149+0000 7f00ca570740 1 -- start start 2024-04-16T17:14:41.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.150+0000 7f00ca570740 1 Processor -- start 2024-04-16T17:14:41.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.150+0000 7f00ca570740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ab65a1af10 0x55ab65a2c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:41.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.150+0000 7f00ca570740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ab65a1b830 0x55ab65a1dd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:41.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.150+0000 7f00ca570740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ab65a24410 0x55ab65a1e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:41.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.150+0000 7f00ca570740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f00c01b4010 con 0x55ab65a1af10 2024-04-16T17:14:41.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.150+0000 7f00ca570740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f00c01b5480 con 0x55ab65a24410 2024-04-16T17:14:41.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.150+0000 7f00ca570740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f00c01b46e0 con 0x55ab65a1b830 2024-04-16T17:14:41.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.150+0000 7f00ca570740 10 client.0 _ll_register_callbacks cb 0x7f00b8011f90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:14:41.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.150+0000 7f00bfe00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ab65a1b830 0x55ab65a1dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:41.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.150+0000 7f00c4e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ab65a1af10 0x55ab65a2c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:41.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.150+0000 7f00bfe00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ab65a1b830 0x55ab65a1dd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:34280/0 (socket says 172.21.15.45:34280) 2024-04-16T17:14:41.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.150+0000 7f00bfe00640 1 -- 172.21.15.45:0/3479463433 learned_addr learned my addr 172.21.15.45:0/3479463433 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:14:41.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.150+0000 7f00c5800640 1 --2- 172.21.15.45:0/3479463433 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ab65a24410 0x55ab65a1e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:41.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.151+0000 7f00c4e00640 1 -- 172.21.15.45:0/3479463433 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ab65a1b830 msgr2=0x55ab65a1dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:14:41.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.151+0000 7f00c4e00640 1 --2- 172.21.15.45:0/3479463433 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ab65a1b830 0x55ab65a1dd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:41.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.151+0000 7f00c4e00640 1 -- 172.21.15.45:0/3479463433 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ab65a24410 msgr2=0x55ab65a1e2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:14:41.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.151+0000 7f00c4e00640 1 --2- 172.21.15.45:0/3479463433 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ab65a24410 0x55ab65a1e2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:41.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.151+0000 7f00c4e00640 1 -- 172.21.15.45:0/3479463433 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ab6575c7d0 con 0x55ab65a1af10 2024-04-16T17:14:41.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.151+0000 7f00c5800640 1 --2- 172.21.15.45:0/3479463433 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ab65a24410 0x55ab65a1e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:14:41.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.151+0000 7f00bfe00640 1 --2- 172.21.15.45:0/3479463433 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ab65a1b830 0x55ab65a1dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:14:41.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.151+0000 7f00c4e00640 1 --2- 172.21.15.45:0/3479463433 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ab65a1af10 0x55ab65a2c800 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x55ab65a37530 tx=0x7f00b402bc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6faf932d21124cd0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:14:41.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.151+0000 7f00a6a00640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-16T17:14:41.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.151+0000 7f00a6a00640 1 -- 172.21.15.45:0/3479463433 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00b402cc80 con 0x55ab65a1af10 2024-04-16T17:14:41.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.151+0000 7f00ca570740 1 -- 172.21.15.45:0/3479463433 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f00ac09ae80 con 0x55ab65a1af10 2024-04-16T17:14:41.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.151+0000 7f00a6a00640 1 -- 172.21.15.45:0/3479463433 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9496+0+0 (secure 0 0 0) 0x7f00b4004040 con 0x55ab65a1af10 2024-04-16T17:14:41.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.151+0000 7f00a6a00640 1 -- 172.21.15.45:0/3479463433 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f00b402c680 con 0x55ab65a1af10 2024-04-16T17:14:41.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.151+0000 7f00ca570740 10 client.17577 fetch_fsmap learned FSMap version 42 2024-04-16T17:14:41.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.152+0000 7f00ca570740 1 -- 172.21.15.45:0/3479463433 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55ab65a9ab20 con 0x55ab65a1af10 2024-04-16T17:14:41.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.152+0000 7f00a6a00640 1 -- 172.21.15.45:0/3479463433 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f00ac09ae80 con 0x55ab65a1af10 2024-04-16T17:14:41.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.152+0000 7f00a6a00640 1 -- 172.21.15.45:0/3479463433 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 42) ==== 8362+0+0 (secure 0 0 0) 0x7f00b4003680 con 0x55ab65a1af10 2024-04-16T17:14:41.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.152+0000 7f00ca570740 10 client.17577 fetch_fsmap finished waiting for FSMap version 42 2024-04-16T17:14:41.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.152+0000 7f00ca570740 10 client.17577 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-16T17:14:41.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.152+0000 7f00ca570740 20 client.17577 populate_metadata read hostname 'smithi045' 2024-04-16T17:14:41.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.152+0000 7f00ca570740 1 --2- 172.21.15.45:0/3479463433 >> [v2:172.21.15.117:6838/2378675949,v1:172.21.15.117:6839/2378675949] conn(0x55ab65a9aeb0 0x55ab65a9d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:41.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.152+0000 7f00ca570740 4 client.17577 mds_command: new command op to 24532 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T17:14:41.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.152+0000 7f00ca570740 1 -- 172.21.15.45:0/3479463433 --> [v2:172.21.15.117:6838/2378675949,v1:172.21.15.117:6839/2378675949] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ab659878b0 con 0x55ab65a9aeb0 2024-04-16T17:14:41.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.153+0000 7f00bfe00640 1 --2- 172.21.15.45:0/3479463433 >> [v2:172.21.15.117:6838/2378675949,v1:172.21.15.117:6839/2378675949] conn(0x55ab65a9aeb0 0x55ab65a9d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:41.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.153+0000 7f00bfe00640 1 --2- 172.21.15.45:0/3479463433 >> [v2:172.21.15.117:6838/2378675949,v1:172.21.15.117:6839/2378675949] conn(0x55ab65a9aeb0 0x55ab65a9d2a0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=59154fdea8099570 server_cookie=e4d0df5486ac88c5 in_seq=0 out_seq=0 2024-04-16T17:14:41.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.153+0000 7f00a6a00640 10 client.17577 ms_handle_connect on v2:172.21.15.117:6838/2378675949 2024-04-16T17:14:41.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.153+0000 7f00a6a00640 1 -- 172.21.15.45:0/3479463433 <== mds.2 v2:172.21.15.117:6838/2378675949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ab659878b0 con 0x55ab65a9aeb0 2024-04-16T17:14:41.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.153+0000 7f00a6a00640 10 client.17577 handle_command_reply: tid=0 2024-04-16T17:14:41.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.153+0000 7f00ca570740 1 client.17577 shutdown 2024-04-16T17:14:41.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.154+0000 7f00ca570740 1 -- 172.21.15.45:0/3479463433 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ab65a1af10 msgr2=0x55ab65a2c800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:41.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.154+0000 7f00ca570740 1 --2- 172.21.15.45:0/3479463433 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ab65a1af10 0x55ab65a2c800 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x55ab65a37530 tx=0x7f00b402bc20 comp rx=0 tx=0).stop 2024-04-16T17:14:41.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.154+0000 7f00ca570740 1 -- 172.21.15.45:0/3479463433 shutdown_connections 2024-04-16T17:14:41.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.154+0000 7f00ca570740 1 --2- 172.21.15.45:0/3479463433 >> [v2:172.21.15.117:6838/2378675949,v1:172.21.15.117:6839/2378675949] conn(0x55ab65a9aeb0 0x55ab65a9d2a0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:41.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.154+0000 7f00ca570740 1 --2- 172.21.15.45:0/3479463433 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ab65a1b830 0x55ab65a1dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:41.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.154+0000 7f00ca570740 1 --2- 172.21.15.45:0/3479463433 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ab65a24410 0x55ab65a1e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:41.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.154+0000 7f00a6a00640 0 client.17577 ms_handle_reset on v2:172.21.15.117:6838/2378675949 2024-04-16T17:14:41.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.154+0000 7f00ca570740 1 --2- 172.21.15.45:0/3479463433 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ab65a1af10 0x55ab65a2c800 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:41.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.154+0000 7f00ca570740 1 -- 172.21.15.45:0/3479463433 >> 172.21.15.45:0/3479463433 conn(0x55ab65a15640 msgr2=0x55ab65a13e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:14:41.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.154+0000 7f00ca570740 1 -- 172.21.15.45:0/3479463433 shutdown_connections 2024-04-16T17:14:41.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.154+0000 7f00ca570740 1 -- 172.21.15.45:0/3479463433 wait complete. 2024-04-16T17:14:41.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.154+0000 7f00ca570740 20 client.17577 trim_cache size 0 max 16384 2024-04-16T17:14:41.167 INFO:teuthology.orchestra.run.smithi045.stdout:[] 2024-04-16T17:14:41.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.155+0000 7f00c6200640 1 -- 172.21.15.45:0/1796674374 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f00a80a3d20 msgr2=0x7f00a80a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:41.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.155+0000 7f00c6200640 1 --2- 172.21.15.45:0/1796674374 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f00a80a3d20 0x7f00a80a61e0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f00c006b440 tx=0x7f00b8009500 comp rx=0 tx=0).stop 2024-04-16T17:14:41.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.155+0000 7f00c6200640 1 -- 172.21.15.45:0/1796674374 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f00c0076e50 msgr2=0x7f00c019f330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:41.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.155+0000 7f00c6200640 1 --2- 172.21.15.45:0/1796674374 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f00c0076e50 0x7f00c019f330 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f00ac00e9f0 tx=0x7f00ac00eeb0 comp rx=0 tx=0).stop 2024-04-16T17:14:41.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.155+0000 7f00c6200640 1 -- 172.21.15.45:0/1796674374 shutdown_connections 2024-04-16T17:14:41.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.155+0000 7f00c6200640 1 --2- 172.21.15.45:0/1796674374 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f00a80a3d20 0x7f00a80a61e0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:41.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.155+0000 7f00c6200640 1 --2- 172.21.15.45:0/1796674374 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f00c0076e50 0x7f00c019f330 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:41.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.155+0000 7f00c6200640 1 --2- 172.21.15.45:0/1796674374 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f00c0112d40 0x7f00c01a3660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:41.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.155+0000 7f00c6200640 1 --2- 172.21.15.45:0/1796674374 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f00c0076530 0x7f00c019edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:41.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.155+0000 7f00c6200640 1 -- 172.21.15.45:0/1796674374 >> 172.21.15.45:0/1796674374 conn(0x7f00c00ffb10 msgr2=0x7f00c0114400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:14:41.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.155+0000 7f00c6200640 1 -- 172.21.15.45:0/1796674374 shutdown_connections 2024-04-16T17:14:41.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.155+0000 7f00c6200640 1 -- 172.21.15.45:0/1796674374 wait complete. 2024-04-16T17:14:41.173 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-16T17:14:41.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.379+0000 7f7095800640 1 -- 172.21.15.45:0/3811298907 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7090112d40 msgr2=0x7f7090115130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:41.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.379+0000 7f7095800640 1 --2- 172.21.15.45:0/3811298907 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7090112d40 0x7f7090115130 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f7088009ca0 tx=0x7f708802c100 comp rx=0 tx=0).stop 2024-04-16T17:14:41.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.379+0000 7f7095800640 1 -- 172.21.15.45:0/3811298907 shutdown_connections 2024-04-16T17:14:41.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.379+0000 7f7095800640 1 --2- 172.21.15.45:0/3811298907 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7090112d40 0x7f7090115130 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:41.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.379+0000 7f7095800640 1 --2- 172.21.15.45:0/3811298907 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7090076e50 0x7f7090112800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:41.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.379+0000 7f7095800640 1 --2- 172.21.15.45:0/3811298907 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7090076530 0x7f7090076910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:41.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.379+0000 7f7095800640 1 -- 172.21.15.45:0/3811298907 >> 172.21.15.45:0/3811298907 conn(0x7f70900ffb10 msgr2=0x7f7090101f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:14:41.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.379+0000 7f7095800640 1 -- 172.21.15.45:0/3811298907 shutdown_connections 2024-04-16T17:14:41.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.379+0000 7f7095800640 1 -- 172.21.15.45:0/3811298907 wait complete. 2024-04-16T17:14:41.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.380+0000 7f7095800640 1 Processor -- start 2024-04-16T17:14:41.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.380+0000 7f7095800640 1 -- start start 2024-04-16T17:14:41.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.381+0000 7f7095800640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7090076530 0x7f709019ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:41.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.381+0000 7f7095800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7090076e50 0x7f709019f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:41.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.381+0000 7f7095800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7090112d40 0x7f70901a35f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:41.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.381+0000 7f7095800640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f7090118860 con 0x7f7090112d40 2024-04-16T17:14:41.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.381+0000 7f7095800640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f70901186e0 con 0x7f7090076530 2024-04-16T17:14:41.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.381+0000 7f7095800640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f70901189e0 con 0x7f7090076e50 2024-04-16T17:14:41.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.381+0000 7f7094e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7090112d40 0x7f70901a35f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:41.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.381+0000 7f708fe00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7090076530 0x7f709019ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:41.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.381+0000 7f7094e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7090112d40 0x7f70901a35f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59330/0 (socket says 172.21.15.45:59330) 2024-04-16T17:14:41.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.381+0000 7f7094e00640 1 -- 172.21.15.45:0/630618113 learned_addr learned my addr 172.21.15.45:0/630618113 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:14:41.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.381+0000 7f708f400640 1 --2- 172.21.15.45:0/630618113 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7090076e50 0x7f709019f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:41.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.382+0000 7f708fe00640 1 -- 172.21.15.45:0/630618113 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7090076e50 msgr2=0x7f709019f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:41.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.382+0000 7f708fe00640 1 --2- 172.21.15.45:0/630618113 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7090076e50 0x7f709019f2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:41.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.382+0000 7f708fe00640 1 -- 172.21.15.45:0/630618113 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7090112d40 msgr2=0x7f70901a35f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:41.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.382+0000 7f708fe00640 1 --2- 172.21.15.45:0/630618113 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7090112d40 0x7f70901a35f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:41.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.382+0000 7f708fe00640 1 -- 172.21.15.45:0/630618113 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70901a3d20 con 0x7f7090076530 2024-04-16T17:14:41.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.382+0000 7f7094e00640 1 --2- 172.21.15.45:0/630618113 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7090112d40 0x7f70901a35f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:14:41.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.382+0000 7f708fe00640 1 --2- 172.21.15.45:0/630618113 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7090076530 0x7f709019ed80 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f708000e9f0 tx=0x7f708000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:14:41.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.382+0000 7f708cc00640 1 -- 172.21.15.45:0/630618113 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f708000ccc0 con 0x7f7090076530 2024-04-16T17:14:41.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.382+0000 7f7095800640 1 -- 172.21.15.45:0/630618113 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70901a4010 con 0x7f7090076530 2024-04-16T17:14:41.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.382+0000 7f708cc00640 1 -- 172.21.15.45:0/630618113 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7080010910 con 0x7f7090076530 2024-04-16T17:14:41.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.382+0000 7f708cc00640 1 -- 172.21.15.45:0/630618113 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7080018960 con 0x7f7090076530 2024-04-16T17:14:41.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.382+0000 7f7095800640 1 -- 172.21.15.45:0/630618113 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70901ab8a0 con 0x7f7090076530 2024-04-16T17:14:41.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.385+0000 7f7099bad740 1 Processor -- start 2024-04-16T17:14:41.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.386+0000 7f7099bad740 1 -- start start 2024-04-16T17:14:41.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.386+0000 7f708cc00640 1 -- 172.21.15.45:0/630618113 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f7080010ac0 con 0x7f7090076530 2024-04-16T17:14:41.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.386+0000 7f7099bad740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d21142ff10 0x55d2114302f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:41.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.386+0000 7f7099bad740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d211430830 0x55d211438ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:41.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.386+0000 7f7099bad740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d211439410 0x55d21143b800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:41.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.386+0000 7f7099bad740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55d211395020 con 0x55d211430830 2024-04-16T17:14:41.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.386+0000 7f7099bad740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55d21133b460 con 0x55d21142ff10 2024-04-16T17:14:41.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.386+0000 7f7099bad740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55d211390fe0 con 0x55d211439410 2024-04-16T17:14:41.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.386+0000 7f708fe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d211430830 0x55d211438ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:41.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.386+0000 7f7094e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d21142ff10 0x55d2114302f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:41.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.386+0000 7f708fe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d211430830 0x55d211438ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59344/0 (socket says 172.21.15.45:59344) 2024-04-16T17:14:41.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.386+0000 7f708fe00640 1 -- 172.21.15.45:0/2629701910 learned_addr learned my addr 172.21.15.45:0/2629701910 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:14:41.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.386+0000 7f708f400640 1 --2- 172.21.15.45:0/2629701910 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d211439410 0x55d21143b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:41.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.387+0000 7f7094e00640 1 -- 172.21.15.45:0/2629701910 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d211439410 msgr2=0x55d21143b800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:14:41.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.387+0000 7f7094e00640 1 --2- 172.21.15.45:0/2629701910 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d211439410 0x55d21143b800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:41.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.387+0000 7f7094e00640 1 -- 172.21.15.45:0/2629701910 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d211430830 msgr2=0x55d211438ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:14:41.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.387+0000 7f7094e00640 1 --2- 172.21.15.45:0/2629701910 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d211430830 0x55d211438ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:41.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.387+0000 7f7094e00640 1 -- 172.21.15.45:0/2629701910 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d211343040 con 0x55d21142ff10 2024-04-16T17:14:41.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.387+0000 7f7094e00640 1 --2- 172.21.15.45:0/2629701910 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d21142ff10 0x55d2114302f0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f7088004af0 tx=0x7f7088005f80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2eb83332437215ac server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:14:41.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.387+0000 7f7085600640 1 -- 172.21.15.45:0/2629701910 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7088045020 con 0x55d21142ff10 2024-04-16T17:14:41.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.387+0000 7f7085600640 1 -- 172.21.15.45:0/2629701910 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f708803a040 con 0x55d21142ff10 2024-04-16T17:14:41.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.387+0000 7f7085600640 1 -- 172.21.15.45:0/2629701910 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7088039040 con 0x55d21142ff10 2024-04-16T17:14:41.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.388+0000 7f708cc00640 1 --2- 172.21.15.45:0/630618113 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f70740a3d20 0x7f70740a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:41.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.388+0000 7f7099bad740 1 -- 172.21.15.45:0/2629701910 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d21142ff10 msgr2=0x55d2114302f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:41.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.388+0000 7f7099bad740 1 --2- 172.21.15.45:0/2629701910 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d21142ff10 0x55d2114302f0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f7088004af0 tx=0x7f7088005f80 comp rx=0 tx=0).stop 2024-04-16T17:14:41.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.388+0000 7f708cc00640 1 -- 172.21.15.45:0/630618113 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9496+0+0 (secure 0 0 0) 0x7f70800d0f60 con 0x7f7090076530 2024-04-16T17:14:41.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.388+0000 7f708fe00640 1 --2- 172.21.15.45:0/630618113 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f70740a3d20 0x7f70740a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:41.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.388+0000 7f7099bad740 1 -- 172.21.15.45:0/2629701910 shutdown_connections 2024-04-16T17:14:41.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.388+0000 7f7099bad740 1 --2- 172.21.15.45:0/2629701910 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d211439410 0x55d21143b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:41.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.388+0000 7f7099bad740 1 --2- 172.21.15.45:0/2629701910 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d211430830 0x55d211438ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:41.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.388+0000 7f7099bad740 1 --2- 172.21.15.45:0/2629701910 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d21142ff10 0x55d2114302f0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:41.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.388+0000 7f7099bad740 1 -- 172.21.15.45:0/2629701910 >> 172.21.15.45:0/2629701910 conn(0x55d21142a640 msgr2=0x55d21142aa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:14:41.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.388+0000 7f7099bad740 1 -- 172.21.15.45:0/2629701910 shutdown_connections 2024-04-16T17:14:41.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.388+0000 7f7099bad740 1 -- 172.21.15.45:0/2629701910 wait complete. 2024-04-16T17:14:41.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.389+0000 7f708fe00640 1 --2- 172.21.15.45:0/630618113 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f70740a3d20 0x7f70740a61e0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f708000e9c0 tx=0x7f70800d4a10 comp rx=0 tx=0).ready entity=mgr.25411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:14:41.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.389+0000 7f7099bad740 1 -- start start 2024-04-16T17:14:41.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.389+0000 7f7099bad740 1 Processor -- start 2024-04-16T17:14:41.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.389+0000 7f7099bad740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d21142ff10 0x55d211436510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:41.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.389+0000 7f7099bad740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d211430830 0x55d211436a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:41.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.389+0000 7f7099bad740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d211439410 0x55d211436f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:41.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.389+0000 7f7099bad740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55d21143c1a0 con 0x55d211439410 2024-04-16T17:14:41.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.390+0000 7f7099bad740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55d21134e830 con 0x55d211430830 2024-04-16T17:14:41.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.390+0000 7f7099bad740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55d21143c320 con 0x55d21142ff10 2024-04-16T17:14:41.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.390+0000 7f7099bad740 10 client.0 _ll_register_callbacks cb 0x55d211356150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:14:41.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.390+0000 7f708f400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d211430830 0x55d211436a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:41.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.390+0000 7f708f400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d211430830 0x55d211436a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:46596/0 (socket says 172.21.15.45:46596) 2024-04-16T17:14:41.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.390+0000 7f708f400640 1 -- 172.21.15.45:0/1536800082 learned_addr learned my addr 172.21.15.45:0/1536800082 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:14:41.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.390+0000 7f7094e00640 1 --2- 172.21.15.45:0/1536800082 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d21142ff10 0x55d211436510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:41.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.390+0000 7f7094e00640 1 --2- 172.21.15.45:0/1536800082 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d211439410 0x55d211436f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:41.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.390+0000 7f708f400640 1 -- 172.21.15.45:0/1536800082 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d21142ff10 msgr2=0x55d211436510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:14:41.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.390+0000 7f708f400640 1 --2- 172.21.15.45:0/1536800082 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d21142ff10 0x55d211436510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:41.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.390+0000 7f708f400640 1 -- 172.21.15.45:0/1536800082 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d211439410 msgr2=0x55d211436f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:14:41.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.390+0000 7f708f400640 1 --2- 172.21.15.45:0/1536800082 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d211439410 0x55d211436f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:41.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.390+0000 7f708f400640 1 -- 172.21.15.45:0/1536800082 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d211352920 con 0x55d211430830 2024-04-16T17:14:41.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.390+0000 7f708f400640 1 --2- 172.21.15.45:0/1536800082 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d211430830 0x55d211436a50 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f707c012700 tx=0x7f707c012bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fd47337f25f2f74b server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:14:41.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.390+0000 7f707a000640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-04-16T17:14:41.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.391+0000 7f707a000640 1 -- 172.21.15.45:0/1536800082 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f707c00f8c0 con 0x55d211430830 2024-04-16T17:14:41.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.391+0000 7f7099bad740 1 -- 172.21.15.45:0/1536800082 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f708803a040 con 0x55d211430830 2024-04-16T17:14:41.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.391+0000 7f707a000640 1 -- 172.21.15.45:0/1536800082 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9496+0+0 (secure 0 0 0) 0x7f707c00fa60 con 0x55d211430830 2024-04-16T17:14:41.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.391+0000 7f707a000640 1 -- 172.21.15.45:0/1536800082 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f707c00fca0 con 0x55d211430830 2024-04-16T17:14:41.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.391+0000 7f7099bad740 10 client.27749 fetch_fsmap learned FSMap version 42 2024-04-16T17:14:41.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.391+0000 7f7099bad740 1 -- 172.21.15.45:0/1536800082 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55d2113207f0 con 0x55d211430830 2024-04-16T17:14:41.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.391+0000 7f707a000640 1 -- 172.21.15.45:0/1536800082 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f708803a040 con 0x55d211430830 2024-04-16T17:14:41.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.392+0000 7f707a000640 1 -- 172.21.15.45:0/1536800082 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 42) ==== 8362+0+0 (secure 0 0 0) 0x7f707c01c420 con 0x55d211430830 2024-04-16T17:14:41.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.392+0000 7f7099bad740 10 client.27749 fetch_fsmap finished waiting for FSMap version 42 2024-04-16T17:14:41.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.392+0000 7f7099bad740 10 client.27749 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-04-16T17:14:41.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.392+0000 7f7099bad740 20 client.27749 populate_metadata read hostname 'smithi045' 2024-04-16T17:14:41.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.392+0000 7f7099bad740 1 --2- 172.21.15.45:0/1536800082 >> [v2:172.21.15.45:6840/3755310822,v1:172.21.15.45:6841/3755310822] conn(0x55d211449840 0x55d21144bc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:41.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.392+0000 7f7099bad740 4 client.27749 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:14:41.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.392+0000 7f7099bad740 1 -- 172.21.15.45:0/1536800082 --> [v2:172.21.15.45:6840/3755310822,v1:172.21.15.45:6841/3755310822] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d2113a2a40 con 0x55d211449840 2024-04-16T17:14:41.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.392+0000 7f7094e00640 1 --2- 172.21.15.45:0/1536800082 >> [v2:172.21.15.45:6840/3755310822,v1:172.21.15.45:6841/3755310822] conn(0x55d211449840 0x55d21144bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:41.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.393+0000 7f7094e00640 1 --2- 172.21.15.45:0/1536800082 >> [v2:172.21.15.45:6840/3755310822,v1:172.21.15.45:6841/3755310822] conn(0x55d211449840 0x55d21144bc50 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=97ca7592fab29cbc server_cookie=bdc2f2bd9c9434c3 in_seq=0 out_seq=0 2024-04-16T17:14:41.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.393+0000 7f707a000640 10 client.27749 ms_handle_connect on v2:172.21.15.45:6840/3755310822 2024-04-16T17:14:41.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.393+0000 7f707a000640 1 -- 172.21.15.45:0/1536800082 <== mds.1 v2:172.21.15.45:6840/3755310822 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d2113a2a40 con 0x55d211449840 2024-04-16T17:14:41.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.393+0000 7f707a000640 10 client.27749 handle_command_reply: tid=0 2024-04-16T17:14:41.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.393+0000 7f7099bad740 1 client.27749 shutdown 2024-04-16T17:14:41.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.394+0000 7f7099bad740 1 -- 172.21.15.45:0/1536800082 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d211430830 msgr2=0x55d211436a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:41.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.394+0000 7f7099bad740 1 --2- 172.21.15.45:0/1536800082 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d211430830 0x55d211436a50 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f707c012700 tx=0x7f707c012bc0 comp rx=0 tx=0).stop 2024-04-16T17:14:41.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.394+0000 7f7099bad740 1 -- 172.21.15.45:0/1536800082 shutdown_connections 2024-04-16T17:14:41.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.394+0000 7f7099bad740 1 --2- 172.21.15.45:0/1536800082 >> [v2:172.21.15.45:6840/3755310822,v1:172.21.15.45:6841/3755310822] conn(0x55d211449840 0x55d21144bc50 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:41.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.394+0000 7f7099bad740 1 --2- 172.21.15.45:0/1536800082 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d211439410 0x55d211436f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:41.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.394+0000 7f7099bad740 1 --2- 172.21.15.45:0/1536800082 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d211430830 0x55d211436a50 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:41.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.394+0000 7f707a000640 0 client.27749 ms_handle_reset on v2:172.21.15.45:6840/3755310822 2024-04-16T17:14:41.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.394+0000 7f7099bad740 1 --2- 172.21.15.45:0/1536800082 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d21142ff10 0x55d211436510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:41.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.394+0000 7f7099bad740 1 -- 172.21.15.45:0/1536800082 >> 172.21.15.45:0/1536800082 conn(0x55d21142a640 msgr2=0x55d21142c390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:14:41.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.394+0000 7f7099bad740 1 -- 172.21.15.45:0/1536800082 shutdown_connections 2024-04-16T17:14:41.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.394+0000 7f7099bad740 1 -- 172.21.15.45:0/1536800082 wait complete. 2024-04-16T17:14:41.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.394+0000 7f7099bad740 20 client.27749 trim_cache size 0 max 16384 2024-04-16T17:14:41.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.413+0000 7f7099bad740 1 Processor -- start 2024-04-16T17:14:41.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.414+0000 7f7099bad740 1 -- start start 2024-04-16T17:14:41.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.414+0000 7f7099bad740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d21142ff10 0x55d211445090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:41.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.414+0000 7f7099bad740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d211430830 0x55d211435690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:41.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.414+0000 7f7099bad740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d211439410 0x55d211436db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:41.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.414+0000 7f7099bad740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f70901b82c0 con 0x55d21142ff10 2024-04-16T17:14:41.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.414+0000 7f7099bad740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f70901b5b30 con 0x55d211439410 2024-04-16T17:14:41.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.414+0000 7f7099bad740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f70901b39a0 con 0x55d211430830 2024-04-16T17:14:41.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.414+0000 7f7094e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d21142ff10 0x55d211445090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:41.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.414+0000 7f7094e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d21142ff10 0x55d211445090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59366/0 (socket says 172.21.15.45:59366) 2024-04-16T17:14:41.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.414+0000 7f7094e00640 1 -- 172.21.15.45:0/1808498036 learned_addr learned my addr 172.21.15.45:0/1808498036 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:14:41.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.414+0000 7f708f400640 1 --2- 172.21.15.45:0/1808498036 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d211430830 0x55d211435690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:41.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.415+0000 7f7094e00640 1 --2- 172.21.15.45:0/1808498036 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d211439410 0x55d211436db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:41.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.415+0000 7f7094e00640 1 -- 172.21.15.45:0/1808498036 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d211430830 msgr2=0x55d211435690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:14:41.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.415+0000 7f7094e00640 1 --2- 172.21.15.45:0/1808498036 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d211430830 0x55d211435690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:41.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.415+0000 7f7094e00640 1 -- 172.21.15.45:0/1808498036 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d211439410 msgr2=0x55d211436db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:14:41.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.415+0000 7f7094e00640 1 --2- 172.21.15.45:0/1808498036 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d211439410 0x55d211436db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:41.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.415+0000 7f7094e00640 1 -- 172.21.15.45:0/1808498036 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d211430660 con 0x55d21142ff10 2024-04-16T17:14:41.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.415+0000 7f708f400640 1 --2- 172.21.15.45:0/1808498036 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d211430830 0x55d211435690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:14:41.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.415+0000 7f7094e00640 1 --2- 172.21.15.45:0/1808498036 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d21142ff10 0x55d211445090 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f7088037040 tx=0x7f7088034710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9711713817fcc8b0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:14:41.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.415+0000 7f7084c00640 1 -- 172.21.15.45:0/1808498036 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7088039040 con 0x55d21142ff10 2024-04-16T17:14:41.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.415+0000 7f7084c00640 1 -- 172.21.15.45:0/1808498036 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7088003970 con 0x55d21142ff10 2024-04-16T17:14:41.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.415+0000 7f7084c00640 1 -- 172.21.15.45:0/1808498036 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7088045040 con 0x55d21142ff10 2024-04-16T17:14:41.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.416+0000 7f7099bad740 1 -- 172.21.15.45:0/1808498036 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d21142ff10 msgr2=0x55d211445090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:41.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.416+0000 7f7099bad740 1 --2- 172.21.15.45:0/1808498036 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d21142ff10 0x55d211445090 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f7088037040 tx=0x7f7088034710 comp rx=0 tx=0).stop 2024-04-16T17:14:41.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.416+0000 7f7099bad740 1 -- 172.21.15.45:0/1808498036 shutdown_connections 2024-04-16T17:14:41.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.416+0000 7f7099bad740 1 --2- 172.21.15.45:0/1808498036 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d211430830 0x55d211435690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:41.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.416+0000 7f7099bad740 1 --2- 172.21.15.45:0/1808498036 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d211439410 0x55d211436db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:41.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.416+0000 7f7099bad740 1 --2- 172.21.15.45:0/1808498036 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d21142ff10 0x55d211445090 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:41.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.416+0000 7f7099bad740 1 -- 172.21.15.45:0/1808498036 >> 172.21.15.45:0/1808498036 conn(0x55d21142a640 msgr2=0x55d211428e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:14:41.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.416+0000 7f7099bad740 1 -- 172.21.15.45:0/1808498036 shutdown_connections 2024-04-16T17:14:41.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.416+0000 7f7099bad740 1 -- 172.21.15.45:0/1808498036 wait complete. 2024-04-16T17:14:41.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.417+0000 7f7099bad740 1 -- start start 2024-04-16T17:14:41.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.417+0000 7f7099bad740 1 Processor -- start 2024-04-16T17:14:41.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.417+0000 7f7099bad740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d21142ff10 0x55d2114415a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:41.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.417+0000 7f7099bad740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d211430830 0x55d211441ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:41.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.417+0000 7f7099bad740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d211439410 0x55d211432980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:41.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.417+0000 7f7099bad740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f70901b3ff0 con 0x55d21142ff10 2024-04-16T17:14:41.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.417+0000 7f7099bad740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f70901b5460 con 0x55d211430830 2024-04-16T17:14:41.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.417+0000 7f7099bad740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f70901b46c0 con 0x55d211439410 2024-04-16T17:14:41.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.417+0000 7f7099bad740 10 client.0 _ll_register_callbacks cb 0x7f707c009850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:14:41.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.417+0000 7f7094e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d21142ff10 0x55d2114415a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:41.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.417+0000 7f7094e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d21142ff10 0x55d2114415a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59368/0 (socket says 172.21.15.45:59368) 2024-04-16T17:14:41.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.417+0000 7f7094e00640 1 -- 172.21.15.45:0/3965940528 learned_addr learned my addr 172.21.15.45:0/3965940528 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:14:41.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.418+0000 7f708f400640 1 --2- 172.21.15.45:0/3965940528 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d211430830 0x55d211441ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:41.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.418+0000 7f7094e00640 1 --2- 172.21.15.45:0/3965940528 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d211439410 0x55d211432980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:41.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.418+0000 7f7094e00640 1 -- 172.21.15.45:0/3965940528 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d211439410 msgr2=0x55d211432980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:14:41.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.418+0000 7f7094e00640 1 --2- 172.21.15.45:0/3965940528 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d211439410 0x55d211432980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:41.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.418+0000 7f7094e00640 1 -- 172.21.15.45:0/3965940528 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d211430830 msgr2=0x55d211441ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:14:41.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.418+0000 7f7094e00640 1 --2- 172.21.15.45:0/3965940528 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d211430830 0x55d211441ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:41.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.418+0000 7f7094e00640 1 -- 172.21.15.45:0/3965940528 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d2110a73f0 con 0x55d21142ff10 2024-04-16T17:14:41.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.418+0000 7f708f400640 1 --2- 172.21.15.45:0/3965940528 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d211430830 0x55d211441ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:14:41.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.418+0000 7f7094e00640 1 --2- 172.21.15.45:0/3965940528 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d21142ff10 0x55d2114415a0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f7088009620 tx=0x7f7088034a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e10f99c00a3ada5 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:14:41.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.418+0000 7f707a000640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-16T17:14:41.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.418+0000 7f707a000640 1 -- 172.21.15.45:0/3965940528 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7088063060 con 0x55d21142ff10 2024-04-16T17:14:41.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.418+0000 7f707a000640 1 -- 172.21.15.45:0/3965940528 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9496+0+0 (secure 0 0 0) 0x7f7088004d40 con 0x55d21142ff10 2024-04-16T17:14:41.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.418+0000 7f7099bad740 1 -- 172.21.15.45:0/3965940528 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7088003970 con 0x55d21142ff10 2024-04-16T17:14:41.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.419+0000 7f707a000640 1 -- 172.21.15.45:0/3965940528 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f708806a050 con 0x55d21142ff10 2024-04-16T17:14:41.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.419+0000 7f7099bad740 10 client.17595 fetch_fsmap learned FSMap version 42 2024-04-16T17:14:41.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.419+0000 7f7099bad740 1 -- 172.21.15.45:0/3965940528 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55d2111717d0 con 0x55d21142ff10 2024-04-16T17:14:41.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.419+0000 7f707a000640 1 -- 172.21.15.45:0/3965940528 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7088003970 con 0x55d21142ff10 2024-04-16T17:14:41.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.419+0000 7f707a000640 1 -- 172.21.15.45:0/3965940528 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 42) ==== 8362+0+0 (secure 0 0 0) 0x7f7088041430 con 0x55d21142ff10 2024-04-16T17:14:41.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.419+0000 7f7099bad740 10 client.17595 fetch_fsmap finished waiting for FSMap version 42 2024-04-16T17:14:41.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.419+0000 7f7099bad740 10 client.17595 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-04-16T17:14:41.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.419+0000 7f7099bad740 20 client.17595 populate_metadata read hostname 'smithi045' 2024-04-16T17:14:41.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.419+0000 7f7099bad740 1 --2- 172.21.15.45:0/3965940528 >> [v2:172.21.15.45:6840/3755310822,v1:172.21.15.45:6841/3755310822] conn(0x55d2114af730 0x55d2114b1b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:41.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.419+0000 7f7099bad740 4 client.17595 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T17:14:41.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.419+0000 7f7099bad740 1 -- 172.21.15.45:0/3965940528 --> [v2:172.21.15.45:6840/3755310822,v1:172.21.15.45:6841/3755310822] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d21139c8b0 con 0x55d2114af730 2024-04-16T17:14:41.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.420+0000 7f708f400640 1 --2- 172.21.15.45:0/3965940528 >> [v2:172.21.15.45:6840/3755310822,v1:172.21.15.45:6841/3755310822] conn(0x55d2114af730 0x55d2114b1b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:41.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.420+0000 7f708f400640 1 --2- 172.21.15.45:0/3965940528 >> [v2:172.21.15.45:6840/3755310822,v1:172.21.15.45:6841/3755310822] conn(0x55d2114af730 0x55d2114b1b50 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2c6d86de63ad88b0 server_cookie=51dbd070a5352c95 in_seq=0 out_seq=0 2024-04-16T17:14:41.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.420+0000 7f707a000640 10 client.17595 ms_handle_connect on v2:172.21.15.45:6840/3755310822 2024-04-16T17:14:41.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.421+0000 7f707a000640 1 -- 172.21.15.45:0/3965940528 <== mds.1 v2:172.21.15.45:6840/3755310822 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d21139c8b0 con 0x55d2114af730 2024-04-16T17:14:41.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.421+0000 7f707a000640 10 client.17595 handle_command_reply: tid=0 2024-04-16T17:14:41.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.421+0000 7f7099bad740 1 client.17595 shutdown 2024-04-16T17:14:41.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.421+0000 7f7099bad740 1 -- 172.21.15.45:0/3965940528 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d21142ff10 msgr2=0x55d2114415a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:41.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.421+0000 7f7099bad740 1 --2- 172.21.15.45:0/3965940528 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d21142ff10 0x55d2114415a0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f7088009620 tx=0x7f7088034a30 comp rx=0 tx=0).stop 2024-04-16T17:14:41.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.421+0000 7f7099bad740 1 -- 172.21.15.45:0/3965940528 shutdown_connections 2024-04-16T17:14:41.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.421+0000 7f7099bad740 1 --2- 172.21.15.45:0/3965940528 >> [v2:172.21.15.45:6840/3755310822,v1:172.21.15.45:6841/3755310822] conn(0x55d2114af730 0x55d2114b1b50 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:41.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.421+0000 7f7099bad740 1 --2- 172.21.15.45:0/3965940528 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d211439410 0x55d211432980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:41.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.421+0000 7f7099bad740 1 --2- 172.21.15.45:0/3965940528 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d211430830 0x55d211441ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:41.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.421+0000 7f707a000640 0 client.17595 ms_handle_reset on v2:172.21.15.45:6840/3755310822 2024-04-16T17:14:41.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.421+0000 7f7099bad740 1 --2- 172.21.15.45:0/3965940528 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d21142ff10 0x55d2114415a0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:41.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.421+0000 7f7099bad740 1 -- 172.21.15.45:0/3965940528 >> 172.21.15.45:0/3965940528 conn(0x55d21142a640 msgr2=0x55d211429700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:14:41.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.421+0000 7f7099bad740 1 -- 172.21.15.45:0/3965940528 shutdown_connections 2024-04-16T17:14:41.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.421+0000 7f7099bad740 1 -- 172.21.15.45:0/3965940528 wait complete. 2024-04-16T17:14:41.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.421+0000 7f7099bad740 20 client.17595 trim_cache size 0 max 16384 2024-04-16T17:14:41.428 INFO:teuthology.orchestra.run.smithi045.stdout:[] 2024-04-16T17:14:41.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.422+0000 7f7095800640 1 -- 172.21.15.45:0/630618113 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f70740a3d20 msgr2=0x7f70740a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:41.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.422+0000 7f7095800640 1 --2- 172.21.15.45:0/630618113 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f70740a3d20 0x7f70740a61e0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f708000e9c0 tx=0x7f70800d4a10 comp rx=0 tx=0).stop 2024-04-16T17:14:41.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.422+0000 7f7095800640 1 -- 172.21.15.45:0/630618113 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7090076530 msgr2=0x7f709019ed80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:41.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.422+0000 7f7095800640 1 --2- 172.21.15.45:0/630618113 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7090076530 0x7f709019ed80 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f708000e9f0 tx=0x7f708000eeb0 comp rx=0 tx=0).stop 2024-04-16T17:14:41.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.422+0000 7f7095800640 1 -- 172.21.15.45:0/630618113 shutdown_connections 2024-04-16T17:14:41.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.422+0000 7f7095800640 1 --2- 172.21.15.45:0/630618113 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f70740a3d20 0x7f70740a61e0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:41.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.422+0000 7f7095800640 1 --2- 172.21.15.45:0/630618113 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7090076e50 0x7f709019f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:41.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.422+0000 7f7095800640 1 --2- 172.21.15.45:0/630618113 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7090112d40 0x7f70901a35f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:41.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.422+0000 7f7095800640 1 --2- 172.21.15.45:0/630618113 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7090076530 0x7f709019ed80 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:41.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.422+0000 7f7095800640 1 -- 172.21.15.45:0/630618113 >> 172.21.15.45:0/630618113 conn(0x7f70900ffb10 msgr2=0x7f7090114390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:14:41.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.422+0000 7f7095800640 1 -- 172.21.15.45:0/630618113 shutdown_connections 2024-04-16T17:14:41.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.422+0000 7f7095800640 1 -- 172.21.15.45:0/630618113 wait complete. 2024-04-16T17:14:41.440 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-16T17:14:41.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.656+0000 7fd209200640 1 -- 172.21.15.45:0/209602673 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd2040743a0 msgr2=0x7fd204074780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:41.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.656+0000 7fd209200640 1 --2- 172.21.15.45:0/209602673 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd2040743a0 0x7fd204074780 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fd1f4009900 tx=0x7fd1f402bcd0 comp rx=0 tx=0).stop 2024-04-16T17:14:41.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.656+0000 7fd209200640 1 -- 172.21.15.45:0/209602673 shutdown_connections 2024-04-16T17:14:41.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.656+0000 7fd209200640 1 --2- 172.21.15.45:0/209602673 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd204074cc0 0x7fd204112970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:41.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.656+0000 7fd209200640 1 --2- 172.21.15.45:0/209602673 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd204112eb0 0x7fd2041152a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:41.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.656+0000 7fd209200640 1 --2- 172.21.15.45:0/209602673 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd2040743a0 0x7fd204074780 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:41.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.656+0000 7fd209200640 1 -- 172.21.15.45:0/209602673 >> 172.21.15.45:0/209602673 conn(0x7fd2040fd9d0 msgr2=0x7fd2040ffdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:14:41.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.656+0000 7fd209200640 1 -- 172.21.15.45:0/209602673 shutdown_connections 2024-04-16T17:14:41.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.656+0000 7fd209200640 1 -- 172.21.15.45:0/209602673 wait complete. 2024-04-16T17:14:41.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.657+0000 7fd209200640 1 Processor -- start 2024-04-16T17:14:41.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.657+0000 7fd209200640 1 -- start start 2024-04-16T17:14:41.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.658+0000 7fd209200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd2040743a0 0x7fd20419edd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:41.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.658+0000 7fd209200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd204074cc0 0x7fd20419f310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:41.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.658+0000 7fd209200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd204112eb0 0x7fd2041a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:41.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.658+0000 7fd209200640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fd2041188f0 con 0x7fd2040743a0 2024-04-16T17:14:41.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.658+0000 7fd209200640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fd204118770 con 0x7fd204074cc0 2024-04-16T17:14:41.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.658+0000 7fd209200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fd204118a70 con 0x7fd204112eb0 2024-04-16T17:14:41.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.658+0000 7fd203400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd2040743a0 0x7fd20419edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:41.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.658+0000 7fd203400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd2040743a0 0x7fd20419edd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59378/0 (socket says 172.21.15.45:59378) 2024-04-16T17:14:41.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.658+0000 7fd203400640 1 -- 172.21.15.45:0/1618976223 learned_addr learned my addr 172.21.15.45:0/1618976223 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:14:41.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.658+0000 7fd203e00640 1 --2- 172.21.15.45:0/1618976223 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd204112eb0 0x7fd2041a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:41.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.658+0000 7fd202a00640 1 --2- 172.21.15.45:0/1618976223 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd204074cc0 0x7fd20419f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:41.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.659+0000 7fd203400640 1 -- 172.21.15.45:0/1618976223 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd204112eb0 msgr2=0x7fd2041a3690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:41.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.659+0000 7fd203400640 1 --2- 172.21.15.45:0/1618976223 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd204112eb0 0x7fd2041a3690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:41.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.659+0000 7fd203400640 1 -- 172.21.15.45:0/1618976223 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd204074cc0 msgr2=0x7fd20419f310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:41.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.659+0000 7fd203400640 1 --2- 172.21.15.45:0/1618976223 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd204074cc0 0x7fd20419f310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:41.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.659+0000 7fd203400640 1 -- 172.21.15.45:0/1618976223 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2041a3dc0 con 0x7fd2040743a0 2024-04-16T17:14:41.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.659+0000 7fd202a00640 1 --2- 172.21.15.45:0/1618976223 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd204074cc0 0x7fd20419f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:14:41.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.659+0000 7fd203e00640 1 --2- 172.21.15.45:0/1618976223 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd204112eb0 0x7fd2041a3690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:14:41.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.659+0000 7fd203400640 1 --2- 172.21.15.45:0/1618976223 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd2040743a0 0x7fd20419edd0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fd1f4000c00 tx=0x7fd1f4004360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:14:41.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.659+0000 7fd1fbe00640 1 -- 172.21.15.45:0/1618976223 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1f4042020 con 0x7fd2040743a0 2024-04-16T17:14:41.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.659+0000 7fd1fbe00640 1 -- 172.21.15.45:0/1618976223 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd1f40054e0 con 0x7fd2040743a0 2024-04-16T17:14:41.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.659+0000 7fd1fbe00640 1 -- 172.21.15.45:0/1618976223 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1f4003af0 con 0x7fd2040743a0 2024-04-16T17:14:41.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.659+0000 7fd209200640 1 -- 172.21.15.45:0/1618976223 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2041a4050 con 0x7fd2040743a0 2024-04-16T17:14:41.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.660+0000 7fd209200640 1 -- 172.21.15.45:0/1618976223 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2041ab940 con 0x7fd2040743a0 2024-04-16T17:14:41.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.663+0000 7fd20d5d9740 1 Processor -- start 2024-04-16T17:14:41.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.663+0000 7fd20d5d9740 1 -- start start 2024-04-16T17:14:41.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.663+0000 7fd1fbe00640 1 -- 172.21.15.45:0/1618976223 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fd1f4045020 con 0x7fd2040743a0 2024-04-16T17:14:41.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.663+0000 7fd20d5d9740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55961dea7f10 0x55961dea82f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:41.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.663+0000 7fd20d5d9740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55961dea8830 0x55961deb0ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:41.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.663+0000 7fd20d5d9740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55961deb1410 0x55961deb3800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:41.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.663+0000 7fd20d5d9740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55961de0d020 con 0x55961deb1410 2024-04-16T17:14:41.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.663+0000 7fd20d5d9740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55961ddb3460 con 0x55961dea7f10 2024-04-16T17:14:41.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.663+0000 7fd20d5d9740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55961de08fe0 con 0x55961dea8830 2024-04-16T17:14:41.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.664+0000 7fd203e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55961dea7f10 0x55961dea82f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:41.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.664+0000 7fd203e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55961dea7f10 0x55961dea82f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:46642/0 (socket says 172.21.15.45:46642) 2024-04-16T17:14:41.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.664+0000 7fd203e00640 1 -- 172.21.15.45:0/549154921 learned_addr learned my addr 172.21.15.45:0/549154921 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:14:41.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.664+0000 7fd203400640 1 --2- 172.21.15.45:0/549154921 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55961dea8830 0x55961deb0ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:41.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.664+0000 7fd202a00640 1 --2- 172.21.15.45:0/549154921 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55961deb1410 0x55961deb3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:41.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.664+0000 7fd203e00640 1 -- 172.21.15.45:0/549154921 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55961dea8830 msgr2=0x55961deb0ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:14:41.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.664+0000 7fd203e00640 1 --2- 172.21.15.45:0/549154921 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55961dea8830 0x55961deb0ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:41.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.664+0000 7fd203e00640 1 -- 172.21.15.45:0/549154921 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55961deb1410 msgr2=0x55961deb3800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:14:41.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.664+0000 7fd203e00640 1 --2- 172.21.15.45:0/549154921 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55961deb1410 0x55961deb3800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:41.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.664+0000 7fd203e00640 1 -- 172.21.15.45:0/549154921 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55961ddbb040 con 0x55961dea7f10 2024-04-16T17:14:41.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.664+0000 7fd202a00640 1 --2- 172.21.15.45:0/549154921 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55961deb1410 0x55961deb3800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:14:41.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.665+0000 7fd203e00640 1 --2- 172.21.15.45:0/549154921 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55961dea7f10 0x55961dea82f0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fd1fc011760 tx=0x7fd1fc011c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=39db351ae58ae973 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:14:41.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.665+0000 7fd1f8c00640 1 -- 172.21.15.45:0/549154921 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1fc004d20 con 0x55961dea7f10 2024-04-16T17:14:41.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.665+0000 7fd1f8c00640 1 -- 172.21.15.45:0/549154921 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd1fc00f9a0 con 0x55961dea7f10 2024-04-16T17:14:41.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.665+0000 7fd1f8c00640 1 -- 172.21.15.45:0/549154921 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1fc004590 con 0x55961dea7f10 2024-04-16T17:14:41.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.665+0000 7fd1fbe00640 1 --2- 172.21.15.45:0/1618976223 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fd1e80a3d20 0x7fd1e80a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:41.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.665+0000 7fd203400640 1 --2- 172.21.15.45:0/1618976223 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fd1e80a3d20 0x7fd1e80a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:41.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.665+0000 7fd1fbe00640 1 -- 172.21.15.45:0/1618976223 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9496+0+0 (secure 0 0 0) 0x7fd1f40f26f0 con 0x7fd2040743a0 2024-04-16T17:14:41.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.666+0000 7fd20d5d9740 1 -- 172.21.15.45:0/549154921 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55961dea7f10 msgr2=0x55961dea82f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:41.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.666+0000 7fd20d5d9740 1 --2- 172.21.15.45:0/549154921 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55961dea7f10 0x55961dea82f0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fd1fc011760 tx=0x7fd1fc011c20 comp rx=0 tx=0).stop 2024-04-16T17:14:41.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.666+0000 7fd20d5d9740 1 -- 172.21.15.45:0/549154921 shutdown_connections 2024-04-16T17:14:41.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.666+0000 7fd20d5d9740 1 --2- 172.21.15.45:0/549154921 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55961dea8830 0x55961deb0ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:41.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.666+0000 7fd20d5d9740 1 --2- 172.21.15.45:0/549154921 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55961deb1410 0x55961deb3800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:41.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.666+0000 7fd20d5d9740 1 --2- 172.21.15.45:0/549154921 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55961dea7f10 0x55961dea82f0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:41.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.666+0000 7fd20d5d9740 1 -- 172.21.15.45:0/549154921 >> 172.21.15.45:0/549154921 conn(0x55961dea2640 msgr2=0x55961dea2a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:14:41.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.666+0000 7fd203400640 1 --2- 172.21.15.45:0/1618976223 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fd1e80a3d20 0x7fd1e80a61e0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x55961dea79e0 tx=0x7fd1f40f8530 comp rx=0 tx=0).ready entity=mgr.25411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:14:41.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.666+0000 7fd20d5d9740 1 -- 172.21.15.45:0/549154921 shutdown_connections 2024-04-16T17:14:41.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.666+0000 7fd20d5d9740 1 -- 172.21.15.45:0/549154921 wait complete. 2024-04-16T17:14:41.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.667+0000 7fd20d5d9740 1 -- start start 2024-04-16T17:14:41.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.667+0000 7fd20d5d9740 1 Processor -- start 2024-04-16T17:14:41.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.667+0000 7fd20d5d9740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55961dea7f10 0x55961deae4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:41.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.667+0000 7fd20d5d9740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55961dea8830 0x55961deae9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:41.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.667+0000 7fd20d5d9740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55961deb1410 0x55961deaef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:41.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.667+0000 7fd20d5d9740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55961deb4200 con 0x55961dea8830 2024-04-16T17:14:41.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.667+0000 7fd20d5d9740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55961ddc6830 con 0x55961dea7f10 2024-04-16T17:14:41.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.667+0000 7fd20d5d9740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55961deb4380 con 0x55961deb1410 2024-04-16T17:14:41.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.667+0000 7fd20d5d9740 10 client.0 _ll_register_callbacks cb 0x55961ddce150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:14:41.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.668+0000 7fd203e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55961dea7f10 0x55961deae4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:41.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.668+0000 7fd202a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55961dea8830 0x55961deae9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:41.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.668+0000 7fd203e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55961dea7f10 0x55961deae4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:46652/0 (socket says 172.21.15.45:46652) 2024-04-16T17:14:41.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.668+0000 7fd203e00640 1 -- 172.21.15.45:0/2564790896 learned_addr learned my addr 172.21.15.45:0/2564790896 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:14:41.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.668+0000 7fd203e00640 1 --2- 172.21.15.45:0/2564790896 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55961deb1410 0x55961deaef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:41.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.668+0000 7fd203e00640 1 -- 172.21.15.45:0/2564790896 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55961deb1410 msgr2=0x55961deaef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:14:41.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.668+0000 7fd203e00640 1 --2- 172.21.15.45:0/2564790896 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55961deb1410 0x55961deaef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:41.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.668+0000 7fd203e00640 1 -- 172.21.15.45:0/2564790896 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55961dea8830 msgr2=0x55961deae9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:14:41.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.668+0000 7fd203e00640 1 --2- 172.21.15.45:0/2564790896 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55961dea8830 0x55961deae9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:41.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.668+0000 7fd203e00640 1 -- 172.21.15.45:0/2564790896 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55961ddca920 con 0x55961dea7f10 2024-04-16T17:14:41.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.668+0000 7fd202a00640 1 --2- 172.21.15.45:0/2564790896 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55961dea8830 0x55961deae9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:14:41.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.668+0000 7fd203e00640 1 --2- 172.21.15.45:0/2564790896 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55961dea7f10 0x55961deae4b0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fd1fc009fd0 tx=0x7fd1fc0040f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=79ce62a4d224b7e2 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:14:41.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.669+0000 7fd1ed600640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-04-16T17:14:41.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.669+0000 7fd1ed600640 1 -- 172.21.15.45:0/2564790896 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1fc005200 con 0x55961dea7f10 2024-04-16T17:14:41.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.669+0000 7fd1ed600640 1 -- 172.21.15.45:0/2564790896 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9496+0+0 (secure 0 0 0) 0x7fd1fc0053a0 con 0x55961dea7f10 2024-04-16T17:14:41.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.669+0000 7fd20d5d9740 1 -- 172.21.15.45:0/2564790896 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd1fc00f9a0 con 0x55961dea7f10 2024-04-16T17:14:41.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.669+0000 7fd1ed600640 1 -- 172.21.15.45:0/2564790896 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7fd1fc011320 con 0x55961dea7f10 2024-04-16T17:14:41.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.669+0000 7fd20d5d9740 10 client.27767 fetch_fsmap learned FSMap version 42 2024-04-16T17:14:41.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.669+0000 7fd20d5d9740 1 -- 172.21.15.45:0/2564790896 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55961dd987f0 con 0x55961dea7f10 2024-04-16T17:14:41.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.669+0000 7fd1ed600640 1 -- 172.21.15.45:0/2564790896 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd1fc00f9a0 con 0x55961dea7f10 2024-04-16T17:14:41.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.670+0000 7fd1ed600640 1 -- 172.21.15.45:0/2564790896 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 42) ==== 8362+0+0 (secure 0 0 0) 0x7fd1fc018070 con 0x55961dea7f10 2024-04-16T17:14:41.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.670+0000 7fd20d5d9740 10 client.27767 fetch_fsmap finished waiting for FSMap version 42 2024-04-16T17:14:41.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.670+0000 7fd20d5d9740 10 client.27767 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:14:41.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.670+0000 7fd20d5d9740 20 client.27767 populate_metadata read hostname 'smithi045' 2024-04-16T17:14:41.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.670+0000 7fd20d5d9740 1 --2- 172.21.15.45:0/2564790896 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55961dec1710 0x55961dec3b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:41.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.670+0000 7fd20d5d9740 4 client.27767 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:14:41.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.670+0000 7fd20d5d9740 1 -- 172.21.15.45:0/2564790896 --> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55961de1aa40 con 0x55961dec1710 2024-04-16T17:14:41.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.670+0000 7fd202a00640 1 --2- 172.21.15.45:0/2564790896 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55961dec1710 0x55961dec3b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:41.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.671+0000 7fd202a00640 1 --2- 172.21.15.45:0/2564790896 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55961dec1710 0x55961dec3b00 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=594442a73bd85d15 server_cookie=a8ae1cb95b09b2b0 in_seq=0 out_seq=0 2024-04-16T17:14:41.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.671+0000 7fd1ed600640 10 client.27767 ms_handle_connect on v2:172.21.15.104:6838/3660070592 2024-04-16T17:14:41.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.671+0000 7fd1ed600640 1 -- 172.21.15.45:0/2564790896 <== mds.0 v2:172.21.15.104:6838/3660070592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55961de1aa40 con 0x55961dec1710 2024-04-16T17:14:41.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.672+0000 7fd1ed600640 10 client.27767 handle_command_reply: tid=0 2024-04-16T17:14:41.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.672+0000 7fd20d5d9740 1 client.27767 shutdown 2024-04-16T17:14:41.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.672+0000 7fd20d5d9740 1 -- 172.21.15.45:0/2564790896 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55961dea7f10 msgr2=0x55961deae4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:41.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.672+0000 7fd20d5d9740 1 --2- 172.21.15.45:0/2564790896 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55961dea7f10 0x55961deae4b0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fd1fc009fd0 tx=0x7fd1fc0040f0 comp rx=0 tx=0).stop 2024-04-16T17:14:41.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.672+0000 7fd20d5d9740 1 -- 172.21.15.45:0/2564790896 shutdown_connections 2024-04-16T17:14:41.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.672+0000 7fd20d5d9740 1 --2- 172.21.15.45:0/2564790896 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55961dec1710 0x55961dec3b00 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:41.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.672+0000 7fd20d5d9740 1 --2- 172.21.15.45:0/2564790896 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55961deb1410 0x55961deaef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:41.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.672+0000 7fd20d5d9740 1 --2- 172.21.15.45:0/2564790896 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55961dea8830 0x55961deae9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:41.679 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.672+0000 7fd20d5d9740 1 --2- 172.21.15.45:0/2564790896 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55961dea7f10 0x55961deae4b0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:41.679 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.672+0000 7fd1ed600640 0 client.27767 ms_handle_reset on v2:172.21.15.104:6838/3660070592 2024-04-16T17:14:41.679 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.672+0000 7fd20d5d9740 1 -- 172.21.15.45:0/2564790896 >> 172.21.15.45:0/2564790896 conn(0x55961dea2640 msgr2=0x55961dea4390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:14:41.679 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.672+0000 7fd20d5d9740 1 -- 172.21.15.45:0/2564790896 shutdown_connections 2024-04-16T17:14:41.679 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.672+0000 7fd20d5d9740 1 -- 172.21.15.45:0/2564790896 wait complete. 2024-04-16T17:14:41.679 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.672+0000 7fd20d5d9740 20 client.27767 trim_cache size 0 max 16384 2024-04-16T17:14:41.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.702+0000 7fd20d5d9740 1 Processor -- start 2024-04-16T17:14:41.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.702+0000 7fd20d5d9740 1 -- start start 2024-04-16T17:14:41.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.703+0000 7fd20d5d9740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55961dea7f10 0x55961debd0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:41.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.703+0000 7fd20d5d9740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55961dea8830 0x55961deaed50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:41.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.703+0000 7fd20d5d9740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55961deb1410 0x55961dea1580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:41.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.703+0000 7fd20d5d9740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fd2041b82b0 con 0x55961deb1410 2024-04-16T17:14:41.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.703+0000 7fd20d5d9740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fd2041b5b20 con 0x55961dea8830 2024-04-16T17:14:41.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.703+0000 7fd20d5d9740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fd2041b3990 con 0x55961dea7f10 2024-04-16T17:14:41.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.703+0000 7fd203e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55961dea7f10 0x55961debd0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:41.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.703+0000 7fd203e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55961deb1410 0x55961dea1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:41.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.703+0000 7fd202a00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55961dea8830 0x55961deaed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:41.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.703+0000 7fd202a00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55961dea8830 0x55961deaed50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:46658/0 (socket says 172.21.15.45:46658) 2024-04-16T17:14:41.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.703+0000 7fd202a00640 1 -- 172.21.15.45:0/1769005369 learned_addr learned my addr 172.21.15.45:0/1769005369 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:14:41.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.703+0000 7fd203e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55961dea7f10 0x55961debd0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:34348/0 (socket says 172.21.15.45:34348) 2024-04-16T17:14:41.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.703+0000 7fd202a00640 1 -- 172.21.15.45:0/1769005369 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55961dea7f10 msgr2=0x55961debd0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:14:41.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.703+0000 7fd202a00640 1 --2- 172.21.15.45:0/1769005369 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55961dea7f10 0x55961debd0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:41.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.703+0000 7fd202a00640 1 -- 172.21.15.45:0/1769005369 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55961deb1410 msgr2=0x55961dea1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:14:41.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.703+0000 7fd202a00640 1 --2- 172.21.15.45:0/1769005369 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55961deb1410 0x55961dea1580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:41.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.704+0000 7fd202a00640 1 -- 172.21.15.45:0/1769005369 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55961db1f3f0 con 0x55961dea8830 2024-04-16T17:14:41.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.704+0000 7fd202a00640 1 --2- 172.21.15.45:0/1769005369 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55961dea8830 0x55961deaed50 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fd1f00128e0 tx=0x7fd1f0005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4b9d21c5c3b527bf server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:14:41.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.704+0000 7fd1efe00640 1 -- 172.21.15.45:0/1769005369 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1f0011af0 con 0x55961dea8830 2024-04-16T17:14:41.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.704+0000 7fd1efe00640 1 -- 172.21.15.45:0/1769005369 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd1f0024030 con 0x55961dea8830 2024-04-16T17:14:41.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.704+0000 7fd1efe00640 1 -- 172.21.15.45:0/1769005369 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1f0004d00 con 0x55961dea8830 2024-04-16T17:14:41.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.705+0000 7fd20d5d9740 1 -- 172.21.15.45:0/1769005369 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55961dea8830 msgr2=0x55961deaed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:41.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.705+0000 7fd20d5d9740 1 --2- 172.21.15.45:0/1769005369 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55961dea8830 0x55961deaed50 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fd1f00128e0 tx=0x7fd1f0005f90 comp rx=0 tx=0).stop 2024-04-16T17:14:41.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.705+0000 7fd20d5d9740 1 -- 172.21.15.45:0/1769005369 shutdown_connections 2024-04-16T17:14:41.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.705+0000 7fd20d5d9740 1 --2- 172.21.15.45:0/1769005369 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55961deb1410 0x55961dea1580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:41.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.705+0000 7fd20d5d9740 1 --2- 172.21.15.45:0/1769005369 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55961dea8830 0x55961deaed50 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:41.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.705+0000 7fd20d5d9740 1 --2- 172.21.15.45:0/1769005369 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55961dea7f10 0x55961debd0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:41.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.705+0000 7fd20d5d9740 1 -- 172.21.15.45:0/1769005369 >> 172.21.15.45:0/1769005369 conn(0x55961dea2640 msgr2=0x55961dea0e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:14:41.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.705+0000 7fd20d5d9740 1 -- 172.21.15.45:0/1769005369 shutdown_connections 2024-04-16T17:14:41.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.705+0000 7fd20d5d9740 1 -- 172.21.15.45:0/1769005369 wait complete. 2024-04-16T17:14:41.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.706+0000 7fd20d5d9740 1 -- start start 2024-04-16T17:14:41.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.706+0000 7fd20d5d9740 1 Processor -- start 2024-04-16T17:14:41.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.707+0000 7fd20d5d9740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55961dea7f10 0x55961deb9950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:41.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.707+0000 7fd20d5d9740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55961dea8830 0x55961deb4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:41.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.707+0000 7fd20d5d9740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55961deb1410 0x55961deb5200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:41.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.707+0000 7fd20d5d9740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fd2041b3fe0 con 0x55961deb1410 2024-04-16T17:14:41.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.707+0000 7fd20d5d9740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fd2041b5450 con 0x55961dea7f10 2024-04-16T17:14:41.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.707+0000 7fd20d5d9740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fd2041b46b0 con 0x55961dea8830 2024-04-16T17:14:41.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.707+0000 7fd20d5d9740 10 client.0 _ll_register_callbacks cb 0x7fd1f40f8400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:14:41.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.707+0000 7fd203e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55961dea7f10 0x55961deb9950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:41.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.707+0000 7fd203e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55961deb1410 0x55961deb5200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:41.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.707+0000 7fd203e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55961dea7f10 0x55961deb9950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:46660/0 (socket says 172.21.15.45:46660) 2024-04-16T17:14:41.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.707+0000 7fd203e00640 1 -- 172.21.15.45:0/2323046118 learned_addr learned my addr 172.21.15.45:0/2323046118 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:14:41.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.707+0000 7fd202a00640 1 --2- 172.21.15.45:0/2323046118 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55961dea8830 0x55961deb4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:41.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.707+0000 7fd203e00640 1 -- 172.21.15.45:0/2323046118 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55961dea8830 msgr2=0x55961deb4cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:14:41.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.707+0000 7fd203e00640 1 --2- 172.21.15.45:0/2323046118 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55961dea8830 0x55961deb4cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:41.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.707+0000 7fd203e00640 1 -- 172.21.15.45:0/2323046118 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55961deb1410 msgr2=0x55961deb5200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:14:41.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.707+0000 7fd203e00640 1 --2- 172.21.15.45:0/2323046118 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55961deb1410 0x55961deb5200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:41.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.707+0000 7fd203e00640 1 -- 172.21.15.45:0/2323046118 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55961dbe97d0 con 0x55961dea7f10 2024-04-16T17:14:41.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.708+0000 7fd203e00640 1 --2- 172.21.15.45:0/2323046118 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55961dea7f10 0x55961deb9950 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fd1fc0040c0 tx=0x7fd1fc014040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=98527a773c84f4d5 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:14:41.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.708+0000 7fd1ed600640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-04-16T17:14:41.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.708+0000 7fd1ed600640 1 -- 172.21.15.45:0/2323046118 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1fc00f9a0 con 0x55961dea7f10 2024-04-16T17:14:41.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.708+0000 7fd20d5d9740 1 -- 172.21.15.45:0/2323046118 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd1f0024030 con 0x55961dea7f10 2024-04-16T17:14:41.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.708+0000 7fd1ed600640 1 -- 172.21.15.45:0/2323046118 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9496+0+0 (secure 0 0 0) 0x7fd1fc003d80 con 0x55961dea7f10 2024-04-16T17:14:41.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.708+0000 7fd1ed600640 1 -- 172.21.15.45:0/2323046118 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7fd1fc00f790 con 0x55961dea7f10 2024-04-16T17:14:41.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.708+0000 7fd20d5d9740 10 client.27773 fetch_fsmap learned FSMap version 42 2024-04-16T17:14:41.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.708+0000 7fd20d5d9740 1 -- 172.21.15.45:0/2323046118 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55961df27870 con 0x55961dea7f10 2024-04-16T17:14:41.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.708+0000 7fd1ed600640 1 -- 172.21.15.45:0/2323046118 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd1f0024030 con 0x55961dea7f10 2024-04-16T17:14:41.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.709+0000 7fd1ed600640 1 -- 172.21.15.45:0/2323046118 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 42) ==== 8362+0+0 (secure 0 0 0) 0x7fd1fc018070 con 0x55961dea7f10 2024-04-16T17:14:41.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.709+0000 7fd20d5d9740 10 client.27773 fetch_fsmap finished waiting for FSMap version 42 2024-04-16T17:14:41.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.709+0000 7fd20d5d9740 10 client.27773 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:14:41.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.709+0000 7fd20d5d9740 20 client.27773 populate_metadata read hostname 'smithi045' 2024-04-16T17:14:41.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.709+0000 7fd20d5d9740 1 --2- 172.21.15.45:0/2323046118 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55961df27f80 0x55961df2a370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:41.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.709+0000 7fd20d5d9740 4 client.27773 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T17:14:41.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.709+0000 7fd20d5d9740 1 -- 172.21.15.45:0/2323046118 --> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55961de148b0 con 0x55961df27f80 2024-04-16T17:14:41.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.709+0000 7fd202a00640 1 --2- 172.21.15.45:0/2323046118 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55961df27f80 0x55961df2a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:41.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.710+0000 7fd202a00640 1 --2- 172.21.15.45:0/2323046118 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55961df27f80 0x55961df2a370 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4ec1b2d86283d377 server_cookie=e174fa7187fc32b5 in_seq=0 out_seq=0 2024-04-16T17:14:41.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.710+0000 7fd1ed600640 10 client.27773 ms_handle_connect on v2:172.21.15.104:6838/3660070592 2024-04-16T17:14:41.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.710+0000 7fd1ed600640 1 -- 172.21.15.45:0/2323046118 <== mds.0 v2:172.21.15.104:6838/3660070592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55961de148b0 con 0x55961df27f80 2024-04-16T17:14:41.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.710+0000 7fd1ed600640 10 client.27773 handle_command_reply: tid=0 2024-04-16T17:14:41.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.710+0000 7fd20d5d9740 1 client.27773 shutdown 2024-04-16T17:14:41.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.710+0000 7fd20d5d9740 1 -- 172.21.15.45:0/2323046118 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55961dea7f10 msgr2=0x55961deb9950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:41.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.710+0000 7fd20d5d9740 1 --2- 172.21.15.45:0/2323046118 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55961dea7f10 0x55961deb9950 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fd1fc0040c0 tx=0x7fd1fc014040 comp rx=0 tx=0).stop 2024-04-16T17:14:41.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.710+0000 7fd20d5d9740 1 -- 172.21.15.45:0/2323046118 shutdown_connections 2024-04-16T17:14:41.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.710+0000 7fd20d5d9740 1 --2- 172.21.15.45:0/2323046118 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55961df27f80 0x55961df2a370 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:41.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.710+0000 7fd20d5d9740 1 --2- 172.21.15.45:0/2323046118 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55961dea8830 0x55961deb4cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:41.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.710+0000 7fd20d5d9740 1 --2- 172.21.15.45:0/2323046118 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55961deb1410 0x55961deb5200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:41.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.710+0000 7fd20d5d9740 1 --2- 172.21.15.45:0/2323046118 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55961dea7f10 0x55961deb9950 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:41.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.710+0000 7fd20d5d9740 1 -- 172.21.15.45:0/2323046118 >> 172.21.15.45:0/2323046118 conn(0x55961dea2640 msgr2=0x55961dea1700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:14:41.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.710+0000 7fd1ed600640 0 client.27773 ms_handle_reset on v2:172.21.15.104:6838/3660070592 2024-04-16T17:14:41.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.710+0000 7fd20d5d9740 1 -- 172.21.15.45:0/2323046118 shutdown_connections 2024-04-16T17:14:41.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.710+0000 7fd20d5d9740 1 -- 172.21.15.45:0/2323046118 wait complete. 2024-04-16T17:14:41.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.710+0000 7fd20d5d9740 20 client.27773 trim_cache size 0 max 16384 2024-04-16T17:14:41.719 INFO:teuthology.orchestra.run.smithi045.stdout:[] 2024-04-16T17:14:41.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.711+0000 7fd209200640 1 -- 172.21.15.45:0/1618976223 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fd1e80a3d20 msgr2=0x7fd1e80a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:41.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.711+0000 7fd209200640 1 --2- 172.21.15.45:0/1618976223 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fd1e80a3d20 0x7fd1e80a61e0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x55961dea79e0 tx=0x7fd1f40f8530 comp rx=0 tx=0).stop 2024-04-16T17:14:41.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.711+0000 7fd209200640 1 -- 172.21.15.45:0/1618976223 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd2040743a0 msgr2=0x7fd20419edd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:41.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.711+0000 7fd209200640 1 --2- 172.21.15.45:0/1618976223 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd2040743a0 0x7fd20419edd0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fd1f4000c00 tx=0x7fd1f4004360 comp rx=0 tx=0).stop 2024-04-16T17:14:41.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.711+0000 7fd209200640 1 -- 172.21.15.45:0/1618976223 shutdown_connections 2024-04-16T17:14:41.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.711+0000 7fd209200640 1 --2- 172.21.15.45:0/1618976223 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fd1e80a3d20 0x7fd1e80a61e0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:41.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.711+0000 7fd209200640 1 --2- 172.21.15.45:0/1618976223 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd204112eb0 0x7fd2041a3690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:41.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.711+0000 7fd209200640 1 --2- 172.21.15.45:0/1618976223 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd204074cc0 0x7fd20419f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:41.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.711+0000 7fd209200640 1 --2- 172.21.15.45:0/1618976223 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd2040743a0 0x7fd20419edd0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:41.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.711+0000 7fd209200640 1 -- 172.21.15.45:0/1618976223 >> 172.21.15.45:0/1618976223 conn(0x7fd2040fd9d0 msgr2=0x7fd2040fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:14:41.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.711+0000 7fd209200640 1 -- 172.21.15.45:0/1618976223 shutdown_connections 2024-04-16T17:14:41.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:41.712+0000 7fd209200640 1 -- 172.21.15.45:0/1618976223 wait complete. 2024-04-16T17:14:41.726 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:14:41 smithi045 ceph-mon[31676]: from='client.? 172.21.15.45:0/1557852504' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:14:41.804 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:14:41 smithi117 ceph-mon[36730]: from='client.? 172.21.15.45:0/1557852504' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:14:41.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:14:41 smithi104 ceph-mon[36457]: from='client.? 172.21.15.45:0/1557852504' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:14:42.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:14:42 smithi045 ceph-mon[31676]: pgmap v246: 129 pgs: 129 active+clean; 60 MiB data, 837 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 372 KiB/s wr, 29 op/s 2024-04-16T17:14:42.735 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-16T17:14:42.735 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T17:14:42.736 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-16T17:14:42.804 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:14:42 smithi117 ceph-mon[36730]: pgmap v246: 129 pgs: 129 active+clean; 60 MiB data, 837 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 372 KiB/s wr, 29 op/s 2024-04-16T17:14:42.930 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:14:42 smithi104 ceph-mon[36457]: pgmap v246: 129 pgs: 129 active+clean; 60 MiB data, 837 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 372 KiB/s wr, 29 op/s 2024-04-16T17:14:42.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:42.952+0000 7fc427e00640 1 -- 172.21.15.45:0/13120111 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc428102ac0 msgr2=0x7fc428106b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:42.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:42.952+0000 7fc427e00640 1 --2- 172.21.15.45:0/13120111 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc428102ac0 0x7fc428106b00 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fc410009900 tx=0x7fc41002bc30 comp rx=0 tx=0).stop 2024-04-16T17:14:42.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:42.952+0000 7fc427e00640 1 -- 172.21.15.45:0/13120111 shutdown_connections 2024-04-16T17:14:42.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:42.952+0000 7fc427e00640 1 --2- 172.21.15.45:0/13120111 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc4281071e0 0x7fc42810edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:42.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:42.952+0000 7fc427e00640 1 --2- 172.21.15.45:0/13120111 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc428102ac0 0x7fc428106b00 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:42.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:42.952+0000 7fc427e00640 1 --2- 172.21.15.45:0/13120111 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc428102110 0x7fc4281024f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:42.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:42.952+0000 7fc427e00640 1 -- 172.21.15.45:0/13120111 >> 172.21.15.45:0/13120111 conn(0x7fc4280fd9d0 msgr2=0x7fc4280ffdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:14:42.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:42.952+0000 7fc427e00640 1 -- 172.21.15.45:0/13120111 shutdown_connections 2024-04-16T17:14:42.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:42.952+0000 7fc427e00640 1 -- 172.21.15.45:0/13120111 wait complete. 2024-04-16T17:14:42.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:42.953+0000 7fc427e00640 1 Processor -- start 2024-04-16T17:14:42.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:42.954+0000 7fc427e00640 1 -- start start 2024-04-16T17:14:42.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:42.954+0000 7fc427e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc428102110 0x7fc428074300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:42.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:42.954+0000 7fc427e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc4281071e0 0x7fc428074840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:42.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:42.954+0000 7fc427e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc428074d80 0x7fc4281a69b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:42.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:42.954+0000 7fc427e00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fc4281122c0 con 0x7fc428102110 2024-04-16T17:14:42.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:42.954+0000 7fc427e00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fc428112140 con 0x7fc428074d80 2024-04-16T17:14:42.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:42.954+0000 7fc427e00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fc428112440 con 0x7fc4281071e0 2024-04-16T17:14:42.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:42.954+0000 7fc426a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc428102110 0x7fc428074300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:42.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:42.954+0000 7fc427400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc428074d80 0x7fc4281a69b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:42.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:42.954+0000 7fc426a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc428102110 0x7fc428074300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59418/0 (socket says 172.21.15.45:59418) 2024-04-16T17:14:42.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:42.955+0000 7fc426a00640 1 -- 172.21.15.45:0/1978611957 learned_addr learned my addr 172.21.15.45:0/1978611957 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:14:42.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:42.955+0000 7fc41e000640 1 --2- 172.21.15.45:0/1978611957 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc4281071e0 0x7fc428074840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:42.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:42.955+0000 7fc426a00640 1 -- 172.21.15.45:0/1978611957 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc4281071e0 msgr2=0x7fc428074840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:42.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:42.955+0000 7fc426a00640 1 --2- 172.21.15.45:0/1978611957 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc4281071e0 0x7fc428074840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:42.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:42.955+0000 7fc426a00640 1 -- 172.21.15.45:0/1978611957 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc428074d80 msgr2=0x7fc4281a69b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:42.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:42.955+0000 7fc426a00640 1 --2- 172.21.15.45:0/1978611957 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc428074d80 0x7fc4281a69b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:42.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:42.955+0000 7fc426a00640 1 -- 172.21.15.45:0/1978611957 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4281a7050 con 0x7fc428102110 2024-04-16T17:14:42.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:42.955+0000 7fc41e000640 1 --2- 172.21.15.45:0/1978611957 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc4281071e0 0x7fc428074840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:14:42.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:42.955+0000 7fc427400640 1 --2- 172.21.15.45:0/1978611957 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc428074d80 0x7fc4281a69b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:14:42.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:42.955+0000 7fc426a00640 1 --2- 172.21.15.45:0/1978611957 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc428102110 0x7fc428074300 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fc41800e9f0 tx=0x7fc41800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:14:42.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:42.955+0000 7fc41fe00640 1 -- 172.21.15.45:0/1978611957 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc41800cd80 con 0x7fc428102110 2024-04-16T17:14:42.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:42.956+0000 7fc41fe00640 1 -- 172.21.15.45:0/1978611957 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc418010910 con 0x7fc428102110 2024-04-16T17:14:42.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:42.956+0000 7fc41fe00640 1 -- 172.21.15.45:0/1978611957 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc418018990 con 0x7fc428102110 2024-04-16T17:14:42.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:42.956+0000 7fc427e00640 1 -- 172.21.15.45:0/1978611957 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4281a7340 con 0x7fc428102110 2024-04-16T17:14:42.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:42.956+0000 7fc427e00640 1 -- 172.21.15.45:0/1978611957 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4281a7860 con 0x7fc428102110 2024-04-16T17:14:42.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:42.959+0000 7fc41fe00640 1 -- 172.21.15.45:0/1978611957 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fc418010ac0 con 0x7fc428102110 2024-04-16T17:14:42.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:42.960+0000 7fc430969740 1 Processor -- start 2024-04-16T17:14:42.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:42.960+0000 7fc430969740 1 -- start start 2024-04-16T17:14:42.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:42.961+0000 7fc41fe00640 1 --2- 172.21.15.45:0/1978611957 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fc40c0a3d20 0x7fc40c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:42.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:42.961+0000 7fc430969740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562b9f153e90 0x562b9f154270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:42.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:42.961+0000 7fc430969740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562b9f1547b0 0x562b9f15ce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:42.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:42.961+0000 7fc430969740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562b9f15d390 0x562b9f15f780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:42.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:42.961+0000 7fc41fe00640 1 -- 172.21.15.45:0/1978611957 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9496+0+0 (secure 0 0 0) 0x7fc4180d0ff0 con 0x7fc428102110 2024-04-16T17:14:42.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:42.961+0000 7fc430969740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x562b9f0b8cb0 con 0x562b9f15d390 2024-04-16T17:14:42.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:42.961+0000 7fc430969740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x562b9f05f370 con 0x562b9f153e90 2024-04-16T17:14:42.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:42.961+0000 7fc430969740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x562b9f0b4c70 con 0x562b9f1547b0 2024-04-16T17:14:42.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:42.961+0000 7fc41e000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562b9f1547b0 0x562b9f15ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:42.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:42.961+0000 7fc427400640 1 --2- 172.21.15.45:0/1978611957 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fc40c0a3d20 0x7fc40c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:42.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:42.961+0000 7fc41e000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562b9f1547b0 0x562b9f15ce50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:34392/0 (socket says 172.21.15.45:34392) 2024-04-16T17:14:42.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:42.961+0000 7fc41e000640 1 -- 172.21.15.45:0/3774490341 learned_addr learned my addr 172.21.15.45:0/3774490341 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:14:42.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:42.961+0000 7fc426a00640 1 --2- 172.21.15.45:0/3774490341 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562b9f153e90 0x562b9f154270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:42.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:42.961+0000 7fc427400640 1 --2- 172.21.15.45:0/3774490341 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562b9f15d390 0x562b9f15f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:42.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:42.962+0000 7fc41e000640 1 -- 172.21.15.45:0/3774490341 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562b9f153e90 msgr2=0x562b9f154270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:14:42.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:42.962+0000 7fc41e000640 1 --2- 172.21.15.45:0/3774490341 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562b9f153e90 0x562b9f154270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:42.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:42.962+0000 7fc41e000640 1 -- 172.21.15.45:0/3774490341 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562b9f15d390 msgr2=0x562b9f15f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:14:42.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:42.962+0000 7fc41e000640 1 --2- 172.21.15.45:0/3774490341 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562b9f15d390 0x562b9f15f780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:42.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:42.962+0000 7fc41e000640 1 -- 172.21.15.45:0/3774490341 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562b9f07e910 con 0x562b9f1547b0 2024-04-16T17:14:42.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:42.962+0000 7fc426a00640 1 --2- 172.21.15.45:0/3774490341 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562b9f153e90 0x562b9f154270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:14:42.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:42.962+0000 7fc427400640 1 --2- 172.21.15.45:0/3774490341 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562b9f15d390 0x562b9f15f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:14:42.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:42.962+0000 7fc41e000640 1 --2- 172.21.15.45:0/3774490341 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562b9f1547b0 0x562b9f15ce50 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fc42806b560 tx=0x7fc410005e80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bcda979bf2c77600 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:14:42.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:42.962+0000 7fc417e00640 1 -- 172.21.15.45:0/3774490341 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc410044020 con 0x562b9f1547b0 2024-04-16T17:14:42.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:42.962+0000 7fc427400640 1 --2- 172.21.15.45:0/1978611957 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fc40c0a3d20 0x7fc40c0a61e0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fc420009fd0 tx=0x7fc420009f60 comp rx=0 tx=0).ready entity=mgr.25411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:14:42.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:42.962+0000 7fc417e00640 1 -- 172.21.15.45:0/3774490341 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc410004240 con 0x562b9f1547b0 2024-04-16T17:14:42.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:42.962+0000 7fc417e00640 1 -- 172.21.15.45:0/3774490341 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4100046c0 con 0x562b9f1547b0 2024-04-16T17:14:42.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:42.963+0000 7fc430969740 1 -- 172.21.15.45:0/3774490341 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562b9f1547b0 msgr2=0x562b9f15ce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:42.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:42.963+0000 7fc430969740 1 --2- 172.21.15.45:0/3774490341 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562b9f1547b0 0x562b9f15ce50 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fc42806b560 tx=0x7fc410005e80 comp rx=0 tx=0).stop 2024-04-16T17:14:42.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:42.963+0000 7fc430969740 1 -- 172.21.15.45:0/3774490341 shutdown_connections 2024-04-16T17:14:42.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:42.963+0000 7fc430969740 1 --2- 172.21.15.45:0/3774490341 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562b9f1547b0 0x562b9f15ce50 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:42.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:42.963+0000 7fc430969740 1 --2- 172.21.15.45:0/3774490341 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562b9f15d390 0x562b9f15f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:42.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:42.963+0000 7fc430969740 1 --2- 172.21.15.45:0/3774490341 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562b9f153e90 0x562b9f154270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:42.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:42.963+0000 7fc430969740 1 -- 172.21.15.45:0/3774490341 >> 172.21.15.45:0/3774490341 conn(0x562b9f14e5c0 msgr2=0x562b9f14e9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:14:42.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:42.964+0000 7fc430969740 1 -- 172.21.15.45:0/3774490341 shutdown_connections 2024-04-16T17:14:42.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:42.964+0000 7fc430969740 1 -- 172.21.15.45:0/3774490341 wait complete. 2024-04-16T17:14:42.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:42.964+0000 7fc430969740 1 -- start start 2024-04-16T17:14:42.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:42.964+0000 7fc430969740 1 Processor -- start 2024-04-16T17:14:42.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:42.965+0000 7fc430969740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562b9f153e90 0x562b9f15a380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:42.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:42.965+0000 7fc430969740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562b9f1547b0 0x562b9f15a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:42.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:42.965+0000 7fc430969740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562b9f15d390 0x562b9f15ae00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:42.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:42.965+0000 7fc430969740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x562b9f160120 con 0x562b9f15d390 2024-04-16T17:14:42.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:42.965+0000 7fc430969740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x562b9f072730 con 0x562b9f153e90 2024-04-16T17:14:42.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:42.965+0000 7fc430969740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x562b9f1602a0 con 0x562b9f1547b0 2024-04-16T17:14:42.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:42.965+0000 7fc430969740 10 client.0 _ll_register_callbacks cb 0x562b9f0411e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:14:42.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:42.965+0000 7fc427400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562b9f15d390 0x562b9f15ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:42.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:42.965+0000 7fc426a00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562b9f153e90 0x562b9f15a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:42.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:42.965+0000 7fc427400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562b9f15d390 0x562b9f15ae00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59426/0 (socket says 172.21.15.45:59426) 2024-04-16T17:14:42.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:42.965+0000 7fc427400640 1 -- 172.21.15.45:0/452957026 learned_addr learned my addr 172.21.15.45:0/452957026 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:14:42.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:42.965+0000 7fc41e000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562b9f1547b0 0x562b9f15a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:42.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:42.966+0000 7fc426a00640 1 -- 172.21.15.45:0/452957026 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562b9f1547b0 msgr2=0x562b9f15a8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:14:42.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:42.966+0000 7fc426a00640 1 --2- 172.21.15.45:0/452957026 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562b9f1547b0 0x562b9f15a8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:42.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:42.966+0000 7fc426a00640 1 -- 172.21.15.45:0/452957026 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562b9f15d390 msgr2=0x562b9f15ae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:14:42.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:42.966+0000 7fc426a00640 1 --2- 172.21.15.45:0/452957026 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562b9f15d390 0x562b9f15ae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:42.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:42.966+0000 7fc426a00640 1 -- 172.21.15.45:0/452957026 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562b9ee937e0 con 0x562b9f153e90 2024-04-16T17:14:42.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:42.966+0000 7fc426a00640 1 --2- 172.21.15.45:0/452957026 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562b9f153e90 0x562b9f15a380 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fc4180ce330 tx=0x7fc41809cc40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d3129f368ade5240 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:14:42.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:42.966+0000 7fc414c00640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-04-16T17:14:42.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:42.966+0000 7fc414c00640 1 -- 172.21.15.45:0/452957026 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc418018b30 con 0x562b9f153e90 2024-04-16T17:14:42.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:42.966+0000 7fc414c00640 1 -- 172.21.15.45:0/452957026 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9496+0+0 (secure 0 0 0) 0x7fc418018cd0 con 0x562b9f153e90 2024-04-16T17:14:42.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:42.966+0000 7fc430969740 1 -- 172.21.15.45:0/452957026 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc410004240 con 0x562b9f153e90 2024-04-16T17:14:42.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:42.967+0000 7fc414c00640 1 -- 172.21.15.45:0/452957026 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7fc41809bfd0 con 0x562b9f153e90 2024-04-16T17:14:42.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:42.967+0000 7fc430969740 10 client.27785 fetch_fsmap learned FSMap version 42 2024-04-16T17:14:42.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:42.967+0000 7fc430969740 1 -- 172.21.15.45:0/452957026 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x562b9eef3f70 con 0x562b9f153e90 2024-04-16T17:14:42.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:42.967+0000 7fc414c00640 1 -- 172.21.15.45:0/452957026 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc410004240 con 0x562b9f153e90 2024-04-16T17:14:42.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:42.967+0000 7fc414c00640 1 -- 172.21.15.45:0/452957026 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 42) ==== 8362+0+0 (secure 0 0 0) 0x7fc41809ba40 con 0x562b9f153e90 2024-04-16T17:14:42.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:42.967+0000 7fc430969740 10 client.27785 fetch_fsmap finished waiting for FSMap version 42 2024-04-16T17:14:42.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:42.967+0000 7fc430969740 10 client.27785 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:14:42.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:42.967+0000 7fc430969740 20 client.27785 populate_metadata read hostname 'smithi045' 2024-04-16T17:14:42.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:42.967+0000 7fc430969740 1 --2- 172.21.15.45:0/452957026 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x562b9f16d620 0x562b9f16fa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:42.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:42.967+0000 7fc430969740 4 client.27785 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:14:42.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:42.967+0000 7fc430969740 1 -- 172.21.15.45:0/452957026 --> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562b9f0c66d0 con 0x562b9f16d620 2024-04-16T17:14:42.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:42.968+0000 7fc41e000640 1 --2- 172.21.15.45:0/452957026 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x562b9f16d620 0x562b9f16fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:42.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:42.968+0000 7fc41e000640 1 --2- 172.21.15.45:0/452957026 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x562b9f16d620 0x562b9f16fa10 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=66fea332931fb5fd server_cookie=c9dc8464b1811219 in_seq=0 out_seq=0 2024-04-16T17:14:42.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:42.968+0000 7fc414c00640 10 client.27785 ms_handle_connect on v2:172.21.15.104:6838/3660070592 2024-04-16T17:14:42.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:42.969+0000 7fc414c00640 1 -- 172.21.15.45:0/452957026 <== mds.0 v2:172.21.15.104:6838/3660070592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562b9f0c66d0 con 0x562b9f16d620 2024-04-16T17:14:42.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:42.969+0000 7fc414c00640 10 client.27785 handle_command_reply: tid=0 2024-04-16T17:14:42.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:42.969+0000 7fc430969740 1 client.27785 shutdown 2024-04-16T17:14:42.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:42.969+0000 7fc430969740 1 -- 172.21.15.45:0/452957026 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562b9f153e90 msgr2=0x562b9f15a380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:42.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:42.969+0000 7fc430969740 1 --2- 172.21.15.45:0/452957026 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562b9f153e90 0x562b9f15a380 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fc4180ce330 tx=0x7fc41809cc40 comp rx=0 tx=0).stop 2024-04-16T17:14:42.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:42.969+0000 7fc430969740 1 -- 172.21.15.45:0/452957026 shutdown_connections 2024-04-16T17:14:42.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:42.969+0000 7fc430969740 1 --2- 172.21.15.45:0/452957026 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x562b9f16d620 0x562b9f16fa10 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:42.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:42.969+0000 7fc430969740 1 --2- 172.21.15.45:0/452957026 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562b9f1547b0 0x562b9f15a8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:42.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:42.969+0000 7fc430969740 1 --2- 172.21.15.45:0/452957026 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562b9f15d390 0x562b9f15ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:42.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:42.969+0000 7fc414c00640 0 client.27785 ms_handle_reset on v2:172.21.15.104:6838/3660070592 2024-04-16T17:14:42.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:42.969+0000 7fc430969740 1 --2- 172.21.15.45:0/452957026 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562b9f153e90 0x562b9f15a380 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:42.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:42.969+0000 7fc430969740 1 -- 172.21.15.45:0/452957026 >> 172.21.15.45:0/452957026 conn(0x562b9f14e5c0 msgr2=0x562b9f150310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:14:42.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:42.970+0000 7fc430969740 1 -- 172.21.15.45:0/452957026 shutdown_connections 2024-04-16T17:14:42.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:42.970+0000 7fc430969740 1 -- 172.21.15.45:0/452957026 wait complete. 2024-04-16T17:14:42.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:42.970+0000 7fc430969740 20 client.27785 trim_cache size 0 max 16384 2024-04-16T17:14:43.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.000+0000 7fc430969740 1 Processor -- start 2024-04-16T17:14:43.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.001+0000 7fc430969740 1 -- start start 2024-04-16T17:14:43.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.001+0000 7fc430969740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562b9f153e90 0x562b9f15cab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:43.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.001+0000 7fc430969740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562b9f1547b0 0x562b9f168fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:43.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.001+0000 7fc430969740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562b9f15d390 0x562b9f175290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:43.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.001+0000 7fc430969740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fc4281b3e70 con 0x562b9f1547b0 2024-04-16T17:14:43.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.001+0000 7fc430969740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fc4281b16e0 con 0x562b9f15d390 2024-04-16T17:14:43.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.001+0000 7fc430969740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fc4281af550 con 0x562b9f153e90 2024-04-16T17:14:43.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.001+0000 7fc41e000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562b9f1547b0 0x562b9f168fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:43.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.001+0000 7fc41e000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562b9f1547b0 0x562b9f168fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59436/0 (socket says 172.21.15.45:59436) 2024-04-16T17:14:43.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.001+0000 7fc41e000640 1 -- 172.21.15.45:0/2131934866 learned_addr learned my addr 172.21.15.45:0/2131934866 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:14:43.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.001+0000 7fc426a00640 1 --2- 172.21.15.45:0/2131934866 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562b9f153e90 0x562b9f15cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:43.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.001+0000 7fc427400640 1 --2- 172.21.15.45:0/2131934866 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562b9f15d390 0x562b9f175290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:43.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.002+0000 7fc41e000640 1 -- 172.21.15.45:0/2131934866 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562b9f153e90 msgr2=0x562b9f15cab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:14:43.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.002+0000 7fc41e000640 1 --2- 172.21.15.45:0/2131934866 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562b9f153e90 0x562b9f15cab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:43.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.002+0000 7fc41e000640 1 -- 172.21.15.45:0/2131934866 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562b9f15d390 msgr2=0x562b9f175290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:14:43.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.002+0000 7fc41e000640 1 --2- 172.21.15.45:0/2131934866 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562b9f15d390 0x562b9f175290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:43.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.002+0000 7fc41e000640 1 -- 172.21.15.45:0/2131934866 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562b9ed8b170 con 0x562b9f1547b0 2024-04-16T17:14:43.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.002+0000 7fc426a00640 1 --2- 172.21.15.45:0/2131934866 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562b9f153e90 0x562b9f15cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:14:43.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.002+0000 7fc41e000640 1 --2- 172.21.15.45:0/2131934866 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562b9f1547b0 0x562b9f168fe0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fc41002c210 tx=0x7fc41002bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83b5db11a1ad9a34 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:14:43.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.002+0000 7fc417400640 1 -- 172.21.15.45:0/2131934866 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc410044040 con 0x562b9f1547b0 2024-04-16T17:14:43.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.002+0000 7fc417400640 1 -- 172.21.15.45:0/2131934866 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc410005520 con 0x562b9f1547b0 2024-04-16T17:14:43.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.002+0000 7fc417400640 1 -- 172.21.15.45:0/2131934866 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4100406d0 con 0x562b9f1547b0 2024-04-16T17:14:43.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.003+0000 7fc430969740 1 -- 172.21.15.45:0/2131934866 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562b9f1547b0 msgr2=0x562b9f168fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:43.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.003+0000 7fc430969740 1 --2- 172.21.15.45:0/2131934866 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562b9f1547b0 0x562b9f168fe0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fc41002c210 tx=0x7fc41002bc30 comp rx=0 tx=0).stop 2024-04-16T17:14:43.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.003+0000 7fc430969740 1 -- 172.21.15.45:0/2131934866 shutdown_connections 2024-04-16T17:14:43.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.003+0000 7fc430969740 1 --2- 172.21.15.45:0/2131934866 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562b9f15d390 0x562b9f175290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:43.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.003+0000 7fc430969740 1 --2- 172.21.15.45:0/2131934866 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562b9f1547b0 0x562b9f168fe0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:43.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.003+0000 7fc430969740 1 --2- 172.21.15.45:0/2131934866 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562b9f153e90 0x562b9f15cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:43.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.003+0000 7fc430969740 1 -- 172.21.15.45:0/2131934866 >> 172.21.15.45:0/2131934866 conn(0x562b9f14e5c0 msgr2=0x562b9f14d560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:14:43.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.003+0000 7fc430969740 1 -- 172.21.15.45:0/2131934866 shutdown_connections 2024-04-16T17:14:43.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.003+0000 7fc430969740 1 -- 172.21.15.45:0/2131934866 wait complete. 2024-04-16T17:14:43.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.004+0000 7fc430969740 1 -- start start 2024-04-16T17:14:43.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.004+0000 7fc430969740 1 Processor -- start 2024-04-16T17:14:43.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.004+0000 7fc430969740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562b9f153e90 0x562b9f1654f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:43.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.004+0000 7fc430969740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562b9f1547b0 0x562b9f165a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:43.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.004+0000 7fc430969740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562b9f15d390 0x562b9f162670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:43.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.004+0000 7fc430969740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fc4281afba0 con 0x562b9f1547b0 2024-04-16T17:14:43.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.004+0000 7fc430969740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fc4281b1010 con 0x562b9f153e90 2024-04-16T17:14:43.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.004+0000 7fc430969740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fc4281b0270 con 0x562b9f15d390 2024-04-16T17:14:43.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.004+0000 7fc430969740 10 client.0 _ll_register_callbacks cb 0x7fc41809d350 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:14:43.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.004+0000 7fc426a00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562b9f153e90 0x562b9f1654f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:43.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.004+0000 7fc41e000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562b9f1547b0 0x562b9f165a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:43.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.005+0000 7fc426a00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562b9f153e90 0x562b9f1654f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:46720/0 (socket says 172.21.15.45:46720) 2024-04-16T17:14:43.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.005+0000 7fc426a00640 1 -- 172.21.15.45:0/3842112513 learned_addr learned my addr 172.21.15.45:0/3842112513 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:14:43.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.005+0000 7fc427400640 1 --2- 172.21.15.45:0/3842112513 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562b9f15d390 0x562b9f162670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:43.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.005+0000 7fc426a00640 1 -- 172.21.15.45:0/3842112513 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562b9f15d390 msgr2=0x562b9f162670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:14:43.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.005+0000 7fc426a00640 1 --2- 172.21.15.45:0/3842112513 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562b9f15d390 0x562b9f162670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:43.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.005+0000 7fc426a00640 1 -- 172.21.15.45:0/3842112513 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562b9f1547b0 msgr2=0x562b9f165a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:14:43.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.005+0000 7fc426a00640 1 --2- 172.21.15.45:0/3842112513 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562b9f1547b0 0x562b9f165a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:43.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.005+0000 7fc426a00640 1 -- 172.21.15.45:0/3842112513 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562b9f162e70 con 0x562b9f153e90 2024-04-16T17:14:43.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.005+0000 7fc427400640 1 --2- 172.21.15.45:0/3842112513 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562b9f15d390 0x562b9f162670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:14:43.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.005+0000 7fc426a00640 1 --2- 172.21.15.45:0/3842112513 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562b9f153e90 0x562b9f1654f0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fc41809bf40 tx=0x7fc4180a20c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6863a982ac8e7a8b server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:14:43.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.005+0000 7fc414c00640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-04-16T17:14:43.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.006+0000 7fc414c00640 1 -- 172.21.15.45:0/3842112513 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4180aa060 con 0x562b9f153e90 2024-04-16T17:14:43.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.006+0000 7fc414c00640 1 -- 172.21.15.45:0/3842112513 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9496+0+0 (secure 0 0 0) 0x7fc4180a21e0 con 0x562b9f153e90 2024-04-16T17:14:43.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.006+0000 7fc430969740 1 -- 172.21.15.45:0/3842112513 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc410005520 con 0x562b9f153e90 2024-04-16T17:14:43.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.006+0000 7fc414c00640 1 -- 172.21.15.45:0/3842112513 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7fc4180b1050 con 0x562b9f153e90 2024-04-16T17:14:43.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.006+0000 7fc430969740 10 client.27791 fetch_fsmap learned FSMap version 42 2024-04-16T17:14:43.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.006+0000 7fc430969740 1 -- 172.21.15.45:0/3842112513 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x562b9f1d4790 con 0x562b9f153e90 2024-04-16T17:14:43.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.006+0000 7fc414c00640 1 -- 172.21.15.45:0/3842112513 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc410005520 con 0x562b9f153e90 2024-04-16T17:14:43.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.006+0000 7fc414c00640 1 -- 172.21.15.45:0/3842112513 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 42) ==== 8362+0+0 (secure 0 0 0) 0x7fc4180b11f0 con 0x562b9f153e90 2024-04-16T17:14:43.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.006+0000 7fc430969740 10 client.27791 fetch_fsmap finished waiting for FSMap version 42 2024-04-16T17:14:43.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.006+0000 7fc430969740 10 client.27791 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:14:43.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.006+0000 7fc430969740 20 client.27791 populate_metadata read hostname 'smithi045' 2024-04-16T17:14:43.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.006+0000 7fc430969740 1 --2- 172.21.15.45:0/3842112513 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x562b9f1d4cf0 0x562b9f1d7110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:43.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.006+0000 7fc430969740 4 client.27791 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-16T17:14:43.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.006+0000 7fc430969740 1 -- 172.21.15.45:0/3842112513 --> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x562b9f0c0540 con 0x562b9f1d4cf0 2024-04-16T17:14:43.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.007+0000 7fc41e000640 1 --2- 172.21.15.45:0/3842112513 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x562b9f1d4cf0 0x562b9f1d7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:43.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.007+0000 7fc41e000640 1 --2- 172.21.15.45:0/3842112513 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x562b9f1d4cf0 0x562b9f1d7110 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3455f8a99f19567a server_cookie=94d4ac94d9cc8647 in_seq=0 out_seq=0 2024-04-16T17:14:43.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.007+0000 7fc414c00640 10 client.27791 ms_handle_connect on v2:172.21.15.104:6838/3660070592 2024-04-16T17:14:43.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.008+0000 7fc414c00640 1 -- 172.21.15.45:0/3842112513 <== mds.0 v2:172.21.15.104:6838/3660070592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x562b9f0c0540 con 0x562b9f1d4cf0 2024-04-16T17:14:43.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.008+0000 7fc414c00640 10 client.27791 handle_command_reply: tid=0 2024-04-16T17:14:43.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.008+0000 7fc430969740 1 client.27791 shutdown 2024-04-16T17:14:43.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.008+0000 7fc430969740 1 -- 172.21.15.45:0/3842112513 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562b9f153e90 msgr2=0x562b9f1654f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:43.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.008+0000 7fc430969740 1 --2- 172.21.15.45:0/3842112513 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562b9f153e90 0x562b9f1654f0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fc41809bf40 tx=0x7fc4180a20c0 comp rx=0 tx=0).stop 2024-04-16T17:14:43.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.008+0000 7fc430969740 1 -- 172.21.15.45:0/3842112513 shutdown_connections 2024-04-16T17:14:43.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.008+0000 7fc430969740 1 --2- 172.21.15.45:0/3842112513 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x562b9f1d4cf0 0x562b9f1d7110 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:43.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.008+0000 7fc430969740 1 --2- 172.21.15.45:0/3842112513 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562b9f15d390 0x562b9f162670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:43.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.008+0000 7fc430969740 1 --2- 172.21.15.45:0/3842112513 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562b9f1547b0 0x562b9f165a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:43.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.008+0000 7fc430969740 1 --2- 172.21.15.45:0/3842112513 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562b9f153e90 0x562b9f1654f0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:43.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.008+0000 7fc414c00640 0 client.27791 ms_handle_reset on v2:172.21.15.104:6838/3660070592 2024-04-16T17:14:43.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.008+0000 7fc430969740 1 -- 172.21.15.45:0/3842112513 >> 172.21.15.45:0/3842112513 conn(0x562b9f14e5c0 msgr2=0x562b9f15a340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:14:43.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.009+0000 7fc430969740 1 -- 172.21.15.45:0/3842112513 shutdown_connections 2024-04-16T17:14:43.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.009+0000 7fc430969740 1 -- 172.21.15.45:0/3842112513 wait complete. 2024-04-16T17:14:43.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.009+0000 7fc430969740 20 client.27791 trim_cache size 0 max 16384 2024-04-16T17:14:43.017 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-16T17:14:43.017 INFO:teuthology.orchestra.run.smithi045.stdout: "return_code": 0, 2024-04-16T17:14:43.017 INFO:teuthology.orchestra.run.smithi045.stdout: "scrub_tag": "140bb269-a067-46c8-ab3a-48a5fe45cc65", 2024-04-16T17:14:43.017 INFO:teuthology.orchestra.run.smithi045.stdout: "mode": "asynchronous" 2024-04-16T17:14:43.017 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-16T17:14:43.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.009+0000 7fc427e00640 1 -- 172.21.15.45:0/1978611957 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fc40c0a3d20 msgr2=0x7fc40c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:43.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.009+0000 7fc427e00640 1 --2- 172.21.15.45:0/1978611957 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fc40c0a3d20 0x7fc40c0a61e0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fc420009fd0 tx=0x7fc420009f60 comp rx=0 tx=0).stop 2024-04-16T17:14:43.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.009+0000 7fc427e00640 1 -- 172.21.15.45:0/1978611957 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc428102110 msgr2=0x7fc428074300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:43.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.009+0000 7fc427e00640 1 --2- 172.21.15.45:0/1978611957 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc428102110 0x7fc428074300 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fc41800e9f0 tx=0x7fc41800eeb0 comp rx=0 tx=0).stop 2024-04-16T17:14:43.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.010+0000 7fc427e00640 1 -- 172.21.15.45:0/1978611957 shutdown_connections 2024-04-16T17:14:43.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.010+0000 7fc427e00640 1 --2- 172.21.15.45:0/1978611957 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fc40c0a3d20 0x7fc40c0a61e0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:43.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.010+0000 7fc427e00640 1 --2- 172.21.15.45:0/1978611957 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc4281071e0 0x7fc428074840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:43.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.010+0000 7fc427e00640 1 --2- 172.21.15.45:0/1978611957 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc428074d80 0x7fc4281a69b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:43.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.010+0000 7fc427e00640 1 --2- 172.21.15.45:0/1978611957 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc428102110 0x7fc428074300 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:43.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.010+0000 7fc427e00640 1 -- 172.21.15.45:0/1978611957 >> 172.21.15.45:0/1978611957 conn(0x7fc4280fd9d0 msgr2=0x7fc4280ffd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:14:43.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.010+0000 7fc427e00640 1 -- 172.21.15.45:0/1978611957 shutdown_connections 2024-04-16T17:14:43.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.010+0000 7fc427e00640 1 -- 172.21.15.45:0/1978611957 wait complete. 2024-04-16T17:14:43.030 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-16T17:14:43.030 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-16T17:14:43.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.229+0000 7f6debe00640 1 -- 172.21.15.45:0/667304081 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6dec101000 msgr2=0x7f6dec1013e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:43.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.229+0000 7f6debe00640 1 --2- 172.21.15.45:0/667304081 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6dec101000 0x7f6dec1013e0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f6ddc009900 tx=0x7f6ddc02bc40 comp rx=0 tx=0).stop 2024-04-16T17:14:43.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.229+0000 7f6debe00640 1 -- 172.21.15.45:0/667304081 shutdown_connections 2024-04-16T17:14:43.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.229+0000 7f6debe00640 1 --2- 172.21.15.45:0/667304081 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6dec1060d0 0x7f6dec10dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:43.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.229+0000 7f6debe00640 1 --2- 172.21.15.45:0/667304081 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6dec1019b0 0x7f6dec1059f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:43.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.229+0000 7f6debe00640 1 --2- 172.21.15.45:0/667304081 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6dec101000 0x7f6dec1013e0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:43.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.229+0000 7f6debe00640 1 -- 172.21.15.45:0/667304081 >> 172.21.15.45:0/667304081 conn(0x7f6dec0fc880 msgr2=0x7f6dec0feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:14:43.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.229+0000 7f6debe00640 1 -- 172.21.15.45:0/667304081 shutdown_connections 2024-04-16T17:14:43.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.229+0000 7f6debe00640 1 -- 172.21.15.45:0/667304081 wait complete. 2024-04-16T17:14:43.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.230+0000 7f6debe00640 1 Processor -- start 2024-04-16T17:14:43.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.230+0000 7f6debe00640 1 -- start start 2024-04-16T17:14:43.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.231+0000 7f6debe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6dec101000 0x7f6dec1a0690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:43.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.231+0000 7f6debe00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6dec1060d0 0x7f6dec1a0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:43.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.231+0000 7f6debe00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6dec19a800 0x7f6dec19acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:43.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.231+0000 7f6debe00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f6dec111190 con 0x7f6dec101000 2024-04-16T17:14:43.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.231+0000 7f6debe00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f6dec111010 con 0x7f6dec19a800 2024-04-16T17:14:43.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.231+0000 7f6debe00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f6dec111310 con 0x7f6dec1060d0 2024-04-16T17:14:43.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.231+0000 7f6deaa00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6dec101000 0x7f6dec1a0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:43.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.231+0000 7f6deaa00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6dec101000 0x7f6dec1a0690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59468/0 (socket says 172.21.15.45:59468) 2024-04-16T17:14:43.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.231+0000 7f6deaa00640 1 -- 172.21.15.45:0/2943040943 learned_addr learned my addr 172.21.15.45:0/2943040943 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:14:43.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.231+0000 7f6dea000640 1 --2- 172.21.15.45:0/2943040943 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6dec1060d0 0x7f6dec1a0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:43.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.231+0000 7f6deb400640 1 --2- 172.21.15.45:0/2943040943 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6dec19a800 0x7f6dec19acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:43.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.231+0000 7f6dea000640 1 -- 172.21.15.45:0/2943040943 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6dec19a800 msgr2=0x7f6dec19acb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:43.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.231+0000 7f6dea000640 1 --2- 172.21.15.45:0/2943040943 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6dec19a800 0x7f6dec19acb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:43.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.231+0000 7f6dea000640 1 -- 172.21.15.45:0/2943040943 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6dec101000 msgr2=0x7f6dec1a0690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:43.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.231+0000 7f6dea000640 1 --2- 172.21.15.45:0/2943040943 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6dec101000 0x7f6dec1a0690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:43.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.231+0000 7f6dea000640 1 -- 172.21.15.45:0/2943040943 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6dec19b390 con 0x7f6dec1060d0 2024-04-16T17:14:43.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.231+0000 7f6deaa00640 1 --2- 172.21.15.45:0/2943040943 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6dec101000 0x7f6dec1a0690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:14:43.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.232+0000 7f6dea000640 1 --2- 172.21.15.45:0/2943040943 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6dec1060d0 0x7f6dec1a0bd0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f6dd800e9c0 tx=0x7f6dd800ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:14:43.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.232+0000 7f6de3400640 1 -- 172.21.15.45:0/2943040943 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dd800ccc0 con 0x7f6dec1060d0 2024-04-16T17:14:43.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.232+0000 7f6debe00640 1 -- 172.21.15.45:0/2943040943 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6dec19b680 con 0x7f6dec1060d0 2024-04-16T17:14:43.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.232+0000 7f6de3400640 1 -- 172.21.15.45:0/2943040943 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6dd8010910 con 0x7f6dec1060d0 2024-04-16T17:14:43.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.232+0000 7f6de3400640 1 -- 172.21.15.45:0/2943040943 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dd8018880 con 0x7f6dec1060d0 2024-04-16T17:14:43.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.232+0000 7f6debe00640 1 -- 172.21.15.45:0/2943040943 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6dec1a7860 con 0x7f6dec1060d0 2024-04-16T17:14:43.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.236+0000 7f6df4799740 1 Processor -- start 2024-04-16T17:14:43.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.236+0000 7f6df4799740 1 -- start start 2024-04-16T17:14:43.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.237+0000 7f6df4799740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5650bc6cb390 0x5650bc6cb770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:43.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.237+0000 7f6df4799740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5650bc6cbcb0 0x5650bc6d4350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:43.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.237+0000 7f6df4799740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5650bc6d4890 0x5650bc6d6c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:43.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.237+0000 7f6df4799740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5650bc631120 con 0x5650bc6cbcb0 2024-04-16T17:14:43.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.237+0000 7f6df4799740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5650bc5d7340 con 0x5650bc6d4890 2024-04-16T17:14:43.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.237+0000 7f6df4799740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5650bc62d0e0 con 0x5650bc6cb390 2024-04-16T17:14:43.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.237+0000 7f6deaa00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5650bc6cbcb0 0x5650bc6d4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:43.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.237+0000 7f6deaa00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5650bc6cbcb0 0x5650bc6d4350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59482/0 (socket says 172.21.15.45:59482) 2024-04-16T17:14:43.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.237+0000 7f6deaa00640 1 -- 172.21.15.45:0/4239269255 learned_addr learned my addr 172.21.15.45:0/4239269255 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:14:43.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.237+0000 7f6deb400640 1 --2- 172.21.15.45:0/4239269255 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5650bc6cb390 0x5650bc6cb770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:43.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.237+0000 7f6de3400640 1 -- 172.21.15.45:0/2943040943 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f6dd8010ac0 con 0x7f6dec1060d0 2024-04-16T17:14:43.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.238+0000 7f6deb400640 1 -- 172.21.15.45:0/4239269255 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5650bc6d4890 msgr2=0x5650bc6d6c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:14:43.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.238+0000 7f6deb400640 1 --2- 172.21.15.45:0/4239269255 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5650bc6d4890 0x5650bc6d6c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:43.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.238+0000 7f6deb400640 1 -- 172.21.15.45:0/4239269255 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5650bc6cbcb0 msgr2=0x5650bc6d4350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:14:43.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.238+0000 7f6deb400640 1 --2- 172.21.15.45:0/4239269255 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5650bc6cbcb0 0x5650bc6d4350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:43.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.238+0000 7f6deb400640 1 -- 172.21.15.45:0/4239269255 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5650bc40d7d0 con 0x5650bc6cb390 2024-04-16T17:14:43.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.238+0000 7f6deaa00640 1 --2- 172.21.15.45:0/4239269255 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5650bc6cbcb0 0x5650bc6d4350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:14:43.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.238+0000 7f6deb400640 1 --2- 172.21.15.45:0/4239269255 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5650bc6cb390 0x5650bc6cb770 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f6de4009f00 tx=0x7f6de4011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4ea5b315868b1bfc server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:14:43.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.239+0000 7f6dd7e00640 1 -- 172.21.15.45:0/4239269255 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6de4004d20 con 0x5650bc6cb390 2024-04-16T17:14:43.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.239+0000 7f6dd7e00640 1 -- 172.21.15.45:0/4239269255 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6de4004050 con 0x5650bc6cb390 2024-04-16T17:14:43.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.239+0000 7f6dd7e00640 1 -- 172.21.15.45:0/4239269255 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6de4005280 con 0x5650bc6cb390 2024-04-16T17:14:43.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.239+0000 7f6de3400640 1 --2- 172.21.15.45:0/2943040943 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f6dd00a3d20 0x7f6dd00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:43.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.239+0000 7f6de3400640 1 -- 172.21.15.45:0/2943040943 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9496+0+0 (secure 0 0 0) 0x7f6dd80d0f60 con 0x7f6dec1060d0 2024-04-16T17:14:43.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.239+0000 7f6deaa00640 1 --2- 172.21.15.45:0/2943040943 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f6dd00a3d20 0x7f6dd00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:43.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.239+0000 7f6df4799740 1 -- 172.21.15.45:0/4239269255 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5650bc6cb390 msgr2=0x5650bc6cb770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:43.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.239+0000 7f6df4799740 1 --2- 172.21.15.45:0/4239269255 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5650bc6cb390 0x5650bc6cb770 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f6de4009f00 tx=0x7f6de4011810 comp rx=0 tx=0).stop 2024-04-16T17:14:43.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.240+0000 7f6df4799740 1 -- 172.21.15.45:0/4239269255 shutdown_connections 2024-04-16T17:14:43.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.240+0000 7f6df4799740 1 --2- 172.21.15.45:0/4239269255 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5650bc6d4890 0x5650bc6d6c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:43.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.240+0000 7f6df4799740 1 --2- 172.21.15.45:0/4239269255 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5650bc6cbcb0 0x5650bc6d4350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:43.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.240+0000 7f6df4799740 1 --2- 172.21.15.45:0/4239269255 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5650bc6cb390 0x5650bc6cb770 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:43.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.240+0000 7f6df4799740 1 -- 172.21.15.45:0/4239269255 >> 172.21.15.45:0/4239269255 conn(0x5650bc6c5ac0 msgr2=0x5650bc6c5ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:14:43.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.240+0000 7f6df4799740 1 -- 172.21.15.45:0/4239269255 shutdown_connections 2024-04-16T17:14:43.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.240+0000 7f6df4799740 1 -- 172.21.15.45:0/4239269255 wait complete. 2024-04-16T17:14:43.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.240+0000 7f6deaa00640 1 --2- 172.21.15.45:0/2943040943 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f6dd00a3d20 0x7f6dd00a61e0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f6ddc0096c0 tx=0x7f6ddc02e040 comp rx=0 tx=0).ready entity=mgr.25411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:14:43.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.240+0000 7f6df4799740 1 -- start start 2024-04-16T17:14:43.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.240+0000 7f6df4799740 1 Processor -- start 2024-04-16T17:14:43.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.241+0000 7f6df4799740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5650bc6cb390 0x5650bc6d18e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:43.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.241+0000 7f6df4799740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5650bc6cbcb0 0x5650bc6d1e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:43.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.241+0000 7f6df4799740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5650bc6d4890 0x5650bc6d2360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:43.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.241+0000 7f6df4799740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5650bc6d7680 con 0x5650bc6d4890 2024-04-16T17:14:43.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.241+0000 7f6df4799740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5650bc5ea710 con 0x5650bc6cbcb0 2024-04-16T17:14:43.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.241+0000 7f6df4799740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5650bc6d7800 con 0x5650bc6cb390 2024-04-16T17:14:43.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.241+0000 7f6df4799740 10 client.0 _ll_register_callbacks cb 0x5650bc5e5af0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:14:43.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.241+0000 7f6deb400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5650bc6d4890 0x5650bc6d2360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:43.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.241+0000 7f6deaa00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5650bc6cb390 0x5650bc6d18e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:43.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.241+0000 7f6deaa00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5650bc6cb390 0x5650bc6d18e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:34442/0 (socket says 172.21.15.45:34442) 2024-04-16T17:14:43.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.241+0000 7f6deaa00640 1 -- 172.21.15.45:0/4260854381 learned_addr learned my addr 172.21.15.45:0/4260854381 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:14:43.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.241+0000 7f6dea000640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5650bc6cbcb0 0x5650bc6d1e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:43.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.242+0000 7f6deaa00640 1 -- 172.21.15.45:0/4260854381 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5650bc6cbcb0 msgr2=0x5650bc6d1e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:14:43.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.242+0000 7f6deaa00640 1 --2- 172.21.15.45:0/4260854381 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5650bc6cbcb0 0x5650bc6d1e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:43.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.242+0000 7f6deaa00640 1 -- 172.21.15.45:0/4260854381 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5650bc6d4890 msgr2=0x5650bc6d2360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:14:43.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.242+0000 7f6deaa00640 1 --2- 172.21.15.45:0/4260854381 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5650bc6d4890 0x5650bc6d2360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:43.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.242+0000 7f6deaa00640 1 -- 172.21.15.45:0/4260854381 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5650bc3433f0 con 0x5650bc6cb390 2024-04-16T17:14:43.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.242+0000 7f6deaa00640 1 --2- 172.21.15.45:0/4260854381 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5650bc6cb390 0x5650bc6d18e0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f6ddc004dc0 tx=0x7f6ddc02bc40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1b299ade2446fd55 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:14:43.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.242+0000 7f6dd4c00640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-04-16T17:14:43.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.242+0000 7f6dd4c00640 1 -- 172.21.15.45:0/4260854381 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ddc04f070 con 0x5650bc6cb390 2024-04-16T17:14:43.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.243+0000 7f6df4799740 1 -- 172.21.15.45:0/4260854381 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6de4004050 con 0x5650bc6cb390 2024-04-16T17:14:43.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.243+0000 7f6dd4c00640 1 -- 172.21.15.45:0/4260854381 <== mon.1 v2:172.21.15.117:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9496+0+0 (secure 0 0 0) 0x7f6ddc004450 con 0x5650bc6cb390 2024-04-16T17:14:43.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.243+0000 7f6dd4c00640 1 -- 172.21.15.45:0/4260854381 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f6ddc05b050 con 0x5650bc6cb390 2024-04-16T17:14:43.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.243+0000 7f6df4799740 10 client.27745 fetch_fsmap learned FSMap version 42 2024-04-16T17:14:43.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.243+0000 7f6df4799740 1 -- 172.21.15.45:0/4260854381 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x5650bc3030f0 con 0x5650bc6cb390 2024-04-16T17:14:43.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.243+0000 7f6dd4c00640 1 -- 172.21.15.45:0/4260854381 <== mon.1 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6de4004050 con 0x5650bc6cb390 2024-04-16T17:14:43.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.243+0000 7f6dd4c00640 1 -- 172.21.15.45:0/4260854381 <== mon.1 v2:172.21.15.117:3300/0 5 ==== fsmap(e 42) ==== 8362+0+0 (secure 0 0 0) 0x7f6ddc003ab0 con 0x5650bc6cb390 2024-04-16T17:14:43.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.243+0000 7f6df4799740 10 client.27745 fetch_fsmap finished waiting for FSMap version 42 2024-04-16T17:14:43.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.243+0000 7f6df4799740 10 client.27745 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:14:43.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.243+0000 7f6df4799740 20 client.27745 populate_metadata read hostname 'smithi045' 2024-04-16T17:14:43.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.243+0000 7f6df4799740 1 --2- 172.21.15.45:0/4260854381 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x5650bc6e4c30 0x5650bc6e7060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:43.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.244+0000 7f6df4799740 4 client.27745 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:14:43.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.244+0000 7f6df4799740 1 -- 172.21.15.45:0/4260854381 --> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5650bc63eb40 con 0x5650bc6e4c30 2024-04-16T17:14:43.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.244+0000 7f6dea000640 1 --2- 172.21.15.45:0/4260854381 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x5650bc6e4c30 0x5650bc6e7060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:43.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.244+0000 7f6dea000640 1 --2- 172.21.15.45:0/4260854381 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x5650bc6e4c30 0x5650bc6e7060 crc :-1 s=READY pgs=223 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=64201ec5e109a489 server_cookie=cdb555ba414c5942 in_seq=0 out_seq=0 2024-04-16T17:14:43.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.244+0000 7f6dd4c00640 10 client.27745 ms_handle_connect on v2:172.21.15.104:6838/3660070592 2024-04-16T17:14:43.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.245+0000 7f6dd4c00640 1 -- 172.21.15.45:0/4260854381 <== mds.0 v2:172.21.15.104:6838/3660070592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5650bc63eb40 con 0x5650bc6e4c30 2024-04-16T17:14:43.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.245+0000 7f6dd4c00640 10 client.27745 handle_command_reply: tid=0 2024-04-16T17:14:43.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.245+0000 7f6df4799740 1 client.27745 shutdown 2024-04-16T17:14:43.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.245+0000 7f6df4799740 1 -- 172.21.15.45:0/4260854381 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5650bc6cb390 msgr2=0x5650bc6d18e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:43.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.245+0000 7f6df4799740 1 --2- 172.21.15.45:0/4260854381 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5650bc6cb390 0x5650bc6d18e0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f6ddc004dc0 tx=0x7f6ddc02bc40 comp rx=0 tx=0).stop 2024-04-16T17:14:43.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.245+0000 7f6df4799740 1 -- 172.21.15.45:0/4260854381 shutdown_connections 2024-04-16T17:14:43.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.246+0000 7f6df4799740 1 --2- 172.21.15.45:0/4260854381 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5650bc6d4890 0x5650bc6d2360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:43.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.246+0000 7f6df4799740 1 --2- 172.21.15.45:0/4260854381 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5650bc6cbcb0 0x5650bc6d1e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:43.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.246+0000 7f6df4799740 1 --2- 172.21.15.45:0/4260854381 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x5650bc6e4c30 0x5650bc6e7060 crc :-1 s=READY pgs=223 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:43.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.246+0000 7f6df4799740 1 --2- 172.21.15.45:0/4260854381 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5650bc6cb390 0x5650bc6d18e0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:43.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.246+0000 7f6df4799740 1 -- 172.21.15.45:0/4260854381 >> 172.21.15.45:0/4260854381 conn(0x5650bc6c5ac0 msgr2=0x5650bc6c7810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:14:43.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.246+0000 7f6dd4c00640 0 client.27745 ms_handle_reset on v2:172.21.15.104:6838/3660070592 2024-04-16T17:14:43.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.246+0000 7f6df4799740 1 -- 172.21.15.45:0/4260854381 shutdown_connections 2024-04-16T17:14:43.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.246+0000 7f6df4799740 1 -- 172.21.15.45:0/4260854381 wait complete. 2024-04-16T17:14:43.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.246+0000 7f6df4799740 20 client.27745 trim_cache size 0 max 16384 2024-04-16T17:14:43.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.265+0000 7f6df4799740 1 Processor -- start 2024-04-16T17:14:43.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.265+0000 7f6df4799740 1 -- start start 2024-04-16T17:14:43.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.266+0000 7f6df4799740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5650bc6cb390 0x5650bc6d4010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:43.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.266+0000 7f6df4799740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5650bc6cbcb0 0x5650bc6d0080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:43.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.266+0000 7f6df4799740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5650bc6d4890 0x5650bc6e05a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:43.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.266+0000 7f6df4799740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f6dec1b3f10 con 0x5650bc6cbcb0 2024-04-16T17:14:43.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.266+0000 7f6df4799740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f6dec1b1780 con 0x5650bc6cb390 2024-04-16T17:14:43.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.266+0000 7f6df4799740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f6dec1af5f0 con 0x5650bc6d4890 2024-04-16T17:14:43.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.266+0000 7f6dea000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5650bc6cbcb0 0x5650bc6d0080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:43.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.266+0000 7f6deaa00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5650bc6cb390 0x5650bc6d4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:43.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.266+0000 7f6dea000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5650bc6cbcb0 0x5650bc6d0080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59502/0 (socket says 172.21.15.45:59502) 2024-04-16T17:14:43.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.266+0000 7f6dea000640 1 -- 172.21.15.45:0/626933613 learned_addr learned my addr 172.21.15.45:0/626933613 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:14:43.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.266+0000 7f6deb400640 1 --2- 172.21.15.45:0/626933613 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5650bc6d4890 0x5650bc6e05a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:43.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.266+0000 7f6dea000640 1 -- 172.21.15.45:0/626933613 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5650bc6d4890 msgr2=0x5650bc6e05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:14:43.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.266+0000 7f6dea000640 1 --2- 172.21.15.45:0/626933613 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5650bc6d4890 0x5650bc6e05a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:43.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.266+0000 7f6dea000640 1 -- 172.21.15.45:0/626933613 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5650bc6cb390 msgr2=0x5650bc6d4010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:14:43.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.266+0000 7f6dea000640 1 --2- 172.21.15.45:0/626933613 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5650bc6cb390 0x5650bc6d4010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:43.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.266+0000 7f6dea000640 1 -- 172.21.15.45:0/626933613 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5650bc6cbae0 con 0x5650bc6cbcb0 2024-04-16T17:14:43.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.267+0000 7f6dea000640 1 --2- 172.21.15.45:0/626933613 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5650bc6cbcb0 0x5650bc6d0080 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f6dd809cb10 tx=0x7f6dd8023040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1eabfa02dea87de server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:14:43.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.267+0000 7f6dd7400640 1 -- 172.21.15.45:0/626933613 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dd809ac20 con 0x5650bc6cbcb0 2024-04-16T17:14:43.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.267+0000 7f6dd7400640 1 -- 172.21.15.45:0/626933613 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6dd80bd0b0 con 0x5650bc6cbcb0 2024-04-16T17:14:43.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.267+0000 7f6dd7400640 1 -- 172.21.15.45:0/626933613 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dd809bfd0 con 0x5650bc6cbcb0 2024-04-16T17:14:43.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.268+0000 7f6df4799740 1 -- 172.21.15.45:0/626933613 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5650bc6cbcb0 msgr2=0x5650bc6d0080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:43.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.268+0000 7f6df4799740 1 --2- 172.21.15.45:0/626933613 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5650bc6cbcb0 0x5650bc6d0080 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f6dd809cb10 tx=0x7f6dd8023040 comp rx=0 tx=0).stop 2024-04-16T17:14:43.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.268+0000 7f6df4799740 1 -- 172.21.15.45:0/626933613 shutdown_connections 2024-04-16T17:14:43.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.268+0000 7f6df4799740 1 --2- 172.21.15.45:0/626933613 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5650bc6d4890 0x5650bc6e05a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:43.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.268+0000 7f6df4799740 1 --2- 172.21.15.45:0/626933613 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5650bc6cbcb0 0x5650bc6d0080 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:43.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.268+0000 7f6df4799740 1 --2- 172.21.15.45:0/626933613 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5650bc6cb390 0x5650bc6d4010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:43.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.268+0000 7f6df4799740 1 -- 172.21.15.45:0/626933613 >> 172.21.15.45:0/626933613 conn(0x5650bc6c5ac0 msgr2=0x5650bc6d2130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:14:43.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.268+0000 7f6df4799740 1 -- 172.21.15.45:0/626933613 shutdown_connections 2024-04-16T17:14:43.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.268+0000 7f6df4799740 1 -- 172.21.15.45:0/626933613 wait complete. 2024-04-16T17:14:43.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.268+0000 7f6df4799740 1 -- start start 2024-04-16T17:14:43.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.269+0000 7f6df4799740 1 Processor -- start 2024-04-16T17:14:43.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.269+0000 7f6df4799740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5650bc6cb390 0x5650bc6dce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:43.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.269+0000 7f6df4799740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5650bc6cbcb0 0x5650bc6d8170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:43.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.269+0000 7f6df4799740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5650bc6d4890 0x5650bc6d86b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:43.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.269+0000 7f6df4799740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f6dec1afc40 con 0x5650bc6cb390 2024-04-16T17:14:43.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.269+0000 7f6df4799740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f6dec1b10b0 con 0x5650bc6d4890 2024-04-16T17:14:43.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.269+0000 7f6df4799740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f6dec1b0310 con 0x5650bc6cbcb0 2024-04-16T17:14:43.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.269+0000 7f6df4799740 10 client.0 _ll_register_callbacks cb 0x7f6ddc043070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:14:43.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.269+0000 7f6deaa00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5650bc6cb390 0x5650bc6dce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:43.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.269+0000 7f6deaa00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5650bc6cb390 0x5650bc6dce00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59510/0 (socket says 172.21.15.45:59510) 2024-04-16T17:14:43.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.269+0000 7f6deaa00640 1 -- 172.21.15.45:0/3914630942 learned_addr learned my addr 172.21.15.45:0/3914630942 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:14:43.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.269+0000 7f6deb400640 1 --2- 172.21.15.45:0/3914630942 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5650bc6d4890 0x5650bc6d86b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:43.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.269+0000 7f6dea000640 1 --2- 172.21.15.45:0/3914630942 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5650bc6cbcb0 0x5650bc6d8170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:43.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.270+0000 7f6deaa00640 1 -- 172.21.15.45:0/3914630942 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5650bc6cbcb0 msgr2=0x5650bc6d8170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:14:43.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.270+0000 7f6deaa00640 1 --2- 172.21.15.45:0/3914630942 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5650bc6cbcb0 0x5650bc6d8170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:43.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.270+0000 7f6deaa00640 1 -- 172.21.15.45:0/3914630942 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5650bc6d4890 msgr2=0x5650bc6d86b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:14:43.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.270+0000 7f6deaa00640 1 --2- 172.21.15.45:0/3914630942 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5650bc6d4890 0x5650bc6d86b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:43.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.270+0000 7f6deaa00640 1 -- 172.21.15.45:0/3914630942 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5650bc6d8ee0 con 0x5650bc6cb390 2024-04-16T17:14:43.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.270+0000 7f6deaa00640 1 --2- 172.21.15.45:0/3914630942 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5650bc6cb390 0x5650bc6dce00 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f6ddc036040 tx=0x7f6ddc005b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d3d1538b584ad1f server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:14:43.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.270+0000 7f6dd4c00640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-16T17:14:43.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.270+0000 7f6dd4c00640 1 -- 172.21.15.45:0/3914630942 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ddc04f040 con 0x5650bc6cb390 2024-04-16T17:14:43.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.270+0000 7f6dd4c00640 1 -- 172.21.15.45:0/3914630942 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9496+0+0 (secure 0 0 0) 0x7f6ddc0544e0 con 0x5650bc6cb390 2024-04-16T17:14:43.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.270+0000 7f6df4799740 1 -- 172.21.15.45:0/3914630942 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6dd80bd0b0 con 0x5650bc6cb390 2024-04-16T17:14:43.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.271+0000 7f6dd4c00640 1 -- 172.21.15.45:0/3914630942 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f6ddc06c050 con 0x5650bc6cb390 2024-04-16T17:14:43.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.271+0000 7f6df4799740 10 client.17646 fetch_fsmap learned FSMap version 42 2024-04-16T17:14:43.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.271+0000 7f6df4799740 1 -- 172.21.15.45:0/3914630942 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x5650bc753920 con 0x5650bc6cb390 2024-04-16T17:14:43.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.271+0000 7f6dd4c00640 1 -- 172.21.15.45:0/3914630942 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6dd80bd0b0 con 0x5650bc6cb390 2024-04-16T17:14:43.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.271+0000 7f6dd4c00640 1 -- 172.21.15.45:0/3914630942 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 42) ==== 8362+0+0 (secure 0 0 0) 0x7f6ddc003ae0 con 0x5650bc6cb390 2024-04-16T17:14:43.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.271+0000 7f6df4799740 10 client.17646 fetch_fsmap finished waiting for FSMap version 42 2024-04-16T17:14:43.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.271+0000 7f6df4799740 10 client.17646 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:14:43.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.271+0000 7f6df4799740 20 client.17646 populate_metadata read hostname 'smithi045' 2024-04-16T17:14:43.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.271+0000 7f6df4799740 1 --2- 172.21.15.45:0/3914630942 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x5650bc754090 0x5650bc7564b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:14:43.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.271+0000 7f6df4799740 4 client.17646 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-16T17:14:43.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.271+0000 7f6df4799740 1 -- 172.21.15.45:0/3914630942 --> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5650bc6389b0 con 0x5650bc754090 2024-04-16T17:14:43.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.272+0000 7f6dea000640 1 --2- 172.21.15.45:0/3914630942 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x5650bc754090 0x5650bc7564b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:14:43.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.272+0000 7f6dd4c00640 10 client.17646 ms_handle_connect on v2:172.21.15.104:6838/3660070592 2024-04-16T17:14:43.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.272+0000 7f6dea000640 1 --2- 172.21.15.45:0/3914630942 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x5650bc754090 0x5650bc7564b0 crc :-1 s=READY pgs=224 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7e866c9c5c7144c4 server_cookie=29884fe0424764a6 in_seq=0 out_seq=0 2024-04-16T17:14:43.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.272+0000 7f6dd4c00640 1 -- 172.21.15.45:0/3914630942 <== mds.0 v2:172.21.15.104:6838/3660070592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5650bc6389b0 con 0x5650bc754090 2024-04-16T17:14:43.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.272+0000 7f6dd4c00640 10 client.17646 handle_command_reply: tid=0 2024-04-16T17:14:43.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.272+0000 7f6df4799740 1 client.17646 shutdown 2024-04-16T17:14:43.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.273+0000 7f6df4799740 1 -- 172.21.15.45:0/3914630942 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5650bc6cb390 msgr2=0x5650bc6dce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:43.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.273+0000 7f6df4799740 1 --2- 172.21.15.45:0/3914630942 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5650bc6cb390 0x5650bc6dce00 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f6ddc036040 tx=0x7f6ddc005b50 comp rx=0 tx=0).stop 2024-04-16T17:14:43.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.273+0000 7f6df4799740 1 -- 172.21.15.45:0/3914630942 shutdown_connections 2024-04-16T17:14:43.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.273+0000 7f6df4799740 1 --2- 172.21.15.45:0/3914630942 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x5650bc754090 0x5650bc7564b0 crc :-1 s=READY pgs=224 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:43.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.273+0000 7f6df4799740 1 --2- 172.21.15.45:0/3914630942 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5650bc6cbcb0 0x5650bc6d8170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:43.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.273+0000 7f6df4799740 1 --2- 172.21.15.45:0/3914630942 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5650bc6d4890 0x5650bc6d86b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:43.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.273+0000 7f6dd4c00640 0 client.17646 ms_handle_reset on v2:172.21.15.104:6838/3660070592 2024-04-16T17:14:43.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.273+0000 7f6df4799740 1 --2- 172.21.15.45:0/3914630942 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5650bc6cb390 0x5650bc6dce00 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:43.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.273+0000 7f6df4799740 1 -- 172.21.15.45:0/3914630942 >> 172.21.15.45:0/3914630942 conn(0x5650bc6c5ac0 msgr2=0x5650bc6d16d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:14:43.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.273+0000 7f6df4799740 1 -- 172.21.15.45:0/3914630942 shutdown_connections 2024-04-16T17:14:43.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.273+0000 7f6df4799740 1 -- 172.21.15.45:0/3914630942 wait complete. 2024-04-16T17:14:43.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.273+0000 7f6df4799740 20 client.17646 trim_cache size 0 max 16384 2024-04-16T17:14:43.279 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-16T17:14:43.279 INFO:teuthology.orchestra.run.smithi045.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-16T17:14:43.279 INFO:teuthology.orchestra.run.smithi045.stdout: "scrubs": { 2024-04-16T17:14:43.280 INFO:teuthology.orchestra.run.smithi045.stdout: "140bb269-a067-46c8-ab3a-48a5fe45cc65": { 2024-04-16T17:14:43.280 INFO:teuthology.orchestra.run.smithi045.stdout: "path": "/", 2024-04-16T17:14:43.280 INFO:teuthology.orchestra.run.smithi045.stdout: "tag": "140bb269-a067-46c8-ab3a-48a5fe45cc65", 2024-04-16T17:14:43.280 INFO:teuthology.orchestra.run.smithi045.stdout: "options": "recursive,force" 2024-04-16T17:14:43.280 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:14:43.280 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:14:43.280 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-16T17:14:43.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.274+0000 7f6debe00640 1 -- 172.21.15.45:0/2943040943 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f6dd00a3d20 msgr2=0x7f6dd00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:43.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.274+0000 7f6debe00640 1 --2- 172.21.15.45:0/2943040943 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f6dd00a3d20 0x7f6dd00a61e0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f6ddc0096c0 tx=0x7f6ddc02e040 comp rx=0 tx=0).stop 2024-04-16T17:14:43.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.274+0000 7f6debe00640 1 -- 172.21.15.45:0/2943040943 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6dec1060d0 msgr2=0x7f6dec1a0bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:14:43.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.274+0000 7f6debe00640 1 --2- 172.21.15.45:0/2943040943 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6dec1060d0 0x7f6dec1a0bd0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f6dd800e9c0 tx=0x7f6dd800ee80 comp rx=0 tx=0).stop 2024-04-16T17:14:43.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.274+0000 7f6debe00640 1 -- 172.21.15.45:0/2943040943 shutdown_connections 2024-04-16T17:14:43.282 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.274+0000 7f6debe00640 1 --2- 172.21.15.45:0/2943040943 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f6dd00a3d20 0x7f6dd00a61e0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:43.282 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.274+0000 7f6debe00640 1 --2- 172.21.15.45:0/2943040943 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6dec1060d0 0x7f6dec1a0bd0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:43.282 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.274+0000 7f6debe00640 1 --2- 172.21.15.45:0/2943040943 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6dec19a800 0x7f6dec19acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:43.282 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.274+0000 7f6debe00640 1 --2- 172.21.15.45:0/2943040943 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6dec101000 0x7f6dec1a0690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:14:43.282 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.274+0000 7f6debe00640 1 -- 172.21.15.45:0/2943040943 >> 172.21.15.45:0/2943040943 conn(0x7f6dec0fc880 msgr2=0x7f6dec0693b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:14:43.282 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.274+0000 7f6debe00640 1 -- 172.21.15.45:0/2943040943 shutdown_connections 2024-04-16T17:14:43.282 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:14:43.274+0000 7f6debe00640 1 -- 172.21.15.45:0/2943040943 wait complete. 2024-04-16T17:14:43.291 INFO:tasks.cephfs.filesystem:scrub status for tag:140bb269-a067-46c8-ab3a-48a5fe45cc65 - {'path': '/', 'tag': '140bb269-a067-46c8-ab3a-48a5fe45cc65', 'options': 'recursive,force'} 2024-04-16T17:14:43.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:14:43 smithi045 ceph-mon[31676]: scrub summary: active paths [/] 2024-04-16T17:14:43.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:14:43 smithi045 ceph-mon[31676]: scrub summary: idle+waiting paths [/] 2024-04-16T17:14:43.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:14:43 smithi045 ceph-mon[31676]: scrub summary: active paths [/] 2024-04-16T17:14:43.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:14:43 smithi045 ceph-mon[31676]: scrub summary: idle+waiting paths [/] 2024-04-16T17:14:43.804 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:14:43 smithi117 ceph-mon[36730]: scrub summary: active paths [/] 2024-04-16T17:14:43.804 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:14:43 smithi117 ceph-mon[36730]: scrub summary: idle+waiting paths [/] 2024-04-16T17:14:43.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:14:43 smithi117 ceph-mon[36730]: scrub summary: active paths [/] 2024-04-16T17:14:43.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:14:43 smithi117 ceph-mon[36730]: scrub summary: idle+waiting paths [/] 2024-04-16T17:14:43.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:14:43 smithi104 ceph-mon[36457]: scrub summary: active paths [/] 2024-04-16T17:14:43.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:14:43 smithi104 ceph-mon[36457]: scrub summary: idle+waiting paths [/] 2024-04-16T17:14:43.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:14:43 smithi104 ceph-mon[36457]: scrub summary: active paths [/] 2024-04-16T17:14:43.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:14:43 smithi104 ceph-mon[36457]: scrub summary: idle+waiting paths [/] 2024-04-16T17:14:44.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:14:44 smithi045 ceph-mon[31676]: scrub queued for path: / 2024-04-16T17:14:44.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:14:44 smithi045 ceph-mon[31676]: scrub summary: idle+waiting paths [/] 2024-04-16T17:14:44.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:14:44 smithi045 ceph-mon[31676]: scrub summary: active paths [/] 2024-04-16T17:14:44.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:14:44 smithi045 ceph-mon[31676]: scrub summary: active paths [/] 2024-04-16T17:14:44.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:14:44 smithi045 ceph-mon[31676]: scrub summary: idle+waiting paths [/] 2024-04-16T17:14:44.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:14:44 smithi045 ceph-mon[31676]: scrub summary: idle+waiting paths [/] 2024-04-16T17:14:44.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:14:44 smithi045 ceph-mon[31676]: pgmap v247: 129 pgs: 129 active+clean; 60 MiB data, 837 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 46 KiB/s wr, 15 op/s 2024-04-16T17:14:44.804 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:14:44 smithi117 ceph-mon[36730]: scrub queued for path: / 2024-04-16T17:14:44.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:14:44 smithi117 ceph-mon[36730]: scrub summary: idle+waiting paths [/] 2024-04-16T17:14:44.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:14:44 smithi117 ceph-mon[36730]: scrub summary: active paths [/] 2024-04-16T17:14:44.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:14:44 smithi117 ceph-mon[36730]: scrub summary: active paths [/] 2024-04-16T17:14:44.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:14:44 smithi117 ceph-mon[36730]: scrub summary: idle+waiting paths [/] 2024-04-16T17:14:44.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:14:44 smithi117 ceph-mon[36730]: scrub summary: idle+waiting paths [/] 2024-04-16T17:14:44.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:14:44 smithi117 ceph-mon[36730]: pgmap v247: 129 pgs: 129 active+clean; 60 MiB data, 837 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 46 KiB/s wr, 15 op/s 2024-04-16T17:14:44.828 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:14:44 smithi104 ceph-mon[36457]: scrub queued for path: / 2024-04-16T17:14:44.828 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:14:44 smithi104 ceph-mon[36457]: scrub summary: idle+waiting paths [/] 2024-04-16T17:14:44.828 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:14:44 smithi104 ceph-mon[36457]: scrub summary: active paths [/] 2024-04-16T17:14:44.828 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:14:44 smithi104 ceph-mon[36457]: scrub summary: active paths [/] 2024-04-16T17:14:44.828 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:14:44 smithi104 ceph-mon[36457]: scrub summary: idle+waiting paths [/] 2024-04-16T17:14:44.828 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:14:44 smithi104 ceph-mon[36457]: scrub summary: idle+waiting paths [/] 2024-04-16T17:14:44.828 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:14:44 smithi104 ceph-mon[36457]: pgmap v247: 129 pgs: 129 active+clean; 60 MiB data, 837 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 46 KiB/s wr, 15 op/s 2024-04-16T17:14:46.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:14:46 smithi045 ceph-mon[31676]: pgmap v248: 129 pgs: 129 active+clean; 60 MiB data, 837 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 46 KiB/s wr, 15 op/s 2024-04-16T17:14:46.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:14:46 smithi117 ceph-mon[36730]: pgmap v248: 129 pgs: 129 active+clean; 60 MiB data, 837 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 46 KiB/s wr, 15 op/s 2024-04-16T17:14:46.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:14:46 smithi104 ceph-mon[36457]: pgmap v248: 129 pgs: 129 active+clean; 60 MiB data, 837 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 46 KiB/s wr, 15 op/s 2024-04-16T17:14:48.726 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:14:48 smithi045 ceph-mon[31676]: pgmap v249: 129 pgs: 129 active+clean; 60 MiB data, 837 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 51 KiB/s wr, 25 op/s 2024-04-16T17:14:48.804 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:14:48 smithi117 ceph-mon[36730]: pgmap v249: 129 pgs: 129 active+clean; 60 MiB data, 837 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 51 KiB/s wr, 25 op/s 2024-04-16T17:14:48.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:14:48 smithi104 ceph-mon[36457]: pgmap v249: 129 pgs: 129 active+clean; 60 MiB data, 837 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 51 KiB/s wr, 25 op/s 2024-04-16T17:14:49.726 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:14:49 smithi045 ceph-mon[31676]: scrub summary: idle 2024-04-16T17:14:49.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:14:49 smithi045 ceph-mon[31676]: scrub summary: idle 2024-04-16T17:14:49.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:14:49 smithi045 ceph-mon[31676]: scrub summary: idle 2024-04-16T17:14:49.804 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:14:49 smithi117 ceph-mon[36730]: scrub summary: idle 2024-04-16T17:14:49.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:14:49 smithi117 ceph-mon[36730]: scrub summary: idle 2024-04-16T17:14:49.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:14:49 smithi117 ceph-mon[36730]: scrub summary: idle 2024-04-16T17:14:49.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:14:49 smithi104 ceph-mon[36457]: scrub summary: idle 2024-04-16T17:14:49.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:14:49 smithi104 ceph-mon[36457]: scrub summary: idle 2024-04-16T17:14:49.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:14:49 smithi104 ceph-mon[36457]: scrub summary: idle 2024-04-16T17:14:50.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:14:50 smithi045 ceph-mon[31676]: pgmap v250: 129 pgs: 129 active+clean; 60 MiB data, 837 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 18 KiB/s wr, 15 op/s 2024-04-16T17:14:50.804 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:14:50 smithi117 ceph-mon[36730]: pgmap v250: 129 pgs: 129 active+clean; 60 MiB data, 837 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 18 KiB/s wr, 15 op/s 2024-04-16T17:14:50.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:14:50 smithi104 ceph-mon[36457]: pgmap v250: 129 pgs: 129 active+clean; 60 MiB data, 837 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 18 KiB/s wr, 15 op/s 2024-04-16T17:14:52.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:14:52 smithi045 ceph-mon[31676]: pgmap v251: 129 pgs: 129 active+clean; 60 MiB data, 837 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 22 KiB/s wr, 18 op/s 2024-04-16T17:14:52.804 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:14:52 smithi117 ceph-mon[36730]: pgmap v251: 129 pgs: 129 active+clean; 60 MiB data, 837 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 22 KiB/s wr, 18 op/s 2024-04-16T17:14:52.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:14:52 smithi104 ceph-mon[36457]: pgmap v251: 129 pgs: 129 active+clean; 60 MiB data, 837 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 22 KiB/s wr, 18 op/s 2024-04-16T17:14:54.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:14:54 smithi117 ceph-mon[36730]: pgmap v252: 129 pgs: 129 active+clean; 60 MiB data, 837 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 9.1 KiB/s wr, 13 op/s 2024-04-16T17:14:54.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:14:54 smithi104 ceph-mon[36457]: pgmap v252: 129 pgs: 129 active+clean; 60 MiB data, 837 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 9.1 KiB/s wr, 13 op/s 2024-04-16T17:14:54.976 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:14:54 smithi045 ceph-mon[31676]: pgmap v252: 129 pgs: 129 active+clean; 60 MiB data, 837 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 9.1 KiB/s wr, 13 op/s 2024-04-16T17:14:56.804 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:14:56 smithi117 ceph-mon[36730]: pgmap v253: 129 pgs: 129 active+clean; 60 MiB data, 837 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 9.1 KiB/s wr, 13 op/s 2024-04-16T17:14:56.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:14:56 smithi104 ceph-mon[36457]: pgmap v253: 129 pgs: 129 active+clean; 60 MiB data, 837 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 9.1 KiB/s wr, 13 op/s 2024-04-16T17:14:56.976 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:14:56 smithi045 ceph-mon[31676]: pgmap v253: 129 pgs: 129 active+clean; 60 MiB data, 837 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 9.1 KiB/s wr, 13 op/s 2024-04-16T17:14:57.804 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:14:57 smithi117 ceph-mon[36730]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-16T17:14:57.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:14:57 smithi104 ceph-mon[36457]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-16T17:14:57.976 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:14:57 smithi045 ceph-mon[31676]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-16T17:14:58.804 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:14:58 smithi117 ceph-mon[36730]: pgmap v254: 129 pgs: 129 active+clean; 60 MiB data, 837 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 13 KiB/s wr, 16 op/s 2024-04-16T17:14:58.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:14:58 smithi104 ceph-mon[36457]: pgmap v254: 129 pgs: 129 active+clean; 60 MiB data, 837 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 13 KiB/s wr, 16 op/s 2024-04-16T17:14:58.976 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:14:58 smithi045 ceph-mon[31676]: pgmap v254: 129 pgs: 129 active+clean; 60 MiB data, 837 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 13 KiB/s wr, 16 op/s 2024-04-16T17:15:00.804 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:15:00 smithi117 ceph-mon[36730]: pgmap v255: 129 pgs: 129 active+clean; 60 MiB data, 837 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 7.3 KiB/s wr, 5 op/s 2024-04-16T17:15:00.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:15:00 smithi104 ceph-mon[36457]: pgmap v255: 129 pgs: 129 active+clean; 60 MiB data, 837 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 7.3 KiB/s wr, 5 op/s 2024-04-16T17:15:00.976 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:15:00 smithi045 ceph-mon[31676]: pgmap v255: 129 pgs: 129 active+clean; 60 MiB data, 837 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 7.3 KiB/s wr, 5 op/s 2024-04-16T17:15:02.804 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:15:02 smithi117 ceph-mon[36730]: pgmap v256: 129 pgs: 129 active+clean; 60 MiB data, 837 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-04-16T17:15:02.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:15:02 smithi104 ceph-mon[36457]: pgmap v256: 129 pgs: 129 active+clean; 60 MiB data, 837 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-04-16T17:15:02.976 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:15:02 smithi045 ceph-mon[31676]: pgmap v256: 129 pgs: 129 active+clean; 60 MiB data, 837 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-04-16T17:15:04.804 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:15:04 smithi117 ceph-mon[36730]: pgmap v257: 129 pgs: 129 active+clean; 60 MiB data, 837 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 50 KiB/s wr, 22 op/s 2024-04-16T17:15:04.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:15:04 smithi104 ceph-mon[36457]: pgmap v257: 129 pgs: 129 active+clean; 60 MiB data, 837 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 50 KiB/s wr, 22 op/s 2024-04-16T17:15:04.976 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:15:04 smithi045 ceph-mon[31676]: pgmap v257: 129 pgs: 129 active+clean; 60 MiB data, 837 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 50 KiB/s wr, 22 op/s 2024-04-16T17:15:05.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:15:05 smithi117 ceph-mon[36730]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T17:15:05.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:15:05 smithi104 ceph-mon[36457]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T17:15:05.976 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:15:05 smithi045 ceph-mon[31676]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T17:15:06.804 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:15:06 smithi117 ceph-mon[36730]: pgmap v258: 129 pgs: 129 active+clean; 60 MiB data, 837 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 50 KiB/s wr, 22 op/s 2024-04-16T17:15:06.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:15:06 smithi117 ceph-mon[36730]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:15:06.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:15:06 smithi117 ceph-mon[36730]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T17:15:06.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:15:06 smithi117 ceph-mon[36730]: from='mgr.25411 ' entity='mgr.x' 2024-04-16T17:15:06.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:15:06 smithi104 ceph-mon[36457]: pgmap v258: 129 pgs: 129 active+clean; 60 MiB data, 837 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 50 KiB/s wr, 22 op/s 2024-04-16T17:15:06.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:15:06 smithi104 ceph-mon[36457]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:15:06.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:15:06 smithi104 ceph-mon[36457]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T17:15:06.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:15:06 smithi104 ceph-mon[36457]: from='mgr.25411 ' entity='mgr.x' 2024-04-16T17:15:06.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:15:06 smithi045 ceph-mon[31676]: pgmap v258: 129 pgs: 129 active+clean; 60 MiB data, 837 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 50 KiB/s wr, 22 op/s 2024-04-16T17:15:06.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:15:06 smithi045 ceph-mon[31676]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:15:06.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:15:06 smithi045 ceph-mon[31676]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T17:15:06.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:15:06 smithi045 ceph-mon[31676]: from='mgr.25411 ' entity='mgr.x' 2024-04-16T17:15:08.804 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:15:08 smithi117 ceph-mon[36730]: pgmap v259: 129 pgs: 129 active+clean; 60 MiB data, 855 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 55 KiB/s wr, 26 op/s 2024-04-16T17:15:08.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:15:08 smithi104 ceph-mon[36457]: pgmap v259: 129 pgs: 129 active+clean; 60 MiB data, 855 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 55 KiB/s wr, 26 op/s 2024-04-16T17:15:08.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:15:08 smithi045 ceph-mon[31676]: pgmap v259: 129 pgs: 129 active+clean; 60 MiB data, 855 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 55 KiB/s wr, 26 op/s 2024-04-16T17:15:09.804 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:15:09 smithi117 ceph-mon[36730]: osdmap e101: 12 total, 12 up, 12 in 2024-04-16T17:15:09.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:15:09 smithi104 ceph-mon[36457]: osdmap e101: 12 total, 12 up, 12 in 2024-04-16T17:15:09.976 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:15:09 smithi045 ceph-mon[31676]: osdmap e101: 12 total, 12 up, 12 in 2024-04-16T17:15:10.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:15:10 smithi104 ceph-mon[36457]: pgmap v261: 129 pgs: 129 active+clean; 60 MiB data, 855 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 62 KiB/s wr, 28 op/s 2024-04-16T17:15:10.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:15:10 smithi045 ceph-mon[31676]: pgmap v261: 129 pgs: 129 active+clean; 60 MiB data, 855 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 62 KiB/s wr, 28 op/s 2024-04-16T17:15:11.054 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:15:10 smithi117 ceph-mon[36730]: pgmap v261: 129 pgs: 129 active+clean; 60 MiB data, 855 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 62 KiB/s wr, 28 op/s 2024-04-16T17:15:12.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:15:12 smithi104 ceph-mon[36457]: pgmap v262: 129 pgs: 129 active+clean; 59 MiB data, 876 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 18 KiB/s wr, 8 op/s 2024-04-16T17:15:12.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:15:12 smithi045 ceph-mon[31676]: pgmap v262: 129 pgs: 129 active+clean; 59 MiB data, 876 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 18 KiB/s wr, 8 op/s 2024-04-16T17:15:13.054 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:15:12 smithi117 ceph-mon[36730]: pgmap v262: 129 pgs: 129 active+clean; 59 MiB data, 876 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 18 KiB/s wr, 8 op/s 2024-04-16T17:15:13.292 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-16T17:15:13.292 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-16T17:15:13.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.515+0000 7fade2000640 1 -- 172.21.15.45:0/3967454036 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faddc103980 msgr2=0x7faddc10be60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:15:13.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.515+0000 7fade2000640 1 --2- 172.21.15.45:0/3967454036 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faddc103980 0x7faddc10be60 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fadc8009900 tx=0x7fadc802bcd0 comp rx=0 tx=0).stop 2024-04-16T17:15:13.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.515+0000 7fade2000640 1 -- 172.21.15.45:0/3967454036 shutdown_connections 2024-04-16T17:15:13.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.515+0000 7fade2000640 1 --2- 172.21.15.45:0/3967454036 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7faddc10c480 0x7faddc10e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:13.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.515+0000 7fade2000640 1 --2- 172.21.15.45:0/3967454036 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faddc103980 0x7faddc10be60 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:13.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.515+0000 7fade2000640 1 --2- 172.21.15.45:0/3967454036 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faddc103060 0x7faddc103440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:13.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.515+0000 7fade2000640 1 -- 172.21.15.45:0/3967454036 >> 172.21.15.45:0/3967454036 conn(0x7faddc0fc880 msgr2=0x7faddc0feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:15:13.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.515+0000 7fade2000640 1 -- 172.21.15.45:0/3967454036 shutdown_connections 2024-04-16T17:15:13.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.515+0000 7fade2000640 1 -- 172.21.15.45:0/3967454036 wait complete. 2024-04-16T17:15:13.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.516+0000 7fade2000640 1 Processor -- start 2024-04-16T17:15:13.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.516+0000 7fade2000640 1 -- start start 2024-04-16T17:15:13.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.516+0000 7fade2000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faddc103060 0x7faddc19e570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:13.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.516+0000 7fade2000640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7faddc103980 0x7faddc19eab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:13.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.516+0000 7fade2000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faddc10c480 0x7faddc198680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:13.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.516+0000 7fade2000640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7faddc112290 con 0x7faddc10c480 2024-04-16T17:15:13.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.516+0000 7fade2000640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7faddc112110 con 0x7faddc103980 2024-04-16T17:15:13.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.516+0000 7fade2000640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7faddc112410 con 0x7faddc103060 2024-04-16T17:15:13.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.517+0000 7fade1600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faddc10c480 0x7faddc198680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:13.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.517+0000 7fade1600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faddc10c480 0x7faddc198680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45946/0 (socket says 172.21.15.45:45946) 2024-04-16T17:15:13.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.517+0000 7fade1600640 1 -- 172.21.15.45:0/2407162481 learned_addr learned my addr 172.21.15.45:0/2407162481 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:15:13.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.517+0000 7faddbe00640 1 --2- 172.21.15.45:0/2407162481 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7faddc103980 0x7faddc19eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:13.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.517+0000 7fade0c00640 1 --2- 172.21.15.45:0/2407162481 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faddc103060 0x7faddc19e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:13.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.517+0000 7fade1600640 1 -- 172.21.15.45:0/2407162481 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faddc103060 msgr2=0x7faddc19e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:15:13.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.517+0000 7fade1600640 1 --2- 172.21.15.45:0/2407162481 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faddc103060 0x7faddc19e570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:13.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.517+0000 7fade1600640 1 -- 172.21.15.45:0/2407162481 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7faddc103980 msgr2=0x7faddc19eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:15:13.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.517+0000 7fade1600640 1 --2- 172.21.15.45:0/2407162481 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7faddc103980 0x7faddc19eab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:13.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.517+0000 7fade1600640 1 -- 172.21.15.45:0/2407162481 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faddc198f40 con 0x7faddc10c480 2024-04-16T17:15:13.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.517+0000 7fade1600640 1 --2- 172.21.15.45:0/2407162481 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faddc10c480 0x7faddc198680 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fadd400eb10 tx=0x7fadd400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:15:13.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.517+0000 7fadd9600640 1 -- 172.21.15.45:0/2407162481 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadd4004b20 con 0x7faddc10c480 2024-04-16T17:15:13.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.518+0000 7fadd9600640 1 -- 172.21.15.45:0/2407162481 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fadd4004cc0 con 0x7faddc10c480 2024-04-16T17:15:13.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.518+0000 7fadd9600640 1 -- 172.21.15.45:0/2407162481 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadd4010760 con 0x7faddc10c480 2024-04-16T17:15:13.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.518+0000 7fade2000640 1 -- 172.21.15.45:0/2407162481 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faddc199230 con 0x7faddc10c480 2024-04-16T17:15:13.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.518+0000 7fade2000640 1 -- 172.21.15.45:0/2407162481 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faddc1a54f0 con 0x7faddc10c480 2024-04-16T17:15:13.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.521+0000 7fade6213740 1 Processor -- start 2024-04-16T17:15:13.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.521+0000 7fadd9600640 1 -- 172.21.15.45:0/2407162481 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fadd4005020 con 0x7faddc10c480 2024-04-16T17:15:13.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.522+0000 7fade6213740 1 -- start start 2024-04-16T17:15:13.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.522+0000 7fade6213740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55af0f735390 0x55af0f735770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:13.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.522+0000 7fade6213740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55af0f735cb0 0x55af0f73e350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:13.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.522+0000 7fade6213740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55af0f73e890 0x55af0f740c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:13.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.522+0000 7fade6213740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55af0f69b120 con 0x55af0f73e890 2024-04-16T17:15:13.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.522+0000 7fade6213740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55af0f641340 con 0x55af0f735390 2024-04-16T17:15:13.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.522+0000 7fade6213740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55af0f6970e0 con 0x55af0f735cb0 2024-04-16T17:15:13.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.522+0000 7faddbe00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55af0f735390 0x55af0f735770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:13.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.522+0000 7fade0c00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55af0f735cb0 0x55af0f73e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:13.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.523+0000 7faddbe00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55af0f735390 0x55af0f735770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:38786/0 (socket says 172.21.15.45:38786) 2024-04-16T17:15:13.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.523+0000 7fade0c00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55af0f735cb0 0x55af0f73e350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:38004/0 (socket says 172.21.15.45:38004) 2024-04-16T17:15:13.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.523+0000 7faddbe00640 1 -- 172.21.15.45:0/3457706862 learned_addr learned my addr 172.21.15.45:0/3457706862 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:15:13.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.523+0000 7faddbe00640 1 --2- 172.21.15.45:0/3457706862 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55af0f73e890 0x55af0f740c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:13.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.523+0000 7fade0c00640 1 -- 172.21.15.45:0/3457706862 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55af0f735390 msgr2=0x55af0f735770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:15:13.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.523+0000 7fade0c00640 1 --2- 172.21.15.45:0/3457706862 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55af0f735390 0x55af0f735770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:13.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.523+0000 7fade0c00640 1 -- 172.21.15.45:0/3457706862 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55af0f73e890 msgr2=0x55af0f740c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:15:13.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.523+0000 7fade0c00640 1 --2- 172.21.15.45:0/3457706862 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55af0f73e890 0x55af0f740c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:13.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.523+0000 7fade0c00640 1 -- 172.21.15.45:0/3457706862 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55af0f4777d0 con 0x55af0f735cb0 2024-04-16T17:15:13.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.523+0000 7faddbe00640 1 --2- 172.21.15.45:0/3457706862 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55af0f73e890 0x55af0f740c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:15:13.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.523+0000 7fadd9600640 1 --2- 172.21.15.45:0/2407162481 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fadc00a3da0 0x7fadc00a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:13.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.523+0000 7fade0c00640 1 --2- 172.21.15.45:0/3457706862 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55af0f735cb0 0x55af0f73e350 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fadcc009c20 tx=0x7fadcc011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aa94983258d89243 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:15:13.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.524+0000 7fadd9600640 1 -- 172.21.15.45:0/2407162481 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9584+0+0 (secure 0 0 0) 0x7fadd40d1110 con 0x7faddc10c480 2024-04-16T17:15:13.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.524+0000 7faddbe00640 1 --2- 172.21.15.45:0/2407162481 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fadc00a3da0 0x7fadc00a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:13.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.524+0000 7fadd1c00640 1 -- 172.21.15.45:0/3457706862 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadcc004050 con 0x55af0f735cb0 2024-04-16T17:15:13.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.524+0000 7fadd1c00640 1 -- 172.21.15.45:0/3457706862 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fadcc005500 con 0x55af0f735cb0 2024-04-16T17:15:13.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.524+0000 7fadd1c00640 1 -- 172.21.15.45:0/3457706862 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadcc004dd0 con 0x55af0f735cb0 2024-04-16T17:15:13.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.525+0000 7fade6213740 1 -- 172.21.15.45:0/3457706862 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55af0f735cb0 msgr2=0x55af0f73e350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:15:13.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.525+0000 7faddbe00640 1 --2- 172.21.15.45:0/2407162481 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fadc00a3da0 0x7fadc00a6260 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x55af0f730c50 tx=0x7fadc8005a20 comp rx=0 tx=0).ready entity=mgr.25411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:15:13.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.525+0000 7fade6213740 1 --2- 172.21.15.45:0/3457706862 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55af0f735cb0 0x55af0f73e350 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fadcc009c20 tx=0x7fadcc011810 comp rx=0 tx=0).stop 2024-04-16T17:15:13.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.525+0000 7fade6213740 1 -- 172.21.15.45:0/3457706862 shutdown_connections 2024-04-16T17:15:13.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.525+0000 7fade6213740 1 --2- 172.21.15.45:0/3457706862 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55af0f735cb0 0x55af0f73e350 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:13.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.525+0000 7fade6213740 1 --2- 172.21.15.45:0/3457706862 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55af0f73e890 0x55af0f740c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:13.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.525+0000 7fade6213740 1 --2- 172.21.15.45:0/3457706862 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55af0f735390 0x55af0f735770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:13.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.525+0000 7fade6213740 1 -- 172.21.15.45:0/3457706862 >> 172.21.15.45:0/3457706862 conn(0x55af0f72fac0 msgr2=0x55af0f72fea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:15:13.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.525+0000 7fade6213740 1 -- 172.21.15.45:0/3457706862 shutdown_connections 2024-04-16T17:15:13.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.525+0000 7fade6213740 1 -- 172.21.15.45:0/3457706862 wait complete. 2024-04-16T17:15:13.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.525+0000 7fade6213740 1 -- start start 2024-04-16T17:15:13.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.526+0000 7fade6213740 1 Processor -- start 2024-04-16T17:15:13.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.526+0000 7fade6213740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55af0f735390 0x55af0f73b840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:13.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.526+0000 7fade6213740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55af0f735cb0 0x55af0f73bd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:13.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.526+0000 7fade6213740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55af0f73e890 0x55af0f73c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:13.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.526+0000 7fade6213740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55af0f741680 con 0x55af0f735cb0 2024-04-16T17:15:13.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.526+0000 7fade6213740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55af0f654710 con 0x55af0f73e890 2024-04-16T17:15:13.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.526+0000 7fade6213740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55af0f741800 con 0x55af0f735390 2024-04-16T17:15:13.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.526+0000 7fade6213740 10 client.0 _ll_register_callbacks cb 0x55af0f64faf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:15:13.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.526+0000 7faddbe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55af0f735cb0 0x55af0f73bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:13.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.527+0000 7faddbe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55af0f735cb0 0x55af0f73bd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45964/0 (socket says 172.21.15.45:45964) 2024-04-16T17:15:13.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.527+0000 7faddbe00640 1 -- 172.21.15.45:0/3559093535 learned_addr learned my addr 172.21.15.45:0/3559093535 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:15:13.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.527+0000 7fade0c00640 1 --2- 172.21.15.45:0/3559093535 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55af0f735390 0x55af0f73b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:13.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.527+0000 7fade1600640 1 --2- 172.21.15.45:0/3559093535 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55af0f73e890 0x55af0f73c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:13.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.527+0000 7faddbe00640 1 -- 172.21.15.45:0/3559093535 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55af0f735390 msgr2=0x55af0f73b840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:15:13.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.527+0000 7faddbe00640 1 --2- 172.21.15.45:0/3559093535 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55af0f735390 0x55af0f73b840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:13.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.527+0000 7faddbe00640 1 -- 172.21.15.45:0/3559093535 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55af0f73e890 msgr2=0x55af0f73c2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:15:13.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.527+0000 7faddbe00640 1 --2- 172.21.15.45:0/3559093535 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55af0f73e890 0x55af0f73c2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:13.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.527+0000 7faddbe00640 1 -- 172.21.15.45:0/3559093535 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55af0f3ad3f0 con 0x55af0f735cb0 2024-04-16T17:15:13.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.527+0000 7fade1600640 1 --2- 172.21.15.45:0/3559093535 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55af0f73e890 0x55af0f73c2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:15:13.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.527+0000 7faddbe00640 1 --2- 172.21.15.45:0/3559093535 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55af0f735cb0 0x55af0f73bd80 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fadc8005ab0 tx=0x7fadc802bcd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4421ef790f5e010 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:15:13.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.527+0000 7fadc6000640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-16T17:15:13.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.528+0000 7fadc6000640 1 -- 172.21.15.45:0/3559093535 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadc8052070 con 0x55af0f735cb0 2024-04-16T17:15:13.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.528+0000 7fadc6000640 1 -- 172.21.15.45:0/3559093535 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9584+0+0 (secure 0 0 0) 0x7fadc8004840 con 0x55af0f735cb0 2024-04-16T17:15:13.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.528+0000 7fade6213740 1 -- 172.21.15.45:0/3559093535 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fadcc005500 con 0x55af0f735cb0 2024-04-16T17:15:13.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.528+0000 7fadc6000640 1 -- 172.21.15.45:0/3559093535 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fadc805e050 con 0x55af0f735cb0 2024-04-16T17:15:13.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.528+0000 7fade6213740 10 client.17658 fetch_fsmap learned FSMap version 43 2024-04-16T17:15:13.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.528+0000 7fade6213740 1 -- 172.21.15.45:0/3559093535 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55af0f36d0f0 con 0x55af0f735cb0 2024-04-16T17:15:13.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.528+0000 7fadc6000640 1 -- 172.21.15.45:0/3559093535 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fadcc005500 con 0x55af0f735cb0 2024-04-16T17:15:13.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.529+0000 7fadc6000640 1 -- 172.21.15.45:0/3559093535 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7fadc804d040 con 0x55af0f735cb0 2024-04-16T17:15:13.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.529+0000 7fade6213740 10 client.17658 fetch_fsmap finished waiting for FSMap version 43 2024-04-16T17:15:13.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.529+0000 7fade6213740 10 client.17658 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:15:13.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.529+0000 7fade6213740 20 client.17658 populate_metadata read hostname 'smithi045' 2024-04-16T17:15:13.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.529+0000 7fade6213740 1 --2- 172.21.15.45:0/3559093535 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55af0f752fa0 0x55af0f755420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:13.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.529+0000 7fade6213740 4 client.17658 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:15:13.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.529+0000 7fade6213740 1 -- 172.21.15.45:0/3559093535 --> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55af0f6a8b40 con 0x55af0f752fa0 2024-04-16T17:15:13.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.529+0000 7fade0c00640 1 --2- 172.21.15.45:0/3559093535 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55af0f752fa0 0x55af0f755420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:13.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.530+0000 7fade0c00640 1 --2- 172.21.15.45:0/3559093535 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55af0f752fa0 0x55af0f755420 crc :-1 s=READY pgs=225 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=17037eeef0f75758 server_cookie=636fb696e8bb793e in_seq=0 out_seq=0 2024-04-16T17:15:13.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.530+0000 7fadc6000640 10 client.17658 ms_handle_connect on v2:172.21.15.104:6838/3660070592 2024-04-16T17:15:13.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.531+0000 7fadc6000640 1 -- 172.21.15.45:0/3559093535 <== mds.0 v2:172.21.15.104:6838/3660070592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55af0f6a8b40 con 0x55af0f752fa0 2024-04-16T17:15:13.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.531+0000 7fadc6000640 10 client.17658 handle_command_reply: tid=0 2024-04-16T17:15:13.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.531+0000 7fade6213740 1 client.17658 shutdown 2024-04-16T17:15:13.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.531+0000 7fade6213740 1 -- 172.21.15.45:0/3559093535 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55af0f735cb0 msgr2=0x55af0f73bd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:15:13.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.531+0000 7fade6213740 1 --2- 172.21.15.45:0/3559093535 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55af0f735cb0 0x55af0f73bd80 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fadc8005ab0 tx=0x7fadc802bcd0 comp rx=0 tx=0).stop 2024-04-16T17:15:13.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.531+0000 7fade6213740 1 -- 172.21.15.45:0/3559093535 shutdown_connections 2024-04-16T17:15:13.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.531+0000 7fade6213740 1 --2- 172.21.15.45:0/3559093535 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55af0f73e890 0x55af0f73c2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:13.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.531+0000 7fade6213740 1 --2- 172.21.15.45:0/3559093535 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55af0f735cb0 0x55af0f73bd80 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:13.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.531+0000 7fade6213740 1 --2- 172.21.15.45:0/3559093535 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55af0f752fa0 0x55af0f755420 crc :-1 s=READY pgs=225 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:13.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.531+0000 7fade6213740 1 --2- 172.21.15.45:0/3559093535 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55af0f735390 0x55af0f73b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:13.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.531+0000 7fade6213740 1 -- 172.21.15.45:0/3559093535 >> 172.21.15.45:0/3559093535 conn(0x55af0f72fac0 msgr2=0x55af0f731810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:15:13.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.531+0000 7fadc6000640 0 client.17658 ms_handle_reset on v2:172.21.15.104:6838/3660070592 2024-04-16T17:15:13.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.531+0000 7fade6213740 1 -- 172.21.15.45:0/3559093535 shutdown_connections 2024-04-16T17:15:13.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.531+0000 7fade6213740 1 -- 172.21.15.45:0/3559093535 wait complete. 2024-04-16T17:15:13.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.531+0000 7fade6213740 20 client.17658 trim_cache size 0 max 16384 2024-04-16T17:15:13.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.551+0000 7fade6213740 1 Processor -- start 2024-04-16T17:15:13.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.551+0000 7fade6213740 1 -- start start 2024-04-16T17:15:13.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.551+0000 7fade6213740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55af0f735390 0x55af0f73df70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:13.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.551+0000 7fade6213740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55af0f735cb0 0x55af0f73bf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:13.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.551+0000 7fade6213740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55af0f73e890 0x55af0f75acf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:13.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.552+0000 7fade6213740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7faddc1afb20 con 0x55af0f73e890 2024-04-16T17:15:13.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.552+0000 7fade6213740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7faddc1ad3f0 con 0x55af0f735390 2024-04-16T17:15:13.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.552+0000 7fade6213740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7faddc1ab240 con 0x55af0f735cb0 2024-04-16T17:15:13.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.552+0000 7fade1600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55af0f73e890 0x55af0f75acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:13.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.552+0000 7fade1600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55af0f73e890 0x55af0f75acf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45976/0 (socket says 172.21.15.45:45976) 2024-04-16T17:15:13.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.552+0000 7fade1600640 1 -- 172.21.15.45:0/1508183866 learned_addr learned my addr 172.21.15.45:0/1508183866 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:15:13.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.552+0000 7faddbe00640 1 --2- 172.21.15.45:0/1508183866 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55af0f735cb0 0x55af0f73bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:13.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.552+0000 7fade0c00640 1 --2- 172.21.15.45:0/1508183866 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55af0f735390 0x55af0f73df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:13.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.552+0000 7fade1600640 1 -- 172.21.15.45:0/1508183866 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55af0f735cb0 msgr2=0x55af0f73bf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:15:13.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.552+0000 7fade1600640 1 --2- 172.21.15.45:0/1508183866 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55af0f735cb0 0x55af0f73bf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:13.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.552+0000 7fade1600640 1 -- 172.21.15.45:0/1508183866 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55af0f735390 msgr2=0x55af0f73df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:15:13.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.552+0000 7fade1600640 1 --2- 172.21.15.45:0/1508183866 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55af0f735390 0x55af0f73df70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:13.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.552+0000 7fade1600640 1 -- 172.21.15.45:0/1508183866 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55af0f729f30 con 0x55af0f73e890 2024-04-16T17:15:13.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.552+0000 7faddbe00640 1 --2- 172.21.15.45:0/1508183866 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55af0f735cb0 0x55af0f73bf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:15:13.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.553+0000 7fade1600640 1 --2- 172.21.15.45:0/1508183866 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55af0f73e890 0x55af0f75acf0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fadd409cbe0 tx=0x7fadd4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a431539f340f3e92 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:15:13.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.553+0000 7fadd1200640 1 -- 172.21.15.45:0/1508183866 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadd40b5020 con 0x55af0f73e890 2024-04-16T17:15:13.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.553+0000 7fadd1200640 1 -- 172.21.15.45:0/1508183866 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fadd409d350 con 0x55af0f73e890 2024-04-16T17:15:13.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.553+0000 7fadd1200640 1 -- 172.21.15.45:0/1508183866 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadd409ceb0 con 0x55af0f73e890 2024-04-16T17:15:13.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.553+0000 7fade6213740 1 -- 172.21.15.45:0/1508183866 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55af0f73e890 msgr2=0x55af0f75acf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:15:13.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.553+0000 7fade6213740 1 --2- 172.21.15.45:0/1508183866 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55af0f73e890 0x55af0f75acf0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fadd409cbe0 tx=0x7fadd4004a40 comp rx=0 tx=0).stop 2024-04-16T17:15:13.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.554+0000 7fade6213740 1 -- 172.21.15.45:0/1508183866 shutdown_connections 2024-04-16T17:15:13.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.554+0000 7fade6213740 1 --2- 172.21.15.45:0/1508183866 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55af0f735cb0 0x55af0f73bf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:13.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.554+0000 7fade6213740 1 --2- 172.21.15.45:0/1508183866 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55af0f73e890 0x55af0f75acf0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:13.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.554+0000 7fade6213740 1 --2- 172.21.15.45:0/1508183866 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55af0f735390 0x55af0f73df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:13.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.554+0000 7fade6213740 1 -- 172.21.15.45:0/1508183866 >> 172.21.15.45:0/1508183866 conn(0x55af0f72fac0 msgr2=0x55af0f73b800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:15:13.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.554+0000 7fade6213740 1 -- 172.21.15.45:0/1508183866 shutdown_connections 2024-04-16T17:15:13.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.554+0000 7fade6213740 1 -- 172.21.15.45:0/1508183866 wait complete. 2024-04-16T17:15:13.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.554+0000 7fade6213740 1 -- start start 2024-04-16T17:15:13.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.554+0000 7fade6213740 1 Processor -- start 2024-04-16T17:15:13.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.555+0000 7fade6213740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55af0f735390 0x55af0f74b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:13.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.555+0000 7fade6213740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55af0f735cb0 0x55af0f7380b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:13.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.555+0000 7fade6213740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55af0f73e890 0x55af0f7385f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:13.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.555+0000 7fade6213740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7faddc1ab8b0 con 0x55af0f735390 2024-04-16T17:15:13.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.555+0000 7fade6213740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7faddc1acd20 con 0x55af0f73e890 2024-04-16T17:15:13.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.555+0000 7fade6213740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7faddc1abf80 con 0x55af0f735cb0 2024-04-16T17:15:13.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.555+0000 7fade6213740 10 client.0 _ll_register_callbacks cb 0x7fadcc005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:15:13.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.555+0000 7fade0c00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55af0f735390 0x55af0f74b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:13.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.555+0000 7fade0c00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55af0f735390 0x55af0f74b1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45978/0 (socket says 172.21.15.45:45978) 2024-04-16T17:15:13.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.555+0000 7fade0c00640 1 -- 172.21.15.45:0/2524177264 learned_addr learned my addr 172.21.15.45:0/2524177264 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:15:13.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.555+0000 7fade1600640 1 --2- 172.21.15.45:0/2524177264 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55af0f73e890 0x55af0f7385f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:13.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.555+0000 7faddbe00640 1 --2- 172.21.15.45:0/2524177264 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55af0f735cb0 0x55af0f7380b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:13.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.555+0000 7fade0c00640 1 -- 172.21.15.45:0/2524177264 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55af0f735cb0 msgr2=0x55af0f7380b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:15:13.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.555+0000 7fade0c00640 1 --2- 172.21.15.45:0/2524177264 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55af0f735cb0 0x55af0f7380b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:13.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.555+0000 7fade0c00640 1 -- 172.21.15.45:0/2524177264 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55af0f73e890 msgr2=0x55af0f7385f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:15:13.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.555+0000 7fade0c00640 1 --2- 172.21.15.45:0/2524177264 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55af0f73e890 0x55af0f7385f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:13.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.555+0000 7fade0c00640 1 -- 172.21.15.45:0/2524177264 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55af0f738c60 con 0x55af0f735390 2024-04-16T17:15:13.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.556+0000 7fade0c00640 1 --2- 172.21.15.45:0/2524177264 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55af0f735390 0x55af0f74b1a0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fadcc014040 tx=0x7fadcc009390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c30c398ba304435 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:15:13.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.556+0000 7fadc6000640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-16T17:15:13.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.556+0000 7fadc6000640 1 -- 172.21.15.45:0/2524177264 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadcc032020 con 0x55af0f735390 2024-04-16T17:15:13.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.556+0000 7fadc6000640 1 -- 172.21.15.45:0/2524177264 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9584+0+0 (secure 0 0 0) 0x7fadcc02edd0 con 0x55af0f735390 2024-04-16T17:15:13.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.556+0000 7fade6213740 1 -- 172.21.15.45:0/2524177264 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fadd409d350 con 0x55af0f735390 2024-04-16T17:15:13.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.557+0000 7fadc6000640 1 -- 172.21.15.45:0/2524177264 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fadcc03d050 con 0x55af0f735390 2024-04-16T17:15:13.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.557+0000 7fade6213740 10 client.17664 fetch_fsmap learned FSMap version 43 2024-04-16T17:15:13.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.557+0000 7fade6213740 1 -- 172.21.15.45:0/2524177264 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55af0f7b9400 con 0x55af0f735390 2024-04-16T17:15:13.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.557+0000 7fadc6000640 1 -- 172.21.15.45:0/2524177264 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fadd409d350 con 0x55af0f735390 2024-04-16T17:15:13.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.557+0000 7fadc6000640 1 -- 172.21.15.45:0/2524177264 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7fadcc02e760 con 0x55af0f735390 2024-04-16T17:15:13.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.557+0000 7fade6213740 10 client.17664 fetch_fsmap finished waiting for FSMap version 43 2024-04-16T17:15:13.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.557+0000 7fade6213740 10 client.17664 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:15:13.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.557+0000 7fade6213740 20 client.17664 populate_metadata read hostname 'smithi045' 2024-04-16T17:15:13.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.557+0000 7fade6213740 1 --2- 172.21.15.45:0/2524177264 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55af0f7b99e0 0x55af0f7bbdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:13.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.557+0000 7fade6213740 4 client.17664 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-16T17:15:13.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.557+0000 7fade6213740 1 -- 172.21.15.45:0/2524177264 --> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55af0f6a29b0 con 0x55af0f7b99e0 2024-04-16T17:15:13.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.558+0000 7faddbe00640 1 --2- 172.21.15.45:0/2524177264 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55af0f7b99e0 0x55af0f7bbdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:13.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.558+0000 7faddbe00640 1 --2- 172.21.15.45:0/2524177264 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55af0f7b99e0 0x55af0f7bbdd0 crc :-1 s=READY pgs=226 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d36d7d78a4de57b1 server_cookie=cae20c9bd1ce980b in_seq=0 out_seq=0 2024-04-16T17:15:13.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.558+0000 7fadc6000640 10 client.17664 ms_handle_connect on v2:172.21.15.104:6838/3660070592 2024-04-16T17:15:13.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.558+0000 7fadc6000640 1 -- 172.21.15.45:0/2524177264 <== mds.0 v2:172.21.15.104:6838/3660070592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55af0f6a29b0 con 0x55af0f7b99e0 2024-04-16T17:15:13.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.559+0000 7fadc6000640 10 client.17664 handle_command_reply: tid=0 2024-04-16T17:15:13.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.559+0000 7fade6213740 1 client.17664 shutdown 2024-04-16T17:15:13.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.559+0000 7fade6213740 1 -- 172.21.15.45:0/2524177264 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55af0f735390 msgr2=0x55af0f74b1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:15:13.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.559+0000 7fade6213740 1 --2- 172.21.15.45:0/2524177264 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55af0f735390 0x55af0f74b1a0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fadcc014040 tx=0x7fadcc009390 comp rx=0 tx=0).stop 2024-04-16T17:15:13.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.559+0000 7fade6213740 1 -- 172.21.15.45:0/2524177264 shutdown_connections 2024-04-16T17:15:13.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.559+0000 7fade6213740 1 --2- 172.21.15.45:0/2524177264 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55af0f7b99e0 0x55af0f7bbdd0 crc :-1 s=READY pgs=226 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:13.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.559+0000 7fade6213740 1 --2- 172.21.15.45:0/2524177264 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55af0f735cb0 0x55af0f7380b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:13.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.559+0000 7fade6213740 1 --2- 172.21.15.45:0/2524177264 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55af0f73e890 0x55af0f7385f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:13.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.559+0000 7fade6213740 1 --2- 172.21.15.45:0/2524177264 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55af0f735390 0x55af0f74b1a0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:13.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.559+0000 7fade6213740 1 -- 172.21.15.45:0/2524177264 >> 172.21.15.45:0/2524177264 conn(0x55af0f72fac0 msgr2=0x55af0f72eb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:15:13.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.559+0000 7fadc6000640 0 client.17664 ms_handle_reset on v2:172.21.15.104:6838/3660070592 2024-04-16T17:15:13.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.559+0000 7fade6213740 1 -- 172.21.15.45:0/2524177264 shutdown_connections 2024-04-16T17:15:13.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.559+0000 7fade6213740 1 -- 172.21.15.45:0/2524177264 wait complete. 2024-04-16T17:15:13.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.559+0000 7fade6213740 20 client.17664 trim_cache size 0 max 16384 2024-04-16T17:15:13.566 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-16T17:15:13.566 INFO:teuthology.orchestra.run.smithi045.stdout: "status": "no active scrubs running", 2024-04-16T17:15:13.566 INFO:teuthology.orchestra.run.smithi045.stdout: "scrubs": {} 2024-04-16T17:15:13.566 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-16T17:15:13.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.560+0000 7fade2000640 1 -- 172.21.15.45:0/2407162481 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fadc00a3da0 msgr2=0x7fadc00a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:15:13.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.560+0000 7fade2000640 1 --2- 172.21.15.45:0/2407162481 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fadc00a3da0 0x7fadc00a6260 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x55af0f730c50 tx=0x7fadc8005a20 comp rx=0 tx=0).stop 2024-04-16T17:15:13.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.560+0000 7fade2000640 1 -- 172.21.15.45:0/2407162481 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faddc10c480 msgr2=0x7faddc198680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:15:13.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.560+0000 7fade2000640 1 --2- 172.21.15.45:0/2407162481 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faddc10c480 0x7faddc198680 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fadd400eb10 tx=0x7fadd400c2d0 comp rx=0 tx=0).stop 2024-04-16T17:15:13.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.560+0000 7fade2000640 1 -- 172.21.15.45:0/2407162481 shutdown_connections 2024-04-16T17:15:13.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.560+0000 7fade2000640 1 --2- 172.21.15.45:0/2407162481 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fadc00a3da0 0x7fadc00a6260 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:13.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.560+0000 7fade2000640 1 --2- 172.21.15.45:0/2407162481 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faddc10c480 0x7faddc198680 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:13.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.560+0000 7fade2000640 1 --2- 172.21.15.45:0/2407162481 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7faddc103980 0x7faddc19eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:13.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.560+0000 7fade2000640 1 --2- 172.21.15.45:0/2407162481 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faddc103060 0x7faddc19e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:13.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.560+0000 7fade2000640 1 -- 172.21.15.45:0/2407162481 >> 172.21.15.45:0/2407162481 conn(0x7faddc0fc880 msgr2=0x7faddc0fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:15:13.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.560+0000 7fade2000640 1 -- 172.21.15.45:0/2407162481 shutdown_connections 2024-04-16T17:15:13.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.560+0000 7fade2000640 1 -- 172.21.15.45:0/2407162481 wait complete. 2024-04-16T17:15:13.577 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-16T17:15:13.578 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-16T17:15:13.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.782+0000 7f2baf200640 1 -- 172.21.15.45:0/1832582505 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2bb0074cf0 msgr2=0x7f2bb010e180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:15:13.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.782+0000 7f2baf200640 1 --2- 172.21.15.45:0/1832582505 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2bb0074cf0 0x7f2bb010e180 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f2b98009860 tx=0x7f2b9802bc10 comp rx=0 tx=0).stop 2024-04-16T17:15:13.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.782+0000 7f2baf200640 1 -- 172.21.15.45:0/1832582505 shutdown_connections 2024-04-16T17:15:13.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.782+0000 7f2baf200640 1 --2- 172.21.15.45:0/1832582505 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2bb0074cf0 0x7f2bb010e180 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:13.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.782+0000 7f2baf200640 1 --2- 172.21.15.45:0/1832582505 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2bb010e6c0 0x7f2bb0110ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:13.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.782+0000 7f2baf200640 1 --2- 172.21.15.45:0/1832582505 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2bb00743d0 0x7f2bb00747b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:13.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.782+0000 7f2baf200640 1 -- 172.21.15.45:0/1832582505 >> 172.21.15.45:0/1832582505 conn(0x7f2bb00fd930 msgr2=0x7f2bb00ffd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:15:13.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.782+0000 7f2baf200640 1 -- 172.21.15.45:0/1832582505 shutdown_connections 2024-04-16T17:15:13.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.782+0000 7f2baf200640 1 -- 172.21.15.45:0/1832582505 wait complete. 2024-04-16T17:15:13.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.783+0000 7f2baf200640 1 Processor -- start 2024-04-16T17:15:13.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.783+0000 7f2baf200640 1 -- start start 2024-04-16T17:15:13.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.783+0000 7f2baf200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2bb00743d0 0x7f2bb01a0630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:13.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.783+0000 7f2baf200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2bb0074cf0 0x7f2bb01a0b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:13.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.783+0000 7f2baf200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2bb010e6c0 0x7f2bb019a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:13.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.784+0000 7f2baf200640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f2bb0114420 con 0x7f2bb00743d0 2024-04-16T17:15:13.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.784+0000 7f2baf200640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f2bb01142a0 con 0x7f2bb0074cf0 2024-04-16T17:15:13.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.784+0000 7f2baf200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f2bb01145a0 con 0x7f2bb010e6c0 2024-04-16T17:15:13.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.784+0000 7f2bade00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2bb00743d0 0x7f2bb01a0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:13.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.784+0000 7f2bade00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2bb00743d0 0x7f2bb01a0630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45996/0 (socket says 172.21.15.45:45996) 2024-04-16T17:15:13.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.784+0000 7f2bade00640 1 -- 172.21.15.45:0/3400439147 learned_addr learned my addr 172.21.15.45:0/3400439147 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:15:13.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.784+0000 7f2bae800640 1 --2- 172.21.15.45:0/3400439147 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2bb010e6c0 0x7f2bb019a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:13.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.784+0000 7f2ba7e00640 1 --2- 172.21.15.45:0/3400439147 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2bb0074cf0 0x7f2bb01a0b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:13.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.784+0000 7f2bade00640 1 -- 172.21.15.45:0/3400439147 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2bb010e6c0 msgr2=0x7f2bb019a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:15:13.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.784+0000 7f2bade00640 1 --2- 172.21.15.45:0/3400439147 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2bb010e6c0 0x7f2bb019a7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:13.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.784+0000 7f2bade00640 1 -- 172.21.15.45:0/3400439147 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2bb0074cf0 msgr2=0x7f2bb01a0b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:15:13.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.784+0000 7f2bade00640 1 --2- 172.21.15.45:0/3400439147 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2bb0074cf0 0x7f2bb01a0b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:13.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.784+0000 7f2bade00640 1 -- 172.21.15.45:0/3400439147 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2bb019af70 con 0x7f2bb00743d0 2024-04-16T17:15:13.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.785+0000 7f2bade00640 1 --2- 172.21.15.45:0/3400439147 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2bb00743d0 0x7f2bb01a0630 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f2ba000e9f0 tx=0x7f2ba000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:15:13.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.785+0000 7f2ba6a00640 1 -- 172.21.15.45:0/3400439147 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ba000cd80 con 0x7f2bb00743d0 2024-04-16T17:15:13.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.785+0000 7f2baf200640 1 -- 172.21.15.45:0/3400439147 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2bb019b260 con 0x7f2bb00743d0 2024-04-16T17:15:13.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.785+0000 7f2ba6a00640 1 -- 172.21.15.45:0/3400439147 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2ba0010910 con 0x7f2bb00743d0 2024-04-16T17:15:13.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.785+0000 7f2ba6a00640 1 -- 172.21.15.45:0/3400439147 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ba00188f0 con 0x7f2bb00743d0 2024-04-16T17:15:13.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.785+0000 7f2baf200640 1 -- 172.21.15.45:0/3400439147 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2bb01a7480 con 0x7f2bb00743d0 2024-04-16T17:15:13.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.786+0000 7f2baf200640 1 -- 172.21.15.45:0/3400439147 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2bb0075a80 con 0x7f2bb00743d0 2024-04-16T17:15:13.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.788+0000 7f2ba6a00640 1 -- 172.21.15.45:0/3400439147 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f2ba0010ac0 con 0x7f2bb00743d0 2024-04-16T17:15:13.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.790+0000 7f2ba6a00640 1 --2- 172.21.15.45:0/3400439147 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f2b940a3d20 0x7f2b940a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:13.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.790+0000 7f2ba6a00640 1 -- 172.21.15.45:0/3400439147 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9584+0+0 (secure 0 0 0) 0x7f2ba00d14a0 con 0x7f2bb00743d0 2024-04-16T17:15:13.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.790+0000 7f2ba7e00640 1 --2- 172.21.15.45:0/3400439147 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f2b940a3d20 0x7f2b940a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:13.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.791+0000 7f2ba7e00640 1 --2- 172.21.15.45:0/3400439147 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f2b940a3d20 0x7f2b940a61e0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f2b9802c220 tx=0x7f2b98035de0 comp rx=0 tx=0).ready entity=mgr.25411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:15:13.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:13.793+0000 7f2ba6a00640 1 -- 172.21.15.45:0/3400439147 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2ba009d6e0 con 0x7f2bb00743d0 2024-04-16T17:15:14.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.039+0000 7f2baf200640 1 -- 172.21.15.45:0/3400439147 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2bb0068d40 con 0x7f2bb00743d0 2024-04-16T17:15:14.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.041+0000 7f2ba6a00640 1 -- 172.21.15.45:0/3400439147 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) ==== 94+0+11890 (secure 0 0 0) 0x7f2bb0068d40 con 0x7f2bb00743d0 2024-04-16T17:15:14.041 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-16T17:15:14.041 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":43,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/1748842443","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":1748842443},{"type":"v1","addr":"172.21.15.45:6835","nonce":1748842443}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6839/1308627051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":1308627051},{"type":"v1","addr":"172.21.15.45:6839","nonce":1308627051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/1768242489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1768242489},{"type":"v1","addr":"172.21.15.117:6833","nonce":1768242489}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6833/3558902585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":3558902585},{"type":"v1","addr":"172.21.15.104:6833","nonce":3558902585}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24478,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/1308142903","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":1308142903},{"type":"v1","addr":"172.21.15.45:6837","nonce":1308142903}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24491,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6835/3430679937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":3430679937},{"type":"v1","addr":"172.21.15.104:6835","nonce":3430679937}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":43,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-16T17:03:24.663611+0000","modified":"2024-04-16T17:14:55.721690+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24581,"mds_1":24569,"mds_2":24532},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24499":{"gid":24499,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/2224123841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2224123841},{"type":"v1","addr":"172.21.15.117:6835","nonce":2224123841}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6837/2471356815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":2471356815},{"type":"v1","addr":"172.21.15.117:6837","nonce":2471356815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24532":{"gid":24532,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":13,"addr":"172.21.15.117:6839/2378675949","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":2378675949},{"type":"v1","addr":"172.21.15.117:6839","nonce":2378675949}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24569":{"gid":24569,"name":"a","rank":1,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.45:6841/3755310822","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6840","nonce":3755310822},{"type":"v1","addr":"172.21.15.45:6841","nonce":3755310822}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24581":{"gid":24581,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.104:6839/3660070592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6838","nonce":3660070592},{"type":"v1","addr":"172.21.15.104:6839","nonce":3660070592}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_27569":{"gid":27569,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6837/1833645246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":1833645246},{"type":"v1","addr":"172.21.15.104:6837","nonce":1833645246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24581,"qdb_cluster":[24569,24581,24532]},"id":1}]} 2024-04-16T17:15:14.041 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 43 2024-04-16T17:15:14.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.043+0000 7f2baf200640 1 -- 172.21.15.45:0/3400439147 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f2b940a3d20 msgr2=0x7f2b940a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:15:14.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.043+0000 7f2baf200640 1 --2- 172.21.15.45:0/3400439147 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f2b940a3d20 0x7f2b940a61e0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f2b9802c220 tx=0x7f2b98035de0 comp rx=0 tx=0).stop 2024-04-16T17:15:14.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.043+0000 7f2baf200640 1 -- 172.21.15.45:0/3400439147 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2bb00743d0 msgr2=0x7f2bb01a0630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:15:14.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.043+0000 7f2baf200640 1 --2- 172.21.15.45:0/3400439147 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2bb00743d0 0x7f2bb01a0630 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f2ba000e9f0 tx=0x7f2ba000eeb0 comp rx=0 tx=0).stop 2024-04-16T17:15:14.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.043+0000 7f2baf200640 1 -- 172.21.15.45:0/3400439147 shutdown_connections 2024-04-16T17:15:14.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.043+0000 7f2baf200640 1 --2- 172.21.15.45:0/3400439147 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f2b940a3d20 0x7f2b940a61e0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:14.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.043+0000 7f2baf200640 1 --2- 172.21.15.45:0/3400439147 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2bb010e6c0 0x7f2bb019a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:14.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.043+0000 7f2baf200640 1 --2- 172.21.15.45:0/3400439147 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2bb0074cf0 0x7f2bb01a0b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:14.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.043+0000 7f2baf200640 1 --2- 172.21.15.45:0/3400439147 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2bb00743d0 0x7f2bb01a0630 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:14.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.043+0000 7f2baf200640 1 -- 172.21.15.45:0/3400439147 >> 172.21.15.45:0/3400439147 conn(0x7f2bb00fd930 msgr2=0x7f2bb00fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:15:14.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.043+0000 7f2baf200640 1 -- 172.21.15.45:0/3400439147 shutdown_connections 2024-04-16T17:15:14.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.043+0000 7f2baf200640 1 -- 172.21.15.45:0/3400439147 wait complete. 2024-04-16T17:15:14.063 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-16T17:15:14.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.272+0000 7fc7fea00640 1 -- 172.21.15.45:0/643253418 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc7f8103100 msgr2=0x7fc7f81034e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:15:14.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.272+0000 7fc7fea00640 1 --2- 172.21.15.45:0/643253418 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc7f8103100 0x7fc7f81034e0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fc7ec009900 tx=0x7fc7ec02bcc0 comp rx=0 tx=0).stop 2024-04-16T17:15:14.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.272+0000 7fc7fea00640 1 -- 172.21.15.45:0/643253418 shutdown_connections 2024-04-16T17:15:14.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.272+0000 7fc7fea00640 1 --2- 172.21.15.45:0/643253418 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc7f810c4d0 0x7fc7f810e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:14.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.272+0000 7fc7fea00640 1 --2- 172.21.15.45:0/643253418 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc7f8103a20 0x7fc7f810bf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:14.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.272+0000 7fc7fea00640 1 --2- 172.21.15.45:0/643253418 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc7f8103100 0x7fc7f81034e0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:14.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.272+0000 7fc7fea00640 1 -- 172.21.15.45:0/643253418 >> 172.21.15.45:0/643253418 conn(0x7fc7f80fc880 msgr2=0x7fc7f80feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:15:14.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.272+0000 7fc7fea00640 1 -- 172.21.15.45:0/643253418 shutdown_connections 2024-04-16T17:15:14.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.272+0000 7fc7fea00640 1 -- 172.21.15.45:0/643253418 wait complete. 2024-04-16T17:15:14.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.273+0000 7fc7fea00640 1 Processor -- start 2024-04-16T17:15:14.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.273+0000 7fc7fea00640 1 -- start start 2024-04-16T17:15:14.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.273+0000 7fc7fea00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc7f8103100 0x7fc7f81a0620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:14.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.273+0000 7fc7fea00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc7f8103a20 0x7fc7f81a0b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:14.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.273+0000 7fc7fea00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc7f810c4d0 0x7fc7f819a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:14.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.273+0000 7fc7fea00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fc7f8112260 con 0x7fc7f810c4d0 2024-04-16T17:15:14.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.273+0000 7fc7fea00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fc7f81120e0 con 0x7fc7f8103100 2024-04-16T17:15:14.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.273+0000 7fc7fea00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fc7f81123e0 con 0x7fc7f8103a20 2024-04-16T17:15:14.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.274+0000 7fc7fd600640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc7f8103100 0x7fc7f81a0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:14.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.274+0000 7fc7fe000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc7f810c4d0 0x7fc7f819a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:14.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.274+0000 7fc7fd600640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc7f8103100 0x7fc7f81a0620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:38860/0 (socket says 172.21.15.45:38860) 2024-04-16T17:15:14.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.274+0000 7fc7fd600640 1 -- 172.21.15.45:0/3017946957 learned_addr learned my addr 172.21.15.45:0/3017946957 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:15:14.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.274+0000 7fc7fcc00640 1 --2- 172.21.15.45:0/3017946957 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc7f8103a20 0x7fc7f81a0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:14.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.274+0000 7fc7fd600640 1 -- 172.21.15.45:0/3017946957 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc7f8103a20 msgr2=0x7fc7f81a0b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:15:14.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.274+0000 7fc7fd600640 1 --2- 172.21.15.45:0/3017946957 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc7f8103a20 0x7fc7f81a0b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:14.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.274+0000 7fc7fd600640 1 -- 172.21.15.45:0/3017946957 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc7f810c4d0 msgr2=0x7fc7f819a880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:15:14.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.274+0000 7fc7fd600640 1 --2- 172.21.15.45:0/3017946957 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc7f810c4d0 0x7fc7f819a880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:14.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.274+0000 7fc7fd600640 1 -- 172.21.15.45:0/3017946957 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7f819b050 con 0x7fc7f8103100 2024-04-16T17:15:14.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.274+0000 7fc7fe000640 1 --2- 172.21.15.45:0/3017946957 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc7f810c4d0 0x7fc7f819a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:15:14.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.275+0000 7fc7fd600640 1 --2- 172.21.15.45:0/3017946957 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc7f8103100 0x7fc7f81a0620 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fc7ec036040 tx=0x7fc7ec02cbc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:15:14.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.275+0000 7fc7f6000640 1 -- 172.21.15.45:0/3017946957 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7ec043020 con 0x7fc7f8103100 2024-04-16T17:15:14.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.275+0000 7fc7fea00640 1 -- 172.21.15.45:0/3017946957 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7f819b2e0 con 0x7fc7f8103100 2024-04-16T17:15:14.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.275+0000 7fc7f6000640 1 -- 172.21.15.45:0/3017946957 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc7ec0054e0 con 0x7fc7f8103100 2024-04-16T17:15:14.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.275+0000 7fc7f6000640 1 -- 172.21.15.45:0/3017946957 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7ec003c20 con 0x7fc7f8103100 2024-04-16T17:15:14.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.275+0000 7fc7fea00640 1 -- 172.21.15.45:0/3017946957 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7f81a7470 con 0x7fc7f8103100 2024-04-16T17:15:14.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.278+0000 7fc802c02740 1 Processor -- start 2024-04-16T17:15:14.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.278+0000 7fc802c02740 1 -- start start 2024-04-16T17:15:14.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.278+0000 7fc7f6000640 1 -- 172.21.15.45:0/3017946957 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fc7ec046020 con 0x7fc7f8103100 2024-04-16T17:15:14.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.278+0000 7fc802c02740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55956b629f10 0x55956b62a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:14.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.278+0000 7fc802c02740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55956b62a830 0x55956b632ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:14.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.278+0000 7fc802c02740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55956b633410 0x55956b635800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:14.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.278+0000 7fc802c02740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55956b58f020 con 0x55956b62a830 2024-04-16T17:15:14.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.278+0000 7fc802c02740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55956b535460 con 0x55956b629f10 2024-04-16T17:15:14.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.279+0000 7fc802c02740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55956b58afe0 con 0x55956b633410 2024-04-16T17:15:14.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.279+0000 7fc7fd600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55956b62a830 0x55956b632ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:14.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.279+0000 7fc7fd600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55956b62a830 0x55956b632ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:46024/0 (socket says 172.21.15.45:46024) 2024-04-16T17:15:14.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.279+0000 7fc7fd600640 1 -- 172.21.15.45:0/2527491937 learned_addr learned my addr 172.21.15.45:0/2527491937 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:15:14.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.279+0000 7fc7fe000640 1 --2- 172.21.15.45:0/2527491937 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55956b629f10 0x55956b62a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:14.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.279+0000 7fc7fcc00640 1 --2- 172.21.15.45:0/2527491937 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55956b633410 0x55956b635800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:14.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.279+0000 7fc7fd600640 1 -- 172.21.15.45:0/2527491937 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55956b633410 msgr2=0x55956b635800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:15:14.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.279+0000 7fc7fd600640 1 --2- 172.21.15.45:0/2527491937 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55956b633410 0x55956b635800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:14.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.279+0000 7fc7fd600640 1 -- 172.21.15.45:0/2527491937 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55956b629f10 msgr2=0x55956b62a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:15:14.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.279+0000 7fc7fd600640 1 --2- 172.21.15.45:0/2527491937 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55956b629f10 0x55956b62a2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:14.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.279+0000 7fc7fd600640 1 -- 172.21.15.45:0/2527491937 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55956b53d040 con 0x55956b62a830 2024-04-16T17:15:14.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.279+0000 7fc7fe000640 1 --2- 172.21.15.45:0/2527491937 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55956b629f10 0x55956b62a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:15:14.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.280+0000 7fc7fd600640 1 --2- 172.21.15.45:0/2527491937 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55956b62a830 0x55956b632ed0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fc7ec0efec0 tx=0x7fc7ec0f6c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=862a19cdd361f310 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:15:14.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.280+0000 7fc7ea600640 1 -- 172.21.15.45:0/2527491937 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7ec102020 con 0x55956b62a830 2024-04-16T17:15:14.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.280+0000 7fc7ea600640 1 -- 172.21.15.45:0/2527491937 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc7ec0fe3f0 con 0x55956b62a830 2024-04-16T17:15:14.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.280+0000 7fc7ea600640 1 -- 172.21.15.45:0/2527491937 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7ec0fd590 con 0x55956b62a830 2024-04-16T17:15:14.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.280+0000 7fc7f6000640 1 --2- 172.21.15.45:0/3017946957 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fc7e00a3d20 0x7fc7e00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:14.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.280+0000 7fc7f6000640 1 -- 172.21.15.45:0/3017946957 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9584+0+0 (secure 0 0 0) 0x7fc7ec0f2b40 con 0x7fc7f8103100 2024-04-16T17:15:14.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.280+0000 7fc7fe000640 1 --2- 172.21.15.45:0/3017946957 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fc7e00a3d20 0x7fc7e00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:14.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.281+0000 7fc802c02740 1 -- 172.21.15.45:0/2527491937 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55956b62a830 msgr2=0x55956b632ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:15:14.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.281+0000 7fc802c02740 1 --2- 172.21.15.45:0/2527491937 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55956b62a830 0x55956b632ed0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fc7ec0efec0 tx=0x7fc7ec0f6c20 comp rx=0 tx=0).stop 2024-04-16T17:15:14.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.281+0000 7fc802c02740 1 -- 172.21.15.45:0/2527491937 shutdown_connections 2024-04-16T17:15:14.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.281+0000 7fc802c02740 1 --2- 172.21.15.45:0/2527491937 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55956b633410 0x55956b635800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:14.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.281+0000 7fc802c02740 1 --2- 172.21.15.45:0/2527491937 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55956b62a830 0x55956b632ed0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:14.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.281+0000 7fc802c02740 1 --2- 172.21.15.45:0/2527491937 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55956b629f10 0x55956b62a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:14.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.281+0000 7fc802c02740 1 -- 172.21.15.45:0/2527491937 >> 172.21.15.45:0/2527491937 conn(0x55956b624640 msgr2=0x55956b624a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:15:14.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.281+0000 7fc802c02740 1 -- 172.21.15.45:0/2527491937 shutdown_connections 2024-04-16T17:15:14.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.281+0000 7fc802c02740 1 -- 172.21.15.45:0/2527491937 wait complete. 2024-04-16T17:15:14.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.281+0000 7fc7fe000640 1 --2- 172.21.15.45:0/3017946957 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fc7e00a3d20 0x7fc7e00a61e0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fc7f0009fd0 tx=0x7fc7f0009500 comp rx=0 tx=0).ready entity=mgr.25411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:15:14.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.282+0000 7fc802c02740 1 -- start start 2024-04-16T17:15:14.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.282+0000 7fc802c02740 1 Processor -- start 2024-04-16T17:15:14.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.282+0000 7fc802c02740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55956b629f10 0x55956b630440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:14.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.282+0000 7fc802c02740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55956b62a830 0x55956b630980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:14.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.282+0000 7fc802c02740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55956b633410 0x55956b630ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:14.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.282+0000 7fc802c02740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55956b635fc0 con 0x55956b62a830 2024-04-16T17:15:14.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.282+0000 7fc802c02740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55956b548830 con 0x55956b629f10 2024-04-16T17:15:14.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.282+0000 7fc802c02740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55956b636140 con 0x55956b633410 2024-04-16T17:15:14.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.282+0000 7fc802c02740 10 client.0 _ll_register_callbacks cb 0x55956b550150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:15:14.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.283+0000 7fc7fcc00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55956b62a830 0x55956b630980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:14.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.283+0000 7fc7fcc00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55956b62a830 0x55956b630980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:46040/0 (socket says 172.21.15.45:46040) 2024-04-16T17:15:14.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.283+0000 7fc7fcc00640 1 -- 172.21.15.45:0/2293842807 learned_addr learned my addr 172.21.15.45:0/2293842807 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:15:14.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.283+0000 7fc7fd600640 1 --2- 172.21.15.45:0/2293842807 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55956b629f10 0x55956b630440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:14.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.283+0000 7fc7fe000640 1 --2- 172.21.15.45:0/2293842807 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55956b633410 0x55956b630ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:14.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.283+0000 7fc7fd600640 1 -- 172.21.15.45:0/2293842807 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55956b633410 msgr2=0x55956b630ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:15:14.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.283+0000 7fc7fd600640 1 --2- 172.21.15.45:0/2293842807 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55956b633410 0x55956b630ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:14.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.283+0000 7fc7fd600640 1 -- 172.21.15.45:0/2293842807 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55956b62a830 msgr2=0x55956b630980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:15:14.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.283+0000 7fc7fd600640 1 --2- 172.21.15.45:0/2293842807 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55956b62a830 0x55956b630980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:14.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.283+0000 7fc7fd600640 1 -- 172.21.15.45:0/2293842807 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55956b54c920 con 0x55956b629f10 2024-04-16T17:15:14.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.283+0000 7fc7fd600640 1 --2- 172.21.15.45:0/2293842807 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55956b629f10 0x55956b630440 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fc7ec0be4d0 tx=0x7fc7ec0feae0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f15c1649e40a191a server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:15:14.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.283+0000 7fc7dea00640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-04-16T17:15:14.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.283+0000 7fc7dea00640 1 -- 172.21.15.45:0/2293842807 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7ec102070 con 0x55956b629f10 2024-04-16T17:15:14.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.284+0000 7fc7dea00640 1 -- 172.21.15.45:0/2293842807 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9584+0+0 (secure 0 0 0) 0x7fc7ec0bcad0 con 0x55956b629f10 2024-04-16T17:15:14.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.284+0000 7fc802c02740 1 -- 172.21.15.45:0/2293842807 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc7ec0fe3f0 con 0x55956b629f10 2024-04-16T17:15:14.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.284+0000 7fc7dea00640 1 -- 172.21.15.45:0/2293842807 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fc7ec0baa70 con 0x55956b629f10 2024-04-16T17:15:14.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.284+0000 7fc802c02740 10 client.27842 fetch_fsmap learned FSMap version 43 2024-04-16T17:15:14.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.284+0000 7fc802c02740 1 -- 172.21.15.45:0/2293842807 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55956b51a7f0 con 0x55956b629f10 2024-04-16T17:15:14.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.284+0000 7fc7dea00640 1 -- 172.21.15.45:0/2293842807 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc7ec0fe3f0 con 0x55956b629f10 2024-04-16T17:15:14.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.284+0000 7fc7dea00640 1 -- 172.21.15.45:0/2293842807 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7fc7ec0bae40 con 0x55956b629f10 2024-04-16T17:15:14.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.284+0000 7fc802c02740 10 client.27842 fetch_fsmap finished waiting for FSMap version 43 2024-04-16T17:15:14.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.284+0000 7fc802c02740 10 client.27842 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-16T17:15:14.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.284+0000 7fc802c02740 20 client.27842 populate_metadata read hostname 'smithi045' 2024-04-16T17:15:14.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.284+0000 7fc802c02740 1 --2- 172.21.15.45:0/2293842807 >> [v2:172.21.15.117:6838/2378675949,v1:172.21.15.117:6839/2378675949] conn(0x55956b643350 0x55956b645740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:14.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.284+0000 7fc802c02740 4 client.27842 mds_command: new command op to 24532 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:15:14.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.284+0000 7fc802c02740 1 -- 172.21.15.45:0/2293842807 --> [v2:172.21.15.117:6838/2378675949,v1:172.21.15.117:6839/2378675949] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55956b59ca40 con 0x55956b643350 2024-04-16T17:15:14.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.285+0000 7fc7fcc00640 1 --2- 172.21.15.45:0/2293842807 >> [v2:172.21.15.117:6838/2378675949,v1:172.21.15.117:6839/2378675949] conn(0x55956b643350 0x55956b645740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:14.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.285+0000 7fc7fcc00640 1 --2- 172.21.15.45:0/2293842807 >> [v2:172.21.15.117:6838/2378675949,v1:172.21.15.117:6839/2378675949] conn(0x55956b643350 0x55956b645740 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3d4ec0b405609237 server_cookie=e158ca63933f248c in_seq=0 out_seq=0 2024-04-16T17:15:14.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.285+0000 7fc7dea00640 10 client.27842 ms_handle_connect on v2:172.21.15.117:6838/2378675949 2024-04-16T17:15:14.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.286+0000 7fc7dea00640 1 -- 172.21.15.45:0/2293842807 <== mds.2 v2:172.21.15.117:6838/2378675949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55956b59ca40 con 0x55956b643350 2024-04-16T17:15:14.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.286+0000 7fc7dea00640 10 client.27842 handle_command_reply: tid=0 2024-04-16T17:15:14.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.286+0000 7fc802c02740 1 client.27842 shutdown 2024-04-16T17:15:14.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.286+0000 7fc802c02740 1 -- 172.21.15.45:0/2293842807 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55956b629f10 msgr2=0x55956b630440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:15:14.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.286+0000 7fc802c02740 1 --2- 172.21.15.45:0/2293842807 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55956b629f10 0x55956b630440 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fc7ec0be4d0 tx=0x7fc7ec0feae0 comp rx=0 tx=0).stop 2024-04-16T17:15:14.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.287+0000 7fc802c02740 1 -- 172.21.15.45:0/2293842807 shutdown_connections 2024-04-16T17:15:14.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.287+0000 7fc802c02740 1 --2- 172.21.15.45:0/2293842807 >> [v2:172.21.15.117:6838/2378675949,v1:172.21.15.117:6839/2378675949] conn(0x55956b643350 0x55956b645740 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:14.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.287+0000 7fc802c02740 1 --2- 172.21.15.45:0/2293842807 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55956b633410 0x55956b630ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:14.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.287+0000 7fc802c02740 1 --2- 172.21.15.45:0/2293842807 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55956b62a830 0x55956b630980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:14.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.287+0000 7fc7dea00640 0 client.27842 ms_handle_reset on v2:172.21.15.117:6838/2378675949 2024-04-16T17:15:14.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.287+0000 7fc802c02740 1 --2- 172.21.15.45:0/2293842807 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55956b629f10 0x55956b630440 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:14.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.287+0000 7fc802c02740 1 -- 172.21.15.45:0/2293842807 >> 172.21.15.45:0/2293842807 conn(0x55956b624640 msgr2=0x55956b626390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:15:14.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.287+0000 7fc802c02740 1 -- 172.21.15.45:0/2293842807 shutdown_connections 2024-04-16T17:15:14.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.287+0000 7fc802c02740 1 -- 172.21.15.45:0/2293842807 wait complete. 2024-04-16T17:15:14.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.287+0000 7fc802c02740 20 client.27842 trim_cache size 0 max 16384 2024-04-16T17:15:14.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.315+0000 7fc802c02740 1 Processor -- start 2024-04-16T17:15:14.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.315+0000 7fc802c02740 1 -- start start 2024-04-16T17:15:14.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.316+0000 7fc802c02740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55956b629f10 0x55956b632b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:14.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.316+0000 7fc802c02740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55956b62a830 0x55956b63edf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:14.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.316+0000 7fc802c02740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55956b633410 0x55956b623580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:14.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.316+0000 7fc802c02740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fc7f81b3fc0 con 0x55956b629f10 2024-04-16T17:15:14.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.316+0000 7fc802c02740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fc7f81b1890 con 0x55956b62a830 2024-04-16T17:15:14.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.316+0000 7fc802c02740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fc7f81af6e0 con 0x55956b633410 2024-04-16T17:15:14.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.316+0000 7fc7fd600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55956b629f10 0x55956b632b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:14.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.316+0000 7fc7fd600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55956b629f10 0x55956b632b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:46054/0 (socket says 172.21.15.45:46054) 2024-04-16T17:15:14.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.316+0000 7fc7fd600640 1 -- 172.21.15.45:0/1485932420 learned_addr learned my addr 172.21.15.45:0/1485932420 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:15:14.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.316+0000 7fc7fcc00640 1 --2- 172.21.15.45:0/1485932420 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55956b62a830 0x55956b63edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:14.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.316+0000 7fc7fe000640 1 --2- 172.21.15.45:0/1485932420 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55956b633410 0x55956b623580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:14.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.316+0000 7fc7fd600640 1 -- 172.21.15.45:0/1485932420 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55956b633410 msgr2=0x55956b623580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:15:14.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.317+0000 7fc7fd600640 1 --2- 172.21.15.45:0/1485932420 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55956b633410 0x55956b623580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:14.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.317+0000 7fc7fd600640 1 -- 172.21.15.45:0/1485932420 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55956b62a830 msgr2=0x55956b63edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:15:14.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.317+0000 7fc7fd600640 1 --2- 172.21.15.45:0/1485932420 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55956b62a830 0x55956b63edf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:14.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.317+0000 7fc7fd600640 1 -- 172.21.15.45:0/1485932420 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55956b62a660 con 0x55956b629f10 2024-04-16T17:15:14.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.317+0000 7fc7fcc00640 1 --2- 172.21.15.45:0/1485932420 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55956b62a830 0x55956b63edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:15:14.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.317+0000 7fc7fd600640 1 --2- 172.21.15.45:0/1485932420 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55956b629f10 0x55956b632b70 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fc7ec0bb1a0 tx=0x7fc7ec04f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27a732d78559a4c4 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:15:14.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.317+0000 7fc7e9c00640 1 -- 172.21.15.45:0/1485932420 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7ec102050 con 0x55956b629f10 2024-04-16T17:15:14.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.317+0000 7fc7e9c00640 1 -- 172.21.15.45:0/1485932420 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc7ec0f64c0 con 0x55956b629f10 2024-04-16T17:15:14.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.317+0000 7fc7e9c00640 1 -- 172.21.15.45:0/1485932420 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7ec0be130 con 0x55956b629f10 2024-04-16T17:15:14.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.318+0000 7fc802c02740 1 -- 172.21.15.45:0/1485932420 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55956b629f10 msgr2=0x55956b632b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:15:14.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.318+0000 7fc802c02740 1 --2- 172.21.15.45:0/1485932420 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55956b629f10 0x55956b632b70 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fc7ec0bb1a0 tx=0x7fc7ec04f000 comp rx=0 tx=0).stop 2024-04-16T17:15:14.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.318+0000 7fc802c02740 1 -- 172.21.15.45:0/1485932420 shutdown_connections 2024-04-16T17:15:14.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.318+0000 7fc802c02740 1 --2- 172.21.15.45:0/1485932420 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55956b633410 0x55956b623580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:14.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.318+0000 7fc802c02740 1 --2- 172.21.15.45:0/1485932420 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55956b62a830 0x55956b63edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:14.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.318+0000 7fc802c02740 1 --2- 172.21.15.45:0/1485932420 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55956b629f10 0x55956b632b70 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:14.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.318+0000 7fc802c02740 1 -- 172.21.15.45:0/1485932420 >> 172.21.15.45:0/1485932420 conn(0x55956b624640 msgr2=0x55956b622e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:15:14.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.318+0000 7fc802c02740 1 -- 172.21.15.45:0/1485932420 shutdown_connections 2024-04-16T17:15:14.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.318+0000 7fc802c02740 1 -- 172.21.15.45:0/1485932420 wait complete. 2024-04-16T17:15:14.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.318+0000 7fc802c02740 1 -- start start 2024-04-16T17:15:14.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.318+0000 7fc802c02740 1 Processor -- start 2024-04-16T17:15:14.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.319+0000 7fc802c02740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55956b629f10 0x55956b62d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:14.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.319+0000 7fc802c02740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55956b62a830 0x55956b62d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:14.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.319+0000 7fc802c02740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55956b633410 0x55956b6a8780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:14.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.319+0000 7fc802c02740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fc7f81afd50 con 0x55956b629f10 2024-04-16T17:15:14.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.319+0000 7fc802c02740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fc7f81b11c0 con 0x55956b62a830 2024-04-16T17:15:14.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.319+0000 7fc802c02740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fc7f81b0420 con 0x55956b633410 2024-04-16T17:15:14.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.319+0000 7fc802c02740 10 client.0 _ll_register_callbacks cb 0x7fc7f0009ee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:15:14.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.319+0000 7fc7fd600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55956b629f10 0x55956b62d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:14.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.319+0000 7fc7fcc00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55956b62a830 0x55956b62d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:14.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.319+0000 7fc7fd600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55956b629f10 0x55956b62d4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:46058/0 (socket says 172.21.15.45:46058) 2024-04-16T17:15:14.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.319+0000 7fc7fd600640 1 -- 172.21.15.45:0/849696849 learned_addr learned my addr 172.21.15.45:0/849696849 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:15:14.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.319+0000 7fc7fe000640 1 --2- 172.21.15.45:0/849696849 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55956b633410 0x55956b6a8780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:14.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.320+0000 7fc7fd600640 1 -- 172.21.15.45:0/849696849 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55956b633410 msgr2=0x55956b6a8780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:15:14.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.320+0000 7fc7fd600640 1 --2- 172.21.15.45:0/849696849 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55956b633410 0x55956b6a8780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:14.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.320+0000 7fc7fd600640 1 -- 172.21.15.45:0/849696849 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55956b62a830 msgr2=0x55956b62d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:15:14.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.320+0000 7fc7fd600640 1 --2- 172.21.15.45:0/849696849 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55956b62a830 0x55956b62d9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:14.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.320+0000 7fc7fd600640 1 -- 172.21.15.45:0/849696849 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55956b2a13f0 con 0x55956b629f10 2024-04-16T17:15:14.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.320+0000 7fc7fcc00640 1 --2- 172.21.15.45:0/849696849 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55956b62a830 0x55956b62d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:15:14.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.320+0000 7fc7fd600640 1 --2- 172.21.15.45:0/849696849 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55956b629f10 0x55956b62d4a0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fc7ec003fd0 tx=0x7fc7ec0042d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a43d9b1b0030dfd server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:15:14.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.320+0000 7fc7dea00640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-16T17:15:14.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.320+0000 7fc7dea00640 1 -- 172.21.15.45:0/849696849 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7ec102070 con 0x55956b629f10 2024-04-16T17:15:14.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.320+0000 7fc7dea00640 1 -- 172.21.15.45:0/849696849 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9584+0+0 (secure 0 0 0) 0x7fc7ec0bae40 con 0x55956b629f10 2024-04-16T17:15:14.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.320+0000 7fc802c02740 1 -- 172.21.15.45:0/849696849 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc7ec0f64c0 con 0x55956b629f10 2024-04-16T17:15:14.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.320+0000 7fc7dea00640 1 -- 172.21.15.45:0/849696849 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fc7ec0be310 con 0x55956b629f10 2024-04-16T17:15:14.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.320+0000 7fc802c02740 10 client.17688 fetch_fsmap learned FSMap version 43 2024-04-16T17:15:14.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.320+0000 7fc802c02740 1 -- 172.21.15.45:0/849696849 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55956b36b7d0 con 0x55956b629f10 2024-04-16T17:15:14.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.321+0000 7fc7dea00640 1 -- 172.21.15.45:0/849696849 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc7ec0f64c0 con 0x55956b629f10 2024-04-16T17:15:14.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.321+0000 7fc7dea00640 1 -- 172.21.15.45:0/849696849 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7fc7ec0d5070 con 0x55956b629f10 2024-04-16T17:15:14.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.321+0000 7fc802c02740 10 client.17688 fetch_fsmap finished waiting for FSMap version 43 2024-04-16T17:15:14.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.321+0000 7fc802c02740 10 client.17688 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-16T17:15:14.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.321+0000 7fc802c02740 20 client.17688 populate_metadata read hostname 'smithi045' 2024-04-16T17:15:14.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.321+0000 7fc802c02740 1 --2- 172.21.15.45:0/849696849 >> [v2:172.21.15.117:6838/2378675949,v1:172.21.15.117:6839/2378675949] conn(0x55956b6a9d40 0x55956b6ac130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:14.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.321+0000 7fc802c02740 4 client.17688 mds_command: new command op to 24532 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T17:15:14.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.321+0000 7fc802c02740 1 -- 172.21.15.45:0/849696849 --> [v2:172.21.15.117:6838/2378675949,v1:172.21.15.117:6839/2378675949] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55956b5968b0 con 0x55956b6a9d40 2024-04-16T17:15:14.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.321+0000 7fc7fcc00640 1 --2- 172.21.15.45:0/849696849 >> [v2:172.21.15.117:6838/2378675949,v1:172.21.15.117:6839/2378675949] conn(0x55956b6a9d40 0x55956b6ac130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:14.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.322+0000 7fc7fcc00640 1 --2- 172.21.15.45:0/849696849 >> [v2:172.21.15.117:6838/2378675949,v1:172.21.15.117:6839/2378675949] conn(0x55956b6a9d40 0x55956b6ac130 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f750e8a4e4737e84 server_cookie=a797140ac0efe3a1 in_seq=0 out_seq=0 2024-04-16T17:15:14.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.322+0000 7fc7dea00640 10 client.17688 ms_handle_connect on v2:172.21.15.117:6838/2378675949 2024-04-16T17:15:14.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.322+0000 7fc7dea00640 1 -- 172.21.15.45:0/849696849 <== mds.2 v2:172.21.15.117:6838/2378675949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55956b5968b0 con 0x55956b6a9d40 2024-04-16T17:15:14.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.322+0000 7fc7dea00640 10 client.17688 handle_command_reply: tid=0 2024-04-16T17:15:14.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.322+0000 7fc802c02740 1 client.17688 shutdown 2024-04-16T17:15:14.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.322+0000 7fc802c02740 1 -- 172.21.15.45:0/849696849 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55956b629f10 msgr2=0x55956b62d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:15:14.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.322+0000 7fc802c02740 1 --2- 172.21.15.45:0/849696849 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55956b629f10 0x55956b62d4a0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fc7ec003fd0 tx=0x7fc7ec0042d0 comp rx=0 tx=0).stop 2024-04-16T17:15:14.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.322+0000 7fc802c02740 1 -- 172.21.15.45:0/849696849 shutdown_connections 2024-04-16T17:15:14.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.322+0000 7fc802c02740 1 --2- 172.21.15.45:0/849696849 >> [v2:172.21.15.117:6838/2378675949,v1:172.21.15.117:6839/2378675949] conn(0x55956b6a9d40 0x55956b6ac130 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:14.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.323+0000 7fc802c02740 1 --2- 172.21.15.45:0/849696849 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55956b633410 0x55956b6a8780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:14.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.323+0000 7fc802c02740 1 --2- 172.21.15.45:0/849696849 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55956b62a830 0x55956b62d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:14.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.323+0000 7fc802c02740 1 --2- 172.21.15.45:0/849696849 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55956b629f10 0x55956b62d4a0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:14.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.323+0000 7fc802c02740 1 -- 172.21.15.45:0/849696849 >> 172.21.15.45:0/849696849 conn(0x55956b624640 msgr2=0x55956b623700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:15:14.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.323+0000 7fc7dea00640 0 client.17688 ms_handle_reset on v2:172.21.15.117:6838/2378675949 2024-04-16T17:15:14.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.323+0000 7fc802c02740 1 -- 172.21.15.45:0/849696849 shutdown_connections 2024-04-16T17:15:14.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.323+0000 7fc802c02740 1 -- 172.21.15.45:0/849696849 wait complete. 2024-04-16T17:15:14.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.323+0000 7fc802c02740 20 client.17688 trim_cache size 0 max 16384 2024-04-16T17:15:14.332 INFO:teuthology.orchestra.run.smithi045.stdout:[] 2024-04-16T17:15:14.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.323+0000 7fc7fea00640 1 -- 172.21.15.45:0/3017946957 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fc7e00a3d20 msgr2=0x7fc7e00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:15:14.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.323+0000 7fc7fea00640 1 --2- 172.21.15.45:0/3017946957 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fc7e00a3d20 0x7fc7e00a61e0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fc7f0009fd0 tx=0x7fc7f0009500 comp rx=0 tx=0).stop 2024-04-16T17:15:14.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.323+0000 7fc7fea00640 1 -- 172.21.15.45:0/3017946957 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc7f8103100 msgr2=0x7fc7f81a0620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:15:14.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.323+0000 7fc7fea00640 1 --2- 172.21.15.45:0/3017946957 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc7f8103100 0x7fc7f81a0620 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fc7ec036040 tx=0x7fc7ec02cbc0 comp rx=0 tx=0).stop 2024-04-16T17:15:14.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.324+0000 7fc7fea00640 1 -- 172.21.15.45:0/3017946957 shutdown_connections 2024-04-16T17:15:14.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.324+0000 7fc7fea00640 1 --2- 172.21.15.45:0/3017946957 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fc7e00a3d20 0x7fc7e00a61e0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:14.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.324+0000 7fc7fea00640 1 --2- 172.21.15.45:0/3017946957 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc7f8103a20 0x7fc7f81a0b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:14.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.324+0000 7fc7fea00640 1 --2- 172.21.15.45:0/3017946957 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc7f810c4d0 0x7fc7f819a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:14.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.324+0000 7fc7fea00640 1 --2- 172.21.15.45:0/3017946957 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc7f8103100 0x7fc7f81a0620 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:14.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.324+0000 7fc7fea00640 1 -- 172.21.15.45:0/3017946957 >> 172.21.15.45:0/3017946957 conn(0x7fc7f80fc880 msgr2=0x7fc7f8101ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:15:14.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.324+0000 7fc7fea00640 1 -- 172.21.15.45:0/3017946957 shutdown_connections 2024-04-16T17:15:14.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.324+0000 7fc7fea00640 1 -- 172.21.15.45:0/3017946957 wait complete. 2024-04-16T17:15:14.342 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-16T17:15:14.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.556+0000 7f03d7200640 1 -- 172.21.15.45:0/4070663048 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f03d8102ac0 msgr2=0x7f03d8106b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:15:14.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.556+0000 7f03d7200640 1 --2- 172.21.15.45:0/4070663048 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f03d8102ac0 0x7f03d8106b00 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f03c8009900 tx=0x7f03c802bc30 comp rx=0 tx=0).stop 2024-04-16T17:15:14.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.556+0000 7f03d7200640 1 -- 172.21.15.45:0/4070663048 shutdown_connections 2024-04-16T17:15:14.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.556+0000 7f03d7200640 1 --2- 172.21.15.45:0/4070663048 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f03d81071e0 0x7f03d810edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:14.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.556+0000 7f03d7200640 1 --2- 172.21.15.45:0/4070663048 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f03d8102ac0 0x7f03d8106b00 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:14.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.556+0000 7f03d7200640 1 --2- 172.21.15.45:0/4070663048 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f03d8102110 0x7f03d81024f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:14.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.556+0000 7f03d7200640 1 -- 172.21.15.45:0/4070663048 >> 172.21.15.45:0/4070663048 conn(0x7f03d80fd9d0 msgr2=0x7f03d80ffdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:15:14.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.556+0000 7f03d7200640 1 -- 172.21.15.45:0/4070663048 shutdown_connections 2024-04-16T17:15:14.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.556+0000 7f03d7200640 1 -- 172.21.15.45:0/4070663048 wait complete. 2024-04-16T17:15:14.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.557+0000 7f03d7200640 1 Processor -- start 2024-04-16T17:15:14.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.557+0000 7f03d7200640 1 -- start start 2024-04-16T17:15:14.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.558+0000 7f03d7200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f03d8102110 0x7f03d81a0620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:14.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.558+0000 7f03d7200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f03d81071e0 0x7f03d81a0b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:14.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.558+0000 7f03d7200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f03d819a870 0x7f03d819ad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:14.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.558+0000 7f03d7200640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f03d8112270 con 0x7f03d81071e0 2024-04-16T17:15:14.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.558+0000 7f03d7200640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f03d81120f0 con 0x7f03d819a870 2024-04-16T17:15:14.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.558+0000 7f03d7200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f03d81123f0 con 0x7f03d8102110 2024-04-16T17:15:14.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.558+0000 7f03d5e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f03d8102110 0x7f03d81a0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:14.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.558+0000 7f03d5e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f03d8102110 0x7f03d81a0620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:38090/0 (socket says 172.21.15.45:38090) 2024-04-16T17:15:14.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.558+0000 7f03d5400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f03d81071e0 0x7f03d81a0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:14.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.558+0000 7f03d5e00640 1 -- 172.21.15.45:0/4051471496 learned_addr learned my addr 172.21.15.45:0/4051471496 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:15:14.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.558+0000 7f03d6800640 1 --2- 172.21.15.45:0/4051471496 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f03d819a870 0x7f03d819ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:14.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.558+0000 7f03d5e00640 1 -- 172.21.15.45:0/4051471496 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f03d819a870 msgr2=0x7f03d819ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:15:14.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.558+0000 7f03d5e00640 1 --2- 172.21.15.45:0/4051471496 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f03d819a870 0x7f03d819ad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:14.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.558+0000 7f03d5e00640 1 -- 172.21.15.45:0/4051471496 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f03d81071e0 msgr2=0x7f03d81a0b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:15:14.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.558+0000 7f03d5e00640 1 --2- 172.21.15.45:0/4051471496 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f03d81071e0 0x7f03d81a0b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:14.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.558+0000 7f03d5e00640 1 -- 172.21.15.45:0/4051471496 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03d819b390 con 0x7f03d8102110 2024-04-16T17:15:14.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.558+0000 7f03d6800640 1 --2- 172.21.15.45:0/4051471496 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f03d819a870 0x7f03d819ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:15:14.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.558+0000 7f03d5400640 1 --2- 172.21.15.45:0/4051471496 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f03d81071e0 0x7f03d81a0b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:15:14.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.559+0000 7f03d5e00640 1 --2- 172.21.15.45:0/4051471496 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f03d8102110 0x7f03d81a0620 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f03d000ebc0 tx=0x7f03d000c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:15:14.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.559+0000 7f03cea00640 1 -- 172.21.15.45:0/4051471496 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03d0010630 con 0x7f03d8102110 2024-04-16T17:15:14.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.559+0000 7f03d7200640 1 -- 172.21.15.45:0/4051471496 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03d819b680 con 0x7f03d8102110 2024-04-16T17:15:14.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.559+0000 7f03cea00640 1 -- 172.21.15.45:0/4051471496 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f03d0010cb0 con 0x7f03d8102110 2024-04-16T17:15:14.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.559+0000 7f03cea00640 1 -- 172.21.15.45:0/4051471496 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03d0018bd0 con 0x7f03d8102110 2024-04-16T17:15:14.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.559+0000 7f03d7200640 1 -- 172.21.15.45:0/4051471496 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03d81a7830 con 0x7f03d8102110 2024-04-16T17:15:14.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.562+0000 7f03df9f2740 1 Processor -- start 2024-04-16T17:15:14.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.562+0000 7f03df9f2740 1 -- start start 2024-04-16T17:15:14.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.562+0000 7f03cea00640 1 -- 172.21.15.45:0/4051471496 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f03d00107d0 con 0x7f03d8102110 2024-04-16T17:15:14.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.563+0000 7f03df9f2740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56278c758f10 0x56278c7592f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:14.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.563+0000 7f03df9f2740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56278c759830 0x56278c761ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:14.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.563+0000 7f03df9f2740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56278c762410 0x56278c764800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:14.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.563+0000 7f03df9f2740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x56278c6be020 con 0x56278c759830 2024-04-16T17:15:14.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.563+0000 7f03df9f2740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x56278c664460 con 0x56278c762410 2024-04-16T17:15:14.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.563+0000 7f03df9f2740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x56278c6b9fe0 con 0x56278c758f10 2024-04-16T17:15:14.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.563+0000 7f03d6800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56278c758f10 0x56278c7592f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:14.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.563+0000 7f03d6800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56278c758f10 0x56278c7592f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:38098/0 (socket says 172.21.15.45:38098) 2024-04-16T17:15:14.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.563+0000 7f03d5e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56278c759830 0x56278c761ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:14.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.563+0000 7f03d6800640 1 -- 172.21.15.45:0/3720895338 learned_addr learned my addr 172.21.15.45:0/3720895338 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:15:14.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.563+0000 7f03d5400640 1 --2- 172.21.15.45:0/3720895338 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56278c762410 0x56278c764800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:14.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.564+0000 7f03d6800640 1 -- 172.21.15.45:0/3720895338 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56278c762410 msgr2=0x56278c764800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:15:14.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.564+0000 7f03d6800640 1 --2- 172.21.15.45:0/3720895338 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56278c762410 0x56278c764800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:14.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.564+0000 7f03d6800640 1 -- 172.21.15.45:0/3720895338 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56278c759830 msgr2=0x56278c761ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:15:14.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.564+0000 7f03d6800640 1 --2- 172.21.15.45:0/3720895338 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56278c759830 0x56278c761ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:14.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.564+0000 7f03d6800640 1 -- 172.21.15.45:0/3720895338 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56278c66c040 con 0x56278c758f10 2024-04-16T17:15:14.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.564+0000 7f03d6800640 1 --2- 172.21.15.45:0/3720895338 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56278c758f10 0x56278c7592f0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f03c0009c20 tx=0x7f03c00117b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=64953d7b719ba79e server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:15:14.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.564+0000 7f03cea00640 1 --2- 172.21.15.45:0/4051471496 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f03bc0a3d20 0x7f03bc0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:14.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.564+0000 7f03c7400640 1 -- 172.21.15.45:0/3720895338 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03c0012450 con 0x56278c758f10 2024-04-16T17:15:14.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.564+0000 7f03c7400640 1 -- 172.21.15.45:0/3720895338 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f03c0012ad0 con 0x56278c758f10 2024-04-16T17:15:14.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.564+0000 7f03c7400640 1 -- 172.21.15.45:0/3720895338 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03c000fc10 con 0x56278c758f10 2024-04-16T17:15:14.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.564+0000 7f03cea00640 1 -- 172.21.15.45:0/4051471496 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9496+0+0 (secure 0 0 0) 0x7f03d00d12d0 con 0x7f03d8102110 2024-04-16T17:15:14.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.565+0000 7f03d5e00640 1 --2- 172.21.15.45:0/4051471496 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f03bc0a3d20 0x7f03bc0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:14.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.565+0000 7f03df9f2740 1 -- 172.21.15.45:0/3720895338 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56278c758f10 msgr2=0x56278c7592f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:15:14.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.565+0000 7f03df9f2740 1 --2- 172.21.15.45:0/3720895338 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56278c758f10 0x56278c7592f0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f03c0009c20 tx=0x7f03c00117b0 comp rx=0 tx=0).stop 2024-04-16T17:15:14.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.565+0000 7f03d5e00640 1 --2- 172.21.15.45:0/4051471496 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f03bc0a3d20 0x7f03bc0a61e0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f03d00d4fd0 tx=0x7f03d00d4d80 comp rx=0 tx=0).ready entity=mgr.25411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:15:14.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.565+0000 7f03df9f2740 1 -- 172.21.15.45:0/3720895338 shutdown_connections 2024-04-16T17:15:14.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.565+0000 7f03df9f2740 1 --2- 172.21.15.45:0/3720895338 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56278c762410 0x56278c764800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:14.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.565+0000 7f03df9f2740 1 --2- 172.21.15.45:0/3720895338 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56278c759830 0x56278c761ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:14.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.565+0000 7f03df9f2740 1 --2- 172.21.15.45:0/3720895338 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56278c758f10 0x56278c7592f0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:14.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.565+0000 7f03df9f2740 1 -- 172.21.15.45:0/3720895338 >> 172.21.15.45:0/3720895338 conn(0x56278c753640 msgr2=0x56278c753a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:15:14.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.566+0000 7f03df9f2740 1 -- 172.21.15.45:0/3720895338 shutdown_connections 2024-04-16T17:15:14.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.566+0000 7f03df9f2740 1 -- 172.21.15.45:0/3720895338 wait complete. 2024-04-16T17:15:14.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.566+0000 7f03df9f2740 1 -- start start 2024-04-16T17:15:14.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.566+0000 7f03df9f2740 1 Processor -- start 2024-04-16T17:15:14.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.567+0000 7f03df9f2740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56278c758f10 0x56278c75f440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:14.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.567+0000 7f03df9f2740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56278c759830 0x56278c75f980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:14.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.567+0000 7f03df9f2740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56278c762410 0x56278c75fec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:14.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.567+0000 7f03df9f2740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x56278c765200 con 0x56278c762410 2024-04-16T17:15:14.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.567+0000 7f03df9f2740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x56278c677830 con 0x56278c759830 2024-04-16T17:15:14.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.567+0000 7f03df9f2740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x56278c765380 con 0x56278c758f10 2024-04-16T17:15:14.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.567+0000 7f03df9f2740 10 client.0 _ll_register_callbacks cb 0x56278c67f150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:15:14.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.567+0000 7f03d5400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56278c759830 0x56278c75f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:14.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.567+0000 7f03d5400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56278c759830 0x56278c75f980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:38952/0 (socket says 172.21.15.45:38952) 2024-04-16T17:15:14.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.567+0000 7f03d5400640 1 -- 172.21.15.45:0/1003601400 learned_addr learned my addr 172.21.15.45:0/1003601400 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:15:14.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.567+0000 7f03d6800640 1 --2- 172.21.15.45:0/1003601400 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56278c758f10 0x56278c75f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:14.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.567+0000 7f03d6800640 1 --2- 172.21.15.45:0/1003601400 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56278c762410 0x56278c75fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:14.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.568+0000 7f03d5400640 1 -- 172.21.15.45:0/1003601400 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56278c758f10 msgr2=0x56278c75f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:15:14.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.568+0000 7f03d5400640 1 --2- 172.21.15.45:0/1003601400 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56278c758f10 0x56278c75f440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:14.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.568+0000 7f03d5400640 1 -- 172.21.15.45:0/1003601400 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56278c762410 msgr2=0x56278c75fec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:15:14.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.568+0000 7f03d5400640 1 --2- 172.21.15.45:0/1003601400 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56278c762410 0x56278c75fec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:14.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.568+0000 7f03d5400640 1 -- 172.21.15.45:0/1003601400 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56278c67b920 con 0x56278c759830 2024-04-16T17:15:14.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.568+0000 7f03d5400640 1 --2- 172.21.15.45:0/1003601400 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56278c759830 0x56278c75f980 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f03d806b560 tx=0x7f03c802c880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=21e97392439fc449 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:15:14.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.568+0000 7f03bbe00640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-04-16T17:15:14.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.569+0000 7f03bbe00640 1 -- 172.21.15.45:0/1003601400 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03c802cbf0 con 0x56278c759830 2024-04-16T17:15:14.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.569+0000 7f03df9f2740 1 -- 172.21.15.45:0/1003601400 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f03c0012ad0 con 0x56278c759830 2024-04-16T17:15:14.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.569+0000 7f03bbe00640 1 -- 172.21.15.45:0/1003601400 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9496+0+0 (secure 0 0 0) 0x7f03c8004040 con 0x56278c759830 2024-04-16T17:15:14.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.570+0000 7f03bbe00640 1 -- 172.21.15.45:0/1003601400 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f03c8004240 con 0x56278c759830 2024-04-16T17:15:14.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.570+0000 7f03df9f2740 10 client.27860 fetch_fsmap learned FSMap version 43 2024-04-16T17:15:14.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.570+0000 7f03df9f2740 1 -- 172.21.15.45:0/1003601400 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x56278c6497f0 con 0x56278c759830 2024-04-16T17:15:14.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.570+0000 7f03bbe00640 1 -- 172.21.15.45:0/1003601400 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f03c0012ad0 con 0x56278c759830 2024-04-16T17:15:14.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.570+0000 7f03bbe00640 1 -- 172.21.15.45:0/1003601400 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f03c8003890 con 0x56278c759830 2024-04-16T17:15:14.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.570+0000 7f03df9f2740 10 client.27860 fetch_fsmap finished waiting for FSMap version 43 2024-04-16T17:15:14.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.570+0000 7f03df9f2740 10 client.27860 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-04-16T17:15:14.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.570+0000 7f03df9f2740 20 client.27860 populate_metadata read hostname 'smithi045' 2024-04-16T17:15:14.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.570+0000 7f03df9f2740 1 --2- 172.21.15.45:0/1003601400 >> [v2:172.21.15.45:6840/3755310822,v1:172.21.15.45:6841/3755310822] conn(0x56278c772800 0x56278c774c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:14.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.570+0000 7f03df9f2740 4 client.27860 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:15:14.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.570+0000 7f03df9f2740 1 -- 172.21.15.45:0/1003601400 --> [v2:172.21.15.45:6840/3755310822,v1:172.21.15.45:6841/3755310822] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56278c6cba40 con 0x56278c772800 2024-04-16T17:15:14.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.571+0000 7f03d6800640 1 --2- 172.21.15.45:0/1003601400 >> [v2:172.21.15.45:6840/3755310822,v1:172.21.15.45:6841/3755310822] conn(0x56278c772800 0x56278c774c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:14.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.571+0000 7f03d6800640 1 --2- 172.21.15.45:0/1003601400 >> [v2:172.21.15.45:6840/3755310822,v1:172.21.15.45:6841/3755310822] conn(0x56278c772800 0x56278c774c20 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=48e40b1bc39b99d0 server_cookie=43147230da591eec in_seq=0 out_seq=0 2024-04-16T17:15:14.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.571+0000 7f03bbe00640 10 client.27860 ms_handle_connect on v2:172.21.15.45:6840/3755310822 2024-04-16T17:15:14.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.572+0000 7f03bbe00640 1 -- 172.21.15.45:0/1003601400 <== mds.1 v2:172.21.15.45:6840/3755310822 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56278c6cba40 con 0x56278c772800 2024-04-16T17:15:14.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.572+0000 7f03bbe00640 10 client.27860 handle_command_reply: tid=0 2024-04-16T17:15:14.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.572+0000 7f03df9f2740 1 client.27860 shutdown 2024-04-16T17:15:14.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.573+0000 7f03df9f2740 1 -- 172.21.15.45:0/1003601400 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56278c759830 msgr2=0x56278c75f980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:15:14.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.573+0000 7f03df9f2740 1 --2- 172.21.15.45:0/1003601400 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56278c759830 0x56278c75f980 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f03d806b560 tx=0x7f03c802c880 comp rx=0 tx=0).stop 2024-04-16T17:15:14.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.573+0000 7f03df9f2740 1 -- 172.21.15.45:0/1003601400 shutdown_connections 2024-04-16T17:15:14.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.573+0000 7f03df9f2740 1 --2- 172.21.15.45:0/1003601400 >> [v2:172.21.15.45:6840/3755310822,v1:172.21.15.45:6841/3755310822] conn(0x56278c772800 0x56278c774c20 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:14.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.573+0000 7f03df9f2740 1 --2- 172.21.15.45:0/1003601400 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56278c762410 0x56278c75fec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:14.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.573+0000 7f03df9f2740 1 --2- 172.21.15.45:0/1003601400 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56278c759830 0x56278c75f980 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:14.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.573+0000 7f03bbe00640 0 client.27860 ms_handle_reset on v2:172.21.15.45:6840/3755310822 2024-04-16T17:15:14.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.573+0000 7f03df9f2740 1 --2- 172.21.15.45:0/1003601400 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56278c758f10 0x56278c75f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:14.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.573+0000 7f03df9f2740 1 -- 172.21.15.45:0/1003601400 >> 172.21.15.45:0/1003601400 conn(0x56278c753640 msgr2=0x56278c755390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:15:14.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.573+0000 7f03df9f2740 1 -- 172.21.15.45:0/1003601400 shutdown_connections 2024-04-16T17:15:14.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.573+0000 7f03df9f2740 1 -- 172.21.15.45:0/1003601400 wait complete. 2024-04-16T17:15:14.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.573+0000 7f03df9f2740 20 client.27860 trim_cache size 0 max 16384 2024-04-16T17:15:14.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.588+0000 7f03df9f2740 1 Processor -- start 2024-04-16T17:15:14.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.588+0000 7f03df9f2740 1 -- start start 2024-04-16T17:15:14.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.588+0000 7f03df9f2740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56278c758f10 0x56278c76e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:14.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.588+0000 7f03df9f2740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56278c759830 0x56278c75fce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:14.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.588+0000 7f03df9f2740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56278c762410 0x56278c77a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:14.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.588+0000 7f03df9f2740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f03d81b3ec0 con 0x56278c762410 2024-04-16T17:15:14.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.588+0000 7f03df9f2740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f03d81b1790 con 0x56278c758f10 2024-04-16T17:15:14.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.588+0000 7f03df9f2740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f03d81af600 con 0x56278c759830 2024-04-16T17:15:14.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.589+0000 7f03d6800640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56278c758f10 0x56278c76e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:14.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.589+0000 7f03d6800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56278c762410 0x56278c77a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:14.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.589+0000 7f03d5400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56278c759830 0x56278c75fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:14.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.589+0000 7f03d6800640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56278c758f10 0x56278c76e0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:38964/0 (socket says 172.21.15.45:38964) 2024-04-16T17:15:14.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.589+0000 7f03d6800640 1 -- 172.21.15.45:0/1339840627 learned_addr learned my addr 172.21.15.45:0/1339840627 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:15:14.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.589+0000 7f03d5400640 1 -- 172.21.15.45:0/1339840627 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56278c758f10 msgr2=0x56278c76e0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:15:14.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.589+0000 7f03d5400640 1 --2- 172.21.15.45:0/1339840627 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56278c758f10 0x56278c76e0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:14.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.589+0000 7f03d5400640 1 -- 172.21.15.45:0/1339840627 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56278c762410 msgr2=0x56278c77a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:15:14.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.589+0000 7f03d5400640 1 --2- 172.21.15.45:0/1339840627 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56278c762410 0x56278c77a4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:14.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.589+0000 7f03d5400640 1 -- 172.21.15.45:0/1339840627 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56278c75e410 con 0x56278c759830 2024-04-16T17:15:14.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.589+0000 7f03d6800640 1 --2- 172.21.15.45:0/1339840627 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56278c762410 0x56278c77a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:15:14.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.590+0000 7f03d5400640 1 --2- 172.21.15.45:0/1339840627 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56278c759830 0x56278c75fce0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f03c802c170 tx=0x7f03c8000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5bac5b05326784b9 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:15:14.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.590+0000 7f03c6a00640 1 -- 172.21.15.45:0/1339840627 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03c8009520 con 0x56278c759830 2024-04-16T17:15:14.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.590+0000 7f03c6a00640 1 -- 172.21.15.45:0/1339840627 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f03c8003e80 con 0x56278c759830 2024-04-16T17:15:14.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.590+0000 7f03c6a00640 1 -- 172.21.15.45:0/1339840627 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03c8005180 con 0x56278c759830 2024-04-16T17:15:14.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.591+0000 7f03df9f2740 1 -- 172.21.15.45:0/1339840627 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56278c759830 msgr2=0x56278c75fce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:15:14.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.591+0000 7f03df9f2740 1 --2- 172.21.15.45:0/1339840627 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56278c759830 0x56278c75fce0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f03c802c170 tx=0x7f03c8000bc0 comp rx=0 tx=0).stop 2024-04-16T17:15:14.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.591+0000 7f03df9f2740 1 -- 172.21.15.45:0/1339840627 shutdown_connections 2024-04-16T17:15:14.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.591+0000 7f03df9f2740 1 --2- 172.21.15.45:0/1339840627 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56278c759830 0x56278c75fce0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:14.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.591+0000 7f03df9f2740 1 --2- 172.21.15.45:0/1339840627 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56278c762410 0x56278c77a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:14.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.591+0000 7f03df9f2740 1 --2- 172.21.15.45:0/1339840627 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56278c758f10 0x56278c76e0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:14.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.591+0000 7f03df9f2740 1 -- 172.21.15.45:0/1339840627 >> 172.21.15.45:0/1339840627 conn(0x56278c753640 msgr2=0x56278c7551d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:15:14.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.591+0000 7f03df9f2740 1 -- 172.21.15.45:0/1339840627 shutdown_connections 2024-04-16T17:15:14.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.591+0000 7f03df9f2740 1 -- 172.21.15.45:0/1339840627 wait complete. 2024-04-16T17:15:14.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.591+0000 7f03df9f2740 1 -- start start 2024-04-16T17:15:14.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.592+0000 7f03df9f2740 1 Processor -- start 2024-04-16T17:15:14.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.592+0000 7f03df9f2740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56278c758f10 0x56278c75c420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:14.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.592+0000 7f03df9f2740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56278c759830 0x56278c75c960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:14.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.592+0000 7f03df9f2740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56278c762410 0x56278c75cea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:14.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.592+0000 7f03df9f2740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f03d81afc50 con 0x56278c762410 2024-04-16T17:15:14.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.592+0000 7f03df9f2740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f03d81b10c0 con 0x56278c758f10 2024-04-16T17:15:14.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.592+0000 7f03df9f2740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f03d81b0320 con 0x56278c759830 2024-04-16T17:15:14.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.592+0000 7f03df9f2740 10 client.0 _ll_register_callbacks cb 0x7f03c8009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:15:14.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.592+0000 7f03d5400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56278c759830 0x56278c75c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:14.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.592+0000 7f03d6800640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56278c758f10 0x56278c75c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:14.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.592+0000 7f03d5400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56278c759830 0x56278c75c960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:38130/0 (socket says 172.21.15.45:38130) 2024-04-16T17:15:14.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.592+0000 7f03d5400640 1 -- 172.21.15.45:0/4029331580 learned_addr learned my addr 172.21.15.45:0/4029331580 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:15:14.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.592+0000 7f03d6800640 1 --2- 172.21.15.45:0/4029331580 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56278c762410 0x56278c75cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:14.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.593+0000 7f03d5400640 1 -- 172.21.15.45:0/4029331580 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56278c758f10 msgr2=0x56278c75c420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:15:14.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.593+0000 7f03d5400640 1 --2- 172.21.15.45:0/4029331580 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56278c758f10 0x56278c75c420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:14.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.593+0000 7f03d5400640 1 -- 172.21.15.45:0/4029331580 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56278c762410 msgr2=0x56278c75cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:15:14.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.593+0000 7f03d5400640 1 --2- 172.21.15.45:0/4029331580 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56278c762410 0x56278c75cea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:14.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.593+0000 7f03d5400640 1 -- 172.21.15.45:0/4029331580 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56278c3d03f0 con 0x56278c759830 2024-04-16T17:15:14.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.593+0000 7f03d5400640 1 --2- 172.21.15.45:0/4029331580 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56278c759830 0x56278c75c960 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f03c8009900 tx=0x7f03c80095a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f526eb95c7d1a8f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:15:14.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.593+0000 7f03bbe00640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-04-16T17:15:14.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.593+0000 7f03bbe00640 1 -- 172.21.15.45:0/4029331580 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03c802c880 con 0x56278c759830 2024-04-16T17:15:14.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.594+0000 7f03df9f2740 1 -- 172.21.15.45:0/4029331580 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f03c8003e80 con 0x56278c759830 2024-04-16T17:15:14.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.594+0000 7f03bbe00640 1 -- 172.21.15.45:0/4029331580 <== mon.1 v2:172.21.15.117:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9496+0+0 (secure 0 0 0) 0x7f03c802ca20 con 0x56278c759830 2024-04-16T17:15:14.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.594+0000 7f03bbe00640 1 -- 172.21.15.45:0/4029331580 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f03c8066050 con 0x56278c759830 2024-04-16T17:15:14.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.594+0000 7f03df9f2740 10 client.27808 fetch_fsmap learned FSMap version 43 2024-04-16T17:15:14.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.594+0000 7f03df9f2740 1 -- 172.21.15.45:0/4029331580 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x56278c49a7d0 con 0x56278c759830 2024-04-16T17:15:14.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.594+0000 7f03bbe00640 1 -- 172.21.15.45:0/4029331580 <== mon.1 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f03c8003e80 con 0x56278c759830 2024-04-16T17:15:14.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.594+0000 7f03bbe00640 1 -- 172.21.15.45:0/4029331580 <== mon.1 v2:172.21.15.117:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f03c80347c0 con 0x56278c759830 2024-04-16T17:15:14.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.594+0000 7f03df9f2740 10 client.27808 fetch_fsmap finished waiting for FSMap version 43 2024-04-16T17:15:14.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.594+0000 7f03df9f2740 10 client.27808 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-04-16T17:15:14.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.594+0000 7f03df9f2740 20 client.27808 populate_metadata read hostname 'smithi045' 2024-04-16T17:15:14.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.594+0000 7f03df9f2740 1 --2- 172.21.15.45:0/4029331580 >> [v2:172.21.15.45:6840/3755310822,v1:172.21.15.45:6841/3755310822] conn(0x56278c7d93d0 0x56278c7db7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:14.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.594+0000 7f03df9f2740 4 client.27808 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T17:15:14.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.594+0000 7f03df9f2740 1 -- 172.21.15.45:0/4029331580 --> [v2:172.21.15.45:6840/3755310822,v1:172.21.15.45:6841/3755310822] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56278c6c58b0 con 0x56278c7d93d0 2024-04-16T17:15:14.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.595+0000 7f03d6800640 1 --2- 172.21.15.45:0/4029331580 >> [v2:172.21.15.45:6840/3755310822,v1:172.21.15.45:6841/3755310822] conn(0x56278c7d93d0 0x56278c7db7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:14.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.595+0000 7f03d6800640 1 --2- 172.21.15.45:0/4029331580 >> [v2:172.21.15.45:6840/3755310822,v1:172.21.15.45:6841/3755310822] conn(0x56278c7d93d0 0x56278c7db7c0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d0811f7a58722f46 server_cookie=ddd7d23e10fc359d in_seq=0 out_seq=0 2024-04-16T17:15:14.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.595+0000 7f03bbe00640 10 client.27808 ms_handle_connect on v2:172.21.15.45:6840/3755310822 2024-04-16T17:15:14.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.595+0000 7f03bbe00640 1 -- 172.21.15.45:0/4029331580 <== mds.1 v2:172.21.15.45:6840/3755310822 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56278c6c58b0 con 0x56278c7d93d0 2024-04-16T17:15:14.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.595+0000 7f03bbe00640 10 client.27808 handle_command_reply: tid=0 2024-04-16T17:15:14.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.596+0000 7f03df9f2740 1 client.27808 shutdown 2024-04-16T17:15:14.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.596+0000 7f03df9f2740 1 -- 172.21.15.45:0/4029331580 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56278c759830 msgr2=0x56278c75c960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:15:14.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.596+0000 7f03df9f2740 1 --2- 172.21.15.45:0/4029331580 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56278c759830 0x56278c75c960 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f03c8009900 tx=0x7f03c80095a0 comp rx=0 tx=0).stop 2024-04-16T17:15:14.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.596+0000 7f03df9f2740 1 -- 172.21.15.45:0/4029331580 shutdown_connections 2024-04-16T17:15:14.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.596+0000 7f03df9f2740 1 --2- 172.21.15.45:0/4029331580 >> [v2:172.21.15.45:6840/3755310822,v1:172.21.15.45:6841/3755310822] conn(0x56278c7d93d0 0x56278c7db7c0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:14.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.596+0000 7f03df9f2740 1 --2- 172.21.15.45:0/4029331580 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56278c759830 0x56278c75c960 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:14.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.596+0000 7f03df9f2740 1 --2- 172.21.15.45:0/4029331580 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56278c762410 0x56278c75cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:14.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.596+0000 7f03df9f2740 1 --2- 172.21.15.45:0/4029331580 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56278c758f10 0x56278c75c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:14.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.596+0000 7f03bbe00640 0 client.27808 ms_handle_reset on v2:172.21.15.45:6840/3755310822 2024-04-16T17:15:14.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.596+0000 7f03df9f2740 1 -- 172.21.15.45:0/4029331580 >> 172.21.15.45:0/4029331580 conn(0x56278c753640 msgr2=0x56278c7547d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:15:14.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.596+0000 7f03df9f2740 1 -- 172.21.15.45:0/4029331580 shutdown_connections 2024-04-16T17:15:14.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.596+0000 7f03df9f2740 1 -- 172.21.15.45:0/4029331580 wait complete. 2024-04-16T17:15:14.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.596+0000 7f03df9f2740 20 client.27808 trim_cache size 0 max 16384 2024-04-16T17:15:14.607 INFO:teuthology.orchestra.run.smithi045.stdout:[] 2024-04-16T17:15:14.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.597+0000 7f03d7200640 1 -- 172.21.15.45:0/4051471496 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f03bc0a3d20 msgr2=0x7f03bc0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:15:14.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.597+0000 7f03d7200640 1 --2- 172.21.15.45:0/4051471496 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f03bc0a3d20 0x7f03bc0a61e0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f03d00d4fd0 tx=0x7f03d00d4d80 comp rx=0 tx=0).stop 2024-04-16T17:15:14.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.597+0000 7f03d7200640 1 -- 172.21.15.45:0/4051471496 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f03d8102110 msgr2=0x7f03d81a0620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:15:14.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.597+0000 7f03d7200640 1 --2- 172.21.15.45:0/4051471496 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f03d8102110 0x7f03d81a0620 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f03d000ebc0 tx=0x7f03d000c6a0 comp rx=0 tx=0).stop 2024-04-16T17:15:14.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.597+0000 7f03d7200640 1 -- 172.21.15.45:0/4051471496 shutdown_connections 2024-04-16T17:15:14.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.597+0000 7f03d7200640 1 --2- 172.21.15.45:0/4051471496 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f03bc0a3d20 0x7f03bc0a61e0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:14.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.597+0000 7f03d7200640 1 --2- 172.21.15.45:0/4051471496 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f03d819a870 0x7f03d819ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:14.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.597+0000 7f03d7200640 1 --2- 172.21.15.45:0/4051471496 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f03d81071e0 0x7f03d81a0b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:14.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.597+0000 7f03d7200640 1 --2- 172.21.15.45:0/4051471496 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f03d8102110 0x7f03d81a0620 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:14.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.597+0000 7f03d7200640 1 -- 172.21.15.45:0/4051471496 >> 172.21.15.45:0/4051471496 conn(0x7f03d80fd9d0 msgr2=0x7f03d80ffd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:15:14.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.597+0000 7f03d7200640 1 -- 172.21.15.45:0/4051471496 shutdown_connections 2024-04-16T17:15:14.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.597+0000 7f03d7200640 1 -- 172.21.15.45:0/4051471496 wait complete. 2024-04-16T17:15:14.612 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:15:14 smithi045 ceph-mon[31676]: pgmap v263: 129 pgs: 129 active+clean; 59 MiB data, 876 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 18 KiB/s wr, 8 op/s 2024-04-16T17:15:14.612 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:15:14 smithi045 ceph-mon[31676]: from='client.? 172.21.15.45:0/3400439147' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:15:14.612 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:15:14 smithi045 ceph-mon[31676]: osdmap e102: 12 total, 12 up, 12 in 2024-04-16T17:15:14.616 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-16T17:15:14.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.823+0000 7fe326000640 1 -- 172.21.15.45:0/1839849420 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe320104e60 msgr2=0x7fe320108ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:15:14.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.823+0000 7fe326000640 1 --2- 172.21.15.45:0/1839849420 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe320104e60 0x7fe320108ea0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fe30c009900 tx=0x7fe30c02bc30 comp rx=0 tx=0).stop 2024-04-16T17:15:14.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.824+0000 7fe326000640 1 -- 172.21.15.45:0/1839849420 shutdown_connections 2024-04-16T17:15:14.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.824+0000 7fe326000640 1 --2- 172.21.15.45:0/1839849420 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe320109580 0x7fe320111150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:14.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.824+0000 7fe326000640 1 --2- 172.21.15.45:0/1839849420 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe320104e60 0x7fe320108ea0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:14.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.824+0000 7fe326000640 1 --2- 172.21.15.45:0/1839849420 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe3201044b0 0x7fe320104890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:14.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.824+0000 7fe326000640 1 -- 172.21.15.45:0/1839849420 >> 172.21.15.45:0/1839849420 conn(0x7fe3200ffd90 msgr2=0x7fe3201021b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:15:14.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.824+0000 7fe326000640 1 -- 172.21.15.45:0/1839849420 shutdown_connections 2024-04-16T17:15:14.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.824+0000 7fe326000640 1 -- 172.21.15.45:0/1839849420 wait complete. 2024-04-16T17:15:14.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.825+0000 7fe326000640 1 Processor -- start 2024-04-16T17:15:14.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.825+0000 7fe326000640 1 -- start start 2024-04-16T17:15:14.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.825+0000 7fe326000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe3201044b0 0x7fe3201a07e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:14.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.825+0000 7fe326000640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe320109580 0x7fe3201a0d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:14.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.825+0000 7fe326000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe32019a990 0x7fe32019ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:14.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.825+0000 7fe326000640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fe320114610 con 0x7fe3201044b0 2024-04-16T17:15:14.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.825+0000 7fe326000640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fe320114490 con 0x7fe320109580 2024-04-16T17:15:14.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.825+0000 7fe326000640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fe320114790 con 0x7fe32019a990 2024-04-16T17:15:14.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.825+0000 7fe31f400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe3201044b0 0x7fe3201a07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:14.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.826+0000 7fe31f400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe3201044b0 0x7fe3201a07e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:46112/0 (socket says 172.21.15.45:46112) 2024-04-16T17:15:14.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.826+0000 7fe31f400640 1 -- 172.21.15.45:0/2172004478 learned_addr learned my addr 172.21.15.45:0/2172004478 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:15:14.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.826+0000 7fe31ea00640 1 --2- 172.21.15.45:0/2172004478 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe320109580 0x7fe3201a0d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:14.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.826+0000 7fe31fe00640 1 --2- 172.21.15.45:0/2172004478 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe32019a990 0x7fe32019ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:14.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.826+0000 7fe31f400640 1 -- 172.21.15.45:0/2172004478 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe32019a990 msgr2=0x7fe32019ae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:15:14.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.826+0000 7fe31f400640 1 --2- 172.21.15.45:0/2172004478 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe32019a990 0x7fe32019ae40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:14.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.826+0000 7fe31f400640 1 -- 172.21.15.45:0/2172004478 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe320109580 msgr2=0x7fe3201a0d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:15:14.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.826+0000 7fe31f400640 1 --2- 172.21.15.45:0/2172004478 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe320109580 0x7fe3201a0d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:14.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.826+0000 7fe31f400640 1 -- 172.21.15.45:0/2172004478 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe32019b4b0 con 0x7fe3201044b0 2024-04-16T17:15:14.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.826+0000 7fe31f400640 1 --2- 172.21.15.45:0/2172004478 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe3201044b0 0x7fe3201a07e0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fe31000e9f0 tx=0x7fe31000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:15:14.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.826+0000 7fe317e00640 1 -- 172.21.15.45:0/2172004478 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe31000cd80 con 0x7fe3201044b0 2024-04-16T17:15:14.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.827+0000 7fe326000640 1 -- 172.21.15.45:0/2172004478 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe32019b7a0 con 0x7fe3201044b0 2024-04-16T17:15:14.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.827+0000 7fe317e00640 1 -- 172.21.15.45:0/2172004478 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe310010910 con 0x7fe3201044b0 2024-04-16T17:15:14.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.827+0000 7fe317e00640 1 -- 172.21.15.45:0/2172004478 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe310018990 con 0x7fe3201044b0 2024-04-16T17:15:14.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.827+0000 7fe326000640 1 -- 172.21.15.45:0/2172004478 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3201a79f0 con 0x7fe3201044b0 2024-04-16T17:15:14.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.830+0000 7fe328d9c740 1 Processor -- start 2024-04-16T17:15:14.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.830+0000 7fe317e00640 1 -- 172.21.15.45:0/2172004478 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fe310010ac0 con 0x7fe3201044b0 2024-04-16T17:15:14.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.830+0000 7fe328d9c740 1 -- start start 2024-04-16T17:15:14.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.830+0000 7fe328d9c740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556316f1dd50 0x556316f1e130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:14.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.830+0000 7fe328d9c740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556316f1e670 0x556316f26d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:14.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.830+0000 7fe328d9c740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556316f27250 0x556316f29640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:14.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.830+0000 7fe328d9c740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x556316e83020 con 0x556316f1dd50 2024-04-16T17:15:14.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.830+0000 7fe328d9c740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x556316e29460 con 0x556316f27250 2024-04-16T17:15:14.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.830+0000 7fe328d9c740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x556316e7efe0 con 0x556316f1e670 2024-04-16T17:15:14.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.831+0000 7fe31fe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556316f1dd50 0x556316f1e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:14.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.831+0000 7fe31fe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556316f1dd50 0x556316f1e130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:46122/0 (socket says 172.21.15.45:46122) 2024-04-16T17:15:14.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.831+0000 7fe31fe00640 1 -- 172.21.15.45:0/3021520256 learned_addr learned my addr 172.21.15.45:0/3021520256 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:15:14.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.831+0000 7fe31f400640 1 --2- 172.21.15.45:0/3021520256 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556316f1e670 0x556316f26d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:14.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.831+0000 7fe31ea00640 1 --2- 172.21.15.45:0/3021520256 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556316f27250 0x556316f29640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:14.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.831+0000 7fe31fe00640 1 -- 172.21.15.45:0/3021520256 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556316f1e670 msgr2=0x556316f26d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:15:14.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.831+0000 7fe31fe00640 1 --2- 172.21.15.45:0/3021520256 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556316f1e670 0x556316f26d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:14.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.831+0000 7fe31fe00640 1 -- 172.21.15.45:0/3021520256 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556316f27250 msgr2=0x556316f29640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:15:14.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.831+0000 7fe31fe00640 1 --2- 172.21.15.45:0/3021520256 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556316f27250 0x556316f29640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:14.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.831+0000 7fe31fe00640 1 -- 172.21.15.45:0/3021520256 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556316e31040 con 0x556316f1dd50 2024-04-16T17:15:14.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.831+0000 7fe31f400640 1 --2- 172.21.15.45:0/3021520256 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556316f1e670 0x556316f26d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:15:14.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.831+0000 7fe317e00640 1 --2- 172.21.15.45:0/2172004478 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fe3040a3d20 0x7fe3040a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:14.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.832+0000 7fe31fe00640 1 --2- 172.21.15.45:0/3021520256 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556316f1dd50 0x556316f1e130 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fe318009f40 tx=0x7fe318010860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98c8bb6b61f097d6 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:15:14.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.832+0000 7fe317e00640 1 -- 172.21.15.45:0/2172004478 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9496+0+0 (secure 0 0 0) 0x7fe3100d0fe0 con 0x7fe3201044b0 2024-04-16T17:15:14.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.832+0000 7fe30be00640 1 -- 172.21.15.45:0/3021520256 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe318003b70 con 0x556316f1dd50 2024-04-16T17:15:14.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.832+0000 7fe31f400640 1 --2- 172.21.15.45:0/2172004478 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fe3040a3d20 0x7fe3040a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:14.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.832+0000 7fe30be00640 1 -- 172.21.15.45:0/3021520256 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe318003d10 con 0x556316f1dd50 2024-04-16T17:15:14.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.832+0000 7fe30be00640 1 -- 172.21.15.45:0/3021520256 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe318011ae0 con 0x556316f1dd50 2024-04-16T17:15:14.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.833+0000 7fe328d9c740 1 -- 172.21.15.45:0/3021520256 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556316f1dd50 msgr2=0x556316f1e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:15:14.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.833+0000 7fe31f400640 1 --2- 172.21.15.45:0/2172004478 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fe3040a3d20 0x7fe3040a61e0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x556316f1d820 tx=0x7fe310018420 comp rx=0 tx=0).ready entity=mgr.25411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:15:14.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.833+0000 7fe328d9c740 1 --2- 172.21.15.45:0/3021520256 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556316f1dd50 0x556316f1e130 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fe318009f40 tx=0x7fe318010860 comp rx=0 tx=0).stop 2024-04-16T17:15:14.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.833+0000 7fe328d9c740 1 -- 172.21.15.45:0/3021520256 shutdown_connections 2024-04-16T17:15:14.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.833+0000 7fe328d9c740 1 --2- 172.21.15.45:0/3021520256 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556316f1e670 0x556316f26d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:14.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.833+0000 7fe328d9c740 1 --2- 172.21.15.45:0/3021520256 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556316f27250 0x556316f29640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:14.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.833+0000 7fe328d9c740 1 --2- 172.21.15.45:0/3021520256 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556316f1dd50 0x556316f1e130 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:14.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.833+0000 7fe328d9c740 1 -- 172.21.15.45:0/3021520256 >> 172.21.15.45:0/3021520256 conn(0x556316f18480 msgr2=0x556316f18860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:15:14.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.833+0000 7fe328d9c740 1 -- 172.21.15.45:0/3021520256 shutdown_connections 2024-04-16T17:15:14.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.833+0000 7fe328d9c740 1 -- 172.21.15.45:0/3021520256 wait complete. 2024-04-16T17:15:14.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.834+0000 7fe328d9c740 1 -- start start 2024-04-16T17:15:14.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.834+0000 7fe328d9c740 1 Processor -- start 2024-04-16T17:15:14.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.834+0000 7fe328d9c740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556316f1dd50 0x556316f242f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:14.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.834+0000 7fe328d9c740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556316f1e670 0x556316f24830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:14.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.834+0000 7fe328d9c740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556316f27250 0x556316f24d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:14.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.834+0000 7fe328d9c740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x556316f2a040 con 0x556316f27250 2024-04-16T17:15:14.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.834+0000 7fe328d9c740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x556316e3c830 con 0x556316f1dd50 2024-04-16T17:15:14.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.834+0000 7fe328d9c740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x556316f2a1c0 con 0x556316f1e670 2024-04-16T17:15:14.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.834+0000 7fe328d9c740 10 client.0 _ll_register_callbacks cb 0x556316e44150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:15:14.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.835+0000 7fe31ea00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556316f1e670 0x556316f24830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:14.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.835+0000 7fe31fe00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556316f1dd50 0x556316f242f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:14.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.835+0000 7fe31ea00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556316f1e670 0x556316f24830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:38178/0 (socket says 172.21.15.45:38178) 2024-04-16T17:15:14.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.835+0000 7fe31ea00640 1 -- 172.21.15.45:0/3961376459 learned_addr learned my addr 172.21.15.45:0/3961376459 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:15:14.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.835+0000 7fe31fe00640 1 --2- 172.21.15.45:0/3961376459 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556316f27250 0x556316f24d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:14.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.835+0000 7fe31ea00640 1 -- 172.21.15.45:0/3961376459 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556316f1dd50 msgr2=0x556316f242f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:15:14.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.835+0000 7fe31ea00640 1 --2- 172.21.15.45:0/3961376459 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556316f1dd50 0x556316f242f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:14.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.835+0000 7fe31ea00640 1 -- 172.21.15.45:0/3961376459 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556316f27250 msgr2=0x556316f24d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:15:14.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.835+0000 7fe31ea00640 1 --2- 172.21.15.45:0/3961376459 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556316f27250 0x556316f24d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:14.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.835+0000 7fe31ea00640 1 -- 172.21.15.45:0/3961376459 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556316e40920 con 0x556316f1e670 2024-04-16T17:15:14.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.835+0000 7fe31fe00640 1 --2- 172.21.15.45:0/3961376459 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556316f1dd50 0x556316f242f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:15:14.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.835+0000 7fe31ea00640 1 --2- 172.21.15.45:0/3961376459 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556316f1e670 0x556316f24830 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fe32006b750 tx=0x7fe30c02ce30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2f2a4a27462c5999 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:15:14.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.835+0000 7fe308c00640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-04-16T17:15:14.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.836+0000 7fe308c00640 1 -- 172.21.15.45:0/3961376459 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe30c004240 con 0x556316f1e670 2024-04-16T17:15:14.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.836+0000 7fe328d9c740 1 -- 172.21.15.45:0/3961376459 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe318003d10 con 0x556316f1e670 2024-04-16T17:15:14.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.836+0000 7fe308c00640 1 -- 172.21.15.45:0/3961376459 <== mon.1 v2:172.21.15.117:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9496+0+0 (secure 0 0 0) 0x7fe30c0043e0 con 0x556316f1e670 2024-04-16T17:15:14.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.836+0000 7fe308c00640 1 -- 172.21.15.45:0/3961376459 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fe30c0046c0 con 0x556316f1e670 2024-04-16T17:15:14.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.836+0000 7fe328d9c740 10 client.27820 fetch_fsmap learned FSMap version 43 2024-04-16T17:15:14.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.836+0000 7fe328d9c740 1 -- 172.21.15.45:0/3961376459 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x556316e0e7f0 con 0x556316f1e670 2024-04-16T17:15:14.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.836+0000 7fe308c00640 1 -- 172.21.15.45:0/3961376459 <== mon.1 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe318003d10 con 0x556316f1e670 2024-04-16T17:15:14.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.837+0000 7fe308c00640 1 -- 172.21.15.45:0/3961376459 <== mon.1 v2:172.21.15.117:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7fe30c0035f0 con 0x556316f1e670 2024-04-16T17:15:14.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.837+0000 7fe328d9c740 10 client.27820 fetch_fsmap finished waiting for FSMap version 43 2024-04-16T17:15:14.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.837+0000 7fe328d9c740 10 client.27820 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:15:14.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.837+0000 7fe328d9c740 20 client.27820 populate_metadata read hostname 'smithi045' 2024-04-16T17:15:14.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.837+0000 7fe328d9c740 1 --2- 172.21.15.45:0/3961376459 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x556316f37620 0x556316f39a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:14.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.837+0000 7fe328d9c740 4 client.27820 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:15:14.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.837+0000 7fe328d9c740 1 -- 172.21.15.45:0/3961376459 --> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556316e90a40 con 0x556316f37620 2024-04-16T17:15:14.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.837+0000 7fe31fe00640 1 --2- 172.21.15.45:0/3961376459 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x556316f37620 0x556316f39a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:14.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.837+0000 7fe31fe00640 1 --2- 172.21.15.45:0/3961376459 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x556316f37620 0x556316f39a60 crc :-1 s=READY pgs=227 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=367d0c443d6d547e server_cookie=ea20b4a469e82758 in_seq=0 out_seq=0 2024-04-16T17:15:14.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.837+0000 7fe308c00640 10 client.27820 ms_handle_connect on v2:172.21.15.104:6838/3660070592 2024-04-16T17:15:14.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.838+0000 7fe308c00640 1 -- 172.21.15.45:0/3961376459 <== mds.0 v2:172.21.15.104:6838/3660070592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556316e90a40 con 0x556316f37620 2024-04-16T17:15:14.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.838+0000 7fe308c00640 10 client.27820 handle_command_reply: tid=0 2024-04-16T17:15:14.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.838+0000 7fe328d9c740 1 client.27820 shutdown 2024-04-16T17:15:14.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.838+0000 7fe328d9c740 1 -- 172.21.15.45:0/3961376459 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556316f1e670 msgr2=0x556316f24830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:15:14.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.838+0000 7fe328d9c740 1 --2- 172.21.15.45:0/3961376459 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556316f1e670 0x556316f24830 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fe32006b750 tx=0x7fe30c02ce30 comp rx=0 tx=0).stop 2024-04-16T17:15:14.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.839+0000 7fe328d9c740 1 -- 172.21.15.45:0/3961376459 shutdown_connections 2024-04-16T17:15:14.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.839+0000 7fe328d9c740 1 --2- 172.21.15.45:0/3961376459 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x556316f37620 0x556316f39a60 crc :-1 s=READY pgs=227 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:14.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.839+0000 7fe328d9c740 1 --2- 172.21.15.45:0/3961376459 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556316f1e670 0x556316f24830 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:14.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.839+0000 7fe328d9c740 1 --2- 172.21.15.45:0/3961376459 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556316f27250 0x556316f24d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:14.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.839+0000 7fe328d9c740 1 --2- 172.21.15.45:0/3961376459 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556316f1dd50 0x556316f242f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:14.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.839+0000 7fe308c00640 0 client.27820 ms_handle_reset on v2:172.21.15.104:6838/3660070592 2024-04-16T17:15:14.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.839+0000 7fe328d9c740 1 -- 172.21.15.45:0/3961376459 >> 172.21.15.45:0/3961376459 conn(0x556316f18480 msgr2=0x556316f1a1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:15:14.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.839+0000 7fe328d9c740 1 -- 172.21.15.45:0/3961376459 shutdown_connections 2024-04-16T17:15:14.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.839+0000 7fe328d9c740 1 -- 172.21.15.45:0/3961376459 wait complete. 2024-04-16T17:15:14.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.839+0000 7fe328d9c740 20 client.27820 trim_cache size 0 max 16384 2024-04-16T17:15:14.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.853+0000 7fe328d9c740 1 Processor -- start 2024-04-16T17:15:14.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.853+0000 7fe328d9c740 1 -- start start 2024-04-16T17:15:14.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.854+0000 7fe328d9c740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556316f1dd50 0x556316f32f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:14.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.854+0000 7fe328d9c740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556316f1e670 0x556316f24b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:14.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.854+0000 7fe328d9c740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556316f27250 0x556316f17360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:14.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.854+0000 7fe328d9c740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fe3201b40f0 con 0x556316f1e670 2024-04-16T17:15:14.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.854+0000 7fe328d9c740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fe3201b19c0 con 0x556316f1dd50 2024-04-16T17:15:14.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.854+0000 7fe328d9c740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fe3201af830 con 0x556316f27250 2024-04-16T17:15:14.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.854+0000 7fe31ea00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556316f1e670 0x556316f24b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:14.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.854+0000 7fe31ea00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556316f1e670 0x556316f24b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:46138/0 (socket says 172.21.15.45:46138) 2024-04-16T17:15:14.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.854+0000 7fe31ea00640 1 -- 172.21.15.45:0/750067888 learned_addr learned my addr 172.21.15.45:0/750067888 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:15:14.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.854+0000 7fe31fe00640 1 --2- 172.21.15.45:0/750067888 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556316f1dd50 0x556316f32f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:14.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.854+0000 7fe31fe00640 1 --2- 172.21.15.45:0/750067888 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556316f27250 0x556316f17360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:14.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.854+0000 7fe31ea00640 1 -- 172.21.15.45:0/750067888 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556316f27250 msgr2=0x556316f17360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:15:14.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.854+0000 7fe31ea00640 1 --2- 172.21.15.45:0/750067888 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556316f27250 0x556316f17360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:14.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.854+0000 7fe31ea00640 1 -- 172.21.15.45:0/750067888 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556316f1dd50 msgr2=0x556316f32f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:15:14.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.854+0000 7fe31ea00640 1 --2- 172.21.15.45:0/750067888 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556316f1dd50 0x556316f32f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:14.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.854+0000 7fe31ea00640 1 -- 172.21.15.45:0/750067888 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556316b953f0 con 0x556316f1e670 2024-04-16T17:15:14.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.855+0000 7fe31ea00640 1 --2- 172.21.15.45:0/750067888 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556316f1e670 0x556316f24b90 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fe30c02c170 tx=0x7fe30c02bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9d04fd30f137be7 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:15:14.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.855+0000 7fe30b400640 1 -- 172.21.15.45:0/750067888 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe30c009520 con 0x556316f1e670 2024-04-16T17:15:14.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.855+0000 7fe30b400640 1 -- 172.21.15.45:0/750067888 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe30c05b070 con 0x556316f1e670 2024-04-16T17:15:14.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.855+0000 7fe30b400640 1 -- 172.21.15.45:0/750067888 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe30c034840 con 0x556316f1e670 2024-04-16T17:15:14.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.856+0000 7fe328d9c740 1 -- 172.21.15.45:0/750067888 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556316f1e670 msgr2=0x556316f24b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:15:14.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.856+0000 7fe328d9c740 1 --2- 172.21.15.45:0/750067888 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556316f1e670 0x556316f24b90 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fe30c02c170 tx=0x7fe30c02bc30 comp rx=0 tx=0).stop 2024-04-16T17:15:14.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.856+0000 7fe328d9c740 1 -- 172.21.15.45:0/750067888 shutdown_connections 2024-04-16T17:15:14.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.856+0000 7fe328d9c740 1 --2- 172.21.15.45:0/750067888 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556316f27250 0x556316f17360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:14.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.856+0000 7fe328d9c740 1 --2- 172.21.15.45:0/750067888 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556316f1e670 0x556316f24b90 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:14.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.856+0000 7fe328d9c740 1 --2- 172.21.15.45:0/750067888 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556316f1dd50 0x556316f32f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:14.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.856+0000 7fe328d9c740 1 -- 172.21.15.45:0/750067888 >> 172.21.15.45:0/750067888 conn(0x556316f18480 msgr2=0x556316f16c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:15:14.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.856+0000 7fe328d9c740 1 -- 172.21.15.45:0/750067888 shutdown_connections 2024-04-16T17:15:14.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.856+0000 7fe328d9c740 1 -- 172.21.15.45:0/750067888 wait complete. 2024-04-16T17:15:14.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.856+0000 7fe328d9c740 1 -- start start 2024-04-16T17:15:14.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.856+0000 7fe328d9c740 1 Processor -- start 2024-04-16T17:15:14.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.857+0000 7fe328d9c740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556316f1dd50 0x556316f2af40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:14.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.857+0000 7fe328d9c740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556316f1e670 0x556316f2b480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:14.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.857+0000 7fe328d9c740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556316f27250 0x556316f2b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:14.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.857+0000 7fe328d9c740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fe3201afe80 con 0x556316f1dd50 2024-04-16T17:15:14.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.857+0000 7fe328d9c740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fe3201b12f0 con 0x556316f27250 2024-04-16T17:15:14.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.857+0000 7fe328d9c740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fe3201b0550 con 0x556316f1e670 2024-04-16T17:15:14.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.857+0000 7fe328d9c740 10 client.0 _ll_register_callbacks cb 0x7fe30c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:15:14.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.857+0000 7fe31ea00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556316f1e670 0x556316f2b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:14.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.857+0000 7fe31ea00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556316f1e670 0x556316f2b480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:38198/0 (socket says 172.21.15.45:38198) 2024-04-16T17:15:14.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.857+0000 7fe31fe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556316f1dd50 0x556316f2af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:14.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.857+0000 7fe31ea00640 1 -- 172.21.15.45:0/3212218673 learned_addr learned my addr 172.21.15.45:0/3212218673 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:15:14.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.858+0000 7fe31ea00640 1 -- 172.21.15.45:0/3212218673 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556316f27250 msgr2=0x556316f2b9c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-16T17:15:14.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.858+0000 7fe31ea00640 1 --2- 172.21.15.45:0/3212218673 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556316f27250 0x556316f2b9c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:14.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.858+0000 7fe31ea00640 1 -- 172.21.15.45:0/3212218673 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556316f1dd50 msgr2=0x556316f2af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:15:14.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.858+0000 7fe31ea00640 1 --2- 172.21.15.45:0/3212218673 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556316f1dd50 0x556316f2af40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:14.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.858+0000 7fe31ea00640 1 -- 172.21.15.45:0/3212218673 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556316c5f7d0 con 0x556316f1e670 2024-04-16T17:15:14.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.858+0000 7fe31fe00640 1 --2- 172.21.15.45:0/3212218673 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556316f1dd50 0x556316f2af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:15:14.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.858+0000 7fe31ea00640 1 --2- 172.21.15.45:0/3212218673 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556316f1e670 0x556316f2b480 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fe30c058040 tx=0x7fe30c003820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bd7d8c4105129d59 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:15:14.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.858+0000 7fe308c00640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-04-16T17:15:14.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.858+0000 7fe308c00640 1 -- 172.21.15.45:0/3212218673 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe30c063070 con 0x556316f1e670 2024-04-16T17:15:14.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.858+0000 7fe308c00640 1 -- 172.21.15.45:0/3212218673 <== mon.1 v2:172.21.15.117:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9496+0+0 (secure 0 0 0) 0x7fe30c004090 con 0x556316f1e670 2024-04-16T17:15:14.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.858+0000 7fe328d9c740 1 -- 172.21.15.45:0/3212218673 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe30c05b070 con 0x556316f1e670 2024-04-16T17:15:14.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.859+0000 7fe308c00640 1 -- 172.21.15.45:0/3212218673 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fe30c06c050 con 0x556316f1e670 2024-04-16T17:15:14.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.859+0000 7fe328d9c740 10 client.27826 fetch_fsmap learned FSMap version 43 2024-04-16T17:15:14.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.859+0000 7fe328d9c740 1 -- 172.21.15.45:0/3212218673 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x556316f9ddd0 con 0x556316f1e670 2024-04-16T17:15:14.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.859+0000 7fe308c00640 1 -- 172.21.15.45:0/3212218673 <== mon.1 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe30c05b070 con 0x556316f1e670 2024-04-16T17:15:14.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.859+0000 7fe308c00640 1 -- 172.21.15.45:0/3212218673 <== mon.1 v2:172.21.15.117:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7fe30c066020 con 0x556316f1e670 2024-04-16T17:15:14.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.859+0000 7fe328d9c740 10 client.27826 fetch_fsmap finished waiting for FSMap version 43 2024-04-16T17:15:14.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.859+0000 7fe328d9c740 10 client.27826 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:15:14.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.859+0000 7fe328d9c740 20 client.27826 populate_metadata read hostname 'smithi045' 2024-04-16T17:15:14.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.859+0000 7fe328d9c740 1 --2- 172.21.15.45:0/3212218673 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x556316f9e310 0x556316fa0750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:14.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.859+0000 7fe328d9c740 4 client.27826 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T17:15:14.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.859+0000 7fe328d9c740 1 -- 172.21.15.45:0/3212218673 --> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556316e8a8b0 con 0x556316f9e310 2024-04-16T17:15:14.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.860+0000 7fe31fe00640 1 --2- 172.21.15.45:0/3212218673 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x556316f9e310 0x556316fa0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:14.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.860+0000 7fe308c00640 10 client.27826 ms_handle_connect on v2:172.21.15.104:6838/3660070592 2024-04-16T17:15:14.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.860+0000 7fe31fe00640 1 --2- 172.21.15.45:0/3212218673 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x556316f9e310 0x556316fa0750 crc :-1 s=READY pgs=228 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b5c1da573cc6ab6c server_cookie=4589da83a4abc270 in_seq=0 out_seq=0 2024-04-16T17:15:14.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.860+0000 7fe308c00640 1 -- 172.21.15.45:0/3212218673 <== mds.0 v2:172.21.15.104:6838/3660070592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556316e8a8b0 con 0x556316f9e310 2024-04-16T17:15:14.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.860+0000 7fe308c00640 10 client.27826 handle_command_reply: tid=0 2024-04-16T17:15:14.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.860+0000 7fe328d9c740 1 client.27826 shutdown 2024-04-16T17:15:14.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.861+0000 7fe328d9c740 1 -- 172.21.15.45:0/3212218673 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556316f1e670 msgr2=0x556316f2b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:15:14.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.861+0000 7fe328d9c740 1 --2- 172.21.15.45:0/3212218673 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556316f1e670 0x556316f2b480 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fe30c058040 tx=0x7fe30c003820 comp rx=0 tx=0).stop 2024-04-16T17:15:14.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.861+0000 7fe328d9c740 1 -- 172.21.15.45:0/3212218673 shutdown_connections 2024-04-16T17:15:14.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.861+0000 7fe328d9c740 1 --2- 172.21.15.45:0/3212218673 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x556316f9e310 0x556316fa0750 crc :-1 s=READY pgs=228 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:14.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.861+0000 7fe328d9c740 1 --2- 172.21.15.45:0/3212218673 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556316f1e670 0x556316f2b480 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:14.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.861+0000 7fe328d9c740 1 --2- 172.21.15.45:0/3212218673 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556316f27250 0x556316f2b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:14.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.861+0000 7fe308c00640 0 client.27826 ms_handle_reset on v2:172.21.15.104:6838/3660070592 2024-04-16T17:15:14.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.861+0000 7fe328d9c740 1 --2- 172.21.15.45:0/3212218673 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556316f1dd50 0x556316f2af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:14.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.861+0000 7fe328d9c740 1 -- 172.21.15.45:0/3212218673 >> 172.21.15.45:0/3212218673 conn(0x556316f18480 msgr2=0x556316f174e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:15:14.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.861+0000 7fe328d9c740 1 -- 172.21.15.45:0/3212218673 shutdown_connections 2024-04-16T17:15:14.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.861+0000 7fe328d9c740 1 -- 172.21.15.45:0/3212218673 wait complete. 2024-04-16T17:15:14.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.861+0000 7fe328d9c740 20 client.27826 trim_cache size 0 max 16384 2024-04-16T17:15:14.868 INFO:teuthology.orchestra.run.smithi045.stdout:[] 2024-04-16T17:15:14.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.862+0000 7fe326000640 1 -- 172.21.15.45:0/2172004478 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fe3040a3d20 msgr2=0x7fe3040a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:15:14.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.862+0000 7fe326000640 1 --2- 172.21.15.45:0/2172004478 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fe3040a3d20 0x7fe3040a61e0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x556316f1d820 tx=0x7fe310018420 comp rx=0 tx=0).stop 2024-04-16T17:15:14.869 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.862+0000 7fe326000640 1 -- 172.21.15.45:0/2172004478 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe3201044b0 msgr2=0x7fe3201a07e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:15:14.869 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.862+0000 7fe326000640 1 --2- 172.21.15.45:0/2172004478 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe3201044b0 0x7fe3201a07e0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fe31000e9f0 tx=0x7fe31000eeb0 comp rx=0 tx=0).stop 2024-04-16T17:15:14.869 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.862+0000 7fe326000640 1 -- 172.21.15.45:0/2172004478 shutdown_connections 2024-04-16T17:15:14.869 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.862+0000 7fe326000640 1 --2- 172.21.15.45:0/2172004478 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fe3040a3d20 0x7fe3040a61e0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:14.869 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.862+0000 7fe326000640 1 --2- 172.21.15.45:0/2172004478 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe32019a990 0x7fe32019ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:14.869 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.862+0000 7fe326000640 1 --2- 172.21.15.45:0/2172004478 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe320109580 0x7fe3201a0d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:14.869 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.862+0000 7fe326000640 1 --2- 172.21.15.45:0/2172004478 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe3201044b0 0x7fe3201a07e0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:14.869 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.862+0000 7fe326000640 1 -- 172.21.15.45:0/2172004478 >> 172.21.15.45:0/2172004478 conn(0x7fe3200ffd90 msgr2=0x7fe3201020f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:15:14.869 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.862+0000 7fe326000640 1 -- 172.21.15.45:0/2172004478 shutdown_connections 2024-04-16T17:15:14.870 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:14.862+0000 7fe326000640 1 -- 172.21.15.45:0/2172004478 wait complete. 2024-04-16T17:15:14.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:15:14 smithi104 ceph-mon[36457]: pgmap v263: 129 pgs: 129 active+clean; 59 MiB data, 876 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 18 KiB/s wr, 8 op/s 2024-04-16T17:15:14.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:15:14 smithi104 ceph-mon[36457]: from='client.? 172.21.15.45:0/3400439147' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:15:14.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:15:14 smithi104 ceph-mon[36457]: osdmap e102: 12 total, 12 up, 12 in 2024-04-16T17:15:15.054 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:15:14 smithi117 ceph-mon[36730]: pgmap v263: 129 pgs: 129 active+clean; 59 MiB data, 876 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 18 KiB/s wr, 8 op/s 2024-04-16T17:15:15.054 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:15:14 smithi117 ceph-mon[36730]: from='client.? 172.21.15.45:0/3400439147' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:15:15.055 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:15:14 smithi117 ceph-mon[36730]: osdmap e102: 12 total, 12 up, 12 in 2024-04-16T17:15:15.880 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-16T17:15:15.880 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T17:15:15.880 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-16T17:15:16.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.104+0000 7fe52b000640 1 -- 172.21.15.45:0/269193971 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe524102230 msgr2=0x7fe524102610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:15:16.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.104+0000 7fe52b000640 1 --2- 172.21.15.45:0/269193971 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe524102230 0x7fe524102610 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fe514009900 tx=0x7fe51402bc30 comp rx=0 tx=0).stop 2024-04-16T17:15:16.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.104+0000 7fe52b000640 1 -- 172.21.15.45:0/269193971 shutdown_connections 2024-04-16T17:15:16.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.104+0000 7fe52b000640 1 --2- 172.21.15.45:0/269193971 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe524107300 0x7fe52410eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:16.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.104+0000 7fe52b000640 1 --2- 172.21.15.45:0/269193971 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe524102be0 0x7fe524106c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:16.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.104+0000 7fe52b000640 1 --2- 172.21.15.45:0/269193971 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe524102230 0x7fe524102610 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:16.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.104+0000 7fe52b000640 1 -- 172.21.15.45:0/269193971 >> 172.21.15.45:0/269193971 conn(0x7fe5240fda70 msgr2=0x7fe5240ffe90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:15:16.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.104+0000 7fe52b000640 1 -- 172.21.15.45:0/269193971 shutdown_connections 2024-04-16T17:15:16.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.104+0000 7fe52b000640 1 -- 172.21.15.45:0/269193971 wait complete. 2024-04-16T17:15:16.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.105+0000 7fe52b000640 1 Processor -- start 2024-04-16T17:15:16.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.105+0000 7fe52b000640 1 -- start start 2024-04-16T17:15:16.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.106+0000 7fe52b000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe524102230 0x7fe524074300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:16.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.106+0000 7fe52b000640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe524107300 0x7fe524074840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:16.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.106+0000 7fe52b000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe524074d80 0x7fe5241a6ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:16.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.106+0000 7fe52b000640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fe524112410 con 0x7fe524074d80 2024-04-16T17:15:16.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.106+0000 7fe52b000640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fe524112290 con 0x7fe524107300 2024-04-16T17:15:16.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.106+0000 7fe52b000640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fe524112590 con 0x7fe524102230 2024-04-16T17:15:16.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.106+0000 7fe529000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe524074d80 0x7fe5241a6ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:16.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.106+0000 7fe529000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe524074d80 0x7fe5241a6ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:46170/0 (socket says 172.21.15.45:46170) 2024-04-16T17:15:16.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.106+0000 7fe523e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe524102230 0x7fe524074300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:16.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.106+0000 7fe523400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe524107300 0x7fe524074840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:16.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.106+0000 7fe529000640 1 -- 172.21.15.45:0/3461616773 learned_addr learned my addr 172.21.15.45:0/3461616773 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:15:16.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.107+0000 7fe523400640 1 -- 172.21.15.45:0/3461616773 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe524102230 msgr2=0x7fe524074300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:15:16.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.107+0000 7fe523400640 1 --2- 172.21.15.45:0/3461616773 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe524102230 0x7fe524074300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:16.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.107+0000 7fe523400640 1 -- 172.21.15.45:0/3461616773 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe524074d80 msgr2=0x7fe5241a6ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:15:16.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.107+0000 7fe523400640 1 --2- 172.21.15.45:0/3461616773 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe524074d80 0x7fe5241a6ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:16.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.107+0000 7fe523400640 1 -- 172.21.15.45:0/3461616773 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5241a7160 con 0x7fe524107300 2024-04-16T17:15:16.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.107+0000 7fe529000640 1 --2- 172.21.15.45:0/3461616773 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe524074d80 0x7fe5241a6ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:15:16.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.107+0000 7fe523400640 1 --2- 172.21.15.45:0/3461616773 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe524107300 0x7fe524074840 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fe51000e9c0 tx=0x7fe51000ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:15:16.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.107+0000 7fe520c00640 1 -- 172.21.15.45:0/3461616773 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe51000ccc0 con 0x7fe524107300 2024-04-16T17:15:16.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.107+0000 7fe52b000640 1 -- 172.21.15.45:0/3461616773 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5241a7450 con 0x7fe524107300 2024-04-16T17:15:16.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.107+0000 7fe520c00640 1 -- 172.21.15.45:0/3461616773 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe510010910 con 0x7fe524107300 2024-04-16T17:15:16.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.107+0000 7fe520c00640 1 -- 172.21.15.45:0/3461616773 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe510018880 con 0x7fe524107300 2024-04-16T17:15:16.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.107+0000 7fe52b000640 1 -- 172.21.15.45:0/3461616773 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5241a79f0 con 0x7fe524107300 2024-04-16T17:15:16.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.111+0000 7fe52dbf1740 1 Processor -- start 2024-04-16T17:15:16.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.112+0000 7fe52dbf1740 1 -- start start 2024-04-16T17:15:16.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.112+0000 7fe52dbf1740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5603242d4ca0 0x5603242d5080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:16.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.112+0000 7fe52dbf1740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5603242d55c0 0x5603242ddc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:16.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.112+0000 7fe52dbf1740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5603242de1a0 0x5603242e0590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:16.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.112+0000 7fe52dbf1740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x560324239cb0 con 0x5603242d4ca0 2024-04-16T17:15:16.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.112+0000 7fe52dbf1740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5603241e0370 con 0x5603242de1a0 2024-04-16T17:15:16.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.112+0000 7fe52dbf1740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x560324235c70 con 0x5603242d55c0 2024-04-16T17:15:16.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.112+0000 7fe529000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5603242d4ca0 0x5603242d5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:16.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.112+0000 7fe529000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5603242d4ca0 0x5603242d5080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:46182/0 (socket says 172.21.15.45:46182) 2024-04-16T17:15:16.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.112+0000 7fe529000640 1 -- 172.21.15.45:0/1049474980 learned_addr learned my addr 172.21.15.45:0/1049474980 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:15:16.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.112+0000 7fe523e00640 1 --2- 172.21.15.45:0/1049474980 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5603242d55c0 0x5603242ddc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:16.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.113+0000 7fe529000640 1 -- 172.21.15.45:0/1049474980 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5603242d55c0 msgr2=0x5603242ddc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:15:16.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.113+0000 7fe529000640 1 --2- 172.21.15.45:0/1049474980 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5603242d55c0 0x5603242ddc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:16.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.113+0000 7fe529000640 1 -- 172.21.15.45:0/1049474980 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5603242de1a0 msgr2=0x5603242e0590 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-16T17:15:16.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.113+0000 7fe529000640 1 --2- 172.21.15.45:0/1049474980 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5603242de1a0 0x5603242e0590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:16.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.113+0000 7fe529000640 1 -- 172.21.15.45:0/1049474980 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5603241ff910 con 0x5603242d4ca0 2024-04-16T17:15:16.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.113+0000 7fe520c00640 1 -- 172.21.15.45:0/3461616773 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fe510010ac0 con 0x7fe524107300 2024-04-16T17:15:16.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.113+0000 7fe529000640 1 --2- 172.21.15.45:0/1049474980 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5603242d4ca0 0x5603242d5080 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fe51c011850 tx=0x7fe51c011d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89d7c43c41cbc647 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:15:16.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.113+0000 7fe519600640 1 -- 172.21.15.45:0/1049474980 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe51c011330 con 0x5603242d4ca0 2024-04-16T17:15:16.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.113+0000 7fe519600640 1 -- 172.21.15.45:0/1049474980 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe51c00fe80 con 0x5603242d4ca0 2024-04-16T17:15:16.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.114+0000 7fe519600640 1 -- 172.21.15.45:0/1049474980 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe51c0193e0 con 0x5603242d4ca0 2024-04-16T17:15:16.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.114+0000 7fe52dbf1740 1 -- 172.21.15.45:0/1049474980 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5603242d4ca0 msgr2=0x5603242d5080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:15:16.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.114+0000 7fe52dbf1740 1 --2- 172.21.15.45:0/1049474980 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5603242d4ca0 0x5603242d5080 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fe51c011850 tx=0x7fe51c011d10 comp rx=0 tx=0).stop 2024-04-16T17:15:16.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.114+0000 7fe52dbf1740 1 -- 172.21.15.45:0/1049474980 shutdown_connections 2024-04-16T17:15:16.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.114+0000 7fe52dbf1740 1 --2- 172.21.15.45:0/1049474980 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5603242d55c0 0x5603242ddc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:16.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.114+0000 7fe52dbf1740 1 --2- 172.21.15.45:0/1049474980 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5603242de1a0 0x5603242e0590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:16.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.114+0000 7fe52dbf1740 1 --2- 172.21.15.45:0/1049474980 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5603242d4ca0 0x5603242d5080 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:16.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.114+0000 7fe52dbf1740 1 -- 172.21.15.45:0/1049474980 >> 172.21.15.45:0/1049474980 conn(0x5603242cf3d0 msgr2=0x5603242cf7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:15:16.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.114+0000 7fe52dbf1740 1 -- 172.21.15.45:0/1049474980 shutdown_connections 2024-04-16T17:15:16.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.114+0000 7fe52dbf1740 1 -- 172.21.15.45:0/1049474980 wait complete. 2024-04-16T17:15:16.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.114+0000 7fe520c00640 1 --2- 172.21.15.45:0/3461616773 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fe5080a3cb0 0x7fe5080a6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:16.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.115+0000 7fe520c00640 1 -- 172.21.15.45:0/3461616773 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9496+0+0 (secure 0 0 0) 0x7fe5100d0f30 con 0x7fe524107300 2024-04-16T17:15:16.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.115+0000 7fe52dbf1740 1 -- start start 2024-04-16T17:15:16.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.115+0000 7fe523e00640 1 --2- 172.21.15.45:0/3461616773 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fe5080a3cb0 0x7fe5080a6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:16.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.115+0000 7fe52dbf1740 1 Processor -- start 2024-04-16T17:15:16.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.116+0000 7fe523e00640 1 --2- 172.21.15.45:0/3461616773 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fe5080a3cb0 0x7fe5080a6170 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fe514000c00 tx=0x7fe51402df90 comp rx=0 tx=0).ready entity=mgr.25411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:15:16.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.116+0000 7fe52dbf1740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5603242d4ca0 0x5603242db0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:16.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.116+0000 7fe52dbf1740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5603242d55c0 0x5603242db5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:16.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.116+0000 7fe52dbf1740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5603242de1a0 0x5603242dbb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:16.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.116+0000 7fe52dbf1740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5603242e0f90 con 0x5603242d55c0 2024-04-16T17:15:16.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.116+0000 7fe52dbf1740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5603241f3730 con 0x5603242d4ca0 2024-04-16T17:15:16.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.116+0000 7fe52dbf1740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5603242e1110 con 0x5603242de1a0 2024-04-16T17:15:16.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.116+0000 7fe52dbf1740 10 client.0 _ll_register_callbacks cb 0x5603241c21e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:15:16.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.116+0000 7fe523400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5603242d55c0 0x5603242db5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:16.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.116+0000 7fe523400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5603242d55c0 0x5603242db5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:46192/0 (socket says 172.21.15.45:46192) 2024-04-16T17:15:16.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.116+0000 7fe523400640 1 -- 172.21.15.45:0/4052978991 learned_addr learned my addr 172.21.15.45:0/4052978991 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:15:16.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.116+0000 7fe529000640 1 --2- 172.21.15.45:0/4052978991 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5603242de1a0 0x5603242dbb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:16.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.116+0000 7fe523e00640 1 --2- 172.21.15.45:0/4052978991 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5603242d4ca0 0x5603242db0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:16.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.116+0000 7fe523400640 1 -- 172.21.15.45:0/4052978991 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5603242de1a0 msgr2=0x5603242dbb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:15:16.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.116+0000 7fe523400640 1 --2- 172.21.15.45:0/4052978991 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5603242de1a0 0x5603242dbb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:16.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.117+0000 7fe523400640 1 -- 172.21.15.45:0/4052978991 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5603242d4ca0 msgr2=0x5603242db0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:15:16.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.117+0000 7fe523400640 1 --2- 172.21.15.45:0/4052978991 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5603242d4ca0 0x5603242db0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:16.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.117+0000 7fe523400640 1 -- 172.21.15.45:0/4052978991 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5603240147e0 con 0x5603242d55c0 2024-04-16T17:15:16.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.117+0000 7fe523e00640 1 --2- 172.21.15.45:0/4052978991 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5603242d4ca0 0x5603242db0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:15:16.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.117+0000 7fe523400640 1 --2- 172.21.15.45:0/4052978991 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5603242d55c0 0x5603242db5f0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fe5100ce2b0 tx=0x7fe51009c780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2af30616c65df55b server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:15:16.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.117+0000 7fe50e000640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-16T17:15:16.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.117+0000 7fe50e000640 1 -- 172.21.15.45:0/4052978991 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5100a0e50 con 0x5603242d55c0 2024-04-16T17:15:16.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.117+0000 7fe50e000640 1 -- 172.21.15.45:0/4052978991 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9496+0+0 (secure 0 0 0) 0x7fe51009bfa0 con 0x5603242d55c0 2024-04-16T17:15:16.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.117+0000 7fe52dbf1740 1 -- 172.21.15.45:0/4052978991 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5603242476d0 con 0x5603242d55c0 2024-04-16T17:15:16.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.117+0000 7fe50e000640 1 -- 172.21.15.45:0/4052978991 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fe5100b6020 con 0x5603242d55c0 2024-04-16T17:15:16.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.117+0000 7fe52dbf1740 10 client.17736 fetch_fsmap learned FSMap version 43 2024-04-16T17:15:16.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.117+0000 7fe52dbf1740 1 -- 172.21.15.45:0/4052978991 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x560324074f70 con 0x5603242d55c0 2024-04-16T17:15:16.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.118+0000 7fe50e000640 1 -- 172.21.15.45:0/4052978991 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x5603242476d0 con 0x5603242d55c0 2024-04-16T17:15:16.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.118+0000 7fe50e000640 1 -- 172.21.15.45:0/4052978991 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7fe51009ba10 con 0x5603242d55c0 2024-04-16T17:15:16.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.118+0000 7fe52dbf1740 10 client.17736 fetch_fsmap finished waiting for FSMap version 43 2024-04-16T17:15:16.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.118+0000 7fe52dbf1740 10 client.17736 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:15:16.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.118+0000 7fe52dbf1740 20 client.17736 populate_metadata read hostname 'smithi045' 2024-04-16T17:15:16.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.118+0000 7fe52dbf1740 1 --2- 172.21.15.45:0/4052978991 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x5603242ee4a0 0x5603242f0890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:16.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.118+0000 7fe52dbf1740 4 client.17736 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:15:16.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.118+0000 7fe52dbf1740 1 -- 172.21.15.45:0/4052978991 --> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560324241540 con 0x5603242ee4a0 2024-04-16T17:15:16.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.118+0000 7fe523e00640 1 --2- 172.21.15.45:0/4052978991 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x5603242ee4a0 0x5603242f0890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:16.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.119+0000 7fe523e00640 1 --2- 172.21.15.45:0/4052978991 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x5603242ee4a0 0x5603242f0890 crc :-1 s=READY pgs=229 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d767357214323664 server_cookie=c6768de808e7a31 in_seq=0 out_seq=0 2024-04-16T17:15:16.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.119+0000 7fe50e000640 10 client.17736 ms_handle_connect on v2:172.21.15.104:6838/3660070592 2024-04-16T17:15:16.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.120+0000 7fe50e000640 1 -- 172.21.15.45:0/4052978991 <== mds.0 v2:172.21.15.104:6838/3660070592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560324241540 con 0x5603242ee4a0 2024-04-16T17:15:16.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.120+0000 7fe50e000640 10 client.17736 handle_command_reply: tid=0 2024-04-16T17:15:16.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.120+0000 7fe52dbf1740 1 client.17736 shutdown 2024-04-16T17:15:16.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.120+0000 7fe52dbf1740 1 -- 172.21.15.45:0/4052978991 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5603242d55c0 msgr2=0x5603242db5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:15:16.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.120+0000 7fe52dbf1740 1 --2- 172.21.15.45:0/4052978991 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5603242d55c0 0x5603242db5f0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fe5100ce2b0 tx=0x7fe51009c780 comp rx=0 tx=0).stop 2024-04-16T17:15:16.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.120+0000 7fe52dbf1740 1 -- 172.21.15.45:0/4052978991 shutdown_connections 2024-04-16T17:15:16.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.120+0000 7fe52dbf1740 1 --2- 172.21.15.45:0/4052978991 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x5603242ee4a0 0x5603242f0890 crc :-1 s=READY pgs=229 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:16.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.120+0000 7fe52dbf1740 1 --2- 172.21.15.45:0/4052978991 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5603242de1a0 0x5603242dbb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:16.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.120+0000 7fe52dbf1740 1 --2- 172.21.15.45:0/4052978991 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5603242d55c0 0x5603242db5f0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:16.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.120+0000 7fe52dbf1740 1 --2- 172.21.15.45:0/4052978991 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5603242d4ca0 0x5603242db0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:16.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.120+0000 7fe52dbf1740 1 -- 172.21.15.45:0/4052978991 >> 172.21.15.45:0/4052978991 conn(0x5603242cf3d0 msgr2=0x5603242d1120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:15:16.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.120+0000 7fe50e000640 0 client.17736 ms_handle_reset on v2:172.21.15.104:6838/3660070592 2024-04-16T17:15:16.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.120+0000 7fe52dbf1740 1 -- 172.21.15.45:0/4052978991 shutdown_connections 2024-04-16T17:15:16.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.120+0000 7fe52dbf1740 1 -- 172.21.15.45:0/4052978991 wait complete. 2024-04-16T17:15:16.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.120+0000 7fe52dbf1740 20 client.17736 trim_cache size 0 max 16384 2024-04-16T17:15:16.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.139+0000 7fe52dbf1740 1 Processor -- start 2024-04-16T17:15:16.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.140+0000 7fe52dbf1740 1 -- start start 2024-04-16T17:15:16.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.140+0000 7fe52dbf1740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5603242d4ca0 0x5603242dd7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:16.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.140+0000 7fe52dbf1740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5603242d55c0 0x5603242e9e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:16.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.140+0000 7fe52dbf1740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5603242de1a0 0x5603242d0ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:16.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.140+0000 7fe52dbf1740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fe5241b4060 con 0x5603242de1a0 2024-04-16T17:15:16.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.140+0000 7fe52dbf1740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fe5241b1930 con 0x5603242d4ca0 2024-04-16T17:15:16.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.140+0000 7fe52dbf1740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fe5241af7a0 con 0x5603242d55c0 2024-04-16T17:15:16.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.140+0000 7fe523400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5603242d55c0 0x5603242e9e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:16.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.140+0000 7fe523400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5603242d55c0 0x5603242e9e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:38244/0 (socket says 172.21.15.45:38244) 2024-04-16T17:15:16.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.140+0000 7fe523400640 1 -- 172.21.15.45:0/2694169685 learned_addr learned my addr 172.21.15.45:0/2694169685 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:15:16.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.140+0000 7fe529000640 1 --2- 172.21.15.45:0/2694169685 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5603242de1a0 0x5603242d0ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:16.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.140+0000 7fe523e00640 1 --2- 172.21.15.45:0/2694169685 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5603242d4ca0 0x5603242dd7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:16.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.141+0000 7fe523400640 1 -- 172.21.15.45:0/2694169685 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5603242d4ca0 msgr2=0x5603242dd7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:15:16.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.141+0000 7fe523400640 1 --2- 172.21.15.45:0/2694169685 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5603242d4ca0 0x5603242dd7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:16.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.141+0000 7fe523400640 1 -- 172.21.15.45:0/2694169685 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5603242de1a0 msgr2=0x5603242d0ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:15:16.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.141+0000 7fe523400640 1 --2- 172.21.15.45:0/2694169685 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5603242de1a0 0x5603242d0ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:16.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.141+0000 7fe523400640 1 -- 172.21.15.45:0/2694169685 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560323f0c170 con 0x5603242d55c0 2024-04-16T17:15:16.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.141+0000 7fe529000640 1 --2- 172.21.15.45:0/2694169685 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5603242de1a0 0x5603242d0ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:15:16.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.141+0000 7fe523400640 1 --2- 172.21.15.45:0/2694169685 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5603242d55c0 0x5603242e9e80 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fe510098fb0 tx=0x7fe51009b060 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1013b7e6bccef1a server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:15:16.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.141+0000 7fe518c00640 1 -- 172.21.15.45:0/2694169685 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5100991c0 con 0x5603242d55c0 2024-04-16T17:15:16.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.141+0000 7fe518c00640 1 -- 172.21.15.45:0/2694169685 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe5100a20b0 con 0x5603242d55c0 2024-04-16T17:15:16.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.141+0000 7fe518c00640 1 -- 172.21.15.45:0/2694169685 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe510098b00 con 0x5603242d55c0 2024-04-16T17:15:16.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.142+0000 7fe52dbf1740 1 -- 172.21.15.45:0/2694169685 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5603242d55c0 msgr2=0x5603242e9e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:15:16.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.142+0000 7fe52dbf1740 1 --2- 172.21.15.45:0/2694169685 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5603242d55c0 0x5603242e9e80 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fe510098fb0 tx=0x7fe51009b060 comp rx=0 tx=0).stop 2024-04-16T17:15:16.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.142+0000 7fe52dbf1740 1 -- 172.21.15.45:0/2694169685 shutdown_connections 2024-04-16T17:15:16.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.142+0000 7fe52dbf1740 1 --2- 172.21.15.45:0/2694169685 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5603242d55c0 0x5603242e9e80 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:16.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.142+0000 7fe52dbf1740 1 --2- 172.21.15.45:0/2694169685 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5603242de1a0 0x5603242d0ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:16.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.142+0000 7fe52dbf1740 1 --2- 172.21.15.45:0/2694169685 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5603242d4ca0 0x5603242dd7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:16.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.142+0000 7fe52dbf1740 1 -- 172.21.15.45:0/2694169685 >> 172.21.15.45:0/2694169685 conn(0x5603242cf3d0 msgr2=0x5603242d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:15:16.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.142+0000 7fe52dbf1740 1 -- 172.21.15.45:0/2694169685 shutdown_connections 2024-04-16T17:15:16.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.142+0000 7fe52dbf1740 1 -- 172.21.15.45:0/2694169685 wait complete. 2024-04-16T17:15:16.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.143+0000 7fe52dbf1740 1 -- start start 2024-04-16T17:15:16.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.143+0000 7fe52dbf1740 1 Processor -- start 2024-04-16T17:15:16.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.144+0000 7fe52dbf1740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5603242d4ca0 0x5603242e67f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:16.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.144+0000 7fe52dbf1740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5603242d55c0 0x5603242d7a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:16.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.144+0000 7fe52dbf1740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5603242de1a0 0x5603242d7fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:16.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.144+0000 7fe52dbf1740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fe5241afdf0 con 0x5603242d4ca0 2024-04-16T17:15:16.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.144+0000 7fe52dbf1740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fe5241b1260 con 0x5603242de1a0 2024-04-16T17:15:16.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.144+0000 7fe52dbf1740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fe5241b04c0 con 0x5603242d55c0 2024-04-16T17:15:16.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.144+0000 7fe52dbf1740 10 client.0 _ll_register_callbacks cb 0x7fe510098ea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:15:16.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.144+0000 7fe523400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5603242d55c0 0x5603242d7a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:16.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.144+0000 7fe523e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5603242d4ca0 0x5603242e67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:16.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.144+0000 7fe523400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5603242d55c0 0x5603242d7a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:38260/0 (socket says 172.21.15.45:38260) 2024-04-16T17:15:16.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.144+0000 7fe523400640 1 -- 172.21.15.45:0/1225907440 learned_addr learned my addr 172.21.15.45:0/1225907440 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:15:16.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.144+0000 7fe529000640 1 --2- 172.21.15.45:0/1225907440 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5603242de1a0 0x5603242d7fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:16.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.144+0000 7fe523400640 1 -- 172.21.15.45:0/1225907440 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5603242de1a0 msgr2=0x5603242d7fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:15:16.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.144+0000 7fe523400640 1 --2- 172.21.15.45:0/1225907440 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5603242de1a0 0x5603242d7fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:16.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.144+0000 7fe523400640 1 -- 172.21.15.45:0/1225907440 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5603242d4ca0 msgr2=0x5603242e67f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:15:16.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.144+0000 7fe523400640 1 --2- 172.21.15.45:0/1225907440 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5603242d4ca0 0x5603242e67f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:16.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.144+0000 7fe523400640 1 -- 172.21.15.45:0/1225907440 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5603242d8810 con 0x5603242d55c0 2024-04-16T17:15:16.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.144+0000 7fe529000640 1 --2- 172.21.15.45:0/1225907440 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5603242de1a0 0x5603242d7fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:15:16.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.145+0000 7fe523400640 1 --2- 172.21.15.45:0/1225907440 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5603242d55c0 0x5603242d7a80 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fe51000cf10 tx=0x7fe51009bfa0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=90cf86998ddfd4f4 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:15:16.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.145+0000 7fe50e000640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-04-16T17:15:16.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.145+0000 7fe50e000640 1 -- 172.21.15.45:0/1225907440 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe510010630 con 0x5603242d55c0 2024-04-16T17:15:16.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.145+0000 7fe52dbf1740 1 -- 172.21.15.45:0/1225907440 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe5100a20b0 con 0x5603242d55c0 2024-04-16T17:15:16.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.145+0000 7fe50e000640 1 -- 172.21.15.45:0/1225907440 <== mon.1 v2:172.21.15.117:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9496+0+0 (secure 0 0 0) 0x7fe5100988a0 con 0x5603242d55c0 2024-04-16T17:15:16.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.145+0000 7fe50e000640 1 -- 172.21.15.45:0/1225907440 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fe5100a0e50 con 0x5603242d55c0 2024-04-16T17:15:16.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.145+0000 7fe52dbf1740 10 client.27844 fetch_fsmap learned FSMap version 43 2024-04-16T17:15:16.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.145+0000 7fe52dbf1740 1 -- 172.21.15.45:0/1225907440 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x560324354ae0 con 0x5603242d55c0 2024-04-16T17:15:16.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.145+0000 7fe50e000640 1 -- 172.21.15.45:0/1225907440 <== mon.1 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe5100a20b0 con 0x5603242d55c0 2024-04-16T17:15:16.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.146+0000 7fe50e000640 1 -- 172.21.15.45:0/1225907440 <== mon.1 v2:172.21.15.117:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7fe5100872a0 con 0x5603242d55c0 2024-04-16T17:15:16.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.146+0000 7fe52dbf1740 10 client.27844 fetch_fsmap finished waiting for FSMap version 43 2024-04-16T17:15:16.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.146+0000 7fe52dbf1740 10 client.27844 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:15:16.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.146+0000 7fe52dbf1740 20 client.27844 populate_metadata read hostname 'smithi045' 2024-04-16T17:15:16.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.146+0000 7fe52dbf1740 1 --2- 172.21.15.45:0/1225907440 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x560324354fb0 0x5603243573a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:16.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.146+0000 7fe52dbf1740 4 client.27844 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-16T17:15:16.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.146+0000 7fe52dbf1740 1 -- 172.21.15.45:0/1225907440 --> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5603241bdaa0 con 0x560324354fb0 2024-04-16T17:15:16.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.146+0000 7fe523e00640 1 --2- 172.21.15.45:0/1225907440 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x560324354fb0 0x5603243573a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:16.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.147+0000 7fe523e00640 1 --2- 172.21.15.45:0/1225907440 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x560324354fb0 0x5603243573a0 crc :-1 s=READY pgs=230 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b8f6026fda5ecf4a server_cookie=4dc56b5080fc8d24 in_seq=0 out_seq=0 2024-04-16T17:15:16.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.147+0000 7fe50e000640 10 client.27844 ms_handle_connect on v2:172.21.15.104:6838/3660070592 2024-04-16T17:15:16.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.148+0000 7fe50e000640 1 -- 172.21.15.45:0/1225907440 <== mds.0 v2:172.21.15.104:6838/3660070592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5603241bdaa0 con 0x560324354fb0 2024-04-16T17:15:16.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.148+0000 7fe50e000640 10 client.27844 handle_command_reply: tid=0 2024-04-16T17:15:16.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.148+0000 7fe52dbf1740 1 client.27844 shutdown 2024-04-16T17:15:16.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.148+0000 7fe52dbf1740 1 -- 172.21.15.45:0/1225907440 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5603242d55c0 msgr2=0x5603242d7a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:15:16.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.148+0000 7fe52dbf1740 1 --2- 172.21.15.45:0/1225907440 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5603242d55c0 0x5603242d7a80 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fe51000cf10 tx=0x7fe51009bfa0 comp rx=0 tx=0).stop 2024-04-16T17:15:16.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.148+0000 7fe52dbf1740 1 -- 172.21.15.45:0/1225907440 shutdown_connections 2024-04-16T17:15:16.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.148+0000 7fe52dbf1740 1 --2- 172.21.15.45:0/1225907440 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x560324354fb0 0x5603243573a0 crc :-1 s=READY pgs=230 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:16.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.148+0000 7fe52dbf1740 1 --2- 172.21.15.45:0/1225907440 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5603242d55c0 0x5603242d7a80 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:16.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.148+0000 7fe52dbf1740 1 --2- 172.21.15.45:0/1225907440 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5603242de1a0 0x5603242d7fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:16.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.148+0000 7fe52dbf1740 1 --2- 172.21.15.45:0/1225907440 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5603242d4ca0 0x5603242e67f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:16.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.148+0000 7fe52dbf1740 1 -- 172.21.15.45:0/1225907440 >> 172.21.15.45:0/1225907440 conn(0x5603242cf3d0 msgr2=0x5603242d1070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:15:16.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.148+0000 7fe50e000640 0 client.27844 ms_handle_reset on v2:172.21.15.104:6838/3660070592 2024-04-16T17:15:16.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.149+0000 7fe52dbf1740 1 -- 172.21.15.45:0/1225907440 shutdown_connections 2024-04-16T17:15:16.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.149+0000 7fe52dbf1740 1 -- 172.21.15.45:0/1225907440 wait complete. 2024-04-16T17:15:16.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.149+0000 7fe52dbf1740 20 client.27844 trim_cache size 0 max 16384 2024-04-16T17:15:16.157 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-16T17:15:16.157 INFO:teuthology.orchestra.run.smithi045.stdout: "return_code": 0, 2024-04-16T17:15:16.157 INFO:teuthology.orchestra.run.smithi045.stdout: "scrub_tag": "fe329381-1740-46b6-9f3c-f31b19a0f3f0", 2024-04-16T17:15:16.157 INFO:teuthology.orchestra.run.smithi045.stdout: "mode": "asynchronous" 2024-04-16T17:15:16.157 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-16T17:15:16.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.149+0000 7fe52b000640 1 -- 172.21.15.45:0/3461616773 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fe5080a3cb0 msgr2=0x7fe5080a6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:15:16.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.149+0000 7fe52b000640 1 --2- 172.21.15.45:0/3461616773 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fe5080a3cb0 0x7fe5080a6170 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fe514000c00 tx=0x7fe51402df90 comp rx=0 tx=0).stop 2024-04-16T17:15:16.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.149+0000 7fe52b000640 1 -- 172.21.15.45:0/3461616773 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe524107300 msgr2=0x7fe524074840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:15:16.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.149+0000 7fe52b000640 1 --2- 172.21.15.45:0/3461616773 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe524107300 0x7fe524074840 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fe51000e9c0 tx=0x7fe51000ee80 comp rx=0 tx=0).stop 2024-04-16T17:15:16.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.150+0000 7fe52b000640 1 -- 172.21.15.45:0/3461616773 shutdown_connections 2024-04-16T17:15:16.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.150+0000 7fe52b000640 1 --2- 172.21.15.45:0/3461616773 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fe5080a3cb0 0x7fe5080a6170 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:16.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.150+0000 7fe52b000640 1 --2- 172.21.15.45:0/3461616773 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe524074d80 0x7fe5241a6ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:16.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.150+0000 7fe52b000640 1 --2- 172.21.15.45:0/3461616773 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe524107300 0x7fe524074840 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:16.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.150+0000 7fe52b000640 1 --2- 172.21.15.45:0/3461616773 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe524102230 0x7fe524074300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:16.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.150+0000 7fe52b000640 1 -- 172.21.15.45:0/3461616773 >> 172.21.15.45:0/3461616773 conn(0x7fe5240fda70 msgr2=0x7fe5240ffdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:15:16.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.150+0000 7fe52b000640 1 -- 172.21.15.45:0/3461616773 shutdown_connections 2024-04-16T17:15:16.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.150+0000 7fe52b000640 1 -- 172.21.15.45:0/3461616773 wait complete. 2024-04-16T17:15:16.168 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-16T17:15:16.168 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-16T17:15:16.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.376+0000 7f7698a00640 1 -- 172.21.15.45:0/4108699414 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7694102ac0 msgr2=0x7f7694106b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:15:16.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.376+0000 7f7698a00640 1 --2- 172.21.15.45:0/4108699414 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7694102ac0 0x7f7694106b00 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f767c009900 tx=0x7f767c02bc30 comp rx=0 tx=0).stop 2024-04-16T17:15:16.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.376+0000 7f7698a00640 1 -- 172.21.15.45:0/4108699414 shutdown_connections 2024-04-16T17:15:16.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.376+0000 7f7698a00640 1 --2- 172.21.15.45:0/4108699414 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f76941071e0 0x7f769410edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:16.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.376+0000 7f7698a00640 1 --2- 172.21.15.45:0/4108699414 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7694102ac0 0x7f7694106b00 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:16.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.376+0000 7f7698a00640 1 --2- 172.21.15.45:0/4108699414 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7694102110 0x7f76941024f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:16.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.376+0000 7f7698a00640 1 -- 172.21.15.45:0/4108699414 >> 172.21.15.45:0/4108699414 conn(0x7f76940fd9d0 msgr2=0x7f76940ffdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:15:16.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.377+0000 7f7698a00640 1 -- 172.21.15.45:0/4108699414 shutdown_connections 2024-04-16T17:15:16.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.377+0000 7f7698a00640 1 -- 172.21.15.45:0/4108699414 wait complete. 2024-04-16T17:15:16.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.378+0000 7f7698a00640 1 Processor -- start 2024-04-16T17:15:16.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.378+0000 7f7698a00640 1 -- start start 2024-04-16T17:15:16.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.378+0000 7f7698a00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7694102110 0x7f7694074300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:16.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.379+0000 7f7698a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f76941071e0 0x7f7694074840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:16.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.379+0000 7f7698a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7694074d80 0x7f76941a69c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:16.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.379+0000 7f7698a00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f7694112250 con 0x7f7694074d80 2024-04-16T17:15:16.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.379+0000 7f7698a00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f76941120d0 con 0x7f7694102110 2024-04-16T17:15:16.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.379+0000 7f7698a00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f76941123d0 con 0x7f76941071e0 2024-04-16T17:15:16.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.379+0000 7f7693200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7694074d80 0x7f76941a69c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:16.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.379+0000 7f7692800640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7694102110 0x7f7694074300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:16.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.379+0000 7f7693200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7694074d80 0x7f76941a69c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:46222/0 (socket says 172.21.15.45:46222) 2024-04-16T17:15:16.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.379+0000 7f7693200640 1 -- 172.21.15.45:0/3211761364 learned_addr learned my addr 172.21.15.45:0/3211761364 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:15:16.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.379+0000 7f7689e00640 1 --2- 172.21.15.45:0/3211761364 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f76941071e0 0x7f7694074840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:16.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.379+0000 7f7693200640 1 -- 172.21.15.45:0/3211761364 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f76941071e0 msgr2=0x7f7694074840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:15:16.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.379+0000 7f7693200640 1 --2- 172.21.15.45:0/3211761364 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f76941071e0 0x7f7694074840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:16.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.379+0000 7f7693200640 1 -- 172.21.15.45:0/3211761364 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7694102110 msgr2=0x7f7694074300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:15:16.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.379+0000 7f7693200640 1 --2- 172.21.15.45:0/3211761364 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7694102110 0x7f7694074300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:16.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.379+0000 7f7693200640 1 -- 172.21.15.45:0/3211761364 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76941a7060 con 0x7f7694074d80 2024-04-16T17:15:16.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.380+0000 7f7693200640 1 --2- 172.21.15.45:0/3211761364 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7694074d80 0x7f76941a69c0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f768c00e970 tx=0x7f768c00ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:15:16.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.380+0000 7f768be00640 1 -- 172.21.15.45:0/3211761364 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f768c00ccc0 con 0x7f7694074d80 2024-04-16T17:15:16.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.380+0000 7f768be00640 1 -- 172.21.15.45:0/3211761364 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f768c00ce60 con 0x7f7694074d80 2024-04-16T17:15:16.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.380+0000 7f7698a00640 1 -- 172.21.15.45:0/3211761364 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76941a7350 con 0x7f7694074d80 2024-04-16T17:15:16.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.380+0000 7f768be00640 1 -- 172.21.15.45:0/3211761364 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f768c010490 con 0x7f7694074d80 2024-04-16T17:15:16.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.380+0000 7f7698a00640 1 -- 172.21.15.45:0/3211761364 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76941a7820 con 0x7f7694074d80 2024-04-16T17:15:16.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.383+0000 7f768be00640 1 -- 172.21.15.45:0/3211761364 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f768c005020 con 0x7f7694074d80 2024-04-16T17:15:16.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.385+0000 7f769c1d3740 1 Processor -- start 2024-04-16T17:15:16.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.385+0000 7f769c1d3740 1 -- start start 2024-04-16T17:15:16.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.385+0000 7f768be00640 1 --2- 172.21.15.45:0/3211761364 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f76780a3d20 0x7f76780a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:16.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.385+0000 7f768be00640 1 -- 172.21.15.45:0/3211761364 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9496+0+0 (secure 0 0 0) 0x7f768c0d1060 con 0x7f7694074d80 2024-04-16T17:15:16.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.385+0000 7f769c1d3740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c8a7659390 0x55c8a7659770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:16.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.385+0000 7f7689e00640 1 --2- 172.21.15.45:0/3211761364 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f76780a3d20 0x7f76780a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:16.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.385+0000 7f769c1d3740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c8a7659cb0 0x55c8a7662350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:16.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.385+0000 7f769c1d3740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c8a7662890 0x55c8a7664c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:16.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.385+0000 7f769c1d3740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55c8a75bf120 con 0x55c8a7659390 2024-04-16T17:15:16.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.385+0000 7f769c1d3740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55c8a7565340 con 0x55c8a7662890 2024-04-16T17:15:16.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.385+0000 7f769c1d3740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55c8a75bb0e0 con 0x55c8a7659cb0 2024-04-16T17:15:16.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.386+0000 7f7692800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c8a7659390 0x55c8a7659770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:16.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.386+0000 7f7692800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c8a7659390 0x55c8a7659770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:46226/0 (socket says 172.21.15.45:46226) 2024-04-16T17:15:16.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.386+0000 7f7692800640 1 -- 172.21.15.45:0/2792385377 learned_addr learned my addr 172.21.15.45:0/2792385377 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:15:16.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.386+0000 7f7693200640 1 --2- 172.21.15.45:0/2792385377 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c8a7662890 0x55c8a7664c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:16.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.386+0000 7f7689e00640 1 --2- 172.21.15.45:0/2792385377 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c8a7659cb0 0x55c8a7662350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:16.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.386+0000 7f7692800640 1 -- 172.21.15.45:0/2792385377 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c8a7659cb0 msgr2=0x55c8a7662350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:15:16.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.386+0000 7f7692800640 1 --2- 172.21.15.45:0/2792385377 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c8a7659cb0 0x55c8a7662350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:16.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.386+0000 7f7692800640 1 -- 172.21.15.45:0/2792385377 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c8a7662890 msgr2=0x55c8a7664c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:15:16.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.386+0000 7f7692800640 1 --2- 172.21.15.45:0/2792385377 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c8a7662890 0x55c8a7664c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:16.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.386+0000 7f7692800640 1 -- 172.21.15.45:0/2792385377 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c8a739b7d0 con 0x55c8a7659390 2024-04-16T17:15:16.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.386+0000 7f7693200640 1 --2- 172.21.15.45:0/2792385377 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c8a7662890 0x55c8a7664c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:15:16.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.386+0000 7f7689e00640 1 --2- 172.21.15.45:0/3211761364 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f76780a3d20 0x7f76780a61e0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f767c009230 tx=0x7f767c02ca00 comp rx=0 tx=0).ready entity=mgr.25411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:15:16.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.387+0000 7f7692800640 1 --2- 172.21.15.45:0/2792385377 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c8a7659390 0x55c8a7659770 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f76840114f0 tx=0x7f76840119b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdea32845d8fcb82 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:15:16.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.387+0000 7f7683400640 1 -- 172.21.15.45:0/2792385377 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f768401e070 con 0x55c8a7659390 2024-04-16T17:15:16.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.387+0000 7f7683400640 1 -- 172.21.15.45:0/2792385377 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f768400fa40 con 0x55c8a7659390 2024-04-16T17:15:16.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.387+0000 7f7683400640 1 -- 172.21.15.45:0/2792385377 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7684019b70 con 0x55c8a7659390 2024-04-16T17:15:16.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.388+0000 7f769c1d3740 1 -- 172.21.15.45:0/2792385377 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c8a7659390 msgr2=0x55c8a7659770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:15:16.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.388+0000 7f769c1d3740 1 --2- 172.21.15.45:0/2792385377 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c8a7659390 0x55c8a7659770 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f76840114f0 tx=0x7f76840119b0 comp rx=0 tx=0).stop 2024-04-16T17:15:16.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.388+0000 7f769c1d3740 1 -- 172.21.15.45:0/2792385377 shutdown_connections 2024-04-16T17:15:16.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.388+0000 7f769c1d3740 1 --2- 172.21.15.45:0/2792385377 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c8a7659cb0 0x55c8a7662350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:16.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.388+0000 7f769c1d3740 1 --2- 172.21.15.45:0/2792385377 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c8a7662890 0x55c8a7664c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:16.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.388+0000 7f769c1d3740 1 --2- 172.21.15.45:0/2792385377 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c8a7659390 0x55c8a7659770 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:16.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.388+0000 7f769c1d3740 1 -- 172.21.15.45:0/2792385377 >> 172.21.15.45:0/2792385377 conn(0x55c8a7653ac0 msgr2=0x55c8a7653ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:15:16.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.388+0000 7f769c1d3740 1 -- 172.21.15.45:0/2792385377 shutdown_connections 2024-04-16T17:15:16.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.388+0000 7f769c1d3740 1 -- 172.21.15.45:0/2792385377 wait complete. 2024-04-16T17:15:16.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.388+0000 7f769c1d3740 1 -- start start 2024-04-16T17:15:16.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.388+0000 7f769c1d3740 1 Processor -- start 2024-04-16T17:15:16.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.389+0000 7f769c1d3740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c8a7659390 0x55c8a765f930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:16.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.389+0000 7f769c1d3740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c8a7659cb0 0x55c8a765fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:16.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.389+0000 7f769c1d3740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c8a7662890 0x55c8a76603b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:16.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.389+0000 7f769c1d3740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55c8a7665680 con 0x55c8a7659cb0 2024-04-16T17:15:16.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.389+0000 7f769c1d3740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55c8a7578710 con 0x55c8a7662890 2024-04-16T17:15:16.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.389+0000 7f769c1d3740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55c8a7665800 con 0x55c8a7659390 2024-04-16T17:15:16.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.389+0000 7f769c1d3740 10 client.0 _ll_register_callbacks cb 0x55c8a7573af0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:15:16.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.389+0000 7f7689e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c8a7659cb0 0x55c8a765fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:16.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.389+0000 7f7692800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c8a7659390 0x55c8a765f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:16.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.389+0000 7f7689e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c8a7659cb0 0x55c8a765fe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:46238/0 (socket says 172.21.15.45:46238) 2024-04-16T17:15:16.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.389+0000 7f7689e00640 1 -- 172.21.15.45:0/2082143636 learned_addr learned my addr 172.21.15.45:0/2082143636 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:15:16.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.389+0000 7f7693200640 1 --2- 172.21.15.45:0/2082143636 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c8a7662890 0x55c8a76603b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:16.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.390+0000 7f7689e00640 1 -- 172.21.15.45:0/2082143636 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c8a7659390 msgr2=0x55c8a765f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:15:16.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.390+0000 7f7689e00640 1 --2- 172.21.15.45:0/2082143636 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c8a7659390 0x55c8a765f930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:16.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.390+0000 7f7689e00640 1 -- 172.21.15.45:0/2082143636 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c8a7662890 msgr2=0x55c8a76603b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:15:16.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.390+0000 7f7689e00640 1 --2- 172.21.15.45:0/2082143636 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c8a7662890 0x55c8a76603b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:16.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.390+0000 7f7689e00640 1 -- 172.21.15.45:0/2082143636 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c8a72d13f0 con 0x55c8a7659cb0 2024-04-16T17:15:16.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.390+0000 7f7692800640 1 --2- 172.21.15.45:0/2082143636 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c8a7659390 0x55c8a765f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:15:16.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.390+0000 7f7689e00640 1 --2- 172.21.15.45:0/2082143636 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c8a7659cb0 0x55c8a765fe70 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f767c009900 tx=0x7f767c02bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9582a9cb14e7a49 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:15:16.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.390+0000 7f7677e00640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-16T17:15:16.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.390+0000 7f7677e00640 1 -- 172.21.15.45:0/2082143636 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f767c0050c0 con 0x55c8a7659cb0 2024-04-16T17:15:16.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.390+0000 7f7677e00640 1 -- 172.21.15.45:0/2082143636 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9496+0+0 (secure 0 0 0) 0x7f767c005260 con 0x55c8a7659cb0 2024-04-16T17:15:16.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.390+0000 7f769c1d3740 1 -- 172.21.15.45:0/2082143636 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f768400fa40 con 0x55c8a7659cb0 2024-04-16T17:15:16.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.391+0000 7f7677e00640 1 -- 172.21.15.45:0/2082143636 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f767c05b050 con 0x55c8a7659cb0 2024-04-16T17:15:16.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.391+0000 7f769c1d3740 10 client.17754 fetch_fsmap learned FSMap version 43 2024-04-16T17:15:16.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.391+0000 7f769c1d3740 1 -- 172.21.15.45:0/2082143636 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55c8a72910f0 con 0x55c8a7659cb0 2024-04-16T17:15:16.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.391+0000 7f7677e00640 1 -- 172.21.15.45:0/2082143636 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f768400fa40 con 0x55c8a7659cb0 2024-04-16T17:15:16.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.391+0000 7f7677e00640 1 -- 172.21.15.45:0/2082143636 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f767c034440 con 0x55c8a7659cb0 2024-04-16T17:15:16.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.391+0000 7f769c1d3740 10 client.17754 fetch_fsmap finished waiting for FSMap version 43 2024-04-16T17:15:16.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.391+0000 7f769c1d3740 10 client.17754 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:15:16.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.391+0000 7f769c1d3740 20 client.17754 populate_metadata read hostname 'smithi045' 2024-04-16T17:15:16.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.391+0000 7f769c1d3740 1 --2- 172.21.15.45:0/2082143636 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55c8a7672cc0 0x55c8a7675140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:16.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.391+0000 7f769c1d3740 4 client.17754 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:15:16.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.391+0000 7f769c1d3740 1 -- 172.21.15.45:0/2082143636 --> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c8a75ccb40 con 0x55c8a7672cc0 2024-04-16T17:15:16.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.392+0000 7f7692800640 1 --2- 172.21.15.45:0/2082143636 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55c8a7672cc0 0x55c8a7675140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:16.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.392+0000 7f7692800640 1 --2- 172.21.15.45:0/2082143636 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55c8a7672cc0 0x55c8a7675140 crc :-1 s=READY pgs=231 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b44fd463f8c5570d server_cookie=b33090ac26149ab6 in_seq=0 out_seq=0 2024-04-16T17:15:16.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.392+0000 7f7677e00640 10 client.17754 ms_handle_connect on v2:172.21.15.104:6838/3660070592 2024-04-16T17:15:16.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.393+0000 7f7677e00640 1 -- 172.21.15.45:0/2082143636 <== mds.0 v2:172.21.15.104:6838/3660070592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c8a75ccb40 con 0x55c8a7672cc0 2024-04-16T17:15:16.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.393+0000 7f7677e00640 10 client.17754 handle_command_reply: tid=0 2024-04-16T17:15:16.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.393+0000 7f769c1d3740 1 client.17754 shutdown 2024-04-16T17:15:16.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.393+0000 7f769c1d3740 1 -- 172.21.15.45:0/2082143636 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c8a7659cb0 msgr2=0x55c8a765fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:15:16.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.393+0000 7f769c1d3740 1 --2- 172.21.15.45:0/2082143636 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c8a7659cb0 0x55c8a765fe70 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f767c009900 tx=0x7f767c02bc30 comp rx=0 tx=0).stop 2024-04-16T17:15:16.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.393+0000 7f769c1d3740 1 -- 172.21.15.45:0/2082143636 shutdown_connections 2024-04-16T17:15:16.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.393+0000 7f769c1d3740 1 --2- 172.21.15.45:0/2082143636 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c8a7662890 0x55c8a76603b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:16.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.393+0000 7f769c1d3740 1 --2- 172.21.15.45:0/2082143636 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c8a7659cb0 0x55c8a765fe70 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:16.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.393+0000 7f769c1d3740 1 --2- 172.21.15.45:0/2082143636 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55c8a7672cc0 0x55c8a7675140 crc :-1 s=READY pgs=231 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:16.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.393+0000 7f769c1d3740 1 --2- 172.21.15.45:0/2082143636 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c8a7659390 0x55c8a765f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:16.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.393+0000 7f7677e00640 0 client.17754 ms_handle_reset on v2:172.21.15.104:6838/3660070592 2024-04-16T17:15:16.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.393+0000 7f769c1d3740 1 -- 172.21.15.45:0/2082143636 >> 172.21.15.45:0/2082143636 conn(0x55c8a7653ac0 msgr2=0x55c8a7655810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:15:16.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.394+0000 7f769c1d3740 1 -- 172.21.15.45:0/2082143636 shutdown_connections 2024-04-16T17:15:16.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.394+0000 7f769c1d3740 1 -- 172.21.15.45:0/2082143636 wait complete. 2024-04-16T17:15:16.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.394+0000 7f769c1d3740 20 client.17754 trim_cache size 0 max 16384 2024-04-16T17:15:16.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.412+0000 7f769c1d3740 1 Processor -- start 2024-04-16T17:15:16.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.412+0000 7f769c1d3740 1 -- start start 2024-04-16T17:15:16.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.413+0000 7f769c1d3740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c8a7659390 0x55c8a7662060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:16.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.413+0000 7f769c1d3740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c8a7659cb0 0x55c8a766e540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:16.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.413+0000 7f769c1d3740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c8a7662890 0x55c8a765ea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:16.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.413+0000 7f769c1d3740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f76941b3e10 con 0x55c8a7659cb0 2024-04-16T17:15:16.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.413+0000 7f769c1d3740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f76941b16e0 con 0x55c8a7662890 2024-04-16T17:15:16.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.413+0000 7f769c1d3740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f76941af550 con 0x55c8a7659390 2024-04-16T17:15:16.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.413+0000 7f7689e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c8a7659cb0 0x55c8a766e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:16.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.413+0000 7f7689e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c8a7659cb0 0x55c8a766e540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:46240/0 (socket says 172.21.15.45:46240) 2024-04-16T17:15:16.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.413+0000 7f7689e00640 1 -- 172.21.15.45:0/3488500096 learned_addr learned my addr 172.21.15.45:0/3488500096 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:15:16.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.413+0000 7f7692800640 1 --2- 172.21.15.45:0/3488500096 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c8a7659390 0x55c8a7662060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:16.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.413+0000 7f7693200640 1 --2- 172.21.15.45:0/3488500096 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c8a7662890 0x55c8a765ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:16.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.414+0000 7f7689e00640 1 -- 172.21.15.45:0/3488500096 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c8a7659390 msgr2=0x55c8a7662060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:15:16.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.414+0000 7f7689e00640 1 --2- 172.21.15.45:0/3488500096 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c8a7659390 0x55c8a7662060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:16.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.414+0000 7f7689e00640 1 -- 172.21.15.45:0/3488500096 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c8a7662890 msgr2=0x55c8a765ea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:15:16.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.414+0000 7f7689e00640 1 --2- 172.21.15.45:0/3488500096 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c8a7662890 0x55c8a765ea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:16.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.414+0000 7f7689e00640 1 -- 172.21.15.45:0/3488500096 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c8a7664fb0 con 0x55c8a7659cb0 2024-04-16T17:15:16.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.414+0000 7f7692800640 1 --2- 172.21.15.45:0/3488500096 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c8a7659390 0x55c8a7662060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:15:16.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.414+0000 7f7689e00640 1 --2- 172.21.15.45:0/3488500096 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c8a7659cb0 0x55c8a766e540 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f767c02c170 tx=0x7f767c005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2d8d5d5881dcb0a server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:15:16.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.414+0000 7f7682a00640 1 -- 172.21.15.45:0/3488500096 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f767c063020 con 0x55c8a7659cb0 2024-04-16T17:15:16.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.414+0000 7f7682a00640 1 -- 172.21.15.45:0/3488500096 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f767c052070 con 0x55c8a7659cb0 2024-04-16T17:15:16.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.414+0000 7f7682a00640 1 -- 172.21.15.45:0/3488500096 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f767c0053a0 con 0x55c8a7659cb0 2024-04-16T17:15:16.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.415+0000 7f769c1d3740 1 -- 172.21.15.45:0/3488500096 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c8a7659cb0 msgr2=0x55c8a766e540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:15:16.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.415+0000 7f769c1d3740 1 --2- 172.21.15.45:0/3488500096 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c8a7659cb0 0x55c8a766e540 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f767c02c170 tx=0x7f767c005fd0 comp rx=0 tx=0).stop 2024-04-16T17:15:16.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.415+0000 7f769c1d3740 1 -- 172.21.15.45:0/3488500096 shutdown_connections 2024-04-16T17:15:16.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.415+0000 7f769c1d3740 1 --2- 172.21.15.45:0/3488500096 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c8a7662890 0x55c8a765ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:16.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.415+0000 7f769c1d3740 1 --2- 172.21.15.45:0/3488500096 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c8a7659cb0 0x55c8a766e540 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:16.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.415+0000 7f769c1d3740 1 --2- 172.21.15.45:0/3488500096 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c8a7659390 0x55c8a7662060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:16.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.415+0000 7f769c1d3740 1 -- 172.21.15.45:0/3488500096 >> 172.21.15.45:0/3488500096 conn(0x55c8a7653ac0 msgr2=0x55c8a765f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:15:16.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.415+0000 7f769c1d3740 1 -- 172.21.15.45:0/3488500096 shutdown_connections 2024-04-16T17:15:16.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.415+0000 7f769c1d3740 1 -- 172.21.15.45:0/3488500096 wait complete. 2024-04-16T17:15:16.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.415+0000 7f769c1d3740 1 -- start start 2024-04-16T17:15:16.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.416+0000 7f769c1d3740 1 Processor -- start 2024-04-16T17:15:16.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.416+0000 7f769c1d3740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c8a7659390 0x55c8a766ae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:16.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.416+0000 7f769c1d3740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c8a7659cb0 0x55c8a76661c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:16.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.416+0000 7f769c1d3740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c8a7662890 0x55c8a7666700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:16.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.416+0000 7f769c1d3740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f76941afba0 con 0x55c8a7659cb0 2024-04-16T17:15:16.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.416+0000 7f769c1d3740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f76941b1010 con 0x55c8a7662890 2024-04-16T17:15:16.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.416+0000 7f769c1d3740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f76941b0270 con 0x55c8a7659390 2024-04-16T17:15:16.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.416+0000 7f769c1d3740 10 client.0 _ll_register_callbacks cb 0x7f76840093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:15:16.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.416+0000 7f7692800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c8a7659390 0x55c8a766ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:16.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.416+0000 7f7689e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c8a7659cb0 0x55c8a76661c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:16.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.416+0000 7f7692800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c8a7659390 0x55c8a766ae50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:38332/0 (socket says 172.21.15.45:38332) 2024-04-16T17:15:16.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.416+0000 7f7692800640 1 -- 172.21.15.45:0/2036287980 learned_addr learned my addr 172.21.15.45:0/2036287980 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:15:16.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.416+0000 7f7693200640 1 --2- 172.21.15.45:0/2036287980 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c8a7662890 0x55c8a7666700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:16.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.417+0000 7f7692800640 1 -- 172.21.15.45:0/2036287980 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c8a7662890 msgr2=0x55c8a7666700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:15:16.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.417+0000 7f7692800640 1 --2- 172.21.15.45:0/2036287980 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c8a7662890 0x55c8a7666700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:16.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.417+0000 7f7692800640 1 -- 172.21.15.45:0/2036287980 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c8a7659cb0 msgr2=0x55c8a76661c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:15:16.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.417+0000 7f7692800640 1 --2- 172.21.15.45:0/2036287980 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c8a7659cb0 0x55c8a76661c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:16.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.417+0000 7f7692800640 1 -- 172.21.15.45:0/2036287980 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c8a7666f30 con 0x55c8a7659390 2024-04-16T17:15:16.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.417+0000 7f7692800640 1 --2- 172.21.15.45:0/2036287980 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c8a7659390 0x55c8a766ae50 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f7684009de0 tx=0x7f7684009990 comp rx=0 tx=0).ready entity=mon.1 client_cookie=211caecb3fbd0cf1 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:15:16.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.417+0000 7f7677e00640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-04-16T17:15:16.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.417+0000 7f7677e00640 1 -- 172.21.15.45:0/2036287980 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f768402d070 con 0x55c8a7659390 2024-04-16T17:15:16.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.417+0000 7f769c1d3740 1 -- 172.21.15.45:0/2036287980 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f767c052070 con 0x55c8a7659390 2024-04-16T17:15:16.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.417+0000 7f7677e00640 1 -- 172.21.15.45:0/2036287980 <== mon.1 v2:172.21.15.117:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9496+0+0 (secure 0 0 0) 0x7f768400f5e0 con 0x55c8a7659390 2024-04-16T17:15:16.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.418+0000 7f7677e00640 1 -- 172.21.15.45:0/2036287980 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f7684039050 con 0x55c8a7659390 2024-04-16T17:15:16.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.418+0000 7f769c1d3740 10 client.27862 fetch_fsmap learned FSMap version 43 2024-04-16T17:15:16.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.418+0000 7f769c1d3740 1 -- 172.21.15.45:0/2036287980 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55c8a76d9350 con 0x55c8a7659390 2024-04-16T17:15:16.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.418+0000 7f7677e00640 1 -- 172.21.15.45:0/2036287980 <== mon.1 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f767c052070 con 0x55c8a7659390 2024-04-16T17:15:16.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.418+0000 7f7677e00640 1 -- 172.21.15.45:0/2036287980 <== mon.1 v2:172.21.15.117:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f76840197c0 con 0x55c8a7659390 2024-04-16T17:15:16.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.418+0000 7f769c1d3740 10 client.27862 fetch_fsmap finished waiting for FSMap version 43 2024-04-16T17:15:16.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.418+0000 7f769c1d3740 10 client.27862 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:15:16.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.418+0000 7f769c1d3740 20 client.27862 populate_metadata read hostname 'smithi045' 2024-04-16T17:15:16.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.418+0000 7f769c1d3740 1 --2- 172.21.15.45:0/2036287980 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55c8a76d98e0 0x55c8a76dbcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:16.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.418+0000 7f769c1d3740 4 client.27862 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-16T17:15:16.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.418+0000 7f769c1d3740 1 -- 172.21.15.45:0/2036287980 --> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55c8a764f180 con 0x55c8a76d98e0 2024-04-16T17:15:16.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.418+0000 7f7689e00640 1 --2- 172.21.15.45:0/2036287980 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55c8a76d98e0 0x55c8a76dbcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:16.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.419+0000 7f7689e00640 1 --2- 172.21.15.45:0/2036287980 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55c8a76d98e0 0x55c8a76dbcd0 crc :-1 s=READY pgs=232 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=85125c56dc945c6 server_cookie=c73eafb4ac0b848a in_seq=0 out_seq=0 2024-04-16T17:15:16.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.419+0000 7f7677e00640 10 client.27862 ms_handle_connect on v2:172.21.15.104:6838/3660070592 2024-04-16T17:15:16.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.419+0000 7f7677e00640 1 -- 172.21.15.45:0/2036287980 <== mds.0 v2:172.21.15.104:6838/3660070592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55c8a764f180 con 0x55c8a76d98e0 2024-04-16T17:15:16.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.419+0000 7f7677e00640 10 client.27862 handle_command_reply: tid=0 2024-04-16T17:15:16.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.419+0000 7f769c1d3740 1 client.27862 shutdown 2024-04-16T17:15:16.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.419+0000 7f769c1d3740 1 -- 172.21.15.45:0/2036287980 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c8a7659390 msgr2=0x55c8a766ae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:15:16.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.419+0000 7f769c1d3740 1 --2- 172.21.15.45:0/2036287980 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c8a7659390 0x55c8a766ae50 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f7684009de0 tx=0x7f7684009990 comp rx=0 tx=0).stop 2024-04-16T17:15:16.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.419+0000 7f769c1d3740 1 -- 172.21.15.45:0/2036287980 shutdown_connections 2024-04-16T17:15:16.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.419+0000 7f769c1d3740 1 --2- 172.21.15.45:0/2036287980 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c8a7662890 0x55c8a7666700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:16.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.419+0000 7f769c1d3740 1 --2- 172.21.15.45:0/2036287980 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c8a7659cb0 0x55c8a76661c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:16.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.419+0000 7f769c1d3740 1 --2- 172.21.15.45:0/2036287980 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55c8a76d98e0 0x55c8a76dbcd0 crc :-1 s=READY pgs=232 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:16.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.419+0000 7f769c1d3740 1 --2- 172.21.15.45:0/2036287980 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c8a7659390 0x55c8a766ae50 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:16.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.419+0000 7f769c1d3740 1 -- 172.21.15.45:0/2036287980 >> 172.21.15.45:0/2036287980 conn(0x55c8a7653ac0 msgr2=0x55c8a765e910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:15:16.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.419+0000 7f7677e00640 0 client.27862 ms_handle_reset on v2:172.21.15.104:6838/3660070592 2024-04-16T17:15:16.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.419+0000 7f769c1d3740 1 -- 172.21.15.45:0/2036287980 shutdown_connections 2024-04-16T17:15:16.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.419+0000 7f769c1d3740 1 -- 172.21.15.45:0/2036287980 wait complete. 2024-04-16T17:15:16.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.420+0000 7f769c1d3740 20 client.27862 trim_cache size 0 max 16384 2024-04-16T17:15:16.428 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-16T17:15:16.429 INFO:teuthology.orchestra.run.smithi045.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-16T17:15:16.429 INFO:teuthology.orchestra.run.smithi045.stdout: "scrubs": { 2024-04-16T17:15:16.429 INFO:teuthology.orchestra.run.smithi045.stdout: "fe329381-1740-46b6-9f3c-f31b19a0f3f0": { 2024-04-16T17:15:16.429 INFO:teuthology.orchestra.run.smithi045.stdout: "path": "/", 2024-04-16T17:15:16.429 INFO:teuthology.orchestra.run.smithi045.stdout: "tag": "fe329381-1740-46b6-9f3c-f31b19a0f3f0", 2024-04-16T17:15:16.429 INFO:teuthology.orchestra.run.smithi045.stdout: "options": "recursive,force" 2024-04-16T17:15:16.429 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:15:16.430 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:15:16.430 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-16T17:15:16.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.420+0000 7f7698a00640 1 -- 172.21.15.45:0/3211761364 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f76780a3d20 msgr2=0x7f76780a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:15:16.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.420+0000 7f7698a00640 1 --2- 172.21.15.45:0/3211761364 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f76780a3d20 0x7f76780a61e0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f767c009230 tx=0x7f767c02ca00 comp rx=0 tx=0).stop 2024-04-16T17:15:16.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.420+0000 7f7698a00640 1 -- 172.21.15.45:0/3211761364 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7694074d80 msgr2=0x7f76941a69c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:15:16.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.420+0000 7f7698a00640 1 --2- 172.21.15.45:0/3211761364 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7694074d80 0x7f76941a69c0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f768c00e970 tx=0x7f768c00ed70 comp rx=0 tx=0).stop 2024-04-16T17:15:16.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.420+0000 7f7698a00640 1 -- 172.21.15.45:0/3211761364 shutdown_connections 2024-04-16T17:15:16.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.420+0000 7f7698a00640 1 --2- 172.21.15.45:0/3211761364 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f76780a3d20 0x7f76780a61e0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:16.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.420+0000 7f7698a00640 1 --2- 172.21.15.45:0/3211761364 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f76941071e0 0x7f7694074840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:16.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.420+0000 7f7698a00640 1 --2- 172.21.15.45:0/3211761364 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7694074d80 0x7f76941a69c0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:16.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.420+0000 7f7698a00640 1 --2- 172.21.15.45:0/3211761364 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7694102110 0x7f7694074300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:16.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.420+0000 7f7698a00640 1 -- 172.21.15.45:0/3211761364 >> 172.21.15.45:0/3211761364 conn(0x7f76940fd9d0 msgr2=0x7f76940fedf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:15:16.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.420+0000 7f7698a00640 1 -- 172.21.15.45:0/3211761364 shutdown_connections 2024-04-16T17:15:16.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:16.420+0000 7f7698a00640 1 -- 172.21.15.45:0/3211761364 wait complete. 2024-04-16T17:15:16.437 INFO:tasks.cephfs.filesystem:scrub status for tag:fe329381-1740-46b6-9f3c-f31b19a0f3f0 - {'path': '/', 'tag': 'fe329381-1740-46b6-9f3c-f31b19a0f3f0', 'options': 'recursive,force'} 2024-04-16T17:15:16.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:15:16 smithi104 ceph-mon[36457]: pgmap v265: 129 pgs: 129 active+clean; 59 MiB data, 876 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 16 KiB/s wr, 5 op/s 2024-04-16T17:15:16.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:15:16 smithi104 ceph-mon[36457]: scrub summary: active paths [/] 2024-04-16T17:15:16.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:15:16 smithi104 ceph-mon[36457]: scrub summary: idle+waiting paths [/] 2024-04-16T17:15:16.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:15:16 smithi045 ceph-mon[31676]: pgmap v265: 129 pgs: 129 active+clean; 59 MiB data, 876 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 16 KiB/s wr, 5 op/s 2024-04-16T17:15:16.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:15:16 smithi045 ceph-mon[31676]: scrub summary: active paths [/] 2024-04-16T17:15:16.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:15:16 smithi045 ceph-mon[31676]: scrub summary: idle+waiting paths [/] 2024-04-16T17:15:17.054 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:15:16 smithi117 ceph-mon[36730]: pgmap v265: 129 pgs: 129 active+clean; 59 MiB data, 876 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 16 KiB/s wr, 5 op/s 2024-04-16T17:15:17.054 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:15:16 smithi117 ceph-mon[36730]: scrub summary: active paths [/] 2024-04-16T17:15:17.055 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:15:16 smithi117 ceph-mon[36730]: scrub summary: idle+waiting paths [/] 2024-04-16T17:15:17.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:15:17 smithi104 ceph-mon[36457]: scrub queued for path: / 2024-04-16T17:15:17.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:15:17 smithi104 ceph-mon[36457]: scrub summary: idle+waiting paths [/] 2024-04-16T17:15:17.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:15:17 smithi104 ceph-mon[36457]: scrub summary: active paths [/] 2024-04-16T17:15:17.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:15:17 smithi104 ceph-mon[36457]: scrub summary: active paths [/] 2024-04-16T17:15:17.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:15:17 smithi104 ceph-mon[36457]: scrub summary: idle+waiting paths [/] 2024-04-16T17:15:17.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:15:17 smithi104 ceph-mon[36457]: scrub summary: idle+waiting paths [/] 2024-04-16T17:15:17.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:15:17 smithi045 ceph-mon[31676]: scrub queued for path: / 2024-04-16T17:15:17.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:15:17 smithi045 ceph-mon[31676]: scrub summary: idle+waiting paths [/] 2024-04-16T17:15:17.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:15:17 smithi045 ceph-mon[31676]: scrub summary: active paths [/] 2024-04-16T17:15:17.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:15:17 smithi045 ceph-mon[31676]: scrub summary: active paths [/] 2024-04-16T17:15:17.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:15:17 smithi045 ceph-mon[31676]: scrub summary: idle+waiting paths [/] 2024-04-16T17:15:17.978 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:15:17 smithi045 ceph-mon[31676]: scrub summary: idle+waiting paths [/] 2024-04-16T17:15:18.054 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:15:17 smithi117 ceph-mon[36730]: scrub queued for path: / 2024-04-16T17:15:18.055 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:15:17 smithi117 ceph-mon[36730]: scrub summary: idle+waiting paths [/] 2024-04-16T17:15:18.055 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:15:17 smithi117 ceph-mon[36730]: scrub summary: active paths [/] 2024-04-16T17:15:18.055 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:15:17 smithi117 ceph-mon[36730]: scrub summary: active paths [/] 2024-04-16T17:15:18.055 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:15:17 smithi117 ceph-mon[36730]: scrub summary: idle+waiting paths [/] 2024-04-16T17:15:18.055 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:15:17 smithi117 ceph-mon[36730]: scrub summary: idle+waiting paths [/] 2024-04-16T17:15:18.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:15:18 smithi104 ceph-mon[36457]: pgmap v266: 129 pgs: 129 active+clean; 60 MiB data, 880 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 25 KiB/s wr, 19 op/s 2024-04-16T17:15:18.976 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:15:18 smithi045 ceph-mon[31676]: pgmap v266: 129 pgs: 129 active+clean; 60 MiB data, 880 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 25 KiB/s wr, 19 op/s 2024-04-16T17:15:19.054 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:15:18 smithi117 ceph-mon[36730]: pgmap v266: 129 pgs: 129 active+clean; 60 MiB data, 880 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 25 KiB/s wr, 19 op/s 2024-04-16T17:15:20.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:15:20 smithi104 ceph-mon[36457]: pgmap v267: 129 pgs: 129 active+clean; 60 MiB data, 880 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-16T17:15:20.976 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:15:20 smithi045 ceph-mon[31676]: pgmap v267: 129 pgs: 129 active+clean; 60 MiB data, 880 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-16T17:15:21.054 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:15:20 smithi117 ceph-mon[36730]: pgmap v267: 129 pgs: 129 active+clean; 60 MiB data, 880 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-16T17:15:22.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:15:22 smithi104 ceph-mon[36457]: pgmap v268: 129 pgs: 129 active+clean; 60 MiB data, 880 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 10 KiB/s wr, 15 op/s 2024-04-16T17:15:22.976 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:15:22 smithi045 ceph-mon[31676]: pgmap v268: 129 pgs: 129 active+clean; 60 MiB data, 880 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 10 KiB/s wr, 15 op/s 2024-04-16T17:15:23.054 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:15:22 smithi117 ceph-mon[36730]: pgmap v268: 129 pgs: 129 active+clean; 60 MiB data, 880 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 10 KiB/s wr, 15 op/s 2024-04-16T17:15:24.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:15:24 smithi104 ceph-mon[36457]: pgmap v269: 129 pgs: 129 active+clean; 60 MiB data, 880 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 10 KiB/s wr, 15 op/s 2024-04-16T17:15:24.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:15:24 smithi104 ceph-mon[36457]: scrub summary: idle 2024-04-16T17:15:24.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:15:24 smithi104 ceph-mon[36457]: scrub summary: idle 2024-04-16T17:15:24.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:15:24 smithi104 ceph-mon[36457]: scrub summary: idle 2024-04-16T17:15:24.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:15:24 smithi045 ceph-mon[31676]: pgmap v269: 129 pgs: 129 active+clean; 60 MiB data, 880 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 10 KiB/s wr, 15 op/s 2024-04-16T17:15:24.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:15:24 smithi045 ceph-mon[31676]: scrub summary: idle 2024-04-16T17:15:24.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:15:24 smithi045 ceph-mon[31676]: scrub summary: idle 2024-04-16T17:15:24.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:15:24 smithi045 ceph-mon[31676]: scrub summary: idle 2024-04-16T17:15:25.054 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:15:24 smithi117 ceph-mon[36730]: pgmap v269: 129 pgs: 129 active+clean; 60 MiB data, 880 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 10 KiB/s wr, 15 op/s 2024-04-16T17:15:25.055 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:15:24 smithi117 ceph-mon[36730]: scrub summary: idle 2024-04-16T17:15:25.055 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:15:24 smithi117 ceph-mon[36730]: scrub summary: idle 2024-04-16T17:15:25.055 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:15:24 smithi117 ceph-mon[36730]: scrub summary: idle 2024-04-16T17:15:26.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:15:26 smithi104 ceph-mon[36457]: pgmap v270: 129 pgs: 129 active+clean; 60 MiB data, 880 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 9.2 KiB/s wr, 14 op/s 2024-04-16T17:15:26.976 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:15:26 smithi045 ceph-mon[31676]: pgmap v270: 129 pgs: 129 active+clean; 60 MiB data, 880 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 9.2 KiB/s wr, 14 op/s 2024-04-16T17:15:27.054 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:15:26 smithi117 ceph-mon[36730]: pgmap v270: 129 pgs: 129 active+clean; 60 MiB data, 880 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 9.2 KiB/s wr, 14 op/s 2024-04-16T17:15:28.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:15:28 smithi104 ceph-mon[36457]: pgmap v271: 129 pgs: 129 active+clean; 60 MiB data, 880 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 8.4 KiB/s wr, 15 op/s 2024-04-16T17:15:28.976 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:15:28 smithi045 ceph-mon[31676]: pgmap v271: 129 pgs: 129 active+clean; 60 MiB data, 880 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 8.4 KiB/s wr, 15 op/s 2024-04-16T17:15:29.054 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:15:28 smithi117 ceph-mon[36730]: pgmap v271: 129 pgs: 129 active+clean; 60 MiB data, 880 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 8.4 KiB/s wr, 15 op/s 2024-04-16T17:15:30.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:15:30 smithi104 ceph-mon[36457]: pgmap v272: 129 pgs: 129 active+clean; 60 MiB data, 880 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 341 B/s wr, 5 op/s 2024-04-16T17:15:30.976 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:15:30 smithi045 ceph-mon[31676]: pgmap v272: 129 pgs: 129 active+clean; 60 MiB data, 880 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 341 B/s wr, 5 op/s 2024-04-16T17:15:31.054 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:15:30 smithi117 ceph-mon[36730]: pgmap v272: 129 pgs: 129 active+clean; 60 MiB data, 880 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 341 B/s wr, 5 op/s 2024-04-16T17:15:32.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:15:32 smithi104 ceph-mon[36457]: pgmap v273: 129 pgs: 129 active+clean; 60 MiB data, 880 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 341 B/s wr, 7 op/s 2024-04-16T17:15:32.976 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:15:32 smithi045 ceph-mon[31676]: pgmap v273: 129 pgs: 129 active+clean; 60 MiB data, 880 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 341 B/s wr, 7 op/s 2024-04-16T17:15:33.054 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:15:32 smithi117 ceph-mon[36730]: pgmap v273: 129 pgs: 129 active+clean; 60 MiB data, 880 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 341 B/s wr, 7 op/s 2024-04-16T17:15:34.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:15:34 smithi104 ceph-mon[36457]: pgmap v274: 129 pgs: 129 active+clean; 60 MiB data, 880 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T17:15:34.976 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:15:34 smithi045 ceph-mon[31676]: pgmap v274: 129 pgs: 129 active+clean; 60 MiB data, 880 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T17:15:35.054 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:15:34 smithi117 ceph-mon[36730]: pgmap v274: 129 pgs: 129 active+clean; 60 MiB data, 880 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T17:15:36.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:15:36 smithi104 ceph-mon[36457]: pgmap v275: 129 pgs: 129 active+clean; 60 MiB data, 880 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T17:15:36.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:15:36 smithi045 ceph-mon[31676]: pgmap v275: 129 pgs: 129 active+clean; 60 MiB data, 880 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T17:15:37.054 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:15:36 smithi117 ceph-mon[36730]: pgmap v275: 129 pgs: 129 active+clean; 60 MiB data, 880 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T17:15:38.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:15:38 smithi104 ceph-mon[36457]: pgmap v276: 129 pgs: 129 active+clean; 60 MiB data, 880 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-16T17:15:38.976 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:15:38 smithi045 ceph-mon[31676]: pgmap v276: 129 pgs: 129 active+clean; 60 MiB data, 880 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-16T17:15:39.054 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:15:38 smithi117 ceph-mon[36730]: pgmap v276: 129 pgs: 129 active+clean; 60 MiB data, 880 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-16T17:15:40.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:15:40 smithi104 ceph-mon[36457]: pgmap v277: 129 pgs: 129 active+clean; 60 MiB data, 880 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T17:15:40.976 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:15:40 smithi045 ceph-mon[31676]: pgmap v277: 129 pgs: 129 active+clean; 60 MiB data, 880 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T17:15:41.054 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:15:40 smithi117 ceph-mon[36730]: pgmap v277: 129 pgs: 129 active+clean; 60 MiB data, 880 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T17:15:42.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:15:42 smithi104 ceph-mon[36457]: pgmap v278: 129 pgs: 129 active+clean; 60 MiB data, 880 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-16T17:15:42.976 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:15:42 smithi045 ceph-mon[31676]: pgmap v278: 129 pgs: 129 active+clean; 60 MiB data, 880 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-16T17:15:43.054 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:15:42 smithi117 ceph-mon[36730]: pgmap v278: 129 pgs: 129 active+clean; 60 MiB data, 880 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-16T17:15:44.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:15:44 smithi104 ceph-mon[36457]: pgmap v279: 129 pgs: 129 active+clean; 60 MiB data, 880 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T17:15:44.976 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:15:44 smithi045 ceph-mon[31676]: pgmap v279: 129 pgs: 129 active+clean; 60 MiB data, 880 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T17:15:45.054 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:15:44 smithi117 ceph-mon[36730]: pgmap v279: 129 pgs: 129 active+clean; 60 MiB data, 880 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T17:15:46.438 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-16T17:15:46.441 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-16T17:15:46.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.664+0000 7f9082200640 1 -- 172.21.15.45:0/2655369042 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f907c1044b0 msgr2=0x7f907c104890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:15:46.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.664+0000 7f9082200640 1 --2- 172.21.15.45:0/2655369042 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f907c1044b0 0x7f907c104890 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f906c009900 tx=0x7f906c02bc30 comp rx=0 tx=0).stop 2024-04-16T17:15:46.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.664+0000 7f9082200640 1 -- 172.21.15.45:0/2655369042 shutdown_connections 2024-04-16T17:15:46.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.664+0000 7f9082200640 1 --2- 172.21.15.45:0/2655369042 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f907c109580 0x7f907c111150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:46.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.664+0000 7f9082200640 1 --2- 172.21.15.45:0/2655369042 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f907c104e60 0x7f907c108ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:46.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.664+0000 7f9082200640 1 --2- 172.21.15.45:0/2655369042 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f907c1044b0 0x7f907c104890 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:46.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.664+0000 7f9082200640 1 -- 172.21.15.45:0/2655369042 >> 172.21.15.45:0/2655369042 conn(0x7f907c0ffd90 msgr2=0x7f907c1021b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:15:46.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.664+0000 7f9082200640 1 -- 172.21.15.45:0/2655369042 shutdown_connections 2024-04-16T17:15:46.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.664+0000 7f9082200640 1 -- 172.21.15.45:0/2655369042 wait complete. 2024-04-16T17:15:46.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.665+0000 7f9082200640 1 Processor -- start 2024-04-16T17:15:46.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.665+0000 7f9082200640 1 -- start start 2024-04-16T17:15:46.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.665+0000 7f9082200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f907c1044b0 0x7f907c1a0880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:46.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.665+0000 7f9082200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f907c109580 0x7f907c1a0dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:46.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.665+0000 7f9082200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f907c19aa30 0x7f907c19aee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:46.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.665+0000 7f9082200640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f907c114690 con 0x7f907c19aa30 2024-04-16T17:15:46.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.665+0000 7f9082200640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f907c114510 con 0x7f907c1044b0 2024-04-16T17:15:46.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.665+0000 7f9082200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f907c114810 con 0x7f907c109580 2024-04-16T17:15:46.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.666+0000 7f907be00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f907c19aa30 0x7f907c19aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:46.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.666+0000 7f907b400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f907c1044b0 0x7f907c1a0880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:46.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.666+0000 7f907be00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f907c19aa30 0x7f907c19aee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:44120/0 (socket says 172.21.15.45:44120) 2024-04-16T17:15:46.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.666+0000 7f907be00640 1 -- 172.21.15.45:0/2562048678 learned_addr learned my addr 172.21.15.45:0/2562048678 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:15:46.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.666+0000 7f907aa00640 1 --2- 172.21.15.45:0/2562048678 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f907c109580 0x7f907c1a0dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:46.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.666+0000 7f907be00640 1 -- 172.21.15.45:0/2562048678 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f907c109580 msgr2=0x7f907c1a0dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:15:46.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.666+0000 7f907be00640 1 --2- 172.21.15.45:0/2562048678 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f907c109580 0x7f907c1a0dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:46.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.666+0000 7f907be00640 1 -- 172.21.15.45:0/2562048678 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f907c1044b0 msgr2=0x7f907c1a0880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:15:46.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.666+0000 7f907be00640 1 --2- 172.21.15.45:0/2562048678 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f907c1044b0 0x7f907c1a0880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:46.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.666+0000 7f907be00640 1 -- 172.21.15.45:0/2562048678 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f907c19b550 con 0x7f907c19aa30 2024-04-16T17:15:46.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.666+0000 7f907b400640 1 --2- 172.21.15.45:0/2562048678 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f907c1044b0 0x7f907c1a0880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:15:46.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.666+0000 7f907be00640 1 --2- 172.21.15.45:0/2562048678 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f907c19aa30 0x7f907c19aee0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f907400eb10 tx=0x7f907400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:15:46.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.667+0000 7f9073e00640 1 -- 172.21.15.45:0/2562048678 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9074004b20 con 0x7f907c19aa30 2024-04-16T17:15:46.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.667+0000 7f9073e00640 1 -- 172.21.15.45:0/2562048678 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9074004cc0 con 0x7f907c19aa30 2024-04-16T17:15:46.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.667+0000 7f9073e00640 1 -- 172.21.15.45:0/2562048678 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9074010760 con 0x7f907c19aa30 2024-04-16T17:15:46.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.667+0000 7f9082200640 1 -- 172.21.15.45:0/2562048678 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f907c19b840 con 0x7f907c19aa30 2024-04-16T17:15:46.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.667+0000 7f9082200640 1 -- 172.21.15.45:0/2562048678 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f907c1a7a40 con 0x7f907c19aa30 2024-04-16T17:15:46.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.670+0000 7f9084bfd740 1 Processor -- start 2024-04-16T17:15:46.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.670+0000 7f9073e00640 1 -- 172.21.15.45:0/2562048678 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f9074005020 con 0x7f907c19aa30 2024-04-16T17:15:46.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.670+0000 7f9084bfd740 1 -- start start 2024-04-16T17:15:46.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.671+0000 7f9084bfd740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d6b2b751b0 0x55d6b2b75590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:46.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.671+0000 7f9084bfd740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d6b2b75ad0 0x55d6b2b7e170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:46.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.671+0000 7f9084bfd740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d6b2b7e6b0 0x55d6b2b80aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:46.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.671+0000 7f9084bfd740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55d6b2adb120 con 0x55d6b2b751b0 2024-04-16T17:15:46.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.671+0000 7f9084bfd740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55d6b2a81340 con 0x55d6b2b7e6b0 2024-04-16T17:15:46.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.671+0000 7f9084bfd740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55d6b2ad70e0 con 0x55d6b2b75ad0 2024-04-16T17:15:46.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.671+0000 7f907aa00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d6b2b751b0 0x55d6b2b75590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:46.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.671+0000 7f907aa00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d6b2b751b0 0x55d6b2b75590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:44124/0 (socket says 172.21.15.45:44124) 2024-04-16T17:15:46.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.671+0000 7f907aa00640 1 -- 172.21.15.45:0/2302479878 learned_addr learned my addr 172.21.15.45:0/2302479878 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:15:46.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.671+0000 7f907b400640 1 --2- 172.21.15.45:0/2302479878 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d6b2b75ad0 0x55d6b2b7e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:46.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.671+0000 7f907aa00640 1 --2- 172.21.15.45:0/2302479878 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d6b2b7e6b0 0x55d6b2b80aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:46.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.671+0000 7f907b400640 1 -- 172.21.15.45:0/2302479878 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d6b2b7e6b0 msgr2=0x55d6b2b80aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:15:46.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.671+0000 7f907b400640 1 --2- 172.21.15.45:0/2302479878 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d6b2b7e6b0 0x55d6b2b80aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:46.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.671+0000 7f907b400640 1 -- 172.21.15.45:0/2302479878 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d6b2b751b0 msgr2=0x55d6b2b75590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:15:46.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.671+0000 7f907b400640 1 --2- 172.21.15.45:0/2302479878 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d6b2b751b0 0x55d6b2b75590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:46.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.671+0000 7f907b400640 1 -- 172.21.15.45:0/2302479878 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d6b28b77d0 con 0x55d6b2b75ad0 2024-04-16T17:15:46.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.671+0000 7f907aa00640 1 --2- 172.21.15.45:0/2302479878 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d6b2b7e6b0 0x55d6b2b80aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:15:46.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.672+0000 7f9073e00640 1 --2- 172.21.15.45:0/2562048678 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f90600a3da0 0x7f90600a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:46.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.672+0000 7f907b400640 1 --2- 172.21.15.45:0/2302479878 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d6b2b75ad0 0x55d6b2b7e170 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f906c02c170 tx=0x7f906c0095e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e5cd261d67d1993c server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:15:46.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.672+0000 7f9073e00640 1 -- 172.21.15.45:0/2562048678 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9496+0+0 (secure 0 0 0) 0x7f90740d12b0 con 0x7f907c19aa30 2024-04-16T17:15:46.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.672+0000 7f907aa00640 1 --2- 172.21.15.45:0/2562048678 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f90600a3da0 0x7f90600a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:46.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.672+0000 7f9067e00640 1 -- 172.21.15.45:0/2302479878 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f906c045070 con 0x55d6b2b75ad0 2024-04-16T17:15:46.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.672+0000 7f9067e00640 1 -- 172.21.15.45:0/2302479878 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f906c004a70 con 0x55d6b2b75ad0 2024-04-16T17:15:46.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.672+0000 7f9067e00640 1 -- 172.21.15.45:0/2302479878 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f906c005180 con 0x55d6b2b75ad0 2024-04-16T17:15:46.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.673+0000 7f9084bfd740 1 -- 172.21.15.45:0/2302479878 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d6b2b75ad0 msgr2=0x55d6b2b7e170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:15:46.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.673+0000 7f9084bfd740 1 --2- 172.21.15.45:0/2302479878 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d6b2b75ad0 0x55d6b2b7e170 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f906c02c170 tx=0x7f906c0095e0 comp rx=0 tx=0).stop 2024-04-16T17:15:46.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.673+0000 7f907aa00640 1 --2- 172.21.15.45:0/2562048678 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f90600a3da0 0x7f90600a6260 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f9068005e00 tx=0x7f906800d610 comp rx=0 tx=0).ready entity=mgr.25411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:15:46.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.673+0000 7f9084bfd740 1 -- 172.21.15.45:0/2302479878 shutdown_connections 2024-04-16T17:15:46.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.673+0000 7f9084bfd740 1 --2- 172.21.15.45:0/2302479878 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d6b2b75ad0 0x55d6b2b7e170 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:46.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.673+0000 7f9084bfd740 1 --2- 172.21.15.45:0/2302479878 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d6b2b7e6b0 0x55d6b2b80aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:46.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.673+0000 7f9084bfd740 1 --2- 172.21.15.45:0/2302479878 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d6b2b751b0 0x55d6b2b75590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:46.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.673+0000 7f9084bfd740 1 -- 172.21.15.45:0/2302479878 >> 172.21.15.45:0/2302479878 conn(0x55d6b2b6f8e0 msgr2=0x55d6b2b6fcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:15:46.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.673+0000 7f9084bfd740 1 -- 172.21.15.45:0/2302479878 shutdown_connections 2024-04-16T17:15:46.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.673+0000 7f9084bfd740 1 -- 172.21.15.45:0/2302479878 wait complete. 2024-04-16T17:15:46.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.674+0000 7f9084bfd740 1 -- start start 2024-04-16T17:15:46.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.674+0000 7f9084bfd740 1 Processor -- start 2024-04-16T17:15:46.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.674+0000 7f9084bfd740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d6b2b751b0 0x55d6b2b7b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:46.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.674+0000 7f9084bfd740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d6b2b75ad0 0x55d6b2b7bc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:46.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.674+0000 7f9084bfd740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d6b2b7e6b0 0x55d6b2b7c160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:46.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.674+0000 7f9084bfd740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55d6b2b81440 con 0x55d6b2b75ad0 2024-04-16T17:15:46.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.674+0000 7f9084bfd740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55d6b2a94710 con 0x55d6b2b7e6b0 2024-04-16T17:15:46.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.674+0000 7f9084bfd740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55d6b2b815c0 con 0x55d6b2b751b0 2024-04-16T17:15:46.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.674+0000 7f9084bfd740 10 client.0 _ll_register_callbacks cb 0x55d6b2a8faf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:15:46.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.675+0000 7f907b400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d6b2b751b0 0x55d6b2b7b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:46.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.675+0000 7f907aa00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d6b2b75ad0 0x55d6b2b7bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:46.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.675+0000 7f907b400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d6b2b751b0 0x55d6b2b7b6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:33934/0 (socket says 172.21.15.45:33934) 2024-04-16T17:15:46.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.675+0000 7f907b400640 1 -- 172.21.15.45:0/1663298130 learned_addr learned my addr 172.21.15.45:0/1663298130 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:15:46.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.675+0000 7f907be00640 1 --2- 172.21.15.45:0/1663298130 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d6b2b7e6b0 0x55d6b2b7c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:46.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.675+0000 7f907b400640 1 -- 172.21.15.45:0/1663298130 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d6b2b7e6b0 msgr2=0x55d6b2b7c160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:15:46.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.675+0000 7f907b400640 1 --2- 172.21.15.45:0/1663298130 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d6b2b7e6b0 0x55d6b2b7c160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:46.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.675+0000 7f907b400640 1 -- 172.21.15.45:0/1663298130 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d6b2b75ad0 msgr2=0x55d6b2b7bc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:15:46.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.675+0000 7f907b400640 1 --2- 172.21.15.45:0/1663298130 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d6b2b75ad0 0x55d6b2b7bc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:46.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.675+0000 7f907b400640 1 -- 172.21.15.45:0/1663298130 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d6b27ed3f0 con 0x55d6b2b751b0 2024-04-16T17:15:46.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.675+0000 7f907b400640 1 --2- 172.21.15.45:0/1663298130 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d6b2b751b0 0x55d6b2b7b6e0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f906c036040 tx=0x7f906c0053f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7c09551ca1427a9 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:15:46.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.675+0000 7f9064c00640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-04-16T17:15:46.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.675+0000 7f9064c00640 1 -- 172.21.15.45:0/1663298130 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f906c045040 con 0x55d6b2b751b0 2024-04-16T17:15:46.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.676+0000 7f9084bfd740 1 -- 172.21.15.45:0/1663298130 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f906c004a70 con 0x55d6b2b751b0 2024-04-16T17:15:46.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.676+0000 7f9064c00640 1 -- 172.21.15.45:0/1663298130 <== mon.1 v2:172.21.15.117:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9496+0+0 (secure 0 0 0) 0x7f906c04a650 con 0x55d6b2b751b0 2024-04-16T17:15:46.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.676+0000 7f9064c00640 1 -- 172.21.15.45:0/1663298130 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f906c05a050 con 0x55d6b2b751b0 2024-04-16T17:15:46.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.676+0000 7f9084bfd740 10 client.27874 fetch_fsmap learned FSMap version 43 2024-04-16T17:15:46.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.676+0000 7f9084bfd740 1 -- 172.21.15.45:0/1663298130 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55d6b27ad0f0 con 0x55d6b2b751b0 2024-04-16T17:15:46.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.676+0000 7f9064c00640 1 -- 172.21.15.45:0/1663298130 <== mon.1 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f906c004a70 con 0x55d6b2b751b0 2024-04-16T17:15:46.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.676+0000 7f9064c00640 1 -- 172.21.15.45:0/1663298130 <== mon.1 v2:172.21.15.117:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f906c03f4d0 con 0x55d6b2b751b0 2024-04-16T17:15:46.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.676+0000 7f9084bfd740 10 client.27874 fetch_fsmap finished waiting for FSMap version 43 2024-04-16T17:15:46.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.676+0000 7f9084bfd740 10 client.27874 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:15:46.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.677+0000 7f9084bfd740 20 client.27874 populate_metadata read hostname 'smithi045' 2024-04-16T17:15:46.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.677+0000 7f9084bfd740 1 --2- 172.21.15.45:0/1663298130 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55d6b2b8e7d0 0x55d6b2b90bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:46.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.677+0000 7f9084bfd740 4 client.27874 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:15:46.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.677+0000 7f9084bfd740 1 -- 172.21.15.45:0/1663298130 --> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d6b2ae8b40 con 0x55d6b2b8e7d0 2024-04-16T17:15:46.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.677+0000 7f907aa00640 1 --2- 172.21.15.45:0/1663298130 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55d6b2b8e7d0 0x55d6b2b90bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:46.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.677+0000 7f907aa00640 1 --2- 172.21.15.45:0/1663298130 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55d6b2b8e7d0 0x55d6b2b90bc0 crc :-1 s=READY pgs=233 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2e45b8b58dacf7ba server_cookie=865b02b268b27475 in_seq=0 out_seq=0 2024-04-16T17:15:46.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.677+0000 7f9064c00640 10 client.27874 ms_handle_connect on v2:172.21.15.104:6838/3660070592 2024-04-16T17:15:46.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.678+0000 7f9064c00640 1 -- 172.21.15.45:0/1663298130 <== mds.0 v2:172.21.15.104:6838/3660070592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d6b2ae8b40 con 0x55d6b2b8e7d0 2024-04-16T17:15:46.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.678+0000 7f9064c00640 10 client.27874 handle_command_reply: tid=0 2024-04-16T17:15:46.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.678+0000 7f9084bfd740 1 client.27874 shutdown 2024-04-16T17:15:46.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.679+0000 7f9084bfd740 1 -- 172.21.15.45:0/1663298130 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d6b2b751b0 msgr2=0x55d6b2b7b6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:15:46.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.679+0000 7f9084bfd740 1 --2- 172.21.15.45:0/1663298130 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d6b2b751b0 0x55d6b2b7b6e0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f906c036040 tx=0x7f906c0053f0 comp rx=0 tx=0).stop 2024-04-16T17:15:46.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.679+0000 7f9084bfd740 1 -- 172.21.15.45:0/1663298130 shutdown_connections 2024-04-16T17:15:46.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.679+0000 7f9084bfd740 1 --2- 172.21.15.45:0/1663298130 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d6b2b7e6b0 0x55d6b2b7c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:46.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.679+0000 7f9084bfd740 1 --2- 172.21.15.45:0/1663298130 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d6b2b75ad0 0x55d6b2b7bc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:46.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.679+0000 7f9084bfd740 1 --2- 172.21.15.45:0/1663298130 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55d6b2b8e7d0 0x55d6b2b90bc0 crc :-1 s=READY pgs=233 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:46.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.679+0000 7f9084bfd740 1 --2- 172.21.15.45:0/1663298130 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d6b2b751b0 0x55d6b2b7b6e0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:46.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.679+0000 7f9084bfd740 1 -- 172.21.15.45:0/1663298130 >> 172.21.15.45:0/1663298130 conn(0x55d6b2b6f8e0 msgr2=0x55d6b2b71630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:15:46.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.679+0000 7f9064c00640 0 client.27874 ms_handle_reset on v2:172.21.15.104:6838/3660070592 2024-04-16T17:15:46.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.679+0000 7f9084bfd740 1 -- 172.21.15.45:0/1663298130 shutdown_connections 2024-04-16T17:15:46.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.679+0000 7f9084bfd740 1 -- 172.21.15.45:0/1663298130 wait complete. 2024-04-16T17:15:46.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.679+0000 7f9084bfd740 20 client.27874 trim_cache size 0 max 16384 2024-04-16T17:15:46.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.698+0000 7f9084bfd740 1 Processor -- start 2024-04-16T17:15:46.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.698+0000 7f9084bfd740 1 -- start start 2024-04-16T17:15:46.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.698+0000 7f9084bfd740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d6b2b751b0 0x55d6b2b7de10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:46.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.698+0000 7f9084bfd740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d6b2b75ad0 0x55d6b2b8a300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:46.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.698+0000 7f9084bfd740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d6b2b7e6b0 0x55d6b2b6e750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:46.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.698+0000 7f9084bfd740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f907c1b4000 con 0x55d6b2b7e6b0 2024-04-16T17:15:46.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.698+0000 7f9084bfd740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f907c1b18d0 con 0x55d6b2b75ad0 2024-04-16T17:15:46.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.698+0000 7f9084bfd740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f907c1af740 con 0x55d6b2b751b0 2024-04-16T17:15:46.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.698+0000 7f907be00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d6b2b7e6b0 0x55d6b2b6e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:46.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.698+0000 7f907be00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d6b2b7e6b0 0x55d6b2b6e750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:44140/0 (socket says 172.21.15.45:44140) 2024-04-16T17:15:46.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.698+0000 7f907b400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d6b2b751b0 0x55d6b2b7de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:46.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.698+0000 7f907be00640 1 -- 172.21.15.45:0/195348925 learned_addr learned my addr 172.21.15.45:0/195348925 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:15:46.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.698+0000 7f907aa00640 1 --2- 172.21.15.45:0/195348925 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d6b2b75ad0 0x55d6b2b8a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:46.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.699+0000 7f907b400640 1 -- 172.21.15.45:0/195348925 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d6b2b75ad0 msgr2=0x55d6b2b8a300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:15:46.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.699+0000 7f907b400640 1 --2- 172.21.15.45:0/195348925 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d6b2b75ad0 0x55d6b2b8a300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:46.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.699+0000 7f907b400640 1 -- 172.21.15.45:0/195348925 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d6b2b7e6b0 msgr2=0x55d6b2b6e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:15:46.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.699+0000 7f907b400640 1 --2- 172.21.15.45:0/195348925 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d6b2b7e6b0 0x55d6b2b6e750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:46.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.699+0000 7f907b400640 1 -- 172.21.15.45:0/195348925 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d6b2b75900 con 0x55d6b2b751b0 2024-04-16T17:15:46.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.699+0000 7f907be00640 1 --2- 172.21.15.45:0/195348925 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d6b2b7e6b0 0x55d6b2b6e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:15:46.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.699+0000 7f907b400640 1 --2- 172.21.15.45:0/195348925 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d6b2b751b0 0x55d6b2b7de10 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f906c03f870 tx=0x7f906c041040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e5092b7765e83277 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:15:46.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.699+0000 7f9067400640 1 -- 172.21.15.45:0/195348925 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f906c000e60 con 0x55d6b2b751b0 2024-04-16T17:15:46.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.699+0000 7f9067400640 1 -- 172.21.15.45:0/195348925 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f906c004a70 con 0x55d6b2b751b0 2024-04-16T17:15:46.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.699+0000 7f9067400640 1 -- 172.21.15.45:0/195348925 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f906c045070 con 0x55d6b2b751b0 2024-04-16T17:15:46.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.700+0000 7f9084bfd740 1 -- 172.21.15.45:0/195348925 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d6b2b751b0 msgr2=0x55d6b2b7de10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:15:46.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.700+0000 7f9084bfd740 1 --2- 172.21.15.45:0/195348925 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d6b2b751b0 0x55d6b2b7de10 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f906c03f870 tx=0x7f906c041040 comp rx=0 tx=0).stop 2024-04-16T17:15:46.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.700+0000 7f9084bfd740 1 -- 172.21.15.45:0/195348925 shutdown_connections 2024-04-16T17:15:46.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.700+0000 7f9084bfd740 1 --2- 172.21.15.45:0/195348925 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d6b2b7e6b0 0x55d6b2b6e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:46.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.700+0000 7f9084bfd740 1 --2- 172.21.15.45:0/195348925 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d6b2b75ad0 0x55d6b2b8a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:46.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.700+0000 7f9084bfd740 1 --2- 172.21.15.45:0/195348925 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d6b2b751b0 0x55d6b2b7de10 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:46.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.700+0000 7f9084bfd740 1 -- 172.21.15.45:0/195348925 >> 172.21.15.45:0/195348925 conn(0x55d6b2b6f8e0 msgr2=0x55d6b2b6e040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:15:46.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.700+0000 7f9084bfd740 1 -- 172.21.15.45:0/195348925 shutdown_connections 2024-04-16T17:15:46.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.700+0000 7f9084bfd740 1 -- 172.21.15.45:0/195348925 wait complete. 2024-04-16T17:15:46.708 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.701+0000 7f9084bfd740 1 -- start start 2024-04-16T17:15:46.708 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.701+0000 7f9084bfd740 1 Processor -- start 2024-04-16T17:15:46.708 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.701+0000 7f9084bfd740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d6b2b751b0 0x55d6b2b78990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:46.708 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.701+0000 7f9084bfd740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d6b2b75ad0 0x55d6b2b78ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:46.708 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.701+0000 7f9084bfd740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d6b2b7e6b0 0x55d6b2bfc300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:46.708 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.701+0000 7f9084bfd740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f907c1afd90 con 0x55d6b2b751b0 2024-04-16T17:15:46.708 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.701+0000 7f9084bfd740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f907c1b1200 con 0x55d6b2b75ad0 2024-04-16T17:15:46.708 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.701+0000 7f9084bfd740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f907c1b0460 con 0x55d6b2b7e6b0 2024-04-16T17:15:46.708 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.701+0000 7f9084bfd740 10 client.0 _ll_register_callbacks cb 0x7f906800d470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:15:46.708 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.702+0000 7f907b400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d6b2b751b0 0x55d6b2b78990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:46.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.702+0000 7f907aa00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d6b2b75ad0 0x55d6b2b78ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:46.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.702+0000 7f907b400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d6b2b751b0 0x55d6b2b78990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:44148/0 (socket says 172.21.15.45:44148) 2024-04-16T17:15:46.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.702+0000 7f907b400640 1 -- 172.21.15.45:0/1892891836 learned_addr learned my addr 172.21.15.45:0/1892891836 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:15:46.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.702+0000 7f907be00640 1 --2- 172.21.15.45:0/1892891836 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d6b2b7e6b0 0x55d6b2bfc300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:46.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.702+0000 7f907aa00640 1 -- 172.21.15.45:0/1892891836 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d6b2b7e6b0 msgr2=0x55d6b2bfc300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:15:46.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.702+0000 7f907aa00640 1 --2- 172.21.15.45:0/1892891836 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d6b2b7e6b0 0x55d6b2bfc300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:46.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.702+0000 7f907aa00640 1 -- 172.21.15.45:0/1892891836 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d6b2b751b0 msgr2=0x55d6b2b78990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:15:46.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.702+0000 7f907aa00640 1 --2- 172.21.15.45:0/1892891836 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d6b2b751b0 0x55d6b2b78990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:46.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.702+0000 7f907aa00640 1 -- 172.21.15.45:0/1892891836 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d6b2bfc840 con 0x55d6b2b75ad0 2024-04-16T17:15:46.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.702+0000 7f907be00640 1 --2- 172.21.15.45:0/1892891836 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d6b2b7e6b0 0x55d6b2bfc300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:15:46.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.702+0000 7f907b400640 1 --2- 172.21.15.45:0/1892891836 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d6b2b751b0 0x55d6b2b78990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:15:46.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.702+0000 7f907aa00640 1 --2- 172.21.15.45:0/1892891836 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d6b2b75ad0 0x55d6b2b78ed0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x55d6b2b91220 tx=0x7f9068004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2cd31acf7bbe9fc2 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:15:46.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.702+0000 7f9064c00640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-04-16T17:15:46.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.703+0000 7f9064c00640 1 -- 172.21.15.45:0/1892891836 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9068004220 con 0x55d6b2b75ad0 2024-04-16T17:15:46.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.703+0000 7f9064c00640 1 -- 172.21.15.45:0/1892891836 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9496+0+0 (secure 0 0 0) 0x7f90680043c0 con 0x55d6b2b75ad0 2024-04-16T17:15:46.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.703+0000 7f9084bfd740 1 -- 172.21.15.45:0/1892891836 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f906c004a70 con 0x55d6b2b75ad0 2024-04-16T17:15:46.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.703+0000 7f9064c00640 1 -- 172.21.15.45:0/1892891836 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f9068020870 con 0x55d6b2b75ad0 2024-04-16T17:15:46.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.703+0000 7f9084bfd740 10 client.27929 fetch_fsmap learned FSMap version 43 2024-04-16T17:15:46.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.703+0000 7f9084bfd740 1 -- 172.21.15.45:0/1892891836 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55d6b2bfda50 con 0x55d6b2b75ad0 2024-04-16T17:15:46.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.703+0000 7f9064c00640 1 -- 172.21.15.45:0/1892891836 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f906c004a70 con 0x55d6b2b75ad0 2024-04-16T17:15:46.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.703+0000 7f9064c00640 1 -- 172.21.15.45:0/1892891836 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f90680035f0 con 0x55d6b2b75ad0 2024-04-16T17:15:46.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.703+0000 7f9084bfd740 10 client.27929 fetch_fsmap finished waiting for FSMap version 43 2024-04-16T17:15:46.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.703+0000 7f9084bfd740 10 client.27929 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:15:46.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.703+0000 7f9084bfd740 20 client.27929 populate_metadata read hostname 'smithi045' 2024-04-16T17:15:46.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.703+0000 7f9084bfd740 1 --2- 172.21.15.45:0/1892891836 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55d6b2bfe1a0 0x55d6b2c005c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:46.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.704+0000 7f9084bfd740 4 client.27929 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-16T17:15:46.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.704+0000 7f9084bfd740 1 -- 172.21.15.45:0/1892891836 --> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55d6b2ae29b0 con 0x55d6b2bfe1a0 2024-04-16T17:15:46.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.704+0000 7f907b400640 1 --2- 172.21.15.45:0/1892891836 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55d6b2bfe1a0 0x55d6b2c005c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:46.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.704+0000 7f907b400640 1 --2- 172.21.15.45:0/1892891836 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55d6b2bfe1a0 0x55d6b2c005c0 crc :-1 s=READY pgs=234 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a67b0383925e0678 server_cookie=5e96a1d86c5df9b4 in_seq=0 out_seq=0 2024-04-16T17:15:46.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.704+0000 7f9064c00640 10 client.27929 ms_handle_connect on v2:172.21.15.104:6838/3660070592 2024-04-16T17:15:46.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.704+0000 7f9064c00640 1 -- 172.21.15.45:0/1892891836 <== mds.0 v2:172.21.15.104:6838/3660070592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55d6b2ae29b0 con 0x55d6b2bfe1a0 2024-04-16T17:15:46.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.704+0000 7f9064c00640 10 client.27929 handle_command_reply: tid=0 2024-04-16T17:15:46.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.705+0000 7f9084bfd740 1 client.27929 shutdown 2024-04-16T17:15:46.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.705+0000 7f9084bfd740 1 -- 172.21.15.45:0/1892891836 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d6b2b75ad0 msgr2=0x55d6b2b78ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:15:46.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.705+0000 7f9084bfd740 1 --2- 172.21.15.45:0/1892891836 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d6b2b75ad0 0x55d6b2b78ed0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x55d6b2b91220 tx=0x7f9068004730 comp rx=0 tx=0).stop 2024-04-16T17:15:46.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.705+0000 7f9084bfd740 1 -- 172.21.15.45:0/1892891836 shutdown_connections 2024-04-16T17:15:46.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.705+0000 7f9084bfd740 1 --2- 172.21.15.45:0/1892891836 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55d6b2bfe1a0 0x55d6b2c005c0 crc :-1 s=READY pgs=234 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:46.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.705+0000 7f9084bfd740 1 --2- 172.21.15.45:0/1892891836 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d6b2b7e6b0 0x55d6b2bfc300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:46.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.705+0000 7f9084bfd740 1 --2- 172.21.15.45:0/1892891836 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d6b2b75ad0 0x55d6b2b78ed0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:46.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.705+0000 7f9064c00640 0 client.27929 ms_handle_reset on v2:172.21.15.104:6838/3660070592 2024-04-16T17:15:46.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.705+0000 7f9084bfd740 1 --2- 172.21.15.45:0/1892891836 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d6b2b751b0 0x55d6b2b78990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:46.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.705+0000 7f9084bfd740 1 -- 172.21.15.45:0/1892891836 >> 172.21.15.45:0/1892891836 conn(0x55d6b2b6f8e0 msgr2=0x55d6b2b71550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:15:46.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.705+0000 7f9084bfd740 1 -- 172.21.15.45:0/1892891836 shutdown_connections 2024-04-16T17:15:46.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.705+0000 7f9084bfd740 1 -- 172.21.15.45:0/1892891836 wait complete. 2024-04-16T17:15:46.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.705+0000 7f9084bfd740 20 client.27929 trim_cache size 0 max 16384 2024-04-16T17:15:46.716 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-16T17:15:46.716 INFO:teuthology.orchestra.run.smithi045.stdout: "status": "no active scrubs running", 2024-04-16T17:15:46.716 INFO:teuthology.orchestra.run.smithi045.stdout: "scrubs": {} 2024-04-16T17:15:46.716 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-16T17:15:46.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.706+0000 7f9082200640 1 -- 172.21.15.45:0/2562048678 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f90600a3da0 msgr2=0x7f90600a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:15:46.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.706+0000 7f9082200640 1 --2- 172.21.15.45:0/2562048678 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f90600a3da0 0x7f90600a6260 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f9068005e00 tx=0x7f906800d610 comp rx=0 tx=0).stop 2024-04-16T17:15:46.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.706+0000 7f9082200640 1 -- 172.21.15.45:0/2562048678 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f907c19aa30 msgr2=0x7f907c19aee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:15:46.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.706+0000 7f9082200640 1 --2- 172.21.15.45:0/2562048678 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f907c19aa30 0x7f907c19aee0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f907400eb10 tx=0x7f907400c2d0 comp rx=0 tx=0).stop 2024-04-16T17:15:46.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.706+0000 7f9082200640 1 -- 172.21.15.45:0/2562048678 shutdown_connections 2024-04-16T17:15:46.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.706+0000 7f9082200640 1 --2- 172.21.15.45:0/2562048678 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f90600a3da0 0x7f90600a6260 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:46.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.706+0000 7f9082200640 1 --2- 172.21.15.45:0/2562048678 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f907c109580 0x7f907c1a0dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:46.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.706+0000 7f9082200640 1 --2- 172.21.15.45:0/2562048678 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f907c19aa30 0x7f907c19aee0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:46.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.706+0000 7f9082200640 1 --2- 172.21.15.45:0/2562048678 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f907c1044b0 0x7f907c1a0880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:46.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.706+0000 7f9082200640 1 -- 172.21.15.45:0/2562048678 >> 172.21.15.45:0/2562048678 conn(0x7f907c0ffd90 msgr2=0x7f907c1020e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:15:46.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.706+0000 7f9082200640 1 -- 172.21.15.45:0/2562048678 shutdown_connections 2024-04-16T17:15:46.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.706+0000 7f9082200640 1 -- 172.21.15.45:0/2562048678 wait complete. 2024-04-16T17:15:46.724 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-16T17:15:46.724 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-16T17:15:46.797 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:15:46 smithi045 ceph-mon[31676]: pgmap v280: 129 pgs: 129 active+clean; 60 MiB data, 880 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T17:15:46.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:15:46 smithi104 ceph-mon[36457]: pgmap v280: 129 pgs: 129 active+clean; 60 MiB data, 880 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T17:15:46.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.931+0000 7f0f15a00640 1 -- 172.21.15.45:0/80391330 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0f10102bf0 msgr2=0x7f0f10106c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:15:46.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.931+0000 7f0f15a00640 1 --2- 172.21.15.45:0/80391330 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0f10102bf0 0x7f0f10106c30 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f0efc009900 tx=0x7f0efc02bc50 comp rx=0 tx=0).stop 2024-04-16T17:15:46.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.931+0000 7f0f15a00640 1 -- 172.21.15.45:0/80391330 shutdown_connections 2024-04-16T17:15:46.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.931+0000 7f0f15a00640 1 --2- 172.21.15.45:0/80391330 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0f10107310 0x7f0f1010eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:46.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.931+0000 7f0f15a00640 1 --2- 172.21.15.45:0/80391330 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0f10102bf0 0x7f0f10106c30 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:46.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.931+0000 7f0f15a00640 1 --2- 172.21.15.45:0/80391330 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0f10102240 0x7f0f10102620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:46.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.931+0000 7f0f15a00640 1 -- 172.21.15.45:0/80391330 >> 172.21.15.45:0/80391330 conn(0x7f0f100793c0 msgr2=0x7f0f100797d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:15:46.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.931+0000 7f0f15a00640 1 -- 172.21.15.45:0/80391330 shutdown_connections 2024-04-16T17:15:46.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.931+0000 7f0f15a00640 1 -- 172.21.15.45:0/80391330 wait complete. 2024-04-16T17:15:46.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.932+0000 7f0f15a00640 1 Processor -- start 2024-04-16T17:15:46.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.933+0000 7f0f15a00640 1 -- start start 2024-04-16T17:15:46.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.933+0000 7f0f15a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0f10102240 0x7f0f101a07c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:46.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.933+0000 7f0f15a00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0f10107310 0x7f0f101a0d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:46.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.933+0000 7f0f15a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0f1019a970 0x7f0f1019ae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:46.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.933+0000 7f0f15a00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f0f101123d0 con 0x7f0f10102240 2024-04-16T17:15:46.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.933+0000 7f0f15a00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f0f10112250 con 0x7f0f10107310 2024-04-16T17:15:46.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.933+0000 7f0f15a00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f0f10112550 con 0x7f0f1019a970 2024-04-16T17:15:46.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.933+0000 7f0f0ea00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0f10102240 0x7f0f101a07c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:46.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.934+0000 7f0f0ea00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0f10102240 0x7f0f101a07c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:44174/0 (socket says 172.21.15.45:44174) 2024-04-16T17:15:46.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.934+0000 7f0f0ea00640 1 -- 172.21.15.45:0/2733113845 learned_addr learned my addr 172.21.15.45:0/2733113845 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:15:46.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.934+0000 7f0f0f400640 1 --2- 172.21.15.45:0/2733113845 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0f1019a970 0x7f0f1019ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:46.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.934+0000 7f0f0e000640 1 --2- 172.21.15.45:0/2733113845 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0f10107310 0x7f0f101a0d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:46.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.934+0000 7f0f0ea00640 1 -- 172.21.15.45:0/2733113845 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0f1019a970 msgr2=0x7f0f1019ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:15:46.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.934+0000 7f0f0ea00640 1 --2- 172.21.15.45:0/2733113845 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0f1019a970 0x7f0f1019ae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:46.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.934+0000 7f0f0ea00640 1 -- 172.21.15.45:0/2733113845 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0f10107310 msgr2=0x7f0f101a0d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:15:46.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.934+0000 7f0f0ea00640 1 --2- 172.21.15.45:0/2733113845 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0f10107310 0x7f0f101a0d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:46.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.934+0000 7f0f0ea00640 1 -- 172.21.15.45:0/2733113845 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f1019b490 con 0x7f0f10102240 2024-04-16T17:15:46.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.934+0000 7f0f0ea00640 1 --2- 172.21.15.45:0/2733113845 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0f10102240 0x7f0f101a07c0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f0f0000ea60 tx=0x7f0f0000ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:15:46.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.935+0000 7f0f07400640 1 -- 172.21.15.45:0/2733113845 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f0000cd20 con 0x7f0f10102240 2024-04-16T17:15:46.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.935+0000 7f0f07400640 1 -- 172.21.15.45:0/2733113845 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0f000035f0 con 0x7f0f10102240 2024-04-16T17:15:46.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.935+0000 7f0f07400640 1 -- 172.21.15.45:0/2733113845 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f00010690 con 0x7f0f10102240 2024-04-16T17:15:46.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.935+0000 7f0f15a00640 1 -- 172.21.15.45:0/2733113845 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f1019b780 con 0x7f0f10102240 2024-04-16T17:15:46.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.935+0000 7f0f15a00640 1 -- 172.21.15.45:0/2733113845 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f101a79d0 con 0x7f0f10102240 2024-04-16T17:15:46.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.936+0000 7f0f15a00640 1 -- 172.21.15.45:0/2733113845 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f10069270 con 0x7f0f10102240 2024-04-16T17:15:46.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.938+0000 7f0f07400640 1 -- 172.21.15.45:0/2733113845 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f0f000037a0 con 0x7f0f10102240 2024-04-16T17:15:46.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.940+0000 7f0f07400640 1 --2- 172.21.15.45:0/2733113845 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f0ef40a3d20 0x7f0ef40a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:46.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.941+0000 7f0f07400640 1 -- 172.21.15.45:0/2733113845 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9496+0+0 (secure 0 0 0) 0x7f0f000d0f40 con 0x7f0f10102240 2024-04-16T17:15:46.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.941+0000 7f0f0e000640 1 --2- 172.21.15.45:0/2733113845 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f0ef40a3d20 0x7f0ef40a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:46.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.941+0000 7f0f0e000640 1 --2- 172.21.15.45:0/2733113845 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f0ef40a3d20 0x7f0ef40a61e0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f0efc02c260 tx=0x7f0efc000bc0 comp rx=0 tx=0).ready entity=mgr.25411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:15:46.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:46.944+0000 7f0f07400640 1 -- 172.21.15.45:0/2733113845 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0f0009d1b0 con 0x7f0f10102240 2024-04-16T17:15:47.054 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:15:46 smithi117 ceph-mon[36730]: pgmap v280: 129 pgs: 129 active+clean; 60 MiB data, 880 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T17:15:47.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.184+0000 7f0f15a00640 1 -- 172.21.15.45:0/2733113845 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0f1019bea0 con 0x7f0f10102240 2024-04-16T17:15:47.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.186+0000 7f0f07400640 1 -- 172.21.15.45:0/2733113845 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) ==== 94+0+11890 (secure 0 0 0) 0x7f0f0009cab0 con 0x7f0f10102240 2024-04-16T17:15:47.187 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-16T17:15:47.187 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":43,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/1748842443","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":1748842443},{"type":"v1","addr":"172.21.15.45:6835","nonce":1748842443}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6839/1308627051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":1308627051},{"type":"v1","addr":"172.21.15.45:6839","nonce":1308627051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/1768242489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1768242489},{"type":"v1","addr":"172.21.15.117:6833","nonce":1768242489}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6833/3558902585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":3558902585},{"type":"v1","addr":"172.21.15.104:6833","nonce":3558902585}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24478,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/1308142903","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":1308142903},{"type":"v1","addr":"172.21.15.45:6837","nonce":1308142903}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24491,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6835/3430679937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":3430679937},{"type":"v1","addr":"172.21.15.104:6835","nonce":3430679937}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":43,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-16T17:03:24.663611+0000","modified":"2024-04-16T17:14:55.721690+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24581,"mds_1":24569,"mds_2":24532},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24499":{"gid":24499,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/2224123841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2224123841},{"type":"v1","addr":"172.21.15.117:6835","nonce":2224123841}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6837/2471356815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":2471356815},{"type":"v1","addr":"172.21.15.117:6837","nonce":2471356815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24532":{"gid":24532,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":13,"addr":"172.21.15.117:6839/2378675949","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":2378675949},{"type":"v1","addr":"172.21.15.117:6839","nonce":2378675949}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24569":{"gid":24569,"name":"a","rank":1,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.45:6841/3755310822","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6840","nonce":3755310822},{"type":"v1","addr":"172.21.15.45:6841","nonce":3755310822}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24581":{"gid":24581,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.104:6839/3660070592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6838","nonce":3660070592},{"type":"v1","addr":"172.21.15.104:6839","nonce":3660070592}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_27569":{"gid":27569,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6837/1833645246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":1833645246},{"type":"v1","addr":"172.21.15.104:6837","nonce":1833645246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24581,"qdb_cluster":[24569,24581,24532]},"id":1}]} 2024-04-16T17:15:47.187 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 43 2024-04-16T17:15:47.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.188+0000 7f0f15a00640 1 -- 172.21.15.45:0/2733113845 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f0ef40a3d20 msgr2=0x7f0ef40a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:15:47.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.188+0000 7f0f15a00640 1 --2- 172.21.15.45:0/2733113845 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f0ef40a3d20 0x7f0ef40a61e0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f0efc02c260 tx=0x7f0efc000bc0 comp rx=0 tx=0).stop 2024-04-16T17:15:47.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.188+0000 7f0f15a00640 1 -- 172.21.15.45:0/2733113845 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0f10102240 msgr2=0x7f0f101a07c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:15:47.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.188+0000 7f0f15a00640 1 --2- 172.21.15.45:0/2733113845 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0f10102240 0x7f0f101a07c0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f0f0000ea60 tx=0x7f0f0000ee60 comp rx=0 tx=0).stop 2024-04-16T17:15:47.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.189+0000 7f0f15a00640 1 -- 172.21.15.45:0/2733113845 shutdown_connections 2024-04-16T17:15:47.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.189+0000 7f0f15a00640 1 --2- 172.21.15.45:0/2733113845 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f0ef40a3d20 0x7f0ef40a61e0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:47.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.189+0000 7f0f15a00640 1 --2- 172.21.15.45:0/2733113845 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0f1019a970 0x7f0f1019ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:47.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.189+0000 7f0f15a00640 1 --2- 172.21.15.45:0/2733113845 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0f10107310 0x7f0f101a0d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:47.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.189+0000 7f0f15a00640 1 --2- 172.21.15.45:0/2733113845 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0f10102240 0x7f0f101a07c0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:47.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.189+0000 7f0f15a00640 1 -- 172.21.15.45:0/2733113845 >> 172.21.15.45:0/2733113845 conn(0x7f0f100793c0 msgr2=0x7f0f100ffec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:15:47.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.189+0000 7f0f15a00640 1 -- 172.21.15.45:0/2733113845 shutdown_connections 2024-04-16T17:15:47.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.189+0000 7f0f15a00640 1 -- 172.21.15.45:0/2733113845 wait complete. 2024-04-16T17:15:47.207 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-16T17:15:47.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.421+0000 7f47ad400640 1 -- 172.21.15.45:0/2094943302 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f47a8103060 msgr2=0x7f47a8103440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:15:47.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.421+0000 7f47ad400640 1 --2- 172.21.15.45:0/2094943302 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f47a8103060 0x7f47a8103440 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f4798009900 tx=0x7f479802bcc0 comp rx=0 tx=0).stop 2024-04-16T17:15:47.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.421+0000 7f47ad400640 1 -- 172.21.15.45:0/2094943302 shutdown_connections 2024-04-16T17:15:47.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.421+0000 7f47ad400640 1 --2- 172.21.15.45:0/2094943302 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f47a810c480 0x7f47a810e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:47.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.421+0000 7f47ad400640 1 --2- 172.21.15.45:0/2094943302 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f47a8103980 0x7f47a810be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:47.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.421+0000 7f47ad400640 1 --2- 172.21.15.45:0/2094943302 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f47a8103060 0x7f47a8103440 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:47.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.421+0000 7f47ad400640 1 -- 172.21.15.45:0/2094943302 >> 172.21.15.45:0/2094943302 conn(0x7f47a80fc880 msgr2=0x7f47a80feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:15:47.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.421+0000 7f47ad400640 1 -- 172.21.15.45:0/2094943302 shutdown_connections 2024-04-16T17:15:47.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.421+0000 7f47ad400640 1 -- 172.21.15.45:0/2094943302 wait complete. 2024-04-16T17:15:47.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.422+0000 7f47ad400640 1 Processor -- start 2024-04-16T17:15:47.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.422+0000 7f47ad400640 1 -- start start 2024-04-16T17:15:47.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.422+0000 7f47ad400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f47a8103060 0x7f47a81a06b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:47.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.423+0000 7f47ad400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f47a8103980 0x7f47a81a0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:47.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.423+0000 7f47ad400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f47a810c480 0x7f47a819a950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:47.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.423+0000 7f47ad400640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f47a81122b0 con 0x7f47a8103060 2024-04-16T17:15:47.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.423+0000 7f47ad400640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f47a8112130 con 0x7f47a8103980 2024-04-16T17:15:47.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.423+0000 7f47ad400640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f47a8112430 con 0x7f47a810c480 2024-04-16T17:15:47.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.423+0000 7f47a7e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f47a8103060 0x7f47a81a06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:47.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.423+0000 7f47a7e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f47a8103060 0x7f47a81a06b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:44190/0 (socket says 172.21.15.45:44190) 2024-04-16T17:15:47.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.423+0000 7f47a7e00640 1 -- 172.21.15.45:0/283432570 learned_addr learned my addr 172.21.15.45:0/283432570 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:15:47.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.423+0000 7f47a7400640 1 --2- 172.21.15.45:0/283432570 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f47a8103980 0x7f47a81a0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:47.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.423+0000 7f47aca00640 1 --2- 172.21.15.45:0/283432570 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f47a810c480 0x7f47a819a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:47.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.423+0000 7f47a7e00640 1 -- 172.21.15.45:0/283432570 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f47a810c480 msgr2=0x7f47a819a950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:15:47.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.423+0000 7f47a7e00640 1 --2- 172.21.15.45:0/283432570 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f47a810c480 0x7f47a819a950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:47.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.423+0000 7f47a7e00640 1 -- 172.21.15.45:0/283432570 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f47a8103980 msgr2=0x7f47a81a0bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:15:47.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.423+0000 7f47a7e00640 1 --2- 172.21.15.45:0/283432570 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f47a8103980 0x7f47a81a0bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:47.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.423+0000 7f47a7e00640 1 -- 172.21.15.45:0/283432570 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47a819b0d0 con 0x7f47a8103060 2024-04-16T17:15:47.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.423+0000 7f47aca00640 1 --2- 172.21.15.45:0/283432570 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f47a810c480 0x7f47a819a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:15:47.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.423+0000 7f47a7400640 1 --2- 172.21.15.45:0/283432570 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f47a8103980 0x7f47a81a0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:15:47.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.424+0000 7f47a7e00640 1 --2- 172.21.15.45:0/283432570 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f47a8103060 0x7f47a81a06b0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f4798000c00 tx=0x7f4798004360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:15:47.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.424+0000 7f47a4c00640 1 -- 172.21.15.45:0/283432570 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4798042020 con 0x7f47a8103060 2024-04-16T17:15:47.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.424+0000 7f47a4c00640 1 -- 172.21.15.45:0/283432570 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4798005000 con 0x7f47a8103060 2024-04-16T17:15:47.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.424+0000 7f47ad400640 1 -- 172.21.15.45:0/283432570 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47a819b360 con 0x7f47a8103060 2024-04-16T17:15:47.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.424+0000 7f47a4c00640 1 -- 172.21.15.45:0/283432570 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4798003b20 con 0x7f47a8103060 2024-04-16T17:15:47.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.424+0000 7f47ad400640 1 -- 172.21.15.45:0/283432570 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47a81a7570 con 0x7f47a8103060 2024-04-16T17:15:47.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.427+0000 7f47b1792740 1 Processor -- start 2024-04-16T17:15:47.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.427+0000 7f47a4c00640 1 -- 172.21.15.45:0/283432570 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f4798045020 con 0x7f47a8103060 2024-04-16T17:15:47.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.428+0000 7f47b1792740 1 -- start start 2024-04-16T17:15:47.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.428+0000 7f47b1792740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d2f51d2f10 0x55d2f51d32f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:47.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.428+0000 7f47b1792740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d2f51d3830 0x55d2f51dbed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:47.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.428+0000 7f47b1792740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d2f51dc410 0x55d2f51de800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:47.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.428+0000 7f47b1792740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55d2f5138020 con 0x55d2f51d2f10 2024-04-16T17:15:47.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.428+0000 7f47b1792740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55d2f50de460 con 0x55d2f51d3830 2024-04-16T17:15:47.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.428+0000 7f47b1792740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55d2f5133fe0 con 0x55d2f51dc410 2024-04-16T17:15:47.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.428+0000 7f47aca00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d2f51d2f10 0x55d2f51d32f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:47.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.428+0000 7f47a7e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d2f51d3830 0x55d2f51dbed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:47.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.428+0000 7f47a7400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d2f51dc410 0x55d2f51de800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:47.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.428+0000 7f47a7e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d2f51d3830 0x55d2f51dbed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:44566/0 (socket says 172.21.15.45:44566) 2024-04-16T17:15:47.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.428+0000 7f47a7e00640 1 -- 172.21.15.45:0/1146691522 learned_addr learned my addr 172.21.15.45:0/1146691522 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:15:47.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.428+0000 7f47aca00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d2f51d2f10 0x55d2f51d32f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:44196/0 (socket says 172.21.15.45:44196) 2024-04-16T17:15:47.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.429+0000 7f47a7400640 1 -- 172.21.15.45:0/1146691522 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d2f51d3830 msgr2=0x55d2f51dbed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:15:47.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.429+0000 7f47a7400640 1 --2- 172.21.15.45:0/1146691522 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d2f51d3830 0x55d2f51dbed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:47.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.429+0000 7f47a7400640 1 -- 172.21.15.45:0/1146691522 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d2f51d2f10 msgr2=0x55d2f51d32f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:15:47.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.429+0000 7f47a7400640 1 --2- 172.21.15.45:0/1146691522 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d2f51d2f10 0x55d2f51d32f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:47.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.429+0000 7f47a7400640 1 -- 172.21.15.45:0/1146691522 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d2f50e6040 con 0x55d2f51dc410 2024-04-16T17:15:47.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.429+0000 7f47a7400640 1 --2- 172.21.15.45:0/1146691522 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d2f51dc410 0x55d2f51de800 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f47a819c0f0 tx=0x7f47940117b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5bea785a8d5e7e8a server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:15:47.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.429+0000 7f479d200640 1 -- 172.21.15.45:0/1146691522 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4794012450 con 0x55d2f51dc410 2024-04-16T17:15:47.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.429+0000 7f479d200640 1 -- 172.21.15.45:0/1146691522 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4794012ad0 con 0x55d2f51dc410 2024-04-16T17:15:47.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.429+0000 7f479d200640 1 -- 172.21.15.45:0/1146691522 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f479400fbc0 con 0x55d2f51dc410 2024-04-16T17:15:47.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.430+0000 7f47b1792740 1 -- 172.21.15.45:0/1146691522 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d2f51dc410 msgr2=0x55d2f51de800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:15:47.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.430+0000 7f47b1792740 1 --2- 172.21.15.45:0/1146691522 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d2f51dc410 0x55d2f51de800 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f47a819c0f0 tx=0x7f47940117b0 comp rx=0 tx=0).stop 2024-04-16T17:15:47.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.430+0000 7f47b1792740 1 -- 172.21.15.45:0/1146691522 shutdown_connections 2024-04-16T17:15:47.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.430+0000 7f47b1792740 1 --2- 172.21.15.45:0/1146691522 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d2f51dc410 0x55d2f51de800 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:47.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.430+0000 7f47b1792740 1 --2- 172.21.15.45:0/1146691522 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d2f51d3830 0x55d2f51dbed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:47.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.430+0000 7f47b1792740 1 --2- 172.21.15.45:0/1146691522 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d2f51d2f10 0x55d2f51d32f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:47.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.430+0000 7f47a4c00640 1 --2- 172.21.15.45:0/283432570 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f478c0a3d20 0x7f478c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:47.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.430+0000 7f47b1792740 1 -- 172.21.15.45:0/1146691522 >> 172.21.15.45:0/1146691522 conn(0x55d2f51cd640 msgr2=0x55d2f51cda20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:15:47.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.431+0000 7f47b1792740 1 -- 172.21.15.45:0/1146691522 shutdown_connections 2024-04-16T17:15:47.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.431+0000 7f47a4c00640 1 -- 172.21.15.45:0/283432570 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9496+0+0 (secure 0 0 0) 0x7f47980f2820 con 0x7f47a8103060 2024-04-16T17:15:47.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.431+0000 7f47aca00640 1 --2- 172.21.15.45:0/283432570 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f478c0a3d20 0x7f478c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:47.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.431+0000 7f47b1792740 1 -- 172.21.15.45:0/1146691522 wait complete. 2024-04-16T17:15:47.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.431+0000 7f47b1792740 1 -- start start 2024-04-16T17:15:47.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.431+0000 7f47aca00640 1 --2- 172.21.15.45:0/283432570 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f478c0a3d20 0x7f478c0a61e0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f47a0005fd0 tx=0x7f47a0009500 comp rx=0 tx=0).ready entity=mgr.25411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:15:47.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.431+0000 7f47b1792740 1 Processor -- start 2024-04-16T17:15:47.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.432+0000 7f47b1792740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d2f51d2f10 0x55d2f51d9410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:47.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.432+0000 7f47b1792740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d2f51d3830 0x55d2f51d9950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:47.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.432+0000 7f47b1792740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d2f51dc410 0x55d2f51d9e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:47.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.432+0000 7f47b1792740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55d2f51df200 con 0x55d2f51dc410 2024-04-16T17:15:47.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.432+0000 7f47b1792740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55d2f50f1830 con 0x55d2f51d3830 2024-04-16T17:15:47.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.432+0000 7f47b1792740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55d2f51df380 con 0x55d2f51d2f10 2024-04-16T17:15:47.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.432+0000 7f47b1792740 10 client.0 _ll_register_callbacks cb 0x55d2f50f9150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:15:47.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.432+0000 7f47a7e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d2f51d2f10 0x55d2f51d9410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:47.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.432+0000 7f47aca00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d2f51dc410 0x55d2f51d9e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:47.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.432+0000 7f47a7400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d2f51d3830 0x55d2f51d9950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:47.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.432+0000 7f47aca00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d2f51dc410 0x55d2f51d9e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:44212/0 (socket says 172.21.15.45:44212) 2024-04-16T17:15:47.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.432+0000 7f47aca00640 1 -- 172.21.15.45:0/394101291 learned_addr learned my addr 172.21.15.45:0/394101291 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:15:47.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.433+0000 7f47a7400640 1 -- 172.21.15.45:0/394101291 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d2f51d2f10 msgr2=0x55d2f51d9410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:15:47.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.433+0000 7f47a7400640 1 --2- 172.21.15.45:0/394101291 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d2f51d2f10 0x55d2f51d9410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:47.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.433+0000 7f47a7400640 1 -- 172.21.15.45:0/394101291 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d2f51dc410 msgr2=0x55d2f51d9e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:15:47.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.433+0000 7f47a7400640 1 --2- 172.21.15.45:0/394101291 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d2f51dc410 0x55d2f51d9e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:47.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.433+0000 7f47a7400640 1 -- 172.21.15.45:0/394101291 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d2f50f5920 con 0x55d2f51d3830 2024-04-16T17:15:47.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.433+0000 7f47a7400640 1 --2- 172.21.15.45:0/394101291 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d2f51d3830 0x55d2f51d9950 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f4794005830 tx=0x7f4794009f60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7df803ad5394a21c server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:15:47.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.433+0000 7f4791600640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-04-16T17:15:47.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.433+0000 7f4791600640 1 -- 172.21.15.45:0/394101291 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4794029020 con 0x55d2f51d3830 2024-04-16T17:15:47.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.433+0000 7f47b1792740 1 -- 172.21.15.45:0/394101291 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4794012ad0 con 0x55d2f51d3830 2024-04-16T17:15:47.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.433+0000 7f4791600640 1 -- 172.21.15.45:0/394101291 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9496+0+0 (secure 0 0 0) 0x7f4794012660 con 0x55d2f51d3830 2024-04-16T17:15:47.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.434+0000 7f4791600640 1 -- 172.21.15.45:0/394101291 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f4794034050 con 0x55d2f51d3830 2024-04-16T17:15:47.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.434+0000 7f47b1792740 10 client.27947 fetch_fsmap learned FSMap version 43 2024-04-16T17:15:47.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.434+0000 7f47b1792740 1 -- 172.21.15.45:0/394101291 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55d2f50c37f0 con 0x55d2f51d3830 2024-04-16T17:15:47.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.434+0000 7f4791600640 1 -- 172.21.15.45:0/394101291 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4794012ad0 con 0x55d2f51d3830 2024-04-16T17:15:47.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.434+0000 7f4791600640 1 -- 172.21.15.45:0/394101291 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f4794005310 con 0x55d2f51d3830 2024-04-16T17:15:47.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.434+0000 7f47b1792740 10 client.27947 fetch_fsmap finished waiting for FSMap version 43 2024-04-16T17:15:47.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.434+0000 7f47b1792740 10 client.27947 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-16T17:15:47.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.434+0000 7f47b1792740 20 client.27947 populate_metadata read hostname 'smithi045' 2024-04-16T17:15:47.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.434+0000 7f47b1792740 1 --2- 172.21.15.45:0/394101291 >> [v2:172.21.15.117:6838/2378675949,v1:172.21.15.117:6839/2378675949] conn(0x55d2f51ec750 0x55d2f51eeb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:47.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.434+0000 7f47b1792740 4 client.27947 mds_command: new command op to 24532 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:15:47.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.434+0000 7f47b1792740 1 -- 172.21.15.45:0/394101291 --> [v2:172.21.15.117:6838/2378675949,v1:172.21.15.117:6839/2378675949] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d2f5145a40 con 0x55d2f51ec750 2024-04-16T17:15:47.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.435+0000 7f47a7e00640 1 --2- 172.21.15.45:0/394101291 >> [v2:172.21.15.117:6838/2378675949,v1:172.21.15.117:6839/2378675949] conn(0x55d2f51ec750 0x55d2f51eeb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:47.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.435+0000 7f47a7e00640 1 --2- 172.21.15.45:0/394101291 >> [v2:172.21.15.117:6838/2378675949,v1:172.21.15.117:6839/2378675949] conn(0x55d2f51ec750 0x55d2f51eeb40 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=47ce6075dd0552b7 server_cookie=158909a3598bcb81 in_seq=0 out_seq=0 2024-04-16T17:15:47.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.435+0000 7f4791600640 10 client.27947 ms_handle_connect on v2:172.21.15.117:6838/2378675949 2024-04-16T17:15:47.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.436+0000 7f4791600640 1 -- 172.21.15.45:0/394101291 <== mds.2 v2:172.21.15.117:6838/2378675949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d2f5145a40 con 0x55d2f51ec750 2024-04-16T17:15:47.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.436+0000 7f4791600640 10 client.27947 handle_command_reply: tid=0 2024-04-16T17:15:47.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.436+0000 7f47b1792740 1 client.27947 shutdown 2024-04-16T17:15:47.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.436+0000 7f47b1792740 1 -- 172.21.15.45:0/394101291 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d2f51d3830 msgr2=0x55d2f51d9950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:15:47.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.436+0000 7f47b1792740 1 --2- 172.21.15.45:0/394101291 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d2f51d3830 0x55d2f51d9950 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f4794005830 tx=0x7f4794009f60 comp rx=0 tx=0).stop 2024-04-16T17:15:47.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.436+0000 7f47b1792740 1 -- 172.21.15.45:0/394101291 shutdown_connections 2024-04-16T17:15:47.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.436+0000 7f47b1792740 1 --2- 172.21.15.45:0/394101291 >> [v2:172.21.15.117:6838/2378675949,v1:172.21.15.117:6839/2378675949] conn(0x55d2f51ec750 0x55d2f51eeb40 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:47.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.436+0000 7f47b1792740 1 --2- 172.21.15.45:0/394101291 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d2f51dc410 0x55d2f51d9e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:47.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.436+0000 7f47b1792740 1 --2- 172.21.15.45:0/394101291 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d2f51d3830 0x55d2f51d9950 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:47.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.436+0000 7f4791600640 0 client.27947 ms_handle_reset on v2:172.21.15.117:6838/2378675949 2024-04-16T17:15:47.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.436+0000 7f47b1792740 1 --2- 172.21.15.45:0/394101291 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d2f51d2f10 0x55d2f51d9410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:47.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.436+0000 7f47b1792740 1 -- 172.21.15.45:0/394101291 >> 172.21.15.45:0/394101291 conn(0x55d2f51cd640 msgr2=0x55d2f51cf390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:15:47.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.436+0000 7f47b1792740 1 -- 172.21.15.45:0/394101291 shutdown_connections 2024-04-16T17:15:47.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.436+0000 7f47b1792740 1 -- 172.21.15.45:0/394101291 wait complete. 2024-04-16T17:15:47.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.436+0000 7f47b1792740 20 client.27947 trim_cache size 0 max 16384 2024-04-16T17:15:47.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.450+0000 7f47b1792740 1 Processor -- start 2024-04-16T17:15:47.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.450+0000 7f47b1792740 1 -- start start 2024-04-16T17:15:47.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.451+0000 7f47b1792740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d2f51d2f10 0x55d2f51e8120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:47.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.451+0000 7f47b1792740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d2f51d3830 0x55d2f51d9ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:47.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.451+0000 7f47b1792740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d2f51dc410 0x55d2f51f42d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:47.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.451+0000 7f47b1792740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f47a81b3ed0 con 0x55d2f51d2f10 2024-04-16T17:15:47.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.451+0000 7f47b1792740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f47a81b17a0 con 0x55d2f51d3830 2024-04-16T17:15:47.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.451+0000 7f47b1792740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f47a81af610 con 0x55d2f51dc410 2024-04-16T17:15:47.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.451+0000 7f47a7e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d2f51d2f10 0x55d2f51e8120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:47.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.451+0000 7f47a7e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d2f51d2f10 0x55d2f51e8120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:44216/0 (socket says 172.21.15.45:44216) 2024-04-16T17:15:47.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.451+0000 7f47a7e00640 1 -- 172.21.15.45:0/3734403238 learned_addr learned my addr 172.21.15.45:0/3734403238 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:15:47.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.451+0000 7f47a7400640 1 --2- 172.21.15.45:0/3734403238 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d2f51d3830 0x55d2f51d9ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:47.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.451+0000 7f47aca00640 1 --2- 172.21.15.45:0/3734403238 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d2f51dc410 0x55d2f51f42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:47.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.451+0000 7f47a7e00640 1 -- 172.21.15.45:0/3734403238 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d2f51dc410 msgr2=0x55d2f51f42d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:15:47.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.451+0000 7f47a7e00640 1 --2- 172.21.15.45:0/3734403238 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d2f51dc410 0x55d2f51f42d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:47.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.451+0000 7f47a7e00640 1 -- 172.21.15.45:0/3734403238 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d2f51d3830 msgr2=0x55d2f51d9ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:15:47.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.451+0000 7f47a7e00640 1 --2- 172.21.15.45:0/3734403238 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d2f51d3830 0x55d2f51d9ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:47.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.452+0000 7f47a7e00640 1 -- 172.21.15.45:0/3734403238 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d2f4e4a3f0 con 0x55d2f51d2f10 2024-04-16T17:15:47.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.452+0000 7f47a7400640 1 --2- 172.21.15.45:0/3734403238 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d2f51d3830 0x55d2f51d9ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:15:47.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.452+0000 7f47a7e00640 1 --2- 172.21.15.45:0/3734403238 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d2f51d2f10 0x55d2f51e8120 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f47980be1a0 tx=0x7f47980f8530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7be71dda951779a5 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:15:47.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.452+0000 7f4793e00640 1 -- 172.21.15.45:0/3734403238 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47980be270 con 0x55d2f51d2f10 2024-04-16T17:15:47.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.452+0000 7f4793e00640 1 -- 172.21.15.45:0/3734403238 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f47980beba0 con 0x55d2f51d2f10 2024-04-16T17:15:47.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.452+0000 7f4793e00640 1 -- 172.21.15.45:0/3734403238 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47980bde00 con 0x55d2f51d2f10 2024-04-16T17:15:47.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.453+0000 7f47b1792740 1 -- 172.21.15.45:0/3734403238 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d2f51d2f10 msgr2=0x55d2f51e8120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:15:47.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.453+0000 7f47b1792740 1 --2- 172.21.15.45:0/3734403238 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d2f51d2f10 0x55d2f51e8120 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f47980be1a0 tx=0x7f47980f8530 comp rx=0 tx=0).stop 2024-04-16T17:15:47.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.453+0000 7f47b1792740 1 -- 172.21.15.45:0/3734403238 shutdown_connections 2024-04-16T17:15:47.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.453+0000 7f47b1792740 1 --2- 172.21.15.45:0/3734403238 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d2f51dc410 0x55d2f51f42d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:47.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.453+0000 7f47b1792740 1 --2- 172.21.15.45:0/3734403238 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d2f51d3830 0x55d2f51d9ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:47.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.453+0000 7f47b1792740 1 --2- 172.21.15.45:0/3734403238 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d2f51d2f10 0x55d2f51e8120 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:47.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.453+0000 7f47b1792740 1 -- 172.21.15.45:0/3734403238 >> 172.21.15.45:0/3734403238 conn(0x55d2f51cd640 msgr2=0x55d2f51d93d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:15:47.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.453+0000 7f47b1792740 1 -- 172.21.15.45:0/3734403238 shutdown_connections 2024-04-16T17:15:47.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.453+0000 7f47b1792740 1 -- 172.21.15.45:0/3734403238 wait complete. 2024-04-16T17:15:47.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.453+0000 7f47b1792740 1 -- start start 2024-04-16T17:15:47.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.453+0000 7f47b1792740 1 Processor -- start 2024-04-16T17:15:47.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.454+0000 7f47b1792740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d2f51d2f10 0x55d2f51e08d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:47.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.454+0000 7f47b1792740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d2f51d3830 0x55d2f51e0e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:47.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.454+0000 7f47b1792740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d2f51dc410 0x55d2f524bb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:47.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.454+0000 7f47b1792740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f47a81afc60 con 0x55d2f51dc410 2024-04-16T17:15:47.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.454+0000 7f47b1792740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f47a81b10d0 con 0x55d2f51d3830 2024-04-16T17:15:47.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.454+0000 7f47b1792740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f47a81b0330 con 0x55d2f51d2f10 2024-04-16T17:15:47.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.454+0000 7f47b1792740 10 client.0 _ll_register_callbacks cb 0x7f47a00099c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:15:47.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.454+0000 7f47aca00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d2f51dc410 0x55d2f524bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:47.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.454+0000 7f47a7e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d2f51d2f10 0x55d2f51e08d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:47.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.454+0000 7f47a7e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d2f51d2f10 0x55d2f51e08d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:34010/0 (socket says 172.21.15.45:34010) 2024-04-16T17:15:47.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.454+0000 7f47aca00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d2f51dc410 0x55d2f524bb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:44218/0 (socket says 172.21.15.45:44218) 2024-04-16T17:15:47.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.454+0000 7f47a7e00640 1 -- 172.21.15.45:0/3735193977 learned_addr learned my addr 172.21.15.45:0/3735193977 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:15:47.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.454+0000 7f47a7400640 1 --2- 172.21.15.45:0/3735193977 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d2f51d3830 0x55d2f51e0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:47.461 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.454+0000 7f47a7e00640 1 -- 172.21.15.45:0/3735193977 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d2f51d3830 msgr2=0x55d2f51e0e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:15:47.461 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.454+0000 7f47a7e00640 1 --2- 172.21.15.45:0/3735193977 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d2f51d3830 0x55d2f51e0e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:47.461 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.454+0000 7f47a7e00640 1 -- 172.21.15.45:0/3735193977 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d2f51dc410 msgr2=0x55d2f524bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:15:47.461 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.454+0000 7f47a7e00640 1 --2- 172.21.15.45:0/3735193977 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d2f51dc410 0x55d2f524bb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:47.461 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.454+0000 7f47a7e00640 1 -- 172.21.15.45:0/3735193977 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d2f4f147d0 con 0x55d2f51d2f10 2024-04-16T17:15:47.461 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.454+0000 7f47a7400640 1 --2- 172.21.15.45:0/3735193977 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d2f51d3830 0x55d2f51e0e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:15:47.461 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.455+0000 7f47a7e00640 1 --2- 172.21.15.45:0/3735193977 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d2f51d2f10 0x55d2f51e08d0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f479803e040 tx=0x7f47980f8150 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e450252a0232f878 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:15:47.461 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.455+0000 7f4791600640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-04-16T17:15:47.461 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.455+0000 7f4791600640 1 -- 172.21.15.45:0/3735193977 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47980bdfe0 con 0x55d2f51d2f10 2024-04-16T17:15:47.462 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.455+0000 7f47b1792740 1 -- 172.21.15.45:0/3735193977 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f47980beba0 con 0x55d2f51d2f10 2024-04-16T17:15:47.462 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.455+0000 7f4791600640 1 -- 172.21.15.45:0/3735193977 <== mon.1 v2:172.21.15.117:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9496+0+0 (secure 0 0 0) 0x7f4798004c20 con 0x55d2f51d2f10 2024-04-16T17:15:47.462 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.455+0000 7f4791600640 1 -- 172.21.15.45:0/3735193977 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f47980be270 con 0x55d2f51d2f10 2024-04-16T17:15:47.462 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.455+0000 7f47b1792740 10 client.27904 fetch_fsmap learned FSMap version 43 2024-04-16T17:15:47.462 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.455+0000 7f47b1792740 1 -- 172.21.15.45:0/3735193977 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55d2f52532e0 con 0x55d2f51d2f10 2024-04-16T17:15:47.462 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.455+0000 7f4791600640 1 -- 172.21.15.45:0/3735193977 <== mon.1 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f47980beba0 con 0x55d2f51d2f10 2024-04-16T17:15:47.462 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.456+0000 7f4791600640 1 -- 172.21.15.45:0/3735193977 <== mon.1 v2:172.21.15.117:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f47980f2070 con 0x55d2f51d2f10 2024-04-16T17:15:47.462 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.456+0000 7f47b1792740 10 client.27904 fetch_fsmap finished waiting for FSMap version 43 2024-04-16T17:15:47.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.456+0000 7f47b1792740 10 client.27904 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-16T17:15:47.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.456+0000 7f47b1792740 20 client.27904 populate_metadata read hostname 'smithi045' 2024-04-16T17:15:47.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.456+0000 7f47b1792740 1 --2- 172.21.15.45:0/3735193977 >> [v2:172.21.15.117:6838/2378675949,v1:172.21.15.117:6839/2378675949] conn(0x55d2f52539b0 0x55d2f5255dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:47.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.456+0000 7f47b1792740 4 client.27904 mds_command: new command op to 24532 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T17:15:47.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.456+0000 7f47b1792740 1 -- 172.21.15.45:0/3735193977 --> [v2:172.21.15.117:6838/2378675949,v1:172.21.15.117:6839/2378675949] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d2f513f8b0 con 0x55d2f52539b0 2024-04-16T17:15:47.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.456+0000 7f47a7400640 1 --2- 172.21.15.45:0/3735193977 >> [v2:172.21.15.117:6838/2378675949,v1:172.21.15.117:6839/2378675949] conn(0x55d2f52539b0 0x55d2f5255dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:47.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.456+0000 7f47a7400640 1 --2- 172.21.15.45:0/3735193977 >> [v2:172.21.15.117:6838/2378675949,v1:172.21.15.117:6839/2378675949] conn(0x55d2f52539b0 0x55d2f5255dd0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7628e2a393ccef35 server_cookie=88250a3b4ef37d3f in_seq=0 out_seq=0 2024-04-16T17:15:47.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.456+0000 7f4791600640 10 client.27904 ms_handle_connect on v2:172.21.15.117:6838/2378675949 2024-04-16T17:15:47.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.457+0000 7f4791600640 1 -- 172.21.15.45:0/3735193977 <== mds.2 v2:172.21.15.117:6838/2378675949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d2f513f8b0 con 0x55d2f52539b0 2024-04-16T17:15:47.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.457+0000 7f4791600640 10 client.27904 handle_command_reply: tid=0 2024-04-16T17:15:47.464 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.457+0000 7f47b1792740 1 client.27904 shutdown 2024-04-16T17:15:47.464 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.457+0000 7f47b1792740 1 -- 172.21.15.45:0/3735193977 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d2f51d2f10 msgr2=0x55d2f51e08d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:15:47.464 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.457+0000 7f47b1792740 1 --2- 172.21.15.45:0/3735193977 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d2f51d2f10 0x55d2f51e08d0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f479803e040 tx=0x7f47980f8150 comp rx=0 tx=0).stop 2024-04-16T17:15:47.464 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.457+0000 7f47b1792740 1 -- 172.21.15.45:0/3735193977 shutdown_connections 2024-04-16T17:15:47.464 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.457+0000 7f47b1792740 1 --2- 172.21.15.45:0/3735193977 >> [v2:172.21.15.117:6838/2378675949,v1:172.21.15.117:6839/2378675949] conn(0x55d2f52539b0 0x55d2f5255dd0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:47.464 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.457+0000 7f47b1792740 1 --2- 172.21.15.45:0/3735193977 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d2f51dc410 0x55d2f524bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:47.464 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.457+0000 7f47b1792740 1 --2- 172.21.15.45:0/3735193977 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d2f51d3830 0x55d2f51e0e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:47.464 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.457+0000 7f4791600640 0 client.27904 ms_handle_reset on v2:172.21.15.117:6838/2378675949 2024-04-16T17:15:47.464 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.457+0000 7f47b1792740 1 --2- 172.21.15.45:0/3735193977 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d2f51d2f10 0x55d2f51e08d0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:47.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.457+0000 7f47b1792740 1 -- 172.21.15.45:0/3735193977 >> 172.21.15.45:0/3735193977 conn(0x55d2f51cd640 msgr2=0x55d2f51ce9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:15:47.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.457+0000 7f47b1792740 1 -- 172.21.15.45:0/3735193977 shutdown_connections 2024-04-16T17:15:47.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.457+0000 7f47b1792740 1 -- 172.21.15.45:0/3735193977 wait complete. 2024-04-16T17:15:47.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.457+0000 7f47b1792740 20 client.27904 trim_cache size 0 max 16384 2024-04-16T17:15:47.465 INFO:teuthology.orchestra.run.smithi045.stdout:[] 2024-04-16T17:15:47.466 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.458+0000 7f47ad400640 1 -- 172.21.15.45:0/283432570 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f478c0a3d20 msgr2=0x7f478c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:15:47.466 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.458+0000 7f47ad400640 1 --2- 172.21.15.45:0/283432570 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f478c0a3d20 0x7f478c0a61e0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f47a0005fd0 tx=0x7f47a0009500 comp rx=0 tx=0).stop 2024-04-16T17:15:47.466 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.458+0000 7f47ad400640 1 -- 172.21.15.45:0/283432570 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f47a8103060 msgr2=0x7f47a81a06b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:15:47.466 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.458+0000 7f47ad400640 1 --2- 172.21.15.45:0/283432570 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f47a8103060 0x7f47a81a06b0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f4798000c00 tx=0x7f4798004360 comp rx=0 tx=0).stop 2024-04-16T17:15:47.466 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.458+0000 7f47ad400640 1 -- 172.21.15.45:0/283432570 shutdown_connections 2024-04-16T17:15:47.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.458+0000 7f47ad400640 1 --2- 172.21.15.45:0/283432570 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f478c0a3d20 0x7f478c0a61e0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:47.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.458+0000 7f47ad400640 1 --2- 172.21.15.45:0/283432570 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f47a810c480 0x7f47a819a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:47.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.458+0000 7f47ad400640 1 --2- 172.21.15.45:0/283432570 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f47a8103980 0x7f47a81a0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:47.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.458+0000 7f47ad400640 1 --2- 172.21.15.45:0/283432570 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f47a8103060 0x7f47a81a06b0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:47.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.458+0000 7f47ad400640 1 -- 172.21.15.45:0/283432570 >> 172.21.15.45:0/283432570 conn(0x7f47a80fc880 msgr2=0x7f47a8101de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:15:47.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.458+0000 7f47ad400640 1 -- 172.21.15.45:0/283432570 shutdown_connections 2024-04-16T17:15:47.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.458+0000 7f47ad400640 1 -- 172.21.15.45:0/283432570 wait complete. 2024-04-16T17:15:47.475 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-16T17:15:47.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.688+0000 7f35d3400640 1 -- 172.21.15.45:0/3570717233 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f35cc109580 msgr2=0x7f35cc111150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:15:47.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.688+0000 7f35d3400640 1 --2- 172.21.15.45:0/3570717233 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f35cc109580 0x7f35cc111150 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f35c4009ca0 tx=0x7f35c402c100 comp rx=0 tx=0).stop 2024-04-16T17:15:47.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.688+0000 7f35d3400640 1 -- 172.21.15.45:0/3570717233 shutdown_connections 2024-04-16T17:15:47.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.688+0000 7f35d3400640 1 --2- 172.21.15.45:0/3570717233 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f35cc104e60 0x7f35cc108ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:47.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.688+0000 7f35d3400640 1 --2- 172.21.15.45:0/3570717233 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f35cc109580 0x7f35cc111150 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:47.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.688+0000 7f35d3400640 1 --2- 172.21.15.45:0/3570717233 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f35cc1044b0 0x7f35cc104890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:47.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.688+0000 7f35d3400640 1 -- 172.21.15.45:0/3570717233 >> 172.21.15.45:0/3570717233 conn(0x7f35cc0ffd90 msgr2=0x7f35cc1021b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:15:47.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.688+0000 7f35d3400640 1 -- 172.21.15.45:0/3570717233 shutdown_connections 2024-04-16T17:15:47.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.688+0000 7f35d3400640 1 -- 172.21.15.45:0/3570717233 wait complete. 2024-04-16T17:15:47.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.689+0000 7f35d3400640 1 Processor -- start 2024-04-16T17:15:47.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.689+0000 7f35d3400640 1 -- start start 2024-04-16T17:15:47.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.690+0000 7f35d3400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f35cc1044b0 0x7f35cc1a0780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:47.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.690+0000 7f35d3400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f35cc109580 0x7f35cc1a0cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:47.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.690+0000 7f35d3400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f35cc19a980 0x7f35cc19ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:47.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.690+0000 7f35d3400640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f35cc114640 con 0x7f35cc109580 2024-04-16T17:15:47.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.690+0000 7f35d3400640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f35cc1144c0 con 0x7f35cc1044b0 2024-04-16T17:15:47.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.690+0000 7f35d3400640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f35cc1147c0 con 0x7f35cc19a980 2024-04-16T17:15:47.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.690+0000 7f35c3e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f35cc109580 0x7f35cc1a0cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:47.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.690+0000 7f35c3e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f35cc109580 0x7f35cc1a0cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:44236/0 (socket says 172.21.15.45:44236) 2024-04-16T17:15:47.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.690+0000 7f35c3e00640 1 -- 172.21.15.45:0/1615486045 learned_addr learned my addr 172.21.15.45:0/1615486045 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:15:47.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.690+0000 7f35d0a00640 1 --2- 172.21.15.45:0/1615486045 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f35cc1044b0 0x7f35cc1a0780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:47.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.690+0000 7f35d1400640 1 --2- 172.21.15.45:0/1615486045 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f35cc19a980 0x7f35cc19ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:47.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.690+0000 7f35c3e00640 1 -- 172.21.15.45:0/1615486045 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f35cc19a980 msgr2=0x7f35cc19ae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:15:47.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.690+0000 7f35c3e00640 1 --2- 172.21.15.45:0/1615486045 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f35cc19a980 0x7f35cc19ae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:47.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.690+0000 7f35c3e00640 1 -- 172.21.15.45:0/1615486045 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f35cc1044b0 msgr2=0x7f35cc1a0780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:15:47.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.690+0000 7f35c3e00640 1 --2- 172.21.15.45:0/1615486045 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f35cc1044b0 0x7f35cc1a0780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:47.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.690+0000 7f35c3e00640 1 -- 172.21.15.45:0/1615486045 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35cc19b4a0 con 0x7f35cc109580 2024-04-16T17:15:47.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.691+0000 7f35c3e00640 1 --2- 172.21.15.45:0/1615486045 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f35cc109580 0x7f35cc1a0cc0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f35b400e9f0 tx=0x7f35b400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:15:47.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.691+0000 7f35ca000640 1 -- 172.21.15.45:0/1615486045 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35b400ccc0 con 0x7f35cc109580 2024-04-16T17:15:47.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.691+0000 7f35ca000640 1 -- 172.21.15.45:0/1615486045 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f35b4005500 con 0x7f35cc109580 2024-04-16T17:15:47.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.691+0000 7f35d3400640 1 -- 172.21.15.45:0/1615486045 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35cc19b790 con 0x7f35cc109580 2024-04-16T17:15:47.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.691+0000 7f35ca000640 1 -- 172.21.15.45:0/1615486045 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35b4010690 con 0x7f35cc109580 2024-04-16T17:15:47.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.691+0000 7f35d3400640 1 -- 172.21.15.45:0/1615486045 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35cc1a7990 con 0x7f35cc109580 2024-04-16T17:15:47.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.694+0000 7f35d61f5740 1 Processor -- start 2024-04-16T17:15:47.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.694+0000 7f35ca000640 1 -- 172.21.15.45:0/1615486045 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f35b40046c0 con 0x7f35cc109580 2024-04-16T17:15:47.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.694+0000 7f35d61f5740 1 -- start start 2024-04-16T17:15:47.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.695+0000 7f35d61f5740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55826de13d50 0x55826de14130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:47.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.695+0000 7f35d61f5740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55826de14670 0x55826de1cd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:47.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.695+0000 7f35d61f5740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55826de1d250 0x55826de1f640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:47.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.695+0000 7f35d61f5740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55826dd79020 con 0x55826de14670 2024-04-16T17:15:47.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.695+0000 7f35d61f5740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55826dd1f460 con 0x55826de13d50 2024-04-16T17:15:47.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.695+0000 7f35d61f5740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55826dd74fe0 con 0x55826de1d250 2024-04-16T17:15:47.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.695+0000 7f35d0a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55826de14670 0x55826de1cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:47.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.695+0000 7f35d0a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55826de14670 0x55826de1cd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:44246/0 (socket says 172.21.15.45:44246) 2024-04-16T17:15:47.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.695+0000 7f35d0a00640 1 -- 172.21.15.45:0/3610457661 learned_addr learned my addr 172.21.15.45:0/3610457661 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:15:47.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.695+0000 7f35d1400640 1 --2- 172.21.15.45:0/3610457661 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55826de13d50 0x55826de14130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:47.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.695+0000 7f35c3e00640 1 --2- 172.21.15.45:0/3610457661 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55826de1d250 0x55826de1f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:47.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.696+0000 7f35d0a00640 1 -- 172.21.15.45:0/3610457661 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55826de1d250 msgr2=0x55826de1f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:15:47.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.696+0000 7f35d0a00640 1 --2- 172.21.15.45:0/3610457661 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55826de1d250 0x55826de1f640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:47.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.696+0000 7f35d0a00640 1 -- 172.21.15.45:0/3610457661 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55826de13d50 msgr2=0x55826de14130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:15:47.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.696+0000 7f35d0a00640 1 --2- 172.21.15.45:0/3610457661 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55826de13d50 0x55826de14130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:47.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.696+0000 7f35d0a00640 1 -- 172.21.15.45:0/3610457661 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55826dd27040 con 0x55826de14670 2024-04-16T17:15:47.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.696+0000 7f35d0a00640 1 --2- 172.21.15.45:0/3610457661 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55826de14670 0x55826de1cd10 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f35bc009c90 tx=0x7f35bc011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4a0a6236b77fdd3 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:15:47.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.696+0000 7f35ca000640 1 --2- 172.21.15.45:0/1615486045 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f35b00a3da0 0x7f35b00a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:47.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.696+0000 7f35ca000640 1 -- 172.21.15.45:0/1615486045 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9496+0+0 (secure 0 0 0) 0x7f35b40d0e40 con 0x7f35cc109580 2024-04-16T17:15:47.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.696+0000 7f35c1c00640 1 -- 172.21.15.45:0/3610457661 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35bc005220 con 0x55826de14670 2024-04-16T17:15:47.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.697+0000 7f35c1c00640 1 -- 172.21.15.45:0/3610457661 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f35bc004c80 con 0x55826de14670 2024-04-16T17:15:47.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.697+0000 7f35d1400640 1 --2- 172.21.15.45:0/1615486045 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f35b00a3da0 0x7f35b00a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:47.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.697+0000 7f35c1c00640 1 -- 172.21.15.45:0/3610457661 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35bc019830 con 0x55826de14670 2024-04-16T17:15:47.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.697+0000 7f35d61f5740 1 -- 172.21.15.45:0/3610457661 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55826de14670 msgr2=0x55826de1cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:15:47.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.697+0000 7f35d61f5740 1 --2- 172.21.15.45:0/3610457661 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55826de14670 0x55826de1cd10 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f35bc009c90 tx=0x7f35bc011810 comp rx=0 tx=0).stop 2024-04-16T17:15:47.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.697+0000 7f35d1400640 1 --2- 172.21.15.45:0/1615486045 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f35b00a3da0 0x7f35b00a6260 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f35c4005e30 tx=0x7f35c4005d80 comp rx=0 tx=0).ready entity=mgr.25411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:15:47.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.697+0000 7f35d61f5740 1 -- 172.21.15.45:0/3610457661 shutdown_connections 2024-04-16T17:15:47.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.697+0000 7f35d61f5740 1 --2- 172.21.15.45:0/3610457661 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55826de1d250 0x55826de1f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:47.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.697+0000 7f35d61f5740 1 --2- 172.21.15.45:0/3610457661 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55826de14670 0x55826de1cd10 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:47.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.697+0000 7f35d61f5740 1 --2- 172.21.15.45:0/3610457661 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55826de13d50 0x55826de14130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:47.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.697+0000 7f35d61f5740 1 -- 172.21.15.45:0/3610457661 >> 172.21.15.45:0/3610457661 conn(0x55826de0e480 msgr2=0x55826de0e860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:15:47.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.698+0000 7f35d61f5740 1 -- 172.21.15.45:0/3610457661 shutdown_connections 2024-04-16T17:15:47.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.698+0000 7f35d61f5740 1 -- 172.21.15.45:0/3610457661 wait complete. 2024-04-16T17:15:47.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.698+0000 7f35d61f5740 1 -- start start 2024-04-16T17:15:47.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.698+0000 7f35d61f5740 1 Processor -- start 2024-04-16T17:15:47.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.699+0000 7f35d61f5740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55826de13d50 0x55826de1a300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:47.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.699+0000 7f35d61f5740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55826de14670 0x55826de1a840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:47.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.699+0000 7f35d61f5740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55826de1d250 0x55826de1ad80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:47.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.699+0000 7f35d61f5740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55826de20040 con 0x55826de14670 2024-04-16T17:15:47.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.699+0000 7f35d61f5740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55826dd32830 con 0x55826de13d50 2024-04-16T17:15:47.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.699+0000 7f35d61f5740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55826de201c0 con 0x55826de1d250 2024-04-16T17:15:47.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.699+0000 7f35d61f5740 10 client.0 _ll_register_callbacks cb 0x55826dd3a150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:15:47.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.699+0000 7f35c3e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55826de14670 0x55826de1a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:47.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.699+0000 7f35c3e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55826de14670 0x55826de1a840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:44250/0 (socket says 172.21.15.45:44250) 2024-04-16T17:15:47.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.699+0000 7f35c3e00640 1 -- 172.21.15.45:0/2553379027 learned_addr learned my addr 172.21.15.45:0/2553379027 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:15:47.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.699+0000 7f35d0a00640 1 --2- 172.21.15.45:0/2553379027 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55826de13d50 0x55826de1a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:47.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.699+0000 7f35d1400640 1 --2- 172.21.15.45:0/2553379027 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55826de1d250 0x55826de1ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:47.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.699+0000 7f35c3e00640 1 -- 172.21.15.45:0/2553379027 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55826de1d250 msgr2=0x55826de1ad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:15:47.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.699+0000 7f35c3e00640 1 --2- 172.21.15.45:0/2553379027 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55826de1d250 0x55826de1ad80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:47.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.699+0000 7f35c3e00640 1 -- 172.21.15.45:0/2553379027 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55826de13d50 msgr2=0x55826de1a300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:15:47.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.699+0000 7f35c3e00640 1 --2- 172.21.15.45:0/2553379027 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55826de13d50 0x55826de1a300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:47.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.699+0000 7f35c3e00640 1 -- 172.21.15.45:0/2553379027 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55826dd36920 con 0x55826de14670 2024-04-16T17:15:47.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.700+0000 7f35c3e00640 1 --2- 172.21.15.45:0/2553379027 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55826de14670 0x55826de1a840 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f35b40ce220 tx=0x7f35b400cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9e58c5bb3317910 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:15:47.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.700+0000 7f35ba000640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-16T17:15:47.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.700+0000 7f35ba000640 1 -- 172.21.15.45:0/2553379027 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35b409c7e0 con 0x55826de14670 2024-04-16T17:15:47.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.700+0000 7f35ba000640 1 -- 172.21.15.45:0/2553379027 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9496+0+0 (secure 0 0 0) 0x7f35b40bbdd0 con 0x55826de14670 2024-04-16T17:15:47.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.700+0000 7f35d61f5740 1 -- 172.21.15.45:0/2553379027 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f35bc004c80 con 0x55826de14670 2024-04-16T17:15:47.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.700+0000 7f35ba000640 1 -- 172.21.15.45:0/2553379027 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f35b40056b0 con 0x55826de14670 2024-04-16T17:15:47.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.700+0000 7f35d61f5740 10 client.17814 fetch_fsmap learned FSMap version 43 2024-04-16T17:15:47.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.700+0000 7f35d61f5740 1 -- 172.21.15.45:0/2553379027 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55826dd047f0 con 0x55826de14670 2024-04-16T17:15:47.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.700+0000 7f35ba000640 1 -- 172.21.15.45:0/2553379027 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f35bc004c80 con 0x55826de14670 2024-04-16T17:15:47.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.700+0000 7f35ba000640 1 -- 172.21.15.45:0/2553379027 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f35b40d2070 con 0x55826de14670 2024-04-16T17:15:47.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.701+0000 7f35d61f5740 10 client.17814 fetch_fsmap finished waiting for FSMap version 43 2024-04-16T17:15:47.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.701+0000 7f35d61f5740 10 client.17814 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-04-16T17:15:47.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.701+0000 7f35d61f5740 20 client.17814 populate_metadata read hostname 'smithi045' 2024-04-16T17:15:47.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.701+0000 7f35d61f5740 1 --2- 172.21.15.45:0/2553379027 >> [v2:172.21.15.45:6840/3755310822,v1:172.21.15.45:6841/3755310822] conn(0x55826de2d660 0x55826de2fa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:47.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.701+0000 7f35d61f5740 4 client.17814 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:15:47.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.701+0000 7f35d61f5740 1 -- 172.21.15.45:0/2553379027 --> [v2:172.21.15.45:6840/3755310822,v1:172.21.15.45:6841/3755310822] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55826dd86a40 con 0x55826de2d660 2024-04-16T17:15:47.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.701+0000 7f35d0a00640 1 --2- 172.21.15.45:0/2553379027 >> [v2:172.21.15.45:6840/3755310822,v1:172.21.15.45:6841/3755310822] conn(0x55826de2d660 0x55826de2fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:47.708 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.701+0000 7f35d0a00640 1 --2- 172.21.15.45:0/2553379027 >> [v2:172.21.15.45:6840/3755310822,v1:172.21.15.45:6841/3755310822] conn(0x55826de2d660 0x55826de2fa50 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=186f80881f04f2d server_cookie=531749096f1c5513 in_seq=0 out_seq=0 2024-04-16T17:15:47.708 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.701+0000 7f35ba000640 10 client.17814 ms_handle_connect on v2:172.21.15.45:6840/3755310822 2024-04-16T17:15:47.708 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.702+0000 7f35ba000640 1 -- 172.21.15.45:0/2553379027 <== mds.1 v2:172.21.15.45:6840/3755310822 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55826dd86a40 con 0x55826de2d660 2024-04-16T17:15:47.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.702+0000 7f35ba000640 10 client.17814 handle_command_reply: tid=0 2024-04-16T17:15:47.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.702+0000 7f35d61f5740 1 client.17814 shutdown 2024-04-16T17:15:47.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.702+0000 7f35d61f5740 1 -- 172.21.15.45:0/2553379027 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55826de14670 msgr2=0x55826de1a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:15:47.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.702+0000 7f35d61f5740 1 --2- 172.21.15.45:0/2553379027 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55826de14670 0x55826de1a840 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f35b40ce220 tx=0x7f35b400cf60 comp rx=0 tx=0).stop 2024-04-16T17:15:47.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.703+0000 7f35d61f5740 1 -- 172.21.15.45:0/2553379027 shutdown_connections 2024-04-16T17:15:47.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.703+0000 7f35d61f5740 1 --2- 172.21.15.45:0/2553379027 >> [v2:172.21.15.45:6840/3755310822,v1:172.21.15.45:6841/3755310822] conn(0x55826de2d660 0x55826de2fa50 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:47.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.703+0000 7f35d61f5740 1 --2- 172.21.15.45:0/2553379027 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55826de1d250 0x55826de1ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:47.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.703+0000 7f35d61f5740 1 --2- 172.21.15.45:0/2553379027 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55826de14670 0x55826de1a840 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:47.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.703+0000 7f35d61f5740 1 --2- 172.21.15.45:0/2553379027 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55826de13d50 0x55826de1a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:47.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.703+0000 7f35d61f5740 1 -- 172.21.15.45:0/2553379027 >> 172.21.15.45:0/2553379027 conn(0x55826de0e480 msgr2=0x55826de101d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:15:47.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.703+0000 7f35ba000640 0 client.17814 ms_handle_reset on v2:172.21.15.45:6840/3755310822 2024-04-16T17:15:47.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.703+0000 7f35d61f5740 1 -- 172.21.15.45:0/2553379027 shutdown_connections 2024-04-16T17:15:47.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.703+0000 7f35d61f5740 1 -- 172.21.15.45:0/2553379027 wait complete. 2024-04-16T17:15:47.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.703+0000 7f35d61f5740 20 client.17814 trim_cache size 0 max 16384 2024-04-16T17:15:47.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.718+0000 7f35d61f5740 1 Processor -- start 2024-04-16T17:15:47.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.718+0000 7f35d61f5740 1 -- start start 2024-04-16T17:15:47.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.718+0000 7f35d61f5740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55826de13d50 0x55826de28f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:47.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.718+0000 7f35d61f5740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55826de14670 0x55826de1c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:47.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.719+0000 7f35d61f5740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55826de1d250 0x55826de0ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:47.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.719+0000 7f35d61f5740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f35cc1b4020 con 0x55826de1d250 2024-04-16T17:15:47.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.719+0000 7f35d61f5740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f35cc1b18f0 con 0x55826de14670 2024-04-16T17:15:47.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.719+0000 7f35d61f5740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f35cc1af760 con 0x55826de13d50 2024-04-16T17:15:47.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.719+0000 7f35d0a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55826de13d50 0x55826de28f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:47.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.719+0000 7f35c3e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55826de14670 0x55826de1c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:47.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.719+0000 7f35d0a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55826de13d50 0x55826de28f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:34052/0 (socket says 172.21.15.45:34052) 2024-04-16T17:15:47.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.719+0000 7f35d0a00640 1 -- 172.21.15.45:0/3970146473 learned_addr learned my addr 172.21.15.45:0/3970146473 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:15:47.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.719+0000 7f35c3e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55826de14670 0x55826de1c9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:44638/0 (socket says 172.21.15.45:44638) 2024-04-16T17:15:47.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.719+0000 7f35d1400640 1 --2- 172.21.15.45:0/3970146473 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55826de1d250 0x55826de0ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:47.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.719+0000 7f35d0a00640 1 -- 172.21.15.45:0/3970146473 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55826de14670 msgr2=0x55826de1c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:15:47.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.719+0000 7f35d0a00640 1 --2- 172.21.15.45:0/3970146473 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55826de14670 0x55826de1c9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:47.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.719+0000 7f35d0a00640 1 -- 172.21.15.45:0/3970146473 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55826de1d250 msgr2=0x55826de0ffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:15:47.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.719+0000 7f35d0a00640 1 --2- 172.21.15.45:0/3970146473 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55826de1d250 0x55826de0ffc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:47.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.719+0000 7f35d0a00640 1 -- 172.21.15.45:0/3970146473 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55826de15ee0 con 0x55826de13d50 2024-04-16T17:15:47.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.720+0000 7f35d0a00640 1 --2- 172.21.15.45:0/3970146473 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55826de13d50 0x55826de28f00 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f35bc009be0 tx=0x7f35bc009930 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ad350c34b6972388 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:15:47.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.720+0000 7f35c1200640 1 -- 172.21.15.45:0/3970146473 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35bc028070 con 0x55826de13d50 2024-04-16T17:15:47.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.720+0000 7f35c1200640 1 -- 172.21.15.45:0/3970146473 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f35bc0052c0 con 0x55826de13d50 2024-04-16T17:15:47.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.720+0000 7f35c1200640 1 -- 172.21.15.45:0/3970146473 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35bc0035f0 con 0x55826de13d50 2024-04-16T17:15:47.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.720+0000 7f35d61f5740 1 -- 172.21.15.45:0/3970146473 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55826de13d50 msgr2=0x55826de28f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:15:47.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.720+0000 7f35d61f5740 1 --2- 172.21.15.45:0/3970146473 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55826de13d50 0x55826de28f00 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f35bc009be0 tx=0x7f35bc009930 comp rx=0 tx=0).stop 2024-04-16T17:15:47.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.721+0000 7f35d61f5740 1 -- 172.21.15.45:0/3970146473 shutdown_connections 2024-04-16T17:15:47.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.721+0000 7f35d61f5740 1 --2- 172.21.15.45:0/3970146473 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55826de1d250 0x55826de0ffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:47.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.721+0000 7f35d61f5740 1 --2- 172.21.15.45:0/3970146473 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55826de14670 0x55826de1c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:47.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.721+0000 7f35d61f5740 1 --2- 172.21.15.45:0/3970146473 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55826de13d50 0x55826de28f00 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:47.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.721+0000 7f35d61f5740 1 -- 172.21.15.45:0/3970146473 >> 172.21.15.45:0/3970146473 conn(0x55826de0e480 msgr2=0x55826de0f8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:15:47.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.721+0000 7f35d61f5740 1 -- 172.21.15.45:0/3970146473 shutdown_connections 2024-04-16T17:15:47.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.721+0000 7f35d61f5740 1 -- 172.21.15.45:0/3970146473 wait complete. 2024-04-16T17:15:47.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.721+0000 7f35d61f5740 1 -- start start 2024-04-16T17:15:47.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.721+0000 7f35d61f5740 1 Processor -- start 2024-04-16T17:15:47.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.722+0000 7f35d61f5740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55826de13d50 0x55826de25970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:47.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.722+0000 7f35d61f5740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55826de14670 0x55826de20ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:47.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.722+0000 7f35d61f5740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55826de1d250 0x55826de213d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:47.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.722+0000 7f35d61f5740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f35cc1afdb0 con 0x55826de14670 2024-04-16T17:15:47.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.722+0000 7f35d61f5740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f35cc1b1220 con 0x55826de1d250 2024-04-16T17:15:47.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.722+0000 7f35d61f5740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f35cc1b0480 con 0x55826de13d50 2024-04-16T17:15:47.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.722+0000 7f35d61f5740 10 client.0 _ll_register_callbacks cb 0x7f35b40d4680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:15:47.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.722+0000 7f35c3e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55826de14670 0x55826de20ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:47.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.722+0000 7f35c3e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55826de14670 0x55826de20ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:44260/0 (socket says 172.21.15.45:44260) 2024-04-16T17:15:47.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.722+0000 7f35c3e00640 1 -- 172.21.15.45:0/3919964938 learned_addr learned my addr 172.21.15.45:0/3919964938 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:15:47.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.722+0000 7f35d1400640 1 --2- 172.21.15.45:0/3919964938 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55826de1d250 0x55826de213d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:47.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.722+0000 7f35d0a00640 1 --2- 172.21.15.45:0/3919964938 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55826de13d50 0x55826de25970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:47.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.722+0000 7f35c3e00640 1 -- 172.21.15.45:0/3919964938 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55826de13d50 msgr2=0x55826de25970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:15:47.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.722+0000 7f35c3e00640 1 --2- 172.21.15.45:0/3919964938 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55826de13d50 0x55826de25970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:47.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.722+0000 7f35c3e00640 1 -- 172.21.15.45:0/3919964938 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55826de1d250 msgr2=0x55826de213d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:15:47.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.722+0000 7f35c3e00640 1 --2- 172.21.15.45:0/3919964938 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55826de1d250 0x55826de213d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:47.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.722+0000 7f35c3e00640 1 -- 172.21.15.45:0/3919964938 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55826da8b3f0 con 0x55826de14670 2024-04-16T17:15:47.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.723+0000 7f35c3e00640 1 --2- 172.21.15.45:0/3919964938 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55826de14670 0x55826de20ce0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f35b40a5150 tx=0x7f35b40d4b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3727246378f02b22 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:15:47.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.723+0000 7f35ba000640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-16T17:15:47.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.723+0000 7f35ba000640 1 -- 172.21.15.45:0/3919964938 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35b409c370 con 0x55826de14670 2024-04-16T17:15:47.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.723+0000 7f35ba000640 1 -- 172.21.15.45:0/3919964938 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9496+0+0 (secure 0 0 0) 0x7f35b409c070 con 0x55826de14670 2024-04-16T17:15:47.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.723+0000 7f35d61f5740 1 -- 172.21.15.45:0/3919964938 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f35bc0052c0 con 0x55826de14670 2024-04-16T17:15:47.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.723+0000 7f35ba000640 1 -- 172.21.15.45:0/3919964938 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f35b409c550 con 0x55826de14670 2024-04-16T17:15:47.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.723+0000 7f35d61f5740 10 client.17820 fetch_fsmap learned FSMap version 43 2024-04-16T17:15:47.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.723+0000 7f35d61f5740 1 -- 172.21.15.45:0/3919964938 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55826db557d0 con 0x55826de14670 2024-04-16T17:15:47.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.724+0000 7f35ba000640 1 -- 172.21.15.45:0/3919964938 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f35bc0052c0 con 0x55826de14670 2024-04-16T17:15:47.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.724+0000 7f35ba000640 1 -- 172.21.15.45:0/3919964938 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f35b4098760 con 0x55826de14670 2024-04-16T17:15:47.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.724+0000 7f35d61f5740 10 client.17820 fetch_fsmap finished waiting for FSMap version 43 2024-04-16T17:15:47.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.724+0000 7f35d61f5740 10 client.17820 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-04-16T17:15:47.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.724+0000 7f35d61f5740 20 client.17820 populate_metadata read hostname 'smithi045' 2024-04-16T17:15:47.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.724+0000 7f35d61f5740 1 --2- 172.21.15.45:0/3919964938 >> [v2:172.21.15.45:6840/3755310822,v1:172.21.15.45:6841/3755310822] conn(0x55826de940b0 0x55826de964d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:47.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.724+0000 7f35d61f5740 4 client.17820 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T17:15:47.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.724+0000 7f35d61f5740 1 -- 172.21.15.45:0/3919964938 --> [v2:172.21.15.45:6840/3755310822,v1:172.21.15.45:6841/3755310822] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55826dd808b0 con 0x55826de940b0 2024-04-16T17:15:47.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.724+0000 7f35d0a00640 1 --2- 172.21.15.45:0/3919964938 >> [v2:172.21.15.45:6840/3755310822,v1:172.21.15.45:6841/3755310822] conn(0x55826de940b0 0x55826de964d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:47.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.725+0000 7f35d0a00640 1 --2- 172.21.15.45:0/3919964938 >> [v2:172.21.15.45:6840/3755310822,v1:172.21.15.45:6841/3755310822] conn(0x55826de940b0 0x55826de964d0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5b198a57672e27d6 server_cookie=59d1d70457b0d788 in_seq=0 out_seq=0 2024-04-16T17:15:47.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.725+0000 7f35ba000640 10 client.17820 ms_handle_connect on v2:172.21.15.45:6840/3755310822 2024-04-16T17:15:47.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.725+0000 7f35ba000640 1 -- 172.21.15.45:0/3919964938 <== mds.1 v2:172.21.15.45:6840/3755310822 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55826dd808b0 con 0x55826de940b0 2024-04-16T17:15:47.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.725+0000 7f35ba000640 10 client.17820 handle_command_reply: tid=0 2024-04-16T17:15:47.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.725+0000 7f35d61f5740 1 client.17820 shutdown 2024-04-16T17:15:47.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.725+0000 7f35d61f5740 1 -- 172.21.15.45:0/3919964938 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55826de14670 msgr2=0x55826de20ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:15:47.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.725+0000 7f35d61f5740 1 --2- 172.21.15.45:0/3919964938 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55826de14670 0x55826de20ce0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f35b40a5150 tx=0x7f35b40d4b40 comp rx=0 tx=0).stop 2024-04-16T17:15:47.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.725+0000 7f35d61f5740 1 -- 172.21.15.45:0/3919964938 shutdown_connections 2024-04-16T17:15:47.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.725+0000 7f35d61f5740 1 --2- 172.21.15.45:0/3919964938 >> [v2:172.21.15.45:6840/3755310822,v1:172.21.15.45:6841/3755310822] conn(0x55826de940b0 0x55826de964d0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:47.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.725+0000 7f35d61f5740 1 --2- 172.21.15.45:0/3919964938 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55826de1d250 0x55826de213d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:47.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.725+0000 7f35d61f5740 1 --2- 172.21.15.45:0/3919964938 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55826de14670 0x55826de20ce0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:47.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.725+0000 7f35d61f5740 1 --2- 172.21.15.45:0/3919964938 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55826de13d50 0x55826de25970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:47.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.725+0000 7f35d61f5740 1 -- 172.21.15.45:0/3919964938 >> 172.21.15.45:0/3919964938 conn(0x55826de0e480 msgr2=0x55826de10140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:15:47.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.725+0000 7f35ba000640 0 client.17820 ms_handle_reset on v2:172.21.15.45:6840/3755310822 2024-04-16T17:15:47.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.725+0000 7f35d61f5740 1 -- 172.21.15.45:0/3919964938 shutdown_connections 2024-04-16T17:15:47.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.725+0000 7f35d61f5740 1 -- 172.21.15.45:0/3919964938 wait complete. 2024-04-16T17:15:47.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.726+0000 7f35d61f5740 20 client.17820 trim_cache size 0 max 16384 2024-04-16T17:15:47.733 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:15:47 smithi045 ceph-mon[31676]: from='client.? 172.21.15.45:0/2733113845' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:15:47.733 INFO:teuthology.orchestra.run.smithi045.stdout:[] 2024-04-16T17:15:47.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.726+0000 7f35d3400640 1 -- 172.21.15.45:0/1615486045 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f35b00a3da0 msgr2=0x7f35b00a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:15:47.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.726+0000 7f35d3400640 1 --2- 172.21.15.45:0/1615486045 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f35b00a3da0 0x7f35b00a6260 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f35c4005e30 tx=0x7f35c4005d80 comp rx=0 tx=0).stop 2024-04-16T17:15:47.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.726+0000 7f35d3400640 1 -- 172.21.15.45:0/1615486045 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f35cc109580 msgr2=0x7f35cc1a0cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:15:47.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.726+0000 7f35d3400640 1 --2- 172.21.15.45:0/1615486045 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f35cc109580 0x7f35cc1a0cc0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f35b400e9f0 tx=0x7f35b400eeb0 comp rx=0 tx=0).stop 2024-04-16T17:15:47.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.727+0000 7f35d3400640 1 -- 172.21.15.45:0/1615486045 shutdown_connections 2024-04-16T17:15:47.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.727+0000 7f35d3400640 1 --2- 172.21.15.45:0/1615486045 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f35b00a3da0 0x7f35b00a6260 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:47.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.727+0000 7f35d3400640 1 --2- 172.21.15.45:0/1615486045 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f35cc19a980 0x7f35cc19ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:47.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.727+0000 7f35d3400640 1 --2- 172.21.15.45:0/1615486045 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f35cc109580 0x7f35cc1a0cc0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:47.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.727+0000 7f35d3400640 1 --2- 172.21.15.45:0/1615486045 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f35cc1044b0 0x7f35cc1a0780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:47.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.727+0000 7f35d3400640 1 -- 172.21.15.45:0/1615486045 >> 172.21.15.45:0/1615486045 conn(0x7f35cc0ffd90 msgr2=0x7f35cc102110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:15:47.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.727+0000 7f35d3400640 1 -- 172.21.15.45:0/1615486045 shutdown_connections 2024-04-16T17:15:47.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.727+0000 7f35d3400640 1 -- 172.21.15.45:0/1615486045 wait complete. 2024-04-16T17:15:47.751 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-16T17:15:47.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:15:47 smithi104 ceph-mon[36457]: from='client.? 172.21.15.45:0/2733113845' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:15:47.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.971+0000 7fc79a200640 1 -- 172.21.15.45:0/2965466271 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc794074cc0 msgr2=0x7fc794112970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:15:47.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.971+0000 7fc79a200640 1 --2- 172.21.15.45:0/2965466271 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc794074cc0 0x7fc794112970 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fc780009900 tx=0x7fc78002bcc0 comp rx=0 tx=0).stop 2024-04-16T17:15:47.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.971+0000 7fc79a200640 1 -- 172.21.15.45:0/2965466271 shutdown_connections 2024-04-16T17:15:47.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.971+0000 7fc79a200640 1 --2- 172.21.15.45:0/2965466271 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc794112eb0 0x7fc7941152a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:47.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.971+0000 7fc79a200640 1 --2- 172.21.15.45:0/2965466271 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc794074cc0 0x7fc794112970 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:47.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.971+0000 7fc79a200640 1 --2- 172.21.15.45:0/2965466271 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc7940743a0 0x7fc794074780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:47.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.971+0000 7fc79a200640 1 -- 172.21.15.45:0/2965466271 >> 172.21.15.45:0/2965466271 conn(0x7fc7940fd9d0 msgr2=0x7fc7940ffdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:15:47.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.971+0000 7fc79a200640 1 -- 172.21.15.45:0/2965466271 shutdown_connections 2024-04-16T17:15:47.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.971+0000 7fc79a200640 1 -- 172.21.15.45:0/2965466271 wait complete. 2024-04-16T17:15:47.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.972+0000 7fc79a200640 1 Processor -- start 2024-04-16T17:15:47.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.973+0000 7fc79a200640 1 -- start start 2024-04-16T17:15:47.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.973+0000 7fc79a200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc7940743a0 0x7fc79419ee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:47.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.973+0000 7fc79a200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc794074cc0 0x7fc79419f360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:47.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.973+0000 7fc79a200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc794112eb0 0x7fc7941a36e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:47.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.973+0000 7fc79a200640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fc7941188a0 con 0x7fc7940743a0 2024-04-16T17:15:47.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.973+0000 7fc79a200640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fc794118720 con 0x7fc794074cc0 2024-04-16T17:15:47.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.973+0000 7fc79a200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fc794118a20 con 0x7fc794112eb0 2024-04-16T17:15:47.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.973+0000 7fc798e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc7940743a0 0x7fc79419ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:47.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.973+0000 7fc798e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc7940743a0 0x7fc79419ee20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:44278/0 (socket says 172.21.15.45:44278) 2024-04-16T17:15:47.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.974+0000 7fc798e00640 1 -- 172.21.15.45:0/193691981 learned_addr learned my addr 172.21.15.45:0/193691981 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:15:47.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.974+0000 7fc793e00640 1 --2- 172.21.15.45:0/193691981 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc794074cc0 0x7fc79419f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:47.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.974+0000 7fc799800640 1 --2- 172.21.15.45:0/193691981 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc794112eb0 0x7fc7941a36e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:47.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.974+0000 7fc793e00640 1 -- 172.21.15.45:0/193691981 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc794112eb0 msgr2=0x7fc7941a36e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:15:47.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.974+0000 7fc793e00640 1 --2- 172.21.15.45:0/193691981 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc794112eb0 0x7fc7941a36e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:47.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.974+0000 7fc793e00640 1 -- 172.21.15.45:0/193691981 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc7940743a0 msgr2=0x7fc79419ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:15:47.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.974+0000 7fc793e00640 1 --2- 172.21.15.45:0/193691981 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc7940743a0 0x7fc79419ee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:47.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.974+0000 7fc793e00640 1 -- 172.21.15.45:0/193691981 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7941a3e10 con 0x7fc794074cc0 2024-04-16T17:15:47.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.974+0000 7fc798e00640 1 --2- 172.21.15.45:0/193691981 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc7940743a0 0x7fc79419ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:15:47.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.974+0000 7fc793e00640 1 --2- 172.21.15.45:0/193691981 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc794074cc0 0x7fc79419f360 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fc780000c00 tx=0x7fc78002cbc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:15:47.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.975+0000 7fc791600640 1 -- 172.21.15.45:0/193691981 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc780043020 con 0x7fc794074cc0 2024-04-16T17:15:47.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.975+0000 7fc791600640 1 -- 172.21.15.45:0/193691981 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc7800054e0 con 0x7fc794074cc0 2024-04-16T17:15:47.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.975+0000 7fc791600640 1 -- 172.21.15.45:0/193691981 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc780003a90 con 0x7fc794074cc0 2024-04-16T17:15:47.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.975+0000 7fc79a200640 1 -- 172.21.15.45:0/193691981 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7941a40a0 con 0x7fc794074cc0 2024-04-16T17:15:47.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.975+0000 7fc79a200640 1 -- 172.21.15.45:0/193691981 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7941ab990 con 0x7fc794074cc0 2024-04-16T17:15:47.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.978+0000 7fc79e5c5740 1 Processor -- start 2024-04-16T17:15:47.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.978+0000 7fc79e5c5740 1 -- start start 2024-04-16T17:15:47.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.978+0000 7fc79e5c5740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ae5eb2af10 0x55ae5eb2b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:47.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.978+0000 7fc79e5c5740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ae5eb2b830 0x55ae5eb33ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:47.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.979+0000 7fc79e5c5740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ae5eb34410 0x55ae5eb36800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:47.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.979+0000 7fc79e5c5740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55ae5ea90020 con 0x55ae5eb34410 2024-04-16T17:15:47.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.979+0000 7fc79e5c5740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55ae5ea36460 con 0x55ae5eb2b830 2024-04-16T17:15:47.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.979+0000 7fc79e5c5740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55ae5ea8bfe0 con 0x55ae5eb2af10 2024-04-16T17:15:47.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.979+0000 7fc791600640 1 -- 172.21.15.45:0/193691981 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fc780046020 con 0x7fc794074cc0 2024-04-16T17:15:47.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.979+0000 7fc799800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ae5eb2af10 0x55ae5eb2b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:47.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.979+0000 7fc799800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ae5eb2af10 0x55ae5eb2b2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:34094/0 (socket says 172.21.15.45:34094) 2024-04-16T17:15:47.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.979+0000 7fc798e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ae5eb2b830 0x55ae5eb33ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:47.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.979+0000 7fc799800640 1 -- 172.21.15.45:0/3718201399 learned_addr learned my addr 172.21.15.45:0/3718201399 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:15:47.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.979+0000 7fc793e00640 1 --2- 172.21.15.45:0/3718201399 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ae5eb34410 0x55ae5eb36800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:47.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.979+0000 7fc798e00640 1 -- 172.21.15.45:0/3718201399 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ae5eb2af10 msgr2=0x55ae5eb2b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:15:47.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.979+0000 7fc798e00640 1 --2- 172.21.15.45:0/3718201399 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ae5eb2af10 0x55ae5eb2b2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:47.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.979+0000 7fc798e00640 1 -- 172.21.15.45:0/3718201399 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ae5eb34410 msgr2=0x55ae5eb36800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:15:47.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.979+0000 7fc798e00640 1 --2- 172.21.15.45:0/3718201399 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ae5eb34410 0x55ae5eb36800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:47.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.979+0000 7fc798e00640 1 -- 172.21.15.45:0/3718201399 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ae5ea3e040 con 0x55ae5eb2b830 2024-04-16T17:15:47.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.980+0000 7fc798e00640 1 --2- 172.21.15.45:0/3718201399 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ae5eb2b830 0x55ae5eb33ed0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fc788011510 tx=0x7fc7880119d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7926a4814b5b9673 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:15:47.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.980+0000 7fc786000640 1 -- 172.21.15.45:0/3718201399 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc78801e070 con 0x55ae5eb2b830 2024-04-16T17:15:47.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.980+0000 7fc786000640 1 -- 172.21.15.45:0/3718201399 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc78800faa0 con 0x55ae5eb2b830 2024-04-16T17:15:47.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.980+0000 7fc786000640 1 -- 172.21.15.45:0/3718201399 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc788019bd0 con 0x55ae5eb2b830 2024-04-16T17:15:47.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.981+0000 7fc791600640 1 --2- 172.21.15.45:0/193691981 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fc77c0a3d20 0x7fc77c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:47.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.981+0000 7fc791600640 1 -- 172.21.15.45:0/193691981 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9496+0+0 (secure 0 0 0) 0x7fc7800f2930 con 0x7fc794074cc0 2024-04-16T17:15:47.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.981+0000 7fc79e5c5740 1 -- 172.21.15.45:0/3718201399 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ae5eb2b830 msgr2=0x55ae5eb33ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:15:47.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.981+0000 7fc79e5c5740 1 --2- 172.21.15.45:0/3718201399 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ae5eb2b830 0x55ae5eb33ed0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fc788011510 tx=0x7fc7880119d0 comp rx=0 tx=0).stop 2024-04-16T17:15:47.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.981+0000 7fc799800640 1 --2- 172.21.15.45:0/193691981 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fc77c0a3d20 0x7fc77c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:47.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.981+0000 7fc79e5c5740 1 -- 172.21.15.45:0/3718201399 shutdown_connections 2024-04-16T17:15:47.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.981+0000 7fc79e5c5740 1 --2- 172.21.15.45:0/3718201399 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ae5eb34410 0x55ae5eb36800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:47.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.981+0000 7fc79e5c5740 1 --2- 172.21.15.45:0/3718201399 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ae5eb2b830 0x55ae5eb33ed0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:47.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.981+0000 7fc79e5c5740 1 --2- 172.21.15.45:0/3718201399 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ae5eb2af10 0x55ae5eb2b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:47.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.981+0000 7fc79e5c5740 1 -- 172.21.15.45:0/3718201399 >> 172.21.15.45:0/3718201399 conn(0x55ae5eb25640 msgr2=0x55ae5eb25a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:15:47.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.981+0000 7fc79e5c5740 1 -- 172.21.15.45:0/3718201399 shutdown_connections 2024-04-16T17:15:47.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.981+0000 7fc79e5c5740 1 -- 172.21.15.45:0/3718201399 wait complete. 2024-04-16T17:15:47.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.982+0000 7fc79e5c5740 1 -- start start 2024-04-16T17:15:47.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.982+0000 7fc799800640 1 --2- 172.21.15.45:0/193691981 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fc77c0a3d20 0x7fc77c0a61e0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fc78c005fd0 tx=0x7fc78c009500 comp rx=0 tx=0).ready entity=mgr.25411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:15:47.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.982+0000 7fc79e5c5740 1 Processor -- start 2024-04-16T17:15:47.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.982+0000 7fc79e5c5740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ae5eb2af10 0x55ae5eb314e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:47.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.982+0000 7fc79e5c5740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ae5eb2b830 0x55ae5eb31a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:47.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.982+0000 7fc79e5c5740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ae5eb34410 0x55ae5eb31f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:47.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.982+0000 7fc79e5c5740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55ae5eb37200 con 0x55ae5eb2b830 2024-04-16T17:15:47.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.982+0000 7fc79e5c5740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55ae5ea49830 con 0x55ae5eb2af10 2024-04-16T17:15:47.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.982+0000 7fc79e5c5740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55ae5eb37380 con 0x55ae5eb34410 2024-04-16T17:15:47.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.983+0000 7fc79e5c5740 10 client.0 _ll_register_callbacks cb 0x55ae5ea51150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:15:47.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.983+0000 7fc793e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ae5eb2b830 0x55ae5eb31a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:47.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.983+0000 7fc793e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ae5eb2b830 0x55ae5eb31a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:44296/0 (socket says 172.21.15.45:44296) 2024-04-16T17:15:47.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.983+0000 7fc793e00640 1 -- 172.21.15.45:0/118105176 learned_addr learned my addr 172.21.15.45:0/118105176 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:15:47.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.983+0000 7fc798e00640 1 --2- 172.21.15.45:0/118105176 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ae5eb2af10 0x55ae5eb314e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:47.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.983+0000 7fc799800640 1 --2- 172.21.15.45:0/118105176 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ae5eb34410 0x55ae5eb31f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:47.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.983+0000 7fc793e00640 1 -- 172.21.15.45:0/118105176 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ae5eb34410 msgr2=0x55ae5eb31f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:15:47.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.983+0000 7fc793e00640 1 --2- 172.21.15.45:0/118105176 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ae5eb34410 0x55ae5eb31f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:47.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.983+0000 7fc793e00640 1 -- 172.21.15.45:0/118105176 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ae5eb2af10 msgr2=0x55ae5eb314e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:15:47.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.983+0000 7fc793e00640 1 --2- 172.21.15.45:0/118105176 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ae5eb2af10 0x55ae5eb314e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:47.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.983+0000 7fc793e00640 1 -- 172.21.15.45:0/118105176 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ae5ea4d920 con 0x55ae5eb2b830 2024-04-16T17:15:47.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.983+0000 7fc799800640 1 --2- 172.21.15.45:0/118105176 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ae5eb34410 0x55ae5eb31f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:15:47.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.983+0000 7fc798e00640 1 --2- 172.21.15.45:0/118105176 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ae5eb2af10 0x55ae5eb314e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:15:47.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.983+0000 7fc793e00640 1 --2- 172.21.15.45:0/118105176 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ae5eb2b830 0x55ae5eb31a20 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fc780006010 tx=0x7fc7800ba250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a86ef85df92e9f22 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:15:47.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.983+0000 7fc77aa00640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-16T17:15:47.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.984+0000 7fc77aa00640 1 -- 172.21.15.45:0/118105176 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7800bdf70 con 0x55ae5eb2b830 2024-04-16T17:15:47.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.984+0000 7fc77aa00640 1 -- 172.21.15.45:0/118105176 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9496+0+0 (secure 0 0 0) 0x7fc7800c2590 con 0x55ae5eb2b830 2024-04-16T17:15:47.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.984+0000 7fc79e5c5740 1 -- 172.21.15.45:0/118105176 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc78800faa0 con 0x55ae5eb2b830 2024-04-16T17:15:47.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.984+0000 7fc77aa00640 1 -- 172.21.15.45:0/118105176 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fc780004c80 con 0x55ae5eb2b830 2024-04-16T17:15:47.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.984+0000 7fc79e5c5740 10 client.17832 fetch_fsmap learned FSMap version 43 2024-04-16T17:15:47.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.984+0000 7fc79e5c5740 1 -- 172.21.15.45:0/118105176 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55ae5ea1b7f0 con 0x55ae5eb2b830 2024-04-16T17:15:47.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.984+0000 7fc77aa00640 1 -- 172.21.15.45:0/118105176 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc78800faa0 con 0x55ae5eb2b830 2024-04-16T17:15:47.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.984+0000 7fc77aa00640 1 -- 172.21.15.45:0/118105176 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7fc7800be870 con 0x55ae5eb2b830 2024-04-16T17:15:47.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.984+0000 7fc79e5c5740 10 client.17832 fetch_fsmap finished waiting for FSMap version 43 2024-04-16T17:15:47.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.984+0000 7fc79e5c5740 10 client.17832 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:15:47.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.984+0000 7fc79e5c5740 20 client.17832 populate_metadata read hostname 'smithi045' 2024-04-16T17:15:47.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.985+0000 7fc79e5c5740 1 --2- 172.21.15.45:0/118105176 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55ae5eb44770 0x55ae5eb46b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:47.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.985+0000 7fc79e5c5740 4 client.17832 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:15:47.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.985+0000 7fc79e5c5740 1 -- 172.21.15.45:0/118105176 --> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ae5ea9da40 con 0x55ae5eb44770 2024-04-16T17:15:47.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.985+0000 7fc798e00640 1 --2- 172.21.15.45:0/118105176 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55ae5eb44770 0x55ae5eb46b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:47.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.985+0000 7fc798e00640 1 --2- 172.21.15.45:0/118105176 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55ae5eb44770 0x55ae5eb46b60 crc :-1 s=READY pgs=235 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f3e7dff8a8a6b3a7 server_cookie=cd1834aaabf5dba6 in_seq=0 out_seq=0 2024-04-16T17:15:47.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.985+0000 7fc77aa00640 10 client.17832 ms_handle_connect on v2:172.21.15.104:6838/3660070592 2024-04-16T17:15:47.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.986+0000 7fc77aa00640 1 -- 172.21.15.45:0/118105176 <== mds.0 v2:172.21.15.104:6838/3660070592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ae5ea9da40 con 0x55ae5eb44770 2024-04-16T17:15:47.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.986+0000 7fc77aa00640 10 client.17832 handle_command_reply: tid=0 2024-04-16T17:15:47.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.986+0000 7fc79e5c5740 1 client.17832 shutdown 2024-04-16T17:15:47.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.987+0000 7fc79e5c5740 1 -- 172.21.15.45:0/118105176 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ae5eb2b830 msgr2=0x55ae5eb31a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:15:47.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.987+0000 7fc79e5c5740 1 --2- 172.21.15.45:0/118105176 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ae5eb2b830 0x55ae5eb31a20 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fc780006010 tx=0x7fc7800ba250 comp rx=0 tx=0).stop 2024-04-16T17:15:47.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.987+0000 7fc79e5c5740 1 -- 172.21.15.45:0/118105176 shutdown_connections 2024-04-16T17:15:47.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.987+0000 7fc79e5c5740 1 --2- 172.21.15.45:0/118105176 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55ae5eb44770 0x55ae5eb46b60 crc :-1 s=READY pgs=235 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:47.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.987+0000 7fc79e5c5740 1 --2- 172.21.15.45:0/118105176 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ae5eb34410 0x55ae5eb31f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:47.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.987+0000 7fc79e5c5740 1 --2- 172.21.15.45:0/118105176 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ae5eb2b830 0x55ae5eb31a20 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:47.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.987+0000 7fc79e5c5740 1 --2- 172.21.15.45:0/118105176 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ae5eb2af10 0x55ae5eb314e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:47.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.987+0000 7fc77aa00640 0 client.17832 ms_handle_reset on v2:172.21.15.104:6838/3660070592 2024-04-16T17:15:47.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.987+0000 7fc79e5c5740 1 -- 172.21.15.45:0/118105176 >> 172.21.15.45:0/118105176 conn(0x55ae5eb25640 msgr2=0x55ae5eb27390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:15:47.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.987+0000 7fc79e5c5740 1 -- 172.21.15.45:0/118105176 shutdown_connections 2024-04-16T17:15:47.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.987+0000 7fc79e5c5740 1 -- 172.21.15.45:0/118105176 wait complete. 2024-04-16T17:15:47.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:47.987+0000 7fc79e5c5740 20 client.17832 trim_cache size 0 max 16384 2024-04-16T17:15:48.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:48.001+0000 7fc79e5c5740 1 Processor -- start 2024-04-16T17:15:48.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:48.001+0000 7fc79e5c5740 1 -- start start 2024-04-16T17:15:48.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:48.002+0000 7fc79e5c5740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ae5eb2af10 0x55ae5eb33c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:48.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:48.002+0000 7fc79e5c5740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ae5eb2b830 0x55ae5eb400c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:48.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:48.002+0000 7fc79e5c5740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ae5eb34410 0x55ae5eb24580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:48.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:48.002+0000 7fc79e5c5740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fc7941b8230 con 0x55ae5eb2af10 2024-04-16T17:15:48.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:48.002+0000 7fc79e5c5740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fc7941b5b00 con 0x55ae5eb2b830 2024-04-16T17:15:48.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:48.002+0000 7fc79e5c5740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fc7941b3970 con 0x55ae5eb34410 2024-04-16T17:15:48.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:48.002+0000 7fc798e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ae5eb2af10 0x55ae5eb33c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:48.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:48.002+0000 7fc798e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ae5eb2af10 0x55ae5eb33c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:44304/0 (socket says 172.21.15.45:44304) 2024-04-16T17:15:48.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:48.002+0000 7fc798e00640 1 -- 172.21.15.45:0/2261666346 learned_addr learned my addr 172.21.15.45:0/2261666346 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:15:48.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:48.002+0000 7fc793e00640 1 --2- 172.21.15.45:0/2261666346 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ae5eb2b830 0x55ae5eb400c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:48.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:48.002+0000 7fc799800640 1 --2- 172.21.15.45:0/2261666346 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ae5eb34410 0x55ae5eb24580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:48.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:48.002+0000 7fc798e00640 1 -- 172.21.15.45:0/2261666346 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ae5eb34410 msgr2=0x55ae5eb24580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:15:48.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:48.002+0000 7fc798e00640 1 --2- 172.21.15.45:0/2261666346 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ae5eb34410 0x55ae5eb24580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:48.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:48.002+0000 7fc798e00640 1 -- 172.21.15.45:0/2261666346 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ae5eb2b830 msgr2=0x55ae5eb400c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:15:48.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:48.002+0000 7fc798e00640 1 --2- 172.21.15.45:0/2261666346 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ae5eb2b830 0x55ae5eb400c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:48.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:48.002+0000 7fc798e00640 1 -- 172.21.15.45:0/2261666346 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ae5eb46ed0 con 0x55ae5eb2af10 2024-04-16T17:15:48.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:48.002+0000 7fc793e00640 1 --2- 172.21.15.45:0/2261666346 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ae5eb2b830 0x55ae5eb400c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:15:48.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:48.002+0000 7fc799800640 1 --2- 172.21.15.45:0/2261666346 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ae5eb34410 0x55ae5eb24580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:15:48.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:48.003+0000 7fc798e00640 1 --2- 172.21.15.45:0/2261666346 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ae5eb2af10 0x55ae5eb33c10 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fc788011510 tx=0x7fc7880196c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3eba6663844746e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:15:48.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:48.003+0000 7fc785600640 1 -- 172.21.15.45:0/2261666346 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc788031070 con 0x55ae5eb2af10 2024-04-16T17:15:48.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:48.003+0000 7fc785600640 1 -- 172.21.15.45:0/2261666346 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc78802a070 con 0x55ae5eb2af10 2024-04-16T17:15:48.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:48.003+0000 7fc785600640 1 -- 172.21.15.45:0/2261666346 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc78801e040 con 0x55ae5eb2af10 2024-04-16T17:15:48.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:48.004+0000 7fc79e5c5740 1 -- 172.21.15.45:0/2261666346 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ae5eb2af10 msgr2=0x55ae5eb33c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:15:48.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:48.004+0000 7fc79e5c5740 1 --2- 172.21.15.45:0/2261666346 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ae5eb2af10 0x55ae5eb33c10 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fc788011510 tx=0x7fc7880196c0 comp rx=0 tx=0).stop 2024-04-16T17:15:48.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:48.004+0000 7fc79e5c5740 1 -- 172.21.15.45:0/2261666346 shutdown_connections 2024-04-16T17:15:48.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:48.004+0000 7fc79e5c5740 1 --2- 172.21.15.45:0/2261666346 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ae5eb34410 0x55ae5eb24580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:48.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:48.004+0000 7fc79e5c5740 1 --2- 172.21.15.45:0/2261666346 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ae5eb2b830 0x55ae5eb400c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:48.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:48.004+0000 7fc79e5c5740 1 --2- 172.21.15.45:0/2261666346 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ae5eb2af10 0x55ae5eb33c10 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:48.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:48.004+0000 7fc79e5c5740 1 -- 172.21.15.45:0/2261666346 >> 172.21.15.45:0/2261666346 conn(0x55ae5eb25640 msgr2=0x55ae5eb23e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:15:48.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:48.004+0000 7fc79e5c5740 1 -- 172.21.15.45:0/2261666346 shutdown_connections 2024-04-16T17:15:48.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:48.004+0000 7fc79e5c5740 1 -- 172.21.15.45:0/2261666346 wait complete. 2024-04-16T17:15:48.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:48.004+0000 7fc79e5c5740 1 -- start start 2024-04-16T17:15:48.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:48.004+0000 7fc79e5c5740 1 Processor -- start 2024-04-16T17:15:48.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:48.005+0000 7fc79e5c5740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ae5eb2af10 0x55ae5eb380e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:48.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:48.005+0000 7fc79e5c5740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ae5eb2b830 0x55ae5eb38620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:48.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:48.005+0000 7fc79e5c5740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ae5eb34410 0x55ae5eb38b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:48.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:48.005+0000 7fc79e5c5740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fc7941b3fc0 con 0x55ae5eb2b830 2024-04-16T17:15:48.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:48.005+0000 7fc79e5c5740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fc7941b5430 con 0x55ae5eb2af10 2024-04-16T17:15:48.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:48.005+0000 7fc79e5c5740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fc7941b4690 con 0x55ae5eb34410 2024-04-16T17:15:48.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:48.005+0000 7fc79e5c5740 10 client.0 _ll_register_callbacks cb 0x7fc78002cdb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:15:48.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:48.005+0000 7fc793e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ae5eb2b830 0x55ae5eb38620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:48.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:48.005+0000 7fc798e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ae5eb2af10 0x55ae5eb380e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:48.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:48.005+0000 7fc793e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ae5eb2b830 0x55ae5eb38620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:44308/0 (socket says 172.21.15.45:44308) 2024-04-16T17:15:48.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:48.005+0000 7fc793e00640 1 -- 172.21.15.45:0/3865851827 learned_addr learned my addr 172.21.15.45:0/3865851827 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:15:48.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:48.005+0000 7fc799800640 1 --2- 172.21.15.45:0/3865851827 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ae5eb34410 0x55ae5eb38b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:48.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:48.005+0000 7fc793e00640 1 -- 172.21.15.45:0/3865851827 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ae5eb34410 msgr2=0x55ae5eb38b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:15:48.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:48.005+0000 7fc793e00640 1 --2- 172.21.15.45:0/3865851827 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ae5eb34410 0x55ae5eb38b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:48.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:48.005+0000 7fc793e00640 1 -- 172.21.15.45:0/3865851827 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ae5eb2af10 msgr2=0x55ae5eb380e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:15:48.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:48.005+0000 7fc793e00640 1 --2- 172.21.15.45:0/3865851827 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ae5eb2af10 0x55ae5eb380e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:48.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:48.005+0000 7fc793e00640 1 -- 172.21.15.45:0/3865851827 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ae5e7a23f0 con 0x55ae5eb2b830 2024-04-16T17:15:48.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:48.006+0000 7fc798e00640 1 --2- 172.21.15.45:0/3865851827 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ae5eb2af10 0x55ae5eb380e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:15:48.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:48.006+0000 7fc793e00640 1 --2- 172.21.15.45:0/3865851827 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ae5eb2b830 0x55ae5eb38620 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fc78002c200 tx=0x7fc780004460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=562cbfd1cf1b54f server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:15:48.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:48.006+0000 7fc77aa00640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-16T17:15:48.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:48.006+0000 7fc77aa00640 1 -- 172.21.15.45:0/3865851827 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7800db030 con 0x55ae5eb2b830 2024-04-16T17:15:48.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:48.006+0000 7fc77aa00640 1 -- 172.21.15.45:0/3865851827 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9496+0+0 (secure 0 0 0) 0x7fc7800c7000 con 0x55ae5eb2b830 2024-04-16T17:15:48.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:48.006+0000 7fc79e5c5740 1 -- 172.21.15.45:0/3865851827 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc78802a070 con 0x55ae5eb2b830 2024-04-16T17:15:48.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:48.006+0000 7fc77aa00640 1 -- 172.21.15.45:0/3865851827 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fc7800a5050 con 0x55ae5eb2b830 2024-04-16T17:15:48.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:48.006+0000 7fc79e5c5740 10 client.17838 fetch_fsmap learned FSMap version 43 2024-04-16T17:15:48.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:48.006+0000 7fc79e5c5740 1 -- 172.21.15.45:0/3865851827 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55ae5e86c7d0 con 0x55ae5eb2b830 2024-04-16T17:15:48.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:48.007+0000 7fc77aa00640 1 -- 172.21.15.45:0/3865851827 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc78802a070 con 0x55ae5eb2b830 2024-04-16T17:15:48.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:48.007+0000 7fc77aa00640 1 -- 172.21.15.45:0/3865851827 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7fc7800dd2c0 con 0x55ae5eb2b830 2024-04-16T17:15:48.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:48.007+0000 7fc79e5c5740 10 client.17838 fetch_fsmap finished waiting for FSMap version 43 2024-04-16T17:15:48.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:48.007+0000 7fc79e5c5740 10 client.17838 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:15:48.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:48.007+0000 7fc79e5c5740 20 client.17838 populate_metadata read hostname 'smithi045' 2024-04-16T17:15:48.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:48.007+0000 7fc79e5c5740 1 --2- 172.21.15.45:0/3865851827 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55ae5ebab370 0x55ae5ebad790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:48.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:48.007+0000 7fc79e5c5740 4 client.17838 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T17:15:48.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:48.007+0000 7fc79e5c5740 1 -- 172.21.15.45:0/3865851827 --> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ae5ea978b0 con 0x55ae5ebab370 2024-04-16T17:15:48.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:48.007+0000 7fc798e00640 1 --2- 172.21.15.45:0/3865851827 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55ae5ebab370 0x55ae5ebad790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:48.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:48.008+0000 7fc798e00640 1 --2- 172.21.15.45:0/3865851827 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55ae5ebab370 0x55ae5ebad790 crc :-1 s=READY pgs=236 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2ee8b5dd24555c4c server_cookie=f15787d08dd0bd76 in_seq=0 out_seq=0 2024-04-16T17:15:48.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:48.008+0000 7fc77aa00640 10 client.17838 ms_handle_connect on v2:172.21.15.104:6838/3660070592 2024-04-16T17:15:48.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:48.008+0000 7fc77aa00640 1 -- 172.21.15.45:0/3865851827 <== mds.0 v2:172.21.15.104:6838/3660070592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ae5ea978b0 con 0x55ae5ebab370 2024-04-16T17:15:48.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:48.008+0000 7fc77aa00640 10 client.17838 handle_command_reply: tid=0 2024-04-16T17:15:48.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:48.008+0000 7fc79e5c5740 1 client.17838 shutdown 2024-04-16T17:15:48.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:48.008+0000 7fc79e5c5740 1 -- 172.21.15.45:0/3865851827 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ae5eb2b830 msgr2=0x55ae5eb38620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:15:48.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:48.008+0000 7fc79e5c5740 1 --2- 172.21.15.45:0/3865851827 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ae5eb2b830 0x55ae5eb38620 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fc78002c200 tx=0x7fc780004460 comp rx=0 tx=0).stop 2024-04-16T17:15:48.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:48.008+0000 7fc79e5c5740 1 -- 172.21.15.45:0/3865851827 shutdown_connections 2024-04-16T17:15:48.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:48.008+0000 7fc79e5c5740 1 --2- 172.21.15.45:0/3865851827 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55ae5ebab370 0x55ae5ebad790 crc :-1 s=READY pgs=236 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:48.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:48.008+0000 7fc79e5c5740 1 --2- 172.21.15.45:0/3865851827 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ae5eb34410 0x55ae5eb38b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:48.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:48.008+0000 7fc79e5c5740 1 --2- 172.21.15.45:0/3865851827 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ae5eb2b830 0x55ae5eb38620 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:48.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:48.008+0000 7fc79e5c5740 1 --2- 172.21.15.45:0/3865851827 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ae5eb2af10 0x55ae5eb380e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:48.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:48.008+0000 7fc77aa00640 0 client.17838 ms_handle_reset on v2:172.21.15.104:6838/3660070592 2024-04-16T17:15:48.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:48.008+0000 7fc79e5c5740 1 -- 172.21.15.45:0/3865851827 >> 172.21.15.45:0/3865851827 conn(0x55ae5eb25640 msgr2=0x55ae5eb2f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:15:48.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:48.008+0000 7fc79e5c5740 1 -- 172.21.15.45:0/3865851827 shutdown_connections 2024-04-16T17:15:48.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:48.008+0000 7fc79e5c5740 1 -- 172.21.15.45:0/3865851827 wait complete. 2024-04-16T17:15:48.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:48.009+0000 7fc79e5c5740 20 client.17838 trim_cache size 0 max 16384 2024-04-16T17:15:48.016 INFO:teuthology.orchestra.run.smithi045.stdout:[] 2024-04-16T17:15:48.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:48.009+0000 7fc79a200640 1 -- 172.21.15.45:0/193691981 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fc77c0a3d20 msgr2=0x7fc77c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:15:48.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:48.009+0000 7fc79a200640 1 --2- 172.21.15.45:0/193691981 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fc77c0a3d20 0x7fc77c0a61e0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fc78c005fd0 tx=0x7fc78c009500 comp rx=0 tx=0).stop 2024-04-16T17:15:48.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:48.009+0000 7fc79a200640 1 -- 172.21.15.45:0/193691981 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc794074cc0 msgr2=0x7fc79419f360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:15:48.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:48.009+0000 7fc79a200640 1 --2- 172.21.15.45:0/193691981 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc794074cc0 0x7fc79419f360 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fc780000c00 tx=0x7fc78002cbc0 comp rx=0 tx=0).stop 2024-04-16T17:15:48.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:48.009+0000 7fc79a200640 1 -- 172.21.15.45:0/193691981 shutdown_connections 2024-04-16T17:15:48.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:48.009+0000 7fc79a200640 1 --2- 172.21.15.45:0/193691981 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fc77c0a3d20 0x7fc77c0a61e0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:48.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:48.009+0000 7fc79a200640 1 --2- 172.21.15.45:0/193691981 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc794112eb0 0x7fc7941a36e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:48.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:48.009+0000 7fc79a200640 1 --2- 172.21.15.45:0/193691981 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc794074cc0 0x7fc79419f360 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:48.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:48.009+0000 7fc79a200640 1 --2- 172.21.15.45:0/193691981 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc7940743a0 0x7fc79419ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:48.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:48.009+0000 7fc79a200640 1 -- 172.21.15.45:0/193691981 >> 172.21.15.45:0/193691981 conn(0x7fc7940fd9d0 msgr2=0x7fc7940fe740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:15:48.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:48.009+0000 7fc79a200640 1 -- 172.21.15.45:0/193691981 shutdown_connections 2024-04-16T17:15:48.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:48.009+0000 7fc79a200640 1 -- 172.21.15.45:0/193691981 wait complete. 2024-04-16T17:15:48.054 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:15:47 smithi117 ceph-mon[36730]: from='client.? 172.21.15.45:0/2733113845' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:15:48.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:15:48 smithi104 ceph-mon[36457]: pgmap v281: 129 pgs: 129 active+clean; 60 MiB data, 880 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-16T17:15:48.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:15:48 smithi045 ceph-mon[31676]: pgmap v281: 129 pgs: 129 active+clean; 60 MiB data, 880 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-16T17:15:49.027 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-16T17:15:49.027 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T17:15:49.027 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-16T17:15:49.054 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:15:48 smithi117 ceph-mon[36730]: pgmap v281: 129 pgs: 129 active+clean; 60 MiB data, 880 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-16T17:15:49.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.234+0000 7f2ef4e00640 1 -- 172.21.15.45:0/1258879721 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2ef010e660 msgr2=0x7f2ef010ea40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:15:49.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.234+0000 7f2ef4e00640 1 --2- 172.21.15.45:0/1258879721 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2ef010e660 0x7f2ef010ea40 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f2ee0009930 tx=0x7f2ee002bc30 comp rx=0 tx=0).stop 2024-04-16T17:15:49.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.234+0000 7f2ef4e00640 1 -- 172.21.15.45:0/1258879721 shutdown_connections 2024-04-16T17:15:49.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.234+0000 7f2ef4e00640 1 --2- 172.21.15.45:0/1258879721 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2ef0108fd0 0x7f2ef0115490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:49.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.234+0000 7f2ef4e00640 1 --2- 172.21.15.45:0/1258879721 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2ef0108610 0x7f2ef0108a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:49.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.234+0000 7f2ef4e00640 1 --2- 172.21.15.45:0/1258879721 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2ef010e660 0x7f2ef010ea40 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:49.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.234+0000 7f2ef4e00640 1 -- 172.21.15.45:0/1258879721 >> 172.21.15.45:0/1258879721 conn(0x7f2ef0076540 msgr2=0x7f2ef0078960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:15:49.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.234+0000 7f2ef4e00640 1 -- 172.21.15.45:0/1258879721 shutdown_connections 2024-04-16T17:15:49.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.234+0000 7f2ef4e00640 1 -- 172.21.15.45:0/1258879721 wait complete. 2024-04-16T17:15:49.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.235+0000 7f2ef4e00640 1 Processor -- start 2024-04-16T17:15:49.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.236+0000 7f2ef4e00640 1 -- start start 2024-04-16T17:15:49.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.236+0000 7f2ef4e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2ef0108610 0x7f2ef019ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:49.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.236+0000 7f2ef4e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2ef0108fd0 0x7f2ef019f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:49.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.236+0000 7f2ef4e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2ef010e660 0x7f2ef01a3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:49.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.236+0000 7f2ef4e00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f2ef01189e0 con 0x7f2ef010e660 2024-04-16T17:15:49.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.236+0000 7f2ef4e00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f2ef0118860 con 0x7f2ef0108fd0 2024-04-16T17:15:49.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.236+0000 7f2ef4e00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f2ef0118b60 con 0x7f2ef0108610 2024-04-16T17:15:49.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.237+0000 7f2eede00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2ef0108610 0x7f2ef019ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:49.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.237+0000 7f2eee800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2ef010e660 0x7f2ef01a3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:49.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.237+0000 7f2eed400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2ef0108fd0 0x7f2ef019f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:49.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.237+0000 7f2eede00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2ef0108610 0x7f2ef019ee80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:34144/0 (socket says 172.21.15.45:34144) 2024-04-16T17:15:49.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.237+0000 7f2eede00640 1 -- 172.21.15.45:0/1840218909 learned_addr learned my addr 172.21.15.45:0/1840218909 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:15:49.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.237+0000 7f2eede00640 1 -- 172.21.15.45:0/1840218909 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2ef0108fd0 msgr2=0x7f2ef019f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:15:49.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.237+0000 7f2eede00640 1 --2- 172.21.15.45:0/1840218909 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2ef0108fd0 0x7f2ef019f3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:49.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.237+0000 7f2eede00640 1 -- 172.21.15.45:0/1840218909 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2ef010e660 msgr2=0x7f2ef01a3740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:15:49.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.237+0000 7f2eede00640 1 --2- 172.21.15.45:0/1840218909 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2ef010e660 0x7f2ef01a3740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:49.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.237+0000 7f2eede00640 1 -- 172.21.15.45:0/1840218909 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ef01a3e70 con 0x7f2ef0108610 2024-04-16T17:15:49.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.237+0000 7f2eede00640 1 --2- 172.21.15.45:0/1840218909 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2ef0108610 0x7f2ef019ee80 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f2ee0009230 tx=0x7f2ee002cb20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:15:49.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.238+0000 7f2ee6a00640 1 -- 172.21.15.45:0/1840218909 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ee0041070 con 0x7f2ef0108610 2024-04-16T17:15:49.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.238+0000 7f2ef4e00640 1 -- 172.21.15.45:0/1840218909 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ef01a4100 con 0x7f2ef0108610 2024-04-16T17:15:49.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.238+0000 7f2ee6a00640 1 -- 172.21.15.45:0/1840218909 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2ee0004450 con 0x7f2ef0108610 2024-04-16T17:15:49.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.238+0000 7f2ee6a00640 1 -- 172.21.15.45:0/1840218909 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ee0003d90 con 0x7f2ef0108610 2024-04-16T17:15:49.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.238+0000 7f2ef4e00640 1 -- 172.21.15.45:0/1840218909 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ef01ab9f0 con 0x7f2ef0108610 2024-04-16T17:15:49.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.242+0000 7f2ef7b92740 1 Processor -- start 2024-04-16T17:15:49.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.242+0000 7f2ef7b92740 1 -- start start 2024-04-16T17:15:49.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.242+0000 7f2ef7b92740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5640a926eca0 0x5640a926f080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:49.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.243+0000 7f2ef7b92740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5640a926f5c0 0x5640a9277c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:49.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.243+0000 7f2ef7b92740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5640a92781a0 0x5640a927a590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:49.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.243+0000 7f2ef7b92740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5640a91d3cb0 con 0x5640a926f5c0 2024-04-16T17:15:49.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.243+0000 7f2ef7b92740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5640a917a370 con 0x5640a926eca0 2024-04-16T17:15:49.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.243+0000 7f2ef7b92740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5640a91cfc70 con 0x5640a92781a0 2024-04-16T17:15:49.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.243+0000 7f2ee6a00640 1 -- 172.21.15.45:0/1840218909 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f2ee0044070 con 0x7f2ef0108610 2024-04-16T17:15:49.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.243+0000 7f2eee800640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5640a926eca0 0x5640a926f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:49.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.243+0000 7f2eee800640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5640a926eca0 0x5640a926f080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:44716/0 (socket says 172.21.15.45:44716) 2024-04-16T17:15:49.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.243+0000 7f2eee800640 1 -- 172.21.15.45:0/891500327 learned_addr learned my addr 172.21.15.45:0/891500327 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:15:49.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.243+0000 7f2eede00640 1 --2- 172.21.15.45:0/891500327 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5640a926f5c0 0x5640a9277c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:49.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.243+0000 7f2eed400640 1 --2- 172.21.15.45:0/891500327 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5640a92781a0 0x5640a927a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:49.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.244+0000 7f2eee800640 1 -- 172.21.15.45:0/891500327 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5640a92781a0 msgr2=0x5640a927a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:15:49.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.244+0000 7f2eee800640 1 --2- 172.21.15.45:0/891500327 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5640a92781a0 0x5640a927a590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:49.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.244+0000 7f2eee800640 1 -- 172.21.15.45:0/891500327 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5640a926f5c0 msgr2=0x5640a9277c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:15:49.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.244+0000 7f2eee800640 1 --2- 172.21.15.45:0/891500327 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5640a926f5c0 0x5640a9277c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:49.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.244+0000 7f2eee800640 1 -- 172.21.15.45:0/891500327 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5640a9199910 con 0x5640a926eca0 2024-04-16T17:15:49.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.244+0000 7f2eee800640 1 --2- 172.21.15.45:0/891500327 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5640a926eca0 0x5640a926f080 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f2ee8009fd0 tx=0x7f2ee80118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a485dc940e9434bb server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:15:49.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.244+0000 7f2edb400640 1 -- 172.21.15.45:0/891500327 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ee8004d20 con 0x5640a926eca0 2024-04-16T17:15:49.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.244+0000 7f2edb400640 1 -- 172.21.15.45:0/891500327 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2ee8004050 con 0x5640a926eca0 2024-04-16T17:15:49.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.244+0000 7f2edb400640 1 -- 172.21.15.45:0/891500327 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ee8005280 con 0x5640a926eca0 2024-04-16T17:15:49.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.244+0000 7f2ee6a00640 1 --2- 172.21.15.45:0/1840218909 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f2ed40a3d20 0x7f2ed40a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:49.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.245+0000 7f2ee6a00640 1 -- 172.21.15.45:0/1840218909 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9496+0+0 (secure 0 0 0) 0x7f2ee00f24a0 con 0x7f2ef0108610 2024-04-16T17:15:49.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.245+0000 7f2eede00640 1 --2- 172.21.15.45:0/1840218909 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f2ed40a3d20 0x7f2ed40a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:49.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.245+0000 7f2ef7b92740 1 -- 172.21.15.45:0/891500327 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5640a926eca0 msgr2=0x5640a926f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:15:49.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.245+0000 7f2ef7b92740 1 --2- 172.21.15.45:0/891500327 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5640a926eca0 0x5640a926f080 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f2ee8009fd0 tx=0x7f2ee80118d0 comp rx=0 tx=0).stop 2024-04-16T17:15:49.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.245+0000 7f2ef7b92740 1 -- 172.21.15.45:0/891500327 shutdown_connections 2024-04-16T17:15:49.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.245+0000 7f2ef7b92740 1 --2- 172.21.15.45:0/891500327 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5640a92781a0 0x5640a927a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:49.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.245+0000 7f2ef7b92740 1 --2- 172.21.15.45:0/891500327 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5640a926f5c0 0x5640a9277c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:49.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.245+0000 7f2ef7b92740 1 --2- 172.21.15.45:0/891500327 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5640a926eca0 0x5640a926f080 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:49.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.245+0000 7f2ef7b92740 1 -- 172.21.15.45:0/891500327 >> 172.21.15.45:0/891500327 conn(0x5640a92693d0 msgr2=0x5640a92697b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:15:49.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.245+0000 7f2ef7b92740 1 -- 172.21.15.45:0/891500327 shutdown_connections 2024-04-16T17:15:49.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.245+0000 7f2eede00640 1 --2- 172.21.15.45:0/1840218909 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f2ed40a3d20 0x7f2ed40a61e0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f2ee00efa80 tx=0x7f2ee00f80e0 comp rx=0 tx=0).ready entity=mgr.25411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:15:49.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.245+0000 7f2ef7b92740 1 -- 172.21.15.45:0/891500327 wait complete. 2024-04-16T17:15:49.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.246+0000 7f2ef7b92740 1 -- start start 2024-04-16T17:15:49.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.246+0000 7f2ef7b92740 1 Processor -- start 2024-04-16T17:15:49.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.246+0000 7f2ef7b92740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5640a926eca0 0x5640a9275230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:49.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.246+0000 7f2ef7b92740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5640a926f5c0 0x5640a9275770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:49.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.246+0000 7f2ef7b92740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5640a92781a0 0x5640a9275cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:49.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.247+0000 7f2ef7b92740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5640a927af90 con 0x5640a926f5c0 2024-04-16T17:15:49.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.247+0000 7f2ef7b92740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5640a918d730 con 0x5640a92781a0 2024-04-16T17:15:49.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.247+0000 7f2ef7b92740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5640a927b110 con 0x5640a926eca0 2024-04-16T17:15:49.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.247+0000 7f2ef7b92740 10 client.0 _ll_register_callbacks cb 0x5640a915c1e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:15:49.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.247+0000 7f2eed400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5640a926f5c0 0x5640a9275770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:49.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.247+0000 7f2eee800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5640a926eca0 0x5640a9275230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:49.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.247+0000 7f2eee800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5640a926eca0 0x5640a9275230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:34168/0 (socket says 172.21.15.45:34168) 2024-04-16T17:15:49.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.247+0000 7f2eee800640 1 -- 172.21.15.45:0/518269344 learned_addr learned my addr 172.21.15.45:0/518269344 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:15:49.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.247+0000 7f2eee800640 1 --2- 172.21.15.45:0/518269344 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5640a92781a0 0x5640a9275cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:49.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.247+0000 7f2eed400640 1 -- 172.21.15.45:0/518269344 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5640a926eca0 msgr2=0x5640a9275230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:15:49.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.247+0000 7f2eed400640 1 --2- 172.21.15.45:0/518269344 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5640a926eca0 0x5640a9275230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:49.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.247+0000 7f2eed400640 1 -- 172.21.15.45:0/518269344 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5640a92781a0 msgr2=0x5640a9275cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:15:49.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.247+0000 7f2eed400640 1 --2- 172.21.15.45:0/518269344 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5640a92781a0 0x5640a9275cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:49.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.247+0000 7f2eed400640 1 -- 172.21.15.45:0/518269344 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5640a8fae7e0 con 0x5640a926f5c0 2024-04-16T17:15:49.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.247+0000 7f2eee800640 1 --2- 172.21.15.45:0/518269344 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5640a92781a0 0x5640a9275cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:15:49.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.248+0000 7f2eed400640 1 --2- 172.21.15.45:0/518269344 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5640a926f5c0 0x5640a9275770 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f2ef010dd40 tx=0x7f2edc012b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4382ed16832c57da server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:15:49.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.248+0000 7f2ecfe00640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-16T17:15:49.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.248+0000 7f2ecfe00640 1 -- 172.21.15.45:0/518269344 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2edc00f920 con 0x5640a926f5c0 2024-04-16T17:15:49.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.248+0000 7f2ef7b92740 1 -- 172.21.15.45:0/518269344 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2ee8004050 con 0x5640a926f5c0 2024-04-16T17:15:49.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.248+0000 7f2ecfe00640 1 -- 172.21.15.45:0/518269344 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9496+0+0 (secure 0 0 0) 0x7f2edc00fac0 con 0x5640a926f5c0 2024-04-16T17:15:49.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.248+0000 7f2ecfe00640 1 -- 172.21.15.45:0/518269344 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f2edc00fcc0 con 0x5640a926f5c0 2024-04-16T17:15:49.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.248+0000 7f2ef7b92740 10 client.17850 fetch_fsmap learned FSMap version 43 2024-04-16T17:15:49.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.248+0000 7f2ef7b92740 1 -- 172.21.15.45:0/518269344 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x5640a900ef70 con 0x5640a926f5c0 2024-04-16T17:15:49.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.248+0000 7f2ecfe00640 1 -- 172.21.15.45:0/518269344 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2ee8004050 con 0x5640a926f5c0 2024-04-16T17:15:49.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.249+0000 7f2ecfe00640 1 -- 172.21.15.45:0/518269344 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f2edc01c420 con 0x5640a926f5c0 2024-04-16T17:15:49.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.249+0000 7f2ef7b92740 10 client.17850 fetch_fsmap finished waiting for FSMap version 43 2024-04-16T17:15:49.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.249+0000 7f2ef7b92740 10 client.17850 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:15:49.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.249+0000 7f2ef7b92740 20 client.17850 populate_metadata read hostname 'smithi045' 2024-04-16T17:15:49.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.249+0000 7f2ef7b92740 1 --2- 172.21.15.45:0/518269344 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x5640a92886d0 0x5640a928aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:49.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.249+0000 7f2ef7b92740 4 client.17850 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:15:49.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.249+0000 7f2ef7b92740 1 -- 172.21.15.45:0/518269344 --> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5640a91e16d0 con 0x5640a92886d0 2024-04-16T17:15:49.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.249+0000 7f2eee800640 1 --2- 172.21.15.45:0/518269344 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x5640a92886d0 0x5640a928aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:49.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.250+0000 7f2eee800640 1 --2- 172.21.15.45:0/518269344 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x5640a92886d0 0x5640a928aaf0 crc :-1 s=READY pgs=237 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f603de7597f83cf3 server_cookie=72cbe34f8443217a in_seq=0 out_seq=0 2024-04-16T17:15:49.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.250+0000 7f2ecfe00640 10 client.17850 ms_handle_connect on v2:172.21.15.104:6838/3660070592 2024-04-16T17:15:49.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.251+0000 7f2ecfe00640 1 -- 172.21.15.45:0/518269344 <== mds.0 v2:172.21.15.104:6838/3660070592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5640a91e16d0 con 0x5640a92886d0 2024-04-16T17:15:49.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.251+0000 7f2ecfe00640 10 client.17850 handle_command_reply: tid=0 2024-04-16T17:15:49.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.251+0000 7f2ef7b92740 1 client.17850 shutdown 2024-04-16T17:15:49.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.251+0000 7f2ef7b92740 1 -- 172.21.15.45:0/518269344 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5640a926f5c0 msgr2=0x5640a9275770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:15:49.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.251+0000 7f2ef7b92740 1 --2- 172.21.15.45:0/518269344 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5640a926f5c0 0x5640a9275770 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f2ef010dd40 tx=0x7f2edc012b60 comp rx=0 tx=0).stop 2024-04-16T17:15:49.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.251+0000 7f2ef7b92740 1 -- 172.21.15.45:0/518269344 shutdown_connections 2024-04-16T17:15:49.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.251+0000 7f2ef7b92740 1 --2- 172.21.15.45:0/518269344 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5640a92781a0 0x5640a9275cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:49.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.251+0000 7f2ef7b92740 1 --2- 172.21.15.45:0/518269344 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5640a926f5c0 0x5640a9275770 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:49.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.251+0000 7f2ef7b92740 1 --2- 172.21.15.45:0/518269344 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x5640a92886d0 0x5640a928aaf0 crc :-1 s=READY pgs=237 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:49.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.251+0000 7f2ef7b92740 1 --2- 172.21.15.45:0/518269344 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5640a926eca0 0x5640a9275230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:49.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.251+0000 7f2ef7b92740 1 -- 172.21.15.45:0/518269344 >> 172.21.15.45:0/518269344 conn(0x5640a92693d0 msgr2=0x5640a926b120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:15:49.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.251+0000 7f2ecfe00640 0 client.17850 ms_handle_reset on v2:172.21.15.104:6838/3660070592 2024-04-16T17:15:49.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.251+0000 7f2ef7b92740 1 -- 172.21.15.45:0/518269344 shutdown_connections 2024-04-16T17:15:49.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.251+0000 7f2ef7b92740 1 -- 172.21.15.45:0/518269344 wait complete. 2024-04-16T17:15:49.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.251+0000 7f2ef7b92740 20 client.17850 trim_cache size 0 max 16384 2024-04-16T17:15:49.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.271+0000 7f2ef7b92740 1 Processor -- start 2024-04-16T17:15:49.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.271+0000 7f2ef7b92740 1 -- start start 2024-04-16T17:15:49.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.271+0000 7f2ef7b92740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5640a926eca0 0x5640a9277960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:49.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.271+0000 7f2ef7b92740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5640a926f5c0 0x5640a9283e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:49.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.271+0000 7f2ef7b92740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5640a92781a0 0x5640a9290400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:49.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.271+0000 7f2ef7b92740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f2ef01b83c0 con 0x5640a926eca0 2024-04-16T17:15:49.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.271+0000 7f2ef7b92740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f2ef01b5c90 con 0x5640a92781a0 2024-04-16T17:15:49.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.271+0000 7f2ef7b92740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f2ef01b3b00 con 0x5640a926f5c0 2024-04-16T17:15:49.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.271+0000 7f2eee800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5640a926eca0 0x5640a9277960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:49.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.271+0000 7f2eee800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5640a926eca0 0x5640a9277960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:44346/0 (socket says 172.21.15.45:44346) 2024-04-16T17:15:49.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.271+0000 7f2eed400640 1 --2- 172.21.15.45:0/3377482240 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5640a926f5c0 0x5640a9283e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:49.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.271+0000 7f2eee800640 1 -- 172.21.15.45:0/3377482240 learned_addr learned my addr 172.21.15.45:0/3377482240 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:15:49.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.272+0000 7f2eee800640 1 --2- 172.21.15.45:0/3377482240 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5640a92781a0 0x5640a9290400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:49.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.272+0000 7f2eed400640 1 -- 172.21.15.45:0/3377482240 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5640a92781a0 msgr2=0x5640a9290400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:15:49.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.272+0000 7f2eed400640 1 --2- 172.21.15.45:0/3377482240 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5640a92781a0 0x5640a9290400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:49.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.272+0000 7f2eed400640 1 -- 172.21.15.45:0/3377482240 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5640a926eca0 msgr2=0x5640a9277960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:15:49.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.272+0000 7f2eed400640 1 --2- 172.21.15.45:0/3377482240 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5640a926eca0 0x5640a9277960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:49.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.272+0000 7f2eed400640 1 -- 172.21.15.45:0/3377482240 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5640a8ea6170 con 0x5640a926f5c0 2024-04-16T17:15:49.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.272+0000 7f2eed400640 1 --2- 172.21.15.45:0/3377482240 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5640a926f5c0 0x5640a9283e50 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f2edc01ca80 tx=0x7f2edc009850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=55fc5f9a7fe110f5 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:15:49.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.273+0000 7f2edaa00640 1 -- 172.21.15.45:0/3377482240 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2edc02f070 con 0x5640a926f5c0 2024-04-16T17:15:49.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.273+0000 7f2edaa00640 1 -- 172.21.15.45:0/3377482240 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2edc005520 con 0x5640a926f5c0 2024-04-16T17:15:49.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.273+0000 7f2edaa00640 1 -- 172.21.15.45:0/3377482240 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2edc01c680 con 0x5640a926f5c0 2024-04-16T17:15:49.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.273+0000 7f2ef7b92740 1 -- 172.21.15.45:0/3377482240 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5640a926f5c0 msgr2=0x5640a9283e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:15:49.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.273+0000 7f2ef7b92740 1 --2- 172.21.15.45:0/3377482240 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5640a926f5c0 0x5640a9283e50 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f2edc01ca80 tx=0x7f2edc009850 comp rx=0 tx=0).stop 2024-04-16T17:15:49.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.273+0000 7f2ef7b92740 1 -- 172.21.15.45:0/3377482240 shutdown_connections 2024-04-16T17:15:49.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.273+0000 7f2ef7b92740 1 --2- 172.21.15.45:0/3377482240 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5640a926f5c0 0x5640a9283e50 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:49.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.273+0000 7f2ef7b92740 1 --2- 172.21.15.45:0/3377482240 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5640a92781a0 0x5640a9290400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:49.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.273+0000 7f2ef7b92740 1 --2- 172.21.15.45:0/3377482240 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5640a926eca0 0x5640a9277960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:49.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.273+0000 7f2ef7b92740 1 -- 172.21.15.45:0/3377482240 >> 172.21.15.45:0/3377482240 conn(0x5640a92693d0 msgr2=0x5640a9275a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:15:49.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.273+0000 7f2ef7b92740 1 -- 172.21.15.45:0/3377482240 shutdown_connections 2024-04-16T17:15:49.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.273+0000 7f2ef7b92740 1 -- 172.21.15.45:0/3377482240 wait complete. 2024-04-16T17:15:49.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.274+0000 7f2ef7b92740 1 -- start start 2024-04-16T17:15:49.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.274+0000 7f2ef7b92740 1 Processor -- start 2024-04-16T17:15:49.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.274+0000 7f2ef7b92740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5640a926eca0 0x5640a92806a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:49.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.274+0000 7f2ef7b92740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5640a926f5c0 0x5640a927d330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:49.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.274+0000 7f2ef7b92740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5640a92781a0 0x5640a927d870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:49.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.274+0000 7f2ef7b92740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f2ef01b4150 con 0x5640a926eca0 2024-04-16T17:15:49.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.274+0000 7f2ef7b92740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f2ef01b55c0 con 0x5640a92781a0 2024-04-16T17:15:49.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.274+0000 7f2ef7b92740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f2ef01b4820 con 0x5640a926f5c0 2024-04-16T17:15:49.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.274+0000 7f2ef7b92740 10 client.0 _ll_register_callbacks cb 0x7f2edc009ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:15:49.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.275+0000 7f2eee800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5640a926eca0 0x5640a92806a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:49.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.275+0000 7f2eee800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5640a926eca0 0x5640a92806a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:44356/0 (socket says 172.21.15.45:44356) 2024-04-16T17:15:49.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.275+0000 7f2eee800640 1 -- 172.21.15.45:0/2401659748 learned_addr learned my addr 172.21.15.45:0/2401659748 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:15:49.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.275+0000 7f2eed400640 1 --2- 172.21.15.45:0/2401659748 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5640a926f5c0 0x5640a927d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:49.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.275+0000 7f2eee800640 1 --2- 172.21.15.45:0/2401659748 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5640a92781a0 0x5640a927d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:49.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.275+0000 7f2eee800640 1 -- 172.21.15.45:0/2401659748 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5640a926f5c0 msgr2=0x5640a927d330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:15:49.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.275+0000 7f2eee800640 1 --2- 172.21.15.45:0/2401659748 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5640a926f5c0 0x5640a927d330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:49.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.275+0000 7f2eee800640 1 -- 172.21.15.45:0/2401659748 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5640a92781a0 msgr2=0x5640a927d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:15:49.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.275+0000 7f2eee800640 1 --2- 172.21.15.45:0/2401659748 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5640a92781a0 0x5640a927d870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:49.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.275+0000 7f2eee800640 1 -- 172.21.15.45:0/2401659748 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5640a927e020 con 0x5640a926eca0 2024-04-16T17:15:49.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.275+0000 7f2eed400640 1 --2- 172.21.15.45:0/2401659748 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5640a926f5c0 0x5640a927d330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:15:49.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.275+0000 7f2eee800640 1 --2- 172.21.15.45:0/2401659748 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5640a926eca0 0x5640a92806a0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f2ee8011dd0 tx=0x7f2ee8009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cdf67f5aac854f9 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:15:49.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.275+0000 7f2ecfe00640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-16T17:15:49.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.276+0000 7f2ecfe00640 1 -- 172.21.15.45:0/2401659748 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ee8003de0 con 0x5640a926eca0 2024-04-16T17:15:49.282 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.276+0000 7f2ef7b92740 1 -- 172.21.15.45:0/2401659748 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2edc005520 con 0x5640a926eca0 2024-04-16T17:15:49.282 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.276+0000 7f2ecfe00640 1 -- 172.21.15.45:0/2401659748 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9496+0+0 (secure 0 0 0) 0x7f2ee802fb50 con 0x5640a926eca0 2024-04-16T17:15:49.282 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.276+0000 7f2ecfe00640 1 -- 172.21.15.45:0/2401659748 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f2ee802fd50 con 0x5640a926eca0 2024-04-16T17:15:49.282 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.276+0000 7f2ef7b92740 10 client.17856 fetch_fsmap learned FSMap version 43 2024-04-16T17:15:49.282 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.276+0000 7f2ef7b92740 1 -- 172.21.15.45:0/2401659748 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x5640a92efc90 con 0x5640a926eca0 2024-04-16T17:15:49.282 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.276+0000 7f2ecfe00640 1 -- 172.21.15.45:0/2401659748 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2edc005520 con 0x5640a926eca0 2024-04-16T17:15:49.282 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.276+0000 7f2ecfe00640 1 -- 172.21.15.45:0/2401659748 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f2ee8025040 con 0x5640a926eca0 2024-04-16T17:15:49.282 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.276+0000 7f2ef7b92740 10 client.17856 fetch_fsmap finished waiting for FSMap version 43 2024-04-16T17:15:49.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.276+0000 7f2ef7b92740 10 client.17856 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:15:49.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.276+0000 7f2ef7b92740 20 client.17856 populate_metadata read hostname 'smithi045' 2024-04-16T17:15:49.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.276+0000 7f2ef7b92740 1 --2- 172.21.15.45:0/2401659748 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x5640a92f01e0 0x5640a92f2600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:49.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.277+0000 7f2ef7b92740 4 client.17856 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-16T17:15:49.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.277+0000 7f2ef7b92740 1 -- 172.21.15.45:0/2401659748 --> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5640a91db540 con 0x5640a92f01e0 2024-04-16T17:15:49.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.277+0000 7f2eed400640 1 --2- 172.21.15.45:0/2401659748 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x5640a92f01e0 0x5640a92f2600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:49.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.277+0000 7f2eed400640 1 --2- 172.21.15.45:0/2401659748 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x5640a92f01e0 0x5640a92f2600 crc :-1 s=READY pgs=238 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8514c6f699d3face server_cookie=92e3f5049c488304 in_seq=0 out_seq=0 2024-04-16T17:15:49.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.277+0000 7f2ecfe00640 10 client.17856 ms_handle_connect on v2:172.21.15.104:6838/3660070592 2024-04-16T17:15:49.284 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.279+0000 7f2ecfe00640 1 -- 172.21.15.45:0/2401659748 <== mds.0 v2:172.21.15.104:6838/3660070592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5640a91db540 con 0x5640a92f01e0 2024-04-16T17:15:49.284 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.279+0000 7f2ecfe00640 10 client.17856 handle_command_reply: tid=0 2024-04-16T17:15:49.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.279+0000 7f2ef7b92740 1 client.17856 shutdown 2024-04-16T17:15:49.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.279+0000 7f2ef7b92740 1 -- 172.21.15.45:0/2401659748 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5640a926eca0 msgr2=0x5640a92806a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:15:49.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.279+0000 7f2ef7b92740 1 --2- 172.21.15.45:0/2401659748 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5640a926eca0 0x5640a92806a0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f2ee8011dd0 tx=0x7f2ee8009500 comp rx=0 tx=0).stop 2024-04-16T17:15:49.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.279+0000 7f2ef7b92740 1 -- 172.21.15.45:0/2401659748 shutdown_connections 2024-04-16T17:15:49.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.279+0000 7f2ef7b92740 1 --2- 172.21.15.45:0/2401659748 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x5640a92f01e0 0x5640a92f2600 crc :-1 s=READY pgs=238 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:49.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.279+0000 7f2ef7b92740 1 --2- 172.21.15.45:0/2401659748 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5640a926f5c0 0x5640a927d330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:49.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.279+0000 7f2ef7b92740 1 --2- 172.21.15.45:0/2401659748 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5640a92781a0 0x5640a927d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:49.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.279+0000 7f2ef7b92740 1 --2- 172.21.15.45:0/2401659748 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5640a926eca0 0x5640a92806a0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:49.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.279+0000 7f2ef7b92740 1 -- 172.21.15.45:0/2401659748 >> 172.21.15.45:0/2401659748 conn(0x5640a92693d0 msgr2=0x5640a9267ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:15:49.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.279+0000 7f2ecfe00640 0 client.17856 ms_handle_reset on v2:172.21.15.104:6838/3660070592 2024-04-16T17:15:49.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.280+0000 7f2ef7b92740 1 -- 172.21.15.45:0/2401659748 shutdown_connections 2024-04-16T17:15:49.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.280+0000 7f2ef7b92740 1 -- 172.21.15.45:0/2401659748 wait complete. 2024-04-16T17:15:49.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.280+0000 7f2ef7b92740 20 client.17856 trim_cache size 0 max 16384 2024-04-16T17:15:49.286 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-16T17:15:49.286 INFO:teuthology.orchestra.run.smithi045.stdout: "return_code": 0, 2024-04-16T17:15:49.287 INFO:teuthology.orchestra.run.smithi045.stdout: "scrub_tag": "fae45a96-bda2-4f23-8a83-72b7bd836a76", 2024-04-16T17:15:49.287 INFO:teuthology.orchestra.run.smithi045.stdout: "mode": "asynchronous" 2024-04-16T17:15:49.287 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-16T17:15:49.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.280+0000 7f2ef4e00640 1 -- 172.21.15.45:0/1840218909 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f2ed40a3d20 msgr2=0x7f2ed40a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:15:49.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.280+0000 7f2ef4e00640 1 --2- 172.21.15.45:0/1840218909 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f2ed40a3d20 0x7f2ed40a61e0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f2ee00efa80 tx=0x7f2ee00f80e0 comp rx=0 tx=0).stop 2024-04-16T17:15:49.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.280+0000 7f2ef4e00640 1 -- 172.21.15.45:0/1840218909 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2ef0108610 msgr2=0x7f2ef019ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:15:49.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.280+0000 7f2ef4e00640 1 --2- 172.21.15.45:0/1840218909 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2ef0108610 0x7f2ef019ee80 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f2ee0009230 tx=0x7f2ee002cb20 comp rx=0 tx=0).stop 2024-04-16T17:15:49.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.280+0000 7f2ef4e00640 1 -- 172.21.15.45:0/1840218909 shutdown_connections 2024-04-16T17:15:49.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.280+0000 7f2ef4e00640 1 --2- 172.21.15.45:0/1840218909 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f2ed40a3d20 0x7f2ed40a61e0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:49.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.280+0000 7f2ef4e00640 1 --2- 172.21.15.45:0/1840218909 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2ef010e660 0x7f2ef01a3740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:49.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.281+0000 7f2ef4e00640 1 --2- 172.21.15.45:0/1840218909 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2ef0108fd0 0x7f2ef019f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:49.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.281+0000 7f2ef4e00640 1 --2- 172.21.15.45:0/1840218909 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2ef0108610 0x7f2ef019ee80 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:49.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.281+0000 7f2ef4e00640 1 -- 172.21.15.45:0/1840218909 >> 172.21.15.45:0/1840218909 conn(0x7f2ef0076540 msgr2=0x7f2ef0077c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:15:49.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.281+0000 7f2ef4e00640 1 -- 172.21.15.45:0/1840218909 shutdown_connections 2024-04-16T17:15:49.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.281+0000 7f2ef4e00640 1 -- 172.21.15.45:0/1840218909 wait complete. 2024-04-16T17:15:49.300 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-16T17:15:49.300 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-16T17:15:49.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.498+0000 7fd48f200640 1 -- 172.21.15.45:0/904475291 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd490102110 msgr2=0x7fd4901024f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:15:49.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.498+0000 7fd48f200640 1 --2- 172.21.15.45:0/904475291 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd490102110 0x7fd4901024f0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fd480009ca0 tx=0x7fd48002c100 comp rx=0 tx=0).stop 2024-04-16T17:15:49.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.499+0000 7fd48f200640 1 -- 172.21.15.45:0/904475291 shutdown_connections 2024-04-16T17:15:49.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.499+0000 7fd48f200640 1 --2- 172.21.15.45:0/904475291 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd4901071e0 0x7fd49010edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:49.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.499+0000 7fd48f200640 1 --2- 172.21.15.45:0/904475291 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd490102ac0 0x7fd490106b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:49.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.499+0000 7fd48f200640 1 --2- 172.21.15.45:0/904475291 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd490102110 0x7fd4901024f0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:49.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.499+0000 7fd48f200640 1 -- 172.21.15.45:0/904475291 >> 172.21.15.45:0/904475291 conn(0x7fd4900fd9d0 msgr2=0x7fd4900ffdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:15:49.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.499+0000 7fd48f200640 1 -- 172.21.15.45:0/904475291 shutdown_connections 2024-04-16T17:15:49.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.499+0000 7fd48f200640 1 -- 172.21.15.45:0/904475291 wait complete. 2024-04-16T17:15:49.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.500+0000 7fd48f200640 1 Processor -- start 2024-04-16T17:15:49.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.500+0000 7fd48f200640 1 -- start start 2024-04-16T17:15:49.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.500+0000 7fd48f200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd490102110 0x7fd4901a0630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:49.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.500+0000 7fd48f200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd4901071e0 0x7fd4901a0b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:49.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.500+0000 7fd48f200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd49019a7a0 0x7fd49019ac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:49.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.500+0000 7fd48f200640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fd490112290 con 0x7fd490102110 2024-04-16T17:15:49.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.500+0000 7fd48f200640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fd490112110 con 0x7fd4901071e0 2024-04-16T17:15:49.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.500+0000 7fd48f200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fd490112410 con 0x7fd49019a7a0 2024-04-16T17:15:49.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.500+0000 7fd48de00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd490102110 0x7fd4901a0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:49.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.500+0000 7fd48de00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd490102110 0x7fd4901a0630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:44368/0 (socket says 172.21.15.45:44368) 2024-04-16T17:15:49.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.500+0000 7fd48de00640 1 -- 172.21.15.45:0/2399701779 learned_addr learned my addr 172.21.15.45:0/2399701779 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:15:49.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.501+0000 7fd485400640 1 --2- 172.21.15.45:0/2399701779 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd4901071e0 0x7fd4901a0b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:49.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.501+0000 7fd48e800640 1 --2- 172.21.15.45:0/2399701779 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd49019a7a0 0x7fd49019ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:49.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.501+0000 7fd48de00640 1 -- 172.21.15.45:0/2399701779 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd49019a7a0 msgr2=0x7fd49019ac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:15:49.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.501+0000 7fd48de00640 1 --2- 172.21.15.45:0/2399701779 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd49019a7a0 0x7fd49019ac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:49.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.501+0000 7fd48de00640 1 -- 172.21.15.45:0/2399701779 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd4901071e0 msgr2=0x7fd4901a0b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:15:49.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.501+0000 7fd48de00640 1 --2- 172.21.15.45:0/2399701779 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd4901071e0 0x7fd4901a0b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:49.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.501+0000 7fd48de00640 1 -- 172.21.15.45:0/2399701779 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd49019b330 con 0x7fd490102110 2024-04-16T17:15:49.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.501+0000 7fd48de00640 1 --2- 172.21.15.45:0/2399701779 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd490102110 0x7fd4901a0630 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fd480005e30 tx=0x7fd480035d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:15:49.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.501+0000 7fd487400640 1 -- 172.21.15.45:0/2399701779 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd480039040 con 0x7fd490102110 2024-04-16T17:15:49.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.502+0000 7fd487400640 1 -- 172.21.15.45:0/2399701779 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd480044070 con 0x7fd490102110 2024-04-16T17:15:49.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.502+0000 7fd48f200640 1 -- 172.21.15.45:0/2399701779 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd49019b5c0 con 0x7fd490102110 2024-04-16T17:15:49.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.502+0000 7fd487400640 1 -- 172.21.15.45:0/2399701779 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4800043c0 con 0x7fd490102110 2024-04-16T17:15:49.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.502+0000 7fd48f200640 1 -- 172.21.15.45:0/2399701779 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4901a7770 con 0x7fd490102110 2024-04-16T17:15:49.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.505+0000 7fd4977f1740 1 Processor -- start 2024-04-16T17:15:49.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.505+0000 7fd4977f1740 1 -- start start 2024-04-16T17:15:49.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.505+0000 7fd487400640 1 -- 172.21.15.45:0/2399701779 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fd480004560 con 0x7fd490102110 2024-04-16T17:15:49.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.505+0000 7fd4977f1740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55891f32a390 0x55891f32a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:49.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.505+0000 7fd4977f1740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55891f32acb0 0x55891f333350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:49.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.505+0000 7fd4977f1740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55891f333890 0x55891f335c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:49.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.505+0000 7fd4977f1740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55891f290120 con 0x55891f333890 2024-04-16T17:15:49.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.505+0000 7fd4977f1740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55891f236340 con 0x55891f32a390 2024-04-16T17:15:49.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.505+0000 7fd4977f1740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55891f28c0e0 con 0x55891f32acb0 2024-04-16T17:15:49.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.506+0000 7fd48de00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55891f32acb0 0x55891f333350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:49.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.506+0000 7fd48e800640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55891f32a390 0x55891f32a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:49.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.506+0000 7fd48de00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55891f32acb0 0x55891f333350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:34222/0 (socket says 172.21.15.45:34222) 2024-04-16T17:15:49.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.506+0000 7fd48de00640 1 -- 172.21.15.45:0/916340377 learned_addr learned my addr 172.21.15.45:0/916340377 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:15:49.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.506+0000 7fd485400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55891f333890 0x55891f335c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:49.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.506+0000 7fd48de00640 1 -- 172.21.15.45:0/916340377 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55891f32a390 msgr2=0x55891f32a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:15:49.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.506+0000 7fd48de00640 1 --2- 172.21.15.45:0/916340377 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55891f32a390 0x55891f32a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:49.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.506+0000 7fd48de00640 1 -- 172.21.15.45:0/916340377 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55891f333890 msgr2=0x55891f335c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:15:49.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.506+0000 7fd48de00640 1 --2- 172.21.15.45:0/916340377 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55891f333890 0x55891f335c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:49.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.506+0000 7fd48de00640 1 -- 172.21.15.45:0/916340377 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55891f06c7d0 con 0x55891f32acb0 2024-04-16T17:15:49.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.506+0000 7fd48e800640 1 --2- 172.21.15.45:0/916340377 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55891f32a390 0x55891f32a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:15:49.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.506+0000 7fd48de00640 1 --2- 172.21.15.45:0/916340377 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55891f32acb0 0x55891f333350 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fd4800f03a0 tx=0x7fd4800f6f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1d4fa5be3ad6c449 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:15:49.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.507+0000 7fd47f000640 1 -- 172.21.15.45:0/916340377 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4800fb760 con 0x55891f32acb0 2024-04-16T17:15:49.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.507+0000 7fd47f000640 1 -- 172.21.15.45:0/916340377 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd480100030 con 0x55891f32acb0 2024-04-16T17:15:49.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.507+0000 7fd487400640 1 --2- 172.21.15.45:0/2399701779 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fd4740a3d20 0x7fd4740a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:49.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.507+0000 7fd47f000640 1 -- 172.21.15.45:0/916340377 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4800f3c60 con 0x55891f32acb0 2024-04-16T17:15:49.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.507+0000 7fd487400640 1 -- 172.21.15.45:0/2399701779 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9496+0+0 (secure 0 0 0) 0x7fd4800f2fa0 con 0x7fd490102110 2024-04-16T17:15:49.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.507+0000 7fd48e800640 1 --2- 172.21.15.45:0/2399701779 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fd4740a3d20 0x7fd4740a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:49.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.507+0000 7fd4977f1740 1 -- 172.21.15.45:0/916340377 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55891f32acb0 msgr2=0x55891f333350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:15:49.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.507+0000 7fd4977f1740 1 --2- 172.21.15.45:0/916340377 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55891f32acb0 0x55891f333350 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fd4800f03a0 tx=0x7fd4800f6f90 comp rx=0 tx=0).stop 2024-04-16T17:15:49.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.508+0000 7fd48e800640 1 --2- 172.21.15.45:0/2399701779 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fd4740a3d20 0x7fd4740a61e0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fd488009ef0 tx=0x7fd488005f90 comp rx=0 tx=0).ready entity=mgr.25411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:15:49.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.508+0000 7fd4977f1740 1 -- 172.21.15.45:0/916340377 shutdown_connections 2024-04-16T17:15:49.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.508+0000 7fd4977f1740 1 --2- 172.21.15.45:0/916340377 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55891f32acb0 0x55891f333350 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:49.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.508+0000 7fd4977f1740 1 --2- 172.21.15.45:0/916340377 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55891f333890 0x55891f335c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:49.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.508+0000 7fd4977f1740 1 --2- 172.21.15.45:0/916340377 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55891f32a390 0x55891f32a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:49.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.508+0000 7fd4977f1740 1 -- 172.21.15.45:0/916340377 >> 172.21.15.45:0/916340377 conn(0x55891f324ac0 msgr2=0x55891f324ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:15:49.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.508+0000 7fd4977f1740 1 -- 172.21.15.45:0/916340377 shutdown_connections 2024-04-16T17:15:49.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.508+0000 7fd4977f1740 1 -- 172.21.15.45:0/916340377 wait complete. 2024-04-16T17:15:49.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.508+0000 7fd4977f1740 1 -- start start 2024-04-16T17:15:49.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.509+0000 7fd4977f1740 1 Processor -- start 2024-04-16T17:15:49.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.509+0000 7fd4977f1740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55891f32a390 0x55891f330910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:49.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.509+0000 7fd4977f1740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55891f32acb0 0x55891f330e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:49.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.509+0000 7fd4977f1740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55891f333890 0x55891f331390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:49.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.509+0000 7fd4977f1740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55891f336440 con 0x55891f333890 2024-04-16T17:15:49.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.509+0000 7fd4977f1740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55891f249710 con 0x55891f32acb0 2024-04-16T17:15:49.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.509+0000 7fd4977f1740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55891f3365c0 con 0x55891f32a390 2024-04-16T17:15:49.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.509+0000 7fd4977f1740 10 client.0 _ll_register_callbacks cb 0x55891f244af0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:15:49.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.509+0000 7fd48de00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55891f32a390 0x55891f330910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:49.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.509+0000 7fd48de00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55891f32a390 0x55891f330910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:34224/0 (socket says 172.21.15.45:34224) 2024-04-16T17:15:49.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.509+0000 7fd48de00640 1 -- 172.21.15.45:0/531180078 learned_addr learned my addr 172.21.15.45:0/531180078 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:15:49.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.509+0000 7fd485400640 1 --2- 172.21.15.45:0/531180078 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55891f32acb0 0x55891f330e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:49.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.509+0000 7fd48e800640 1 --2- 172.21.15.45:0/531180078 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55891f333890 0x55891f331390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:49.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.510+0000 7fd48de00640 1 -- 172.21.15.45:0/531180078 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55891f32acb0 msgr2=0x55891f330e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:15:49.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.510+0000 7fd48de00640 1 --2- 172.21.15.45:0/531180078 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55891f32acb0 0x55891f330e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:49.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.510+0000 7fd48de00640 1 -- 172.21.15.45:0/531180078 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55891f333890 msgr2=0x55891f331390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:15:49.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.510+0000 7fd48de00640 1 --2- 172.21.15.45:0/531180078 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55891f333890 0x55891f331390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:49.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.510+0000 7fd48de00640 1 -- 172.21.15.45:0/531180078 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55891efa23f0 con 0x55891f32a390 2024-04-16T17:15:49.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.510+0000 7fd48e800640 1 --2- 172.21.15.45:0/531180078 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55891f333890 0x55891f331390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:15:49.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.510+0000 7fd48de00640 1 --2- 172.21.15.45:0/531180078 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55891f32a390 0x55891f330910 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fd4800bec10 tx=0x7fd4800f3500 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4643214ca9731448 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:15:49.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.510+0000 7fd473400640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-04-16T17:15:49.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.510+0000 7fd473400640 1 -- 172.21.15.45:0/531180078 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4800baf20 con 0x55891f32a390 2024-04-16T17:15:49.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.510+0000 7fd473400640 1 -- 172.21.15.45:0/531180078 <== mon.1 v2:172.21.15.117:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9496+0+0 (secure 0 0 0) 0x7fd4800bcd20 con 0x55891f32a390 2024-04-16T17:15:49.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.510+0000 7fd4977f1740 1 -- 172.21.15.45:0/531180078 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd480100030 con 0x55891f32a390 2024-04-16T17:15:49.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.511+0000 7fd473400640 1 -- 172.21.15.45:0/531180078 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fd4800be5e0 con 0x55891f32a390 2024-04-16T17:15:49.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.511+0000 7fd4977f1740 10 client.27970 fetch_fsmap learned FSMap version 43 2024-04-16T17:15:49.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.511+0000 7fd4977f1740 1 -- 172.21.15.45:0/531180078 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55891ef620f0 con 0x55891f32a390 2024-04-16T17:15:49.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.511+0000 7fd473400640 1 -- 172.21.15.45:0/531180078 <== mon.1 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd480100030 con 0x55891f32a390 2024-04-16T17:15:49.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.511+0000 7fd473400640 1 -- 172.21.15.45:0/531180078 <== mon.1 v2:172.21.15.117:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7fd48003c030 con 0x55891f32a390 2024-04-16T17:15:49.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.511+0000 7fd4977f1740 10 client.27970 fetch_fsmap finished waiting for FSMap version 43 2024-04-16T17:15:49.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.511+0000 7fd4977f1740 10 client.27970 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:15:49.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.511+0000 7fd4977f1740 20 client.27970 populate_metadata read hostname 'smithi045' 2024-04-16T17:15:49.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.511+0000 7fd4977f1740 1 --2- 172.21.15.45:0/531180078 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55891f343820 0x55891f345c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:49.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.511+0000 7fd4977f1740 4 client.27970 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:15:49.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.511+0000 7fd4977f1740 1 -- 172.21.15.45:0/531180078 --> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55891f29db40 con 0x55891f343820 2024-04-16T17:15:49.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.512+0000 7fd485400640 1 --2- 172.21.15.45:0/531180078 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55891f343820 0x55891f345c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:49.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.512+0000 7fd485400640 1 --2- 172.21.15.45:0/531180078 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55891f343820 0x55891f345c10 crc :-1 s=READY pgs=239 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cad0533e3ad42a72 server_cookie=4391d3d0278bf608 in_seq=0 out_seq=0 2024-04-16T17:15:49.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.512+0000 7fd473400640 10 client.27970 ms_handle_connect on v2:172.21.15.104:6838/3660070592 2024-04-16T17:15:49.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.513+0000 7fd473400640 1 -- 172.21.15.45:0/531180078 <== mds.0 v2:172.21.15.104:6838/3660070592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55891f29db40 con 0x55891f343820 2024-04-16T17:15:49.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.513+0000 7fd473400640 10 client.27970 handle_command_reply: tid=0 2024-04-16T17:15:49.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.513+0000 7fd4977f1740 1 client.27970 shutdown 2024-04-16T17:15:49.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.513+0000 7fd4977f1740 1 -- 172.21.15.45:0/531180078 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55891f32a390 msgr2=0x55891f330910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:15:49.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.513+0000 7fd4977f1740 1 --2- 172.21.15.45:0/531180078 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55891f32a390 0x55891f330910 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fd4800bec10 tx=0x7fd4800f3500 comp rx=0 tx=0).stop 2024-04-16T17:15:49.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.513+0000 7fd4977f1740 1 -- 172.21.15.45:0/531180078 shutdown_connections 2024-04-16T17:15:49.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.513+0000 7fd4977f1740 1 --2- 172.21.15.45:0/531180078 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55891f333890 0x55891f331390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:49.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.513+0000 7fd4977f1740 1 --2- 172.21.15.45:0/531180078 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55891f32acb0 0x55891f330e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:49.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.513+0000 7fd4977f1740 1 --2- 172.21.15.45:0/531180078 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55891f343820 0x55891f345c10 crc :-1 s=READY pgs=239 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:49.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.513+0000 7fd4977f1740 1 --2- 172.21.15.45:0/531180078 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55891f32a390 0x55891f330910 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:49.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.513+0000 7fd4977f1740 1 -- 172.21.15.45:0/531180078 >> 172.21.15.45:0/531180078 conn(0x55891f324ac0 msgr2=0x55891f326810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:15:49.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.513+0000 7fd473400640 0 client.27970 ms_handle_reset on v2:172.21.15.104:6838/3660070592 2024-04-16T17:15:49.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.513+0000 7fd4977f1740 1 -- 172.21.15.45:0/531180078 shutdown_connections 2024-04-16T17:15:49.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.513+0000 7fd4977f1740 1 -- 172.21.15.45:0/531180078 wait complete. 2024-04-16T17:15:49.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.514+0000 7fd4977f1740 20 client.27970 trim_cache size 0 max 16384 2024-04-16T17:15:49.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.532+0000 7fd4977f1740 1 Processor -- start 2024-04-16T17:15:49.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.532+0000 7fd4977f1740 1 -- start start 2024-04-16T17:15:49.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.533+0000 7fd4977f1740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55891f32a390 0x55891f323b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:49.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.533+0000 7fd4977f1740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55891f32acb0 0x55891f33f130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:49.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.533+0000 7fd4977f1740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55891f333890 0x55891f32fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:49.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.533+0000 7fd4977f1740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fd4901b3e80 con 0x55891f32acb0 2024-04-16T17:15:49.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.533+0000 7fd4977f1740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fd4901b1750 con 0x55891f333890 2024-04-16T17:15:49.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.533+0000 7fd4977f1740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fd4901af5c0 con 0x55891f32a390 2024-04-16T17:15:49.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.533+0000 7fd485400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55891f32acb0 0x55891f33f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:49.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.533+0000 7fd48de00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55891f32a390 0x55891f323b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:49.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.533+0000 7fd48e800640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55891f333890 0x55891f32fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:49.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.533+0000 7fd485400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55891f32acb0 0x55891f33f130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:44398/0 (socket says 172.21.15.45:44398) 2024-04-16T17:15:49.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.533+0000 7fd485400640 1 -- 172.21.15.45:0/2486579246 learned_addr learned my addr 172.21.15.45:0/2486579246 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:15:49.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.533+0000 7fd485400640 1 -- 172.21.15.45:0/2486579246 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55891f32a390 msgr2=0x55891f323b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:15:49.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.533+0000 7fd485400640 1 --2- 172.21.15.45:0/2486579246 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55891f32a390 0x55891f323b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:49.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.533+0000 7fd485400640 1 -- 172.21.15.45:0/2486579246 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55891f333890 msgr2=0x55891f32fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:15:49.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.533+0000 7fd485400640 1 --2- 172.21.15.45:0/2486579246 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55891f333890 0x55891f32fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:49.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.533+0000 7fd485400640 1 -- 172.21.15.45:0/2486579246 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55891f335c80 con 0x55891f32acb0 2024-04-16T17:15:49.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.534+0000 7fd485400640 1 --2- 172.21.15.45:0/2486579246 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55891f32acb0 0x55891f33f130 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fd478000c00 tx=0x7fd478005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4da288f81d13d5eb server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:15:49.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.534+0000 7fd47e600640 1 -- 172.21.15.45:0/2486579246 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd478024070 con 0x55891f32acb0 2024-04-16T17:15:49.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.534+0000 7fd47e600640 1 -- 172.21.15.45:0/2486579246 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd478010e50 con 0x55891f32acb0 2024-04-16T17:15:49.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.534+0000 7fd47e600640 1 -- 172.21.15.45:0/2486579246 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd478003830 con 0x55891f32acb0 2024-04-16T17:15:49.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.535+0000 7fd4977f1740 1 -- 172.21.15.45:0/2486579246 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55891f32acb0 msgr2=0x55891f33f130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:15:49.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.535+0000 7fd4977f1740 1 --2- 172.21.15.45:0/2486579246 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55891f32acb0 0x55891f33f130 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fd478000c00 tx=0x7fd478005f90 comp rx=0 tx=0).stop 2024-04-16T17:15:49.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.535+0000 7fd4977f1740 1 -- 172.21.15.45:0/2486579246 shutdown_connections 2024-04-16T17:15:49.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.535+0000 7fd4977f1740 1 --2- 172.21.15.45:0/2486579246 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55891f333890 0x55891f32fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:49.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.535+0000 7fd4977f1740 1 --2- 172.21.15.45:0/2486579246 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55891f32acb0 0x55891f33f130 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:49.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.535+0000 7fd4977f1740 1 --2- 172.21.15.45:0/2486579246 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55891f32a390 0x55891f323b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:49.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.535+0000 7fd4977f1740 1 -- 172.21.15.45:0/2486579246 >> 172.21.15.45:0/2486579246 conn(0x55891f324ac0 msgr2=0x55891f3308d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:15:49.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.535+0000 7fd4977f1740 1 -- 172.21.15.45:0/2486579246 shutdown_connections 2024-04-16T17:15:49.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.535+0000 7fd4977f1740 1 -- 172.21.15.45:0/2486579246 wait complete. 2024-04-16T17:15:49.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.535+0000 7fd4977f1740 1 -- start start 2024-04-16T17:15:49.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.535+0000 7fd4977f1740 1 Processor -- start 2024-04-16T17:15:49.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.536+0000 7fd4977f1740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55891f32a390 0x55891f33bba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:49.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.536+0000 7fd4977f1740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55891f32acb0 0x55891f336f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:49.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.536+0000 7fd4977f1740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55891f333890 0x55891f337600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:49.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.536+0000 7fd4977f1740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fd4901afc10 con 0x55891f333890 2024-04-16T17:15:49.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.536+0000 7fd4977f1740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fd4901b1080 con 0x55891f32acb0 2024-04-16T17:15:49.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.536+0000 7fd4977f1740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fd4901b02e0 con 0x55891f32a390 2024-04-16T17:15:49.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.536+0000 7fd4977f1740 10 client.0 _ll_register_callbacks cb 0x7fd478009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:15:49.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.536+0000 7fd48e800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55891f333890 0x55891f337600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:49.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.536+0000 7fd48de00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55891f32a390 0x55891f33bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:49.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.536+0000 7fd48e800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55891f333890 0x55891f337600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:44400/0 (socket says 172.21.15.45:44400) 2024-04-16T17:15:49.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.536+0000 7fd485400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55891f32acb0 0x55891f336f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:49.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.536+0000 7fd48e800640 1 -- 172.21.15.45:0/3921524501 learned_addr learned my addr 172.21.15.45:0/3921524501 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:15:49.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.536+0000 7fd48e800640 1 -- 172.21.15.45:0/3921524501 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55891f32a390 msgr2=0x55891f33bba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:15:49.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.536+0000 7fd48e800640 1 --2- 172.21.15.45:0/3921524501 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55891f32a390 0x55891f33bba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:49.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.536+0000 7fd48e800640 1 -- 172.21.15.45:0/3921524501 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55891f32acb0 msgr2=0x55891f336f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:15:49.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.537+0000 7fd48e800640 1 --2- 172.21.15.45:0/3921524501 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55891f32acb0 0x55891f336f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:49.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.537+0000 7fd48e800640 1 -- 172.21.15.45:0/3921524501 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55891f3a8be0 con 0x55891f333890 2024-04-16T17:15:49.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.537+0000 7fd48de00640 1 --2- 172.21.15.45:0/3921524501 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55891f32a390 0x55891f33bba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:15:49.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.537+0000 7fd48e800640 1 --2- 172.21.15.45:0/3921524501 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55891f333890 0x55891f337600 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fd4880045d0 tx=0x7fd488004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c86e96a80712e2a5 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:15:49.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.537+0000 7fd473400640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-16T17:15:49.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.537+0000 7fd473400640 1 -- 172.21.15.45:0/3921524501 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd48802f060 con 0x55891f333890 2024-04-16T17:15:49.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.537+0000 7fd4977f1740 1 -- 172.21.15.45:0/3921524501 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd478010e50 con 0x55891f333890 2024-04-16T17:15:49.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.537+0000 7fd473400640 1 -- 172.21.15.45:0/3921524501 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9496+0+0 (secure 0 0 0) 0x7fd488003e00 con 0x55891f333890 2024-04-16T17:15:49.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.537+0000 7fd473400640 1 -- 172.21.15.45:0/3921524501 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fd488036050 con 0x55891f333890 2024-04-16T17:15:49.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.538+0000 7fd4977f1740 10 client.17874 fetch_fsmap learned FSMap version 43 2024-04-16T17:15:49.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.538+0000 7fd4977f1740 1 -- 172.21.15.45:0/3921524501 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55891f3a9e40 con 0x55891f333890 2024-04-16T17:15:49.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.538+0000 7fd473400640 1 -- 172.21.15.45:0/3921524501 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd478010e50 con 0x55891f333890 2024-04-16T17:15:49.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.538+0000 7fd473400640 1 -- 172.21.15.45:0/3921524501 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7fd488029070 con 0x55891f333890 2024-04-16T17:15:49.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.538+0000 7fd4977f1740 10 client.17874 fetch_fsmap finished waiting for FSMap version 43 2024-04-16T17:15:49.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.538+0000 7fd4977f1740 10 client.17874 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:15:49.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.538+0000 7fd4977f1740 20 client.17874 populate_metadata read hostname 'smithi045' 2024-04-16T17:15:49.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.538+0000 7fd4977f1740 1 --2- 172.21.15.45:0/3921524501 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55891f3aa540 0x55891f3ac960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:15:49.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.538+0000 7fd4977f1740 4 client.17874 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-16T17:15:49.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.538+0000 7fd4977f1740 1 -- 172.21.15.45:0/3921524501 --> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55891f2979b0 con 0x55891f3aa540 2024-04-16T17:15:49.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.538+0000 7fd48de00640 1 --2- 172.21.15.45:0/3921524501 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55891f3aa540 0x55891f3ac960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:15:49.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.539+0000 7fd48de00640 1 --2- 172.21.15.45:0/3921524501 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55891f3aa540 0x55891f3ac960 crc :-1 s=READY pgs=240 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e2ed38c7fca47cb8 server_cookie=9f19f996483c97bf in_seq=0 out_seq=0 2024-04-16T17:15:49.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.539+0000 7fd473400640 10 client.17874 ms_handle_connect on v2:172.21.15.104:6838/3660070592 2024-04-16T17:15:49.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.539+0000 7fd473400640 1 -- 172.21.15.45:0/3921524501 <== mds.0 v2:172.21.15.104:6838/3660070592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55891f2979b0 con 0x55891f3aa540 2024-04-16T17:15:49.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.539+0000 7fd473400640 10 client.17874 handle_command_reply: tid=0 2024-04-16T17:15:49.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.539+0000 7fd4977f1740 1 client.17874 shutdown 2024-04-16T17:15:49.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.539+0000 7fd4977f1740 1 -- 172.21.15.45:0/3921524501 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55891f333890 msgr2=0x55891f337600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:15:49.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.539+0000 7fd4977f1740 1 --2- 172.21.15.45:0/3921524501 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55891f333890 0x55891f337600 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fd4880045d0 tx=0x7fd488004730 comp rx=0 tx=0).stop 2024-04-16T17:15:49.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.539+0000 7fd4977f1740 1 -- 172.21.15.45:0/3921524501 shutdown_connections 2024-04-16T17:15:49.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.540+0000 7fd4977f1740 1 --2- 172.21.15.45:0/3921524501 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55891f333890 0x55891f337600 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:49.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.540+0000 7fd4977f1740 1 --2- 172.21.15.45:0/3921524501 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55891f32acb0 0x55891f336f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:49.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.540+0000 7fd4977f1740 1 --2- 172.21.15.45:0/3921524501 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55891f3aa540 0x55891f3ac960 crc :-1 s=READY pgs=240 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:49.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.540+0000 7fd4977f1740 1 --2- 172.21.15.45:0/3921524501 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55891f32a390 0x55891f33bba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:49.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.540+0000 7fd4977f1740 1 -- 172.21.15.45:0/3921524501 >> 172.21.15.45:0/3921524501 conn(0x55891f324ac0 msgr2=0x55891f3264e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:15:49.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.540+0000 7fd473400640 0 client.17874 ms_handle_reset on v2:172.21.15.104:6838/3660070592 2024-04-16T17:15:49.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.540+0000 7fd4977f1740 1 -- 172.21.15.45:0/3921524501 shutdown_connections 2024-04-16T17:15:49.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.540+0000 7fd4977f1740 1 -- 172.21.15.45:0/3921524501 wait complete. 2024-04-16T17:15:49.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.540+0000 7fd4977f1740 20 client.17874 trim_cache size 0 max 16384 2024-04-16T17:15:49.548 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-16T17:15:49.548 INFO:teuthology.orchestra.run.smithi045.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-16T17:15:49.548 INFO:teuthology.orchestra.run.smithi045.stdout: "scrubs": { 2024-04-16T17:15:49.548 INFO:teuthology.orchestra.run.smithi045.stdout: "fae45a96-bda2-4f23-8a83-72b7bd836a76": { 2024-04-16T17:15:49.549 INFO:teuthology.orchestra.run.smithi045.stdout: "path": "/", 2024-04-16T17:15:49.549 INFO:teuthology.orchestra.run.smithi045.stdout: "tag": "fae45a96-bda2-4f23-8a83-72b7bd836a76", 2024-04-16T17:15:49.549 INFO:teuthology.orchestra.run.smithi045.stdout: "options": "recursive,force" 2024-04-16T17:15:49.549 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:15:49.549 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:15:49.549 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-16T17:15:49.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.540+0000 7fd48f200640 1 -- 172.21.15.45:0/2399701779 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fd4740a3d20 msgr2=0x7fd4740a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:15:49.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.540+0000 7fd48f200640 1 --2- 172.21.15.45:0/2399701779 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fd4740a3d20 0x7fd4740a61e0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fd488009ef0 tx=0x7fd488005f90 comp rx=0 tx=0).stop 2024-04-16T17:15:49.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.540+0000 7fd48f200640 1 -- 172.21.15.45:0/2399701779 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd490102110 msgr2=0x7fd4901a0630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:15:49.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.540+0000 7fd48f200640 1 --2- 172.21.15.45:0/2399701779 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd490102110 0x7fd4901a0630 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fd480005e30 tx=0x7fd480035d90 comp rx=0 tx=0).stop 2024-04-16T17:15:49.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.541+0000 7fd48f200640 1 -- 172.21.15.45:0/2399701779 shutdown_connections 2024-04-16T17:15:49.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.541+0000 7fd48f200640 1 --2- 172.21.15.45:0/2399701779 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fd4740a3d20 0x7fd4740a61e0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:49.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.541+0000 7fd48f200640 1 --2- 172.21.15.45:0/2399701779 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd49019a7a0 0x7fd49019ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:49.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.541+0000 7fd48f200640 1 --2- 172.21.15.45:0/2399701779 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd4901071e0 0x7fd4901a0b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:49.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.541+0000 7fd48f200640 1 --2- 172.21.15.45:0/2399701779 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd490102110 0x7fd4901a0630 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:15:49.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.541+0000 7fd48f200640 1 -- 172.21.15.45:0/2399701779 >> 172.21.15.45:0/2399701779 conn(0x7fd4900fd9d0 msgr2=0x7fd4900ffcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:15:49.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.541+0000 7fd48f200640 1 -- 172.21.15.45:0/2399701779 shutdown_connections 2024-04-16T17:15:49.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:15:49.541+0000 7fd48f200640 1 -- 172.21.15.45:0/2399701779 wait complete. 2024-04-16T17:15:49.557 INFO:tasks.cephfs.filesystem:scrub status for tag:fae45a96-bda2-4f23-8a83-72b7bd836a76 - {'path': '/', 'tag': 'fae45a96-bda2-4f23-8a83-72b7bd836a76', 'options': 'recursive,force'} 2024-04-16T17:15:49.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:15:49 smithi104 ceph-mon[36457]: scrub queued for path: / 2024-04-16T17:15:49.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:15:49 smithi104 ceph-mon[36457]: scrub summary: idle+waiting paths [/] 2024-04-16T17:15:49.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:15:49 smithi104 ceph-mon[36457]: scrub summary: active paths [/] 2024-04-16T17:15:49.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:15:49 smithi104 ceph-mon[36457]: scrub summary: active paths [/] 2024-04-16T17:15:49.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:15:49 smithi104 ceph-mon[36457]: scrub summary: idle+waiting paths [/] 2024-04-16T17:15:49.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:15:49 smithi104 ceph-mon[36457]: scrub summary: active paths [/] 2024-04-16T17:15:49.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:15:49 smithi104 ceph-mon[36457]: scrub summary: idle+waiting paths [/] 2024-04-16T17:15:49.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:15:49 smithi104 ceph-mon[36457]: scrub summary: idle+waiting paths [/] 2024-04-16T17:15:49.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:15:49 smithi045 ceph-mon[31676]: scrub queued for path: / 2024-04-16T17:15:49.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:15:49 smithi045 ceph-mon[31676]: scrub summary: idle+waiting paths [/] 2024-04-16T17:15:49.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:15:49 smithi045 ceph-mon[31676]: scrub summary: active paths [/] 2024-04-16T17:15:49.978 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:15:49 smithi045 ceph-mon[31676]: scrub summary: active paths [/] 2024-04-16T17:15:49.978 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:15:49 smithi045 ceph-mon[31676]: scrub summary: idle+waiting paths [/] 2024-04-16T17:15:49.978 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:15:49 smithi045 ceph-mon[31676]: scrub summary: active paths [/] 2024-04-16T17:15:49.978 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:15:49 smithi045 ceph-mon[31676]: scrub summary: idle+waiting paths [/] 2024-04-16T17:15:49.978 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:15:49 smithi045 ceph-mon[31676]: scrub summary: idle+waiting paths [/] 2024-04-16T17:15:50.054 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:15:49 smithi117 ceph-mon[36730]: scrub queued for path: / 2024-04-16T17:15:50.054 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:15:49 smithi117 ceph-mon[36730]: scrub summary: idle+waiting paths [/] 2024-04-16T17:15:50.055 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:15:49 smithi117 ceph-mon[36730]: scrub summary: active paths [/] 2024-04-16T17:15:50.055 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:15:49 smithi117 ceph-mon[36730]: scrub summary: active paths [/] 2024-04-16T17:15:50.055 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:15:49 smithi117 ceph-mon[36730]: scrub summary: idle+waiting paths [/] 2024-04-16T17:15:50.055 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:15:49 smithi117 ceph-mon[36730]: scrub summary: active paths [/] 2024-04-16T17:15:50.055 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:15:49 smithi117 ceph-mon[36730]: scrub summary: idle+waiting paths [/] 2024-04-16T17:15:50.055 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:15:49 smithi117 ceph-mon[36730]: scrub summary: idle+waiting paths [/] 2024-04-16T17:15:50.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:15:50 smithi104 ceph-mon[36457]: scrub summary: active paths [/] 2024-04-16T17:15:50.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:15:50 smithi104 ceph-mon[36457]: scrub summary: idle+waiting paths [/] 2024-04-16T17:15:50.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:15:50 smithi104 ceph-mon[36457]: pgmap v282: 129 pgs: 129 active+clean; 60 MiB data, 880 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T17:15:50.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:15:50 smithi045 ceph-mon[31676]: scrub summary: active paths [/] 2024-04-16T17:15:50.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:15:50 smithi045 ceph-mon[31676]: scrub summary: idle+waiting paths [/] 2024-04-16T17:15:50.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:15:50 smithi045 ceph-mon[31676]: pgmap v282: 129 pgs: 129 active+clean; 60 MiB data, 880 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T17:15:51.054 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:15:50 smithi117 ceph-mon[36730]: scrub summary: active paths [/] 2024-04-16T17:15:51.054 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:15:50 smithi117 ceph-mon[36730]: scrub summary: idle+waiting paths [/] 2024-04-16T17:15:51.054 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:15:50 smithi117 ceph-mon[36730]: pgmap v282: 129 pgs: 129 active+clean; 60 MiB data, 880 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T17:15:52.978 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:15:52 smithi045 ceph-mon[31676]: pgmap v283: 129 pgs: 129 active+clean; 60 MiB data, 880 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 14 op/s 2024-04-16T17:15:53.054 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:15:52 smithi117 ceph-mon[36730]: pgmap v283: 129 pgs: 129 active+clean; 60 MiB data, 880 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 14 op/s 2024-04-16T17:15:53.177 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:15:52 smithi104 ceph-mon[36457]: pgmap v283: 129 pgs: 129 active+clean; 60 MiB data, 880 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 14 op/s 2024-04-16T17:15:53.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:15:53 smithi045 ceph-mon[31676]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-16T17:15:54.054 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:15:53 smithi117 ceph-mon[36730]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-16T17:15:54.177 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:15:53 smithi104 ceph-mon[36457]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-16T17:15:54.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:15:54 smithi045 ceph-mon[31676]: pgmap v284: 129 pgs: 129 active+clean; 60 MiB data, 880 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 11 op/s 2024-04-16T17:15:55.054 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:15:54 smithi117 ceph-mon[36730]: pgmap v284: 129 pgs: 129 active+clean; 60 MiB data, 880 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 11 op/s 2024-04-16T17:15:55.177 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:15:54 smithi104 ceph-mon[36457]: pgmap v284: 129 pgs: 129 active+clean; 60 MiB data, 880 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 11 op/s 2024-04-16T17:15:56.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:15:56 smithi045 ceph-mon[31676]: pgmap v285: 129 pgs: 129 active+clean; 60 MiB data, 880 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 11 op/s 2024-04-16T17:15:57.054 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:15:56 smithi117 ceph-mon[36730]: pgmap v285: 129 pgs: 129 active+clean; 60 MiB data, 880 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 11 op/s 2024-04-16T17:15:57.177 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:15:56 smithi104 ceph-mon[36457]: pgmap v285: 129 pgs: 129 active+clean; 60 MiB data, 880 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 11 op/s 2024-04-16T17:15:58.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:15:58 smithi045 ceph-mon[31676]: pgmap v286: 129 pgs: 129 active+clean; 60 MiB data, 880 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 14 op/s 2024-04-16T17:15:58.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:15:58 smithi045 ceph-mon[31676]: scrub summary: idle 2024-04-16T17:15:59.054 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:15:58 smithi117 ceph-mon[36730]: pgmap v286: 129 pgs: 129 active+clean; 60 MiB data, 880 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 14 op/s 2024-04-16T17:15:59.054 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:15:58 smithi117 ceph-mon[36730]: scrub summary: idle 2024-04-16T17:15:59.177 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:15:58 smithi104 ceph-mon[36457]: pgmap v286: 129 pgs: 129 active+clean; 60 MiB data, 880 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 14 op/s 2024-04-16T17:15:59.177 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:15:58 smithi104 ceph-mon[36457]: scrub summary: idle 2024-04-16T17:15:59.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:15:59 smithi045 ceph-mon[31676]: scrub summary: idle 2024-04-16T17:15:59.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:15:59 smithi045 ceph-mon[31676]: scrub summary: idle 2024-04-16T17:16:00.054 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:15:59 smithi117 ceph-mon[36730]: scrub summary: idle 2024-04-16T17:16:00.055 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:15:59 smithi117 ceph-mon[36730]: scrub summary: idle 2024-04-16T17:16:00.177 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:15:59 smithi104 ceph-mon[36457]: scrub summary: idle 2024-04-16T17:16:00.250 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:15:59 smithi104 ceph-mon[36457]: scrub summary: idle 2024-04-16T17:16:00.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:16:00 smithi045 ceph-mon[31676]: pgmap v287: 129 pgs: 129 active+clean; 60 MiB data, 880 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 11 op/s 2024-04-16T17:16:01.054 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:16:00 smithi117 ceph-mon[36730]: pgmap v287: 129 pgs: 129 active+clean; 60 MiB data, 880 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 11 op/s 2024-04-16T17:16:01.177 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:16:00 smithi104 ceph-mon[36457]: pgmap v287: 129 pgs: 129 active+clean; 60 MiB data, 880 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 11 op/s 2024-04-16T17:16:02.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:16:02 smithi045 ceph-mon[31676]: pgmap v288: 129 pgs: 129 active+clean; 60 MiB data, 880 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 45 KiB/s wr, 31 op/s 2024-04-16T17:16:03.054 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:16:02 smithi117 ceph-mon[36730]: pgmap v288: 129 pgs: 129 active+clean; 60 MiB data, 880 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 45 KiB/s wr, 31 op/s 2024-04-16T17:16:03.177 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:16:02 smithi104 ceph-mon[36457]: pgmap v288: 129 pgs: 129 active+clean; 60 MiB data, 880 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 45 KiB/s wr, 31 op/s 2024-04-16T17:16:04.054 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:16:03 smithi117 ceph-mon[36730]: pgmap v289: 129 pgs: 129 active+clean; 60 MiB data, 880 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-04-16T17:16:04.177 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:16:03 smithi104 ceph-mon[36457]: pgmap v289: 129 pgs: 129 active+clean; 60 MiB data, 880 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-04-16T17:16:04.226 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:16:03 smithi045 ceph-mon[31676]: pgmap v289: 129 pgs: 129 active+clean; 60 MiB data, 880 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-04-16T17:16:06.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:16:06 smithi045 ceph-mon[31676]: pgmap v290: 129 pgs: 129 active+clean; 60 MiB data, 880 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-04-16T17:16:06.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:16:06 smithi045 ceph-mon[31676]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T17:16:06.804 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:16:06 smithi117 ceph-mon[36730]: pgmap v290: 129 pgs: 129 active+clean; 60 MiB data, 880 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-04-16T17:16:06.804 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:16:06 smithi117 ceph-mon[36730]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T17:16:06.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:16:06 smithi104 ceph-mon[36457]: pgmap v290: 129 pgs: 129 active+clean; 60 MiB data, 880 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-04-16T17:16:06.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:16:06 smithi104 ceph-mon[36457]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T17:16:08.177 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:16:07 smithi104 ceph-mon[36457]: from='mgr.25411 ' entity='mgr.x' 2024-04-16T17:16:08.178 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:16:07 smithi104 ceph-mon[36457]: from='mgr.25411 ' entity='mgr.x' 2024-04-16T17:16:08.178 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:16:07 smithi104 ceph-mon[36457]: from='mgr.25411 ' entity='mgr.x' 2024-04-16T17:16:08.178 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:16:07 smithi104 ceph-mon[36457]: from='mgr.25411 ' entity='mgr.x' 2024-04-16T17:16:08.178 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:16:07 smithi104 ceph-mon[36457]: from='mgr.25411 ' entity='mgr.x' 2024-04-16T17:16:08.179 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:16:07 smithi104 ceph-mon[36457]: from='mgr.25411 ' entity='mgr.x' 2024-04-16T17:16:08.179 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:16:07 smithi104 ceph-mon[36457]: pgmap v291: 129 pgs: 129 active+clean; 60 MiB data, 880 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 24 op/s 2024-04-16T17:16:08.227 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:16:07 smithi045 ceph-mon[31676]: from='mgr.25411 ' entity='mgr.x' 2024-04-16T17:16:08.227 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:16:07 smithi045 ceph-mon[31676]: from='mgr.25411 ' entity='mgr.x' 2024-04-16T17:16:08.227 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:16:07 smithi045 ceph-mon[31676]: from='mgr.25411 ' entity='mgr.x' 2024-04-16T17:16:08.227 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:16:07 smithi045 ceph-mon[31676]: from='mgr.25411 ' entity='mgr.x' 2024-04-16T17:16:08.228 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:16:07 smithi045 ceph-mon[31676]: from='mgr.25411 ' entity='mgr.x' 2024-04-16T17:16:08.228 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:16:07 smithi045 ceph-mon[31676]: from='mgr.25411 ' entity='mgr.x' 2024-04-16T17:16:08.228 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:16:07 smithi045 ceph-mon[31676]: pgmap v291: 129 pgs: 129 active+clean; 60 MiB data, 880 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 24 op/s 2024-04-16T17:16:08.304 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:16:07 smithi117 ceph-mon[36730]: from='mgr.25411 ' entity='mgr.x' 2024-04-16T17:16:08.305 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:16:07 smithi117 ceph-mon[36730]: from='mgr.25411 ' entity='mgr.x' 2024-04-16T17:16:08.305 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:16:07 smithi117 ceph-mon[36730]: from='mgr.25411 ' entity='mgr.x' 2024-04-16T17:16:08.306 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:16:07 smithi117 ceph-mon[36730]: from='mgr.25411 ' entity='mgr.x' 2024-04-16T17:16:08.306 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:16:07 smithi117 ceph-mon[36730]: from='mgr.25411 ' entity='mgr.x' 2024-04-16T17:16:08.306 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:16:07 smithi117 ceph-mon[36730]: from='mgr.25411 ' entity='mgr.x' 2024-04-16T17:16:08.306 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:16:07 smithi117 ceph-mon[36730]: pgmap v291: 129 pgs: 129 active+clean; 60 MiB data, 880 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 24 op/s 2024-04-16T17:16:09.427 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:16:09 smithi104 ceph-mon[36457]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:16:09.427 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:16:09 smithi104 ceph-mon[36457]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T17:16:09.427 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:16:09 smithi104 ceph-mon[36457]: from='mgr.25411 ' entity='mgr.x' 2024-04-16T17:16:09.476 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:16:09 smithi045 ceph-mon[31676]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:16:09.476 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:16:09 smithi045 ceph-mon[31676]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T17:16:09.477 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:16:09 smithi045 ceph-mon[31676]: from='mgr.25411 ' entity='mgr.x' 2024-04-16T17:16:09.554 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:16:09 smithi117 ceph-mon[36730]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:16:09.554 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:16:09 smithi117 ceph-mon[36730]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T17:16:09.554 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:16:09 smithi117 ceph-mon[36730]: from='mgr.25411 ' entity='mgr.x' 2024-04-16T17:16:10.477 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:16:10 smithi045 ceph-mon[31676]: osdmap e103: 12 total, 12 up, 12 in 2024-04-16T17:16:10.477 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:16:10 smithi045 ceph-mon[31676]: pgmap v293: 129 pgs: 129 active+clean; 60 MiB data, 880 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-04-16T17:16:10.554 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:16:10 smithi117 ceph-mon[36730]: osdmap e103: 12 total, 12 up, 12 in 2024-04-16T17:16:10.554 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:16:10 smithi117 ceph-mon[36730]: pgmap v293: 129 pgs: 129 active+clean; 60 MiB data, 880 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-04-16T17:16:10.677 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:16:10 smithi104 ceph-mon[36457]: osdmap e103: 12 total, 12 up, 12 in 2024-04-16T17:16:10.677 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:16:10 smithi104 ceph-mon[36457]: pgmap v293: 129 pgs: 129 active+clean; 60 MiB data, 880 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-04-16T17:16:12.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:16:12 smithi045 ceph-mon[31676]: pgmap v294: 129 pgs: 129 active+clean; 56 MiB data, 866 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-16T17:16:12.804 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:16:12 smithi117 ceph-mon[36730]: pgmap v294: 129 pgs: 129 active+clean; 56 MiB data, 866 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-16T17:16:12.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:16:12 smithi104 ceph-mon[36457]: pgmap v294: 129 pgs: 129 active+clean; 56 MiB data, 866 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-16T17:16:14.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:16:14 smithi045 ceph-mon[31676]: pgmap v295: 129 pgs: 129 active+clean; 56 MiB data, 866 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-16T17:16:14.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:16:14 smithi045 ceph-mon[31676]: osdmap e104: 12 total, 12 up, 12 in 2024-04-16T17:16:14.729 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:16:14 smithi104 ceph-mon[36457]: pgmap v295: 129 pgs: 129 active+clean; 56 MiB data, 866 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-16T17:16:14.729 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:16:14 smithi104 ceph-mon[36457]: osdmap e104: 12 total, 12 up, 12 in 2024-04-16T17:16:14.804 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:16:14 smithi117 ceph-mon[36730]: pgmap v295: 129 pgs: 129 active+clean; 56 MiB data, 866 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-16T17:16:14.804 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:16:14 smithi117 ceph-mon[36730]: osdmap e104: 12 total, 12 up, 12 in 2024-04-16T17:16:16.804 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:16:16 smithi117 ceph-mon[36730]: pgmap v297: 129 pgs: 129 active+clean; 56 MiB data, 866 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 127 B/s wr, 3 op/s 2024-04-16T17:16:16.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:16:16 smithi104 ceph-mon[36457]: pgmap v297: 129 pgs: 129 active+clean; 56 MiB data, 866 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 127 B/s wr, 3 op/s 2024-04-16T17:16:16.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:16:16 smithi045 ceph-mon[31676]: pgmap v297: 129 pgs: 129 active+clean; 56 MiB data, 866 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 127 B/s wr, 3 op/s 2024-04-16T17:16:18.804 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:16:18 smithi117 ceph-mon[36730]: pgmap v298: 129 pgs: 129 active+clean; 56 MiB data, 870 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 248 B/s wr, 7 op/s 2024-04-16T17:16:18.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:16:18 smithi104 ceph-mon[36457]: pgmap v298: 129 pgs: 129 active+clean; 56 MiB data, 870 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 248 B/s wr, 7 op/s 2024-04-16T17:16:18.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:16:18 smithi045 ceph-mon[31676]: pgmap v298: 129 pgs: 129 active+clean; 56 MiB data, 870 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 248 B/s wr, 7 op/s 2024-04-16T17:16:19.558 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-16T17:16:19.558 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-16T17:16:19.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:19.798+0000 7f62eb200640 1 -- 172.21.15.45:0/3713241863 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f62ec074cd0 msgr2=0x7f62ec10e230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:19.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:19.798+0000 7f62eb200640 1 --2- 172.21.15.45:0/3713241863 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f62ec074cd0 0x7f62ec10e230 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f62d8009900 tx=0x7f62d802bcc0 comp rx=0 tx=0).stop 2024-04-16T17:16:19.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:19.798+0000 7f62eb200640 1 -- 172.21.15.45:0/3713241863 shutdown_connections 2024-04-16T17:16:19.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:19.798+0000 7f62eb200640 1 --2- 172.21.15.45:0/3713241863 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f62ec074cd0 0x7f62ec10e230 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:19.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:19.798+0000 7f62eb200640 1 --2- 172.21.15.45:0/3713241863 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f62ec10e770 0x7f62ec110b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:19.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:19.798+0000 7f62eb200640 1 --2- 172.21.15.45:0/3713241863 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f62ec0743b0 0x7f62ec074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:19.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:19.798+0000 7f62eb200640 1 -- 172.21.15.45:0/3713241863 >> 172.21.15.45:0/3713241863 conn(0x7f62ec0fd9a0 msgr2=0x7f62ec0ffdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:16:19.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:19.798+0000 7f62eb200640 1 -- 172.21.15.45:0/3713241863 shutdown_connections 2024-04-16T17:16:19.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:19.799+0000 7f62eb200640 1 -- 172.21.15.45:0/3713241863 wait complete. 2024-04-16T17:16:19.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:19.799+0000 7f62eb200640 1 Processor -- start 2024-04-16T17:16:19.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:19.800+0000 7f62eb200640 1 -- start start 2024-04-16T17:16:19.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:19.800+0000 7f62eb200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f62ec0743b0 0x7f62ec1a0670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:19.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:19.800+0000 7f62eb200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f62ec074cd0 0x7f62ec1a0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:19.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:19.800+0000 7f62eb200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f62ec10e770 0x7f62ec19a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:19.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:19.800+0000 7f62eb200640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f62ec1144b0 con 0x7f62ec0743b0 2024-04-16T17:16:19.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:19.800+0000 7f62eb200640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f62ec114330 con 0x7f62ec10e770 2024-04-16T17:16:19.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:19.800+0000 7f62eb200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f62ec114630 con 0x7f62ec074cd0 2024-04-16T17:16:19.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:19.800+0000 7f62e9e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f62ec0743b0 0x7f62ec1a0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:19.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:19.800+0000 7f62e9e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f62ec0743b0 0x7f62ec1a0670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:48946/0 (socket says 172.21.15.45:48946) 2024-04-16T17:16:19.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:19.800+0000 7f62e9e00640 1 -- 172.21.15.45:0/2031778513 learned_addr learned my addr 172.21.15.45:0/2031778513 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:16:19.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:19.800+0000 7f62e9400640 1 --2- 172.21.15.45:0/2031778513 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f62ec074cd0 0x7f62ec1a0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:19.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:19.801+0000 7f62ea800640 1 --2- 172.21.15.45:0/2031778513 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f62ec10e770 0x7f62ec19a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:19.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:19.801+0000 7f62e9e00640 1 -- 172.21.15.45:0/2031778513 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f62ec074cd0 msgr2=0x7f62ec1a0bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:19.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:19.801+0000 7f62e9e00640 1 --2- 172.21.15.45:0/2031778513 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f62ec074cd0 0x7f62ec1a0bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:19.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:19.801+0000 7f62e9e00640 1 -- 172.21.15.45:0/2031778513 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f62ec10e770 msgr2=0x7f62ec19a830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:19.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:19.801+0000 7f62e9e00640 1 --2- 172.21.15.45:0/2031778513 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f62ec10e770 0x7f62ec19a830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:19.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:19.801+0000 7f62e9e00640 1 -- 172.21.15.45:0/2031778513 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62ec19b000 con 0x7f62ec0743b0 2024-04-16T17:16:19.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:19.801+0000 7f62e9e00640 1 --2- 172.21.15.45:0/2031778513 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f62ec0743b0 0x7f62ec1a0670 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f62dc00ea60 tx=0x7f62dc00ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:16:19.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:19.801+0000 7f62e2a00640 1 -- 172.21.15.45:0/2031778513 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62dc00cd20 con 0x7f62ec0743b0 2024-04-16T17:16:19.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:19.802+0000 7f62eb200640 1 -- 172.21.15.45:0/2031778513 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62ec19b2f0 con 0x7f62ec0743b0 2024-04-16T17:16:19.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:19.802+0000 7f62e2a00640 1 -- 172.21.15.45:0/2031778513 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f62dc0035f0 con 0x7f62ec0743b0 2024-04-16T17:16:19.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:19.802+0000 7f62e2a00640 1 -- 172.21.15.45:0/2031778513 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62dc010690 con 0x7f62ec0743b0 2024-04-16T17:16:19.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:19.802+0000 7f62eb200640 1 -- 172.21.15.45:0/2031778513 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62ec1a74c0 con 0x7f62ec0743b0 2024-04-16T17:16:19.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:19.805+0000 7f62e2a00640 1 -- 172.21.15.45:0/2031778513 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f62dc0037a0 con 0x7f62ec0743b0 2024-04-16T17:16:19.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:19.805+0000 7f62f3bd1740 1 Processor -- start 2024-04-16T17:16:19.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:19.805+0000 7f62f3bd1740 1 -- start start 2024-04-16T17:16:19.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:19.805+0000 7f62f3bd1740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558936fb8390 0x558936fb8770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:19.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:19.805+0000 7f62f3bd1740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558936fb8cb0 0x558936fc1350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:19.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:19.805+0000 7f62f3bd1740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558936fc1890 0x558936fc3c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:19.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:19.805+0000 7f62f3bd1740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x558936f1e120 con 0x558936fc1890 2024-04-16T17:16:19.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:19.805+0000 7f62f3bd1740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x558936ec4340 con 0x558936fb8390 2024-04-16T17:16:19.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:19.805+0000 7f62f3bd1740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x558936f1a0e0 con 0x558936fb8cb0 2024-04-16T17:16:19.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:19.806+0000 7f62e9400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558936fc1890 0x558936fc3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:19.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:19.806+0000 7f62ea800640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558936fb8390 0x558936fb8770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:19.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:19.806+0000 7f62e9400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558936fc1890 0x558936fc3c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:48962/0 (socket says 172.21.15.45:48962) 2024-04-16T17:16:19.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:19.806+0000 7f62e9400640 1 -- 172.21.15.45:0/1177073530 learned_addr learned my addr 172.21.15.45:0/1177073530 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:16:19.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:19.806+0000 7f62e9e00640 1 --2- 172.21.15.45:0/1177073530 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558936fb8cb0 0x558936fc1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:19.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:19.806+0000 7f62e9400640 1 -- 172.21.15.45:0/1177073530 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558936fb8cb0 msgr2=0x558936fc1350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:16:19.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:19.806+0000 7f62e9400640 1 --2- 172.21.15.45:0/1177073530 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558936fb8cb0 0x558936fc1350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:19.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:19.806+0000 7f62e9400640 1 -- 172.21.15.45:0/1177073530 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558936fb8390 msgr2=0x558936fb8770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:16:19.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:19.806+0000 7f62e9400640 1 --2- 172.21.15.45:0/1177073530 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558936fb8390 0x558936fb8770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:19.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:19.806+0000 7f62e9400640 1 -- 172.21.15.45:0/1177073530 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558936cfa7d0 con 0x558936fc1890 2024-04-16T17:16:19.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:19.806+0000 7f62e9e00640 1 --2- 172.21.15.45:0/1177073530 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558936fb8cb0 0x558936fc1350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:16:19.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:19.807+0000 7f62e9400640 1 --2- 172.21.15.45:0/1177073530 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558936fc1890 0x558936fc3c80 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f62d8000c00 tx=0x7f62d8009560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f29dfd8efa6f4356 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:16:19.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:19.807+0000 7f62e2a00640 1 --2- 172.21.15.45:0/2031778513 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f62d00a3d20 0x7f62d00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:19.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:19.807+0000 7f62d7400640 1 -- 172.21.15.45:0/1177073530 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62d8043070 con 0x558936fc1890 2024-04-16T17:16:19.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:19.807+0000 7f62d7400640 1 -- 172.21.15.45:0/1177073530 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f62d802c6e0 con 0x558936fc1890 2024-04-16T17:16:19.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:19.807+0000 7f62d7400640 1 -- 172.21.15.45:0/1177073530 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62d8004a40 con 0x558936fc1890 2024-04-16T17:16:19.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:19.807+0000 7f62e2a00640 1 -- 172.21.15.45:0/2031778513 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 9496+0+0 (secure 0 0 0) 0x7f62dc0d0f20 con 0x7f62ec0743b0 2024-04-16T17:16:19.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:19.807+0000 7f62ea800640 1 --2- 172.21.15.45:0/2031778513 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f62d00a3d20 0x7f62d00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:19.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:19.808+0000 7f62f3bd1740 1 -- 172.21.15.45:0/1177073530 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558936fc1890 msgr2=0x558936fc3c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:19.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:19.808+0000 7f62f3bd1740 1 --2- 172.21.15.45:0/1177073530 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558936fc1890 0x558936fc3c80 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f62d8000c00 tx=0x7f62d8009560 comp rx=0 tx=0).stop 2024-04-16T17:16:19.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:19.808+0000 7f62f3bd1740 1 -- 172.21.15.45:0/1177073530 shutdown_connections 2024-04-16T17:16:19.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:19.808+0000 7f62f3bd1740 1 --2- 172.21.15.45:0/1177073530 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558936fb8cb0 0x558936fc1350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:19.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:19.808+0000 7f62f3bd1740 1 --2- 172.21.15.45:0/1177073530 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558936fc1890 0x558936fc3c80 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:19.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:19.808+0000 7f62f3bd1740 1 --2- 172.21.15.45:0/1177073530 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558936fb8390 0x558936fb8770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:19.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:19.808+0000 7f62f3bd1740 1 -- 172.21.15.45:0/1177073530 >> 172.21.15.45:0/1177073530 conn(0x558936fb2ac0 msgr2=0x558936fb2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:16:19.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:19.808+0000 7f62ea800640 1 --2- 172.21.15.45:0/2031778513 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f62d00a3d20 0x7f62d00a61e0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f62e4005fd0 tx=0x7f62e4009500 comp rx=0 tx=0).ready entity=mgr.25411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:16:19.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:19.808+0000 7f62f3bd1740 1 -- 172.21.15.45:0/1177073530 shutdown_connections 2024-04-16T17:16:19.814 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:19.808+0000 7f62f3bd1740 1 -- 172.21.15.45:0/1177073530 wait complete. 2024-04-16T17:16:19.814 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:19.808+0000 7f62f3bd1740 1 -- start start 2024-04-16T17:16:19.814 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:19.809+0000 7f62f3bd1740 1 Processor -- start 2024-04-16T17:16:19.814 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:19.809+0000 7f62f3bd1740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558936fb8390 0x558936fbe890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:19.814 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:19.809+0000 7f62f3bd1740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558936fb8cb0 0x558936fbedd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:19.814 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:19.809+0000 7f62f3bd1740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558936fc1890 0x558936fbf310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:19.814 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:19.809+0000 7f62f3bd1740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x558936fc4620 con 0x558936fb8390 2024-04-16T17:16:19.814 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:19.809+0000 7f62f3bd1740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x558936ed7710 con 0x558936fb8cb0 2024-04-16T17:16:19.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:19.809+0000 7f62f3bd1740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x558936fc47a0 con 0x558936fc1890 2024-04-16T17:16:19.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:19.809+0000 7f62f3bd1740 10 client.0 _ll_register_callbacks cb 0x558936ed2af0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:16:19.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:19.809+0000 7f62e9e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558936fb8390 0x558936fbe890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:19.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:19.809+0000 7f62e9400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558936fb8cb0 0x558936fbedd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:19.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:19.810+0000 7f62e9e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558936fb8390 0x558936fbe890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:48972/0 (socket says 172.21.15.45:48972) 2024-04-16T17:16:19.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:19.810+0000 7f62ea800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558936fc1890 0x558936fbf310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:19.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:19.810+0000 7f62e9e00640 1 -- 172.21.15.45:0/9952009 learned_addr learned my addr 172.21.15.45:0/9952009 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:16:19.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:19.810+0000 7f62e9e00640 1 -- 172.21.15.45:0/9952009 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558936fc1890 msgr2=0x558936fbf310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:16:19.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:19.810+0000 7f62e9e00640 1 --2- 172.21.15.45:0/9952009 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558936fc1890 0x558936fbf310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:19.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:19.810+0000 7f62e9e00640 1 -- 172.21.15.45:0/9952009 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558936fb8cb0 msgr2=0x558936fbedd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:16:19.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:19.810+0000 7f62e9e00640 1 --2- 172.21.15.45:0/9952009 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558936fb8cb0 0x558936fbedd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:19.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:19.810+0000 7f62e9e00640 1 -- 172.21.15.45:0/9952009 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558936c303f0 con 0x558936fb8390 2024-04-16T17:16:19.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:19.810+0000 7f62e9400640 1 --2- 172.21.15.45:0/9952009 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558936fb8cb0 0x558936fbedd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:16:19.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:19.810+0000 7f62e9e00640 1 --2- 172.21.15.45:0/9952009 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558936fb8390 0x558936fbe890 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f62dc01f000 tx=0x7f62dc09c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b50f5fca347916f3 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:16:19.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:19.810+0000 7f62cbe00640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-16T17:16:19.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:19.810+0000 7f62cbe00640 1 -- 172.21.15.45:0/9952009 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62dc09d190 con 0x558936fb8390 2024-04-16T17:16:19.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:19.811+0000 7f62cbe00640 1 -- 172.21.15.45:0/9952009 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9496+0+0 (secure 0 0 0) 0x7f62dc099100 con 0x558936fb8390 2024-04-16T17:16:19.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:19.811+0000 7f62f3bd1740 1 -- 172.21.15.45:0/9952009 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f62d802c6e0 con 0x558936fb8390 2024-04-16T17:16:19.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:19.811+0000 7f62cbe00640 1 -- 172.21.15.45:0/9952009 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f62dc09ca90 con 0x558936fb8390 2024-04-16T17:16:19.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:19.811+0000 7f62f3bd1740 10 client.17886 fetch_fsmap learned FSMap version 44 2024-04-16T17:16:19.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:19.811+0000 7f62f3bd1740 1 -- 172.21.15.45:0/9952009 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x558936bf00f0 con 0x558936fb8390 2024-04-16T17:16:19.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:19.811+0000 7f62cbe00640 1 -- 172.21.15.45:0/9952009 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f62d802c6e0 con 0x558936fb8390 2024-04-16T17:16:19.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:19.811+0000 7f62cbe00640 1 -- 172.21.15.45:0/9952009 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 44) ==== 8346+0+0 (secure 0 0 0) 0x7f62dc09bee0 con 0x558936fb8390 2024-04-16T17:16:19.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:19.811+0000 7f62f3bd1740 10 client.17886 fetch_fsmap finished waiting for FSMap version 44 2024-04-16T17:16:19.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:19.811+0000 7f62f3bd1740 10 client.17886 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:16:19.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:19.811+0000 7f62f3bd1740 20 client.17886 populate_metadata read hostname 'smithi045' 2024-04-16T17:16:19.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:19.811+0000 7f62f3bd1740 1 --2- 172.21.15.45:0/9952009 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x558936fd1b60 0x558936fd3f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:19.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:19.811+0000 7f62f3bd1740 4 client.17886 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:16:19.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:19.811+0000 7f62f3bd1740 1 -- 172.21.15.45:0/9952009 --> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558936f2bb40 con 0x558936fd1b60 2024-04-16T17:16:19.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:19.812+0000 7f62e9400640 1 --2- 172.21.15.45:0/9952009 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x558936fd1b60 0x558936fd3f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:19.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:19.812+0000 7f62e9400640 1 --2- 172.21.15.45:0/9952009 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x558936fd1b60 0x558936fd3f90 crc :-1 s=READY pgs=241 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b9f91e8f3ca90fef server_cookie=3605d4b8998aa18b in_seq=0 out_seq=0 2024-04-16T17:16:19.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:19.812+0000 7f62cbe00640 10 client.17886 ms_handle_connect on v2:172.21.15.104:6838/3660070592 2024-04-16T17:16:19.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:19.813+0000 7f62cbe00640 1 -- 172.21.15.45:0/9952009 <== mds.0 v2:172.21.15.104:6838/3660070592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558936f2bb40 con 0x558936fd1b60 2024-04-16T17:16:19.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:19.813+0000 7f62cbe00640 10 client.17886 handle_command_reply: tid=0 2024-04-16T17:16:19.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:19.813+0000 7f62f3bd1740 1 client.17886 shutdown 2024-04-16T17:16:19.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:19.813+0000 7f62f3bd1740 1 -- 172.21.15.45:0/9952009 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558936fb8390 msgr2=0x558936fbe890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:19.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:19.813+0000 7f62f3bd1740 1 --2- 172.21.15.45:0/9952009 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558936fb8390 0x558936fbe890 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f62dc01f000 tx=0x7f62dc09c860 comp rx=0 tx=0).stop 2024-04-16T17:16:19.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:19.814+0000 7f62f3bd1740 1 -- 172.21.15.45:0/9952009 shutdown_connections 2024-04-16T17:16:19.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:19.814+0000 7f62f3bd1740 1 --2- 172.21.15.45:0/9952009 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x558936fd1b60 0x558936fd3f90 crc :-1 s=READY pgs=241 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:19.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:19.814+0000 7f62f3bd1740 1 --2- 172.21.15.45:0/9952009 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558936fc1890 0x558936fbf310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:19.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:19.814+0000 7f62f3bd1740 1 --2- 172.21.15.45:0/9952009 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558936fb8cb0 0x558936fbedd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:19.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:19.814+0000 7f62f3bd1740 1 --2- 172.21.15.45:0/9952009 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558936fb8390 0x558936fbe890 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:19.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:19.814+0000 7f62cbe00640 0 client.17886 ms_handle_reset on v2:172.21.15.104:6838/3660070592 2024-04-16T17:16:19.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:19.814+0000 7f62f3bd1740 1 -- 172.21.15.45:0/9952009 >> 172.21.15.45:0/9952009 conn(0x558936fb2ac0 msgr2=0x558936fb4810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:16:19.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:19.814+0000 7f62f3bd1740 1 -- 172.21.15.45:0/9952009 shutdown_connections 2024-04-16T17:16:19.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:19.814+0000 7f62f3bd1740 1 -- 172.21.15.45:0/9952009 wait complete. 2024-04-16T17:16:19.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:19.814+0000 7f62f3bd1740 20 client.17886 trim_cache size 0 max 16384 2024-04-16T17:16:19.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:19.833+0000 7f62f3bd1740 1 Processor -- start 2024-04-16T17:16:19.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:19.833+0000 7f62f3bd1740 1 -- start start 2024-04-16T17:16:19.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:19.833+0000 7f62f3bd1740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558936fb8390 0x558936fcd510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:19.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:19.833+0000 7f62f3bd1740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558936fb8cb0 0x558936fd9860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:19.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:19.834+0000 7f62f3bd1740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558936fc1890 0x558936fd9da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:19.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:19.834+0000 7f62f3bd1740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f62ec1b3e60 con 0x558936fb8cb0 2024-04-16T17:16:19.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:19.834+0000 7f62f3bd1740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f62ec1b1730 con 0x558936fc1890 2024-04-16T17:16:19.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:19.834+0000 7f62f3bd1740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f62ec1af5a0 con 0x558936fb8390 2024-04-16T17:16:19.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:19.834+0000 7f62e9400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558936fb8cb0 0x558936fd9860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:19.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:19.834+0000 7f62e9400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558936fb8cb0 0x558936fd9860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58990/0 (socket says 172.21.15.45:58990) 2024-04-16T17:16:19.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:19.834+0000 7f62e9400640 1 -- 172.21.15.45:0/309374414 learned_addr learned my addr 172.21.15.45:0/309374414 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:16:19.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:19.834+0000 7f62e9e00640 1 --2- 172.21.15.45:0/309374414 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558936fb8390 0x558936fcd510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:19.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:19.834+0000 7f62ea800640 1 --2- 172.21.15.45:0/309374414 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558936fc1890 0x558936fd9da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:19.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:19.834+0000 7f62ea800640 1 -- 172.21.15.45:0/309374414 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558936fb8390 msgr2=0x558936fcd510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:16:19.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:19.834+0000 7f62ea800640 1 --2- 172.21.15.45:0/309374414 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558936fb8390 0x558936fcd510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:19.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:19.834+0000 7f62ea800640 1 -- 172.21.15.45:0/309374414 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558936fb8cb0 msgr2=0x558936fd9860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:16:19.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:19.834+0000 7f62ea800640 1 --2- 172.21.15.45:0/309374414 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558936fb8cb0 0x558936fd9860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:19.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:19.834+0000 7f62ea800640 1 -- 172.21.15.45:0/309374414 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558936fb8ae0 con 0x558936fc1890 2024-04-16T17:16:19.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:19.835+0000 7f62ea800640 1 --2- 172.21.15.45:0/309374414 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558936fc1890 0x558936fd9da0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f62e4009fd0 tx=0x7f62e4017000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f7fce9ae1adaa4 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:16:19.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:19.835+0000 7f62d6a00640 1 -- 172.21.15.45:0/309374414 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62e4029460 con 0x558936fc1890 2024-04-16T17:16:19.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:19.835+0000 7f62d6a00640 1 -- 172.21.15.45:0/309374414 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f62e402e070 con 0x558936fc1890 2024-04-16T17:16:19.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:19.835+0000 7f62d6a00640 1 -- 172.21.15.45:0/309374414 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62e401da40 con 0x558936fc1890 2024-04-16T17:16:19.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:19.836+0000 7f62f3bd1740 1 -- 172.21.15.45:0/309374414 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558936fc1890 msgr2=0x558936fd9da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:19.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:19.836+0000 7f62f3bd1740 1 --2- 172.21.15.45:0/309374414 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558936fc1890 0x558936fd9da0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f62e4009fd0 tx=0x7f62e4017000 comp rx=0 tx=0).stop 2024-04-16T17:16:19.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:19.836+0000 7f62f3bd1740 1 -- 172.21.15.45:0/309374414 shutdown_connections 2024-04-16T17:16:19.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:19.836+0000 7f62f3bd1740 1 --2- 172.21.15.45:0/309374414 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558936fc1890 0x558936fd9da0 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:19.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:19.836+0000 7f62f3bd1740 1 --2- 172.21.15.45:0/309374414 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558936fb8cb0 0x558936fd9860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:19.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:19.836+0000 7f62f3bd1740 1 --2- 172.21.15.45:0/309374414 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558936fb8390 0x558936fcd510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:19.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:19.836+0000 7f62f3bd1740 1 -- 172.21.15.45:0/309374414 >> 172.21.15.45:0/309374414 conn(0x558936fb2ac0 msgr2=0x558936fbd160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:16:19.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:19.836+0000 7f62f3bd1740 1 -- 172.21.15.45:0/309374414 shutdown_connections 2024-04-16T17:16:19.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:19.836+0000 7f62f3bd1740 1 -- 172.21.15.45:0/309374414 wait complete. 2024-04-16T17:16:19.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:19.836+0000 7f62f3bd1740 1 -- start start 2024-04-16T17:16:19.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:19.837+0000 7f62f3bd1740 1 Processor -- start 2024-04-16T17:16:19.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:19.837+0000 7f62f3bd1740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558936fb8390 0x558936fc9ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:19.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:19.837+0000 7f62f3bd1740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558936fb8cb0 0x558936fc4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:19.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:19.837+0000 7f62f3bd1740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558936fc1890 0x558936fc54e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:19.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:19.837+0000 7f62f3bd1740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f62ec1afbf0 con 0x558936fb8390 2024-04-16T17:16:19.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:19.837+0000 7f62f3bd1740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f62ec1b1060 con 0x558936fc1890 2024-04-16T17:16:19.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:19.837+0000 7f62f3bd1740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f62ec1b02c0 con 0x558936fb8cb0 2024-04-16T17:16:19.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:19.837+0000 7f62f3bd1740 10 client.0 _ll_register_callbacks cb 0x7f62dc00ef90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:16:19.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:19.837+0000 7f62e9e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558936fb8390 0x558936fc9ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:19.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:19.837+0000 7f62e9e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558936fb8390 0x558936fc9ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59000/0 (socket says 172.21.15.45:59000) 2024-04-16T17:16:19.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:19.837+0000 7f62e9e00640 1 -- 172.21.15.45:0/4180077802 learned_addr learned my addr 172.21.15.45:0/4180077802 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:16:19.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:19.837+0000 7f62ea800640 1 --2- 172.21.15.45:0/4180077802 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558936fc1890 0x558936fc54e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:19.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:19.837+0000 7f62e9400640 1 --2- 172.21.15.45:0/4180077802 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558936fb8cb0 0x558936fc4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:19.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:19.838+0000 7f62e9e00640 1 -- 172.21.15.45:0/4180077802 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558936fb8cb0 msgr2=0x558936fc4fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:16:19.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:19.838+0000 7f62e9e00640 1 --2- 172.21.15.45:0/4180077802 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558936fb8cb0 0x558936fc4fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:19.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:19.838+0000 7f62e9e00640 1 -- 172.21.15.45:0/4180077802 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558936fc1890 msgr2=0x558936fc54e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:16:19.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:19.838+0000 7f62e9e00640 1 --2- 172.21.15.45:0/4180077802 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558936fc1890 0x558936fc54e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:19.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:19.838+0000 7f62e9e00640 1 -- 172.21.15.45:0/4180077802 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558936fc5a20 con 0x558936fb8390 2024-04-16T17:16:19.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:19.838+0000 7f62ea800640 1 --2- 172.21.15.45:0/4180077802 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558936fc1890 0x558936fc54e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:16:19.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:19.838+0000 7f62e9e00640 1 --2- 172.21.15.45:0/4180077802 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558936fb8390 0x558936fc9ac0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f62dc09c320 tx=0x7f62dc09c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec32848ba071773d server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:16:19.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:19.838+0000 7f62cbe00640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-16T17:16:19.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:19.838+0000 7f62cbe00640 1 -- 172.21.15.45:0/4180077802 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62dc09c5d0 con 0x558936fb8390 2024-04-16T17:16:19.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:19.838+0000 7f62cbe00640 1 -- 172.21.15.45:0/4180077802 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9496+0+0 (secure 0 0 0) 0x7f62dc0ce4c0 con 0x558936fb8390 2024-04-16T17:16:19.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:19.838+0000 7f62f3bd1740 1 -- 172.21.15.45:0/4180077802 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f62e402e070 con 0x558936fb8390 2024-04-16T17:16:19.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:19.839+0000 7f62cbe00640 1 -- 172.21.15.45:0/4180077802 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f62dc0ce6c0 con 0x558936fb8390 2024-04-16T17:16:19.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:19.839+0000 7f62f3bd1740 10 client.17892 fetch_fsmap learned FSMap version 44 2024-04-16T17:16:19.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:19.839+0000 7f62f3bd1740 1 -- 172.21.15.45:0/4180077802 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x558937037df0 con 0x558936fb8390 2024-04-16T17:16:19.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:19.839+0000 7f62cbe00640 1 -- 172.21.15.45:0/4180077802 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f62e402e070 con 0x558936fb8390 2024-04-16T17:16:19.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:19.839+0000 7f62cbe00640 1 -- 172.21.15.45:0/4180077802 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 44) ==== 8346+0+0 (secure 0 0 0) 0x7f62dc0d4020 con 0x558936fb8390 2024-04-16T17:16:19.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:19.839+0000 7f62f3bd1740 10 client.17892 fetch_fsmap finished waiting for FSMap version 44 2024-04-16T17:16:19.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:19.839+0000 7f62f3bd1740 10 client.17892 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:16:19.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:19.839+0000 7f62f3bd1740 20 client.17892 populate_metadata read hostname 'smithi045' 2024-04-16T17:16:19.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:19.839+0000 7f62f3bd1740 1 --2- 172.21.15.45:0/4180077802 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x5589370384f0 0x55893703a910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:19.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:19.839+0000 7f62f3bd1740 4 client.17892 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-16T17:16:19.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:19.839+0000 7f62f3bd1740 1 -- 172.21.15.45:0/4180077802 --> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] -- command(tid 0: {"prefix": "scrub status"}) -- 0x558936f259b0 con 0x5589370384f0 2024-04-16T17:16:19.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:19.840+0000 7f62e9400640 1 --2- 172.21.15.45:0/4180077802 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x5589370384f0 0x55893703a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:19.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:19.840+0000 7f62e9400640 1 --2- 172.21.15.45:0/4180077802 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x5589370384f0 0x55893703a910 crc :-1 s=READY pgs=242 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fa274939e9d7ab7f server_cookie=bf4a580cd68eec2a in_seq=0 out_seq=0 2024-04-16T17:16:19.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:19.840+0000 7f62cbe00640 10 client.17892 ms_handle_connect on v2:172.21.15.104:6838/3660070592 2024-04-16T17:16:19.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:19.840+0000 7f62cbe00640 1 -- 172.21.15.45:0/4180077802 <== mds.0 v2:172.21.15.104:6838/3660070592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x558936f259b0 con 0x5589370384f0 2024-04-16T17:16:19.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:19.840+0000 7f62cbe00640 10 client.17892 handle_command_reply: tid=0 2024-04-16T17:16:19.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:19.840+0000 7f62f3bd1740 1 client.17892 shutdown 2024-04-16T17:16:19.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:19.840+0000 7f62f3bd1740 1 -- 172.21.15.45:0/4180077802 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558936fb8390 msgr2=0x558936fc9ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:19.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:19.840+0000 7f62f3bd1740 1 --2- 172.21.15.45:0/4180077802 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558936fb8390 0x558936fc9ac0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f62dc09c320 tx=0x7f62dc09c990 comp rx=0 tx=0).stop 2024-04-16T17:16:19.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:19.841+0000 7f62f3bd1740 1 -- 172.21.15.45:0/4180077802 shutdown_connections 2024-04-16T17:16:19.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:19.841+0000 7f62f3bd1740 1 --2- 172.21.15.45:0/4180077802 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x5589370384f0 0x55893703a910 crc :-1 s=READY pgs=242 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:19.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:19.841+0000 7f62f3bd1740 1 --2- 172.21.15.45:0/4180077802 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558936fb8cb0 0x558936fc4fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:19.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:19.841+0000 7f62f3bd1740 1 --2- 172.21.15.45:0/4180077802 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558936fc1890 0x558936fc54e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:19.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:19.841+0000 7f62cbe00640 0 client.17892 ms_handle_reset on v2:172.21.15.104:6838/3660070592 2024-04-16T17:16:19.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:19.841+0000 7f62f3bd1740 1 --2- 172.21.15.45:0/4180077802 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558936fb8390 0x558936fc9ac0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:19.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:19.841+0000 7f62f3bd1740 1 -- 172.21.15.45:0/4180077802 >> 172.21.15.45:0/4180077802 conn(0x558936fb2ac0 msgr2=0x558936fb1290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:16:19.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:19.841+0000 7f62f3bd1740 1 -- 172.21.15.45:0/4180077802 shutdown_connections 2024-04-16T17:16:19.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:19.841+0000 7f62f3bd1740 1 -- 172.21.15.45:0/4180077802 wait complete. 2024-04-16T17:16:19.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:19.841+0000 7f62f3bd1740 20 client.17892 trim_cache size 0 max 16384 2024-04-16T17:16:19.847 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-16T17:16:19.847 INFO:teuthology.orchestra.run.smithi045.stdout: "status": "no active scrubs running", 2024-04-16T17:16:19.847 INFO:teuthology.orchestra.run.smithi045.stdout: "scrubs": {} 2024-04-16T17:16:19.847 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-16T17:16:19.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:19.842+0000 7f62eb200640 1 -- 172.21.15.45:0/2031778513 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f62d00a3d20 msgr2=0x7f62d00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:19.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:19.842+0000 7f62eb200640 1 --2- 172.21.15.45:0/2031778513 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f62d00a3d20 0x7f62d00a61e0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f62e4005fd0 tx=0x7f62e4009500 comp rx=0 tx=0).stop 2024-04-16T17:16:19.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:19.842+0000 7f62eb200640 1 -- 172.21.15.45:0/2031778513 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f62ec0743b0 msgr2=0x7f62ec1a0670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:19.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:19.842+0000 7f62eb200640 1 --2- 172.21.15.45:0/2031778513 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f62ec0743b0 0x7f62ec1a0670 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f62dc00ea60 tx=0x7f62dc00ee60 comp rx=0 tx=0).stop 2024-04-16T17:16:19.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:19.842+0000 7f62eb200640 1 -- 172.21.15.45:0/2031778513 shutdown_connections 2024-04-16T17:16:19.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:19.842+0000 7f62eb200640 1 --2- 172.21.15.45:0/2031778513 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f62d00a3d20 0x7f62d00a61e0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:19.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:19.842+0000 7f62eb200640 1 --2- 172.21.15.45:0/2031778513 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f62ec074cd0 0x7f62ec1a0bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:19.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:19.842+0000 7f62eb200640 1 --2- 172.21.15.45:0/2031778513 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f62ec10e770 0x7f62ec19a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:19.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:19.842+0000 7f62eb200640 1 --2- 172.21.15.45:0/2031778513 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f62ec0743b0 0x7f62ec1a0670 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:19.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:19.842+0000 7f62eb200640 1 -- 172.21.15.45:0/2031778513 >> 172.21.15.45:0/2031778513 conn(0x7f62ec0fd9a0 msgr2=0x7f62ec0ff1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:16:19.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:19.842+0000 7f62eb200640 1 -- 172.21.15.45:0/2031778513 shutdown_connections 2024-04-16T17:16:19.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:19.842+0000 7f62eb200640 1 -- 172.21.15.45:0/2031778513 wait complete. 2024-04-16T17:16:19.859 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-16T17:16:19.859 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-16T17:16:20.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.081+0000 7ff4cac00640 1 -- 172.21.15.45:0/3189856114 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff4c4104260 msgr2=0x7ff4c4104640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:20.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.081+0000 7ff4cac00640 1 --2- 172.21.15.45:0/3189856114 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff4c4104260 0x7ff4c4104640 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7ff4b8009900 tx=0x7ff4b802bc40 comp rx=0 tx=0).stop 2024-04-16T17:16:20.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.082+0000 7ff4cac00640 1 -- 172.21.15.45:0/3189856114 shutdown_connections 2024-04-16T17:16:20.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.082+0000 7ff4cac00640 1 --2- 172.21.15.45:0/3189856114 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff4c4109330 0x7ff4c4110f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:20.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.082+0000 7ff4cac00640 1 --2- 172.21.15.45:0/3189856114 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff4c4104c10 0x7ff4c4108c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:20.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.082+0000 7ff4cac00640 1 --2- 172.21.15.45:0/3189856114 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff4c4104260 0x7ff4c4104640 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:20.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.082+0000 7ff4cac00640 1 -- 172.21.15.45:0/3189856114 >> 172.21.15.45:0/3189856114 conn(0x7ff4c40ffaa0 msgr2=0x7ff4c4101ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:16:20.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.082+0000 7ff4cac00640 1 -- 172.21.15.45:0/3189856114 shutdown_connections 2024-04-16T17:16:20.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.082+0000 7ff4cac00640 1 -- 172.21.15.45:0/3189856114 wait complete. 2024-04-16T17:16:20.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.083+0000 7ff4cac00640 1 Processor -- start 2024-04-16T17:16:20.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.083+0000 7ff4cac00640 1 -- start start 2024-04-16T17:16:20.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.083+0000 7ff4cac00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff4c4104260 0x7ff4c41a0630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:20.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.083+0000 7ff4cac00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff4c4109330 0x7ff4c41a0b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:20.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.083+0000 7ff4cac00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff4c419a6f0 0x7ff4c419ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:20.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.083+0000 7ff4cac00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7ff4c4114410 con 0x7ff4c4104260 2024-04-16T17:16:20.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.083+0000 7ff4cac00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7ff4c4114290 con 0x7ff4c419a6f0 2024-04-16T17:16:20.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.083+0000 7ff4cac00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7ff4c4114590 con 0x7ff4c4109330 2024-04-16T17:16:20.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.084+0000 7ff4c9800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff4c4104260 0x7ff4c41a0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:20.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.084+0000 7ff4c9800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff4c4104260 0x7ff4c41a0630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59020/0 (socket says 172.21.15.45:59020) 2024-04-16T17:16:20.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.084+0000 7ff4c9800640 1 -- 172.21.15.45:0/3291154102 learned_addr learned my addr 172.21.15.45:0/3291154102 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:16:20.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.084+0000 7ff4ca200640 1 --2- 172.21.15.45:0/3291154102 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff4c419a6f0 0x7ff4c419ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:20.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.084+0000 7ff4c8e00640 1 --2- 172.21.15.45:0/3291154102 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff4c4109330 0x7ff4c41a0b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:20.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.084+0000 7ff4c9800640 1 -- 172.21.15.45:0/3291154102 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff4c4109330 msgr2=0x7ff4c41a0b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:20.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.084+0000 7ff4c9800640 1 --2- 172.21.15.45:0/3291154102 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff4c4109330 0x7ff4c41a0b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:20.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.084+0000 7ff4c9800640 1 -- 172.21.15.45:0/3291154102 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff4c419a6f0 msgr2=0x7ff4c419ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:20.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.084+0000 7ff4c9800640 1 --2- 172.21.15.45:0/3291154102 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff4c419a6f0 0x7ff4c419ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:20.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.084+0000 7ff4c9800640 1 -- 172.21.15.45:0/3291154102 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4c419b2f0 con 0x7ff4c4104260 2024-04-16T17:16:20.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.084+0000 7ff4c9800640 1 --2- 172.21.15.45:0/3291154102 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff4c4104260 0x7ff4c41a0630 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7ff4b8005ab0 tx=0x7ff4b80043c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:16:20.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.085+0000 7ff4c2000640 1 -- 172.21.15.45:0/3291154102 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4b8041070 con 0x7ff4c4104260 2024-04-16T17:16:20.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.085+0000 7ff4c2000640 1 -- 172.21.15.45:0/3291154102 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff4b802cca0 con 0x7ff4c4104260 2024-04-16T17:16:20.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.085+0000 7ff4cac00640 1 -- 172.21.15.45:0/3291154102 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4c419b580 con 0x7ff4c4104260 2024-04-16T17:16:20.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.085+0000 7ff4c2000640 1 -- 172.21.15.45:0/3291154102 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4b8034440 con 0x7ff4c4104260 2024-04-16T17:16:20.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.085+0000 7ff4cac00640 1 -- 172.21.15.45:0/3291154102 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4c41a76c0 con 0x7ff4c4104260 2024-04-16T17:16:20.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.086+0000 7ff4cac00640 1 -- 172.21.15.45:0/3291154102 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4c4105730 con 0x7ff4c4104260 2024-04-16T17:16:20.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.088+0000 7ff4c2000640 1 -- 172.21.15.45:0/3291154102 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7ff4b8044070 con 0x7ff4c4104260 2024-04-16T17:16:20.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.090+0000 7ff4c2000640 1 --2- 172.21.15.45:0/3291154102 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7ff4ac0a3d20 0x7ff4ac0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:20.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.090+0000 7ff4c2000640 1 -- 172.21.15.45:0/3291154102 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 9496+0+0 (secure 0 0 0) 0x7ff4b80f25b0 con 0x7ff4c4104260 2024-04-16T17:16:20.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.091+0000 7ff4c8e00640 1 --2- 172.21.15.45:0/3291154102 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7ff4ac0a3d20 0x7ff4ac0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:20.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.091+0000 7ff4c8e00640 1 --2- 172.21.15.45:0/3291154102 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7ff4ac0a3d20 0x7ff4ac0a61e0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7ff4c419bf40 tx=0x7ff4b000b410 comp rx=0 tx=0).ready entity=mgr.25411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:16:20.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.095+0000 7ff4c2000640 1 -- 172.21.15.45:0/3291154102 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff4b80be2e0 con 0x7ff4c4104260 2024-04-16T17:16:20.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.355+0000 7ff4cac00640 1 -- 172.21.15.45:0/3291154102 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff4c410c590 con 0x7ff4c4104260 2024-04-16T17:16:20.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.357+0000 7ff4c2000640 1 -- 172.21.15.45:0/3291154102 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) ==== 94+0+11887 (secure 0 0 0) 0x7ff4b80c25e0 con 0x7ff4c4104260 2024-04-16T17:16:20.356 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-16T17:16:20.357 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":44,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/1748842443","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":1748842443},{"type":"v1","addr":"172.21.15.45:6835","nonce":1748842443}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6839/1308627051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":1308627051},{"type":"v1","addr":"172.21.15.45:6839","nonce":1308627051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/1768242489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1768242489},{"type":"v1","addr":"172.21.15.117:6833","nonce":1768242489}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6833/3558902585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":3558902585},{"type":"v1","addr":"172.21.15.104:6833","nonce":3558902585}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24478,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/1308142903","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":1308142903},{"type":"v1","addr":"172.21.15.45:6837","nonce":1308142903}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24491,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6835/3430679937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":3430679937},{"type":"v1","addr":"172.21.15.104:6835","nonce":3430679937}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":44,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-16T17:03:24.663611+0000","modified":"2024-04-16T17:15:52.264451+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24581,"mds_1":24569,"mds_2":24532},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24499":{"gid":24499,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/2224123841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2224123841},{"type":"v1","addr":"172.21.15.117:6835","nonce":2224123841}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6837/2471356815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":2471356815},{"type":"v1","addr":"172.21.15.117:6837","nonce":2471356815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24532":{"gid":24532,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":13,"addr":"172.21.15.117:6839/2378675949","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":2378675949},{"type":"v1","addr":"172.21.15.117:6839","nonce":2378675949}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24569":{"gid":24569,"name":"a","rank":1,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.45:6841/3755310822","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6840","nonce":3755310822},{"type":"v1","addr":"172.21.15.45:6841","nonce":3755310822}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24581":{"gid":24581,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.104:6839/3660070592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6838","nonce":3660070592},{"type":"v1","addr":"172.21.15.104:6839","nonce":3660070592}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_27569":{"gid":27569,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6837/1833645246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":1833645246},{"type":"v1","addr":"172.21.15.104:6837","nonce":1833645246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24581,"qdb_cluster":[24569,24532,24581]},"id":1}]} 2024-04-16T17:16:20.357 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 44 2024-04-16T17:16:20.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.360+0000 7ff4cac00640 1 -- 172.21.15.45:0/3291154102 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7ff4ac0a3d20 msgr2=0x7ff4ac0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:20.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.360+0000 7ff4cac00640 1 --2- 172.21.15.45:0/3291154102 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7ff4ac0a3d20 0x7ff4ac0a61e0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7ff4c419bf40 tx=0x7ff4b000b410 comp rx=0 tx=0).stop 2024-04-16T17:16:20.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.360+0000 7ff4cac00640 1 -- 172.21.15.45:0/3291154102 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff4c4104260 msgr2=0x7ff4c41a0630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:20.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.360+0000 7ff4cac00640 1 --2- 172.21.15.45:0/3291154102 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff4c4104260 0x7ff4c41a0630 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7ff4b8005ab0 tx=0x7ff4b80043c0 comp rx=0 tx=0).stop 2024-04-16T17:16:20.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.360+0000 7ff4cac00640 1 -- 172.21.15.45:0/3291154102 shutdown_connections 2024-04-16T17:16:20.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.360+0000 7ff4cac00640 1 --2- 172.21.15.45:0/3291154102 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7ff4ac0a3d20 0x7ff4ac0a61e0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:20.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.360+0000 7ff4cac00640 1 --2- 172.21.15.45:0/3291154102 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff4c4109330 0x7ff4c41a0b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:20.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.360+0000 7ff4cac00640 1 --2- 172.21.15.45:0/3291154102 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff4c419a6f0 0x7ff4c419ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:20.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.360+0000 7ff4cac00640 1 --2- 172.21.15.45:0/3291154102 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff4c4104260 0x7ff4c41a0630 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:20.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.360+0000 7ff4cac00640 1 -- 172.21.15.45:0/3291154102 >> 172.21.15.45:0/3291154102 conn(0x7ff4c40ffaa0 msgr2=0x7ff4c4101d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:16:20.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.360+0000 7ff4cac00640 1 -- 172.21.15.45:0/3291154102 shutdown_connections 2024-04-16T17:16:20.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.360+0000 7ff4cac00640 1 -- 172.21.15.45:0/3291154102 wait complete. 2024-04-16T17:16:20.380 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-16T17:16:20.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.610+0000 7ff9e4a00640 1 -- 172.21.15.45:0/1362046715 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff9e0074cc0 msgr2=0x7ff9e0112980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:20.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.610+0000 7ff9e4a00640 1 --2- 172.21.15.45:0/1362046715 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff9e0074cc0 0x7ff9e0112980 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7ff9cc009900 tx=0x7ff9cc02bcd0 comp rx=0 tx=0).stop 2024-04-16T17:16:20.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.610+0000 7ff9e4a00640 1 -- 172.21.15.45:0/1362046715 shutdown_connections 2024-04-16T17:16:20.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.610+0000 7ff9e4a00640 1 --2- 172.21.15.45:0/1362046715 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff9e0112ec0 0x7ff9e01152b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:20.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.610+0000 7ff9e4a00640 1 --2- 172.21.15.45:0/1362046715 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff9e0074cc0 0x7ff9e0112980 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:20.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.610+0000 7ff9e4a00640 1 --2- 172.21.15.45:0/1362046715 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff9e00743a0 0x7ff9e0074780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:20.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.610+0000 7ff9e4a00640 1 -- 172.21.15.45:0/1362046715 >> 172.21.15.45:0/1362046715 conn(0x7ff9e00fd9a0 msgr2=0x7ff9e00ffdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:16:20.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.610+0000 7ff9e4a00640 1 -- 172.21.15.45:0/1362046715 shutdown_connections 2024-04-16T17:16:20.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.611+0000 7ff9e4a00640 1 -- 172.21.15.45:0/1362046715 wait complete. 2024-04-16T17:16:20.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.612+0000 7ff9e4a00640 1 Processor -- start 2024-04-16T17:16:20.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.612+0000 7ff9e4a00640 1 -- start start 2024-04-16T17:16:20.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.612+0000 7ff9e4a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff9e00743a0 0x7ff9e019ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:20.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.612+0000 7ff9e4a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff9e0074cc0 0x7ff9e019f340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:20.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.612+0000 7ff9e4a00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff9e0112ec0 0x7ff9e01a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:20.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.612+0000 7ff9e4a00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7ff9e01188e0 con 0x7ff9e0074cc0 2024-04-16T17:16:20.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.612+0000 7ff9e4a00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7ff9e0118760 con 0x7ff9e0112ec0 2024-04-16T17:16:20.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.612+0000 7ff9e4a00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7ff9e0118a60 con 0x7ff9e00743a0 2024-04-16T17:16:20.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.613+0000 7ff9dde00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff9e0074cc0 0x7ff9e019f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:20.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.613+0000 7ff9dde00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff9e0074cc0 0x7ff9e019f340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59044/0 (socket says 172.21.15.45:59044) 2024-04-16T17:16:20.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.613+0000 7ff9dde00640 1 -- 172.21.15.45:0/535148231 learned_addr learned my addr 172.21.15.45:0/535148231 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:16:20.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.613+0000 7ff9de800640 1 --2- 172.21.15.45:0/535148231 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff9e00743a0 0x7ff9e019ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:20.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.613+0000 7ff9df200640 1 --2- 172.21.15.45:0/535148231 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff9e0112ec0 0x7ff9e01a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:20.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.613+0000 7ff9dde00640 1 -- 172.21.15.45:0/535148231 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff9e00743a0 msgr2=0x7ff9e019ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:20.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.613+0000 7ff9dde00640 1 --2- 172.21.15.45:0/535148231 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff9e00743a0 0x7ff9e019ee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:20.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.613+0000 7ff9dde00640 1 -- 172.21.15.45:0/535148231 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff9e0112ec0 msgr2=0x7ff9e01a36c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:20.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.613+0000 7ff9dde00640 1 --2- 172.21.15.45:0/535148231 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff9e0112ec0 0x7ff9e01a36c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:20.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.613+0000 7ff9dde00640 1 -- 172.21.15.45:0/535148231 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9e01a3df0 con 0x7ff9e0074cc0 2024-04-16T17:16:20.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.614+0000 7ff9dde00640 1 --2- 172.21.15.45:0/535148231 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff9e0074cc0 0x7ff9e019f340 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7ff9cc000c00 tx=0x7ff9cc0043c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:16:20.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.614+0000 7ff9d7400640 1 -- 172.21.15.45:0/535148231 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9cc042020 con 0x7ff9e0074cc0 2024-04-16T17:16:20.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.614+0000 7ff9d7400640 1 -- 172.21.15.45:0/535148231 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff9cc0054e0 con 0x7ff9e0074cc0 2024-04-16T17:16:20.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.614+0000 7ff9e4a00640 1 -- 172.21.15.45:0/535148231 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9e01a4080 con 0x7ff9e0074cc0 2024-04-16T17:16:20.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.614+0000 7ff9d7400640 1 -- 172.21.15.45:0/535148231 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9cc003b20 con 0x7ff9e0074cc0 2024-04-16T17:16:20.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.614+0000 7ff9e4a00640 1 -- 172.21.15.45:0/535148231 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9e01ab970 con 0x7ff9e0074cc0 2024-04-16T17:16:20.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.617+0000 7ff9e8074740 1 Processor -- start 2024-04-16T17:16:20.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.617+0000 7ff9d7400640 1 -- 172.21.15.45:0/535148231 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7ff9cc045020 con 0x7ff9e0074cc0 2024-04-16T17:16:20.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.617+0000 7ff9e8074740 1 -- start start 2024-04-16T17:16:20.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.618+0000 7ff9e8074740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559a426aff10 0x559a426b02f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:20.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.618+0000 7ff9e8074740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559a426b0830 0x559a426b8ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:20.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.618+0000 7ff9e8074740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559a426b9410 0x559a426bb800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:20.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.618+0000 7ff9e8074740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x559a42615020 con 0x559a426b9410 2024-04-16T17:16:20.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.618+0000 7ff9e8074740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x559a425bb460 con 0x559a426b0830 2024-04-16T17:16:20.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.618+0000 7ff9e8074740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x559a42610fe0 con 0x559a426aff10 2024-04-16T17:16:20.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.618+0000 7ff9df200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559a426aff10 0x559a426b02f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:20.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.618+0000 7ff9dde00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559a426b9410 0x559a426bb800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:20.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.618+0000 7ff9de800640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559a426b0830 0x559a426b8ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:20.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.618+0000 7ff9df200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559a426aff10 0x559a426b02f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:41916/0 (socket says 172.21.15.45:41916) 2024-04-16T17:16:20.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.618+0000 7ff9df200640 1 -- 172.21.15.45:0/3534779333 learned_addr learned my addr 172.21.15.45:0/3534779333 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:16:20.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.619+0000 7ff9df200640 1 -- 172.21.15.45:0/3534779333 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559a426b0830 msgr2=0x559a426b8ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:16:20.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.619+0000 7ff9df200640 1 --2- 172.21.15.45:0/3534779333 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559a426b0830 0x559a426b8ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:20.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.619+0000 7ff9df200640 1 -- 172.21.15.45:0/3534779333 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559a426b9410 msgr2=0x559a426bb800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:16:20.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.619+0000 7ff9df200640 1 --2- 172.21.15.45:0/3534779333 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559a426b9410 0x559a426bb800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:20.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.619+0000 7ff9df200640 1 -- 172.21.15.45:0/3534779333 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559a425c3040 con 0x559a426aff10 2024-04-16T17:16:20.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.619+0000 7ff9dde00640 1 --2- 172.21.15.45:0/3534779333 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559a426b9410 0x559a426bb800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:16:20.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.619+0000 7ff9de800640 1 --2- 172.21.15.45:0/3534779333 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559a426b0830 0x559a426b8ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:16:20.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.619+0000 7ff9df200640 1 --2- 172.21.15.45:0/3534779333 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559a426aff10 0x559a426b02f0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7ff9d8009f40 tx=0x7ff9d8010800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c8f272f8a6edc3f1 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:16:20.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.619+0000 7ff9d7400640 1 --2- 172.21.15.45:0/535148231 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7ff9c40a3d20 0x7ff9c40a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:20.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.619+0000 7ff9d7400640 1 -- 172.21.15.45:0/535148231 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 9496+0+0 (secure 0 0 0) 0x7ff9cc0f2860 con 0x7ff9e0074cc0 2024-04-16T17:16:20.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.619+0000 7ff9cb400640 1 -- 172.21.15.45:0/3534779333 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9d8016070 con 0x559a426aff10 2024-04-16T17:16:20.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.620+0000 7ff9cb400640 1 -- 172.21.15.45:0/3534779333 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff9d8004d20 con 0x559a426aff10 2024-04-16T17:16:20.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.620+0000 7ff9cb400640 1 -- 172.21.15.45:0/3534779333 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9d80128b0 con 0x559a426aff10 2024-04-16T17:16:20.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.620+0000 7ff9de800640 1 --2- 172.21.15.45:0/535148231 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7ff9c40a3d20 0x7ff9c40a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:20.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.620+0000 7ff9e8074740 1 -- 172.21.15.45:0/3534779333 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559a426aff10 msgr2=0x559a426b02f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:20.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.620+0000 7ff9e8074740 1 --2- 172.21.15.45:0/3534779333 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559a426aff10 0x559a426b02f0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7ff9d8009f40 tx=0x7ff9d8010800 comp rx=0 tx=0).stop 2024-04-16T17:16:20.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.620+0000 7ff9de800640 1 --2- 172.21.15.45:0/535148231 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7ff9c40a3d20 0x7ff9c40a61e0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x559a426af9e0 tx=0x7ff9d0005f90 comp rx=0 tx=0).ready entity=mgr.25411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:16:20.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.620+0000 7ff9e8074740 1 -- 172.21.15.45:0/3534779333 shutdown_connections 2024-04-16T17:16:20.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.620+0000 7ff9e8074740 1 --2- 172.21.15.45:0/3534779333 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559a426b9410 0x559a426bb800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:20.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.620+0000 7ff9e8074740 1 --2- 172.21.15.45:0/3534779333 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559a426b0830 0x559a426b8ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:20.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.620+0000 7ff9e8074740 1 --2- 172.21.15.45:0/3534779333 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559a426aff10 0x559a426b02f0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:20.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.620+0000 7ff9e8074740 1 -- 172.21.15.45:0/3534779333 >> 172.21.15.45:0/3534779333 conn(0x559a426aa640 msgr2=0x559a426aaa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:16:20.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.621+0000 7ff9e8074740 1 -- 172.21.15.45:0/3534779333 shutdown_connections 2024-04-16T17:16:20.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.621+0000 7ff9e8074740 1 -- 172.21.15.45:0/3534779333 wait complete. 2024-04-16T17:16:20.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.621+0000 7ff9e8074740 1 -- start start 2024-04-16T17:16:20.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.621+0000 7ff9e8074740 1 Processor -- start 2024-04-16T17:16:20.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.622+0000 7ff9e8074740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559a426aff10 0x559a426b64b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:20.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.622+0000 7ff9e8074740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559a426b0830 0x559a426b69f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:20.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.622+0000 7ff9e8074740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559a426b9410 0x559a426b6f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:20.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.622+0000 7ff9e8074740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x559a426bc200 con 0x559a426b9410 2024-04-16T17:16:20.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.622+0000 7ff9e8074740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x559a425ce830 con 0x559a426aff10 2024-04-16T17:16:20.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.622+0000 7ff9e8074740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x559a426bc380 con 0x559a426b0830 2024-04-16T17:16:20.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.622+0000 7ff9e8074740 10 client.0 _ll_register_callbacks cb 0x559a425d6150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:16:20.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.622+0000 7ff9de800640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559a426aff10 0x559a426b64b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:20.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.622+0000 7ff9de800640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559a426aff10 0x559a426b64b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:42780/0 (socket says 172.21.15.45:42780) 2024-04-16T17:16:20.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.622+0000 7ff9de800640 1 -- 172.21.15.45:0/3195780163 learned_addr learned my addr 172.21.15.45:0/3195780163 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:16:20.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.622+0000 7ff9df200640 1 --2- 172.21.15.45:0/3195780163 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559a426b9410 0x559a426b6f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:20.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.622+0000 7ff9dde00640 1 --2- 172.21.15.45:0/3195780163 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559a426b0830 0x559a426b69f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:20.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.622+0000 7ff9de800640 1 -- 172.21.15.45:0/3195780163 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559a426b0830 msgr2=0x559a426b69f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:16:20.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.622+0000 7ff9de800640 1 --2- 172.21.15.45:0/3195780163 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559a426b0830 0x559a426b69f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:20.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.622+0000 7ff9de800640 1 -- 172.21.15.45:0/3195780163 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559a426b9410 msgr2=0x559a426b6f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:16:20.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.622+0000 7ff9de800640 1 --2- 172.21.15.45:0/3195780163 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559a426b9410 0x559a426b6f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:20.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.623+0000 7ff9de800640 1 -- 172.21.15.45:0/3195780163 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559a425d2920 con 0x559a426aff10 2024-04-16T17:16:20.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.623+0000 7ff9de800640 1 --2- 172.21.15.45:0/3195780163 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559a426aff10 0x559a426b64b0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7ff9d0004600 tx=0x7ff9d0004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=766681cd2e0f6f99 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:16:20.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.623+0000 7ff9bfe00640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-04-16T17:16:20.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.623+0000 7ff9bfe00640 1 -- 172.21.15.45:0/3195780163 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9d0009be0 con 0x559a426aff10 2024-04-16T17:16:20.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.623+0000 7ff9bfe00640 1 -- 172.21.15.45:0/3195780163 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9496+0+0 (secure 0 0 0) 0x7ff9d001f640 con 0x559a426aff10 2024-04-16T17:16:20.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.623+0000 7ff9e8074740 1 -- 172.21.15.45:0/3195780163 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff9d8004d20 con 0x559a426aff10 2024-04-16T17:16:20.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.624+0000 7ff9bfe00640 1 -- 172.21.15.45:0/3195780163 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7ff9d001f880 con 0x559a426aff10 2024-04-16T17:16:20.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.624+0000 7ff9e8074740 10 client.28061 fetch_fsmap learned FSMap version 44 2024-04-16T17:16:20.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.624+0000 7ff9e8074740 1 -- 172.21.15.45:0/3195780163 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x559a425a07f0 con 0x559a426aff10 2024-04-16T17:16:20.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.624+0000 7ff9bfe00640 1 -- 172.21.15.45:0/3195780163 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff9d8004d20 con 0x559a426aff10 2024-04-16T17:16:20.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.624+0000 7ff9bfe00640 1 -- 172.21.15.45:0/3195780163 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 44) ==== 8346+0+0 (secure 0 0 0) 0x7ff9d0027070 con 0x559a426aff10 2024-04-16T17:16:20.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.624+0000 7ff9e8074740 10 client.28061 fetch_fsmap finished waiting for FSMap version 44 2024-04-16T17:16:20.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.624+0000 7ff9e8074740 10 client.28061 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-16T17:16:20.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.624+0000 7ff9e8074740 20 client.28061 populate_metadata read hostname 'smithi045' 2024-04-16T17:16:20.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.624+0000 7ff9e8074740 1 --2- 172.21.15.45:0/3195780163 >> [v2:172.21.15.117:6838/2378675949,v1:172.21.15.117:6839/2378675949] conn(0x559a426c9900 0x559a426cbd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:20.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.624+0000 7ff9e8074740 4 client.28061 mds_command: new command op to 24532 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:16:20.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.624+0000 7ff9e8074740 1 -- 172.21.15.45:0/3195780163 --> [v2:172.21.15.117:6838/2378675949,v1:172.21.15.117:6839/2378675949] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559a42622a40 con 0x559a426c9900 2024-04-16T17:16:20.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.625+0000 7ff9dde00640 1 --2- 172.21.15.45:0/3195780163 >> [v2:172.21.15.117:6838/2378675949,v1:172.21.15.117:6839/2378675949] conn(0x559a426c9900 0x559a426cbd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:20.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.625+0000 7ff9dde00640 1 --2- 172.21.15.45:0/3195780163 >> [v2:172.21.15.117:6838/2378675949,v1:172.21.15.117:6839/2378675949] conn(0x559a426c9900 0x559a426cbd50 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ee341884839eaa46 server_cookie=73d45174210fa696 in_seq=0 out_seq=0 2024-04-16T17:16:20.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.625+0000 7ff9bfe00640 10 client.28061 ms_handle_connect on v2:172.21.15.117:6838/2378675949 2024-04-16T17:16:20.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.626+0000 7ff9bfe00640 1 -- 172.21.15.45:0/3195780163 <== mds.2 v2:172.21.15.117:6838/2378675949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559a42622a40 con 0x559a426c9900 2024-04-16T17:16:20.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.626+0000 7ff9bfe00640 10 client.28061 handle_command_reply: tid=0 2024-04-16T17:16:20.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.626+0000 7ff9e8074740 1 client.28061 shutdown 2024-04-16T17:16:20.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.626+0000 7ff9e8074740 1 -- 172.21.15.45:0/3195780163 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559a426aff10 msgr2=0x559a426b64b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:20.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.626+0000 7ff9e8074740 1 --2- 172.21.15.45:0/3195780163 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559a426aff10 0x559a426b64b0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7ff9d0004600 tx=0x7ff9d0004730 comp rx=0 tx=0).stop 2024-04-16T17:16:20.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.626+0000 7ff9e8074740 1 -- 172.21.15.45:0/3195780163 shutdown_connections 2024-04-16T17:16:20.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.626+0000 7ff9e8074740 1 --2- 172.21.15.45:0/3195780163 >> [v2:172.21.15.117:6838/2378675949,v1:172.21.15.117:6839/2378675949] conn(0x559a426c9900 0x559a426cbd50 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:20.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.626+0000 7ff9e8074740 1 --2- 172.21.15.45:0/3195780163 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559a426b0830 0x559a426b69f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:20.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.626+0000 7ff9e8074740 1 --2- 172.21.15.45:0/3195780163 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559a426b9410 0x559a426b6f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:20.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.626+0000 7ff9e8074740 1 --2- 172.21.15.45:0/3195780163 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559a426aff10 0x559a426b64b0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:20.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.626+0000 7ff9e8074740 1 -- 172.21.15.45:0/3195780163 >> 172.21.15.45:0/3195780163 conn(0x559a426aa640 msgr2=0x559a426ac390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:16:20.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.626+0000 7ff9bfe00640 0 client.28061 ms_handle_reset on v2:172.21.15.117:6838/2378675949 2024-04-16T17:16:20.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.627+0000 7ff9e8074740 1 -- 172.21.15.45:0/3195780163 shutdown_connections 2024-04-16T17:16:20.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.627+0000 7ff9e8074740 1 -- 172.21.15.45:0/3195780163 wait complete. 2024-04-16T17:16:20.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.627+0000 7ff9e8074740 20 client.28061 trim_cache size 0 max 16384 2024-04-16T17:16:20.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.642+0000 7ff9e8074740 1 Processor -- start 2024-04-16T17:16:20.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.642+0000 7ff9e8074740 1 -- start start 2024-04-16T17:16:20.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.642+0000 7ff9e8074740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559a426aff10 0x559a426b8be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:20.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.642+0000 7ff9e8074740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559a426b0830 0x559a426c5120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:20.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.642+0000 7ff9e8074740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559a426b9410 0x559a426d1620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:20.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.642+0000 7ff9e8074740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7ff9e01b8340 con 0x559a426b0830 2024-04-16T17:16:20.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.642+0000 7ff9e8074740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7ff9e01b5c10 con 0x559a426aff10 2024-04-16T17:16:20.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.642+0000 7ff9e8074740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7ff9e01b3a80 con 0x559a426b9410 2024-04-16T17:16:20.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.643+0000 7ff9dde00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559a426b0830 0x559a426c5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:20.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.643+0000 7ff9dde00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559a426b0830 0x559a426c5120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59082/0 (socket says 172.21.15.45:59082) 2024-04-16T17:16:20.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.643+0000 7ff9dde00640 1 -- 172.21.15.45:0/417640198 learned_addr learned my addr 172.21.15.45:0/417640198 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:16:20.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.643+0000 7ff9de800640 1 --2- 172.21.15.45:0/417640198 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559a426aff10 0x559a426b8be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:20.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.643+0000 7ff9df200640 1 --2- 172.21.15.45:0/417640198 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559a426b9410 0x559a426d1620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:20.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.643+0000 7ff9dde00640 1 -- 172.21.15.45:0/417640198 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559a426b9410 msgr2=0x559a426d1620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:16:20.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.643+0000 7ff9dde00640 1 --2- 172.21.15.45:0/417640198 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559a426b9410 0x559a426d1620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:20.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.643+0000 7ff9dde00640 1 -- 172.21.15.45:0/417640198 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559a426aff10 msgr2=0x559a426b8be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:16:20.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.643+0000 7ff9dde00640 1 --2- 172.21.15.45:0/417640198 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559a426aff10 0x559a426b8be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:20.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.643+0000 7ff9dde00640 1 -- 172.21.15.45:0/417640198 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559a426b0660 con 0x559a426b0830 2024-04-16T17:16:20.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.643+0000 7ff9df200640 1 --2- 172.21.15.45:0/417640198 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559a426b9410 0x559a426d1620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:16:20.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.643+0000 7ff9dde00640 1 --2- 172.21.15.45:0/417640198 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559a426b0830 0x559a426c5120 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7ff9cc03c040 tx=0x7ff9cc0f8530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54fcd6c1f2fcb7da server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:16:20.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.644+0000 7ff9caa00640 1 -- 172.21.15.45:0/417640198 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9cc0ba780 con 0x559a426b0830 2024-04-16T17:16:20.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.644+0000 7ff9caa00640 1 -- 172.21.15.45:0/417640198 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff9cc004c80 con 0x559a426b0830 2024-04-16T17:16:20.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.644+0000 7ff9caa00640 1 -- 172.21.15.45:0/417640198 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9cc0bde40 con 0x559a426b0830 2024-04-16T17:16:20.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.644+0000 7ff9e8074740 1 -- 172.21.15.45:0/417640198 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559a426b0830 msgr2=0x559a426c5120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:20.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.644+0000 7ff9e8074740 1 --2- 172.21.15.45:0/417640198 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559a426b0830 0x559a426c5120 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7ff9cc03c040 tx=0x7ff9cc0f8530 comp rx=0 tx=0).stop 2024-04-16T17:16:20.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.645+0000 7ff9e8074740 1 -- 172.21.15.45:0/417640198 shutdown_connections 2024-04-16T17:16:20.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.645+0000 7ff9e8074740 1 --2- 172.21.15.45:0/417640198 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559a426b9410 0x559a426d1620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:20.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.645+0000 7ff9e8074740 1 --2- 172.21.15.45:0/417640198 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559a426b0830 0x559a426c5120 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:20.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.645+0000 7ff9e8074740 1 --2- 172.21.15.45:0/417640198 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559a426aff10 0x559a426b8be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:20.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.645+0000 7ff9e8074740 1 -- 172.21.15.45:0/417640198 >> 172.21.15.45:0/417640198 conn(0x559a426aa640 msgr2=0x559a426b6d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:16:20.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.645+0000 7ff9e8074740 1 -- 172.21.15.45:0/417640198 shutdown_connections 2024-04-16T17:16:20.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.645+0000 7ff9e8074740 1 -- 172.21.15.45:0/417640198 wait complete. 2024-04-16T17:16:20.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.645+0000 7ff9e8074740 1 -- start start 2024-04-16T17:16:20.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.645+0000 7ff9e8074740 1 Processor -- start 2024-04-16T17:16:20.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.646+0000 7ff9e8074740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559a426aff10 0x559a426bd2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:20.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.646+0000 7ff9e8074740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559a426b9410 0x559a426bd7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:20.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.646+0000 7ff9e8074740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559a426bdd20 0x559a4272ee90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:20.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.646+0000 7ff9e8074740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7ff9e01b40d0 con 0x559a426bdd20 2024-04-16T17:16:20.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.646+0000 7ff9e8074740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7ff9e01b5540 con 0x559a426aff10 2024-04-16T17:16:20.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.646+0000 7ff9e8074740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7ff9e01b47a0 con 0x559a426b9410 2024-04-16T17:16:20.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.646+0000 7ff9e8074740 10 client.0 _ll_register_callbacks cb 0x7ff9d0009850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:16:20.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.646+0000 7ff9de800640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559a426aff10 0x559a426bd2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:20.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.646+0000 7ff9de800640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559a426aff10 0x559a426bd2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:42802/0 (socket says 172.21.15.45:42802) 2024-04-16T17:16:20.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.646+0000 7ff9de800640 1 -- 172.21.15.45:0/3994699333 learned_addr learned my addr 172.21.15.45:0/3994699333 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:16:20.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.646+0000 7ff9df200640 1 --2- 172.21.15.45:0/3994699333 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559a426bdd20 0x559a4272ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:20.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.646+0000 7ff9dde00640 1 --2- 172.21.15.45:0/3994699333 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559a426b9410 0x559a426bd7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:20.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.647+0000 7ff9de800640 1 -- 172.21.15.45:0/3994699333 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559a426b9410 msgr2=0x559a426bd7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:16:20.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.647+0000 7ff9de800640 1 --2- 172.21.15.45:0/3994699333 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559a426b9410 0x559a426bd7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:20.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.647+0000 7ff9de800640 1 -- 172.21.15.45:0/3994699333 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559a426bdd20 msgr2=0x559a4272ee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:16:20.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.647+0000 7ff9de800640 1 --2- 172.21.15.45:0/3994699333 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559a426bdd20 0x559a4272ee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:20.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.647+0000 7ff9de800640 1 -- 172.21.15.45:0/3994699333 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559a423273f0 con 0x559a426aff10 2024-04-16T17:16:20.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.647+0000 7ff9dde00640 1 --2- 172.21.15.45:0/3994699333 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559a426b9410 0x559a426bd7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:16:20.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.647+0000 7ff9de800640 1 --2- 172.21.15.45:0/3994699333 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559a426aff10 0x559a426bd2a0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7ff9d002aff0 tx=0x7ff9d0029c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=602298e07e3c7b9a server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:16:20.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.647+0000 7ff9bfe00640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-04-16T17:16:20.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.647+0000 7ff9bfe00640 1 -- 172.21.15.45:0/3994699333 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9d003c030 con 0x559a426aff10 2024-04-16T17:16:20.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.647+0000 7ff9bfe00640 1 -- 172.21.15.45:0/3994699333 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9496+0+0 (secure 0 0 0) 0x7ff9d0009be0 con 0x559a426aff10 2024-04-16T17:16:20.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.647+0000 7ff9e8074740 1 -- 172.21.15.45:0/3994699333 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff9cc004c80 con 0x559a426aff10 2024-04-16T17:16:20.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.647+0000 7ff9bfe00640 1 -- 172.21.15.45:0/3994699333 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7ff9d00043e0 con 0x559a426aff10 2024-04-16T17:16:20.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.647+0000 7ff9e8074740 10 client.28067 fetch_fsmap learned FSMap version 44 2024-04-16T17:16:20.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.647+0000 7ff9e8074740 1 -- 172.21.15.45:0/3994699333 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x559a423f17d0 con 0x559a426aff10 2024-04-16T17:16:20.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.648+0000 7ff9bfe00640 1 -- 172.21.15.45:0/3994699333 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff9cc004c80 con 0x559a426aff10 2024-04-16T17:16:20.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.648+0000 7ff9bfe00640 1 -- 172.21.15.45:0/3994699333 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 44) ==== 8346+0+0 (secure 0 0 0) 0x7ff9d0027040 con 0x559a426aff10 2024-04-16T17:16:20.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.648+0000 7ff9e8074740 10 client.28067 fetch_fsmap finished waiting for FSMap version 44 2024-04-16T17:16:20.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.648+0000 7ff9e8074740 10 client.28067 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-16T17:16:20.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.648+0000 7ff9e8074740 20 client.28067 populate_metadata read hostname 'smithi045' 2024-04-16T17:16:20.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.648+0000 7ff9e8074740 1 --2- 172.21.15.45:0/3994699333 >> [v2:172.21.15.117:6838/2378675949,v1:172.21.15.117:6839/2378675949] conn(0x559a42730ce0 0x559a42733100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:20.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.648+0000 7ff9e8074740 4 client.28067 mds_command: new command op to 24532 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T17:16:20.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.648+0000 7ff9e8074740 1 -- 172.21.15.45:0/3994699333 --> [v2:172.21.15.117:6838/2378675949,v1:172.21.15.117:6839/2378675949] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559a4261c8b0 con 0x559a42730ce0 2024-04-16T17:16:20.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.648+0000 7ff9dde00640 1 --2- 172.21.15.45:0/3994699333 >> [v2:172.21.15.117:6838/2378675949,v1:172.21.15.117:6839/2378675949] conn(0x559a42730ce0 0x559a42733100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:20.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.649+0000 7ff9dde00640 1 --2- 172.21.15.45:0/3994699333 >> [v2:172.21.15.117:6838/2378675949,v1:172.21.15.117:6839/2378675949] conn(0x559a42730ce0 0x559a42733100 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=256fedbd99035afa server_cookie=a7715c4568fef589 in_seq=0 out_seq=0 2024-04-16T17:16:20.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.649+0000 7ff9bfe00640 10 client.28067 ms_handle_connect on v2:172.21.15.117:6838/2378675949 2024-04-16T17:16:20.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.649+0000 7ff9bfe00640 1 -- 172.21.15.45:0/3994699333 <== mds.2 v2:172.21.15.117:6838/2378675949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559a4261c8b0 con 0x559a42730ce0 2024-04-16T17:16:20.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.649+0000 7ff9bfe00640 10 client.28067 handle_command_reply: tid=0 2024-04-16T17:16:20.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.649+0000 7ff9e8074740 1 client.28067 shutdown 2024-04-16T17:16:20.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.649+0000 7ff9e8074740 1 -- 172.21.15.45:0/3994699333 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559a426aff10 msgr2=0x559a426bd2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:20.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.649+0000 7ff9e8074740 1 --2- 172.21.15.45:0/3994699333 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559a426aff10 0x559a426bd2a0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7ff9d002aff0 tx=0x7ff9d0029c00 comp rx=0 tx=0).stop 2024-04-16T17:16:20.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.649+0000 7ff9e8074740 1 -- 172.21.15.45:0/3994699333 shutdown_connections 2024-04-16T17:16:20.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.649+0000 7ff9e8074740 1 --2- 172.21.15.45:0/3994699333 >> [v2:172.21.15.117:6838/2378675949,v1:172.21.15.117:6839/2378675949] conn(0x559a42730ce0 0x559a42733100 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:20.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.649+0000 7ff9e8074740 1 --2- 172.21.15.45:0/3994699333 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559a426b9410 0x559a426bd7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:20.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.649+0000 7ff9e8074740 1 --2- 172.21.15.45:0/3994699333 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559a426bdd20 0x559a4272ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:20.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.649+0000 7ff9e8074740 1 --2- 172.21.15.45:0/3994699333 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559a426aff10 0x559a426bd2a0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:20.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.649+0000 7ff9e8074740 1 -- 172.21.15.45:0/3994699333 >> 172.21.15.45:0/3994699333 conn(0x559a426aa640 msgr2=0x559a426ab640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:16:20.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.649+0000 7ff9bfe00640 0 client.28067 ms_handle_reset on v2:172.21.15.117:6838/2378675949 2024-04-16T17:16:20.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.650+0000 7ff9e8074740 1 -- 172.21.15.45:0/3994699333 shutdown_connections 2024-04-16T17:16:20.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.650+0000 7ff9e8074740 1 -- 172.21.15.45:0/3994699333 wait complete. 2024-04-16T17:16:20.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.650+0000 7ff9e8074740 20 client.28067 trim_cache size 0 max 16384 2024-04-16T17:16:20.657 INFO:teuthology.orchestra.run.smithi045.stdout:[] 2024-04-16T17:16:20.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.650+0000 7ff9e4a00640 1 -- 172.21.15.45:0/535148231 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7ff9c40a3d20 msgr2=0x7ff9c40a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:20.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.650+0000 7ff9e4a00640 1 --2- 172.21.15.45:0/535148231 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7ff9c40a3d20 0x7ff9c40a61e0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x559a426af9e0 tx=0x7ff9d0005f90 comp rx=0 tx=0).stop 2024-04-16T17:16:20.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.650+0000 7ff9e4a00640 1 -- 172.21.15.45:0/535148231 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff9e0074cc0 msgr2=0x7ff9e019f340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:20.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.650+0000 7ff9e4a00640 1 --2- 172.21.15.45:0/535148231 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff9e0074cc0 0x7ff9e019f340 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7ff9cc000c00 tx=0x7ff9cc0043c0 comp rx=0 tx=0).stop 2024-04-16T17:16:20.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.650+0000 7ff9e4a00640 1 -- 172.21.15.45:0/535148231 shutdown_connections 2024-04-16T17:16:20.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.650+0000 7ff9e4a00640 1 --2- 172.21.15.45:0/535148231 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7ff9c40a3d20 0x7ff9c40a61e0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:20.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.650+0000 7ff9e4a00640 1 --2- 172.21.15.45:0/535148231 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff9e0112ec0 0x7ff9e01a36c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:20.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.650+0000 7ff9e4a00640 1 --2- 172.21.15.45:0/535148231 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff9e0074cc0 0x7ff9e019f340 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:20.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.650+0000 7ff9e4a00640 1 --2- 172.21.15.45:0/535148231 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff9e00743a0 0x7ff9e019ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:20.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.650+0000 7ff9e4a00640 1 -- 172.21.15.45:0/535148231 >> 172.21.15.45:0/535148231 conn(0x7ff9e00fd9a0 msgr2=0x7ff9e00ff130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:16:20.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.651+0000 7ff9e4a00640 1 -- 172.21.15.45:0/535148231 shutdown_connections 2024-04-16T17:16:20.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.651+0000 7ff9e4a00640 1 -- 172.21.15.45:0/535148231 wait complete. 2024-04-16T17:16:20.665 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:16:20 smithi045 ceph-mon[31676]: pgmap v299: 129 pgs: 129 active+clean; 56 MiB data, 870 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-16T17:16:20.665 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:16:20 smithi045 ceph-mon[31676]: from='client.? 172.21.15.45:0/3291154102' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:16:20.669 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-16T17:16:20.804 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:16:20 smithi117 ceph-mon[36730]: pgmap v299: 129 pgs: 129 active+clean; 56 MiB data, 870 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-16T17:16:20.804 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:16:20 smithi117 ceph-mon[36730]: from='client.? 172.21.15.45:0/3291154102' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:16:20.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.900+0000 7fce8e400640 1 -- 172.21.15.45:0/2033549358 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fce88104cd0 msgr2=0x7fce88108d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:20.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.900+0000 7fce8e400640 1 --2- 172.21.15.45:0/2033549358 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fce88104cd0 0x7fce88108d10 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fce74009900 tx=0x7fce7402bc30 comp rx=0 tx=0).stop 2024-04-16T17:16:20.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.901+0000 7fce8e400640 1 -- 172.21.15.45:0/2033549358 shutdown_connections 2024-04-16T17:16:20.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.901+0000 7fce8e400640 1 --2- 172.21.15.45:0/2033549358 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fce881093f0 0x7fce88110fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:20.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.901+0000 7fce8e400640 1 --2- 172.21.15.45:0/2033549358 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fce88104cd0 0x7fce88108d10 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:20.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.901+0000 7fce8e400640 1 --2- 172.21.15.45:0/2033549358 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fce88104320 0x7fce88104700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:20.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.901+0000 7fce8e400640 1 -- 172.21.15.45:0/2033549358 >> 172.21.15.45:0/2033549358 conn(0x7fce880ffb40 msgr2=0x7fce88101f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:16:20.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.901+0000 7fce8e400640 1 -- 172.21.15.45:0/2033549358 shutdown_connections 2024-04-16T17:16:20.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.901+0000 7fce8e400640 1 -- 172.21.15.45:0/2033549358 wait complete. 2024-04-16T17:16:20.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.902+0000 7fce8e400640 1 Processor -- start 2024-04-16T17:16:20.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.902+0000 7fce8e400640 1 -- start start 2024-04-16T17:16:20.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.903+0000 7fce8e400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fce88104320 0x7fce881a0680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:20.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.903+0000 7fce8e400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fce881093f0 0x7fce881a0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:20.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.903+0000 7fce8e400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fce8819a840 0x7fce8819acf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:20.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.903+0000 7fce8e400640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fce88114460 con 0x7fce8819a840 2024-04-16T17:16:20.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.903+0000 7fce8e400640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fce881142e0 con 0x7fce88104320 2024-04-16T17:16:20.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.903+0000 7fce8e400640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fce881145e0 con 0x7fce881093f0 2024-04-16T17:16:20.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.903+0000 7fce8da00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fce8819a840 0x7fce8819acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:20.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.903+0000 7fce8d000640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fce88104320 0x7fce881a0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:20.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.903+0000 7fce87e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fce881093f0 0x7fce881a0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:20.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.903+0000 7fce8da00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fce8819a840 0x7fce8819acf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59106/0 (socket says 172.21.15.45:59106) 2024-04-16T17:16:20.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.903+0000 7fce8da00640 1 -- 172.21.15.45:0/3402208960 learned_addr learned my addr 172.21.15.45:0/3402208960 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:16:20.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.904+0000 7fce8da00640 1 -- 172.21.15.45:0/3402208960 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fce881093f0 msgr2=0x7fce881a0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:20.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.904+0000 7fce8da00640 1 --2- 172.21.15.45:0/3402208960 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fce881093f0 0x7fce881a0bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:20.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.904+0000 7fce8da00640 1 -- 172.21.15.45:0/3402208960 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fce88104320 msgr2=0x7fce881a0680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:20.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.904+0000 7fce8da00640 1 --2- 172.21.15.45:0/3402208960 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fce88104320 0x7fce881a0680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:20.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.904+0000 7fce8da00640 1 -- 172.21.15.45:0/3402208960 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce8819b3d0 con 0x7fce8819a840 2024-04-16T17:16:20.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.904+0000 7fce87e00640 1 --2- 172.21.15.45:0/3402208960 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fce881093f0 0x7fce881a0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:16:20.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.904+0000 7fce8da00640 1 --2- 172.21.15.45:0/3402208960 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fce8819a840 0x7fce8819acf0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fce8000efd0 tx=0x7fce8000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:16:20.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.904+0000 7fce85600640 1 -- 172.21.15.45:0/3402208960 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce80003b70 con 0x7fce8819a840 2024-04-16T17:16:20.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.904+0000 7fce8e400640 1 -- 172.21.15.45:0/3402208960 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce8819b6c0 con 0x7fce8819a840 2024-04-16T17:16:20.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.905+0000 7fce85600640 1 -- 172.21.15.45:0/3402208960 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fce80003d10 con 0x7fce8819a840 2024-04-16T17:16:20.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.905+0000 7fce85600640 1 -- 172.21.15.45:0/3402208960 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce800109f0 con 0x7fce8819a840 2024-04-16T17:16:20.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.905+0000 7fce8e400640 1 -- 172.21.15.45:0/3402208960 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce881a77d0 con 0x7fce8819a840 2024-04-16T17:16:20.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.908+0000 7fce9260e740 1 Processor -- start 2024-04-16T17:16:20.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.908+0000 7fce85600640 1 -- 172.21.15.45:0/3402208960 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fce80004020 con 0x7fce8819a840 2024-04-16T17:16:20.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.908+0000 7fce9260e740 1 -- start start 2024-04-16T17:16:20.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.909+0000 7fce9260e740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555a91be4f10 0x555a91be52f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:20.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.909+0000 7fce9260e740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555a91be5830 0x555a91beded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:20.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.909+0000 7fce9260e740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555a91bee410 0x555a91bf0800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:20.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.909+0000 7fce9260e740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x555a91b4a020 con 0x555a91be5830 2024-04-16T17:16:20.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.909+0000 7fce9260e740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x555a91af0460 con 0x555a91bee410 2024-04-16T17:16:20.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.909+0000 7fce9260e740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x555a91b45fe0 con 0x555a91be4f10 2024-04-16T17:16:20.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.909+0000 7fce8d000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555a91be5830 0x555a91beded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:20.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.909+0000 7fce87e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555a91be4f10 0x555a91be52f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:20.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.909+0000 7fce8d000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555a91be5830 0x555a91beded0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59116/0 (socket says 172.21.15.45:59116) 2024-04-16T17:16:20.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.909+0000 7fce8d000640 1 -- 172.21.15.45:0/1766631961 learned_addr learned my addr 172.21.15.45:0/1766631961 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:16:20.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.909+0000 7fce87e00640 1 --2- 172.21.15.45:0/1766631961 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555a91bee410 0x555a91bf0800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:20.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.910+0000 7fce8d000640 1 -- 172.21.15.45:0/1766631961 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555a91be4f10 msgr2=0x555a91be52f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:16:20.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.910+0000 7fce8d000640 1 --2- 172.21.15.45:0/1766631961 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555a91be4f10 0x555a91be52f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:20.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.910+0000 7fce8d000640 1 -- 172.21.15.45:0/1766631961 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555a91bee410 msgr2=0x555a91bf0800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:16:20.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.910+0000 7fce8d000640 1 --2- 172.21.15.45:0/1766631961 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555a91bee410 0x555a91bf0800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:20.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.910+0000 7fce8d000640 1 -- 172.21.15.45:0/1766631961 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555a91af8040 con 0x555a91be5830 2024-04-16T17:16:20.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.910+0000 7fce87e00640 1 --2- 172.21.15.45:0/1766631961 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555a91bee410 0x555a91bf0800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:16:20.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.910+0000 7fce85600640 1 --2- 172.21.15.45:0/3402208960 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fce700a3d20 0x7fce700a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:20.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.910+0000 7fce85600640 1 -- 172.21.15.45:0/3402208960 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 9496+0+0 (secure 0 0 0) 0x7fce800d0ff0 con 0x7fce8819a840 2024-04-16T17:16:20.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.910+0000 7fce8d000640 1 --2- 172.21.15.45:0/1766631961 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555a91be5830 0x555a91beded0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fce7c009cf0 tx=0x7fce7c011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=520da8ebb564b156 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:16:20.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.910+0000 7fce87e00640 1 --2- 172.21.15.45:0/3402208960 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fce700a3d20 0x7fce700a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:20.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.910+0000 7fce7a000640 1 -- 172.21.15.45:0/1766631961 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce7c004230 con 0x555a91be5830 2024-04-16T17:16:20.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.911+0000 7fce7a000640 1 -- 172.21.15.45:0/1766631961 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fce7c0043d0 con 0x555a91be5830 2024-04-16T17:16:20.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.911+0000 7fce7a000640 1 -- 172.21.15.45:0/1766631961 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce7c019900 con 0x555a91be5830 2024-04-16T17:16:20.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.911+0000 7fce87e00640 1 --2- 172.21.15.45:0/3402208960 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fce700a3d20 0x7fce700a61e0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fce74009230 tx=0x7fce74005fd0 comp rx=0 tx=0).ready entity=mgr.25411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:16:20.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.911+0000 7fce9260e740 1 -- 172.21.15.45:0/1766631961 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555a91be5830 msgr2=0x555a91beded0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:20.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.911+0000 7fce9260e740 1 --2- 172.21.15.45:0/1766631961 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555a91be5830 0x555a91beded0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fce7c009cf0 tx=0x7fce7c011810 comp rx=0 tx=0).stop 2024-04-16T17:16:20.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.912+0000 7fce9260e740 1 -- 172.21.15.45:0/1766631961 shutdown_connections 2024-04-16T17:16:20.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.912+0000 7fce9260e740 1 --2- 172.21.15.45:0/1766631961 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555a91bee410 0x555a91bf0800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:20.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.912+0000 7fce9260e740 1 --2- 172.21.15.45:0/1766631961 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555a91be5830 0x555a91beded0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:20.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.912+0000 7fce9260e740 1 --2- 172.21.15.45:0/1766631961 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555a91be4f10 0x555a91be52f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:20.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.912+0000 7fce9260e740 1 -- 172.21.15.45:0/1766631961 >> 172.21.15.45:0/1766631961 conn(0x555a91bdf640 msgr2=0x555a91bdfa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:16:20.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.912+0000 7fce9260e740 1 -- 172.21.15.45:0/1766631961 shutdown_connections 2024-04-16T17:16:20.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.912+0000 7fce9260e740 1 -- 172.21.15.45:0/1766631961 wait complete. 2024-04-16T17:16:20.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.912+0000 7fce9260e740 1 -- start start 2024-04-16T17:16:20.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.912+0000 7fce9260e740 1 Processor -- start 2024-04-16T17:16:20.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.913+0000 7fce9260e740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555a91be4f10 0x555a91beb470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:20.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.913+0000 7fce9260e740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555a91be5830 0x555a91beb9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:20.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.913+0000 7fce9260e740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555a91bee410 0x555a91bebef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:20.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.913+0000 7fce9260e740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x555a91bf1200 con 0x555a91bee410 2024-04-16T17:16:20.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.913+0000 7fce9260e740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x555a91b03830 con 0x555a91be5830 2024-04-16T17:16:20.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.913+0000 7fce9260e740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x555a91bf1380 con 0x555a91be4f10 2024-04-16T17:16:20.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.913+0000 7fce9260e740 10 client.0 _ll_register_callbacks cb 0x555a91b0b150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:16:20.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.913+0000 7fce8da00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555a91bee410 0x555a91bebef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:20.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.913+0000 7fce8d000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555a91be4f10 0x555a91beb470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:20.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.913+0000 7fce8da00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555a91bee410 0x555a91bebef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59132/0 (socket says 172.21.15.45:59132) 2024-04-16T17:16:20.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.913+0000 7fce8da00640 1 -- 172.21.15.45:0/1246051047 learned_addr learned my addr 172.21.15.45:0/1246051047 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:16:20.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.913+0000 7fce87e00640 1 --2- 172.21.15.45:0/1246051047 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555a91be5830 0x555a91beb9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:20.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.914+0000 7fce8da00640 1 -- 172.21.15.45:0/1246051047 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555a91be4f10 msgr2=0x555a91beb470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:16:20.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.914+0000 7fce8da00640 1 --2- 172.21.15.45:0/1246051047 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555a91be4f10 0x555a91beb470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:20.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.914+0000 7fce8da00640 1 -- 172.21.15.45:0/1246051047 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555a91be5830 msgr2=0x555a91beb9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:16:20.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.914+0000 7fce8da00640 1 --2- 172.21.15.45:0/1246051047 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555a91be5830 0x555a91beb9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:20.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.914+0000 7fce8da00640 1 -- 172.21.15.45:0/1246051047 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555a91b07920 con 0x555a91bee410 2024-04-16T17:16:20.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.914+0000 7fce8d000640 1 --2- 172.21.15.45:0/1246051047 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555a91be4f10 0x555a91beb470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:16:20.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.914+0000 7fce8da00640 1 --2- 172.21.15.45:0/1246051047 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555a91bee410 0x555a91bebef0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fce800a52a0 tx=0x7fce8009c4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a07d7d9a9b95fc8 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:16:20.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.914+0000 7fce6ea00640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-16T17:16:20.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.914+0000 7fce6ea00640 1 -- 172.21.15.45:0/1246051047 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce800b6060 con 0x555a91bee410 2024-04-16T17:16:20.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.914+0000 7fce6ea00640 1 -- 172.21.15.45:0/1246051047 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9496+0+0 (secure 0 0 0) 0x7fce80004310 con 0x555a91bee410 2024-04-16T17:16:20.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.914+0000 7fce9260e740 1 -- 172.21.15.45:0/1246051047 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fce7c0043d0 con 0x555a91bee410 2024-04-16T17:16:20.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.914+0000 7fce6ea00640 1 -- 172.21.15.45:0/1246051047 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fce8009cb60 con 0x555a91bee410 2024-04-16T17:16:20.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.914+0000 7fce9260e740 10 client.17928 fetch_fsmap learned FSMap version 44 2024-04-16T17:16:20.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.914+0000 7fce9260e740 1 -- 172.21.15.45:0/1246051047 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x555a91ad57f0 con 0x555a91bee410 2024-04-16T17:16:20.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.915+0000 7fce6ea00640 1 -- 172.21.15.45:0/1246051047 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fce7c0043d0 con 0x555a91bee410 2024-04-16T17:16:20.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.915+0000 7fce6ea00640 1 -- 172.21.15.45:0/1246051047 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 44) ==== 8346+0+0 (secure 0 0 0) 0x7fce8001f020 con 0x555a91bee410 2024-04-16T17:16:20.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.915+0000 7fce9260e740 10 client.17928 fetch_fsmap finished waiting for FSMap version 44 2024-04-16T17:16:20.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.915+0000 7fce9260e740 10 client.17928 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-04-16T17:16:20.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.915+0000 7fce9260e740 20 client.17928 populate_metadata read hostname 'smithi045' 2024-04-16T17:16:20.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.915+0000 7fce9260e740 1 --2- 172.21.15.45:0/1246051047 >> [v2:172.21.15.45:6840/3755310822,v1:172.21.15.45:6841/3755310822] conn(0x555a91bfe7c0 0x555a91c00bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:20.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.915+0000 7fce9260e740 4 client.17928 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:16:20.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.915+0000 7fce9260e740 1 -- 172.21.15.45:0/1246051047 --> [v2:172.21.15.45:6840/3755310822,v1:172.21.15.45:6841/3755310822] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555a91b57a40 con 0x555a91bfe7c0 2024-04-16T17:16:20.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.915+0000 7fce8d000640 1 --2- 172.21.15.45:0/1246051047 >> [v2:172.21.15.45:6840/3755310822,v1:172.21.15.45:6841/3755310822] conn(0x555a91bfe7c0 0x555a91c00bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:20.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.916+0000 7fce8d000640 1 --2- 172.21.15.45:0/1246051047 >> [v2:172.21.15.45:6840/3755310822,v1:172.21.15.45:6841/3755310822] conn(0x555a91bfe7c0 0x555a91c00bb0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3e0e9a61e9398948 server_cookie=8db6713552acf3a in_seq=0 out_seq=0 2024-04-16T17:16:20.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.916+0000 7fce6ea00640 10 client.17928 ms_handle_connect on v2:172.21.15.45:6840/3755310822 2024-04-16T17:16:20.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.917+0000 7fce6ea00640 1 -- 172.21.15.45:0/1246051047 <== mds.1 v2:172.21.15.45:6840/3755310822 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555a91b57a40 con 0x555a91bfe7c0 2024-04-16T17:16:20.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.917+0000 7fce6ea00640 10 client.17928 handle_command_reply: tid=0 2024-04-16T17:16:20.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.917+0000 7fce9260e740 1 client.17928 shutdown 2024-04-16T17:16:20.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.917+0000 7fce9260e740 1 -- 172.21.15.45:0/1246051047 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555a91bee410 msgr2=0x555a91bebef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:20.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.917+0000 7fce9260e740 1 --2- 172.21.15.45:0/1246051047 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555a91bee410 0x555a91bebef0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fce800a52a0 tx=0x7fce8009c4c0 comp rx=0 tx=0).stop 2024-04-16T17:16:20.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.917+0000 7fce9260e740 1 -- 172.21.15.45:0/1246051047 shutdown_connections 2024-04-16T17:16:20.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.917+0000 7fce9260e740 1 --2- 172.21.15.45:0/1246051047 >> [v2:172.21.15.45:6840/3755310822,v1:172.21.15.45:6841/3755310822] conn(0x555a91bfe7c0 0x555a91c00bb0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:20.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.917+0000 7fce9260e740 1 --2- 172.21.15.45:0/1246051047 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555a91bee410 0x555a91bebef0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:20.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.917+0000 7fce9260e740 1 --2- 172.21.15.45:0/1246051047 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555a91be5830 0x555a91beb9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:20.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.917+0000 7fce9260e740 1 --2- 172.21.15.45:0/1246051047 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555a91be4f10 0x555a91beb470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:20.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.917+0000 7fce6ea00640 0 client.17928 ms_handle_reset on v2:172.21.15.45:6840/3755310822 2024-04-16T17:16:20.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.917+0000 7fce9260e740 1 -- 172.21.15.45:0/1246051047 >> 172.21.15.45:0/1246051047 conn(0x555a91bdf640 msgr2=0x555a91be1390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:16:20.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.917+0000 7fce9260e740 1 -- 172.21.15.45:0/1246051047 shutdown_connections 2024-04-16T17:16:20.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.917+0000 7fce9260e740 1 -- 172.21.15.45:0/1246051047 wait complete. 2024-04-16T17:16:20.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.918+0000 7fce9260e740 20 client.17928 trim_cache size 0 max 16384 2024-04-16T17:16:20.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:16:20 smithi104 ceph-mon[36457]: pgmap v299: 129 pgs: 129 active+clean; 56 MiB data, 870 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-16T17:16:20.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:16:20 smithi104 ceph-mon[36457]: from='client.? 172.21.15.45:0/3291154102' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:16:20.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.933+0000 7fce9260e740 1 Processor -- start 2024-04-16T17:16:20.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.933+0000 7fce9260e740 1 -- start start 2024-04-16T17:16:20.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.933+0000 7fce9260e740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555a91be4f10 0x555a91bedba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:20.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.933+0000 7fce9260e740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555a91be5830 0x555a91bfa0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:20.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.933+0000 7fce9260e740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555a91bee410 0x555a91c06460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:20.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.933+0000 7fce9260e740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fce881b3db0 con 0x555a91be5830 2024-04-16T17:16:20.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.933+0000 7fce9260e740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fce881b1680 con 0x555a91be4f10 2024-04-16T17:16:20.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.933+0000 7fce9260e740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fce881af4f0 con 0x555a91bee410 2024-04-16T17:16:20.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.933+0000 7fce8d000640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555a91be4f10 0x555a91bedba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:20.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.933+0000 7fce8da00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555a91bee410 0x555a91c06460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:20.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.933+0000 7fce8da00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555a91bee410 0x555a91c06460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:42020/0 (socket says 172.21.15.45:42020) 2024-04-16T17:16:20.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.933+0000 7fce8d000640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555a91be4f10 0x555a91bedba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:42854/0 (socket says 172.21.15.45:42854) 2024-04-16T17:16:20.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.933+0000 7fce8da00640 1 -- 172.21.15.45:0/3558506375 learned_addr learned my addr 172.21.15.45:0/3558506375 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:16:20.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.934+0000 7fce87e00640 1 --2- 172.21.15.45:0/3558506375 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555a91be5830 0x555a91bfa0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:20.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.934+0000 7fce8da00640 1 -- 172.21.15.45:0/3558506375 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555a91be4f10 msgr2=0x555a91bedba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:16:20.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.934+0000 7fce8da00640 1 --2- 172.21.15.45:0/3558506375 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555a91be4f10 0x555a91bedba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:20.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.934+0000 7fce8da00640 1 -- 172.21.15.45:0/3558506375 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555a91be5830 msgr2=0x555a91bfa0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:16:20.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.934+0000 7fce8da00640 1 --2- 172.21.15.45:0/3558506375 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555a91be5830 0x555a91bfa0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:20.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.934+0000 7fce8da00640 1 -- 172.21.15.45:0/3558506375 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555a9185c3f0 con 0x555a91bee410 2024-04-16T17:16:20.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.934+0000 7fce8da00640 1 --2- 172.21.15.45:0/3558506375 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555a91bee410 0x555a91c06460 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fce80010df0 tx=0x7fce800bd050 comp rx=0 tx=0).ready entity=mon.1 client_cookie=447e7cd98674b03a server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:16:20.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.934+0000 7fce79600640 1 -- 172.21.15.45:0/3558506375 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce800b6040 con 0x555a91bee410 2024-04-16T17:16:20.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.934+0000 7fce79600640 1 -- 172.21.15.45:0/3558506375 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fce800a57d0 con 0x555a91bee410 2024-04-16T17:16:20.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.934+0000 7fce79600640 1 -- 172.21.15.45:0/3558506375 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce8009bfb0 con 0x555a91bee410 2024-04-16T17:16:20.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.935+0000 7fce9260e740 1 -- 172.21.15.45:0/3558506375 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555a91bee410 msgr2=0x555a91c06460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:20.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.935+0000 7fce9260e740 1 --2- 172.21.15.45:0/3558506375 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555a91bee410 0x555a91c06460 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fce80010df0 tx=0x7fce800bd050 comp rx=0 tx=0).stop 2024-04-16T17:16:20.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.935+0000 7fce9260e740 1 -- 172.21.15.45:0/3558506375 shutdown_connections 2024-04-16T17:16:20.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.935+0000 7fce9260e740 1 --2- 172.21.15.45:0/3558506375 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555a91bee410 0x555a91c06460 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:20.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.935+0000 7fce9260e740 1 --2- 172.21.15.45:0/3558506375 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555a91be5830 0x555a91bfa0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:20.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.935+0000 7fce9260e740 1 --2- 172.21.15.45:0/3558506375 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555a91be4f10 0x555a91bedba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:20.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.935+0000 7fce9260e740 1 -- 172.21.15.45:0/3558506375 >> 172.21.15.45:0/3558506375 conn(0x555a91bdf640 msgr2=0x555a91be5c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:16:20.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.935+0000 7fce9260e740 1 -- 172.21.15.45:0/3558506375 shutdown_connections 2024-04-16T17:16:20.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.935+0000 7fce9260e740 1 -- 172.21.15.45:0/3558506375 wait complete. 2024-04-16T17:16:20.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.936+0000 7fce9260e740 1 -- start start 2024-04-16T17:16:20.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.936+0000 7fce9260e740 1 Processor -- start 2024-04-16T17:16:20.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.936+0000 7fce9260e740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555a91be4f10 0x555a91be81c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:20.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.936+0000 7fce9260e740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555a91be5830 0x555a91be8700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:20.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.936+0000 7fce9260e740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555a91bee410 0x555a91be8c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:20.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.936+0000 7fce9260e740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fce881afb40 con 0x555a91bee410 2024-04-16T17:16:20.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.936+0000 7fce9260e740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fce881b0fb0 con 0x555a91be4f10 2024-04-16T17:16:20.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.936+0000 7fce9260e740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fce881b0210 con 0x555a91be5830 2024-04-16T17:16:20.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.936+0000 7fce9260e740 10 client.0 _ll_register_callbacks cb 0x7fce80009b40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:16:20.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.937+0000 7fce8d000640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555a91be4f10 0x555a91be81c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:20.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.937+0000 7fce8da00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555a91bee410 0x555a91be8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:20.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.937+0000 7fce8d000640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555a91be4f10 0x555a91be81c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:42858/0 (socket says 172.21.15.45:42858) 2024-04-16T17:16:20.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.937+0000 7fce8d000640 1 -- 172.21.15.45:0/3084817799 learned_addr learned my addr 172.21.15.45:0/3084817799 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:16:20.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.937+0000 7fce8da00640 1 --2- 172.21.15.45:0/3084817799 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555a91bee410 0x555a91be8c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59152/0 (socket says 172.21.15.45:59152) 2024-04-16T17:16:20.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.937+0000 7fce87e00640 1 --2- 172.21.15.45:0/3084817799 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555a91be5830 0x555a91be8700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:20.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.937+0000 7fce8d000640 1 -- 172.21.15.45:0/3084817799 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555a91be5830 msgr2=0x555a91be8700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:16:20.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.937+0000 7fce8d000640 1 --2- 172.21.15.45:0/3084817799 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555a91be5830 0x555a91be8700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:20.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.937+0000 7fce8d000640 1 -- 172.21.15.45:0/3084817799 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555a91bee410 msgr2=0x555a91be8c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:16:20.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.937+0000 7fce8d000640 1 --2- 172.21.15.45:0/3084817799 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555a91bee410 0x555a91be8c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:20.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.937+0000 7fce8d000640 1 -- 172.21.15.45:0/3084817799 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555a919267d0 con 0x555a91be4f10 2024-04-16T17:16:20.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.937+0000 7fce8da00640 1 --2- 172.21.15.45:0/3084817799 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555a91bee410 0x555a91be8c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:16:20.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.937+0000 7fce8d000640 1 --2- 172.21.15.45:0/3084817799 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555a91be4f10 0x555a91be81c0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fce7c005fd0 tx=0x7fce7c0194c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a44b4b418548fb26 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:16:20.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.937+0000 7fce6ea00640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-04-16T17:16:20.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.938+0000 7fce6ea00640 1 -- 172.21.15.45:0/3084817799 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce7c005040 con 0x555a91be4f10 2024-04-16T17:16:20.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.938+0000 7fce6ea00640 1 -- 172.21.15.45:0/3084817799 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9496+0+0 (secure 0 0 0) 0x7fce7c0051e0 con 0x555a91be4f10 2024-04-16T17:16:20.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.938+0000 7fce9260e740 1 -- 172.21.15.45:0/3084817799 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fce800a57d0 con 0x555a91be4f10 2024-04-16T17:16:20.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.938+0000 7fce6ea00640 1 -- 172.21.15.45:0/3084817799 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fce7c03b050 con 0x555a91be4f10 2024-04-16T17:16:20.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.938+0000 7fce9260e740 10 client.28085 fetch_fsmap learned FSMap version 44 2024-04-16T17:16:20.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.938+0000 7fce9260e740 1 -- 172.21.15.45:0/3084817799 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x555a91c64f00 con 0x555a91be4f10 2024-04-16T17:16:20.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.938+0000 7fce6ea00640 1 -- 172.21.15.45:0/3084817799 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fce800a57d0 con 0x555a91be4f10 2024-04-16T17:16:20.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.938+0000 7fce6ea00640 1 -- 172.21.15.45:0/3084817799 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 44) ==== 8346+0+0 (secure 0 0 0) 0x7fce7c02b1a0 con 0x555a91be4f10 2024-04-16T17:16:20.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.938+0000 7fce9260e740 10 client.28085 fetch_fsmap finished waiting for FSMap version 44 2024-04-16T17:16:20.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.938+0000 7fce9260e740 10 client.28085 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-04-16T17:16:20.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.938+0000 7fce9260e740 20 client.28085 populate_metadata read hostname 'smithi045' 2024-04-16T17:16:20.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.938+0000 7fce9260e740 1 --2- 172.21.15.45:0/3084817799 >> [v2:172.21.15.45:6840/3755310822,v1:172.21.15.45:6841/3755310822] conn(0x555a91c65430 0x555a91c67820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:20.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.938+0000 7fce9260e740 4 client.28085 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T17:16:20.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.938+0000 7fce9260e740 1 -- 172.21.15.45:0/3084817799 --> [v2:172.21.15.45:6840/3755310822,v1:172.21.15.45:6841/3755310822] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555a91b518b0 con 0x555a91c65430 2024-04-16T17:16:20.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.939+0000 7fce87e00640 1 --2- 172.21.15.45:0/3084817799 >> [v2:172.21.15.45:6840/3755310822,v1:172.21.15.45:6841/3755310822] conn(0x555a91c65430 0x555a91c67820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:20.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.939+0000 7fce87e00640 1 --2- 172.21.15.45:0/3084817799 >> [v2:172.21.15.45:6840/3755310822,v1:172.21.15.45:6841/3755310822] conn(0x555a91c65430 0x555a91c67820 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6c94839ee24a2095 server_cookie=f0dd3c71d5e59828 in_seq=0 out_seq=0 2024-04-16T17:16:20.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.939+0000 7fce6ea00640 10 client.28085 ms_handle_connect on v2:172.21.15.45:6840/3755310822 2024-04-16T17:16:20.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.940+0000 7fce6ea00640 1 -- 172.21.15.45:0/3084817799 <== mds.1 v2:172.21.15.45:6840/3755310822 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555a91b518b0 con 0x555a91c65430 2024-04-16T17:16:20.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.940+0000 7fce6ea00640 10 client.28085 handle_command_reply: tid=0 2024-04-16T17:16:20.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.940+0000 7fce9260e740 1 client.28085 shutdown 2024-04-16T17:16:20.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.940+0000 7fce9260e740 1 -- 172.21.15.45:0/3084817799 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555a91be4f10 msgr2=0x555a91be81c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:20.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.940+0000 7fce9260e740 1 --2- 172.21.15.45:0/3084817799 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555a91be4f10 0x555a91be81c0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fce7c005fd0 tx=0x7fce7c0194c0 comp rx=0 tx=0).stop 2024-04-16T17:16:20.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.940+0000 7fce9260e740 1 -- 172.21.15.45:0/3084817799 shutdown_connections 2024-04-16T17:16:20.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.940+0000 7fce9260e740 1 --2- 172.21.15.45:0/3084817799 >> [v2:172.21.15.45:6840/3755310822,v1:172.21.15.45:6841/3755310822] conn(0x555a91c65430 0x555a91c67820 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:20.947 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.940+0000 7fce9260e740 1 --2- 172.21.15.45:0/3084817799 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555a91be5830 0x555a91be8700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:20.947 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.940+0000 7fce9260e740 1 --2- 172.21.15.45:0/3084817799 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555a91bee410 0x555a91be8c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:20.947 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.940+0000 7fce9260e740 1 --2- 172.21.15.45:0/3084817799 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555a91be4f10 0x555a91be81c0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:20.947 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.940+0000 7fce6ea00640 0 client.28085 ms_handle_reset on v2:172.21.15.45:6840/3755310822 2024-04-16T17:16:20.947 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.940+0000 7fce9260e740 1 -- 172.21.15.45:0/3084817799 >> 172.21.15.45:0/3084817799 conn(0x555a91bdf640 msgr2=0x555a91bdde70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:16:20.947 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.940+0000 7fce9260e740 1 -- 172.21.15.45:0/3084817799 shutdown_connections 2024-04-16T17:16:20.947 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.940+0000 7fce9260e740 1 -- 172.21.15.45:0/3084817799 wait complete. 2024-04-16T17:16:20.947 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.940+0000 7fce9260e740 20 client.28085 trim_cache size 0 max 16384 2024-04-16T17:16:20.948 INFO:teuthology.orchestra.run.smithi045.stdout:[] 2024-04-16T17:16:20.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.941+0000 7fce8e400640 1 -- 172.21.15.45:0/3402208960 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fce700a3d20 msgr2=0x7fce700a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:20.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.941+0000 7fce8e400640 1 --2- 172.21.15.45:0/3402208960 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fce700a3d20 0x7fce700a61e0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fce74009230 tx=0x7fce74005fd0 comp rx=0 tx=0).stop 2024-04-16T17:16:20.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.941+0000 7fce8e400640 1 -- 172.21.15.45:0/3402208960 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fce8819a840 msgr2=0x7fce8819acf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:20.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.941+0000 7fce8e400640 1 --2- 172.21.15.45:0/3402208960 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fce8819a840 0x7fce8819acf0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fce8000efd0 tx=0x7fce8000c6a0 comp rx=0 tx=0).stop 2024-04-16T17:16:20.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.941+0000 7fce8e400640 1 -- 172.21.15.45:0/3402208960 shutdown_connections 2024-04-16T17:16:20.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.941+0000 7fce8e400640 1 --2- 172.21.15.45:0/3402208960 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fce700a3d20 0x7fce700a61e0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:20.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.941+0000 7fce8e400640 1 --2- 172.21.15.45:0/3402208960 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fce881093f0 0x7fce881a0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:20.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.941+0000 7fce8e400640 1 --2- 172.21.15.45:0/3402208960 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fce8819a840 0x7fce8819acf0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:20.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.941+0000 7fce8e400640 1 --2- 172.21.15.45:0/3402208960 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fce88104320 0x7fce881a0680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:20.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.941+0000 7fce8e400640 1 -- 172.21.15.45:0/3402208960 >> 172.21.15.45:0/3402208960 conn(0x7fce880ffb40 msgr2=0x7fce88100e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:16:20.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.941+0000 7fce8e400640 1 -- 172.21.15.45:0/3402208960 shutdown_connections 2024-04-16T17:16:20.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:20.941+0000 7fce8e400640 1 -- 172.21.15.45:0/3402208960 wait complete. 2024-04-16T17:16:20.959 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-16T17:16:21.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:21.192+0000 7fbc21a00640 1 -- 172.21.15.45:0/2756987986 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbc1c104310 msgr2=0x7fbc1c1046f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:21.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:21.192+0000 7fbc21a00640 1 --2- 172.21.15.45:0/2756987986 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbc1c104310 0x7fbc1c1046f0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fbc0c009900 tx=0x7fbc0c02bc30 comp rx=0 tx=0).stop 2024-04-16T17:16:21.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:21.193+0000 7fbc21a00640 1 -- 172.21.15.45:0/2756987986 shutdown_connections 2024-04-16T17:16:21.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:21.193+0000 7fbc21a00640 1 --2- 172.21.15.45:0/2756987986 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbc1c1093e0 0x7fbc1c110fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:21.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:21.193+0000 7fbc21a00640 1 --2- 172.21.15.45:0/2756987986 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbc1c104cc0 0x7fbc1c108d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:21.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:21.193+0000 7fbc21a00640 1 --2- 172.21.15.45:0/2756987986 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbc1c104310 0x7fbc1c1046f0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:21.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:21.193+0000 7fbc21a00640 1 -- 172.21.15.45:0/2756987986 >> 172.21.15.45:0/2756987986 conn(0x7fbc1c0ffb70 msgr2=0x7fbc1c101f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:16:21.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:21.193+0000 7fbc21a00640 1 -- 172.21.15.45:0/2756987986 shutdown_connections 2024-04-16T17:16:21.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:21.193+0000 7fbc21a00640 1 -- 172.21.15.45:0/2756987986 wait complete. 2024-04-16T17:16:21.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:21.194+0000 7fbc21a00640 1 Processor -- start 2024-04-16T17:16:21.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:21.194+0000 7fbc21a00640 1 -- start start 2024-04-16T17:16:21.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:21.194+0000 7fbc21a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbc1c104310 0x7fbc1c1a0620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:21.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:21.194+0000 7fbc21a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbc1c1093e0 0x7fbc1c1a0b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:21.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:21.194+0000 7fbc21a00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbc1c19a7e0 0x7fbc1c19ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:21.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:21.194+0000 7fbc21a00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fbc1c1144c0 con 0x7fbc1c104310 2024-04-16T17:16:21.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:21.194+0000 7fbc21a00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fbc1c114340 con 0x7fbc1c19a7e0 2024-04-16T17:16:21.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:21.194+0000 7fbc21a00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fbc1c114640 con 0x7fbc1c1093e0 2024-04-16T17:16:21.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:21.195+0000 7fbc1be00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbc1c104310 0x7fbc1c1a0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:21.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:21.195+0000 7fbc21000640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbc1c19a7e0 0x7fbc1c19ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:21.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:21.195+0000 7fbc13e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbc1c1093e0 0x7fbc1c1a0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:21.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:21.195+0000 7fbc1be00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbc1c104310 0x7fbc1c1a0620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59172/0 (socket says 172.21.15.45:59172) 2024-04-16T17:16:21.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:21.195+0000 7fbc1be00640 1 -- 172.21.15.45:0/2014672771 learned_addr learned my addr 172.21.15.45:0/2014672771 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:16:21.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:21.195+0000 7fbc13e00640 1 -- 172.21.15.45:0/2014672771 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbc1c19a7e0 msgr2=0x7fbc1c19ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:21.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:21.195+0000 7fbc13e00640 1 --2- 172.21.15.45:0/2014672771 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbc1c19a7e0 0x7fbc1c19ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:21.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:21.195+0000 7fbc13e00640 1 -- 172.21.15.45:0/2014672771 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbc1c104310 msgr2=0x7fbc1c1a0620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:21.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:21.195+0000 7fbc13e00640 1 --2- 172.21.15.45:0/2014672771 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbc1c104310 0x7fbc1c1a0620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:21.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:21.195+0000 7fbc13e00640 1 -- 172.21.15.45:0/2014672771 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc1c19b370 con 0x7fbc1c1093e0 2024-04-16T17:16:21.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:21.195+0000 7fbc13e00640 1 --2- 172.21.15.45:0/2014672771 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbc1c1093e0 0x7fbc1c1a0b60 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fbc04005fd0 tx=0x7fbc0400dde0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:16:21.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:21.196+0000 7fbc19600640 1 -- 172.21.15.45:0/2014672771 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc04009c00 con 0x7fbc1c1093e0 2024-04-16T17:16:21.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:21.196+0000 7fbc19600640 1 -- 172.21.15.45:0/2014672771 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbc04009da0 con 0x7fbc1c1093e0 2024-04-16T17:16:21.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:21.196+0000 7fbc19600640 1 -- 172.21.15.45:0/2014672771 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc04010460 con 0x7fbc1c1093e0 2024-04-16T17:16:21.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:21.196+0000 7fbc21a00640 1 -- 172.21.15.45:0/2014672771 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc1c19b660 con 0x7fbc1c1093e0 2024-04-16T17:16:21.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:21.196+0000 7fbc21a00640 1 -- 172.21.15.45:0/2014672771 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc1c1a77a0 con 0x7fbc1c1093e0 2024-04-16T17:16:21.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:21.199+0000 7fbc25c1f740 1 Processor -- start 2024-04-16T17:16:21.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:21.199+0000 7fbc19600640 1 -- 172.21.15.45:0/2014672771 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fbc04003f50 con 0x7fbc1c1093e0 2024-04-16T17:16:21.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:21.199+0000 7fbc25c1f740 1 -- start start 2024-04-16T17:16:21.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:21.200+0000 7fbc25c1f740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a0fec62f10 0x55a0fec632f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:21.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:21.200+0000 7fbc25c1f740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a0fec63830 0x55a0fec6bed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:21.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:21.200+0000 7fbc25c1f740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a0fec6c410 0x55a0fec6e800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:21.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:21.200+0000 7fbc25c1f740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55a0febc8020 con 0x55a0fec62f10 2024-04-16T17:16:21.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:21.200+0000 7fbc25c1f740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55a0feb6e460 con 0x55a0fec6c410 2024-04-16T17:16:21.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:21.200+0000 7fbc25c1f740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55a0febc3fe0 con 0x55a0fec63830 2024-04-16T17:16:21.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:21.200+0000 7fbc21000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a0fec62f10 0x55a0fec632f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:21.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:21.200+0000 7fbc21000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a0fec62f10 0x55a0fec632f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59174/0 (socket says 172.21.15.45:59174) 2024-04-16T17:16:21.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:21.200+0000 7fbc21000640 1 -- 172.21.15.45:0/2683699 learned_addr learned my addr 172.21.15.45:0/2683699 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:16:21.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:21.200+0000 7fbc1be00640 1 --2- 172.21.15.45:0/2683699 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a0fec63830 0x55a0fec6bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:21.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:21.200+0000 7fbc13e00640 1 --2- 172.21.15.45:0/2683699 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a0fec6c410 0x55a0fec6e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:21.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:21.200+0000 7fbc21000640 1 -- 172.21.15.45:0/2683699 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a0fec63830 msgr2=0x55a0fec6bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:16:21.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:21.201+0000 7fbc21000640 1 --2- 172.21.15.45:0/2683699 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a0fec63830 0x55a0fec6bed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:21.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:21.201+0000 7fbc21000640 1 -- 172.21.15.45:0/2683699 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a0fec6c410 msgr2=0x55a0fec6e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:16:21.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:21.201+0000 7fbc21000640 1 --2- 172.21.15.45:0/2683699 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a0fec6c410 0x55a0fec6e800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:21.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:21.201+0000 7fbc21000640 1 -- 172.21.15.45:0/2683699 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a0feb76040 con 0x55a0fec62f10 2024-04-16T17:16:21.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:21.201+0000 7fbc21000640 1 --2- 172.21.15.45:0/2683699 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a0fec62f10 0x55a0fec632f0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fbc14009f50 tx=0x7fbc140118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=777d8bf7c8ec2080 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:16:21.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:21.201+0000 7fbc19600640 1 --2- 172.21.15.45:0/2014672771 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fbc000a3da0 0x7fbc000a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:21.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:21.201+0000 7fbc19600640 1 -- 172.21.15.45:0/2014672771 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 9496+0+0 (secure 0 0 0) 0x7fbc040d06e0 con 0x7fbc1c1093e0 2024-04-16T17:16:21.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:21.201+0000 7fbc11600640 1 -- 172.21.15.45:0/2683699 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc14004d20 con 0x55a0fec62f10 2024-04-16T17:16:21.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:21.201+0000 7fbc1be00640 1 --2- 172.21.15.45:0/2014672771 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fbc000a3da0 0x7fbc000a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:21.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:21.201+0000 7fbc11600640 1 -- 172.21.15.45:0/2683699 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbc14004050 con 0x55a0fec62f10 2024-04-16T17:16:21.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:21.201+0000 7fbc11600640 1 -- 172.21.15.45:0/2683699 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc14005280 con 0x55a0fec62f10 2024-04-16T17:16:21.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:21.202+0000 7fbc25c1f740 1 -- 172.21.15.45:0/2683699 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a0fec62f10 msgr2=0x55a0fec632f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:21.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:21.202+0000 7fbc25c1f740 1 --2- 172.21.15.45:0/2683699 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a0fec62f10 0x55a0fec632f0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fbc14009f50 tx=0x7fbc140118d0 comp rx=0 tx=0).stop 2024-04-16T17:16:21.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:21.202+0000 7fbc1be00640 1 --2- 172.21.15.45:0/2014672771 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fbc000a3da0 0x7fbc000a6260 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fbc0c0096c0 tx=0x7fbc0c02df90 comp rx=0 tx=0).ready entity=mgr.25411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:16:21.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:21.202+0000 7fbc25c1f740 1 -- 172.21.15.45:0/2683699 shutdown_connections 2024-04-16T17:16:21.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:21.202+0000 7fbc25c1f740 1 --2- 172.21.15.45:0/2683699 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a0fec63830 0x55a0fec6bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:21.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:21.202+0000 7fbc25c1f740 1 --2- 172.21.15.45:0/2683699 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a0fec6c410 0x55a0fec6e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:21.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:21.202+0000 7fbc25c1f740 1 --2- 172.21.15.45:0/2683699 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a0fec62f10 0x55a0fec632f0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:21.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:21.202+0000 7fbc25c1f740 1 -- 172.21.15.45:0/2683699 >> 172.21.15.45:0/2683699 conn(0x55a0fec5d640 msgr2=0x55a0fec5da20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:16:21.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:21.202+0000 7fbc25c1f740 1 -- 172.21.15.45:0/2683699 shutdown_connections 2024-04-16T17:16:21.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:21.202+0000 7fbc25c1f740 1 -- 172.21.15.45:0/2683699 wait complete. 2024-04-16T17:16:21.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:21.203+0000 7fbc25c1f740 1 -- start start 2024-04-16T17:16:21.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:21.203+0000 7fbc25c1f740 1 Processor -- start 2024-04-16T17:16:21.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:21.203+0000 7fbc25c1f740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a0fec62f10 0x55a0fec694e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:21.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:21.204+0000 7fbc25c1f740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a0fec63830 0x55a0fec69a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:21.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:21.204+0000 7fbc25c1f740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a0fec6c410 0x55a0fec69f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:21.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:21.204+0000 7fbc25c1f740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55a0fec6f200 con 0x55a0fec63830 2024-04-16T17:16:21.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:21.204+0000 7fbc25c1f740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55a0feb81830 con 0x55a0fec6c410 2024-04-16T17:16:21.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:21.204+0000 7fbc25c1f740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55a0fec6f380 con 0x55a0fec62f10 2024-04-16T17:16:21.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:21.204+0000 7fbc25c1f740 10 client.0 _ll_register_callbacks cb 0x55a0feb89150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:16:21.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:21.204+0000 7fbc13e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a0fec63830 0x55a0fec69a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:21.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:21.204+0000 7fbc13e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a0fec63830 0x55a0fec69a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59182/0 (socket says 172.21.15.45:59182) 2024-04-16T17:16:21.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:21.204+0000 7fbc13e00640 1 -- 172.21.15.45:0/2096591974 learned_addr learned my addr 172.21.15.45:0/2096591974 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:16:21.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:21.204+0000 7fbc1be00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a0fec62f10 0x55a0fec694e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:21.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:21.204+0000 7fbc21000640 1 --2- 172.21.15.45:0/2096591974 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a0fec6c410 0x55a0fec69f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:21.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:21.204+0000 7fbc13e00640 1 -- 172.21.15.45:0/2096591974 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a0fec62f10 msgr2=0x55a0fec694e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:16:21.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:21.204+0000 7fbc13e00640 1 --2- 172.21.15.45:0/2096591974 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a0fec62f10 0x55a0fec694e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:21.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:21.204+0000 7fbc13e00640 1 -- 172.21.15.45:0/2096591974 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a0fec6c410 msgr2=0x55a0fec69f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:16:21.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:21.204+0000 7fbc13e00640 1 --2- 172.21.15.45:0/2096591974 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a0fec6c410 0x55a0fec69f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:21.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:21.204+0000 7fbc13e00640 1 -- 172.21.15.45:0/2096591974 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a0feb85920 con 0x55a0fec63830 2024-04-16T17:16:21.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:21.205+0000 7fbc13e00640 1 --2- 172.21.15.45:0/2096591974 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a0fec63830 0x55a0fec69a20 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fbc040cda60 tx=0x7fbc0409bf00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97d6bcda46b8cdf4 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:16:21.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:21.205+0000 7fbc0a000640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-16T17:16:21.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:21.205+0000 7fbc0a000640 1 -- 172.21.15.45:0/2096591974 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc04020070 con 0x55a0fec63830 2024-04-16T17:16:21.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:21.205+0000 7fbc0a000640 1 -- 172.21.15.45:0/2096591974 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9496+0+0 (secure 0 0 0) 0x7fbc040cdc80 con 0x55a0fec63830 2024-04-16T17:16:21.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:21.205+0000 7fbc25c1f740 1 -- 172.21.15.45:0/2096591974 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbc14004050 con 0x55a0fec63830 2024-04-16T17:16:21.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:21.205+0000 7fbc0a000640 1 -- 172.21.15.45:0/2096591974 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fbc040cde80 con 0x55a0fec63830 2024-04-16T17:16:21.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:21.205+0000 7fbc25c1f740 10 client.17946 fetch_fsmap learned FSMap version 44 2024-04-16T17:16:21.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:21.205+0000 7fbc25c1f740 1 -- 172.21.15.45:0/2096591974 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55a0feb537f0 con 0x55a0fec63830 2024-04-16T17:16:21.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:21.206+0000 7fbc0a000640 1 -- 172.21.15.45:0/2096591974 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbc14004050 con 0x55a0fec63830 2024-04-16T17:16:21.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:21.206+0000 7fbc0a000640 1 -- 172.21.15.45:0/2096591974 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 44) ==== 8346+0+0 (secure 0 0 0) 0x7fbc040d0240 con 0x55a0fec63830 2024-04-16T17:16:21.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:21.206+0000 7fbc25c1f740 10 client.17946 fetch_fsmap finished waiting for FSMap version 44 2024-04-16T17:16:21.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:21.206+0000 7fbc25c1f740 10 client.17946 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:16:21.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:21.206+0000 7fbc25c1f740 20 client.17946 populate_metadata read hostname 'smithi045' 2024-04-16T17:16:21.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:21.206+0000 7fbc25c1f740 1 --2- 172.21.15.45:0/2096591974 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55a0fec7c820 0x55a0fec7ec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:21.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:21.206+0000 7fbc25c1f740 4 client.17946 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:16:21.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:21.206+0000 7fbc25c1f740 1 -- 172.21.15.45:0/2096591974 --> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a0febd5a40 con 0x55a0fec7c820 2024-04-16T17:16:21.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:21.206+0000 7fbc1be00640 1 --2- 172.21.15.45:0/2096591974 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55a0fec7c820 0x55a0fec7ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:21.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:21.207+0000 7fbc1be00640 1 --2- 172.21.15.45:0/2096591974 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55a0fec7c820 0x55a0fec7ec10 crc :-1 s=READY pgs=243 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1023f3ef903863f0 server_cookie=4ddfab1d7738e581 in_seq=0 out_seq=0 2024-04-16T17:16:21.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:21.207+0000 7fbc0a000640 10 client.17946 ms_handle_connect on v2:172.21.15.104:6838/3660070592 2024-04-16T17:16:21.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:21.208+0000 7fbc0a000640 1 -- 172.21.15.45:0/2096591974 <== mds.0 v2:172.21.15.104:6838/3660070592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a0febd5a40 con 0x55a0fec7c820 2024-04-16T17:16:21.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:21.208+0000 7fbc0a000640 10 client.17946 handle_command_reply: tid=0 2024-04-16T17:16:21.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:21.208+0000 7fbc25c1f740 1 client.17946 shutdown 2024-04-16T17:16:21.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:21.208+0000 7fbc25c1f740 1 -- 172.21.15.45:0/2096591974 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a0fec63830 msgr2=0x55a0fec69a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:21.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:21.208+0000 7fbc25c1f740 1 --2- 172.21.15.45:0/2096591974 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a0fec63830 0x55a0fec69a20 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fbc040cda60 tx=0x7fbc0409bf00 comp rx=0 tx=0).stop 2024-04-16T17:16:21.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:21.208+0000 7fbc25c1f740 1 -- 172.21.15.45:0/2096591974 shutdown_connections 2024-04-16T17:16:21.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:21.208+0000 7fbc25c1f740 1 --2- 172.21.15.45:0/2096591974 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a0fec6c410 0x55a0fec69f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:21.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:21.208+0000 7fbc25c1f740 1 --2- 172.21.15.45:0/2096591974 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a0fec63830 0x55a0fec69a20 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:21.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:21.208+0000 7fbc25c1f740 1 --2- 172.21.15.45:0/2096591974 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55a0fec7c820 0x55a0fec7ec10 crc :-1 s=READY pgs=243 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:21.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:21.208+0000 7fbc25c1f740 1 --2- 172.21.15.45:0/2096591974 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a0fec62f10 0x55a0fec694e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:21.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:21.208+0000 7fbc0a000640 0 client.17946 ms_handle_reset on v2:172.21.15.104:6838/3660070592 2024-04-16T17:16:21.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:21.208+0000 7fbc25c1f740 1 -- 172.21.15.45:0/2096591974 >> 172.21.15.45:0/2096591974 conn(0x55a0fec5d640 msgr2=0x55a0fec5f390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:16:21.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:21.208+0000 7fbc25c1f740 1 -- 172.21.15.45:0/2096591974 shutdown_connections 2024-04-16T17:16:21.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:21.208+0000 7fbc25c1f740 1 -- 172.21.15.45:0/2096591974 wait complete. 2024-04-16T17:16:21.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:21.208+0000 7fbc25c1f740 20 client.17946 trim_cache size 0 max 16384 2024-04-16T17:16:21.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:21.224+0000 7fbc25c1f740 1 Processor -- start 2024-04-16T17:16:21.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:21.224+0000 7fbc25c1f740 1 -- start start 2024-04-16T17:16:21.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:21.224+0000 7fbc25c1f740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a0fec62f10 0x55a0fec6bc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:21.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:21.224+0000 7fbc25c1f740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a0fec63830 0x55a0fec780c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:21.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:21.224+0000 7fbc25c1f740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a0fec6c410 0x55a0fec69bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:21.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:21.224+0000 7fbc25c1f740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fbc1c1b3ec0 con 0x55a0fec6c410 2024-04-16T17:16:21.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:21.224+0000 7fbc25c1f740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fbc1c1b1790 con 0x55a0fec62f10 2024-04-16T17:16:21.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:21.224+0000 7fbc25c1f740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fbc1c1af600 con 0x55a0fec63830 2024-04-16T17:16:21.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:21.225+0000 7fbc21000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a0fec6c410 0x55a0fec69bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:21.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:21.225+0000 7fbc21000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a0fec6c410 0x55a0fec69bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59194/0 (socket says 172.21.15.45:59194) 2024-04-16T17:16:21.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:21.225+0000 7fbc21000640 1 -- 172.21.15.45:0/1581736542 learned_addr learned my addr 172.21.15.45:0/1581736542 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:16:21.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:21.225+0000 7fbc13e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a0fec63830 0x55a0fec780c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:21.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:21.225+0000 7fbc1be00640 1 --2- 172.21.15.45:0/1581736542 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a0fec62f10 0x55a0fec6bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:21.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:21.225+0000 7fbc13e00640 1 -- 172.21.15.45:0/1581736542 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a0fec62f10 msgr2=0x55a0fec6bc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:16:21.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:21.225+0000 7fbc13e00640 1 --2- 172.21.15.45:0/1581736542 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a0fec62f10 0x55a0fec6bc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:21.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:21.225+0000 7fbc13e00640 1 -- 172.21.15.45:0/1581736542 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a0fec6c410 msgr2=0x55a0fec69bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:16:21.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:21.225+0000 7fbc13e00640 1 --2- 172.21.15.45:0/1581736542 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a0fec6c410 0x55a0fec69bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:21.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:21.225+0000 7fbc13e00640 1 -- 172.21.15.45:0/1581736542 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a0fe8da3f0 con 0x55a0fec63830 2024-04-16T17:16:21.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:21.225+0000 7fbc21000640 1 --2- 172.21.15.45:0/1581736542 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a0fec6c410 0x55a0fec69bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:16:21.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:21.225+0000 7fbc13e00640 1 --2- 172.21.15.45:0/1581736542 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a0fec63830 0x55a0fec780c0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fbc0409a5b0 tx=0x7fbc04003800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=130bb457feab7ea0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:16:21.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:21.226+0000 7fbc10c00640 1 -- 172.21.15.45:0/1581736542 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc040a02f0 con 0x55a0fec63830 2024-04-16T17:16:21.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:21.226+0000 7fbc10c00640 1 -- 172.21.15.45:0/1581736542 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbc040d6140 con 0x55a0fec63830 2024-04-16T17:16:21.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:21.226+0000 7fbc10c00640 1 -- 172.21.15.45:0/1581736542 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc040a0100 con 0x55a0fec63830 2024-04-16T17:16:21.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:21.227+0000 7fbc25c1f740 1 -- 172.21.15.45:0/1581736542 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a0fec63830 msgr2=0x55a0fec780c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:21.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:21.227+0000 7fbc25c1f740 1 --2- 172.21.15.45:0/1581736542 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a0fec63830 0x55a0fec780c0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fbc0409a5b0 tx=0x7fbc04003800 comp rx=0 tx=0).stop 2024-04-16T17:16:21.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:21.227+0000 7fbc25c1f740 1 -- 172.21.15.45:0/1581736542 shutdown_connections 2024-04-16T17:16:21.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:21.227+0000 7fbc25c1f740 1 --2- 172.21.15.45:0/1581736542 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a0fec63830 0x55a0fec780c0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:21.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:21.227+0000 7fbc25c1f740 1 --2- 172.21.15.45:0/1581736542 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a0fec6c410 0x55a0fec69bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:21.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:21.227+0000 7fbc25c1f740 1 --2- 172.21.15.45:0/1581736542 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a0fec62f10 0x55a0fec6bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:21.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:21.227+0000 7fbc25c1f740 1 -- 172.21.15.45:0/1581736542 >> 172.21.15.45:0/1581736542 conn(0x55a0fec5d640 msgr2=0x55a0fec694a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:16:21.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:21.227+0000 7fbc25c1f740 1 -- 172.21.15.45:0/1581736542 shutdown_connections 2024-04-16T17:16:21.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:21.227+0000 7fbc25c1f740 1 -- 172.21.15.45:0/1581736542 wait complete. 2024-04-16T17:16:21.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:21.227+0000 7fbc25c1f740 1 -- start start 2024-04-16T17:16:21.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:21.228+0000 7fbc25c1f740 1 Processor -- start 2024-04-16T17:16:21.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:21.228+0000 7fbc25c1f740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a0fec62f10 0x55a0fec749b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:21.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:21.228+0000 7fbc25c1f740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a0fec63830 0x55a0fec6fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:21.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:21.228+0000 7fbc25c1f740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a0fec6c410 0x55a0fec70260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:21.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:21.228+0000 7fbc25c1f740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fbc1c1afc50 con 0x55a0fec62f10 2024-04-16T17:16:21.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:21.228+0000 7fbc25c1f740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fbc1c1b10c0 con 0x55a0fec63830 2024-04-16T17:16:21.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:21.228+0000 7fbc25c1f740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fbc1c1b0320 con 0x55a0fec6c410 2024-04-16T17:16:21.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:21.228+0000 7fbc25c1f740 10 client.0 _ll_register_callbacks cb 0x7fbc04098650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:16:21.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:21.228+0000 7fbc1be00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a0fec62f10 0x55a0fec749b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:21.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:21.228+0000 7fbc1be00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a0fec62f10 0x55a0fec749b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59196/0 (socket says 172.21.15.45:59196) 2024-04-16T17:16:21.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:21.228+0000 7fbc1be00640 1 -- 172.21.15.45:0/2483200749 learned_addr learned my addr 172.21.15.45:0/2483200749 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:16:21.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:21.228+0000 7fbc13e00640 1 --2- 172.21.15.45:0/2483200749 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a0fec63830 0x55a0fec6fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:21.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:21.228+0000 7fbc21000640 1 --2- 172.21.15.45:0/2483200749 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a0fec6c410 0x55a0fec70260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:21.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:21.229+0000 7fbc13e00640 1 -- 172.21.15.45:0/2483200749 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a0fec6c410 msgr2=0x55a0fec70260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:16:21.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:21.229+0000 7fbc13e00640 1 --2- 172.21.15.45:0/2483200749 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a0fec6c410 0x55a0fec70260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:21.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:21.229+0000 7fbc13e00640 1 -- 172.21.15.45:0/2483200749 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a0fec62f10 msgr2=0x55a0fec749b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:16:21.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:21.229+0000 7fbc13e00640 1 --2- 172.21.15.45:0/2483200749 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a0fec62f10 0x55a0fec749b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:21.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:21.229+0000 7fbc13e00640 1 -- 172.21.15.45:0/2483200749 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a0fe9a47d0 con 0x55a0fec63830 2024-04-16T17:16:21.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:21.229+0000 7fbc13e00640 1 --2- 172.21.15.45:0/2483200749 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a0fec63830 0x55a0fec6fd20 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x55a0febd5b10 tx=0x7fbc040a4df0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=84f6d6ad3f36e8f0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:16:21.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:21.229+0000 7fbc0a000640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-04-16T17:16:21.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:21.229+0000 7fbc0a000640 1 -- 172.21.15.45:0/2483200749 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc040a4ec0 con 0x55a0fec63830 2024-04-16T17:16:21.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:21.229+0000 7fbc0a000640 1 -- 172.21.15.45:0/2483200749 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9496+0+0 (secure 0 0 0) 0x7fbc040a0110 con 0x55a0fec63830 2024-04-16T17:16:21.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:21.229+0000 7fbc25c1f740 1 -- 172.21.15.45:0/2483200749 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55a0febcf8b0 con 0x55a0fec63830 2024-04-16T17:16:21.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:21.229+0000 7fbc0a000640 1 -- 172.21.15.45:0/2483200749 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fbc040cdc80 con 0x55a0fec63830 2024-04-16T17:16:21.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:21.229+0000 7fbc25c1f740 10 client.28103 fetch_fsmap learned FSMap version 44 2024-04-16T17:16:21.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:21.229+0000 7fbc25c1f740 1 -- 172.21.15.45:0/2483200749 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55a0fece2b40 con 0x55a0fec63830 2024-04-16T17:16:21.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:21.230+0000 7fbc0a000640 1 -- 172.21.15.45:0/2483200749 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55a0febcf8b0 con 0x55a0fec63830 2024-04-16T17:16:21.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:21.230+0000 7fbc0a000640 1 -- 172.21.15.45:0/2483200749 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 44) ==== 8346+0+0 (secure 0 0 0) 0x7fbc0409b1c0 con 0x55a0fec63830 2024-04-16T17:16:21.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:21.230+0000 7fbc25c1f740 10 client.28103 fetch_fsmap finished waiting for FSMap version 44 2024-04-16T17:16:21.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:21.230+0000 7fbc25c1f740 10 client.28103 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:16:21.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:21.230+0000 7fbc25c1f740 20 client.28103 populate_metadata read hostname 'smithi045' 2024-04-16T17:16:21.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:21.230+0000 7fbc25c1f740 1 --2- 172.21.15.45:0/2483200749 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55a0fece31a0 0x55a0fece5590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:21.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:21.230+0000 7fbc25c1f740 4 client.28103 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T17:16:21.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:21.230+0000 7fbc25c1f740 1 -- 172.21.15.45:0/2483200749 --> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a0feb4b570 con 0x55a0fece31a0 2024-04-16T17:16:21.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:21.230+0000 7fbc1be00640 1 --2- 172.21.15.45:0/2483200749 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55a0fece31a0 0x55a0fece5590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:21.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:21.231+0000 7fbc1be00640 1 --2- 172.21.15.45:0/2483200749 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55a0fece31a0 0x55a0fece5590 crc :-1 s=READY pgs=244 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=caf8b9e573098d87 server_cookie=27ed36f96db43ce0 in_seq=0 out_seq=0 2024-04-16T17:16:21.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:21.231+0000 7fbc0a000640 10 client.28103 ms_handle_connect on v2:172.21.15.104:6838/3660070592 2024-04-16T17:16:21.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:21.231+0000 7fbc0a000640 1 -- 172.21.15.45:0/2483200749 <== mds.0 v2:172.21.15.104:6838/3660070592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a0feb4b570 con 0x55a0fece31a0 2024-04-16T17:16:21.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:21.231+0000 7fbc0a000640 10 client.28103 handle_command_reply: tid=0 2024-04-16T17:16:21.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:21.231+0000 7fbc25c1f740 1 client.28103 shutdown 2024-04-16T17:16:21.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:21.231+0000 7fbc25c1f740 1 -- 172.21.15.45:0/2483200749 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a0fec63830 msgr2=0x55a0fec6fd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:21.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:21.231+0000 7fbc25c1f740 1 --2- 172.21.15.45:0/2483200749 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a0fec63830 0x55a0fec6fd20 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x55a0febd5b10 tx=0x7fbc040a4df0 comp rx=0 tx=0).stop 2024-04-16T17:16:21.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:21.231+0000 7fbc25c1f740 1 -- 172.21.15.45:0/2483200749 shutdown_connections 2024-04-16T17:16:21.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:21.231+0000 7fbc25c1f740 1 --2- 172.21.15.45:0/2483200749 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55a0fece31a0 0x55a0fece5590 crc :-1 s=READY pgs=244 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:21.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:21.231+0000 7fbc25c1f740 1 --2- 172.21.15.45:0/2483200749 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a0fec6c410 0x55a0fec70260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:21.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:21.231+0000 7fbc25c1f740 1 --2- 172.21.15.45:0/2483200749 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a0fec63830 0x55a0fec6fd20 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:21.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:21.231+0000 7fbc25c1f740 1 --2- 172.21.15.45:0/2483200749 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a0fec62f10 0x55a0fec749b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:21.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:21.231+0000 7fbc25c1f740 1 -- 172.21.15.45:0/2483200749 >> 172.21.15.45:0/2483200749 conn(0x55a0fec5d640 msgr2=0x55a0fec694a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:16:21.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:21.231+0000 7fbc0a000640 0 client.28103 ms_handle_reset on v2:172.21.15.104:6838/3660070592 2024-04-16T17:16:21.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:21.232+0000 7fbc25c1f740 1 -- 172.21.15.45:0/2483200749 shutdown_connections 2024-04-16T17:16:21.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:21.232+0000 7fbc25c1f740 1 -- 172.21.15.45:0/2483200749 wait complete. 2024-04-16T17:16:21.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:21.232+0000 7fbc25c1f740 20 client.28103 trim_cache size 0 max 16384 2024-04-16T17:16:21.239 INFO:teuthology.orchestra.run.smithi045.stdout:[] 2024-04-16T17:16:21.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:21.232+0000 7fbc21a00640 1 -- 172.21.15.45:0/2014672771 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fbc000a3da0 msgr2=0x7fbc000a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:21.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:21.232+0000 7fbc21a00640 1 --2- 172.21.15.45:0/2014672771 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fbc000a3da0 0x7fbc000a6260 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fbc0c0096c0 tx=0x7fbc0c02df90 comp rx=0 tx=0).stop 2024-04-16T17:16:21.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:21.232+0000 7fbc21a00640 1 -- 172.21.15.45:0/2014672771 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbc1c1093e0 msgr2=0x7fbc1c1a0b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:21.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:21.232+0000 7fbc21a00640 1 --2- 172.21.15.45:0/2014672771 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbc1c1093e0 0x7fbc1c1a0b60 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fbc04005fd0 tx=0x7fbc0400dde0 comp rx=0 tx=0).stop 2024-04-16T17:16:21.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:21.233+0000 7fbc21a00640 1 -- 172.21.15.45:0/2014672771 shutdown_connections 2024-04-16T17:16:21.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:21.233+0000 7fbc21a00640 1 --2- 172.21.15.45:0/2014672771 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fbc000a3da0 0x7fbc000a6260 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:21.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:21.233+0000 7fbc21a00640 1 --2- 172.21.15.45:0/2014672771 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbc1c1093e0 0x7fbc1c1a0b60 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:21.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:21.233+0000 7fbc21a00640 1 --2- 172.21.15.45:0/2014672771 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbc1c19a7e0 0x7fbc1c19ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:21.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:21.233+0000 7fbc21a00640 1 --2- 172.21.15.45:0/2014672771 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbc1c104310 0x7fbc1c1a0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:21.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:21.233+0000 7fbc21a00640 1 -- 172.21.15.45:0/2014672771 >> 172.21.15.45:0/2014672771 conn(0x7fbc1c0ffb70 msgr2=0x7fbc1c101ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:16:21.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:21.233+0000 7fbc21a00640 1 -- 172.21.15.45:0/2014672771 shutdown_connections 2024-04-16T17:16:21.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:21.233+0000 7fbc21a00640 1 -- 172.21.15.45:0/2014672771 wait complete. 2024-04-16T17:16:22.250 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-16T17:16:22.250 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T17:16:22.251 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-16T17:16:22.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.490+0000 7f3ea7600640 1 -- 172.21.15.45:0/2007624796 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3ea0076e50 msgr2=0x7f3ea01129a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:22.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.491+0000 7f3ea7600640 1 --2- 172.21.15.45:0/2007624796 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3ea0076e50 0x7f3ea01129a0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f3e8c009930 tx=0x7f3e8c02bc20 comp rx=0 tx=0).stop 2024-04-16T17:16:22.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.491+0000 7f3ea7600640 1 -- 172.21.15.45:0/2007624796 shutdown_connections 2024-04-16T17:16:22.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.491+0000 7f3ea7600640 1 --2- 172.21.15.45:0/2007624796 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3ea0112ee0 0x7f3ea01152d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:22.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.491+0000 7f3ea7600640 1 --2- 172.21.15.45:0/2007624796 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3ea0076e50 0x7f3ea01129a0 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:22.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.491+0000 7f3ea7600640 1 --2- 172.21.15.45:0/2007624796 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3ea0076530 0x7f3ea0076910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:22.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.491+0000 7f3ea7600640 1 -- 172.21.15.45:0/2007624796 >> 172.21.15.45:0/2007624796 conn(0x7f3ea00ffd90 msgr2=0x7f3ea01021b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:16:22.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.491+0000 7f3ea7600640 1 -- 172.21.15.45:0/2007624796 shutdown_connections 2024-04-16T17:16:22.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.491+0000 7f3ea7600640 1 -- 172.21.15.45:0/2007624796 wait complete. 2024-04-16T17:16:22.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.492+0000 7f3ea7600640 1 Processor -- start 2024-04-16T17:16:22.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.492+0000 7f3ea7600640 1 -- start start 2024-04-16T17:16:22.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.493+0000 7f3ea7600640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3ea0076530 0x7f3ea019f040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:22.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.493+0000 7f3ea7600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3ea0076e50 0x7f3ea019f580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:22.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.493+0000 7f3ea7600640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3ea0112ee0 0x7f3ea01a38b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:22.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.493+0000 7f3ea7600640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f3ea0118a40 con 0x7f3ea0076e50 2024-04-16T17:16:22.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.493+0000 7f3ea7600640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f3ea01188c0 con 0x7f3ea0112ee0 2024-04-16T17:16:22.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.493+0000 7f3ea7600640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f3ea0118bc0 con 0x7f3ea0076530 2024-04-16T17:16:22.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.493+0000 7f3e9fe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3ea0076e50 0x7f3ea019f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:22.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.493+0000 7f3e9fe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3ea0076e50 0x7f3ea019f580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59212/0 (socket says 172.21.15.45:59212) 2024-04-16T17:16:22.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.493+0000 7f3e9fe00640 1 -- 172.21.15.45:0/584683857 learned_addr learned my addr 172.21.15.45:0/584683857 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:16:22.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.493+0000 7f3ea4c00640 1 --2- 172.21.15.45:0/584683857 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3ea0076530 0x7f3ea019f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:22.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.494+0000 7f3ea5600640 1 --2- 172.21.15.45:0/584683857 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3ea0112ee0 0x7f3ea01a38b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:22.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.494+0000 7f3e9fe00640 1 -- 172.21.15.45:0/584683857 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3ea0076530 msgr2=0x7f3ea019f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:22.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.494+0000 7f3e9fe00640 1 --2- 172.21.15.45:0/584683857 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3ea0076530 0x7f3ea019f040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:22.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.494+0000 7f3e9fe00640 1 -- 172.21.15.45:0/584683857 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3ea0112ee0 msgr2=0x7f3ea01a38b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:22.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.494+0000 7f3e9fe00640 1 --2- 172.21.15.45:0/584683857 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3ea0112ee0 0x7f3ea01a38b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:22.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.494+0000 7f3e9fe00640 1 -- 172.21.15.45:0/584683857 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ea01a3fe0 con 0x7f3ea0076e50 2024-04-16T17:16:22.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.494+0000 7f3ea4c00640 1 --2- 172.21.15.45:0/584683857 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3ea0076530 0x7f3ea019f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:16:22.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.494+0000 7f3e9fe00640 1 --2- 172.21.15.45:0/584683857 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3ea0076e50 0x7f3ea019f580 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f3e8c009430 tx=0x7f3e8c02cb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:16:22.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.495+0000 7f3e9d600640 1 -- 172.21.15.45:0/584683857 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e8c041070 con 0x7f3ea0076e50 2024-04-16T17:16:22.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.495+0000 7f3e9d600640 1 -- 172.21.15.45:0/584683857 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3e8c004450 con 0x7f3ea0076e50 2024-04-16T17:16:22.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.495+0000 7f3e9d600640 1 -- 172.21.15.45:0/584683857 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e8c003d00 con 0x7f3ea0076e50 2024-04-16T17:16:22.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.495+0000 7f3ea7600640 1 -- 172.21.15.45:0/584683857 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ea01a4270 con 0x7f3ea0076e50 2024-04-16T17:16:22.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.495+0000 7f3ea7600640 1 -- 172.21.15.45:0/584683857 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ea01abb60 con 0x7f3ea0076e50 2024-04-16T17:16:22.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.498+0000 7f3e9d600640 1 -- 172.21.15.45:0/584683857 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f3e8c044070 con 0x7f3ea0076e50 2024-04-16T17:16:22.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.498+0000 7f3eaa372740 1 Processor -- start 2024-04-16T17:16:22.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.498+0000 7f3eaa372740 1 -- start start 2024-04-16T17:16:22.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.499+0000 7f3eaa372740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e216ad4ca0 0x55e216ad5080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:22.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.499+0000 7f3eaa372740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e216ad55c0 0x55e216addc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:22.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.499+0000 7f3eaa372740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e216ade1a0 0x55e216ae0590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:22.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.499+0000 7f3eaa372740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55e216a39cb0 con 0x55e216ad4ca0 2024-04-16T17:16:22.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.499+0000 7f3eaa372740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55e2169e0370 con 0x55e216ad55c0 2024-04-16T17:16:22.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.499+0000 7f3eaa372740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55e216a35c70 con 0x55e216ade1a0 2024-04-16T17:16:22.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.499+0000 7f3ea5600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e216ad4ca0 0x55e216ad5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:22.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.499+0000 7f3ea4c00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e216ad55c0 0x55e216addc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:22.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.499+0000 7f3e9fe00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e216ade1a0 0x55e216ae0590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:22.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.499+0000 7f3ea5600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e216ad4ca0 0x55e216ad5080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59222/0 (socket says 172.21.15.45:59222) 2024-04-16T17:16:22.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.499+0000 7f3ea5600640 1 -- 172.21.15.45:0/1966666328 learned_addr learned my addr 172.21.15.45:0/1966666328 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:16:22.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.500+0000 7f3ea4c00640 1 -- 172.21.15.45:0/1966666328 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e216ade1a0 msgr2=0x55e216ae0590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:16:22.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.500+0000 7f3ea4c00640 1 --2- 172.21.15.45:0/1966666328 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e216ade1a0 0x55e216ae0590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:22.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.500+0000 7f3ea4c00640 1 -- 172.21.15.45:0/1966666328 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e216ad4ca0 msgr2=0x55e216ad5080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:16:22.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.500+0000 7f3ea4c00640 1 --2- 172.21.15.45:0/1966666328 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e216ad4ca0 0x55e216ad5080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:22.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.500+0000 7f3ea4c00640 1 -- 172.21.15.45:0/1966666328 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e2169ff910 con 0x55e216ad55c0 2024-04-16T17:16:22.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.500+0000 7f3ea5600640 1 --2- 172.21.15.45:0/1966666328 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e216ad4ca0 0x55e216ad5080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:16:22.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.500+0000 7f3ea4c00640 1 --2- 172.21.15.45:0/1966666328 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e216ad55c0 0x55e216addc60 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f3e94009fd0 tx=0x7f3e940118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9e2132e532a69496 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:16:22.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.500+0000 7f3e9d600640 1 --2- 172.21.15.45:0/584683857 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f3e880a3d20 0x7f3e880a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:22.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.500+0000 7f3e9d600640 1 -- 172.21.15.45:0/584683857 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 9496+0+0 (secure 0 0 0) 0x7f3e8c0f24e0 con 0x7f3ea0076e50 2024-04-16T17:16:22.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.501+0000 7f3ea5600640 1 --2- 172.21.15.45:0/584683857 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f3e880a3d20 0x7f3e880a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:22.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.501+0000 7f3ea5600640 1 --2- 172.21.15.45:0/584683857 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f3e880a3d20 0x7f3e880a61e0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f3e98012780 tx=0x7f3e98009500 comp rx=0 tx=0).ready entity=mgr.25411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:16:22.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.504+0000 7f3e91c00640 1 -- 172.21.15.45:0/1966666328 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e94004310 con 0x55e216ad55c0 2024-04-16T17:16:22.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.504+0000 7f3e91c00640 1 -- 172.21.15.45:0/1966666328 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3e940044b0 con 0x55e216ad55c0 2024-04-16T17:16:22.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.504+0000 7f3e91c00640 1 -- 172.21.15.45:0/1966666328 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e940035f0 con 0x55e216ad55c0 2024-04-16T17:16:22.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.505+0000 7f3eaa372740 1 -- 172.21.15.45:0/1966666328 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e216ad55c0 msgr2=0x55e216addc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:22.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.505+0000 7f3eaa372740 1 --2- 172.21.15.45:0/1966666328 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e216ad55c0 0x55e216addc60 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f3e94009fd0 tx=0x7f3e940118d0 comp rx=0 tx=0).stop 2024-04-16T17:16:22.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.505+0000 7f3eaa372740 1 -- 172.21.15.45:0/1966666328 shutdown_connections 2024-04-16T17:16:22.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.505+0000 7f3eaa372740 1 --2- 172.21.15.45:0/1966666328 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e216ade1a0 0x55e216ae0590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:22.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.505+0000 7f3eaa372740 1 --2- 172.21.15.45:0/1966666328 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e216ad55c0 0x55e216addc60 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:22.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.505+0000 7f3eaa372740 1 --2- 172.21.15.45:0/1966666328 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e216ad4ca0 0x55e216ad5080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:22.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.505+0000 7f3eaa372740 1 -- 172.21.15.45:0/1966666328 >> 172.21.15.45:0/1966666328 conn(0x55e216acf3d0 msgr2=0x55e216acf7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:16:22.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.505+0000 7f3eaa372740 1 -- 172.21.15.45:0/1966666328 shutdown_connections 2024-04-16T17:16:22.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.505+0000 7f3eaa372740 1 -- 172.21.15.45:0/1966666328 wait complete. 2024-04-16T17:16:22.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.505+0000 7f3eaa372740 1 -- start start 2024-04-16T17:16:22.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.506+0000 7f3eaa372740 1 Processor -- start 2024-04-16T17:16:22.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.506+0000 7f3eaa372740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e216ad4ca0 0x55e216adb230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:22.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.506+0000 7f3eaa372740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e216ad55c0 0x55e216adb770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:22.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.506+0000 7f3eaa372740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e216ade1a0 0x55e216adbcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:22.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.506+0000 7f3eaa372740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55e216ae0f90 con 0x55e216ad4ca0 2024-04-16T17:16:22.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.506+0000 7f3eaa372740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55e2169f3730 con 0x55e216ade1a0 2024-04-16T17:16:22.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.506+0000 7f3eaa372740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55e216ae1110 con 0x55e216ad55c0 2024-04-16T17:16:22.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.506+0000 7f3eaa372740 10 client.0 _ll_register_callbacks cb 0x55e2169c21e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:16:22.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.506+0000 7f3ea4c00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e216ad4ca0 0x55e216adb230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:22.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.507+0000 7f3ea5600640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e216ade1a0 0x55e216adbcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:22.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.507+0000 7f3e9fe00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e216ad55c0 0x55e216adb770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:22.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.507+0000 7f3ea5600640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e216ade1a0 0x55e216adbcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:42950/0 (socket says 172.21.15.45:42950) 2024-04-16T17:16:22.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.507+0000 7f3ea5600640 1 -- 172.21.15.45:0/2756277321 learned_addr learned my addr 172.21.15.45:0/2756277321 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:16:22.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.507+0000 7f3ea5600640 1 -- 172.21.15.45:0/2756277321 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e216ad55c0 msgr2=0x55e216adb770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:16:22.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.507+0000 7f3ea5600640 1 --2- 172.21.15.45:0/2756277321 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e216ad55c0 0x55e216adb770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:22.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.507+0000 7f3ea5600640 1 -- 172.21.15.45:0/2756277321 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e216ad4ca0 msgr2=0x55e216adb230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:16:22.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.507+0000 7f3ea5600640 1 --2- 172.21.15.45:0/2756277321 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e216ad4ca0 0x55e216adb230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:22.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.507+0000 7f3ea5600640 1 -- 172.21.15.45:0/2756277321 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e2168147e0 con 0x55e216ade1a0 2024-04-16T17:16:22.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.507+0000 7f3e9fe00640 1 --2- 172.21.15.45:0/2756277321 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e216ad55c0 0x55e216adb770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:16:22.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.507+0000 7f3ea5600640 1 --2- 172.21.15.45:0/2756277321 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e216ade1a0 0x55e216adbcb0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f3e980122f0 tx=0x7f3e98004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5145e811c3f7a52b server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:16:22.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.507+0000 7f3e86000640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-04-16T17:16:22.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.508+0000 7f3e86000640 1 -- 172.21.15.45:0/2756277321 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e980123e0 con 0x55e216ade1a0 2024-04-16T17:16:22.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.508+0000 7f3e86000640 1 -- 172.21.15.45:0/2756277321 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9496+0+0 (secure 0 0 0) 0x7f3e980119c0 con 0x55e216ade1a0 2024-04-16T17:16:22.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.508+0000 7f3eaa372740 1 -- 172.21.15.45:0/2756277321 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3e940044b0 con 0x55e216ade1a0 2024-04-16T17:16:22.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.508+0000 7f3e86000640 1 -- 172.21.15.45:0/2756277321 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f3e98011bc0 con 0x55e216ade1a0 2024-04-16T17:16:22.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.508+0000 7f3eaa372740 10 client.28115 fetch_fsmap learned FSMap version 44 2024-04-16T17:16:22.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.508+0000 7f3eaa372740 1 -- 172.21.15.45:0/2756277321 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55e216874f70 con 0x55e216ade1a0 2024-04-16T17:16:22.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.508+0000 7f3e86000640 1 -- 172.21.15.45:0/2756277321 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3e940044b0 con 0x55e216ade1a0 2024-04-16T17:16:22.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.508+0000 7f3e86000640 1 -- 172.21.15.45:0/2756277321 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 44) ==== 8346+0+0 (secure 0 0 0) 0x7f3e98027070 con 0x55e216ade1a0 2024-04-16T17:16:22.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.508+0000 7f3eaa372740 10 client.28115 fetch_fsmap finished waiting for FSMap version 44 2024-04-16T17:16:22.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.508+0000 7f3eaa372740 10 client.28115 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:16:22.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.509+0000 7f3eaa372740 20 client.28115 populate_metadata read hostname 'smithi045' 2024-04-16T17:16:22.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.509+0000 7f3eaa372740 1 --2- 172.21.15.45:0/2756277321 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55e216aee660 0x55e216af0ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:22.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.509+0000 7f3eaa372740 4 client.28115 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:16:22.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.509+0000 7f3eaa372740 1 -- 172.21.15.45:0/2756277321 --> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e216a476d0 con 0x55e216aee660 2024-04-16T17:16:22.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.509+0000 7f3ea4c00640 1 --2- 172.21.15.45:0/2756277321 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55e216aee660 0x55e216af0ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:22.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.509+0000 7f3ea4c00640 1 --2- 172.21.15.45:0/2756277321 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55e216aee660 0x55e216af0ac0 crc :-1 s=READY pgs=245 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4b80f61e5139d2b7 server_cookie=bdfb03f15d96885f in_seq=0 out_seq=0 2024-04-16T17:16:22.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.510+0000 7f3e86000640 10 client.28115 ms_handle_connect on v2:172.21.15.104:6838/3660070592 2024-04-16T17:16:22.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.510+0000 7f3e86000640 1 -- 172.21.15.45:0/2756277321 <== mds.0 v2:172.21.15.104:6838/3660070592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e216a476d0 con 0x55e216aee660 2024-04-16T17:16:22.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.510+0000 7f3e86000640 10 client.28115 handle_command_reply: tid=0 2024-04-16T17:16:22.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.510+0000 7f3eaa372740 1 client.28115 shutdown 2024-04-16T17:16:22.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.511+0000 7f3eaa372740 1 -- 172.21.15.45:0/2756277321 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e216ade1a0 msgr2=0x55e216adbcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:22.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.511+0000 7f3eaa372740 1 --2- 172.21.15.45:0/2756277321 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e216ade1a0 0x55e216adbcb0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f3e980122f0 tx=0x7f3e98004ab0 comp rx=0 tx=0).stop 2024-04-16T17:16:22.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.511+0000 7f3eaa372740 1 -- 172.21.15.45:0/2756277321 shutdown_connections 2024-04-16T17:16:22.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.511+0000 7f3eaa372740 1 --2- 172.21.15.45:0/2756277321 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55e216aee660 0x55e216af0ac0 crc :-1 s=READY pgs=245 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:22.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.511+0000 7f3eaa372740 1 --2- 172.21.15.45:0/2756277321 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e216ad55c0 0x55e216adb770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:22.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.511+0000 7f3eaa372740 1 --2- 172.21.15.45:0/2756277321 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e216ade1a0 0x55e216adbcb0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:22.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.511+0000 7f3eaa372740 1 --2- 172.21.15.45:0/2756277321 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e216ad4ca0 0x55e216adb230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:22.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.511+0000 7f3e86000640 0 client.28115 ms_handle_reset on v2:172.21.15.104:6838/3660070592 2024-04-16T17:16:22.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.511+0000 7f3eaa372740 1 -- 172.21.15.45:0/2756277321 >> 172.21.15.45:0/2756277321 conn(0x55e216acf3d0 msgr2=0x55e216ad1120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:16:22.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.511+0000 7f3eaa372740 1 -- 172.21.15.45:0/2756277321 shutdown_connections 2024-04-16T17:16:22.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.511+0000 7f3eaa372740 1 -- 172.21.15.45:0/2756277321 wait complete. 2024-04-16T17:16:22.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.511+0000 7f3eaa372740 20 client.28115 trim_cache size 0 max 16384 2024-04-16T17:16:22.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.530+0000 7f3eaa372740 1 Processor -- start 2024-04-16T17:16:22.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.531+0000 7f3eaa372740 1 -- start start 2024-04-16T17:16:22.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.531+0000 7f3eaa372740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e216ad4ca0 0x55e216add960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:22.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.531+0000 7f3eaa372740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e216ad55c0 0x55e216ae9e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:22.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.531+0000 7f3eaa372740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e216ade1a0 0x55e216af6390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:22.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.531+0000 7f3eaa372740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f3ea01b8440 con 0x55e216ad4ca0 2024-04-16T17:16:22.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.531+0000 7f3eaa372740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f3ea01b5d10 con 0x55e216ad55c0 2024-04-16T17:16:22.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.531+0000 7f3eaa372740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f3ea01b3b80 con 0x55e216ade1a0 2024-04-16T17:16:22.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.531+0000 7f3ea4c00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e216ad4ca0 0x55e216add960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:22.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.531+0000 7f3ea4c00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e216ad4ca0 0x55e216add960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59236/0 (socket says 172.21.15.45:59236) 2024-04-16T17:16:22.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.531+0000 7f3ea4c00640 1 -- 172.21.15.45:0/89051290 learned_addr learned my addr 172.21.15.45:0/89051290 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:16:22.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.531+0000 7f3ea5600640 1 --2- 172.21.15.45:0/89051290 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e216ade1a0 0x55e216af6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:22.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.531+0000 7f3e9fe00640 1 --2- 172.21.15.45:0/89051290 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e216ad55c0 0x55e216ae9e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:22.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.532+0000 7f3ea5600640 1 -- 172.21.15.45:0/89051290 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e216ad55c0 msgr2=0x55e216ae9e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:16:22.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.532+0000 7f3ea5600640 1 --2- 172.21.15.45:0/89051290 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e216ad55c0 0x55e216ae9e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:22.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.532+0000 7f3ea5600640 1 -- 172.21.15.45:0/89051290 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e216ad4ca0 msgr2=0x55e216add960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:16:22.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.532+0000 7f3ea5600640 1 --2- 172.21.15.45:0/89051290 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e216ad4ca0 0x55e216add960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:22.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.532+0000 7f3ea5600640 1 -- 172.21.15.45:0/89051290 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e216ad6e50 con 0x55e216ade1a0 2024-04-16T17:16:22.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.532+0000 7f3e9fe00640 1 --2- 172.21.15.45:0/89051290 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e216ad55c0 0x55e216ae9e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:16:22.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.532+0000 7f3ea5600640 1 --2- 172.21.15.45:0/89051290 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e216ade1a0 0x55e216af6390 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f3e9801f470 tx=0x7f3e98005bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b881fc8fd9d37c13 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:16:22.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.532+0000 7f3e91200640 1 -- 172.21.15.45:0/89051290 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e98011a10 con 0x55e216ade1a0 2024-04-16T17:16:22.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.532+0000 7f3e91200640 1 -- 172.21.15.45:0/89051290 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3e98003e50 con 0x55e216ade1a0 2024-04-16T17:16:22.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.532+0000 7f3e91200640 1 -- 172.21.15.45:0/89051290 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e9801f660 con 0x55e216ade1a0 2024-04-16T17:16:22.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.533+0000 7f3eaa372740 1 -- 172.21.15.45:0/89051290 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e216ade1a0 msgr2=0x55e216af6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:22.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.533+0000 7f3eaa372740 1 --2- 172.21.15.45:0/89051290 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e216ade1a0 0x55e216af6390 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f3e9801f470 tx=0x7f3e98005bd0 comp rx=0 tx=0).stop 2024-04-16T17:16:22.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.533+0000 7f3eaa372740 1 -- 172.21.15.45:0/89051290 shutdown_connections 2024-04-16T17:16:22.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.533+0000 7f3eaa372740 1 --2- 172.21.15.45:0/89051290 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e216ade1a0 0x55e216af6390 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:22.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.533+0000 7f3eaa372740 1 --2- 172.21.15.45:0/89051290 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e216ad55c0 0x55e216ae9e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:22.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.533+0000 7f3eaa372740 1 --2- 172.21.15.45:0/89051290 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e216ad4ca0 0x55e216add960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:22.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.533+0000 7f3eaa372740 1 -- 172.21.15.45:0/89051290 >> 172.21.15.45:0/89051290 conn(0x55e216acf3d0 msgr2=0x55e216ad1090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:16:22.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.533+0000 7f3eaa372740 1 -- 172.21.15.45:0/89051290 shutdown_connections 2024-04-16T17:16:22.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.533+0000 7f3eaa372740 1 -- 172.21.15.45:0/89051290 wait complete. 2024-04-16T17:16:22.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.533+0000 7f3eaa372740 1 -- start start 2024-04-16T17:16:22.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.534+0000 7f3eaa372740 1 Processor -- start 2024-04-16T17:16:22.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.534+0000 7f3eaa372740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e216ad4ca0 0x55e216ae6390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:22.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.534+0000 7f3eaa372740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e216ad55c0 0x55e216ae68d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:22.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.534+0000 7f3eaa372740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e216ade1a0 0x55e216ae3360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:22.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.534+0000 7f3eaa372740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f3ea01b41d0 con 0x55e216ade1a0 2024-04-16T17:16:22.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.534+0000 7f3eaa372740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f3ea01b5640 con 0x55e216ad4ca0 2024-04-16T17:16:22.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.534+0000 7f3eaa372740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f3ea01b48a0 con 0x55e216ad55c0 2024-04-16T17:16:22.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.534+0000 7f3eaa372740 10 client.0 _ll_register_callbacks cb 0x7f3e980121b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:16:22.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.534+0000 7f3ea4c00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e216ad4ca0 0x55e216ae6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:22.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.535+0000 7f3ea4c00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e216ad4ca0 0x55e216ae6390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:42968/0 (socket says 172.21.15.45:42968) 2024-04-16T17:16:22.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.535+0000 7f3ea4c00640 1 -- 172.21.15.45:0/346008365 learned_addr learned my addr 172.21.15.45:0/346008365 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:16:22.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.535+0000 7f3ea5600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e216ade1a0 0x55e216ae3360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:22.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.535+0000 7f3e9fe00640 1 --2- 172.21.15.45:0/346008365 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e216ad55c0 0x55e216ae68d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:22.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.535+0000 7f3ea4c00640 1 -- 172.21.15.45:0/346008365 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e216ad55c0 msgr2=0x55e216ae68d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:16:22.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.535+0000 7f3ea4c00640 1 --2- 172.21.15.45:0/346008365 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e216ad55c0 0x55e216ae68d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:22.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.535+0000 7f3ea4c00640 1 -- 172.21.15.45:0/346008365 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e216ade1a0 msgr2=0x55e216ae3360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:16:22.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.535+0000 7f3ea4c00640 1 --2- 172.21.15.45:0/346008365 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e216ade1a0 0x55e216ae3360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:22.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.535+0000 7f3ea4c00640 1 -- 172.21.15.45:0/346008365 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e21670c170 con 0x55e216ad4ca0 2024-04-16T17:16:22.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.535+0000 7f3ea4c00640 1 --2- 172.21.15.45:0/346008365 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e216ad4ca0 0x55e216ae6390 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f3e94011dd0 tx=0x7f3e94005850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=439a626f8b00ceac server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:16:22.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.535+0000 7f3e86000640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-04-16T17:16:22.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.536+0000 7f3e86000640 1 -- 172.21.15.45:0/346008365 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e9402ede0 con 0x55e216ad4ca0 2024-04-16T17:16:22.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.536+0000 7f3e86000640 1 -- 172.21.15.45:0/346008365 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9496+0+0 (secure 0 0 0) 0x7f3e94005040 con 0x55e216ad4ca0 2024-04-16T17:16:22.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.536+0000 7f3eaa372740 1 -- 172.21.15.45:0/346008365 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3e98003e50 con 0x55e216ad4ca0 2024-04-16T17:16:22.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.536+0000 7f3e86000640 1 -- 172.21.15.45:0/346008365 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f3e9403d050 con 0x55e216ad4ca0 2024-04-16T17:16:22.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.536+0000 7f3eaa372740 10 client.28121 fetch_fsmap learned FSMap version 44 2024-04-16T17:16:22.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.536+0000 7f3eaa372740 1 -- 172.21.15.45:0/346008365 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55e216b55370 con 0x55e216ad4ca0 2024-04-16T17:16:22.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.536+0000 7f3e86000640 1 -- 172.21.15.45:0/346008365 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3e98003e50 con 0x55e216ad4ca0 2024-04-16T17:16:22.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.536+0000 7f3e86000640 1 -- 172.21.15.45:0/346008365 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 44) ==== 8346+0+0 (secure 0 0 0) 0x7f3e940035f0 con 0x55e216ad4ca0 2024-04-16T17:16:22.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.536+0000 7f3eaa372740 10 client.28121 fetch_fsmap finished waiting for FSMap version 44 2024-04-16T17:16:22.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.536+0000 7f3eaa372740 10 client.28121 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:16:22.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.536+0000 7f3eaa372740 20 client.28121 populate_metadata read hostname 'smithi045' 2024-04-16T17:16:22.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.537+0000 7f3eaa372740 1 --2- 172.21.15.45:0/346008365 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55e216b55950 0x55e216b57d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:22.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.537+0000 7f3eaa372740 4 client.28121 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-16T17:16:22.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.537+0000 7f3eaa372740 1 -- 172.21.15.45:0/346008365 --> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55e216a41540 con 0x55e216b55950 2024-04-16T17:16:22.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.537+0000 7f3e9fe00640 1 --2- 172.21.15.45:0/346008365 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55e216b55950 0x55e216b57d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:22.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.537+0000 7f3e9fe00640 1 --2- 172.21.15.45:0/346008365 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55e216b55950 0x55e216b57d70 crc :-1 s=READY pgs=246 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ea89985534bd4a48 server_cookie=c68125b4f7a3e16d in_seq=0 out_seq=0 2024-04-16T17:16:22.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.537+0000 7f3e86000640 10 client.28121 ms_handle_connect on v2:172.21.15.104:6838/3660070592 2024-04-16T17:16:22.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.539+0000 7f3e86000640 1 -- 172.21.15.45:0/346008365 <== mds.0 v2:172.21.15.104:6838/3660070592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55e216a41540 con 0x55e216b55950 2024-04-16T17:16:22.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.539+0000 7f3e86000640 10 client.28121 handle_command_reply: tid=0 2024-04-16T17:16:22.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.539+0000 7f3eaa372740 1 client.28121 shutdown 2024-04-16T17:16:22.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.539+0000 7f3eaa372740 1 -- 172.21.15.45:0/346008365 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e216ad4ca0 msgr2=0x55e216ae6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:22.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.539+0000 7f3eaa372740 1 --2- 172.21.15.45:0/346008365 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e216ad4ca0 0x55e216ae6390 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f3e94011dd0 tx=0x7f3e94005850 comp rx=0 tx=0).stop 2024-04-16T17:16:22.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.539+0000 7f3eaa372740 1 -- 172.21.15.45:0/346008365 shutdown_connections 2024-04-16T17:16:22.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.539+0000 7f3eaa372740 1 --2- 172.21.15.45:0/346008365 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55e216b55950 0x55e216b57d70 crc :-1 s=READY pgs=246 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:22.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.539+0000 7f3eaa372740 1 --2- 172.21.15.45:0/346008365 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e216ad55c0 0x55e216ae68d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:22.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.539+0000 7f3eaa372740 1 --2- 172.21.15.45:0/346008365 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e216ade1a0 0x55e216ae3360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:22.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.539+0000 7f3e86000640 0 client.28121 ms_handle_reset on v2:172.21.15.104:6838/3660070592 2024-04-16T17:16:22.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.539+0000 7f3eaa372740 1 --2- 172.21.15.45:0/346008365 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e216ad4ca0 0x55e216ae6390 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:22.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.539+0000 7f3eaa372740 1 -- 172.21.15.45:0/346008365 >> 172.21.15.45:0/346008365 conn(0x55e216acf3d0 msgr2=0x55e216acdba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:16:22.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.539+0000 7f3eaa372740 1 -- 172.21.15.45:0/346008365 shutdown_connections 2024-04-16T17:16:22.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.539+0000 7f3eaa372740 1 -- 172.21.15.45:0/346008365 wait complete. 2024-04-16T17:16:22.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.539+0000 7f3eaa372740 20 client.28121 trim_cache size 0 max 16384 2024-04-16T17:16:22.545 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-16T17:16:22.545 INFO:teuthology.orchestra.run.smithi045.stdout: "return_code": 0, 2024-04-16T17:16:22.545 INFO:teuthology.orchestra.run.smithi045.stdout: "scrub_tag": "2e4db62b-8dee-4c44-9204-a20321eae1b1", 2024-04-16T17:16:22.546 INFO:teuthology.orchestra.run.smithi045.stdout: "mode": "asynchronous" 2024-04-16T17:16:22.546 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-16T17:16:22.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.540+0000 7f3ea7600640 1 -- 172.21.15.45:0/584683857 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f3e880a3d20 msgr2=0x7f3e880a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:22.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.540+0000 7f3ea7600640 1 --2- 172.21.15.45:0/584683857 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f3e880a3d20 0x7f3e880a61e0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f3e98012780 tx=0x7f3e98009500 comp rx=0 tx=0).stop 2024-04-16T17:16:22.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.540+0000 7f3ea7600640 1 -- 172.21.15.45:0/584683857 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3ea0076e50 msgr2=0x7f3ea019f580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:22.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.540+0000 7f3ea7600640 1 --2- 172.21.15.45:0/584683857 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3ea0076e50 0x7f3ea019f580 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f3e8c009430 tx=0x7f3e8c02cb10 comp rx=0 tx=0).stop 2024-04-16T17:16:22.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.540+0000 7f3ea7600640 1 -- 172.21.15.45:0/584683857 shutdown_connections 2024-04-16T17:16:22.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.540+0000 7f3ea7600640 1 --2- 172.21.15.45:0/584683857 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f3e880a3d20 0x7f3e880a61e0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:22.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.540+0000 7f3ea7600640 1 --2- 172.21.15.45:0/584683857 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3ea0112ee0 0x7f3ea01a38b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:22.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.540+0000 7f3ea7600640 1 --2- 172.21.15.45:0/584683857 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3ea0076e50 0x7f3ea019f580 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:22.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.540+0000 7f3ea7600640 1 --2- 172.21.15.45:0/584683857 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3ea0076530 0x7f3ea019f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:22.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.540+0000 7f3ea7600640 1 -- 172.21.15.45:0/584683857 >> 172.21.15.45:0/584683857 conn(0x7f3ea00ffd90 msgr2=0x7f3ea01145a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:16:22.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.540+0000 7f3ea7600640 1 -- 172.21.15.45:0/584683857 shutdown_connections 2024-04-16T17:16:22.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.541+0000 7f3ea7600640 1 -- 172.21.15.45:0/584683857 wait complete. 2024-04-16T17:16:22.563 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-16T17:16:22.563 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-16T17:16:22.641 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:16:22 smithi045 ceph-mon[31676]: pgmap v300: 129 pgs: 129 active+clean; 56 MiB data, 870 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-16T17:16:22.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.783+0000 7fb373e00640 1 -- 172.21.15.45:0/1026341251 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb374104cd0 msgr2=0x7fb374108d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:22.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.783+0000 7fb373e00640 1 --2- 172.21.15.45:0/1026341251 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb374104cd0 0x7fb374108d10 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fb360009900 tx=0x7fb36002bc50 comp rx=0 tx=0).stop 2024-04-16T17:16:22.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.783+0000 7fb373e00640 1 -- 172.21.15.45:0/1026341251 shutdown_connections 2024-04-16T17:16:22.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.783+0000 7fb373e00640 1 --2- 172.21.15.45:0/1026341251 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb3741093f0 0x7fb374110fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:22.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.783+0000 7fb373e00640 1 --2- 172.21.15.45:0/1026341251 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb374104cd0 0x7fb374108d10 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:22.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.783+0000 7fb373e00640 1 --2- 172.21.15.45:0/1026341251 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb374104320 0x7fb374104700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:22.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.783+0000 7fb373e00640 1 -- 172.21.15.45:0/1026341251 >> 172.21.15.45:0/1026341251 conn(0x7fb3740ffb40 msgr2=0x7fb374101f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:16:22.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.783+0000 7fb373e00640 1 -- 172.21.15.45:0/1026341251 shutdown_connections 2024-04-16T17:16:22.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.783+0000 7fb373e00640 1 -- 172.21.15.45:0/1026341251 wait complete. 2024-04-16T17:16:22.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.784+0000 7fb373e00640 1 Processor -- start 2024-04-16T17:16:22.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.784+0000 7fb373e00640 1 -- start start 2024-04-16T17:16:22.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.785+0000 7fb373e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb374104320 0x7fb3741a0710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:22.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.785+0000 7fb373e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb3741093f0 0x7fb3741a0c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:22.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.785+0000 7fb373e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb37419a8c0 0x7fb37419ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:22.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.785+0000 7fb373e00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fb374114480 con 0x7fb374104320 2024-04-16T17:16:22.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.785+0000 7fb373e00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fb374114300 con 0x7fb3741093f0 2024-04-16T17:16:22.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.785+0000 7fb373e00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb374114600 con 0x7fb37419a8c0 2024-04-16T17:16:22.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.785+0000 7fb372a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb374104320 0x7fb3741a0710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:22.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.785+0000 7fb372000640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb3741093f0 0x7fb3741a0c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:22.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.785+0000 7fb372000640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb3741093f0 0x7fb3741a0c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:42986/0 (socket says 172.21.15.45:42986) 2024-04-16T17:16:22.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.785+0000 7fb372000640 1 -- 172.21.15.45:0/1811317789 learned_addr learned my addr 172.21.15.45:0/1811317789 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:16:22.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.785+0000 7fb373400640 1 --2- 172.21.15.45:0/1811317789 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb37419a8c0 0x7fb37419ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:22.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.786+0000 7fb372000640 1 -- 172.21.15.45:0/1811317789 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb37419a8c0 msgr2=0x7fb37419ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:22.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.786+0000 7fb372000640 1 --2- 172.21.15.45:0/1811317789 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb37419a8c0 0x7fb37419ad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:22.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.786+0000 7fb372000640 1 -- 172.21.15.45:0/1811317789 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb374104320 msgr2=0x7fb3741a0710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:22.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.786+0000 7fb372000640 1 --2- 172.21.15.45:0/1811317789 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb374104320 0x7fb3741a0710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:22.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.786+0000 7fb372000640 1 -- 172.21.15.45:0/1811317789 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb37419b3e0 con 0x7fb3741093f0 2024-04-16T17:16:22.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.786+0000 7fb372000640 1 --2- 172.21.15.45:0/1811317789 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb3741093f0 0x7fb3741a0c50 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fb360036040 tx=0x7fb36002cb40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:16:22.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.786+0000 7fb36b400640 1 -- 172.21.15.45:0/1811317789 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb360042020 con 0x7fb3741093f0 2024-04-16T17:16:22.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.786+0000 7fb36b400640 1 -- 172.21.15.45:0/1811317789 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb360004450 con 0x7fb3741093f0 2024-04-16T17:16:22.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.786+0000 7fb36b400640 1 -- 172.21.15.45:0/1811317789 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb360003d00 con 0x7fb3741093f0 2024-04-16T17:16:22.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.786+0000 7fb373e00640 1 -- 172.21.15.45:0/1811317789 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb37419b670 con 0x7fb3741093f0 2024-04-16T17:16:22.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.786+0000 7fb373e00640 1 -- 172.21.15.45:0/1811317789 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3741a7870 con 0x7fb3741093f0 2024-04-16T17:16:22.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.790+0000 7fb36b400640 1 -- 172.21.15.45:0/1811317789 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fb360045020 con 0x7fb3741093f0 2024-04-16T17:16:22.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.790+0000 7fb37c5da740 1 Processor -- start 2024-04-16T17:16:22.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.790+0000 7fb37c5da740 1 -- start start 2024-04-16T17:16:22.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.790+0000 7fb37c5da740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55df3fe65390 0x55df3fe65770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:22.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.790+0000 7fb37c5da740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55df3fe65cb0 0x55df3fe6e350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:22.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.790+0000 7fb37c5da740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55df3fe6e890 0x55df3fe70c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:22.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.790+0000 7fb37c5da740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55df3fdcb120 con 0x55df3fe65390 2024-04-16T17:16:22.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.790+0000 7fb37c5da740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55df3fd71340 con 0x55df3fe65cb0 2024-04-16T17:16:22.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.790+0000 7fb37c5da740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55df3fdc70e0 con 0x55df3fe6e890 2024-04-16T17:16:22.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.790+0000 7fb373400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55df3fe65390 0x55df3fe65770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:22.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.791+0000 7fb373400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55df3fe65390 0x55df3fe65770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59280/0 (socket says 172.21.15.45:59280) 2024-04-16T17:16:22.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.791+0000 7fb372a00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55df3fe65cb0 0x55df3fe6e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:22.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.791+0000 7fb373400640 1 -- 172.21.15.45:0/4196573999 learned_addr learned my addr 172.21.15.45:0/4196573999 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:16:22.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.791+0000 7fb372000640 1 --2- 172.21.15.45:0/4196573999 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55df3fe6e890 0x55df3fe70c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:22.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.791+0000 7fb373400640 1 -- 172.21.15.45:0/4196573999 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55df3fe6e890 msgr2=0x55df3fe70c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:16:22.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.791+0000 7fb373400640 1 --2- 172.21.15.45:0/4196573999 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55df3fe6e890 0x55df3fe70c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:22.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.791+0000 7fb373400640 1 -- 172.21.15.45:0/4196573999 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55df3fe65cb0 msgr2=0x55df3fe6e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:16:22.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.791+0000 7fb373400640 1 --2- 172.21.15.45:0/4196573999 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55df3fe65cb0 0x55df3fe6e350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:22.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.791+0000 7fb373400640 1 -- 172.21.15.45:0/4196573999 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55df3fba77d0 con 0x55df3fe65390 2024-04-16T17:16:22.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.791+0000 7fb372a00640 1 --2- 172.21.15.45:0/4196573999 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55df3fe65cb0 0x55df3fe6e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:16:22.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.791+0000 7fb373400640 1 --2- 172.21.15.45:0/4196573999 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55df3fe65390 0x55df3fe65770 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fb36c009f60 tx=0x7fb36c009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43f571dc3e37c9e6 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:16:22.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.792+0000 7fb36b400640 1 --2- 172.21.15.45:0/1811317789 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fb3580a3d20 0x7fb3580a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:22.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.792+0000 7fb35fe00640 1 -- 172.21.15.45:0/4196573999 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb36c01e070 con 0x55df3fe65390 2024-04-16T17:16:22.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.792+0000 7fb35fe00640 1 -- 172.21.15.45:0/4196573999 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb36c003c30 con 0x55df3fe65390 2024-04-16T17:16:22.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.792+0000 7fb35fe00640 1 -- 172.21.15.45:0/4196573999 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb36c00f980 con 0x55df3fe65390 2024-04-16T17:16:22.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.792+0000 7fb36b400640 1 -- 172.21.15.45:0/1811317789 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 9496+0+0 (secure 0 0 0) 0x7fb3600f29e0 con 0x7fb3741093f0 2024-04-16T17:16:22.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.792+0000 7fb372a00640 1 --2- 172.21.15.45:0/1811317789 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fb3580a3d20 0x7fb3580a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:22.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.792+0000 7fb37c5da740 1 -- 172.21.15.45:0/4196573999 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55df3fe65390 msgr2=0x55df3fe65770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:22.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.792+0000 7fb37c5da740 1 --2- 172.21.15.45:0/4196573999 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55df3fe65390 0x55df3fe65770 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fb36c009f60 tx=0x7fb36c009f90 comp rx=0 tx=0).stop 2024-04-16T17:16:22.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.793+0000 7fb37c5da740 1 -- 172.21.15.45:0/4196573999 shutdown_connections 2024-04-16T17:16:22.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.793+0000 7fb37c5da740 1 --2- 172.21.15.45:0/4196573999 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55df3fe6e890 0x55df3fe70c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:22.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.793+0000 7fb37c5da740 1 --2- 172.21.15.45:0/4196573999 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55df3fe65cb0 0x55df3fe6e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:22.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.793+0000 7fb37c5da740 1 --2- 172.21.15.45:0/4196573999 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55df3fe65390 0x55df3fe65770 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:22.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.793+0000 7fb37c5da740 1 -- 172.21.15.45:0/4196573999 >> 172.21.15.45:0/4196573999 conn(0x55df3fe5fac0 msgr2=0x55df3fe5fea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:16:22.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.793+0000 7fb37c5da740 1 -- 172.21.15.45:0/4196573999 shutdown_connections 2024-04-16T17:16:22.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.793+0000 7fb372a00640 1 --2- 172.21.15.45:0/1811317789 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fb3580a3d20 0x7fb3580a61e0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x55df3fe64e60 tx=0x7fb364005f90 comp rx=0 tx=0).ready entity=mgr.25411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:16:22.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.793+0000 7fb37c5da740 1 -- 172.21.15.45:0/4196573999 wait complete. 2024-04-16T17:16:22.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.793+0000 7fb37c5da740 1 -- start start 2024-04-16T17:16:22.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.793+0000 7fb37c5da740 1 Processor -- start 2024-04-16T17:16:22.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.794+0000 7fb37c5da740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55df3fe65390 0x55df3fe6b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:22.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.794+0000 7fb37c5da740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55df3fe65cb0 0x55df3fe6bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:22.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.794+0000 7fb37c5da740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55df3fe6e890 0x55df3fe6c420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:22.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.794+0000 7fb37c5da740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55df3fe71680 con 0x55df3fe65cb0 2024-04-16T17:16:22.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.794+0000 7fb37c5da740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55df3fd84710 con 0x55df3fe6e890 2024-04-16T17:16:22.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.794+0000 7fb37c5da740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55df3fe71800 con 0x55df3fe65390 2024-04-16T17:16:22.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.794+0000 7fb37c5da740 10 client.0 _ll_register_callbacks cb 0x55df3fd7faf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:16:22.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.794+0000 7fb372000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55df3fe65cb0 0x55df3fe6bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:22.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.794+0000 7fb372a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55df3fe65390 0x55df3fe6b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:22.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.794+0000 7fb372000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55df3fe65cb0 0x55df3fe6bee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59284/0 (socket says 172.21.15.45:59284) 2024-04-16T17:16:22.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.794+0000 7fb372000640 1 -- 172.21.15.45:0/2782310346 learned_addr learned my addr 172.21.15.45:0/2782310346 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:16:22.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.794+0000 7fb373400640 1 --2- 172.21.15.45:0/2782310346 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55df3fe6e890 0x55df3fe6c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:22.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.795+0000 7fb372000640 1 -- 172.21.15.45:0/2782310346 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55df3fe65390 msgr2=0x55df3fe6b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:16:22.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.795+0000 7fb372000640 1 --2- 172.21.15.45:0/2782310346 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55df3fe65390 0x55df3fe6b9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:22.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.795+0000 7fb372000640 1 -- 172.21.15.45:0/2782310346 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55df3fe6e890 msgr2=0x55df3fe6c420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:16:22.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.795+0000 7fb372000640 1 --2- 172.21.15.45:0/2782310346 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55df3fe6e890 0x55df3fe6c420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:22.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.795+0000 7fb372000640 1 -- 172.21.15.45:0/2782310346 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55df3fadd3f0 con 0x55df3fe65cb0 2024-04-16T17:16:22.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.795+0000 7fb372000640 1 --2- 172.21.15.45:0/2782310346 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55df3fe65cb0 0x55df3fe6bee0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fb360000c00 tx=0x7fb360005c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67bec8269d93414a server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:16:22.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.795+0000 7fb35cc00640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-16T17:16:22.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.795+0000 7fb35cc00640 1 -- 172.21.15.45:0/2782310346 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3600ba960 con 0x55df3fe65cb0 2024-04-16T17:16:22.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.795+0000 7fb35cc00640 1 -- 172.21.15.45:0/2782310346 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9496+0+0 (secure 0 0 0) 0x7fb3600c70a0 con 0x55df3fe65cb0 2024-04-16T17:16:22.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.795+0000 7fb37c5da740 1 -- 172.21.15.45:0/2782310346 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb36c003c30 con 0x55df3fe65cb0 2024-04-16T17:16:22.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.796+0000 7fb35cc00640 1 -- 172.21.15.45:0/2782310346 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fb3600be020 con 0x55df3fe65cb0 2024-04-16T17:16:22.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.796+0000 7fb37c5da740 10 client.17982 fetch_fsmap learned FSMap version 44 2024-04-16T17:16:22.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.796+0000 7fb37c5da740 1 -- 172.21.15.45:0/2782310346 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55df3fa9d0f0 con 0x55df3fe65cb0 2024-04-16T17:16:22.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.796+0000 7fb35cc00640 1 -- 172.21.15.45:0/2782310346 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb36c003c30 con 0x55df3fe65cb0 2024-04-16T17:16:22.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.796+0000 7fb35cc00640 1 -- 172.21.15.45:0/2782310346 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 44) ==== 8346+0+0 (secure 0 0 0) 0x7fb3600bdb10 con 0x55df3fe65cb0 2024-04-16T17:16:22.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.796+0000 7fb37c5da740 10 client.17982 fetch_fsmap finished waiting for FSMap version 44 2024-04-16T17:16:22.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.796+0000 7fb37c5da740 10 client.17982 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:16:22.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.796+0000 7fb37c5da740 20 client.17982 populate_metadata read hostname 'smithi045' 2024-04-16T17:16:22.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.796+0000 7fb37c5da740 1 --2- 172.21.15.45:0/2782310346 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55df3fe7ec40 0x55df3fe810c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:22.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.796+0000 7fb37c5da740 4 client.17982 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:16:22.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.796+0000 7fb37c5da740 1 -- 172.21.15.45:0/2782310346 --> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55df3fdd8b40 con 0x55df3fe7ec40 2024-04-16T17:16:22.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.797+0000 7fb372a00640 1 --2- 172.21.15.45:0/2782310346 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55df3fe7ec40 0x55df3fe810c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:22.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.797+0000 7fb372a00640 1 --2- 172.21.15.45:0/2782310346 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55df3fe7ec40 0x55df3fe810c0 crc :-1 s=READY pgs=247 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f69b3a1bb79eef30 server_cookie=96c5a7a3b9edc79b in_seq=0 out_seq=0 2024-04-16T17:16:22.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.797+0000 7fb35cc00640 10 client.17982 ms_handle_connect on v2:172.21.15.104:6838/3660070592 2024-04-16T17:16:22.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.798+0000 7fb35cc00640 1 -- 172.21.15.45:0/2782310346 <== mds.0 v2:172.21.15.104:6838/3660070592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55df3fdd8b40 con 0x55df3fe7ec40 2024-04-16T17:16:22.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.798+0000 7fb35cc00640 10 client.17982 handle_command_reply: tid=0 2024-04-16T17:16:22.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.798+0000 7fb37c5da740 1 client.17982 shutdown 2024-04-16T17:16:22.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.798+0000 7fb37c5da740 1 -- 172.21.15.45:0/2782310346 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55df3fe65cb0 msgr2=0x55df3fe6bee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:22.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.798+0000 7fb37c5da740 1 --2- 172.21.15.45:0/2782310346 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55df3fe65cb0 0x55df3fe6bee0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fb360000c00 tx=0x7fb360005c90 comp rx=0 tx=0).stop 2024-04-16T17:16:22.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.799+0000 7fb37c5da740 1 -- 172.21.15.45:0/2782310346 shutdown_connections 2024-04-16T17:16:22.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.799+0000 7fb37c5da740 1 --2- 172.21.15.45:0/2782310346 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55df3fe6e890 0x55df3fe6c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:22.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.799+0000 7fb37c5da740 1 --2- 172.21.15.45:0/2782310346 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55df3fe65cb0 0x55df3fe6bee0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:22.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.799+0000 7fb37c5da740 1 --2- 172.21.15.45:0/2782310346 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55df3fe7ec40 0x55df3fe810c0 crc :-1 s=READY pgs=247 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:22.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.799+0000 7fb37c5da740 1 --2- 172.21.15.45:0/2782310346 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55df3fe65390 0x55df3fe6b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:22.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.799+0000 7fb37c5da740 1 -- 172.21.15.45:0/2782310346 >> 172.21.15.45:0/2782310346 conn(0x55df3fe5fac0 msgr2=0x55df3fe61810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:16:22.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.799+0000 7fb35cc00640 0 client.17982 ms_handle_reset on v2:172.21.15.104:6838/3660070592 2024-04-16T17:16:22.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.799+0000 7fb37c5da740 1 -- 172.21.15.45:0/2782310346 shutdown_connections 2024-04-16T17:16:22.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.799+0000 7fb37c5da740 1 -- 172.21.15.45:0/2782310346 wait complete. 2024-04-16T17:16:22.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.799+0000 7fb37c5da740 20 client.17982 trim_cache size 0 max 16384 2024-04-16T17:16:22.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:16:22 smithi117 ceph-mon[36730]: pgmap v300: 129 pgs: 129 active+clean; 56 MiB data, 870 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-16T17:16:22.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.818+0000 7fb37c5da740 1 Processor -- start 2024-04-16T17:16:22.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.818+0000 7fb37c5da740 1 -- start start 2024-04-16T17:16:22.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.819+0000 7fb37c5da740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55df3fe65390 0x55df3fe7a540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:22.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.819+0000 7fb37c5da740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55df3fe65cb0 0x55df3fe6e060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:22.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.819+0000 7fb37c5da740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55df3fe6e890 0x55df3fe80c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:22.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.819+0000 7fb37c5da740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fb3741b3f30 con 0x55df3fe6e890 2024-04-16T17:16:22.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.819+0000 7fb37c5da740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fb3741b1800 con 0x55df3fe65390 2024-04-16T17:16:22.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.819+0000 7fb37c5da740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb3741af670 con 0x55df3fe65cb0 2024-04-16T17:16:22.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.819+0000 7fb372000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55df3fe65cb0 0x55df3fe6e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:22.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.819+0000 7fb372000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55df3fe65cb0 0x55df3fe6e060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:42174/0 (socket says 172.21.15.45:42174) 2024-04-16T17:16:22.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.819+0000 7fb372000640 1 -- 172.21.15.45:0/1937615322 learned_addr learned my addr 172.21.15.45:0/1937615322 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:16:22.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.819+0000 7fb373400640 1 --2- 172.21.15.45:0/1937615322 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55df3fe6e890 0x55df3fe80c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:22.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.819+0000 7fb372a00640 1 --2- 172.21.15.45:0/1937615322 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55df3fe65390 0x55df3fe7a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:22.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.820+0000 7fb372000640 1 -- 172.21.15.45:0/1937615322 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55df3fe65390 msgr2=0x55df3fe7a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:16:22.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.820+0000 7fb372000640 1 --2- 172.21.15.45:0/1937615322 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55df3fe65390 0x55df3fe7a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:22.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.820+0000 7fb372000640 1 -- 172.21.15.45:0/1937615322 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55df3fe6e890 msgr2=0x55df3fe80c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:16:22.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.820+0000 7fb372000640 1 --2- 172.21.15.45:0/1937615322 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55df3fe6e890 0x55df3fe80c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:22.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.820+0000 7fb372000640 1 -- 172.21.15.45:0/1937615322 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55df3fe6a950 con 0x55df3fe65cb0 2024-04-16T17:16:22.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.820+0000 7fb372000640 1 --2- 172.21.15.45:0/1937615322 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55df3fe65cb0 0x55df3fe6e060 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fb3600096c0 tx=0x7fb360003f60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=65012287c7010184 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:16:22.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.820+0000 7fb35f400640 1 -- 172.21.15.45:0/1937615322 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3600be200 con 0x55df3fe65cb0 2024-04-16T17:16:22.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.820+0000 7fb35f400640 1 -- 172.21.15.45:0/1937615322 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb360004c80 con 0x55df3fe65cb0 2024-04-16T17:16:22.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.820+0000 7fb35f400640 1 -- 172.21.15.45:0/1937615322 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3600be490 con 0x55df3fe65cb0 2024-04-16T17:16:22.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.821+0000 7fb37c5da740 1 -- 172.21.15.45:0/1937615322 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55df3fe65cb0 msgr2=0x55df3fe6e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:22.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.821+0000 7fb37c5da740 1 --2- 172.21.15.45:0/1937615322 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55df3fe65cb0 0x55df3fe6e060 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fb3600096c0 tx=0x7fb360003f60 comp rx=0 tx=0).stop 2024-04-16T17:16:22.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.821+0000 7fb37c5da740 1 -- 172.21.15.45:0/1937615322 shutdown_connections 2024-04-16T17:16:22.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.821+0000 7fb37c5da740 1 --2- 172.21.15.45:0/1937615322 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55df3fe65cb0 0x55df3fe6e060 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:22.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.821+0000 7fb37c5da740 1 --2- 172.21.15.45:0/1937615322 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55df3fe6e890 0x55df3fe80c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:22.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.821+0000 7fb37c5da740 1 --2- 172.21.15.45:0/1937615322 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55df3fe65390 0x55df3fe7a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:22.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.821+0000 7fb37c5da740 1 -- 172.21.15.45:0/1937615322 >> 172.21.15.45:0/1937615322 conn(0x55df3fe5fac0 msgr2=0x55df3fe60ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:16:22.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.821+0000 7fb37c5da740 1 -- 172.21.15.45:0/1937615322 shutdown_connections 2024-04-16T17:16:22.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.821+0000 7fb37c5da740 1 -- 172.21.15.45:0/1937615322 wait complete. 2024-04-16T17:16:22.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.822+0000 7fb37c5da740 1 -- start start 2024-04-16T17:16:22.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.822+0000 7fb37c5da740 1 Processor -- start 2024-04-16T17:16:22.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.822+0000 7fb37c5da740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55df3fe65390 0x55df3fe727a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:22.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.822+0000 7fb37c5da740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55df3fe65cb0 0x55df3fe72ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:22.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.822+0000 7fb37c5da740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55df3fe6e890 0x55df3fee4240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:22.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.822+0000 7fb37c5da740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fb3741afcc0 con 0x55df3fe65390 2024-04-16T17:16:22.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.822+0000 7fb37c5da740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fb3741b1130 con 0x55df3fe6e890 2024-04-16T17:16:22.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.822+0000 7fb37c5da740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb3741b0390 con 0x55df3fe65cb0 2024-04-16T17:16:22.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.822+0000 7fb37c5da740 10 client.0 _ll_register_callbacks cb 0x7fb360005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:16:22.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.823+0000 7fb372a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55df3fe65390 0x55df3fe727a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:22.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.823+0000 7fb372000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55df3fe65cb0 0x55df3fe72ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:22.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.823+0000 7fb372a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55df3fe65390 0x55df3fe727a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59294/0 (socket says 172.21.15.45:59294) 2024-04-16T17:16:22.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.823+0000 7fb372a00640 1 -- 172.21.15.45:0/340660514 learned_addr learned my addr 172.21.15.45:0/340660514 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:16:22.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.823+0000 7fb372000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55df3fe65cb0 0x55df3fe72ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:42180/0 (socket says 172.21.15.45:42180) 2024-04-16T17:16:22.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.823+0000 7fb373400640 1 --2- 172.21.15.45:0/340660514 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55df3fe6e890 0x55df3fee4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:22.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.823+0000 7fb372a00640 1 -- 172.21.15.45:0/340660514 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55df3fe65cb0 msgr2=0x55df3fe72ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:16:22.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.823+0000 7fb372a00640 1 --2- 172.21.15.45:0/340660514 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55df3fe65cb0 0x55df3fe72ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:22.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.823+0000 7fb372a00640 1 -- 172.21.15.45:0/340660514 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55df3fe6e890 msgr2=0x55df3fee4240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:16:22.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.823+0000 7fb372a00640 1 --2- 172.21.15.45:0/340660514 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55df3fe6e890 0x55df3fee4240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:22.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.823+0000 7fb372a00640 1 -- 172.21.15.45:0/340660514 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55df3fe65ae0 con 0x55df3fe65390 2024-04-16T17:16:22.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.823+0000 7fb372000640 1 --2- 172.21.15.45:0/340660514 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55df3fe65cb0 0x55df3fe72ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:16:22.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.823+0000 7fb372a00640 1 --2- 172.21.15.45:0/340660514 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55df3fe65390 0x55df3fe727a0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fb364009d50 tx=0x7fb364017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5da25ccda66da431 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:16:22.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.823+0000 7fb35cc00640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-16T17:16:22.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.824+0000 7fb35cc00640 1 -- 172.21.15.45:0/340660514 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb36401db20 con 0x55df3fe65390 2024-04-16T17:16:22.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.824+0000 7fb35cc00640 1 -- 172.21.15.45:0/340660514 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9496+0+0 (secure 0 0 0) 0x7fb36401dcc0 con 0x55df3fe65390 2024-04-16T17:16:22.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.824+0000 7fb37c5da740 1 -- 172.21.15.45:0/340660514 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb360004c80 con 0x55df3fe65390 2024-04-16T17:16:22.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.824+0000 7fb35cc00640 1 -- 172.21.15.45:0/340660514 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fb36403c050 con 0x55df3fe65390 2024-04-16T17:16:22.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.824+0000 7fb37c5da740 10 client.17988 fetch_fsmap learned FSMap version 44 2024-04-16T17:16:22.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.824+0000 7fb37c5da740 1 -- 172.21.15.45:0/340660514 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55df3fee5700 con 0x55df3fe65390 2024-04-16T17:16:22.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.824+0000 7fb35cc00640 1 -- 172.21.15.45:0/340660514 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb360004c80 con 0x55df3fe65390 2024-04-16T17:16:22.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.824+0000 7fb35cc00640 1 -- 172.21.15.45:0/340660514 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 44) ==== 8346+0+0 (secure 0 0 0) 0x7fb36402d0e0 con 0x55df3fe65390 2024-04-16T17:16:22.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.824+0000 7fb37c5da740 10 client.17988 fetch_fsmap finished waiting for FSMap version 44 2024-04-16T17:16:22.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.824+0000 7fb37c5da740 10 client.17988 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:16:22.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.825+0000 7fb37c5da740 20 client.17988 populate_metadata read hostname 'smithi045' 2024-04-16T17:16:22.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.825+0000 7fb37c5da740 1 --2- 172.21.15.45:0/340660514 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55df3fee5e80 0x55df3fee82a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:22.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.825+0000 7fb37c5da740 4 client.17988 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-16T17:16:22.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.825+0000 7fb37c5da740 1 -- 172.21.15.45:0/340660514 --> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55df3fdd29b0 con 0x55df3fee5e80 2024-04-16T17:16:22.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.825+0000 7fb372000640 1 --2- 172.21.15.45:0/340660514 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55df3fee5e80 0x55df3fee82a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:22.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.825+0000 7fb372000640 1 --2- 172.21.15.45:0/340660514 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55df3fee5e80 0x55df3fee82a0 crc :-1 s=READY pgs=248 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5da6280c9887a16b server_cookie=f4d8bee130585ef9 in_seq=0 out_seq=0 2024-04-16T17:16:22.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.825+0000 7fb35cc00640 10 client.17988 ms_handle_connect on v2:172.21.15.104:6838/3660070592 2024-04-16T17:16:22.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.826+0000 7fb35cc00640 1 -- 172.21.15.45:0/340660514 <== mds.0 v2:172.21.15.104:6838/3660070592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55df3fdd29b0 con 0x55df3fee5e80 2024-04-16T17:16:22.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.826+0000 7fb35cc00640 10 client.17988 handle_command_reply: tid=0 2024-04-16T17:16:22.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.826+0000 7fb37c5da740 1 client.17988 shutdown 2024-04-16T17:16:22.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.826+0000 7fb37c5da740 1 -- 172.21.15.45:0/340660514 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55df3fe65390 msgr2=0x55df3fe727a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:22.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.826+0000 7fb37c5da740 1 --2- 172.21.15.45:0/340660514 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55df3fe65390 0x55df3fe727a0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fb364009d50 tx=0x7fb364017000 comp rx=0 tx=0).stop 2024-04-16T17:16:22.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.826+0000 7fb37c5da740 1 -- 172.21.15.45:0/340660514 shutdown_connections 2024-04-16T17:16:22.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.826+0000 7fb37c5da740 1 --2- 172.21.15.45:0/340660514 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55df3fee5e80 0x55df3fee82a0 crc :-1 s=READY pgs=248 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:22.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.826+0000 7fb37c5da740 1 --2- 172.21.15.45:0/340660514 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55df3fe65cb0 0x55df3fe72ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:22.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.826+0000 7fb37c5da740 1 --2- 172.21.15.45:0/340660514 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55df3fe6e890 0x55df3fee4240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:22.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.826+0000 7fb35cc00640 0 client.17988 ms_handle_reset on v2:172.21.15.104:6838/3660070592 2024-04-16T17:16:22.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.826+0000 7fb37c5da740 1 --2- 172.21.15.45:0/340660514 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55df3fe65390 0x55df3fe727a0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:22.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.826+0000 7fb37c5da740 1 -- 172.21.15.45:0/340660514 >> 172.21.15.45:0/340660514 conn(0x55df3fe5fac0 msgr2=0x55df3fe61780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:16:22.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.826+0000 7fb37c5da740 1 -- 172.21.15.45:0/340660514 shutdown_connections 2024-04-16T17:16:22.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.826+0000 7fb37c5da740 1 -- 172.21.15.45:0/340660514 wait complete. 2024-04-16T17:16:22.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.826+0000 7fb37c5da740 20 client.17988 trim_cache size 0 max 16384 2024-04-16T17:16:22.833 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-16T17:16:22.833 INFO:teuthology.orchestra.run.smithi045.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-16T17:16:22.833 INFO:teuthology.orchestra.run.smithi045.stdout: "scrubs": { 2024-04-16T17:16:22.833 INFO:teuthology.orchestra.run.smithi045.stdout: "2e4db62b-8dee-4c44-9204-a20321eae1b1": { 2024-04-16T17:16:22.833 INFO:teuthology.orchestra.run.smithi045.stdout: "path": "/", 2024-04-16T17:16:22.833 INFO:teuthology.orchestra.run.smithi045.stdout: "tag": "2e4db62b-8dee-4c44-9204-a20321eae1b1", 2024-04-16T17:16:22.834 INFO:teuthology.orchestra.run.smithi045.stdout: "options": "recursive,force" 2024-04-16T17:16:22.834 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:16:22.834 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:16:22.834 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-16T17:16:22.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.827+0000 7fb373e00640 1 -- 172.21.15.45:0/1811317789 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fb3580a3d20 msgr2=0x7fb3580a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:22.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.827+0000 7fb373e00640 1 --2- 172.21.15.45:0/1811317789 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fb3580a3d20 0x7fb3580a61e0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x55df3fe64e60 tx=0x7fb364005f90 comp rx=0 tx=0).stop 2024-04-16T17:16:22.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.827+0000 7fb373e00640 1 -- 172.21.15.45:0/1811317789 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb3741093f0 msgr2=0x7fb3741a0c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:22.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.827+0000 7fb373e00640 1 --2- 172.21.15.45:0/1811317789 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb3741093f0 0x7fb3741a0c50 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fb360036040 tx=0x7fb36002cb40 comp rx=0 tx=0).stop 2024-04-16T17:16:22.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.827+0000 7fb373e00640 1 -- 172.21.15.45:0/1811317789 shutdown_connections 2024-04-16T17:16:22.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.827+0000 7fb373e00640 1 --2- 172.21.15.45:0/1811317789 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fb3580a3d20 0x7fb3580a61e0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:22.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.827+0000 7fb373e00640 1 --2- 172.21.15.45:0/1811317789 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb37419a8c0 0x7fb37419ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:22.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.827+0000 7fb373e00640 1 --2- 172.21.15.45:0/1811317789 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb3741093f0 0x7fb3741a0c50 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:22.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.827+0000 7fb373e00640 1 --2- 172.21.15.45:0/1811317789 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb374104320 0x7fb3741a0710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:22.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.827+0000 7fb373e00640 1 -- 172.21.15.45:0/1811317789 >> 172.21.15.45:0/1811317789 conn(0x7fb3740ffb40 msgr2=0x7fb374100f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:16:22.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.827+0000 7fb373e00640 1 -- 172.21.15.45:0/1811317789 shutdown_connections 2024-04-16T17:16:22.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:22.827+0000 7fb373e00640 1 -- 172.21.15.45:0/1811317789 wait complete. 2024-04-16T17:16:22.847 INFO:tasks.cephfs.filesystem:scrub status for tag:2e4db62b-8dee-4c44-9204-a20321eae1b1 - {'path': '/', 'tag': '2e4db62b-8dee-4c44-9204-a20321eae1b1', 'options': 'recursive,force'} 2024-04-16T17:16:22.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:16:22 smithi104 ceph-mon[36457]: pgmap v300: 129 pgs: 129 active+clean; 56 MiB data, 870 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-16T17:16:23.804 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:16:23 smithi117 ceph-mon[36730]: scrub queued for path: / 2024-04-16T17:16:23.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:16:23 smithi117 ceph-mon[36730]: scrub summary: idle+waiting paths [/] 2024-04-16T17:16:23.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:16:23 smithi117 ceph-mon[36730]: scrub summary: active paths [/] 2024-04-16T17:16:23.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:16:23 smithi117 ceph-mon[36730]: scrub summary: active paths [/] 2024-04-16T17:16:23.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:16:23 smithi117 ceph-mon[36730]: scrub summary: active paths [/] 2024-04-16T17:16:23.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:16:23 smithi117 ceph-mon[36730]: scrub summary: idle+waiting paths [/] 2024-04-16T17:16:23.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:16:23 smithi117 ceph-mon[36730]: scrub summary: active paths [/] 2024-04-16T17:16:23.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:16:23 smithi117 ceph-mon[36730]: scrub summary: idle+waiting paths [/] 2024-04-16T17:16:23.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:16:23 smithi117 ceph-mon[36730]: scrub summary: idle+waiting paths [/] 2024-04-16T17:16:23.806 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:16:23 smithi117 ceph-mon[36730]: scrub summary: idle+waiting paths [/] 2024-04-16T17:16:23.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:16:23 smithi104 ceph-mon[36457]: scrub queued for path: / 2024-04-16T17:16:23.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:16:23 smithi104 ceph-mon[36457]: scrub summary: idle+waiting paths [/] 2024-04-16T17:16:23.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:16:23 smithi104 ceph-mon[36457]: scrub summary: active paths [/] 2024-04-16T17:16:23.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:16:23 smithi104 ceph-mon[36457]: scrub summary: active paths [/] 2024-04-16T17:16:23.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:16:23 smithi104 ceph-mon[36457]: scrub summary: active paths [/] 2024-04-16T17:16:23.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:16:23 smithi104 ceph-mon[36457]: scrub summary: idle+waiting paths [/] 2024-04-16T17:16:23.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:16:23 smithi104 ceph-mon[36457]: scrub summary: active paths [/] 2024-04-16T17:16:23.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:16:23 smithi104 ceph-mon[36457]: scrub summary: idle+waiting paths [/] 2024-04-16T17:16:23.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:16:23 smithi104 ceph-mon[36457]: scrub summary: idle+waiting paths [/] 2024-04-16T17:16:23.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:16:23 smithi104 ceph-mon[36457]: scrub summary: idle+waiting paths [/] 2024-04-16T17:16:23.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:16:23 smithi045 ceph-mon[31676]: scrub queued for path: / 2024-04-16T17:16:23.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:16:23 smithi045 ceph-mon[31676]: scrub summary: idle+waiting paths [/] 2024-04-16T17:16:23.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:16:23 smithi045 ceph-mon[31676]: scrub summary: active paths [/] 2024-04-16T17:16:23.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:16:23 smithi045 ceph-mon[31676]: scrub summary: active paths [/] 2024-04-16T17:16:23.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:16:23 smithi045 ceph-mon[31676]: scrub summary: active paths [/] 2024-04-16T17:16:23.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:16:23 smithi045 ceph-mon[31676]: scrub summary: idle+waiting paths [/] 2024-04-16T17:16:23.978 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:16:23 smithi045 ceph-mon[31676]: scrub summary: active paths [/] 2024-04-16T17:16:23.978 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:16:23 smithi045 ceph-mon[31676]: scrub summary: idle+waiting paths [/] 2024-04-16T17:16:23.978 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:16:23 smithi045 ceph-mon[31676]: scrub summary: idle+waiting paths [/] 2024-04-16T17:16:23.978 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:16:23 smithi045 ceph-mon[31676]: scrub summary: idle+waiting paths [/] 2024-04-16T17:16:24.804 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:16:24 smithi117 ceph-mon[36730]: pgmap v301: 129 pgs: 129 active+clean; 56 MiB data, 870 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-16T17:16:24.926 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:16:24 smithi104 ceph-mon[36457]: pgmap v301: 129 pgs: 129 active+clean; 56 MiB data, 870 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-16T17:16:24.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:16:24 smithi045 ceph-mon[31676]: pgmap v301: 129 pgs: 129 active+clean; 56 MiB data, 870 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-16T17:16:26.804 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:16:26 smithi117 ceph-mon[36730]: pgmap v302: 129 pgs: 129 active+clean; 56 MiB data, 870 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 92 B/s wr, 5 op/s 2024-04-16T17:16:26.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:16:26 smithi104 ceph-mon[36457]: pgmap v302: 129 pgs: 129 active+clean; 56 MiB data, 870 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 92 B/s wr, 5 op/s 2024-04-16T17:16:26.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:16:26 smithi045 ceph-mon[31676]: pgmap v302: 129 pgs: 129 active+clean; 56 MiB data, 870 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 92 B/s wr, 5 op/s 2024-04-16T17:16:28.804 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:16:28 smithi117 ceph-mon[36730]: pgmap v303: 129 pgs: 129 active+clean; 56 MiB data, 870 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 85 B/s wr, 14 op/s 2024-04-16T17:16:28.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:16:28 smithi104 ceph-mon[36457]: pgmap v303: 129 pgs: 129 active+clean; 56 MiB data, 870 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 85 B/s wr, 14 op/s 2024-04-16T17:16:28.976 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:16:28 smithi045 ceph-mon[31676]: pgmap v303: 129 pgs: 129 active+clean; 56 MiB data, 870 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 85 B/s wr, 14 op/s 2024-04-16T17:16:29.804 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:16:29 smithi117 ceph-mon[36730]: scrub summary: idle 2024-04-16T17:16:29.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:16:29 smithi117 ceph-mon[36730]: scrub summary: idle 2024-04-16T17:16:29.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:16:29 smithi117 ceph-mon[36730]: scrub summary: idle 2024-04-16T17:16:29.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:16:29 smithi104 ceph-mon[36457]: scrub summary: idle 2024-04-16T17:16:29.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:16:29 smithi104 ceph-mon[36457]: scrub summary: idle 2024-04-16T17:16:29.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:16:29 smithi104 ceph-mon[36457]: scrub summary: idle 2024-04-16T17:16:29.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:16:29 smithi045 ceph-mon[31676]: scrub summary: idle 2024-04-16T17:16:29.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:16:29 smithi045 ceph-mon[31676]: scrub summary: idle 2024-04-16T17:16:29.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:16:29 smithi045 ceph-mon[31676]: scrub summary: idle 2024-04-16T17:16:30.804 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:16:30 smithi117 ceph-mon[36730]: pgmap v304: 129 pgs: 129 active+clean; 56 MiB data, 870 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 11 op/s 2024-04-16T17:16:30.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:16:30 smithi104 ceph-mon[36457]: pgmap v304: 129 pgs: 129 active+clean; 56 MiB data, 870 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 11 op/s 2024-04-16T17:16:30.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:16:30 smithi045 ceph-mon[31676]: pgmap v304: 129 pgs: 129 active+clean; 56 MiB data, 870 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 11 op/s 2024-04-16T17:16:32.804 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:16:32 smithi117 ceph-mon[36730]: pgmap v305: 129 pgs: 129 active+clean; 56 MiB data, 870 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 14 op/s 2024-04-16T17:16:32.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:16:32 smithi104 ceph-mon[36457]: pgmap v305: 129 pgs: 129 active+clean; 56 MiB data, 870 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 14 op/s 2024-04-16T17:16:32.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:16:32 smithi045 ceph-mon[31676]: pgmap v305: 129 pgs: 129 active+clean; 56 MiB data, 870 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 14 op/s 2024-04-16T17:16:34.804 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:16:34 smithi117 ceph-mon[36730]: pgmap v306: 129 pgs: 129 active+clean; 56 MiB data, 870 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 11 op/s 2024-04-16T17:16:34.854 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:16:34 smithi045 ceph-mon[31676]: pgmap v306: 129 pgs: 129 active+clean; 56 MiB data, 870 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 11 op/s 2024-04-16T17:16:34.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:16:34 smithi104 ceph-mon[36457]: pgmap v306: 129 pgs: 129 active+clean; 56 MiB data, 870 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 11 op/s 2024-04-16T17:16:36.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:16:36 smithi104 ceph-mon[36457]: pgmap v307: 129 pgs: 129 active+clean; 56 MiB data, 870 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 11 op/s 2024-04-16T17:16:36.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:16:36 smithi045 ceph-mon[31676]: pgmap v307: 129 pgs: 129 active+clean; 56 MiB data, 870 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 11 op/s 2024-04-16T17:16:37.054 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:16:36 smithi117 ceph-mon[36730]: pgmap v307: 129 pgs: 129 active+clean; 56 MiB data, 870 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 11 op/s 2024-04-16T17:16:38.857 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:16:38 smithi117 ceph-mon[36730]: pgmap v308: 129 pgs: 129 active+clean; 56 MiB data, 870 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 14 op/s 2024-04-16T17:16:38.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:16:38 smithi104 ceph-mon[36457]: pgmap v308: 129 pgs: 129 active+clean; 56 MiB data, 870 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 14 op/s 2024-04-16T17:16:38.976 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:16:38 smithi045 ceph-mon[31676]: pgmap v308: 129 pgs: 129 active+clean; 56 MiB data, 870 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 14 op/s 2024-04-16T17:16:40.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:16:40 smithi104 ceph-mon[36457]: pgmap v309: 129 pgs: 129 active+clean; 56 MiB data, 870 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T17:16:40.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:16:40 smithi045 ceph-mon[31676]: pgmap v309: 129 pgs: 129 active+clean; 56 MiB data, 870 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T17:16:41.054 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:16:40 smithi117 ceph-mon[36730]: pgmap v309: 129 pgs: 129 active+clean; 56 MiB data, 870 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T17:16:42.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:16:42 smithi104 ceph-mon[36457]: pgmap v310: 129 pgs: 129 active+clean; 56 MiB data, 870 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-16T17:16:42.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:16:42 smithi045 ceph-mon[31676]: pgmap v310: 129 pgs: 129 active+clean; 56 MiB data, 870 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-16T17:16:43.054 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:16:42 smithi117 ceph-mon[36730]: pgmap v310: 129 pgs: 129 active+clean; 56 MiB data, 870 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-16T17:16:44.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:16:44 smithi104 ceph-mon[36457]: pgmap v311: 129 pgs: 129 active+clean; 56 MiB data, 870 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T17:16:44.976 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:16:44 smithi045 ceph-mon[31676]: pgmap v311: 129 pgs: 129 active+clean; 56 MiB data, 870 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T17:16:45.054 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:16:44 smithi117 ceph-mon[36730]: pgmap v311: 129 pgs: 129 active+clean; 56 MiB data, 870 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T17:16:46.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:16:46 smithi104 ceph-mon[36457]: pgmap v312: 129 pgs: 129 active+clean; 56 MiB data, 870 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T17:16:46.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:16:46 smithi045 ceph-mon[31676]: pgmap v312: 129 pgs: 129 active+clean; 56 MiB data, 870 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T17:16:47.054 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:16:46 smithi117 ceph-mon[36730]: pgmap v312: 129 pgs: 129 active+clean; 56 MiB data, 870 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T17:16:48.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:16:48 smithi104 ceph-mon[36457]: pgmap v313: 129 pgs: 129 active+clean; 56 MiB data, 870 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-16T17:16:48.976 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:16:48 smithi045 ceph-mon[31676]: pgmap v313: 129 pgs: 129 active+clean; 56 MiB data, 870 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-16T17:16:49.054 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:16:48 smithi117 ceph-mon[36730]: pgmap v313: 129 pgs: 129 active+clean; 56 MiB data, 870 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-16T17:16:50.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:16:50 smithi104 ceph-mon[36457]: pgmap v314: 129 pgs: 129 active+clean; 56 MiB data, 870 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-04-16T17:16:50.976 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:16:50 smithi045 ceph-mon[31676]: pgmap v314: 129 pgs: 129 active+clean; 56 MiB data, 870 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-04-16T17:16:51.054 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:16:50 smithi117 ceph-mon[36730]: pgmap v314: 129 pgs: 129 active+clean; 56 MiB data, 870 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-04-16T17:16:52.848 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-16T17:16:52.849 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-16T17:16:52.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:16:52 smithi104 ceph-mon[36457]: pgmap v315: 129 pgs: 129 active+clean; 56 MiB data, 870 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-16T17:16:52.945 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:16:52 smithi045 ceph-mon[31676]: pgmap v315: 129 pgs: 129 active+clean; 56 MiB data, 870 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-16T17:16:53.054 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:16:52 smithi117 ceph-mon[36730]: pgmap v315: 129 pgs: 129 active+clean; 56 MiB data, 870 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-16T17:16:53.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.096+0000 7fe76c600640 1 -- 172.21.15.45:0/1351114748 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe764108fd0 msgr2=0x7fe764115490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:53.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.096+0000 7fe76c600640 1 --2- 172.21.15.45:0/1351114748 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe764108fd0 0x7fe764115490 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fe75c009cd0 tx=0x7fe75c02c0e0 comp rx=0 tx=0).stop 2024-04-16T17:16:53.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.096+0000 7fe76c600640 1 -- 172.21.15.45:0/1351114748 shutdown_connections 2024-04-16T17:16:53.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.096+0000 7fe76c600640 1 --2- 172.21.15.45:0/1351114748 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe764108610 0x7fe764108a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:53.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.096+0000 7fe76c600640 1 --2- 172.21.15.45:0/1351114748 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe764108fd0 0x7fe764115490 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:53.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.096+0000 7fe76c600640 1 --2- 172.21.15.45:0/1351114748 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe76410e660 0x7fe76410ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:53.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.096+0000 7fe76c600640 1 -- 172.21.15.45:0/1351114748 >> 172.21.15.45:0/1351114748 conn(0x7fe764076540 msgr2=0x7fe764078960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:16:53.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.097+0000 7fe76c600640 1 -- 172.21.15.45:0/1351114748 shutdown_connections 2024-04-16T17:16:53.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.097+0000 7fe76c600640 1 -- 172.21.15.45:0/1351114748 wait complete. 2024-04-16T17:16:53.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.098+0000 7fe76c600640 1 Processor -- start 2024-04-16T17:16:53.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.098+0000 7fe76c600640 1 -- start start 2024-04-16T17:16:53.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.098+0000 7fe76c600640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe764108610 0x7fe76419ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:53.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.098+0000 7fe76c600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe764108fd0 0x7fe76419f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:53.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.098+0000 7fe76c600640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe76410e660 0x7fe7641a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:53.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.098+0000 7fe76c600640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fe7641189e0 con 0x7fe764108fd0 2024-04-16T17:16:53.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.098+0000 7fe76c600640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fe764118860 con 0x7fe764108610 2024-04-16T17:16:53.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.098+0000 7fe76c600640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fe764118b60 con 0x7fe76410e660 2024-04-16T17:16:53.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.098+0000 7fe769c00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe764108610 0x7fe76419ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:53.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.098+0000 7fe769c00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe764108610 0x7fe76419ee60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:45296/0 (socket says 172.21.15.45:45296) 2024-04-16T17:16:53.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.098+0000 7fe769c00640 1 -- 172.21.15.45:0/2043049797 learned_addr learned my addr 172.21.15.45:0/2043049797 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:16:53.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.099+0000 7fe769200640 1 --2- 172.21.15.45:0/2043049797 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe764108fd0 0x7fe76419f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:53.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.099+0000 7fe76a600640 1 --2- 172.21.15.45:0/2043049797 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe76410e660 0x7fe7641a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:53.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.099+0000 7fe769c00640 1 -- 172.21.15.45:0/2043049797 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe76410e660 msgr2=0x7fe7641a36d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:53.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.099+0000 7fe769c00640 1 --2- 172.21.15.45:0/2043049797 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe76410e660 0x7fe7641a36d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:53.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.099+0000 7fe769c00640 1 -- 172.21.15.45:0/2043049797 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe764108fd0 msgr2=0x7fe76419f3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:53.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.099+0000 7fe769c00640 1 --2- 172.21.15.45:0/2043049797 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe764108fd0 0x7fe76419f3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:53.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.099+0000 7fe769c00640 1 -- 172.21.15.45:0/2043049797 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7641a3e00 con 0x7fe764108610 2024-04-16T17:16:53.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.099+0000 7fe769200640 1 --2- 172.21.15.45:0/2043049797 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe764108fd0 0x7fe76419f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:16:53.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.099+0000 7fe769c00640 1 --2- 172.21.15.45:0/2043049797 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe764108610 0x7fe76419ee60 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fe75800e9c0 tx=0x7fe75800ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:16:53.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.100+0000 7fe762000640 1 -- 172.21.15.45:0/2043049797 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe75800cde0 con 0x7fe764108610 2024-04-16T17:16:53.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.100+0000 7fe762000640 1 -- 172.21.15.45:0/2043049797 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe758010910 con 0x7fe764108610 2024-04-16T17:16:53.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.100+0000 7fe762000640 1 -- 172.21.15.45:0/2043049797 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe758018a00 con 0x7fe764108610 2024-04-16T17:16:53.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.100+0000 7fe76c600640 1 -- 172.21.15.45:0/2043049797 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7641a40f0 con 0x7fe764108610 2024-04-16T17:16:53.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.100+0000 7fe76c600640 1 -- 172.21.15.45:0/2043049797 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7641ab980 con 0x7fe764108610 2024-04-16T17:16:53.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.103+0000 7fe76f3b0740 1 Processor -- start 2024-04-16T17:16:53.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.103+0000 7fe762000640 1 -- 172.21.15.45:0/2043049797 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fe75801f020 con 0x7fe764108610 2024-04-16T17:16:53.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.103+0000 7fe76f3b0740 1 -- start start 2024-04-16T17:16:53.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.104+0000 7fe76f3b0740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cbad9551b0 0x55cbad955590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:53.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.104+0000 7fe76f3b0740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cbad955ad0 0x55cbad95e170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:53.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.104+0000 7fe76f3b0740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55cbad95e6b0 0x55cbad960aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:53.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.104+0000 7fe76f3b0740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55cbad8bb120 con 0x55cbad955ad0 2024-04-16T17:16:53.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.104+0000 7fe76f3b0740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55cbad861340 con 0x55cbad95e6b0 2024-04-16T17:16:53.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.104+0000 7fe76f3b0740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55cbad8b70e0 con 0x55cbad9551b0 2024-04-16T17:16:53.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.104+0000 7fe769c00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cbad955ad0 0x55cbad95e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:53.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.104+0000 7fe769c00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cbad955ad0 0x55cbad95e170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56756/0 (socket says 172.21.15.45:56756) 2024-04-16T17:16:53.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.104+0000 7fe769c00640 1 -- 172.21.15.45:0/1702844217 learned_addr learned my addr 172.21.15.45:0/1702844217 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:16:53.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.104+0000 7fe769200640 1 --2- 172.21.15.45:0/1702844217 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55cbad95e6b0 0x55cbad960aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:53.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.104+0000 7fe76a600640 1 --2- 172.21.15.45:0/1702844217 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cbad9551b0 0x55cbad955590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:53.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.104+0000 7fe769c00640 1 -- 172.21.15.45:0/1702844217 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cbad9551b0 msgr2=0x55cbad955590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:16:53.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.104+0000 7fe769c00640 1 --2- 172.21.15.45:0/1702844217 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cbad9551b0 0x55cbad955590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:53.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.104+0000 7fe769c00640 1 -- 172.21.15.45:0/1702844217 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55cbad95e6b0 msgr2=0x55cbad960aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:16:53.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.104+0000 7fe769c00640 1 --2- 172.21.15.45:0/1702844217 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55cbad95e6b0 0x55cbad960aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:53.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.104+0000 7fe769c00640 1 -- 172.21.15.45:0/1702844217 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cbad6977d0 con 0x55cbad955ad0 2024-04-16T17:16:53.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.105+0000 7fe769c00640 1 --2- 172.21.15.45:0/1702844217 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cbad955ad0 0x55cbad95e170 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fe7580ce100 tx=0x7fe7580d91a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28f031223e93471e server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:16:53.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.105+0000 7fe762000640 1 --2- 172.21.15.45:0/2043049797 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fe74c0a3d20 0x7fe74c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:53.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.105+0000 7fe756a00640 1 -- 172.21.15.45:0/1702844217 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7580e4070 con 0x55cbad955ad0 2024-04-16T17:16:53.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.105+0000 7fe762000640 1 -- 172.21.15.45:0/2043049797 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 9496+0+0 (secure 0 0 0) 0x7fe7580d0ea0 con 0x7fe764108610 2024-04-16T17:16:53.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.105+0000 7fe76a600640 1 --2- 172.21.15.45:0/2043049797 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fe74c0a3d20 0x7fe74c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:53.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.105+0000 7fe756a00640 1 -- 172.21.15.45:0/1702844217 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe758053240 con 0x55cbad955ad0 2024-04-16T17:16:53.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.105+0000 7fe756a00640 1 -- 172.21.15.45:0/1702844217 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe75809c7e0 con 0x55cbad955ad0 2024-04-16T17:16:53.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.106+0000 7fe76f3b0740 1 -- 172.21.15.45:0/1702844217 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cbad955ad0 msgr2=0x55cbad95e170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:53.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.106+0000 7fe76f3b0740 1 --2- 172.21.15.45:0/1702844217 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cbad955ad0 0x55cbad95e170 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fe7580ce100 tx=0x7fe7580d91a0 comp rx=0 tx=0).stop 2024-04-16T17:16:53.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.106+0000 7fe76a600640 1 --2- 172.21.15.45:0/2043049797 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fe74c0a3d20 0x7fe74c0a61e0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fe75c005cc0 tx=0x7fe75c005c50 comp rx=0 tx=0).ready entity=mgr.25411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:16:53.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.106+0000 7fe76f3b0740 1 -- 172.21.15.45:0/1702844217 shutdown_connections 2024-04-16T17:16:53.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.106+0000 7fe76f3b0740 1 --2- 172.21.15.45:0/1702844217 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55cbad95e6b0 0x55cbad960aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:53.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.106+0000 7fe76f3b0740 1 --2- 172.21.15.45:0/1702844217 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cbad955ad0 0x55cbad95e170 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:53.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.106+0000 7fe76f3b0740 1 --2- 172.21.15.45:0/1702844217 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cbad9551b0 0x55cbad955590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:53.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.106+0000 7fe76f3b0740 1 -- 172.21.15.45:0/1702844217 >> 172.21.15.45:0/1702844217 conn(0x55cbad94f8e0 msgr2=0x55cbad94fcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:16:53.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.106+0000 7fe76f3b0740 1 -- 172.21.15.45:0/1702844217 shutdown_connections 2024-04-16T17:16:53.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.106+0000 7fe76f3b0740 1 -- 172.21.15.45:0/1702844217 wait complete. 2024-04-16T17:16:53.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.107+0000 7fe76f3b0740 1 -- start start 2024-04-16T17:16:53.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.107+0000 7fe76f3b0740 1 Processor -- start 2024-04-16T17:16:53.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.107+0000 7fe76f3b0740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55cbad9551b0 0x55cbad95b710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:53.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.107+0000 7fe76f3b0740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cbad955ad0 0x55cbad95bc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:53.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.108+0000 7fe76f3b0740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cbad95e6b0 0x55cbad95c190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:53.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.108+0000 7fe76f3b0740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55cbad9614a0 con 0x55cbad955ad0 2024-04-16T17:16:53.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.108+0000 7fe76f3b0740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55cbad874710 con 0x55cbad9551b0 2024-04-16T17:16:53.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.108+0000 7fe76f3b0740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55cbad961620 con 0x55cbad95e6b0 2024-04-16T17:16:53.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.108+0000 7fe76f3b0740 10 client.0 _ll_register_callbacks cb 0x55cbad86faf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:16:53.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.108+0000 7fe769200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cbad955ad0 0x55cbad95bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:53.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.108+0000 7fe769200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cbad955ad0 0x55cbad95bc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56760/0 (socket says 172.21.15.45:56760) 2024-04-16T17:16:53.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.108+0000 7fe769200640 1 -- 172.21.15.45:0/3700164656 learned_addr learned my addr 172.21.15.45:0/3700164656 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:16:53.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.108+0000 7fe769c00640 1 --2- 172.21.15.45:0/3700164656 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55cbad9551b0 0x55cbad95b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:53.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.108+0000 7fe76a600640 1 --2- 172.21.15.45:0/3700164656 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cbad95e6b0 0x55cbad95c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:53.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.108+0000 7fe769200640 1 -- 172.21.15.45:0/3700164656 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cbad95e6b0 msgr2=0x55cbad95c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:16:53.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.108+0000 7fe769200640 1 --2- 172.21.15.45:0/3700164656 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cbad95e6b0 0x55cbad95c190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:53.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.108+0000 7fe769200640 1 -- 172.21.15.45:0/3700164656 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55cbad9551b0 msgr2=0x55cbad95b710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:16:53.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.108+0000 7fe769200640 1 --2- 172.21.15.45:0/3700164656 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55cbad9551b0 0x55cbad95b710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:53.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.108+0000 7fe769200640 1 -- 172.21.15.45:0/3700164656 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cbad5cd3f0 con 0x55cbad955ad0 2024-04-16T17:16:53.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.108+0000 7fe76a600640 1 --2- 172.21.15.45:0/3700164656 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cbad95e6b0 0x55cbad95c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:16:53.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.109+0000 7fe769200640 1 --2- 172.21.15.45:0/3700164656 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cbad955ad0 0x55cbad95bc50 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fe750012770 tx=0x7fe750012c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d09d3166bddf4a1 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:16:53.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.109+0000 7fe74b400640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-16T17:16:53.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.109+0000 7fe74b400640 1 -- 172.21.15.45:0/3700164656 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe750011bb0 con 0x55cbad955ad0 2024-04-16T17:16:53.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.109+0000 7fe74b400640 1 -- 172.21.15.45:0/3700164656 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9496+0+0 (secure 0 0 0) 0x7fe750011d50 con 0x55cbad955ad0 2024-04-16T17:16:53.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.109+0000 7fe76f3b0740 1 -- 172.21.15.45:0/3700164656 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe758053240 con 0x55cbad955ad0 2024-04-16T17:16:53.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.109+0000 7fe74b400640 1 -- 172.21.15.45:0/3700164656 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fe75000f460 con 0x55cbad955ad0 2024-04-16T17:16:53.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.109+0000 7fe76f3b0740 10 client.18000 fetch_fsmap learned FSMap version 44 2024-04-16T17:16:53.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.109+0000 7fe76f3b0740 1 -- 172.21.15.45:0/3700164656 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55cbad58d0f0 con 0x55cbad955ad0 2024-04-16T17:16:53.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.109+0000 7fe74b400640 1 -- 172.21.15.45:0/3700164656 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe758053240 con 0x55cbad955ad0 2024-04-16T17:16:53.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.110+0000 7fe74b400640 1 -- 172.21.15.45:0/3700164656 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 44) ==== 8346+0+0 (secure 0 0 0) 0x7fe75001c7c0 con 0x55cbad955ad0 2024-04-16T17:16:53.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.110+0000 7fe76f3b0740 10 client.18000 fetch_fsmap finished waiting for FSMap version 44 2024-04-16T17:16:53.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.110+0000 7fe76f3b0740 10 client.18000 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:16:53.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.110+0000 7fe76f3b0740 20 client.18000 populate_metadata read hostname 'smithi045' 2024-04-16T17:16:53.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.110+0000 7fe76f3b0740 1 --2- 172.21.15.45:0/3700164656 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55cbad96eb90 0x55cbad970fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:53.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.110+0000 7fe76f3b0740 4 client.18000 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:16:53.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.110+0000 7fe76f3b0740 1 -- 172.21.15.45:0/3700164656 --> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cbad8c8b40 con 0x55cbad96eb90 2024-04-16T17:16:53.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.110+0000 7fe769c00640 1 --2- 172.21.15.45:0/3700164656 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55cbad96eb90 0x55cbad970fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:53.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.111+0000 7fe769c00640 1 --2- 172.21.15.45:0/3700164656 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55cbad96eb90 0x55cbad970fd0 crc :-1 s=READY pgs=249 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=76b316517431ea0f server_cookie=3c1fed7f37c3a654 in_seq=0 out_seq=0 2024-04-16T17:16:53.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.111+0000 7fe74b400640 10 client.18000 ms_handle_connect on v2:172.21.15.104:6838/3660070592 2024-04-16T17:16:53.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.112+0000 7fe74b400640 1 -- 172.21.15.45:0/3700164656 <== mds.0 v2:172.21.15.104:6838/3660070592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55cbad8c8b40 con 0x55cbad96eb90 2024-04-16T17:16:53.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.112+0000 7fe74b400640 10 client.18000 handle_command_reply: tid=0 2024-04-16T17:16:53.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.112+0000 7fe76f3b0740 1 client.18000 shutdown 2024-04-16T17:16:53.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.112+0000 7fe76f3b0740 1 -- 172.21.15.45:0/3700164656 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cbad955ad0 msgr2=0x55cbad95bc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:53.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.112+0000 7fe76f3b0740 1 --2- 172.21.15.45:0/3700164656 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cbad955ad0 0x55cbad95bc50 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fe750012770 tx=0x7fe750012c30 comp rx=0 tx=0).stop 2024-04-16T17:16:53.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.112+0000 7fe76f3b0740 1 -- 172.21.15.45:0/3700164656 shutdown_connections 2024-04-16T17:16:53.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.112+0000 7fe76f3b0740 1 --2- 172.21.15.45:0/3700164656 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55cbad96eb90 0x55cbad970fd0 crc :-1 s=READY pgs=249 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:53.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.112+0000 7fe76f3b0740 1 --2- 172.21.15.45:0/3700164656 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cbad95e6b0 0x55cbad95c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:53.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.112+0000 7fe76f3b0740 1 --2- 172.21.15.45:0/3700164656 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cbad955ad0 0x55cbad95bc50 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:53.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.112+0000 7fe76f3b0740 1 --2- 172.21.15.45:0/3700164656 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55cbad9551b0 0x55cbad95b710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:53.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.112+0000 7fe76f3b0740 1 -- 172.21.15.45:0/3700164656 >> 172.21.15.45:0/3700164656 conn(0x55cbad94f8e0 msgr2=0x55cbad951630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:16:53.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.112+0000 7fe74b400640 0 client.18000 ms_handle_reset on v2:172.21.15.104:6838/3660070592 2024-04-16T17:16:53.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.112+0000 7fe76f3b0740 1 -- 172.21.15.45:0/3700164656 shutdown_connections 2024-04-16T17:16:53.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.112+0000 7fe76f3b0740 1 -- 172.21.15.45:0/3700164656 wait complete. 2024-04-16T17:16:53.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.112+0000 7fe76f3b0740 20 client.18000 trim_cache size 0 max 16384 2024-04-16T17:16:53.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.131+0000 7fe76f3b0740 1 Processor -- start 2024-04-16T17:16:53.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.131+0000 7fe76f3b0740 1 -- start start 2024-04-16T17:16:53.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.132+0000 7fe76f3b0740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55cbad9551b0 0x55cbad95de40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:53.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.132+0000 7fe76f3b0740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cbad955ad0 0x55cbad95a650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:53.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.132+0000 7fe76f3b0740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cbad95e6b0 0x55cbad956c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:53.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.132+0000 7fe76f3b0740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fe7641b84a0 con 0x55cbad955ad0 2024-04-16T17:16:53.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.132+0000 7fe76f3b0740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fe7641b5d70 con 0x55cbad9551b0 2024-04-16T17:16:53.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.132+0000 7fe76f3b0740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fe7641b3be0 con 0x55cbad95e6b0 2024-04-16T17:16:53.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.132+0000 7fe769200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cbad955ad0 0x55cbad95a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:53.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.132+0000 7fe769200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cbad955ad0 0x55cbad95a650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56764/0 (socket says 172.21.15.45:56764) 2024-04-16T17:16:53.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.132+0000 7fe769200640 1 -- 172.21.15.45:0/3779347334 learned_addr learned my addr 172.21.15.45:0/3779347334 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:16:53.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.132+0000 7fe769c00640 1 --2- 172.21.15.45:0/3779347334 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55cbad9551b0 0x55cbad95de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:53.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.132+0000 7fe76a600640 1 --2- 172.21.15.45:0/3779347334 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cbad95e6b0 0x55cbad956c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:53.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.132+0000 7fe769200640 1 -- 172.21.15.45:0/3779347334 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cbad95e6b0 msgr2=0x55cbad956c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:16:53.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.133+0000 7fe769200640 1 --2- 172.21.15.45:0/3779347334 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cbad95e6b0 0x55cbad956c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:53.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.133+0000 7fe769200640 1 -- 172.21.15.45:0/3779347334 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55cbad9551b0 msgr2=0x55cbad95de40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:16:53.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.133+0000 7fe769200640 1 --2- 172.21.15.45:0/3779347334 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55cbad9551b0 0x55cbad95de40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:53.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.133+0000 7fe769200640 1 -- 172.21.15.45:0/3779347334 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cbad960aa0 con 0x55cbad955ad0 2024-04-16T17:16:53.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.133+0000 7fe76a600640 1 --2- 172.21.15.45:0/3779347334 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cbad95e6b0 0x55cbad956c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:16:53.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.133+0000 7fe769200640 1 --2- 172.21.15.45:0/3779347334 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cbad955ad0 0x55cbad95a650 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fe75001ce20 tx=0x7fe750009f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8b142ec387f7a91 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:16:53.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.133+0000 7fe756000640 1 -- 172.21.15.45:0/3779347334 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe750009680 con 0x55cbad955ad0 2024-04-16T17:16:53.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.133+0000 7fe756000640 1 -- 172.21.15.45:0/3779347334 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe75002f070 con 0x55cbad955ad0 2024-04-16T17:16:53.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.133+0000 7fe756000640 1 -- 172.21.15.45:0/3779347334 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe75000f8f0 con 0x55cbad955ad0 2024-04-16T17:16:53.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.134+0000 7fe76f3b0740 1 -- 172.21.15.45:0/3779347334 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cbad955ad0 msgr2=0x55cbad95a650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:53.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.134+0000 7fe76f3b0740 1 --2- 172.21.15.45:0/3779347334 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cbad955ad0 0x55cbad95a650 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fe75001ce20 tx=0x7fe750009f50 comp rx=0 tx=0).stop 2024-04-16T17:16:53.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.134+0000 7fe76f3b0740 1 -- 172.21.15.45:0/3779347334 shutdown_connections 2024-04-16T17:16:53.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.134+0000 7fe76f3b0740 1 --2- 172.21.15.45:0/3779347334 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cbad95e6b0 0x55cbad956c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:53.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.134+0000 7fe76f3b0740 1 --2- 172.21.15.45:0/3779347334 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cbad955ad0 0x55cbad95a650 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:53.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.134+0000 7fe76f3b0740 1 --2- 172.21.15.45:0/3779347334 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55cbad9551b0 0x55cbad95de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:53.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.134+0000 7fe76f3b0740 1 -- 172.21.15.45:0/3779347334 >> 172.21.15.45:0/3779347334 conn(0x55cbad94f8e0 msgr2=0x55cbad94e8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:16:53.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.134+0000 7fe76f3b0740 1 -- 172.21.15.45:0/3779347334 shutdown_connections 2024-04-16T17:16:53.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.134+0000 7fe76f3b0740 1 -- 172.21.15.45:0/3779347334 wait complete. 2024-04-16T17:16:53.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.134+0000 7fe76f3b0740 1 -- start start 2024-04-16T17:16:53.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.135+0000 7fe76f3b0740 1 Processor -- start 2024-04-16T17:16:53.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.135+0000 7fe76f3b0740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55cbad9551b0 0x55cbad966b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:53.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.135+0000 7fe76f3b0740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cbad955ad0 0x55cbad961f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:53.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.135+0000 7fe76f3b0740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cbad95e6b0 0x55cbad962440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:53.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.135+0000 7fe76f3b0740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fe7641b4230 con 0x55cbad95e6b0 2024-04-16T17:16:53.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.135+0000 7fe76f3b0740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fe7641b56a0 con 0x55cbad9551b0 2024-04-16T17:16:53.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.135+0000 7fe76f3b0740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fe7641b4900 con 0x55cbad955ad0 2024-04-16T17:16:53.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.135+0000 7fe76f3b0740 10 client.0 _ll_register_callbacks cb 0x7fe750009860 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:16:53.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.135+0000 7fe769c00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55cbad9551b0 0x55cbad966b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:53.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.135+0000 7fe76a600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cbad95e6b0 0x55cbad962440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:53.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.135+0000 7fe769c00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55cbad9551b0 0x55cbad966b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:45338/0 (socket says 172.21.15.45:45338) 2024-04-16T17:16:53.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.135+0000 7fe769c00640 1 -- 172.21.15.45:0/4185804869 learned_addr learned my addr 172.21.15.45:0/4185804869 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:16:53.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.135+0000 7fe769200640 1 --2- 172.21.15.45:0/4185804869 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cbad955ad0 0x55cbad961f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:53.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.136+0000 7fe769c00640 1 -- 172.21.15.45:0/4185804869 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cbad955ad0 msgr2=0x55cbad961f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:16:53.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.136+0000 7fe769c00640 1 --2- 172.21.15.45:0/4185804869 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cbad955ad0 0x55cbad961f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:53.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.136+0000 7fe769c00640 1 -- 172.21.15.45:0/4185804869 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cbad95e6b0 msgr2=0x55cbad962440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:16:53.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.136+0000 7fe769c00640 1 --2- 172.21.15.45:0/4185804869 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cbad95e6b0 0x55cbad962440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:53.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.136+0000 7fe769c00640 1 -- 172.21.15.45:0/4185804869 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cbad962b30 con 0x55cbad9551b0 2024-04-16T17:16:53.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.136+0000 7fe769c00640 1 --2- 172.21.15.45:0/4185804869 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55cbad9551b0 0x55cbad966b60 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fe7580993e0 tx=0x7fe7580bd040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6c561ac9e09ad3c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:16:53.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.136+0000 7fe74b400640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-04-16T17:16:53.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.136+0000 7fe74b400640 1 -- 172.21.15.45:0/4185804869 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe75809c370 con 0x55cbad9551b0 2024-04-16T17:16:53.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.136+0000 7fe76f3b0740 1 -- 172.21.15.45:0/4185804869 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe75002f070 con 0x55cbad9551b0 2024-04-16T17:16:53.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.136+0000 7fe74b400640 1 -- 172.21.15.45:0/4185804869 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9496+0+0 (secure 0 0 0) 0x7fe75809ad10 con 0x55cbad9551b0 2024-04-16T17:16:53.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.137+0000 7fe74b400640 1 -- 172.21.15.45:0/4185804869 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fe75809c550 con 0x55cbad9551b0 2024-04-16T17:16:53.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.137+0000 7fe76f3b0740 10 client.28157 fetch_fsmap learned FSMap version 44 2024-04-16T17:16:53.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.137+0000 7fe76f3b0740 1 -- 172.21.15.45:0/4185804869 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55cbad9d5020 con 0x55cbad9551b0 2024-04-16T17:16:53.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.137+0000 7fe74b400640 1 -- 172.21.15.45:0/4185804869 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe75002f070 con 0x55cbad9551b0 2024-04-16T17:16:53.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.137+0000 7fe74b400640 1 -- 172.21.15.45:0/4185804869 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 44) ==== 8346+0+0 (secure 0 0 0) 0x7fe7580d3040 con 0x55cbad9551b0 2024-04-16T17:16:53.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.137+0000 7fe76f3b0740 10 client.28157 fetch_fsmap finished waiting for FSMap version 44 2024-04-16T17:16:53.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.137+0000 7fe76f3b0740 10 client.28157 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:16:53.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.137+0000 7fe76f3b0740 20 client.28157 populate_metadata read hostname 'smithi045' 2024-04-16T17:16:53.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.137+0000 7fe76f3b0740 1 --2- 172.21.15.45:0/4185804869 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55cbad9d5310 0x55cbad9d7700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:53.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.137+0000 7fe76f3b0740 4 client.28157 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-16T17:16:53.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.137+0000 7fe76f3b0740 1 -- 172.21.15.45:0/4185804869 --> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55cbad8c29b0 con 0x55cbad9d5310 2024-04-16T17:16:53.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.137+0000 7fe769200640 1 --2- 172.21.15.45:0/4185804869 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55cbad9d5310 0x55cbad9d7700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:53.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.138+0000 7fe769200640 1 --2- 172.21.15.45:0/4185804869 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55cbad9d5310 0x55cbad9d7700 crc :-1 s=READY pgs=250 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fb0e19ddc953e7d4 server_cookie=ceec60dcc5e87ba0 in_seq=0 out_seq=0 2024-04-16T17:16:53.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.138+0000 7fe74b400640 10 client.28157 ms_handle_connect on v2:172.21.15.104:6838/3660070592 2024-04-16T17:16:53.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.138+0000 7fe74b400640 1 -- 172.21.15.45:0/4185804869 <== mds.0 v2:172.21.15.104:6838/3660070592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55cbad8c29b0 con 0x55cbad9d5310 2024-04-16T17:16:53.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.138+0000 7fe74b400640 10 client.28157 handle_command_reply: tid=0 2024-04-16T17:16:53.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.138+0000 7fe76f3b0740 1 client.28157 shutdown 2024-04-16T17:16:53.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.138+0000 7fe76f3b0740 1 -- 172.21.15.45:0/4185804869 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55cbad9551b0 msgr2=0x55cbad966b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:53.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.138+0000 7fe76f3b0740 1 --2- 172.21.15.45:0/4185804869 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55cbad9551b0 0x55cbad966b60 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fe7580993e0 tx=0x7fe7580bd040 comp rx=0 tx=0).stop 2024-04-16T17:16:53.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.139+0000 7fe76f3b0740 1 -- 172.21.15.45:0/4185804869 shutdown_connections 2024-04-16T17:16:53.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.139+0000 7fe76f3b0740 1 --2- 172.21.15.45:0/4185804869 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55cbad9d5310 0x55cbad9d7700 crc :-1 s=READY pgs=250 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:53.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.139+0000 7fe76f3b0740 1 --2- 172.21.15.45:0/4185804869 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cbad955ad0 0x55cbad961f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:53.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.139+0000 7fe76f3b0740 1 --2- 172.21.15.45:0/4185804869 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cbad95e6b0 0x55cbad962440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:53.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.139+0000 7fe74b400640 0 client.28157 ms_handle_reset on v2:172.21.15.104:6838/3660070592 2024-04-16T17:16:53.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.139+0000 7fe76f3b0740 1 --2- 172.21.15.45:0/4185804869 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55cbad9551b0 0x55cbad966b60 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:53.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.139+0000 7fe76f3b0740 1 -- 172.21.15.45:0/4185804869 >> 172.21.15.45:0/4185804869 conn(0x55cbad94f8e0 msgr2=0x55cbad95b640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:16:53.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.139+0000 7fe76f3b0740 1 -- 172.21.15.45:0/4185804869 shutdown_connections 2024-04-16T17:16:53.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.139+0000 7fe76f3b0740 1 -- 172.21.15.45:0/4185804869 wait complete. 2024-04-16T17:16:53.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.139+0000 7fe76f3b0740 20 client.28157 trim_cache size 0 max 16384 2024-04-16T17:16:53.149 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-16T17:16:53.149 INFO:teuthology.orchestra.run.smithi045.stdout: "status": "no active scrubs running", 2024-04-16T17:16:53.149 INFO:teuthology.orchestra.run.smithi045.stdout: "scrubs": {} 2024-04-16T17:16:53.150 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-16T17:16:53.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.140+0000 7fe76c600640 1 -- 172.21.15.45:0/2043049797 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fe74c0a3d20 msgr2=0x7fe74c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:53.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.140+0000 7fe76c600640 1 --2- 172.21.15.45:0/2043049797 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fe74c0a3d20 0x7fe74c0a61e0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fe75c005cc0 tx=0x7fe75c005c50 comp rx=0 tx=0).stop 2024-04-16T17:16:53.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.140+0000 7fe76c600640 1 -- 172.21.15.45:0/2043049797 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe764108610 msgr2=0x7fe76419ee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:53.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.140+0000 7fe76c600640 1 --2- 172.21.15.45:0/2043049797 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe764108610 0x7fe76419ee60 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fe75800e9c0 tx=0x7fe75800ee80 comp rx=0 tx=0).stop 2024-04-16T17:16:53.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.140+0000 7fe76c600640 1 -- 172.21.15.45:0/2043049797 shutdown_connections 2024-04-16T17:16:53.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.140+0000 7fe76c600640 1 --2- 172.21.15.45:0/2043049797 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fe74c0a3d20 0x7fe74c0a61e0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:53.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.140+0000 7fe76c600640 1 --2- 172.21.15.45:0/2043049797 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe76410e660 0x7fe7641a36d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:53.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.140+0000 7fe76c600640 1 --2- 172.21.15.45:0/2043049797 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe764108fd0 0x7fe76419f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:53.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.140+0000 7fe76c600640 1 --2- 172.21.15.45:0/2043049797 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe764108610 0x7fe76419ee60 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:53.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.140+0000 7fe76c600640 1 -- 172.21.15.45:0/2043049797 >> 172.21.15.45:0/2043049797 conn(0x7fe764076540 msgr2=0x7fe764077b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:16:53.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.140+0000 7fe76c600640 1 -- 172.21.15.45:0/2043049797 shutdown_connections 2024-04-16T17:16:53.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.140+0000 7fe76c600640 1 -- 172.21.15.45:0/2043049797 wait complete. 2024-04-16T17:16:53.158 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-16T17:16:53.159 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-16T17:16:53.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.371+0000 7f0766c00640 1 -- 172.21.15.45:0/2311552464 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0760104290 msgr2=0x7f0760104670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:53.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.371+0000 7f0766c00640 1 --2- 172.21.15.45:0/2311552464 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0760104290 0x7f0760104670 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f0754009900 tx=0x7f075402bc30 comp rx=0 tx=0).stop 2024-04-16T17:16:53.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.371+0000 7f0766c00640 1 -- 172.21.15.45:0/2311552464 shutdown_connections 2024-04-16T17:16:53.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.371+0000 7f0766c00640 1 --2- 172.21.15.45:0/2311552464 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0760109360 0x7f0760110f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:53.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.371+0000 7f0766c00640 1 --2- 172.21.15.45:0/2311552464 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0760104c40 0x7f0760108c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:53.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.371+0000 7f0766c00640 1 --2- 172.21.15.45:0/2311552464 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0760104290 0x7f0760104670 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:53.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.371+0000 7f0766c00640 1 -- 172.21.15.45:0/2311552464 >> 172.21.15.45:0/2311552464 conn(0x7f07600ffad0 msgr2=0x7f0760101ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:16:53.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.372+0000 7f0766c00640 1 -- 172.21.15.45:0/2311552464 shutdown_connections 2024-04-16T17:16:53.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.372+0000 7f0766c00640 1 -- 172.21.15.45:0/2311552464 wait complete. 2024-04-16T17:16:53.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.372+0000 7f0766c00640 1 Processor -- start 2024-04-16T17:16:53.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.373+0000 7f0766c00640 1 -- start start 2024-04-16T17:16:53.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.373+0000 7f0766c00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0760104290 0x7f07601a0630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:53.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.373+0000 7f0766c00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0760109360 0x7f07601a0b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:53.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.373+0000 7f0766c00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f076019a7a0 0x7f076019ac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:53.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.373+0000 7f0766c00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f0760114460 con 0x7f0760104290 2024-04-16T17:16:53.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.373+0000 7f0766c00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f07601142e0 con 0x7f076019a7a0 2024-04-16T17:16:53.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.373+0000 7f0766c00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f07601145e0 con 0x7f0760109360 2024-04-16T17:16:53.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.373+0000 7f0765800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0760104290 0x7f07601a0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:53.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.373+0000 7f0765800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0760104290 0x7f07601a0630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56794/0 (socket says 172.21.15.45:56794) 2024-04-16T17:16:53.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.373+0000 7f0765800640 1 -- 172.21.15.45:0/1713286111 learned_addr learned my addr 172.21.15.45:0/1713286111 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:16:53.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.373+0000 7f0766200640 1 --2- 172.21.15.45:0/1713286111 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f076019a7a0 0x7f076019ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:53.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.374+0000 7f0764e00640 1 --2- 172.21.15.45:0/1713286111 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0760109360 0x7f07601a0b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:53.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.374+0000 7f0765800640 1 -- 172.21.15.45:0/1713286111 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0760109360 msgr2=0x7f07601a0b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:53.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.374+0000 7f0765800640 1 --2- 172.21.15.45:0/1713286111 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0760109360 0x7f07601a0b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:53.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.374+0000 7f0765800640 1 -- 172.21.15.45:0/1713286111 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f076019a7a0 msgr2=0x7f076019ac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:53.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.374+0000 7f0765800640 1 --2- 172.21.15.45:0/1713286111 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f076019a7a0 0x7f076019ac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:53.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.374+0000 7f0765800640 1 -- 172.21.15.45:0/1713286111 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f076019b330 con 0x7f0760104290 2024-04-16T17:16:53.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.374+0000 7f0766200640 1 --2- 172.21.15.45:0/1713286111 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f076019a7a0 0x7f076019ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:16:53.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.374+0000 7f0765800640 1 --2- 172.21.15.45:0/1713286111 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0760104290 0x7f07601a0630 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f075402c240 tx=0x7f07540042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:16:53.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.374+0000 7f075e000640 1 -- 172.21.15.45:0/1713286111 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0754043020 con 0x7f0760104290 2024-04-16T17:16:53.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.375+0000 7f0766c00640 1 -- 172.21.15.45:0/1713286111 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f076019b5c0 con 0x7f0760104290 2024-04-16T17:16:53.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.375+0000 7f075e000640 1 -- 172.21.15.45:0/1713286111 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f07540054e0 con 0x7f0760104290 2024-04-16T17:16:53.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.375+0000 7f075e000640 1 -- 172.21.15.45:0/1713286111 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0754003b20 con 0x7f0760104290 2024-04-16T17:16:53.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.375+0000 7f0766c00640 1 -- 172.21.15.45:0/1713286111 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07601a77a0 con 0x7f0760104290 2024-04-16T17:16:53.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.376+0000 7f0766c00640 1 -- 172.21.15.45:0/1713286111 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0760105760 con 0x7f0760104290 2024-04-16T17:16:53.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.378+0000 7f075e000640 1 -- 172.21.15.45:0/1713286111 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f0754046020 con 0x7f0760104290 2024-04-16T17:16:53.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.379+0000 7f075e000640 1 --2- 172.21.15.45:0/1713286111 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f07480a3d20 0x7f07480a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:53.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.379+0000 7f075e000640 1 -- 172.21.15.45:0/1713286111 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 9496+0+0 (secure 0 0 0) 0x7f07540f2a20 con 0x7f0760104290 2024-04-16T17:16:53.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.380+0000 7f0764e00640 1 --2- 172.21.15.45:0/1713286111 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f07480a3d20 0x7f07480a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:53.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.380+0000 7f0764e00640 1 --2- 172.21.15.45:0/1713286111 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f07480a3d20 0x7f07480a61e0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f076010a910 tx=0x7f074c005f90 comp rx=0 tx=0).ready entity=mgr.25411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:16:53.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.385+0000 7f075e000640 1 -- 172.21.15.45:0/1713286111 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f075404d020 con 0x7f0760104290 2024-04-16T17:16:53.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.636+0000 7f0766c00640 1 -- 172.21.15.45:0/1713286111 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0760104c40 con 0x7f0760104290 2024-04-16T17:16:53.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.637+0000 7f075e000640 1 -- 172.21.15.45:0/1713286111 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) ==== 94+0+11887 (secure 0 0 0) 0x7f07540beda0 con 0x7f0760104290 2024-04-16T17:16:53.638 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-16T17:16:53.638 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":44,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/1748842443","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":1748842443},{"type":"v1","addr":"172.21.15.45:6835","nonce":1748842443}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6839/1308627051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":1308627051},{"type":"v1","addr":"172.21.15.45:6839","nonce":1308627051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/1768242489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1768242489},{"type":"v1","addr":"172.21.15.117:6833","nonce":1768242489}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6833/3558902585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":3558902585},{"type":"v1","addr":"172.21.15.104:6833","nonce":3558902585}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24478,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/1308142903","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":1308142903},{"type":"v1","addr":"172.21.15.45:6837","nonce":1308142903}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24491,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6835/3430679937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":3430679937},{"type":"v1","addr":"172.21.15.104:6835","nonce":3430679937}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":44,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-16T17:03:24.663611+0000","modified":"2024-04-16T17:15:52.264451+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24581,"mds_1":24569,"mds_2":24532},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24499":{"gid":24499,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/2224123841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2224123841},{"type":"v1","addr":"172.21.15.117:6835","nonce":2224123841}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6837/2471356815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":2471356815},{"type":"v1","addr":"172.21.15.117:6837","nonce":2471356815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24532":{"gid":24532,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":13,"addr":"172.21.15.117:6839/2378675949","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":2378675949},{"type":"v1","addr":"172.21.15.117:6839","nonce":2378675949}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24569":{"gid":24569,"name":"a","rank":1,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.45:6841/3755310822","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6840","nonce":3755310822},{"type":"v1","addr":"172.21.15.45:6841","nonce":3755310822}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24581":{"gid":24581,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.104:6839/3660070592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6838","nonce":3660070592},{"type":"v1","addr":"172.21.15.104:6839","nonce":3660070592}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_27569":{"gid":27569,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6837/1833645246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":1833645246},{"type":"v1","addr":"172.21.15.104:6837","nonce":1833645246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24581,"qdb_cluster":[24569,24532,24581]},"id":1}]} 2024-04-16T17:16:53.639 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 44 2024-04-16T17:16:53.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.640+0000 7f0766c00640 1 -- 172.21.15.45:0/1713286111 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f07480a3d20 msgr2=0x7f07480a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:53.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.640+0000 7f0766c00640 1 --2- 172.21.15.45:0/1713286111 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f07480a3d20 0x7f07480a61e0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f076010a910 tx=0x7f074c005f90 comp rx=0 tx=0).stop 2024-04-16T17:16:53.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.640+0000 7f0766c00640 1 -- 172.21.15.45:0/1713286111 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0760104290 msgr2=0x7f07601a0630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:53.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.640+0000 7f0766c00640 1 --2- 172.21.15.45:0/1713286111 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0760104290 0x7f07601a0630 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f075402c240 tx=0x7f07540042f0 comp rx=0 tx=0).stop 2024-04-16T17:16:53.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.640+0000 7f0766c00640 1 -- 172.21.15.45:0/1713286111 shutdown_connections 2024-04-16T17:16:53.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.640+0000 7f0766c00640 1 --2- 172.21.15.45:0/1713286111 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f07480a3d20 0x7f07480a61e0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:53.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.640+0000 7f0766c00640 1 --2- 172.21.15.45:0/1713286111 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0760109360 0x7f07601a0b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:53.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.640+0000 7f0766c00640 1 --2- 172.21.15.45:0/1713286111 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f076019a7a0 0x7f076019ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:53.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.640+0000 7f0766c00640 1 --2- 172.21.15.45:0/1713286111 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0760104290 0x7f07601a0630 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:53.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.640+0000 7f0766c00640 1 -- 172.21.15.45:0/1713286111 >> 172.21.15.45:0/1713286111 conn(0x7f07600ffad0 msgr2=0x7f0760101e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:16:53.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.640+0000 7f0766c00640 1 -- 172.21.15.45:0/1713286111 shutdown_connections 2024-04-16T17:16:53.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.640+0000 7f0766c00640 1 -- 172.21.15.45:0/1713286111 wait complete. 2024-04-16T17:16:53.661 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-16T17:16:53.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.884+0000 7fd2af200640 1 -- 172.21.15.45:0/2904869925 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd2a8076530 msgr2=0x7fd2a8076910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:53.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.884+0000 7fd2af200640 1 --2- 172.21.15.45:0/2904869925 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd2a8076530 0x7fd2a8076910 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fd29c009890 tx=0x7fd29c02bb70 comp rx=0 tx=0).stop 2024-04-16T17:16:53.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.884+0000 7fd2af200640 1 -- 172.21.15.45:0/2904869925 shutdown_connections 2024-04-16T17:16:53.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.885+0000 7fd2af200640 1 --2- 172.21.15.45:0/2904869925 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd2a8112d40 0x7fd2a8115130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:53.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.885+0000 7fd2af200640 1 --2- 172.21.15.45:0/2904869925 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd2a8076e50 0x7fd2a8112800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:53.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.885+0000 7fd2af200640 1 --2- 172.21.15.45:0/2904869925 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd2a8076530 0x7fd2a8076910 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:53.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.885+0000 7fd2af200640 1 -- 172.21.15.45:0/2904869925 >> 172.21.15.45:0/2904869925 conn(0x7fd2a80ffb70 msgr2=0x7fd2a8101f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:16:53.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.885+0000 7fd2af200640 1 -- 172.21.15.45:0/2904869925 shutdown_connections 2024-04-16T17:16:53.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.885+0000 7fd2af200640 1 -- 172.21.15.45:0/2904869925 wait complete. 2024-04-16T17:16:53.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.886+0000 7fd2af200640 1 Processor -- start 2024-04-16T17:16:53.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.886+0000 7fd2af200640 1 -- start start 2024-04-16T17:16:53.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.886+0000 7fd2af200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd2a8076530 0x7fd2a819ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:53.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.886+0000 7fd2af200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd2a8076e50 0x7fd2a819f350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:53.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.886+0000 7fd2af200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd2a8112d40 0x7fd2a81a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:53.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.886+0000 7fd2af200640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fd2a8118890 con 0x7fd2a8112d40 2024-04-16T17:16:53.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.886+0000 7fd2af200640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fd2a8118710 con 0x7fd2a8076e50 2024-04-16T17:16:53.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.886+0000 7fd2af200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fd2a8118a10 con 0x7fd2a8076530 2024-04-16T17:16:53.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.887+0000 7fd2ade00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd2a8076530 0x7fd2a819ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:53.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.887+0000 7fd2ade00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd2a8076530 0x7fd2a819ee10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:40272/0 (socket says 172.21.15.45:40272) 2024-04-16T17:16:53.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.887+0000 7fd2ade00640 1 -- 172.21.15.45:0/3979021457 learned_addr learned my addr 172.21.15.45:0/3979021457 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:16:53.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.887+0000 7fd2ad400640 1 --2- 172.21.15.45:0/3979021457 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd2a8076e50 0x7fd2a819f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:53.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.887+0000 7fd2ae800640 1 --2- 172.21.15.45:0/3979021457 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd2a8112d40 0x7fd2a81a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:53.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.887+0000 7fd2ade00640 1 -- 172.21.15.45:0/3979021457 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd2a8076e50 msgr2=0x7fd2a819f350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:53.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.887+0000 7fd2ade00640 1 --2- 172.21.15.45:0/3979021457 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd2a8076e50 0x7fd2a819f350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:53.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.887+0000 7fd2ade00640 1 -- 172.21.15.45:0/3979021457 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd2a8112d40 msgr2=0x7fd2a81a36d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:53.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.887+0000 7fd2ade00640 1 --2- 172.21.15.45:0/3979021457 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd2a8112d40 0x7fd2a81a36d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:53.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.887+0000 7fd2ade00640 1 -- 172.21.15.45:0/3979021457 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2a81a3e00 con 0x7fd2a8076530 2024-04-16T17:16:53.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.887+0000 7fd2ade00640 1 --2- 172.21.15.45:0/3979021457 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd2a8076530 0x7fd2a819ee10 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fd29c0043c0 tx=0x7fd29c0043f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:16:53.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.888+0000 7fd2a2a00640 1 -- 172.21.15.45:0/3979021457 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd29c041020 con 0x7fd2a8076530 2024-04-16T17:16:53.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.888+0000 7fd2a2a00640 1 -- 172.21.15.45:0/3979021457 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd29c02c9f0 con 0x7fd2a8076530 2024-04-16T17:16:53.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.888+0000 7fd2af200640 1 -- 172.21.15.45:0/3979021457 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2a81a4090 con 0x7fd2a8076530 2024-04-16T17:16:53.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.888+0000 7fd2a2a00640 1 -- 172.21.15.45:0/3979021457 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd29c039040 con 0x7fd2a8076530 2024-04-16T17:16:53.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.888+0000 7fd2af200640 1 -- 172.21.15.45:0/3979021457 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2a81ab980 con 0x7fd2a8076530 2024-04-16T17:16:53.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.891+0000 7fd2b357a740 1 Processor -- start 2024-04-16T17:16:53.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.891+0000 7fd2b357a740 1 -- start start 2024-04-16T17:16:53.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.891+0000 7fd2a2a00640 1 -- 172.21.15.45:0/3979021457 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fd29c044020 con 0x7fd2a8076530 2024-04-16T17:16:53.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.891+0000 7fd2b357a740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5557e617bf10 0x5557e617c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:53.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.891+0000 7fd2b357a740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5557e617c830 0x5557e6184ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:53.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.891+0000 7fd2b357a740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5557e6185410 0x5557e6187800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:53.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.891+0000 7fd2b357a740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5557e60e1020 con 0x5557e617bf10 2024-04-16T17:16:53.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.891+0000 7fd2b357a740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5557e6087460 con 0x5557e6185410 2024-04-16T17:16:53.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.891+0000 7fd2b357a740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5557e60dcfe0 con 0x5557e617c830 2024-04-16T17:16:53.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.892+0000 7fd2ae800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5557e617bf10 0x5557e617c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:53.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.892+0000 7fd2ae800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5557e617bf10 0x5557e617c2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56816/0 (socket says 172.21.15.45:56816) 2024-04-16T17:16:53.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.892+0000 7fd2ae800640 1 -- 172.21.15.45:0/3172921377 learned_addr learned my addr 172.21.15.45:0/3172921377 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:16:53.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.892+0000 7fd2ade00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5557e617c830 0x5557e6184ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:53.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.892+0000 7fd2ad400640 1 --2- 172.21.15.45:0/3172921377 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5557e6185410 0x5557e6187800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:53.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.892+0000 7fd2ae800640 1 -- 172.21.15.45:0/3172921377 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5557e617c830 msgr2=0x5557e6184ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:16:53.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.892+0000 7fd2ae800640 1 --2- 172.21.15.45:0/3172921377 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5557e617c830 0x5557e6184ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:53.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.892+0000 7fd2ae800640 1 -- 172.21.15.45:0/3172921377 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5557e6185410 msgr2=0x5557e6187800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:16:53.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.892+0000 7fd2ae800640 1 --2- 172.21.15.45:0/3172921377 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5557e6185410 0x5557e6187800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:53.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.892+0000 7fd2ae800640 1 -- 172.21.15.45:0/3172921377 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5557e608f040 con 0x5557e617bf10 2024-04-16T17:16:53.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.892+0000 7fd2ade00640 1 --2- 172.21.15.45:0/3172921377 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5557e617c830 0x5557e6184ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:16:53.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.892+0000 7fd2a2a00640 1 --2- 172.21.15.45:0/3979021457 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fd2900a3da0 0x7fd2900a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:53.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.893+0000 7fd2ae800640 1 --2- 172.21.15.45:0/3172921377 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5557e617bf10 0x5557e617c2f0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fd2a4009ef0 tx=0x7fd2a400f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9595ade0d540cb05 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:16:53.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.893+0000 7fd2a2a00640 1 -- 172.21.15.45:0/3979021457 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 9496+0+0 (secure 0 0 0) 0x7fd29c0f2490 con 0x7fd2a8076530 2024-04-16T17:16:53.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.893+0000 7fd2ade00640 1 --2- 172.21.15.45:0/3979021457 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fd2900a3da0 0x7fd2900a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:53.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.893+0000 7fd29b000640 1 -- 172.21.15.45:0/3172921377 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2a4015070 con 0x5557e617bf10 2024-04-16T17:16:53.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.893+0000 7fd29b000640 1 -- 172.21.15.45:0/3172921377 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd2a4011920 con 0x5557e617bf10 2024-04-16T17:16:53.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.893+0000 7fd29b000640 1 -- 172.21.15.45:0/3172921377 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2a40109a0 con 0x5557e617bf10 2024-04-16T17:16:53.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.894+0000 7fd2b357a740 1 -- 172.21.15.45:0/3172921377 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5557e617bf10 msgr2=0x5557e617c2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:53.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.894+0000 7fd2ade00640 1 --2- 172.21.15.45:0/3979021457 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fd2900a3da0 0x7fd2900a6260 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fd29c0f5fd0 tx=0x7fd29c0f5f40 comp rx=0 tx=0).ready entity=mgr.25411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:16:53.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.894+0000 7fd2b357a740 1 --2- 172.21.15.45:0/3172921377 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5557e617bf10 0x5557e617c2f0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fd2a4009ef0 tx=0x7fd2a400f730 comp rx=0 tx=0).stop 2024-04-16T17:16:53.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.894+0000 7fd2b357a740 1 -- 172.21.15.45:0/3172921377 shutdown_connections 2024-04-16T17:16:53.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.894+0000 7fd2b357a740 1 --2- 172.21.15.45:0/3172921377 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5557e617c830 0x5557e6184ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:53.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.894+0000 7fd2b357a740 1 --2- 172.21.15.45:0/3172921377 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5557e6185410 0x5557e6187800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:53.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.894+0000 7fd2b357a740 1 --2- 172.21.15.45:0/3172921377 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5557e617bf10 0x5557e617c2f0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:53.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.894+0000 7fd2b357a740 1 -- 172.21.15.45:0/3172921377 >> 172.21.15.45:0/3172921377 conn(0x5557e6176640 msgr2=0x5557e6176a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:16:53.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.894+0000 7fd2b357a740 1 -- 172.21.15.45:0/3172921377 shutdown_connections 2024-04-16T17:16:53.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.894+0000 7fd2b357a740 1 -- 172.21.15.45:0/3172921377 wait complete. 2024-04-16T17:16:53.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.894+0000 7fd2b357a740 1 -- start start 2024-04-16T17:16:53.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.895+0000 7fd2b357a740 1 Processor -- start 2024-04-16T17:16:53.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.895+0000 7fd2b357a740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5557e617bf10 0x5557e61824a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:53.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.895+0000 7fd2b357a740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5557e617c830 0x5557e61829e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:53.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.895+0000 7fd2b357a740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5557e6185410 0x5557e6182f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:53.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.895+0000 7fd2b357a740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5557e6188200 con 0x5557e617c830 2024-04-16T17:16:53.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.895+0000 7fd2b357a740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5557e609a830 con 0x5557e617bf10 2024-04-16T17:16:53.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.895+0000 7fd2b357a740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5557e6188380 con 0x5557e6185410 2024-04-16T17:16:53.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.895+0000 7fd2b357a740 10 client.0 _ll_register_callbacks cb 0x5557e60a2150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:16:53.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.895+0000 7fd2ad400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5557e617c830 0x5557e61829e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:53.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.895+0000 7fd2ae800640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5557e617bf10 0x5557e61824a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:53.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.895+0000 7fd2ad400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5557e617c830 0x5557e61829e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56826/0 (socket says 172.21.15.45:56826) 2024-04-16T17:16:53.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.895+0000 7fd2ad400640 1 -- 172.21.15.45:0/1481476544 learned_addr learned my addr 172.21.15.45:0/1481476544 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:16:53.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.896+0000 7fd2ae800640 1 --2- 172.21.15.45:0/1481476544 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5557e6185410 0x5557e6182f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:53.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.896+0000 7fd2ad400640 1 -- 172.21.15.45:0/1481476544 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5557e6185410 msgr2=0x5557e6182f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:16:53.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.896+0000 7fd2ad400640 1 --2- 172.21.15.45:0/1481476544 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5557e6185410 0x5557e6182f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:53.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.896+0000 7fd2ad400640 1 -- 172.21.15.45:0/1481476544 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5557e617bf10 msgr2=0x5557e61824a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:16:53.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.896+0000 7fd2ad400640 1 --2- 172.21.15.45:0/1481476544 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5557e617bf10 0x5557e61824a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:53.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.896+0000 7fd2ad400640 1 -- 172.21.15.45:0/1481476544 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5557e609e920 con 0x5557e617c830 2024-04-16T17:16:53.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.896+0000 7fd2ae800640 1 --2- 172.21.15.45:0/1481476544 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5557e6185410 0x5557e6182f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:16:53.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.896+0000 7fd2ad400640 1 --2- 172.21.15.45:0/1481476544 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5557e617c830 0x5557e61829e0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fd2940116f0 tx=0x7fd294011bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb84d6f80e6acf68 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:16:53.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.896+0000 7fd28f400640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-16T17:16:53.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.896+0000 7fd28f400640 1 -- 172.21.15.45:0/1481476544 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd29400fbf0 con 0x5557e617c830 2024-04-16T17:16:53.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.896+0000 7fd28f400640 1 -- 172.21.15.45:0/1481476544 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9496+0+0 (secure 0 0 0) 0x7fd294017050 con 0x5557e617c830 2024-04-16T17:16:53.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.897+0000 7fd2b357a740 1 -- 172.21.15.45:0/1481476544 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd2a4011920 con 0x5557e617c830 2024-04-16T17:16:53.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.897+0000 7fd28f400640 1 -- 172.21.15.45:0/1481476544 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fd294027050 con 0x5557e617c830 2024-04-16T17:16:53.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.897+0000 7fd2b357a740 10 client.18024 fetch_fsmap learned FSMap version 44 2024-04-16T17:16:53.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.897+0000 7fd2b357a740 1 -- 172.21.15.45:0/1481476544 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x5557e606c7f0 con 0x5557e617c830 2024-04-16T17:16:53.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.897+0000 7fd28f400640 1 -- 172.21.15.45:0/1481476544 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd2a4011920 con 0x5557e617c830 2024-04-16T17:16:53.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.897+0000 7fd28f400640 1 -- 172.21.15.45:0/1481476544 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 44) ==== 8346+0+0 (secure 0 0 0) 0x7fd294022030 con 0x5557e617c830 2024-04-16T17:16:53.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.897+0000 7fd2b357a740 10 client.18024 fetch_fsmap finished waiting for FSMap version 44 2024-04-16T17:16:53.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.897+0000 7fd2b357a740 10 client.18024 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-16T17:16:53.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.897+0000 7fd2b357a740 20 client.18024 populate_metadata read hostname 'smithi045' 2024-04-16T17:16:53.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.897+0000 7fd2b357a740 1 --2- 172.21.15.45:0/1481476544 >> [v2:172.21.15.117:6838/2378675949,v1:172.21.15.117:6839/2378675949] conn(0x5557e6195840 0x5557e6197ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:53.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.897+0000 7fd2b357a740 4 client.18024 mds_command: new command op to 24532 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:16:53.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.897+0000 7fd2b357a740 1 -- 172.21.15.45:0/1481476544 --> [v2:172.21.15.117:6838/2378675949,v1:172.21.15.117:6839/2378675949] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5557e60eea40 con 0x5557e6195840 2024-04-16T17:16:53.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.898+0000 7fd2ae800640 1 --2- 172.21.15.45:0/1481476544 >> [v2:172.21.15.117:6838/2378675949,v1:172.21.15.117:6839/2378675949] conn(0x5557e6195840 0x5557e6197ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:53.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.899+0000 7fd2ae800640 1 --2- 172.21.15.45:0/1481476544 >> [v2:172.21.15.117:6838/2378675949,v1:172.21.15.117:6839/2378675949] conn(0x5557e6195840 0x5557e6197ca0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=23924b82a4ee4302 server_cookie=a02867aef8bef569 in_seq=0 out_seq=0 2024-04-16T17:16:53.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.899+0000 7fd28f400640 10 client.18024 ms_handle_connect on v2:172.21.15.117:6838/2378675949 2024-04-16T17:16:53.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.899+0000 7fd28f400640 1 -- 172.21.15.45:0/1481476544 <== mds.2 v2:172.21.15.117:6838/2378675949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5557e60eea40 con 0x5557e6195840 2024-04-16T17:16:53.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.899+0000 7fd28f400640 10 client.18024 handle_command_reply: tid=0 2024-04-16T17:16:53.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.899+0000 7fd2b357a740 1 client.18024 shutdown 2024-04-16T17:16:53.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.900+0000 7fd2b357a740 1 -- 172.21.15.45:0/1481476544 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5557e617c830 msgr2=0x5557e61829e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:53.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.900+0000 7fd2b357a740 1 --2- 172.21.15.45:0/1481476544 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5557e617c830 0x5557e61829e0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fd2940116f0 tx=0x7fd294011bb0 comp rx=0 tx=0).stop 2024-04-16T17:16:53.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.900+0000 7fd2b357a740 1 -- 172.21.15.45:0/1481476544 shutdown_connections 2024-04-16T17:16:53.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.900+0000 7fd2b357a740 1 --2- 172.21.15.45:0/1481476544 >> [v2:172.21.15.117:6838/2378675949,v1:172.21.15.117:6839/2378675949] conn(0x5557e6195840 0x5557e6197ca0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:53.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.900+0000 7fd2b357a740 1 --2- 172.21.15.45:0/1481476544 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5557e6185410 0x5557e6182f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:53.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.900+0000 7fd2b357a740 1 --2- 172.21.15.45:0/1481476544 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5557e617c830 0x5557e61829e0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:53.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.900+0000 7fd2b357a740 1 --2- 172.21.15.45:0/1481476544 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5557e617bf10 0x5557e61824a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:53.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.900+0000 7fd28f400640 0 client.18024 ms_handle_reset on v2:172.21.15.117:6838/2378675949 2024-04-16T17:16:53.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.900+0000 7fd2b357a740 1 -- 172.21.15.45:0/1481476544 >> 172.21.15.45:0/1481476544 conn(0x5557e6176640 msgr2=0x5557e6178390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:16:53.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.900+0000 7fd2b357a740 1 -- 172.21.15.45:0/1481476544 shutdown_connections 2024-04-16T17:16:53.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.900+0000 7fd2b357a740 1 -- 172.21.15.45:0/1481476544 wait complete. 2024-04-16T17:16:53.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.900+0000 7fd2b357a740 20 client.18024 trim_cache size 0 max 16384 2024-04-16T17:16:53.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.915+0000 7fd2b357a740 1 Processor -- start 2024-04-16T17:16:53.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.915+0000 7fd2b357a740 1 -- start start 2024-04-16T17:16:53.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.915+0000 7fd2b357a740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5557e617bf10 0x5557e6184bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:53.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.915+0000 7fd2b357a740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5557e617c830 0x5557e61910c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:53.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.915+0000 7fd2b357a740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5557e6185410 0x5557e619d570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:53.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.915+0000 7fd2b357a740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fd2a81b82b0 con 0x5557e6185410 2024-04-16T17:16:53.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.915+0000 7fd2b357a740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fd2a81b5b80 con 0x5557e617bf10 2024-04-16T17:16:53.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.915+0000 7fd2b357a740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fd2a81b39f0 con 0x5557e617c830 2024-04-16T17:16:53.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.915+0000 7fd2ae800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5557e6185410 0x5557e619d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:53.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.915+0000 7fd2ae800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5557e6185410 0x5557e619d570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56836/0 (socket says 172.21.15.45:56836) 2024-04-16T17:16:53.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.915+0000 7fd2ae800640 1 -- 172.21.15.45:0/3811847502 learned_addr learned my addr 172.21.15.45:0/3811847502 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:16:53.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.915+0000 7fd2ad400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5557e617c830 0x5557e61910c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:53.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.915+0000 7fd2ae800640 1 --2- 172.21.15.45:0/3811847502 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5557e617bf10 0x5557e6184bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:53.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.916+0000 7fd2ad400640 1 -- 172.21.15.45:0/3811847502 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5557e617bf10 msgr2=0x5557e6184bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:16:53.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.916+0000 7fd2ad400640 1 --2- 172.21.15.45:0/3811847502 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5557e617bf10 0x5557e6184bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:53.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.916+0000 7fd2ad400640 1 -- 172.21.15.45:0/3811847502 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5557e6185410 msgr2=0x5557e619d570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:16:53.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.916+0000 7fd2ad400640 1 --2- 172.21.15.45:0/3811847502 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5557e6185410 0x5557e619d570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:53.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.916+0000 7fd2ad400640 1 -- 172.21.15.45:0/3811847502 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5557e617c660 con 0x5557e617c830 2024-04-16T17:16:53.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.916+0000 7fd2ae800640 1 --2- 172.21.15.45:0/3811847502 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5557e6185410 0x5557e619d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:16:53.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.916+0000 7fd2ad400640 1 --2- 172.21.15.45:0/3811847502 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5557e617c830 0x5557e61910c0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fd294001eb0 tx=0x7fd294009460 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9759ea3942db87ea server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:16:53.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.916+0000 7fd29a600640 1 -- 172.21.15.45:0/3811847502 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd294000e60 con 0x5557e617c830 2024-04-16T17:16:53.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.916+0000 7fd29a600640 1 -- 172.21.15.45:0/3811847502 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd294019620 con 0x5557e617c830 2024-04-16T17:16:53.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.916+0000 7fd29a600640 1 -- 172.21.15.45:0/3811847502 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2940050b0 con 0x5557e617c830 2024-04-16T17:16:53.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.917+0000 7fd2b357a740 1 -- 172.21.15.45:0/3811847502 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5557e617c830 msgr2=0x5557e61910c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:53.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.917+0000 7fd2b357a740 1 --2- 172.21.15.45:0/3811847502 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5557e617c830 0x5557e61910c0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fd294001eb0 tx=0x7fd294009460 comp rx=0 tx=0).stop 2024-04-16T17:16:53.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.917+0000 7fd2b357a740 1 -- 172.21.15.45:0/3811847502 shutdown_connections 2024-04-16T17:16:53.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.917+0000 7fd2b357a740 1 --2- 172.21.15.45:0/3811847502 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5557e617c830 0x5557e61910c0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:53.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.917+0000 7fd2b357a740 1 --2- 172.21.15.45:0/3811847502 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5557e6185410 0x5557e619d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:53.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.917+0000 7fd2b357a740 1 --2- 172.21.15.45:0/3811847502 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5557e617bf10 0x5557e6184bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:53.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.917+0000 7fd2b357a740 1 -- 172.21.15.45:0/3811847502 >> 172.21.15.45:0/3811847502 conn(0x5557e6176640 msgr2=0x5557e61782e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:16:53.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.917+0000 7fd2b357a740 1 -- 172.21.15.45:0/3811847502 shutdown_connections 2024-04-16T17:16:53.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.917+0000 7fd2b357a740 1 -- 172.21.15.45:0/3811847502 wait complete. 2024-04-16T17:16:53.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.918+0000 7fd2b357a740 1 -- start start 2024-04-16T17:16:53.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.918+0000 7fd2b357a740 1 Processor -- start 2024-04-16T17:16:53.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.918+0000 7fd2b357a740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5557e617bf10 0x5557e617f460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:53.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.918+0000 7fd2b357a740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5557e617c830 0x5557e617f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:53.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.918+0000 7fd2b357a740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5557e6185410 0x5557e61fade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:53.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.918+0000 7fd2b357a740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fd2a81b4040 con 0x5557e617c830 2024-04-16T17:16:53.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.918+0000 7fd2b357a740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fd2a81b54b0 con 0x5557e6185410 2024-04-16T17:16:53.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.918+0000 7fd2b357a740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fd2a81b4710 con 0x5557e617bf10 2024-04-16T17:16:53.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.918+0000 7fd2b357a740 10 client.0 _ll_register_callbacks cb 0x7fd294009890 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:16:53.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.919+0000 7fd2ad400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5557e617c830 0x5557e617f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:53.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.919+0000 7fd2ad400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5557e617c830 0x5557e617f9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56850/0 (socket says 172.21.15.45:56850) 2024-04-16T17:16:53.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.919+0000 7fd2ad400640 1 -- 172.21.15.45:0/1230428097 learned_addr learned my addr 172.21.15.45:0/1230428097 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:16:53.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.919+0000 7fd2ae800640 1 --2- 172.21.15.45:0/1230428097 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5557e6185410 0x5557e61fade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:53.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.919+0000 7fd2ae800640 1 --2- 172.21.15.45:0/1230428097 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5557e617bf10 0x5557e617f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:53.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.919+0000 7fd2ad400640 1 -- 172.21.15.45:0/1230428097 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5557e617bf10 msgr2=0x5557e617f460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:16:53.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.919+0000 7fd2ad400640 1 --2- 172.21.15.45:0/1230428097 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5557e617bf10 0x5557e617f460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:53.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.919+0000 7fd2ad400640 1 -- 172.21.15.45:0/1230428097 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5557e6185410 msgr2=0x5557e61fade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:16:53.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.919+0000 7fd2ad400640 1 --2- 172.21.15.45:0/1230428097 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5557e6185410 0x5557e61fade0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:53.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.919+0000 7fd2ad400640 1 -- 172.21.15.45:0/1230428097 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5557e5df33f0 con 0x5557e617c830 2024-04-16T17:16:53.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.919+0000 7fd2ae800640 1 --2- 172.21.15.45:0/1230428097 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5557e6185410 0x5557e61fade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:16:53.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.919+0000 7fd2ad400640 1 --2- 172.21.15.45:0/1230428097 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5557e617c830 0x5557e617f9a0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fd294006010 tx=0x7fd294009700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81c2c9912f75c78f server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:16:53.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.919+0000 7fd28f400640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-16T17:16:53.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.920+0000 7fd28f400640 1 -- 172.21.15.45:0/1230428097 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2940052f0 con 0x5557e617c830 2024-04-16T17:16:53.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.920+0000 7fd2b357a740 1 -- 172.21.15.45:0/1230428097 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd294019620 con 0x5557e617c830 2024-04-16T17:16:53.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.920+0000 7fd28f400640 1 -- 172.21.15.45:0/1230428097 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9496+0+0 (secure 0 0 0) 0x7fd294005490 con 0x5557e617c830 2024-04-16T17:16:53.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.920+0000 7fd28f400640 1 -- 172.21.15.45:0/1230428097 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fd2940197d0 con 0x5557e617c830 2024-04-16T17:16:53.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.920+0000 7fd2b357a740 10 client.18030 fetch_fsmap learned FSMap version 44 2024-04-16T17:16:53.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.920+0000 7fd2b357a740 1 -- 172.21.15.45:0/1230428097 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x5557e5ebd7d0 con 0x5557e617c830 2024-04-16T17:16:53.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.920+0000 7fd28f400640 1 -- 172.21.15.45:0/1230428097 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd294019620 con 0x5557e617c830 2024-04-16T17:16:53.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.920+0000 7fd28f400640 1 -- 172.21.15.45:0/1230428097 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 44) ==== 8346+0+0 (secure 0 0 0) 0x7fd294022070 con 0x5557e617c830 2024-04-16T17:16:53.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.920+0000 7fd2b357a740 10 client.18030 fetch_fsmap finished waiting for FSMap version 44 2024-04-16T17:16:53.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.920+0000 7fd2b357a740 10 client.18030 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-16T17:16:53.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.920+0000 7fd2b357a740 20 client.18030 populate_metadata read hostname 'smithi045' 2024-04-16T17:16:53.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.920+0000 7fd2b357a740 1 --2- 172.21.15.45:0/1230428097 >> [v2:172.21.15.117:6838/2378675949,v1:172.21.15.117:6839/2378675949] conn(0x5557e61fc700 0x5557e61feaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:53.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.920+0000 7fd2b357a740 4 client.18030 mds_command: new command op to 24532 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T17:16:53.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.920+0000 7fd2b357a740 1 -- 172.21.15.45:0/1230428097 --> [v2:172.21.15.117:6838/2378675949,v1:172.21.15.117:6839/2378675949] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5557e60e88b0 con 0x5557e61fc700 2024-04-16T17:16:53.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.921+0000 7fd2ae800640 1 --2- 172.21.15.45:0/1230428097 >> [v2:172.21.15.117:6838/2378675949,v1:172.21.15.117:6839/2378675949] conn(0x5557e61fc700 0x5557e61feaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:53.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.921+0000 7fd2ae800640 1 --2- 172.21.15.45:0/1230428097 >> [v2:172.21.15.117:6838/2378675949,v1:172.21.15.117:6839/2378675949] conn(0x5557e61fc700 0x5557e61feaf0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=fca50eddfe55bb75 server_cookie=b189741a2827a0d0 in_seq=0 out_seq=0 2024-04-16T17:16:53.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.921+0000 7fd28f400640 10 client.18030 ms_handle_connect on v2:172.21.15.117:6838/2378675949 2024-04-16T17:16:53.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.922+0000 7fd28f400640 1 -- 172.21.15.45:0/1230428097 <== mds.2 v2:172.21.15.117:6838/2378675949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5557e60e88b0 con 0x5557e61fc700 2024-04-16T17:16:53.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.922+0000 7fd28f400640 10 client.18030 handle_command_reply: tid=0 2024-04-16T17:16:53.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.922+0000 7fd2b357a740 1 client.18030 shutdown 2024-04-16T17:16:53.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.922+0000 7fd2b357a740 1 -- 172.21.15.45:0/1230428097 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5557e617c830 msgr2=0x5557e617f9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:53.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.922+0000 7fd2b357a740 1 --2- 172.21.15.45:0/1230428097 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5557e617c830 0x5557e617f9a0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fd294006010 tx=0x7fd294009700 comp rx=0 tx=0).stop 2024-04-16T17:16:53.929 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.922+0000 7fd2b357a740 1 -- 172.21.15.45:0/1230428097 shutdown_connections 2024-04-16T17:16:53.929 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.922+0000 7fd2b357a740 1 --2- 172.21.15.45:0/1230428097 >> [v2:172.21.15.117:6838/2378675949,v1:172.21.15.117:6839/2378675949] conn(0x5557e61fc700 0x5557e61feaf0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:53.929 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.922+0000 7fd2b357a740 1 --2- 172.21.15.45:0/1230428097 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5557e6185410 0x5557e61fade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:53.929 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.922+0000 7fd28f400640 0 client.18030 ms_handle_reset on v2:172.21.15.117:6838/2378675949 2024-04-16T17:16:53.929 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.922+0000 7fd2b357a740 1 --2- 172.21.15.45:0/1230428097 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5557e617c830 0x5557e617f9a0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:53.929 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.922+0000 7fd2b357a740 1 --2- 172.21.15.45:0/1230428097 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5557e617bf10 0x5557e617f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:53.929 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.922+0000 7fd2b357a740 1 -- 172.21.15.45:0/1230428097 >> 172.21.15.45:0/1230428097 conn(0x5557e6176640 msgr2=0x5557e6174e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:16:53.929 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.922+0000 7fd2b357a740 1 -- 172.21.15.45:0/1230428097 shutdown_connections 2024-04-16T17:16:53.929 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.922+0000 7fd2b357a740 1 -- 172.21.15.45:0/1230428097 wait complete. 2024-04-16T17:16:53.930 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.922+0000 7fd2b357a740 20 client.18030 trim_cache size 0 max 16384 2024-04-16T17:16:53.930 INFO:teuthology.orchestra.run.smithi045.stdout:[] 2024-04-16T17:16:53.930 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.923+0000 7fd2af200640 1 -- 172.21.15.45:0/3979021457 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fd2900a3da0 msgr2=0x7fd2900a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:53.931 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.923+0000 7fd2af200640 1 --2- 172.21.15.45:0/3979021457 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fd2900a3da0 0x7fd2900a6260 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fd29c0f5fd0 tx=0x7fd29c0f5f40 comp rx=0 tx=0).stop 2024-04-16T17:16:53.931 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.923+0000 7fd2af200640 1 -- 172.21.15.45:0/3979021457 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd2a8076530 msgr2=0x7fd2a819ee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:53.931 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.923+0000 7fd2af200640 1 --2- 172.21.15.45:0/3979021457 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd2a8076530 0x7fd2a819ee10 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fd29c0043c0 tx=0x7fd29c0043f0 comp rx=0 tx=0).stop 2024-04-16T17:16:53.931 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.923+0000 7fd2af200640 1 -- 172.21.15.45:0/3979021457 shutdown_connections 2024-04-16T17:16:53.931 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.923+0000 7fd2af200640 1 --2- 172.21.15.45:0/3979021457 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fd2900a3da0 0x7fd2900a6260 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:53.931 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.923+0000 7fd2af200640 1 --2- 172.21.15.45:0/3979021457 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd2a8112d40 0x7fd2a81a36d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:53.931 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.923+0000 7fd2af200640 1 --2- 172.21.15.45:0/3979021457 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd2a8076e50 0x7fd2a819f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:53.931 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.923+0000 7fd2af200640 1 --2- 172.21.15.45:0/3979021457 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd2a8076530 0x7fd2a819ee10 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:53.931 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.923+0000 7fd2af200640 1 -- 172.21.15.45:0/3979021457 >> 172.21.15.45:0/3979021457 conn(0x7fd2a80ffb70 msgr2=0x7fd2a8101f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:16:53.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.923+0000 7fd2af200640 1 -- 172.21.15.45:0/3979021457 shutdown_connections 2024-04-16T17:16:53.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:53.923+0000 7fd2af200640 1 -- 172.21.15.45:0/3979021457 wait complete. 2024-04-16T17:16:53.943 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-16T17:16:54.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.166+0000 7f8d20a00640 1 -- 172.21.15.45:0/4095712667 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8d1c102ac0 msgr2=0x7f8d1c106b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:54.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.166+0000 7f8d20a00640 1 --2- 172.21.15.45:0/4095712667 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8d1c102ac0 0x7f8d1c106b00 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f8d08009860 tx=0x7f8d0802bb70 comp rx=0 tx=0).stop 2024-04-16T17:16:54.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.166+0000 7f8d20a00640 1 -- 172.21.15.45:0/4095712667 shutdown_connections 2024-04-16T17:16:54.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.166+0000 7f8d20a00640 1 --2- 172.21.15.45:0/4095712667 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8d1c1071e0 0x7f8d1c10edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:54.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.166+0000 7f8d20a00640 1 --2- 172.21.15.45:0/4095712667 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8d1c102ac0 0x7f8d1c106b00 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:54.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.166+0000 7f8d20a00640 1 --2- 172.21.15.45:0/4095712667 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8d1c102110 0x7f8d1c1024f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:54.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.166+0000 7f8d20a00640 1 -- 172.21.15.45:0/4095712667 >> 172.21.15.45:0/4095712667 conn(0x7f8d1c0fd9d0 msgr2=0x7f8d1c0ffdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:16:54.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.166+0000 7f8d20a00640 1 -- 172.21.15.45:0/4095712667 shutdown_connections 2024-04-16T17:16:54.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.166+0000 7f8d20a00640 1 -- 172.21.15.45:0/4095712667 wait complete. 2024-04-16T17:16:54.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.167+0000 7f8d20a00640 1 Processor -- start 2024-04-16T17:16:54.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.168+0000 7f8d20a00640 1 -- start start 2024-04-16T17:16:54.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.168+0000 7f8d20a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8d1c102110 0x7f8d1c1a0670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:54.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.168+0000 7f8d20a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8d1c1071e0 0x7f8d1c1a0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:54.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.168+0000 7f8d20a00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8d1c19a7e0 0x7f8d1c19ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:54.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.168+0000 7f8d20a00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f8d1c112270 con 0x7f8d1c102110 2024-04-16T17:16:54.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.168+0000 7f8d20a00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f8d1c1120f0 con 0x7f8d1c19a7e0 2024-04-16T17:16:54.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.168+0000 7f8d20a00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f8d1c1123f0 con 0x7f8d1c1071e0 2024-04-16T17:16:54.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.168+0000 7f8d1b400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8d1c102110 0x7f8d1c1a0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:54.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.168+0000 7f8d1b400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8d1c102110 0x7f8d1c1a0670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56870/0 (socket says 172.21.15.45:56870) 2024-04-16T17:16:54.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.168+0000 7f8d1b400640 1 -- 172.21.15.45:0/643278657 learned_addr learned my addr 172.21.15.45:0/643278657 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:16:54.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.168+0000 7f8d1aa00640 1 --2- 172.21.15.45:0/643278657 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8d1c1071e0 0x7f8d1c1a0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:54.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.169+0000 7f8d1be00640 1 --2- 172.21.15.45:0/643278657 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8d1c19a7e0 0x7f8d1c19ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:54.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.169+0000 7f8d1b400640 1 -- 172.21.15.45:0/643278657 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8d1c1071e0 msgr2=0x7f8d1c1a0bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:54.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.169+0000 7f8d1b400640 1 --2- 172.21.15.45:0/643278657 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8d1c1071e0 0x7f8d1c1a0bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:54.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.169+0000 7f8d1b400640 1 -- 172.21.15.45:0/643278657 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8d1c19a7e0 msgr2=0x7f8d1c19ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:54.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.169+0000 7f8d1b400640 1 --2- 172.21.15.45:0/643278657 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8d1c19a7e0 0x7f8d1c19ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:54.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.169+0000 7f8d1b400640 1 -- 172.21.15.45:0/643278657 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d1c19b370 con 0x7f8d1c102110 2024-04-16T17:16:54.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.169+0000 7f8d1be00640 1 --2- 172.21.15.45:0/643278657 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8d1c19a7e0 0x7f8d1c19ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:16:54.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.169+0000 7f8d1b400640 1 --2- 172.21.15.45:0/643278657 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8d1c102110 0x7f8d1c1a0670 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f8d0c00eb90 tx=0x7f8d0c00c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:16:54.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.169+0000 7f8d13e00640 1 -- 172.21.15.45:0/643278657 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d0c004d20 con 0x7f8d1c102110 2024-04-16T17:16:54.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.169+0000 7f8d20a00640 1 -- 172.21.15.45:0/643278657 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d1c19b660 con 0x7f8d1c102110 2024-04-16T17:16:54.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.169+0000 7f8d13e00640 1 -- 172.21.15.45:0/643278657 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8d0c00ee90 con 0x7f8d1c102110 2024-04-16T17:16:54.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.169+0000 7f8d13e00640 1 -- 172.21.15.45:0/643278657 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d0c010830 con 0x7f8d1c102110 2024-04-16T17:16:54.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.170+0000 7f8d20a00640 1 -- 172.21.15.45:0/643278657 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d1c1a7810 con 0x7f8d1c102110 2024-04-16T17:16:54.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.173+0000 7f8d24c83740 1 Processor -- start 2024-04-16T17:16:54.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.173+0000 7f8d24c83740 1 -- start start 2024-04-16T17:16:54.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.173+0000 7f8d13e00640 1 -- 172.21.15.45:0/643278657 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f8d0c005020 con 0x7f8d1c102110 2024-04-16T17:16:54.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.173+0000 7f8d24c83740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c0ee6d5f10 0x55c0ee6d62f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:54.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.173+0000 7f8d24c83740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c0ee6d6830 0x55c0ee6deed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:54.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.173+0000 7f8d24c83740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c0ee6df410 0x55c0ee6e1800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:54.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.173+0000 7f8d24c83740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55c0ee63b020 con 0x55c0ee6d6830 2024-04-16T17:16:54.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.173+0000 7f8d24c83740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55c0ee5e1460 con 0x55c0ee6d5f10 2024-04-16T17:16:54.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.173+0000 7f8d24c83740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55c0ee636fe0 con 0x55c0ee6df410 2024-04-16T17:16:54.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.173+0000 7f8d1be00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c0ee6d5f10 0x55c0ee6d62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:54.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.173+0000 7f8d1be00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c0ee6d5f10 0x55c0ee6d62f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:45422/0 (socket says 172.21.15.45:45422) 2024-04-16T17:16:54.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.173+0000 7f8d1be00640 1 -- 172.21.15.45:0/1189224641 learned_addr learned my addr 172.21.15.45:0/1189224641 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:16:54.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.173+0000 7f8d1b400640 1 --2- 172.21.15.45:0/1189224641 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c0ee6d6830 0x55c0ee6deed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:54.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.173+0000 7f8d1aa00640 1 --2- 172.21.15.45:0/1189224641 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c0ee6df410 0x55c0ee6e1800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:54.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.174+0000 7f8d1be00640 1 -- 172.21.15.45:0/1189224641 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c0ee6df410 msgr2=0x55c0ee6e1800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:16:54.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.174+0000 7f8d1be00640 1 --2- 172.21.15.45:0/1189224641 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c0ee6df410 0x55c0ee6e1800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:54.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.174+0000 7f8d1be00640 1 -- 172.21.15.45:0/1189224641 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c0ee6d6830 msgr2=0x55c0ee6deed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:16:54.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.174+0000 7f8d1be00640 1 --2- 172.21.15.45:0/1189224641 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c0ee6d6830 0x55c0ee6deed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:54.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.174+0000 7f8d1be00640 1 -- 172.21.15.45:0/1189224641 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c0ee5e9040 con 0x55c0ee6d5f10 2024-04-16T17:16:54.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.174+0000 7f8d1be00640 1 --2- 172.21.15.45:0/1189224641 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c0ee6d5f10 0x55c0ee6d62f0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f8d14009c60 tx=0x7f8d140107a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1a8a6efc82d1315 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:16:54.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.174+0000 7f8d07e00640 1 -- 172.21.15.45:0/1189224641 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d1401c480 con 0x55c0ee6d5f10 2024-04-16T17:16:54.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.175+0000 7f8d13e00640 1 --2- 172.21.15.45:0/643278657 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f8d000a3d20 0x7f8d000a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:54.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.175+0000 7f8d07e00640 1 -- 172.21.15.45:0/1189224641 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8d1401cb00 con 0x55c0ee6d5f10 2024-04-16T17:16:54.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.175+0000 7f8d07e00640 1 -- 172.21.15.45:0/1189224641 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d14011b50 con 0x55c0ee6d5f10 2024-04-16T17:16:54.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.175+0000 7f8d13e00640 1 -- 172.21.15.45:0/643278657 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 9496+0+0 (secure 0 0 0) 0x7f8d0c0d12f0 con 0x7f8d1c102110 2024-04-16T17:16:54.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.175+0000 7f8d1b400640 1 --2- 172.21.15.45:0/643278657 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f8d000a3d20 0x7f8d000a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:54.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.175+0000 7f8d24c83740 1 -- 172.21.15.45:0/1189224641 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c0ee6d5f10 msgr2=0x55c0ee6d62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:54.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.175+0000 7f8d24c83740 1 --2- 172.21.15.45:0/1189224641 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c0ee6d5f10 0x55c0ee6d62f0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f8d14009c60 tx=0x7f8d140107a0 comp rx=0 tx=0).stop 2024-04-16T17:16:54.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.175+0000 7f8d1b400640 1 --2- 172.21.15.45:0/643278657 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f8d000a3d20 0x7f8d000a61e0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f8d0c0ce720 tx=0x7f8d0c0d3800 comp rx=0 tx=0).ready entity=mgr.25411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:16:54.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.175+0000 7f8d24c83740 1 -- 172.21.15.45:0/1189224641 shutdown_connections 2024-04-16T17:16:54.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.176+0000 7f8d24c83740 1 --2- 172.21.15.45:0/1189224641 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c0ee6df410 0x55c0ee6e1800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:54.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.176+0000 7f8d24c83740 1 --2- 172.21.15.45:0/1189224641 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c0ee6d6830 0x55c0ee6deed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:54.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.176+0000 7f8d24c83740 1 --2- 172.21.15.45:0/1189224641 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c0ee6d5f10 0x55c0ee6d62f0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:54.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.176+0000 7f8d24c83740 1 -- 172.21.15.45:0/1189224641 >> 172.21.15.45:0/1189224641 conn(0x55c0ee6d0640 msgr2=0x55c0ee6d0a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:16:54.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.176+0000 7f8d24c83740 1 -- 172.21.15.45:0/1189224641 shutdown_connections 2024-04-16T17:16:54.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.176+0000 7f8d24c83740 1 -- 172.21.15.45:0/1189224641 wait complete. 2024-04-16T17:16:54.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.176+0000 7f8d24c83740 1 -- start start 2024-04-16T17:16:54.184 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.176+0000 7f8d24c83740 1 Processor -- start 2024-04-16T17:16:54.184 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.177+0000 7f8d24c83740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c0ee6d5f10 0x55c0ee6dc520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:54.184 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.177+0000 7f8d24c83740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c0ee6d6830 0x55c0ee6dca60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:54.184 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.177+0000 7f8d24c83740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c0ee6df410 0x55c0ee6dcfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:54.184 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.177+0000 7f8d24c83740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55c0ee6e2200 con 0x55c0ee6d5f10 2024-04-16T17:16:54.184 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.177+0000 7f8d24c83740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55c0ee5f4830 con 0x55c0ee6d6830 2024-04-16T17:16:54.184 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.177+0000 7f8d24c83740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55c0ee6e2380 con 0x55c0ee6df410 2024-04-16T17:16:54.184 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.177+0000 7f8d24c83740 10 client.0 _ll_register_callbacks cb 0x55c0ee5fc150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:16:54.184 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.177+0000 7f8d1be00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c0ee6d5f10 0x55c0ee6dc520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:54.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.177+0000 7f8d1be00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c0ee6d5f10 0x55c0ee6dc520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56880/0 (socket says 172.21.15.45:56880) 2024-04-16T17:16:54.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.177+0000 7f8d1be00640 1 -- 172.21.15.45:0/3106556747 learned_addr learned my addr 172.21.15.45:0/3106556747 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:16:54.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.177+0000 7f8d1aa00640 1 --2- 172.21.15.45:0/3106556747 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c0ee6d6830 0x55c0ee6dca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:54.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.177+0000 7f8d1be00640 1 --2- 172.21.15.45:0/3106556747 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c0ee6df410 0x55c0ee6dcfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:54.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.177+0000 7f8d1be00640 1 -- 172.21.15.45:0/3106556747 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c0ee6df410 msgr2=0x55c0ee6dcfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:16:54.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.177+0000 7f8d1be00640 1 --2- 172.21.15.45:0/3106556747 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c0ee6df410 0x55c0ee6dcfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:54.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.177+0000 7f8d1be00640 1 -- 172.21.15.45:0/3106556747 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c0ee6d6830 msgr2=0x55c0ee6dca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:16:54.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.177+0000 7f8d1be00640 1 --2- 172.21.15.45:0/3106556747 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c0ee6d6830 0x55c0ee6dca60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:54.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.177+0000 7f8d1be00640 1 -- 172.21.15.45:0/3106556747 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c0ee5f8920 con 0x55c0ee6d5f10 2024-04-16T17:16:54.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.178+0000 7f8d1be00640 1 --2- 172.21.15.45:0/3106556747 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c0ee6d5f10 0x55c0ee6dc520 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x55c0ee6cfec0 tx=0x7f8d14011830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93e7c6bd5bb4ce89 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:16:54.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.178+0000 7f8d04c00640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-16T17:16:54.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.178+0000 7f8d04c00640 1 -- 172.21.15.45:0/3106556747 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d14011b40 con 0x55c0ee6d5f10 2024-04-16T17:16:54.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.178+0000 7f8d04c00640 1 -- 172.21.15.45:0/3106556747 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9496+0+0 (secure 0 0 0) 0x7f8d14011ce0 con 0x55c0ee6d5f10 2024-04-16T17:16:54.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.178+0000 7f8d24c83740 1 -- 172.21.15.45:0/3106556747 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8d1401cb00 con 0x55c0ee6d5f10 2024-04-16T17:16:54.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.178+0000 7f8d04c00640 1 -- 172.21.15.45:0/3106556747 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f8d14034050 con 0x55c0ee6d5f10 2024-04-16T17:16:54.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.178+0000 7f8d24c83740 10 client.18042 fetch_fsmap learned FSMap version 44 2024-04-16T17:16:54.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.178+0000 7f8d24c83740 1 -- 172.21.15.45:0/3106556747 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55c0ee5c67f0 con 0x55c0ee6d5f10 2024-04-16T17:16:54.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.179+0000 7f8d04c00640 1 -- 172.21.15.45:0/3106556747 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8d1401cb00 con 0x55c0ee6d5f10 2024-04-16T17:16:54.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.179+0000 7f8d04c00640 1 -- 172.21.15.45:0/3106556747 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 44) ==== 8346+0+0 (secure 0 0 0) 0x7f8d14029020 con 0x55c0ee6d5f10 2024-04-16T17:16:54.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.179+0000 7f8d24c83740 10 client.18042 fetch_fsmap finished waiting for FSMap version 44 2024-04-16T17:16:54.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.179+0000 7f8d24c83740 10 client.18042 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-04-16T17:16:54.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.179+0000 7f8d24c83740 20 client.18042 populate_metadata read hostname 'smithi045' 2024-04-16T17:16:54.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.179+0000 7f8d24c83740 1 --2- 172.21.15.45:0/3106556747 >> [v2:172.21.15.45:6840/3755310822,v1:172.21.15.45:6841/3755310822] conn(0x55c0ee6ef810 0x55c0ee6f1c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:54.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.179+0000 7f8d24c83740 4 client.18042 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:16:54.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.179+0000 7f8d24c83740 1 -- 172.21.15.45:0/3106556747 --> [v2:172.21.15.45:6840/3755310822,v1:172.21.15.45:6841/3755310822] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c0ee648a40 con 0x55c0ee6ef810 2024-04-16T17:16:54.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.179+0000 7f8d1aa00640 1 --2- 172.21.15.45:0/3106556747 >> [v2:172.21.15.45:6840/3755310822,v1:172.21.15.45:6841/3755310822] conn(0x55c0ee6ef810 0x55c0ee6f1c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:54.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.180+0000 7f8d04c00640 10 client.18042 ms_handle_connect on v2:172.21.15.45:6840/3755310822 2024-04-16T17:16:54.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.180+0000 7f8d1aa00640 1 --2- 172.21.15.45:0/3106556747 >> [v2:172.21.15.45:6840/3755310822,v1:172.21.15.45:6841/3755310822] conn(0x55c0ee6ef810 0x55c0ee6f1c00 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=189ba2a7413915d0 server_cookie=f949a340bdd80aa in_seq=0 out_seq=0 2024-04-16T17:16:54.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.181+0000 7f8d04c00640 1 -- 172.21.15.45:0/3106556747 <== mds.1 v2:172.21.15.45:6840/3755310822 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c0ee648a40 con 0x55c0ee6ef810 2024-04-16T17:16:54.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.181+0000 7f8d04c00640 10 client.18042 handle_command_reply: tid=0 2024-04-16T17:16:54.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.181+0000 7f8d24c83740 1 client.18042 shutdown 2024-04-16T17:16:54.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.181+0000 7f8d24c83740 1 -- 172.21.15.45:0/3106556747 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c0ee6d5f10 msgr2=0x55c0ee6dc520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:54.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.181+0000 7f8d24c83740 1 --2- 172.21.15.45:0/3106556747 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c0ee6d5f10 0x55c0ee6dc520 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x55c0ee6cfec0 tx=0x7f8d14011830 comp rx=0 tx=0).stop 2024-04-16T17:16:54.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.181+0000 7f8d24c83740 1 -- 172.21.15.45:0/3106556747 shutdown_connections 2024-04-16T17:16:54.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.181+0000 7f8d24c83740 1 --2- 172.21.15.45:0/3106556747 >> [v2:172.21.15.45:6840/3755310822,v1:172.21.15.45:6841/3755310822] conn(0x55c0ee6ef810 0x55c0ee6f1c00 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:54.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.181+0000 7f8d24c83740 1 --2- 172.21.15.45:0/3106556747 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c0ee6df410 0x55c0ee6dcfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:54.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.181+0000 7f8d24c83740 1 --2- 172.21.15.45:0/3106556747 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c0ee6d6830 0x55c0ee6dca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:54.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.181+0000 7f8d24c83740 1 --2- 172.21.15.45:0/3106556747 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c0ee6d5f10 0x55c0ee6dc520 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:54.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.181+0000 7f8d04c00640 0 client.18042 ms_handle_reset on v2:172.21.15.45:6840/3755310822 2024-04-16T17:16:54.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.181+0000 7f8d24c83740 1 -- 172.21.15.45:0/3106556747 >> 172.21.15.45:0/3106556747 conn(0x55c0ee6d0640 msgr2=0x55c0ee6d2390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:16:54.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.181+0000 7f8d24c83740 1 -- 172.21.15.45:0/3106556747 shutdown_connections 2024-04-16T17:16:54.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.181+0000 7f8d24c83740 1 -- 172.21.15.45:0/3106556747 wait complete. 2024-04-16T17:16:54.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.181+0000 7f8d24c83740 20 client.18042 trim_cache size 0 max 16384 2024-04-16T17:16:54.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.197+0000 7f8d24c83740 1 Processor -- start 2024-04-16T17:16:54.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.197+0000 7f8d24c83740 1 -- start start 2024-04-16T17:16:54.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.198+0000 7f8d24c83740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c0ee6d5f10 0x55c0ee6dec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:54.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.198+0000 7f8d24c83740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c0ee6d6830 0x55c0ee6eb0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:54.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.198+0000 7f8d24c83740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c0ee6df410 0x55c0ee6db500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:54.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.198+0000 7f8d24c83740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f8d1c1b3eb0 con 0x55c0ee6d6830 2024-04-16T17:16:54.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.198+0000 7f8d24c83740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f8d1c1b1780 con 0x55c0ee6d5f10 2024-04-16T17:16:54.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.198+0000 7f8d24c83740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f8d1c1af5f0 con 0x55c0ee6df410 2024-04-16T17:16:54.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.198+0000 7f8d1be00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c0ee6d5f10 0x55c0ee6dec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:54.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.198+0000 7f8d1be00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c0ee6d5f10 0x55c0ee6dec50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:45440/0 (socket says 172.21.15.45:45440) 2024-04-16T17:16:54.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.198+0000 7f8d1be00640 1 -- 172.21.15.45:0/1480146119 learned_addr learned my addr 172.21.15.45:0/1480146119 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:16:54.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.198+0000 7f8d1be00640 1 --2- 172.21.15.45:0/1480146119 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c0ee6df410 0x55c0ee6db500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:54.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.198+0000 7f8d1aa00640 1 --2- 172.21.15.45:0/1480146119 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c0ee6d6830 0x55c0ee6eb0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:54.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.198+0000 7f8d1be00640 1 -- 172.21.15.45:0/1480146119 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c0ee6df410 msgr2=0x55c0ee6db500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:16:54.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.198+0000 7f8d1be00640 1 --2- 172.21.15.45:0/1480146119 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c0ee6df410 0x55c0ee6db500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:54.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.198+0000 7f8d1be00640 1 -- 172.21.15.45:0/1480146119 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c0ee6d6830 msgr2=0x55c0ee6eb0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:16:54.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.198+0000 7f8d1be00640 1 --2- 172.21.15.45:0/1480146119 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c0ee6d6830 0x55c0ee6eb0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:54.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.198+0000 7f8d1be00640 1 -- 172.21.15.45:0/1480146119 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c0ee34d3f0 con 0x55c0ee6d5f10 2024-04-16T17:16:54.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.198+0000 7f8d1aa00640 1 --2- 172.21.15.45:0/1480146119 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c0ee6d6830 0x55c0ee6eb0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:16:54.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.199+0000 7f8d1be00640 1 --2- 172.21.15.45:0/1480146119 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c0ee6d5f10 0x55c0ee6dec50 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f8d1402b580 tx=0x7f8d14013040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e6d22da64620e2e2 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:16:54.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.199+0000 7f8d07400640 1 -- 172.21.15.45:0/1480146119 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d14010ca0 con 0x55c0ee6d5f10 2024-04-16T17:16:54.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.199+0000 7f8d07400640 1 -- 172.21.15.45:0/1480146119 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8d14020030 con 0x55c0ee6d5f10 2024-04-16T17:16:54.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.199+0000 7f8d07400640 1 -- 172.21.15.45:0/1480146119 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d14011da0 con 0x55c0ee6d5f10 2024-04-16T17:16:54.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.200+0000 7f8d24c83740 1 -- 172.21.15.45:0/1480146119 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c0ee6d5f10 msgr2=0x55c0ee6dec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:54.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.200+0000 7f8d24c83740 1 --2- 172.21.15.45:0/1480146119 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c0ee6d5f10 0x55c0ee6dec50 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f8d1402b580 tx=0x7f8d14013040 comp rx=0 tx=0).stop 2024-04-16T17:16:54.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.200+0000 7f8d24c83740 1 -- 172.21.15.45:0/1480146119 shutdown_connections 2024-04-16T17:16:54.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.200+0000 7f8d24c83740 1 --2- 172.21.15.45:0/1480146119 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c0ee6df410 0x55c0ee6db500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:54.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.200+0000 7f8d24c83740 1 --2- 172.21.15.45:0/1480146119 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c0ee6d6830 0x55c0ee6eb0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:54.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.200+0000 7f8d24c83740 1 --2- 172.21.15.45:0/1480146119 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c0ee6d5f10 0x55c0ee6dec50 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:54.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.200+0000 7f8d24c83740 1 -- 172.21.15.45:0/1480146119 >> 172.21.15.45:0/1480146119 conn(0x55c0ee6d0640 msgr2=0x55c0ee6dadf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:16:54.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.200+0000 7f8d24c83740 1 -- 172.21.15.45:0/1480146119 shutdown_connections 2024-04-16T17:16:54.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.200+0000 7f8d24c83740 1 -- 172.21.15.45:0/1480146119 wait complete. 2024-04-16T17:16:54.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.200+0000 7f8d24c83740 1 -- start start 2024-04-16T17:16:54.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.200+0000 7f8d24c83740 1 Processor -- start 2024-04-16T17:16:54.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.201+0000 7f8d24c83740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c0ee6d5f10 0x55c0ee6e2590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:54.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.201+0000 7f8d24c83740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c0ee6d6830 0x55c0ee6e2ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:54.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.201+0000 7f8d24c83740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c0ee6df410 0x55c0ee6e3010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:54.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.201+0000 7f8d24c83740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f8d1c1afc40 con 0x55c0ee6d6830 2024-04-16T17:16:54.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.201+0000 7f8d24c83740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f8d1c1b10b0 con 0x55c0ee6d5f10 2024-04-16T17:16:54.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.201+0000 7f8d24c83740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f8d1c1b0310 con 0x55c0ee6df410 2024-04-16T17:16:54.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.201+0000 7f8d24c83740 10 client.0 _ll_register_callbacks cb 0x7f8d0c0d4d10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:16:54.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.201+0000 7f8d1aa00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c0ee6d6830 0x55c0ee6e2ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:54.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.201+0000 7f8d1be00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c0ee6d5f10 0x55c0ee6e2590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:54.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.201+0000 7f8d1aa00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c0ee6d6830 0x55c0ee6e2ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56892/0 (socket says 172.21.15.45:56892) 2024-04-16T17:16:54.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.201+0000 7f8d1aa00640 1 -- 172.21.15.45:0/3980330325 learned_addr learned my addr 172.21.15.45:0/3980330325 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:16:54.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.201+0000 7f8d1be00640 1 --2- 172.21.15.45:0/3980330325 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c0ee6df410 0x55c0ee6e3010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:54.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.202+0000 7f8d1aa00640 1 -- 172.21.15.45:0/3980330325 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c0ee6df410 msgr2=0x55c0ee6e3010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:16:54.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.202+0000 7f8d1aa00640 1 --2- 172.21.15.45:0/3980330325 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c0ee6df410 0x55c0ee6e3010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:54.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.202+0000 7f8d1aa00640 1 -- 172.21.15.45:0/3980330325 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c0ee6d5f10 msgr2=0x55c0ee6e2590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:16:54.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.202+0000 7f8d1aa00640 1 --2- 172.21.15.45:0/3980330325 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c0ee6d5f10 0x55c0ee6e2590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:54.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.202+0000 7f8d1aa00640 1 -- 172.21.15.45:0/3980330325 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c0ee4177d0 con 0x55c0ee6d6830 2024-04-16T17:16:54.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.202+0000 7f8d1aa00640 1 --2- 172.21.15.45:0/3980330325 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c0ee6d6830 0x55c0ee6e2ad0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f8d0802c180 tx=0x7f8d08009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=776d802c8e3f951c server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:16:54.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.202+0000 7f8d04c00640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-16T17:16:54.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.202+0000 7f8d04c00640 1 -- 172.21.15.45:0/3980330325 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d08039040 con 0x55c0ee6d6830 2024-04-16T17:16:54.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.202+0000 7f8d04c00640 1 -- 172.21.15.45:0/3980330325 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9496+0+0 (secure 0 0 0) 0x7f8d08004400 con 0x55c0ee6d6830 2024-04-16T17:16:54.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.202+0000 7f8d24c83740 1 -- 172.21.15.45:0/3980330325 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8d14020030 con 0x55c0ee6d6830 2024-04-16T17:16:54.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.203+0000 7f8d04c00640 1 -- 172.21.15.45:0/3980330325 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f8d08058050 con 0x55c0ee6d6830 2024-04-16T17:16:54.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.203+0000 7f8d24c83740 10 client.18048 fetch_fsmap learned FSMap version 44 2024-04-16T17:16:54.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.203+0000 7f8d24c83740 1 -- 172.21.15.45:0/3980330325 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55c0ee75d490 con 0x55c0ee6d6830 2024-04-16T17:16:54.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.203+0000 7f8d04c00640 1 -- 172.21.15.45:0/3980330325 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8d14020030 con 0x55c0ee6d6830 2024-04-16T17:16:54.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.203+0000 7f8d04c00640 1 -- 172.21.15.45:0/3980330325 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 44) ==== 8346+0+0 (secure 0 0 0) 0x7f8d08032640 con 0x55c0ee6d6830 2024-04-16T17:16:54.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.203+0000 7f8d24c83740 10 client.18048 fetch_fsmap finished waiting for FSMap version 44 2024-04-16T17:16:54.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.203+0000 7f8d24c83740 10 client.18048 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-04-16T17:16:54.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.203+0000 7f8d24c83740 20 client.18048 populate_metadata read hostname 'smithi045' 2024-04-16T17:16:54.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.203+0000 7f8d24c83740 1 --2- 172.21.15.45:0/3980330325 >> [v2:172.21.15.45:6840/3755310822,v1:172.21.15.45:6841/3755310822] conn(0x55c0ee75dc40 0x55c0ee760060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:54.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.203+0000 7f8d24c83740 4 client.18048 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T17:16:54.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.203+0000 7f8d24c83740 1 -- 172.21.15.45:0/3980330325 --> [v2:172.21.15.45:6840/3755310822,v1:172.21.15.45:6841/3755310822] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c0ee6428b0 con 0x55c0ee75dc40 2024-04-16T17:16:54.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.204+0000 7f8d1be00640 1 --2- 172.21.15.45:0/3980330325 >> [v2:172.21.15.45:6840/3755310822,v1:172.21.15.45:6841/3755310822] conn(0x55c0ee75dc40 0x55c0ee760060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:54.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.204+0000 7f8d1be00640 1 --2- 172.21.15.45:0/3980330325 >> [v2:172.21.15.45:6840/3755310822,v1:172.21.15.45:6841/3755310822] conn(0x55c0ee75dc40 0x55c0ee760060 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=30ab269aa63aed61 server_cookie=3df4ba66242159b in_seq=0 out_seq=0 2024-04-16T17:16:54.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.204+0000 7f8d04c00640 10 client.18048 ms_handle_connect on v2:172.21.15.45:6840/3755310822 2024-04-16T17:16:54.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.204+0000 7f8d04c00640 1 -- 172.21.15.45:0/3980330325 <== mds.1 v2:172.21.15.45:6840/3755310822 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c0ee6428b0 con 0x55c0ee75dc40 2024-04-16T17:16:54.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.204+0000 7f8d04c00640 10 client.18048 handle_command_reply: tid=0 2024-04-16T17:16:54.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.204+0000 7f8d24c83740 1 client.18048 shutdown 2024-04-16T17:16:54.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.205+0000 7f8d24c83740 1 -- 172.21.15.45:0/3980330325 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c0ee6d6830 msgr2=0x55c0ee6e2ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:54.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.205+0000 7f8d24c83740 1 --2- 172.21.15.45:0/3980330325 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c0ee6d6830 0x55c0ee6e2ad0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f8d0802c180 tx=0x7f8d08009580 comp rx=0 tx=0).stop 2024-04-16T17:16:54.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.205+0000 7f8d24c83740 1 -- 172.21.15.45:0/3980330325 shutdown_connections 2024-04-16T17:16:54.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.205+0000 7f8d24c83740 1 --2- 172.21.15.45:0/3980330325 >> [v2:172.21.15.45:6840/3755310822,v1:172.21.15.45:6841/3755310822] conn(0x55c0ee75dc40 0x55c0ee760060 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:54.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.205+0000 7f8d24c83740 1 --2- 172.21.15.45:0/3980330325 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c0ee6df410 0x55c0ee6e3010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:54.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.205+0000 7f8d24c83740 1 --2- 172.21.15.45:0/3980330325 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c0ee6d6830 0x55c0ee6e2ad0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:54.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.205+0000 7f8d24c83740 1 --2- 172.21.15.45:0/3980330325 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c0ee6d5f10 0x55c0ee6e2590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:54.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.205+0000 7f8d24c83740 1 -- 172.21.15.45:0/3980330325 >> 172.21.15.45:0/3980330325 conn(0x55c0ee6d0640 msgr2=0x55c0ee6db680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:16:54.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.205+0000 7f8d04c00640 0 client.18048 ms_handle_reset on v2:172.21.15.45:6840/3755310822 2024-04-16T17:16:54.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.205+0000 7f8d24c83740 1 -- 172.21.15.45:0/3980330325 shutdown_connections 2024-04-16T17:16:54.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.205+0000 7f8d24c83740 1 -- 172.21.15.45:0/3980330325 wait complete. 2024-04-16T17:16:54.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.205+0000 7f8d24c83740 20 client.18048 trim_cache size 0 max 16384 2024-04-16T17:16:54.212 INFO:teuthology.orchestra.run.smithi045.stdout:[] 2024-04-16T17:16:54.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.206+0000 7f8d20a00640 1 -- 172.21.15.45:0/643278657 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f8d000a3d20 msgr2=0x7f8d000a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:54.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.206+0000 7f8d20a00640 1 --2- 172.21.15.45:0/643278657 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f8d000a3d20 0x7f8d000a61e0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f8d0c0ce720 tx=0x7f8d0c0d3800 comp rx=0 tx=0).stop 2024-04-16T17:16:54.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.206+0000 7f8d20a00640 1 -- 172.21.15.45:0/643278657 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8d1c102110 msgr2=0x7f8d1c1a0670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:54.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.206+0000 7f8d20a00640 1 --2- 172.21.15.45:0/643278657 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8d1c102110 0x7f8d1c1a0670 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f8d0c00eb90 tx=0x7f8d0c00c6a0 comp rx=0 tx=0).stop 2024-04-16T17:16:54.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.206+0000 7f8d20a00640 1 -- 172.21.15.45:0/643278657 shutdown_connections 2024-04-16T17:16:54.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.206+0000 7f8d20a00640 1 --2- 172.21.15.45:0/643278657 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f8d000a3d20 0x7f8d000a61e0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:54.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.206+0000 7f8d20a00640 1 --2- 172.21.15.45:0/643278657 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8d1c1071e0 0x7f8d1c1a0bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:54.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.206+0000 7f8d20a00640 1 --2- 172.21.15.45:0/643278657 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8d1c19a7e0 0x7f8d1c19ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:54.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.206+0000 7f8d20a00640 1 --2- 172.21.15.45:0/643278657 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8d1c102110 0x7f8d1c1a0670 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:54.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.206+0000 7f8d20a00640 1 -- 172.21.15.45:0/643278657 >> 172.21.15.45:0/643278657 conn(0x7f8d1c0fd9d0 msgr2=0x7f8d1c0ffd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:16:54.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.206+0000 7f8d20a00640 1 -- 172.21.15.45:0/643278657 shutdown_connections 2024-04-16T17:16:54.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.206+0000 7f8d20a00640 1 -- 172.21.15.45:0/643278657 wait complete. 2024-04-16T17:16:54.223 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-16T17:16:54.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.446+0000 7f99f5200640 1 -- 172.21.15.45:0/3679313321 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f99f006a870 msgr2=0x7f99f006ac50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:54.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.446+0000 7f99f5200640 1 --2- 172.21.15.45:0/3679313321 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f99f006a870 0x7f99f006ac50 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f99e0009900 tx=0x7f99e002bc30 comp rx=0 tx=0).stop 2024-04-16T17:16:54.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.446+0000 7f99f5200640 1 -- 172.21.15.45:0/3679313321 shutdown_connections 2024-04-16T17:16:54.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.446+0000 7f99f5200640 1 --2- 172.21.15.45:0/3679313321 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f99f006b220 0x7f99f0106c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:54.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.446+0000 7f99f5200640 1 --2- 172.21.15.45:0/3679313321 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f99f0107330 0x7f99f010eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:54.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.446+0000 7f99f5200640 1 --2- 172.21.15.45:0/3679313321 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f99f006a870 0x7f99f006ac50 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:54.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.446+0000 7f99f5200640 1 -- 172.21.15.45:0/3679313321 >> 172.21.15.45:0/3679313321 conn(0x7f99f0076d00 msgr2=0x7f99f0077110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:16:54.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.446+0000 7f99f5200640 1 -- 172.21.15.45:0/3679313321 shutdown_connections 2024-04-16T17:16:54.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.446+0000 7f99f5200640 1 -- 172.21.15.45:0/3679313321 wait complete. 2024-04-16T17:16:54.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.447+0000 7f99f5200640 1 Processor -- start 2024-04-16T17:16:54.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.447+0000 7f99f5200640 1 -- start start 2024-04-16T17:16:54.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.448+0000 7f99f5200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f99f006a870 0x7f99f0101320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:54.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.448+0000 7f99f5200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f99f0107330 0x7f99f0101860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:54.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.448+0000 7f99f5200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f99f0101da0 0x7f99f01a6a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:54.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.448+0000 7f99f5200640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f99f0112430 con 0x7f99f0101da0 2024-04-16T17:16:54.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.448+0000 7f99f5200640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f99f01122b0 con 0x7f99f006a870 2024-04-16T17:16:54.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.448+0000 7f99f5200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f99f01125b0 con 0x7f99f0107330 2024-04-16T17:16:54.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.448+0000 7f99ed400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f99f0107330 0x7f99f0101860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:54.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.448+0000 7f99ee800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f99f0101da0 0x7f99f01a6a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:54.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.448+0000 7f99ed400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f99f0107330 0x7f99f0101860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:40350/0 (socket says 172.21.15.45:40350) 2024-04-16T17:16:54.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.448+0000 7f99ed400640 1 -- 172.21.15.45:0/1595439625 learned_addr learned my addr 172.21.15.45:0/1595439625 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:16:54.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.448+0000 7f99ede00640 1 --2- 172.21.15.45:0/1595439625 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f99f006a870 0x7f99f0101320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:54.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.449+0000 7f99ed400640 1 -- 172.21.15.45:0/1595439625 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f99f006a870 msgr2=0x7f99f0101320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:54.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.449+0000 7f99ed400640 1 --2- 172.21.15.45:0/1595439625 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f99f006a870 0x7f99f0101320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:54.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.449+0000 7f99ed400640 1 -- 172.21.15.45:0/1595439625 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f99f0101da0 msgr2=0x7f99f01a6a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:54.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.449+0000 7f99ed400640 1 --2- 172.21.15.45:0/1595439625 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f99f0101da0 0x7f99f01a6a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:54.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.449+0000 7f99ed400640 1 -- 172.21.15.45:0/1595439625 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99f01a7160 con 0x7f99f0107330 2024-04-16T17:16:54.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.449+0000 7f99ed400640 1 --2- 172.21.15.45:0/1595439625 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f99f0107330 0x7f99f0101860 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f99d800eb90 tx=0x7f99d800c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:16:54.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.449+0000 7f99e6a00640 1 -- 172.21.15.45:0/1595439625 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99d8004d20 con 0x7f99f0107330 2024-04-16T17:16:54.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.449+0000 7f99e6a00640 1 -- 172.21.15.45:0/1595439625 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f99d800ee90 con 0x7f99f0107330 2024-04-16T17:16:54.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.449+0000 7f99e6a00640 1 -- 172.21.15.45:0/1595439625 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99d8010860 con 0x7f99f0107330 2024-04-16T17:16:54.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.449+0000 7f99f5200640 1 -- 172.21.15.45:0/1595439625 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99f01a7450 con 0x7f99f0107330 2024-04-16T17:16:54.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.449+0000 7f99f5200640 1 -- 172.21.15.45:0/1595439625 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99f01a79a0 con 0x7f99f0107330 2024-04-16T17:16:54.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.453+0000 7f99f7fb9740 1 Processor -- start 2024-04-16T17:16:54.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.453+0000 7f99e6a00640 1 -- 172.21.15.45:0/1595439625 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f99d8004ec0 con 0x7f99f0107330 2024-04-16T17:16:54.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.453+0000 7f99f7fb9740 1 -- start start 2024-04-16T17:16:54.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.453+0000 7f99f7fb9740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563e53ab1b60 0x563e53ab1f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:54.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.453+0000 7f99f7fb9740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563e53ab2510 0x563e53ac3520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:54.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.453+0000 7f99f7fb9740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563e53ac3a60 0x563e53ac5e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:54.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.453+0000 7f99f7fb9740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x563e53a17020 con 0x563e53ab2510 2024-04-16T17:16:54.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.453+0000 7f99f7fb9740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x563e539bd460 con 0x563e53ac3a60 2024-04-16T17:16:54.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.453+0000 7f99f7fb9740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x563e53a12fe0 con 0x563e53ab1b60 2024-04-16T17:16:54.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.453+0000 7f99ede00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563e53ab2510 0x563e53ac3520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:54.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.454+0000 7f99ee800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563e53ab1b60 0x563e53ab1f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:54.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.454+0000 7f99ee800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563e53ab1b60 0x563e53ab1f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:40360/0 (socket says 172.21.15.45:40360) 2024-04-16T17:16:54.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.454+0000 7f99ed400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563e53ac3a60 0x563e53ac5e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:54.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.454+0000 7f99ee800640 1 -- 172.21.15.45:0/3573206026 learned_addr learned my addr 172.21.15.45:0/3573206026 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:16:54.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.454+0000 7f99ee800640 1 -- 172.21.15.45:0/3573206026 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563e53ac3a60 msgr2=0x563e53ac5e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:16:54.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.454+0000 7f99ee800640 1 --2- 172.21.15.45:0/3573206026 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563e53ac3a60 0x563e53ac5e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:54.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.454+0000 7f99ee800640 1 -- 172.21.15.45:0/3573206026 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563e53ab2510 msgr2=0x563e53ac3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:16:54.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.454+0000 7f99ee800640 1 --2- 172.21.15.45:0/3573206026 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563e53ab2510 0x563e53ac3520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:54.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.454+0000 7f99ee800640 1 -- 172.21.15.45:0/3573206026 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563e539c5040 con 0x563e53ab1b60 2024-04-16T17:16:54.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.454+0000 7f99ee800640 1 --2- 172.21.15.45:0/3573206026 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563e53ab1b60 0x563e53ab1f40 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f99e8009c50 tx=0x7f99e8011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=496199eda7fc5905 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:16:54.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.455+0000 7f99e6a00640 1 --2- 172.21.15.45:0/1595439625 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f99d00a3dd0 0x7f99d00a6290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:54.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.455+0000 7f99df400640 1 -- 172.21.15.45:0/3573206026 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99e801e070 con 0x563e53ab1b60 2024-04-16T17:16:54.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.455+0000 7f99df400640 1 -- 172.21.15.45:0/3573206026 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f99e8019b10 con 0x563e53ab1b60 2024-04-16T17:16:54.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.455+0000 7f99df400640 1 -- 172.21.15.45:0/3573206026 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99e800ea80 con 0x563e53ab1b60 2024-04-16T17:16:54.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.455+0000 7f99e6a00640 1 -- 172.21.15.45:0/1595439625 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 9496+0+0 (secure 0 0 0) 0x7f99d80d1310 con 0x7f99f0107330 2024-04-16T17:16:54.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.455+0000 7f99ede00640 1 --2- 172.21.15.45:0/1595439625 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f99d00a3dd0 0x7f99d00a6290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:54.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.455+0000 7f99f7fb9740 1 -- 172.21.15.45:0/3573206026 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563e53ab1b60 msgr2=0x563e53ab1f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:54.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.455+0000 7f99f7fb9740 1 --2- 172.21.15.45:0/3573206026 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563e53ab1b60 0x563e53ab1f40 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f99e8009c50 tx=0x7f99e8011810 comp rx=0 tx=0).stop 2024-04-16T17:16:54.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.456+0000 7f99f7fb9740 1 -- 172.21.15.45:0/3573206026 shutdown_connections 2024-04-16T17:16:54.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.456+0000 7f99f7fb9740 1 --2- 172.21.15.45:0/3573206026 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563e53ac3a60 0x563e53ac5e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:54.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.456+0000 7f99f7fb9740 1 --2- 172.21.15.45:0/3573206026 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563e53ab2510 0x563e53ac3520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:54.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.456+0000 7f99f7fb9740 1 --2- 172.21.15.45:0/3573206026 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563e53ab1b60 0x563e53ab1f40 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:54.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.456+0000 7f99f7fb9740 1 -- 172.21.15.45:0/3573206026 >> 172.21.15.45:0/3573206026 conn(0x563e53aac480 msgr2=0x563e53aac860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:16:54.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.456+0000 7f99ede00640 1 --2- 172.21.15.45:0/1595439625 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f99d00a3dd0 0x7f99d00a6290 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f99e0000c00 tx=0x7f99e002df90 comp rx=0 tx=0).ready entity=mgr.25411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:16:54.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.456+0000 7f99f7fb9740 1 -- 172.21.15.45:0/3573206026 shutdown_connections 2024-04-16T17:16:54.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.456+0000 7f99f7fb9740 1 -- 172.21.15.45:0/3573206026 wait complete. 2024-04-16T17:16:54.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.456+0000 7f99f7fb9740 1 -- start start 2024-04-16T17:16:54.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.456+0000 7f99f7fb9740 1 Processor -- start 2024-04-16T17:16:54.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.457+0000 7f99f7fb9740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563e53ab1b60 0x563e53ac0b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:54.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.457+0000 7f99f7fb9740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563e53ab2510 0x563e53ac1080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:54.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.457+0000 7f99f7fb9740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563e53ac3a60 0x563e53ac15c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:54.464 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.457+0000 7f99f7fb9740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x563e53ac6820 con 0x563e53ac3a60 2024-04-16T17:16:54.464 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.457+0000 7f99f7fb9740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x563e539d0830 con 0x563e53ab2510 2024-04-16T17:16:54.464 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.457+0000 7f99f7fb9740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x563e53ac69a0 con 0x563e53ab1b60 2024-04-16T17:16:54.464 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.457+0000 7f99f7fb9740 10 client.0 _ll_register_callbacks cb 0x563e539d8150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:16:54.464 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.457+0000 7f99ee800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563e53ac3a60 0x563e53ac15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:54.464 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.457+0000 7f99ede00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563e53ab1b60 0x563e53ac0b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:54.464 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.457+0000 7f99ee800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563e53ac3a60 0x563e53ac15c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56938/0 (socket says 172.21.15.45:56938) 2024-04-16T17:16:54.464 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.457+0000 7f99ee800640 1 -- 172.21.15.45:0/1122152522 learned_addr learned my addr 172.21.15.45:0/1122152522 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:16:54.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.457+0000 7f99ed400640 1 --2- 172.21.15.45:0/1122152522 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563e53ab2510 0x563e53ac1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:54.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.458+0000 7f99ede00640 1 -- 172.21.15.45:0/1122152522 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563e53ab2510 msgr2=0x563e53ac1080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:16:54.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.458+0000 7f99ede00640 1 --2- 172.21.15.45:0/1122152522 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563e53ab2510 0x563e53ac1080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:54.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.458+0000 7f99ede00640 1 -- 172.21.15.45:0/1122152522 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563e53ac3a60 msgr2=0x563e53ac15c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:16:54.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.458+0000 7f99ede00640 1 --2- 172.21.15.45:0/1122152522 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563e53ac3a60 0x563e53ac15c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:54.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.458+0000 7f99ede00640 1 -- 172.21.15.45:0/1122152522 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563e539d4920 con 0x563e53ab1b60 2024-04-16T17:16:54.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.458+0000 7f99ee800640 1 --2- 172.21.15.45:0/1122152522 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563e53ac3a60 0x563e53ac15c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:16:54.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.458+0000 7f99ed400640 1 --2- 172.21.15.45:0/1122152522 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563e53ab2510 0x563e53ac1080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:16:54.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.458+0000 7f99ede00640 1 --2- 172.21.15.45:0/1122152522 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563e53ab1b60 0x563e53ac0b40 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f99e002c170 tx=0x7f99e0009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c0989b7e43b1c277 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:16:54.466 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.458+0000 7f99d7e00640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-04-16T17:16:54.466 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.459+0000 7f99d7e00640 1 -- 172.21.15.45:0/1122152522 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99e0046070 con 0x563e53ab1b60 2024-04-16T17:16:54.466 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.459+0000 7f99d7e00640 1 -- 172.21.15.45:0/1122152522 <== mon.1 v2:172.21.15.117:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9496+0+0 (secure 0 0 0) 0x7f99e0004c10 con 0x563e53ab1b60 2024-04-16T17:16:54.466 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.459+0000 7f99f7fb9740 1 -- 172.21.15.45:0/1122152522 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f99e8019b10 con 0x563e53ab1b60 2024-04-16T17:16:54.466 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.459+0000 7f99d7e00640 1 -- 172.21.15.45:0/1122152522 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f99e002cbc0 con 0x563e53ab1b60 2024-04-16T17:16:54.466 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.459+0000 7f99f7fb9740 10 client.28162 fetch_fsmap learned FSMap version 44 2024-04-16T17:16:54.466 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.459+0000 7f99f7fb9740 1 -- 172.21.15.45:0/1122152522 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x563e539a27f0 con 0x563e53ab1b60 2024-04-16T17:16:54.466 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.459+0000 7f99d7e00640 1 -- 172.21.15.45:0/1122152522 <== mon.1 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f99e8019b10 con 0x563e53ab1b60 2024-04-16T17:16:54.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.460+0000 7f99d7e00640 1 -- 172.21.15.45:0/1122152522 <== mon.1 v2:172.21.15.117:3300/0 5 ==== fsmap(e 44) ==== 8346+0+0 (secure 0 0 0) 0x7f99e00038d0 con 0x563e53ab1b60 2024-04-16T17:16:54.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.460+0000 7f99f7fb9740 10 client.28162 fetch_fsmap finished waiting for FSMap version 44 2024-04-16T17:16:54.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.460+0000 7f99f7fb9740 10 client.28162 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:16:54.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.460+0000 7f99f7fb9740 20 client.28162 populate_metadata read hostname 'smithi045' 2024-04-16T17:16:54.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.460+0000 7f99f7fb9740 1 --2- 172.21.15.45:0/1122152522 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x563e53ad3eb0 0x563e53ad6300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:54.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.460+0000 7f99f7fb9740 4 client.28162 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:16:54.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.460+0000 7f99f7fb9740 1 -- 172.21.15.45:0/1122152522 --> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563e53a24a40 con 0x563e53ad3eb0 2024-04-16T17:16:54.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.460+0000 7f99ed400640 1 --2- 172.21.15.45:0/1122152522 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x563e53ad3eb0 0x563e53ad6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:54.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.461+0000 7f99ed400640 1 --2- 172.21.15.45:0/1122152522 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x563e53ad3eb0 0x563e53ad6300 crc :-1 s=READY pgs=251 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7ef84787c0610fcc server_cookie=555e2d98de8fd764 in_seq=0 out_seq=0 2024-04-16T17:16:54.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.461+0000 7f99d7e00640 10 client.28162 ms_handle_connect on v2:172.21.15.104:6838/3660070592 2024-04-16T17:16:54.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.461+0000 7f99d7e00640 1 -- 172.21.15.45:0/1122152522 <== mds.0 v2:172.21.15.104:6838/3660070592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563e53a24a40 con 0x563e53ad3eb0 2024-04-16T17:16:54.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.461+0000 7f99d7e00640 10 client.28162 handle_command_reply: tid=0 2024-04-16T17:16:54.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.462+0000 7f99f7fb9740 1 client.28162 shutdown 2024-04-16T17:16:54.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.462+0000 7f99f7fb9740 1 -- 172.21.15.45:0/1122152522 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563e53ab1b60 msgr2=0x563e53ac0b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:54.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.462+0000 7f99f7fb9740 1 --2- 172.21.15.45:0/1122152522 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563e53ab1b60 0x563e53ac0b40 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f99e002c170 tx=0x7f99e0009260 comp rx=0 tx=0).stop 2024-04-16T17:16:54.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.462+0000 7f99f7fb9740 1 -- 172.21.15.45:0/1122152522 shutdown_connections 2024-04-16T17:16:54.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.462+0000 7f99f7fb9740 1 --2- 172.21.15.45:0/1122152522 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563e53ac3a60 0x563e53ac15c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:54.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.462+0000 7f99f7fb9740 1 --2- 172.21.15.45:0/1122152522 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563e53ab2510 0x563e53ac1080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:54.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.462+0000 7f99f7fb9740 1 --2- 172.21.15.45:0/1122152522 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x563e53ad3eb0 0x563e53ad6300 crc :-1 s=READY pgs=251 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:54.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.462+0000 7f99f7fb9740 1 --2- 172.21.15.45:0/1122152522 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563e53ab1b60 0x563e53ac0b40 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:54.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.462+0000 7f99f7fb9740 1 -- 172.21.15.45:0/1122152522 >> 172.21.15.45:0/1122152522 conn(0x563e53aac480 msgr2=0x563e53aae1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:16:54.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.462+0000 7f99d7e00640 0 client.28162 ms_handle_reset on v2:172.21.15.104:6838/3660070592 2024-04-16T17:16:54.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.462+0000 7f99f7fb9740 1 -- 172.21.15.45:0/1122152522 shutdown_connections 2024-04-16T17:16:54.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.462+0000 7f99f7fb9740 1 -- 172.21.15.45:0/1122152522 wait complete. 2024-04-16T17:16:54.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.462+0000 7f99f7fb9740 20 client.28162 trim_cache size 0 max 16384 2024-04-16T17:16:54.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.477+0000 7f99f7fb9740 1 Processor -- start 2024-04-16T17:16:54.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.477+0000 7f99f7fb9740 1 -- start start 2024-04-16T17:16:54.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.478+0000 7f99f7fb9740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563e53ab1b60 0x563e53ac3270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:54.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.478+0000 7f99f7fb9740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563e53ab2510 0x563e53acf720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:54.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.478+0000 7f99f7fb9740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563e53ac3a60 0x563e53abfca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:54.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.478+0000 7f99f7fb9740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f99f01afc30 con 0x563e53ac3a60 2024-04-16T17:16:54.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.478+0000 7f99f7fb9740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f99f01ad500 con 0x563e53ab1b60 2024-04-16T17:16:54.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.478+0000 7f99f7fb9740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f99f01ab370 con 0x563e53ab2510 2024-04-16T17:16:54.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.478+0000 7f99ee800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563e53ac3a60 0x563e53abfca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:54.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.478+0000 7f99ee800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563e53ac3a60 0x563e53abfca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56952/0 (socket says 172.21.15.45:56952) 2024-04-16T17:16:54.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.478+0000 7f99ed400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563e53ab2510 0x563e53acf720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:54.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.478+0000 7f99ee800640 1 -- 172.21.15.45:0/2490549537 learned_addr learned my addr 172.21.15.45:0/2490549537 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:16:54.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.478+0000 7f99ede00640 1 --2- 172.21.15.45:0/2490549537 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563e53ab1b60 0x563e53ac3270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:54.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.479+0000 7f99ed400640 1 -- 172.21.15.45:0/2490549537 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563e53ab1b60 msgr2=0x563e53ac3270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:16:54.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.479+0000 7f99ed400640 1 --2- 172.21.15.45:0/2490549537 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563e53ab1b60 0x563e53ac3270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:54.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.479+0000 7f99ed400640 1 -- 172.21.15.45:0/2490549537 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563e53ac3a60 msgr2=0x563e53abfca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:16:54.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.479+0000 7f99ed400640 1 --2- 172.21.15.45:0/2490549537 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563e53ac3a60 0x563e53abfca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:54.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.479+0000 7f99ed400640 1 -- 172.21.15.45:0/2490549537 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563e537293f0 con 0x563e53ab2510 2024-04-16T17:16:54.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.479+0000 7f99ee800640 1 --2- 172.21.15.45:0/2490549537 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563e53ac3a60 0x563e53abfca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:16:54.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.479+0000 7f99ed400640 1 --2- 172.21.15.45:0/2490549537 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563e53ab2510 0x563e53acf720 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f99d80ce630 tx=0x7f99d80bd040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=10f15c1567e3f1bf server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:16:54.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.479+0000 7f99dea00640 1 -- 172.21.15.45:0/2490549537 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99d809c7e0 con 0x563e53ab2510 2024-04-16T17:16:54.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.479+0000 7f99dea00640 1 -- 172.21.15.45:0/2490549537 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f99d809d580 con 0x563e53ab2510 2024-04-16T17:16:54.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.479+0000 7f99dea00640 1 -- 172.21.15.45:0/2490549537 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99d80ce8b0 con 0x563e53ab2510 2024-04-16T17:16:54.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.480+0000 7f99f7fb9740 1 -- 172.21.15.45:0/2490549537 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563e53ab2510 msgr2=0x563e53acf720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:54.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.480+0000 7f99f7fb9740 1 --2- 172.21.15.45:0/2490549537 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563e53ab2510 0x563e53acf720 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f99d80ce630 tx=0x7f99d80bd040 comp rx=0 tx=0).stop 2024-04-16T17:16:54.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.480+0000 7f99f7fb9740 1 -- 172.21.15.45:0/2490549537 shutdown_connections 2024-04-16T17:16:54.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.480+0000 7f99f7fb9740 1 --2- 172.21.15.45:0/2490549537 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563e53ab2510 0x563e53acf720 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:54.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.480+0000 7f99f7fb9740 1 --2- 172.21.15.45:0/2490549537 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563e53ac3a60 0x563e53abfca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:54.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.480+0000 7f99f7fb9740 1 --2- 172.21.15.45:0/2490549537 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563e53ab1b60 0x563e53ac3270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:54.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.480+0000 7f99f7fb9740 1 -- 172.21.15.45:0/2490549537 >> 172.21.15.45:0/2490549537 conn(0x563e53aac480 msgr2=0x563e53aaac50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:16:54.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.480+0000 7f99f7fb9740 1 -- 172.21.15.45:0/2490549537 shutdown_connections 2024-04-16T17:16:54.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.480+0000 7f99f7fb9740 1 -- 172.21.15.45:0/2490549537 wait complete. 2024-04-16T17:16:54.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.480+0000 7f99f7fb9740 1 -- start start 2024-04-16T17:16:54.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.481+0000 7f99f7fb9740 1 Processor -- start 2024-04-16T17:16:54.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.481+0000 7f99f7fb9740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563e53ab1b60 0x563e53acbee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:54.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.481+0000 7f99f7fb9740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563e53ab2510 0x563e53ac7250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:54.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.481+0000 7f99f7fb9740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563e53ac3a60 0x563e53ac7790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:54.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.481+0000 7f99f7fb9740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f99f01ab9c0 con 0x563e53ab2510 2024-04-16T17:16:54.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.481+0000 7f99f7fb9740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f99f01ace30 con 0x563e53ac3a60 2024-04-16T17:16:54.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.481+0000 7f99f7fb9740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f99f01ac090 con 0x563e53ab1b60 2024-04-16T17:16:54.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.481+0000 7f99f7fb9740 10 client.0 _ll_register_callbacks cb 0x7f99e0009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:16:54.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.481+0000 7f99ede00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563e53ab1b60 0x563e53acbee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:54.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.481+0000 7f99ede00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563e53ab1b60 0x563e53acbee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:40372/0 (socket says 172.21.15.45:40372) 2024-04-16T17:16:54.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.481+0000 7f99ede00640 1 -- 172.21.15.45:0/2272764792 learned_addr learned my addr 172.21.15.45:0/2272764792 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:16:54.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.481+0000 7f99ee800640 1 --2- 172.21.15.45:0/2272764792 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563e53ac3a60 0x563e53ac7790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:54.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.481+0000 7f99ed400640 1 --2- 172.21.15.45:0/2272764792 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563e53ab2510 0x563e53ac7250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:54.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.482+0000 7f99ee800640 1 -- 172.21.15.45:0/2272764792 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563e53ab1b60 msgr2=0x563e53acbee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:16:54.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.482+0000 7f99ee800640 1 --2- 172.21.15.45:0/2272764792 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563e53ab1b60 0x563e53acbee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:54.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.482+0000 7f99ee800640 1 -- 172.21.15.45:0/2272764792 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563e53ab2510 msgr2=0x563e53ac7250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:16:54.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.482+0000 7f99ee800640 1 --2- 172.21.15.45:0/2272764792 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563e53ab2510 0x563e53ac7250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:54.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.482+0000 7f99ee800640 1 -- 172.21.15.45:0/2272764792 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563e537f37d0 con 0x563e53ac3a60 2024-04-16T17:16:54.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.482+0000 7f99ed400640 1 --2- 172.21.15.45:0/2272764792 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563e53ab2510 0x563e53ac7250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:16:54.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.482+0000 7f99ee800640 1 --2- 172.21.15.45:0/2272764792 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563e53ac3a60 0x563e53ac7790 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f99e8005b90 tx=0x7f99e8005710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=786d3d2335de9a89 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:16:54.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.482+0000 7f99d7e00640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-04-16T17:16:54.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.482+0000 7f99d7e00640 1 -- 172.21.15.45:0/2272764792 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99e801e040 con 0x563e53ac3a60 2024-04-16T17:16:54.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.482+0000 7f99d7e00640 1 -- 172.21.15.45:0/2272764792 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9496+0+0 (secure 0 0 0) 0x7f99e800e6e0 con 0x563e53ac3a60 2024-04-16T17:16:54.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.482+0000 7f99f7fb9740 1 -- 172.21.15.45:0/2272764792 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f99d809d580 con 0x563e53ac3a60 2024-04-16T17:16:54.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.483+0000 7f99d7e00640 1 -- 172.21.15.45:0/2272764792 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f99e8034050 con 0x563e53ac3a60 2024-04-16T17:16:54.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.483+0000 7f99f7fb9740 10 client.28217 fetch_fsmap learned FSMap version 44 2024-04-16T17:16:54.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.483+0000 7f99f7fb9740 1 -- 172.21.15.45:0/2272764792 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x563e53b3a1c0 con 0x563e53ac3a60 2024-04-16T17:16:54.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.483+0000 7f99d7e00640 1 -- 172.21.15.45:0/2272764792 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f99d809d580 con 0x563e53ac3a60 2024-04-16T17:16:54.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.483+0000 7f99d7e00640 1 -- 172.21.15.45:0/2272764792 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 44) ==== 8346+0+0 (secure 0 0 0) 0x7f99e8004bd0 con 0x563e53ac3a60 2024-04-16T17:16:54.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.483+0000 7f99f7fb9740 10 client.28217 fetch_fsmap finished waiting for FSMap version 44 2024-04-16T17:16:54.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.483+0000 7f99f7fb9740 10 client.28217 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:16:54.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.483+0000 7f99f7fb9740 20 client.28217 populate_metadata read hostname 'smithi045' 2024-04-16T17:16:54.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.483+0000 7f99f7fb9740 1 --2- 172.21.15.45:0/2272764792 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x563e53b3a690 0x563e53b3ca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:54.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.483+0000 7f99f7fb9740 4 client.28217 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T17:16:54.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.483+0000 7f99f7fb9740 1 -- 172.21.15.45:0/2272764792 --> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563e53a1e8b0 con 0x563e53b3a690 2024-04-16T17:16:54.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.484+0000 7f99ede00640 1 --2- 172.21.15.45:0/2272764792 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x563e53b3a690 0x563e53b3ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:54.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.484+0000 7f99ede00640 1 --2- 172.21.15.45:0/2272764792 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x563e53b3a690 0x563e53b3ca80 crc :-1 s=READY pgs=252 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4eab319757de3396 server_cookie=5dc6ee81678e9cb9 in_seq=0 out_seq=0 2024-04-16T17:16:54.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.484+0000 7f99d7e00640 10 client.28217 ms_handle_connect on v2:172.21.15.104:6838/3660070592 2024-04-16T17:16:54.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.485+0000 7f99d7e00640 1 -- 172.21.15.45:0/2272764792 <== mds.0 v2:172.21.15.104:6838/3660070592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563e53a1e8b0 con 0x563e53b3a690 2024-04-16T17:16:54.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.485+0000 7f99d7e00640 10 client.28217 handle_command_reply: tid=0 2024-04-16T17:16:54.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.485+0000 7f99f7fb9740 1 client.28217 shutdown 2024-04-16T17:16:54.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.485+0000 7f99f7fb9740 1 -- 172.21.15.45:0/2272764792 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563e53ac3a60 msgr2=0x563e53ac7790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:54.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.485+0000 7f99f7fb9740 1 --2- 172.21.15.45:0/2272764792 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563e53ac3a60 0x563e53ac7790 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f99e8005b90 tx=0x7f99e8005710 comp rx=0 tx=0).stop 2024-04-16T17:16:54.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.485+0000 7f99f7fb9740 1 -- 172.21.15.45:0/2272764792 shutdown_connections 2024-04-16T17:16:54.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.485+0000 7f99f7fb9740 1 --2- 172.21.15.45:0/2272764792 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563e53ac3a60 0x563e53ac7790 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:54.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.485+0000 7f99f7fb9740 1 --2- 172.21.15.45:0/2272764792 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563e53ab2510 0x563e53ac7250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:54.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.485+0000 7f99f7fb9740 1 --2- 172.21.15.45:0/2272764792 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x563e53b3a690 0x563e53b3ca80 crc :-1 s=READY pgs=252 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:54.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.485+0000 7f99f7fb9740 1 --2- 172.21.15.45:0/2272764792 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563e53ab1b60 0x563e53acbee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:54.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.485+0000 7f99f7fb9740 1 -- 172.21.15.45:0/2272764792 >> 172.21.15.45:0/2272764792 conn(0x563e53aac480 msgr2=0x563e53aad810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:16:54.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.485+0000 7f99d7e00640 0 client.28217 ms_handle_reset on v2:172.21.15.104:6838/3660070592 2024-04-16T17:16:54.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.485+0000 7f99f7fb9740 1 -- 172.21.15.45:0/2272764792 shutdown_connections 2024-04-16T17:16:54.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.485+0000 7f99f7fb9740 1 -- 172.21.15.45:0/2272764792 wait complete. 2024-04-16T17:16:54.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.485+0000 7f99f7fb9740 20 client.28217 trim_cache size 0 max 16384 2024-04-16T17:16:54.493 INFO:teuthology.orchestra.run.smithi045.stdout:[] 2024-04-16T17:16:54.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.486+0000 7f99f5200640 1 -- 172.21.15.45:0/1595439625 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f99d00a3dd0 msgr2=0x7f99d00a6290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:54.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.486+0000 7f99f5200640 1 --2- 172.21.15.45:0/1595439625 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f99d00a3dd0 0x7f99d00a6290 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f99e0000c00 tx=0x7f99e002df90 comp rx=0 tx=0).stop 2024-04-16T17:16:54.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.486+0000 7f99f5200640 1 -- 172.21.15.45:0/1595439625 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f99f0107330 msgr2=0x7f99f0101860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:54.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.486+0000 7f99f5200640 1 --2- 172.21.15.45:0/1595439625 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f99f0107330 0x7f99f0101860 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f99d800eb90 tx=0x7f99d800c6a0 comp rx=0 tx=0).stop 2024-04-16T17:16:54.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.486+0000 7f99f5200640 1 -- 172.21.15.45:0/1595439625 shutdown_connections 2024-04-16T17:16:54.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.486+0000 7f99f5200640 1 --2- 172.21.15.45:0/1595439625 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f99d00a3dd0 0x7f99d00a6290 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:54.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.486+0000 7f99f5200640 1 --2- 172.21.15.45:0/1595439625 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f99f0107330 0x7f99f0101860 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:54.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.486+0000 7f99f5200640 1 --2- 172.21.15.45:0/1595439625 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f99f0101da0 0x7f99f01a6a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:54.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.486+0000 7f99f5200640 1 --2- 172.21.15.45:0/1595439625 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f99f006a870 0x7f99f0101320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:54.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.486+0000 7f99f5200640 1 -- 172.21.15.45:0/1595439625 >> 172.21.15.45:0/1595439625 conn(0x7f99f0076d00 msgr2=0x7f99f00ff3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:16:54.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.486+0000 7f99f5200640 1 -- 172.21.15.45:0/1595439625 shutdown_connections 2024-04-16T17:16:54.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:54.486+0000 7f99f5200640 1 -- 172.21.15.45:0/1595439625 wait complete. 2024-04-16T17:16:54.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:16:54 smithi104 ceph-mon[36457]: pgmap v316: 129 pgs: 129 active+clean; 56 MiB data, 870 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-04-16T17:16:54.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:16:54 smithi104 ceph-mon[36457]: from='client.? 172.21.15.45:0/1713286111' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:16:54.976 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:16:54 smithi045 ceph-mon[31676]: pgmap v316: 129 pgs: 129 active+clean; 56 MiB data, 870 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-04-16T17:16:54.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:16:54 smithi045 ceph-mon[31676]: from='client.? 172.21.15.45:0/1713286111' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:16:55.055 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:16:54 smithi117 ceph-mon[36730]: pgmap v316: 129 pgs: 129 active+clean; 56 MiB data, 870 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-04-16T17:16:55.055 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:16:54 smithi117 ceph-mon[36730]: from='client.? 172.21.15.45:0/1713286111' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:16:55.506 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-16T17:16:55.507 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T17:16:55.507 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-16T17:16:55.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:55.749+0000 7f75abe00640 1 -- 172.21.15.45:0/4206991989 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f75ac104ca0 msgr2=0x7f75ac108ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:55.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:55.749+0000 7f75abe00640 1 --2- 172.21.15.45:0/4206991989 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f75ac104ca0 0x7f75ac108ce0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f7598009860 tx=0x7f759802bb70 comp rx=0 tx=0).stop 2024-04-16T17:16:55.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:55.749+0000 7f75abe00640 1 -- 172.21.15.45:0/4206991989 shutdown_connections 2024-04-16T17:16:55.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:55.749+0000 7f75abe00640 1 --2- 172.21.15.45:0/4206991989 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f75ac104ca0 0x7f75ac108ce0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:55.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:55.749+0000 7f75abe00640 1 --2- 172.21.15.45:0/4206991989 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f75ac1093c0 0x7f75ac110f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:55.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:55.749+0000 7f75abe00640 1 --2- 172.21.15.45:0/4206991989 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f75ac1042f0 0x7f75ac1046d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:55.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:55.749+0000 7f75abe00640 1 -- 172.21.15.45:0/4206991989 >> 172.21.15.45:0/4206991989 conn(0x7f75ac0ffb10 msgr2=0x7f75ac101f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:16:55.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:55.749+0000 7f75abe00640 1 -- 172.21.15.45:0/4206991989 shutdown_connections 2024-04-16T17:16:55.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:55.749+0000 7f75abe00640 1 -- 172.21.15.45:0/4206991989 wait complete. 2024-04-16T17:16:55.753 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:55.750+0000 7f75abe00640 1 Processor -- start 2024-04-16T17:16:55.753 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:55.751+0000 7f75abe00640 1 -- start start 2024-04-16T17:16:55.753 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:55.751+0000 7f75abe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f75ac1042f0 0x7f75ac1a06e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:55.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:55.751+0000 7f75abe00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f75ac1093c0 0x7f75ac1a0c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:55.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:55.751+0000 7f75abe00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f75ac19a8a0 0x7f75ac19ad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:55.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:55.751+0000 7f75abe00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f75ac114430 con 0x7f75ac1042f0 2024-04-16T17:16:55.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:55.751+0000 7f75abe00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f75ac1142b0 con 0x7f75ac19a8a0 2024-04-16T17:16:55.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:55.751+0000 7f75abe00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f75ac1145b0 con 0x7f75ac1093c0 2024-04-16T17:16:55.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:55.752+0000 7f75aaa00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f75ac1042f0 0x7f75ac1a06e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:55.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:55.752+0000 7f75aaa00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f75ac1042f0 0x7f75ac1a06e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56978/0 (socket says 172.21.15.45:56978) 2024-04-16T17:16:55.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:55.752+0000 7f75aaa00640 1 -- 172.21.15.45:0/114788835 learned_addr learned my addr 172.21.15.45:0/114788835 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:16:55.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:55.752+0000 7f75aa000640 1 --2- 172.21.15.45:0/114788835 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f75ac1093c0 0x7f75ac1a0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:55.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:55.752+0000 7f75ab400640 1 --2- 172.21.15.45:0/114788835 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f75ac19a8a0 0x7f75ac19ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:55.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:55.752+0000 7f75aaa00640 1 -- 172.21.15.45:0/114788835 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f75ac1093c0 msgr2=0x7f75ac1a0c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:55.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:55.752+0000 7f75aaa00640 1 --2- 172.21.15.45:0/114788835 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f75ac1093c0 0x7f75ac1a0c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:55.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:55.752+0000 7f75aaa00640 1 -- 172.21.15.45:0/114788835 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f75ac19a8a0 msgr2=0x7f75ac19ad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:55.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:55.752+0000 7f75aaa00640 1 --2- 172.21.15.45:0/114788835 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f75ac19a8a0 0x7f75ac19ad50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:55.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:55.752+0000 7f75aaa00640 1 -- 172.21.15.45:0/114788835 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75ac19b430 con 0x7f75ac1042f0 2024-04-16T17:16:55.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:55.752+0000 7f75aaa00640 1 --2- 172.21.15.45:0/114788835 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f75ac1042f0 0x7f75ac1a06e0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f759c00e9f0 tx=0x7f759c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:16:55.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:55.753+0000 7f75a3400640 1 -- 172.21.15.45:0/114788835 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f759c00cd80 con 0x7f75ac1042f0 2024-04-16T17:16:55.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:55.753+0000 7f75abe00640 1 -- 172.21.15.45:0/114788835 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75ac19b720 con 0x7f75ac1042f0 2024-04-16T17:16:55.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:55.753+0000 7f75a3400640 1 -- 172.21.15.45:0/114788835 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f759c010910 con 0x7f75ac1042f0 2024-04-16T17:16:55.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:55.753+0000 7f75abe00640 1 -- 172.21.15.45:0/114788835 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75ac1a78d0 con 0x7f75ac1042f0 2024-04-16T17:16:55.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:55.753+0000 7f75a3400640 1 -- 172.21.15.45:0/114788835 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f759c018910 con 0x7f75ac1042f0 2024-04-16T17:16:55.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:55.757+0000 7f75b4a19740 1 Processor -- start 2024-04-16T17:16:55.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:55.758+0000 7f75b4a19740 1 -- start start 2024-04-16T17:16:55.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:55.758+0000 7f75a3400640 1 -- 172.21.15.45:0/114788835 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f759c010ac0 con 0x7f75ac1042f0 2024-04-16T17:16:55.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:55.758+0000 7f75b4a19740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cfaca8fca0 0x55cfaca90080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:55.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:55.758+0000 7f75b4a19740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cfaca90650 0x55cfacaa1660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:55.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:55.758+0000 7f75b4a19740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55cfacaa1ba0 0x55cfacaa3f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:55.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:55.758+0000 7f75b4a19740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55cfac9f4cb0 con 0x55cfaca8fca0 2024-04-16T17:16:55.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:55.758+0000 7f75b4a19740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55cfac99b370 con 0x55cfacaa1ba0 2024-04-16T17:16:55.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:55.758+0000 7f75b4a19740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55cfac9f0c70 con 0x55cfaca90650 2024-04-16T17:16:55.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:55.758+0000 7f75ab400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cfaca8fca0 0x55cfaca90080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:55.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:55.759+0000 7f75ab400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cfaca8fca0 0x55cfaca90080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56988/0 (socket says 172.21.15.45:56988) 2024-04-16T17:16:55.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:55.759+0000 7f75ab400640 1 -- 172.21.15.45:0/2612718829 learned_addr learned my addr 172.21.15.45:0/2612718829 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:16:55.761 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:55.759+0000 7f75aaa00640 1 --2- 172.21.15.45:0/2612718829 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cfaca90650 0x55cfacaa1660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:55.761 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:55.759+0000 7f75aa000640 1 --2- 172.21.15.45:0/2612718829 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55cfacaa1ba0 0x55cfacaa3f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:55.761 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:55.759+0000 7f75ab400640 1 -- 172.21.15.45:0/2612718829 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cfaca90650 msgr2=0x55cfacaa1660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:16:55.761 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:55.759+0000 7f75ab400640 1 --2- 172.21.15.45:0/2612718829 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cfaca90650 0x55cfacaa1660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:55.761 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:55.759+0000 7f75ab400640 1 -- 172.21.15.45:0/2612718829 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55cfacaa1ba0 msgr2=0x55cfacaa3f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:16:55.761 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:55.759+0000 7f75ab400640 1 --2- 172.21.15.45:0/2612718829 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55cfacaa1ba0 0x55cfacaa3f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:55.761 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:55.759+0000 7f75ab400640 1 -- 172.21.15.45:0/2612718829 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cfac9ba910 con 0x55cfaca8fca0 2024-04-16T17:16:55.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:55.759+0000 7f75ab400640 1 --2- 172.21.15.45:0/2612718829 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cfaca8fca0 0x55cfaca90080 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f75a40099f0 tx=0x7f75a40095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90cce4d334547e10 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:16:55.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:55.759+0000 7f75a3400640 1 --2- 172.21.15.45:0/114788835 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f75900a3d20 0x7f75900a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:55.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:55.760+0000 7f75a3400640 1 -- 172.21.15.45:0/114788835 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 9496+0+0 (secure 0 0 0) 0x7f759c0d0f80 con 0x7f75ac1042f0 2024-04-16T17:16:55.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:55.760+0000 7f7597e00640 1 -- 172.21.15.45:0/2612718829 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75a401d070 con 0x55cfaca8fca0 2024-04-16T17:16:55.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:55.760+0000 7f7597e00640 1 -- 172.21.15.45:0/2612718829 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f75a4003e40 con 0x55cfaca8fca0 2024-04-16T17:16:55.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:55.760+0000 7f7597e00640 1 -- 172.21.15.45:0/2612718829 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75a400db00 con 0x55cfaca8fca0 2024-04-16T17:16:55.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:55.760+0000 7f75aaa00640 1 --2- 172.21.15.45:0/114788835 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f75900a3d20 0x7f75900a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:55.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:55.761+0000 7f75b4a19740 1 -- 172.21.15.45:0/2612718829 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cfaca8fca0 msgr2=0x55cfaca90080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:55.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:55.761+0000 7f75b4a19740 1 --2- 172.21.15.45:0/2612718829 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cfaca8fca0 0x55cfaca90080 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f75a40099f0 tx=0x7f75a40095c0 comp rx=0 tx=0).stop 2024-04-16T17:16:55.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:55.761+0000 7f75aaa00640 1 --2- 172.21.15.45:0/114788835 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f75900a3d20 0x7f75900a61e0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f759c0ce3b0 tx=0x7f759c0d4ad0 comp rx=0 tx=0).ready entity=mgr.25411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:16:55.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:55.761+0000 7f75b4a19740 1 -- 172.21.15.45:0/2612718829 shutdown_connections 2024-04-16T17:16:55.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:55.761+0000 7f75b4a19740 1 --2- 172.21.15.45:0/2612718829 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cfaca90650 0x55cfacaa1660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:55.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:55.761+0000 7f75b4a19740 1 --2- 172.21.15.45:0/2612718829 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55cfacaa1ba0 0x55cfacaa3f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:55.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:55.761+0000 7f75b4a19740 1 --2- 172.21.15.45:0/2612718829 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cfaca8fca0 0x55cfaca90080 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:55.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:55.761+0000 7f75b4a19740 1 -- 172.21.15.45:0/2612718829 >> 172.21.15.45:0/2612718829 conn(0x55cfaca8a5c0 msgr2=0x55cfaca8a9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:16:55.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:55.761+0000 7f75b4a19740 1 -- 172.21.15.45:0/2612718829 shutdown_connections 2024-04-16T17:16:55.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:55.761+0000 7f75b4a19740 1 -- 172.21.15.45:0/2612718829 wait complete. 2024-04-16T17:16:55.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:55.761+0000 7f75b4a19740 1 -- start start 2024-04-16T17:16:55.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:55.762+0000 7f75b4a19740 1 Processor -- start 2024-04-16T17:16:55.768 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:55.762+0000 7f75b4a19740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cfaca8fca0 0x55cfaca9ec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:55.768 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:55.762+0000 7f75b4a19740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cfaca90650 0x55cfaca9f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:55.768 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:55.762+0000 7f75b4a19740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55cfacaa1ba0 0x55cfaca9f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:55.768 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:55.762+0000 7f75b4a19740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55cfacaa4960 con 0x55cfaca8fca0 2024-04-16T17:16:55.768 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:55.762+0000 7f75b4a19740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55cfac9ae730 con 0x55cfacaa1ba0 2024-04-16T17:16:55.768 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:55.762+0000 7f75b4a19740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55cfacaa4ae0 con 0x55cfaca90650 2024-04-16T17:16:55.768 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:55.762+0000 7f75b4a19740 10 client.0 _ll_register_callbacks cb 0x55cfac97d1e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:16:55.768 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:55.762+0000 7f75ab400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cfaca8fca0 0x55cfaca9ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:55.768 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:55.762+0000 7f75ab400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cfaca8fca0 0x55cfaca9ec60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56992/0 (socket says 172.21.15.45:56992) 2024-04-16T17:16:55.769 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:55.762+0000 7f75ab400640 1 -- 172.21.15.45:0/2175616579 learned_addr learned my addr 172.21.15.45:0/2175616579 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:16:55.769 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:55.763+0000 7f75ab400640 1 --2- 172.21.15.45:0/2175616579 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55cfacaa1ba0 0x55cfaca9f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:55.769 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:55.763+0000 7f75aa000640 1 --2- 172.21.15.45:0/2175616579 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cfaca90650 0x55cfaca9f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:55.769 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:55.763+0000 7f75ab400640 1 -- 172.21.15.45:0/2175616579 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cfaca90650 msgr2=0x55cfaca9f1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:16:55.769 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:55.763+0000 7f75ab400640 1 --2- 172.21.15.45:0/2175616579 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cfaca90650 0x55cfaca9f1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:55.769 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:55.763+0000 7f75ab400640 1 -- 172.21.15.45:0/2175616579 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55cfacaa1ba0 msgr2=0x55cfaca9f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:16:55.769 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:55.763+0000 7f75ab400640 1 --2- 172.21.15.45:0/2175616579 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55cfacaa1ba0 0x55cfaca9f6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:55.769 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:55.763+0000 7f75ab400640 1 -- 172.21.15.45:0/2175616579 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cfac7cf7e0 con 0x55cfaca8fca0 2024-04-16T17:16:55.769 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:55.763+0000 7f75ab400640 1 --2- 172.21.15.45:0/2175616579 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cfaca8fca0 0x55cfaca9ec60 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x55cfaca8b410 tx=0x7f75a400d6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c9b89d47c7e45fb server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:16:55.769 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:55.763+0000 7f7594c00640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-16T17:16:55.770 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:55.763+0000 7f7594c00640 1 -- 172.21.15.45:0/2175616579 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75a401d030 con 0x55cfaca8fca0 2024-04-16T17:16:55.770 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:55.763+0000 7f7594c00640 1 -- 172.21.15.45:0/2175616579 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9496+0+0 (secure 0 0 0) 0x7f75a400dd50 con 0x55cfaca8fca0 2024-04-16T17:16:55.770 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:55.764+0000 7f75b4a19740 1 -- 172.21.15.45:0/2175616579 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55cfaca026d0 con 0x55cfaca8fca0 2024-04-16T17:16:55.770 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:55.764+0000 7f7594c00640 1 -- 172.21.15.45:0/2175616579 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f75a4004a40 con 0x55cfaca8fca0 2024-04-16T17:16:55.770 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:55.764+0000 7f75b4a19740 10 client.18078 fetch_fsmap learned FSMap version 44 2024-04-16T17:16:55.770 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:55.764+0000 7f75b4a19740 1 -- 172.21.15.45:0/2175616579 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55cfac82ff70 con 0x55cfaca8fca0 2024-04-16T17:16:55.770 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:55.764+0000 7f7594c00640 1 -- 172.21.15.45:0/2175616579 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55cfaca026d0 con 0x55cfaca8fca0 2024-04-16T17:16:55.770 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:55.764+0000 7f7594c00640 1 -- 172.21.15.45:0/2175616579 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 44) ==== 8346+0+0 (secure 0 0 0) 0x7f75a400b040 con 0x55cfaca8fca0 2024-04-16T17:16:55.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:55.765+0000 7f75b4a19740 10 client.18078 fetch_fsmap finished waiting for FSMap version 44 2024-04-16T17:16:55.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:55.765+0000 7f75b4a19740 10 client.18078 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:16:55.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:55.765+0000 7f75b4a19740 20 client.18078 populate_metadata read hostname 'smithi045' 2024-04-16T17:16:55.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:55.765+0000 7f75b4a19740 1 --2- 172.21.15.45:0/2175616579 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55cfacab2010 0x55cfacab4400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:55.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:55.765+0000 7f75b4a19740 4 client.18078 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:16:55.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:55.765+0000 7f75b4a19740 1 -- 172.21.15.45:0/2175616579 --> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cfac9fc540 con 0x55cfacab2010 2024-04-16T17:16:55.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:55.765+0000 7f75aa000640 1 --2- 172.21.15.45:0/2175616579 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55cfacab2010 0x55cfacab4400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:55.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:55.765+0000 7f75aa000640 1 --2- 172.21.15.45:0/2175616579 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55cfacab2010 0x55cfacab4400 crc :-1 s=READY pgs=253 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4e963bba17fec8f9 server_cookie=3cbccb642aa7fa27 in_seq=0 out_seq=0 2024-04-16T17:16:55.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:55.765+0000 7f7594c00640 10 client.18078 ms_handle_connect on v2:172.21.15.104:6838/3660070592 2024-04-16T17:16:55.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:55.766+0000 7f7594c00640 1 -- 172.21.15.45:0/2175616579 <== mds.0 v2:172.21.15.104:6838/3660070592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55cfac9fc540 con 0x55cfacab2010 2024-04-16T17:16:55.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:55.766+0000 7f7594c00640 10 client.18078 handle_command_reply: tid=0 2024-04-16T17:16:55.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:55.766+0000 7f75b4a19740 1 client.18078 shutdown 2024-04-16T17:16:55.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:55.767+0000 7f75b4a19740 1 -- 172.21.15.45:0/2175616579 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cfaca8fca0 msgr2=0x55cfaca9ec60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:55.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:55.767+0000 7f75b4a19740 1 --2- 172.21.15.45:0/2175616579 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cfaca8fca0 0x55cfaca9ec60 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x55cfaca8b410 tx=0x7f75a400d6e0 comp rx=0 tx=0).stop 2024-04-16T17:16:55.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:55.767+0000 7f75b4a19740 1 -- 172.21.15.45:0/2175616579 shutdown_connections 2024-04-16T17:16:55.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:55.767+0000 7f75b4a19740 1 --2- 172.21.15.45:0/2175616579 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55cfacab2010 0x55cfacab4400 crc :-1 s=READY pgs=253 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:55.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:55.767+0000 7f75b4a19740 1 --2- 172.21.15.45:0/2175616579 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cfaca90650 0x55cfaca9f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:55.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:55.767+0000 7f75b4a19740 1 --2- 172.21.15.45:0/2175616579 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55cfacaa1ba0 0x55cfaca9f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:55.774 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:55.767+0000 7f75b4a19740 1 --2- 172.21.15.45:0/2175616579 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cfaca8fca0 0x55cfaca9ec60 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:55.774 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:55.767+0000 7f7594c00640 0 client.18078 ms_handle_reset on v2:172.21.15.104:6838/3660070592 2024-04-16T17:16:55.774 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:55.767+0000 7f75b4a19740 1 -- 172.21.15.45:0/2175616579 >> 172.21.15.45:0/2175616579 conn(0x55cfaca8a5c0 msgr2=0x55cfaca8c310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:16:55.774 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:55.767+0000 7f75b4a19740 1 -- 172.21.15.45:0/2175616579 shutdown_connections 2024-04-16T17:16:55.774 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:55.767+0000 7f75b4a19740 1 -- 172.21.15.45:0/2175616579 wait complete. 2024-04-16T17:16:55.774 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:55.767+0000 7f75b4a19740 20 client.18078 trim_cache size 0 max 16384 2024-04-16T17:16:55.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:55.786+0000 7f75b4a19740 1 Processor -- start 2024-04-16T17:16:55.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:55.787+0000 7f75b4a19740 1 -- start start 2024-04-16T17:16:55.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:55.787+0000 7f75b4a19740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cfaca8fca0 0x55cfacaa1390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:55.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:55.787+0000 7f75b4a19740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cfaca90650 0x55cfacaad860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:55.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:55.787+0000 7f75b4a19740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55cfacaa1ba0 0x55cfaca9db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:55.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:55.787+0000 7f75b4a19740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f75ac1afa30 con 0x55cfaca90650 2024-04-16T17:16:55.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:55.787+0000 7f75b4a19740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f75ac1ad300 con 0x55cfacaa1ba0 2024-04-16T17:16:55.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:55.787+0000 7f75b4a19740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f75ac1ab170 con 0x55cfaca8fca0 2024-04-16T17:16:55.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:55.787+0000 7f75aa000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cfaca90650 0x55cfacaad860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:55.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:55.787+0000 7f75aa000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cfaca90650 0x55cfacaad860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56996/0 (socket says 172.21.15.45:56996) 2024-04-16T17:16:55.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:55.787+0000 7f75aa000640 1 -- 172.21.15.45:0/1796353799 learned_addr learned my addr 172.21.15.45:0/1796353799 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:16:55.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:55.788+0000 7f75ab400640 1 --2- 172.21.15.45:0/1796353799 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cfaca8fca0 0x55cfacaa1390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:55.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:55.788+0000 7f75ab400640 1 --2- 172.21.15.45:0/1796353799 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55cfacaa1ba0 0x55cfaca9db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:55.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:55.788+0000 7f75aa000640 1 -- 172.21.15.45:0/1796353799 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cfaca8fca0 msgr2=0x55cfacaa1390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:16:55.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:55.788+0000 7f75aa000640 1 --2- 172.21.15.45:0/1796353799 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cfaca8fca0 0x55cfacaa1390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:55.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:55.788+0000 7f75aa000640 1 -- 172.21.15.45:0/1796353799 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55cfacaa1ba0 msgr2=0x55cfaca9db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:16:55.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:55.788+0000 7f75aa000640 1 --2- 172.21.15.45:0/1796353799 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55cfacaa1ba0 0x55cfaca9db10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:55.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:55.788+0000 7f75aa000640 1 -- 172.21.15.45:0/1796353799 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cfac6c7170 con 0x55cfaca90650 2024-04-16T17:16:55.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:55.788+0000 7f75ab400640 1 --2- 172.21.15.45:0/1796353799 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cfaca8fca0 0x55cfacaa1390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:16:55.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:55.788+0000 7f75aa000640 1 --2- 172.21.15.45:0/1796353799 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cfaca90650 0x55cfacaad860 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f75ac06b6b0 tx=0x7f75980056d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5dc3a2cab6dc5675 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:16:55.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:55.788+0000 7f7597400640 1 -- 172.21.15.45:0/1796353799 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f759804e070 con 0x55cfaca90650 2024-04-16T17:16:55.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:55.788+0000 7f7597400640 1 -- 172.21.15.45:0/1796353799 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7598034bb0 con 0x55cfaca90650 2024-04-16T17:16:55.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:55.788+0000 7f7597400640 1 -- 172.21.15.45:0/1796353799 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7598039040 con 0x55cfaca90650 2024-04-16T17:16:55.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:55.789+0000 7f75b4a19740 1 -- 172.21.15.45:0/1796353799 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cfaca90650 msgr2=0x55cfacaad860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:55.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:55.789+0000 7f75b4a19740 1 --2- 172.21.15.45:0/1796353799 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cfaca90650 0x55cfacaad860 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f75ac06b6b0 tx=0x7f75980056d0 comp rx=0 tx=0).stop 2024-04-16T17:16:55.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:55.789+0000 7f75b4a19740 1 -- 172.21.15.45:0/1796353799 shutdown_connections 2024-04-16T17:16:55.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:55.789+0000 7f75b4a19740 1 --2- 172.21.15.45:0/1796353799 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55cfacaa1ba0 0x55cfaca9db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:55.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:55.789+0000 7f75b4a19740 1 --2- 172.21.15.45:0/1796353799 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cfaca90650 0x55cfacaad860 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:55.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:55.789+0000 7f75b4a19740 1 --2- 172.21.15.45:0/1796353799 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cfaca8fca0 0x55cfacaa1390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:55.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:55.789+0000 7f75b4a19740 1 -- 172.21.15.45:0/1796353799 >> 172.21.15.45:0/1796353799 conn(0x55cfaca8a5c0 msgr2=0x55cfaca9d400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:16:55.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:55.789+0000 7f75b4a19740 1 -- 172.21.15.45:0/1796353799 shutdown_connections 2024-04-16T17:16:55.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:55.789+0000 7f75b4a19740 1 -- 172.21.15.45:0/1796353799 wait complete. 2024-04-16T17:16:55.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:55.790+0000 7f75b4a19740 1 -- start start 2024-04-16T17:16:55.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:55.790+0000 7f75b4a19740 1 Processor -- start 2024-04-16T17:16:55.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:55.790+0000 7f75b4a19740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cfaca8fca0 0x55cfacaaa060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:55.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:55.791+0000 7f75b4a19740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55cfaca90650 0x55cfacaa53d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:55.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:55.791+0000 7f75b4a19740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cfacaa1ba0 0x55cfacaa5910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:55.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:55.791+0000 7f75b4a19740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f75ac1ab7c0 con 0x55cfacaa1ba0 2024-04-16T17:16:55.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:55.791+0000 7f75b4a19740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f75ac1acc30 con 0x55cfaca90650 2024-04-16T17:16:55.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:55.791+0000 7f75b4a19740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f75ac1abe90 con 0x55cfaca8fca0 2024-04-16T17:16:55.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:55.791+0000 7f75b4a19740 10 client.0 _ll_register_callbacks cb 0x7f75a4009ee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:16:55.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:55.791+0000 7f75ab400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cfacaa1ba0 0x55cfacaa5910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:55.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:55.791+0000 7f75ab400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cfacaa1ba0 0x55cfacaa5910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57000/0 (socket says 172.21.15.45:57000) 2024-04-16T17:16:55.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:55.791+0000 7f75ab400640 1 -- 172.21.15.45:0/1054827760 learned_addr learned my addr 172.21.15.45:0/1054827760 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:16:55.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:55.791+0000 7f75ab400640 1 --2- 172.21.15.45:0/1054827760 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cfaca8fca0 0x55cfacaaa060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:55.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:55.791+0000 7f75aa000640 1 --2- 172.21.15.45:0/1054827760 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55cfaca90650 0x55cfacaa53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:55.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:55.791+0000 7f75ab400640 1 -- 172.21.15.45:0/1054827760 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cfaca8fca0 msgr2=0x55cfacaaa060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:16:55.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:55.791+0000 7f75ab400640 1 --2- 172.21.15.45:0/1054827760 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cfaca8fca0 0x55cfacaaa060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:55.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:55.791+0000 7f75ab400640 1 -- 172.21.15.45:0/1054827760 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55cfaca90650 msgr2=0x55cfacaa53d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:16:55.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:55.791+0000 7f75ab400640 1 --2- 172.21.15.45:0/1054827760 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55cfaca90650 0x55cfacaa53d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:55.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:55.791+0000 7f75ab400640 1 -- 172.21.15.45:0/1054827760 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cfacaa6050 con 0x55cfacaa1ba0 2024-04-16T17:16:55.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:55.792+0000 7f75aa000640 1 --2- 172.21.15.45:0/1054827760 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55cfaca90650 0x55cfacaa53d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:16:55.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:55.792+0000 7f75ab400640 1 --2- 172.21.15.45:0/1054827760 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cfacaa1ba0 0x55cfacaa5910 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f75a4005d50 tx=0x7f75a400d6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7f905b6fbeb0563 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:16:55.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:55.792+0000 7f7594c00640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-16T17:16:55.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:55.792+0000 7f7594c00640 1 -- 172.21.15.45:0/1054827760 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75a401d040 con 0x55cfacaa1ba0 2024-04-16T17:16:55.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:55.792+0000 7f75b4a19740 1 -- 172.21.15.45:0/1054827760 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7598034bb0 con 0x55cfacaa1ba0 2024-04-16T17:16:55.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:55.792+0000 7f7594c00640 1 -- 172.21.15.45:0/1054827760 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9496+0+0 (secure 0 0 0) 0x7f75a4005640 con 0x55cfacaa1ba0 2024-04-16T17:16:55.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:55.792+0000 7f7594c00640 1 -- 172.21.15.45:0/1054827760 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f75a4005840 con 0x55cfacaa1ba0 2024-04-16T17:16:55.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:55.793+0000 7f75b4a19740 10 client.18084 fetch_fsmap learned FSMap version 44 2024-04-16T17:16:55.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:55.793+0000 7f75b4a19740 1 -- 172.21.15.45:0/1054827760 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55cfacb183a0 con 0x55cfacaa1ba0 2024-04-16T17:16:55.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:55.793+0000 7f7594c00640 1 -- 172.21.15.45:0/1054827760 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7598034bb0 con 0x55cfacaa1ba0 2024-04-16T17:16:55.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:55.793+0000 7f7594c00640 1 -- 172.21.15.45:0/1054827760 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 44) ==== 8346+0+0 (secure 0 0 0) 0x7f75a400b070 con 0x55cfacaa1ba0 2024-04-16T17:16:55.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:55.793+0000 7f75b4a19740 10 client.18084 fetch_fsmap finished waiting for FSMap version 44 2024-04-16T17:16:55.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:55.793+0000 7f75b4a19740 10 client.18084 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:16:55.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:55.793+0000 7f75b4a19740 20 client.18084 populate_metadata read hostname 'smithi045' 2024-04-16T17:16:55.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:55.793+0000 7f75b4a19740 1 --2- 172.21.15.45:0/1054827760 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55cfacb18780 0x55cfacb1ab70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:55.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:55.793+0000 7f75b4a19740 4 client.18084 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-16T17:16:55.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:55.793+0000 7f75b4a19740 1 -- 172.21.15.45:0/1054827760 --> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55cfac978aa0 con 0x55cfacb18780 2024-04-16T17:16:55.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:55.794+0000 7f75aa000640 1 --2- 172.21.15.45:0/1054827760 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55cfacb18780 0x55cfacb1ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:55.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:55.794+0000 7f75aa000640 1 --2- 172.21.15.45:0/1054827760 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55cfacb18780 0x55cfacb1ab70 crc :-1 s=READY pgs=254 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2acf9333185454f4 server_cookie=b823db8013b2bd12 in_seq=0 out_seq=0 2024-04-16T17:16:55.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:55.794+0000 7f7594c00640 10 client.18084 ms_handle_connect on v2:172.21.15.104:6838/3660070592 2024-04-16T17:16:55.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:55.795+0000 7f7594c00640 1 -- 172.21.15.45:0/1054827760 <== mds.0 v2:172.21.15.104:6838/3660070592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55cfac978aa0 con 0x55cfacb18780 2024-04-16T17:16:55.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:55.795+0000 7f7594c00640 10 client.18084 handle_command_reply: tid=0 2024-04-16T17:16:55.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:55.795+0000 7f75b4a19740 1 client.18084 shutdown 2024-04-16T17:16:55.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:55.796+0000 7f75b4a19740 1 -- 172.21.15.45:0/1054827760 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cfacaa1ba0 msgr2=0x55cfacaa5910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:55.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:55.796+0000 7f75b4a19740 1 --2- 172.21.15.45:0/1054827760 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cfacaa1ba0 0x55cfacaa5910 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f75a4005d50 tx=0x7f75a400d6e0 comp rx=0 tx=0).stop 2024-04-16T17:16:55.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:55.796+0000 7f75b4a19740 1 -- 172.21.15.45:0/1054827760 shutdown_connections 2024-04-16T17:16:55.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:55.796+0000 7f75b4a19740 1 --2- 172.21.15.45:0/1054827760 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cfacaa1ba0 0x55cfacaa5910 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:55.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:55.796+0000 7f75b4a19740 1 --2- 172.21.15.45:0/1054827760 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55cfaca90650 0x55cfacaa53d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:55.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:55.796+0000 7f75b4a19740 1 --2- 172.21.15.45:0/1054827760 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x55cfacb18780 0x55cfacb1ab70 crc :-1 s=READY pgs=254 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:55.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:55.796+0000 7f75b4a19740 1 --2- 172.21.15.45:0/1054827760 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cfaca8fca0 0x55cfacaaa060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:55.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:55.796+0000 7f75b4a19740 1 -- 172.21.15.45:0/1054827760 >> 172.21.15.45:0/1054827760 conn(0x55cfaca8a5c0 msgr2=0x55cfaca9dc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:16:55.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:55.796+0000 7f7594c00640 0 client.18084 ms_handle_reset on v2:172.21.15.104:6838/3660070592 2024-04-16T17:16:55.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:55.796+0000 7f75b4a19740 1 -- 172.21.15.45:0/1054827760 shutdown_connections 2024-04-16T17:16:55.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:55.796+0000 7f75b4a19740 1 -- 172.21.15.45:0/1054827760 wait complete. 2024-04-16T17:16:55.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:55.796+0000 7f75b4a19740 20 client.18084 trim_cache size 0 max 16384 2024-04-16T17:16:55.803 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-16T17:16:55.803 INFO:teuthology.orchestra.run.smithi045.stdout: "return_code": 0, 2024-04-16T17:16:55.803 INFO:teuthology.orchestra.run.smithi045.stdout: "scrub_tag": "081db9a7-a429-4e71-b94d-55251de0e4fb", 2024-04-16T17:16:55.803 INFO:teuthology.orchestra.run.smithi045.stdout: "mode": "asynchronous" 2024-04-16T17:16:55.803 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-16T17:16:55.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:55.797+0000 7f75abe00640 1 -- 172.21.15.45:0/114788835 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f75900a3d20 msgr2=0x7f75900a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:55.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:55.797+0000 7f75abe00640 1 --2- 172.21.15.45:0/114788835 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f75900a3d20 0x7f75900a61e0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f759c0ce3b0 tx=0x7f759c0d4ad0 comp rx=0 tx=0).stop 2024-04-16T17:16:55.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:55.797+0000 7f75abe00640 1 -- 172.21.15.45:0/114788835 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f75ac1042f0 msgr2=0x7f75ac1a06e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:55.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:55.797+0000 7f75abe00640 1 --2- 172.21.15.45:0/114788835 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f75ac1042f0 0x7f75ac1a06e0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f759c00e9f0 tx=0x7f759c00eeb0 comp rx=0 tx=0).stop 2024-04-16T17:16:55.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:55.797+0000 7f75abe00640 1 -- 172.21.15.45:0/114788835 shutdown_connections 2024-04-16T17:16:55.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:55.797+0000 7f75abe00640 1 --2- 172.21.15.45:0/114788835 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f75900a3d20 0x7f75900a61e0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:55.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:55.797+0000 7f75abe00640 1 --2- 172.21.15.45:0/114788835 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f75ac1093c0 0x7f75ac1a0c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:55.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:55.797+0000 7f75abe00640 1 --2- 172.21.15.45:0/114788835 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f75ac19a8a0 0x7f75ac19ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:55.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:55.797+0000 7f75abe00640 1 --2- 172.21.15.45:0/114788835 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f75ac1042f0 0x7f75ac1a06e0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:55.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:55.797+0000 7f75abe00640 1 -- 172.21.15.45:0/114788835 >> 172.21.15.45:0/114788835 conn(0x7f75ac0ffb10 msgr2=0x7f75ac100e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:16:55.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:55.797+0000 7f75abe00640 1 -- 172.21.15.45:0/114788835 shutdown_connections 2024-04-16T17:16:55.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:55.797+0000 7f75abe00640 1 -- 172.21.15.45:0/114788835 wait complete. 2024-04-16T17:16:55.821 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-16T17:16:55.821 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-16T17:16:56.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.038+0000 7f0452c00640 1 -- 172.21.15.45:0/1319100173 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f044c103980 msgr2=0x7f044c10be60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:56.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.038+0000 7f0452c00640 1 --2- 172.21.15.45:0/1319100173 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f044c103980 0x7f044c10be60 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f0438009860 tx=0x7f043802bc20 comp rx=0 tx=0).stop 2024-04-16T17:16:56.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.038+0000 7f0452c00640 1 -- 172.21.15.45:0/1319100173 shutdown_connections 2024-04-16T17:16:56.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.038+0000 7f0452c00640 1 --2- 172.21.15.45:0/1319100173 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f044c10c480 0x7f044c10e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:56.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.038+0000 7f0452c00640 1 --2- 172.21.15.45:0/1319100173 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f044c103980 0x7f044c10be60 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:56.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.038+0000 7f0452c00640 1 --2- 172.21.15.45:0/1319100173 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f044c103060 0x7f044c103440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:56.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.038+0000 7f0452c00640 1 -- 172.21.15.45:0/1319100173 >> 172.21.15.45:0/1319100173 conn(0x7f044c0fc880 msgr2=0x7f044c0feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:16:56.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.038+0000 7f0452c00640 1 -- 172.21.15.45:0/1319100173 shutdown_connections 2024-04-16T17:16:56.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.038+0000 7f0452c00640 1 -- 172.21.15.45:0/1319100173 wait complete. 2024-04-16T17:16:56.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.039+0000 7f0452c00640 1 Processor -- start 2024-04-16T17:16:56.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.039+0000 7f0452c00640 1 -- start start 2024-04-16T17:16:56.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.040+0000 7f0452c00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f044c103060 0x7f044c1a0630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:56.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.040+0000 7f0452c00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f044c103980 0x7f044c1a0b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:56.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.040+0000 7f0452c00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f044c10c480 0x7f044c19a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:56.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.040+0000 7f0452c00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f044c112260 con 0x7f044c103980 2024-04-16T17:16:56.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.040+0000 7f0452c00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f044c1120e0 con 0x7f044c10c480 2024-04-16T17:16:56.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.040+0000 7f0452c00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f044c1123e0 con 0x7f044c103060 2024-04-16T17:16:56.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.040+0000 7f0450e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f044c103980 0x7f044c1a0b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:56.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.040+0000 7f0450e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f044c103980 0x7f044c1a0b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57016/0 (socket says 172.21.15.45:57016) 2024-04-16T17:16:56.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.040+0000 7f0450e00640 1 -- 172.21.15.45:0/2014236198 learned_addr learned my addr 172.21.15.45:0/2014236198 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:16:56.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.040+0000 7f0451800640 1 --2- 172.21.15.45:0/2014236198 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f044c103060 0x7f044c1a0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:56.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.040+0000 7f0452200640 1 --2- 172.21.15.45:0/2014236198 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f044c10c480 0x7f044c19a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:56.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.040+0000 7f0450e00640 1 -- 172.21.15.45:0/2014236198 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f044c103060 msgr2=0x7f044c1a0630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:56.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.040+0000 7f0450e00640 1 --2- 172.21.15.45:0/2014236198 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f044c103060 0x7f044c1a0630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:56.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.040+0000 7f0450e00640 1 -- 172.21.15.45:0/2014236198 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f044c10c480 msgr2=0x7f044c19a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:56.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.040+0000 7f0450e00640 1 --2- 172.21.15.45:0/2014236198 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f044c10c480 0x7f044c19a7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:56.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.040+0000 7f0450e00640 1 -- 172.21.15.45:0/2014236198 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f044c19af70 con 0x7f044c103980 2024-04-16T17:16:56.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.041+0000 7f0450e00640 1 --2- 172.21.15.45:0/2014236198 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f044c103980 0x7f044c1a0b70 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f0438037040 tx=0x7f04380344c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:16:56.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.041+0000 7f044a000640 1 -- 172.21.15.45:0/2014236198 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0438041020 con 0x7f044c103980 2024-04-16T17:16:56.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.041+0000 7f044a000640 1 -- 172.21.15.45:0/2014236198 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f043802ce50 con 0x7f044c103980 2024-04-16T17:16:56.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.041+0000 7f044a000640 1 -- 172.21.15.45:0/2014236198 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0438039040 con 0x7f044c103980 2024-04-16T17:16:56.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.041+0000 7f0452c00640 1 -- 172.21.15.45:0/2014236198 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f044c19b200 con 0x7f044c103980 2024-04-16T17:16:56.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.041+0000 7f0452c00640 1 -- 172.21.15.45:0/2014236198 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f044c1a7480 con 0x7f044c103980 2024-04-16T17:16:56.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.045+0000 7f0457009740 1 Processor -- start 2024-04-16T17:16:56.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.045+0000 7f0457009740 1 -- start start 2024-04-16T17:16:56.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.045+0000 7f044a000640 1 -- 172.21.15.45:0/2014236198 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f0438044020 con 0x7f044c103980 2024-04-16T17:16:56.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.045+0000 7f0457009740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564191075390 0x564191075770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:56.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.045+0000 7f0457009740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564191075cb0 0x56419107e350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:56.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.045+0000 7f0457009740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56419107e890 0x564191080c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:56.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.045+0000 7f0457009740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x564190fdb120 con 0x564191075390 2024-04-16T17:16:56.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.045+0000 7f0457009740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x564190f81340 con 0x56419107e890 2024-04-16T17:16:56.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.045+0000 7f0457009740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x564190fd70e0 con 0x564191075cb0 2024-04-16T17:16:56.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.046+0000 7f0452200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564191075390 0x564191075770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:56.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.046+0000 7f0452200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564191075390 0x564191075770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57030/0 (socket says 172.21.15.45:57030) 2024-04-16T17:16:56.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.046+0000 7f0452200640 1 -- 172.21.15.45:0/766370785 learned_addr learned my addr 172.21.15.45:0/766370785 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:16:56.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.046+0000 7f0450e00640 1 --2- 172.21.15.45:0/766370785 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56419107e890 0x564191080c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:56.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.046+0000 7f0451800640 1 --2- 172.21.15.45:0/766370785 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564191075cb0 0x56419107e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:56.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.046+0000 7f0452200640 1 -- 172.21.15.45:0/766370785 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564191075cb0 msgr2=0x56419107e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:16:56.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.046+0000 7f0452200640 1 --2- 172.21.15.45:0/766370785 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564191075cb0 0x56419107e350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:56.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.046+0000 7f0452200640 1 -- 172.21.15.45:0/766370785 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56419107e890 msgr2=0x564191080c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:16:56.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.046+0000 7f0452200640 1 --2- 172.21.15.45:0/766370785 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56419107e890 0x564191080c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:56.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.046+0000 7f0452200640 1 -- 172.21.15.45:0/766370785 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564190db77d0 con 0x564191075390 2024-04-16T17:16:56.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.047+0000 7f0452200640 1 --2- 172.21.15.45:0/766370785 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564191075390 0x564191075770 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f0444009c20 tx=0x7f0444011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4625d1a0a1350872 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:16:56.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.047+0000 7f044a000640 1 --2- 172.21.15.45:0/2014236198 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f04300a3da0 0x7f04300a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:56.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.047+0000 7f0442a00640 1 -- 172.21.15.45:0/766370785 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f044401e070 con 0x564191075390 2024-04-16T17:16:56.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.047+0000 7f0442a00640 1 -- 172.21.15.45:0/766370785 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0444019910 con 0x564191075390 2024-04-16T17:16:56.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.047+0000 7f0442a00640 1 -- 172.21.15.45:0/766370785 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f044400e910 con 0x564191075390 2024-04-16T17:16:56.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.047+0000 7f044a000640 1 -- 172.21.15.45:0/2014236198 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 9496+0+0 (secure 0 0 0) 0x7f04380f1ee0 con 0x7f044c103980 2024-04-16T17:16:56.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.047+0000 7f0451800640 1 --2- 172.21.15.45:0/2014236198 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f04300a3da0 0x7f04300a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:56.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.048+0000 7f0457009740 1 -- 172.21.15.45:0/766370785 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564191075390 msgr2=0x564191075770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:56.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.048+0000 7f0457009740 1 --2- 172.21.15.45:0/766370785 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564191075390 0x564191075770 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f0444009c20 tx=0x7f0444011810 comp rx=0 tx=0).stop 2024-04-16T17:16:56.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.048+0000 7f0457009740 1 -- 172.21.15.45:0/766370785 shutdown_connections 2024-04-16T17:16:56.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.048+0000 7f0457009740 1 --2- 172.21.15.45:0/766370785 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564191075cb0 0x56419107e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:56.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.048+0000 7f0457009740 1 --2- 172.21.15.45:0/766370785 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56419107e890 0x564191080c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:56.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.048+0000 7f0457009740 1 --2- 172.21.15.45:0/766370785 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564191075390 0x564191075770 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:56.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.048+0000 7f0457009740 1 -- 172.21.15.45:0/766370785 >> 172.21.15.45:0/766370785 conn(0x56419106fac0 msgr2=0x56419106fea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:16:56.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.048+0000 7f0457009740 1 -- 172.21.15.45:0/766370785 shutdown_connections 2024-04-16T17:16:56.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.048+0000 7f0451800640 1 --2- 172.21.15.45:0/2014236198 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f04300a3da0 0x7f04300a6260 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x564191074e60 tx=0x7f043c010040 comp rx=0 tx=0).ready entity=mgr.25411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:16:56.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.048+0000 7f0457009740 1 -- 172.21.15.45:0/766370785 wait complete. 2024-04-16T17:16:56.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.049+0000 7f0457009740 1 -- start start 2024-04-16T17:16:56.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.049+0000 7f0457009740 1 Processor -- start 2024-04-16T17:16:56.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.049+0000 7f0457009740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564191075390 0x56419107b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:56.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.049+0000 7f0457009740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564191075cb0 0x56419107bf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:56.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.049+0000 7f0457009740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56419107e890 0x56419107c470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:56.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.049+0000 7f0457009740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x564191081680 con 0x564191075390 2024-04-16T17:16:56.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.049+0000 7f0457009740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x564190f94710 con 0x564191075cb0 2024-04-16T17:16:56.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.049+0000 7f0457009740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x564191081800 con 0x56419107e890 2024-04-16T17:16:56.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.050+0000 7f0457009740 10 client.0 _ll_register_callbacks cb 0x564190f8faf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:16:56.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.050+0000 7f0451800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564191075390 0x56419107b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:56.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.050+0000 7f0451800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564191075390 0x56419107b9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57046/0 (socket says 172.21.15.45:57046) 2024-04-16T17:16:56.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.050+0000 7f0451800640 1 -- 172.21.15.45:0/2930740346 learned_addr learned my addr 172.21.15.45:0/2930740346 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:16:56.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.050+0000 7f0452200640 1 --2- 172.21.15.45:0/2930740346 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56419107e890 0x56419107c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:56.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.050+0000 7f0450e00640 1 --2- 172.21.15.45:0/2930740346 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564191075cb0 0x56419107bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:56.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.050+0000 7f0451800640 1 -- 172.21.15.45:0/2930740346 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56419107e890 msgr2=0x56419107c470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:16:56.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.050+0000 7f0451800640 1 --2- 172.21.15.45:0/2930740346 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56419107e890 0x56419107c470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:56.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.050+0000 7f0451800640 1 -- 172.21.15.45:0/2930740346 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564191075cb0 msgr2=0x56419107bf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:16:56.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.050+0000 7f0451800640 1 --2- 172.21.15.45:0/2930740346 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564191075cb0 0x56419107bf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:56.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.050+0000 7f0451800640 1 -- 172.21.15.45:0/2930740346 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564190ced3f0 con 0x564191075390 2024-04-16T17:16:56.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.050+0000 7f0452200640 1 --2- 172.21.15.45:0/2930740346 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56419107e890 0x56419107c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:16:56.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.051+0000 7f0451800640 1 --2- 172.21.15.45:0/2930740346 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564191075390 0x56419107b9f0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f043c00ffd0 tx=0x7f043c004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3e20e81ecc82168 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:16:56.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.051+0000 7f0437400640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-16T17:16:56.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.051+0000 7f0437400640 1 -- 172.21.15.45:0/2930740346 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f043c018460 con 0x564191075390 2024-04-16T17:16:56.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.051+0000 7f0437400640 1 -- 172.21.15.45:0/2930740346 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9496+0+0 (secure 0 0 0) 0x7f043c018600 con 0x564191075390 2024-04-16T17:16:56.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.051+0000 7f0457009740 1 -- 172.21.15.45:0/2930740346 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0444019910 con 0x564191075390 2024-04-16T17:16:56.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.051+0000 7f0437400640 1 -- 172.21.15.45:0/2930740346 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f043c032050 con 0x564191075390 2024-04-16T17:16:56.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.051+0000 7f0457009740 10 client.18096 fetch_fsmap learned FSMap version 44 2024-04-16T17:16:56.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.051+0000 7f0457009740 1 -- 172.21.15.45:0/2930740346 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x564190cad0f0 con 0x564191075390 2024-04-16T17:16:56.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.052+0000 7f0437400640 1 -- 172.21.15.45:0/2930740346 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0444019910 con 0x564191075390 2024-04-16T17:16:56.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.052+0000 7f0437400640 1 -- 172.21.15.45:0/2930740346 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 44) ==== 8346+0+0 (secure 0 0 0) 0x7f043c00b040 con 0x564191075390 2024-04-16T17:16:56.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.052+0000 7f0457009740 10 client.18096 fetch_fsmap finished waiting for FSMap version 44 2024-04-16T17:16:56.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.052+0000 7f0457009740 10 client.18096 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:16:56.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.052+0000 7f0457009740 20 client.18096 populate_metadata read hostname 'smithi045' 2024-04-16T17:16:56.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.052+0000 7f0457009740 1 --2- 172.21.15.45:0/2930740346 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x56419108ee20 0x564191091290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:56.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.052+0000 7f0457009740 4 client.18096 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:16:56.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.052+0000 7f0457009740 1 -- 172.21.15.45:0/2930740346 --> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564190fe8b40 con 0x56419108ee20 2024-04-16T17:16:56.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.052+0000 7f0450e00640 1 --2- 172.21.15.45:0/2930740346 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x56419108ee20 0x564191091290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:56.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.053+0000 7f0450e00640 1 --2- 172.21.15.45:0/2930740346 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x56419108ee20 0x564191091290 crc :-1 s=READY pgs=255 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6d9f850f51351dac server_cookie=6460e371186640df in_seq=0 out_seq=0 2024-04-16T17:16:56.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.053+0000 7f0437400640 10 client.18096 ms_handle_connect on v2:172.21.15.104:6838/3660070592 2024-04-16T17:16:56.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.054+0000 7f0437400640 1 -- 172.21.15.45:0/2930740346 <== mds.0 v2:172.21.15.104:6838/3660070592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564190fe8b40 con 0x56419108ee20 2024-04-16T17:16:56.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.054+0000 7f0437400640 10 client.18096 handle_command_reply: tid=0 2024-04-16T17:16:56.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.054+0000 7f0457009740 1 client.18096 shutdown 2024-04-16T17:16:56.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.054+0000 7f0457009740 1 -- 172.21.15.45:0/2930740346 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564191075390 msgr2=0x56419107b9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:56.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.054+0000 7f0457009740 1 --2- 172.21.15.45:0/2930740346 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564191075390 0x56419107b9f0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f043c00ffd0 tx=0x7f043c004ab0 comp rx=0 tx=0).stop 2024-04-16T17:16:56.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.054+0000 7f0457009740 1 -- 172.21.15.45:0/2930740346 shutdown_connections 2024-04-16T17:16:56.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.054+0000 7f0457009740 1 --2- 172.21.15.45:0/2930740346 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x56419108ee20 0x564191091290 crc :-1 s=READY pgs=255 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:56.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.054+0000 7f0457009740 1 --2- 172.21.15.45:0/2930740346 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56419107e890 0x56419107c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:56.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.055+0000 7f0437400640 0 client.18096 ms_handle_reset on v2:172.21.15.104:6838/3660070592 2024-04-16T17:16:56.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.054+0000 7f0457009740 1 --2- 172.21.15.45:0/2930740346 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564191075cb0 0x56419107bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:56.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.055+0000 7f0457009740 1 --2- 172.21.15.45:0/2930740346 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564191075390 0x56419107b9f0 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:56.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.055+0000 7f0457009740 1 -- 172.21.15.45:0/2930740346 >> 172.21.15.45:0/2930740346 conn(0x56419106fac0 msgr2=0x564191071810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:16:56.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.055+0000 7f0457009740 1 -- 172.21.15.45:0/2930740346 shutdown_connections 2024-04-16T17:16:56.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.055+0000 7f0457009740 1 -- 172.21.15.45:0/2930740346 wait complete. 2024-04-16T17:16:56.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.055+0000 7f0457009740 20 client.18096 trim_cache size 0 max 16384 2024-04-16T17:16:56.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.074+0000 7f0457009740 1 Processor -- start 2024-04-16T17:16:56.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.074+0000 7f0457009740 1 -- start start 2024-04-16T17:16:56.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.074+0000 7f0457009740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56419106fac0 0x56419107e120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:56.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.074+0000 7f0457009740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564191075390 0x56419108a570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:56.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.074+0000 7f0457009740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564191075cb0 0x56419106e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:56.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.074+0000 7f0457009740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f044c1b3e20 con 0x56419106fac0 2024-04-16T17:16:56.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.074+0000 7f0457009740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f044c1b16f0 con 0x564191075390 2024-04-16T17:16:56.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.074+0000 7f0457009740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f044c1af560 con 0x564191075cb0 2024-04-16T17:16:56.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.075+0000 7f0451800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56419106fac0 0x56419107e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:56.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.075+0000 7f0451800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56419106fac0 0x56419107e120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57052/0 (socket says 172.21.15.45:57052) 2024-04-16T17:16:56.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.075+0000 7f0451800640 1 -- 172.21.15.45:0/2833879574 learned_addr learned my addr 172.21.15.45:0/2833879574 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:16:56.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.075+0000 7f0452200640 1 --2- 172.21.15.45:0/2833879574 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564191075cb0 0x56419106e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:56.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.075+0000 7f0450e00640 1 --2- 172.21.15.45:0/2833879574 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564191075390 0x56419108a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:56.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.075+0000 7f0451800640 1 -- 172.21.15.45:0/2833879574 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564191075cb0 msgr2=0x56419106e9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:16:56.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.075+0000 7f0451800640 1 --2- 172.21.15.45:0/2833879574 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564191075cb0 0x56419106e9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:56.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.075+0000 7f0451800640 1 -- 172.21.15.45:0/2833879574 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564191075390 msgr2=0x56419108a570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:16:56.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.075+0000 7f0451800640 1 --2- 172.21.15.45:0/2833879574 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564191075390 0x56419108a570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:56.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.075+0000 7f0451800640 1 -- 172.21.15.45:0/2833879574 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56419106b380 con 0x56419106fac0 2024-04-16T17:16:56.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.076+0000 7f0451800640 1 --2- 172.21.15.45:0/2833879574 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56419106fac0 0x56419107e120 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f043c01afd0 tx=0x7f043c0099b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12891b6604a77226 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:16:56.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.076+0000 7f0442000640 1 -- 172.21.15.45:0/2833879574 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f043c01ac00 con 0x56419106fac0 2024-04-16T17:16:56.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.076+0000 7f0442000640 1 -- 172.21.15.45:0/2833879574 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f043c03d070 con 0x56419106fac0 2024-04-16T17:16:56.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.076+0000 7f0442000640 1 -- 172.21.15.45:0/2833879574 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f043c02eaa0 con 0x56419106fac0 2024-04-16T17:16:56.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.077+0000 7f0457009740 1 -- 172.21.15.45:0/2833879574 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56419106fac0 msgr2=0x56419107e120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:56.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.077+0000 7f0457009740 1 --2- 172.21.15.45:0/2833879574 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56419106fac0 0x56419107e120 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f043c01afd0 tx=0x7f043c0099b0 comp rx=0 tx=0).stop 2024-04-16T17:16:56.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.077+0000 7f0457009740 1 -- 172.21.15.45:0/2833879574 shutdown_connections 2024-04-16T17:16:56.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.077+0000 7f0457009740 1 --2- 172.21.15.45:0/2833879574 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564191075cb0 0x56419106e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:56.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.077+0000 7f0457009740 1 --2- 172.21.15.45:0/2833879574 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564191075390 0x56419108a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:56.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.077+0000 7f0457009740 1 --2- 172.21.15.45:0/2833879574 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56419106fac0 0x56419107e120 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:56.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.077+0000 7f0457009740 1 -- 172.21.15.45:0/2833879574 >> 172.21.15.45:0/2833879574 conn(0x56419106af50 msgr2=0x56419106e290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:16:56.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.077+0000 7f0457009740 1 -- 172.21.15.45:0/2833879574 shutdown_connections 2024-04-16T17:16:56.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.077+0000 7f0457009740 1 -- 172.21.15.45:0/2833879574 wait complete. 2024-04-16T17:16:56.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.077+0000 7f0457009740 1 -- start start 2024-04-16T17:16:56.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.078+0000 7f0457009740 1 Processor -- start 2024-04-16T17:16:56.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.078+0000 7f0457009740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56419106fac0 0x5641910827f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:56.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.078+0000 7f0457009740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564191075390 0x564191082d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:56.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.078+0000 7f0457009740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564191075cb0 0x5641910fcbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:56.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.078+0000 7f0457009740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f044c1afbb0 con 0x564191075cb0 2024-04-16T17:16:56.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.078+0000 7f0457009740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f044c1b1020 con 0x564191075390 2024-04-16T17:16:56.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.078+0000 7f0457009740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f044c1b0280 con 0x56419106fac0 2024-04-16T17:16:56.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.078+0000 7f0457009740 10 client.0 _ll_register_callbacks cb 0x7f043c009900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:16:56.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.078+0000 7f0452200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564191075cb0 0x5641910fcbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:56.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.078+0000 7f0451800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56419106fac0 0x5641910827f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:56.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.078+0000 7f0451800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56419106fac0 0x5641910827f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:40456/0 (socket says 172.21.15.45:40456) 2024-04-16T17:16:56.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.078+0000 7f0451800640 1 -- 172.21.15.45:0/4285773911 learned_addr learned my addr 172.21.15.45:0/4285773911 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:16:56.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.078+0000 7f0450e00640 1 --2- 172.21.15.45:0/4285773911 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564191075390 0x564191082d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:56.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.079+0000 7f0451800640 1 -- 172.21.15.45:0/4285773911 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564191075390 msgr2=0x564191082d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:16:56.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.079+0000 7f0451800640 1 --2- 172.21.15.45:0/4285773911 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564191075390 0x564191082d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:56.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.079+0000 7f0451800640 1 -- 172.21.15.45:0/4285773911 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564191075cb0 msgr2=0x5641910fcbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:16:56.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.079+0000 7f0451800640 1 --2- 172.21.15.45:0/4285773911 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564191075cb0 0x5641910fcbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:56.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.079+0000 7f0451800640 1 -- 172.21.15.45:0/4285773911 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564191083600 con 0x56419106fac0 2024-04-16T17:16:56.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.079+0000 7f0450e00640 1 --2- 172.21.15.45:0/4285773911 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564191075390 0x564191082d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:16:56.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.079+0000 7f0451800640 1 --2- 172.21.15.45:0/4285773911 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56419106fac0 0x5641910827f0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f043c01a990 tx=0x7f043c009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dae8ad2757e0b4f4 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:16:56.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.079+0000 7f0437400640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-04-16T17:16:56.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.079+0000 7f0437400640 1 -- 172.21.15.45:0/4285773911 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f043c045020 con 0x56419106fac0 2024-04-16T17:16:56.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.079+0000 7f0437400640 1 -- 172.21.15.45:0/4285773911 <== mon.1 v2:172.21.15.117:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9496+0+0 (secure 0 0 0) 0x7f043c01ad80 con 0x56419106fac0 2024-04-16T17:16:56.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.079+0000 7f0457009740 1 -- 172.21.15.45:0/4285773911 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f043c03d070 con 0x56419106fac0 2024-04-16T17:16:56.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.080+0000 7f0437400640 1 -- 172.21.15.45:0/4285773911 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f043c050050 con 0x56419106fac0 2024-04-16T17:16:56.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.080+0000 7f0457009740 10 client.28204 fetch_fsmap learned FSMap version 44 2024-04-16T17:16:56.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.080+0000 7f0457009740 1 -- 172.21.15.45:0/4285773911 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x5641910fe230 con 0x56419106fac0 2024-04-16T17:16:56.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.080+0000 7f0437400640 1 -- 172.21.15.45:0/4285773911 <== mon.1 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f043c03d070 con 0x56419106fac0 2024-04-16T17:16:56.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.080+0000 7f0437400640 1 -- 172.21.15.45:0/4285773911 <== mon.1 v2:172.21.15.117:3300/0 5 ==== fsmap(e 44) ==== 8346+0+0 (secure 0 0 0) 0x7f043c02e7e0 con 0x56419106fac0 2024-04-16T17:16:56.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.080+0000 7f0457009740 10 client.28204 fetch_fsmap finished waiting for FSMap version 44 2024-04-16T17:16:56.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.080+0000 7f0457009740 10 client.28204 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:16:56.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.080+0000 7f0457009740 20 client.28204 populate_metadata read hostname 'smithi045' 2024-04-16T17:16:56.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.080+0000 7f0457009740 1 --2- 172.21.15.45:0/4285773911 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x56419107e890 0x5641911008a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:56.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.080+0000 7f0457009740 4 client.28204 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-16T17:16:56.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.080+0000 7f0457009740 1 -- 172.21.15.45:0/4285773911 --> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] -- command(tid 0: {"prefix": "scrub status"}) -- 0x564190fe29b0 con 0x56419107e890 2024-04-16T17:16:56.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.081+0000 7f0450e00640 1 --2- 172.21.15.45:0/4285773911 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x56419107e890 0x5641911008a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:56.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.081+0000 7f0450e00640 1 --2- 172.21.15.45:0/4285773911 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x56419107e890 0x5641911008a0 crc :-1 s=READY pgs=256 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ea5e0dfde05cc7f6 server_cookie=a25b9b8b4f15cae8 in_seq=0 out_seq=0 2024-04-16T17:16:56.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.081+0000 7f0437400640 10 client.28204 ms_handle_connect on v2:172.21.15.104:6838/3660070592 2024-04-16T17:16:56.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.081+0000 7f0437400640 1 -- 172.21.15.45:0/4285773911 <== mds.0 v2:172.21.15.104:6838/3660070592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x564190fe29b0 con 0x56419107e890 2024-04-16T17:16:56.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.081+0000 7f0437400640 10 client.28204 handle_command_reply: tid=0 2024-04-16T17:16:56.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.081+0000 7f0457009740 1 client.28204 shutdown 2024-04-16T17:16:56.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.082+0000 7f0457009740 1 -- 172.21.15.45:0/4285773911 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56419106fac0 msgr2=0x5641910827f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:56.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.082+0000 7f0457009740 1 --2- 172.21.15.45:0/4285773911 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56419106fac0 0x5641910827f0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f043c01a990 tx=0x7f043c009250 comp rx=0 tx=0).stop 2024-04-16T17:16:56.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.082+0000 7f0457009740 1 -- 172.21.15.45:0/4285773911 shutdown_connections 2024-04-16T17:16:56.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.082+0000 7f0457009740 1 --2- 172.21.15.45:0/4285773911 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564191075cb0 0x5641910fcbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:56.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.082+0000 7f0457009740 1 --2- 172.21.15.45:0/4285773911 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564191075390 0x564191082d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:56.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.082+0000 7f0457009740 1 --2- 172.21.15.45:0/4285773911 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x56419107e890 0x5641911008a0 crc :-1 s=READY pgs=256 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:56.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.082+0000 7f0457009740 1 --2- 172.21.15.45:0/4285773911 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56419106fac0 0x5641910827f0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:56.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.082+0000 7f0457009740 1 -- 172.21.15.45:0/4285773911 >> 172.21.15.45:0/4285773911 conn(0x56419106af50 msgr2=0x56419107c220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:16:56.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.082+0000 7f0437400640 0 client.28204 ms_handle_reset on v2:172.21.15.104:6838/3660070592 2024-04-16T17:16:56.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.082+0000 7f0457009740 1 -- 172.21.15.45:0/4285773911 shutdown_connections 2024-04-16T17:16:56.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.082+0000 7f0457009740 1 -- 172.21.15.45:0/4285773911 wait complete. 2024-04-16T17:16:56.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.082+0000 7f0457009740 20 client.28204 trim_cache size 0 max 16384 2024-04-16T17:16:56.090 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-16T17:16:56.090 INFO:teuthology.orchestra.run.smithi045.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-16T17:16:56.090 INFO:teuthology.orchestra.run.smithi045.stdout: "scrubs": { 2024-04-16T17:16:56.090 INFO:teuthology.orchestra.run.smithi045.stdout: "081db9a7-a429-4e71-b94d-55251de0e4fb": { 2024-04-16T17:16:56.090 INFO:teuthology.orchestra.run.smithi045.stdout: "path": "/", 2024-04-16T17:16:56.090 INFO:teuthology.orchestra.run.smithi045.stdout: "tag": "081db9a7-a429-4e71-b94d-55251de0e4fb", 2024-04-16T17:16:56.091 INFO:teuthology.orchestra.run.smithi045.stdout: "options": "recursive,force" 2024-04-16T17:16:56.091 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:16:56.091 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:16:56.091 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-16T17:16:56.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.083+0000 7f0452c00640 1 -- 172.21.15.45:0/2014236198 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f04300a3da0 msgr2=0x7f04300a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:56.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.083+0000 7f0452c00640 1 --2- 172.21.15.45:0/2014236198 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f04300a3da0 0x7f04300a6260 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x564191074e60 tx=0x7f043c010040 comp rx=0 tx=0).stop 2024-04-16T17:16:56.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.083+0000 7f0452c00640 1 -- 172.21.15.45:0/2014236198 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f044c103980 msgr2=0x7f044c1a0b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:56.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.083+0000 7f0452c00640 1 --2- 172.21.15.45:0/2014236198 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f044c103980 0x7f044c1a0b70 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f0438037040 tx=0x7f04380344c0 comp rx=0 tx=0).stop 2024-04-16T17:16:56.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.083+0000 7f0452c00640 1 -- 172.21.15.45:0/2014236198 shutdown_connections 2024-04-16T17:16:56.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.083+0000 7f0452c00640 1 --2- 172.21.15.45:0/2014236198 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f04300a3da0 0x7f04300a6260 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:56.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.083+0000 7f0452c00640 1 --2- 172.21.15.45:0/2014236198 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f044c10c480 0x7f044c19a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:56.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.083+0000 7f0452c00640 1 --2- 172.21.15.45:0/2014236198 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f044c103980 0x7f044c1a0b70 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:56.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.083+0000 7f0452c00640 1 --2- 172.21.15.45:0/2014236198 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f044c103060 0x7f044c1a0630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:56.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.083+0000 7f0452c00640 1 -- 172.21.15.45:0/2014236198 >> 172.21.15.45:0/2014236198 conn(0x7f044c0fc880 msgr2=0x7f044c101ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:16:56.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.083+0000 7f0452c00640 1 -- 172.21.15.45:0/2014236198 shutdown_connections 2024-04-16T17:16:56.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.083+0000 7f0452c00640 1 -- 172.21.15.45:0/2014236198 wait complete. 2024-04-16T17:16:56.101 INFO:tasks.cephfs.filesystem:scrub status for tag:081db9a7-a429-4e71-b94d-55251de0e4fb - {'path': '/', 'tag': '081db9a7-a429-4e71-b94d-55251de0e4fb', 'options': 'recursive,force'} 2024-04-16T17:16:56.398 DEBUG:teuthology.orchestra.run.smithi045:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-04-16T17:16:56.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.628+0000 7f0d74a00640 1 -- 172.21.15.45:0/4054485561 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0d7010e4d0 msgr2=0x7f0d7010e8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:56.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.628+0000 7f0d74a00640 1 --2- 172.21.15.45:0/4054485561 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0d7010e4d0 0x7f0d7010e8b0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f0d60009930 tx=0x7f0d6002bc30 comp rx=0 tx=0).stop 2024-04-16T17:16:56.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.628+0000 7f0d74a00640 1 -- 172.21.15.45:0/4054485561 shutdown_connections 2024-04-16T17:16:56.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.628+0000 7f0d74a00640 1 --2- 172.21.15.45:0/4054485561 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0d70108ec0 0x7f0d70115300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:56.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.628+0000 7f0d74a00640 1 --2- 172.21.15.45:0/4054485561 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0d70108540 0x7f0d70108980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:56.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.628+0000 7f0d74a00640 1 --2- 172.21.15.45:0/4054485561 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0d7010e4d0 0x7f0d7010e8b0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:56.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.628+0000 7f0d74a00640 1 -- 172.21.15.45:0/4054485561 >> 172.21.15.45:0/4054485561 conn(0x7f0d70077de0 msgr2=0x7f0d70106e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:16:56.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.628+0000 7f0d74a00640 1 -- 172.21.15.45:0/4054485561 shutdown_connections 2024-04-16T17:16:56.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.628+0000 7f0d74a00640 1 -- 172.21.15.45:0/4054485561 wait complete. 2024-04-16T17:16:56.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.629+0000 7f0d74a00640 1 Processor -- start 2024-04-16T17:16:56.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.629+0000 7f0d74a00640 1 -- start start 2024-04-16T17:16:56.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.630+0000 7f0d74a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0d70108540 0x7f0d7019ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:56.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.630+0000 7f0d74a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0d70108ec0 0x7f0d7019f290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:56.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.630+0000 7f0d74a00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0d7010e4d0 0x7f0d701a3610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:56.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.630+0000 7f0d74a00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f0d701187f0 con 0x7f0d70108540 2024-04-16T17:16:56.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.630+0000 7f0d74a00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f0d70118670 con 0x7f0d7010e4d0 2024-04-16T17:16:56.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.630+0000 7f0d74a00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f0d70118970 con 0x7f0d70108ec0 2024-04-16T17:16:56.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.630+0000 7f0d6e800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0d70108540 0x7f0d7019ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:56.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.630+0000 7f0d65e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0d70108ec0 0x7f0d7019f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:56.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.630+0000 7f0d6e800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0d70108540 0x7f0d7019ed50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57080/0 (socket says 172.21.15.45:57080) 2024-04-16T17:16:56.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.631+0000 7f0d6e800640 1 -- 172.21.15.45:0/1261112313 learned_addr learned my addr 172.21.15.45:0/1261112313 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:16:56.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.631+0000 7f0d6f200640 1 --2- 172.21.15.45:0/1261112313 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0d7010e4d0 0x7f0d701a3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:56.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.631+0000 7f0d65e00640 1 -- 172.21.15.45:0/1261112313 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0d7010e4d0 msgr2=0x7f0d701a3610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:56.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.631+0000 7f0d65e00640 1 --2- 172.21.15.45:0/1261112313 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0d7010e4d0 0x7f0d701a3610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:56.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.631+0000 7f0d65e00640 1 -- 172.21.15.45:0/1261112313 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0d70108540 msgr2=0x7f0d7019ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:56.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.631+0000 7f0d65e00640 1 --2- 172.21.15.45:0/1261112313 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0d70108540 0x7f0d7019ed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:56.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.631+0000 7f0d65e00640 1 -- 172.21.15.45:0/1261112313 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d701a3d40 con 0x7f0d70108ec0 2024-04-16T17:16:56.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.631+0000 7f0d6e800640 1 --2- 172.21.15.45:0/1261112313 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0d70108540 0x7f0d7019ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:16:56.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.631+0000 7f0d65e00640 1 --2- 172.21.15.45:0/1261112313 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0d70108ec0 0x7f0d7019f290 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f0d58009fd0 tx=0x7f0d5800eed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:16:56.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.632+0000 7f0d67e00640 1 -- 172.21.15.45:0/1261112313 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d5800cde0 con 0x7f0d70108ec0 2024-04-16T17:16:56.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.632+0000 7f0d74a00640 1 -- 172.21.15.45:0/1261112313 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d701a4030 con 0x7f0d70108ec0 2024-04-16T17:16:56.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.632+0000 7f0d67e00640 1 -- 172.21.15.45:0/1261112313 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d58005020 con 0x7f0d70108ec0 2024-04-16T17:16:56.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.632+0000 7f0d67e00640 1 -- 172.21.15.45:0/1261112313 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d580106d0 con 0x7f0d70108ec0 2024-04-16T17:16:56.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.632+0000 7f0d74a00640 1 -- 172.21.15.45:0/1261112313 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d701ab8c0 con 0x7f0d70108ec0 2024-04-16T17:16:56.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.633+0000 7f0d74a00640 1 -- 172.21.15.45:0/1261112313 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d700505e0 con 0x7f0d70108ec0 2024-04-16T17:16:56.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.635+0000 7f0d67e00640 1 -- 172.21.15.45:0/1261112313 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f0d58005540 con 0x7f0d70108ec0 2024-04-16T17:16:56.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.636+0000 7f0d67e00640 1 --2- 172.21.15.45:0/1261112313 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f0d540a3d90 0x7f0d540a6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:56.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.637+0000 7f0d67e00640 1 -- 172.21.15.45:0/1261112313 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 9496+0+0 (secure 0 0 0) 0x7f0d580d18a0 con 0x7f0d70108ec0 2024-04-16T17:16:56.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.637+0000 7f0d6e800640 1 --2- 172.21.15.45:0/1261112313 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f0d540a3d90 0x7f0d540a6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:56.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.638+0000 7f0d6e800640 1 --2- 172.21.15.45:0/1261112313 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f0d540a3d90 0x7f0d540a6250 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f0d60000c00 tx=0x7f0d600058d0 comp rx=0 tx=0).ready entity=mgr.25411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:16:56.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.639+0000 7f0d67e00640 1 -- 172.21.15.45:0/1261112313 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0d5809db10 con 0x7f0d70108ec0 2024-04-16T17:16:56.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:16:56 smithi045 ceph-mon[31676]: pgmap v317: 129 pgs: 129 active+clean; 56 MiB data, 870 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-04-16T17:16:56.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:16:56 smithi045 ceph-mon[31676]: scrub summary: active paths [/] 2024-04-16T17:16:56.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:16:56 smithi045 ceph-mon[31676]: scrub summary: active paths [/] 2024-04-16T17:16:56.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:16:56 smithi045 ceph-mon[31676]: scrub summary: idle+waiting paths [/] 2024-04-16T17:16:56.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:16:56 smithi045 ceph-mon[31676]: scrub summary: active paths [/] 2024-04-16T17:16:56.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:16:56 smithi045 ceph-mon[31676]: scrub summary: idle+waiting paths [/] 2024-04-16T17:16:56.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:16:56 smithi045 ceph-mon[31676]: scrub summary: idle+waiting paths [/] 2024-04-16T17:16:56.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.874+0000 7f0d74a00640 1 -- 172.21.15.45:0/1261112313 --> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7f0d70062440 con 0x7f0d540a3d90 2024-04-16T17:16:56.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.889+0000 7f0d67e00640 1 -- 172.21.15.45:0/1261112313 <== mgr.25411 v2:172.21.15.45:6800/751128195 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+350 (secure 0 0 0) 0x7f0d70062440 con 0x7f0d540a3d90 2024-04-16T17:16:56.890 INFO:teuthology.orchestra.run.smithi045.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-04-16T00:00:00", "created": "2024-04-16T17:06:55", "first": "2024-04-16T17:07:00", "last": "2024-04-16T17:16:00", "last_pruned": "2024-04-16T17:16:00", "created_count": 10, "pruned_count": 4, "active": true} 2024-04-16T17:16:56.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.892+0000 7f0d74a00640 1 -- 172.21.15.45:0/1261112313 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f0d540a3d90 msgr2=0x7f0d540a6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:56.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.892+0000 7f0d74a00640 1 --2- 172.21.15.45:0/1261112313 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f0d540a3d90 0x7f0d540a6250 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f0d60000c00 tx=0x7f0d600058d0 comp rx=0 tx=0).stop 2024-04-16T17:16:56.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.892+0000 7f0d74a00640 1 -- 172.21.15.45:0/1261112313 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0d70108ec0 msgr2=0x7f0d7019f290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:56.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.892+0000 7f0d74a00640 1 --2- 172.21.15.45:0/1261112313 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0d70108ec0 0x7f0d7019f290 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f0d58009fd0 tx=0x7f0d5800eed0 comp rx=0 tx=0).stop 2024-04-16T17:16:56.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.892+0000 7f0d74a00640 1 -- 172.21.15.45:0/1261112313 shutdown_connections 2024-04-16T17:16:56.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.892+0000 7f0d74a00640 1 --2- 172.21.15.45:0/1261112313 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f0d540a3d90 0x7f0d540a6250 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:56.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.892+0000 7f0d74a00640 1 --2- 172.21.15.45:0/1261112313 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0d70108ec0 0x7f0d7019f290 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:56.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.892+0000 7f0d74a00640 1 --2- 172.21.15.45:0/1261112313 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0d7010e4d0 0x7f0d701a3610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:56.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.892+0000 7f0d74a00640 1 --2- 172.21.15.45:0/1261112313 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0d70108540 0x7f0d7019ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:56.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.892+0000 7f0d74a00640 1 -- 172.21.15.45:0/1261112313 >> 172.21.15.45:0/1261112313 conn(0x7f0d70077de0 msgr2=0x7f0d70079000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:16:56.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.892+0000 7f0d74a00640 1 -- 172.21.15.45:0/1261112313 shutdown_connections 2024-04-16T17:16:56.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:56.892+0000 7f0d74a00640 1 -- 172.21.15.45:0/1261112313 wait complete. 2024-04-16T17:16:56.912 DEBUG:teuthology.orchestra.run.smithi045:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-04-16T17:16:56.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:16:56 smithi104 ceph-mon[36457]: pgmap v317: 129 pgs: 129 active+clean; 56 MiB data, 870 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-04-16T17:16:56.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:16:56 smithi104 ceph-mon[36457]: scrub summary: active paths [/] 2024-04-16T17:16:56.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:16:56 smithi104 ceph-mon[36457]: scrub summary: active paths [/] 2024-04-16T17:16:56.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:16:56 smithi104 ceph-mon[36457]: scrub summary: idle+waiting paths [/] 2024-04-16T17:16:56.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:16:56 smithi104 ceph-mon[36457]: scrub summary: active paths [/] 2024-04-16T17:16:56.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:16:56 smithi104 ceph-mon[36457]: scrub summary: idle+waiting paths [/] 2024-04-16T17:16:56.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:16:56 smithi104 ceph-mon[36457]: scrub summary: idle+waiting paths [/] 2024-04-16T17:16:57.054 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:16:56 smithi117 ceph-mon[36730]: pgmap v317: 129 pgs: 129 active+clean; 56 MiB data, 870 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-04-16T17:16:57.055 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:16:56 smithi117 ceph-mon[36730]: scrub summary: active paths [/] 2024-04-16T17:16:57.055 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:16:56 smithi117 ceph-mon[36730]: scrub summary: active paths [/] 2024-04-16T17:16:57.055 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:16:56 smithi117 ceph-mon[36730]: scrub summary: idle+waiting paths [/] 2024-04-16T17:16:57.055 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:16:56 smithi117 ceph-mon[36730]: scrub summary: active paths [/] 2024-04-16T17:16:57.055 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:16:56 smithi117 ceph-mon[36730]: scrub summary: idle+waiting paths [/] 2024-04-16T17:16:57.055 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:16:56 smithi117 ceph-mon[36730]: scrub summary: idle+waiting paths [/] 2024-04-16T17:16:57.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:57.115+0000 7fc3e7200640 1 -- 172.21.15.45:0/223567834 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc3e8112ce0 msgr2=0x7fc3e81150d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:57.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:57.115+0000 7fc3e7200640 1 --2- 172.21.15.45:0/223567834 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc3e8112ce0 0x7fc3e81150d0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fc3e0009c10 tx=0x7fc3e002bef0 comp rx=0 tx=0).stop 2024-04-16T17:16:57.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:57.115+0000 7fc3e7200640 1 -- 172.21.15.45:0/223567834 shutdown_connections 2024-04-16T17:16:57.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:57.115+0000 7fc3e7200640 1 --2- 172.21.15.45:0/223567834 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc3e8112ce0 0x7fc3e81150d0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:57.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:57.115+0000 7fc3e7200640 1 --2- 172.21.15.45:0/223567834 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc3e8076e50 0x7fc3e81127a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:57.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:57.115+0000 7fc3e7200640 1 --2- 172.21.15.45:0/223567834 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc3e8076530 0x7fc3e8076910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:57.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:57.115+0000 7fc3e7200640 1 -- 172.21.15.45:0/223567834 >> 172.21.15.45:0/223567834 conn(0x7fc3e80ffb10 msgr2=0x7fc3e8101f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:16:57.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:57.116+0000 7fc3e7200640 1 -- 172.21.15.45:0/223567834 shutdown_connections 2024-04-16T17:16:57.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:57.116+0000 7fc3e7200640 1 -- 172.21.15.45:0/223567834 wait complete. 2024-04-16T17:16:57.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:57.117+0000 7fc3e7200640 1 Processor -- start 2024-04-16T17:16:57.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:57.117+0000 7fc3e7200640 1 -- start start 2024-04-16T17:16:57.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:57.117+0000 7fc3e7200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc3e8076530 0x7fc3e81a4a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:57.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:57.117+0000 7fc3e7200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc3e8076e50 0x7fc3e81a4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:57.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:57.117+0000 7fc3e7200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc3e8112ce0 0x7fc3e819eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:57.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:57.118+0000 7fc3e7200640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fc3e8118850 con 0x7fc3e8112ce0 2024-04-16T17:16:57.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:57.118+0000 7fc3e7200640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fc3e81186d0 con 0x7fc3e8076e50 2024-04-16T17:16:57.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:57.118+0000 7fc3e7200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fc3e81189d0 con 0x7fc3e8076530 2024-04-16T17:16:57.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:57.118+0000 7fc3e6800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc3e8112ce0 0x7fc3e819eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:57.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:57.118+0000 7fc3e5e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc3e8076530 0x7fc3e81a4a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:57.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:57.118+0000 7fc3e6800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc3e8112ce0 0x7fc3e819eb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57096/0 (socket says 172.21.15.45:57096) 2024-04-16T17:16:57.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:57.118+0000 7fc3e6800640 1 -- 172.21.15.45:0/948259648 learned_addr learned my addr 172.21.15.45:0/948259648 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:16:57.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:57.118+0000 7fc3e5400640 1 --2- 172.21.15.45:0/948259648 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc3e8076e50 0x7fc3e81a4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:57.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:57.118+0000 7fc3e6800640 1 -- 172.21.15.45:0/948259648 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc3e8076530 msgr2=0x7fc3e81a4a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:57.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:57.118+0000 7fc3e6800640 1 --2- 172.21.15.45:0/948259648 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc3e8076530 0x7fc3e81a4a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:57.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:57.118+0000 7fc3e6800640 1 -- 172.21.15.45:0/948259648 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc3e8076e50 msgr2=0x7fc3e81a4fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:57.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:57.118+0000 7fc3e6800640 1 --2- 172.21.15.45:0/948259648 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc3e8076e50 0x7fc3e81a4fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:57.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:57.118+0000 7fc3e6800640 1 -- 172.21.15.45:0/948259648 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3e819f3f0 con 0x7fc3e8112ce0 2024-04-16T17:16:57.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:57.118+0000 7fc3e5e00640 1 --2- 172.21.15.45:0/948259648 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc3e8076530 0x7fc3e81a4a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:16:57.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:57.119+0000 7fc3e6800640 1 --2- 172.21.15.45:0/948259648 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc3e8112ce0 0x7fc3e819eb30 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fc3e0004af0 tx=0x7fc3e00336b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:16:57.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:57.119+0000 7fc3dea00640 1 -- 172.21.15.45:0/948259648 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3e0043020 con 0x7fc3e8112ce0 2024-04-16T17:16:57.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:57.119+0000 7fc3dea00640 1 -- 172.21.15.45:0/948259648 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc3e0003e50 con 0x7fc3e8112ce0 2024-04-16T17:16:57.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:57.119+0000 7fc3dea00640 1 -- 172.21.15.45:0/948259648 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3e003e6b0 con 0x7fc3e8112ce0 2024-04-16T17:16:57.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:57.119+0000 7fc3e7200640 1 -- 172.21.15.45:0/948259648 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3e819f680 con 0x7fc3e8112ce0 2024-04-16T17:16:57.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:57.119+0000 7fc3e7200640 1 -- 172.21.15.45:0/948259648 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3e81ab810 con 0x7fc3e8112ce0 2024-04-16T17:16:57.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:57.121+0000 7fc3e7200640 1 -- 172.21.15.45:0/948259648 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3e8077c70 con 0x7fc3e8112ce0 2024-04-16T17:16:57.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:57.123+0000 7fc3dea00640 1 -- 172.21.15.45:0/948259648 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fc3e0046020 con 0x7fc3e8112ce0 2024-04-16T17:16:57.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:57.124+0000 7fc3dea00640 1 --2- 172.21.15.45:0/948259648 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fc3cc0a3d90 0x7fc3cc0a6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:16:57.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:57.124+0000 7fc3dea00640 1 -- 172.21.15.45:0/948259648 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 9496+0+0 (secure 0 0 0) 0x7fc3e00f3380 con 0x7fc3e8112ce0 2024-04-16T17:16:57.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:57.124+0000 7fc3e5e00640 1 --2- 172.21.15.45:0/948259648 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fc3cc0a3d90 0x7fc3cc0a6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:16:57.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:57.125+0000 7fc3e5e00640 1 --2- 172.21.15.45:0/948259648 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fc3cc0a3d90 0x7fc3cc0a6250 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fc3d80096c0 tx=0x7fc3d8009260 comp rx=0 tx=0).ready entity=mgr.25411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:16:57.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:57.128+0000 7fc3dea00640 1 -- 172.21.15.45:0/948259648 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc3e004d020 con 0x7fc3e8112ce0 2024-04-16T17:16:57.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:57.367+0000 7fc3e7200640 1 -- 172.21.15.45:0/948259648 --> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7fc3e8062440 con 0x7fc3cc0a3d90 2024-04-16T17:16:57.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:57.383+0000 7fc3dea00640 1 -- 172.21.15.45:0/948259648 <== mgr.25411 v2:172.21.15.45:6800/751128195 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7fc3e8062440 con 0x7fc3cc0a3d90 2024-04-16T17:16:57.384 INFO:teuthology.orchestra.run.smithi045.stdout:/ 1m 6m3h 2024-04-16T17:16:57.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:57.386+0000 7fc3e7200640 1 -- 172.21.15.45:0/948259648 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fc3cc0a3d90 msgr2=0x7fc3cc0a6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:57.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:57.386+0000 7fc3e7200640 1 --2- 172.21.15.45:0/948259648 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fc3cc0a3d90 0x7fc3cc0a6250 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fc3d80096c0 tx=0x7fc3d8009260 comp rx=0 tx=0).stop 2024-04-16T17:16:57.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:57.386+0000 7fc3e7200640 1 -- 172.21.15.45:0/948259648 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc3e8112ce0 msgr2=0x7fc3e819eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:16:57.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:57.386+0000 7fc3e7200640 1 --2- 172.21.15.45:0/948259648 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc3e8112ce0 0x7fc3e819eb30 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fc3e0004af0 tx=0x7fc3e00336b0 comp rx=0 tx=0).stop 2024-04-16T17:16:57.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:57.387+0000 7fc3e7200640 1 -- 172.21.15.45:0/948259648 shutdown_connections 2024-04-16T17:16:57.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:57.387+0000 7fc3e7200640 1 --2- 172.21.15.45:0/948259648 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fc3cc0a3d90 0x7fc3cc0a6250 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:57.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:57.387+0000 7fc3e7200640 1 --2- 172.21.15.45:0/948259648 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc3e8112ce0 0x7fc3e819eb30 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:57.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:57.387+0000 7fc3e7200640 1 --2- 172.21.15.45:0/948259648 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc3e8076e50 0x7fc3e81a4fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:57.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:57.387+0000 7fc3e7200640 1 --2- 172.21.15.45:0/948259648 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc3e8076530 0x7fc3e81a4a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:16:57.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:57.387+0000 7fc3e7200640 1 -- 172.21.15.45:0/948259648 >> 172.21.15.45:0/948259648 conn(0x7fc3e80ffb10 msgr2=0x7fc3e8101f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:16:57.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:57.387+0000 7fc3e7200640 1 -- 172.21.15.45:0/948259648 shutdown_connections 2024-04-16T17:16:57.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:16:57.387+0000 7fc3e7200640 1 -- 172.21.15.45:0/948259648 wait complete. 2024-04-16T17:16:57.408 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2024-04-16T17:16:57.419 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2024-04-16T17:16:57.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:16:57 smithi045 ceph-mon[31676]: scrub queued for path: / 2024-04-16T17:16:57.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:16:57 smithi045 ceph-mon[31676]: scrub summary: idle+waiting paths [/] 2024-04-16T17:16:57.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:16:57 smithi045 ceph-mon[31676]: scrub summary: active paths [/] 2024-04-16T17:16:57.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:16:57 smithi045 ceph-mon[31676]: scrub summary: idle+waiting paths [/] 2024-04-16T17:16:57.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:16:57 smithi104 ceph-mon[36457]: scrub queued for path: / 2024-04-16T17:16:57.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:16:57 smithi104 ceph-mon[36457]: scrub summary: idle+waiting paths [/] 2024-04-16T17:16:57.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:16:57 smithi104 ceph-mon[36457]: scrub summary: active paths [/] 2024-04-16T17:16:57.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:16:57 smithi104 ceph-mon[36457]: scrub summary: idle+waiting paths [/] 2024-04-16T17:16:58.054 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:16:57 smithi117 ceph-mon[36730]: scrub queued for path: / 2024-04-16T17:16:58.054 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:16:57 smithi117 ceph-mon[36730]: scrub summary: idle+waiting paths [/] 2024-04-16T17:16:58.054 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:16:57 smithi117 ceph-mon[36730]: scrub summary: active paths [/] 2024-04-16T17:16:58.054 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:16:57 smithi117 ceph-mon[36730]: scrub summary: idle+waiting paths [/] 2024-04-16T17:16:58.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:16:58 smithi104 ceph-mon[36457]: from='client.28210 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T17:16:58.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:16:58 smithi104 ceph-mon[36457]: from='client.18114 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T17:16:58.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:16:58 smithi104 ceph-mon[36457]: pgmap v318: 129 pgs: 129 active+clean; 56 MiB data, 870 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 14 op/s 2024-04-16T17:16:58.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:16:58 smithi045 ceph-mon[31676]: from='client.28210 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T17:16:58.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:16:58 smithi045 ceph-mon[31676]: from='client.18114 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T17:16:58.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:16:58 smithi045 ceph-mon[31676]: pgmap v318: 129 pgs: 129 active+clean; 56 MiB data, 870 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 14 op/s 2024-04-16T17:16:59.054 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:16:58 smithi117 ceph-mon[36730]: from='client.28210 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T17:16:59.054 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:16:58 smithi117 ceph-mon[36730]: from='client.18114 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-16T17:16:59.054 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:16:58 smithi117 ceph-mon[36730]: pgmap v318: 129 pgs: 129 active+clean; 56 MiB data, 870 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 14 op/s 2024-04-16T17:17:00.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:17:00 smithi104 ceph-mon[36457]: pgmap v319: 129 pgs: 129 active+clean; 56 MiB data, 870 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 11 op/s 2024-04-16T17:17:00.937 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:17:00 smithi045 ceph-mon[31676]: pgmap v319: 129 pgs: 129 active+clean; 56 MiB data, 870 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 11 op/s 2024-04-16T17:17:01.054 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:17:00 smithi117 ceph-mon[36730]: pgmap v319: 129 pgs: 129 active+clean; 56 MiB data, 870 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 11 op/s 2024-04-16T17:17:02.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:17:02 smithi104 ceph-mon[36457]: pgmap v320: 129 pgs: 129 active+clean; 56 MiB data, 870 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 45 KiB/s wr, 34 op/s 2024-04-16T17:17:02.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:17:02 smithi045 ceph-mon[31676]: pgmap v320: 129 pgs: 129 active+clean; 56 MiB data, 870 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 45 KiB/s wr, 34 op/s 2024-04-16T17:17:03.054 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:17:02 smithi117 ceph-mon[36730]: pgmap v320: 129 pgs: 129 active+clean; 56 MiB data, 870 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 45 KiB/s wr, 34 op/s 2024-04-16T17:17:03.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:17:03 smithi104 ceph-mon[36457]: scrub summary: idle 2024-04-16T17:17:03.976 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:17:03 smithi045 ceph-mon[31676]: scrub summary: idle 2024-04-16T17:17:04.054 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:17:03 smithi117 ceph-mon[36730]: scrub summary: idle 2024-04-16T17:17:04.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:17:04 smithi104 ceph-mon[36457]: pgmap v321: 129 pgs: 129 active+clean; 56 MiB data, 870 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 45 KiB/s wr, 31 op/s 2024-04-16T17:17:04.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:17:04 smithi104 ceph-mon[36457]: scrub summary: idle 2024-04-16T17:17:04.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:17:04 smithi104 ceph-mon[36457]: scrub summary: idle 2024-04-16T17:17:04.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:17:04 smithi045 ceph-mon[31676]: pgmap v321: 129 pgs: 129 active+clean; 56 MiB data, 870 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 45 KiB/s wr, 31 op/s 2024-04-16T17:17:04.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:17:04 smithi045 ceph-mon[31676]: scrub summary: idle 2024-04-16T17:17:04.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:17:04 smithi045 ceph-mon[31676]: scrub summary: idle 2024-04-16T17:17:05.054 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:17:04 smithi117 ceph-mon[36730]: pgmap v321: 129 pgs: 129 active+clean; 56 MiB data, 870 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 45 KiB/s wr, 31 op/s 2024-04-16T17:17:05.054 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:17:04 smithi117 ceph-mon[36730]: scrub summary: idle 2024-04-16T17:17:05.055 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:17:04 smithi117 ceph-mon[36730]: scrub summary: idle 2024-04-16T17:17:06.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:17:06 smithi104 ceph-mon[36457]: pgmap v322: 129 pgs: 129 active+clean; 56 MiB data, 870 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 45 KiB/s wr, 31 op/s 2024-04-16T17:17:06.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:17:06 smithi045 ceph-mon[31676]: pgmap v322: 129 pgs: 129 active+clean; 56 MiB data, 870 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 45 KiB/s wr, 31 op/s 2024-04-16T17:17:07.054 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:17:06 smithi117 ceph-mon[36730]: pgmap v322: 129 pgs: 129 active+clean; 56 MiB data, 870 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 45 KiB/s wr, 31 op/s 2024-04-16T17:17:08.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:17:08 smithi104 ceph-mon[36457]: pgmap v323: 129 pgs: 129 active+clean; 56 MiB data, 870 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 45 KiB/s wr, 34 op/s 2024-04-16T17:17:08.976 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:17:08 smithi045 ceph-mon[31676]: pgmap v323: 129 pgs: 129 active+clean; 56 MiB data, 870 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 45 KiB/s wr, 34 op/s 2024-04-16T17:17:09.054 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:17:08 smithi117 ceph-mon[36730]: pgmap v323: 129 pgs: 129 active+clean; 56 MiB data, 870 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 45 KiB/s wr, 34 op/s 2024-04-16T17:17:09.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:17:09 smithi104 ceph-mon[36457]: osdmap e105: 12 total, 12 up, 12 in 2024-04-16T17:17:09.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:17:09 smithi104 ceph-mon[36457]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T17:17:09.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:17:09 smithi045 ceph-mon[31676]: osdmap e105: 12 total, 12 up, 12 in 2024-04-16T17:17:09.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:17:09 smithi045 ceph-mon[31676]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T17:17:10.054 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:17:09 smithi117 ceph-mon[36730]: osdmap e105: 12 total, 12 up, 12 in 2024-04-16T17:17:10.054 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:17:09 smithi117 ceph-mon[36730]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T17:17:10.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:17:10 smithi104 ceph-mon[36457]: pgmap v325: 129 pgs: 129 active+clean; 56 MiB data, 870 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 29 op/s 2024-04-16T17:17:10.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:17:10 smithi104 ceph-mon[36457]: from='mgr.25411 ' entity='mgr.x' 2024-04-16T17:17:10.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:17:10 smithi104 ceph-mon[36457]: from='mgr.25411 ' entity='mgr.x' 2024-04-16T17:17:10.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:17:10 smithi104 ceph-mon[36457]: from='mgr.25411 ' entity='mgr.x' 2024-04-16T17:17:10.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:17:10 smithi104 ceph-mon[36457]: from='mgr.25411 ' entity='mgr.x' 2024-04-16T17:17:10.976 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:17:10 smithi045 ceph-mon[31676]: pgmap v325: 129 pgs: 129 active+clean; 56 MiB data, 870 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 29 op/s 2024-04-16T17:17:10.976 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:17:10 smithi045 ceph-mon[31676]: from='mgr.25411 ' entity='mgr.x' 2024-04-16T17:17:10.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:17:10 smithi045 ceph-mon[31676]: from='mgr.25411 ' entity='mgr.x' 2024-04-16T17:17:10.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:17:10 smithi045 ceph-mon[31676]: from='mgr.25411 ' entity='mgr.x' 2024-04-16T17:17:10.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:17:10 smithi045 ceph-mon[31676]: from='mgr.25411 ' entity='mgr.x' 2024-04-16T17:17:11.054 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:17:10 smithi117 ceph-mon[36730]: pgmap v325: 129 pgs: 129 active+clean; 56 MiB data, 870 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 29 op/s 2024-04-16T17:17:11.054 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:17:10 smithi117 ceph-mon[36730]: from='mgr.25411 ' entity='mgr.x' 2024-04-16T17:17:11.055 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:17:10 smithi117 ceph-mon[36730]: from='mgr.25411 ' entity='mgr.x' 2024-04-16T17:17:11.055 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:17:10 smithi117 ceph-mon[36730]: from='mgr.25411 ' entity='mgr.x' 2024-04-16T17:17:11.055 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:17:10 smithi117 ceph-mon[36730]: from='mgr.25411 ' entity='mgr.x' 2024-04-16T17:17:11.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:17:11 smithi045 ceph-mon[31676]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:17:11.978 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:17:11 smithi045 ceph-mon[31676]: from='mgr.25411 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:17:11.978 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:17:11 smithi045 ceph-mon[31676]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:17:11.978 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:17:11 smithi045 ceph-mon[31676]: from='mgr.25411 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:17:11.978 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:17:11 smithi045 ceph-mon[31676]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:17:11.978 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:17:11 smithi045 ceph-mon[31676]: from='mgr.25411 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:17:11.978 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:17:11 smithi045 ceph-mon[31676]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:17:11.978 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:17:11 smithi045 ceph-mon[31676]: from='mgr.25411 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:17:11.979 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:17:11 smithi045 ceph-mon[31676]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:17:11.979 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:17:11 smithi045 ceph-mon[31676]: from='mgr.25411 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:17:11.979 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:17:11 smithi045 ceph-mon[31676]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:17:11.979 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:17:11 smithi045 ceph-mon[31676]: from='mgr.25411 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:17:11.979 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:17:11 smithi045 ceph-mon[31676]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:17:11.979 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:17:11 smithi045 ceph-mon[31676]: from='mgr.25411 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:17:11.979 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:17:11 smithi045 ceph-mon[31676]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:17:11.979 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:17:11 smithi045 ceph-mon[31676]: from='mgr.25411 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:17:12.054 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:17:11 smithi117 ceph-mon[36730]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:17:12.055 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:17:11 smithi117 ceph-mon[36730]: from='mgr.25411 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:17:12.055 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:17:11 smithi117 ceph-mon[36730]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:17:12.055 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:17:11 smithi117 ceph-mon[36730]: from='mgr.25411 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:17:12.055 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:17:11 smithi117 ceph-mon[36730]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:17:12.055 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:17:11 smithi117 ceph-mon[36730]: from='mgr.25411 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:17:12.055 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:17:11 smithi117 ceph-mon[36730]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:17:12.055 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:17:11 smithi117 ceph-mon[36730]: from='mgr.25411 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:17:12.055 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:17:11 smithi117 ceph-mon[36730]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:17:12.055 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:17:11 smithi117 ceph-mon[36730]: from='mgr.25411 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:17:12.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:17:11 smithi117 ceph-mon[36730]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:17:12.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:17:11 smithi117 ceph-mon[36730]: from='mgr.25411 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:17:12.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:17:11 smithi117 ceph-mon[36730]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:17:12.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:17:11 smithi117 ceph-mon[36730]: from='mgr.25411 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:17:12.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:17:11 smithi117 ceph-mon[36730]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:17:12.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:17:11 smithi117 ceph-mon[36730]: from='mgr.25411 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:17:12.177 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:17:11 smithi104 ceph-mon[36457]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:17:12.177 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:17:11 smithi104 ceph-mon[36457]: from='mgr.25411 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:17:12.177 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:17:11 smithi104 ceph-mon[36457]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:17:12.177 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:17:11 smithi104 ceph-mon[36457]: from='mgr.25411 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:17:12.177 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:17:11 smithi104 ceph-mon[36457]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:17:12.177 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:17:11 smithi104 ceph-mon[36457]: from='mgr.25411 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:17:12.177 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:17:11 smithi104 ceph-mon[36457]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:17:12.178 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:17:11 smithi104 ceph-mon[36457]: from='mgr.25411 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:17:12.178 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:17:11 smithi104 ceph-mon[36457]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:17:12.178 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:17:11 smithi104 ceph-mon[36457]: from='mgr.25411 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:17:12.178 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:17:11 smithi104 ceph-mon[36457]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:17:12.178 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:17:11 smithi104 ceph-mon[36457]: from='mgr.25411 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:17:12.178 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:17:11 smithi104 ceph-mon[36457]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:17:12.178 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:17:11 smithi104 ceph-mon[36457]: from='mgr.25411 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:17:12.178 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:17:11 smithi104 ceph-mon[36457]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:17:12.178 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:17:11 smithi104 ceph-mon[36457]: from='mgr.25411 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:17:12.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:17:12 smithi045 ceph-mon[31676]: Adjusting osd_memory_target on smithi104 to 174.1M 2024-04-16T17:17:12.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:17:12 smithi045 ceph-mon[31676]: Unable to set osd_memory_target on smithi104 to 182633472: error parsing value: Value '182633472' is below minimum 939524096 2024-04-16T17:17:12.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:17:12 smithi045 ceph-mon[31676]: Adjusting osd_memory_target on smithi117 to 174.1M 2024-04-16T17:17:12.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:17:12 smithi045 ceph-mon[31676]: Unable to set osd_memory_target on smithi117 to 182632038: error parsing value: Value '182632038' is below minimum 939524096 2024-04-16T17:17:12.978 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:17:12 smithi045 ceph-mon[31676]: pgmap v326: 129 pgs: 129 active+clean; 55 MiB data, 856 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-16T17:17:12.978 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:17:12 smithi045 ceph-mon[31676]: from='mgr.25411 ' entity='mgr.x' 2024-04-16T17:17:12.978 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:17:12 smithi045 ceph-mon[31676]: from='mgr.25411 ' entity='mgr.x' 2024-04-16T17:17:13.054 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:17:12 smithi117 ceph-mon[36730]: Adjusting osd_memory_target on smithi104 to 174.1M 2024-04-16T17:17:13.055 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:17:12 smithi117 ceph-mon[36730]: Unable to set osd_memory_target on smithi104 to 182633472: error parsing value: Value '182633472' is below minimum 939524096 2024-04-16T17:17:13.055 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:17:12 smithi117 ceph-mon[36730]: Adjusting osd_memory_target on smithi117 to 174.1M 2024-04-16T17:17:13.055 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:17:12 smithi117 ceph-mon[36730]: Unable to set osd_memory_target on smithi117 to 182632038: error parsing value: Value '182632038' is below minimum 939524096 2024-04-16T17:17:13.055 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:17:12 smithi117 ceph-mon[36730]: pgmap v326: 129 pgs: 129 active+clean; 55 MiB data, 856 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-16T17:17:13.055 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:17:12 smithi117 ceph-mon[36730]: from='mgr.25411 ' entity='mgr.x' 2024-04-16T17:17:13.055 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:17:12 smithi117 ceph-mon[36730]: from='mgr.25411 ' entity='mgr.x' 2024-04-16T17:17:13.177 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:17:12 smithi104 ceph-mon[36457]: Adjusting osd_memory_target on smithi104 to 174.1M 2024-04-16T17:17:13.177 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:17:12 smithi104 ceph-mon[36457]: Unable to set osd_memory_target on smithi104 to 182633472: error parsing value: Value '182633472' is below minimum 939524096 2024-04-16T17:17:13.177 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:17:12 smithi104 ceph-mon[36457]: Adjusting osd_memory_target on smithi117 to 174.1M 2024-04-16T17:17:13.177 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:17:12 smithi104 ceph-mon[36457]: Unable to set osd_memory_target on smithi117 to 182632038: error parsing value: Value '182632038' is below minimum 939524096 2024-04-16T17:17:13.177 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:17:12 smithi104 ceph-mon[36457]: pgmap v326: 129 pgs: 129 active+clean; 55 MiB data, 856 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-16T17:17:13.177 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:17:12 smithi104 ceph-mon[36457]: from='mgr.25411 ' entity='mgr.x' 2024-04-16T17:17:13.177 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:17:12 smithi104 ceph-mon[36457]: from='mgr.25411 ' entity='mgr.x' 2024-04-16T17:17:13.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:17:13 smithi045 ceph-mon[31676]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:17:13.978 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:17:13 smithi045 ceph-mon[31676]: from='mgr.25411 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:17:13.978 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:17:13 smithi045 ceph-mon[31676]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:17:13.978 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:17:13 smithi045 ceph-mon[31676]: from='mgr.25411 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:17:13.978 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:17:13 smithi045 ceph-mon[31676]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:17:13.978 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:17:13 smithi045 ceph-mon[31676]: from='mgr.25411 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:17:13.978 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:17:13 smithi045 ceph-mon[31676]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:17:13.978 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:17:13 smithi045 ceph-mon[31676]: from='mgr.25411 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:17:13.979 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:17:13 smithi045 ceph-mon[31676]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:17:13.979 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:17:13 smithi045 ceph-mon[31676]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T17:17:13.979 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:17:13 smithi045 ceph-mon[31676]: from='mgr.25411 ' entity='mgr.x' 2024-04-16T17:17:14.055 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:17:13 smithi117 ceph-mon[36730]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:17:14.055 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:17:13 smithi117 ceph-mon[36730]: from='mgr.25411 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:17:14.055 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:17:13 smithi117 ceph-mon[36730]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:17:14.055 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:17:13 smithi117 ceph-mon[36730]: from='mgr.25411 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:17:14.055 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:17:13 smithi117 ceph-mon[36730]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:17:14.055 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:17:13 smithi117 ceph-mon[36730]: from='mgr.25411 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:17:14.055 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:17:13 smithi117 ceph-mon[36730]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:17:14.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:17:13 smithi117 ceph-mon[36730]: from='mgr.25411 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:17:14.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:17:13 smithi117 ceph-mon[36730]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:17:14.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:17:13 smithi117 ceph-mon[36730]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T17:17:14.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:17:13 smithi117 ceph-mon[36730]: from='mgr.25411 ' entity='mgr.x' 2024-04-16T17:17:14.078 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:17:13 smithi104 ceph-mon[36457]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:17:14.078 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:17:13 smithi104 ceph-mon[36457]: from='mgr.25411 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:17:14.079 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:17:13 smithi104 ceph-mon[36457]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:17:14.079 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:17:13 smithi104 ceph-mon[36457]: from='mgr.25411 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:17:14.079 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:17:13 smithi104 ceph-mon[36457]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:17:14.079 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:17:13 smithi104 ceph-mon[36457]: from='mgr.25411 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:17:14.079 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:17:13 smithi104 ceph-mon[36457]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:17:14.079 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:17:13 smithi104 ceph-mon[36457]: from='mgr.25411 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-16T17:17:14.079 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:17:13 smithi104 ceph-mon[36457]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:17:14.079 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:17:13 smithi104 ceph-mon[36457]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T17:17:14.080 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:17:13 smithi104 ceph-mon[36457]: from='mgr.25411 ' entity='mgr.x' 2024-04-16T17:17:14.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:17:14 smithi045 ceph-mon[31676]: Adjusting osd_memory_target on smithi045 to 177.5M 2024-04-16T17:17:14.978 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:17:14 smithi045 ceph-mon[31676]: Unable to set osd_memory_target on smithi045 to 186173030: error parsing value: Value '186173030' is below minimum 939524096 2024-04-16T17:17:14.978 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:17:14 smithi045 ceph-mon[31676]: pgmap v327: 129 pgs: 129 active+clean; 55 MiB data, 856 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-16T17:17:14.978 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:17:14 smithi045 ceph-mon[31676]: osdmap e106: 12 total, 12 up, 12 in 2024-04-16T17:17:15.054 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:17:14 smithi117 ceph-mon[36730]: Adjusting osd_memory_target on smithi045 to 177.5M 2024-04-16T17:17:15.054 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:17:14 smithi117 ceph-mon[36730]: Unable to set osd_memory_target on smithi045 to 186173030: error parsing value: Value '186173030' is below minimum 939524096 2024-04-16T17:17:15.055 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:17:14 smithi117 ceph-mon[36730]: pgmap v327: 129 pgs: 129 active+clean; 55 MiB data, 856 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-16T17:17:15.055 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:17:14 smithi117 ceph-mon[36730]: osdmap e106: 12 total, 12 up, 12 in 2024-04-16T17:17:15.093 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:17:14 smithi104 ceph-mon[36457]: Adjusting osd_memory_target on smithi045 to 177.5M 2024-04-16T17:17:15.093 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:17:14 smithi104 ceph-mon[36457]: Unable to set osd_memory_target on smithi045 to 186173030: error parsing value: Value '186173030' is below minimum 939524096 2024-04-16T17:17:15.093 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:17:14 smithi104 ceph-mon[36457]: pgmap v327: 129 pgs: 129 active+clean; 55 MiB data, 856 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-16T17:17:15.093 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:17:14 smithi104 ceph-mon[36457]: osdmap e106: 12 total, 12 up, 12 in 2024-04-16T17:17:16.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:17:16 smithi045 ceph-mon[31676]: pgmap v329: 129 pgs: 129 active+clean; 55 MiB data, 856 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 127 B/s wr, 3 op/s 2024-04-16T17:17:17.054 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:17:16 smithi117 ceph-mon[36730]: pgmap v329: 129 pgs: 129 active+clean; 55 MiB data, 856 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 127 B/s wr, 3 op/s 2024-04-16T17:17:17.177 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:17:16 smithi104 ceph-mon[36457]: pgmap v329: 129 pgs: 129 active+clean; 55 MiB data, 856 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 127 B/s wr, 3 op/s 2024-04-16T17:17:18.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:17:18 smithi045 ceph-mon[31676]: pgmap v330: 129 pgs: 129 active+clean; 55 MiB data, 874 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 233 B/s wr, 7 op/s 2024-04-16T17:17:19.054 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:17:18 smithi117 ceph-mon[36730]: pgmap v330: 129 pgs: 129 active+clean; 55 MiB data, 874 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 233 B/s wr, 7 op/s 2024-04-16T17:17:19.177 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:17:18 smithi104 ceph-mon[36457]: pgmap v330: 129 pgs: 129 active+clean; 55 MiB data, 874 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 233 B/s wr, 7 op/s 2024-04-16T17:17:20.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:17:20 smithi045 ceph-mon[31676]: pgmap v331: 129 pgs: 129 active+clean; 55 MiB data, 874 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-16T17:17:21.054 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:17:20 smithi117 ceph-mon[36730]: pgmap v331: 129 pgs: 129 active+clean; 55 MiB data, 874 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-16T17:17:21.177 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:17:20 smithi104 ceph-mon[36457]: pgmap v331: 129 pgs: 129 active+clean; 55 MiB data, 874 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-16T17:17:22.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:17:22 smithi045 ceph-mon[31676]: pgmap v332: 129 pgs: 129 active+clean; 55 MiB data, 874 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-16T17:17:23.054 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:17:22 smithi117 ceph-mon[36730]: pgmap v332: 129 pgs: 129 active+clean; 55 MiB data, 874 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-16T17:17:23.177 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:17:22 smithi104 ceph-mon[36457]: pgmap v332: 129 pgs: 129 active+clean; 55 MiB data, 874 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-16T17:17:25.055 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:17:24 smithi117 ceph-mon[36730]: pgmap v333: 129 pgs: 129 active+clean; 55 MiB data, 874 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-16T17:17:25.177 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:17:24 smithi104 ceph-mon[36457]: pgmap v333: 129 pgs: 129 active+clean; 55 MiB data, 874 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-16T17:17:25.226 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:17:24 smithi045 ceph-mon[31676]: pgmap v333: 129 pgs: 129 active+clean; 55 MiB data, 874 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-16T17:17:26.102 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-16T17:17:26.102 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-16T17:17:26.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.345+0000 7fea4fe00640 1 -- 172.21.15.45:0/2367668522 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fea50074430 msgr2=0x7fea50074810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:17:26.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.345+0000 7fea4fe00640 1 --2- 172.21.15.45:0/2367668522 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fea50074430 0x7fea50074810 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fea38009860 tx=0x7fea3802bc00 comp rx=0 tx=0).stop 2024-04-16T17:17:26.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.345+0000 7fea4fe00640 1 -- 172.21.15.45:0/2367668522 shutdown_connections 2024-04-16T17:17:26.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.345+0000 7fea4fe00640 1 --2- 172.21.15.45:0/2367668522 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fea5010e6e0 0x7fea50110b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:26.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.345+0000 7fea4fe00640 1 --2- 172.21.15.45:0/2367668522 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fea50074d50 0x7fea5010e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:26.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.345+0000 7fea4fe00640 1 --2- 172.21.15.45:0/2367668522 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fea50074430 0x7fea50074810 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:26.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.345+0000 7fea4fe00640 1 -- 172.21.15.45:0/2367668522 >> 172.21.15.45:0/2367668522 conn(0x7fea500fd9d0 msgr2=0x7fea500ffdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:17:26.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.345+0000 7fea4fe00640 1 -- 172.21.15.45:0/2367668522 shutdown_connections 2024-04-16T17:17:26.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.345+0000 7fea4fe00640 1 -- 172.21.15.45:0/2367668522 wait complete. 2024-04-16T17:17:26.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.346+0000 7fea4fe00640 1 Processor -- start 2024-04-16T17:17:26.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.346+0000 7fea4fe00640 1 -- start start 2024-04-16T17:17:26.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.347+0000 7fea4fe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fea50074430 0x7fea501a0620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:26.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.347+0000 7fea4fe00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fea50074d50 0x7fea501a0b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:26.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.347+0000 7fea4fe00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fea5010e6e0 0x7fea5019a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:26.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.347+0000 7fea4fe00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fea50114470 con 0x7fea50074430 2024-04-16T17:17:26.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.347+0000 7fea4fe00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fea501142f0 con 0x7fea50074d50 2024-04-16T17:17:26.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.347+0000 7fea4fe00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fea501145f0 con 0x7fea5010e6e0 2024-04-16T17:17:26.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.347+0000 7fea4ea00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fea50074430 0x7fea501a0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:26.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.347+0000 7fea4ea00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fea50074430 0x7fea501a0620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45058/0 (socket says 172.21.15.45:45058) 2024-04-16T17:17:26.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.347+0000 7fea4ea00640 1 -- 172.21.15.45:0/702577115 learned_addr learned my addr 172.21.15.45:0/702577115 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:17:26.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.347+0000 7fea4f400640 1 --2- 172.21.15.45:0/702577115 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fea5010e6e0 0x7fea5019a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:26.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.347+0000 7fea4e000640 1 --2- 172.21.15.45:0/702577115 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fea50074d50 0x7fea501a0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:26.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.348+0000 7fea4f400640 1 -- 172.21.15.45:0/702577115 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fea50074d50 msgr2=0x7fea501a0b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:17:26.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.348+0000 7fea4f400640 1 --2- 172.21.15.45:0/702577115 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fea50074d50 0x7fea501a0b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:26.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.348+0000 7fea4f400640 1 -- 172.21.15.45:0/702577115 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fea50074430 msgr2=0x7fea501a0620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:17:26.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.348+0000 7fea4f400640 1 --2- 172.21.15.45:0/702577115 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fea50074430 0x7fea501a0620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:26.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.348+0000 7fea4f400640 1 -- 172.21.15.45:0/702577115 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea5019b000 con 0x7fea5010e6e0 2024-04-16T17:17:26.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.348+0000 7fea4f400640 1 --2- 172.21.15.45:0/702577115 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fea5010e6e0 0x7fea5019a830 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fea4800e970 tx=0x7fea4800ee30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:17:26.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.348+0000 7fea47400640 1 -- 172.21.15.45:0/702577115 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea4800ccc0 con 0x7fea5010e6e0 2024-04-16T17:17:26.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.348+0000 7fea47400640 1 -- 172.21.15.45:0/702577115 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea48005020 con 0x7fea5010e6e0 2024-04-16T17:17:26.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.348+0000 7fea4fe00640 1 -- 172.21.15.45:0/702577115 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea5019b2f0 con 0x7fea5010e6e0 2024-04-16T17:17:26.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.348+0000 7fea47400640 1 -- 172.21.15.45:0/702577115 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea48004a00 con 0x7fea5010e6e0 2024-04-16T17:17:26.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.349+0000 7fea4fe00640 1 -- 172.21.15.45:0/702577115 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea501a7470 con 0x7fea5010e6e0 2024-04-16T17:17:26.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.352+0000 7fea47400640 1 -- 172.21.15.45:0/702577115 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fea480051d0 con 0x7fea5010e6e0 2024-04-16T17:17:26.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.352+0000 7fea58574740 1 Processor -- start 2024-04-16T17:17:26.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.352+0000 7fea58574740 1 -- start start 2024-04-16T17:17:26.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.352+0000 7fea58574740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561d97fd5390 0x561d97fd5770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:26.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.352+0000 7fea58574740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561d97fd5cb0 0x561d97fde350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:26.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.352+0000 7fea58574740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561d97fde890 0x561d97fe0c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:26.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.352+0000 7fea58574740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x561d97f3b120 con 0x561d97fde890 2024-04-16T17:17:26.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.352+0000 7fea58574740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x561d97ee1340 con 0x561d97fd5390 2024-04-16T17:17:26.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.352+0000 7fea58574740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x561d97f370e0 con 0x561d97fd5cb0 2024-04-16T17:17:26.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.353+0000 7fea4ea00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561d97fd5cb0 0x561d97fde350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:26.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.353+0000 7fea4e000640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561d97fd5390 0x561d97fd5770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:26.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.353+0000 7fea4ea00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561d97fd5cb0 0x561d97fde350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:35756/0 (socket says 172.21.15.45:35756) 2024-04-16T17:17:26.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.353+0000 7fea4ea00640 1 -- 172.21.15.45:0/953955681 learned_addr learned my addr 172.21.15.45:0/953955681 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:17:26.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.353+0000 7fea4e000640 1 --2- 172.21.15.45:0/953955681 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561d97fde890 0x561d97fe0c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:26.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.353+0000 7fea4ea00640 1 -- 172.21.15.45:0/953955681 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561d97fd5390 msgr2=0x561d97fd5770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:17:26.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.353+0000 7fea4ea00640 1 --2- 172.21.15.45:0/953955681 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561d97fd5390 0x561d97fd5770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:26.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.353+0000 7fea4ea00640 1 -- 172.21.15.45:0/953955681 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561d97fde890 msgr2=0x561d97fe0c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:17:26.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.353+0000 7fea4ea00640 1 --2- 172.21.15.45:0/953955681 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561d97fde890 0x561d97fe0c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:26.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.353+0000 7fea4ea00640 1 -- 172.21.15.45:0/953955681 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561d97d177d0 con 0x561d97fd5cb0 2024-04-16T17:17:26.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.353+0000 7fea4e000640 1 --2- 172.21.15.45:0/953955681 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561d97fde890 0x561d97fe0c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:17:26.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.354+0000 7fea4ea00640 1 --2- 172.21.15.45:0/953955681 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561d97fd5cb0 0x561d97fde350 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fea38037040 tx=0x7fea38009520 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bdc823c3c62a7284 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:17:26.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.354+0000 7fea47400640 1 --2- 172.21.15.45:0/702577115 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fea340a3d20 0x7fea340a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:26.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.354+0000 7fea3fe00640 1 -- 172.21.15.45:0/953955681 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea38043020 con 0x561d97fd5cb0 2024-04-16T17:17:26.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.354+0000 7fea3fe00640 1 -- 172.21.15.45:0/953955681 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea38034440 con 0x561d97fd5cb0 2024-04-16T17:17:26.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.354+0000 7fea47400640 1 -- 172.21.15.45:0/702577115 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 9496+0+0 (secure 0 0 0) 0x7fea480d0b00 con 0x7fea5010e6e0 2024-04-16T17:17:26.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.354+0000 7fea3fe00640 1 -- 172.21.15.45:0/953955681 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea38039040 con 0x561d97fd5cb0 2024-04-16T17:17:26.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.354+0000 7fea4e000640 1 --2- 172.21.15.45:0/702577115 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fea340a3d20 0x7fea340a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:26.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.355+0000 7fea58574740 1 -- 172.21.15.45:0/953955681 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561d97fd5cb0 msgr2=0x561d97fde350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:17:26.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.355+0000 7fea4e000640 1 --2- 172.21.15.45:0/702577115 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fea340a3d20 0x7fea340a61e0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x561d97fd0c50 tx=0x7fea400095c0 comp rx=0 tx=0).ready entity=mgr.25411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:17:26.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.355+0000 7fea58574740 1 --2- 172.21.15.45:0/953955681 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561d97fd5cb0 0x561d97fde350 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fea38037040 tx=0x7fea38009520 comp rx=0 tx=0).stop 2024-04-16T17:17:26.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.355+0000 7fea58574740 1 -- 172.21.15.45:0/953955681 shutdown_connections 2024-04-16T17:17:26.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.355+0000 7fea58574740 1 --2- 172.21.15.45:0/953955681 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561d97fd5cb0 0x561d97fde350 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:26.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.355+0000 7fea58574740 1 --2- 172.21.15.45:0/953955681 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561d97fde890 0x561d97fe0c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:26.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.355+0000 7fea58574740 1 --2- 172.21.15.45:0/953955681 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561d97fd5390 0x561d97fd5770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:26.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.355+0000 7fea58574740 1 -- 172.21.15.45:0/953955681 >> 172.21.15.45:0/953955681 conn(0x561d97fcfac0 msgr2=0x561d97fcfea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:17:26.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.355+0000 7fea58574740 1 -- 172.21.15.45:0/953955681 shutdown_connections 2024-04-16T17:17:26.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.355+0000 7fea58574740 1 -- 172.21.15.45:0/953955681 wait complete. 2024-04-16T17:17:26.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.355+0000 7fea58574740 1 -- start start 2024-04-16T17:17:26.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.356+0000 7fea58574740 1 Processor -- start 2024-04-16T17:17:26.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.356+0000 7fea58574740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561d97fd5390 0x561d97fdb8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:26.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.356+0000 7fea58574740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561d97fd5cb0 0x561d97fdbe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:26.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.356+0000 7fea58574740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561d97fde890 0x561d97fdc370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:26.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.356+0000 7fea58574740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x561d97fe1620 con 0x561d97fde890 2024-04-16T17:17:26.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.356+0000 7fea58574740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x561d97ef4710 con 0x561d97fd5cb0 2024-04-16T17:17:26.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.356+0000 7fea58574740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x561d97fe17a0 con 0x561d97fd5390 2024-04-16T17:17:26.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.356+0000 7fea58574740 10 client.0 _ll_register_callbacks cb 0x561d97eefaf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:17:26.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.357+0000 7fea4f400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561d97fde890 0x561d97fdc370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:26.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.357+0000 7fea4e000640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561d97fd5cb0 0x561d97fdbe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:26.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.357+0000 7fea4e000640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561d97fd5cb0 0x561d97fdbe30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:43624/0 (socket says 172.21.15.45:43624) 2024-04-16T17:17:26.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.357+0000 7fea4e000640 1 -- 172.21.15.45:0/2556743989 learned_addr learned my addr 172.21.15.45:0/2556743989 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:17:26.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.357+0000 7fea4ea00640 1 --2- 172.21.15.45:0/2556743989 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561d97fd5390 0x561d97fdb8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:26.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.357+0000 7fea4f400640 1 -- 172.21.15.45:0/2556743989 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561d97fd5390 msgr2=0x561d97fdb8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:17:26.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.357+0000 7fea4f400640 1 --2- 172.21.15.45:0/2556743989 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561d97fd5390 0x561d97fdb8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:26.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.357+0000 7fea4f400640 1 -- 172.21.15.45:0/2556743989 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561d97fd5cb0 msgr2=0x561d97fdbe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:17:26.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.357+0000 7fea4f400640 1 --2- 172.21.15.45:0/2556743989 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561d97fd5cb0 0x561d97fdbe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:26.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.357+0000 7fea4f400640 1 -- 172.21.15.45:0/2556743989 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561d97c4d3f0 con 0x561d97fde890 2024-04-16T17:17:26.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.357+0000 7fea4e000640 1 --2- 172.21.15.45:0/2556743989 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561d97fd5cb0 0x561d97fdbe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:17:26.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.357+0000 7fea4ea00640 1 --2- 172.21.15.45:0/2556743989 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561d97fd5390 0x561d97fdb8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:17:26.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.357+0000 7fea4f400640 1 --2- 172.21.15.45:0/2556743989 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561d97fde890 0x561d97fdc370 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fea480cde20 tx=0x7fea48005620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a26273dad1c1006a server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:17:26.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.357+0000 7fea3cc00640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-16T17:17:26.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.358+0000 7fea3cc00640 1 -- 172.21.15.45:0/2556743989 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea4801c070 con 0x561d97fde890 2024-04-16T17:17:26.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.358+0000 7fea3cc00640 1 -- 172.21.15.45:0/2556743989 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9496+0+0 (secure 0 0 0) 0x7fea480a4e50 con 0x561d97fde890 2024-04-16T17:17:26.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.358+0000 7fea58574740 1 -- 172.21.15.45:0/2556743989 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fea38034440 con 0x561d97fde890 2024-04-16T17:17:26.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.358+0000 7fea3cc00640 1 -- 172.21.15.45:0/2556743989 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fea4809c670 con 0x561d97fde890 2024-04-16T17:17:26.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.358+0000 7fea58574740 10 client.18126 fetch_fsmap learned FSMap version 44 2024-04-16T17:17:26.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.358+0000 7fea58574740 1 -- 172.21.15.45:0/2556743989 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x561d97c0d0f0 con 0x561d97fde890 2024-04-16T17:17:26.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.358+0000 7fea3cc00640 1 -- 172.21.15.45:0/2556743989 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea38034440 con 0x561d97fde890 2024-04-16T17:17:26.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.358+0000 7fea3cc00640 1 -- 172.21.15.45:0/2556743989 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 44) ==== 8346+0+0 (secure 0 0 0) 0x7fea48004ba0 con 0x561d97fde890 2024-04-16T17:17:26.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.358+0000 7fea58574740 10 client.18126 fetch_fsmap finished waiting for FSMap version 44 2024-04-16T17:17:26.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.358+0000 7fea58574740 10 client.18126 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:17:26.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.359+0000 7fea58574740 20 client.18126 populate_metadata read hostname 'smithi045' 2024-04-16T17:17:26.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.359+0000 7fea58574740 1 --2- 172.21.15.45:0/2556743989 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x561d97feebe0 0x561d97ff1050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:26.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.359+0000 7fea58574740 4 client.18126 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:17:26.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.359+0000 7fea58574740 1 -- 172.21.15.45:0/2556743989 --> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561d97f48b40 con 0x561d97feebe0 2024-04-16T17:17:26.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.359+0000 7fea4ea00640 1 --2- 172.21.15.45:0/2556743989 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x561d97feebe0 0x561d97ff1050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:26.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.360+0000 7fea4ea00640 1 --2- 172.21.15.45:0/2556743989 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x561d97feebe0 0x561d97ff1050 crc :-1 s=READY pgs=257 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=531235d454d0a8bc server_cookie=fa119f124d82457e in_seq=0 out_seq=0 2024-04-16T17:17:26.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.360+0000 7fea3cc00640 10 client.18126 ms_handle_connect on v2:172.21.15.104:6838/3660070592 2024-04-16T17:17:26.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.360+0000 7fea3cc00640 1 -- 172.21.15.45:0/2556743989 <== mds.0 v2:172.21.15.104:6838/3660070592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561d97f48b40 con 0x561d97feebe0 2024-04-16T17:17:26.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.360+0000 7fea3cc00640 10 client.18126 handle_command_reply: tid=0 2024-04-16T17:17:26.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.361+0000 7fea58574740 1 client.18126 shutdown 2024-04-16T17:17:26.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.361+0000 7fea58574740 1 -- 172.21.15.45:0/2556743989 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561d97fde890 msgr2=0x561d97fdc370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:17:26.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.361+0000 7fea58574740 1 --2- 172.21.15.45:0/2556743989 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561d97fde890 0x561d97fdc370 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fea480cde20 tx=0x7fea48005620 comp rx=0 tx=0).stop 2024-04-16T17:17:26.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.361+0000 7fea58574740 1 -- 172.21.15.45:0/2556743989 shutdown_connections 2024-04-16T17:17:26.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.361+0000 7fea58574740 1 --2- 172.21.15.45:0/2556743989 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561d97fde890 0x561d97fdc370 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:26.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.361+0000 7fea58574740 1 --2- 172.21.15.45:0/2556743989 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561d97fd5cb0 0x561d97fdbe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:26.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.361+0000 7fea58574740 1 --2- 172.21.15.45:0/2556743989 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x561d97feebe0 0x561d97ff1050 crc :-1 s=READY pgs=257 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:26.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.361+0000 7fea58574740 1 --2- 172.21.15.45:0/2556743989 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561d97fd5390 0x561d97fdb8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:26.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.361+0000 7fea58574740 1 -- 172.21.15.45:0/2556743989 >> 172.21.15.45:0/2556743989 conn(0x561d97fcfac0 msgr2=0x561d97fd1810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:17:26.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.361+0000 7fea3cc00640 0 client.18126 ms_handle_reset on v2:172.21.15.104:6838/3660070592 2024-04-16T17:17:26.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.361+0000 7fea58574740 1 -- 172.21.15.45:0/2556743989 shutdown_connections 2024-04-16T17:17:26.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.361+0000 7fea58574740 1 -- 172.21.15.45:0/2556743989 wait complete. 2024-04-16T17:17:26.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.361+0000 7fea58574740 20 client.18126 trim_cache size 0 max 16384 2024-04-16T17:17:26.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.381+0000 7fea58574740 1 Processor -- start 2024-04-16T17:17:26.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.381+0000 7fea58574740 1 -- start start 2024-04-16T17:17:26.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.381+0000 7fea58574740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561d97fd5390 0x561d97fde020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:26.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.381+0000 7fea58574740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561d97fd5cb0 0x561d97fea4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:26.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.381+0000 7fea58574740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561d97fde890 0x561d97fdaa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:26.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.381+0000 7fea58574740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fea501b3e90 con 0x561d97fd5cb0 2024-04-16T17:17:26.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.381+0000 7fea58574740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fea501b1760 con 0x561d97fd5390 2024-04-16T17:17:26.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.381+0000 7fea58574740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fea501af5d0 con 0x561d97fde890 2024-04-16T17:17:26.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.381+0000 7fea4e000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561d97fd5cb0 0x561d97fea4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:26.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.381+0000 7fea4ea00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561d97fd5390 0x561d97fde020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:26.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.381+0000 7fea4e000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561d97fd5cb0 0x561d97fea4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45080/0 (socket says 172.21.15.45:45080) 2024-04-16T17:17:26.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.382+0000 7fea4e000640 1 -- 172.21.15.45:0/573165428 learned_addr learned my addr 172.21.15.45:0/573165428 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:17:26.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.382+0000 7fea4f400640 1 --2- 172.21.15.45:0/573165428 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561d97fde890 0x561d97fdaa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:26.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.382+0000 7fea4ea00640 1 -- 172.21.15.45:0/573165428 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561d97fde890 msgr2=0x561d97fdaa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:17:26.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.382+0000 7fea4ea00640 1 --2- 172.21.15.45:0/573165428 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561d97fde890 0x561d97fdaa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:26.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.382+0000 7fea4ea00640 1 -- 172.21.15.45:0/573165428 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561d97fd5cb0 msgr2=0x561d97fea4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:17:26.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.382+0000 7fea4ea00640 1 --2- 172.21.15.45:0/573165428 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561d97fd5cb0 0x561d97fea4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:26.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.382+0000 7fea4ea00640 1 -- 172.21.15.45:0/573165428 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561d97fe0c80 con 0x561d97fd5390 2024-04-16T17:17:26.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.382+0000 7fea4ea00640 1 --2- 172.21.15.45:0/573165428 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561d97fd5390 0x561d97fde020 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fea38009830 tx=0x7fea3802bc00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f34a272ac57e1f1f server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:17:26.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.382+0000 7fea3f400640 1 -- 172.21.15.45:0/573165428 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea38043040 con 0x561d97fd5390 2024-04-16T17:17:26.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.383+0000 7fea3f400640 1 -- 172.21.15.45:0/573165428 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea38034440 con 0x561d97fd5390 2024-04-16T17:17:26.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.383+0000 7fea3f400640 1 -- 172.21.15.45:0/573165428 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea38039040 con 0x561d97fd5390 2024-04-16T17:17:26.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.383+0000 7fea58574740 1 -- 172.21.15.45:0/573165428 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561d97fd5390 msgr2=0x561d97fde020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:17:26.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.383+0000 7fea58574740 1 --2- 172.21.15.45:0/573165428 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561d97fd5390 0x561d97fde020 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fea38009830 tx=0x7fea3802bc00 comp rx=0 tx=0).stop 2024-04-16T17:17:26.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.383+0000 7fea58574740 1 -- 172.21.15.45:0/573165428 shutdown_connections 2024-04-16T17:17:26.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.383+0000 7fea58574740 1 --2- 172.21.15.45:0/573165428 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561d97fde890 0x561d97fdaa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:26.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.383+0000 7fea58574740 1 --2- 172.21.15.45:0/573165428 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561d97fd5cb0 0x561d97fea4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:26.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.383+0000 7fea58574740 1 --2- 172.21.15.45:0/573165428 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561d97fd5390 0x561d97fde020 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:26.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.383+0000 7fea58574740 1 -- 172.21.15.45:0/573165428 >> 172.21.15.45:0/573165428 conn(0x561d97fcfac0 msgr2=0x561d97fdb8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:17:26.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.383+0000 7fea58574740 1 -- 172.21.15.45:0/573165428 shutdown_connections 2024-04-16T17:17:26.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.383+0000 7fea58574740 1 -- 172.21.15.45:0/573165428 wait complete. 2024-04-16T17:17:26.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.384+0000 7fea58574740 1 -- start start 2024-04-16T17:17:26.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.384+0000 7fea58574740 1 Processor -- start 2024-04-16T17:17:26.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.385+0000 7fea58574740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561d97fd5390 0x561d97fe6ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:26.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.385+0000 7fea58574740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561d97fd5cb0 0x561d97fe1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:26.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.385+0000 7fea58574740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561d97fde890 0x561d97fe24f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:26.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.385+0000 7fea58574740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fea501afc20 con 0x561d97fd5390 2024-04-16T17:17:26.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.385+0000 7fea58574740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fea501b1090 con 0x561d97fd5cb0 2024-04-16T17:17:26.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.385+0000 7fea58574740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fea501b02f0 con 0x561d97fde890 2024-04-16T17:17:26.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.385+0000 7fea58574740 10 client.0 _ll_register_callbacks cb 0x7fea480046c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:17:26.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.385+0000 7fea4ea00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561d97fd5390 0x561d97fe6ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:26.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.385+0000 7fea4ea00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561d97fd5390 0x561d97fe6ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45090/0 (socket says 172.21.15.45:45090) 2024-04-16T17:17:26.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.385+0000 7fea4ea00640 1 -- 172.21.15.45:0/2302370758 learned_addr learned my addr 172.21.15.45:0/2302370758 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:17:26.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.385+0000 7fea4e000640 1 --2- 172.21.15.45:0/2302370758 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561d97fd5cb0 0x561d97fe1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:26.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.385+0000 7fea4f400640 1 --2- 172.21.15.45:0/2302370758 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561d97fde890 0x561d97fe24f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:26.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.385+0000 7fea4ea00640 1 -- 172.21.15.45:0/2302370758 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561d97fde890 msgr2=0x561d97fe24f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:17:26.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.385+0000 7fea4ea00640 1 --2- 172.21.15.45:0/2302370758 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561d97fde890 0x561d97fe24f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:26.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.385+0000 7fea4ea00640 1 -- 172.21.15.45:0/2302370758 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561d97fd5cb0 msgr2=0x561d97fe1fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:17:26.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.385+0000 7fea4ea00640 1 --2- 172.21.15.45:0/2302370758 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561d97fd5cb0 0x561d97fe1fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:26.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.385+0000 7fea4ea00640 1 -- 172.21.15.45:0/2302370758 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561d97fe2a30 con 0x561d97fd5390 2024-04-16T17:17:26.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.386+0000 7fea4ea00640 1 --2- 172.21.15.45:0/2302370758 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561d97fd5390 0x561d97fe6ad0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fea380040e0 tx=0x7fea38005d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5b62ac01e8c5943 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:17:26.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.386+0000 7fea3cc00640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-16T17:17:26.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.386+0000 7fea3cc00640 1 -- 172.21.15.45:0/2302370758 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea38039040 con 0x561d97fd5390 2024-04-16T17:17:26.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.386+0000 7fea3cc00640 1 -- 172.21.15.45:0/2302370758 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9496+0+0 (secure 0 0 0) 0x7fea38053dd0 con 0x561d97fd5390 2024-04-16T17:17:26.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.386+0000 7fea58574740 1 -- 172.21.15.45:0/2302370758 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fea38034440 con 0x561d97fd5390 2024-04-16T17:17:26.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.386+0000 7fea3cc00640 1 -- 172.21.15.45:0/2302370758 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fea38064050 con 0x561d97fd5390 2024-04-16T17:17:26.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.386+0000 7fea58574740 10 client.18132 fetch_fsmap learned FSMap version 44 2024-04-16T17:17:26.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.386+0000 7fea58574740 1 -- 172.21.15.45:0/2302370758 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x561d98054eb0 con 0x561d97fd5390 2024-04-16T17:17:26.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.387+0000 7fea3cc00640 1 -- 172.21.15.45:0/2302370758 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea38034440 con 0x561d97fd5390 2024-04-16T17:17:26.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.387+0000 7fea3cc00640 1 -- 172.21.15.45:0/2302370758 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 44) ==== 8346+0+0 (secure 0 0 0) 0x7fea3802c890 con 0x561d97fd5390 2024-04-16T17:17:26.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.387+0000 7fea58574740 10 client.18132 fetch_fsmap finished waiting for FSMap version 44 2024-04-16T17:17:26.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.387+0000 7fea58574740 10 client.18132 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:17:26.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.387+0000 7fea58574740 20 client.18132 populate_metadata read hostname 'smithi045' 2024-04-16T17:17:26.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.387+0000 7fea58574740 1 --2- 172.21.15.45:0/2302370758 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x561d98055250 0x561d98057640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:26.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.387+0000 7fea58574740 4 client.18132 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-16T17:17:26.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.387+0000 7fea58574740 1 -- 172.21.15.45:0/2302370758 --> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] -- command(tid 0: {"prefix": "scrub status"}) -- 0x561d97f429b0 con 0x561d98055250 2024-04-16T17:17:26.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.387+0000 7fea4e000640 1 --2- 172.21.15.45:0/2302370758 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x561d98055250 0x561d98057640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:26.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.388+0000 7fea4e000640 1 --2- 172.21.15.45:0/2302370758 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x561d98055250 0x561d98057640 crc :-1 s=READY pgs=258 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=75441cf74084c693 server_cookie=6f0098c5645af3c in_seq=0 out_seq=0 2024-04-16T17:17:26.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.388+0000 7fea3cc00640 10 client.18132 ms_handle_connect on v2:172.21.15.104:6838/3660070592 2024-04-16T17:17:26.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.388+0000 7fea3cc00640 1 -- 172.21.15.45:0/2302370758 <== mds.0 v2:172.21.15.104:6838/3660070592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x561d97f429b0 con 0x561d98055250 2024-04-16T17:17:26.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.388+0000 7fea3cc00640 10 client.18132 handle_command_reply: tid=0 2024-04-16T17:17:26.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.388+0000 7fea58574740 1 client.18132 shutdown 2024-04-16T17:17:26.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.388+0000 7fea58574740 1 -- 172.21.15.45:0/2302370758 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561d97fd5390 msgr2=0x561d97fe6ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:17:26.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.388+0000 7fea58574740 1 --2- 172.21.15.45:0/2302370758 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561d97fd5390 0x561d97fe6ad0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fea380040e0 tx=0x7fea38005d20 comp rx=0 tx=0).stop 2024-04-16T17:17:26.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.388+0000 7fea58574740 1 -- 172.21.15.45:0/2302370758 shutdown_connections 2024-04-16T17:17:26.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.388+0000 7fea58574740 1 --2- 172.21.15.45:0/2302370758 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x561d98055250 0x561d98057640 crc :-1 s=READY pgs=258 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:26.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.388+0000 7fea58574740 1 --2- 172.21.15.45:0/2302370758 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561d97fde890 0x561d97fe24f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:26.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.388+0000 7fea58574740 1 --2- 172.21.15.45:0/2302370758 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561d97fd5cb0 0x561d97fe1fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:26.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.388+0000 7fea58574740 1 --2- 172.21.15.45:0/2302370758 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561d97fd5390 0x561d97fe6ad0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:26.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.388+0000 7fea3cc00640 0 client.18132 ms_handle_reset on v2:172.21.15.104:6838/3660070592 2024-04-16T17:17:26.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.388+0000 7fea58574740 1 -- 172.21.15.45:0/2302370758 >> 172.21.15.45:0/2302370758 conn(0x561d97fcfac0 msgr2=0x561d97fdb8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:17:26.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.389+0000 7fea58574740 1 -- 172.21.15.45:0/2302370758 shutdown_connections 2024-04-16T17:17:26.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.389+0000 7fea58574740 1 -- 172.21.15.45:0/2302370758 wait complete. 2024-04-16T17:17:26.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.389+0000 7fea58574740 20 client.18132 trim_cache size 0 max 16384 2024-04-16T17:17:26.399 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-16T17:17:26.399 INFO:teuthology.orchestra.run.smithi045.stdout: "status": "no active scrubs running", 2024-04-16T17:17:26.399 INFO:teuthology.orchestra.run.smithi045.stdout: "scrubs": {} 2024-04-16T17:17:26.400 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-16T17:17:26.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.389+0000 7fea4fe00640 1 -- 172.21.15.45:0/702577115 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fea340a3d20 msgr2=0x7fea340a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:17:26.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.389+0000 7fea4fe00640 1 --2- 172.21.15.45:0/702577115 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fea340a3d20 0x7fea340a61e0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x561d97fd0c50 tx=0x7fea400095c0 comp rx=0 tx=0).stop 2024-04-16T17:17:26.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.389+0000 7fea4fe00640 1 -- 172.21.15.45:0/702577115 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fea5010e6e0 msgr2=0x7fea5019a830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:17:26.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.389+0000 7fea4fe00640 1 --2- 172.21.15.45:0/702577115 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fea5010e6e0 0x7fea5019a830 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fea4800e970 tx=0x7fea4800ee30 comp rx=0 tx=0).stop 2024-04-16T17:17:26.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.389+0000 7fea4fe00640 1 -- 172.21.15.45:0/702577115 shutdown_connections 2024-04-16T17:17:26.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.389+0000 7fea4fe00640 1 --2- 172.21.15.45:0/702577115 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fea340a3d20 0x7fea340a61e0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:26.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.389+0000 7fea4fe00640 1 --2- 172.21.15.45:0/702577115 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fea5010e6e0 0x7fea5019a830 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:26.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.389+0000 7fea4fe00640 1 --2- 172.21.15.45:0/702577115 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fea50074d50 0x7fea501a0b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:26.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.389+0000 7fea4fe00640 1 --2- 172.21.15.45:0/702577115 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fea50074430 0x7fea501a0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:26.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.389+0000 7fea4fe00640 1 -- 172.21.15.45:0/702577115 >> 172.21.15.45:0/702577115 conn(0x7fea500fd9d0 msgr2=0x7fea500fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:17:26.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.390+0000 7fea4fe00640 1 -- 172.21.15.45:0/702577115 shutdown_connections 2024-04-16T17:17:26.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.390+0000 7fea4fe00640 1 -- 172.21.15.45:0/702577115 wait complete. 2024-04-16T17:17:26.407 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-16T17:17:26.407 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-16T17:17:26.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.633+0000 7f39bf400640 1 -- 172.21.15.45:0/2044806469 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f39b8109520 msgr2=0x7f39b81110f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:17:26.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.633+0000 7f39bf400640 1 --2- 172.21.15.45:0/2044806469 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f39b8109520 0x7f39b81110f0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f39b0009be0 tx=0x7f39b002bef0 comp rx=0 tx=0).stop 2024-04-16T17:17:26.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.633+0000 7f39bf400640 1 -- 172.21.15.45:0/2044806469 shutdown_connections 2024-04-16T17:17:26.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.633+0000 7f39bf400640 1 --2- 172.21.15.45:0/2044806469 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f39b8104e00 0x7f39b8108e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:26.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.633+0000 7f39bf400640 1 --2- 172.21.15.45:0/2044806469 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f39b8109520 0x7f39b81110f0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:26.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.633+0000 7f39bf400640 1 --2- 172.21.15.45:0/2044806469 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f39b8104450 0x7f39b8104830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:26.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.633+0000 7f39bf400640 1 -- 172.21.15.45:0/2044806469 >> 172.21.15.45:0/2044806469 conn(0x7f39b80ffcb0 msgr2=0x7f39b81020d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:17:26.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.633+0000 7f39bf400640 1 -- 172.21.15.45:0/2044806469 shutdown_connections 2024-04-16T17:17:26.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.633+0000 7f39bf400640 1 -- 172.21.15.45:0/2044806469 wait complete. 2024-04-16T17:17:26.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.634+0000 7f39bf400640 1 Processor -- start 2024-04-16T17:17:26.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.635+0000 7f39bf400640 1 -- start start 2024-04-16T17:17:26.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.635+0000 7f39bf400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f39b8104450 0x7f39b81a0760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:26.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.635+0000 7f39bf400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f39b8109520 0x7f39b81a0ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:26.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.635+0000 7f39bf400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f39b819a910 0x7f39b819adc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:26.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.635+0000 7f39bf400640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f39b8114630 con 0x7f39b8104450 2024-04-16T17:17:26.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.635+0000 7f39bf400640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f39b81144b0 con 0x7f39b8109520 2024-04-16T17:17:26.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.635+0000 7f39bf400640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f39b81147b0 con 0x7f39b819a910 2024-04-16T17:17:26.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.635+0000 7f39bca00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f39b8104450 0x7f39b81a0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:26.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.635+0000 7f39bca00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f39b8104450 0x7f39b81a0760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45104/0 (socket says 172.21.15.45:45104) 2024-04-16T17:17:26.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.635+0000 7f39b7e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f39b8109520 0x7f39b81a0ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:26.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.635+0000 7f39bca00640 1 -- 172.21.15.45:0/3368150411 learned_addr learned my addr 172.21.15.45:0/3368150411 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:17:26.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.635+0000 7f39bd400640 1 --2- 172.21.15.45:0/3368150411 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f39b819a910 0x7f39b819adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:26.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.636+0000 7f39b7e00640 1 -- 172.21.15.45:0/3368150411 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f39b819a910 msgr2=0x7f39b819adc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:17:26.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.636+0000 7f39b7e00640 1 --2- 172.21.15.45:0/3368150411 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f39b819a910 0x7f39b819adc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:26.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.636+0000 7f39b7e00640 1 -- 172.21.15.45:0/3368150411 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f39b8104450 msgr2=0x7f39b81a0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:17:26.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.636+0000 7f39b7e00640 1 --2- 172.21.15.45:0/3368150411 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f39b8104450 0x7f39b81a0760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:26.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.636+0000 7f39b7e00640 1 -- 172.21.15.45:0/3368150411 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39b819b430 con 0x7f39b8109520 2024-04-16T17:17:26.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.636+0000 7f39b7e00640 1 --2- 172.21.15.45:0/3368150411 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f39b8109520 0x7f39b81a0ca0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f39a400e9c0 tx=0x7f39a400ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:17:26.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.636+0000 7f39b5600640 1 -- 172.21.15.45:0/3368150411 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39a400ccc0 con 0x7f39b8109520 2024-04-16T17:17:26.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.636+0000 7f39b5600640 1 -- 172.21.15.45:0/3368150411 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f39a4005020 con 0x7f39b8109520 2024-04-16T17:17:26.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.636+0000 7f39b5600640 1 -- 172.21.15.45:0/3368150411 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39a4004a00 con 0x7f39b8109520 2024-04-16T17:17:26.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.636+0000 7f39bf400640 1 -- 172.21.15.45:0/3368150411 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39b819b720 con 0x7f39b8109520 2024-04-16T17:17:26.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.637+0000 7f39bf400640 1 -- 172.21.15.45:0/3368150411 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39b81a7970 con 0x7f39b8109520 2024-04-16T17:17:26.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.638+0000 7f39bf400640 1 -- 172.21.15.45:0/3368150411 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39b8068e20 con 0x7f39b8109520 2024-04-16T17:17:26.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.640+0000 7f39b5600640 1 -- 172.21.15.45:0/3368150411 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f39a40051d0 con 0x7f39b8109520 2024-04-16T17:17:26.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.642+0000 7f39b5600640 1 --2- 172.21.15.45:0/3368150411 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f39a00a3da0 0x7f39a00a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:26.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.642+0000 7f39b5600640 1 -- 172.21.15.45:0/3368150411 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 9496+0+0 (secure 0 0 0) 0x7f39a40d0bd0 con 0x7f39b8109520 2024-04-16T17:17:26.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.642+0000 7f39bca00640 1 --2- 172.21.15.45:0/3368150411 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f39a00a3da0 0x7f39a00a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:26.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.643+0000 7f39bca00640 1 --2- 172.21.15.45:0/3368150411 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f39a00a3da0 0x7f39a00a6260 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f39b819ba50 tx=0x7f39ac009260 comp rx=0 tx=0).ready entity=mgr.25411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:17:26.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.644+0000 7f39b5600640 1 -- 172.21.15.45:0/3368150411 <== mon.2 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f39a409ce40 con 0x7f39b8109520 2024-04-16T17:17:26.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.893+0000 7f39bf400640 1 -- 172.21.15.45:0/3368150411 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f39b819be40 con 0x7f39b8109520 2024-04-16T17:17:26.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.894+0000 7f39b5600640 1 -- 172.21.15.45:0/3368150411 <== mon.2 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) ==== 94+0+11887 (secure 0 0 0) 0x7f39b819be40 con 0x7f39b8109520 2024-04-16T17:17:26.894 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-16T17:17:26.894 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":44,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/1748842443","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":1748842443},{"type":"v1","addr":"172.21.15.45:6835","nonce":1748842443}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6839/1308627051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":1308627051},{"type":"v1","addr":"172.21.15.45:6839","nonce":1308627051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/1768242489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1768242489},{"type":"v1","addr":"172.21.15.117:6833","nonce":1768242489}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6833/3558902585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":3558902585},{"type":"v1","addr":"172.21.15.104:6833","nonce":3558902585}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24478,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/1308142903","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":1308142903},{"type":"v1","addr":"172.21.15.45:6837","nonce":1308142903}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24491,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6835/3430679937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":3430679937},{"type":"v1","addr":"172.21.15.104:6835","nonce":3430679937}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":44,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-16T17:03:24.663611+0000","modified":"2024-04-16T17:15:52.264451+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24581,"mds_1":24569,"mds_2":24532},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24499":{"gid":24499,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/2224123841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2224123841},{"type":"v1","addr":"172.21.15.117:6835","nonce":2224123841}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6837/2471356815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":2471356815},{"type":"v1","addr":"172.21.15.117:6837","nonce":2471356815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24532":{"gid":24532,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":13,"addr":"172.21.15.117:6839/2378675949","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":2378675949},{"type":"v1","addr":"172.21.15.117:6839","nonce":2378675949}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24569":{"gid":24569,"name":"a","rank":1,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.45:6841/3755310822","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6840","nonce":3755310822},{"type":"v1","addr":"172.21.15.45:6841","nonce":3755310822}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24581":{"gid":24581,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.104:6839/3660070592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6838","nonce":3660070592},{"type":"v1","addr":"172.21.15.104:6839","nonce":3660070592}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_27569":{"gid":27569,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6837/1833645246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":1833645246},{"type":"v1","addr":"172.21.15.104:6837","nonce":1833645246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24581,"qdb_cluster":[24569,24532,24581]},"id":1}]} 2024-04-16T17:17:26.894 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 44 2024-04-16T17:17:26.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.897+0000 7f39bf400640 1 -- 172.21.15.45:0/3368150411 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f39a00a3da0 msgr2=0x7f39a00a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:17:26.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.897+0000 7f39bf400640 1 --2- 172.21.15.45:0/3368150411 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f39a00a3da0 0x7f39a00a6260 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f39b819ba50 tx=0x7f39ac009260 comp rx=0 tx=0).stop 2024-04-16T17:17:26.897 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.897+0000 7f39bf400640 1 -- 172.21.15.45:0/3368150411 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f39b8109520 msgr2=0x7f39b81a0ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:17:26.897 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.897+0000 7f39bf400640 1 --2- 172.21.15.45:0/3368150411 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f39b8109520 0x7f39b81a0ca0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f39a400e9c0 tx=0x7f39a400ee80 comp rx=0 tx=0).stop 2024-04-16T17:17:26.897 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.897+0000 7f39bf400640 1 -- 172.21.15.45:0/3368150411 shutdown_connections 2024-04-16T17:17:26.897 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.897+0000 7f39bf400640 1 --2- 172.21.15.45:0/3368150411 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f39a00a3da0 0x7f39a00a6260 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:26.897 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.897+0000 7f39bf400640 1 --2- 172.21.15.45:0/3368150411 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f39b819a910 0x7f39b819adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:26.897 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.897+0000 7f39bf400640 1 --2- 172.21.15.45:0/3368150411 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f39b8109520 0x7f39b81a0ca0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:26.897 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.897+0000 7f39bf400640 1 --2- 172.21.15.45:0/3368150411 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f39b8104450 0x7f39b81a0760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:26.897 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.897+0000 7f39bf400640 1 -- 172.21.15.45:0/3368150411 >> 172.21.15.45:0/3368150411 conn(0x7f39b80ffcb0 msgr2=0x7f39b8101f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:17:26.897 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.897+0000 7f39bf400640 1 -- 172.21.15.45:0/3368150411 shutdown_connections 2024-04-16T17:17:26.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:26.897+0000 7f39bf400640 1 -- 172.21.15.45:0/3368150411 wait complete. 2024-04-16T17:17:26.913 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:17:26 smithi045 ceph-mon[31676]: pgmap v334: 129 pgs: 129 active+clean; 55 MiB data, 874 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 92 B/s wr, 5 op/s 2024-04-16T17:17:26.917 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-16T17:17:27.054 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:17:26 smithi117 ceph-mon[36730]: pgmap v334: 129 pgs: 129 active+clean; 55 MiB data, 874 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 92 B/s wr, 5 op/s 2024-04-16T17:17:27.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.146+0000 7fbdd6600640 1 -- 172.21.15.45:0/3475856071 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbdd0104cc0 msgr2=0x7fbdd0108d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:17:27.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.146+0000 7fbdd6600640 1 --2- 172.21.15.45:0/3475856071 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbdd0104cc0 0x7fbdd0108d00 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fbdbc009860 tx=0x7fbdbc02bb70 comp rx=0 tx=0).stop 2024-04-16T17:17:27.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.146+0000 7fbdd6600640 1 -- 172.21.15.45:0/3475856071 shutdown_connections 2024-04-16T17:17:27.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.146+0000 7fbdd6600640 1 --2- 172.21.15.45:0/3475856071 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbdd0104cc0 0x7fbdd0108d00 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:27.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.146+0000 7fbdd6600640 1 --2- 172.21.15.45:0/3475856071 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbdd01093e0 0x7fbdd0110fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:27.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.146+0000 7fbdd6600640 1 --2- 172.21.15.45:0/3475856071 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbdd0104310 0x7fbdd01046f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:27.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.146+0000 7fbdd6600640 1 -- 172.21.15.45:0/3475856071 >> 172.21.15.45:0/3475856071 conn(0x7fbdd00ffb10 msgr2=0x7fbdd0101f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:17:27.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.147+0000 7fbdd6600640 1 -- 172.21.15.45:0/3475856071 shutdown_connections 2024-04-16T17:17:27.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.147+0000 7fbdd6600640 1 -- 172.21.15.45:0/3475856071 wait complete. 2024-04-16T17:17:27.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.148+0000 7fbdd6600640 1 Processor -- start 2024-04-16T17:17:27.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.148+0000 7fbdd6600640 1 -- start start 2024-04-16T17:17:27.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.148+0000 7fbdd6600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbdd0104310 0x7fbdd01a0630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:27.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.148+0000 7fbdd6600640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbdd01093e0 0x7fbdd01a0b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:27.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.148+0000 7fbdd6600640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbdd019a7f0 0x7fbdd019aca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:27.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.148+0000 7fbdd6600640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fbdd0114450 con 0x7fbdd0104310 2024-04-16T17:17:27.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.148+0000 7fbdd6600640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fbdd01142d0 con 0x7fbdd019a7f0 2024-04-16T17:17:27.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.148+0000 7fbdd6600640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fbdd01145d0 con 0x7fbdd01093e0 2024-04-16T17:17:27.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.148+0000 7fbdd5200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbdd0104310 0x7fbdd01a0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:27.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.148+0000 7fbdd5c00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbdd019a7f0 0x7fbdd019aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:27.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.148+0000 7fbdd5200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbdd0104310 0x7fbdd01a0630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45128/0 (socket says 172.21.15.45:45128) 2024-04-16T17:17:27.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.149+0000 7fbdd5c00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbdd019a7f0 0x7fbdd019aca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:43684/0 (socket says 172.21.15.45:43684) 2024-04-16T17:17:27.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.149+0000 7fbdd5200640 1 -- 172.21.15.45:0/2501177050 learned_addr learned my addr 172.21.15.45:0/2501177050 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:17:27.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.149+0000 7fbdcfe00640 1 --2- 172.21.15.45:0/2501177050 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbdd01093e0 0x7fbdd01a0b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:27.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.149+0000 7fbdd5c00640 1 -- 172.21.15.45:0/2501177050 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbdd01093e0 msgr2=0x7fbdd01a0b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:17:27.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.149+0000 7fbdd5c00640 1 --2- 172.21.15.45:0/2501177050 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbdd01093e0 0x7fbdd01a0b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:27.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.149+0000 7fbdd5c00640 1 -- 172.21.15.45:0/2501177050 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbdd0104310 msgr2=0x7fbdd01a0630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:17:27.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.149+0000 7fbdd5c00640 1 --2- 172.21.15.45:0/2501177050 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbdd0104310 0x7fbdd01a0630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:27.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.149+0000 7fbdd5c00640 1 -- 172.21.15.45:0/2501177050 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbdd019b380 con 0x7fbdd019a7f0 2024-04-16T17:17:27.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.149+0000 7fbdcfe00640 1 --2- 172.21.15.45:0/2501177050 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbdd01093e0 0x7fbdd01a0b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:17:27.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.149+0000 7fbdd5200640 1 --2- 172.21.15.45:0/2501177050 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbdd0104310 0x7fbdd01a0630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:17:27.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.149+0000 7fbdd5c00640 1 --2- 172.21.15.45:0/2501177050 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbdd019a7f0 0x7fbdd019aca0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fbdc8009fd0 tx=0x7fbdc800ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:17:27.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.149+0000 7fbdcd600640 1 -- 172.21.15.45:0/2501177050 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdc800ce20 con 0x7fbdd019a7f0 2024-04-16T17:17:27.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.150+0000 7fbdcd600640 1 -- 172.21.15.45:0/2501177050 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbdc8005020 con 0x7fbdd019a7f0 2024-04-16T17:17:27.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.150+0000 7fbdd6600640 1 -- 172.21.15.45:0/2501177050 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbdd019b670 con 0x7fbdd019a7f0 2024-04-16T17:17:27.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.150+0000 7fbdcd600640 1 -- 172.21.15.45:0/2501177050 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdc8010720 con 0x7fbdd019a7f0 2024-04-16T17:17:27.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.150+0000 7fbdd6600640 1 -- 172.21.15.45:0/2501177050 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbdd01a77d0 con 0x7fbdd019a7f0 2024-04-16T17:17:27.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.154+0000 7fbdda967740 1 Processor -- start 2024-04-16T17:17:27.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.154+0000 7fbdcd600640 1 -- 172.21.15.45:0/2501177050 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fbdc8005540 con 0x7fbdd019a7f0 2024-04-16T17:17:27.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.155+0000 7fbdda967740 1 -- start start 2024-04-16T17:17:27.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.155+0000 7fbdda967740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561c20e8bf10 0x561c20e8c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:27.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.155+0000 7fbdda967740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561c20e8c830 0x561c20e94ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:27.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.155+0000 7fbdda967740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561c20e95410 0x561c20e97800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:27.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.155+0000 7fbdda967740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x561c20df1020 con 0x561c20e8bf10 2024-04-16T17:17:27.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.155+0000 7fbdda967740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x561c20d97460 con 0x561c20e95410 2024-04-16T17:17:27.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.155+0000 7fbdda967740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x561c20decfe0 con 0x561c20e8c830 2024-04-16T17:17:27.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.155+0000 7fbdd5200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561c20e8c830 0x561c20e94ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:27.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.155+0000 7fbdcfe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561c20e8bf10 0x561c20e8c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:27.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.155+0000 7fbdd5200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561c20e8c830 0x561c20e94ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:35800/0 (socket says 172.21.15.45:35800) 2024-04-16T17:17:27.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.155+0000 7fbdd5200640 1 -- 172.21.15.45:0/1434626499 learned_addr learned my addr 172.21.15.45:0/1434626499 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:17:27.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.155+0000 7fbdcfe00640 1 --2- 172.21.15.45:0/1434626499 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561c20e95410 0x561c20e97800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:27.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.156+0000 7fbdd5200640 1 -- 172.21.15.45:0/1434626499 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561c20e95410 msgr2=0x561c20e97800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:17:27.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.156+0000 7fbdd5200640 1 --2- 172.21.15.45:0/1434626499 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561c20e95410 0x561c20e97800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:27.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.156+0000 7fbdd5200640 1 -- 172.21.15.45:0/1434626499 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561c20e8bf10 msgr2=0x561c20e8c2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:17:27.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.156+0000 7fbdd5200640 1 --2- 172.21.15.45:0/1434626499 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561c20e8bf10 0x561c20e8c2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:27.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.156+0000 7fbdd5200640 1 -- 172.21.15.45:0/1434626499 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561c20d9f040 con 0x561c20e8c830 2024-04-16T17:17:27.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.156+0000 7fbdcfe00640 1 --2- 172.21.15.45:0/1434626499 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561c20e8bf10 0x561c20e8c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:17:27.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.156+0000 7fbdd5200640 1 --2- 172.21.15.45:0/1434626499 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561c20e8c830 0x561c20e94ed0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fbdc40115c0 tx=0x7fbdc4011a80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7bd7fbb1523c5d16 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:17:27.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.156+0000 7fbdc2000640 1 -- 172.21.15.45:0/1434626499 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdc400f6c0 con 0x561c20e8c830 2024-04-16T17:17:27.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.156+0000 7fbdcd600640 1 --2- 172.21.15.45:0/2501177050 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fbdb80a3da0 0x7fbdb80a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:27.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.156+0000 7fbdc2000640 1 -- 172.21.15.45:0/1434626499 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbdc400fd40 con 0x561c20e8c830 2024-04-16T17:17:27.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.156+0000 7fbdc2000640 1 -- 172.21.15.45:0/1434626499 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdc40183b0 con 0x561c20e8c830 2024-04-16T17:17:27.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.157+0000 7fbdcd600640 1 -- 172.21.15.45:0/2501177050 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 9496+0+0 (secure 0 0 0) 0x7fbdc80d1200 con 0x7fbdd019a7f0 2024-04-16T17:17:27.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.157+0000 7fbdcfe00640 1 --2- 172.21.15.45:0/2501177050 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fbdb80a3da0 0x7fbdb80a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:27.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.157+0000 7fbdda967740 1 -- 172.21.15.45:0/1434626499 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561c20e8c830 msgr2=0x561c20e94ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:17:27.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.157+0000 7fbdda967740 1 --2- 172.21.15.45:0/1434626499 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561c20e8c830 0x561c20e94ed0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fbdc40115c0 tx=0x7fbdc4011a80 comp rx=0 tx=0).stop 2024-04-16T17:17:27.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.157+0000 7fbdda967740 1 -- 172.21.15.45:0/1434626499 shutdown_connections 2024-04-16T17:17:27.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.157+0000 7fbdda967740 1 --2- 172.21.15.45:0/1434626499 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561c20e8c830 0x561c20e94ed0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:27.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.157+0000 7fbdda967740 1 --2- 172.21.15.45:0/1434626499 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561c20e95410 0x561c20e97800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:27.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.157+0000 7fbdda967740 1 --2- 172.21.15.45:0/1434626499 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561c20e8bf10 0x561c20e8c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:27.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.157+0000 7fbdda967740 1 -- 172.21.15.45:0/1434626499 >> 172.21.15.45:0/1434626499 conn(0x561c20e86640 msgr2=0x561c20e86a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:17:27.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.157+0000 7fbdcfe00640 1 --2- 172.21.15.45:0/2501177050 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fbdb80a3da0 0x7fbdb80a6260 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fbdbc009830 tx=0x7fbdbc0058d0 comp rx=0 tx=0).ready entity=mgr.25411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:17:27.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.157+0000 7fbdda967740 1 -- 172.21.15.45:0/1434626499 shutdown_connections 2024-04-16T17:17:27.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.157+0000 7fbdda967740 1 -- 172.21.15.45:0/1434626499 wait complete. 2024-04-16T17:17:27.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.158+0000 7fbdda967740 1 -- start start 2024-04-16T17:17:27.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.158+0000 7fbdda967740 1 Processor -- start 2024-04-16T17:17:27.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.159+0000 7fbdda967740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561c20e8bf10 0x561c20e92400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:27.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.159+0000 7fbdda967740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561c20e8c830 0x561c20e92940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:27.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.159+0000 7fbdda967740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561c20e95410 0x561c20e92e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:27.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.159+0000 7fbdda967740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x561c20e98200 con 0x561c20e95410 2024-04-16T17:17:27.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.159+0000 7fbdda967740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x561c20daa830 con 0x561c20e8c830 2024-04-16T17:17:27.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.159+0000 7fbdda967740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x561c20e98380 con 0x561c20e8bf10 2024-04-16T17:17:27.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.159+0000 7fbdda967740 10 client.0 _ll_register_callbacks cb 0x561c20db2150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:17:27.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.159+0000 7fbdd5200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561c20e8bf10 0x561c20e92400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:27.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.159+0000 7fbdd5200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561c20e8bf10 0x561c20e92400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:35816/0 (socket says 172.21.15.45:35816) 2024-04-16T17:17:27.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.159+0000 7fbdd5c00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561c20e95410 0x561c20e92e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:27.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.159+0000 7fbdd5200640 1 -- 172.21.15.45:0/3210953786 learned_addr learned my addr 172.21.15.45:0/3210953786 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:17:27.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.159+0000 7fbdcfe00640 1 --2- 172.21.15.45:0/3210953786 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561c20e8c830 0x561c20e92940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:27.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.160+0000 7fbdd5c00640 1 -- 172.21.15.45:0/3210953786 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561c20e8bf10 msgr2=0x561c20e92400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:17:27.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.160+0000 7fbdd5c00640 1 --2- 172.21.15.45:0/3210953786 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561c20e8bf10 0x561c20e92400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:27.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.160+0000 7fbdd5c00640 1 -- 172.21.15.45:0/3210953786 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561c20e8c830 msgr2=0x561c20e92940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:17:27.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.160+0000 7fbdd5c00640 1 --2- 172.21.15.45:0/3210953786 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561c20e8c830 0x561c20e92940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:27.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.160+0000 7fbdd5c00640 1 -- 172.21.15.45:0/3210953786 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561c20dae920 con 0x561c20e95410 2024-04-16T17:17:27.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.160+0000 7fbdd5200640 1 --2- 172.21.15.45:0/3210953786 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561c20e8bf10 0x561c20e92400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:17:27.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.160+0000 7fbdd5c00640 1 --2- 172.21.15.45:0/3210953786 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561c20e95410 0x561c20e92e80 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fbdc80ce520 tx=0x7fbdc809cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a48bea95298361c server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:17:27.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.160+0000 7fbdb6a00640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-16T17:17:27.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.160+0000 7fbdb6a00640 1 -- 172.21.15.45:0/3210953786 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdc80b6060 con 0x561c20e95410 2024-04-16T17:17:27.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.160+0000 7fbdb6a00640 1 -- 172.21.15.45:0/3210953786 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9496+0+0 (secure 0 0 0) 0x7fbdc80993e0 con 0x561c20e95410 2024-04-16T17:17:27.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.160+0000 7fbdda967740 1 -- 172.21.15.45:0/3210953786 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbdc400fd40 con 0x561c20e95410 2024-04-16T17:17:27.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.160+0000 7fbdb6a00640 1 -- 172.21.15.45:0/3210953786 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fbdc809cfd0 con 0x561c20e95410 2024-04-16T17:17:27.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.160+0000 7fbdda967740 10 client.18150 fetch_fsmap learned FSMap version 44 2024-04-16T17:17:27.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.160+0000 7fbdda967740 1 -- 172.21.15.45:0/3210953786 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x561c20d7c7f0 con 0x561c20e95410 2024-04-16T17:17:27.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.161+0000 7fbdb6a00640 1 -- 172.21.15.45:0/3210953786 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbdc400fd40 con 0x561c20e95410 2024-04-16T17:17:27.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.161+0000 7fbdb6a00640 1 -- 172.21.15.45:0/3210953786 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 44) ==== 8346+0+0 (secure 0 0 0) 0x7fbdc80bd0d0 con 0x561c20e95410 2024-04-16T17:17:27.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.161+0000 7fbdda967740 10 client.18150 fetch_fsmap finished waiting for FSMap version 44 2024-04-16T17:17:27.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.161+0000 7fbdda967740 10 client.18150 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-16T17:17:27.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.161+0000 7fbdda967740 20 client.18150 populate_metadata read hostname 'smithi045' 2024-04-16T17:17:27.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.161+0000 7fbdda967740 1 --2- 172.21.15.45:0/3210953786 >> [v2:172.21.15.117:6838/2378675949,v1:172.21.15.117:6839/2378675949] conn(0x561c20ea57a0 0x561c20ea7bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:27.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.161+0000 7fbdda967740 4 client.18150 mds_command: new command op to 24532 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:17:27.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.161+0000 7fbdda967740 1 -- 172.21.15.45:0/3210953786 --> [v2:172.21.15.117:6838/2378675949,v1:172.21.15.117:6839/2378675949] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561c20dfea40 con 0x561c20ea57a0 2024-04-16T17:17:27.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.161+0000 7fbdd5200640 1 --2- 172.21.15.45:0/3210953786 >> [v2:172.21.15.117:6838/2378675949,v1:172.21.15.117:6839/2378675949] conn(0x561c20ea57a0 0x561c20ea7bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:27.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.162+0000 7fbdd5200640 1 --2- 172.21.15.45:0/3210953786 >> [v2:172.21.15.117:6838/2378675949,v1:172.21.15.117:6839/2378675949] conn(0x561c20ea57a0 0x561c20ea7bb0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3ef702081971b3d8 server_cookie=5f1f6ec414c5fa79 in_seq=0 out_seq=0 2024-04-16T17:17:27.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.162+0000 7fbdb6a00640 10 client.18150 ms_handle_connect on v2:172.21.15.117:6838/2378675949 2024-04-16T17:17:27.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.163+0000 7fbdb6a00640 1 -- 172.21.15.45:0/3210953786 <== mds.2 v2:172.21.15.117:6838/2378675949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561c20dfea40 con 0x561c20ea57a0 2024-04-16T17:17:27.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.163+0000 7fbdb6a00640 10 client.18150 handle_command_reply: tid=0 2024-04-16T17:17:27.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.163+0000 7fbdda967740 1 client.18150 shutdown 2024-04-16T17:17:27.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.163+0000 7fbdda967740 1 -- 172.21.15.45:0/3210953786 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561c20e95410 msgr2=0x561c20e92e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:17:27.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.163+0000 7fbdda967740 1 --2- 172.21.15.45:0/3210953786 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561c20e95410 0x561c20e92e80 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fbdc80ce520 tx=0x7fbdc809cd70 comp rx=0 tx=0).stop 2024-04-16T17:17:27.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.163+0000 7fbdda967740 1 -- 172.21.15.45:0/3210953786 shutdown_connections 2024-04-16T17:17:27.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.163+0000 7fbdda967740 1 --2- 172.21.15.45:0/3210953786 >> [v2:172.21.15.117:6838/2378675949,v1:172.21.15.117:6839/2378675949] conn(0x561c20ea57a0 0x561c20ea7bb0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:27.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.164+0000 7fbdda967740 1 --2- 172.21.15.45:0/3210953786 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561c20e95410 0x561c20e92e80 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:27.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.164+0000 7fbdda967740 1 --2- 172.21.15.45:0/3210953786 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561c20e8c830 0x561c20e92940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:27.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.164+0000 7fbdda967740 1 --2- 172.21.15.45:0/3210953786 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561c20e8bf10 0x561c20e92400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:27.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.164+0000 7fbdb6a00640 0 client.18150 ms_handle_reset on v2:172.21.15.117:6838/2378675949 2024-04-16T17:17:27.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.164+0000 7fbdda967740 1 -- 172.21.15.45:0/3210953786 >> 172.21.15.45:0/3210953786 conn(0x561c20e86640 msgr2=0x561c20e88390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:17:27.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.164+0000 7fbdda967740 1 -- 172.21.15.45:0/3210953786 shutdown_connections 2024-04-16T17:17:27.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.164+0000 7fbdda967740 1 -- 172.21.15.45:0/3210953786 wait complete. 2024-04-16T17:17:27.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.164+0000 7fbdda967740 20 client.18150 trim_cache size 0 max 16384 2024-04-16T17:17:27.177 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:17:26 smithi104 ceph-mon[36457]: pgmap v334: 129 pgs: 129 active+clean; 55 MiB data, 874 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 92 B/s wr, 5 op/s 2024-04-16T17:17:27.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.179+0000 7fbdda967740 1 Processor -- start 2024-04-16T17:17:27.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.179+0000 7fbdda967740 1 -- start start 2024-04-16T17:17:27.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.179+0000 7fbdda967740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561c20e8bf10 0x561c20e94b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:27.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.179+0000 7fbdda967740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561c20e8c830 0x561c20ea10c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:27.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.179+0000 7fbdda967740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561c20e95410 0x561c20e91560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:27.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.179+0000 7fbdda967740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fbdd01b3e90 con 0x561c20e8bf10 2024-04-16T17:17:27.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.179+0000 7fbdda967740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fbdd01b1760 con 0x561c20e95410 2024-04-16T17:17:27.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.179+0000 7fbdda967740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fbdd01af5d0 con 0x561c20e8c830 2024-04-16T17:17:27.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.180+0000 7fbdd5c00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561c20e95410 0x561c20e91560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:27.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.180+0000 7fbdd5c00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561c20e95410 0x561c20e91560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:43704/0 (socket says 172.21.15.45:43704) 2024-04-16T17:17:27.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.180+0000 7fbdcfe00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561c20e8c830 0x561c20ea10c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:27.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.180+0000 7fbdd5c00640 1 -- 172.21.15.45:0/1498284810 learned_addr learned my addr 172.21.15.45:0/1498284810 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:17:27.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.180+0000 7fbdd5200640 1 --2- 172.21.15.45:0/1498284810 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561c20e8bf10 0x561c20e94b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:27.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.180+0000 7fbdd5c00640 1 -- 172.21.15.45:0/1498284810 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561c20e8c830 msgr2=0x561c20ea10c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:17:27.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.180+0000 7fbdd5c00640 1 --2- 172.21.15.45:0/1498284810 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561c20e8c830 0x561c20ea10c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:27.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.180+0000 7fbdd5c00640 1 -- 172.21.15.45:0/1498284810 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561c20e8bf10 msgr2=0x561c20e94b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:17:27.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.180+0000 7fbdd5c00640 1 --2- 172.21.15.45:0/1498284810 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561c20e8bf10 0x561c20e94b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:27.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.180+0000 7fbdd5c00640 1 -- 172.21.15.45:0/1498284810 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561c20b033f0 con 0x561c20e95410 2024-04-16T17:17:27.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.180+0000 7fbdcfe00640 1 --2- 172.21.15.45:0/1498284810 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561c20e8c830 0x561c20ea10c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:17:27.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.180+0000 7fbdd5200640 1 --2- 172.21.15.45:0/1498284810 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561c20e8bf10 0x561c20e94b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:17:27.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.180+0000 7fbdd5c00640 1 --2- 172.21.15.45:0/1498284810 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561c20e95410 0x561c20e91560 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fbdc800c760 tx=0x7fbdc80a5b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8e555075ccd801a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:17:27.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.181+0000 7fbdc1600640 1 -- 172.21.15.45:0/1498284810 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdc80b6020 con 0x561c20e95410 2024-04-16T17:17:27.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.181+0000 7fbdc1600640 1 -- 172.21.15.45:0/1498284810 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbdc809c2c0 con 0x561c20e95410 2024-04-16T17:17:27.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.181+0000 7fbdc1600640 1 -- 172.21.15.45:0/1498284810 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdc80bd0d0 con 0x561c20e95410 2024-04-16T17:17:27.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.181+0000 7fbdda967740 1 -- 172.21.15.45:0/1498284810 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561c20e95410 msgr2=0x561c20e91560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:17:27.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.181+0000 7fbdda967740 1 --2- 172.21.15.45:0/1498284810 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561c20e95410 0x561c20e91560 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fbdc800c760 tx=0x7fbdc80a5b60 comp rx=0 tx=0).stop 2024-04-16T17:17:27.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.181+0000 7fbdda967740 1 -- 172.21.15.45:0/1498284810 shutdown_connections 2024-04-16T17:17:27.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.181+0000 7fbdda967740 1 --2- 172.21.15.45:0/1498284810 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561c20e8c830 0x561c20ea10c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:27.184 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.181+0000 7fbdda967740 1 --2- 172.21.15.45:0/1498284810 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561c20e95410 0x561c20e91560 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:27.184 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.181+0000 7fbdda967740 1 --2- 172.21.15.45:0/1498284810 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561c20e8bf10 0x561c20e94b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:27.184 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.181+0000 7fbdda967740 1 -- 172.21.15.45:0/1498284810 >> 172.21.15.45:0/1498284810 conn(0x561c20e86640 msgr2=0x561c20e87a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:17:27.184 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.181+0000 7fbdda967740 1 -- 172.21.15.45:0/1498284810 shutdown_connections 2024-04-16T17:17:27.184 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.181+0000 7fbdda967740 1 -- 172.21.15.45:0/1498284810 wait complete. 2024-04-16T17:17:27.184 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.182+0000 7fbdda967740 1 -- start start 2024-04-16T17:17:27.184 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.182+0000 7fbdda967740 1 Processor -- start 2024-04-16T17:17:27.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.182+0000 7fbdda967740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561c20e8bf10 0x561c20e9d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:27.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.183+0000 7fbdda967740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561c20e8c830 0x561c20e8ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:27.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.183+0000 7fbdda967740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561c20e95410 0x561c20e8f110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:27.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.183+0000 7fbdda967740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fbdd01afc20 con 0x561c20e8c830 2024-04-16T17:17:27.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.183+0000 7fbdda967740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fbdd01b1090 con 0x561c20e95410 2024-04-16T17:17:27.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.183+0000 7fbdda967740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fbdd01b02f0 con 0x561c20e8bf10 2024-04-16T17:17:27.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.183+0000 7fbdda967740 10 client.0 _ll_register_callbacks cb 0x7fbdc8099010 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:17:27.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.183+0000 7fbdcfe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561c20e8c830 0x561c20e8ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:27.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.183+0000 7fbdcfe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561c20e8c830 0x561c20e8ebd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45152/0 (socket says 172.21.15.45:45152) 2024-04-16T17:17:27.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.183+0000 7fbdcfe00640 1 -- 172.21.15.45:0/46170119 learned_addr learned my addr 172.21.15.45:0/46170119 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:17:27.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.183+0000 7fbdd5c00640 1 --2- 172.21.15.45:0/46170119 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561c20e95410 0x561c20e8f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:27.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.183+0000 7fbdd5200640 1 --2- 172.21.15.45:0/46170119 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561c20e8bf10 0x561c20e9d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:27.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.183+0000 7fbdd5c00640 1 -- 172.21.15.45:0/46170119 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561c20e8bf10 msgr2=0x561c20e9d7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:17:27.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.183+0000 7fbdd5c00640 1 --2- 172.21.15.45:0/46170119 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561c20e8bf10 0x561c20e9d7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:27.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.183+0000 7fbdd5c00640 1 -- 172.21.15.45:0/46170119 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561c20e8c830 msgr2=0x561c20e8ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:17:27.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.183+0000 7fbdd5c00640 1 --2- 172.21.15.45:0/46170119 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561c20e8c830 0x561c20e8ebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:27.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.183+0000 7fbdd5c00640 1 -- 172.21.15.45:0/46170119 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561c20bcd7d0 con 0x561c20e95410 2024-04-16T17:17:27.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.183+0000 7fbdcfe00640 1 --2- 172.21.15.45:0/46170119 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561c20e8c830 0x561c20e8ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:17:27.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.184+0000 7fbdd5c00640 1 --2- 172.21.15.45:0/46170119 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561c20e95410 0x561c20e8f110 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fbdc80a12e0 tx=0x7fbdc8009590 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a6c99fd8e316b65a server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:17:27.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.184+0000 7fbdb6a00640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-04-16T17:17:27.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.184+0000 7fbdb6a00640 1 -- 172.21.15.45:0/46170119 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdc80b6220 con 0x561c20e95410 2024-04-16T17:17:27.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.184+0000 7fbdda967740 1 -- 172.21.15.45:0/46170119 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbdc809c2c0 con 0x561c20e95410 2024-04-16T17:17:27.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.184+0000 7fbdb6a00640 1 -- 172.21.15.45:0/46170119 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9496+0+0 (secure 0 0 0) 0x7fbdc80a5550 con 0x561c20e95410 2024-04-16T17:17:27.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.184+0000 7fbdb6a00640 1 -- 172.21.15.45:0/46170119 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fbdc80a5790 con 0x561c20e95410 2024-04-16T17:17:27.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.184+0000 7fbdda967740 10 client.28307 fetch_fsmap learned FSMap version 44 2024-04-16T17:17:27.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.184+0000 7fbdda967740 1 -- 172.21.15.45:0/46170119 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x561c20f14070 con 0x561c20e95410 2024-04-16T17:17:27.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.184+0000 7fbdb6a00640 1 -- 172.21.15.45:0/46170119 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbdc809c2c0 con 0x561c20e95410 2024-04-16T17:17:27.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.185+0000 7fbdb6a00640 1 -- 172.21.15.45:0/46170119 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 44) ==== 8346+0+0 (secure 0 0 0) 0x7fbdc809be30 con 0x561c20e95410 2024-04-16T17:17:27.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.185+0000 7fbdda967740 10 client.28307 fetch_fsmap finished waiting for FSMap version 44 2024-04-16T17:17:27.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.185+0000 7fbdda967740 10 client.28307 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-16T17:17:27.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.185+0000 7fbdda967740 20 client.28307 populate_metadata read hostname 'smithi045' 2024-04-16T17:17:27.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.185+0000 7fbdda967740 1 --2- 172.21.15.45:0/46170119 >> [v2:172.21.15.117:6838/2378675949,v1:172.21.15.117:6839/2378675949] conn(0x561c20f14540 0x561c20f16930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:27.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.185+0000 7fbdda967740 4 client.28307 mds_command: new command op to 24532 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T17:17:27.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.185+0000 7fbdda967740 1 -- 172.21.15.45:0/46170119 --> [v2:172.21.15.117:6838/2378675949,v1:172.21.15.117:6839/2378675949] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561c20df88b0 con 0x561c20f14540 2024-04-16T17:17:27.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.185+0000 7fbdd5200640 1 --2- 172.21.15.45:0/46170119 >> [v2:172.21.15.117:6838/2378675949,v1:172.21.15.117:6839/2378675949] conn(0x561c20f14540 0x561c20f16930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:27.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.186+0000 7fbdd5200640 1 --2- 172.21.15.45:0/46170119 >> [v2:172.21.15.117:6838/2378675949,v1:172.21.15.117:6839/2378675949] conn(0x561c20f14540 0x561c20f16930 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6cb8160e208141cf server_cookie=26d24b8dd573ea85 in_seq=0 out_seq=0 2024-04-16T17:17:27.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.186+0000 7fbdb6a00640 10 client.28307 ms_handle_connect on v2:172.21.15.117:6838/2378675949 2024-04-16T17:17:27.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.186+0000 7fbdb6a00640 1 -- 172.21.15.45:0/46170119 <== mds.2 v2:172.21.15.117:6838/2378675949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561c20df88b0 con 0x561c20f14540 2024-04-16T17:17:27.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.186+0000 7fbdb6a00640 10 client.28307 handle_command_reply: tid=0 2024-04-16T17:17:27.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.186+0000 7fbdda967740 1 client.28307 shutdown 2024-04-16T17:17:27.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.186+0000 7fbdda967740 1 -- 172.21.15.45:0/46170119 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561c20e95410 msgr2=0x561c20e8f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:17:27.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.186+0000 7fbdda967740 1 --2- 172.21.15.45:0/46170119 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561c20e95410 0x561c20e8f110 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fbdc80a12e0 tx=0x7fbdc8009590 comp rx=0 tx=0).stop 2024-04-16T17:17:27.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.186+0000 7fbdda967740 1 -- 172.21.15.45:0/46170119 shutdown_connections 2024-04-16T17:17:27.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.186+0000 7fbdda967740 1 --2- 172.21.15.45:0/46170119 >> [v2:172.21.15.117:6838/2378675949,v1:172.21.15.117:6839/2378675949] conn(0x561c20f14540 0x561c20f16930 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:27.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.186+0000 7fbdda967740 1 --2- 172.21.15.45:0/46170119 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561c20e95410 0x561c20e8f110 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:27.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.186+0000 7fbdda967740 1 --2- 172.21.15.45:0/46170119 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561c20e8c830 0x561c20e8ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:27.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.186+0000 7fbdda967740 1 --2- 172.21.15.45:0/46170119 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561c20e8bf10 0x561c20e9d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:27.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.186+0000 7fbdb6a00640 0 client.28307 ms_handle_reset on v2:172.21.15.117:6838/2378675949 2024-04-16T17:17:27.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.186+0000 7fbdda967740 1 -- 172.21.15.45:0/46170119 >> 172.21.15.45:0/46170119 conn(0x561c20e86640 msgr2=0x561c20e87a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:17:27.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.186+0000 7fbdda967740 1 -- 172.21.15.45:0/46170119 shutdown_connections 2024-04-16T17:17:27.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.186+0000 7fbdda967740 1 -- 172.21.15.45:0/46170119 wait complete. 2024-04-16T17:17:27.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.186+0000 7fbdda967740 20 client.28307 trim_cache size 0 max 16384 2024-04-16T17:17:27.194 INFO:teuthology.orchestra.run.smithi045.stdout:[] 2024-04-16T17:17:27.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.187+0000 7fbdd6600640 1 -- 172.21.15.45:0/2501177050 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fbdb80a3da0 msgr2=0x7fbdb80a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:17:27.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.187+0000 7fbdd6600640 1 --2- 172.21.15.45:0/2501177050 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fbdb80a3da0 0x7fbdb80a6260 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fbdbc009830 tx=0x7fbdbc0058d0 comp rx=0 tx=0).stop 2024-04-16T17:17:27.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.187+0000 7fbdd6600640 1 -- 172.21.15.45:0/2501177050 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbdd019a7f0 msgr2=0x7fbdd019aca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:17:27.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.187+0000 7fbdd6600640 1 --2- 172.21.15.45:0/2501177050 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbdd019a7f0 0x7fbdd019aca0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fbdc8009fd0 tx=0x7fbdc800ef10 comp rx=0 tx=0).stop 2024-04-16T17:17:27.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.187+0000 7fbdd6600640 1 -- 172.21.15.45:0/2501177050 shutdown_connections 2024-04-16T17:17:27.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.187+0000 7fbdd6600640 1 --2- 172.21.15.45:0/2501177050 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fbdb80a3da0 0x7fbdb80a6260 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:27.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.187+0000 7fbdd6600640 1 --2- 172.21.15.45:0/2501177050 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbdd01093e0 0x7fbdd01a0b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:27.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.187+0000 7fbdd6600640 1 --2- 172.21.15.45:0/2501177050 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbdd019a7f0 0x7fbdd019aca0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:27.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.187+0000 7fbdd6600640 1 --2- 172.21.15.45:0/2501177050 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbdd0104310 0x7fbdd01a0630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:27.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.187+0000 7fbdd6600640 1 -- 172.21.15.45:0/2501177050 >> 172.21.15.45:0/2501177050 conn(0x7fbdd00ffb10 msgr2=0x7fbdd0101e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:17:27.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.187+0000 7fbdd6600640 1 -- 172.21.15.45:0/2501177050 shutdown_connections 2024-04-16T17:17:27.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.187+0000 7fbdd6600640 1 -- 172.21.15.45:0/2501177050 wait complete. 2024-04-16T17:17:27.207 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-16T17:17:27.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.433+0000 7f4dd6400640 1 -- 172.21.15.45:0/1231398769 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4dd006a870 msgr2=0x7f4dd006ac50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:17:27.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.433+0000 7f4dd6400640 1 --2- 172.21.15.45:0/1231398769 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4dd006a870 0x7f4dd006ac50 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f4dc0009900 tx=0x7f4dc002bc30 comp rx=0 tx=0).stop 2024-04-16T17:17:27.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.433+0000 7f4dd6400640 1 -- 172.21.15.45:0/1231398769 shutdown_connections 2024-04-16T17:17:27.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.433+0000 7f4dd6400640 1 --2- 172.21.15.45:0/1231398769 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4dd0107330 0x7f4dd010eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:27.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.433+0000 7f4dd6400640 1 --2- 172.21.15.45:0/1231398769 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4dd006b220 0x7f4dd0106c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:27.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.433+0000 7f4dd6400640 1 --2- 172.21.15.45:0/1231398769 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4dd006a870 0x7f4dd006ac50 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:27.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.433+0000 7f4dd6400640 1 -- 172.21.15.45:0/1231398769 >> 172.21.15.45:0/1231398769 conn(0x7f4dd0076d00 msgr2=0x7f4dd0077110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:17:27.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.433+0000 7f4dd6400640 1 -- 172.21.15.45:0/1231398769 shutdown_connections 2024-04-16T17:17:27.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.433+0000 7f4dd6400640 1 -- 172.21.15.45:0/1231398769 wait complete. 2024-04-16T17:17:27.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.434+0000 7f4dd6400640 1 Processor -- start 2024-04-16T17:17:27.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.435+0000 7f4dd6400640 1 -- start start 2024-04-16T17:17:27.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.435+0000 7f4dd6400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4dd006a870 0x7f4dd01a06f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:27.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.435+0000 7f4dd6400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4dd0107330 0x7f4dd01a0c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:27.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.435+0000 7f4dd6400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4dd019a8f0 0x7f4dd019ada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:27.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.435+0000 7f4dd6400640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f4dd0112430 con 0x7f4dd019a8f0 2024-04-16T17:17:27.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.435+0000 7f4dd6400640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f4dd01122b0 con 0x7f4dd006a870 2024-04-16T17:17:27.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.435+0000 7f4dd6400640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f4dd01125b0 con 0x7f4dd0107330 2024-04-16T17:17:27.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.435+0000 7f4dcfe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4dd019a8f0 0x7f4dd019ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:27.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.435+0000 7f4dcf400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4dd006a870 0x7f4dd01a06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:27.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.435+0000 7f4dcea00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4dd0107330 0x7f4dd01a0c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:27.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.435+0000 7f4dcf400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4dd006a870 0x7f4dd01a06f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:43738/0 (socket says 172.21.15.45:43738) 2024-04-16T17:17:27.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.435+0000 7f4dcf400640 1 -- 172.21.15.45:0/2864735300 learned_addr learned my addr 172.21.15.45:0/2864735300 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:17:27.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.436+0000 7f4dcf400640 1 -- 172.21.15.45:0/2864735300 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4dd0107330 msgr2=0x7f4dd01a0c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:17:27.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.436+0000 7f4dcf400640 1 --2- 172.21.15.45:0/2864735300 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4dd0107330 0x7f4dd01a0c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:27.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.436+0000 7f4dcf400640 1 -- 172.21.15.45:0/2864735300 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4dd019a8f0 msgr2=0x7f4dd019ada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:17:27.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.436+0000 7f4dcf400640 1 --2- 172.21.15.45:0/2864735300 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4dd019a8f0 0x7f4dd019ada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:27.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.436+0000 7f4dcf400640 1 -- 172.21.15.45:0/2864735300 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4dd019b410 con 0x7f4dd006a870 2024-04-16T17:17:27.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.436+0000 7f4dcf400640 1 --2- 172.21.15.45:0/2864735300 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4dd006a870 0x7f4dd01a06f0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f4dc002c240 tx=0x7f4dc00042f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:17:27.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.437+0000 7f4dc7e00640 1 -- 172.21.15.45:0/2864735300 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dc0043020 con 0x7f4dd006a870 2024-04-16T17:17:27.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.437+0000 7f4dc7e00640 1 -- 172.21.15.45:0/2864735300 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4dc00054e0 con 0x7f4dd006a870 2024-04-16T17:17:27.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.437+0000 7f4dd6400640 1 -- 172.21.15.45:0/2864735300 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4dd019b6a0 con 0x7f4dd006a870 2024-04-16T17:17:27.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.437+0000 7f4dc7e00640 1 -- 172.21.15.45:0/2864735300 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dc0003b20 con 0x7f4dd006a870 2024-04-16T17:17:27.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.437+0000 7f4dd6400640 1 -- 172.21.15.45:0/2864735300 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4dd01a78a0 con 0x7f4dd006a870 2024-04-16T17:17:27.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.441+0000 7f4dd8ffc740 1 Processor -- start 2024-04-16T17:17:27.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.441+0000 7f4dc7e00640 1 -- 172.21.15.45:0/2864735300 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f4dc0046020 con 0x7f4dd006a870 2024-04-16T17:17:27.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.441+0000 7f4dd8ffc740 1 -- start start 2024-04-16T17:17:27.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.441+0000 7f4dd8ffc740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fa43599d50 0x55fa4359a130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:27.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.441+0000 7f4dd8ffc740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fa4359a670 0x55fa435a2d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:27.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.441+0000 7f4dd8ffc740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fa435a3250 0x55fa435a5640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:27.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.441+0000 7f4dd8ffc740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55fa434ff020 con 0x55fa4359a670 2024-04-16T17:17:27.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.442+0000 7f4dd8ffc740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55fa434a5460 con 0x55fa43599d50 2024-04-16T17:17:27.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.442+0000 7f4dd8ffc740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55fa434fafe0 con 0x55fa435a3250 2024-04-16T17:17:27.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.442+0000 7f4dcfe00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fa43599d50 0x55fa4359a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:27.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.442+0000 7f4dcfe00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fa43599d50 0x55fa4359a130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:43744/0 (socket says 172.21.15.45:43744) 2024-04-16T17:17:27.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.442+0000 7f4dcf400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fa4359a670 0x55fa435a2d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:27.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.442+0000 7f4dcfe00640 1 -- 172.21.15.45:0/3027954374 learned_addr learned my addr 172.21.15.45:0/3027954374 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:17:27.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.442+0000 7f4dcea00640 1 --2- 172.21.15.45:0/3027954374 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fa435a3250 0x55fa435a5640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:27.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.442+0000 7f4dcfe00640 1 -- 172.21.15.45:0/3027954374 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fa435a3250 msgr2=0x55fa435a5640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:17:27.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.442+0000 7f4dcfe00640 1 --2- 172.21.15.45:0/3027954374 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fa435a3250 0x55fa435a5640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:27.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.442+0000 7f4dcfe00640 1 -- 172.21.15.45:0/3027954374 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fa4359a670 msgr2=0x55fa435a2d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:17:27.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.442+0000 7f4dcfe00640 1 --2- 172.21.15.45:0/3027954374 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fa4359a670 0x55fa435a2d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:27.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.442+0000 7f4dcfe00640 1 -- 172.21.15.45:0/3027954374 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fa434ad040 con 0x55fa43599d50 2024-04-16T17:17:27.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.443+0000 7f4dc7e00640 1 --2- 172.21.15.45:0/2864735300 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f4db40a3d20 0x7f4db40a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:27.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.443+0000 7f4dcfe00640 1 --2- 172.21.15.45:0/3027954374 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fa43599d50 0x55fa4359a130 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f4dc8009fd0 tx=0x7f4dc80118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ed68627b0a952e36 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:17:27.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.443+0000 7f4dc7e00640 1 -- 172.21.15.45:0/2864735300 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 9496+0+0 (secure 0 0 0) 0x7f4dc00f2a90 con 0x7f4dd006a870 2024-04-16T17:17:27.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.443+0000 7f4dcf400640 1 --2- 172.21.15.45:0/2864735300 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f4db40a3d20 0x7f4db40a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:27.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.443+0000 7f4dc4c00640 1 -- 172.21.15.45:0/3027954374 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dc8004d20 con 0x55fa43599d50 2024-04-16T17:17:27.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.443+0000 7f4dc4c00640 1 -- 172.21.15.45:0/3027954374 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4dc8004050 con 0x55fa43599d50 2024-04-16T17:17:27.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.443+0000 7f4dc4c00640 1 -- 172.21.15.45:0/3027954374 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dc8005240 con 0x55fa43599d50 2024-04-16T17:17:27.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.444+0000 7f4dd8ffc740 1 -- 172.21.15.45:0/3027954374 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fa43599d50 msgr2=0x55fa4359a130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:17:27.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.444+0000 7f4dcf400640 1 --2- 172.21.15.45:0/2864735300 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f4db40a3d20 0x7f4db40a61e0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x55fa43599820 tx=0x7f4dc00f6540 comp rx=0 tx=0).ready entity=mgr.25411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:17:27.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.444+0000 7f4dd8ffc740 1 --2- 172.21.15.45:0/3027954374 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fa43599d50 0x55fa4359a130 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f4dc8009fd0 tx=0x7f4dc80118d0 comp rx=0 tx=0).stop 2024-04-16T17:17:27.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.444+0000 7f4dd8ffc740 1 -- 172.21.15.45:0/3027954374 shutdown_connections 2024-04-16T17:17:27.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.444+0000 7f4dd8ffc740 1 --2- 172.21.15.45:0/3027954374 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fa435a3250 0x55fa435a5640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:27.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.444+0000 7f4dd8ffc740 1 --2- 172.21.15.45:0/3027954374 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fa4359a670 0x55fa435a2d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:27.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.444+0000 7f4dd8ffc740 1 --2- 172.21.15.45:0/3027954374 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fa43599d50 0x55fa4359a130 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:27.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.444+0000 7f4dd8ffc740 1 -- 172.21.15.45:0/3027954374 >> 172.21.15.45:0/3027954374 conn(0x55fa43594480 msgr2=0x55fa43594860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:17:27.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.444+0000 7f4dd8ffc740 1 -- 172.21.15.45:0/3027954374 shutdown_connections 2024-04-16T17:17:27.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.444+0000 7f4dd8ffc740 1 -- 172.21.15.45:0/3027954374 wait complete. 2024-04-16T17:17:27.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.445+0000 7f4dd8ffc740 1 -- start start 2024-04-16T17:17:27.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.445+0000 7f4dd8ffc740 1 Processor -- start 2024-04-16T17:17:27.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.446+0000 7f4dd8ffc740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fa43599d50 0x55fa435a02a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:27.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.446+0000 7f4dd8ffc740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fa4359a670 0x55fa435a07e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:27.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.446+0000 7f4dd8ffc740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fa435a3250 0x55fa435a0d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:27.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.446+0000 7f4dd8ffc740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55fa435a6040 con 0x55fa43599d50 2024-04-16T17:17:27.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.446+0000 7f4dd8ffc740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55fa434b8830 con 0x55fa435a3250 2024-04-16T17:17:27.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.446+0000 7f4dd8ffc740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55fa435a61c0 con 0x55fa4359a670 2024-04-16T17:17:27.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.446+0000 7f4dd8ffc740 10 client.0 _ll_register_callbacks cb 0x55fa434c0150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:17:27.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.446+0000 7f4dcfe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fa43599d50 0x55fa435a02a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:27.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.446+0000 7f4dcea00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fa4359a670 0x55fa435a07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:27.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.446+0000 7f4dcfe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fa43599d50 0x55fa435a02a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45188/0 (socket says 172.21.15.45:45188) 2024-04-16T17:17:27.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.446+0000 7f4dcea00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fa4359a670 0x55fa435a07e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:35866/0 (socket says 172.21.15.45:35866) 2024-04-16T17:17:27.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.446+0000 7f4dcfe00640 1 -- 172.21.15.45:0/99797004 learned_addr learned my addr 172.21.15.45:0/99797004 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:17:27.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.446+0000 7f4dcfe00640 1 --2- 172.21.15.45:0/99797004 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fa435a3250 0x55fa435a0d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:27.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.446+0000 7f4dcea00640 1 -- 172.21.15.45:0/99797004 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fa435a3250 msgr2=0x55fa435a0d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:17:27.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.446+0000 7f4dcea00640 1 --2- 172.21.15.45:0/99797004 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fa435a3250 0x55fa435a0d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:27.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.446+0000 7f4dcea00640 1 -- 172.21.15.45:0/99797004 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fa43599d50 msgr2=0x55fa435a02a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:17:27.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.446+0000 7f4dcea00640 1 --2- 172.21.15.45:0/99797004 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fa43599d50 0x55fa435a02a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:27.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.446+0000 7f4dcea00640 1 -- 172.21.15.45:0/99797004 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fa434bc920 con 0x55fa4359a670 2024-04-16T17:17:27.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.447+0000 7f4dcea00640 1 --2- 172.21.15.45:0/99797004 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fa4359a670 0x55fa435a07e0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f4dd019c090 tx=0x7f4dbc012b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=733fecf906bfa970 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:17:27.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.447+0000 7f4db9600640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-04-16T17:17:27.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.447+0000 7f4db9600640 1 -- 172.21.15.45:0/99797004 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dbc004020 con 0x55fa4359a670 2024-04-16T17:17:27.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.447+0000 7f4dd8ffc740 1 -- 172.21.15.45:0/99797004 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4dc8004050 con 0x55fa4359a670 2024-04-16T17:17:27.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.447+0000 7f4db9600640 1 -- 172.21.15.45:0/99797004 <== mon.1 v2:172.21.15.117:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9496+0+0 (secure 0 0 0) 0x7f4dbc0041c0 con 0x55fa4359a670 2024-04-16T17:17:27.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.447+0000 7f4db9600640 1 -- 172.21.15.45:0/99797004 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f4dbc025050 con 0x55fa4359a670 2024-04-16T17:17:27.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.447+0000 7f4dd8ffc740 10 client.28270 fetch_fsmap learned FSMap version 44 2024-04-16T17:17:27.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.447+0000 7f4dd8ffc740 1 -- 172.21.15.45:0/99797004 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55fa4348a7f0 con 0x55fa4359a670 2024-04-16T17:17:27.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.448+0000 7f4db9600640 1 -- 172.21.15.45:0/99797004 <== mon.1 v2:172.21.15.117:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4dc8004050 con 0x55fa4359a670 2024-04-16T17:17:27.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.448+0000 7f4db9600640 1 -- 172.21.15.45:0/99797004 <== mon.1 v2:172.21.15.117:3300/0 5 ==== fsmap(e 44) ==== 8346+0+0 (secure 0 0 0) 0x7f4dbc003a60 con 0x55fa4359a670 2024-04-16T17:17:27.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.448+0000 7f4dd8ffc740 10 client.28270 fetch_fsmap finished waiting for FSMap version 44 2024-04-16T17:17:27.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.448+0000 7f4dd8ffc740 10 client.28270 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-04-16T17:17:27.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.448+0000 7f4dd8ffc740 20 client.28270 populate_metadata read hostname 'smithi045' 2024-04-16T17:17:27.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.448+0000 7f4dd8ffc740 1 --2- 172.21.15.45:0/99797004 >> [v2:172.21.15.45:6840/3755310822,v1:172.21.15.45:6841/3755310822] conn(0x55fa435b35f0 0x55fa435b5a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:27.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.448+0000 7f4dd8ffc740 4 client.28270 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:17:27.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.448+0000 7f4dd8ffc740 1 -- 172.21.15.45:0/99797004 --> [v2:172.21.15.45:6840/3755310822,v1:172.21.15.45:6841/3755310822] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fa4350ca40 con 0x55fa435b35f0 2024-04-16T17:17:27.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.448+0000 7f4dcfe00640 1 --2- 172.21.15.45:0/99797004 >> [v2:172.21.15.45:6840/3755310822,v1:172.21.15.45:6841/3755310822] conn(0x55fa435b35f0 0x55fa435b5a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:27.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.449+0000 7f4dcfe00640 1 --2- 172.21.15.45:0/99797004 >> [v2:172.21.15.45:6840/3755310822,v1:172.21.15.45:6841/3755310822] conn(0x55fa435b35f0 0x55fa435b5a00 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ee6ced96a91477b4 server_cookie=960d85534b24aa19 in_seq=0 out_seq=0 2024-04-16T17:17:27.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.449+0000 7f4db9600640 10 client.28270 ms_handle_connect on v2:172.21.15.45:6840/3755310822 2024-04-16T17:17:27.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.450+0000 7f4db9600640 1 -- 172.21.15.45:0/99797004 <== mds.1 v2:172.21.15.45:6840/3755310822 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55fa4350ca40 con 0x55fa435b35f0 2024-04-16T17:17:27.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.450+0000 7f4db9600640 10 client.28270 handle_command_reply: tid=0 2024-04-16T17:17:27.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.450+0000 7f4dd8ffc740 1 client.28270 shutdown 2024-04-16T17:17:27.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.450+0000 7f4dd8ffc740 1 -- 172.21.15.45:0/99797004 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fa4359a670 msgr2=0x55fa435a07e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:17:27.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.450+0000 7f4dd8ffc740 1 --2- 172.21.15.45:0/99797004 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fa4359a670 0x55fa435a07e0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f4dd019c090 tx=0x7f4dbc012b60 comp rx=0 tx=0).stop 2024-04-16T17:17:27.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.450+0000 7f4dd8ffc740 1 -- 172.21.15.45:0/99797004 shutdown_connections 2024-04-16T17:17:27.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.450+0000 7f4dd8ffc740 1 --2- 172.21.15.45:0/99797004 >> [v2:172.21.15.45:6840/3755310822,v1:172.21.15.45:6841/3755310822] conn(0x55fa435b35f0 0x55fa435b5a00 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:27.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.450+0000 7f4dd8ffc740 1 --2- 172.21.15.45:0/99797004 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fa4359a670 0x55fa435a07e0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:27.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.450+0000 7f4dd8ffc740 1 --2- 172.21.15.45:0/99797004 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fa435a3250 0x55fa435a0d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:27.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.450+0000 7f4dd8ffc740 1 --2- 172.21.15.45:0/99797004 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fa43599d50 0x55fa435a02a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:27.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.450+0000 7f4dd8ffc740 1 -- 172.21.15.45:0/99797004 >> 172.21.15.45:0/99797004 conn(0x55fa43594480 msgr2=0x55fa435961d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:17:27.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.450+0000 7f4db9600640 0 client.28270 ms_handle_reset on v2:172.21.15.45:6840/3755310822 2024-04-16T17:17:27.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.450+0000 7f4dd8ffc740 1 -- 172.21.15.45:0/99797004 shutdown_connections 2024-04-16T17:17:27.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.450+0000 7f4dd8ffc740 1 -- 172.21.15.45:0/99797004 wait complete. 2024-04-16T17:17:27.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.450+0000 7f4dd8ffc740 20 client.28270 trim_cache size 0 max 16384 2024-04-16T17:17:27.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.466+0000 7f4dd8ffc740 1 Processor -- start 2024-04-16T17:17:27.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.466+0000 7f4dd8ffc740 1 -- start start 2024-04-16T17:17:27.466 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.466+0000 7f4dd8ffc740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fa43599d50 0x55fa435a29d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:27.466 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.466+0000 7f4dd8ffc740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fa4359a670 0x55fa435aef00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:27.466 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.466+0000 7f4dd8ffc740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fa435a3250 0x55fa435a0970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:27.466 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.466+0000 7f4dd8ffc740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f4dd01b4090 con 0x55fa435a3250 2024-04-16T17:17:27.466 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.466+0000 7f4dd8ffc740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f4dd01b1960 con 0x55fa4359a670 2024-04-16T17:17:27.466 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.466+0000 7f4dd8ffc740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f4dd01af7d0 con 0x55fa43599d50 2024-04-16T17:17:27.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.467+0000 7f4dcea00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fa4359a670 0x55fa435aef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:27.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.467+0000 7f4dcfe00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fa43599d50 0x55fa435a29d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:27.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.467+0000 7f4dcea00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fa4359a670 0x55fa435aef00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:43756/0 (socket says 172.21.15.45:43756) 2024-04-16T17:17:27.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.467+0000 7f4dcea00640 1 -- 172.21.15.45:0/1204891069 learned_addr learned my addr 172.21.15.45:0/1204891069 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:17:27.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.467+0000 7f4dcfe00640 1 --2- 172.21.15.45:0/1204891069 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fa435a3250 0x55fa435a0970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:27.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.467+0000 7f4dcea00640 1 -- 172.21.15.45:0/1204891069 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fa43599d50 msgr2=0x55fa435a29d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:17:27.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.467+0000 7f4dcea00640 1 --2- 172.21.15.45:0/1204891069 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fa43599d50 0x55fa435a29d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:27.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.467+0000 7f4dcea00640 1 -- 172.21.15.45:0/1204891069 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fa435a3250 msgr2=0x55fa435a0970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:17:27.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.467+0000 7f4dcea00640 1 --2- 172.21.15.45:0/1204891069 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fa435a3250 0x55fa435a0970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:27.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.467+0000 7f4dcea00640 1 -- 172.21.15.45:0/1204891069 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fa4359a4a0 con 0x55fa4359a670 2024-04-16T17:17:27.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.467+0000 7f4dcfe00640 1 --2- 172.21.15.45:0/1204891069 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fa43599d50 0x55fa435a29d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:17:27.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.467+0000 7f4dcea00640 1 --2- 172.21.15.45:0/1204891069 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fa4359a670 0x55fa435aef00 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f4dbc003e30 tx=0x7f4dbc009620 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a2e0774a71f00da6 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:17:27.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.468+0000 7f4dbbe00640 1 -- 172.21.15.45:0/1204891069 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dbc010580 con 0x55fa4359a670 2024-04-16T17:17:27.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.468+0000 7f4dbbe00640 1 -- 172.21.15.45:0/1204891069 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4dbc010e00 con 0x55fa4359a670 2024-04-16T17:17:27.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.468+0000 7f4dbbe00640 1 -- 172.21.15.45:0/1204891069 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dbc003850 con 0x55fa4359a670 2024-04-16T17:17:27.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.469+0000 7f4dd8ffc740 1 -- 172.21.15.45:0/1204891069 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fa4359a670 msgr2=0x55fa435aef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:17:27.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.469+0000 7f4dd8ffc740 1 --2- 172.21.15.45:0/1204891069 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fa4359a670 0x55fa435aef00 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f4dbc003e30 tx=0x7f4dbc009620 comp rx=0 tx=0).stop 2024-04-16T17:17:27.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.469+0000 7f4dd8ffc740 1 -- 172.21.15.45:0/1204891069 shutdown_connections 2024-04-16T17:17:27.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.469+0000 7f4dd8ffc740 1 --2- 172.21.15.45:0/1204891069 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fa435a3250 0x55fa435a0970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:27.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.469+0000 7f4dd8ffc740 1 --2- 172.21.15.45:0/1204891069 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fa4359a670 0x55fa435aef00 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:27.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.469+0000 7f4dd8ffc740 1 --2- 172.21.15.45:0/1204891069 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fa43599d50 0x55fa435a29d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:27.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.469+0000 7f4dd8ffc740 1 -- 172.21.15.45:0/1204891069 >> 172.21.15.45:0/1204891069 conn(0x55fa43594480 msgr2=0x55fa435a0260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:17:27.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.469+0000 7f4dd8ffc740 1 -- 172.21.15.45:0/1204891069 shutdown_connections 2024-04-16T17:17:27.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.469+0000 7f4dd8ffc740 1 -- 172.21.15.45:0/1204891069 wait complete. 2024-04-16T17:17:27.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.469+0000 7f4dd8ffc740 1 -- start start 2024-04-16T17:17:27.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.469+0000 7f4dd8ffc740 1 Processor -- start 2024-04-16T17:17:27.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.470+0000 7f4dd8ffc740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fa43599d50 0x55fa435ab8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:27.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.470+0000 7f4dd8ffc740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fa4359a670 0x55fa4359cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:27.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.470+0000 7f4dd8ffc740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fa435a3250 0x55fa4359d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:27.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.470+0000 7f4dd8ffc740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f4dd01afe20 con 0x55fa435a3250 2024-04-16T17:17:27.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.470+0000 7f4dd8ffc740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f4dd01b1290 con 0x55fa4359a670 2024-04-16T17:17:27.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.470+0000 7f4dd8ffc740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f4dd01b04f0 con 0x55fa43599d50 2024-04-16T17:17:27.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.470+0000 7f4dd8ffc740 10 client.0 _ll_register_callbacks cb 0x7f4dbc009910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:17:27.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.470+0000 7f4dcea00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fa4359a670 0x55fa4359cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:27.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.470+0000 7f4dcfe00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fa43599d50 0x55fa435ab8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:27.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.470+0000 7f4dcea00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fa4359a670 0x55fa4359cdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:43772/0 (socket says 172.21.15.45:43772) 2024-04-16T17:17:27.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.470+0000 7f4dcea00640 1 -- 172.21.15.45:0/1767783854 learned_addr learned my addr 172.21.15.45:0/1767783854 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:17:27.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.470+0000 7f4dcfe00640 1 --2- 172.21.15.45:0/1767783854 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fa435a3250 0x55fa4359d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:27.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.471+0000 7f4dcea00640 1 -- 172.21.15.45:0/1767783854 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fa43599d50 msgr2=0x55fa435ab8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:17:27.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.471+0000 7f4dcea00640 1 --2- 172.21.15.45:0/1767783854 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fa43599d50 0x55fa435ab8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:27.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.471+0000 7f4dcea00640 1 -- 172.21.15.45:0/1767783854 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fa435a3250 msgr2=0x55fa4359d3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:17:27.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.471+0000 7f4dcea00640 1 --2- 172.21.15.45:0/1767783854 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fa435a3250 0x55fa4359d3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:27.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.471+0000 7f4dcea00640 1 -- 172.21.15.45:0/1767783854 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fa432113f0 con 0x55fa4359a670 2024-04-16T17:17:27.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.471+0000 7f4dcfe00640 1 --2- 172.21.15.45:0/1767783854 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fa435a3250 0x55fa4359d3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-16T17:17:27.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.471+0000 7f4dcea00640 1 --2- 172.21.15.45:0/1767783854 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fa4359a670 0x55fa4359cdd0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f4dbc012ba0 tx=0x7f4dbc0107b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5c7373f0cdb53c18 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:17:27.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.471+0000 7f4db9600640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-04-16T17:17:27.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.471+0000 7f4db9600640 1 -- 172.21.15.45:0/1767783854 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dbc00fb00 con 0x55fa4359a670 2024-04-16T17:17:27.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.471+0000 7f4db9600640 1 -- 172.21.15.45:0/1767783854 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9496+0+0 (secure 0 0 0) 0x7f4dbc0035f0 con 0x55fa4359a670 2024-04-16T17:17:27.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.471+0000 7f4dd8ffc740 1 -- 172.21.15.45:0/1767783854 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4dbc010e00 con 0x55fa4359a670 2024-04-16T17:17:27.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.472+0000 7f4db9600640 1 -- 172.21.15.45:0/1767783854 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f4dbc03b050 con 0x55fa4359a670 2024-04-16T17:17:27.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.472+0000 7f4dd8ffc740 10 client.28325 fetch_fsmap learned FSMap version 44 2024-04-16T17:17:27.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.472+0000 7f4dd8ffc740 1 -- 172.21.15.45:0/1767783854 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55fa432db7d0 con 0x55fa4359a670 2024-04-16T17:17:27.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.472+0000 7f4db9600640 1 -- 172.21.15.45:0/1767783854 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4dbc010e00 con 0x55fa4359a670 2024-04-16T17:17:27.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.472+0000 7f4db9600640 1 -- 172.21.15.45:0/1767783854 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 44) ==== 8346+0+0 (secure 0 0 0) 0x7f4dbc028020 con 0x55fa4359a670 2024-04-16T17:17:27.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.472+0000 7f4dd8ffc740 10 client.28325 fetch_fsmap finished waiting for FSMap version 44 2024-04-16T17:17:27.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.472+0000 7f4dd8ffc740 10 client.28325 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-04-16T17:17:27.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.472+0000 7f4dd8ffc740 20 client.28325 populate_metadata read hostname 'smithi045' 2024-04-16T17:17:27.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.473+0000 7f4dd8ffc740 1 --2- 172.21.15.45:0/1767783854 >> [v2:172.21.15.45:6840/3755310822,v1:172.21.15.45:6841/3755310822] conn(0x55fa43619e60 0x55fa4361c280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:27.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.473+0000 7f4dd8ffc740 4 client.28325 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T17:17:27.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.473+0000 7f4dd8ffc740 1 -- 172.21.15.45:0/1767783854 --> [v2:172.21.15.45:6840/3755310822,v1:172.21.15.45:6841/3755310822] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55fa435068b0 con 0x55fa43619e60 2024-04-16T17:17:27.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.473+0000 7f4dcfe00640 1 --2- 172.21.15.45:0/1767783854 >> [v2:172.21.15.45:6840/3755310822,v1:172.21.15.45:6841/3755310822] conn(0x55fa43619e60 0x55fa4361c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:27.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.473+0000 7f4dcfe00640 1 --2- 172.21.15.45:0/1767783854 >> [v2:172.21.15.45:6840/3755310822,v1:172.21.15.45:6841/3755310822] conn(0x55fa43619e60 0x55fa4361c280 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=df79b0095e4534f7 server_cookie=699ef0b2dd1ee154 in_seq=0 out_seq=0 2024-04-16T17:17:27.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.473+0000 7f4db9600640 10 client.28325 ms_handle_connect on v2:172.21.15.45:6840/3755310822 2024-04-16T17:17:27.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.474+0000 7f4db9600640 1 -- 172.21.15.45:0/1767783854 <== mds.1 v2:172.21.15.45:6840/3755310822 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55fa435068b0 con 0x55fa43619e60 2024-04-16T17:17:27.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.474+0000 7f4db9600640 10 client.28325 handle_command_reply: tid=0 2024-04-16T17:17:27.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.474+0000 7f4dd8ffc740 1 client.28325 shutdown 2024-04-16T17:17:27.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.474+0000 7f4dd8ffc740 1 -- 172.21.15.45:0/1767783854 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fa4359a670 msgr2=0x55fa4359cdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:17:27.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.474+0000 7f4dd8ffc740 1 --2- 172.21.15.45:0/1767783854 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fa4359a670 0x55fa4359cdd0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f4dbc012ba0 tx=0x7f4dbc0107b0 comp rx=0 tx=0).stop 2024-04-16T17:17:27.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.474+0000 7f4dd8ffc740 1 -- 172.21.15.45:0/1767783854 shutdown_connections 2024-04-16T17:17:27.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.474+0000 7f4dd8ffc740 1 --2- 172.21.15.45:0/1767783854 >> [v2:172.21.15.45:6840/3755310822,v1:172.21.15.45:6841/3755310822] conn(0x55fa43619e60 0x55fa4361c280 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:27.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.474+0000 7f4dd8ffc740 1 --2- 172.21.15.45:0/1767783854 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fa435a3250 0x55fa4359d3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:27.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.474+0000 7f4dd8ffc740 1 --2- 172.21.15.45:0/1767783854 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fa4359a670 0x55fa4359cdd0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:27.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.474+0000 7f4db9600640 0 client.28325 ms_handle_reset on v2:172.21.15.45:6840/3755310822 2024-04-16T17:17:27.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.474+0000 7f4dd8ffc740 1 --2- 172.21.15.45:0/1767783854 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fa43599d50 0x55fa435ab8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:27.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.474+0000 7f4dd8ffc740 1 -- 172.21.15.45:0/1767783854 >> 172.21.15.45:0/1767783854 conn(0x55fa43594480 msgr2=0x55fa435a0af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:17:27.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.474+0000 7f4dd8ffc740 1 -- 172.21.15.45:0/1767783854 shutdown_connections 2024-04-16T17:17:27.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.474+0000 7f4dd8ffc740 1 -- 172.21.15.45:0/1767783854 wait complete. 2024-04-16T17:17:27.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.474+0000 7f4dd8ffc740 20 client.28325 trim_cache size 0 max 16384 2024-04-16T17:17:27.480 INFO:teuthology.orchestra.run.smithi045.stdout:[] 2024-04-16T17:17:27.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.475+0000 7f4dd6400640 1 -- 172.21.15.45:0/2864735300 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f4db40a3d20 msgr2=0x7f4db40a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:17:27.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.475+0000 7f4dd6400640 1 --2- 172.21.15.45:0/2864735300 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f4db40a3d20 0x7f4db40a61e0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x55fa43599820 tx=0x7f4dc00f6540 comp rx=0 tx=0).stop 2024-04-16T17:17:27.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.475+0000 7f4dd6400640 1 -- 172.21.15.45:0/2864735300 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4dd006a870 msgr2=0x7f4dd01a06f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:17:27.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.475+0000 7f4dd6400640 1 --2- 172.21.15.45:0/2864735300 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4dd006a870 0x7f4dd01a06f0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f4dc002c240 tx=0x7f4dc00042f0 comp rx=0 tx=0).stop 2024-04-16T17:17:27.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.475+0000 7f4dd6400640 1 -- 172.21.15.45:0/2864735300 shutdown_connections 2024-04-16T17:17:27.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.475+0000 7f4dd6400640 1 --2- 172.21.15.45:0/2864735300 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f4db40a3d20 0x7f4db40a61e0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:27.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.475+0000 7f4dd6400640 1 --2- 172.21.15.45:0/2864735300 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4dd0107330 0x7f4dd01a0c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:27.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.475+0000 7f4dd6400640 1 --2- 172.21.15.45:0/2864735300 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4dd019a8f0 0x7f4dd019ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:27.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.475+0000 7f4dd6400640 1 --2- 172.21.15.45:0/2864735300 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4dd006a870 0x7f4dd01a06f0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:27.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.475+0000 7f4dd6400640 1 -- 172.21.15.45:0/2864735300 >> 172.21.15.45:0/2864735300 conn(0x7f4dd0076d00 msgr2=0x7f4dd010d380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:17:27.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.475+0000 7f4dd6400640 1 -- 172.21.15.45:0/2864735300 shutdown_connections 2024-04-16T17:17:27.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.475+0000 7f4dd6400640 1 -- 172.21.15.45:0/2864735300 wait complete. 2024-04-16T17:17:27.496 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-16T17:17:27.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.719+0000 7f9d5ea00640 1 -- 172.21.15.45:0/1476118962 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9d58104310 msgr2=0x7f9d581046f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:17:27.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.719+0000 7f9d5ea00640 1 --2- 172.21.15.45:0/1476118962 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9d58104310 0x7f9d581046f0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f9d48009be0 tx=0x7f9d4802bef0 comp rx=0 tx=0).stop 2024-04-16T17:17:27.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.720+0000 7f9d5ea00640 1 -- 172.21.15.45:0/1476118962 shutdown_connections 2024-04-16T17:17:27.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.720+0000 7f9d5ea00640 1 --2- 172.21.15.45:0/1476118962 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9d58104cc0 0x7f9d58108d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:27.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.720+0000 7f9d5ea00640 1 --2- 172.21.15.45:0/1476118962 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9d581093e0 0x7f9d58110fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:27.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.720+0000 7f9d5ea00640 1 --2- 172.21.15.45:0/1476118962 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9d58104310 0x7f9d581046f0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:27.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.720+0000 7f9d5ea00640 1 -- 172.21.15.45:0/1476118962 >> 172.21.15.45:0/1476118962 conn(0x7f9d580ffb70 msgr2=0x7f9d58101f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:17:27.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.720+0000 7f9d5ea00640 1 -- 172.21.15.45:0/1476118962 shutdown_connections 2024-04-16T17:17:27.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.720+0000 7f9d5ea00640 1 -- 172.21.15.45:0/1476118962 wait complete. 2024-04-16T17:17:27.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.721+0000 7f9d5ea00640 1 Processor -- start 2024-04-16T17:17:27.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.721+0000 7f9d5ea00640 1 -- start start 2024-04-16T17:17:27.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.721+0000 7f9d5ea00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9d58104310 0x7f9d581a06c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:27.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.722+0000 7f9d5ea00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9d581093e0 0x7f9d581a0c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:27.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.722+0000 7f9d5ea00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9d5819a8c0 0x7f9d5819ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:27.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.722+0000 7f9d5ea00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f9d581144e0 con 0x7f9d58104310 2024-04-16T17:17:27.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.722+0000 7f9d5ea00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f9d58114360 con 0x7f9d5819a8c0 2024-04-16T17:17:27.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.722+0000 7f9d5ea00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f9d58114660 con 0x7f9d581093e0 2024-04-16T17:17:27.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.722+0000 7f9d5d600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9d58104310 0x7f9d581a06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:27.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.722+0000 7f9d5d600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9d58104310 0x7f9d581a06c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45236/0 (socket says 172.21.15.45:45236) 2024-04-16T17:17:27.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.722+0000 7f9d5d600640 1 -- 172.21.15.45:0/792415807 learned_addr learned my addr 172.21.15.45:0/792415807 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:17:27.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.722+0000 7f9d5cc00640 1 --2- 172.21.15.45:0/792415807 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9d581093e0 0x7f9d581a0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:27.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.722+0000 7f9d5e000640 1 --2- 172.21.15.45:0/792415807 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9d5819a8c0 0x7f9d5819ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:27.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.722+0000 7f9d5cc00640 1 -- 172.21.15.45:0/792415807 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9d5819a8c0 msgr2=0x7f9d5819ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:17:27.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.722+0000 7f9d5cc00640 1 --2- 172.21.15.45:0/792415807 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9d5819a8c0 0x7f9d5819ad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:27.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.722+0000 7f9d5cc00640 1 -- 172.21.15.45:0/792415807 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9d58104310 msgr2=0x7f9d581a06c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:17:27.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.722+0000 7f9d5cc00640 1 --2- 172.21.15.45:0/792415807 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9d58104310 0x7f9d581a06c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:27.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.722+0000 7f9d5cc00640 1 -- 172.21.15.45:0/792415807 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d5819b3e0 con 0x7f9d581093e0 2024-04-16T17:17:27.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.722+0000 7f9d5d600640 1 --2- 172.21.15.45:0/792415807 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9d58104310 0x7f9d581a06c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:17:27.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.723+0000 7f9d5cc00640 1 --2- 172.21.15.45:0/792415807 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9d581093e0 0x7f9d581a0c00 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f9d4400e9c0 tx=0x7f9d4400ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:17:27.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.723+0000 7f9d56000640 1 -- 172.21.15.45:0/792415807 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d4400ccc0 con 0x7f9d581093e0 2024-04-16T17:17:27.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.723+0000 7f9d56000640 1 -- 172.21.15.45:0/792415807 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9d44005020 con 0x7f9d581093e0 2024-04-16T17:17:27.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.723+0000 7f9d56000640 1 -- 172.21.15.45:0/792415807 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d44004a00 con 0x7f9d581093e0 2024-04-16T17:17:27.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.723+0000 7f9d5ea00640 1 -- 172.21.15.45:0/792415807 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d5819b6d0 con 0x7f9d581093e0 2024-04-16T17:17:27.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.723+0000 7f9d5ea00640 1 -- 172.21.15.45:0/792415807 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d581a78d0 con 0x7f9d581093e0 2024-04-16T17:17:27.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.727+0000 7f9d62e17740 1 Processor -- start 2024-04-16T17:17:27.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.728+0000 7f9d62e17740 1 -- start start 2024-04-16T17:17:27.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.728+0000 7f9d62e17740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5630b17eff10 0x5630b17f02f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:27.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.728+0000 7f9d62e17740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5630b17f0830 0x5630b17f8ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:27.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.728+0000 7f9d62e17740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5630b17f9410 0x5630b17fb800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:27.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.728+0000 7f9d62e17740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5630b1755020 con 0x5630b17f9410 2024-04-16T17:17:27.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.728+0000 7f9d62e17740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5630b16fb460 con 0x5630b17f0830 2024-04-16T17:17:27.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.728+0000 7f9d62e17740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5630b1750fe0 con 0x5630b17eff10 2024-04-16T17:17:27.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.729+0000 7f9d5d600640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5630b17f0830 0x5630b17f8ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:27.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.729+0000 7f9d5d600640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5630b17f0830 0x5630b17f8ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:43804/0 (socket says 172.21.15.45:43804) 2024-04-16T17:17:27.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.729+0000 7f9d5d600640 1 -- 172.21.15.45:0/3948650541 learned_addr learned my addr 172.21.15.45:0/3948650541 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:17:27.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.729+0000 7f9d5e000640 1 --2- 172.21.15.45:0/3948650541 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5630b17eff10 0x5630b17f02f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:27.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.729+0000 7f9d56000640 1 -- 172.21.15.45:0/792415807 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f9d440051d0 con 0x7f9d581093e0 2024-04-16T17:17:27.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.729+0000 7f9d5d600640 1 -- 172.21.15.45:0/3948650541 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5630b17eff10 msgr2=0x5630b17f02f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:17:27.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.729+0000 7f9d5d600640 1 --2- 172.21.15.45:0/3948650541 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5630b17eff10 0x5630b17f02f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:27.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.729+0000 7f9d5d600640 1 -- 172.21.15.45:0/3948650541 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5630b17f9410 msgr2=0x5630b17fb800 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-16T17:17:27.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.729+0000 7f9d5cc00640 1 --2- 172.21.15.45:0/3948650541 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5630b17f9410 0x5630b17fb800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:27.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.729+0000 7f9d5d600640 1 --2- 172.21.15.45:0/3948650541 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5630b17f9410 0x5630b17fb800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:27.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.729+0000 7f9d5d600640 1 -- 172.21.15.45:0/3948650541 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5630b1703040 con 0x5630b17f0830 2024-04-16T17:17:27.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.729+0000 7f9d5cc00640 1 --2- 172.21.15.45:0/3948650541 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5630b17f9410 0x5630b17fb800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-16T17:17:27.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.730+0000 7f9d5d600640 1 --2- 172.21.15.45:0/3948650541 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5630b17f0830 0x5630b17f8ed0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f9d48000f80 tx=0x7f9d48009840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a6449e4d2cd9e0ff server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:17:27.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.730+0000 7f9d4e600640 1 -- 172.21.15.45:0/3948650541 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d48043030 con 0x5630b17f0830 2024-04-16T17:17:27.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.730+0000 7f9d4e600640 1 -- 172.21.15.45:0/3948650541 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9d48003970 con 0x5630b17f0830 2024-04-16T17:17:27.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.730+0000 7f9d4e600640 1 -- 172.21.15.45:0/3948650541 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d4802cd40 con 0x5630b17f0830 2024-04-16T17:17:27.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.730+0000 7f9d56000640 1 --2- 172.21.15.45:0/792415807 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f9d3c0a3da0 0x7f9d3c0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:27.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.730+0000 7f9d56000640 1 -- 172.21.15.45:0/792415807 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 9496+0+0 (secure 0 0 0) 0x7f9d440d0ae0 con 0x7f9d581093e0 2024-04-16T17:17:27.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.731+0000 7f9d5e000640 1 --2- 172.21.15.45:0/792415807 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f9d3c0a3da0 0x7f9d3c0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:27.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.731+0000 7f9d62e17740 1 -- 172.21.15.45:0/3948650541 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5630b17f0830 msgr2=0x5630b17f8ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:17:27.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.731+0000 7f9d62e17740 1 --2- 172.21.15.45:0/3948650541 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5630b17f0830 0x5630b17f8ed0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f9d48000f80 tx=0x7f9d48009840 comp rx=0 tx=0).stop 2024-04-16T17:17:27.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.731+0000 7f9d62e17740 1 -- 172.21.15.45:0/3948650541 shutdown_connections 2024-04-16T17:17:27.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.731+0000 7f9d62e17740 1 --2- 172.21.15.45:0/3948650541 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5630b17f9410 0x5630b17fb800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:27.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.731+0000 7f9d62e17740 1 --2- 172.21.15.45:0/3948650541 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5630b17f0830 0x5630b17f8ed0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:27.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.731+0000 7f9d62e17740 1 --2- 172.21.15.45:0/3948650541 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5630b17eff10 0x5630b17f02f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:27.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.731+0000 7f9d62e17740 1 -- 172.21.15.45:0/3948650541 >> 172.21.15.45:0/3948650541 conn(0x5630b17ea640 msgr2=0x5630b17eaa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:17:27.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.731+0000 7f9d62e17740 1 -- 172.21.15.45:0/3948650541 shutdown_connections 2024-04-16T17:17:27.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.731+0000 7f9d62e17740 1 -- 172.21.15.45:0/3948650541 wait complete. 2024-04-16T17:17:27.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.731+0000 7f9d5e000640 1 --2- 172.21.15.45:0/792415807 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f9d3c0a3da0 0x7f9d3c0a6260 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f9d50009530 tx=0x7f9d50009260 comp rx=0 tx=0).ready entity=mgr.25411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:17:27.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.732+0000 7f9d62e17740 1 -- start start 2024-04-16T17:17:27.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.732+0000 7f9d62e17740 1 Processor -- start 2024-04-16T17:17:27.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.732+0000 7f9d62e17740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5630b17eff10 0x5630b17f6550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:27.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.733+0000 7f9d62e17740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5630b17f0830 0x5630b17f6a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:27.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.733+0000 7f9d62e17740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5630b17f9410 0x5630b17f6fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:27.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.733+0000 7f9d62e17740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5630b17fc1a0 con 0x5630b17eff10 2024-04-16T17:17:27.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.733+0000 7f9d62e17740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5630b170e830 con 0x5630b17f9410 2024-04-16T17:17:27.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.733+0000 7f9d62e17740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5630b17fc320 con 0x5630b17f0830 2024-04-16T17:17:27.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.733+0000 7f9d62e17740 10 client.0 _ll_register_callbacks cb 0x5630b1716150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:17:27.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.733+0000 7f9d5d600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5630b17eff10 0x5630b17f6550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:27.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.733+0000 7f9d5d600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5630b17eff10 0x5630b17f6550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45268/0 (socket says 172.21.15.45:45268) 2024-04-16T17:17:27.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.733+0000 7f9d5d600640 1 -- 172.21.15.45:0/3724306436 learned_addr learned my addr 172.21.15.45:0/3724306436 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:17:27.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.733+0000 7f9d5e000640 1 --2- 172.21.15.45:0/3724306436 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5630b17f9410 0x5630b17f6fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:27.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.733+0000 7f9d5cc00640 1 --2- 172.21.15.45:0/3724306436 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5630b17f0830 0x5630b17f6a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:27.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.733+0000 7f9d5d600640 1 -- 172.21.15.45:0/3724306436 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5630b17f0830 msgr2=0x5630b17f6a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:17:27.738 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.733+0000 7f9d5d600640 1 --2- 172.21.15.45:0/3724306436 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5630b17f0830 0x5630b17f6a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:27.738 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.733+0000 7f9d5d600640 1 -- 172.21.15.45:0/3724306436 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5630b17f9410 msgr2=0x5630b17f6fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:17:27.738 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.733+0000 7f9d5d600640 1 --2- 172.21.15.45:0/3724306436 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5630b17f9410 0x5630b17f6fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:27.738 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.733+0000 7f9d5d600640 1 -- 172.21.15.45:0/3724306436 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5630b1712920 con 0x5630b17eff10 2024-04-16T17:17:27.738 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.733+0000 7f9d5e000640 1 --2- 172.21.15.45:0/3724306436 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5630b17f9410 0x5630b17f6fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:17:27.738 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.734+0000 7f9d5d600640 1 --2- 172.21.15.45:0/3724306436 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5630b17eff10 0x5630b17f6550 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f9d4802c500 tx=0x7f9d4802bef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8aba934d935196bf server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:17:27.738 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.734+0000 7f9d42a00640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-16T17:17:27.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.737+0000 7f9d42a00640 1 -- 172.21.15.45:0/3724306436 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d4804c030 con 0x5630b17eff10 2024-04-16T17:17:27.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.737+0000 7f9d42a00640 1 -- 172.21.15.45:0/3724306436 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9496+0+0 (secure 0 0 0) 0x7f9d48033710 con 0x5630b17eff10 2024-04-16T17:17:27.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.737+0000 7f9d62e17740 1 -- 172.21.15.45:0/3724306436 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9d48003970 con 0x5630b17eff10 2024-04-16T17:17:27.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.737+0000 7f9d42a00640 1 -- 172.21.15.45:0/3724306436 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f9d48056050 con 0x5630b17eff10 2024-04-16T17:17:27.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.737+0000 7f9d62e17740 10 client.18183 fetch_fsmap learned FSMap version 44 2024-04-16T17:17:27.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.738+0000 7f9d62e17740 1 -- 172.21.15.45:0/3724306436 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x5630b16e07f0 con 0x5630b17eff10 2024-04-16T17:17:27.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.738+0000 7f9d42a00640 1 -- 172.21.15.45:0/3724306436 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9d48003970 con 0x5630b17eff10 2024-04-16T17:17:27.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.738+0000 7f9d42a00640 1 -- 172.21.15.45:0/3724306436 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 44) ==== 8346+0+0 (secure 0 0 0) 0x7f9d4803f6a0 con 0x5630b17eff10 2024-04-16T17:17:27.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.738+0000 7f9d62e17740 10 client.18183 fetch_fsmap finished waiting for FSMap version 44 2024-04-16T17:17:27.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.738+0000 7f9d62e17740 10 client.18183 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:17:27.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.738+0000 7f9d62e17740 20 client.18183 populate_metadata read hostname 'smithi045' 2024-04-16T17:17:27.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.738+0000 7f9d62e17740 1 --2- 172.21.15.45:0/3724306436 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x5630b1809640 0x5630b180ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:27.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.738+0000 7f9d62e17740 4 client.18183 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-16T17:17:27.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.738+0000 7f9d62e17740 1 -- 172.21.15.45:0/3724306436 --> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5630b1762a40 con 0x5630b1809640 2024-04-16T17:17:27.743 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.739+0000 7f9d5cc00640 1 --2- 172.21.15.45:0/3724306436 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x5630b1809640 0x5630b180ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:27.743 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.739+0000 7f9d5cc00640 1 --2- 172.21.15.45:0/3724306436 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x5630b1809640 0x5630b180ba30 crc :-1 s=READY pgs=259 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c0f5d7ac9470f3c8 server_cookie=f08ff9c6b6728fc5 in_seq=0 out_seq=0 2024-04-16T17:17:27.743 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.739+0000 7f9d42a00640 10 client.18183 ms_handle_connect on v2:172.21.15.104:6838/3660070592 2024-04-16T17:17:27.743 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.740+0000 7f9d42a00640 1 -- 172.21.15.45:0/3724306436 <== mds.0 v2:172.21.15.104:6838/3660070592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5630b1762a40 con 0x5630b1809640 2024-04-16T17:17:27.743 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.740+0000 7f9d42a00640 10 client.18183 handle_command_reply: tid=0 2024-04-16T17:17:27.743 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.740+0000 7f9d62e17740 1 client.18183 shutdown 2024-04-16T17:17:27.743 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.740+0000 7f9d62e17740 1 -- 172.21.15.45:0/3724306436 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5630b17eff10 msgr2=0x5630b17f6550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:17:27.743 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.740+0000 7f9d62e17740 1 --2- 172.21.15.45:0/3724306436 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5630b17eff10 0x5630b17f6550 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f9d4802c500 tx=0x7f9d4802bef0 comp rx=0 tx=0).stop 2024-04-16T17:17:27.744 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.740+0000 7f9d62e17740 1 -- 172.21.15.45:0/3724306436 shutdown_connections 2024-04-16T17:17:27.744 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.740+0000 7f9d62e17740 1 --2- 172.21.15.45:0/3724306436 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x5630b1809640 0x5630b180ba30 crc :-1 s=READY pgs=259 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:27.744 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.741+0000 7f9d62e17740 1 --2- 172.21.15.45:0/3724306436 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5630b17f0830 0x5630b17f6a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:27.744 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.741+0000 7f9d62e17740 1 --2- 172.21.15.45:0/3724306436 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5630b17f9410 0x5630b17f6fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:27.744 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.741+0000 7f9d62e17740 1 --2- 172.21.15.45:0/3724306436 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5630b17eff10 0x5630b17f6550 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:27.744 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.741+0000 7f9d42a00640 0 client.18183 ms_handle_reset on v2:172.21.15.104:6838/3660070592 2024-04-16T17:17:27.744 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.741+0000 7f9d62e17740 1 -- 172.21.15.45:0/3724306436 >> 172.21.15.45:0/3724306436 conn(0x5630b17ea640 msgr2=0x5630b17ec390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:17:27.744 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.741+0000 7f9d62e17740 1 -- 172.21.15.45:0/3724306436 shutdown_connections 2024-04-16T17:17:27.744 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.741+0000 7f9d62e17740 1 -- 172.21.15.45:0/3724306436 wait complete. 2024-04-16T17:17:27.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.741+0000 7f9d62e17740 20 client.18183 trim_cache size 0 max 16384 2024-04-16T17:17:27.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.756+0000 7f9d62e17740 1 Processor -- start 2024-04-16T17:17:27.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.756+0000 7f9d62e17740 1 -- start start 2024-04-16T17:17:27.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.756+0000 7f9d62e17740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5630b17eff10 0x5630b17f8c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:27.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.756+0000 7f9d62e17740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5630b17f0830 0x5630b1805060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:27.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.756+0000 7f9d62e17740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5630b17f9410 0x5630b17f56b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:27.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.756+0000 7f9d62e17740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f9d581b3d80 con 0x5630b17f0830 2024-04-16T17:17:27.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.756+0000 7f9d62e17740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f9d581b1650 con 0x5630b17eff10 2024-04-16T17:17:27.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.756+0000 7f9d62e17740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f9d581af4c0 con 0x5630b17f9410 2024-04-16T17:17:27.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.757+0000 7f9d5d600640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5630b17eff10 0x5630b17f8c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:27.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.757+0000 7f9d5d600640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5630b17eff10 0x5630b17f8c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:43814/0 (socket says 172.21.15.45:43814) 2024-04-16T17:17:27.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.757+0000 7f9d5d600640 1 -- 172.21.15.45:0/510196245 learned_addr learned my addr 172.21.15.45:0/510196245 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:17:27.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.757+0000 7f9d5cc00640 1 --2- 172.21.15.45:0/510196245 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5630b17f0830 0x5630b1805060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:27.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.757+0000 7f9d5e000640 1 --2- 172.21.15.45:0/510196245 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5630b17f9410 0x5630b17f56b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:27.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.757+0000 7f9d5d600640 1 -- 172.21.15.45:0/510196245 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5630b17f9410 msgr2=0x5630b17f56b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:17:27.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.757+0000 7f9d5d600640 1 --2- 172.21.15.45:0/510196245 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5630b17f9410 0x5630b17f56b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:27.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.757+0000 7f9d5d600640 1 -- 172.21.15.45:0/510196245 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5630b17f0830 msgr2=0x5630b1805060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:17:27.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.757+0000 7f9d5d600640 1 --2- 172.21.15.45:0/510196245 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5630b17f0830 0x5630b1805060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:27.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.757+0000 7f9d5d600640 1 -- 172.21.15.45:0/510196245 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5630b14673f0 con 0x5630b17eff10 2024-04-16T17:17:27.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.757+0000 7f9d5d600640 1 --2- 172.21.15.45:0/510196245 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5630b17eff10 0x5630b17f8c80 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f9d48002230 tx=0x7f9d48009a80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=717c4f5bea8cc13f server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:17:27.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.758+0000 7f9d4dc00640 1 -- 172.21.15.45:0/510196245 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d4804c070 con 0x5630b17eff10 2024-04-16T17:17:27.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.758+0000 7f9d4dc00640 1 -- 172.21.15.45:0/510196245 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9d4805f070 con 0x5630b17eff10 2024-04-16T17:17:27.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.758+0000 7f9d4dc00640 1 -- 172.21.15.45:0/510196245 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d48043070 con 0x5630b17eff10 2024-04-16T17:17:27.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.759+0000 7f9d62e17740 1 -- 172.21.15.45:0/510196245 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5630b17eff10 msgr2=0x5630b17f8c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:17:27.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.759+0000 7f9d62e17740 1 --2- 172.21.15.45:0/510196245 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5630b17eff10 0x5630b17f8c80 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f9d48002230 tx=0x7f9d48009a80 comp rx=0 tx=0).stop 2024-04-16T17:17:27.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.759+0000 7f9d62e17740 1 -- 172.21.15.45:0/510196245 shutdown_connections 2024-04-16T17:17:27.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.759+0000 7f9d62e17740 1 --2- 172.21.15.45:0/510196245 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5630b17f9410 0x5630b17f56b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:27.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.759+0000 7f9d62e17740 1 --2- 172.21.15.45:0/510196245 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5630b17f0830 0x5630b1805060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:27.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.759+0000 7f9d62e17740 1 --2- 172.21.15.45:0/510196245 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5630b17eff10 0x5630b17f8c80 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:27.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.759+0000 7f9d62e17740 1 -- 172.21.15.45:0/510196245 >> 172.21.15.45:0/510196245 conn(0x5630b17ea640 msgr2=0x5630b17e8e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:17:27.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.759+0000 7f9d62e17740 1 -- 172.21.15.45:0/510196245 shutdown_connections 2024-04-16T17:17:27.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.759+0000 7f9d62e17740 1 -- 172.21.15.45:0/510196245 wait complete. 2024-04-16T17:17:27.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.759+0000 7f9d62e17740 1 -- start start 2024-04-16T17:17:27.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.760+0000 7f9d62e17740 1 Processor -- start 2024-04-16T17:17:27.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.760+0000 7f9d62e17740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5630b17eff10 0x5630b1801930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:27.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.760+0000 7f9d62e17740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5630b17f0830 0x5630b17fcca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:27.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.760+0000 7f9d62e17740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5630b17f9410 0x5630b17fd1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:27.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.760+0000 7f9d62e17740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f9d581afb10 con 0x5630b17f9410 2024-04-16T17:17:27.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.760+0000 7f9d62e17740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f9d581b0f80 con 0x5630b17f0830 2024-04-16T17:17:27.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.760+0000 7f9d62e17740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f9d581b01e0 con 0x5630b17eff10 2024-04-16T17:17:27.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.760+0000 7f9d62e17740 10 client.0 _ll_register_callbacks cb 0x7f9d50009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-16T17:17:27.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.761+0000 7f9d5cc00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5630b17f0830 0x5630b17fcca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:27.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.761+0000 7f9d5e000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5630b17f9410 0x5630b17fd1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:27.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.761+0000 7f9d5d600640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5630b17eff10 0x5630b1801930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:27.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.761+0000 7f9d5cc00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5630b17f0830 0x5630b17fcca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:43818/0 (socket says 172.21.15.45:43818) 2024-04-16T17:17:27.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.761+0000 7f9d5cc00640 1 -- 172.21.15.45:0/2856831406 learned_addr learned my addr 172.21.15.45:0/2856831406 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:17:27.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.761+0000 7f9d5e000640 1 -- 172.21.15.45:0/2856831406 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5630b17eff10 msgr2=0x5630b1801930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:17:27.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.761+0000 7f9d5e000640 1 --2- 172.21.15.45:0/2856831406 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5630b17eff10 0x5630b1801930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:27.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.761+0000 7f9d5e000640 1 -- 172.21.15.45:0/2856831406 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5630b17f0830 msgr2=0x5630b17fcca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-16T17:17:27.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.761+0000 7f9d5e000640 1 --2- 172.21.15.45:0/2856831406 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5630b17f0830 0x5630b17fcca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:27.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.761+0000 7f9d5e000640 1 -- 172.21.15.45:0/2856831406 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5630b15317d0 con 0x5630b17f9410 2024-04-16T17:17:27.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.761+0000 7f9d5d600640 1 --2- 172.21.15.45:0/2856831406 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5630b17eff10 0x5630b1801930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:17:27.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.761+0000 7f9d5cc00640 1 --2- 172.21.15.45:0/2856831406 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5630b17f0830 0x5630b17fcca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:17:27.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.761+0000 7f9d5e000640 1 --2- 172.21.15.45:0/2856831406 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5630b17f9410 0x5630b17fd1e0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x5630b1805720 tx=0x7f9d50017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7df81bd45ff79262 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:17:27.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.761+0000 7f9d42a00640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-16T17:17:27.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.762+0000 7f9d42a00640 1 -- 172.21.15.45:0/2856831406 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d5002f060 con 0x5630b17f9410 2024-04-16T17:17:27.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.762+0000 7f9d42a00640 1 -- 172.21.15.45:0/2856831406 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9496+0+0 (secure 0 0 0) 0x7f9d5001eb50 con 0x5630b17f9410 2024-04-16T17:17:27.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.762+0000 7f9d62e17740 1 -- 172.21.15.45:0/2856831406 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9d4805f070 con 0x5630b17f9410 2024-04-16T17:17:27.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.762+0000 7f9d42a00640 1 -- 172.21.15.45:0/2856831406 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f9d50036050 con 0x5630b17f9410 2024-04-16T17:17:27.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.762+0000 7f9d62e17740 10 client.18189 fetch_fsmap learned FSMap version 44 2024-04-16T17:17:27.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.762+0000 7f9d62e17740 1 -- 172.21.15.45:0/2856831406 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x5630b186f720 con 0x5630b17f9410 2024-04-16T17:17:27.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.762+0000 7f9d42a00640 1 -- 172.21.15.45:0/2856831406 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9d4805f070 con 0x5630b17f9410 2024-04-16T17:17:27.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.763+0000 7f9d42a00640 1 -- 172.21.15.45:0/2856831406 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 44) ==== 8346+0+0 (secure 0 0 0) 0x7f9d5001d3e0 con 0x5630b17f9410 2024-04-16T17:17:27.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.763+0000 7f9d62e17740 10 client.18189 fetch_fsmap finished waiting for FSMap version 44 2024-04-16T17:17:27.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.763+0000 7f9d62e17740 10 client.18189 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-16T17:17:27.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.763+0000 7f9d62e17740 20 client.18189 populate_metadata read hostname 'smithi045' 2024-04-16T17:17:27.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.763+0000 7f9d62e17740 1 --2- 172.21.15.45:0/2856831406 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x5630b186fea0 0x5630b18722c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:27.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.763+0000 7f9d62e17740 4 client.18189 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-16T17:17:27.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.763+0000 7f9d62e17740 1 -- 172.21.15.45:0/2856831406 --> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5630b175c8b0 con 0x5630b186fea0 2024-04-16T17:17:27.768 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.763+0000 7f9d5d600640 1 --2- 172.21.15.45:0/2856831406 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x5630b186fea0 0x5630b18722c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:27.769 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.764+0000 7f9d5d600640 1 --2- 172.21.15.45:0/2856831406 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x5630b186fea0 0x5630b18722c0 crc :-1 s=READY pgs=260 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fc4bffd3d1e47f5 server_cookie=2c78fbf5c3e82661 in_seq=0 out_seq=0 2024-04-16T17:17:27.769 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.764+0000 7f9d42a00640 10 client.18189 ms_handle_connect on v2:172.21.15.104:6838/3660070592 2024-04-16T17:17:27.769 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.764+0000 7f9d42a00640 1 -- 172.21.15.45:0/2856831406 <== mds.0 v2:172.21.15.104:6838/3660070592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5630b175c8b0 con 0x5630b186fea0 2024-04-16T17:17:27.769 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.764+0000 7f9d42a00640 10 client.18189 handle_command_reply: tid=0 2024-04-16T17:17:27.769 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.764+0000 7f9d62e17740 1 client.18189 shutdown 2024-04-16T17:17:27.769 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.764+0000 7f9d62e17740 1 -- 172.21.15.45:0/2856831406 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5630b17f9410 msgr2=0x5630b17fd1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:17:27.769 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.764+0000 7f9d62e17740 1 --2- 172.21.15.45:0/2856831406 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5630b17f9410 0x5630b17fd1e0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x5630b1805720 tx=0x7f9d50017000 comp rx=0 tx=0).stop 2024-04-16T17:17:27.770 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.764+0000 7f9d62e17740 1 -- 172.21.15.45:0/2856831406 shutdown_connections 2024-04-16T17:17:27.770 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.764+0000 7f9d62e17740 1 --2- 172.21.15.45:0/2856831406 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5630b17f9410 0x5630b17fd1e0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:27.770 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.764+0000 7f9d62e17740 1 --2- 172.21.15.45:0/2856831406 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5630b17f0830 0x5630b17fcca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:27.770 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.764+0000 7f9d62e17740 1 --2- 172.21.15.45:0/2856831406 >> [v2:172.21.15.104:6838/3660070592,v1:172.21.15.104:6839/3660070592] conn(0x5630b186fea0 0x5630b18722c0 crc :-1 s=READY pgs=260 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:27.770 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.764+0000 7f9d62e17740 1 --2- 172.21.15.45:0/2856831406 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5630b17eff10 0x5630b1801930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:27.770 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.764+0000 7f9d42a00640 0 client.18189 ms_handle_reset on v2:172.21.15.104:6838/3660070592 2024-04-16T17:17:27.770 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.764+0000 7f9d62e17740 1 -- 172.21.15.45:0/2856831406 >> 172.21.15.45:0/2856831406 conn(0x5630b17ea640 msgr2=0x5630b17f64f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:17:27.770 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.764+0000 7f9d62e17740 1 -- 172.21.15.45:0/2856831406 shutdown_connections 2024-04-16T17:17:27.770 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.764+0000 7f9d62e17740 1 -- 172.21.15.45:0/2856831406 wait complete. 2024-04-16T17:17:27.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.764+0000 7f9d62e17740 20 client.18189 trim_cache size 0 max 16384 2024-04-16T17:17:27.771 INFO:teuthology.orchestra.run.smithi045.stdout:[] 2024-04-16T17:17:27.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.765+0000 7f9d5ea00640 1 -- 172.21.15.45:0/792415807 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f9d3c0a3da0 msgr2=0x7f9d3c0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:17:27.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.765+0000 7f9d5ea00640 1 --2- 172.21.15.45:0/792415807 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f9d3c0a3da0 0x7f9d3c0a6260 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f9d50009530 tx=0x7f9d50009260 comp rx=0 tx=0).stop 2024-04-16T17:17:27.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.765+0000 7f9d5ea00640 1 -- 172.21.15.45:0/792415807 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9d581093e0 msgr2=0x7f9d581a0c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:17:27.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.765+0000 7f9d5ea00640 1 --2- 172.21.15.45:0/792415807 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9d581093e0 0x7f9d581a0c00 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f9d4400e9c0 tx=0x7f9d4400ee80 comp rx=0 tx=0).stop 2024-04-16T17:17:27.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.765+0000 7f9d5ea00640 1 -- 172.21.15.45:0/792415807 shutdown_connections 2024-04-16T17:17:27.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.765+0000 7f9d5ea00640 1 --2- 172.21.15.45:0/792415807 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f9d3c0a3da0 0x7f9d3c0a6260 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:27.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.765+0000 7f9d5ea00640 1 --2- 172.21.15.45:0/792415807 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9d581093e0 0x7f9d581a0c00 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:27.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.765+0000 7f9d5ea00640 1 --2- 172.21.15.45:0/792415807 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9d5819a8c0 0x7f9d5819ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:27.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.765+0000 7f9d5ea00640 1 --2- 172.21.15.45:0/792415807 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9d58104310 0x7f9d581a06c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:27.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.765+0000 7f9d5ea00640 1 -- 172.21.15.45:0/792415807 >> 172.21.15.45:0/792415807 conn(0x7f9d580ffb70 msgr2=0x7f9d58101ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:17:27.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.765+0000 7f9d5ea00640 1 -- 172.21.15.45:0/792415807 shutdown_connections 2024-04-16T17:17:27.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:27.765+0000 7f9d5ea00640 1 -- 172.21.15.45:0/792415807 wait complete. 2024-04-16T17:17:27.780 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:17:27 smithi045 ceph-mon[31676]: from='client.? 172.21.15.45:0/3368150411' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:17:27.780 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:17:27 smithi045 ceph-mon[31676]: pgmap v335: 129 pgs: 129 active+clean; 55 MiB data, 874 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2024-04-16T17:17:28.054 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:17:27 smithi117 ceph-mon[36730]: from='client.? 172.21.15.45:0/3368150411' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:17:28.054 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:17:27 smithi117 ceph-mon[36730]: pgmap v335: 129 pgs: 129 active+clean; 55 MiB data, 874 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2024-04-16T17:17:28.176 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:17:27 smithi104 ceph-mon[36457]: from='client.? 172.21.15.45:0/3368150411' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:17:28.177 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:17:27 smithi104 ceph-mon[36457]: pgmap v335: 129 pgs: 129 active+clean; 55 MiB data, 874 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2024-04-16T17:17:28.785 INFO:tasks.fwd_scrub.fs.[cephfs]:end scrubbing fs: cephfs 2024-04-16T17:17:28.786 INFO:tasks.fwd_scrub:done joining 2024-04-16T17:17:28.786 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-04-16T17:17:28.797 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-16T17:17:29.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:29.033+0000 7f180ce00640 1 -- 172.21.15.45:0/2852770379 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1808105420 msgr2=0x7f180810cf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:17:29.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:29.033+0000 7f180ce00640 1 --2- 172.21.15.45:0/2852770379 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1808105420 0x7f180810cf60 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f1800009ca0 tx=0x7f180002c100 comp rx=0 tx=0).stop 2024-04-16T17:17:29.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:29.033+0000 7f180ce00640 1 -- 172.21.15.45:0/2852770379 shutdown_connections 2024-04-16T17:17:29.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:29.033+0000 7f180ce00640 1 --2- 172.21.15.45:0/2852770379 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1808105420 0x7f180810cf60 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:29.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:29.033+0000 7f180ce00640 1 --2- 172.21.15.45:0/2852770379 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1808100f10 0x7f1808104cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:29.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:29.033+0000 7f180ce00640 1 --2- 172.21.15.45:0/2852770379 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1808100560 0x7f1808100940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:29.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:29.033+0000 7f180ce00640 1 -- 172.21.15.45:0/2852770379 >> 172.21.15.45:0/2852770379 conn(0x7f1808076c80 msgr2=0x7f1808077090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:17:29.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:29.034+0000 7f180ce00640 1 -- 172.21.15.45:0/2852770379 shutdown_connections 2024-04-16T17:17:29.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:29.034+0000 7f180ce00640 1 -- 172.21.15.45:0/2852770379 wait complete. 2024-04-16T17:17:29.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:29.035+0000 7f180ce00640 1 Processor -- start 2024-04-16T17:17:29.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:29.035+0000 7f180ce00640 1 -- start start 2024-04-16T17:17:29.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:29.035+0000 7f180ce00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1808100560 0x7f1808103920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:29.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:29.035+0000 7f180ce00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1808105420 0x7f1808101d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:29.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:29.035+0000 7f180ce00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f18081022b0 0x7f1808102740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:29.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:29.035+0000 7f180ce00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f180810fff0 con 0x7f1808100560 2024-04-16T17:17:29.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:29.035+0000 7f180ce00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f180810fe70 con 0x7f1808105420 2024-04-16T17:17:29.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:29.035+0000 7f180ce00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f1808110170 con 0x7f18081022b0 2024-04-16T17:17:29.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:29.036+0000 7f1807400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1808100560 0x7f1808103920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:29.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:29.036+0000 7f17ffe00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1808105420 0x7f1808101d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:29.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:29.036+0000 7f1807400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1808100560 0x7f1808103920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45328/0 (socket says 172.21.15.45:45328) 2024-04-16T17:17:29.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:29.036+0000 7f1807400640 1 -- 172.21.15.45:0/3424148822 learned_addr learned my addr 172.21.15.45:0/3424148822 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:17:29.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:29.036+0000 7f1807e00640 1 --2- 172.21.15.45:0/3424148822 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f18081022b0 0x7f1808102740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:29.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:29.036+0000 7f17ffe00640 1 -- 172.21.15.45:0/3424148822 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f18081022b0 msgr2=0x7f1808102740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:17:29.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:29.036+0000 7f17ffe00640 1 --2- 172.21.15.45:0/3424148822 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f18081022b0 0x7f1808102740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:29.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:29.036+0000 7f17ffe00640 1 -- 172.21.15.45:0/3424148822 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1808100560 msgr2=0x7f1808103920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:17:29.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:29.036+0000 7f17ffe00640 1 --2- 172.21.15.45:0/3424148822 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1808100560 0x7f1808103920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:29.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:29.036+0000 7f17ffe00640 1 -- 172.21.15.45:0/3424148822 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18081ab330 con 0x7f1808105420 2024-04-16T17:17:29.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:29.036+0000 7f1807400640 1 --2- 172.21.15.45:0/3424148822 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1808100560 0x7f1808103920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:17:29.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:29.036+0000 7f17ffe00640 1 --2- 172.21.15.45:0/3424148822 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1808105420 0x7f1808101d20 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f18080781d0 tx=0x7f17f000ee50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:17:29.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:29.037+0000 7f1804c00640 1 -- 172.21.15.45:0/3424148822 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17f000cc10 con 0x7f1808105420 2024-04-16T17:17:29.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:29.037+0000 7f180ce00640 1 -- 172.21.15.45:0/3424148822 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18081ab620 con 0x7f1808105420 2024-04-16T17:17:29.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:29.037+0000 7f1804c00640 1 -- 172.21.15.45:0/3424148822 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f17f000cdb0 con 0x7f1808105420 2024-04-16T17:17:29.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:29.037+0000 7f1804c00640 1 -- 172.21.15.45:0/3424148822 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17f0018800 con 0x7f1808105420 2024-04-16T17:17:29.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:29.037+0000 7f180ce00640 1 -- 172.21.15.45:0/3424148822 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18081abbc0 con 0x7f1808105420 2024-04-16T17:17:29.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:29.038+0000 7f180ce00640 1 -- 172.21.15.45:0/3424148822 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1808101760 con 0x7f1808105420 2024-04-16T17:17:29.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:29.040+0000 7f1804c00640 1 -- 172.21.15.45:0/3424148822 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f17f0010b30 con 0x7f1808105420 2024-04-16T17:17:29.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:29.042+0000 7f1804c00640 1 --2- 172.21.15.45:0/3424148822 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f17e80a3d20 0x7f17e80a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:29.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:29.042+0000 7f1804c00640 1 -- 172.21.15.45:0/3424148822 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 9496+0+0 (secure 0 0 0) 0x7f17f00d12f0 con 0x7f1808105420 2024-04-16T17:17:29.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:29.042+0000 7f1807400640 1 --2- 172.21.15.45:0/3424148822 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f17e80a3d20 0x7f17e80a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:29.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:29.043+0000 7f1807400640 1 --2- 172.21.15.45:0/3424148822 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f17e80a3d20 0x7f17e80a61e0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f17f800ab50 tx=0x7f17f800c040 comp rx=0 tx=0).ready entity=mgr.25411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:17:29.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:29.044+0000 7f1804c00640 1 -- 172.21.15.45:0/3424148822 <== mon.2 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f17f009cf10 con 0x7f1808105420 2024-04-16T17:17:29.254 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:29.254+0000 7f180ce00640 1 -- 172.21.15.45:0/3424148822 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f180810a5a0 con 0x7f1808105420 2024-04-16T17:17:29.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:29.257+0000 7f1804c00640 1 -- 172.21.15.45:0/3424148822 <== mon.2 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v106) ==== 74+0+25285 (secure 0 0 0) 0x7f17f00a1210 con 0x7f1808105420 2024-04-16T17:17:29.257 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-16T17:17:29.257 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":106,"fsid":"b1fecd38-fc11-11ee-bc90-c7b262605968","created":"2024-04-16T16:54:02.385906+0000","modified":"2024-04-16T17:17:14.403755+0000","last_up_change":"2024-04-16T17:02:25.162624+0000","last_in_change":"2024-04-16T17:02:04.903274+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-16T16:58:06.166659+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-16T17:03:22.304198+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-16T17:03:23.616450+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"105","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":17,"snap_epoch":105,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-16T17:03:27.226289+0000","flags":8197,"flags_names":"hashpspool,ec_overwrites,selfmanaged_snaps","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"105","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":17,"snap_epoch":105,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b655de2c-59f8-4fd4-ac98-0cc222229f56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":2290170881},{"type":"v1","addr":"172.21.15.45:6803","nonce":2290170881}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":2290170881},{"type":"v1","addr":"172.21.15.45:6805","nonce":2290170881}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":2290170881},{"type":"v1","addr":"172.21.15.45:6809","nonce":2290170881}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":2290170881},{"type":"v1","addr":"172.21.15.45:6807","nonce":2290170881}]},"public_addr":"172.21.15.45:6803/2290170881","cluster_addr":"172.21.15.45:6805/2290170881","heartbeat_back_addr":"172.21.15.45:6809/2290170881","heartbeat_front_addr":"172.21.15.45:6807/2290170881","state":["exists","up"]},{"osd":1,"uuid":"ebd450ec-08fd-4c3d-8b24-4e10634ddb7e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6800","nonce":1462942067},{"type":"v1","addr":"172.21.15.104:6801","nonce":1462942067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":1462942067},{"type":"v1","addr":"172.21.15.104:6803","nonce":1462942067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":1462942067},{"type":"v1","addr":"172.21.15.104:6807","nonce":1462942067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":1462942067},{"type":"v1","addr":"172.21.15.104:6805","nonce":1462942067}]},"public_addr":"172.21.15.104:6801/1462942067","cluster_addr":"172.21.15.104:6803/1462942067","heartbeat_back_addr":"172.21.15.104:6807/1462942067","heartbeat_front_addr":"172.21.15.104:6805/1462942067","state":["exists","up"]},{"osd":2,"uuid":"b318a512-4190-42a2-8104-dcc4026ad8ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":2829260735},{"type":"v1","addr":"172.21.15.117:6801","nonce":2829260735}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":2829260735},{"type":"v1","addr":"172.21.15.117:6803","nonce":2829260735}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":2829260735},{"type":"v1","addr":"172.21.15.117:6807","nonce":2829260735}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":2829260735},{"type":"v1","addr":"172.21.15.117:6805","nonce":2829260735}]},"public_addr":"172.21.15.117:6801/2829260735","cluster_addr":"172.21.15.117:6803/2829260735","heartbeat_back_addr":"172.21.15.117:6807/2829260735","heartbeat_front_addr":"172.21.15.117:6805/2829260735","state":["exists","up"]},{"osd":3,"uuid":"ec6490cb-a715-4c12-a54a-225aae26fcb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":2823712137},{"type":"v1","addr":"172.21.15.45:6811","nonce":2823712137}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":2823712137},{"type":"v1","addr":"172.21.15.45:6813","nonce":2823712137}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":2823712137},{"type":"v1","addr":"172.21.15.45:6817","nonce":2823712137}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":2823712137},{"type":"v1","addr":"172.21.15.45:6815","nonce":2823712137}]},"public_addr":"172.21.15.45:6811/2823712137","cluster_addr":"172.21.15.45:6813/2823712137","heartbeat_back_addr":"172.21.15.45:6817/2823712137","heartbeat_front_addr":"172.21.15.45:6815/2823712137","state":["exists","up"]},{"osd":4,"uuid":"5d91d568-0250-4aae-8dc2-a0e079f92062","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":1396278127},{"type":"v1","addr":"172.21.15.104:6809","nonce":1396278127}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":1396278127},{"type":"v1","addr":"172.21.15.104:6811","nonce":1396278127}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":1396278127},{"type":"v1","addr":"172.21.15.104:6815","nonce":1396278127}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":1396278127},{"type":"v1","addr":"172.21.15.104:6813","nonce":1396278127}]},"public_addr":"172.21.15.104:6809/1396278127","cluster_addr":"172.21.15.104:6811/1396278127","heartbeat_back_addr":"172.21.15.104:6815/1396278127","heartbeat_front_addr":"172.21.15.104:6813/1396278127","state":["exists","up"]},{"osd":5,"uuid":"ac510b43-6fb2-4d0f-bc5f-16ac080d9e9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":4145720047},{"type":"v1","addr":"172.21.15.117:6809","nonce":4145720047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":4145720047},{"type":"v1","addr":"172.21.15.117:6811","nonce":4145720047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":4145720047},{"type":"v1","addr":"172.21.15.117:6815","nonce":4145720047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":4145720047},{"type":"v1","addr":"172.21.15.117:6813","nonce":4145720047}]},"public_addr":"172.21.15.117:6809/4145720047","cluster_addr":"172.21.15.117:6811/4145720047","heartbeat_back_addr":"172.21.15.117:6815/4145720047","heartbeat_front_addr":"172.21.15.117:6813/4145720047","state":["exists","up"]},{"osd":6,"uuid":"a41adc18-d88a-419e-b2c2-b089c1f32472","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":3254229176},{"type":"v1","addr":"172.21.15.45:6819","nonce":3254229176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":3254229176},{"type":"v1","addr":"172.21.15.45:6821","nonce":3254229176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":3254229176},{"type":"v1","addr":"172.21.15.45:6825","nonce":3254229176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":3254229176},{"type":"v1","addr":"172.21.15.45:6823","nonce":3254229176}]},"public_addr":"172.21.15.45:6819/3254229176","cluster_addr":"172.21.15.45:6821/3254229176","heartbeat_back_addr":"172.21.15.45:6825/3254229176","heartbeat_front_addr":"172.21.15.45:6823/3254229176","state":["exists","up"]},{"osd":7,"uuid":"b71712a4-7cf6-454d-9fd1-565e3df21c1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":4151077186},{"type":"v1","addr":"172.21.15.104:6817","nonce":4151077186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":4151077186},{"type":"v1","addr":"172.21.15.104:6819","nonce":4151077186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":4151077186},{"type":"v1","addr":"172.21.15.104:6823","nonce":4151077186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":4151077186},{"type":"v1","addr":"172.21.15.104:6821","nonce":4151077186}]},"public_addr":"172.21.15.104:6817/4151077186","cluster_addr":"172.21.15.104:6819/4151077186","heartbeat_back_addr":"172.21.15.104:6823/4151077186","heartbeat_front_addr":"172.21.15.104:6821/4151077186","state":["exists","up"]},{"osd":8,"uuid":"9e572aef-59a8-450e-a92e-df9dff4d57a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":1500286830},{"type":"v1","addr":"172.21.15.117:6817","nonce":1500286830}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":1500286830},{"type":"v1","addr":"172.21.15.117:6819","nonce":1500286830}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":1500286830},{"type":"v1","addr":"172.21.15.117:6823","nonce":1500286830}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":1500286830},{"type":"v1","addr":"172.21.15.117:6821","nonce":1500286830}]},"public_addr":"172.21.15.117:6817/1500286830","cluster_addr":"172.21.15.117:6819/1500286830","heartbeat_back_addr":"172.21.15.117:6823/1500286830","heartbeat_front_addr":"172.21.15.117:6821/1500286830","state":["exists","up"]},{"osd":9,"uuid":"014e9598-c413-4cc4-9bb8-61c6e8f59d30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":3106734074},{"type":"v1","addr":"172.21.15.45:6827","nonce":3106734074}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":3106734074},{"type":"v1","addr":"172.21.15.45:6829","nonce":3106734074}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":3106734074},{"type":"v1","addr":"172.21.15.45:6833","nonce":3106734074}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":3106734074},{"type":"v1","addr":"172.21.15.45:6831","nonce":3106734074}]},"public_addr":"172.21.15.45:6827/3106734074","cluster_addr":"172.21.15.45:6829/3106734074","heartbeat_back_addr":"172.21.15.45:6833/3106734074","heartbeat_front_addr":"172.21.15.45:6831/3106734074","state":["exists","up"]},{"osd":10,"uuid":"2deba364-3e39-47be-8a1f-8256df5b82d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":1404794652},{"type":"v1","addr":"172.21.15.104:6825","nonce":1404794652}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":1404794652},{"type":"v1","addr":"172.21.15.104:6827","nonce":1404794652}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":1404794652},{"type":"v1","addr":"172.21.15.104:6831","nonce":1404794652}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":1404794652},{"type":"v1","addr":"172.21.15.104:6829","nonce":1404794652}]},"public_addr":"172.21.15.104:6825/1404794652","cluster_addr":"172.21.15.104:6827/1404794652","heartbeat_back_addr":"172.21.15.104:6831/1404794652","heartbeat_front_addr":"172.21.15.104:6829/1404794652","state":["exists","up"]},{"osd":11,"uuid":"3fda4fea-59ad-4d1a-9fe6-fcb30e31fd7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":3384003243},{"type":"v1","addr":"172.21.15.117:6825","nonce":3384003243}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":3384003243},{"type":"v1","addr":"172.21.15.117:6827","nonce":3384003243}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":3384003243},{"type":"v1","addr":"172.21.15.117:6831","nonce":3384003243}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":3384003243},{"type":"v1","addr":"172.21.15.117:6829","nonce":3384003243}]},"public_addr":"172.21.15.117:6825/3384003243","cluster_addr":"172.21.15.117:6827/3384003243","heartbeat_back_addr":"172.21.15.117:6831/3384003243","heartbeat_front_addr":"172.21.15.117:6829/3384003243","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:57:14.799608+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:57:39.884668+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:58:03.327057+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:58:37.314531+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:59:03.667650+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:59:28.462823+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T17:00:02.968574+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T17:00:29.559190+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T17:00:55.291007+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T17:01:29.815977+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T17:01:57.015580+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T17:02:23.233151+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.5","mappings":[{"from":9,"to":10}]},{"pgid":"4.7","mappings":[{"from":5,"to":8}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.45:6800/268030699":"2024-04-17T17:06:45.093215+0000","172.21.15.45:6801/268030699":"2024-04-17T17:06:45.093215+0000","172.21.15.45:0/4074982598":"2024-04-17T16:55:19.948001+0000","172.21.15.45:0/1915281176":"2024-04-17T17:06:45.093215+0000","172.21.15.45:0/2443572576":"2024-04-17T16:55:19.948001+0000","172.21.15.45:0/1500147666":"2024-04-17T17:06:45.093215+0000","172.21.15.45:6801/1983630136":"2024-04-17T16:55:19.948001+0000","172.21.15.45:6800/1983630136":"2024-04-17T16:55:19.948001+0000","172.21.15.45:0/12001708":"2024-04-17T17:06:45.093215+0000","172.21.15.45:0/3150465248":"2024-04-17T16:54:43.692466+0000","172.21.15.45:0/556738623":"2024-04-17T16:54:43.692466+0000","172.21.15.45:0/445676297":"2024-04-17T16:55:19.948001+0000","172.21.15.45:6800/2553659622":"2024-04-17T16:54:43.692466+0000","172.21.15.45:6801/2553659622":"2024-04-17T16:54:43.692466+0000","172.21.15.45:0/265066921":"2024-04-17T17:06:45.093215+0000","172.21.15.45:0/1184615408":"2024-04-17T16:54:43.692466+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":6,"length":1},{"begin":17,"length":1}]},{"pool":4,"snaps":[{"begin":6,"length":1},{"begin":17,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-16T17:17:29.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:29.260+0000 7f180ce00640 1 -- 172.21.15.45:0/3424148822 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f17e80a3d20 msgr2=0x7f17e80a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:17:29.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:29.260+0000 7f180ce00640 1 --2- 172.21.15.45:0/3424148822 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f17e80a3d20 0x7f17e80a61e0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f17f800ab50 tx=0x7f17f800c040 comp rx=0 tx=0).stop 2024-04-16T17:17:29.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:29.260+0000 7f180ce00640 1 -- 172.21.15.45:0/3424148822 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1808105420 msgr2=0x7f1808101d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:17:29.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:29.260+0000 7f180ce00640 1 --2- 172.21.15.45:0/3424148822 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1808105420 0x7f1808101d20 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f18080781d0 tx=0x7f17f000ee50 comp rx=0 tx=0).stop 2024-04-16T17:17:29.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:29.260+0000 7f180ce00640 1 -- 172.21.15.45:0/3424148822 shutdown_connections 2024-04-16T17:17:29.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:29.260+0000 7f180ce00640 1 --2- 172.21.15.45:0/3424148822 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f17e80a3d20 0x7f17e80a61e0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:29.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:29.260+0000 7f180ce00640 1 --2- 172.21.15.45:0/3424148822 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f18081022b0 0x7f1808102740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:29.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:29.260+0000 7f180ce00640 1 --2- 172.21.15.45:0/3424148822 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1808105420 0x7f1808101d20 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:29.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:29.260+0000 7f180ce00640 1 --2- 172.21.15.45:0/3424148822 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1808100560 0x7f1808103920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:29.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:29.260+0000 7f180ce00640 1 -- 172.21.15.45:0/3424148822 >> 172.21.15.45:0/3424148822 conn(0x7f1808076c80 msgr2=0x7f18081040e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:17:29.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:29.260+0000 7f180ce00640 1 -- 172.21.15.45:0/3424148822 shutdown_connections 2024-04-16T17:17:29.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:29.261+0000 7f180ce00640 1 -- 172.21.15.45:0/3424148822 wait complete. 2024-04-16T17:17:29.281 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-16T16:58:06.166659+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-16T17:03:22.304198+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-16T17:03:23.616450+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '105', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 17, 'snap_epoch': 105, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-16T17:03:27.226289+0000', 'flags': 8197, 'flags_names': 'hashpspool,ec_overwrites,selfmanaged_snaps', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '105', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 17, 'snap_epoch': 105, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-16T17:17:29.281 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-16T17:17:29.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:29.505+0000 7fb493200640 1 -- 172.21.15.45:0/1670611967 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb4941029c0 msgr2=0x7fb494106a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:17:29.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:29.505+0000 7fb493200640 1 --2- 172.21.15.45:0/1670611967 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb4941029c0 0x7fb494106a00 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fb480009900 tx=0x7fb48002bc30 comp rx=0 tx=0).stop 2024-04-16T17:17:29.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:29.505+0000 7fb493200640 1 -- 172.21.15.45:0/1670611967 shutdown_connections 2024-04-16T17:17:29.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:29.505+0000 7fb493200640 1 --2- 172.21.15.45:0/1670611967 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb4941070e0 0x7fb49410ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:29.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:29.505+0000 7fb493200640 1 --2- 172.21.15.45:0/1670611967 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb4941029c0 0x7fb494106a00 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:29.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:29.505+0000 7fb493200640 1 --2- 172.21.15.45:0/1670611967 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb494102010 0x7fb4941023f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:29.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:29.506+0000 7fb493200640 1 -- 172.21.15.45:0/1670611967 >> 172.21.15.45:0/1670611967 conn(0x7fb4940fd8d0 msgr2=0x7fb4940ffcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:17:29.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:29.506+0000 7fb493200640 1 -- 172.21.15.45:0/1670611967 shutdown_connections 2024-04-16T17:17:29.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:29.506+0000 7fb493200640 1 -- 172.21.15.45:0/1670611967 wait complete. 2024-04-16T17:17:29.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:29.507+0000 7fb493200640 1 Processor -- start 2024-04-16T17:17:29.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:29.507+0000 7fb493200640 1 -- start start 2024-04-16T17:17:29.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:29.508+0000 7fb493200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb494102010 0x7fb4941a0610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:29.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:29.508+0000 7fb493200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb4941070e0 0x7fb4941a0b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:29.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:29.508+0000 7fb493200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb49419a6d0 0x7fb49419ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:29.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:29.508+0000 7fb493200640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fb494112150 con 0x7fb49419a6d0 2024-04-16T17:17:29.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:29.508+0000 7fb493200640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fb494111fd0 con 0x7fb4941070e0 2024-04-16T17:17:29.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:29.508+0000 7fb493200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb4941122d0 con 0x7fb494102010 2024-04-16T17:17:29.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:29.508+0000 7fb492800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb49419a6d0 0x7fb49419ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:29.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:29.508+0000 7fb491e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb494102010 0x7fb4941a0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:29.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:29.508+0000 7fb491400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb4941070e0 0x7fb4941a0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:29.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:29.508+0000 7fb491e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb494102010 0x7fb4941a0610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:35984/0 (socket says 172.21.15.45:35984) 2024-04-16T17:17:29.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:29.508+0000 7fb491e00640 1 -- 172.21.15.45:0/2837034753 learned_addr learned my addr 172.21.15.45:0/2837034753 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:17:29.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:29.508+0000 7fb491e00640 1 -- 172.21.15.45:0/2837034753 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb4941070e0 msgr2=0x7fb4941a0b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:17:29.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:29.508+0000 7fb491e00640 1 --2- 172.21.15.45:0/2837034753 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb4941070e0 0x7fb4941a0b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:29.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:29.508+0000 7fb491e00640 1 -- 172.21.15.45:0/2837034753 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb49419a6d0 msgr2=0x7fb49419ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:17:29.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:29.508+0000 7fb491e00640 1 --2- 172.21.15.45:0/2837034753 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb49419a6d0 0x7fb49419ab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:29.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:29.508+0000 7fb491e00640 1 -- 172.21.15.45:0/2837034753 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb49419b2d0 con 0x7fb494102010 2024-04-16T17:17:29.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:29.508+0000 7fb491400640 1 --2- 172.21.15.45:0/2837034753 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb4941070e0 0x7fb4941a0b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:17:29.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:29.508+0000 7fb492800640 1 --2- 172.21.15.45:0/2837034753 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb49419a6d0 0x7fb49419ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:17:29.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:29.509+0000 7fb491e00640 1 --2- 172.21.15.45:0/2837034753 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb494102010 0x7fb4941a0610 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fb48400e9f0 tx=0x7fb48400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:17:29.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:29.509+0000 7fb48aa00640 1 -- 172.21.15.45:0/2837034753 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb48400cd80 con 0x7fb494102010 2024-04-16T17:17:29.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:29.509+0000 7fb48aa00640 1 -- 172.21.15.45:0/2837034753 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb484010910 con 0x7fb494102010 2024-04-16T17:17:29.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:29.509+0000 7fb493200640 1 -- 172.21.15.45:0/2837034753 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4941a73b0 con 0x7fb494102010 2024-04-16T17:17:29.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:29.509+0000 7fb48aa00640 1 -- 172.21.15.45:0/2837034753 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb484018990 con 0x7fb494102010 2024-04-16T17:17:29.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:29.509+0000 7fb493200640 1 -- 172.21.15.45:0/2837034753 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4941a77b0 con 0x7fb494102010 2024-04-16T17:17:29.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:29.510+0000 7fb493200640 1 -- 172.21.15.45:0/2837034753 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb4941034e0 con 0x7fb494102010 2024-04-16T17:17:29.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:29.512+0000 7fb48aa00640 1 -- 172.21.15.45:0/2837034753 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fb484010ac0 con 0x7fb494102010 2024-04-16T17:17:29.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:29.514+0000 7fb48aa00640 1 --2- 172.21.15.45:0/2837034753 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fb4780a3da0 0x7fb4780a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:29.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:29.514+0000 7fb48aa00640 1 -- 172.21.15.45:0/2837034753 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 9496+0+0 (secure 0 0 0) 0x7fb4840d1090 con 0x7fb494102010 2024-04-16T17:17:29.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:29.514+0000 7fb491400640 1 --2- 172.21.15.45:0/2837034753 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fb4780a3da0 0x7fb4780a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:29.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:29.515+0000 7fb491400640 1 --2- 172.21.15.45:0/2837034753 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fb4780a3da0 0x7fb4780a6260 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fb480009230 tx=0x7fb480005fd0 comp rx=0 tx=0).ready entity=mgr.25411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:17:29.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:29.516+0000 7fb48aa00640 1 -- 172.21.15.45:0/2837034753 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb48409ccb0 con 0x7fb494102010 2024-04-16T17:17:29.676 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:17:29 smithi104 ceph-mon[36457]: from='client.? 172.21.15.45:0/3424148822' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T17:17:29.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:29.724+0000 7fb493200640 1 -- 172.21.15.45:0/2837034753 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fb494069190 con 0x7fb494102010 2024-04-16T17:17:29.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:29.725+0000 7fb48aa00640 1 -- 172.21.15.45:0/2837034753 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v106) ==== 93+0+10 (secure 0 0 0) 0x7fb4840a0fb0 con 0x7fb494102010 2024-04-16T17:17:29.724 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 1 2024-04-16T17:17:29.726 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:17:29 smithi045 ceph-mon[31676]: from='client.? 172.21.15.45:0/3424148822' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T17:17:29.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:29.728+0000 7fb493200640 1 -- 172.21.15.45:0/2837034753 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fb4780a3da0 msgr2=0x7fb4780a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:17:29.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:29.728+0000 7fb493200640 1 --2- 172.21.15.45:0/2837034753 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fb4780a3da0 0x7fb4780a6260 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fb480009230 tx=0x7fb480005fd0 comp rx=0 tx=0).stop 2024-04-16T17:17:29.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:29.728+0000 7fb493200640 1 -- 172.21.15.45:0/2837034753 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb494102010 msgr2=0x7fb4941a0610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:17:29.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:29.728+0000 7fb493200640 1 --2- 172.21.15.45:0/2837034753 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb494102010 0x7fb4941a0610 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fb48400e9f0 tx=0x7fb48400eeb0 comp rx=0 tx=0).stop 2024-04-16T17:17:29.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:29.728+0000 7fb493200640 1 -- 172.21.15.45:0/2837034753 shutdown_connections 2024-04-16T17:17:29.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:29.728+0000 7fb493200640 1 --2- 172.21.15.45:0/2837034753 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fb4780a3da0 0x7fb4780a6260 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:29.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:29.728+0000 7fb493200640 1 --2- 172.21.15.45:0/2837034753 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb49419a6d0 0x7fb49419ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:29.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:29.728+0000 7fb493200640 1 --2- 172.21.15.45:0/2837034753 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb4941070e0 0x7fb4941a0b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:29.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:29.728+0000 7fb493200640 1 --2- 172.21.15.45:0/2837034753 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb494102010 0x7fb4941a0610 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:29.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:29.728+0000 7fb493200640 1 -- 172.21.15.45:0/2837034753 >> 172.21.15.45:0/2837034753 conn(0x7fb4940fd8d0 msgr2=0x7fb4940ffc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:17:29.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:29.728+0000 7fb493200640 1 -- 172.21.15.45:0/2837034753 shutdown_connections 2024-04-16T17:17:29.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:29.728+0000 7fb493200640 1 -- 172.21.15.45:0/2837034753 wait complete. 2024-04-16T17:17:29.749 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-16T17:17:29.804 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:17:29 smithi117 ceph-mon[36730]: from='client.? 172.21.15.45:0/3424148822' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T17:17:29.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:29.976+0000 7f3f3a000640 1 -- 172.21.15.45:0/2529818481 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3f34103980 msgr2=0x7f3f3410be60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:17:29.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:29.976+0000 7f3f3a000640 1 --2- 172.21.15.45:0/2529818481 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3f34103980 0x7f3f3410be60 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f3f20009860 tx=0x7f3f2002bc00 comp rx=0 tx=0).stop 2024-04-16T17:17:29.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:29.976+0000 7f3f3a000640 1 -- 172.21.15.45:0/2529818481 shutdown_connections 2024-04-16T17:17:29.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:29.976+0000 7f3f3a000640 1 --2- 172.21.15.45:0/2529818481 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3f3410c480 0x7f3f3410e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:29.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:29.976+0000 7f3f3a000640 1 --2- 172.21.15.45:0/2529818481 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3f34103980 0x7f3f3410be60 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:29.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:29.976+0000 7f3f3a000640 1 --2- 172.21.15.45:0/2529818481 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3f34103060 0x7f3f34103440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:29.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:29.976+0000 7f3f3a000640 1 -- 172.21.15.45:0/2529818481 >> 172.21.15.45:0/2529818481 conn(0x7f3f340fc880 msgr2=0x7f3f340feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:17:29.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:29.977+0000 7f3f3a000640 1 -- 172.21.15.45:0/2529818481 shutdown_connections 2024-04-16T17:17:29.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:29.977+0000 7f3f3a000640 1 -- 172.21.15.45:0/2529818481 wait complete. 2024-04-16T17:17:29.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:29.978+0000 7f3f3a000640 1 Processor -- start 2024-04-16T17:17:29.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:29.978+0000 7f3f3a000640 1 -- start start 2024-04-16T17:17:29.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:29.979+0000 7f3f3a000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3f34103060 0x7f3f3419e430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:29.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:29.979+0000 7f3f3a000640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3f34103980 0x7f3f3419e970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:29.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:29.979+0000 7f3f3a000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3f3410c480 0x7f3f341985e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:29.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:29.979+0000 7f3f3a000640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f3f34112260 con 0x7f3f34103060 2024-04-16T17:17:29.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:29.979+0000 7f3f3a000640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f3f341120e0 con 0x7f3f34103980 2024-04-16T17:17:29.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:29.979+0000 7f3f3a000640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f3f341123e0 con 0x7f3f3410c480 2024-04-16T17:17:29.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:29.979+0000 7f3f38c00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3f34103060 0x7f3f3419e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:29.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:29.979+0000 7f3f38c00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3f34103060 0x7f3f3419e430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55122/0 (socket says 172.21.15.45:55122) 2024-04-16T17:17:29.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:29.979+0000 7f3f38c00640 1 -- 172.21.15.45:0/304335570 learned_addr learned my addr 172.21.15.45:0/304335570 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:17:29.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:29.979+0000 7f3f33e00640 1 --2- 172.21.15.45:0/304335570 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3f34103980 0x7f3f3419e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:29.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:29.979+0000 7f3f39600640 1 --2- 172.21.15.45:0/304335570 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3f3410c480 0x7f3f341985e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:29.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:29.979+0000 7f3f38c00640 1 -- 172.21.15.45:0/304335570 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3f3410c480 msgr2=0x7f3f341985e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:17:29.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:29.979+0000 7f3f38c00640 1 --2- 172.21.15.45:0/304335570 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3f3410c480 0x7f3f341985e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:29.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:29.980+0000 7f3f38c00640 1 -- 172.21.15.45:0/304335570 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3f34103980 msgr2=0x7f3f3419e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:17:29.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:29.980+0000 7f3f38c00640 1 --2- 172.21.15.45:0/304335570 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3f34103980 0x7f3f3419e970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:29.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:29.980+0000 7f3f38c00640 1 -- 172.21.15.45:0/304335570 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f34198d60 con 0x7f3f34103060 2024-04-16T17:17:29.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:29.980+0000 7f3f38c00640 1 --2- 172.21.15.45:0/304335570 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3f34103060 0x7f3f3419e430 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f3f2800e9f0 tx=0x7f3f2800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:17:29.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:29.980+0000 7f3f31600640 1 -- 172.21.15.45:0/304335570 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f2800cd80 con 0x7f3f34103060 2024-04-16T17:17:29.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:29.980+0000 7f3f31600640 1 -- 172.21.15.45:0/304335570 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3f28005500 con 0x7f3f34103060 2024-04-16T17:17:29.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:29.980+0000 7f3f3a000640 1 -- 172.21.15.45:0/304335570 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f34199050 con 0x7f3f34103060 2024-04-16T17:17:29.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:29.980+0000 7f3f31600640 1 -- 172.21.15.45:0/304335570 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f28010690 con 0x7f3f34103060 2024-04-16T17:17:29.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:29.980+0000 7f3f3a000640 1 -- 172.21.15.45:0/304335570 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f341a52f0 con 0x7f3f34103060 2024-04-16T17:17:29.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:29.982+0000 7f3f3a000640 1 -- 172.21.15.45:0/304335570 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f34068d40 con 0x7f3f34103060 2024-04-16T17:17:29.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:29.984+0000 7f3f31600640 1 -- 172.21.15.45:0/304335570 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f3f280046c0 con 0x7f3f34103060 2024-04-16T17:17:29.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:29.985+0000 7f3f31600640 1 --2- 172.21.15.45:0/304335570 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f3f1c0a3da0 0x7f3f1c0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:29.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:29.985+0000 7f3f31600640 1 -- 172.21.15.45:0/304335570 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 9496+0+0 (secure 0 0 0) 0x7f3f280d0fc0 con 0x7f3f34103060 2024-04-16T17:17:29.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:29.985+0000 7f3f33e00640 1 --2- 172.21.15.45:0/304335570 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f3f1c0a3da0 0x7f3f1c0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:29.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:29.986+0000 7f3f33e00640 1 --2- 172.21.15.45:0/304335570 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f3f1c0a3da0 0x7f3f1c0a6260 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f3f2002c210 tx=0x7f3f2002e040 comp rx=0 tx=0).ready entity=mgr.25411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:17:29.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:29.990+0000 7f3f31600640 1 -- 172.21.15.45:0/304335570 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3f2809d230 con 0x7f3f34103060 2024-04-16T17:17:30.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:30.197+0000 7f3f3a000640 1 -- 172.21.15.45:0/304335570 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f3f34107520 con 0x7f3f34103060 2024-04-16T17:17:30.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:30.198+0000 7f3f31600640 1 -- 172.21.15.45:0/304335570 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v106) ==== 104+0+11 (secure 0 0 0) 0x7f3f2809cb30 con 0x7f3f34103060 2024-04-16T17:17:30.198 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 32 2024-04-16T17:17:30.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:30.201+0000 7f3f3a000640 1 -- 172.21.15.45:0/304335570 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f3f1c0a3da0 msgr2=0x7f3f1c0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:17:30.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:30.201+0000 7f3f3a000640 1 --2- 172.21.15.45:0/304335570 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f3f1c0a3da0 0x7f3f1c0a6260 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f3f2002c210 tx=0x7f3f2002e040 comp rx=0 tx=0).stop 2024-04-16T17:17:30.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:30.201+0000 7f3f3a000640 1 -- 172.21.15.45:0/304335570 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3f34103060 msgr2=0x7f3f3419e430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:17:30.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:30.201+0000 7f3f3a000640 1 --2- 172.21.15.45:0/304335570 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3f34103060 0x7f3f3419e430 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f3f2800e9f0 tx=0x7f3f2800edf0 comp rx=0 tx=0).stop 2024-04-16T17:17:30.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:30.201+0000 7f3f3a000640 1 -- 172.21.15.45:0/304335570 shutdown_connections 2024-04-16T17:17:30.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:30.201+0000 7f3f3a000640 1 --2- 172.21.15.45:0/304335570 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f3f1c0a3da0 0x7f3f1c0a6260 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:30.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:30.201+0000 7f3f3a000640 1 --2- 172.21.15.45:0/304335570 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3f3410c480 0x7f3f341985e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:30.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:30.201+0000 7f3f3a000640 1 --2- 172.21.15.45:0/304335570 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3f34103980 0x7f3f3419e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:30.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:30.201+0000 7f3f3a000640 1 --2- 172.21.15.45:0/304335570 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3f34103060 0x7f3f3419e430 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:30.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:30.201+0000 7f3f3a000640 1 -- 172.21.15.45:0/304335570 >> 172.21.15.45:0/304335570 conn(0x7f3f340fc880 msgr2=0x7f3f340fe140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:17:30.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:30.201+0000 7f3f3a000640 1 -- 172.21.15.45:0/304335570 shutdown_connections 2024-04-16T17:17:30.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:30.201+0000 7f3f3a000640 1 -- 172.21.15.45:0/304335570 wait complete. 2024-04-16T17:17:30.219 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-16T17:17:30.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:30.451+0000 7fb2ca200640 1 -- 172.21.15.45:0/2098960008 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb2c4107260 msgr2=0x7fb2c410ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:17:30.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:30.451+0000 7fb2ca200640 1 --2- 172.21.15.45:0/2098960008 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb2c4107260 0x7fb2c410ede0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fb2bc009ca0 tx=0x7fb2bc02c100 comp rx=0 tx=0).stop 2024-04-16T17:17:30.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:30.452+0000 7fb2ca200640 1 -- 172.21.15.45:0/2098960008 shutdown_connections 2024-04-16T17:17:30.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:30.452+0000 7fb2ca200640 1 --2- 172.21.15.45:0/2098960008 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb2c4101400 0x7fb2c4106b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:30.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:30.452+0000 7fb2ca200640 1 --2- 172.21.15.45:0/2098960008 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb2c4107260 0x7fb2c410ede0 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:30.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:30.452+0000 7fb2ca200640 1 --2- 172.21.15.45:0/2098960008 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb2c4100a50 0x7fb2c4100e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:30.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:30.452+0000 7fb2ca200640 1 -- 172.21.15.45:0/2098960008 >> 172.21.15.45:0/2098960008 conn(0x7fb2c4076090 msgr2=0x7fb2c40764a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:17:30.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:30.452+0000 7fb2ca200640 1 -- 172.21.15.45:0/2098960008 shutdown_connections 2024-04-16T17:17:30.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:30.452+0000 7fb2ca200640 1 -- 172.21.15.45:0/2098960008 wait complete. 2024-04-16T17:17:30.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:30.453+0000 7fb2ca200640 1 Processor -- start 2024-04-16T17:17:30.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:30.453+0000 7fb2ca200640 1 -- start start 2024-04-16T17:17:30.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:30.453+0000 7fb2ca200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb2c4100a50 0x7fb2c41a0660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:30.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:30.453+0000 7fb2ca200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb2c4107260 0x7fb2c41a0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:30.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:30.453+0000 7fb2ca200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb2c419a7d0 0x7fb2c419ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:30.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:30.453+0000 7fb2ca200640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fb2c41122d0 con 0x7fb2c4100a50 2024-04-16T17:17:30.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:30.453+0000 7fb2ca200640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fb2c4112150 con 0x7fb2c419a7d0 2024-04-16T17:17:30.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:30.453+0000 7fb2ca200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb2c4112450 con 0x7fb2c4107260 2024-04-16T17:17:30.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:30.454+0000 7fb2c8e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb2c4100a50 0x7fb2c41a0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:30.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:30.454+0000 7fb2c9800640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb2c419a7d0 0x7fb2c419ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:30.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:30.454+0000 7fb2c8e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb2c4100a50 0x7fb2c41a0660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55132/0 (socket says 172.21.15.45:55132) 2024-04-16T17:17:30.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:30.454+0000 7fb2c8e00640 1 -- 172.21.15.45:0/3453408999 learned_addr learned my addr 172.21.15.45:0/3453408999 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:17:30.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:30.454+0000 7fb2bbe00640 1 --2- 172.21.15.45:0/3453408999 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb2c4107260 0x7fb2c41a0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:30.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:30.454+0000 7fb2c8e00640 1 -- 172.21.15.45:0/3453408999 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb2c4107260 msgr2=0x7fb2c41a0ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:17:30.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:30.454+0000 7fb2c8e00640 1 --2- 172.21.15.45:0/3453408999 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb2c4107260 0x7fb2c41a0ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:30.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:30.454+0000 7fb2c8e00640 1 -- 172.21.15.45:0/3453408999 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb2c419a7d0 msgr2=0x7fb2c419ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:17:30.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:30.454+0000 7fb2c8e00640 1 --2- 172.21.15.45:0/3453408999 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb2c419a7d0 0x7fb2c419ac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:30.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:30.454+0000 7fb2c8e00640 1 -- 172.21.15.45:0/3453408999 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2c419b360 con 0x7fb2c4100a50 2024-04-16T17:17:30.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:30.454+0000 7fb2c9800640 1 --2- 172.21.15.45:0/3453408999 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb2c419a7d0 0x7fb2c419ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:17:30.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:30.454+0000 7fb2c8e00640 1 --2- 172.21.15.45:0/3453408999 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb2c4100a50 0x7fb2c41a0660 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fb2b400e9f0 tx=0x7fb2b400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:17:30.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:30.455+0000 7fb2c2000640 1 -- 172.21.15.45:0/3453408999 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2b400cd80 con 0x7fb2c4100a50 2024-04-16T17:17:30.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:30.455+0000 7fb2c2000640 1 -- 172.21.15.45:0/3453408999 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb2b4010910 con 0x7fb2c4100a50 2024-04-16T17:17:30.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:30.455+0000 7fb2ca200640 1 -- 172.21.15.45:0/3453408999 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2c419b650 con 0x7fb2c4100a50 2024-04-16T17:17:30.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:30.455+0000 7fb2c2000640 1 -- 172.21.15.45:0/3453408999 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2b4018990 con 0x7fb2c4100a50 2024-04-16T17:17:30.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:30.455+0000 7fb2ca200640 1 -- 172.21.15.45:0/3453408999 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2c41a7880 con 0x7fb2c4100a50 2024-04-16T17:17:30.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:30.456+0000 7fb2ca200640 1 -- 172.21.15.45:0/3453408999 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb2c406a110 con 0x7fb2c4100a50 2024-04-16T17:17:30.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:30.458+0000 7fb2c2000640 1 -- 172.21.15.45:0/3453408999 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fb2b4010ac0 con 0x7fb2c4100a50 2024-04-16T17:17:30.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:30.460+0000 7fb2c2000640 1 --2- 172.21.15.45:0/3453408999 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fb2a80a3da0 0x7fb2a80a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:30.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:30.460+0000 7fb2c2000640 1 -- 172.21.15.45:0/3453408999 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 9496+0+0 (secure 0 0 0) 0x7fb2b40d0eb0 con 0x7fb2c4100a50 2024-04-16T17:17:30.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:30.460+0000 7fb2bbe00640 1 --2- 172.21.15.45:0/3453408999 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fb2a80a3da0 0x7fb2a80a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:30.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:30.461+0000 7fb2bbe00640 1 --2- 172.21.15.45:0/3453408999 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fb2a80a3da0 0x7fb2a80a6260 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fb2c410f190 tx=0x7fb2ac005f90 comp rx=0 tx=0).ready entity=mgr.25411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:17:30.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:30.464+0000 7fb2c2000640 1 -- 172.21.15.45:0/3453408999 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb2c419b650 con 0x7fb2c4100a50 2024-04-16T17:17:30.476 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:17:30 smithi045 ceph-mon[31676]: pgmap v336: 129 pgs: 129 active+clean; 55 MiB data, 874 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T17:17:30.476 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:17:30 smithi045 ceph-mon[31676]: from='client.? 172.21.15.45:0/2837034753' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-16T17:17:30.477 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:17:30 smithi045 ceph-mon[31676]: from='client.? 172.21.15.45:0/304335570' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-16T17:17:30.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:30.673+0000 7fb2ca200640 1 -- 172.21.15.45:0/3453408999 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fb2c4104040 con 0x7fb2c4100a50 2024-04-16T17:17:30.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:30.675+0000 7fb2c2000640 1 -- 172.21.15.45:0/3453408999 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v106) ==== 100+0+11 (secure 0 0 0) 0x7fb2b409cad0 con 0x7fb2c4100a50 2024-04-16T17:17:30.674 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 64 2024-04-16T17:17:30.677 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:17:30 smithi104 ceph-mon[36457]: pgmap v336: 129 pgs: 129 active+clean; 55 MiB data, 874 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T17:17:30.677 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:17:30 smithi104 ceph-mon[36457]: from='client.? 172.21.15.45:0/2837034753' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-16T17:17:30.677 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:17:30 smithi104 ceph-mon[36457]: from='client.? 172.21.15.45:0/304335570' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-16T17:17:30.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:30.677+0000 7fb2ca200640 1 -- 172.21.15.45:0/3453408999 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fb2a80a3da0 msgr2=0x7fb2a80a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:17:30.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:30.677+0000 7fb2ca200640 1 --2- 172.21.15.45:0/3453408999 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fb2a80a3da0 0x7fb2a80a6260 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fb2c410f190 tx=0x7fb2ac005f90 comp rx=0 tx=0).stop 2024-04-16T17:17:30.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:30.678+0000 7fb2ca200640 1 -- 172.21.15.45:0/3453408999 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb2c4100a50 msgr2=0x7fb2c41a0660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:17:30.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:30.678+0000 7fb2ca200640 1 --2- 172.21.15.45:0/3453408999 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb2c4100a50 0x7fb2c41a0660 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fb2b400e9f0 tx=0x7fb2b400eeb0 comp rx=0 tx=0).stop 2024-04-16T17:17:30.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:30.678+0000 7fb2ca200640 1 -- 172.21.15.45:0/3453408999 shutdown_connections 2024-04-16T17:17:30.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:30.678+0000 7fb2ca200640 1 --2- 172.21.15.45:0/3453408999 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fb2a80a3da0 0x7fb2a80a6260 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:30.679 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:30.678+0000 7fb2ca200640 1 --2- 172.21.15.45:0/3453408999 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb2c4107260 0x7fb2c41a0ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:30.679 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:30.678+0000 7fb2ca200640 1 --2- 172.21.15.45:0/3453408999 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb2c419a7d0 0x7fb2c419ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:30.679 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:30.678+0000 7fb2ca200640 1 --2- 172.21.15.45:0/3453408999 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb2c4100a50 0x7fb2c41a0660 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:30.679 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:30.678+0000 7fb2ca200640 1 -- 172.21.15.45:0/3453408999 >> 172.21.15.45:0/3453408999 conn(0x7fb2c4076090 msgr2=0x7fb2c40feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:17:30.679 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:30.678+0000 7fb2ca200640 1 -- 172.21.15.45:0/3453408999 shutdown_connections 2024-04-16T17:17:30.679 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:30.678+0000 7fb2ca200640 1 -- 172.21.15.45:0/3453408999 wait complete. 2024-04-16T17:17:30.698 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-16T17:17:30.804 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:17:30 smithi117 ceph-mon[36730]: pgmap v336: 129 pgs: 129 active+clean; 55 MiB data, 874 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T17:17:30.804 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:17:30 smithi117 ceph-mon[36730]: from='client.? 172.21.15.45:0/2837034753' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-16T17:17:30.804 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:17:30 smithi117 ceph-mon[36730]: from='client.? 172.21.15.45:0/304335570' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-16T17:17:30.931 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:30.931+0000 7fe993200640 1 -- 172.21.15.45:0/2966574243 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe994102d30 msgr2=0x7fe994103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:17:30.931 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:30.931+0000 7fe993200640 1 --2- 172.21.15.45:0/2966574243 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe994102d30 0x7fe994103110 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fe984009860 tx=0x7fe98402bb70 comp rx=0 tx=0).stop 2024-04-16T17:17:30.931 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:30.931+0000 7fe993200640 1 -- 172.21.15.45:0/2966574243 shutdown_connections 2024-04-16T17:17:30.931 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:30.931+0000 7fe993200640 1 --2- 172.21.15.45:0/2966574243 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe9941011a0 0x7fe994110f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:30.931 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:30.931+0000 7fe993200640 1 --2- 172.21.15.45:0/2966574243 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe9941007e0 0x7fe994100c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:30.931 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:30.931+0000 7fe993200640 1 --2- 172.21.15.45:0/2966574243 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe994102d30 0x7fe994103110 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:30.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:30.931+0000 7fe993200640 1 -- 172.21.15.45:0/2966574243 >> 172.21.15.45:0/2966574243 conn(0x7fe994076090 msgr2=0x7fe9940764a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:17:30.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:30.931+0000 7fe993200640 1 -- 172.21.15.45:0/2966574243 shutdown_connections 2024-04-16T17:17:30.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:30.931+0000 7fe993200640 1 -- 172.21.15.45:0/2966574243 wait complete. 2024-04-16T17:17:30.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:30.932+0000 7fe993200640 1 Processor -- start 2024-04-16T17:17:30.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:30.933+0000 7fe993200640 1 -- start start 2024-04-16T17:17:30.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:30.933+0000 7fe993200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe9941007e0 0x7fe9941a0660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:30.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:30.933+0000 7fe993200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe9941011a0 0x7fe9941a0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:30.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:30.933+0000 7fe993200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe994102d30 0x7fe99419a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:30.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:30.933+0000 7fe993200640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fe994114510 con 0x7fe9941011a0 2024-04-16T17:17:30.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:30.933+0000 7fe993200640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fe994114390 con 0x7fe994102d30 2024-04-16T17:17:30.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:30.933+0000 7fe993200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fe994114690 con 0x7fe9941007e0 2024-04-16T17:17:30.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:30.933+0000 7fe989400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe9941011a0 0x7fe9941a0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:30.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:30.933+0000 7fe989400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe9941011a0 0x7fe9941a0ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55156/0 (socket says 172.21.15.45:55156) 2024-04-16T17:17:30.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:30.933+0000 7fe989400640 1 -- 172.21.15.45:0/780442721 learned_addr learned my addr 172.21.15.45:0/780442721 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:17:30.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:30.933+0000 7fe991e00640 1 --2- 172.21.15.45:0/780442721 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe9941007e0 0x7fe9941a0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:30.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:30.933+0000 7fe992800640 1 --2- 172.21.15.45:0/780442721 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe994102d30 0x7fe99419a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:30.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:30.934+0000 7fe992800640 1 -- 172.21.15.45:0/780442721 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe9941007e0 msgr2=0x7fe9941a0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:17:30.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:30.934+0000 7fe992800640 1 --2- 172.21.15.45:0/780442721 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe9941007e0 0x7fe9941a0660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:30.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:30.934+0000 7fe992800640 1 -- 172.21.15.45:0/780442721 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe9941011a0 msgr2=0x7fe9941a0ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:17:30.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:30.934+0000 7fe992800640 1 --2- 172.21.15.45:0/780442721 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe9941011a0 0x7fe9941a0ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:30.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:30.934+0000 7fe992800640 1 -- 172.21.15.45:0/780442721 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe99419afa0 con 0x7fe994102d30 2024-04-16T17:17:30.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:30.934+0000 7fe989400640 1 --2- 172.21.15.45:0/780442721 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe9941011a0 0x7fe9941a0ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:17:30.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:30.934+0000 7fe991e00640 1 --2- 172.21.15.45:0/780442721 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe9941007e0 0x7fe9941a0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:17:30.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:30.934+0000 7fe992800640 1 --2- 172.21.15.45:0/780442721 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe994102d30 0x7fe99419a7d0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fe98c00ef00 tx=0x7fe98c00c740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:17:30.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:30.934+0000 7fe98b400640 1 -- 172.21.15.45:0/780442721 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe98c004b20 con 0x7fe994102d30 2024-04-16T17:17:30.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:30.934+0000 7fe98b400640 1 -- 172.21.15.45:0/780442721 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe98c003e50 con 0x7fe994102d30 2024-04-16T17:17:30.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:30.934+0000 7fe993200640 1 -- 172.21.15.45:0/780442721 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe99419b290 con 0x7fe994102d30 2024-04-16T17:17:30.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:30.934+0000 7fe98b400640 1 -- 172.21.15.45:0/780442721 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe98c00b570 con 0x7fe994102d30 2024-04-16T17:17:30.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:30.935+0000 7fe993200640 1 -- 172.21.15.45:0/780442721 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9941a74b0 con 0x7fe994102d30 2024-04-16T17:17:30.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:30.936+0000 7fe993200640 1 -- 172.21.15.45:0/780442721 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe99406a1f0 con 0x7fe994102d30 2024-04-16T17:17:30.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:30.938+0000 7fe98b400640 1 -- 172.21.15.45:0/780442721 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fe98c01d020 con 0x7fe994102d30 2024-04-16T17:17:30.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:30.939+0000 7fe98b400640 1 --2- 172.21.15.45:0/780442721 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fe9780a3d20 0x7fe9780a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:30.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:30.939+0000 7fe98b400640 1 -- 172.21.15.45:0/780442721 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 9496+0+0 (secure 0 0 0) 0x7fe98c0d0aa0 con 0x7fe994102d30 2024-04-16T17:17:30.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:30.940+0000 7fe991e00640 1 --2- 172.21.15.45:0/780442721 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fe9780a3d20 0x7fe9780a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:30.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:30.940+0000 7fe991e00640 1 --2- 172.21.15.45:0/780442721 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fe9780a3d20 0x7fe9780a61e0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fe984009830 tx=0x7fe98402e040 comp rx=0 tx=0).ready entity=mgr.25411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:17:30.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:30.944+0000 7fe98b400640 1 -- 172.21.15.45:0/780442721 <== mon.2 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe98c09cd10 con 0x7fe994102d30 2024-04-16T17:17:31.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:31.154+0000 7fe993200640 1 -- 172.21.15.45:0/780442721 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fe994069190 con 0x7fe994102d30 2024-04-16T17:17:31.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:31.155+0000 7fe98b400640 1 -- 172.21.15.45:0/780442721 <== mon.2 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v106) ==== 103+0+11 (secure 0 0 0) 0x7fe98c09c610 con 0x7fe994102d30 2024-04-16T17:17:31.154 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 32 2024-04-16T17:17:31.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:31.158+0000 7fe993200640 1 -- 172.21.15.45:0/780442721 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fe9780a3d20 msgr2=0x7fe9780a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:17:31.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:31.158+0000 7fe993200640 1 --2- 172.21.15.45:0/780442721 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fe9780a3d20 0x7fe9780a61e0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fe984009830 tx=0x7fe98402e040 comp rx=0 tx=0).stop 2024-04-16T17:17:31.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:31.158+0000 7fe993200640 1 -- 172.21.15.45:0/780442721 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe994102d30 msgr2=0x7fe99419a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:17:31.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:31.158+0000 7fe993200640 1 --2- 172.21.15.45:0/780442721 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe994102d30 0x7fe99419a7d0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fe98c00ef00 tx=0x7fe98c00c740 comp rx=0 tx=0).stop 2024-04-16T17:17:31.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:31.158+0000 7fe993200640 1 -- 172.21.15.45:0/780442721 shutdown_connections 2024-04-16T17:17:31.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:31.158+0000 7fe993200640 1 --2- 172.21.15.45:0/780442721 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fe9780a3d20 0x7fe9780a61e0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:31.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:31.158+0000 7fe993200640 1 --2- 172.21.15.45:0/780442721 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe994102d30 0x7fe99419a7d0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:31.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:31.158+0000 7fe993200640 1 --2- 172.21.15.45:0/780442721 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe9941011a0 0x7fe9941a0ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:31.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:31.158+0000 7fe993200640 1 --2- 172.21.15.45:0/780442721 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe9941007e0 0x7fe9941a0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:31.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:31.158+0000 7fe993200640 1 -- 172.21.15.45:0/780442721 >> 172.21.15.45:0/780442721 conn(0x7fe994076090 msgr2=0x7fe9940fe310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:17:31.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:31.158+0000 7fe993200640 1 -- 172.21.15.45:0/780442721 shutdown_connections 2024-04-16T17:17:31.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:31.158+0000 7fe993200640 1 -- 172.21.15.45:0/780442721 wait complete. 2024-04-16T17:17:31.179 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-04-16T17:17:31.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:31.410+0000 7f9387e00640 1 -- 172.21.15.45:0/4292968559 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f938810e490 msgr2=0x7f938810e870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:17:31.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:31.410+0000 7f9387e00640 1 --2- 172.21.15.45:0/4292968559 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f938810e490 0x7f938810e870 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f9378009930 tx=0x7f937802bc30 comp rx=0 tx=0).stop 2024-04-16T17:17:31.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:31.411+0000 7f9387e00640 1 -- 172.21.15.45:0/4292968559 shutdown_connections 2024-04-16T17:17:31.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:31.411+0000 7f9387e00640 1 --2- 172.21.15.45:0/4292968559 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9388108400 0x7f9388108880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:31.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:31.411+0000 7f9387e00640 1 --2- 172.21.15.45:0/4292968559 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9388108dc0 0x7f93881152c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:31.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:31.411+0000 7f9387e00640 1 --2- 172.21.15.45:0/4292968559 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f938810e490 0x7f938810e870 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:31.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:31.411+0000 7f9387e00640 1 -- 172.21.15.45:0/4292968559 >> 172.21.15.45:0/4292968559 conn(0x7f9388077b80 msgr2=0x7f9388079fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:17:31.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:31.411+0000 7f9387e00640 1 -- 172.21.15.45:0/4292968559 shutdown_connections 2024-04-16T17:17:31.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:31.411+0000 7f9387e00640 1 -- 172.21.15.45:0/4292968559 wait complete. 2024-04-16T17:17:31.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:31.412+0000 7f9387e00640 1 Processor -- start 2024-04-16T17:17:31.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:31.412+0000 7f9387e00640 1 -- start start 2024-04-16T17:17:31.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:31.412+0000 7f9387e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9388108400 0x7f93881a4900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:31.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:31.412+0000 7f9387e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9388108dc0 0x7f93881a4e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:31.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:31.412+0000 7f9387e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f938810e490 0x7f938819eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:31.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:31.412+0000 7f9387e00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f93881187e0 con 0x7f9388108400 2024-04-16T17:17:31.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:31.412+0000 7f9387e00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f9388118660 con 0x7f9388108dc0 2024-04-16T17:17:31.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:31.412+0000 7f9387e00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f9388118960 con 0x7f938810e490 2024-04-16T17:17:31.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:31.412+0000 7f9386a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9388108400 0x7f93881a4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:31.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:31.413+0000 7f9386a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9388108400 0x7f93881a4900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55176/0 (socket says 172.21.15.45:55176) 2024-04-16T17:17:31.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:31.413+0000 7f9386a00640 1 -- 172.21.15.45:0/4190718114 learned_addr learned my addr 172.21.15.45:0/4190718114 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:17:31.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:31.413+0000 7f9386000640 1 --2- 172.21.15.45:0/4190718114 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9388108dc0 0x7f93881a4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:31.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:31.413+0000 7f9387400640 1 --2- 172.21.15.45:0/4190718114 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f938810e490 0x7f938819eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:31.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:31.413+0000 7f9386a00640 1 -- 172.21.15.45:0/4190718114 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f938810e490 msgr2=0x7f938819eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:17:31.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:31.413+0000 7f9386a00640 1 --2- 172.21.15.45:0/4190718114 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f938810e490 0x7f938819eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:31.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:31.413+0000 7f9386a00640 1 -- 172.21.15.45:0/4190718114 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9388108dc0 msgr2=0x7f93881a4e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:17:31.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:31.413+0000 7f9386a00640 1 --2- 172.21.15.45:0/4190718114 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9388108dc0 0x7f93881a4e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:31.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:31.413+0000 7f9386a00640 1 -- 172.21.15.45:0/4190718114 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f938819f280 con 0x7f9388108400 2024-04-16T17:17:31.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:31.413+0000 7f9386a00640 1 --2- 172.21.15.45:0/4190718114 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9388108400 0x7f93881a4900 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f9378009900 tx=0x7f9378004360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:17:31.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:31.414+0000 7f937f400640 1 -- 172.21.15.45:0/4190718114 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9378043020 con 0x7f9388108400 2024-04-16T17:17:31.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:31.414+0000 7f937f400640 1 -- 172.21.15.45:0/4190718114 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f93780054e0 con 0x7f9388108400 2024-04-16T17:17:31.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:31.414+0000 7f9387e00640 1 -- 172.21.15.45:0/4190718114 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f938819f510 con 0x7f9388108400 2024-04-16T17:17:31.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:31.414+0000 7f937f400640 1 -- 172.21.15.45:0/4190718114 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9378003b80 con 0x7f9388108400 2024-04-16T17:17:31.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:31.414+0000 7f9387e00640 1 -- 172.21.15.45:0/4190718114 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93881ab7c0 con 0x7f9388108400 2024-04-16T17:17:31.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:31.415+0000 7f9387e00640 1 -- 172.21.15.45:0/4190718114 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9388069190 con 0x7f9388108400 2024-04-16T17:17:31.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:31.417+0000 7f937f400640 1 -- 172.21.15.45:0/4190718114 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f9378046020 con 0x7f9388108400 2024-04-16T17:17:31.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:31.418+0000 7f937f400640 1 --2- 172.21.15.45:0/4190718114 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f936c0a3d20 0x7f936c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:17:31.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:31.419+0000 7f937f400640 1 -- 172.21.15.45:0/4190718114 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 9496+0+0 (secure 0 0 0) 0x7f93780f2bd0 con 0x7f9388108400 2024-04-16T17:17:31.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:31.419+0000 7f9386000640 1 --2- 172.21.15.45:0/4190718114 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f936c0a3d20 0x7f936c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:17:31.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:31.419+0000 7f9386000640 1 --2- 172.21.15.45:0/4190718114 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f936c0a3d20 0x7f936c0a61e0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f938807a280 tx=0x7f937000b390 comp rx=0 tx=0).ready entity=mgr.25411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:17:31.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:31.423+0000 7f937f400640 1 -- 172.21.15.45:0/4190718114 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f937804d020 con 0x7f9388108400 2024-04-16T17:17:31.476 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:17:31 smithi045 ceph-mon[31676]: from='client.? 172.21.15.45:0/3453408999' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-16T17:17:31.477 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:17:31 smithi045 ceph-mon[31676]: from='client.? 172.21.15.45:0/780442721' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-16T17:17:31.677 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:17:31 smithi104 ceph-mon[36457]: from='client.? 172.21.15.45:0/3453408999' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-16T17:17:31.677 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:17:31 smithi104 ceph-mon[36457]: from='client.? 172.21.15.45:0/780442721' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-16T17:17:31.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:31.690+0000 7f9387e00640 1 -- 172.21.15.45:0/4190718114 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7f9388110990 con 0x7f9388108400 2024-04-16T17:17:31.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:31.713+0000 7f937f400640 1 -- 172.21.15.45:0/4190718114 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v26) ==== 81+0+746058 (secure 0 0 0) 0x7f93780bef50 con 0x7f9388108400 2024-04-16T17:17:31.721 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-16T17:17:31.721 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-16T17:17:31.721 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 26, 2024-04-16T17:17:31.721 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:31.721 INFO:teuthology.orchestra.run.smithi045.stdout: "active_gid": 25411, 2024-04-16T17:17:31.721 INFO:teuthology.orchestra.run.smithi045.stdout: "active_name": "x", 2024-04-16T17:17:31.721 INFO:teuthology.orchestra.run.smithi045.stdout: "active_addrs": { 2024-04-16T17:17:31.722 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-04-16T17:17:31.722 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-16T17:17:31.722 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v2", 2024-04-16T17:17:31.722 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6800", 2024-04-16T17:17:31.722 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 751128195 2024-04-16T17:17:31.722 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:31.722 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-16T17:17:31.722 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-04-16T17:17:31.722 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6801", 2024-04-16T17:17:31.723 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 751128195 2024-04-16T17:17:31.723 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:17:31.723 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-04-16T17:17:31.723 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:31.723 INFO:teuthology.orchestra.run.smithi045.stdout: "active_addr": "172.21.15.45:6801/751128195", 2024-04-16T17:17:31.723 INFO:teuthology.orchestra.run.smithi045.stdout: "active_change": "2024-04-16T17:06:45.093574+0000", 2024-04-16T17:17:31.723 INFO:teuthology.orchestra.run.smithi045.stdout: "active_mgr_features": 4540701547738038271, 2024-04-16T17:17:31.723 INFO:teuthology.orchestra.run.smithi045.stdout: "available": true, 2024-04-16T17:17:31.723 INFO:teuthology.orchestra.run.smithi045.stdout: "standbys": [ 2024-04-16T17:17:31.724 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-16T17:17:31.724 INFO:teuthology.orchestra.run.smithi045.stdout: "gid": 25408, 2024-04-16T17:17:31.724 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "z", 2024-04-16T17:17:31.724 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr_features": 4540701547738038271, 2024-04-16T17:17:31.724 INFO:teuthology.orchestra.run.smithi045.stdout: "available_modules": [ 2024-04-16T17:17:31.724 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-16T17:17:31.724 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "alerts", 2024-04-16T17:17:31.724 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-16T17:17:31.724 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-16T17:17:31.725 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-16T17:17:31.725 INFO:teuthology.orchestra.run.smithi045.stdout: "interval": { 2024-04-16T17:17:31.725 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "interval", 2024-04-16T17:17:31.725 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-04-16T17:17:31.725 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:31.725 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:31.725 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "60", 2024-04-16T17:17:31.725 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:31.725 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:31.726 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:31.726 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "How frequently to reexamine health status", 2024-04-16T17:17:31.726 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:31.726 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:31.726 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:31.726 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:31.726 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-16T17:17:31.726 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-16T17:17:31.727 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:31.727 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:31.727 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:31.727 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:31.727 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:31.727 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:31.727 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-16T17:17:31.727 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-16T17:17:31.727 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-16T17:17:31.727 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-16T17:17:31.728 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-16T17:17:31.728 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-16T17:17:31.728 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-16T17:17:31.728 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-16T17:17:31.728 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:31.728 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:31.728 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:31.728 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:31.728 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:31.728 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-16T17:17:31.729 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-16T17:17:31.729 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:31.729 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:31.729 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:31.729 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:31.729 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:31.729 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:31.729 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:31.729 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:31.729 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:31.730 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:31.730 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:31.730 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:31.730 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-16T17:17:31.730 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-16T17:17:31.730 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:31.730 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:31.730 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:31.730 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-16T17:17:31.731 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:31.731 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:31.731 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-16T17:17:31.731 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-16T17:17:31.731 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-16T17:17:31.731 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-16T17:17:31.731 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-16T17:17:31.731 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-16T17:17:31.731 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-16T17:17:31.732 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-16T17:17:31.732 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:31.732 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:31.732 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:31.732 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:31.732 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:31.732 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-16T17:17:31.732 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-16T17:17:31.732 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:31.732 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:31.733 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:31.733 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:31.733 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:31.733 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:31.733 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:31.733 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:31.733 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:31.733 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:31.733 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:31.733 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:31.734 INFO:teuthology.orchestra.run.smithi045.stdout: "smtp_destination": { 2024-04-16T17:17:31.734 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "smtp_destination", 2024-04-16T17:17:31.734 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:31.734 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:31.734 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:31.734 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:31.734 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:31.734 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:31.734 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:31.735 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Email address to send alerts to", 2024-04-16T17:17:31.735 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:31.735 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:31.735 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:31.735 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:31.735 INFO:teuthology.orchestra.run.smithi045.stdout: "smtp_from_name": { 2024-04-16T17:17:31.735 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "smtp_from_name", 2024-04-16T17:17:31.735 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:31.735 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:31.736 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:31.736 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "Ceph", 2024-04-16T17:17:31.736 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:31.736 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:31.736 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:31.736 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Email From: name", 2024-04-16T17:17:31.736 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:31.736 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:31.736 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:31.737 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:31.737 INFO:teuthology.orchestra.run.smithi045.stdout: "smtp_host": { 2024-04-16T17:17:31.737 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "smtp_host", 2024-04-16T17:17:31.737 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:31.737 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:31.737 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:31.737 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:31.737 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:31.737 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:31.738 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:31.738 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "SMTP server", 2024-04-16T17:17:31.738 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:31.738 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:31.738 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:31.738 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:31.738 INFO:teuthology.orchestra.run.smithi045.stdout: "smtp_password": { 2024-04-16T17:17:31.738 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "smtp_password", 2024-04-16T17:17:31.738 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:31.739 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:31.739 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:31.739 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:31.739 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:31.739 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:31.739 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:31.739 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Password to authenticate with", 2024-04-16T17:17:31.739 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:31.739 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:31.740 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:31.740 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:31.740 INFO:teuthology.orchestra.run.smithi045.stdout: "smtp_port": { 2024-04-16T17:17:31.740 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "smtp_port", 2024-04-16T17:17:31.740 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-16T17:17:31.740 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:31.740 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:31.740 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "465", 2024-04-16T17:17:31.740 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:31.741 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:31.741 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:31.741 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "SMTP port", 2024-04-16T17:17:31.741 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:31.741 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:31.741 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:31.741 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:31.741 INFO:teuthology.orchestra.run.smithi045.stdout: "smtp_sender": { 2024-04-16T17:17:31.741 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "smtp_sender", 2024-04-16T17:17:31.741 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:31.742 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:31.742 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:31.742 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:31.742 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:31.742 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:31.742 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:31.742 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "SMTP envelope sender", 2024-04-16T17:17:31.742 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:31.742 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:31.742 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:31.743 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:31.743 INFO:teuthology.orchestra.run.smithi045.stdout: "smtp_ssl": { 2024-04-16T17:17:31.743 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "smtp_ssl", 2024-04-16T17:17:31.743 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:31.743 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:31.743 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:31.743 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-16T17:17:31.743 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:31.743 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:31.744 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:31.744 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-16T17:17:31.744 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:31.744 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:31.744 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:31.744 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:31.744 INFO:teuthology.orchestra.run.smithi045.stdout: "smtp_user": { 2024-04-16T17:17:31.744 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "smtp_user", 2024-04-16T17:17:31.744 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:31.744 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:31.745 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:31.745 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:31.745 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:31.745 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:31.745 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:31.745 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "User to authenticate as", 2024-04-16T17:17:31.745 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:31.745 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:31.745 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:31.745 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:17:31.746 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:17:31.746 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:31.746 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-16T17:17:31.746 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "balancer", 2024-04-16T17:17:31.746 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-16T17:17:31.746 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-16T17:17:31.746 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-16T17:17:31.746 INFO:teuthology.orchestra.run.smithi045.stdout: "active": { 2024-04-16T17:17:31.746 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "active", 2024-04-16T17:17:31.747 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:31.747 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:31.747 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:31.747 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-16T17:17:31.747 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:31.747 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:31.747 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:31.747 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "automatically balance PGs across cluster", 2024-04-16T17:17:31.747 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:31.748 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:31.748 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:31.748 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:31.748 INFO:teuthology.orchestra.run.smithi045.stdout: "begin_time": { 2024-04-16T17:17:31.748 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "begin_time", 2024-04-16T17:17:31.748 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:31.748 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:31.748 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:31.748 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0000", 2024-04-16T17:17:31.748 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:31.749 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:31.749 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:31.749 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "beginning time of day to automatically balance", 2024-04-16T17:17:31.749 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-16T17:17:31.749 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:31.749 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:31.749 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:31.749 INFO:teuthology.orchestra.run.smithi045.stdout: "begin_weekday": { 2024-04-16T17:17:31.749 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "begin_weekday", 2024-04-16T17:17:31.750 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "uint", 2024-04-16T17:17:31.750 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:31.750 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:31.750 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0", 2024-04-16T17:17:31.750 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "0", 2024-04-16T17:17:31.750 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "6", 2024-04-16T17:17:31.750 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:31.750 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-16T17:17:31.750 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-16T17:17:31.751 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:31.751 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:31.751 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:31.751 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_compat_max_iterations": { 2024-04-16T17:17:31.751 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "crush_compat_max_iterations", 2024-04-16T17:17:31.751 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "uint", 2024-04-16T17:17:31.751 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:31.751 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:31.751 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "25", 2024-04-16T17:17:31.751 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "1", 2024-04-16T17:17:31.752 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "250", 2024-04-16T17:17:31.752 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:31.752 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-16T17:17:31.752 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:31.752 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:31.752 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:31.752 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:31.752 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_compat_metrics": { 2024-04-16T17:17:31.752 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "crush_compat_metrics", 2024-04-16T17:17:31.752 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:31.753 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:31.753 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:31.753 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "pgs,objects,bytes", 2024-04-16T17:17:31.753 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:31.753 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:31.753 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:31.753 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-16T17:17:31.753 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-16T17:17:31.753 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:31.754 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:31.754 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:31.754 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_compat_step": { 2024-04-16T17:17:31.754 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "crush_compat_step", 2024-04-16T17:17:31.754 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "float", 2024-04-16T17:17:31.754 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:31.754 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:31.754 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0.5", 2024-04-16T17:17:31.754 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "0.001", 2024-04-16T17:17:31.755 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "0.999", 2024-04-16T17:17:31.755 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:31.755 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "aggressiveness of optimization", 2024-04-16T17:17:31.755 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-16T17:17:31.755 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:31.755 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:31.755 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:31.755 INFO:teuthology.orchestra.run.smithi045.stdout: "end_time": { 2024-04-16T17:17:31.755 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "end_time", 2024-04-16T17:17:31.756 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:31.756 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:31.756 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:31.756 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "2359", 2024-04-16T17:17:31.756 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:31.756 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:31.756 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:31.756 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "ending time of day to automatically balance", 2024-04-16T17:17:31.756 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-16T17:17:31.756 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:31.757 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:31.757 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:31.757 INFO:teuthology.orchestra.run.smithi045.stdout: "end_weekday": { 2024-04-16T17:17:31.757 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "end_weekday", 2024-04-16T17:17:31.757 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "uint", 2024-04-16T17:17:31.757 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:31.757 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:31.757 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0", 2024-04-16T17:17:31.757 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "0", 2024-04-16T17:17:31.757 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "6", 2024-04-16T17:17:31.758 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:31.758 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-16T17:17:31.758 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-16T17:17:31.758 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:31.758 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:31.758 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:31.758 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-16T17:17:31.758 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-16T17:17:31.758 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:31.759 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:31.759 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:31.759 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:31.759 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:31.759 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:31.759 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-16T17:17:31.759 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-16T17:17:31.759 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-16T17:17:31.759 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-16T17:17:31.759 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-16T17:17:31.760 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-16T17:17:31.760 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-16T17:17:31.760 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-16T17:17:31.760 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:31.760 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:31.760 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:31.760 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:31.760 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:31.760 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-16T17:17:31.760 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-16T17:17:31.761 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:31.761 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:31.761 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:31.761 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:31.761 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:31.761 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:31.761 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:31.761 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:31.761 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:31.762 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:31.762 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:31.762 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:31.762 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-16T17:17:31.762 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-16T17:17:31.762 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:31.762 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:31.762 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:31.763 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-16T17:17:31.763 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:31.763 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:31.763 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-16T17:17:31.763 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-16T17:17:31.763 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-16T17:17:31.763 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-16T17:17:31.763 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-16T17:17:31.763 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-16T17:17:31.764 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-16T17:17:31.764 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-16T17:17:31.764 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:31.764 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:31.764 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:31.764 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:31.764 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:31.764 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-16T17:17:31.764 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-16T17:17:31.765 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:31.765 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:31.765 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:31.765 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:31.765 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:31.765 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:31.765 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:31.765 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:31.765 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:31.766 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:31.766 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:31.766 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:31.766 INFO:teuthology.orchestra.run.smithi045.stdout: "min_score": { 2024-04-16T17:17:31.766 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "min_score", 2024-04-16T17:17:31.766 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "float", 2024-04-16T17:17:31.766 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:31.766 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:31.767 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0", 2024-04-16T17:17:31.767 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:31.767 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:31.767 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:31.767 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-16T17:17:31.767 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:31.767 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:31.767 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:31.767 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:31.767 INFO:teuthology.orchestra.run.smithi045.stdout: "mode": { 2024-04-16T17:17:31.768 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "mode", 2024-04-16T17:17:31.768 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:31.768 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:31.768 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:31.768 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "upmap", 2024-04-16T17:17:31.768 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:31.768 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:31.768 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-16T17:17:31.768 INFO:teuthology.orchestra.run.smithi045.stdout: "crush-compat", 2024-04-16T17:17:31.768 INFO:teuthology.orchestra.run.smithi045.stdout: "none", 2024-04-16T17:17:31.769 INFO:teuthology.orchestra.run.smithi045.stdout: "read", 2024-04-16T17:17:31.769 INFO:teuthology.orchestra.run.smithi045.stdout: "upmap", 2024-04-16T17:17:31.769 INFO:teuthology.orchestra.run.smithi045.stdout: "upmap-read" 2024-04-16T17:17:31.769 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-16T17:17:31.769 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Balancer mode", 2024-04-16T17:17:31.769 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:31.769 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:31.769 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:31.769 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:31.770 INFO:teuthology.orchestra.run.smithi045.stdout: "pool_ids": { 2024-04-16T17:17:31.770 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "pool_ids", 2024-04-16T17:17:31.770 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:31.770 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:31.770 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:31.770 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:31.770 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:31.770 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:31.770 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:31.770 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-16T17:17:31.771 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:31.771 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:31.771 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:31.771 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:31.771 INFO:teuthology.orchestra.run.smithi045.stdout: "sleep_interval": { 2024-04-16T17:17:31.771 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "sleep_interval", 2024-04-16T17:17:31.771 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-04-16T17:17:31.771 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:31.771 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:31.772 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "60", 2024-04-16T17:17:31.772 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:31.772 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:31.772 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:31.772 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-16T17:17:31.772 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:31.772 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:31.772 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:31.772 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:31.772 INFO:teuthology.orchestra.run.smithi045.stdout: "upmap_max_deviation": { 2024-04-16T17:17:31.773 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "upmap_max_deviation", 2024-04-16T17:17:31.773 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-16T17:17:31.773 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:31.773 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:31.773 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "5", 2024-04-16T17:17:31.773 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "1", 2024-04-16T17:17:31.773 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:31.773 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:31.773 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-16T17:17:31.774 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-16T17:17:31.774 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:31.774 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:31.774 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:31.774 INFO:teuthology.orchestra.run.smithi045.stdout: "upmap_max_optimizations": { 2024-04-16T17:17:31.774 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "upmap_max_optimizations", 2024-04-16T17:17:31.774 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "uint", 2024-04-16T17:17:31.774 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:31.774 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:31.775 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "10", 2024-04-16T17:17:31.775 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:31.775 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:31.775 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:31.775 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-16T17:17:31.775 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:31.775 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:31.775 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:31.775 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:17:31.776 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:17:31.776 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:31.776 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-16T17:17:31.776 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "cephadm", 2024-04-16T17:17:31.776 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-16T17:17:31.776 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-16T17:17:31.776 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-16T17:17:31.776 INFO:teuthology.orchestra.run.smithi045.stdout: "agent_down_multiplier": { 2024-04-16T17:17:31.776 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "agent_down_multiplier", 2024-04-16T17:17:31.776 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "float", 2024-04-16T17:17:31.777 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:31.777 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:31.777 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "3.0", 2024-04-16T17:17:31.777 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:31.777 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:31.777 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:31.777 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-16T17:17:31.777 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:31.777 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:31.777 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:31.778 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:31.778 INFO:teuthology.orchestra.run.smithi045.stdout: "agent_refresh_rate": { 2024-04-16T17:17:31.778 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "agent_refresh_rate", 2024-04-16T17:17:31.778 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-04-16T17:17:31.778 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:31.778 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:31.778 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "20", 2024-04-16T17:17:31.778 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:31.779 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:31.779 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:31.779 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-16T17:17:31.779 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:31.779 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:31.779 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:31.779 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:31.779 INFO:teuthology.orchestra.run.smithi045.stdout: "agent_starting_port": { 2024-04-16T17:17:31.779 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "agent_starting_port", 2024-04-16T17:17:31.779 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-16T17:17:31.780 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:31.780 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:31.780 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "4721", 2024-04-16T17:17:31.780 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:31.780 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:31.780 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:31.780 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-16T17:17:31.780 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:31.780 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:31.780 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:31.781 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:31.781 INFO:teuthology.orchestra.run.smithi045.stdout: "allow_ptrace": { 2024-04-16T17:17:31.781 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "allow_ptrace", 2024-04-16T17:17:31.781 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:31.781 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:31.781 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:31.781 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:31.781 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:31.781 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:31.782 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:31.782 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-16T17:17:31.782 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-16T17:17:31.782 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:31.782 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:31.782 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:31.782 INFO:teuthology.orchestra.run.smithi045.stdout: "autotune_interval": { 2024-04-16T17:17:31.782 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "autotune_interval", 2024-04-16T17:17:31.782 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-04-16T17:17:31.782 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:31.783 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:31.783 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "600", 2024-04-16T17:17:31.783 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:31.783 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:31.783 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:31.783 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-16T17:17:31.783 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:31.783 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:31.783 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:31.784 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:31.784 INFO:teuthology.orchestra.run.smithi045.stdout: "autotune_memory_target_ratio": { 2024-04-16T17:17:31.784 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "autotune_memory_target_ratio", 2024-04-16T17:17:31.784 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "float", 2024-04-16T17:17:31.784 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:31.784 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:31.784 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0.7", 2024-04-16T17:17:31.784 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:31.785 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:31.785 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:31.785 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-16T17:17:31.785 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:31.785 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:31.785 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:31.785 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:31.785 INFO:teuthology.orchestra.run.smithi045.stdout: "cephadm_log_destination": { 2024-04-16T17:17:31.785 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "cephadm_log_destination", 2024-04-16T17:17:31.785 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:31.786 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:31.786 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:31.786 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:31.786 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:31.786 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:31.786 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-16T17:17:31.786 INFO:teuthology.orchestra.run.smithi045.stdout: "file", 2024-04-16T17:17:31.786 INFO:teuthology.orchestra.run.smithi045.stdout: "file,syslog", 2024-04-16T17:17:31.786 INFO:teuthology.orchestra.run.smithi045.stdout: "syslog" 2024-04-16T17:17:31.787 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-16T17:17:31.787 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-16T17:17:31.787 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:31.787 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:31.787 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:31.787 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:31.787 INFO:teuthology.orchestra.run.smithi045.stdout: "cgroups_split": { 2024-04-16T17:17:31.787 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "cgroups_split", 2024-04-16T17:17:31.787 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:31.787 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:31.788 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:31.788 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-16T17:17:31.788 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:31.788 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:31.788 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:31.788 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-16T17:17:31.788 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:31.788 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:31.788 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:31.788 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:31.789 INFO:teuthology.orchestra.run.smithi045.stdout: "config_checks_enabled": { 2024-04-16T17:17:31.789 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "config_checks_enabled", 2024-04-16T17:17:31.789 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:31.789 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:31.789 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:31.789 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:31.789 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:31.789 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:31.789 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:31.790 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-16T17:17:31.790 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:31.790 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:31.790 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:31.790 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:31.790 INFO:teuthology.orchestra.run.smithi045.stdout: "config_dashboard": { 2024-04-16T17:17:31.790 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "config_dashboard", 2024-04-16T17:17:31.790 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:31.790 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:31.791 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:31.791 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-16T17:17:31.791 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:31.791 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:31.791 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:31.791 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-16T17:17:31.791 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:31.791 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:31.791 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:31.791 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:31.792 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_alertmanager": { 2024-04-16T17:17:31.792 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_alertmanager", 2024-04-16T17:17:31.792 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:31.792 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:31.792 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:31.792 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-16T17:17:31.792 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:31.792 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:31.792 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:31.793 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Prometheus container image", 2024-04-16T17:17:31.793 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:31.793 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:31.793 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:31.793 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:31.793 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_base": { 2024-04-16T17:17:31.793 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_base", 2024-04-16T17:17:31.793 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:31.793 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:31.794 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:31.794 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-16T17:17:31.794 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:31.794 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:31.794 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:31.794 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Container image name, without the tag", 2024-04-16T17:17:31.794 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:31.794 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:31.794 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:31.794 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:31.795 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_elasticsearch": { 2024-04-16T17:17:31.795 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_elasticsearch", 2024-04-16T17:17:31.795 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:31.795 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:31.795 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:31.795 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-16T17:17:31.795 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:31.795 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:31.795 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:31.795 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "elasticsearch container image", 2024-04-16T17:17:31.796 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:31.796 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:31.796 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:31.796 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:31.796 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_grafana": { 2024-04-16T17:17:31.796 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_grafana", 2024-04-16T17:17:31.796 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:31.796 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:31.796 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:31.797 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-16T17:17:31.797 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:31.797 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:31.797 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:31.797 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Prometheus container image", 2024-04-16T17:17:31.797 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:31.797 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:31.797 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:31.797 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:31.798 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_haproxy": { 2024-04-16T17:17:31.798 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_haproxy", 2024-04-16T17:17:31.798 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:31.798 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:31.798 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:31.798 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-16T17:17:31.798 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:31.798 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:31.798 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:31.798 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "HAproxy container image", 2024-04-16T17:17:31.799 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:31.799 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:31.799 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:31.799 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:31.799 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_jaeger_agent": { 2024-04-16T17:17:31.799 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_jaeger_agent", 2024-04-16T17:17:31.799 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:31.799 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:31.799 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:31.800 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-16T17:17:31.800 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:31.800 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:31.800 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:31.800 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Jaeger agent container image", 2024-04-16T17:17:31.800 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:31.800 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:31.800 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:31.800 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:31.800 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_jaeger_collector": { 2024-04-16T17:17:31.801 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_jaeger_collector", 2024-04-16T17:17:31.801 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:31.801 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:31.801 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:31.801 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-16T17:17:31.801 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:31.801 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:31.801 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:31.801 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Jaeger collector container image", 2024-04-16T17:17:31.802 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:31.802 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:31.802 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:31.802 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:31.802 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_jaeger_query": { 2024-04-16T17:17:31.802 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_jaeger_query", 2024-04-16T17:17:31.802 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:31.802 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:31.802 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:31.803 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-16T17:17:31.803 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:31.803 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:31.803 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:31.803 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Jaeger query container image", 2024-04-16T17:17:31.803 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:31.803 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:31.803 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:31.803 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:31.803 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_keepalived": { 2024-04-16T17:17:31.804 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_keepalived", 2024-04-16T17:17:31.804 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:31.804 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:31.804 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:31.804 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-16T17:17:31.804 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:31.804 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:31.804 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:31.804 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Keepalived container image", 2024-04-16T17:17:31.805 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:31.805 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:31.805 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:31.805 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:31.805 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_loki": { 2024-04-16T17:17:31.805 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_loki", 2024-04-16T17:17:31.805 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:31.805 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:31.805 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:31.806 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-16T17:17:31.806 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:31.806 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:31.806 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:31.806 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Loki container image", 2024-04-16T17:17:31.806 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:31.806 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:31.806 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:31.806 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:31.807 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_node_exporter": { 2024-04-16T17:17:31.807 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_node_exporter", 2024-04-16T17:17:31.807 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:31.807 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:31.807 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:31.807 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-16T17:17:31.807 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:31.807 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:31.807 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:31.808 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Prometheus container image", 2024-04-16T17:17:31.808 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:31.808 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:31.808 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:31.808 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:31.808 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_nvmeof": { 2024-04-16T17:17:31.808 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_nvmeof", 2024-04-16T17:17:31.808 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:31.808 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:31.808 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:31.809 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-04-16T17:17:31.809 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:31.809 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:31.809 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:31.809 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Nvme-of container image", 2024-04-16T17:17:31.809 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:31.809 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:31.809 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:31.809 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:31.810 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_prometheus": { 2024-04-16T17:17:31.810 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_prometheus", 2024-04-16T17:17:31.810 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:31.810 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:31.810 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:31.810 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-16T17:17:31.810 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:31.810 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:31.810 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:31.811 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Prometheus container image", 2024-04-16T17:17:31.811 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:31.811 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:31.811 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:31.811 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:31.811 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_promtail": { 2024-04-16T17:17:31.811 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_promtail", 2024-04-16T17:17:31.811 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:31.811 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:31.812 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:31.812 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-16T17:17:31.812 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:31.812 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:31.812 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:31.812 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Promtail container image", 2024-04-16T17:17:31.812 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:31.812 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:31.812 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:31.813 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:31.813 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_samba": { 2024-04-16T17:17:31.813 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_samba", 2024-04-16T17:17:31.813 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:31.813 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:31.813 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:31.813 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-16T17:17:31.813 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:31.813 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:31.814 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:31.814 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Samba/SMB container image", 2024-04-16T17:17:31.814 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:31.814 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:31.814 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:31.814 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:31.814 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_snmp_gateway": { 2024-04-16T17:17:31.814 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_snmp_gateway", 2024-04-16T17:17:31.814 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:31.815 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:31.815 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:31.815 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-16T17:17:31.815 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:31.815 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:31.815 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:31.815 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "SNMP Gateway container image", 2024-04-16T17:17:31.815 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:31.815 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:31.816 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:31.816 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:31.816 INFO:teuthology.orchestra.run.smithi045.stdout: "container_init": { 2024-04-16T17:17:31.816 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_init", 2024-04-16T17:17:31.816 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:31.816 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:31.816 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:31.816 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-16T17:17:31.816 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:31.816 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:31.817 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:31.817 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Run podman/docker with `--init`", 2024-04-16T17:17:31.817 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:31.817 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:31.817 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:31.817 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:31.817 INFO:teuthology.orchestra.run.smithi045.stdout: "daemon_cache_timeout": { 2024-04-16T17:17:31.817 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "daemon_cache_timeout", 2024-04-16T17:17:31.817 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-04-16T17:17:31.818 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:31.818 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:31.818 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "600", 2024-04-16T17:17:31.818 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:31.818 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:31.818 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:31.818 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-16T17:17:31.818 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:31.818 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:31.818 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:31.819 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:31.819 INFO:teuthology.orchestra.run.smithi045.stdout: "default_cephadm_command_timeout": { 2024-04-16T17:17:31.819 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "default_cephadm_command_timeout", 2024-04-16T17:17:31.819 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-16T17:17:31.819 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:31.819 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:31.819 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "900", 2024-04-16T17:17:31.819 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:31.819 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:31.819 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:31.820 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-16T17:17:31.820 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:31.820 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:31.820 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:31.820 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:31.820 INFO:teuthology.orchestra.run.smithi045.stdout: "default_registry": { 2024-04-16T17:17:31.820 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "default_registry", 2024-04-16T17:17:31.820 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:31.820 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:31.821 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:31.821 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "docker.io", 2024-04-16T17:17:31.821 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:31.821 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:31.821 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:31.821 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-16T17:17:31.821 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:31.821 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:31.821 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:31.822 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:31.822 INFO:teuthology.orchestra.run.smithi045.stdout: "device_cache_timeout": { 2024-04-16T17:17:31.822 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "device_cache_timeout", 2024-04-16T17:17:31.822 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-04-16T17:17:31.822 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:31.822 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:31.822 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "1800", 2024-04-16T17:17:31.822 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:31.823 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:31.823 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:31.823 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "seconds to cache device inventory", 2024-04-16T17:17:31.823 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:31.823 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:31.823 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:31.823 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:31.823 INFO:teuthology.orchestra.run.smithi045.stdout: "device_enhanced_scan": { 2024-04-16T17:17:31.823 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "device_enhanced_scan", 2024-04-16T17:17:31.824 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:31.824 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:31.824 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:31.824 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:31.824 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:31.824 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:31.824 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:31.824 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-16T17:17:31.824 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:31.825 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:31.825 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:31.825 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:31.825 INFO:teuthology.orchestra.run.smithi045.stdout: "facts_cache_timeout": { 2024-04-16T17:17:31.825 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "facts_cache_timeout", 2024-04-16T17:17:31.825 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-04-16T17:17:31.825 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:31.825 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:31.825 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "60", 2024-04-16T17:17:31.826 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:31.826 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:31.826 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:31.826 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "seconds to cache host facts data", 2024-04-16T17:17:31.826 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:31.826 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:31.826 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:31.826 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:31.826 INFO:teuthology.orchestra.run.smithi045.stdout: "grafana_dashboards_path": { 2024-04-16T17:17:31.827 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "grafana_dashboards_path", 2024-04-16T17:17:31.827 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:31.827 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:31.827 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:31.827 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-16T17:17:31.827 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:31.827 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:31.827 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:31.827 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-16T17:17:31.828 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:31.828 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:31.828 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:31.828 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:31.828 INFO:teuthology.orchestra.run.smithi045.stdout: "host_check_interval": { 2024-04-16T17:17:31.828 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "host_check_interval", 2024-04-16T17:17:31.828 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-04-16T17:17:31.828 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:31.828 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:31.828 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "600", 2024-04-16T17:17:31.829 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:31.829 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:31.829 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:31.829 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "how frequently to perform a host check", 2024-04-16T17:17:31.829 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:31.829 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:31.829 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:31.829 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:31.829 INFO:teuthology.orchestra.run.smithi045.stdout: "hw_monitoring": { 2024-04-16T17:17:31.830 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "hw_monitoring", 2024-04-16T17:17:31.830 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:31.830 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:31.830 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:31.830 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:31.830 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:31.830 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:31.830 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:31.830 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-16T17:17:31.830 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:31.831 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:31.831 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:31.831 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:31.831 INFO:teuthology.orchestra.run.smithi045.stdout: "inventory_list_all": { 2024-04-16T17:17:31.831 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "inventory_list_all", 2024-04-16T17:17:31.831 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:31.831 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:31.831 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:31.831 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:31.832 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:31.832 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:31.832 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:31.832 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-16T17:17:31.832 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:31.832 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:31.832 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:31.832 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:31.832 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-16T17:17:31.832 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-16T17:17:31.833 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:31.833 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:31.833 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:31.833 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:31.833 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:31.833 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:31.833 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-16T17:17:31.833 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-16T17:17:31.833 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-16T17:17:31.833 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-16T17:17:31.834 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-16T17:17:31.834 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-16T17:17:31.834 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-16T17:17:31.834 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-16T17:17:31.834 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:31.834 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:31.834 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:31.834 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:31.834 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:31.834 INFO:teuthology.orchestra.run.smithi045.stdout: "log_refresh_metadata": { 2024-04-16T17:17:31.835 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_refresh_metadata", 2024-04-16T17:17:31.835 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:31.835 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:31.835 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:31.835 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:31.835 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:31.835 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:31.835 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:31.836 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-16T17:17:31.836 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:31.836 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:31.836 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:31.836 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:31.836 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-16T17:17:31.836 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-16T17:17:31.836 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:31.836 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:31.837 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:31.837 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-16T17:17:31.837 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:31.837 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:31.837 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:31.837 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-16T17:17:31.837 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:31.837 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:31.837 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:31.837 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:31.838 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-16T17:17:31.838 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-16T17:17:31.838 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:31.838 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:31.838 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:31.838 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-16T17:17:31.838 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:31.838 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:31.839 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-16T17:17:31.839 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-16T17:17:31.839 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-16T17:17:31.839 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-16T17:17:31.839 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-16T17:17:31.839 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-16T17:17:31.839 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-16T17:17:31.839 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-16T17:17:31.839 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:31.839 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:31.840 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:31.840 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:31.840 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:31.840 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-16T17:17:31.840 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-16T17:17:31.840 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:31.840 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:31.840 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:31.840 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:31.841 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:31.841 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:31.841 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:31.841 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:31.841 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:31.841 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:31.841 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:31.841 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:31.841 INFO:teuthology.orchestra.run.smithi045.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-16T17:17:31.841 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-16T17:17:31.842 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:31.842 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:31.842 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:31.842 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:31.842 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:31.842 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:31.842 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:31.842 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-16T17:17:31.842 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:31.843 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:31.843 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:31.843 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:31.843 INFO:teuthology.orchestra.run.smithi045.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-16T17:17:31.843 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-16T17:17:31.843 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:31.843 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:31.843 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:31.843 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "*", 2024-04-16T17:17:31.844 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:31.844 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:31.844 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:31.844 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-16T17:17:31.844 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:31.844 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:31.844 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:31.844 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:31.844 INFO:teuthology.orchestra.run.smithi045.stdout: "max_count_per_host": { 2024-04-16T17:17:31.844 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "max_count_per_host", 2024-04-16T17:17:31.845 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-16T17:17:31.845 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:31.845 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:31.845 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "10", 2024-04-16T17:17:31.845 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:31.845 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:31.845 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:31.845 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "max number of daemons per service per host", 2024-04-16T17:17:31.845 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:31.845 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:31.846 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:31.846 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:31.846 INFO:teuthology.orchestra.run.smithi045.stdout: "max_osd_draining_count": { 2024-04-16T17:17:31.846 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "max_osd_draining_count", 2024-04-16T17:17:31.846 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-16T17:17:31.846 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:31.846 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:31.846 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "10", 2024-04-16T17:17:31.846 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:31.847 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:31.847 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:31.847 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-16T17:17:31.847 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:31.847 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:31.847 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:31.847 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:31.847 INFO:teuthology.orchestra.run.smithi045.stdout: "migration_current": { 2024-04-16T17:17:31.847 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "migration_current", 2024-04-16T17:17:31.847 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-16T17:17:31.848 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:31.848 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:31.848 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:31.848 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:31.848 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:31.848 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:31.848 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "internal - do not modify", 2024-04-16T17:17:31.848 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:31.848 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:31.848 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:31.849 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:31.849 INFO:teuthology.orchestra.run.smithi045.stdout: "mode": { 2024-04-16T17:17:31.849 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "mode", 2024-04-16T17:17:31.849 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:31.849 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:31.849 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:31.849 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "root", 2024-04-16T17:17:31.849 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:31.850 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:31.850 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-16T17:17:31.850 INFO:teuthology.orchestra.run.smithi045.stdout: "cephadm-package", 2024-04-16T17:17:31.850 INFO:teuthology.orchestra.run.smithi045.stdout: "root" 2024-04-16T17:17:31.850 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-16T17:17:31.850 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "mode for remote execution of cephadm", 2024-04-16T17:17:31.850 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:31.850 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:31.850 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:31.851 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:31.851 INFO:teuthology.orchestra.run.smithi045.stdout: "oob_default_addr": { 2024-04-16T17:17:31.851 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "oob_default_addr", 2024-04-16T17:17:31.851 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:31.851 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:31.851 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:31.851 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "169.254.1.1", 2024-04-16T17:17:31.851 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:31.851 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:31.851 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:31.852 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-16T17:17:31.852 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:31.852 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:31.852 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:31.852 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:31.852 INFO:teuthology.orchestra.run.smithi045.stdout: "prometheus_alerts_path": { 2024-04-16T17:17:31.852 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "prometheus_alerts_path", 2024-04-16T17:17:31.852 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:31.852 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:31.852 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:31.853 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-16T17:17:31.853 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:31.853 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:31.853 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:31.853 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-16T17:17:31.853 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:31.853 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:31.853 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:31.853 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:31.854 INFO:teuthology.orchestra.run.smithi045.stdout: "registry_insecure": { 2024-04-16T17:17:31.854 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "registry_insecure", 2024-04-16T17:17:31.854 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:31.854 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:31.854 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:31.854 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:31.854 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:31.854 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:31.854 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:31.855 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-16T17:17:31.855 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:31.855 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:31.855 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:31.855 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:31.855 INFO:teuthology.orchestra.run.smithi045.stdout: "registry_password": { 2024-04-16T17:17:31.855 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "registry_password", 2024-04-16T17:17:31.855 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:31.855 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:31.856 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:31.856 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:31.856 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:31.856 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:31.856 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:31.856 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-16T17:17:31.856 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:31.856 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:31.856 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:31.857 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:31.857 INFO:teuthology.orchestra.run.smithi045.stdout: "registry_url": { 2024-04-16T17:17:31.857 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "registry_url", 2024-04-16T17:17:31.857 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:31.857 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:31.857 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:31.857 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:31.857 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:31.857 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:31.857 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:31.858 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-16T17:17:31.858 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:31.858 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:31.858 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:31.858 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:31.858 INFO:teuthology.orchestra.run.smithi045.stdout: "registry_username": { 2024-04-16T17:17:31.858 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "registry_username", 2024-04-16T17:17:31.858 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:31.858 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:31.858 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:31.859 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:31.859 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:31.859 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:31.859 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:31.859 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-16T17:17:31.859 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:31.859 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:31.859 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:31.859 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:31.860 INFO:teuthology.orchestra.run.smithi045.stdout: "secure_monitoring_stack": { 2024-04-16T17:17:31.860 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "secure_monitoring_stack", 2024-04-16T17:17:31.860 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:31.860 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:31.860 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:31.860 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:31.860 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:31.860 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:31.860 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:31.860 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-16T17:17:31.861 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:31.861 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:31.861 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:31.861 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:31.861 INFO:teuthology.orchestra.run.smithi045.stdout: "service_discovery_port": { 2024-04-16T17:17:31.861 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "service_discovery_port", 2024-04-16T17:17:31.861 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-16T17:17:31.861 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:31.862 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:31.862 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "8765", 2024-04-16T17:17:31.862 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:31.862 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:31.862 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:31.862 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "cephadm service discovery port", 2024-04-16T17:17:31.862 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:31.862 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:31.862 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:31.862 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:31.863 INFO:teuthology.orchestra.run.smithi045.stdout: "ssh_config_file": { 2024-04-16T17:17:31.863 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ssh_config_file", 2024-04-16T17:17:31.863 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:31.863 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:31.863 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:31.863 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:31.863 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:31.863 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:31.863 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:31.864 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-16T17:17:31.864 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:31.864 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:31.864 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:31.864 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:31.864 INFO:teuthology.orchestra.run.smithi045.stdout: "use_agent": { 2024-04-16T17:17:31.864 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "use_agent", 2024-04-16T17:17:31.864 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:31.864 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:31.865 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:31.865 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:31.865 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:31.865 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:31.865 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:31.865 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-16T17:17:31.865 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:31.865 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:31.865 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:31.866 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:31.866 INFO:teuthology.orchestra.run.smithi045.stdout: "use_repo_digest": { 2024-04-16T17:17:31.866 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "use_repo_digest", 2024-04-16T17:17:31.866 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:31.866 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:31.866 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:31.866 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-16T17:17:31.866 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:31.867 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:31.867 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:31.867 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-16T17:17:31.867 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:31.867 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:31.867 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:31.867 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:31.867 INFO:teuthology.orchestra.run.smithi045.stdout: "warn_on_failed_host_check": { 2024-04-16T17:17:31.867 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "warn_on_failed_host_check", 2024-04-16T17:17:31.868 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:31.868 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:31.868 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:31.868 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-16T17:17:31.868 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:31.868 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:31.868 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:31.868 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "raise a health warning if the host check fails", 2024-04-16T17:17:31.868 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:31.868 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:31.869 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:31.869 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:31.869 INFO:teuthology.orchestra.run.smithi045.stdout: "warn_on_stray_daemons": { 2024-04-16T17:17:31.869 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "warn_on_stray_daemons", 2024-04-16T17:17:31.869 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:31.869 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:31.869 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:31.869 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-16T17:17:31.870 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:31.870 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:31.870 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:31.870 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-16T17:17:31.870 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:31.870 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:31.870 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:31.870 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:31.870 INFO:teuthology.orchestra.run.smithi045.stdout: "warn_on_stray_hosts": { 2024-04-16T17:17:31.871 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "warn_on_stray_hosts", 2024-04-16T17:17:31.871 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:31.871 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:31.871 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:31.871 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-16T17:17:31.871 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:31.871 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:31.871 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:31.871 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-16T17:17:31.871 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:31.872 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:31.872 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:31.872 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:17:31.872 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:17:31.872 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:31.872 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-16T17:17:31.872 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "crash", 2024-04-16T17:17:31.872 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-16T17:17:31.872 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-16T17:17:31.873 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-16T17:17:31.873 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-16T17:17:31.873 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-16T17:17:31.873 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:31.873 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:31.873 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:31.873 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:31.873 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:31.873 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:31.873 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-16T17:17:31.874 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-16T17:17:31.874 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-16T17:17:31.874 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-16T17:17:31.874 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-16T17:17:31.874 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-16T17:17:31.874 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-16T17:17:31.874 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-16T17:17:31.874 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:31.874 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:31.875 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:31.875 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:31.875 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:31.875 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-16T17:17:31.875 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-16T17:17:31.875 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:31.875 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:31.875 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:31.875 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:31.876 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:31.876 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:31.876 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:31.876 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:31.876 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:31.876 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:31.876 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:31.876 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:31.876 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-16T17:17:31.877 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-16T17:17:31.877 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:31.877 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:31.877 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:31.877 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-16T17:17:31.877 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:31.877 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:31.877 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-16T17:17:31.877 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-16T17:17:31.877 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-16T17:17:31.878 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-16T17:17:31.878 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-16T17:17:31.878 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-16T17:17:31.878 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-16T17:17:31.878 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-16T17:17:31.878 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:31.878 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:31.878 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:31.878 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:31.879 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:31.879 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-16T17:17:31.879 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-16T17:17:31.879 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:31.879 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:31.879 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:31.879 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:31.879 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:31.879 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:31.880 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:31.880 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:31.880 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:31.880 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:31.880 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:31.880 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:31.880 INFO:teuthology.orchestra.run.smithi045.stdout: "retain_interval": { 2024-04-16T17:17:31.880 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "retain_interval", 2024-04-16T17:17:31.880 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-04-16T17:17:31.880 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:31.880 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:31.881 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "31536000", 2024-04-16T17:17:31.881 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:31.881 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:31.881 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:31.881 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-16T17:17:31.881 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:31.881 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:31.881 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:31.881 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:31.882 INFO:teuthology.orchestra.run.smithi045.stdout: "warn_recent_interval": { 2024-04-16T17:17:31.882 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "warn_recent_interval", 2024-04-16T17:17:31.882 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-04-16T17:17:31.882 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:31.882 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:31.882 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "1209600", 2024-04-16T17:17:31.882 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:31.882 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:31.882 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:31.882 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-16T17:17:31.883 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:31.883 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:31.883 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:31.883 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:17:31.883 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:17:31.883 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:31.883 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-16T17:17:31.883 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "dashboard", 2024-04-16T17:17:31.883 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-16T17:17:31.884 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-16T17:17:31.884 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-16T17:17:31.884 INFO:teuthology.orchestra.run.smithi045.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-16T17:17:31.884 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-16T17:17:31.884 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-16T17:17:31.884 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:31.884 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:31.884 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "10", 2024-04-16T17:17:31.884 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:31.885 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:31.885 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:31.885 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:31.885 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:31.885 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:31.885 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:31.885 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:31.885 INFO:teuthology.orchestra.run.smithi045.stdout: "ALERTMANAGER_API_HOST": { 2024-04-16T17:17:31.885 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-16T17:17:31.885 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:31.886 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:31.886 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:31.886 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:31.886 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:31.886 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:31.886 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:31.886 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:31.886 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:31.886 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:31.887 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:31.887 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:31.887 INFO:teuthology.orchestra.run.smithi045.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-16T17:17:31.887 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-16T17:17:31.887 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:31.887 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:31.887 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:31.887 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-16T17:17:31.887 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:31.888 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:31.888 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:31.888 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:31.888 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:31.888 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:31.888 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:31.888 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:31.888 INFO:teuthology.orchestra.run.smithi045.stdout: "AUDIT_API_ENABLED": { 2024-04-16T17:17:31.888 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "AUDIT_API_ENABLED", 2024-04-16T17:17:31.888 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:31.889 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:31.889 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:31.889 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:31.889 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:31.889 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:31.889 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:31.889 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:31.889 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:31.889 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:31.890 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:31.890 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:31.890 INFO:teuthology.orchestra.run.smithi045.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-16T17:17:31.890 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-16T17:17:31.890 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:31.890 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:31.890 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:31.890 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-16T17:17:31.890 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:31.891 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:31.891 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:31.891 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:31.891 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:31.891 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:31.891 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:31.891 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:31.891 INFO:teuthology.orchestra.run.smithi045.stdout: "ENABLE_BROWSABLE_API": { 2024-04-16T17:17:31.891 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-16T17:17:31.892 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:31.892 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:31.892 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:31.892 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-16T17:17:31.892 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:31.892 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:31.892 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:31.892 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:31.892 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:31.893 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:31.893 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:31.893 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:31.893 INFO:teuthology.orchestra.run.smithi045.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-16T17:17:31.893 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-16T17:17:31.893 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:31.893 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:31.893 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:31.893 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-16T17:17:31.894 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:31.894 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:31.894 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:31.894 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:31.894 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:31.894 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:31.894 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:31.894 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:31.894 INFO:teuthology.orchestra.run.smithi045.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-16T17:17:31.894 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-16T17:17:31.895 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:31.895 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:31.895 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:31.895 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-16T17:17:31.895 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:31.895 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:31.895 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:31.895 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:31.895 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:31.895 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:31.896 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:31.896 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:31.896 INFO:teuthology.orchestra.run.smithi045.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-16T17:17:31.896 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-16T17:17:31.896 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:31.896 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:31.896 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:31.896 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-16T17:17:31.896 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:31.896 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:31.897 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:31.897 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:31.897 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:31.897 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:31.897 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:31.897 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:31.897 INFO:teuthology.orchestra.run.smithi045.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-16T17:17:31.897 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-16T17:17:31.897 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:31.897 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:31.898 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:31.898 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-16T17:17:31.898 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:31.898 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:31.898 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:31.898 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:31.898 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:31.898 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:31.898 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:31.899 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:31.899 INFO:teuthology.orchestra.run.smithi045.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-16T17:17:31.899 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-16T17:17:31.899 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:31.899 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:31.899 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:31.899 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-16T17:17:31.899 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:31.899 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:31.899 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:31.900 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:31.900 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:31.900 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:31.900 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:31.900 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:31.900 INFO:teuthology.orchestra.run.smithi045.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-16T17:17:31.900 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-16T17:17:31.900 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:31.900 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:31.901 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:31.901 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-16T17:17:31.901 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:31.901 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:31.901 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:31.901 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:31.901 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:31.901 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:31.901 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:31.901 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:31.902 INFO:teuthology.orchestra.run.smithi045.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-16T17:17:31.902 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-16T17:17:31.902 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:31.902 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:31.902 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:31.902 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-16T17:17:31.902 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:31.902 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:31.902 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:31.903 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:31.903 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:31.903 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:31.903 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:31.903 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:31.903 INFO:teuthology.orchestra.run.smithi045.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-16T17:17:31.903 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-16T17:17:31.903 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:31.903 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:31.904 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:31.904 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:31.904 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:31.904 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:31.904 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:31.904 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:31.904 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:31.904 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:31.904 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:31.904 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:31.905 INFO:teuthology.orchestra.run.smithi045.stdout: "GRAFANA_API_PASSWORD": { 2024-04-16T17:17:31.905 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-16T17:17:31.905 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:31.905 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:31.905 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:31.905 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "admin", 2024-04-16T17:17:31.905 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:31.905 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:31.905 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:31.905 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:31.906 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:31.906 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:31.906 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:31.906 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:31.906 INFO:teuthology.orchestra.run.smithi045.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-16T17:17:31.906 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-16T17:17:31.906 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:31.906 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:31.907 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:31.907 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-16T17:17:31.907 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:31.907 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:31.907 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:31.907 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:31.907 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:31.907 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:31.907 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:31.907 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:31.908 INFO:teuthology.orchestra.run.smithi045.stdout: "GRAFANA_API_URL": { 2024-04-16T17:17:31.908 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "GRAFANA_API_URL", 2024-04-16T17:17:31.908 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:31.908 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:31.908 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:31.908 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:31.908 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:31.908 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:31.908 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:31.908 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:31.909 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:31.909 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:31.909 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:31.909 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:31.909 INFO:teuthology.orchestra.run.smithi045.stdout: "GRAFANA_API_USERNAME": { 2024-04-16T17:17:31.909 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-16T17:17:31.909 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:31.909 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:31.909 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:31.909 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "admin", 2024-04-16T17:17:31.910 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:31.910 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:31.910 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:31.910 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:31.910 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:31.910 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:31.910 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:31.910 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:31.910 INFO:teuthology.orchestra.run.smithi045.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-16T17:17:31.911 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-16T17:17:31.911 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:31.911 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:31.911 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:31.911 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:31.911 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:31.911 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:31.911 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:31.911 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:31.912 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:31.912 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:31.912 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:31.912 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:31.912 INFO:teuthology.orchestra.run.smithi045.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-16T17:17:31.912 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-16T17:17:31.912 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:31.912 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:31.913 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:31.913 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:31.913 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:31.913 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:31.913 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:31.913 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:31.913 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:31.913 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:31.913 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:31.914 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:31.914 INFO:teuthology.orchestra.run.smithi045.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-16T17:17:31.914 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-16T17:17:31.914 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:31.914 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:31.914 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:31.914 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-16T17:17:31.914 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:31.914 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:31.914 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:31.915 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:31.915 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:31.915 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:31.915 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:31.915 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:31.915 INFO:teuthology.orchestra.run.smithi045.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-16T17:17:31.915 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-16T17:17:31.915 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:31.916 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:31.916 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:31.916 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:31.916 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:31.916 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:31.916 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:31.916 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:31.916 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:31.916 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:31.917 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:31.917 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:31.917 INFO:teuthology.orchestra.run.smithi045.stdout: "MANAGED_BY_CLUSTERS": { 2024-04-16T17:17:31.917 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-04-16T17:17:31.917 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:31.917 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:31.917 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:31.917 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "[]", 2024-04-16T17:17:31.917 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:31.918 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:31.918 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:31.918 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:31.918 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:31.918 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:31.918 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:31.918 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:31.918 INFO:teuthology.orchestra.run.smithi045.stdout: "MULTICLUSTER_CONFIG": { 2024-04-16T17:17:31.918 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "MULTICLUSTER_CONFIG", 2024-04-16T17:17:31.918 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:31.919 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:31.919 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:31.919 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "{}", 2024-04-16T17:17:31.919 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:31.919 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:31.919 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:31.919 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:31.919 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:31.919 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:31.919 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:31.920 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:31.920 INFO:teuthology.orchestra.run.smithi045.stdout: "PROMETHEUS_API_HOST": { 2024-04-16T17:17:31.920 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-16T17:17:31.920 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:31.920 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:31.920 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:31.920 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:31.920 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:31.920 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:31.920 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:31.921 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:31.921 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:31.921 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:31.921 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:31.921 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:31.921 INFO:teuthology.orchestra.run.smithi045.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-16T17:17:31.921 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-16T17:17:31.921 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:31.921 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:31.922 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:31.922 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-16T17:17:31.922 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:31.922 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:31.922 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:31.922 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:31.922 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:31.922 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:31.923 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:31.923 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:31.923 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-16T17:17:31.923 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-16T17:17:31.923 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:31.923 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:31.923 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:31.923 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:31.923 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:31.924 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:31.924 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:31.924 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:31.924 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:31.924 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:31.924 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:31.924 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:31.924 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-16T17:17:31.924 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-16T17:17:31.925 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:31.925 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:31.925 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:31.925 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:31.925 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:31.925 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:31.925 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:31.925 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:31.925 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:31.926 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:31.926 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:31.926 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:31.926 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-16T17:17:31.926 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-16T17:17:31.926 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:31.926 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:31.926 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:31.926 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-16T17:17:31.927 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:31.927 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:31.927 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:31.927 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:31.927 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:31.927 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:31.927 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:31.927 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:31.927 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-16T17:17:31.927 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-16T17:17:31.928 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:31.928 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:31.928 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:31.928 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-16T17:17:31.928 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:31.928 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:31.928 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:31.928 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:31.928 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:31.928 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:31.929 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:31.929 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:31.929 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-16T17:17:31.929 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-16T17:17:31.929 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:31.929 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:31.929 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:31.929 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:31.929 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:31.930 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:31.930 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:31.930 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:31.930 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:31.930 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:31.930 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:31.930 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:31.930 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-16T17:17:31.930 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-16T17:17:31.931 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:31.931 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:31.931 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:31.931 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:31.931 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:31.931 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:31.931 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:31.931 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:31.931 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:31.932 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:31.932 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:31.932 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:31.932 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-16T17:17:31.932 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-16T17:17:31.932 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:31.932 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:31.932 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:31.932 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:31.933 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:31.933 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:31.933 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:31.933 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:31.933 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:31.933 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:31.933 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:31.933 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:31.933 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_ENABLED": { 2024-04-16T17:17:31.934 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-16T17:17:31.934 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:31.934 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:31.934 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:31.934 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-16T17:17:31.934 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:31.934 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:31.934 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:31.934 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:31.934 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:31.935 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:31.935 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:31.935 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:31.935 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-16T17:17:31.935 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-16T17:17:31.935 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:31.935 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:31.935 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:31.936 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-16T17:17:31.936 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:31.936 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:31.936 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:31.936 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:31.936 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:31.936 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:31.936 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:31.936 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:31.937 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-16T17:17:31.937 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-16T17:17:31.937 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-16T17:17:31.937 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:31.937 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:31.937 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "10", 2024-04-16T17:17:31.937 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:31.937 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:31.937 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:31.938 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:31.938 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:31.938 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:31.938 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:31.938 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:31.938 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-16T17:17:31.938 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-16T17:17:31.938 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-16T17:17:31.938 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:31.939 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:31.939 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "8", 2024-04-16T17:17:31.939 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:31.939 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:31.939 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:31.939 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:31.939 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:31.939 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:31.939 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:31.940 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:31.940 INFO:teuthology.orchestra.run.smithi045.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-16T17:17:31.940 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-16T17:17:31.940 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-16T17:17:31.940 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:31.940 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:31.940 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "45", 2024-04-16T17:17:31.940 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:31.940 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:31.940 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:31.941 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:31.941 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:31.941 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:31.941 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:31.941 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:31.941 INFO:teuthology.orchestra.run.smithi045.stdout: "RGW_API_ACCESS_KEY": { 2024-04-16T17:17:31.941 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-16T17:17:31.941 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:31.941 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:31.942 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:31.942 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:31.942 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:31.942 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:31.942 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:31.942 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:31.942 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:31.942 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:31.942 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:31.942 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:31.943 INFO:teuthology.orchestra.run.smithi045.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-16T17:17:31.943 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-16T17:17:31.943 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:31.943 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:31.943 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:31.943 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "admin", 2024-04-16T17:17:31.943 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:31.943 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:31.943 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:31.944 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:31.944 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:31.944 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:31.944 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:31.944 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:31.944 INFO:teuthology.orchestra.run.smithi045.stdout: "RGW_API_SECRET_KEY": { 2024-04-16T17:17:31.944 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-16T17:17:31.944 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:31.944 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:31.944 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:31.945 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:31.945 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:31.945 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:31.945 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:31.945 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:31.945 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:31.945 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:31.945 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:31.945 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:31.946 INFO:teuthology.orchestra.run.smithi045.stdout: "RGW_API_SSL_VERIFY": { 2024-04-16T17:17:31.946 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-16T17:17:31.946 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:31.946 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:31.946 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:31.946 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-16T17:17:31.946 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:31.946 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:31.946 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:31.946 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:31.947 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:31.947 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:31.947 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:31.947 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:31.947 INFO:teuthology.orchestra.run.smithi045.stdout: "UNSAFE_TLS_v1_2": { 2024-04-16T17:17:31.947 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "UNSAFE_TLS_v1_2", 2024-04-16T17:17:31.947 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:31.947 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:31.947 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:31.948 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:31.948 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:31.948 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:31.948 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:31.948 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:31.948 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:31.948 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:31.948 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:31.948 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:31.948 INFO:teuthology.orchestra.run.smithi045.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-16T17:17:31.948 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-16T17:17:31.949 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-16T17:17:31.949 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:31.949 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:31.949 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0", 2024-04-16T17:17:31.949 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:31.949 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:31.949 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:31.949 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:31.950 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:31.950 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:31.950 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:31.950 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:31.950 INFO:teuthology.orchestra.run.smithi045.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-16T17:17:31.950 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-16T17:17:31.950 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-16T17:17:31.950 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:31.950 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:31.951 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "10", 2024-04-16T17:17:31.951 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:31.951 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:31.951 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:31.951 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:31.951 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:31.951 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:31.951 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:31.951 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:31.952 INFO:teuthology.orchestra.run.smithi045.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-16T17:17:31.952 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-16T17:17:31.952 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-16T17:17:31.952 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:31.952 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:31.952 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "5", 2024-04-16T17:17:31.952 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:31.952 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:31.952 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:31.953 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:31.953 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:31.953 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:31.953 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:31.953 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:31.953 INFO:teuthology.orchestra.run.smithi045.stdout: "cross_origin_url": { 2024-04-16T17:17:31.953 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "cross_origin_url", 2024-04-16T17:17:31.953 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:31.954 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:31.954 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:31.954 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:31.954 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:31.954 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:31.954 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:31.954 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:31.954 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:31.954 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:31.955 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:31.955 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:31.955 INFO:teuthology.orchestra.run.smithi045.stdout: "crt_file": { 2024-04-16T17:17:31.955 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "crt_file", 2024-04-16T17:17:31.955 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:31.955 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:31.955 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:31.955 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:31.955 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:31.956 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:31.956 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:31.956 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:31.956 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:31.956 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:31.956 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:31.956 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:31.956 INFO:teuthology.orchestra.run.smithi045.stdout: "debug": { 2024-04-16T17:17:31.956 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "debug", 2024-04-16T17:17:31.957 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:31.957 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:31.957 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:31.957 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:31.957 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:31.957 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:31.957 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:31.957 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Enable/disable debug options", 2024-04-16T17:17:31.957 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:31.957 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:31.958 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:31.958 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:31.958 INFO:teuthology.orchestra.run.smithi045.stdout: "jwt_token_ttl": { 2024-04-16T17:17:31.958 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "jwt_token_ttl", 2024-04-16T17:17:31.958 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-16T17:17:31.958 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:31.958 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:31.958 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "28800", 2024-04-16T17:17:31.958 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:31.959 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:31.959 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:31.959 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:31.959 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:31.959 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:31.959 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:31.959 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:31.959 INFO:teuthology.orchestra.run.smithi045.stdout: "key_file": { 2024-04-16T17:17:31.959 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "key_file", 2024-04-16T17:17:31.959 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:31.960 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:31.960 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:31.960 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:31.960 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:31.960 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:31.960 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:31.960 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:31.960 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:31.960 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:31.961 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:31.961 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:31.961 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-16T17:17:31.961 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-16T17:17:31.961 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:31.961 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:31.961 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:31.961 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:31.961 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:31.961 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:31.962 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-16T17:17:31.962 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-16T17:17:31.962 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-16T17:17:31.962 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-16T17:17:31.962 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-16T17:17:31.962 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-16T17:17:31.962 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-16T17:17:31.962 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-16T17:17:31.962 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:31.963 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:31.963 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:31.963 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:31.963 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:31.963 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-16T17:17:31.963 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-16T17:17:31.963 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:31.963 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:31.963 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:31.963 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:31.964 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:31.964 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:31.964 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:31.964 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:31.964 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:31.964 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:31.964 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:31.964 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:31.964 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-16T17:17:31.965 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-16T17:17:31.965 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:31.965 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:31.965 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:31.965 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-16T17:17:31.965 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:31.965 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:31.965 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-16T17:17:31.965 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-16T17:17:31.965 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-16T17:17:31.966 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-16T17:17:31.966 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-16T17:17:31.966 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-16T17:17:31.966 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-16T17:17:31.966 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-16T17:17:31.966 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:31.966 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:31.966 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:31.966 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:31.967 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:31.967 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-16T17:17:31.967 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-16T17:17:31.967 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:31.967 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:31.967 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:31.967 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:31.967 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:31.967 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:31.968 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:31.968 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:31.968 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:31.968 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:31.968 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:31.968 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:31.968 INFO:teuthology.orchestra.run.smithi045.stdout: "motd": { 2024-04-16T17:17:31.968 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "motd", 2024-04-16T17:17:31.968 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:31.968 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:31.969 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:31.969 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:31.969 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:31.969 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:31.969 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:31.969 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "The message of the day", 2024-04-16T17:17:31.969 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:31.969 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:31.970 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:31.970 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:31.970 INFO:teuthology.orchestra.run.smithi045.stdout: "redirect_resolve_ip_addr": { 2024-04-16T17:17:31.970 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "redirect_resolve_ip_addr", 2024-04-16T17:17:31.970 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:31.970 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:31.970 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:31.970 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:31.970 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:31.971 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:31.971 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:31.971 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:31.971 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:31.971 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:31.971 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:31.971 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:31.971 INFO:teuthology.orchestra.run.smithi045.stdout: "server_addr": { 2024-04-16T17:17:31.971 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "server_addr", 2024-04-16T17:17:31.972 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:31.972 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:31.972 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:31.972 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "::", 2024-04-16T17:17:31.972 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:31.972 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:31.972 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:31.972 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:31.973 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:31.973 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:31.973 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:31.973 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:31.973 INFO:teuthology.orchestra.run.smithi045.stdout: "server_port": { 2024-04-16T17:17:31.973 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "server_port", 2024-04-16T17:17:31.973 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-16T17:17:31.973 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:31.973 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:31.974 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "8080", 2024-04-16T17:17:31.974 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:31.974 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:31.974 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:31.974 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:31.974 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:31.974 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:31.974 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:31.974 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:31.975 INFO:teuthology.orchestra.run.smithi045.stdout: "ssl": { 2024-04-16T17:17:31.975 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ssl", 2024-04-16T17:17:31.975 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:31.975 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:31.975 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:31.975 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-16T17:17:31.975 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:31.975 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:31.975 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:31.976 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:31.976 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:31.976 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:31.976 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:31.976 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:31.976 INFO:teuthology.orchestra.run.smithi045.stdout: "ssl_server_port": { 2024-04-16T17:17:31.976 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ssl_server_port", 2024-04-16T17:17:31.976 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-16T17:17:31.976 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:31.977 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:31.977 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "8443", 2024-04-16T17:17:31.977 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:31.977 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:31.977 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:31.977 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:31.977 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:31.977 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:31.977 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:31.977 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:31.978 INFO:teuthology.orchestra.run.smithi045.stdout: "standby_behaviour": { 2024-04-16T17:17:31.978 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "standby_behaviour", 2024-04-16T17:17:31.978 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:31.978 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:31.978 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:31.978 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "redirect", 2024-04-16T17:17:31.978 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:31.978 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:31.978 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-16T17:17:31.979 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-16T17:17:31.979 INFO:teuthology.orchestra.run.smithi045.stdout: "redirect" 2024-04-16T17:17:31.979 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-16T17:17:31.979 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:31.979 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:31.979 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:31.979 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:31.979 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:31.979 INFO:teuthology.orchestra.run.smithi045.stdout: "standby_error_status_code": { 2024-04-16T17:17:31.980 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "standby_error_status_code", 2024-04-16T17:17:31.980 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-16T17:17:31.980 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:31.980 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:31.980 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "500", 2024-04-16T17:17:31.980 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "400", 2024-04-16T17:17:31.980 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "599", 2024-04-16T17:17:31.980 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:31.980 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:31.980 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:31.981 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:31.981 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:31.981 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:31.981 INFO:teuthology.orchestra.run.smithi045.stdout: "url_prefix": { 2024-04-16T17:17:31.981 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "url_prefix", 2024-04-16T17:17:31.981 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:31.981 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:31.981 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:31.981 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:31.982 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:31.982 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:31.982 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:31.982 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:31.982 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:31.982 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:31.982 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:31.982 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:17:31.983 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:17:31.983 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:31.983 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-16T17:17:31.983 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "devicehealth", 2024-04-16T17:17:31.983 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-16T17:17:31.983 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-16T17:17:31.983 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-16T17:17:31.983 INFO:teuthology.orchestra.run.smithi045.stdout: "enable_monitoring": { 2024-04-16T17:17:31.983 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "enable_monitoring", 2024-04-16T17:17:31.983 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:31.984 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:31.984 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:31.984 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-16T17:17:31.984 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:31.984 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:31.984 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:31.984 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "monitor device health metrics", 2024-04-16T17:17:31.984 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:31.984 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:31.985 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:31.985 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:31.985 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-16T17:17:31.985 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-16T17:17:31.985 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:31.985 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:31.985 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:31.985 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:31.985 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:31.985 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:31.986 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-16T17:17:31.986 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-16T17:17:31.986 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-16T17:17:31.986 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-16T17:17:31.986 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-16T17:17:31.986 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-16T17:17:31.986 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-16T17:17:31.986 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-16T17:17:31.986 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:31.987 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:31.987 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:31.987 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:31.987 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:31.987 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-16T17:17:31.987 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-16T17:17:31.987 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:31.987 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:31.987 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:31.987 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:31.988 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:31.988 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:31.988 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:31.988 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:31.988 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:31.988 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:31.988 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:31.988 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:31.988 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-16T17:17:31.989 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-16T17:17:31.989 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:31.989 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:31.989 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:31.989 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-16T17:17:31.989 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:31.989 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:31.989 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-16T17:17:31.989 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-16T17:17:31.990 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-16T17:17:31.990 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-16T17:17:31.990 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-16T17:17:31.990 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-16T17:17:31.990 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-16T17:17:31.990 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-16T17:17:31.990 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:31.990 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:31.990 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:31.991 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:31.991 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:31.991 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-16T17:17:31.991 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-16T17:17:31.991 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:31.991 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:31.991 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:31.991 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:31.991 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:31.992 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:31.992 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:31.992 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:31.992 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:31.992 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:31.992 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:31.992 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:31.992 INFO:teuthology.orchestra.run.smithi045.stdout: "mark_out_threshold": { 2024-04-16T17:17:31.992 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "mark_out_threshold", 2024-04-16T17:17:31.993 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-04-16T17:17:31.993 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:31.993 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:31.993 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "2419200", 2024-04-16T17:17:31.993 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:31.993 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:31.993 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:31.993 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-16T17:17:31.993 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:31.993 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:31.994 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:31.994 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:31.994 INFO:teuthology.orchestra.run.smithi045.stdout: "pool_name": { 2024-04-16T17:17:31.994 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "pool_name", 2024-04-16T17:17:31.994 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:31.994 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:31.994 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:31.994 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "device_health_metrics", 2024-04-16T17:17:31.994 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:31.995 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:31.995 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:31.995 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-16T17:17:31.995 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:31.995 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:31.995 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:31.995 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:31.995 INFO:teuthology.orchestra.run.smithi045.stdout: "retention_period": { 2024-04-16T17:17:31.995 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "retention_period", 2024-04-16T17:17:31.996 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-04-16T17:17:31.996 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:31.996 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:31.996 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "15552000", 2024-04-16T17:17:31.996 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:31.996 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:31.996 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:31.996 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "how long to retain device health metrics", 2024-04-16T17:17:31.996 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:31.996 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:31.997 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:31.997 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:31.997 INFO:teuthology.orchestra.run.smithi045.stdout: "scrape_frequency": { 2024-04-16T17:17:31.997 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "scrape_frequency", 2024-04-16T17:17:31.997 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-04-16T17:17:31.997 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:31.997 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:31.997 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "86400", 2024-04-16T17:17:31.997 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:31.998 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:31.998 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:31.998 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-16T17:17:31.998 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:31.998 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:31.998 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:31.998 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:31.998 INFO:teuthology.orchestra.run.smithi045.stdout: "self_heal": { 2024-04-16T17:17:31.998 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "self_heal", 2024-04-16T17:17:31.999 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:31.999 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:31.999 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:31.999 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-16T17:17:31.999 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:31.999 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:31.999 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:31.999 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-16T17:17:31.999 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:31.999 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.000 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.000 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.000 INFO:teuthology.orchestra.run.smithi045.stdout: "sleep_interval": { 2024-04-16T17:17:32.000 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "sleep_interval", 2024-04-16T17:17:32.000 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-04-16T17:17:32.000 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.000 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.000 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "600", 2024-04-16T17:17:32.000 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.001 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.001 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.001 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "how frequently to wake up and check device health", 2024-04-16T17:17:32.001 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.001 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.001 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.001 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.001 INFO:teuthology.orchestra.run.smithi045.stdout: "warn_threshold": { 2024-04-16T17:17:32.001 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "warn_threshold", 2024-04-16T17:17:32.002 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-04-16T17:17:32.002 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.002 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.002 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "7257600", 2024-04-16T17:17:32.002 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.002 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.002 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.002 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-16T17:17:32.002 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.002 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.003 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.003 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:17:32.003 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:17:32.003 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.003 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-16T17:17:32.003 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "diskprediction_local", 2024-04-16T17:17:32.003 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-16T17:17:32.003 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-16T17:17:32.003 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-16T17:17:32.004 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-16T17:17:32.004 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-16T17:17:32.004 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.004 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.004 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.004 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:32.004 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.004 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.004 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-16T17:17:32.004 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-16T17:17:32.005 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-16T17:17:32.005 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-16T17:17:32.005 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-16T17:17:32.005 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-16T17:17:32.005 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-16T17:17:32.005 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-16T17:17:32.005 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.005 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.005 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.006 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.006 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.006 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-16T17:17:32.006 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-16T17:17:32.006 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:32.006 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.006 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.006 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:32.006 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.007 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.007 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.007 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.007 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.007 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.007 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.007 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.007 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-16T17:17:32.007 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-16T17:17:32.008 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.008 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.008 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.008 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-16T17:17:32.008 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.008 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.008 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-16T17:17:32.008 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-16T17:17:32.008 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-16T17:17:32.008 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-16T17:17:32.009 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-16T17:17:32.009 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-16T17:17:32.009 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-16T17:17:32.009 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-16T17:17:32.009 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.009 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.009 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.009 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.009 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.009 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-16T17:17:32.010 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-16T17:17:32.010 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:32.010 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.010 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.010 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:32.010 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.010 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.010 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.010 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.011 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.011 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.011 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.011 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.011 INFO:teuthology.orchestra.run.smithi045.stdout: "predict_interval": { 2024-04-16T17:17:32.011 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "predict_interval", 2024-04-16T17:17:32.011 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.011 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.011 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.011 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "86400", 2024-04-16T17:17:32.012 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.012 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.012 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.012 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.012 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.012 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.012 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.012 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.013 INFO:teuthology.orchestra.run.smithi045.stdout: "predictor_model": { 2024-04-16T17:17:32.013 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "predictor_model", 2024-04-16T17:17:32.013 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.013 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.013 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.013 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "prophetstor", 2024-04-16T17:17:32.013 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.013 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.013 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.013 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.014 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.014 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.014 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.014 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.014 INFO:teuthology.orchestra.run.smithi045.stdout: "sleep_interval": { 2024-04-16T17:17:32.014 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "sleep_interval", 2024-04-16T17:17:32.014 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.014 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.014 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.015 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "600", 2024-04-16T17:17:32.015 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.015 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.015 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.015 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.015 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.015 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.015 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.015 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:17:32.015 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:17:32.016 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.016 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-16T17:17:32.016 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "influx", 2024-04-16T17:17:32.016 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": false, 2024-04-16T17:17:32.016 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "influxdb python module not found", 2024-04-16T17:17:32.016 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-16T17:17:32.016 INFO:teuthology.orchestra.run.smithi045.stdout: "batch_size": { 2024-04-16T17:17:32.016 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "batch_size", 2024-04-16T17:17:32.016 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-16T17:17:32.017 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.017 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.017 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "5000", 2024-04-16T17:17:32.017 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.017 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.017 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.017 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-16T17:17:32.017 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.017 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.018 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.018 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.018 INFO:teuthology.orchestra.run.smithi045.stdout: "database": { 2024-04-16T17:17:32.018 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "database", 2024-04-16T17:17:32.018 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.018 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.018 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.018 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "ceph", 2024-04-16T17:17:32.018 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.019 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.019 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.019 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-16T17:17:32.019 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.019 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.019 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.019 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.019 INFO:teuthology.orchestra.run.smithi045.stdout: "hostname": { 2024-04-16T17:17:32.020 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "hostname", 2024-04-16T17:17:32.020 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.020 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.020 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.020 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:32.020 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.020 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.020 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.020 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "InfluxDB server hostname", 2024-04-16T17:17:32.020 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.021 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.021 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.021 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.021 INFO:teuthology.orchestra.run.smithi045.stdout: "interval": { 2024-04-16T17:17:32.021 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "interval", 2024-04-16T17:17:32.021 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-04-16T17:17:32.021 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.021 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.021 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "30", 2024-04-16T17:17:32.022 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "5", 2024-04-16T17:17:32.022 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.022 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.022 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-16T17:17:32.022 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.022 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.022 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.022 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.022 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-16T17:17:32.022 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-16T17:17:32.023 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.023 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.023 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.023 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:32.023 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.023 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.023 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-16T17:17:32.023 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-16T17:17:32.023 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-16T17:17:32.023 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-16T17:17:32.024 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-16T17:17:32.024 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-16T17:17:32.024 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-16T17:17:32.024 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-16T17:17:32.024 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.024 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.024 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.024 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.024 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.025 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-16T17:17:32.025 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-16T17:17:32.025 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:32.025 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.025 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.025 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:32.025 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.025 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.025 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.026 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.026 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.026 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.026 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.026 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.026 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-16T17:17:32.026 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-16T17:17:32.026 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.026 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.027 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.027 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-16T17:17:32.027 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.027 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.027 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-16T17:17:32.027 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-16T17:17:32.027 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-16T17:17:32.027 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-16T17:17:32.027 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-16T17:17:32.028 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-16T17:17:32.028 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-16T17:17:32.028 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-16T17:17:32.028 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.028 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.028 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.028 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.028 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.028 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-16T17:17:32.028 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-16T17:17:32.029 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:32.029 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.029 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.029 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:32.029 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.029 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.029 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.029 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.029 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.029 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.030 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.030 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.030 INFO:teuthology.orchestra.run.smithi045.stdout: "password": { 2024-04-16T17:17:32.030 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "password", 2024-04-16T17:17:32.030 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.030 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.030 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.030 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:32.030 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.031 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.031 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.031 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "password of InfluxDB server user", 2024-04-16T17:17:32.031 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.031 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.031 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.031 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.031 INFO:teuthology.orchestra.run.smithi045.stdout: "port": { 2024-04-16T17:17:32.031 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "port", 2024-04-16T17:17:32.031 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-16T17:17:32.032 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.032 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.032 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "8086", 2024-04-16T17:17:32.032 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.032 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.032 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.032 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "InfluxDB server port", 2024-04-16T17:17:32.032 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.033 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.033 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.033 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.033 INFO:teuthology.orchestra.run.smithi045.stdout: "ssl": { 2024-04-16T17:17:32.033 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ssl", 2024-04-16T17:17:32.033 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.033 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.033 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.033 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "false", 2024-04-16T17:17:32.034 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.034 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.034 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.034 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-16T17:17:32.034 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.034 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.034 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.034 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.034 INFO:teuthology.orchestra.run.smithi045.stdout: "threads": { 2024-04-16T17:17:32.035 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "threads", 2024-04-16T17:17:32.035 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-16T17:17:32.035 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.035 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.035 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "5", 2024-04-16T17:17:32.035 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "1", 2024-04-16T17:17:32.035 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "32", 2024-04-16T17:17:32.035 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.035 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-16T17:17:32.036 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.036 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.036 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.036 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.036 INFO:teuthology.orchestra.run.smithi045.stdout: "username": { 2024-04-16T17:17:32.036 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "username", 2024-04-16T17:17:32.036 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.036 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.036 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.037 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:32.037 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.037 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.037 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.037 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "username of InfluxDB server user", 2024-04-16T17:17:32.037 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.037 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.037 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.037 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.038 INFO:teuthology.orchestra.run.smithi045.stdout: "verify_ssl": { 2024-04-16T17:17:32.038 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "verify_ssl", 2024-04-16T17:17:32.038 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.038 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.038 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.038 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "true", 2024-04-16T17:17:32.038 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.038 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.038 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.039 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-16T17:17:32.039 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.039 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.039 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.039 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:17:32.039 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:17:32.039 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.039 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-16T17:17:32.039 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "insights", 2024-04-16T17:17:32.040 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-16T17:17:32.040 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-16T17:17:32.040 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-16T17:17:32.040 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-16T17:17:32.040 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-16T17:17:32.040 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.040 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.040 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.040 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:32.040 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.041 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.041 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-16T17:17:32.041 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-16T17:17:32.041 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-16T17:17:32.041 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-16T17:17:32.041 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-16T17:17:32.041 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-16T17:17:32.041 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-16T17:17:32.041 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-16T17:17:32.042 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.042 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.042 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.042 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.042 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.042 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-16T17:17:32.042 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-16T17:17:32.042 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:32.042 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.042 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.043 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:32.043 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.043 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.043 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.043 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.043 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.043 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.043 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.043 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.043 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-16T17:17:32.044 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-16T17:17:32.044 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.044 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.044 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.044 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-16T17:17:32.044 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.044 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.044 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-16T17:17:32.044 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-16T17:17:32.045 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-16T17:17:32.045 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-16T17:17:32.045 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-16T17:17:32.045 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-16T17:17:32.045 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-16T17:17:32.045 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-16T17:17:32.045 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.045 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.045 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.046 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.046 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.046 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-16T17:17:32.046 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-16T17:17:32.046 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:32.046 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.046 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.046 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:32.046 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.047 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.047 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.047 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.047 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.047 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.047 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.047 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:17:32.047 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:17:32.047 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.048 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-16T17:17:32.048 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "iostat", 2024-04-16T17:17:32.048 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-16T17:17:32.048 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-16T17:17:32.048 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-16T17:17:32.048 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-16T17:17:32.048 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-16T17:17:32.048 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.048 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.048 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.049 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:32.049 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.049 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.049 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-16T17:17:32.049 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-16T17:17:32.049 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-16T17:17:32.049 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-16T17:17:32.049 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-16T17:17:32.049 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-16T17:17:32.050 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-16T17:17:32.050 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-16T17:17:32.050 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.050 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.050 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.050 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.050 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.050 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-16T17:17:32.051 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-16T17:17:32.051 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:32.051 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.051 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.051 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:32.051 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.051 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.051 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.051 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.052 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.052 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.052 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.052 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.052 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-16T17:17:32.052 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-16T17:17:32.052 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.052 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.052 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.052 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-16T17:17:32.053 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.053 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.053 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-16T17:17:32.053 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-16T17:17:32.053 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-16T17:17:32.053 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-16T17:17:32.053 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-16T17:17:32.053 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-16T17:17:32.053 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-16T17:17:32.053 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-16T17:17:32.054 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.054 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.054 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.054 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.054 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.054 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-16T17:17:32.054 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-16T17:17:32.054 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:32.055 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.055 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.055 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:32.055 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.055 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.055 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.055 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.055 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.055 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.056 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.056 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:17:32.056 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:17:32.056 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.056 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-16T17:17:32.056 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "k8sevents", 2024-04-16T17:17:32.056 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-16T17:17:32.056 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-16T17:17:32.056 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-16T17:17:32.057 INFO:teuthology.orchestra.run.smithi045.stdout: "ceph_event_retention_days": { 2024-04-16T17:17:32.057 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ceph_event_retention_days", 2024-04-16T17:17:32.057 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-16T17:17:32.057 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.057 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.057 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "7", 2024-04-16T17:17:32.057 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.057 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.057 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.057 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-16T17:17:32.058 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.058 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.058 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.058 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.058 INFO:teuthology.orchestra.run.smithi045.stdout: "config_check_secs": { 2024-04-16T17:17:32.058 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "config_check_secs", 2024-04-16T17:17:32.058 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-16T17:17:32.058 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.058 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.059 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "10", 2024-04-16T17:17:32.059 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "10", 2024-04-16T17:17:32.059 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.059 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.059 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-16T17:17:32.059 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.059 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.059 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.059 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.060 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-16T17:17:32.060 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-16T17:17:32.060 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.060 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.060 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.060 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:32.060 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.060 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.060 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-16T17:17:32.060 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-16T17:17:32.061 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-16T17:17:32.061 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-16T17:17:32.061 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-16T17:17:32.061 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-16T17:17:32.061 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-16T17:17:32.061 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-16T17:17:32.061 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.061 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.061 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.062 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.062 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.062 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-16T17:17:32.062 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-16T17:17:32.062 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:32.062 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.062 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.062 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:32.062 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.063 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.063 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.063 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.063 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.063 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.063 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.063 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.063 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-16T17:17:32.063 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-16T17:17:32.063 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.064 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.064 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.064 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-16T17:17:32.064 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.064 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.064 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-16T17:17:32.064 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-16T17:17:32.064 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-16T17:17:32.064 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-16T17:17:32.064 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-16T17:17:32.065 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-16T17:17:32.065 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-16T17:17:32.065 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-16T17:17:32.065 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.065 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.065 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.065 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.065 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.065 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-16T17:17:32.066 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-16T17:17:32.066 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:32.066 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.066 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.066 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:32.066 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.066 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.066 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.066 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.066 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.067 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.067 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.067 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:17:32.067 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:17:32.067 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.067 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-16T17:17:32.067 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "localpool", 2024-04-16T17:17:32.067 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-16T17:17:32.067 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-16T17:17:32.068 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-16T17:17:32.068 INFO:teuthology.orchestra.run.smithi045.stdout: "failure_domain": { 2024-04-16T17:17:32.068 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "failure_domain", 2024-04-16T17:17:32.068 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.068 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.068 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.068 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "host", 2024-04-16T17:17:32.068 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.068 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.069 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.069 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "failure domain for any created local pool", 2024-04-16T17:17:32.069 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-16T17:17:32.069 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.069 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.069 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.069 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-16T17:17:32.069 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-16T17:17:32.069 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.070 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.070 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.070 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:32.070 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.070 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.070 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-16T17:17:32.070 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-16T17:17:32.070 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-16T17:17:32.070 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-16T17:17:32.071 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-16T17:17:32.071 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-16T17:17:32.071 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-16T17:17:32.071 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-16T17:17:32.071 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.071 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.071 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.071 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.071 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.071 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-16T17:17:32.072 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-16T17:17:32.072 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:32.072 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.072 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.072 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:32.072 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.072 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.072 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.072 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.073 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.073 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.073 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.073 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.073 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-16T17:17:32.073 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-16T17:17:32.073 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.073 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.073 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.074 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-16T17:17:32.074 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.074 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.074 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-16T17:17:32.074 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-16T17:17:32.074 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-16T17:17:32.074 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-16T17:17:32.074 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-16T17:17:32.074 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-16T17:17:32.074 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-16T17:17:32.075 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-16T17:17:32.075 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.075 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.075 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.075 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.075 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.075 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-16T17:17:32.075 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-16T17:17:32.076 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:32.076 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.076 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.076 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:32.076 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.076 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.076 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.076 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.076 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.077 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.077 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.077 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.077 INFO:teuthology.orchestra.run.smithi045.stdout: "min_size": { 2024-04-16T17:17:32.077 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "min_size", 2024-04-16T17:17:32.077 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-16T17:17:32.077 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.077 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.077 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:32.078 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.078 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.078 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.078 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "default min_size for any created local pool", 2024-04-16T17:17:32.078 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-16T17:17:32.078 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.078 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.078 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.078 INFO:teuthology.orchestra.run.smithi045.stdout: "num_rep": { 2024-04-16T17:17:32.078 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "num_rep", 2024-04-16T17:17:32.079 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-16T17:17:32.079 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.079 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.079 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "3", 2024-04-16T17:17:32.079 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.079 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.079 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.079 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "default replica count for any created local pool", 2024-04-16T17:17:32.079 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.079 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.080 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.080 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.080 INFO:teuthology.orchestra.run.smithi045.stdout: "pg_num": { 2024-04-16T17:17:32.080 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "pg_num", 2024-04-16T17:17:32.080 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-16T17:17:32.080 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.080 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.080 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "128", 2024-04-16T17:17:32.080 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.080 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.081 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.081 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "default pg_num for any created local pool", 2024-04-16T17:17:32.081 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.081 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.081 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.081 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.081 INFO:teuthology.orchestra.run.smithi045.stdout: "prefix": { 2024-04-16T17:17:32.081 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "prefix", 2024-04-16T17:17:32.081 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.082 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.082 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.082 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:32.082 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.082 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.082 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.082 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "name prefix for any created local pool", 2024-04-16T17:17:32.082 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.082 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.082 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.083 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.083 INFO:teuthology.orchestra.run.smithi045.stdout: "subtree": { 2024-04-16T17:17:32.083 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "subtree", 2024-04-16T17:17:32.083 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.083 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.083 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.083 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "rack", 2024-04-16T17:17:32.083 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.083 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.083 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.084 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-16T17:17:32.084 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-16T17:17:32.084 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.084 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.084 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:17:32.084 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:17:32.084 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.084 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-16T17:17:32.084 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "mds_autoscaler", 2024-04-16T17:17:32.085 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-16T17:17:32.085 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-16T17:17:32.085 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-16T17:17:32.085 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-16T17:17:32.085 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-16T17:17:32.085 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.085 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.085 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.085 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:32.085 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.086 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.086 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-16T17:17:32.086 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-16T17:17:32.086 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-16T17:17:32.086 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-16T17:17:32.086 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-16T17:17:32.086 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-16T17:17:32.086 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-16T17:17:32.086 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-16T17:17:32.087 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.087 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.087 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.087 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.087 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.087 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-16T17:17:32.087 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-16T17:17:32.087 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:32.087 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.088 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.088 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:32.088 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.088 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.088 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.088 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.088 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.088 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.088 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.088 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.089 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-16T17:17:32.089 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-16T17:17:32.089 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.089 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.089 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.089 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-16T17:17:32.089 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.089 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.089 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-16T17:17:32.090 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-16T17:17:32.090 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-16T17:17:32.090 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-16T17:17:32.090 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-16T17:17:32.090 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-16T17:17:32.090 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-16T17:17:32.090 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-16T17:17:32.090 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.091 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.091 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.091 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.091 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.091 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-16T17:17:32.091 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-16T17:17:32.091 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:32.091 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.091 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.092 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:32.092 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.092 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.092 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.092 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.092 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.092 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.092 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.092 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:17:32.093 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:17:32.093 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.093 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-16T17:17:32.093 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "mirroring", 2024-04-16T17:17:32.093 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-16T17:17:32.093 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-16T17:17:32.093 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-16T17:17:32.093 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-16T17:17:32.093 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-16T17:17:32.094 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.094 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.094 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.094 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:32.094 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.094 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.094 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-16T17:17:32.094 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-16T17:17:32.094 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-16T17:17:32.095 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-16T17:17:32.095 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-16T17:17:32.095 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-16T17:17:32.095 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-16T17:17:32.095 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-16T17:17:32.095 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.095 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.095 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.095 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.096 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.096 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-16T17:17:32.096 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-16T17:17:32.096 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:32.096 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.096 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.096 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:32.096 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.096 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.096 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.097 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.097 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.097 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.097 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.097 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.097 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-16T17:17:32.097 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-16T17:17:32.097 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.098 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.098 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.098 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-16T17:17:32.098 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.098 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.098 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-16T17:17:32.098 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-16T17:17:32.098 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-16T17:17:32.098 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-16T17:17:32.099 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-16T17:17:32.099 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-16T17:17:32.099 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-16T17:17:32.099 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-16T17:17:32.099 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.099 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.099 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.099 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.099 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.100 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-16T17:17:32.100 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-16T17:17:32.100 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:32.100 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.100 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.100 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:32.100 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.100 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.100 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.100 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.101 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.101 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.101 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.101 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:17:32.101 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:17:32.101 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.101 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-16T17:17:32.101 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "nfs", 2024-04-16T17:17:32.102 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-16T17:17:32.102 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-16T17:17:32.102 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-16T17:17:32.102 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-16T17:17:32.102 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-16T17:17:32.102 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.102 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.102 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.103 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:32.103 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.103 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.103 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-16T17:17:32.103 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-16T17:17:32.103 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-16T17:17:32.103 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-16T17:17:32.103 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-16T17:17:32.103 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-16T17:17:32.104 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-16T17:17:32.104 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-16T17:17:32.104 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.104 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.104 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.104 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.104 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.104 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-16T17:17:32.104 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-16T17:17:32.104 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:32.105 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.105 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.105 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:32.105 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.105 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.105 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.105 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.105 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.105 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.105 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.106 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.106 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-16T17:17:32.106 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-16T17:17:32.106 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.106 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.106 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.106 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-16T17:17:32.106 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.106 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.107 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-16T17:17:32.107 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-16T17:17:32.107 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-16T17:17:32.107 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-16T17:17:32.107 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-16T17:17:32.107 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-16T17:17:32.107 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-16T17:17:32.107 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-16T17:17:32.107 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.108 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.108 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.108 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.108 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.108 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-16T17:17:32.108 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-16T17:17:32.108 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:32.108 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.109 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.109 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:32.109 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.109 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.109 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.109 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.109 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.109 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.109 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.110 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:17:32.110 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:17:32.110 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.110 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-16T17:17:32.110 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "orchestrator", 2024-04-16T17:17:32.110 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-16T17:17:32.110 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-16T17:17:32.110 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-16T17:17:32.110 INFO:teuthology.orchestra.run.smithi045.stdout: "fail_fs": { 2024-04-16T17:17:32.111 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "fail_fs", 2024-04-16T17:17:32.111 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:32.111 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.111 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.111 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:32.111 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.111 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.111 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.111 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-16T17:17:32.112 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.112 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.112 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.112 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.112 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-16T17:17:32.112 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-16T17:17:32.112 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.112 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.113 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.113 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:32.113 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.113 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.113 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-16T17:17:32.113 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-16T17:17:32.113 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-16T17:17:32.113 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-16T17:17:32.113 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-16T17:17:32.113 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-16T17:17:32.114 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-16T17:17:32.114 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-16T17:17:32.114 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.114 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.114 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.114 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.114 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.114 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-16T17:17:32.114 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-16T17:17:32.115 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:32.115 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.115 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.115 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:32.115 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.115 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.115 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.115 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.115 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.116 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.116 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.116 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.116 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-16T17:17:32.116 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-16T17:17:32.116 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.116 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.116 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.116 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-16T17:17:32.117 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.117 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.117 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-16T17:17:32.117 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-16T17:17:32.117 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-16T17:17:32.117 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-16T17:17:32.117 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-16T17:17:32.117 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-16T17:17:32.117 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-16T17:17:32.118 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-16T17:17:32.118 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.118 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.118 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.118 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.118 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.118 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-16T17:17:32.118 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-16T17:17:32.118 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:32.119 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.119 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.119 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:32.119 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.119 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.119 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.119 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.119 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.119 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.119 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.120 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.120 INFO:teuthology.orchestra.run.smithi045.stdout: "orchestrator": { 2024-04-16T17:17:32.120 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "orchestrator", 2024-04-16T17:17:32.120 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.120 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.120 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.120 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:32.120 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.120 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.120 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-16T17:17:32.121 INFO:teuthology.orchestra.run.smithi045.stdout: "cephadm", 2024-04-16T17:17:32.121 INFO:teuthology.orchestra.run.smithi045.stdout: "rook", 2024-04-16T17:17:32.121 INFO:teuthology.orchestra.run.smithi045.stdout: "test_orchestrator" 2024-04-16T17:17:32.121 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-16T17:17:32.121 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Orchestrator backend", 2024-04-16T17:17:32.121 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.121 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.121 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.121 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:17:32.122 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:17:32.122 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.122 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-16T17:17:32.122 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "osd_perf_query", 2024-04-16T17:17:32.122 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-16T17:17:32.122 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-16T17:17:32.122 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-16T17:17:32.122 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-16T17:17:32.122 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-16T17:17:32.123 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.123 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.123 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.123 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:32.123 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.123 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.123 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-16T17:17:32.123 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-16T17:17:32.123 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-16T17:17:32.123 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-16T17:17:32.124 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-16T17:17:32.124 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-16T17:17:32.124 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-16T17:17:32.124 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-16T17:17:32.124 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.124 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.124 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.124 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.124 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.124 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-16T17:17:32.125 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-16T17:17:32.125 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:32.125 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.125 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.125 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:32.125 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.125 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.125 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.125 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.126 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.126 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.126 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.126 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.126 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-16T17:17:32.126 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-16T17:17:32.126 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.126 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.127 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.127 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-16T17:17:32.127 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.127 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.127 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-16T17:17:32.127 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-16T17:17:32.127 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-16T17:17:32.127 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-16T17:17:32.127 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-16T17:17:32.127 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-16T17:17:32.128 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-16T17:17:32.128 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-16T17:17:32.128 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.128 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.128 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.128 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.128 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.128 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-16T17:17:32.128 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-16T17:17:32.128 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:32.129 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.129 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.129 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:32.129 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.129 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.129 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.129 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.129 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.129 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.130 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.130 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:17:32.130 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:17:32.130 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.130 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-16T17:17:32.130 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "osd_support", 2024-04-16T17:17:32.130 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-16T17:17:32.130 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-16T17:17:32.130 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-16T17:17:32.131 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-16T17:17:32.131 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-16T17:17:32.131 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.131 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.131 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.131 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:32.131 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.131 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.131 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-16T17:17:32.131 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-16T17:17:32.132 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-16T17:17:32.132 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-16T17:17:32.132 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-16T17:17:32.132 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-16T17:17:32.132 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-16T17:17:32.132 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-16T17:17:32.132 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.132 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.132 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.132 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.133 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.133 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-16T17:17:32.133 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-16T17:17:32.133 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:32.133 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.133 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.133 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:32.133 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.133 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.133 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.134 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.134 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.134 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.134 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.134 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.134 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-16T17:17:32.134 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-16T17:17:32.134 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.134 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.135 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.135 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-16T17:17:32.135 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.135 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.135 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-16T17:17:32.135 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-16T17:17:32.135 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-16T17:17:32.135 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-16T17:17:32.135 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-16T17:17:32.136 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-16T17:17:32.136 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-16T17:17:32.136 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-16T17:17:32.136 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.136 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.136 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.136 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.136 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.136 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-16T17:17:32.137 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-16T17:17:32.137 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:32.137 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.137 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.137 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:32.137 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.137 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.137 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.137 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.138 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.138 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.138 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.138 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:17:32.138 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:17:32.138 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.138 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-16T17:17:32.138 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "pg_autoscaler", 2024-04-16T17:17:32.139 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-16T17:17:32.139 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-16T17:17:32.139 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-16T17:17:32.139 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-16T17:17:32.139 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-16T17:17:32.139 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.139 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.139 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.139 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:32.140 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.140 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.140 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-16T17:17:32.140 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-16T17:17:32.140 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-16T17:17:32.140 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-16T17:17:32.140 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-16T17:17:32.140 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-16T17:17:32.140 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-16T17:17:32.141 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-16T17:17:32.141 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.141 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.141 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.141 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.141 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.141 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-16T17:17:32.141 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-16T17:17:32.141 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:32.141 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.142 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.142 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:32.142 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.142 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.142 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.142 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.142 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.142 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.142 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.143 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.143 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-16T17:17:32.143 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-16T17:17:32.143 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.143 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.143 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.143 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-16T17:17:32.143 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.143 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.143 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-16T17:17:32.144 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-16T17:17:32.144 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-16T17:17:32.144 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-16T17:17:32.144 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-16T17:17:32.144 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-16T17:17:32.144 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-16T17:17:32.144 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-16T17:17:32.144 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.144 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.144 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.145 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.145 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.145 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-16T17:17:32.145 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-16T17:17:32.145 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:32.145 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.145 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.145 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:32.145 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.145 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.146 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.146 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.146 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.146 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.146 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.146 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.146 INFO:teuthology.orchestra.run.smithi045.stdout: "sleep_interval": { 2024-04-16T17:17:32.146 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "sleep_interval", 2024-04-16T17:17:32.146 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-04-16T17:17:32.147 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.147 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.147 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "60", 2024-04-16T17:17:32.147 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.147 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.147 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.147 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.147 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.148 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.148 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.148 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.148 INFO:teuthology.orchestra.run.smithi045.stdout: "threshold": { 2024-04-16T17:17:32.148 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "threshold", 2024-04-16T17:17:32.148 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "float", 2024-04-16T17:17:32.148 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.148 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.148 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "3.0", 2024-04-16T17:17:32.149 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "1.0", 2024-04-16T17:17:32.149 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.149 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.149 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "scaling threshold", 2024-04-16T17:17:32.149 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-16T17:17:32.149 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.149 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.149 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:17:32.149 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:17:32.149 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.150 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-16T17:17:32.150 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "progress", 2024-04-16T17:17:32.150 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-16T17:17:32.150 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-16T17:17:32.150 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-16T17:17:32.150 INFO:teuthology.orchestra.run.smithi045.stdout: "allow_pg_recovery_event": { 2024-04-16T17:17:32.150 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "allow_pg_recovery_event", 2024-04-16T17:17:32.150 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:32.150 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.151 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.151 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:32.151 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.151 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.151 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.151 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-16T17:17:32.151 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.151 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.151 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.152 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.152 INFO:teuthology.orchestra.run.smithi045.stdout: "enabled": { 2024-04-16T17:17:32.152 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "enabled", 2024-04-16T17:17:32.152 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:32.152 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.152 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.152 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-16T17:17:32.152 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.152 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.153 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.153 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.153 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.153 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.153 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.153 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.153 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-16T17:17:32.153 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-16T17:17:32.153 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.154 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.154 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.154 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:32.154 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.154 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.154 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-16T17:17:32.154 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-16T17:17:32.154 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-16T17:17:32.155 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-16T17:17:32.155 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-16T17:17:32.155 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-16T17:17:32.155 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-16T17:17:32.155 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-16T17:17:32.155 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.155 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.155 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.155 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.155 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.156 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-16T17:17:32.156 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-16T17:17:32.156 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:32.156 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.156 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.156 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:32.156 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.156 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.157 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.157 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.157 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.157 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.157 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.157 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.157 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-16T17:17:32.157 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-16T17:17:32.157 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.157 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.158 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.158 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-16T17:17:32.158 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.158 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.158 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-16T17:17:32.158 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-16T17:17:32.158 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-16T17:17:32.158 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-16T17:17:32.158 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-16T17:17:32.158 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-16T17:17:32.159 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-16T17:17:32.159 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-16T17:17:32.159 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.159 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.159 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.159 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.159 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.159 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-16T17:17:32.159 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-16T17:17:32.159 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:32.160 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.160 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.160 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:32.160 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.160 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.160 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.160 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.160 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.160 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.161 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.161 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.161 INFO:teuthology.orchestra.run.smithi045.stdout: "max_completed_events": { 2024-04-16T17:17:32.161 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "max_completed_events", 2024-04-16T17:17:32.161 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-16T17:17:32.161 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.161 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.161 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "50", 2024-04-16T17:17:32.161 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.162 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.162 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.162 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "number of past completed events to remember", 2024-04-16T17:17:32.162 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.162 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.162 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.162 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.162 INFO:teuthology.orchestra.run.smithi045.stdout: "sleep_interval": { 2024-04-16T17:17:32.162 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "sleep_interval", 2024-04-16T17:17:32.162 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-04-16T17:17:32.163 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.163 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.163 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "5", 2024-04-16T17:17:32.163 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.163 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.163 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.163 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "how long the module is going to sleep", 2024-04-16T17:17:32.163 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.163 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.163 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.164 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:17:32.164 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:17:32.164 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.164 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-16T17:17:32.164 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "prometheus", 2024-04-16T17:17:32.164 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-16T17:17:32.164 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-16T17:17:32.164 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-16T17:17:32.164 INFO:teuthology.orchestra.run.smithi045.stdout: "cache": { 2024-04-16T17:17:32.164 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "cache", 2024-04-16T17:17:32.165 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:32.165 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.165 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.165 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-16T17:17:32.165 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.165 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.165 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.165 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.165 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.166 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.166 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.166 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.166 INFO:teuthology.orchestra.run.smithi045.stdout: "exclude_perf_counters": { 2024-04-16T17:17:32.166 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "exclude_perf_counters", 2024-04-16T17:17:32.166 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:32.166 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.166 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.166 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-16T17:17:32.166 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.167 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.167 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.167 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-16T17:17:32.167 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-16T17:17:32.167 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.167 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.167 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.167 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-16T17:17:32.167 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-16T17:17:32.168 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.168 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.168 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.168 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:32.168 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.168 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.168 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-16T17:17:32.168 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-16T17:17:32.168 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-16T17:17:32.169 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-16T17:17:32.169 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-16T17:17:32.169 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-16T17:17:32.169 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-16T17:17:32.169 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-16T17:17:32.169 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.169 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.169 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.169 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.169 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.170 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-16T17:17:32.170 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-16T17:17:32.170 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:32.170 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.170 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.170 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:32.170 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.170 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.170 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.170 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.171 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.171 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.171 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.171 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.171 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-16T17:17:32.171 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-16T17:17:32.171 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.171 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.171 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.172 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-16T17:17:32.172 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.172 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.172 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-16T17:17:32.172 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-16T17:17:32.172 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-16T17:17:32.172 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-16T17:17:32.172 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-16T17:17:32.172 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-16T17:17:32.173 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-16T17:17:32.173 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-16T17:17:32.173 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.173 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.173 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.173 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.173 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.173 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-16T17:17:32.173 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-16T17:17:32.173 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:32.174 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.174 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.174 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:32.174 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.174 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.174 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.174 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.174 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.174 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.175 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.175 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.175 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_stats_pools": { 2024-04-16T17:17:32.175 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rbd_stats_pools", 2024-04-16T17:17:32.175 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.175 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.175 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.175 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:32.175 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.175 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.176 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.176 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.176 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.176 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.176 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.176 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.176 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-16T17:17:32.176 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-16T17:17:32.176 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-16T17:17:32.177 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.177 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.177 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "300", 2024-04-16T17:17:32.177 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.177 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.177 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.177 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.177 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.177 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.178 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.178 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.178 INFO:teuthology.orchestra.run.smithi045.stdout: "scrape_interval": { 2024-04-16T17:17:32.178 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "scrape_interval", 2024-04-16T17:17:32.178 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "float", 2024-04-16T17:17:32.178 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.178 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.178 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "15.0", 2024-04-16T17:17:32.179 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.179 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.179 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.179 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.179 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.179 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.179 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.179 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.179 INFO:teuthology.orchestra.run.smithi045.stdout: "server_addr": { 2024-04-16T17:17:32.179 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "server_addr", 2024-04-16T17:17:32.180 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.180 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.180 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.180 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "::", 2024-04-16T17:17:32.180 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.180 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.180 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.180 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-16T17:17:32.180 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.181 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.181 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.181 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.181 INFO:teuthology.orchestra.run.smithi045.stdout: "server_port": { 2024-04-16T17:17:32.181 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "server_port", 2024-04-16T17:17:32.181 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-16T17:17:32.181 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.181 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.181 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "9283", 2024-04-16T17:17:32.181 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.182 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.182 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.182 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-16T17:17:32.182 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.182 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.182 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.182 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.182 INFO:teuthology.orchestra.run.smithi045.stdout: "stale_cache_strategy": { 2024-04-16T17:17:32.182 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "stale_cache_strategy", 2024-04-16T17:17:32.182 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.183 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.183 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.183 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "log", 2024-04-16T17:17:32.183 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.183 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.183 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.183 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.183 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.183 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.183 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.184 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.184 INFO:teuthology.orchestra.run.smithi045.stdout: "standby_behaviour": { 2024-04-16T17:17:32.184 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "standby_behaviour", 2024-04-16T17:17:32.184 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.184 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.184 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.184 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "default", 2024-04-16T17:17:32.184 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.184 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.185 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-16T17:17:32.185 INFO:teuthology.orchestra.run.smithi045.stdout: "default", 2024-04-16T17:17:32.185 INFO:teuthology.orchestra.run.smithi045.stdout: "error" 2024-04-16T17:17:32.185 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-16T17:17:32.185 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.185 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.185 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.185 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.185 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.185 INFO:teuthology.orchestra.run.smithi045.stdout: "standby_error_status_code": { 2024-04-16T17:17:32.186 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "standby_error_status_code", 2024-04-16T17:17:32.186 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-16T17:17:32.186 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.186 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.186 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "500", 2024-04-16T17:17:32.186 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "400", 2024-04-16T17:17:32.186 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "599", 2024-04-16T17:17:32.186 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.186 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.187 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.187 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.187 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.187 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:17:32.187 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:17:32.187 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.187 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-16T17:17:32.187 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rbd_support", 2024-04-16T17:17:32.187 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-16T17:17:32.187 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-16T17:17:32.188 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-16T17:17:32.188 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-16T17:17:32.188 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-16T17:17:32.188 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.188 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.188 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.188 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:32.188 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.188 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.189 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-16T17:17:32.189 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-16T17:17:32.189 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-16T17:17:32.189 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-16T17:17:32.189 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-16T17:17:32.189 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-16T17:17:32.189 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-16T17:17:32.189 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-16T17:17:32.189 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.189 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.190 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.190 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.190 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.190 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-16T17:17:32.190 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-16T17:17:32.190 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:32.190 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.190 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.191 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:32.191 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.191 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.191 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.191 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.191 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.191 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.191 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.191 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.192 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-16T17:17:32.192 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-16T17:17:32.192 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.192 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.192 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.192 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-16T17:17:32.192 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.192 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.192 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-16T17:17:32.192 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-16T17:17:32.193 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-16T17:17:32.193 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-16T17:17:32.193 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-16T17:17:32.193 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-16T17:17:32.193 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-16T17:17:32.193 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-16T17:17:32.193 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.193 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.193 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.193 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.194 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.194 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-16T17:17:32.194 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-16T17:17:32.194 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:32.194 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.194 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.194 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:32.194 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.194 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.195 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.195 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.195 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.195 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.195 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.195 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.195 INFO:teuthology.orchestra.run.smithi045.stdout: "max_concurrent_snap_create": { 2024-04-16T17:17:32.195 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "max_concurrent_snap_create", 2024-04-16T17:17:32.195 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-16T17:17:32.195 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.196 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.196 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "10", 2024-04-16T17:17:32.196 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.196 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.196 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.196 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.196 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.196 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.196 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.196 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.197 INFO:teuthology.orchestra.run.smithi045.stdout: "mirror_snapshot_schedule": { 2024-04-16T17:17:32.197 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "mirror_snapshot_schedule", 2024-04-16T17:17:32.197 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.197 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.197 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.197 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:32.197 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.197 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.197 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.197 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.198 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.198 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.198 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.198 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.198 INFO:teuthology.orchestra.run.smithi045.stdout: "trash_purge_schedule": { 2024-04-16T17:17:32.198 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "trash_purge_schedule", 2024-04-16T17:17:32.198 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.198 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.199 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.199 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:32.199 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.199 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.199 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.199 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.199 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.199 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.199 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.199 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:17:32.200 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:17:32.200 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.200 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-16T17:17:32.200 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "restful", 2024-04-16T17:17:32.200 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-16T17:17:32.200 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-16T17:17:32.200 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-16T17:17:32.200 INFO:teuthology.orchestra.run.smithi045.stdout: "enable_auth": { 2024-04-16T17:17:32.200 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "enable_auth", 2024-04-16T17:17:32.201 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:32.201 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.201 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.201 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-16T17:17:32.201 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.201 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.201 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.201 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.201 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.201 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.202 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.202 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.202 INFO:teuthology.orchestra.run.smithi045.stdout: "key_file": { 2024-04-16T17:17:32.202 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "key_file", 2024-04-16T17:17:32.202 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.202 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.202 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.202 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:32.203 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.203 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.203 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.203 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.203 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.203 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.203 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.203 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.203 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-16T17:17:32.204 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-16T17:17:32.204 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.204 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.204 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.204 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:32.204 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.204 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.204 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-16T17:17:32.204 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-16T17:17:32.205 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-16T17:17:32.205 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-16T17:17:32.205 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-16T17:17:32.205 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-16T17:17:32.205 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-16T17:17:32.205 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-16T17:17:32.205 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.205 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.205 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.206 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.206 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.206 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-16T17:17:32.206 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-16T17:17:32.206 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:32.206 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.206 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.206 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:32.206 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.206 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.207 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.207 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.207 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.207 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.207 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.207 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.207 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-16T17:17:32.207 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-16T17:17:32.207 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.207 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.208 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.208 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-16T17:17:32.208 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.208 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.208 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-16T17:17:32.208 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-16T17:17:32.208 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-16T17:17:32.208 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-16T17:17:32.208 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-16T17:17:32.208 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-16T17:17:32.209 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-16T17:17:32.209 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-16T17:17:32.209 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.209 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.209 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.209 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.209 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.209 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-16T17:17:32.209 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-16T17:17:32.209 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:32.210 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.210 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.210 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:32.210 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.210 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.210 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.210 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.210 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.210 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.211 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.211 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.211 INFO:teuthology.orchestra.run.smithi045.stdout: "server_addr": { 2024-04-16T17:17:32.211 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "server_addr", 2024-04-16T17:17:32.211 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.211 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.211 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.211 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:32.211 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.211 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.212 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.212 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.212 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.212 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.212 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.212 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.212 INFO:teuthology.orchestra.run.smithi045.stdout: "server_port": { 2024-04-16T17:17:32.212 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "server_port", 2024-04-16T17:17:32.212 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.213 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.213 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.213 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:32.213 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.213 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.213 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.213 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.213 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.213 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.213 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.214 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:17:32.214 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:17:32.214 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.214 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-16T17:17:32.214 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rgw", 2024-04-16T17:17:32.214 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-16T17:17:32.214 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-16T17:17:32.214 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-16T17:17:32.214 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-16T17:17:32.215 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-16T17:17:32.215 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.215 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.215 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.215 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:32.215 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.215 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.215 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-16T17:17:32.215 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-16T17:17:32.216 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-16T17:17:32.216 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-16T17:17:32.216 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-16T17:17:32.216 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-16T17:17:32.216 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-16T17:17:32.216 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-16T17:17:32.216 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.216 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.216 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.216 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.217 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.217 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-16T17:17:32.217 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-16T17:17:32.217 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:32.217 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.217 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.217 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:32.217 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.217 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.218 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.218 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.218 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.218 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.218 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.218 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.218 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-16T17:17:32.218 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-16T17:17:32.218 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.219 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.219 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.219 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-16T17:17:32.219 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.219 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.219 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-16T17:17:32.219 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-16T17:17:32.219 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-16T17:17:32.219 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-16T17:17:32.219 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-16T17:17:32.220 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-16T17:17:32.220 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-16T17:17:32.220 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-16T17:17:32.220 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.220 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.220 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.220 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.220 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.220 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-16T17:17:32.220 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-16T17:17:32.221 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:32.221 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.221 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.221 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:32.221 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.221 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.221 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.221 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.221 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.222 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.222 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.222 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:17:32.222 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:17:32.222 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.222 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-16T17:17:32.222 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rook", 2024-04-16T17:17:32.222 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-16T17:17:32.222 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-16T17:17:32.223 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-16T17:17:32.223 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-16T17:17:32.223 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-16T17:17:32.223 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.223 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.223 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.223 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:32.223 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.223 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.223 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-16T17:17:32.224 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-16T17:17:32.224 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-16T17:17:32.224 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-16T17:17:32.224 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-16T17:17:32.224 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-16T17:17:32.224 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-16T17:17:32.224 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-16T17:17:32.224 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.224 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.225 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.225 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.225 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.225 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-16T17:17:32.225 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-16T17:17:32.225 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:32.225 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.225 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.225 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:32.226 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.226 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.226 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.226 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.226 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.226 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.226 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.226 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.226 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-16T17:17:32.227 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-16T17:17:32.227 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.227 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.227 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.227 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-16T17:17:32.227 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.227 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.227 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-16T17:17:32.227 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-16T17:17:32.228 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-16T17:17:32.228 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-16T17:17:32.228 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-16T17:17:32.228 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-16T17:17:32.228 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-16T17:17:32.228 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-16T17:17:32.228 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.228 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.228 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.229 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.229 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.229 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-16T17:17:32.229 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-16T17:17:32.229 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:32.229 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.229 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.229 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:32.229 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.229 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.230 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.230 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.230 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.230 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.230 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.230 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.230 INFO:teuthology.orchestra.run.smithi045.stdout: "storage_class": { 2024-04-16T17:17:32.230 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "storage_class", 2024-04-16T17:17:32.230 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.231 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.231 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.231 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "local", 2024-04-16T17:17:32.231 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.231 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.231 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.231 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-16T17:17:32.231 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.231 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.232 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.232 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:17:32.232 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:17:32.232 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.232 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-16T17:17:32.232 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "selftest", 2024-04-16T17:17:32.232 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-16T17:17:32.232 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-16T17:17:32.232 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-16T17:17:32.232 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-16T17:17:32.233 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-16T17:17:32.233 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.233 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.233 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.233 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:32.233 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.233 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.233 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-16T17:17:32.233 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-16T17:17:32.233 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-16T17:17:32.234 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-16T17:17:32.234 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-16T17:17:32.234 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-16T17:17:32.234 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-16T17:17:32.234 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-16T17:17:32.234 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.234 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.234 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.234 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.235 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.235 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-16T17:17:32.235 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-16T17:17:32.235 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:32.235 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.235 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.235 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:32.235 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.235 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.236 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.236 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.236 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.236 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.236 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.236 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.236 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-16T17:17:32.236 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-16T17:17:32.237 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.237 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.237 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.237 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-16T17:17:32.237 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.237 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.237 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-16T17:17:32.237 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-16T17:17:32.237 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-16T17:17:32.237 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-16T17:17:32.238 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-16T17:17:32.238 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-16T17:17:32.238 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-16T17:17:32.238 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-16T17:17:32.238 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.238 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.238 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.238 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.238 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.239 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-16T17:17:32.239 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-16T17:17:32.239 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:32.239 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.239 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.239 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:32.239 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.239 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.240 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.240 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.240 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.240 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.240 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.240 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.240 INFO:teuthology.orchestra.run.smithi045.stdout: "roption1": { 2024-04-16T17:17:32.240 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "roption1", 2024-04-16T17:17:32.240 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.241 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.241 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.241 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:32.241 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.241 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.241 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.241 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.241 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.241 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.242 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.242 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.242 INFO:teuthology.orchestra.run.smithi045.stdout: "roption2": { 2024-04-16T17:17:32.242 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "roption2", 2024-04-16T17:17:32.242 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.242 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.242 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.242 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "xyz", 2024-04-16T17:17:32.242 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.243 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.243 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.243 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.243 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.243 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.243 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.243 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.243 INFO:teuthology.orchestra.run.smithi045.stdout: "rwoption1": { 2024-04-16T17:17:32.243 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rwoption1", 2024-04-16T17:17:32.244 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.244 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.244 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.244 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:32.244 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.244 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.244 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.244 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.244 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.244 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.245 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.245 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.245 INFO:teuthology.orchestra.run.smithi045.stdout: "rwoption2": { 2024-04-16T17:17:32.245 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rwoption2", 2024-04-16T17:17:32.245 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-16T17:17:32.245 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.245 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.245 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:32.245 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.245 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.246 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.246 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.246 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.246 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.246 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.246 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.246 INFO:teuthology.orchestra.run.smithi045.stdout: "rwoption3": { 2024-04-16T17:17:32.246 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rwoption3", 2024-04-16T17:17:32.246 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "float", 2024-04-16T17:17:32.246 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.247 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.247 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:32.247 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.247 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.247 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.247 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.247 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.247 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.247 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.248 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.248 INFO:teuthology.orchestra.run.smithi045.stdout: "rwoption4": { 2024-04-16T17:17:32.248 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rwoption4", 2024-04-16T17:17:32.248 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.248 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.248 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.248 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:32.248 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.248 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.249 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.249 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.249 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.249 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.249 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.249 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.249 INFO:teuthology.orchestra.run.smithi045.stdout: "rwoption5": { 2024-04-16T17:17:32.249 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rwoption5", 2024-04-16T17:17:32.249 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:32.249 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.250 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.250 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:32.250 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.250 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.250 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.250 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.250 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.250 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.250 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.251 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.251 INFO:teuthology.orchestra.run.smithi045.stdout: "rwoption6": { 2024-04-16T17:17:32.251 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rwoption6", 2024-04-16T17:17:32.251 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:32.251 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.251 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.251 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-16T17:17:32.251 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.251 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.252 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.252 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.252 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.252 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.252 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.252 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.252 INFO:teuthology.orchestra.run.smithi045.stdout: "rwoption7": { 2024-04-16T17:17:32.252 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rwoption7", 2024-04-16T17:17:32.252 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-16T17:17:32.252 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.253 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.253 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:32.253 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "1", 2024-04-16T17:17:32.253 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "42", 2024-04-16T17:17:32.253 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.253 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.253 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.253 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.253 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.253 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.254 INFO:teuthology.orchestra.run.smithi045.stdout: "testkey": { 2024-04-16T17:17:32.254 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "testkey", 2024-04-16T17:17:32.254 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.254 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.254 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.254 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:32.254 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.254 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.254 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.255 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.255 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.255 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.255 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.255 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.255 INFO:teuthology.orchestra.run.smithi045.stdout: "testlkey": { 2024-04-16T17:17:32.255 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "testlkey", 2024-04-16T17:17:32.255 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.256 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.256 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.256 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:32.256 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.256 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.256 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.256 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.256 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.256 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.256 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.257 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.257 INFO:teuthology.orchestra.run.smithi045.stdout: "testnewline": { 2024-04-16T17:17:32.257 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "testnewline", 2024-04-16T17:17:32.257 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.257 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.257 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.257 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:32.257 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.257 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.257 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.258 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.258 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.258 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.258 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.258 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:17:32.258 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:17:32.258 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.258 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-16T17:17:32.258 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "snap_schedule", 2024-04-16T17:17:32.259 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-16T17:17:32.259 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-16T17:17:32.259 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-16T17:17:32.259 INFO:teuthology.orchestra.run.smithi045.stdout: "allow_m_granularity": { 2024-04-16T17:17:32.259 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "allow_m_granularity", 2024-04-16T17:17:32.259 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:32.259 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.259 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.259 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:32.260 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.260 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.260 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.260 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "allow minute scheduled snapshots", 2024-04-16T17:17:32.260 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.260 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.260 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.260 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.260 INFO:teuthology.orchestra.run.smithi045.stdout: "dump_on_update": { 2024-04-16T17:17:32.260 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "dump_on_update", 2024-04-16T17:17:32.261 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:32.261 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.261 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.261 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:32.261 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.261 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.261 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.261 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "dump database to debug log on update", 2024-04-16T17:17:32.262 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.262 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.262 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.262 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.262 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-16T17:17:32.262 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-16T17:17:32.262 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.262 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.262 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.262 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:32.263 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.263 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.263 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-16T17:17:32.263 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-16T17:17:32.263 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-16T17:17:32.263 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-16T17:17:32.263 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-16T17:17:32.263 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-16T17:17:32.263 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-16T17:17:32.264 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-16T17:17:32.264 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.264 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.264 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.264 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.264 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.264 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-16T17:17:32.264 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-16T17:17:32.264 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:32.264 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.265 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.265 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:32.265 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.265 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.265 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.265 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.265 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.265 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.265 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.266 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.266 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-16T17:17:32.266 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-16T17:17:32.266 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.266 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.266 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.266 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-16T17:17:32.266 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.266 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.267 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-16T17:17:32.267 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-16T17:17:32.267 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-16T17:17:32.267 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-16T17:17:32.267 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-16T17:17:32.267 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-16T17:17:32.267 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-16T17:17:32.267 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-16T17:17:32.267 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.268 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.268 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.268 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.268 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.268 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-16T17:17:32.268 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-16T17:17:32.268 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:32.268 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.268 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.269 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:32.269 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.269 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.269 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.269 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.269 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.269 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.269 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.269 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:17:32.270 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:17:32.270 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.270 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-16T17:17:32.270 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "stats", 2024-04-16T17:17:32.270 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-16T17:17:32.270 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-16T17:17:32.270 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-16T17:17:32.270 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-16T17:17:32.270 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-16T17:17:32.271 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.271 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.271 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.271 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:32.271 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.271 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.271 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-16T17:17:32.271 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-16T17:17:32.271 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-16T17:17:32.271 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-16T17:17:32.272 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-16T17:17:32.272 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-16T17:17:32.272 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-16T17:17:32.272 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-16T17:17:32.272 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.272 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.272 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.272 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.272 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.272 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-16T17:17:32.273 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-16T17:17:32.273 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:32.273 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.273 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.273 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:32.273 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.273 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.273 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.273 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.274 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.274 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.274 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.274 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.274 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-16T17:17:32.274 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-16T17:17:32.274 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.274 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.274 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.274 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-16T17:17:32.275 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.275 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.275 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-16T17:17:32.275 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-16T17:17:32.275 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-16T17:17:32.275 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-16T17:17:32.275 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-16T17:17:32.275 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-16T17:17:32.275 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-16T17:17:32.275 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-16T17:17:32.276 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.276 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.276 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.276 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.276 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.276 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-16T17:17:32.276 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-16T17:17:32.276 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:32.277 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.277 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.277 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:32.277 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.277 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.277 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.277 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.277 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.277 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.277 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.278 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:17:32.278 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:17:32.278 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.278 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-16T17:17:32.278 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "status", 2024-04-16T17:17:32.278 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-16T17:17:32.278 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-16T17:17:32.278 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-16T17:17:32.278 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-16T17:17:32.279 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-16T17:17:32.279 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.279 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.279 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.279 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:32.279 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.279 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.279 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-16T17:17:32.279 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-16T17:17:32.280 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-16T17:17:32.280 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-16T17:17:32.280 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-16T17:17:32.280 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-16T17:17:32.280 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-16T17:17:32.280 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-16T17:17:32.280 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.280 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.280 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.280 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.281 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.281 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-16T17:17:32.281 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-16T17:17:32.281 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:32.281 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.281 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.281 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:32.281 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.281 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.282 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.282 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.282 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.282 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.282 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.282 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.282 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-16T17:17:32.282 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-16T17:17:32.282 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.282 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.283 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.283 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-16T17:17:32.283 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.283 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.283 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-16T17:17:32.283 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-16T17:17:32.283 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-16T17:17:32.283 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-16T17:17:32.284 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-16T17:17:32.284 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-16T17:17:32.284 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-16T17:17:32.284 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-16T17:17:32.284 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.284 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.284 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.284 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.284 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.285 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-16T17:17:32.285 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-16T17:17:32.285 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:32.285 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.285 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.285 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:32.285 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.285 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.285 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.286 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.286 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.286 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.286 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.286 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:17:32.286 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:17:32.286 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.286 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-16T17:17:32.286 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "telegraf", 2024-04-16T17:17:32.287 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-16T17:17:32.287 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-16T17:17:32.287 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-16T17:17:32.287 INFO:teuthology.orchestra.run.smithi045.stdout: "address": { 2024-04-16T17:17:32.287 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "address", 2024-04-16T17:17:32.287 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.287 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.287 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.287 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-16T17:17:32.287 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.288 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.288 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.288 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.288 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.288 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.288 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.288 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.288 INFO:teuthology.orchestra.run.smithi045.stdout: "interval": { 2024-04-16T17:17:32.288 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "interval", 2024-04-16T17:17:32.288 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-04-16T17:17:32.289 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.289 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.289 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "15", 2024-04-16T17:17:32.289 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.289 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.289 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.289 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.289 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.289 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.290 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.290 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.290 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-16T17:17:32.290 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-16T17:17:32.290 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.290 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.290 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.290 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:32.290 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.291 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.291 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-16T17:17:32.291 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-16T17:17:32.291 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-16T17:17:32.291 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-16T17:17:32.291 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-16T17:17:32.291 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-16T17:17:32.291 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-16T17:17:32.291 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-16T17:17:32.292 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.292 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.292 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.292 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.292 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.292 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-16T17:17:32.292 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-16T17:17:32.292 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:32.292 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.293 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.293 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:32.293 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.293 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.293 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.293 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.293 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.293 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.293 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.293 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.294 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-16T17:17:32.294 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-16T17:17:32.294 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.294 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.294 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.294 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-16T17:17:32.294 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.294 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.294 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-16T17:17:32.294 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-16T17:17:32.295 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-16T17:17:32.295 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-16T17:17:32.295 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-16T17:17:32.295 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-16T17:17:32.295 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-16T17:17:32.295 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-16T17:17:32.295 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.295 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.295 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.296 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.296 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.296 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-16T17:17:32.296 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-16T17:17:32.296 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:32.296 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.296 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.296 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:32.296 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.296 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.297 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.297 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.297 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.297 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.297 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.297 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:17:32.297 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:17:32.297 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.297 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-16T17:17:32.298 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "telemetry", 2024-04-16T17:17:32.298 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-16T17:17:32.298 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-16T17:17:32.298 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-16T17:17:32.298 INFO:teuthology.orchestra.run.smithi045.stdout: "channel_basic": { 2024-04-16T17:17:32.298 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "channel_basic", 2024-04-16T17:17:32.298 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:32.298 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.298 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.298 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-16T17:17:32.299 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.299 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.299 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.299 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-16T17:17:32.299 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.299 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.299 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.299 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.299 INFO:teuthology.orchestra.run.smithi045.stdout: "channel_crash": { 2024-04-16T17:17:32.299 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "channel_crash", 2024-04-16T17:17:32.300 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:32.300 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.300 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.300 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-16T17:17:32.300 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.300 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.300 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.300 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-16T17:17:32.301 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.301 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.301 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.301 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.301 INFO:teuthology.orchestra.run.smithi045.stdout: "channel_device": { 2024-04-16T17:17:32.301 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "channel_device", 2024-04-16T17:17:32.301 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:32.301 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.301 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.301 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-16T17:17:32.302 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.302 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.302 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.302 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-16T17:17:32.302 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.302 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.302 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.302 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.302 INFO:teuthology.orchestra.run.smithi045.stdout: "channel_ident": { 2024-04-16T17:17:32.303 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "channel_ident", 2024-04-16T17:17:32.303 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:32.303 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.303 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.303 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:32.303 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.303 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.303 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.303 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-16T17:17:32.303 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.304 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.304 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.304 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.304 INFO:teuthology.orchestra.run.smithi045.stdout: "channel_perf": { 2024-04-16T17:17:32.304 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "channel_perf", 2024-04-16T17:17:32.304 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:32.304 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.304 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.304 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:32.305 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.305 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.305 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.305 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-16T17:17:32.305 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.305 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.305 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.305 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.305 INFO:teuthology.orchestra.run.smithi045.stdout: "contact": { 2024-04-16T17:17:32.305 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "contact", 2024-04-16T17:17:32.306 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.306 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.306 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.306 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:32.306 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.306 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.306 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.306 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.306 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.307 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.307 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.307 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.307 INFO:teuthology.orchestra.run.smithi045.stdout: "description": { 2024-04-16T17:17:32.307 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "description", 2024-04-16T17:17:32.307 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.307 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.307 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.307 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:32.307 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.308 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.308 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.308 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.308 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.308 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.308 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.308 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.308 INFO:teuthology.orchestra.run.smithi045.stdout: "device_url": { 2024-04-16T17:17:32.308 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "device_url", 2024-04-16T17:17:32.308 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.309 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.309 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.309 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-16T17:17:32.309 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.309 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.309 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.309 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.309 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.309 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.310 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.310 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.310 INFO:teuthology.orchestra.run.smithi045.stdout: "enabled": { 2024-04-16T17:17:32.310 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "enabled", 2024-04-16T17:17:32.310 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:32.310 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.310 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.310 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:32.310 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.311 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.311 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.311 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.311 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.311 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.311 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.311 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.311 INFO:teuthology.orchestra.run.smithi045.stdout: "interval": { 2024-04-16T17:17:32.311 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "interval", 2024-04-16T17:17:32.312 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-16T17:17:32.312 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.312 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.312 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "24", 2024-04-16T17:17:32.312 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "8", 2024-04-16T17:17:32.312 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.312 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.312 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.312 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.313 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.313 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.313 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.313 INFO:teuthology.orchestra.run.smithi045.stdout: "last_opt_revision": { 2024-04-16T17:17:32.313 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "last_opt_revision", 2024-04-16T17:17:32.313 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-16T17:17:32.313 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.313 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.313 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "1", 2024-04-16T17:17:32.314 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.314 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.314 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.314 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.314 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.314 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.314 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.314 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.315 INFO:teuthology.orchestra.run.smithi045.stdout: "leaderboard": { 2024-04-16T17:17:32.315 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "leaderboard", 2024-04-16T17:17:32.315 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:32.315 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.315 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.315 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:32.315 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.315 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.315 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.316 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.316 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.316 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.316 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.316 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.316 INFO:teuthology.orchestra.run.smithi045.stdout: "leaderboard_description": { 2024-04-16T17:17:32.316 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "leaderboard_description", 2024-04-16T17:17:32.316 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.316 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.316 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.317 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:32.317 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.317 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.317 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.317 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.317 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.317 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.317 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.317 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.318 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-16T17:17:32.318 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-16T17:17:32.318 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.318 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.318 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.318 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:32.318 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.318 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.318 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-16T17:17:32.319 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-16T17:17:32.319 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-16T17:17:32.319 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-16T17:17:32.319 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-16T17:17:32.319 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-16T17:17:32.319 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-16T17:17:32.319 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-16T17:17:32.319 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.319 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.320 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.320 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.320 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.320 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-16T17:17:32.320 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-16T17:17:32.320 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:32.320 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.320 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.320 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:32.320 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.321 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.321 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.321 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.321 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.321 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.321 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.321 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.321 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-16T17:17:32.321 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-16T17:17:32.322 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.322 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.322 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.322 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-16T17:17:32.322 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.322 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.322 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-16T17:17:32.322 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-16T17:17:32.322 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-16T17:17:32.323 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-16T17:17:32.323 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-16T17:17:32.323 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-16T17:17:32.323 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-16T17:17:32.323 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-16T17:17:32.323 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.323 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.323 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.323 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.324 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.324 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-16T17:17:32.324 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-16T17:17:32.324 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:32.324 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.324 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.324 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:32.324 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.324 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.324 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.325 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.325 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.325 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.325 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.325 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.325 INFO:teuthology.orchestra.run.smithi045.stdout: "organization": { 2024-04-16T17:17:32.325 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "organization", 2024-04-16T17:17:32.325 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.325 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.326 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.326 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:32.326 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.326 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.326 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.326 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.326 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.326 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.326 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.327 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.327 INFO:teuthology.orchestra.run.smithi045.stdout: "proxy": { 2024-04-16T17:17:32.327 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "proxy", 2024-04-16T17:17:32.327 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.327 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.327 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.327 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:32.327 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.327 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.328 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.328 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.328 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.328 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.328 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.328 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.328 INFO:teuthology.orchestra.run.smithi045.stdout: "url": { 2024-04-16T17:17:32.328 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "url", 2024-04-16T17:17:32.328 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.329 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.329 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.329 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-16T17:17:32.329 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.329 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.329 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.329 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.329 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.329 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.329 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.330 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:17:32.330 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:17:32.330 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.330 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-16T17:17:32.330 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "test_orchestrator", 2024-04-16T17:17:32.330 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-16T17:17:32.330 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-16T17:17:32.330 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-16T17:17:32.330 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-16T17:17:32.331 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-16T17:17:32.331 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.331 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.331 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.331 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:32.331 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.331 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.331 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-16T17:17:32.331 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-16T17:17:32.331 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-16T17:17:32.332 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-16T17:17:32.332 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-16T17:17:32.332 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-16T17:17:32.332 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-16T17:17:32.332 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-16T17:17:32.332 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.332 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.332 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.332 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.332 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.333 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-16T17:17:32.333 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-16T17:17:32.333 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:32.333 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.333 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.333 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:32.333 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.333 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.333 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.333 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.334 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.334 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.334 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.334 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.334 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-16T17:17:32.334 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-16T17:17:32.334 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.334 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.334 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.335 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-16T17:17:32.335 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.335 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.335 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-16T17:17:32.335 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-16T17:17:32.335 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-16T17:17:32.335 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-16T17:17:32.335 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-16T17:17:32.335 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-16T17:17:32.336 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-16T17:17:32.336 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-16T17:17:32.336 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.336 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.336 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.336 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.336 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.336 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-16T17:17:32.336 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-16T17:17:32.337 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:32.337 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.337 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.337 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:32.337 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.337 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.337 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.337 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.337 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.338 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.338 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.338 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:17:32.338 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:17:32.338 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.338 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-16T17:17:32.338 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "volumes", 2024-04-16T17:17:32.338 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-16T17:17:32.338 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-16T17:17:32.339 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-16T17:17:32.339 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-16T17:17:32.339 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-16T17:17:32.339 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.339 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.339 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.339 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:32.339 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.339 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.340 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-16T17:17:32.340 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-16T17:17:32.340 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-16T17:17:32.340 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-16T17:17:32.340 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-16T17:17:32.340 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-16T17:17:32.340 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-16T17:17:32.340 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-16T17:17:32.340 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.341 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.341 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.341 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.341 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.341 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-16T17:17:32.341 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-16T17:17:32.341 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:32.341 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.341 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.342 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:32.342 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.342 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.342 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.342 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.342 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.342 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.342 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.342 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.343 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-16T17:17:32.343 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-16T17:17:32.343 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.343 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.343 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.343 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-16T17:17:32.343 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.343 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.343 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-16T17:17:32.343 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-16T17:17:32.344 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-16T17:17:32.344 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-16T17:17:32.344 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-16T17:17:32.344 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-16T17:17:32.344 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-16T17:17:32.344 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-16T17:17:32.344 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.344 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.344 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.344 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.345 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.345 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-16T17:17:32.345 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-16T17:17:32.345 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:32.345 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.345 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.345 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:32.345 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.346 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.346 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.346 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.346 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.346 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.346 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.346 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.346 INFO:teuthology.orchestra.run.smithi045.stdout: "max_concurrent_clones": { 2024-04-16T17:17:32.346 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "max_concurrent_clones", 2024-04-16T17:17:32.347 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-16T17:17:32.347 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.347 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.347 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "4", 2024-04-16T17:17:32.347 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.347 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.347 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.347 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-16T17:17:32.347 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.347 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.348 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.348 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.348 INFO:teuthology.orchestra.run.smithi045.stdout: "periodic_async_work": { 2024-04-16T17:17:32.348 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "periodic_async_work", 2024-04-16T17:17:32.348 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:32.348 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.348 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.348 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:32.348 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.349 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.349 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.349 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Periodically check for async work", 2024-04-16T17:17:32.349 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.349 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.349 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.349 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.349 INFO:teuthology.orchestra.run.smithi045.stdout: "snapshot_clone_delay": { 2024-04-16T17:17:32.349 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "snapshot_clone_delay", 2024-04-16T17:17:32.349 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-16T17:17:32.350 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.350 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.350 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0", 2024-04-16T17:17:32.350 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.350 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.350 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.350 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-16T17:17:32.350 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.350 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.351 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.351 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.351 INFO:teuthology.orchestra.run.smithi045.stdout: "snapshot_clone_no_wait": { 2024-04-16T17:17:32.351 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "snapshot_clone_no_wait", 2024-04-16T17:17:32.351 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:32.351 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.351 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.351 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-16T17:17:32.351 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.351 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.352 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.352 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-16T17:17:32.352 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.352 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.352 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.352 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:17:32.352 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:17:32.352 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.352 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-16T17:17:32.353 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "zabbix", 2024-04-16T17:17:32.353 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-16T17:17:32.353 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-16T17:17:32.353 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-16T17:17:32.353 INFO:teuthology.orchestra.run.smithi045.stdout: "discovery_interval": { 2024-04-16T17:17:32.353 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "discovery_interval", 2024-04-16T17:17:32.353 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "uint", 2024-04-16T17:17:32.353 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.353 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.353 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "100", 2024-04-16T17:17:32.354 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.354 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.354 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.354 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.354 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.354 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.354 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.354 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.354 INFO:teuthology.orchestra.run.smithi045.stdout: "identifier": { 2024-04-16T17:17:32.355 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "identifier", 2024-04-16T17:17:32.355 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.355 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.355 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.355 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:32.355 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.355 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.355 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.356 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.356 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.356 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.356 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.356 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.356 INFO:teuthology.orchestra.run.smithi045.stdout: "interval": { 2024-04-16T17:17:32.356 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "interval", 2024-04-16T17:17:32.356 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-04-16T17:17:32.356 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.357 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.357 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "60", 2024-04-16T17:17:32.357 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.357 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.357 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.357 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.357 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.357 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.357 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.358 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.358 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-16T17:17:32.358 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-16T17:17:32.358 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.358 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.358 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.358 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:32.358 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.358 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.358 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-16T17:17:32.359 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-16T17:17:32.359 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-16T17:17:32.359 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-16T17:17:32.359 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-16T17:17:32.359 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-16T17:17:32.359 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-16T17:17:32.359 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-16T17:17:32.359 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.359 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.360 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.360 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.360 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.360 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-16T17:17:32.360 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-16T17:17:32.360 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:32.360 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.360 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.360 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:32.361 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.361 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.361 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.361 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.361 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.361 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.361 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.361 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.361 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-16T17:17:32.362 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-16T17:17:32.362 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.362 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.362 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.362 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-16T17:17:32.362 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.362 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.362 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-16T17:17:32.362 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-16T17:17:32.363 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-16T17:17:32.363 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-16T17:17:32.363 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-16T17:17:32.363 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-16T17:17:32.363 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-16T17:17:32.363 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-16T17:17:32.363 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.363 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.363 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.363 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.363 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.364 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-16T17:17:32.364 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-16T17:17:32.364 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:32.364 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.364 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.364 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:32.364 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.364 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.364 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.365 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.365 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.365 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.365 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.365 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.365 INFO:teuthology.orchestra.run.smithi045.stdout: "zabbix_host": { 2024-04-16T17:17:32.365 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "zabbix_host", 2024-04-16T17:17:32.365 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.365 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.366 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.366 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:32.366 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.366 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.366 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.366 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.366 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.366 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.366 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.366 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.367 INFO:teuthology.orchestra.run.smithi045.stdout: "zabbix_port": { 2024-04-16T17:17:32.367 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "zabbix_port", 2024-04-16T17:17:32.367 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-16T17:17:32.367 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.367 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.367 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "10051", 2024-04-16T17:17:32.367 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.367 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.367 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.368 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.368 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.368 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.368 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.368 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.368 INFO:teuthology.orchestra.run.smithi045.stdout: "zabbix_sender": { 2024-04-16T17:17:32.368 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "zabbix_sender", 2024-04-16T17:17:32.368 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.368 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.369 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.369 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-16T17:17:32.369 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.369 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.369 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.369 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.369 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.369 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.369 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.370 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:17:32.370 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:17:32.370 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:17:32.370 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-04-16T17:17:32.370 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.370 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-16T17:17:32.370 INFO:teuthology.orchestra.run.smithi045.stdout: "gid": 25409, 2024-04-16T17:17:32.370 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "y", 2024-04-16T17:17:32.370 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr_features": 4540701547738038271, 2024-04-16T17:17:32.371 INFO:teuthology.orchestra.run.smithi045.stdout: "available_modules": [ 2024-04-16T17:17:32.371 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-16T17:17:32.371 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "alerts", 2024-04-16T17:17:32.371 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-16T17:17:32.371 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-16T17:17:32.371 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-16T17:17:32.371 INFO:teuthology.orchestra.run.smithi045.stdout: "interval": { 2024-04-16T17:17:32.371 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "interval", 2024-04-16T17:17:32.371 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-04-16T17:17:32.372 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.372 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.372 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "60", 2024-04-16T17:17:32.372 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.372 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.372 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.372 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "How frequently to reexamine health status", 2024-04-16T17:17:32.372 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.372 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.373 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.373 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.373 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-16T17:17:32.373 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-16T17:17:32.373 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.373 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.373 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.373 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:32.374 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.374 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.374 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-16T17:17:32.374 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-16T17:17:32.374 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-16T17:17:32.374 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-16T17:17:32.374 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-16T17:17:32.374 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-16T17:17:32.374 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-16T17:17:32.374 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-16T17:17:32.375 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.375 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.375 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.375 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.375 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.375 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-16T17:17:32.375 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-16T17:17:32.375 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:32.375 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.375 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.376 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:32.376 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.376 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.376 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.376 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.376 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.376 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.376 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.376 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.376 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-16T17:17:32.377 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-16T17:17:32.377 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.377 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.377 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.377 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-16T17:17:32.377 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.377 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.377 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-16T17:17:32.377 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-16T17:17:32.378 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-16T17:17:32.378 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-16T17:17:32.378 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-16T17:17:32.378 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-16T17:17:32.378 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-16T17:17:32.378 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-16T17:17:32.378 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.378 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.379 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.379 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.379 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.379 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-16T17:17:32.379 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-16T17:17:32.379 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:32.379 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.379 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.379 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:32.379 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.380 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.380 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.380 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.380 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.380 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.380 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.380 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.380 INFO:teuthology.orchestra.run.smithi045.stdout: "smtp_destination": { 2024-04-16T17:17:32.380 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "smtp_destination", 2024-04-16T17:17:32.381 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.381 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.381 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.381 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:32.381 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.381 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.381 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.381 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Email address to send alerts to", 2024-04-16T17:17:32.381 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.382 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.382 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.382 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.382 INFO:teuthology.orchestra.run.smithi045.stdout: "smtp_from_name": { 2024-04-16T17:17:32.382 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "smtp_from_name", 2024-04-16T17:17:32.382 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.382 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.382 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.382 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "Ceph", 2024-04-16T17:17:32.383 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.383 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.383 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.383 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Email From: name", 2024-04-16T17:17:32.383 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.383 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.383 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.383 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.383 INFO:teuthology.orchestra.run.smithi045.stdout: "smtp_host": { 2024-04-16T17:17:32.383 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "smtp_host", 2024-04-16T17:17:32.384 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.384 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.384 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.384 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:32.384 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.384 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.384 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.384 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "SMTP server", 2024-04-16T17:17:32.384 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.384 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.385 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.385 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.385 INFO:teuthology.orchestra.run.smithi045.stdout: "smtp_password": { 2024-04-16T17:17:32.385 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "smtp_password", 2024-04-16T17:17:32.385 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.385 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.385 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.385 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:32.385 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.385 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.386 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.386 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Password to authenticate with", 2024-04-16T17:17:32.386 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.386 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.386 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.386 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.386 INFO:teuthology.orchestra.run.smithi045.stdout: "smtp_port": { 2024-04-16T17:17:32.386 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "smtp_port", 2024-04-16T17:17:32.387 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-16T17:17:32.387 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.387 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.387 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "465", 2024-04-16T17:17:32.387 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.387 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.387 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.387 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "SMTP port", 2024-04-16T17:17:32.387 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.387 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.388 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.388 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.388 INFO:teuthology.orchestra.run.smithi045.stdout: "smtp_sender": { 2024-04-16T17:17:32.388 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "smtp_sender", 2024-04-16T17:17:32.388 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.388 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.388 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.388 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:32.388 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.389 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.389 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.389 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "SMTP envelope sender", 2024-04-16T17:17:32.389 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.389 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.389 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.389 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.389 INFO:teuthology.orchestra.run.smithi045.stdout: "smtp_ssl": { 2024-04-16T17:17:32.389 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "smtp_ssl", 2024-04-16T17:17:32.389 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:32.390 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.390 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.390 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-16T17:17:32.390 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.390 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.390 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.390 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-16T17:17:32.390 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.391 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.391 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.391 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.391 INFO:teuthology.orchestra.run.smithi045.stdout: "smtp_user": { 2024-04-16T17:17:32.391 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "smtp_user", 2024-04-16T17:17:32.391 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.391 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.391 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.391 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:32.391 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.392 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.392 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.392 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "User to authenticate as", 2024-04-16T17:17:32.392 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.392 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.392 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.392 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:17:32.392 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:17:32.393 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.393 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-16T17:17:32.393 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "balancer", 2024-04-16T17:17:32.393 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-16T17:17:32.393 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-16T17:17:32.393 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-16T17:17:32.393 INFO:teuthology.orchestra.run.smithi045.stdout: "active": { 2024-04-16T17:17:32.393 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "active", 2024-04-16T17:17:32.393 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:32.393 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.394 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.394 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-16T17:17:32.394 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.394 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.394 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.394 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "automatically balance PGs across cluster", 2024-04-16T17:17:32.394 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.394 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.394 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.394 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.395 INFO:teuthology.orchestra.run.smithi045.stdout: "begin_time": { 2024-04-16T17:17:32.395 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "begin_time", 2024-04-16T17:17:32.395 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.395 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.395 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.395 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0000", 2024-04-16T17:17:32.395 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.395 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.395 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.395 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "beginning time of day to automatically balance", 2024-04-16T17:17:32.396 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-16T17:17:32.396 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.396 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.396 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.396 INFO:teuthology.orchestra.run.smithi045.stdout: "begin_weekday": { 2024-04-16T17:17:32.396 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "begin_weekday", 2024-04-16T17:17:32.396 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "uint", 2024-04-16T17:17:32.396 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.396 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.396 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0", 2024-04-16T17:17:32.397 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "0", 2024-04-16T17:17:32.397 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "6", 2024-04-16T17:17:32.397 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.397 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-16T17:17:32.397 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-16T17:17:32.397 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.397 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.397 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.397 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_compat_max_iterations": { 2024-04-16T17:17:32.397 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "crush_compat_max_iterations", 2024-04-16T17:17:32.398 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "uint", 2024-04-16T17:17:32.398 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.398 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.398 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "25", 2024-04-16T17:17:32.398 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "1", 2024-04-16T17:17:32.398 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "250", 2024-04-16T17:17:32.398 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.398 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-16T17:17:32.398 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.399 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.399 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.399 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.399 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_compat_metrics": { 2024-04-16T17:17:32.399 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "crush_compat_metrics", 2024-04-16T17:17:32.399 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.399 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.399 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.399 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "pgs,objects,bytes", 2024-04-16T17:17:32.400 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.400 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.400 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.400 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-16T17:17:32.400 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-16T17:17:32.400 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.400 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.400 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.400 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_compat_step": { 2024-04-16T17:17:32.401 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "crush_compat_step", 2024-04-16T17:17:32.401 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "float", 2024-04-16T17:17:32.401 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.401 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.401 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0.5", 2024-04-16T17:17:32.401 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "0.001", 2024-04-16T17:17:32.401 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "0.999", 2024-04-16T17:17:32.401 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.401 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "aggressiveness of optimization", 2024-04-16T17:17:32.401 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-16T17:17:32.402 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.402 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.402 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.402 INFO:teuthology.orchestra.run.smithi045.stdout: "end_time": { 2024-04-16T17:17:32.402 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "end_time", 2024-04-16T17:17:32.402 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.402 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.402 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.402 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "2359", 2024-04-16T17:17:32.403 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.403 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.403 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.403 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "ending time of day to automatically balance", 2024-04-16T17:17:32.403 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-16T17:17:32.403 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.403 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.403 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.403 INFO:teuthology.orchestra.run.smithi045.stdout: "end_weekday": { 2024-04-16T17:17:32.403 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "end_weekday", 2024-04-16T17:17:32.404 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "uint", 2024-04-16T17:17:32.404 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.404 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.404 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0", 2024-04-16T17:17:32.404 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "0", 2024-04-16T17:17:32.404 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "6", 2024-04-16T17:17:32.404 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.404 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-16T17:17:32.405 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-16T17:17:32.405 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.405 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.405 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.405 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-16T17:17:32.405 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-16T17:17:32.405 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.405 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.405 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.406 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:32.406 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.406 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.406 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-16T17:17:32.406 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-16T17:17:32.406 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-16T17:17:32.406 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-16T17:17:32.406 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-16T17:17:32.406 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-16T17:17:32.407 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-16T17:17:32.407 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-16T17:17:32.407 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.407 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.407 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.407 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.407 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.407 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-16T17:17:32.407 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-16T17:17:32.408 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:32.408 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.408 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.408 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:32.408 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.408 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.408 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.408 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.408 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.408 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.409 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.409 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.409 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-16T17:17:32.409 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-16T17:17:32.409 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.409 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.409 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.409 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-16T17:17:32.409 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.409 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.410 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-16T17:17:32.410 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-16T17:17:32.410 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-16T17:17:32.410 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-16T17:17:32.410 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-16T17:17:32.410 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-16T17:17:32.410 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-16T17:17:32.410 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-16T17:17:32.410 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.410 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.411 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.411 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.411 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.411 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-16T17:17:32.411 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-16T17:17:32.411 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:32.411 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.411 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.411 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:32.412 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.412 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.412 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.412 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.412 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.412 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.412 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.412 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.412 INFO:teuthology.orchestra.run.smithi045.stdout: "min_score": { 2024-04-16T17:17:32.412 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "min_score", 2024-04-16T17:17:32.413 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "float", 2024-04-16T17:17:32.413 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.413 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.413 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0", 2024-04-16T17:17:32.413 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.413 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.413 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.413 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-16T17:17:32.413 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.414 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.414 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.414 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.414 INFO:teuthology.orchestra.run.smithi045.stdout: "mode": { 2024-04-16T17:17:32.414 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "mode", 2024-04-16T17:17:32.414 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.414 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.414 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.414 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "upmap", 2024-04-16T17:17:32.415 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.415 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.415 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-16T17:17:32.415 INFO:teuthology.orchestra.run.smithi045.stdout: "crush-compat", 2024-04-16T17:17:32.415 INFO:teuthology.orchestra.run.smithi045.stdout: "none", 2024-04-16T17:17:32.415 INFO:teuthology.orchestra.run.smithi045.stdout: "read", 2024-04-16T17:17:32.415 INFO:teuthology.orchestra.run.smithi045.stdout: "upmap", 2024-04-16T17:17:32.415 INFO:teuthology.orchestra.run.smithi045.stdout: "upmap-read" 2024-04-16T17:17:32.416 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-16T17:17:32.416 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Balancer mode", 2024-04-16T17:17:32.416 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.416 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.416 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.416 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.416 INFO:teuthology.orchestra.run.smithi045.stdout: "pool_ids": { 2024-04-16T17:17:32.416 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "pool_ids", 2024-04-16T17:17:32.416 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.416 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.417 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.417 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:32.417 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.417 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.417 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.417 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-16T17:17:32.417 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.417 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.417 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.417 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.418 INFO:teuthology.orchestra.run.smithi045.stdout: "sleep_interval": { 2024-04-16T17:17:32.418 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "sleep_interval", 2024-04-16T17:17:32.418 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-04-16T17:17:32.418 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.418 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.418 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "60", 2024-04-16T17:17:32.418 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.418 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.419 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.419 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-16T17:17:32.419 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.419 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.419 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.419 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.419 INFO:teuthology.orchestra.run.smithi045.stdout: "upmap_max_deviation": { 2024-04-16T17:17:32.419 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "upmap_max_deviation", 2024-04-16T17:17:32.419 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-16T17:17:32.419 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.420 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.420 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "5", 2024-04-16T17:17:32.420 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "1", 2024-04-16T17:17:32.420 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.420 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.420 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-16T17:17:32.420 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-16T17:17:32.420 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.420 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.420 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.421 INFO:teuthology.orchestra.run.smithi045.stdout: "upmap_max_optimizations": { 2024-04-16T17:17:32.421 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "upmap_max_optimizations", 2024-04-16T17:17:32.421 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "uint", 2024-04-16T17:17:32.421 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.421 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.421 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "10", 2024-04-16T17:17:32.421 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.421 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.421 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.422 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-16T17:17:32.422 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.422 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.422 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.422 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:17:32.422 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:17:32.422 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.422 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-16T17:17:32.422 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "cephadm", 2024-04-16T17:17:32.423 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-16T17:17:32.423 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-16T17:17:32.423 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-16T17:17:32.423 INFO:teuthology.orchestra.run.smithi045.stdout: "agent_down_multiplier": { 2024-04-16T17:17:32.423 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "agent_down_multiplier", 2024-04-16T17:17:32.423 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "float", 2024-04-16T17:17:32.423 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.423 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.423 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "3.0", 2024-04-16T17:17:32.424 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.424 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.424 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.424 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-16T17:17:32.424 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.424 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.424 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.424 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.424 INFO:teuthology.orchestra.run.smithi045.stdout: "agent_refresh_rate": { 2024-04-16T17:17:32.424 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "agent_refresh_rate", 2024-04-16T17:17:32.425 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-04-16T17:17:32.425 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.425 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.425 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "20", 2024-04-16T17:17:32.425 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.425 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.425 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.425 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-16T17:17:32.425 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.426 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.426 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.426 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.426 INFO:teuthology.orchestra.run.smithi045.stdout: "agent_starting_port": { 2024-04-16T17:17:32.426 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "agent_starting_port", 2024-04-16T17:17:32.426 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-16T17:17:32.426 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.426 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.427 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "4721", 2024-04-16T17:17:32.427 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.427 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.427 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.427 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-16T17:17:32.427 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.427 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.427 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.427 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.428 INFO:teuthology.orchestra.run.smithi045.stdout: "allow_ptrace": { 2024-04-16T17:17:32.428 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "allow_ptrace", 2024-04-16T17:17:32.428 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:32.428 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.428 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.428 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:32.428 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.428 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.428 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.429 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-16T17:17:32.429 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-16T17:17:32.429 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.429 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.429 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.429 INFO:teuthology.orchestra.run.smithi045.stdout: "autotune_interval": { 2024-04-16T17:17:32.429 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "autotune_interval", 2024-04-16T17:17:32.429 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-04-16T17:17:32.429 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.430 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.430 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "600", 2024-04-16T17:17:32.430 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.430 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.430 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.430 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-16T17:17:32.430 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.430 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.430 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.431 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.431 INFO:teuthology.orchestra.run.smithi045.stdout: "autotune_memory_target_ratio": { 2024-04-16T17:17:32.431 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "autotune_memory_target_ratio", 2024-04-16T17:17:32.431 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "float", 2024-04-16T17:17:32.431 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.431 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.431 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0.7", 2024-04-16T17:17:32.431 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.431 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.431 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.432 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-16T17:17:32.432 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.432 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.432 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.432 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.432 INFO:teuthology.orchestra.run.smithi045.stdout: "cephadm_log_destination": { 2024-04-16T17:17:32.432 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "cephadm_log_destination", 2024-04-16T17:17:32.432 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.432 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.432 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.433 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:32.433 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.433 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.433 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-16T17:17:32.433 INFO:teuthology.orchestra.run.smithi045.stdout: "file", 2024-04-16T17:17:32.433 INFO:teuthology.orchestra.run.smithi045.stdout: "file,syslog", 2024-04-16T17:17:32.433 INFO:teuthology.orchestra.run.smithi045.stdout: "syslog" 2024-04-16T17:17:32.433 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-16T17:17:32.433 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-16T17:17:32.434 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.434 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.434 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.434 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.434 INFO:teuthology.orchestra.run.smithi045.stdout: "cgroups_split": { 2024-04-16T17:17:32.434 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "cgroups_split", 2024-04-16T17:17:32.434 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:32.434 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.434 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.434 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-16T17:17:32.435 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.435 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.435 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.435 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-16T17:17:32.435 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.435 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.435 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.435 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.435 INFO:teuthology.orchestra.run.smithi045.stdout: "config_checks_enabled": { 2024-04-16T17:17:32.436 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "config_checks_enabled", 2024-04-16T17:17:32.436 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:32.436 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.436 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.436 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:32.436 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.436 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.436 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.436 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-16T17:17:32.437 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.437 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.437 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.437 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.437 INFO:teuthology.orchestra.run.smithi045.stdout: "config_dashboard": { 2024-04-16T17:17:32.437 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "config_dashboard", 2024-04-16T17:17:32.437 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:32.437 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.437 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.438 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-16T17:17:32.438 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.438 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.438 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.438 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-16T17:17:32.438 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.438 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.438 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.438 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.439 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_alertmanager": { 2024-04-16T17:17:32.439 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_alertmanager", 2024-04-16T17:17:32.439 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.439 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.439 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.439 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-16T17:17:32.439 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.439 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.439 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.440 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Prometheus container image", 2024-04-16T17:17:32.440 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.440 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.440 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.440 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.440 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_base": { 2024-04-16T17:17:32.440 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_base", 2024-04-16T17:17:32.440 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.440 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.441 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.441 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-16T17:17:32.441 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.441 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.441 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.441 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Container image name, without the tag", 2024-04-16T17:17:32.441 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.441 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.441 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.442 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.442 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_elasticsearch": { 2024-04-16T17:17:32.442 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_elasticsearch", 2024-04-16T17:17:32.442 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.442 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.442 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.442 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-16T17:17:32.442 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.442 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.442 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.443 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "elasticsearch container image", 2024-04-16T17:17:32.443 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.443 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.443 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.443 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.443 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_grafana": { 2024-04-16T17:17:32.443 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_grafana", 2024-04-16T17:17:32.443 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.443 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.444 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.444 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-16T17:17:32.444 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.444 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.444 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.444 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Prometheus container image", 2024-04-16T17:17:32.444 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.444 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.444 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.444 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.445 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_haproxy": { 2024-04-16T17:17:32.445 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_haproxy", 2024-04-16T17:17:32.445 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.445 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.445 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.445 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-16T17:17:32.445 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.445 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.445 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.445 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "HAproxy container image", 2024-04-16T17:17:32.446 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.446 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.446 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.446 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.446 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_jaeger_agent": { 2024-04-16T17:17:32.446 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_jaeger_agent", 2024-04-16T17:17:32.446 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.446 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.446 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.446 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-16T17:17:32.447 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.447 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.447 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.447 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Jaeger agent container image", 2024-04-16T17:17:32.447 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.447 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.447 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.447 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.447 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_jaeger_collector": { 2024-04-16T17:17:32.448 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_jaeger_collector", 2024-04-16T17:17:32.448 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.448 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.448 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.448 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-16T17:17:32.448 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.448 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.448 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.448 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Jaeger collector container image", 2024-04-16T17:17:32.449 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.449 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.449 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.449 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.449 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_jaeger_query": { 2024-04-16T17:17:32.449 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_jaeger_query", 2024-04-16T17:17:32.449 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.449 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.449 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.449 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-16T17:17:32.450 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.450 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.450 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.450 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Jaeger query container image", 2024-04-16T17:17:32.450 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.450 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.450 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.450 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.450 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_keepalived": { 2024-04-16T17:17:32.451 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_keepalived", 2024-04-16T17:17:32.451 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.451 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.451 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.451 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-16T17:17:32.451 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.451 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.451 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.451 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Keepalived container image", 2024-04-16T17:17:32.452 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.452 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.452 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.452 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.452 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_loki": { 2024-04-16T17:17:32.452 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_loki", 2024-04-16T17:17:32.452 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.452 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.452 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.452 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-16T17:17:32.453 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.453 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.453 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.453 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Loki container image", 2024-04-16T17:17:32.453 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.453 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.453 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.453 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.453 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_node_exporter": { 2024-04-16T17:17:32.453 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_node_exporter", 2024-04-16T17:17:32.454 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.454 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.454 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.454 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-16T17:17:32.454 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.454 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.454 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.454 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Prometheus container image", 2024-04-16T17:17:32.454 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.455 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.455 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.455 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.455 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_nvmeof": { 2024-04-16T17:17:32.455 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_nvmeof", 2024-04-16T17:17:32.455 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.455 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.455 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.455 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-04-16T17:17:32.456 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.456 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.456 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.456 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Nvme-of container image", 2024-04-16T17:17:32.456 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.456 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.456 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.456 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.456 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_prometheus": { 2024-04-16T17:17:32.457 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_prometheus", 2024-04-16T17:17:32.457 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.457 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.457 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.457 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-16T17:17:32.457 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.457 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.457 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.457 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Prometheus container image", 2024-04-16T17:17:32.457 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.458 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.458 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.458 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.458 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_promtail": { 2024-04-16T17:17:32.458 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_promtail", 2024-04-16T17:17:32.458 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.458 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.458 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.458 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-16T17:17:32.458 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.459 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.459 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.459 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Promtail container image", 2024-04-16T17:17:32.459 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.459 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.459 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.459 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.459 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_samba": { 2024-04-16T17:17:32.459 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_samba", 2024-04-16T17:17:32.459 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.460 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.460 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.460 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-16T17:17:32.460 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.460 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.460 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.460 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Samba/SMB container image", 2024-04-16T17:17:32.460 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.460 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.461 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.461 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.461 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_snmp_gateway": { 2024-04-16T17:17:32.461 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_snmp_gateway", 2024-04-16T17:17:32.461 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.461 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.461 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.461 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-16T17:17:32.461 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.461 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.462 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.462 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "SNMP Gateway container image", 2024-04-16T17:17:32.462 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.462 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.462 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.462 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.462 INFO:teuthology.orchestra.run.smithi045.stdout: "container_init": { 2024-04-16T17:17:32.462 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_init", 2024-04-16T17:17:32.462 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:32.463 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.463 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.463 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-16T17:17:32.463 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.463 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.463 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.463 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Run podman/docker with `--init`", 2024-04-16T17:17:32.463 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.463 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.463 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.464 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.464 INFO:teuthology.orchestra.run.smithi045.stdout: "daemon_cache_timeout": { 2024-04-16T17:17:32.464 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "daemon_cache_timeout", 2024-04-16T17:17:32.464 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-04-16T17:17:32.464 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.464 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.464 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "600", 2024-04-16T17:17:32.464 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.464 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.464 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.465 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-16T17:17:32.465 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.465 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.465 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.465 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.465 INFO:teuthology.orchestra.run.smithi045.stdout: "default_cephadm_command_timeout": { 2024-04-16T17:17:32.465 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "default_cephadm_command_timeout", 2024-04-16T17:17:32.465 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-16T17:17:32.465 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.466 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.466 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "900", 2024-04-16T17:17:32.466 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.466 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.466 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.466 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-16T17:17:32.466 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.466 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.466 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.466 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.467 INFO:teuthology.orchestra.run.smithi045.stdout: "default_registry": { 2024-04-16T17:17:32.467 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "default_registry", 2024-04-16T17:17:32.467 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.467 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.467 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.467 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "docker.io", 2024-04-16T17:17:32.467 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.467 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.467 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.467 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-16T17:17:32.468 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.468 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.468 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.468 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.468 INFO:teuthology.orchestra.run.smithi045.stdout: "device_cache_timeout": { 2024-04-16T17:17:32.468 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "device_cache_timeout", 2024-04-16T17:17:32.468 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-04-16T17:17:32.468 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.468 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.469 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "1800", 2024-04-16T17:17:32.469 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.469 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.469 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.469 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "seconds to cache device inventory", 2024-04-16T17:17:32.469 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.469 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.469 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.469 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.470 INFO:teuthology.orchestra.run.smithi045.stdout: "device_enhanced_scan": { 2024-04-16T17:17:32.470 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "device_enhanced_scan", 2024-04-16T17:17:32.470 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:32.470 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.470 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.470 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:32.470 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.470 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.470 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.471 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-16T17:17:32.471 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.471 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.471 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.471 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.471 INFO:teuthology.orchestra.run.smithi045.stdout: "facts_cache_timeout": { 2024-04-16T17:17:32.471 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "facts_cache_timeout", 2024-04-16T17:17:32.471 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-04-16T17:17:32.471 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.471 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.472 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "60", 2024-04-16T17:17:32.472 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.472 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.472 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.472 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "seconds to cache host facts data", 2024-04-16T17:17:32.472 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.472 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.472 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.472 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.472 INFO:teuthology.orchestra.run.smithi045.stdout: "grafana_dashboards_path": { 2024-04-16T17:17:32.473 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "grafana_dashboards_path", 2024-04-16T17:17:32.473 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.473 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.473 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.473 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-16T17:17:32.473 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.473 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.473 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.473 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-16T17:17:32.474 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.474 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.474 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.474 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.474 INFO:teuthology.orchestra.run.smithi045.stdout: "host_check_interval": { 2024-04-16T17:17:32.474 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "host_check_interval", 2024-04-16T17:17:32.474 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-04-16T17:17:32.474 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.474 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.475 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "600", 2024-04-16T17:17:32.475 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.475 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.475 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.475 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "how frequently to perform a host check", 2024-04-16T17:17:32.475 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.475 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.475 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.475 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.476 INFO:teuthology.orchestra.run.smithi045.stdout: "hw_monitoring": { 2024-04-16T17:17:32.476 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "hw_monitoring", 2024-04-16T17:17:32.476 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:32.476 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.476 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.476 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:32.476 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.476 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.476 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.476 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-16T17:17:32.477 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.477 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.477 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.477 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.477 INFO:teuthology.orchestra.run.smithi045.stdout: "inventory_list_all": { 2024-04-16T17:17:32.477 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "inventory_list_all", 2024-04-16T17:17:32.477 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:32.477 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.478 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.478 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:32.478 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.478 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.478 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.478 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-16T17:17:32.478 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.478 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.478 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.479 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.479 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-16T17:17:32.479 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-16T17:17:32.479 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.479 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.479 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.479 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:32.479 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.479 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.479 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-16T17:17:32.480 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-16T17:17:32.480 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-16T17:17:32.480 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-16T17:17:32.480 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-16T17:17:32.480 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-16T17:17:32.480 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-16T17:17:32.480 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-16T17:17:32.480 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.480 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.481 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.481 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.481 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.481 INFO:teuthology.orchestra.run.smithi045.stdout: "log_refresh_metadata": { 2024-04-16T17:17:32.481 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_refresh_metadata", 2024-04-16T17:17:32.481 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:32.481 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.481 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.481 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:32.482 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.482 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.482 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.482 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-16T17:17:32.482 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.482 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.482 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.482 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.482 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-16T17:17:32.482 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-16T17:17:32.483 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:32.483 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.483 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.483 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-16T17:17:32.483 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.483 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.483 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.483 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-16T17:17:32.483 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.484 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.484 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.484 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.484 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-16T17:17:32.484 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-16T17:17:32.484 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.484 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.484 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.484 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-16T17:17:32.484 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.485 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.485 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-16T17:17:32.485 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-16T17:17:32.485 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-16T17:17:32.485 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-16T17:17:32.485 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-16T17:17:32.485 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-16T17:17:32.485 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-16T17:17:32.485 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-16T17:17:32.486 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.486 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.486 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.486 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.486 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.486 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-16T17:17:32.486 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-16T17:17:32.486 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:32.486 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.487 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.487 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:32.487 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.487 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.487 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.487 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.487 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.487 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.487 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.488 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.488 INFO:teuthology.orchestra.run.smithi045.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-16T17:17:32.488 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-16T17:17:32.488 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:32.488 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.488 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.488 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:32.488 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.488 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.488 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.489 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-16T17:17:32.489 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.489 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.489 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.489 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.489 INFO:teuthology.orchestra.run.smithi045.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-16T17:17:32.489 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-16T17:17:32.489 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.489 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.490 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.490 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "*", 2024-04-16T17:17:32.490 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.490 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.490 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.490 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-16T17:17:32.490 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.490 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.490 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.490 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.491 INFO:teuthology.orchestra.run.smithi045.stdout: "max_count_per_host": { 2024-04-16T17:17:32.491 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "max_count_per_host", 2024-04-16T17:17:32.491 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-16T17:17:32.491 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.491 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.491 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "10", 2024-04-16T17:17:32.491 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.491 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.491 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.492 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "max number of daemons per service per host", 2024-04-16T17:17:32.492 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.492 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.492 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.492 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.492 INFO:teuthology.orchestra.run.smithi045.stdout: "max_osd_draining_count": { 2024-04-16T17:17:32.492 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "max_osd_draining_count", 2024-04-16T17:17:32.492 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-16T17:17:32.492 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.493 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.493 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "10", 2024-04-16T17:17:32.493 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.493 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.493 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.493 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-16T17:17:32.493 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.493 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.493 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.494 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.494 INFO:teuthology.orchestra.run.smithi045.stdout: "migration_current": { 2024-04-16T17:17:32.494 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "migration_current", 2024-04-16T17:17:32.494 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-16T17:17:32.494 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.494 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.494 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:32.494 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.494 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.494 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.495 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "internal - do not modify", 2024-04-16T17:17:32.495 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.495 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.495 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.495 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.495 INFO:teuthology.orchestra.run.smithi045.stdout: "mode": { 2024-04-16T17:17:32.495 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "mode", 2024-04-16T17:17:32.495 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.495 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.496 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.496 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "root", 2024-04-16T17:17:32.496 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.496 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.496 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-16T17:17:32.496 INFO:teuthology.orchestra.run.smithi045.stdout: "cephadm-package", 2024-04-16T17:17:32.496 INFO:teuthology.orchestra.run.smithi045.stdout: "root" 2024-04-16T17:17:32.496 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-16T17:17:32.496 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "mode for remote execution of cephadm", 2024-04-16T17:17:32.497 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.497 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.497 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.497 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.497 INFO:teuthology.orchestra.run.smithi045.stdout: "oob_default_addr": { 2024-04-16T17:17:32.497 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "oob_default_addr", 2024-04-16T17:17:32.497 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.497 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.497 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.497 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "169.254.1.1", 2024-04-16T17:17:32.498 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.498 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.498 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.498 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-16T17:17:32.498 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.498 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.498 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.498 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.498 INFO:teuthology.orchestra.run.smithi045.stdout: "prometheus_alerts_path": { 2024-04-16T17:17:32.498 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "prometheus_alerts_path", 2024-04-16T17:17:32.499 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.499 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.499 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.499 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-16T17:17:32.499 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.499 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.499 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.499 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-16T17:17:32.499 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.499 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.500 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.500 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.500 INFO:teuthology.orchestra.run.smithi045.stdout: "registry_insecure": { 2024-04-16T17:17:32.500 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "registry_insecure", 2024-04-16T17:17:32.500 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:32.500 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.500 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.500 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:32.500 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.501 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.501 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.501 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-16T17:17:32.501 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.501 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.501 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.501 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.501 INFO:teuthology.orchestra.run.smithi045.stdout: "registry_password": { 2024-04-16T17:17:32.501 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "registry_password", 2024-04-16T17:17:32.502 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.502 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.502 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.502 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:32.502 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.502 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.502 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.502 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-16T17:17:32.502 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.503 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.503 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.503 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.503 INFO:teuthology.orchestra.run.smithi045.stdout: "registry_url": { 2024-04-16T17:17:32.503 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "registry_url", 2024-04-16T17:17:32.503 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.503 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.503 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.503 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:32.504 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.504 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.504 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.504 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-16T17:17:32.504 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.504 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.504 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.504 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.505 INFO:teuthology.orchestra.run.smithi045.stdout: "registry_username": { 2024-04-16T17:17:32.505 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "registry_username", 2024-04-16T17:17:32.505 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.505 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.505 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.505 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:32.505 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.505 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.505 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.506 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-16T17:17:32.506 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.506 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.506 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.506 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.506 INFO:teuthology.orchestra.run.smithi045.stdout: "secure_monitoring_stack": { 2024-04-16T17:17:32.506 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "secure_monitoring_stack", 2024-04-16T17:17:32.506 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:32.506 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.507 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.507 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:32.507 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.507 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.507 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.507 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-16T17:17:32.507 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.507 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.507 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.508 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.508 INFO:teuthology.orchestra.run.smithi045.stdout: "service_discovery_port": { 2024-04-16T17:17:32.508 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "service_discovery_port", 2024-04-16T17:17:32.508 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-16T17:17:32.508 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.508 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.508 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "8765", 2024-04-16T17:17:32.508 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.508 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.509 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.509 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "cephadm service discovery port", 2024-04-16T17:17:32.509 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.509 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.509 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.509 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.509 INFO:teuthology.orchestra.run.smithi045.stdout: "ssh_config_file": { 2024-04-16T17:17:32.509 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ssh_config_file", 2024-04-16T17:17:32.509 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.510 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.510 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.510 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:32.510 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.510 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.510 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.510 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-16T17:17:32.510 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.511 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.511 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.511 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.511 INFO:teuthology.orchestra.run.smithi045.stdout: "use_agent": { 2024-04-16T17:17:32.511 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "use_agent", 2024-04-16T17:17:32.511 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:32.511 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.511 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.511 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:32.512 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.512 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.512 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.512 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-16T17:17:32.512 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.512 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.512 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.512 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.513 INFO:teuthology.orchestra.run.smithi045.stdout: "use_repo_digest": { 2024-04-16T17:17:32.513 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "use_repo_digest", 2024-04-16T17:17:32.513 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:32.513 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.513 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.513 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-16T17:17:32.513 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.513 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.514 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.514 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-16T17:17:32.514 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.514 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.514 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.514 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.514 INFO:teuthology.orchestra.run.smithi045.stdout: "warn_on_failed_host_check": { 2024-04-16T17:17:32.514 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "warn_on_failed_host_check", 2024-04-16T17:17:32.514 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:32.515 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.515 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.515 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-16T17:17:32.515 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.515 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.515 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.515 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "raise a health warning if the host check fails", 2024-04-16T17:17:32.515 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.515 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.516 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.516 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.516 INFO:teuthology.orchestra.run.smithi045.stdout: "warn_on_stray_daemons": { 2024-04-16T17:17:32.516 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "warn_on_stray_daemons", 2024-04-16T17:17:32.516 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:32.516 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.516 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.516 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-16T17:17:32.516 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.517 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.517 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.517 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-16T17:17:32.517 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.517 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.517 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.517 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.517 INFO:teuthology.orchestra.run.smithi045.stdout: "warn_on_stray_hosts": { 2024-04-16T17:17:32.518 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "warn_on_stray_hosts", 2024-04-16T17:17:32.518 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:32.518 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.518 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.518 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-16T17:17:32.518 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.518 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.518 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.519 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-16T17:17:32.519 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.519 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.519 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.519 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:17:32.519 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:17:32.519 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.519 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-16T17:17:32.519 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "crash", 2024-04-16T17:17:32.520 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-16T17:17:32.520 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-16T17:17:32.520 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-16T17:17:32.520 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-16T17:17:32.520 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-16T17:17:32.520 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.520 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.520 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.520 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:32.521 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.521 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.521 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-16T17:17:32.521 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-16T17:17:32.521 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-16T17:17:32.521 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-16T17:17:32.521 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-16T17:17:32.521 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-16T17:17:32.522 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-16T17:17:32.522 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-16T17:17:32.522 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.522 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.522 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.522 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.522 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.522 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-16T17:17:32.522 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-16T17:17:32.523 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:32.523 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.523 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.523 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:32.523 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.523 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.523 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.523 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.523 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.524 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.524 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.524 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.524 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-16T17:17:32.524 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-16T17:17:32.524 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.524 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.524 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.524 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-16T17:17:32.525 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.525 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.525 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-16T17:17:32.525 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-16T17:17:32.525 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-16T17:17:32.525 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-16T17:17:32.525 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-16T17:17:32.525 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-16T17:17:32.525 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-16T17:17:32.526 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-16T17:17:32.526 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.526 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.526 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.526 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.526 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.526 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-16T17:17:32.526 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-16T17:17:32.526 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:32.527 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.527 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.527 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:32.527 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.527 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.527 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.527 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.527 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.528 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.528 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.528 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.528 INFO:teuthology.orchestra.run.smithi045.stdout: "retain_interval": { 2024-04-16T17:17:32.528 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "retain_interval", 2024-04-16T17:17:32.528 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-04-16T17:17:32.528 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.528 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.528 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "31536000", 2024-04-16T17:17:32.528 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.529 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.529 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.529 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-16T17:17:32.529 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.529 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.529 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.529 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.529 INFO:teuthology.orchestra.run.smithi045.stdout: "warn_recent_interval": { 2024-04-16T17:17:32.529 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "warn_recent_interval", 2024-04-16T17:17:32.530 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-04-16T17:17:32.530 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.530 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.530 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "1209600", 2024-04-16T17:17:32.530 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.530 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.530 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.530 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-16T17:17:32.530 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.531 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.531 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.531 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:17:32.531 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:17:32.531 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.531 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-16T17:17:32.531 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "dashboard", 2024-04-16T17:17:32.531 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-16T17:17:32.531 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-16T17:17:32.532 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-16T17:17:32.532 INFO:teuthology.orchestra.run.smithi045.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-16T17:17:32.532 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-16T17:17:32.532 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-16T17:17:32.532 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.532 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.532 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "10", 2024-04-16T17:17:32.532 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.532 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.533 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.533 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.533 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.533 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.533 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.533 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.533 INFO:teuthology.orchestra.run.smithi045.stdout: "ALERTMANAGER_API_HOST": { 2024-04-16T17:17:32.533 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-16T17:17:32.533 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.534 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.534 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.534 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:32.534 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.534 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.534 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.534 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.534 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.534 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.535 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.535 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.535 INFO:teuthology.orchestra.run.smithi045.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-16T17:17:32.535 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-16T17:17:32.535 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:32.535 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.535 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.535 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-16T17:17:32.535 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.535 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.536 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.536 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.536 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.536 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.536 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.536 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.536 INFO:teuthology.orchestra.run.smithi045.stdout: "AUDIT_API_ENABLED": { 2024-04-16T17:17:32.536 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "AUDIT_API_ENABLED", 2024-04-16T17:17:32.536 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:32.537 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.537 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.537 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:32.537 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.537 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.537 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.537 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.537 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.537 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.538 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.538 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.538 INFO:teuthology.orchestra.run.smithi045.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-16T17:17:32.538 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-16T17:17:32.538 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:32.538 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.538 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.538 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-16T17:17:32.538 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.539 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.539 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.539 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.539 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.539 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.539 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.539 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.539 INFO:teuthology.orchestra.run.smithi045.stdout: "ENABLE_BROWSABLE_API": { 2024-04-16T17:17:32.539 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-16T17:17:32.539 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:32.540 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.540 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.540 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-16T17:17:32.540 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.540 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.540 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.540 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.540 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.541 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.541 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.541 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.541 INFO:teuthology.orchestra.run.smithi045.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-16T17:17:32.541 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-16T17:17:32.541 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:32.541 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.541 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.541 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-16T17:17:32.541 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.542 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.542 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.542 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.542 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.542 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.542 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.542 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.542 INFO:teuthology.orchestra.run.smithi045.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-16T17:17:32.543 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-16T17:17:32.543 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:32.543 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.543 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.543 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-16T17:17:32.543 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.543 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.543 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.543 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.543 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.544 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.544 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.544 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.544 INFO:teuthology.orchestra.run.smithi045.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-16T17:17:32.544 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-16T17:17:32.544 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:32.544 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.544 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.544 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-16T17:17:32.544 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.545 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.545 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.545 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.545 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.545 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.545 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.545 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.545 INFO:teuthology.orchestra.run.smithi045.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-16T17:17:32.545 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-16T17:17:32.546 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:32.546 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.546 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.546 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-16T17:17:32.546 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.546 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.546 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.546 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.546 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.546 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.547 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.547 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.547 INFO:teuthology.orchestra.run.smithi045.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-16T17:17:32.547 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-16T17:17:32.547 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:32.547 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.547 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.547 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-16T17:17:32.547 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.548 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.548 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.548 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.548 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.548 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.548 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.548 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.548 INFO:teuthology.orchestra.run.smithi045.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-16T17:17:32.548 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-16T17:17:32.548 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:32.549 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.549 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.549 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-16T17:17:32.549 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.549 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.549 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.549 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.549 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.549 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.549 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.550 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.550 INFO:teuthology.orchestra.run.smithi045.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-16T17:17:32.550 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-16T17:17:32.550 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:32.550 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.550 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.550 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-16T17:17:32.550 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.551 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.551 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.551 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.551 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.551 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.551 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.551 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.551 INFO:teuthology.orchestra.run.smithi045.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-16T17:17:32.551 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-16T17:17:32.551 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.552 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.552 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.552 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:32.552 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.552 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.552 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.552 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.552 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.552 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.552 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.553 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.553 INFO:teuthology.orchestra.run.smithi045.stdout: "GRAFANA_API_PASSWORD": { 2024-04-16T17:17:32.553 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-16T17:17:32.553 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.553 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.553 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.553 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "admin", 2024-04-16T17:17:32.553 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.553 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.553 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.554 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.554 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.554 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.554 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.554 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.554 INFO:teuthology.orchestra.run.smithi045.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-16T17:17:32.554 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-16T17:17:32.554 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:32.555 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.555 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.555 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-16T17:17:32.555 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.555 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.555 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.555 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.555 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.555 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.555 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.556 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.556 INFO:teuthology.orchestra.run.smithi045.stdout: "GRAFANA_API_URL": { 2024-04-16T17:17:32.556 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "GRAFANA_API_URL", 2024-04-16T17:17:32.556 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.556 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.556 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.556 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:32.556 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.556 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.557 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.557 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.557 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.557 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.557 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.557 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.557 INFO:teuthology.orchestra.run.smithi045.stdout: "GRAFANA_API_USERNAME": { 2024-04-16T17:17:32.557 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-16T17:17:32.557 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.557 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.558 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.558 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "admin", 2024-04-16T17:17:32.558 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.558 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.558 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.558 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.558 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.558 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.558 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.559 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.559 INFO:teuthology.orchestra.run.smithi045.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-16T17:17:32.559 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-16T17:17:32.559 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.559 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.559 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.559 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:32.559 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.559 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.560 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.560 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.560 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.560 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.560 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.560 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.560 INFO:teuthology.orchestra.run.smithi045.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-16T17:17:32.560 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-16T17:17:32.560 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:32.561 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.561 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.561 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:32.561 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.561 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.561 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.561 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.561 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.561 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.562 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.562 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.562 INFO:teuthology.orchestra.run.smithi045.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-16T17:17:32.562 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-16T17:17:32.562 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:32.562 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.562 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.562 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-16T17:17:32.562 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.563 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.563 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.563 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.563 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.563 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.563 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.563 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.563 INFO:teuthology.orchestra.run.smithi045.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-16T17:17:32.563 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-16T17:17:32.563 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.564 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.564 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.564 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:32.564 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.564 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.564 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.564 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.564 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.564 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.564 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.565 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.565 INFO:teuthology.orchestra.run.smithi045.stdout: "MANAGED_BY_CLUSTERS": { 2024-04-16T17:17:32.565 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-04-16T17:17:32.565 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.565 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.565 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.565 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "[]", 2024-04-16T17:17:32.565 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.565 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.566 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.566 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.566 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.566 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.566 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.566 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.566 INFO:teuthology.orchestra.run.smithi045.stdout: "MULTICLUSTER_CONFIG": { 2024-04-16T17:17:32.566 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "MULTICLUSTER_CONFIG", 2024-04-16T17:17:32.566 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.567 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.567 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.567 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "{}", 2024-04-16T17:17:32.567 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.567 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.567 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.567 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.567 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.567 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.568 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.568 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.568 INFO:teuthology.orchestra.run.smithi045.stdout: "PROMETHEUS_API_HOST": { 2024-04-16T17:17:32.568 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-16T17:17:32.568 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.568 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.568 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.568 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:32.568 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.569 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.569 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.569 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.569 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.569 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.569 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.569 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.569 INFO:teuthology.orchestra.run.smithi045.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-16T17:17:32.569 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-16T17:17:32.570 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:32.570 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.570 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.570 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-16T17:17:32.570 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.570 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.570 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.570 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.570 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.571 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.571 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.571 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.571 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-16T17:17:32.571 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-16T17:17:32.571 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:32.571 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.571 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.571 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:32.572 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.572 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.572 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.572 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.572 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.572 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.572 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.572 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.572 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-16T17:17:32.572 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-16T17:17:32.573 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:32.573 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.573 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.573 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:32.573 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.573 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.573 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.573 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.573 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.574 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.574 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.574 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.574 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-16T17:17:32.574 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-16T17:17:32.574 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:32.574 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.574 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.574 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-16T17:17:32.575 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.575 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.575 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.575 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.575 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.575 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.575 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.575 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.575 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-16T17:17:32.575 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-16T17:17:32.576 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:32.576 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.576 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.576 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-16T17:17:32.576 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.576 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.576 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.576 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.576 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.577 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.577 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.577 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.577 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-16T17:17:32.577 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-16T17:17:32.577 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:32.577 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.577 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.577 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:32.577 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.578 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.578 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.578 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.578 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.578 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.578 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.578 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.578 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-16T17:17:32.579 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-16T17:17:32.579 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:32.579 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.579 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.579 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:32.579 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.579 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.579 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.579 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.579 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.580 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.580 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.580 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.580 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-16T17:17:32.580 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-16T17:17:32.580 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:32.580 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.580 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.580 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:32.581 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.581 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.581 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.581 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.581 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.581 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.581 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.581 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.581 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_ENABLED": { 2024-04-16T17:17:32.581 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-16T17:17:32.582 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:32.582 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.582 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.582 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-16T17:17:32.582 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.582 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.582 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.582 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.582 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.582 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.583 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.583 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.583 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-16T17:17:32.583 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-16T17:17:32.583 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.583 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.583 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.583 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-16T17:17:32.583 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.583 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.584 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.584 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.584 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.584 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.584 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.584 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.584 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-16T17:17:32.584 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-16T17:17:32.584 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-16T17:17:32.585 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.585 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.585 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "10", 2024-04-16T17:17:32.585 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.585 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.585 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.585 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.585 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.585 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.585 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.586 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.586 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-16T17:17:32.586 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-16T17:17:32.586 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-16T17:17:32.586 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.586 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.586 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "8", 2024-04-16T17:17:32.586 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.586 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.587 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.587 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.587 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.587 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.587 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.587 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.587 INFO:teuthology.orchestra.run.smithi045.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-16T17:17:32.587 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-16T17:17:32.587 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-16T17:17:32.588 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.588 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.588 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "45", 2024-04-16T17:17:32.588 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.588 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.588 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.588 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.588 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.588 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.588 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.589 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.589 INFO:teuthology.orchestra.run.smithi045.stdout: "RGW_API_ACCESS_KEY": { 2024-04-16T17:17:32.589 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-16T17:17:32.589 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.589 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.589 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.589 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:32.589 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.589 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.589 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.590 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.590 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.590 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.590 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.590 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.590 INFO:teuthology.orchestra.run.smithi045.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-16T17:17:32.590 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-16T17:17:32.590 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.590 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.591 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.591 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "admin", 2024-04-16T17:17:32.591 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.591 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.591 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.591 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.591 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.591 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.591 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.592 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.592 INFO:teuthology.orchestra.run.smithi045.stdout: "RGW_API_SECRET_KEY": { 2024-04-16T17:17:32.592 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-16T17:17:32.592 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.592 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.592 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.592 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:32.592 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.592 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.593 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.593 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.593 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.593 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.593 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.593 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.593 INFO:teuthology.orchestra.run.smithi045.stdout: "RGW_API_SSL_VERIFY": { 2024-04-16T17:17:32.593 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-16T17:17:32.593 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:32.594 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.594 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.594 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-16T17:17:32.594 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.594 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.594 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.594 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.594 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.594 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.595 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.595 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.595 INFO:teuthology.orchestra.run.smithi045.stdout: "UNSAFE_TLS_v1_2": { 2024-04-16T17:17:32.595 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "UNSAFE_TLS_v1_2", 2024-04-16T17:17:32.595 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:32.595 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.595 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.595 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:32.595 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.596 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.596 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.596 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.596 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.596 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.596 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.596 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.597 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:17:31 smithi117 ceph-mon[36730]: from='client.? 172.21.15.45:0/3453408999' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-16T17:17:32.597 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:17:31 smithi117 ceph-mon[36730]: from='client.? 172.21.15.45:0/780442721' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-16T17:17:32.600 INFO:teuthology.orchestra.run.smithi045.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-16T17:17:32.600 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-16T17:17:32.600 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-16T17:17:32.600 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.600 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.601 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0", 2024-04-16T17:17:32.601 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.601 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.601 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.601 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.601 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.601 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.601 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.601 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.601 INFO:teuthology.orchestra.run.smithi045.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-16T17:17:32.602 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-16T17:17:32.602 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-16T17:17:32.602 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.602 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.602 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "10", 2024-04-16T17:17:32.602 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.602 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.602 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.602 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.603 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.603 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.603 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.603 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.603 INFO:teuthology.orchestra.run.smithi045.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-16T17:17:32.603 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-16T17:17:32.603 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-16T17:17:32.603 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.603 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.603 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "5", 2024-04-16T17:17:32.604 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.604 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.604 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.604 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.604 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.604 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.604 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.604 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.604 INFO:teuthology.orchestra.run.smithi045.stdout: "cross_origin_url": { 2024-04-16T17:17:32.604 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "cross_origin_url", 2024-04-16T17:17:32.605 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.605 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.605 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.605 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:32.605 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.605 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.605 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.605 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.605 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.606 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.606 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.606 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.606 INFO:teuthology.orchestra.run.smithi045.stdout: "crt_file": { 2024-04-16T17:17:32.606 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "crt_file", 2024-04-16T17:17:32.606 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.606 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.606 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.606 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:32.607 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.607 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.607 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.607 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.607 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.607 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.607 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.607 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.607 INFO:teuthology.orchestra.run.smithi045.stdout: "debug": { 2024-04-16T17:17:32.607 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "debug", 2024-04-16T17:17:32.608 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:32.608 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.608 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.608 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:32.608 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.608 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.608 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.608 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Enable/disable debug options", 2024-04-16T17:17:32.608 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.608 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.609 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.609 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.609 INFO:teuthology.orchestra.run.smithi045.stdout: "jwt_token_ttl": { 2024-04-16T17:17:32.609 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "jwt_token_ttl", 2024-04-16T17:17:32.609 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-16T17:17:32.609 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.609 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.609 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "28800", 2024-04-16T17:17:32.609 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.610 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.610 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.610 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.610 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.610 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.610 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.610 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.610 INFO:teuthology.orchestra.run.smithi045.stdout: "key_file": { 2024-04-16T17:17:32.610 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "key_file", 2024-04-16T17:17:32.610 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.611 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.611 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.611 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:32.611 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.611 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.611 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.611 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.611 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.611 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.612 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.612 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.612 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-16T17:17:32.612 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-16T17:17:32.612 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.612 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.612 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.612 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:32.612 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.612 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.613 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-16T17:17:32.613 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-16T17:17:32.613 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-16T17:17:32.613 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-16T17:17:32.613 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-16T17:17:32.613 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-16T17:17:32.613 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-16T17:17:32.613 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-16T17:17:32.613 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.614 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.614 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.614 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.614 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.614 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-16T17:17:32.614 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-16T17:17:32.614 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:32.614 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.614 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.615 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:32.615 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.615 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.615 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.615 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.615 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.615 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.615 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.615 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.616 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-16T17:17:32.616 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-16T17:17:32.616 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.616 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.616 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.616 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-16T17:17:32.616 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.616 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.616 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-16T17:17:32.616 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-16T17:17:32.617 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-16T17:17:32.617 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-16T17:17:32.617 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-16T17:17:32.617 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-16T17:17:32.617 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-16T17:17:32.617 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-16T17:17:32.617 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.617 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.617 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.617 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.618 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.618 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-16T17:17:32.618 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-16T17:17:32.618 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:32.618 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.618 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.618 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:32.618 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.618 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.619 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.619 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.619 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.619 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.619 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.619 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.619 INFO:teuthology.orchestra.run.smithi045.stdout: "motd": { 2024-04-16T17:17:32.619 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "motd", 2024-04-16T17:17:32.619 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.620 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.620 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.620 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:32.620 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.620 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.620 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.620 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "The message of the day", 2024-04-16T17:17:32.620 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.620 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.621 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.621 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.621 INFO:teuthology.orchestra.run.smithi045.stdout: "redirect_resolve_ip_addr": { 2024-04-16T17:17:32.621 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "redirect_resolve_ip_addr", 2024-04-16T17:17:32.621 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:32.621 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.621 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.621 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:32.621 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.622 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.622 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.622 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.622 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.622 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.622 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.622 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.622 INFO:teuthology.orchestra.run.smithi045.stdout: "server_addr": { 2024-04-16T17:17:32.622 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "server_addr", 2024-04-16T17:17:32.623 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.623 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.623 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.623 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "::", 2024-04-16T17:17:32.623 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.623 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.623 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.623 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.624 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.624 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.624 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.624 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.624 INFO:teuthology.orchestra.run.smithi045.stdout: "server_port": { 2024-04-16T17:17:32.624 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "server_port", 2024-04-16T17:17:32.624 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-16T17:17:32.624 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.624 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.625 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "8080", 2024-04-16T17:17:32.625 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.625 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.625 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.625 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.625 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.625 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.625 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.625 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.626 INFO:teuthology.orchestra.run.smithi045.stdout: "ssl": { 2024-04-16T17:17:32.626 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ssl", 2024-04-16T17:17:32.626 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:32.626 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.626 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.626 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-16T17:17:32.626 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.626 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.626 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.626 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.627 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.627 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.627 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.627 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.627 INFO:teuthology.orchestra.run.smithi045.stdout: "ssl_server_port": { 2024-04-16T17:17:32.627 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ssl_server_port", 2024-04-16T17:17:32.627 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-16T17:17:32.627 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.627 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.628 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "8443", 2024-04-16T17:17:32.628 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.628 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.628 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.628 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.628 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.628 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.628 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.628 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.629 INFO:teuthology.orchestra.run.smithi045.stdout: "standby_behaviour": { 2024-04-16T17:17:32.629 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "standby_behaviour", 2024-04-16T17:17:32.629 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.629 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.629 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.629 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "redirect", 2024-04-16T17:17:32.629 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.629 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.629 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-16T17:17:32.629 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-16T17:17:32.630 INFO:teuthology.orchestra.run.smithi045.stdout: "redirect" 2024-04-16T17:17:32.630 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-16T17:17:32.630 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.630 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.630 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.630 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.630 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.630 INFO:teuthology.orchestra.run.smithi045.stdout: "standby_error_status_code": { 2024-04-16T17:17:32.630 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "standby_error_status_code", 2024-04-16T17:17:32.631 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-16T17:17:32.631 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.631 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.631 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "500", 2024-04-16T17:17:32.631 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "400", 2024-04-16T17:17:32.631 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "599", 2024-04-16T17:17:32.631 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.631 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.631 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.631 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.632 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.632 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.632 INFO:teuthology.orchestra.run.smithi045.stdout: "url_prefix": { 2024-04-16T17:17:32.632 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "url_prefix", 2024-04-16T17:17:32.632 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.632 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.632 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.632 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:32.632 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.632 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.633 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.633 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.633 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.633 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.633 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.633 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:17:32.633 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:17:32.633 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.633 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-16T17:17:32.634 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "devicehealth", 2024-04-16T17:17:32.634 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-16T17:17:32.634 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-16T17:17:32.634 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-16T17:17:32.634 INFO:teuthology.orchestra.run.smithi045.stdout: "enable_monitoring": { 2024-04-16T17:17:32.634 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "enable_monitoring", 2024-04-16T17:17:32.634 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:32.634 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.634 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.634 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-16T17:17:32.635 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.635 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.635 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.635 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "monitor device health metrics", 2024-04-16T17:17:32.635 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.635 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.635 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.635 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.635 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-16T17:17:32.635 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-16T17:17:32.636 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.636 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.636 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.636 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:32.636 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.636 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.636 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-16T17:17:32.636 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-16T17:17:32.637 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-16T17:17:32.637 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-16T17:17:32.637 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-16T17:17:32.637 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-16T17:17:32.637 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-16T17:17:32.637 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-16T17:17:32.637 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.637 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.637 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.638 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.638 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.638 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-16T17:17:32.638 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-16T17:17:32.638 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:32.638 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.638 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.638 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:32.638 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.638 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.639 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.639 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.639 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.639 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.639 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.639 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.639 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-16T17:17:32.639 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-16T17:17:32.639 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.640 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.640 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.640 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-16T17:17:32.640 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.640 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.640 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-16T17:17:32.640 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-16T17:17:32.640 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-16T17:17:32.640 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-16T17:17:32.640 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-16T17:17:32.641 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-16T17:17:32.641 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-16T17:17:32.641 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-16T17:17:32.641 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.641 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.641 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.641 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.641 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.641 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-16T17:17:32.642 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-16T17:17:32.642 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:32.642 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.642 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.642 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:32.642 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.642 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.642 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.642 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.643 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.643 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.643 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.643 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.643 INFO:teuthology.orchestra.run.smithi045.stdout: "mark_out_threshold": { 2024-04-16T17:17:32.643 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "mark_out_threshold", 2024-04-16T17:17:32.643 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-04-16T17:17:32.643 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.643 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.644 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "2419200", 2024-04-16T17:17:32.644 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.644 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.644 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.644 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-16T17:17:32.644 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.644 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.644 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.644 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.644 INFO:teuthology.orchestra.run.smithi045.stdout: "pool_name": { 2024-04-16T17:17:32.645 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "pool_name", 2024-04-16T17:17:32.645 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.645 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.645 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.645 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "device_health_metrics", 2024-04-16T17:17:32.645 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.645 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.645 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.645 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-16T17:17:32.645 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.646 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.646 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.646 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.646 INFO:teuthology.orchestra.run.smithi045.stdout: "retention_period": { 2024-04-16T17:17:32.646 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "retention_period", 2024-04-16T17:17:32.646 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-04-16T17:17:32.646 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.646 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.646 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "15552000", 2024-04-16T17:17:32.647 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.647 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.647 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.647 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "how long to retain device health metrics", 2024-04-16T17:17:32.647 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.647 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.647 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.647 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.647 INFO:teuthology.orchestra.run.smithi045.stdout: "scrape_frequency": { 2024-04-16T17:17:32.647 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "scrape_frequency", 2024-04-16T17:17:32.648 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-04-16T17:17:32.648 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.648 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.648 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "86400", 2024-04-16T17:17:32.648 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.648 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.648 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.648 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-16T17:17:32.648 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.649 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.649 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.649 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.649 INFO:teuthology.orchestra.run.smithi045.stdout: "self_heal": { 2024-04-16T17:17:32.649 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "self_heal", 2024-04-16T17:17:32.649 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:32.649 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.649 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.649 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-16T17:17:32.649 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.650 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.650 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.650 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-16T17:17:32.650 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.650 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.650 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.650 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.650 INFO:teuthology.orchestra.run.smithi045.stdout: "sleep_interval": { 2024-04-16T17:17:32.650 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "sleep_interval", 2024-04-16T17:17:32.651 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-04-16T17:17:32.651 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.651 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.651 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "600", 2024-04-16T17:17:32.651 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.651 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.651 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.651 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "how frequently to wake up and check device health", 2024-04-16T17:17:32.651 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.652 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.652 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.652 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.652 INFO:teuthology.orchestra.run.smithi045.stdout: "warn_threshold": { 2024-04-16T17:17:32.652 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "warn_threshold", 2024-04-16T17:17:32.652 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-04-16T17:17:32.652 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.652 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.652 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "7257600", 2024-04-16T17:17:32.652 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.653 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.653 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.653 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-16T17:17:32.653 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.653 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.653 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.653 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:17:32.653 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:17:32.653 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.654 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-16T17:17:32.654 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "diskprediction_local", 2024-04-16T17:17:32.654 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-16T17:17:32.654 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-16T17:17:32.654 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-16T17:17:32.654 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-16T17:17:32.654 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-16T17:17:32.654 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.654 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.655 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.655 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:32.655 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.655 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.655 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-16T17:17:32.655 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-16T17:17:32.655 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-16T17:17:32.655 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-16T17:17:32.655 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-16T17:17:32.655 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-16T17:17:32.656 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-16T17:17:32.656 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-16T17:17:32.656 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.656 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.656 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.656 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.656 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.656 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-16T17:17:32.656 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-16T17:17:32.656 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:32.657 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.657 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.657 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:32.657 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.657 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.657 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.657 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.657 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.657 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.658 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.658 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.658 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-16T17:17:32.658 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-16T17:17:32.658 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.658 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.658 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.658 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-16T17:17:32.658 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.658 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.659 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-16T17:17:32.659 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-16T17:17:32.659 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-16T17:17:32.659 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-16T17:17:32.659 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-16T17:17:32.659 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-16T17:17:32.659 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-16T17:17:32.659 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-16T17:17:32.659 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.659 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.660 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.660 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.660 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.660 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-16T17:17:32.660 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-16T17:17:32.660 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:32.660 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.660 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.661 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:32.661 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.661 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.661 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.661 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.661 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.661 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.661 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.661 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.662 INFO:teuthology.orchestra.run.smithi045.stdout: "predict_interval": { 2024-04-16T17:17:32.662 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "predict_interval", 2024-04-16T17:17:32.662 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.662 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.662 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.662 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "86400", 2024-04-16T17:17:32.662 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.662 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.662 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.663 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.663 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.663 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.663 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.663 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.663 INFO:teuthology.orchestra.run.smithi045.stdout: "predictor_model": { 2024-04-16T17:17:32.663 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "predictor_model", 2024-04-16T17:17:32.663 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.663 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.664 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.664 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "prophetstor", 2024-04-16T17:17:32.664 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.664 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.664 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.664 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.664 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.664 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.664 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.664 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.665 INFO:teuthology.orchestra.run.smithi045.stdout: "sleep_interval": { 2024-04-16T17:17:32.665 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "sleep_interval", 2024-04-16T17:17:32.665 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.665 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.665 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.665 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "600", 2024-04-16T17:17:32.665 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.665 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.665 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.665 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.666 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.666 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.666 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.666 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:17:32.666 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:17:32.666 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.666 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-16T17:17:32.666 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "influx", 2024-04-16T17:17:32.666 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": false, 2024-04-16T17:17:32.667 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "influxdb python module not found", 2024-04-16T17:17:32.667 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-16T17:17:32.667 INFO:teuthology.orchestra.run.smithi045.stdout: "batch_size": { 2024-04-16T17:17:32.667 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "batch_size", 2024-04-16T17:17:32.667 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-16T17:17:32.667 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.667 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.667 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "5000", 2024-04-16T17:17:32.667 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.667 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.668 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.668 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-16T17:17:32.668 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.668 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.668 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.668 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.668 INFO:teuthology.orchestra.run.smithi045.stdout: "database": { 2024-04-16T17:17:32.668 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "database", 2024-04-16T17:17:32.668 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.669 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.669 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.669 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "ceph", 2024-04-16T17:17:32.669 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.669 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.669 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.669 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-16T17:17:32.669 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.669 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.670 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.670 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.670 INFO:teuthology.orchestra.run.smithi045.stdout: "hostname": { 2024-04-16T17:17:32.670 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "hostname", 2024-04-16T17:17:32.670 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.670 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.670 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.670 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:32.670 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.671 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.671 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.671 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "InfluxDB server hostname", 2024-04-16T17:17:32.671 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.671 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.671 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.671 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.671 INFO:teuthology.orchestra.run.smithi045.stdout: "interval": { 2024-04-16T17:17:32.671 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "interval", 2024-04-16T17:17:32.672 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-04-16T17:17:32.672 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.672 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.672 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "30", 2024-04-16T17:17:32.672 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "5", 2024-04-16T17:17:32.672 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.672 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.672 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-16T17:17:32.672 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.673 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.673 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.673 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.673 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-16T17:17:32.673 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-16T17:17:32.673 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.673 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.673 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.673 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:32.673 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.674 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.674 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-16T17:17:32.674 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-16T17:17:32.674 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-16T17:17:32.674 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-16T17:17:32.674 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-16T17:17:32.674 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-16T17:17:32.674 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-16T17:17:32.674 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-16T17:17:32.675 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.675 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.675 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.675 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.675 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.675 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-16T17:17:32.675 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-16T17:17:32.675 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:32.675 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.675 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.676 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:32.676 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.676 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.676 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.676 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.676 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.676 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.676 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.676 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.677 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-16T17:17:32.677 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-16T17:17:32.677 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.677 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.677 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.677 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-16T17:17:32.677 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.677 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.677 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-16T17:17:32.678 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-16T17:17:32.678 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-16T17:17:32.678 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-16T17:17:32.678 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-16T17:17:32.678 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-16T17:17:32.678 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-16T17:17:32.678 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-16T17:17:32.678 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.678 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.679 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.679 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.679 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.679 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-16T17:17:32.679 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-16T17:17:32.679 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:32.679 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.679 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.679 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:32.680 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.680 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.680 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.680 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.680 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.680 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.680 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.680 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.680 INFO:teuthology.orchestra.run.smithi045.stdout: "password": { 2024-04-16T17:17:32.680 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "password", 2024-04-16T17:17:32.681 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.681 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.681 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.681 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:32.681 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.681 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.681 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.681 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "password of InfluxDB server user", 2024-04-16T17:17:32.681 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.682 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.682 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.682 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.682 INFO:teuthology.orchestra.run.smithi045.stdout: "port": { 2024-04-16T17:17:32.682 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "port", 2024-04-16T17:17:32.682 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-16T17:17:32.682 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.682 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.683 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "8086", 2024-04-16T17:17:32.683 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.683 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.683 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.683 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "InfluxDB server port", 2024-04-16T17:17:32.683 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.683 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.683 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.683 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.683 INFO:teuthology.orchestra.run.smithi045.stdout: "ssl": { 2024-04-16T17:17:32.684 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ssl", 2024-04-16T17:17:32.684 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.684 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.684 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.684 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "false", 2024-04-16T17:17:32.684 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.684 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.684 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.684 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-16T17:17:32.684 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.685 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.685 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.685 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.685 INFO:teuthology.orchestra.run.smithi045.stdout: "threads": { 2024-04-16T17:17:32.685 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "threads", 2024-04-16T17:17:32.685 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-16T17:17:32.685 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.685 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.685 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "5", 2024-04-16T17:17:32.685 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "1", 2024-04-16T17:17:32.686 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "32", 2024-04-16T17:17:32.686 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.686 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-16T17:17:32.686 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.686 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.686 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.686 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.686 INFO:teuthology.orchestra.run.smithi045.stdout: "username": { 2024-04-16T17:17:32.687 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "username", 2024-04-16T17:17:32.687 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.687 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.687 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.687 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:32.687 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.687 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.687 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.687 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "username of InfluxDB server user", 2024-04-16T17:17:32.688 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.688 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.688 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.688 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.688 INFO:teuthology.orchestra.run.smithi045.stdout: "verify_ssl": { 2024-04-16T17:17:32.688 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "verify_ssl", 2024-04-16T17:17:32.688 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.688 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.688 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.688 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "true", 2024-04-16T17:17:32.689 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.689 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.689 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.689 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-16T17:17:32.689 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.689 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.689 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.689 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:17:32.689 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:17:32.689 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.690 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-16T17:17:32.690 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "insights", 2024-04-16T17:17:32.690 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-16T17:17:32.690 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-16T17:17:32.690 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-16T17:17:32.690 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-16T17:17:32.690 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-16T17:17:32.690 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.690 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.691 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.691 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:32.691 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.691 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.691 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-16T17:17:32.691 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-16T17:17:32.691 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-16T17:17:32.691 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-16T17:17:32.691 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-16T17:17:32.691 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-16T17:17:32.692 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-16T17:17:32.692 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-16T17:17:32.692 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.692 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.692 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.692 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.692 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.692 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-16T17:17:32.692 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-16T17:17:32.693 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:32.693 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.693 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.693 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:32.693 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.693 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.693 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.693 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.693 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.694 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.694 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.694 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.694 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-16T17:17:32.694 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-16T17:17:32.694 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.694 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.694 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.694 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-16T17:17:32.694 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.695 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.695 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-16T17:17:32.695 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-16T17:17:32.695 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-16T17:17:32.695 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-16T17:17:32.695 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-16T17:17:32.695 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-16T17:17:32.695 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-16T17:17:32.695 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-16T17:17:32.695 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.696 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.696 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.696 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.696 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.696 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-16T17:17:32.696 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-16T17:17:32.696 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:32.696 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.696 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.697 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:32.697 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.697 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.697 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.697 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.697 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.697 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.697 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.697 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:17:32.697 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:17:32.698 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.698 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-16T17:17:32.698 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "iostat", 2024-04-16T17:17:32.698 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-16T17:17:32.698 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-16T17:17:32.698 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-16T17:17:32.698 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-16T17:17:32.698 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-16T17:17:32.699 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.699 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.699 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.699 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:32.699 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.699 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.699 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-16T17:17:32.699 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-16T17:17:32.699 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-16T17:17:32.699 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-16T17:17:32.700 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-16T17:17:32.700 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-16T17:17:32.700 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-16T17:17:32.700 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-16T17:17:32.700 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.700 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.700 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.700 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.700 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.700 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-16T17:17:32.701 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-16T17:17:32.701 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:32.701 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.701 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.701 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:32.701 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.701 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.701 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.701 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.702 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.702 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.702 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.702 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.702 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-16T17:17:32.702 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-16T17:17:32.705 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.706 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.706 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.706 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-16T17:17:32.707 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.707 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.707 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-16T17:17:32.707 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-16T17:17:32.707 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-16T17:17:32.707 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-16T17:17:32.707 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-16T17:17:32.707 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-16T17:17:32.707 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-16T17:17:32.707 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-16T17:17:32.708 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.708 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.708 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.708 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.708 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.708 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-16T17:17:32.708 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-16T17:17:32.708 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:32.708 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.708 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.709 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:32.709 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.709 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.709 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.709 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.709 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.709 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.709 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.709 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:17:32.709 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:17:32.710 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.710 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-16T17:17:32.710 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "k8sevents", 2024-04-16T17:17:32.710 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-16T17:17:32.710 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-16T17:17:32.710 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-16T17:17:32.710 INFO:teuthology.orchestra.run.smithi045.stdout: "ceph_event_retention_days": { 2024-04-16T17:17:32.710 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ceph_event_retention_days", 2024-04-16T17:17:32.710 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-16T17:17:32.711 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.711 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.711 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "7", 2024-04-16T17:17:32.711 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.711 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.711 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.711 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-16T17:17:32.711 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.711 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.712 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.712 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.712 INFO:teuthology.orchestra.run.smithi045.stdout: "config_check_secs": { 2024-04-16T17:17:32.712 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "config_check_secs", 2024-04-16T17:17:32.712 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-16T17:17:32.712 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.712 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.712 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "10", 2024-04-16T17:17:32.712 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "10", 2024-04-16T17:17:32.712 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.713 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.713 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-16T17:17:32.713 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.713 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.713 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.713 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.713 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-16T17:17:32.713 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-16T17:17:32.713 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.714 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.714 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.714 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:32.714 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.714 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.714 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-16T17:17:32.714 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-16T17:17:32.714 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-16T17:17:32.714 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-16T17:17:32.714 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-16T17:17:32.715 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-16T17:17:32.715 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-16T17:17:32.715 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-16T17:17:32.715 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.715 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.715 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.715 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.715 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.715 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-16T17:17:32.716 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-16T17:17:32.716 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:32.716 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.716 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.716 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:32.716 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.716 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.716 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.716 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.716 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.717 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.717 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.717 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.717 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-16T17:17:32.717 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-16T17:17:32.717 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.717 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.717 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.717 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-16T17:17:32.718 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.718 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.718 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-16T17:17:32.718 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-16T17:17:32.718 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-16T17:17:32.718 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-16T17:17:32.718 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-16T17:17:32.718 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-16T17:17:32.718 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-16T17:17:32.719 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-16T17:17:32.719 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.719 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.719 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.719 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.719 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.719 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-16T17:17:32.719 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-16T17:17:32.719 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:32.720 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.720 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.720 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:32.720 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.720 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.720 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.720 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.720 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.720 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.721 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.721 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:17:32.721 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:17:32.721 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.721 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-16T17:17:32.721 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "localpool", 2024-04-16T17:17:32.721 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-16T17:17:32.721 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-16T17:17:32.721 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-16T17:17:32.721 INFO:teuthology.orchestra.run.smithi045.stdout: "failure_domain": { 2024-04-16T17:17:32.722 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "failure_domain", 2024-04-16T17:17:32.722 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.722 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.722 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.722 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "host", 2024-04-16T17:17:32.722 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.722 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.722 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.722 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "failure domain for any created local pool", 2024-04-16T17:17:32.722 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-16T17:17:32.723 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.723 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.723 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.723 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-16T17:17:32.723 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-16T17:17:32.723 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.723 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.723 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.723 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:32.724 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.724 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.724 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-16T17:17:32.724 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-16T17:17:32.724 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-16T17:17:32.724 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-16T17:17:32.724 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-16T17:17:32.724 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-16T17:17:32.724 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-16T17:17:32.724 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-16T17:17:32.725 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.725 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.725 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.725 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.725 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.725 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-16T17:17:32.725 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-16T17:17:32.726 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:32.726 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.726 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.726 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:32.726 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.726 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.726 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.726 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.727 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.727 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.727 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.727 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.727 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-16T17:17:32.727 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-16T17:17:32.727 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.727 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.727 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.727 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-16T17:17:32.728 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.728 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.728 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-16T17:17:32.728 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-16T17:17:32.728 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-16T17:17:32.728 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-16T17:17:32.728 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-16T17:17:32.728 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-16T17:17:32.728 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-16T17:17:32.728 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-16T17:17:32.729 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.729 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.729 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.729 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.729 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.729 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-16T17:17:32.729 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-16T17:17:32.729 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:32.729 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.730 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.730 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:32.730 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.730 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.730 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.730 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.730 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.730 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.731 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.731 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.731 INFO:teuthology.orchestra.run.smithi045.stdout: "min_size": { 2024-04-16T17:17:32.731 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "min_size", 2024-04-16T17:17:32.731 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-16T17:17:32.731 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.731 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.731 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:32.731 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.732 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.732 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.732 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "default min_size for any created local pool", 2024-04-16T17:17:32.732 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-16T17:17:32.732 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.732 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.732 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.732 INFO:teuthology.orchestra.run.smithi045.stdout: "num_rep": { 2024-04-16T17:17:32.732 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "num_rep", 2024-04-16T17:17:32.732 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-16T17:17:32.733 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.733 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.733 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "3", 2024-04-16T17:17:32.733 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.733 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.733 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.733 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "default replica count for any created local pool", 2024-04-16T17:17:32.733 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.733 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.733 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.734 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.734 INFO:teuthology.orchestra.run.smithi045.stdout: "pg_num": { 2024-04-16T17:17:32.734 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "pg_num", 2024-04-16T17:17:32.734 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-16T17:17:32.734 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.734 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.734 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "128", 2024-04-16T17:17:32.734 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.734 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.734 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.735 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "default pg_num for any created local pool", 2024-04-16T17:17:32.735 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.735 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.735 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.735 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.735 INFO:teuthology.orchestra.run.smithi045.stdout: "prefix": { 2024-04-16T17:17:32.735 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "prefix", 2024-04-16T17:17:32.735 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.735 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.735 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.736 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:32.736 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.736 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.736 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.736 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "name prefix for any created local pool", 2024-04-16T17:17:32.736 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.736 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.736 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.736 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.737 INFO:teuthology.orchestra.run.smithi045.stdout: "subtree": { 2024-04-16T17:17:32.737 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "subtree", 2024-04-16T17:17:32.737 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.737 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.737 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.737 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "rack", 2024-04-16T17:17:32.737 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.737 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.737 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.737 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-16T17:17:32.738 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-16T17:17:32.738 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.738 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.738 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:17:32.738 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:17:32.738 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.738 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-16T17:17:32.738 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "mds_autoscaler", 2024-04-16T17:17:32.738 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-16T17:17:32.739 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-16T17:17:32.739 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-16T17:17:32.739 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-16T17:17:32.739 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-16T17:17:32.739 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.739 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.739 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.739 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:32.739 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.739 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.740 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-16T17:17:32.740 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-16T17:17:32.740 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-16T17:17:32.740 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-16T17:17:32.740 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-16T17:17:32.740 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-16T17:17:32.740 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-16T17:17:32.740 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-16T17:17:32.740 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.741 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.741 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.741 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.741 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.741 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-16T17:17:32.741 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-16T17:17:32.741 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:32.741 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.741 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.742 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:32.742 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.742 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.742 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.742 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.742 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.742 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.742 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.742 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.742 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-16T17:17:32.743 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-16T17:17:32.743 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.743 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.743 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.743 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-16T17:17:32.743 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.743 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.743 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-16T17:17:32.743 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-16T17:17:32.743 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-16T17:17:32.744 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-16T17:17:32.744 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-16T17:17:32.744 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-16T17:17:32.744 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-16T17:17:32.744 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-16T17:17:32.744 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.744 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.744 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.744 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.744 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.745 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-16T17:17:32.745 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-16T17:17:32.745 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:32.745 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.745 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.745 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:32.745 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.745 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.745 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.746 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.746 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.746 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.746 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.746 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:17:32.746 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:17:32.746 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.746 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-16T17:17:32.747 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "mirroring", 2024-04-16T17:17:32.747 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-16T17:17:32.747 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-16T17:17:32.747 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-16T17:17:32.747 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-16T17:17:32.747 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-16T17:17:32.747 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.747 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.747 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.747 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:32.748 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.748 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.748 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-16T17:17:32.748 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-16T17:17:32.748 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-16T17:17:32.748 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-16T17:17:32.748 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-16T17:17:32.748 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-16T17:17:32.748 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-16T17:17:32.749 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-16T17:17:32.749 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.749 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.749 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.749 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.749 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.749 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-16T17:17:32.749 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-16T17:17:32.749 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:32.750 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.750 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.750 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:32.750 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.750 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.750 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.750 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.750 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.751 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.751 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.751 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.751 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-16T17:17:32.751 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-16T17:17:32.751 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.751 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.751 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.751 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-16T17:17:32.752 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.752 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.752 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-16T17:17:32.752 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-16T17:17:32.752 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-16T17:17:32.752 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-16T17:17:32.752 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-16T17:17:32.752 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-16T17:17:32.752 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-16T17:17:32.752 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-16T17:17:32.753 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.753 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.753 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.753 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.753 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.753 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-16T17:17:32.753 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-16T17:17:32.753 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:32.753 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.754 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.754 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:32.754 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.754 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.754 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.754 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.754 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.754 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.754 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.755 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:17:32.755 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:17:32.755 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.755 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-16T17:17:32.755 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "nfs", 2024-04-16T17:17:32.755 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-16T17:17:32.755 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-16T17:17:32.755 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-16T17:17:32.755 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-16T17:17:32.756 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-16T17:17:32.756 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.756 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.756 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.756 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:32.756 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.756 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.756 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-16T17:17:32.756 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-16T17:17:32.757 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-16T17:17:32.757 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-16T17:17:32.757 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-16T17:17:32.757 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-16T17:17:32.757 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-16T17:17:32.757 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-16T17:17:32.757 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.757 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.757 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.758 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.758 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.758 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-16T17:17:32.758 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-16T17:17:32.758 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:32.758 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.758 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.758 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:32.758 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.759 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.759 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.759 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.759 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.759 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.759 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.759 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.759 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-16T17:17:32.759 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-16T17:17:32.760 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.760 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.760 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.760 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-16T17:17:32.760 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.760 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.760 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-16T17:17:32.760 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-16T17:17:32.760 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-16T17:17:32.760 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-16T17:17:32.761 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-16T17:17:32.761 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-16T17:17:32.761 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-16T17:17:32.761 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-16T17:17:32.761 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.761 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.761 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.761 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.762 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.762 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-16T17:17:32.762 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-16T17:17:32.762 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:32.762 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.762 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.762 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:32.762 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.763 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.763 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.763 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.763 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.763 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.763 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.763 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:17:32.763 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:17:32.763 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.764 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-16T17:17:32.764 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "orchestrator", 2024-04-16T17:17:32.764 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-16T17:17:32.764 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-16T17:17:32.764 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-16T17:17:32.764 INFO:teuthology.orchestra.run.smithi045.stdout: "fail_fs": { 2024-04-16T17:17:32.764 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "fail_fs", 2024-04-16T17:17:32.764 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:32.764 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.765 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.765 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:32.765 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.765 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.765 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.765 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-16T17:17:32.765 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.765 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.765 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.766 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.766 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-16T17:17:32.766 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-16T17:17:32.766 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.766 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.766 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.766 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:32.766 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.766 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.767 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-16T17:17:32.767 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-16T17:17:32.767 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-16T17:17:32.767 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-16T17:17:32.767 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-16T17:17:32.767 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-16T17:17:32.767 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-16T17:17:32.767 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-16T17:17:32.768 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.768 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.768 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.768 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.768 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.768 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-16T17:17:32.768 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-16T17:17:32.768 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:32.768 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.768 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.769 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:32.769 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.769 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.769 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.769 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.769 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.769 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.769 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.769 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.769 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-16T17:17:32.770 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-16T17:17:32.770 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.770 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.770 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.770 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-16T17:17:32.770 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.770 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.771 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-16T17:17:32.771 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-16T17:17:32.771 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-16T17:17:32.771 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-16T17:17:32.771 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-16T17:17:32.771 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-16T17:17:32.771 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-16T17:17:32.771 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-16T17:17:32.771 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.771 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.772 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.772 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.772 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.772 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-16T17:17:32.772 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-16T17:17:32.772 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:32.772 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.772 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.772 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:32.772 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.773 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.773 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.773 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.773 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.773 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.773 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.773 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.773 INFO:teuthology.orchestra.run.smithi045.stdout: "orchestrator": { 2024-04-16T17:17:32.773 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "orchestrator", 2024-04-16T17:17:32.774 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.774 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.774 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.774 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:32.774 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.774 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.774 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-16T17:17:32.774 INFO:teuthology.orchestra.run.smithi045.stdout: "cephadm", 2024-04-16T17:17:32.775 INFO:teuthology.orchestra.run.smithi045.stdout: "rook", 2024-04-16T17:17:32.775 INFO:teuthology.orchestra.run.smithi045.stdout: "test_orchestrator" 2024-04-16T17:17:32.775 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-16T17:17:32.775 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Orchestrator backend", 2024-04-16T17:17:32.775 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.775 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.775 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.775 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:17:32.775 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:17:32.775 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.776 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-16T17:17:32.776 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "osd_perf_query", 2024-04-16T17:17:32.776 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-16T17:17:32.776 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-16T17:17:32.776 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-16T17:17:32.776 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-16T17:17:32.776 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-16T17:17:32.776 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.776 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.776 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.777 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:32.777 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.777 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.777 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-16T17:17:32.777 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-16T17:17:32.777 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-16T17:17:32.777 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-16T17:17:32.777 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-16T17:17:32.777 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-16T17:17:32.778 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-16T17:17:32.778 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-16T17:17:32.778 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.778 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.778 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.778 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.778 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.778 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-16T17:17:32.778 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-16T17:17:32.779 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:32.779 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.779 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.779 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:32.779 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.779 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.779 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.779 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.779 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.780 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.780 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.780 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.780 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-16T17:17:32.780 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-16T17:17:32.780 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.780 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.780 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.780 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-16T17:17:32.781 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.781 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.781 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-16T17:17:32.781 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-16T17:17:32.781 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-16T17:17:32.781 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-16T17:17:32.781 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-16T17:17:32.781 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-16T17:17:32.781 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-16T17:17:32.781 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-16T17:17:32.782 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.782 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.782 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.782 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.782 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.782 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-16T17:17:32.782 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-16T17:17:32.782 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:32.782 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.782 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.783 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:32.783 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.783 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.783 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.783 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.783 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.783 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.783 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.783 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:17:32.783 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:17:32.784 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.784 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-16T17:17:32.784 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "osd_support", 2024-04-16T17:17:32.784 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-16T17:17:32.784 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-16T17:17:32.784 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-16T17:17:32.784 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-16T17:17:32.784 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-16T17:17:32.784 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.784 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.785 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.785 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:32.785 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.785 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.785 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-16T17:17:32.785 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-16T17:17:32.785 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-16T17:17:32.785 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-16T17:17:32.785 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-16T17:17:32.785 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-16T17:17:32.786 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-16T17:17:32.786 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-16T17:17:32.786 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.786 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.786 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.786 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.786 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.786 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-16T17:17:32.787 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-16T17:17:32.787 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:32.787 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.787 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.787 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:32.787 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.787 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.787 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.787 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.788 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.788 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.788 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.788 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.788 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-16T17:17:32.788 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-16T17:17:32.788 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.788 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.788 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.789 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-16T17:17:32.789 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.789 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.789 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-16T17:17:32.789 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-16T17:17:32.789 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-16T17:17:32.789 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-16T17:17:32.789 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-16T17:17:32.789 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-16T17:17:32.789 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-16T17:17:32.790 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-16T17:17:32.790 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.790 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.790 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.790 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.790 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.790 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-16T17:17:32.790 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-16T17:17:32.790 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:32.791 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.791 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.791 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:32.791 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.791 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.791 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.791 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.791 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.791 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.792 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.792 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:17:32.792 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:17:32.792 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.792 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-16T17:17:32.792 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "pg_autoscaler", 2024-04-16T17:17:32.792 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-16T17:17:32.792 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-16T17:17:32.792 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-16T17:17:32.793 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-16T17:17:32.793 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-16T17:17:32.793 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.793 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.793 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.793 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:32.793 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.793 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.793 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-16T17:17:32.793 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-16T17:17:32.794 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-16T17:17:32.794 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-16T17:17:32.794 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-16T17:17:32.794 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-16T17:17:32.794 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-16T17:17:32.794 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-16T17:17:32.794 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.794 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.794 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.795 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.795 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.795 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-16T17:17:32.795 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-16T17:17:32.795 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:32.795 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.795 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.795 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:32.795 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.795 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.796 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.796 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.796 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.796 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.796 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.796 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.796 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-16T17:17:32.796 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-16T17:17:32.796 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.796 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.797 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.797 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-16T17:17:32.797 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.797 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.797 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-16T17:17:32.797 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-16T17:17:32.797 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-16T17:17:32.797 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-16T17:17:32.797 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-16T17:17:32.797 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-16T17:17:32.798 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-16T17:17:32.798 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-16T17:17:32.798 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.798 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.798 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.798 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.798 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.798 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-16T17:17:32.799 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-16T17:17:32.799 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:32.799 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.799 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.799 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:32.799 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.799 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.799 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.799 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.799 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.800 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.800 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.800 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.800 INFO:teuthology.orchestra.run.smithi045.stdout: "sleep_interval": { 2024-04-16T17:17:32.800 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "sleep_interval", 2024-04-16T17:17:32.800 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-04-16T17:17:32.800 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.800 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.800 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "60", 2024-04-16T17:17:32.801 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.801 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.801 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.801 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.801 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.801 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.801 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.801 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.801 INFO:teuthology.orchestra.run.smithi045.stdout: "threshold": { 2024-04-16T17:17:32.802 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "threshold", 2024-04-16T17:17:32.802 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "float", 2024-04-16T17:17:32.802 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.802 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.802 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "3.0", 2024-04-16T17:17:32.802 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "1.0", 2024-04-16T17:17:32.802 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.802 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.803 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "scaling threshold", 2024-04-16T17:17:32.803 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-16T17:17:32.803 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.803 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.803 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:17:32.803 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:17:32.803 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.803 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-16T17:17:32.803 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "progress", 2024-04-16T17:17:32.803 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-16T17:17:32.804 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-16T17:17:32.804 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-16T17:17:32.804 INFO:teuthology.orchestra.run.smithi045.stdout: "allow_pg_recovery_event": { 2024-04-16T17:17:32.804 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "allow_pg_recovery_event", 2024-04-16T17:17:32.804 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:32.804 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.804 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.804 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:32.804 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.804 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.805 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.805 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-16T17:17:32.805 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.805 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.805 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.805 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.805 INFO:teuthology.orchestra.run.smithi045.stdout: "enabled": { 2024-04-16T17:17:32.805 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "enabled", 2024-04-16T17:17:32.805 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:32.805 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.806 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.806 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-16T17:17:32.806 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.806 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.806 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.806 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.806 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.806 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.807 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.807 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.807 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-16T17:17:32.807 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-16T17:17:32.807 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.807 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.807 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.807 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:32.807 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.807 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.808 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-16T17:17:32.808 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-16T17:17:32.808 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-16T17:17:32.808 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-16T17:17:32.808 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-16T17:17:32.808 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-16T17:17:32.808 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-16T17:17:32.808 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-16T17:17:32.808 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.808 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.809 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.809 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.809 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.809 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-16T17:17:32.809 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-16T17:17:32.809 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:32.809 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.809 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.809 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:32.810 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.810 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.810 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.810 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.810 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.810 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.810 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.810 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.810 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-16T17:17:32.811 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-16T17:17:32.811 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.811 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.811 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.811 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-16T17:17:32.811 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.811 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.811 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-16T17:17:32.811 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-16T17:17:32.812 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-16T17:17:32.812 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-16T17:17:32.812 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-16T17:17:32.812 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-16T17:17:32.812 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-16T17:17:32.812 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-16T17:17:32.812 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.812 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.812 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.812 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.813 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.813 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-16T17:17:32.813 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-16T17:17:32.813 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:32.813 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.813 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.813 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:32.813 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.813 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.813 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.814 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.814 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.814 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.814 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.814 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.814 INFO:teuthology.orchestra.run.smithi045.stdout: "max_completed_events": { 2024-04-16T17:17:32.814 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "max_completed_events", 2024-04-16T17:17:32.814 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-16T17:17:32.815 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.815 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.815 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "50", 2024-04-16T17:17:32.815 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.815 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.815 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.815 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "number of past completed events to remember", 2024-04-16T17:17:32.815 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.815 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.816 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.816 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.816 INFO:teuthology.orchestra.run.smithi045.stdout: "sleep_interval": { 2024-04-16T17:17:32.816 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "sleep_interval", 2024-04-16T17:17:32.816 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-04-16T17:17:32.816 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.816 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.816 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "5", 2024-04-16T17:17:32.816 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.816 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.817 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.817 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "how long the module is going to sleep", 2024-04-16T17:17:32.817 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.817 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.817 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.817 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:17:32.817 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:17:32.817 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.817 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-16T17:17:32.818 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "prometheus", 2024-04-16T17:17:32.818 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-16T17:17:32.818 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-16T17:17:32.818 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-16T17:17:32.818 INFO:teuthology.orchestra.run.smithi045.stdout: "cache": { 2024-04-16T17:17:32.818 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "cache", 2024-04-16T17:17:32.818 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:32.818 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.818 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.819 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-16T17:17:32.819 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.819 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.819 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.819 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.819 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.819 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.819 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.819 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.820 INFO:teuthology.orchestra.run.smithi045.stdout: "exclude_perf_counters": { 2024-04-16T17:17:32.820 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "exclude_perf_counters", 2024-04-16T17:17:32.820 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:32.820 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.820 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.820 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-16T17:17:32.820 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.820 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.820 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.820 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-16T17:17:32.821 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-16T17:17:32.821 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.821 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.821 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.821 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-16T17:17:32.821 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-16T17:17:32.821 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.821 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.821 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.822 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:32.822 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.822 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.822 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-16T17:17:32.822 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-16T17:17:32.822 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-16T17:17:32.822 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-16T17:17:32.822 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-16T17:17:32.822 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-16T17:17:32.823 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-16T17:17:32.823 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-16T17:17:32.823 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.823 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.823 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.823 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.823 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.823 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-16T17:17:32.823 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-16T17:17:32.824 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:32.824 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.824 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.824 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:32.824 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.824 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.824 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.824 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.824 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.825 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.825 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.825 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.825 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-16T17:17:32.825 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-16T17:17:32.825 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.825 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.825 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.825 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-16T17:17:32.825 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.826 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.826 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-16T17:17:32.826 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-16T17:17:32.826 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-16T17:17:32.826 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-16T17:17:32.826 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-16T17:17:32.826 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-16T17:17:32.826 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-16T17:17:32.827 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-16T17:17:32.827 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.827 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.827 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.827 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.827 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.827 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-16T17:17:32.827 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-16T17:17:32.827 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:32.828 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.828 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.828 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:32.828 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.828 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.828 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.828 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.828 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.828 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.828 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.829 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.829 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_stats_pools": { 2024-04-16T17:17:32.829 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rbd_stats_pools", 2024-04-16T17:17:32.829 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.829 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.829 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.829 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:32.829 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.829 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.830 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.830 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.830 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.830 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.830 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.830 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.830 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-16T17:17:32.830 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-16T17:17:32.830 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-16T17:17:32.831 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.831 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.831 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "300", 2024-04-16T17:17:32.831 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.831 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.831 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.831 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.831 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.831 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.832 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.832 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.832 INFO:teuthology.orchestra.run.smithi045.stdout: "scrape_interval": { 2024-04-16T17:17:32.832 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "scrape_interval", 2024-04-16T17:17:32.832 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "float", 2024-04-16T17:17:32.832 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.832 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.832 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "15.0", 2024-04-16T17:17:32.832 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.832 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.833 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.833 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.833 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.833 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.833 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.833 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.833 INFO:teuthology.orchestra.run.smithi045.stdout: "server_addr": { 2024-04-16T17:17:32.833 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "server_addr", 2024-04-16T17:17:32.833 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.833 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.834 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.834 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "::", 2024-04-16T17:17:32.834 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.834 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.834 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.834 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-16T17:17:32.834 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.834 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.834 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.835 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.835 INFO:teuthology.orchestra.run.smithi045.stdout: "server_port": { 2024-04-16T17:17:32.835 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "server_port", 2024-04-16T17:17:32.835 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-16T17:17:32.835 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.835 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.835 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "9283", 2024-04-16T17:17:32.835 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.835 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.835 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.836 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-16T17:17:32.836 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.836 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.836 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.836 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.836 INFO:teuthology.orchestra.run.smithi045.stdout: "stale_cache_strategy": { 2024-04-16T17:17:32.836 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "stale_cache_strategy", 2024-04-16T17:17:32.836 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.836 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.836 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.837 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "log", 2024-04-16T17:17:32.837 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.837 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.837 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.837 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.837 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.837 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.837 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.837 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.838 INFO:teuthology.orchestra.run.smithi045.stdout: "standby_behaviour": { 2024-04-16T17:17:32.838 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "standby_behaviour", 2024-04-16T17:17:32.838 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.838 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.838 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.838 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "default", 2024-04-16T17:17:32.838 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.838 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.839 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-16T17:17:32.839 INFO:teuthology.orchestra.run.smithi045.stdout: "default", 2024-04-16T17:17:32.839 INFO:teuthology.orchestra.run.smithi045.stdout: "error" 2024-04-16T17:17:32.839 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-16T17:17:32.839 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.839 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.839 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.839 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.839 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.839 INFO:teuthology.orchestra.run.smithi045.stdout: "standby_error_status_code": { 2024-04-16T17:17:32.840 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "standby_error_status_code", 2024-04-16T17:17:32.840 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-16T17:17:32.840 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.840 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.840 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "500", 2024-04-16T17:17:32.840 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "400", 2024-04-16T17:17:32.840 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "599", 2024-04-16T17:17:32.840 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.840 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.841 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.841 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.841 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.841 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:17:32.841 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:17:32.841 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.841 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-16T17:17:32.841 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rbd_support", 2024-04-16T17:17:32.841 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-16T17:17:32.842 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-16T17:17:32.842 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-16T17:17:32.842 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-16T17:17:32.842 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-16T17:17:32.842 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.842 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.842 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.842 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:32.842 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.842 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.843 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-16T17:17:32.843 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-16T17:17:32.843 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-16T17:17:32.843 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-16T17:17:32.843 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-16T17:17:32.843 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-16T17:17:32.843 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-16T17:17:32.843 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-16T17:17:32.843 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.843 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.844 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.844 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.844 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.844 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-16T17:17:32.844 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-16T17:17:32.844 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:32.844 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.844 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.844 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:32.844 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.845 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.845 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.845 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.845 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.845 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.845 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.845 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.845 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-16T17:17:32.845 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-16T17:17:32.846 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.846 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.846 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.846 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-16T17:17:32.846 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.846 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.846 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-16T17:17:32.846 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-16T17:17:32.846 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-16T17:17:32.847 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-16T17:17:32.847 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-16T17:17:32.847 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-16T17:17:32.847 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-16T17:17:32.847 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-16T17:17:32.847 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.847 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.847 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.847 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.847 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.848 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-16T17:17:32.848 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-16T17:17:32.848 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:32.848 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.848 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.848 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:32.848 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.848 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.848 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.848 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.849 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.849 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.849 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.849 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.849 INFO:teuthology.orchestra.run.smithi045.stdout: "max_concurrent_snap_create": { 2024-04-16T17:17:32.849 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "max_concurrent_snap_create", 2024-04-16T17:17:32.849 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-16T17:17:32.849 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.849 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.850 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "10", 2024-04-16T17:17:32.850 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.850 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.850 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.850 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.850 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.850 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.850 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.850 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.850 INFO:teuthology.orchestra.run.smithi045.stdout: "mirror_snapshot_schedule": { 2024-04-16T17:17:32.851 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "mirror_snapshot_schedule", 2024-04-16T17:17:32.851 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.851 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.851 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.851 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:32.851 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.851 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.851 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.851 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.852 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.852 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.852 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.852 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.852 INFO:teuthology.orchestra.run.smithi045.stdout: "trash_purge_schedule": { 2024-04-16T17:17:32.852 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "trash_purge_schedule", 2024-04-16T17:17:32.852 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.852 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.852 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.852 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:32.853 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.853 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.853 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.853 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.853 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.853 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.853 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.853 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:17:32.853 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:17:32.853 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.854 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-16T17:17:32.854 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "restful", 2024-04-16T17:17:32.854 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-16T17:17:32.854 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-16T17:17:32.854 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-16T17:17:32.854 INFO:teuthology.orchestra.run.smithi045.stdout: "enable_auth": { 2024-04-16T17:17:32.854 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "enable_auth", 2024-04-16T17:17:32.854 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:32.854 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.855 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.855 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-16T17:17:32.855 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.855 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.855 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.855 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.855 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.855 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.855 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.856 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.856 INFO:teuthology.orchestra.run.smithi045.stdout: "key_file": { 2024-04-16T17:17:32.856 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "key_file", 2024-04-16T17:17:32.856 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.856 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.856 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.856 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:32.856 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.856 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.856 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.857 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.857 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.857 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.857 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.857 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.857 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-16T17:17:32.857 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-16T17:17:32.857 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.857 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.858 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.858 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:32.858 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.858 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.858 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-16T17:17:32.858 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-16T17:17:32.858 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-16T17:17:32.858 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-16T17:17:32.858 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-16T17:17:32.858 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-16T17:17:32.859 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-16T17:17:32.859 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-16T17:17:32.859 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.859 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.859 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.859 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.859 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.859 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-16T17:17:32.859 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-16T17:17:32.859 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:32.860 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.860 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.860 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:32.860 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.860 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.860 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.860 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.860 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.860 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.861 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.861 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.861 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-16T17:17:32.861 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-16T17:17:32.861 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.861 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.861 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.861 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-16T17:17:32.861 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.861 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.862 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-16T17:17:32.862 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-16T17:17:32.862 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-16T17:17:32.862 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-16T17:17:32.862 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-16T17:17:32.862 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-16T17:17:32.862 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-16T17:17:32.862 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-16T17:17:32.862 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.863 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.863 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.863 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.863 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.863 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-16T17:17:32.863 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-16T17:17:32.863 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:32.863 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.863 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.863 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:32.864 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.864 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.864 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.864 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.864 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.864 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.864 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.864 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.864 INFO:teuthology.orchestra.run.smithi045.stdout: "server_addr": { 2024-04-16T17:17:32.864 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "server_addr", 2024-04-16T17:17:32.865 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.865 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.865 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.865 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:32.865 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.865 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.865 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.865 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.865 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.865 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.866 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.866 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.866 INFO:teuthology.orchestra.run.smithi045.stdout: "server_port": { 2024-04-16T17:17:32.866 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "server_port", 2024-04-16T17:17:32.866 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.866 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.866 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.866 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:32.866 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.867 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.867 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.867 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.867 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.867 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.867 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.867 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:17:32.867 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:17:32.867 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.867 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-16T17:17:32.868 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rgw", 2024-04-16T17:17:32.868 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-16T17:17:32.868 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-16T17:17:32.868 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-16T17:17:32.868 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-16T17:17:32.868 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-16T17:17:32.868 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.868 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.868 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.868 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:32.869 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.869 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.869 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-16T17:17:32.869 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-16T17:17:32.869 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-16T17:17:32.869 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-16T17:17:32.869 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-16T17:17:32.869 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-16T17:17:32.869 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-16T17:17:32.870 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-16T17:17:32.870 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.870 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.870 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.870 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.870 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.870 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-16T17:17:32.870 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-16T17:17:32.870 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:32.871 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.871 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.871 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:32.871 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.871 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.871 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.871 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.871 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.871 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.872 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.872 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.872 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-16T17:17:32.872 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-16T17:17:32.872 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.872 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.872 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.872 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-16T17:17:32.872 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.872 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.873 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-16T17:17:32.873 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-16T17:17:32.873 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-16T17:17:32.873 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-16T17:17:32.873 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-16T17:17:32.873 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-16T17:17:32.873 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-16T17:17:32.873 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-16T17:17:32.873 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.873 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.874 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.874 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.874 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.874 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-16T17:17:32.874 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-16T17:17:32.874 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:32.874 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.874 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.875 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:32.875 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.875 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.875 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.875 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.875 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.875 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.875 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.875 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:17:32.875 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:17:32.876 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.876 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-16T17:17:32.876 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rook", 2024-04-16T17:17:32.876 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-16T17:17:32.876 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-16T17:17:32.876 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-16T17:17:32.876 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-16T17:17:32.876 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-16T17:17:32.876 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.876 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.877 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.877 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:32.877 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.877 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.877 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-16T17:17:32.877 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-16T17:17:32.877 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-16T17:17:32.877 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-16T17:17:32.877 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-16T17:17:32.877 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-16T17:17:32.878 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-16T17:17:32.878 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-16T17:17:32.878 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.878 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.878 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.878 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.878 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.878 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-16T17:17:32.878 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-16T17:17:32.878 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:32.879 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.879 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.879 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:32.879 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.879 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.879 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.879 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.879 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.879 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.880 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.880 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.880 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-16T17:17:32.880 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-16T17:17:32.880 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.880 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.880 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.880 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-16T17:17:32.880 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.880 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.881 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-16T17:17:32.881 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-16T17:17:32.881 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-16T17:17:32.881 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-16T17:17:32.881 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-16T17:17:32.881 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-16T17:17:32.881 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-16T17:17:32.881 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-16T17:17:32.881 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.882 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.882 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.882 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.882 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.882 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-16T17:17:32.882 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-16T17:17:32.882 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:32.882 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.882 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.882 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:32.883 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.883 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.883 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.883 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.883 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.883 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.883 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.883 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.883 INFO:teuthology.orchestra.run.smithi045.stdout: "storage_class": { 2024-04-16T17:17:32.884 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "storage_class", 2024-04-16T17:17:32.884 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.884 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.884 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.884 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "local", 2024-04-16T17:17:32.884 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.884 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.884 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.884 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-16T17:17:32.884 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.885 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.885 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.885 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:17:32.885 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:17:32.885 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.885 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-16T17:17:32.885 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "selftest", 2024-04-16T17:17:32.885 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-16T17:17:32.885 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-16T17:17:32.885 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-16T17:17:32.886 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-16T17:17:32.886 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-16T17:17:32.886 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.886 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.886 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.886 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:32.886 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.886 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.886 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-16T17:17:32.886 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-16T17:17:32.887 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-16T17:17:32.887 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-16T17:17:32.887 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-16T17:17:32.887 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-16T17:17:32.887 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-16T17:17:32.887 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-16T17:17:32.887 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.887 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.888 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.888 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.888 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.888 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-16T17:17:32.888 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-16T17:17:32.888 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:32.888 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.888 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.888 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:32.889 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.889 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.889 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.889 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.889 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.889 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.889 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.889 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.889 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-16T17:17:32.889 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-16T17:17:32.890 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.890 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.890 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.890 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-16T17:17:32.890 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.890 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.890 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-16T17:17:32.890 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-16T17:17:32.890 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-16T17:17:32.891 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-16T17:17:32.891 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-16T17:17:32.891 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-16T17:17:32.891 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-16T17:17:32.891 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-16T17:17:32.891 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.891 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.891 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.891 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.892 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.892 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-16T17:17:32.892 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-16T17:17:32.892 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:32.892 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.892 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.892 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:32.892 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.892 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.892 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.893 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.893 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.893 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.893 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.893 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.893 INFO:teuthology.orchestra.run.smithi045.stdout: "roption1": { 2024-04-16T17:17:32.893 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "roption1", 2024-04-16T17:17:32.893 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.893 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.893 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.894 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:32.894 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.894 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.894 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.894 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.894 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.894 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.894 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.894 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.895 INFO:teuthology.orchestra.run.smithi045.stdout: "roption2": { 2024-04-16T17:17:32.895 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "roption2", 2024-04-16T17:17:32.895 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.895 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.895 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.895 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "xyz", 2024-04-16T17:17:32.895 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.895 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.895 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.896 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.896 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.896 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.896 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.896 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.896 INFO:teuthology.orchestra.run.smithi045.stdout: "rwoption1": { 2024-04-16T17:17:32.896 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rwoption1", 2024-04-16T17:17:32.896 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.896 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.896 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.897 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:32.897 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.897 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.897 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.897 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.897 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.897 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.897 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.897 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.897 INFO:teuthology.orchestra.run.smithi045.stdout: "rwoption2": { 2024-04-16T17:17:32.898 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rwoption2", 2024-04-16T17:17:32.898 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-16T17:17:32.898 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.898 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.898 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:32.898 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.898 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.898 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.898 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.899 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.899 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.899 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.899 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.899 INFO:teuthology.orchestra.run.smithi045.stdout: "rwoption3": { 2024-04-16T17:17:32.899 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rwoption3", 2024-04-16T17:17:32.899 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "float", 2024-04-16T17:17:32.899 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.899 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.899 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:32.900 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.900 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.900 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.900 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.900 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.900 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.900 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.900 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.901 INFO:teuthology.orchestra.run.smithi045.stdout: "rwoption4": { 2024-04-16T17:17:32.901 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rwoption4", 2024-04-16T17:17:32.901 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.901 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.901 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.901 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:32.901 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.901 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.901 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.901 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.902 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.902 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.902 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.902 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.902 INFO:teuthology.orchestra.run.smithi045.stdout: "rwoption5": { 2024-04-16T17:17:32.902 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rwoption5", 2024-04-16T17:17:32.902 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:32.902 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.902 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.903 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:32.903 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.903 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.903 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.903 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.903 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.903 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.903 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.903 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.903 INFO:teuthology.orchestra.run.smithi045.stdout: "rwoption6": { 2024-04-16T17:17:32.904 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rwoption6", 2024-04-16T17:17:32.904 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:32.904 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.904 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.904 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-16T17:17:32.904 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.904 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.904 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.904 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.905 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.905 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.905 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.905 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.905 INFO:teuthology.orchestra.run.smithi045.stdout: "rwoption7": { 2024-04-16T17:17:32.905 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rwoption7", 2024-04-16T17:17:32.905 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-16T17:17:32.905 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.905 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.905 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:32.906 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "1", 2024-04-16T17:17:32.906 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "42", 2024-04-16T17:17:32.906 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.906 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.906 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.906 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.906 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.906 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.907 INFO:teuthology.orchestra.run.smithi045.stdout: "testkey": { 2024-04-16T17:17:32.907 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "testkey", 2024-04-16T17:17:32.907 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.907 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.907 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.907 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:32.907 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.907 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.907 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.908 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.908 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.908 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.908 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.908 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.908 INFO:teuthology.orchestra.run.smithi045.stdout: "testlkey": { 2024-04-16T17:17:32.908 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "testlkey", 2024-04-16T17:17:32.908 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.909 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.909 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.909 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:32.909 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.909 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.909 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.909 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.909 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.909 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.910 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.910 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.910 INFO:teuthology.orchestra.run.smithi045.stdout: "testnewline": { 2024-04-16T17:17:32.910 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "testnewline", 2024-04-16T17:17:32.910 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.910 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.910 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.910 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:32.910 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.911 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.911 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.911 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.911 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.911 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.911 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.911 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:17:32.911 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:17:32.911 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.912 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-16T17:17:32.912 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "snap_schedule", 2024-04-16T17:17:32.912 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-16T17:17:32.912 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-16T17:17:32.912 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-16T17:17:32.912 INFO:teuthology.orchestra.run.smithi045.stdout: "allow_m_granularity": { 2024-04-16T17:17:32.912 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "allow_m_granularity", 2024-04-16T17:17:32.912 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:32.912 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.913 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.913 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:32.913 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.913 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.913 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.913 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "allow minute scheduled snapshots", 2024-04-16T17:17:32.913 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.913 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.913 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.913 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.914 INFO:teuthology.orchestra.run.smithi045.stdout: "dump_on_update": { 2024-04-16T17:17:32.914 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "dump_on_update", 2024-04-16T17:17:32.914 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:32.914 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.914 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.914 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:32.914 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.914 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.914 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.915 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "dump database to debug log on update", 2024-04-16T17:17:32.915 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.915 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.915 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.915 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.915 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-16T17:17:32.915 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-16T17:17:32.915 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.916 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.916 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.916 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:32.916 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.916 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.916 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-16T17:17:32.916 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-16T17:17:32.916 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-16T17:17:32.916 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-16T17:17:32.917 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-16T17:17:32.917 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-16T17:17:32.917 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-16T17:17:32.917 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-16T17:17:32.917 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.917 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.917 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.917 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.917 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.918 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-16T17:17:32.918 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-16T17:17:32.918 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:32.918 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.918 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.918 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:32.918 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.918 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.919 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.919 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.919 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.919 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.919 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.919 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.919 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-16T17:17:32.919 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-16T17:17:32.919 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.920 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.920 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.920 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-16T17:17:32.920 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.920 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.920 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-16T17:17:32.920 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-16T17:17:32.920 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-16T17:17:32.920 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-16T17:17:32.921 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-16T17:17:32.921 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-16T17:17:32.921 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-16T17:17:32.921 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-16T17:17:32.921 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.921 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.921 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.921 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.921 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.922 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-16T17:17:32.922 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-16T17:17:32.922 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:32.922 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.922 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.922 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:32.922 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.922 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.923 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.923 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.923 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.923 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.923 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.923 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:17:32.923 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:17:32.923 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.923 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-16T17:17:32.924 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "stats", 2024-04-16T17:17:32.924 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-16T17:17:32.924 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-16T17:17:32.924 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-16T17:17:32.924 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-16T17:17:32.924 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-16T17:17:32.924 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.924 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.924 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.925 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:32.925 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.925 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.925 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-16T17:17:32.925 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-16T17:17:32.925 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-16T17:17:32.925 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-16T17:17:32.925 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-16T17:17:32.925 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-16T17:17:32.925 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-16T17:17:32.926 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-16T17:17:32.926 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.926 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.926 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.926 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.926 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.926 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-16T17:17:32.926 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-16T17:17:32.927 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:32.927 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.927 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.927 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:32.927 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.927 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.927 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.927 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.927 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.928 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.928 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.928 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.928 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-16T17:17:32.928 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-16T17:17:32.928 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.928 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.928 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.928 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-16T17:17:32.929 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.929 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.929 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-16T17:17:32.929 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-16T17:17:32.929 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-16T17:17:32.929 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-16T17:17:32.929 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-16T17:17:32.929 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-16T17:17:32.929 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-16T17:17:32.930 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-16T17:17:32.930 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.930 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.930 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.930 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.930 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.930 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-16T17:17:32.930 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-16T17:17:32.930 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:32.931 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.931 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.931 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:32.931 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.931 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.931 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.931 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.931 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.932 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.932 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.932 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:17:32.932 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:17:32.932 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.932 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-16T17:17:32.932 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "status", 2024-04-16T17:17:32.932 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-16T17:17:32.932 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-16T17:17:32.932 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-16T17:17:32.933 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-16T17:17:32.933 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-16T17:17:32.933 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.933 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.933 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.933 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:32.933 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.933 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.933 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-16T17:17:32.933 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-16T17:17:32.934 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-16T17:17:32.934 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-16T17:17:32.934 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-16T17:17:32.934 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-16T17:17:32.934 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-16T17:17:32.934 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-16T17:17:32.934 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.934 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.934 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.935 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.935 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.935 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-16T17:17:32.935 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-16T17:17:32.935 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:32.935 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.935 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.935 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:32.935 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.935 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.936 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.936 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.936 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.936 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.936 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.936 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.936 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-16T17:17:32.936 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-16T17:17:32.936 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.936 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.937 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.937 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-16T17:17:32.937 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.937 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.937 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-16T17:17:32.937 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-16T17:17:32.937 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-16T17:17:32.937 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-16T17:17:32.937 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-16T17:17:32.938 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-16T17:17:32.938 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-16T17:17:32.938 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-16T17:17:32.938 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.938 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.938 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.938 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.938 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.938 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-16T17:17:32.939 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-16T17:17:32.939 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:32.939 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.939 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.939 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:32.939 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.939 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.939 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.939 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.940 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.940 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.940 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.940 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:17:32.940 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:17:32.940 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.940 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-16T17:17:32.940 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "telegraf", 2024-04-16T17:17:32.940 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-16T17:17:32.940 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-16T17:17:32.940 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-16T17:17:32.941 INFO:teuthology.orchestra.run.smithi045.stdout: "address": { 2024-04-16T17:17:32.941 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "address", 2024-04-16T17:17:32.941 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.941 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.941 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.941 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-16T17:17:32.941 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.941 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.941 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.942 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.942 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.942 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.942 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.942 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.942 INFO:teuthology.orchestra.run.smithi045.stdout: "interval": { 2024-04-16T17:17:32.942 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "interval", 2024-04-16T17:17:32.942 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-04-16T17:17:32.942 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.943 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.943 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "15", 2024-04-16T17:17:32.943 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.943 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.943 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.943 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.943 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.943 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.943 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.944 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.944 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-16T17:17:32.944 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-16T17:17:32.944 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.944 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.944 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.944 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:32.944 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.944 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.944 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-16T17:17:32.945 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-16T17:17:32.945 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-16T17:17:32.945 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-16T17:17:32.945 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-16T17:17:32.945 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-16T17:17:32.945 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-16T17:17:32.945 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-16T17:17:32.945 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.945 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.946 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.946 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.946 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.946 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-16T17:17:32.946 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-16T17:17:32.946 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:32.946 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.946 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.947 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:32.947 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.947 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.947 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.947 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.947 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.947 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.947 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.947 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.948 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-16T17:17:32.948 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-16T17:17:32.948 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.948 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.948 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.948 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-16T17:17:32.948 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.948 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.948 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-16T17:17:32.948 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-16T17:17:32.949 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-16T17:17:32.949 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-16T17:17:32.949 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-16T17:17:32.949 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-16T17:17:32.949 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-16T17:17:32.949 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-16T17:17:32.949 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.949 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.949 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.950 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.950 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.950 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-16T17:17:32.950 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-16T17:17:32.950 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:32.950 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.950 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.950 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:32.951 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.951 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.951 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.951 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.951 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.951 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.951 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.951 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:17:32.951 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:17:32.952 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.952 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-16T17:17:32.952 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "telemetry", 2024-04-16T17:17:32.952 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-16T17:17:32.952 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-16T17:17:32.952 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-16T17:17:32.952 INFO:teuthology.orchestra.run.smithi045.stdout: "channel_basic": { 2024-04-16T17:17:32.952 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "channel_basic", 2024-04-16T17:17:32.952 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:32.953 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.953 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.953 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-16T17:17:32.953 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.953 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.953 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.953 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-16T17:17:32.953 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.953 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.953 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.954 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.954 INFO:teuthology.orchestra.run.smithi045.stdout: "channel_crash": { 2024-04-16T17:17:32.954 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "channel_crash", 2024-04-16T17:17:32.954 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:32.954 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.954 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.954 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-16T17:17:32.954 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.955 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.955 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.955 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-16T17:17:32.955 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.955 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.955 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.955 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.955 INFO:teuthology.orchestra.run.smithi045.stdout: "channel_device": { 2024-04-16T17:17:32.956 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "channel_device", 2024-04-16T17:17:32.956 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:32.956 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.956 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.956 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-16T17:17:32.956 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.956 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.956 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.956 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-16T17:17:32.957 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.957 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.957 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.957 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.957 INFO:teuthology.orchestra.run.smithi045.stdout: "channel_ident": { 2024-04-16T17:17:32.957 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "channel_ident", 2024-04-16T17:17:32.957 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:32.957 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.957 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.958 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:32.958 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.958 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.958 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.958 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-16T17:17:32.958 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.958 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.958 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.958 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.959 INFO:teuthology.orchestra.run.smithi045.stdout: "channel_perf": { 2024-04-16T17:17:32.959 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "channel_perf", 2024-04-16T17:17:32.959 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:32.959 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.959 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.959 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:32.959 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.959 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.959 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.960 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-16T17:17:32.960 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.960 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.960 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.960 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.960 INFO:teuthology.orchestra.run.smithi045.stdout: "contact": { 2024-04-16T17:17:32.960 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "contact", 2024-04-16T17:17:32.960 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.960 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.960 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.961 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:32.961 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.961 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.961 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.961 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.961 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.961 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.961 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.961 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.962 INFO:teuthology.orchestra.run.smithi045.stdout: "description": { 2024-04-16T17:17:32.962 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "description", 2024-04-16T17:17:32.962 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.962 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.962 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.962 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:32.962 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.962 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.962 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.962 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.963 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.963 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.963 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.963 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.963 INFO:teuthology.orchestra.run.smithi045.stdout: "device_url": { 2024-04-16T17:17:32.963 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "device_url", 2024-04-16T17:17:32.963 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.963 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.963 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.964 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-16T17:17:32.964 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.964 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.964 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.964 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.964 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.964 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.964 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.964 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.964 INFO:teuthology.orchestra.run.smithi045.stdout: "enabled": { 2024-04-16T17:17:32.965 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "enabled", 2024-04-16T17:17:32.965 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:32.965 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.965 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.965 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:32.965 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.965 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.965 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.965 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.965 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.966 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.966 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.966 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.966 INFO:teuthology.orchestra.run.smithi045.stdout: "interval": { 2024-04-16T17:17:32.966 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "interval", 2024-04-16T17:17:32.966 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-16T17:17:32.966 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.966 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.966 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "24", 2024-04-16T17:17:32.967 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "8", 2024-04-16T17:17:32.967 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.967 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.967 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.967 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.967 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.967 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.967 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.967 INFO:teuthology.orchestra.run.smithi045.stdout: "last_opt_revision": { 2024-04-16T17:17:32.967 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "last_opt_revision", 2024-04-16T17:17:32.968 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-16T17:17:32.968 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.968 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.968 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "1", 2024-04-16T17:17:32.968 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.968 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.968 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.968 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.968 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.968 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.968 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.969 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.969 INFO:teuthology.orchestra.run.smithi045.stdout: "leaderboard": { 2024-04-16T17:17:32.969 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "leaderboard", 2024-04-16T17:17:32.969 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:32.969 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.969 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.969 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:32.969 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.970 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.970 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.970 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.970 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.970 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.970 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.970 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.970 INFO:teuthology.orchestra.run.smithi045.stdout: "leaderboard_description": { 2024-04-16T17:17:32.970 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "leaderboard_description", 2024-04-16T17:17:32.971 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.971 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.971 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.971 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:32.971 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.971 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.971 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.971 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.971 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.972 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.972 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.972 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.972 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-16T17:17:32.972 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-16T17:17:32.972 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.972 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.972 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.972 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:32.972 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.973 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.973 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-16T17:17:32.973 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-16T17:17:32.973 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-16T17:17:32.973 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-16T17:17:32.973 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-16T17:17:32.973 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-16T17:17:32.973 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-16T17:17:32.973 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-16T17:17:32.973 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.974 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.974 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.974 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.974 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.974 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-16T17:17:32.974 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-16T17:17:32.974 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:32.974 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.974 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.975 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:32.975 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.975 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.975 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.975 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.975 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.975 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.975 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.975 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.976 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-16T17:17:32.976 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-16T17:17:32.976 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.976 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.976 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.976 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-16T17:17:32.976 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.976 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.976 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-16T17:17:32.976 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-16T17:17:32.977 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-16T17:17:32.977 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-16T17:17:32.977 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-16T17:17:32.977 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-16T17:17:32.977 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-16T17:17:32.977 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-16T17:17:32.977 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.977 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.977 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.977 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.978 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.978 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-16T17:17:32.978 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-16T17:17:32.978 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:32.978 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.978 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.978 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:32.978 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.978 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.979 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.979 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.979 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.979 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.979 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.979 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.979 INFO:teuthology.orchestra.run.smithi045.stdout: "organization": { 2024-04-16T17:17:32.979 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "organization", 2024-04-16T17:17:32.979 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.979 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.980 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.980 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:32.980 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.980 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.980 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.980 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.980 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.980 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.980 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.980 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.981 INFO:teuthology.orchestra.run.smithi045.stdout: "proxy": { 2024-04-16T17:17:32.981 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "proxy", 2024-04-16T17:17:32.981 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.981 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.981 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.981 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:32.981 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.981 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.982 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.982 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.982 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.982 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.982 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.982 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.982 INFO:teuthology.orchestra.run.smithi045.stdout: "url": { 2024-04-16T17:17:32.982 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "url", 2024-04-16T17:17:32.982 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.982 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.983 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:32.983 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-16T17:17:32.983 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.983 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.983 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.983 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.983 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.983 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.983 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.984 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:17:32.984 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:17:32.984 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.984 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-16T17:17:32.984 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "test_orchestrator", 2024-04-16T17:17:32.984 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-16T17:17:32.984 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-16T17:17:32.984 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-16T17:17:32.984 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-16T17:17:32.984 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-16T17:17:32.985 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.985 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.985 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.985 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:32.985 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.985 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.985 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-16T17:17:32.985 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-16T17:17:32.985 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-16T17:17:32.985 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-16T17:17:32.986 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-16T17:17:32.986 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-16T17:17:32.986 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-16T17:17:32.986 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-16T17:17:32.986 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.986 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.986 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.986 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.986 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.986 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-16T17:17:32.987 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-16T17:17:32.987 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:32.987 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.987 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.987 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:32.987 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.987 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.987 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.987 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.988 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.988 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.988 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.988 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.988 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-16T17:17:32.988 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-16T17:17:32.988 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.988 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.988 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.988 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-16T17:17:32.989 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.989 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.989 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-16T17:17:32.989 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-16T17:17:32.989 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-16T17:17:32.989 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-16T17:17:32.989 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-16T17:17:32.989 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-16T17:17:32.990 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-16T17:17:32.990 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-16T17:17:32.990 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.990 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.990 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.990 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.990 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.990 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-16T17:17:32.990 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-16T17:17:32.990 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:32.991 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.991 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.991 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:32.991 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.991 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.991 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.991 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.991 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.991 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.991 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.992 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:17:32.992 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:17:32.992 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.992 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-16T17:17:32.992 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "volumes", 2024-04-16T17:17:32.992 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-16T17:17:32.992 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-16T17:17:32.992 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-16T17:17:32.992 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-16T17:17:32.993 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-16T17:17:32.993 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.993 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.993 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.993 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:32.993 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.993 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.993 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-16T17:17:32.993 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-16T17:17:32.994 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-16T17:17:32.994 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-16T17:17:32.994 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-16T17:17:32.994 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-16T17:17:32.994 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-16T17:17:32.994 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-16T17:17:32.994 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.994 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.994 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.994 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.995 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.995 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-16T17:17:32.995 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-16T17:17:32.995 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:32.995 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.995 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.995 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:32.995 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.995 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.996 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.996 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.996 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.996 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.996 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.996 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.996 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-16T17:17:32.996 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-16T17:17:32.996 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:32.997 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.997 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.997 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-16T17:17:32.997 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.997 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.997 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-16T17:17:32.997 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-16T17:17:32.997 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-16T17:17:32.997 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-16T17:17:32.997 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-16T17:17:32.998 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-16T17:17:32.998 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-16T17:17:32.998 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-16T17:17:32.998 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:32.998 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:32.998 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:32.998 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:32.998 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:32.998 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-16T17:17:32.999 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-16T17:17:32.999 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:32.999 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:32.999 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:32.999 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:32.999 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:32.999 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:32.999 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:32.999 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.000 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.000 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.000 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.000 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.000 INFO:teuthology.orchestra.run.smithi045.stdout: "max_concurrent_clones": { 2024-04-16T17:17:33.000 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "max_concurrent_clones", 2024-04-16T17:17:33.000 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-16T17:17:33.000 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.000 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:33.000 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "4", 2024-04-16T17:17:33.001 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.001 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.001 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.001 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-16T17:17:33.001 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.001 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.001 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.001 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.001 INFO:teuthology.orchestra.run.smithi045.stdout: "periodic_async_work": { 2024-04-16T17:17:33.001 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "periodic_async_work", 2024-04-16T17:17:33.002 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:33.002 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.002 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:33.002 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:33.002 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.002 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.002 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.002 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Periodically check for async work", 2024-04-16T17:17:33.002 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.003 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.003 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.003 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.003 INFO:teuthology.orchestra.run.smithi045.stdout: "snapshot_clone_delay": { 2024-04-16T17:17:33.003 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "snapshot_clone_delay", 2024-04-16T17:17:33.003 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-16T17:17:33.003 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.003 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:33.004 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0", 2024-04-16T17:17:33.004 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.004 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.004 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.004 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-16T17:17:33.004 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.004 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.004 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.004 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.005 INFO:teuthology.orchestra.run.smithi045.stdout: "snapshot_clone_no_wait": { 2024-04-16T17:17:33.005 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "snapshot_clone_no_wait", 2024-04-16T17:17:33.005 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:33.005 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.005 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:33.005 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-16T17:17:33.005 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.005 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.005 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.005 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-16T17:17:33.006 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.006 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.006 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.006 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:17:33.006 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:17:33.006 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.006 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-16T17:17:33.006 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "zabbix", 2024-04-16T17:17:33.006 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-16T17:17:33.007 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-16T17:17:33.007 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-16T17:17:33.007 INFO:teuthology.orchestra.run.smithi045.stdout: "discovery_interval": { 2024-04-16T17:17:33.007 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "discovery_interval", 2024-04-16T17:17:33.007 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "uint", 2024-04-16T17:17:33.007 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.007 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:33.007 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "100", 2024-04-16T17:17:33.007 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.008 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.008 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.008 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.008 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.008 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.008 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.008 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.008 INFO:teuthology.orchestra.run.smithi045.stdout: "identifier": { 2024-04-16T17:17:33.008 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "identifier", 2024-04-16T17:17:33.009 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:33.009 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.009 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:33.009 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:33.009 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.009 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.009 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.009 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.009 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.009 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.010 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.010 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.010 INFO:teuthology.orchestra.run.smithi045.stdout: "interval": { 2024-04-16T17:17:33.010 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "interval", 2024-04-16T17:17:33.010 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-04-16T17:17:33.010 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.010 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:33.010 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "60", 2024-04-16T17:17:33.010 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.010 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.011 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.011 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.011 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.011 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.011 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.011 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.011 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-16T17:17:33.011 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-16T17:17:33.011 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:33.011 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.012 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:33.012 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:33.012 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.012 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.012 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-16T17:17:33.012 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-16T17:17:33.012 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-16T17:17:33.012 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-16T17:17:33.012 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-16T17:17:33.013 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-16T17:17:33.013 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-16T17:17:33.013 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-16T17:17:33.013 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.013 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.013 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.013 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.013 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.013 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-16T17:17:33.013 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-16T17:17:33.014 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:33.014 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.014 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:33.014 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:33.014 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.014 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.014 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.014 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.014 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.015 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.015 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.015 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.015 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-16T17:17:33.015 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-16T17:17:33.015 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:33.015 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.015 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:33.015 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-16T17:17:33.016 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.016 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.016 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-16T17:17:33.016 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-16T17:17:33.016 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-16T17:17:33.016 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-16T17:17:33.016 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-16T17:17:33.016 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-16T17:17:33.016 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-16T17:17:33.017 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-16T17:17:33.017 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.017 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.017 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.017 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.017 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.017 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-16T17:17:33.017 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-16T17:17:33.017 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:33.017 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.018 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:33.018 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:33.018 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.018 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.018 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.018 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.018 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.018 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.018 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.019 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.019 INFO:teuthology.orchestra.run.smithi045.stdout: "zabbix_host": { 2024-04-16T17:17:33.019 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "zabbix_host", 2024-04-16T17:17:33.019 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:33.019 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.019 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:33.019 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:33.019 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.019 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.020 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.020 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.020 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.020 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.020 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.020 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.020 INFO:teuthology.orchestra.run.smithi045.stdout: "zabbix_port": { 2024-04-16T17:17:33.020 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "zabbix_port", 2024-04-16T17:17:33.020 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-16T17:17:33.020 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.021 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:33.021 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "10051", 2024-04-16T17:17:33.021 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.021 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.021 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.021 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.021 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.021 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.021 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.022 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.022 INFO:teuthology.orchestra.run.smithi045.stdout: "zabbix_sender": { 2024-04-16T17:17:33.022 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "zabbix_sender", 2024-04-16T17:17:33.022 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:33.022 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.022 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:33.022 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-16T17:17:33.022 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.022 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.022 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.023 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.023 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.023 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.023 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.023 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:17:33.023 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:17:33.023 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:17:33.023 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-04-16T17:17:33.023 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:17:33.023 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-16T17:17:33.024 INFO:teuthology.orchestra.run.smithi045.stdout: "modules": [ 2024-04-16T17:17:33.024 INFO:teuthology.orchestra.run.smithi045.stdout: "cephadm", 2024-04-16T17:17:33.024 INFO:teuthology.orchestra.run.smithi045.stdout: "dashboard", 2024-04-16T17:17:33.024 INFO:teuthology.orchestra.run.smithi045.stdout: "iostat", 2024-04-16T17:17:33.024 INFO:teuthology.orchestra.run.smithi045.stdout: "nfs", 2024-04-16T17:17:33.024 INFO:teuthology.orchestra.run.smithi045.stdout: "restful", 2024-04-16T17:17:33.024 INFO:teuthology.orchestra.run.smithi045.stdout: "snap_schedule" 2024-04-16T17:17:33.024 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-16T17:17:33.024 INFO:teuthology.orchestra.run.smithi045.stdout: "available_modules": [ 2024-04-16T17:17:33.025 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-16T17:17:33.025 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "alerts", 2024-04-16T17:17:33.025 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-16T17:17:33.025 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-16T17:17:33.025 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-16T17:17:33.025 INFO:teuthology.orchestra.run.smithi045.stdout: "interval": { 2024-04-16T17:17:33.025 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "interval", 2024-04-16T17:17:33.025 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-04-16T17:17:33.025 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.025 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:33.026 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "60", 2024-04-16T17:17:33.026 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.026 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.026 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.026 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "How frequently to reexamine health status", 2024-04-16T17:17:33.026 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.026 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.026 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.026 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.026 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-16T17:17:33.027 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-16T17:17:33.027 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:33.027 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.027 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:33.027 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:33.027 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.027 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.027 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-16T17:17:33.027 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-16T17:17:33.028 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-16T17:17:33.028 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-16T17:17:33.028 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-16T17:17:33.028 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-16T17:17:33.028 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-16T17:17:33.028 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-16T17:17:33.028 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.028 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.028 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.028 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.029 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.029 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-16T17:17:33.029 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-16T17:17:33.029 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:33.029 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.029 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:33.029 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:33.029 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.029 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.030 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.030 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.030 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.030 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.030 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.030 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.030 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-16T17:17:33.030 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-16T17:17:33.030 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:33.030 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.031 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:33.031 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-16T17:17:33.031 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.031 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.031 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-16T17:17:33.031 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-16T17:17:33.031 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-16T17:17:33.031 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-16T17:17:33.031 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-16T17:17:33.032 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-16T17:17:33.032 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-16T17:17:33.032 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-16T17:17:33.032 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.032 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.032 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.032 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.032 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.032 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-16T17:17:33.033 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-16T17:17:33.033 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:33.033 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.033 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:33.033 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:33.033 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.033 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.033 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.033 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.033 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.034 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.034 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.034 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.034 INFO:teuthology.orchestra.run.smithi045.stdout: "smtp_destination": { 2024-04-16T17:17:33.034 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "smtp_destination", 2024-04-16T17:17:33.034 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:33.034 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.034 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:33.034 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:33.034 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.035 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.035 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.035 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Email address to send alerts to", 2024-04-16T17:17:33.035 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.035 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.035 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.035 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.035 INFO:teuthology.orchestra.run.smithi045.stdout: "smtp_from_name": { 2024-04-16T17:17:33.035 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "smtp_from_name", 2024-04-16T17:17:33.035 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:33.036 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.036 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:33.036 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "Ceph", 2024-04-16T17:17:33.036 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.036 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.036 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.036 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Email From: name", 2024-04-16T17:17:33.036 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.036 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.036 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.037 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.037 INFO:teuthology.orchestra.run.smithi045.stdout: "smtp_host": { 2024-04-16T17:17:33.037 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "smtp_host", 2024-04-16T17:17:33.037 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:33.037 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.037 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:33.037 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:33.037 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.037 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.038 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.038 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "SMTP server", 2024-04-16T17:17:33.038 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.038 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.038 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.038 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.038 INFO:teuthology.orchestra.run.smithi045.stdout: "smtp_password": { 2024-04-16T17:17:33.038 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "smtp_password", 2024-04-16T17:17:33.038 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:33.039 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.039 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:33.039 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:33.039 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.039 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.039 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.039 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Password to authenticate with", 2024-04-16T17:17:33.039 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.039 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.040 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.040 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.040 INFO:teuthology.orchestra.run.smithi045.stdout: "smtp_port": { 2024-04-16T17:17:33.040 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "smtp_port", 2024-04-16T17:17:33.040 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-16T17:17:33.040 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.040 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:33.040 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "465", 2024-04-16T17:17:33.040 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.040 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.041 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.041 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "SMTP port", 2024-04-16T17:17:33.041 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.041 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.041 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.041 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.041 INFO:teuthology.orchestra.run.smithi045.stdout: "smtp_sender": { 2024-04-16T17:17:33.041 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "smtp_sender", 2024-04-16T17:17:33.041 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:33.042 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.042 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:33.042 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:33.042 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.042 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.042 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.042 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "SMTP envelope sender", 2024-04-16T17:17:33.042 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.042 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.043 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.043 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.043 INFO:teuthology.orchestra.run.smithi045.stdout: "smtp_ssl": { 2024-04-16T17:17:33.043 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "smtp_ssl", 2024-04-16T17:17:33.043 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:33.043 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.043 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:33.043 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-16T17:17:33.043 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.043 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.043 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.044 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-16T17:17:33.044 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.044 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.044 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.044 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.044 INFO:teuthology.orchestra.run.smithi045.stdout: "smtp_user": { 2024-04-16T17:17:33.044 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "smtp_user", 2024-04-16T17:17:33.044 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:33.044 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.045 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:33.045 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:33.045 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.045 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.045 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.045 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "User to authenticate as", 2024-04-16T17:17:33.045 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.045 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.045 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.045 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:17:33.046 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:17:33.046 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.046 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-16T17:17:33.046 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "balancer", 2024-04-16T17:17:33.046 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-16T17:17:33.046 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-16T17:17:33.046 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-16T17:17:33.046 INFO:teuthology.orchestra.run.smithi045.stdout: "active": { 2024-04-16T17:17:33.046 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "active", 2024-04-16T17:17:33.047 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:33.047 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.047 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:33.047 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-16T17:17:33.047 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.047 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.047 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.047 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "automatically balance PGs across cluster", 2024-04-16T17:17:33.047 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.047 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.048 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.048 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.048 INFO:teuthology.orchestra.run.smithi045.stdout: "begin_time": { 2024-04-16T17:17:33.048 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "begin_time", 2024-04-16T17:17:33.048 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:33.048 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.048 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:33.048 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0000", 2024-04-16T17:17:33.048 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.048 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.049 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.049 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "beginning time of day to automatically balance", 2024-04-16T17:17:33.049 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-16T17:17:33.049 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.049 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.049 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.049 INFO:teuthology.orchestra.run.smithi045.stdout: "begin_weekday": { 2024-04-16T17:17:33.049 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "begin_weekday", 2024-04-16T17:17:33.050 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "uint", 2024-04-16T17:17:33.050 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.050 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:33.050 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0", 2024-04-16T17:17:33.050 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "0", 2024-04-16T17:17:33.050 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "6", 2024-04-16T17:17:33.050 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.050 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-16T17:17:33.050 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-16T17:17:33.050 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.051 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.051 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.051 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_compat_max_iterations": { 2024-04-16T17:17:33.051 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "crush_compat_max_iterations", 2024-04-16T17:17:33.051 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "uint", 2024-04-16T17:17:33.051 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.051 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:33.051 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "25", 2024-04-16T17:17:33.051 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "1", 2024-04-16T17:17:33.052 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "250", 2024-04-16T17:17:33.052 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.052 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-16T17:17:33.052 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.052 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.052 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.052 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.052 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_compat_metrics": { 2024-04-16T17:17:33.052 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "crush_compat_metrics", 2024-04-16T17:17:33.052 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:33.053 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.053 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:33.053 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "pgs,objects,bytes", 2024-04-16T17:17:33.053 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.053 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.053 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.053 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-16T17:17:33.053 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-16T17:17:33.053 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.054 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.054 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.054 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_compat_step": { 2024-04-16T17:17:33.054 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "crush_compat_step", 2024-04-16T17:17:33.054 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "float", 2024-04-16T17:17:33.054 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.054 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:33.054 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0.5", 2024-04-16T17:17:33.054 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "0.001", 2024-04-16T17:17:33.055 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "0.999", 2024-04-16T17:17:33.055 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.055 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "aggressiveness of optimization", 2024-04-16T17:17:33.055 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-16T17:17:33.055 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.055 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.055 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.055 INFO:teuthology.orchestra.run.smithi045.stdout: "end_time": { 2024-04-16T17:17:33.055 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "end_time", 2024-04-16T17:17:33.056 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:33.056 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.056 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:33.056 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "2359", 2024-04-16T17:17:33.056 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.056 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.056 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.056 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "ending time of day to automatically balance", 2024-04-16T17:17:33.056 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-16T17:17:33.057 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.057 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.057 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.057 INFO:teuthology.orchestra.run.smithi045.stdout: "end_weekday": { 2024-04-16T17:17:33.057 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "end_weekday", 2024-04-16T17:17:33.057 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "uint", 2024-04-16T17:17:33.057 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.057 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:33.057 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0", 2024-04-16T17:17:33.057 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "0", 2024-04-16T17:17:33.058 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "6", 2024-04-16T17:17:33.058 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.058 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-16T17:17:33.058 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-16T17:17:33.058 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.058 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.058 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.058 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-16T17:17:33.058 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-16T17:17:33.058 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:33.059 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.059 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:33.059 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:33.059 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.059 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.059 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-16T17:17:33.059 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-16T17:17:33.059 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-16T17:17:33.060 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-16T17:17:33.060 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-16T17:17:33.060 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-16T17:17:33.060 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-16T17:17:33.060 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-16T17:17:33.060 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.060 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.060 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.060 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.060 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.061 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-16T17:17:33.061 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-16T17:17:33.061 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:33.061 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.061 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:33.061 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:33.061 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.061 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.061 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.061 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.062 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.062 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.062 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.062 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.062 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-16T17:17:33.062 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-16T17:17:33.062 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:33.062 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.062 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:33.063 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-16T17:17:33.063 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.063 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.063 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-16T17:17:33.063 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-16T17:17:33.063 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-16T17:17:33.063 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-16T17:17:33.063 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-16T17:17:33.063 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-16T17:17:33.063 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-16T17:17:33.064 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-16T17:17:33.064 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.064 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.064 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.064 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.064 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.064 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-16T17:17:33.064 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-16T17:17:33.064 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:33.064 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.065 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:33.065 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:33.065 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.065 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.065 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.065 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.065 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.065 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.065 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.065 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.066 INFO:teuthology.orchestra.run.smithi045.stdout: "min_score": { 2024-04-16T17:17:33.066 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "min_score", 2024-04-16T17:17:33.066 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "float", 2024-04-16T17:17:33.066 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.066 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:33.066 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0", 2024-04-16T17:17:33.066 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.066 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.066 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.067 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-16T17:17:33.067 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.067 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.067 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.067 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.067 INFO:teuthology.orchestra.run.smithi045.stdout: "mode": { 2024-04-16T17:17:33.067 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "mode", 2024-04-16T17:17:33.067 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:33.067 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.067 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:33.068 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "upmap", 2024-04-16T17:17:33.068 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.068 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.068 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-16T17:17:33.068 INFO:teuthology.orchestra.run.smithi045.stdout: "crush-compat", 2024-04-16T17:17:33.068 INFO:teuthology.orchestra.run.smithi045.stdout: "none", 2024-04-16T17:17:33.068 INFO:teuthology.orchestra.run.smithi045.stdout: "read", 2024-04-16T17:17:33.068 INFO:teuthology.orchestra.run.smithi045.stdout: "upmap", 2024-04-16T17:17:33.068 INFO:teuthology.orchestra.run.smithi045.stdout: "upmap-read" 2024-04-16T17:17:33.068 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-16T17:17:33.069 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Balancer mode", 2024-04-16T17:17:33.069 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.069 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.069 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.069 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.069 INFO:teuthology.orchestra.run.smithi045.stdout: "pool_ids": { 2024-04-16T17:17:33.069 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "pool_ids", 2024-04-16T17:17:33.069 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:33.069 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.070 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:33.070 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:33.070 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.070 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.070 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.070 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-16T17:17:33.070 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.070 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.070 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.071 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.071 INFO:teuthology.orchestra.run.smithi045.stdout: "sleep_interval": { 2024-04-16T17:17:33.071 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "sleep_interval", 2024-04-16T17:17:33.071 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-04-16T17:17:33.071 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.071 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:33.071 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "60", 2024-04-16T17:17:33.071 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.071 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.072 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.072 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-16T17:17:33.072 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.072 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.072 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.072 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.072 INFO:teuthology.orchestra.run.smithi045.stdout: "upmap_max_deviation": { 2024-04-16T17:17:33.072 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "upmap_max_deviation", 2024-04-16T17:17:33.072 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-16T17:17:33.073 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.073 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:33.073 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "5", 2024-04-16T17:17:33.073 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "1", 2024-04-16T17:17:33.073 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.073 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.073 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-16T17:17:33.073 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-16T17:17:33.073 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.073 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.074 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.074 INFO:teuthology.orchestra.run.smithi045.stdout: "upmap_max_optimizations": { 2024-04-16T17:17:33.074 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "upmap_max_optimizations", 2024-04-16T17:17:33.074 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "uint", 2024-04-16T17:17:33.074 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.074 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:33.074 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "10", 2024-04-16T17:17:33.074 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.074 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.075 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.075 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-16T17:17:33.075 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.075 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.075 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.075 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:17:33.075 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:17:33.075 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.075 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-16T17:17:33.075 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "cephadm", 2024-04-16T17:17:33.076 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-16T17:17:33.076 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-16T17:17:33.076 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-16T17:17:33.076 INFO:teuthology.orchestra.run.smithi045.stdout: "agent_down_multiplier": { 2024-04-16T17:17:33.076 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "agent_down_multiplier", 2024-04-16T17:17:33.076 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "float", 2024-04-16T17:17:33.076 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.076 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:33.076 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "3.0", 2024-04-16T17:17:33.077 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.077 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.077 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.077 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-16T17:17:33.077 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.077 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.077 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.077 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.077 INFO:teuthology.orchestra.run.smithi045.stdout: "agent_refresh_rate": { 2024-04-16T17:17:33.077 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "agent_refresh_rate", 2024-04-16T17:17:33.078 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-04-16T17:17:33.078 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.078 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:33.078 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "20", 2024-04-16T17:17:33.078 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.078 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.078 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.078 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-16T17:17:33.078 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.078 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.079 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.079 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.079 INFO:teuthology.orchestra.run.smithi045.stdout: "agent_starting_port": { 2024-04-16T17:17:33.079 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "agent_starting_port", 2024-04-16T17:17:33.079 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-16T17:17:33.079 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.079 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:33.079 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "4721", 2024-04-16T17:17:33.079 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.079 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.080 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.080 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-16T17:17:33.080 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.080 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.080 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.080 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.080 INFO:teuthology.orchestra.run.smithi045.stdout: "allow_ptrace": { 2024-04-16T17:17:33.080 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "allow_ptrace", 2024-04-16T17:17:33.080 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:33.081 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.081 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:33.081 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:33.081 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.081 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.081 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.081 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-16T17:17:33.081 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-16T17:17:33.081 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.082 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.082 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.082 INFO:teuthology.orchestra.run.smithi045.stdout: "autotune_interval": { 2024-04-16T17:17:33.082 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "autotune_interval", 2024-04-16T17:17:33.082 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-04-16T17:17:33.082 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.082 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:33.082 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "600", 2024-04-16T17:17:33.082 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.083 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.083 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.083 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-16T17:17:33.083 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.083 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.083 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.083 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.083 INFO:teuthology.orchestra.run.smithi045.stdout: "autotune_memory_target_ratio": { 2024-04-16T17:17:33.083 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "autotune_memory_target_ratio", 2024-04-16T17:17:33.084 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "float", 2024-04-16T17:17:33.084 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.084 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:33.084 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0.7", 2024-04-16T17:17:33.084 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.084 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.084 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.084 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-16T17:17:33.084 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.084 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.085 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.085 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.085 INFO:teuthology.orchestra.run.smithi045.stdout: "cephadm_log_destination": { 2024-04-16T17:17:33.085 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "cephadm_log_destination", 2024-04-16T17:17:33.085 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:33.085 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.085 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:33.085 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:33.085 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.085 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.086 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-16T17:17:33.086 INFO:teuthology.orchestra.run.smithi045.stdout: "file", 2024-04-16T17:17:33.086 INFO:teuthology.orchestra.run.smithi045.stdout: "file,syslog", 2024-04-16T17:17:33.086 INFO:teuthology.orchestra.run.smithi045.stdout: "syslog" 2024-04-16T17:17:33.086 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-16T17:17:33.086 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-16T17:17:33.086 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.086 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.087 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.087 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.087 INFO:teuthology.orchestra.run.smithi045.stdout: "cgroups_split": { 2024-04-16T17:17:33.087 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "cgroups_split", 2024-04-16T17:17:33.087 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:33.087 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.087 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:33.087 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-16T17:17:33.087 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.088 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.088 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.088 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-16T17:17:33.088 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.088 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.088 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.088 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.088 INFO:teuthology.orchestra.run.smithi045.stdout: "config_checks_enabled": { 2024-04-16T17:17:33.088 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "config_checks_enabled", 2024-04-16T17:17:33.089 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:33.089 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.089 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:33.089 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:33.089 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.089 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.089 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.089 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-16T17:17:33.089 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.090 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.090 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.090 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.090 INFO:teuthology.orchestra.run.smithi045.stdout: "config_dashboard": { 2024-04-16T17:17:33.090 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "config_dashboard", 2024-04-16T17:17:33.090 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:33.090 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.090 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:33.090 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-16T17:17:33.091 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.091 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.091 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.091 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-16T17:17:33.091 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.091 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.091 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.091 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.091 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_alertmanager": { 2024-04-16T17:17:33.092 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_alertmanager", 2024-04-16T17:17:33.092 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:33.092 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.092 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:33.092 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-16T17:17:33.092 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.092 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.092 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.092 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Prometheus container image", 2024-04-16T17:17:33.092 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.093 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.093 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.093 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.093 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_base": { 2024-04-16T17:17:33.093 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_base", 2024-04-16T17:17:33.093 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:33.093 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.093 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:33.093 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-16T17:17:33.093 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.094 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.094 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.094 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Container image name, without the tag", 2024-04-16T17:17:33.094 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.094 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.094 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.094 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.094 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_elasticsearch": { 2024-04-16T17:17:33.094 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_elasticsearch", 2024-04-16T17:17:33.095 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:33.095 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.095 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:33.095 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-16T17:17:33.095 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.095 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.095 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.095 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "elasticsearch container image", 2024-04-16T17:17:33.095 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.096 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.096 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.096 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.096 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_grafana": { 2024-04-16T17:17:33.096 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_grafana", 2024-04-16T17:17:33.096 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:33.096 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.096 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:33.096 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-16T17:17:33.097 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.097 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.097 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.097 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Prometheus container image", 2024-04-16T17:17:33.097 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.097 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.097 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.097 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.097 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_haproxy": { 2024-04-16T17:17:33.097 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_haproxy", 2024-04-16T17:17:33.098 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:33.098 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.098 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:33.098 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-16T17:17:33.098 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.098 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.098 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.098 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "HAproxy container image", 2024-04-16T17:17:33.098 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.099 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.099 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.099 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.099 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_jaeger_agent": { 2024-04-16T17:17:33.099 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_jaeger_agent", 2024-04-16T17:17:33.099 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:33.099 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.099 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:33.099 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-16T17:17:33.099 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.100 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.100 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.100 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Jaeger agent container image", 2024-04-16T17:17:33.100 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.100 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.100 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.100 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.100 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_jaeger_collector": { 2024-04-16T17:17:33.100 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_jaeger_collector", 2024-04-16T17:17:33.101 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:33.101 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.101 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:33.101 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-16T17:17:33.101 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.101 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.101 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.101 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Jaeger collector container image", 2024-04-16T17:17:33.101 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.101 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.102 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.102 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.102 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_jaeger_query": { 2024-04-16T17:17:33.102 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_jaeger_query", 2024-04-16T17:17:33.102 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:33.102 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.102 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:33.102 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-16T17:17:33.102 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.102 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.103 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.103 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Jaeger query container image", 2024-04-16T17:17:33.103 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.103 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.103 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.103 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.103 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_keepalived": { 2024-04-16T17:17:33.103 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_keepalived", 2024-04-16T17:17:33.103 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:33.103 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.104 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:33.104 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-16T17:17:33.104 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.104 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.104 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.104 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Keepalived container image", 2024-04-16T17:17:33.104 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.104 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.104 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.104 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.105 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_loki": { 2024-04-16T17:17:33.105 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_loki", 2024-04-16T17:17:33.105 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:33.105 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.105 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:33.105 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-16T17:17:33.105 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.105 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.105 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.106 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Loki container image", 2024-04-16T17:17:33.106 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.106 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.106 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.106 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.106 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_node_exporter": { 2024-04-16T17:17:33.106 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_node_exporter", 2024-04-16T17:17:33.106 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:33.106 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.107 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:33.107 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-16T17:17:33.107 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.107 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.107 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.107 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Prometheus container image", 2024-04-16T17:17:33.107 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.107 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.107 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.108 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.108 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_nvmeof": { 2024-04-16T17:17:33.108 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_nvmeof", 2024-04-16T17:17:33.108 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:33.108 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.108 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:33.108 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-04-16T17:17:33.108 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.108 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.109 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.109 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Nvme-of container image", 2024-04-16T17:17:33.109 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.109 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.109 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.109 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.109 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_prometheus": { 2024-04-16T17:17:33.109 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_prometheus", 2024-04-16T17:17:33.109 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:33.110 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.110 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:33.110 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-16T17:17:33.110 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.110 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.110 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.110 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Prometheus container image", 2024-04-16T17:17:33.110 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.110 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.110 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.111 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.111 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_promtail": { 2024-04-16T17:17:33.111 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_promtail", 2024-04-16T17:17:33.111 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:33.111 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.111 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:33.111 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-16T17:17:33.111 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.112 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.112 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.112 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Promtail container image", 2024-04-16T17:17:33.112 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.112 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.112 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.112 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.112 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_samba": { 2024-04-16T17:17:33.112 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_samba", 2024-04-16T17:17:33.112 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:33.113 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.113 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:33.113 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-16T17:17:33.113 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.113 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.113 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.113 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Samba/SMB container image", 2024-04-16T17:17:33.113 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.113 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.114 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.114 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.114 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_snmp_gateway": { 2024-04-16T17:17:33.114 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_snmp_gateway", 2024-04-16T17:17:33.114 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:33.114 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.114 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:33.114 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-16T17:17:33.114 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.115 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.115 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.115 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "SNMP Gateway container image", 2024-04-16T17:17:33.115 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.115 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.115 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.115 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.115 INFO:teuthology.orchestra.run.smithi045.stdout: "container_init": { 2024-04-16T17:17:33.115 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_init", 2024-04-16T17:17:33.115 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:33.116 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.116 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:33.116 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-16T17:17:33.116 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.116 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.116 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.116 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Run podman/docker with `--init`", 2024-04-16T17:17:33.116 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.116 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.116 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.117 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.117 INFO:teuthology.orchestra.run.smithi045.stdout: "daemon_cache_timeout": { 2024-04-16T17:17:33.117 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "daemon_cache_timeout", 2024-04-16T17:17:33.117 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-04-16T17:17:33.117 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.117 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:33.117 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "600", 2024-04-16T17:17:33.117 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.117 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.118 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.118 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-16T17:17:33.118 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.118 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.118 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.118 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.118 INFO:teuthology.orchestra.run.smithi045.stdout: "default_cephadm_command_timeout": { 2024-04-16T17:17:33.118 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "default_cephadm_command_timeout", 2024-04-16T17:17:33.118 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-16T17:17:33.119 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.119 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:33.119 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "900", 2024-04-16T17:17:33.119 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.119 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.119 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.119 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-16T17:17:33.119 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.120 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.120 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.120 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.120 INFO:teuthology.orchestra.run.smithi045.stdout: "default_registry": { 2024-04-16T17:17:33.120 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "default_registry", 2024-04-16T17:17:33.120 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:33.120 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.120 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:33.120 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "docker.io", 2024-04-16T17:17:33.120 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.121 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.121 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.121 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-16T17:17:33.121 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.121 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.121 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.121 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.121 INFO:teuthology.orchestra.run.smithi045.stdout: "device_cache_timeout": { 2024-04-16T17:17:33.121 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "device_cache_timeout", 2024-04-16T17:17:33.122 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-04-16T17:17:33.122 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.122 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:33.122 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "1800", 2024-04-16T17:17:33.122 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.122 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.122 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.122 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "seconds to cache device inventory", 2024-04-16T17:17:33.122 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.122 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.123 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.123 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.123 INFO:teuthology.orchestra.run.smithi045.stdout: "device_enhanced_scan": { 2024-04-16T17:17:33.123 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "device_enhanced_scan", 2024-04-16T17:17:33.123 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:33.123 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.123 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:33.123 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:33.123 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.124 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.124 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.124 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-16T17:17:33.124 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.124 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.124 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.124 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.124 INFO:teuthology.orchestra.run.smithi045.stdout: "facts_cache_timeout": { 2024-04-16T17:17:33.124 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "facts_cache_timeout", 2024-04-16T17:17:33.125 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-04-16T17:17:33.125 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.125 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:33.125 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "60", 2024-04-16T17:17:33.125 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.125 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.125 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.125 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "seconds to cache host facts data", 2024-04-16T17:17:33.125 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.125 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.126 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.126 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.126 INFO:teuthology.orchestra.run.smithi045.stdout: "grafana_dashboards_path": { 2024-04-16T17:17:33.126 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "grafana_dashboards_path", 2024-04-16T17:17:33.126 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:33.126 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.126 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:33.126 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-16T17:17:33.127 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.127 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.127 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.127 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-16T17:17:33.127 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.127 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.127 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.127 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.127 INFO:teuthology.orchestra.run.smithi045.stdout: "host_check_interval": { 2024-04-16T17:17:33.127 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "host_check_interval", 2024-04-16T17:17:33.128 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-04-16T17:17:33.128 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.128 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:33.128 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "600", 2024-04-16T17:17:33.128 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.128 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.128 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.128 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "how frequently to perform a host check", 2024-04-16T17:17:33.128 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.128 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.129 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.129 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.129 INFO:teuthology.orchestra.run.smithi045.stdout: "hw_monitoring": { 2024-04-16T17:17:33.129 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "hw_monitoring", 2024-04-16T17:17:33.129 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:33.129 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.129 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:33.129 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:33.129 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.129 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.130 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.130 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-16T17:17:33.130 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.130 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.130 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.130 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.130 INFO:teuthology.orchestra.run.smithi045.stdout: "inventory_list_all": { 2024-04-16T17:17:33.130 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "inventory_list_all", 2024-04-16T17:17:33.130 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:33.131 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.131 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:33.131 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:33.131 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.131 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.131 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.131 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-16T17:17:33.131 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.131 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.132 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.132 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.132 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-16T17:17:33.132 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-16T17:17:33.132 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:33.132 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.132 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:33.132 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:33.132 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.132 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.133 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-16T17:17:33.133 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-16T17:17:33.133 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-16T17:17:33.133 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-16T17:17:33.133 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-16T17:17:33.133 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-16T17:17:33.133 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-16T17:17:33.133 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-16T17:17:33.133 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.134 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.134 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.134 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.134 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.134 INFO:teuthology.orchestra.run.smithi045.stdout: "log_refresh_metadata": { 2024-04-16T17:17:33.134 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_refresh_metadata", 2024-04-16T17:17:33.134 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:33.134 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.134 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:33.135 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:33.135 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.135 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.135 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.135 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-16T17:17:33.135 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.135 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.135 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.135 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.136 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-16T17:17:33.136 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-16T17:17:33.136 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:33.136 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.136 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:33.136 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-16T17:17:33.136 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.136 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.136 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.136 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-16T17:17:33.137 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.137 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.137 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.137 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.137 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-16T17:17:33.137 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-16T17:17:33.137 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:33.137 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.137 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:33.137 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-16T17:17:33.138 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.138 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.138 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-16T17:17:33.138 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-16T17:17:33.138 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-16T17:17:33.138 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-16T17:17:33.138 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-16T17:17:33.138 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-16T17:17:33.139 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-16T17:17:33.139 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-16T17:17:33.139 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.139 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.139 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.139 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.139 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.139 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-16T17:17:33.139 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-16T17:17:33.139 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:33.140 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.140 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:33.140 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:33.140 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.140 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.140 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.140 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.140 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.140 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.141 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.141 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.141 INFO:teuthology.orchestra.run.smithi045.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-16T17:17:33.141 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-16T17:17:33.141 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:33.141 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.141 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:33.141 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:33.141 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.141 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.142 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.142 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-16T17:17:33.142 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.142 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.142 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.142 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.142 INFO:teuthology.orchestra.run.smithi045.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-16T17:17:33.142 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-16T17:17:33.142 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:33.143 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.143 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:33.143 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "*", 2024-04-16T17:17:33.143 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.143 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.143 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.143 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-16T17:17:33.143 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.143 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.143 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.144 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.144 INFO:teuthology.orchestra.run.smithi045.stdout: "max_count_per_host": { 2024-04-16T17:17:33.144 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "max_count_per_host", 2024-04-16T17:17:33.144 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-16T17:17:33.144 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.144 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:33.144 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "10", 2024-04-16T17:17:33.144 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.144 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.144 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.145 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "max number of daemons per service per host", 2024-04-16T17:17:33.145 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.145 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.145 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.145 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.145 INFO:teuthology.orchestra.run.smithi045.stdout: "max_osd_draining_count": { 2024-04-16T17:17:33.145 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "max_osd_draining_count", 2024-04-16T17:17:33.145 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-16T17:17:33.145 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.145 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:33.146 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "10", 2024-04-16T17:17:33.146 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.146 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.146 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.146 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-16T17:17:33.146 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.146 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.146 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.146 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.147 INFO:teuthology.orchestra.run.smithi045.stdout: "migration_current": { 2024-04-16T17:17:33.147 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "migration_current", 2024-04-16T17:17:33.147 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-16T17:17:33.147 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.147 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:33.147 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:33.147 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.147 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.147 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.147 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "internal - do not modify", 2024-04-16T17:17:33.148 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.148 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.148 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.148 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.148 INFO:teuthology.orchestra.run.smithi045.stdout: "mode": { 2024-04-16T17:17:33.148 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "mode", 2024-04-16T17:17:33.148 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:33.148 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.148 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:33.148 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "root", 2024-04-16T17:17:33.149 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.149 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.149 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-16T17:17:33.149 INFO:teuthology.orchestra.run.smithi045.stdout: "cephadm-package", 2024-04-16T17:17:33.149 INFO:teuthology.orchestra.run.smithi045.stdout: "root" 2024-04-16T17:17:33.149 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-16T17:17:33.149 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "mode for remote execution of cephadm", 2024-04-16T17:17:33.149 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.149 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.150 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.150 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.150 INFO:teuthology.orchestra.run.smithi045.stdout: "oob_default_addr": { 2024-04-16T17:17:33.150 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "oob_default_addr", 2024-04-16T17:17:33.150 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:33.150 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.150 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:33.150 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "169.254.1.1", 2024-04-16T17:17:33.151 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.151 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.151 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.151 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-16T17:17:33.151 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.151 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.151 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.151 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.151 INFO:teuthology.orchestra.run.smithi045.stdout: "prometheus_alerts_path": { 2024-04-16T17:17:33.152 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "prometheus_alerts_path", 2024-04-16T17:17:33.152 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:33.152 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.152 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:33.152 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-16T17:17:33.152 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.152 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.152 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.152 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-16T17:17:33.152 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.153 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.153 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.153 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.153 INFO:teuthology.orchestra.run.smithi045.stdout: "registry_insecure": { 2024-04-16T17:17:33.153 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "registry_insecure", 2024-04-16T17:17:33.153 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:33.153 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.153 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:33.153 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:33.153 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.154 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.154 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.154 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-16T17:17:33.154 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.154 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.154 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.154 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.154 INFO:teuthology.orchestra.run.smithi045.stdout: "registry_password": { 2024-04-16T17:17:33.154 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "registry_password", 2024-04-16T17:17:33.155 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:33.155 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.155 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:33.155 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:33.155 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.155 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.155 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.155 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-16T17:17:33.155 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.155 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.156 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.156 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.156 INFO:teuthology.orchestra.run.smithi045.stdout: "registry_url": { 2024-04-16T17:17:33.156 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "registry_url", 2024-04-16T17:17:33.156 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:33.156 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.156 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:33.156 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:33.156 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.156 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.157 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.157 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-16T17:17:33.157 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.157 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.157 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.157 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.157 INFO:teuthology.orchestra.run.smithi045.stdout: "registry_username": { 2024-04-16T17:17:33.157 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "registry_username", 2024-04-16T17:17:33.157 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:33.157 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.158 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:33.158 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:33.158 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.158 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.158 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.158 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-16T17:17:33.158 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.158 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.158 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.159 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.159 INFO:teuthology.orchestra.run.smithi045.stdout: "secure_monitoring_stack": { 2024-04-16T17:17:33.159 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "secure_monitoring_stack", 2024-04-16T17:17:33.159 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:33.159 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.159 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:33.159 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:33.159 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.159 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.159 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.160 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-16T17:17:33.160 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.160 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.160 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.160 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.160 INFO:teuthology.orchestra.run.smithi045.stdout: "service_discovery_port": { 2024-04-16T17:17:33.160 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "service_discovery_port", 2024-04-16T17:17:33.160 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-16T17:17:33.160 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.160 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:33.161 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "8765", 2024-04-16T17:17:33.161 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.161 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.161 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.161 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "cephadm service discovery port", 2024-04-16T17:17:33.161 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.161 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.161 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.161 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.161 INFO:teuthology.orchestra.run.smithi045.stdout: "ssh_config_file": { 2024-04-16T17:17:33.162 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ssh_config_file", 2024-04-16T17:17:33.162 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:33.162 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.162 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:33.162 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:33.162 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.162 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.162 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.162 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-16T17:17:33.163 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.163 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.163 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.163 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.163 INFO:teuthology.orchestra.run.smithi045.stdout: "use_agent": { 2024-04-16T17:17:33.163 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "use_agent", 2024-04-16T17:17:33.163 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:33.163 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.163 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:33.164 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:33.164 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.164 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.164 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.164 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-16T17:17:33.164 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.164 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.164 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.164 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.164 INFO:teuthology.orchestra.run.smithi045.stdout: "use_repo_digest": { 2024-04-16T17:17:33.165 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "use_repo_digest", 2024-04-16T17:17:33.165 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:33.165 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.165 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:33.165 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-16T17:17:33.165 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.165 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.165 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.165 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-16T17:17:33.165 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.166 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.166 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.166 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.166 INFO:teuthology.orchestra.run.smithi045.stdout: "warn_on_failed_host_check": { 2024-04-16T17:17:33.166 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "warn_on_failed_host_check", 2024-04-16T17:17:33.166 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:33.166 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.166 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:33.166 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-16T17:17:33.166 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.167 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.167 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.167 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "raise a health warning if the host check fails", 2024-04-16T17:17:33.167 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.167 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.167 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.167 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.167 INFO:teuthology.orchestra.run.smithi045.stdout: "warn_on_stray_daemons": { 2024-04-16T17:17:33.167 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "warn_on_stray_daemons", 2024-04-16T17:17:33.168 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:33.168 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.168 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:33.168 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-16T17:17:33.168 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.168 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.168 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.168 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-16T17:17:33.168 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.168 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.169 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.169 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.169 INFO:teuthology.orchestra.run.smithi045.stdout: "warn_on_stray_hosts": { 2024-04-16T17:17:33.169 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "warn_on_stray_hosts", 2024-04-16T17:17:33.169 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:33.169 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.169 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:33.169 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-16T17:17:33.169 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.169 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.170 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.170 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-16T17:17:33.170 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.170 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.170 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.170 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:17:33.170 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:17:33.170 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.170 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-16T17:17:33.171 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "crash", 2024-04-16T17:17:33.171 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-16T17:17:33.171 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-16T17:17:33.171 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-16T17:17:33.171 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-16T17:17:33.171 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-16T17:17:33.171 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:33.171 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.171 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:33.171 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:33.172 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.172 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.172 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-16T17:17:33.172 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-16T17:17:33.172 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-16T17:17:33.172 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-16T17:17:33.172 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-16T17:17:33.172 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-16T17:17:33.172 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-16T17:17:33.173 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-16T17:17:33.173 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.173 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.173 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.173 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.173 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.173 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-16T17:17:33.173 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-16T17:17:33.173 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:33.174 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.174 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:33.174 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:33.174 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.174 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.174 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.174 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.174 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.174 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.175 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.175 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.175 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-16T17:17:33.175 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-16T17:17:33.175 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:33.175 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.175 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:33.175 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-16T17:17:33.175 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.176 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.176 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-16T17:17:33.176 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-16T17:17:33.176 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-16T17:17:33.176 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-16T17:17:33.176 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-16T17:17:33.176 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-16T17:17:33.176 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-16T17:17:33.176 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-16T17:17:33.176 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.177 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.177 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.177 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.177 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.177 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-16T17:17:33.177 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-16T17:17:33.177 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:33.177 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.177 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:33.177 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:33.178 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.178 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.178 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.178 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.178 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.178 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.178 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.178 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.178 INFO:teuthology.orchestra.run.smithi045.stdout: "retain_interval": { 2024-04-16T17:17:33.179 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "retain_interval", 2024-04-16T17:17:33.179 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-04-16T17:17:33.179 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.179 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:33.179 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "31536000", 2024-04-16T17:17:33.179 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.179 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.179 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.179 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-16T17:17:33.179 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.180 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.180 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.180 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.180 INFO:teuthology.orchestra.run.smithi045.stdout: "warn_recent_interval": { 2024-04-16T17:17:33.180 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "warn_recent_interval", 2024-04-16T17:17:33.180 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-04-16T17:17:33.180 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.180 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:33.180 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "1209600", 2024-04-16T17:17:33.180 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.181 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.181 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.181 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-16T17:17:33.181 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.181 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.181 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.181 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:17:33.181 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:17:33.182 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.182 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-16T17:17:33.182 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "dashboard", 2024-04-16T17:17:33.182 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-16T17:17:33.182 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-16T17:17:33.182 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-16T17:17:33.182 INFO:teuthology.orchestra.run.smithi045.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-16T17:17:33.182 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-16T17:17:33.182 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-16T17:17:33.183 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.183 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:33.183 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "10", 2024-04-16T17:17:33.183 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.183 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.183 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.183 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.183 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.183 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.183 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.184 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.184 INFO:teuthology.orchestra.run.smithi045.stdout: "ALERTMANAGER_API_HOST": { 2024-04-16T17:17:33.184 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-16T17:17:33.184 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:33.184 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.184 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:33.184 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:33.184 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.184 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.184 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.185 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.185 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.185 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.185 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.185 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.185 INFO:teuthology.orchestra.run.smithi045.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-16T17:17:33.185 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-16T17:17:33.185 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:33.185 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.185 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:33.186 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-16T17:17:33.186 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.186 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.186 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.186 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.186 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.186 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.186 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.186 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.186 INFO:teuthology.orchestra.run.smithi045.stdout: "AUDIT_API_ENABLED": { 2024-04-16T17:17:33.187 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "AUDIT_API_ENABLED", 2024-04-16T17:17:33.187 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:33.187 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.187 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:33.187 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:33.187 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.187 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.187 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.187 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.188 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.188 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.188 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.188 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.188 INFO:teuthology.orchestra.run.smithi045.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-16T17:17:33.188 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-16T17:17:33.188 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:33.188 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.189 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:33.189 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-16T17:17:33.189 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.189 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.189 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.189 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.189 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.189 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.189 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.190 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.190 INFO:teuthology.orchestra.run.smithi045.stdout: "ENABLE_BROWSABLE_API": { 2024-04-16T17:17:33.190 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-16T17:17:33.190 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:33.190 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.190 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:33.190 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-16T17:17:33.190 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.191 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.191 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.191 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.191 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.191 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.191 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.191 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.191 INFO:teuthology.orchestra.run.smithi045.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-16T17:17:33.191 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-16T17:17:33.192 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:33.192 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.192 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:33.192 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-16T17:17:33.192 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.192 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.192 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.192 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.192 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.192 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.193 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.193 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.193 INFO:teuthology.orchestra.run.smithi045.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-16T17:17:33.193 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-16T17:17:33.193 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:33.193 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.193 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:33.193 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-16T17:17:33.193 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.193 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.194 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.194 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.194 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.194 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.194 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.194 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.194 INFO:teuthology.orchestra.run.smithi045.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-16T17:17:33.194 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-16T17:17:33.194 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:33.195 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.195 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:33.195 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-16T17:17:33.195 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.195 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.195 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.195 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.195 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.195 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.195 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.195 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.196 INFO:teuthology.orchestra.run.smithi045.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-16T17:17:33.196 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-16T17:17:33.196 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:33.196 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.196 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:33.196 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-16T17:17:33.196 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.196 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.196 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.197 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.197 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.197 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.197 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.197 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.197 INFO:teuthology.orchestra.run.smithi045.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-16T17:17:33.197 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-16T17:17:33.197 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:33.197 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.198 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:33.198 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-16T17:17:33.198 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.198 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.198 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.198 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.198 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.198 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.198 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.198 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.199 INFO:teuthology.orchestra.run.smithi045.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-16T17:17:33.199 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-16T17:17:33.199 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:33.199 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.199 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:33.199 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-16T17:17:33.199 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.199 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.199 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.200 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.200 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.200 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.200 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.200 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.200 INFO:teuthology.orchestra.run.smithi045.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-16T17:17:33.200 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-16T17:17:33.200 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:33.200 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.200 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:33.201 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-16T17:17:33.201 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.201 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.201 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.201 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.201 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.201 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.201 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.201 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.201 INFO:teuthology.orchestra.run.smithi045.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-16T17:17:33.202 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-16T17:17:33.202 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:33.202 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.202 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:33.202 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:33.202 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.202 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.202 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.202 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.203 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.203 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.203 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.203 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.203 INFO:teuthology.orchestra.run.smithi045.stdout: "GRAFANA_API_PASSWORD": { 2024-04-16T17:17:33.203 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-16T17:17:33.203 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:33.203 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.203 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:33.203 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "admin", 2024-04-16T17:17:33.204 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.204 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.204 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.204 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.204 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.204 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.204 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.204 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.204 INFO:teuthology.orchestra.run.smithi045.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-16T17:17:33.204 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-16T17:17:33.205 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:33.205 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.205 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:33.205 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-16T17:17:33.205 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.205 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.205 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.205 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.205 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.205 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.206 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.206 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.206 INFO:teuthology.orchestra.run.smithi045.stdout: "GRAFANA_API_URL": { 2024-04-16T17:17:33.206 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "GRAFANA_API_URL", 2024-04-16T17:17:33.206 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:33.206 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.206 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:33.206 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:33.206 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.207 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.207 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.207 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.207 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.207 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.207 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.207 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.207 INFO:teuthology.orchestra.run.smithi045.stdout: "GRAFANA_API_USERNAME": { 2024-04-16T17:17:33.207 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-16T17:17:33.208 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:33.208 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.208 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:33.208 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "admin", 2024-04-16T17:17:33.208 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.208 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.208 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.208 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.208 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.208 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.209 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.209 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.209 INFO:teuthology.orchestra.run.smithi045.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-16T17:17:33.209 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-16T17:17:33.209 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:33.209 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.209 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:33.209 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:33.209 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.210 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.210 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.210 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.210 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.210 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.210 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.210 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.210 INFO:teuthology.orchestra.run.smithi045.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-16T17:17:33.210 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-16T17:17:33.211 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:33.211 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.211 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:33.211 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:33.211 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.211 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.211 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.211 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.211 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.211 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.212 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.212 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.212 INFO:teuthology.orchestra.run.smithi045.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-16T17:17:33.212 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-16T17:17:33.212 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:33.212 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.212 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:33.212 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-16T17:17:33.212 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.213 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.213 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.213 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.213 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.213 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.213 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.213 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.213 INFO:teuthology.orchestra.run.smithi045.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-16T17:17:33.213 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-16T17:17:33.213 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:33.214 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.214 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:33.214 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:33.214 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.214 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.214 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.214 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.214 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.215 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.215 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.215 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.215 INFO:teuthology.orchestra.run.smithi045.stdout: "MANAGED_BY_CLUSTERS": { 2024-04-16T17:17:33.215 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-04-16T17:17:33.215 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:33.215 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.215 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:33.215 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "[]", 2024-04-16T17:17:33.216 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.216 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.216 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.216 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.216 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.216 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.216 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.216 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.216 INFO:teuthology.orchestra.run.smithi045.stdout: "MULTICLUSTER_CONFIG": { 2024-04-16T17:17:33.217 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "MULTICLUSTER_CONFIG", 2024-04-16T17:17:33.217 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:33.217 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.217 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:33.217 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "{}", 2024-04-16T17:17:33.217 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.217 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.217 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.217 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.218 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.218 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.218 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.218 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.218 INFO:teuthology.orchestra.run.smithi045.stdout: "PROMETHEUS_API_HOST": { 2024-04-16T17:17:33.218 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-16T17:17:33.218 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:33.218 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.219 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:33.219 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:33.219 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.219 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.219 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.219 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.219 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.219 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.219 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.220 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.220 INFO:teuthology.orchestra.run.smithi045.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-16T17:17:33.220 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-16T17:17:33.220 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:33.220 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.220 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:33.220 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-16T17:17:33.220 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.220 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.221 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.221 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.221 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.221 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.221 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.221 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.221 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-16T17:17:33.221 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-16T17:17:33.221 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:33.222 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.222 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:33.222 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:33.222 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.222 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.222 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.222 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.222 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.223 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.223 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.223 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.223 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-16T17:17:33.223 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-16T17:17:33.223 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:33.223 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.223 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:33.223 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:33.224 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.224 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.224 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.224 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.224 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.224 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.224 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.224 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.224 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-16T17:17:33.224 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-16T17:17:33.225 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:33.225 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.225 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:33.225 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-16T17:17:33.225 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.225 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.225 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.225 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.225 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.226 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.226 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.226 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.226 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-16T17:17:33.226 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-16T17:17:33.226 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:33.226 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.226 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:33.227 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-16T17:17:33.227 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.227 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.227 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.227 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.227 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.227 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.227 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.227 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.228 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-16T17:17:33.228 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-16T17:17:33.228 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:33.228 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.228 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:33.228 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:33.228 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.228 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.228 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.229 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.229 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.229 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.229 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.229 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.229 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-16T17:17:33.229 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-16T17:17:33.229 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:33.229 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.229 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:33.230 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:33.230 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.230 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.230 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.230 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.230 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.230 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.230 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.230 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.231 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-16T17:17:33.231 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-16T17:17:33.231 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:33.231 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.231 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:33.231 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:33.231 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.231 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.231 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.231 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.232 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.232 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.232 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.232 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.232 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_ENABLED": { 2024-04-16T17:17:33.232 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-16T17:17:33.232 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:33.232 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.232 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:33.233 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-16T17:17:33.233 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.233 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.233 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.233 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.233 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.233 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.233 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.233 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.234 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-16T17:17:33.234 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-16T17:17:33.234 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:33.234 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.234 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:33.234 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-16T17:17:33.234 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.234 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.235 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.235 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.235 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.235 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.235 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.235 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.235 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-16T17:17:33.235 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-16T17:17:33.235 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-16T17:17:33.236 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.236 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:33.236 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "10", 2024-04-16T17:17:33.236 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.236 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.236 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.236 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.236 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.236 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.237 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.237 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.237 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-16T17:17:33.237 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-16T17:17:33.237 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-16T17:17:33.237 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.237 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:33.237 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "8", 2024-04-16T17:17:33.237 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.237 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.238 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.238 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.238 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.238 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.238 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.238 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.238 INFO:teuthology.orchestra.run.smithi045.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-16T17:17:33.238 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-16T17:17:33.239 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-16T17:17:33.239 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.239 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:33.239 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "45", 2024-04-16T17:17:33.239 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.239 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.239 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.239 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.239 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.239 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.240 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.240 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.240 INFO:teuthology.orchestra.run.smithi045.stdout: "RGW_API_ACCESS_KEY": { 2024-04-16T17:17:33.240 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-16T17:17:33.240 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:33.240 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.240 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:33.240 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:33.240 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.241 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.241 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.241 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.241 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.241 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.241 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.241 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.241 INFO:teuthology.orchestra.run.smithi045.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-16T17:17:33.241 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-16T17:17:33.241 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:33.242 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.242 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:33.242 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "admin", 2024-04-16T17:17:33.242 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.242 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.242 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.242 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.242 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.243 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.243 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.243 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.243 INFO:teuthology.orchestra.run.smithi045.stdout: "RGW_API_SECRET_KEY": { 2024-04-16T17:17:33.243 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-16T17:17:33.243 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:33.243 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.243 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:33.243 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:33.243 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.244 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.244 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.244 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.244 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.244 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.244 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.244 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.244 INFO:teuthology.orchestra.run.smithi045.stdout: "RGW_API_SSL_VERIFY": { 2024-04-16T17:17:33.244 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-16T17:17:33.245 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:33.245 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.245 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:33.245 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-16T17:17:33.245 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.245 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.245 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.245 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.245 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.246 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.246 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.246 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.246 INFO:teuthology.orchestra.run.smithi045.stdout: "UNSAFE_TLS_v1_2": { 2024-04-16T17:17:33.246 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "UNSAFE_TLS_v1_2", 2024-04-16T17:17:33.246 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:33.246 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.246 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:33.247 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:33.247 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.247 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.247 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.247 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.247 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.247 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.247 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.247 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.247 INFO:teuthology.orchestra.run.smithi045.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-16T17:17:33.248 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-16T17:17:33.248 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-16T17:17:33.248 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.248 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:33.248 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0", 2024-04-16T17:17:33.248 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.248 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.248 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.248 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.248 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.249 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.249 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.249 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.249 INFO:teuthology.orchestra.run.smithi045.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-16T17:17:33.249 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-16T17:17:33.249 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-16T17:17:33.249 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.249 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:33.249 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "10", 2024-04-16T17:17:33.249 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.250 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.250 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.250 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.250 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.250 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.250 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.250 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.250 INFO:teuthology.orchestra.run.smithi045.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-16T17:17:33.250 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-16T17:17:33.251 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-16T17:17:33.251 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.251 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:33.251 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "5", 2024-04-16T17:17:33.251 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.251 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.251 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.251 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.251 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.252 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.252 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.252 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.252 INFO:teuthology.orchestra.run.smithi045.stdout: "cross_origin_url": { 2024-04-16T17:17:33.252 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "cross_origin_url", 2024-04-16T17:17:33.252 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:33.252 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.252 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:33.252 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:33.252 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.253 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.253 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.253 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.253 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.253 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.253 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.253 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.253 INFO:teuthology.orchestra.run.smithi045.stdout: "crt_file": { 2024-04-16T17:17:33.253 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "crt_file", 2024-04-16T17:17:33.253 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:33.254 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.254 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:33.254 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:33.254 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.254 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.254 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.254 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.254 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.254 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.255 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.255 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.255 INFO:teuthology.orchestra.run.smithi045.stdout: "debug": { 2024-04-16T17:17:33.255 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "debug", 2024-04-16T17:17:33.255 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:33.255 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.255 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:33.255 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:33.255 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.255 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.256 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.256 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Enable/disable debug options", 2024-04-16T17:17:33.256 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.256 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.256 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.256 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.256 INFO:teuthology.orchestra.run.smithi045.stdout: "jwt_token_ttl": { 2024-04-16T17:17:33.256 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "jwt_token_ttl", 2024-04-16T17:17:33.256 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-16T17:17:33.257 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.257 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:33.257 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "28800", 2024-04-16T17:17:33.257 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.257 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.257 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.257 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.257 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.257 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.258 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.258 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.258 INFO:teuthology.orchestra.run.smithi045.stdout: "key_file": { 2024-04-16T17:17:33.258 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "key_file", 2024-04-16T17:17:33.258 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:33.258 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.258 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:33.258 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:33.258 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.258 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.259 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.259 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.259 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.259 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.259 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.259 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.259 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-16T17:17:33.259 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-16T17:17:33.259 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:33.259 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.260 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:33.260 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:33.260 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.260 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.260 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-16T17:17:33.260 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-16T17:17:33.260 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-16T17:17:33.260 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-16T17:17:33.260 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-16T17:17:33.261 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-16T17:17:33.261 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-16T17:17:33.261 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-16T17:17:33.261 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.261 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.261 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.261 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.261 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.261 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-16T17:17:33.261 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-16T17:17:33.262 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:33.262 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.262 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:33.262 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:33.262 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.262 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.262 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.262 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.262 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.262 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.263 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.263 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.263 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-16T17:17:33.263 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-16T17:17:33.263 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:33.263 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.263 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:33.263 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-16T17:17:33.263 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.263 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.264 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-16T17:17:33.264 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-16T17:17:33.264 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-16T17:17:33.264 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-16T17:17:33.264 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-16T17:17:33.264 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-16T17:17:33.264 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-16T17:17:33.264 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-16T17:17:33.264 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.265 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.265 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.265 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.265 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.265 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-16T17:17:33.265 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-16T17:17:33.265 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:33.265 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.265 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:33.265 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:33.266 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.266 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.266 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.266 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.266 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.266 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.266 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.266 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.266 INFO:teuthology.orchestra.run.smithi045.stdout: "motd": { 2024-04-16T17:17:33.266 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "motd", 2024-04-16T17:17:33.267 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:33.267 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.267 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:33.267 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:33.267 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.267 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.267 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.267 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "The message of the day", 2024-04-16T17:17:33.267 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.268 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.268 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.268 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.268 INFO:teuthology.orchestra.run.smithi045.stdout: "redirect_resolve_ip_addr": { 2024-04-16T17:17:33.268 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "redirect_resolve_ip_addr", 2024-04-16T17:17:33.268 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:33.268 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.268 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:33.268 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:33.268 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.269 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.269 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.269 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.269 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.269 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.269 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.269 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.269 INFO:teuthology.orchestra.run.smithi045.stdout: "server_addr": { 2024-04-16T17:17:33.270 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "server_addr", 2024-04-16T17:17:33.270 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:33.270 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.270 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:33.270 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "::", 2024-04-16T17:17:33.270 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.270 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.270 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.270 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.271 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.271 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.271 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.271 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.271 INFO:teuthology.orchestra.run.smithi045.stdout: "server_port": { 2024-04-16T17:17:33.271 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "server_port", 2024-04-16T17:17:33.271 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-16T17:17:33.271 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.271 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:33.272 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "8080", 2024-04-16T17:17:33.272 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.272 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.272 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.272 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.272 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.272 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.272 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.272 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.272 INFO:teuthology.orchestra.run.smithi045.stdout: "ssl": { 2024-04-16T17:17:33.273 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ssl", 2024-04-16T17:17:33.273 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:33.273 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.273 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:33.273 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-16T17:17:33.273 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.273 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.273 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.273 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.274 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.274 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.274 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.274 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.274 INFO:teuthology.orchestra.run.smithi045.stdout: "ssl_server_port": { 2024-04-16T17:17:33.274 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ssl_server_port", 2024-04-16T17:17:33.274 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-16T17:17:33.274 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.274 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:33.275 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "8443", 2024-04-16T17:17:33.275 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.275 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.275 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.275 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.275 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.275 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.275 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.275 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.275 INFO:teuthology.orchestra.run.smithi045.stdout: "standby_behaviour": { 2024-04-16T17:17:33.276 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "standby_behaviour", 2024-04-16T17:17:33.276 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:33.276 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.276 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:33.276 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "redirect", 2024-04-16T17:17:33.276 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.276 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.276 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-16T17:17:33.276 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-16T17:17:33.277 INFO:teuthology.orchestra.run.smithi045.stdout: "redirect" 2024-04-16T17:17:33.277 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-16T17:17:33.277 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.277 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.277 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.277 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.277 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.277 INFO:teuthology.orchestra.run.smithi045.stdout: "standby_error_status_code": { 2024-04-16T17:17:33.277 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "standby_error_status_code", 2024-04-16T17:17:33.277 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-16T17:17:33.278 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.278 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:33.278 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "500", 2024-04-16T17:17:33.278 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "400", 2024-04-16T17:17:33.278 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "599", 2024-04-16T17:17:33.278 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.278 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.278 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.278 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.279 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.279 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.279 INFO:teuthology.orchestra.run.smithi045.stdout: "url_prefix": { 2024-04-16T17:17:33.279 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "url_prefix", 2024-04-16T17:17:33.279 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:33.279 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.279 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:33.279 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:33.279 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.279 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.280 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.280 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.280 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.280 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.280 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.280 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:17:33.280 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:17:33.280 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.280 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-16T17:17:33.280 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "devicehealth", 2024-04-16T17:17:33.281 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-16T17:17:33.281 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-16T17:17:33.281 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-16T17:17:33.281 INFO:teuthology.orchestra.run.smithi045.stdout: "enable_monitoring": { 2024-04-16T17:17:33.281 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "enable_monitoring", 2024-04-16T17:17:33.281 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:33.281 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.281 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:33.281 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-16T17:17:33.282 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.282 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.282 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.282 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "monitor device health metrics", 2024-04-16T17:17:33.282 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.282 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.282 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.282 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.283 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-16T17:17:33.283 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-16T17:17:33.283 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:33.283 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.283 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:33.283 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:33.283 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.283 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.283 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-16T17:17:33.284 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-16T17:17:33.284 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-16T17:17:33.284 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-16T17:17:33.284 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-16T17:17:33.284 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-16T17:17:33.284 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-16T17:17:33.284 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-16T17:17:33.284 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.284 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.285 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.285 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.285 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.285 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-16T17:17:33.285 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-16T17:17:33.285 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:33.285 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.285 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:33.285 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:33.286 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.286 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.286 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.286 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.286 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.286 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.286 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.286 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.286 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-16T17:17:33.286 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-16T17:17:33.287 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:33.287 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.287 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:33.287 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-16T17:17:33.287 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.287 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.287 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-16T17:17:33.287 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-16T17:17:33.287 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-16T17:17:33.287 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-16T17:17:33.288 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-16T17:17:33.288 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-16T17:17:33.288 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-16T17:17:33.288 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-16T17:17:33.288 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.288 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.288 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.288 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.288 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.289 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-16T17:17:33.289 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-16T17:17:33.289 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:33.289 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.289 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:33.289 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:33.289 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.289 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.289 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.290 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.290 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.290 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.290 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.290 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.290 INFO:teuthology.orchestra.run.smithi045.stdout: "mark_out_threshold": { 2024-04-16T17:17:33.290 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "mark_out_threshold", 2024-04-16T17:17:33.290 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-04-16T17:17:33.290 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.291 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:33.291 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "2419200", 2024-04-16T17:17:33.291 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.291 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.291 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.291 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-16T17:17:33.291 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.291 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.291 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.291 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.292 INFO:teuthology.orchestra.run.smithi045.stdout: "pool_name": { 2024-04-16T17:17:33.292 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "pool_name", 2024-04-16T17:17:33.292 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:33.292 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.292 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:33.292 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "device_health_metrics", 2024-04-16T17:17:33.292 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.292 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.292 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.293 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-16T17:17:33.293 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.293 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.293 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.293 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.293 INFO:teuthology.orchestra.run.smithi045.stdout: "retention_period": { 2024-04-16T17:17:33.293 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "retention_period", 2024-04-16T17:17:33.293 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-04-16T17:17:33.293 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.294 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:33.294 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "15552000", 2024-04-16T17:17:33.294 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.294 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.294 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.294 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "how long to retain device health metrics", 2024-04-16T17:17:33.294 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.295 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.295 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.295 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.295 INFO:teuthology.orchestra.run.smithi045.stdout: "scrape_frequency": { 2024-04-16T17:17:33.295 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "scrape_frequency", 2024-04-16T17:17:33.295 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-04-16T17:17:33.295 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.295 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:33.295 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "86400", 2024-04-16T17:17:33.295 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.296 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.296 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.296 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-16T17:17:33.296 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.296 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.296 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.296 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.296 INFO:teuthology.orchestra.run.smithi045.stdout: "self_heal": { 2024-04-16T17:17:33.296 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "self_heal", 2024-04-16T17:17:33.296 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:33.297 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.297 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:33.297 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-16T17:17:33.297 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.297 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.297 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.297 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-16T17:17:33.297 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.297 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.298 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.298 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.298 INFO:teuthology.orchestra.run.smithi045.stdout: "sleep_interval": { 2024-04-16T17:17:33.298 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "sleep_interval", 2024-04-16T17:17:33.298 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-04-16T17:17:33.298 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.298 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:33.298 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "600", 2024-04-16T17:17:33.298 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.299 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.299 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.299 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "how frequently to wake up and check device health", 2024-04-16T17:17:33.299 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.299 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.299 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.299 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.299 INFO:teuthology.orchestra.run.smithi045.stdout: "warn_threshold": { 2024-04-16T17:17:33.299 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "warn_threshold", 2024-04-16T17:17:33.299 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-04-16T17:17:33.299 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.300 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:33.300 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "7257600", 2024-04-16T17:17:33.300 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.300 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.300 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.300 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-16T17:17:33.300 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.300 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.300 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.301 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:17:33.301 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:17:33.301 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.301 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-16T17:17:33.301 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "diskprediction_local", 2024-04-16T17:17:33.301 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-16T17:17:33.301 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-16T17:17:33.301 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-16T17:17:33.301 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-16T17:17:33.301 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-16T17:17:33.302 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:33.302 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.302 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:33.302 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:33.302 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.302 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.302 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-16T17:17:33.302 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-16T17:17:33.302 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-16T17:17:33.303 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-16T17:17:33.303 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-16T17:17:33.303 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-16T17:17:33.303 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-16T17:17:33.303 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-16T17:17:33.303 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.303 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.303 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.303 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.303 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.303 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-16T17:17:33.304 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-16T17:17:33.304 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:33.304 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.304 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:33.304 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:33.304 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.304 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.304 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.304 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.304 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.305 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.305 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.305 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.305 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-16T17:17:33.305 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-16T17:17:33.305 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:33.305 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.305 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:33.305 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-16T17:17:33.305 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.306 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.306 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-16T17:17:33.306 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-16T17:17:33.306 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-16T17:17:33.306 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-16T17:17:33.306 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-16T17:17:33.306 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-16T17:17:33.306 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-16T17:17:33.306 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-16T17:17:33.307 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.307 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.307 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.307 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.307 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.307 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-16T17:17:33.307 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-16T17:17:33.307 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:33.307 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.307 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:33.308 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:33.308 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.308 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.308 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.308 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.308 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.308 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.308 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.308 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.308 INFO:teuthology.orchestra.run.smithi045.stdout: "predict_interval": { 2024-04-16T17:17:33.309 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "predict_interval", 2024-04-16T17:17:33.309 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:33.309 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.309 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:33.309 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "86400", 2024-04-16T17:17:33.309 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.309 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.309 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.309 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.309 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.310 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.310 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.310 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.310 INFO:teuthology.orchestra.run.smithi045.stdout: "predictor_model": { 2024-04-16T17:17:33.310 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "predictor_model", 2024-04-16T17:17:33.310 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:33.310 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.310 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:33.310 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "prophetstor", 2024-04-16T17:17:33.311 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.311 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.311 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.311 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.311 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.311 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.311 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.311 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.311 INFO:teuthology.orchestra.run.smithi045.stdout: "sleep_interval": { 2024-04-16T17:17:33.311 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "sleep_interval", 2024-04-16T17:17:33.312 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:33.312 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.312 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:33.312 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "600", 2024-04-16T17:17:33.312 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.312 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.312 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.312 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.312 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.312 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.313 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.313 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:17:33.313 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:17:33.313 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.313 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-16T17:17:33.313 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "influx", 2024-04-16T17:17:33.313 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": false, 2024-04-16T17:17:33.313 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "influxdb python module not found", 2024-04-16T17:17:33.314 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-16T17:17:33.314 INFO:teuthology.orchestra.run.smithi045.stdout: "batch_size": { 2024-04-16T17:17:33.314 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "batch_size", 2024-04-16T17:17:33.314 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-16T17:17:33.314 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.314 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:33.314 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "5000", 2024-04-16T17:17:33.314 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.314 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.315 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.315 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-16T17:17:33.315 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.315 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.315 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.315 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.315 INFO:teuthology.orchestra.run.smithi045.stdout: "database": { 2024-04-16T17:17:33.315 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "database", 2024-04-16T17:17:33.315 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:33.316 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.316 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:33.316 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "ceph", 2024-04-16T17:17:33.316 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.316 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.316 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.316 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-16T17:17:33.316 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.316 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.316 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.317 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.317 INFO:teuthology.orchestra.run.smithi045.stdout: "hostname": { 2024-04-16T17:17:33.317 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "hostname", 2024-04-16T17:17:33.317 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:33.317 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.317 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:33.317 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:33.317 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.317 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.317 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.318 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "InfluxDB server hostname", 2024-04-16T17:17:33.318 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.318 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.318 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.318 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.318 INFO:teuthology.orchestra.run.smithi045.stdout: "interval": { 2024-04-16T17:17:33.318 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "interval", 2024-04-16T17:17:33.318 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-04-16T17:17:33.318 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.318 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:33.319 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "30", 2024-04-16T17:17:33.319 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "5", 2024-04-16T17:17:33.319 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.319 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.319 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-16T17:17:33.319 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.319 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.319 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.319 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.320 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-16T17:17:33.320 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-16T17:17:33.320 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:33.320 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.320 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:33.320 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:33.320 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.320 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.320 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-16T17:17:33.321 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-16T17:17:33.321 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-16T17:17:33.321 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-16T17:17:33.321 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-16T17:17:33.321 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-16T17:17:33.321 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-16T17:17:33.321 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-16T17:17:33.321 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.321 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.322 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.322 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.322 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.322 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-16T17:17:33.322 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-16T17:17:33.322 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:33.322 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.322 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:33.322 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:33.322 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.323 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.323 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.323 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.323 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.323 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.323 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.323 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.323 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-16T17:17:33.323 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-16T17:17:33.323 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:33.324 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.324 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:33.324 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-16T17:17:33.324 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.324 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.324 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-16T17:17:33.324 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-16T17:17:33.324 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-16T17:17:33.324 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-16T17:17:33.325 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-16T17:17:33.325 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-16T17:17:33.325 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-16T17:17:33.325 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-16T17:17:33.325 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.325 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.325 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.325 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.326 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.326 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-16T17:17:33.326 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-16T17:17:33.326 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:33.326 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.326 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:33.326 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:33.326 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.326 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.327 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.327 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.327 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.327 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.327 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.327 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.327 INFO:teuthology.orchestra.run.smithi045.stdout: "password": { 2024-04-16T17:17:33.327 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "password", 2024-04-16T17:17:33.327 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:33.328 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.328 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:33.328 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:33.328 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.328 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.328 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.328 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "password of InfluxDB server user", 2024-04-16T17:17:33.328 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.328 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.328 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.329 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.329 INFO:teuthology.orchestra.run.smithi045.stdout: "port": { 2024-04-16T17:17:33.329 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "port", 2024-04-16T17:17:33.329 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-16T17:17:33.329 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.329 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:33.329 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "8086", 2024-04-16T17:17:33.329 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.329 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.329 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.330 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "InfluxDB server port", 2024-04-16T17:17:33.330 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.330 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.330 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.330 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.330 INFO:teuthology.orchestra.run.smithi045.stdout: "ssl": { 2024-04-16T17:17:33.330 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ssl", 2024-04-16T17:17:33.330 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:33.330 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.331 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:33.331 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "false", 2024-04-16T17:17:33.331 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.331 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.331 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.331 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-16T17:17:33.331 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.331 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.331 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.332 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.332 INFO:teuthology.orchestra.run.smithi045.stdout: "threads": { 2024-04-16T17:17:33.332 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "threads", 2024-04-16T17:17:33.332 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-16T17:17:33.332 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.332 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:33.332 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "5", 2024-04-16T17:17:33.332 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "1", 2024-04-16T17:17:33.332 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "32", 2024-04-16T17:17:33.332 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.333 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-16T17:17:33.333 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.333 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.333 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.333 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.333 INFO:teuthology.orchestra.run.smithi045.stdout: "username": { 2024-04-16T17:17:33.333 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "username", 2024-04-16T17:17:33.333 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:33.333 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.334 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:33.334 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:33.334 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.334 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.334 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.334 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "username of InfluxDB server user", 2024-04-16T17:17:33.334 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.334 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.334 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.335 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.335 INFO:teuthology.orchestra.run.smithi045.stdout: "verify_ssl": { 2024-04-16T17:17:33.335 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "verify_ssl", 2024-04-16T17:17:33.335 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:33.335 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.335 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:33.335 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "true", 2024-04-16T17:17:33.335 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.335 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.336 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.336 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-16T17:17:33.336 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.336 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.336 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.336 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:17:33.336 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:17:33.336 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.336 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-16T17:17:33.336 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "insights", 2024-04-16T17:17:33.337 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-16T17:17:33.337 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-16T17:17:33.337 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-16T17:17:33.337 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-16T17:17:33.337 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-16T17:17:33.337 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:33.337 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.337 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:33.337 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:33.337 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.338 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.338 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-16T17:17:33.338 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-16T17:17:33.338 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-16T17:17:33.338 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-16T17:17:33.338 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-16T17:17:33.338 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-16T17:17:33.338 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-16T17:17:33.339 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-16T17:17:33.339 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.339 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.339 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.339 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.339 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.339 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-16T17:17:33.339 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-16T17:17:33.339 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:33.340 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.340 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:33.340 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:33.340 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.340 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.340 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.340 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.340 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.340 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.340 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.341 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.341 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-16T17:17:33.341 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-16T17:17:33.341 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:33.341 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.341 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:33.341 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-16T17:17:33.341 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.341 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.341 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-16T17:17:33.342 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-16T17:17:33.342 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-16T17:17:33.342 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-16T17:17:33.342 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-16T17:17:33.342 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-16T17:17:33.342 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-16T17:17:33.342 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-16T17:17:33.342 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:31.729+0000 7f9387e00640 1 -- 172.21.15.45:0/4190718114 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f936c0a3d20 msgr2=0x7f936c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:17:33.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:31.729+0000 7f9387e00640 1 --2- 172.21.15.45:0/4190718114 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f936c0a3d20 0x7f936c0a61e0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f938807a280 tx=0x7f937000b390 comp rx=0 tx=0).stop 2024-04-16T17:17:33.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:31.729+0000 7f9387e00640 1 -- 172.21.15.45:0/4190718114 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9388108400 msgr2=0x7f93881a4900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:17:33.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:31.729+0000 7f9387e00640 1 --2- 172.21.15.45:0/4190718114 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9388108400 0x7f93881a4900 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f9378009900 tx=0x7f9378004360 comp rx=0 tx=0).stop 2024-04-16T17:17:33.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:31.729+0000 7f9387e00640 1 -- 172.21.15.45:0/4190718114 shutdown_connections 2024-04-16T17:17:33.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:31.729+0000 7f9387e00640 1 --2- 172.21.15.45:0/4190718114 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f936c0a3d20 0x7f936c0a61e0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:33.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:31.729+0000 7f9387e00640 1 --2- 172.21.15.45:0/4190718114 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f938810e490 0x7f938819eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:33.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:31.729+0000 7f9387e00640 1 --2- 172.21.15.45:0/4190718114 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9388108dc0 0x7f93881a4e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:33.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:31.729+0000 7f9387e00640 1 --2- 172.21.15.45:0/4190718114 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9388108400 0x7f93881a4900 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:17:33.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:31.729+0000 7f9387e00640 1 -- 172.21.15.45:0/4190718114 >> 172.21.15.45:0/4190718114 conn(0x7f9388077b80 msgr2=0x7f93880782d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:17:33.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:31.729+0000 7f9387e00640 1 -- 172.21.15.45:0/4190718114 shutdown_connections 2024-04-16T17:17:33.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:17:31.729+0000 7f9387e00640 1 -- 172.21.15.45:0/4190718114 wait complete. 2024-04-16T17:17:33.346 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:17:32 smithi104 ceph-mon[36457]: pgmap v337: 129 pgs: 129 active+clean; 55 MiB data, 874 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-16T17:17:33.346 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:17:32 smithi104 ceph-mon[36457]: from='client.? 172.21.15.45:0/4190718114' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-16T17:17:33.346 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:17:32 smithi117 ceph-mon[36730]: pgmap v337: 129 pgs: 129 active+clean; 55 MiB data, 874 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-16T17:17:33.346 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:17:32 smithi117 ceph-mon[36730]: from='client.? 172.21.15.45:0/4190718114' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-16T17:17:33.347 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.347 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.347 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.347 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.347 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-16T17:17:33.347 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-16T17:17:33.347 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:33.347 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.347 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:33.348 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:33.348 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.348 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.348 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.348 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.348 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.348 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.348 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.348 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:17:33.349 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:17:33.349 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.349 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-16T17:17:33.349 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "iostat", 2024-04-16T17:17:33.349 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-16T17:17:33.349 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-16T17:17:33.349 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-16T17:17:33.349 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-16T17:17:33.349 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-16T17:17:33.350 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:33.350 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.350 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:33.350 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:33.350 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.350 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.350 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-16T17:17:33.350 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-16T17:17:33.351 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-16T17:17:33.351 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-16T17:17:33.351 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-16T17:17:33.351 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-16T17:17:33.351 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-16T17:17:33.351 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-16T17:17:33.351 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.351 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.351 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.352 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.352 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.352 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-16T17:17:33.352 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-16T17:17:33.352 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:33.352 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.352 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:33.352 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:33.352 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.353 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.353 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.353 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.353 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.353 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.353 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.353 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.353 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-16T17:17:33.353 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-16T17:17:33.353 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:33.354 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.354 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:33.354 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-16T17:17:33.354 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.354 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.354 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-16T17:17:33.354 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-16T17:17:33.354 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-16T17:17:33.354 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-16T17:17:33.355 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-16T17:17:33.355 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-16T17:17:33.355 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-16T17:17:33.355 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-16T17:17:33.355 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.355 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.355 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.355 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.355 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.355 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-16T17:17:33.356 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-16T17:17:33.356 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:33.356 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.356 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:33.356 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:33.356 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.356 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.356 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.356 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.356 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.357 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.357 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.357 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:17:33.357 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:17:33.357 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.357 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-16T17:17:33.357 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "k8sevents", 2024-04-16T17:17:33.357 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-16T17:17:33.357 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-16T17:17:33.358 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-16T17:17:33.358 INFO:teuthology.orchestra.run.smithi045.stdout: "ceph_event_retention_days": { 2024-04-16T17:17:33.358 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ceph_event_retention_days", 2024-04-16T17:17:33.358 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-16T17:17:33.358 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.358 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:33.358 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "7", 2024-04-16T17:17:33.358 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.358 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.358 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.359 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-16T17:17:33.359 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.359 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.359 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.359 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.359 INFO:teuthology.orchestra.run.smithi045.stdout: "config_check_secs": { 2024-04-16T17:17:33.359 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "config_check_secs", 2024-04-16T17:17:33.359 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-16T17:17:33.359 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.360 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:33.360 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "10", 2024-04-16T17:17:33.360 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "10", 2024-04-16T17:17:33.360 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.360 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.360 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-16T17:17:33.360 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.360 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.360 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.360 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.361 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-16T17:17:33.361 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-16T17:17:33.361 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:33.361 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.361 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:33.361 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:33.361 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.361 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.361 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-16T17:17:33.362 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-16T17:17:33.362 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-16T17:17:33.362 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-16T17:17:33.362 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-16T17:17:33.362 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-16T17:17:33.362 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-16T17:17:33.362 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-16T17:17:33.362 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.362 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.363 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.363 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.363 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.363 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-16T17:17:33.363 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-16T17:17:33.363 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:33.363 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.363 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:33.363 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:33.364 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.364 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.364 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.364 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.364 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.364 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.364 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.364 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.364 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-16T17:17:33.364 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-16T17:17:33.365 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:33.365 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.365 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:33.365 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-16T17:17:33.365 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.365 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.365 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-16T17:17:33.365 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-16T17:17:33.365 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-16T17:17:33.365 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-16T17:17:33.366 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-16T17:17:33.366 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-16T17:17:33.366 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-16T17:17:33.366 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-16T17:17:33.366 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.366 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.366 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.366 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.366 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.367 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-16T17:17:33.367 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-16T17:17:33.367 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:33.367 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.367 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:33.367 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:33.367 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.367 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.367 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.367 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.368 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.368 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.368 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.368 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:17:33.368 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:17:33.368 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.368 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-16T17:17:33.368 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "localpool", 2024-04-16T17:17:33.368 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-16T17:17:33.369 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-16T17:17:33.369 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-16T17:17:33.369 INFO:teuthology.orchestra.run.smithi045.stdout: "failure_domain": { 2024-04-16T17:17:33.369 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "failure_domain", 2024-04-16T17:17:33.369 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:33.369 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.369 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:33.369 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "host", 2024-04-16T17:17:33.369 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.370 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.370 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.370 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "failure domain for any created local pool", 2024-04-16T17:17:33.370 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-16T17:17:33.370 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.370 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.370 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.370 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-16T17:17:33.370 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-16T17:17:33.371 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:33.371 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.371 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:33.371 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:33.371 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.371 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.371 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-16T17:17:33.371 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-16T17:17:33.371 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-16T17:17:33.372 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-16T17:17:33.372 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-16T17:17:33.372 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-16T17:17:33.372 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-16T17:17:33.372 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-16T17:17:33.372 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.372 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.372 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.372 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.372 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.373 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-16T17:17:33.373 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-16T17:17:33.373 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:33.373 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.373 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:33.373 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:33.373 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.373 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.373 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.374 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.374 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.374 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.374 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.374 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.374 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-16T17:17:33.374 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-16T17:17:33.374 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:33.374 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.375 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:33.375 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-16T17:17:33.375 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.375 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.375 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-16T17:17:33.375 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-16T17:17:33.375 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-16T17:17:33.375 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-16T17:17:33.375 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-16T17:17:33.376 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-16T17:17:33.376 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-16T17:17:33.376 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-16T17:17:33.376 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.376 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.376 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.376 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.376 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.376 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-16T17:17:33.377 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-16T17:17:33.377 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:33.377 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.377 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:33.377 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:33.377 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.377 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.377 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.377 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.377 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.378 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.378 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.378 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.378 INFO:teuthology.orchestra.run.smithi045.stdout: "min_size": { 2024-04-16T17:17:33.378 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "min_size", 2024-04-16T17:17:33.378 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-16T17:17:33.378 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.378 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:33.378 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:33.379 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.379 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.379 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.379 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "default min_size for any created local pool", 2024-04-16T17:17:33.379 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-16T17:17:33.379 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.379 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.379 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.379 INFO:teuthology.orchestra.run.smithi045.stdout: "num_rep": { 2024-04-16T17:17:33.380 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "num_rep", 2024-04-16T17:17:33.380 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-16T17:17:33.380 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.380 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:33.380 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "3", 2024-04-16T17:17:33.380 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.380 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.380 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.380 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "default replica count for any created local pool", 2024-04-16T17:17:33.380 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.381 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.381 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.381 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.381 INFO:teuthology.orchestra.run.smithi045.stdout: "pg_num": { 2024-04-16T17:17:33.381 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "pg_num", 2024-04-16T17:17:33.381 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-16T17:17:33.381 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.381 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:33.381 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "128", 2024-04-16T17:17:33.381 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.382 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.382 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.382 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "default pg_num for any created local pool", 2024-04-16T17:17:33.382 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.382 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.382 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.382 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.382 INFO:teuthology.orchestra.run.smithi045.stdout: "prefix": { 2024-04-16T17:17:33.383 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "prefix", 2024-04-16T17:17:33.383 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:33.383 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.383 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:33.383 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:33.383 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.383 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.383 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.383 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "name prefix for any created local pool", 2024-04-16T17:17:33.384 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.384 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.384 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.384 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.384 INFO:teuthology.orchestra.run.smithi045.stdout: "subtree": { 2024-04-16T17:17:33.384 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "subtree", 2024-04-16T17:17:33.384 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:33.384 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.384 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:33.385 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "rack", 2024-04-16T17:17:33.385 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.385 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.385 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.385 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-16T17:17:33.385 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-16T17:17:33.385 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.385 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.385 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:17:33.385 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:17:33.386 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.386 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-16T17:17:33.386 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "mds_autoscaler", 2024-04-16T17:17:33.386 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-16T17:17:33.386 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-16T17:17:33.386 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-16T17:17:33.386 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-16T17:17:33.386 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-16T17:17:33.386 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:33.387 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.387 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:33.387 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:33.387 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.387 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.387 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-16T17:17:33.387 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-16T17:17:33.387 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-16T17:17:33.387 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-16T17:17:33.387 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-16T17:17:33.388 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-16T17:17:33.388 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-16T17:17:33.388 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-16T17:17:33.388 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.388 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.388 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.388 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.388 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.388 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-16T17:17:33.389 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-16T17:17:33.389 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:33.389 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.389 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:33.389 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:33.389 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.389 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.389 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.389 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.390 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.390 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.390 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.390 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.390 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-16T17:17:33.390 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-16T17:17:33.390 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:33.390 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.390 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:33.391 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-16T17:17:33.391 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.391 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.391 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-16T17:17:33.391 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-16T17:17:33.391 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-16T17:17:33.391 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-16T17:17:33.391 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-16T17:17:33.391 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-16T17:17:33.392 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-16T17:17:33.392 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-16T17:17:33.392 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.392 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.392 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.392 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.392 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.392 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-16T17:17:33.392 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-16T17:17:33.392 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:33.393 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.393 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:33.393 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:33.393 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.393 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.393 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.393 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.393 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.393 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.394 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.394 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:17:33.394 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:17:33.394 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.394 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-16T17:17:33.394 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "mirroring", 2024-04-16T17:17:33.394 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-16T17:17:33.394 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-16T17:17:33.394 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-16T17:17:33.394 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-16T17:17:33.395 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-16T17:17:33.395 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:33.395 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.395 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:33.395 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:33.395 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.395 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.395 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-16T17:17:33.395 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-16T17:17:33.395 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-16T17:17:33.396 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-16T17:17:33.396 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-16T17:17:33.396 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-16T17:17:33.396 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-16T17:17:33.396 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-16T17:17:33.396 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.396 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.396 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.396 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.397 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.397 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-16T17:17:33.397 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-16T17:17:33.397 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:33.397 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.397 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:33.397 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:33.397 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.397 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.398 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.398 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.398 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.398 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.398 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.398 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.398 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-16T17:17:33.398 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-16T17:17:33.398 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:33.398 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.399 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:33.399 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-16T17:17:33.399 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.399 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.399 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-16T17:17:33.399 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-16T17:17:33.399 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-16T17:17:33.399 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-16T17:17:33.399 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-16T17:17:33.400 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-16T17:17:33.400 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-16T17:17:33.400 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-16T17:17:33.400 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.400 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.400 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.400 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.400 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.400 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-16T17:17:33.400 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-16T17:17:33.401 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:33.401 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.401 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:33.401 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:33.401 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.401 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.401 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.401 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.401 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.401 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.402 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.402 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:17:33.402 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:17:33.402 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.402 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-16T17:17:33.402 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "nfs", 2024-04-16T17:17:33.402 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-16T17:17:33.402 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-16T17:17:33.402 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-16T17:17:33.402 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-16T17:17:33.403 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-16T17:17:33.403 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:33.403 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.403 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:33.403 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:33.403 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.403 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.403 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-16T17:17:33.403 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-16T17:17:33.404 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-16T17:17:33.404 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-16T17:17:33.404 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-16T17:17:33.404 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-16T17:17:33.404 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-16T17:17:33.404 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-16T17:17:33.404 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.404 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.404 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.404 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.405 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.405 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-16T17:17:33.405 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-16T17:17:33.405 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:33.405 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.405 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:33.405 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:33.405 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.405 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.406 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.406 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.406 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.406 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.406 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.406 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.406 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-16T17:17:33.406 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-16T17:17:33.406 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:33.406 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.407 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:33.407 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-16T17:17:33.407 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.407 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.407 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-16T17:17:33.407 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-16T17:17:33.407 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-16T17:17:33.407 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-16T17:17:33.407 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-16T17:17:33.407 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-16T17:17:33.408 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-16T17:17:33.408 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-16T17:17:33.408 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.408 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.408 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.408 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.408 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.408 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-16T17:17:33.408 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-16T17:17:33.408 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:33.409 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.409 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:33.409 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:33.409 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.409 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.409 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.409 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.409 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.409 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.409 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.410 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:17:33.410 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:17:33.410 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.410 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-16T17:17:33.410 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "orchestrator", 2024-04-16T17:17:33.410 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-16T17:17:33.410 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-16T17:17:33.410 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-16T17:17:33.411 INFO:teuthology.orchestra.run.smithi045.stdout: "fail_fs": { 2024-04-16T17:17:33.411 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "fail_fs", 2024-04-16T17:17:33.411 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:33.411 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.411 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:33.411 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:33.411 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.411 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.411 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.411 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-16T17:17:33.412 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.412 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.412 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.412 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.412 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-16T17:17:33.412 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-16T17:17:33.412 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:33.412 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.412 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:33.412 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:33.413 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.413 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.413 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-16T17:17:33.413 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-16T17:17:33.413 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-16T17:17:33.413 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-16T17:17:33.413 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-16T17:17:33.413 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-16T17:17:33.413 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-16T17:17:33.414 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-16T17:17:33.414 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.414 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.414 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.414 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.414 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.414 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-16T17:17:33.414 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-16T17:17:33.414 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:33.415 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.415 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:33.415 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:33.415 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.415 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.415 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.415 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.415 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.416 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.416 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.416 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.416 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-16T17:17:33.416 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-16T17:17:33.416 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:33.416 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.416 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:33.416 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-16T17:17:33.416 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.417 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.417 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-16T17:17:33.417 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-16T17:17:33.417 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-16T17:17:33.417 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-16T17:17:33.417 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-16T17:17:33.417 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-16T17:17:33.417 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-16T17:17:33.417 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-16T17:17:33.418 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.418 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.418 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.418 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.418 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.418 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-16T17:17:33.418 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-16T17:17:33.418 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:33.418 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.419 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:33.419 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:33.419 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.419 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.419 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.419 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.419 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.419 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.419 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.420 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.420 INFO:teuthology.orchestra.run.smithi045.stdout: "orchestrator": { 2024-04-16T17:17:33.420 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "orchestrator", 2024-04-16T17:17:33.420 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:33.420 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.420 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:33.420 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:33.420 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.420 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.420 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-16T17:17:33.421 INFO:teuthology.orchestra.run.smithi045.stdout: "cephadm", 2024-04-16T17:17:33.421 INFO:teuthology.orchestra.run.smithi045.stdout: "rook", 2024-04-16T17:17:33.421 INFO:teuthology.orchestra.run.smithi045.stdout: "test_orchestrator" 2024-04-16T17:17:33.421 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-16T17:17:33.421 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Orchestrator backend", 2024-04-16T17:17:33.421 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.421 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.421 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.421 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:17:33.422 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:17:33.422 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.422 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-16T17:17:33.422 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "osd_perf_query", 2024-04-16T17:17:33.422 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-16T17:17:33.422 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-16T17:17:33.422 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-16T17:17:33.422 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-16T17:17:33.422 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-16T17:17:33.423 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:33.423 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.423 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:33.423 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:33.423 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.423 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.423 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-16T17:17:33.423 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-16T17:17:33.423 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-16T17:17:33.423 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-16T17:17:33.424 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-16T17:17:33.424 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-16T17:17:33.424 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-16T17:17:33.424 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-16T17:17:33.424 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.424 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.424 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.424 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.424 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.425 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-16T17:17:33.425 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-16T17:17:33.425 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:33.425 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.425 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:33.425 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:33.425 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.425 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.425 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.426 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.426 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.426 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.426 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.426 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.426 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-16T17:17:33.426 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-16T17:17:33.426 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:33.426 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.426 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:33.427 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-16T17:17:33.427 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.427 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.427 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-16T17:17:33.427 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-16T17:17:33.427 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-16T17:17:33.427 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-16T17:17:33.427 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-16T17:17:33.427 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-16T17:17:33.428 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-16T17:17:33.428 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-16T17:17:33.428 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.428 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.428 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.428 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.428 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.428 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-16T17:17:33.428 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-16T17:17:33.429 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:33.429 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.429 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:33.429 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:33.429 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.429 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.429 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.429 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.429 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.429 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.430 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.430 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:17:33.430 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:17:33.430 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.430 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-16T17:17:33.430 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "osd_support", 2024-04-16T17:17:33.430 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-16T17:17:33.430 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-16T17:17:33.431 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-16T17:17:33.431 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-16T17:17:33.431 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-16T17:17:33.431 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:33.431 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.431 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:33.431 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:33.431 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.431 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.431 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-16T17:17:33.432 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-16T17:17:33.432 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-16T17:17:33.432 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-16T17:17:33.432 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-16T17:17:33.432 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-16T17:17:33.432 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-16T17:17:33.432 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-16T17:17:33.432 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.432 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.432 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.433 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.433 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.433 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-16T17:17:33.433 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-16T17:17:33.433 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:33.433 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.433 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:33.433 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:33.433 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.433 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.434 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.434 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.434 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.434 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.434 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.434 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.434 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-16T17:17:33.434 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-16T17:17:33.434 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:33.435 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.435 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:33.435 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-16T17:17:33.435 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.435 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.435 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-16T17:17:33.435 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-16T17:17:33.435 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-16T17:17:33.435 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-16T17:17:33.436 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-16T17:17:33.436 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-16T17:17:33.436 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-16T17:17:33.436 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-16T17:17:33.436 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.436 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.436 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.436 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.436 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.436 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-16T17:17:33.437 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-16T17:17:33.437 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:33.437 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.437 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:33.437 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:33.437 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.437 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.437 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.437 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.437 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.438 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.438 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.438 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:17:33.438 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:17:33.438 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.438 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-16T17:17:33.438 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "pg_autoscaler", 2024-04-16T17:17:33.438 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-16T17:17:33.438 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-16T17:17:33.439 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-16T17:17:33.439 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-16T17:17:33.439 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-16T17:17:33.439 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:33.439 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.439 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:33.439 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:33.439 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.439 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.440 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-16T17:17:33.440 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-16T17:17:33.440 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-16T17:17:33.440 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-16T17:17:33.440 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-16T17:17:33.440 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-16T17:17:33.440 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-16T17:17:33.440 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-16T17:17:33.440 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.441 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.441 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.441 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.441 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.441 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-16T17:17:33.441 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-16T17:17:33.441 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:33.441 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.441 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:33.442 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:33.442 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.442 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.442 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.442 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.442 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.442 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.442 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.442 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.443 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-16T17:17:33.443 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-16T17:17:33.443 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:33.443 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.443 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:33.443 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-16T17:17:33.443 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.443 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.443 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-16T17:17:33.443 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-16T17:17:33.444 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-16T17:17:33.444 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-16T17:17:33.444 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-16T17:17:33.444 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-16T17:17:33.444 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-16T17:17:33.444 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-16T17:17:33.444 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.444 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.444 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.444 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.445 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.445 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-16T17:17:33.445 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-16T17:17:33.445 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:33.445 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.445 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:33.445 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:33.445 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.445 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.445 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.446 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.446 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.446 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.446 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.446 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.446 INFO:teuthology.orchestra.run.smithi045.stdout: "sleep_interval": { 2024-04-16T17:17:33.446 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "sleep_interval", 2024-04-16T17:17:33.446 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-04-16T17:17:33.446 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.447 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:33.447 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "60", 2024-04-16T17:17:33.447 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.447 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.447 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.447 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.447 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.447 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.447 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.447 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.448 INFO:teuthology.orchestra.run.smithi045.stdout: "threshold": { 2024-04-16T17:17:33.448 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "threshold", 2024-04-16T17:17:33.448 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "float", 2024-04-16T17:17:33.448 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.448 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:33.448 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "3.0", 2024-04-16T17:17:33.448 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "1.0", 2024-04-16T17:17:33.448 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.448 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.449 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "scaling threshold", 2024-04-16T17:17:33.449 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-16T17:17:33.449 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.449 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.449 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:17:33.449 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:17:33.449 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.449 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-16T17:17:33.449 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "progress", 2024-04-16T17:17:33.449 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-16T17:17:33.450 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-16T17:17:33.450 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-16T17:17:33.450 INFO:teuthology.orchestra.run.smithi045.stdout: "allow_pg_recovery_event": { 2024-04-16T17:17:33.450 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "allow_pg_recovery_event", 2024-04-16T17:17:33.450 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:33.450 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.450 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:33.450 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:33.450 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.451 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.451 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.451 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-16T17:17:33.451 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.451 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.451 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.451 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.451 INFO:teuthology.orchestra.run.smithi045.stdout: "enabled": { 2024-04-16T17:17:33.451 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "enabled", 2024-04-16T17:17:33.451 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:33.452 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.452 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:33.452 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-16T17:17:33.452 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.452 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.452 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.452 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.452 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.452 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.453 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.453 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.453 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-16T17:17:33.453 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-16T17:17:33.453 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:33.453 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.453 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:33.453 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:33.453 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.453 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.454 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-16T17:17:33.454 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-16T17:17:33.454 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-16T17:17:33.454 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-16T17:17:33.454 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-16T17:17:33.454 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-16T17:17:33.454 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-16T17:17:33.454 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-16T17:17:33.454 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.454 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.455 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.455 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.455 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.455 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-16T17:17:33.455 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-16T17:17:33.455 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:33.455 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.455 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:33.455 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:33.456 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.456 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.456 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.456 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.456 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.456 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.456 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.456 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.456 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-16T17:17:33.457 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-16T17:17:33.457 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:33.457 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.457 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:33.457 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-16T17:17:33.457 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.457 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.457 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-16T17:17:33.457 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-16T17:17:33.457 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-16T17:17:33.458 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-16T17:17:33.458 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-16T17:17:33.458 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-16T17:17:33.458 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-16T17:17:33.458 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-16T17:17:33.458 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.458 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.458 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.458 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.459 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.459 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-16T17:17:33.459 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-16T17:17:33.459 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:33.459 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.459 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:33.459 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:33.459 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.459 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.459 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.460 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.460 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.460 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.460 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.460 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.460 INFO:teuthology.orchestra.run.smithi045.stdout: "max_completed_events": { 2024-04-16T17:17:33.460 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "max_completed_events", 2024-04-16T17:17:33.460 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-16T17:17:33.460 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.460 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:33.461 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "50", 2024-04-16T17:17:33.461 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.461 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.461 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.461 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "number of past completed events to remember", 2024-04-16T17:17:33.461 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.461 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.461 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.461 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.462 INFO:teuthology.orchestra.run.smithi045.stdout: "sleep_interval": { 2024-04-16T17:17:33.462 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "sleep_interval", 2024-04-16T17:17:33.462 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-04-16T17:17:33.462 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.462 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:33.462 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "5", 2024-04-16T17:17:33.462 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.462 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.462 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.462 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "how long the module is going to sleep", 2024-04-16T17:17:33.463 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.463 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.463 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.463 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:17:33.463 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:17:33.463 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.463 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-16T17:17:33.463 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "prometheus", 2024-04-16T17:17:33.463 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-16T17:17:33.463 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-16T17:17:33.464 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-16T17:17:33.464 INFO:teuthology.orchestra.run.smithi045.stdout: "cache": { 2024-04-16T17:17:33.464 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "cache", 2024-04-16T17:17:33.464 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:33.464 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.464 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:33.464 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-16T17:17:33.464 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.464 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.465 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.465 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.465 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.465 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.465 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.465 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.465 INFO:teuthology.orchestra.run.smithi045.stdout: "exclude_perf_counters": { 2024-04-16T17:17:33.465 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "exclude_perf_counters", 2024-04-16T17:17:33.465 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:33.465 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.466 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:33.466 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-16T17:17:33.466 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.466 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.466 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.466 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-16T17:17:33.466 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-16T17:17:33.466 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.466 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.467 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.467 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-16T17:17:33.467 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-16T17:17:33.467 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:33.467 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.467 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:33.467 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:33.467 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.467 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.467 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-16T17:17:33.468 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-16T17:17:33.468 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-16T17:17:33.468 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-16T17:17:33.468 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-16T17:17:33.468 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-16T17:17:33.468 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-16T17:17:33.468 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-16T17:17:33.468 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.468 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.469 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.469 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.469 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.469 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-16T17:17:33.469 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-16T17:17:33.469 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:33.469 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.469 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:33.469 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:33.469 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.470 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.470 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.470 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.470 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.470 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.470 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.470 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.470 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-16T17:17:33.471 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-16T17:17:33.471 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:33.471 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.471 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:33.471 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-16T17:17:33.471 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.471 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.471 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-16T17:17:33.471 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-16T17:17:33.472 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-16T17:17:33.472 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-16T17:17:33.472 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-16T17:17:33.472 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-16T17:17:33.472 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-16T17:17:33.472 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-16T17:17:33.472 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.472 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.472 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.472 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.473 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.473 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-16T17:17:33.473 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-16T17:17:33.473 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:33.473 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.473 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:33.473 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:33.473 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.473 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.474 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.474 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.474 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.474 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.474 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.474 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.474 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_stats_pools": { 2024-04-16T17:17:33.474 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rbd_stats_pools", 2024-04-16T17:17:33.474 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:33.475 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.475 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:33.475 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:33.475 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.475 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.475 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.475 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.475 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.475 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.476 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.476 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.476 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-16T17:17:33.476 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-16T17:17:33.476 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-16T17:17:33.476 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.476 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:33.476 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "300", 2024-04-16T17:17:33.476 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.476 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.477 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.477 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.477 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.477 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.477 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.477 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.477 INFO:teuthology.orchestra.run.smithi045.stdout: "scrape_interval": { 2024-04-16T17:17:33.477 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "scrape_interval", 2024-04-16T17:17:33.477 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "float", 2024-04-16T17:17:33.478 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.478 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:33.478 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "15.0", 2024-04-16T17:17:33.478 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.478 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.478 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.478 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.478 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.478 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.478 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.479 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.479 INFO:teuthology.orchestra.run.smithi045.stdout: "server_addr": { 2024-04-16T17:17:33.479 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "server_addr", 2024-04-16T17:17:33.479 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:33.479 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.479 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:33.479 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "::", 2024-04-16T17:17:33.479 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.479 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.480 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.480 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-16T17:17:33.480 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.480 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.480 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.480 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.480 INFO:teuthology.orchestra.run.smithi045.stdout: "server_port": { 2024-04-16T17:17:33.480 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "server_port", 2024-04-16T17:17:33.480 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-16T17:17:33.480 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.481 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:33.481 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "9283", 2024-04-16T17:17:33.481 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.481 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.481 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.481 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-16T17:17:33.481 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.481 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.481 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.481 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.482 INFO:teuthology.orchestra.run.smithi045.stdout: "stale_cache_strategy": { 2024-04-16T17:17:33.482 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "stale_cache_strategy", 2024-04-16T17:17:33.482 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:33.482 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.482 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:33.482 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "log", 2024-04-16T17:17:33.482 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.482 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.482 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.483 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.483 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.483 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.483 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.483 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.483 INFO:teuthology.orchestra.run.smithi045.stdout: "standby_behaviour": { 2024-04-16T17:17:33.483 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "standby_behaviour", 2024-04-16T17:17:33.483 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:33.483 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.483 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:33.484 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "default", 2024-04-16T17:17:33.484 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.484 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.484 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-16T17:17:33.484 INFO:teuthology.orchestra.run.smithi045.stdout: "default", 2024-04-16T17:17:33.484 INFO:teuthology.orchestra.run.smithi045.stdout: "error" 2024-04-16T17:17:33.484 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-16T17:17:33.484 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.484 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.485 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.485 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.485 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.485 INFO:teuthology.orchestra.run.smithi045.stdout: "standby_error_status_code": { 2024-04-16T17:17:33.485 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "standby_error_status_code", 2024-04-16T17:17:33.485 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-16T17:17:33.485 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.485 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:33.485 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "500", 2024-04-16T17:17:33.486 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "400", 2024-04-16T17:17:33.486 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "599", 2024-04-16T17:17:33.486 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.486 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.486 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.486 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.486 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.486 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:17:33.486 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:17:33.487 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.487 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-16T17:17:33.487 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rbd_support", 2024-04-16T17:17:33.487 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-16T17:17:33.487 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-16T17:17:33.487 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-16T17:17:33.487 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-16T17:17:33.487 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-16T17:17:33.487 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:33.487 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.488 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:33.488 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:33.488 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.488 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.488 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-16T17:17:33.488 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-16T17:17:33.488 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-16T17:17:33.488 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-16T17:17:33.488 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-16T17:17:33.489 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-16T17:17:33.489 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-16T17:17:33.489 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-16T17:17:33.489 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.489 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.489 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.489 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.489 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.489 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-16T17:17:33.489 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-16T17:17:33.490 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:33.490 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.490 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:33.490 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:33.490 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.490 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.490 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.490 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.491 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.491 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.491 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.491 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.491 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-16T17:17:33.491 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-16T17:17:33.491 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:33.491 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.491 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:33.491 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-16T17:17:33.492 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.492 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.492 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-16T17:17:33.492 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-16T17:17:33.492 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-16T17:17:33.492 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-16T17:17:33.492 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-16T17:17:33.492 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-16T17:17:33.492 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-16T17:17:33.493 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-16T17:17:33.493 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.493 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.493 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.493 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.493 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.493 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-16T17:17:33.493 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-16T17:17:33.493 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:33.493 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.494 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:33.494 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:33.494 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.494 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.494 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.494 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.494 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.494 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.494 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.494 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.495 INFO:teuthology.orchestra.run.smithi045.stdout: "max_concurrent_snap_create": { 2024-04-16T17:17:33.495 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "max_concurrent_snap_create", 2024-04-16T17:17:33.495 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-16T17:17:33.495 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.495 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:33.495 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "10", 2024-04-16T17:17:33.495 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.495 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.495 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.496 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.496 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.496 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.496 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.496 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.496 INFO:teuthology.orchestra.run.smithi045.stdout: "mirror_snapshot_schedule": { 2024-04-16T17:17:33.496 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "mirror_snapshot_schedule", 2024-04-16T17:17:33.496 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:33.496 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.496 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:33.496 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:33.497 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.497 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.497 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.497 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.497 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.497 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.497 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.497 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.497 INFO:teuthology.orchestra.run.smithi045.stdout: "trash_purge_schedule": { 2024-04-16T17:17:33.498 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "trash_purge_schedule", 2024-04-16T17:17:33.498 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:33.498 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.498 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:33.498 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:33.498 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.498 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.498 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.498 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.499 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.499 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.499 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.499 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:17:33.499 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:17:33.499 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.499 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-16T17:17:33.499 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "restful", 2024-04-16T17:17:33.499 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-16T17:17:33.499 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-16T17:17:33.500 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-16T17:17:33.500 INFO:teuthology.orchestra.run.smithi045.stdout: "enable_auth": { 2024-04-16T17:17:33.500 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "enable_auth", 2024-04-16T17:17:33.500 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:33.500 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.500 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:33.500 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-16T17:17:33.500 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.500 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.501 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.501 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.501 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.501 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.501 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.501 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.501 INFO:teuthology.orchestra.run.smithi045.stdout: "key_file": { 2024-04-16T17:17:33.501 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "key_file", 2024-04-16T17:17:33.501 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:33.502 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.502 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:33.502 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:33.502 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.502 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.502 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.502 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.502 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.502 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.503 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.503 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.503 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-16T17:17:33.503 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-16T17:17:33.503 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:33.503 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.503 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:33.503 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:33.503 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.503 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.504 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-16T17:17:33.504 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-16T17:17:33.504 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-16T17:17:33.504 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-16T17:17:33.504 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-16T17:17:33.504 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-16T17:17:33.504 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-16T17:17:33.504 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-16T17:17:33.504 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.504 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.505 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.505 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.505 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.505 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-16T17:17:33.505 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-16T17:17:33.505 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:33.505 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.505 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:33.505 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:33.506 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.506 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.506 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.506 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.506 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.506 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.506 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.506 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.506 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-16T17:17:33.506 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-16T17:17:33.507 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:33.507 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.507 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:33.507 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-16T17:17:33.507 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.507 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.507 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-16T17:17:33.507 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-16T17:17:33.507 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-16T17:17:33.508 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-16T17:17:33.508 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-16T17:17:33.508 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-16T17:17:33.508 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-16T17:17:33.508 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-16T17:17:33.508 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.508 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.508 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.508 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.509 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.509 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-16T17:17:33.509 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-16T17:17:33.509 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:33.509 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.509 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:33.509 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:33.509 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.509 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.509 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.510 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.510 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.510 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.510 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.510 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.510 INFO:teuthology.orchestra.run.smithi045.stdout: "server_addr": { 2024-04-16T17:17:33.510 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "server_addr", 2024-04-16T17:17:33.510 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:33.511 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.511 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:33.511 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:33.511 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.511 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.511 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.511 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.511 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.511 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.512 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.512 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.512 INFO:teuthology.orchestra.run.smithi045.stdout: "server_port": { 2024-04-16T17:17:33.512 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "server_port", 2024-04-16T17:17:33.512 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:33.512 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.512 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:33.512 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:33.512 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.513 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.513 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.513 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.513 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.513 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.513 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.513 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:17:33.513 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:17:33.514 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.514 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-16T17:17:33.514 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rgw", 2024-04-16T17:17:33.514 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-16T17:17:33.514 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-16T17:17:33.514 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-16T17:17:33.514 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-16T17:17:33.514 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-16T17:17:33.514 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:33.515 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.515 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:33.515 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:33.515 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.515 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.515 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-16T17:17:33.515 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-16T17:17:33.515 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-16T17:17:33.515 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-16T17:17:33.516 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-16T17:17:33.516 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-16T17:17:33.516 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-16T17:17:33.516 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-16T17:17:33.516 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.516 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.516 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.516 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.516 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.517 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-16T17:17:33.517 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-16T17:17:33.517 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:33.517 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.517 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:33.517 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:33.517 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.517 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.517 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.518 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.518 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.518 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.518 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.518 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.518 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-16T17:17:33.518 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-16T17:17:33.518 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:33.518 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.519 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:33.519 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-16T17:17:33.519 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.519 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.519 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-16T17:17:33.519 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-16T17:17:33.519 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-16T17:17:33.519 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-16T17:17:33.520 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-16T17:17:33.520 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-16T17:17:33.520 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-16T17:17:33.520 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-16T17:17:33.520 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.520 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.520 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.520 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.520 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.521 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-16T17:17:33.521 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-16T17:17:33.521 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:33.521 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.521 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:33.521 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:33.521 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.521 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.521 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.522 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.522 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.522 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.522 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.522 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:17:33.522 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:17:33.522 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.522 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-16T17:17:33.523 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rook", 2024-04-16T17:17:33.523 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-16T17:17:33.523 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-16T17:17:33.523 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-16T17:17:33.523 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-16T17:17:33.523 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-16T17:17:33.523 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:33.523 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.523 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:33.523 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:33.524 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.524 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.524 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-16T17:17:33.524 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-16T17:17:33.524 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-16T17:17:33.524 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-16T17:17:33.524 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-16T17:17:33.524 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-16T17:17:33.524 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-16T17:17:33.525 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-16T17:17:33.525 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.525 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.525 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.525 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.525 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.525 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-16T17:17:33.525 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-16T17:17:33.525 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:33.526 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.526 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:33.526 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:33.526 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.526 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.526 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.526 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.526 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.526 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.527 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.527 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.527 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-16T17:17:33.527 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-16T17:17:33.527 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:33.527 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.527 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:33.527 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-16T17:17:33.527 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.527 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.528 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-16T17:17:33.528 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-16T17:17:33.528 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-16T17:17:33.528 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-16T17:17:33.528 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-16T17:17:33.528 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-16T17:17:33.528 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-16T17:17:33.528 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-16T17:17:33.529 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.529 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.529 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.529 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.529 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.529 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-16T17:17:33.529 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-16T17:17:33.529 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:33.529 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.529 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:33.530 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:33.530 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.530 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.530 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.530 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.530 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.530 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.530 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.530 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.531 INFO:teuthology.orchestra.run.smithi045.stdout: "storage_class": { 2024-04-16T17:17:33.531 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "storage_class", 2024-04-16T17:17:33.531 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:33.531 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.531 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:33.531 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "local", 2024-04-16T17:17:33.531 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.531 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.531 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.532 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-16T17:17:33.532 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.532 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.532 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.532 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:17:33.532 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:17:33.532 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.532 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-16T17:17:33.532 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "selftest", 2024-04-16T17:17:33.532 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-16T17:17:33.533 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-16T17:17:33.533 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-16T17:17:33.533 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-16T17:17:33.533 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-16T17:17:33.533 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:33.533 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.533 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:33.533 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:33.533 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.533 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.534 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-16T17:17:33.534 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-16T17:17:33.534 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-16T17:17:33.534 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-16T17:17:33.534 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-16T17:17:33.534 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-16T17:17:33.534 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-16T17:17:33.534 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-16T17:17:33.534 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.535 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.535 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.535 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.535 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.535 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-16T17:17:33.535 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-16T17:17:33.535 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:33.535 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.535 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:33.536 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:33.536 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.536 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.536 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.536 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.536 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.536 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.536 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.536 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.536 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-16T17:17:33.537 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-16T17:17:33.537 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:33.537 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.537 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:33.537 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-16T17:17:33.537 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.537 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.537 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-16T17:17:33.537 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-16T17:17:33.538 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-16T17:17:33.538 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-16T17:17:33.538 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-16T17:17:33.538 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-16T17:17:33.538 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-16T17:17:33.538 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-16T17:17:33.538 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.538 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.538 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.539 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.539 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.539 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-16T17:17:33.539 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-16T17:17:33.539 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:33.539 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.539 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:33.539 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:33.539 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.540 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.540 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.540 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.540 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.540 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.540 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.540 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.540 INFO:teuthology.orchestra.run.smithi045.stdout: "roption1": { 2024-04-16T17:17:33.541 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "roption1", 2024-04-16T17:17:33.541 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:33.541 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.541 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:33.541 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:33.541 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.541 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.541 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.541 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.541 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.542 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.542 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.542 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.542 INFO:teuthology.orchestra.run.smithi045.stdout: "roption2": { 2024-04-16T17:17:33.542 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "roption2", 2024-04-16T17:17:33.542 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:33.542 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.542 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:33.542 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "xyz", 2024-04-16T17:17:33.543 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.543 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.543 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.543 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.543 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.543 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.543 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.543 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.543 INFO:teuthology.orchestra.run.smithi045.stdout: "rwoption1": { 2024-04-16T17:17:33.544 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rwoption1", 2024-04-16T17:17:33.544 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:33.544 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.544 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:33.544 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:33.544 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.544 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.544 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.544 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.545 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.545 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.545 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.545 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.545 INFO:teuthology.orchestra.run.smithi045.stdout: "rwoption2": { 2024-04-16T17:17:33.545 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rwoption2", 2024-04-16T17:17:33.545 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-16T17:17:33.545 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.545 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:33.545 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:33.546 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.546 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.546 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.546 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.546 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.546 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.546 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.546 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.546 INFO:teuthology.orchestra.run.smithi045.stdout: "rwoption3": { 2024-04-16T17:17:33.547 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rwoption3", 2024-04-16T17:17:33.547 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "float", 2024-04-16T17:17:33.547 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.547 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:33.547 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:33.547 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.547 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.547 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.547 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.547 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.548 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.548 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.548 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.548 INFO:teuthology.orchestra.run.smithi045.stdout: "rwoption4": { 2024-04-16T17:17:33.548 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rwoption4", 2024-04-16T17:17:33.548 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:33.548 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.548 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:33.548 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:33.548 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.549 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.549 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.549 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.549 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.549 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.549 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.549 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.549 INFO:teuthology.orchestra.run.smithi045.stdout: "rwoption5": { 2024-04-16T17:17:33.549 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rwoption5", 2024-04-16T17:17:33.550 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:33.550 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.550 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:33.550 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:33.550 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.550 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.550 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.550 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.550 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.551 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.551 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.551 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.551 INFO:teuthology.orchestra.run.smithi045.stdout: "rwoption6": { 2024-04-16T17:17:33.551 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rwoption6", 2024-04-16T17:17:33.551 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:33.551 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.551 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:33.551 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-16T17:17:33.552 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.552 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.552 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.552 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.552 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.552 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.552 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.552 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.552 INFO:teuthology.orchestra.run.smithi045.stdout: "rwoption7": { 2024-04-16T17:17:33.552 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rwoption7", 2024-04-16T17:17:33.553 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-16T17:17:33.553 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.553 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:33.553 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:33.553 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "1", 2024-04-16T17:17:33.553 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "42", 2024-04-16T17:17:33.553 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.553 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.553 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.553 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.554 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.554 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.554 INFO:teuthology.orchestra.run.smithi045.stdout: "testkey": { 2024-04-16T17:17:33.554 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "testkey", 2024-04-16T17:17:33.554 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:33.554 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.554 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:33.554 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:33.554 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.555 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.555 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.555 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.555 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.555 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.555 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.555 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.555 INFO:teuthology.orchestra.run.smithi045.stdout: "testlkey": { 2024-04-16T17:17:33.555 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "testlkey", 2024-04-16T17:17:33.556 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:33.556 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.556 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:33.556 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:33.556 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.556 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.556 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.556 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.556 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.557 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.557 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.557 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.557 INFO:teuthology.orchestra.run.smithi045.stdout: "testnewline": { 2024-04-16T17:17:33.557 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "testnewline", 2024-04-16T17:17:33.557 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:33.557 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.557 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:33.557 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:33.557 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.558 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.558 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.558 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.558 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.558 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.558 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.558 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:17:33.558 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:17:33.558 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.559 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-16T17:17:33.559 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "snap_schedule", 2024-04-16T17:17:33.559 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-16T17:17:33.559 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-16T17:17:33.559 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-16T17:17:33.559 INFO:teuthology.orchestra.run.smithi045.stdout: "allow_m_granularity": { 2024-04-16T17:17:33.559 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "allow_m_granularity", 2024-04-16T17:17:33.559 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:33.559 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.559 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:33.560 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:33.560 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.560 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.560 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.560 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "allow minute scheduled snapshots", 2024-04-16T17:17:33.560 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.560 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.560 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.560 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.560 INFO:teuthology.orchestra.run.smithi045.stdout: "dump_on_update": { 2024-04-16T17:17:33.561 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "dump_on_update", 2024-04-16T17:17:33.561 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:33.561 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.561 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:33.561 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:33.561 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.561 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.561 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.561 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "dump database to debug log on update", 2024-04-16T17:17:33.562 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.562 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.562 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.562 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.562 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-16T17:17:33.562 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-16T17:17:33.562 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:33.562 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.562 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:33.563 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:33.563 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.563 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.563 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-16T17:17:33.563 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-16T17:17:33.563 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-16T17:17:33.563 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-16T17:17:33.563 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-16T17:17:33.563 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-16T17:17:33.563 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-16T17:17:33.564 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-16T17:17:33.564 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.564 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.564 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.564 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.564 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.564 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-16T17:17:33.564 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-16T17:17:33.564 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:33.564 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.565 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:33.565 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:33.565 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.565 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.565 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.565 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.565 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.565 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.565 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.566 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.566 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-16T17:17:33.566 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-16T17:17:33.566 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:33.566 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.566 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:33.566 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-16T17:17:33.566 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.566 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.567 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-16T17:17:33.567 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-16T17:17:33.567 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-16T17:17:33.567 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-16T17:17:33.567 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-16T17:17:33.567 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-16T17:17:33.567 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-16T17:17:33.567 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-16T17:17:33.567 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.567 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.568 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.568 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.568 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.568 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-16T17:17:33.568 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-16T17:17:33.568 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:33.568 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.568 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:33.568 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:33.568 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.569 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.569 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.569 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.569 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.569 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.569 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.569 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:17:33.569 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:17:33.569 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.569 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-16T17:17:33.570 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "stats", 2024-04-16T17:17:33.570 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-16T17:17:33.570 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-16T17:17:33.570 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-16T17:17:33.570 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-16T17:17:33.570 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-16T17:17:33.570 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:33.570 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.570 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:33.571 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:33.571 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.571 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.571 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-16T17:17:33.571 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-16T17:17:33.571 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-16T17:17:33.571 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-16T17:17:33.571 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-16T17:17:33.571 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-16T17:17:33.572 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-16T17:17:33.572 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-16T17:17:33.572 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.572 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.572 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.572 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.572 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.572 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-16T17:17:33.572 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-16T17:17:33.573 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:33.573 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.573 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:33.573 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:33.573 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.573 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.573 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.573 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.573 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.573 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.574 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.574 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.574 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-16T17:17:33.574 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-16T17:17:33.574 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:33.574 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.574 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:33.574 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-16T17:17:33.575 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.575 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.575 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-16T17:17:33.575 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-16T17:17:33.575 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-16T17:17:33.575 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-16T17:17:33.575 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-16T17:17:33.575 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-16T17:17:33.575 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-16T17:17:33.575 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-16T17:17:33.576 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.576 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.576 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.576 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.576 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.576 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-16T17:17:33.576 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-16T17:17:33.576 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:33.576 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.576 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:33.577 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:33.577 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.577 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.577 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.577 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.577 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.577 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.577 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.577 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:17:33.578 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:17:33.578 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.578 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-16T17:17:33.578 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "status", 2024-04-16T17:17:33.578 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-16T17:17:33.578 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-16T17:17:33.578 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-16T17:17:33.578 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-16T17:17:33.578 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-16T17:17:33.578 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:33.579 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.579 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:33.579 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:33.579 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.579 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.579 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-16T17:17:33.579 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-16T17:17:33.579 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-16T17:17:33.579 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-16T17:17:33.579 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-16T17:17:33.580 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-16T17:17:33.580 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-16T17:17:33.580 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-16T17:17:33.580 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.580 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.580 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.580 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.580 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.580 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-16T17:17:33.580 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-16T17:17:33.581 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:33.581 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.581 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:33.581 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:33.581 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.581 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.581 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.581 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.581 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.582 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.582 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.582 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.582 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-16T17:17:33.582 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-16T17:17:33.582 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:33.582 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.582 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:33.582 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-16T17:17:33.583 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.583 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.583 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-16T17:17:33.583 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-16T17:17:33.583 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-16T17:17:33.583 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-16T17:17:33.583 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-16T17:17:33.583 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-16T17:17:33.583 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-16T17:17:33.584 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-16T17:17:33.584 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.584 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.584 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.584 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.584 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.584 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-16T17:17:33.584 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-16T17:17:33.584 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:33.585 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.585 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:33.585 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:33.585 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.585 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.585 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.585 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.585 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.585 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.585 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.586 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:17:33.586 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:17:33.586 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.586 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-16T17:17:33.586 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "telegraf", 2024-04-16T17:17:33.586 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-16T17:17:33.586 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-16T17:17:33.586 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-16T17:17:33.586 INFO:teuthology.orchestra.run.smithi045.stdout: "address": { 2024-04-16T17:17:33.586 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "address", 2024-04-16T17:17:33.587 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:33.587 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.587 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:33.587 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-16T17:17:33.587 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.587 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.587 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.587 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.587 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.588 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.588 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.588 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.588 INFO:teuthology.orchestra.run.smithi045.stdout: "interval": { 2024-04-16T17:17:33.588 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "interval", 2024-04-16T17:17:33.588 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-04-16T17:17:33.588 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.588 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:33.588 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "15", 2024-04-16T17:17:33.588 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.589 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.589 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.589 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.589 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.589 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.589 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.589 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.589 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-16T17:17:33.589 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-16T17:17:33.590 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:33.590 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.590 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:33.590 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:33.590 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.590 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.590 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-16T17:17:33.590 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-16T17:17:33.590 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-16T17:17:33.591 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-16T17:17:33.591 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-16T17:17:33.591 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-16T17:17:33.591 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-16T17:17:33.591 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-16T17:17:33.591 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.591 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.591 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.591 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.592 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.592 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-16T17:17:33.592 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-16T17:17:33.592 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:33.592 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.592 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:33.592 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:33.592 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.592 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.592 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.593 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.593 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.593 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.593 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.593 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.593 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-16T17:17:33.593 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-16T17:17:33.593 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:33.593 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.594 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:33.594 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-16T17:17:33.594 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.594 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.594 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-16T17:17:33.594 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-16T17:17:33.594 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-16T17:17:33.594 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-16T17:17:33.594 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-16T17:17:33.594 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-16T17:17:33.595 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-16T17:17:33.595 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-16T17:17:33.595 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.595 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.595 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.595 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.595 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.595 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-16T17:17:33.595 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-16T17:17:33.596 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:33.596 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.596 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:33.596 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:33.596 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.596 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.596 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.596 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.596 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.596 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.597 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.597 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:17:33.597 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:17:33.597 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.597 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-16T17:17:33.597 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "telemetry", 2024-04-16T17:17:33.597 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-16T17:17:33.597 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-16T17:17:33.597 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-16T17:17:33.597 INFO:teuthology.orchestra.run.smithi045.stdout: "channel_basic": { 2024-04-16T17:17:33.598 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "channel_basic", 2024-04-16T17:17:33.598 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:33.598 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.598 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:33.598 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-16T17:17:33.598 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.598 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.598 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.598 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-16T17:17:33.598 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.599 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.599 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.599 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.599 INFO:teuthology.orchestra.run.smithi045.stdout: "channel_crash": { 2024-04-16T17:17:33.599 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "channel_crash", 2024-04-16T17:17:33.599 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:33.599 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.599 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:33.600 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-16T17:17:33.600 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.600 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.600 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.600 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-16T17:17:33.600 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.600 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.600 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.600 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.601 INFO:teuthology.orchestra.run.smithi045.stdout: "channel_device": { 2024-04-16T17:17:33.601 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "channel_device", 2024-04-16T17:17:33.601 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:33.601 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.601 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:33.601 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-16T17:17:33.601 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.601 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.601 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.601 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-16T17:17:33.602 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.602 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.602 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.602 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.602 INFO:teuthology.orchestra.run.smithi045.stdout: "channel_ident": { 2024-04-16T17:17:33.602 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "channel_ident", 2024-04-16T17:17:33.602 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:33.602 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.602 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:33.602 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:33.603 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.603 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.603 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.603 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-16T17:17:33.603 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.603 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.603 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.603 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.603 INFO:teuthology.orchestra.run.smithi045.stdout: "channel_perf": { 2024-04-16T17:17:33.603 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "channel_perf", 2024-04-16T17:17:33.604 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:33.604 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.604 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:33.604 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:33.604 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.604 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.604 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.604 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-16T17:17:33.604 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.605 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.605 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.605 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.605 INFO:teuthology.orchestra.run.smithi045.stdout: "contact": { 2024-04-16T17:17:33.605 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "contact", 2024-04-16T17:17:33.605 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:33.605 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.605 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:33.605 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:33.605 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.606 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.606 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.606 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.606 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.606 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.606 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.606 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.606 INFO:teuthology.orchestra.run.smithi045.stdout: "description": { 2024-04-16T17:17:33.606 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "description", 2024-04-16T17:17:33.606 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:33.607 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.607 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:33.607 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:33.607 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.607 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.607 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.607 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.607 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.607 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.607 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.608 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.608 INFO:teuthology.orchestra.run.smithi045.stdout: "device_url": { 2024-04-16T17:17:33.608 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "device_url", 2024-04-16T17:17:33.608 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:33.608 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.608 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:33.608 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-16T17:17:33.608 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.608 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.608 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.609 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.609 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.609 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.609 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.609 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.609 INFO:teuthology.orchestra.run.smithi045.stdout: "enabled": { 2024-04-16T17:17:33.609 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "enabled", 2024-04-16T17:17:33.609 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:33.609 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.609 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:33.610 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:33.610 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.610 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.610 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.610 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.610 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.610 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.610 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.610 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.611 INFO:teuthology.orchestra.run.smithi045.stdout: "interval": { 2024-04-16T17:17:33.611 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "interval", 2024-04-16T17:17:33.611 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-16T17:17:33.611 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.611 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:33.611 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "24", 2024-04-16T17:17:33.611 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "8", 2024-04-16T17:17:33.611 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.611 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.612 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.612 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.612 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.612 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.612 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.612 INFO:teuthology.orchestra.run.smithi045.stdout: "last_opt_revision": { 2024-04-16T17:17:33.612 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "last_opt_revision", 2024-04-16T17:17:33.612 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-16T17:17:33.612 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.612 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:33.613 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "1", 2024-04-16T17:17:33.613 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.613 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.613 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.613 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.613 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.613 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.613 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.613 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.614 INFO:teuthology.orchestra.run.smithi045.stdout: "leaderboard": { 2024-04-16T17:17:33.614 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "leaderboard", 2024-04-16T17:17:33.614 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:33.614 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.614 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:33.614 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:33.614 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.614 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.614 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.615 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.615 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.615 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.615 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.615 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.615 INFO:teuthology.orchestra.run.smithi045.stdout: "leaderboard_description": { 2024-04-16T17:17:33.615 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "leaderboard_description", 2024-04-16T17:17:33.615 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:33.615 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.616 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:33.616 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:33.616 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.616 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.616 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.616 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.616 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.616 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.617 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.617 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.617 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-16T17:17:33.617 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-16T17:17:33.617 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:33.617 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.617 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:33.617 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:33.617 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.617 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.618 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-16T17:17:33.618 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-16T17:17:33.618 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-16T17:17:33.618 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-16T17:17:33.618 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-16T17:17:33.618 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-16T17:17:33.618 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-16T17:17:33.618 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-16T17:17:33.618 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.619 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.619 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.619 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.619 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.619 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-16T17:17:33.619 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-16T17:17:33.619 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:33.619 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.619 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:33.620 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:33.620 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.620 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.620 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.620 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.620 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.620 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.620 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.620 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.620 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-16T17:17:33.621 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-16T17:17:33.621 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:33.621 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.621 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:33.621 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-16T17:17:33.621 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.621 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.621 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-16T17:17:33.621 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-16T17:17:33.622 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-16T17:17:33.622 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-16T17:17:33.622 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-16T17:17:33.622 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-16T17:17:33.622 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-16T17:17:33.622 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-16T17:17:33.622 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.622 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.622 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.622 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.623 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.623 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-16T17:17:33.623 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-16T17:17:33.623 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:33.623 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.623 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:33.623 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:33.623 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.623 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.624 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.624 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.624 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.624 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.624 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.624 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.624 INFO:teuthology.orchestra.run.smithi045.stdout: "organization": { 2024-04-16T17:17:33.624 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "organization", 2024-04-16T17:17:33.624 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:33.624 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.625 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:33.625 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:33.625 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.625 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.625 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.625 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.625 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.625 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.625 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.626 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.626 INFO:teuthology.orchestra.run.smithi045.stdout: "proxy": { 2024-04-16T17:17:33.626 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "proxy", 2024-04-16T17:17:33.626 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:33.626 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.626 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:33.626 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:33.626 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.626 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.627 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.627 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.627 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.627 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.627 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.627 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.627 INFO:teuthology.orchestra.run.smithi045.stdout: "url": { 2024-04-16T17:17:33.627 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "url", 2024-04-16T17:17:33.627 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:33.627 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.628 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:33.628 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-16T17:17:33.628 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.628 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.628 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.628 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.628 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.628 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.628 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.629 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:17:33.629 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:17:33.629 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.629 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-16T17:17:33.629 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "test_orchestrator", 2024-04-16T17:17:33.629 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-16T17:17:33.629 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-16T17:17:33.629 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-16T17:17:33.629 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-16T17:17:33.629 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-16T17:17:33.630 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:33.630 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.630 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:33.630 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:33.630 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.630 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.630 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-16T17:17:33.630 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-16T17:17:33.630 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-16T17:17:33.631 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-16T17:17:33.631 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-16T17:17:33.631 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-16T17:17:33.631 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-16T17:17:33.631 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-16T17:17:33.631 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.631 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.631 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.631 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.631 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.632 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-16T17:17:33.632 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-16T17:17:33.632 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:33.632 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.632 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:33.632 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:33.632 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.632 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.632 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.633 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.633 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.633 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.633 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.633 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.633 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-16T17:17:33.633 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-16T17:17:33.633 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:33.633 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.633 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:33.634 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-16T17:17:33.634 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.634 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.634 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-16T17:17:33.634 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-16T17:17:33.634 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-16T17:17:33.634 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-16T17:17:33.634 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-16T17:17:33.634 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-16T17:17:33.634 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-16T17:17:33.635 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-16T17:17:33.635 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.635 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.635 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.635 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.635 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.635 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-16T17:17:33.635 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-16T17:17:33.635 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:33.636 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.636 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:33.636 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:33.636 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.636 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.636 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.636 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.636 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.636 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.637 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.637 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:17:33.637 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:17:33.637 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.637 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-16T17:17:33.637 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "volumes", 2024-04-16T17:17:33.637 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-16T17:17:33.637 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-16T17:17:33.637 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-16T17:17:33.638 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-16T17:17:33.638 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-16T17:17:33.638 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:33.638 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.638 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:33.638 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:33.638 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.638 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.638 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-16T17:17:33.638 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-16T17:17:33.639 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-16T17:17:33.639 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-16T17:17:33.639 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-16T17:17:33.639 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-16T17:17:33.639 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-16T17:17:33.639 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-16T17:17:33.639 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.639 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.639 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.640 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.640 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.640 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-16T17:17:33.640 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-16T17:17:33.640 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:33.640 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.640 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:33.640 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:33.640 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.641 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.641 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.641 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.641 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.641 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.641 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.641 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.641 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-16T17:17:33.641 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-16T17:17:33.641 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:33.642 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.642 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:33.642 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-16T17:17:33.642 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.642 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.642 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-16T17:17:33.642 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-16T17:17:33.642 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-16T17:17:33.642 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-16T17:17:33.643 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-16T17:17:33.643 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-16T17:17:33.643 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-16T17:17:33.643 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-16T17:17:33.643 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.643 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.643 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.643 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.643 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.643 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-16T17:17:33.644 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-16T17:17:33.644 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:33.644 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.644 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:33.644 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:33.644 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.644 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.644 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.645 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.645 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.645 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.645 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.645 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.645 INFO:teuthology.orchestra.run.smithi045.stdout: "max_concurrent_clones": { 2024-04-16T17:17:33.645 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "max_concurrent_clones", 2024-04-16T17:17:33.645 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-16T17:17:33.645 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.645 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:33.646 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "4", 2024-04-16T17:17:33.646 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.646 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.646 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.646 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-16T17:17:33.646 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.646 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.646 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.646 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.647 INFO:teuthology.orchestra.run.smithi045.stdout: "periodic_async_work": { 2024-04-16T17:17:33.647 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "periodic_async_work", 2024-04-16T17:17:33.647 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:33.647 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.647 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:33.647 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:33.647 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.647 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.647 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.647 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Periodically check for async work", 2024-04-16T17:17:33.648 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.648 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.648 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.648 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.648 INFO:teuthology.orchestra.run.smithi045.stdout: "snapshot_clone_delay": { 2024-04-16T17:17:33.648 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "snapshot_clone_delay", 2024-04-16T17:17:33.648 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-16T17:17:33.648 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.648 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:33.648 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0", 2024-04-16T17:17:33.649 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.649 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.649 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.649 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-16T17:17:33.649 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.649 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.649 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.649 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.649 INFO:teuthology.orchestra.run.smithi045.stdout: "snapshot_clone_no_wait": { 2024-04-16T17:17:33.650 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "snapshot_clone_no_wait", 2024-04-16T17:17:33.650 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:33.650 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.650 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:33.650 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-16T17:17:33.650 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.650 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.650 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.651 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-16T17:17:33.651 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.651 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.651 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.651 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:17:33.651 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:17:33.651 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.651 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-16T17:17:33.652 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "zabbix", 2024-04-16T17:17:33.652 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-16T17:17:33.652 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-16T17:17:33.652 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-16T17:17:33.652 INFO:teuthology.orchestra.run.smithi045.stdout: "discovery_interval": { 2024-04-16T17:17:33.652 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "discovery_interval", 2024-04-16T17:17:33.652 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "uint", 2024-04-16T17:17:33.652 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.652 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:33.653 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "100", 2024-04-16T17:17:33.653 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.653 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.653 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.653 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.653 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.653 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.653 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.653 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.653 INFO:teuthology.orchestra.run.smithi045.stdout: "identifier": { 2024-04-16T17:17:33.654 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "identifier", 2024-04-16T17:17:33.654 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:33.654 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.654 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:33.654 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:33.654 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.654 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.654 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.654 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.655 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.655 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.655 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.655 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.655 INFO:teuthology.orchestra.run.smithi045.stdout: "interval": { 2024-04-16T17:17:33.655 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "interval", 2024-04-16T17:17:33.655 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-04-16T17:17:33.655 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.655 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:33.655 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "60", 2024-04-16T17:17:33.656 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.656 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.656 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.656 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.656 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.656 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.656 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.656 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.656 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-16T17:17:33.656 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-16T17:17:33.657 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:33.657 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.657 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:33.657 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:33.657 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.657 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.657 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-16T17:17:33.657 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-16T17:17:33.657 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-16T17:17:33.658 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-16T17:17:33.658 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-16T17:17:33.658 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-16T17:17:33.658 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-16T17:17:33.658 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-16T17:17:33.658 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.658 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.658 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.658 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.658 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.659 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-16T17:17:33.659 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-16T17:17:33.659 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:33.659 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.659 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:33.659 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:33.659 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.659 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.659 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.659 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.660 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.660 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.660 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.660 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.660 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-16T17:17:33.660 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-16T17:17:33.660 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:33.660 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.660 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:33.660 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-16T17:17:33.661 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.661 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.661 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-16T17:17:33.661 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-16T17:17:33.661 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-16T17:17:33.661 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-16T17:17:33.661 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-16T17:17:33.661 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-16T17:17:33.661 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-16T17:17:33.662 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-16T17:17:33.662 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.662 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.662 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.662 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.662 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.662 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-16T17:17:33.662 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-16T17:17:33.662 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-16T17:17:33.663 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.663 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-16T17:17:33.663 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-16T17:17:33.663 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.663 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.663 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.663 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.663 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.663 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.663 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.664 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.664 INFO:teuthology.orchestra.run.smithi045.stdout: "zabbix_host": { 2024-04-16T17:17:33.664 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "zabbix_host", 2024-04-16T17:17:33.664 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:33.664 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.664 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:33.664 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-16T17:17:33.664 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.664 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.664 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.665 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.665 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.665 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.665 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.665 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.665 INFO:teuthology.orchestra.run.smithi045.stdout: "zabbix_port": { 2024-04-16T17:17:33.665 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "zabbix_port", 2024-04-16T17:17:33.665 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-16T17:17:33.665 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.665 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:33.666 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "10051", 2024-04-16T17:17:33.666 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.666 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.666 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.666 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.666 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.666 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.666 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.666 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.666 INFO:teuthology.orchestra.run.smithi045.stdout: "zabbix_sender": { 2024-04-16T17:17:33.667 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "zabbix_sender", 2024-04-16T17:17:33.667 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-16T17:17:33.667 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-16T17:17:33.667 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-16T17:17:33.667 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-16T17:17:33.667 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-16T17:17:33.667 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-16T17:17:33.667 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-16T17:17:33.667 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-16T17:17:33.668 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-16T17:17:33.668 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-16T17:17:33.668 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-16T17:17:33.668 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:17:33.668 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:17:33.668 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:17:33.668 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-16T17:17:33.668 INFO:teuthology.orchestra.run.smithi045.stdout: "services": { 2024-04-16T17:17:33.668 INFO:teuthology.orchestra.run.smithi045.stdout: "dashboard": "https://172.21.15.45:8443/" 2024-04-16T17:17:33.668 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.669 INFO:teuthology.orchestra.run.smithi045.stdout: "always_on_modules": { 2024-04-16T17:17:33.669 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus": [ 2024-04-16T17:17:33.669 INFO:teuthology.orchestra.run.smithi045.stdout: "balancer", 2024-04-16T17:17:33.669 INFO:teuthology.orchestra.run.smithi045.stdout: "crash", 2024-04-16T17:17:33.669 INFO:teuthology.orchestra.run.smithi045.stdout: "devicehealth", 2024-04-16T17:17:33.669 INFO:teuthology.orchestra.run.smithi045.stdout: "orchestrator", 2024-04-16T17:17:33.669 INFO:teuthology.orchestra.run.smithi045.stdout: "pg_autoscaler", 2024-04-16T17:17:33.669 INFO:teuthology.orchestra.run.smithi045.stdout: "progress", 2024-04-16T17:17:33.669 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_support", 2024-04-16T17:17:33.669 INFO:teuthology.orchestra.run.smithi045.stdout: "status", 2024-04-16T17:17:33.670 INFO:teuthology.orchestra.run.smithi045.stdout: "telemetry", 2024-04-16T17:17:33.670 INFO:teuthology.orchestra.run.smithi045.stdout: "volumes" 2024-04-16T17:17:33.670 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-16T17:17:33.670 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific": [ 2024-04-16T17:17:33.670 INFO:teuthology.orchestra.run.smithi045.stdout: "balancer", 2024-04-16T17:17:33.670 INFO:teuthology.orchestra.run.smithi045.stdout: "crash", 2024-04-16T17:17:33.670 INFO:teuthology.orchestra.run.smithi045.stdout: "devicehealth", 2024-04-16T17:17:33.670 INFO:teuthology.orchestra.run.smithi045.stdout: "orchestrator", 2024-04-16T17:17:33.670 INFO:teuthology.orchestra.run.smithi045.stdout: "pg_autoscaler", 2024-04-16T17:17:33.671 INFO:teuthology.orchestra.run.smithi045.stdout: "progress", 2024-04-16T17:17:33.671 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_support", 2024-04-16T17:17:33.671 INFO:teuthology.orchestra.run.smithi045.stdout: "status", 2024-04-16T17:17:33.671 INFO:teuthology.orchestra.run.smithi045.stdout: "telemetry", 2024-04-16T17:17:33.671 INFO:teuthology.orchestra.run.smithi045.stdout: "volumes" 2024-04-16T17:17:33.671 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-16T17:17:33.671 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy": [ 2024-04-16T17:17:33.671 INFO:teuthology.orchestra.run.smithi045.stdout: "balancer", 2024-04-16T17:17:33.671 INFO:teuthology.orchestra.run.smithi045.stdout: "crash", 2024-04-16T17:17:33.671 INFO:teuthology.orchestra.run.smithi045.stdout: "devicehealth", 2024-04-16T17:17:33.672 INFO:teuthology.orchestra.run.smithi045.stdout: "orchestrator", 2024-04-16T17:17:33.672 INFO:teuthology.orchestra.run.smithi045.stdout: "pg_autoscaler", 2024-04-16T17:17:33.672 INFO:teuthology.orchestra.run.smithi045.stdout: "progress", 2024-04-16T17:17:33.672 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_support", 2024-04-16T17:17:33.672 INFO:teuthology.orchestra.run.smithi045.stdout: "status", 2024-04-16T17:17:33.672 INFO:teuthology.orchestra.run.smithi045.stdout: "telemetry", 2024-04-16T17:17:33.672 INFO:teuthology.orchestra.run.smithi045.stdout: "volumes" 2024-04-16T17:17:33.672 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-16T17:17:33.672 INFO:teuthology.orchestra.run.smithi045.stdout: "reef": [ 2024-04-16T17:17:33.672 INFO:teuthology.orchestra.run.smithi045.stdout: "balancer", 2024-04-16T17:17:33.673 INFO:teuthology.orchestra.run.smithi045.stdout: "crash", 2024-04-16T17:17:33.673 INFO:teuthology.orchestra.run.smithi045.stdout: "devicehealth", 2024-04-16T17:17:33.673 INFO:teuthology.orchestra.run.smithi045.stdout: "orchestrator", 2024-04-16T17:17:33.673 INFO:teuthology.orchestra.run.smithi045.stdout: "pg_autoscaler", 2024-04-16T17:17:33.673 INFO:teuthology.orchestra.run.smithi045.stdout: "progress", 2024-04-16T17:17:33.673 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_support", 2024-04-16T17:17:33.673 INFO:teuthology.orchestra.run.smithi045.stdout: "status", 2024-04-16T17:17:33.673 INFO:teuthology.orchestra.run.smithi045.stdout: "telemetry", 2024-04-16T17:17:33.674 INFO:teuthology.orchestra.run.smithi045.stdout: "volumes" 2024-04-16T17:17:33.674 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-16T17:17:33.674 INFO:teuthology.orchestra.run.smithi045.stdout: "squid": [ 2024-04-16T17:17:33.674 INFO:teuthology.orchestra.run.smithi045.stdout: "balancer", 2024-04-16T17:17:33.674 INFO:teuthology.orchestra.run.smithi045.stdout: "crash", 2024-04-16T17:17:33.674 INFO:teuthology.orchestra.run.smithi045.stdout: "devicehealth", 2024-04-16T17:17:33.674 INFO:teuthology.orchestra.run.smithi045.stdout: "orchestrator", 2024-04-16T17:17:33.674 INFO:teuthology.orchestra.run.smithi045.stdout: "pg_autoscaler", 2024-04-16T17:17:33.674 INFO:teuthology.orchestra.run.smithi045.stdout: "progress", 2024-04-16T17:17:33.675 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_support", 2024-04-16T17:17:33.675 INFO:teuthology.orchestra.run.smithi045.stdout: "status", 2024-04-16T17:17:33.675 INFO:teuthology.orchestra.run.smithi045.stdout: "telemetry", 2024-04-16T17:17:33.675 INFO:teuthology.orchestra.run.smithi045.stdout: "volumes" 2024-04-16T17:17:33.675 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-04-16T17:17:33.675 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.675 INFO:teuthology.orchestra.run.smithi045.stdout: "last_failure_osd_epoch": 95, 2024-04-16T17:17:33.675 INFO:teuthology.orchestra.run.smithi045.stdout: "active_clients": [ 2024-04-16T17:17:33.675 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-16T17:17:33.676 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "devicehealth", 2024-04-16T17:17:33.676 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-04-16T17:17:33.676 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-16T17:17:33.676 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v2", 2024-04-16T17:17:33.676 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:0", 2024-04-16T17:17:33.676 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 3873430409 2024-04-16T17:17:33.676 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:17:33.676 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-04-16T17:17:33.676 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.676 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-16T17:17:33.677 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "libcephsqlite", 2024-04-16T17:17:33.677 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-04-16T17:17:33.677 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-16T17:17:33.677 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v2", 2024-04-16T17:17:33.677 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:0", 2024-04-16T17:17:33.677 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 400136323 2024-04-16T17:17:33.677 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:17:33.677 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-04-16T17:17:33.677 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.677 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-16T17:17:33.678 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rbd_support", 2024-04-16T17:17:33.678 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-04-16T17:17:33.678 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-16T17:17:33.678 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v2", 2024-04-16T17:17:33.678 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:0", 2024-04-16T17:17:33.678 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 3189946854 2024-04-16T17:17:33.678 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:17:33.678 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-04-16T17:17:33.678 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.678 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-16T17:17:33.679 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "snap_schedule", 2024-04-16T17:17:33.679 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-04-16T17:17:33.679 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-16T17:17:33.679 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v2", 2024-04-16T17:17:33.679 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:0", 2024-04-16T17:17:33.679 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 1515811809 2024-04-16T17:17:33.679 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:17:33.679 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-04-16T17:17:33.679 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.680 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-16T17:17:33.680 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "snap_schedule", 2024-04-16T17:17:33.680 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-04-16T17:17:33.680 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-16T17:17:33.680 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v2", 2024-04-16T17:17:33.680 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:0", 2024-04-16T17:17:33.680 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 194342036 2024-04-16T17:17:33.680 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:17:33.680 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-04-16T17:17:33.680 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:33.681 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-16T17:17:33.681 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "volumes", 2024-04-16T17:17:33.681 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-04-16T17:17:33.681 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-16T17:17:33.681 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v2", 2024-04-16T17:17:33.681 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:0", 2024-04-16T17:17:33.681 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 4115866412 2024-04-16T17:17:33.681 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:17:33.681 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-04-16T17:17:33.681 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:17:33.682 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-04-16T17:17:33.682 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-16T17:17:33.689 DEBUG:tasks.check_counter:Getting stats from a 2024-04-16T17:17:33.689 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45 shell --fsid b1fecd38-fc11-11ee-bc90-c7b262605968 -- ceph daemon mds.a perf dump 2024-04-16T17:17:33.690 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:17:32 smithi045 ceph-mon[31676]: pgmap v337: 129 pgs: 129 active+clean; 55 MiB data, 874 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-16T17:17:33.690 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:17:32 smithi045 ceph-mon[31676]: from='client.? 172.21.15.45:0/4190718114' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-16T17:17:34.190 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/mon.a/config 2024-04-16T17:17:34.600 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:17:34 smithi045 ceph-mon[31676]: pgmap v338: 129 pgs: 129 active+clean; 55 MiB data, 874 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T17:17:34.804 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:17:34 smithi117 ceph-mon[36730]: pgmap v338: 129 pgs: 129 active+clean; 55 MiB data, 874 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T17:17:34.926 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:17:34 smithi104 ceph-mon[36457]: pgmap v338: 129 pgs: 129 active+clean; 55 MiB data, 874 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T17:17:36.573 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:17:36 smithi045 ceph-mon[31676]: pgmap v339: 129 pgs: 129 active+clean; 55 MiB data, 874 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T17:17:36.699 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-16T17:17:36.700 INFO:teuthology.orchestra.run.smithi045.stdout: "AsyncMessenger::Worker-0": { 2024-04-16T17:17:36.700 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_messages": 14315, 2024-04-16T17:17:36.700 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages": 18814, 2024-04-16T17:17:36.700 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_bytes": 4666710, 2024-04-16T17:17:36.700 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_bytes": 29129483, 2024-04-16T17:17:36.700 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_created_connections": 56, 2024-04-16T17:17:36.700 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_active_connections": 5, 2024-04-16T17:17:36.701 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_total_time": 4.547166671, 2024-04-16T17:17:36.701 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_send_time": 2.968674126, 2024-04-16T17:17:36.701 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_recv_time": 1.795062893, 2024-04-16T17:17:36.701 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_fast_dispatch_time": 0.119716667, 2024-04-16T17:17:36.701 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages_queue_lat": { 2024-04-16T17:17:36.701 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 18808, 2024-04-16T17:17:36.701 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 1.888889592, 2024-04-16T17:17:36.701 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000100430 2024-04-16T17:17:36.701 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:36.702 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_handle_ack_lat": { 2024-04-16T17:17:36.702 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 24718, 2024-04-16T17:17:36.702 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.011243814, 2024-04-16T17:17:36.702 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000454 2024-04-16T17:17:36.702 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:36.702 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_encrypted_bytes": 5504, 2024-04-16T17:17:36.702 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_encrypted_bytes": 3104 2024-04-16T17:17:36.702 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:36.702 INFO:teuthology.orchestra.run.smithi045.stdout: "AsyncMessenger::Worker-1": { 2024-04-16T17:17:36.703 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_messages": 11870, 2024-04-16T17:17:36.703 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages": 10372, 2024-04-16T17:17:36.703 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_bytes": 4021100, 2024-04-16T17:17:36.703 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_bytes": 3380064, 2024-04-16T17:17:36.703 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_created_connections": 11, 2024-04-16T17:17:36.703 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_active_connections": 5, 2024-04-16T17:17:36.703 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_total_time": 1.794007965, 2024-04-16T17:17:36.703 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_send_time": 0.835444717, 2024-04-16T17:17:36.703 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_recv_time": 1.174251307, 2024-04-16T17:17:36.704 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_fast_dispatch_time": 0.028433261, 2024-04-16T17:17:36.704 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages_queue_lat": { 2024-04-16T17:17:36.704 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 10374, 2024-04-16T17:17:36.704 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.849861583, 2024-04-16T17:17:36.704 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000081922 2024-04-16T17:17:36.704 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:36.704 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_handle_ack_lat": { 2024-04-16T17:17:36.704 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 20745, 2024-04-16T17:17:36.704 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.007094201, 2024-04-16T17:17:36.704 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000341 2024-04-16T17:17:36.705 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:36.705 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-16T17:17:36.705 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_encrypted_bytes": 469760 2024-04-16T17:17:36.705 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:36.705 INFO:teuthology.orchestra.run.smithi045.stdout: "AsyncMessenger::Worker-2": { 2024-04-16T17:17:36.705 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_messages": 3848, 2024-04-16T17:17:36.705 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages": 3315, 2024-04-16T17:17:36.705 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_bytes": 3124535, 2024-04-16T17:17:36.705 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_bytes": 1274694, 2024-04-16T17:17:36.705 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_created_connections": 7, 2024-04-16T17:17:36.706 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_active_connections": 6, 2024-04-16T17:17:36.706 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_total_time": 0.626386698, 2024-04-16T17:17:36.706 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_send_time": 0.261589908, 2024-04-16T17:17:36.706 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_recv_time": 0.402920850, 2024-04-16T17:17:36.706 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_fast_dispatch_time": 0.055597675, 2024-04-16T17:17:36.706 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages_queue_lat": { 2024-04-16T17:17:36.706 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 3314, 2024-04-16T17:17:36.706 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.373409429, 2024-04-16T17:17:36.706 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000112676 2024-04-16T17:17:36.706 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:36.707 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_handle_ack_lat": { 2024-04-16T17:17:36.707 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 4101, 2024-04-16T17:17:36.707 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.001480488, 2024-04-16T17:17:36.707 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000361 2024-04-16T17:17:36.707 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:36.707 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_encrypted_bytes": 1727168, 2024-04-16T17:17:36.707 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_encrypted_bytes": 165616 2024-04-16T17:17:36.707 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:36.707 INFO:teuthology.orchestra.run.smithi045.stdout: "cct": { 2024-04-16T17:17:36.707 INFO:teuthology.orchestra.run.smithi045.stdout: "total_workers": 1, 2024-04-16T17:17:36.708 INFO:teuthology.orchestra.run.smithi045.stdout: "unhealthy_workers": 0 2024-04-16T17:17:36.708 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:36.708 INFO:teuthology.orchestra.run.smithi045.stdout: "finisher-MDSRank": { 2024-04-16T17:17:36.708 INFO:teuthology.orchestra.run.smithi045.stdout: "queue_len": 0, 2024-04-16T17:17:36.708 INFO:teuthology.orchestra.run.smithi045.stdout: "complete_latency": { 2024-04-16T17:17:36.708 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 3852, 2024-04-16T17:17:36.708 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 3.783827872, 2024-04-16T17:17:36.708 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000982302 2024-04-16T17:17:36.708 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:17:36.708 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:36.709 INFO:teuthology.orchestra.run.smithi045.stdout: "finisher-PurgeQueue": { 2024-04-16T17:17:36.709 INFO:teuthology.orchestra.run.smithi045.stdout: "queue_len": 0, 2024-04-16T17:17:36.709 INFO:teuthology.orchestra.run.smithi045.stdout: "complete_latency": { 2024-04-16T17:17:36.709 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 1503, 2024-04-16T17:17:36.709 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.218021869, 2024-04-16T17:17:36.709 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000145057 2024-04-16T17:17:36.709 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:17:36.709 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:36.709 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": { 2024-04-16T17:17:36.710 INFO:teuthology.orchestra.run.smithi045.stdout: "request": 7334, 2024-04-16T17:17:36.710 INFO:teuthology.orchestra.run.smithi045.stdout: "reply": 7135, 2024-04-16T17:17:36.710 INFO:teuthology.orchestra.run.smithi045.stdout: "reply_latency": { 2024-04-16T17:17:36.710 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 7135, 2024-04-16T17:17:36.710 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 91.688434832, 2024-04-16T17:17:36.710 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.012850516 2024-04-16T17:17:36.710 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:36.710 INFO:teuthology.orchestra.run.smithi045.stdout: "slow_reply": 0, 2024-04-16T17:17:36.710 INFO:teuthology.orchestra.run.smithi045.stdout: "forward": 199, 2024-04-16T17:17:36.710 INFO:teuthology.orchestra.run.smithi045.stdout: "dir_fetch_complete": 4, 2024-04-16T17:17:36.711 INFO:teuthology.orchestra.run.smithi045.stdout: "dir_fetch_keys": 0, 2024-04-16T17:17:36.711 INFO:teuthology.orchestra.run.smithi045.stdout: "dir_commit": 19, 2024-04-16T17:17:36.711 INFO:teuthology.orchestra.run.smithi045.stdout: "dir_split": 1, 2024-04-16T17:17:36.711 INFO:teuthology.orchestra.run.smithi045.stdout: "dir_merge": 0, 2024-04-16T17:17:36.711 INFO:teuthology.orchestra.run.smithi045.stdout: "inodes": 243, 2024-04-16T17:17:36.711 INFO:teuthology.orchestra.run.smithi045.stdout: "inodes_top": 60, 2024-04-16T17:17:36.711 INFO:teuthology.orchestra.run.smithi045.stdout: "inodes_bottom": 170, 2024-04-16T17:17:36.711 INFO:teuthology.orchestra.run.smithi045.stdout: "inodes_pin_tail": 13, 2024-04-16T17:17:36.711 INFO:teuthology.orchestra.run.smithi045.stdout: "inodes_pinned": 158, 2024-04-16T17:17:36.712 INFO:teuthology.orchestra.run.smithi045.stdout: "inodes_expired": 148, 2024-04-16T17:17:36.712 INFO:teuthology.orchestra.run.smithi045.stdout: "inodes_with_caps": 22, 2024-04-16T17:17:36.712 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": 22, 2024-04-16T17:17:36.712 INFO:teuthology.orchestra.run.smithi045.stdout: "subtrees": 11, 2024-04-16T17:17:36.712 INFO:teuthology.orchestra.run.smithi045.stdout: "traverse": 13457, 2024-04-16T17:17:36.712 INFO:teuthology.orchestra.run.smithi045.stdout: "traverse_hit": 7138, 2024-04-16T17:17:36.712 INFO:teuthology.orchestra.run.smithi045.stdout: "traverse_forward": 193, 2024-04-16T17:17:36.712 INFO:teuthology.orchestra.run.smithi045.stdout: "traverse_discover": 19, 2024-04-16T17:17:36.712 INFO:teuthology.orchestra.run.smithi045.stdout: "traverse_dir_fetch": 0, 2024-04-16T17:17:36.712 INFO:teuthology.orchestra.run.smithi045.stdout: "traverse_remote_ino": 0, 2024-04-16T17:17:36.713 INFO:teuthology.orchestra.run.smithi045.stdout: "traverse_lock": 141, 2024-04-16T17:17:36.713 INFO:teuthology.orchestra.run.smithi045.stdout: "load_cent": 0, 2024-04-16T17:17:36.713 INFO:teuthology.orchestra.run.smithi045.stdout: "q": 0, 2024-04-16T17:17:36.713 INFO:teuthology.orchestra.run.smithi045.stdout: "exported": 5, 2024-04-16T17:17:36.713 INFO:teuthology.orchestra.run.smithi045.stdout: "exported_inodes": 68, 2024-04-16T17:17:36.713 INFO:teuthology.orchestra.run.smithi045.stdout: "imported": 4, 2024-04-16T17:17:36.713 INFO:teuthology.orchestra.run.smithi045.stdout: "imported_inodes": 137, 2024-04-16T17:17:36.713 INFO:teuthology.orchestra.run.smithi045.stdout: "openino_dir_fetch": 0, 2024-04-16T17:17:36.713 INFO:teuthology.orchestra.run.smithi045.stdout: "openino_backtrace_fetch": 503, 2024-04-16T17:17:36.713 INFO:teuthology.orchestra.run.smithi045.stdout: "openino_peer_discover": 0, 2024-04-16T17:17:36.714 INFO:teuthology.orchestra.run.smithi045.stdout: "root_rfiles": 68, 2024-04-16T17:17:36.714 INFO:teuthology.orchestra.run.smithi045.stdout: "root_rbytes": 21196213, 2024-04-16T17:17:36.714 INFO:teuthology.orchestra.run.smithi045.stdout: "root_rsnaps": 6, 2024-04-16T17:17:36.714 INFO:teuthology.orchestra.run.smithi045.stdout: "scrub_backtrace_fetch": 503, 2024-04-16T17:17:36.714 INFO:teuthology.orchestra.run.smithi045.stdout: "scrub_set_tag": 0, 2024-04-16T17:17:36.714 INFO:teuthology.orchestra.run.smithi045.stdout: "scrub_backtrace_repaired": 0, 2024-04-16T17:17:36.714 INFO:teuthology.orchestra.run.smithi045.stdout: "scrub_inotable_repaired": 0, 2024-04-16T17:17:36.714 INFO:teuthology.orchestra.run.smithi045.stdout: "scrub_dir_inodes": 0, 2024-04-16T17:17:36.714 INFO:teuthology.orchestra.run.smithi045.stdout: "scrub_dir_base_inodes": 0, 2024-04-16T17:17:36.715 INFO:teuthology.orchestra.run.smithi045.stdout: "scrub_dirfrag_rstats": 0, 2024-04-16T17:17:36.715 INFO:teuthology.orchestra.run.smithi045.stdout: "scrub_file_inodes": 503, 2024-04-16T17:17:36.715 INFO:teuthology.orchestra.run.smithi045.stdout: "handle_inode_file_caps": 0, 2024-04-16T17:17:36.715 INFO:teuthology.orchestra.run.smithi045.stdout: "ceph_cap_op_revoke": 990, 2024-04-16T17:17:36.715 INFO:teuthology.orchestra.run.smithi045.stdout: "ceph_cap_op_grant": 4719, 2024-04-16T17:17:36.715 INFO:teuthology.orchestra.run.smithi045.stdout: "ceph_cap_op_trunc": 0, 2024-04-16T17:17:36.715 INFO:teuthology.orchestra.run.smithi045.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-16T17:17:36.715 INFO:teuthology.orchestra.run.smithi045.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-16T17:17:36.715 INFO:teuthology.orchestra.run.smithi045.stdout: "handle_client_caps": 3170, 2024-04-16T17:17:36.716 INFO:teuthology.orchestra.run.smithi045.stdout: "handle_client_caps_dirty": 893, 2024-04-16T17:17:36.716 INFO:teuthology.orchestra.run.smithi045.stdout: "handle_client_cap_release": 494, 2024-04-16T17:17:36.716 INFO:teuthology.orchestra.run.smithi045.stdout: "process_request_cap_release": 6834 2024-04-16T17:17:36.716 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:36.716 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_cache": { 2024-04-16T17:17:36.716 INFO:teuthology.orchestra.run.smithi045.stdout: "dir_update": 2, 2024-04-16T17:17:36.716 INFO:teuthology.orchestra.run.smithi045.stdout: "dir_update_receipt": 3, 2024-04-16T17:17:36.716 INFO:teuthology.orchestra.run.smithi045.stdout: "dir_try_discover": 2, 2024-04-16T17:17:36.716 INFO:teuthology.orchestra.run.smithi045.stdout: "dir_send_discover": 21, 2024-04-16T17:17:36.717 INFO:teuthology.orchestra.run.smithi045.stdout: "dir_handle_discover": 11, 2024-04-16T17:17:36.717 INFO:teuthology.orchestra.run.smithi045.stdout: "num_strays": 0, 2024-04-16T17:17:36.717 INFO:teuthology.orchestra.run.smithi045.stdout: "num_strays_delayed": 0, 2024-04-16T17:17:36.717 INFO:teuthology.orchestra.run.smithi045.stdout: "num_strays_enqueuing": 0, 2024-04-16T17:17:36.717 INFO:teuthology.orchestra.run.smithi045.stdout: "strays_created": 710, 2024-04-16T17:17:36.717 INFO:teuthology.orchestra.run.smithi045.stdout: "strays_enqueued": 710, 2024-04-16T17:17:36.717 INFO:teuthology.orchestra.run.smithi045.stdout: "strays_reintegrated": 0, 2024-04-16T17:17:36.717 INFO:teuthology.orchestra.run.smithi045.stdout: "strays_migrated": 0, 2024-04-16T17:17:36.717 INFO:teuthology.orchestra.run.smithi045.stdout: "num_recovering_processing": 0, 2024-04-16T17:17:36.717 INFO:teuthology.orchestra.run.smithi045.stdout: "num_recovering_enqueued": 0, 2024-04-16T17:17:36.717 INFO:teuthology.orchestra.run.smithi045.stdout: "num_recovering_prioritized": 0, 2024-04-16T17:17:36.718 INFO:teuthology.orchestra.run.smithi045.stdout: "recovery_started": 0, 2024-04-16T17:17:36.718 INFO:teuthology.orchestra.run.smithi045.stdout: "recovery_completed": 0, 2024-04-16T17:17:36.718 INFO:teuthology.orchestra.run.smithi045.stdout: "ireq_quiesce_path": 0, 2024-04-16T17:17:36.718 INFO:teuthology.orchestra.run.smithi045.stdout: "ireq_quiesce_inode": 0, 2024-04-16T17:17:36.718 INFO:teuthology.orchestra.run.smithi045.stdout: "ireq_enqueue_scrub": 0, 2024-04-16T17:17:36.718 INFO:teuthology.orchestra.run.smithi045.stdout: "ireq_exportdir": 11, 2024-04-16T17:17:36.718 INFO:teuthology.orchestra.run.smithi045.stdout: "ireq_flush": 0, 2024-04-16T17:17:36.718 INFO:teuthology.orchestra.run.smithi045.stdout: "ireq_fragmentdir": 1, 2024-04-16T17:17:36.718 INFO:teuthology.orchestra.run.smithi045.stdout: "ireq_fragstats": 0, 2024-04-16T17:17:36.719 INFO:teuthology.orchestra.run.smithi045.stdout: "ireq_inodestats": 0 2024-04-16T17:17:36.719 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:36.719 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_log": { 2024-04-16T17:17:36.719 INFO:teuthology.orchestra.run.smithi045.stdout: "evlrg": 0, 2024-04-16T17:17:36.719 INFO:teuthology.orchestra.run.smithi045.stdout: "evadd": 7979, 2024-04-16T17:17:36.719 INFO:teuthology.orchestra.run.smithi045.stdout: "evex": 0, 2024-04-16T17:17:36.719 INFO:teuthology.orchestra.run.smithi045.stdout: "evtrm": 0, 2024-04-16T17:17:36.719 INFO:teuthology.orchestra.run.smithi045.stdout: "ev": 7979, 2024-04-16T17:17:36.719 INFO:teuthology.orchestra.run.smithi045.stdout: "evexg": 0, 2024-04-16T17:17:36.719 INFO:teuthology.orchestra.run.smithi045.stdout: "evexd": 0, 2024-04-16T17:17:36.720 INFO:teuthology.orchestra.run.smithi045.stdout: "segadd": 10, 2024-04-16T17:17:36.720 INFO:teuthology.orchestra.run.smithi045.stdout: "segex": 0, 2024-04-16T17:17:36.720 INFO:teuthology.orchestra.run.smithi045.stdout: "segtrm": 0, 2024-04-16T17:17:36.720 INFO:teuthology.orchestra.run.smithi045.stdout: "seg": 10, 2024-04-16T17:17:36.720 INFO:teuthology.orchestra.run.smithi045.stdout: "segmjr": 2, 2024-04-16T17:17:36.720 INFO:teuthology.orchestra.run.smithi045.stdout: "segexg": 0, 2024-04-16T17:17:36.720 INFO:teuthology.orchestra.run.smithi045.stdout: "segexd": 0, 2024-04-16T17:17:36.720 INFO:teuthology.orchestra.run.smithi045.stdout: "expos": 4194304, 2024-04-16T17:17:36.720 INFO:teuthology.orchestra.run.smithi045.stdout: "wrpos": 14562744, 2024-04-16T17:17:36.720 INFO:teuthology.orchestra.run.smithi045.stdout: "rdpos": 0, 2024-04-16T17:17:36.721 INFO:teuthology.orchestra.run.smithi045.stdout: "jlat": { 2024-04-16T17:17:36.721 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 1679, 2024-04-16T17:17:36.721 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 3.893404543, 2024-04-16T17:17:36.721 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.002318882 2024-04-16T17:17:36.721 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:36.721 INFO:teuthology.orchestra.run.smithi045.stdout: "replayed": 0 2024-04-16T17:17:36.721 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:36.721 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_mem": { 2024-04-16T17:17:36.721 INFO:teuthology.orchestra.run.smithi045.stdout: "ino": 222, 2024-04-16T17:17:36.722 INFO:teuthology.orchestra.run.smithi045.stdout: "ino+": 1040, 2024-04-16T17:17:36.722 INFO:teuthology.orchestra.run.smithi045.stdout: "ino-": 818, 2024-04-16T17:17:36.722 INFO:teuthology.orchestra.run.smithi045.stdout: "dir": 41, 2024-04-16T17:17:36.722 INFO:teuthology.orchestra.run.smithi045.stdout: "dir+": 43, 2024-04-16T17:17:36.722 INFO:teuthology.orchestra.run.smithi045.stdout: "dir-": 2, 2024-04-16T17:17:36.722 INFO:teuthology.orchestra.run.smithi045.stdout: "dn": 243, 2024-04-16T17:17:36.722 INFO:teuthology.orchestra.run.smithi045.stdout: "dn+": 1687, 2024-04-16T17:17:36.722 INFO:teuthology.orchestra.run.smithi045.stdout: "dn-": 1444, 2024-04-16T17:17:36.722 INFO:teuthology.orchestra.run.smithi045.stdout: "cap": 22, 2024-04-16T17:17:36.723 INFO:teuthology.orchestra.run.smithi045.stdout: "cap+": 1146, 2024-04-16T17:17:36.723 INFO:teuthology.orchestra.run.smithi045.stdout: "cap-": 1124, 2024-04-16T17:17:36.723 INFO:teuthology.orchestra.run.smithi045.stdout: "rss": 59468, 2024-04-16T17:17:36.723 INFO:teuthology.orchestra.run.smithi045.stdout: "heap": 223608 2024-04-16T17:17:36.723 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:36.723 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_server": { 2024-04-16T17:17:36.723 INFO:teuthology.orchestra.run.smithi045.stdout: "dispatch_client_request": 11926, 2024-04-16T17:17:36.723 INFO:teuthology.orchestra.run.smithi045.stdout: "dispatch_server_request": 33, 2024-04-16T17:17:36.723 INFO:teuthology.orchestra.run.smithi045.stdout: "handle_client_request": 7334, 2024-04-16T17:17:36.724 INFO:teuthology.orchestra.run.smithi045.stdout: "handle_client_session": 97, 2024-04-16T17:17:36.724 INFO:teuthology.orchestra.run.smithi045.stdout: "handle_peer_request": 1627, 2024-04-16T17:17:36.724 INFO:teuthology.orchestra.run.smithi045.stdout: "req_create_latency": { 2024-04-16T17:17:36.724 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 711, 2024-04-16T17:17:36.724 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 8.085493747, 2024-04-16T17:17:36.724 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.011372002 2024-04-16T17:17:36.724 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:36.724 INFO:teuthology.orchestra.run.smithi045.stdout: "req_getattr_latency": { 2024-04-16T17:17:36.724 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 9, 2024-04-16T17:17:36.724 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.018682649, 2024-04-16T17:17:36.725 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.002075849 2024-04-16T17:17:36.725 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:36.725 INFO:teuthology.orchestra.run.smithi045.stdout: "req_getfilelock_latency": { 2024-04-16T17:17:36.725 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-16T17:17:36.725 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-16T17:17:36.725 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-16T17:17:36.725 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:36.725 INFO:teuthology.orchestra.run.smithi045.stdout: "req_link_latency": { 2024-04-16T17:17:36.725 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-16T17:17:36.725 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-16T17:17:36.726 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-16T17:17:36.726 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:36.726 INFO:teuthology.orchestra.run.smithi045.stdout: "req_lookup_latency": { 2024-04-16T17:17:36.726 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 1213, 2024-04-16T17:17:36.726 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 67.484862902, 2024-04-16T17:17:36.726 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.055634676 2024-04-16T17:17:36.726 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:36.726 INFO:teuthology.orchestra.run.smithi045.stdout: "req_lookuphash_latency": { 2024-04-16T17:17:36.726 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-16T17:17:36.727 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-16T17:17:36.727 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-16T17:17:36.727 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:36.727 INFO:teuthology.orchestra.run.smithi045.stdout: "req_lookupino_latency": { 2024-04-16T17:17:36.727 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-16T17:17:36.727 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-16T17:17:36.727 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-16T17:17:36.727 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:36.727 INFO:teuthology.orchestra.run.smithi045.stdout: "req_lookupname_latency": { 2024-04-16T17:17:36.727 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-16T17:17:36.728 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-16T17:17:36.728 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-16T17:17:36.728 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:36.728 INFO:teuthology.orchestra.run.smithi045.stdout: "req_lookupparent_latency": { 2024-04-16T17:17:36.728 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-16T17:17:36.728 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-16T17:17:36.728 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-16T17:17:36.728 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:36.728 INFO:teuthology.orchestra.run.smithi045.stdout: "req_lookupsnap_latency": { 2024-04-16T17:17:36.728 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-16T17:17:36.729 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-16T17:17:36.729 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-16T17:17:36.729 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:36.729 INFO:teuthology.orchestra.run.smithi045.stdout: "req_lssnap_latency": { 2024-04-16T17:17:36.729 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-16T17:17:36.730 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-16T17:17:36.730 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-16T17:17:36.730 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:36.730 INFO:teuthology.orchestra.run.smithi045.stdout: "req_mkdir_latency": { 2024-04-16T17:17:36.730 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-16T17:17:36.730 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-16T17:17:36.730 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-16T17:17:36.730 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:36.730 INFO:teuthology.orchestra.run.smithi045.stdout: "req_mknod_latency": { 2024-04-16T17:17:36.731 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-16T17:17:36.731 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-16T17:17:36.731 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-16T17:17:36.731 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:36.731 INFO:teuthology.orchestra.run.smithi045.stdout: "req_mksnap_latency": { 2024-04-16T17:17:36.731 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-16T17:17:36.731 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-16T17:17:36.731 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-16T17:17:36.731 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:36.731 INFO:teuthology.orchestra.run.smithi045.stdout: "req_open_latency": { 2024-04-16T17:17:36.732 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-16T17:17:36.732 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-16T17:17:36.732 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-16T17:17:36.732 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:36.732 INFO:teuthology.orchestra.run.smithi045.stdout: "req_readdir_latency": { 2024-04-16T17:17:36.732 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 4364, 2024-04-16T17:17:36.732 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 4.143126984, 2024-04-16T17:17:36.732 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000949387 2024-04-16T17:17:36.732 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:36.733 INFO:teuthology.orchestra.run.smithi045.stdout: "req_rename_latency": { 2024-04-16T17:17:36.733 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 128, 2024-04-16T17:17:36.733 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.751591126, 2024-04-16T17:17:36.733 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.005871805 2024-04-16T17:17:36.733 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:36.733 INFO:teuthology.orchestra.run.smithi045.stdout: "req_renamesnap_latency": { 2024-04-16T17:17:36.733 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-16T17:17:36.733 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-16T17:17:36.733 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-16T17:17:36.733 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:36.734 INFO:teuthology.orchestra.run.smithi045.stdout: "req_snapdiff_latency": { 2024-04-16T17:17:36.734 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-16T17:17:36.734 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-16T17:17:36.734 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-16T17:17:36.734 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:36.734 INFO:teuthology.orchestra.run.smithi045.stdout: "req_rmdir_latency": { 2024-04-16T17:17:36.734 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-16T17:17:36.734 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-16T17:17:36.737 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-16T17:17:36.738 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:36.738 INFO:teuthology.orchestra.run.smithi045.stdout: "req_rmsnap_latency": { 2024-04-16T17:17:36.738 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-16T17:17:36.738 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-16T17:17:36.738 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-16T17:17:36.738 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:36.738 INFO:teuthology.orchestra.run.smithi045.stdout: "req_rmxattr_latency": { 2024-04-16T17:17:36.738 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-16T17:17:36.739 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-16T17:17:36.739 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-16T17:17:36.739 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:36.739 INFO:teuthology.orchestra.run.smithi045.stdout: "req_setattr_latency": { 2024-04-16T17:17:36.739 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-16T17:17:36.739 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-16T17:17:36.739 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-16T17:17:36.739 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:36.739 INFO:teuthology.orchestra.run.smithi045.stdout: "req_setdirlayout_latency": { 2024-04-16T17:17:36.739 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-16T17:17:36.740 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-16T17:17:36.740 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-16T17:17:36.740 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:36.740 INFO:teuthology.orchestra.run.smithi045.stdout: "req_setfilelock_latency": { 2024-04-16T17:17:36.740 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-16T17:17:36.740 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-16T17:17:36.740 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-16T17:17:36.740 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:36.740 INFO:teuthology.orchestra.run.smithi045.stdout: "req_setlayout_latency": { 2024-04-16T17:17:36.740 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-16T17:17:36.741 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-16T17:17:36.741 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-16T17:17:36.741 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:36.741 INFO:teuthology.orchestra.run.smithi045.stdout: "req_setxattr_latency": { 2024-04-16T17:17:36.741 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-16T17:17:36.741 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-16T17:17:36.741 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-16T17:17:36.741 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:36.741 INFO:teuthology.orchestra.run.smithi045.stdout: "req_symlink_latency": { 2024-04-16T17:17:36.742 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-16T17:17:36.742 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-16T17:17:36.742 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-16T17:17:36.742 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:36.742 INFO:teuthology.orchestra.run.smithi045.stdout: "req_unlink_latency": { 2024-04-16T17:17:36.742 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 710, 2024-04-16T17:17:36.742 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 11.204677424, 2024-04-16T17:17:36.742 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.015781235 2024-04-16T17:17:36.742 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:36.742 INFO:teuthology.orchestra.run.smithi045.stdout: "cap_revoke_eviction": 0, 2024-04-16T17:17:36.743 INFO:teuthology.orchestra.run.smithi045.stdout: "cap_acquisition_throttle": 0, 2024-04-16T17:17:36.743 INFO:teuthology.orchestra.run.smithi045.stdout: "req_getvxattr_latency": { 2024-04-16T17:17:36.743 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-16T17:17:36.743 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-16T17:17:36.743 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-16T17:17:36.743 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:17:36.743 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:36.743 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_sessions": { 2024-04-16T17:17:36.743 INFO:teuthology.orchestra.run.smithi045.stdout: "session_count": 1, 2024-04-16T17:17:36.743 INFO:teuthology.orchestra.run.smithi045.stdout: "session_add": 3, 2024-04-16T17:17:36.744 INFO:teuthology.orchestra.run.smithi045.stdout: "session_remove": 2, 2024-04-16T17:17:36.744 INFO:teuthology.orchestra.run.smithi045.stdout: "sessions_open": 1, 2024-04-16T17:17:36.744 INFO:teuthology.orchestra.run.smithi045.stdout: "sessions_stale": 0, 2024-04-16T17:17:36.744 INFO:teuthology.orchestra.run.smithi045.stdout: "total_load": 2305, 2024-04-16T17:17:36.744 INFO:teuthology.orchestra.run.smithi045.stdout: "average_load": 2305, 2024-04-16T17:17:36.744 INFO:teuthology.orchestra.run.smithi045.stdout: "avg_session_uptime": 565, 2024-04-16T17:17:36.744 INFO:teuthology.orchestra.run.smithi045.stdout: "mdthresh_evicted": 0 2024-04-16T17:17:36.744 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:36.744 INFO:teuthology.orchestra.run.smithi045.stdout: "mempool": { 2024-04-16T17:17:36.745 INFO:teuthology.orchestra.run.smithi045.stdout: "bloom_filter_bytes": 0, 2024-04-16T17:17:36.745 INFO:teuthology.orchestra.run.smithi045.stdout: "bloom_filter_items": 0, 2024-04-16T17:17:36.745 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_alloc_bytes": 0, 2024-04-16T17:17:36.745 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_alloc_items": 0, 2024-04-16T17:17:36.745 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_data_bytes": 0, 2024-04-16T17:17:36.745 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_data_items": 0, 2024-04-16T17:17:36.745 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-16T17:17:36.745 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_onode_items": 0, 2024-04-16T17:17:36.745 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-16T17:17:36.745 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_meta_items": 0, 2024-04-16T17:17:36.746 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_other_bytes": 0, 2024-04-16T17:17:36.746 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_other_items": 0, 2024-04-16T17:17:36.746 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-16T17:17:36.746 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_buffer_items": 0, 2024-04-16T17:17:36.746 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_extent_bytes": 0, 2024-04-16T17:17:36.746 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_extent_items": 0, 2024-04-16T17:17:36.746 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_blob_bytes": 0, 2024-04-16T17:17:36.746 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_blob_items": 0, 2024-04-16T17:17:36.746 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-16T17:17:36.746 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_shared_blob_items": 0, 2024-04-16T17:17:36.747 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-16T17:17:36.747 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_inline_bl_items": 0, 2024-04-16T17:17:36.747 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_fsck_bytes": 0, 2024-04-16T17:17:36.747 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_fsck_items": 0, 2024-04-16T17:17:36.747 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_txc_bytes": 0, 2024-04-16T17:17:36.747 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_txc_items": 0, 2024-04-16T17:17:36.747 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-16T17:17:36.747 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_writing_deferred_items": 0, 2024-04-16T17:17:36.747 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_writing_bytes": 0, 2024-04-16T17:17:36.748 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_writing_items": 0, 2024-04-16T17:17:36.748 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_bytes": 0, 2024-04-16T17:17:36.748 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_items": 0, 2024-04-16T17:17:36.748 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_file_reader_bytes": 0, 2024-04-16T17:17:36.748 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_file_reader_items": 0, 2024-04-16T17:17:36.748 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_file_writer_bytes": 0, 2024-04-16T17:17:36.748 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_file_writer_items": 0, 2024-04-16T17:17:36.748 INFO:teuthology.orchestra.run.smithi045.stdout: "buffer_anon_bytes": 315620, 2024-04-16T17:17:36.748 INFO:teuthology.orchestra.run.smithi045.stdout: "buffer_anon_items": 227, 2024-04-16T17:17:36.748 INFO:teuthology.orchestra.run.smithi045.stdout: "buffer_meta_bytes": 352, 2024-04-16T17:17:36.749 INFO:teuthology.orchestra.run.smithi045.stdout: "buffer_meta_items": 4, 2024-04-16T17:17:36.749 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_bytes": 0, 2024-04-16T17:17:36.749 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_items": 0, 2024-04-16T17:17:36.749 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_mapbl_bytes": 0, 2024-04-16T17:17:36.749 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_mapbl_items": 0, 2024-04-16T17:17:36.749 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_pglog_bytes": 0, 2024-04-16T17:17:36.749 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_pglog_items": 0, 2024-04-16T17:17:36.749 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_bytes": 9512, 2024-04-16T17:17:36.749 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_items": 182, 2024-04-16T17:17:36.750 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_mapping_bytes": 0, 2024-04-16T17:17:36.750 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_mapping_items": 0, 2024-04-16T17:17:36.750 INFO:teuthology.orchestra.run.smithi045.stdout: "pgmap_bytes": 0, 2024-04-16T17:17:36.750 INFO:teuthology.orchestra.run.smithi045.stdout: "pgmap_items": 0, 2024-04-16T17:17:36.750 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_co_bytes": 783925, 2024-04-16T17:17:36.750 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_co_items": 5603, 2024-04-16T17:17:36.750 INFO:teuthology.orchestra.run.smithi045.stdout: "unittest_1_bytes": 0, 2024-04-16T17:17:36.750 INFO:teuthology.orchestra.run.smithi045.stdout: "unittest_1_items": 0, 2024-04-16T17:17:36.751 INFO:teuthology.orchestra.run.smithi045.stdout: "unittest_2_bytes": 0, 2024-04-16T17:17:36.751 INFO:teuthology.orchestra.run.smithi045.stdout: "unittest_2_items": 0 2024-04-16T17:17:36.751 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:36.751 INFO:teuthology.orchestra.run.smithi045.stdout: "objecter": { 2024-04-16T17:17:36.751 INFO:teuthology.orchestra.run.smithi045.stdout: "op_active": 0, 2024-04-16T17:17:36.751 INFO:teuthology.orchestra.run.smithi045.stdout: "op_laggy": 0, 2024-04-16T17:17:36.751 INFO:teuthology.orchestra.run.smithi045.stdout: "op_send": 4283, 2024-04-16T17:17:36.751 INFO:teuthology.orchestra.run.smithi045.stdout: "op_send_bytes": 10656007, 2024-04-16T17:17:36.751 INFO:teuthology.orchestra.run.smithi045.stdout: "op_resend": 33, 2024-04-16T17:17:36.751 INFO:teuthology.orchestra.run.smithi045.stdout: "op_reply": 4283, 2024-04-16T17:17:36.752 INFO:teuthology.orchestra.run.smithi045.stdout: "op_latency": { 2024-04-16T17:17:36.752 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 4283, 2024-04-16T17:17:36.752 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 16.058749761, 2024-04-16T17:17:36.752 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.003749416 2024-04-16T17:17:36.752 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:36.752 INFO:teuthology.orchestra.run.smithi045.stdout: "op_inflight": 0, 2024-04-16T17:17:36.752 INFO:teuthology.orchestra.run.smithi045.stdout: "oplen_avg": { 2024-04-16T17:17:36.752 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 4283, 2024-04-16T17:17:36.752 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 4619 2024-04-16T17:17:36.752 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:36.753 INFO:teuthology.orchestra.run.smithi045.stdout: "op": 4283, 2024-04-16T17:17:36.753 INFO:teuthology.orchestra.run.smithi045.stdout: "op_r": 654, 2024-04-16T17:17:36.753 INFO:teuthology.orchestra.run.smithi045.stdout: "op_w": 3629, 2024-04-16T17:17:36.753 INFO:teuthology.orchestra.run.smithi045.stdout: "op_rmw": 0, 2024-04-16T17:17:36.753 INFO:teuthology.orchestra.run.smithi045.stdout: "op_pg": 0, 2024-04-16T17:17:36.753 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_stat": 0, 2024-04-16T17:17:36.753 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_create": 10, 2024-04-16T17:17:36.753 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_read": 147, 2024-04-16T17:17:36.753 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_write": 1826, 2024-04-16T17:17:36.753 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_writefull": 118, 2024-04-16T17:17:36.754 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_writesame": 0, 2024-04-16T17:17:36.754 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_append": 0, 2024-04-16T17:17:36.754 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_zero": 0, 2024-04-16T17:17:36.754 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_truncate": 1, 2024-04-16T17:17:36.754 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_delete": 1434, 2024-04-16T17:17:36.754 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_mapext": 0, 2024-04-16T17:17:36.754 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_sparse_read": 0, 2024-04-16T17:17:36.754 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_clonerange": 0, 2024-04-16T17:17:36.754 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_getxattr": 503, 2024-04-16T17:17:36.754 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_setxattr": 20, 2024-04-16T17:17:36.755 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_cmpxattr": 0, 2024-04-16T17:17:36.755 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_rmxattr": 0, 2024-04-16T17:17:36.755 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_resetxattrs": 0, 2024-04-16T17:17:36.755 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_call": 0, 2024-04-16T17:17:36.755 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_watch": 0, 2024-04-16T17:17:36.755 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_notify": 0, 2024-04-16T17:17:36.755 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_src_cmpxattr": 0, 2024-04-16T17:17:36.755 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_pgls": 0, 2024-04-16T17:17:36.755 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_pgls_filter": 0, 2024-04-16T17:17:36.755 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_other": 0, 2024-04-16T17:17:36.756 INFO:teuthology.orchestra.run.smithi045.stdout: "linger_active": 0, 2024-04-16T17:17:36.756 INFO:teuthology.orchestra.run.smithi045.stdout: "linger_send": 0, 2024-04-16T17:17:36.756 INFO:teuthology.orchestra.run.smithi045.stdout: "linger_resend": 0, 2024-04-16T17:17:36.756 INFO:teuthology.orchestra.run.smithi045.stdout: "linger_ping": 0, 2024-04-16T17:17:36.756 INFO:teuthology.orchestra.run.smithi045.stdout: "poolop_active": 0, 2024-04-16T17:17:36.756 INFO:teuthology.orchestra.run.smithi045.stdout: "poolop_send": 0, 2024-04-16T17:17:36.756 INFO:teuthology.orchestra.run.smithi045.stdout: "poolop_resend": 0, 2024-04-16T17:17:36.756 INFO:teuthology.orchestra.run.smithi045.stdout: "poolstat_active": 0, 2024-04-16T17:17:36.756 INFO:teuthology.orchestra.run.smithi045.stdout: "poolstat_send": 0, 2024-04-16T17:17:36.756 INFO:teuthology.orchestra.run.smithi045.stdout: "poolstat_resend": 0, 2024-04-16T17:17:36.757 INFO:teuthology.orchestra.run.smithi045.stdout: "statfs_active": 0, 2024-04-16T17:17:36.757 INFO:teuthology.orchestra.run.smithi045.stdout: "statfs_send": 0, 2024-04-16T17:17:36.757 INFO:teuthology.orchestra.run.smithi045.stdout: "statfs_resend": 0, 2024-04-16T17:17:36.757 INFO:teuthology.orchestra.run.smithi045.stdout: "command_active": 0, 2024-04-16T17:17:36.757 INFO:teuthology.orchestra.run.smithi045.stdout: "command_send": 0, 2024-04-16T17:17:36.757 INFO:teuthology.orchestra.run.smithi045.stdout: "command_resend": 0, 2024-04-16T17:17:36.757 INFO:teuthology.orchestra.run.smithi045.stdout: "map_epoch": 106, 2024-04-16T17:17:36.757 INFO:teuthology.orchestra.run.smithi045.stdout: "map_full": 0, 2024-04-16T17:17:36.757 INFO:teuthology.orchestra.run.smithi045.stdout: "map_inc": 16, 2024-04-16T17:17:36.758 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_sessions": 12, 2024-04-16T17:17:36.758 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_session_open": 12, 2024-04-16T17:17:36.758 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_session_close": 0, 2024-04-16T17:17:36.758 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_laggy": 0, 2024-04-16T17:17:36.758 INFO:teuthology.orchestra.run.smithi045.stdout: "omap_wr": 442, 2024-04-16T17:17:36.758 INFO:teuthology.orchestra.run.smithi045.stdout: "omap_rd": 8, 2024-04-16T17:17:36.758 INFO:teuthology.orchestra.run.smithi045.stdout: "omap_del": 110 2024-04-16T17:17:36.758 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:36.758 INFO:teuthology.orchestra.run.smithi045.stdout: "oft": { 2024-04-16T17:17:36.758 INFO:teuthology.orchestra.run.smithi045.stdout: "omap_total_objs": 1, 2024-04-16T17:17:36.759 INFO:teuthology.orchestra.run.smithi045.stdout: "omap_total_kv_pairs": 0, 2024-04-16T17:17:36.759 INFO:teuthology.orchestra.run.smithi045.stdout: "omap_total_updates": 0, 2024-04-16T17:17:36.759 INFO:teuthology.orchestra.run.smithi045.stdout: "omap_total_removes": 0 2024-04-16T17:17:36.759 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:36.759 INFO:teuthology.orchestra.run.smithi045.stdout: "osd-slow-ops": { 2024-04-16T17:17:36.759 INFO:teuthology.orchestra.run.smithi045.stdout: "slow_ops_count": 1 2024-04-16T17:17:36.759 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:36.759 INFO:teuthology.orchestra.run.smithi045.stdout: "purge_queue": { 2024-04-16T17:17:36.759 INFO:teuthology.orchestra.run.smithi045.stdout: "pq_executing_ops": 0, 2024-04-16T17:17:36.759 INFO:teuthology.orchestra.run.smithi045.stdout: "pq_executing_ops_high_water": 16, 2024-04-16T17:17:36.760 INFO:teuthology.orchestra.run.smithi045.stdout: "pq_executing": 0, 2024-04-16T17:17:36.760 INFO:teuthology.orchestra.run.smithi045.stdout: "pq_executing_high_water": 8, 2024-04-16T17:17:36.760 INFO:teuthology.orchestra.run.smithi045.stdout: "pq_executed_ops": 1420, 2024-04-16T17:17:36.760 INFO:teuthology.orchestra.run.smithi045.stdout: "pq_executed": 710, 2024-04-16T17:17:36.760 INFO:teuthology.orchestra.run.smithi045.stdout: "pq_item_in_journal": 0 2024-04-16T17:17:36.760 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:36.760 INFO:teuthology.orchestra.run.smithi045.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-16T17:17:36.760 INFO:teuthology.orchestra.run.smithi045.stdout: "val": 0, 2024-04-16T17:17:36.760 INFO:teuthology.orchestra.run.smithi045.stdout: "max": 104857600, 2024-04-16T17:17:36.761 INFO:teuthology.orchestra.run.smithi045.stdout: "get_started": 0, 2024-04-16T17:17:36.761 INFO:teuthology.orchestra.run.smithi045.stdout: "get": 30029, 2024-04-16T17:17:36.761 INFO:teuthology.orchestra.run.smithi045.stdout: "get_sum": 9088937, 2024-04-16T17:17:36.761 INFO:teuthology.orchestra.run.smithi045.stdout: "get_or_fail_fail": 0, 2024-04-16T17:17:36.761 INFO:teuthology.orchestra.run.smithi045.stdout: "get_or_fail_success": 30029, 2024-04-16T17:17:36.761 INFO:teuthology.orchestra.run.smithi045.stdout: "take": 0, 2024-04-16T17:17:36.761 INFO:teuthology.orchestra.run.smithi045.stdout: "take_sum": 0, 2024-04-16T17:17:36.761 INFO:teuthology.orchestra.run.smithi045.stdout: "put": 30029, 2024-04-16T17:17:36.761 INFO:teuthology.orchestra.run.smithi045.stdout: "put_sum": 9088937, 2024-04-16T17:17:36.762 INFO:teuthology.orchestra.run.smithi045.stdout: "wait": { 2024-04-16T17:17:36.762 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-16T17:17:36.762 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-16T17:17:36.762 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-16T17:17:36.762 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:17:36.762 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:36.762 INFO:teuthology.orchestra.run.smithi045.stdout: "throttle-objecter_bytes": { 2024-04-16T17:17:36.762 INFO:teuthology.orchestra.run.smithi045.stdout: "val": 0, 2024-04-16T17:17:36.762 INFO:teuthology.orchestra.run.smithi045.stdout: "max": 104857600, 2024-04-16T17:17:36.762 INFO:teuthology.orchestra.run.smithi045.stdout: "get_started": 0, 2024-04-16T17:17:36.763 INFO:teuthology.orchestra.run.smithi045.stdout: "get": 0, 2024-04-16T17:17:36.763 INFO:teuthology.orchestra.run.smithi045.stdout: "get_sum": 0, 2024-04-16T17:17:36.763 INFO:teuthology.orchestra.run.smithi045.stdout: "get_or_fail_fail": 0, 2024-04-16T17:17:36.763 INFO:teuthology.orchestra.run.smithi045.stdout: "get_or_fail_success": 0, 2024-04-16T17:17:36.763 INFO:teuthology.orchestra.run.smithi045.stdout: "take": 4283, 2024-04-16T17:17:36.763 INFO:teuthology.orchestra.run.smithi045.stdout: "take_sum": 10733485, 2024-04-16T17:17:36.763 INFO:teuthology.orchestra.run.smithi045.stdout: "put": 4283, 2024-04-16T17:17:36.763 INFO:teuthology.orchestra.run.smithi045.stdout: "put_sum": 10733485, 2024-04-16T17:17:36.763 INFO:teuthology.orchestra.run.smithi045.stdout: "wait": { 2024-04-16T17:17:36.764 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-16T17:17:36.764 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-16T17:17:36.764 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-16T17:17:36.764 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:17:36.764 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:36.764 INFO:teuthology.orchestra.run.smithi045.stdout: "throttle-objecter_ops": { 2024-04-16T17:17:36.764 INFO:teuthology.orchestra.run.smithi045.stdout: "val": 0, 2024-04-16T17:17:36.764 INFO:teuthology.orchestra.run.smithi045.stdout: "max": 1024, 2024-04-16T17:17:36.764 INFO:teuthology.orchestra.run.smithi045.stdout: "get_started": 0, 2024-04-16T17:17:36.765 INFO:teuthology.orchestra.run.smithi045.stdout: "get": 0, 2024-04-16T17:17:36.765 INFO:teuthology.orchestra.run.smithi045.stdout: "get_sum": 0, 2024-04-16T17:17:36.765 INFO:teuthology.orchestra.run.smithi045.stdout: "get_or_fail_fail": 0, 2024-04-16T17:17:36.765 INFO:teuthology.orchestra.run.smithi045.stdout: "get_or_fail_success": 0, 2024-04-16T17:17:36.765 INFO:teuthology.orchestra.run.smithi045.stdout: "take": 4283, 2024-04-16T17:17:36.765 INFO:teuthology.orchestra.run.smithi045.stdout: "take_sum": 4283, 2024-04-16T17:17:36.765 INFO:teuthology.orchestra.run.smithi045.stdout: "put": 4283, 2024-04-16T17:17:36.765 INFO:teuthology.orchestra.run.smithi045.stdout: "put_sum": 4283, 2024-04-16T17:17:36.765 INFO:teuthology.orchestra.run.smithi045.stdout: "wait": { 2024-04-16T17:17:36.765 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-16T17:17:36.766 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-16T17:17:36.766 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-16T17:17:36.766 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:17:36.766 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:36.766 INFO:teuthology.orchestra.run.smithi045.stdout: "throttle-write_buf_throttle": { 2024-04-16T17:17:36.766 INFO:teuthology.orchestra.run.smithi045.stdout: "val": 0, 2024-04-16T17:17:36.766 INFO:teuthology.orchestra.run.smithi045.stdout: "max": 3758096384, 2024-04-16T17:17:36.766 INFO:teuthology.orchestra.run.smithi045.stdout: "get_started": 0, 2024-04-16T17:17:36.766 INFO:teuthology.orchestra.run.smithi045.stdout: "get": 710, 2024-04-16T17:17:36.767 INFO:teuthology.orchestra.run.smithi045.stdout: "get_sum": 77542, 2024-04-16T17:17:36.767 INFO:teuthology.orchestra.run.smithi045.stdout: "get_or_fail_fail": 0, 2024-04-16T17:17:36.767 INFO:teuthology.orchestra.run.smithi045.stdout: "get_or_fail_success": 710, 2024-04-16T17:17:36.767 INFO:teuthology.orchestra.run.smithi045.stdout: "take": 0, 2024-04-16T17:17:36.767 INFO:teuthology.orchestra.run.smithi045.stdout: "take_sum": 0, 2024-04-16T17:17:36.767 INFO:teuthology.orchestra.run.smithi045.stdout: "put": 147, 2024-04-16T17:17:36.767 INFO:teuthology.orchestra.run.smithi045.stdout: "put_sum": 77542, 2024-04-16T17:17:36.767 INFO:teuthology.orchestra.run.smithi045.stdout: "wait": { 2024-04-16T17:17:36.767 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-16T17:17:36.767 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-16T17:17:36.768 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-16T17:17:36.768 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:17:36.768 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:36.768 INFO:teuthology.orchestra.run.smithi045.stdout: "throttle-write_buf_throttle-0x5651ae95c500": { 2024-04-16T17:17:36.768 INFO:teuthology.orchestra.run.smithi045.stdout: "val": 0, 2024-04-16T17:17:36.768 INFO:teuthology.orchestra.run.smithi045.stdout: "max": 3758096384, 2024-04-16T17:17:36.768 INFO:teuthology.orchestra.run.smithi045.stdout: "get_started": 0, 2024-04-16T17:17:36.768 INFO:teuthology.orchestra.run.smithi045.stdout: "get": 7979, 2024-04-16T17:17:36.768 INFO:teuthology.orchestra.run.smithi045.stdout: "get_sum": 10368440, 2024-04-16T17:17:36.768 INFO:teuthology.orchestra.run.smithi045.stdout: "get_or_fail_fail": 0, 2024-04-16T17:17:36.769 INFO:teuthology.orchestra.run.smithi045.stdout: "get_or_fail_success": 7979, 2024-04-16T17:17:36.769 INFO:teuthology.orchestra.run.smithi045.stdout: "take": 0, 2024-04-16T17:17:36.769 INFO:teuthology.orchestra.run.smithi045.stdout: "take_sum": 0, 2024-04-16T17:17:36.769 INFO:teuthology.orchestra.run.smithi045.stdout: "put": 1679, 2024-04-16T17:17:36.769 INFO:teuthology.orchestra.run.smithi045.stdout: "put_sum": 10368440, 2024-04-16T17:17:36.769 INFO:teuthology.orchestra.run.smithi045.stdout: "wait": { 2024-04-16T17:17:36.769 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-16T17:17:36.769 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-16T17:17:36.769 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-16T17:17:36.769 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:17:36.769 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:17:36.770 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-16T17:17:36.804 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:17:36 smithi117 ceph-mon[36730]: pgmap v339: 129 pgs: 129 active+clean; 55 MiB data, 874 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T17:17:36.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:17:36 smithi104 ceph-mon[36457]: pgmap v339: 129 pgs: 129 active+clean; 55 MiB data, 874 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T17:17:37.717 INFO:tasks.check_counter:Daemon mds.a mds.exported=5 2024-04-16T17:17:37.717 INFO:tasks.check_counter:Daemon mds.a mds.imported=4 2024-04-16T17:17:37.717 INFO:tasks.check_counter:Daemon mds.a mds.root_rsnaps=6 2024-04-16T17:17:37.717 INFO:tasks.check_counter:Daemon mds.a mds_server.req_mksnap_latency.avgcount=0 2024-04-16T17:17:37.718 INFO:tasks.check_counter:Daemon mds.a mds_server.req_rmsnap_latency.avgcount=0 2024-04-16T17:17:37.718 DEBUG:tasks.check_counter:Getting stats from d 2024-04-16T17:17:37.719 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45 shell --fsid b1fecd38-fc11-11ee-bc90-c7b262605968 -- ceph daemon mds.d perf dump 2024-04-16T17:17:38.211 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/mon.a/config 2024-04-16T17:17:38.674 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:17:38 smithi045 ceph-mon[31676]: pgmap v340: 129 pgs: 129 active+clean; 55 MiB data, 874 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-16T17:17:38.804 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:17:38 smithi117 ceph-mon[36730]: pgmap v340: 129 pgs: 129 active+clean; 55 MiB data, 874 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-16T17:17:38.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:17:38 smithi104 ceph-mon[36457]: pgmap v340: 129 pgs: 129 active+clean; 55 MiB data, 874 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-16T17:17:40.763 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:17:40 smithi045 ceph-mon[31676]: pgmap v341: 129 pgs: 129 active+clean; 55 MiB data, 874 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T17:17:40.804 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:17:40 smithi117 ceph-mon[36730]: pgmap v341: 129 pgs: 129 active+clean; 55 MiB data, 874 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T17:17:40.868 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-16T17:17:40.868 INFO:teuthology.orchestra.run.smithi045.stdout: "AsyncMessenger::Worker-0": { 2024-04-16T17:17:40.869 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_messages": 0, 2024-04-16T17:17:40.869 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages": 2, 2024-04-16T17:17:40.869 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_bytes": 0, 2024-04-16T17:17:40.869 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_bytes": 0, 2024-04-16T17:17:40.869 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_created_connections": 4, 2024-04-16T17:17:40.869 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_active_connections": 0, 2024-04-16T17:17:40.869 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_total_time": 0.004510166, 2024-04-16T17:17:40.869 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_send_time": 0.000000000, 2024-04-16T17:17:40.869 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_recv_time": 0.000973503, 2024-04-16T17:17:40.870 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-16T17:17:40.870 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages_queue_lat": { 2024-04-16T17:17:40.870 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-16T17:17:40.870 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-16T17:17:40.870 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-16T17:17:40.870 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:40.870 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_handle_ack_lat": { 2024-04-16T17:17:40.870 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-16T17:17:40.871 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-16T17:17:40.871 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-16T17:17:40.871 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:40.871 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-16T17:17:40.871 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_encrypted_bytes": 0 2024-04-16T17:17:40.871 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:40.871 INFO:teuthology.orchestra.run.smithi045.stdout: "AsyncMessenger::Worker-1": { 2024-04-16T17:17:40.871 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_messages": 251, 2024-04-16T17:17:40.872 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages": 238, 2024-04-16T17:17:40.872 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_bytes": 1599488, 2024-04-16T17:17:40.872 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_bytes": 146272, 2024-04-16T17:17:40.872 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_created_connections": 2, 2024-04-16T17:17:40.872 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_active_connections": 1, 2024-04-16T17:17:40.872 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_total_time": 0.106350323, 2024-04-16T17:17:40.872 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_send_time": 0.023955781, 2024-04-16T17:17:40.872 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_recv_time": 0.109668146, 2024-04-16T17:17:40.872 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_fast_dispatch_time": 0.010773449, 2024-04-16T17:17:40.873 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages_queue_lat": { 2024-04-16T17:17:40.873 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 238, 2024-04-16T17:17:40.873 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.021251951, 2024-04-16T17:17:40.873 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000089293 2024-04-16T17:17:40.873 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:40.873 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_handle_ack_lat": { 2024-04-16T17:17:40.873 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-16T17:17:40.873 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-16T17:17:40.873 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-16T17:17:40.874 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:40.874 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_encrypted_bytes": 1599488, 2024-04-16T17:17:40.874 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_encrypted_bytes": 146272 2024-04-16T17:17:40.874 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:40.874 INFO:teuthology.orchestra.run.smithi045.stdout: "AsyncMessenger::Worker-2": { 2024-04-16T17:17:40.874 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_messages": 2, 2024-04-16T17:17:40.874 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages": 181, 2024-04-16T17:17:40.874 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_bytes": 320, 2024-04-16T17:17:40.875 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_bytes": 188032, 2024-04-16T17:17:40.875 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_created_connections": 5, 2024-04-16T17:17:40.875 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_active_connections": 3, 2024-04-16T17:17:40.875 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_total_time": 0.034412100, 2024-04-16T17:17:40.875 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_send_time": 0.024005507, 2024-04-16T17:17:40.875 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_recv_time": 0.004870584, 2024-04-16T17:17:40.875 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-16T17:17:40.875 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages_queue_lat": { 2024-04-16T17:17:40.875 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 178, 2024-04-16T17:17:40.875 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.020118347, 2024-04-16T17:17:40.876 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000113024 2024-04-16T17:17:40.876 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:40.876 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_handle_ack_lat": { 2024-04-16T17:17:40.876 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-16T17:17:40.876 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-16T17:17:40.876 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-16T17:17:40.876 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:40.876 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-16T17:17:40.876 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_encrypted_bytes": 188032 2024-04-16T17:17:40.877 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:40.877 INFO:teuthology.orchestra.run.smithi045.stdout: "cct": { 2024-04-16T17:17:40.877 INFO:teuthology.orchestra.run.smithi045.stdout: "total_workers": 0, 2024-04-16T17:17:40.877 INFO:teuthology.orchestra.run.smithi045.stdout: "unhealthy_workers": 0 2024-04-16T17:17:40.877 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:40.877 INFO:teuthology.orchestra.run.smithi045.stdout: "mempool": { 2024-04-16T17:17:40.877 INFO:teuthology.orchestra.run.smithi045.stdout: "bloom_filter_bytes": 0, 2024-04-16T17:17:40.877 INFO:teuthology.orchestra.run.smithi045.stdout: "bloom_filter_items": 0, 2024-04-16T17:17:40.877 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_alloc_bytes": 0, 2024-04-16T17:17:40.877 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_alloc_items": 0, 2024-04-16T17:17:40.878 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_data_bytes": 0, 2024-04-16T17:17:40.878 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_data_items": 0, 2024-04-16T17:17:40.878 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-16T17:17:40.878 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_onode_items": 0, 2024-04-16T17:17:40.878 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-16T17:17:40.878 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_meta_items": 0, 2024-04-16T17:17:40.878 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_other_bytes": 0, 2024-04-16T17:17:40.878 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_other_items": 0, 2024-04-16T17:17:40.878 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-16T17:17:40.879 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_buffer_items": 0, 2024-04-16T17:17:40.879 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_extent_bytes": 0, 2024-04-16T17:17:40.879 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_extent_items": 0, 2024-04-16T17:17:40.879 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_blob_bytes": 0, 2024-04-16T17:17:40.879 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_blob_items": 0, 2024-04-16T17:17:40.879 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-16T17:17:40.879 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_shared_blob_items": 0, 2024-04-16T17:17:40.879 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-16T17:17:40.879 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_inline_bl_items": 0, 2024-04-16T17:17:40.879 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_fsck_bytes": 0, 2024-04-16T17:17:40.880 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_fsck_items": 0, 2024-04-16T17:17:40.880 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_txc_bytes": 0, 2024-04-16T17:17:40.880 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_txc_items": 0, 2024-04-16T17:17:40.880 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-16T17:17:40.880 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_writing_deferred_items": 0, 2024-04-16T17:17:40.880 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_writing_bytes": 0, 2024-04-16T17:17:40.880 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_writing_items": 0, 2024-04-16T17:17:40.880 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_bytes": 0, 2024-04-16T17:17:40.880 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_items": 0, 2024-04-16T17:17:40.881 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_file_reader_bytes": 0, 2024-04-16T17:17:40.881 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_file_reader_items": 0, 2024-04-16T17:17:40.881 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_file_writer_bytes": 0, 2024-04-16T17:17:40.881 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_file_writer_items": 0, 2024-04-16T17:17:40.881 INFO:teuthology.orchestra.run.smithi045.stdout: "buffer_anon_bytes": 148731, 2024-04-16T17:17:40.881 INFO:teuthology.orchestra.run.smithi045.stdout: "buffer_anon_items": 31, 2024-04-16T17:17:40.881 INFO:teuthology.orchestra.run.smithi045.stdout: "buffer_meta_bytes": 0, 2024-04-16T17:17:40.881 INFO:teuthology.orchestra.run.smithi045.stdout: "buffer_meta_items": 0, 2024-04-16T17:17:40.881 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_bytes": 0, 2024-04-16T17:17:40.881 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_items": 0, 2024-04-16T17:17:40.882 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_mapbl_bytes": 0, 2024-04-16T17:17:40.882 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_mapbl_items": 0, 2024-04-16T17:17:40.882 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_pglog_bytes": 0, 2024-04-16T17:17:40.882 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_pglog_items": 0, 2024-04-16T17:17:40.882 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_bytes": 0, 2024-04-16T17:17:40.882 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_items": 0, 2024-04-16T17:17:40.882 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_mapping_bytes": 0, 2024-04-16T17:17:40.882 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_mapping_items": 0, 2024-04-16T17:17:40.882 INFO:teuthology.orchestra.run.smithi045.stdout: "pgmap_bytes": 0, 2024-04-16T17:17:40.883 INFO:teuthology.orchestra.run.smithi045.stdout: "pgmap_items": 0, 2024-04-16T17:17:40.883 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_co_bytes": 928, 2024-04-16T17:17:40.883 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_co_items": 2, 2024-04-16T17:17:40.883 INFO:teuthology.orchestra.run.smithi045.stdout: "unittest_1_bytes": 0, 2024-04-16T17:17:40.883 INFO:teuthology.orchestra.run.smithi045.stdout: "unittest_1_items": 0, 2024-04-16T17:17:40.883 INFO:teuthology.orchestra.run.smithi045.stdout: "unittest_2_bytes": 0, 2024-04-16T17:17:40.883 INFO:teuthology.orchestra.run.smithi045.stdout: "unittest_2_items": 0 2024-04-16T17:17:40.883 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:40.883 INFO:teuthology.orchestra.run.smithi045.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-16T17:17:40.884 INFO:teuthology.orchestra.run.smithi045.stdout: "val": 0, 2024-04-16T17:17:40.884 INFO:teuthology.orchestra.run.smithi045.stdout: "max": 104857600, 2024-04-16T17:17:40.884 INFO:teuthology.orchestra.run.smithi045.stdout: "get_started": 0, 2024-04-16T17:17:40.884 INFO:teuthology.orchestra.run.smithi045.stdout: "get": 249, 2024-04-16T17:17:40.884 INFO:teuthology.orchestra.run.smithi045.stdout: "get_sum": 1561649, 2024-04-16T17:17:40.884 INFO:teuthology.orchestra.run.smithi045.stdout: "get_or_fail_fail": 0, 2024-04-16T17:17:40.884 INFO:teuthology.orchestra.run.smithi045.stdout: "get_or_fail_success": 249, 2024-04-16T17:17:40.884 INFO:teuthology.orchestra.run.smithi045.stdout: "take": 0, 2024-04-16T17:17:40.884 INFO:teuthology.orchestra.run.smithi045.stdout: "take_sum": 0, 2024-04-16T17:17:40.884 INFO:teuthology.orchestra.run.smithi045.stdout: "put": 249, 2024-04-16T17:17:40.885 INFO:teuthology.orchestra.run.smithi045.stdout: "put_sum": 1561649, 2024-04-16T17:17:40.885 INFO:teuthology.orchestra.run.smithi045.stdout: "wait": { 2024-04-16T17:17:40.885 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-16T17:17:40.885 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-16T17:17:40.885 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-16T17:17:40.885 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:17:40.885 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:17:40.885 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-16T17:17:40.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:17:40 smithi104 ceph-mon[36457]: pgmap v341: 129 pgs: 129 active+clean; 55 MiB data, 874 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T17:17:41.926 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-04-16T17:17:41.927 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-04-16T17:17:41.927 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.d 2024-04-16T17:17:41.927 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.d 2024-04-16T17:17:41.927 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.d 2024-04-16T17:17:41.927 DEBUG:tasks.check_counter:Getting stats from g 2024-04-16T17:17:41.927 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45 shell --fsid b1fecd38-fc11-11ee-bc90-c7b262605968 -- ceph daemon mds.g perf dump 2024-04-16T17:17:42.417 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/mon.a/config 2024-04-16T17:17:42.779 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:17:42 smithi045 ceph-mon[31676]: pgmap v342: 129 pgs: 129 active+clean; 55 MiB data, 874 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-16T17:17:42.804 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:17:42 smithi117 ceph-mon[36730]: pgmap v342: 129 pgs: 129 active+clean; 55 MiB data, 874 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-16T17:17:42.926 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:17:42 smithi104 ceph-mon[36457]: pgmap v342: 129 pgs: 129 active+clean; 55 MiB data, 874 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-16T17:17:44.711 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:17:44 smithi045 ceph-mon[31676]: pgmap v343: 129 pgs: 129 active+clean; 55 MiB data, 874 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T17:17:44.804 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:17:44 smithi117 ceph-mon[36730]: pgmap v343: 129 pgs: 129 active+clean; 55 MiB data, 874 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T17:17:44.819 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-16T17:17:44.819 INFO:teuthology.orchestra.run.smithi045.stdout: "AsyncMessenger::Worker-0": { 2024-04-16T17:17:44.819 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_messages": 248, 2024-04-16T17:17:44.819 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages": 235, 2024-04-16T17:17:44.819 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_bytes": 1598672, 2024-04-16T17:17:44.819 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_bytes": 144352, 2024-04-16T17:17:44.819 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_created_connections": 4, 2024-04-16T17:17:44.820 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_active_connections": 0, 2024-04-16T17:17:44.820 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_total_time": 0.091912246, 2024-04-16T17:17:44.820 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_send_time": 0.032172354, 2024-04-16T17:17:44.820 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_recv_time": 0.062514303, 2024-04-16T17:17:44.820 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_fast_dispatch_time": 0.009221376, 2024-04-16T17:17:44.820 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages_queue_lat": { 2024-04-16T17:17:44.820 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 235, 2024-04-16T17:17:44.820 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.023213191, 2024-04-16T17:17:44.820 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000098779 2024-04-16T17:17:44.821 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:44.821 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_handle_ack_lat": { 2024-04-16T17:17:44.821 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-16T17:17:44.821 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-16T17:17:44.821 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-16T17:17:44.821 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:44.821 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_encrypted_bytes": 1598672, 2024-04-16T17:17:44.821 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_encrypted_bytes": 144352 2024-04-16T17:17:44.822 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:44.822 INFO:teuthology.orchestra.run.smithi045.stdout: "AsyncMessenger::Worker-1": { 2024-04-16T17:17:44.822 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_messages": 2, 2024-04-16T17:17:44.822 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages": 178, 2024-04-16T17:17:44.822 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_bytes": 320, 2024-04-16T17:17:44.822 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_bytes": 187504, 2024-04-16T17:17:44.822 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_created_connections": 5, 2024-04-16T17:17:44.822 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_active_connections": 3, 2024-04-16T17:17:44.822 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_total_time": 0.032791265, 2024-04-16T17:17:44.823 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_send_time": 0.023278400, 2024-04-16T17:17:44.823 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_recv_time": 0.003845109, 2024-04-16T17:17:44.823 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-16T17:17:44.823 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages_queue_lat": { 2024-04-16T17:17:44.823 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 175, 2024-04-16T17:17:44.823 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.026929329, 2024-04-16T17:17:44.823 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000153881 2024-04-16T17:17:44.823 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:44.823 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_handle_ack_lat": { 2024-04-16T17:17:44.824 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-16T17:17:44.824 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-16T17:17:44.824 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-16T17:17:44.824 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:44.824 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-16T17:17:44.824 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_encrypted_bytes": 187504 2024-04-16T17:17:44.824 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:44.824 INFO:teuthology.orchestra.run.smithi045.stdout: "AsyncMessenger::Worker-2": { 2024-04-16T17:17:44.824 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_messages": 0, 2024-04-16T17:17:44.825 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages": 2, 2024-04-16T17:17:44.825 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_bytes": 0, 2024-04-16T17:17:44.825 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_bytes": 0, 2024-04-16T17:17:44.825 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_created_connections": 2, 2024-04-16T17:17:44.825 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_active_connections": 1, 2024-04-16T17:17:44.825 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_total_time": 0.000896827, 2024-04-16T17:17:44.825 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_send_time": 0.000000000, 2024-04-16T17:17:44.825 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_recv_time": 0.000745928, 2024-04-16T17:17:44.825 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-16T17:17:44.825 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages_queue_lat": { 2024-04-16T17:17:44.826 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-16T17:17:44.826 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-16T17:17:44.826 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-16T17:17:44.826 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:44.826 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_handle_ack_lat": { 2024-04-16T17:17:44.826 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-16T17:17:44.826 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-16T17:17:44.826 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-16T17:17:44.826 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:44.827 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-16T17:17:44.827 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_encrypted_bytes": 0 2024-04-16T17:17:44.827 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:44.827 INFO:teuthology.orchestra.run.smithi045.stdout: "cct": { 2024-04-16T17:17:44.827 INFO:teuthology.orchestra.run.smithi045.stdout: "total_workers": 0, 2024-04-16T17:17:44.827 INFO:teuthology.orchestra.run.smithi045.stdout: "unhealthy_workers": 0 2024-04-16T17:17:44.827 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:44.827 INFO:teuthology.orchestra.run.smithi045.stdout: "mempool": { 2024-04-16T17:17:44.827 INFO:teuthology.orchestra.run.smithi045.stdout: "bloom_filter_bytes": 0, 2024-04-16T17:17:44.828 INFO:teuthology.orchestra.run.smithi045.stdout: "bloom_filter_items": 0, 2024-04-16T17:17:44.828 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_alloc_bytes": 0, 2024-04-16T17:17:44.828 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_alloc_items": 0, 2024-04-16T17:17:44.828 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_data_bytes": 0, 2024-04-16T17:17:44.828 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_data_items": 0, 2024-04-16T17:17:44.828 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-16T17:17:44.828 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_onode_items": 0, 2024-04-16T17:17:44.828 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-16T17:17:44.828 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_meta_items": 0, 2024-04-16T17:17:44.829 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_other_bytes": 0, 2024-04-16T17:17:44.829 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_other_items": 0, 2024-04-16T17:17:44.829 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-16T17:17:44.829 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_buffer_items": 0, 2024-04-16T17:17:44.829 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_extent_bytes": 0, 2024-04-16T17:17:44.829 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_extent_items": 0, 2024-04-16T17:17:44.829 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_blob_bytes": 0, 2024-04-16T17:17:44.829 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_blob_items": 0, 2024-04-16T17:17:44.829 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-16T17:17:44.830 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_shared_blob_items": 0, 2024-04-16T17:17:44.830 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-16T17:17:44.830 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_inline_bl_items": 0, 2024-04-16T17:17:44.830 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_fsck_bytes": 0, 2024-04-16T17:17:44.830 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_fsck_items": 0, 2024-04-16T17:17:44.830 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_txc_bytes": 0, 2024-04-16T17:17:44.830 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_txc_items": 0, 2024-04-16T17:17:44.830 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-16T17:17:44.831 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_writing_deferred_items": 0, 2024-04-16T17:17:44.831 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_writing_bytes": 0, 2024-04-16T17:17:44.831 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_writing_items": 0, 2024-04-16T17:17:44.831 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_bytes": 0, 2024-04-16T17:17:44.831 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_items": 0, 2024-04-16T17:17:44.831 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_file_reader_bytes": 0, 2024-04-16T17:17:44.831 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_file_reader_items": 0, 2024-04-16T17:17:44.831 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_file_writer_bytes": 0, 2024-04-16T17:17:44.831 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_file_writer_items": 0, 2024-04-16T17:17:44.832 INFO:teuthology.orchestra.run.smithi045.stdout: "buffer_anon_bytes": 150776, 2024-04-16T17:17:44.832 INFO:teuthology.orchestra.run.smithi045.stdout: "buffer_anon_items": 32, 2024-04-16T17:17:44.832 INFO:teuthology.orchestra.run.smithi045.stdout: "buffer_meta_bytes": 0, 2024-04-16T17:17:44.832 INFO:teuthology.orchestra.run.smithi045.stdout: "buffer_meta_items": 0, 2024-04-16T17:17:44.832 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_bytes": 0, 2024-04-16T17:17:44.832 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_items": 0, 2024-04-16T17:17:44.832 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_mapbl_bytes": 0, 2024-04-16T17:17:44.832 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_mapbl_items": 0, 2024-04-16T17:17:44.832 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_pglog_bytes": 0, 2024-04-16T17:17:44.832 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_pglog_items": 0, 2024-04-16T17:17:44.833 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_bytes": 0, 2024-04-16T17:17:44.833 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_items": 0, 2024-04-16T17:17:44.833 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_mapping_bytes": 0, 2024-04-16T17:17:44.833 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_mapping_items": 0, 2024-04-16T17:17:44.833 INFO:teuthology.orchestra.run.smithi045.stdout: "pgmap_bytes": 0, 2024-04-16T17:17:44.833 INFO:teuthology.orchestra.run.smithi045.stdout: "pgmap_items": 0, 2024-04-16T17:17:44.833 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_co_bytes": 928, 2024-04-16T17:17:44.833 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_co_items": 2, 2024-04-16T17:17:44.833 INFO:teuthology.orchestra.run.smithi045.stdout: "unittest_1_bytes": 0, 2024-04-16T17:17:44.834 INFO:teuthology.orchestra.run.smithi045.stdout: "unittest_1_items": 0, 2024-04-16T17:17:44.834 INFO:teuthology.orchestra.run.smithi045.stdout: "unittest_2_bytes": 0, 2024-04-16T17:17:44.834 INFO:teuthology.orchestra.run.smithi045.stdout: "unittest_2_items": 0 2024-04-16T17:17:44.834 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:44.834 INFO:teuthology.orchestra.run.smithi045.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-16T17:17:44.834 INFO:teuthology.orchestra.run.smithi045.stdout: "val": 0, 2024-04-16T17:17:44.834 INFO:teuthology.orchestra.run.smithi045.stdout: "max": 104857600, 2024-04-16T17:17:44.834 INFO:teuthology.orchestra.run.smithi045.stdout: "get_started": 0, 2024-04-16T17:17:44.834 INFO:teuthology.orchestra.run.smithi045.stdout: "get": 246, 2024-04-16T17:17:44.835 INFO:teuthology.orchestra.run.smithi045.stdout: "get_sum": 1561259, 2024-04-16T17:17:44.835 INFO:teuthology.orchestra.run.smithi045.stdout: "get_or_fail_fail": 0, 2024-04-16T17:17:44.835 INFO:teuthology.orchestra.run.smithi045.stdout: "get_or_fail_success": 246, 2024-04-16T17:17:44.835 INFO:teuthology.orchestra.run.smithi045.stdout: "take": 0, 2024-04-16T17:17:44.835 INFO:teuthology.orchestra.run.smithi045.stdout: "take_sum": 0, 2024-04-16T17:17:44.835 INFO:teuthology.orchestra.run.smithi045.stdout: "put": 246, 2024-04-16T17:17:44.835 INFO:teuthology.orchestra.run.smithi045.stdout: "put_sum": 1561259, 2024-04-16T17:17:44.835 INFO:teuthology.orchestra.run.smithi045.stdout: "wait": { 2024-04-16T17:17:44.835 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-16T17:17:44.836 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-16T17:17:44.836 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-16T17:17:44.836 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:17:44.836 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:17:44.836 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-16T17:17:44.926 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:17:44 smithi104 ceph-mon[36457]: pgmap v343: 129 pgs: 129 active+clean; 55 MiB data, 874 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T17:17:45.833 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.g 2024-04-16T17:17:45.833 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.g 2024-04-16T17:17:45.833 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.g 2024-04-16T17:17:45.833 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.g 2024-04-16T17:17:45.833 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.g 2024-04-16T17:17:45.833 DEBUG:tasks.check_counter:Getting stats from j 2024-04-16T17:17:45.834 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45 shell --fsid b1fecd38-fc11-11ee-bc90-c7b262605968 -- ceph daemon mds.j perf dump 2024-04-16T17:17:46.321 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/mon.a/config 2024-04-16T17:17:46.804 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:17:46 smithi117 ceph-mon[36730]: pgmap v344: 129 pgs: 129 active+clean; 55 MiB data, 874 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T17:17:46.828 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:17:46 smithi045 ceph-mon[31676]: pgmap v344: 129 pgs: 129 active+clean; 55 MiB data, 874 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T17:17:46.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:17:46 smithi104 ceph-mon[36457]: pgmap v344: 129 pgs: 129 active+clean; 55 MiB data, 874 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T17:17:48.643 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:17:48 smithi045 ceph-mon[31676]: pgmap v345: 129 pgs: 129 active+clean; 55 MiB data, 874 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-16T17:17:48.750 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-16T17:17:48.751 INFO:teuthology.orchestra.run.smithi045.stdout: "AsyncMessenger::Worker-0": { 2024-04-16T17:17:48.751 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_messages": 0, 2024-04-16T17:17:48.751 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages": 2, 2024-04-16T17:17:48.751 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_bytes": 0, 2024-04-16T17:17:48.751 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_bytes": 0, 2024-04-16T17:17:48.751 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_created_connections": 4, 2024-04-16T17:17:48.751 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_active_connections": 0, 2024-04-16T17:17:48.751 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_total_time": 0.004556623, 2024-04-16T17:17:48.752 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_send_time": 0.000000000, 2024-04-16T17:17:48.752 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_recv_time": 0.001018525, 2024-04-16T17:17:48.752 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-16T17:17:48.752 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages_queue_lat": { 2024-04-16T17:17:48.752 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-16T17:17:48.752 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-16T17:17:48.752 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-16T17:17:48.752 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:48.753 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_handle_ack_lat": { 2024-04-16T17:17:48.753 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-16T17:17:48.753 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-16T17:17:48.753 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-16T17:17:48.753 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:48.753 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-16T17:17:48.753 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_encrypted_bytes": 0 2024-04-16T17:17:48.753 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:48.754 INFO:teuthology.orchestra.run.smithi045.stdout: "AsyncMessenger::Worker-1": { 2024-04-16T17:17:48.754 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_messages": 254, 2024-04-16T17:17:48.754 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages": 241, 2024-04-16T17:17:48.754 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_bytes": 1600304, 2024-04-16T17:17:48.754 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_bytes": 148192, 2024-04-16T17:17:48.754 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_created_connections": 2, 2024-04-16T17:17:48.754 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_active_connections": 1, 2024-04-16T17:17:48.754 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_total_time": 0.096598593, 2024-04-16T17:17:48.755 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_send_time": 0.034551403, 2024-04-16T17:17:48.755 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_recv_time": 0.071635716, 2024-04-16T17:17:48.755 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_fast_dispatch_time": 0.009509149, 2024-04-16T17:17:48.755 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages_queue_lat": { 2024-04-16T17:17:48.755 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 241, 2024-04-16T17:17:48.755 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.023401207, 2024-04-16T17:17:48.755 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000097100 2024-04-16T17:17:48.755 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:48.756 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_handle_ack_lat": { 2024-04-16T17:17:48.756 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-16T17:17:48.756 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-16T17:17:48.756 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-16T17:17:48.756 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:48.756 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_encrypted_bytes": 1600304, 2024-04-16T17:17:48.756 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_encrypted_bytes": 148192 2024-04-16T17:17:48.756 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:48.756 INFO:teuthology.orchestra.run.smithi045.stdout: "AsyncMessenger::Worker-2": { 2024-04-16T17:17:48.757 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_messages": 2, 2024-04-16T17:17:48.757 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages": 184, 2024-04-16T17:17:48.757 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_bytes": 320, 2024-04-16T17:17:48.757 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_bytes": 188560, 2024-04-16T17:17:48.757 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_created_connections": 5, 2024-04-16T17:17:48.757 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_active_connections": 3, 2024-04-16T17:17:48.757 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_total_time": 0.035337006, 2024-04-16T17:17:48.757 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_send_time": 0.026213570, 2024-04-16T17:17:48.758 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_recv_time": 0.003416813, 2024-04-16T17:17:48.758 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-16T17:17:48.758 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages_queue_lat": { 2024-04-16T17:17:48.758 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 181, 2024-04-16T17:17:48.758 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.026635533, 2024-04-16T17:17:48.758 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000147157 2024-04-16T17:17:48.758 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:48.758 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_handle_ack_lat": { 2024-04-16T17:17:48.759 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-16T17:17:48.759 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-16T17:17:48.759 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-16T17:17:48.759 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:48.759 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-16T17:17:48.759 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_encrypted_bytes": 188560 2024-04-16T17:17:48.759 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:48.759 INFO:teuthology.orchestra.run.smithi045.stdout: "cct": { 2024-04-16T17:17:48.760 INFO:teuthology.orchestra.run.smithi045.stdout: "total_workers": 0, 2024-04-16T17:17:48.760 INFO:teuthology.orchestra.run.smithi045.stdout: "unhealthy_workers": 0 2024-04-16T17:17:48.760 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:48.760 INFO:teuthology.orchestra.run.smithi045.stdout: "mempool": { 2024-04-16T17:17:48.760 INFO:teuthology.orchestra.run.smithi045.stdout: "bloom_filter_bytes": 0, 2024-04-16T17:17:48.760 INFO:teuthology.orchestra.run.smithi045.stdout: "bloom_filter_items": 0, 2024-04-16T17:17:48.760 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_alloc_bytes": 0, 2024-04-16T17:17:48.760 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_alloc_items": 0, 2024-04-16T17:17:48.760 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_data_bytes": 0, 2024-04-16T17:17:48.761 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_data_items": 0, 2024-04-16T17:17:48.761 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-16T17:17:48.761 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_onode_items": 0, 2024-04-16T17:17:48.761 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-16T17:17:48.761 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_meta_items": 0, 2024-04-16T17:17:48.761 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_other_bytes": 0, 2024-04-16T17:17:48.761 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_other_items": 0, 2024-04-16T17:17:48.761 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-16T17:17:48.762 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_buffer_items": 0, 2024-04-16T17:17:48.762 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_extent_bytes": 0, 2024-04-16T17:17:48.762 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_extent_items": 0, 2024-04-16T17:17:48.762 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_blob_bytes": 0, 2024-04-16T17:17:48.762 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_blob_items": 0, 2024-04-16T17:17:48.762 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-16T17:17:48.762 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_shared_blob_items": 0, 2024-04-16T17:17:48.763 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-16T17:17:48.763 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_inline_bl_items": 0, 2024-04-16T17:17:48.763 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_fsck_bytes": 0, 2024-04-16T17:17:48.763 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_fsck_items": 0, 2024-04-16T17:17:48.763 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_txc_bytes": 0, 2024-04-16T17:17:48.763 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_txc_items": 0, 2024-04-16T17:17:48.763 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-16T17:17:48.763 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_writing_deferred_items": 0, 2024-04-16T17:17:48.764 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_writing_bytes": 0, 2024-04-16T17:17:48.764 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_writing_items": 0, 2024-04-16T17:17:48.764 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_bytes": 0, 2024-04-16T17:17:48.764 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_items": 0, 2024-04-16T17:17:48.764 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_file_reader_bytes": 0, 2024-04-16T17:17:48.764 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_file_reader_items": 0, 2024-04-16T17:17:48.764 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_file_writer_bytes": 0, 2024-04-16T17:17:48.764 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_file_writer_items": 0, 2024-04-16T17:17:48.765 INFO:teuthology.orchestra.run.smithi045.stdout: "buffer_anon_bytes": 149745, 2024-04-16T17:17:48.765 INFO:teuthology.orchestra.run.smithi045.stdout: "buffer_anon_items": 31, 2024-04-16T17:17:48.765 INFO:teuthology.orchestra.run.smithi045.stdout: "buffer_meta_bytes": 0, 2024-04-16T17:17:48.765 INFO:teuthology.orchestra.run.smithi045.stdout: "buffer_meta_items": 0, 2024-04-16T17:17:48.765 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_bytes": 0, 2024-04-16T17:17:48.765 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_items": 0, 2024-04-16T17:17:48.765 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_mapbl_bytes": 0, 2024-04-16T17:17:48.766 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_mapbl_items": 0, 2024-04-16T17:17:48.766 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_pglog_bytes": 0, 2024-04-16T17:17:48.766 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_pglog_items": 0, 2024-04-16T17:17:48.766 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_bytes": 0, 2024-04-16T17:17:48.766 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_items": 0, 2024-04-16T17:17:48.766 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_mapping_bytes": 0, 2024-04-16T17:17:48.766 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_mapping_items": 0, 2024-04-16T17:17:48.766 INFO:teuthology.orchestra.run.smithi045.stdout: "pgmap_bytes": 0, 2024-04-16T17:17:48.767 INFO:teuthology.orchestra.run.smithi045.stdout: "pgmap_items": 0, 2024-04-16T17:17:48.767 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_co_bytes": 928, 2024-04-16T17:17:48.767 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_co_items": 2, 2024-04-16T17:17:48.767 INFO:teuthology.orchestra.run.smithi045.stdout: "unittest_1_bytes": 0, 2024-04-16T17:17:48.767 INFO:teuthology.orchestra.run.smithi045.stdout: "unittest_1_items": 0, 2024-04-16T17:17:48.767 INFO:teuthology.orchestra.run.smithi045.stdout: "unittest_2_bytes": 0, 2024-04-16T17:17:48.767 INFO:teuthology.orchestra.run.smithi045.stdout: "unittest_2_items": 0 2024-04-16T17:17:48.767 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-16T17:17:48.768 INFO:teuthology.orchestra.run.smithi045.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-16T17:17:48.768 INFO:teuthology.orchestra.run.smithi045.stdout: "val": 0, 2024-04-16T17:17:48.768 INFO:teuthology.orchestra.run.smithi045.stdout: "max": 104857600, 2024-04-16T17:17:48.768 INFO:teuthology.orchestra.run.smithi045.stdout: "get_started": 0, 2024-04-16T17:17:48.768 INFO:teuthology.orchestra.run.smithi045.stdout: "get": 252, 2024-04-16T17:17:48.768 INFO:teuthology.orchestra.run.smithi045.stdout: "get_sum": 1562039, 2024-04-16T17:17:48.768 INFO:teuthology.orchestra.run.smithi045.stdout: "get_or_fail_fail": 0, 2024-04-16T17:17:48.768 INFO:teuthology.orchestra.run.smithi045.stdout: "get_or_fail_success": 252, 2024-04-16T17:17:48.769 INFO:teuthology.orchestra.run.smithi045.stdout: "take": 0, 2024-04-16T17:17:48.769 INFO:teuthology.orchestra.run.smithi045.stdout: "take_sum": 0, 2024-04-16T17:17:48.769 INFO:teuthology.orchestra.run.smithi045.stdout: "put": 252, 2024-04-16T17:17:48.769 INFO:teuthology.orchestra.run.smithi045.stdout: "put_sum": 1562039, 2024-04-16T17:17:48.769 INFO:teuthology.orchestra.run.smithi045.stdout: "wait": { 2024-04-16T17:17:48.769 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-16T17:17:48.769 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-16T17:17:48.769 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-16T17:17:48.770 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:17:48.770 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-16T17:17:48.770 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-16T17:17:48.804 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:17:48 smithi117 ceph-mon[36730]: pgmap v345: 129 pgs: 129 active+clean; 55 MiB data, 874 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-16T17:17:48.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:17:48 smithi104 ceph-mon[36457]: pgmap v345: 129 pgs: 129 active+clean; 55 MiB data, 874 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-16T17:17:49.609 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-04-16T17:17:49.610 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-04-16T17:17:49.610 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.j 2024-04-16T17:17:49.610 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.j 2024-04-16T17:17:49.610 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.j 2024-04-16T17:17:49.610 DEBUG:tasks.check_counter:Getting stats from b 2024-04-16T17:17:49.610 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45 shell --fsid b1fecd38-fc11-11ee-bc90-c7b262605968 -- ceph daemon mds.b perf dump 2024-04-16T17:17:49.862 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/mon.b/config 2024-04-16T17:17:50.804 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:17:50 smithi117 ceph-mon[36730]: pgmap v346: 129 pgs: 129 active+clean; 55 MiB data, 874 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T17:17:50.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:17:50 smithi104 ceph-mon[36457]: pgmap v346: 129 pgs: 129 active+clean; 55 MiB data, 874 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T17:17:50.976 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:17:50 smithi045 ceph-mon[31676]: pgmap v346: 129 pgs: 129 active+clean; 55 MiB data, 874 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T17:17:51.795 INFO:teuthology.orchestra.run.smithi104.stdout:{ 2024-04-16T17:17:51.796 INFO:teuthology.orchestra.run.smithi104.stdout: "AsyncMessenger::Worker-0": { 2024-04-16T17:17:51.796 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_messages": 23344, 2024-04-16T17:17:51.796 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_messages": 24405, 2024-04-16T17:17:51.796 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_bytes": 7722319, 2024-04-16T17:17:51.796 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_bytes": 32740294, 2024-04-16T17:17:51.796 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_created_connections": 228, 2024-04-16T17:17:51.796 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_active_connections": 5, 2024-04-16T17:17:51.797 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_total_time": 1.515213721, 2024-04-16T17:17:51.797 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_send_time": 0.582312861, 2024-04-16T17:17:51.797 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_recv_time": 0.939994349, 2024-04-16T17:17:51.797 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_fast_dispatch_time": 0.133829240, 2024-04-16T17:17:51.797 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_messages_queue_lat": { 2024-04-16T17:17:51.797 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 24404, 2024-04-16T17:17:51.797 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 1.133881009, 2024-04-16T17:17:51.797 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000046462 2024-04-16T17:17:51.797 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-16T17:17:51.798 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_handle_ack_lat": { 2024-04-16T17:17:51.798 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 37774, 2024-04-16T17:17:51.798 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.004924873, 2024-04-16T17:17:51.798 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000130 2024-04-16T17:17:51.798 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-16T17:17:51.798 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_encrypted_bytes": 1783296, 2024-04-16T17:17:51.798 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_encrypted_bytes": 180352 2024-04-16T17:17:51.798 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-16T17:17:51.798 INFO:teuthology.orchestra.run.smithi104.stdout: "AsyncMessenger::Worker-1": { 2024-04-16T17:17:51.799 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_messages": 27859, 2024-04-16T17:17:51.799 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_messages": 37215, 2024-04-16T17:17:51.799 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_bytes": 9126466, 2024-04-16T17:17:51.799 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_bytes": 41751134, 2024-04-16T17:17:51.799 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_created_connections": 17, 2024-04-16T17:17:51.799 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_active_connections": 5, 2024-04-16T17:17:51.799 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_total_time": 2.031084917, 2024-04-16T17:17:51.799 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_send_time": 0.813387315, 2024-04-16T17:17:51.799 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_recv_time": 1.330744676, 2024-04-16T17:17:51.799 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_fast_dispatch_time": 0.181791971, 2024-04-16T17:17:51.800 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_messages_queue_lat": { 2024-04-16T17:17:51.800 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 37212, 2024-04-16T17:17:51.800 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 2.231220541, 2024-04-16T17:17:51.800 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000059959 2024-04-16T17:17:51.800 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-16T17:17:51.800 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_handle_ack_lat": { 2024-04-16T17:17:51.800 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 41224, 2024-04-16T17:17:51.800 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.008583284, 2024-04-16T17:17:51.801 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000208 2024-04-16T17:17:51.801 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-16T17:17:51.801 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_encrypted_bytes": 99152, 2024-04-16T17:17:51.801 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_encrypted_bytes": 519200 2024-04-16T17:17:51.801 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-16T17:17:51.801 INFO:teuthology.orchestra.run.smithi104.stdout: "AsyncMessenger::Worker-2": { 2024-04-16T17:17:51.801 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_messages": 17262, 2024-04-16T17:17:51.801 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_messages": 18300, 2024-04-16T17:17:51.801 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_bytes": 5052249, 2024-04-16T17:17:51.802 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_bytes": 33475214, 2024-04-16T17:17:51.802 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_created_connections": 16, 2024-04-16T17:17:51.802 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_active_connections": 7, 2024-04-16T17:17:51.802 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_total_time": 1.240471303, 2024-04-16T17:17:51.802 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_send_time": 0.484684648, 2024-04-16T17:17:51.802 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_recv_time": 0.604639743, 2024-04-16T17:17:51.802 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_fast_dispatch_time": 0.213766524, 2024-04-16T17:17:51.802 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_messages_queue_lat": { 2024-04-16T17:17:51.802 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 18299, 2024-04-16T17:17:51.802 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 1.503668053, 2024-04-16T17:17:51.803 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000082172 2024-04-16T17:17:51.803 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-16T17:17:51.803 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_handle_ack_lat": { 2024-04-16T17:17:51.803 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 21323, 2024-04-16T17:17:51.803 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.003177896, 2024-04-16T17:17:51.803 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000149 2024-04-16T17:17:51.803 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-16T17:17:51.803 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_encrypted_bytes": 344624, 2024-04-16T17:17:51.803 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_encrypted_bytes": 97744 2024-04-16T17:17:51.804 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-16T17:17:51.804 INFO:teuthology.orchestra.run.smithi104.stdout: "cct": { 2024-04-16T17:17:51.804 INFO:teuthology.orchestra.run.smithi104.stdout: "total_workers": 1, 2024-04-16T17:17:51.804 INFO:teuthology.orchestra.run.smithi104.stdout: "unhealthy_workers": 0 2024-04-16T17:17:51.804 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-16T17:17:51.804 INFO:teuthology.orchestra.run.smithi104.stdout: "finisher-MDSRank": { 2024-04-16T17:17:51.804 INFO:teuthology.orchestra.run.smithi104.stdout: "queue_len": 0, 2024-04-16T17:17:51.804 INFO:teuthology.orchestra.run.smithi104.stdout: "complete_latency": { 2024-04-16T17:17:51.804 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 15394, 2024-04-16T17:17:51.804 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 8.501040542, 2024-04-16T17:17:51.805 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000552230 2024-04-16T17:17:51.805 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-04-16T17:17:51.805 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-16T17:17:51.805 INFO:teuthology.orchestra.run.smithi104.stdout: "finisher-PurgeQueue": { 2024-04-16T17:17:51.805 INFO:teuthology.orchestra.run.smithi104.stdout: "queue_len": 0, 2024-04-16T17:17:51.805 INFO:teuthology.orchestra.run.smithi104.stdout: "complete_latency": { 2024-04-16T17:17:51.805 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 3971, 2024-04-16T17:17:51.805 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.226003883, 2024-04-16T17:17:51.805 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000056913 2024-04-16T17:17:51.805 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-04-16T17:17:51.806 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-16T17:17:51.806 INFO:teuthology.orchestra.run.smithi104.stdout: "mds": { 2024-04-16T17:17:51.806 INFO:teuthology.orchestra.run.smithi104.stdout: "request": 13925, 2024-04-16T17:17:51.806 INFO:teuthology.orchestra.run.smithi104.stdout: "reply": 12451, 2024-04-16T17:17:51.806 INFO:teuthology.orchestra.run.smithi104.stdout: "reply_latency": { 2024-04-16T17:17:51.806 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 12451, 2024-04-16T17:17:51.806 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 57.449460407, 2024-04-16T17:17:51.806 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.004614043 2024-04-16T17:17:51.806 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-16T17:17:51.806 INFO:teuthology.orchestra.run.smithi104.stdout: "slow_reply": 0, 2024-04-16T17:17:51.807 INFO:teuthology.orchestra.run.smithi104.stdout: "forward": 1474, 2024-04-16T17:17:51.807 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_fetch_complete": 10, 2024-04-16T17:17:51.807 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_fetch_keys": 2, 2024-04-16T17:17:51.807 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_commit": 567, 2024-04-16T17:17:51.807 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_split": 0, 2024-04-16T17:17:51.807 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_merge": 0, 2024-04-16T17:17:51.807 INFO:teuthology.orchestra.run.smithi104.stdout: "inodes": 292, 2024-04-16T17:17:51.807 INFO:teuthology.orchestra.run.smithi104.stdout: "inodes_top": 96, 2024-04-16T17:17:51.807 INFO:teuthology.orchestra.run.smithi104.stdout: "inodes_bottom": 161, 2024-04-16T17:17:51.807 INFO:teuthology.orchestra.run.smithi104.stdout: "inodes_pin_tail": 35, 2024-04-16T17:17:51.808 INFO:teuthology.orchestra.run.smithi104.stdout: "inodes_pinned": 155, 2024-04-16T17:17:51.808 INFO:teuthology.orchestra.run.smithi104.stdout: "inodes_expired": 259, 2024-04-16T17:17:51.808 INFO:teuthology.orchestra.run.smithi104.stdout: "inodes_with_caps": 36, 2024-04-16T17:17:51.808 INFO:teuthology.orchestra.run.smithi104.stdout: "caps": 37, 2024-04-16T17:17:51.808 INFO:teuthology.orchestra.run.smithi104.stdout: "subtrees": 12, 2024-04-16T17:17:51.808 INFO:teuthology.orchestra.run.smithi104.stdout: "traverse": 18787, 2024-04-16T17:17:51.808 INFO:teuthology.orchestra.run.smithi104.stdout: "traverse_hit": 15601, 2024-04-16T17:17:51.808 INFO:teuthology.orchestra.run.smithi104.stdout: "traverse_forward": 1467, 2024-04-16T17:17:51.808 INFO:teuthology.orchestra.run.smithi104.stdout: "traverse_discover": 7, 2024-04-16T17:17:51.808 INFO:teuthology.orchestra.run.smithi104.stdout: "traverse_dir_fetch": 1, 2024-04-16T17:17:51.809 INFO:teuthology.orchestra.run.smithi104.stdout: "traverse_remote_ino": 0, 2024-04-16T17:17:51.809 INFO:teuthology.orchestra.run.smithi104.stdout: "traverse_lock": 9, 2024-04-16T17:17:51.809 INFO:teuthology.orchestra.run.smithi104.stdout: "load_cent": 0, 2024-04-16T17:17:51.809 INFO:teuthology.orchestra.run.smithi104.stdout: "q": 0, 2024-04-16T17:17:51.809 INFO:teuthology.orchestra.run.smithi104.stdout: "exported": 5, 2024-04-16T17:17:51.809 INFO:teuthology.orchestra.run.smithi104.stdout: "exported_inodes": 173, 2024-04-16T17:17:51.809 INFO:teuthology.orchestra.run.smithi104.stdout: "imported": 3, 2024-04-16T17:17:51.809 INFO:teuthology.orchestra.run.smithi104.stdout: "imported_inodes": 33, 2024-04-16T17:17:51.809 INFO:teuthology.orchestra.run.smithi104.stdout: "openino_dir_fetch": 0, 2024-04-16T17:17:51.810 INFO:teuthology.orchestra.run.smithi104.stdout: "openino_backtrace_fetch": 891, 2024-04-16T17:17:51.810 INFO:teuthology.orchestra.run.smithi104.stdout: "openino_peer_discover": 0, 2024-04-16T17:17:51.810 INFO:teuthology.orchestra.run.smithi104.stdout: "root_rfiles": 68, 2024-04-16T17:17:51.810 INFO:teuthology.orchestra.run.smithi104.stdout: "root_rbytes": 21196213, 2024-04-16T17:17:51.810 INFO:teuthology.orchestra.run.smithi104.stdout: "root_rsnaps": 6, 2024-04-16T17:17:51.810 INFO:teuthology.orchestra.run.smithi104.stdout: "scrub_backtrace_fetch": 891, 2024-04-16T17:17:51.810 INFO:teuthology.orchestra.run.smithi104.stdout: "scrub_set_tag": 0, 2024-04-16T17:17:51.810 INFO:teuthology.orchestra.run.smithi104.stdout: "scrub_backtrace_repaired": 0, 2024-04-16T17:17:51.810 INFO:teuthology.orchestra.run.smithi104.stdout: "scrub_inotable_repaired": 0, 2024-04-16T17:17:51.810 INFO:teuthology.orchestra.run.smithi104.stdout: "scrub_dir_inodes": 380, 2024-04-16T17:17:51.811 INFO:teuthology.orchestra.run.smithi104.stdout: "scrub_dir_base_inodes": 20, 2024-04-16T17:17:51.811 INFO:teuthology.orchestra.run.smithi104.stdout: "scrub_dirfrag_rstats": 360, 2024-04-16T17:17:51.811 INFO:teuthology.orchestra.run.smithi104.stdout: "scrub_file_inodes": 511, 2024-04-16T17:17:51.811 INFO:teuthology.orchestra.run.smithi104.stdout: "handle_inode_file_caps": 9, 2024-04-16T17:17:51.811 INFO:teuthology.orchestra.run.smithi104.stdout: "ceph_cap_op_revoke": 3210, 2024-04-16T17:17:51.811 INFO:teuthology.orchestra.run.smithi104.stdout: "ceph_cap_op_grant": 7810, 2024-04-16T17:17:51.811 INFO:teuthology.orchestra.run.smithi104.stdout: "ceph_cap_op_trunc": 0, 2024-04-16T17:17:51.811 INFO:teuthology.orchestra.run.smithi104.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-16T17:17:51.812 INFO:teuthology.orchestra.run.smithi104.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-16T17:17:51.812 INFO:teuthology.orchestra.run.smithi104.stdout: "handle_client_caps": 6264, 2024-04-16T17:17:51.812 INFO:teuthology.orchestra.run.smithi104.stdout: "handle_client_caps_dirty": 1871, 2024-04-16T17:17:51.812 INFO:teuthology.orchestra.run.smithi104.stdout: "handle_client_cap_release": 878, 2024-04-16T17:17:51.812 INFO:teuthology.orchestra.run.smithi104.stdout: "process_request_cap_release": 6408 2024-04-16T17:17:51.812 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-16T17:17:51.812 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_cache": { 2024-04-16T17:17:51.812 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_update": 2, 2024-04-16T17:17:51.812 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_update_receipt": 0, 2024-04-16T17:17:51.813 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_try_discover": 0, 2024-04-16T17:17:51.813 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_send_discover": 7, 2024-04-16T17:17:51.813 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_handle_discover": 24, 2024-04-16T17:17:51.813 INFO:teuthology.orchestra.run.smithi104.stdout: "num_strays": 0, 2024-04-16T17:17:51.813 INFO:teuthology.orchestra.run.smithi104.stdout: "num_strays_delayed": 0, 2024-04-16T17:17:51.813 INFO:teuthology.orchestra.run.smithi104.stdout: "num_strays_enqueuing": 0, 2024-04-16T17:17:51.813 INFO:teuthology.orchestra.run.smithi104.stdout: "strays_created": 1967, 2024-04-16T17:17:51.813 INFO:teuthology.orchestra.run.smithi104.stdout: "strays_enqueued": 1967, 2024-04-16T17:17:51.813 INFO:teuthology.orchestra.run.smithi104.stdout: "strays_reintegrated": 0, 2024-04-16T17:17:51.813 INFO:teuthology.orchestra.run.smithi104.stdout: "strays_migrated": 0, 2024-04-16T17:17:51.814 INFO:teuthology.orchestra.run.smithi104.stdout: "num_recovering_processing": 0, 2024-04-16T17:17:51.814 INFO:teuthology.orchestra.run.smithi104.stdout: "num_recovering_enqueued": 0, 2024-04-16T17:17:51.814 INFO:teuthology.orchestra.run.smithi104.stdout: "num_recovering_prioritized": 0, 2024-04-16T17:17:51.814 INFO:teuthology.orchestra.run.smithi104.stdout: "recovery_started": 0, 2024-04-16T17:17:51.814 INFO:teuthology.orchestra.run.smithi104.stdout: "recovery_completed": 0, 2024-04-16T17:17:51.814 INFO:teuthology.orchestra.run.smithi104.stdout: "ireq_quiesce_path": 0, 2024-04-16T17:17:51.814 INFO:teuthology.orchestra.run.smithi104.stdout: "ireq_quiesce_inode": 0, 2024-04-16T17:17:51.814 INFO:teuthology.orchestra.run.smithi104.stdout: "ireq_enqueue_scrub": 20, 2024-04-16T17:17:51.815 INFO:teuthology.orchestra.run.smithi104.stdout: "ireq_exportdir": 5, 2024-04-16T17:17:51.815 INFO:teuthology.orchestra.run.smithi104.stdout: "ireq_flush": 0, 2024-04-16T17:17:51.815 INFO:teuthology.orchestra.run.smithi104.stdout: "ireq_fragmentdir": 0, 2024-04-16T17:17:51.815 INFO:teuthology.orchestra.run.smithi104.stdout: "ireq_fragstats": 0, 2024-04-16T17:17:51.815 INFO:teuthology.orchestra.run.smithi104.stdout: "ireq_inodestats": 0 2024-04-16T17:17:51.815 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-16T17:17:51.815 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_log": { 2024-04-16T17:17:51.815 INFO:teuthology.orchestra.run.smithi104.stdout: "evlrg": 0, 2024-04-16T17:17:51.816 INFO:teuthology.orchestra.run.smithi104.stdout: "evadd": 13477, 2024-04-16T17:17:51.816 INFO:teuthology.orchestra.run.smithi104.stdout: "evex": 13168, 2024-04-16T17:17:51.816 INFO:teuthology.orchestra.run.smithi104.stdout: "evtrm": 13168, 2024-04-16T17:17:51.816 INFO:teuthology.orchestra.run.smithi104.stdout: "ev": 309, 2024-04-16T17:17:51.816 INFO:teuthology.orchestra.run.smithi104.stdout: "evexg": 0, 2024-04-16T17:17:51.816 INFO:teuthology.orchestra.run.smithi104.stdout: "evexd": 547, 2024-04-16T17:17:51.816 INFO:teuthology.orchestra.run.smithi104.stdout: "segadd": 50, 2024-04-16T17:17:51.816 INFO:teuthology.orchestra.run.smithi104.stdout: "segex": 48, 2024-04-16T17:17:51.816 INFO:teuthology.orchestra.run.smithi104.stdout: "segtrm": 48, 2024-04-16T17:17:51.816 INFO:teuthology.orchestra.run.smithi104.stdout: "seg": 2, 2024-04-16T17:17:51.817 INFO:teuthology.orchestra.run.smithi104.stdout: "segmjr": 1, 2024-04-16T17:17:51.817 INFO:teuthology.orchestra.run.smithi104.stdout: "segexg": 0, 2024-04-16T17:17:51.817 INFO:teuthology.orchestra.run.smithi104.stdout: "segexd": 1, 2024-04-16T17:17:51.817 INFO:teuthology.orchestra.run.smithi104.stdout: "expos": 66916694, 2024-04-16T17:17:51.817 INFO:teuthology.orchestra.run.smithi104.stdout: "wrpos": 68496824, 2024-04-16T17:17:51.817 INFO:teuthology.orchestra.run.smithi104.stdout: "rdpos": 0, 2024-04-16T17:17:51.817 INFO:teuthology.orchestra.run.smithi104.stdout: "jlat": { 2024-04-16T17:17:51.817 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 7179, 2024-04-16T17:17:51.817 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 20.868942668, 2024-04-16T17:17:51.818 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.002906942 2024-04-16T17:17:51.818 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-16T17:17:51.818 INFO:teuthology.orchestra.run.smithi104.stdout: "replayed": 0 2024-04-16T17:17:51.818 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-16T17:17:51.818 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_mem": { 2024-04-16T17:17:51.818 INFO:teuthology.orchestra.run.smithi104.stdout: "ino": 164, 2024-04-16T17:17:51.818 INFO:teuthology.orchestra.run.smithi104.stdout: "ino+": 2318, 2024-04-16T17:17:51.818 INFO:teuthology.orchestra.run.smithi104.stdout: "ino-": 2154, 2024-04-16T17:17:51.818 INFO:teuthology.orchestra.run.smithi104.stdout: "dir": 41, 2024-04-16T17:17:51.818 INFO:teuthology.orchestra.run.smithi104.stdout: "dir+": 46, 2024-04-16T17:17:51.819 INFO:teuthology.orchestra.run.smithi104.stdout: "dir-": 5, 2024-04-16T17:17:51.819 INFO:teuthology.orchestra.run.smithi104.stdout: "dn": 292, 2024-04-16T17:17:51.819 INFO:teuthology.orchestra.run.smithi104.stdout: "dn+": 3595, 2024-04-16T17:17:51.819 INFO:teuthology.orchestra.run.smithi104.stdout: "dn-": 3303, 2024-04-16T17:17:51.819 INFO:teuthology.orchestra.run.smithi104.stdout: "cap": 37, 2024-04-16T17:17:51.819 INFO:teuthology.orchestra.run.smithi104.stdout: "cap+": 2085, 2024-04-16T17:17:51.819 INFO:teuthology.orchestra.run.smithi104.stdout: "cap-": 2048, 2024-04-16T17:17:51.819 INFO:teuthology.orchestra.run.smithi104.stdout: "rss": 65568, 2024-04-16T17:17:51.819 INFO:teuthology.orchestra.run.smithi104.stdout: "heap": 231800 2024-04-16T17:17:51.819 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-16T17:17:51.820 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_server": { 2024-04-16T17:17:51.820 INFO:teuthology.orchestra.run.smithi104.stdout: "dispatch_client_request": 19865, 2024-04-16T17:17:51.820 INFO:teuthology.orchestra.run.smithi104.stdout: "dispatch_server_request": 5227, 2024-04-16T17:17:51.820 INFO:teuthology.orchestra.run.smithi104.stdout: "handle_client_request": 13925, 2024-04-16T17:17:51.820 INFO:teuthology.orchestra.run.smithi104.stdout: "handle_client_session": 408, 2024-04-16T17:17:51.820 INFO:teuthology.orchestra.run.smithi104.stdout: "handle_peer_request": 10499, 2024-04-16T17:17:51.820 INFO:teuthology.orchestra.run.smithi104.stdout: "req_create_latency": { 2024-04-16T17:17:51.820 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 2030, 2024-04-16T17:17:51.820 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 2.179698358, 2024-04-16T17:17:51.821 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.001073743 2024-04-16T17:17:51.821 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-16T17:17:51.821 INFO:teuthology.orchestra.run.smithi104.stdout: "req_getattr_latency": { 2024-04-16T17:17:51.821 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 3328, 2024-04-16T17:17:51.821 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 21.467891242, 2024-04-16T17:17:51.821 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.006450688 2024-04-16T17:17:51.821 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-16T17:17:51.821 INFO:teuthology.orchestra.run.smithi104.stdout: "req_getfilelock_latency": { 2024-04-16T17:17:51.821 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-04-16T17:17:51.821 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-04-16T17:17:51.822 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-04-16T17:17:51.822 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-16T17:17:51.822 INFO:teuthology.orchestra.run.smithi104.stdout: "req_link_latency": { 2024-04-16T17:17:51.822 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-04-16T17:17:51.822 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-04-16T17:17:51.822 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-04-16T17:17:51.822 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-16T17:17:51.822 INFO:teuthology.orchestra.run.smithi104.stdout: "req_lookup_latency": { 2024-04-16T17:17:51.822 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 814, 2024-04-16T17:17:51.823 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 5.060526009, 2024-04-16T17:17:51.823 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.006216862 2024-04-16T17:17:51.823 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-16T17:17:51.823 INFO:teuthology.orchestra.run.smithi104.stdout: "req_lookuphash_latency": { 2024-04-16T17:17:51.823 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-04-16T17:17:51.823 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-04-16T17:17:51.823 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-04-16T17:17:51.823 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-16T17:17:51.823 INFO:teuthology.orchestra.run.smithi104.stdout: "req_lookupino_latency": { 2024-04-16T17:17:51.823 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 2, 2024-04-16T17:17:51.824 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000312036, 2024-04-16T17:17:51.824 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000156018 2024-04-16T17:17:51.824 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-16T17:17:51.824 INFO:teuthology.orchestra.run.smithi104.stdout: "req_lookupname_latency": { 2024-04-16T17:17:51.824 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-04-16T17:17:51.824 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-04-16T17:17:51.824 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-04-16T17:17:51.824 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-16T17:17:51.824 INFO:teuthology.orchestra.run.smithi104.stdout: "req_lookupparent_latency": { 2024-04-16T17:17:51.824 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-04-16T17:17:51.825 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-04-16T17:17:51.825 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-04-16T17:17:51.825 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-16T17:17:51.825 INFO:teuthology.orchestra.run.smithi104.stdout: "req_lookupsnap_latency": { 2024-04-16T17:17:51.825 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-04-16T17:17:51.825 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-04-16T17:17:51.825 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-04-16T17:17:51.825 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-16T17:17:51.826 INFO:teuthology.orchestra.run.smithi104.stdout: "req_lssnap_latency": { 2024-04-16T17:17:51.826 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 11, 2024-04-16T17:17:51.826 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.002067304, 2024-04-16T17:17:51.826 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000187936 2024-04-16T17:17:51.826 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-16T17:17:51.827 INFO:teuthology.orchestra.run.smithi104.stdout: "req_mkdir_latency": { 2024-04-16T17:17:51.827 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 20, 2024-04-16T17:17:51.827 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.026708042, 2024-04-16T17:17:51.827 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.001335402 2024-04-16T17:17:51.827 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-16T17:17:51.827 INFO:teuthology.orchestra.run.smithi104.stdout: "req_mknod_latency": { 2024-04-16T17:17:51.827 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-04-16T17:17:51.827 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-04-16T17:17:51.827 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-04-16T17:17:51.828 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-16T17:17:51.828 INFO:teuthology.orchestra.run.smithi104.stdout: "req_mksnap_latency": { 2024-04-16T17:17:51.828 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 11, 2024-04-16T17:17:51.828 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 3.710275551, 2024-04-16T17:17:51.828 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.337297777 2024-04-16T17:17:51.828 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-16T17:17:51.828 INFO:teuthology.orchestra.run.smithi104.stdout: "req_open_latency": { 2024-04-16T17:17:51.828 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 1, 2024-04-16T17:17:51.829 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000215213, 2024-04-16T17:17:51.829 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000215213 2024-04-16T17:17:51.829 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-16T17:17:51.829 INFO:teuthology.orchestra.run.smithi104.stdout: "req_readdir_latency": { 2024-04-16T17:17:51.829 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 3575, 2024-04-16T17:17:51.829 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 7.898390938, 2024-04-16T17:17:51.829 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.002209340 2024-04-16T17:17:51.829 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-16T17:17:51.829 INFO:teuthology.orchestra.run.smithi104.stdout: "req_rename_latency": { 2024-04-16T17:17:51.830 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 527, 2024-04-16T17:17:51.830 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 13.609546316, 2024-04-16T17:17:51.830 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.025824566 2024-04-16T17:17:51.830 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-16T17:17:51.830 INFO:teuthology.orchestra.run.smithi104.stdout: "req_renamesnap_latency": { 2024-04-16T17:17:51.830 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-04-16T17:17:51.830 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-04-16T17:17:51.830 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-04-16T17:17:51.830 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-16T17:17:51.831 INFO:teuthology.orchestra.run.smithi104.stdout: "req_snapdiff_latency": { 2024-04-16T17:17:51.831 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-04-16T17:17:51.831 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-04-16T17:17:51.831 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-04-16T17:17:51.831 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-16T17:17:51.831 INFO:teuthology.orchestra.run.smithi104.stdout: "req_rmdir_latency": { 2024-04-16T17:17:51.831 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-04-16T17:17:51.831 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-04-16T17:17:51.832 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-04-16T17:17:51.832 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-16T17:17:51.832 INFO:teuthology.orchestra.run.smithi104.stdout: "req_rmsnap_latency": { 2024-04-16T17:17:51.832 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 5, 2024-04-16T17:17:51.832 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.031448451, 2024-04-16T17:17:51.832 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.006289690 2024-04-16T17:17:51.832 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-16T17:17:51.832 INFO:teuthology.orchestra.run.smithi104.stdout: "req_rmxattr_latency": { 2024-04-16T17:17:51.832 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-04-16T17:17:51.833 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-04-16T17:17:51.833 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-04-16T17:17:51.833 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-16T17:17:51.833 INFO:teuthology.orchestra.run.smithi104.stdout: "req_setattr_latency": { 2024-04-16T17:17:51.833 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 12, 2024-04-16T17:17:51.833 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.034863441, 2024-04-16T17:17:51.833 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.002905286 2024-04-16T17:17:51.833 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-16T17:17:51.833 INFO:teuthology.orchestra.run.smithi104.stdout: "req_setdirlayout_latency": { 2024-04-16T17:17:51.834 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-04-16T17:17:51.834 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-04-16T17:17:51.834 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-04-16T17:17:51.834 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-16T17:17:51.834 INFO:teuthology.orchestra.run.smithi104.stdout: "req_setfilelock_latency": { 2024-04-16T17:17:51.834 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 140, 2024-04-16T17:17:51.834 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.046109603, 2024-04-16T17:17:51.834 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000329354 2024-04-16T17:17:51.834 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-16T17:17:51.835 INFO:teuthology.orchestra.run.smithi104.stdout: "req_setlayout_latency": { 2024-04-16T17:17:51.835 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-04-16T17:17:51.835 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-04-16T17:17:51.835 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-04-16T17:17:51.835 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-16T17:17:51.835 INFO:teuthology.orchestra.run.smithi104.stdout: "req_setxattr_latency": { 2024-04-16T17:17:51.835 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 8, 2024-04-16T17:17:51.835 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.022856823, 2024-04-16T17:17:51.836 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.002857102 2024-04-16T17:17:51.836 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-16T17:17:51.836 INFO:teuthology.orchestra.run.smithi104.stdout: "req_symlink_latency": { 2024-04-16T17:17:51.836 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-04-16T17:17:51.836 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-04-16T17:17:51.836 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-04-16T17:17:51.836 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-16T17:17:51.836 INFO:teuthology.orchestra.run.smithi104.stdout: "req_unlink_latency": { 2024-04-16T17:17:51.836 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 1967, 2024-04-16T17:17:51.837 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 3.358551080, 2024-04-16T17:17:51.837 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.001707448 2024-04-16T17:17:51.837 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-16T17:17:51.837 INFO:teuthology.orchestra.run.smithi104.stdout: "cap_revoke_eviction": 0, 2024-04-16T17:17:51.837 INFO:teuthology.orchestra.run.smithi104.stdout: "cap_acquisition_throttle": 0, 2024-04-16T17:17:51.837 INFO:teuthology.orchestra.run.smithi104.stdout: "req_getvxattr_latency": { 2024-04-16T17:17:51.837 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-04-16T17:17:51.837 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-04-16T17:17:51.837 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-04-16T17:17:51.838 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-04-16T17:17:51.838 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-16T17:17:51.838 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_sessions": { 2024-04-16T17:17:51.838 INFO:teuthology.orchestra.run.smithi104.stdout: "session_count": 2, 2024-04-16T17:17:51.838 INFO:teuthology.orchestra.run.smithi104.stdout: "session_add": 6, 2024-04-16T17:17:51.838 INFO:teuthology.orchestra.run.smithi104.stdout: "session_remove": 4, 2024-04-16T17:17:51.838 INFO:teuthology.orchestra.run.smithi104.stdout: "sessions_open": 2, 2024-04-16T17:17:51.838 INFO:teuthology.orchestra.run.smithi104.stdout: "sessions_stale": 0, 2024-04-16T17:17:51.839 INFO:teuthology.orchestra.run.smithi104.stdout: "total_load": 414, 2024-04-16T17:17:51.839 INFO:teuthology.orchestra.run.smithi104.stdout: "average_load": 207, 2024-04-16T17:17:51.839 INFO:teuthology.orchestra.run.smithi104.stdout: "avg_session_uptime": 656, 2024-04-16T17:17:51.839 INFO:teuthology.orchestra.run.smithi104.stdout: "mdthresh_evicted": 0 2024-04-16T17:17:51.839 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-16T17:17:51.839 INFO:teuthology.orchestra.run.smithi104.stdout: "mempool": { 2024-04-16T17:17:51.839 INFO:teuthology.orchestra.run.smithi104.stdout: "bloom_filter_bytes": 0, 2024-04-16T17:17:51.839 INFO:teuthology.orchestra.run.smithi104.stdout: "bloom_filter_items": 0, 2024-04-16T17:17:51.839 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_alloc_bytes": 0, 2024-04-16T17:17:51.840 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_alloc_items": 0, 2024-04-16T17:17:51.840 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_data_bytes": 0, 2024-04-16T17:17:51.840 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_data_items": 0, 2024-04-16T17:17:51.840 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-16T17:17:51.840 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_onode_items": 0, 2024-04-16T17:17:51.840 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-16T17:17:51.840 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_meta_items": 0, 2024-04-16T17:17:51.840 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_other_bytes": 0, 2024-04-16T17:17:51.840 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_other_items": 0, 2024-04-16T17:17:51.841 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-16T17:17:51.841 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_buffer_items": 0, 2024-04-16T17:17:51.841 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_extent_bytes": 0, 2024-04-16T17:17:51.841 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_extent_items": 0, 2024-04-16T17:17:51.841 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_blob_bytes": 0, 2024-04-16T17:17:51.841 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_blob_items": 0, 2024-04-16T17:17:51.841 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-16T17:17:51.841 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_shared_blob_items": 0, 2024-04-16T17:17:51.841 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-16T17:17:51.842 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_inline_bl_items": 0, 2024-04-16T17:17:51.842 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_fsck_bytes": 0, 2024-04-16T17:17:51.842 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_fsck_items": 0, 2024-04-16T17:17:51.842 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_txc_bytes": 0, 2024-04-16T17:17:51.842 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_txc_items": 0, 2024-04-16T17:17:51.842 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-16T17:17:51.842 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_writing_deferred_items": 0, 2024-04-16T17:17:51.842 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_writing_bytes": 0, 2024-04-16T17:17:51.842 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_writing_items": 0, 2024-04-16T17:17:51.843 INFO:teuthology.orchestra.run.smithi104.stdout: "bluefs_bytes": 0, 2024-04-16T17:17:51.843 INFO:teuthology.orchestra.run.smithi104.stdout: "bluefs_items": 0, 2024-04-16T17:17:51.843 INFO:teuthology.orchestra.run.smithi104.stdout: "bluefs_file_reader_bytes": 0, 2024-04-16T17:17:51.843 INFO:teuthology.orchestra.run.smithi104.stdout: "bluefs_file_reader_items": 0, 2024-04-16T17:17:51.843 INFO:teuthology.orchestra.run.smithi104.stdout: "bluefs_file_writer_bytes": 0, 2024-04-16T17:17:51.843 INFO:teuthology.orchestra.run.smithi104.stdout: "bluefs_file_writer_items": 0, 2024-04-16T17:17:51.843 INFO:teuthology.orchestra.run.smithi104.stdout: "buffer_anon_bytes": 315426, 2024-04-16T17:17:51.843 INFO:teuthology.orchestra.run.smithi104.stdout: "buffer_anon_items": 207, 2024-04-16T17:17:51.843 INFO:teuthology.orchestra.run.smithi104.stdout: "buffer_meta_bytes": 0, 2024-04-16T17:17:51.844 INFO:teuthology.orchestra.run.smithi104.stdout: "buffer_meta_items": 0, 2024-04-16T17:17:51.844 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_bytes": 0, 2024-04-16T17:17:51.844 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_items": 0, 2024-04-16T17:17:51.844 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_mapbl_bytes": 0, 2024-04-16T17:17:51.844 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_mapbl_items": 0, 2024-04-16T17:17:51.844 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_pglog_bytes": 0, 2024-04-16T17:17:51.844 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_pglog_items": 0, 2024-04-16T17:17:51.844 INFO:teuthology.orchestra.run.smithi104.stdout: "osdmap_bytes": 9512, 2024-04-16T17:17:51.844 INFO:teuthology.orchestra.run.smithi104.stdout: "osdmap_items": 182, 2024-04-16T17:17:51.845 INFO:teuthology.orchestra.run.smithi104.stdout: "osdmap_mapping_bytes": 0, 2024-04-16T17:17:51.845 INFO:teuthology.orchestra.run.smithi104.stdout: "osdmap_mapping_items": 0, 2024-04-16T17:17:51.845 INFO:teuthology.orchestra.run.smithi104.stdout: "pgmap_bytes": 0, 2024-04-16T17:17:51.845 INFO:teuthology.orchestra.run.smithi104.stdout: "pgmap_items": 0, 2024-04-16T17:17:51.845 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_co_bytes": 770879, 2024-04-16T17:17:51.845 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_co_items": 7558, 2024-04-16T17:17:51.845 INFO:teuthology.orchestra.run.smithi104.stdout: "unittest_1_bytes": 0, 2024-04-16T17:17:51.845 INFO:teuthology.orchestra.run.smithi104.stdout: "unittest_1_items": 0, 2024-04-16T17:17:51.845 INFO:teuthology.orchestra.run.smithi104.stdout: "unittest_2_bytes": 0, 2024-04-16T17:17:51.846 INFO:teuthology.orchestra.run.smithi104.stdout: "unittest_2_items": 0 2024-04-16T17:17:51.846 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-16T17:17:51.846 INFO:teuthology.orchestra.run.smithi104.stdout: "objecter": { 2024-04-16T17:17:51.846 INFO:teuthology.orchestra.run.smithi104.stdout: "op_active": 0, 2024-04-16T17:17:51.846 INFO:teuthology.orchestra.run.smithi104.stdout: "op_laggy": 0, 2024-04-16T17:17:51.846 INFO:teuthology.orchestra.run.smithi104.stdout: "op_send": 16554, 2024-04-16T17:17:51.846 INFO:teuthology.orchestra.run.smithi104.stdout: "op_send_bytes": 67995939, 2024-04-16T17:17:51.846 INFO:teuthology.orchestra.run.smithi104.stdout: "op_resend": 31, 2024-04-16T17:17:51.846 INFO:teuthology.orchestra.run.smithi104.stdout: "op_reply": 16554, 2024-04-16T17:17:51.847 INFO:teuthology.orchestra.run.smithi104.stdout: "op_latency": { 2024-04-16T17:17:51.847 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 16554, 2024-04-16T17:17:51.847 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 174.396716717, 2024-04-16T17:17:51.847 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.010535019 2024-04-16T17:17:51.847 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-16T17:17:51.847 INFO:teuthology.orchestra.run.smithi104.stdout: "op_inflight": 0, 2024-04-16T17:17:51.847 INFO:teuthology.orchestra.run.smithi104.stdout: "oplen_avg": { 2024-04-16T17:17:51.847 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 16554, 2024-04-16T17:17:51.847 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 21913 2024-04-16T17:17:51.848 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-16T17:17:51.848 INFO:teuthology.orchestra.run.smithi104.stdout: "op": 16554, 2024-04-16T17:17:51.848 INFO:teuthology.orchestra.run.smithi104.stdout: "op_r": 1113, 2024-04-16T17:17:51.848 INFO:teuthology.orchestra.run.smithi104.stdout: "op_w": 15441, 2024-04-16T17:17:51.848 INFO:teuthology.orchestra.run.smithi104.stdout: "op_rmw": 0, 2024-04-16T17:17:51.848 INFO:teuthology.orchestra.run.smithi104.stdout: "op_pg": 0, 2024-04-16T17:17:51.848 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_stat": 535, 2024-04-16T17:17:51.848 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_create": 2508, 2024-04-16T17:17:51.848 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_read": 190, 2024-04-16T17:17:51.849 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_write": 7349, 2024-04-16T17:17:51.849 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_writefull": 245, 2024-04-16T17:17:51.849 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_writesame": 0, 2024-04-16T17:17:51.849 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_append": 0, 2024-04-16T17:17:51.849 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_zero": 0, 2024-04-16T17:17:51.849 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_truncate": 0, 2024-04-16T17:17:51.849 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_delete": 4475, 2024-04-16T17:17:51.849 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_mapext": 0, 2024-04-16T17:17:51.849 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_sparse_read": 0, 2024-04-16T17:17:51.850 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_clonerange": 0, 2024-04-16T17:17:51.850 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_getxattr": 923, 2024-04-16T17:17:51.850 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_setxattr": 3834, 2024-04-16T17:17:51.850 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_cmpxattr": 0, 2024-04-16T17:17:51.850 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_rmxattr": 0, 2024-04-16T17:17:51.850 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_resetxattrs": 0, 2024-04-16T17:17:51.850 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_call": 0, 2024-04-16T17:17:51.850 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_watch": 0, 2024-04-16T17:17:51.850 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_notify": 0, 2024-04-16T17:17:51.851 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_src_cmpxattr": 0, 2024-04-16T17:17:51.851 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_pgls": 0, 2024-04-16T17:17:51.851 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_pgls_filter": 0, 2024-04-16T17:17:51.851 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_other": 0, 2024-04-16T17:17:51.851 INFO:teuthology.orchestra.run.smithi104.stdout: "linger_active": 0, 2024-04-16T17:17:51.851 INFO:teuthology.orchestra.run.smithi104.stdout: "linger_send": 0, 2024-04-16T17:17:51.851 INFO:teuthology.orchestra.run.smithi104.stdout: "linger_resend": 0, 2024-04-16T17:17:51.851 INFO:teuthology.orchestra.run.smithi104.stdout: "linger_ping": 0, 2024-04-16T17:17:51.851 INFO:teuthology.orchestra.run.smithi104.stdout: "poolop_active": 0, 2024-04-16T17:17:51.852 INFO:teuthology.orchestra.run.smithi104.stdout: "poolop_send": 0, 2024-04-16T17:17:51.852 INFO:teuthology.orchestra.run.smithi104.stdout: "poolop_resend": 0, 2024-04-16T17:17:51.852 INFO:teuthology.orchestra.run.smithi104.stdout: "poolstat_active": 0, 2024-04-16T17:17:51.852 INFO:teuthology.orchestra.run.smithi104.stdout: "poolstat_send": 0, 2024-04-16T17:17:51.852 INFO:teuthology.orchestra.run.smithi104.stdout: "poolstat_resend": 0, 2024-04-16T17:17:51.852 INFO:teuthology.orchestra.run.smithi104.stdout: "statfs_active": 0, 2024-04-16T17:17:51.852 INFO:teuthology.orchestra.run.smithi104.stdout: "statfs_send": 0, 2024-04-16T17:17:51.852 INFO:teuthology.orchestra.run.smithi104.stdout: "statfs_resend": 0, 2024-04-16T17:17:51.852 INFO:teuthology.orchestra.run.smithi104.stdout: "command_active": 0, 2024-04-16T17:17:51.853 INFO:teuthology.orchestra.run.smithi104.stdout: "command_send": 0, 2024-04-16T17:17:51.853 INFO:teuthology.orchestra.run.smithi104.stdout: "command_resend": 0, 2024-04-16T17:17:51.853 INFO:teuthology.orchestra.run.smithi104.stdout: "map_epoch": 106, 2024-04-16T17:17:51.853 INFO:teuthology.orchestra.run.smithi104.stdout: "map_full": 0, 2024-04-16T17:17:51.853 INFO:teuthology.orchestra.run.smithi104.stdout: "map_inc": 21, 2024-04-16T17:17:51.853 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_sessions": 12, 2024-04-16T17:17:51.853 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_session_open": 12, 2024-04-16T17:17:51.853 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_session_close": 0, 2024-04-16T17:17:51.853 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_laggy": 0, 2024-04-16T17:17:51.854 INFO:teuthology.orchestra.run.smithi104.stdout: "omap_wr": 1558, 2024-04-16T17:17:51.854 INFO:teuthology.orchestra.run.smithi104.stdout: "omap_rd": 24, 2024-04-16T17:17:51.854 INFO:teuthology.orchestra.run.smithi104.stdout: "omap_del": 272 2024-04-16T17:17:51.854 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-16T17:17:51.854 INFO:teuthology.orchestra.run.smithi104.stdout: "oft": { 2024-04-16T17:17:51.854 INFO:teuthology.orchestra.run.smithi104.stdout: "omap_total_objs": 1, 2024-04-16T17:17:51.854 INFO:teuthology.orchestra.run.smithi104.stdout: "omap_total_kv_pairs": 0, 2024-04-16T17:17:51.854 INFO:teuthology.orchestra.run.smithi104.stdout: "omap_total_updates": 0, 2024-04-16T17:17:51.854 INFO:teuthology.orchestra.run.smithi104.stdout: "omap_total_removes": 0 2024-04-16T17:17:51.855 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-16T17:17:51.855 INFO:teuthology.orchestra.run.smithi104.stdout: "osd-slow-ops": { 2024-04-16T17:17:51.855 INFO:teuthology.orchestra.run.smithi104.stdout: "slow_ops_count": 0 2024-04-16T17:17:51.855 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-16T17:17:51.855 INFO:teuthology.orchestra.run.smithi104.stdout: "purge_queue": { 2024-04-16T17:17:51.855 INFO:teuthology.orchestra.run.smithi104.stdout: "pq_executing_ops": 0, 2024-04-16T17:17:51.855 INFO:teuthology.orchestra.run.smithi104.stdout: "pq_executing_ops_high_water": 16, 2024-04-16T17:17:51.855 INFO:teuthology.orchestra.run.smithi104.stdout: "pq_executing": 0, 2024-04-16T17:17:51.855 INFO:teuthology.orchestra.run.smithi104.stdout: "pq_executing_high_water": 8, 2024-04-16T17:17:51.856 INFO:teuthology.orchestra.run.smithi104.stdout: "pq_executed_ops": 3934, 2024-04-16T17:17:51.856 INFO:teuthology.orchestra.run.smithi104.stdout: "pq_executed": 1967, 2024-04-16T17:17:51.856 INFO:teuthology.orchestra.run.smithi104.stdout: "pq_item_in_journal": 0 2024-04-16T17:17:51.856 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-16T17:17:51.856 INFO:teuthology.orchestra.run.smithi104.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-16T17:17:51.856 INFO:teuthology.orchestra.run.smithi104.stdout: "val": 0, 2024-04-16T17:17:51.856 INFO:teuthology.orchestra.run.smithi104.stdout: "max": 104857600, 2024-04-16T17:17:51.856 INFO:teuthology.orchestra.run.smithi104.stdout: "get_started": 0, 2024-04-16T17:17:51.856 INFO:teuthology.orchestra.run.smithi104.stdout: "get": 68461, 2024-04-16T17:17:51.856 INFO:teuthology.orchestra.run.smithi104.stdout: "get_sum": 15667689, 2024-04-16T17:17:51.857 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_fail": 0, 2024-04-16T17:17:51.857 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_success": 68461, 2024-04-16T17:17:51.857 INFO:teuthology.orchestra.run.smithi104.stdout: "take": 0, 2024-04-16T17:17:51.857 INFO:teuthology.orchestra.run.smithi104.stdout: "take_sum": 0, 2024-04-16T17:17:51.857 INFO:teuthology.orchestra.run.smithi104.stdout: "put": 68461, 2024-04-16T17:17:51.857 INFO:teuthology.orchestra.run.smithi104.stdout: "put_sum": 15667689, 2024-04-16T17:17:51.857 INFO:teuthology.orchestra.run.smithi104.stdout: "wait": { 2024-04-16T17:17:51.857 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-04-16T17:17:51.857 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-04-16T17:17:51.858 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-04-16T17:17:51.858 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-04-16T17:17:51.858 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-16T17:17:51.858 INFO:teuthology.orchestra.run.smithi104.stdout: "throttle-objecter_bytes": { 2024-04-16T17:17:51.858 INFO:teuthology.orchestra.run.smithi104.stdout: "val": 0, 2024-04-16T17:17:51.858 INFO:teuthology.orchestra.run.smithi104.stdout: "max": 104857600, 2024-04-16T17:17:51.858 INFO:teuthology.orchestra.run.smithi104.stdout: "get_started": 0, 2024-04-16T17:17:51.858 INFO:teuthology.orchestra.run.smithi104.stdout: "get": 0, 2024-04-16T17:17:51.858 INFO:teuthology.orchestra.run.smithi104.stdout: "get_sum": 0, 2024-04-16T17:17:51.859 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_fail": 0, 2024-04-16T17:17:51.859 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_success": 0, 2024-04-16T17:17:51.859 INFO:teuthology.orchestra.run.smithi104.stdout: "take": 16554, 2024-04-16T17:17:51.859 INFO:teuthology.orchestra.run.smithi104.stdout: "take_sum": 68210462, 2024-04-16T17:17:51.859 INFO:teuthology.orchestra.run.smithi104.stdout: "put": 16554, 2024-04-16T17:17:51.859 INFO:teuthology.orchestra.run.smithi104.stdout: "put_sum": 68210462, 2024-04-16T17:17:51.859 INFO:teuthology.orchestra.run.smithi104.stdout: "wait": { 2024-04-16T17:17:51.859 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-04-16T17:17:51.859 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-04-16T17:17:51.860 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-04-16T17:17:51.860 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-04-16T17:17:51.860 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-16T17:17:51.860 INFO:teuthology.orchestra.run.smithi104.stdout: "throttle-objecter_ops": { 2024-04-16T17:17:51.860 INFO:teuthology.orchestra.run.smithi104.stdout: "val": 0, 2024-04-16T17:17:51.860 INFO:teuthology.orchestra.run.smithi104.stdout: "max": 1024, 2024-04-16T17:17:51.860 INFO:teuthology.orchestra.run.smithi104.stdout: "get_started": 0, 2024-04-16T17:17:51.860 INFO:teuthology.orchestra.run.smithi104.stdout: "get": 0, 2024-04-16T17:17:51.860 INFO:teuthology.orchestra.run.smithi104.stdout: "get_sum": 0, 2024-04-16T17:17:51.861 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_fail": 0, 2024-04-16T17:17:51.861 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_success": 0, 2024-04-16T17:17:51.861 INFO:teuthology.orchestra.run.smithi104.stdout: "take": 16554, 2024-04-16T17:17:51.861 INFO:teuthology.orchestra.run.smithi104.stdout: "take_sum": 16554, 2024-04-16T17:17:51.861 INFO:teuthology.orchestra.run.smithi104.stdout: "put": 16554, 2024-04-16T17:17:51.861 INFO:teuthology.orchestra.run.smithi104.stdout: "put_sum": 16554, 2024-04-16T17:17:51.861 INFO:teuthology.orchestra.run.smithi104.stdout: "wait": { 2024-04-16T17:17:51.861 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-04-16T17:17:51.862 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-04-16T17:17:51.862 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-04-16T17:17:51.862 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-04-16T17:17:51.862 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-16T17:17:51.862 INFO:teuthology.orchestra.run.smithi104.stdout: "throttle-write_buf_throttle": { 2024-04-16T17:17:51.862 INFO:teuthology.orchestra.run.smithi104.stdout: "val": 0, 2024-04-16T17:17:51.862 INFO:teuthology.orchestra.run.smithi104.stdout: "max": 3758096384, 2024-04-16T17:17:51.862 INFO:teuthology.orchestra.run.smithi104.stdout: "get_started": 0, 2024-04-16T17:17:51.862 INFO:teuthology.orchestra.run.smithi104.stdout: "get": 1967, 2024-04-16T17:17:51.863 INFO:teuthology.orchestra.run.smithi104.stdout: "get_sum": 214691, 2024-04-16T17:17:51.863 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_fail": 0, 2024-04-16T17:17:51.863 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_success": 1967, 2024-04-16T17:17:51.863 INFO:teuthology.orchestra.run.smithi104.stdout: "take": 0, 2024-04-16T17:17:51.863 INFO:teuthology.orchestra.run.smithi104.stdout: "take_sum": 0, 2024-04-16T17:17:51.863 INFO:teuthology.orchestra.run.smithi104.stdout: "put": 170, 2024-04-16T17:17:51.863 INFO:teuthology.orchestra.run.smithi104.stdout: "put_sum": 214691, 2024-04-16T17:17:51.863 INFO:teuthology.orchestra.run.smithi104.stdout: "wait": { 2024-04-16T17:17:51.863 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-04-16T17:17:51.863 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-04-16T17:17:51.864 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-04-16T17:17:51.864 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-04-16T17:17:51.864 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-16T17:17:51.864 INFO:teuthology.orchestra.run.smithi104.stdout: "throttle-write_buf_throttle-0x55c732ca0500": { 2024-04-16T17:17:51.864 INFO:teuthology.orchestra.run.smithi104.stdout: "val": 0, 2024-04-16T17:17:51.864 INFO:teuthology.orchestra.run.smithi104.stdout: "max": 3758096384, 2024-04-16T17:17:51.864 INFO:teuthology.orchestra.run.smithi104.stdout: "get_started": 0, 2024-04-16T17:17:51.864 INFO:teuthology.orchestra.run.smithi104.stdout: "get": 13477, 2024-04-16T17:17:51.864 INFO:teuthology.orchestra.run.smithi104.stdout: "get_sum": 64302520, 2024-04-16T17:17:51.865 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_fail": 0, 2024-04-16T17:17:51.865 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_success": 13477, 2024-04-16T17:17:51.865 INFO:teuthology.orchestra.run.smithi104.stdout: "take": 0, 2024-04-16T17:17:51.865 INFO:teuthology.orchestra.run.smithi104.stdout: "take_sum": 0, 2024-04-16T17:17:51.865 INFO:teuthology.orchestra.run.smithi104.stdout: "put": 7179, 2024-04-16T17:17:51.865 INFO:teuthology.orchestra.run.smithi104.stdout: "put_sum": 64302520, 2024-04-16T17:17:51.865 INFO:teuthology.orchestra.run.smithi104.stdout: "wait": { 2024-04-16T17:17:51.865 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-04-16T17:17:51.865 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-04-16T17:17:51.866 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-04-16T17:17:51.866 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-04-16T17:17:51.866 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-04-16T17:17:51.866 INFO:teuthology.orchestra.run.smithi104.stdout:} 2024-04-16T17:17:52.507 INFO:tasks.check_counter:Daemon mds.b mds.exported=5 2024-04-16T17:17:52.507 INFO:tasks.check_counter:Daemon mds.b mds.imported=3 2024-04-16T17:17:52.508 INFO:tasks.check_counter:Daemon mds.b mds.root_rsnaps=6 2024-04-16T17:17:52.508 INFO:tasks.check_counter:Daemon mds.b mds_server.req_mksnap_latency.avgcount=11 2024-04-16T17:17:52.508 INFO:tasks.check_counter:Daemon mds.b mds_server.req_rmsnap_latency.avgcount=5 2024-04-16T17:17:52.508 DEBUG:tasks.check_counter:Getting stats from e 2024-04-16T17:17:52.508 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45 shell --fsid b1fecd38-fc11-11ee-bc90-c7b262605968 -- ceph daemon mds.e perf dump 2024-04-16T17:17:52.758 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/mon.b/config 2024-04-16T17:17:52.778 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:17:52 smithi104 ceph-mon[36457]: pgmap v347: 129 pgs: 129 active+clean; 55 MiB data, 874 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-16T17:17:52.804 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:17:52 smithi117 ceph-mon[36730]: pgmap v347: 129 pgs: 129 active+clean; 55 MiB data, 874 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-16T17:17:52.976 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:17:52 smithi045 ceph-mon[31676]: pgmap v347: 129 pgs: 129 active+clean; 55 MiB data, 874 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-16T17:17:54.564 INFO:teuthology.orchestra.run.smithi104.stdout:{ 2024-04-16T17:17:54.564 INFO:teuthology.orchestra.run.smithi104.stdout: "AsyncMessenger::Worker-0": { 2024-04-16T17:17:54.565 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_messages": 0, 2024-04-16T17:17:54.565 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_messages": 2, 2024-04-16T17:17:54.565 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_bytes": 0, 2024-04-16T17:17:54.565 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_bytes": 0, 2024-04-16T17:17:54.565 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_created_connections": 4, 2024-04-16T17:17:54.565 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_active_connections": 0, 2024-04-16T17:17:54.565 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_total_time": 0.001027944, 2024-04-16T17:17:54.565 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_send_time": 0.000000000, 2024-04-16T17:17:54.566 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_recv_time": 0.000178501, 2024-04-16T17:17:54.566 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-16T17:17:54.566 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_messages_queue_lat": { 2024-04-16T17:17:54.566 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-04-16T17:17:54.566 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-04-16T17:17:54.566 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-04-16T17:17:54.566 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-16T17:17:54.566 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_handle_ack_lat": { 2024-04-16T17:17:54.567 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-04-16T17:17:54.567 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-04-16T17:17:54.567 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-04-16T17:17:54.567 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-16T17:17:54.567 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-16T17:17:54.567 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_encrypted_bytes": 0 2024-04-16T17:17:54.567 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-16T17:17:54.567 INFO:teuthology.orchestra.run.smithi104.stdout: "AsyncMessenger::Worker-1": { 2024-04-16T17:17:54.568 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_messages": 254, 2024-04-16T17:17:54.568 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_messages": 241, 2024-04-16T17:17:54.568 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_bytes": 1600304, 2024-04-16T17:17:54.568 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_bytes": 148192, 2024-04-16T17:17:54.568 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_created_connections": 2, 2024-04-16T17:17:54.568 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_active_connections": 1, 2024-04-16T17:17:54.568 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_total_time": 0.048581726, 2024-04-16T17:17:54.568 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_send_time": 0.014903495, 2024-04-16T17:17:54.568 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_recv_time": 0.036458027, 2024-04-16T17:17:54.569 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_fast_dispatch_time": 0.005964786, 2024-04-16T17:17:54.569 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_messages_queue_lat": { 2024-04-16T17:17:54.569 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 241, 2024-04-16T17:17:54.569 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.013266026, 2024-04-16T17:17:54.569 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000055045 2024-04-16T17:17:54.569 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-16T17:17:54.569 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_handle_ack_lat": { 2024-04-16T17:17:54.569 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-04-16T17:17:54.570 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-04-16T17:17:54.570 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-04-16T17:17:54.570 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-16T17:17:54.570 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_encrypted_bytes": 1600304, 2024-04-16T17:17:54.570 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_encrypted_bytes": 148192 2024-04-16T17:17:54.570 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-16T17:17:54.570 INFO:teuthology.orchestra.run.smithi104.stdout: "AsyncMessenger::Worker-2": { 2024-04-16T17:17:54.570 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_messages": 2, 2024-04-16T17:17:54.570 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_messages": 183, 2024-04-16T17:17:54.571 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_bytes": 320, 2024-04-16T17:17:54.571 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_bytes": 188384, 2024-04-16T17:17:54.571 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_created_connections": 5, 2024-04-16T17:17:54.571 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_active_connections": 3, 2024-04-16T17:17:54.571 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_total_time": 0.011978826, 2024-04-16T17:17:54.571 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_send_time": 0.008304818, 2024-04-16T17:17:54.571 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_recv_time": 0.001190145, 2024-04-16T17:17:54.571 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-16T17:17:54.572 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_messages_queue_lat": { 2024-04-16T17:17:54.572 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 180, 2024-04-16T17:17:54.572 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.029172707, 2024-04-16T17:17:54.572 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000162070 2024-04-16T17:17:54.572 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-16T17:17:54.572 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_handle_ack_lat": { 2024-04-16T17:17:54.572 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-04-16T17:17:54.572 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-04-16T17:17:54.572 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-04-16T17:17:54.573 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-16T17:17:54.573 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-16T17:17:54.573 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_encrypted_bytes": 188384 2024-04-16T17:17:54.573 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-16T17:17:54.573 INFO:teuthology.orchestra.run.smithi104.stdout: "cct": { 2024-04-16T17:17:54.573 INFO:teuthology.orchestra.run.smithi104.stdout: "total_workers": 0, 2024-04-16T17:17:54.573 INFO:teuthology.orchestra.run.smithi104.stdout: "unhealthy_workers": 0 2024-04-16T17:17:54.573 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-16T17:17:54.574 INFO:teuthology.orchestra.run.smithi104.stdout: "mempool": { 2024-04-16T17:17:54.574 INFO:teuthology.orchestra.run.smithi104.stdout: "bloom_filter_bytes": 0, 2024-04-16T17:17:54.574 INFO:teuthology.orchestra.run.smithi104.stdout: "bloom_filter_items": 0, 2024-04-16T17:17:54.574 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_alloc_bytes": 0, 2024-04-16T17:17:54.574 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_alloc_items": 0, 2024-04-16T17:17:54.574 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_data_bytes": 0, 2024-04-16T17:17:54.574 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_data_items": 0, 2024-04-16T17:17:54.574 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-16T17:17:54.574 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_onode_items": 0, 2024-04-16T17:17:54.575 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-16T17:17:54.575 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_meta_items": 0, 2024-04-16T17:17:54.575 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_other_bytes": 0, 2024-04-16T17:17:54.575 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_other_items": 0, 2024-04-16T17:17:54.575 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-16T17:17:54.575 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_buffer_items": 0, 2024-04-16T17:17:54.575 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_extent_bytes": 0, 2024-04-16T17:17:54.575 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_extent_items": 0, 2024-04-16T17:17:54.576 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_blob_bytes": 0, 2024-04-16T17:17:54.576 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_blob_items": 0, 2024-04-16T17:17:54.576 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-16T17:17:54.576 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_shared_blob_items": 0, 2024-04-16T17:17:54.576 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-16T17:17:54.576 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_inline_bl_items": 0, 2024-04-16T17:17:54.576 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_fsck_bytes": 0, 2024-04-16T17:17:54.576 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_fsck_items": 0, 2024-04-16T17:17:54.576 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_txc_bytes": 0, 2024-04-16T17:17:54.577 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_txc_items": 0, 2024-04-16T17:17:54.577 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-16T17:17:54.577 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_writing_deferred_items": 0, 2024-04-16T17:17:54.577 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_writing_bytes": 0, 2024-04-16T17:17:54.577 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_writing_items": 0, 2024-04-16T17:17:54.577 INFO:teuthology.orchestra.run.smithi104.stdout: "bluefs_bytes": 0, 2024-04-16T17:17:54.577 INFO:teuthology.orchestra.run.smithi104.stdout: "bluefs_items": 0, 2024-04-16T17:17:54.577 INFO:teuthology.orchestra.run.smithi104.stdout: "bluefs_file_reader_bytes": 0, 2024-04-16T17:17:54.578 INFO:teuthology.orchestra.run.smithi104.stdout: "bluefs_file_reader_items": 0, 2024-04-16T17:17:54.578 INFO:teuthology.orchestra.run.smithi104.stdout: "bluefs_file_writer_bytes": 0, 2024-04-16T17:17:54.578 INFO:teuthology.orchestra.run.smithi104.stdout: "bluefs_file_writer_items": 0, 2024-04-16T17:17:54.578 INFO:teuthology.orchestra.run.smithi104.stdout: "buffer_anon_bytes": 148714, 2024-04-16T17:17:54.578 INFO:teuthology.orchestra.run.smithi104.stdout: "buffer_anon_items": 30, 2024-04-16T17:17:54.578 INFO:teuthology.orchestra.run.smithi104.stdout: "buffer_meta_bytes": 0, 2024-04-16T17:17:54.578 INFO:teuthology.orchestra.run.smithi104.stdout: "buffer_meta_items": 0, 2024-04-16T17:17:54.578 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_bytes": 0, 2024-04-16T17:17:54.578 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_items": 0, 2024-04-16T17:17:54.579 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_mapbl_bytes": 0, 2024-04-16T17:17:54.579 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_mapbl_items": 0, 2024-04-16T17:17:54.579 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_pglog_bytes": 0, 2024-04-16T17:17:54.579 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_pglog_items": 0, 2024-04-16T17:17:54.579 INFO:teuthology.orchestra.run.smithi104.stdout: "osdmap_bytes": 0, 2024-04-16T17:17:54.579 INFO:teuthology.orchestra.run.smithi104.stdout: "osdmap_items": 0, 2024-04-16T17:17:54.580 INFO:teuthology.orchestra.run.smithi104.stdout: "osdmap_mapping_bytes": 0, 2024-04-16T17:17:54.580 INFO:teuthology.orchestra.run.smithi104.stdout: "osdmap_mapping_items": 0, 2024-04-16T17:17:54.580 INFO:teuthology.orchestra.run.smithi104.stdout: "pgmap_bytes": 0, 2024-04-16T17:17:54.580 INFO:teuthology.orchestra.run.smithi104.stdout: "pgmap_items": 0, 2024-04-16T17:17:54.580 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_co_bytes": 928, 2024-04-16T17:17:54.580 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_co_items": 2, 2024-04-16T17:17:54.580 INFO:teuthology.orchestra.run.smithi104.stdout: "unittest_1_bytes": 0, 2024-04-16T17:17:54.580 INFO:teuthology.orchestra.run.smithi104.stdout: "unittest_1_items": 0, 2024-04-16T17:17:54.580 INFO:teuthology.orchestra.run.smithi104.stdout: "unittest_2_bytes": 0, 2024-04-16T17:17:54.581 INFO:teuthology.orchestra.run.smithi104.stdout: "unittest_2_items": 0 2024-04-16T17:17:54.581 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-16T17:17:54.581 INFO:teuthology.orchestra.run.smithi104.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-16T17:17:54.581 INFO:teuthology.orchestra.run.smithi104.stdout: "val": 0, 2024-04-16T17:17:54.581 INFO:teuthology.orchestra.run.smithi104.stdout: "max": 104857600, 2024-04-16T17:17:54.581 INFO:teuthology.orchestra.run.smithi104.stdout: "get_started": 0, 2024-04-16T17:17:54.581 INFO:teuthology.orchestra.run.smithi104.stdout: "get": 252, 2024-04-16T17:17:54.581 INFO:teuthology.orchestra.run.smithi104.stdout: "get_sum": 1562039, 2024-04-16T17:17:54.582 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_fail": 0, 2024-04-16T17:17:54.582 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_success": 252, 2024-04-16T17:17:54.582 INFO:teuthology.orchestra.run.smithi104.stdout: "take": 0, 2024-04-16T17:17:54.582 INFO:teuthology.orchestra.run.smithi104.stdout: "take_sum": 0, 2024-04-16T17:17:54.582 INFO:teuthology.orchestra.run.smithi104.stdout: "put": 252, 2024-04-16T17:17:54.582 INFO:teuthology.orchestra.run.smithi104.stdout: "put_sum": 1562039, 2024-04-16T17:17:54.582 INFO:teuthology.orchestra.run.smithi104.stdout: "wait": { 2024-04-16T17:17:54.582 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-04-16T17:17:54.582 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-04-16T17:17:54.583 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-04-16T17:17:54.583 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-04-16T17:17:54.583 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-04-16T17:17:54.583 INFO:teuthology.orchestra.run.smithi104.stdout:} 2024-04-16T17:17:54.926 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:17:54 smithi104 ceph-mon[36457]: pgmap v348: 129 pgs: 129 active+clean; 55 MiB data, 874 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T17:17:54.976 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:17:54 smithi045 ceph-mon[31676]: pgmap v348: 129 pgs: 129 active+clean; 55 MiB data, 874 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T17:17:55.054 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:17:54 smithi117 ceph-mon[36730]: pgmap v348: 129 pgs: 129 active+clean; 55 MiB data, 874 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T17:17:55.259 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.e 2024-04-16T17:17:55.260 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.e 2024-04-16T17:17:55.260 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.e 2024-04-16T17:17:55.260 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.e 2024-04-16T17:17:55.260 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.e 2024-04-16T17:17:55.260 DEBUG:tasks.check_counter:Getting stats from h 2024-04-16T17:17:55.260 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45 shell --fsid b1fecd38-fc11-11ee-bc90-c7b262605968 -- ceph daemon mds.h perf dump 2024-04-16T17:17:55.500 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/mon.b/config 2024-04-16T17:17:56.648 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:17:56 smithi104 ceph-mon[36457]: pgmap v349: 129 pgs: 129 active+clean; 55 MiB data, 874 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T17:17:56.976 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:17:56 smithi045 ceph-mon[31676]: pgmap v349: 129 pgs: 129 active+clean; 55 MiB data, 874 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T17:17:57.054 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:17:56 smithi117 ceph-mon[36730]: pgmap v349: 129 pgs: 129 active+clean; 55 MiB data, 874 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T17:17:57.228 INFO:teuthology.orchestra.run.smithi104.stdout:{ 2024-04-16T17:17:57.228 INFO:teuthology.orchestra.run.smithi104.stdout: "AsyncMessenger::Worker-0": { 2024-04-16T17:17:57.228 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_messages": 471, 2024-04-16T17:17:57.228 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_messages": 473, 2024-04-16T17:17:57.228 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_bytes": 142233, 2024-04-16T17:17:57.228 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_bytes": 149788, 2024-04-16T17:17:57.228 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_created_connections": 7, 2024-04-16T17:17:57.229 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_active_connections": 3, 2024-04-16T17:17:57.229 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_total_time": 0.047037033, 2024-04-16T17:17:57.229 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_send_time": 0.017463596, 2024-04-16T17:17:57.229 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_recv_time": 0.010264145, 2024-04-16T17:17:57.229 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_fast_dispatch_time": 0.014118012, 2024-04-16T17:17:57.229 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_messages_queue_lat": { 2024-04-16T17:17:57.229 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 471, 2024-04-16T17:17:57.229 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.018981317, 2024-04-16T17:17:57.229 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000040300 2024-04-16T17:17:57.230 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-16T17:17:57.230 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_handle_ack_lat": { 2024-04-16T17:17:57.230 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-04-16T17:17:57.230 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-04-16T17:17:57.230 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-04-16T17:17:57.230 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-16T17:17:57.230 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-16T17:17:57.230 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_encrypted_bytes": 0 2024-04-16T17:17:57.231 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-16T17:17:57.231 INFO:teuthology.orchestra.run.smithi104.stdout: "AsyncMessenger::Worker-1": { 2024-04-16T17:17:57.231 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_messages": 135, 2024-04-16T17:17:57.231 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_messages": 132, 2024-04-16T17:17:57.231 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_bytes": 4281749, 2024-04-16T17:17:57.231 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_bytes": 59835, 2024-04-16T17:17:57.231 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_created_connections": 4, 2024-04-16T17:17:57.231 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_active_connections": 3, 2024-04-16T17:17:57.231 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_total_time": 0.017350353, 2024-04-16T17:17:57.232 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_send_time": 0.005826037, 2024-04-16T17:17:57.232 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_recv_time": 0.008550468, 2024-04-16T17:17:57.232 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_fast_dispatch_time": 0.002740740, 2024-04-16T17:17:57.232 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_messages_queue_lat": { 2024-04-16T17:17:57.232 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 131, 2024-04-16T17:17:57.232 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.006990049, 2024-04-16T17:17:57.232 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000053359 2024-04-16T17:17:57.232 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-16T17:17:57.232 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_handle_ack_lat": { 2024-04-16T17:17:57.233 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-04-16T17:17:57.233 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-04-16T17:17:57.233 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-04-16T17:17:57.233 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-16T17:17:57.233 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_encrypted_bytes": 192160, 2024-04-16T17:17:57.233 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_encrypted_bytes": 44832 2024-04-16T17:17:57.233 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-16T17:17:57.233 INFO:teuthology.orchestra.run.smithi104.stdout: "AsyncMessenger::Worker-2": { 2024-04-16T17:17:57.233 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_messages": 295, 2024-04-16T17:17:57.234 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_messages": 345, 2024-04-16T17:17:57.234 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_bytes": 1467367, 2024-04-16T17:17:57.234 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_bytes": 270086, 2024-04-16T17:17:57.234 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_created_connections": 5, 2024-04-16T17:17:57.234 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_active_connections": 4, 2024-04-16T17:17:57.234 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_total_time": 0.027454343, 2024-04-16T17:17:57.234 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_send_time": 0.009331930, 2024-04-16T17:17:57.234 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_recv_time": 0.008689644, 2024-04-16T17:17:57.234 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_fast_dispatch_time": 0.006979524, 2024-04-16T17:17:57.235 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_messages_queue_lat": { 2024-04-16T17:17:57.235 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 344, 2024-04-16T17:17:57.235 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.017117195, 2024-04-16T17:17:57.235 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000049759 2024-04-16T17:17:57.235 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-16T17:17:57.235 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_handle_ack_lat": { 2024-04-16T17:17:57.235 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-04-16T17:17:57.235 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-04-16T17:17:57.235 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-04-16T17:17:57.236 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-16T17:17:57.236 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_encrypted_bytes": 3152, 2024-04-16T17:17:57.236 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_encrypted_bytes": 177856 2024-04-16T17:17:57.236 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-16T17:17:57.236 INFO:teuthology.orchestra.run.smithi104.stdout: "cct": { 2024-04-16T17:17:57.236 INFO:teuthology.orchestra.run.smithi104.stdout: "total_workers": 1, 2024-04-16T17:17:57.236 INFO:teuthology.orchestra.run.smithi104.stdout: "unhealthy_workers": 0 2024-04-16T17:17:57.236 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-16T17:17:57.237 INFO:teuthology.orchestra.run.smithi104.stdout: "finisher-MDSRank": { 2024-04-16T17:17:57.237 INFO:teuthology.orchestra.run.smithi104.stdout: "queue_len": 0, 2024-04-16T17:17:57.237 INFO:teuthology.orchestra.run.smithi104.stdout: "complete_latency": { 2024-04-16T17:17:57.237 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 1354, 2024-04-16T17:17:57.237 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.034041975, 2024-04-16T17:17:57.237 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000025141 2024-04-16T17:17:57.237 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-04-16T17:17:57.238 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-16T17:17:57.238 INFO:teuthology.orchestra.run.smithi104.stdout: "finisher-PurgeQueue": { 2024-04-16T17:17:57.238 INFO:teuthology.orchestra.run.smithi104.stdout: "queue_len": 0, 2024-04-16T17:17:57.238 INFO:teuthology.orchestra.run.smithi104.stdout: "complete_latency": { 2024-04-16T17:17:57.238 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-04-16T17:17:57.238 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-04-16T17:17:57.238 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-04-16T17:17:57.238 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-04-16T17:17:57.238 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-16T17:17:57.239 INFO:teuthology.orchestra.run.smithi104.stdout: "mds": { 2024-04-16T17:17:57.239 INFO:teuthology.orchestra.run.smithi104.stdout: "request": 0, 2024-04-16T17:17:57.239 INFO:teuthology.orchestra.run.smithi104.stdout: "reply": 0, 2024-04-16T17:17:57.239 INFO:teuthology.orchestra.run.smithi104.stdout: "reply_latency": { 2024-04-16T17:17:57.239 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-04-16T17:17:57.239 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-04-16T17:17:57.239 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-04-16T17:17:57.239 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-16T17:17:57.239 INFO:teuthology.orchestra.run.smithi104.stdout: "slow_reply": 0, 2024-04-16T17:17:57.240 INFO:teuthology.orchestra.run.smithi104.stdout: "forward": 0, 2024-04-16T17:17:57.240 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_fetch_complete": 0, 2024-04-16T17:17:57.240 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_fetch_keys": 0, 2024-04-16T17:17:57.240 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_commit": 0, 2024-04-16T17:17:57.240 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_split": 0, 2024-04-16T17:17:57.240 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_merge": 0, 2024-04-16T17:17:57.240 INFO:teuthology.orchestra.run.smithi104.stdout: "inodes": 235, 2024-04-16T17:17:57.240 INFO:teuthology.orchestra.run.smithi104.stdout: "inodes_top": 65, 2024-04-16T17:17:57.240 INFO:teuthology.orchestra.run.smithi104.stdout: "inodes_bottom": 170, 2024-04-16T17:17:57.241 INFO:teuthology.orchestra.run.smithi104.stdout: "inodes_pin_tail": 0, 2024-04-16T17:17:57.241 INFO:teuthology.orchestra.run.smithi104.stdout: "inodes_pinned": 144, 2024-04-16T17:17:57.241 INFO:teuthology.orchestra.run.smithi104.stdout: "inodes_expired": 0, 2024-04-16T17:17:57.241 INFO:teuthology.orchestra.run.smithi104.stdout: "inodes_with_caps": 0, 2024-04-16T17:17:57.241 INFO:teuthology.orchestra.run.smithi104.stdout: "caps": 0, 2024-04-16T17:17:57.241 INFO:teuthology.orchestra.run.smithi104.stdout: "subtrees": 11, 2024-04-16T17:17:57.241 INFO:teuthology.orchestra.run.smithi104.stdout: "traverse": 0, 2024-04-16T17:17:57.241 INFO:teuthology.orchestra.run.smithi104.stdout: "traverse_hit": 0, 2024-04-16T17:17:57.241 INFO:teuthology.orchestra.run.smithi104.stdout: "traverse_forward": 0, 2024-04-16T17:17:57.242 INFO:teuthology.orchestra.run.smithi104.stdout: "traverse_discover": 0, 2024-04-16T17:17:57.242 INFO:teuthology.orchestra.run.smithi104.stdout: "traverse_dir_fetch": 0, 2024-04-16T17:17:57.242 INFO:teuthology.orchestra.run.smithi104.stdout: "traverse_remote_ino": 0, 2024-04-16T17:17:57.242 INFO:teuthology.orchestra.run.smithi104.stdout: "traverse_lock": 0, 2024-04-16T17:17:57.242 INFO:teuthology.orchestra.run.smithi104.stdout: "load_cent": 0, 2024-04-16T17:17:57.242 INFO:teuthology.orchestra.run.smithi104.stdout: "q": 0, 2024-04-16T17:17:57.242 INFO:teuthology.orchestra.run.smithi104.stdout: "exported": 0, 2024-04-16T17:17:57.242 INFO:teuthology.orchestra.run.smithi104.stdout: "exported_inodes": 0, 2024-04-16T17:17:57.242 INFO:teuthology.orchestra.run.smithi104.stdout: "imported": 0, 2024-04-16T17:17:57.243 INFO:teuthology.orchestra.run.smithi104.stdout: "imported_inodes": 0, 2024-04-16T17:17:57.243 INFO:teuthology.orchestra.run.smithi104.stdout: "openino_dir_fetch": 0, 2024-04-16T17:17:57.243 INFO:teuthology.orchestra.run.smithi104.stdout: "openino_backtrace_fetch": 0, 2024-04-16T17:17:57.243 INFO:teuthology.orchestra.run.smithi104.stdout: "openino_peer_discover": 0, 2024-04-16T17:17:57.243 INFO:teuthology.orchestra.run.smithi104.stdout: "root_rfiles": 68, 2024-04-16T17:17:57.243 INFO:teuthology.orchestra.run.smithi104.stdout: "root_rbytes": 21196213, 2024-04-16T17:17:57.243 INFO:teuthology.orchestra.run.smithi104.stdout: "root_rsnaps": 6, 2024-04-16T17:17:57.243 INFO:teuthology.orchestra.run.smithi104.stdout: "scrub_backtrace_fetch": 0, 2024-04-16T17:17:57.243 INFO:teuthology.orchestra.run.smithi104.stdout: "scrub_set_tag": 0, 2024-04-16T17:17:57.244 INFO:teuthology.orchestra.run.smithi104.stdout: "scrub_backtrace_repaired": 0, 2024-04-16T17:17:57.244 INFO:teuthology.orchestra.run.smithi104.stdout: "scrub_inotable_repaired": 0, 2024-04-16T17:17:57.244 INFO:teuthology.orchestra.run.smithi104.stdout: "scrub_dir_inodes": 0, 2024-04-16T17:17:57.244 INFO:teuthology.orchestra.run.smithi104.stdout: "scrub_dir_base_inodes": 0, 2024-04-16T17:17:57.244 INFO:teuthology.orchestra.run.smithi104.stdout: "scrub_dirfrag_rstats": 0, 2024-04-16T17:17:57.244 INFO:teuthology.orchestra.run.smithi104.stdout: "scrub_file_inodes": 0, 2024-04-16T17:17:57.244 INFO:teuthology.orchestra.run.smithi104.stdout: "handle_inode_file_caps": 0, 2024-04-16T17:17:57.244 INFO:teuthology.orchestra.run.smithi104.stdout: "ceph_cap_op_revoke": 0, 2024-04-16T17:17:57.244 INFO:teuthology.orchestra.run.smithi104.stdout: "ceph_cap_op_grant": 0, 2024-04-16T17:17:57.245 INFO:teuthology.orchestra.run.smithi104.stdout: "ceph_cap_op_trunc": 0, 2024-04-16T17:17:57.245 INFO:teuthology.orchestra.run.smithi104.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-16T17:17:57.245 INFO:teuthology.orchestra.run.smithi104.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-16T17:17:57.245 INFO:teuthology.orchestra.run.smithi104.stdout: "handle_client_caps": 0, 2024-04-16T17:17:57.245 INFO:teuthology.orchestra.run.smithi104.stdout: "handle_client_caps_dirty": 0, 2024-04-16T17:17:57.245 INFO:teuthology.orchestra.run.smithi104.stdout: "handle_client_cap_release": 0, 2024-04-16T17:17:57.245 INFO:teuthology.orchestra.run.smithi104.stdout: "process_request_cap_release": 0 2024-04-16T17:17:57.245 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-16T17:17:57.245 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_cache": { 2024-04-16T17:17:57.246 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_update": 0, 2024-04-16T17:17:57.246 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_update_receipt": 0, 2024-04-16T17:17:57.246 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_try_discover": 0, 2024-04-16T17:17:57.246 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_send_discover": 0, 2024-04-16T17:17:57.246 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_handle_discover": 0, 2024-04-16T17:17:57.246 INFO:teuthology.orchestra.run.smithi104.stdout: "num_strays": 0, 2024-04-16T17:17:57.246 INFO:teuthology.orchestra.run.smithi104.stdout: "num_strays_delayed": 0, 2024-04-16T17:17:57.246 INFO:teuthology.orchestra.run.smithi104.stdout: "num_strays_enqueuing": 0, 2024-04-16T17:17:57.247 INFO:teuthology.orchestra.run.smithi104.stdout: "strays_created": 0, 2024-04-16T17:17:57.247 INFO:teuthology.orchestra.run.smithi104.stdout: "strays_enqueued": 0, 2024-04-16T17:17:57.247 INFO:teuthology.orchestra.run.smithi104.stdout: "strays_reintegrated": 0, 2024-04-16T17:17:57.247 INFO:teuthology.orchestra.run.smithi104.stdout: "strays_migrated": 0, 2024-04-16T17:17:57.247 INFO:teuthology.orchestra.run.smithi104.stdout: "num_recovering_processing": 0, 2024-04-16T17:17:57.247 INFO:teuthology.orchestra.run.smithi104.stdout: "num_recovering_enqueued": 0, 2024-04-16T17:17:57.247 INFO:teuthology.orchestra.run.smithi104.stdout: "num_recovering_prioritized": 0, 2024-04-16T17:17:57.247 INFO:teuthology.orchestra.run.smithi104.stdout: "recovery_started": 0, 2024-04-16T17:17:57.247 INFO:teuthology.orchestra.run.smithi104.stdout: "recovery_completed": 0, 2024-04-16T17:17:57.248 INFO:teuthology.orchestra.run.smithi104.stdout: "ireq_quiesce_path": 0, 2024-04-16T17:17:57.248 INFO:teuthology.orchestra.run.smithi104.stdout: "ireq_quiesce_inode": 0, 2024-04-16T17:17:57.248 INFO:teuthology.orchestra.run.smithi104.stdout: "ireq_enqueue_scrub": 0, 2024-04-16T17:17:57.248 INFO:teuthology.orchestra.run.smithi104.stdout: "ireq_exportdir": 0, 2024-04-16T17:17:57.248 INFO:teuthology.orchestra.run.smithi104.stdout: "ireq_flush": 0, 2024-04-16T17:17:57.248 INFO:teuthology.orchestra.run.smithi104.stdout: "ireq_fragmentdir": 0, 2024-04-16T17:17:57.248 INFO:teuthology.orchestra.run.smithi104.stdout: "ireq_fragstats": 0, 2024-04-16T17:17:57.248 INFO:teuthology.orchestra.run.smithi104.stdout: "ireq_inodestats": 0 2024-04-16T17:17:57.248 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-16T17:17:57.249 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_log": { 2024-04-16T17:17:57.249 INFO:teuthology.orchestra.run.smithi104.stdout: "evlrg": 0, 2024-04-16T17:17:57.249 INFO:teuthology.orchestra.run.smithi104.stdout: "evadd": 0, 2024-04-16T17:17:57.249 INFO:teuthology.orchestra.run.smithi104.stdout: "evex": 0, 2024-04-16T17:17:57.249 INFO:teuthology.orchestra.run.smithi104.stdout: "evtrm": 0, 2024-04-16T17:17:57.249 INFO:teuthology.orchestra.run.smithi104.stdout: "ev": 856, 2024-04-16T17:17:57.249 INFO:teuthology.orchestra.run.smithi104.stdout: "evexg": 0, 2024-04-16T17:17:57.249 INFO:teuthology.orchestra.run.smithi104.stdout: "evexd": 0, 2024-04-16T17:17:57.249 INFO:teuthology.orchestra.run.smithi104.stdout: "segadd": 0, 2024-04-16T17:17:57.250 INFO:teuthology.orchestra.run.smithi104.stdout: "segex": 0, 2024-04-16T17:17:57.250 INFO:teuthology.orchestra.run.smithi104.stdout: "segtrm": 0, 2024-04-16T17:17:57.250 INFO:teuthology.orchestra.run.smithi104.stdout: "seg": 2, 2024-04-16T17:17:57.250 INFO:teuthology.orchestra.run.smithi104.stdout: "segmjr": 2, 2024-04-16T17:17:57.250 INFO:teuthology.orchestra.run.smithi104.stdout: "segexg": 0, 2024-04-16T17:17:57.250 INFO:teuthology.orchestra.run.smithi104.stdout: "segexd": 0, 2024-04-16T17:17:57.250 INFO:teuthology.orchestra.run.smithi104.stdout: "expos": 66916694, 2024-04-16T17:17:57.250 INFO:teuthology.orchestra.run.smithi104.stdout: "wrpos": 68496824, 2024-04-16T17:17:57.251 INFO:teuthology.orchestra.run.smithi104.stdout: "rdpos": 68496688, 2024-04-16T17:17:57.251 INFO:teuthology.orchestra.run.smithi104.stdout: "jlat": { 2024-04-16T17:17:57.251 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-04-16T17:17:57.251 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-04-16T17:17:57.251 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-04-16T17:17:57.251 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-16T17:17:57.251 INFO:teuthology.orchestra.run.smithi104.stdout: "replayed": 856 2024-04-16T17:17:57.251 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-16T17:17:57.251 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_mem": { 2024-04-16T17:17:57.252 INFO:teuthology.orchestra.run.smithi104.stdout: "ino": 47, 2024-04-16T17:17:57.252 INFO:teuthology.orchestra.run.smithi104.stdout: "ino+": 194, 2024-04-16T17:17:57.252 INFO:teuthology.orchestra.run.smithi104.stdout: "ino-": 147, 2024-04-16T17:17:57.252 INFO:teuthology.orchestra.run.smithi104.stdout: "dir": 32, 2024-04-16T17:17:57.252 INFO:teuthology.orchestra.run.smithi104.stdout: "dir+": 32, 2024-04-16T17:17:57.252 INFO:teuthology.orchestra.run.smithi104.stdout: "dir-": 0, 2024-04-16T17:17:57.252 INFO:teuthology.orchestra.run.smithi104.stdout: "dn": 235, 2024-04-16T17:17:57.252 INFO:teuthology.orchestra.run.smithi104.stdout: "dn+": 235, 2024-04-16T17:17:57.252 INFO:teuthology.orchestra.run.smithi104.stdout: "dn-": 0, 2024-04-16T17:17:57.253 INFO:teuthology.orchestra.run.smithi104.stdout: "cap": 0, 2024-04-16T17:17:57.253 INFO:teuthology.orchestra.run.smithi104.stdout: "cap+": 0, 2024-04-16T17:17:57.253 INFO:teuthology.orchestra.run.smithi104.stdout: "cap-": 0, 2024-04-16T17:17:57.253 INFO:teuthology.orchestra.run.smithi104.stdout: "rss": 46552, 2024-04-16T17:17:57.253 INFO:teuthology.orchestra.run.smithi104.stdout: "heap": 215392 2024-04-16T17:17:57.253 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-16T17:17:57.253 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_server": { 2024-04-16T17:17:57.253 INFO:teuthology.orchestra.run.smithi104.stdout: "dispatch_client_request": 0, 2024-04-16T17:17:57.253 INFO:teuthology.orchestra.run.smithi104.stdout: "dispatch_server_request": 0, 2024-04-16T17:17:57.254 INFO:teuthology.orchestra.run.smithi104.stdout: "handle_client_request": 0, 2024-04-16T17:17:57.254 INFO:teuthology.orchestra.run.smithi104.stdout: "handle_client_session": 0, 2024-04-16T17:17:57.254 INFO:teuthology.orchestra.run.smithi104.stdout: "handle_peer_request": 0, 2024-04-16T17:17:57.254 INFO:teuthology.orchestra.run.smithi104.stdout: "req_create_latency": { 2024-04-16T17:17:57.254 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-04-16T17:17:57.254 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-04-16T17:17:57.254 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-04-16T17:17:57.254 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-16T17:17:57.255 INFO:teuthology.orchestra.run.smithi104.stdout: "req_getattr_latency": { 2024-04-16T17:17:57.255 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-04-16T17:17:57.255 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-04-16T17:17:57.255 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-04-16T17:17:57.255 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-16T17:17:57.255 INFO:teuthology.orchestra.run.smithi104.stdout: "req_getfilelock_latency": { 2024-04-16T17:17:57.255 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-04-16T17:17:57.255 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-04-16T17:17:57.255 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-04-16T17:17:57.256 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-16T17:17:57.256 INFO:teuthology.orchestra.run.smithi104.stdout: "req_link_latency": { 2024-04-16T17:17:57.256 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-04-16T17:17:57.256 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-04-16T17:17:57.256 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-04-16T17:17:57.256 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-16T17:17:57.256 INFO:teuthology.orchestra.run.smithi104.stdout: "req_lookup_latency": { 2024-04-16T17:17:57.256 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-04-16T17:17:57.257 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-04-16T17:17:57.257 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-04-16T17:17:57.257 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-16T17:17:57.257 INFO:teuthology.orchestra.run.smithi104.stdout: "req_lookuphash_latency": { 2024-04-16T17:17:57.257 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-04-16T17:17:57.257 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-04-16T17:17:57.257 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-04-16T17:17:57.257 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-16T17:17:57.257 INFO:teuthology.orchestra.run.smithi104.stdout: "req_lookupino_latency": { 2024-04-16T17:17:57.258 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-04-16T17:17:57.258 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-04-16T17:17:57.258 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-04-16T17:17:57.258 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-16T17:17:57.258 INFO:teuthology.orchestra.run.smithi104.stdout: "req_lookupname_latency": { 2024-04-16T17:17:57.258 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-04-16T17:17:57.258 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-04-16T17:17:57.258 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-04-16T17:17:57.258 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-16T17:17:57.259 INFO:teuthology.orchestra.run.smithi104.stdout: "req_lookupparent_latency": { 2024-04-16T17:17:57.259 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-04-16T17:17:57.259 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-04-16T17:17:57.259 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-04-16T17:17:57.259 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-16T17:17:57.259 INFO:teuthology.orchestra.run.smithi104.stdout: "req_lookupsnap_latency": { 2024-04-16T17:17:57.259 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-04-16T17:17:57.259 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-04-16T17:17:57.259 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-04-16T17:17:57.260 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-16T17:17:57.260 INFO:teuthology.orchestra.run.smithi104.stdout: "req_lssnap_latency": { 2024-04-16T17:17:57.260 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-04-16T17:17:57.260 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-04-16T17:17:57.260 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-04-16T17:17:57.260 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-16T17:17:57.260 INFO:teuthology.orchestra.run.smithi104.stdout: "req_mkdir_latency": { 2024-04-16T17:17:57.261 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-04-16T17:17:57.261 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-04-16T17:17:57.261 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-04-16T17:17:57.261 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-16T17:17:57.261 INFO:teuthology.orchestra.run.smithi104.stdout: "req_mknod_latency": { 2024-04-16T17:17:57.261 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-04-16T17:17:57.262 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-04-16T17:17:57.262 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-04-16T17:17:57.262 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-16T17:17:57.262 INFO:teuthology.orchestra.run.smithi104.stdout: "req_mksnap_latency": { 2024-04-16T17:17:57.262 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-04-16T17:17:57.262 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-04-16T17:17:57.262 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-04-16T17:17:57.262 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-16T17:17:57.263 INFO:teuthology.orchestra.run.smithi104.stdout: "req_open_latency": { 2024-04-16T17:17:57.263 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-04-16T17:17:57.263 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-04-16T17:17:57.263 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-04-16T17:17:57.263 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-16T17:17:57.263 INFO:teuthology.orchestra.run.smithi104.stdout: "req_readdir_latency": { 2024-04-16T17:17:57.263 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-04-16T17:17:57.263 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-04-16T17:17:57.263 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-04-16T17:17:57.263 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-16T17:17:57.264 INFO:teuthology.orchestra.run.smithi104.stdout: "req_rename_latency": { 2024-04-16T17:17:57.264 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-04-16T17:17:57.264 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-04-16T17:17:57.264 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-04-16T17:17:57.264 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-16T17:17:57.264 INFO:teuthology.orchestra.run.smithi104.stdout: "req_renamesnap_latency": { 2024-04-16T17:17:57.264 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-04-16T17:17:57.264 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-04-16T17:17:57.264 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-04-16T17:17:57.265 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-16T17:17:57.265 INFO:teuthology.orchestra.run.smithi104.stdout: "req_snapdiff_latency": { 2024-04-16T17:17:57.265 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-04-16T17:17:57.265 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-04-16T17:17:57.265 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-04-16T17:17:57.265 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-16T17:17:57.265 INFO:teuthology.orchestra.run.smithi104.stdout: "req_rmdir_latency": { 2024-04-16T17:17:57.265 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-04-16T17:17:57.265 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-04-16T17:17:57.266 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-04-16T17:17:57.266 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-16T17:17:57.266 INFO:teuthology.orchestra.run.smithi104.stdout: "req_rmsnap_latency": { 2024-04-16T17:17:57.266 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-04-16T17:17:57.266 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-04-16T17:17:57.266 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-04-16T17:17:57.266 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-16T17:17:57.266 INFO:teuthology.orchestra.run.smithi104.stdout: "req_rmxattr_latency": { 2024-04-16T17:17:57.266 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-04-16T17:17:57.267 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-04-16T17:17:57.267 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-04-16T17:17:57.267 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-16T17:17:57.267 INFO:teuthology.orchestra.run.smithi104.stdout: "req_setattr_latency": { 2024-04-16T17:17:57.267 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-04-16T17:17:57.267 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-04-16T17:17:57.267 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-04-16T17:17:57.267 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-16T17:17:57.267 INFO:teuthology.orchestra.run.smithi104.stdout: "req_setdirlayout_latency": { 2024-04-16T17:17:57.268 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-04-16T17:17:57.268 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-04-16T17:17:57.268 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-04-16T17:17:57.268 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-16T17:17:57.268 INFO:teuthology.orchestra.run.smithi104.stdout: "req_setfilelock_latency": { 2024-04-16T17:17:57.268 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-04-16T17:17:57.268 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-04-16T17:17:57.268 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-04-16T17:17:57.268 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-16T17:17:57.268 INFO:teuthology.orchestra.run.smithi104.stdout: "req_setlayout_latency": { 2024-04-16T17:17:57.269 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-04-16T17:17:57.269 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-04-16T17:17:57.269 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-04-16T17:17:57.269 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-16T17:17:57.269 INFO:teuthology.orchestra.run.smithi104.stdout: "req_setxattr_latency": { 2024-04-16T17:17:57.269 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-04-16T17:17:57.269 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-04-16T17:17:57.269 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-04-16T17:17:57.269 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-16T17:17:57.270 INFO:teuthology.orchestra.run.smithi104.stdout: "req_symlink_latency": { 2024-04-16T17:17:57.270 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-04-16T17:17:57.270 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-04-16T17:17:57.270 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-04-16T17:17:57.270 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-16T17:17:57.270 INFO:teuthology.orchestra.run.smithi104.stdout: "req_unlink_latency": { 2024-04-16T17:17:57.270 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-04-16T17:17:57.270 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-04-16T17:17:57.270 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-04-16T17:17:57.271 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-16T17:17:57.271 INFO:teuthology.orchestra.run.smithi104.stdout: "cap_revoke_eviction": 0, 2024-04-16T17:17:57.271 INFO:teuthology.orchestra.run.smithi104.stdout: "cap_acquisition_throttle": 0, 2024-04-16T17:17:57.271 INFO:teuthology.orchestra.run.smithi104.stdout: "req_getvxattr_latency": { 2024-04-16T17:17:57.271 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-04-16T17:17:57.271 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-04-16T17:17:57.271 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-04-16T17:17:57.271 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-04-16T17:17:57.272 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-16T17:17:57.272 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_sessions": { 2024-04-16T17:17:57.272 INFO:teuthology.orchestra.run.smithi104.stdout: "session_count": 2, 2024-04-16T17:17:57.272 INFO:teuthology.orchestra.run.smithi104.stdout: "session_add": 2, 2024-04-16T17:17:57.272 INFO:teuthology.orchestra.run.smithi104.stdout: "session_remove": 0, 2024-04-16T17:17:57.272 INFO:teuthology.orchestra.run.smithi104.stdout: "sessions_open": 0, 2024-04-16T17:17:57.272 INFO:teuthology.orchestra.run.smithi104.stdout: "sessions_stale": 0, 2024-04-16T17:17:57.272 INFO:teuthology.orchestra.run.smithi104.stdout: "total_load": 0, 2024-04-16T17:17:57.272 INFO:teuthology.orchestra.run.smithi104.stdout: "average_load": 0, 2024-04-16T17:17:57.273 INFO:teuthology.orchestra.run.smithi104.stdout: "avg_session_uptime": 2398, 2024-04-16T17:17:57.273 INFO:teuthology.orchestra.run.smithi104.stdout: "mdthresh_evicted": 0 2024-04-16T17:17:57.273 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-16T17:17:57.273 INFO:teuthology.orchestra.run.smithi104.stdout: "mempool": { 2024-04-16T17:17:57.273 INFO:teuthology.orchestra.run.smithi104.stdout: "bloom_filter_bytes": 0, 2024-04-16T17:17:57.273 INFO:teuthology.orchestra.run.smithi104.stdout: "bloom_filter_items": 0, 2024-04-16T17:17:57.273 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_alloc_bytes": 0, 2024-04-16T17:17:57.273 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_alloc_items": 0, 2024-04-16T17:17:57.273 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_data_bytes": 0, 2024-04-16T17:17:57.273 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_data_items": 0, 2024-04-16T17:17:57.274 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-16T17:17:57.274 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_onode_items": 0, 2024-04-16T17:17:57.274 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-16T17:17:57.274 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_meta_items": 0, 2024-04-16T17:17:57.274 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_other_bytes": 0, 2024-04-16T17:17:57.274 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_other_items": 0, 2024-04-16T17:17:57.274 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-16T17:17:57.275 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_buffer_items": 0, 2024-04-16T17:17:57.275 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_extent_bytes": 0, 2024-04-16T17:17:57.275 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_extent_items": 0, 2024-04-16T17:17:57.275 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_blob_bytes": 0, 2024-04-16T17:17:57.275 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_blob_items": 0, 2024-04-16T17:17:57.275 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-16T17:17:57.275 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_shared_blob_items": 0, 2024-04-16T17:17:57.275 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-16T17:17:57.275 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_inline_bl_items": 0, 2024-04-16T17:17:57.276 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_fsck_bytes": 0, 2024-04-16T17:17:57.276 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_fsck_items": 0, 2024-04-16T17:17:57.276 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_txc_bytes": 0, 2024-04-16T17:17:57.276 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_txc_items": 0, 2024-04-16T17:17:57.276 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-16T17:17:57.276 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_writing_deferred_items": 0, 2024-04-16T17:17:57.276 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_writing_bytes": 0, 2024-04-16T17:17:57.276 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_writing_items": 0, 2024-04-16T17:17:57.276 INFO:teuthology.orchestra.run.smithi104.stdout: "bluefs_bytes": 0, 2024-04-16T17:17:57.277 INFO:teuthology.orchestra.run.smithi104.stdout: "bluefs_items": 0, 2024-04-16T17:17:57.277 INFO:teuthology.orchestra.run.smithi104.stdout: "bluefs_file_reader_bytes": 0, 2024-04-16T17:17:57.277 INFO:teuthology.orchestra.run.smithi104.stdout: "bluefs_file_reader_items": 0, 2024-04-16T17:17:57.277 INFO:teuthology.orchestra.run.smithi104.stdout: "bluefs_file_writer_bytes": 0, 2024-04-16T17:17:57.277 INFO:teuthology.orchestra.run.smithi104.stdout: "bluefs_file_writer_items": 0, 2024-04-16T17:17:57.277 INFO:teuthology.orchestra.run.smithi104.stdout: "buffer_anon_bytes": 210078, 2024-04-16T17:17:57.277 INFO:teuthology.orchestra.run.smithi104.stdout: "buffer_anon_items": 183, 2024-04-16T17:17:57.277 INFO:teuthology.orchestra.run.smithi104.stdout: "buffer_meta_bytes": 0, 2024-04-16T17:17:57.277 INFO:teuthology.orchestra.run.smithi104.stdout: "buffer_meta_items": 0, 2024-04-16T17:17:57.277 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_bytes": 0, 2024-04-16T17:17:57.278 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_items": 0, 2024-04-16T17:17:57.278 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_mapbl_bytes": 0, 2024-04-16T17:17:57.278 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_mapbl_items": 0, 2024-04-16T17:17:57.278 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_pglog_bytes": 0, 2024-04-16T17:17:57.278 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_pglog_items": 0, 2024-04-16T17:17:57.278 INFO:teuthology.orchestra.run.smithi104.stdout: "osdmap_bytes": 9512, 2024-04-16T17:17:57.278 INFO:teuthology.orchestra.run.smithi104.stdout: "osdmap_items": 182, 2024-04-16T17:17:57.278 INFO:teuthology.orchestra.run.smithi104.stdout: "osdmap_mapping_bytes": 0, 2024-04-16T17:17:57.279 INFO:teuthology.orchestra.run.smithi104.stdout: "osdmap_mapping_items": 0, 2024-04-16T17:17:57.279 INFO:teuthology.orchestra.run.smithi104.stdout: "pgmap_bytes": 0, 2024-04-16T17:17:57.279 INFO:teuthology.orchestra.run.smithi104.stdout: "pgmap_items": 0, 2024-04-16T17:17:57.279 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_co_bytes": 423435, 2024-04-16T17:17:57.279 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_co_items": 3766, 2024-04-16T17:17:57.279 INFO:teuthology.orchestra.run.smithi104.stdout: "unittest_1_bytes": 0, 2024-04-16T17:17:57.279 INFO:teuthology.orchestra.run.smithi104.stdout: "unittest_1_items": 0, 2024-04-16T17:17:57.279 INFO:teuthology.orchestra.run.smithi104.stdout: "unittest_2_bytes": 0, 2024-04-16T17:17:57.279 INFO:teuthology.orchestra.run.smithi104.stdout: "unittest_2_items": 0 2024-04-16T17:17:57.280 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-16T17:17:57.280 INFO:teuthology.orchestra.run.smithi104.stdout: "objecter": { 2024-04-16T17:17:57.280 INFO:teuthology.orchestra.run.smithi104.stdout: "op_active": 0, 2024-04-16T17:17:57.280 INFO:teuthology.orchestra.run.smithi104.stdout: "op_laggy": 0, 2024-04-16T17:17:57.280 INFO:teuthology.orchestra.run.smithi104.stdout: "op_send": 808, 2024-04-16T17:17:57.280 INFO:teuthology.orchestra.run.smithi104.stdout: "op_send_bytes": 26, 2024-04-16T17:17:57.280 INFO:teuthology.orchestra.run.smithi104.stdout: "op_resend": 4, 2024-04-16T17:17:57.280 INFO:teuthology.orchestra.run.smithi104.stdout: "op_reply": 808, 2024-04-16T17:17:57.280 INFO:teuthology.orchestra.run.smithi104.stdout: "op_latency": { 2024-04-16T17:17:57.281 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 808, 2024-04-16T17:17:57.281 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.402994715, 2024-04-16T17:17:57.281 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000498755 2024-04-16T17:17:57.281 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-16T17:17:57.281 INFO:teuthology.orchestra.run.smithi104.stdout: "op_inflight": 0, 2024-04-16T17:17:57.281 INFO:teuthology.orchestra.run.smithi104.stdout: "oplen_avg": { 2024-04-16T17:17:57.281 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 808, 2024-04-16T17:17:57.281 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 809 2024-04-16T17:17:57.281 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-16T17:17:57.282 INFO:teuthology.orchestra.run.smithi104.stdout: "op": 808, 2024-04-16T17:17:57.282 INFO:teuthology.orchestra.run.smithi104.stdout: "op_r": 808, 2024-04-16T17:17:57.282 INFO:teuthology.orchestra.run.smithi104.stdout: "op_w": 0, 2024-04-16T17:17:57.282 INFO:teuthology.orchestra.run.smithi104.stdout: "op_rmw": 0, 2024-04-16T17:17:57.282 INFO:teuthology.orchestra.run.smithi104.stdout: "op_pg": 0, 2024-04-16T17:17:57.282 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_stat": 498, 2024-04-16T17:17:57.282 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_create": 0, 2024-04-16T17:17:57.282 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_read": 307, 2024-04-16T17:17:57.282 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_write": 0, 2024-04-16T17:17:57.282 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_writefull": 0, 2024-04-16T17:17:57.283 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_writesame": 0, 2024-04-16T17:17:57.283 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_append": 0, 2024-04-16T17:17:57.283 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_zero": 0, 2024-04-16T17:17:57.283 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_truncate": 0, 2024-04-16T17:17:57.283 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_delete": 0, 2024-04-16T17:17:57.283 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_mapext": 0, 2024-04-16T17:17:57.283 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_sparse_read": 0, 2024-04-16T17:17:57.283 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_clonerange": 0, 2024-04-16T17:17:57.283 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_getxattr": 2, 2024-04-16T17:17:57.284 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_setxattr": 0, 2024-04-16T17:17:57.284 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_cmpxattr": 0, 2024-04-16T17:17:57.284 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_rmxattr": 0, 2024-04-16T17:17:57.284 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_resetxattrs": 0, 2024-04-16T17:17:57.284 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_call": 0, 2024-04-16T17:17:57.284 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_watch": 0, 2024-04-16T17:17:57.284 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_notify": 0, 2024-04-16T17:17:57.284 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_src_cmpxattr": 0, 2024-04-16T17:17:57.284 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_pgls": 0, 2024-04-16T17:17:57.285 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_pgls_filter": 0, 2024-04-16T17:17:57.285 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_other": 0, 2024-04-16T17:17:57.285 INFO:teuthology.orchestra.run.smithi104.stdout: "linger_active": 0, 2024-04-16T17:17:57.285 INFO:teuthology.orchestra.run.smithi104.stdout: "linger_send": 0, 2024-04-16T17:17:57.285 INFO:teuthology.orchestra.run.smithi104.stdout: "linger_resend": 0, 2024-04-16T17:17:57.285 INFO:teuthology.orchestra.run.smithi104.stdout: "linger_ping": 0, 2024-04-16T17:17:57.285 INFO:teuthology.orchestra.run.smithi104.stdout: "poolop_active": 0, 2024-04-16T17:17:57.285 INFO:teuthology.orchestra.run.smithi104.stdout: "poolop_send": 0, 2024-04-16T17:17:57.285 INFO:teuthology.orchestra.run.smithi104.stdout: "poolop_resend": 0, 2024-04-16T17:17:57.285 INFO:teuthology.orchestra.run.smithi104.stdout: "poolstat_active": 0, 2024-04-16T17:17:57.286 INFO:teuthology.orchestra.run.smithi104.stdout: "poolstat_send": 0, 2024-04-16T17:17:57.286 INFO:teuthology.orchestra.run.smithi104.stdout: "poolstat_resend": 0, 2024-04-16T17:17:57.286 INFO:teuthology.orchestra.run.smithi104.stdout: "statfs_active": 0, 2024-04-16T17:17:57.286 INFO:teuthology.orchestra.run.smithi104.stdout: "statfs_send": 0, 2024-04-16T17:17:57.286 INFO:teuthology.orchestra.run.smithi104.stdout: "statfs_resend": 0, 2024-04-16T17:17:57.286 INFO:teuthology.orchestra.run.smithi104.stdout: "command_active": 0, 2024-04-16T17:17:57.286 INFO:teuthology.orchestra.run.smithi104.stdout: "command_send": 0, 2024-04-16T17:17:57.286 INFO:teuthology.orchestra.run.smithi104.stdout: "command_resend": 0, 2024-04-16T17:17:57.286 INFO:teuthology.orchestra.run.smithi104.stdout: "map_epoch": 106, 2024-04-16T17:17:57.287 INFO:teuthology.orchestra.run.smithi104.stdout: "map_full": 0, 2024-04-16T17:17:57.287 INFO:teuthology.orchestra.run.smithi104.stdout: "map_inc": 8, 2024-04-16T17:17:57.287 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_sessions": 7, 2024-04-16T17:17:57.287 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_session_open": 7, 2024-04-16T17:17:57.287 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_session_close": 0, 2024-04-16T17:17:57.287 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_laggy": 0, 2024-04-16T17:17:57.287 INFO:teuthology.orchestra.run.smithi104.stdout: "omap_wr": 0, 2024-04-16T17:17:57.287 INFO:teuthology.orchestra.run.smithi104.stdout: "omap_rd": 2, 2024-04-16T17:17:57.287 INFO:teuthology.orchestra.run.smithi104.stdout: "omap_del": 0 2024-04-16T17:17:57.288 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-16T17:17:57.288 INFO:teuthology.orchestra.run.smithi104.stdout: "oft": { 2024-04-16T17:17:57.288 INFO:teuthology.orchestra.run.smithi104.stdout: "omap_total_objs": 0, 2024-04-16T17:17:57.288 INFO:teuthology.orchestra.run.smithi104.stdout: "omap_total_kv_pairs": 0, 2024-04-16T17:17:57.288 INFO:teuthology.orchestra.run.smithi104.stdout: "omap_total_updates": 0, 2024-04-16T17:17:57.288 INFO:teuthology.orchestra.run.smithi104.stdout: "omap_total_removes": 0 2024-04-16T17:17:57.288 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-16T17:17:57.288 INFO:teuthology.orchestra.run.smithi104.stdout: "osd-slow-ops": { 2024-04-16T17:17:57.288 INFO:teuthology.orchestra.run.smithi104.stdout: "slow_ops_count": 0 2024-04-16T17:17:57.288 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-16T17:17:57.289 INFO:teuthology.orchestra.run.smithi104.stdout: "purge_queue": { 2024-04-16T17:17:57.289 INFO:teuthology.orchestra.run.smithi104.stdout: "pq_executing_ops": 0, 2024-04-16T17:17:57.289 INFO:teuthology.orchestra.run.smithi104.stdout: "pq_executing_ops_high_water": 0, 2024-04-16T17:17:57.289 INFO:teuthology.orchestra.run.smithi104.stdout: "pq_executing": 0, 2024-04-16T17:17:57.289 INFO:teuthology.orchestra.run.smithi104.stdout: "pq_executing_high_water": 0, 2024-04-16T17:17:57.289 INFO:teuthology.orchestra.run.smithi104.stdout: "pq_executed_ops": 0, 2024-04-16T17:17:57.289 INFO:teuthology.orchestra.run.smithi104.stdout: "pq_executed": 0, 2024-04-16T17:17:57.289 INFO:teuthology.orchestra.run.smithi104.stdout: "pq_item_in_journal": 0 2024-04-16T17:17:57.289 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-16T17:17:57.290 INFO:teuthology.orchestra.run.smithi104.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-16T17:17:57.290 INFO:teuthology.orchestra.run.smithi104.stdout: "val": 0, 2024-04-16T17:17:57.290 INFO:teuthology.orchestra.run.smithi104.stdout: "max": 104857600, 2024-04-16T17:17:57.290 INFO:teuthology.orchestra.run.smithi104.stdout: "get_started": 0, 2024-04-16T17:17:57.290 INFO:teuthology.orchestra.run.smithi104.stdout: "get": 897, 2024-04-16T17:17:57.290 INFO:teuthology.orchestra.run.smithi104.stdout: "get_sum": 5803147, 2024-04-16T17:17:57.290 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_fail": 0, 2024-04-16T17:17:57.290 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_success": 897, 2024-04-16T17:17:57.290 INFO:teuthology.orchestra.run.smithi104.stdout: "take": 0, 2024-04-16T17:17:57.291 INFO:teuthology.orchestra.run.smithi104.stdout: "take_sum": 0, 2024-04-16T17:17:57.291 INFO:teuthology.orchestra.run.smithi104.stdout: "put": 897, 2024-04-16T17:17:57.291 INFO:teuthology.orchestra.run.smithi104.stdout: "put_sum": 5803147, 2024-04-16T17:17:57.291 INFO:teuthology.orchestra.run.smithi104.stdout: "wait": { 2024-04-16T17:17:57.291 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-04-16T17:17:57.291 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-04-16T17:17:57.291 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-04-16T17:17:57.291 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-04-16T17:17:57.292 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-16T17:17:57.292 INFO:teuthology.orchestra.run.smithi104.stdout: "throttle-objecter_bytes": { 2024-04-16T17:17:57.292 INFO:teuthology.orchestra.run.smithi104.stdout: "val": 0, 2024-04-16T17:17:57.292 INFO:teuthology.orchestra.run.smithi104.stdout: "max": 104857600, 2024-04-16T17:17:57.292 INFO:teuthology.orchestra.run.smithi104.stdout: "get_started": 0, 2024-04-16T17:17:57.292 INFO:teuthology.orchestra.run.smithi104.stdout: "get": 0, 2024-04-16T17:17:57.292 INFO:teuthology.orchestra.run.smithi104.stdout: "get_sum": 0, 2024-04-16T17:17:57.292 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_fail": 0, 2024-04-16T17:17:57.292 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_success": 0, 2024-04-16T17:17:57.293 INFO:teuthology.orchestra.run.smithi104.stdout: "take": 808, 2024-04-16T17:17:57.293 INFO:teuthology.orchestra.run.smithi104.stdout: "take_sum": 5463614, 2024-04-16T17:17:57.293 INFO:teuthology.orchestra.run.smithi104.stdout: "put": 808, 2024-04-16T17:17:57.293 INFO:teuthology.orchestra.run.smithi104.stdout: "put_sum": 5463614, 2024-04-16T17:17:57.293 INFO:teuthology.orchestra.run.smithi104.stdout: "wait": { 2024-04-16T17:17:57.293 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-04-16T17:17:57.293 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-04-16T17:17:57.293 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-04-16T17:17:57.293 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-04-16T17:17:57.294 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-16T17:17:57.294 INFO:teuthology.orchestra.run.smithi104.stdout: "throttle-objecter_ops": { 2024-04-16T17:17:57.294 INFO:teuthology.orchestra.run.smithi104.stdout: "val": 0, 2024-04-16T17:17:57.294 INFO:teuthology.orchestra.run.smithi104.stdout: "max": 1024, 2024-04-16T17:17:57.294 INFO:teuthology.orchestra.run.smithi104.stdout: "get_started": 0, 2024-04-16T17:17:57.294 INFO:teuthology.orchestra.run.smithi104.stdout: "get": 0, 2024-04-16T17:17:57.294 INFO:teuthology.orchestra.run.smithi104.stdout: "get_sum": 0, 2024-04-16T17:17:57.294 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_fail": 0, 2024-04-16T17:17:57.294 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_success": 0, 2024-04-16T17:17:57.294 INFO:teuthology.orchestra.run.smithi104.stdout: "take": 808, 2024-04-16T17:17:57.295 INFO:teuthology.orchestra.run.smithi104.stdout: "take_sum": 808, 2024-04-16T17:17:57.295 INFO:teuthology.orchestra.run.smithi104.stdout: "put": 808, 2024-04-16T17:17:57.295 INFO:teuthology.orchestra.run.smithi104.stdout: "put_sum": 808, 2024-04-16T17:17:57.295 INFO:teuthology.orchestra.run.smithi104.stdout: "wait": { 2024-04-16T17:17:57.295 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-04-16T17:17:57.295 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-04-16T17:17:57.295 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-04-16T17:17:57.295 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-04-16T17:17:57.295 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-16T17:17:57.296 INFO:teuthology.orchestra.run.smithi104.stdout: "throttle-write_buf_throttle": { 2024-04-16T17:17:57.296 INFO:teuthology.orchestra.run.smithi104.stdout: "val": 0, 2024-04-16T17:17:57.296 INFO:teuthology.orchestra.run.smithi104.stdout: "max": 3758096384, 2024-04-16T17:17:57.296 INFO:teuthology.orchestra.run.smithi104.stdout: "get_started": 0, 2024-04-16T17:17:57.296 INFO:teuthology.orchestra.run.smithi104.stdout: "get": 0, 2024-04-16T17:17:57.296 INFO:teuthology.orchestra.run.smithi104.stdout: "get_sum": 0, 2024-04-16T17:17:57.296 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_fail": 0, 2024-04-16T17:17:57.296 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_success": 0, 2024-04-16T17:17:57.296 INFO:teuthology.orchestra.run.smithi104.stdout: "take": 0, 2024-04-16T17:17:57.297 INFO:teuthology.orchestra.run.smithi104.stdout: "take_sum": 0, 2024-04-16T17:17:57.297 INFO:teuthology.orchestra.run.smithi104.stdout: "put": 0, 2024-04-16T17:17:57.297 INFO:teuthology.orchestra.run.smithi104.stdout: "put_sum": 0, 2024-04-16T17:17:57.297 INFO:teuthology.orchestra.run.smithi104.stdout: "wait": { 2024-04-16T17:17:57.297 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-04-16T17:17:57.297 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-04-16T17:17:57.297 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-04-16T17:17:57.297 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-04-16T17:17:57.297 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-16T17:17:57.298 INFO:teuthology.orchestra.run.smithi104.stdout: "throttle-write_buf_throttle-0x5568785586e0": { 2024-04-16T17:17:57.298 INFO:teuthology.orchestra.run.smithi104.stdout: "val": 0, 2024-04-16T17:17:57.298 INFO:teuthology.orchestra.run.smithi104.stdout: "max": 3758096384, 2024-04-16T17:17:57.298 INFO:teuthology.orchestra.run.smithi104.stdout: "get_started": 0, 2024-04-16T17:17:57.298 INFO:teuthology.orchestra.run.smithi104.stdout: "get": 0, 2024-04-16T17:17:57.298 INFO:teuthology.orchestra.run.smithi104.stdout: "get_sum": 0, 2024-04-16T17:17:57.298 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_fail": 0, 2024-04-16T17:17:57.298 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_success": 0, 2024-04-16T17:17:57.299 INFO:teuthology.orchestra.run.smithi104.stdout: "take": 0, 2024-04-16T17:17:57.299 INFO:teuthology.orchestra.run.smithi104.stdout: "take_sum": 0, 2024-04-16T17:17:57.299 INFO:teuthology.orchestra.run.smithi104.stdout: "put": 0, 2024-04-16T17:17:57.299 INFO:teuthology.orchestra.run.smithi104.stdout: "put_sum": 0, 2024-04-16T17:17:57.299 INFO:teuthology.orchestra.run.smithi104.stdout: "wait": { 2024-04-16T17:17:57.299 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-04-16T17:17:57.299 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-04-16T17:17:57.299 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-04-16T17:17:57.299 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-04-16T17:17:57.300 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-04-16T17:17:57.300 INFO:teuthology.orchestra.run.smithi104.stdout:} 2024-04-16T17:17:58.018 INFO:tasks.check_counter:Daemon mds.h mds.exported=0 2024-04-16T17:17:58.018 INFO:tasks.check_counter:Daemon mds.h mds.imported=0 2024-04-16T17:17:58.018 INFO:tasks.check_counter:Daemon mds.h mds.root_rsnaps=6 2024-04-16T17:17:58.019 INFO:tasks.check_counter:Daemon mds.h mds_server.req_mksnap_latency.avgcount=0 2024-04-16T17:17:58.019 INFO:tasks.check_counter:Daemon mds.h mds_server.req_rmsnap_latency.avgcount=0 2024-04-16T17:17:58.019 DEBUG:tasks.check_counter:Getting stats from k 2024-04-16T17:17:58.019 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45 shell --fsid b1fecd38-fc11-11ee-bc90-c7b262605968 -- ceph daemon mds.k perf dump 2024-04-16T17:17:58.259 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/mon.b/config 2024-04-16T17:17:58.677 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:17:58 smithi104 ceph-mon[36457]: pgmap v350: 129 pgs: 129 active+clean; 55 MiB data, 874 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-16T17:17:58.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:17:58 smithi045 ceph-mon[31676]: pgmap v350: 129 pgs: 129 active+clean; 55 MiB data, 874 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-16T17:17:59.054 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:17:58 smithi117 ceph-mon[36730]: pgmap v350: 129 pgs: 129 active+clean; 55 MiB data, 874 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-16T17:18:00.056 INFO:teuthology.orchestra.run.smithi104.stdout:{ 2024-04-16T17:18:00.056 INFO:teuthology.orchestra.run.smithi104.stdout: "AsyncMessenger::Worker-0": { 2024-04-16T17:18:00.056 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_messages": 255, 2024-04-16T17:18:00.057 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_messages": 244, 2024-04-16T17:18:00.057 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_bytes": 1599296, 2024-04-16T17:18:00.057 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_bytes": 150640, 2024-04-16T17:18:00.057 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_created_connections": 4, 2024-04-16T17:18:00.057 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_active_connections": 0, 2024-04-16T17:18:00.057 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_total_time": 0.051342485, 2024-04-16T17:18:00.057 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_send_time": 0.015025920, 2024-04-16T17:18:00.057 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_recv_time": 0.038096775, 2024-04-16T17:18:00.057 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_fast_dispatch_time": 0.005904230, 2024-04-16T17:18:00.057 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_messages_queue_lat": { 2024-04-16T17:18:00.058 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 243, 2024-04-16T17:18:00.058 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.012039378, 2024-04-16T17:18:00.058 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000049544 2024-04-16T17:18:00.058 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-16T17:18:00.058 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_handle_ack_lat": { 2024-04-16T17:18:00.058 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-04-16T17:18:00.058 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-04-16T17:18:00.058 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-04-16T17:18:00.058 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-16T17:18:00.059 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_encrypted_bytes": 1599296, 2024-04-16T17:18:00.059 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_encrypted_bytes": 150640 2024-04-16T17:18:00.059 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-16T17:18:00.059 INFO:teuthology.orchestra.run.smithi104.stdout: "AsyncMessenger::Worker-1": { 2024-04-16T17:18:00.059 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_messages": 7, 2024-04-16T17:18:00.059 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_messages": 190, 2024-04-16T17:18:00.059 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_bytes": 3472, 2024-04-16T17:18:00.059 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_bytes": 268064, 2024-04-16T17:18:00.059 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_created_connections": 6, 2024-04-16T17:18:00.059 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_active_connections": 3, 2024-04-16T17:18:00.060 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_total_time": 0.012418169, 2024-04-16T17:18:00.060 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_send_time": 0.008296613, 2024-04-16T17:18:00.060 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_recv_time": 0.001920874, 2024-04-16T17:18:00.060 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-16T17:18:00.060 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_messages_queue_lat": { 2024-04-16T17:18:00.060 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 188, 2024-04-16T17:18:00.060 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.031862605, 2024-04-16T17:18:00.060 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000169481 2024-04-16T17:18:00.060 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-16T17:18:00.060 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_handle_ack_lat": { 2024-04-16T17:18:00.061 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-04-16T17:18:00.061 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-04-16T17:18:00.061 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-04-16T17:18:00.061 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-16T17:18:00.061 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_encrypted_bytes": 3472, 2024-04-16T17:18:00.061 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_encrypted_bytes": 268064 2024-04-16T17:18:00.061 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-16T17:18:00.061 INFO:teuthology.orchestra.run.smithi104.stdout: "AsyncMessenger::Worker-2": { 2024-04-16T17:18:00.061 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_messages": 0, 2024-04-16T17:18:00.061 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_messages": 2, 2024-04-16T17:18:00.062 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_bytes": 0, 2024-04-16T17:18:00.062 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_bytes": 0, 2024-04-16T17:18:00.062 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_created_connections": 2, 2024-04-16T17:18:00.062 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_active_connections": 1, 2024-04-16T17:18:00.062 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_total_time": 0.000257924, 2024-04-16T17:18:00.062 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_send_time": 0.000000000, 2024-04-16T17:18:00.062 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_recv_time": 0.000199265, 2024-04-16T17:18:00.062 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-16T17:18:00.062 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_messages_queue_lat": { 2024-04-16T17:18:00.063 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-04-16T17:18:00.063 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-04-16T17:18:00.063 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-04-16T17:18:00.063 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-16T17:18:00.063 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_handle_ack_lat": { 2024-04-16T17:18:00.063 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-04-16T17:18:00.063 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-04-16T17:18:00.063 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-04-16T17:18:00.063 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-16T17:18:00.064 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-16T17:18:00.064 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_encrypted_bytes": 0 2024-04-16T17:18:00.064 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-16T17:18:00.064 INFO:teuthology.orchestra.run.smithi104.stdout: "cct": { 2024-04-16T17:18:00.064 INFO:teuthology.orchestra.run.smithi104.stdout: "total_workers": 0, 2024-04-16T17:18:00.064 INFO:teuthology.orchestra.run.smithi104.stdout: "unhealthy_workers": 0 2024-04-16T17:18:00.064 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-16T17:18:00.064 INFO:teuthology.orchestra.run.smithi104.stdout: "mempool": { 2024-04-16T17:18:00.064 INFO:teuthology.orchestra.run.smithi104.stdout: "bloom_filter_bytes": 0, 2024-04-16T17:18:00.064 INFO:teuthology.orchestra.run.smithi104.stdout: "bloom_filter_items": 0, 2024-04-16T17:18:00.065 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_alloc_bytes": 0, 2024-04-16T17:18:00.065 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_alloc_items": 0, 2024-04-16T17:18:00.065 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_data_bytes": 0, 2024-04-16T17:18:00.065 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_data_items": 0, 2024-04-16T17:18:00.065 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-16T17:18:00.065 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_onode_items": 0, 2024-04-16T17:18:00.065 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-16T17:18:00.065 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_meta_items": 0, 2024-04-16T17:18:00.065 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_other_bytes": 0, 2024-04-16T17:18:00.065 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_other_items": 0, 2024-04-16T17:18:00.066 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-16T17:18:00.066 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_buffer_items": 0, 2024-04-16T17:18:00.066 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_extent_bytes": 0, 2024-04-16T17:18:00.066 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_extent_items": 0, 2024-04-16T17:18:00.066 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_blob_bytes": 0, 2024-04-16T17:18:00.066 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_blob_items": 0, 2024-04-16T17:18:00.066 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-16T17:18:00.066 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_shared_blob_items": 0, 2024-04-16T17:18:00.066 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-16T17:18:00.066 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_inline_bl_items": 0, 2024-04-16T17:18:00.067 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_fsck_bytes": 0, 2024-04-16T17:18:00.067 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_fsck_items": 0, 2024-04-16T17:18:00.067 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_txc_bytes": 0, 2024-04-16T17:18:00.067 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_txc_items": 0, 2024-04-16T17:18:00.067 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-16T17:18:00.067 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_writing_deferred_items": 0, 2024-04-16T17:18:00.067 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_writing_bytes": 0, 2024-04-16T17:18:00.067 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_writing_items": 0, 2024-04-16T17:18:00.067 INFO:teuthology.orchestra.run.smithi104.stdout: "bluefs_bytes": 0, 2024-04-16T17:18:00.068 INFO:teuthology.orchestra.run.smithi104.stdout: "bluefs_items": 0, 2024-04-16T17:18:00.068 INFO:teuthology.orchestra.run.smithi104.stdout: "bluefs_file_reader_bytes": 0, 2024-04-16T17:18:00.068 INFO:teuthology.orchestra.run.smithi104.stdout: "bluefs_file_reader_items": 0, 2024-04-16T17:18:00.068 INFO:teuthology.orchestra.run.smithi104.stdout: "bluefs_file_writer_bytes": 0, 2024-04-16T17:18:00.068 INFO:teuthology.orchestra.run.smithi104.stdout: "bluefs_file_writer_items": 0, 2024-04-16T17:18:00.068 INFO:teuthology.orchestra.run.smithi104.stdout: "buffer_anon_bytes": 148714, 2024-04-16T17:18:00.068 INFO:teuthology.orchestra.run.smithi104.stdout: "buffer_anon_items": 30, 2024-04-16T17:18:00.068 INFO:teuthology.orchestra.run.smithi104.stdout: "buffer_meta_bytes": 0, 2024-04-16T17:18:00.068 INFO:teuthology.orchestra.run.smithi104.stdout: "buffer_meta_items": 0, 2024-04-16T17:18:00.068 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_bytes": 0, 2024-04-16T17:18:00.069 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_items": 0, 2024-04-16T17:18:00.069 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_mapbl_bytes": 0, 2024-04-16T17:18:00.069 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_mapbl_items": 0, 2024-04-16T17:18:00.069 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_pglog_bytes": 0, 2024-04-16T17:18:00.069 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_pglog_items": 0, 2024-04-16T17:18:00.069 INFO:teuthology.orchestra.run.smithi104.stdout: "osdmap_bytes": 0, 2024-04-16T17:18:00.069 INFO:teuthology.orchestra.run.smithi104.stdout: "osdmap_items": 0, 2024-04-16T17:18:00.069 INFO:teuthology.orchestra.run.smithi104.stdout: "osdmap_mapping_bytes": 0, 2024-04-16T17:18:00.069 INFO:teuthology.orchestra.run.smithi104.stdout: "osdmap_mapping_items": 0, 2024-04-16T17:18:00.069 INFO:teuthology.orchestra.run.smithi104.stdout: "pgmap_bytes": 0, 2024-04-16T17:18:00.070 INFO:teuthology.orchestra.run.smithi104.stdout: "pgmap_items": 0, 2024-04-16T17:18:00.070 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_co_bytes": 928, 2024-04-16T17:18:00.070 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_co_items": 2, 2024-04-16T17:18:00.070 INFO:teuthology.orchestra.run.smithi104.stdout: "unittest_1_bytes": 0, 2024-04-16T17:18:00.070 INFO:teuthology.orchestra.run.smithi104.stdout: "unittest_1_items": 0, 2024-04-16T17:18:00.070 INFO:teuthology.orchestra.run.smithi104.stdout: "unittest_2_bytes": 0, 2024-04-16T17:18:00.070 INFO:teuthology.orchestra.run.smithi104.stdout: "unittest_2_items": 0 2024-04-16T17:18:00.070 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-04-16T17:18:00.070 INFO:teuthology.orchestra.run.smithi104.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-16T17:18:00.070 INFO:teuthology.orchestra.run.smithi104.stdout: "val": 0, 2024-04-16T17:18:00.071 INFO:teuthology.orchestra.run.smithi104.stdout: "max": 104857600, 2024-04-16T17:18:00.071 INFO:teuthology.orchestra.run.smithi104.stdout: "get_started": 0, 2024-04-16T17:18:00.071 INFO:teuthology.orchestra.run.smithi104.stdout: "get": 258, 2024-04-16T17:18:00.071 INFO:teuthology.orchestra.run.smithi104.stdout: "get_sum": 1563353, 2024-04-16T17:18:00.071 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_fail": 0, 2024-04-16T17:18:00.071 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_success": 258, 2024-04-16T17:18:00.071 INFO:teuthology.orchestra.run.smithi104.stdout: "take": 0, 2024-04-16T17:18:00.071 INFO:teuthology.orchestra.run.smithi104.stdout: "take_sum": 0, 2024-04-16T17:18:00.071 INFO:teuthology.orchestra.run.smithi104.stdout: "put": 258, 2024-04-16T17:18:00.072 INFO:teuthology.orchestra.run.smithi104.stdout: "put_sum": 1563353, 2024-04-16T17:18:00.072 INFO:teuthology.orchestra.run.smithi104.stdout: "wait": { 2024-04-16T17:18:00.072 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-04-16T17:18:00.072 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-04-16T17:18:00.072 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-04-16T17:18:00.072 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-04-16T17:18:00.072 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-04-16T17:18:00.072 INFO:teuthology.orchestra.run.smithi104.stdout:} 2024-04-16T17:18:00.677 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:18:00 smithi104 ceph-mon[36457]: pgmap v351: 129 pgs: 129 active+clean; 55 MiB data, 874 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T17:18:00.802 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.k 2024-04-16T17:18:00.802 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.k 2024-04-16T17:18:00.802 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.k 2024-04-16T17:18:00.802 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.k 2024-04-16T17:18:00.803 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.k 2024-04-16T17:18:00.803 DEBUG:tasks.check_counter:Getting stats from c 2024-04-16T17:18:00.803 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45 shell --fsid b1fecd38-fc11-11ee-bc90-c7b262605968 -- ceph daemon mds.c perf dump 2024-04-16T17:18:00.834 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:18:00 smithi117 ceph-mon[36730]: pgmap v351: 129 pgs: 129 active+clean; 55 MiB data, 874 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T17:18:00.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:18:00 smithi045 ceph-mon[31676]: pgmap v351: 129 pgs: 129 active+clean; 55 MiB data, 874 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-16T17:18:01.052 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/mon.c/config 2024-04-16T17:18:02.804 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:18:02 smithi117 ceph-mon[36730]: pgmap v352: 129 pgs: 129 active+clean; 55 MiB data, 874 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 24 op/s 2024-04-16T17:18:02.896 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-04-16T17:18:02.896 INFO:teuthology.orchestra.run.smithi117.stdout: "AsyncMessenger::Worker-0": { 2024-04-16T17:18:02.896 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_messages": 2396, 2024-04-16T17:18:02.897 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_messages": 2398, 2024-04-16T17:18:02.897 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_bytes": 11049532, 2024-04-16T17:18:02.897 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_bytes": 761935, 2024-04-16T17:18:02.897 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_created_connections": 6, 2024-04-16T17:18:02.897 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_active_connections": 2, 2024-04-16T17:18:02.897 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_total_time": 0.208967049, 2024-04-16T17:18:02.897 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_send_time": 0.070225501, 2024-04-16T17:18:02.897 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_recv_time": 0.056112076, 2024-04-16T17:18:02.897 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_fast_dispatch_time": 0.065563568, 2024-04-16T17:18:02.898 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_messages_queue_lat": { 2024-04-16T17:18:02.898 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 2396, 2024-04-16T17:18:02.898 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.084106927, 2024-04-16T17:18:02.898 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000035103 2024-04-16T17:18:02.898 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-16T17:18:02.898 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_handle_ack_lat": { 2024-04-16T17:18:02.898 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-04-16T17:18:02.898 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-04-16T17:18:02.899 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-04-16T17:18:02.899 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-16T17:18:02.899 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-16T17:18:02.899 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_encrypted_bytes": 0 2024-04-16T17:18:02.899 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-16T17:18:02.899 INFO:teuthology.orchestra.run.smithi117.stdout: "AsyncMessenger::Worker-1": { 2024-04-16T17:18:02.899 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_messages": 576, 2024-04-16T17:18:02.899 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_messages": 545, 2024-04-16T17:18:02.899 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_bytes": 1781026, 2024-04-16T17:18:02.900 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_bytes": 242424, 2024-04-16T17:18:02.900 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_created_connections": 4, 2024-04-16T17:18:02.900 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_active_connections": 3, 2024-04-16T17:18:02.900 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_total_time": 0.080319925, 2024-04-16T17:18:02.900 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_send_time": 0.024941392, 2024-04-16T17:18:02.900 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_recv_time": 0.045583908, 2024-04-16T17:18:02.900 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_fast_dispatch_time": 0.016317600, 2024-04-16T17:18:02.900 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_messages_queue_lat": { 2024-04-16T17:18:02.901 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 544, 2024-04-16T17:18:02.901 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.026095527, 2024-04-16T17:18:02.901 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000047969 2024-04-16T17:18:02.901 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-16T17:18:02.901 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_handle_ack_lat": { 2024-04-16T17:18:02.901 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-04-16T17:18:02.901 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-04-16T17:18:02.901 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-04-16T17:18:02.901 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-16T17:18:02.901 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_encrypted_bytes": 1711344, 2024-04-16T17:18:02.902 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_encrypted_bytes": 152368 2024-04-16T17:18:02.902 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-16T17:18:02.902 INFO:teuthology.orchestra.run.smithi117.stdout: "AsyncMessenger::Worker-2": { 2024-04-16T17:18:02.902 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_messages": 8, 2024-04-16T17:18:02.902 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_messages": 189, 2024-04-16T17:18:02.902 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_bytes": 3740, 2024-04-16T17:18:02.902 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_bytes": 557902, 2024-04-16T17:18:02.902 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_created_connections": 7, 2024-04-16T17:18:02.902 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_active_connections": 4, 2024-04-16T17:18:02.903 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_total_time": 0.011739628, 2024-04-16T17:18:02.903 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_send_time": 0.007039647, 2024-04-16T17:18:02.903 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_recv_time": 0.002549142, 2024-04-16T17:18:02.903 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_fast_dispatch_time": 0.000032425, 2024-04-16T17:18:02.903 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_messages_queue_lat": { 2024-04-16T17:18:02.903 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 186, 2024-04-16T17:18:02.903 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.025522359, 2024-04-16T17:18:02.903 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000137216 2024-04-16T17:18:02.904 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-16T17:18:02.904 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_handle_ack_lat": { 2024-04-16T17:18:02.904 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-04-16T17:18:02.904 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-04-16T17:18:02.904 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-04-16T17:18:02.904 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-16T17:18:02.904 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_encrypted_bytes": 3472, 2024-04-16T17:18:02.904 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_encrypted_bytes": 557584 2024-04-16T17:18:02.904 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-16T17:18:02.905 INFO:teuthology.orchestra.run.smithi117.stdout: "cct": { 2024-04-16T17:18:02.905 INFO:teuthology.orchestra.run.smithi117.stdout: "total_workers": 1, 2024-04-16T17:18:02.905 INFO:teuthology.orchestra.run.smithi117.stdout: "unhealthy_workers": 0 2024-04-16T17:18:02.905 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-16T17:18:02.905 INFO:teuthology.orchestra.run.smithi117.stdout: "finisher-MDSRank": { 2024-04-16T17:18:02.905 INFO:teuthology.orchestra.run.smithi117.stdout: "queue_len": 0, 2024-04-16T17:18:02.905 INFO:teuthology.orchestra.run.smithi117.stdout: "complete_latency": { 2024-04-16T17:18:02.905 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 4590, 2024-04-16T17:18:02.905 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.117211080, 2024-04-16T17:18:02.906 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000025536 2024-04-16T17:18:02.906 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-04-16T17:18:02.906 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-16T17:18:02.906 INFO:teuthology.orchestra.run.smithi117.stdout: "finisher-PurgeQueue": { 2024-04-16T17:18:02.906 INFO:teuthology.orchestra.run.smithi117.stdout: "queue_len": 0, 2024-04-16T17:18:02.906 INFO:teuthology.orchestra.run.smithi117.stdout: "complete_latency": { 2024-04-16T17:18:02.906 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-04-16T17:18:02.906 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-04-16T17:18:02.906 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-04-16T17:18:02.907 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-04-16T17:18:02.907 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-16T17:18:02.907 INFO:teuthology.orchestra.run.smithi117.stdout: "mds": { 2024-04-16T17:18:02.907 INFO:teuthology.orchestra.run.smithi117.stdout: "request": 0, 2024-04-16T17:18:02.907 INFO:teuthology.orchestra.run.smithi117.stdout: "reply": 0, 2024-04-16T17:18:02.907 INFO:teuthology.orchestra.run.smithi117.stdout: "reply_latency": { 2024-04-16T17:18:02.907 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-04-16T17:18:02.907 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-04-16T17:18:02.907 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-04-16T17:18:02.907 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-16T17:18:02.908 INFO:teuthology.orchestra.run.smithi117.stdout: "slow_reply": 0, 2024-04-16T17:18:02.908 INFO:teuthology.orchestra.run.smithi117.stdout: "forward": 0, 2024-04-16T17:18:02.908 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_fetch_complete": 0, 2024-04-16T17:18:02.908 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_fetch_keys": 0, 2024-04-16T17:18:02.908 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_commit": 0, 2024-04-16T17:18:02.908 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_split": 0, 2024-04-16T17:18:02.908 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_merge": 0, 2024-04-16T17:18:02.908 INFO:teuthology.orchestra.run.smithi117.stdout: "inodes": 942, 2024-04-16T17:18:02.909 INFO:teuthology.orchestra.run.smithi117.stdout: "inodes_top": 11, 2024-04-16T17:18:02.909 INFO:teuthology.orchestra.run.smithi117.stdout: "inodes_bottom": 931, 2024-04-16T17:18:02.909 INFO:teuthology.orchestra.run.smithi117.stdout: "inodes_pin_tail": 0, 2024-04-16T17:18:02.909 INFO:teuthology.orchestra.run.smithi117.stdout: "inodes_pinned": 926, 2024-04-16T17:18:02.909 INFO:teuthology.orchestra.run.smithi117.stdout: "inodes_expired": 0, 2024-04-16T17:18:02.909 INFO:teuthology.orchestra.run.smithi117.stdout: "inodes_with_caps": 0, 2024-04-16T17:18:02.909 INFO:teuthology.orchestra.run.smithi117.stdout: "caps": 0, 2024-04-16T17:18:02.909 INFO:teuthology.orchestra.run.smithi117.stdout: "subtrees": 8, 2024-04-16T17:18:02.909 INFO:teuthology.orchestra.run.smithi117.stdout: "traverse": 0, 2024-04-16T17:18:02.910 INFO:teuthology.orchestra.run.smithi117.stdout: "traverse_hit": 0, 2024-04-16T17:18:02.910 INFO:teuthology.orchestra.run.smithi117.stdout: "traverse_forward": 0, 2024-04-16T17:18:02.910 INFO:teuthology.orchestra.run.smithi117.stdout: "traverse_discover": 0, 2024-04-16T17:18:02.910 INFO:teuthology.orchestra.run.smithi117.stdout: "traverse_dir_fetch": 0, 2024-04-16T17:18:02.910 INFO:teuthology.orchestra.run.smithi117.stdout: "traverse_remote_ino": 0, 2024-04-16T17:18:02.910 INFO:teuthology.orchestra.run.smithi117.stdout: "traverse_lock": 0, 2024-04-16T17:18:02.910 INFO:teuthology.orchestra.run.smithi117.stdout: "load_cent": 0, 2024-04-16T17:18:02.910 INFO:teuthology.orchestra.run.smithi117.stdout: "q": 0, 2024-04-16T17:18:02.910 INFO:teuthology.orchestra.run.smithi117.stdout: "exported": 0, 2024-04-16T17:18:02.911 INFO:teuthology.orchestra.run.smithi117.stdout: "exported_inodes": 0, 2024-04-16T17:18:02.911 INFO:teuthology.orchestra.run.smithi117.stdout: "imported": 0, 2024-04-16T17:18:02.911 INFO:teuthology.orchestra.run.smithi117.stdout: "imported_inodes": 0, 2024-04-16T17:18:02.911 INFO:teuthology.orchestra.run.smithi117.stdout: "openino_dir_fetch": 0, 2024-04-16T17:18:02.911 INFO:teuthology.orchestra.run.smithi117.stdout: "openino_backtrace_fetch": 0, 2024-04-16T17:18:02.911 INFO:teuthology.orchestra.run.smithi117.stdout: "openino_peer_discover": 0, 2024-04-16T17:18:02.911 INFO:teuthology.orchestra.run.smithi117.stdout: "root_rfiles": 0, 2024-04-16T17:18:02.911 INFO:teuthology.orchestra.run.smithi117.stdout: "root_rbytes": 0, 2024-04-16T17:18:02.911 INFO:teuthology.orchestra.run.smithi117.stdout: "root_rsnaps": 0, 2024-04-16T17:18:02.912 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_backtrace_fetch": 0, 2024-04-16T17:18:02.912 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_set_tag": 0, 2024-04-16T17:18:02.912 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_backtrace_repaired": 0, 2024-04-16T17:18:02.912 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_inotable_repaired": 0, 2024-04-16T17:18:02.912 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_dir_inodes": 0, 2024-04-16T17:18:02.912 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_dir_base_inodes": 0, 2024-04-16T17:18:02.912 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_dirfrag_rstats": 0, 2024-04-16T17:18:02.912 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_file_inodes": 0, 2024-04-16T17:18:02.912 INFO:teuthology.orchestra.run.smithi117.stdout: "handle_inode_file_caps": 0, 2024-04-16T17:18:02.913 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph_cap_op_revoke": 0, 2024-04-16T17:18:02.913 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph_cap_op_grant": 0, 2024-04-16T17:18:02.913 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph_cap_op_trunc": 0, 2024-04-16T17:18:02.913 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-16T17:18:02.913 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-16T17:18:02.913 INFO:teuthology.orchestra.run.smithi117.stdout: "handle_client_caps": 0, 2024-04-16T17:18:02.913 INFO:teuthology.orchestra.run.smithi117.stdout: "handle_client_caps_dirty": 0, 2024-04-16T17:18:02.913 INFO:teuthology.orchestra.run.smithi117.stdout: "handle_client_cap_release": 0, 2024-04-16T17:18:02.913 INFO:teuthology.orchestra.run.smithi117.stdout: "process_request_cap_release": 0 2024-04-16T17:18:02.914 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-16T17:18:02.914 INFO:teuthology.orchestra.run.smithi117.stdout: "mds_cache": { 2024-04-16T17:18:02.914 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_update": 0, 2024-04-16T17:18:02.914 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_update_receipt": 0, 2024-04-16T17:18:02.914 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_try_discover": 0, 2024-04-16T17:18:02.914 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_send_discover": 0, 2024-04-16T17:18:02.914 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_handle_discover": 0, 2024-04-16T17:18:02.914 INFO:teuthology.orchestra.run.smithi117.stdout: "num_strays": 0, 2024-04-16T17:18:02.915 INFO:teuthology.orchestra.run.smithi117.stdout: "num_strays_delayed": 0, 2024-04-16T17:18:02.915 INFO:teuthology.orchestra.run.smithi117.stdout: "num_strays_enqueuing": 0, 2024-04-16T17:18:02.915 INFO:teuthology.orchestra.run.smithi117.stdout: "strays_created": 0, 2024-04-16T17:18:02.915 INFO:teuthology.orchestra.run.smithi117.stdout: "strays_enqueued": 0, 2024-04-16T17:18:02.915 INFO:teuthology.orchestra.run.smithi117.stdout: "strays_reintegrated": 0, 2024-04-16T17:18:02.915 INFO:teuthology.orchestra.run.smithi117.stdout: "strays_migrated": 0, 2024-04-16T17:18:02.915 INFO:teuthology.orchestra.run.smithi117.stdout: "num_recovering_processing": 0, 2024-04-16T17:18:02.915 INFO:teuthology.orchestra.run.smithi117.stdout: "num_recovering_enqueued": 0, 2024-04-16T17:18:02.915 INFO:teuthology.orchestra.run.smithi117.stdout: "num_recovering_prioritized": 0, 2024-04-16T17:18:02.916 INFO:teuthology.orchestra.run.smithi117.stdout: "recovery_started": 0, 2024-04-16T17:18:02.916 INFO:teuthology.orchestra.run.smithi117.stdout: "recovery_completed": 0, 2024-04-16T17:18:02.916 INFO:teuthology.orchestra.run.smithi117.stdout: "ireq_quiesce_path": 0, 2024-04-16T17:18:02.916 INFO:teuthology.orchestra.run.smithi117.stdout: "ireq_quiesce_inode": 0, 2024-04-16T17:18:02.916 INFO:teuthology.orchestra.run.smithi117.stdout: "ireq_enqueue_scrub": 0, 2024-04-16T17:18:02.916 INFO:teuthology.orchestra.run.smithi117.stdout: "ireq_exportdir": 0, 2024-04-16T17:18:02.916 INFO:teuthology.orchestra.run.smithi117.stdout: "ireq_flush": 0, 2024-04-16T17:18:02.916 INFO:teuthology.orchestra.run.smithi117.stdout: "ireq_fragmentdir": 0, 2024-04-16T17:18:02.916 INFO:teuthology.orchestra.run.smithi117.stdout: "ireq_fragstats": 0, 2024-04-16T17:18:02.917 INFO:teuthology.orchestra.run.smithi117.stdout: "ireq_inodestats": 0 2024-04-16T17:18:02.917 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-16T17:18:02.917 INFO:teuthology.orchestra.run.smithi117.stdout: "mds_log": { 2024-04-16T17:18:02.917 INFO:teuthology.orchestra.run.smithi117.stdout: "evlrg": 0, 2024-04-16T17:18:02.917 INFO:teuthology.orchestra.run.smithi117.stdout: "evadd": 0, 2024-04-16T17:18:02.917 INFO:teuthology.orchestra.run.smithi117.stdout: "evex": 0, 2024-04-16T17:18:02.917 INFO:teuthology.orchestra.run.smithi117.stdout: "evtrm": 0, 2024-04-16T17:18:02.917 INFO:teuthology.orchestra.run.smithi117.stdout: "ev": 7979, 2024-04-16T17:18:02.917 INFO:teuthology.orchestra.run.smithi117.stdout: "evexg": 0, 2024-04-16T17:18:02.918 INFO:teuthology.orchestra.run.smithi117.stdout: "evexd": 0, 2024-04-16T17:18:02.918 INFO:teuthology.orchestra.run.smithi117.stdout: "segadd": 0, 2024-04-16T17:18:02.918 INFO:teuthology.orchestra.run.smithi117.stdout: "segex": 0, 2024-04-16T17:18:02.918 INFO:teuthology.orchestra.run.smithi117.stdout: "segtrm": 0, 2024-04-16T17:18:02.918 INFO:teuthology.orchestra.run.smithi117.stdout: "seg": 10, 2024-04-16T17:18:02.918 INFO:teuthology.orchestra.run.smithi117.stdout: "segmjr": 2, 2024-04-16T17:18:02.918 INFO:teuthology.orchestra.run.smithi117.stdout: "segexg": 0, 2024-04-16T17:18:02.918 INFO:teuthology.orchestra.run.smithi117.stdout: "segexd": 0, 2024-04-16T17:18:02.918 INFO:teuthology.orchestra.run.smithi117.stdout: "expos": 4194304, 2024-04-16T17:18:02.919 INFO:teuthology.orchestra.run.smithi117.stdout: "wrpos": 14562744, 2024-04-16T17:18:02.919 INFO:teuthology.orchestra.run.smithi117.stdout: "rdpos": 14562170, 2024-04-16T17:18:02.919 INFO:teuthology.orchestra.run.smithi117.stdout: "jlat": { 2024-04-16T17:18:02.919 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-04-16T17:18:02.919 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-04-16T17:18:02.919 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-04-16T17:18:02.919 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-16T17:18:02.919 INFO:teuthology.orchestra.run.smithi117.stdout: "replayed": 7979 2024-04-16T17:18:02.919 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-16T17:18:02.920 INFO:teuthology.orchestra.run.smithi117.stdout: "mds_mem": { 2024-04-16T17:18:02.920 INFO:teuthology.orchestra.run.smithi117.stdout: "ino": 210, 2024-04-16T17:18:02.920 INFO:teuthology.orchestra.run.smithi117.stdout: "ino+": 970, 2024-04-16T17:18:02.920 INFO:teuthology.orchestra.run.smithi117.stdout: "ino-": 760, 2024-04-16T17:18:02.920 INFO:teuthology.orchestra.run.smithi117.stdout: "dir": 26, 2024-04-16T17:18:02.920 INFO:teuthology.orchestra.run.smithi117.stdout: "dir+": 32, 2024-04-16T17:18:02.920 INFO:teuthology.orchestra.run.smithi117.stdout: "dir-": 6, 2024-04-16T17:18:02.920 INFO:teuthology.orchestra.run.smithi117.stdout: "dn": 942, 2024-04-16T17:18:02.921 INFO:teuthology.orchestra.run.smithi117.stdout: "dn+": 1011, 2024-04-16T17:18:02.921 INFO:teuthology.orchestra.run.smithi117.stdout: "dn-": 69, 2024-04-16T17:18:02.921 INFO:teuthology.orchestra.run.smithi117.stdout: "cap": 0, 2024-04-16T17:18:02.921 INFO:teuthology.orchestra.run.smithi117.stdout: "cap+": 0, 2024-04-16T17:18:02.921 INFO:teuthology.orchestra.run.smithi117.stdout: "cap-": 0, 2024-04-16T17:18:02.921 INFO:teuthology.orchestra.run.smithi117.stdout: "rss": 50108, 2024-04-16T17:18:02.921 INFO:teuthology.orchestra.run.smithi117.stdout: "heap": 215416 2024-04-16T17:18:02.921 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-16T17:18:02.921 INFO:teuthology.orchestra.run.smithi117.stdout: "mds_server": { 2024-04-16T17:18:02.921 INFO:teuthology.orchestra.run.smithi117.stdout: "dispatch_client_request": 0, 2024-04-16T17:18:02.922 INFO:teuthology.orchestra.run.smithi117.stdout: "dispatch_server_request": 0, 2024-04-16T17:18:02.922 INFO:teuthology.orchestra.run.smithi117.stdout: "handle_client_request": 0, 2024-04-16T17:18:02.922 INFO:teuthology.orchestra.run.smithi117.stdout: "handle_client_session": 0, 2024-04-16T17:18:02.922 INFO:teuthology.orchestra.run.smithi117.stdout: "handle_peer_request": 0, 2024-04-16T17:18:02.922 INFO:teuthology.orchestra.run.smithi117.stdout: "req_create_latency": { 2024-04-16T17:18:02.922 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-04-16T17:18:02.922 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-04-16T17:18:02.922 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-04-16T17:18:02.922 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-16T17:18:02.923 INFO:teuthology.orchestra.run.smithi117.stdout: "req_getattr_latency": { 2024-04-16T17:18:02.923 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-04-16T17:18:02.923 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-04-16T17:18:02.923 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-04-16T17:18:02.923 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-16T17:18:02.923 INFO:teuthology.orchestra.run.smithi117.stdout: "req_getfilelock_latency": { 2024-04-16T17:18:02.923 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-04-16T17:18:02.923 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-04-16T17:18:02.923 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-04-16T17:18:02.924 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-16T17:18:02.924 INFO:teuthology.orchestra.run.smithi117.stdout: "req_link_latency": { 2024-04-16T17:18:02.924 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-04-16T17:18:02.924 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-04-16T17:18:02.924 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-04-16T17:18:02.924 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-16T17:18:02.924 INFO:teuthology.orchestra.run.smithi117.stdout: "req_lookup_latency": { 2024-04-16T17:18:02.924 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-04-16T17:18:02.924 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-04-16T17:18:02.924 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-04-16T17:18:02.925 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-16T17:18:02.925 INFO:teuthology.orchestra.run.smithi117.stdout: "req_lookuphash_latency": { 2024-04-16T17:18:02.925 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-04-16T17:18:02.925 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-04-16T17:18:02.925 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-04-16T17:18:02.925 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-16T17:18:02.925 INFO:teuthology.orchestra.run.smithi117.stdout: "req_lookupino_latency": { 2024-04-16T17:18:02.925 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-04-16T17:18:02.925 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-04-16T17:18:02.926 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-04-16T17:18:02.926 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-16T17:18:02.926 INFO:teuthology.orchestra.run.smithi117.stdout: "req_lookupname_latency": { 2024-04-16T17:18:02.926 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-04-16T17:18:02.926 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-04-16T17:18:02.926 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-04-16T17:18:02.926 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-16T17:18:02.926 INFO:teuthology.orchestra.run.smithi117.stdout: "req_lookupparent_latency": { 2024-04-16T17:18:02.926 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-04-16T17:18:02.927 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-04-16T17:18:02.927 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-04-16T17:18:02.927 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-16T17:18:02.927 INFO:teuthology.orchestra.run.smithi117.stdout: "req_lookupsnap_latency": { 2024-04-16T17:18:02.927 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-04-16T17:18:02.927 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-04-16T17:18:02.927 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-04-16T17:18:02.927 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-16T17:18:02.927 INFO:teuthology.orchestra.run.smithi117.stdout: "req_lssnap_latency": { 2024-04-16T17:18:02.928 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-04-16T17:18:02.928 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-04-16T17:18:02.928 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-04-16T17:18:02.928 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-16T17:18:02.928 INFO:teuthology.orchestra.run.smithi117.stdout: "req_mkdir_latency": { 2024-04-16T17:18:02.929 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:18:02 smithi104 ceph-mon[36457]: pgmap v352: 129 pgs: 129 active+clean; 55 MiB data, 874 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 24 op/s 2024-04-16T17:18:02.929 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:18:02 smithi045 ceph-mon[31676]: pgmap v352: 129 pgs: 129 active+clean; 55 MiB data, 874 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 24 op/s 2024-04-16T17:18:02.929 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-04-16T17:18:02.929 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-04-16T17:18:02.929 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-04-16T17:18:02.930 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-16T17:18:02.930 INFO:teuthology.orchestra.run.smithi117.stdout: "req_mknod_latency": { 2024-04-16T17:18:02.930 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-04-16T17:18:02.930 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-04-16T17:18:02.930 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-04-16T17:18:02.930 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-16T17:18:02.930 INFO:teuthology.orchestra.run.smithi117.stdout: "req_mksnap_latency": { 2024-04-16T17:18:02.930 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-04-16T17:18:02.931 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-04-16T17:18:02.931 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-04-16T17:18:02.931 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-16T17:18:02.931 INFO:teuthology.orchestra.run.smithi117.stdout: "req_open_latency": { 2024-04-16T17:18:02.931 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-04-16T17:18:02.931 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-04-16T17:18:02.931 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-04-16T17:18:02.931 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-16T17:18:02.931 INFO:teuthology.orchestra.run.smithi117.stdout: "req_readdir_latency": { 2024-04-16T17:18:02.931 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-04-16T17:18:02.932 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-04-16T17:18:02.932 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-04-16T17:18:02.932 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-16T17:18:02.932 INFO:teuthology.orchestra.run.smithi117.stdout: "req_rename_latency": { 2024-04-16T17:18:02.932 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-04-16T17:18:02.932 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-04-16T17:18:02.932 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-04-16T17:18:02.932 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-16T17:18:02.932 INFO:teuthology.orchestra.run.smithi117.stdout: "req_renamesnap_latency": { 2024-04-16T17:18:02.933 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-04-16T17:18:02.933 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-04-16T17:18:02.933 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-04-16T17:18:02.933 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-16T17:18:02.933 INFO:teuthology.orchestra.run.smithi117.stdout: "req_snapdiff_latency": { 2024-04-16T17:18:02.933 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-04-16T17:18:02.933 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-04-16T17:18:02.933 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-04-16T17:18:02.933 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-16T17:18:02.934 INFO:teuthology.orchestra.run.smithi117.stdout: "req_rmdir_latency": { 2024-04-16T17:18:02.934 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-04-16T17:18:02.934 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-04-16T17:18:02.934 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-04-16T17:18:02.934 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-16T17:18:02.934 INFO:teuthology.orchestra.run.smithi117.stdout: "req_rmsnap_latency": { 2024-04-16T17:18:02.934 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-04-16T17:18:02.934 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-04-16T17:18:02.934 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-04-16T17:18:02.934 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-16T17:18:02.935 INFO:teuthology.orchestra.run.smithi117.stdout: "req_rmxattr_latency": { 2024-04-16T17:18:02.935 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-04-16T17:18:02.935 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-04-16T17:18:02.935 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-04-16T17:18:02.935 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-16T17:18:02.935 INFO:teuthology.orchestra.run.smithi117.stdout: "req_setattr_latency": { 2024-04-16T17:18:02.935 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-04-16T17:18:02.935 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-04-16T17:18:02.935 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-04-16T17:18:02.936 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-16T17:18:02.936 INFO:teuthology.orchestra.run.smithi117.stdout: "req_setdirlayout_latency": { 2024-04-16T17:18:02.936 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-04-16T17:18:02.936 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-04-16T17:18:02.936 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-04-16T17:18:02.936 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-16T17:18:02.936 INFO:teuthology.orchestra.run.smithi117.stdout: "req_setfilelock_latency": { 2024-04-16T17:18:02.936 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-04-16T17:18:02.936 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-04-16T17:18:02.937 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-04-16T17:18:02.937 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-16T17:18:02.937 INFO:teuthology.orchestra.run.smithi117.stdout: "req_setlayout_latency": { 2024-04-16T17:18:02.937 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-04-16T17:18:02.937 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-04-16T17:18:02.937 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-04-16T17:18:02.937 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-16T17:18:02.937 INFO:teuthology.orchestra.run.smithi117.stdout: "req_setxattr_latency": { 2024-04-16T17:18:02.937 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-04-16T17:18:02.938 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-04-16T17:18:02.938 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-04-16T17:18:02.938 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-16T17:18:02.938 INFO:teuthology.orchestra.run.smithi117.stdout: "req_symlink_latency": { 2024-04-16T17:18:02.938 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-04-16T17:18:02.938 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-04-16T17:18:02.938 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-04-16T17:18:02.938 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-16T17:18:02.938 INFO:teuthology.orchestra.run.smithi117.stdout: "req_unlink_latency": { 2024-04-16T17:18:02.939 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-04-16T17:18:02.939 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-04-16T17:18:02.939 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-04-16T17:18:02.939 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-16T17:18:02.939 INFO:teuthology.orchestra.run.smithi117.stdout: "cap_revoke_eviction": 0, 2024-04-16T17:18:02.939 INFO:teuthology.orchestra.run.smithi117.stdout: "cap_acquisition_throttle": 0, 2024-04-16T17:18:02.939 INFO:teuthology.orchestra.run.smithi117.stdout: "req_getvxattr_latency": { 2024-04-16T17:18:02.939 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-04-16T17:18:02.939 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-04-16T17:18:02.939 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-04-16T17:18:02.940 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-04-16T17:18:02.940 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-16T17:18:02.940 INFO:teuthology.orchestra.run.smithi117.stdout: "mds_sessions": { 2024-04-16T17:18:02.940 INFO:teuthology.orchestra.run.smithi117.stdout: "session_count": 1, 2024-04-16T17:18:02.940 INFO:teuthology.orchestra.run.smithi117.stdout: "session_add": 3, 2024-04-16T17:18:02.940 INFO:teuthology.orchestra.run.smithi117.stdout: "session_remove": 2, 2024-04-16T17:18:02.940 INFO:teuthology.orchestra.run.smithi117.stdout: "sessions_open": 2, 2024-04-16T17:18:02.940 INFO:teuthology.orchestra.run.smithi117.stdout: "sessions_stale": 0, 2024-04-16T17:18:02.940 INFO:teuthology.orchestra.run.smithi117.stdout: "total_load": 0, 2024-04-16T17:18:02.940 INFO:teuthology.orchestra.run.smithi117.stdout: "average_load": 0, 2024-04-16T17:18:02.941 INFO:teuthology.orchestra.run.smithi117.stdout: "avg_session_uptime": 4119, 2024-04-16T17:18:02.941 INFO:teuthology.orchestra.run.smithi117.stdout: "mdthresh_evicted": 0 2024-04-16T17:18:02.941 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-16T17:18:02.941 INFO:teuthology.orchestra.run.smithi117.stdout: "mempool": { 2024-04-16T17:18:02.941 INFO:teuthology.orchestra.run.smithi117.stdout: "bloom_filter_bytes": 0, 2024-04-16T17:18:02.941 INFO:teuthology.orchestra.run.smithi117.stdout: "bloom_filter_items": 0, 2024-04-16T17:18:02.941 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_alloc_bytes": 0, 2024-04-16T17:18:02.941 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_alloc_items": 0, 2024-04-16T17:18:02.941 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_data_bytes": 0, 2024-04-16T17:18:02.942 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_data_items": 0, 2024-04-16T17:18:02.942 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-16T17:18:02.942 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_onode_items": 0, 2024-04-16T17:18:02.942 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-16T17:18:02.942 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_meta_items": 0, 2024-04-16T17:18:02.942 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_other_bytes": 0, 2024-04-16T17:18:02.942 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_other_items": 0, 2024-04-16T17:18:02.942 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-16T17:18:02.942 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_buffer_items": 0, 2024-04-16T17:18:02.943 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_extent_bytes": 0, 2024-04-16T17:18:02.943 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_extent_items": 0, 2024-04-16T17:18:02.943 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_blob_bytes": 0, 2024-04-16T17:18:02.943 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_blob_items": 0, 2024-04-16T17:18:02.943 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-16T17:18:02.943 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_shared_blob_items": 0, 2024-04-16T17:18:02.943 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-16T17:18:02.943 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_inline_bl_items": 0, 2024-04-16T17:18:02.943 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_fsck_bytes": 0, 2024-04-16T17:18:02.944 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_fsck_items": 0, 2024-04-16T17:18:02.944 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_txc_bytes": 0, 2024-04-16T17:18:02.944 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_txc_items": 0, 2024-04-16T17:18:02.944 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-16T17:18:02.944 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_writing_deferred_items": 0, 2024-04-16T17:18:02.944 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_writing_bytes": 0, 2024-04-16T17:18:02.944 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_writing_items": 0, 2024-04-16T17:18:02.944 INFO:teuthology.orchestra.run.smithi117.stdout: "bluefs_bytes": 0, 2024-04-16T17:18:02.944 INFO:teuthology.orchestra.run.smithi117.stdout: "bluefs_items": 0, 2024-04-16T17:18:02.944 INFO:teuthology.orchestra.run.smithi117.stdout: "bluefs_file_reader_bytes": 0, 2024-04-16T17:18:02.945 INFO:teuthology.orchestra.run.smithi117.stdout: "bluefs_file_reader_items": 0, 2024-04-16T17:18:02.945 INFO:teuthology.orchestra.run.smithi117.stdout: "bluefs_file_writer_bytes": 0, 2024-04-16T17:18:02.945 INFO:teuthology.orchestra.run.smithi117.stdout: "bluefs_file_writer_items": 0, 2024-04-16T17:18:02.945 INFO:teuthology.orchestra.run.smithi117.stdout: "buffer_anon_bytes": 198434, 2024-04-16T17:18:02.945 INFO:teuthology.orchestra.run.smithi117.stdout: "buffer_anon_items": 257, 2024-04-16T17:18:02.945 INFO:teuthology.orchestra.run.smithi117.stdout: "buffer_meta_bytes": 0, 2024-04-16T17:18:02.945 INFO:teuthology.orchestra.run.smithi117.stdout: "buffer_meta_items": 0, 2024-04-16T17:18:02.945 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_bytes": 0, 2024-04-16T17:18:02.945 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_items": 0, 2024-04-16T17:18:02.946 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_mapbl_bytes": 0, 2024-04-16T17:18:02.946 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_mapbl_items": 0, 2024-04-16T17:18:02.946 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_pglog_bytes": 0, 2024-04-16T17:18:02.946 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_pglog_items": 0, 2024-04-16T17:18:02.946 INFO:teuthology.orchestra.run.smithi117.stdout: "osdmap_bytes": 9512, 2024-04-16T17:18:02.946 INFO:teuthology.orchestra.run.smithi117.stdout: "osdmap_items": 182, 2024-04-16T17:18:02.946 INFO:teuthology.orchestra.run.smithi117.stdout: "osdmap_mapping_bytes": 0, 2024-04-16T17:18:02.946 INFO:teuthology.orchestra.run.smithi117.stdout: "osdmap_mapping_items": 0, 2024-04-16T17:18:02.946 INFO:teuthology.orchestra.run.smithi117.stdout: "pgmap_bytes": 0, 2024-04-16T17:18:02.947 INFO:teuthology.orchestra.run.smithi117.stdout: "pgmap_items": 0, 2024-04-16T17:18:02.947 INFO:teuthology.orchestra.run.smithi117.stdout: "mds_co_bytes": 1217424, 2024-04-16T17:18:02.947 INFO:teuthology.orchestra.run.smithi117.stdout: "mds_co_items": 8700, 2024-04-16T17:18:02.947 INFO:teuthology.orchestra.run.smithi117.stdout: "unittest_1_bytes": 0, 2024-04-16T17:18:02.947 INFO:teuthology.orchestra.run.smithi117.stdout: "unittest_1_items": 0, 2024-04-16T17:18:02.947 INFO:teuthology.orchestra.run.smithi117.stdout: "unittest_2_bytes": 0, 2024-04-16T17:18:02.947 INFO:teuthology.orchestra.run.smithi117.stdout: "unittest_2_items": 0 2024-04-16T17:18:02.947 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-16T17:18:02.947 INFO:teuthology.orchestra.run.smithi117.stdout: "objecter": { 2024-04-16T17:18:02.947 INFO:teuthology.orchestra.run.smithi117.stdout: "op_active": 0, 2024-04-16T17:18:02.948 INFO:teuthology.orchestra.run.smithi117.stdout: "op_laggy": 0, 2024-04-16T17:18:02.948 INFO:teuthology.orchestra.run.smithi117.stdout: "op_send": 2680, 2024-04-16T17:18:02.948 INFO:teuthology.orchestra.run.smithi117.stdout: "op_send_bytes": 21, 2024-04-16T17:18:02.948 INFO:teuthology.orchestra.run.smithi117.stdout: "op_resend": 3, 2024-04-16T17:18:02.948 INFO:teuthology.orchestra.run.smithi117.stdout: "op_reply": 2680, 2024-04-16T17:18:02.948 INFO:teuthology.orchestra.run.smithi117.stdout: "op_latency": { 2024-04-16T17:18:02.948 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 2680, 2024-04-16T17:18:02.948 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 1.302975765, 2024-04-16T17:18:02.948 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000486184 2024-04-16T17:18:02.949 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-16T17:18:02.949 INFO:teuthology.orchestra.run.smithi117.stdout: "op_inflight": 0, 2024-04-16T17:18:02.949 INFO:teuthology.orchestra.run.smithi117.stdout: "oplen_avg": { 2024-04-16T17:18:02.949 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 2680, 2024-04-16T17:18:02.949 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 2681 2024-04-16T17:18:02.949 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-16T17:18:02.949 INFO:teuthology.orchestra.run.smithi117.stdout: "op": 2680, 2024-04-16T17:18:02.949 INFO:teuthology.orchestra.run.smithi117.stdout: "op_r": 2680, 2024-04-16T17:18:02.949 INFO:teuthology.orchestra.run.smithi117.stdout: "op_w": 0, 2024-04-16T17:18:02.949 INFO:teuthology.orchestra.run.smithi117.stdout: "op_rmw": 0, 2024-04-16T17:18:02.950 INFO:teuthology.orchestra.run.smithi117.stdout: "op_pg": 0, 2024-04-16T17:18:02.950 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_stat": 1609, 2024-04-16T17:18:02.950 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_create": 0, 2024-04-16T17:18:02.950 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_read": 1069, 2024-04-16T17:18:02.950 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_write": 0, 2024-04-16T17:18:02.950 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_writefull": 0, 2024-04-16T17:18:02.950 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_writesame": 0, 2024-04-16T17:18:02.950 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_append": 0, 2024-04-16T17:18:02.950 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_zero": 0, 2024-04-16T17:18:02.951 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_truncate": 0, 2024-04-16T17:18:02.951 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_delete": 0, 2024-04-16T17:18:02.951 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_mapext": 0, 2024-04-16T17:18:02.951 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_sparse_read": 0, 2024-04-16T17:18:02.951 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_clonerange": 0, 2024-04-16T17:18:02.951 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_getxattr": 1, 2024-04-16T17:18:02.951 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_setxattr": 0, 2024-04-16T17:18:02.951 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_cmpxattr": 0, 2024-04-16T17:18:02.951 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_rmxattr": 0, 2024-04-16T17:18:02.951 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_resetxattrs": 0, 2024-04-16T17:18:02.952 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_call": 0, 2024-04-16T17:18:02.952 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_watch": 0, 2024-04-16T17:18:02.952 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_notify": 0, 2024-04-16T17:18:02.952 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_src_cmpxattr": 0, 2024-04-16T17:18:02.952 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_pgls": 0, 2024-04-16T17:18:02.952 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_pgls_filter": 0, 2024-04-16T17:18:02.952 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_other": 0, 2024-04-16T17:18:02.952 INFO:teuthology.orchestra.run.smithi117.stdout: "linger_active": 0, 2024-04-16T17:18:02.952 INFO:teuthology.orchestra.run.smithi117.stdout: "linger_send": 0, 2024-04-16T17:18:02.952 INFO:teuthology.orchestra.run.smithi117.stdout: "linger_resend": 0, 2024-04-16T17:18:02.953 INFO:teuthology.orchestra.run.smithi117.stdout: "linger_ping": 0, 2024-04-16T17:18:02.953 INFO:teuthology.orchestra.run.smithi117.stdout: "poolop_active": 0, 2024-04-16T17:18:02.953 INFO:teuthology.orchestra.run.smithi117.stdout: "poolop_send": 0, 2024-04-16T17:18:02.953 INFO:teuthology.orchestra.run.smithi117.stdout: "poolop_resend": 0, 2024-04-16T17:18:02.953 INFO:teuthology.orchestra.run.smithi117.stdout: "poolstat_active": 0, 2024-04-16T17:18:02.953 INFO:teuthology.orchestra.run.smithi117.stdout: "poolstat_send": 0, 2024-04-16T17:18:02.953 INFO:teuthology.orchestra.run.smithi117.stdout: "poolstat_resend": 0, 2024-04-16T17:18:02.953 INFO:teuthology.orchestra.run.smithi117.stdout: "statfs_active": 0, 2024-04-16T17:18:02.953 INFO:teuthology.orchestra.run.smithi117.stdout: "statfs_send": 0, 2024-04-16T17:18:02.953 INFO:teuthology.orchestra.run.smithi117.stdout: "statfs_resend": 0, 2024-04-16T17:18:02.954 INFO:teuthology.orchestra.run.smithi117.stdout: "command_active": 0, 2024-04-16T17:18:02.954 INFO:teuthology.orchestra.run.smithi117.stdout: "command_send": 0, 2024-04-16T17:18:02.954 INFO:teuthology.orchestra.run.smithi117.stdout: "command_resend": 0, 2024-04-16T17:18:02.954 INFO:teuthology.orchestra.run.smithi117.stdout: "map_epoch": 106, 2024-04-16T17:18:02.954 INFO:teuthology.orchestra.run.smithi117.stdout: "map_full": 0, 2024-04-16T17:18:02.954 INFO:teuthology.orchestra.run.smithi117.stdout: "map_inc": 16, 2024-04-16T17:18:02.954 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_sessions": 5, 2024-04-16T17:18:02.954 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_session_open": 5, 2024-04-16T17:18:02.955 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_session_close": 0, 2024-04-16T17:18:02.955 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_laggy": 0, 2024-04-16T17:18:02.955 INFO:teuthology.orchestra.run.smithi117.stdout: "omap_wr": 0, 2024-04-16T17:18:02.955 INFO:teuthology.orchestra.run.smithi117.stdout: "omap_rd": 2, 2024-04-16T17:18:02.955 INFO:teuthology.orchestra.run.smithi117.stdout: "omap_del": 0 2024-04-16T17:18:02.955 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-16T17:18:02.955 INFO:teuthology.orchestra.run.smithi117.stdout: "oft": { 2024-04-16T17:18:02.955 INFO:teuthology.orchestra.run.smithi117.stdout: "omap_total_objs": 0, 2024-04-16T17:18:02.955 INFO:teuthology.orchestra.run.smithi117.stdout: "omap_total_kv_pairs": 0, 2024-04-16T17:18:02.955 INFO:teuthology.orchestra.run.smithi117.stdout: "omap_total_updates": 0, 2024-04-16T17:18:02.956 INFO:teuthology.orchestra.run.smithi117.stdout: "omap_total_removes": 0 2024-04-16T17:18:02.956 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-16T17:18:02.956 INFO:teuthology.orchestra.run.smithi117.stdout: "osd-slow-ops": { 2024-04-16T17:18:02.956 INFO:teuthology.orchestra.run.smithi117.stdout: "slow_ops_count": 0 2024-04-16T17:18:02.956 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-16T17:18:02.956 INFO:teuthology.orchestra.run.smithi117.stdout: "purge_queue": { 2024-04-16T17:18:02.956 INFO:teuthology.orchestra.run.smithi117.stdout: "pq_executing_ops": 0, 2024-04-16T17:18:02.956 INFO:teuthology.orchestra.run.smithi117.stdout: "pq_executing_ops_high_water": 0, 2024-04-16T17:18:02.956 INFO:teuthology.orchestra.run.smithi117.stdout: "pq_executing": 0, 2024-04-16T17:18:02.956 INFO:teuthology.orchestra.run.smithi117.stdout: "pq_executing_high_water": 0, 2024-04-16T17:18:02.957 INFO:teuthology.orchestra.run.smithi117.stdout: "pq_executed_ops": 0, 2024-04-16T17:18:02.957 INFO:teuthology.orchestra.run.smithi117.stdout: "pq_executed": 0, 2024-04-16T17:18:02.957 INFO:teuthology.orchestra.run.smithi117.stdout: "pq_item_in_journal": 0 2024-04-16T17:18:02.957 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-16T17:18:02.957 INFO:teuthology.orchestra.run.smithi117.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-16T17:18:02.957 INFO:teuthology.orchestra.run.smithi117.stdout: "val": 0, 2024-04-16T17:18:02.957 INFO:teuthology.orchestra.run.smithi117.stdout: "max": 104857600, 2024-04-16T17:18:02.957 INFO:teuthology.orchestra.run.smithi117.stdout: "get_started": 0, 2024-04-16T17:18:02.957 INFO:teuthology.orchestra.run.smithi117.stdout: "get": 2976, 2024-04-16T17:18:02.958 INFO:teuthology.orchestra.run.smithi117.stdout: "get_sum": 12548485, 2024-04-16T17:18:02.958 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_fail": 0, 2024-04-16T17:18:02.958 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_success": 2976, 2024-04-16T17:18:02.958 INFO:teuthology.orchestra.run.smithi117.stdout: "take": 0, 2024-04-16T17:18:02.958 INFO:teuthology.orchestra.run.smithi117.stdout: "take_sum": 0, 2024-04-16T17:18:02.958 INFO:teuthology.orchestra.run.smithi117.stdout: "put": 2976, 2024-04-16T17:18:02.958 INFO:teuthology.orchestra.run.smithi117.stdout: "put_sum": 12548485, 2024-04-16T17:18:02.958 INFO:teuthology.orchestra.run.smithi117.stdout: "wait": { 2024-04-16T17:18:02.958 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-04-16T17:18:02.959 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-04-16T17:18:02.959 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-04-16T17:18:02.959 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-04-16T17:18:02.959 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-16T17:18:02.959 INFO:teuthology.orchestra.run.smithi117.stdout: "throttle-objecter_bytes": { 2024-04-16T17:18:02.959 INFO:teuthology.orchestra.run.smithi117.stdout: "val": 0, 2024-04-16T17:18:02.959 INFO:teuthology.orchestra.run.smithi117.stdout: "max": 104857600, 2024-04-16T17:18:02.959 INFO:teuthology.orchestra.run.smithi117.stdout: "get_started": 0, 2024-04-16T17:18:02.959 INFO:teuthology.orchestra.run.smithi117.stdout: "get": 0, 2024-04-16T17:18:02.959 INFO:teuthology.orchestra.run.smithi117.stdout: "get_sum": 0, 2024-04-16T17:18:02.960 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_fail": 0, 2024-04-16T17:18:02.960 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_success": 0, 2024-04-16T17:18:02.960 INFO:teuthology.orchestra.run.smithi117.stdout: "take": 2680, 2024-04-16T17:18:02.960 INFO:teuthology.orchestra.run.smithi117.stdout: "take_sum": 10369322, 2024-04-16T17:18:02.960 INFO:teuthology.orchestra.run.smithi117.stdout: "put": 2680, 2024-04-16T17:18:02.960 INFO:teuthology.orchestra.run.smithi117.stdout: "put_sum": 10369322, 2024-04-16T17:18:02.960 INFO:teuthology.orchestra.run.smithi117.stdout: "wait": { 2024-04-16T17:18:02.960 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-04-16T17:18:02.960 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-04-16T17:18:02.960 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-04-16T17:18:02.961 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-04-16T17:18:02.961 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-16T17:18:02.961 INFO:teuthology.orchestra.run.smithi117.stdout: "throttle-objecter_ops": { 2024-04-16T17:18:02.961 INFO:teuthology.orchestra.run.smithi117.stdout: "val": 0, 2024-04-16T17:18:02.961 INFO:teuthology.orchestra.run.smithi117.stdout: "max": 1024, 2024-04-16T17:18:02.961 INFO:teuthology.orchestra.run.smithi117.stdout: "get_started": 0, 2024-04-16T17:18:02.961 INFO:teuthology.orchestra.run.smithi117.stdout: "get": 0, 2024-04-16T17:18:02.961 INFO:teuthology.orchestra.run.smithi117.stdout: "get_sum": 0, 2024-04-16T17:18:02.961 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_fail": 0, 2024-04-16T17:18:02.962 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_success": 0, 2024-04-16T17:18:02.962 INFO:teuthology.orchestra.run.smithi117.stdout: "take": 2680, 2024-04-16T17:18:02.962 INFO:teuthology.orchestra.run.smithi117.stdout: "take_sum": 2680, 2024-04-16T17:18:02.962 INFO:teuthology.orchestra.run.smithi117.stdout: "put": 2680, 2024-04-16T17:18:02.962 INFO:teuthology.orchestra.run.smithi117.stdout: "put_sum": 2680, 2024-04-16T17:18:02.962 INFO:teuthology.orchestra.run.smithi117.stdout: "wait": { 2024-04-16T17:18:02.962 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-04-16T17:18:02.962 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-04-16T17:18:02.962 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-04-16T17:18:02.963 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-04-16T17:18:02.963 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-16T17:18:02.963 INFO:teuthology.orchestra.run.smithi117.stdout: "throttle-write_buf_throttle": { 2024-04-16T17:18:02.963 INFO:teuthology.orchestra.run.smithi117.stdout: "val": 0, 2024-04-16T17:18:02.963 INFO:teuthology.orchestra.run.smithi117.stdout: "max": 3758096384, 2024-04-16T17:18:02.963 INFO:teuthology.orchestra.run.smithi117.stdout: "get_started": 0, 2024-04-16T17:18:02.963 INFO:teuthology.orchestra.run.smithi117.stdout: "get": 0, 2024-04-16T17:18:02.963 INFO:teuthology.orchestra.run.smithi117.stdout: "get_sum": 0, 2024-04-16T17:18:02.963 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_fail": 0, 2024-04-16T17:18:02.963 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_success": 0, 2024-04-16T17:18:02.964 INFO:teuthology.orchestra.run.smithi117.stdout: "take": 0, 2024-04-16T17:18:02.964 INFO:teuthology.orchestra.run.smithi117.stdout: "take_sum": 0, 2024-04-16T17:18:02.964 INFO:teuthology.orchestra.run.smithi117.stdout: "put": 0, 2024-04-16T17:18:02.964 INFO:teuthology.orchestra.run.smithi117.stdout: "put_sum": 0, 2024-04-16T17:18:02.964 INFO:teuthology.orchestra.run.smithi117.stdout: "wait": { 2024-04-16T17:18:02.964 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-04-16T17:18:02.964 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-04-16T17:18:02.964 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-04-16T17:18:02.964 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-04-16T17:18:02.964 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-16T17:18:02.965 INFO:teuthology.orchestra.run.smithi117.stdout: "throttle-write_buf_throttle-0x556c9b0bc6e0": { 2024-04-16T17:18:02.965 INFO:teuthology.orchestra.run.smithi117.stdout: "val": 0, 2024-04-16T17:18:02.965 INFO:teuthology.orchestra.run.smithi117.stdout: "max": 3758096384, 2024-04-16T17:18:02.965 INFO:teuthology.orchestra.run.smithi117.stdout: "get_started": 0, 2024-04-16T17:18:02.965 INFO:teuthology.orchestra.run.smithi117.stdout: "get": 0, 2024-04-16T17:18:02.965 INFO:teuthology.orchestra.run.smithi117.stdout: "get_sum": 0, 2024-04-16T17:18:02.965 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_fail": 0, 2024-04-16T17:18:02.965 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_success": 0, 2024-04-16T17:18:02.965 INFO:teuthology.orchestra.run.smithi117.stdout: "take": 0, 2024-04-16T17:18:02.965 INFO:teuthology.orchestra.run.smithi117.stdout: "take_sum": 0, 2024-04-16T17:18:02.966 INFO:teuthology.orchestra.run.smithi117.stdout: "put": 0, 2024-04-16T17:18:02.966 INFO:teuthology.orchestra.run.smithi117.stdout: "put_sum": 0, 2024-04-16T17:18:02.966 INFO:teuthology.orchestra.run.smithi117.stdout: "wait": { 2024-04-16T17:18:02.966 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-04-16T17:18:02.966 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-04-16T17:18:02.966 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-04-16T17:18:02.966 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-04-16T17:18:02.966 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-04-16T17:18:02.966 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-04-16T17:18:03.647 INFO:tasks.check_counter:Daemon mds.c mds.exported=0 2024-04-16T17:18:03.647 INFO:tasks.check_counter:Daemon mds.c mds.imported=0 2024-04-16T17:18:03.647 INFO:tasks.check_counter:Daemon mds.c mds.root_rsnaps=0 2024-04-16T17:18:03.647 INFO:tasks.check_counter:Daemon mds.c mds_server.req_mksnap_latency.avgcount=0 2024-04-16T17:18:03.647 INFO:tasks.check_counter:Daemon mds.c mds_server.req_rmsnap_latency.avgcount=0 2024-04-16T17:18:03.648 DEBUG:tasks.check_counter:Getting stats from f 2024-04-16T17:18:03.648 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45 shell --fsid b1fecd38-fc11-11ee-bc90-c7b262605968 -- ceph daemon mds.f perf dump 2024-04-16T17:18:03.937 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/mon.c/config 2024-04-16T17:18:04.858 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:18:04 smithi117 ceph-mon[36730]: pgmap v353: 129 pgs: 129 active+clean; 55 MiB data, 874 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-04-16T17:18:04.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:18:04 smithi104 ceph-mon[36457]: pgmap v353: 129 pgs: 129 active+clean; 55 MiB data, 874 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-04-16T17:18:04.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:18:04 smithi045 ceph-mon[31676]: pgmap v353: 129 pgs: 129 active+clean; 55 MiB data, 874 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-04-16T17:18:05.647 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-04-16T17:18:05.647 INFO:teuthology.orchestra.run.smithi117.stdout: "AsyncMessenger::Worker-0": { 2024-04-16T17:18:05.647 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_messages": 532, 2024-04-16T17:18:05.647 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_messages": 499, 2024-04-16T17:18:05.648 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_bytes": 1772760, 2024-04-16T17:18:05.648 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_bytes": 227874, 2024-04-16T17:18:05.648 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_created_connections": 6, 2024-04-16T17:18:05.648 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_active_connections": 2, 2024-04-16T17:18:05.648 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_total_time": 0.072919900, 2024-04-16T17:18:05.648 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_send_time": 0.021825372, 2024-04-16T17:18:05.649 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_recv_time": 0.043360689, 2024-04-16T17:18:05.649 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_fast_dispatch_time": 0.014120690, 2024-04-16T17:18:05.649 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_messages_queue_lat": { 2024-04-16T17:18:05.649 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 499, 2024-04-16T17:18:05.649 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.022838931, 2024-04-16T17:18:05.649 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000045769 2024-04-16T17:18:05.649 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-16T17:18:05.650 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_handle_ack_lat": { 2024-04-16T17:18:05.650 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-04-16T17:18:05.650 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-04-16T17:18:05.650 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-04-16T17:18:05.650 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-16T17:18:05.650 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_encrypted_bytes": 1714608, 2024-04-16T17:18:05.651 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_encrypted_bytes": 153456 2024-04-16T17:18:05.651 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-16T17:18:05.651 INFO:teuthology.orchestra.run.smithi117.stdout: "AsyncMessenger::Worker-1": { 2024-04-16T17:18:05.651 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_messages": 838, 2024-04-16T17:18:05.651 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_messages": 1021, 2024-04-16T17:18:05.651 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_bytes": 5137099, 2024-04-16T17:18:05.652 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_bytes": 746253, 2024-04-16T17:18:05.652 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_created_connections": 7, 2024-04-16T17:18:05.652 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_active_connections": 5, 2024-04-16T17:18:05.652 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_total_time": 0.093502592, 2024-04-16T17:18:05.652 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_send_time": 0.031466443, 2024-04-16T17:18:05.652 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_recv_time": 0.025272805, 2024-04-16T17:18:05.652 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_fast_dispatch_time": 0.027126790, 2024-04-16T17:18:05.653 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_messages_queue_lat": { 2024-04-16T17:18:05.653 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 1018, 2024-04-16T17:18:05.653 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.060837916, 2024-04-16T17:18:05.653 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000059762 2024-04-16T17:18:05.653 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-16T17:18:05.653 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_handle_ack_lat": { 2024-04-16T17:18:05.653 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-04-16T17:18:05.654 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-04-16T17:18:05.654 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-04-16T17:18:05.654 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-16T17:18:05.654 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-16T17:18:05.654 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_encrypted_bytes": 480400 2024-04-16T17:18:05.654 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-16T17:18:05.655 INFO:teuthology.orchestra.run.smithi117.stdout: "AsyncMessenger::Worker-2": { 2024-04-16T17:18:05.655 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_messages": 1595, 2024-04-16T17:18:05.655 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_messages": 1597, 2024-04-16T17:18:05.655 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_bytes": 8865897, 2024-04-16T17:18:05.655 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_bytes": 507268, 2024-04-16T17:18:05.655 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_created_connections": 5, 2024-04-16T17:18:05.655 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_active_connections": 4, 2024-04-16T17:18:05.656 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_total_time": 0.167676147, 2024-04-16T17:18:05.656 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_send_time": 0.046949369, 2024-04-16T17:18:05.656 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_recv_time": 0.051782075, 2024-04-16T17:18:05.656 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_fast_dispatch_time": 0.054342543, 2024-04-16T17:18:05.656 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_messages_queue_lat": { 2024-04-16T17:18:05.656 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 1595, 2024-04-16T17:18:05.656 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.063258347, 2024-04-16T17:18:05.657 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000039660 2024-04-16T17:18:05.657 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-16T17:18:05.657 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_handle_ack_lat": { 2024-04-16T17:18:05.657 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-04-16T17:18:05.657 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-04-16T17:18:05.657 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-04-16T17:18:05.657 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-16T17:18:05.658 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-16T17:18:05.658 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_encrypted_bytes": 0 2024-04-16T17:18:05.658 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-16T17:18:05.658 INFO:teuthology.orchestra.run.smithi117.stdout: "cct": { 2024-04-16T17:18:05.658 INFO:teuthology.orchestra.run.smithi117.stdout: "total_workers": 1, 2024-04-16T17:18:05.658 INFO:teuthology.orchestra.run.smithi117.stdout: "unhealthy_workers": 0 2024-04-16T17:18:05.659 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-16T17:18:05.659 INFO:teuthology.orchestra.run.smithi117.stdout: "finisher-MDSRank": { 2024-04-16T17:18:05.659 INFO:teuthology.orchestra.run.smithi117.stdout: "queue_len": 0, 2024-04-16T17:18:05.659 INFO:teuthology.orchestra.run.smithi117.stdout: "complete_latency": { 2024-04-16T17:18:05.659 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 4514, 2024-04-16T17:18:05.659 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.126864471, 2024-04-16T17:18:05.659 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000028104 2024-04-16T17:18:05.660 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-04-16T17:18:05.660 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-16T17:18:05.660 INFO:teuthology.orchestra.run.smithi117.stdout: "finisher-PurgeQueue": { 2024-04-16T17:18:05.660 INFO:teuthology.orchestra.run.smithi117.stdout: "queue_len": 0, 2024-04-16T17:18:05.660 INFO:teuthology.orchestra.run.smithi117.stdout: "complete_latency": { 2024-04-16T17:18:05.660 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-04-16T17:18:05.660 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-04-16T17:18:05.661 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-04-16T17:18:05.661 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-04-16T17:18:05.661 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-16T17:18:05.661 INFO:teuthology.orchestra.run.smithi117.stdout: "mds": { 2024-04-16T17:18:05.661 INFO:teuthology.orchestra.run.smithi117.stdout: "request": 0, 2024-04-16T17:18:05.661 INFO:teuthology.orchestra.run.smithi117.stdout: "reply": 0, 2024-04-16T17:18:05.662 INFO:teuthology.orchestra.run.smithi117.stdout: "reply_latency": { 2024-04-16T17:18:05.662 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-04-16T17:18:05.662 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-04-16T17:18:05.662 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-04-16T17:18:05.662 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-16T17:18:05.662 INFO:teuthology.orchestra.run.smithi117.stdout: "slow_reply": 0, 2024-04-16T17:18:05.662 INFO:teuthology.orchestra.run.smithi117.stdout: "forward": 0, 2024-04-16T17:18:05.663 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_fetch_complete": 0, 2024-04-16T17:18:05.663 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_fetch_keys": 0, 2024-04-16T17:18:05.663 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_commit": 0, 2024-04-16T17:18:05.663 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_split": 0, 2024-04-16T17:18:05.663 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_merge": 0, 2024-04-16T17:18:05.663 INFO:teuthology.orchestra.run.smithi117.stdout: "inodes": 1863, 2024-04-16T17:18:05.664 INFO:teuthology.orchestra.run.smithi117.stdout: "inodes_top": 21, 2024-04-16T17:18:05.664 INFO:teuthology.orchestra.run.smithi117.stdout: "inodes_bottom": 1842, 2024-04-16T17:18:05.664 INFO:teuthology.orchestra.run.smithi117.stdout: "inodes_pin_tail": 0, 2024-04-16T17:18:05.664 INFO:teuthology.orchestra.run.smithi117.stdout: "inodes_pinned": 1834, 2024-04-16T17:18:05.664 INFO:teuthology.orchestra.run.smithi117.stdout: "inodes_expired": 0, 2024-04-16T17:18:05.664 INFO:teuthology.orchestra.run.smithi117.stdout: "inodes_with_caps": 0, 2024-04-16T17:18:05.664 INFO:teuthology.orchestra.run.smithi117.stdout: "caps": 0, 2024-04-16T17:18:05.665 INFO:teuthology.orchestra.run.smithi117.stdout: "subtrees": 5, 2024-04-16T17:18:05.665 INFO:teuthology.orchestra.run.smithi117.stdout: "traverse": 0, 2024-04-16T17:18:05.665 INFO:teuthology.orchestra.run.smithi117.stdout: "traverse_hit": 0, 2024-04-16T17:18:05.665 INFO:teuthology.orchestra.run.smithi117.stdout: "traverse_forward": 0, 2024-04-16T17:18:05.665 INFO:teuthology.orchestra.run.smithi117.stdout: "traverse_discover": 0, 2024-04-16T17:18:05.665 INFO:teuthology.orchestra.run.smithi117.stdout: "traverse_dir_fetch": 0, 2024-04-16T17:18:05.665 INFO:teuthology.orchestra.run.smithi117.stdout: "traverse_remote_ino": 0, 2024-04-16T17:18:05.665 INFO:teuthology.orchestra.run.smithi117.stdout: "traverse_lock": 0, 2024-04-16T17:18:05.666 INFO:teuthology.orchestra.run.smithi117.stdout: "load_cent": 0, 2024-04-16T17:18:05.666 INFO:teuthology.orchestra.run.smithi117.stdout: "q": 0, 2024-04-16T17:18:05.666 INFO:teuthology.orchestra.run.smithi117.stdout: "exported": 0, 2024-04-16T17:18:05.666 INFO:teuthology.orchestra.run.smithi117.stdout: "exported_inodes": 0, 2024-04-16T17:18:05.666 INFO:teuthology.orchestra.run.smithi117.stdout: "imported": 0, 2024-04-16T17:18:05.666 INFO:teuthology.orchestra.run.smithi117.stdout: "imported_inodes": 0, 2024-04-16T17:18:05.666 INFO:teuthology.orchestra.run.smithi117.stdout: "openino_dir_fetch": 0, 2024-04-16T17:18:05.667 INFO:teuthology.orchestra.run.smithi117.stdout: "openino_backtrace_fetch": 0, 2024-04-16T17:18:05.667 INFO:teuthology.orchestra.run.smithi117.stdout: "openino_peer_discover": 0, 2024-04-16T17:18:05.667 INFO:teuthology.orchestra.run.smithi117.stdout: "root_rfiles": 0, 2024-04-16T17:18:05.667 INFO:teuthology.orchestra.run.smithi117.stdout: "root_rbytes": 0, 2024-04-16T17:18:05.667 INFO:teuthology.orchestra.run.smithi117.stdout: "root_rsnaps": 0, 2024-04-16T17:18:05.667 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_backtrace_fetch": 0, 2024-04-16T17:18:05.667 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_set_tag": 0, 2024-04-16T17:18:05.667 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_backtrace_repaired": 0, 2024-04-16T17:18:05.668 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_inotable_repaired": 0, 2024-04-16T17:18:05.668 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_dir_inodes": 0, 2024-04-16T17:18:05.668 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_dir_base_inodes": 0, 2024-04-16T17:18:05.668 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_dirfrag_rstats": 0, 2024-04-16T17:18:05.668 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_file_inodes": 0, 2024-04-16T17:18:05.668 INFO:teuthology.orchestra.run.smithi117.stdout: "handle_inode_file_caps": 0, 2024-04-16T17:18:05.668 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph_cap_op_revoke": 0, 2024-04-16T17:18:05.668 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph_cap_op_grant": 0, 2024-04-16T17:18:05.669 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph_cap_op_trunc": 0, 2024-04-16T17:18:05.669 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-16T17:18:05.669 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-16T17:18:05.669 INFO:teuthology.orchestra.run.smithi117.stdout: "handle_client_caps": 0, 2024-04-16T17:18:05.669 INFO:teuthology.orchestra.run.smithi117.stdout: "handle_client_caps_dirty": 0, 2024-04-16T17:18:05.669 INFO:teuthology.orchestra.run.smithi117.stdout: "handle_client_cap_release": 0, 2024-04-16T17:18:05.669 INFO:teuthology.orchestra.run.smithi117.stdout: "process_request_cap_release": 0 2024-04-16T17:18:05.670 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-16T17:18:05.670 INFO:teuthology.orchestra.run.smithi117.stdout: "mds_cache": { 2024-04-16T17:18:05.670 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_update": 0, 2024-04-16T17:18:05.670 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_update_receipt": 0, 2024-04-16T17:18:05.670 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_try_discover": 0, 2024-04-16T17:18:05.670 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_send_discover": 0, 2024-04-16T17:18:05.671 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_handle_discover": 0, 2024-04-16T17:18:05.671 INFO:teuthology.orchestra.run.smithi117.stdout: "num_strays": 0, 2024-04-16T17:18:05.671 INFO:teuthology.orchestra.run.smithi117.stdout: "num_strays_delayed": 0, 2024-04-16T17:18:05.671 INFO:teuthology.orchestra.run.smithi117.stdout: "num_strays_enqueuing": 0, 2024-04-16T17:18:05.671 INFO:teuthology.orchestra.run.smithi117.stdout: "strays_created": 0, 2024-04-16T17:18:05.671 INFO:teuthology.orchestra.run.smithi117.stdout: "strays_enqueued": 0, 2024-04-16T17:18:05.671 INFO:teuthology.orchestra.run.smithi117.stdout: "strays_reintegrated": 0, 2024-04-16T17:18:05.671 INFO:teuthology.orchestra.run.smithi117.stdout: "strays_migrated": 0, 2024-04-16T17:18:05.672 INFO:teuthology.orchestra.run.smithi117.stdout: "num_recovering_processing": 0, 2024-04-16T17:18:05.672 INFO:teuthology.orchestra.run.smithi117.stdout: "num_recovering_enqueued": 0, 2024-04-16T17:18:05.672 INFO:teuthology.orchestra.run.smithi117.stdout: "num_recovering_prioritized": 0, 2024-04-16T17:18:05.672 INFO:teuthology.orchestra.run.smithi117.stdout: "recovery_started": 0, 2024-04-16T17:18:05.672 INFO:teuthology.orchestra.run.smithi117.stdout: "recovery_completed": 0, 2024-04-16T17:18:05.672 INFO:teuthology.orchestra.run.smithi117.stdout: "ireq_quiesce_path": 0, 2024-04-16T17:18:05.672 INFO:teuthology.orchestra.run.smithi117.stdout: "ireq_quiesce_inode": 0, 2024-04-16T17:18:05.673 INFO:teuthology.orchestra.run.smithi117.stdout: "ireq_enqueue_scrub": 0, 2024-04-16T17:18:05.673 INFO:teuthology.orchestra.run.smithi117.stdout: "ireq_exportdir": 0, 2024-04-16T17:18:05.673 INFO:teuthology.orchestra.run.smithi117.stdout: "ireq_flush": 0, 2024-04-16T17:18:05.673 INFO:teuthology.orchestra.run.smithi117.stdout: "ireq_fragmentdir": 0, 2024-04-16T17:18:05.673 INFO:teuthology.orchestra.run.smithi117.stdout: "ireq_fragstats": 0, 2024-04-16T17:18:05.673 INFO:teuthology.orchestra.run.smithi117.stdout: "ireq_inodestats": 0 2024-04-16T17:18:05.673 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-16T17:18:05.673 INFO:teuthology.orchestra.run.smithi117.stdout: "mds_log": { 2024-04-16T17:18:05.674 INFO:teuthology.orchestra.run.smithi117.stdout: "evlrg": 0, 2024-04-16T17:18:05.674 INFO:teuthology.orchestra.run.smithi117.stdout: "evadd": 0, 2024-04-16T17:18:05.674 INFO:teuthology.orchestra.run.smithi117.stdout: "evex": 0, 2024-04-16T17:18:05.674 INFO:teuthology.orchestra.run.smithi117.stdout: "evtrm": 0, 2024-04-16T17:18:05.674 INFO:teuthology.orchestra.run.smithi117.stdout: "ev": 9496, 2024-04-16T17:18:05.674 INFO:teuthology.orchestra.run.smithi117.stdout: "evexg": 0, 2024-04-16T17:18:05.674 INFO:teuthology.orchestra.run.smithi117.stdout: "evexd": 0, 2024-04-16T17:18:05.674 INFO:teuthology.orchestra.run.smithi117.stdout: "segadd": 0, 2024-04-16T17:18:05.675 INFO:teuthology.orchestra.run.smithi117.stdout: "segex": 0, 2024-04-16T17:18:05.675 INFO:teuthology.orchestra.run.smithi117.stdout: "segtrm": 0, 2024-04-16T17:18:05.675 INFO:teuthology.orchestra.run.smithi117.stdout: "seg": 12, 2024-04-16T17:18:05.675 INFO:teuthology.orchestra.run.smithi117.stdout: "segmjr": 2, 2024-04-16T17:18:05.675 INFO:teuthology.orchestra.run.smithi117.stdout: "segexg": 0, 2024-04-16T17:18:05.675 INFO:teuthology.orchestra.run.smithi117.stdout: "segexd": 0, 2024-04-16T17:18:05.675 INFO:teuthology.orchestra.run.smithi117.stdout: "expos": 4194304, 2024-04-16T17:18:05.676 INFO:teuthology.orchestra.run.smithi117.stdout: "wrpos": 17508205, 2024-04-16T17:18:05.676 INFO:teuthology.orchestra.run.smithi117.stdout: "rdpos": 17507631, 2024-04-16T17:18:05.676 INFO:teuthology.orchestra.run.smithi117.stdout: "jlat": { 2024-04-16T17:18:05.676 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-04-16T17:18:05.676 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-04-16T17:18:05.676 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-04-16T17:18:05.676 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-16T17:18:05.676 INFO:teuthology.orchestra.run.smithi117.stdout: "replayed": 9496 2024-04-16T17:18:05.677 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-16T17:18:05.677 INFO:teuthology.orchestra.run.smithi117.stdout: "mds_mem": { 2024-04-16T17:18:05.677 INFO:teuthology.orchestra.run.smithi117.stdout: "ino": 124, 2024-04-16T17:18:05.677 INFO:teuthology.orchestra.run.smithi117.stdout: "ino+": 1878, 2024-04-16T17:18:05.677 INFO:teuthology.orchestra.run.smithi117.stdout: "ino-": 1754, 2024-04-16T17:18:05.677 INFO:teuthology.orchestra.run.smithi117.stdout: "dir": 24, 2024-04-16T17:18:05.677 INFO:teuthology.orchestra.run.smithi117.stdout: "dir+": 28, 2024-04-16T17:18:05.677 INFO:teuthology.orchestra.run.smithi117.stdout: "dir-": 4, 2024-04-16T17:18:05.678 INFO:teuthology.orchestra.run.smithi117.stdout: "dn": 1863, 2024-04-16T17:18:05.678 INFO:teuthology.orchestra.run.smithi117.stdout: "dn+": 1938, 2024-04-16T17:18:05.678 INFO:teuthology.orchestra.run.smithi117.stdout: "dn-": 75, 2024-04-16T17:18:05.678 INFO:teuthology.orchestra.run.smithi117.stdout: "cap": 0, 2024-04-16T17:18:05.678 INFO:teuthology.orchestra.run.smithi117.stdout: "cap+": 0, 2024-04-16T17:18:05.678 INFO:teuthology.orchestra.run.smithi117.stdout: "cap-": 0, 2024-04-16T17:18:05.678 INFO:teuthology.orchestra.run.smithi117.stdout: "rss": 53060, 2024-04-16T17:18:05.679 INFO:teuthology.orchestra.run.smithi117.stdout: "heap": 215416 2024-04-16T17:18:05.679 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-16T17:18:05.679 INFO:teuthology.orchestra.run.smithi117.stdout: "mds_server": { 2024-04-16T17:18:05.679 INFO:teuthology.orchestra.run.smithi117.stdout: "dispatch_client_request": 0, 2024-04-16T17:18:05.679 INFO:teuthology.orchestra.run.smithi117.stdout: "dispatch_server_request": 0, 2024-04-16T17:18:05.679 INFO:teuthology.orchestra.run.smithi117.stdout: "handle_client_request": 0, 2024-04-16T17:18:05.679 INFO:teuthology.orchestra.run.smithi117.stdout: "handle_client_session": 0, 2024-04-16T17:18:05.679 INFO:teuthology.orchestra.run.smithi117.stdout: "handle_peer_request": 0, 2024-04-16T17:18:05.680 INFO:teuthology.orchestra.run.smithi117.stdout: "req_create_latency": { 2024-04-16T17:18:05.680 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-04-16T17:18:05.680 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-04-16T17:18:05.680 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-04-16T17:18:05.680 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-16T17:18:05.680 INFO:teuthology.orchestra.run.smithi117.stdout: "req_getattr_latency": { 2024-04-16T17:18:05.680 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-04-16T17:18:05.680 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-04-16T17:18:05.681 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-04-16T17:18:05.681 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-16T17:18:05.681 INFO:teuthology.orchestra.run.smithi117.stdout: "req_getfilelock_latency": { 2024-04-16T17:18:05.681 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-04-16T17:18:05.681 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-04-16T17:18:05.681 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-04-16T17:18:05.681 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-16T17:18:05.681 INFO:teuthology.orchestra.run.smithi117.stdout: "req_link_latency": { 2024-04-16T17:18:05.682 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-04-16T17:18:05.682 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-04-16T17:18:05.682 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-04-16T17:18:05.682 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-16T17:18:05.682 INFO:teuthology.orchestra.run.smithi117.stdout: "req_lookup_latency": { 2024-04-16T17:18:05.682 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-04-16T17:18:05.682 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-04-16T17:18:05.682 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-04-16T17:18:05.683 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-16T17:18:05.683 INFO:teuthology.orchestra.run.smithi117.stdout: "req_lookuphash_latency": { 2024-04-16T17:18:05.683 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-04-16T17:18:05.683 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-04-16T17:18:05.683 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-04-16T17:18:05.683 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-16T17:18:05.683 INFO:teuthology.orchestra.run.smithi117.stdout: "req_lookupino_latency": { 2024-04-16T17:18:05.684 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-04-16T17:18:05.684 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-04-16T17:18:05.684 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-04-16T17:18:05.684 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-16T17:18:05.684 INFO:teuthology.orchestra.run.smithi117.stdout: "req_lookupname_latency": { 2024-04-16T17:18:05.684 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-04-16T17:18:05.684 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-04-16T17:18:05.684 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-04-16T17:18:05.685 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-16T17:18:05.685 INFO:teuthology.orchestra.run.smithi117.stdout: "req_lookupparent_latency": { 2024-04-16T17:18:05.685 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-04-16T17:18:05.685 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-04-16T17:18:05.685 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-04-16T17:18:05.685 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-16T17:18:05.685 INFO:teuthology.orchestra.run.smithi117.stdout: "req_lookupsnap_latency": { 2024-04-16T17:18:05.685 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-04-16T17:18:05.686 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-04-16T17:18:05.686 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-04-16T17:18:05.686 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-16T17:18:05.686 INFO:teuthology.orchestra.run.smithi117.stdout: "req_lssnap_latency": { 2024-04-16T17:18:05.686 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-04-16T17:18:05.686 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-04-16T17:18:05.686 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-04-16T17:18:05.687 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-16T17:18:05.687 INFO:teuthology.orchestra.run.smithi117.stdout: "req_mkdir_latency": { 2024-04-16T17:18:05.687 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-04-16T17:18:05.688 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-04-16T17:18:05.688 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-04-16T17:18:05.688 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-16T17:18:05.688 INFO:teuthology.orchestra.run.smithi117.stdout: "req_mknod_latency": { 2024-04-16T17:18:05.688 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-04-16T17:18:05.688 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-04-16T17:18:05.688 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-04-16T17:18:05.689 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-16T17:18:05.689 INFO:teuthology.orchestra.run.smithi117.stdout: "req_mksnap_latency": { 2024-04-16T17:18:05.689 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-04-16T17:18:05.689 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-04-16T17:18:05.689 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-04-16T17:18:05.689 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-16T17:18:05.689 INFO:teuthology.orchestra.run.smithi117.stdout: "req_open_latency": { 2024-04-16T17:18:05.689 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-04-16T17:18:05.690 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-04-16T17:18:05.690 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-04-16T17:18:05.690 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-16T17:18:05.690 INFO:teuthology.orchestra.run.smithi117.stdout: "req_readdir_latency": { 2024-04-16T17:18:05.690 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-04-16T17:18:05.690 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-04-16T17:18:05.690 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-04-16T17:18:05.691 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-16T17:18:05.691 INFO:teuthology.orchestra.run.smithi117.stdout: "req_rename_latency": { 2024-04-16T17:18:05.691 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-04-16T17:18:05.691 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-04-16T17:18:05.691 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-04-16T17:18:05.691 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-16T17:18:05.691 INFO:teuthology.orchestra.run.smithi117.stdout: "req_renamesnap_latency": { 2024-04-16T17:18:05.691 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-04-16T17:18:05.692 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-04-16T17:18:05.692 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-04-16T17:18:05.692 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-16T17:18:05.692 INFO:teuthology.orchestra.run.smithi117.stdout: "req_snapdiff_latency": { 2024-04-16T17:18:05.692 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-04-16T17:18:05.692 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-04-16T17:18:05.692 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-04-16T17:18:05.692 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-16T17:18:05.692 INFO:teuthology.orchestra.run.smithi117.stdout: "req_rmdir_latency": { 2024-04-16T17:18:05.693 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-04-16T17:18:05.693 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-04-16T17:18:05.693 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-04-16T17:18:05.693 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-16T17:18:05.693 INFO:teuthology.orchestra.run.smithi117.stdout: "req_rmsnap_latency": { 2024-04-16T17:18:05.693 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-04-16T17:18:05.693 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-04-16T17:18:05.693 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-04-16T17:18:05.694 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-16T17:18:05.694 INFO:teuthology.orchestra.run.smithi117.stdout: "req_rmxattr_latency": { 2024-04-16T17:18:05.694 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-04-16T17:18:05.694 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-04-16T17:18:05.694 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-04-16T17:18:05.694 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-16T17:18:05.694 INFO:teuthology.orchestra.run.smithi117.stdout: "req_setattr_latency": { 2024-04-16T17:18:05.695 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-04-16T17:18:05.695 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-04-16T17:18:05.695 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-04-16T17:18:05.695 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-16T17:18:05.695 INFO:teuthology.orchestra.run.smithi117.stdout: "req_setdirlayout_latency": { 2024-04-16T17:18:05.695 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-04-16T17:18:05.695 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-04-16T17:18:05.695 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-04-16T17:18:05.696 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-16T17:18:05.696 INFO:teuthology.orchestra.run.smithi117.stdout: "req_setfilelock_latency": { 2024-04-16T17:18:05.696 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-04-16T17:18:05.696 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-04-16T17:18:05.696 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-04-16T17:18:05.696 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-16T17:18:05.696 INFO:teuthology.orchestra.run.smithi117.stdout: "req_setlayout_latency": { 2024-04-16T17:18:05.696 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-04-16T17:18:05.697 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-04-16T17:18:05.697 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-04-16T17:18:05.697 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-16T17:18:05.697 INFO:teuthology.orchestra.run.smithi117.stdout: "req_setxattr_latency": { 2024-04-16T17:18:05.697 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-04-16T17:18:05.697 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-04-16T17:18:05.697 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-04-16T17:18:05.697 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-16T17:18:05.698 INFO:teuthology.orchestra.run.smithi117.stdout: "req_symlink_latency": { 2024-04-16T17:18:05.698 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-04-16T17:18:05.698 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-04-16T17:18:05.698 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-04-16T17:18:05.698 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-16T17:18:05.698 INFO:teuthology.orchestra.run.smithi117.stdout: "req_unlink_latency": { 2024-04-16T17:18:05.698 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-04-16T17:18:05.699 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-04-16T17:18:05.699 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-04-16T17:18:05.699 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-16T17:18:05.699 INFO:teuthology.orchestra.run.smithi117.stdout: "cap_revoke_eviction": 0, 2024-04-16T17:18:05.699 INFO:teuthology.orchestra.run.smithi117.stdout: "cap_acquisition_throttle": 0, 2024-04-16T17:18:05.699 INFO:teuthology.orchestra.run.smithi117.stdout: "req_getvxattr_latency": { 2024-04-16T17:18:05.699 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-04-16T17:18:05.699 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-04-16T17:18:05.700 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-04-16T17:18:05.700 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-04-16T17:18:05.700 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-16T17:18:05.700 INFO:teuthology.orchestra.run.smithi117.stdout: "mds_sessions": { 2024-04-16T17:18:05.700 INFO:teuthology.orchestra.run.smithi117.stdout: "session_count": 2, 2024-04-16T17:18:05.700 INFO:teuthology.orchestra.run.smithi117.stdout: "session_add": 5, 2024-04-16T17:18:05.700 INFO:teuthology.orchestra.run.smithi117.stdout: "session_remove": 3, 2024-04-16T17:18:05.700 INFO:teuthology.orchestra.run.smithi117.stdout: "sessions_open": 4, 2024-04-16T17:18:05.700 INFO:teuthology.orchestra.run.smithi117.stdout: "sessions_stale": 0, 2024-04-16T17:18:05.701 INFO:teuthology.orchestra.run.smithi117.stdout: "total_load": 0, 2024-04-16T17:18:05.701 INFO:teuthology.orchestra.run.smithi117.stdout: "average_load": 0, 2024-04-16T17:18:05.701 INFO:teuthology.orchestra.run.smithi117.stdout: "avg_session_uptime": 4122, 2024-04-16T17:18:05.701 INFO:teuthology.orchestra.run.smithi117.stdout: "mdthresh_evicted": 0 2024-04-16T17:18:05.701 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-16T17:18:05.701 INFO:teuthology.orchestra.run.smithi117.stdout: "mempool": { 2024-04-16T17:18:05.701 INFO:teuthology.orchestra.run.smithi117.stdout: "bloom_filter_bytes": 0, 2024-04-16T17:18:05.701 INFO:teuthology.orchestra.run.smithi117.stdout: "bloom_filter_items": 0, 2024-04-16T17:18:05.702 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_alloc_bytes": 0, 2024-04-16T17:18:05.702 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_alloc_items": 0, 2024-04-16T17:18:05.702 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_data_bytes": 0, 2024-04-16T17:18:05.702 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_data_items": 0, 2024-04-16T17:18:05.702 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-16T17:18:05.702 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_onode_items": 0, 2024-04-16T17:18:05.702 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-16T17:18:05.702 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_meta_items": 0, 2024-04-16T17:18:05.703 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_other_bytes": 0, 2024-04-16T17:18:05.703 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_other_items": 0, 2024-04-16T17:18:05.703 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-16T17:18:05.703 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_buffer_items": 0, 2024-04-16T17:18:05.703 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_extent_bytes": 0, 2024-04-16T17:18:05.703 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_extent_items": 0, 2024-04-16T17:18:05.703 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_blob_bytes": 0, 2024-04-16T17:18:05.703 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_blob_items": 0, 2024-04-16T17:18:05.704 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-16T17:18:05.704 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_shared_blob_items": 0, 2024-04-16T17:18:05.704 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-16T17:18:05.704 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_inline_bl_items": 0, 2024-04-16T17:18:05.704 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_fsck_bytes": 0, 2024-04-16T17:18:05.704 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_fsck_items": 0, 2024-04-16T17:18:05.704 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_txc_bytes": 0, 2024-04-16T17:18:05.704 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_txc_items": 0, 2024-04-16T17:18:05.705 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-16T17:18:05.705 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_writing_deferred_items": 0, 2024-04-16T17:18:05.705 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_writing_bytes": 0, 2024-04-16T17:18:05.705 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_writing_items": 0, 2024-04-16T17:18:05.705 INFO:teuthology.orchestra.run.smithi117.stdout: "bluefs_bytes": 0, 2024-04-16T17:18:05.705 INFO:teuthology.orchestra.run.smithi117.stdout: "bluefs_items": 0, 2024-04-16T17:18:05.705 INFO:teuthology.orchestra.run.smithi117.stdout: "bluefs_file_reader_bytes": 0, 2024-04-16T17:18:05.705 INFO:teuthology.orchestra.run.smithi117.stdout: "bluefs_file_reader_items": 0, 2024-04-16T17:18:05.706 INFO:teuthology.orchestra.run.smithi117.stdout: "bluefs_file_writer_bytes": 0, 2024-04-16T17:18:05.706 INFO:teuthology.orchestra.run.smithi117.stdout: "bluefs_file_writer_items": 0, 2024-04-16T17:18:05.706 INFO:teuthology.orchestra.run.smithi117.stdout: "buffer_anon_bytes": 212196, 2024-04-16T17:18:05.706 INFO:teuthology.orchestra.run.smithi117.stdout: "buffer_anon_items": 181, 2024-04-16T17:18:05.706 INFO:teuthology.orchestra.run.smithi117.stdout: "buffer_meta_bytes": 0, 2024-04-16T17:18:05.706 INFO:teuthology.orchestra.run.smithi117.stdout: "buffer_meta_items": 0, 2024-04-16T17:18:05.706 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_bytes": 0, 2024-04-16T17:18:05.706 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_items": 0, 2024-04-16T17:18:05.707 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_mapbl_bytes": 0, 2024-04-16T17:18:05.707 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_mapbl_items": 0, 2024-04-16T17:18:05.707 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_pglog_bytes": 0, 2024-04-16T17:18:05.707 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_pglog_items": 0, 2024-04-16T17:18:05.707 INFO:teuthology.orchestra.run.smithi117.stdout: "osdmap_bytes": 9512, 2024-04-16T17:18:05.707 INFO:teuthology.orchestra.run.smithi117.stdout: "osdmap_items": 182, 2024-04-16T17:18:05.707 INFO:teuthology.orchestra.run.smithi117.stdout: "osdmap_mapping_bytes": 0, 2024-04-16T17:18:05.707 INFO:teuthology.orchestra.run.smithi117.stdout: "osdmap_mapping_items": 0, 2024-04-16T17:18:05.708 INFO:teuthology.orchestra.run.smithi117.stdout: "pgmap_bytes": 0, 2024-04-16T17:18:05.708 INFO:teuthology.orchestra.run.smithi117.stdout: "pgmap_items": 0, 2024-04-16T17:18:05.708 INFO:teuthology.orchestra.run.smithi117.stdout: "mds_co_bytes": 1662434, 2024-04-16T17:18:05.708 INFO:teuthology.orchestra.run.smithi117.stdout: "mds_co_items": 10357, 2024-04-16T17:18:05.708 INFO:teuthology.orchestra.run.smithi117.stdout: "unittest_1_bytes": 0, 2024-04-16T17:18:05.708 INFO:teuthology.orchestra.run.smithi117.stdout: "unittest_1_items": 0, 2024-04-16T17:18:05.708 INFO:teuthology.orchestra.run.smithi117.stdout: "unittest_2_bytes": 0, 2024-04-16T17:18:05.708 INFO:teuthology.orchestra.run.smithi117.stdout: "unittest_2_items": 0 2024-04-16T17:18:05.708 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-16T17:18:05.709 INFO:teuthology.orchestra.run.smithi117.stdout: "objecter": { 2024-04-16T17:18:05.709 INFO:teuthology.orchestra.run.smithi117.stdout: "op_active": 0, 2024-04-16T17:18:05.709 INFO:teuthology.orchestra.run.smithi117.stdout: "op_laggy": 0, 2024-04-16T17:18:05.709 INFO:teuthology.orchestra.run.smithi117.stdout: "op_send": 2665, 2024-04-16T17:18:05.709 INFO:teuthology.orchestra.run.smithi117.stdout: "op_send_bytes": 21, 2024-04-16T17:18:05.709 INFO:teuthology.orchestra.run.smithi117.stdout: "op_resend": 3, 2024-04-16T17:18:05.709 INFO:teuthology.orchestra.run.smithi117.stdout: "op_reply": 2665, 2024-04-16T17:18:05.709 INFO:teuthology.orchestra.run.smithi117.stdout: "op_latency": { 2024-04-16T17:18:05.710 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 2665, 2024-04-16T17:18:05.710 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 1.988963027, 2024-04-16T17:18:05.710 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000746327 2024-04-16T17:18:05.710 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-16T17:18:05.710 INFO:teuthology.orchestra.run.smithi117.stdout: "op_inflight": 0, 2024-04-16T17:18:05.710 INFO:teuthology.orchestra.run.smithi117.stdout: "oplen_avg": { 2024-04-16T17:18:05.710 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 2665, 2024-04-16T17:18:05.710 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 2666 2024-04-16T17:18:05.711 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-16T17:18:05.711 INFO:teuthology.orchestra.run.smithi117.stdout: "op": 2665, 2024-04-16T17:18:05.711 INFO:teuthology.orchestra.run.smithi117.stdout: "op_r": 2665, 2024-04-16T17:18:05.711 INFO:teuthology.orchestra.run.smithi117.stdout: "op_w": 0, 2024-04-16T17:18:05.711 INFO:teuthology.orchestra.run.smithi117.stdout: "op_rmw": 0, 2024-04-16T17:18:05.711 INFO:teuthology.orchestra.run.smithi117.stdout: "op_pg": 0, 2024-04-16T17:18:05.711 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_stat": 1613, 2024-04-16T17:18:05.711 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_create": 0, 2024-04-16T17:18:05.712 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_read": 1050, 2024-04-16T17:18:05.712 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_write": 0, 2024-04-16T17:18:05.712 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_writefull": 0, 2024-04-16T17:18:05.712 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_writesame": 0, 2024-04-16T17:18:05.712 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_append": 0, 2024-04-16T17:18:05.712 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_zero": 0, 2024-04-16T17:18:05.712 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_truncate": 0, 2024-04-16T17:18:05.712 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_delete": 0, 2024-04-16T17:18:05.712 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_mapext": 0, 2024-04-16T17:18:05.713 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_sparse_read": 0, 2024-04-16T17:18:05.713 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_clonerange": 0, 2024-04-16T17:18:05.713 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_getxattr": 1, 2024-04-16T17:18:05.713 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_setxattr": 0, 2024-04-16T17:18:05.713 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_cmpxattr": 0, 2024-04-16T17:18:05.713 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_rmxattr": 0, 2024-04-16T17:18:05.713 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_resetxattrs": 0, 2024-04-16T17:18:05.713 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_call": 0, 2024-04-16T17:18:05.714 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_watch": 0, 2024-04-16T17:18:05.714 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_notify": 0, 2024-04-16T17:18:05.714 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_src_cmpxattr": 0, 2024-04-16T17:18:05.714 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_pgls": 0, 2024-04-16T17:18:05.714 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_pgls_filter": 0, 2024-04-16T17:18:05.714 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_other": 0, 2024-04-16T17:18:05.714 INFO:teuthology.orchestra.run.smithi117.stdout: "linger_active": 0, 2024-04-16T17:18:05.714 INFO:teuthology.orchestra.run.smithi117.stdout: "linger_send": 0, 2024-04-16T17:18:05.715 INFO:teuthology.orchestra.run.smithi117.stdout: "linger_resend": 0, 2024-04-16T17:18:05.715 INFO:teuthology.orchestra.run.smithi117.stdout: "linger_ping": 0, 2024-04-16T17:18:05.715 INFO:teuthology.orchestra.run.smithi117.stdout: "poolop_active": 0, 2024-04-16T17:18:05.715 INFO:teuthology.orchestra.run.smithi117.stdout: "poolop_send": 0, 2024-04-16T17:18:05.715 INFO:teuthology.orchestra.run.smithi117.stdout: "poolop_resend": 0, 2024-04-16T17:18:05.715 INFO:teuthology.orchestra.run.smithi117.stdout: "poolstat_active": 0, 2024-04-16T17:18:05.715 INFO:teuthology.orchestra.run.smithi117.stdout: "poolstat_send": 0, 2024-04-16T17:18:05.715 INFO:teuthology.orchestra.run.smithi117.stdout: "poolstat_resend": 0, 2024-04-16T17:18:05.716 INFO:teuthology.orchestra.run.smithi117.stdout: "statfs_active": 0, 2024-04-16T17:18:05.716 INFO:teuthology.orchestra.run.smithi117.stdout: "statfs_send": 0, 2024-04-16T17:18:05.716 INFO:teuthology.orchestra.run.smithi117.stdout: "statfs_resend": 0, 2024-04-16T17:18:05.716 INFO:teuthology.orchestra.run.smithi117.stdout: "command_active": 0, 2024-04-16T17:18:05.716 INFO:teuthology.orchestra.run.smithi117.stdout: "command_send": 0, 2024-04-16T17:18:05.716 INFO:teuthology.orchestra.run.smithi117.stdout: "command_resend": 0, 2024-04-16T17:18:05.716 INFO:teuthology.orchestra.run.smithi117.stdout: "map_epoch": 106, 2024-04-16T17:18:05.716 INFO:teuthology.orchestra.run.smithi117.stdout: "map_full": 0, 2024-04-16T17:18:05.717 INFO:teuthology.orchestra.run.smithi117.stdout: "map_inc": 16, 2024-04-16T17:18:05.717 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_sessions": 7, 2024-04-16T17:18:05.717 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_session_open": 7, 2024-04-16T17:18:05.717 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_session_close": 0, 2024-04-16T17:18:05.717 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_laggy": 0, 2024-04-16T17:18:05.717 INFO:teuthology.orchestra.run.smithi117.stdout: "omap_wr": 0, 2024-04-16T17:18:05.717 INFO:teuthology.orchestra.run.smithi117.stdout: "omap_rd": 2, 2024-04-16T17:18:05.717 INFO:teuthology.orchestra.run.smithi117.stdout: "omap_del": 0 2024-04-16T17:18:05.717 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-16T17:18:05.718 INFO:teuthology.orchestra.run.smithi117.stdout: "oft": { 2024-04-16T17:18:05.718 INFO:teuthology.orchestra.run.smithi117.stdout: "omap_total_objs": 0, 2024-04-16T17:18:05.718 INFO:teuthology.orchestra.run.smithi117.stdout: "omap_total_kv_pairs": 0, 2024-04-16T17:18:05.718 INFO:teuthology.orchestra.run.smithi117.stdout: "omap_total_updates": 0, 2024-04-16T17:18:05.718 INFO:teuthology.orchestra.run.smithi117.stdout: "omap_total_removes": 0 2024-04-16T17:18:05.718 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-16T17:18:05.718 INFO:teuthology.orchestra.run.smithi117.stdout: "osd-slow-ops": { 2024-04-16T17:18:05.718 INFO:teuthology.orchestra.run.smithi117.stdout: "slow_ops_count": 0 2024-04-16T17:18:05.718 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-16T17:18:05.719 INFO:teuthology.orchestra.run.smithi117.stdout: "purge_queue": { 2024-04-16T17:18:05.719 INFO:teuthology.orchestra.run.smithi117.stdout: "pq_executing_ops": 0, 2024-04-16T17:18:05.719 INFO:teuthology.orchestra.run.smithi117.stdout: "pq_executing_ops_high_water": 0, 2024-04-16T17:18:05.719 INFO:teuthology.orchestra.run.smithi117.stdout: "pq_executing": 0, 2024-04-16T17:18:05.719 INFO:teuthology.orchestra.run.smithi117.stdout: "pq_executing_high_water": 0, 2024-04-16T17:18:05.719 INFO:teuthology.orchestra.run.smithi117.stdout: "pq_executed_ops": 0, 2024-04-16T17:18:05.719 INFO:teuthology.orchestra.run.smithi117.stdout: "pq_executed": 0, 2024-04-16T17:18:05.719 INFO:teuthology.orchestra.run.smithi117.stdout: "pq_item_in_journal": 0 2024-04-16T17:18:05.720 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-16T17:18:05.720 INFO:teuthology.orchestra.run.smithi117.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-16T17:18:05.720 INFO:teuthology.orchestra.run.smithi117.stdout: "val": 0, 2024-04-16T17:18:05.720 INFO:teuthology.orchestra.run.smithi117.stdout: "max": 104857600, 2024-04-16T17:18:05.720 INFO:teuthology.orchestra.run.smithi117.stdout: "get_started": 0, 2024-04-16T17:18:05.720 INFO:teuthology.orchestra.run.smithi117.stdout: "get": 2961, 2024-04-16T17:18:05.720 INFO:teuthology.orchestra.run.smithi117.stdout: "get_sum": 15491284, 2024-04-16T17:18:05.720 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_fail": 0, 2024-04-16T17:18:05.721 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_success": 2961, 2024-04-16T17:18:05.721 INFO:teuthology.orchestra.run.smithi117.stdout: "take": 0, 2024-04-16T17:18:05.721 INFO:teuthology.orchestra.run.smithi117.stdout: "take_sum": 0, 2024-04-16T17:18:05.721 INFO:teuthology.orchestra.run.smithi117.stdout: "put": 2961, 2024-04-16T17:18:05.721 INFO:teuthology.orchestra.run.smithi117.stdout: "put_sum": 15491284, 2024-04-16T17:18:05.721 INFO:teuthology.orchestra.run.smithi117.stdout: "wait": { 2024-04-16T17:18:05.721 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-04-16T17:18:05.721 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-04-16T17:18:05.722 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-04-16T17:18:05.722 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-04-16T17:18:05.722 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-16T17:18:05.722 INFO:teuthology.orchestra.run.smithi117.stdout: "throttle-objecter_bytes": { 2024-04-16T17:18:05.722 INFO:teuthology.orchestra.run.smithi117.stdout: "val": 0, 2024-04-16T17:18:05.722 INFO:teuthology.orchestra.run.smithi117.stdout: "max": 104857600, 2024-04-16T17:18:05.722 INFO:teuthology.orchestra.run.smithi117.stdout: "get_started": 0, 2024-04-16T17:18:05.722 INFO:teuthology.orchestra.run.smithi117.stdout: "get": 0, 2024-04-16T17:18:05.723 INFO:teuthology.orchestra.run.smithi117.stdout: "get_sum": 0, 2024-04-16T17:18:05.723 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_fail": 0, 2024-04-16T17:18:05.723 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_success": 0, 2024-04-16T17:18:05.723 INFO:teuthology.orchestra.run.smithi117.stdout: "take": 2665, 2024-04-16T17:18:05.723 INFO:teuthology.orchestra.run.smithi117.stdout: "take_sum": 13313906, 2024-04-16T17:18:05.723 INFO:teuthology.orchestra.run.smithi117.stdout: "put": 2665, 2024-04-16T17:18:05.723 INFO:teuthology.orchestra.run.smithi117.stdout: "put_sum": 13313906, 2024-04-16T17:18:05.723 INFO:teuthology.orchestra.run.smithi117.stdout: "wait": { 2024-04-16T17:18:05.723 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-04-16T17:18:05.724 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-04-16T17:18:05.724 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-04-16T17:18:05.724 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-04-16T17:18:05.724 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-16T17:18:05.724 INFO:teuthology.orchestra.run.smithi117.stdout: "throttle-objecter_ops": { 2024-04-16T17:18:05.724 INFO:teuthology.orchestra.run.smithi117.stdout: "val": 0, 2024-04-16T17:18:05.724 INFO:teuthology.orchestra.run.smithi117.stdout: "max": 1024, 2024-04-16T17:18:05.724 INFO:teuthology.orchestra.run.smithi117.stdout: "get_started": 0, 2024-04-16T17:18:05.725 INFO:teuthology.orchestra.run.smithi117.stdout: "get": 0, 2024-04-16T17:18:05.725 INFO:teuthology.orchestra.run.smithi117.stdout: "get_sum": 0, 2024-04-16T17:18:05.725 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_fail": 0, 2024-04-16T17:18:05.725 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_success": 0, 2024-04-16T17:18:05.725 INFO:teuthology.orchestra.run.smithi117.stdout: "take": 2665, 2024-04-16T17:18:05.725 INFO:teuthology.orchestra.run.smithi117.stdout: "take_sum": 2665, 2024-04-16T17:18:05.725 INFO:teuthology.orchestra.run.smithi117.stdout: "put": 2665, 2024-04-16T17:18:05.725 INFO:teuthology.orchestra.run.smithi117.stdout: "put_sum": 2665, 2024-04-16T17:18:05.726 INFO:teuthology.orchestra.run.smithi117.stdout: "wait": { 2024-04-16T17:18:05.726 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-04-16T17:18:05.726 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-04-16T17:18:05.726 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-04-16T17:18:05.726 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-04-16T17:18:05.726 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-16T17:18:05.726 INFO:teuthology.orchestra.run.smithi117.stdout: "throttle-write_buf_throttle": { 2024-04-16T17:18:05.726 INFO:teuthology.orchestra.run.smithi117.stdout: "val": 0, 2024-04-16T17:18:05.726 INFO:teuthology.orchestra.run.smithi117.stdout: "max": 3758096384, 2024-04-16T17:18:05.727 INFO:teuthology.orchestra.run.smithi117.stdout: "get_started": 0, 2024-04-16T17:18:05.727 INFO:teuthology.orchestra.run.smithi117.stdout: "get": 0, 2024-04-16T17:18:05.727 INFO:teuthology.orchestra.run.smithi117.stdout: "get_sum": 0, 2024-04-16T17:18:05.727 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_fail": 0, 2024-04-16T17:18:05.727 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_success": 0, 2024-04-16T17:18:05.727 INFO:teuthology.orchestra.run.smithi117.stdout: "take": 0, 2024-04-16T17:18:05.727 INFO:teuthology.orchestra.run.smithi117.stdout: "take_sum": 0, 2024-04-16T17:18:05.727 INFO:teuthology.orchestra.run.smithi117.stdout: "put": 0, 2024-04-16T17:18:05.728 INFO:teuthology.orchestra.run.smithi117.stdout: "put_sum": 0, 2024-04-16T17:18:05.728 INFO:teuthology.orchestra.run.smithi117.stdout: "wait": { 2024-04-16T17:18:05.728 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-04-16T17:18:05.728 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-04-16T17:18:05.728 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-04-16T17:18:05.728 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-04-16T17:18:05.728 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-16T17:18:05.728 INFO:teuthology.orchestra.run.smithi117.stdout: "throttle-write_buf_throttle-0x55b6ba30e6e0": { 2024-04-16T17:18:05.728 INFO:teuthology.orchestra.run.smithi117.stdout: "val": 0, 2024-04-16T17:18:05.729 INFO:teuthology.orchestra.run.smithi117.stdout: "max": 3758096384, 2024-04-16T17:18:05.729 INFO:teuthology.orchestra.run.smithi117.stdout: "get_started": 0, 2024-04-16T17:18:05.729 INFO:teuthology.orchestra.run.smithi117.stdout: "get": 0, 2024-04-16T17:18:05.729 INFO:teuthology.orchestra.run.smithi117.stdout: "get_sum": 0, 2024-04-16T17:18:05.729 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_fail": 0, 2024-04-16T17:18:05.729 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_success": 0, 2024-04-16T17:18:05.729 INFO:teuthology.orchestra.run.smithi117.stdout: "take": 0, 2024-04-16T17:18:05.729 INFO:teuthology.orchestra.run.smithi117.stdout: "take_sum": 0, 2024-04-16T17:18:05.730 INFO:teuthology.orchestra.run.smithi117.stdout: "put": 0, 2024-04-16T17:18:05.730 INFO:teuthology.orchestra.run.smithi117.stdout: "put_sum": 0, 2024-04-16T17:18:05.730 INFO:teuthology.orchestra.run.smithi117.stdout: "wait": { 2024-04-16T17:18:05.730 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-04-16T17:18:05.730 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-04-16T17:18:05.730 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-04-16T17:18:05.730 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-04-16T17:18:05.730 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-04-16T17:18:05.730 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-04-16T17:18:06.442 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2024-04-16T17:18:06.442 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2024-04-16T17:18:06.443 INFO:tasks.check_counter:Daemon mds.f mds.root_rsnaps=0 2024-04-16T17:18:06.443 INFO:tasks.check_counter:Daemon mds.f mds_server.req_mksnap_latency.avgcount=0 2024-04-16T17:18:06.443 INFO:tasks.check_counter:Daemon mds.f mds_server.req_rmsnap_latency.avgcount=0 2024-04-16T17:18:06.443 DEBUG:tasks.check_counter:Getting stats from i 2024-04-16T17:18:06.443 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45 shell --fsid b1fecd38-fc11-11ee-bc90-c7b262605968 -- ceph daemon mds.i perf dump 2024-04-16T17:18:06.684 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/mon.c/config 2024-04-16T17:18:06.804 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:18:06 smithi117 ceph-mon[36730]: pgmap v354: 129 pgs: 129 active+clean; 55 MiB data, 874 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-04-16T17:18:06.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:18:06 smithi104 ceph-mon[36457]: pgmap v354: 129 pgs: 129 active+clean; 55 MiB data, 874 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-04-16T17:18:06.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:18:06 smithi045 ceph-mon[31676]: pgmap v354: 129 pgs: 129 active+clean; 55 MiB data, 874 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-04-16T17:18:08.389 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-04-16T17:18:08.389 INFO:teuthology.orchestra.run.smithi117.stdout: "AsyncMessenger::Worker-0": { 2024-04-16T17:18:08.389 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_messages": 21778, 2024-04-16T17:18:08.389 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_messages": 20607, 2024-04-16T17:18:08.389 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_bytes": 8297553, 2024-04-16T17:18:08.390 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_bytes": 9718696, 2024-04-16T17:18:08.390 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_created_connections": 10, 2024-04-16T17:18:08.390 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_active_connections": 5, 2024-04-16T17:18:08.390 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_total_time": 1.230276046, 2024-04-16T17:18:08.390 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_send_time": 0.500807299, 2024-04-16T17:18:08.390 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_recv_time": 0.831751368, 2024-04-16T17:18:08.390 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_fast_dispatch_time": 0.065764580, 2024-04-16T17:18:08.391 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_messages_queue_lat": { 2024-04-16T17:18:08.391 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 20608, 2024-04-16T17:18:08.391 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.414486793, 2024-04-16T17:18:08.391 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000020112 2024-04-16T17:18:08.391 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-16T17:18:08.391 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_handle_ack_lat": { 2024-04-16T17:18:08.391 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 37322, 2024-04-16T17:18:08.391 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.004604709, 2024-04-16T17:18:08.392 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000123 2024-04-16T17:18:08.392 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-16T17:18:08.392 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_encrypted_bytes": 1730128, 2024-04-16T17:18:08.392 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_encrypted_bytes": 187696 2024-04-16T17:18:08.392 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-16T17:18:08.392 INFO:teuthology.orchestra.run.smithi117.stdout: "AsyncMessenger::Worker-1": { 2024-04-16T17:18:08.392 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_messages": 4703, 2024-04-16T17:18:08.393 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_messages": 5257, 2024-04-16T17:18:08.393 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_bytes": 1402362, 2024-04-16T17:18:08.393 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_bytes": 7505791, 2024-04-16T17:18:08.393 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_created_connections": 46, 2024-04-16T17:18:08.393 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_active_connections": 7, 2024-04-16T17:18:08.393 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_total_time": 0.337615682, 2024-04-16T17:18:08.393 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_send_time": 0.125900631, 2024-04-16T17:18:08.394 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_recv_time": 0.141662294, 2024-04-16T17:18:08.394 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_fast_dispatch_time": 0.069679438, 2024-04-16T17:18:08.394 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_messages_queue_lat": { 2024-04-16T17:18:08.394 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 5257, 2024-04-16T17:18:08.394 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.157922521, 2024-04-16T17:18:08.394 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000030040 2024-04-16T17:18:08.394 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-16T17:18:08.395 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_handle_ack_lat": { 2024-04-16T17:18:08.395 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 3949, 2024-04-16T17:18:08.395 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000568159, 2024-04-16T17:18:08.395 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000143 2024-04-16T17:18:08.395 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-16T17:18:08.395 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_encrypted_bytes": 7168, 2024-04-16T17:18:08.395 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_encrypted_bytes": 155824 2024-04-16T17:18:08.396 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-16T17:18:08.396 INFO:teuthology.orchestra.run.smithi117.stdout: "AsyncMessenger::Worker-2": { 2024-04-16T17:18:08.396 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_messages": 16863, 2024-04-16T17:18:08.396 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_messages": 26328, 2024-04-16T17:18:08.396 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_bytes": 5699664, 2024-04-16T17:18:08.396 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_bytes": 22763128, 2024-04-16T17:18:08.396 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_created_connections": 20, 2024-04-16T17:18:08.396 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_active_connections": 7, 2024-04-16T17:18:08.397 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_total_time": 1.368647398, 2024-04-16T17:18:08.397 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_send_time": 0.548892580, 2024-04-16T17:18:08.397 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_recv_time": 0.936318678, 2024-04-16T17:18:08.397 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_fast_dispatch_time": 0.066489359, 2024-04-16T17:18:08.397 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_messages_queue_lat": { 2024-04-16T17:18:08.397 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 26322, 2024-04-16T17:18:08.397 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.574118965, 2024-04-16T17:18:08.398 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000021811 2024-04-16T17:18:08.398 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-16T17:18:08.398 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_handle_ack_lat": { 2024-04-16T17:18:08.398 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 32120, 2024-04-16T17:18:08.398 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.007457456, 2024-04-16T17:18:08.398 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000232 2024-04-16T17:18:08.399 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-16T17:18:08.399 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_encrypted_bytes": 1088, 2024-04-16T17:18:08.399 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_encrypted_bytes": 334752 2024-04-16T17:18:08.399 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-16T17:18:08.399 INFO:teuthology.orchestra.run.smithi117.stdout: "cct": { 2024-04-16T17:18:08.399 INFO:teuthology.orchestra.run.smithi117.stdout: "total_workers": 1, 2024-04-16T17:18:08.399 INFO:teuthology.orchestra.run.smithi117.stdout: "unhealthy_workers": 0 2024-04-16T17:18:08.399 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-16T17:18:08.400 INFO:teuthology.orchestra.run.smithi117.stdout: "finisher-MDSRank": { 2024-04-16T17:18:08.400 INFO:teuthology.orchestra.run.smithi117.stdout: "queue_len": 0, 2024-04-16T17:18:08.400 INFO:teuthology.orchestra.run.smithi117.stdout: "complete_latency": { 2024-04-16T17:18:08.400 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 6005, 2024-04-16T17:18:08.400 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 2.822670862, 2024-04-16T17:18:08.400 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000470053 2024-04-16T17:18:08.400 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-04-16T17:18:08.401 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-16T17:18:08.401 INFO:teuthology.orchestra.run.smithi117.stdout: "finisher-PurgeQueue": { 2024-04-16T17:18:08.401 INFO:teuthology.orchestra.run.smithi117.stdout: "queue_len": 0, 2024-04-16T17:18:08.401 INFO:teuthology.orchestra.run.smithi117.stdout: "complete_latency": { 2024-04-16T17:18:08.401 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 3434, 2024-04-16T17:18:08.401 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.189457159, 2024-04-16T17:18:08.401 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000055170 2024-04-16T17:18:08.401 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-04-16T17:18:08.402 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-16T17:18:08.402 INFO:teuthology.orchestra.run.smithi117.stdout: "mds": { 2024-04-16T17:18:08.402 INFO:teuthology.orchestra.run.smithi117.stdout: "request": 10268, 2024-04-16T17:18:08.402 INFO:teuthology.orchestra.run.smithi117.stdout: "reply": 10111, 2024-04-16T17:18:08.402 INFO:teuthology.orchestra.run.smithi117.stdout: "reply_latency": { 2024-04-16T17:18:08.402 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 10111, 2024-04-16T17:18:08.402 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 43.080637913, 2024-04-16T17:18:08.402 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.004260769 2024-04-16T17:18:08.403 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-16T17:18:08.403 INFO:teuthology.orchestra.run.smithi117.stdout: "slow_reply": 0, 2024-04-16T17:18:08.403 INFO:teuthology.orchestra.run.smithi117.stdout: "forward": 157, 2024-04-16T17:18:08.403 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_fetch_complete": 4, 2024-04-16T17:18:08.403 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_fetch_keys": 0, 2024-04-16T17:18:08.403 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_commit": 11, 2024-04-16T17:18:08.403 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_split": 0, 2024-04-16T17:18:08.403 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_merge": 0, 2024-04-16T17:18:08.404 INFO:teuthology.orchestra.run.smithi117.stdout: "inodes": 190, 2024-04-16T17:18:08.404 INFO:teuthology.orchestra.run.smithi117.stdout: "inodes_top": 31, 2024-04-16T17:18:08.404 INFO:teuthology.orchestra.run.smithi117.stdout: "inodes_bottom": 116, 2024-04-16T17:18:08.404 INFO:teuthology.orchestra.run.smithi117.stdout: "inodes_pin_tail": 43, 2024-04-16T17:18:08.404 INFO:teuthology.orchestra.run.smithi117.stdout: "inodes_pinned": 146, 2024-04-16T17:18:08.404 INFO:teuthology.orchestra.run.smithi117.stdout: "inodes_expired": 151, 2024-04-16T17:18:08.404 INFO:teuthology.orchestra.run.smithi117.stdout: "inodes_with_caps": 7, 2024-04-16T17:18:08.404 INFO:teuthology.orchestra.run.smithi117.stdout: "caps": 7, 2024-04-16T17:18:08.405 INFO:teuthology.orchestra.run.smithi117.stdout: "subtrees": 9, 2024-04-16T17:18:08.405 INFO:teuthology.orchestra.run.smithi117.stdout: "traverse": 23000, 2024-04-16T17:18:08.405 INFO:teuthology.orchestra.run.smithi117.stdout: "traverse_hit": 10325, 2024-04-16T17:18:08.405 INFO:teuthology.orchestra.run.smithi117.stdout: "traverse_forward": 151, 2024-04-16T17:18:08.405 INFO:teuthology.orchestra.run.smithi117.stdout: "traverse_discover": 22, 2024-04-16T17:18:08.405 INFO:teuthology.orchestra.run.smithi117.stdout: "traverse_dir_fetch": 0, 2024-04-16T17:18:08.405 INFO:teuthology.orchestra.run.smithi117.stdout: "traverse_remote_ino": 0, 2024-04-16T17:18:08.406 INFO:teuthology.orchestra.run.smithi117.stdout: "traverse_lock": 264, 2024-04-16T17:18:08.406 INFO:teuthology.orchestra.run.smithi117.stdout: "load_cent": 0, 2024-04-16T17:18:08.406 INFO:teuthology.orchestra.run.smithi117.stdout: "q": 0, 2024-04-16T17:18:08.406 INFO:teuthology.orchestra.run.smithi117.stdout: "exported": 4, 2024-04-16T17:18:08.406 INFO:teuthology.orchestra.run.smithi117.stdout: "exported_inodes": 81, 2024-04-16T17:18:08.406 INFO:teuthology.orchestra.run.smithi117.stdout: "imported": 7, 2024-04-16T17:18:08.406 INFO:teuthology.orchestra.run.smithi117.stdout: "imported_inodes": 152, 2024-04-16T17:18:08.406 INFO:teuthology.orchestra.run.smithi117.stdout: "openino_dir_fetch": 0, 2024-04-16T17:18:08.407 INFO:teuthology.orchestra.run.smithi117.stdout: "openino_backtrace_fetch": 268, 2024-04-16T17:18:08.407 INFO:teuthology.orchestra.run.smithi117.stdout: "openino_peer_discover": 0, 2024-04-16T17:18:08.407 INFO:teuthology.orchestra.run.smithi117.stdout: "root_rfiles": 68, 2024-04-16T17:18:08.407 INFO:teuthology.orchestra.run.smithi117.stdout: "root_rbytes": 21196213, 2024-04-16T17:18:08.407 INFO:teuthology.orchestra.run.smithi117.stdout: "root_rsnaps": 6, 2024-04-16T17:18:08.407 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_backtrace_fetch": 268, 2024-04-16T17:18:08.407 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_set_tag": 0, 2024-04-16T17:18:08.407 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_backtrace_repaired": 0, 2024-04-16T17:18:08.408 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_inotable_repaired": 0, 2024-04-16T17:18:08.408 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_dir_inodes": 0, 2024-04-16T17:18:08.408 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_dir_base_inodes": 0, 2024-04-16T17:18:08.408 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_dirfrag_rstats": 0, 2024-04-16T17:18:08.408 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_file_inodes": 268, 2024-04-16T17:18:08.408 INFO:teuthology.orchestra.run.smithi117.stdout: "handle_inode_file_caps": 0, 2024-04-16T17:18:08.408 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph_cap_op_revoke": 739, 2024-04-16T17:18:08.409 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph_cap_op_grant": 6627, 2024-04-16T17:18:08.409 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph_cap_op_trunc": 0, 2024-04-16T17:18:08.409 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph_cap_op_flushsnap_ack": 2, 2024-04-16T17:18:08.409 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-16T17:18:08.409 INFO:teuthology.orchestra.run.smithi117.stdout: "handle_client_caps": 2383, 2024-04-16T17:18:08.409 INFO:teuthology.orchestra.run.smithi117.stdout: "handle_client_caps_dirty": 1080, 2024-04-16T17:18:08.409 INFO:teuthology.orchestra.run.smithi117.stdout: "handle_client_cap_release": 332, 2024-04-16T17:18:08.409 INFO:teuthology.orchestra.run.smithi117.stdout: "process_request_cap_release": 9877 2024-04-16T17:18:08.410 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-16T17:18:08.410 INFO:teuthology.orchestra.run.smithi117.stdout: "mds_cache": { 2024-04-16T17:18:08.410 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_update": 0, 2024-04-16T17:18:08.410 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_update_receipt": 8, 2024-04-16T17:18:08.410 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_try_discover": 5, 2024-04-16T17:18:08.410 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_send_discover": 28, 2024-04-16T17:18:08.410 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_handle_discover": 21, 2024-04-16T17:18:08.411 INFO:teuthology.orchestra.run.smithi117.stdout: "num_strays": 0, 2024-04-16T17:18:08.411 INFO:teuthology.orchestra.run.smithi117.stdout: "num_strays_delayed": 0, 2024-04-16T17:18:08.411 INFO:teuthology.orchestra.run.smithi117.stdout: "num_strays_enqueuing": 0, 2024-04-16T17:18:08.411 INFO:teuthology.orchestra.run.smithi117.stdout: "strays_created": 1690, 2024-04-16T17:18:08.411 INFO:teuthology.orchestra.run.smithi117.stdout: "strays_enqueued": 1690, 2024-04-16T17:18:08.411 INFO:teuthology.orchestra.run.smithi117.stdout: "strays_reintegrated": 0, 2024-04-16T17:18:08.411 INFO:teuthology.orchestra.run.smithi117.stdout: "strays_migrated": 0, 2024-04-16T17:18:08.411 INFO:teuthology.orchestra.run.smithi117.stdout: "num_recovering_processing": 0, 2024-04-16T17:18:08.412 INFO:teuthology.orchestra.run.smithi117.stdout: "num_recovering_enqueued": 0, 2024-04-16T17:18:08.412 INFO:teuthology.orchestra.run.smithi117.stdout: "num_recovering_prioritized": 0, 2024-04-16T17:18:08.412 INFO:teuthology.orchestra.run.smithi117.stdout: "recovery_started": 0, 2024-04-16T17:18:08.412 INFO:teuthology.orchestra.run.smithi117.stdout: "recovery_completed": 0, 2024-04-16T17:18:08.412 INFO:teuthology.orchestra.run.smithi117.stdout: "ireq_quiesce_path": 0, 2024-04-16T17:18:08.412 INFO:teuthology.orchestra.run.smithi117.stdout: "ireq_quiesce_inode": 0, 2024-04-16T17:18:08.412 INFO:teuthology.orchestra.run.smithi117.stdout: "ireq_enqueue_scrub": 0, 2024-04-16T17:18:08.412 INFO:teuthology.orchestra.run.smithi117.stdout: "ireq_exportdir": 6, 2024-04-16T17:18:08.413 INFO:teuthology.orchestra.run.smithi117.stdout: "ireq_flush": 0, 2024-04-16T17:18:08.413 INFO:teuthology.orchestra.run.smithi117.stdout: "ireq_fragmentdir": 0, 2024-04-16T17:18:08.413 INFO:teuthology.orchestra.run.smithi117.stdout: "ireq_fragstats": 0, 2024-04-16T17:18:08.413 INFO:teuthology.orchestra.run.smithi117.stdout: "ireq_inodestats": 0 2024-04-16T17:18:08.413 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-16T17:18:08.413 INFO:teuthology.orchestra.run.smithi117.stdout: "mds_log": { 2024-04-16T17:18:08.413 INFO:teuthology.orchestra.run.smithi117.stdout: "evlrg": 0, 2024-04-16T17:18:08.413 INFO:teuthology.orchestra.run.smithi117.stdout: "evadd": 9496, 2024-04-16T17:18:08.414 INFO:teuthology.orchestra.run.smithi117.stdout: "evex": 0, 2024-04-16T17:18:08.414 INFO:teuthology.orchestra.run.smithi117.stdout: "evtrm": 0, 2024-04-16T17:18:08.414 INFO:teuthology.orchestra.run.smithi117.stdout: "ev": 9496, 2024-04-16T17:18:08.414 INFO:teuthology.orchestra.run.smithi117.stdout: "evexg": 0, 2024-04-16T17:18:08.414 INFO:teuthology.orchestra.run.smithi117.stdout: "evexd": 0, 2024-04-16T17:18:08.414 INFO:teuthology.orchestra.run.smithi117.stdout: "segadd": 12, 2024-04-16T17:18:08.414 INFO:teuthology.orchestra.run.smithi117.stdout: "segex": 0, 2024-04-16T17:18:08.415 INFO:teuthology.orchestra.run.smithi117.stdout: "segtrm": 0, 2024-04-16T17:18:08.415 INFO:teuthology.orchestra.run.smithi117.stdout: "seg": 12, 2024-04-16T17:18:08.415 INFO:teuthology.orchestra.run.smithi117.stdout: "segmjr": 2, 2024-04-16T17:18:08.415 INFO:teuthology.orchestra.run.smithi117.stdout: "segexg": 0, 2024-04-16T17:18:08.415 INFO:teuthology.orchestra.run.smithi117.stdout: "segexd": 0, 2024-04-16T17:18:08.415 INFO:teuthology.orchestra.run.smithi117.stdout: "expos": 4194304, 2024-04-16T17:18:08.415 INFO:teuthology.orchestra.run.smithi117.stdout: "wrpos": 17508205, 2024-04-16T17:18:08.416 INFO:teuthology.orchestra.run.smithi117.stdout: "rdpos": 0, 2024-04-16T17:18:08.416 INFO:teuthology.orchestra.run.smithi117.stdout: "jlat": { 2024-04-16T17:18:08.416 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 2792, 2024-04-16T17:18:08.416 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 7.089763329, 2024-04-16T17:18:08.416 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.002539313 2024-04-16T17:18:08.416 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-16T17:18:08.416 INFO:teuthology.orchestra.run.smithi117.stdout: "replayed": 0 2024-04-16T17:18:08.416 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-16T17:18:08.417 INFO:teuthology.orchestra.run.smithi117.stdout: "mds_mem": { 2024-04-16T17:18:08.417 INFO:teuthology.orchestra.run.smithi117.stdout: "ino": 138, 2024-04-16T17:18:08.417 INFO:teuthology.orchestra.run.smithi117.stdout: "ino+": 1933, 2024-04-16T17:18:08.417 INFO:teuthology.orchestra.run.smithi117.stdout: "ino-": 1795, 2024-04-16T17:18:08.417 INFO:teuthology.orchestra.run.smithi117.stdout: "dir": 39, 2024-04-16T17:18:08.417 INFO:teuthology.orchestra.run.smithi117.stdout: "dir+": 44, 2024-04-16T17:18:08.417 INFO:teuthology.orchestra.run.smithi117.stdout: "dir-": 5, 2024-04-16T17:18:08.417 INFO:teuthology.orchestra.run.smithi117.stdout: "dn": 190, 2024-04-16T17:18:08.418 INFO:teuthology.orchestra.run.smithi117.stdout: "dn+": 2816, 2024-04-16T17:18:08.418 INFO:teuthology.orchestra.run.smithi117.stdout: "dn-": 2626, 2024-04-16T17:18:08.418 INFO:teuthology.orchestra.run.smithi117.stdout: "cap": 7, 2024-04-16T17:18:08.418 INFO:teuthology.orchestra.run.smithi117.stdout: "cap+": 1816, 2024-04-16T17:18:08.418 INFO:teuthology.orchestra.run.smithi117.stdout: "cap-": 1809, 2024-04-16T17:18:08.418 INFO:teuthology.orchestra.run.smithi117.stdout: "rss": 58160, 2024-04-16T17:18:08.418 INFO:teuthology.orchestra.run.smithi117.stdout: "heap": 223608 2024-04-16T17:18:08.419 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-16T17:18:08.419 INFO:teuthology.orchestra.run.smithi117.stdout: "mds_server": { 2024-04-16T17:18:08.419 INFO:teuthology.orchestra.run.smithi117.stdout: "dispatch_client_request": 20283, 2024-04-16T17:18:08.419 INFO:teuthology.orchestra.run.smithi117.stdout: "dispatch_server_request": 116, 2024-04-16T17:18:08.419 INFO:teuthology.orchestra.run.smithi117.stdout: "handle_client_request": 10268, 2024-04-16T17:18:08.419 INFO:teuthology.orchestra.run.smithi117.stdout: "handle_client_session": 196, 2024-04-16T17:18:08.419 INFO:teuthology.orchestra.run.smithi117.stdout: "handle_peer_request": 3768, 2024-04-16T17:18:08.419 INFO:teuthology.orchestra.run.smithi117.stdout: "req_create_latency": { 2024-04-16T17:18:08.420 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 1694, 2024-04-16T17:18:08.420 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 9.084676780, 2024-04-16T17:18:08.420 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.005362855 2024-04-16T17:18:08.420 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-16T17:18:08.420 INFO:teuthology.orchestra.run.smithi117.stdout: "req_getattr_latency": { 2024-04-16T17:18:08.420 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 55, 2024-04-16T17:18:08.420 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.030517537, 2024-04-16T17:18:08.420 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000554864 2024-04-16T17:18:08.421 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-16T17:18:08.421 INFO:teuthology.orchestra.run.smithi117.stdout: "req_getfilelock_latency": { 2024-04-16T17:18:08.421 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-04-16T17:18:08.421 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-04-16T17:18:08.421 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-04-16T17:18:08.421 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-16T17:18:08.421 INFO:teuthology.orchestra.run.smithi117.stdout: "req_link_latency": { 2024-04-16T17:18:08.421 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-04-16T17:18:08.422 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-04-16T17:18:08.422 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-04-16T17:18:08.422 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-16T17:18:08.422 INFO:teuthology.orchestra.run.smithi117.stdout: "req_lookup_latency": { 2024-04-16T17:18:08.422 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 2394, 2024-04-16T17:18:08.422 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 23.941192071, 2024-04-16T17:18:08.422 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.010000497 2024-04-16T17:18:08.423 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-16T17:18:08.423 INFO:teuthology.orchestra.run.smithi117.stdout: "req_lookuphash_latency": { 2024-04-16T17:18:08.423 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-04-16T17:18:08.423 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-04-16T17:18:08.423 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-04-16T17:18:08.423 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-16T17:18:08.423 INFO:teuthology.orchestra.run.smithi117.stdout: "req_lookupino_latency": { 2024-04-16T17:18:08.423 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-04-16T17:18:08.424 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-04-16T17:18:08.424 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-04-16T17:18:08.424 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-16T17:18:08.424 INFO:teuthology.orchestra.run.smithi117.stdout: "req_lookupname_latency": { 2024-04-16T17:18:08.424 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-04-16T17:18:08.424 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-04-16T17:18:08.424 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-04-16T17:18:08.424 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-16T17:18:08.425 INFO:teuthology.orchestra.run.smithi117.stdout: "req_lookupparent_latency": { 2024-04-16T17:18:08.425 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-04-16T17:18:08.425 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-04-16T17:18:08.425 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-04-16T17:18:08.425 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-16T17:18:08.425 INFO:teuthology.orchestra.run.smithi117.stdout: "req_lookupsnap_latency": { 2024-04-16T17:18:08.425 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-04-16T17:18:08.425 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-04-16T17:18:08.426 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-04-16T17:18:08.426 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-16T17:18:08.426 INFO:teuthology.orchestra.run.smithi117.stdout: "req_lssnap_latency": { 2024-04-16T17:18:08.426 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-04-16T17:18:08.427 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-04-16T17:18:08.427 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-04-16T17:18:08.427 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-16T17:18:08.427 INFO:teuthology.orchestra.run.smithi117.stdout: "req_mkdir_latency": { 2024-04-16T17:18:08.427 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-04-16T17:18:08.427 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-04-16T17:18:08.427 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-04-16T17:18:08.428 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-16T17:18:08.428 INFO:teuthology.orchestra.run.smithi117.stdout: "req_mknod_latency": { 2024-04-16T17:18:08.428 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-04-16T17:18:08.428 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-04-16T17:18:08.428 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-04-16T17:18:08.428 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-16T17:18:08.428 INFO:teuthology.orchestra.run.smithi117.stdout: "req_mksnap_latency": { 2024-04-16T17:18:08.429 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-04-16T17:18:08.429 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-04-16T17:18:08.429 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-04-16T17:18:08.429 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-16T17:18:08.429 INFO:teuthology.orchestra.run.smithi117.stdout: "req_open_latency": { 2024-04-16T17:18:08.429 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-04-16T17:18:08.429 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-04-16T17:18:08.429 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-04-16T17:18:08.430 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-16T17:18:08.430 INFO:teuthology.orchestra.run.smithi117.stdout: "req_readdir_latency": { 2024-04-16T17:18:08.430 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 4010, 2024-04-16T17:18:08.430 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 1.811696691, 2024-04-16T17:18:08.430 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000451794 2024-04-16T17:18:08.430 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-16T17:18:08.430 INFO:teuthology.orchestra.run.smithi117.stdout: "req_rename_latency": { 2024-04-16T17:18:08.431 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 268, 2024-04-16T17:18:08.431 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 4.583790617, 2024-04-16T17:18:08.431 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.017103696 2024-04-16T17:18:08.431 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-16T17:18:08.431 INFO:teuthology.orchestra.run.smithi117.stdout: "req_renamesnap_latency": { 2024-04-16T17:18:08.431 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-04-16T17:18:08.431 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-04-16T17:18:08.431 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-04-16T17:18:08.431 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-16T17:18:08.432 INFO:teuthology.orchestra.run.smithi117.stdout: "req_snapdiff_latency": { 2024-04-16T17:18:08.432 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-04-16T17:18:08.432 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-04-16T17:18:08.432 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-04-16T17:18:08.432 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-16T17:18:08.432 INFO:teuthology.orchestra.run.smithi117.stdout: "req_rmdir_latency": { 2024-04-16T17:18:08.432 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-04-16T17:18:08.432 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-04-16T17:18:08.433 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-04-16T17:18:08.433 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-16T17:18:08.433 INFO:teuthology.orchestra.run.smithi117.stdout: "req_rmsnap_latency": { 2024-04-16T17:18:08.433 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-04-16T17:18:08.433 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-04-16T17:18:08.433 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-04-16T17:18:08.433 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-16T17:18:08.433 INFO:teuthology.orchestra.run.smithi117.stdout: "req_rmxattr_latency": { 2024-04-16T17:18:08.434 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-04-16T17:18:08.434 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-04-16T17:18:08.434 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-04-16T17:18:08.434 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-16T17:18:08.434 INFO:teuthology.orchestra.run.smithi117.stdout: "req_setattr_latency": { 2024-04-16T17:18:08.434 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-04-16T17:18:08.434 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-04-16T17:18:08.434 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-04-16T17:18:08.435 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-16T17:18:08.435 INFO:teuthology.orchestra.run.smithi117.stdout: "req_setdirlayout_latency": { 2024-04-16T17:18:08.435 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-04-16T17:18:08.435 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-04-16T17:18:08.435 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-04-16T17:18:08.435 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-16T17:18:08.435 INFO:teuthology.orchestra.run.smithi117.stdout: "req_setfilelock_latency": { 2024-04-16T17:18:08.435 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-04-16T17:18:08.435 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-04-16T17:18:08.436 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-04-16T17:18:08.436 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-16T17:18:08.436 INFO:teuthology.orchestra.run.smithi117.stdout: "req_setlayout_latency": { 2024-04-16T17:18:08.436 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-04-16T17:18:08.436 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-04-16T17:18:08.436 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-04-16T17:18:08.436 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-16T17:18:08.436 INFO:teuthology.orchestra.run.smithi117.stdout: "req_setxattr_latency": { 2024-04-16T17:18:08.437 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-04-16T17:18:08.437 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-04-16T17:18:08.437 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-04-16T17:18:08.437 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-16T17:18:08.437 INFO:teuthology.orchestra.run.smithi117.stdout: "req_symlink_latency": { 2024-04-16T17:18:08.437 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-04-16T17:18:08.437 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-04-16T17:18:08.437 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-04-16T17:18:08.437 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-16T17:18:08.438 INFO:teuthology.orchestra.run.smithi117.stdout: "req_unlink_latency": { 2024-04-16T17:18:08.438 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 1690, 2024-04-16T17:18:08.438 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 3.628764217, 2024-04-16T17:18:08.438 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.002147197 2024-04-16T17:18:08.438 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-16T17:18:08.438 INFO:teuthology.orchestra.run.smithi117.stdout: "cap_revoke_eviction": 0, 2024-04-16T17:18:08.438 INFO:teuthology.orchestra.run.smithi117.stdout: "cap_acquisition_throttle": 0, 2024-04-16T17:18:08.438 INFO:teuthology.orchestra.run.smithi117.stdout: "req_getvxattr_latency": { 2024-04-16T17:18:08.439 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-04-16T17:18:08.439 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-04-16T17:18:08.439 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-04-16T17:18:08.439 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-04-16T17:18:08.439 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-16T17:18:08.439 INFO:teuthology.orchestra.run.smithi117.stdout: "mds_sessions": { 2024-04-16T17:18:08.439 INFO:teuthology.orchestra.run.smithi117.stdout: "session_count": 2, 2024-04-16T17:18:08.439 INFO:teuthology.orchestra.run.smithi117.stdout: "session_add": 5, 2024-04-16T17:18:08.440 INFO:teuthology.orchestra.run.smithi117.stdout: "session_remove": 3, 2024-04-16T17:18:08.440 INFO:teuthology.orchestra.run.smithi117.stdout: "sessions_open": 2, 2024-04-16T17:18:08.440 INFO:teuthology.orchestra.run.smithi117.stdout: "sessions_stale": 0, 2024-04-16T17:18:08.440 INFO:teuthology.orchestra.run.smithi117.stdout: "total_load": 3469, 2024-04-16T17:18:08.440 INFO:teuthology.orchestra.run.smithi117.stdout: "average_load": 1734, 2024-04-16T17:18:08.440 INFO:teuthology.orchestra.run.smithi117.stdout: "avg_session_uptime": 675, 2024-04-16T17:18:08.440 INFO:teuthology.orchestra.run.smithi117.stdout: "mdthresh_evicted": 0 2024-04-16T17:18:08.440 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-16T17:18:08.440 INFO:teuthology.orchestra.run.smithi117.stdout: "mempool": { 2024-04-16T17:18:08.441 INFO:teuthology.orchestra.run.smithi117.stdout: "bloom_filter_bytes": 0, 2024-04-16T17:18:08.441 INFO:teuthology.orchestra.run.smithi117.stdout: "bloom_filter_items": 0, 2024-04-16T17:18:08.441 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_alloc_bytes": 0, 2024-04-16T17:18:08.441 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_alloc_items": 0, 2024-04-16T17:18:08.441 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_data_bytes": 0, 2024-04-16T17:18:08.441 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_data_items": 0, 2024-04-16T17:18:08.441 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-16T17:18:08.441 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_onode_items": 0, 2024-04-16T17:18:08.442 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-16T17:18:08.442 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_meta_items": 0, 2024-04-16T17:18:08.442 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_other_bytes": 0, 2024-04-16T17:18:08.442 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_other_items": 0, 2024-04-16T17:18:08.442 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-16T17:18:08.442 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_buffer_items": 0, 2024-04-16T17:18:08.442 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_extent_bytes": 0, 2024-04-16T17:18:08.442 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_extent_items": 0, 2024-04-16T17:18:08.443 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_blob_bytes": 0, 2024-04-16T17:18:08.443 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_blob_items": 0, 2024-04-16T17:18:08.443 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-16T17:18:08.443 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_shared_blob_items": 0, 2024-04-16T17:18:08.443 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-16T17:18:08.443 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_inline_bl_items": 0, 2024-04-16T17:18:08.443 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_fsck_bytes": 0, 2024-04-16T17:18:08.443 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_fsck_items": 0, 2024-04-16T17:18:08.443 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_txc_bytes": 0, 2024-04-16T17:18:08.444 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_txc_items": 0, 2024-04-16T17:18:08.444 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-16T17:18:08.444 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_writing_deferred_items": 0, 2024-04-16T17:18:08.444 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_writing_bytes": 0, 2024-04-16T17:18:08.444 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_writing_items": 0, 2024-04-16T17:18:08.444 INFO:teuthology.orchestra.run.smithi117.stdout: "bluefs_bytes": 0, 2024-04-16T17:18:08.444 INFO:teuthology.orchestra.run.smithi117.stdout: "bluefs_items": 0, 2024-04-16T17:18:08.444 INFO:teuthology.orchestra.run.smithi117.stdout: "bluefs_file_reader_bytes": 0, 2024-04-16T17:18:08.444 INFO:teuthology.orchestra.run.smithi117.stdout: "bluefs_file_reader_items": 0, 2024-04-16T17:18:08.445 INFO:teuthology.orchestra.run.smithi117.stdout: "bluefs_file_writer_bytes": 0, 2024-04-16T17:18:08.445 INFO:teuthology.orchestra.run.smithi117.stdout: "bluefs_file_writer_items": 0, 2024-04-16T17:18:08.445 INFO:teuthology.orchestra.run.smithi117.stdout: "buffer_anon_bytes": 299924, 2024-04-16T17:18:08.445 INFO:teuthology.orchestra.run.smithi117.stdout: "buffer_anon_items": 198, 2024-04-16T17:18:08.445 INFO:teuthology.orchestra.run.smithi117.stdout: "buffer_meta_bytes": 528, 2024-04-16T17:18:08.445 INFO:teuthology.orchestra.run.smithi117.stdout: "buffer_meta_items": 6, 2024-04-16T17:18:08.445 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_bytes": 0, 2024-04-16T17:18:08.445 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_items": 0, 2024-04-16T17:18:08.446 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_mapbl_bytes": 0, 2024-04-16T17:18:08.446 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_mapbl_items": 0, 2024-04-16T17:18:08.446 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_pglog_bytes": 0, 2024-04-16T17:18:08.446 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_pglog_items": 0, 2024-04-16T17:18:08.446 INFO:teuthology.orchestra.run.smithi117.stdout: "osdmap_bytes": 9512, 2024-04-16T17:18:08.446 INFO:teuthology.orchestra.run.smithi117.stdout: "osdmap_items": 182, 2024-04-16T17:18:08.446 INFO:teuthology.orchestra.run.smithi117.stdout: "osdmap_mapping_bytes": 0, 2024-04-16T17:18:08.446 INFO:teuthology.orchestra.run.smithi117.stdout: "osdmap_mapping_items": 0, 2024-04-16T17:18:08.447 INFO:teuthology.orchestra.run.smithi117.stdout: "pgmap_bytes": 0, 2024-04-16T17:18:08.447 INFO:teuthology.orchestra.run.smithi117.stdout: "pgmap_items": 0, 2024-04-16T17:18:08.447 INFO:teuthology.orchestra.run.smithi117.stdout: "mds_co_bytes": 552709, 2024-04-16T17:18:08.447 INFO:teuthology.orchestra.run.smithi117.stdout: "mds_co_items": 3954, 2024-04-16T17:18:08.447 INFO:teuthology.orchestra.run.smithi117.stdout: "unittest_1_bytes": 0, 2024-04-16T17:18:08.447 INFO:teuthology.orchestra.run.smithi117.stdout: "unittest_1_items": 0, 2024-04-16T17:18:08.447 INFO:teuthology.orchestra.run.smithi117.stdout: "unittest_2_bytes": 0, 2024-04-16T17:18:08.447 INFO:teuthology.orchestra.run.smithi117.stdout: "unittest_2_items": 0 2024-04-16T17:18:08.448 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-16T17:18:08.448 INFO:teuthology.orchestra.run.smithi117.stdout: "objecter": { 2024-04-16T17:18:08.448 INFO:teuthology.orchestra.run.smithi117.stdout: "op_active": 0, 2024-04-16T17:18:08.448 INFO:teuthology.orchestra.run.smithi117.stdout: "op_laggy": 0, 2024-04-16T17:18:08.448 INFO:teuthology.orchestra.run.smithi117.stdout: "op_send": 7184, 2024-04-16T17:18:08.448 INFO:teuthology.orchestra.run.smithi117.stdout: "op_send_bytes": 13696068, 2024-04-16T17:18:08.448 INFO:teuthology.orchestra.run.smithi117.stdout: "op_resend": 33, 2024-04-16T17:18:08.448 INFO:teuthology.orchestra.run.smithi117.stdout: "op_reply": 7184, 2024-04-16T17:18:08.449 INFO:teuthology.orchestra.run.smithi117.stdout: "op_latency": { 2024-04-16T17:18:08.449 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 7184, 2024-04-16T17:18:08.449 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 29.846444321, 2024-04-16T17:18:08.449 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.004154571 2024-04-16T17:18:08.449 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-16T17:18:08.449 INFO:teuthology.orchestra.run.smithi117.stdout: "op_inflight": 0, 2024-04-16T17:18:08.449 INFO:teuthology.orchestra.run.smithi117.stdout: "oplen_avg": { 2024-04-16T17:18:08.449 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 7184, 2024-04-16T17:18:08.449 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 7535 2024-04-16T17:18:08.450 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-16T17:18:08.450 INFO:teuthology.orchestra.run.smithi117.stdout: "op": 7184, 2024-04-16T17:18:08.450 INFO:teuthology.orchestra.run.smithi117.stdout: "op_r": 447, 2024-04-16T17:18:08.450 INFO:teuthology.orchestra.run.smithi117.stdout: "op_w": 6737, 2024-04-16T17:18:08.450 INFO:teuthology.orchestra.run.smithi117.stdout: "op_rmw": 0, 2024-04-16T17:18:08.450 INFO:teuthology.orchestra.run.smithi117.stdout: "op_pg": 0, 2024-04-16T17:18:08.450 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_stat": 0, 2024-04-16T17:18:08.450 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_create": 10, 2024-04-16T17:18:08.451 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_read": 175, 2024-04-16T17:18:08.451 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_write": 2967, 2024-04-16T17:18:08.451 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_writefull": 132, 2024-04-16T17:18:08.451 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_writesame": 0, 2024-04-16T17:18:08.451 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_append": 0, 2024-04-16T17:18:08.451 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_zero": 0, 2024-04-16T17:18:08.451 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_truncate": 0, 2024-04-16T17:18:08.451 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_delete": 3395, 2024-04-16T17:18:08.452 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_mapext": 0, 2024-04-16T17:18:08.452 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_sparse_read": 0, 2024-04-16T17:18:08.452 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_clonerange": 0, 2024-04-16T17:18:08.452 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_getxattr": 272, 2024-04-16T17:18:08.452 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_setxattr": 20, 2024-04-16T17:18:08.452 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_cmpxattr": 0, 2024-04-16T17:18:08.452 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_rmxattr": 0, 2024-04-16T17:18:08.452 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_resetxattrs": 0, 2024-04-16T17:18:08.452 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_call": 0, 2024-04-16T17:18:08.453 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_watch": 0, 2024-04-16T17:18:08.453 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_notify": 0, 2024-04-16T17:18:08.453 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_src_cmpxattr": 0, 2024-04-16T17:18:08.453 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_pgls": 0, 2024-04-16T17:18:08.453 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_pgls_filter": 0, 2024-04-16T17:18:08.453 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_other": 0, 2024-04-16T17:18:08.453 INFO:teuthology.orchestra.run.smithi117.stdout: "linger_active": 0, 2024-04-16T17:18:08.453 INFO:teuthology.orchestra.run.smithi117.stdout: "linger_send": 0, 2024-04-16T17:18:08.453 INFO:teuthology.orchestra.run.smithi117.stdout: "linger_resend": 0, 2024-04-16T17:18:08.454 INFO:teuthology.orchestra.run.smithi117.stdout: "linger_ping": 0, 2024-04-16T17:18:08.454 INFO:teuthology.orchestra.run.smithi117.stdout: "poolop_active": 0, 2024-04-16T17:18:08.454 INFO:teuthology.orchestra.run.smithi117.stdout: "poolop_send": 0, 2024-04-16T17:18:08.454 INFO:teuthology.orchestra.run.smithi117.stdout: "poolop_resend": 0, 2024-04-16T17:18:08.454 INFO:teuthology.orchestra.run.smithi117.stdout: "poolstat_active": 0, 2024-04-16T17:18:08.454 INFO:teuthology.orchestra.run.smithi117.stdout: "poolstat_send": 0, 2024-04-16T17:18:08.454 INFO:teuthology.orchestra.run.smithi117.stdout: "poolstat_resend": 0, 2024-04-16T17:18:08.454 INFO:teuthology.orchestra.run.smithi117.stdout: "statfs_active": 0, 2024-04-16T17:18:08.454 INFO:teuthology.orchestra.run.smithi117.stdout: "statfs_send": 0, 2024-04-16T17:18:08.455 INFO:teuthology.orchestra.run.smithi117.stdout: "statfs_resend": 0, 2024-04-16T17:18:08.455 INFO:teuthology.orchestra.run.smithi117.stdout: "command_active": 0, 2024-04-16T17:18:08.455 INFO:teuthology.orchestra.run.smithi117.stdout: "command_send": 0, 2024-04-16T17:18:08.455 INFO:teuthology.orchestra.run.smithi117.stdout: "command_resend": 0, 2024-04-16T17:18:08.455 INFO:teuthology.orchestra.run.smithi117.stdout: "map_epoch": 106, 2024-04-16T17:18:08.455 INFO:teuthology.orchestra.run.smithi117.stdout: "map_full": 0, 2024-04-16T17:18:08.455 INFO:teuthology.orchestra.run.smithi117.stdout: "map_inc": 16, 2024-04-16T17:18:08.455 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_sessions": 12, 2024-04-16T17:18:08.455 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_session_open": 12, 2024-04-16T17:18:08.456 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_session_close": 0, 2024-04-16T17:18:08.456 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_laggy": 0, 2024-04-16T17:18:08.456 INFO:teuthology.orchestra.run.smithi117.stdout: "omap_wr": 417, 2024-04-16T17:18:08.456 INFO:teuthology.orchestra.run.smithi117.stdout: "omap_rd": 8, 2024-04-16T17:18:08.456 INFO:teuthology.orchestra.run.smithi117.stdout: "omap_del": 139 2024-04-16T17:18:08.456 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-16T17:18:08.456 INFO:teuthology.orchestra.run.smithi117.stdout: "oft": { 2024-04-16T17:18:08.456 INFO:teuthology.orchestra.run.smithi117.stdout: "omap_total_objs": 1, 2024-04-16T17:18:08.456 INFO:teuthology.orchestra.run.smithi117.stdout: "omap_total_kv_pairs": 0, 2024-04-16T17:18:08.457 INFO:teuthology.orchestra.run.smithi117.stdout: "omap_total_updates": 0, 2024-04-16T17:18:08.457 INFO:teuthology.orchestra.run.smithi117.stdout: "omap_total_removes": 0 2024-04-16T17:18:08.457 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-16T17:18:08.457 INFO:teuthology.orchestra.run.smithi117.stdout: "osd-slow-ops": { 2024-04-16T17:18:08.457 INFO:teuthology.orchestra.run.smithi117.stdout: "slow_ops_count": 0 2024-04-16T17:18:08.457 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-16T17:18:08.457 INFO:teuthology.orchestra.run.smithi117.stdout: "purge_queue": { 2024-04-16T17:18:08.457 INFO:teuthology.orchestra.run.smithi117.stdout: "pq_executing_ops": 0, 2024-04-16T17:18:08.458 INFO:teuthology.orchestra.run.smithi117.stdout: "pq_executing_ops_high_water": 16, 2024-04-16T17:18:08.458 INFO:teuthology.orchestra.run.smithi117.stdout: "pq_executing": 0, 2024-04-16T17:18:08.458 INFO:teuthology.orchestra.run.smithi117.stdout: "pq_executing_high_water": 8, 2024-04-16T17:18:08.458 INFO:teuthology.orchestra.run.smithi117.stdout: "pq_executed_ops": 3380, 2024-04-16T17:18:08.458 INFO:teuthology.orchestra.run.smithi117.stdout: "pq_executed": 1690, 2024-04-16T17:18:08.458 INFO:teuthology.orchestra.run.smithi117.stdout: "pq_item_in_journal": 0 2024-04-16T17:18:08.458 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-16T17:18:08.458 INFO:teuthology.orchestra.run.smithi117.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-16T17:18:08.458 INFO:teuthology.orchestra.run.smithi117.stdout: "val": 0, 2024-04-16T17:18:08.459 INFO:teuthology.orchestra.run.smithi117.stdout: "max": 104857600, 2024-04-16T17:18:08.459 INFO:teuthology.orchestra.run.smithi117.stdout: "get_started": 0, 2024-04-16T17:18:08.459 INFO:teuthology.orchestra.run.smithi117.stdout: "get": 43340, 2024-04-16T17:18:08.459 INFO:teuthology.orchestra.run.smithi117.stdout: "get_sum": 11476731, 2024-04-16T17:18:08.459 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_fail": 0, 2024-04-16T17:18:08.459 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_success": 43340, 2024-04-16T17:18:08.459 INFO:teuthology.orchestra.run.smithi117.stdout: "take": 0, 2024-04-16T17:18:08.459 INFO:teuthology.orchestra.run.smithi117.stdout: "take_sum": 0, 2024-04-16T17:18:08.459 INFO:teuthology.orchestra.run.smithi117.stdout: "put": 43340, 2024-04-16T17:18:08.460 INFO:teuthology.orchestra.run.smithi117.stdout: "put_sum": 11476731, 2024-04-16T17:18:08.460 INFO:teuthology.orchestra.run.smithi117.stdout: "wait": { 2024-04-16T17:18:08.460 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-04-16T17:18:08.460 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-04-16T17:18:08.460 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-04-16T17:18:08.460 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-04-16T17:18:08.460 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-16T17:18:08.460 INFO:teuthology.orchestra.run.smithi117.stdout: "throttle-objecter_bytes": { 2024-04-16T17:18:08.460 INFO:teuthology.orchestra.run.smithi117.stdout: "val": 0, 2024-04-16T17:18:08.461 INFO:teuthology.orchestra.run.smithi117.stdout: "max": 104857600, 2024-04-16T17:18:08.461 INFO:teuthology.orchestra.run.smithi117.stdout: "get_started": 0, 2024-04-16T17:18:08.461 INFO:teuthology.orchestra.run.smithi117.stdout: "get": 0, 2024-04-16T17:18:08.461 INFO:teuthology.orchestra.run.smithi117.stdout: "get_sum": 0, 2024-04-16T17:18:08.461 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_fail": 0, 2024-04-16T17:18:08.461 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_success": 0, 2024-04-16T17:18:08.461 INFO:teuthology.orchestra.run.smithi117.stdout: "take": 7184, 2024-04-16T17:18:08.461 INFO:teuthology.orchestra.run.smithi117.stdout: "take_sum": 13880334, 2024-04-16T17:18:08.462 INFO:teuthology.orchestra.run.smithi117.stdout: "put": 7184, 2024-04-16T17:18:08.462 INFO:teuthology.orchestra.run.smithi117.stdout: "put_sum": 13880334, 2024-04-16T17:18:08.462 INFO:teuthology.orchestra.run.smithi117.stdout: "wait": { 2024-04-16T17:18:08.462 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-04-16T17:18:08.462 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-04-16T17:18:08.462 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-04-16T17:18:08.462 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-04-16T17:18:08.462 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-16T17:18:08.462 INFO:teuthology.orchestra.run.smithi117.stdout: "throttle-objecter_ops": { 2024-04-16T17:18:08.463 INFO:teuthology.orchestra.run.smithi117.stdout: "val": 0, 2024-04-16T17:18:08.463 INFO:teuthology.orchestra.run.smithi117.stdout: "max": 1024, 2024-04-16T17:18:08.463 INFO:teuthology.orchestra.run.smithi117.stdout: "get_started": 0, 2024-04-16T17:18:08.463 INFO:teuthology.orchestra.run.smithi117.stdout: "get": 0, 2024-04-16T17:18:08.463 INFO:teuthology.orchestra.run.smithi117.stdout: "get_sum": 0, 2024-04-16T17:18:08.463 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_fail": 0, 2024-04-16T17:18:08.463 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_success": 0, 2024-04-16T17:18:08.463 INFO:teuthology.orchestra.run.smithi117.stdout: "take": 7184, 2024-04-16T17:18:08.463 INFO:teuthology.orchestra.run.smithi117.stdout: "take_sum": 7184, 2024-04-16T17:18:08.464 INFO:teuthology.orchestra.run.smithi117.stdout: "put": 7184, 2024-04-16T17:18:08.464 INFO:teuthology.orchestra.run.smithi117.stdout: "put_sum": 7184, 2024-04-16T17:18:08.464 INFO:teuthology.orchestra.run.smithi117.stdout: "wait": { 2024-04-16T17:18:08.464 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-04-16T17:18:08.464 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-04-16T17:18:08.464 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-04-16T17:18:08.464 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-04-16T17:18:08.464 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-16T17:18:08.464 INFO:teuthology.orchestra.run.smithi117.stdout: "throttle-write_buf_throttle": { 2024-04-16T17:18:08.465 INFO:teuthology.orchestra.run.smithi117.stdout: "val": 0, 2024-04-16T17:18:08.465 INFO:teuthology.orchestra.run.smithi117.stdout: "max": 3758096384, 2024-04-16T17:18:08.465 INFO:teuthology.orchestra.run.smithi117.stdout: "get_started": 0, 2024-04-16T17:18:08.465 INFO:teuthology.orchestra.run.smithi117.stdout: "get": 1690, 2024-04-16T17:18:08.465 INFO:teuthology.orchestra.run.smithi117.stdout: "get_sum": 184330, 2024-04-16T17:18:08.465 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_fail": 0, 2024-04-16T17:18:08.465 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_success": 1690, 2024-04-16T17:18:08.465 INFO:teuthology.orchestra.run.smithi117.stdout: "take": 0, 2024-04-16T17:18:08.465 INFO:teuthology.orchestra.run.smithi117.stdout: "take_sum": 0, 2024-04-16T17:18:08.466 INFO:teuthology.orchestra.run.smithi117.stdout: "put": 175, 2024-04-16T17:18:08.466 INFO:teuthology.orchestra.run.smithi117.stdout: "put_sum": 184330, 2024-04-16T17:18:08.466 INFO:teuthology.orchestra.run.smithi117.stdout: "wait": { 2024-04-16T17:18:08.466 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-04-16T17:18:08.466 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-04-16T17:18:08.466 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-04-16T17:18:08.466 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-04-16T17:18:08.466 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-16T17:18:08.466 INFO:teuthology.orchestra.run.smithi117.stdout: "throttle-write_buf_throttle-0x55965f57e500": { 2024-04-16T17:18:08.467 INFO:teuthology.orchestra.run.smithi117.stdout: "val": 0, 2024-04-16T17:18:08.467 INFO:teuthology.orchestra.run.smithi117.stdout: "max": 3758096384, 2024-04-16T17:18:08.467 INFO:teuthology.orchestra.run.smithi117.stdout: "get_started": 0, 2024-04-16T17:18:08.467 INFO:teuthology.orchestra.run.smithi117.stdout: "get": 9496, 2024-04-16T17:18:08.467 INFO:teuthology.orchestra.run.smithi117.stdout: "get_sum": 13313901, 2024-04-16T17:18:08.467 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_fail": 0, 2024-04-16T17:18:08.467 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_success": 9496, 2024-04-16T17:18:08.467 INFO:teuthology.orchestra.run.smithi117.stdout: "take": 0, 2024-04-16T17:18:08.467 INFO:teuthology.orchestra.run.smithi117.stdout: "take_sum": 0, 2024-04-16T17:18:08.468 INFO:teuthology.orchestra.run.smithi117.stdout: "put": 2792, 2024-04-16T17:18:08.468 INFO:teuthology.orchestra.run.smithi117.stdout: "put_sum": 13313901, 2024-04-16T17:18:08.468 INFO:teuthology.orchestra.run.smithi117.stdout: "wait": { 2024-04-16T17:18:08.468 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-04-16T17:18:08.468 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-04-16T17:18:08.468 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-04-16T17:18:08.468 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-04-16T17:18:08.468 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-04-16T17:18:08.468 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-04-16T17:18:08.804 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:18:08 smithi117 ceph-mon[36730]: pgmap v355: 129 pgs: 129 active+clean; 55 MiB data, 874 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 24 op/s 2024-04-16T17:18:08.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:18:08 smithi104 ceph-mon[36457]: pgmap v355: 129 pgs: 129 active+clean; 55 MiB data, 874 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 24 op/s 2024-04-16T17:18:08.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:18:08 smithi045 ceph-mon[31676]: pgmap v355: 129 pgs: 129 active+clean; 55 MiB data, 874 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 24 op/s 2024-04-16T17:18:09.197 INFO:tasks.check_counter:Daemon mds.i mds.exported=4 2024-04-16T17:18:09.197 INFO:tasks.check_counter:Daemon mds.i mds.imported=7 2024-04-16T17:18:09.197 INFO:tasks.check_counter:Daemon mds.i mds.root_rsnaps=6 2024-04-16T17:18:09.197 INFO:tasks.check_counter:Daemon mds.i mds_server.req_mksnap_latency.avgcount=0 2024-04-16T17:18:09.197 INFO:tasks.check_counter:Daemon mds.i mds_server.req_rmsnap_latency.avgcount=0 2024-04-16T17:18:09.197 DEBUG:tasks.check_counter:Getting stats from l 2024-04-16T17:18:09.198 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45 shell --fsid b1fecd38-fc11-11ee-bc90-c7b262605968 -- ceph daemon mds.l perf dump 2024-04-16T17:18:09.440 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/mon.c/config 2024-04-16T17:18:09.831 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:18:09 smithi117 ceph-mon[36730]: osdmap e107: 12 total, 12 up, 12 in 2024-04-16T17:18:09.926 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:18:09 smithi104 ceph-mon[36457]: osdmap e107: 12 total, 12 up, 12 in 2024-04-16T17:18:09.976 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:18:09 smithi045 ceph-mon[31676]: osdmap e107: 12 total, 12 up, 12 in 2024-04-16T17:18:10.918 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:18:10 smithi117 ceph-mon[36730]: pgmap v357: 129 pgs: 129 active+clean; 55 MiB data, 874 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-04-16T17:18:10.926 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:18:10 smithi104 ceph-mon[36457]: pgmap v357: 129 pgs: 129 active+clean; 55 MiB data, 874 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-04-16T17:18:10.976 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:18:10 smithi045 ceph-mon[31676]: pgmap v357: 129 pgs: 129 active+clean; 55 MiB data, 874 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-04-16T17:18:11.139 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-04-16T17:18:11.139 INFO:teuthology.orchestra.run.smithi117.stdout: "AsyncMessenger::Worker-0": { 2024-04-16T17:18:11.139 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_messages": 4, 2024-04-16T17:18:11.140 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_messages": 4, 2024-04-16T17:18:11.140 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_bytes": 2992, 2024-04-16T17:18:11.140 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_bytes": 448, 2024-04-16T17:18:11.140 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_created_connections": 4, 2024-04-16T17:18:11.140 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_active_connections": 0, 2024-04-16T17:18:11.140 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_total_time": 0.001288416, 2024-04-16T17:18:11.140 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_send_time": 0.000037992, 2024-04-16T17:18:11.140 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_recv_time": 0.000498005, 2024-04-16T17:18:11.140 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-16T17:18:11.140 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_messages_queue_lat": { 2024-04-16T17:18:11.141 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 3, 2024-04-16T17:18:11.141 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000776927, 2024-04-16T17:18:11.141 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000258975 2024-04-16T17:18:11.141 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-16T17:18:11.141 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_handle_ack_lat": { 2024-04-16T17:18:11.141 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-04-16T17:18:11.141 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-04-16T17:18:11.141 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-04-16T17:18:11.141 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-16T17:18:11.142 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-04-16T17:18:11.142 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_encrypted_bytes": 448 2024-04-16T17:18:11.142 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-16T17:18:11.142 INFO:teuthology.orchestra.run.smithi117.stdout: "AsyncMessenger::Worker-1": { 2024-04-16T17:18:11.142 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_messages": 2, 2024-04-16T17:18:11.142 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_messages": 189, 2024-04-16T17:18:11.142 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_bytes": 320, 2024-04-16T17:18:11.142 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_bytes": 189440, 2024-04-16T17:18:11.142 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_created_connections": 5, 2024-04-16T17:18:11.143 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_active_connections": 3, 2024-04-16T17:18:11.143 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_total_time": 0.011796347, 2024-04-16T17:18:11.143 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_send_time": 0.008041412, 2024-04-16T17:18:11.143 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_recv_time": 0.001216441, 2024-04-16T17:18:11.143 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-16T17:18:11.143 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_messages_queue_lat": { 2024-04-16T17:18:11.143 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 186, 2024-04-16T17:18:11.143 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.031315250, 2024-04-16T17:18:11.144 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000168361 2024-04-16T17:18:11.144 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-16T17:18:11.144 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_handle_ack_lat": { 2024-04-16T17:18:11.144 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-04-16T17:18:11.144 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-04-16T17:18:11.144 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-04-16T17:18:11.144 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-16T17:18:11.144 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-16T17:18:11.144 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_encrypted_bytes": 189440 2024-04-16T17:18:11.144 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-16T17:18:11.145 INFO:teuthology.orchestra.run.smithi117.stdout: "AsyncMessenger::Worker-2": { 2024-04-16T17:18:11.145 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_messages": 259, 2024-04-16T17:18:11.145 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_messages": 248, 2024-04-16T17:18:11.145 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_bytes": 1600384, 2024-04-16T17:18:11.145 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_bytes": 153200, 2024-04-16T17:18:11.145 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_created_connections": 2, 2024-04-16T17:18:11.145 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_active_connections": 1, 2024-04-16T17:18:11.145 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_total_time": 0.052947561, 2024-04-16T17:18:11.145 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_send_time": 0.015682351, 2024-04-16T17:18:11.146 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_recv_time": 0.039156373, 2024-04-16T17:18:11.146 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_fast_dispatch_time": 0.006517958, 2024-04-16T17:18:11.146 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_messages_queue_lat": { 2024-04-16T17:18:11.146 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 247, 2024-04-16T17:18:11.146 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.013042193, 2024-04-16T17:18:11.146 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000052802 2024-04-16T17:18:11.146 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-16T17:18:11.146 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_handle_ack_lat": { 2024-04-16T17:18:11.146 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-04-16T17:18:11.147 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-04-16T17:18:11.147 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-04-16T17:18:11.147 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-16T17:18:11.147 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_encrypted_bytes": 1600384, 2024-04-16T17:18:11.147 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_encrypted_bytes": 153200 2024-04-16T17:18:11.147 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-16T17:18:11.147 INFO:teuthology.orchestra.run.smithi117.stdout: "cct": { 2024-04-16T17:18:11.147 INFO:teuthology.orchestra.run.smithi117.stdout: "total_workers": 0, 2024-04-16T17:18:11.147 INFO:teuthology.orchestra.run.smithi117.stdout: "unhealthy_workers": 0 2024-04-16T17:18:11.148 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-16T17:18:11.148 INFO:teuthology.orchestra.run.smithi117.stdout: "mempool": { 2024-04-16T17:18:11.148 INFO:teuthology.orchestra.run.smithi117.stdout: "bloom_filter_bytes": 0, 2024-04-16T17:18:11.148 INFO:teuthology.orchestra.run.smithi117.stdout: "bloom_filter_items": 0, 2024-04-16T17:18:11.148 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_alloc_bytes": 0, 2024-04-16T17:18:11.148 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_alloc_items": 0, 2024-04-16T17:18:11.148 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_data_bytes": 0, 2024-04-16T17:18:11.148 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_data_items": 0, 2024-04-16T17:18:11.148 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-16T17:18:11.149 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_onode_items": 0, 2024-04-16T17:18:11.149 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-16T17:18:11.149 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_meta_items": 0, 2024-04-16T17:18:11.149 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_other_bytes": 0, 2024-04-16T17:18:11.149 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_other_items": 0, 2024-04-16T17:18:11.149 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-16T17:18:11.149 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_buffer_items": 0, 2024-04-16T17:18:11.149 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_extent_bytes": 0, 2024-04-16T17:18:11.149 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_extent_items": 0, 2024-04-16T17:18:11.149 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_blob_bytes": 0, 2024-04-16T17:18:11.150 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_blob_items": 0, 2024-04-16T17:18:11.150 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-16T17:18:11.150 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_shared_blob_items": 0, 2024-04-16T17:18:11.150 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-16T17:18:11.150 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_inline_bl_items": 0, 2024-04-16T17:18:11.150 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_fsck_bytes": 0, 2024-04-16T17:18:11.150 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_fsck_items": 0, 2024-04-16T17:18:11.150 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_txc_bytes": 0, 2024-04-16T17:18:11.150 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_txc_items": 0, 2024-04-16T17:18:11.151 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-16T17:18:11.151 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_writing_deferred_items": 0, 2024-04-16T17:18:11.151 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_writing_bytes": 0, 2024-04-16T17:18:11.151 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_writing_items": 0, 2024-04-16T17:18:11.151 INFO:teuthology.orchestra.run.smithi117.stdout: "bluefs_bytes": 0, 2024-04-16T17:18:11.151 INFO:teuthology.orchestra.run.smithi117.stdout: "bluefs_items": 0, 2024-04-16T17:18:11.151 INFO:teuthology.orchestra.run.smithi117.stdout: "bluefs_file_reader_bytes": 0, 2024-04-16T17:18:11.151 INFO:teuthology.orchestra.run.smithi117.stdout: "bluefs_file_reader_items": 0, 2024-04-16T17:18:11.151 INFO:teuthology.orchestra.run.smithi117.stdout: "bluefs_file_writer_bytes": 0, 2024-04-16T17:18:11.152 INFO:teuthology.orchestra.run.smithi117.stdout: "bluefs_file_writer_items": 0, 2024-04-16T17:18:11.152 INFO:teuthology.orchestra.run.smithi117.stdout: "buffer_anon_bytes": 148670, 2024-04-16T17:18:11.152 INFO:teuthology.orchestra.run.smithi117.stdout: "buffer_anon_items": 30, 2024-04-16T17:18:11.152 INFO:teuthology.orchestra.run.smithi117.stdout: "buffer_meta_bytes": 0, 2024-04-16T17:18:11.152 INFO:teuthology.orchestra.run.smithi117.stdout: "buffer_meta_items": 0, 2024-04-16T17:18:11.152 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_bytes": 0, 2024-04-16T17:18:11.152 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_items": 0, 2024-04-16T17:18:11.152 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_mapbl_bytes": 0, 2024-04-16T17:18:11.152 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_mapbl_items": 0, 2024-04-16T17:18:11.153 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_pglog_bytes": 0, 2024-04-16T17:18:11.153 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_pglog_items": 0, 2024-04-16T17:18:11.153 INFO:teuthology.orchestra.run.smithi117.stdout: "osdmap_bytes": 0, 2024-04-16T17:18:11.153 INFO:teuthology.orchestra.run.smithi117.stdout: "osdmap_items": 0, 2024-04-16T17:18:11.153 INFO:teuthology.orchestra.run.smithi117.stdout: "osdmap_mapping_bytes": 0, 2024-04-16T17:18:11.153 INFO:teuthology.orchestra.run.smithi117.stdout: "osdmap_mapping_items": 0, 2024-04-16T17:18:11.153 INFO:teuthology.orchestra.run.smithi117.stdout: "pgmap_bytes": 0, 2024-04-16T17:18:11.153 INFO:teuthology.orchestra.run.smithi117.stdout: "pgmap_items": 0, 2024-04-16T17:18:11.153 INFO:teuthology.orchestra.run.smithi117.stdout: "mds_co_bytes": 928, 2024-04-16T17:18:11.154 INFO:teuthology.orchestra.run.smithi117.stdout: "mds_co_items": 2, 2024-04-16T17:18:11.154 INFO:teuthology.orchestra.run.smithi117.stdout: "unittest_1_bytes": 0, 2024-04-16T17:18:11.154 INFO:teuthology.orchestra.run.smithi117.stdout: "unittest_1_items": 0, 2024-04-16T17:18:11.154 INFO:teuthology.orchestra.run.smithi117.stdout: "unittest_2_bytes": 0, 2024-04-16T17:18:11.154 INFO:teuthology.orchestra.run.smithi117.stdout: "unittest_2_items": 0 2024-04-16T17:18:11.154 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-04-16T17:18:11.154 INFO:teuthology.orchestra.run.smithi117.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-16T17:18:11.154 INFO:teuthology.orchestra.run.smithi117.stdout: "val": 0, 2024-04-16T17:18:11.154 INFO:teuthology.orchestra.run.smithi117.stdout: "max": 104857600, 2024-04-16T17:18:11.155 INFO:teuthology.orchestra.run.smithi117.stdout: "get_started": 0, 2024-04-16T17:18:11.155 INFO:teuthology.orchestra.run.smithi117.stdout: "get": 261, 2024-04-16T17:18:11.155 INFO:teuthology.orchestra.run.smithi117.stdout: "get_sum": 1563846, 2024-04-16T17:18:11.155 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_fail": 0, 2024-04-16T17:18:11.155 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_success": 261, 2024-04-16T17:18:11.155 INFO:teuthology.orchestra.run.smithi117.stdout: "take": 0, 2024-04-16T17:18:11.155 INFO:teuthology.orchestra.run.smithi117.stdout: "take_sum": 0, 2024-04-16T17:18:11.155 INFO:teuthology.orchestra.run.smithi117.stdout: "put": 261, 2024-04-16T17:18:11.155 INFO:teuthology.orchestra.run.smithi117.stdout: "put_sum": 1563846, 2024-04-16T17:18:11.156 INFO:teuthology.orchestra.run.smithi117.stdout: "wait": { 2024-04-16T17:18:11.156 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-04-16T17:18:11.156 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-04-16T17:18:11.156 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-04-16T17:18:11.156 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-04-16T17:18:11.156 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-04-16T17:18:11.156 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-04-16T17:18:11.849 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.l 2024-04-16T17:18:11.849 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.l 2024-04-16T17:18:11.849 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.l 2024-04-16T17:18:11.849 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.l 2024-04-16T17:18:11.849 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.l 2024-04-16T17:18:11.850 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-04-16T17:18:11.859 INFO:tasks.kclient:Unmounting kernel clients... 2024-04-16T17:18:11.859 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-04-16T17:18:11.860 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-16T17:18:11.924 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-04-16T17:18:11.924 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-16T17:18:11.981 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-16T17:18:12.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:12.216+0000 7ff5f2800640 1 -- 172.21.15.45:0/472072313 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff5ec108610 msgr2=0x7ff5ec108a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:18:12.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:12.216+0000 7ff5f2800640 1 --2- 172.21.15.45:0/472072313 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff5ec108610 0x7ff5ec108a90 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7ff5d4009890 tx=0x7ff5d402bb70 comp rx=0 tx=0).stop 2024-04-16T17:18:12.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:12.216+0000 7ff5f2800640 1 -- 172.21.15.45:0/472072313 shutdown_connections 2024-04-16T17:18:12.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:12.216+0000 7ff5f2800640 1 --2- 172.21.15.45:0/472072313 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff5ec108610 0x7ff5ec108a90 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:12.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:12.216+0000 7ff5f2800640 1 --2- 172.21.15.45:0/472072313 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff5ec108fd0 0x7ff5ec1154e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:12.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:12.216+0000 7ff5f2800640 1 --2- 172.21.15.45:0/472072313 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff5ec10e6b0 0x7ff5ec10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:12.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:12.216+0000 7ff5f2800640 1 -- 172.21.15.45:0/472072313 >> 172.21.15.45:0/472072313 conn(0x7ff5ec075800 msgr2=0x7ff5ec106e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:18:12.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:12.216+0000 7ff5f2800640 1 -- 172.21.15.45:0/472072313 shutdown_connections 2024-04-16T17:18:12.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:12.216+0000 7ff5f2800640 1 -- 172.21.15.45:0/472072313 wait complete. 2024-04-16T17:18:12.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:12.217+0000 7ff5f2800640 1 Processor -- start 2024-04-16T17:18:12.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:12.217+0000 7ff5f2800640 1 -- start start 2024-04-16T17:18:12.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:12.217+0000 7ff5f2800640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff5ec108610 0x7ff5ec19ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:12.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:12.218+0000 7ff5f2800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff5ec108fd0 0x7ff5ec19f490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:12.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:12.218+0000 7ff5f2800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff5ec10e6b0 0x7ff5ec1a3810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:12.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:12.218+0000 7ff5f2800640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7ff5ec1189f0 con 0x7ff5ec108fd0 2024-04-16T17:18:12.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:12.218+0000 7ff5f2800640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7ff5ec118870 con 0x7ff5ec108610 2024-04-16T17:18:12.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:12.218+0000 7ff5f2800640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7ff5ec118b70 con 0x7ff5ec10e6b0 2024-04-16T17:18:12.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:12.218+0000 7ff5eb400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff5ec108610 0x7ff5ec19ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:12.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:12.218+0000 7ff5eb400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff5ec108610 0x7ff5ec19ef50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:59176/0 (socket says 172.21.15.45:59176) 2024-04-16T17:18:12.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:12.218+0000 7ff5e3e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff5ec108fd0 0x7ff5ec19f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:12.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:12.218+0000 7ff5eb400640 1 -- 172.21.15.45:0/1304953557 learned_addr learned my addr 172.21.15.45:0/1304953557 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:18:12.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:12.218+0000 7ff5ebe00640 1 --2- 172.21.15.45:0/1304953557 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff5ec10e6b0 0x7ff5ec1a3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:12.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:12.218+0000 7ff5eb400640 1 -- 172.21.15.45:0/1304953557 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff5ec10e6b0 msgr2=0x7ff5ec1a3810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:18:12.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:12.218+0000 7ff5eb400640 1 --2- 172.21.15.45:0/1304953557 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff5ec10e6b0 0x7ff5ec1a3810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:12.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:12.218+0000 7ff5eb400640 1 -- 172.21.15.45:0/1304953557 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff5ec108fd0 msgr2=0x7ff5ec19f490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:18:12.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:12.218+0000 7ff5eb400640 1 --2- 172.21.15.45:0/1304953557 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff5ec108fd0 0x7ff5ec19f490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:12.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:12.218+0000 7ff5eb400640 1 -- 172.21.15.45:0/1304953557 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5ec1a3f40 con 0x7ff5ec108610 2024-04-16T17:18:12.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:12.218+0000 7ff5e3e00640 1 --2- 172.21.15.45:0/1304953557 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff5ec108fd0 0x7ff5ec19f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:18:12.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:12.219+0000 7ff5eb400640 1 --2- 172.21.15.45:0/1304953557 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff5ec108610 0x7ff5ec19ef50 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7ff5dc00e9f0 tx=0x7ff5dc00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:18:12.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:12.219+0000 7ff5e8c00640 1 -- 172.21.15.45:0/1304953557 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5dc00ccc0 con 0x7ff5ec108610 2024-04-16T17:18:12.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:12.219+0000 7ff5f2800640 1 -- 172.21.15.45:0/1304953557 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5ec1a4230 con 0x7ff5ec108610 2024-04-16T17:18:12.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:12.219+0000 7ff5e8c00640 1 -- 172.21.15.45:0/1304953557 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff5dc0035f0 con 0x7ff5ec108610 2024-04-16T17:18:12.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:12.219+0000 7ff5e8c00640 1 -- 172.21.15.45:0/1304953557 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5dc00ce60 con 0x7ff5ec108610 2024-04-16T17:18:12.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:12.219+0000 7ff5f2800640 1 -- 172.21.15.45:0/1304953557 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5ec1abac0 con 0x7ff5ec108610 2024-04-16T17:18:12.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:12.220+0000 7ff5f2800640 1 -- 172.21.15.45:0/1304953557 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5ec069270 con 0x7ff5ec108610 2024-04-16T17:18:12.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:12.223+0000 7ff5e8c00640 1 -- 172.21.15.45:0/1304953557 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7ff5dc0037a0 con 0x7ff5ec108610 2024-04-16T17:18:12.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:12.224+0000 7ff5e8c00640 1 --2- 172.21.15.45:0/1304953557 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7ff5d00a3d20 0x7ff5d00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:12.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:12.224+0000 7ff5e8c00640 1 -- 172.21.15.45:0/1304953557 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 9584+0+0 (secure 0 0 0) 0x7ff5dc0d0dc0 con 0x7ff5ec108610 2024-04-16T17:18:12.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:12.224+0000 7ff5e3e00640 1 --2- 172.21.15.45:0/1304953557 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7ff5d00a3d20 0x7ff5d00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:12.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:12.225+0000 7ff5e3e00640 1 --2- 172.21.15.45:0/1304953557 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7ff5d00a3d20 0x7ff5d00a61e0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7ff5d402c180 tx=0x7ff5d402e040 comp rx=0 tx=0).ready entity=mgr.25411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:18:12.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:12.227+0000 7ff5e8c00640 1 -- 172.21.15.45:0/1304953557 <== mon.2 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff5dc09d000 con 0x7ff5ec108610 2024-04-16T17:18:12.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:12.435+0000 7ff5f2800640 1 -- 172.21.15.45:0/1304953557 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff5ec1a02a0 con 0x7ff5ec108610 2024-04-16T17:18:12.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:12.438+0000 7ff5e8c00640 1 -- 172.21.15.45:0/1304953557 <== mon.2 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v107) ==== 74+0+25420 (secure 0 0 0) 0x7ff5dc09c900 con 0x7ff5ec108610 2024-04-16T17:18:12.439 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-16T17:18:12.439 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":107,"fsid":"b1fecd38-fc11-11ee-bc90-c7b262605968","created":"2024-04-16T16:54:02.385906+0000","modified":"2024-04-16T17:18:08.588246+0000","last_up_change":"2024-04-16T17:02:25.162624+0000","last_in_change":"2024-04-16T17:02:04.903274+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-16T16:58:06.166659+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-16T17:03:22.304198+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-16T17:03:23.616450+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"107","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":19,"snap_epoch":107,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-16T17:03:27.226289+0000","flags":8197,"flags_names":"hashpspool,ec_overwrites,selfmanaged_snaps","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"107","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":19,"snap_epoch":107,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b655de2c-59f8-4fd4-ac98-0cc222229f56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":2290170881},{"type":"v1","addr":"172.21.15.45:6803","nonce":2290170881}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":2290170881},{"type":"v1","addr":"172.21.15.45:6805","nonce":2290170881}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":2290170881},{"type":"v1","addr":"172.21.15.45:6809","nonce":2290170881}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":2290170881},{"type":"v1","addr":"172.21.15.45:6807","nonce":2290170881}]},"public_addr":"172.21.15.45:6803/2290170881","cluster_addr":"172.21.15.45:6805/2290170881","heartbeat_back_addr":"172.21.15.45:6809/2290170881","heartbeat_front_addr":"172.21.15.45:6807/2290170881","state":["exists","up"]},{"osd":1,"uuid":"ebd450ec-08fd-4c3d-8b24-4e10634ddb7e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6800","nonce":1462942067},{"type":"v1","addr":"172.21.15.104:6801","nonce":1462942067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":1462942067},{"type":"v1","addr":"172.21.15.104:6803","nonce":1462942067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":1462942067},{"type":"v1","addr":"172.21.15.104:6807","nonce":1462942067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":1462942067},{"type":"v1","addr":"172.21.15.104:6805","nonce":1462942067}]},"public_addr":"172.21.15.104:6801/1462942067","cluster_addr":"172.21.15.104:6803/1462942067","heartbeat_back_addr":"172.21.15.104:6807/1462942067","heartbeat_front_addr":"172.21.15.104:6805/1462942067","state":["exists","up"]},{"osd":2,"uuid":"b318a512-4190-42a2-8104-dcc4026ad8ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":2829260735},{"type":"v1","addr":"172.21.15.117:6801","nonce":2829260735}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":2829260735},{"type":"v1","addr":"172.21.15.117:6803","nonce":2829260735}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":2829260735},{"type":"v1","addr":"172.21.15.117:6807","nonce":2829260735}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":2829260735},{"type":"v1","addr":"172.21.15.117:6805","nonce":2829260735}]},"public_addr":"172.21.15.117:6801/2829260735","cluster_addr":"172.21.15.117:6803/2829260735","heartbeat_back_addr":"172.21.15.117:6807/2829260735","heartbeat_front_addr":"172.21.15.117:6805/2829260735","state":["exists","up"]},{"osd":3,"uuid":"ec6490cb-a715-4c12-a54a-225aae26fcb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":2823712137},{"type":"v1","addr":"172.21.15.45:6811","nonce":2823712137}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":2823712137},{"type":"v1","addr":"172.21.15.45:6813","nonce":2823712137}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":2823712137},{"type":"v1","addr":"172.21.15.45:6817","nonce":2823712137}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":2823712137},{"type":"v1","addr":"172.21.15.45:6815","nonce":2823712137}]},"public_addr":"172.21.15.45:6811/2823712137","cluster_addr":"172.21.15.45:6813/2823712137","heartbeat_back_addr":"172.21.15.45:6817/2823712137","heartbeat_front_addr":"172.21.15.45:6815/2823712137","state":["exists","up"]},{"osd":4,"uuid":"5d91d568-0250-4aae-8dc2-a0e079f92062","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":1396278127},{"type":"v1","addr":"172.21.15.104:6809","nonce":1396278127}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":1396278127},{"type":"v1","addr":"172.21.15.104:6811","nonce":1396278127}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":1396278127},{"type":"v1","addr":"172.21.15.104:6815","nonce":1396278127}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":1396278127},{"type":"v1","addr":"172.21.15.104:6813","nonce":1396278127}]},"public_addr":"172.21.15.104:6809/1396278127","cluster_addr":"172.21.15.104:6811/1396278127","heartbeat_back_addr":"172.21.15.104:6815/1396278127","heartbeat_front_addr":"172.21.15.104:6813/1396278127","state":["exists","up"]},{"osd":5,"uuid":"ac510b43-6fb2-4d0f-bc5f-16ac080d9e9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":4145720047},{"type":"v1","addr":"172.21.15.117:6809","nonce":4145720047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":4145720047},{"type":"v1","addr":"172.21.15.117:6811","nonce":4145720047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":4145720047},{"type":"v1","addr":"172.21.15.117:6815","nonce":4145720047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":4145720047},{"type":"v1","addr":"172.21.15.117:6813","nonce":4145720047}]},"public_addr":"172.21.15.117:6809/4145720047","cluster_addr":"172.21.15.117:6811/4145720047","heartbeat_back_addr":"172.21.15.117:6815/4145720047","heartbeat_front_addr":"172.21.15.117:6813/4145720047","state":["exists","up"]},{"osd":6,"uuid":"a41adc18-d88a-419e-b2c2-b089c1f32472","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":3254229176},{"type":"v1","addr":"172.21.15.45:6819","nonce":3254229176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":3254229176},{"type":"v1","addr":"172.21.15.45:6821","nonce":3254229176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":3254229176},{"type":"v1","addr":"172.21.15.45:6825","nonce":3254229176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":3254229176},{"type":"v1","addr":"172.21.15.45:6823","nonce":3254229176}]},"public_addr":"172.21.15.45:6819/3254229176","cluster_addr":"172.21.15.45:6821/3254229176","heartbeat_back_addr":"172.21.15.45:6825/3254229176","heartbeat_front_addr":"172.21.15.45:6823/3254229176","state":["exists","up"]},{"osd":7,"uuid":"b71712a4-7cf6-454d-9fd1-565e3df21c1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":4151077186},{"type":"v1","addr":"172.21.15.104:6817","nonce":4151077186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":4151077186},{"type":"v1","addr":"172.21.15.104:6819","nonce":4151077186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":4151077186},{"type":"v1","addr":"172.21.15.104:6823","nonce":4151077186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":4151077186},{"type":"v1","addr":"172.21.15.104:6821","nonce":4151077186}]},"public_addr":"172.21.15.104:6817/4151077186","cluster_addr":"172.21.15.104:6819/4151077186","heartbeat_back_addr":"172.21.15.104:6823/4151077186","heartbeat_front_addr":"172.21.15.104:6821/4151077186","state":["exists","up"]},{"osd":8,"uuid":"9e572aef-59a8-450e-a92e-df9dff4d57a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":1500286830},{"type":"v1","addr":"172.21.15.117:6817","nonce":1500286830}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":1500286830},{"type":"v1","addr":"172.21.15.117:6819","nonce":1500286830}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":1500286830},{"type":"v1","addr":"172.21.15.117:6823","nonce":1500286830}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":1500286830},{"type":"v1","addr":"172.21.15.117:6821","nonce":1500286830}]},"public_addr":"172.21.15.117:6817/1500286830","cluster_addr":"172.21.15.117:6819/1500286830","heartbeat_back_addr":"172.21.15.117:6823/1500286830","heartbeat_front_addr":"172.21.15.117:6821/1500286830","state":["exists","up"]},{"osd":9,"uuid":"014e9598-c413-4cc4-9bb8-61c6e8f59d30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":3106734074},{"type":"v1","addr":"172.21.15.45:6827","nonce":3106734074}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":3106734074},{"type":"v1","addr":"172.21.15.45:6829","nonce":3106734074}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":3106734074},{"type":"v1","addr":"172.21.15.45:6833","nonce":3106734074}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":3106734074},{"type":"v1","addr":"172.21.15.45:6831","nonce":3106734074}]},"public_addr":"172.21.15.45:6827/3106734074","cluster_addr":"172.21.15.45:6829/3106734074","heartbeat_back_addr":"172.21.15.45:6833/3106734074","heartbeat_front_addr":"172.21.15.45:6831/3106734074","state":["exists","up"]},{"osd":10,"uuid":"2deba364-3e39-47be-8a1f-8256df5b82d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":1404794652},{"type":"v1","addr":"172.21.15.104:6825","nonce":1404794652}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":1404794652},{"type":"v1","addr":"172.21.15.104:6827","nonce":1404794652}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":1404794652},{"type":"v1","addr":"172.21.15.104:6831","nonce":1404794652}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":1404794652},{"type":"v1","addr":"172.21.15.104:6829","nonce":1404794652}]},"public_addr":"172.21.15.104:6825/1404794652","cluster_addr":"172.21.15.104:6827/1404794652","heartbeat_back_addr":"172.21.15.104:6831/1404794652","heartbeat_front_addr":"172.21.15.104:6829/1404794652","state":["exists","up"]},{"osd":11,"uuid":"3fda4fea-59ad-4d1a-9fe6-fcb30e31fd7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":3384003243},{"type":"v1","addr":"172.21.15.117:6825","nonce":3384003243}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":3384003243},{"type":"v1","addr":"172.21.15.117:6827","nonce":3384003243}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":3384003243},{"type":"v1","addr":"172.21.15.117:6831","nonce":3384003243}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":3384003243},{"type":"v1","addr":"172.21.15.117:6829","nonce":3384003243}]},"public_addr":"172.21.15.117:6825/3384003243","cluster_addr":"172.21.15.117:6827/3384003243","heartbeat_back_addr":"172.21.15.117:6831/3384003243","heartbeat_front_addr":"172.21.15.117:6829/3384003243","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:57:14.799608+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:57:39.884668+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:58:03.327057+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:58:37.314531+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:59:03.667650+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:59:28.462823+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T17:00:02.968574+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T17:00:29.559190+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T17:00:55.291007+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T17:01:29.815977+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T17:01:57.015580+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T17:02:23.233151+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.5","mappings":[{"from":9,"to":10}]},{"pgid":"4.7","mappings":[{"from":5,"to":8}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.45:6800/268030699":"2024-04-17T17:06:45.093215+0000","172.21.15.45:6801/268030699":"2024-04-17T17:06:45.093215+0000","172.21.15.45:0/4074982598":"2024-04-17T16:55:19.948001+0000","172.21.15.45:0/1915281176":"2024-04-17T17:06:45.093215+0000","172.21.15.45:0/2443572576":"2024-04-17T16:55:19.948001+0000","172.21.15.45:0/1500147666":"2024-04-17T17:06:45.093215+0000","172.21.15.45:6801/1983630136":"2024-04-17T16:55:19.948001+0000","172.21.15.45:6800/1983630136":"2024-04-17T16:55:19.948001+0000","172.21.15.45:0/12001708":"2024-04-17T17:06:45.093215+0000","172.21.15.45:0/3150465248":"2024-04-17T16:54:43.692466+0000","172.21.15.45:0/556738623":"2024-04-17T16:54:43.692466+0000","172.21.15.45:0/445676297":"2024-04-17T16:55:19.948001+0000","172.21.15.45:6800/2553659622":"2024-04-17T16:54:43.692466+0000","172.21.15.45:6801/2553659622":"2024-04-17T16:54:43.692466+0000","172.21.15.45:0/265066921":"2024-04-17T17:06:45.093215+0000","172.21.15.45:0/1184615408":"2024-04-17T16:54:43.692466+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":3,"snaps":[{"begin":7,"length":1},{"begin":19,"length":1}]},{"pool":4,"snaps":[{"begin":7,"length":1},{"begin":19,"length":1}]}],"new_removed_snaps":[{"pool":3,"snaps":[{"begin":7,"length":1},{"begin":19,"length":1}]},{"pool":4,"snaps":[{"begin":7,"length":1},{"begin":19,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-16T17:18:12.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:12.440+0000 7ff5f2800640 1 -- 172.21.15.45:0/1304953557 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7ff5d00a3d20 msgr2=0x7ff5d00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:18:12.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:12.440+0000 7ff5f2800640 1 --2- 172.21.15.45:0/1304953557 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7ff5d00a3d20 0x7ff5d00a61e0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7ff5d402c180 tx=0x7ff5d402e040 comp rx=0 tx=0).stop 2024-04-16T17:18:12.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:12.440+0000 7ff5f2800640 1 -- 172.21.15.45:0/1304953557 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff5ec108610 msgr2=0x7ff5ec19ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:18:12.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:12.440+0000 7ff5f2800640 1 --2- 172.21.15.45:0/1304953557 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff5ec108610 0x7ff5ec19ef50 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7ff5dc00e9f0 tx=0x7ff5dc00eeb0 comp rx=0 tx=0).stop 2024-04-16T17:18:12.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:12.440+0000 7ff5f2800640 1 -- 172.21.15.45:0/1304953557 shutdown_connections 2024-04-16T17:18:12.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:12.440+0000 7ff5f2800640 1 --2- 172.21.15.45:0/1304953557 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7ff5d00a3d20 0x7ff5d00a61e0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:12.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:12.440+0000 7ff5f2800640 1 --2- 172.21.15.45:0/1304953557 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff5ec10e6b0 0x7ff5ec1a3810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:12.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:12.440+0000 7ff5f2800640 1 --2- 172.21.15.45:0/1304953557 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff5ec108fd0 0x7ff5ec19f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:12.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:12.440+0000 7ff5f2800640 1 --2- 172.21.15.45:0/1304953557 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff5ec108610 0x7ff5ec19ef50 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:12.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:12.440+0000 7ff5f2800640 1 -- 172.21.15.45:0/1304953557 >> 172.21.15.45:0/1304953557 conn(0x7ff5ec075800 msgr2=0x7ff5ec076880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:18:12.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:12.440+0000 7ff5f2800640 1 -- 172.21.15.45:0/1304953557 shutdown_connections 2024-04-16T17:18:12.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:12.441+0000 7ff5f2800640 1 -- 172.21.15.45:0/1304953557 wait complete. 2024-04-16T17:18:12.462 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-16T16:58:06.166659+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-16T17:03:22.304198+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-16T17:03:23.616450+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '107', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 19, 'snap_epoch': 107, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-16T17:03:27.226289+0000', 'flags': 8197, 'flags_names': 'hashpspool,ec_overwrites,selfmanaged_snaps', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '107', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 19, 'snap_epoch': 107, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-16T17:18:12.462 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-16T17:18:12.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:12.697+0000 7ff74fe00640 1 -- 172.21.15.45:0/1516691973 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff75010e490 msgr2=0x7ff75010e870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:18:12.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:12.697+0000 7ff74fe00640 1 --2- 172.21.15.45:0/1516691973 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff75010e490 0x7ff75010e870 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7ff740009900 tx=0x7ff74002bc30 comp rx=0 tx=0).stop 2024-04-16T17:18:12.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:12.697+0000 7ff74fe00640 1 -- 172.21.15.45:0/1516691973 shutdown_connections 2024-04-16T17:18:12.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:12.697+0000 7ff74fe00640 1 --2- 172.21.15.45:0/1516691973 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff750108400 0x7ff750108880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:12.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:12.697+0000 7ff74fe00640 1 --2- 172.21.15.45:0/1516691973 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff750108dc0 0x7ff7501152c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:12.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:12.697+0000 7ff74fe00640 1 --2- 172.21.15.45:0/1516691973 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff75010e490 0x7ff75010e870 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:12.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:12.697+0000 7ff74fe00640 1 -- 172.21.15.45:0/1516691973 >> 172.21.15.45:0/1516691973 conn(0x7ff750077c10 msgr2=0x7ff75007a030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:18:12.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:12.697+0000 7ff74fe00640 1 -- 172.21.15.45:0/1516691973 shutdown_connections 2024-04-16T17:18:12.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:12.697+0000 7ff74fe00640 1 -- 172.21.15.45:0/1516691973 wait complete. 2024-04-16T17:18:12.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:12.698+0000 7ff74fe00640 1 Processor -- start 2024-04-16T17:18:12.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:12.698+0000 7ff74fe00640 1 -- start start 2024-04-16T17:18:12.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:12.698+0000 7ff74fe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff750108400 0x7ff7501a4900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:12.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:12.698+0000 7ff74fe00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff750108dc0 0x7ff7501a4e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:12.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:12.698+0000 7ff74fe00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff75010e490 0x7ff75019eab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:12.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:12.698+0000 7ff74fe00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7ff7501187b0 con 0x7ff750108400 2024-04-16T17:18:12.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:12.698+0000 7ff74fe00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7ff750118630 con 0x7ff750108dc0 2024-04-16T17:18:12.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:12.698+0000 7ff74fe00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7ff750118930 con 0x7ff75010e490 2024-04-16T17:18:12.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:12.699+0000 7ff74ea00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff750108400 0x7ff7501a4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:12.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:12.699+0000 7ff74ea00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff750108400 0x7ff7501a4900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59800/0 (socket says 172.21.15.45:59800) 2024-04-16T17:18:12.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:12.699+0000 7ff74ea00640 1 -- 172.21.15.45:0/2900844816 learned_addr learned my addr 172.21.15.45:0/2900844816 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:18:12.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:12.699+0000 7ff74e000640 1 --2- 172.21.15.45:0/2900844816 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff750108dc0 0x7ff7501a4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:12.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:12.699+0000 7ff74f400640 1 --2- 172.21.15.45:0/2900844816 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff75010e490 0x7ff75019eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:12.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:12.699+0000 7ff74ea00640 1 -- 172.21.15.45:0/2900844816 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff75010e490 msgr2=0x7ff75019eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:18:12.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:12.699+0000 7ff74ea00640 1 --2- 172.21.15.45:0/2900844816 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff75010e490 0x7ff75019eab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:12.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:12.699+0000 7ff74ea00640 1 -- 172.21.15.45:0/2900844816 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff750108dc0 msgr2=0x7ff7501a4e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:18:12.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:12.699+0000 7ff74ea00640 1 --2- 172.21.15.45:0/2900844816 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff750108dc0 0x7ff7501a4e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:12.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:12.699+0000 7ff74ea00640 1 -- 172.21.15.45:0/2900844816 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff75019f230 con 0x7ff750108400 2024-04-16T17:18:12.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:12.700+0000 7ff74ea00640 1 --2- 172.21.15.45:0/2900844816 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff750108400 0x7ff7501a4900 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7ff740005940 tx=0x7ff740004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:18:12.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:12.700+0000 7ff747400640 1 -- 172.21.15.45:0/2900844816 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff740041070 con 0x7ff750108400 2024-04-16T17:18:12.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:12.700+0000 7ff747400640 1 -- 172.21.15.45:0/2900844816 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff7400054e0 con 0x7ff750108400 2024-04-16T17:18:12.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:12.700+0000 7ff747400640 1 -- 172.21.15.45:0/2900844816 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff740003d00 con 0x7ff750108400 2024-04-16T17:18:12.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:12.700+0000 7ff74fe00640 1 -- 172.21.15.45:0/2900844816 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff75019f4c0 con 0x7ff750108400 2024-04-16T17:18:12.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:12.700+0000 7ff74fe00640 1 -- 172.21.15.45:0/2900844816 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7501ab7c0 con 0x7ff750108400 2024-04-16T17:18:12.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:12.701+0000 7ff74fe00640 1 -- 172.21.15.45:0/2900844816 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff750069190 con 0x7ff750108400 2024-04-16T17:18:12.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:12.703+0000 7ff747400640 1 -- 172.21.15.45:0/2900844816 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7ff740044070 con 0x7ff750108400 2024-04-16T17:18:12.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:12.706+0000 7ff747400640 1 --2- 172.21.15.45:0/2900844816 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7ff7340a3d20 0x7ff7340a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:12.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:12.706+0000 7ff747400640 1 -- 172.21.15.45:0/2900844816 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 9584+0+0 (secure 0 0 0) 0x7ff7400f27c0 con 0x7ff750108400 2024-04-16T17:18:12.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:12.706+0000 7ff74e000640 1 --2- 172.21.15.45:0/2900844816 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7ff7340a3d20 0x7ff7340a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:12.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:12.707+0000 7ff74e000640 1 --2- 172.21.15.45:0/2900844816 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7ff7340a3d20 0x7ff7340a61e0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7ff75007a330 tx=0x7ff73c005f90 comp rx=0 tx=0).ready entity=mgr.25411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:18:12.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:12.711+0000 7ff747400640 1 -- 172.21.15.45:0/2900844816 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff7400be4c0 con 0x7ff750108400 2024-04-16T17:18:12.726 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:18:12 smithi045 ceph-mon[31676]: pgmap v358: 129 pgs: 129 active+clean; 51 MiB data, 871 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-16T17:18:12.726 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:18:12 smithi045 ceph-mon[31676]: from='client.? 172.21.15.45:0/1304953557' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T17:18:12.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:12.919+0000 7ff74fe00640 1 -- 172.21.15.45:0/2900844816 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7ff750110960 con 0x7ff750108400 2024-04-16T17:18:12.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:12.919+0000 7ff747400640 1 -- 172.21.15.45:0/2900844816 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v107) ==== 93+0+10 (secure 0 0 0) 0x7ff7400c27c0 con 0x7ff750108400 2024-04-16T17:18:12.920 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 1 2024-04-16T17:18:12.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:12.922+0000 7ff74fe00640 1 -- 172.21.15.45:0/2900844816 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7ff7340a3d20 msgr2=0x7ff7340a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:18:12.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:12.922+0000 7ff74fe00640 1 --2- 172.21.15.45:0/2900844816 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7ff7340a3d20 0x7ff7340a61e0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7ff75007a330 tx=0x7ff73c005f90 comp rx=0 tx=0).stop 2024-04-16T17:18:12.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:12.922+0000 7ff74fe00640 1 -- 172.21.15.45:0/2900844816 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff750108400 msgr2=0x7ff7501a4900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:18:12.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:12.922+0000 7ff74fe00640 1 --2- 172.21.15.45:0/2900844816 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff750108400 0x7ff7501a4900 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7ff740005940 tx=0x7ff740004270 comp rx=0 tx=0).stop 2024-04-16T17:18:12.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:12.922+0000 7ff74fe00640 1 -- 172.21.15.45:0/2900844816 shutdown_connections 2024-04-16T17:18:12.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:12.922+0000 7ff74fe00640 1 --2- 172.21.15.45:0/2900844816 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7ff7340a3d20 0x7ff7340a61e0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:12.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:12.922+0000 7ff74fe00640 1 --2- 172.21.15.45:0/2900844816 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff75010e490 0x7ff75019eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:12.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:12.922+0000 7ff74fe00640 1 --2- 172.21.15.45:0/2900844816 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff750108dc0 0x7ff7501a4e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:12.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:12.922+0000 7ff74fe00640 1 --2- 172.21.15.45:0/2900844816 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff750108400 0x7ff7501a4900 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:12.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:12.922+0000 7ff74fe00640 1 -- 172.21.15.45:0/2900844816 >> 172.21.15.45:0/2900844816 conn(0x7ff750077c10 msgr2=0x7ff750078450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:18:12.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:12.922+0000 7ff74fe00640 1 -- 172.21.15.45:0/2900844816 shutdown_connections 2024-04-16T17:18:12.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:12.922+0000 7ff74fe00640 1 -- 172.21.15.45:0/2900844816 wait complete. 2024-04-16T17:18:12.926 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:18:12 smithi104 ceph-mon[36457]: pgmap v358: 129 pgs: 129 active+clean; 51 MiB data, 871 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-16T17:18:12.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:18:12 smithi104 ceph-mon[36457]: from='client.? 172.21.15.45:0/1304953557' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T17:18:12.940 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-16T17:18:13.054 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:18:12 smithi117 ceph-mon[36730]: pgmap v358: 129 pgs: 129 active+clean; 51 MiB data, 871 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-16T17:18:13.054 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:18:12 smithi117 ceph-mon[36730]: from='client.? 172.21.15.45:0/1304953557' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T17:18:13.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:13.177+0000 7ffb7aa00640 1 -- 172.21.15.45:0/107014394 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffb74101400 msgr2=0x7ffb74106b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:18:13.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:13.177+0000 7ffb7aa00640 1 --2- 172.21.15.45:0/107014394 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffb74101400 0x7ffb74106b30 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7ffb60009900 tx=0x7ffb6002bc30 comp rx=0 tx=0).stop 2024-04-16T17:18:13.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:13.177+0000 7ffb7aa00640 1 -- 172.21.15.45:0/107014394 shutdown_connections 2024-04-16T17:18:13.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:13.177+0000 7ffb7aa00640 1 --2- 172.21.15.45:0/107014394 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffb74101400 0x7ffb74106b30 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:13.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:13.177+0000 7ffb7aa00640 1 --2- 172.21.15.45:0/107014394 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffb74107260 0x7ffb7410ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:13.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:13.177+0000 7ffb7aa00640 1 --2- 172.21.15.45:0/107014394 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ffb74100a50 0x7ffb74100e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:13.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:13.177+0000 7ffb7aa00640 1 -- 172.21.15.45:0/107014394 >> 172.21.15.45:0/107014394 conn(0x7ffb74076090 msgr2=0x7ffb740764a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:18:13.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:13.177+0000 7ffb7aa00640 1 -- 172.21.15.45:0/107014394 shutdown_connections 2024-04-16T17:18:13.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:13.177+0000 7ffb7aa00640 1 -- 172.21.15.45:0/107014394 wait complete. 2024-04-16T17:18:13.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:13.178+0000 7ffb7aa00640 1 Processor -- start 2024-04-16T17:18:13.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:13.178+0000 7ffb7aa00640 1 -- start start 2024-04-16T17:18:13.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:13.178+0000 7ffb7aa00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ffb74100a50 0x7ffb7419e430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:13.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:13.178+0000 7ffb7aa00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffb74107260 0x7ffb7419e970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:13.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:13.178+0000 7ffb7aa00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffb741985e0 0x7ffb74198a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:13.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:13.178+0000 7ffb7aa00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7ffb74112280 con 0x7ffb741985e0 2024-04-16T17:18:13.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:13.178+0000 7ffb7aa00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7ffb74112100 con 0x7ffb74100a50 2024-04-16T17:18:13.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:13.178+0000 7ffb7aa00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7ffb74112400 con 0x7ffb74107260 2024-04-16T17:18:13.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:13.179+0000 7ffb79600640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ffb74100a50 0x7ffb7419e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:13.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:13.179+0000 7ffb7a000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffb741985e0 0x7ffb74198a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:13.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:13.179+0000 7ffb79600640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ffb74100a50 0x7ffb7419e430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:59208/0 (socket says 172.21.15.45:59208) 2024-04-16T17:18:13.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:13.179+0000 7ffb79600640 1 -- 172.21.15.45:0/2317997556 learned_addr learned my addr 172.21.15.45:0/2317997556 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:18:13.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:13.179+0000 7ffb78c00640 1 --2- 172.21.15.45:0/2317997556 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffb74107260 0x7ffb7419e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:13.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:13.179+0000 7ffb7a000640 1 -- 172.21.15.45:0/2317997556 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffb74107260 msgr2=0x7ffb7419e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:18:13.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:13.179+0000 7ffb7a000640 1 --2- 172.21.15.45:0/2317997556 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffb74107260 0x7ffb7419e970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:13.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:13.179+0000 7ffb7a000640 1 -- 172.21.15.45:0/2317997556 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ffb74100a50 msgr2=0x7ffb7419e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:18:13.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:13.179+0000 7ffb7a000640 1 --2- 172.21.15.45:0/2317997556 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ffb74100a50 0x7ffb7419e430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:13.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:13.179+0000 7ffb7a000640 1 -- 172.21.15.45:0/2317997556 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb74199100 con 0x7ffb741985e0 2024-04-16T17:18:13.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:13.179+0000 7ffb78c00640 1 --2- 172.21.15.45:0/2317997556 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffb74107260 0x7ffb7419e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:18:13.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:13.179+0000 7ffb7a000640 1 --2- 172.21.15.45:0/2317997556 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffb741985e0 0x7ffb74198a90 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7ffb6c009fd0 tx=0x7ffb6c00ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:18:13.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:13.180+0000 7ffb72000640 1 -- 172.21.15.45:0/2317997556 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb6c00ce20 con 0x7ffb741985e0 2024-04-16T17:18:13.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:13.180+0000 7ffb72000640 1 -- 172.21.15.45:0/2317997556 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffb6c005020 con 0x7ffb741985e0 2024-04-16T17:18:13.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:13.180+0000 7ffb72000640 1 -- 172.21.15.45:0/2317997556 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb6c010720 con 0x7ffb741985e0 2024-04-16T17:18:13.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:13.180+0000 7ffb7aa00640 1 -- 172.21.15.45:0/2317997556 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb741993f0 con 0x7ffb741985e0 2024-04-16T17:18:13.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:13.180+0000 7ffb7aa00640 1 -- 172.21.15.45:0/2317997556 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb741a55f0 con 0x7ffb741985e0 2024-04-16T17:18:13.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:13.181+0000 7ffb7aa00640 1 -- 172.21.15.45:0/2317997556 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb7406a110 con 0x7ffb741985e0 2024-04-16T17:18:13.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:13.183+0000 7ffb72000640 1 -- 172.21.15.45:0/2317997556 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7ffb6c005540 con 0x7ffb741985e0 2024-04-16T17:18:13.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:13.184+0000 7ffb72000640 1 --2- 172.21.15.45:0/2317997556 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7ffb5c0a3d20 0x7ffb5c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:13.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:13.185+0000 7ffb72000640 1 -- 172.21.15.45:0/2317997556 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 9584+0+0 (secure 0 0 0) 0x7ffb6c0d12d0 con 0x7ffb741985e0 2024-04-16T17:18:13.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:13.185+0000 7ffb79600640 1 --2- 172.21.15.45:0/2317997556 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7ffb5c0a3d20 0x7ffb5c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:13.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:13.186+0000 7ffb79600640 1 --2- 172.21.15.45:0/2317997556 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7ffb5c0a3d20 0x7ffb5c0a61e0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7ffb68009cd0 tx=0x7ffb68005f90 comp rx=0 tx=0).ready entity=mgr.25411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:18:13.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:13.189+0000 7ffb72000640 1 -- 172.21.15.45:0/2317997556 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ffb6c09d510 con 0x7ffb741985e0 2024-04-16T17:18:13.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:13.396+0000 7ffb7aa00640 1 -- 172.21.15.45:0/2317997556 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7ffb74101400 con 0x7ffb741985e0 2024-04-16T17:18:13.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:13.397+0000 7ffb72000640 1 -- 172.21.15.45:0/2317997556 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v107) ==== 104+0+11 (secure 0 0 0) 0x7ffb6c09ce10 con 0x7ffb741985e0 2024-04-16T17:18:13.397 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 32 2024-04-16T17:18:13.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:13.399+0000 7ffb7aa00640 1 -- 172.21.15.45:0/2317997556 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7ffb5c0a3d20 msgr2=0x7ffb5c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:18:13.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:13.399+0000 7ffb7aa00640 1 --2- 172.21.15.45:0/2317997556 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7ffb5c0a3d20 0x7ffb5c0a61e0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7ffb68009cd0 tx=0x7ffb68005f90 comp rx=0 tx=0).stop 2024-04-16T17:18:13.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:13.399+0000 7ffb7aa00640 1 -- 172.21.15.45:0/2317997556 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffb741985e0 msgr2=0x7ffb74198a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:18:13.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:13.399+0000 7ffb7aa00640 1 --2- 172.21.15.45:0/2317997556 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffb741985e0 0x7ffb74198a90 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7ffb6c009fd0 tx=0x7ffb6c00ef10 comp rx=0 tx=0).stop 2024-04-16T17:18:13.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:13.400+0000 7ffb7aa00640 1 -- 172.21.15.45:0/2317997556 shutdown_connections 2024-04-16T17:18:13.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:13.400+0000 7ffb7aa00640 1 --2- 172.21.15.45:0/2317997556 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7ffb5c0a3d20 0x7ffb5c0a61e0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:13.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:13.400+0000 7ffb7aa00640 1 --2- 172.21.15.45:0/2317997556 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffb74107260 0x7ffb7419e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:13.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:13.400+0000 7ffb7aa00640 1 --2- 172.21.15.45:0/2317997556 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffb741985e0 0x7ffb74198a90 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:13.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:13.400+0000 7ffb7aa00640 1 --2- 172.21.15.45:0/2317997556 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ffb74100a50 0x7ffb7419e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:13.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:13.400+0000 7ffb7aa00640 1 -- 172.21.15.45:0/2317997556 >> 172.21.15.45:0/2317997556 conn(0x7ffb74076090 msgr2=0x7ffb740fea60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:18:13.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:13.400+0000 7ffb7aa00640 1 -- 172.21.15.45:0/2317997556 shutdown_connections 2024-04-16T17:18:13.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:13.400+0000 7ffb7aa00640 1 -- 172.21.15.45:0/2317997556 wait complete. 2024-04-16T17:18:13.420 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-16T17:18:13.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:13.634+0000 7f8099600640 1 -- 172.21.15.45:0/2899352214 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8094103060 msgr2=0x7f8094103440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:18:13.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:13.634+0000 7f8099600640 1 --2- 172.21.15.45:0/2899352214 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8094103060 0x7f8094103440 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f8084009900 tx=0x7f808402bcd0 comp rx=0 tx=0).stop 2024-04-16T17:18:13.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:13.634+0000 7f8099600640 1 -- 172.21.15.45:0/2899352214 shutdown_connections 2024-04-16T17:18:13.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:13.634+0000 7f8099600640 1 --2- 172.21.15.45:0/2899352214 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f809410c480 0x7f809410e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:13.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:13.634+0000 7f8099600640 1 --2- 172.21.15.45:0/2899352214 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8094103980 0x7f809410be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:13.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:13.634+0000 7f8099600640 1 --2- 172.21.15.45:0/2899352214 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8094103060 0x7f8094103440 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:13.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:13.634+0000 7f8099600640 1 -- 172.21.15.45:0/2899352214 >> 172.21.15.45:0/2899352214 conn(0x7f80940fc880 msgr2=0x7f80940feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:18:13.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:13.634+0000 7f8099600640 1 -- 172.21.15.45:0/2899352214 shutdown_connections 2024-04-16T17:18:13.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:13.634+0000 7f8099600640 1 -- 172.21.15.45:0/2899352214 wait complete. 2024-04-16T17:18:13.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:13.636+0000 7f8099600640 1 Processor -- start 2024-04-16T17:18:13.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:13.636+0000 7f8099600640 1 -- start start 2024-04-16T17:18:13.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:13.636+0000 7f8099600640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8094103060 0x7f8094100cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:13.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:13.636+0000 7f8099600640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8094103980 0x7f80941011f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:13.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:13.637+0000 7f8099600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f809410c480 0x7f8094101730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:13.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:13.637+0000 7f8099600640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f8094112290 con 0x7f809410c480 2024-04-16T17:18:13.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:13.637+0000 7f8099600640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f8094112110 con 0x7f8094103980 2024-04-16T17:18:13.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:13.637+0000 7f8099600640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f8094112410 con 0x7f8094103060 2024-04-16T17:18:13.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:13.637+0000 7f808be00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8094103980 0x7f80941011f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:13.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:13.637+0000 7f8093e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8094103060 0x7f8094100cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:13.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:13.637+0000 7f808be00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8094103980 0x7f80941011f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:59234/0 (socket says 172.21.15.45:59234) 2024-04-16T17:18:13.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:13.637+0000 7f808be00640 1 -- 172.21.15.45:0/1035673585 learned_addr learned my addr 172.21.15.45:0/1035673585 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:18:13.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:13.637+0000 7f8098c00640 1 --2- 172.21.15.45:0/1035673585 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f809410c480 0x7f8094101730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:13.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:13.637+0000 7f808be00640 1 -- 172.21.15.45:0/1035673585 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8094103060 msgr2=0x7f8094100cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:18:13.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:13.637+0000 7f808be00640 1 --2- 172.21.15.45:0/1035673585 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8094103060 0x7f8094100cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:13.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:13.637+0000 7f808be00640 1 -- 172.21.15.45:0/1035673585 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f809410c480 msgr2=0x7f8094101730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:18:13.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:13.637+0000 7f808be00640 1 --2- 172.21.15.45:0/1035673585 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f809410c480 0x7f8094101730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:13.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:13.637+0000 7f808be00640 1 -- 172.21.15.45:0/1035673585 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80941a6c30 con 0x7f8094103980 2024-04-16T17:18:13.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:13.638+0000 7f808be00640 1 --2- 172.21.15.45:0/1035673585 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8094103980 0x7f80941011f0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f807c00e9f0 tx=0x7f807c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:18:13.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:13.638+0000 7f8091600640 1 -- 172.21.15.45:0/1035673585 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f807c00ccc0 con 0x7f8094103980 2024-04-16T17:18:13.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:13.638+0000 7f8091600640 1 -- 172.21.15.45:0/1035673585 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f807c010910 con 0x7f8094103980 2024-04-16T17:18:13.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:13.638+0000 7f8091600640 1 -- 172.21.15.45:0/1035673585 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f807c018920 con 0x7f8094103980 2024-04-16T17:18:13.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:13.638+0000 7f8099600640 1 -- 172.21.15.45:0/1035673585 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80941a6f20 con 0x7f8094103980 2024-04-16T17:18:13.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:13.638+0000 7f8099600640 1 -- 172.21.15.45:0/1035673585 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80941a7330 con 0x7f8094103980 2024-04-16T17:18:13.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:13.639+0000 7f8099600640 1 -- 172.21.15.45:0/1035673585 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8094068d40 con 0x7f8094103980 2024-04-16T17:18:13.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:13.641+0000 7f8091600640 1 -- 172.21.15.45:0/1035673585 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f807c010430 con 0x7f8094103980 2024-04-16T17:18:13.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:13.643+0000 7f8091600640 1 --2- 172.21.15.45:0/1035673585 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f80780a3dd0 0x7f80780a6290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:13.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:13.643+0000 7f8091600640 1 -- 172.21.15.45:0/1035673585 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 9584+0+0 (secure 0 0 0) 0x7f807c0d0e30 con 0x7f8094103980 2024-04-16T17:18:13.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:13.643+0000 7f8093e00640 1 --2- 172.21.15.45:0/1035673585 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f80780a3dd0 0x7f80780a6290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:13.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:13.644+0000 7f8093e00640 1 --2- 172.21.15.45:0/1035673585 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f80780a3dd0 0x7f80780a6290 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f8084005ab0 tx=0x7f8084005a20 comp rx=0 tx=0).ready entity=mgr.25411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:18:13.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:13.647+0000 7f8091600640 1 -- 172.21.15.45:0/1035673585 <== mon.2 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f807c09d070 con 0x7f8094103980 2024-04-16T17:18:13.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:18:13 smithi045 ceph-mon[31676]: from='client.? 172.21.15.45:0/2900844816' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-16T17:18:13.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:18:13 smithi045 ceph-mon[31676]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T17:18:13.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:18:13 smithi045 ceph-mon[31676]: from='client.? 172.21.15.45:0/2317997556' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-16T17:18:13.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:13.857+0000 7f8099600640 1 -- 172.21.15.45:0/1035673585 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f8094102190 con 0x7f8094103980 2024-04-16T17:18:13.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:13.857+0000 7f8091600640 1 -- 172.21.15.45:0/1035673585 <== mon.2 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v107) ==== 100+0+11 (secure 0 0 0) 0x7f807c09c970 con 0x7f8094103980 2024-04-16T17:18:13.858 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 64 2024-04-16T17:18:13.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:13.860+0000 7f8099600640 1 -- 172.21.15.45:0/1035673585 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f80780a3dd0 msgr2=0x7f80780a6290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:18:13.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:13.860+0000 7f8099600640 1 --2- 172.21.15.45:0/1035673585 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f80780a3dd0 0x7f80780a6290 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f8084005ab0 tx=0x7f8084005a20 comp rx=0 tx=0).stop 2024-04-16T17:18:13.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:13.860+0000 7f8099600640 1 -- 172.21.15.45:0/1035673585 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8094103980 msgr2=0x7f80941011f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:18:13.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:13.860+0000 7f8099600640 1 --2- 172.21.15.45:0/1035673585 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8094103980 0x7f80941011f0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f807c00e9f0 tx=0x7f807c00eeb0 comp rx=0 tx=0).stop 2024-04-16T17:18:13.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:13.860+0000 7f8099600640 1 -- 172.21.15.45:0/1035673585 shutdown_connections 2024-04-16T17:18:13.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:13.861+0000 7f8099600640 1 --2- 172.21.15.45:0/1035673585 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f80780a3dd0 0x7f80780a6290 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:13.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:13.861+0000 7f8099600640 1 --2- 172.21.15.45:0/1035673585 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f809410c480 0x7f8094101730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:13.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:13.861+0000 7f8099600640 1 --2- 172.21.15.45:0/1035673585 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8094103980 0x7f80941011f0 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:13.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:13.861+0000 7f8099600640 1 --2- 172.21.15.45:0/1035673585 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8094103060 0x7f8094100cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:13.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:13.861+0000 7f8099600640 1 -- 172.21.15.45:0/1035673585 >> 172.21.15.45:0/1035673585 conn(0x7f80940fc880 msgr2=0x7f80940fdfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:18:13.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:13.861+0000 7f8099600640 1 -- 172.21.15.45:0/1035673585 shutdown_connections 2024-04-16T17:18:13.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:13.861+0000 7f8099600640 1 -- 172.21.15.45:0/1035673585 wait complete. 2024-04-16T17:18:13.891 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-16T17:18:13.926 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:18:13 smithi104 ceph-mon[36457]: from='client.? 172.21.15.45:0/2900844816' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-16T17:18:13.926 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:18:13 smithi104 ceph-mon[36457]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T17:18:13.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:18:13 smithi104 ceph-mon[36457]: from='client.? 172.21.15.45:0/2317997556' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-16T17:18:14.054 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:18:13 smithi117 ceph-mon[36730]: from='client.? 172.21.15.45:0/2900844816' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-16T17:18:14.054 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:18:13 smithi117 ceph-mon[36730]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-16T17:18:14.054 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:18:13 smithi117 ceph-mon[36730]: from='client.? 172.21.15.45:0/2317997556' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-16T17:18:14.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:14.112+0000 7f2f5bc00640 1 -- 172.21.15.45:0/3088516809 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2f54102bf0 msgr2=0x7f2f54106c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:18:14.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:14.112+0000 7f2f5bc00640 1 --2- 172.21.15.45:0/3088516809 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2f54102bf0 0x7f2f54106c30 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f2f40009900 tx=0x7f2f4002bc40 comp rx=0 tx=0).stop 2024-04-16T17:18:14.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:14.112+0000 7f2f5bc00640 1 -- 172.21.15.45:0/3088516809 shutdown_connections 2024-04-16T17:18:14.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:14.112+0000 7f2f5bc00640 1 --2- 172.21.15.45:0/3088516809 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2f54102bf0 0x7f2f54106c30 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:14.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:14.112+0000 7f2f5bc00640 1 --2- 172.21.15.45:0/3088516809 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2f54107310 0x7f2f5410eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:14.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:14.112+0000 7f2f5bc00640 1 --2- 172.21.15.45:0/3088516809 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2f54102240 0x7f2f54102620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:14.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:14.112+0000 7f2f5bc00640 1 -- 172.21.15.45:0/3088516809 >> 172.21.15.45:0/3088516809 conn(0x7f2f54079450 msgr2=0x7f2f54079860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:18:14.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:14.112+0000 7f2f5bc00640 1 -- 172.21.15.45:0/3088516809 shutdown_connections 2024-04-16T17:18:14.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:14.112+0000 7f2f5bc00640 1 -- 172.21.15.45:0/3088516809 wait complete. 2024-04-16T17:18:14.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:14.113+0000 7f2f5bc00640 1 Processor -- start 2024-04-16T17:18:14.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:14.114+0000 7f2f5bc00640 1 -- start start 2024-04-16T17:18:14.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:14.114+0000 7f2f5bc00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2f54102240 0x7f2f541a0790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:14.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:14.114+0000 7f2f5bc00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2f54107310 0x7f2f541a0cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:14.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:14.114+0000 7f2f5bc00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2f5419a940 0x7f2f5419adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:14.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:14.114+0000 7f2f5bc00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f2f541123f0 con 0x7f2f54107310 2024-04-16T17:18:14.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:14.114+0000 7f2f5bc00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f2f54112270 con 0x7f2f5419a940 2024-04-16T17:18:14.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:14.114+0000 7f2f5bc00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f2f54112570 con 0x7f2f54102240 2024-04-16T17:18:14.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:14.114+0000 7f2f59c00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2f5419a940 0x7f2f5419adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:14.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:14.114+0000 7f2f53e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2f54107310 0x7f2f541a0cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:14.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:14.114+0000 7f2f59c00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2f5419a940 0x7f2f5419adf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:59258/0 (socket says 172.21.15.45:59258) 2024-04-16T17:18:14.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:14.114+0000 7f2f59c00640 1 -- 172.21.15.45:0/2700594599 learned_addr learned my addr 172.21.15.45:0/2700594599 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:18:14.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:14.114+0000 7f2f59200640 1 --2- 172.21.15.45:0/2700594599 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2f54102240 0x7f2f541a0790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:14.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:14.115+0000 7f2f59c00640 1 -- 172.21.15.45:0/2700594599 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2f54102240 msgr2=0x7f2f541a0790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:18:14.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:14.115+0000 7f2f59c00640 1 --2- 172.21.15.45:0/2700594599 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2f54102240 0x7f2f541a0790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:14.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:14.115+0000 7f2f59c00640 1 -- 172.21.15.45:0/2700594599 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2f54107310 msgr2=0x7f2f541a0cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:18:14.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:14.115+0000 7f2f59c00640 1 --2- 172.21.15.45:0/2700594599 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2f54107310 0x7f2f541a0cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:14.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:14.115+0000 7f2f59c00640 1 -- 172.21.15.45:0/2700594599 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f5419b460 con 0x7f2f5419a940 2024-04-16T17:18:14.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:14.115+0000 7f2f59c00640 1 --2- 172.21.15.45:0/2700594599 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2f5419a940 0x7f2f5419adf0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f2f4c00eb10 tx=0x7f2f4c00c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:18:14.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:14.115+0000 7f2f51600640 1 -- 172.21.15.45:0/2700594599 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f4c004b20 con 0x7f2f5419a940 2024-04-16T17:18:14.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:14.115+0000 7f2f51600640 1 -- 172.21.15.45:0/2700594599 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2f4c004cc0 con 0x7f2f5419a940 2024-04-16T17:18:14.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:14.115+0000 7f2f5bc00640 1 -- 172.21.15.45:0/2700594599 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f5419b750 con 0x7f2f5419a940 2024-04-16T17:18:14.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:14.115+0000 7f2f51600640 1 -- 172.21.15.45:0/2700594599 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f4c010760 con 0x7f2f5419a940 2024-04-16T17:18:14.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:14.115+0000 7f2f5bc00640 1 -- 172.21.15.45:0/2700594599 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f541a79a0 con 0x7f2f5419a940 2024-04-16T17:18:14.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:14.117+0000 7f2f5bc00640 1 -- 172.21.15.45:0/2700594599 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f54069270 con 0x7f2f5419a940 2024-04-16T17:18:14.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:14.118+0000 7f2f51600640 1 -- 172.21.15.45:0/2700594599 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f2f4c005020 con 0x7f2f5419a940 2024-04-16T17:18:14.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:14.120+0000 7f2f51600640 1 --2- 172.21.15.45:0/2700594599 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f2f3c0a3da0 0x7f2f3c0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:14.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:14.120+0000 7f2f51600640 1 -- 172.21.15.45:0/2700594599 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 9584+0+0 (secure 0 0 0) 0x7f2f4c0d1110 con 0x7f2f5419a940 2024-04-16T17:18:14.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:14.120+0000 7f2f59200640 1 --2- 172.21.15.45:0/2700594599 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f2f3c0a3da0 0x7f2f3c0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:14.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:14.121+0000 7f2f59200640 1 --2- 172.21.15.45:0/2700594599 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f2f3c0a3da0 0x7f2f3c0a6260 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f2f5419ba80 tx=0x7f2f48005f90 comp rx=0 tx=0).ready entity=mgr.25411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:18:14.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:14.123+0000 7f2f51600640 1 -- 172.21.15.45:0/2700594599 <== mon.2 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2f4c09d350 con 0x7f2f5419a940 2024-04-16T17:18:14.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:14.346+0000 7f2f5bc00640 1 -- 172.21.15.45:0/2700594599 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f2f54102dd0 con 0x7f2f5419a940 2024-04-16T17:18:14.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:14.347+0000 7f2f51600640 1 -- 172.21.15.45:0/2700594599 <== mon.2 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v107) ==== 103+0+11 (secure 0 0 0) 0x7f2f4c09cc50 con 0x7f2f5419a940 2024-04-16T17:18:14.347 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 32 2024-04-16T17:18:14.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:14.350+0000 7f2f5bc00640 1 -- 172.21.15.45:0/2700594599 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f2f3c0a3da0 msgr2=0x7f2f3c0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:18:14.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:14.350+0000 7f2f5bc00640 1 --2- 172.21.15.45:0/2700594599 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f2f3c0a3da0 0x7f2f3c0a6260 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f2f5419ba80 tx=0x7f2f48005f90 comp rx=0 tx=0).stop 2024-04-16T17:18:14.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:14.350+0000 7f2f5bc00640 1 -- 172.21.15.45:0/2700594599 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2f5419a940 msgr2=0x7f2f5419adf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:18:14.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:14.350+0000 7f2f5bc00640 1 --2- 172.21.15.45:0/2700594599 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2f5419a940 0x7f2f5419adf0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f2f4c00eb10 tx=0x7f2f4c00c2d0 comp rx=0 tx=0).stop 2024-04-16T17:18:14.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:14.350+0000 7f2f5bc00640 1 -- 172.21.15.45:0/2700594599 shutdown_connections 2024-04-16T17:18:14.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:14.350+0000 7f2f5bc00640 1 --2- 172.21.15.45:0/2700594599 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f2f3c0a3da0 0x7f2f3c0a6260 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:14.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:14.350+0000 7f2f5bc00640 1 --2- 172.21.15.45:0/2700594599 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2f5419a940 0x7f2f5419adf0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:14.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:14.350+0000 7f2f5bc00640 1 --2- 172.21.15.45:0/2700594599 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2f54107310 0x7f2f541a0cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:14.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:14.350+0000 7f2f5bc00640 1 --2- 172.21.15.45:0/2700594599 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2f54102240 0x7f2f541a0790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:14.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:14.350+0000 7f2f5bc00640 1 -- 172.21.15.45:0/2700594599 >> 172.21.15.45:0/2700594599 conn(0x7f2f54079450 msgr2=0x7f2f541001a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:18:14.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:14.350+0000 7f2f5bc00640 1 -- 172.21.15.45:0/2700594599 shutdown_connections 2024-04-16T17:18:14.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:14.350+0000 7f2f5bc00640 1 -- 172.21.15.45:0/2700594599 wait complete. 2024-04-16T17:18:14.369 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-16T17:18:14.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:14.574+0000 7f6c02800640 1 -- 172.21.15.45:0/2341622083 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6bfc102240 msgr2=0x7f6bfc102620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:18:14.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:14.574+0000 7f6c02800640 1 --2- 172.21.15.45:0/2341622083 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6bfc102240 0x7f6bfc102620 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f6bec009860 tx=0x7f6bec02bb70 comp rx=0 tx=0).stop 2024-04-16T17:18:14.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:14.574+0000 7f6c02800640 1 -- 172.21.15.45:0/2341622083 shutdown_connections 2024-04-16T17:18:14.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:14.574+0000 7f6c02800640 1 --2- 172.21.15.45:0/2341622083 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6bfc102bf0 0x7f6bfc106c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:14.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:14.574+0000 7f6c02800640 1 --2- 172.21.15.45:0/2341622083 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6bfc107310 0x7f6bfc10eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:14.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:14.574+0000 7f6c02800640 1 --2- 172.21.15.45:0/2341622083 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6bfc102240 0x7f6bfc102620 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:14.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:14.574+0000 7f6c02800640 1 -- 172.21.15.45:0/2341622083 >> 172.21.15.45:0/2341622083 conn(0x7f6bfc079450 msgr2=0x7f6bfc079860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:18:14.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:14.574+0000 7f6c02800640 1 -- 172.21.15.45:0/2341622083 shutdown_connections 2024-04-16T17:18:14.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:14.574+0000 7f6c02800640 1 -- 172.21.15.45:0/2341622083 wait complete. 2024-04-16T17:18:14.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:14.576+0000 7f6c02800640 1 Processor -- start 2024-04-16T17:18:14.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:14.576+0000 7f6c02800640 1 -- start start 2024-04-16T17:18:14.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:14.576+0000 7f6c02800640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6bfc102240 0x7f6bfc1a07c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:14.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:14.576+0000 7f6c02800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6bfc107310 0x7f6bfc1a0d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:14.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:14.576+0000 7f6c02800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6bfc19a970 0x7f6bfc19ae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:14.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:14.576+0000 7f6c02800640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f6bfc1123a0 con 0x7f6bfc107310 2024-04-16T17:18:14.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:14.576+0000 7f6c02800640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f6bfc112220 con 0x7f6bfc102240 2024-04-16T17:18:14.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:14.576+0000 7f6c02800640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f6bfc112520 con 0x7f6bfc19a970 2024-04-16T17:18:14.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:14.577+0000 7f6bf2a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6bfc107310 0x7f6bfc1a0d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:14.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:14.577+0000 7f6bfbe00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6bfc19a970 0x7f6bfc19ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:14.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:14.577+0000 7f6bfbe00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6bfc19a970 0x7f6bfc19ae20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.45:58838/0 (socket says 172.21.15.45:58838) 2024-04-16T17:18:14.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:14.577+0000 7f6bfbe00640 1 -- 172.21.15.45:0/2614354912 learned_addr learned my addr 172.21.15.45:0/2614354912 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:18:14.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:14.577+0000 7f6bfb400640 1 --2- 172.21.15.45:0/2614354912 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6bfc102240 0x7f6bfc1a07c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:14.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:14.577+0000 7f6bfbe00640 1 -- 172.21.15.45:0/2614354912 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6bfc102240 msgr2=0x7f6bfc1a07c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:18:14.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:14.577+0000 7f6bfbe00640 1 --2- 172.21.15.45:0/2614354912 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6bfc102240 0x7f6bfc1a07c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:14.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:14.577+0000 7f6bfbe00640 1 -- 172.21.15.45:0/2614354912 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6bfc107310 msgr2=0x7f6bfc1a0d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:18:14.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:14.577+0000 7f6bfbe00640 1 --2- 172.21.15.45:0/2614354912 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6bfc107310 0x7f6bfc1a0d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:14.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:14.577+0000 7f6bfbe00640 1 -- 172.21.15.45:0/2614354912 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6bfc19b490 con 0x7f6bfc19a970 2024-04-16T17:18:14.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:14.577+0000 7f6bf2a00640 1 --2- 172.21.15.45:0/2614354912 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6bfc107310 0x7f6bfc1a0d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:18:14.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:14.577+0000 7f6bfbe00640 1 --2- 172.21.15.45:0/2614354912 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6bfc19a970 0x7f6bfc19ae20 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f6bf400efc0 tx=0x7f6bf400c800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:18:14.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:14.578+0000 7f6bf8c00640 1 -- 172.21.15.45:0/2614354912 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bf4003970 con 0x7f6bfc19a970 2024-04-16T17:18:14.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:14.578+0000 7f6bf8c00640 1 -- 172.21.15.45:0/2614354912 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6bf4004020 con 0x7f6bfc19a970 2024-04-16T17:18:14.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:14.578+0000 7f6bf8c00640 1 -- 172.21.15.45:0/2614354912 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bf400b9f0 con 0x7f6bfc19a970 2024-04-16T17:18:14.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:14.578+0000 7f6c02800640 1 -- 172.21.15.45:0/2614354912 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6bfc19b780 con 0x7f6bfc19a970 2024-04-16T17:18:14.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:14.578+0000 7f6c02800640 1 -- 172.21.15.45:0/2614354912 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6bfc1a79d0 con 0x7f6bfc19a970 2024-04-16T17:18:14.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:14.579+0000 7f6c02800640 1 -- 172.21.15.45:0/2614354912 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6bfc069270 con 0x7f6bfc19a970 2024-04-16T17:18:14.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:14.581+0000 7f6bf8c00640 1 -- 172.21.15.45:0/2614354912 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f6bf401d020 con 0x7f6bfc19a970 2024-04-16T17:18:14.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:14.582+0000 7f6bf8c00640 1 --2- 172.21.15.45:0/2614354912 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f6be00a3da0 0x7f6be00a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:14.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:14.582+0000 7f6bf8c00640 1 -- 172.21.15.45:0/2614354912 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 9496+0+0 (secure 0 0 0) 0x7f6bf40d0a00 con 0x7f6bfc19a970 2024-04-16T17:18:14.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:14.583+0000 7f6bfb400640 1 --2- 172.21.15.45:0/2614354912 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f6be00a3da0 0x7f6be00a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:14.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:14.584+0000 7f6bfb400640 1 --2- 172.21.15.45:0/2614354912 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f6be00a3da0 0x7f6be00a6260 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f6bec009830 tx=0x7f6bec035ae0 comp rx=0 tx=0).ready entity=mgr.25411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:18:14.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:14.586+0000 7f6bf8c00640 1 -- 172.21.15.45:0/2614354912 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6bf409ccd0 con 0x7f6bfc19a970 2024-04-16T17:18:14.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:14.828+0000 7f6c02800640 1 -- 172.21.15.45:0/2614354912 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f6bfc102d90 con 0x7f6bfc19a970 2024-04-16T17:18:14.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:14.829+0000 7f6bf8c00640 1 -- 172.21.15.45:0/2614354912 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 16 entries v108) ==== 81+0+939 (secure 0 0 0) 0x7f6bf409c5d0 con 0x7f6bfc19a970 2024-04-16T17:18:14.829 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.45:6800/268030699 2024-04-17T17:06:45.093215+0000 2024-04-16T17:18:14.830 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.45:6801/268030699 2024-04-17T17:06:45.093215+0000 2024-04-16T17:18:14.830 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.45:0/4074982598 2024-04-17T16:55:19.948001+0000 2024-04-16T17:18:14.830 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.45:0/1915281176 2024-04-17T17:06:45.093215+0000 2024-04-16T17:18:14.830 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.45:0/2443572576 2024-04-17T16:55:19.948001+0000 2024-04-16T17:18:14.830 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.45:0/1500147666 2024-04-17T17:06:45.093215+0000 2024-04-16T17:18:14.830 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.45:6801/1983630136 2024-04-17T16:55:19.948001+0000 2024-04-16T17:18:14.830 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.45:6800/1983630136 2024-04-17T16:55:19.948001+0000 2024-04-16T17:18:14.830 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.45:0/12001708 2024-04-17T17:06:45.093215+0000 2024-04-16T17:18:14.831 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.45:0/3150465248 2024-04-17T16:54:43.692466+0000 2024-04-16T17:18:14.831 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.45:0/556738623 2024-04-17T16:54:43.692466+0000 2024-04-16T17:18:14.831 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.45:0/445676297 2024-04-17T16:55:19.948001+0000 2024-04-16T17:18:14.831 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.45:6800/2553659622 2024-04-17T16:54:43.692466+0000 2024-04-16T17:18:14.831 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.45:6801/2553659622 2024-04-17T16:54:43.692466+0000 2024-04-16T17:18:14.831 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.45:0/265066921 2024-04-17T17:06:45.093215+0000 2024-04-16T17:18:14.831 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.45:0/1184615408 2024-04-17T16:54:43.692466+0000 2024-04-16T17:18:14.832 INFO:teuthology.orchestra.run.smithi045.stderr:listed 16 entries 2024-04-16T17:18:14.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:14.831+0000 7f6c02800640 1 -- 172.21.15.45:0/2614354912 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f6be00a3da0 msgr2=0x7f6be00a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:18:14.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:14.831+0000 7f6c02800640 1 --2- 172.21.15.45:0/2614354912 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f6be00a3da0 0x7f6be00a6260 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f6bec009830 tx=0x7f6bec035ae0 comp rx=0 tx=0).stop 2024-04-16T17:18:14.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:14.831+0000 7f6c02800640 1 -- 172.21.15.45:0/2614354912 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6bfc19a970 msgr2=0x7f6bfc19ae20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:18:14.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:14.831+0000 7f6c02800640 1 --2- 172.21.15.45:0/2614354912 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6bfc19a970 0x7f6bfc19ae20 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f6bf400efc0 tx=0x7f6bf400c800 comp rx=0 tx=0).stop 2024-04-16T17:18:14.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:14.831+0000 7f6c02800640 1 -- 172.21.15.45:0/2614354912 shutdown_connections 2024-04-16T17:18:14.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:14.831+0000 7f6c02800640 1 --2- 172.21.15.45:0/2614354912 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f6be00a3da0 0x7f6be00a6260 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:14.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:14.831+0000 7f6c02800640 1 --2- 172.21.15.45:0/2614354912 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6bfc19a970 0x7f6bfc19ae20 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:14.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:14.831+0000 7f6c02800640 1 --2- 172.21.15.45:0/2614354912 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6bfc107310 0x7f6bfc1a0d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:14.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:14.831+0000 7f6c02800640 1 --2- 172.21.15.45:0/2614354912 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6bfc102240 0x7f6bfc1a07c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:14.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:14.831+0000 7f6c02800640 1 -- 172.21.15.45:0/2614354912 >> 172.21.15.45:0/2614354912 conn(0x7f6bfc079450 msgr2=0x7f6bfc100160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:18:14.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:14.832+0000 7f6c02800640 1 -- 172.21.15.45:0/2614354912 shutdown_connections 2024-04-16T17:18:14.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:14.832+0000 7f6c02800640 1 -- 172.21.15.45:0/2614354912 wait complete. 2024-04-16T17:18:14.853 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-16T17:18:14.854 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T17:18:14.854 DEBUG:teuthology.orchestra.run.smithi045:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-16T17:18:14.926 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:18:14 smithi104 ceph-mon[36457]: pgmap v359: 129 pgs: 129 active+clean; 51 MiB data, 871 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-16T17:18:14.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:18:14 smithi104 ceph-mon[36457]: from='client.? 172.21.15.45:0/1035673585' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-16T17:18:14.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:18:14 smithi104 ceph-mon[36457]: from='client.? 172.21.15.45:0/2700594599' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-16T17:18:14.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:18:14 smithi104 ceph-mon[36457]: osdmap e108: 12 total, 12 up, 12 in 2024-04-16T17:18:14.937 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:18:14 smithi045 ceph-mon[31676]: pgmap v359: 129 pgs: 129 active+clean; 51 MiB data, 871 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-16T17:18:14.937 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:18:14 smithi045 ceph-mon[31676]: from='client.? 172.21.15.45:0/1035673585' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-16T17:18:14.937 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:18:14 smithi045 ceph-mon[31676]: from='client.? 172.21.15.45:0/2700594599' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-16T17:18:14.937 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:18:14 smithi045 ceph-mon[31676]: osdmap e108: 12 total, 12 up, 12 in 2024-04-16T17:18:15.054 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:18:14 smithi117 ceph-mon[36730]: pgmap v359: 129 pgs: 129 active+clean; 51 MiB data, 871 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-16T17:18:15.054 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:18:14 smithi117 ceph-mon[36730]: from='client.? 172.21.15.45:0/1035673585' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-16T17:18:15.054 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:18:14 smithi117 ceph-mon[36730]: from='client.? 172.21.15.45:0/2700594599' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-16T17:18:15.054 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:18:14 smithi117 ceph-mon[36730]: osdmap e108: 12 total, 12 up, 12 in 2024-04-16T17:18:15.081 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi045.front.sepia.ceph.com 2024-04-16T17:18:15.081 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T17:18:15.082 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-16T17:18:15.155 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T17:18:15.155 DEBUG:teuthology.orchestra.run.smithi045:> ip netns list 2024-04-16T17:18:15.193 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-16T17:18:15.194 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T17:18:15.194 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-16T17:18:15.309 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-16T17:18:15.309 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip link delete ceph-brx 2024-04-16T17:18:15.433 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-04-16T17:18:15.445 INFO:tasks.cephadm:Teardown begin 2024-04-16T17:18:15.445 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-16T17:18:15.445 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-16T17:18:15.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:15.643+0000 7ff1a3e00640 1 -- 172.21.15.45:0/2608207964 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff1a4069db0 msgr2=0x7ff1a4106a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:18:15.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:15.643+0000 7ff1a3e00640 1 --2- 172.21.15.45:0/2608207964 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff1a4069db0 0x7ff1a4106a70 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7ff18c009900 tx=0x7ff18c02bc40 comp rx=0 tx=0).stop 2024-04-16T17:18:15.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:15.644+0000 7ff1a3e00640 1 -- 172.21.15.45:0/2608207964 shutdown_connections 2024-04-16T17:18:15.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:15.644+0000 7ff1a3e00640 1 --2- 172.21.15.45:0/2608207964 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff1a41071a0 0x7ff1a410ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:15.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:15.644+0000 7ff1a3e00640 1 --2- 172.21.15.45:0/2608207964 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff1a4069db0 0x7ff1a4106a70 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:15.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:15.644+0000 7ff1a3e00640 1 --2- 172.21.15.45:0/2608207964 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff1a4069400 0x7ff1a40697e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:15.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:15.644+0000 7ff1a3e00640 1 -- 172.21.15.45:0/2608207964 >> 172.21.15.45:0/2608207964 conn(0x7ff1a4076840 msgr2=0x7ff1a4076c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:18:15.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:15.644+0000 7ff1a3e00640 1 -- 172.21.15.45:0/2608207964 shutdown_connections 2024-04-16T17:18:15.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:15.644+0000 7ff1a3e00640 1 -- 172.21.15.45:0/2608207964 wait complete. 2024-04-16T17:18:15.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:15.645+0000 7ff1a3e00640 1 Processor -- start 2024-04-16T17:18:15.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:15.646+0000 7ff1a3e00640 1 -- start start 2024-04-16T17:18:15.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:15.646+0000 7ff1a3e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff1a4069400 0x7ff1a41010c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:15.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:15.646+0000 7ff1a3e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff1a41071a0 0x7ff1a4101600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:15.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:15.646+0000 7ff1a3e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff1a4101b40 0x7ff1a41a6930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:15.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:15.646+0000 7ff1a3e00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7ff1a4112240 con 0x7ff1a41071a0 2024-04-16T17:18:15.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:15.646+0000 7ff1a3e00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7ff1a41120c0 con 0x7ff1a4069400 2024-04-16T17:18:15.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:15.646+0000 7ff1a3e00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7ff1a41123c0 con 0x7ff1a4101b40 2024-04-16T17:18:15.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:15.646+0000 7ff1a2a00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff1a4069400 0x7ff1a41010c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:15.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:15.646+0000 7ff19be00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff1a41071a0 0x7ff1a4101600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:15.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:15.646+0000 7ff1a2a00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff1a4069400 0x7ff1a41010c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:59304/0 (socket says 172.21.15.45:59304) 2024-04-16T17:18:15.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:15.646+0000 7ff1a2a00640 1 -- 172.21.15.45:0/3175400285 learned_addr learned my addr 172.21.15.45:0/3175400285 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:18:15.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:15.646+0000 7ff1a3400640 1 --2- 172.21.15.45:0/3175400285 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff1a4101b40 0x7ff1a41a6930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:15.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:15.647+0000 7ff1a2a00640 1 -- 172.21.15.45:0/3175400285 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff1a4101b40 msgr2=0x7ff1a41a6930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:18:15.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:15.647+0000 7ff1a2a00640 1 --2- 172.21.15.45:0/3175400285 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff1a4101b40 0x7ff1a41a6930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:15.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:15.647+0000 7ff1a2a00640 1 -- 172.21.15.45:0/3175400285 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff1a41071a0 msgr2=0x7ff1a4101600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:18:15.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:15.647+0000 7ff1a2a00640 1 --2- 172.21.15.45:0/3175400285 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff1a41071a0 0x7ff1a4101600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:15.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:15.647+0000 7ff1a2a00640 1 -- 172.21.15.45:0/3175400285 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1a41a7030 con 0x7ff1a4069400 2024-04-16T17:18:15.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:15.647+0000 7ff1a2a00640 1 --2- 172.21.15.45:0/3175400285 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff1a4069400 0x7ff1a41010c0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7ff19400ea60 tx=0x7ff19400ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:18:15.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:15.647+0000 7ff19b400640 1 -- 172.21.15.45:0/3175400285 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff19400cde0 con 0x7ff1a4069400 2024-04-16T17:18:15.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:15.648+0000 7ff1a3e00640 1 -- 172.21.15.45:0/3175400285 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1a41a7320 con 0x7ff1a4069400 2024-04-16T17:18:15.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:15.648+0000 7ff19b400640 1 -- 172.21.15.45:0/3175400285 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff1940035f0 con 0x7ff1a4069400 2024-04-16T17:18:15.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:15.648+0000 7ff19b400640 1 -- 172.21.15.45:0/3175400285 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff194010670 con 0x7ff1a4069400 2024-04-16T17:18:15.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:15.648+0000 7ff1a3e00640 1 -- 172.21.15.45:0/3175400285 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1a41a7890 con 0x7ff1a4069400 2024-04-16T17:18:15.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:15.649+0000 7ff1a3e00640 1 -- 172.21.15.45:0/3175400285 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1a4068d40 con 0x7ff1a4069400 2024-04-16T17:18:15.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:15.650+0000 7ff19b400640 1 -- 172.21.15.45:0/3175400285 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7ff1940037a0 con 0x7ff1a4069400 2024-04-16T17:18:15.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:15.652+0000 7ff19b400640 1 --2- 172.21.15.45:0/3175400285 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7ff1880a3dd0 0x7ff1880a6290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:15.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:15.652+0000 7ff19b400640 1 -- 172.21.15.45:0/3175400285 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 9496+0+0 (secure 0 0 0) 0x7ff1940d0e40 con 0x7ff1a4069400 2024-04-16T17:18:15.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:15.652+0000 7ff19be00640 1 --2- 172.21.15.45:0/3175400285 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7ff1880a3dd0 0x7ff1880a6290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:15.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:15.653+0000 7ff19be00640 1 --2- 172.21.15.45:0/3175400285 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7ff1880a3dd0 0x7ff1880a6290 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7ff18c02c180 tx=0x7ff18c02cf90 comp rx=0 tx=0).ready entity=mgr.25411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:18:15.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:15.656+0000 7ff19b400640 1 -- 172.21.15.45:0/3175400285 <== mon.2 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff19409d0b0 con 0x7ff1a4069400 2024-04-16T17:18:15.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:15.905+0000 7ff1a3e00640 1 -- 172.21.15.45:0/3175400285 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff1a410a390 con 0x7ff1a4069400 2024-04-16T17:18:15.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:15.906+0000 7ff19b400640 1 -- 172.21.15.45:0/3175400285 <== mon.2 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) ==== 94+0+11887 (secure 0 0 0) 0x7ff19409c9b0 con 0x7ff1a4069400 2024-04-16T17:18:15.907 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-16T17:18:15.907 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":44,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/1748842443","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":1748842443},{"type":"v1","addr":"172.21.15.45:6835","nonce":1748842443}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6839/1308627051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":1308627051},{"type":"v1","addr":"172.21.15.45:6839","nonce":1308627051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/1768242489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1768242489},{"type":"v1","addr":"172.21.15.117:6833","nonce":1768242489}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6833/3558902585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":3558902585},{"type":"v1","addr":"172.21.15.104:6833","nonce":3558902585}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24478,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/1308142903","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":1308142903},{"type":"v1","addr":"172.21.15.45:6837","nonce":1308142903}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24491,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6835/3430679937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":3430679937},{"type":"v1","addr":"172.21.15.104:6835","nonce":3430679937}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":44,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-16T17:03:24.663611+0000","modified":"2024-04-16T17:15:52.264451+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24581,"mds_1":24569,"mds_2":24532},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24499":{"gid":24499,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/2224123841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2224123841},{"type":"v1","addr":"172.21.15.117:6835","nonce":2224123841}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6837/2471356815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":2471356815},{"type":"v1","addr":"172.21.15.117:6837","nonce":2471356815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24532":{"gid":24532,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":13,"addr":"172.21.15.117:6839/2378675949","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":2378675949},{"type":"v1","addr":"172.21.15.117:6839","nonce":2378675949}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24569":{"gid":24569,"name":"a","rank":1,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.45:6841/3755310822","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6840","nonce":3755310822},{"type":"v1","addr":"172.21.15.45:6841","nonce":3755310822}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24581":{"gid":24581,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.104:6839/3660070592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6838","nonce":3660070592},{"type":"v1","addr":"172.21.15.104:6839","nonce":3660070592}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_27569":{"gid":27569,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6837/1833645246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":1833645246},{"type":"v1","addr":"172.21.15.104:6837","nonce":1833645246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24581,"qdb_cluster":[24569,24532,24581]},"id":1}]} 2024-04-16T17:18:15.908 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 44 2024-04-16T17:18:15.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:15.909+0000 7ff1a3e00640 1 -- 172.21.15.45:0/3175400285 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7ff1880a3dd0 msgr2=0x7ff1880a6290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:18:15.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:15.909+0000 7ff1a3e00640 1 --2- 172.21.15.45:0/3175400285 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7ff1880a3dd0 0x7ff1880a6290 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7ff18c02c180 tx=0x7ff18c02cf90 comp rx=0 tx=0).stop 2024-04-16T17:18:15.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:15.909+0000 7ff1a3e00640 1 -- 172.21.15.45:0/3175400285 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff1a4069400 msgr2=0x7ff1a41010c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:18:15.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:15.909+0000 7ff1a3e00640 1 --2- 172.21.15.45:0/3175400285 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff1a4069400 0x7ff1a41010c0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7ff19400ea60 tx=0x7ff19400ef20 comp rx=0 tx=0).stop 2024-04-16T17:18:15.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:15.909+0000 7ff1a3e00640 1 -- 172.21.15.45:0/3175400285 shutdown_connections 2024-04-16T17:18:15.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:15.909+0000 7ff1a3e00640 1 --2- 172.21.15.45:0/3175400285 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7ff1880a3dd0 0x7ff1880a6290 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:15.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:15.909+0000 7ff1a3e00640 1 --2- 172.21.15.45:0/3175400285 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff1a4101b40 0x7ff1a41a6930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:15.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:15.909+0000 7ff1a3e00640 1 --2- 172.21.15.45:0/3175400285 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff1a41071a0 0x7ff1a4101600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:15.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:15.909+0000 7ff1a3e00640 1 --2- 172.21.15.45:0/3175400285 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff1a4069400 0x7ff1a41010c0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:15.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:15.909+0000 7ff1a3e00640 1 -- 172.21.15.45:0/3175400285 >> 172.21.15.45:0/3175400285 conn(0x7ff1a4076840 msgr2=0x7ff1a40ff1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:18:15.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:15.909+0000 7ff1a3e00640 1 -- 172.21.15.45:0/3175400285 shutdown_connections 2024-04-16T17:18:15.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:15.910+0000 7ff1a3e00640 1 -- 172.21.15.45:0/3175400285 wait complete. 2024-04-16T17:18:15.926 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:18:15 smithi045 ceph-mon[31676]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:18:15.926 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:18:15 smithi045 ceph-mon[31676]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T17:18:15.927 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:18:15 smithi045 ceph-mon[31676]: from='mgr.25411 ' entity='mgr.x' 2024-04-16T17:18:15.927 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:18:15 smithi045 ceph-mon[31676]: from='client.? 172.21.15.45:0/2614354912' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-16T17:18:15.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:18:15 smithi104 ceph-mon[36457]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:18:15.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:18:15 smithi104 ceph-mon[36457]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T17:18:15.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:18:15 smithi104 ceph-mon[36457]: from='mgr.25411 ' entity='mgr.x' 2024-04-16T17:18:15.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:18:15 smithi104 ceph-mon[36457]: from='client.? 172.21.15.45:0/2614354912' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-16T17:18:15.929 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-16T17:18:16.054 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:18:15 smithi117 ceph-mon[36730]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-16T17:18:16.054 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:18:15 smithi117 ceph-mon[36730]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-16T17:18:16.055 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:18:15 smithi117 ceph-mon[36730]: from='mgr.25411 ' entity='mgr.x' 2024-04-16T17:18:16.055 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:18:15 smithi117 ceph-mon[36730]: from='client.? 172.21.15.45:0/2614354912' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-16T17:18:16.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:16.152+0000 7fc870a00640 1 -- 172.21.15.45:0/723804812 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc86c069400 msgr2=0x7fc86c0697e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:18:16.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:16.152+0000 7fc870a00640 1 --2- 172.21.15.45:0/723804812 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc86c069400 0x7fc86c0697e0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fc85c009860 tx=0x7fc85c02bb70 comp rx=0 tx=0).stop 2024-04-16T17:18:16.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:16.153+0000 7fc870a00640 1 -- 172.21.15.45:0/723804812 shutdown_connections 2024-04-16T17:18:16.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:16.153+0000 7fc870a00640 1 --2- 172.21.15.45:0/723804812 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc86c1071a0 0x7fc86c10ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:16.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:16.153+0000 7fc870a00640 1 --2- 172.21.15.45:0/723804812 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc86c069db0 0x7fc86c106a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:16.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:16.153+0000 7fc870a00640 1 --2- 172.21.15.45:0/723804812 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc86c069400 0x7fc86c0697e0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:16.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:16.153+0000 7fc870a00640 1 -- 172.21.15.45:0/723804812 >> 172.21.15.45:0/723804812 conn(0x7fc86c076840 msgr2=0x7fc86c076c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:18:16.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:16.153+0000 7fc870a00640 1 -- 172.21.15.45:0/723804812 shutdown_connections 2024-04-16T17:18:16.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:16.153+0000 7fc870a00640 1 -- 172.21.15.45:0/723804812 wait complete. 2024-04-16T17:18:16.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:16.154+0000 7fc870a00640 1 Processor -- start 2024-04-16T17:18:16.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:16.154+0000 7fc870a00640 1 -- start start 2024-04-16T17:18:16.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:16.154+0000 7fc870a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc86c069400 0x7fc86c1a06a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:16.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:16.154+0000 7fc870a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc86c1071a0 0x7fc86c1a0be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:16.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:16.154+0000 7fc870a00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc86c19a860 0x7fc86c19ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:16.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:16.154+0000 7fc870a00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fc86c112200 con 0x7fc86c1071a0 2024-04-16T17:18:16.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:16.154+0000 7fc870a00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fc86c112080 con 0x7fc86c19a860 2024-04-16T17:18:16.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:16.154+0000 7fc870a00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fc86c112380 con 0x7fc86c069400 2024-04-16T17:18:16.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:16.155+0000 7fc86aa00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc86c1071a0 0x7fc86c1a0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:16.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:16.155+0000 7fc86b400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc86c069400 0x7fc86c1a06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:16.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:16.155+0000 7fc86aa00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc86c1071a0 0x7fc86c1a0be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59894/0 (socket says 172.21.15.45:59894) 2024-04-16T17:18:16.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:16.155+0000 7fc86aa00640 1 -- 172.21.15.45:0/3530341840 learned_addr learned my addr 172.21.15.45:0/3530341840 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:18:16.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:16.155+0000 7fc86be00640 1 --2- 172.21.15.45:0/3530341840 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc86c19a860 0x7fc86c19ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:16.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:16.155+0000 7fc86b400640 1 -- 172.21.15.45:0/3530341840 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc86c19a860 msgr2=0x7fc86c19ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:18:16.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:16.155+0000 7fc86b400640 1 --2- 172.21.15.45:0/3530341840 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc86c19a860 0x7fc86c19ad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:16.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:16.155+0000 7fc86b400640 1 -- 172.21.15.45:0/3530341840 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc86c1071a0 msgr2=0x7fc86c1a0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:18:16.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:16.155+0000 7fc86b400640 1 --2- 172.21.15.45:0/3530341840 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc86c1071a0 0x7fc86c1a0be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:16.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:16.155+0000 7fc86b400640 1 -- 172.21.15.45:0/3530341840 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc86c19b3f0 con 0x7fc86c069400 2024-04-16T17:18:16.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:16.156+0000 7fc86b400640 1 --2- 172.21.15.45:0/3530341840 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc86c069400 0x7fc86c1a06a0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fc85c037040 tx=0x7fc85c0045b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:18:16.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:16.156+0000 7fc863e00640 1 -- 172.21.15.45:0/3530341840 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc85c041020 con 0x7fc86c069400 2024-04-16T17:18:16.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:16.156+0000 7fc863e00640 1 -- 172.21.15.45:0/3530341840 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc85c02c9f0 con 0x7fc86c069400 2024-04-16T17:18:16.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:16.156+0000 7fc870a00640 1 -- 172.21.15.45:0/3530341840 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc86c19b680 con 0x7fc86c069400 2024-04-16T17:18:16.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:16.156+0000 7fc863e00640 1 -- 172.21.15.45:0/3530341840 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc85c039040 con 0x7fc86c069400 2024-04-16T17:18:16.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:16.156+0000 7fc870a00640 1 -- 172.21.15.45:0/3530341840 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc86c1a77e0 con 0x7fc86c069400 2024-04-16T17:18:16.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:16.157+0000 7fc870a00640 1 -- 172.21.15.45:0/3530341840 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc86c103550 con 0x7fc86c069400 2024-04-16T17:18:16.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:16.159+0000 7fc863e00640 1 -- 172.21.15.45:0/3530341840 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fc85c044020 con 0x7fc86c069400 2024-04-16T17:18:16.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:16.161+0000 7fc863e00640 1 --2- 172.21.15.45:0/3530341840 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fc8500a3d20 0x7fc8500a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:16.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:16.161+0000 7fc863e00640 1 -- 172.21.15.45:0/3530341840 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 9496+0+0 (secure 0 0 0) 0x7fc85c0f2710 con 0x7fc86c069400 2024-04-16T17:18:16.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:16.161+0000 7fc86aa00640 1 --2- 172.21.15.45:0/3530341840 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fc8500a3d20 0x7fc8500a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:16.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:16.162+0000 7fc86aa00640 1 --2- 172.21.15.45:0/3530341840 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fc8500a3d20 0x7fc8500a61e0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fc86c10ece0 tx=0x7fc854009500 comp rx=0 tx=0).ready entity=mgr.25411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:18:16.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:16.163+0000 7fc863e00640 1 -- 172.21.15.45:0/3530341840 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc85c04b020 con 0x7fc86c069400 2024-04-16T17:18:16.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:16.414+0000 7fc870a00640 1 -- 172.21.15.45:0/3530341840 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc86c068d40 con 0x7fc86c069400 2024-04-16T17:18:16.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:16.415+0000 7fc863e00640 1 -- 172.21.15.45:0/3530341840 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) ==== 94+0+11887 (secure 0 0 0) 0x7fc85c0bea90 con 0x7fc86c069400 2024-04-16T17:18:16.416 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-16T17:18:16.416 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":44,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14382,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/1748842443","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":1748842443},{"type":"v1","addr":"172.21.15.45:6835","nonce":1748842443}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6839/1308627051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":1308627051},{"type":"v1","addr":"172.21.15.45:6839","nonce":1308627051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/1768242489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1768242489},{"type":"v1","addr":"172.21.15.117:6833","nonce":1768242489}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6833/3558902585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":3558902585},{"type":"v1","addr":"172.21.15.104:6833","nonce":3558902585}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24478,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/1308142903","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":1308142903},{"type":"v1","addr":"172.21.15.45:6837","nonce":1308142903}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24491,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6835/3430679937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":3430679937},{"type":"v1","addr":"172.21.15.104:6835","nonce":3430679937}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":44,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-16T17:03:24.663611+0000","modified":"2024-04-16T17:15:52.264451+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24581,"mds_1":24569,"mds_2":24532},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24499":{"gid":24499,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/2224123841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2224123841},{"type":"v1","addr":"172.21.15.117:6835","nonce":2224123841}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6837/2471356815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":2471356815},{"type":"v1","addr":"172.21.15.117:6837","nonce":2471356815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24532":{"gid":24532,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":13,"addr":"172.21.15.117:6839/2378675949","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":2378675949},{"type":"v1","addr":"172.21.15.117:6839","nonce":2378675949}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24569":{"gid":24569,"name":"a","rank":1,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.45:6841/3755310822","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6840","nonce":3755310822},{"type":"v1","addr":"172.21.15.45:6841","nonce":3755310822}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24581":{"gid":24581,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.104:6839/3660070592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6838","nonce":3660070592},{"type":"v1","addr":"172.21.15.104:6839","nonce":3660070592}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_27569":{"gid":27569,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6837/1833645246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":1833645246},{"type":"v1","addr":"172.21.15.104:6837","nonce":1833645246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24581,"qdb_cluster":[24569,24532,24581]},"id":1}]} 2024-04-16T17:18:16.417 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 44 2024-04-16T17:18:16.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:16.418+0000 7fc870a00640 1 -- 172.21.15.45:0/3530341840 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fc8500a3d20 msgr2=0x7fc8500a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:18:16.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:16.418+0000 7fc870a00640 1 --2- 172.21.15.45:0/3530341840 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fc8500a3d20 0x7fc8500a61e0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fc86c10ece0 tx=0x7fc854009500 comp rx=0 tx=0).stop 2024-04-16T17:18:16.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:16.418+0000 7fc870a00640 1 -- 172.21.15.45:0/3530341840 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc86c069400 msgr2=0x7fc86c1a06a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:18:16.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:16.418+0000 7fc870a00640 1 --2- 172.21.15.45:0/3530341840 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc86c069400 0x7fc86c1a06a0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fc85c037040 tx=0x7fc85c0045b0 comp rx=0 tx=0).stop 2024-04-16T17:18:16.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:16.418+0000 7fc870a00640 1 -- 172.21.15.45:0/3530341840 shutdown_connections 2024-04-16T17:18:16.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:16.418+0000 7fc870a00640 1 --2- 172.21.15.45:0/3530341840 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fc8500a3d20 0x7fc8500a61e0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:16.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:16.418+0000 7fc870a00640 1 --2- 172.21.15.45:0/3530341840 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc86c19a860 0x7fc86c19ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:16.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:16.418+0000 7fc870a00640 1 --2- 172.21.15.45:0/3530341840 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc86c1071a0 0x7fc86c1a0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:16.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:16.418+0000 7fc870a00640 1 --2- 172.21.15.45:0/3530341840 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc86c069400 0x7fc86c1a06a0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:16.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:16.418+0000 7fc870a00640 1 -- 172.21.15.45:0/3530341840 >> 172.21.15.45:0/3530341840 conn(0x7fc86c076840 msgr2=0x7fc86c10d1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:18:16.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:16.419+0000 7fc870a00640 1 -- 172.21.15.45:0/3530341840 shutdown_connections 2024-04-16T17:18:16.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:16.419+0000 7fc870a00640 1 -- 172.21.15.45:0/3530341840 wait complete. 2024-04-16T17:18:16.440 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-16T17:18:16.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:16.677+0000 7f3382400640 1 -- 172.21.15.45:0/813031256 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f337c1071a0 msgr2=0x7f337c10ed40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:18:16.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:16.677+0000 7f3382400640 1 --2- 172.21.15.45:0/813031256 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f337c1071a0 0x7f337c10ed40 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f3374009be0 tx=0x7f337402bef0 comp rx=0 tx=0).stop 2024-04-16T17:18:16.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:16.677+0000 7f3382400640 1 -- 172.21.15.45:0/813031256 shutdown_connections 2024-04-16T17:18:16.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:16.677+0000 7f3382400640 1 --2- 172.21.15.45:0/813031256 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f337c069db0 0x7f337c106a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:16.679 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:16.677+0000 7f3382400640 1 --2- 172.21.15.45:0/813031256 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f337c1071a0 0x7f337c10ed40 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:16.679 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:16.677+0000 7f3382400640 1 --2- 172.21.15.45:0/813031256 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f337c069400 0x7f337c0697e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:16.679 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:16.677+0000 7f3382400640 1 -- 172.21.15.45:0/813031256 >> 172.21.15.45:0/813031256 conn(0x7f337c076840 msgr2=0x7f337c076c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:18:16.679 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:16.677+0000 7f3382400640 1 -- 172.21.15.45:0/813031256 shutdown_connections 2024-04-16T17:18:16.679 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:16.678+0000 7f3382400640 1 -- 172.21.15.45:0/813031256 wait complete. 2024-04-16T17:18:16.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:16.678+0000 7f3382400640 1 Processor -- start 2024-04-16T17:18:16.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:16.679+0000 7f3382400640 1 -- start start 2024-04-16T17:18:16.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:16.679+0000 7f3382400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f337c069400 0x7f337c1010c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:16.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:16.679+0000 7f3382400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f337c1071a0 0x7f337c101600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:16.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:16.679+0000 7f3382400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f337c102d60 0x7f337c101b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:16.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:16.679+0000 7f3382400640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f337c112280 con 0x7f337c1071a0 2024-04-16T17:18:16.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:16.679+0000 7f3382400640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f337c112100 con 0x7f337c069400 2024-04-16T17:18:16.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:16.679+0000 7f3382400640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f337c112400 con 0x7f337c102d60 2024-04-16T17:18:16.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:16.679+0000 7f3381000640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f337c069400 0x7f337c1010c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:16.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:16.679+0000 7f3373e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f337c1071a0 0x7f337c101600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:16.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:16.679+0000 7f3381000640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f337c069400 0x7f337c1010c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:59338/0 (socket says 172.21.15.45:59338) 2024-04-16T17:18:16.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:16.679+0000 7f3381000640 1 -- 172.21.15.45:0/1215557425 learned_addr learned my addr 172.21.15.45:0/1215557425 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:18:16.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:16.680+0000 7f3381a00640 1 --2- 172.21.15.45:0/1215557425 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f337c102d60 0x7f337c101b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:16.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:16.680+0000 7f3373e00640 1 -- 172.21.15.45:0/1215557425 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f337c102d60 msgr2=0x7f337c101b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:18:16.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:16.680+0000 7f3373e00640 1 --2- 172.21.15.45:0/1215557425 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f337c102d60 0x7f337c101b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:16.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:16.680+0000 7f3373e00640 1 -- 172.21.15.45:0/1215557425 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f337c069400 msgr2=0x7f337c1010c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:18:16.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:16.680+0000 7f3373e00640 1 --2- 172.21.15.45:0/1215557425 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f337c069400 0x7f337c1010c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:16.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:16.680+0000 7f3373e00640 1 -- 172.21.15.45:0/1215557425 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f337c1ab3f0 con 0x7f337c1071a0 2024-04-16T17:18:16.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:16.680+0000 7f3381000640 1 --2- 172.21.15.45:0/1215557425 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f337c069400 0x7f337c1010c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:18:16.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:16.680+0000 7f3373e00640 1 --2- 172.21.15.45:0/1215557425 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f337c1071a0 0x7f337c101600 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f337c077ec0 tx=0x7f336400ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:18:16.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:16.680+0000 7f337a000640 1 -- 172.21.15.45:0/1215557425 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f336400cc40 con 0x7f337c1071a0 2024-04-16T17:18:16.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:16.681+0000 7f3382400640 1 -- 172.21.15.45:0/1215557425 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f337c1ab6e0 con 0x7f337c1071a0 2024-04-16T17:18:16.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:16.681+0000 7f337a000640 1 -- 172.21.15.45:0/1215557425 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f336400cde0 con 0x7f337c1071a0 2024-04-16T17:18:16.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:16.681+0000 7f337a000640 1 -- 172.21.15.45:0/1215557425 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3364004a00 con 0x7f337c1071a0 2024-04-16T17:18:16.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:16.681+0000 7f3382400640 1 -- 172.21.15.45:0/1215557425 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f337c1abbb0 con 0x7f337c1071a0 2024-04-16T17:18:16.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:16.682+0000 7f3382400640 1 -- 172.21.15.45:0/1215557425 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f337c103550 con 0x7f337c1071a0 2024-04-16T17:18:16.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:16.684+0000 7f337a000640 1 -- 172.21.15.45:0/1215557425 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f3364004ba0 con 0x7f337c1071a0 2024-04-16T17:18:16.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:16.685+0000 7f337a000640 1 --2- 172.21.15.45:0/1215557425 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f33600a3d20 0x7f33600a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:16.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:16.686+0000 7f337a000640 1 -- 172.21.15.45:0/1215557425 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 9496+0+0 (secure 0 0 0) 0x7f33640d0bc0 con 0x7f337c1071a0 2024-04-16T17:18:16.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:16.686+0000 7f3381000640 1 --2- 172.21.15.45:0/1215557425 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f33600a3d20 0x7f33600a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:16.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:16.687+0000 7f3381000640 1 --2- 172.21.15.45:0/1215557425 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f33600a3d20 0x7f33600a61e0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f336c0058c0 tx=0x7f336c009260 comp rx=0 tx=0).ready entity=mgr.25411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:18:16.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:16.689+0000 7f337a000640 1 -- 172.21.15.45:0/1215557425 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f336409cec0 con 0x7f337c1071a0 2024-04-16T17:18:16.726 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:18:16 smithi045 ceph-mon[31676]: pgmap v361: 129 pgs: 129 active+clean; 51 MiB data, 871 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 127 B/s wr, 3 op/s 2024-04-16T17:18:16.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:18:16 smithi045 ceph-mon[31676]: from='client.? 172.21.15.45:0/3175400285' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:18:16.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:18:16 smithi045 ceph-mon[31676]: from='client.? 172.21.15.45:0/3530341840' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:18:16.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:16.896+0000 7f3382400640 1 -- 172.21.15.45:0/1215557425 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f337c10a390 con 0x7f337c1071a0 2024-04-16T17:18:16.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:16.900+0000 7f337a000640 1 -- 172.21.15.45:0/1215557425 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v108) ==== 74+0+25285 (secure 0 0 0) 0x7f336409c7c0 con 0x7f337c1071a0 2024-04-16T17:18:16.901 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-16T17:18:16.902 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":108,"fsid":"b1fecd38-fc11-11ee-bc90-c7b262605968","created":"2024-04-16T16:54:02.385906+0000","modified":"2024-04-16T17:18:14.429571+0000","last_up_change":"2024-04-16T17:02:25.162624+0000","last_in_change":"2024-04-16T17:02:04.903274+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-16T16:58:06.166659+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-16T17:03:22.304198+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-16T17:03:23.616450+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"107","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":19,"snap_epoch":107,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-16T17:03:27.226289+0000","flags":8197,"flags_names":"hashpspool,ec_overwrites,selfmanaged_snaps","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"107","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":19,"snap_epoch":107,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b655de2c-59f8-4fd4-ac98-0cc222229f56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":2290170881},{"type":"v1","addr":"172.21.15.45:6803","nonce":2290170881}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":2290170881},{"type":"v1","addr":"172.21.15.45:6805","nonce":2290170881}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":2290170881},{"type":"v1","addr":"172.21.15.45:6809","nonce":2290170881}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":2290170881},{"type":"v1","addr":"172.21.15.45:6807","nonce":2290170881}]},"public_addr":"172.21.15.45:6803/2290170881","cluster_addr":"172.21.15.45:6805/2290170881","heartbeat_back_addr":"172.21.15.45:6809/2290170881","heartbeat_front_addr":"172.21.15.45:6807/2290170881","state":["exists","up"]},{"osd":1,"uuid":"ebd450ec-08fd-4c3d-8b24-4e10634ddb7e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6800","nonce":1462942067},{"type":"v1","addr":"172.21.15.104:6801","nonce":1462942067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":1462942067},{"type":"v1","addr":"172.21.15.104:6803","nonce":1462942067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":1462942067},{"type":"v1","addr":"172.21.15.104:6807","nonce":1462942067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":1462942067},{"type":"v1","addr":"172.21.15.104:6805","nonce":1462942067}]},"public_addr":"172.21.15.104:6801/1462942067","cluster_addr":"172.21.15.104:6803/1462942067","heartbeat_back_addr":"172.21.15.104:6807/1462942067","heartbeat_front_addr":"172.21.15.104:6805/1462942067","state":["exists","up"]},{"osd":2,"uuid":"b318a512-4190-42a2-8104-dcc4026ad8ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":2829260735},{"type":"v1","addr":"172.21.15.117:6801","nonce":2829260735}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":2829260735},{"type":"v1","addr":"172.21.15.117:6803","nonce":2829260735}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":2829260735},{"type":"v1","addr":"172.21.15.117:6807","nonce":2829260735}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":2829260735},{"type":"v1","addr":"172.21.15.117:6805","nonce":2829260735}]},"public_addr":"172.21.15.117:6801/2829260735","cluster_addr":"172.21.15.117:6803/2829260735","heartbeat_back_addr":"172.21.15.117:6807/2829260735","heartbeat_front_addr":"172.21.15.117:6805/2829260735","state":["exists","up"]},{"osd":3,"uuid":"ec6490cb-a715-4c12-a54a-225aae26fcb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":2823712137},{"type":"v1","addr":"172.21.15.45:6811","nonce":2823712137}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":2823712137},{"type":"v1","addr":"172.21.15.45:6813","nonce":2823712137}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":2823712137},{"type":"v1","addr":"172.21.15.45:6817","nonce":2823712137}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":2823712137},{"type":"v1","addr":"172.21.15.45:6815","nonce":2823712137}]},"public_addr":"172.21.15.45:6811/2823712137","cluster_addr":"172.21.15.45:6813/2823712137","heartbeat_back_addr":"172.21.15.45:6817/2823712137","heartbeat_front_addr":"172.21.15.45:6815/2823712137","state":["exists","up"]},{"osd":4,"uuid":"5d91d568-0250-4aae-8dc2-a0e079f92062","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":1396278127},{"type":"v1","addr":"172.21.15.104:6809","nonce":1396278127}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":1396278127},{"type":"v1","addr":"172.21.15.104:6811","nonce":1396278127}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":1396278127},{"type":"v1","addr":"172.21.15.104:6815","nonce":1396278127}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":1396278127},{"type":"v1","addr":"172.21.15.104:6813","nonce":1396278127}]},"public_addr":"172.21.15.104:6809/1396278127","cluster_addr":"172.21.15.104:6811/1396278127","heartbeat_back_addr":"172.21.15.104:6815/1396278127","heartbeat_front_addr":"172.21.15.104:6813/1396278127","state":["exists","up"]},{"osd":5,"uuid":"ac510b43-6fb2-4d0f-bc5f-16ac080d9e9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":4145720047},{"type":"v1","addr":"172.21.15.117:6809","nonce":4145720047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":4145720047},{"type":"v1","addr":"172.21.15.117:6811","nonce":4145720047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":4145720047},{"type":"v1","addr":"172.21.15.117:6815","nonce":4145720047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":4145720047},{"type":"v1","addr":"172.21.15.117:6813","nonce":4145720047}]},"public_addr":"172.21.15.117:6809/4145720047","cluster_addr":"172.21.15.117:6811/4145720047","heartbeat_back_addr":"172.21.15.117:6815/4145720047","heartbeat_front_addr":"172.21.15.117:6813/4145720047","state":["exists","up"]},{"osd":6,"uuid":"a41adc18-d88a-419e-b2c2-b089c1f32472","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":3254229176},{"type":"v1","addr":"172.21.15.45:6819","nonce":3254229176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":3254229176},{"type":"v1","addr":"172.21.15.45:6821","nonce":3254229176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":3254229176},{"type":"v1","addr":"172.21.15.45:6825","nonce":3254229176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":3254229176},{"type":"v1","addr":"172.21.15.45:6823","nonce":3254229176}]},"public_addr":"172.21.15.45:6819/3254229176","cluster_addr":"172.21.15.45:6821/3254229176","heartbeat_back_addr":"172.21.15.45:6825/3254229176","heartbeat_front_addr":"172.21.15.45:6823/3254229176","state":["exists","up"]},{"osd":7,"uuid":"b71712a4-7cf6-454d-9fd1-565e3df21c1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":4151077186},{"type":"v1","addr":"172.21.15.104:6817","nonce":4151077186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":4151077186},{"type":"v1","addr":"172.21.15.104:6819","nonce":4151077186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":4151077186},{"type":"v1","addr":"172.21.15.104:6823","nonce":4151077186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":4151077186},{"type":"v1","addr":"172.21.15.104:6821","nonce":4151077186}]},"public_addr":"172.21.15.104:6817/4151077186","cluster_addr":"172.21.15.104:6819/4151077186","heartbeat_back_addr":"172.21.15.104:6823/4151077186","heartbeat_front_addr":"172.21.15.104:6821/4151077186","state":["exists","up"]},{"osd":8,"uuid":"9e572aef-59a8-450e-a92e-df9dff4d57a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":1500286830},{"type":"v1","addr":"172.21.15.117:6817","nonce":1500286830}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":1500286830},{"type":"v1","addr":"172.21.15.117:6819","nonce":1500286830}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":1500286830},{"type":"v1","addr":"172.21.15.117:6823","nonce":1500286830}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":1500286830},{"type":"v1","addr":"172.21.15.117:6821","nonce":1500286830}]},"public_addr":"172.21.15.117:6817/1500286830","cluster_addr":"172.21.15.117:6819/1500286830","heartbeat_back_addr":"172.21.15.117:6823/1500286830","heartbeat_front_addr":"172.21.15.117:6821/1500286830","state":["exists","up"]},{"osd":9,"uuid":"014e9598-c413-4cc4-9bb8-61c6e8f59d30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":3106734074},{"type":"v1","addr":"172.21.15.45:6827","nonce":3106734074}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":3106734074},{"type":"v1","addr":"172.21.15.45:6829","nonce":3106734074}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":3106734074},{"type":"v1","addr":"172.21.15.45:6833","nonce":3106734074}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":3106734074},{"type":"v1","addr":"172.21.15.45:6831","nonce":3106734074}]},"public_addr":"172.21.15.45:6827/3106734074","cluster_addr":"172.21.15.45:6829/3106734074","heartbeat_back_addr":"172.21.15.45:6833/3106734074","heartbeat_front_addr":"172.21.15.45:6831/3106734074","state":["exists","up"]},{"osd":10,"uuid":"2deba364-3e39-47be-8a1f-8256df5b82d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":1404794652},{"type":"v1","addr":"172.21.15.104:6825","nonce":1404794652}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":1404794652},{"type":"v1","addr":"172.21.15.104:6827","nonce":1404794652}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":1404794652},{"type":"v1","addr":"172.21.15.104:6831","nonce":1404794652}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":1404794652},{"type":"v1","addr":"172.21.15.104:6829","nonce":1404794652}]},"public_addr":"172.21.15.104:6825/1404794652","cluster_addr":"172.21.15.104:6827/1404794652","heartbeat_back_addr":"172.21.15.104:6831/1404794652","heartbeat_front_addr":"172.21.15.104:6829/1404794652","state":["exists","up"]},{"osd":11,"uuid":"3fda4fea-59ad-4d1a-9fe6-fcb30e31fd7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":3384003243},{"type":"v1","addr":"172.21.15.117:6825","nonce":3384003243}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":3384003243},{"type":"v1","addr":"172.21.15.117:6827","nonce":3384003243}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":3384003243},{"type":"v1","addr":"172.21.15.117:6831","nonce":3384003243}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":3384003243},{"type":"v1","addr":"172.21.15.117:6829","nonce":3384003243}]},"public_addr":"172.21.15.117:6825/3384003243","cluster_addr":"172.21.15.117:6827/3384003243","heartbeat_back_addr":"172.21.15.117:6831/3384003243","heartbeat_front_addr":"172.21.15.117:6829/3384003243","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:57:14.799608+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:57:39.884668+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:58:03.327057+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:58:37.314531+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:59:03.667650+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T16:59:28.462823+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T17:00:02.968574+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T17:00:29.559190+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T17:00:55.291007+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T17:01:29.815977+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T17:01:57.015580+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-16T17:02:23.233151+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.5","mappings":[{"from":9,"to":10}]},{"pgid":"4.7","mappings":[{"from":5,"to":8}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.45:6800/268030699":"2024-04-17T17:06:45.093215+0000","172.21.15.45:6801/268030699":"2024-04-17T17:06:45.093215+0000","172.21.15.45:0/4074982598":"2024-04-17T16:55:19.948001+0000","172.21.15.45:0/1915281176":"2024-04-17T17:06:45.093215+0000","172.21.15.45:0/2443572576":"2024-04-17T16:55:19.948001+0000","172.21.15.45:0/1500147666":"2024-04-17T17:06:45.093215+0000","172.21.15.45:6801/1983630136":"2024-04-17T16:55:19.948001+0000","172.21.15.45:6800/1983630136":"2024-04-17T16:55:19.948001+0000","172.21.15.45:0/12001708":"2024-04-17T17:06:45.093215+0000","172.21.15.45:0/3150465248":"2024-04-17T16:54:43.692466+0000","172.21.15.45:0/556738623":"2024-04-17T16:54:43.692466+0000","172.21.15.45:0/445676297":"2024-04-17T16:55:19.948001+0000","172.21.15.45:6800/2553659622":"2024-04-17T16:54:43.692466+0000","172.21.15.45:6801/2553659622":"2024-04-17T16:54:43.692466+0000","172.21.15.45:0/265066921":"2024-04-17T17:06:45.093215+0000","172.21.15.45:0/1184615408":"2024-04-17T16:54:43.692466+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":7,"length":1},{"begin":19,"length":1}]},{"pool":4,"snaps":[{"begin":7,"length":1},{"begin":19,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-16T17:18:16.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:16.903+0000 7f3382400640 1 -- 172.21.15.45:0/1215557425 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f33600a3d20 msgr2=0x7f33600a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:18:16.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:16.903+0000 7f3382400640 1 --2- 172.21.15.45:0/1215557425 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f33600a3d20 0x7f33600a61e0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f336c0058c0 tx=0x7f336c009260 comp rx=0 tx=0).stop 2024-04-16T17:18:16.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:16.903+0000 7f3382400640 1 -- 172.21.15.45:0/1215557425 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f337c1071a0 msgr2=0x7f337c101600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:18:16.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:16.903+0000 7f3382400640 1 --2- 172.21.15.45:0/1215557425 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f337c1071a0 0x7f337c101600 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f337c077ec0 tx=0x7f336400ee50 comp rx=0 tx=0).stop 2024-04-16T17:18:16.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:16.903+0000 7f3382400640 1 -- 172.21.15.45:0/1215557425 shutdown_connections 2024-04-16T17:18:16.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:16.903+0000 7f3382400640 1 --2- 172.21.15.45:0/1215557425 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f33600a3d20 0x7f33600a61e0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:16.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:16.903+0000 7f3382400640 1 --2- 172.21.15.45:0/1215557425 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f337c102d60 0x7f337c101b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:16.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:16.903+0000 7f3382400640 1 --2- 172.21.15.45:0/1215557425 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f337c1071a0 0x7f337c101600 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:16.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:16.903+0000 7f3382400640 1 --2- 172.21.15.45:0/1215557425 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f337c069400 0x7f337c1010c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:16.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:16.903+0000 7f3382400640 1 -- 172.21.15.45:0/1215557425 >> 172.21.15.45:0/1215557425 conn(0x7f337c076840 msgr2=0x7f337c0ff0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:18:16.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:16.904+0000 7f3382400640 1 -- 172.21.15.45:0/1215557425 shutdown_connections 2024-04-16T17:18:16.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:16.904+0000 7f3382400640 1 -- 172.21.15.45:0/1215557425 wait complete. 2024-04-16T17:18:16.923 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-16T17:18:16.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:18:16 smithi104 ceph-mon[36457]: pgmap v361: 129 pgs: 129 active+clean; 51 MiB data, 871 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 127 B/s wr, 3 op/s 2024-04-16T17:18:16.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:18:16 smithi104 ceph-mon[36457]: from='client.? 172.21.15.45:0/3175400285' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:18:16.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:18:16 smithi104 ceph-mon[36457]: from='client.? 172.21.15.45:0/3530341840' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:18:17.054 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:18:16 smithi117 ceph-mon[36730]: pgmap v361: 129 pgs: 129 active+clean; 51 MiB data, 871 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 127 B/s wr, 3 op/s 2024-04-16T17:18:17.054 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:18:16 smithi117 ceph-mon[36730]: from='client.? 172.21.15.45:0/3175400285' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:18:17.054 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:18:16 smithi117 ceph-mon[36730]: from='client.? 172.21.15.45:0/3530341840' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-16T17:18:17.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:17.157+0000 7fdc49400640 1 -- 172.21.15.45:0/1883252594 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdc44109580 msgr2=0x7fdc44111150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:18:17.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:17.157+0000 7fdc49400640 1 --2- 172.21.15.45:0/1883252594 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdc44109580 0x7fdc44111150 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fdc3c009900 tx=0x7fdc3c02bc50 comp rx=0 tx=0).stop 2024-04-16T17:18:17.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:17.157+0000 7fdc49400640 1 -- 172.21.15.45:0/1883252594 shutdown_connections 2024-04-16T17:18:17.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:17.157+0000 7fdc49400640 1 --2- 172.21.15.45:0/1883252594 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdc44109580 0x7fdc44111150 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:17.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:17.157+0000 7fdc49400640 1 --2- 172.21.15.45:0/1883252594 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdc44104e60 0x7fdc44108ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:17.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:17.157+0000 7fdc49400640 1 --2- 172.21.15.45:0/1883252594 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdc441044b0 0x7fdc44104890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:17.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:17.157+0000 7fdc49400640 1 -- 172.21.15.45:0/1883252594 >> 172.21.15.45:0/1883252594 conn(0x7fdc440ffd90 msgr2=0x7fdc441021b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:18:17.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:17.157+0000 7fdc49400640 1 -- 172.21.15.45:0/1883252594 shutdown_connections 2024-04-16T17:18:17.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:17.157+0000 7fdc49400640 1 -- 172.21.15.45:0/1883252594 wait complete. 2024-04-16T17:18:17.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:17.158+0000 7fdc49400640 1 Processor -- start 2024-04-16T17:18:17.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:17.158+0000 7fdc49400640 1 -- start start 2024-04-16T17:18:17.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:17.159+0000 7fdc49400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdc441044b0 0x7fdc441a0840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:17.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:17.159+0000 7fdc49400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdc44109580 0x7fdc441a0d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:17.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:17.159+0000 7fdc49400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdc4419a9f0 0x7fdc4419aea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:17.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:17.159+0000 7fdc49400640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fdc44114690 con 0x7fdc4419a9f0 2024-04-16T17:18:17.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:17.159+0000 7fdc49400640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fdc44114510 con 0x7fdc441044b0 2024-04-16T17:18:17.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:17.159+0000 7fdc49400640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fdc44114810 con 0x7fdc44109580 2024-04-16T17:18:17.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:17.159+0000 7fdc43200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdc4419a9f0 0x7fdc4419aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:17.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:17.159+0000 7fdc43200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdc4419a9f0 0x7fdc4419aea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59938/0 (socket says 172.21.15.45:59938) 2024-04-16T17:18:17.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:17.159+0000 7fdc43200640 1 -- 172.21.15.45:0/1131743550 learned_addr learned my addr 172.21.15.45:0/1131743550 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:18:17.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:17.159+0000 7fdc42800640 1 --2- 172.21.15.45:0/1131743550 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdc441044b0 0x7fdc441a0840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:17.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:17.159+0000 7fdc39e00640 1 --2- 172.21.15.45:0/1131743550 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdc44109580 0x7fdc441a0d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:17.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:17.160+0000 7fdc43200640 1 -- 172.21.15.45:0/1131743550 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdc44109580 msgr2=0x7fdc441a0d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:18:17.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:17.160+0000 7fdc43200640 1 --2- 172.21.15.45:0/1131743550 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdc44109580 0x7fdc441a0d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:17.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:17.160+0000 7fdc43200640 1 -- 172.21.15.45:0/1131743550 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdc441044b0 msgr2=0x7fdc441a0840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:18:17.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:17.160+0000 7fdc43200640 1 --2- 172.21.15.45:0/1131743550 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdc441044b0 0x7fdc441a0840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:17.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:17.160+0000 7fdc43200640 1 -- 172.21.15.45:0/1131743550 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc4419b510 con 0x7fdc4419a9f0 2024-04-16T17:18:17.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:17.160+0000 7fdc42800640 1 --2- 172.21.15.45:0/1131743550 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdc441044b0 0x7fdc441a0840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:18:17.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:17.160+0000 7fdc39e00640 1 --2- 172.21.15.45:0/1131743550 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdc44109580 0x7fdc441a0d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:18:17.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:17.160+0000 7fdc43200640 1 --2- 172.21.15.45:0/1131743550 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdc4419a9f0 0x7fdc4419aea0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fdc3c02c190 tx=0x7fdc3c0042e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:18:17.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:17.160+0000 7fdc3be00640 1 -- 172.21.15.45:0/1131743550 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc3c041070 con 0x7fdc4419a9f0 2024-04-16T17:18:17.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:17.160+0000 7fdc49400640 1 -- 172.21.15.45:0/1131743550 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc4419b7a0 con 0x7fdc4419a9f0 2024-04-16T17:18:17.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:17.160+0000 7fdc3be00640 1 -- 172.21.15.45:0/1131743550 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc3c02ccb0 con 0x7fdc4419a9f0 2024-04-16T17:18:17.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:17.160+0000 7fdc3be00640 1 -- 172.21.15.45:0/1131743550 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc3c003dc0 con 0x7fdc4419a9f0 2024-04-16T17:18:17.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:17.161+0000 7fdc49400640 1 -- 172.21.15.45:0/1131743550 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc441a7a40 con 0x7fdc4419a9f0 2024-04-16T17:18:17.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:17.162+0000 7fdc49400640 1 -- 172.21.15.45:0/1131743550 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc44069270 con 0x7fdc4419a9f0 2024-04-16T17:18:17.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:17.164+0000 7fdc3be00640 1 -- 172.21.15.45:0/1131743550 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fdc3c044070 con 0x7fdc4419a9f0 2024-04-16T17:18:17.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:17.166+0000 7fdc3be00640 1 --2- 172.21.15.45:0/1131743550 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fdc280a3da0 0x7fdc280a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:17.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:17.167+0000 7fdc3be00640 1 -- 172.21.15.45:0/1131743550 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 9496+0+0 (secure 0 0 0) 0x7fdc3c0f24e0 con 0x7fdc4419a9f0 2024-04-16T17:18:17.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:17.167+0000 7fdc42800640 1 --2- 172.21.15.45:0/1131743550 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fdc280a3da0 0x7fdc280a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:17.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:17.168+0000 7fdc42800640 1 --2- 172.21.15.45:0/1131743550 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fdc280a3da0 0x7fdc280a6260 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fdc34009fd0 tx=0x7fdc34009680 comp rx=0 tx=0).ready entity=mgr.25411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:18:17.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:17.171+0000 7fdc3be00640 1 -- 172.21.15.45:0/1131743550 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fdc3c0be210 con 0x7fdc4419a9f0 2024-04-16T17:18:17.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:17.425+0000 7fdc49400640 1 -- 172.21.15.45:0/1131743550 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fdc4419bf20 con 0x7fdc4419a9f0 2024-04-16T17:18:17.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:17.429+0000 7fdc3be00640 1 -- 172.21.15.45:0/1131743550 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v45) ==== 161+0+0 (secure 0 0 0) 0x7fdc3c0c2510 con 0x7fdc4419a9f0 2024-04-16T17:18:17.430 INFO:teuthology.orchestra.run.smithi045.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-16T17:18:17.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:17.432+0000 7fdc49400640 1 -- 172.21.15.45:0/1131743550 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fdc280a3da0 msgr2=0x7fdc280a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:18:17.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:17.432+0000 7fdc49400640 1 --2- 172.21.15.45:0/1131743550 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fdc280a3da0 0x7fdc280a6260 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fdc34009fd0 tx=0x7fdc34009680 comp rx=0 tx=0).stop 2024-04-16T17:18:17.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:17.432+0000 7fdc49400640 1 -- 172.21.15.45:0/1131743550 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdc4419a9f0 msgr2=0x7fdc4419aea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:18:17.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:17.432+0000 7fdc49400640 1 --2- 172.21.15.45:0/1131743550 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdc4419a9f0 0x7fdc4419aea0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fdc3c02c190 tx=0x7fdc3c0042e0 comp rx=0 tx=0).stop 2024-04-16T17:18:17.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:17.433+0000 7fdc49400640 1 -- 172.21.15.45:0/1131743550 shutdown_connections 2024-04-16T17:18:17.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:17.433+0000 7fdc49400640 1 --2- 172.21.15.45:0/1131743550 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7fdc280a3da0 0x7fdc280a6260 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:17.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:17.433+0000 7fdc49400640 1 --2- 172.21.15.45:0/1131743550 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdc44109580 0x7fdc441a0d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:17.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:17.433+0000 7fdc49400640 1 --2- 172.21.15.45:0/1131743550 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdc4419a9f0 0x7fdc4419aea0 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:17.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:17.433+0000 7fdc49400640 1 --2- 172.21.15.45:0/1131743550 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdc441044b0 0x7fdc441a0840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:17.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:17.433+0000 7fdc49400640 1 -- 172.21.15.45:0/1131743550 >> 172.21.15.45:0/1131743550 conn(0x7fdc440ffd90 msgr2=0x7fdc441020e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:18:17.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:17.434+0000 7fdc49400640 1 -- 172.21.15.45:0/1131743550 shutdown_connections 2024-04-16T17:18:17.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:17.435+0000 7fdc49400640 1 -- 172.21.15.45:0/1131743550 wait complete. 2024-04-16T17:18:17.454 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-16T17:18:17.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:17.662+0000 7f31ada00640 1 -- 172.21.15.45:0/308067559 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f31a8100d60 msgr2=0x7f31a810a120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:18:17.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:17.662+0000 7f31ada00640 1 --2- 172.21.15.45:0/308067559 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f31a8100d60 0x7f31a810a120 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f3194009900 tx=0x7f319402bc30 comp rx=0 tx=0).stop 2024-04-16T17:18:17.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:17.663+0000 7f31ada00640 1 -- 172.21.15.45:0/308067559 shutdown_connections 2024-04-16T17:18:17.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:17.663+0000 7f31ada00640 1 --2- 172.21.15.45:0/308067559 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f31a8100d60 0x7f31a810a120 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:17.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:17.663+0000 7f31ada00640 1 --2- 172.21.15.45:0/308067559 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f31a810a800 0x7f31a81111d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:17.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:17.663+0000 7f31ada00640 1 --2- 172.21.15.45:0/308067559 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f31a81003b0 0x7f31a8100790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:17.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:17.663+0000 7f31ada00640 1 -- 172.21.15.45:0/308067559 >> 172.21.15.45:0/308067559 conn(0x7f31a8077060 msgr2=0x7f31a80fe450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:18:17.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:17.663+0000 7f31ada00640 1 -- 172.21.15.45:0/308067559 shutdown_connections 2024-04-16T17:18:17.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:17.663+0000 7f31ada00640 1 -- 172.21.15.45:0/308067559 wait complete. 2024-04-16T17:18:17.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:17.664+0000 7f31ada00640 1 Processor -- start 2024-04-16T17:18:17.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:17.664+0000 7f31ada00640 1 -- start start 2024-04-16T17:18:17.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:17.665+0000 7f31ada00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f31a81003b0 0x7f31a81a0800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:17.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:17.665+0000 7f31ada00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f31a810a800 0x7f31a81a0d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:17.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:17.665+0000 7f31ada00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f31a819a9b0 0x7f31a819ae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:17.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:17.665+0000 7f31ada00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f31a8114650 con 0x7f31a810a800 2024-04-16T17:18:17.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:17.665+0000 7f31ada00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f31a81144d0 con 0x7f31a819a9b0 2024-04-16T17:18:17.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:17.665+0000 7f31ada00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f31a81147d0 con 0x7f31a81003b0 2024-04-16T17:18:17.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:17.665+0000 7f31a7400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f31a819a9b0 0x7f31a819ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:17.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:17.665+0000 7f31a7400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f31a819a9b0 0x7f31a819ae60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.45:59392/0 (socket says 172.21.15.45:59392) 2024-04-16T17:18:17.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:17.665+0000 7f31a7400640 1 -- 172.21.15.45:0/3050343946 learned_addr learned my addr 172.21.15.45:0/3050343946 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:18:17.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:17.665+0000 7f31a6000640 1 --2- 172.21.15.45:0/3050343946 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f31a810a800 0x7f31a81a0d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:17.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:17.665+0000 7f31a6a00640 1 --2- 172.21.15.45:0/3050343946 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f31a81003b0 0x7f31a81a0800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:17.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:17.665+0000 7f31a7400640 1 -- 172.21.15.45:0/3050343946 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f31a81003b0 msgr2=0x7f31a81a0800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:18:17.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:17.665+0000 7f31a7400640 1 --2- 172.21.15.45:0/3050343946 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f31a81003b0 0x7f31a81a0800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:17.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:17.665+0000 7f31a7400640 1 -- 172.21.15.45:0/3050343946 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f31a810a800 msgr2=0x7f31a81a0d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:18:17.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:17.665+0000 7f31a7400640 1 --2- 172.21.15.45:0/3050343946 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f31a810a800 0x7f31a81a0d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:17.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:17.665+0000 7f31a7400640 1 -- 172.21.15.45:0/3050343946 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31a819b4d0 con 0x7f31a819a9b0 2024-04-16T17:18:17.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:17.666+0000 7f31a7400640 1 --2- 172.21.15.45:0/3050343946 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f31a819a9b0 0x7f31a819ae60 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f31a000e970 tx=0x7f31a000ee30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:18:17.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:17.666+0000 7f319f400640 1 -- 172.21.15.45:0/3050343946 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31a000ccc0 con 0x7f31a819a9b0 2024-04-16T17:18:17.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:17.666+0000 7f319f400640 1 -- 172.21.15.45:0/3050343946 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f31a000ce60 con 0x7f31a819a9b0 2024-04-16T17:18:17.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:17.666+0000 7f31ada00640 1 -- 172.21.15.45:0/3050343946 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31a819b7c0 con 0x7f31a819a9b0 2024-04-16T17:18:17.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:17.666+0000 7f319f400640 1 -- 172.21.15.45:0/3050343946 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31a0010430 con 0x7f31a819a9b0 2024-04-16T17:18:17.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:17.666+0000 7f31ada00640 1 -- 172.21.15.45:0/3050343946 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31a81a7a10 con 0x7f31a819a9b0 2024-04-16T17:18:17.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:17.667+0000 7f31ada00640 1 -- 172.21.15.45:0/3050343946 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f31a8069270 con 0x7f31a819a9b0 2024-04-16T17:18:17.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:17.669+0000 7f319f400640 1 -- 172.21.15.45:0/3050343946 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f31a0005000 con 0x7f31a819a9b0 2024-04-16T17:18:17.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:17.670+0000 7f319f400640 1 --2- 172.21.15.45:0/3050343946 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f318c0a3da0 0x7f318c0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:17.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:17.671+0000 7f319f400640 1 -- 172.21.15.45:0/3050343946 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 9666+0+0 (secure 0 0 0) 0x7f31a00d0f40 con 0x7f31a819a9b0 2024-04-16T17:18:17.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:17.671+0000 7f31a6a00640 1 --2- 172.21.15.45:0/3050343946 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f318c0a3da0 0x7f318c0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:17.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:17.671+0000 7f31a6a00640 1 --2- 172.21.15.45:0/3050343946 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f318c0a3da0 0x7f318c0a6260 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f31a819baf0 tx=0x7f3198009500 comp rx=0 tx=0).ready entity=mgr.25411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:18:17.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:17.675+0000 7f319f400640 1 -- 172.21.15.45:0/3050343946 <== mon.2 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f31a009d100 con 0x7f31a819a9b0 2024-04-16T17:18:17.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:18:17 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mon-a[31672]: 2024-04-16T17:18:17.426+0000 7f71ea200700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-16T17:18:17.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:18:17 smithi045 ceph-mon[31676]: from='client.? 172.21.15.45:0/1215557425' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T17:18:17.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:18:17 smithi045 ceph-mon[31676]: from='client.? 172.21.15.45:0/1131743550' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-16T17:18:17.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:18:17 smithi045 ceph-mon[31676]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-16T17:18:17.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:18:17 smithi045 ceph-mon[31676]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-16T17:18:17.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:18:17 smithi045 ceph-mon[31676]: from='client.? 172.21.15.45:0/1131743550' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-16T17:18:17.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:18:17 smithi045 ceph-mon[31676]: osdmap e109: 12 total, 12 up, 12 in 2024-04-16T17:18:17.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:18:17 smithi045 ceph-mon[31676]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-04-16T17:18:17.729 INFO:journalctl@ceph.mds.a.smithi045.stdout:Apr 16 17:18:17 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mds-a[73384]: ignoring --setuser ceph since I am not root 2024-04-16T17:18:17.729 INFO:journalctl@ceph.mds.a.smithi045.stdout:Apr 16 17:18:17 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mds-a[73384]: ignoring --setgroup ceph since I am not root 2024-04-16T17:18:17.729 INFO:journalctl@ceph.mds.a.smithi045.stdout:Apr 16 17:18:17 smithi045 ceph-mds[73388]: -- 172.21.15.45:0/656398600 <== mon.0 v2:172.21.15.45:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d448311a40 con 0x55d4482e4800 2024-04-16T17:18:17.729 INFO:journalctl@ceph.mds.a.smithi045.stdout:Apr 16 17:18:17 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mds-a[73384]: starting mds.a at 2024-04-16T17:18:17.804 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:18:17 smithi117 ceph-mon[36730]: from='client.? 172.21.15.45:0/1215557425' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T17:18:17.804 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:18:17 smithi117 ceph-mon[36730]: from='client.? 172.21.15.45:0/1131743550' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-16T17:18:17.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:18:17 smithi117 ceph-mon[36730]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-16T17:18:17.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:18:17 smithi117 ceph-mon[36730]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-16T17:18:17.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:18:17 smithi117 ceph-mon[36730]: from='client.? 172.21.15.45:0/1131743550' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-16T17:18:17.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:18:17 smithi117 ceph-mon[36730]: osdmap e109: 12 total, 12 up, 12 in 2024-04-16T17:18:17.805 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:18:17 smithi117 ceph-mon[36730]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-04-16T17:18:17.805 INFO:journalctl@ceph.mds.c.smithi117.stdout:Apr 16 17:18:17 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mds-c[67668]: ignoring --setuser ceph since I am not root 2024-04-16T17:18:17.806 INFO:journalctl@ceph.mds.c.smithi117.stdout:Apr 16 17:18:17 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mds-c[67668]: ignoring --setgroup ceph since I am not root 2024-04-16T17:18:17.806 INFO:journalctl@ceph.mds.c.smithi117.stdout:Apr 16 17:18:17 smithi117 ceph-mds[67672]: -- 172.21.15.117:0/209283684 <== mon.2 v2:172.21.15.104:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x564e2e655a40 con 0x564e2e628c00 2024-04-16T17:18:17.806 INFO:journalctl@ceph.mds.c.smithi117.stdout:Apr 16 17:18:17 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mds-c[67668]: starting mds.c at 2024-04-16T17:18:17.806 INFO:journalctl@ceph.mds.f.smithi117.stdout:Apr 16 17:18:17 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mds-f[67363]: ignoring --setuser ceph since I am not root 2024-04-16T17:18:17.806 INFO:journalctl@ceph.mds.f.smithi117.stdout:Apr 16 17:18:17 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mds-f[67363]: ignoring --setgroup ceph since I am not root 2024-04-16T17:18:17.806 INFO:journalctl@ceph.mds.f.smithi117.stdout:Apr 16 17:18:17 smithi117 ceph-mds[67367]: -- 172.21.15.117:0/4240922751 <== mon.2 v2:172.21.15.104:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56120e961a40 con 0x56120e934c00 2024-04-16T17:18:17.807 INFO:journalctl@ceph.mds.f.smithi117.stdout:Apr 16 17:18:17 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mds-f[67363]: starting mds.f at 2024-04-16T17:18:17.807 INFO:journalctl@ceph.mds.i.smithi117.stdout:Apr 16 17:18:17 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mds-i[67974]: ignoring --setuser ceph since I am not root 2024-04-16T17:18:17.807 INFO:journalctl@ceph.mds.i.smithi117.stdout:Apr 16 17:18:17 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mds-i[67974]: ignoring --setgroup ceph since I am not root 2024-04-16T17:18:17.807 INFO:journalctl@ceph.mds.i.smithi117.stdout:Apr 16 17:18:17 smithi117 ceph-mds[67979]: -- 172.21.15.117:0/1731089973 <== mon.1 v2:172.21.15.117:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x557821e5da40 con 0x557821e30800 2024-04-16T17:18:17.807 INFO:journalctl@ceph.mds.i.smithi117.stdout:Apr 16 17:18:17 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mds-i[67974]: starting mds.i at 2024-04-16T17:18:17.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:17.920+0000 7f31ada00640 1 -- 172.21.15.45:0/3050343946 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f31a810c7c0 con 0x7f31a819a9b0 2024-04-16T17:18:17.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:18:17 smithi104 ceph-mon[36457]: from='client.? 172.21.15.45:0/1215557425' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-16T17:18:17.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:18:17 smithi104 ceph-mon[36457]: from='client.? 172.21.15.45:0/1131743550' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-16T17:18:17.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:18:17 smithi104 ceph-mon[36457]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-16T17:18:17.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:18:17 smithi104 ceph-mon[36457]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-16T17:18:17.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:18:17 smithi104 ceph-mon[36457]: from='client.? 172.21.15.45:0/1131743550' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-16T17:18:17.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:18:17 smithi104 ceph-mon[36457]: osdmap e109: 12 total, 12 up, 12 in 2024-04-16T17:18:17.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:18:17 smithi104 ceph-mon[36457]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-04-16T17:18:17.928 INFO:journalctl@ceph.mds.b.smithi104.stdout:Apr 16 17:18:17 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mds-b[67798]: ignoring --setuser ceph since I am not root 2024-04-16T17:18:17.928 INFO:journalctl@ceph.mds.b.smithi104.stdout:Apr 16 17:18:17 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mds-b[67798]: ignoring --setgroup ceph since I am not root 2024-04-16T17:18:17.928 INFO:journalctl@ceph.mds.b.smithi104.stdout:Apr 16 17:18:17 smithi104 ceph-mds[67802]: -- 172.21.15.104:0/2116984862 <== mon.1 v2:172.21.15.117:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5621f460ba40 con 0x5621f45df000 2024-04-16T17:18:17.929 INFO:journalctl@ceph.mds.b.smithi104.stdout:Apr 16 17:18:17 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mds-b[67798]: starting mds.b at 2024-04-16T17:18:17.929 INFO:journalctl@ceph.mds.h.smithi104.stdout:Apr 16 17:18:17 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mds-h[67477]: ignoring --setuser ceph since I am not root 2024-04-16T17:18:17.929 INFO:journalctl@ceph.mds.h.smithi104.stdout:Apr 16 17:18:17 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mds-h[67477]: ignoring --setgroup ceph since I am not root 2024-04-16T17:18:17.929 INFO:journalctl@ceph.mds.h.smithi104.stdout:Apr 16 17:18:17 smithi104 ceph-mds[67481]: -- 172.21.15.104:0/3430266170 <== mon.2 v2:172.21.15.104:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x563d8bc29a40 con 0x563d8bbfd000 2024-04-16T17:18:17.929 INFO:journalctl@ceph.mds.h.smithi104.stdout:Apr 16 17:18:17 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mds-h[67477]: starting mds.h at 2024-04-16T17:18:18.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:18.443+0000 7f319f400640 1 -- 172.21.15.45:0/3050343946 <== mon.2 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v46) ==== 104+0+0 (secure 0 0 0) 0x7f31a009ca00 con 0x7f31a819a9b0 2024-04-16T17:18:18.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:18.448+0000 7f31ada00640 1 -- 172.21.15.45:0/3050343946 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f318c0a3da0 msgr2=0x7f318c0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:18:18.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:18.448+0000 7f31ada00640 1 --2- 172.21.15.45:0/3050343946 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f318c0a3da0 0x7f318c0a6260 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f31a819baf0 tx=0x7f3198009500 comp rx=0 tx=0).stop 2024-04-16T17:18:18.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:18.448+0000 7f31ada00640 1 -- 172.21.15.45:0/3050343946 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f31a819a9b0 msgr2=0x7f31a819ae60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:18:18.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:18.448+0000 7f31ada00640 1 --2- 172.21.15.45:0/3050343946 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f31a819a9b0 0x7f31a819ae60 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f31a000e970 tx=0x7f31a000ee30 comp rx=0 tx=0).stop 2024-04-16T17:18:18.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:18.449+0000 7f31ada00640 1 -- 172.21.15.45:0/3050343946 shutdown_connections 2024-04-16T17:18:18.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:18.450+0000 7f31ada00640 1 --2- 172.21.15.45:0/3050343946 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f318c0a3da0 0x7f318c0a6260 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:18.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:18.450+0000 7f31ada00640 1 --2- 172.21.15.45:0/3050343946 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f31a819a9b0 0x7f31a819ae60 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:18.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:18.450+0000 7f31ada00640 1 --2- 172.21.15.45:0/3050343946 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f31a810a800 0x7f31a81a0d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:18.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:18.450+0000 7f31ada00640 1 --2- 172.21.15.45:0/3050343946 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f31a81003b0 0x7f31a81a0800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:18.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:18.450+0000 7f31ada00640 1 -- 172.21.15.45:0/3050343946 >> 172.21.15.45:0/3050343946 conn(0x7f31a8077060 msgr2=0x7f31a80fe450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:18:18.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:18.451+0000 7f31ada00640 1 -- 172.21.15.45:0/3050343946 shutdown_connections 2024-04-16T17:18:18.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:18.451+0000 7f31ada00640 1 -- 172.21.15.45:0/3050343946 wait complete. 2024-04-16T17:18:18.475 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-16T17:18:18.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:18.686+0000 7f9891c00640 1 -- 172.21.15.45:0/966952730 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f988c076530 msgr2=0x7f988c076910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:18:18.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:18.686+0000 7f9891c00640 1 --2- 172.21.15.45:0/966952730 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f988c076530 0x7f988c076910 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f987c009890 tx=0x7f987c02bb70 comp rx=0 tx=0).stop 2024-04-16T17:18:18.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:18.686+0000 7f9891c00640 1 -- 172.21.15.45:0/966952730 shutdown_connections 2024-04-16T17:18:18.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:18.686+0000 7f9891c00640 1 --2- 172.21.15.45:0/966952730 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f988c112c50 0x7f988c115040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:18.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:18.686+0000 7f9891c00640 1 --2- 172.21.15.45:0/966952730 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f988c076e50 0x7f988c112710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:18.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:18.686+0000 7f9891c00640 1 --2- 172.21.15.45:0/966952730 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f988c076530 0x7f988c076910 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:18.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:18.686+0000 7f9891c00640 1 -- 172.21.15.45:0/966952730 >> 172.21.15.45:0/966952730 conn(0x7f988c0ffa60 msgr2=0x7f988c101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:18:18.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:18.686+0000 7f9891c00640 1 -- 172.21.15.45:0/966952730 shutdown_connections 2024-04-16T17:18:18.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:18.686+0000 7f9891c00640 1 -- 172.21.15.45:0/966952730 wait complete. 2024-04-16T17:18:18.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:18.687+0000 7f9891c00640 1 Processor -- start 2024-04-16T17:18:18.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:18.687+0000 7f9891c00640 1 -- start start 2024-04-16T17:18:18.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:18.688+0000 7f9891c00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f988c076530 0x7f988c1a4a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:18.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:18.688+0000 7f9891c00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f988c076e50 0x7f988c1a4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:18.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:18.688+0000 7f9891c00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f988c112c50 0x7f988c19eb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:18.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:18.688+0000 7f9891c00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f988c1187c0 con 0x7f988c076e50 2024-04-16T17:18:18.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:18.688+0000 7f9891c00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f988c118640 con 0x7f988c112c50 2024-04-16T17:18:18.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:18.688+0000 7f9891c00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f988c118940 con 0x7f988c076530 2024-04-16T17:18:18.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:18.688+0000 7f988b400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f988c076e50 0x7f988c1a4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:18.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:18.688+0000 7f988b400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f988c076e50 0x7f988c1a4fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59986/0 (socket says 172.21.15.45:59986) 2024-04-16T17:18:18.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:18.688+0000 7f988b400640 1 -- 172.21.15.45:0/3838323001 learned_addr learned my addr 172.21.15.45:0/3838323001 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:18:18.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:18.688+0000 7f988be00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f988c076530 0x7f988c1a4a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:18.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:18.688+0000 7f9891200640 1 --2- 172.21.15.45:0/3838323001 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f988c112c50 0x7f988c19eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:18.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:18.689+0000 7f988b400640 1 -- 172.21.15.45:0/3838323001 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f988c076530 msgr2=0x7f988c1a4a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:18:18.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:18.689+0000 7f988b400640 1 --2- 172.21.15.45:0/3838323001 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f988c076530 0x7f988c1a4a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:18.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:18.689+0000 7f988b400640 1 -- 172.21.15.45:0/3838323001 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f988c112c50 msgr2=0x7f988c19eb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:18:18.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:18.689+0000 7f988b400640 1 --2- 172.21.15.45:0/3838323001 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f988c112c50 0x7f988c19eb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:18.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:18.689+0000 7f988b400640 1 -- 172.21.15.45:0/3838323001 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f988c19f3e0 con 0x7f988c076e50 2024-04-16T17:18:18.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:18.689+0000 7f988be00640 1 --2- 172.21.15.45:0/3838323001 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f988c076530 0x7f988c1a4a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:18:18.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:18.689+0000 7f988b400640 1 --2- 172.21.15.45:0/3838323001 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f988c076e50 0x7f988c1a4fa0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f987800efb0 tx=0x7f987800c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:18:18.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:18.689+0000 7f9888c00640 1 -- 172.21.15.45:0/3838323001 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f987801b070 con 0x7f988c076e50 2024-04-16T17:18:18.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:18.689+0000 7f9888c00640 1 -- 172.21.15.45:0/3838323001 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f987800ba50 con 0x7f988c076e50 2024-04-16T17:18:18.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:18.689+0000 7f9888c00640 1 -- 172.21.15.45:0/3838323001 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9878016c00 con 0x7f988c076e50 2024-04-16T17:18:18.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:18.689+0000 7f9891c00640 1 -- 172.21.15.45:0/3838323001 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f988c19f6d0 con 0x7f988c076e50 2024-04-16T17:18:18.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:18.690+0000 7f9891c00640 1 -- 172.21.15.45:0/3838323001 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f988c1ab9e0 con 0x7f988c076e50 2024-04-16T17:18:18.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:18.691+0000 7f9891c00640 1 -- 172.21.15.45:0/3838323001 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f988c077c70 con 0x7f988c076e50 2024-04-16T17:18:18.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:18.694+0000 7f9888c00640 1 -- 172.21.15.45:0/3838323001 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f987801e070 con 0x7f988c076e50 2024-04-16T17:18:18.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:18.695+0000 7f9888c00640 1 --2- 172.21.15.45:0/3838323001 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f98700a3d20 0x7f98700a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:18.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:18.695+0000 7f9888c00640 1 -- 172.21.15.45:0/3838323001 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 9666+0+0 (secure 0 0 0) 0x7f98780d0c00 con 0x7f988c076e50 2024-04-16T17:18:18.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:18.695+0000 7f988be00640 1 --2- 172.21.15.45:0/3838323001 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f98700a3d20 0x7f98700a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:18.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:18.696+0000 7f988be00640 1 --2- 172.21.15.45:0/3838323001 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f98700a3d20 0x7f98700a61e0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f987c037040 tx=0x7f987c02e040 comp rx=0 tx=0).ready entity=mgr.25411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:18:18.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:18.700+0000 7f9888c00640 1 -- 172.21.15.45:0/3838323001 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f987809c770 con 0x7f988c076e50 2024-04-16T17:18:18.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:18:18 smithi045 ceph-mon[31676]: pgmap v363: 129 pgs: 129 active+clean; 51 MiB data, 871 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1.4 KiB/s wr, 10 op/s 2024-04-16T17:18:18.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:18:18 smithi045 ceph-mon[31676]: from='client.? 172.21.15.45:0/3050343946' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-16T17:18:18.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:18:18 smithi045 ceph-mon[31676]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-16T17:18:18.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:18:18 smithi045 ceph-mon[31676]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-16T17:18:18.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:18:18 smithi045 ceph-mon[31676]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-16T17:18:18.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:18:18 smithi045 ceph-mon[31676]: Cluster is now healthy 2024-04-16T17:18:18.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:18:18 smithi045 ceph-mon[31676]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-16T17:18:18.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:18:18 smithi045 ceph-mon[31676]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-16T17:18:18.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:18:18 smithi045 ceph-mon[31676]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-16T17:18:18.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:18:18 smithi045 ceph-mon[31676]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-16T17:18:18.729 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:18:18 smithi045 ceph-mon[31676]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-16T17:18:18.729 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:18:18 smithi045 ceph-mon[31676]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-16T17:18:18.729 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:18:18 smithi045 ceph-mon[31676]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-16T17:18:18.729 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:18:18 smithi045 ceph-mon[31676]: mds.? [v2:172.21.15.117:6836/3545882875,v1:172.21.15.117:6838/3545882875] up:boot 2024-04-16T17:18:18.729 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:18:18 smithi045 ceph-mon[31676]: mds.? [v2:172.21.15.117:6834/3238446395,v1:172.21.15.117:6835/3238446395] up:boot 2024-04-16T17:18:18.729 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:18:18 smithi045 ceph-mon[31676]: mds.? [v2:172.21.15.104:6837/2236645881,v1:172.21.15.104:6839/2236645881] up:boot 2024-04-16T17:18:18.729 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:18:18 smithi045 ceph-mon[31676]: mds.? [v2:172.21.15.104:6836/954085536,v1:172.21.15.104:6838/954085536] up:boot 2024-04-16T17:18:18.729 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:18:18 smithi045 ceph-mon[31676]: mds.? [v2:172.21.15.117:6837/4129461385,v1:172.21.15.117:6839/4129461385] up:boot 2024-04-16T17:18:18.729 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:18:18 smithi045 ceph-mon[31676]: mds.? [v2:172.21.15.45:6840/2426996029,v1:172.21.15.45:6841/2426996029] up:boot 2024-04-16T17:18:18.730 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:18:18 smithi045 ceph-mon[31676]: fsmap 12 up:standby 2024-04-16T17:18:18.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:18.907+0000 7f9891c00640 1 -- 172.21.15.45:0/3838323001 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f988c10dde0 con 0x7f988c076e50 2024-04-16T17:18:18.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:18:18 smithi104 ceph-mon[36457]: pgmap v363: 129 pgs: 129 active+clean; 51 MiB data, 871 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1.4 KiB/s wr, 10 op/s 2024-04-16T17:18:18.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:18:18 smithi104 ceph-mon[36457]: from='client.? 172.21.15.45:0/3050343946' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-16T17:18:18.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:18:18 smithi104 ceph-mon[36457]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-16T17:18:18.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:18:18 smithi104 ceph-mon[36457]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-16T17:18:18.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:18:18 smithi104 ceph-mon[36457]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-16T17:18:18.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:18:18 smithi104 ceph-mon[36457]: Cluster is now healthy 2024-04-16T17:18:18.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:18:18 smithi104 ceph-mon[36457]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-16T17:18:18.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:18:18 smithi104 ceph-mon[36457]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-16T17:18:18.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:18:18 smithi104 ceph-mon[36457]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-16T17:18:18.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:18:18 smithi104 ceph-mon[36457]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-16T17:18:18.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:18:18 smithi104 ceph-mon[36457]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-16T17:18:18.928 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:18:18 smithi104 ceph-mon[36457]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-16T17:18:18.929 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:18:18 smithi104 ceph-mon[36457]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-16T17:18:18.929 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:18:18 smithi104 ceph-mon[36457]: mds.? [v2:172.21.15.117:6836/3545882875,v1:172.21.15.117:6838/3545882875] up:boot 2024-04-16T17:18:18.929 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:18:18 smithi104 ceph-mon[36457]: mds.? [v2:172.21.15.117:6834/3238446395,v1:172.21.15.117:6835/3238446395] up:boot 2024-04-16T17:18:18.929 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:18:18 smithi104 ceph-mon[36457]: mds.? [v2:172.21.15.104:6837/2236645881,v1:172.21.15.104:6839/2236645881] up:boot 2024-04-16T17:18:18.929 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:18:18 smithi104 ceph-mon[36457]: mds.? [v2:172.21.15.104:6836/954085536,v1:172.21.15.104:6838/954085536] up:boot 2024-04-16T17:18:18.929 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:18:18 smithi104 ceph-mon[36457]: mds.? [v2:172.21.15.117:6837/4129461385,v1:172.21.15.117:6839/4129461385] up:boot 2024-04-16T17:18:18.929 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:18:18 smithi104 ceph-mon[36457]: mds.? [v2:172.21.15.45:6840/2426996029,v1:172.21.15.45:6841/2426996029] up:boot 2024-04-16T17:18:18.929 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:18:18 smithi104 ceph-mon[36457]: fsmap 12 up:standby 2024-04-16T17:18:19.054 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:18:18 smithi117 ceph-mon[36730]: pgmap v363: 129 pgs: 129 active+clean; 51 MiB data, 871 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1.4 KiB/s wr, 10 op/s 2024-04-16T17:18:19.055 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:18:18 smithi117 ceph-mon[36730]: from='client.? 172.21.15.45:0/3050343946' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-16T17:18:19.055 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:18:18 smithi117 ceph-mon[36730]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-16T17:18:19.055 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:18:18 smithi117 ceph-mon[36730]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-16T17:18:19.055 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:18:18 smithi117 ceph-mon[36730]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-16T17:18:19.055 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:18:18 smithi117 ceph-mon[36730]: Cluster is now healthy 2024-04-16T17:18:19.055 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:18:18 smithi117 ceph-mon[36730]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-16T17:18:19.055 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:18:18 smithi117 ceph-mon[36730]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-16T17:18:19.055 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:18:18 smithi117 ceph-mon[36730]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-16T17:18:19.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:18:18 smithi117 ceph-mon[36730]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-16T17:18:19.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:18:18 smithi117 ceph-mon[36730]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-16T17:18:19.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:18:18 smithi117 ceph-mon[36730]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-16T17:18:19.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:18:18 smithi117 ceph-mon[36730]: from='mgr.25411 172.21.15.45:0/3110654480' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-16T17:18:19.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:18:18 smithi117 ceph-mon[36730]: mds.? [v2:172.21.15.117:6836/3545882875,v1:172.21.15.117:6838/3545882875] up:boot 2024-04-16T17:18:19.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:18:18 smithi117 ceph-mon[36730]: mds.? [v2:172.21.15.117:6834/3238446395,v1:172.21.15.117:6835/3238446395] up:boot 2024-04-16T17:18:19.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:18:18 smithi117 ceph-mon[36730]: mds.? [v2:172.21.15.104:6837/2236645881,v1:172.21.15.104:6839/2236645881] up:boot 2024-04-16T17:18:19.056 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:18:18 smithi117 ceph-mon[36730]: mds.? [v2:172.21.15.104:6836/954085536,v1:172.21.15.104:6838/954085536] up:boot 2024-04-16T17:18:19.057 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:18:18 smithi117 ceph-mon[36730]: mds.? [v2:172.21.15.117:6837/4129461385,v1:172.21.15.117:6839/4129461385] up:boot 2024-04-16T17:18:19.057 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:18:18 smithi117 ceph-mon[36730]: mds.? [v2:172.21.15.45:6840/2426996029,v1:172.21.15.45:6841/2426996029] up:boot 2024-04-16T17:18:19.057 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:18:18 smithi117 ceph-mon[36730]: fsmap 12 up:standby 2024-04-16T17:18:19.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:19.665+0000 7f9888c00640 1 -- 172.21.15.45:0/3838323001 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v110) ==== 181+0+0 (secure 0 0 0) 0x7f98780a0a70 con 0x7f988c076e50 2024-04-16T17:18:19.666 INFO:teuthology.orchestra.run.smithi045.stderr:pool 'cephfs_metadata' removed 2024-04-16T17:18:19.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:19.669+0000 7f9891c00640 1 -- 172.21.15.45:0/3838323001 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f98700a3d20 msgr2=0x7f98700a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:18:19.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:19.669+0000 7f9891c00640 1 --2- 172.21.15.45:0/3838323001 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f98700a3d20 0x7f98700a61e0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f987c037040 tx=0x7f987c02e040 comp rx=0 tx=0).stop 2024-04-16T17:18:19.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:19.669+0000 7f9891c00640 1 -- 172.21.15.45:0/3838323001 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f988c076e50 msgr2=0x7f988c1a4fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:18:19.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:19.669+0000 7f9891c00640 1 --2- 172.21.15.45:0/3838323001 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f988c076e50 0x7f988c1a4fa0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f987800efb0 tx=0x7f987800c800 comp rx=0 tx=0).stop 2024-04-16T17:18:19.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:19.669+0000 7f9891c00640 1 -- 172.21.15.45:0/3838323001 shutdown_connections 2024-04-16T17:18:19.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:19.669+0000 7f9891c00640 1 --2- 172.21.15.45:0/3838323001 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f98700a3d20 0x7f98700a61e0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:19.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:19.669+0000 7f9891c00640 1 --2- 172.21.15.45:0/3838323001 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f988c112c50 0x7f988c19eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:19.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:19.669+0000 7f9891c00640 1 --2- 172.21.15.45:0/3838323001 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f988c076e50 0x7f988c1a4fa0 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:19.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:19.669+0000 7f9891c00640 1 --2- 172.21.15.45:0/3838323001 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f988c076530 0x7f988c1a4a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:19.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:19.669+0000 7f9891c00640 1 -- 172.21.15.45:0/3838323001 >> 172.21.15.45:0/3838323001 conn(0x7f988c0ffa60 msgr2=0x7f988c114370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:18:19.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:19.671+0000 7f9891c00640 1 -- 172.21.15.45:0/3838323001 shutdown_connections 2024-04-16T17:18:19.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:19.673+0000 7f9891c00640 1 -- 172.21.15.45:0/3838323001 wait complete. 2024-04-16T17:18:19.701 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-16T17:18:19.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:19.924+0000 7f0545e00640 1 -- 172.21.15.45:0/929464742 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0540108610 msgr2=0x7f0540108a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:18:19.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:19.924+0000 7f0545e00640 1 --2- 172.21.15.45:0/929464742 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0540108610 0x7f0540108a90 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f052c009860 tx=0x7f052c02bb70 comp rx=0 tx=0).stop 2024-04-16T17:18:19.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:19.924+0000 7f0545e00640 1 -- 172.21.15.45:0/929464742 shutdown_connections 2024-04-16T17:18:19.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:19.924+0000 7f0545e00640 1 --2- 172.21.15.45:0/929464742 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0540108fd0 0x7f0540115490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:19.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:19.924+0000 7f0545e00640 1 --2- 172.21.15.45:0/929464742 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0540108610 0x7f0540108a90 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:19.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:19.924+0000 7f0545e00640 1 --2- 172.21.15.45:0/929464742 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f054010e660 0x7f054010ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:19.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:19.924+0000 7f0545e00640 1 -- 172.21.15.45:0/929464742 >> 172.21.15.45:0/929464742 conn(0x7f0540076540 msgr2=0x7f0540078960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:18:19.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:19.924+0000 7f0545e00640 1 -- 172.21.15.45:0/929464742 shutdown_connections 2024-04-16T17:18:19.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:19.924+0000 7f0545e00640 1 -- 172.21.15.45:0/929464742 wait complete. 2024-04-16T17:18:19.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:18:19 smithi104 ceph-mon[36457]: from='client.? 172.21.15.45:0/3838323001' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-16T17:18:19.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:19.925+0000 7f0545e00640 1 Processor -- start 2024-04-16T17:18:19.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:19.925+0000 7f0545e00640 1 -- start start 2024-04-16T17:18:19.929 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:19.926+0000 7f0545e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0540108610 0x7f054019ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:19.929 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:19.926+0000 7f0545e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0540108fd0 0x7f054019f380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:19.929 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:19.926+0000 7f0545e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f054010e660 0x7f05401a3700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:19.929 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:19.926+0000 7f0545e00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f05401189a0 con 0x7f0540108610 2024-04-16T17:18:19.929 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:19.926+0000 7f0545e00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f0540118820 con 0x7f054010e660 2024-04-16T17:18:19.929 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:19.926+0000 7f0545e00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f0540118b20 con 0x7f0540108fd0 2024-04-16T17:18:19.929 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:19.926+0000 7f053ea00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0540108610 0x7f054019ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:19.929 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:19.926+0000 7f053f400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f054010e660 0x7f05401a3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:19.929 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:19.926+0000 7f053ea00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0540108610 0x7f054019ee40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33404/0 (socket says 172.21.15.45:33404) 2024-04-16T17:18:19.930 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:19.926+0000 7f053ea00640 1 -- 172.21.15.45:0/53610318 learned_addr learned my addr 172.21.15.45:0/53610318 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:18:19.930 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:19.926+0000 7f053e000640 1 --2- 172.21.15.45:0/53610318 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0540108fd0 0x7f054019f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:19.930 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:19.926+0000 7f053ea00640 1 -- 172.21.15.45:0/53610318 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0540108fd0 msgr2=0x7f054019f380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:18:19.930 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:19.926+0000 7f053ea00640 1 --2- 172.21.15.45:0/53610318 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0540108fd0 0x7f054019f380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:19.930 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:19.926+0000 7f053ea00640 1 -- 172.21.15.45:0/53610318 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f054010e660 msgr2=0x7f05401a3700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:18:19.930 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:19.926+0000 7f053ea00640 1 --2- 172.21.15.45:0/53610318 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f054010e660 0x7f05401a3700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:19.930 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:19.926+0000 7f053ea00640 1 -- 172.21.15.45:0/53610318 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05401a3e30 con 0x7f0540108610 2024-04-16T17:18:19.930 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:19.927+0000 7f053e000640 1 --2- 172.21.15.45:0/53610318 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0540108fd0 0x7f054019f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:18:19.930 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:19.927+0000 7f053ea00640 1 --2- 172.21.15.45:0/53610318 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0540108610 0x7f054019ee40 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f053000e9a0 tx=0x7f053000eda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:18:19.931 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:19.927+0000 7f0544a00640 1 -- 172.21.15.45:0/53610318 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f053000ccc0 con 0x7f0540108610 2024-04-16T17:18:19.931 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:19.927+0000 7f0545e00640 1 -- 172.21.15.45:0/53610318 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05401a4120 con 0x7f0540108610 2024-04-16T17:18:19.931 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:19.927+0000 7f0544a00640 1 -- 172.21.15.45:0/53610318 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0530010910 con 0x7f0540108610 2024-04-16T17:18:19.931 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:19.927+0000 7f0544a00640 1 -- 172.21.15.45:0/53610318 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0530018a30 con 0x7f0540108610 2024-04-16T17:18:19.931 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:19.927+0000 7f0545e00640 1 -- 172.21.15.45:0/53610318 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05401ab9b0 con 0x7f0540108610 2024-04-16T17:18:19.931 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:19.929+0000 7f0545e00640 1 -- 172.21.15.45:0/53610318 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0540069270 con 0x7f0540108610 2024-04-16T17:18:19.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:19.932+0000 7f0544a00640 1 -- 172.21.15.45:0/53610318 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f053001f020 con 0x7f0540108610 2024-04-16T17:18:19.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:19.933+0000 7f0544a00640 1 --2- 172.21.15.45:0/53610318 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f05240a3d20 0x7f05240a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:19.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:19.933+0000 7f0544a00640 1 -- 172.21.15.45:0/53610318 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 9112+0+0 (secure 0 0 0) 0x7f05300d11f0 con 0x7f0540108610 2024-04-16T17:18:19.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:19.934+0000 7f053e000640 1 --2- 172.21.15.45:0/53610318 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f05240a3d20 0x7f05240a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:19.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:19.934+0000 7f053e000640 1 --2- 172.21.15.45:0/53610318 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f05240a3d20 0x7f05240a61e0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f052c02c180 tx=0x7f052c02e040 comp rx=0 tx=0).ready entity=mgr.25411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:18:19.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:19.937+0000 7f0544a00640 1 -- 172.21.15.45:0/53610318 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f053009d640 con 0x7f0540108610 2024-04-16T17:18:19.976 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:18:19 smithi045 ceph-mon[31676]: from='client.? 172.21.15.45:0/3838323001' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-16T17:18:20.054 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:18:19 smithi117 ceph-mon[36730]: from='client.? 172.21.15.45:0/3838323001' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-16T17:18:20.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:20.154+0000 7f0545e00640 1 -- 172.21.15.45:0/53610318 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f0540110b50 con 0x7f0540108610 2024-04-16T17:18:20.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:20.669+0000 7f0544a00640 1 -- 172.21.15.45:0/53610318 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v111) ==== 169+0+0 (secure 0 0 0) 0x7f053009cf40 con 0x7f0540108610 2024-04-16T17:18:20.669 INFO:teuthology.orchestra.run.smithi045.stderr:pool 'cephfs_data' removed 2024-04-16T17:18:20.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:20.675+0000 7f0545e00640 1 -- 172.21.15.45:0/53610318 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f05240a3d20 msgr2=0x7f05240a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:18:20.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:20.675+0000 7f0545e00640 1 --2- 172.21.15.45:0/53610318 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f05240a3d20 0x7f05240a61e0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f052c02c180 tx=0x7f052c02e040 comp rx=0 tx=0).stop 2024-04-16T17:18:20.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:20.675+0000 7f0545e00640 1 -- 172.21.15.45:0/53610318 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0540108610 msgr2=0x7f054019ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:18:20.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:20.675+0000 7f0545e00640 1 --2- 172.21.15.45:0/53610318 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0540108610 0x7f054019ee40 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f053000e9a0 tx=0x7f053000eda0 comp rx=0 tx=0).stop 2024-04-16T17:18:20.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:20.676+0000 7f0545e00640 1 -- 172.21.15.45:0/53610318 shutdown_connections 2024-04-16T17:18:20.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:20.676+0000 7f0545e00640 1 --2- 172.21.15.45:0/53610318 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f05240a3d20 0x7f05240a61e0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:20.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:20.676+0000 7f0545e00640 1 --2- 172.21.15.45:0/53610318 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0540108fd0 0x7f054019f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:20.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:20.676+0000 7f0545e00640 1 --2- 172.21.15.45:0/53610318 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f054010e660 0x7f05401a3700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:20.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:20.676+0000 7f0545e00640 1 --2- 172.21.15.45:0/53610318 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0540108610 0x7f054019ee40 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:20.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:20.676+0000 7f0545e00640 1 -- 172.21.15.45:0/53610318 >> 172.21.15.45:0/53610318 conn(0x7f0540076540 msgr2=0x7f0540077c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:18:20.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:20.676+0000 7f0545e00640 1 -- 172.21.15.45:0/53610318 shutdown_connections 2024-04-16T17:18:20.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:20.676+0000 7f0545e00640 1 -- 172.21.15.45:0/53610318 wait complete. 2024-04-16T17:18:20.702 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2024-04-16T17:18:20.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:20.917+0000 7f99ed200640 1 -- 172.21.15.45:0/3525730192 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f99e80743a0 msgr2=0x7f99e8074780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:18:20.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:20.917+0000 7f99ed200640 1 --2- 172.21.15.45:0/3525730192 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f99e80743a0 0x7f99e8074780 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f99d8009900 tx=0x7f99d802bcd0 comp rx=0 tx=0).stop 2024-04-16T17:18:20.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:20.917+0000 7f99ed200640 1 -- 172.21.15.45:0/3525730192 shutdown_connections 2024-04-16T17:18:20.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:20.917+0000 7f99ed200640 1 --2- 172.21.15.45:0/3525730192 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f99e8074cc0 0x7f99e8112870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:20.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:20.917+0000 7f99ed200640 1 --2- 172.21.15.45:0/3525730192 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f99e8112db0 0x7f99e81151a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:20.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:20.917+0000 7f99ed200640 1 --2- 172.21.15.45:0/3525730192 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f99e80743a0 0x7f99e8074780 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:20.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:20.917+0000 7f99ed200640 1 -- 172.21.15.45:0/3525730192 >> 172.21.15.45:0/3525730192 conn(0x7f99e80fd8d0 msgr2=0x7f99e80ffcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:18:20.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:20.917+0000 7f99ed200640 1 -- 172.21.15.45:0/3525730192 shutdown_connections 2024-04-16T17:18:20.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:20.917+0000 7f99ed200640 1 -- 172.21.15.45:0/3525730192 wait complete. 2024-04-16T17:18:20.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:20.918+0000 7f99ed200640 1 Processor -- start 2024-04-16T17:18:20.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:20.918+0000 7f99ed200640 1 -- start start 2024-04-16T17:18:20.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:20.919+0000 7f99ed200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f99e80743a0 0x7f99e81a4950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:20.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:20.919+0000 7f99ed200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f99e8074cc0 0x7f99e81a4e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:20.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:20.919+0000 7f99ed200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f99e8112db0 0x7f99e819eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:20.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:20.919+0000 7f99ed200640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f99e81187a0 con 0x7f99e80743a0 2024-04-16T17:18:20.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:20.919+0000 7f99ed200640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f99e8118620 con 0x7f99e8074cc0 2024-04-16T17:18:20.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:20.919+0000 7f99ed200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f99e8118920 con 0x7f99e8112db0 2024-04-16T17:18:20.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:20.919+0000 7f99e7400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f99e80743a0 0x7f99e81a4950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:20.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:20.919+0000 7f99e7400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f99e80743a0 0x7f99e81a4950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33420/0 (socket says 172.21.15.45:33420) 2024-04-16T17:18:20.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:20.919+0000 7f99e7400640 1 -- 172.21.15.45:0/3027262831 learned_addr learned my addr 172.21.15.45:0/3027262831 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-16T17:18:20.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:20.919+0000 7f99e6a00640 1 --2- 172.21.15.45:0/3027262831 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f99e8074cc0 0x7f99e81a4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:20.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:20.919+0000 7f99e7e00640 1 --2- 172.21.15.45:0/3027262831 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f99e8112db0 0x7f99e819eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:20.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:20.920+0000 7f99e7400640 1 -- 172.21.15.45:0/3027262831 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f99e8112db0 msgr2=0x7f99e819eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:18:20.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:20.920+0000 7f99e7400640 1 --2- 172.21.15.45:0/3027262831 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f99e8112db0 0x7f99e819eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:20.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:20.920+0000 7f99e7400640 1 -- 172.21.15.45:0/3027262831 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f99e8074cc0 msgr2=0x7f99e81a4e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:18:20.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:20.920+0000 7f99e7400640 1 --2- 172.21.15.45:0/3027262831 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f99e8074cc0 0x7f99e81a4e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:20.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:20.920+0000 7f99e7400640 1 -- 172.21.15.45:0/3027262831 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99e819f280 con 0x7f99e80743a0 2024-04-16T17:18:20.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:20.920+0000 7f99e7e00640 1 --2- 172.21.15.45:0/3027262831 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f99e8112db0 0x7f99e819eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:18:20.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:20.920+0000 7f99e6a00640 1 --2- 172.21.15.45:0/3027262831 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f99e8074cc0 0x7f99e81a4e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-16T17:18:20.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:20.920+0000 7f99e7400640 1 --2- 172.21.15.45:0/3027262831 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f99e80743a0 0x7f99e81a4950 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f99d8005940 tx=0x7f99d802cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:18:20.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:20.920+0000 7f99dfe00640 1 -- 172.21.15.45:0/3027262831 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99d8041070 con 0x7f99e80743a0 2024-04-16T17:18:20.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:20.920+0000 7f99dfe00640 1 -- 172.21.15.45:0/3027262831 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f99d8004450 con 0x7f99e80743a0 2024-04-16T17:18:20.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:20.920+0000 7f99ed200640 1 -- 172.21.15.45:0/3027262831 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99e819f510 con 0x7f99e80743a0 2024-04-16T17:18:20.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:20.920+0000 7f99dfe00640 1 -- 172.21.15.45:0/3027262831 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99d8003d80 con 0x7f99e80743a0 2024-04-16T17:18:20.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:20.921+0000 7f99ed200640 1 -- 172.21.15.45:0/3027262831 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99e81ab810 con 0x7f99e80743a0 2024-04-16T17:18:20.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:20.922+0000 7f99ed200640 1 -- 172.21.15.45:0/3027262831 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99e8069190 con 0x7f99e80743a0 2024-04-16T17:18:20.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:20.924+0000 7f99dfe00640 1 -- 172.21.15.45:0/3027262831 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f99d8044070 con 0x7f99e80743a0 2024-04-16T17:18:20.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:20.925+0000 7f99dfe00640 1 --2- 172.21.15.45:0/3027262831 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f99cc0a3d20 0x7f99cc0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-16T17:18:20.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:20.926+0000 7f99dfe00640 1 -- 172.21.15.45:0/3027262831 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 8582+0+0 (secure 0 0 0) 0x7f99d80f2240 con 0x7f99e80743a0 2024-04-16T17:18:20.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:20.926+0000 7f99e6a00640 1 --2- 172.21.15.45:0/3027262831 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f99cc0a3d20 0x7f99cc0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-16T17:18:20.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:20.927+0000 7f99e6a00640 1 --2- 172.21.15.45:0/3027262831 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f99cc0a3d20 0x7f99cc0a61e0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f99e80fffd0 tx=0x7f99d4005f90 comp rx=0 tx=0).ready entity=mgr.25411 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-16T17:18:20.929 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:20.930+0000 7f99dfe00640 1 -- 172.21.15.45:0/3027262831 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f99d80be300 con 0x7f99e80743a0 2024-04-16T17:18:20.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:18:20 smithi045 ceph-mon[31676]: pgmap v364: 129 pgs: 129 active+clean; 51 MiB data, 871 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-04-16T17:18:20.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:18:20 smithi045 ceph-mon[31676]: from='client.? 172.21.15.45:0/3838323001' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-16T17:18:20.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:18:20 smithi045 ceph-mon[31676]: osdmap e110: 12 total, 12 up, 12 in 2024-04-16T17:18:20.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:18:20 smithi045 ceph-mon[31676]: from='client.? 172.21.15.45:0/53610318' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-16T17:18:21.054 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:18:20 smithi117 ceph-mon[36730]: pgmap v364: 129 pgs: 129 active+clean; 51 MiB data, 871 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-04-16T17:18:21.054 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:18:20 smithi117 ceph-mon[36730]: from='client.? 172.21.15.45:0/3838323001' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-16T17:18:21.054 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:18:20 smithi117 ceph-mon[36730]: osdmap e110: 12 total, 12 up, 12 in 2024-04-16T17:18:21.054 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:18:20 smithi117 ceph-mon[36730]: from='client.? 172.21.15.45:0/53610318' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-16T17:18:21.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:21.153+0000 7f99ed200640 1 -- 172.21.15.45:0/3027262831 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) -- 0x7f99e802d620 con 0x7f99e80743a0 2024-04-16T17:18:21.177 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:18:20 smithi104 ceph-mon[36457]: pgmap v364: 129 pgs: 129 active+clean; 51 MiB data, 871 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-04-16T17:18:21.177 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:18:20 smithi104 ceph-mon[36457]: from='client.? 172.21.15.45:0/3838323001' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-16T17:18:21.177 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:18:20 smithi104 ceph-mon[36457]: osdmap e110: 12 total, 12 up, 12 in 2024-04-16T17:18:21.177 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:18:20 smithi104 ceph-mon[36457]: from='client.? 172.21.15.45:0/53610318' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-16T17:18:21.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:21.688+0000 7f99dfe00640 1 -- 172.21.15.45:0/3027262831 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v112) ==== 178+0+0 (secure 0 0 0) 0x7f99d80c2600 con 0x7f99e80743a0 2024-04-16T17:18:21.688 INFO:teuthology.orchestra.run.smithi045.stderr:pool 'cephfs_data_ec' removed 2024-04-16T17:18:21.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:21.693+0000 7f99ed200640 1 -- 172.21.15.45:0/3027262831 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f99cc0a3d20 msgr2=0x7f99cc0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:18:21.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:21.693+0000 7f99ed200640 1 --2- 172.21.15.45:0/3027262831 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f99cc0a3d20 0x7f99cc0a61e0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f99e80fffd0 tx=0x7f99d4005f90 comp rx=0 tx=0).stop 2024-04-16T17:18:21.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:21.693+0000 7f99ed200640 1 -- 172.21.15.45:0/3027262831 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f99e80743a0 msgr2=0x7f99e81a4950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-16T17:18:21.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:21.693+0000 7f99ed200640 1 --2- 172.21.15.45:0/3027262831 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f99e80743a0 0x7f99e81a4950 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f99d8005940 tx=0x7f99d802cbd0 comp rx=0 tx=0).stop 2024-04-16T17:18:21.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:21.693+0000 7f99ed200640 1 -- 172.21.15.45:0/3027262831 shutdown_connections 2024-04-16T17:18:21.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:21.693+0000 7f99ed200640 1 --2- 172.21.15.45:0/3027262831 >> [v2:172.21.15.45:6800/751128195,v1:172.21.15.45:6801/751128195] conn(0x7f99cc0a3d20 0x7f99cc0a61e0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:21.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:21.693+0000 7f99ed200640 1 --2- 172.21.15.45:0/3027262831 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f99e8112db0 0x7f99e819eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:21.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:21.693+0000 7f99ed200640 1 --2- 172.21.15.45:0/3027262831 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f99e8074cc0 0x7f99e81a4e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:21.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:21.693+0000 7f99ed200640 1 --2- 172.21.15.45:0/3027262831 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f99e80743a0 0x7f99e81a4950 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-16T17:18:21.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:21.693+0000 7f99ed200640 1 -- 172.21.15.45:0/3027262831 >> 172.21.15.45:0/3027262831 conn(0x7f99e80fd8d0 msgr2=0x7f99e80ff0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-16T17:18:21.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:21.694+0000 7f99ed200640 1 -- 172.21.15.45:0/3027262831 shutdown_connections 2024-04-16T17:18:21.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-16T17:18:21.694+0000 7f99ed200640 1 -- 172.21.15.45:0/3027262831 wait complete. 2024-04-16T17:18:21.720 DEBUG:teuthology.orchestra.run.smithi045:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-16T17:18:21.803 DEBUG:teuthology.orchestra.run.smithi104:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-16T17:18:21.839 DEBUG:teuthology.orchestra.run.smithi117:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-16T17:18:21.874 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-04-16T17:18:21.874 DEBUG:teuthology.orchestra.run.smithi045:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-16T17:18:21.931 DEBUG:teuthology.orchestra.run.smithi104:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-16T17:18:21.949 DEBUG:teuthology.orchestra.run.smithi117:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-16T17:18:21.966 INFO:tasks.cephadm:Stopping all daemons... 2024-04-16T17:18:21.966 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-04-16T17:18:21.966 DEBUG:teuthology.orchestra.run.smithi045:> sudo systemctl stop ceph-b1fecd38-fc11-11ee-bc90-c7b262605968@mon.a 2024-04-16T17:18:21.976 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:18:21 smithi045 ceph-mon[31676]: from='client.? 172.21.15.45:0/53610318' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-04-16T17:18:21.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:18:21 smithi045 ceph-mon[31676]: osdmap e111: 12 total, 12 up, 12 in 2024-04-16T17:18:21.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:18:21 smithi045 ceph-mon[31676]: from='client.? 172.21.15.45:0/3027262831' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-16T17:18:22.054 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:18:21 smithi117 ceph-mon[36730]: from='client.? 172.21.15.45:0/53610318' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-04-16T17:18:22.054 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:18:21 smithi117 ceph-mon[36730]: osdmap e111: 12 total, 12 up, 12 in 2024-04-16T17:18:22.054 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:18:21 smithi117 ceph-mon[36730]: from='client.? 172.21.15.45:0/3027262831' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-16T17:18:22.177 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:18:21 smithi104 ceph-mon[36457]: from='client.? 172.21.15.45:0/53610318' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-04-16T17:18:22.177 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:18:21 smithi104 ceph-mon[36457]: osdmap e111: 12 total, 12 up, 12 in 2024-04-16T17:18:22.177 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:18:21 smithi104 ceph-mon[36457]: from='client.? 172.21.15.45:0/3027262831' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-16T17:18:22.477 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:18:22 smithi045 systemd[1]: Stopping Ceph mon.a for b1fecd38-fc11-11ee-bc90-c7b262605968... 2024-04-16T17:18:22.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:18:22 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mon-a[31672]: 2024-04-16T17:18:22.599+0000 7f71f1000700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-16T17:18:22.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:18:22 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mon-a[31672]: 2024-04-16T17:18:22.599+0000 7f71f1000700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-04-16T17:18:22.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:18:22 smithi045 podman[103457]: 2024-04-16 17:18:22.774957689 +0000 UTC m=+0.351621565 container died 5e03aa10df484d017679f33bd15f25d228119ead2a392d47d4e2705506ca956f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45, name=ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mon-a, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, org.label-schema.build-date=20240409, org.label-schema.license=GPLv2) 2024-04-16T17:18:23.477 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:18:23 smithi045 podman[103457]: 2024-04-16 17:18:23.266257177 +0000 UTC m=+0.842921089 container cleanup 5e03aa10df484d017679f33bd15f25d228119ead2a392d47d4e2705506ca956f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45, name=ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mon-a, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240409, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-16T17:18:23.477 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:18:23 smithi045 bash[103457]: ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mon-a 2024-04-16T17:18:23.977 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 16 17:18:23 smithi045 podman[103471]: 2024-04-16 17:18:23.517667171 +0000 UTC m=+0.726219135 container remove 5e03aa10df484d017679f33bd15f25d228119ead2a392d47d4e2705506ca956f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45, name=ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mon-a, ceph=True, GIT_CLEAN=True, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, org.label-schema.build-date=20240409, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-16T17:18:24.067 DEBUG:teuthology.orchestra.run.smithi045:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b1fecd38-fc11-11ee-bc90-c7b262605968@mon.a.service' 2024-04-16T17:18:24.277 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-16T17:18:24.278 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-04-16T17:18:24.278 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-04-16T17:18:24.278 DEBUG:teuthology.orchestra.run.smithi104:> sudo systemctl stop ceph-b1fecd38-fc11-11ee-bc90-c7b262605968@mon.b 2024-04-16T17:18:24.594 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:18:24 smithi104 systemd[1]: Stopping Ceph mon.b for b1fecd38-fc11-11ee-bc90-c7b262605968... 2024-04-16T17:18:24.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:18:24 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mon-b[36453]: 2024-04-16T17:18:24.594+0000 7fb90cbec700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-16T17:18:24.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:18:24 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mon-b[36453]: 2024-04-16T17:18:24.594+0000 7fb90cbec700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-04-16T17:18:24.927 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:18:24 smithi104 podman[74196]: 2024-04-16 17:18:24.635995871 +0000 UTC m=+0.162803520 container died 68a342e84dea98df618d136c2aca253c177fb630c8b2f18d159f2106c3f1c4f7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mon-b, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240409, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb) 2024-04-16T17:18:25.296 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:18:25 smithi104 podman[74196]: 2024-04-16 17:18:25.037321753 +0000 UTC m=+0.564129395 container cleanup 68a342e84dea98df618d136c2aca253c177fb630c8b2f18d159f2106c3f1c4f7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mon-b, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240409, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2) 2024-04-16T17:18:25.296 INFO:journalctl@ceph.mon.b.smithi104.stdout:Apr 16 17:18:25 smithi104 bash[74196]: ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mon-b 2024-04-16T17:18:25.455 DEBUG:teuthology.orchestra.run.smithi104:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b1fecd38-fc11-11ee-bc90-c7b262605968@mon.b.service' 2024-04-16T17:18:25.501 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-16T17:18:25.501 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-04-16T17:18:25.502 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-04-16T17:18:25.502 DEBUG:teuthology.orchestra.run.smithi117:> sudo systemctl stop ceph-b1fecd38-fc11-11ee-bc90-c7b262605968@mon.c 2024-04-16T17:18:25.804 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:18:25 smithi117 systemd[1]: Stopping Ceph mon.c for b1fecd38-fc11-11ee-bc90-c7b262605968... 2024-04-16T17:18:26.304 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:18:25 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mon-c[36726]: 2024-04-16T17:18:25.831+0000 7f17df499700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-16T17:18:26.305 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:18:25 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mon-c[36726]: 2024-04-16T17:18:25.831+0000 7f17df499700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-04-16T17:18:26.305 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:18:26 smithi117 podman[74423]: 2024-04-16 17:18:26.046093462 +0000 UTC m=+0.337821819 container died a26181e132ad7b008b206cb47b80c6b217ca6d390204bc4a88c6fd7eb7a28317 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mon-c, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240409) 2024-04-16T17:18:26.699 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:18:26 smithi117 podman[74423]: 2024-04-16 17:18:26.457516993 +0000 UTC m=+0.749245348 container cleanup a26181e132ad7b008b206cb47b80c6b217ca6d390204bc4a88c6fd7eb7a28317 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mon-c, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240409, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb) 2024-04-16T17:18:26.700 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:18:26 smithi117 bash[74423]: ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mon-c 2024-04-16T17:18:26.931 DEBUG:teuthology.orchestra.run.smithi117:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b1fecd38-fc11-11ee-bc90-c7b262605968@mon.c.service' 2024-04-16T17:18:26.955 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:18:26 smithi117 podman[74437]: 2024-04-16 17:18:26.700773702 +0000 UTC m=+0.658907920 container remove a26181e132ad7b008b206cb47b80c6b217ca6d390204bc4a88c6fd7eb7a28317 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mon-c, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240409, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-04-16T17:18:26.955 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:18:26 smithi117 systemd[1]: ceph-b1fecd38-fc11-11ee-bc90-c7b262605968@mon.c.service: Deactivated successfully. 2024-04-16T17:18:26.955 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:18:26 smithi117 systemd[1]: Stopped Ceph mon.c for b1fecd38-fc11-11ee-bc90-c7b262605968. 2024-04-16T17:18:26.956 INFO:journalctl@ceph.mon.c.smithi117.stdout:Apr 16 17:18:26 smithi117 systemd[1]: ceph-b1fecd38-fc11-11ee-bc90-c7b262605968@mon.c.service: Consumed 14.928s CPU time. 2024-04-16T17:18:26.979 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-16T17:18:26.980 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-04-16T17:18:26.980 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-04-16T17:18:26.980 DEBUG:teuthology.orchestra.run.smithi045:> sudo systemctl stop ceph-b1fecd38-fc11-11ee-bc90-c7b262605968@mgr.x 2024-04-16T17:18:27.476 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 16 17:18:27 smithi045 systemd[1]: Stopping Ceph mgr.x for b1fecd38-fc11-11ee-bc90-c7b262605968... 2024-04-16T17:18:27.960 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 16 17:18:27 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-x[31905]: 2024-04-16T17:18:27.562+0000 7f6d52400700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-16T17:18:27.960 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 16 17:18:27 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-x[31905]: 2024-04-16T17:18:27.562+0000 7f6d52400700 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2024-04-16T17:18:27.960 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 16 17:18:27 smithi045 podman[103578]: 2024-04-16 17:18:27.653951525 +0000 UTC m=+0.216561525 container died 9b59d149dd293fab06b5d2104c31d27b9ae666e74f598d5ac2c35f8f89b6fdfe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45, name=ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-x, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240409) 2024-04-16T17:18:28.497 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 16 17:18:28 smithi045 podman[103578]: 2024-04-16 17:18:28.237648284 +0000 UTC m=+0.800258341 container cleanup 9b59d149dd293fab06b5d2104c31d27b9ae666e74f598d5ac2c35f8f89b6fdfe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45, name=ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-x, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240409, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb) 2024-04-16T17:18:28.498 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 16 17:18:28 smithi045 bash[103578]: ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-x 2024-04-16T17:18:28.958 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 16 17:18:28 smithi045 podman[103592]: 2024-04-16 17:18:28.497287435 +0000 UTC m=+0.823662837 container remove 9b59d149dd293fab06b5d2104c31d27b9ae666e74f598d5ac2c35f8f89b6fdfe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b42bfbeda298874861ece590aa565cf64ac8c45, name=ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-x, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240409, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, maintainer=Guillaume Abrioux ) 2024-04-16T17:18:28.989 DEBUG:teuthology.orchestra.run.smithi045:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b1fecd38-fc11-11ee-bc90-c7b262605968@mgr.x.service' 2024-04-16T17:18:29.115 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-16T17:18:29.116 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-04-16T17:18:29.116 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-04-16T17:18:29.116 DEBUG:teuthology.orchestra.run.smithi104:> sudo systemctl stop ceph-b1fecd38-fc11-11ee-bc90-c7b262605968@mgr.y 2024-04-16T17:18:29.426 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Apr 16 17:18:29 smithi104 systemd[1]: Stopping Ceph mgr.y for b1fecd38-fc11-11ee-bc90-c7b262605968... 2024-04-16T17:18:29.701 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Apr 16 17:18:29 smithi104 podman[74318]: 2024-04-16 17:18:29.437535904 +0000 UTC m=+0.130020531 container died ae9050b2d6c5b6535e7df159b86e5de8d321a91a254ac5f4b6821fd18700a66b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-y, GIT_BRANCH=HEAD, org.label-schema.build-date=20240409, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-04-16T17:18:30.102 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Apr 16 17:18:29 smithi104 podman[74318]: 2024-04-16 17:18:29.817641833 +0000 UTC m=+0.510126451 container cleanup ae9050b2d6c5b6535e7df159b86e5de8d321a91a254ac5f4b6821fd18700a66b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-y, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240409, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, ceph=True, org.label-schema.schema-version=1.0) 2024-04-16T17:18:30.102 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Apr 16 17:18:29 smithi104 bash[74318]: ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-y 2024-04-16T17:18:30.244 DEBUG:teuthology.orchestra.run.smithi104:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b1fecd38-fc11-11ee-bc90-c7b262605968@mgr.y.service' 2024-04-16T17:18:30.280 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-16T17:18:30.281 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-04-16T17:18:30.281 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-04-16T17:18:30.281 DEBUG:teuthology.orchestra.run.smithi117:> sudo systemctl stop ceph-b1fecd38-fc11-11ee-bc90-c7b262605968@mgr.z 2024-04-16T17:18:30.606 INFO:journalctl@ceph.mgr.z.smithi117.stdout:Apr 16 17:18:30 smithi117 systemd[1]: Stopping Ceph mgr.z for b1fecd38-fc11-11ee-bc90-c7b262605968... 2024-04-16T17:18:30.929 INFO:journalctl@ceph.mgr.z.smithi117.stdout:Apr 16 17:18:30 smithi117 podman[74543]: 2024-04-16 17:18:30.60743638 +0000 UTC m=+0.137756887 container died 08195d3a5a957145dbd9722387eac83b6d2e2e5aa30ef32e995d2df61803426a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-z, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240409) 2024-04-16T17:18:31.288 INFO:journalctl@ceph.mgr.z.smithi117.stdout:Apr 16 17:18:31 smithi117 podman[74543]: 2024-04-16 17:18:31.037391929 +0000 UTC m=+0.567712417 container cleanup 08195d3a5a957145dbd9722387eac83b6d2e2e5aa30ef32e995d2df61803426a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-z, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240409, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-04-16T17:18:31.288 INFO:journalctl@ceph.mgr.z.smithi117.stdout:Apr 16 17:18:31 smithi117 bash[74543]: ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mgr-z 2024-04-16T17:18:31.493 DEBUG:teuthology.orchestra.run.smithi117:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b1fecd38-fc11-11ee-bc90-c7b262605968@mgr.z.service' 2024-04-16T17:18:31.530 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-16T17:18:31.530 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-04-16T17:18:31.531 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-04-16T17:18:31.531 DEBUG:teuthology.orchestra.run.smithi045:> sudo systemctl stop ceph-b1fecd38-fc11-11ee-bc90-c7b262605968@osd.0 2024-04-16T17:18:31.976 INFO:journalctl@ceph.osd.0.smithi045.stdout:Apr 16 17:18:31 smithi045 systemd[1]: Stopping Ceph osd.0 for b1fecd38-fc11-11ee-bc90-c7b262605968... 2024-04-16T17:18:32.477 INFO:journalctl@ceph.osd.0.smithi045.stdout:Apr 16 17:18:32 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-0[46414]: 2024-04-16T17:18:32.157+0000 7fad23400700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-16T17:18:32.477 INFO:journalctl@ceph.osd.0.smithi045.stdout:Apr 16 17:18:32 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-0[46414]: 2024-04-16T17:18:32.157+0000 7fad23400700 -1 osd.0 112 *** Got signal Terminated *** 2024-04-16T17:18:32.477 INFO:journalctl@ceph.osd.0.smithi045.stdout:Apr 16 17:18:32 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-0[46414]: 2024-04-16T17:18:32.157+0000 7fad23400700 -1 osd.0 112 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-16T17:18:37.977 INFO:journalctl@ceph.osd.0.smithi045.stdout:Apr 16 17:18:37 smithi045 podman[103698]: 2024-04-16 17:18:37.693662826 +0000 UTC m=+5.682749861 container died e52617cc8fdb6fb13851327976edbb338f4514d54557a4f3057bfb798c244eca (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240409, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-04-16T17:18:38.468 INFO:journalctl@ceph.osd.0.smithi045.stdout:Apr 16 17:18:38 smithi045 podman[103698]: 2024-04-16 17:18:38.233417401 +0000 UTC m=+6.222504380 container cleanup e52617cc8fdb6fb13851327976edbb338f4514d54557a4f3057bfb798c244eca (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-0, org.label-schema.license=GPLv2, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240409, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-04-16T17:18:38.468 INFO:journalctl@ceph.osd.0.smithi045.stdout:Apr 16 17:18:38 smithi045 bash[103698]: ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-0 2024-04-16T17:18:38.726 INFO:journalctl@ceph.osd.0.smithi045.stdout:Apr 16 17:18:38 smithi045 podman[103719]: 2024-04-16 17:18:38.4675522 +0000 UTC m=+0.760180943 container remove e52617cc8fdb6fb13851327976edbb338f4514d54557a4f3057bfb798c244eca (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-0, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240409) 2024-04-16T17:18:39.544 INFO:journalctl@ceph.osd.0.smithi045.stdout:Apr 16 17:18:39 smithi045 podman[103809]: 2024-04-16 17:18:39.159875135 +0000 UTC m=+0.034168533 image pull e5ce10bdffbeeffffd1125f67b369e2151f0f17cdec854e93b6adf9321f65907 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2 2024-04-16T17:18:39.545 INFO:journalctl@ceph.osd.0.smithi045.stdout:Apr 16 17:18:39 smithi045 podman[103809]: 2024-04-16 17:18:39.310129725 +0000 UTC m=+0.184423160 container create 5993130bf8be26618a7e8f427aa85950306c9c12613c3b850c79794a057be5cf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-0-deactivate, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240409, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, io.buildah.version=1.33.5) 2024-04-16T17:18:39.821 INFO:journalctl@ceph.osd.0.smithi045.stdout:Apr 16 17:18:39 smithi045 podman[103809]: 2024-04-16 17:18:39.672326036 +0000 UTC m=+0.546619515 container init 5993130bf8be26618a7e8f427aa85950306c9c12613c3b850c79794a057be5cf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-0-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240409, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True) 2024-04-16T17:18:39.822 INFO:journalctl@ceph.osd.0.smithi045.stdout:Apr 16 17:18:39 smithi045 podman[103809]: 2024-04-16 17:18:39.684778625 +0000 UTC m=+0.559072067 container start 5993130bf8be26618a7e8f427aa85950306c9c12613c3b850c79794a057be5cf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-0-deactivate, org.label-schema.build-date=20240409, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, GIT_CLEAN=True) 2024-04-16T17:18:40.153 INFO:journalctl@ceph.osd.0.smithi045.stdout:Apr 16 17:18:39 smithi045 podman[103809]: 2024-04-16 17:18:39.820976489 +0000 UTC m=+0.695269964 container attach 5993130bf8be26618a7e8f427aa85950306c9c12613c3b850c79794a057be5cf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-0-deactivate, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240409, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb) 2024-04-16T17:18:40.153 INFO:journalctl@ceph.osd.0.smithi045.stdout:Apr 16 17:18:40 smithi045 podman[103809]: 2024-04-16 17:18:40.15321601 +0000 UTC m=+1.027509502 container died 5993130bf8be26618a7e8f427aa85950306c9c12613c3b850c79794a057be5cf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-0-deactivate, org.label-schema.build-date=20240409, CEPH_POINT_RELEASE=, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-04-16T17:18:40.915 DEBUG:teuthology.orchestra.run.smithi045:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b1fecd38-fc11-11ee-bc90-c7b262605968@osd.0.service' 2024-04-16T17:18:41.050 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-16T17:18:41.050 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-04-16T17:18:41.050 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-04-16T17:18:41.050 DEBUG:teuthology.orchestra.run.smithi104:> sudo systemctl stop ceph-b1fecd38-fc11-11ee-bc90-c7b262605968@osd.1 2024-04-16T17:18:41.362 INFO:journalctl@ceph.osd.1.smithi104.stdout:Apr 16 17:18:41 smithi104 systemd[1]: Stopping Ceph osd.1 for b1fecd38-fc11-11ee-bc90-c7b262605968... 2024-04-16T17:18:41.676 INFO:journalctl@ceph.osd.1.smithi104.stdout:Apr 16 17:18:41 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-1[41050]: 2024-04-16T17:18:41.362+0000 7f42cbe15700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-16T17:18:41.677 INFO:journalctl@ceph.osd.1.smithi104.stdout:Apr 16 17:18:41 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-1[41050]: 2024-04-16T17:18:41.362+0000 7f42cbe15700 -1 osd.1 112 *** Got signal Terminated *** 2024-04-16T17:18:41.677 INFO:journalctl@ceph.osd.1.smithi104.stdout:Apr 16 17:18:41 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-1[41050]: 2024-04-16T17:18:41.362+0000 7f42cbe15700 -1 osd.1 112 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-16T17:18:47.177 INFO:journalctl@ceph.osd.1.smithi104.stdout:Apr 16 17:18:46 smithi104 podman[74443]: 2024-04-16 17:18:46.866600813 +0000 UTC m=+5.626836786 container died 04c09ef53b429d5cef3eafc4d05f88ce89f578a267ef5679dd17b5246cc3e3b3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-1, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240409, GIT_CLEAN=True, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-04-16T17:18:47.677 INFO:journalctl@ceph.osd.1.smithi104.stdout:Apr 16 17:18:47 smithi104 podman[74443]: 2024-04-16 17:18:47.205583501 +0000 UTC m=+5.965819464 container cleanup 04c09ef53b429d5cef3eafc4d05f88ce89f578a267ef5679dd17b5246cc3e3b3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-1, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240409, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb) 2024-04-16T17:18:47.677 INFO:journalctl@ceph.osd.1.smithi104.stdout:Apr 16 17:18:47 smithi104 bash[74443]: ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-1 2024-04-16T17:18:47.678 INFO:journalctl@ceph.osd.1.smithi104.stdout:Apr 16 17:18:47 smithi104 podman[74460]: 2024-04-16 17:18:47.430919544 +0000 UTC m=+0.564551758 container remove 04c09ef53b429d5cef3eafc4d05f88ce89f578a267ef5679dd17b5246cc3e3b3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-1, org.label-schema.vendor=CentOS, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, GIT_CLEAN=True, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240409) 2024-04-16T17:18:48.111 INFO:journalctl@ceph.osd.1.smithi104.stdout:Apr 16 17:18:47 smithi104 podman[74549]: 2024-04-16 17:18:47.765256112 +0000 UTC m=+0.017794501 image pull e5ce10bdffbeeffffd1125f67b369e2151f0f17cdec854e93b6adf9321f65907 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2 2024-04-16T17:18:48.112 INFO:journalctl@ceph.osd.1.smithi104.stdout:Apr 16 17:18:47 smithi104 podman[74549]: 2024-04-16 17:18:47.881799779 +0000 UTC m=+0.134338160 container create 285148f3343bb7cbe15aa83a77a027f2850ac85cfe2181f3d8e330016a4b4449 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-1-deactivate, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240409, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-04-16T17:18:48.112 INFO:journalctl@ceph.osd.1.smithi104.stdout:Apr 16 17:18:48 smithi104 podman[74549]: 2024-04-16 17:18:48.109593377 +0000 UTC m=+0.362131767 container init 285148f3343bb7cbe15aa83a77a027f2850ac85cfe2181f3d8e330016a4b4449 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-1-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.build-date=20240409, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-04-16T17:18:48.112 INFO:journalctl@ceph.osd.1.smithi104.stdout:Apr 16 17:18:48 smithi104 podman[74549]: 2024-04-16 17:18:48.112815401 +0000 UTC m=+0.365353792 container start 285148f3343bb7cbe15aa83a77a027f2850ac85cfe2181f3d8e330016a4b4449 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-1-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240409, maintainer=Guillaume Abrioux , RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb) 2024-04-16T17:18:48.398 INFO:journalctl@ceph.osd.1.smithi104.stdout:Apr 16 17:18:48 smithi104 podman[74549]: 2024-04-16 17:18:48.192769422 +0000 UTC m=+0.445307835 container attach 285148f3343bb7cbe15aa83a77a027f2850ac85cfe2181f3d8e330016a4b4449 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-1-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240409, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-04-16T17:18:48.677 INFO:journalctl@ceph.osd.1.smithi104.stdout:Apr 16 17:18:48 smithi104 podman[74549]: 2024-04-16 17:18:48.399981911 +0000 UTC m=+0.652520303 container died 285148f3343bb7cbe15aa83a77a027f2850ac85cfe2181f3d8e330016a4b4449 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-1-deactivate, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240409) 2024-04-16T17:18:49.029 DEBUG:teuthology.orchestra.run.smithi104:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b1fecd38-fc11-11ee-bc90-c7b262605968@osd.1.service' 2024-04-16T17:18:49.071 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-16T17:18:49.071 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-04-16T17:18:49.072 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-04-16T17:18:49.072 DEBUG:teuthology.orchestra.run.smithi117:> sudo systemctl stop ceph-b1fecd38-fc11-11ee-bc90-c7b262605968@osd.2 2024-04-16T17:18:49.406 INFO:journalctl@ceph.osd.2.smithi117.stdout:Apr 16 17:18:49 smithi117 systemd[1]: Stopping Ceph osd.2 for b1fecd38-fc11-11ee-bc90-c7b262605968... 2024-04-16T17:18:49.804 INFO:journalctl@ceph.osd.2.smithi117.stdout:Apr 16 17:18:49 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-2[41881]: 2024-04-16T17:18:49.406+0000 7fb2a6b20700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-16T17:18:49.804 INFO:journalctl@ceph.osd.2.smithi117.stdout:Apr 16 17:18:49 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-2[41881]: 2024-04-16T17:18:49.406+0000 7fb2a6b20700 -1 osd.2 112 *** Got signal Terminated *** 2024-04-16T17:18:49.804 INFO:journalctl@ceph.osd.2.smithi117.stdout:Apr 16 17:18:49 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-2[41881]: 2024-04-16T17:18:49.406+0000 7fb2a6b20700 -1 osd.2 112 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-16T17:18:55.231 INFO:journalctl@ceph.osd.2.smithi117.stdout:Apr 16 17:18:54 smithi117 podman[74671]: 2024-04-16 17:18:54.927591092 +0000 UTC m=+5.644301144 container died 36470e2c52461e7f20b8969c5f66acdeb6eae79d20a9c4c927a29302876574df (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240409, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb) 2024-04-16T17:18:55.554 INFO:journalctl@ceph.osd.2.smithi117.stdout:Apr 16 17:18:55 smithi117 podman[74671]: 2024-04-16 17:18:55.332054246 +0000 UTC m=+6.048764294 container cleanup 36470e2c52461e7f20b8969c5f66acdeb6eae79d20a9c4c927a29302876574df (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240409, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-16T17:18:55.554 INFO:journalctl@ceph.osd.2.smithi117.stdout:Apr 16 17:18:55 smithi117 bash[74671]: ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-2 2024-04-16T17:18:55.995 INFO:journalctl@ceph.osd.2.smithi117.stdout:Apr 16 17:18:55 smithi117 podman[74699]: 2024-04-16 17:18:55.575341842 +0000 UTC m=+0.648463716 container remove 36470e2c52461e7f20b8969c5f66acdeb6eae79d20a9c4c927a29302876574df (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240409, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-04-16T17:18:56.261 INFO:journalctl@ceph.osd.2.smithi117.stdout:Apr 16 17:18:55 smithi117 podman[74790]: 2024-04-16 17:18:55.899458571 +0000 UTC m=+0.017979886 image pull e5ce10bdffbeeffffd1125f67b369e2151f0f17cdec854e93b6adf9321f65907 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2 2024-04-16T17:18:56.261 INFO:journalctl@ceph.osd.2.smithi117.stdout:Apr 16 17:18:56 smithi117 podman[74790]: 2024-04-16 17:18:56.026407516 +0000 UTC m=+0.144928816 container create 76aa110a23a62888a0225ae6128dbd2b06d2a7f29259f33b55a0745b25c49c34 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-2-deactivate, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240409) 2024-04-16T17:18:56.554 INFO:journalctl@ceph.osd.2.smithi117.stdout:Apr 16 17:18:56 smithi117 podman[74790]: 2024-04-16 17:18:56.262459985 +0000 UTC m=+0.380981285 container init 76aa110a23a62888a0225ae6128dbd2b06d2a7f29259f33b55a0745b25c49c34 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-2-deactivate, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240409, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-04-16T17:18:56.554 INFO:journalctl@ceph.osd.2.smithi117.stdout:Apr 16 17:18:56 smithi117 podman[74790]: 2024-04-16 17:18:56.265764418 +0000 UTC m=+0.384285711 container start 76aa110a23a62888a0225ae6128dbd2b06d2a7f29259f33b55a0745b25c49c34 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-2-deactivate, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240409, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-04-16T17:18:56.555 INFO:journalctl@ceph.osd.2.smithi117.stdout:Apr 16 17:18:56 smithi117 podman[74790]: 2024-04-16 17:18:56.3541447 +0000 UTC m=+0.472666017 container attach 76aa110a23a62888a0225ae6128dbd2b06d2a7f29259f33b55a0745b25c49c34 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-2-deactivate, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240409, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-04-16T17:18:56.964 INFO:journalctl@ceph.osd.2.smithi117.stdout:Apr 16 17:18:56 smithi117 podman[74790]: 2024-04-16 17:18:56.55572348 +0000 UTC m=+0.674244787 container died 76aa110a23a62888a0225ae6128dbd2b06d2a7f29259f33b55a0745b25c49c34 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-2-deactivate, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240409) 2024-04-16T17:18:57.328 DEBUG:teuthology.orchestra.run.smithi117:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b1fecd38-fc11-11ee-bc90-c7b262605968@osd.2.service' 2024-04-16T17:18:57.371 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-16T17:18:57.371 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-04-16T17:18:57.372 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-04-16T17:18:57.372 DEBUG:teuthology.orchestra.run.smithi045:> sudo systemctl stop ceph-b1fecd38-fc11-11ee-bc90-c7b262605968@osd.3 2024-04-16T17:18:57.727 INFO:journalctl@ceph.osd.3.smithi045.stdout:Apr 16 17:18:57 smithi045 systemd[1]: Stopping Ceph osd.3 for b1fecd38-fc11-11ee-bc90-c7b262605968... 2024-04-16T17:18:58.227 INFO:journalctl@ceph.osd.3.smithi045.stdout:Apr 16 17:18:57 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-3[53367]: 2024-04-16T17:18:57.925+0000 7fc3d2400700 -1 osd.3 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-16T17:18:33.208725+0000 front 2024-04-16T17:18:33.208664+0000 (oldest deadline 2024-04-16T17:18:57.908576+0000) 2024-04-16T17:18:58.228 INFO:journalctl@ceph.osd.3.smithi045.stdout:Apr 16 17:18:58 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-3[53367]: 2024-04-16T17:18:58.033+0000 7fc3dbc00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-16T17:18:58.228 INFO:journalctl@ceph.osd.3.smithi045.stdout:Apr 16 17:18:58 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-3[53367]: 2024-04-16T17:18:58.033+0000 7fc3dbc00700 -1 osd.3 112 *** Got signal Terminated *** 2024-04-16T17:18:58.228 INFO:journalctl@ceph.osd.3.smithi045.stdout:Apr 16 17:18:58 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-3[53367]: 2024-04-16T17:18:58.033+0000 7fc3dbc00700 -1 osd.3 112 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-16T17:18:58.805 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:18:58 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:18:58.544+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-16T17:18:34.374061+0000 front 2024-04-16T17:18:34.374275+0000 (oldest deadline 2024-04-16T17:18:57.873879+0000) 2024-04-16T17:18:59.227 INFO:journalctl@ceph.osd.3.smithi045.stdout:Apr 16 17:18:58 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-3[53367]: 2024-04-16T17:18:58.957+0000 7fc3d2400700 -1 osd.3 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-16T17:18:33.208725+0000 front 2024-04-16T17:18:33.208664+0000 (oldest deadline 2024-04-16T17:18:57.908576+0000) 2024-04-16T17:18:59.839 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:18:59 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:18:59.578+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-16T17:18:34.374061+0000 front 2024-04-16T17:18:34.374275+0000 (oldest deadline 2024-04-16T17:18:57.873879+0000) 2024-04-16T17:19:00.227 INFO:journalctl@ceph.osd.3.smithi045.stdout:Apr 16 17:18:59 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-3[53367]: 2024-04-16T17:18:59.922+0000 7fc3d2400700 -1 osd.3 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-16T17:18:33.208725+0000 front 2024-04-16T17:18:33.208664+0000 (oldest deadline 2024-04-16T17:18:57.908576+0000) 2024-04-16T17:19:00.804 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:19:00 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:19:00.543+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-16T17:18:34.374061+0000 front 2024-04-16T17:18:34.374275+0000 (oldest deadline 2024-04-16T17:18:57.873879+0000) 2024-04-16T17:19:01.227 INFO:journalctl@ceph.osd.3.smithi045.stdout:Apr 16 17:19:00 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-3[53367]: 2024-04-16T17:19:00.932+0000 7fc3d2400700 -1 osd.3 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-16T17:18:33.208725+0000 front 2024-04-16T17:18:33.208664+0000 (oldest deadline 2024-04-16T17:18:57.908576+0000) 2024-04-16T17:19:01.677 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:19:01 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:19:01.178+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-16T17:18:36.158791+0000 front 2024-04-16T17:18:36.158992+0000 (oldest deadline 2024-04-16T17:19:00.258603+0000) 2024-04-16T17:19:01.804 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:19:01 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:19:01.547+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-16T17:18:34.374061+0000 front 2024-04-16T17:18:34.374275+0000 (oldest deadline 2024-04-16T17:18:57.873879+0000) 2024-04-16T17:19:02.229 INFO:journalctl@ceph.osd.3.smithi045.stdout:Apr 16 17:19:01 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-3[53367]: 2024-04-16T17:19:01.926+0000 7fc3d2400700 -1 osd.3 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-16T17:18:33.208725+0000 front 2024-04-16T17:18:33.208664+0000 (oldest deadline 2024-04-16T17:18:57.908576+0000) 2024-04-16T17:19:02.676 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:19:02 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:19:02.180+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-16T17:18:36.158791+0000 front 2024-04-16T17:18:36.158992+0000 (oldest deadline 2024-04-16T17:19:00.258603+0000) 2024-04-16T17:19:02.727 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 16 17:19:02 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-9[66545]: 2024-04-16T17:19:02.384+0000 7f5937600700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-16T17:18:36.968505+0000 front 2024-04-16T17:18:36.968670+0000 (oldest deadline 2024-04-16T17:19:02.268762+0000) 2024-04-16T17:19:02.804 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:19:02 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:19:02.540+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-16T17:18:34.374061+0000 front 2024-04-16T17:18:34.374275+0000 (oldest deadline 2024-04-16T17:18:57.873879+0000) 2024-04-16T17:19:03.227 INFO:journalctl@ceph.osd.3.smithi045.stdout:Apr 16 17:19:02 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-3[53367]: 2024-04-16T17:19:02.922+0000 7fc3d2400700 -1 osd.3 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-16T17:18:33.208725+0000 front 2024-04-16T17:18:33.208664+0000 (oldest deadline 2024-04-16T17:18:57.908576+0000) 2024-04-16T17:19:03.304 INFO:journalctl@ceph.osd.5.smithi117.stdout:Apr 16 17:19:02 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-5[48830]: 2024-04-16T17:19:02.939+0000 7fe9dcc0c700 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-16T17:18:35.703278+0000 front 2024-04-16T17:18:35.702939+0000 (oldest deadline 2024-04-16T17:19:02.703378+0000) 2024-04-16T17:19:03.426 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:19:03 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:19:03.140+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-16T17:18:36.158791+0000 front 2024-04-16T17:18:36.158992+0000 (oldest deadline 2024-04-16T17:19:00.258603+0000) 2024-04-16T17:19:03.658 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 16 17:19:03 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-9[66545]: 2024-04-16T17:19:03.335+0000 7f5937600700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-16T17:18:36.968505+0000 front 2024-04-16T17:18:36.968670+0000 (oldest deadline 2024-04-16T17:19:02.268762+0000) 2024-04-16T17:19:03.909 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:19:03 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:19:03.556+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-16T17:18:34.374061+0000 front 2024-04-16T17:18:34.374275+0000 (oldest deadline 2024-04-16T17:18:57.873879+0000) 2024-04-16T17:19:03.977 INFO:journalctl@ceph.osd.3.smithi045.stdout:Apr 16 17:19:03 smithi045 podman[103913]: 2024-04-16 17:19:03.658048069 +0000 UTC m=+5.783544053 container died cf9586aa61786aacf1e18e3a00b89f9c501424a3560fbf5b97c5d51ec857792d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-3, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240409) 2024-04-16T17:19:04.304 INFO:journalctl@ceph.osd.5.smithi117.stdout:Apr 16 17:19:03 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-5[48830]: 2024-04-16T17:19:03.909+0000 7fe9dcc0c700 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-16T17:18:35.703278+0000 front 2024-04-16T17:18:35.702939+0000 (oldest deadline 2024-04-16T17:19:02.703378+0000) 2024-04-16T17:19:04.328 INFO:journalctl@ceph.osd.3.smithi045.stdout:Apr 16 17:19:04 smithi045 podman[103913]: 2024-04-16 17:19:04.176112179 +0000 UTC m=+6.301608203 container cleanup cf9586aa61786aacf1e18e3a00b89f9c501424a3560fbf5b97c5d51ec857792d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-3, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240409, org.label-schema.schema-version=1.0) 2024-04-16T17:19:04.328 INFO:journalctl@ceph.osd.3.smithi045.stdout:Apr 16 17:19:04 smithi045 bash[103913]: ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-3 2024-04-16T17:19:04.426 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:19:04 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:19:04.175+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-16T17:18:36.158791+0000 front 2024-04-16T17:18:36.158992+0000 (oldest deadline 2024-04-16T17:19:00.258603+0000) 2024-04-16T17:19:04.726 INFO:journalctl@ceph.osd.3.smithi045.stdout:Apr 16 17:19:04 smithi045 podman[103934]: 2024-04-16 17:19:04.410792908 +0000 UTC m=+0.736937231 container remove cf9586aa61786aacf1e18e3a00b89f9c501424a3560fbf5b97c5d51ec857792d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-3, org.label-schema.license=GPLv2, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240409, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-16T17:19:04.727 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 16 17:19:04 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-9[66545]: 2024-04-16T17:19:04.326+0000 7f5937600700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-16T17:18:36.968505+0000 front 2024-04-16T17:18:36.968670+0000 (oldest deadline 2024-04-16T17:19:02.268762+0000) 2024-04-16T17:19:04.804 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:19:04 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:19:04.535+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-16T17:18:34.374061+0000 front 2024-04-16T17:18:34.374275+0000 (oldest deadline 2024-04-16T17:18:57.873879+0000) 2024-04-16T17:19:05.304 INFO:journalctl@ceph.osd.5.smithi117.stdout:Apr 16 17:19:04 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-5[48830]: 2024-04-16T17:19:04.910+0000 7fe9dcc0c700 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-16T17:18:35.703278+0000 front 2024-04-16T17:18:35.702939+0000 (oldest deadline 2024-04-16T17:19:02.703378+0000) 2024-04-16T17:19:05.426 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:19:05 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:19:05.171+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-16T17:18:36.158791+0000 front 2024-04-16T17:18:36.158992+0000 (oldest deadline 2024-04-16T17:19:00.258603+0000) 2024-04-16T17:19:05.565 INFO:journalctl@ceph.osd.3.smithi045.stdout:Apr 16 17:19:05 smithi045 podman[104023]: 2024-04-16 17:19:05.156568275 +0000 UTC m=+0.034763299 image pull e5ce10bdffbeeffffd1125f67b369e2151f0f17cdec854e93b6adf9321f65907 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2 2024-04-16T17:19:05.565 INFO:journalctl@ceph.osd.3.smithi045.stdout:Apr 16 17:19:05 smithi045 podman[104023]: 2024-04-16 17:19:05.261481189 +0000 UTC m=+0.139676250 container create 4be7267e8a099e8bada014b6d792ab2b2e9cd697f84ea17adc3cc0336c90f843 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-3-deactivate, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240409) 2024-04-16T17:19:05.565 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 16 17:19:05 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-9[66545]: 2024-04-16T17:19:05.345+0000 7f5937600700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-16T17:18:36.968505+0000 front 2024-04-16T17:18:36.968670+0000 (oldest deadline 2024-04-16T17:19:02.268762+0000) 2024-04-16T17:19:05.804 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:19:05 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:19:05.548+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-16T17:18:34.374061+0000 front 2024-04-16T17:18:34.374275+0000 (oldest deadline 2024-04-16T17:18:57.873879+0000) 2024-04-16T17:19:05.839 INFO:journalctl@ceph.osd.3.smithi045.stdout:Apr 16 17:19:05 smithi045 podman[104023]: 2024-04-16 17:19:05.565424517 +0000 UTC m=+0.443619585 container init 4be7267e8a099e8bada014b6d792ab2b2e9cd697f84ea17adc3cc0336c90f843 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-3-deactivate, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240409, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-04-16T17:19:05.839 INFO:journalctl@ceph.osd.3.smithi045.stdout:Apr 16 17:19:05 smithi045 podman[104023]: 2024-04-16 17:19:05.578134245 +0000 UTC m=+0.456329310 container start 4be7267e8a099e8bada014b6d792ab2b2e9cd697f84ea17adc3cc0336c90f843 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-3-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240409, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-04-16T17:19:05.839 INFO:journalctl@ceph.osd.3.smithi045.stdout:Apr 16 17:19:05 smithi045 podman[104023]: 2024-04-16 17:19:05.839054757 +0000 UTC m=+0.717249848 container attach 4be7267e8a099e8bada014b6d792ab2b2e9cd697f84ea17adc3cc0336c90f843 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-3-deactivate, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.build-date=20240409, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb) 2024-04-16T17:19:06.227 INFO:journalctl@ceph.osd.3.smithi045.stdout:Apr 16 17:19:06 smithi045 podman[104023]: 2024-04-16 17:19:06.052942167 +0000 UTC m=+0.931137250 container died 4be7267e8a099e8bada014b6d792ab2b2e9cd697f84ea17adc3cc0336c90f843 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-3-deactivate, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240409, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-04-16T17:19:06.304 INFO:journalctl@ceph.osd.5.smithi117.stdout:Apr 16 17:19:05 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-5[48830]: 2024-04-16T17:19:05.960+0000 7fe9dcc0c700 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-16T17:18:35.703278+0000 front 2024-04-16T17:18:35.702939+0000 (oldest deadline 2024-04-16T17:19:02.703378+0000) 2024-04-16T17:19:06.427 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:19:06 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:19:06.122+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-16T17:18:36.158791+0000 front 2024-04-16T17:18:36.158992+0000 (oldest deadline 2024-04-16T17:19:00.258603+0000) 2024-04-16T17:19:06.612 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 16 17:19:06 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-9[66545]: 2024-04-16T17:19:06.338+0000 7f5937600700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-16T17:18:36.968505+0000 front 2024-04-16T17:18:36.968670+0000 (oldest deadline 2024-04-16T17:19:02.268762+0000) 2024-04-16T17:19:06.978 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:19:06 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:19:06.562+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-16T17:18:34.374061+0000 front 2024-04-16T17:18:34.374275+0000 (oldest deadline 2024-04-16T17:18:57.873879+0000) 2024-04-16T17:19:07.082 DEBUG:teuthology.orchestra.run.smithi045:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b1fecd38-fc11-11ee-bc90-c7b262605968@osd.3.service' 2024-04-16T17:19:07.206 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-16T17:19:07.207 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-04-16T17:19:07.207 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-04-16T17:19:07.207 DEBUG:teuthology.orchestra.run.smithi104:> sudo systemctl stop ceph-b1fecd38-fc11-11ee-bc90-c7b262605968@osd.4 2024-04-16T17:19:07.304 INFO:journalctl@ceph.osd.5.smithi117.stdout:Apr 16 17:19:06 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-5[48830]: 2024-04-16T17:19:06.978+0000 7fe9dcc0c700 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-16T17:18:35.703278+0000 front 2024-04-16T17:18:35.702939+0000 (oldest deadline 2024-04-16T17:19:02.703378+0000) 2024-04-16T17:19:07.426 INFO:journalctl@ceph.osd.4.smithi104.stdout:Apr 16 17:19:07 smithi104 systemd[1]: Stopping Ceph osd.4 for b1fecd38-fc11-11ee-bc90-c7b262605968... 2024-04-16T17:19:07.427 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:19:07 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:19:07.150+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-16T17:18:36.158791+0000 front 2024-04-16T17:18:36.158992+0000 (oldest deadline 2024-04-16T17:19:00.258603+0000) 2024-04-16T17:19:07.726 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 16 17:19:07 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-9[66545]: 2024-04-16T17:19:07.337+0000 7f5937600700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-16T17:18:36.968505+0000 front 2024-04-16T17:18:36.968670+0000 (oldest deadline 2024-04-16T17:19:02.268762+0000) 2024-04-16T17:19:07.804 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:19:07 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:19:07.513+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-16T17:18:34.374061+0000 front 2024-04-16T17:18:34.374275+0000 (oldest deadline 2024-04-16T17:18:57.873879+0000) 2024-04-16T17:19:07.927 INFO:journalctl@ceph.osd.4.smithi104.stdout:Apr 16 17:19:07 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-4[48045]: 2024-04-16T17:19:07.513+0000 7fa47d9ec700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-16T17:19:07.927 INFO:journalctl@ceph.osd.4.smithi104.stdout:Apr 16 17:19:07 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-4[48045]: 2024-04-16T17:19:07.513+0000 7fa47d9ec700 -1 osd.4 112 *** Got signal Terminated *** 2024-04-16T17:19:07.927 INFO:journalctl@ceph.osd.4.smithi104.stdout:Apr 16 17:19:07 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-4[48045]: 2024-04-16T17:19:07.513+0000 7fa47d9ec700 -1 osd.4 112 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-16T17:19:08.304 INFO:journalctl@ceph.osd.5.smithi117.stdout:Apr 16 17:19:07 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-5[48830]: 2024-04-16T17:19:07.990+0000 7fe9dcc0c700 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-16T17:18:35.703278+0000 front 2024-04-16T17:18:35.702939+0000 (oldest deadline 2024-04-16T17:19:02.703378+0000) 2024-04-16T17:19:08.305 INFO:journalctl@ceph.osd.8.smithi117.stdout:Apr 16 17:19:07 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-8[55817]: 2024-04-16T17:19:07.855+0000 7f65ab398700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-16T17:18:41.860244+0000 front 2024-04-16T17:18:41.860176+0000 (oldest deadline 2024-04-16T17:19:07.760349+0000) 2024-04-16T17:19:08.427 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:19:08 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:19:08.138+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-16T17:18:36.158791+0000 front 2024-04-16T17:18:36.158992+0000 (oldest deadline 2024-04-16T17:19:00.258603+0000) 2024-04-16T17:19:08.427 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:19:08 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:19:08.138+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-16T17:18:44.260046+0000 front 2024-04-16T17:18:44.260171+0000 (oldest deadline 2024-04-16T17:19:07.160168+0000) 2024-04-16T17:19:08.727 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 16 17:19:08 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-9[66545]: 2024-04-16T17:19:08.350+0000 7f5937600700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-16T17:18:36.968505+0000 front 2024-04-16T17:18:36.968670+0000 (oldest deadline 2024-04-16T17:19:02.268762+0000) 2024-04-16T17:19:08.804 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:19:08 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:19:08.487+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-16T17:18:34.374061+0000 front 2024-04-16T17:18:34.374275+0000 (oldest deadline 2024-04-16T17:18:57.873879+0000) 2024-04-16T17:19:08.926 INFO:journalctl@ceph.osd.7.smithi104.stdout:Apr 16 17:19:08 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-7[55033]: 2024-04-16T17:19:08.609+0000 7f08b34d5700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-16T17:18:42.259445+0000 front 2024-04-16T17:18:42.259491+0000 (oldest deadline 2024-04-16T17:19:08.159569+0000) 2024-04-16T17:19:09.304 INFO:journalctl@ceph.osd.8.smithi117.stdout:Apr 16 17:19:08 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-8[55817]: 2024-04-16T17:19:08.871+0000 7f65ab398700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-16T17:18:41.860244+0000 front 2024-04-16T17:18:41.860176+0000 (oldest deadline 2024-04-16T17:19:07.760349+0000) 2024-04-16T17:19:09.305 INFO:journalctl@ceph.osd.5.smithi117.stdout:Apr 16 17:19:08 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-5[48830]: 2024-04-16T17:19:08.974+0000 7fe9dcc0c700 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-16T17:18:35.703278+0000 front 2024-04-16T17:18:35.702939+0000 (oldest deadline 2024-04-16T17:19:02.703378+0000) 2024-04-16T17:19:09.305 INFO:journalctl@ceph.osd.5.smithi117.stdout:Apr 16 17:19:08 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-5[48830]: 2024-04-16T17:19:08.974+0000 7fe9dcc0c700 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-16T17:18:45.604466+0000 front 2024-04-16T17:18:45.604549+0000 (oldest deadline 2024-04-16T17:19:08.504560+0000) 2024-04-16T17:19:09.427 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:19:09 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:19:09.105+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-16T17:18:36.158791+0000 front 2024-04-16T17:18:36.158992+0000 (oldest deadline 2024-04-16T17:19:00.258603+0000) 2024-04-16T17:19:09.427 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:19:09 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:19:09.105+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-16T17:18:44.260046+0000 front 2024-04-16T17:18:44.260171+0000 (oldest deadline 2024-04-16T17:19:07.160168+0000) 2024-04-16T17:19:09.727 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 16 17:19:09 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-9[66545]: 2024-04-16T17:19:09.347+0000 7f5937600700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-16T17:18:36.968505+0000 front 2024-04-16T17:18:36.968670+0000 (oldest deadline 2024-04-16T17:19:02.268762+0000) 2024-04-16T17:19:09.727 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 16 17:19:09 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-9[66545]: 2024-04-16T17:19:09.347+0000 7f5937600700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-16T17:18:43.371969+0000 front 2024-04-16T17:18:43.371904+0000 (oldest deadline 2024-04-16T17:19:08.671642+0000) 2024-04-16T17:19:09.804 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:19:09 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:19:09.438+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-16T17:18:34.374061+0000 front 2024-04-16T17:18:34.374275+0000 (oldest deadline 2024-04-16T17:18:57.873879+0000) 2024-04-16T17:19:09.927 INFO:journalctl@ceph.osd.7.smithi104.stdout:Apr 16 17:19:09 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-7[55033]: 2024-04-16T17:19:09.604+0000 7f08b34d5700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-16T17:18:42.259445+0000 front 2024-04-16T17:18:42.259491+0000 (oldest deadline 2024-04-16T17:19:08.159569+0000) 2024-04-16T17:19:10.304 INFO:journalctl@ceph.osd.8.smithi117.stdout:Apr 16 17:19:09 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-8[55817]: 2024-04-16T17:19:09.885+0000 7f65ab398700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-16T17:18:41.860244+0000 front 2024-04-16T17:18:41.860176+0000 (oldest deadline 2024-04-16T17:19:07.760349+0000) 2024-04-16T17:19:10.304 INFO:journalctl@ceph.osd.5.smithi117.stdout:Apr 16 17:19:10 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-5[48830]: 2024-04-16T17:19:10.022+0000 7fe9dcc0c700 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-16T17:18:35.703278+0000 front 2024-04-16T17:18:35.702939+0000 (oldest deadline 2024-04-16T17:19:02.703378+0000) 2024-04-16T17:19:10.305 INFO:journalctl@ceph.osd.5.smithi117.stdout:Apr 16 17:19:10 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-5[48830]: 2024-04-16T17:19:10.022+0000 7fe9dcc0c700 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-16T17:18:45.604466+0000 front 2024-04-16T17:18:45.604549+0000 (oldest deadline 2024-04-16T17:19:08.504560+0000) 2024-04-16T17:19:10.427 INFO:journalctl@ceph.osd.4.smithi104.stdout:Apr 16 17:19:10 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-4[48045]: 2024-04-16T17:19:10.096+0000 7fa479fe0700 -1 osd.4 112 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-16T17:18:40.645865+0000 front 2024-04-16T17:18:40.646010+0000 (oldest deadline 2024-04-16T17:19:10.046543+0000) 2024-04-16T17:19:10.427 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:19:10 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:19:10.116+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-16T17:18:36.158791+0000 front 2024-04-16T17:18:36.158992+0000 (oldest deadline 2024-04-16T17:19:00.258603+0000) 2024-04-16T17:19:10.427 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:19:10 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:19:10.116+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-16T17:18:44.260046+0000 front 2024-04-16T17:18:44.260171+0000 (oldest deadline 2024-04-16T17:19:07.160168+0000) 2024-04-16T17:19:10.727 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 16 17:19:10 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-9[66545]: 2024-04-16T17:19:10.361+0000 7f5937600700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-16T17:18:36.968505+0000 front 2024-04-16T17:18:36.968670+0000 (oldest deadline 2024-04-16T17:19:02.268762+0000) 2024-04-16T17:19:10.727 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 16 17:19:10 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-9[66545]: 2024-04-16T17:19:10.361+0000 7f5937600700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-16T17:18:43.371969+0000 front 2024-04-16T17:18:43.371904+0000 (oldest deadline 2024-04-16T17:19:08.671642+0000) 2024-04-16T17:19:10.728 INFO:journalctl@ceph.osd.6.smithi045.stdout:Apr 16 17:19:10 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-6[59926]: 2024-04-16T17:19:10.579+0000 7fae88800700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-16T17:18:45.105720+0000 front 2024-04-16T17:18:45.105642+0000 (oldest deadline 2024-04-16T17:19:09.805838+0000) 2024-04-16T17:19:10.804 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:19:10 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:19:10.424+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-16T17:18:34.374061+0000 front 2024-04-16T17:18:34.374275+0000 (oldest deadline 2024-04-16T17:18:57.873879+0000) 2024-04-16T17:19:10.927 INFO:journalctl@ceph.osd.7.smithi104.stdout:Apr 16 17:19:10 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-7[55033]: 2024-04-16T17:19:10.567+0000 7f08b34d5700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-16T17:18:42.259445+0000 front 2024-04-16T17:18:42.259491+0000 (oldest deadline 2024-04-16T17:19:08.159569+0000) 2024-04-16T17:19:11.304 INFO:journalctl@ceph.osd.8.smithi117.stdout:Apr 16 17:19:10 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-8[55817]: 2024-04-16T17:19:10.907+0000 7f65ab398700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-16T17:18:41.860244+0000 front 2024-04-16T17:18:41.860176+0000 (oldest deadline 2024-04-16T17:19:07.760349+0000) 2024-04-16T17:19:11.305 INFO:journalctl@ceph.osd.5.smithi117.stdout:Apr 16 17:19:11 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-5[48830]: 2024-04-16T17:19:11.041+0000 7fe9dcc0c700 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-16T17:18:35.703278+0000 front 2024-04-16T17:18:35.702939+0000 (oldest deadline 2024-04-16T17:19:02.703378+0000) 2024-04-16T17:19:11.305 INFO:journalctl@ceph.osd.5.smithi117.stdout:Apr 16 17:19:11 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-5[48830]: 2024-04-16T17:19:11.041+0000 7fe9dcc0c700 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-16T17:18:45.604466+0000 front 2024-04-16T17:18:45.604549+0000 (oldest deadline 2024-04-16T17:19:08.504560+0000) 2024-04-16T17:19:11.427 INFO:journalctl@ceph.osd.4.smithi104.stdout:Apr 16 17:19:11 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-4[48045]: 2024-04-16T17:19:11.067+0000 7fa479fe0700 -1 osd.4 112 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-16T17:18:40.645865+0000 front 2024-04-16T17:18:40.646010+0000 (oldest deadline 2024-04-16T17:19:10.046543+0000) 2024-04-16T17:19:11.427 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:19:11 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:19:11.138+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-16T17:18:36.158791+0000 front 2024-04-16T17:18:36.158992+0000 (oldest deadline 2024-04-16T17:19:00.258603+0000) 2024-04-16T17:19:11.427 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:19:11 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:19:11.138+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-16T17:18:44.260046+0000 front 2024-04-16T17:18:44.260171+0000 (oldest deadline 2024-04-16T17:19:07.160168+0000) 2024-04-16T17:19:11.727 INFO:journalctl@ceph.osd.6.smithi045.stdout:Apr 16 17:19:11 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-6[59926]: 2024-04-16T17:19:11.567+0000 7fae88800700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-16T17:18:45.105720+0000 front 2024-04-16T17:18:45.105642+0000 (oldest deadline 2024-04-16T17:19:09.805838+0000) 2024-04-16T17:19:11.727 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 16 17:19:11 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-9[66545]: 2024-04-16T17:19:11.318+0000 7f5937600700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-16T17:18:36.968505+0000 front 2024-04-16T17:18:36.968670+0000 (oldest deadline 2024-04-16T17:19:02.268762+0000) 2024-04-16T17:19:11.727 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 16 17:19:11 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-9[66545]: 2024-04-16T17:19:11.318+0000 7f5937600700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-16T17:18:43.371969+0000 front 2024-04-16T17:18:43.371904+0000 (oldest deadline 2024-04-16T17:19:08.671642+0000) 2024-04-16T17:19:11.804 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:19:11 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:19:11.397+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-16T17:18:34.374061+0000 front 2024-04-16T17:18:34.374275+0000 (oldest deadline 2024-04-16T17:18:57.873879+0000) 2024-04-16T17:19:11.927 INFO:journalctl@ceph.osd.7.smithi104.stdout:Apr 16 17:19:11 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-7[55033]: 2024-04-16T17:19:11.526+0000 7f08b34d5700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-16T17:18:42.259445+0000 front 2024-04-16T17:18:42.259491+0000 (oldest deadline 2024-04-16T17:19:08.159569+0000) 2024-04-16T17:19:12.304 INFO:journalctl@ceph.osd.8.smithi117.stdout:Apr 16 17:19:11 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-8[55817]: 2024-04-16T17:19:11.948+0000 7f65ab398700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-16T17:18:41.860244+0000 front 2024-04-16T17:18:41.860176+0000 (oldest deadline 2024-04-16T17:19:07.760349+0000) 2024-04-16T17:19:12.305 INFO:journalctl@ceph.osd.5.smithi117.stdout:Apr 16 17:19:12 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-5[48830]: 2024-04-16T17:19:12.023+0000 7fe9dcc0c700 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-16T17:18:35.703278+0000 front 2024-04-16T17:18:35.702939+0000 (oldest deadline 2024-04-16T17:19:02.703378+0000) 2024-04-16T17:19:12.305 INFO:journalctl@ceph.osd.5.smithi117.stdout:Apr 16 17:19:12 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-5[48830]: 2024-04-16T17:19:12.023+0000 7fe9dcc0c700 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-16T17:18:45.604466+0000 front 2024-04-16T17:18:45.604549+0000 (oldest deadline 2024-04-16T17:19:08.504560+0000) 2024-04-16T17:19:12.427 INFO:journalctl@ceph.osd.4.smithi104.stdout:Apr 16 17:19:12 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-4[48045]: 2024-04-16T17:19:12.021+0000 7fa479fe0700 -1 osd.4 112 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-16T17:18:40.645865+0000 front 2024-04-16T17:18:40.646010+0000 (oldest deadline 2024-04-16T17:19:10.046543+0000) 2024-04-16T17:19:12.427 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:19:12 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:19:12.143+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-16T17:18:36.158791+0000 front 2024-04-16T17:18:36.158992+0000 (oldest deadline 2024-04-16T17:19:00.258603+0000) 2024-04-16T17:19:12.427 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:19:12 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:19:12.143+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-16T17:18:44.260046+0000 front 2024-04-16T17:18:44.260171+0000 (oldest deadline 2024-04-16T17:19:07.160168+0000) 2024-04-16T17:19:12.601 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 16 17:19:12 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-9[66545]: 2024-04-16T17:19:12.344+0000 7f5937600700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-16T17:18:36.968505+0000 front 2024-04-16T17:18:36.968670+0000 (oldest deadline 2024-04-16T17:19:02.268762+0000) 2024-04-16T17:19:12.601 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 16 17:19:12 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-9[66545]: 2024-04-16T17:19:12.344+0000 7f5937600700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-16T17:18:43.371969+0000 front 2024-04-16T17:18:43.371904+0000 (oldest deadline 2024-04-16T17:19:08.671642+0000) 2024-04-16T17:19:12.804 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:19:12 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:19:12.390+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-16T17:18:34.374061+0000 front 2024-04-16T17:18:34.374275+0000 (oldest deadline 2024-04-16T17:18:57.873879+0000) 2024-04-16T17:19:12.894 INFO:journalctl@ceph.osd.7.smithi104.stdout:Apr 16 17:19:12 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-7[55033]: 2024-04-16T17:19:12.533+0000 7f08b34d5700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-16T17:18:42.259445+0000 front 2024-04-16T17:18:42.259491+0000 (oldest deadline 2024-04-16T17:19:08.159569+0000) 2024-04-16T17:19:12.976 INFO:journalctl@ceph.osd.6.smithi045.stdout:Apr 16 17:19:12 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-6[59926]: 2024-04-16T17:19:12.600+0000 7fae88800700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-16T17:18:45.105720+0000 front 2024-04-16T17:18:45.105642+0000 (oldest deadline 2024-04-16T17:19:09.805838+0000) 2024-04-16T17:19:13.164 INFO:journalctl@ceph.osd.4.smithi104.stdout:Apr 16 17:19:12 smithi104 podman[74666]: 2024-04-16 17:19:12.89547457 +0000 UTC m=+5.486622071 container died 90c38fbb7da6fc45bddb51d639593c4c81ecb8ffcff7f04cb78c924540d69b6b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-4, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240409, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.5, ceph=True) 2024-04-16T17:19:13.164 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:19:13 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:19:13.097+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-16T17:18:36.158791+0000 front 2024-04-16T17:18:36.158992+0000 (oldest deadline 2024-04-16T17:19:00.258603+0000) 2024-04-16T17:19:13.165 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:19:13 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:19:13.097+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-16T17:18:44.260046+0000 front 2024-04-16T17:18:44.260171+0000 (oldest deadline 2024-04-16T17:19:07.160168+0000) 2024-04-16T17:19:13.305 INFO:journalctl@ceph.osd.8.smithi117.stdout:Apr 16 17:19:12 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-8[55817]: 2024-04-16T17:19:12.974+0000 7f65ab398700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-16T17:18:41.860244+0000 front 2024-04-16T17:18:41.860176+0000 (oldest deadline 2024-04-16T17:19:07.760349+0000) 2024-04-16T17:19:13.305 INFO:journalctl@ceph.osd.5.smithi117.stdout:Apr 16 17:19:13 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-5[48830]: 2024-04-16T17:19:13.042+0000 7fe9dcc0c700 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-16T17:18:35.703278+0000 front 2024-04-16T17:18:35.702939+0000 (oldest deadline 2024-04-16T17:19:02.703378+0000) 2024-04-16T17:19:13.305 INFO:journalctl@ceph.osd.5.smithi117.stdout:Apr 16 17:19:13 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-5[48830]: 2024-04-16T17:19:13.042+0000 7fe9dcc0c700 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-16T17:18:45.604466+0000 front 2024-04-16T17:18:45.604549+0000 (oldest deadline 2024-04-16T17:19:08.504560+0000) 2024-04-16T17:19:13.426 INFO:journalctl@ceph.osd.4.smithi104.stdout:Apr 16 17:19:13 smithi104 podman[74666]: 2024-04-16 17:19:13.247495165 +0000 UTC m=+5.838642704 container cleanup 90c38fbb7da6fc45bddb51d639593c4c81ecb8ffcff7f04cb78c924540d69b6b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-4, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240409, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-04-16T17:19:13.426 INFO:journalctl@ceph.osd.4.smithi104.stdout:Apr 16 17:19:13 smithi104 bash[74666]: ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-4 2024-04-16T17:19:13.630 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 16 17:19:13 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-9[66545]: 2024-04-16T17:19:13.311+0000 7f5937600700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-16T17:18:36.968505+0000 front 2024-04-16T17:18:36.968670+0000 (oldest deadline 2024-04-16T17:19:02.268762+0000) 2024-04-16T17:19:13.631 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 16 17:19:13 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-9[66545]: 2024-04-16T17:19:13.311+0000 7f5937600700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-16T17:18:43.371969+0000 front 2024-04-16T17:18:43.371904+0000 (oldest deadline 2024-04-16T17:19:08.671642+0000) 2024-04-16T17:19:13.804 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:19:13 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:19:13.438+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-16T17:18:34.374061+0000 front 2024-04-16T17:18:34.374275+0000 (oldest deadline 2024-04-16T17:18:57.873879+0000) 2024-04-16T17:19:13.850 INFO:journalctl@ceph.osd.7.smithi104.stdout:Apr 16 17:19:13 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-7[55033]: 2024-04-16T17:19:13.485+0000 7f08b34d5700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-16T17:18:42.259445+0000 front 2024-04-16T17:18:42.259491+0000 (oldest deadline 2024-04-16T17:19:08.159569+0000) 2024-04-16T17:19:13.851 INFO:journalctl@ceph.osd.4.smithi104.stdout:Apr 16 17:19:13 smithi104 podman[74681]: 2024-04-16 17:19:13.524402502 +0000 UTC m=+0.627706792 container remove 90c38fbb7da6fc45bddb51d639593c4c81ecb8ffcff7f04cb78c924540d69b6b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-4, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240409, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-04-16T17:19:13.977 INFO:journalctl@ceph.osd.6.smithi045.stdout:Apr 16 17:19:13 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-6[59926]: 2024-04-16T17:19:13.630+0000 7fae88800700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-16T17:18:45.105720+0000 front 2024-04-16T17:18:45.105642+0000 (oldest deadline 2024-04-16T17:19:09.805838+0000) 2024-04-16T17:19:14.119 INFO:journalctl@ceph.osd.4.smithi104.stdout:Apr 16 17:19:13 smithi104 podman[74776]: 2024-04-16 17:19:13.755320977 +0000 UTC m=+0.017705094 image pull e5ce10bdffbeeffffd1125f67b369e2151f0f17cdec854e93b6adf9321f65907 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2 2024-04-16T17:19:14.119 INFO:journalctl@ceph.osd.4.smithi104.stdout:Apr 16 17:19:13 smithi104 podman[74776]: 2024-04-16 17:19:13.891982955 +0000 UTC m=+0.154367103 container create 43015a08963350bc45bcecbc51ef2c1dc74a52f2d8c61b25ba548dbf32e45510 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-4-deactivate, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240409, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-16T17:19:14.304 INFO:journalctl@ceph.osd.8.smithi117.stdout:Apr 16 17:19:13 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-8[55817]: 2024-04-16T17:19:13.973+0000 7f65ab398700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-16T17:18:41.860244+0000 front 2024-04-16T17:18:41.860176+0000 (oldest deadline 2024-04-16T17:19:07.760349+0000) 2024-04-16T17:19:14.304 INFO:journalctl@ceph.osd.5.smithi117.stdout:Apr 16 17:19:14 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-5[48830]: 2024-04-16T17:19:14.090+0000 7fe9dcc0c700 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-16T17:18:35.703278+0000 front 2024-04-16T17:18:35.702939+0000 (oldest deadline 2024-04-16T17:19:02.703378+0000) 2024-04-16T17:19:14.304 INFO:journalctl@ceph.osd.5.smithi117.stdout:Apr 16 17:19:14 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-5[48830]: 2024-04-16T17:19:14.090+0000 7fe9dcc0c700 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-16T17:18:45.604466+0000 front 2024-04-16T17:18:45.604549+0000 (oldest deadline 2024-04-16T17:19:08.504560+0000) 2024-04-16T17:19:14.427 INFO:journalctl@ceph.osd.4.smithi104.stdout:Apr 16 17:19:14 smithi104 podman[74776]: 2024-04-16 17:19:14.161079434 +0000 UTC m=+0.423463539 container init 43015a08963350bc45bcecbc51ef2c1dc74a52f2d8c61b25ba548dbf32e45510 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-4-deactivate, GIT_CLEAN=True, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, org.label-schema.build-date=20240409, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True) 2024-04-16T17:19:14.427 INFO:journalctl@ceph.osd.4.smithi104.stdout:Apr 16 17:19:14 smithi104 podman[74776]: 2024-04-16 17:19:14.164164643 +0000 UTC m=+0.426548742 container start 43015a08963350bc45bcecbc51ef2c1dc74a52f2d8c61b25ba548dbf32e45510 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-4-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240409, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2) 2024-04-16T17:19:14.427 INFO:journalctl@ceph.osd.4.smithi104.stdout:Apr 16 17:19:14 smithi104 podman[74776]: 2024-04-16 17:19:14.261150289 +0000 UTC m=+0.523534413 container attach 43015a08963350bc45bcecbc51ef2c1dc74a52f2d8c61b25ba548dbf32e45510 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-4-deactivate, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240409, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-04-16T17:19:14.428 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:19:14 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:19:14.119+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-16T17:18:36.158791+0000 front 2024-04-16T17:18:36.158992+0000 (oldest deadline 2024-04-16T17:19:00.258603+0000) 2024-04-16T17:19:14.428 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:19:14 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:19:14.119+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-16T17:18:44.260046+0000 front 2024-04-16T17:18:44.260171+0000 (oldest deadline 2024-04-16T17:19:07.160168+0000) 2024-04-16T17:19:14.677 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 16 17:19:14 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-9[66545]: 2024-04-16T17:19:14.304+0000 7f5937600700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-16T17:18:36.968505+0000 front 2024-04-16T17:18:36.968670+0000 (oldest deadline 2024-04-16T17:19:02.268762+0000) 2024-04-16T17:19:14.678 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 16 17:19:14 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-9[66545]: 2024-04-16T17:19:14.304+0000 7f5937600700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-16T17:18:43.371969+0000 front 2024-04-16T17:18:43.371904+0000 (oldest deadline 2024-04-16T17:19:08.671642+0000) 2024-04-16T17:19:14.746 INFO:journalctl@ceph.osd.7.smithi104.stdout:Apr 16 17:19:14 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-7[55033]: 2024-04-16T17:19:14.527+0000 7f08b34d5700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-16T17:18:42.259445+0000 front 2024-04-16T17:18:42.259491+0000 (oldest deadline 2024-04-16T17:19:08.159569+0000) 2024-04-16T17:19:14.746 INFO:journalctl@ceph.osd.4.smithi104.stdout:Apr 16 17:19:14 smithi104 podman[74776]: 2024-04-16 17:19:14.448268443 +0000 UTC m=+0.710652551 container died 43015a08963350bc45bcecbc51ef2c1dc74a52f2d8c61b25ba548dbf32e45510 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-4-deactivate, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, ceph=True, org.label-schema.build-date=20240409, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-04-16T17:19:14.804 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:19:14 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:19:14.471+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-16T17:18:34.374061+0000 front 2024-04-16T17:18:34.374275+0000 (oldest deadline 2024-04-16T17:18:57.873879+0000) 2024-04-16T17:19:14.977 INFO:journalctl@ceph.osd.6.smithi045.stdout:Apr 16 17:19:14 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-6[59926]: 2024-04-16T17:19:14.677+0000 7fae88800700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-16T17:18:45.105720+0000 front 2024-04-16T17:18:45.105642+0000 (oldest deadline 2024-04-16T17:19:09.805838+0000) 2024-04-16T17:19:15.020 DEBUG:teuthology.orchestra.run.smithi104:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b1fecd38-fc11-11ee-bc90-c7b262605968@osd.4.service' 2024-04-16T17:19:15.062 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-16T17:19:15.062 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-04-16T17:19:15.062 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-04-16T17:19:15.062 DEBUG:teuthology.orchestra.run.smithi117:> sudo systemctl stop ceph-b1fecd38-fc11-11ee-bc90-c7b262605968@osd.5 2024-04-16T17:19:15.304 INFO:journalctl@ceph.osd.8.smithi117.stdout:Apr 16 17:19:14 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-8[55817]: 2024-04-16T17:19:14.933+0000 7f65ab398700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-16T17:18:41.860244+0000 front 2024-04-16T17:18:41.860176+0000 (oldest deadline 2024-04-16T17:19:07.760349+0000) 2024-04-16T17:19:15.304 INFO:journalctl@ceph.osd.5.smithi117.stdout:Apr 16 17:19:15 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-5[48830]: 2024-04-16T17:19:15.049+0000 7fe9dcc0c700 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-16T17:18:35.703278+0000 front 2024-04-16T17:18:35.702939+0000 (oldest deadline 2024-04-16T17:19:02.703378+0000) 2024-04-16T17:19:15.304 INFO:journalctl@ceph.osd.5.smithi117.stdout:Apr 16 17:19:15 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-5[48830]: 2024-04-16T17:19:15.049+0000 7fe9dcc0c700 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-16T17:18:45.604466+0000 front 2024-04-16T17:18:45.604549+0000 (oldest deadline 2024-04-16T17:19:08.504560+0000) 2024-04-16T17:19:15.305 INFO:journalctl@ceph.osd.5.smithi117.stdout:Apr 16 17:19:15 smithi117 systemd[1]: Stopping Ceph osd.5 for b1fecd38-fc11-11ee-bc90-c7b262605968... 2024-04-16T17:19:15.427 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:19:15 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:19:15.096+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-16T17:18:36.158791+0000 front 2024-04-16T17:18:36.158992+0000 (oldest deadline 2024-04-16T17:19:00.258603+0000) 2024-04-16T17:19:15.427 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:19:15 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:19:15.096+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-16T17:18:44.260046+0000 front 2024-04-16T17:18:44.260171+0000 (oldest deadline 2024-04-16T17:19:07.160168+0000) 2024-04-16T17:19:15.690 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 16 17:19:15 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-9[66545]: 2024-04-16T17:19:15.309+0000 7f5937600700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-16T17:18:36.968505+0000 front 2024-04-16T17:18:36.968670+0000 (oldest deadline 2024-04-16T17:19:02.268762+0000) 2024-04-16T17:19:15.690 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 16 17:19:15 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-9[66545]: 2024-04-16T17:19:15.309+0000 7f5937600700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-16T17:18:43.371969+0000 front 2024-04-16T17:18:43.371904+0000 (oldest deadline 2024-04-16T17:19:08.671642+0000) 2024-04-16T17:19:15.804 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:19:15 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:19:15.519+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-16T17:18:34.374061+0000 front 2024-04-16T17:18:34.374275+0000 (oldest deadline 2024-04-16T17:18:57.873879+0000) 2024-04-16T17:19:15.804 INFO:journalctl@ceph.osd.5.smithi117.stdout:Apr 16 17:19:15 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-5[48830]: 2024-04-16T17:19:15.417+0000 7fe9e0618700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-16T17:19:15.804 INFO:journalctl@ceph.osd.5.smithi117.stdout:Apr 16 17:19:15 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-5[48830]: 2024-04-16T17:19:15.417+0000 7fe9e0618700 -1 osd.5 112 *** Got signal Terminated *** 2024-04-16T17:19:15.804 INFO:journalctl@ceph.osd.5.smithi117.stdout:Apr 16 17:19:15 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-5[48830]: 2024-04-16T17:19:15.417+0000 7fe9e0618700 -1 osd.5 112 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-16T17:19:15.926 INFO:journalctl@ceph.osd.7.smithi104.stdout:Apr 16 17:19:15 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-7[55033]: 2024-04-16T17:19:15.548+0000 7f08b34d5700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-16T17:18:42.259445+0000 front 2024-04-16T17:18:42.259491+0000 (oldest deadline 2024-04-16T17:19:08.159569+0000) 2024-04-16T17:19:15.976 INFO:journalctl@ceph.osd.6.smithi045.stdout:Apr 16 17:19:15 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-6[59926]: 2024-04-16T17:19:15.690+0000 7fae88800700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-16T17:18:45.105720+0000 front 2024-04-16T17:18:45.105642+0000 (oldest deadline 2024-04-16T17:19:09.805838+0000) 2024-04-16T17:19:15.977 INFO:journalctl@ceph.osd.6.smithi045.stdout:Apr 16 17:19:15 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-6[59926]: 2024-04-16T17:19:15.690+0000 7fae88800700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-16T17:18:52.708348+0000 front 2024-04-16T17:18:52.707998+0000 (oldest deadline 2024-04-16T17:19:15.007885+0000) 2024-04-16T17:19:16.304 INFO:journalctl@ceph.osd.5.smithi117.stdout:Apr 16 17:19:16 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-5[48830]: 2024-04-16T17:19:16.052+0000 7fe9dcc0c700 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-16T17:18:35.703278+0000 front 2024-04-16T17:18:35.702939+0000 (oldest deadline 2024-04-16T17:19:02.703378+0000) 2024-04-16T17:19:16.304 INFO:journalctl@ceph.osd.5.smithi117.stdout:Apr 16 17:19:16 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-5[48830]: 2024-04-16T17:19:16.052+0000 7fe9dcc0c700 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-16T17:18:45.604466+0000 front 2024-04-16T17:18:45.604549+0000 (oldest deadline 2024-04-16T17:19:08.504560+0000) 2024-04-16T17:19:16.305 INFO:journalctl@ceph.osd.8.smithi117.stdout:Apr 16 17:19:15 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-8[55817]: 2024-04-16T17:19:15.900+0000 7f65ab398700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-16T17:18:41.860244+0000 front 2024-04-16T17:18:41.860176+0000 (oldest deadline 2024-04-16T17:19:07.760349+0000) 2024-04-16T17:19:16.427 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:19:16 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:19:16.111+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-16T17:18:36.158791+0000 front 2024-04-16T17:18:36.158992+0000 (oldest deadline 2024-04-16T17:19:00.258603+0000) 2024-04-16T17:19:16.427 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:19:16 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:19:16.111+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-16T17:18:44.260046+0000 front 2024-04-16T17:18:44.260171+0000 (oldest deadline 2024-04-16T17:19:07.160168+0000) 2024-04-16T17:19:16.702 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 16 17:19:16 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-9[66545]: 2024-04-16T17:19:16.284+0000 7f5937600700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-16T17:18:36.968505+0000 front 2024-04-16T17:18:36.968670+0000 (oldest deadline 2024-04-16T17:19:02.268762+0000) 2024-04-16T17:19:16.702 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 16 17:19:16 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-9[66545]: 2024-04-16T17:19:16.284+0000 7f5937600700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-16T17:18:43.371969+0000 front 2024-04-16T17:18:43.371904+0000 (oldest deadline 2024-04-16T17:19:08.671642+0000) 2024-04-16T17:19:16.703 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 16 17:19:16 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-9[66545]: 2024-04-16T17:19:16.284+0000 7f5937600700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-16T17:18:53.373968+0000 front 2024-04-16T17:18:53.374043+0000 (oldest deadline 2024-04-16T17:19:15.673755+0000) 2024-04-16T17:19:16.804 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:19:16 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:19:16.502+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-16T17:18:34.374061+0000 front 2024-04-16T17:18:34.374275+0000 (oldest deadline 2024-04-16T17:18:57.873879+0000) 2024-04-16T17:19:16.804 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:19:16 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:19:16.502+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-16T17:18:50.676854+0000 front 2024-04-16T17:18:50.676874+0000 (oldest deadline 2024-04-16T17:19:15.977004+0000) 2024-04-16T17:19:16.927 INFO:journalctl@ceph.osd.7.smithi104.stdout:Apr 16 17:19:16 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-7[55033]: 2024-04-16T17:19:16.530+0000 7f08b34d5700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-16T17:18:42.259445+0000 front 2024-04-16T17:18:42.259491+0000 (oldest deadline 2024-04-16T17:19:08.159569+0000) 2024-04-16T17:19:16.928 INFO:journalctl@ceph.osd.7.smithi104.stdout:Apr 16 17:19:16 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-7[55033]: 2024-04-16T17:19:16.530+0000 7f08b34d5700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-16T17:18:50.460516+0000 front 2024-04-16T17:18:50.460605+0000 (oldest deadline 2024-04-16T17:19:16.360615+0000) 2024-04-16T17:19:16.976 INFO:journalctl@ceph.osd.6.smithi045.stdout:Apr 16 17:19:16 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-6[59926]: 2024-04-16T17:19:16.702+0000 7fae88800700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-16T17:18:45.105720+0000 front 2024-04-16T17:18:45.105642+0000 (oldest deadline 2024-04-16T17:19:09.805838+0000) 2024-04-16T17:19:16.977 INFO:journalctl@ceph.osd.6.smithi045.stdout:Apr 16 17:19:16 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-6[59926]: 2024-04-16T17:19:16.702+0000 7fae88800700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-16T17:18:52.708348+0000 front 2024-04-16T17:18:52.707998+0000 (oldest deadline 2024-04-16T17:19:15.007885+0000) 2024-04-16T17:19:17.304 INFO:journalctl@ceph.osd.8.smithi117.stdout:Apr 16 17:19:16 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-8[55817]: 2024-04-16T17:19:16.900+0000 7f65ab398700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-16T17:18:41.860244+0000 front 2024-04-16T17:18:41.860176+0000 (oldest deadline 2024-04-16T17:19:07.760349+0000) 2024-04-16T17:19:17.305 INFO:journalctl@ceph.osd.5.smithi117.stdout:Apr 16 17:19:17 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-5[48830]: 2024-04-16T17:19:17.015+0000 7fe9dcc0c700 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-16T17:18:35.703278+0000 front 2024-04-16T17:18:35.702939+0000 (oldest deadline 2024-04-16T17:19:02.703378+0000) 2024-04-16T17:19:17.305 INFO:journalctl@ceph.osd.5.smithi117.stdout:Apr 16 17:19:17 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-5[48830]: 2024-04-16T17:19:17.015+0000 7fe9dcc0c700 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-16T17:18:45.604466+0000 front 2024-04-16T17:18:45.604549+0000 (oldest deadline 2024-04-16T17:19:08.504560+0000) 2024-04-16T17:19:17.305 INFO:journalctl@ceph.osd.5.smithi117.stdout:Apr 16 17:19:17 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-5[48830]: 2024-04-16T17:19:17.015+0000 7fe9dcc0c700 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-16T17:18:50.205325+0000 front 2024-04-16T17:18:50.205346+0000 (oldest deadline 2024-04-16T17:19:16.105450+0000) 2024-04-16T17:19:17.426 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:19:17 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:19:17.111+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-16T17:18:36.158791+0000 front 2024-04-16T17:18:36.158992+0000 (oldest deadline 2024-04-16T17:19:00.258603+0000) 2024-04-16T17:19:17.427 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:19:17 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:19:17.111+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-16T17:18:44.260046+0000 front 2024-04-16T17:18:44.260171+0000 (oldest deadline 2024-04-16T17:19:07.160168+0000) 2024-04-16T17:19:17.661 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 16 17:19:17 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-9[66545]: 2024-04-16T17:19:17.307+0000 7f5937600700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-16T17:18:36.968505+0000 front 2024-04-16T17:18:36.968670+0000 (oldest deadline 2024-04-16T17:19:02.268762+0000) 2024-04-16T17:19:17.661 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 16 17:19:17 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-9[66545]: 2024-04-16T17:19:17.307+0000 7f5937600700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-16T17:18:43.371969+0000 front 2024-04-16T17:18:43.371904+0000 (oldest deadline 2024-04-16T17:19:08.671642+0000) 2024-04-16T17:19:17.661 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 16 17:19:17 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-9[66545]: 2024-04-16T17:19:17.307+0000 7f5937600700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-16T17:18:53.373968+0000 front 2024-04-16T17:18:53.374043+0000 (oldest deadline 2024-04-16T17:19:15.673755+0000) 2024-04-16T17:19:17.699 INFO:journalctl@ceph.osd.7.smithi104.stdout:Apr 16 17:19:17 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-7[55033]: 2024-04-16T17:19:17.521+0000 7f08b34d5700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-16T17:18:42.259445+0000 front 2024-04-16T17:18:42.259491+0000 (oldest deadline 2024-04-16T17:19:08.159569+0000) 2024-04-16T17:19:17.699 INFO:journalctl@ceph.osd.7.smithi104.stdout:Apr 16 17:19:17 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-7[55033]: 2024-04-16T17:19:17.521+0000 7f08b34d5700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-16T17:18:50.460516+0000 front 2024-04-16T17:18:50.460605+0000 (oldest deadline 2024-04-16T17:19:16.360615+0000) 2024-04-16T17:19:17.804 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:19:17 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:19:17.527+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-16T17:18:34.374061+0000 front 2024-04-16T17:18:34.374275+0000 (oldest deadline 2024-04-16T17:18:57.873879+0000) 2024-04-16T17:19:17.804 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:19:17 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:19:17.527+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-16T17:18:50.676854+0000 front 2024-04-16T17:18:50.676874+0000 (oldest deadline 2024-04-16T17:19:15.977004+0000) 2024-04-16T17:19:17.976 INFO:journalctl@ceph.osd.6.smithi045.stdout:Apr 16 17:19:17 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-6[59926]: 2024-04-16T17:19:17.660+0000 7fae88800700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-16T17:18:45.105720+0000 front 2024-04-16T17:18:45.105642+0000 (oldest deadline 2024-04-16T17:19:09.805838+0000) 2024-04-16T17:19:17.977 INFO:journalctl@ceph.osd.6.smithi045.stdout:Apr 16 17:19:17 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-6[59926]: 2024-04-16T17:19:17.660+0000 7fae88800700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-16T17:18:52.708348+0000 front 2024-04-16T17:18:52.707998+0000 (oldest deadline 2024-04-16T17:19:15.007885+0000) 2024-04-16T17:19:18.304 INFO:journalctl@ceph.osd.8.smithi117.stdout:Apr 16 17:19:17 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-8[55817]: 2024-04-16T17:19:17.920+0000 7f65ab398700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-16T17:18:41.860244+0000 front 2024-04-16T17:18:41.860176+0000 (oldest deadline 2024-04-16T17:19:07.760349+0000) 2024-04-16T17:19:18.304 INFO:journalctl@ceph.osd.5.smithi117.stdout:Apr 16 17:19:18 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-5[48830]: 2024-04-16T17:19:18.058+0000 7fe9dcc0c700 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-16T17:18:35.703278+0000 front 2024-04-16T17:18:35.702939+0000 (oldest deadline 2024-04-16T17:19:02.703378+0000) 2024-04-16T17:19:18.304 INFO:journalctl@ceph.osd.5.smithi117.stdout:Apr 16 17:19:18 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-5[48830]: 2024-04-16T17:19:18.058+0000 7fe9dcc0c700 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-16T17:18:45.604466+0000 front 2024-04-16T17:18:45.604549+0000 (oldest deadline 2024-04-16T17:19:08.504560+0000) 2024-04-16T17:19:18.305 INFO:journalctl@ceph.osd.5.smithi117.stdout:Apr 16 17:19:18 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-5[48830]: 2024-04-16T17:19:18.058+0000 7fe9dcc0c700 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-16T17:18:50.205325+0000 front 2024-04-16T17:18:50.205346+0000 (oldest deadline 2024-04-16T17:19:16.105450+0000) 2024-04-16T17:19:18.426 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:19:18 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:19:18.158+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-16T17:18:36.158791+0000 front 2024-04-16T17:18:36.158992+0000 (oldest deadline 2024-04-16T17:19:00.258603+0000) 2024-04-16T17:19:18.426 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:19:18 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:19:18.158+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-16T17:18:44.260046+0000 front 2024-04-16T17:18:44.260171+0000 (oldest deadline 2024-04-16T17:19:07.160168+0000) 2024-04-16T17:19:18.686 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 16 17:19:18 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-9[66545]: 2024-04-16T17:19:18.264+0000 7f5937600700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-16T17:18:36.968505+0000 front 2024-04-16T17:18:36.968670+0000 (oldest deadline 2024-04-16T17:19:02.268762+0000) 2024-04-16T17:19:18.686 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 16 17:19:18 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-9[66545]: 2024-04-16T17:19:18.264+0000 7f5937600700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-16T17:18:43.371969+0000 front 2024-04-16T17:18:43.371904+0000 (oldest deadline 2024-04-16T17:19:08.671642+0000) 2024-04-16T17:19:18.686 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 16 17:19:18 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-9[66545]: 2024-04-16T17:19:18.264+0000 7f5937600700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-16T17:18:53.373968+0000 front 2024-04-16T17:18:53.374043+0000 (oldest deadline 2024-04-16T17:19:15.673755+0000) 2024-04-16T17:19:18.804 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:19:18 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:19:18.507+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-16T17:18:34.374061+0000 front 2024-04-16T17:18:34.374275+0000 (oldest deadline 2024-04-16T17:18:57.873879+0000) 2024-04-16T17:19:18.804 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:19:18 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:19:18.507+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-16T17:18:50.676854+0000 front 2024-04-16T17:18:50.676874+0000 (oldest deadline 2024-04-16T17:19:15.977004+0000) 2024-04-16T17:19:18.927 INFO:journalctl@ceph.osd.7.smithi104.stdout:Apr 16 17:19:18 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-7[55033]: 2024-04-16T17:19:18.545+0000 7f08b34d5700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-16T17:18:42.259445+0000 front 2024-04-16T17:18:42.259491+0000 (oldest deadline 2024-04-16T17:19:08.159569+0000) 2024-04-16T17:19:18.927 INFO:journalctl@ceph.osd.7.smithi104.stdout:Apr 16 17:19:18 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-7[55033]: 2024-04-16T17:19:18.545+0000 7f08b34d5700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-16T17:18:50.460516+0000 front 2024-04-16T17:18:50.460605+0000 (oldest deadline 2024-04-16T17:19:16.360615+0000) 2024-04-16T17:19:18.976 INFO:journalctl@ceph.osd.6.smithi045.stdout:Apr 16 17:19:18 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-6[59926]: 2024-04-16T17:19:18.685+0000 7fae88800700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-16T17:18:45.105720+0000 front 2024-04-16T17:18:45.105642+0000 (oldest deadline 2024-04-16T17:19:09.805838+0000) 2024-04-16T17:19:18.977 INFO:journalctl@ceph.osd.6.smithi045.stdout:Apr 16 17:19:18 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-6[59926]: 2024-04-16T17:19:18.685+0000 7fae88800700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-16T17:18:52.708348+0000 front 2024-04-16T17:18:52.707998+0000 (oldest deadline 2024-04-16T17:19:15.007885+0000) 2024-04-16T17:19:19.304 INFO:journalctl@ceph.osd.8.smithi117.stdout:Apr 16 17:19:18 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-8[55817]: 2024-04-16T17:19:18.922+0000 7f65ab398700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-16T17:18:41.860244+0000 front 2024-04-16T17:18:41.860176+0000 (oldest deadline 2024-04-16T17:19:07.760349+0000) 2024-04-16T17:19:19.305 INFO:journalctl@ceph.osd.5.smithi117.stdout:Apr 16 17:19:19 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-5[48830]: 2024-04-16T17:19:19.035+0000 7fe9dcc0c700 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-16T17:18:35.703278+0000 front 2024-04-16T17:18:35.702939+0000 (oldest deadline 2024-04-16T17:19:02.703378+0000) 2024-04-16T17:19:19.305 INFO:journalctl@ceph.osd.5.smithi117.stdout:Apr 16 17:19:19 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-5[48830]: 2024-04-16T17:19:19.035+0000 7fe9dcc0c700 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-16T17:18:45.604466+0000 front 2024-04-16T17:18:45.604549+0000 (oldest deadline 2024-04-16T17:19:08.504560+0000) 2024-04-16T17:19:19.305 INFO:journalctl@ceph.osd.5.smithi117.stdout:Apr 16 17:19:19 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-5[48830]: 2024-04-16T17:19:19.035+0000 7fe9dcc0c700 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-16T17:18:50.205325+0000 front 2024-04-16T17:18:50.205346+0000 (oldest deadline 2024-04-16T17:19:16.105450+0000) 2024-04-16T17:19:19.427 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:19:19 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:19:19.155+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-16T17:18:36.158791+0000 front 2024-04-16T17:18:36.158992+0000 (oldest deadline 2024-04-16T17:19:00.258603+0000) 2024-04-16T17:19:19.427 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:19:19 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:19:19.155+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-16T17:18:44.260046+0000 front 2024-04-16T17:18:44.260171+0000 (oldest deadline 2024-04-16T17:19:07.160168+0000) 2024-04-16T17:19:19.427 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:19:19 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:19:19.155+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-16T17:18:54.062069+0000 front 2024-04-16T17:18:54.061971+0000 (oldest deadline 2024-04-16T17:19:18.761808+0000) 2024-04-16T17:19:19.574 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:19:19 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:19:19.505+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-16T17:18:34.374061+0000 front 2024-04-16T17:18:34.374275+0000 (oldest deadline 2024-04-16T17:18:57.873879+0000) 2024-04-16T17:19:19.575 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:19:19 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:19:19.505+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-16T17:18:50.676854+0000 front 2024-04-16T17:18:50.676874+0000 (oldest deadline 2024-04-16T17:19:15.977004+0000) 2024-04-16T17:19:19.723 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 16 17:19:19 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-9[66545]: 2024-04-16T17:19:19.260+0000 7f5937600700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-16T17:18:36.968505+0000 front 2024-04-16T17:18:36.968670+0000 (oldest deadline 2024-04-16T17:19:02.268762+0000) 2024-04-16T17:19:19.724 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 16 17:19:19 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-9[66545]: 2024-04-16T17:19:19.260+0000 7f5937600700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-16T17:18:43.371969+0000 front 2024-04-16T17:18:43.371904+0000 (oldest deadline 2024-04-16T17:19:08.671642+0000) 2024-04-16T17:19:19.724 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 16 17:19:19 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-9[66545]: 2024-04-16T17:19:19.260+0000 7f5937600700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-16T17:18:53.373968+0000 front 2024-04-16T17:18:53.374043+0000 (oldest deadline 2024-04-16T17:19:15.673755+0000) 2024-04-16T17:19:19.927 INFO:journalctl@ceph.osd.7.smithi104.stdout:Apr 16 17:19:19 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-7[55033]: 2024-04-16T17:19:19.582+0000 7f08b34d5700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-16T17:18:42.259445+0000 front 2024-04-16T17:18:42.259491+0000 (oldest deadline 2024-04-16T17:19:08.159569+0000) 2024-04-16T17:19:19.927 INFO:journalctl@ceph.osd.7.smithi104.stdout:Apr 16 17:19:19 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-7[55033]: 2024-04-16T17:19:19.582+0000 7f08b34d5700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-16T17:18:50.460516+0000 front 2024-04-16T17:18:50.460605+0000 (oldest deadline 2024-04-16T17:19:16.360615+0000) 2024-04-16T17:19:19.976 INFO:journalctl@ceph.osd.6.smithi045.stdout:Apr 16 17:19:19 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-6[59926]: 2024-04-16T17:19:19.723+0000 7fae88800700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-16T17:18:45.105720+0000 front 2024-04-16T17:18:45.105642+0000 (oldest deadline 2024-04-16T17:19:09.805838+0000) 2024-04-16T17:19:19.977 INFO:journalctl@ceph.osd.6.smithi045.stdout:Apr 16 17:19:19 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-6[59926]: 2024-04-16T17:19:19.723+0000 7fae88800700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-16T17:18:52.708348+0000 front 2024-04-16T17:18:52.707998+0000 (oldest deadline 2024-04-16T17:19:15.007885+0000) 2024-04-16T17:19:20.304 INFO:journalctl@ceph.osd.8.smithi117.stdout:Apr 16 17:19:19 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-8[55817]: 2024-04-16T17:19:19.877+0000 7f65ab398700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-16T17:18:41.860244+0000 front 2024-04-16T17:18:41.860176+0000 (oldest deadline 2024-04-16T17:19:07.760349+0000) 2024-04-16T17:19:20.304 INFO:journalctl@ceph.osd.5.smithi117.stdout:Apr 16 17:19:20 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-5[48830]: 2024-04-16T17:19:20.049+0000 7fe9dcc0c700 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-16T17:18:35.703278+0000 front 2024-04-16T17:18:35.702939+0000 (oldest deadline 2024-04-16T17:19:02.703378+0000) 2024-04-16T17:19:20.305 INFO:journalctl@ceph.osd.5.smithi117.stdout:Apr 16 17:19:20 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-5[48830]: 2024-04-16T17:19:20.049+0000 7fe9dcc0c700 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-16T17:18:45.604466+0000 front 2024-04-16T17:18:45.604549+0000 (oldest deadline 2024-04-16T17:19:08.504560+0000) 2024-04-16T17:19:20.305 INFO:journalctl@ceph.osd.5.smithi117.stdout:Apr 16 17:19:20 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-5[48830]: 2024-04-16T17:19:20.049+0000 7fe9dcc0c700 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-16T17:18:50.205325+0000 front 2024-04-16T17:18:50.205346+0000 (oldest deadline 2024-04-16T17:19:16.105450+0000) 2024-04-16T17:19:20.426 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:19:20 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:19:20.172+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-16T17:18:36.158791+0000 front 2024-04-16T17:18:36.158992+0000 (oldest deadline 2024-04-16T17:19:00.258603+0000) 2024-04-16T17:19:20.426 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:19:20 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:19:20.172+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-16T17:18:44.260046+0000 front 2024-04-16T17:18:44.260171+0000 (oldest deadline 2024-04-16T17:19:07.160168+0000) 2024-04-16T17:19:20.427 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:19:20 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:19:20.172+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-16T17:18:54.062069+0000 front 2024-04-16T17:18:54.061971+0000 (oldest deadline 2024-04-16T17:19:18.761808+0000) 2024-04-16T17:19:20.727 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 16 17:19:20 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-9[66545]: 2024-04-16T17:19:20.287+0000 7f5937600700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-16T17:18:36.968505+0000 front 2024-04-16T17:18:36.968670+0000 (oldest deadline 2024-04-16T17:19:02.268762+0000) 2024-04-16T17:19:20.727 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 16 17:19:20 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-9[66545]: 2024-04-16T17:19:20.287+0000 7f5937600700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-16T17:18:43.371969+0000 front 2024-04-16T17:18:43.371904+0000 (oldest deadline 2024-04-16T17:19:08.671642+0000) 2024-04-16T17:19:20.727 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 16 17:19:20 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-9[66545]: 2024-04-16T17:19:20.287+0000 7f5937600700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-16T17:18:53.373968+0000 front 2024-04-16T17:18:53.374043+0000 (oldest deadline 2024-04-16T17:19:15.673755+0000) 2024-04-16T17:19:20.804 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:19:20 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:19:20.543+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-16T17:18:34.374061+0000 front 2024-04-16T17:18:34.374275+0000 (oldest deadline 2024-04-16T17:18:57.873879+0000) 2024-04-16T17:19:20.804 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:19:20 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:19:20.543+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-16T17:18:50.676854+0000 front 2024-04-16T17:18:50.676874+0000 (oldest deadline 2024-04-16T17:19:15.977004+0000) 2024-04-16T17:19:20.926 INFO:journalctl@ceph.osd.7.smithi104.stdout:Apr 16 17:19:20 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-7[55033]: 2024-04-16T17:19:20.580+0000 7f08b34d5700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-16T17:18:42.259445+0000 front 2024-04-16T17:18:42.259491+0000 (oldest deadline 2024-04-16T17:19:08.159569+0000) 2024-04-16T17:19:20.927 INFO:journalctl@ceph.osd.7.smithi104.stdout:Apr 16 17:19:20 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-7[55033]: 2024-04-16T17:19:20.580+0000 7f08b34d5700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-16T17:18:50.460516+0000 front 2024-04-16T17:18:50.460605+0000 (oldest deadline 2024-04-16T17:19:16.360615+0000) 2024-04-16T17:19:21.226 INFO:journalctl@ceph.osd.6.smithi045.stdout:Apr 16 17:19:20 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-6[59926]: 2024-04-16T17:19:20.746+0000 7fae88800700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-16T17:18:45.105720+0000 front 2024-04-16T17:18:45.105642+0000 (oldest deadline 2024-04-16T17:19:09.805838+0000) 2024-04-16T17:19:21.227 INFO:journalctl@ceph.osd.6.smithi045.stdout:Apr 16 17:19:20 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-6[59926]: 2024-04-16T17:19:20.746+0000 7fae88800700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-16T17:18:52.708348+0000 front 2024-04-16T17:18:52.707998+0000 (oldest deadline 2024-04-16T17:19:15.007885+0000) 2024-04-16T17:19:21.233 INFO:journalctl@ceph.osd.5.smithi117.stdout:Apr 16 17:19:20 smithi117 podman[74898]: 2024-04-16 17:19:20.929335353 +0000 UTC m=+5.677478280 container died 5b060c3476ff95b088241a1ae7c807f06423c895454763406a4fbcd84a8d0a42 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-5, GIT_BRANCH=HEAD, org.label-schema.build-date=20240409, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-04-16T17:19:21.234 INFO:journalctl@ceph.osd.8.smithi117.stdout:Apr 16 17:19:20 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-8[55817]: 2024-04-16T17:19:20.854+0000 7f65ab398700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-16T17:18:41.860244+0000 front 2024-04-16T17:18:41.860176+0000 (oldest deadline 2024-04-16T17:19:07.760349+0000) 2024-04-16T17:19:21.234 INFO:journalctl@ceph.osd.8.smithi117.stdout:Apr 16 17:19:20 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-8[55817]: 2024-04-16T17:19:20.854+0000 7f65ab398700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-16T17:18:52.461265+0000 front 2024-04-16T17:18:52.461355+0000 (oldest deadline 2024-04-16T17:19:20.662233+0000) 2024-04-16T17:19:21.504 INFO:journalctl@ceph.osd.5.smithi117.stdout:Apr 16 17:19:21 smithi117 podman[74898]: 2024-04-16 17:19:21.334300117 +0000 UTC m=+6.082443037 container cleanup 5b060c3476ff95b088241a1ae7c807f06423c895454763406a4fbcd84a8d0a42 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240409, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-04-16T17:19:21.504 INFO:journalctl@ceph.osd.5.smithi117.stdout:Apr 16 17:19:21 smithi117 bash[74898]: ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-5 2024-04-16T17:19:21.559 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:19:21 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:19:21.206+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-16T17:18:36.158791+0000 front 2024-04-16T17:18:36.158992+0000 (oldest deadline 2024-04-16T17:19:00.258603+0000) 2024-04-16T17:19:21.560 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:19:21 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:19:21.206+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-16T17:18:44.260046+0000 front 2024-04-16T17:18:44.260171+0000 (oldest deadline 2024-04-16T17:19:07.160168+0000) 2024-04-16T17:19:21.560 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:19:21 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:19:21.206+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-16T17:18:54.062069+0000 front 2024-04-16T17:18:54.061971+0000 (oldest deadline 2024-04-16T17:19:18.761808+0000) 2024-04-16T17:19:21.727 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 16 17:19:21 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-9[66545]: 2024-04-16T17:19:21.313+0000 7f5937600700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-16T17:18:36.968505+0000 front 2024-04-16T17:18:36.968670+0000 (oldest deadline 2024-04-16T17:19:02.268762+0000) 2024-04-16T17:19:21.727 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 16 17:19:21 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-9[66545]: 2024-04-16T17:19:21.313+0000 7f5937600700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-16T17:18:43.371969+0000 front 2024-04-16T17:18:43.371904+0000 (oldest deadline 2024-04-16T17:19:08.671642+0000) 2024-04-16T17:19:21.727 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 16 17:19:21 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-9[66545]: 2024-04-16T17:19:21.313+0000 7f5937600700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-16T17:18:53.373968+0000 front 2024-04-16T17:18:53.374043+0000 (oldest deadline 2024-04-16T17:19:15.673755+0000) 2024-04-16T17:19:21.804 INFO:journalctl@ceph.osd.5.smithi117.stdout:Apr 16 17:19:21 smithi117 podman[74928]: 2024-04-16 17:19:21.585856828 +0000 UTC m=+0.659329978 container remove 5b060c3476ff95b088241a1ae7c807f06423c895454763406a4fbcd84a8d0a42 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-5, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240409, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-04-16T17:19:21.804 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:19:21 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:19:21.504+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-16T17:18:34.374061+0000 front 2024-04-16T17:18:34.374275+0000 (oldest deadline 2024-04-16T17:18:57.873879+0000) 2024-04-16T17:19:21.804 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:19:21 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:19:21.504+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-16T17:18:50.676854+0000 front 2024-04-16T17:18:50.676874+0000 (oldest deadline 2024-04-16T17:19:15.977004+0000) 2024-04-16T17:19:21.926 INFO:journalctl@ceph.osd.7.smithi104.stdout:Apr 16 17:19:21 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-7[55033]: 2024-04-16T17:19:21.560+0000 7f08b34d5700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-16T17:18:42.259445+0000 front 2024-04-16T17:18:42.259491+0000 (oldest deadline 2024-04-16T17:19:08.159569+0000) 2024-04-16T17:19:21.927 INFO:journalctl@ceph.osd.7.smithi104.stdout:Apr 16 17:19:21 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-7[55033]: 2024-04-16T17:19:21.560+0000 7f08b34d5700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-16T17:18:50.460516+0000 front 2024-04-16T17:18:50.460605+0000 (oldest deadline 2024-04-16T17:19:16.360615+0000) 2024-04-16T17:19:22.180 INFO:journalctl@ceph.osd.8.smithi117.stdout:Apr 16 17:19:21 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-8[55817]: 2024-04-16T17:19:21.872+0000 7f65ab398700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-16T17:18:41.860244+0000 front 2024-04-16T17:18:41.860176+0000 (oldest deadline 2024-04-16T17:19:07.760349+0000) 2024-04-16T17:19:22.181 INFO:journalctl@ceph.osd.8.smithi117.stdout:Apr 16 17:19:21 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-8[55817]: 2024-04-16T17:19:21.872+0000 7f65ab398700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-16T17:18:52.461265+0000 front 2024-04-16T17:18:52.461355+0000 (oldest deadline 2024-04-16T17:19:20.662233+0000) 2024-04-16T17:19:22.181 INFO:journalctl@ceph.osd.5.smithi117.stdout:Apr 16 17:19:21 smithi117 podman[75024]: 2024-04-16 17:19:21.844057171 +0000 UTC m=+0.017852705 image pull e5ce10bdffbeeffffd1125f67b369e2151f0f17cdec854e93b6adf9321f65907 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2 2024-04-16T17:19:22.181 INFO:journalctl@ceph.osd.5.smithi117.stdout:Apr 16 17:19:21 smithi117 podman[75024]: 2024-04-16 17:19:21.953610061 +0000 UTC m=+0.127405586 container create fdf0e7d127499010a1a6db849351569b239ec6998d95f6139bf83e43957a239b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-5-deactivate, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.build-date=20240409, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.license=GPLv2) 2024-04-16T17:19:22.226 INFO:journalctl@ceph.osd.6.smithi045.stdout:Apr 16 17:19:21 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-6[59926]: 2024-04-16T17:19:21.728+0000 7fae88800700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-16T17:18:45.105720+0000 front 2024-04-16T17:18:45.105642+0000 (oldest deadline 2024-04-16T17:19:09.805838+0000) 2024-04-16T17:19:22.227 INFO:journalctl@ceph.osd.6.smithi045.stdout:Apr 16 17:19:21 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-6[59926]: 2024-04-16T17:19:21.728+0000 7fae88800700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-16T17:18:52.708348+0000 front 2024-04-16T17:18:52.707998+0000 (oldest deadline 2024-04-16T17:19:15.007885+0000) 2024-04-16T17:19:22.475 INFO:journalctl@ceph.osd.5.smithi117.stdout:Apr 16 17:19:22 smithi117 podman[75024]: 2024-04-16 17:19:22.181519454 +0000 UTC m=+0.355314988 container init fdf0e7d127499010a1a6db849351569b239ec6998d95f6139bf83e43957a239b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-5-deactivate, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.build-date=20240409, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-04-16T17:19:22.475 INFO:journalctl@ceph.osd.5.smithi117.stdout:Apr 16 17:19:22 smithi117 podman[75024]: 2024-04-16 17:19:22.18544291 +0000 UTC m=+0.359238432 container start fdf0e7d127499010a1a6db849351569b239ec6998d95f6139bf83e43957a239b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-5-deactivate, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240409, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-04-16T17:19:22.475 INFO:journalctl@ceph.osd.5.smithi117.stdout:Apr 16 17:19:22 smithi117 podman[75024]: 2024-04-16 17:19:22.281288048 +0000 UTC m=+0.455083582 container attach fdf0e7d127499010a1a6db849351569b239ec6998d95f6139bf83e43957a239b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-5-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240409, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True) 2024-04-16T17:19:22.523 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:19:22 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:19:22.197+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-16T17:18:36.158791+0000 front 2024-04-16T17:18:36.158992+0000 (oldest deadline 2024-04-16T17:19:00.258603+0000) 2024-04-16T17:19:22.523 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:19:22 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:19:22.197+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-16T17:18:44.260046+0000 front 2024-04-16T17:18:44.260171+0000 (oldest deadline 2024-04-16T17:19:07.160168+0000) 2024-04-16T17:19:22.523 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:19:22 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:19:22.197+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-16T17:18:54.062069+0000 front 2024-04-16T17:18:54.061971+0000 (oldest deadline 2024-04-16T17:19:18.761808+0000) 2024-04-16T17:19:22.698 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 16 17:19:22 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-9[66545]: 2024-04-16T17:19:22.361+0000 7f5937600700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-16T17:18:36.968505+0000 front 2024-04-16T17:18:36.968670+0000 (oldest deadline 2024-04-16T17:19:02.268762+0000) 2024-04-16T17:19:22.698 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 16 17:19:22 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-9[66545]: 2024-04-16T17:19:22.361+0000 7f5937600700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-16T17:18:43.371969+0000 front 2024-04-16T17:18:43.371904+0000 (oldest deadline 2024-04-16T17:19:08.671642+0000) 2024-04-16T17:19:22.699 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 16 17:19:22 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-9[66545]: 2024-04-16T17:19:22.361+0000 7f5937600700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-16T17:18:53.373968+0000 front 2024-04-16T17:18:53.374043+0000 (oldest deadline 2024-04-16T17:19:15.673755+0000) 2024-04-16T17:19:22.749 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:19:22 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:19:22.549+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-16T17:18:34.374061+0000 front 2024-04-16T17:18:34.374275+0000 (oldest deadline 2024-04-16T17:18:57.873879+0000) 2024-04-16T17:19:22.750 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:19:22 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:19:22.549+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-16T17:18:50.676854+0000 front 2024-04-16T17:18:50.676874+0000 (oldest deadline 2024-04-16T17:19:15.977004+0000) 2024-04-16T17:19:22.750 INFO:journalctl@ceph.osd.5.smithi117.stdout:Apr 16 17:19:22 smithi117 podman[75024]: 2024-04-16 17:19:22.476087338 +0000 UTC m=+0.649882870 container died fdf0e7d127499010a1a6db849351569b239ec6998d95f6139bf83e43957a239b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-5-deactivate, io.buildah.version=1.33.5, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240409, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-04-16T17:19:22.926 INFO:journalctl@ceph.osd.7.smithi104.stdout:Apr 16 17:19:22 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-7[55033]: 2024-04-16T17:19:22.524+0000 7f08b34d5700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-16T17:18:42.259445+0000 front 2024-04-16T17:18:42.259491+0000 (oldest deadline 2024-04-16T17:19:08.159569+0000) 2024-04-16T17:19:22.927 INFO:journalctl@ceph.osd.7.smithi104.stdout:Apr 16 17:19:22 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-7[55033]: 2024-04-16T17:19:22.524+0000 7f08b34d5700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-16T17:18:50.460516+0000 front 2024-04-16T17:18:50.460605+0000 (oldest deadline 2024-04-16T17:19:16.360615+0000) 2024-04-16T17:19:22.976 INFO:journalctl@ceph.osd.6.smithi045.stdout:Apr 16 17:19:22 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-6[59926]: 2024-04-16T17:19:22.698+0000 7fae88800700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-16T17:18:45.105720+0000 front 2024-04-16T17:18:45.105642+0000 (oldest deadline 2024-04-16T17:19:09.805838+0000) 2024-04-16T17:19:22.977 INFO:journalctl@ceph.osd.6.smithi045.stdout:Apr 16 17:19:22 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-6[59926]: 2024-04-16T17:19:22.698+0000 7fae88800700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-16T17:18:52.708348+0000 front 2024-04-16T17:18:52.707998+0000 (oldest deadline 2024-04-16T17:19:15.007885+0000) 2024-04-16T17:19:23.025 INFO:journalctl@ceph.osd.8.smithi117.stdout:Apr 16 17:19:22 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-8[55817]: 2024-04-16T17:19:22.899+0000 7f65ab398700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-16T17:18:41.860244+0000 front 2024-04-16T17:18:41.860176+0000 (oldest deadline 2024-04-16T17:19:07.760349+0000) 2024-04-16T17:19:23.025 INFO:journalctl@ceph.osd.8.smithi117.stdout:Apr 16 17:19:22 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-8[55817]: 2024-04-16T17:19:22.899+0000 7f65ab398700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-16T17:18:52.461265+0000 front 2024-04-16T17:18:52.461355+0000 (oldest deadline 2024-04-16T17:19:20.662233+0000) 2024-04-16T17:19:23.059 DEBUG:teuthology.orchestra.run.smithi117:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b1fecd38-fc11-11ee-bc90-c7b262605968@osd.5.service' 2024-04-16T17:19:23.099 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-16T17:19:23.100 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-04-16T17:19:23.100 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-04-16T17:19:23.100 DEBUG:teuthology.orchestra.run.smithi045:> sudo systemctl stop ceph-b1fecd38-fc11-11ee-bc90-c7b262605968@osd.6 2024-04-16T17:19:23.477 INFO:journalctl@ceph.osd.6.smithi045.stdout:Apr 16 17:19:23 smithi045 systemd[1]: Stopping Ceph osd.6 for b1fecd38-fc11-11ee-bc90-c7b262605968... 2024-04-16T17:19:23.477 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 16 17:19:23 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-9[66545]: 2024-04-16T17:19:23.367+0000 7f5937600700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-16T17:18:36.968505+0000 front 2024-04-16T17:18:36.968670+0000 (oldest deadline 2024-04-16T17:19:02.268762+0000) 2024-04-16T17:19:23.477 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 16 17:19:23 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-9[66545]: 2024-04-16T17:19:23.367+0000 7f5937600700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-16T17:18:43.371969+0000 front 2024-04-16T17:18:43.371904+0000 (oldest deadline 2024-04-16T17:19:08.671642+0000) 2024-04-16T17:19:23.477 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 16 17:19:23 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-9[66545]: 2024-04-16T17:19:23.367+0000 7f5937600700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-16T17:18:53.373968+0000 front 2024-04-16T17:18:53.374043+0000 (oldest deadline 2024-04-16T17:19:15.673755+0000) 2024-04-16T17:19:23.506 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:19:23 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:19:23.229+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-16T17:18:36.158791+0000 front 2024-04-16T17:18:36.158992+0000 (oldest deadline 2024-04-16T17:19:00.258603+0000) 2024-04-16T17:19:23.506 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:19:23 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:19:23.229+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-16T17:18:44.260046+0000 front 2024-04-16T17:18:44.260171+0000 (oldest deadline 2024-04-16T17:19:07.160168+0000) 2024-04-16T17:19:23.506 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:19:23 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:19:23.229+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-16T17:18:54.062069+0000 front 2024-04-16T17:18:54.061971+0000 (oldest deadline 2024-04-16T17:19:18.761808+0000) 2024-04-16T17:19:23.804 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:19:23 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:19:23.499+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-16T17:18:34.374061+0000 front 2024-04-16T17:18:34.374275+0000 (oldest deadline 2024-04-16T17:18:57.873879+0000) 2024-04-16T17:19:23.804 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:19:23 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:19:23.499+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-16T17:18:50.676854+0000 front 2024-04-16T17:18:50.676874+0000 (oldest deadline 2024-04-16T17:19:15.977004+0000) 2024-04-16T17:19:23.926 INFO:journalctl@ceph.osd.7.smithi104.stdout:Apr 16 17:19:23 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-7[55033]: 2024-04-16T17:19:23.506+0000 7f08b34d5700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-16T17:18:42.259445+0000 front 2024-04-16T17:18:42.259491+0000 (oldest deadline 2024-04-16T17:19:08.159569+0000) 2024-04-16T17:19:23.926 INFO:journalctl@ceph.osd.7.smithi104.stdout:Apr 16 17:19:23 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-7[55033]: 2024-04-16T17:19:23.506+0000 7f08b34d5700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-16T17:18:50.460516+0000 front 2024-04-16T17:18:50.460605+0000 (oldest deadline 2024-04-16T17:19:16.360615+0000) 2024-04-16T17:19:23.977 INFO:journalctl@ceph.osd.6.smithi045.stdout:Apr 16 17:19:23 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-6[59926]: 2024-04-16T17:19:23.653+0000 7fae88800700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-16T17:18:45.105720+0000 front 2024-04-16T17:18:45.105642+0000 (oldest deadline 2024-04-16T17:19:09.805838+0000) 2024-04-16T17:19:23.977 INFO:journalctl@ceph.osd.6.smithi045.stdout:Apr 16 17:19:23 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-6[59926]: 2024-04-16T17:19:23.653+0000 7fae88800700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-16T17:18:52.708348+0000 front 2024-04-16T17:18:52.707998+0000 (oldest deadline 2024-04-16T17:19:15.007885+0000) 2024-04-16T17:19:23.977 INFO:journalctl@ceph.osd.6.smithi045.stdout:Apr 16 17:19:23 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-6[59926]: 2024-04-16T17:19:23.751+0000 7fae92000700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-16T17:19:23.977 INFO:journalctl@ceph.osd.6.smithi045.stdout:Apr 16 17:19:23 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-6[59926]: 2024-04-16T17:19:23.751+0000 7fae92000700 -1 osd.6 112 *** Got signal Terminated *** 2024-04-16T17:19:23.977 INFO:journalctl@ceph.osd.6.smithi045.stdout:Apr 16 17:19:23 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-6[59926]: 2024-04-16T17:19:23.751+0000 7fae92000700 -1 osd.6 112 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-16T17:19:24.304 INFO:journalctl@ceph.osd.8.smithi117.stdout:Apr 16 17:19:23 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-8[55817]: 2024-04-16T17:19:23.923+0000 7f65ab398700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-16T17:18:41.860244+0000 front 2024-04-16T17:18:41.860176+0000 (oldest deadline 2024-04-16T17:19:07.760349+0000) 2024-04-16T17:19:24.304 INFO:journalctl@ceph.osd.8.smithi117.stdout:Apr 16 17:19:23 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-8[55817]: 2024-04-16T17:19:23.923+0000 7f65ab398700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-16T17:18:52.461265+0000 front 2024-04-16T17:18:52.461355+0000 (oldest deadline 2024-04-16T17:19:20.662233+0000) 2024-04-16T17:19:24.514 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:19:24 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:19:24.195+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-16T17:18:36.158791+0000 front 2024-04-16T17:18:36.158992+0000 (oldest deadline 2024-04-16T17:19:00.258603+0000) 2024-04-16T17:19:24.515 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:19:24 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:19:24.195+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-16T17:18:44.260046+0000 front 2024-04-16T17:18:44.260171+0000 (oldest deadline 2024-04-16T17:19:07.160168+0000) 2024-04-16T17:19:24.515 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:19:24 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:19:24.195+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-16T17:18:54.062069+0000 front 2024-04-16T17:18:54.061971+0000 (oldest deadline 2024-04-16T17:19:18.761808+0000) 2024-04-16T17:19:24.676 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 16 17:19:24 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-9[66545]: 2024-04-16T17:19:24.372+0000 7f5937600700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-16T17:18:36.968505+0000 front 2024-04-16T17:18:36.968670+0000 (oldest deadline 2024-04-16T17:19:02.268762+0000) 2024-04-16T17:19:24.676 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 16 17:19:24 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-9[66545]: 2024-04-16T17:19:24.372+0000 7f5937600700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-16T17:18:43.371969+0000 front 2024-04-16T17:18:43.371904+0000 (oldest deadline 2024-04-16T17:19:08.671642+0000) 2024-04-16T17:19:24.676 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 16 17:19:24 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-9[66545]: 2024-04-16T17:19:24.372+0000 7f5937600700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-16T17:18:53.373968+0000 front 2024-04-16T17:18:53.374043+0000 (oldest deadline 2024-04-16T17:19:15.673755+0000) 2024-04-16T17:19:24.804 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:19:24 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:19:24.509+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-16T17:18:34.374061+0000 front 2024-04-16T17:18:34.374275+0000 (oldest deadline 2024-04-16T17:18:57.873879+0000) 2024-04-16T17:19:24.804 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:19:24 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:19:24.509+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-16T17:18:50.676854+0000 front 2024-04-16T17:18:50.676874+0000 (oldest deadline 2024-04-16T17:19:15.977004+0000) 2024-04-16T17:19:24.926 INFO:journalctl@ceph.osd.7.smithi104.stdout:Apr 16 17:19:24 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-7[55033]: 2024-04-16T17:19:24.515+0000 7f08b34d5700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-16T17:18:42.259445+0000 front 2024-04-16T17:18:42.259491+0000 (oldest deadline 2024-04-16T17:19:08.159569+0000) 2024-04-16T17:19:24.927 INFO:journalctl@ceph.osd.7.smithi104.stdout:Apr 16 17:19:24 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-7[55033]: 2024-04-16T17:19:24.515+0000 7f08b34d5700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-16T17:18:50.460516+0000 front 2024-04-16T17:18:50.460605+0000 (oldest deadline 2024-04-16T17:19:16.360615+0000) 2024-04-16T17:19:24.976 INFO:journalctl@ceph.osd.6.smithi045.stdout:Apr 16 17:19:24 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-6[59926]: 2024-04-16T17:19:24.675+0000 7fae88800700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-16T17:18:45.105720+0000 front 2024-04-16T17:18:45.105642+0000 (oldest deadline 2024-04-16T17:19:09.805838+0000) 2024-04-16T17:19:24.987 INFO:journalctl@ceph.osd.6.smithi045.stdout:Apr 16 17:19:24 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-6[59926]: 2024-04-16T17:19:24.675+0000 7fae88800700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-16T17:18:52.708348+0000 front 2024-04-16T17:18:52.707998+0000 (oldest deadline 2024-04-16T17:19:15.007885+0000) 2024-04-16T17:19:24.987 INFO:journalctl@ceph.osd.6.smithi045.stdout:Apr 16 17:19:24 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-6[59926]: 2024-04-16T17:19:24.675+0000 7fae88800700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-16T17:18:58.509715+0000 front 2024-04-16T17:18:58.509773+0000 (oldest deadline 2024-04-16T17:19:23.809929+0000) 2024-04-16T17:19:25.304 INFO:journalctl@ceph.osd.8.smithi117.stdout:Apr 16 17:19:24 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-8[55817]: 2024-04-16T17:19:24.875+0000 7f65ab398700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-16T17:18:41.860244+0000 front 2024-04-16T17:18:41.860176+0000 (oldest deadline 2024-04-16T17:19:07.760349+0000) 2024-04-16T17:19:25.305 INFO:journalctl@ceph.osd.8.smithi117.stdout:Apr 16 17:19:24 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-8[55817]: 2024-04-16T17:19:24.875+0000 7f65ab398700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-16T17:18:52.461265+0000 front 2024-04-16T17:18:52.461355+0000 (oldest deadline 2024-04-16T17:19:20.662233+0000) 2024-04-16T17:19:25.305 INFO:journalctl@ceph.osd.8.smithi117.stdout:Apr 16 17:19:24 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-8[55817]: 2024-04-16T17:19:24.875+0000 7f65ab398700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-16T17:19:00.663078+0000 front 2024-04-16T17:19:00.662927+0000 (oldest deadline 2024-04-16T17:19:24.762742+0000) 2024-04-16T17:19:25.427 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:19:25 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:19:25.159+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-16T17:18:36.158791+0000 front 2024-04-16T17:18:36.158992+0000 (oldest deadline 2024-04-16T17:19:00.258603+0000) 2024-04-16T17:19:25.427 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:19:25 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:19:25.159+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-16T17:18:44.260046+0000 front 2024-04-16T17:18:44.260171+0000 (oldest deadline 2024-04-16T17:19:07.160168+0000) 2024-04-16T17:19:25.427 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:19:25 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:19:25.159+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-16T17:18:54.062069+0000 front 2024-04-16T17:18:54.061971+0000 (oldest deadline 2024-04-16T17:19:18.761808+0000) 2024-04-16T17:19:25.427 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:19:25 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:19:25.159+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-16T17:19:01.063265+0000 front 2024-04-16T17:19:01.063419+0000 (oldest deadline 2024-04-16T17:19:24.563191+0000) 2024-04-16T17:19:25.644 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 16 17:19:25 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-9[66545]: 2024-04-16T17:19:25.377+0000 7f5937600700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-16T17:18:36.968505+0000 front 2024-04-16T17:18:36.968670+0000 (oldest deadline 2024-04-16T17:19:02.268762+0000) 2024-04-16T17:19:25.644 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 16 17:19:25 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-9[66545]: 2024-04-16T17:19:25.377+0000 7f5937600700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-16T17:18:43.371969+0000 front 2024-04-16T17:18:43.371904+0000 (oldest deadline 2024-04-16T17:19:08.671642+0000) 2024-04-16T17:19:25.644 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 16 17:19:25 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-9[66545]: 2024-04-16T17:19:25.377+0000 7f5937600700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-16T17:18:53.373968+0000 front 2024-04-16T17:18:53.374043+0000 (oldest deadline 2024-04-16T17:19:15.673755+0000) 2024-04-16T17:19:25.804 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:19:25 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:19:25.512+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-16T17:18:34.374061+0000 front 2024-04-16T17:18:34.374275+0000 (oldest deadline 2024-04-16T17:18:57.873879+0000) 2024-04-16T17:19:25.804 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:19:25 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:19:25.512+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-16T17:18:50.676854+0000 front 2024-04-16T17:18:50.676874+0000 (oldest deadline 2024-04-16T17:19:15.977004+0000) 2024-04-16T17:19:25.927 INFO:journalctl@ceph.osd.7.smithi104.stdout:Apr 16 17:19:25 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-7[55033]: 2024-04-16T17:19:25.478+0000 7f08b34d5700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-16T17:18:42.259445+0000 front 2024-04-16T17:18:42.259491+0000 (oldest deadline 2024-04-16T17:19:08.159569+0000) 2024-04-16T17:19:25.927 INFO:journalctl@ceph.osd.7.smithi104.stdout:Apr 16 17:19:25 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-7[55033]: 2024-04-16T17:19:25.478+0000 7f08b34d5700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-16T17:18:50.460516+0000 front 2024-04-16T17:18:50.460605+0000 (oldest deadline 2024-04-16T17:19:16.360615+0000) 2024-04-16T17:19:25.977 INFO:journalctl@ceph.osd.6.smithi045.stdout:Apr 16 17:19:25 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-6[59926]: 2024-04-16T17:19:25.643+0000 7fae88800700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-16T17:18:45.105720+0000 front 2024-04-16T17:18:45.105642+0000 (oldest deadline 2024-04-16T17:19:09.805838+0000) 2024-04-16T17:19:25.977 INFO:journalctl@ceph.osd.6.smithi045.stdout:Apr 16 17:19:25 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-6[59926]: 2024-04-16T17:19:25.643+0000 7fae88800700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-16T17:18:52.708348+0000 front 2024-04-16T17:18:52.707998+0000 (oldest deadline 2024-04-16T17:19:15.007885+0000) 2024-04-16T17:19:25.977 INFO:journalctl@ceph.osd.6.smithi045.stdout:Apr 16 17:19:25 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-6[59926]: 2024-04-16T17:19:25.643+0000 7fae88800700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-16T17:18:58.509715+0000 front 2024-04-16T17:18:58.509773+0000 (oldest deadline 2024-04-16T17:19:23.809929+0000) 2024-04-16T17:19:26.304 INFO:journalctl@ceph.osd.8.smithi117.stdout:Apr 16 17:19:25 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-8[55817]: 2024-04-16T17:19:25.876+0000 7f65ab398700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-16T17:18:41.860244+0000 front 2024-04-16T17:18:41.860176+0000 (oldest deadline 2024-04-16T17:19:07.760349+0000) 2024-04-16T17:19:26.304 INFO:journalctl@ceph.osd.8.smithi117.stdout:Apr 16 17:19:25 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-8[55817]: 2024-04-16T17:19:25.876+0000 7f65ab398700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-16T17:18:52.461265+0000 front 2024-04-16T17:18:52.461355+0000 (oldest deadline 2024-04-16T17:19:20.662233+0000) 2024-04-16T17:19:26.305 INFO:journalctl@ceph.osd.8.smithi117.stdout:Apr 16 17:19:25 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-8[55817]: 2024-04-16T17:19:25.876+0000 7f65ab398700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-16T17:19:00.663078+0000 front 2024-04-16T17:19:00.662927+0000 (oldest deadline 2024-04-16T17:19:24.762742+0000) 2024-04-16T17:19:26.506 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:19:26 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:19:26.180+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-16T17:18:36.158791+0000 front 2024-04-16T17:18:36.158992+0000 (oldest deadline 2024-04-16T17:19:00.258603+0000) 2024-04-16T17:19:26.506 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:19:26 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:19:26.180+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-16T17:18:44.260046+0000 front 2024-04-16T17:18:44.260171+0000 (oldest deadline 2024-04-16T17:19:07.160168+0000) 2024-04-16T17:19:26.506 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:19:26 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:19:26.180+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-16T17:18:54.062069+0000 front 2024-04-16T17:18:54.061971+0000 (oldest deadline 2024-04-16T17:19:18.761808+0000) 2024-04-16T17:19:26.506 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:19:26 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:19:26.180+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-16T17:19:01.063265+0000 front 2024-04-16T17:19:01.063419+0000 (oldest deadline 2024-04-16T17:19:24.563191+0000) 2024-04-16T17:19:26.727 INFO:journalctl@ceph.osd.6.smithi045.stdout:Apr 16 17:19:26 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-6[59926]: 2024-04-16T17:19:26.645+0000 7fae88800700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-16T17:18:45.105720+0000 front 2024-04-16T17:18:45.105642+0000 (oldest deadline 2024-04-16T17:19:09.805838+0000) 2024-04-16T17:19:26.727 INFO:journalctl@ceph.osd.6.smithi045.stdout:Apr 16 17:19:26 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-6[59926]: 2024-04-16T17:19:26.645+0000 7fae88800700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-16T17:18:52.708348+0000 front 2024-04-16T17:18:52.707998+0000 (oldest deadline 2024-04-16T17:19:15.007885+0000) 2024-04-16T17:19:26.727 INFO:journalctl@ceph.osd.6.smithi045.stdout:Apr 16 17:19:26 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-6[59926]: 2024-04-16T17:19:26.645+0000 7fae88800700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-16T17:18:58.509715+0000 front 2024-04-16T17:18:58.509773+0000 (oldest deadline 2024-04-16T17:19:23.809929+0000) 2024-04-16T17:19:26.727 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 16 17:19:26 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-9[66545]: 2024-04-16T17:19:26.414+0000 7f5937600700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-16T17:18:36.968505+0000 front 2024-04-16T17:18:36.968670+0000 (oldest deadline 2024-04-16T17:19:02.268762+0000) 2024-04-16T17:19:26.728 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 16 17:19:26 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-9[66545]: 2024-04-16T17:19:26.414+0000 7f5937600700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-16T17:18:43.371969+0000 front 2024-04-16T17:18:43.371904+0000 (oldest deadline 2024-04-16T17:19:08.671642+0000) 2024-04-16T17:19:26.728 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 16 17:19:26 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-9[66545]: 2024-04-16T17:19:26.414+0000 7f5937600700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-16T17:18:53.373968+0000 front 2024-04-16T17:18:53.374043+0000 (oldest deadline 2024-04-16T17:19:15.673755+0000) 2024-04-16T17:19:26.804 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:19:26 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:19:26.532+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-16T17:18:34.374061+0000 front 2024-04-16T17:18:34.374275+0000 (oldest deadline 2024-04-16T17:18:57.873879+0000) 2024-04-16T17:19:26.804 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:19:26 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:19:26.532+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-16T17:18:50.676854+0000 front 2024-04-16T17:18:50.676874+0000 (oldest deadline 2024-04-16T17:19:15.977004+0000) 2024-04-16T17:19:26.927 INFO:journalctl@ceph.osd.7.smithi104.stdout:Apr 16 17:19:26 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-7[55033]: 2024-04-16T17:19:26.506+0000 7f08b34d5700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-16T17:18:42.259445+0000 front 2024-04-16T17:18:42.259491+0000 (oldest deadline 2024-04-16T17:19:08.159569+0000) 2024-04-16T17:19:26.927 INFO:journalctl@ceph.osd.7.smithi104.stdout:Apr 16 17:19:26 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-7[55033]: 2024-04-16T17:19:26.506+0000 7f08b34d5700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-16T17:18:50.460516+0000 front 2024-04-16T17:18:50.460605+0000 (oldest deadline 2024-04-16T17:19:16.360615+0000) 2024-04-16T17:19:27.305 INFO:journalctl@ceph.osd.8.smithi117.stdout:Apr 16 17:19:26 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-8[55817]: 2024-04-16T17:19:26.882+0000 7f65ab398700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-16T17:18:41.860244+0000 front 2024-04-16T17:18:41.860176+0000 (oldest deadline 2024-04-16T17:19:07.760349+0000) 2024-04-16T17:19:27.305 INFO:journalctl@ceph.osd.8.smithi117.stdout:Apr 16 17:19:26 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-8[55817]: 2024-04-16T17:19:26.882+0000 7f65ab398700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-16T17:18:52.461265+0000 front 2024-04-16T17:18:52.461355+0000 (oldest deadline 2024-04-16T17:19:20.662233+0000) 2024-04-16T17:19:27.305 INFO:journalctl@ceph.osd.8.smithi117.stdout:Apr 16 17:19:26 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-8[55817]: 2024-04-16T17:19:26.882+0000 7f65ab398700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-16T17:19:00.663078+0000 front 2024-04-16T17:19:00.662927+0000 (oldest deadline 2024-04-16T17:19:24.762742+0000) 2024-04-16T17:19:27.427 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:19:27 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:19:27.154+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-16T17:18:36.158791+0000 front 2024-04-16T17:18:36.158992+0000 (oldest deadline 2024-04-16T17:19:00.258603+0000) 2024-04-16T17:19:27.427 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:19:27 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:19:27.154+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-16T17:18:44.260046+0000 front 2024-04-16T17:18:44.260171+0000 (oldest deadline 2024-04-16T17:19:07.160168+0000) 2024-04-16T17:19:27.427 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:19:27 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:19:27.154+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-16T17:18:54.062069+0000 front 2024-04-16T17:18:54.061971+0000 (oldest deadline 2024-04-16T17:19:18.761808+0000) 2024-04-16T17:19:27.427 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:19:27 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:19:27.154+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-16T17:19:01.063265+0000 front 2024-04-16T17:19:01.063419+0000 (oldest deadline 2024-04-16T17:19:24.563191+0000) 2024-04-16T17:19:27.689 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 16 17:19:27 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-9[66545]: 2024-04-16T17:19:27.369+0000 7f5937600700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-16T17:18:36.968505+0000 front 2024-04-16T17:18:36.968670+0000 (oldest deadline 2024-04-16T17:19:02.268762+0000) 2024-04-16T17:19:27.689 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 16 17:19:27 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-9[66545]: 2024-04-16T17:19:27.369+0000 7f5937600700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-16T17:18:43.371969+0000 front 2024-04-16T17:18:43.371904+0000 (oldest deadline 2024-04-16T17:19:08.671642+0000) 2024-04-16T17:19:27.689 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 16 17:19:27 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-9[66545]: 2024-04-16T17:19:27.369+0000 7f5937600700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-16T17:18:53.373968+0000 front 2024-04-16T17:18:53.374043+0000 (oldest deadline 2024-04-16T17:19:15.673755+0000) 2024-04-16T17:19:27.689 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 16 17:19:27 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-9[66545]: 2024-04-16T17:19:27.369+0000 7f5937600700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-16T17:19:01.880446+0000 front 2024-04-16T17:19:01.880936+0000 (oldest deadline 2024-04-16T17:19:27.180477+0000) 2024-04-16T17:19:27.868 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:19:27 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:19:27.574+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-16T17:18:34.374061+0000 front 2024-04-16T17:18:34.374275+0000 (oldest deadline 2024-04-16T17:18:57.873879+0000) 2024-04-16T17:19:27.869 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:19:27 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:19:27.574+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-16T17:18:50.676854+0000 front 2024-04-16T17:18:50.676874+0000 (oldest deadline 2024-04-16T17:19:15.977004+0000) 2024-04-16T17:19:27.869 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:19:27 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:19:27.574+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-16T17:19:01.878146+0000 front 2024-04-16T17:19:01.878262+0000 (oldest deadline 2024-04-16T17:19:27.178048+0000) 2024-04-16T17:19:27.927 INFO:journalctl@ceph.osd.7.smithi104.stdout:Apr 16 17:19:27 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-7[55033]: 2024-04-16T17:19:27.477+0000 7f08b34d5700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-16T17:18:42.259445+0000 front 2024-04-16T17:18:42.259491+0000 (oldest deadline 2024-04-16T17:19:08.159569+0000) 2024-04-16T17:19:27.927 INFO:journalctl@ceph.osd.7.smithi104.stdout:Apr 16 17:19:27 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-7[55033]: 2024-04-16T17:19:27.477+0000 7f08b34d5700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-16T17:18:50.460516+0000 front 2024-04-16T17:18:50.460605+0000 (oldest deadline 2024-04-16T17:19:16.360615+0000) 2024-04-16T17:19:27.977 INFO:journalctl@ceph.osd.6.smithi045.stdout:Apr 16 17:19:27 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-6[59926]: 2024-04-16T17:19:27.688+0000 7fae88800700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-16T17:18:45.105720+0000 front 2024-04-16T17:18:45.105642+0000 (oldest deadline 2024-04-16T17:19:09.805838+0000) 2024-04-16T17:19:27.978 INFO:journalctl@ceph.osd.6.smithi045.stdout:Apr 16 17:19:27 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-6[59926]: 2024-04-16T17:19:27.688+0000 7fae88800700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-16T17:18:52.708348+0000 front 2024-04-16T17:18:52.707998+0000 (oldest deadline 2024-04-16T17:19:15.007885+0000) 2024-04-16T17:19:27.978 INFO:journalctl@ceph.osd.6.smithi045.stdout:Apr 16 17:19:27 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-6[59926]: 2024-04-16T17:19:27.688+0000 7fae88800700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-16T17:18:58.509715+0000 front 2024-04-16T17:18:58.509773+0000 (oldest deadline 2024-04-16T17:19:23.809929+0000) 2024-04-16T17:19:28.304 INFO:journalctl@ceph.osd.8.smithi117.stdout:Apr 16 17:19:27 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-8[55817]: 2024-04-16T17:19:27.869+0000 7f65ab398700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-16T17:18:41.860244+0000 front 2024-04-16T17:18:41.860176+0000 (oldest deadline 2024-04-16T17:19:07.760349+0000) 2024-04-16T17:19:28.304 INFO:journalctl@ceph.osd.8.smithi117.stdout:Apr 16 17:19:27 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-8[55817]: 2024-04-16T17:19:27.869+0000 7f65ab398700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-16T17:18:52.461265+0000 front 2024-04-16T17:18:52.461355+0000 (oldest deadline 2024-04-16T17:19:20.662233+0000) 2024-04-16T17:19:28.305 INFO:journalctl@ceph.osd.8.smithi117.stdout:Apr 16 17:19:27 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-8[55817]: 2024-04-16T17:19:27.869+0000 7f65ab398700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-16T17:19:00.663078+0000 front 2024-04-16T17:19:00.662927+0000 (oldest deadline 2024-04-16T17:19:24.762742+0000) 2024-04-16T17:19:28.427 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:19:28 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:19:28.162+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-16T17:18:36.158791+0000 front 2024-04-16T17:18:36.158992+0000 (oldest deadline 2024-04-16T17:19:00.258603+0000) 2024-04-16T17:19:28.427 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:19:28 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:19:28.162+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-16T17:18:44.260046+0000 front 2024-04-16T17:18:44.260171+0000 (oldest deadline 2024-04-16T17:19:07.160168+0000) 2024-04-16T17:19:28.427 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:19:28 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:19:28.162+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-16T17:18:54.062069+0000 front 2024-04-16T17:18:54.061971+0000 (oldest deadline 2024-04-16T17:19:18.761808+0000) 2024-04-16T17:19:28.428 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:19:28 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:19:28.162+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-16T17:19:01.063265+0000 front 2024-04-16T17:19:01.063419+0000 (oldest deadline 2024-04-16T17:19:24.563191+0000) 2024-04-16T17:19:28.712 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 16 17:19:28 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-9[66545]: 2024-04-16T17:19:28.352+0000 7f5937600700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-16T17:18:36.968505+0000 front 2024-04-16T17:18:36.968670+0000 (oldest deadline 2024-04-16T17:19:02.268762+0000) 2024-04-16T17:19:28.712 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 16 17:19:28 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-9[66545]: 2024-04-16T17:19:28.352+0000 7f5937600700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-16T17:18:43.371969+0000 front 2024-04-16T17:18:43.371904+0000 (oldest deadline 2024-04-16T17:19:08.671642+0000) 2024-04-16T17:19:28.712 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 16 17:19:28 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-9[66545]: 2024-04-16T17:19:28.352+0000 7f5937600700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-16T17:18:53.373968+0000 front 2024-04-16T17:18:53.374043+0000 (oldest deadline 2024-04-16T17:19:15.673755+0000) 2024-04-16T17:19:28.712 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 16 17:19:28 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-9[66545]: 2024-04-16T17:19:28.352+0000 7f5937600700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-16T17:19:01.880446+0000 front 2024-04-16T17:19:01.880936+0000 (oldest deadline 2024-04-16T17:19:27.180477+0000) 2024-04-16T17:19:28.804 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:19:28 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:19:28.551+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-16T17:18:34.374061+0000 front 2024-04-16T17:18:34.374275+0000 (oldest deadline 2024-04-16T17:18:57.873879+0000) 2024-04-16T17:19:28.804 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:19:28 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:19:28.551+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-16T17:18:50.676854+0000 front 2024-04-16T17:18:50.676874+0000 (oldest deadline 2024-04-16T17:19:15.977004+0000) 2024-04-16T17:19:28.804 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:19:28 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:19:28.551+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-16T17:19:01.878146+0000 front 2024-04-16T17:19:01.878262+0000 (oldest deadline 2024-04-16T17:19:27.178048+0000) 2024-04-16T17:19:28.927 INFO:journalctl@ceph.osd.7.smithi104.stdout:Apr 16 17:19:28 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-7[55033]: 2024-04-16T17:19:28.463+0000 7f08b34d5700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-16T17:18:42.259445+0000 front 2024-04-16T17:18:42.259491+0000 (oldest deadline 2024-04-16T17:19:08.159569+0000) 2024-04-16T17:19:28.927 INFO:journalctl@ceph.osd.7.smithi104.stdout:Apr 16 17:19:28 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-7[55033]: 2024-04-16T17:19:28.463+0000 7f08b34d5700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-16T17:18:50.460516+0000 front 2024-04-16T17:18:50.460605+0000 (oldest deadline 2024-04-16T17:19:16.360615+0000) 2024-04-16T17:19:28.927 INFO:journalctl@ceph.osd.7.smithi104.stdout:Apr 16 17:19:28 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-7[55033]: 2024-04-16T17:19:28.463+0000 7f08b34d5700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-16T17:19:02.662964+0000 front 2024-04-16T17:19:02.662819+0000 (oldest deadline 2024-04-16T17:19:27.863073+0000) 2024-04-16T17:19:28.977 INFO:journalctl@ceph.osd.6.smithi045.stdout:Apr 16 17:19:28 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-6[59926]: 2024-04-16T17:19:28.711+0000 7fae88800700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-16T17:18:45.105720+0000 front 2024-04-16T17:18:45.105642+0000 (oldest deadline 2024-04-16T17:19:09.805838+0000) 2024-04-16T17:19:28.977 INFO:journalctl@ceph.osd.6.smithi045.stdout:Apr 16 17:19:28 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-6[59926]: 2024-04-16T17:19:28.711+0000 7fae88800700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-16T17:18:52.708348+0000 front 2024-04-16T17:18:52.707998+0000 (oldest deadline 2024-04-16T17:19:15.007885+0000) 2024-04-16T17:19:28.977 INFO:journalctl@ceph.osd.6.smithi045.stdout:Apr 16 17:19:28 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-6[59926]: 2024-04-16T17:19:28.711+0000 7fae88800700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-16T17:18:58.509715+0000 front 2024-04-16T17:18:58.509773+0000 (oldest deadline 2024-04-16T17:19:23.809929+0000) 2024-04-16T17:19:29.304 INFO:journalctl@ceph.osd.8.smithi117.stdout:Apr 16 17:19:28 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-8[55817]: 2024-04-16T17:19:28.830+0000 7f65ab398700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-16T17:18:41.860244+0000 front 2024-04-16T17:18:41.860176+0000 (oldest deadline 2024-04-16T17:19:07.760349+0000) 2024-04-16T17:19:29.304 INFO:journalctl@ceph.osd.8.smithi117.stdout:Apr 16 17:19:28 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-8[55817]: 2024-04-16T17:19:28.830+0000 7f65ab398700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-16T17:18:52.461265+0000 front 2024-04-16T17:18:52.461355+0000 (oldest deadline 2024-04-16T17:19:20.662233+0000) 2024-04-16T17:19:29.305 INFO:journalctl@ceph.osd.8.smithi117.stdout:Apr 16 17:19:28 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-8[55817]: 2024-04-16T17:19:28.830+0000 7f65ab398700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-16T17:19:00.663078+0000 front 2024-04-16T17:19:00.662927+0000 (oldest deadline 2024-04-16T17:19:24.762742+0000) 2024-04-16T17:19:29.677 INFO:journalctl@ceph.osd.7.smithi104.stdout:Apr 16 17:19:29 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-7[55033]: 2024-04-16T17:19:29.438+0000 7f08b34d5700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-16T17:18:42.259445+0000 front 2024-04-16T17:18:42.259491+0000 (oldest deadline 2024-04-16T17:19:08.159569+0000) 2024-04-16T17:19:29.677 INFO:journalctl@ceph.osd.7.smithi104.stdout:Apr 16 17:19:29 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-7[55033]: 2024-04-16T17:19:29.438+0000 7f08b34d5700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-16T17:18:50.460516+0000 front 2024-04-16T17:18:50.460605+0000 (oldest deadline 2024-04-16T17:19:16.360615+0000) 2024-04-16T17:19:29.677 INFO:journalctl@ceph.osd.7.smithi104.stdout:Apr 16 17:19:29 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-7[55033]: 2024-04-16T17:19:29.438+0000 7f08b34d5700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-16T17:19:02.662964+0000 front 2024-04-16T17:19:02.662819+0000 (oldest deadline 2024-04-16T17:19:27.863073+0000) 2024-04-16T17:19:29.677 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:19:29 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:19:29.191+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-16T17:18:36.158791+0000 front 2024-04-16T17:18:36.158992+0000 (oldest deadline 2024-04-16T17:19:00.258603+0000) 2024-04-16T17:19:29.678 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:19:29 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:19:29.191+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-16T17:18:44.260046+0000 front 2024-04-16T17:18:44.260171+0000 (oldest deadline 2024-04-16T17:19:07.160168+0000) 2024-04-16T17:19:29.678 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:19:29 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:19:29.191+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-16T17:18:54.062069+0000 front 2024-04-16T17:18:54.061971+0000 (oldest deadline 2024-04-16T17:19:18.761808+0000) 2024-04-16T17:19:29.678 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:19:29 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:19:29.191+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-16T17:19:01.063265+0000 front 2024-04-16T17:19:01.063419+0000 (oldest deadline 2024-04-16T17:19:24.563191+0000) 2024-04-16T17:19:29.725 INFO:journalctl@ceph.osd.6.smithi045.stdout:Apr 16 17:19:29 smithi045 podman[104129]: 2024-04-16 17:19:29.27629707 +0000 UTC m=+5.683945642 container died 717404df8e8ce66dcee82eb831d5c6471dfc6a23416c0ac5eb456820d7a801c8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-6, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, org.label-schema.build-date=20240409) 2024-04-16T17:19:29.726 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 16 17:19:29 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-9[66545]: 2024-04-16T17:19:29.343+0000 7f5937600700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-16T17:18:36.968505+0000 front 2024-04-16T17:18:36.968670+0000 (oldest deadline 2024-04-16T17:19:02.268762+0000) 2024-04-16T17:19:29.726 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 16 17:19:29 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-9[66545]: 2024-04-16T17:19:29.343+0000 7f5937600700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-16T17:18:43.371969+0000 front 2024-04-16T17:18:43.371904+0000 (oldest deadline 2024-04-16T17:19:08.671642+0000) 2024-04-16T17:19:29.727 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 16 17:19:29 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-9[66545]: 2024-04-16T17:19:29.343+0000 7f5937600700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-16T17:18:53.373968+0000 front 2024-04-16T17:18:53.374043+0000 (oldest deadline 2024-04-16T17:19:15.673755+0000) 2024-04-16T17:19:29.727 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 16 17:19:29 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-9[66545]: 2024-04-16T17:19:29.343+0000 7f5937600700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-16T17:19:01.880446+0000 front 2024-04-16T17:19:01.880936+0000 (oldest deadline 2024-04-16T17:19:27.180477+0000) 2024-04-16T17:19:29.804 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:19:29 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:19:29.502+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-16T17:18:34.374061+0000 front 2024-04-16T17:18:34.374275+0000 (oldest deadline 2024-04-16T17:18:57.873879+0000) 2024-04-16T17:19:29.804 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:19:29 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:19:29.502+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-16T17:18:50.676854+0000 front 2024-04-16T17:18:50.676874+0000 (oldest deadline 2024-04-16T17:19:15.977004+0000) 2024-04-16T17:19:29.805 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:19:29 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:19:29.502+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-16T17:19:01.878146+0000 front 2024-04-16T17:19:01.878262+0000 (oldest deadline 2024-04-16T17:19:27.178048+0000) 2024-04-16T17:19:29.977 INFO:journalctl@ceph.osd.6.smithi045.stdout:Apr 16 17:19:29 smithi045 podman[104129]: 2024-04-16 17:19:29.894393628 +0000 UTC m=+6.302042197 container cleanup 717404df8e8ce66dcee82eb831d5c6471dfc6a23416c0ac5eb456820d7a801c8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-6, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240409, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-04-16T17:19:29.977 INFO:journalctl@ceph.osd.6.smithi045.stdout:Apr 16 17:19:29 smithi045 bash[104129]: ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-6 2024-04-16T17:19:30.304 INFO:journalctl@ceph.osd.8.smithi117.stdout:Apr 16 17:19:29 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-8[55817]: 2024-04-16T17:19:29.844+0000 7f65ab398700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-16T17:18:41.860244+0000 front 2024-04-16T17:18:41.860176+0000 (oldest deadline 2024-04-16T17:19:07.760349+0000) 2024-04-16T17:19:30.304 INFO:journalctl@ceph.osd.8.smithi117.stdout:Apr 16 17:19:29 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-8[55817]: 2024-04-16T17:19:29.844+0000 7f65ab398700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-16T17:18:52.461265+0000 front 2024-04-16T17:18:52.461355+0000 (oldest deadline 2024-04-16T17:19:20.662233+0000) 2024-04-16T17:19:30.305 INFO:journalctl@ceph.osd.8.smithi117.stdout:Apr 16 17:19:29 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-8[55817]: 2024-04-16T17:19:29.844+0000 7f65ab398700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-16T17:19:00.663078+0000 front 2024-04-16T17:19:00.662927+0000 (oldest deadline 2024-04-16T17:19:24.762742+0000) 2024-04-16T17:19:30.477 INFO:journalctl@ceph.osd.6.smithi045.stdout:Apr 16 17:19:30 smithi045 podman[104148]: 2024-04-16 17:19:30.112179386 +0000 UTC m=+0.819208203 container remove 717404df8e8ce66dcee82eb831d5c6471dfc6a23416c0ac5eb456820d7a801c8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-6, org.label-schema.license=GPLv2, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240409, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-04-16T17:19:30.477 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 16 17:19:30 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-9[66545]: 2024-04-16T17:19:30.338+0000 7f5937600700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-16T17:18:36.968505+0000 front 2024-04-16T17:18:36.968670+0000 (oldest deadline 2024-04-16T17:19:02.268762+0000) 2024-04-16T17:19:30.477 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 16 17:19:30 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-9[66545]: 2024-04-16T17:19:30.338+0000 7f5937600700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-16T17:18:43.371969+0000 front 2024-04-16T17:18:43.371904+0000 (oldest deadline 2024-04-16T17:19:08.671642+0000) 2024-04-16T17:19:30.477 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 16 17:19:30 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-9[66545]: 2024-04-16T17:19:30.338+0000 7f5937600700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-16T17:18:53.373968+0000 front 2024-04-16T17:18:53.374043+0000 (oldest deadline 2024-04-16T17:19:15.673755+0000) 2024-04-16T17:19:30.477 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 16 17:19:30 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-9[66545]: 2024-04-16T17:19:30.338+0000 7f5937600700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-16T17:19:01.880446+0000 front 2024-04-16T17:19:01.880936+0000 (oldest deadline 2024-04-16T17:19:27.180477+0000) 2024-04-16T17:19:30.677 INFO:journalctl@ceph.osd.7.smithi104.stdout:Apr 16 17:19:30 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-7[55033]: 2024-04-16T17:19:30.420+0000 7f08b34d5700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-16T17:18:42.259445+0000 front 2024-04-16T17:18:42.259491+0000 (oldest deadline 2024-04-16T17:19:08.159569+0000) 2024-04-16T17:19:30.677 INFO:journalctl@ceph.osd.7.smithi104.stdout:Apr 16 17:19:30 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-7[55033]: 2024-04-16T17:19:30.420+0000 7f08b34d5700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-16T17:18:50.460516+0000 front 2024-04-16T17:18:50.460605+0000 (oldest deadline 2024-04-16T17:19:16.360615+0000) 2024-04-16T17:19:30.677 INFO:journalctl@ceph.osd.7.smithi104.stdout:Apr 16 17:19:30 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-7[55033]: 2024-04-16T17:19:30.420+0000 7f08b34d5700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-16T17:19:02.662964+0000 front 2024-04-16T17:19:02.662819+0000 (oldest deadline 2024-04-16T17:19:27.863073+0000) 2024-04-16T17:19:30.677 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:19:30 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:19:30.224+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-16T17:18:36.158791+0000 front 2024-04-16T17:18:36.158992+0000 (oldest deadline 2024-04-16T17:19:00.258603+0000) 2024-04-16T17:19:30.678 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:19:30 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:19:30.224+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-16T17:18:44.260046+0000 front 2024-04-16T17:18:44.260171+0000 (oldest deadline 2024-04-16T17:19:07.160168+0000) 2024-04-16T17:19:30.678 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:19:30 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:19:30.224+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-16T17:18:54.062069+0000 front 2024-04-16T17:18:54.061971+0000 (oldest deadline 2024-04-16T17:19:18.761808+0000) 2024-04-16T17:19:30.678 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:19:30 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:19:30.224+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-16T17:19:01.063265+0000 front 2024-04-16T17:19:01.063419+0000 (oldest deadline 2024-04-16T17:19:24.563191+0000) 2024-04-16T17:19:30.804 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:19:30 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:19:30.519+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-16T17:18:34.374061+0000 front 2024-04-16T17:18:34.374275+0000 (oldest deadline 2024-04-16T17:18:57.873879+0000) 2024-04-16T17:19:30.805 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:19:30 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:19:30.519+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-16T17:18:50.676854+0000 front 2024-04-16T17:18:50.676874+0000 (oldest deadline 2024-04-16T17:19:15.977004+0000) 2024-04-16T17:19:30.805 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:19:30 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:19:30.519+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-16T17:19:01.878146+0000 front 2024-04-16T17:19:01.878262+0000 (oldest deadline 2024-04-16T17:19:27.178048+0000) 2024-04-16T17:19:31.228 INFO:journalctl@ceph.osd.6.smithi045.stdout:Apr 16 17:19:30 smithi045 podman[104237]: 2024-04-16 17:19:30.859538808 +0000 UTC m=+0.036927798 image pull e5ce10bdffbeeffffd1125f67b369e2151f0f17cdec854e93b6adf9321f65907 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2 2024-04-16T17:19:31.228 INFO:journalctl@ceph.osd.6.smithi045.stdout:Apr 16 17:19:31 smithi045 podman[104237]: 2024-04-16 17:19:31.012924998 +0000 UTC m=+0.190314009 container create 7c0494dd1a1405d0061493fbc83faffcc378663fbcc1ea11796e23fcc38680d8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-6-deactivate, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240409, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-04-16T17:19:31.304 INFO:journalctl@ceph.osd.8.smithi117.stdout:Apr 16 17:19:30 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-8[55817]: 2024-04-16T17:19:30.855+0000 7f65ab398700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-16T17:18:41.860244+0000 front 2024-04-16T17:18:41.860176+0000 (oldest deadline 2024-04-16T17:19:07.760349+0000) 2024-04-16T17:19:31.304 INFO:journalctl@ceph.osd.8.smithi117.stdout:Apr 16 17:19:30 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-8[55817]: 2024-04-16T17:19:30.855+0000 7f65ab398700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-16T17:18:52.461265+0000 front 2024-04-16T17:18:52.461355+0000 (oldest deadline 2024-04-16T17:19:20.662233+0000) 2024-04-16T17:19:31.304 INFO:journalctl@ceph.osd.8.smithi117.stdout:Apr 16 17:19:30 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-8[55817]: 2024-04-16T17:19:30.855+0000 7f65ab398700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-16T17:19:00.663078+0000 front 2024-04-16T17:19:00.662927+0000 (oldest deadline 2024-04-16T17:19:24.762742+0000) 2024-04-16T17:19:31.677 INFO:journalctl@ceph.osd.7.smithi104.stdout:Apr 16 17:19:31 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-7[55033]: 2024-04-16T17:19:31.409+0000 7f08b34d5700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-16T17:18:42.259445+0000 front 2024-04-16T17:18:42.259491+0000 (oldest deadline 2024-04-16T17:19:08.159569+0000) 2024-04-16T17:19:31.677 INFO:journalctl@ceph.osd.7.smithi104.stdout:Apr 16 17:19:31 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-7[55033]: 2024-04-16T17:19:31.409+0000 7f08b34d5700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-16T17:18:50.460516+0000 front 2024-04-16T17:18:50.460605+0000 (oldest deadline 2024-04-16T17:19:16.360615+0000) 2024-04-16T17:19:31.677 INFO:journalctl@ceph.osd.7.smithi104.stdout:Apr 16 17:19:31 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-7[55033]: 2024-04-16T17:19:31.409+0000 7f08b34d5700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-16T17:19:02.662964+0000 front 2024-04-16T17:19:02.662819+0000 (oldest deadline 2024-04-16T17:19:27.863073+0000) 2024-04-16T17:19:31.677 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:19:31 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:19:31.234+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-16T17:18:36.158791+0000 front 2024-04-16T17:18:36.158992+0000 (oldest deadline 2024-04-16T17:19:00.258603+0000) 2024-04-16T17:19:31.678 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:19:31 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:19:31.234+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-16T17:18:44.260046+0000 front 2024-04-16T17:18:44.260171+0000 (oldest deadline 2024-04-16T17:19:07.160168+0000) 2024-04-16T17:19:31.678 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:19:31 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:19:31.234+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-16T17:18:54.062069+0000 front 2024-04-16T17:18:54.061971+0000 (oldest deadline 2024-04-16T17:19:18.761808+0000) 2024-04-16T17:19:31.678 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:19:31 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:19:31.234+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-16T17:19:01.063265+0000 front 2024-04-16T17:19:01.063419+0000 (oldest deadline 2024-04-16T17:19:24.563191+0000) 2024-04-16T17:19:31.727 INFO:journalctl@ceph.osd.6.smithi045.stdout:Apr 16 17:19:31 smithi045 podman[104237]: 2024-04-16 17:19:31.435426676 +0000 UTC m=+0.612815681 container init 7c0494dd1a1405d0061493fbc83faffcc378663fbcc1ea11796e23fcc38680d8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-6-deactivate, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, org.label-schema.build-date=20240409, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-16T17:19:31.727 INFO:journalctl@ceph.osd.6.smithi045.stdout:Apr 16 17:19:31 smithi045 podman[104237]: 2024-04-16 17:19:31.448739498 +0000 UTC m=+0.626128515 container start 7c0494dd1a1405d0061493fbc83faffcc378663fbcc1ea11796e23fcc38680d8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-6-deactivate, org.label-schema.build-date=20240409, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-04-16T17:19:31.727 INFO:journalctl@ceph.osd.6.smithi045.stdout:Apr 16 17:19:31 smithi045 podman[104237]: 2024-04-16 17:19:31.53192489 +0000 UTC m=+0.709313954 container attach 7c0494dd1a1405d0061493fbc83faffcc378663fbcc1ea11796e23fcc38680d8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-6-deactivate, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240409, org.label-schema.license=GPLv2, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, io.buildah.version=1.33.5, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-16T17:19:31.727 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 16 17:19:31 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-9[66545]: 2024-04-16T17:19:31.317+0000 7f5937600700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-16T17:18:36.968505+0000 front 2024-04-16T17:18:36.968670+0000 (oldest deadline 2024-04-16T17:19:02.268762+0000) 2024-04-16T17:19:31.727 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 16 17:19:31 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-9[66545]: 2024-04-16T17:19:31.317+0000 7f5937600700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-16T17:18:43.371969+0000 front 2024-04-16T17:18:43.371904+0000 (oldest deadline 2024-04-16T17:19:08.671642+0000) 2024-04-16T17:19:31.728 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 16 17:19:31 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-9[66545]: 2024-04-16T17:19:31.317+0000 7f5937600700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-16T17:18:53.373968+0000 front 2024-04-16T17:18:53.374043+0000 (oldest deadline 2024-04-16T17:19:15.673755+0000) 2024-04-16T17:19:31.728 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 16 17:19:31 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-9[66545]: 2024-04-16T17:19:31.317+0000 7f5937600700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-16T17:19:01.880446+0000 front 2024-04-16T17:19:01.880936+0000 (oldest deadline 2024-04-16T17:19:27.180477+0000) 2024-04-16T17:19:31.804 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:19:31 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:19:31.510+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-16T17:18:34.374061+0000 front 2024-04-16T17:18:34.374275+0000 (oldest deadline 2024-04-16T17:18:57.873879+0000) 2024-04-16T17:19:31.804 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:19:31 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:19:31.510+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-16T17:18:50.676854+0000 front 2024-04-16T17:18:50.676874+0000 (oldest deadline 2024-04-16T17:19:15.977004+0000) 2024-04-16T17:19:31.804 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:19:31 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:19:31.510+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-16T17:19:01.878146+0000 front 2024-04-16T17:19:01.878262+0000 (oldest deadline 2024-04-16T17:19:27.178048+0000) 2024-04-16T17:19:32.227 INFO:journalctl@ceph.osd.6.smithi045.stdout:Apr 16 17:19:31 smithi045 podman[104237]: 2024-04-16 17:19:31.910484856 +0000 UTC m=+1.087873885 container died 7c0494dd1a1405d0061493fbc83faffcc378663fbcc1ea11796e23fcc38680d8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-6-deactivate, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240409, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.license=GPLv2) 2024-04-16T17:19:32.304 INFO:journalctl@ceph.osd.8.smithi117.stdout:Apr 16 17:19:31 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-8[55817]: 2024-04-16T17:19:31.885+0000 7f65ab398700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-16T17:18:41.860244+0000 front 2024-04-16T17:18:41.860176+0000 (oldest deadline 2024-04-16T17:19:07.760349+0000) 2024-04-16T17:19:32.304 INFO:journalctl@ceph.osd.8.smithi117.stdout:Apr 16 17:19:31 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-8[55817]: 2024-04-16T17:19:31.885+0000 7f65ab398700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-16T17:18:52.461265+0000 front 2024-04-16T17:18:52.461355+0000 (oldest deadline 2024-04-16T17:19:20.662233+0000) 2024-04-16T17:19:32.304 INFO:journalctl@ceph.osd.8.smithi117.stdout:Apr 16 17:19:31 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-8[55817]: 2024-04-16T17:19:31.885+0000 7f65ab398700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-16T17:19:00.663078+0000 front 2024-04-16T17:19:00.662927+0000 (oldest deadline 2024-04-16T17:19:24.762742+0000) 2024-04-16T17:19:32.677 INFO:journalctl@ceph.osd.7.smithi104.stdout:Apr 16 17:19:32 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-7[55033]: 2024-04-16T17:19:32.423+0000 7f08b34d5700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-16T17:18:42.259445+0000 front 2024-04-16T17:18:42.259491+0000 (oldest deadline 2024-04-16T17:19:08.159569+0000) 2024-04-16T17:19:32.677 INFO:journalctl@ceph.osd.7.smithi104.stdout:Apr 16 17:19:32 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-7[55033]: 2024-04-16T17:19:32.423+0000 7f08b34d5700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-16T17:18:50.460516+0000 front 2024-04-16T17:18:50.460605+0000 (oldest deadline 2024-04-16T17:19:16.360615+0000) 2024-04-16T17:19:32.677 INFO:journalctl@ceph.osd.7.smithi104.stdout:Apr 16 17:19:32 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-7[55033]: 2024-04-16T17:19:32.423+0000 7f08b34d5700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-16T17:19:02.662964+0000 front 2024-04-16T17:19:02.662819+0000 (oldest deadline 2024-04-16T17:19:27.863073+0000) 2024-04-16T17:19:32.677 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:19:32 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:19:32.248+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-16T17:18:36.158791+0000 front 2024-04-16T17:18:36.158992+0000 (oldest deadline 2024-04-16T17:19:00.258603+0000) 2024-04-16T17:19:32.677 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:19:32 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:19:32.248+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-16T17:18:44.260046+0000 front 2024-04-16T17:18:44.260171+0000 (oldest deadline 2024-04-16T17:19:07.160168+0000) 2024-04-16T17:19:32.678 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:19:32 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:19:32.248+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-16T17:18:54.062069+0000 front 2024-04-16T17:18:54.061971+0000 (oldest deadline 2024-04-16T17:19:18.761808+0000) 2024-04-16T17:19:32.678 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:19:32 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:19:32.248+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-16T17:19:01.063265+0000 front 2024-04-16T17:19:01.063419+0000 (oldest deadline 2024-04-16T17:19:24.563191+0000) 2024-04-16T17:19:32.726 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 16 17:19:32 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-9[66545]: 2024-04-16T17:19:32.338+0000 7f5937600700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-16T17:18:36.968505+0000 front 2024-04-16T17:18:36.968670+0000 (oldest deadline 2024-04-16T17:19:02.268762+0000) 2024-04-16T17:19:32.727 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 16 17:19:32 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-9[66545]: 2024-04-16T17:19:32.338+0000 7f5937600700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-16T17:18:43.371969+0000 front 2024-04-16T17:18:43.371904+0000 (oldest deadline 2024-04-16T17:19:08.671642+0000) 2024-04-16T17:19:32.727 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 16 17:19:32 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-9[66545]: 2024-04-16T17:19:32.338+0000 7f5937600700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-16T17:18:53.373968+0000 front 2024-04-16T17:18:53.374043+0000 (oldest deadline 2024-04-16T17:19:15.673755+0000) 2024-04-16T17:19:32.727 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 16 17:19:32 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-9[66545]: 2024-04-16T17:19:32.338+0000 7f5937600700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-16T17:19:01.880446+0000 front 2024-04-16T17:19:01.880936+0000 (oldest deadline 2024-04-16T17:19:27.180477+0000) 2024-04-16T17:19:32.783 DEBUG:teuthology.orchestra.run.smithi045:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b1fecd38-fc11-11ee-bc90-c7b262605968@osd.6.service' 2024-04-16T17:19:32.804 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:19:32 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:19:32.493+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-16T17:18:34.374061+0000 front 2024-04-16T17:18:34.374275+0000 (oldest deadline 2024-04-16T17:18:57.873879+0000) 2024-04-16T17:19:32.804 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:19:32 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:19:32.493+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-16T17:18:50.676854+0000 front 2024-04-16T17:18:50.676874+0000 (oldest deadline 2024-04-16T17:19:15.977004+0000) 2024-04-16T17:19:32.804 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:19:32 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:19:32.493+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-16T17:19:01.878146+0000 front 2024-04-16T17:19:01.878262+0000 (oldest deadline 2024-04-16T17:19:27.178048+0000) 2024-04-16T17:19:32.915 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-16T17:19:32.915 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-04-16T17:19:32.916 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-04-16T17:19:32.916 DEBUG:teuthology.orchestra.run.smithi104:> sudo systemctl stop ceph-b1fecd38-fc11-11ee-bc90-c7b262605968@osd.7 2024-04-16T17:19:33.226 INFO:journalctl@ceph.osd.7.smithi104.stdout:Apr 16 17:19:32 smithi104 systemd[1]: Stopping Ceph osd.7 for b1fecd38-fc11-11ee-bc90-c7b262605968... 2024-04-16T17:19:33.304 INFO:journalctl@ceph.osd.8.smithi117.stdout:Apr 16 17:19:32 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-8[55817]: 2024-04-16T17:19:32.847+0000 7f65ab398700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-16T17:18:41.860244+0000 front 2024-04-16T17:18:41.860176+0000 (oldest deadline 2024-04-16T17:19:07.760349+0000) 2024-04-16T17:19:33.304 INFO:journalctl@ceph.osd.8.smithi117.stdout:Apr 16 17:19:32 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-8[55817]: 2024-04-16T17:19:32.847+0000 7f65ab398700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-16T17:18:52.461265+0000 front 2024-04-16T17:18:52.461355+0000 (oldest deadline 2024-04-16T17:19:20.662233+0000) 2024-04-16T17:19:33.304 INFO:journalctl@ceph.osd.8.smithi117.stdout:Apr 16 17:19:32 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-8[55817]: 2024-04-16T17:19:32.847+0000 7f65ab398700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-16T17:19:00.663078+0000 front 2024-04-16T17:19:00.662927+0000 (oldest deadline 2024-04-16T17:19:24.762742+0000) 2024-04-16T17:19:33.677 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:19:33 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:19:33.254+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-16T17:18:36.158791+0000 front 2024-04-16T17:18:36.158992+0000 (oldest deadline 2024-04-16T17:19:00.258603+0000) 2024-04-16T17:19:33.677 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:19:33 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:19:33.254+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-16T17:18:44.260046+0000 front 2024-04-16T17:18:44.260171+0000 (oldest deadline 2024-04-16T17:19:07.160168+0000) 2024-04-16T17:19:33.677 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:19:33 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:19:33.254+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-16T17:18:54.062069+0000 front 2024-04-16T17:18:54.061971+0000 (oldest deadline 2024-04-16T17:19:18.761808+0000) 2024-04-16T17:19:33.677 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:19:33 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:19:33.254+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-16T17:19:01.063265+0000 front 2024-04-16T17:19:01.063419+0000 (oldest deadline 2024-04-16T17:19:24.563191+0000) 2024-04-16T17:19:33.677 INFO:journalctl@ceph.osd.7.smithi104.stdout:Apr 16 17:19:33 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-7[55033]: 2024-04-16T17:19:33.226+0000 7f08baee9700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-16T17:19:33.677 INFO:journalctl@ceph.osd.7.smithi104.stdout:Apr 16 17:19:33 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-7[55033]: 2024-04-16T17:19:33.226+0000 7f08baee9700 -1 osd.7 112 *** Got signal Terminated *** 2024-04-16T17:19:33.677 INFO:journalctl@ceph.osd.7.smithi104.stdout:Apr 16 17:19:33 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-7[55033]: 2024-04-16T17:19:33.226+0000 7f08baee9700 -1 osd.7 112 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-16T17:19:33.678 INFO:journalctl@ceph.osd.7.smithi104.stdout:Apr 16 17:19:33 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-7[55033]: 2024-04-16T17:19:33.410+0000 7f08b34d5700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-16T17:18:42.259445+0000 front 2024-04-16T17:18:42.259491+0000 (oldest deadline 2024-04-16T17:19:08.159569+0000) 2024-04-16T17:19:33.678 INFO:journalctl@ceph.osd.7.smithi104.stdout:Apr 16 17:19:33 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-7[55033]: 2024-04-16T17:19:33.410+0000 7f08b34d5700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-16T17:18:50.460516+0000 front 2024-04-16T17:18:50.460605+0000 (oldest deadline 2024-04-16T17:19:16.360615+0000) 2024-04-16T17:19:33.678 INFO:journalctl@ceph.osd.7.smithi104.stdout:Apr 16 17:19:33 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-7[55033]: 2024-04-16T17:19:33.410+0000 7f08b34d5700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-16T17:19:02.662964+0000 front 2024-04-16T17:19:02.662819+0000 (oldest deadline 2024-04-16T17:19:27.863073+0000) 2024-04-16T17:19:33.678 INFO:journalctl@ceph.osd.7.smithi104.stdout:Apr 16 17:19:33 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-7[55033]: 2024-04-16T17:19:33.410+0000 7f08b34d5700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-04-16T17:19:07.863349+0000 front 2024-04-16T17:19:07.863461+0000 (oldest deadline 2024-04-16T17:19:33.163433+0000) 2024-04-16T17:19:33.727 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 16 17:19:33 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-9[66545]: 2024-04-16T17:19:33.309+0000 7f5937600700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-16T17:18:36.968505+0000 front 2024-04-16T17:18:36.968670+0000 (oldest deadline 2024-04-16T17:19:02.268762+0000) 2024-04-16T17:19:33.727 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 16 17:19:33 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-9[66545]: 2024-04-16T17:19:33.309+0000 7f5937600700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-16T17:18:43.371969+0000 front 2024-04-16T17:18:43.371904+0000 (oldest deadline 2024-04-16T17:19:08.671642+0000) 2024-04-16T17:19:33.727 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 16 17:19:33 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-9[66545]: 2024-04-16T17:19:33.309+0000 7f5937600700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-16T17:18:53.373968+0000 front 2024-04-16T17:18:53.374043+0000 (oldest deadline 2024-04-16T17:19:15.673755+0000) 2024-04-16T17:19:33.727 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 16 17:19:33 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-9[66545]: 2024-04-16T17:19:33.309+0000 7f5937600700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-16T17:19:01.880446+0000 front 2024-04-16T17:19:01.880936+0000 (oldest deadline 2024-04-16T17:19:27.180477+0000) 2024-04-16T17:19:33.727 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 16 17:19:33 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-9[66545]: 2024-04-16T17:19:33.309+0000 7f5937600700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-04-16T17:19:09.482762+0000 front 2024-04-16T17:19:09.482816+0000 (oldest deadline 2024-04-16T17:19:32.982945+0000) 2024-04-16T17:19:33.804 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:19:33 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:19:33.469+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-16T17:18:34.374061+0000 front 2024-04-16T17:18:34.374275+0000 (oldest deadline 2024-04-16T17:18:57.873879+0000) 2024-04-16T17:19:33.804 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:19:33 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:19:33.469+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-16T17:18:50.676854+0000 front 2024-04-16T17:18:50.676874+0000 (oldest deadline 2024-04-16T17:19:15.977004+0000) 2024-04-16T17:19:33.804 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:19:33 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:19:33.469+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-16T17:19:01.878146+0000 front 2024-04-16T17:19:01.878262+0000 (oldest deadline 2024-04-16T17:19:27.178048+0000) 2024-04-16T17:19:33.804 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:19:33 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:19:33.469+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-04-16T17:19:07.178684+0000 front 2024-04-16T17:19:07.178598+0000 (oldest deadline 2024-04-16T17:19:33.078600+0000) 2024-04-16T17:19:34.304 INFO:journalctl@ceph.osd.8.smithi117.stdout:Apr 16 17:19:33 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-8[55817]: 2024-04-16T17:19:33.842+0000 7f65ab398700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-16T17:18:41.860244+0000 front 2024-04-16T17:18:41.860176+0000 (oldest deadline 2024-04-16T17:19:07.760349+0000) 2024-04-16T17:19:34.304 INFO:journalctl@ceph.osd.8.smithi117.stdout:Apr 16 17:19:33 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-8[55817]: 2024-04-16T17:19:33.842+0000 7f65ab398700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-16T17:18:52.461265+0000 front 2024-04-16T17:18:52.461355+0000 (oldest deadline 2024-04-16T17:19:20.662233+0000) 2024-04-16T17:19:34.305 INFO:journalctl@ceph.osd.8.smithi117.stdout:Apr 16 17:19:33 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-8[55817]: 2024-04-16T17:19:33.842+0000 7f65ab398700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-16T17:19:00.663078+0000 front 2024-04-16T17:19:00.662927+0000 (oldest deadline 2024-04-16T17:19:24.762742+0000) 2024-04-16T17:19:34.677 INFO:journalctl@ceph.osd.7.smithi104.stdout:Apr 16 17:19:34 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-7[55033]: 2024-04-16T17:19:34.440+0000 7f08b34d5700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-16T17:18:42.259445+0000 front 2024-04-16T17:18:42.259491+0000 (oldest deadline 2024-04-16T17:19:08.159569+0000) 2024-04-16T17:19:34.677 INFO:journalctl@ceph.osd.7.smithi104.stdout:Apr 16 17:19:34 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-7[55033]: 2024-04-16T17:19:34.440+0000 7f08b34d5700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-16T17:18:50.460516+0000 front 2024-04-16T17:18:50.460605+0000 (oldest deadline 2024-04-16T17:19:16.360615+0000) 2024-04-16T17:19:34.677 INFO:journalctl@ceph.osd.7.smithi104.stdout:Apr 16 17:19:34 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-7[55033]: 2024-04-16T17:19:34.440+0000 7f08b34d5700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-16T17:19:02.662964+0000 front 2024-04-16T17:19:02.662819+0000 (oldest deadline 2024-04-16T17:19:27.863073+0000) 2024-04-16T17:19:34.677 INFO:journalctl@ceph.osd.7.smithi104.stdout:Apr 16 17:19:34 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-7[55033]: 2024-04-16T17:19:34.440+0000 7f08b34d5700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-04-16T17:19:07.863349+0000 front 2024-04-16T17:19:07.863461+0000 (oldest deadline 2024-04-16T17:19:33.163433+0000) 2024-04-16T17:19:34.678 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:19:34 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:19:34.230+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-16T17:18:36.158791+0000 front 2024-04-16T17:18:36.158992+0000 (oldest deadline 2024-04-16T17:19:00.258603+0000) 2024-04-16T17:19:34.678 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:19:34 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:19:34.230+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-16T17:18:44.260046+0000 front 2024-04-16T17:18:44.260171+0000 (oldest deadline 2024-04-16T17:19:07.160168+0000) 2024-04-16T17:19:34.678 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:19:34 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:19:34.230+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-16T17:18:54.062069+0000 front 2024-04-16T17:18:54.061971+0000 (oldest deadline 2024-04-16T17:19:18.761808+0000) 2024-04-16T17:19:34.678 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:19:34 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:19:34.230+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-16T17:19:01.063265+0000 front 2024-04-16T17:19:01.063419+0000 (oldest deadline 2024-04-16T17:19:24.563191+0000) 2024-04-16T17:19:34.727 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 16 17:19:34 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-9[66545]: 2024-04-16T17:19:34.348+0000 7f5937600700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-16T17:18:36.968505+0000 front 2024-04-16T17:18:36.968670+0000 (oldest deadline 2024-04-16T17:19:02.268762+0000) 2024-04-16T17:19:34.727 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 16 17:19:34 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-9[66545]: 2024-04-16T17:19:34.348+0000 7f5937600700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-16T17:18:43.371969+0000 front 2024-04-16T17:18:43.371904+0000 (oldest deadline 2024-04-16T17:19:08.671642+0000) 2024-04-16T17:19:34.727 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 16 17:19:34 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-9[66545]: 2024-04-16T17:19:34.348+0000 7f5937600700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-16T17:18:53.373968+0000 front 2024-04-16T17:18:53.374043+0000 (oldest deadline 2024-04-16T17:19:15.673755+0000) 2024-04-16T17:19:34.727 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 16 17:19:34 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-9[66545]: 2024-04-16T17:19:34.348+0000 7f5937600700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-16T17:19:01.880446+0000 front 2024-04-16T17:19:01.880936+0000 (oldest deadline 2024-04-16T17:19:27.180477+0000) 2024-04-16T17:19:34.727 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 16 17:19:34 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-9[66545]: 2024-04-16T17:19:34.348+0000 7f5937600700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-04-16T17:19:09.482762+0000 front 2024-04-16T17:19:09.482816+0000 (oldest deadline 2024-04-16T17:19:32.982945+0000) 2024-04-16T17:19:34.804 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:19:34 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:19:34.513+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-16T17:18:34.374061+0000 front 2024-04-16T17:18:34.374275+0000 (oldest deadline 2024-04-16T17:18:57.873879+0000) 2024-04-16T17:19:34.804 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:19:34 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:19:34.513+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-16T17:18:50.676854+0000 front 2024-04-16T17:18:50.676874+0000 (oldest deadline 2024-04-16T17:19:15.977004+0000) 2024-04-16T17:19:34.804 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:19:34 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:19:34.513+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-16T17:19:01.878146+0000 front 2024-04-16T17:19:01.878262+0000 (oldest deadline 2024-04-16T17:19:27.178048+0000) 2024-04-16T17:19:34.804 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:19:34 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:19:34.513+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-04-16T17:19:07.178684+0000 front 2024-04-16T17:19:07.178598+0000 (oldest deadline 2024-04-16T17:19:33.078600+0000) 2024-04-16T17:19:35.304 INFO:journalctl@ceph.osd.8.smithi117.stdout:Apr 16 17:19:34 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-8[55817]: 2024-04-16T17:19:34.857+0000 7f65ab398700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-16T17:18:41.860244+0000 front 2024-04-16T17:18:41.860176+0000 (oldest deadline 2024-04-16T17:19:07.760349+0000) 2024-04-16T17:19:35.305 INFO:journalctl@ceph.osd.8.smithi117.stdout:Apr 16 17:19:34 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-8[55817]: 2024-04-16T17:19:34.857+0000 7f65ab398700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-16T17:18:52.461265+0000 front 2024-04-16T17:18:52.461355+0000 (oldest deadline 2024-04-16T17:19:20.662233+0000) 2024-04-16T17:19:35.305 INFO:journalctl@ceph.osd.8.smithi117.stdout:Apr 16 17:19:34 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-8[55817]: 2024-04-16T17:19:34.857+0000 7f65ab398700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-16T17:19:00.663078+0000 front 2024-04-16T17:19:00.662927+0000 (oldest deadline 2024-04-16T17:19:24.762742+0000) 2024-04-16T17:19:35.305 INFO:journalctl@ceph.osd.8.smithi117.stdout:Apr 16 17:19:34 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-8[55817]: 2024-04-16T17:19:34.857+0000 7f65ab398700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-04-16T17:19:11.565691+0000 front 2024-04-16T17:19:11.565594+0000 (oldest deadline 2024-04-16T17:19:34.465535+0000) 2024-04-16T17:19:35.677 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:19:35 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:19:35.274+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-16T17:18:36.158791+0000 front 2024-04-16T17:18:36.158992+0000 (oldest deadline 2024-04-16T17:19:00.258603+0000) 2024-04-16T17:19:35.677 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:19:35 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:19:35.274+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-16T17:18:44.260046+0000 front 2024-04-16T17:18:44.260171+0000 (oldest deadline 2024-04-16T17:19:07.160168+0000) 2024-04-16T17:19:35.677 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:19:35 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:19:35.274+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-16T17:18:54.062069+0000 front 2024-04-16T17:18:54.061971+0000 (oldest deadline 2024-04-16T17:19:18.761808+0000) 2024-04-16T17:19:35.677 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:19:35 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:19:35.274+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-16T17:19:01.063265+0000 front 2024-04-16T17:19:01.063419+0000 (oldest deadline 2024-04-16T17:19:24.563191+0000) 2024-04-16T17:19:35.677 INFO:journalctl@ceph.osd.7.smithi104.stdout:Apr 16 17:19:35 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-7[55033]: 2024-04-16T17:19:35.481+0000 7f08b34d5700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-16T17:18:42.259445+0000 front 2024-04-16T17:18:42.259491+0000 (oldest deadline 2024-04-16T17:19:08.159569+0000) 2024-04-16T17:19:35.678 INFO:journalctl@ceph.osd.7.smithi104.stdout:Apr 16 17:19:35 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-7[55033]: 2024-04-16T17:19:35.481+0000 7f08b34d5700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-16T17:18:50.460516+0000 front 2024-04-16T17:18:50.460605+0000 (oldest deadline 2024-04-16T17:19:16.360615+0000) 2024-04-16T17:19:35.678 INFO:journalctl@ceph.osd.7.smithi104.stdout:Apr 16 17:19:35 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-7[55033]: 2024-04-16T17:19:35.481+0000 7f08b34d5700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-16T17:19:02.662964+0000 front 2024-04-16T17:19:02.662819+0000 (oldest deadline 2024-04-16T17:19:27.863073+0000) 2024-04-16T17:19:35.678 INFO:journalctl@ceph.osd.7.smithi104.stdout:Apr 16 17:19:35 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-7[55033]: 2024-04-16T17:19:35.481+0000 7f08b34d5700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-04-16T17:19:07.863349+0000 front 2024-04-16T17:19:07.863461+0000 (oldest deadline 2024-04-16T17:19:33.163433+0000) 2024-04-16T17:19:35.727 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 16 17:19:35 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-9[66545]: 2024-04-16T17:19:35.312+0000 7f5937600700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-16T17:18:36.968505+0000 front 2024-04-16T17:18:36.968670+0000 (oldest deadline 2024-04-16T17:19:02.268762+0000) 2024-04-16T17:19:35.727 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 16 17:19:35 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-9[66545]: 2024-04-16T17:19:35.312+0000 7f5937600700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-16T17:18:43.371969+0000 front 2024-04-16T17:18:43.371904+0000 (oldest deadline 2024-04-16T17:19:08.671642+0000) 2024-04-16T17:19:35.727 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 16 17:19:35 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-9[66545]: 2024-04-16T17:19:35.312+0000 7f5937600700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-16T17:18:53.373968+0000 front 2024-04-16T17:18:53.374043+0000 (oldest deadline 2024-04-16T17:19:15.673755+0000) 2024-04-16T17:19:35.727 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 16 17:19:35 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-9[66545]: 2024-04-16T17:19:35.312+0000 7f5937600700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-16T17:19:01.880446+0000 front 2024-04-16T17:19:01.880936+0000 (oldest deadline 2024-04-16T17:19:27.180477+0000) 2024-04-16T17:19:35.727 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 16 17:19:35 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-9[66545]: 2024-04-16T17:19:35.312+0000 7f5937600700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-04-16T17:19:09.482762+0000 front 2024-04-16T17:19:09.482816+0000 (oldest deadline 2024-04-16T17:19:32.982945+0000) 2024-04-16T17:19:35.804 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:19:35 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:19:35.524+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-16T17:18:34.374061+0000 front 2024-04-16T17:18:34.374275+0000 (oldest deadline 2024-04-16T17:18:57.873879+0000) 2024-04-16T17:19:35.804 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:19:35 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:19:35.524+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-16T17:18:50.676854+0000 front 2024-04-16T17:18:50.676874+0000 (oldest deadline 2024-04-16T17:19:15.977004+0000) 2024-04-16T17:19:35.804 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:19:35 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:19:35.524+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-16T17:19:01.878146+0000 front 2024-04-16T17:19:01.878262+0000 (oldest deadline 2024-04-16T17:19:27.178048+0000) 2024-04-16T17:19:35.805 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:19:35 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:19:35.524+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-04-16T17:19:07.178684+0000 front 2024-04-16T17:19:07.178598+0000 (oldest deadline 2024-04-16T17:19:33.078600+0000) 2024-04-16T17:19:36.304 INFO:journalctl@ceph.osd.8.smithi117.stdout:Apr 16 17:19:35 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-8[55817]: 2024-04-16T17:19:35.813+0000 7f65ab398700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-16T17:18:41.860244+0000 front 2024-04-16T17:18:41.860176+0000 (oldest deadline 2024-04-16T17:19:07.760349+0000) 2024-04-16T17:19:36.304 INFO:journalctl@ceph.osd.8.smithi117.stdout:Apr 16 17:19:35 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-8[55817]: 2024-04-16T17:19:35.813+0000 7f65ab398700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-16T17:18:52.461265+0000 front 2024-04-16T17:18:52.461355+0000 (oldest deadline 2024-04-16T17:19:20.662233+0000) 2024-04-16T17:19:36.305 INFO:journalctl@ceph.osd.8.smithi117.stdout:Apr 16 17:19:35 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-8[55817]: 2024-04-16T17:19:35.813+0000 7f65ab398700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-16T17:19:00.663078+0000 front 2024-04-16T17:19:00.662927+0000 (oldest deadline 2024-04-16T17:19:24.762742+0000) 2024-04-16T17:19:36.305 INFO:journalctl@ceph.osd.8.smithi117.stdout:Apr 16 17:19:35 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-8[55817]: 2024-04-16T17:19:35.813+0000 7f65ab398700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-04-16T17:19:11.565691+0000 front 2024-04-16T17:19:11.565594+0000 (oldest deadline 2024-04-16T17:19:34.465535+0000) 2024-04-16T17:19:36.507 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:19:36 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:19:36.245+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-16T17:18:36.158791+0000 front 2024-04-16T17:18:36.158992+0000 (oldest deadline 2024-04-16T17:19:00.258603+0000) 2024-04-16T17:19:36.507 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:19:36 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:19:36.245+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-16T17:18:44.260046+0000 front 2024-04-16T17:18:44.260171+0000 (oldest deadline 2024-04-16T17:19:07.160168+0000) 2024-04-16T17:19:36.508 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:19:36 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:19:36.245+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-16T17:18:54.062069+0000 front 2024-04-16T17:18:54.061971+0000 (oldest deadline 2024-04-16T17:19:18.761808+0000) 2024-04-16T17:19:36.508 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:19:36 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:19:36.245+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-16T17:19:01.063265+0000 front 2024-04-16T17:19:01.063419+0000 (oldest deadline 2024-04-16T17:19:24.563191+0000) 2024-04-16T17:19:36.508 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:19:36 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:19:36.245+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-04-16T17:19:10.865363+0000 front 2024-04-16T17:19:10.865389+0000 (oldest deadline 2024-04-16T17:19:35.466166+0000) 2024-04-16T17:19:36.726 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 16 17:19:36 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-9[66545]: 2024-04-16T17:19:36.267+0000 7f5937600700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-16T17:18:36.968505+0000 front 2024-04-16T17:18:36.968670+0000 (oldest deadline 2024-04-16T17:19:02.268762+0000) 2024-04-16T17:19:36.726 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 16 17:19:36 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-9[66545]: 2024-04-16T17:19:36.267+0000 7f5937600700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-16T17:18:43.371969+0000 front 2024-04-16T17:18:43.371904+0000 (oldest deadline 2024-04-16T17:19:08.671642+0000) 2024-04-16T17:19:36.727 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 16 17:19:36 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-9[66545]: 2024-04-16T17:19:36.267+0000 7f5937600700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-16T17:18:53.373968+0000 front 2024-04-16T17:18:53.374043+0000 (oldest deadline 2024-04-16T17:19:15.673755+0000) 2024-04-16T17:19:36.727 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 16 17:19:36 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-9[66545]: 2024-04-16T17:19:36.267+0000 7f5937600700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-16T17:19:01.880446+0000 front 2024-04-16T17:19:01.880936+0000 (oldest deadline 2024-04-16T17:19:27.180477+0000) 2024-04-16T17:19:36.727 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 16 17:19:36 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-9[66545]: 2024-04-16T17:19:36.267+0000 7f5937600700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-04-16T17:19:09.482762+0000 front 2024-04-16T17:19:09.482816+0000 (oldest deadline 2024-04-16T17:19:32.982945+0000) 2024-04-16T17:19:36.804 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:19:36 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:19:36.506+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-16T17:18:34.374061+0000 front 2024-04-16T17:18:34.374275+0000 (oldest deadline 2024-04-16T17:18:57.873879+0000) 2024-04-16T17:19:36.805 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:19:36 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:19:36.506+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-16T17:18:50.676854+0000 front 2024-04-16T17:18:50.676874+0000 (oldest deadline 2024-04-16T17:19:15.977004+0000) 2024-04-16T17:19:36.805 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:19:36 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:19:36.506+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-16T17:19:01.878146+0000 front 2024-04-16T17:19:01.878262+0000 (oldest deadline 2024-04-16T17:19:27.178048+0000) 2024-04-16T17:19:36.805 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:19:36 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:19:36.506+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-04-16T17:19:07.178684+0000 front 2024-04-16T17:19:07.178598+0000 (oldest deadline 2024-04-16T17:19:33.078600+0000) 2024-04-16T17:19:36.926 INFO:journalctl@ceph.osd.7.smithi104.stdout:Apr 16 17:19:36 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-7[55033]: 2024-04-16T17:19:36.508+0000 7f08b34d5700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-16T17:18:42.259445+0000 front 2024-04-16T17:18:42.259491+0000 (oldest deadline 2024-04-16T17:19:08.159569+0000) 2024-04-16T17:19:36.927 INFO:journalctl@ceph.osd.7.smithi104.stdout:Apr 16 17:19:36 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-7[55033]: 2024-04-16T17:19:36.508+0000 7f08b34d5700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-16T17:18:50.460516+0000 front 2024-04-16T17:18:50.460605+0000 (oldest deadline 2024-04-16T17:19:16.360615+0000) 2024-04-16T17:19:36.927 INFO:journalctl@ceph.osd.7.smithi104.stdout:Apr 16 17:19:36 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-7[55033]: 2024-04-16T17:19:36.508+0000 7f08b34d5700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-16T17:19:02.662964+0000 front 2024-04-16T17:19:02.662819+0000 (oldest deadline 2024-04-16T17:19:27.863073+0000) 2024-04-16T17:19:36.927 INFO:journalctl@ceph.osd.7.smithi104.stdout:Apr 16 17:19:36 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-7[55033]: 2024-04-16T17:19:36.508+0000 7f08b34d5700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-04-16T17:19:07.863349+0000 front 2024-04-16T17:19:07.863461+0000 (oldest deadline 2024-04-16T17:19:33.163433+0000) 2024-04-16T17:19:37.304 INFO:journalctl@ceph.osd.8.smithi117.stdout:Apr 16 17:19:36 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-8[55817]: 2024-04-16T17:19:36.854+0000 7f65ab398700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-16T17:18:41.860244+0000 front 2024-04-16T17:18:41.860176+0000 (oldest deadline 2024-04-16T17:19:07.760349+0000) 2024-04-16T17:19:37.305 INFO:journalctl@ceph.osd.8.smithi117.stdout:Apr 16 17:19:36 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-8[55817]: 2024-04-16T17:19:36.854+0000 7f65ab398700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-16T17:18:52.461265+0000 front 2024-04-16T17:18:52.461355+0000 (oldest deadline 2024-04-16T17:19:20.662233+0000) 2024-04-16T17:19:37.305 INFO:journalctl@ceph.osd.8.smithi117.stdout:Apr 16 17:19:36 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-8[55817]: 2024-04-16T17:19:36.854+0000 7f65ab398700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-16T17:19:00.663078+0000 front 2024-04-16T17:19:00.662927+0000 (oldest deadline 2024-04-16T17:19:24.762742+0000) 2024-04-16T17:19:37.305 INFO:journalctl@ceph.osd.8.smithi117.stdout:Apr 16 17:19:36 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-8[55817]: 2024-04-16T17:19:36.854+0000 7f65ab398700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-04-16T17:19:11.565691+0000 front 2024-04-16T17:19:11.565594+0000 (oldest deadline 2024-04-16T17:19:34.465535+0000) 2024-04-16T17:19:37.508 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:19:37 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:19:37.246+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-16T17:18:36.158791+0000 front 2024-04-16T17:18:36.158992+0000 (oldest deadline 2024-04-16T17:19:00.258603+0000) 2024-04-16T17:19:37.508 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:19:37 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:19:37.246+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-16T17:18:44.260046+0000 front 2024-04-16T17:18:44.260171+0000 (oldest deadline 2024-04-16T17:19:07.160168+0000) 2024-04-16T17:19:37.508 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:19:37 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:19:37.246+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-16T17:18:54.062069+0000 front 2024-04-16T17:18:54.061971+0000 (oldest deadline 2024-04-16T17:19:18.761808+0000) 2024-04-16T17:19:37.508 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:19:37 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:19:37.246+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-16T17:19:01.063265+0000 front 2024-04-16T17:19:01.063419+0000 (oldest deadline 2024-04-16T17:19:24.563191+0000) 2024-04-16T17:19:37.508 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:19:37 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:19:37.246+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-04-16T17:19:10.865363+0000 front 2024-04-16T17:19:10.865389+0000 (oldest deadline 2024-04-16T17:19:35.466166+0000) 2024-04-16T17:19:37.727 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 16 17:19:37 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-9[66545]: 2024-04-16T17:19:37.296+0000 7f5937600700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-16T17:18:36.968505+0000 front 2024-04-16T17:18:36.968670+0000 (oldest deadline 2024-04-16T17:19:02.268762+0000) 2024-04-16T17:19:37.727 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 16 17:19:37 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-9[66545]: 2024-04-16T17:19:37.296+0000 7f5937600700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-16T17:18:43.371969+0000 front 2024-04-16T17:18:43.371904+0000 (oldest deadline 2024-04-16T17:19:08.671642+0000) 2024-04-16T17:19:37.727 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 16 17:19:37 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-9[66545]: 2024-04-16T17:19:37.296+0000 7f5937600700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-16T17:18:53.373968+0000 front 2024-04-16T17:18:53.374043+0000 (oldest deadline 2024-04-16T17:19:15.673755+0000) 2024-04-16T17:19:37.727 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 16 17:19:37 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-9[66545]: 2024-04-16T17:19:37.296+0000 7f5937600700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-16T17:19:01.880446+0000 front 2024-04-16T17:19:01.880936+0000 (oldest deadline 2024-04-16T17:19:27.180477+0000) 2024-04-16T17:19:37.727 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 16 17:19:37 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-9[66545]: 2024-04-16T17:19:37.296+0000 7f5937600700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-04-16T17:19:09.482762+0000 front 2024-04-16T17:19:09.482816+0000 (oldest deadline 2024-04-16T17:19:32.982945+0000) 2024-04-16T17:19:37.804 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:19:37 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:19:37.498+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-16T17:18:34.374061+0000 front 2024-04-16T17:18:34.374275+0000 (oldest deadline 2024-04-16T17:18:57.873879+0000) 2024-04-16T17:19:37.805 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:19:37 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:19:37.498+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-16T17:18:50.676854+0000 front 2024-04-16T17:18:50.676874+0000 (oldest deadline 2024-04-16T17:19:15.977004+0000) 2024-04-16T17:19:37.805 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:19:37 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:19:37.498+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-16T17:19:01.878146+0000 front 2024-04-16T17:19:01.878262+0000 (oldest deadline 2024-04-16T17:19:27.178048+0000) 2024-04-16T17:19:37.805 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:19:37 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:19:37.498+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-04-16T17:19:07.178684+0000 front 2024-04-16T17:19:07.178598+0000 (oldest deadline 2024-04-16T17:19:33.078600+0000) 2024-04-16T17:19:37.927 INFO:journalctl@ceph.osd.7.smithi104.stdout:Apr 16 17:19:37 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-7[55033]: 2024-04-16T17:19:37.508+0000 7f08b34d5700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-16T17:18:42.259445+0000 front 2024-04-16T17:18:42.259491+0000 (oldest deadline 2024-04-16T17:19:08.159569+0000) 2024-04-16T17:19:37.927 INFO:journalctl@ceph.osd.7.smithi104.stdout:Apr 16 17:19:37 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-7[55033]: 2024-04-16T17:19:37.508+0000 7f08b34d5700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-16T17:18:50.460516+0000 front 2024-04-16T17:18:50.460605+0000 (oldest deadline 2024-04-16T17:19:16.360615+0000) 2024-04-16T17:19:37.927 INFO:journalctl@ceph.osd.7.smithi104.stdout:Apr 16 17:19:37 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-7[55033]: 2024-04-16T17:19:37.508+0000 7f08b34d5700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-16T17:19:02.662964+0000 front 2024-04-16T17:19:02.662819+0000 (oldest deadline 2024-04-16T17:19:27.863073+0000) 2024-04-16T17:19:37.927 INFO:journalctl@ceph.osd.7.smithi104.stdout:Apr 16 17:19:37 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-7[55033]: 2024-04-16T17:19:37.508+0000 7f08b34d5700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-04-16T17:19:07.863349+0000 front 2024-04-16T17:19:07.863461+0000 (oldest deadline 2024-04-16T17:19:33.163433+0000) 2024-04-16T17:19:38.304 INFO:journalctl@ceph.osd.8.smithi117.stdout:Apr 16 17:19:37 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-8[55817]: 2024-04-16T17:19:37.887+0000 7f65ab398700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-16T17:18:41.860244+0000 front 2024-04-16T17:18:41.860176+0000 (oldest deadline 2024-04-16T17:19:07.760349+0000) 2024-04-16T17:19:38.304 INFO:journalctl@ceph.osd.8.smithi117.stdout:Apr 16 17:19:37 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-8[55817]: 2024-04-16T17:19:37.887+0000 7f65ab398700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-16T17:18:52.461265+0000 front 2024-04-16T17:18:52.461355+0000 (oldest deadline 2024-04-16T17:19:20.662233+0000) 2024-04-16T17:19:38.304 INFO:journalctl@ceph.osd.8.smithi117.stdout:Apr 16 17:19:37 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-8[55817]: 2024-04-16T17:19:37.887+0000 7f65ab398700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-16T17:19:00.663078+0000 front 2024-04-16T17:19:00.662927+0000 (oldest deadline 2024-04-16T17:19:24.762742+0000) 2024-04-16T17:19:38.305 INFO:journalctl@ceph.osd.8.smithi117.stdout:Apr 16 17:19:37 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-8[55817]: 2024-04-16T17:19:37.887+0000 7f65ab398700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-04-16T17:19:11.565691+0000 front 2024-04-16T17:19:11.565594+0000 (oldest deadline 2024-04-16T17:19:34.465535+0000) 2024-04-16T17:19:38.565 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:19:38 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:19:38.219+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-16T17:18:36.158791+0000 front 2024-04-16T17:18:36.158992+0000 (oldest deadline 2024-04-16T17:19:00.258603+0000) 2024-04-16T17:19:38.565 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:19:38 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:19:38.219+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-16T17:18:44.260046+0000 front 2024-04-16T17:18:44.260171+0000 (oldest deadline 2024-04-16T17:19:07.160168+0000) 2024-04-16T17:19:38.565 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:19:38 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:19:38.219+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-16T17:18:54.062069+0000 front 2024-04-16T17:18:54.061971+0000 (oldest deadline 2024-04-16T17:19:18.761808+0000) 2024-04-16T17:19:38.565 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:19:38 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:19:38.219+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-16T17:19:01.063265+0000 front 2024-04-16T17:19:01.063419+0000 (oldest deadline 2024-04-16T17:19:24.563191+0000) 2024-04-16T17:19:38.565 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:19:38 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:19:38.219+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-04-16T17:19:10.865363+0000 front 2024-04-16T17:19:10.865389+0000 (oldest deadline 2024-04-16T17:19:35.466166+0000) 2024-04-16T17:19:38.727 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 16 17:19:38 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-9[66545]: 2024-04-16T17:19:38.338+0000 7f5937600700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-16T17:18:36.968505+0000 front 2024-04-16T17:18:36.968670+0000 (oldest deadline 2024-04-16T17:19:02.268762+0000) 2024-04-16T17:19:38.727 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 16 17:19:38 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-9[66545]: 2024-04-16T17:19:38.338+0000 7f5937600700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-16T17:18:43.371969+0000 front 2024-04-16T17:18:43.371904+0000 (oldest deadline 2024-04-16T17:19:08.671642+0000) 2024-04-16T17:19:38.727 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 16 17:19:38 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-9[66545]: 2024-04-16T17:19:38.338+0000 7f5937600700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-16T17:18:53.373968+0000 front 2024-04-16T17:18:53.374043+0000 (oldest deadline 2024-04-16T17:19:15.673755+0000) 2024-04-16T17:19:38.727 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 16 17:19:38 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-9[66545]: 2024-04-16T17:19:38.338+0000 7f5937600700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-16T17:19:01.880446+0000 front 2024-04-16T17:19:01.880936+0000 (oldest deadline 2024-04-16T17:19:27.180477+0000) 2024-04-16T17:19:38.727 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 16 17:19:38 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-9[66545]: 2024-04-16T17:19:38.338+0000 7f5937600700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-04-16T17:19:09.482762+0000 front 2024-04-16T17:19:09.482816+0000 (oldest deadline 2024-04-16T17:19:32.982945+0000) 2024-04-16T17:19:38.804 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:19:38 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:19:38.494+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-16T17:18:34.374061+0000 front 2024-04-16T17:18:34.374275+0000 (oldest deadline 2024-04-16T17:18:57.873879+0000) 2024-04-16T17:19:38.804 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:19:38 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:19:38.494+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-16T17:18:50.676854+0000 front 2024-04-16T17:18:50.676874+0000 (oldest deadline 2024-04-16T17:19:15.977004+0000) 2024-04-16T17:19:38.804 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:19:38 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:19:38.494+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-16T17:19:01.878146+0000 front 2024-04-16T17:19:01.878262+0000 (oldest deadline 2024-04-16T17:19:27.178048+0000) 2024-04-16T17:19:38.804 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:19:38 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:19:38.494+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-04-16T17:19:07.178684+0000 front 2024-04-16T17:19:07.178598+0000 (oldest deadline 2024-04-16T17:19:33.078600+0000) 2024-04-16T17:19:38.860 INFO:journalctl@ceph.osd.7.smithi104.stdout:Apr 16 17:19:38 smithi104 podman[74894]: 2024-04-16 17:19:38.565928773 +0000 UTC m=+5.450325752 container died 8eb3a9b475d1564ab96d7ad5f219433bdc41bc37f4d659e5bc9abec5d604cc71 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-7, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, org.label-schema.build-date=20240409, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-04-16T17:19:39.152 INFO:journalctl@ceph.osd.7.smithi104.stdout:Apr 16 17:19:38 smithi104 podman[74894]: 2024-04-16 17:19:38.976814832 +0000 UTC m=+5.861211786 container cleanup 8eb3a9b475d1564ab96d7ad5f219433bdc41bc37f4d659e5bc9abec5d604cc71 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-7, org.label-schema.build-date=20240409, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-16T17:19:39.152 INFO:journalctl@ceph.osd.7.smithi104.stdout:Apr 16 17:19:38 smithi104 bash[74894]: ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-7 2024-04-16T17:19:39.304 INFO:journalctl@ceph.osd.8.smithi117.stdout:Apr 16 17:19:38 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-8[55817]: 2024-04-16T17:19:38.910+0000 7f65ab398700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-16T17:18:41.860244+0000 front 2024-04-16T17:18:41.860176+0000 (oldest deadline 2024-04-16T17:19:07.760349+0000) 2024-04-16T17:19:39.305 INFO:journalctl@ceph.osd.8.smithi117.stdout:Apr 16 17:19:38 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-8[55817]: 2024-04-16T17:19:38.910+0000 7f65ab398700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-16T17:18:52.461265+0000 front 2024-04-16T17:18:52.461355+0000 (oldest deadline 2024-04-16T17:19:20.662233+0000) 2024-04-16T17:19:39.305 INFO:journalctl@ceph.osd.8.smithi117.stdout:Apr 16 17:19:38 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-8[55817]: 2024-04-16T17:19:38.910+0000 7f65ab398700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-16T17:19:00.663078+0000 front 2024-04-16T17:19:00.662927+0000 (oldest deadline 2024-04-16T17:19:24.762742+0000) 2024-04-16T17:19:39.305 INFO:journalctl@ceph.osd.8.smithi117.stdout:Apr 16 17:19:38 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-8[55817]: 2024-04-16T17:19:38.910+0000 7f65ab398700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-04-16T17:19:11.565691+0000 front 2024-04-16T17:19:11.565594+0000 (oldest deadline 2024-04-16T17:19:34.465535+0000) 2024-04-16T17:19:39.426 INFO:journalctl@ceph.osd.7.smithi104.stdout:Apr 16 17:19:39 smithi104 podman[74911]: 2024-04-16 17:19:39.153148036 +0000 UTC m=+0.584627108 container remove 8eb3a9b475d1564ab96d7ad5f219433bdc41bc37f4d659e5bc9abec5d604cc71 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-7, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240409, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-04-16T17:19:39.427 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:19:39 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:19:39.185+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-16T17:18:36.158791+0000 front 2024-04-16T17:18:36.158992+0000 (oldest deadline 2024-04-16T17:19:00.258603+0000) 2024-04-16T17:19:39.427 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:19:39 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:19:39.185+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-16T17:18:44.260046+0000 front 2024-04-16T17:18:44.260171+0000 (oldest deadline 2024-04-16T17:19:07.160168+0000) 2024-04-16T17:19:39.427 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:19:39 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:19:39.185+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-16T17:18:54.062069+0000 front 2024-04-16T17:18:54.061971+0000 (oldest deadline 2024-04-16T17:19:18.761808+0000) 2024-04-16T17:19:39.427 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:19:39 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:19:39.185+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-16T17:19:01.063265+0000 front 2024-04-16T17:19:01.063419+0000 (oldest deadline 2024-04-16T17:19:24.563191+0000) 2024-04-16T17:19:39.427 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:19:39 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:19:39.185+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-04-16T17:19:10.865363+0000 front 2024-04-16T17:19:10.865389+0000 (oldest deadline 2024-04-16T17:19:35.466166+0000) 2024-04-16T17:19:39.727 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 16 17:19:39 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-9[66545]: 2024-04-16T17:19:39.342+0000 7f5937600700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-16T17:18:36.968505+0000 front 2024-04-16T17:18:36.968670+0000 (oldest deadline 2024-04-16T17:19:02.268762+0000) 2024-04-16T17:19:39.727 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 16 17:19:39 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-9[66545]: 2024-04-16T17:19:39.342+0000 7f5937600700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-16T17:18:43.371969+0000 front 2024-04-16T17:18:43.371904+0000 (oldest deadline 2024-04-16T17:19:08.671642+0000) 2024-04-16T17:19:39.727 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 16 17:19:39 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-9[66545]: 2024-04-16T17:19:39.342+0000 7f5937600700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-16T17:18:53.373968+0000 front 2024-04-16T17:18:53.374043+0000 (oldest deadline 2024-04-16T17:19:15.673755+0000) 2024-04-16T17:19:39.727 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 16 17:19:39 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-9[66545]: 2024-04-16T17:19:39.342+0000 7f5937600700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-16T17:19:01.880446+0000 front 2024-04-16T17:19:01.880936+0000 (oldest deadline 2024-04-16T17:19:27.180477+0000) 2024-04-16T17:19:39.727 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 16 17:19:39 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-9[66545]: 2024-04-16T17:19:39.342+0000 7f5937600700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-04-16T17:19:09.482762+0000 front 2024-04-16T17:19:09.482816+0000 (oldest deadline 2024-04-16T17:19:32.982945+0000) 2024-04-16T17:19:39.804 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:19:39 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:19:39.502+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-16T17:18:34.374061+0000 front 2024-04-16T17:18:34.374275+0000 (oldest deadline 2024-04-16T17:18:57.873879+0000) 2024-04-16T17:19:39.805 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:19:39 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:19:39.502+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-16T17:18:50.676854+0000 front 2024-04-16T17:18:50.676874+0000 (oldest deadline 2024-04-16T17:19:15.977004+0000) 2024-04-16T17:19:39.805 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:19:39 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:19:39.502+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-16T17:19:01.878146+0000 front 2024-04-16T17:19:01.878262+0000 (oldest deadline 2024-04-16T17:19:27.178048+0000) 2024-04-16T17:19:39.805 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:19:39 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:19:39.502+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-04-16T17:19:07.178684+0000 front 2024-04-16T17:19:07.178598+0000 (oldest deadline 2024-04-16T17:19:33.078600+0000) 2024-04-16T17:19:39.813 INFO:journalctl@ceph.osd.7.smithi104.stdout:Apr 16 17:19:39 smithi104 podman[75004]: 2024-04-16 17:19:39.457705071 +0000 UTC m=+0.017803093 image pull e5ce10bdffbeeffffd1125f67b369e2151f0f17cdec854e93b6adf9321f65907 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2 2024-04-16T17:19:39.813 INFO:journalctl@ceph.osd.7.smithi104.stdout:Apr 16 17:19:39 smithi104 podman[75004]: 2024-04-16 17:19:39.57895678 +0000 UTC m=+0.139054837 container create 133280cf62ea626493aa7130c480822bd43f428db43bc7032ab89ccc1964004c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-7-deactivate, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240409, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, CEPH_POINT_RELEASE=) 2024-04-16T17:19:39.813 INFO:journalctl@ceph.osd.7.smithi104.stdout:Apr 16 17:19:39 smithi104 podman[75004]: 2024-04-16 17:19:39.814548699 +0000 UTC m=+0.374646716 container init 133280cf62ea626493aa7130c480822bd43f428db43bc7032ab89ccc1964004c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-7-deactivate, CEPH_POINT_RELEASE=, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240409, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-04-16T17:19:40.107 INFO:journalctl@ceph.osd.7.smithi104.stdout:Apr 16 17:19:39 smithi104 podman[75004]: 2024-04-16 17:19:39.817843774 +0000 UTC m=+0.377941795 container start 133280cf62ea626493aa7130c480822bd43f428db43bc7032ab89ccc1964004c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-7-deactivate, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240409) 2024-04-16T17:19:40.107 INFO:journalctl@ceph.osd.7.smithi104.stdout:Apr 16 17:19:39 smithi104 podman[75004]: 2024-04-16 17:19:39.922897843 +0000 UTC m=+0.482995868 container attach 133280cf62ea626493aa7130c480822bd43f428db43bc7032ab89ccc1964004c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-7-deactivate, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, io.buildah.version=1.33.5, org.label-schema.build-date=20240409, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-04-16T17:19:40.304 INFO:journalctl@ceph.osd.8.smithi117.stdout:Apr 16 17:19:39 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-8[55817]: 2024-04-16T17:19:39.892+0000 7f65ab398700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-16T17:18:41.860244+0000 front 2024-04-16T17:18:41.860176+0000 (oldest deadline 2024-04-16T17:19:07.760349+0000) 2024-04-16T17:19:40.304 INFO:journalctl@ceph.osd.8.smithi117.stdout:Apr 16 17:19:39 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-8[55817]: 2024-04-16T17:19:39.892+0000 7f65ab398700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-16T17:18:52.461265+0000 front 2024-04-16T17:18:52.461355+0000 (oldest deadline 2024-04-16T17:19:20.662233+0000) 2024-04-16T17:19:40.305 INFO:journalctl@ceph.osd.8.smithi117.stdout:Apr 16 17:19:39 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-8[55817]: 2024-04-16T17:19:39.892+0000 7f65ab398700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-16T17:19:00.663078+0000 front 2024-04-16T17:19:00.662927+0000 (oldest deadline 2024-04-16T17:19:24.762742+0000) 2024-04-16T17:19:40.305 INFO:journalctl@ceph.osd.8.smithi117.stdout:Apr 16 17:19:39 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-8[55817]: 2024-04-16T17:19:39.892+0000 7f65ab398700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-04-16T17:19:11.565691+0000 front 2024-04-16T17:19:11.565594+0000 (oldest deadline 2024-04-16T17:19:34.465535+0000) 2024-04-16T17:19:40.379 INFO:journalctl@ceph.osd.7.smithi104.stdout:Apr 16 17:19:40 smithi104 podman[75004]: 2024-04-16 17:19:40.108706116 +0000 UTC m=+0.668804134 container died 133280cf62ea626493aa7130c480822bd43f428db43bc7032ab89ccc1964004c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-7-deactivate, org.label-schema.build-date=20240409, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0) 2024-04-16T17:19:40.379 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:19:40 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:19:40.150+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-16T17:18:36.158791+0000 front 2024-04-16T17:18:36.158992+0000 (oldest deadline 2024-04-16T17:19:00.258603+0000) 2024-04-16T17:19:40.380 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:19:40 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:19:40.150+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-16T17:18:44.260046+0000 front 2024-04-16T17:18:44.260171+0000 (oldest deadline 2024-04-16T17:19:07.160168+0000) 2024-04-16T17:19:40.380 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:19:40 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:19:40.150+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-16T17:18:54.062069+0000 front 2024-04-16T17:18:54.061971+0000 (oldest deadline 2024-04-16T17:19:18.761808+0000) 2024-04-16T17:19:40.380 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:19:40 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:19:40.150+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-16T17:19:01.063265+0000 front 2024-04-16T17:19:01.063419+0000 (oldest deadline 2024-04-16T17:19:24.563191+0000) 2024-04-16T17:19:40.380 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:19:40 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:19:40.150+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-04-16T17:19:10.865363+0000 front 2024-04-16T17:19:10.865389+0000 (oldest deadline 2024-04-16T17:19:35.466166+0000) 2024-04-16T17:19:40.699 DEBUG:teuthology.orchestra.run.smithi104:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b1fecd38-fc11-11ee-bc90-c7b262605968@osd.7.service' 2024-04-16T17:19:40.727 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 16 17:19:40 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-9[66545]: 2024-04-16T17:19:40.295+0000 7f5937600700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-16T17:18:36.968505+0000 front 2024-04-16T17:18:36.968670+0000 (oldest deadline 2024-04-16T17:19:02.268762+0000) 2024-04-16T17:19:40.727 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 16 17:19:40 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-9[66545]: 2024-04-16T17:19:40.295+0000 7f5937600700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-16T17:18:43.371969+0000 front 2024-04-16T17:18:43.371904+0000 (oldest deadline 2024-04-16T17:19:08.671642+0000) 2024-04-16T17:19:40.727 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 16 17:19:40 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-9[66545]: 2024-04-16T17:19:40.295+0000 7f5937600700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-16T17:18:53.373968+0000 front 2024-04-16T17:18:53.374043+0000 (oldest deadline 2024-04-16T17:19:15.673755+0000) 2024-04-16T17:19:40.727 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 16 17:19:40 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-9[66545]: 2024-04-16T17:19:40.295+0000 7f5937600700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-16T17:19:01.880446+0000 front 2024-04-16T17:19:01.880936+0000 (oldest deadline 2024-04-16T17:19:27.180477+0000) 2024-04-16T17:19:40.728 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 16 17:19:40 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-9[66545]: 2024-04-16T17:19:40.295+0000 7f5937600700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-04-16T17:19:09.482762+0000 front 2024-04-16T17:19:09.482816+0000 (oldest deadline 2024-04-16T17:19:32.982945+0000) 2024-04-16T17:19:40.741 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-16T17:19:40.741 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-04-16T17:19:40.742 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-04-16T17:19:40.742 DEBUG:teuthology.orchestra.run.smithi117:> sudo systemctl stop ceph-b1fecd38-fc11-11ee-bc90-c7b262605968@osd.8 2024-04-16T17:19:40.771 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:19:40 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:19:40.512+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-16T17:18:34.374061+0000 front 2024-04-16T17:18:34.374275+0000 (oldest deadline 2024-04-16T17:18:57.873879+0000) 2024-04-16T17:19:40.772 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:19:40 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:19:40.512+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-16T17:18:50.676854+0000 front 2024-04-16T17:18:50.676874+0000 (oldest deadline 2024-04-16T17:19:15.977004+0000) 2024-04-16T17:19:40.772 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:19:40 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:19:40.512+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-16T17:19:01.878146+0000 front 2024-04-16T17:19:01.878262+0000 (oldest deadline 2024-04-16T17:19:27.178048+0000) 2024-04-16T17:19:40.772 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:19:40 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:19:40.512+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-04-16T17:19:07.178684+0000 front 2024-04-16T17:19:07.178598+0000 (oldest deadline 2024-04-16T17:19:33.078600+0000) 2024-04-16T17:19:41.037 INFO:journalctl@ceph.osd.8.smithi117.stdout:Apr 16 17:19:40 smithi117 systemd[1]: Stopping Ceph osd.8 for b1fecd38-fc11-11ee-bc90-c7b262605968... 2024-04-16T17:19:41.037 INFO:journalctl@ceph.osd.8.smithi117.stdout:Apr 16 17:19:40 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-8[55817]: 2024-04-16T17:19:40.867+0000 7f65ab398700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-16T17:18:41.860244+0000 front 2024-04-16T17:18:41.860176+0000 (oldest deadline 2024-04-16T17:19:07.760349+0000) 2024-04-16T17:19:41.038 INFO:journalctl@ceph.osd.8.smithi117.stdout:Apr 16 17:19:40 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-8[55817]: 2024-04-16T17:19:40.867+0000 7f65ab398700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-16T17:18:52.461265+0000 front 2024-04-16T17:18:52.461355+0000 (oldest deadline 2024-04-16T17:19:20.662233+0000) 2024-04-16T17:19:41.038 INFO:journalctl@ceph.osd.8.smithi117.stdout:Apr 16 17:19:40 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-8[55817]: 2024-04-16T17:19:40.867+0000 7f65ab398700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-16T17:19:00.663078+0000 front 2024-04-16T17:19:00.662927+0000 (oldest deadline 2024-04-16T17:19:24.762742+0000) 2024-04-16T17:19:41.038 INFO:journalctl@ceph.osd.8.smithi117.stdout:Apr 16 17:19:40 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-8[55817]: 2024-04-16T17:19:40.867+0000 7f65ab398700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-04-16T17:19:11.565691+0000 front 2024-04-16T17:19:11.565594+0000 (oldest deadline 2024-04-16T17:19:34.465535+0000) 2024-04-16T17:19:41.038 INFO:journalctl@ceph.osd.8.smithi117.stdout:Apr 16 17:19:41 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-8[55817]: 2024-04-16T17:19:41.037+0000 7f65b2dac700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-16T17:19:41.038 INFO:journalctl@ceph.osd.8.smithi117.stdout:Apr 16 17:19:41 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-8[55817]: 2024-04-16T17:19:41.037+0000 7f65b2dac700 -1 osd.8 112 *** Got signal Terminated *** 2024-04-16T17:19:41.038 INFO:journalctl@ceph.osd.8.smithi117.stdout:Apr 16 17:19:41 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-8[55817]: 2024-04-16T17:19:41.037+0000 7f65b2dac700 -1 osd.8 112 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-16T17:19:41.427 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:19:41 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:19:41.170+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-16T17:18:36.158791+0000 front 2024-04-16T17:18:36.158992+0000 (oldest deadline 2024-04-16T17:19:00.258603+0000) 2024-04-16T17:19:41.427 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:19:41 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:19:41.170+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-16T17:18:44.260046+0000 front 2024-04-16T17:18:44.260171+0000 (oldest deadline 2024-04-16T17:19:07.160168+0000) 2024-04-16T17:19:41.427 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:19:41 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:19:41.170+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-16T17:18:54.062069+0000 front 2024-04-16T17:18:54.061971+0000 (oldest deadline 2024-04-16T17:19:18.761808+0000) 2024-04-16T17:19:41.427 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:19:41 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:19:41.170+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-16T17:19:01.063265+0000 front 2024-04-16T17:19:01.063419+0000 (oldest deadline 2024-04-16T17:19:24.563191+0000) 2024-04-16T17:19:41.428 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:19:41 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:19:41.170+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-04-16T17:19:10.865363+0000 front 2024-04-16T17:19:10.865389+0000 (oldest deadline 2024-04-16T17:19:35.466166+0000) 2024-04-16T17:19:41.730 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 16 17:19:41 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-9[66545]: 2024-04-16T17:19:41.292+0000 7f5937600700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-16T17:18:36.968505+0000 front 2024-04-16T17:18:36.968670+0000 (oldest deadline 2024-04-16T17:19:02.268762+0000) 2024-04-16T17:19:41.730 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 16 17:19:41 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-9[66545]: 2024-04-16T17:19:41.292+0000 7f5937600700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-16T17:18:43.371969+0000 front 2024-04-16T17:18:43.371904+0000 (oldest deadline 2024-04-16T17:19:08.671642+0000) 2024-04-16T17:19:41.730 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 16 17:19:41 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-9[66545]: 2024-04-16T17:19:41.292+0000 7f5937600700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-16T17:18:53.373968+0000 front 2024-04-16T17:18:53.374043+0000 (oldest deadline 2024-04-16T17:19:15.673755+0000) 2024-04-16T17:19:41.731 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 16 17:19:41 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-9[66545]: 2024-04-16T17:19:41.292+0000 7f5937600700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-16T17:19:01.880446+0000 front 2024-04-16T17:19:01.880936+0000 (oldest deadline 2024-04-16T17:19:27.180477+0000) 2024-04-16T17:19:41.731 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 16 17:19:41 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-9[66545]: 2024-04-16T17:19:41.292+0000 7f5937600700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-04-16T17:19:09.482762+0000 front 2024-04-16T17:19:09.482816+0000 (oldest deadline 2024-04-16T17:19:32.982945+0000) 2024-04-16T17:19:41.804 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:19:41 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:19:41.504+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-16T17:18:34.374061+0000 front 2024-04-16T17:18:34.374275+0000 (oldest deadline 2024-04-16T17:18:57.873879+0000) 2024-04-16T17:19:41.804 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:19:41 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:19:41.504+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-16T17:18:50.676854+0000 front 2024-04-16T17:18:50.676874+0000 (oldest deadline 2024-04-16T17:19:15.977004+0000) 2024-04-16T17:19:41.805 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:19:41 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:19:41.504+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-16T17:19:01.878146+0000 front 2024-04-16T17:19:01.878262+0000 (oldest deadline 2024-04-16T17:19:27.178048+0000) 2024-04-16T17:19:41.805 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:19:41 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:19:41.504+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-04-16T17:19:07.178684+0000 front 2024-04-16T17:19:07.178598+0000 (oldest deadline 2024-04-16T17:19:33.078600+0000) 2024-04-16T17:19:42.304 INFO:journalctl@ceph.osd.8.smithi117.stdout:Apr 16 17:19:41 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-8[55817]: 2024-04-16T17:19:41.847+0000 7f65ab398700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-16T17:18:41.860244+0000 front 2024-04-16T17:18:41.860176+0000 (oldest deadline 2024-04-16T17:19:07.760349+0000) 2024-04-16T17:19:42.305 INFO:journalctl@ceph.osd.8.smithi117.stdout:Apr 16 17:19:41 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-8[55817]: 2024-04-16T17:19:41.847+0000 7f65ab398700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-16T17:18:52.461265+0000 front 2024-04-16T17:18:52.461355+0000 (oldest deadline 2024-04-16T17:19:20.662233+0000) 2024-04-16T17:19:42.305 INFO:journalctl@ceph.osd.8.smithi117.stdout:Apr 16 17:19:41 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-8[55817]: 2024-04-16T17:19:41.847+0000 7f65ab398700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-16T17:19:00.663078+0000 front 2024-04-16T17:19:00.662927+0000 (oldest deadline 2024-04-16T17:19:24.762742+0000) 2024-04-16T17:19:42.305 INFO:journalctl@ceph.osd.8.smithi117.stdout:Apr 16 17:19:41 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-8[55817]: 2024-04-16T17:19:41.847+0000 7f65ab398700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-04-16T17:19:11.565691+0000 front 2024-04-16T17:19:11.565594+0000 (oldest deadline 2024-04-16T17:19:34.465535+0000) 2024-04-16T17:19:42.427 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:19:42 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:19:42.131+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-16T17:18:36.158791+0000 front 2024-04-16T17:18:36.158992+0000 (oldest deadline 2024-04-16T17:19:00.258603+0000) 2024-04-16T17:19:42.427 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:19:42 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:19:42.131+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-16T17:18:44.260046+0000 front 2024-04-16T17:18:44.260171+0000 (oldest deadline 2024-04-16T17:19:07.160168+0000) 2024-04-16T17:19:42.427 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:19:42 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:19:42.131+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-16T17:18:54.062069+0000 front 2024-04-16T17:18:54.061971+0000 (oldest deadline 2024-04-16T17:19:18.761808+0000) 2024-04-16T17:19:42.427 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:19:42 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:19:42.131+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-16T17:19:01.063265+0000 front 2024-04-16T17:19:01.063419+0000 (oldest deadline 2024-04-16T17:19:24.563191+0000) 2024-04-16T17:19:42.427 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:19:42 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:19:42.131+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-04-16T17:19:10.865363+0000 front 2024-04-16T17:19:10.865389+0000 (oldest deadline 2024-04-16T17:19:35.466166+0000) 2024-04-16T17:19:42.727 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 16 17:19:42 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-9[66545]: 2024-04-16T17:19:42.286+0000 7f5937600700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-16T17:18:36.968505+0000 front 2024-04-16T17:18:36.968670+0000 (oldest deadline 2024-04-16T17:19:02.268762+0000) 2024-04-16T17:19:42.727 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 16 17:19:42 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-9[66545]: 2024-04-16T17:19:42.286+0000 7f5937600700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-16T17:18:43.371969+0000 front 2024-04-16T17:18:43.371904+0000 (oldest deadline 2024-04-16T17:19:08.671642+0000) 2024-04-16T17:19:42.727 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 16 17:19:42 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-9[66545]: 2024-04-16T17:19:42.286+0000 7f5937600700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-16T17:18:53.373968+0000 front 2024-04-16T17:18:53.374043+0000 (oldest deadline 2024-04-16T17:19:15.673755+0000) 2024-04-16T17:19:42.727 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 16 17:19:42 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-9[66545]: 2024-04-16T17:19:42.286+0000 7f5937600700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-16T17:19:01.880446+0000 front 2024-04-16T17:19:01.880936+0000 (oldest deadline 2024-04-16T17:19:27.180477+0000) 2024-04-16T17:19:42.728 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 16 17:19:42 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-9[66545]: 2024-04-16T17:19:42.286+0000 7f5937600700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-04-16T17:19:09.482762+0000 front 2024-04-16T17:19:09.482816+0000 (oldest deadline 2024-04-16T17:19:32.982945+0000) 2024-04-16T17:19:42.804 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:19:42 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:19:42.540+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-16T17:18:34.374061+0000 front 2024-04-16T17:18:34.374275+0000 (oldest deadline 2024-04-16T17:18:57.873879+0000) 2024-04-16T17:19:42.804 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:19:42 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:19:42.540+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-16T17:18:50.676854+0000 front 2024-04-16T17:18:50.676874+0000 (oldest deadline 2024-04-16T17:19:15.977004+0000) 2024-04-16T17:19:42.805 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:19:42 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:19:42.540+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-16T17:19:01.878146+0000 front 2024-04-16T17:19:01.878262+0000 (oldest deadline 2024-04-16T17:19:27.178048+0000) 2024-04-16T17:19:42.805 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:19:42 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:19:42.540+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-04-16T17:19:07.178684+0000 front 2024-04-16T17:19:07.178598+0000 (oldest deadline 2024-04-16T17:19:33.078600+0000) 2024-04-16T17:19:43.304 INFO:journalctl@ceph.osd.8.smithi117.stdout:Apr 16 17:19:42 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-8[55817]: 2024-04-16T17:19:42.854+0000 7f65ab398700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-16T17:18:41.860244+0000 front 2024-04-16T17:18:41.860176+0000 (oldest deadline 2024-04-16T17:19:07.760349+0000) 2024-04-16T17:19:43.305 INFO:journalctl@ceph.osd.8.smithi117.stdout:Apr 16 17:19:42 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-8[55817]: 2024-04-16T17:19:42.854+0000 7f65ab398700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-16T17:18:52.461265+0000 front 2024-04-16T17:18:52.461355+0000 (oldest deadline 2024-04-16T17:19:20.662233+0000) 2024-04-16T17:19:43.305 INFO:journalctl@ceph.osd.8.smithi117.stdout:Apr 16 17:19:42 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-8[55817]: 2024-04-16T17:19:42.854+0000 7f65ab398700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-16T17:19:00.663078+0000 front 2024-04-16T17:19:00.662927+0000 (oldest deadline 2024-04-16T17:19:24.762742+0000) 2024-04-16T17:19:43.305 INFO:journalctl@ceph.osd.8.smithi117.stdout:Apr 16 17:19:42 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-8[55817]: 2024-04-16T17:19:42.854+0000 7f65ab398700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-04-16T17:19:11.565691+0000 front 2024-04-16T17:19:11.565594+0000 (oldest deadline 2024-04-16T17:19:34.465535+0000) 2024-04-16T17:19:43.427 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:19:43 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:19:43.090+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-16T17:18:36.158791+0000 front 2024-04-16T17:18:36.158992+0000 (oldest deadline 2024-04-16T17:19:00.258603+0000) 2024-04-16T17:19:43.427 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:19:43 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:19:43.090+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-16T17:18:44.260046+0000 front 2024-04-16T17:18:44.260171+0000 (oldest deadline 2024-04-16T17:19:07.160168+0000) 2024-04-16T17:19:43.427 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:19:43 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:19:43.090+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-16T17:18:54.062069+0000 front 2024-04-16T17:18:54.061971+0000 (oldest deadline 2024-04-16T17:19:18.761808+0000) 2024-04-16T17:19:43.427 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:19:43 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:19:43.090+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-16T17:19:01.063265+0000 front 2024-04-16T17:19:01.063419+0000 (oldest deadline 2024-04-16T17:19:24.563191+0000) 2024-04-16T17:19:43.428 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:19:43 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:19:43.090+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-04-16T17:19:10.865363+0000 front 2024-04-16T17:19:10.865389+0000 (oldest deadline 2024-04-16T17:19:35.466166+0000) 2024-04-16T17:19:43.727 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 16 17:19:43 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-9[66545]: 2024-04-16T17:19:43.320+0000 7f5937600700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-16T17:18:36.968505+0000 front 2024-04-16T17:18:36.968670+0000 (oldest deadline 2024-04-16T17:19:02.268762+0000) 2024-04-16T17:19:43.727 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 16 17:19:43 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-9[66545]: 2024-04-16T17:19:43.320+0000 7f5937600700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-16T17:18:43.371969+0000 front 2024-04-16T17:18:43.371904+0000 (oldest deadline 2024-04-16T17:19:08.671642+0000) 2024-04-16T17:19:43.728 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 16 17:19:43 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-9[66545]: 2024-04-16T17:19:43.320+0000 7f5937600700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-16T17:18:53.373968+0000 front 2024-04-16T17:18:53.374043+0000 (oldest deadline 2024-04-16T17:19:15.673755+0000) 2024-04-16T17:19:43.728 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 16 17:19:43 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-9[66545]: 2024-04-16T17:19:43.320+0000 7f5937600700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-16T17:19:01.880446+0000 front 2024-04-16T17:19:01.880936+0000 (oldest deadline 2024-04-16T17:19:27.180477+0000) 2024-04-16T17:19:43.728 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 16 17:19:43 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-9[66545]: 2024-04-16T17:19:43.320+0000 7f5937600700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-04-16T17:19:09.482762+0000 front 2024-04-16T17:19:09.482816+0000 (oldest deadline 2024-04-16T17:19:32.982945+0000) 2024-04-16T17:19:43.805 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:19:43 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:19:43.547+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-16T17:18:34.374061+0000 front 2024-04-16T17:18:34.374275+0000 (oldest deadline 2024-04-16T17:18:57.873879+0000) 2024-04-16T17:19:43.805 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:19:43 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:19:43.547+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-16T17:18:50.676854+0000 front 2024-04-16T17:18:50.676874+0000 (oldest deadline 2024-04-16T17:19:15.977004+0000) 2024-04-16T17:19:43.805 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:19:43 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:19:43.547+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-16T17:19:01.878146+0000 front 2024-04-16T17:19:01.878262+0000 (oldest deadline 2024-04-16T17:19:27.178048+0000) 2024-04-16T17:19:43.805 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:19:43 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:19:43.547+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-04-16T17:19:07.178684+0000 front 2024-04-16T17:19:07.178598+0000 (oldest deadline 2024-04-16T17:19:33.078600+0000) 2024-04-16T17:19:43.805 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:19:43 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:19:43.547+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.117:6812 osd.5 since back 2024-04-16T17:19:18.379616+0000 front 2024-04-16T17:19:18.379681+0000 (oldest deadline 2024-04-16T17:19:43.079667+0000) 2024-04-16T17:19:44.304 INFO:journalctl@ceph.osd.8.smithi117.stdout:Apr 16 17:19:43 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-8[55817]: 2024-04-16T17:19:43.852+0000 7f65ab398700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-16T17:18:41.860244+0000 front 2024-04-16T17:18:41.860176+0000 (oldest deadline 2024-04-16T17:19:07.760349+0000) 2024-04-16T17:19:44.305 INFO:journalctl@ceph.osd.8.smithi117.stdout:Apr 16 17:19:43 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-8[55817]: 2024-04-16T17:19:43.852+0000 7f65ab398700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-16T17:18:52.461265+0000 front 2024-04-16T17:18:52.461355+0000 (oldest deadline 2024-04-16T17:19:20.662233+0000) 2024-04-16T17:19:44.305 INFO:journalctl@ceph.osd.8.smithi117.stdout:Apr 16 17:19:43 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-8[55817]: 2024-04-16T17:19:43.852+0000 7f65ab398700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-16T17:19:00.663078+0000 front 2024-04-16T17:19:00.662927+0000 (oldest deadline 2024-04-16T17:19:24.762742+0000) 2024-04-16T17:19:44.305 INFO:journalctl@ceph.osd.8.smithi117.stdout:Apr 16 17:19:43 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-8[55817]: 2024-04-16T17:19:43.852+0000 7f65ab398700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-04-16T17:19:11.565691+0000 front 2024-04-16T17:19:11.565594+0000 (oldest deadline 2024-04-16T17:19:34.465535+0000) 2024-04-16T17:19:44.427 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:19:44 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:19:44.094+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-16T17:18:36.158791+0000 front 2024-04-16T17:18:36.158992+0000 (oldest deadline 2024-04-16T17:19:00.258603+0000) 2024-04-16T17:19:44.427 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:19:44 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:19:44.094+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-16T17:18:44.260046+0000 front 2024-04-16T17:18:44.260171+0000 (oldest deadline 2024-04-16T17:19:07.160168+0000) 2024-04-16T17:19:44.427 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:19:44 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:19:44.094+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-16T17:18:54.062069+0000 front 2024-04-16T17:18:54.061971+0000 (oldest deadline 2024-04-16T17:19:18.761808+0000) 2024-04-16T17:19:44.427 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:19:44 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:19:44.094+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-16T17:19:01.063265+0000 front 2024-04-16T17:19:01.063419+0000 (oldest deadline 2024-04-16T17:19:24.563191+0000) 2024-04-16T17:19:44.428 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:19:44 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:19:44.094+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-04-16T17:19:10.865363+0000 front 2024-04-16T17:19:10.865389+0000 (oldest deadline 2024-04-16T17:19:35.466166+0000) 2024-04-16T17:19:44.727 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 16 17:19:44 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-9[66545]: 2024-04-16T17:19:44.352+0000 7f5937600700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-16T17:18:36.968505+0000 front 2024-04-16T17:18:36.968670+0000 (oldest deadline 2024-04-16T17:19:02.268762+0000) 2024-04-16T17:19:44.727 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 16 17:19:44 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-9[66545]: 2024-04-16T17:19:44.352+0000 7f5937600700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-16T17:18:43.371969+0000 front 2024-04-16T17:18:43.371904+0000 (oldest deadline 2024-04-16T17:19:08.671642+0000) 2024-04-16T17:19:44.727 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 16 17:19:44 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-9[66545]: 2024-04-16T17:19:44.352+0000 7f5937600700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-16T17:18:53.373968+0000 front 2024-04-16T17:18:53.374043+0000 (oldest deadline 2024-04-16T17:19:15.673755+0000) 2024-04-16T17:19:44.727 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 16 17:19:44 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-9[66545]: 2024-04-16T17:19:44.352+0000 7f5937600700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-16T17:19:01.880446+0000 front 2024-04-16T17:19:01.880936+0000 (oldest deadline 2024-04-16T17:19:27.180477+0000) 2024-04-16T17:19:44.727 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 16 17:19:44 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-9[66545]: 2024-04-16T17:19:44.352+0000 7f5937600700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-04-16T17:19:09.482762+0000 front 2024-04-16T17:19:09.482816+0000 (oldest deadline 2024-04-16T17:19:32.982945+0000) 2024-04-16T17:19:44.728 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 16 17:19:44 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-9[66545]: 2024-04-16T17:19:44.352+0000 7f5937600700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.117:6812 osd.5 since back 2024-04-16T17:19:15.284908+0000 front 2024-04-16T17:19:15.285136+0000 (oldest deadline 2024-04-16T17:19:44.085731+0000) 2024-04-16T17:19:44.846 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:19:44 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:19:44.567+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-16T17:18:34.374061+0000 front 2024-04-16T17:18:34.374275+0000 (oldest deadline 2024-04-16T17:18:57.873879+0000) 2024-04-16T17:19:44.847 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:19:44 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:19:44.567+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-16T17:18:50.676854+0000 front 2024-04-16T17:18:50.676874+0000 (oldest deadline 2024-04-16T17:19:15.977004+0000) 2024-04-16T17:19:44.847 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:19:44 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:19:44.567+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-16T17:19:01.878146+0000 front 2024-04-16T17:19:01.878262+0000 (oldest deadline 2024-04-16T17:19:27.178048+0000) 2024-04-16T17:19:44.847 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:19:44 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:19:44.567+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-04-16T17:19:07.178684+0000 front 2024-04-16T17:19:07.178598+0000 (oldest deadline 2024-04-16T17:19:33.078600+0000) 2024-04-16T17:19:44.847 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:19:44 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:19:44.567+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.117:6812 osd.5 since back 2024-04-16T17:19:18.379616+0000 front 2024-04-16T17:19:18.379681+0000 (oldest deadline 2024-04-16T17:19:43.079667+0000) 2024-04-16T17:19:45.304 INFO:journalctl@ceph.osd.8.smithi117.stdout:Apr 16 17:19:44 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-8[55817]: 2024-04-16T17:19:44.846+0000 7f65ab398700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-16T17:18:41.860244+0000 front 2024-04-16T17:18:41.860176+0000 (oldest deadline 2024-04-16T17:19:07.760349+0000) 2024-04-16T17:19:45.305 INFO:journalctl@ceph.osd.8.smithi117.stdout:Apr 16 17:19:44 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-8[55817]: 2024-04-16T17:19:44.846+0000 7f65ab398700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-16T17:18:52.461265+0000 front 2024-04-16T17:18:52.461355+0000 (oldest deadline 2024-04-16T17:19:20.662233+0000) 2024-04-16T17:19:45.305 INFO:journalctl@ceph.osd.8.smithi117.stdout:Apr 16 17:19:44 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-8[55817]: 2024-04-16T17:19:44.846+0000 7f65ab398700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-16T17:19:00.663078+0000 front 2024-04-16T17:19:00.662927+0000 (oldest deadline 2024-04-16T17:19:24.762742+0000) 2024-04-16T17:19:45.305 INFO:journalctl@ceph.osd.8.smithi117.stdout:Apr 16 17:19:44 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-8[55817]: 2024-04-16T17:19:44.846+0000 7f65ab398700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-04-16T17:19:11.565691+0000 front 2024-04-16T17:19:11.565594+0000 (oldest deadline 2024-04-16T17:19:34.465535+0000) 2024-04-16T17:19:45.427 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:19:45 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:19:45.118+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-16T17:18:36.158791+0000 front 2024-04-16T17:18:36.158992+0000 (oldest deadline 2024-04-16T17:19:00.258603+0000) 2024-04-16T17:19:45.427 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:19:45 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:19:45.118+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-16T17:18:44.260046+0000 front 2024-04-16T17:18:44.260171+0000 (oldest deadline 2024-04-16T17:19:07.160168+0000) 2024-04-16T17:19:45.427 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:19:45 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:19:45.118+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-16T17:18:54.062069+0000 front 2024-04-16T17:18:54.061971+0000 (oldest deadline 2024-04-16T17:19:18.761808+0000) 2024-04-16T17:19:45.427 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:19:45 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:19:45.118+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-16T17:19:01.063265+0000 front 2024-04-16T17:19:01.063419+0000 (oldest deadline 2024-04-16T17:19:24.563191+0000) 2024-04-16T17:19:45.427 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:19:45 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:19:45.118+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-04-16T17:19:10.865363+0000 front 2024-04-16T17:19:10.865389+0000 (oldest deadline 2024-04-16T17:19:35.466166+0000) 2024-04-16T17:19:45.427 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:19:45 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:19:45.118+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.117:6812 osd.5 since back 2024-04-16T17:19:19.567307+0000 front 2024-04-16T17:19:19.567365+0000 (oldest deadline 2024-04-16T17:19:44.267279+0000) 2024-04-16T17:19:45.726 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 16 17:19:45 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-9[66545]: 2024-04-16T17:19:45.346+0000 7f5937600700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-16T17:18:36.968505+0000 front 2024-04-16T17:18:36.968670+0000 (oldest deadline 2024-04-16T17:19:02.268762+0000) 2024-04-16T17:19:45.727 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 16 17:19:45 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-9[66545]: 2024-04-16T17:19:45.346+0000 7f5937600700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-16T17:18:43.371969+0000 front 2024-04-16T17:18:43.371904+0000 (oldest deadline 2024-04-16T17:19:08.671642+0000) 2024-04-16T17:19:45.727 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 16 17:19:45 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-9[66545]: 2024-04-16T17:19:45.346+0000 7f5937600700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-16T17:18:53.373968+0000 front 2024-04-16T17:18:53.374043+0000 (oldest deadline 2024-04-16T17:19:15.673755+0000) 2024-04-16T17:19:45.727 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 16 17:19:45 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-9[66545]: 2024-04-16T17:19:45.346+0000 7f5937600700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-16T17:19:01.880446+0000 front 2024-04-16T17:19:01.880936+0000 (oldest deadline 2024-04-16T17:19:27.180477+0000) 2024-04-16T17:19:45.727 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 16 17:19:45 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-9[66545]: 2024-04-16T17:19:45.346+0000 7f5937600700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-04-16T17:19:09.482762+0000 front 2024-04-16T17:19:09.482816+0000 (oldest deadline 2024-04-16T17:19:32.982945+0000) 2024-04-16T17:19:45.727 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 16 17:19:45 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-9[66545]: 2024-04-16T17:19:45.346+0000 7f5937600700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.117:6812 osd.5 since back 2024-04-16T17:19:15.284908+0000 front 2024-04-16T17:19:15.285136+0000 (oldest deadline 2024-04-16T17:19:44.085731+0000) 2024-04-16T17:19:46.054 INFO:journalctl@ceph.osd.8.smithi117.stdout:Apr 16 17:19:45 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-8[55817]: 2024-04-16T17:19:45.831+0000 7f65ab398700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-16T17:18:41.860244+0000 front 2024-04-16T17:18:41.860176+0000 (oldest deadline 2024-04-16T17:19:07.760349+0000) 2024-04-16T17:19:46.055 INFO:journalctl@ceph.osd.8.smithi117.stdout:Apr 16 17:19:45 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-8[55817]: 2024-04-16T17:19:45.831+0000 7f65ab398700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-16T17:18:52.461265+0000 front 2024-04-16T17:18:52.461355+0000 (oldest deadline 2024-04-16T17:19:20.662233+0000) 2024-04-16T17:19:46.055 INFO:journalctl@ceph.osd.8.smithi117.stdout:Apr 16 17:19:45 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-8[55817]: 2024-04-16T17:19:45.831+0000 7f65ab398700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-16T17:19:00.663078+0000 front 2024-04-16T17:19:00.662927+0000 (oldest deadline 2024-04-16T17:19:24.762742+0000) 2024-04-16T17:19:46.055 INFO:journalctl@ceph.osd.8.smithi117.stdout:Apr 16 17:19:45 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-8[55817]: 2024-04-16T17:19:45.831+0000 7f65ab398700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-04-16T17:19:11.565691+0000 front 2024-04-16T17:19:11.565594+0000 (oldest deadline 2024-04-16T17:19:34.465535+0000) 2024-04-16T17:19:46.055 INFO:journalctl@ceph.osd.8.smithi117.stdout:Apr 16 17:19:45 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-8[55817]: 2024-04-16T17:19:45.831+0000 7f65ab398700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.117:6812 osd.5 since back 2024-04-16T17:19:19.166518+0000 front 2024-04-16T17:19:19.166487+0000 (oldest deadline 2024-04-16T17:19:45.066678+0000) 2024-04-16T17:19:46.056 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:19:45 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:19:45.593+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-16T17:18:34.374061+0000 front 2024-04-16T17:18:34.374275+0000 (oldest deadline 2024-04-16T17:18:57.873879+0000) 2024-04-16T17:19:46.056 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:19:45 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:19:45.593+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-16T17:18:50.676854+0000 front 2024-04-16T17:18:50.676874+0000 (oldest deadline 2024-04-16T17:19:15.977004+0000) 2024-04-16T17:19:46.056 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:19:45 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:19:45.593+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-16T17:19:01.878146+0000 front 2024-04-16T17:19:01.878262+0000 (oldest deadline 2024-04-16T17:19:27.178048+0000) 2024-04-16T17:19:46.056 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:19:45 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:19:45.593+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-04-16T17:19:07.178684+0000 front 2024-04-16T17:19:07.178598+0000 (oldest deadline 2024-04-16T17:19:33.078600+0000) 2024-04-16T17:19:46.056 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:19:45 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:19:45.593+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.117:6812 osd.5 since back 2024-04-16T17:19:18.379616+0000 front 2024-04-16T17:19:18.379681+0000 (oldest deadline 2024-04-16T17:19:43.079667+0000) 2024-04-16T17:19:46.427 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:19:46 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:19:46.166+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-16T17:18:36.158791+0000 front 2024-04-16T17:18:36.158992+0000 (oldest deadline 2024-04-16T17:19:00.258603+0000) 2024-04-16T17:19:46.427 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:19:46 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:19:46.166+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-16T17:18:44.260046+0000 front 2024-04-16T17:18:44.260171+0000 (oldest deadline 2024-04-16T17:19:07.160168+0000) 2024-04-16T17:19:46.427 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:19:46 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:19:46.166+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-16T17:18:54.062069+0000 front 2024-04-16T17:18:54.061971+0000 (oldest deadline 2024-04-16T17:19:18.761808+0000) 2024-04-16T17:19:46.427 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:19:46 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:19:46.166+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-16T17:19:01.063265+0000 front 2024-04-16T17:19:01.063419+0000 (oldest deadline 2024-04-16T17:19:24.563191+0000) 2024-04-16T17:19:46.427 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:19:46 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:19:46.166+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-04-16T17:19:10.865363+0000 front 2024-04-16T17:19:10.865389+0000 (oldest deadline 2024-04-16T17:19:35.466166+0000) 2024-04-16T17:19:46.428 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:19:46 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:19:46.166+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.117:6812 osd.5 since back 2024-04-16T17:19:19.567307+0000 front 2024-04-16T17:19:19.567365+0000 (oldest deadline 2024-04-16T17:19:44.267279+0000) 2024-04-16T17:19:46.704 INFO:journalctl@ceph.osd.8.smithi117.stdout:Apr 16 17:19:46 smithi117 podman[75130]: 2024-04-16 17:19:46.400960735 +0000 UTC m=+5.462934080 container died f23150d642f1fc7a61d328248009ae2e531db7212280cdab5ef10c1911ffd1d2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-8, org.label-schema.build-date=20240409, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb) 2024-04-16T17:19:46.705 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:19:46 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:19:46.631+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-16T17:18:34.374061+0000 front 2024-04-16T17:18:34.374275+0000 (oldest deadline 2024-04-16T17:18:57.873879+0000) 2024-04-16T17:19:46.705 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:19:46 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:19:46.631+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-16T17:18:50.676854+0000 front 2024-04-16T17:18:50.676874+0000 (oldest deadline 2024-04-16T17:19:15.977004+0000) 2024-04-16T17:19:46.705 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:19:46 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:19:46.631+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-16T17:19:01.878146+0000 front 2024-04-16T17:19:01.878262+0000 (oldest deadline 2024-04-16T17:19:27.178048+0000) 2024-04-16T17:19:46.705 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:19:46 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:19:46.631+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-04-16T17:19:07.178684+0000 front 2024-04-16T17:19:07.178598+0000 (oldest deadline 2024-04-16T17:19:33.078600+0000) 2024-04-16T17:19:46.705 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:19:46 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:19:46.631+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.117:6812 osd.5 since back 2024-04-16T17:19:18.379616+0000 front 2024-04-16T17:19:18.379681+0000 (oldest deadline 2024-04-16T17:19:43.079667+0000) 2024-04-16T17:19:46.727 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 16 17:19:46 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-9[66545]: 2024-04-16T17:19:46.347+0000 7f5937600700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-16T17:18:36.968505+0000 front 2024-04-16T17:18:36.968670+0000 (oldest deadline 2024-04-16T17:19:02.268762+0000) 2024-04-16T17:19:46.727 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 16 17:19:46 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-9[66545]: 2024-04-16T17:19:46.347+0000 7f5937600700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-16T17:18:43.371969+0000 front 2024-04-16T17:18:43.371904+0000 (oldest deadline 2024-04-16T17:19:08.671642+0000) 2024-04-16T17:19:46.727 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 16 17:19:46 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-9[66545]: 2024-04-16T17:19:46.347+0000 7f5937600700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-16T17:18:53.373968+0000 front 2024-04-16T17:18:53.374043+0000 (oldest deadline 2024-04-16T17:19:15.673755+0000) 2024-04-16T17:19:46.727 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 16 17:19:46 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-9[66545]: 2024-04-16T17:19:46.347+0000 7f5937600700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-16T17:19:01.880446+0000 front 2024-04-16T17:19:01.880936+0000 (oldest deadline 2024-04-16T17:19:27.180477+0000) 2024-04-16T17:19:46.728 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 16 17:19:46 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-9[66545]: 2024-04-16T17:19:46.347+0000 7f5937600700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-04-16T17:19:09.482762+0000 front 2024-04-16T17:19:09.482816+0000 (oldest deadline 2024-04-16T17:19:32.982945+0000) 2024-04-16T17:19:46.728 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 16 17:19:46 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-9[66545]: 2024-04-16T17:19:46.347+0000 7f5937600700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.117:6812 osd.5 since back 2024-04-16T17:19:15.284908+0000 front 2024-04-16T17:19:15.285136+0000 (oldest deadline 2024-04-16T17:19:44.085731+0000) 2024-04-16T17:19:46.988 INFO:journalctl@ceph.osd.8.smithi117.stdout:Apr 16 17:19:46 smithi117 podman[75130]: 2024-04-16 17:19:46.804266418 +0000 UTC m=+5.866239752 container cleanup f23150d642f1fc7a61d328248009ae2e531db7212280cdab5ef10c1911ffd1d2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-8, org.label-schema.schema-version=1.0, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240409) 2024-04-16T17:19:46.988 INFO:journalctl@ceph.osd.8.smithi117.stdout:Apr 16 17:19:46 smithi117 bash[75130]: ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-8 2024-04-16T17:19:47.304 INFO:journalctl@ceph.osd.8.smithi117.stdout:Apr 16 17:19:46 smithi117 podman[75148]: 2024-04-16 17:19:46.9888545 +0000 UTC m=+0.585607595 container remove f23150d642f1fc7a61d328248009ae2e531db7212280cdab5ef10c1911ffd1d2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-8, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240409, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-04-16T17:19:47.661 INFO:journalctl@ceph.osd.8.smithi117.stdout:Apr 16 17:19:47 smithi117 podman[75238]: 2024-04-16 17:19:47.313143167 +0000 UTC m=+0.017746816 image pull e5ce10bdffbeeffffd1125f67b369e2151f0f17cdec854e93b6adf9321f65907 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2 2024-04-16T17:19:47.662 INFO:journalctl@ceph.osd.8.smithi117.stdout:Apr 16 17:19:47 smithi117 podman[75238]: 2024-04-16 17:19:47.439674632 +0000 UTC m=+0.144278253 container create 139815a000f8362fccbaf1c759b6a5cb6fb1981e15eb5c8771dfcf3facb99638 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-8-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240409, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True) 2024-04-16T17:19:47.662 INFO:journalctl@ceph.osd.8.smithi117.stdout:Apr 16 17:19:47 smithi117 podman[75238]: 2024-04-16 17:19:47.659412538 +0000 UTC m=+0.364016160 container init 139815a000f8362fccbaf1c759b6a5cb6fb1981e15eb5c8771dfcf3facb99638 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-8-deactivate, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240409, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-04-16T17:19:47.662 INFO:journalctl@ceph.osd.8.smithi117.stdout:Apr 16 17:19:47 smithi117 podman[75238]: 2024-04-16 17:19:47.662574083 +0000 UTC m=+0.367177715 container start 139815a000f8362fccbaf1c759b6a5cb6fb1981e15eb5c8771dfcf3facb99638 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-8-deactivate, org.label-schema.build-date=20240409, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True) 2024-04-16T17:19:47.662 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:19:47 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:19:47.618+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-16T17:18:34.374061+0000 front 2024-04-16T17:18:34.374275+0000 (oldest deadline 2024-04-16T17:18:57.873879+0000) 2024-04-16T17:19:47.663 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:19:47 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:19:47.618+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-16T17:18:50.676854+0000 front 2024-04-16T17:18:50.676874+0000 (oldest deadline 2024-04-16T17:19:15.977004+0000) 2024-04-16T17:19:47.663 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:19:47 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:19:47.618+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-16T17:19:01.878146+0000 front 2024-04-16T17:19:01.878262+0000 (oldest deadline 2024-04-16T17:19:27.178048+0000) 2024-04-16T17:19:47.663 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:19:47 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:19:47.618+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-04-16T17:19:07.178684+0000 front 2024-04-16T17:19:07.178598+0000 (oldest deadline 2024-04-16T17:19:33.078600+0000) 2024-04-16T17:19:47.663 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:19:47 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:19:47.618+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.117:6812 osd.5 since back 2024-04-16T17:19:18.379616+0000 front 2024-04-16T17:19:18.379681+0000 (oldest deadline 2024-04-16T17:19:43.079667+0000) 2024-04-16T17:19:47.677 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:19:47 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:19:47.182+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-16T17:18:36.158791+0000 front 2024-04-16T17:18:36.158992+0000 (oldest deadline 2024-04-16T17:19:00.258603+0000) 2024-04-16T17:19:47.677 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:19:47 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:19:47.182+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-16T17:18:44.260046+0000 front 2024-04-16T17:18:44.260171+0000 (oldest deadline 2024-04-16T17:19:07.160168+0000) 2024-04-16T17:19:47.677 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:19:47 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:19:47.182+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-16T17:18:54.062069+0000 front 2024-04-16T17:18:54.061971+0000 (oldest deadline 2024-04-16T17:19:18.761808+0000) 2024-04-16T17:19:47.677 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:19:47 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:19:47.182+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-16T17:19:01.063265+0000 front 2024-04-16T17:19:01.063419+0000 (oldest deadline 2024-04-16T17:19:24.563191+0000) 2024-04-16T17:19:47.677 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:19:47 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:19:47.182+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-04-16T17:19:10.865363+0000 front 2024-04-16T17:19:10.865389+0000 (oldest deadline 2024-04-16T17:19:35.466166+0000) 2024-04-16T17:19:47.678 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:19:47 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:19:47.182+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.117:6812 osd.5 since back 2024-04-16T17:19:19.567307+0000 front 2024-04-16T17:19:19.567365+0000 (oldest deadline 2024-04-16T17:19:44.267279+0000) 2024-04-16T17:19:47.727 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 16 17:19:47 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-9[66545]: 2024-04-16T17:19:47.378+0000 7f5937600700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-16T17:18:36.968505+0000 front 2024-04-16T17:18:36.968670+0000 (oldest deadline 2024-04-16T17:19:02.268762+0000) 2024-04-16T17:19:47.727 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 16 17:19:47 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-9[66545]: 2024-04-16T17:19:47.378+0000 7f5937600700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-16T17:18:43.371969+0000 front 2024-04-16T17:18:43.371904+0000 (oldest deadline 2024-04-16T17:19:08.671642+0000) 2024-04-16T17:19:47.727 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 16 17:19:47 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-9[66545]: 2024-04-16T17:19:47.378+0000 7f5937600700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-16T17:18:53.373968+0000 front 2024-04-16T17:18:53.374043+0000 (oldest deadline 2024-04-16T17:19:15.673755+0000) 2024-04-16T17:19:47.727 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 16 17:19:47 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-9[66545]: 2024-04-16T17:19:47.378+0000 7f5937600700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-16T17:19:01.880446+0000 front 2024-04-16T17:19:01.880936+0000 (oldest deadline 2024-04-16T17:19:27.180477+0000) 2024-04-16T17:19:47.728 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 16 17:19:47 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-9[66545]: 2024-04-16T17:19:47.378+0000 7f5937600700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-04-16T17:19:09.482762+0000 front 2024-04-16T17:19:09.482816+0000 (oldest deadline 2024-04-16T17:19:32.982945+0000) 2024-04-16T17:19:47.728 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 16 17:19:47 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-9[66545]: 2024-04-16T17:19:47.378+0000 7f5937600700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.117:6812 osd.5 since back 2024-04-16T17:19:15.284908+0000 front 2024-04-16T17:19:15.285136+0000 (oldest deadline 2024-04-16T17:19:44.085731+0000) 2024-04-16T17:19:47.941 INFO:journalctl@ceph.osd.8.smithi117.stdout:Apr 16 17:19:47 smithi117 podman[75238]: 2024-04-16 17:19:47.751203056 +0000 UTC m=+0.455806696 container attach 139815a000f8362fccbaf1c759b6a5cb6fb1981e15eb5c8771dfcf3facb99638 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-8-deactivate, GIT_BRANCH=HEAD, org.label-schema.build-date=20240409, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux ) 2024-04-16T17:19:48.304 INFO:journalctl@ceph.osd.8.smithi117.stdout:Apr 16 17:19:47 smithi117 podman[75238]: 2024-04-16 17:19:47.943043511 +0000 UTC m=+0.647647153 container died 139815a000f8362fccbaf1c759b6a5cb6fb1981e15eb5c8771dfcf3facb99638 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-8-deactivate, org.label-schema.build-date=20240409, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-04-16T17:19:48.427 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:19:48 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:19:48.171+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-16T17:18:36.158791+0000 front 2024-04-16T17:18:36.158992+0000 (oldest deadline 2024-04-16T17:19:00.258603+0000) 2024-04-16T17:19:48.427 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:19:48 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:19:48.171+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-16T17:18:44.260046+0000 front 2024-04-16T17:18:44.260171+0000 (oldest deadline 2024-04-16T17:19:07.160168+0000) 2024-04-16T17:19:48.427 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:19:48 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:19:48.171+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-16T17:18:54.062069+0000 front 2024-04-16T17:18:54.061971+0000 (oldest deadline 2024-04-16T17:19:18.761808+0000) 2024-04-16T17:19:48.427 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:19:48 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:19:48.171+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-16T17:19:01.063265+0000 front 2024-04-16T17:19:01.063419+0000 (oldest deadline 2024-04-16T17:19:24.563191+0000) 2024-04-16T17:19:48.427 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:19:48 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:19:48.172+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-04-16T17:19:10.865363+0000 front 2024-04-16T17:19:10.865389+0000 (oldest deadline 2024-04-16T17:19:35.466166+0000) 2024-04-16T17:19:48.427 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:19:48 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:19:48.172+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.117:6812 osd.5 since back 2024-04-16T17:19:19.567307+0000 front 2024-04-16T17:19:19.567365+0000 (oldest deadline 2024-04-16T17:19:44.267279+0000) 2024-04-16T17:19:48.614 DEBUG:teuthology.orchestra.run.smithi117:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b1fecd38-fc11-11ee-bc90-c7b262605968@osd.8.service' 2024-04-16T17:19:48.696 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-16T17:19:48.697 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-04-16T17:19:48.697 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-04-16T17:19:48.697 DEBUG:teuthology.orchestra.run.smithi045:> sudo systemctl stop ceph-b1fecd38-fc11-11ee-bc90-c7b262605968@osd.9 2024-04-16T17:19:48.727 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 16 17:19:48 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-9[66545]: 2024-04-16T17:19:48.410+0000 7f5937600700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-16T17:18:36.968505+0000 front 2024-04-16T17:18:36.968670+0000 (oldest deadline 2024-04-16T17:19:02.268762+0000) 2024-04-16T17:19:48.727 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 16 17:19:48 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-9[66545]: 2024-04-16T17:19:48.410+0000 7f5937600700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-16T17:18:43.371969+0000 front 2024-04-16T17:18:43.371904+0000 (oldest deadline 2024-04-16T17:19:08.671642+0000) 2024-04-16T17:19:48.727 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 16 17:19:48 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-9[66545]: 2024-04-16T17:19:48.410+0000 7f5937600700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-16T17:18:53.373968+0000 front 2024-04-16T17:18:53.374043+0000 (oldest deadline 2024-04-16T17:19:15.673755+0000) 2024-04-16T17:19:48.727 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 16 17:19:48 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-9[66545]: 2024-04-16T17:19:48.410+0000 7f5937600700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-16T17:19:01.880446+0000 front 2024-04-16T17:19:01.880936+0000 (oldest deadline 2024-04-16T17:19:27.180477+0000) 2024-04-16T17:19:48.727 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 16 17:19:48 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-9[66545]: 2024-04-16T17:19:48.410+0000 7f5937600700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-04-16T17:19:09.482762+0000 front 2024-04-16T17:19:09.482816+0000 (oldest deadline 2024-04-16T17:19:32.982945+0000) 2024-04-16T17:19:48.727 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 16 17:19:48 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-9[66545]: 2024-04-16T17:19:48.410+0000 7f5937600700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.117:6812 osd.5 since back 2024-04-16T17:19:15.284908+0000 front 2024-04-16T17:19:15.285136+0000 (oldest deadline 2024-04-16T17:19:44.085731+0000) 2024-04-16T17:19:48.865 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:19:48 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:19:48.573+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-16T17:18:34.374061+0000 front 2024-04-16T17:18:34.374275+0000 (oldest deadline 2024-04-16T17:18:57.873879+0000) 2024-04-16T17:19:48.866 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:19:48 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:19:48.573+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-16T17:18:50.676854+0000 front 2024-04-16T17:18:50.676874+0000 (oldest deadline 2024-04-16T17:19:15.977004+0000) 2024-04-16T17:19:48.866 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:19:48 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:19:48.573+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-16T17:19:01.878146+0000 front 2024-04-16T17:19:01.878262+0000 (oldest deadline 2024-04-16T17:19:27.178048+0000) 2024-04-16T17:19:48.866 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:19:48 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:19:48.573+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-04-16T17:19:07.178684+0000 front 2024-04-16T17:19:07.178598+0000 (oldest deadline 2024-04-16T17:19:33.078600+0000) 2024-04-16T17:19:48.866 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:19:48 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:19:48.573+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.117:6812 osd.5 since back 2024-04-16T17:19:18.379616+0000 front 2024-04-16T17:19:18.379681+0000 (oldest deadline 2024-04-16T17:19:43.079667+0000) 2024-04-16T17:19:49.227 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 16 17:19:48 smithi045 systemd[1]: Stopping Ceph osd.9 for b1fecd38-fc11-11ee-bc90-c7b262605968... 2024-04-16T17:19:49.427 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:19:49 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:19:49.128+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-16T17:18:36.158791+0000 front 2024-04-16T17:18:36.158992+0000 (oldest deadline 2024-04-16T17:19:00.258603+0000) 2024-04-16T17:19:49.427 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:19:49 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:19:49.128+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-16T17:18:44.260046+0000 front 2024-04-16T17:18:44.260171+0000 (oldest deadline 2024-04-16T17:19:07.160168+0000) 2024-04-16T17:19:49.428 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:19:49 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:19:49.128+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-16T17:18:54.062069+0000 front 2024-04-16T17:18:54.061971+0000 (oldest deadline 2024-04-16T17:19:18.761808+0000) 2024-04-16T17:19:49.428 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:19:49 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:19:49.128+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-16T17:19:01.063265+0000 front 2024-04-16T17:19:01.063419+0000 (oldest deadline 2024-04-16T17:19:24.563191+0000) 2024-04-16T17:19:49.428 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:19:49 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:19:49.128+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-04-16T17:19:10.865363+0000 front 2024-04-16T17:19:10.865389+0000 (oldest deadline 2024-04-16T17:19:35.466166+0000) 2024-04-16T17:19:49.428 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:19:49 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:19:49.128+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.117:6812 osd.5 since back 2024-04-16T17:19:19.567307+0000 front 2024-04-16T17:19:19.567365+0000 (oldest deadline 2024-04-16T17:19:44.267279+0000) 2024-04-16T17:19:49.727 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 16 17:19:49 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-9[66545]: 2024-04-16T17:19:49.249+0000 7f5940e00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-16T17:19:49.728 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 16 17:19:49 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-9[66545]: 2024-04-16T17:19:49.249+0000 7f5940e00700 -1 osd.9 112 *** Got signal Terminated *** 2024-04-16T17:19:49.728 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 16 17:19:49 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-9[66545]: 2024-04-16T17:19:49.250+0000 7f5940e00700 -1 osd.9 112 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-16T17:19:49.728 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 16 17:19:49 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-9[66545]: 2024-04-16T17:19:49.409+0000 7f5937600700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-16T17:18:36.968505+0000 front 2024-04-16T17:18:36.968670+0000 (oldest deadline 2024-04-16T17:19:02.268762+0000) 2024-04-16T17:19:49.728 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 16 17:19:49 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-9[66545]: 2024-04-16T17:19:49.409+0000 7f5937600700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-16T17:18:43.371969+0000 front 2024-04-16T17:18:43.371904+0000 (oldest deadline 2024-04-16T17:19:08.671642+0000) 2024-04-16T17:19:49.728 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 16 17:19:49 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-9[66545]: 2024-04-16T17:19:49.409+0000 7f5937600700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-16T17:18:53.373968+0000 front 2024-04-16T17:18:53.374043+0000 (oldest deadline 2024-04-16T17:19:15.673755+0000) 2024-04-16T17:19:49.728 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 16 17:19:49 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-9[66545]: 2024-04-16T17:19:49.409+0000 7f5937600700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-16T17:19:01.880446+0000 front 2024-04-16T17:19:01.880936+0000 (oldest deadline 2024-04-16T17:19:27.180477+0000) 2024-04-16T17:19:49.729 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 16 17:19:49 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-9[66545]: 2024-04-16T17:19:49.409+0000 7f5937600700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-04-16T17:19:09.482762+0000 front 2024-04-16T17:19:09.482816+0000 (oldest deadline 2024-04-16T17:19:32.982945+0000) 2024-04-16T17:19:49.729 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 16 17:19:49 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-9[66545]: 2024-04-16T17:19:49.409+0000 7f5937600700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.117:6812 osd.5 since back 2024-04-16T17:19:15.284908+0000 front 2024-04-16T17:19:15.285136+0000 (oldest deadline 2024-04-16T17:19:44.085731+0000) 2024-04-16T17:19:50.054 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:19:49 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:19:49.606+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-16T17:18:34.374061+0000 front 2024-04-16T17:18:34.374275+0000 (oldest deadline 2024-04-16T17:18:57.873879+0000) 2024-04-16T17:19:50.055 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:19:49 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:19:49.606+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-16T17:18:50.676854+0000 front 2024-04-16T17:18:50.676874+0000 (oldest deadline 2024-04-16T17:19:15.977004+0000) 2024-04-16T17:19:50.055 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:19:49 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:19:49.606+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-16T17:19:01.878146+0000 front 2024-04-16T17:19:01.878262+0000 (oldest deadline 2024-04-16T17:19:27.178048+0000) 2024-04-16T17:19:50.055 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:19:49 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:19:49.606+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-04-16T17:19:07.178684+0000 front 2024-04-16T17:19:07.178598+0000 (oldest deadline 2024-04-16T17:19:33.078600+0000) 2024-04-16T17:19:50.055 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:19:49 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:19:49.606+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.117:6812 osd.5 since back 2024-04-16T17:19:18.379616+0000 front 2024-04-16T17:19:18.379681+0000 (oldest deadline 2024-04-16T17:19:43.079667+0000) 2024-04-16T17:19:50.427 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:19:50 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:19:50.106+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-16T17:18:36.158791+0000 front 2024-04-16T17:18:36.158992+0000 (oldest deadline 2024-04-16T17:19:00.258603+0000) 2024-04-16T17:19:50.427 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:19:50 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:19:50.106+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-16T17:18:44.260046+0000 front 2024-04-16T17:18:44.260171+0000 (oldest deadline 2024-04-16T17:19:07.160168+0000) 2024-04-16T17:19:50.427 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:19:50 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:19:50.106+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-16T17:18:54.062069+0000 front 2024-04-16T17:18:54.061971+0000 (oldest deadline 2024-04-16T17:19:18.761808+0000) 2024-04-16T17:19:50.427 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:19:50 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:19:50.106+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-16T17:19:01.063265+0000 front 2024-04-16T17:19:01.063419+0000 (oldest deadline 2024-04-16T17:19:24.563191+0000) 2024-04-16T17:19:50.427 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:19:50 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:19:50.106+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-04-16T17:19:10.865363+0000 front 2024-04-16T17:19:10.865389+0000 (oldest deadline 2024-04-16T17:19:35.466166+0000) 2024-04-16T17:19:50.427 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:19:50 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:19:50.106+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.117:6812 osd.5 since back 2024-04-16T17:19:19.567307+0000 front 2024-04-16T17:19:19.567365+0000 (oldest deadline 2024-04-16T17:19:44.267279+0000) 2024-04-16T17:19:50.727 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 16 17:19:50 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-9[66545]: 2024-04-16T17:19:50.406+0000 7f5937600700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-16T17:18:36.968505+0000 front 2024-04-16T17:18:36.968670+0000 (oldest deadline 2024-04-16T17:19:02.268762+0000) 2024-04-16T17:19:50.727 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 16 17:19:50 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-9[66545]: 2024-04-16T17:19:50.406+0000 7f5937600700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-16T17:18:43.371969+0000 front 2024-04-16T17:18:43.371904+0000 (oldest deadline 2024-04-16T17:19:08.671642+0000) 2024-04-16T17:19:50.727 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 16 17:19:50 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-9[66545]: 2024-04-16T17:19:50.406+0000 7f5937600700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-16T17:18:53.373968+0000 front 2024-04-16T17:18:53.374043+0000 (oldest deadline 2024-04-16T17:19:15.673755+0000) 2024-04-16T17:19:50.727 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 16 17:19:50 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-9[66545]: 2024-04-16T17:19:50.406+0000 7f5937600700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-16T17:19:01.880446+0000 front 2024-04-16T17:19:01.880936+0000 (oldest deadline 2024-04-16T17:19:27.180477+0000) 2024-04-16T17:19:50.727 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 16 17:19:50 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-9[66545]: 2024-04-16T17:19:50.406+0000 7f5937600700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-04-16T17:19:09.482762+0000 front 2024-04-16T17:19:09.482816+0000 (oldest deadline 2024-04-16T17:19:32.982945+0000) 2024-04-16T17:19:50.728 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 16 17:19:50 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-9[66545]: 2024-04-16T17:19:50.406+0000 7f5937600700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.117:6812 osd.5 since back 2024-04-16T17:19:15.284908+0000 front 2024-04-16T17:19:15.285136+0000 (oldest deadline 2024-04-16T17:19:44.085731+0000) 2024-04-16T17:19:51.054 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:19:50 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:19:50.597+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-16T17:18:34.374061+0000 front 2024-04-16T17:18:34.374275+0000 (oldest deadline 2024-04-16T17:18:57.873879+0000) 2024-04-16T17:19:51.054 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:19:50 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:19:50.597+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-16T17:18:50.676854+0000 front 2024-04-16T17:18:50.676874+0000 (oldest deadline 2024-04-16T17:19:15.977004+0000) 2024-04-16T17:19:51.054 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:19:50 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:19:50.597+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-16T17:19:01.878146+0000 front 2024-04-16T17:19:01.878262+0000 (oldest deadline 2024-04-16T17:19:27.178048+0000) 2024-04-16T17:19:51.055 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:19:50 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:19:50.597+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-04-16T17:19:07.178684+0000 front 2024-04-16T17:19:07.178598+0000 (oldest deadline 2024-04-16T17:19:33.078600+0000) 2024-04-16T17:19:51.055 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:19:50 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:19:50.597+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.117:6812 osd.5 since back 2024-04-16T17:19:18.379616+0000 front 2024-04-16T17:19:18.379681+0000 (oldest deadline 2024-04-16T17:19:43.079667+0000) 2024-04-16T17:19:51.427 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:19:51 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:19:51.111+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-16T17:18:36.158791+0000 front 2024-04-16T17:18:36.158992+0000 (oldest deadline 2024-04-16T17:19:00.258603+0000) 2024-04-16T17:19:51.427 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:19:51 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:19:51.111+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-16T17:18:44.260046+0000 front 2024-04-16T17:18:44.260171+0000 (oldest deadline 2024-04-16T17:19:07.160168+0000) 2024-04-16T17:19:51.427 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:19:51 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:19:51.111+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-16T17:18:54.062069+0000 front 2024-04-16T17:18:54.061971+0000 (oldest deadline 2024-04-16T17:19:18.761808+0000) 2024-04-16T17:19:51.427 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:19:51 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:19:51.111+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-16T17:19:01.063265+0000 front 2024-04-16T17:19:01.063419+0000 (oldest deadline 2024-04-16T17:19:24.563191+0000) 2024-04-16T17:19:51.427 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:19:51 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:19:51.111+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-04-16T17:19:10.865363+0000 front 2024-04-16T17:19:10.865389+0000 (oldest deadline 2024-04-16T17:19:35.466166+0000) 2024-04-16T17:19:51.427 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:19:51 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:19:51.111+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.117:6812 osd.5 since back 2024-04-16T17:19:19.567307+0000 front 2024-04-16T17:19:19.567365+0000 (oldest deadline 2024-04-16T17:19:44.267279+0000) 2024-04-16T17:19:51.727 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 16 17:19:51 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-9[66545]: 2024-04-16T17:19:51.366+0000 7f5937600700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-16T17:18:36.968505+0000 front 2024-04-16T17:18:36.968670+0000 (oldest deadline 2024-04-16T17:19:02.268762+0000) 2024-04-16T17:19:51.727 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 16 17:19:51 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-9[66545]: 2024-04-16T17:19:51.366+0000 7f5937600700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-16T17:18:43.371969+0000 front 2024-04-16T17:18:43.371904+0000 (oldest deadline 2024-04-16T17:19:08.671642+0000) 2024-04-16T17:19:51.727 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 16 17:19:51 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-9[66545]: 2024-04-16T17:19:51.366+0000 7f5937600700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-16T17:18:53.373968+0000 front 2024-04-16T17:18:53.374043+0000 (oldest deadline 2024-04-16T17:19:15.673755+0000) 2024-04-16T17:19:51.727 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 16 17:19:51 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-9[66545]: 2024-04-16T17:19:51.366+0000 7f5937600700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-16T17:19:01.880446+0000 front 2024-04-16T17:19:01.880936+0000 (oldest deadline 2024-04-16T17:19:27.180477+0000) 2024-04-16T17:19:51.727 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 16 17:19:51 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-9[66545]: 2024-04-16T17:19:51.366+0000 7f5937600700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-04-16T17:19:09.482762+0000 front 2024-04-16T17:19:09.482816+0000 (oldest deadline 2024-04-16T17:19:32.982945+0000) 2024-04-16T17:19:51.727 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 16 17:19:51 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-9[66545]: 2024-04-16T17:19:51.366+0000 7f5937600700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.117:6812 osd.5 since back 2024-04-16T17:19:15.284908+0000 front 2024-04-16T17:19:15.285136+0000 (oldest deadline 2024-04-16T17:19:44.085731+0000) 2024-04-16T17:19:52.054 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:19:51 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:19:51.605+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-16T17:18:34.374061+0000 front 2024-04-16T17:18:34.374275+0000 (oldest deadline 2024-04-16T17:18:57.873879+0000) 2024-04-16T17:19:52.054 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:19:51 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:19:51.605+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-16T17:18:50.676854+0000 front 2024-04-16T17:18:50.676874+0000 (oldest deadline 2024-04-16T17:19:15.977004+0000) 2024-04-16T17:19:52.054 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:19:51 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:19:51.605+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-16T17:19:01.878146+0000 front 2024-04-16T17:19:01.878262+0000 (oldest deadline 2024-04-16T17:19:27.178048+0000) 2024-04-16T17:19:52.055 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:19:51 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:19:51.605+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-04-16T17:19:07.178684+0000 front 2024-04-16T17:19:07.178598+0000 (oldest deadline 2024-04-16T17:19:33.078600+0000) 2024-04-16T17:19:52.055 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:19:51 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:19:51.605+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.117:6812 osd.5 since back 2024-04-16T17:19:18.379616+0000 front 2024-04-16T17:19:18.379681+0000 (oldest deadline 2024-04-16T17:19:43.079667+0000) 2024-04-16T17:19:52.427 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:19:52 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:19:52.149+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-16T17:18:36.158791+0000 front 2024-04-16T17:18:36.158992+0000 (oldest deadline 2024-04-16T17:19:00.258603+0000) 2024-04-16T17:19:52.427 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:19:52 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:19:52.149+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-16T17:18:44.260046+0000 front 2024-04-16T17:18:44.260171+0000 (oldest deadline 2024-04-16T17:19:07.160168+0000) 2024-04-16T17:19:52.427 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:19:52 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:19:52.149+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-16T17:18:54.062069+0000 front 2024-04-16T17:18:54.061971+0000 (oldest deadline 2024-04-16T17:19:18.761808+0000) 2024-04-16T17:19:52.427 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:19:52 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:19:52.149+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-16T17:19:01.063265+0000 front 2024-04-16T17:19:01.063419+0000 (oldest deadline 2024-04-16T17:19:24.563191+0000) 2024-04-16T17:19:52.427 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:19:52 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:19:52.149+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-04-16T17:19:10.865363+0000 front 2024-04-16T17:19:10.865389+0000 (oldest deadline 2024-04-16T17:19:35.466166+0000) 2024-04-16T17:19:52.427 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:19:52 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:19:52.149+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.117:6812 osd.5 since back 2024-04-16T17:19:19.567307+0000 front 2024-04-16T17:19:19.567365+0000 (oldest deadline 2024-04-16T17:19:44.267279+0000) 2024-04-16T17:19:52.427 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:19:52 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:19:52.149+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.45:6822 osd.6 since back 2024-04-16T17:19:27.768425+0000 front 2024-04-16T17:19:27.768357+0000 (oldest deadline 2024-04-16T17:19:51.868043+0000) 2024-04-16T17:19:52.727 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 16 17:19:52 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-9[66545]: 2024-04-16T17:19:52.381+0000 7f5937600700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-16T17:18:36.968505+0000 front 2024-04-16T17:18:36.968670+0000 (oldest deadline 2024-04-16T17:19:02.268762+0000) 2024-04-16T17:19:52.727 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 16 17:19:52 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-9[66545]: 2024-04-16T17:19:52.381+0000 7f5937600700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-16T17:18:43.371969+0000 front 2024-04-16T17:18:43.371904+0000 (oldest deadline 2024-04-16T17:19:08.671642+0000) 2024-04-16T17:19:52.727 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 16 17:19:52 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-9[66545]: 2024-04-16T17:19:52.381+0000 7f5937600700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-16T17:18:53.373968+0000 front 2024-04-16T17:18:53.374043+0000 (oldest deadline 2024-04-16T17:19:15.673755+0000) 2024-04-16T17:19:52.727 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 16 17:19:52 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-9[66545]: 2024-04-16T17:19:52.381+0000 7f5937600700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-16T17:19:01.880446+0000 front 2024-04-16T17:19:01.880936+0000 (oldest deadline 2024-04-16T17:19:27.180477+0000) 2024-04-16T17:19:52.727 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 16 17:19:52 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-9[66545]: 2024-04-16T17:19:52.381+0000 7f5937600700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-04-16T17:19:09.482762+0000 front 2024-04-16T17:19:09.482816+0000 (oldest deadline 2024-04-16T17:19:32.982945+0000) 2024-04-16T17:19:52.727 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 16 17:19:52 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-9[66545]: 2024-04-16T17:19:52.381+0000 7f5937600700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.117:6812 osd.5 since back 2024-04-16T17:19:15.284908+0000 front 2024-04-16T17:19:15.285136+0000 (oldest deadline 2024-04-16T17:19:44.085731+0000) 2024-04-16T17:19:53.054 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:19:52 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:19:52.620+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-16T17:18:34.374061+0000 front 2024-04-16T17:18:34.374275+0000 (oldest deadline 2024-04-16T17:18:57.873879+0000) 2024-04-16T17:19:53.054 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:19:52 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:19:52.620+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-16T17:18:50.676854+0000 front 2024-04-16T17:18:50.676874+0000 (oldest deadline 2024-04-16T17:19:15.977004+0000) 2024-04-16T17:19:53.054 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:19:52 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:19:52.620+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-16T17:19:01.878146+0000 front 2024-04-16T17:19:01.878262+0000 (oldest deadline 2024-04-16T17:19:27.178048+0000) 2024-04-16T17:19:53.054 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:19:52 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:19:52.620+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-04-16T17:19:07.178684+0000 front 2024-04-16T17:19:07.178598+0000 (oldest deadline 2024-04-16T17:19:33.078600+0000) 2024-04-16T17:19:53.054 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:19:52 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:19:52.620+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.117:6812 osd.5 since back 2024-04-16T17:19:18.379616+0000 front 2024-04-16T17:19:18.379681+0000 (oldest deadline 2024-04-16T17:19:43.079667+0000) 2024-04-16T17:19:53.426 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:19:53 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:19:53.159+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-16T17:18:36.158791+0000 front 2024-04-16T17:18:36.158992+0000 (oldest deadline 2024-04-16T17:19:00.258603+0000) 2024-04-16T17:19:53.427 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:19:53 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:19:53.159+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-16T17:18:44.260046+0000 front 2024-04-16T17:18:44.260171+0000 (oldest deadline 2024-04-16T17:19:07.160168+0000) 2024-04-16T17:19:53.427 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:19:53 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:19:53.159+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-16T17:18:54.062069+0000 front 2024-04-16T17:18:54.061971+0000 (oldest deadline 2024-04-16T17:19:18.761808+0000) 2024-04-16T17:19:53.427 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:19:53 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:19:53.159+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-16T17:19:01.063265+0000 front 2024-04-16T17:19:01.063419+0000 (oldest deadline 2024-04-16T17:19:24.563191+0000) 2024-04-16T17:19:53.427 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:19:53 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:19:53.159+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-04-16T17:19:10.865363+0000 front 2024-04-16T17:19:10.865389+0000 (oldest deadline 2024-04-16T17:19:35.466166+0000) 2024-04-16T17:19:53.427 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:19:53 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:19:53.159+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.117:6812 osd.5 since back 2024-04-16T17:19:19.567307+0000 front 2024-04-16T17:19:19.567365+0000 (oldest deadline 2024-04-16T17:19:44.267279+0000) 2024-04-16T17:19:53.427 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:19:53 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:19:53.159+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.45:6822 osd.6 since back 2024-04-16T17:19:27.768425+0000 front 2024-04-16T17:19:27.768357+0000 (oldest deadline 2024-04-16T17:19:51.868043+0000) 2024-04-16T17:19:53.727 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 16 17:19:53 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-9[66545]: 2024-04-16T17:19:53.342+0000 7f5937600700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-16T17:18:36.968505+0000 front 2024-04-16T17:18:36.968670+0000 (oldest deadline 2024-04-16T17:19:02.268762+0000) 2024-04-16T17:19:53.727 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 16 17:19:53 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-9[66545]: 2024-04-16T17:19:53.342+0000 7f5937600700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-16T17:18:43.371969+0000 front 2024-04-16T17:18:43.371904+0000 (oldest deadline 2024-04-16T17:19:08.671642+0000) 2024-04-16T17:19:53.727 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 16 17:19:53 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-9[66545]: 2024-04-16T17:19:53.342+0000 7f5937600700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-16T17:18:53.373968+0000 front 2024-04-16T17:18:53.374043+0000 (oldest deadline 2024-04-16T17:19:15.673755+0000) 2024-04-16T17:19:53.727 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 16 17:19:53 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-9[66545]: 2024-04-16T17:19:53.342+0000 7f5937600700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-16T17:19:01.880446+0000 front 2024-04-16T17:19:01.880936+0000 (oldest deadline 2024-04-16T17:19:27.180477+0000) 2024-04-16T17:19:53.728 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 16 17:19:53 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-9[66545]: 2024-04-16T17:19:53.342+0000 7f5937600700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-04-16T17:19:09.482762+0000 front 2024-04-16T17:19:09.482816+0000 (oldest deadline 2024-04-16T17:19:32.982945+0000) 2024-04-16T17:19:53.728 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 16 17:19:53 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-9[66545]: 2024-04-16T17:19:53.342+0000 7f5937600700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.117:6812 osd.5 since back 2024-04-16T17:19:15.284908+0000 front 2024-04-16T17:19:15.285136+0000 (oldest deadline 2024-04-16T17:19:44.085731+0000) 2024-04-16T17:19:54.054 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:19:53 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:19:53.647+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-16T17:18:34.374061+0000 front 2024-04-16T17:18:34.374275+0000 (oldest deadline 2024-04-16T17:18:57.873879+0000) 2024-04-16T17:19:54.054 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:19:53 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:19:53.647+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-16T17:18:50.676854+0000 front 2024-04-16T17:18:50.676874+0000 (oldest deadline 2024-04-16T17:19:15.977004+0000) 2024-04-16T17:19:54.054 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:19:53 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:19:53.647+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-16T17:19:01.878146+0000 front 2024-04-16T17:19:01.878262+0000 (oldest deadline 2024-04-16T17:19:27.178048+0000) 2024-04-16T17:19:54.054 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:19:53 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:19:53.647+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-04-16T17:19:07.178684+0000 front 2024-04-16T17:19:07.178598+0000 (oldest deadline 2024-04-16T17:19:33.078600+0000) 2024-04-16T17:19:54.055 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:19:53 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:19:53.647+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.117:6812 osd.5 since back 2024-04-16T17:19:18.379616+0000 front 2024-04-16T17:19:18.379681+0000 (oldest deadline 2024-04-16T17:19:43.079667+0000) 2024-04-16T17:19:54.427 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:19:54 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:19:54.171+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-16T17:18:36.158791+0000 front 2024-04-16T17:18:36.158992+0000 (oldest deadline 2024-04-16T17:19:00.258603+0000) 2024-04-16T17:19:54.427 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:19:54 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:19:54.171+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-16T17:18:44.260046+0000 front 2024-04-16T17:18:44.260171+0000 (oldest deadline 2024-04-16T17:19:07.160168+0000) 2024-04-16T17:19:54.427 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:19:54 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:19:54.171+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-16T17:18:54.062069+0000 front 2024-04-16T17:18:54.061971+0000 (oldest deadline 2024-04-16T17:19:18.761808+0000) 2024-04-16T17:19:54.427 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:19:54 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:19:54.171+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-16T17:19:01.063265+0000 front 2024-04-16T17:19:01.063419+0000 (oldest deadline 2024-04-16T17:19:24.563191+0000) 2024-04-16T17:19:54.428 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:19:54 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:19:54.171+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-04-16T17:19:10.865363+0000 front 2024-04-16T17:19:10.865389+0000 (oldest deadline 2024-04-16T17:19:35.466166+0000) 2024-04-16T17:19:54.428 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:19:54 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:19:54.171+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.117:6812 osd.5 since back 2024-04-16T17:19:19.567307+0000 front 2024-04-16T17:19:19.567365+0000 (oldest deadline 2024-04-16T17:19:44.267279+0000) 2024-04-16T17:19:54.428 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:19:54 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:19:54.171+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.45:6822 osd.6 since back 2024-04-16T17:19:27.768425+0000 front 2024-04-16T17:19:27.768357+0000 (oldest deadline 2024-04-16T17:19:51.868043+0000) 2024-04-16T17:19:55.054 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:19:54 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:19:54.603+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-16T17:18:34.374061+0000 front 2024-04-16T17:18:34.374275+0000 (oldest deadline 2024-04-16T17:18:57.873879+0000) 2024-04-16T17:19:55.054 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:19:54 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:19:54.603+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-16T17:18:50.676854+0000 front 2024-04-16T17:18:50.676874+0000 (oldest deadline 2024-04-16T17:19:15.977004+0000) 2024-04-16T17:19:55.055 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:19:54 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:19:54.603+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-16T17:19:01.878146+0000 front 2024-04-16T17:19:01.878262+0000 (oldest deadline 2024-04-16T17:19:27.178048+0000) 2024-04-16T17:19:55.055 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:19:54 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:19:54.603+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-04-16T17:19:07.178684+0000 front 2024-04-16T17:19:07.178598+0000 (oldest deadline 2024-04-16T17:19:33.078600+0000) 2024-04-16T17:19:55.055 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:19:54 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:19:54.603+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.117:6812 osd.5 since back 2024-04-16T17:19:18.379616+0000 front 2024-04-16T17:19:18.379681+0000 (oldest deadline 2024-04-16T17:19:43.079667+0000) 2024-04-16T17:19:55.055 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:19:54 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:19:54.603+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.45:6822 osd.6 since back 2024-04-16T17:19:27.780981+0000 front 2024-04-16T17:19:27.781111+0000 (oldest deadline 2024-04-16T17:19:53.680822+0000) 2024-04-16T17:19:55.223 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 16 17:19:54 smithi045 podman[104343]: 2024-04-16 17:19:54.880996572 +0000 UTC m=+5.761526850 container died 4c51f92b25f2c151ec7b0fe0bfd9043e5b78daf4f5d6411aa8f8a6aa58adbf19 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-9, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240409, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, maintainer=Guillaume Abrioux ) 2024-04-16T17:19:55.477 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 16 17:19:55 smithi045 podman[104343]: 2024-04-16 17:19:55.333859856 +0000 UTC m=+6.214390159 container cleanup 4c51f92b25f2c151ec7b0fe0bfd9043e5b78daf4f5d6411aa8f8a6aa58adbf19 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-9, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240409, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb) 2024-04-16T17:19:55.478 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 16 17:19:55 smithi045 bash[104343]: ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-9 2024-04-16T17:19:55.677 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:19:55 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:19:55.198+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-16T17:18:36.158791+0000 front 2024-04-16T17:18:36.158992+0000 (oldest deadline 2024-04-16T17:19:00.258603+0000) 2024-04-16T17:19:55.677 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:19:55 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:19:55.198+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-16T17:18:44.260046+0000 front 2024-04-16T17:18:44.260171+0000 (oldest deadline 2024-04-16T17:19:07.160168+0000) 2024-04-16T17:19:55.677 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:19:55 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:19:55.198+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-16T17:18:54.062069+0000 front 2024-04-16T17:18:54.061971+0000 (oldest deadline 2024-04-16T17:19:18.761808+0000) 2024-04-16T17:19:55.677 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:19:55 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:19:55.198+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-16T17:19:01.063265+0000 front 2024-04-16T17:19:01.063419+0000 (oldest deadline 2024-04-16T17:19:24.563191+0000) 2024-04-16T17:19:55.677 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:19:55 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:19:55.198+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-04-16T17:19:10.865363+0000 front 2024-04-16T17:19:10.865389+0000 (oldest deadline 2024-04-16T17:19:35.466166+0000) 2024-04-16T17:19:55.677 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:19:55 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:19:55.198+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.117:6812 osd.5 since back 2024-04-16T17:19:19.567307+0000 front 2024-04-16T17:19:19.567365+0000 (oldest deadline 2024-04-16T17:19:44.267279+0000) 2024-04-16T17:19:55.677 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:19:55 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:19:55.198+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.45:6822 osd.6 since back 2024-04-16T17:19:27.768425+0000 front 2024-04-16T17:19:27.768357+0000 (oldest deadline 2024-04-16T17:19:51.868043+0000) 2024-04-16T17:19:55.977 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 16 17:19:55 smithi045 podman[104362]: 2024-04-16 17:19:55.726334338 +0000 UTC m=+0.831079327 container remove 4c51f92b25f2c151ec7b0fe0bfd9043e5b78daf4f5d6411aa8f8a6aa58adbf19 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-9, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240409, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, io.buildah.version=1.33.5) 2024-04-16T17:19:56.054 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:19:55 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:19:55.621+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-16T17:18:34.374061+0000 front 2024-04-16T17:18:34.374275+0000 (oldest deadline 2024-04-16T17:18:57.873879+0000) 2024-04-16T17:19:56.054 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:19:55 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:19:55.621+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-16T17:18:50.676854+0000 front 2024-04-16T17:18:50.676874+0000 (oldest deadline 2024-04-16T17:19:15.977004+0000) 2024-04-16T17:19:56.054 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:19:55 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:19:55.621+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-16T17:19:01.878146+0000 front 2024-04-16T17:19:01.878262+0000 (oldest deadline 2024-04-16T17:19:27.178048+0000) 2024-04-16T17:19:56.055 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:19:55 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:19:55.621+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-04-16T17:19:07.178684+0000 front 2024-04-16T17:19:07.178598+0000 (oldest deadline 2024-04-16T17:19:33.078600+0000) 2024-04-16T17:19:56.055 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:19:55 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:19:55.621+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.117:6812 osd.5 since back 2024-04-16T17:19:18.379616+0000 front 2024-04-16T17:19:18.379681+0000 (oldest deadline 2024-04-16T17:19:43.079667+0000) 2024-04-16T17:19:56.055 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:19:55 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:19:55.621+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.45:6822 osd.6 since back 2024-04-16T17:19:27.780981+0000 front 2024-04-16T17:19:27.781111+0000 (oldest deadline 2024-04-16T17:19:53.680822+0000) 2024-04-16T17:19:56.677 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:19:56 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:19:56.239+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-16T17:18:36.158791+0000 front 2024-04-16T17:18:36.158992+0000 (oldest deadline 2024-04-16T17:19:00.258603+0000) 2024-04-16T17:19:56.677 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:19:56 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:19:56.239+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-16T17:18:44.260046+0000 front 2024-04-16T17:18:44.260171+0000 (oldest deadline 2024-04-16T17:19:07.160168+0000) 2024-04-16T17:19:56.677 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:19:56 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:19:56.239+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-16T17:18:54.062069+0000 front 2024-04-16T17:18:54.061971+0000 (oldest deadline 2024-04-16T17:19:18.761808+0000) 2024-04-16T17:19:56.677 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:19:56 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:19:56.239+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-16T17:19:01.063265+0000 front 2024-04-16T17:19:01.063419+0000 (oldest deadline 2024-04-16T17:19:24.563191+0000) 2024-04-16T17:19:56.677 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:19:56 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:19:56.239+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-04-16T17:19:10.865363+0000 front 2024-04-16T17:19:10.865389+0000 (oldest deadline 2024-04-16T17:19:35.466166+0000) 2024-04-16T17:19:56.677 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:19:56 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:19:56.239+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.117:6812 osd.5 since back 2024-04-16T17:19:19.567307+0000 front 2024-04-16T17:19:19.567365+0000 (oldest deadline 2024-04-16T17:19:44.267279+0000) 2024-04-16T17:19:56.677 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:19:56 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:19:56.239+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.45:6822 osd.6 since back 2024-04-16T17:19:27.768425+0000 front 2024-04-16T17:19:27.768357+0000 (oldest deadline 2024-04-16T17:19:51.868043+0000) 2024-04-16T17:19:56.889 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 16 17:19:56 smithi045 podman[104450]: 2024-04-16 17:19:56.477452634 +0000 UTC m=+0.033305432 image pull e5ce10bdffbeeffffd1125f67b369e2151f0f17cdec854e93b6adf9321f65907 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2 2024-04-16T17:19:56.889 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 16 17:19:56 smithi045 podman[104450]: 2024-04-16 17:19:56.643821327 +0000 UTC m=+0.199674172 container create b5e992b26b8c75efa266d3e12bcb7a8e3a3fbbfc72ce87dd576407b818c69b97 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-9-deactivate, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240409, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-04-16T17:19:56.889 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 16 17:19:56 smithi045 podman[104450]: 2024-04-16 17:19:56.889172284 +0000 UTC m=+0.445025172 container init b5e992b26b8c75efa266d3e12bcb7a8e3a3fbbfc72ce87dd576407b818c69b97 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-9-deactivate, maintainer=Guillaume Abrioux , RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240409, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-16T17:19:57.054 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:19:56 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:19:56.657+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-16T17:18:34.374061+0000 front 2024-04-16T17:18:34.374275+0000 (oldest deadline 2024-04-16T17:18:57.873879+0000) 2024-04-16T17:19:57.054 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:19:56 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:19:56.657+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-16T17:18:50.676854+0000 front 2024-04-16T17:18:50.676874+0000 (oldest deadline 2024-04-16T17:19:15.977004+0000) 2024-04-16T17:19:57.054 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:19:56 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:19:56.657+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-16T17:19:01.878146+0000 front 2024-04-16T17:19:01.878262+0000 (oldest deadline 2024-04-16T17:19:27.178048+0000) 2024-04-16T17:19:57.055 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:19:56 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:19:56.657+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-04-16T17:19:07.178684+0000 front 2024-04-16T17:19:07.178598+0000 (oldest deadline 2024-04-16T17:19:33.078600+0000) 2024-04-16T17:19:57.055 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:19:56 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:19:56.657+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.117:6812 osd.5 since back 2024-04-16T17:19:18.379616+0000 front 2024-04-16T17:19:18.379681+0000 (oldest deadline 2024-04-16T17:19:43.079667+0000) 2024-04-16T17:19:57.055 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:19:56 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:19:56.657+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.45:6822 osd.6 since back 2024-04-16T17:19:27.780981+0000 front 2024-04-16T17:19:27.781111+0000 (oldest deadline 2024-04-16T17:19:53.680822+0000) 2024-04-16T17:19:57.227 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 16 17:19:56 smithi045 podman[104450]: 2024-04-16 17:19:56.901972344 +0000 UTC m=+0.457825209 container start b5e992b26b8c75efa266d3e12bcb7a8e3a3fbbfc72ce87dd576407b818c69b97 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-9-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, org.label-schema.build-date=20240409) 2024-04-16T17:19:57.227 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 16 17:19:56 smithi045 podman[104450]: 2024-04-16 17:19:56.999087564 +0000 UTC m=+0.554940446 container attach b5e992b26b8c75efa266d3e12bcb7a8e3a3fbbfc72ce87dd576407b818c69b97 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-9-deactivate, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240409) 2024-04-16T17:19:57.677 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:19:57 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:19:57.252+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-16T17:18:36.158791+0000 front 2024-04-16T17:18:36.158992+0000 (oldest deadline 2024-04-16T17:19:00.258603+0000) 2024-04-16T17:19:57.677 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:19:57 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:19:57.252+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-16T17:18:44.260046+0000 front 2024-04-16T17:18:44.260171+0000 (oldest deadline 2024-04-16T17:19:07.160168+0000) 2024-04-16T17:19:57.677 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:19:57 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:19:57.252+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-16T17:18:54.062069+0000 front 2024-04-16T17:18:54.061971+0000 (oldest deadline 2024-04-16T17:19:18.761808+0000) 2024-04-16T17:19:57.678 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:19:57 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:19:57.252+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-16T17:19:01.063265+0000 front 2024-04-16T17:19:01.063419+0000 (oldest deadline 2024-04-16T17:19:24.563191+0000) 2024-04-16T17:19:57.678 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:19:57 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:19:57.252+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-04-16T17:19:10.865363+0000 front 2024-04-16T17:19:10.865389+0000 (oldest deadline 2024-04-16T17:19:35.466166+0000) 2024-04-16T17:19:57.678 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:19:57 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:19:57.252+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.117:6812 osd.5 since back 2024-04-16T17:19:19.567307+0000 front 2024-04-16T17:19:19.567365+0000 (oldest deadline 2024-04-16T17:19:44.267279+0000) 2024-04-16T17:19:57.678 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:19:57 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:19:57.252+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.45:6822 osd.6 since back 2024-04-16T17:19:27.768425+0000 front 2024-04-16T17:19:27.768357+0000 (oldest deadline 2024-04-16T17:19:51.868043+0000) 2024-04-16T17:19:57.727 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 16 17:19:57 smithi045 podman[104450]: 2024-04-16 17:19:57.372216236 +0000 UTC m=+0.928069089 container died b5e992b26b8c75efa266d3e12bcb7a8e3a3fbbfc72ce87dd576407b818c69b97 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-9-deactivate, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240409, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-16T17:19:58.054 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:19:57 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:19:57.688+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-16T17:18:34.374061+0000 front 2024-04-16T17:18:34.374275+0000 (oldest deadline 2024-04-16T17:18:57.873879+0000) 2024-04-16T17:19:58.054 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:19:57 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:19:57.688+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-16T17:18:50.676854+0000 front 2024-04-16T17:18:50.676874+0000 (oldest deadline 2024-04-16T17:19:15.977004+0000) 2024-04-16T17:19:58.054 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:19:57 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:19:57.688+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-16T17:19:01.878146+0000 front 2024-04-16T17:19:01.878262+0000 (oldest deadline 2024-04-16T17:19:27.178048+0000) 2024-04-16T17:19:58.055 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:19:57 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:19:57.688+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-04-16T17:19:07.178684+0000 front 2024-04-16T17:19:07.178598+0000 (oldest deadline 2024-04-16T17:19:33.078600+0000) 2024-04-16T17:19:58.055 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:19:57 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:19:57.688+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.117:6812 osd.5 since back 2024-04-16T17:19:18.379616+0000 front 2024-04-16T17:19:18.379681+0000 (oldest deadline 2024-04-16T17:19:43.079667+0000) 2024-04-16T17:19:58.055 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:19:57 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:19:57.688+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.45:6822 osd.6 since back 2024-04-16T17:19:27.780981+0000 front 2024-04-16T17:19:27.781111+0000 (oldest deadline 2024-04-16T17:19:53.680822+0000) 2024-04-16T17:19:58.180 DEBUG:teuthology.orchestra.run.smithi045:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b1fecd38-fc11-11ee-bc90-c7b262605968@osd.9.service' 2024-04-16T17:19:58.350 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-16T17:19:58.351 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-04-16T17:19:58.351 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-04-16T17:19:58.351 DEBUG:teuthology.orchestra.run.smithi104:> sudo systemctl stop ceph-b1fecd38-fc11-11ee-bc90-c7b262605968@osd.10 2024-04-16T17:19:58.677 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:19:58 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:19:58.276+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-16T17:18:36.158791+0000 front 2024-04-16T17:18:36.158992+0000 (oldest deadline 2024-04-16T17:19:00.258603+0000) 2024-04-16T17:19:58.677 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:19:58 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:19:58.276+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-16T17:18:44.260046+0000 front 2024-04-16T17:18:44.260171+0000 (oldest deadline 2024-04-16T17:19:07.160168+0000) 2024-04-16T17:19:58.678 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:19:58 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:19:58.276+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-16T17:18:54.062069+0000 front 2024-04-16T17:18:54.061971+0000 (oldest deadline 2024-04-16T17:19:18.761808+0000) 2024-04-16T17:19:58.678 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:19:58 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:19:58.276+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-16T17:19:01.063265+0000 front 2024-04-16T17:19:01.063419+0000 (oldest deadline 2024-04-16T17:19:24.563191+0000) 2024-04-16T17:19:58.678 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:19:58 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:19:58.276+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-04-16T17:19:10.865363+0000 front 2024-04-16T17:19:10.865389+0000 (oldest deadline 2024-04-16T17:19:35.466166+0000) 2024-04-16T17:19:58.678 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:19:58 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:19:58.276+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.117:6812 osd.5 since back 2024-04-16T17:19:19.567307+0000 front 2024-04-16T17:19:19.567365+0000 (oldest deadline 2024-04-16T17:19:44.267279+0000) 2024-04-16T17:19:58.678 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:19:58 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:19:58.276+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.45:6822 osd.6 since back 2024-04-16T17:19:27.768425+0000 front 2024-04-16T17:19:27.768357+0000 (oldest deadline 2024-04-16T17:19:51.868043+0000) 2024-04-16T17:19:58.678 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:19:58 smithi104 systemd[1]: Stopping Ceph osd.10 for b1fecd38-fc11-11ee-bc90-c7b262605968... 2024-04-16T17:19:59.054 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:19:58 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:19:58.735+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-16T17:18:34.374061+0000 front 2024-04-16T17:18:34.374275+0000 (oldest deadline 2024-04-16T17:18:57.873879+0000) 2024-04-16T17:19:59.055 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:19:58 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:19:58.735+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-16T17:18:50.676854+0000 front 2024-04-16T17:18:50.676874+0000 (oldest deadline 2024-04-16T17:19:15.977004+0000) 2024-04-16T17:19:59.055 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:19:58 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:19:58.735+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-16T17:19:01.878146+0000 front 2024-04-16T17:19:01.878262+0000 (oldest deadline 2024-04-16T17:19:27.178048+0000) 2024-04-16T17:19:59.055 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:19:58 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:19:58.735+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-04-16T17:19:07.178684+0000 front 2024-04-16T17:19:07.178598+0000 (oldest deadline 2024-04-16T17:19:33.078600+0000) 2024-04-16T17:19:59.055 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:19:58 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:19:58.735+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.117:6812 osd.5 since back 2024-04-16T17:19:18.379616+0000 front 2024-04-16T17:19:18.379681+0000 (oldest deadline 2024-04-16T17:19:43.079667+0000) 2024-04-16T17:19:59.055 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:19:58 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:19:58.735+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.45:6822 osd.6 since back 2024-04-16T17:19:27.780981+0000 front 2024-04-16T17:19:27.781111+0000 (oldest deadline 2024-04-16T17:19:53.680822+0000) 2024-04-16T17:19:59.177 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:19:58 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:19:58.678+0000 7fc225710700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-16T17:19:59.177 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:19:58 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:19:58.678+0000 7fc225710700 -1 osd.10 112 *** Got signal Terminated *** 2024-04-16T17:19:59.177 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:19:58 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:19:58.678+0000 7fc225710700 -1 osd.10 112 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-16T17:19:59.677 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:19:59 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:19:59.277+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-16T17:18:36.158791+0000 front 2024-04-16T17:18:36.158992+0000 (oldest deadline 2024-04-16T17:19:00.258603+0000) 2024-04-16T17:19:59.677 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:19:59 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:19:59.277+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-16T17:18:44.260046+0000 front 2024-04-16T17:18:44.260171+0000 (oldest deadline 2024-04-16T17:19:07.160168+0000) 2024-04-16T17:19:59.677 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:19:59 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:19:59.277+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-16T17:18:54.062069+0000 front 2024-04-16T17:18:54.061971+0000 (oldest deadline 2024-04-16T17:19:18.761808+0000) 2024-04-16T17:19:59.677 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:19:59 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:19:59.277+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-16T17:19:01.063265+0000 front 2024-04-16T17:19:01.063419+0000 (oldest deadline 2024-04-16T17:19:24.563191+0000) 2024-04-16T17:19:59.678 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:19:59 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:19:59.277+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-04-16T17:19:10.865363+0000 front 2024-04-16T17:19:10.865389+0000 (oldest deadline 2024-04-16T17:19:35.466166+0000) 2024-04-16T17:19:59.678 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:19:59 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:19:59.277+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.117:6812 osd.5 since back 2024-04-16T17:19:19.567307+0000 front 2024-04-16T17:19:19.567365+0000 (oldest deadline 2024-04-16T17:19:44.267279+0000) 2024-04-16T17:19:59.678 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:19:59 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:19:59.277+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.45:6822 osd.6 since back 2024-04-16T17:19:27.768425+0000 front 2024-04-16T17:19:27.768357+0000 (oldest deadline 2024-04-16T17:19:51.868043+0000) 2024-04-16T17:20:00.054 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:19:59 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:19:59.723+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-16T17:18:34.374061+0000 front 2024-04-16T17:18:34.374275+0000 (oldest deadline 2024-04-16T17:18:57.873879+0000) 2024-04-16T17:20:00.055 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:19:59 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:19:59.723+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-16T17:18:50.676854+0000 front 2024-04-16T17:18:50.676874+0000 (oldest deadline 2024-04-16T17:19:15.977004+0000) 2024-04-16T17:20:00.055 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:19:59 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:19:59.723+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-16T17:19:01.878146+0000 front 2024-04-16T17:19:01.878262+0000 (oldest deadline 2024-04-16T17:19:27.178048+0000) 2024-04-16T17:20:00.055 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:19:59 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:19:59.723+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-04-16T17:19:07.178684+0000 front 2024-04-16T17:19:07.178598+0000 (oldest deadline 2024-04-16T17:19:33.078600+0000) 2024-04-16T17:20:00.055 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:19:59 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:19:59.723+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.117:6812 osd.5 since back 2024-04-16T17:19:18.379616+0000 front 2024-04-16T17:19:18.379681+0000 (oldest deadline 2024-04-16T17:19:43.079667+0000) 2024-04-16T17:20:00.055 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:19:59 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:19:59.723+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.45:6822 osd.6 since back 2024-04-16T17:19:27.780981+0000 front 2024-04-16T17:19:27.781111+0000 (oldest deadline 2024-04-16T17:19:53.680822+0000) 2024-04-16T17:20:00.677 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:20:00 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:20:00.277+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-16T17:18:36.158791+0000 front 2024-04-16T17:18:36.158992+0000 (oldest deadline 2024-04-16T17:19:00.258603+0000) 2024-04-16T17:20:00.677 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:20:00 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:20:00.277+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-16T17:18:44.260046+0000 front 2024-04-16T17:18:44.260171+0000 (oldest deadline 2024-04-16T17:19:07.160168+0000) 2024-04-16T17:20:00.677 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:20:00 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:20:00.277+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-16T17:18:54.062069+0000 front 2024-04-16T17:18:54.061971+0000 (oldest deadline 2024-04-16T17:19:18.761808+0000) 2024-04-16T17:20:00.677 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:20:00 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:20:00.277+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-16T17:19:01.063265+0000 front 2024-04-16T17:19:01.063419+0000 (oldest deadline 2024-04-16T17:19:24.563191+0000) 2024-04-16T17:20:00.677 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:20:00 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:20:00.277+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-04-16T17:19:10.865363+0000 front 2024-04-16T17:19:10.865389+0000 (oldest deadline 2024-04-16T17:19:35.466166+0000) 2024-04-16T17:20:00.677 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:20:00 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:20:00.277+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.117:6812 osd.5 since back 2024-04-16T17:19:19.567307+0000 front 2024-04-16T17:19:19.567365+0000 (oldest deadline 2024-04-16T17:19:44.267279+0000) 2024-04-16T17:20:00.677 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:20:00 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:20:00.277+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.45:6822 osd.6 since back 2024-04-16T17:19:27.768425+0000 front 2024-04-16T17:19:27.768357+0000 (oldest deadline 2024-04-16T17:19:51.868043+0000) 2024-04-16T17:20:01.054 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:20:00 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:20:00.701+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-16T17:18:34.374061+0000 front 2024-04-16T17:18:34.374275+0000 (oldest deadline 2024-04-16T17:18:57.873879+0000) 2024-04-16T17:20:01.054 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:20:00 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:20:00.701+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-16T17:18:50.676854+0000 front 2024-04-16T17:18:50.676874+0000 (oldest deadline 2024-04-16T17:19:15.977004+0000) 2024-04-16T17:20:01.054 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:20:00 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:20:00.701+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-16T17:19:01.878146+0000 front 2024-04-16T17:19:01.878262+0000 (oldest deadline 2024-04-16T17:19:27.178048+0000) 2024-04-16T17:20:01.054 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:20:00 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:20:00.701+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-04-16T17:19:07.178684+0000 front 2024-04-16T17:19:07.178598+0000 (oldest deadline 2024-04-16T17:19:33.078600+0000) 2024-04-16T17:20:01.055 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:20:00 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:20:00.701+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.117:6812 osd.5 since back 2024-04-16T17:19:18.379616+0000 front 2024-04-16T17:19:18.379681+0000 (oldest deadline 2024-04-16T17:19:43.079667+0000) 2024-04-16T17:20:01.055 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:20:00 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:20:00.701+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.45:6822 osd.6 since back 2024-04-16T17:19:27.780981+0000 front 2024-04-16T17:19:27.781111+0000 (oldest deadline 2024-04-16T17:19:53.680822+0000) 2024-04-16T17:20:01.055 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:20:00 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:20:00.701+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.104:6820 osd.7 since back 2024-04-16T17:19:36.581793+0000 front 2024-04-16T17:19:36.581901+0000 (oldest deadline 2024-04-16T17:20:00.681641+0000) 2024-04-16T17:20:01.677 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:20:01 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:20:01.256+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-16T17:18:36.158791+0000 front 2024-04-16T17:18:36.158992+0000 (oldest deadline 2024-04-16T17:19:00.258603+0000) 2024-04-16T17:20:01.677 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:20:01 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:20:01.256+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-16T17:18:44.260046+0000 front 2024-04-16T17:18:44.260171+0000 (oldest deadline 2024-04-16T17:19:07.160168+0000) 2024-04-16T17:20:01.677 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:20:01 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:20:01.256+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-16T17:18:54.062069+0000 front 2024-04-16T17:18:54.061971+0000 (oldest deadline 2024-04-16T17:19:18.761808+0000) 2024-04-16T17:20:01.677 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:20:01 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:20:01.256+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-16T17:19:01.063265+0000 front 2024-04-16T17:19:01.063419+0000 (oldest deadline 2024-04-16T17:19:24.563191+0000) 2024-04-16T17:20:01.677 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:20:01 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:20:01.256+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-04-16T17:19:10.865363+0000 front 2024-04-16T17:19:10.865389+0000 (oldest deadline 2024-04-16T17:19:35.466166+0000) 2024-04-16T17:20:01.677 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:20:01 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:20:01.256+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.117:6812 osd.5 since back 2024-04-16T17:19:19.567307+0000 front 2024-04-16T17:19:19.567365+0000 (oldest deadline 2024-04-16T17:19:44.267279+0000) 2024-04-16T17:20:01.678 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:20:01 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:20:01.256+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.45:6822 osd.6 since back 2024-04-16T17:19:27.768425+0000 front 2024-04-16T17:19:27.768357+0000 (oldest deadline 2024-04-16T17:19:51.868043+0000) 2024-04-16T17:20:02.054 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:20:01 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:20:01.738+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-16T17:18:34.374061+0000 front 2024-04-16T17:18:34.374275+0000 (oldest deadline 2024-04-16T17:18:57.873879+0000) 2024-04-16T17:20:02.054 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:20:01 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:20:01.738+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-16T17:18:50.676854+0000 front 2024-04-16T17:18:50.676874+0000 (oldest deadline 2024-04-16T17:19:15.977004+0000) 2024-04-16T17:20:02.055 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:20:01 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:20:01.738+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-16T17:19:01.878146+0000 front 2024-04-16T17:19:01.878262+0000 (oldest deadline 2024-04-16T17:19:27.178048+0000) 2024-04-16T17:20:02.055 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:20:01 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:20:01.738+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-04-16T17:19:07.178684+0000 front 2024-04-16T17:19:07.178598+0000 (oldest deadline 2024-04-16T17:19:33.078600+0000) 2024-04-16T17:20:02.055 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:20:01 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:20:01.738+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.117:6812 osd.5 since back 2024-04-16T17:19:18.379616+0000 front 2024-04-16T17:19:18.379681+0000 (oldest deadline 2024-04-16T17:19:43.079667+0000) 2024-04-16T17:20:02.055 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:20:01 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:20:01.738+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.45:6822 osd.6 since back 2024-04-16T17:19:27.780981+0000 front 2024-04-16T17:19:27.781111+0000 (oldest deadline 2024-04-16T17:19:53.680822+0000) 2024-04-16T17:20:02.055 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:20:01 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:20:01.738+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.104:6820 osd.7 since back 2024-04-16T17:19:36.581793+0000 front 2024-04-16T17:19:36.581901+0000 (oldest deadline 2024-04-16T17:20:00.681641+0000) 2024-04-16T17:20:02.676 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:20:02 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:20:02.245+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-16T17:18:36.158791+0000 front 2024-04-16T17:18:36.158992+0000 (oldest deadline 2024-04-16T17:19:00.258603+0000) 2024-04-16T17:20:02.677 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:20:02 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:20:02.245+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-16T17:18:44.260046+0000 front 2024-04-16T17:18:44.260171+0000 (oldest deadline 2024-04-16T17:19:07.160168+0000) 2024-04-16T17:20:02.677 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:20:02 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:20:02.245+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-16T17:18:54.062069+0000 front 2024-04-16T17:18:54.061971+0000 (oldest deadline 2024-04-16T17:19:18.761808+0000) 2024-04-16T17:20:02.677 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:20:02 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:20:02.245+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-16T17:19:01.063265+0000 front 2024-04-16T17:19:01.063419+0000 (oldest deadline 2024-04-16T17:19:24.563191+0000) 2024-04-16T17:20:02.677 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:20:02 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:20:02.245+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-04-16T17:19:10.865363+0000 front 2024-04-16T17:19:10.865389+0000 (oldest deadline 2024-04-16T17:19:35.466166+0000) 2024-04-16T17:20:02.677 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:20:02 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:20:02.245+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.117:6812 osd.5 since back 2024-04-16T17:19:19.567307+0000 front 2024-04-16T17:19:19.567365+0000 (oldest deadline 2024-04-16T17:19:44.267279+0000) 2024-04-16T17:20:02.677 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:20:02 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:20:02.245+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.45:6822 osd.6 since back 2024-04-16T17:19:27.768425+0000 front 2024-04-16T17:19:27.768357+0000 (oldest deadline 2024-04-16T17:19:51.868043+0000) 2024-04-16T17:20:03.054 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:20:02 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:20:02.774+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-16T17:18:34.374061+0000 front 2024-04-16T17:18:34.374275+0000 (oldest deadline 2024-04-16T17:18:57.873879+0000) 2024-04-16T17:20:03.055 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:20:02 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:20:02.774+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-16T17:18:50.676854+0000 front 2024-04-16T17:18:50.676874+0000 (oldest deadline 2024-04-16T17:19:15.977004+0000) 2024-04-16T17:20:03.055 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:20:02 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:20:02.774+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-16T17:19:01.878146+0000 front 2024-04-16T17:19:01.878262+0000 (oldest deadline 2024-04-16T17:19:27.178048+0000) 2024-04-16T17:20:03.055 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:20:02 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:20:02.774+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-04-16T17:19:07.178684+0000 front 2024-04-16T17:19:07.178598+0000 (oldest deadline 2024-04-16T17:19:33.078600+0000) 2024-04-16T17:20:03.055 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:20:02 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:20:02.774+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.117:6812 osd.5 since back 2024-04-16T17:19:18.379616+0000 front 2024-04-16T17:19:18.379681+0000 (oldest deadline 2024-04-16T17:19:43.079667+0000) 2024-04-16T17:20:03.055 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:20:02 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:20:02.774+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.45:6822 osd.6 since back 2024-04-16T17:19:27.780981+0000 front 2024-04-16T17:19:27.781111+0000 (oldest deadline 2024-04-16T17:19:53.680822+0000) 2024-04-16T17:20:03.055 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:20:02 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:20:02.774+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.104:6820 osd.7 since back 2024-04-16T17:19:36.581793+0000 front 2024-04-16T17:19:36.581901+0000 (oldest deadline 2024-04-16T17:20:00.681641+0000) 2024-04-16T17:20:03.677 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:20:03 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:20:03.267+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-16T17:18:36.158791+0000 front 2024-04-16T17:18:36.158992+0000 (oldest deadline 2024-04-16T17:19:00.258603+0000) 2024-04-16T17:20:03.677 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:20:03 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:20:03.267+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-04-16T17:18:44.260046+0000 front 2024-04-16T17:18:44.260171+0000 (oldest deadline 2024-04-16T17:19:07.160168+0000) 2024-04-16T17:20:03.678 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:20:03 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:20:03.267+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-16T17:18:54.062069+0000 front 2024-04-16T17:18:54.061971+0000 (oldest deadline 2024-04-16T17:19:18.761808+0000) 2024-04-16T17:20:03.678 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:20:03 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:20:03.267+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-16T17:19:01.063265+0000 front 2024-04-16T17:19:01.063419+0000 (oldest deadline 2024-04-16T17:19:24.563191+0000) 2024-04-16T17:20:03.678 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:20:03 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:20:03.267+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-04-16T17:19:10.865363+0000 front 2024-04-16T17:19:10.865389+0000 (oldest deadline 2024-04-16T17:19:35.466166+0000) 2024-04-16T17:20:03.678 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:20:03 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:20:03.267+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.117:6812 osd.5 since back 2024-04-16T17:19:19.567307+0000 front 2024-04-16T17:19:19.567365+0000 (oldest deadline 2024-04-16T17:19:44.267279+0000) 2024-04-16T17:20:03.678 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:20:03 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10[62090]: 2024-04-16T17:20:03.267+0000 7fc21dcfc700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.45:6822 osd.6 since back 2024-04-16T17:19:27.768425+0000 front 2024-04-16T17:19:27.768357+0000 (oldest deadline 2024-04-16T17:19:51.868043+0000) 2024-04-16T17:20:04.054 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:20:03 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:20:03.757+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-16T17:18:34.374061+0000 front 2024-04-16T17:18:34.374275+0000 (oldest deadline 2024-04-16T17:18:57.873879+0000) 2024-04-16T17:20:04.054 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:20:03 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:20:03.757+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-16T17:18:50.676854+0000 front 2024-04-16T17:18:50.676874+0000 (oldest deadline 2024-04-16T17:19:15.977004+0000) 2024-04-16T17:20:04.055 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:20:03 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:20:03.757+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-16T17:19:01.878146+0000 front 2024-04-16T17:19:01.878262+0000 (oldest deadline 2024-04-16T17:19:27.178048+0000) 2024-04-16T17:20:04.055 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:20:03 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:20:03.757+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-04-16T17:19:07.178684+0000 front 2024-04-16T17:19:07.178598+0000 (oldest deadline 2024-04-16T17:19:33.078600+0000) 2024-04-16T17:20:04.055 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:20:03 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:20:03.757+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.117:6812 osd.5 since back 2024-04-16T17:19:18.379616+0000 front 2024-04-16T17:19:18.379681+0000 (oldest deadline 2024-04-16T17:19:43.079667+0000) 2024-04-16T17:20:04.055 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:20:03 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:20:03.757+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.45:6822 osd.6 since back 2024-04-16T17:19:27.780981+0000 front 2024-04-16T17:19:27.781111+0000 (oldest deadline 2024-04-16T17:19:53.680822+0000) 2024-04-16T17:20:04.055 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:20:03 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:20:03.757+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.104:6820 osd.7 since back 2024-04-16T17:19:36.581793+0000 front 2024-04-16T17:19:36.581901+0000 (oldest deadline 2024-04-16T17:20:00.681641+0000) 2024-04-16T17:20:04.494 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:20:04 smithi104 podman[75121]: 2024-04-16 17:20:04.234467482 +0000 UTC m=+5.693608556 container died 3e58719018ae72957ffbf7e8b2ce4ee35e39e6a30a15a36d667f388911b68be3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240409, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-16T17:20:04.494 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:20:04 smithi104 podman[75121]: 2024-04-16 17:20:04.494989344 +0000 UTC m=+5.954130428 container cleanup 3e58719018ae72957ffbf7e8b2ce4ee35e39e6a30a15a36d667f388911b68be3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240409, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-04-16T17:20:04.927 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:20:04 smithi104 bash[75121]: ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10 2024-04-16T17:20:04.927 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:20:04 smithi104 podman[75140]: 2024-04-16 17:20:04.721517876 +0000 UTC m=+0.485626826 container remove 3e58719018ae72957ffbf7e8b2ce4ee35e39e6a30a15a36d667f388911b68be3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240409, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-16T17:20:05.054 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:20:04 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:20:04.754+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-16T17:18:34.374061+0000 front 2024-04-16T17:18:34.374275+0000 (oldest deadline 2024-04-16T17:18:57.873879+0000) 2024-04-16T17:20:05.054 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:20:04 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:20:04.754+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-16T17:18:50.676854+0000 front 2024-04-16T17:18:50.676874+0000 (oldest deadline 2024-04-16T17:19:15.977004+0000) 2024-04-16T17:20:05.054 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:20:04 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:20:04.754+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-16T17:19:01.878146+0000 front 2024-04-16T17:19:01.878262+0000 (oldest deadline 2024-04-16T17:19:27.178048+0000) 2024-04-16T17:20:05.055 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:20:04 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:20:04.754+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-04-16T17:19:07.178684+0000 front 2024-04-16T17:19:07.178598+0000 (oldest deadline 2024-04-16T17:19:33.078600+0000) 2024-04-16T17:20:05.055 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:20:04 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:20:04.754+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.117:6812 osd.5 since back 2024-04-16T17:19:18.379616+0000 front 2024-04-16T17:19:18.379681+0000 (oldest deadline 2024-04-16T17:19:43.079667+0000) 2024-04-16T17:20:05.055 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:20:04 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:20:04.754+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.45:6822 osd.6 since back 2024-04-16T17:19:27.780981+0000 front 2024-04-16T17:19:27.781111+0000 (oldest deadline 2024-04-16T17:19:53.680822+0000) 2024-04-16T17:20:05.055 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:20:04 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:20:04.754+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.104:6820 osd.7 since back 2024-04-16T17:19:36.581793+0000 front 2024-04-16T17:19:36.581901+0000 (oldest deadline 2024-04-16T17:20:00.681641+0000) 2024-04-16T17:20:05.349 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:20:05 smithi104 podman[75232]: 2024-04-16 17:20:04.949496972 +0000 UTC m=+0.017535743 image pull e5ce10bdffbeeffffd1125f67b369e2151f0f17cdec854e93b6adf9321f65907 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2 2024-04-16T17:20:05.349 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:20:05 smithi104 podman[75232]: 2024-04-16 17:20:05.060504795 +0000 UTC m=+0.128543543 container create 46f92258f056b310435ea103b7bbd453204818a3ebe52cb96c0a0db0d64089c0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10-deactivate, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240409, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-04-16T17:20:05.349 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:20:05 smithi104 podman[75232]: 2024-04-16 17:20:05.284033029 +0000 UTC m=+0.352071786 container init 46f92258f056b310435ea103b7bbd453204818a3ebe52cb96c0a0db0d64089c0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10-deactivate, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240409, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, ceph=True) 2024-04-16T17:20:05.350 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:20:05 smithi104 podman[75232]: 2024-04-16 17:20:05.287259555 +0000 UTC m=+0.355298302 container start 46f92258f056b310435ea103b7bbd453204818a3ebe52cb96c0a0db0d64089c0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10-deactivate, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240409, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-04-16T17:20:05.350 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:20:05 smithi104 podman[75232]: 2024-04-16 17:20:05.350279301 +0000 UTC m=+0.418318071 container attach 46f92258f056b310435ea103b7bbd453204818a3ebe52cb96c0a0db0d64089c0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10-deactivate, org.label-schema.build-date=20240409, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-16T17:20:05.676 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:20:05 smithi104 podman[75232]: 2024-04-16 17:20:05.578358687 +0000 UTC m=+0.646397443 container died 46f92258f056b310435ea103b7bbd453204818a3ebe52cb96c0a0db0d64089c0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.build-date=20240409, maintainer=Guillaume Abrioux ) 2024-04-16T17:20:06.054 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:20:05 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:20:05.738+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-16T17:18:34.374061+0000 front 2024-04-16T17:18:34.374275+0000 (oldest deadline 2024-04-16T17:18:57.873879+0000) 2024-04-16T17:20:06.055 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:20:05 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:20:05.738+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-16T17:18:50.676854+0000 front 2024-04-16T17:18:50.676874+0000 (oldest deadline 2024-04-16T17:19:15.977004+0000) 2024-04-16T17:20:06.055 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:20:05 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:20:05.738+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-16T17:19:01.878146+0000 front 2024-04-16T17:19:01.878262+0000 (oldest deadline 2024-04-16T17:19:27.178048+0000) 2024-04-16T17:20:06.055 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:20:05 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:20:05.738+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-04-16T17:19:07.178684+0000 front 2024-04-16T17:19:07.178598+0000 (oldest deadline 2024-04-16T17:19:33.078600+0000) 2024-04-16T17:20:06.055 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:20:05 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:20:05.738+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.117:6812 osd.5 since back 2024-04-16T17:19:18.379616+0000 front 2024-04-16T17:19:18.379681+0000 (oldest deadline 2024-04-16T17:19:43.079667+0000) 2024-04-16T17:20:06.055 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:20:05 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:20:05.738+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.45:6822 osd.6 since back 2024-04-16T17:19:27.780981+0000 front 2024-04-16T17:19:27.781111+0000 (oldest deadline 2024-04-16T17:19:53.680822+0000) 2024-04-16T17:20:06.055 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:20:05 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:20:05.738+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.104:6820 osd.7 since back 2024-04-16T17:19:36.581793+0000 front 2024-04-16T17:19:36.581901+0000 (oldest deadline 2024-04-16T17:20:00.681641+0000) 2024-04-16T17:20:06.094 DEBUG:teuthology.orchestra.run.smithi104:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b1fecd38-fc11-11ee-bc90-c7b262605968@osd.10.service' 2024-04-16T17:20:06.119 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:20:06 smithi104 podman[75258]: 2024-04-16 17:20:06.070699398 +0000 UTC m=+0.482362655 container remove 46f92258f056b310435ea103b7bbd453204818a3ebe52cb96c0a0db0d64089c0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-10-deactivate, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, org.label-schema.license=GPLv2, org.label-schema.build-date=20240409, maintainer=Guillaume Abrioux ) 2024-04-16T17:20:06.119 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:20:06 smithi104 systemd[1]: ceph-b1fecd38-fc11-11ee-bc90-c7b262605968@osd.10.service: Deactivated successfully. 2024-04-16T17:20:06.120 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:20:06 smithi104 systemd[1]: Stopped Ceph osd.10 for b1fecd38-fc11-11ee-bc90-c7b262605968. 2024-04-16T17:20:06.120 INFO:journalctl@ceph.osd.10.smithi104.stdout:Apr 16 17:20:06 smithi104 systemd[1]: ceph-b1fecd38-fc11-11ee-bc90-c7b262605968@osd.10.service: Consumed 25.414s CPU time. 2024-04-16T17:20:06.134 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-16T17:20:06.134 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-04-16T17:20:06.134 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-04-16T17:20:06.134 DEBUG:teuthology.orchestra.run.smithi117:> sudo systemctl stop ceph-b1fecd38-fc11-11ee-bc90-c7b262605968@osd.11 2024-04-16T17:20:06.446 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:20:06 smithi117 systemd[1]: Stopping Ceph osd.11 for b1fecd38-fc11-11ee-bc90-c7b262605968... 2024-04-16T17:20:06.715 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:20:06 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:20:06.446+0000 7f713d50d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-16T17:20:06.715 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:20:06 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:20:06.446+0000 7f713d50d700 -1 osd.11 112 *** Got signal Terminated *** 2024-04-16T17:20:06.715 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:20:06 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:20:06.446+0000 7f713d50d700 -1 osd.11 112 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-16T17:20:06.715 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:20:06 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:20:06.715+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-16T17:18:34.374061+0000 front 2024-04-16T17:18:34.374275+0000 (oldest deadline 2024-04-16T17:18:57.873879+0000) 2024-04-16T17:20:06.715 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:20:06 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:20:06.715+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-16T17:18:50.676854+0000 front 2024-04-16T17:18:50.676874+0000 (oldest deadline 2024-04-16T17:19:15.977004+0000) 2024-04-16T17:20:06.715 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:20:06 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:20:06.715+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-16T17:19:01.878146+0000 front 2024-04-16T17:19:01.878262+0000 (oldest deadline 2024-04-16T17:19:27.178048+0000) 2024-04-16T17:20:06.715 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:20:06 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:20:06.715+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-04-16T17:19:07.178684+0000 front 2024-04-16T17:19:07.178598+0000 (oldest deadline 2024-04-16T17:19:33.078600+0000) 2024-04-16T17:20:06.716 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:20:06 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:20:06.715+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.117:6812 osd.5 since back 2024-04-16T17:19:18.379616+0000 front 2024-04-16T17:19:18.379681+0000 (oldest deadline 2024-04-16T17:19:43.079667+0000) 2024-04-16T17:20:06.716 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:20:06 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:20:06.715+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.45:6822 osd.6 since back 2024-04-16T17:19:27.780981+0000 front 2024-04-16T17:19:27.781111+0000 (oldest deadline 2024-04-16T17:19:53.680822+0000) 2024-04-16T17:20:06.716 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:20:06 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:20:06.715+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.104:6820 osd.7 since back 2024-04-16T17:19:36.581793+0000 front 2024-04-16T17:19:36.581901+0000 (oldest deadline 2024-04-16T17:20:00.681641+0000) 2024-04-16T17:20:08.054 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:20:07 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:20:07.696+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-16T17:18:34.374061+0000 front 2024-04-16T17:18:34.374275+0000 (oldest deadline 2024-04-16T17:18:57.873879+0000) 2024-04-16T17:20:08.054 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:20:07 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:20:07.696+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-16T17:18:50.676854+0000 front 2024-04-16T17:18:50.676874+0000 (oldest deadline 2024-04-16T17:19:15.977004+0000) 2024-04-16T17:20:08.054 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:20:07 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:20:07.696+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-16T17:19:01.878146+0000 front 2024-04-16T17:19:01.878262+0000 (oldest deadline 2024-04-16T17:19:27.178048+0000) 2024-04-16T17:20:08.055 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:20:07 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:20:07.696+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-04-16T17:19:07.178684+0000 front 2024-04-16T17:19:07.178598+0000 (oldest deadline 2024-04-16T17:19:33.078600+0000) 2024-04-16T17:20:08.055 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:20:07 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:20:07.696+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.117:6812 osd.5 since back 2024-04-16T17:19:18.379616+0000 front 2024-04-16T17:19:18.379681+0000 (oldest deadline 2024-04-16T17:19:43.079667+0000) 2024-04-16T17:20:08.055 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:20:07 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:20:07.696+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.45:6822 osd.6 since back 2024-04-16T17:19:27.780981+0000 front 2024-04-16T17:19:27.781111+0000 (oldest deadline 2024-04-16T17:19:53.680822+0000) 2024-04-16T17:20:08.055 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:20:07 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:20:07.696+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.104:6820 osd.7 since back 2024-04-16T17:19:36.581793+0000 front 2024-04-16T17:19:36.581901+0000 (oldest deadline 2024-04-16T17:20:00.681641+0000) 2024-04-16T17:20:09.054 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:20:08 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:20:08.667+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-16T17:18:34.374061+0000 front 2024-04-16T17:18:34.374275+0000 (oldest deadline 2024-04-16T17:18:57.873879+0000) 2024-04-16T17:20:09.055 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:20:08 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:20:08.667+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-16T17:18:50.676854+0000 front 2024-04-16T17:18:50.676874+0000 (oldest deadline 2024-04-16T17:19:15.977004+0000) 2024-04-16T17:20:09.055 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:20:08 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:20:08.667+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-16T17:19:01.878146+0000 front 2024-04-16T17:19:01.878262+0000 (oldest deadline 2024-04-16T17:19:27.178048+0000) 2024-04-16T17:20:09.055 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:20:08 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:20:08.667+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-04-16T17:19:07.178684+0000 front 2024-04-16T17:19:07.178598+0000 (oldest deadline 2024-04-16T17:19:33.078600+0000) 2024-04-16T17:20:09.055 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:20:08 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:20:08.667+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.117:6812 osd.5 since back 2024-04-16T17:19:18.379616+0000 front 2024-04-16T17:19:18.379681+0000 (oldest deadline 2024-04-16T17:19:43.079667+0000) 2024-04-16T17:20:09.055 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:20:08 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:20:08.667+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.45:6822 osd.6 since back 2024-04-16T17:19:27.780981+0000 front 2024-04-16T17:19:27.781111+0000 (oldest deadline 2024-04-16T17:19:53.680822+0000) 2024-04-16T17:20:09.055 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:20:08 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:20:08.667+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.104:6820 osd.7 since back 2024-04-16T17:19:36.581793+0000 front 2024-04-16T17:19:36.581901+0000 (oldest deadline 2024-04-16T17:20:00.681641+0000) 2024-04-16T17:20:09.055 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:20:08 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:20:08.667+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.117:6820 osd.8 since back 2024-04-16T17:19:45.382387+0000 front 2024-04-16T17:19:45.382439+0000 (oldest deadline 2024-04-16T17:20:08.282395+0000) 2024-04-16T17:20:10.054 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:20:09 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:20:09.700+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-16T17:18:34.374061+0000 front 2024-04-16T17:18:34.374275+0000 (oldest deadline 2024-04-16T17:18:57.873879+0000) 2024-04-16T17:20:10.055 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:20:09 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:20:09.700+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-16T17:18:50.676854+0000 front 2024-04-16T17:18:50.676874+0000 (oldest deadline 2024-04-16T17:19:15.977004+0000) 2024-04-16T17:20:10.055 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:20:09 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:20:09.700+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-16T17:19:01.878146+0000 front 2024-04-16T17:19:01.878262+0000 (oldest deadline 2024-04-16T17:19:27.178048+0000) 2024-04-16T17:20:10.055 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:20:09 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:20:09.700+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-04-16T17:19:07.178684+0000 front 2024-04-16T17:19:07.178598+0000 (oldest deadline 2024-04-16T17:19:33.078600+0000) 2024-04-16T17:20:10.055 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:20:09 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:20:09.700+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.117:6812 osd.5 since back 2024-04-16T17:19:18.379616+0000 front 2024-04-16T17:19:18.379681+0000 (oldest deadline 2024-04-16T17:19:43.079667+0000) 2024-04-16T17:20:10.055 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:20:09 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:20:09.700+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.45:6822 osd.6 since back 2024-04-16T17:19:27.780981+0000 front 2024-04-16T17:19:27.781111+0000 (oldest deadline 2024-04-16T17:19:53.680822+0000) 2024-04-16T17:20:10.055 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:20:09 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:20:09.700+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.104:6820 osd.7 since back 2024-04-16T17:19:36.581793+0000 front 2024-04-16T17:19:36.581901+0000 (oldest deadline 2024-04-16T17:20:00.681641+0000) 2024-04-16T17:20:10.055 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:20:09 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:20:09.700+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.117:6820 osd.8 since back 2024-04-16T17:19:45.382387+0000 front 2024-04-16T17:19:45.382439+0000 (oldest deadline 2024-04-16T17:20:08.282395+0000) 2024-04-16T17:20:11.054 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:20:10 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:20:10.652+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-16T17:18:34.374061+0000 front 2024-04-16T17:18:34.374275+0000 (oldest deadline 2024-04-16T17:18:57.873879+0000) 2024-04-16T17:20:11.055 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:20:10 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:20:10.652+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.117:6804 osd.2 since back 2024-04-16T17:18:50.676854+0000 front 2024-04-16T17:18:50.676874+0000 (oldest deadline 2024-04-16T17:19:15.977004+0000) 2024-04-16T17:20:11.055 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:20:10 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:20:10.652+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-16T17:19:01.878146+0000 front 2024-04-16T17:19:01.878262+0000 (oldest deadline 2024-04-16T17:19:27.178048+0000) 2024-04-16T17:20:11.055 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:20:10 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:20:10.652+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-04-16T17:19:07.178684+0000 front 2024-04-16T17:19:07.178598+0000 (oldest deadline 2024-04-16T17:19:33.078600+0000) 2024-04-16T17:20:11.055 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:20:10 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:20:10.652+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.117:6812 osd.5 since back 2024-04-16T17:19:18.379616+0000 front 2024-04-16T17:19:18.379681+0000 (oldest deadline 2024-04-16T17:19:43.079667+0000) 2024-04-16T17:20:11.055 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:20:10 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:20:10.652+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.45:6822 osd.6 since back 2024-04-16T17:19:27.780981+0000 front 2024-04-16T17:19:27.781111+0000 (oldest deadline 2024-04-16T17:19:53.680822+0000) 2024-04-16T17:20:11.055 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:20:10 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:20:10.652+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.104:6820 osd.7 since back 2024-04-16T17:19:36.581793+0000 front 2024-04-16T17:19:36.581901+0000 (oldest deadline 2024-04-16T17:20:00.681641+0000) 2024-04-16T17:20:11.055 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:20:10 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11[62651]: 2024-04-16T17:20:10.652+0000 7f7139b01700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.117:6820 osd.8 since back 2024-04-16T17:19:45.382387+0000 front 2024-04-16T17:19:45.382439+0000 (oldest deadline 2024-04-16T17:20:08.282395+0000) 2024-04-16T17:20:12.130 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:20:11 smithi117 podman[75357]: 2024-04-16 17:20:11.872744379 +0000 UTC m=+5.544476180 container died 6ef904b3d0029e522b675e0f475e4f63f63e1c2e8e61bc23c43a2a44d0635ec2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240409, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-04-16T17:20:12.396 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:20:12 smithi117 podman[75357]: 2024-04-16 17:20:12.196652534 +0000 UTC m=+5.868384333 container cleanup 6ef904b3d0029e522b675e0f475e4f63f63e1c2e8e61bc23c43a2a44d0635ec2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240409, org.label-schema.schema-version=1.0, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS) 2024-04-16T17:20:12.396 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:20:12 smithi117 bash[75357]: ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11 2024-04-16T17:20:12.797 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:20:12 smithi117 podman[75374]: 2024-04-16 17:20:12.397661503 +0000 UTC m=+0.520346452 container remove 6ef904b3d0029e522b675e0f475e4f63f63e1c2e8e61bc23c43a2a44d0635ec2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240409, CEPH_POINT_RELEASE=, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-04-16T17:20:13.054 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:20:12 smithi117 podman[75464]: 2024-04-16 17:20:12.798283592 +0000 UTC m=+0.111912681 container create 4301b6ce30a233342505a0219f7b3b1fd24cd1cb3da2d36d898ed1bec36d9d7d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11-deactivate, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240409, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-04-16T17:20:13.054 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:20:12 smithi117 podman[75464]: 2024-04-16 17:20:12.703572868 +0000 UTC m=+0.017201970 image pull e5ce10bdffbeeffffd1125f67b369e2151f0f17cdec854e93b6adf9321f65907 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2 2024-04-16T17:20:13.055 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:20:13 smithi117 podman[75464]: 2024-04-16 17:20:13.029632544 +0000 UTC m=+0.343261640 container init 4301b6ce30a233342505a0219f7b3b1fd24cd1cb3da2d36d898ed1bec36d9d7d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11-deactivate, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240409, ceph=True, org.label-schema.license=GPLv2) 2024-04-16T17:20:13.055 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:20:13 smithi117 podman[75464]: 2024-04-16 17:20:13.033104005 +0000 UTC m=+0.346733091 container start 4301b6ce30a233342505a0219f7b3b1fd24cd1cb3da2d36d898ed1bec36d9d7d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11-deactivate, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240409, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-04-16T17:20:13.554 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:20:13 smithi117 podman[75464]: 2024-04-16 17:20:13.134037291 +0000 UTC m=+0.447666383 container attach 4301b6ce30a233342505a0219f7b3b1fd24cd1cb3da2d36d898ed1bec36d9d7d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240409, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-16T17:20:13.554 INFO:journalctl@ceph.osd.11.smithi117.stdout:Apr 16 17:20:13 smithi117 podman[75464]: 2024-04-16 17:20:13.326696876 +0000 UTC m=+0.640325978 container died 4301b6ce30a233342505a0219f7b3b1fd24cd1cb3da2d36d898ed1bec36d9d7d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-osd-11-deactivate, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240409, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-16T17:20:13.954 DEBUG:teuthology.orchestra.run.smithi117:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b1fecd38-fc11-11ee-bc90-c7b262605968@osd.11.service' 2024-04-16T17:20:13.995 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-16T17:20:13.996 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-04-16T17:20:13.996 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-04-16T17:20:13.996 DEBUG:teuthology.orchestra.run.smithi045:> sudo systemctl stop ceph-b1fecd38-fc11-11ee-bc90-c7b262605968@mds.a 2024-04-16T17:20:14.476 INFO:journalctl@ceph.mds.a.smithi045.stdout:Apr 16 17:20:14 smithi045 systemd[1]: Stopping Ceph mds.a for b1fecd38-fc11-11ee-bc90-c7b262605968... 2024-04-16T17:20:14.977 INFO:journalctl@ceph.mds.a.smithi045.stdout:Apr 16 17:20:14 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mds-a[73384]: 2024-04-16T17:20:14.589+0000 7f4999400700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-16T17:20:14.977 INFO:journalctl@ceph.mds.a.smithi045.stdout:Apr 16 17:20:14 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mds-a[73384]: 2024-04-16T17:20:14.589+0000 7f4999400700 -1 mds.a *** got signal Terminated *** 2024-04-16T17:20:17.929 INFO:journalctl@ceph.mds.a.smithi045.stdout:Apr 16 17:20:17 smithi045 podman[104556]: 2024-04-16 17:20:17.590801038 +0000 UTC m=+3.128375532 container stop 070e1db7eaf3b2fb2d8a18804a330f93c3a73c41dc144ec08371030cf1c4d759 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mds-a, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.build-date=20240409, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-04-16T17:20:17.929 INFO:journalctl@ceph.mds.a.smithi045.stdout:Apr 16 17:20:17 smithi045 podman[104556]: 2024-04-16 17:20:17.617325854 +0000 UTC m=+3.154900366 container died 070e1db7eaf3b2fb2d8a18804a330f93c3a73c41dc144ec08371030cf1c4d759 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mds-a, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, org.label-schema.build-date=20240409, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-04-16T17:20:18.227 INFO:journalctl@ceph.mds.a.smithi045.stdout:Apr 16 17:20:18 smithi045 podman[104556]: 2024-04-16 17:20:18.041043185 +0000 UTC m=+3.578617704 container cleanup 070e1db7eaf3b2fb2d8a18804a330f93c3a73c41dc144ec08371030cf1c4d759 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mds-a, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240409, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb) 2024-04-16T17:20:18.227 INFO:journalctl@ceph.mds.a.smithi045.stdout:Apr 16 17:20:18 smithi045 bash[104556]: ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mds-a 2024-04-16T17:20:18.707 INFO:journalctl@ceph.mds.a.smithi045.stdout:Apr 16 17:20:18 smithi045 podman[104569]: 2024-04-16 17:20:18.241945957 +0000 UTC m=+0.631658614 container remove 070e1db7eaf3b2fb2d8a18804a330f93c3a73c41dc144ec08371030cf1c4d759 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mds-a, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240409, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-04-16T17:20:18.746 DEBUG:teuthology.orchestra.run.smithi045:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b1fecd38-fc11-11ee-bc90-c7b262605968@mds.a.service' 2024-04-16T17:20:18.871 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-16T17:20:18.872 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-04-16T17:20:18.872 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-04-16T17:20:18.872 DEBUG:teuthology.orchestra.run.smithi045:> sudo systemctl stop ceph-b1fecd38-fc11-11ee-bc90-c7b262605968@mds.d 2024-04-16T17:20:19.477 INFO:journalctl@ceph.mds.d.smithi045.stdout:Apr 16 17:20:19 smithi045 systemd[1]: Stopping Ceph mds.d for b1fecd38-fc11-11ee-bc90-c7b262605968... 2024-04-16T17:20:19.977 INFO:journalctl@ceph.mds.d.smithi045.stdout:Apr 16 17:20:19 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mds-d[72397]: 2024-04-16T17:20:19.700+0000 7f5f67c00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-16T17:20:19.977 INFO:journalctl@ceph.mds.d.smithi045.stdout:Apr 16 17:20:19 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mds-d[72397]: 2024-04-16T17:20:19.700+0000 7f5f67c00700 -1 mds.d *** got signal Terminated *** 2024-04-16T17:20:21.227 INFO:journalctl@ceph.mds.d.smithi045.stdout:Apr 16 17:20:20 smithi045 podman[104676]: 2024-04-16 17:20:20.949264498 +0000 UTC m=+1.582071307 container died 37490c52e388b887637a5e5c5e9e541c5b2348bcf21350cb0a0102eada69ad44 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mds-d, org.label-schema.build-date=20240409, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-04-16T17:20:21.695 INFO:journalctl@ceph.mds.d.smithi045.stdout:Apr 16 17:20:21 smithi045 podman[104676]: 2024-04-16 17:20:21.503220267 +0000 UTC m=+2.136027101 container cleanup 37490c52e388b887637a5e5c5e9e541c5b2348bcf21350cb0a0102eada69ad44 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mds-d, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240409, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, org.label-schema.schema-version=1.0) 2024-04-16T17:20:21.696 INFO:journalctl@ceph.mds.d.smithi045.stdout:Apr 16 17:20:21 smithi045 bash[104676]: ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mds-d 2024-04-16T17:20:21.976 INFO:journalctl@ceph.mds.d.smithi045.stdout:Apr 16 17:20:21 smithi045 podman[104690]: 2024-04-16 17:20:21.695682115 +0000 UTC m=+0.731358832 container remove 37490c52e388b887637a5e5c5e9e541c5b2348bcf21350cb0a0102eada69ad44 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mds-d, org.label-schema.build-date=20240409, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, ceph=True, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-04-16T17:20:22.244 DEBUG:teuthology.orchestra.run.smithi045:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b1fecd38-fc11-11ee-bc90-c7b262605968@mds.d.service' 2024-04-16T17:20:22.366 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-16T17:20:22.367 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-04-16T17:20:22.367 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-04-16T17:20:22.367 DEBUG:teuthology.orchestra.run.smithi045:> sudo systemctl stop ceph-b1fecd38-fc11-11ee-bc90-c7b262605968@mds.g 2024-04-16T17:20:22.977 INFO:journalctl@ceph.mds.g.smithi045.stdout:Apr 16 17:20:22 smithi045 systemd[1]: Stopping Ceph mds.g for b1fecd38-fc11-11ee-bc90-c7b262605968... 2024-04-16T17:20:23.477 INFO:journalctl@ceph.mds.g.smithi045.stdout:Apr 16 17:20:23 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mds-g[73003]: 2024-04-16T17:20:23.013+0000 7fe0a3c00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-16T17:20:23.477 INFO:journalctl@ceph.mds.g.smithi045.stdout:Apr 16 17:20:23 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mds-g[73003]: 2024-04-16T17:20:23.014+0000 7fe0a3c00700 -1 mds.g *** got signal Terminated *** 2024-04-16T17:20:27.360 INFO:journalctl@ceph.mds.g.smithi045.stdout:Apr 16 17:20:27 smithi045 podman[104797]: 2024-04-16 17:20:27.031715248 +0000 UTC m=+4.182063356 container died 0cf613ae6b13474562a8d1843026f3ca9a669b3b6a1ea1583c2c761e145a7b80 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mds-g, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.build-date=20240409, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-04-16T17:20:27.726 INFO:journalctl@ceph.mds.g.smithi045.stdout:Apr 16 17:20:27 smithi045 podman[104797]: 2024-04-16 17:20:27.472588823 +0000 UTC m=+4.622936924 container cleanup 0cf613ae6b13474562a8d1843026f3ca9a669b3b6a1ea1583c2c761e145a7b80 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mds-g, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240409) 2024-04-16T17:20:27.727 INFO:journalctl@ceph.mds.g.smithi045.stdout:Apr 16 17:20:27 smithi045 bash[104797]: ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mds-g 2024-04-16T17:20:28.226 INFO:journalctl@ceph.mds.g.smithi045.stdout:Apr 16 17:20:27 smithi045 podman[104811]: 2024-04-16 17:20:27.841267037 +0000 UTC m=+0.789297373 container remove 0cf613ae6b13474562a8d1843026f3ca9a669b3b6a1ea1583c2c761e145a7b80 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mds-g, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, GIT_CLEAN=True, org.label-schema.build-date=20240409, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-04-16T17:20:28.381 DEBUG:teuthology.orchestra.run.smithi045:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b1fecd38-fc11-11ee-bc90-c7b262605968@mds.g.service' 2024-04-16T17:20:28.507 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-16T17:20:28.507 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-04-16T17:20:28.507 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-04-16T17:20:28.507 DEBUG:teuthology.orchestra.run.smithi045:> sudo systemctl stop ceph-b1fecd38-fc11-11ee-bc90-c7b262605968@mds.j 2024-04-16T17:20:28.976 INFO:journalctl@ceph.mds.j.smithi045.stdout:Apr 16 17:20:28 smithi045 systemd[1]: Stopping Ceph mds.j for b1fecd38-fc11-11ee-bc90-c7b262605968... 2024-04-16T17:20:29.477 INFO:journalctl@ceph.mds.j.smithi045.stdout:Apr 16 17:20:29 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mds-j[72016]: 2024-04-16T17:20:29.140+0000 7f0687e00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-16T17:20:29.477 INFO:journalctl@ceph.mds.j.smithi045.stdout:Apr 16 17:20:29 smithi045 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mds-j[72016]: 2024-04-16T17:20:29.140+0000 7f0687e00700 -1 mds.j *** got signal Terminated *** 2024-04-16T17:20:33.199 INFO:journalctl@ceph.mds.j.smithi045.stdout:Apr 16 17:20:32 smithi045 podman[104918]: 2024-04-16 17:20:32.796898233 +0000 UTC m=+3.775242980 container died f9a5b8751be267ac3b34bf014287140eda4b857862f23800aff7207d6d31d163 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mds-j, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240409, org.label-schema.schema-version=1.0, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-04-16T17:20:33.477 INFO:journalctl@ceph.mds.j.smithi045.stdout:Apr 16 17:20:33 smithi045 podman[104918]: 2024-04-16 17:20:33.292170609 +0000 UTC m=+4.270515363 container cleanup f9a5b8751be267ac3b34bf014287140eda4b857862f23800aff7207d6d31d163 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mds-j, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240409, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-16T17:20:33.477 INFO:journalctl@ceph.mds.j.smithi045.stdout:Apr 16 17:20:33 smithi045 bash[104918]: ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mds-j 2024-04-16T17:20:33.856 INFO:journalctl@ceph.mds.j.smithi045.stdout:Apr 16 17:20:33 smithi045 podman[104932]: 2024-04-16 17:20:33.601616771 +0000 UTC m=+0.793941813 container remove f9a5b8751be267ac3b34bf014287140eda4b857862f23800aff7207d6d31d163 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mds-j, org.label-schema.build-date=20240409, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-04-16T17:20:33.875 DEBUG:teuthology.orchestra.run.smithi045:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b1fecd38-fc11-11ee-bc90-c7b262605968@mds.j.service' 2024-04-16T17:20:33.996 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-16T17:20:33.996 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-04-16T17:20:33.996 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-04-16T17:20:33.996 DEBUG:teuthology.orchestra.run.smithi104:> sudo systemctl stop ceph-b1fecd38-fc11-11ee-bc90-c7b262605968@mds.b 2024-04-16T17:20:34.309 INFO:journalctl@ceph.mds.b.smithi104.stdout:Apr 16 17:20:34 smithi104 systemd[1]: Stopping Ceph mds.b for b1fecd38-fc11-11ee-bc90-c7b262605968... 2024-04-16T17:20:34.574 INFO:journalctl@ceph.mds.b.smithi104.stdout:Apr 16 17:20:34 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mds-b[67798]: 2024-04-16T17:20:34.309+0000 7f9d7713d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-16T17:20:34.575 INFO:journalctl@ceph.mds.b.smithi104.stdout:Apr 16 17:20:34 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mds-b[67798]: 2024-04-16T17:20:34.309+0000 7f9d7713d700 -1 mds.b *** got signal Terminated *** 2024-04-16T17:20:37.868 INFO:journalctl@ceph.mds.b.smithi104.stdout:Apr 16 17:20:37 smithi104 podman[75338]: 2024-04-16 17:20:37.490525259 +0000 UTC m=+3.300183082 container stop 7f17776ddeedcf6bda64cedb064a9ae5afbea853a9470eff8ce2335d835a2b7a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mds-b, org.label-schema.build-date=20240409, ceph=True, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, GIT_CLEAN=True) 2024-04-16T17:20:37.868 INFO:journalctl@ceph.mds.b.smithi104.stdout:Apr 16 17:20:37 smithi104 podman[75338]: 2024-04-16 17:20:37.516796031 +0000 UTC m=+3.326453855 container died 7f17776ddeedcf6bda64cedb064a9ae5afbea853a9470eff8ce2335d835a2b7a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mds-b, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240409, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-04-16T17:20:38.178 INFO:journalctl@ceph.mds.b.smithi104.stdout:Apr 16 17:20:37 smithi104 podman[75338]: 2024-04-16 17:20:37.951162175 +0000 UTC m=+3.760820017 container cleanup 7f17776ddeedcf6bda64cedb064a9ae5afbea853a9470eff8ce2335d835a2b7a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mds-b, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240409, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-04-16T17:20:38.179 INFO:journalctl@ceph.mds.b.smithi104.stdout:Apr 16 17:20:37 smithi104 bash[75338]: ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mds-b 2024-04-16T17:20:38.422 DEBUG:teuthology.orchestra.run.smithi104:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b1fecd38-fc11-11ee-bc90-c7b262605968@mds.b.service' 2024-04-16T17:20:38.460 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-16T17:20:38.460 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-04-16T17:20:38.461 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-04-16T17:20:38.461 DEBUG:teuthology.orchestra.run.smithi104:> sudo systemctl stop ceph-b1fecd38-fc11-11ee-bc90-c7b262605968@mds.e 2024-04-16T17:20:38.779 INFO:journalctl@ceph.mds.e.smithi104.stdout:Apr 16 17:20:38 smithi104 systemd[1]: Stopping Ceph mds.e for b1fecd38-fc11-11ee-bc90-c7b262605968... 2024-04-16T17:20:39.176 INFO:journalctl@ceph.mds.e.smithi104.stdout:Apr 16 17:20:38 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mds-e[67167]: 2024-04-16T17:20:38.779+0000 7fb94de7e700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-16T17:20:39.177 INFO:journalctl@ceph.mds.e.smithi104.stdout:Apr 16 17:20:38 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mds-e[67167]: 2024-04-16T17:20:38.779+0000 7fb94de7e700 -1 mds.e *** got signal Terminated *** 2024-04-16T17:20:40.205 INFO:journalctl@ceph.mds.e.smithi104.stdout:Apr 16 17:20:39 smithi104 podman[75470]: 2024-04-16 17:20:39.955349529 +0000 UTC m=+1.273240713 container died 27e929370e371825dd4c5123c564baa959d6ca7f6cde48e83b6d5f6e03e2eb1a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mds-e, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240409, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True) 2024-04-16T17:20:40.463 INFO:journalctl@ceph.mds.e.smithi104.stdout:Apr 16 17:20:40 smithi104 podman[75470]: 2024-04-16 17:20:40.271287249 +0000 UTC m=+1.589178403 container cleanup 27e929370e371825dd4c5123c564baa959d6ca7f6cde48e83b6d5f6e03e2eb1a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mds-e, io.buildah.version=1.33.5, org.label-schema.build-date=20240409, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-04-16T17:20:40.463 INFO:journalctl@ceph.mds.e.smithi104.stdout:Apr 16 17:20:40 smithi104 bash[75470]: ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mds-e 2024-04-16T17:20:40.582 DEBUG:teuthology.orchestra.run.smithi104:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b1fecd38-fc11-11ee-bc90-c7b262605968@mds.e.service' 2024-04-16T17:20:40.617 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-16T17:20:40.617 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-04-16T17:20:40.618 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-04-16T17:20:40.618 DEBUG:teuthology.orchestra.run.smithi104:> sudo systemctl stop ceph-b1fecd38-fc11-11ee-bc90-c7b262605968@mds.h 2024-04-16T17:20:40.926 INFO:journalctl@ceph.mds.h.smithi104.stdout:Apr 16 17:20:40 smithi104 systemd[1]: Stopping Ceph mds.h for b1fecd38-fc11-11ee-bc90-c7b262605968... 2024-04-16T17:20:41.427 INFO:journalctl@ceph.mds.h.smithi104.stdout:Apr 16 17:20:40 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mds-h[67477]: 2024-04-16T17:20:40.974+0000 7f2d48532700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-16T17:20:41.427 INFO:journalctl@ceph.mds.h.smithi104.stdout:Apr 16 17:20:40 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mds-h[67477]: 2024-04-16T17:20:40.974+0000 7f2d48532700 -1 mds.h *** got signal Terminated *** 2024-04-16T17:20:45.383 INFO:journalctl@ceph.mds.h.smithi104.stdout:Apr 16 17:20:44 smithi104 podman[75590]: 2024-04-16 17:20:44.980233536 +0000 UTC m=+4.148161622 container died 664cad82bac4b0446c60c0ad8d361a02cf8d3a5f87e8c95af01034eaaf37cd2f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mds-h, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240409, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, GIT_BRANCH=HEAD) 2024-04-16T17:20:45.677 INFO:journalctl@ceph.mds.h.smithi104.stdout:Apr 16 17:20:45 smithi104 podman[75590]: 2024-04-16 17:20:45.491206807 +0000 UTC m=+4.659134894 container cleanup 664cad82bac4b0446c60c0ad8d361a02cf8d3a5f87e8c95af01034eaaf37cd2f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mds-h, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240409, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-16T17:20:45.677 INFO:journalctl@ceph.mds.h.smithi104.stdout:Apr 16 17:20:45 smithi104 bash[75590]: ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mds-h 2024-04-16T17:20:45.843 DEBUG:teuthology.orchestra.run.smithi104:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b1fecd38-fc11-11ee-bc90-c7b262605968@mds.h.service' 2024-04-16T17:20:45.877 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-16T17:20:45.877 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-04-16T17:20:45.878 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-04-16T17:20:45.878 DEBUG:teuthology.orchestra.run.smithi104:> sudo systemctl stop ceph-b1fecd38-fc11-11ee-bc90-c7b262605968@mds.k 2024-04-16T17:20:46.177 INFO:journalctl@ceph.mds.k.smithi104.stdout:Apr 16 17:20:45 smithi104 systemd[1]: Stopping Ceph mds.k for b1fecd38-fc11-11ee-bc90-c7b262605968... 2024-04-16T17:20:46.677 INFO:journalctl@ceph.mds.k.smithi104.stdout:Apr 16 17:20:46 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mds-k[66845]: 2024-04-16T17:20:46.211+0000 7f30491c4700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-16T17:20:46.677 INFO:journalctl@ceph.mds.k.smithi104.stdout:Apr 16 17:20:46 smithi104 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mds-k[66845]: 2024-04-16T17:20:46.211+0000 7f30491c4700 -1 mds.k *** got signal Terminated *** 2024-04-16T17:20:48.903 INFO:journalctl@ceph.mds.k.smithi104.stdout:Apr 16 17:20:48 smithi104 podman[75711]: 2024-04-16 17:20:48.630683425 +0000 UTC m=+2.536724624 container died c06d93d410ffc0ac248cc71d43f501f82405fade8d5b13d9ff1b8bbcf7678129 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mds-k, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240409, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD) 2024-04-16T17:20:49.177 INFO:journalctl@ceph.mds.k.smithi104.stdout:Apr 16 17:20:48 smithi104 podman[75711]: 2024-04-16 17:20:48.994604623 +0000 UTC m=+2.900645806 container cleanup c06d93d410ffc0ac248cc71d43f501f82405fade8d5b13d9ff1b8bbcf7678129 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mds-k, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240409, org.label-schema.vendor=CentOS) 2024-04-16T17:20:49.177 INFO:journalctl@ceph.mds.k.smithi104.stdout:Apr 16 17:20:48 smithi104 bash[75711]: ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mds-k 2024-04-16T17:20:49.363 DEBUG:teuthology.orchestra.run.smithi104:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b1fecd38-fc11-11ee-bc90-c7b262605968@mds.k.service' 2024-04-16T17:20:49.397 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-16T17:20:49.397 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-04-16T17:20:49.398 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-04-16T17:20:49.398 DEBUG:teuthology.orchestra.run.smithi117:> sudo systemctl stop ceph-b1fecd38-fc11-11ee-bc90-c7b262605968@mds.c 2024-04-16T17:20:49.717 INFO:journalctl@ceph.mds.c.smithi117.stdout:Apr 16 17:20:49 smithi117 systemd[1]: Stopping Ceph mds.c for b1fecd38-fc11-11ee-bc90-c7b262605968... 2024-04-16T17:20:50.054 INFO:journalctl@ceph.mds.c.smithi117.stdout:Apr 16 17:20:49 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mds-c[67668]: 2024-04-16T17:20:49.775+0000 7fbf74196700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-16T17:20:50.054 INFO:journalctl@ceph.mds.c.smithi117.stdout:Apr 16 17:20:49 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mds-c[67668]: 2024-04-16T17:20:49.775+0000 7fbf74196700 -1 mds.c *** got signal Terminated *** 2024-04-16T17:20:53.804 INFO:journalctl@ceph.mds.c.smithi117.stdout:Apr 16 17:20:53 smithi117 podman[75570]: 2024-04-16 17:20:53.507718309 +0000 UTC m=+3.914271117 container died 13fd9e10d5813e0ed2550e2603de9ddae30bddb663b77b3fee60d61ba0b890cb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mds-c, org.label-schema.license=GPLv2, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240409, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5) 2024-04-16T17:20:54.241 INFO:journalctl@ceph.mds.c.smithi117.stdout:Apr 16 17:20:53 smithi117 podman[75570]: 2024-04-16 17:20:53.982981817 +0000 UTC m=+4.389534612 container cleanup 13fd9e10d5813e0ed2550e2603de9ddae30bddb663b77b3fee60d61ba0b890cb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mds-c, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240409, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, maintainer=Guillaume Abrioux ) 2024-04-16T17:20:54.241 INFO:journalctl@ceph.mds.c.smithi117.stdout:Apr 16 17:20:53 smithi117 bash[75570]: ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mds-c 2024-04-16T17:20:54.408 DEBUG:teuthology.orchestra.run.smithi117:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b1fecd38-fc11-11ee-bc90-c7b262605968@mds.c.service' 2024-04-16T17:20:54.447 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-16T17:20:54.447 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-04-16T17:20:54.447 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-04-16T17:20:54.448 DEBUG:teuthology.orchestra.run.smithi117:> sudo systemctl stop ceph-b1fecd38-fc11-11ee-bc90-c7b262605968@mds.f 2024-04-16T17:20:54.769 INFO:journalctl@ceph.mds.f.smithi117.stdout:Apr 16 17:20:54 smithi117 systemd[1]: Stopping Ceph mds.f for b1fecd38-fc11-11ee-bc90-c7b262605968... 2024-04-16T17:20:55.054 INFO:journalctl@ceph.mds.f.smithi117.stdout:Apr 16 17:20:54 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mds-f[67363]: 2024-04-16T17:20:54.769+0000 7f47f9ff6700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-16T17:20:55.054 INFO:journalctl@ceph.mds.f.smithi117.stdout:Apr 16 17:20:54 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mds-f[67363]: 2024-04-16T17:20:54.769+0000 7f47f9ff6700 -1 mds.f *** got signal Terminated *** 2024-04-16T17:20:57.804 INFO:journalctl@ceph.mds.f.smithi117.stdout:Apr 16 17:20:57 smithi117 podman[75702]: 2024-04-16 17:20:57.506518795 +0000 UTC m=+2.842665371 container died a9eb34cce0d4060ed494edc24b958a7702fc5d52a29ed990a4a05231c7677f93 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mds-f, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240409, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-04-16T17:20:58.232 INFO:journalctl@ceph.mds.f.smithi117.stdout:Apr 16 17:20:57 smithi117 podman[75702]: 2024-04-16 17:20:57.82834302 +0000 UTC m=+3.164489645 container cleanup a9eb34cce0d4060ed494edc24b958a7702fc5d52a29ed990a4a05231c7677f93 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mds-f, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240409, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb) 2024-04-16T17:20:58.233 INFO:journalctl@ceph.mds.f.smithi117.stdout:Apr 16 17:20:57 smithi117 bash[75702]: ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mds-f 2024-04-16T17:20:58.233 INFO:journalctl@ceph.mds.f.smithi117.stdout:Apr 16 17:20:58 smithi117 podman[75716]: 2024-04-16 17:20:58.029338866 +0000 UTC m=+0.519680502 container remove a9eb34cce0d4060ed494edc24b958a7702fc5d52a29ed990a4a05231c7677f93 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mds-f, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240409, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS) 2024-04-16T17:20:58.251 DEBUG:teuthology.orchestra.run.smithi117:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b1fecd38-fc11-11ee-bc90-c7b262605968@mds.f.service' 2024-04-16T17:20:58.287 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-16T17:20:58.287 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-04-16T17:20:58.288 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-04-16T17:20:58.288 DEBUG:teuthology.orchestra.run.smithi117:> sudo systemctl stop ceph-b1fecd38-fc11-11ee-bc90-c7b262605968@mds.i 2024-04-16T17:20:58.555 INFO:journalctl@ceph.mds.i.smithi117.stdout:Apr 16 17:20:58 smithi117 systemd[1]: Stopping Ceph mds.i for b1fecd38-fc11-11ee-bc90-c7b262605968... 2024-04-16T17:20:59.054 INFO:journalctl@ceph.mds.i.smithi117.stdout:Apr 16 17:20:58 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mds-i[67974]: 2024-04-16T17:20:58.606+0000 7f9602ebe700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-16T17:20:59.054 INFO:journalctl@ceph.mds.i.smithi117.stdout:Apr 16 17:20:58 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mds-i[67974]: 2024-04-16T17:20:58.606+0000 7f9602ebe700 -1 mds.i *** got signal Terminated *** 2024-04-16T17:21:01.773 INFO:journalctl@ceph.mds.i.smithi117.stdout:Apr 16 17:21:01 smithi117 podman[75823]: 2024-04-16 17:21:01.495790362 +0000 UTC m=+2.994093933 container died 02de05e9e9c27b0cde2597955d7d8807e919a9894e529a77f59804366d42bca2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mds-i, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, io.buildah.version=1.33.5, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240409, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-04-16T17:21:02.054 INFO:journalctl@ceph.mds.i.smithi117.stdout:Apr 16 17:21:01 smithi117 podman[75823]: 2024-04-16 17:21:01.865164835 +0000 UTC m=+3.363468400 container cleanup 02de05e9e9c27b0cde2597955d7d8807e919a9894e529a77f59804366d42bca2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mds-i, org.label-schema.build-date=20240409, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-04-16T17:21:02.054 INFO:journalctl@ceph.mds.i.smithi117.stdout:Apr 16 17:21:01 smithi117 bash[75823]: ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mds-i 2024-04-16T17:21:02.287 DEBUG:teuthology.orchestra.run.smithi117:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b1fecd38-fc11-11ee-bc90-c7b262605968@mds.i.service' 2024-04-16T17:21:02.322 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-16T17:21:02.322 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-04-16T17:21:02.322 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-04-16T17:21:02.322 DEBUG:teuthology.orchestra.run.smithi117:> sudo systemctl stop ceph-b1fecd38-fc11-11ee-bc90-c7b262605968@mds.l 2024-04-16T17:21:02.804 INFO:journalctl@ceph.mds.l.smithi117.stdout:Apr 16 17:21:02 smithi117 systemd[1]: Stopping Ceph mds.l for b1fecd38-fc11-11ee-bc90-c7b262605968... 2024-04-16T17:21:02.804 INFO:journalctl@ceph.mds.l.smithi117.stdout:Apr 16 17:21:02 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mds-l[67052]: 2024-04-16T17:21:02.626+0000 7feb9dc44700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-16T17:21:02.804 INFO:journalctl@ceph.mds.l.smithi117.stdout:Apr 16 17:21:02 smithi117 ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mds-l[67052]: 2024-04-16T17:21:02.626+0000 7feb9dc44700 -1 mds.l *** got signal Terminated *** 2024-04-16T17:21:05.919 INFO:journalctl@ceph.mds.l.smithi117.stdout:Apr 16 17:21:05 smithi117 podman[75947]: 2024-04-16 17:21:05.661250562 +0000 UTC m=+3.123922094 container died 888d9071f1761f69b31f6e2d66fdaa707f15a1c9447b8bcfd2af62059fa3c184 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mds-l, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240409, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, org.label-schema.license=GPLv2, io.buildah.version=1.33.5) 2024-04-16T17:21:06.277 INFO:journalctl@ceph.mds.l.smithi117.stdout:Apr 16 17:21:06 smithi117 podman[75947]: 2024-04-16 17:21:06.018978027 +0000 UTC m=+3.481649541 container cleanup 888d9071f1761f69b31f6e2d66fdaa707f15a1c9447b8bcfd2af62059fa3c184 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:59f471e3549b85d55eaf023b1ca9fa19dc9cc95510b852c5bac234ebe142daa2, name=ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mds-l, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-9b42bfb, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240409, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-04-16T17:21:06.277 INFO:journalctl@ceph.mds.l.smithi117.stdout:Apr 16 17:21:06 smithi117 bash[75947]: ceph-b1fecd38-fc11-11ee-bc90-c7b262605968-mds-l 2024-04-16T17:21:06.425 DEBUG:teuthology.orchestra.run.smithi117:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b1fecd38-fc11-11ee-bc90-c7b262605968@mds.l.service' 2024-04-16T17:21:06.459 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-16T17:21:06.459 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-04-16T17:21:06.460 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid b1fecd38-fc11-11ee-bc90-c7b262605968 --force --keep-logs 2024-04-16T17:21:06.798 INFO:teuthology.orchestra.run.smithi045.stdout:Deleting cluster with fsid: b1fecd38-fc11-11ee-bc90-c7b262605968 2024-04-16T17:21:12.171 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid b1fecd38-fc11-11ee-bc90-c7b262605968 --force --keep-logs 2024-04-16T17:21:12.364 INFO:teuthology.orchestra.run.smithi104.stdout:Deleting cluster with fsid: b1fecd38-fc11-11ee-bc90-c7b262605968 2024-04-16T17:21:14.940 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid b1fecd38-fc11-11ee-bc90-c7b262605968 --force --keep-logs 2024-04-16T17:21:15.129 INFO:teuthology.orchestra.run.smithi117.stdout:Deleting cluster with fsid: b1fecd38-fc11-11ee-bc90-c7b262605968 2024-04-16T17:21:17.553 DEBUG:teuthology.orchestra.run.smithi045:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-16T17:21:17.661 DEBUG:teuthology.orchestra.run.smithi104:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-16T17:21:17.696 DEBUG:teuthology.orchestra.run.smithi117:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-16T17:21:17.725 INFO:tasks.cephadm:Archiving crash dumps... 2024-04-16T17:21:17.727 DEBUG:teuthology.misc:Transferring archived files from smithi045:/var/lib/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/crash to /home/teuthworker/archive/mchangir-2024-04-16_16:32:10-fs-wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-distro-default-smithi/7658439/remote/smithi045/crash 2024-04-16T17:21:17.727 DEBUG:teuthology.orchestra.run.smithi045:> sudo tar c -f - -C /var/lib/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/crash -- . 2024-04-16T17:21:17.845 INFO:teuthology.orchestra.run.smithi045.stderr:tar: /var/lib/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/crash: Cannot open: No such file or directory 2024-04-16T17:21:17.846 INFO:teuthology.orchestra.run.smithi045.stderr:tar: Error is not recoverable: exiting now 2024-04-16T17:21:17.851 DEBUG:teuthology.misc:Transferring archived files from smithi104:/var/lib/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/crash to /home/teuthworker/archive/mchangir-2024-04-16_16:32:10-fs-wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-distro-default-smithi/7658439/remote/smithi104/crash 2024-04-16T17:21:17.852 DEBUG:teuthology.orchestra.run.smithi104:> sudo tar c -f - -C /var/lib/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/crash -- . 2024-04-16T17:21:17.909 INFO:teuthology.orchestra.run.smithi104.stderr:tar: /var/lib/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/crash: Cannot open: No such file or directory 2024-04-16T17:21:17.909 INFO:teuthology.orchestra.run.smithi104.stderr:tar: Error is not recoverable: exiting now 2024-04-16T17:21:17.911 DEBUG:teuthology.misc:Transferring archived files from smithi117:/var/lib/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/crash to /home/teuthworker/archive/mchangir-2024-04-16_16:32:10-fs-wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-distro-default-smithi/7658439/remote/smithi117/crash 2024-04-16T17:21:17.912 DEBUG:teuthology.orchestra.run.smithi117:> sudo tar c -f - -C /var/lib/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/crash -- . 2024-04-16T17:21:17.948 INFO:teuthology.orchestra.run.smithi117.stderr:tar: /var/lib/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/crash: Cannot open: No such file or directory 2024-04-16T17:21:17.949 INFO:teuthology.orchestra.run.smithi117.stderr:tar: Error is not recoverable: exiting now 2024-04-16T17:21:17.950 INFO:tasks.cephadm:Checking cluster log for badness... 2024-04-16T17:21:17.950 DEBUG:teuthology.orchestra.run.smithi045:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-04-16T17:21:18.066 INFO:teuthology.orchestra.run.smithi045.stdout:2024-04-16T17:09:44.194555+0000 mds.b (mds.0) 34 : cluster [WRN] Scrub error on inode 0x10000000225 (/volumes/qa/sv_1/584ee6b0-43a5-48e1-b843-9914b8e65254/client.0/tmp/clients/client0/~dmtmp/PM) see mds.b log and `damage ls` output for details 2024-04-16T17:21:18.068 WARNING:tasks.cephadm:Found errors (ERR|WRN|SEC) in cluster log 2024-04-16T17:21:18.068 INFO:tasks.cephadm:Compressing logs... 2024-04-16T17:21:18.068 DEBUG:teuthology.orchestra.run.smithi045:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-16T17:21:18.072 DEBUG:teuthology.orchestra.run.smithi104:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-16T17:21:18.075 DEBUG:teuthology.orchestra.run.smithi117:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-16T17:21:18.102 INFO:teuthology.orchestra.run.smithi117.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-16T17:21:18.102 INFO:teuthology.orchestra.run.smithi117.stderr:: No such file or directory 2024-04-16T17:21:18.102 INFO:teuthology.orchestra.run.smithi117.stderr:gzip -5 --verbose -- /var/log/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/ceph-mds.l.log 2024-04-16T17:21:18.102 INFO:teuthology.orchestra.run.smithi117.stderr:gzip -5 --verbose -- /var/log/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/ceph.audit.log 2024-04-16T17:21:18.102 INFO:teuthology.orchestra.run.smithi117.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/ceph-mds.i.log 2024-04-16T17:21:18.102 INFO:teuthology.orchestra.run.smithi117.stderr:/var/log/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/ceph-mds.l.log: /var/log/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/ceph-osd.11.log 2024-04-16T17:21:18.110 INFO:teuthology.orchestra.run.smithi104.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-16T17:21:18.110 INFO:teuthology.orchestra.run.smithi104.stderr:: No such file or directory 2024-04-16T17:21:18.110 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/ceph-osd.10.log 2024-04-16T17:21:18.110 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/ceph.audit.log 2024-04-16T17:21:18.110 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/ceph-mds.b.log 2024-04-16T17:21:18.111 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/ceph-osd.4.log 2024-04-16T17:21:18.111 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/ceph-mds.b.log: 92.1% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-16T17:21:18.111 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/ceph-osd.1.log 2024-04-16T17:21:18.111 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/ceph-mds.k.log 2024-04-16T17:21:18.111 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/ceph-osd.4.log: /var/log/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/ceph-mds.h.log 2024-04-16T17:21:18.111 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/ceph-mds.k.log: gzip -5 --verbose -- /var/log/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/ceph-osd.7.log 2024-04-16T17:21:18.111 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/ceph-mds.e.log 2024-04-16T17:21:18.111 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/ceph-osd.7.log: /var/log/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/ceph-mds.h.log: 90.2% -- replaced with /var/log/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/ceph.audit.log.gz 2024-04-16T17:21:18.112 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/ceph-mgr.y.log 2024-04-16T17:21:18.114 INFO:teuthology.orchestra.run.smithi117.stderr:gzip -5 --verbose -- /var/log/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/ceph-mgr.z.log 2024-04-16T17:21:18.114 INFO:teuthology.orchestra.run.smithi117.stderr:/var/log/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/ceph-mds.i.log: /var/log/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/ceph-osd.8.log 2024-04-16T17:21:18.114 INFO:teuthology.orchestra.run.smithi117.stderr: 92.4% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-16T17:21:18.114 INFO:teuthology.orchestra.run.smithi117.stderr:/var/log/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/ceph-mgr.z.log: gzip -5 --verbose -- /var/log/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/ceph-osd.5.log 2024-04-16T17:21:18.114 INFO:teuthology.orchestra.run.smithi117.stderr:/var/log/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/ceph-mds.f.log 2024-04-16T17:21:18.114 INFO:teuthology.orchestra.run.smithi117.stderr:/var/log/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/ceph.cephadm.log 2024-04-16T17:21:18.115 INFO:teuthology.orchestra.run.smithi117.stderr:/var/log/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/ceph-mds.f.log: 90.2% -- replaced with /var/log/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/ceph.audit.log.gz 2024-04-16T17:21:18.115 INFO:teuthology.orchestra.run.smithi117.stderr:gzip -5 --verbose -- /var/log/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/ceph.log 2024-04-16T17:21:18.115 INFO:teuthology.orchestra.run.smithi117.stderr:/var/log/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/ceph.cephadm.log: 83.5% -- replaced with /var/log/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/ceph.cephadm.log.gz 2024-04-16T17:21:18.115 INFO:teuthology.orchestra.run.smithi117.stderr:gzip -5 --verbose -- /var/log/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/ceph-osd.2.log 2024-04-16T17:21:18.115 INFO:teuthology.orchestra.run.smithi117.stderr:/var/log/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/ceph.log: 94.6% -- replaced with /var/log/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/ceph-mds.l.log.gz 2024-04-16T17:21:18.115 INFO:teuthology.orchestra.run.smithi117.stderr:gzip -5 --verbose -- /var/log/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/ceph-mds.c.log 2024-04-16T17:21:18.115 INFO:teuthology.orchestra.run.smithi117.stderr:/var/log/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/ceph-osd.2.log: 87.0% -- replaced with /var/log/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/ceph.log.gz 2024-04-16T17:21:18.115 INFO:teuthology.orchestra.run.smithi117.stderr:gzip -5 --verbose -- /var/log/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/ceph-volume.log 2024-04-16T17:21:18.115 INFO:teuthology.orchestra.run.smithi117.stderr:/var/log/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/ceph-mds.c.log: 92.0% -- replaced with /var/log/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/ceph-mgr.z.log.gz 2024-04-16T17:21:18.116 INFO:teuthology.orchestra.run.smithi117.stderr:gzip -5 --verbose -- /var/log/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/ceph-mon.c.log 2024-04-16T17:21:18.117 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/ceph-mds.e.log: 94.4% -- replaced with /var/log/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/ceph-mds.k.log.gz 2024-04-16T17:21:18.118 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/ceph.cephadm.log 2024-04-16T17:21:18.118 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/ceph-mgr.y.log: 94.4% -- replaced with /var/log/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/ceph-mds.e.log.gz 2024-04-16T17:21:18.118 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/ceph.log 2024-04-16T17:21:18.118 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/ceph.cephadm.log: 83.5% -- replaced with /var/log/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/ceph.cephadm.log.gz 2024-04-16T17:21:18.118 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/ceph-volume.log 2024-04-16T17:21:18.119 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/ceph.log: 87.0% -- replaced with /var/log/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/ceph.log.gz 2024-04-16T17:21:18.119 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/ceph-mon.b.log 2024-04-16T17:21:18.122 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/ceph-volume.log: 91.9% -- replaced with /var/log/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/ceph-mgr.y.log.gz 2024-04-16T17:21:18.127 INFO:teuthology.orchestra.run.smithi117.stderr:/var/log/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/ceph-volume.log: /var/log/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/ceph-mon.c.log: 90.9% -- replaced with /var/log/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/ceph-volume.log.gz 2024-04-16T17:21:18.133 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/ceph-mon.b.log: 90.9% -- replaced with /var/log/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/ceph-volume.log.gz 2024-04-16T17:21:18.138 INFO:teuthology.orchestra.run.smithi045.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-04-16T17:21:18.141 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-16T17:21:18.142 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.log 2024-04-16T17:21:18.143 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/ceph-osd.6.log 2024-04-16T17:21:18.144 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.log: gzip -5 --verbose -- /var/log/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/ceph-mon.a.log 2024-04-16T17:21:18.145 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/ceph.audit.log 2024-04-16T17:21:18.146 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/ceph-mon.a.log: 92.3% -- replaced with /var/log/ceph/ceph-client.admin.log.gz 2024-04-16T17:21:18.146 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/ceph-mds.g.log 2024-04-16T17:21:18.148 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/ceph-mds.j.log 2024-04-16T17:21:18.149 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/ceph-osd.3.log 2024-04-16T17:21:18.151 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/ceph-mds.j.log: gzip -5 --verbose -- /var/log/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/ceph-mgr.x.log 2024-04-16T17:21:18.153 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/ceph-osd.3.log: /var/log/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/ceph-mgr.x.log: 90.1% -- replaced with /var/log/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/ceph.audit.log.gz 2024-04-16T17:21:18.154 INFO:teuthology.orchestra.run.smithi045.stderr: 93.2% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-16T17:21:18.154 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/ceph-osd.0.log 2024-04-16T17:21:18.159 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/ceph.cephadm.log 2024-04-16T17:21:18.159 INFO:teuthology.orchestra.run.smithi045.stderr: 94.5% -- replaced with /var/log/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/ceph-mds.g.log.gz 2024-04-16T17:21:18.159 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/ceph.log 2024-04-16T17:21:18.159 INFO:teuthology.orchestra.run.smithi045.stderr: 94.5% -- replaced with /var/log/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/ceph-mds.j.log.gz 2024-04-16T17:21:18.160 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/ceph-volume.log 2024-04-16T17:21:18.160 INFO:teuthology.orchestra.run.smithi045.stderr: 83.9% -- replaced with /var/log/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/ceph.cephadm.log.gz 2024-04-16T17:21:18.161 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/ceph-osd.9.log 2024-04-16T17:21:18.163 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/ceph-volume.log: /var/log/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/ceph-osd.9.log: gzip -5 --verbose -- /var/log/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/ceph-mds.d.log 2024-04-16T17:21:18.164 INFO:teuthology.orchestra.run.smithi045.stderr: 87.0% -- replaced with /var/log/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/ceph.log.gz 2024-04-16T17:21:18.170 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/ceph-mds.a.log 2024-04-16T17:21:18.177 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/ceph-mds.a.log: 90.9% -- replaced with /var/log/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/ceph-volume.log.gz 2024-04-16T17:21:18.182 INFO:teuthology.orchestra.run.smithi045.stderr: 94.3% -- replaced with /var/log/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/ceph-mds.d.log.gz 2024-04-16T17:21:18.555 INFO:teuthology.orchestra.run.smithi117.stderr: 94.8% -- replaced with /var/log/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/ceph-mds.c.log.gz 2024-04-16T17:21:18.601 INFO:teuthology.orchestra.run.smithi045.stderr: 90.1% -- replaced with /var/log/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/ceph-mgr.x.log.gz 2024-04-16T17:21:18.697 INFO:teuthology.orchestra.run.smithi117.stderr: 91.7% -- replaced with /var/log/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/ceph-mon.c.log.gz 2024-04-16T17:21:18.697 INFO:teuthology.orchestra.run.smithi117.stderr: 95.1% -- replaced with /var/log/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/ceph-mds.f.log.gz 2024-04-16T17:21:18.864 INFO:teuthology.orchestra.run.smithi104.stderr: 91.7% -- replaced with /var/log/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/ceph-mon.b.log.gz 2024-04-16T17:21:18.965 INFO:teuthology.orchestra.run.smithi104.stderr: 94.9% -- replaced with /var/log/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/ceph-mds.h.log.gz 2024-04-16T17:21:19.294 INFO:teuthology.orchestra.run.smithi045.stderr: 90.4% -- replaced with /var/log/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/ceph-mon.a.log.gz 2024-04-16T17:21:21.523 INFO:teuthology.orchestra.run.smithi117.stderr: 91.7% -- replaced with /var/log/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/ceph-osd.11.log.gz 2024-04-16T17:21:22.374 INFO:teuthology.orchestra.run.smithi045.stderr: 92.1% -- replaced with /var/log/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/ceph-osd.0.log.gz 2024-04-16T17:21:22.453 INFO:teuthology.orchestra.run.smithi117.stderr: 91.7% -- replaced with /var/log/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/ceph-osd.8.log.gz 2024-04-16T17:21:22.516 INFO:teuthology.orchestra.run.smithi104.stderr: 92.0% -- replaced with /var/log/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/ceph-osd.1.log.gz 2024-04-16T17:21:22.841 INFO:teuthology.orchestra.run.smithi117.stderr: 92.0% -- replaced with /var/log/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/ceph-osd.2.log.gz 2024-04-16T17:21:23.425 INFO:teuthology.orchestra.run.smithi045.stderr: 91.8% -- replaced with /var/log/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/ceph-osd.9.log.gz 2024-04-16T17:21:23.715 INFO:teuthology.orchestra.run.smithi045.stderr: 93.8% -- replaced with /var/log/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/ceph-mds.a.log.gz 2024-04-16T17:21:23.822 INFO:teuthology.orchestra.run.smithi104.stderr: 91.9% -- replaced with /var/log/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/ceph-osd.10.log.gz 2024-04-16T17:21:23.893 INFO:teuthology.orchestra.run.smithi045.stderr: 91.9% -- replaced with /var/log/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/ceph-osd.6.log.gz 2024-04-16T17:21:24.334 INFO:teuthology.orchestra.run.smithi104.stderr: 91.6% -- replaced with /var/log/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/ceph-osd.4.log.gz 2024-04-16T17:21:24.495 INFO:teuthology.orchestra.run.smithi045.stderr: 91.7% -- replaced with /var/log/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/ceph-osd.3.log.gz 2024-04-16T17:21:24.498 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-04-16T17:21:24.498 INFO:teuthology.orchestra.run.smithi045.stderr:real 0m6.394s 2024-04-16T17:21:24.499 INFO:teuthology.orchestra.run.smithi045.stderr:user 0m26.810s 2024-04-16T17:21:24.499 INFO:teuthology.orchestra.run.smithi045.stderr:sys 0m1.992s 2024-04-16T17:21:24.654 INFO:teuthology.orchestra.run.smithi117.stderr: 91.4% -- replaced with /var/log/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/ceph-osd.5.log.gz 2024-04-16T17:21:24.928 INFO:teuthology.orchestra.run.smithi104.stderr: 91.6% -- replaced with /var/log/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/ceph-osd.7.log.gz 2024-04-16T17:21:26.589 INFO:teuthology.orchestra.run.smithi117.stderr: 93.9% -- replaced with /var/log/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/ceph-mds.i.log.gz 2024-04-16T17:21:26.590 INFO:teuthology.orchestra.run.smithi117.stderr: 2024-04-16T17:21:26.591 INFO:teuthology.orchestra.run.smithi117.stderr:real 0m8.502s 2024-04-16T17:21:26.591 INFO:teuthology.orchestra.run.smithi117.stderr:user 0m28.267s 2024-04-16T17:21:26.591 INFO:teuthology.orchestra.run.smithi117.stderr:sys 0m0.892s 2024-04-16T17:21:31.031 INFO:teuthology.orchestra.run.smithi104.stderr: 93.5% -- replaced with /var/log/ceph/b1fecd38-fc11-11ee-bc90-c7b262605968/ceph-mds.b.log.gz 2024-04-16T17:21:31.033 INFO:teuthology.orchestra.run.smithi104.stderr: 2024-04-16T17:21:31.033 INFO:teuthology.orchestra.run.smithi104.stderr:real 0m12.944s 2024-04-16T17:21:31.033 INFO:teuthology.orchestra.run.smithi104.stderr:user 0m36.604s 2024-04-16T17:21:31.033 INFO:teuthology.orchestra.run.smithi104.stderr:sys 0m1.109s 2024-04-16T17:21:31.034 INFO:tasks.cephadm:Archiving logs... 2024-04-16T17:21:31.034 DEBUG:teuthology.misc:Transferring archived files from smithi045:/var/log/ceph to /home/teuthworker/archive/mchangir-2024-04-16_16:32:10-fs-wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-distro-default-smithi/7658439/remote/smithi045/log 2024-04-16T17:21:31.035 DEBUG:teuthology.orchestra.run.smithi045:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-16T17:21:34.604 DEBUG:teuthology.misc:Transferring archived files from smithi104:/var/log/ceph to /home/teuthworker/archive/mchangir-2024-04-16_16:32:10-fs-wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-distro-default-smithi/7658439/remote/smithi104/log 2024-04-16T17:21:34.605 DEBUG:teuthology.orchestra.run.smithi104:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-16T17:21:39.540 DEBUG:teuthology.misc:Transferring archived files from smithi117:/var/log/ceph to /home/teuthworker/archive/mchangir-2024-04-16_16:32:10-fs-wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-distro-default-smithi/7658439/remote/smithi117/log 2024-04-16T17:21:39.541 DEBUG:teuthology.orchestra.run.smithi117:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-16T17:21:43.516 INFO:tasks.cephadm:Removing cluster... 2024-04-16T17:21:43.612 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid b1fecd38-fc11-11ee-bc90-c7b262605968 --force 2024-04-16T17:21:43.954 INFO:teuthology.orchestra.run.smithi045.stdout:Deleting cluster with fsid: b1fecd38-fc11-11ee-bc90-c7b262605968 2024-04-16T17:21:44.689 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid b1fecd38-fc11-11ee-bc90-c7b262605968 --force 2024-04-16T17:21:44.883 INFO:teuthology.orchestra.run.smithi104.stdout:Deleting cluster with fsid: b1fecd38-fc11-11ee-bc90-c7b262605968 2024-04-16T17:21:45.180 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid b1fecd38-fc11-11ee-bc90-c7b262605968 --force 2024-04-16T17:21:45.373 INFO:teuthology.orchestra.run.smithi117.stdout:Deleting cluster with fsid: b1fecd38-fc11-11ee-bc90-c7b262605968 2024-04-16T17:21:45.662 INFO:tasks.cephadm:Removing cephadm ... 2024-04-16T17:21:45.663 DEBUG:teuthology.orchestra.run.smithi045:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-16T17:21:45.725 DEBUG:teuthology.orchestra.run.smithi104:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-16T17:21:45.740 DEBUG:teuthology.orchestra.run.smithi117:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-16T17:21:45.755 INFO:tasks.cephadm:Teardown complete 2024-04-16T17:21:45.756 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-04-16T17:21:45.768 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-04-16T17:21:45.768 DEBUG:teuthology.orchestra.run.smithi045:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-16T17:21:45.776 DEBUG:teuthology.orchestra.run.smithi104:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-16T17:21:45.782 DEBUG:teuthology.orchestra.run.smithi117:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-16T17:21:45.951 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-04-16T17:21:45.962 INFO:teuthology.task.clock:Checking final clock skew... 2024-04-16T17:21:45.962 DEBUG:teuthology.orchestra.run.smithi045:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-16T17:21:45.970 DEBUG:teuthology.orchestra.run.smithi104:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-16T17:21:45.972 DEBUG:teuthology.orchestra.run.smithi117:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-16T17:21:45.983 INFO:teuthology.orchestra.run.smithi104.stderr:bash: line 1: ntpq: command not found 2024-04-16T17:21:45.986 INFO:teuthology.orchestra.run.smithi117.stderr:bash: line 1: ntpq: command not found 2024-04-16T17:21:45.987 INFO:teuthology.orchestra.run.smithi104.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-16T17:21:45.988 INFO:teuthology.orchestra.run.smithi104.stdout:=============================================================================== 2024-04-16T17:21:45.988 INFO:teuthology.orchestra.run.smithi104.stdout:^* hv01.front.sepia.ceph.com 3 6 377 38 +1090us[+1073us] +/- 54ms 2024-04-16T17:21:45.988 INFO:teuthology.orchestra.run.smithi104.stdout:^+ hv02.front.sepia.ceph.com 3 7 377 109 -2548us[-2566us] +/- 75ms 2024-04-16T17:21:45.988 INFO:teuthology.orchestra.run.smithi104.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 43 +1579us[+1563us] +/- 70ms 2024-04-16T17:21:45.988 INFO:teuthology.orchestra.run.smithi104.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-16T17:21:45.990 INFO:teuthology.orchestra.run.smithi117.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-16T17:21:45.991 INFO:teuthology.orchestra.run.smithi117.stdout:=============================================================================== 2024-04-16T17:21:45.991 INFO:teuthology.orchestra.run.smithi117.stdout:^* hv01.front.sepia.ceph.com 3 6 377 38 +981us[ +952us] +/- 54ms 2024-04-16T17:21:45.991 INFO:teuthology.orchestra.run.smithi117.stdout:^+ hv02.front.sepia.ceph.com 3 7 377 40 -2642us[-2671us] +/- 76ms 2024-04-16T17:21:45.991 INFO:teuthology.orchestra.run.smithi117.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 39 +1477us[+1448us] +/- 70ms 2024-04-16T17:21:45.991 INFO:teuthology.orchestra.run.smithi117.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-16T17:21:46.018 INFO:teuthology.orchestra.run.smithi045.stderr:bash: line 1: ntpq: command not found 2024-04-16T17:21:46.028 INFO:teuthology.orchestra.run.smithi045.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-16T17:21:46.028 INFO:teuthology.orchestra.run.smithi045.stdout:=============================================================================== 2024-04-16T17:21:46.028 INFO:teuthology.orchestra.run.smithi045.stdout:^* hv01.front.sepia.ceph.com 3 6 377 41 +958us[ +940us] +/- 54ms 2024-04-16T17:21:46.028 INFO:teuthology.orchestra.run.smithi045.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 39 -2706us[-2706us] +/- 76ms 2024-04-16T17:21:46.029 INFO:teuthology.orchestra.run.smithi045.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 37 +1448us[+1448us] +/- 70ms 2024-04-16T17:21:46.029 INFO:teuthology.orchestra.run.smithi045.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-16T17:21:46.030 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-04-16T17:21:46.043 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-04-16T17:21:46.043 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-04-16T17:21:46.106 DEBUG:teuthology.orchestra.run.smithi045:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-16T17:21:46.362 DEBUG:teuthology.orchestra.run.smithi104:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-16T17:21:46.434 DEBUG:teuthology.orchestra.run.smithi117:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-16T17:21:46.510 DEBUG:teuthology.orchestra.run.smithi045:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-16T17:21:46.622 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-16T17:21:46.622 DEBUG:teuthology.orchestra.run.smithi104:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-16T17:21:46.652 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-16T17:21:46.653 DEBUG:teuthology.orchestra.run.smithi117:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-16T17:21:46.684 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-16T17:21:46.685 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-04-16T17:21:46.700 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-04-16T17:21:46.717 INFO:teuthology.task.internal:Duration was 2294.906812 seconds 2024-04-16T17:21:46.718 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-04-16T17:21:46.733 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-04-16T17:21:46.733 DEBUG:teuthology.orchestra.run.smithi045:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-16T17:21:46.741 DEBUG:teuthology.orchestra.run.smithi104:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-16T17:21:46.744 DEBUG:teuthology.orchestra.run.smithi117:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-16T17:21:46.784 INFO:teuthology.orchestra.run.smithi104.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-16T17:21:46.788 INFO:teuthology.orchestra.run.smithi117.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-16T17:21:46.914 INFO:teuthology.orchestra.run.smithi045.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-16T17:21:47.373 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-04-16T17:21:47.374 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi045.front.sepia.ceph.com 2024-04-16T17:21:47.374 DEBUG:teuthology.orchestra.run.smithi045:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-16T17:21:47.434 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi104.front.sepia.ceph.com 2024-04-16T17:21:47.434 DEBUG:teuthology.orchestra.run.smithi104:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-16T17:21:47.468 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi117.front.sepia.ceph.com 2024-04-16T17:21:47.468 DEBUG:teuthology.orchestra.run.smithi117:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-16T17:21:47.505 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-04-16T17:21:47.505 DEBUG:teuthology.orchestra.run.smithi045:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-16T17:21:47.509 DEBUG:teuthology.orchestra.run.smithi104:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-16T17:21:47.511 DEBUG:teuthology.orchestra.run.smithi117:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-16T17:21:47.577 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-04-16T17:21:47.578 DEBUG:teuthology.orchestra.run.smithi045:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-16T17:21:47.582 DEBUG:teuthology.orchestra.run.smithi104:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-16T17:21:47.584 DEBUG:teuthology.orchestra.run.smithi117:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-16T17:22:02.057 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-04-16T17:22:02.069 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-04-16T17:22:02.069 DEBUG:teuthology.orchestra.run.smithi045:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-16T17:22:02.156 DEBUG:teuthology.orchestra.run.smithi104:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-16T17:22:02.188 DEBUG:teuthology.orchestra.run.smithi117:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-16T17:22:02.220 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-04-16T17:22:02.232 DEBUG:teuthology.orchestra.run.smithi045:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-16T17:22:02.240 DEBUG:teuthology.orchestra.run.smithi104:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-16T17:22:02.242 DEBUG:teuthology.orchestra.run.smithi117:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-16T17:22:02.268 INFO:teuthology.orchestra.run.smithi104.stdout:kernel.core_pattern = core 2024-04-16T17:22:02.289 INFO:teuthology.orchestra.run.smithi117.stdout:kernel.core_pattern = core 2024-04-16T17:22:02.343 INFO:teuthology.orchestra.run.smithi045.stdout:kernel.core_pattern = core 2024-04-16T17:22:02.399 DEBUG:teuthology.orchestra.run.smithi045:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-16T17:22:02.450 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-16T17:22:02.451 DEBUG:teuthology.orchestra.run.smithi104:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-16T17:22:02.467 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-16T17:22:02.467 DEBUG:teuthology.orchestra.run.smithi117:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-16T17:22:02.483 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-16T17:22:02.483 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-04-16T17:22:02.495 INFO:teuthology.task.internal:Transferring archived files... 2024-04-16T17:22:02.496 DEBUG:teuthology.misc:Transferring archived files from smithi045:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/mchangir-2024-04-16_16:32:10-fs-wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-distro-default-smithi/7658439/remote/smithi045 2024-04-16T17:22:02.496 DEBUG:teuthology.orchestra.run.smithi045:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-16T17:22:02.897 DEBUG:teuthology.misc:Transferring archived files from smithi104:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/mchangir-2024-04-16_16:32:10-fs-wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-distro-default-smithi/7658439/remote/smithi104 2024-04-16T17:22:02.898 DEBUG:teuthology.orchestra.run.smithi104:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-16T17:22:02.940 DEBUG:teuthology.misc:Transferring archived files from smithi117:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/mchangir-2024-04-16_16:32:10-fs-wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-distro-default-smithi/7658439/remote/smithi117 2024-04-16T17:22:02.940 DEBUG:teuthology.orchestra.run.smithi117:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-16T17:22:02.984 INFO:teuthology.task.internal:Removing archive directory... 2024-04-16T17:22:02.984 DEBUG:teuthology.orchestra.run.smithi045:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-16T17:22:02.992 DEBUG:teuthology.orchestra.run.smithi104:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-16T17:22:02.994 DEBUG:teuthology.orchestra.run.smithi117:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-16T17:22:03.056 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-04-16T17:22:03.069 INFO:teuthology.task.internal:Not uploading archives. 2024-04-16T17:22:03.069 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-04-16T17:22:03.082 INFO:teuthology.task.internal:Tidying up after the test... 2024-04-16T17:22:03.082 DEBUG:teuthology.orchestra.run.smithi045:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-16T17:22:03.090 DEBUG:teuthology.orchestra.run.smithi104:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-16T17:22:03.092 DEBUG:teuthology.orchestra.run.smithi117:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-16T17:22:03.106 INFO:teuthology.orchestra.run.smithi104.stdout: 262162 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 16 17:22 /home/ubuntu/cephtest 2024-04-16T17:22:03.109 INFO:teuthology.orchestra.run.smithi117.stdout: 262162 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 16 17:22 /home/ubuntu/cephtest 2024-04-16T17:22:03.140 INFO:teuthology.orchestra.run.smithi045.stdout: 395174 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 16 17:22 /home/ubuntu/cephtest 2024-04-16T17:22:03.144 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-04-16T17:22:03.156 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-04-16T17:22:03.181 INFO:teuthology.nuke:Checking targets against current locks 2024-04-16T17:22:03.210 DEBUG:teuthology.nuke:shortname: smithi045 2024-04-16T17:22:03.211 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-16T17:22:03.234 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi045.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/mchangir-2024-04-16_16:32:10-fs-wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-distro-default-smithi/7658439', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-16 16:34:01.459561', 'locked_by': 'scheduled_mchangir@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNRd4Z5lIkG+aHVcNKB5bIR33xLSkjJ2+IrwRHGZWgiBdmi5xc7WlihH4ZBjgTq0kQvwjbwStiHVrSWiMstz70I='} 2024-04-16T17:22:03.239 DEBUG:teuthology.nuke:shortname: smithi104 2024-04-16T17:22:03.239 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-16T17:22:03.262 DEBUG:teuthology.nuke:shortname: smithi117 2024-04-16T17:22:03.262 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-16T17:22:03.268 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi104.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/mchangir-2024-04-16_16:32:10-fs-wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-distro-default-smithi/7658439', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-16 16:34:01.457884', 'locked_by': 'scheduled_mchangir@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBE2MaJ9xDICTcctgG59YkLRA+XHL7Tqw5uUYQV9k69LVhiELc9OQJywfIZiI9Tko916Qu0n1c8yH0ufeobgw8NQ='} 2024-04-16T17:22:03.281 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi117.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/mchangir-2024-04-16_16:32:10-fs-wip-mchangir-qa-debug-resource-temporarily-unavailable-issue-distro-default-smithi/7658439', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-16 16:34:01.455850', 'locked_by': 'scheduled_mchangir@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGRRWVkC0WMOkt1Y4/S+07VTH5BhW2jpEz0phigKIpIr5kqVDK2y1ROw0hAw3CSDTz1vShxtrByguWwQ+BTKSQ0='} 2024-04-16T17:22:03.298 INFO:teuthology.orchestra.console.smithi045:Power off 2024-04-16T17:22:03.299 DEBUG:teuthology.orchestra.console.smithi045:pexpect command: ipmitool -H smithi045.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-16T17:22:03.324 DEBUG:teuthology.orchestra.console.smithi045:power off output: Chassis Power Control: Down/Off 2024-04-16T17:22:03.325 DEBUG:teuthology.orchestra.console.smithi045:pexpect command: ipmitool -H smithi045.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-16T17:22:03.341 INFO:teuthology.orchestra.console.smithi117:Power off 2024-04-16T17:22:03.342 DEBUG:teuthology.orchestra.console.smithi117:pexpect command: ipmitool -H smithi117.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-16T17:22:03.357 INFO:teuthology.orchestra.console.smithi104:Power off 2024-04-16T17:22:03.357 DEBUG:teuthology.orchestra.console.smithi104:pexpect command: ipmitool -H smithi104.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-16T17:22:03.365 DEBUG:teuthology.orchestra.console.smithi045:check power output: Chassis Power is on 2024-04-16T17:22:03.366 DEBUG:teuthology.orchestra.console.smithi117:power off output: Chassis Power Control: Down/Off 2024-04-16T17:22:03.366 DEBUG:teuthology.orchestra.console.smithi117:pexpect command: ipmitool -H smithi117.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-16T17:22:03.378 DEBUG:teuthology.orchestra.console.smithi104:power off output: Chassis Power Control: Down/Off 2024-04-16T17:22:03.379 DEBUG:teuthology.orchestra.console.smithi104:pexpect command: ipmitool -H smithi104.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-16T17:22:03.388 DEBUG:teuthology.orchestra.console.smithi117:check power output: Chassis Power is on 2024-04-16T17:22:03.400 DEBUG:teuthology.orchestra.console.smithi104:check power output: Chassis Power is on 2024-04-16T17:22:07.367 DEBUG:teuthology.orchestra.console.smithi045:pexpect command: ipmitool -H smithi045.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-16T17:22:07.389 DEBUG:teuthology.orchestra.console.smithi117:pexpect command: ipmitool -H smithi117.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-16T17:22:07.401 DEBUG:teuthology.orchestra.console.smithi104:pexpect command: ipmitool -H smithi104.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-16T17:22:07.480 DEBUG:teuthology.orchestra.console.smithi045:check power output: Chassis Power is on 2024-04-16T17:22:07.501 DEBUG:teuthology.orchestra.console.smithi117:check power output: Chassis Power is on 2024-04-16T17:22:07.513 DEBUG:teuthology.orchestra.console.smithi104:check power output: Chassis Power is on 2024-04-16T17:22:11.485 DEBUG:teuthology.orchestra.console.smithi045:pexpect command: ipmitool -H smithi045.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-16T17:22:11.502 DEBUG:teuthology.orchestra.console.smithi117:pexpect command: ipmitool -H smithi117.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-16T17:22:11.515 DEBUG:teuthology.orchestra.console.smithi104:pexpect command: ipmitool -H smithi104.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-16T17:22:11.596 DEBUG:teuthology.orchestra.console.smithi045:check power output: Chassis Power is off 2024-04-16T17:22:11.612 DEBUG:teuthology.orchestra.console.smithi117:check power output: Chassis Power is off 2024-04-16T17:22:11.624 DEBUG:teuthology.orchestra.console.smithi104:check power output: Chassis Power is off 2024-04-16T17:22:11.697 INFO:teuthology.orchestra.console.smithi045:Power off completed 2024-04-16T17:22:11.712 INFO:teuthology.orchestra.console.smithi117:Power off completed 2024-04-16T17:22:11.725 INFO:teuthology.orchestra.console.smithi104:Power off completed 2024-04-16T17:22:11.831 INFO:teuthology.run:Summary data: client.0-kernel-sha1: 3f60ed14f7c110c7ce3c8d83dd2ed0d08e80d0c8 description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-workunit/suites/dbench}} duration: 2294.9068117141724 failure_reason: 'Command failed (workunit test suites/dbench.sh) on smithi045 with status 1: ''mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=b137e4a675398e295567f9652210865a4d573fac TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh''' flavor: default owner: scheduled_mchangir@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=6df16c87d16b4e09ad8cc292d074c972 status: fail success: false 2024-04-16T17:22:11.832 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-16T17:22:11.943 INFO:teuthology.run:FAIL